2024-09-21T07:21:06.486 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T07:21:06.508 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T07:21:06.610 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913602 branch: wip-pdonnell-testing-20240920.212106-debug description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/snapshots} email: pdonnell@redhat.com first_in_suite: false job_id: '7913602' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240920.212106-debug ceph: cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon warn on pool no app: false mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: client: pid file: '' global: ms die on skipped message: false mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - evicting unresponsive client - RECENT_CRASH sha1: 1843221895adccfb3b692060251dd6a0183bc410 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs check-counter: dry_run: true install: ceph: flavor: default sha1: 1843221895adccfb3b692060251dd6a0183bc410 kclient: syntax: v2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240920.212106-debug sha1: 1843221895adccfb3b692060251dd6a0183bc410 owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 7864 sha1: 1843221895adccfb3b692060251dd6a0183bc410 sleep_before_teardown: 0 subset: 67/512 suite: fs suite_branch: wip-pdonnell-testing-20240920.212106-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1843221895adccfb3b692060251dd6a0183bc410 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi039.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE/444ear+/QNL2dzlGKlS6MtvL7iVEZnYbJfsrfKdpvTUU0FPzjwxrgA6zk8lGwMcqoardajhSGzIQxKKKzZxE= smithi148.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGTY8Z8iIfpfOzg9VGQxWzpaUnnGwoUVPqRfeKLhaZfAuri1kEyYYnjeRZbta8xMl46Zk2Iwx2ZMrS7DfhUxV4s= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - exec: mon.a: - ceph config set mgr mgr/crash/warn_recent_interval 0 - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_snapshots teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-20_23:48:22 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-21T07:21:06.610 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa; will attempt to use it 2024-09-21T07:21:06.611 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks 2024-09-21T07:21:06.611 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T07:21:06.613 INFO:teuthology.task.internal:Checking packages... 2024-09-21T07:21:06.635 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1843221895adccfb3b692060251dd6a0183bc410' 2024-09-21T07:21:06.635 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T07:21:06.635 INFO:teuthology.packaging:ref: None 2024-09-21T07:21:06.635 INFO:teuthology.packaging:tag: None 2024-09-21T07:21:06.635 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240920.212106-debug 2024-09-21T07:21:06.635 INFO:teuthology.packaging:sha1: 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T07:21:06.635 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240920.212106-debug 2024-09-21T07:21:06.790 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-5120.g18432218 2024-09-21T07:21:06.791 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T07:21:06.805 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T07:21:06.806 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T07:21:06.843 INFO:teuthology.task.internal:Saving configuration 2024-09-21T07:21:06.860 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T07:21:06.868 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T07:21:06.893 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi039.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913602', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 07:15:12.525380', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE/444ear+/QNL2dzlGKlS6MtvL7iVEZnYbJfsrfKdpvTUU0FPzjwxrgA6zk8lGwMcqoardajhSGzIQxKKKzZxE='} 2024-09-21T07:21:06.917 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi148.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913602', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 07:15:12.528014', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGTY8Z8iIfpfOzg9VGQxWzpaUnnGwoUVPqRfeKLhaZfAuri1kEyYYnjeRZbta8xMl46Zk2Iwx2ZMrS7DfhUxV4s='} 2024-09-21T07:21:06.917 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T07:21:06.925 INFO:teuthology.task.internal:roles: ubuntu@smithi039.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-21T07:21:06.925 INFO:teuthology.task.internal:roles: ubuntu@smithi148.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-21T07:21:06.925 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T07:21:06.987 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f1eb44c0160>, signals=[15]) 2024-09-21T07:21:06.987 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T07:21:06.992 INFO:teuthology.task.internal:Opening connections... 2024-09-21T07:21:06.992 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T07:21:06.993 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T07:21:07.071 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi148.front.sepia.ceph.com 2024-09-21T07:21:07.072 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T07:21:07.143 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T07:21:07.150 DEBUG:teuthology.orchestra.run.smithi039:> uname -m 2024-09-21T07:21:07.167 INFO:teuthology.orchestra.run.smithi039.stdout:x86_64 2024-09-21T07:21:07.167 DEBUG:teuthology.orchestra.run.smithi039:> cat /etc/os-release 2024-09-21T07:21:07.224 INFO:teuthology.orchestra.run.smithi039.stdout:NAME="CentOS Stream" 2024-09-21T07:21:07.224 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION="9" 2024-09-21T07:21:07.224 INFO:teuthology.orchestra.run.smithi039.stdout:ID="centos" 2024-09-21T07:21:07.224 INFO:teuthology.orchestra.run.smithi039.stdout:ID_LIKE="rhel fedora" 2024-09-21T07:21:07.224 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION_ID="9" 2024-09-21T07:21:07.224 INFO:teuthology.orchestra.run.smithi039.stdout:PLATFORM_ID="platform:el9" 2024-09-21T07:21:07.225 INFO:teuthology.orchestra.run.smithi039.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T07:21:07.225 INFO:teuthology.orchestra.run.smithi039.stdout:ANSI_COLOR="0;31" 2024-09-21T07:21:07.225 INFO:teuthology.orchestra.run.smithi039.stdout:LOGO="fedora-logo-icon" 2024-09-21T07:21:07.225 INFO:teuthology.orchestra.run.smithi039.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T07:21:07.225 INFO:teuthology.orchestra.run.smithi039.stdout:HOME_URL="https://centos.org/" 2024-09-21T07:21:07.225 INFO:teuthology.orchestra.run.smithi039.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T07:21:07.225 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T07:21:07.225 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T07:21:07.226 INFO:teuthology.lock.ops:Updating smithi039.front.sepia.ceph.com on lock server 2024-09-21T07:21:07.251 DEBUG:teuthology.orchestra.run.smithi148:> uname -m 2024-09-21T07:21:07.270 INFO:teuthology.orchestra.run.smithi148.stdout:x86_64 2024-09-21T07:21:07.270 DEBUG:teuthology.orchestra.run.smithi148:> cat /etc/os-release 2024-09-21T07:21:07.328 INFO:teuthology.orchestra.run.smithi148.stdout:NAME="CentOS Stream" 2024-09-21T07:21:07.329 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION="9" 2024-09-21T07:21:07.329 INFO:teuthology.orchestra.run.smithi148.stdout:ID="centos" 2024-09-21T07:21:07.329 INFO:teuthology.orchestra.run.smithi148.stdout:ID_LIKE="rhel fedora" 2024-09-21T07:21:07.329 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION_ID="9" 2024-09-21T07:21:07.329 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM_ID="platform:el9" 2024-09-21T07:21:07.329 INFO:teuthology.orchestra.run.smithi148.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T07:21:07.329 INFO:teuthology.orchestra.run.smithi148.stdout:ANSI_COLOR="0;31" 2024-09-21T07:21:07.329 INFO:teuthology.orchestra.run.smithi148.stdout:LOGO="fedora-logo-icon" 2024-09-21T07:21:07.329 INFO:teuthology.orchestra.run.smithi148.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T07:21:07.329 INFO:teuthology.orchestra.run.smithi148.stdout:HOME_URL="https://centos.org/" 2024-09-21T07:21:07.329 INFO:teuthology.orchestra.run.smithi148.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T07:21:07.329 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T07:21:07.330 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T07:21:07.330 INFO:teuthology.lock.ops:Updating smithi148.front.sepia.ceph.com on lock server 2024-09-21T07:21:07.351 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T07:21:07.361 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T07:21:07.368 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T07:21:07.368 DEBUG:teuthology.orchestra.run.smithi039:> test '!' -e /home/ubuntu/cephtest 2024-09-21T07:21:07.371 DEBUG:teuthology.orchestra.run.smithi148:> test '!' -e /home/ubuntu/cephtest 2024-09-21T07:21:07.388 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T07:21:07.395 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T07:21:07.396 DEBUG:teuthology.orchestra.run.smithi039:> test -z $(ls -A /var/lib/ceph) 2024-09-21T07:21:07.427 DEBUG:teuthology.orchestra.run.smithi148:> test -z $(ls -A /var/lib/ceph) 2024-09-21T07:21:07.543 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T07:21:07.592 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T07:21:07.619 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-21T07:21:07.620 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-09-21T07:21:07.620 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-09-21T07:21:07.620 DEBUG:teuthology.orchestra.run.smithi039:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T07:21:07.620 DEBUG:teuthology.orchestra.run.smithi148:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T07:21:07.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:21:07.637 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-21T07:21:07.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:21:07.640 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-21T07:21:07.796 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-21T07:21:07.796 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-21T07:21:07.796 DEBUG:teuthology.orchestra.run.smithi039:> uname -r 2024-09-21T07:21:07.806 DEBUG:teuthology.task.kernel:sha1 for client.1 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-21T07:21:07.806 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-21T07:21:07.806 DEBUG:teuthology.orchestra.run.smithi148:> uname -r 2024-09-21T07:21:07.813 INFO:teuthology.orchestra.run.smithi039.stdout:5.14.0-503.el9.x86_64 2024-09-21T07:21:07.813 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-21T07:21:07.814 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-21T07:21:07.814 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum upgrade -y linux-firmware 2024-09-21T07:21:07.824 INFO:teuthology.orchestra.run.smithi148.stdout:5.14.0-503.el9.x86_64 2024-09-21T07:21:07.824 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-21T07:21:07.825 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-21T07:21:07.825 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum upgrade -y linux-firmware 2024-09-21T07:21:10.008 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 8.3 MB 00:00 2024-09-21T07:21:10.158 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.3 MB 00:00 2024-09-21T07:21:11.776 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - AppStream 32 MB/s | 20 MB 00:00 2024-09-21T07:21:11.817 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - AppStream 36 MB/s | 20 MB 00:00 2024-09-21T07:21:16.235 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-21T07:21:17.585 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 6.5 MB 00:01 2024-09-21T07:21:17.777 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 19 kB 00:00 2024-09-21T07:21:18.715 INFO:teuthology.orchestra.run.smithi148.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-21T07:21:19.927 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 19 kB 00:01 2024-09-21T07:21:20.831 INFO:teuthology.orchestra.run.smithi039.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-21T07:21:24.054 INFO:teuthology.orchestra.run.smithi148.stdout:lab-extras 46 kB/s | 1.7 kB 00:00 2024-09-21T07:21:26.030 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-21T07:21:26.039 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T07:21:26.039 INFO:teuthology.orchestra.run.smithi148.stdout: Package Architecture Version Repository Size 2024-09-21T07:21:26.039 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T07:21:26.040 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrading: 2024-09-21T07:21:26.040 INFO:teuthology.orchestra.run.smithi148.stdout: linux-firmware noarch 20240905-146.el9 baseos 388 M 2024-09-21T07:21:26.040 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:21:26.040 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-21T07:21:26.040 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-09-21T07:21:26.040 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrade 1 Package 2024-09-21T07:21:26.040 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:21:26.040 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 388 M 2024-09-21T07:21:26.040 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-21T07:21:26.503 INFO:teuthology.orchestra.run.smithi039.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2024-09-21T07:21:28.543 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-09-21T07:21:28.549 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-21T07:21:28.549 INFO:teuthology.orchestra.run.smithi039.stdout: Package Architecture Version Repository Size 2024-09-21T07:21:28.549 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-21T07:21:28.549 INFO:teuthology.orchestra.run.smithi039.stdout:Upgrading: 2024-09-21T07:21:28.549 INFO:teuthology.orchestra.run.smithi039.stdout: linux-firmware noarch 20240905-146.el9 baseos 388 M 2024-09-21T07:21:28.549 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:21:28.549 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-09-21T07:21:28.549 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-21T07:21:28.549 INFO:teuthology.orchestra.run.smithi039.stdout:Upgrade 1 Package 2024-09-21T07:21:28.549 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:21:28.550 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 388 M 2024-09-21T07:21:28.550 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-09-21T07:21:39.546 INFO:teuthology.orchestra.run.smithi148.stdout:linux-firmware-20240905-146.el9.noarch.rpm 29 MB/s | 388 MB 00:13 2024-09-21T07:21:39.546 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-21T07:21:39.546 INFO:teuthology.orchestra.run.smithi148.stdout:Total 29 MB/s | 388 MB 00:13 2024-09-21T07:21:42.653 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-21T07:21:42.755 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-21T07:21:42.755 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-21T07:21:44.504 INFO:teuthology.orchestra.run.smithi039.stdout:linux-firmware-20240905-146.el9.noarch.rpm 24 MB/s | 388 MB 00:15 2024-09-21T07:21:44.504 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-09-21T07:21:44.504 INFO:teuthology.orchestra.run.smithi039.stdout:Total 24 MB/s | 388 MB 00:15 2024-09-21T07:21:46.049 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-21T07:21:46.050 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-21T07:21:47.773 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-09-21T07:21:47.873 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-09-21T07:21:47.873 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-09-21T07:21:51.312 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-09-21T07:21:51.313 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-09-21T07:21:55.396 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-21T07:21:56.096 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : linux-firmware-20240905-146.el9.noarch 1/2 2024-09-21T07:21:56.185 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : linux-firmware-20240716-145.el9.noarch 2/2 2024-09-21T07:21:56.654 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: linux-firmware-20240716-145.el9.noarch 2/2 2024-09-21T07:21:56.655 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : linux-firmware-20240905-146.el9.noarch 1/2 2024-09-21T07:21:57.394 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : linux-firmware-20240716-145.el9.noarch 2/2 2024-09-21T07:21:57.395 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:21:57.395 INFO:teuthology.orchestra.run.smithi148.stdout:Upgraded: 2024-09-21T07:21:57.395 INFO:teuthology.orchestra.run.smithi148.stdout: linux-firmware-20240905-146.el9.noarch 2024-09-21T07:21:57.395 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:21:57.395 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-21T07:21:57.741 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-21T07:21:57.741 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-21T07:21:57.892 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-21T07:21:57.892 DEBUG:teuthology.orchestra.run.smithi148:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-21T07:21:57.895 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi148.front.sepia.ceph.com to complete... 2024-09-21T07:22:00.870 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-09-21T07:22:01.630 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : linux-firmware-20240905-146.el9.noarch 1/2 2024-09-21T07:22:01.714 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : linux-firmware-20240716-145.el9.noarch 2/2 2024-09-21T07:22:02.181 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: linux-firmware-20240716-145.el9.noarch 2/2 2024-09-21T07:22:02.181 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : linux-firmware-20240905-146.el9.noarch 1/2 2024-09-21T07:22:02.934 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : linux-firmware-20240716-145.el9.noarch 2/2 2024-09-21T07:22:02.934 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:22:02.934 INFO:teuthology.orchestra.run.smithi039.stdout:Upgraded: 2024-09-21T07:22:02.934 INFO:teuthology.orchestra.run.smithi039.stdout: linux-firmware-20240905-146.el9.noarch 2024-09-21T07:22:02.934 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:22:02.934 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-09-21T07:22:03.274 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-21T07:22:03.274 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-21T07:22:03.429 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-21T07:22:03.429 DEBUG:teuthology.orchestra.run.smithi039:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-21T07:22:03.432 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi039.front.sepia.ceph.com to complete... 2024-09-21T07:22:04.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21 07:22:04 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-21T07:22:04.218 INFO:teuthology.orchestra.run.smithi148.stderr:FINISHED --2024-09-21 07:22:04-- 2024-09-21T07:22:04.218 INFO:teuthology.orchestra.run.smithi148.stderr:Total wall clock time: 6.3s 2024-09-21T07:22:04.218 INFO:teuthology.orchestra.run.smithi148.stderr:Downloaded: 1 files, 343M in 6.0s (56.9 MB/s) 2024-09-21T07:22:04.220 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-21T07:22:04.220 DEBUG:teuthology.orchestra.run.smithi148:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-21T07:22:06.287 INFO:teuthology.orchestra.run.smithi148.stdout:Verifying... ######################################## 2024-09-21T07:22:06.523 INFO:teuthology.orchestra.run.smithi148.stdout:Preparing... ######################################## 2024-09-21T07:22:06.582 INFO:teuthology.orchestra.run.smithi148.stdout:Updating / installing... 2024-09-21T07:22:10.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21 07:22:10 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-21T07:22:10.357 INFO:teuthology.orchestra.run.smithi039.stderr:FINISHED --2024-09-21 07:22:10-- 2024-09-21T07:22:10.357 INFO:teuthology.orchestra.run.smithi039.stderr:Total wall clock time: 6.9s 2024-09-21T07:22:10.357 INFO:teuthology.orchestra.run.smithi039.stderr:Downloaded: 1 files, 343M in 6.5s (52.9 MB/s) 2024-09-21T07:22:10.364 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-21T07:22:10.364 DEBUG:teuthology.orchestra.run.smithi039:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-21T07:22:12.525 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying... ######################################## 2024-09-21T07:22:12.586 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing... ######################################## 2024-09-21T07:22:12.650 INFO:teuthology.orchestra.run.smithi039.stdout:Updating / installing... 2024-09-21T07:22:14.602 INFO:teuthology.orchestra.run.smithi148.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-21T07:22:20.992 INFO:teuthology.orchestra.run.smithi039.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-21T07:22:54.728 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi148.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-21T07:22:54.728 DEBUG:teuthology.orchestra.run.smithi148:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-21T07:22:54.805 INFO:teuthology.orchestra.run.smithi148.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:22:54.805 INFO:teuthology.orchestra.run.smithi148.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:22:54.805 INFO:teuthology.orchestra.run.smithi148.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:22:54.805 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:22:54.805 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-21T07:22:54.805 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-21T07:22:54.805 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-21T07:22:54.806 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-21T07:22:54.806 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-21T07:22:54.806 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-21T07:22:54.806 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-21T07:22:54.806 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-21T07:22:54.806 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-21T07:22:54.806 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-21T07:22:54.806 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-21T07:22:54.806 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-21T07:22:54.806 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-21T07:22:54.806 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-21T07:22:54.806 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-21T07:22:54.806 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-21T07:22:54.807 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-21T07:22:54.807 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-21T07:22:54.807 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-21T07:22:54.807 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-21T07:22:54.807 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-21T07:22:54.807 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-21T07:22:54.807 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-21T07:22:54.807 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-21T07:22:54.807 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-21T07:22:54.807 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-21T07:22:54.807 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-21T07:22:54.807 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-21T07:22:54.807 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-21T07:22:54.807 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-21T07:22:54.808 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-21T07:22:54.808 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-21T07:22:54.808 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-21T07:22:54.808 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-21T07:22:54.808 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-21T07:22:54.808 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-21T07:22:54.808 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-21T07:22:54.808 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-21T07:22:54.808 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-21T07:22:54.808 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-21T07:22:54.808 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-21T07:22:54.808 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-21T07:22:54.808 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-21T07:22:54.808 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-21T07:22:54.809 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-21T07:22:54.809 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-21T07:22:54.809 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-21T07:22:54.809 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-21T07:22:54.809 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-21T07:22:54.809 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-21T07:22:54.809 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-21T07:22:54.809 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-21T07:22:54.809 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-21T07:22:54.809 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-21T07:22:54.809 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-21T07:22:54.809 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-21T07:22:54.809 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-21T07:22:54.809 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-21T07:22:54.810 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-21T07:22:54.810 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-21T07:22:54.810 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-21T07:22:54.810 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-21T07:22:54.810 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-21T07:22:54.810 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-21T07:22:54.810 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-21T07:22:54.810 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-21T07:22:54.810 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-21T07:22:54.810 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-21T07:22:54.810 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-21T07:22:54.810 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-21T07:22:54.810 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-21T07:22:54.810 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-21T07:22:54.811 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-21T07:22:54.811 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-21T07:22:54.811 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-21T07:22:54.811 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-21T07:22:54.811 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-21T07:22:54.811 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-21T07:22:54.811 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-21T07:22:54.811 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-21T07:22:54.811 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-21T07:22:54.811 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-21T07:22:54.811 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-21T07:22:54.811 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-21T07:22:54.811 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-21T07:22:54.811 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-21T07:22:54.812 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-21T07:22:54.812 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-21T07:22:54.812 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-21T07:22:54.812 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-21T07:22:54.812 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-21T07:22:54.812 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-21T07:22:54.812 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-21T07:22:54.812 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-21T07:22:54.812 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-21T07:22:54.812 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-21T07:22:54.812 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-21T07:22:54.812 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-21T07:22:54.812 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-21T07:22:54.812 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-21T07:22:54.812 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-21T07:22:54.813 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-21T07:22:54.813 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-21T07:22:54.813 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-21T07:22:54.813 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-21T07:22:54.813 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-21T07:22:54.813 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-21T07:22:54.813 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-21T07:22:54.813 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-21T07:22:54.813 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-21T07:22:54.813 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-21T07:22:54.813 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-21T07:22:54.813 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-21T07:22:54.813 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-21T07:22:54.813 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-21T07:22:54.813 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-21T07:22:54.814 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-21T07:22:54.814 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-21T07:22:54.814 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-21T07:22:54.814 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-21T07:22:54.814 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-21T07:22:54.814 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-21T07:22:54.814 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-21T07:22:54.814 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-21T07:22:54.814 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-21T07:22:54.814 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-21T07:22:54.814 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-21T07:22:54.814 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-21T07:22:54.814 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-21T07:22:54.815 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-21T07:22:54.815 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-21T07:22:54.815 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-21T07:22:54.815 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-21T07:22:54.815 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-21T07:22:54.815 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-21T07:22:54.815 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-21T07:22:54.815 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-21T07:22:54.815 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-21T07:22:54.815 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-21T07:22:54.815 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-21T07:22:54.815 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-21T07:22:54.815 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-21T07:22:54.815 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-21T07:22:54.816 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-21T07:22:54.816 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-21T07:22:54.816 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-21T07:22:54.817 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-21T07:22:54.817 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-21T07:22:54.818 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-21T07:22:54.818 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-21T07:22:54.818 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-21T07:22:54.818 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-21T07:22:54.818 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-21T07:22:54.818 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-21T07:22:54.818 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-21T07:22:54.818 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-21T07:22:54.818 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-21T07:22:54.818 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-21T07:22:54.818 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-21T07:22:54.818 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-21T07:22:54.818 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-21T07:22:54.819 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-21T07:22:54.819 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-21T07:22:54.819 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-21T07:22:54.819 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-21T07:22:54.819 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-21T07:22:54.819 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-21T07:22:54.819 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-21T07:22:54.819 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-21T07:22:54.819 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-21T07:22:54.819 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-21T07:22:54.819 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-21T07:22:54.819 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-21T07:22:54.819 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-21T07:22:54.820 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-21T07:22:54.820 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-21T07:22:54.820 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-21T07:22:54.820 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-21T07:22:54.820 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-21T07:22:54.820 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-21T07:22:54.820 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-21T07:22:54.820 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-21T07:22:54.820 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-21T07:22:54.820 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-21T07:22:54.820 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-21T07:22:54.820 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-21T07:22:54.820 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-21T07:22:54.820 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-21T07:22:54.821 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-21T07:22:54.821 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-21T07:22:54.821 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-21T07:22:54.821 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-21T07:22:54.821 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-21T07:22:54.821 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-21T07:22:54.821 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-21T07:22:54.821 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-21T07:22:54.821 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-21T07:22:54.821 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-21T07:22:54.821 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-21T07:22:54.821 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-21T07:22:54.821 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-21T07:22:54.821 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-21T07:22:54.822 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-21T07:22:54.822 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-21T07:22:54.822 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-21T07:22:54.822 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-21T07:22:54.822 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-21T07:22:54.822 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-21T07:22:54.822 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-21T07:22:54.822 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-21T07:22:54.822 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-21T07:22:54.822 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-21T07:22:54.822 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-21T07:22:54.822 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-21T07:22:54.822 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-21T07:22:54.823 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-21T07:22:54.823 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-21T07:22:54.823 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-21T07:22:54.823 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-21T07:22:54.823 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-21T07:22:54.823 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-21T07:22:54.823 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-21T07:22:54.823 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-21T07:22:54.823 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-21T07:22:54.823 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-21T07:22:54.823 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-21T07:22:54.823 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-21T07:22:54.823 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-21T07:22:54.823 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-21T07:22:54.824 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-21T07:22:54.824 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-21T07:22:54.824 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-21T07:22:54.824 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-21T07:22:54.824 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-21T07:22:54.824 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-21T07:22:54.824 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-21T07:22:54.824 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-21T07:22:54.824 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-21T07:22:54.824 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-21T07:22:54.824 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-21T07:22:54.824 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-21T07:22:54.824 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-21T07:22:54.825 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-21T07:22:54.825 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-21T07:22:54.825 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-21T07:22:54.825 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-21T07:22:54.825 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-21T07:22:54.825 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-21T07:22:54.825 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-21T07:22:54.825 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-21T07:22:54.825 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-21T07:22:54.825 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-21T07:22:54.825 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-21T07:22:54.825 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-21T07:22:54.825 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-21T07:22:54.825 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-21T07:22:54.826 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-21T07:22:54.826 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-21T07:22:54.826 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-21T07:22:54.826 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-21T07:22:54.826 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-21T07:22:54.826 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-21T07:22:54.826 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-21T07:22:54.826 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-21T07:22:54.826 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-21T07:22:54.826 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-21T07:22:54.826 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-21T07:22:54.826 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-21T07:22:54.826 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-21T07:22:54.827 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-21T07:22:54.827 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-21T07:22:54.827 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-21T07:22:54.827 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-21T07:22:54.827 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-21T07:22:54.827 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-21T07:22:54.827 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-21T07:22:54.827 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-21T07:22:54.827 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-21T07:22:54.827 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-21T07:22:54.827 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-21T07:22:54.827 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-21T07:22:54.827 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-21T07:22:54.827 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-21T07:22:54.828 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-21T07:22:54.828 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-21T07:22:54.828 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-21T07:22:54.828 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-21T07:22:54.828 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-21T07:22:54.828 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-21T07:22:54.828 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-21T07:22:54.828 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-21T07:22:54.828 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-21T07:22:54.828 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-21T07:22:54.828 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-21T07:22:54.828 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-21T07:22:54.828 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-21T07:22:54.828 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-21T07:22:54.829 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-21T07:22:54.829 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-21T07:22:54.829 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-21T07:22:54.829 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-21T07:22:54.829 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-21T07:22:54.829 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-21T07:22:54.829 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-21T07:22:54.829 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-21T07:22:54.829 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-21T07:22:54.829 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-21T07:22:54.829 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-21T07:22:54.829 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-21T07:22:54.829 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-21T07:22:54.829 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-21T07:22:54.829 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-21T07:22:54.830 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-21T07:22:54.830 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-21T07:22:54.830 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-21T07:22:54.830 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-21T07:22:54.830 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-21T07:22:54.830 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-21T07:22:54.830 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-21T07:22:54.830 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-21T07:22:54.830 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-21T07:22:54.830 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-21T07:22:54.830 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-21T07:22:54.830 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-21T07:22:54.830 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-21T07:22:54.831 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-21T07:22:54.831 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-21T07:22:54.831 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-21T07:22:54.831 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-21T07:22:54.831 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-21T07:22:54.831 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-21T07:22:54.831 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-21T07:22:54.831 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-21T07:22:54.831 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-21T07:22:54.831 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-21T07:22:54.831 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-21T07:22:54.831 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-21T07:22:54.831 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-21T07:22:54.831 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-21T07:22:54.832 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-21T07:22:54.832 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-21T07:22:54.832 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-21T07:22:54.832 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-21T07:22:54.832 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-21T07:22:54.832 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-21T07:22:54.832 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-21T07:22:54.832 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-21T07:22:54.832 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-21T07:22:54.832 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-21T07:22:54.832 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-21T07:22:54.832 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-21T07:22:54.832 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-21T07:22:54.832 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-21T07:22:54.833 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-21T07:22:54.833 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-21T07:22:54.833 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-21T07:22:54.833 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-21T07:22:54.833 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-21T07:22:54.833 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-21T07:22:54.833 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-21T07:22:54.833 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-21T07:22:54.833 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-21T07:22:54.833 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-21T07:22:54.833 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-21T07:22:54.833 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-21T07:22:54.833 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-21T07:22:54.834 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-21T07:22:54.834 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-21T07:22:54.834 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-21T07:22:54.834 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-21T07:22:54.834 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-21T07:22:54.834 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-21T07:22:54.834 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-21T07:22:54.834 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-21T07:22:54.834 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-21T07:22:54.834 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-21T07:22:54.834 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-21T07:22:54.834 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-21T07:22:54.834 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-21T07:22:54.834 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-21T07:22:54.835 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-21T07:22:54.835 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-21T07:22:54.835 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-21T07:22:54.835 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-21T07:22:54.835 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-21T07:22:54.835 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-21T07:22:54.835 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-21T07:22:54.835 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-21T07:22:54.835 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-21T07:22:54.835 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-21T07:22:54.835 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-21T07:22:54.835 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-21T07:22:54.835 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-21T07:22:54.835 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-21T07:22:54.836 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-21T07:22:54.836 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-21T07:22:54.836 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-21T07:22:54.836 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-21T07:22:54.836 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-21T07:22:54.836 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-21T07:22:54.836 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-21T07:22:54.836 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-21T07:22:54.836 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-21T07:22:54.836 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-21T07:22:54.836 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-21T07:22:54.836 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-21T07:22:54.836 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-21T07:22:54.836 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-21T07:22:54.836 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-21T07:22:54.837 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-21T07:22:54.837 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-21T07:22:54.837 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-21T07:22:54.837 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-21T07:22:54.837 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-21T07:22:54.837 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-21T07:22:54.837 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-21T07:22:54.837 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-21T07:22:54.837 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-21T07:22:54.837 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-21T07:22:54.837 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-21T07:22:54.837 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-21T07:22:54.837 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-21T07:22:54.838 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-21T07:22:54.838 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-21T07:22:54.838 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-21T07:22:54.838 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-21T07:22:54.838 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-21T07:22:54.838 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-21T07:22:54.838 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-21T07:22:54.838 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-21T07:22:54.838 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-21T07:22:54.838 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-21T07:22:54.838 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-21T07:22:54.838 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-21T07:22:54.838 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-21T07:22:54.838 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-21T07:22:54.839 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-21T07:22:54.839 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-21T07:22:54.839 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-21T07:22:54.839 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-21T07:22:54.839 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-21T07:22:54.839 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-21T07:22:54.839 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-21T07:22:54.839 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-21T07:22:54.839 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-21T07:22:54.839 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-21T07:22:54.839 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-21T07:22:54.839 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-21T07:22:54.839 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-21T07:22:54.839 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-21T07:22:54.839 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-21T07:22:54.840 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-21T07:22:54.840 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-21T07:22:54.840 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-21T07:22:54.840 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-21T07:22:54.840 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-21T07:22:54.840 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-21T07:22:54.840 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-21T07:22:54.840 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-21T07:22:54.840 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-21T07:22:54.840 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-21T07:22:54.840 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-21T07:22:54.840 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-21T07:22:54.840 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-21T07:22:54.841 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-21T07:22:54.841 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-21T07:22:54.841 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-21T07:22:54.841 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-21T07:22:54.841 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-21T07:22:54.841 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-21T07:22:54.841 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-21T07:22:54.841 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-21T07:22:54.841 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-21T07:22:54.841 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-21T07:22:54.841 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-21T07:22:54.841 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-21T07:22:54.841 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-21T07:22:54.842 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-21T07:22:54.842 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-21T07:22:54.842 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-21T07:22:54.842 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-21T07:22:54.842 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-21T07:22:54.842 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-21T07:22:54.842 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-21T07:22:54.842 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-21T07:22:54.842 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-21T07:22:54.842 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-21T07:22:54.842 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-21T07:22:54.842 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-21T07:22:54.842 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-21T07:22:54.842 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-21T07:22:54.843 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-21T07:22:54.843 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-21T07:22:54.843 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-21T07:22:54.843 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-21T07:22:54.843 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-21T07:22:54.843 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-21T07:22:54.843 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-21T07:22:54.843 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-21T07:22:54.843 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-21T07:22:54.843 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-21T07:22:54.843 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-21T07:22:54.843 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-21T07:22:54.843 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-21T07:22:54.843 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-21T07:22:54.844 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-21T07:22:54.844 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-21T07:22:54.844 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-21T07:22:54.844 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-21T07:22:54.844 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-21T07:22:54.844 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-21T07:22:54.844 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-21T07:22:54.844 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-21T07:22:54.844 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-21T07:22:54.844 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-21T07:22:54.844 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-21T07:22:54.844 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-21T07:22:54.844 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-21T07:22:54.844 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-21T07:22:54.845 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-21T07:22:54.845 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-21T07:22:54.845 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-21T07:22:54.845 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-21T07:22:54.845 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-21T07:22:54.845 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-21T07:22:54.845 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-21T07:22:54.845 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-21T07:22:54.845 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-21T07:22:54.845 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-21T07:22:54.845 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-21T07:22:54.845 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-21T07:22:54.845 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-21T07:22:54.845 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-21T07:22:54.846 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-21T07:22:54.846 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-21T07:22:54.846 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-21T07:22:54.846 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-21T07:22:54.846 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-21T07:22:54.846 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-21T07:22:54.846 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-21T07:22:54.846 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-21T07:22:54.846 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-21T07:22:54.846 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-21T07:22:54.846 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-21T07:22:54.846 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-21T07:22:54.846 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-21T07:22:54.846 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-21T07:22:54.847 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-21T07:22:54.847 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-21T07:22:54.847 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-21T07:22:54.847 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-21T07:22:54.847 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-21T07:22:54.847 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-21T07:22:54.847 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-21T07:22:54.847 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-21T07:22:54.847 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-21T07:22:54.847 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-21T07:22:54.847 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-21T07:22:54.847 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-21T07:22:54.847 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-21T07:22:54.848 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-21T07:22:54.848 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-21T07:22:54.848 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-21T07:22:54.848 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-21T07:22:54.848 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-21T07:22:54.848 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-21T07:22:54.848 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-21T07:22:54.848 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-21T07:22:54.848 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-21T07:22:54.848 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-21T07:22:54.848 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-21T07:22:54.848 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-21T07:22:54.848 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-21T07:22:54.848 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-21T07:22:54.849 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-21T07:22:54.849 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-21T07:22:54.849 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-21T07:22:54.849 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-21T07:22:54.849 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-21T07:22:54.849 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-21T07:22:54.849 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-21T07:22:54.849 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-21T07:22:54.849 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-21T07:22:54.849 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-21T07:22:54.849 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-21T07:22:54.849 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-21T07:22:54.849 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-21T07:22:54.849 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-21T07:22:54.850 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-21T07:22:54.850 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-21T07:22:54.850 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-21T07:22:54.850 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-21T07:22:54.850 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-21T07:22:54.850 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-21T07:22:54.850 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-21T07:22:54.850 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-21T07:22:54.850 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-21T07:22:54.850 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-21T07:22:54.850 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-21T07:22:54.850 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-21T07:22:54.850 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-21T07:22:54.850 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-21T07:22:54.851 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-21T07:22:54.851 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-21T07:22:54.851 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-21T07:22:54.851 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-21T07:22:54.851 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-21T07:22:54.851 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-21T07:22:54.851 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-21T07:22:54.851 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-21T07:22:54.851 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-21T07:22:54.851 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-21T07:22:54.851 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-21T07:22:54.851 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-21T07:22:54.851 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-21T07:22:54.852 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-21T07:22:54.852 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-21T07:22:54.852 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-21T07:22:54.852 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-21T07:22:54.852 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-21T07:22:54.852 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-21T07:22:54.852 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-21T07:22:54.852 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-21T07:22:54.852 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-21T07:22:54.852 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-21T07:22:54.852 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-21T07:22:54.852 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-21T07:22:54.852 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-21T07:22:54.852 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-21T07:22:54.852 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-21T07:22:54.853 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-21T07:22:54.853 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-21T07:22:54.853 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-21T07:22:54.853 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-21T07:22:54.853 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-21T07:22:54.853 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-21T07:22:54.853 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-21T07:22:54.853 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-21T07:22:54.853 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-21T07:22:54.853 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-21T07:22:54.853 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-21T07:22:54.853 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-21T07:22:54.853 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-21T07:22:54.854 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-21T07:22:54.854 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-21T07:22:54.854 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-21T07:22:54.854 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-21T07:22:54.854 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-21T07:22:54.854 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-21T07:22:54.854 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-21T07:22:54.854 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-21T07:22:54.854 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-21T07:22:54.854 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-21T07:22:54.854 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-21T07:22:54.854 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-21T07:22:54.855 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-21T07:22:54.855 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-21T07:22:54.855 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-21T07:22:54.855 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-21T07:22:54.855 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-21T07:22:54.855 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-21T07:22:54.855 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-21T07:22:54.855 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-21T07:22:54.855 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-21T07:22:54.855 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-21T07:22:54.855 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-21T07:22:54.855 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-21T07:22:54.855 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-21T07:22:54.855 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-21T07:22:54.856 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-21T07:22:54.856 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-21T07:22:54.856 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-21T07:22:54.856 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-21T07:22:54.856 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-21T07:22:54.856 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-21T07:22:54.856 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-21T07:22:54.856 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-21T07:22:54.856 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-21T07:22:54.856 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-21T07:22:54.856 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-21T07:22:54.856 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-21T07:22:54.856 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-21T07:22:54.857 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-21T07:22:54.857 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-21T07:22:54.857 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-21T07:22:54.857 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-21T07:22:54.857 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-21T07:22:54.857 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-21T07:22:54.857 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-21T07:22:54.857 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-21T07:22:54.857 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-21T07:22:54.857 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-21T07:22:54.857 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-21T07:22:54.857 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-21T07:22:54.857 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-21T07:22:54.858 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-21T07:22:54.858 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-21T07:22:54.858 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-21T07:22:54.858 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-21T07:22:54.858 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-21T07:22:54.858 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-21T07:22:54.858 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-21T07:22:54.858 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-21T07:22:54.858 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-21T07:22:54.858 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-21T07:22:54.858 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-21T07:22:54.858 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-21T07:22:54.858 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-21T07:22:54.859 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-21T07:22:54.859 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-21T07:22:54.859 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-21T07:22:54.859 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-21T07:22:54.859 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-21T07:22:54.859 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-21T07:22:54.859 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-21T07:22:54.859 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-21T07:22:54.859 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-21T07:22:54.859 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-21T07:22:54.859 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-21T07:22:54.859 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-21T07:22:54.859 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-21T07:22:54.859 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-21T07:22:54.860 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-21T07:22:54.860 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-21T07:22:54.860 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-21T07:22:54.860 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-21T07:22:54.860 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-21T07:22:54.860 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-21T07:22:54.860 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-21T07:22:54.860 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-21T07:22:54.860 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-21T07:22:54.860 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-21T07:22:54.860 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-21T07:22:54.860 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-21T07:22:54.860 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-21T07:22:54.861 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-21T07:22:54.861 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-21T07:22:54.861 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-21T07:22:54.861 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-21T07:22:54.861 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-21T07:22:54.861 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-21T07:22:54.861 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-21T07:22:54.861 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-21T07:22:54.861 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-21T07:22:54.861 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-21T07:22:54.861 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-21T07:22:54.861 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-21T07:22:54.861 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-21T07:22:54.861 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-21T07:22:54.862 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-21T07:22:54.862 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-21T07:22:54.862 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-21T07:22:54.862 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-21T07:22:54.862 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-21T07:22:54.862 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-21T07:22:54.862 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-21T07:22:54.862 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-21T07:22:54.862 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-21T07:22:54.862 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-21T07:22:54.862 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-21T07:22:54.862 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-21T07:22:54.862 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-21T07:22:54.863 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-21T07:22:54.863 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-21T07:22:54.863 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-21T07:22:54.863 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-21T07:22:54.863 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-21T07:22:54.863 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-21T07:22:54.863 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-21T07:22:54.863 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-21T07:22:54.863 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-21T07:22:54.863 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-21T07:22:54.863 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-21T07:22:54.863 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-21T07:22:54.863 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-21T07:22:54.863 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-21T07:22:54.864 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-21T07:22:54.864 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-21T07:22:54.864 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-21T07:22:54.864 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-21T07:22:54.864 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-21T07:22:54.864 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-21T07:22:54.864 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-21T07:22:54.864 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-21T07:22:54.864 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-21T07:22:54.864 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-21T07:22:54.864 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-21T07:22:54.864 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-21T07:22:54.864 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-21T07:22:54.865 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-21T07:22:54.865 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-21T07:22:54.865 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-21T07:22:54.865 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-21T07:22:54.865 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-21T07:22:54.865 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-21T07:22:54.865 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-21T07:22:54.865 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-21T07:22:54.865 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-21T07:22:54.865 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-21T07:22:54.865 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-21T07:22:54.865 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-21T07:22:54.865 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-21T07:22:54.865 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-21T07:22:54.866 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-21T07:22:54.866 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-21T07:22:54.866 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-21T07:22:54.866 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-21T07:22:54.866 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-21T07:22:54.866 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-21T07:22:54.866 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-21T07:22:54.866 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-21T07:22:54.866 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-21T07:22:54.866 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-21T07:22:54.866 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-21T07:22:54.866 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-21T07:22:54.867 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-21T07:22:54.867 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-21T07:22:54.867 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-21T07:22:54.867 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-21T07:22:54.867 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-21T07:22:54.867 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-21T07:22:54.867 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-21T07:22:54.867 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-21T07:22:54.867 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-21T07:22:54.867 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-21T07:22:54.867 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-21T07:22:54.867 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-21T07:22:54.867 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-21T07:22:54.867 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-21T07:22:54.867 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-21T07:22:54.868 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-21T07:22:54.868 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-21T07:22:54.868 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-21T07:22:54.868 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-21T07:22:54.868 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-21T07:22:54.868 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-21T07:22:54.868 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-21T07:22:54.868 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-21T07:22:54.868 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-21T07:22:54.868 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-21T07:22:54.868 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-21T07:22:54.868 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-21T07:22:54.868 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-21T07:22:54.868 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-21T07:22:54.869 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-21T07:22:54.869 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-21T07:22:54.869 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-21T07:22:54.869 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-21T07:22:54.869 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-21T07:22:54.869 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-21T07:22:54.869 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-21T07:22:54.869 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-21T07:22:54.869 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-21T07:22:54.869 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-21T07:22:54.869 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-21T07:22:54.869 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-21T07:22:54.869 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-21T07:22:54.869 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-21T07:22:54.870 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-21T07:22:54.870 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-21T07:22:54.870 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-21T07:22:54.870 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-21T07:22:54.870 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-21T07:22:54.870 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-21T07:22:54.870 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-21T07:22:54.870 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-21T07:22:54.870 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-21T07:22:54.870 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-21T07:22:54.870 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-21T07:22:54.870 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-21T07:22:54.870 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-21T07:22:54.870 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-21T07:22:54.871 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-21T07:22:54.871 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-21T07:22:54.871 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-21T07:22:54.871 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-21T07:22:54.871 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-21T07:22:54.871 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-21T07:22:54.871 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-21T07:22:54.871 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-21T07:22:54.871 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-21T07:22:54.871 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-21T07:22:54.871 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-21T07:22:54.871 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-21T07:22:54.871 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-21T07:22:54.871 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-21T07:22:54.872 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-21T07:22:54.872 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-21T07:22:54.872 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-21T07:22:54.872 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-21T07:22:54.872 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-21T07:22:54.872 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-21T07:22:54.872 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-21T07:22:54.872 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-21T07:22:54.872 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-21T07:22:54.872 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-21T07:22:54.872 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-21T07:22:54.872 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-21T07:22:54.872 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-21T07:22:54.872 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-21T07:22:54.872 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-21T07:22:54.873 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-21T07:22:54.873 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-21T07:22:54.873 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-21T07:22:54.873 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-21T07:22:54.873 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-21T07:22:54.873 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-21T07:22:54.873 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-21T07:22:54.873 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-21T07:22:54.873 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-21T07:22:54.873 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-21T07:22:54.873 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-21T07:22:54.873 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-21T07:22:54.873 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-21T07:22:54.873 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-21T07:22:54.874 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-21T07:22:54.874 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-21T07:22:54.874 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-21T07:22:54.874 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-21T07:22:54.874 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-21T07:22:54.874 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-21T07:22:54.874 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-21T07:22:54.874 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-21T07:22:54.874 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-21T07:22:54.874 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-21T07:22:54.874 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-21T07:22:54.874 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-21T07:22:54.874 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-21T07:22:54.874 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-21T07:22:54.875 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-21T07:22:54.875 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-21T07:22:54.875 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-21T07:22:54.875 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-21T07:22:54.875 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-21T07:22:54.875 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-21T07:22:54.875 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-21T07:22:54.875 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-21T07:22:54.875 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-21T07:22:54.875 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-21T07:22:54.875 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-21T07:22:54.875 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-21T07:22:54.875 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-21T07:22:54.875 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-21T07:22:54.876 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-21T07:22:54.876 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-21T07:22:54.876 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-21T07:22:54.876 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-21T07:22:54.876 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-21T07:22:54.876 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-21T07:22:54.876 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-21T07:22:54.876 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-21T07:22:54.876 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-21T07:22:54.876 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-21T07:22:54.876 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-21T07:22:54.876 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-21T07:22:54.876 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-21T07:22:54.876 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-21T07:22:54.876 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-21T07:22:54.877 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-21T07:22:54.877 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-21T07:22:54.877 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-21T07:22:54.877 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-21T07:22:54.877 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-21T07:22:54.877 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-21T07:22:54.877 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-21T07:22:54.877 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-21T07:22:54.877 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-21T07:22:54.877 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-21T07:22:54.877 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-21T07:22:54.877 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-21T07:22:54.877 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-21T07:22:54.877 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-21T07:22:54.878 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-21T07:22:54.878 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-21T07:22:54.878 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-21T07:22:54.878 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-21T07:22:54.878 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-21T07:22:54.878 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-21T07:22:54.878 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-21T07:22:54.878 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-21T07:22:54.878 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-21T07:22:54.878 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-21T07:22:54.878 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-21T07:22:54.878 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-21T07:22:54.878 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-21T07:22:54.878 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-21T07:22:54.879 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-21T07:22:54.879 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-21T07:22:54.879 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-21T07:22:54.879 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-21T07:22:54.879 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-21T07:22:54.879 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-21T07:22:54.879 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-21T07:22:54.879 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-21T07:22:54.879 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-21T07:22:54.879 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-21T07:22:54.879 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-21T07:22:54.879 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-21T07:22:54.879 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-21T07:22:54.879 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-21T07:22:54.880 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-21T07:22:54.880 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-21T07:22:54.880 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-21T07:22:54.880 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-21T07:22:54.880 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-21T07:22:54.880 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-21T07:22:54.880 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-21T07:22:54.880 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-21T07:22:54.880 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-21T07:22:54.880 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-21T07:22:54.880 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-21T07:22:54.880 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-21T07:22:54.880 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-21T07:22:54.880 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-21T07:22:54.881 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-21T07:22:54.881 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-21T07:22:54.881 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-21T07:22:54.881 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-21T07:22:54.881 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-21T07:22:54.881 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-21T07:22:54.881 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-21T07:22:54.881 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-21T07:22:54.881 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-21T07:22:54.881 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-21T07:22:54.881 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-21T07:22:54.881 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-21T07:22:54.881 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-21T07:22:54.881 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-21T07:22:54.882 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-21T07:22:54.882 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-21T07:22:54.882 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-21T07:22:54.882 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-21T07:22:54.882 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-21T07:22:54.882 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-21T07:22:54.882 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-21T07:22:54.882 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-21T07:22:54.882 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-21T07:22:54.882 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-21T07:22:54.882 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-21T07:22:54.882 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-21T07:22:54.882 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-21T07:22:54.882 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-21T07:22:54.883 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-21T07:22:54.883 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-21T07:22:54.883 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-21T07:22:54.883 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-21T07:22:54.883 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-21T07:22:54.883 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-21T07:22:54.883 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-21T07:22:54.883 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-21T07:22:54.883 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-21T07:22:54.883 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-21T07:22:54.883 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-21T07:22:54.883 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-21T07:22:54.883 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-21T07:22:54.884 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-21T07:22:54.884 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-21T07:22:54.884 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-21T07:22:54.884 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-21T07:22:54.884 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-21T07:22:54.884 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-21T07:22:54.884 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-21T07:22:54.884 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-21T07:22:54.884 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-21T07:22:54.884 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-21T07:22:54.884 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-21T07:22:54.884 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-21T07:22:54.884 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-21T07:22:54.884 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-21T07:22:54.884 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-21T07:22:54.885 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-21T07:22:54.885 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-21T07:22:54.885 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-21T07:22:54.885 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-21T07:22:54.885 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-21T07:22:54.885 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-21T07:22:54.885 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-21T07:22:54.885 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-21T07:22:54.885 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-21T07:22:54.885 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-21T07:22:54.885 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-21T07:22:54.885 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-21T07:22:54.885 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-21T07:22:54.885 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-21T07:22:54.886 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-21T07:22:54.886 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-21T07:22:54.886 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-21T07:22:54.886 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-21T07:22:54.886 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-21T07:22:54.886 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-21T07:22:54.886 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-21T07:22:54.886 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-21T07:22:54.886 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-21T07:22:54.886 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-21T07:22:54.886 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-21T07:22:54.886 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-21T07:22:54.886 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-21T07:22:54.886 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-21T07:22:54.887 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-21T07:22:54.887 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-21T07:22:54.887 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-21T07:22:54.887 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-21T07:22:54.887 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-21T07:22:54.887 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-21T07:22:54.887 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-21T07:22:54.887 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-21T07:22:54.887 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-21T07:22:54.887 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-21T07:22:54.887 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-21T07:22:54.887 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-21T07:22:54.887 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-21T07:22:54.887 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-21T07:22:54.888 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-21T07:22:54.888 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-21T07:22:54.888 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-21T07:22:54.888 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-21T07:22:54.888 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-21T07:22:54.888 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-21T07:22:54.888 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-21T07:22:54.888 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-21T07:22:54.888 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-21T07:22:54.888 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-21T07:22:54.888 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-21T07:22:54.888 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-21T07:22:54.888 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-21T07:22:54.888 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-21T07:22:54.889 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-21T07:22:54.889 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-21T07:22:54.889 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-21T07:22:54.889 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-21T07:22:54.889 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-21T07:22:54.889 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-21T07:22:54.889 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-21T07:22:54.889 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-21T07:22:54.889 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-21T07:22:54.889 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-21T07:22:54.889 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-21T07:22:54.889 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-21T07:22:54.889 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-21T07:22:54.889 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-21T07:22:54.889 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-21T07:22:54.890 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-21T07:22:54.890 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-21T07:22:54.890 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-21T07:22:54.890 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-21T07:22:54.890 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-21T07:22:54.890 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-21T07:22:54.890 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-21T07:22:54.890 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-21T07:22:54.890 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-21T07:22:54.890 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-21T07:22:54.890 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-21T07:22:54.890 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-21T07:22:54.890 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-21T07:22:54.890 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-21T07:22:54.891 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-21T07:22:54.891 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-21T07:22:54.891 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-21T07:22:54.891 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-21T07:22:54.891 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-21T07:22:54.891 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-21T07:22:54.891 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-21T07:22:54.891 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-21T07:22:54.891 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-21T07:22:54.891 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-21T07:22:54.891 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-21T07:22:54.891 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-21T07:22:54.891 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-21T07:22:54.891 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-21T07:22:54.892 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-21T07:22:54.892 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-21T07:22:54.892 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-21T07:22:54.892 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-21T07:22:54.892 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-21T07:22:54.892 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-21T07:22:54.892 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-21T07:22:54.892 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-21T07:22:54.892 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-21T07:22:54.892 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-21T07:22:54.892 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-21T07:22:54.892 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-21T07:22:54.892 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-21T07:22:54.892 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-21T07:22:54.892 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-21T07:22:54.893 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-21T07:22:54.893 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-21T07:22:54.893 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-21T07:22:54.893 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-21T07:22:54.893 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-21T07:22:54.893 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-21T07:22:54.893 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-21T07:22:54.893 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-21T07:22:54.893 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-21T07:22:54.893 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-21T07:22:54.893 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-21T07:22:54.893 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-21T07:22:54.893 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-21T07:22:54.893 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-21T07:22:54.893 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-21T07:22:54.894 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-21T07:22:54.894 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-21T07:22:54.894 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-21T07:22:54.894 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-21T07:22:54.894 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-21T07:22:54.894 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-21T07:22:54.894 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-21T07:22:54.894 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-21T07:22:54.894 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-21T07:22:54.894 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-21T07:22:54.894 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-21T07:22:54.894 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-21T07:22:54.894 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-21T07:22:54.895 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-21T07:22:54.895 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-21T07:22:54.895 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-21T07:22:54.895 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-21T07:22:54.895 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-21T07:22:54.895 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-21T07:22:54.895 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-21T07:22:54.895 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-21T07:22:54.895 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-21T07:22:54.895 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-21T07:22:54.895 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-21T07:22:54.895 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-21T07:22:54.895 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-21T07:22:54.895 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-21T07:22:54.895 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-21T07:22:54.896 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-21T07:22:54.896 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-21T07:22:54.896 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-21T07:22:54.896 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-21T07:22:54.896 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-21T07:22:54.896 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-21T07:22:54.896 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-21T07:22:54.896 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-21T07:22:54.896 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-21T07:22:54.896 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-21T07:22:54.896 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-21T07:22:54.896 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-21T07:22:54.896 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-21T07:22:54.896 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-21T07:22:54.897 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-21T07:22:54.897 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-21T07:22:54.897 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-21T07:22:54.897 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-21T07:22:54.897 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-21T07:22:54.897 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-21T07:22:54.897 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-21T07:22:54.897 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-21T07:22:54.897 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-21T07:22:54.897 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-21T07:22:54.897 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-21T07:22:54.897 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-21T07:22:54.897 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-21T07:22:54.898 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-21T07:22:54.898 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-21T07:22:54.898 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-21T07:22:54.898 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-21T07:22:54.898 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-21T07:22:54.898 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-21T07:22:54.898 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-21T07:22:54.898 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-21T07:22:54.898 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-21T07:22:54.898 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-21T07:22:54.898 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-21T07:22:54.898 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-21T07:22:54.898 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-21T07:22:54.898 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-21T07:22:54.899 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-21T07:22:54.899 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-21T07:22:54.899 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-21T07:22:54.899 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-21T07:22:54.899 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-21T07:22:54.899 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-21T07:22:54.899 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-21T07:22:54.899 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-21T07:22:54.899 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-21T07:22:54.899 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-21T07:22:54.899 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-21T07:22:54.899 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-21T07:22:54.899 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-21T07:22:54.899 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-21T07:22:54.899 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-21T07:22:54.900 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-21T07:22:54.900 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-21T07:22:54.900 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-21T07:22:54.900 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-21T07:22:54.900 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-21T07:22:54.900 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-21T07:22:54.900 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-21T07:22:54.900 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-21T07:22:54.900 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-21T07:22:54.900 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-21T07:22:54.900 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-21T07:22:54.900 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-21T07:22:54.900 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-21T07:22:54.900 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-21T07:22:54.901 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-21T07:22:54.901 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-21T07:22:54.901 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-21T07:22:54.901 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-21T07:22:54.901 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-21T07:22:54.901 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-21T07:22:54.901 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-21T07:22:54.901 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-21T07:22:54.901 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-21T07:22:54.901 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-21T07:22:54.901 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-21T07:22:54.901 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-21T07:22:54.901 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-21T07:22:54.901 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-21T07:22:54.902 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-21T07:22:54.902 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-21T07:22:54.902 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-21T07:22:54.902 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-21T07:22:54.902 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-21T07:22:54.902 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-21T07:22:54.902 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-21T07:22:54.902 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-21T07:22:54.902 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-21T07:22:54.902 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-21T07:22:54.902 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-21T07:22:54.902 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-21T07:22:54.902 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-21T07:22:54.902 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-21T07:22:54.902 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-21T07:22:54.903 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-21T07:22:54.903 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-21T07:22:54.903 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-21T07:22:54.903 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-21T07:22:54.903 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-21T07:22:54.903 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-21T07:22:54.903 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-21T07:22:54.903 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-21T07:22:54.903 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-21T07:22:54.903 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-21T07:22:54.903 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-21T07:22:54.903 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-21T07:22:54.903 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-21T07:22:54.903 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-21T07:22:54.904 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-21T07:22:54.904 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-21T07:22:54.904 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-21T07:22:54.904 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-21T07:22:54.904 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-21T07:22:54.904 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-21T07:22:54.904 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-21T07:22:54.904 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-21T07:22:54.904 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-21T07:22:54.904 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-21T07:22:54.904 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-21T07:22:54.904 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-21T07:22:54.904 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-21T07:22:54.904 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-21T07:22:54.905 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-21T07:22:54.905 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-21T07:22:54.905 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-21T07:22:54.905 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-21T07:22:54.905 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-21T07:22:54.905 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-21T07:22:54.905 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-21T07:22:54.905 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-21T07:22:54.905 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-21T07:22:54.905 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-21T07:22:54.905 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-21T07:22:54.905 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-21T07:22:54.905 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-21T07:22:54.905 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-21T07:22:54.906 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-21T07:22:54.906 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-21T07:22:54.906 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-21T07:22:54.906 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-21T07:22:54.906 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-21T07:22:54.906 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-21T07:22:54.906 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-21T07:22:54.906 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-21T07:22:54.906 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-21T07:22:54.906 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-21T07:22:54.906 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-21T07:22:54.906 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-21T07:22:54.906 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-21T07:22:54.906 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-21T07:22:54.907 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-21T07:22:54.907 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-21T07:22:54.907 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-21T07:22:54.907 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-21T07:22:54.907 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-21T07:22:54.907 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-21T07:22:54.907 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-21T07:22:54.907 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-21T07:22:54.907 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-21T07:22:54.907 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-21T07:22:54.907 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-21T07:22:54.907 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-21T07:22:54.907 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-21T07:22:54.907 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-21T07:22:54.908 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-21T07:22:54.908 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-21T07:22:54.908 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-21T07:22:54.908 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-21T07:22:54.908 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-21T07:22:54.908 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-21T07:22:54.908 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-21T07:22:54.908 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-21T07:22:54.908 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-21T07:22:54.908 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-21T07:22:54.908 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-21T07:22:54.908 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-21T07:22:54.908 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-21T07:22:54.908 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-21T07:22:54.909 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-21T07:22:54.909 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-21T07:22:54.909 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-21T07:22:54.909 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-21T07:22:54.909 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-21T07:22:54.909 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-21T07:22:54.909 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-21T07:22:54.909 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-21T07:22:54.909 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-21T07:22:54.909 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-21T07:22:54.909 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-21T07:22:54.909 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-21T07:22:54.909 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-21T07:22:54.909 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-21T07:22:54.910 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-21T07:22:54.910 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-21T07:22:54.910 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-21T07:22:54.910 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-21T07:22:54.910 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-21T07:22:54.910 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-21T07:22:54.910 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-21T07:22:54.910 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-21T07:22:54.910 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-21T07:22:54.910 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-21T07:22:54.910 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-21T07:22:54.910 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-21T07:22:54.910 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-21T07:22:54.910 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-21T07:22:54.911 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-21T07:22:54.911 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-21T07:22:54.911 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-21T07:22:54.911 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-21T07:22:54.911 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-21T07:22:54.911 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-21T07:22:54.911 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-21T07:22:54.911 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-21T07:22:54.911 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-21T07:22:54.911 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-21T07:22:54.911 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-21T07:22:54.911 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-21T07:22:54.911 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-21T07:22:54.911 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-21T07:22:54.912 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-21T07:22:54.912 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-21T07:22:54.912 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-21T07:22:54.912 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-21T07:22:54.912 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-21T07:22:54.912 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-21T07:22:54.912 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-21T07:22:54.912 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-21T07:22:54.912 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-21T07:22:54.912 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-21T07:22:54.912 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-21T07:22:54.912 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-21T07:22:54.912 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-21T07:22:54.912 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-21T07:22:54.913 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-21T07:22:54.913 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-21T07:22:54.913 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-21T07:22:54.913 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-21T07:22:54.913 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-21T07:22:54.913 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-21T07:22:54.913 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-21T07:22:54.913 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-21T07:22:54.913 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-21T07:22:54.913 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-21T07:22:54.913 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-21T07:22:54.913 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-21T07:22:54.913 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-21T07:22:54.913 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-21T07:22:54.914 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-21T07:22:54.914 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-21T07:22:54.914 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-21T07:22:54.914 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-21T07:22:54.914 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-21T07:22:54.914 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-21T07:22:54.914 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-21T07:22:54.914 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-21T07:22:54.914 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-21T07:22:54.914 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-21T07:22:54.914 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-21T07:22:54.914 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-21T07:22:54.914 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-21T07:22:54.914 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-21T07:22:54.915 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-21T07:22:54.915 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-21T07:22:54.915 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-21T07:22:54.915 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-21T07:22:54.915 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-21T07:22:54.915 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-21T07:22:54.915 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-21T07:22:54.915 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-21T07:22:54.915 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-21T07:22:54.915 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-21T07:22:54.915 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-21T07:22:54.915 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-21T07:22:54.915 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-21T07:22:54.915 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-21T07:22:54.915 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-21T07:22:54.916 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-21T07:22:54.916 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-21T07:22:54.916 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-21T07:22:54.916 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-21T07:22:54.916 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-21T07:22:54.916 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-21T07:22:54.916 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-21T07:22:54.916 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-21T07:22:54.916 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-21T07:22:54.916 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-21T07:22:54.916 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-21T07:22:54.916 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-21T07:22:54.916 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-21T07:22:54.916 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-21T07:22:54.916 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-21T07:22:54.917 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-21T07:22:54.917 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-21T07:22:54.917 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-21T07:22:54.917 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-21T07:22:54.917 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-21T07:22:54.917 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-21T07:22:54.917 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-21T07:22:54.917 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-21T07:22:54.917 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-21T07:22:54.917 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-21T07:22:54.917 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-21T07:22:54.917 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-21T07:22:54.917 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-21T07:22:54.917 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-21T07:22:54.918 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-21T07:22:54.918 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-21T07:22:54.918 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-21T07:22:54.918 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-21T07:22:54.918 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-21T07:22:54.918 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-21T07:22:54.918 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-21T07:22:54.918 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-21T07:22:54.918 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-21T07:22:54.918 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-21T07:22:54.918 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-21T07:22:54.918 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-21T07:22:54.918 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-21T07:22:54.918 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-21T07:22:54.919 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-21T07:22:54.919 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-21T07:22:54.919 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-21T07:22:54.919 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-21T07:22:54.919 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-21T07:22:54.919 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-21T07:22:54.919 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-21T07:22:54.919 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-21T07:22:54.919 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-21T07:22:54.919 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-21T07:22:54.919 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-21T07:22:54.919 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-21T07:22:54.919 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-21T07:22:54.919 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-21T07:22:54.919 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-21T07:22:54.920 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-21T07:22:54.920 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-21T07:22:54.920 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-21T07:22:54.920 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-21T07:22:54.920 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-21T07:22:54.920 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-21T07:22:54.920 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-21T07:22:54.920 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-21T07:22:54.920 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-21T07:22:54.920 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-21T07:22:54.920 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-21T07:22:54.920 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-21T07:22:54.920 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-21T07:22:54.920 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-21T07:22:54.920 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-21T07:22:54.921 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-21T07:22:54.921 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-21T07:22:54.921 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-21T07:22:54.921 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-21T07:22:54.921 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-21T07:22:54.921 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-21T07:22:54.921 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-21T07:22:54.921 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-21T07:22:54.921 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-21T07:22:54.921 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-21T07:22:54.921 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-21T07:22:54.921 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-21T07:22:54.921 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-21T07:22:54.921 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-21T07:22:54.922 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-21T07:22:54.922 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-21T07:22:54.922 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-21T07:22:54.922 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-21T07:22:54.922 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-21T07:22:54.922 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-21T07:22:54.922 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-21T07:22:54.922 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-21T07:22:54.922 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-21T07:22:54.922 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-21T07:22:54.922 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-21T07:22:54.922 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-21T07:22:54.922 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-21T07:22:54.922 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-21T07:22:54.923 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-21T07:22:54.923 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-21T07:22:54.923 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-21T07:22:54.923 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-21T07:22:54.923 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-21T07:22:54.923 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-21T07:22:54.923 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-21T07:22:54.923 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-21T07:22:54.923 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-21T07:22:54.923 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-21T07:22:54.923 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-21T07:22:54.923 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-21T07:22:54.923 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-21T07:22:54.923 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-21T07:22:54.924 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-21T07:22:54.924 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-21T07:22:54.924 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-21T07:22:54.924 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-21T07:22:54.924 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-21T07:22:54.924 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-21T07:22:54.924 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-21T07:22:54.924 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-21T07:22:54.924 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-21T07:22:54.924 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-21T07:22:54.924 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-21T07:22:54.924 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-21T07:22:54.924 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-21T07:22:54.924 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-21T07:22:54.925 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-21T07:22:54.925 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-21T07:22:54.925 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-21T07:22:54.925 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-21T07:22:54.925 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-21T07:22:54.925 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-21T07:22:54.925 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-21T07:22:54.925 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-21T07:22:54.925 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-21T07:22:54.925 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-21T07:22:54.925 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-21T07:22:54.925 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-21T07:22:54.925 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-21T07:22:54.925 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-21T07:22:54.925 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-21T07:22:54.926 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-21T07:22:54.926 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-21T07:22:54.926 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-21T07:22:54.926 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-21T07:22:54.926 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-21T07:22:54.926 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-21T07:22:54.926 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-21T07:22:54.926 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-21T07:22:54.926 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-21T07:22:54.926 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-21T07:22:54.926 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-21T07:22:54.926 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-21T07:22:54.926 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-21T07:22:54.926 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-21T07:22:54.927 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-21T07:22:54.927 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-21T07:22:54.927 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-21T07:22:54.927 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-21T07:22:54.927 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-21T07:22:54.927 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-21T07:22:54.927 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-21T07:22:54.927 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-21T07:22:54.927 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-21T07:22:54.927 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-21T07:22:54.927 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-21T07:22:54.927 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-21T07:22:54.927 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-21T07:22:54.927 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-21T07:22:54.928 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-21T07:22:54.928 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-21T07:22:54.928 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-21T07:22:54.928 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-21T07:22:54.928 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-21T07:22:54.928 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-21T07:22:54.928 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-21T07:22:54.928 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-21T07:22:54.928 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-21T07:22:54.928 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-21T07:22:54.928 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-21T07:22:54.928 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-21T07:22:54.928 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-21T07:22:54.928 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-21T07:22:54.928 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-21T07:22:54.929 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-21T07:22:54.929 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-21T07:22:54.929 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-21T07:22:54.929 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-21T07:22:54.929 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-21T07:22:54.929 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-21T07:22:54.929 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-21T07:22:54.929 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-21T07:22:54.929 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-21T07:22:54.929 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-21T07:22:54.929 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-21T07:22:54.929 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-21T07:22:54.929 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-21T07:22:54.929 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-21T07:22:54.930 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-21T07:22:54.930 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-21T07:22:54.930 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-21T07:22:54.930 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-21T07:22:54.930 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-21T07:22:54.930 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-21T07:22:54.930 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-21T07:22:54.930 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-21T07:22:54.930 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-21T07:22:54.930 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-21T07:22:54.930 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-21T07:22:54.930 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-21T07:22:54.930 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-21T07:22:54.930 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-21T07:22:54.930 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-21T07:22:54.931 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-21T07:22:54.931 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-21T07:22:54.931 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-21T07:22:54.931 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-21T07:22:54.931 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-21T07:22:54.931 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-21T07:22:54.931 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-21T07:22:54.931 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-21T07:22:54.931 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-21T07:22:54.931 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-21T07:22:54.931 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-21T07:22:54.931 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-21T07:22:54.931 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-21T07:22:54.931 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-21T07:22:54.932 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-21T07:22:54.932 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-21T07:22:54.932 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-21T07:22:54.932 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-21T07:22:54.932 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-21T07:22:54.932 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-21T07:22:54.932 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-21T07:22:54.932 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-21T07:22:54.932 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-21T07:22:54.932 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-21T07:22:54.932 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-21T07:22:54.932 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-21T07:22:54.932 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-21T07:22:54.932 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-21T07:22:54.933 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-21T07:22:54.933 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-21T07:22:54.933 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-21T07:22:54.933 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-21T07:22:54.933 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-21T07:22:54.933 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-21T07:22:54.933 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-21T07:22:54.933 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-21T07:22:54.933 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-21T07:22:54.933 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-21T07:22:54.933 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-21T07:22:54.933 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-21T07:22:54.933 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-21T07:22:54.933 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-21T07:22:54.934 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-21T07:22:54.934 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-21T07:22:54.934 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-21T07:22:54.934 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-21T07:22:54.934 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-21T07:22:54.934 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-21T07:22:54.934 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-21T07:22:54.934 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-21T07:22:54.934 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-21T07:22:54.934 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-21T07:22:54.934 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-21T07:22:54.934 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-21T07:22:54.934 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-21T07:22:54.934 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-21T07:22:54.935 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-21T07:22:54.935 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-21T07:22:54.935 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-21T07:22:54.935 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-21T07:22:54.935 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-21T07:22:54.935 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-21T07:22:54.935 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-21T07:22:54.935 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-21T07:22:54.935 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-21T07:22:54.935 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-21T07:22:54.935 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-21T07:22:54.935 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-21T07:22:54.935 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-21T07:22:54.935 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-21T07:22:54.936 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-21T07:22:54.936 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-21T07:22:54.936 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-21T07:22:54.936 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-21T07:22:54.936 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-21T07:22:54.936 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-21T07:22:54.936 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-21T07:22:54.936 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-21T07:22:54.936 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-21T07:22:54.936 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-21T07:22:54.936 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-21T07:22:54.936 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-21T07:22:54.936 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-21T07:22:54.937 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-21T07:22:54.937 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-21T07:22:54.937 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-21T07:22:54.937 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-21T07:22:54.937 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-21T07:22:54.937 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-21T07:22:54.937 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-21T07:22:54.937 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-21T07:22:54.937 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-21T07:22:54.937 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-21T07:22:54.937 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-21T07:22:54.937 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-21T07:22:54.937 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-21T07:22:54.937 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-21T07:22:54.938 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-21T07:22:54.938 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-21T07:22:54.938 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-21T07:22:54.938 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-21T07:22:54.938 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-21T07:22:54.938 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-21T07:22:54.938 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-21T07:22:54.938 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-21T07:22:54.938 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-21T07:22:54.938 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-21T07:22:54.938 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-21T07:22:54.938 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-21T07:22:54.939 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-21T07:22:54.939 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-21T07:22:54.939 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-21T07:22:54.939 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-21T07:22:54.939 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-21T07:22:54.939 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-21T07:22:54.939 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-21T07:22:54.939 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-21T07:22:54.939 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-21T07:22:54.939 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-21T07:22:54.939 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-21T07:22:54.939 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-21T07:22:54.939 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-21T07:22:54.939 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-21T07:22:54.940 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-21T07:22:54.940 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-21T07:22:54.940 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-21T07:22:54.940 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-21T07:22:54.940 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-21T07:22:54.940 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-21T07:22:54.940 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-21T07:22:54.940 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-21T07:22:54.940 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-21T07:22:54.940 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-21T07:22:54.940 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-21T07:22:54.940 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-21T07:22:54.940 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-21T07:22:54.940 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-21T07:22:54.941 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-21T07:22:54.941 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-21T07:22:54.941 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-21T07:22:54.941 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-21T07:22:54.941 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-21T07:22:54.941 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-21T07:22:54.941 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-21T07:22:54.941 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-21T07:22:54.941 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-21T07:22:54.941 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-21T07:22:54.941 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-21T07:22:54.941 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-21T07:22:54.941 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-21T07:22:54.941 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-21T07:22:54.942 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-21T07:22:54.942 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-21T07:22:54.942 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-21T07:22:54.942 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-21T07:22:54.942 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-21T07:22:54.942 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-21T07:22:54.942 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-21T07:22:54.942 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-21T07:22:54.942 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-21T07:22:54.942 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-21T07:22:54.942 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-21T07:22:54.942 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-21T07:22:54.942 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-21T07:22:54.942 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-21T07:22:54.942 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-21T07:22:54.943 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-21T07:22:54.943 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-21T07:22:54.943 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-21T07:22:54.943 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-21T07:22:54.943 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-21T07:22:54.943 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-21T07:22:54.943 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-21T07:22:54.943 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-21T07:22:54.943 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-21T07:22:54.943 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-21T07:22:54.943 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-21T07:22:54.943 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-21T07:22:54.943 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-21T07:22:54.943 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-21T07:22:54.944 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-21T07:22:54.944 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-21T07:22:54.944 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-21T07:22:54.944 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-21T07:22:54.944 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-21T07:22:54.944 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-21T07:22:54.944 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-21T07:22:54.944 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-21T07:22:54.944 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-21T07:22:54.944 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-21T07:22:54.944 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-21T07:22:54.944 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-21T07:22:54.944 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-21T07:22:54.944 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-21T07:22:54.945 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-21T07:22:54.945 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-21T07:22:54.945 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-21T07:22:54.945 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-21T07:22:54.945 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-21T07:22:54.945 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-21T07:22:54.945 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-21T07:22:54.945 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-21T07:22:54.945 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-21T07:22:54.945 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-21T07:22:54.945 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-21T07:22:54.945 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-21T07:22:54.945 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-21T07:22:54.946 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-21T07:22:54.946 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-21T07:22:54.946 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-21T07:22:54.946 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-21T07:22:54.946 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-21T07:22:54.946 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-21T07:22:54.946 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-21T07:22:54.946 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-21T07:22:54.946 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-21T07:22:54.946 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-21T07:22:54.946 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-21T07:22:54.946 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-21T07:22:54.946 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-21T07:22:54.946 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-21T07:22:54.947 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-21T07:22:54.947 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-21T07:22:54.947 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-21T07:22:54.947 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-21T07:22:54.947 INFO:teuthology.orchestra.run.smithi148.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-21T07:22:54.951 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:22:54.951 DEBUG:teuthology.orchestra.run.smithi148:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-21T07:22:54.975 INFO:teuthology.orchestra.run.smithi148.stdout:postinstall scriptlet (using /bin/sh): 2024-09-21T07:22:54.975 INFO:teuthology.orchestra.run.smithi148.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-21T07:22:54.975 INFO:teuthology.orchestra.run.smithi148.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-21T07:22:54.975 INFO:teuthology.orchestra.run.smithi148.stdout:fi 2024-09-21T07:22:54.975 INFO:teuthology.orchestra.run.smithi148.stdout:for file in vmlinuz System.map config; do 2024-09-21T07:22:54.975 INFO:teuthology.orchestra.run.smithi148.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-21T07:22:54.975 INFO:teuthology.orchestra.run.smithi148.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-21T07:22:54.975 INFO:teuthology.orchestra.run.smithi148.stdout: fi 2024-09-21T07:22:54.975 INFO:teuthology.orchestra.run.smithi148.stdout:done 2024-09-21T07:22:54.975 INFO:teuthology.orchestra.run.smithi148.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-21T07:22:54.975 INFO:teuthology.orchestra.run.smithi148.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-21T07:22:54.976 INFO:teuthology.orchestra.run.smithi148.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-21T07:22:54.976 INFO:teuthology.orchestra.run.smithi148.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-21T07:22:54.976 INFO:teuthology.orchestra.run.smithi148.stdout:fi 2024-09-21T07:22:54.976 INFO:teuthology.orchestra.run.smithi148.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-21T07:22:54.976 INFO:teuthology.orchestra.run.smithi148.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:22:54.976 INFO:teuthology.orchestra.run.smithi148.stdout:fi 2024-09-21T07:22:54.976 INFO:teuthology.orchestra.run.smithi148.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-21T07:22:54.976 INFO:teuthology.orchestra.run.smithi148.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-21T07:22:54.976 INFO:teuthology.orchestra.run.smithi148.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:22:54.976 INFO:teuthology.orchestra.run.smithi148.stdout:fi 2024-09-21T07:22:54.977 DEBUG:teuthology.orchestra.run.smithi148:> sudo rpm -qi grub2-tools 2024-09-21T07:22:55.060 INFO:teuthology.orchestra.run.smithi148.stdout:Name : grub2-tools 2024-09-21T07:22:55.060 INFO:teuthology.orchestra.run.smithi148.stdout:Epoch : 1 2024-09-21T07:22:55.060 INFO:teuthology.orchestra.run.smithi148.stdout:Version : 2.06 2024-09-21T07:22:55.060 INFO:teuthology.orchestra.run.smithi148.stdout:Release : 82.el9 2024-09-21T07:22:55.060 INFO:teuthology.orchestra.run.smithi148.stdout:Architecture: x86_64 2024-09-21T07:22:55.061 INFO:teuthology.orchestra.run.smithi148.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T07:22:55.061 INFO:teuthology.orchestra.run.smithi148.stdout:Group : Unspecified 2024-09-21T07:22:55.061 INFO:teuthology.orchestra.run.smithi148.stdout:Size : 8274670 2024-09-21T07:22:55.061 INFO:teuthology.orchestra.run.smithi148.stdout:License : GPLv3+ 2024-09-21T07:22:55.061 INFO:teuthology.orchestra.run.smithi148.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T07:22:55.061 INFO:teuthology.orchestra.run.smithi148.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T07:22:55.061 INFO:teuthology.orchestra.run.smithi148.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T07:22:55.061 INFO:teuthology.orchestra.run.smithi148.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T07:22:55.061 INFO:teuthology.orchestra.run.smithi148.stdout:Packager : builder@centos.org 2024-09-21T07:22:55.061 INFO:teuthology.orchestra.run.smithi148.stdout:Vendor : CentOS 2024-09-21T07:22:55.061 INFO:teuthology.orchestra.run.smithi148.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T07:22:55.061 INFO:teuthology.orchestra.run.smithi148.stdout:Summary : Support tools for GRUB. 2024-09-21T07:22:55.062 INFO:teuthology.orchestra.run.smithi148.stdout:Description : 2024-09-21T07:22:55.062 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:22:55.062 INFO:teuthology.orchestra.run.smithi148.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T07:22:55.062 INFO:teuthology.orchestra.run.smithi148.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T07:22:55.062 INFO:teuthology.orchestra.run.smithi148.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T07:22:55.062 INFO:teuthology.orchestra.run.smithi148.stdout:hardware devices. 2024-09-21T07:22:55.062 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:22:55.062 INFO:teuthology.orchestra.run.smithi148.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T07:22:55.063 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T07:22:55.063 INFO:teuthology.task.kernel:Updating grub on smithi148 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:22:55.063 DEBUG:teuthology.orchestra.run.smithi148:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T07:22:55.666 INFO:teuthology.orchestra.run.smithi148.stderr:Generating grub configuration file ... 2024-09-21T07:22:57.027 INFO:teuthology.orchestra.run.smithi148.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T07:22:57.059 INFO:teuthology.orchestra.run.smithi148.stderr:done 2024-09-21T07:22:57.061 DEBUG:teuthology.orchestra.run.smithi148:> mktemp 2024-09-21T07:22:57.076 INFO:teuthology.orchestra.run.smithi148.stdout:/tmp/tmp.Ti5A9L5Lzr 2024-09-21T07:22:57.076 DEBUG:teuthology.orchestra.run.smithi148:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Ti5A9L5Lzr 2024-09-21T07:22:57.143 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0666 /tmp/tmp.Ti5A9L5Lzr 2024-09-21T07:22:57.279 DEBUG:teuthology.orchestra.remote:smithi148:/tmp/tmp.Ti5A9L5Lzr is 6KB 2024-09-21T07:22:57.299 DEBUG:teuthology.orchestra.run.smithi148:> rm -fr /tmp/tmp.Ti5A9L5Lzr 2024-09-21T07:22:57.313 DEBUG:teuthology.orchestra.run.smithi148:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T07:22:57.379 INFO:teuthology.orchestra.run.smithi148.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T07:22:57.379 INFO:teuthology.orchestra.run.smithi148.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T07:22:57.380 INFO:teuthology.orchestra.run.smithi148.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T07:22:57.380 INFO:teuthology.orchestra.run.smithi148.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-21T07:22:57.381 DEBUG:teuthology.orchestra.run.smithi148:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:22:57.538 DEBUG:teuthology.orchestra.run.smithi148:> sudo shutdown -r now 2024-09-21T07:23:01.290 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi039.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-21T07:23:01.290 DEBUG:teuthology.orchestra.run.smithi039:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-21T07:23:01.381 INFO:teuthology.orchestra.run.smithi039.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:23:01.381 INFO:teuthology.orchestra.run.smithi039.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:23:01.381 INFO:teuthology.orchestra.run.smithi039.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:23:01.381 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:23:01.381 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-21T07:23:01.381 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-21T07:23:01.381 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-21T07:23:01.382 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-21T07:23:01.382 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-21T07:23:01.382 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-21T07:23:01.382 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-21T07:23:01.382 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-21T07:23:01.382 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-21T07:23:01.382 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-21T07:23:01.382 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-21T07:23:01.382 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-21T07:23:01.382 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-21T07:23:01.382 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-21T07:23:01.383 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-21T07:23:01.383 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-21T07:23:01.383 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-21T07:23:01.383 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-21T07:23:01.383 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-21T07:23:01.383 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-21T07:23:01.383 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-21T07:23:01.383 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-21T07:23:01.383 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-21T07:23:01.383 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-21T07:23:01.383 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-21T07:23:01.383 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-21T07:23:01.383 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-21T07:23:01.383 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-21T07:23:01.383 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-21T07:23:01.384 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-21T07:23:01.384 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-21T07:23:01.384 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-21T07:23:01.384 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-21T07:23:01.384 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-21T07:23:01.384 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-21T07:23:01.384 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-21T07:23:01.384 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-21T07:23:01.384 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-21T07:23:01.384 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-21T07:23:01.384 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-21T07:23:01.384 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-21T07:23:01.384 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-21T07:23:01.384 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-21T07:23:01.385 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-21T07:23:01.385 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-21T07:23:01.385 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-21T07:23:01.385 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-21T07:23:01.385 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-21T07:23:01.385 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-21T07:23:01.385 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-21T07:23:01.385 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-21T07:23:01.385 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-21T07:23:01.385 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-21T07:23:01.385 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-21T07:23:01.385 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-21T07:23:01.385 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-21T07:23:01.385 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-21T07:23:01.386 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-21T07:23:01.386 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-21T07:23:01.386 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-21T07:23:01.386 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-21T07:23:01.386 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-21T07:23:01.386 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-21T07:23:01.386 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-21T07:23:01.386 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-21T07:23:01.386 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-21T07:23:01.386 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-21T07:23:01.386 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-21T07:23:01.386 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-21T07:23:01.386 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-21T07:23:01.387 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-21T07:23:01.387 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-21T07:23:01.387 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-21T07:23:01.387 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-21T07:23:01.387 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-21T07:23:01.387 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-21T07:23:01.387 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-21T07:23:01.387 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-21T07:23:01.387 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-21T07:23:01.387 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-21T07:23:01.387 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-21T07:23:01.387 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-21T07:23:01.387 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-21T07:23:01.387 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-21T07:23:01.387 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-21T07:23:01.388 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-21T07:23:01.388 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-21T07:23:01.388 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-21T07:23:01.388 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-21T07:23:01.388 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-21T07:23:01.388 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-21T07:23:01.388 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-21T07:23:01.388 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-21T07:23:01.388 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-21T07:23:01.388 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-21T07:23:01.388 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-21T07:23:01.388 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-21T07:23:01.388 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-21T07:23:01.388 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-21T07:23:01.389 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-21T07:23:01.389 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-21T07:23:01.389 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-21T07:23:01.389 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-21T07:23:01.389 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-21T07:23:01.389 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-21T07:23:01.389 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-21T07:23:01.389 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-21T07:23:01.389 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-21T07:23:01.389 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-21T07:23:01.389 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-21T07:23:01.389 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-21T07:23:01.389 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-21T07:23:01.389 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-21T07:23:01.389 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-21T07:23:01.390 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-21T07:23:01.390 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-21T07:23:01.390 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-21T07:23:01.390 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-21T07:23:01.390 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-21T07:23:01.390 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-21T07:23:01.390 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-21T07:23:01.390 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-21T07:23:01.390 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-21T07:23:01.390 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-21T07:23:01.390 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-21T07:23:01.390 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-21T07:23:01.390 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-21T07:23:01.391 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-21T07:23:01.391 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-21T07:23:01.391 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-21T07:23:01.391 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-21T07:23:01.391 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-21T07:23:01.391 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-21T07:23:01.391 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-21T07:23:01.391 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-21T07:23:01.391 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-21T07:23:01.391 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-21T07:23:01.391 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-21T07:23:01.391 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-21T07:23:01.391 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-21T07:23:01.391 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-21T07:23:01.391 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-21T07:23:01.392 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-21T07:23:01.392 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-21T07:23:01.392 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-21T07:23:01.392 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-21T07:23:01.392 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-21T07:23:01.392 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-21T07:23:01.392 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-21T07:23:01.392 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-21T07:23:01.392 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-21T07:23:01.392 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-21T07:23:01.392 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-21T07:23:01.392 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-21T07:23:01.392 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-21T07:23:01.392 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-21T07:23:01.393 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-21T07:23:01.393 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-21T07:23:01.393 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-21T07:23:01.393 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-21T07:23:01.393 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-21T07:23:01.393 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-21T07:23:01.393 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-21T07:23:01.393 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-21T07:23:01.393 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-21T07:23:01.393 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-21T07:23:01.393 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-21T07:23:01.393 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-21T07:23:01.393 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-21T07:23:01.393 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-21T07:23:01.394 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-21T07:23:01.394 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-21T07:23:01.394 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-21T07:23:01.394 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-21T07:23:01.394 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-21T07:23:01.394 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-21T07:23:01.394 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-21T07:23:01.394 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-21T07:23:01.394 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-21T07:23:01.394 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-21T07:23:01.394 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-21T07:23:01.394 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-21T07:23:01.394 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-21T07:23:01.394 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-21T07:23:01.395 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-21T07:23:01.395 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-21T07:23:01.395 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-21T07:23:01.395 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-21T07:23:01.395 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-21T07:23:01.395 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-21T07:23:01.395 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-21T07:23:01.395 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-21T07:23:01.395 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-21T07:23:01.395 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-21T07:23:01.395 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-21T07:23:01.395 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-21T07:23:01.395 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-21T07:23:01.395 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-21T07:23:01.396 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-21T07:23:01.396 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-21T07:23:01.396 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-21T07:23:01.396 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-21T07:23:01.396 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-21T07:23:01.396 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-21T07:23:01.396 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-21T07:23:01.396 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-21T07:23:01.396 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-21T07:23:01.396 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-21T07:23:01.396 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-21T07:23:01.396 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-21T07:23:01.396 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-21T07:23:01.396 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-21T07:23:01.397 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-21T07:23:01.397 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-21T07:23:01.397 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-21T07:23:01.397 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-21T07:23:01.397 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-21T07:23:01.397 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-21T07:23:01.397 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-21T07:23:01.397 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-21T07:23:01.397 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-21T07:23:01.397 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-21T07:23:01.398 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-21T07:23:01.398 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-21T07:23:01.398 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-21T07:23:01.398 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-21T07:23:01.398 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-21T07:23:01.398 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-21T07:23:01.398 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-21T07:23:01.398 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-21T07:23:01.398 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-21T07:23:01.398 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-21T07:23:01.399 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-21T07:23:01.399 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-21T07:23:01.399 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-21T07:23:01.399 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-21T07:23:01.399 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-21T07:23:01.399 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-21T07:23:01.399 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-21T07:23:01.399 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-21T07:23:01.399 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-21T07:23:01.399 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-21T07:23:01.399 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-21T07:23:01.399 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-21T07:23:01.399 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-21T07:23:01.399 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-21T07:23:01.400 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-21T07:23:01.400 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-21T07:23:01.400 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-21T07:23:01.400 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-21T07:23:01.400 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-21T07:23:01.400 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-21T07:23:01.400 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-21T07:23:01.400 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-21T07:23:01.400 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-21T07:23:01.400 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-21T07:23:01.400 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-21T07:23:01.400 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-21T07:23:01.400 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-21T07:23:01.400 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-21T07:23:01.401 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-21T07:23:01.401 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-21T07:23:01.401 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-21T07:23:01.401 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-21T07:23:01.401 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-21T07:23:01.401 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-21T07:23:01.401 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-21T07:23:01.401 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-21T07:23:01.401 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-21T07:23:01.401 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-21T07:23:01.401 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-21T07:23:01.401 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-21T07:23:01.401 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-21T07:23:01.401 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-21T07:23:01.402 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-21T07:23:01.402 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-21T07:23:01.402 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-21T07:23:01.402 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-21T07:23:01.402 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-21T07:23:01.402 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-21T07:23:01.402 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-21T07:23:01.402 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-21T07:23:01.402 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-21T07:23:01.402 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-21T07:23:01.402 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-21T07:23:01.402 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-21T07:23:01.402 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-21T07:23:01.403 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-21T07:23:01.403 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-21T07:23:01.403 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-21T07:23:01.403 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-21T07:23:01.403 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-21T07:23:01.403 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-21T07:23:01.403 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-21T07:23:01.403 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-21T07:23:01.403 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-21T07:23:01.403 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-21T07:23:01.403 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-21T07:23:01.403 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-21T07:23:01.403 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-21T07:23:01.403 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-21T07:23:01.404 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-21T07:23:01.404 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-21T07:23:01.404 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-21T07:23:01.404 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-21T07:23:01.404 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-21T07:23:01.404 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-21T07:23:01.404 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-21T07:23:01.404 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-21T07:23:01.404 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-21T07:23:01.404 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-21T07:23:01.404 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-21T07:23:01.404 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-21T07:23:01.404 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-21T07:23:01.404 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-21T07:23:01.405 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-21T07:23:01.405 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-21T07:23:01.405 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-21T07:23:01.405 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-21T07:23:01.405 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-21T07:23:01.405 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-21T07:23:01.405 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-21T07:23:01.405 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-21T07:23:01.405 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-21T07:23:01.405 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-21T07:23:01.405 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-21T07:23:01.405 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-21T07:23:01.405 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-21T07:23:01.405 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-21T07:23:01.406 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-21T07:23:01.406 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-21T07:23:01.406 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-21T07:23:01.406 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-21T07:23:01.406 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-21T07:23:01.406 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-21T07:23:01.406 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-21T07:23:01.406 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-21T07:23:01.406 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-21T07:23:01.406 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-21T07:23:01.406 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-21T07:23:01.406 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-21T07:23:01.406 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-21T07:23:01.407 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-21T07:23:01.407 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-21T07:23:01.407 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-21T07:23:01.407 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-21T07:23:01.407 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-21T07:23:01.407 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-21T07:23:01.407 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-21T07:23:01.407 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-21T07:23:01.407 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-21T07:23:01.407 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-21T07:23:01.407 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-21T07:23:01.407 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-21T07:23:01.408 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-21T07:23:01.408 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-21T07:23:01.408 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-21T07:23:01.408 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-21T07:23:01.408 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-21T07:23:01.408 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-21T07:23:01.408 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-21T07:23:01.408 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-21T07:23:01.408 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-21T07:23:01.408 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-21T07:23:01.408 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-21T07:23:01.408 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-21T07:23:01.408 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-21T07:23:01.409 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-21T07:23:01.409 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-21T07:23:01.409 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-21T07:23:01.409 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-21T07:23:01.409 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-21T07:23:01.409 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-21T07:23:01.409 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-21T07:23:01.409 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-21T07:23:01.409 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-21T07:23:01.409 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-21T07:23:01.409 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-21T07:23:01.409 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-21T07:23:01.409 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-21T07:23:01.410 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-21T07:23:01.410 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-21T07:23:01.410 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-21T07:23:01.410 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-21T07:23:01.410 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-21T07:23:01.410 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-21T07:23:01.410 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-21T07:23:01.410 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-21T07:23:01.410 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-21T07:23:01.410 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-21T07:23:01.411 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-21T07:23:01.411 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-21T07:23:01.411 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-21T07:23:01.411 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-21T07:23:01.411 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-21T07:23:01.411 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-21T07:23:01.411 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-21T07:23:01.411 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-21T07:23:01.411 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-21T07:23:01.411 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-21T07:23:01.411 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-21T07:23:01.411 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-21T07:23:01.411 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-21T07:23:01.412 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-21T07:23:01.412 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-21T07:23:01.412 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-21T07:23:01.412 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-21T07:23:01.412 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-21T07:23:01.412 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-21T07:23:01.412 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-21T07:23:01.412 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-21T07:23:01.412 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-21T07:23:01.412 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-21T07:23:01.412 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-21T07:23:01.412 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-21T07:23:01.413 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-21T07:23:01.413 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-21T07:23:01.413 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-21T07:23:01.413 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-21T07:23:01.413 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-21T07:23:01.413 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-21T07:23:01.413 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-21T07:23:01.413 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-21T07:23:01.413 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-21T07:23:01.413 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-21T07:23:01.413 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-21T07:23:01.413 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-21T07:23:01.413 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-21T07:23:01.413 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-21T07:23:01.414 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-21T07:23:01.414 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-21T07:23:01.414 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-21T07:23:01.414 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-21T07:23:01.414 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-21T07:23:01.414 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-21T07:23:01.414 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-21T07:23:01.414 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-21T07:23:01.414 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-21T07:23:01.414 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-21T07:23:01.414 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-21T07:23:01.414 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-21T07:23:01.414 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-21T07:23:01.414 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-21T07:23:01.415 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-21T07:23:01.415 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-21T07:23:01.415 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-21T07:23:01.415 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-21T07:23:01.415 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-21T07:23:01.415 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-21T07:23:01.415 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-21T07:23:01.415 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-21T07:23:01.415 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-21T07:23:01.415 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-21T07:23:01.415 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-21T07:23:01.415 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-21T07:23:01.415 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-21T07:23:01.416 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-21T07:23:01.416 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-21T07:23:01.416 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-21T07:23:01.416 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-21T07:23:01.416 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-21T07:23:01.416 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-21T07:23:01.416 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-21T07:23:01.416 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-21T07:23:01.416 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-21T07:23:01.416 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-21T07:23:01.416 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-21T07:23:01.416 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-21T07:23:01.416 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-21T07:23:01.416 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-21T07:23:01.417 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-21T07:23:01.417 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-21T07:23:01.417 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-21T07:23:01.417 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-21T07:23:01.417 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-21T07:23:01.417 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-21T07:23:01.417 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-21T07:23:01.417 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-21T07:23:01.417 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-21T07:23:01.417 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-21T07:23:01.417 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-21T07:23:01.417 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-21T07:23:01.417 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-21T07:23:01.418 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-21T07:23:01.418 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-21T07:23:01.418 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-21T07:23:01.418 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-21T07:23:01.418 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-21T07:23:01.418 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-21T07:23:01.418 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-21T07:23:01.418 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-21T07:23:01.418 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-21T07:23:01.418 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-21T07:23:01.419 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-21T07:23:01.419 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-21T07:23:01.419 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-21T07:23:01.419 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-21T07:23:01.419 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-21T07:23:01.419 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-21T07:23:01.419 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-21T07:23:01.419 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-21T07:23:01.419 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-21T07:23:01.419 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-21T07:23:01.419 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-21T07:23:01.419 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-21T07:23:01.419 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-21T07:23:01.419 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-21T07:23:01.420 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-21T07:23:01.420 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-21T07:23:01.420 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-21T07:23:01.420 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-21T07:23:01.420 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-21T07:23:01.420 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-21T07:23:01.420 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-21T07:23:01.420 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-21T07:23:01.420 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-21T07:23:01.420 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-21T07:23:01.420 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-21T07:23:01.420 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-21T07:23:01.421 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-21T07:23:01.421 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-21T07:23:01.421 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-21T07:23:01.421 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-21T07:23:01.421 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-21T07:23:01.421 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-21T07:23:01.421 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-21T07:23:01.421 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-21T07:23:01.421 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-21T07:23:01.421 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-21T07:23:01.421 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-21T07:23:01.421 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-21T07:23:01.421 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-21T07:23:01.421 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-21T07:23:01.421 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-21T07:23:01.422 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-21T07:23:01.422 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-21T07:23:01.422 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-21T07:23:01.422 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-21T07:23:01.422 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-21T07:23:01.422 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-21T07:23:01.422 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-21T07:23:01.422 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-21T07:23:01.422 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-21T07:23:01.422 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-21T07:23:01.422 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-21T07:23:01.422 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-21T07:23:01.423 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-21T07:23:01.423 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-21T07:23:01.423 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-21T07:23:01.423 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-21T07:23:01.423 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-21T07:23:01.423 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-21T07:23:01.423 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-21T07:23:01.423 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-21T07:23:01.423 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-21T07:23:01.423 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-21T07:23:01.423 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-21T07:23:01.423 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-21T07:23:01.423 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-21T07:23:01.424 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-21T07:23:01.424 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-21T07:23:01.424 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-21T07:23:01.424 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-21T07:23:01.424 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-21T07:23:01.424 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-21T07:23:01.424 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-21T07:23:01.424 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-21T07:23:01.424 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-21T07:23:01.424 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-21T07:23:01.424 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-21T07:23:01.425 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-21T07:23:01.425 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-21T07:23:01.425 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-21T07:23:01.425 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-21T07:23:01.425 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-21T07:23:01.425 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-21T07:23:01.425 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-21T07:23:01.425 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-21T07:23:01.425 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-21T07:23:01.425 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-21T07:23:01.425 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-21T07:23:01.425 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-21T07:23:01.425 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-21T07:23:01.426 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-21T07:23:01.426 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-21T07:23:01.426 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-21T07:23:01.426 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-21T07:23:01.426 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-21T07:23:01.426 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-21T07:23:01.426 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-21T07:23:01.426 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-21T07:23:01.426 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-21T07:23:01.426 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-21T07:23:01.426 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-21T07:23:01.426 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-21T07:23:01.426 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-21T07:23:01.427 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-21T07:23:01.427 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-21T07:23:01.427 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-21T07:23:01.427 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-21T07:23:01.427 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-21T07:23:01.427 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-21T07:23:01.427 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-21T07:23:01.427 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-21T07:23:01.427 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-21T07:23:01.427 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-21T07:23:01.427 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-21T07:23:01.427 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-21T07:23:01.427 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-21T07:23:01.427 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-21T07:23:01.427 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-21T07:23:01.428 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-21T07:23:01.428 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-21T07:23:01.428 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-21T07:23:01.428 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-21T07:23:01.428 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-21T07:23:01.428 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-21T07:23:01.428 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-21T07:23:01.428 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-21T07:23:01.428 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-21T07:23:01.428 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-21T07:23:01.428 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-21T07:23:01.428 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-21T07:23:01.428 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-21T07:23:01.428 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-21T07:23:01.428 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-21T07:23:01.429 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-21T07:23:01.429 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-21T07:23:01.429 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-21T07:23:01.429 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-21T07:23:01.429 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-21T07:23:01.429 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-21T07:23:01.429 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-21T07:23:01.429 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-21T07:23:01.429 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-21T07:23:01.429 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-21T07:23:01.429 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-21T07:23:01.429 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-21T07:23:01.429 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-21T07:23:01.429 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-21T07:23:01.429 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-21T07:23:01.430 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-21T07:23:01.430 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-21T07:23:01.430 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-21T07:23:01.430 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-21T07:23:01.430 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-21T07:23:01.430 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-21T07:23:01.430 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-21T07:23:01.430 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-21T07:23:01.430 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-21T07:23:01.430 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-21T07:23:01.430 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-21T07:23:01.430 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-21T07:23:01.431 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-21T07:23:01.431 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-21T07:23:01.431 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-21T07:23:01.431 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-21T07:23:01.431 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-21T07:23:01.431 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-21T07:23:01.431 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-21T07:23:01.431 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-21T07:23:01.431 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-21T07:23:01.431 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-21T07:23:01.431 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-21T07:23:01.431 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-21T07:23:01.431 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-21T07:23:01.432 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-21T07:23:01.432 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-21T07:23:01.432 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-21T07:23:01.432 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-21T07:23:01.432 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-21T07:23:01.432 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-21T07:23:01.432 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-21T07:23:01.432 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-21T07:23:01.432 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-21T07:23:01.432 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-21T07:23:01.432 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-21T07:23:01.432 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-21T07:23:01.432 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-21T07:23:01.432 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-21T07:23:01.433 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-21T07:23:01.433 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-21T07:23:01.433 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-21T07:23:01.433 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-21T07:23:01.433 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-21T07:23:01.433 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-21T07:23:01.433 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-21T07:23:01.433 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-21T07:23:01.433 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-21T07:23:01.433 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-21T07:23:01.433 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-21T07:23:01.433 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-21T07:23:01.433 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-21T07:23:01.433 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-21T07:23:01.433 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-21T07:23:01.434 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-21T07:23:01.434 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-21T07:23:01.434 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-21T07:23:01.434 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-21T07:23:01.434 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-21T07:23:01.434 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-21T07:23:01.434 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-21T07:23:01.434 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-21T07:23:01.434 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-21T07:23:01.434 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-21T07:23:01.434 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-21T07:23:01.434 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-21T07:23:01.435 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-21T07:23:01.435 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-21T07:23:01.435 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-21T07:23:01.435 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-21T07:23:01.435 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-21T07:23:01.435 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-21T07:23:01.435 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-21T07:23:01.435 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-21T07:23:01.435 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-21T07:23:01.435 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-21T07:23:01.435 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-21T07:23:01.435 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-21T07:23:01.436 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-21T07:23:01.436 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-21T07:23:01.436 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-21T07:23:01.436 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-21T07:23:01.436 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-21T07:23:01.436 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-21T07:23:01.436 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-21T07:23:01.436 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-21T07:23:01.436 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-21T07:23:01.436 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-21T07:23:01.436 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-21T07:23:01.436 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-21T07:23:01.436 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-21T07:23:01.436 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-21T07:23:01.437 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-21T07:23:01.437 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-21T07:23:01.437 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-21T07:23:01.437 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-21T07:23:01.437 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-21T07:23:01.437 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-21T07:23:01.437 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-21T07:23:01.437 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-21T07:23:01.437 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-21T07:23:01.437 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-21T07:23:01.437 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-21T07:23:01.437 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-21T07:23:01.437 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-21T07:23:01.438 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-21T07:23:01.438 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-21T07:23:01.438 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-21T07:23:01.438 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-21T07:23:01.438 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-21T07:23:01.438 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-21T07:23:01.438 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-21T07:23:01.438 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-21T07:23:01.438 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-21T07:23:01.438 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-21T07:23:01.438 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-21T07:23:01.438 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-21T07:23:01.438 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-21T07:23:01.438 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-21T07:23:01.439 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-21T07:23:01.439 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-21T07:23:01.439 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-21T07:23:01.439 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-21T07:23:01.439 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-21T07:23:01.439 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-21T07:23:01.439 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-21T07:23:01.439 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-21T07:23:01.439 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-21T07:23:01.439 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-21T07:23:01.439 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-21T07:23:01.440 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-21T07:23:01.440 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-21T07:23:01.440 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-21T07:23:01.440 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-21T07:23:01.440 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-21T07:23:01.440 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-21T07:23:01.440 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-21T07:23:01.440 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-21T07:23:01.440 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-21T07:23:01.440 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-21T07:23:01.440 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-21T07:23:01.441 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-21T07:23:01.441 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-21T07:23:01.441 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-21T07:23:01.441 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-21T07:23:01.441 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-21T07:23:01.441 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-21T07:23:01.441 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-21T07:23:01.441 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-21T07:23:01.441 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-21T07:23:01.441 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-21T07:23:01.441 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-21T07:23:01.441 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-21T07:23:01.441 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-21T07:23:01.441 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-21T07:23:01.442 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-21T07:23:01.442 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-21T07:23:01.442 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-21T07:23:01.442 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-21T07:23:01.442 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-21T07:23:01.442 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-21T07:23:01.442 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-21T07:23:01.442 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-21T07:23:01.442 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-21T07:23:01.442 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-21T07:23:01.442 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-21T07:23:01.442 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-21T07:23:01.442 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-21T07:23:01.443 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-21T07:23:01.443 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-21T07:23:01.443 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-21T07:23:01.443 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-21T07:23:01.443 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-21T07:23:01.443 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-21T07:23:01.443 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-21T07:23:01.443 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-21T07:23:01.443 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-21T07:23:01.443 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-21T07:23:01.443 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-21T07:23:01.443 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-21T07:23:01.443 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-21T07:23:01.443 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-21T07:23:01.444 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-21T07:23:01.444 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-21T07:23:01.444 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-21T07:23:01.444 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-21T07:23:01.444 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-21T07:23:01.444 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-21T07:23:01.444 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-21T07:23:01.444 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-21T07:23:01.444 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-21T07:23:01.444 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-21T07:23:01.444 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-21T07:23:01.445 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-21T07:23:01.445 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-21T07:23:01.445 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-21T07:23:01.445 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-21T07:23:01.445 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-21T07:23:01.445 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-21T07:23:01.445 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-21T07:23:01.445 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-21T07:23:01.445 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-21T07:23:01.445 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-21T07:23:01.445 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-21T07:23:01.446 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-21T07:23:01.446 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-21T07:23:01.446 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-21T07:23:01.446 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-21T07:23:01.446 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-21T07:23:01.446 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-21T07:23:01.446 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-21T07:23:01.446 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-21T07:23:01.446 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-21T07:23:01.446 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-21T07:23:01.446 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-21T07:23:01.446 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-21T07:23:01.446 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-21T07:23:01.446 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-21T07:23:01.447 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-21T07:23:01.447 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-21T07:23:01.447 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-21T07:23:01.447 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-21T07:23:01.447 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-21T07:23:01.447 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-21T07:23:01.447 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-21T07:23:01.447 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-21T07:23:01.447 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-21T07:23:01.447 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-21T07:23:01.447 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-21T07:23:01.447 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-21T07:23:01.447 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-21T07:23:01.447 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-21T07:23:01.448 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-21T07:23:01.448 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-21T07:23:01.448 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-21T07:23:01.448 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-21T07:23:01.448 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-21T07:23:01.448 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-21T07:23:01.448 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-21T07:23:01.448 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-21T07:23:01.448 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-21T07:23:01.448 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-21T07:23:01.448 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-21T07:23:01.448 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-21T07:23:01.449 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-21T07:23:01.449 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-21T07:23:01.449 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-21T07:23:01.449 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-21T07:23:01.449 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-21T07:23:01.449 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-21T07:23:01.449 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-21T07:23:01.449 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-21T07:23:01.449 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-21T07:23:01.449 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-21T07:23:01.449 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-21T07:23:01.449 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-21T07:23:01.449 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-21T07:23:01.450 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-21T07:23:01.450 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-21T07:23:01.450 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-21T07:23:01.450 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-21T07:23:01.450 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-21T07:23:01.450 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-21T07:23:01.450 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-21T07:23:01.450 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-21T07:23:01.450 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-21T07:23:01.450 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-21T07:23:01.450 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-21T07:23:01.450 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-21T07:23:01.450 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-21T07:23:01.450 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-21T07:23:01.451 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-21T07:23:01.451 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-21T07:23:01.451 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-21T07:23:01.451 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-21T07:23:01.451 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-21T07:23:01.451 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-21T07:23:01.451 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-21T07:23:01.451 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-21T07:23:01.451 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-21T07:23:01.451 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-21T07:23:01.451 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-21T07:23:01.451 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-21T07:23:01.451 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-21T07:23:01.451 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-21T07:23:01.451 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-21T07:23:01.452 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-21T07:23:01.452 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-21T07:23:01.452 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-21T07:23:01.452 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-21T07:23:01.452 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-21T07:23:01.452 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-21T07:23:01.452 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-21T07:23:01.452 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-21T07:23:01.452 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-21T07:23:01.452 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-21T07:23:01.452 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-21T07:23:01.452 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-21T07:23:01.452 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-21T07:23:01.452 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-21T07:23:01.452 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-21T07:23:01.453 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-21T07:23:01.453 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-21T07:23:01.453 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-21T07:23:01.453 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-21T07:23:01.453 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-21T07:23:01.453 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-21T07:23:01.453 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-21T07:23:01.453 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-21T07:23:01.453 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-21T07:23:01.453 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-21T07:23:01.453 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-21T07:23:01.453 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-21T07:23:01.453 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-21T07:23:01.453 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-21T07:23:01.454 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-21T07:23:01.454 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-21T07:23:01.454 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-21T07:23:01.454 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-21T07:23:01.454 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-21T07:23:01.454 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-21T07:23:01.454 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-21T07:23:01.454 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-21T07:23:01.454 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-21T07:23:01.454 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-21T07:23:01.454 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-21T07:23:01.454 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-21T07:23:01.454 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-21T07:23:01.455 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-21T07:23:01.455 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-21T07:23:01.455 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-21T07:23:01.455 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-21T07:23:01.455 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-21T07:23:01.455 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-21T07:23:01.455 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-21T07:23:01.455 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-21T07:23:01.455 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-21T07:23:01.455 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-21T07:23:01.455 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-21T07:23:01.455 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-21T07:23:01.455 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-21T07:23:01.455 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-21T07:23:01.455 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-21T07:23:01.456 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-21T07:23:01.456 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-21T07:23:01.456 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-21T07:23:01.456 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-21T07:23:01.456 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-21T07:23:01.456 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-21T07:23:01.456 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-21T07:23:01.456 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-21T07:23:01.456 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-21T07:23:01.456 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-21T07:23:01.456 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-21T07:23:01.456 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-21T07:23:01.456 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-21T07:23:01.456 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-21T07:23:01.456 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-21T07:23:01.457 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-21T07:23:01.457 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-21T07:23:01.457 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-21T07:23:01.457 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-21T07:23:01.457 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-21T07:23:01.457 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-21T07:23:01.457 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-21T07:23:01.457 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-21T07:23:01.457 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-21T07:23:01.457 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-21T07:23:01.457 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-21T07:23:01.457 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-21T07:23:01.457 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-21T07:23:01.457 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-21T07:23:01.458 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-21T07:23:01.458 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-21T07:23:01.458 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-21T07:23:01.458 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-21T07:23:01.458 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-21T07:23:01.458 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-21T07:23:01.458 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-21T07:23:01.458 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-21T07:23:01.458 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-21T07:23:01.458 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-21T07:23:01.458 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-21T07:23:01.458 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-21T07:23:01.458 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-21T07:23:01.459 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-21T07:23:01.459 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-21T07:23:01.459 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-21T07:23:01.459 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-21T07:23:01.459 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-21T07:23:01.459 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-21T07:23:01.459 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-21T07:23:01.459 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-21T07:23:01.460 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-21T07:23:01.460 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-21T07:23:01.460 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-21T07:23:01.460 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-21T07:23:01.460 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-21T07:23:01.460 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-21T07:23:01.460 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-21T07:23:01.460 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-21T07:23:01.460 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-21T07:23:01.460 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-21T07:23:01.460 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-21T07:23:01.460 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-21T07:23:01.460 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-21T07:23:01.460 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-21T07:23:01.461 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-21T07:23:01.461 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-21T07:23:01.461 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-21T07:23:01.461 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-21T07:23:01.461 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-21T07:23:01.461 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-21T07:23:01.461 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-21T07:23:01.461 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-21T07:23:01.461 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-21T07:23:01.461 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-21T07:23:01.461 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-21T07:23:01.461 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-21T07:23:01.461 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-21T07:23:01.461 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-21T07:23:01.462 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-21T07:23:01.462 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-21T07:23:01.462 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-21T07:23:01.462 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-21T07:23:01.462 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-21T07:23:01.462 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-21T07:23:01.462 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-21T07:23:01.462 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-21T07:23:01.462 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-21T07:23:01.462 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-21T07:23:01.462 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-21T07:23:01.463 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-21T07:23:01.463 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-21T07:23:01.463 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-21T07:23:01.463 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-21T07:23:01.463 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-21T07:23:01.463 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-21T07:23:01.463 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-21T07:23:01.463 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-21T07:23:01.463 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-21T07:23:01.463 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-21T07:23:01.463 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-21T07:23:01.463 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-21T07:23:01.463 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-21T07:23:01.463 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-21T07:23:01.464 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-21T07:23:01.464 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-21T07:23:01.464 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-21T07:23:01.464 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-21T07:23:01.464 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-21T07:23:01.464 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-21T07:23:01.464 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-21T07:23:01.464 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-21T07:23:01.464 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-21T07:23:01.464 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-21T07:23:01.464 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-21T07:23:01.464 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-21T07:23:01.464 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-21T07:23:01.464 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-21T07:23:01.465 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-21T07:23:01.465 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-21T07:23:01.465 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-21T07:23:01.465 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-21T07:23:01.465 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-21T07:23:01.465 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-21T07:23:01.465 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-21T07:23:01.465 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-21T07:23:01.465 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-21T07:23:01.465 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-21T07:23:01.465 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-21T07:23:01.465 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-21T07:23:01.465 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-21T07:23:01.466 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-21T07:23:01.466 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-21T07:23:01.466 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-21T07:23:01.466 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-21T07:23:01.466 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-21T07:23:01.466 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-21T07:23:01.466 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-21T07:23:01.466 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-21T07:23:01.466 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-21T07:23:01.466 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-21T07:23:01.466 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-21T07:23:01.466 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-21T07:23:01.467 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-21T07:23:01.467 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-21T07:23:01.467 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-21T07:23:01.467 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-21T07:23:01.467 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-21T07:23:01.467 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-21T07:23:01.467 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-21T07:23:01.467 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-21T07:23:01.467 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-21T07:23:01.467 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-21T07:23:01.467 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-21T07:23:01.467 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-21T07:23:01.467 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-21T07:23:01.467 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-21T07:23:01.468 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-21T07:23:01.468 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-21T07:23:01.468 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-21T07:23:01.468 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-21T07:23:01.468 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-21T07:23:01.468 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-21T07:23:01.468 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-21T07:23:01.468 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-21T07:23:01.468 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-21T07:23:01.468 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-21T07:23:01.468 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-21T07:23:01.468 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-21T07:23:01.468 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-21T07:23:01.468 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-21T07:23:01.469 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-21T07:23:01.469 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-21T07:23:01.469 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-21T07:23:01.469 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-21T07:23:01.469 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-21T07:23:01.469 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-21T07:23:01.469 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-21T07:23:01.469 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-21T07:23:01.469 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-21T07:23:01.469 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-21T07:23:01.469 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-21T07:23:01.469 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-21T07:23:01.469 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-21T07:23:01.469 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-21T07:23:01.470 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-21T07:23:01.470 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-21T07:23:01.470 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-21T07:23:01.470 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-21T07:23:01.470 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-21T07:23:01.470 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-21T07:23:01.470 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-21T07:23:01.470 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-21T07:23:01.470 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-21T07:23:01.471 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-21T07:23:01.471 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-21T07:23:01.471 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-21T07:23:01.471 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-21T07:23:01.471 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-21T07:23:01.471 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-21T07:23:01.471 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-21T07:23:01.471 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-21T07:23:01.471 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-21T07:23:01.471 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-21T07:23:01.471 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-21T07:23:01.472 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-21T07:23:01.472 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-21T07:23:01.472 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-21T07:23:01.472 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-21T07:23:01.472 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-21T07:23:01.472 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-21T07:23:01.472 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-21T07:23:01.472 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-21T07:23:01.472 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-21T07:23:01.472 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-21T07:23:01.472 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-21T07:23:01.472 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-21T07:23:01.472 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-21T07:23:01.473 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-21T07:23:01.473 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-21T07:23:01.473 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-21T07:23:01.473 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-21T07:23:01.473 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-21T07:23:01.473 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-21T07:23:01.474 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-21T07:23:01.474 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-21T07:23:01.474 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-21T07:23:01.474 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-21T07:23:01.474 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-21T07:23:01.474 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-21T07:23:01.474 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-21T07:23:01.475 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-21T07:23:01.475 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-21T07:23:01.475 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-21T07:23:01.475 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-21T07:23:01.475 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-21T07:23:01.475 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-21T07:23:01.475 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-21T07:23:01.475 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-21T07:23:01.475 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-21T07:23:01.475 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-21T07:23:01.476 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-21T07:23:01.476 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-21T07:23:01.476 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-21T07:23:01.476 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-21T07:23:01.476 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-21T07:23:01.476 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-21T07:23:01.476 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-21T07:23:01.476 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-21T07:23:01.476 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-21T07:23:01.476 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-21T07:23:01.476 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-21T07:23:01.476 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-21T07:23:01.476 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-21T07:23:01.476 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-21T07:23:01.476 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-21T07:23:01.477 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-21T07:23:01.477 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-21T07:23:01.477 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-21T07:23:01.477 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-21T07:23:01.477 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-21T07:23:01.477 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-21T07:23:01.477 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-21T07:23:01.477 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-21T07:23:01.477 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-21T07:23:01.477 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-21T07:23:01.477 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-21T07:23:01.477 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-21T07:23:01.477 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-21T07:23:01.477 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-21T07:23:01.478 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-21T07:23:01.478 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-21T07:23:01.478 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-21T07:23:01.478 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-21T07:23:01.478 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-21T07:23:01.478 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-21T07:23:01.478 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-21T07:23:01.478 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-21T07:23:01.478 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-21T07:23:01.478 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-21T07:23:01.478 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-21T07:23:01.478 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-21T07:23:01.478 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-21T07:23:01.478 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-21T07:23:01.479 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-21T07:23:01.479 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-21T07:23:01.479 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-21T07:23:01.479 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-21T07:23:01.479 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-21T07:23:01.479 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-21T07:23:01.479 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-21T07:23:01.479 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-21T07:23:01.479 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-21T07:23:01.480 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-21T07:23:01.480 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-21T07:23:01.480 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-21T07:23:01.480 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-21T07:23:01.480 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-21T07:23:01.480 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-21T07:23:01.480 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-21T07:23:01.481 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-21T07:23:01.481 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-21T07:23:01.481 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-21T07:23:01.481 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-21T07:23:01.481 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-21T07:23:01.481 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-21T07:23:01.481 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-21T07:23:01.481 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-21T07:23:01.481 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-21T07:23:01.482 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-21T07:23:01.482 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-21T07:23:01.482 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-21T07:23:01.482 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-21T07:23:01.482 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-21T07:23:01.482 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-21T07:23:01.482 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-21T07:23:01.482 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-21T07:23:01.482 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-21T07:23:01.482 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-21T07:23:01.482 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-21T07:23:01.482 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-21T07:23:01.482 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-21T07:23:01.482 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-21T07:23:01.483 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-21T07:23:01.483 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-21T07:23:01.483 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-21T07:23:01.483 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-21T07:23:01.483 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-21T07:23:01.483 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-21T07:23:01.483 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-21T07:23:01.483 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-21T07:23:01.483 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-21T07:23:01.483 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-21T07:23:01.483 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-21T07:23:01.484 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-21T07:23:01.484 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-21T07:23:01.484 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-21T07:23:01.484 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-21T07:23:01.484 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-21T07:23:01.484 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-21T07:23:01.484 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-21T07:23:01.484 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-21T07:23:01.484 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-21T07:23:01.484 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-21T07:23:01.484 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-21T07:23:01.484 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-21T07:23:01.484 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-21T07:23:01.485 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-21T07:23:01.485 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-21T07:23:01.485 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-21T07:23:01.485 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-21T07:23:01.485 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-21T07:23:01.485 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-21T07:23:01.485 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-21T07:23:01.485 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-21T07:23:01.485 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-21T07:23:01.485 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-21T07:23:01.485 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-21T07:23:01.486 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-21T07:23:01.486 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-21T07:23:01.486 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-21T07:23:01.486 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-21T07:23:01.486 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-21T07:23:01.486 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-21T07:23:01.486 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-21T07:23:01.486 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-21T07:23:01.486 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-21T07:23:01.486 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-21T07:23:01.486 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-21T07:23:01.487 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-21T07:23:01.487 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-21T07:23:01.487 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-21T07:23:01.487 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-21T07:23:01.487 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-21T07:23:01.487 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-21T07:23:01.487 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-21T07:23:01.487 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-21T07:23:01.487 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-21T07:23:01.487 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-21T07:23:01.487 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-21T07:23:01.487 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-21T07:23:01.488 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-21T07:23:01.488 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-21T07:23:01.488 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-21T07:23:01.488 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-21T07:23:01.488 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-21T07:23:01.488 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-21T07:23:01.489 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-21T07:23:01.489 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-21T07:23:01.489 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-21T07:23:01.489 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-21T07:23:01.489 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-21T07:23:01.489 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-21T07:23:01.490 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-21T07:23:01.490 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-21T07:23:01.490 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-21T07:23:01.490 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-21T07:23:01.490 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-21T07:23:01.490 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-21T07:23:01.490 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-21T07:23:01.490 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-21T07:23:01.490 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-21T07:23:01.491 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-21T07:23:01.491 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-21T07:23:01.491 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-21T07:23:01.491 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-21T07:23:01.491 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-21T07:23:01.491 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-21T07:23:01.491 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-21T07:23:01.491 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-21T07:23:01.491 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-21T07:23:01.492 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-21T07:23:01.492 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-21T07:23:01.492 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-21T07:23:01.492 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-21T07:23:01.492 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-21T07:23:01.492 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-21T07:23:01.492 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-21T07:23:01.492 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-21T07:23:01.492 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-21T07:23:01.492 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-21T07:23:01.492 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-21T07:23:01.492 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-21T07:23:01.492 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-21T07:23:01.492 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-21T07:23:01.492 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-21T07:23:01.493 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-21T07:23:01.493 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-21T07:23:01.493 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-21T07:23:01.493 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-21T07:23:01.493 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-21T07:23:01.493 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-21T07:23:01.493 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-21T07:23:01.493 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-21T07:23:01.493 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-21T07:23:01.493 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-21T07:23:01.493 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-21T07:23:01.493 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-21T07:23:01.493 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-21T07:23:01.493 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-21T07:23:01.494 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-21T07:23:01.494 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-21T07:23:01.494 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-21T07:23:01.494 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-21T07:23:01.494 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-21T07:23:01.494 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-21T07:23:01.494 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-21T07:23:01.494 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-21T07:23:01.494 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-21T07:23:01.494 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-21T07:23:01.494 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-21T07:23:01.494 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-21T07:23:01.494 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-21T07:23:01.495 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-21T07:23:01.495 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-21T07:23:01.495 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-21T07:23:01.495 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-21T07:23:01.495 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-21T07:23:01.495 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-21T07:23:01.495 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-21T07:23:01.495 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-21T07:23:01.495 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-21T07:23:01.495 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-21T07:23:01.495 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-21T07:23:01.495 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-21T07:23:01.495 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-21T07:23:01.495 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-21T07:23:01.496 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-21T07:23:01.496 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-21T07:23:01.496 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-21T07:23:01.496 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-21T07:23:01.496 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-21T07:23:01.496 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-21T07:23:01.496 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-21T07:23:01.496 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-21T07:23:01.496 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-21T07:23:01.497 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-21T07:23:01.497 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-21T07:23:01.497 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-21T07:23:01.497 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-21T07:23:01.497 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-21T07:23:01.497 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-21T07:23:01.498 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-21T07:23:01.498 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-21T07:23:01.498 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-21T07:23:01.498 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-21T07:23:01.498 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-21T07:23:01.498 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-21T07:23:01.498 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-21T07:23:01.499 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-21T07:23:01.499 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-21T07:23:01.499 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-21T07:23:01.499 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-21T07:23:01.499 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-21T07:23:01.499 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-21T07:23:01.499 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-21T07:23:01.499 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-21T07:23:01.499 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-21T07:23:01.499 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-21T07:23:01.499 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-21T07:23:01.499 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-21T07:23:01.499 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-21T07:23:01.499 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-21T07:23:01.500 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-21T07:23:01.500 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-21T07:23:01.500 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-21T07:23:01.500 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-21T07:23:01.500 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-21T07:23:01.500 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-21T07:23:01.500 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-21T07:23:01.500 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-21T07:23:01.500 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-21T07:23:01.500 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-21T07:23:01.500 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-21T07:23:01.500 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-21T07:23:01.500 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-21T07:23:01.500 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-21T07:23:01.501 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-21T07:23:01.501 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-21T07:23:01.501 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-21T07:23:01.501 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-21T07:23:01.501 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-21T07:23:01.501 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-21T07:23:01.501 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-21T07:23:01.501 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-21T07:23:01.501 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-21T07:23:01.501 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-21T07:23:01.501 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-21T07:23:01.502 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-21T07:23:01.502 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-21T07:23:01.502 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-21T07:23:01.502 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-21T07:23:01.502 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-21T07:23:01.502 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-21T07:23:01.502 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-21T07:23:01.502 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-21T07:23:01.502 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-21T07:23:01.502 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-21T07:23:01.502 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-21T07:23:01.502 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-21T07:23:01.502 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-21T07:23:01.503 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-21T07:23:01.503 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-21T07:23:01.503 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-21T07:23:01.503 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-21T07:23:01.503 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-21T07:23:01.503 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-21T07:23:01.503 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-21T07:23:01.503 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-21T07:23:01.503 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-21T07:23:01.503 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-21T07:23:01.503 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-21T07:23:01.503 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-21T07:23:01.503 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-21T07:23:01.504 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-21T07:23:01.504 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-21T07:23:01.504 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-21T07:23:01.504 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-21T07:23:01.504 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-21T07:23:01.504 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-21T07:23:01.504 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-21T07:23:01.504 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-21T07:23:01.504 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-21T07:23:01.504 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-21T07:23:01.504 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-21T07:23:01.504 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-21T07:23:01.504 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-21T07:23:01.504 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-21T07:23:01.504 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-21T07:23:01.505 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-21T07:23:01.505 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-21T07:23:01.505 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-21T07:23:01.505 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-21T07:23:01.505 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-21T07:23:01.505 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-21T07:23:01.505 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-21T07:23:01.505 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-21T07:23:01.505 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-21T07:23:01.505 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-21T07:23:01.505 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-21T07:23:01.505 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-21T07:23:01.505 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-21T07:23:01.505 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-21T07:23:01.505 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-21T07:23:01.506 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-21T07:23:01.506 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-21T07:23:01.506 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-21T07:23:01.506 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-21T07:23:01.506 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-21T07:23:01.506 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-21T07:23:01.506 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-21T07:23:01.506 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-21T07:23:01.506 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-21T07:23:01.506 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-21T07:23:01.506 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-21T07:23:01.506 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-21T07:23:01.506 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-21T07:23:01.506 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-21T07:23:01.507 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-21T07:23:01.507 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-21T07:23:01.507 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-21T07:23:01.507 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-21T07:23:01.507 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-21T07:23:01.507 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-21T07:23:01.507 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-21T07:23:01.507 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-21T07:23:01.507 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-21T07:23:01.507 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-21T07:23:01.507 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-21T07:23:01.507 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-21T07:23:01.507 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-21T07:23:01.507 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-21T07:23:01.508 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-21T07:23:01.508 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-21T07:23:01.508 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-21T07:23:01.508 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-21T07:23:01.508 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-21T07:23:01.508 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-21T07:23:01.508 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-21T07:23:01.508 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-21T07:23:01.508 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-21T07:23:01.508 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-21T07:23:01.508 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-21T07:23:01.508 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-21T07:23:01.508 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-21T07:23:01.509 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-21T07:23:01.509 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-21T07:23:01.509 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-21T07:23:01.509 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-21T07:23:01.509 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-21T07:23:01.509 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-21T07:23:01.509 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-21T07:23:01.509 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-21T07:23:01.509 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-21T07:23:01.509 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-21T07:23:01.509 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-21T07:23:01.509 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-21T07:23:01.509 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-21T07:23:01.509 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-21T07:23:01.510 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-21T07:23:01.510 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-21T07:23:01.510 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-21T07:23:01.510 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-21T07:23:01.510 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-21T07:23:01.510 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-21T07:23:01.510 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-21T07:23:01.510 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-21T07:23:01.510 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-21T07:23:01.510 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-21T07:23:01.510 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-21T07:23:01.510 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-21T07:23:01.510 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-21T07:23:01.511 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-21T07:23:01.511 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-21T07:23:01.511 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-21T07:23:01.511 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-21T07:23:01.511 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-21T07:23:01.511 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-21T07:23:01.511 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-21T07:23:01.511 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-21T07:23:01.511 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-21T07:23:01.511 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-21T07:23:01.511 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-21T07:23:01.512 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-21T07:23:01.512 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-21T07:23:01.512 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-21T07:23:01.512 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-21T07:23:01.512 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-21T07:23:01.512 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-21T07:23:01.512 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-21T07:23:01.512 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-21T07:23:01.512 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-21T07:23:01.512 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-21T07:23:01.513 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-21T07:23:01.513 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-21T07:23:01.513 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-21T07:23:01.513 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-21T07:23:01.513 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-21T07:23:01.513 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-21T07:23:01.513 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-21T07:23:01.513 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-21T07:23:01.513 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-21T07:23:01.513 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-21T07:23:01.513 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-21T07:23:01.513 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-21T07:23:01.514 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-21T07:23:01.514 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-21T07:23:01.514 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-21T07:23:01.514 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-21T07:23:01.514 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-21T07:23:01.514 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-21T07:23:01.514 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-21T07:23:01.514 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-21T07:23:01.514 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-21T07:23:01.514 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-21T07:23:01.514 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-21T07:23:01.515 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-21T07:23:01.515 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-21T07:23:01.515 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-21T07:23:01.515 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-21T07:23:01.515 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-21T07:23:01.515 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-21T07:23:01.515 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-21T07:23:01.515 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-21T07:23:01.515 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-21T07:23:01.515 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-21T07:23:01.516 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-21T07:23:01.516 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-21T07:23:01.516 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-21T07:23:01.516 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-21T07:23:01.516 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-21T07:23:01.516 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-21T07:23:01.517 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-21T07:23:01.517 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-21T07:23:01.517 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-21T07:23:01.517 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-21T07:23:01.517 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-21T07:23:01.517 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-21T07:23:01.517 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-21T07:23:01.517 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-21T07:23:01.517 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-21T07:23:01.517 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-21T07:23:01.517 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-21T07:23:01.517 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-21T07:23:01.517 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-21T07:23:01.517 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-21T07:23:01.517 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-21T07:23:01.518 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-21T07:23:01.518 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-21T07:23:01.518 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-21T07:23:01.518 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-21T07:23:01.518 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-21T07:23:01.518 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-21T07:23:01.518 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-21T07:23:01.518 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-21T07:23:01.518 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-21T07:23:01.518 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-21T07:23:01.518 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-21T07:23:01.518 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-21T07:23:01.519 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-21T07:23:01.519 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-21T07:23:01.519 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-21T07:23:01.519 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-21T07:23:01.519 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-21T07:23:01.519 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-21T07:23:01.519 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-21T07:23:01.519 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-21T07:23:01.519 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-21T07:23:01.519 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-21T07:23:01.519 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-21T07:23:01.519 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-21T07:23:01.519 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-21T07:23:01.520 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-21T07:23:01.520 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-21T07:23:01.520 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-21T07:23:01.520 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-21T07:23:01.520 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-21T07:23:01.520 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-21T07:23:01.520 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-21T07:23:01.520 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-21T07:23:01.520 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-21T07:23:01.520 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-21T07:23:01.520 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-21T07:23:01.520 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-21T07:23:01.520 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-21T07:23:01.521 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-21T07:23:01.521 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-21T07:23:01.521 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-21T07:23:01.521 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-21T07:23:01.521 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-21T07:23:01.521 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-21T07:23:01.521 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-21T07:23:01.521 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-21T07:23:01.521 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-21T07:23:01.521 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-21T07:23:01.521 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-21T07:23:01.521 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-21T07:23:01.521 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-21T07:23:01.521 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-21T07:23:01.522 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-21T07:23:01.522 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-21T07:23:01.522 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-21T07:23:01.522 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-21T07:23:01.522 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-21T07:23:01.522 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-21T07:23:01.522 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-21T07:23:01.522 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-21T07:23:01.522 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-21T07:23:01.522 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-21T07:23:01.522 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-21T07:23:01.523 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-21T07:23:01.523 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-21T07:23:01.523 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-21T07:23:01.523 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-21T07:23:01.523 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-21T07:23:01.523 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-21T07:23:01.523 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-21T07:23:01.523 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-21T07:23:01.523 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-21T07:23:01.523 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-21T07:23:01.523 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-21T07:23:01.523 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-21T07:23:01.523 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-21T07:23:01.524 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-21T07:23:01.524 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-21T07:23:01.524 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-21T07:23:01.524 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-21T07:23:01.524 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-21T07:23:01.524 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-21T07:23:01.524 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-21T07:23:01.524 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-21T07:23:01.524 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-21T07:23:01.524 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-21T07:23:01.524 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-21T07:23:01.524 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-21T07:23:01.524 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-21T07:23:01.524 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-21T07:23:01.525 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-21T07:23:01.525 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-21T07:23:01.525 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-21T07:23:01.525 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-21T07:23:01.525 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-21T07:23:01.525 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-21T07:23:01.525 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-21T07:23:01.525 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-21T07:23:01.525 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-21T07:23:01.525 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-21T07:23:01.525 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-21T07:23:01.525 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-21T07:23:01.525 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-21T07:23:01.526 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-21T07:23:01.526 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-21T07:23:01.526 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-21T07:23:01.526 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-21T07:23:01.526 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-21T07:23:01.526 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-21T07:23:01.526 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-21T07:23:01.526 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-21T07:23:01.526 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-21T07:23:01.526 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-21T07:23:01.526 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-21T07:23:01.526 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-21T07:23:01.527 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-21T07:23:01.527 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-21T07:23:01.527 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-21T07:23:01.527 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-21T07:23:01.527 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-21T07:23:01.527 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-21T07:23:01.527 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-21T07:23:01.527 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-21T07:23:01.527 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-21T07:23:01.527 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-21T07:23:01.527 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-21T07:23:01.527 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-21T07:23:01.528 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-21T07:23:01.528 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-21T07:23:01.528 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-21T07:23:01.528 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-21T07:23:01.528 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-21T07:23:01.528 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-21T07:23:01.528 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-21T07:23:01.528 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-21T07:23:01.528 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-21T07:23:01.528 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-21T07:23:01.528 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-21T07:23:01.528 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-21T07:23:01.528 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-21T07:23:01.528 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-21T07:23:01.528 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-21T07:23:01.529 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-21T07:23:01.529 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-21T07:23:01.529 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-21T07:23:01.529 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-21T07:23:01.529 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-21T07:23:01.529 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-21T07:23:01.529 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-21T07:23:01.529 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-21T07:23:01.529 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-21T07:23:01.529 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-21T07:23:01.529 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-21T07:23:01.529 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-21T07:23:01.529 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-21T07:23:01.529 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-21T07:23:01.529 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-21T07:23:01.530 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-21T07:23:01.530 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-21T07:23:01.530 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-21T07:23:01.530 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-21T07:23:01.530 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-21T07:23:01.530 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-21T07:23:01.530 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-21T07:23:01.530 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-21T07:23:01.530 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-21T07:23:01.530 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-21T07:23:01.530 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-21T07:23:01.530 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-21T07:23:01.530 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-21T07:23:01.530 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-21T07:23:01.531 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-21T07:23:01.531 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-21T07:23:01.531 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-21T07:23:01.531 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-21T07:23:01.531 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-21T07:23:01.531 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-21T07:23:01.531 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-21T07:23:01.531 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-21T07:23:01.531 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-21T07:23:01.531 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-21T07:23:01.531 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-21T07:23:01.531 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-21T07:23:01.531 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-21T07:23:01.531 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-21T07:23:01.532 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-21T07:23:01.532 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-21T07:23:01.532 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-21T07:23:01.532 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-21T07:23:01.532 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-21T07:23:01.532 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-21T07:23:01.532 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-21T07:23:01.532 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-21T07:23:01.532 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-21T07:23:01.532 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-21T07:23:01.532 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-21T07:23:01.532 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-21T07:23:01.532 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-21T07:23:01.532 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-21T07:23:01.533 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-21T07:23:01.533 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-21T07:23:01.533 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-21T07:23:01.533 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-21T07:23:01.533 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-21T07:23:01.533 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-21T07:23:01.533 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-21T07:23:01.533 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-21T07:23:01.533 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-21T07:23:01.534 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-21T07:23:01.534 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-21T07:23:01.534 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-21T07:23:01.534 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-21T07:23:01.534 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-21T07:23:01.534 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-21T07:23:01.534 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-21T07:23:01.534 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-21T07:23:01.534 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-21T07:23:01.534 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-21T07:23:01.534 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-21T07:23:01.534 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-21T07:23:01.534 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-21T07:23:01.535 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-21T07:23:01.535 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-21T07:23:01.535 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-21T07:23:01.535 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-21T07:23:01.535 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-21T07:23:01.535 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-21T07:23:01.535 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-21T07:23:01.535 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-21T07:23:01.535 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-21T07:23:01.535 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-21T07:23:01.535 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-21T07:23:01.535 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-21T07:23:01.536 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-21T07:23:01.536 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-21T07:23:01.536 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-21T07:23:01.536 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-21T07:23:01.536 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-21T07:23:01.536 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-21T07:23:01.536 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-21T07:23:01.536 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-21T07:23:01.540 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:23:01.541 DEBUG:teuthology.orchestra.run.smithi039:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-21T07:23:01.565 INFO:teuthology.orchestra.run.smithi039.stdout:postinstall scriptlet (using /bin/sh): 2024-09-21T07:23:01.565 INFO:teuthology.orchestra.run.smithi039.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-21T07:23:01.565 INFO:teuthology.orchestra.run.smithi039.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-21T07:23:01.565 INFO:teuthology.orchestra.run.smithi039.stdout:fi 2024-09-21T07:23:01.565 INFO:teuthology.orchestra.run.smithi039.stdout:for file in vmlinuz System.map config; do 2024-09-21T07:23:01.565 INFO:teuthology.orchestra.run.smithi039.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-21T07:23:01.565 INFO:teuthology.orchestra.run.smithi039.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-21T07:23:01.565 INFO:teuthology.orchestra.run.smithi039.stdout: fi 2024-09-21T07:23:01.565 INFO:teuthology.orchestra.run.smithi039.stdout:done 2024-09-21T07:23:01.565 INFO:teuthology.orchestra.run.smithi039.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-21T07:23:01.566 INFO:teuthology.orchestra.run.smithi039.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-21T07:23:01.566 INFO:teuthology.orchestra.run.smithi039.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-21T07:23:01.566 INFO:teuthology.orchestra.run.smithi039.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-21T07:23:01.566 INFO:teuthology.orchestra.run.smithi039.stdout:fi 2024-09-21T07:23:01.566 INFO:teuthology.orchestra.run.smithi039.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-21T07:23:01.566 INFO:teuthology.orchestra.run.smithi039.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:23:01.566 INFO:teuthology.orchestra.run.smithi039.stdout:fi 2024-09-21T07:23:01.566 INFO:teuthology.orchestra.run.smithi039.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-21T07:23:01.566 INFO:teuthology.orchestra.run.smithi039.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-21T07:23:01.566 INFO:teuthology.orchestra.run.smithi039.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:23:01.566 INFO:teuthology.orchestra.run.smithi039.stdout:fi 2024-09-21T07:23:01.567 DEBUG:teuthology.orchestra.run.smithi039:> sudo rpm -qi grub2-tools 2024-09-21T07:23:01.640 INFO:teuthology.orchestra.run.smithi039.stdout:Name : grub2-tools 2024-09-21T07:23:01.640 INFO:teuthology.orchestra.run.smithi039.stdout:Epoch : 1 2024-09-21T07:23:01.640 INFO:teuthology.orchestra.run.smithi039.stdout:Version : 2.06 2024-09-21T07:23:01.640 INFO:teuthology.orchestra.run.smithi039.stdout:Release : 82.el9 2024-09-21T07:23:01.640 INFO:teuthology.orchestra.run.smithi039.stdout:Architecture: x86_64 2024-09-21T07:23:01.640 INFO:teuthology.orchestra.run.smithi039.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T07:23:01.640 INFO:teuthology.orchestra.run.smithi039.stdout:Group : Unspecified 2024-09-21T07:23:01.641 INFO:teuthology.orchestra.run.smithi039.stdout:Size : 8274670 2024-09-21T07:23:01.641 INFO:teuthology.orchestra.run.smithi039.stdout:License : GPLv3+ 2024-09-21T07:23:01.641 INFO:teuthology.orchestra.run.smithi039.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T07:23:01.641 INFO:teuthology.orchestra.run.smithi039.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T07:23:01.641 INFO:teuthology.orchestra.run.smithi039.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T07:23:01.641 INFO:teuthology.orchestra.run.smithi039.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T07:23:01.641 INFO:teuthology.orchestra.run.smithi039.stdout:Packager : builder@centos.org 2024-09-21T07:23:01.641 INFO:teuthology.orchestra.run.smithi039.stdout:Vendor : CentOS 2024-09-21T07:23:01.641 INFO:teuthology.orchestra.run.smithi039.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T07:23:01.641 INFO:teuthology.orchestra.run.smithi039.stdout:Summary : Support tools for GRUB. 2024-09-21T07:23:01.641 INFO:teuthology.orchestra.run.smithi039.stdout:Description : 2024-09-21T07:23:01.641 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:23:01.641 INFO:teuthology.orchestra.run.smithi039.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T07:23:01.641 INFO:teuthology.orchestra.run.smithi039.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T07:23:01.642 INFO:teuthology.orchestra.run.smithi039.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T07:23:01.642 INFO:teuthology.orchestra.run.smithi039.stdout:hardware devices. 2024-09-21T07:23:01.642 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:23:01.642 INFO:teuthology.orchestra.run.smithi039.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T07:23:01.643 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T07:23:01.643 INFO:teuthology.task.kernel:Updating grub on smithi039 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:23:01.643 DEBUG:teuthology.orchestra.run.smithi039:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T07:23:02.314 INFO:teuthology.orchestra.run.smithi039.stderr:Generating grub configuration file ... 2024-09-21T07:23:03.640 INFO:teuthology.orchestra.run.smithi039.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T07:23:03.669 INFO:teuthology.orchestra.run.smithi039.stderr:done 2024-09-21T07:23:03.672 DEBUG:teuthology.orchestra.run.smithi039:> mktemp 2024-09-21T07:23:03.687 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/tmp.oEJGAWh2lF 2024-09-21T07:23:03.688 DEBUG:teuthology.orchestra.run.smithi039:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.oEJGAWh2lF 2024-09-21T07:23:03.754 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0666 /tmp/tmp.oEJGAWh2lF 2024-09-21T07:23:03.913 DEBUG:teuthology.orchestra.remote:smithi039:/tmp/tmp.oEJGAWh2lF is 6KB 2024-09-21T07:23:03.966 DEBUG:teuthology.orchestra.run.smithi039:> rm -fr /tmp/tmp.oEJGAWh2lF 2024-09-21T07:23:03.982 DEBUG:teuthology.orchestra.run.smithi039:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T07:23:04.045 INFO:teuthology.orchestra.run.smithi039.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T07:23:04.045 INFO:teuthology.orchestra.run.smithi039.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T07:23:04.046 INFO:teuthology.orchestra.run.smithi039.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T07:23:04.046 INFO:teuthology.orchestra.run.smithi039.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-21T07:23:04.047 DEBUG:teuthology.orchestra.run.smithi039:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:23:04.185 DEBUG:teuthology.orchestra.run.smithi039:> sudo shutdown -r now 2024-09-21T07:23:27.564 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-09-21T07:23:27.564 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-09-21T07:23:27.565 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T07:23:34.194 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-21T07:23:34.195 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-09-21T07:23:34.195 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T07:23:46.031 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.148 2024-09-21T07:23:55.038 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-09-21T07:23:55.039 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T07:23:58.099 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.148 2024-09-21T07:24:08.819 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-09-21T07:24:10.100 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-09-21T07:24:10.101 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T07:24:17.826 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-09-21T07:24:17.827 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T07:24:20.883 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-09-21T07:24:28.526 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.148 2024-09-21T07:24:32.886 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-09-21T07:24:32.887 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T07:24:35.951 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-09-21T07:24:43.534 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-09-21T07:24:43.535 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T07:24:45.869 DEBUG:teuthology.orchestra.run.smithi148:> true 2024-09-21T07:24:46.522 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-09-21T07:24:46.522 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-21T07:24:46.523 DEBUG:teuthology.orchestra.run.smithi148:> uname -r 2024-09-21T07:24:46.556 INFO:teuthology.orchestra.run.smithi148.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:24:46.557 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:24:46.557 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T07:24:46.557 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T07:24:47.558 DEBUG:teuthology.parallel:result is None 2024-09-21T07:24:50.951 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-09-21T07:24:50.952 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T07:25:22.891 DEBUG:teuthology.orchestra.run.smithi039:> true 2024-09-21T07:25:23.404 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-09-21T07:25:23.405 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-21T07:25:23.405 DEBUG:teuthology.orchestra.run.smithi039:> uname -r 2024-09-21T07:25:23.440 INFO:teuthology.orchestra.run.smithi039.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:25:23.440 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:25:23.440 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T07:25:23.440 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T07:25:24.443 DEBUG:teuthology.parallel:result is None 2024-09-21T07:25:24.443 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T07:25:24.453 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T07:25:24.453 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T07:25:24.457 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T07:25:24.499 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T07:25:24.505 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T07:25:24.521 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T07:25:24.522 DEBUG:teuthology.orchestra.run.smithi039:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T07:25:24.529 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T07:25:24.595 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T07:25:24.602 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T07:25:24.602 DEBUG:teuthology.orchestra.run.smithi039:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T07:25:24.606 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T07:25:24.729 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T07:25:24.745 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T07:25:24.766 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T07:25:24.909 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T07:25:24.913 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T07:25:24.920 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T07:25:24.920 DEBUG:teuthology.orchestra.run.smithi039:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T07:25:24.924 DEBUG:teuthology.orchestra.run.smithi148:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T07:25:24.980 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T07:25:24.991 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T07:25:24.992 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T07:25:24.995 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T07:25:25.029 DEBUG:teuthology.orchestra.run.smithi039:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T07:25:25.129 DEBUG:teuthology.orchestra.run.smithi039:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T07:25:25.203 DEBUG:teuthology.orchestra.run.smithi039:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T07:25:25.266 DEBUG:teuthology.orchestra.run.smithi039:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T07:25:25.335 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:25:25.335 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T07:25:25.413 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T07:25:25.471 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T07:25:25.557 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T07:25:25.621 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T07:25:25.690 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:25:25.690 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T07:25:25.744 DEBUG:teuthology.orchestra.run.smithi039:> sudo service rsyslog restart 2024-09-21T07:25:25.748 DEBUG:teuthology.orchestra.run.smithi148:> sudo service rsyslog restart 2024-09-21T07:25:25.834 INFO:teuthology.orchestra.run.smithi039.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T07:25:25.865 INFO:teuthology.orchestra.run.smithi148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T07:25:26.144 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T07:25:26.151 INFO:teuthology.task.internal:Starting timer... 2024-09-21T07:25:26.152 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T07:25:26.162 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T07:25:26.170 DEBUG:teuthology.orchestra.run.smithi039:> sudo service auditd rotate 2024-09-21T07:25:26.267 INFO:teuthology.orchestra.run.smithi039.stdout:Rotating logs: 2024-09-21T07:25:26.270 DEBUG:teuthology.orchestra.run.smithi148:> sudo service auditd rotate 2024-09-21T07:25:26.357 INFO:teuthology.orchestra.run.smithi148.stdout:Rotating logs: 2024-09-21T07:25:26.360 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T07:25:26.360 DEBUG:teuthology.orchestra.run.smithi039:> /usr/sbin/getenforce 2024-09-21T07:25:26.402 INFO:teuthology.orchestra.run.smithi039.stdout:Permissive 2024-09-21T07:25:26.402 DEBUG:teuthology.orchestra.run.smithi148:> /usr/sbin/getenforce 2024-09-21T07:25:26.446 INFO:teuthology.orchestra.run.smithi148.stdout:Permissive 2024-09-21T07:25:26.446 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi039.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi148.front.sepia.ceph.com': 'permissive'} 2024-09-21T07:25:26.446 DEBUG:teuthology.orchestra.run.smithi039:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-21T07:25:26.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:25:26.511 DEBUG:teuthology.orchestra.run.smithi148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-21T07:25:26.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:25:26.568 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T07:25:26.569 DEBUG:teuthology.orchestra.run.smithi039:> sudo /usr/sbin/setenforce permissive 2024-09-21T07:25:26.736 DEBUG:teuthology.orchestra.run.smithi148:> sudo /usr/sbin/setenforce permissive 2024-09-21T07:25:26.802 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T07:25:26.816 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T07:25:26.827 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T07:25:26.827 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi039.front.sepia.ceph.com,smithi148.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T07:30:09.629 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi039.front.sepia.ceph.com'), Remote(name='ubuntu@smithi148.front.sepia.ceph.com')] 2024-09-21T07:30:09.630 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-09-21T07:30:09.631 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T07:30:09.724 DEBUG:teuthology.orchestra.run.smithi039:> true 2024-09-21T07:30:09.834 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-09-21T07:30:09.835 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-09-21T07:30:09.835 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T07:30:09.929 DEBUG:teuthology.orchestra.run.smithi148:> true 2024-09-21T07:30:10.045 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-09-21T07:30:10.046 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T07:30:10.055 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T07:30:10.056 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T07:30:10.056 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T07:30:10.060 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T07:30:10.060 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T07:30:10.121 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T07:30:10.150 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T07:30:10.151 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T07:30:10.180 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T07:30:10.202 INFO:teuthology.orchestra.run.smithi039.stderr:sudo: ntpd: command not found 2024-09-21T07:30:10.227 INFO:teuthology.orchestra.run.smithi039.stdout:506 Cannot talk to daemon 2024-09-21T07:30:10.231 INFO:teuthology.orchestra.run.smithi148.stderr:sudo: ntpd: command not found 2024-09-21T07:30:10.257 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T07:30:10.258 INFO:teuthology.orchestra.run.smithi148.stdout:506 Cannot talk to daemon 2024-09-21T07:30:10.284 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T07:30:10.285 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T07:30:10.314 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T07:30:10.387 INFO:teuthology.orchestra.run.smithi039.stderr:bash: line 1: ntpq: command not found 2024-09-21T07:30:10.394 INFO:teuthology.orchestra.run.smithi039.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T07:30:10.394 INFO:teuthology.orchestra.run.smithi039.stdout:=============================================================================== 2024-09-21T07:30:10.394 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T07:30:10.394 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T07:30:10.394 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T07:30:10.394 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T07:30:10.409 INFO:teuthology.orchestra.run.smithi148.stderr:bash: line 1: ntpq: command not found 2024-09-21T07:30:10.416 INFO:teuthology.orchestra.run.smithi148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T07:30:10.416 INFO:teuthology.orchestra.run.smithi148.stdout:=============================================================================== 2024-09-21T07:30:10.416 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T07:30:10.416 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T07:30:10.416 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T07:30:10.416 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T07:30:10.417 INFO:teuthology.run_tasks:Running task install... 2024-09-21T07:30:10.426 DEBUG:teuthology.task.install:project ceph 2024-09-21T07:30:10.426 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}} 2024-09-21T07:30:10.426 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'} 2024-09-21T07:30:10.426 INFO:teuthology.task.install:Using flavor: default 2024-09-21T07:30:10.431 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T07:30:10.432 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-21T07:30:10.432 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T07:30:10.432 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T07:30:10.434 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T07:30:10.434 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T07:30:10.585 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T07:30:10.585 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T07:30:10.589 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T07:30:10.589 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T07:30:10.693 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T07:30:10.693 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:30:10.693 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T07:30:10.699 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T07:30:10.700 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:30:10.700 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T07:30:10.750 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T07:30:10.750 DEBUG:teuthology.orchestra.run.smithi039:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T07:30:10.757 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T07:30:10.757 DEBUG:teuthology.orchestra.run.smithi148:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T07:30:10.848 DEBUG:teuthology.orchestra.run.smithi039:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T07:30:10.855 DEBUG:teuthology.orchestra.run.smithi148:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T07:30:10.927 DEBUG:teuthology.orchestra.run.smithi039:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T07:30:10.933 DEBUG:teuthology.orchestra.run.smithi148:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T07:30:10.982 INFO:teuthology.orchestra.run.smithi039.stdout:check_obsoletes = 1 2024-09-21T07:30:10.985 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum clean all 2024-09-21T07:30:10.988 INFO:teuthology.orchestra.run.smithi148.stdout:check_obsoletes = 1 2024-09-21T07:30:10.992 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum clean all 2024-09-21T07:30:11.266 INFO:teuthology.orchestra.run.smithi039.stdout:45 files removed 2024-09-21T07:30:11.291 INFO:teuthology.orchestra.run.smithi148.stdout:45 files removed 2024-09-21T07:30:11.299 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T07:30:11.327 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T07:30:11.999 INFO:teuthology.orchestra.run.smithi039.stdout:ceph packages for x86_64 286 kB/s | 84 kB 00:00 2024-09-21T07:30:12.012 INFO:teuthology.orchestra.run.smithi148.stdout:ceph packages for x86_64 287 kB/s | 84 kB 00:00 2024-09-21T07:30:12.299 INFO:teuthology.orchestra.run.smithi039.stdout:ceph noarch packages 95 kB/s | 24 kB 00:00 2024-09-21T07:30:12.312 INFO:teuthology.orchestra.run.smithi148.stdout:ceph noarch packages 94 kB/s | 24 kB 00:00 2024-09-21T07:30:12.591 INFO:teuthology.orchestra.run.smithi039.stdout:ceph source packages 7.5 kB/s | 1.9 kB 00:00 2024-09-21T07:30:12.605 INFO:teuthology.orchestra.run.smithi148.stdout:ceph source packages 7.5 kB/s | 1.9 kB 00:00 2024-09-21T07:30:13.075 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 8.3 MB 00:00 2024-09-21T07:30:13.497 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - BaseOS 9.6 MB/s | 8.3 MB 00:00 2024-09-21T07:30:14.919 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - AppStream 29 MB/s | 20 MB 00:00 2024-09-21T07:30:15.398 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - AppStream 26 MB/s | 20 MB 00:00 2024-09-21T07:30:19.635 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-21T07:30:20.474 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - CRB 6.4 MB/s | 6.5 MB 00:01 2024-09-21T07:30:21.320 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 19 kB 00:00 2024-09-21T07:30:21.974 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - Extras packages 106 kB/s | 19 kB 00:00 2024-09-21T07:30:22.507 INFO:teuthology.orchestra.run.smithi039.stdout:Extra Packages for Enterprise Linux 21 MB/s | 23 MB 00:01 2024-09-21T07:30:23.145 INFO:teuthology.orchestra.run.smithi148.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-21T07:30:29.321 INFO:teuthology.orchestra.run.smithi039.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-09-21T07:30:29.468 INFO:teuthology.orchestra.run.smithi148.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-21T07:30:31.812 INFO:teuthology.orchestra.run.smithi039.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T07:30:31.813 INFO:teuthology.orchestra.run.smithi039.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T07:30:31.816 INFO:teuthology.orchestra.run.smithi148.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T07:30:31.817 INFO:teuthology.orchestra.run.smithi148.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T07:30:31.827 INFO:teuthology.orchestra.run.smithi039.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T07:30:31.828 INFO:teuthology.orchestra.run.smithi039.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T07:30:31.829 INFO:teuthology.orchestra.run.smithi039.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T07:30:31.830 INFO:teuthology.orchestra.run.smithi039.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T07:30:31.831 INFO:teuthology.orchestra.run.smithi148.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T07:30:31.832 INFO:teuthology.orchestra.run.smithi148.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T07:30:31.832 INFO:teuthology.orchestra.run.smithi039.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T07:30:31.833 INFO:teuthology.orchestra.run.smithi148.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T07:30:31.834 INFO:teuthology.orchestra.run.smithi148.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T07:30:31.835 INFO:teuthology.orchestra.run.smithi039.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T07:30:31.836 INFO:teuthology.orchestra.run.smithi148.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T07:30:31.836 INFO:teuthology.orchestra.run.smithi039.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T07:30:31.839 INFO:teuthology.orchestra.run.smithi148.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T07:30:31.840 INFO:teuthology.orchestra.run.smithi148.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T07:30:31.905 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-21T07:30:31.907 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-09-21T07:30:31.925 INFO:teuthology.orchestra.run.smithi148.stdout:======================================================================================= 2024-09-21T07:30:31.925 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-09-21T07:30:31.925 INFO:teuthology.orchestra.run.smithi148.stdout:======================================================================================= 2024-09-21T07:30:31.925 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-21T07:30:31.925 INFO:teuthology.orchestra.run.smithi148.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T07:30:31.925 INFO:teuthology.orchestra.run.smithi148.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T07:30:31.926 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T07:30:31.926 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T07:30:31.926 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T07:30:31.926 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T07:30:31.926 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T07:30:31.926 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T07:30:31.926 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T07:30:31.926 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T07:30:31.926 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T07:30:31.926 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T07:30:31.926 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T07:30:31.927 INFO:teuthology.orchestra.run.smithi148.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T07:30:31.927 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T07:30:31.927 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T07:30:31.927 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T07:30:31.927 INFO:teuthology.orchestra.run.smithi148.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T07:30:31.927 INFO:teuthology.orchestra.run.smithi148.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T07:30:31.927 INFO:teuthology.orchestra.run.smithi148.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T07:30:31.927 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T07:30:31.927 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T07:30:31.927 INFO:teuthology.orchestra.run.smithi148.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T07:30:31.927 INFO:teuthology.orchestra.run.smithi148.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T07:30:31.927 INFO:teuthology.orchestra.run.smithi148.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T07:30:31.927 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T07:30:31.927 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T07:30:31.928 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T07:30:31.928 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T07:30:31.928 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T07:30:31.928 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T07:30:31.928 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T07:30:31.928 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T07:30:31.928 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T07:30:31.928 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T07:30:31.928 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T07:30:31.928 INFO:teuthology.orchestra.run.smithi148.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T07:30:31.928 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrading: 2024-09-21T07:30:31.928 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T07:30:31.929 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T07:30:31.929 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T07:30:31.929 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T07:30:31.929 INFO:teuthology.orchestra.run.smithi148.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T07:30:31.929 INFO:teuthology.orchestra.run.smithi148.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T07:30:31.929 INFO:teuthology.orchestra.run.smithi148.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T07:30:31.929 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T07:30:31.929 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-09-21T07:30:31.929 INFO:teuthology.orchestra.run.smithi148.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T07:30:31.929 INFO:teuthology.orchestra.run.smithi148.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T07:30:31.929 INFO:teuthology.orchestra.run.smithi148.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T07:30:31.929 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T07:30:31.929 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T07:30:31.929 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T07:30:31.930 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T07:30:31.930 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T07:30:31.930 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T07:30:31.930 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T07:30:31.930 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T07:30:31.930 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T07:30:31.930 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T07:30:31.930 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T07:30:31.930 INFO:teuthology.orchestra.run.smithi148.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T07:30:31.930 INFO:teuthology.orchestra.run.smithi148.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T07:30:31.930 INFO:teuthology.orchestra.run.smithi148.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T07:30:31.930 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T07:30:31.930 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T07:30:31.931 INFO:teuthology.orchestra.run.smithi148.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T07:30:31.931 INFO:teuthology.orchestra.run.smithi148.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T07:30:31.931 INFO:teuthology.orchestra.run.smithi148.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T07:30:31.931 INFO:teuthology.orchestra.run.smithi148.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T07:30:31.931 INFO:teuthology.orchestra.run.smithi148.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T07:30:31.931 INFO:teuthology.orchestra.run.smithi148.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T07:30:31.931 INFO:teuthology.orchestra.run.smithi148.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T07:30:31.931 INFO:teuthology.orchestra.run.smithi148.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T07:30:31.931 INFO:teuthology.orchestra.run.smithi148.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T07:30:31.931 INFO:teuthology.orchestra.run.smithi148.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T07:30:31.931 INFO:teuthology.orchestra.run.smithi148.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T07:30:31.931 INFO:teuthology.orchestra.run.smithi148.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T07:30:31.931 INFO:teuthology.orchestra.run.smithi148.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T07:30:31.931 INFO:teuthology.orchestra.run.smithi148.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T07:30:31.932 INFO:teuthology.orchestra.run.smithi148.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T07:30:31.932 INFO:teuthology.orchestra.run.smithi148.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T07:30:31.932 INFO:teuthology.orchestra.run.smithi148.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T07:30:31.932 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T07:30:31.932 INFO:teuthology.orchestra.run.smithi148.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T07:30:31.932 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T07:30:31.932 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T07:30:31.932 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T07:30:31.932 INFO:teuthology.orchestra.run.smithi148.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T07:30:31.932 INFO:teuthology.orchestra.run.smithi148.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T07:30:31.932 INFO:teuthology.orchestra.run.smithi148.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T07:30:31.932 INFO:teuthology.orchestra.run.smithi148.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T07:30:31.932 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T07:30:31.932 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T07:30:31.932 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T07:30:31.933 INFO:teuthology.orchestra.run.smithi148.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T07:30:31.933 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T07:30:31.933 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T07:30:31.933 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T07:30:31.933 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T07:30:31.933 INFO:teuthology.orchestra.run.smithi148.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T07:30:31.933 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T07:30:31.933 INFO:teuthology.orchestra.run.smithi148.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T07:30:31.933 INFO:teuthology.orchestra.run.smithi148.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T07:30:31.933 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T07:30:31.933 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T07:30:31.933 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T07:30:31.933 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T07:30:31.933 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T07:30:31.934 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T07:30:31.934 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T07:30:31.934 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T07:30:31.934 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T07:30:31.934 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T07:30:31.934 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T07:30:31.934 INFO:teuthology.orchestra.run.smithi148.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T07:30:31.934 INFO:teuthology.orchestra.run.smithi148.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T07:30:31.934 INFO:teuthology.orchestra.run.smithi148.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T07:30:31.934 INFO:teuthology.orchestra.run.smithi148.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T07:30:31.934 INFO:teuthology.orchestra.run.smithi148.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T07:30:31.934 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T07:30:31.934 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T07:30:31.934 INFO:teuthology.orchestra.run.smithi148.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T07:30:31.935 INFO:teuthology.orchestra.run.smithi148.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T07:30:31.935 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T07:30:31.935 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T07:30:31.935 INFO:teuthology.orchestra.run.smithi148.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T07:30:31.935 INFO:teuthology.orchestra.run.smithi148.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T07:30:31.935 INFO:teuthology.orchestra.run.smithi148.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T07:30:31.935 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T07:30:31.935 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T07:30:31.935 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T07:30:31.935 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T07:30:31.935 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T07:30:31.936 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T07:30:31.936 INFO:teuthology.orchestra.run.smithi148.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T07:30:31.936 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T07:30:31.936 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T07:30:31.936 INFO:teuthology.orchestra.run.smithi148.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T07:30:31.936 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T07:30:31.936 INFO:teuthology.orchestra.run.smithi148.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T07:30:31.936 INFO:teuthology.orchestra.run.smithi148.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T07:30:31.936 INFO:teuthology.orchestra.run.smithi148.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T07:30:31.936 INFO:teuthology.orchestra.run.smithi148.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T07:30:31.936 INFO:teuthology.orchestra.run.smithi148.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T07:30:31.936 INFO:teuthology.orchestra.run.smithi148.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T07:30:31.936 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T07:30:31.936 INFO:teuthology.orchestra.run.smithi148.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T07:30:31.937 INFO:teuthology.orchestra.run.smithi148.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T07:30:31.937 INFO:teuthology.orchestra.run.smithi148.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T07:30:31.937 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T07:30:31.937 INFO:teuthology.orchestra.run.smithi148.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T07:30:31.937 INFO:teuthology.orchestra.run.smithi148.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T07:30:31.937 INFO:teuthology.orchestra.run.smithi148.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T07:30:31.937 INFO:teuthology.orchestra.run.smithi148.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T07:30:31.937 INFO:teuthology.orchestra.run.smithi148.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T07:30:31.937 INFO:teuthology.orchestra.run.smithi148.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T07:30:31.937 INFO:teuthology.orchestra.run.smithi148.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T07:30:31.937 INFO:teuthology.orchestra.run.smithi148.stdout:Installing weak dependencies: 2024-09-21T07:30:31.937 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T07:30:31.937 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:30:31.937 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-21T07:30:31.938 INFO:teuthology.orchestra.run.smithi148.stdout:======================================================================================= 2024-09-21T07:30:31.938 INFO:teuthology.orchestra.run.smithi148.stdout:Install 149 Packages 2024-09-21T07:30:31.938 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrade 8 Packages 2024-09-21T07:30:31.938 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:30:31.939 INFO:teuthology.orchestra.run.smithi039.stdout:======================================================================================= 2024-09-21T07:30:31.939 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repository Size 2024-09-21T07:30:31.939 INFO:teuthology.orchestra.run.smithi039.stdout:======================================================================================= 2024-09-21T07:30:31.939 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-09-21T07:30:31.939 INFO:teuthology.orchestra.run.smithi039.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T07:30:31.939 INFO:teuthology.orchestra.run.smithi039.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T07:30:31.939 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T07:30:31.939 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T07:30:31.939 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T07:30:31.939 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T07:30:31.939 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T07:30:31.939 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T07:30:31.940 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T07:30:31.940 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T07:30:31.940 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T07:30:31.940 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T07:30:31.940 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T07:30:31.940 INFO:teuthology.orchestra.run.smithi039.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T07:30:31.940 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T07:30:31.940 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T07:30:31.940 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T07:30:31.941 INFO:teuthology.orchestra.run.smithi039.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T07:30:31.941 INFO:teuthology.orchestra.run.smithi039.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T07:30:31.941 INFO:teuthology.orchestra.run.smithi039.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T07:30:31.941 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T07:30:31.941 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T07:30:31.941 INFO:teuthology.orchestra.run.smithi039.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T07:30:31.941 INFO:teuthology.orchestra.run.smithi039.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T07:30:31.941 INFO:teuthology.orchestra.run.smithi039.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T07:30:31.941 INFO:teuthology.orchestra.run.smithi039.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T07:30:31.941 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T07:30:31.941 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T07:30:31.941 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T07:30:31.941 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T07:30:31.942 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T07:30:31.942 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T07:30:31.942 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T07:30:31.942 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T07:30:31.942 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T07:30:31.942 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T07:30:31.942 INFO:teuthology.orchestra.run.smithi039.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T07:30:31.942 INFO:teuthology.orchestra.run.smithi039.stdout:Upgrading: 2024-09-21T07:30:31.942 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T07:30:31.943 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T07:30:31.943 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T07:30:31.943 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T07:30:31.943 INFO:teuthology.orchestra.run.smithi039.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T07:30:31.943 INFO:teuthology.orchestra.run.smithi039.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T07:30:31.943 INFO:teuthology.orchestra.run.smithi039.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T07:30:31.943 INFO:teuthology.orchestra.run.smithi039.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T07:30:31.943 INFO:teuthology.orchestra.run.smithi039.stdout:Installing dependencies: 2024-09-21T07:30:31.943 INFO:teuthology.orchestra.run.smithi039.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T07:30:31.943 INFO:teuthology.orchestra.run.smithi039.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T07:30:31.943 INFO:teuthology.orchestra.run.smithi039.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T07:30:31.943 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T07:30:31.943 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T07:30:31.944 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T07:30:31.944 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T07:30:31.944 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T07:30:31.944 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T07:30:31.944 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T07:30:31.944 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T07:30:31.944 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T07:30:31.944 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T07:30:31.944 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T07:30:31.945 INFO:teuthology.orchestra.run.smithi039.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T07:30:31.945 INFO:teuthology.orchestra.run.smithi039.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T07:30:31.945 INFO:teuthology.orchestra.run.smithi039.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T07:30:31.945 INFO:teuthology.orchestra.run.smithi039.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T07:30:31.945 INFO:teuthology.orchestra.run.smithi039.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T07:30:31.945 INFO:teuthology.orchestra.run.smithi039.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T07:30:31.945 INFO:teuthology.orchestra.run.smithi039.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T07:30:31.945 INFO:teuthology.orchestra.run.smithi039.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T07:30:31.945 INFO:teuthology.orchestra.run.smithi039.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T07:30:31.945 INFO:teuthology.orchestra.run.smithi039.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T07:30:31.945 INFO:teuthology.orchestra.run.smithi039.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T07:30:31.945 INFO:teuthology.orchestra.run.smithi039.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T07:30:31.946 INFO:teuthology.orchestra.run.smithi039.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T07:30:31.946 INFO:teuthology.orchestra.run.smithi039.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T07:30:31.946 INFO:teuthology.orchestra.run.smithi039.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T07:30:31.946 INFO:teuthology.orchestra.run.smithi039.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T07:30:31.946 INFO:teuthology.orchestra.run.smithi039.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T07:30:31.946 INFO:teuthology.orchestra.run.smithi039.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T07:30:31.946 INFO:teuthology.orchestra.run.smithi039.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T07:30:31.946 INFO:teuthology.orchestra.run.smithi039.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T07:30:31.946 INFO:teuthology.orchestra.run.smithi039.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T07:30:31.946 INFO:teuthology.orchestra.run.smithi039.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T07:30:31.947 INFO:teuthology.orchestra.run.smithi039.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T07:30:31.947 INFO:teuthology.orchestra.run.smithi039.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T07:30:31.947 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T07:30:31.947 INFO:teuthology.orchestra.run.smithi039.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T07:30:31.947 INFO:teuthology.orchestra.run.smithi039.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T07:30:31.947 INFO:teuthology.orchestra.run.smithi039.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T07:30:31.947 INFO:teuthology.orchestra.run.smithi039.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T07:30:31.947 INFO:teuthology.orchestra.run.smithi039.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T07:30:31.947 INFO:teuthology.orchestra.run.smithi039.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T07:30:31.947 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T07:30:31.948 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T07:30:31.948 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T07:30:31.948 INFO:teuthology.orchestra.run.smithi039.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T07:30:31.948 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T07:30:31.948 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T07:30:31.948 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T07:30:31.948 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T07:30:31.948 INFO:teuthology.orchestra.run.smithi039.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T07:30:31.948 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T07:30:31.949 INFO:teuthology.orchestra.run.smithi039.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T07:30:31.949 INFO:teuthology.orchestra.run.smithi039.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T07:30:31.949 INFO:teuthology.orchestra.run.smithi039.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T07:30:31.949 INFO:teuthology.orchestra.run.smithi039.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T07:30:31.949 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T07:30:31.949 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T07:30:31.949 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T07:30:31.949 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T07:30:31.949 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T07:30:31.949 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T07:30:31.949 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T07:30:31.949 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T07:30:31.949 INFO:teuthology.orchestra.run.smithi039.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T07:30:31.949 INFO:teuthology.orchestra.run.smithi039.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T07:30:31.950 INFO:teuthology.orchestra.run.smithi039.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T07:30:31.950 INFO:teuthology.orchestra.run.smithi039.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T07:30:31.950 INFO:teuthology.orchestra.run.smithi039.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T07:30:31.950 INFO:teuthology.orchestra.run.smithi039.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T07:30:31.950 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T07:30:31.950 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T07:30:31.950 INFO:teuthology.orchestra.run.smithi039.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T07:30:31.950 INFO:teuthology.orchestra.run.smithi039.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T07:30:31.950 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T07:30:31.950 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T07:30:31.950 INFO:teuthology.orchestra.run.smithi039.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T07:30:31.950 INFO:teuthology.orchestra.run.smithi039.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T07:30:31.950 INFO:teuthology.orchestra.run.smithi039.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T07:30:31.951 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T07:30:31.951 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T07:30:31.951 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T07:30:31.951 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T07:30:31.951 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T07:30:31.951 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T07:30:31.951 INFO:teuthology.orchestra.run.smithi039.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T07:30:31.951 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T07:30:31.951 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T07:30:31.951 INFO:teuthology.orchestra.run.smithi039.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T07:30:31.951 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T07:30:31.951 INFO:teuthology.orchestra.run.smithi039.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T07:30:31.951 INFO:teuthology.orchestra.run.smithi039.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T07:30:31.951 INFO:teuthology.orchestra.run.smithi039.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T07:30:31.951 INFO:teuthology.orchestra.run.smithi039.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T07:30:31.952 INFO:teuthology.orchestra.run.smithi039.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T07:30:31.952 INFO:teuthology.orchestra.run.smithi039.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T07:30:31.952 INFO:teuthology.orchestra.run.smithi039.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T07:30:31.952 INFO:teuthology.orchestra.run.smithi039.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T07:30:31.952 INFO:teuthology.orchestra.run.smithi039.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T07:30:31.952 INFO:teuthology.orchestra.run.smithi039.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T07:30:31.952 INFO:teuthology.orchestra.run.smithi039.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T07:30:31.952 INFO:teuthology.orchestra.run.smithi039.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T07:30:31.952 INFO:teuthology.orchestra.run.smithi039.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T07:30:31.952 INFO:teuthology.orchestra.run.smithi039.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T07:30:31.952 INFO:teuthology.orchestra.run.smithi039.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T07:30:31.952 INFO:teuthology.orchestra.run.smithi039.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T07:30:31.952 INFO:teuthology.orchestra.run.smithi039.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T07:30:31.952 INFO:teuthology.orchestra.run.smithi039.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T07:30:31.953 INFO:teuthology.orchestra.run.smithi039.stdout:Installing weak dependencies: 2024-09-21T07:30:31.953 INFO:teuthology.orchestra.run.smithi039.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T07:30:31.953 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:30:31.953 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-09-21T07:30:31.953 INFO:teuthology.orchestra.run.smithi039.stdout:======================================================================================= 2024-09-21T07:30:31.953 INFO:teuthology.orchestra.run.smithi039.stdout:Install 149 Packages 2024-09-21T07:30:31.953 INFO:teuthology.orchestra.run.smithi039.stdout:Upgrade 8 Packages 2024-09-21T07:30:31.953 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:30:31.953 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 228 M 2024-09-21T07:30:31.953 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 228 M 2024-09-21T07:30:31.953 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-21T07:30:31.954 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-09-21T07:30:32.444 INFO:teuthology.orchestra.run.smithi148.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 48 kB/s | 6.6 kB 00:00 2024-09-21T07:30:32.531 INFO:teuthology.orchestra.run.smithi039.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 42 kB/s | 6.6 kB 00:00 2024-09-21T07:30:32.636 INFO:teuthology.orchestra.run.smithi148.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 4.5 MB/s | 873 kB 00:00 2024-09-21T07:30:32.686 INFO:teuthology.orchestra.run.smithi148.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 2.9 MB/s | 148 kB 00:00 2024-09-21T07:30:32.714 INFO:teuthology.orchestra.run.smithi039.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 4.7 MB/s | 873 kB 00:00 2024-09-21T07:30:32.773 INFO:teuthology.orchestra.run.smithi039.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 2.5 MB/s | 148 kB 00:00 2024-09-21T07:30:32.995 INFO:teuthology.orchestra.run.smithi148.stdout:(4/157): ceph-base-19.3.0-5120.g18432218.el9.x8 8.0 MB/s | 5.5 MB 00:00 2024-09-21T07:30:33.053 INFO:teuthology.orchestra.run.smithi148.stdout:(5/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 6.2 MB/s | 2.3 MB 00:00 2024-09-21T07:30:33.065 INFO:teuthology.orchestra.run.smithi039.stdout:(4/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 7.8 MB/s | 2.3 MB 00:00 2024-09-21T07:30:33.140 INFO:teuthology.orchestra.run.smithi039.stdout:(5/157): ceph-base-19.3.0-5120.g18432218.el9.x8 7.2 MB/s | 5.5 MB 00:00 2024-09-21T07:30:33.162 INFO:teuthology.orchestra.run.smithi148.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 9.1 MB/s | 1.5 MB 00:00 2024-09-21T07:30:33.299 INFO:teuthology.orchestra.run.smithi039.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 6.5 MB/s | 1.5 MB 00:00 2024-09-21T07:30:33.712 INFO:teuthology.orchestra.run.smithi148.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 7.5 MB/s | 5.0 MB 00:00 2024-09-21T07:30:33.733 INFO:teuthology.orchestra.run.smithi039.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 8.4 MB/s | 5.0 MB 00:00 2024-09-21T07:30:34.188 INFO:teuthology.orchestra.run.smithi148.stdout:(8/157): ceph-common-19.3.0-5120.g18432218.el9. 11 MB/s | 20 MB 00:01 2024-09-21T07:30:34.271 INFO:teuthology.orchestra.run.smithi148.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 303 kB/s | 25 kB 00:00 2024-09-21T07:30:34.450 INFO:teuthology.orchestra.run.smithi039.stdout:(8/157): ceph-common-19.3.0-5120.g18432218.el9. 9.6 MB/s | 20 MB 00:02 2024-09-21T07:30:34.533 INFO:teuthology.orchestra.run.smithi039.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 303 kB/s | 25 kB 00:00 2024-09-21T07:30:34.821 INFO:teuthology.orchestra.run.smithi148.stdout:(10/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 11 MB/s | 18 MB 00:01 2024-09-21T07:30:34.988 INFO:teuthology.orchestra.run.smithi148.stdout:(11/157): ceph-radosgw-19.3.0-5120.g18432218.el 8.2 MB/s | 10 MB 00:01 2024-09-21T07:30:34.992 INFO:teuthology.orchestra.run.smithi039.stdout:(10/157): ceph-radosgw-19.3.0-5120.g18432218.el 8.3 MB/s | 10 MB 00:01 2024-09-21T07:30:35.014 INFO:teuthology.orchestra.run.smithi148.stdout:(12/157): cephfs-mirror-19.3.0-5120.g18432218.e 1.2 MB/s | 226 kB 00:00 2024-09-21T07:30:35.039 INFO:teuthology.orchestra.run.smithi148.stdout:(13/157): cephfs-shell-19.3.0-5120.g18432218.el 539 kB/s | 27 kB 00:00 2024-09-21T07:30:35.059 INFO:teuthology.orchestra.run.smithi039.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 3.3 MB/s | 226 kB 00:00 2024-09-21T07:30:35.081 INFO:teuthology.orchestra.run.smithi148.stdout:(14/157): libcephfs-devel-19.3.0-5120.g18432218 476 kB/s | 32 kB 00:00 2024-09-21T07:30:35.117 INFO:teuthology.orchestra.run.smithi039.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 467 kB/s | 27 kB 00:00 2024-09-21T07:30:35.148 INFO:teuthology.orchestra.run.smithi148.stdout:(15/157): libcephfs2-19.3.0-5120.g18432218.el9. 6.9 MB/s | 766 kB 00:00 2024-09-21T07:30:35.173 INFO:teuthology.orchestra.run.smithi148.stdout:(16/157): libcephsqlite-19.3.0-5120.g18432218.e 1.7 MB/s | 158 kB 00:00 2024-09-21T07:30:35.175 INFO:teuthology.orchestra.run.smithi039.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 544 kB/s | 32 kB 00:00 2024-09-21T07:30:35.232 INFO:teuthology.orchestra.run.smithi148.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 1.5 MB/s | 127 kB 00:00 2024-09-21T07:30:35.265 INFO:teuthology.orchestra.run.smithi148.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 5.4 MB/s | 507 kB 00:00 2024-09-21T07:30:35.324 INFO:teuthology.orchestra.run.smithi148.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 773 kB/s | 45 kB 00:00 2024-09-21T07:30:35.382 INFO:teuthology.orchestra.run.smithi148.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 2.5 MB/s | 148 kB 00:00 2024-09-21T07:30:35.401 INFO:teuthology.orchestra.run.smithi039.stdout:(14/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 8.5 MB/s | 18 MB 00:02 2024-09-21T07:30:35.434 INFO:teuthology.orchestra.run.smithi039.stdout:(15/157): libcephfs2-19.3.0-5120.g18432218.el9. 2.9 MB/s | 766 kB 00:00 2024-09-21T07:30:35.450 INFO:teuthology.orchestra.run.smithi148.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 2.4 MB/s | 161 kB 00:00 2024-09-21T07:30:35.460 INFO:teuthology.orchestra.run.smithi039.stdout:(16/157): libcephsqlite-19.3.0-5120.g18432218.e 2.7 MB/s | 158 kB 00:00 2024-09-21T07:30:35.493 INFO:teuthology.orchestra.run.smithi039.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 2.1 MB/s | 127 kB 00:00 2024-09-21T07:30:35.551 INFO:teuthology.orchestra.run.smithi148.stdout:(22/157): python3-rados-19.3.0-5120.g18432218.e 3.1 MB/s | 321 kB 00:00 2024-09-21T07:30:35.594 INFO:teuthology.orchestra.run.smithi039.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 3.7 MB/s | 507 kB 00:00 2024-09-21T07:30:35.642 INFO:teuthology.orchestra.run.smithi148.stdout:(23/157): python3-rbd-19.3.0-5120.g18432218.el9 3.3 MB/s | 307 kB 00:00 2024-09-21T07:30:35.677 INFO:teuthology.orchestra.run.smithi039.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 543 kB/s | 45 kB 00:00 2024-09-21T07:30:35.726 INFO:teuthology.orchestra.run.smithi148.stdout:(24/157): librgw2-19.3.0-5120.g18432218.el9.x86 11 MB/s | 5.3 MB 00:00 2024-09-21T07:30:35.744 INFO:teuthology.orchestra.run.smithi039.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 2.2 MB/s | 148 kB 00:00 2024-09-21T07:30:35.751 INFO:teuthology.orchestra.run.smithi148.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 918 kB/s | 99 kB 00:00 2024-09-21T07:30:35.803 INFO:teuthology.orchestra.run.smithi039.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 2.7 MB/s | 161 kB 00:00 2024-09-21T07:30:35.809 INFO:teuthology.orchestra.run.smithi148.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.0 MB/s | 86 kB 00:00 2024-09-21T07:30:35.878 INFO:teuthology.orchestra.run.smithi039.stdout:(22/157): python3-rados-19.3.0-5120.g18432218.e 4.2 MB/s | 321 kB 00:00 2024-09-21T07:30:35.910 INFO:teuthology.orchestra.run.smithi148.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 1.7 MB/s | 170 kB 00:00 2024-09-21T07:30:35.962 INFO:teuthology.orchestra.run.smithi039.stdout:(23/157): python3-rbd-19.3.0-5120.g18432218.el9 3.6 MB/s | 307 kB 00:00 2024-09-21T07:30:36.002 INFO:teuthology.orchestra.run.smithi148.stdout:(28/157): ceph-grafana-dashboards-19.3.0-5120.g 374 kB/s | 34 kB 00:00 2024-09-21T07:30:36.062 INFO:teuthology.orchestra.run.smithi039.stdout:(24/157): librgw2-19.3.0-5120.g18432218.el9.x86 9.3 MB/s | 5.3 MB 00:00 2024-09-21T07:30:36.078 INFO:teuthology.orchestra.run.smithi148.stdout:(29/157): rbd-mirror-19.3.0-5120.g18432218.el9. 9.5 MB/s | 3.1 MB 00:00 2024-09-21T07:30:36.087 INFO:teuthology.orchestra.run.smithi039.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 792 kB/s | 99 kB 00:00 2024-09-21T07:30:36.102 INFO:teuthology.orchestra.run.smithi148.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 1.5 MB/s | 154 kB 00:00 2024-09-21T07:30:36.113 INFO:teuthology.orchestra.run.smithi039.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.7 MB/s | 86 kB 00:00 2024-09-21T07:30:36.204 INFO:teuthology.orchestra.run.smithi039.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 1.8 MB/s | 170 kB 00:00 2024-09-21T07:30:36.298 INFO:teuthology.orchestra.run.smithi039.stdout:(28/157): ceph-grafana-dashboards-19.3.0-5120.g 374 kB/s | 34 kB 00:00 2024-09-21T07:30:36.388 INFO:teuthology.orchestra.run.smithi039.stdout:(29/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 1.6 MB/s | 154 kB 00:00 2024-09-21T07:30:36.540 INFO:teuthology.orchestra.run.smithi039.stdout:(30/157): rbd-mirror-19.3.0-5120.g18432218.el9. 6.9 MB/s | 3.1 MB 00:00 2024-09-21T07:30:36.911 INFO:teuthology.orchestra.run.smithi148.stdout:(31/157): ceph-test-19.3.0-5120.g18432218.el9.x 15 MB/s | 40 MB 00:02 2024-09-21T07:30:37.003 INFO:teuthology.orchestra.run.smithi148.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 3.1 MB/s | 292 kB 00:00 2024-09-21T07:30:37.095 INFO:teuthology.orchestra.run.smithi148.stdout:(33/157): ceph-mgr-diskprediction-local-19.3.0- 7.4 MB/s | 7.4 MB 00:00 2024-09-21T07:30:37.120 INFO:teuthology.orchestra.run.smithi148.stdout:(34/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 423 kB/s | 49 kB 00:00 2024-09-21T07:30:37.145 INFO:teuthology.orchestra.run.smithi148.stdout:(35/157): ceph-prometheus-alerts-19.3.0-5120.g1 329 kB/s | 16 kB 00:00 2024-09-21T07:30:37.179 INFO:teuthology.orchestra.run.smithi148.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 4.8 MB/s | 285 kB 00:00 2024-09-21T07:30:37.238 INFO:teuthology.orchestra.run.smithi148.stdout:(37/157): cephadm-19.3.0-5120.g18432218.el9.noa 8.3 MB/s | 784 kB 00:00 2024-09-21T07:30:37.254 INFO:teuthology.orchestra.run.smithi148.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 245 kB/s | 18 kB 00:00 2024-09-21T07:30:37.405 INFO:teuthology.orchestra.run.smithi148.stdout:(39/157): ceph-mgr-dashboard-19.3.0-5120.g18432 11 MB/s | 14 MB 00:01 2024-09-21T07:30:37.440 INFO:teuthology.orchestra.run.smithi039.stdout:(31/157): ceph-test-19.3.0-5120.g18432218.el9.x 14 MB/s | 40 MB 00:02 2024-09-21T07:30:37.505 INFO:teuthology.orchestra.run.smithi148.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 299 kB/s | 80 kB 00:00 2024-09-21T07:30:37.548 INFO:teuthology.orchestra.run.smithi039.stdout:(32/157): ceph-mgr-diskprediction-local-19.3.0- 7.3 MB/s | 7.4 MB 00:01 2024-09-21T07:30:37.580 INFO:teuthology.orchestra.run.smithi148.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 191 kB 00:00 2024-09-21T07:30:37.581 INFO:teuthology.orchestra.run.smithi039.stdout:(33/157): ceph-mgr-modules-core-19.3.0-5120.g18 2.0 MB/s | 292 kB 00:00 2024-09-21T07:30:37.607 INFO:teuthology.orchestra.run.smithi039.stdout:(34/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 849 kB/s | 49 kB 00:00 2024-09-21T07:30:37.614 INFO:teuthology.orchestra.run.smithi148.stdout:(42/157): libgfortran-11.5.0-2.el9.x86_64.rpm 2.2 MB/s | 801 kB 00:00 2024-09-21T07:30:37.632 INFO:teuthology.orchestra.run.smithi039.stdout:(35/157): ceph-prometheus-alerts-19.3.0-5120.g1 328 kB/s | 16 kB 00:00 2024-09-21T07:30:37.639 INFO:teuthology.orchestra.run.smithi148.stdout:(43/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.8 MB/s | 253 kB 00:00 2024-09-21T07:30:37.657 INFO:teuthology.orchestra.run.smithi039.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 5.6 MB/s | 285 kB 00:00 2024-09-21T07:30:37.673 INFO:teuthology.orchestra.run.smithi148.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-21T07:30:37.690 INFO:teuthology.orchestra.run.smithi148.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-21T07:30:37.699 INFO:teuthology.orchestra.run.smithi039.stdout:(37/157): cephfs-top-19.3.0-5120.g18432218.el9. 444 kB/s | 18 kB 00:00 2024-09-21T07:30:37.732 INFO:teuthology.orchestra.run.smithi148.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-21T07:30:37.733 INFO:teuthology.orchestra.run.smithi039.stdout:(38/157): cephadm-19.3.0-5120.g18432218.el9.noa 7.6 MB/s | 784 kB 00:00 2024-09-21T07:30:37.774 INFO:teuthology.orchestra.run.smithi148.stdout:(47/157): python3-cryptography-36.0.1-4.el9.x86 6.4 MB/s | 1.2 MB 00:00 2024-09-21T07:30:37.799 INFO:teuthology.orchestra.run.smithi148.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-09-21T07:30:37.892 INFO:teuthology.orchestra.run.smithi039.stdout:(39/157): ceph-mgr-dashboard-19.3.0-5120.g18432 9.4 MB/s | 14 MB 00:01 2024-09-21T07:30:37.908 INFO:teuthology.orchestra.run.smithi148.stdout:(49/157): automake-1.16.2-8.el9.noarch.rpm 3.8 MB/s | 689 kB 00:00 2024-09-21T07:30:37.917 INFO:teuthology.orchestra.run.smithi039.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 368 kB/s | 80 kB 00:00 2024-09-21T07:30:37.950 INFO:teuthology.orchestra.run.smithi148.stdout:(50/157): bison-3.7.4-5.el9.x86_64.rpm 5.3 MB/s | 943 kB 00:00 2024-09-21T07:30:37.966 INFO:teuthology.orchestra.run.smithi148.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 372 kB/s | 22 kB 00:00 2024-09-21T07:30:37.992 INFO:teuthology.orchestra.run.smithi148.stdout:(52/157): boost-program-options-1.75.0-8.el9.x8 555 kB/s | 107 kB 00:00 2024-09-21T07:30:38.025 INFO:teuthology.orchestra.run.smithi148.stdout:(53/157): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 313 kB 00:00 2024-09-21T07:30:38.026 INFO:teuthology.orchestra.run.smithi039.stdout:(41/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2024-09-21T07:30:38.042 INFO:teuthology.orchestra.run.smithi148.stdout:(54/157): flexiblas-3.0.4-8.el9.x86_64.rpm 434 kB/s | 33 kB 00:00 2024-09-21T07:30:38.051 INFO:teuthology.orchestra.run.smithi039.stdout:(42/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.2 MB/s | 191 kB 00:00 2024-09-21T07:30:38.084 INFO:teuthology.orchestra.run.smithi148.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 310 kB/s | 18 kB 00:00 2024-09-21T07:30:38.085 INFO:teuthology.orchestra.run.smithi039.stdout:(43/157): libgfortran-11.5.0-2.el9.x86_64.rpm 2.2 MB/s | 801 kB 00:00 2024-09-21T07:30:38.101 INFO:teuthology.orchestra.run.smithi148.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 190 kB/s | 11 kB 00:00 2024-09-21T07:30:38.109 INFO:teuthology.orchestra.run.smithi039.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-21T07:30:38.126 INFO:teuthology.orchestra.run.smithi039.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-09-21T07:30:38.160 INFO:teuthology.orchestra.run.smithi039.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-21T07:30:38.186 INFO:teuthology.orchestra.run.smithi039.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-21T07:30:38.189 INFO:teuthology.orchestra.run.smithi148.stdout:(57/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 15 MB/s | 3.0 MB 00:00 2024-09-21T07:30:38.218 INFO:teuthology.orchestra.run.smithi148.stdout:(58/157): libnbd-1.20.2-2.el9.x86_64.rpm 1.2 MB/s | 171 kB 00:00 2024-09-21T07:30:38.243 INFO:teuthology.orchestra.run.smithi148.stdout:(59/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.1 MB/s | 160 kB 00:00 2024-09-21T07:30:38.252 INFO:teuthology.orchestra.run.smithi039.stdout:(48/157): python3-cryptography-36.0.1-4.el9.x86 5.4 MB/s | 1.2 MB 00:00 2024-09-21T07:30:38.261 INFO:teuthology.orchestra.run.smithi148.stdout:(60/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 636 kB/s | 45 kB 00:00 2024-09-21T07:30:38.311 INFO:teuthology.orchestra.run.smithi148.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-09-21T07:30:38.344 INFO:teuthology.orchestra.run.smithi148.stdout:(62/157): libtool-2.4.6-46.el9.x86_64.rpm 5.7 MB/s | 581 kB 00:00 2024-09-21T07:30:38.361 INFO:teuthology.orchestra.run.smithi148.stdout:(63/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 243 kB/s | 24 kB 00:00 2024-09-21T07:30:38.378 INFO:teuthology.orchestra.run.smithi148.stdout:(64/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 705 kB/s | 47 kB 00:00 2024-09-21T07:30:38.420 INFO:teuthology.orchestra.run.smithi148.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-09-21T07:30:38.428 INFO:teuthology.orchestra.run.smithi039.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 612 kB/s | 107 kB 00:00 2024-09-21T07:30:38.437 INFO:teuthology.orchestra.run.smithi148.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 659 kB/s | 38 kB 00:00 2024-09-21T07:30:38.454 INFO:teuthology.orchestra.run.smithi148.stdout:(67/157): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-09-21T07:30:38.461 INFO:teuthology.orchestra.run.smithi039.stdout:(50/157): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 689 kB 00:00 2024-09-21T07:30:38.487 INFO:teuthology.orchestra.run.smithi039.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 372 kB/s | 22 kB 00:00 2024-09-21T07:30:38.520 INFO:teuthology.orchestra.run.smithi039.stdout:(52/157): bison-3.7.4-5.el9.x86_64.rpm 2.8 MB/s | 943 kB 00:00 2024-09-21T07:30:38.546 INFO:teuthology.orchestra.run.smithi039.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 558 kB/s | 33 kB 00:00 2024-09-21T07:30:38.571 INFO:teuthology.orchestra.run.smithi039.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-09-21T07:30:38.571 INFO:teuthology.orchestra.run.smithi148.stdout:(68/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 33 MB/s | 4.4 MB 00:00 2024-09-21T07:30:38.596 INFO:teuthology.orchestra.run.smithi039.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-09-21T07:30:38.630 INFO:teuthology.orchestra.run.smithi039.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 191 kB/s | 11 kB 00:00 2024-09-21T07:30:38.646 INFO:teuthology.orchestra.run.smithi148.stdout:(69/157): openblas-openmp-0.3.26-2.el9.x86_64.r 22 MB/s | 4.9 MB 00:00 2024-09-21T07:30:38.696 INFO:teuthology.orchestra.run.smithi148.stdout:(70/157): postgresql-13.16-1.el9.x86_64.rpm 6.5 MB/s | 1.6 MB 00:00 2024-09-21T07:30:38.697 INFO:teuthology.orchestra.run.smithi039.stdout:(57/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 17 MB/s | 3.0 MB 00:00 2024-09-21T07:30:38.722 INFO:teuthology.orchestra.run.smithi039.stdout:(58/157): libnbd-1.20.2-2.el9.x86_64.rpm 1.3 MB/s | 171 kB 00:00 2024-09-21T07:30:38.730 INFO:teuthology.orchestra.run.smithi148.stdout:(71/157): postgresql-contrib-13.16-1.el9.x86_64 5.6 MB/s | 910 kB 00:00 2024-09-21T07:30:38.739 INFO:teuthology.orchestra.run.smithi039.stdout:(59/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-09-21T07:30:38.755 INFO:teuthology.orchestra.run.smithi148.stdout:(72/157): postgresql-private-libs-13.16-1.el9.x 1.2 MB/s | 137 kB 00:00 2024-09-21T07:30:38.764 INFO:teuthology.orchestra.run.smithi039.stdout:(60/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 673 kB/s | 45 kB 00:00 2024-09-21T07:30:38.806 INFO:teuthology.orchestra.run.smithi148.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-09-21T07:30:38.806 INFO:teuthology.orchestra.run.smithi039.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 8.5 MB/s | 581 kB 00:00 2024-09-21T07:30:38.831 INFO:teuthology.orchestra.run.smithi039.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 367 kB/s | 24 kB 00:00 2024-09-21T07:30:38.856 INFO:teuthology.orchestra.run.smithi148.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 874 kB/s | 87 kB 00:00 2024-09-21T07:30:38.865 INFO:teuthology.orchestra.run.smithi039.stdout:(63/157): librdkafka-1.6.1-102.el9.x86_64.rpm 4.5 MB/s | 662 kB 00:00 2024-09-21T07:30:38.891 INFO:teuthology.orchestra.run.smithi039.stdout:(64/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 566 kB/s | 47 kB 00:00 2024-09-21T07:30:38.924 INFO:teuthology.orchestra.run.smithi039.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-09-21T07:30:38.948 INFO:teuthology.orchestra.run.smithi148.stdout:(75/157): postgresql-server-13.16-1.el9.x86_64. 23 MB/s | 5.8 MB 00:00 2024-09-21T07:30:38.949 INFO:teuthology.orchestra.run.smithi039.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 659 kB/s | 38 kB 00:00 2024-09-21T07:30:38.973 INFO:teuthology.orchestra.run.smithi148.stdout:(76/157): python3-colorama-0.4.6-3.el9.noarch.r 523 kB/s | 61 kB 00:00 2024-09-21T07:30:38.974 INFO:teuthology.orchestra.run.smithi039.stdout:(67/157): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-09-21T07:30:39.073 INFO:teuthology.orchestra.run.smithi148.stdout:(77/157): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 6.0 MB 00:00 2024-09-21T07:30:39.099 INFO:teuthology.orchestra.run.smithi148.stdout:(78/157): python3-devel-3.9.19-8.el9.x86_64.rpm 1.6 MB/s | 245 kB 00:00 2024-09-21T07:30:39.108 INFO:teuthology.orchestra.run.smithi039.stdout:(68/157): openblas-openmp-0.3.26-2.el9.x86_64.r 27 MB/s | 4.9 MB 00:00 2024-09-21T07:30:39.124 INFO:teuthology.orchestra.run.smithi148.stdout:(79/157): python3-jinja2-2.11.3-6.el9.noarch.rp 1.6 MB/s | 248 kB 00:00 2024-09-21T07:30:39.158 INFO:teuthology.orchestra.run.smithi039.stdout:(69/157): postgresql-13.16-1.el9.x86_64.rpm 8.6 MB/s | 1.6 MB 00:00 2024-09-21T07:30:39.172 INFO:teuthology.orchestra.run.smithi148.stdout:(80/157): python3-jsonpatch-1.21-16.el9.noarch. 261 kB/s | 26 kB 00:00 2024-09-21T07:30:39.200 INFO:teuthology.orchestra.run.smithi148.stdout:(81/157): python3-jsonpointer-2.0-4.el9.noarch. 254 kB/s | 19 kB 00:00 2024-09-21T07:30:39.200 INFO:teuthology.orchestra.run.smithi039.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 9.7 MB/s | 910 kB 00:00 2024-09-21T07:30:39.225 INFO:teuthology.orchestra.run.smithi148.stdout:(82/157): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-09-21T07:30:39.250 INFO:teuthology.orchestra.run.smithi148.stdout:(83/157): python3-markupsafe-1.1.1-12.el9.x86_6 447 kB/s | 35 kB 00:00 2024-09-21T07:30:39.300 INFO:teuthology.orchestra.run.smithi148.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-09-21T07:30:39.313 INFO:teuthology.orchestra.run.smithi039.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 888 kB/s | 137 kB 00:00 2024-09-21T07:30:39.325 INFO:teuthology.orchestra.run.smithi148.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 222 kB 00:00 2024-09-21T07:30:39.392 INFO:teuthology.orchestra.run.smithi148.stdout:(86/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 26 MB/s | 5.1 MB 00:00 2024-09-21T07:30:39.409 INFO:teuthology.orchestra.run.smithi039.stdout:(72/157): postgresql-server-13.16-1.el9.x86_64. 28 MB/s | 5.8 MB 00:00 2024-09-21T07:30:39.417 INFO:teuthology.orchestra.run.smithi148.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 460 kB/s | 42 kB 00:00 2024-09-21T07:30:39.443 INFO:teuthology.orchestra.run.smithi148.stdout:(88/157): python3-packaging-20.9-5.el9.noarch.r 542 kB/s | 77 kB 00:00 2024-09-21T07:30:39.451 INFO:teuthology.orchestra.run.smithi039.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 2.1 MB/s | 87 kB 00:00 2024-09-21T07:30:39.468 INFO:teuthology.orchestra.run.smithi148.stdout:(89/157): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-09-21T07:30:39.518 INFO:teuthology.orchestra.run.smithi148.stdout:(90/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 159 kB 00:00 2024-09-21T07:30:39.535 INFO:teuthology.orchestra.run.smithi148.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 763 kB/s | 51 kB 00:00 2024-09-21T07:30:39.560 INFO:teuthology.orchestra.run.smithi148.stdout:(92/157): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-09-21T07:30:39.577 INFO:teuthology.orchestra.run.smithi148.stdout:(93/157): python3-requests-oauthlib-1.3.0-12.el 915 kB/s | 54 kB 00:00 2024-09-21T07:30:39.611 INFO:teuthology.orchestra.run.smithi148.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 831 kB/s | 42 kB 00:00 2024-09-21T07:30:39.626 INFO:teuthology.orchestra.run.smithi039.stdout:(74/157): python3-babel-2.9.1-2.el9.noarch.rpm 34 MB/s | 6.0 MB 00:00 2024-09-21T07:30:39.628 INFO:teuthology.orchestra.run.smithi148.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 880 kB/s | 44 kB 00:00 2024-09-21T07:30:39.678 INFO:teuthology.orchestra.run.smithi148.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-09-21T07:30:39.693 INFO:teuthology.orchestra.run.smithi039.stdout:(75/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 6.0 MB/s | 4.4 MB 00:00 2024-09-21T07:30:39.695 INFO:teuthology.orchestra.run.smithi148.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 537 kB/s | 36 kB 00:00 2024-09-21T07:30:39.719 INFO:teuthology.orchestra.run.smithi039.stdout:(76/157): python3-colorama-0.4.6-3.el9.noarch.r 666 kB/s | 61 kB 00:00 2024-09-21T07:30:39.737 INFO:teuthology.orchestra.run.smithi148.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 812 kB/s | 47 kB 00:00 2024-09-21T07:30:39.752 INFO:teuthology.orchestra.run.smithi039.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 4.1 MB/s | 245 kB 00:00 2024-09-21T07:30:39.778 INFO:teuthology.orchestra.run.smithi039.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-09-21T07:30:39.795 INFO:teuthology.orchestra.run.smithi039.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 610 kB/s | 26 kB 00:00 2024-09-21T07:30:39.820 INFO:teuthology.orchestra.run.smithi039.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 457 kB/s | 19 kB 00:00 2024-09-21T07:30:39.854 INFO:teuthology.orchestra.run.smithi039.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-21T07:30:39.879 INFO:teuthology.orchestra.run.smithi039.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 591 kB/s | 35 kB 00:00 2024-09-21T07:30:39.921 INFO:teuthology.orchestra.run.smithi148.stdout:(99/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 42 MB/s | 16 MB 00:00 2024-09-21T07:30:39.937 INFO:teuthology.orchestra.run.smithi039.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-09-21T07:30:39.938 INFO:teuthology.orchestra.run.smithi148.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 289 kB/s | 58 kB 00:00 2024-09-21T07:30:39.963 INFO:teuthology.orchestra.run.smithi148.stdout:(101/157): socat-1.7.4.1-6.el9.x86_64.rpm 1.1 MB/s | 304 kB 00:00 2024-09-21T07:30:39.971 INFO:teuthology.orchestra.run.smithi039.stdout:(84/157): protobuf-3.14.0-13.el9.x86_64.rpm 1.5 MB/s | 1.0 MB 00:00 2024-09-21T07:30:40.005 INFO:teuthology.orchestra.run.smithi039.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 3.2 MB/s | 222 kB 00:00 2024-09-21T07:30:40.005 INFO:teuthology.orchestra.run.smithi148.stdout:(102/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.1 MB/s | 96 kB 00:00 2024-09-21T07:30:40.022 INFO:teuthology.orchestra.run.smithi148.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 763 kB/s | 45 kB 00:00 2024-09-21T07:30:40.039 INFO:teuthology.orchestra.run.smithi148.stdout:(104/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 631 kB/s | 64 kB 00:00 2024-09-21T07:30:40.055 INFO:teuthology.orchestra.run.smithi039.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 927 kB/s | 77 kB 00:00 2024-09-21T07:30:40.064 INFO:teuthology.orchestra.run.smithi148.stdout:(105/157): lua-devel-5.4.4-4.el9.x86_64.rpm 379 kB/s | 22 kB 00:00 2024-09-21T07:30:40.080 INFO:teuthology.orchestra.run.smithi039.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 563 kB/s | 42 kB 00:00 2024-09-21T07:30:40.115 INFO:teuthology.orchestra.run.smithi148.stdout:(106/157): protobuf-compiler-3.14.0-13.el9.x86_ 9.1 MB/s | 863 kB 00:00 2024-09-21T07:30:40.139 INFO:teuthology.orchestra.run.smithi039.stdout:(88/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-09-21T07:30:40.156 INFO:teuthology.orchestra.run.smithi148.stdout:(107/157): abseil-cpp-20211102.0-3.el9.x86_64.r 4.6 MB/s | 548 kB 00:00 2024-09-21T07:30:40.190 INFO:teuthology.orchestra.run.smithi148.stdout:(108/157): gperftools-libs-2.9.1-3.el9.x86_64.r 2.4 MB/s | 308 kB 00:00 2024-09-21T07:30:40.206 INFO:teuthology.orchestra.run.smithi039.stdout:(89/157): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-09-21T07:30:40.233 INFO:teuthology.orchestra.run.smithi148.stdout:(109/157): grpc-data-1.46.7-10.el9.noarch.rpm 167 kB/s | 19 kB 00:00 2024-09-21T07:30:40.257 INFO:teuthology.orchestra.run.smithi148.stdout:(110/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 390 kB/s | 26 kB 00:00 2024-09-21T07:30:40.265 INFO:teuthology.orchestra.run.smithi039.stdout:(90/157): python3-protobuf-3.14.0-13.el9.noarch 1.3 MB/s | 269 kB 00:00 2024-09-21T07:30:40.283 INFO:teuthology.orchestra.run.smithi148.stdout:(111/157): liboath-2.6.7-2.el9.x86_64.rpm 984 kB/s | 49 kB 00:00 2024-09-21T07:30:40.290 INFO:teuthology.orchestra.run.smithi039.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 609 kB/s | 51 kB 00:00 2024-09-21T07:30:40.308 INFO:teuthology.orchestra.run.smithi148.stdout:(112/157): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-21T07:30:40.334 INFO:teuthology.orchestra.run.smithi148.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-09-21T07:30:40.340 INFO:teuthology.orchestra.run.smithi039.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 717 kB/s | 54 kB 00:00 2024-09-21T07:30:40.375 INFO:teuthology.orchestra.run.smithi148.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-09-21T07:30:40.399 INFO:teuthology.orchestra.run.smithi039.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 715 kB/s | 42 kB 00:00 2024-09-21T07:30:40.400 INFO:teuthology.orchestra.run.smithi148.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-21T07:30:40.426 INFO:teuthology.orchestra.run.smithi148.stdout:(116/157): python3-asyncssh-2.13.2-5.el9.noarch 5.8 MB/s | 548 kB 00:00 2024-09-21T07:30:40.451 INFO:teuthology.orchestra.run.smithi148.stdout:(117/157): python3-cachetools-4.2.4-1.el9.noarc 641 kB/s | 32 kB 00:00 2024-09-21T07:30:40.474 INFO:teuthology.orchestra.run.smithi039.stdout:(94/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.2 MB/s | 5.1 MB 00:00 2024-09-21T07:30:40.476 INFO:teuthology.orchestra.run.smithi148.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-21T07:30:40.499 INFO:teuthology.orchestra.run.smithi039.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 441 kB/s | 44 kB 00:00 2024-09-21T07:30:40.502 INFO:teuthology.orchestra.run.smithi148.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-21T07:30:40.533 INFO:teuthology.orchestra.run.smithi039.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-21T07:30:40.558 INFO:teuthology.orchestra.run.smithi039.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 614 kB/s | 36 kB 00:00 2024-09-21T07:30:40.561 INFO:teuthology.orchestra.run.smithi148.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-09-21T07:30:40.583 INFO:teuthology.orchestra.run.smithi039.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-09-21T07:30:40.594 INFO:teuthology.orchestra.run.smithi148.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-09-21T07:30:40.619 INFO:teuthology.orchestra.run.smithi148.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-21T07:30:40.633 INFO:teuthology.orchestra.run.smithi039.stdout:(99/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-21T07:30:40.653 INFO:teuthology.orchestra.run.smithi148.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-21T07:30:40.678 INFO:teuthology.orchestra.run.smithi148.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-09-21T07:30:40.692 INFO:teuthology.orchestra.run.smithi039.stdout:(100/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-21T07:30:40.703 INFO:teuthology.orchestra.run.smithi148.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 713 kB/s | 18 kB 00:00 2024-09-21T07:30:40.729 INFO:teuthology.orchestra.run.smithi148.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 926 kB/s | 23 kB 00:00 2024-09-21T07:30:40.754 INFO:teuthology.orchestra.run.smithi148.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 784 kB/s | 19 kB 00:00 2024-09-21T07:30:40.779 INFO:teuthology.orchestra.run.smithi148.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 785 kB/s | 20 kB 00:00 2024-09-21T07:30:40.838 INFO:teuthology.orchestra.run.smithi148.stdout:(129/157): python3-grpcio-1.46.7-10.el9.x86_64. 8.4 MB/s | 2.0 MB 00:00 2024-09-21T07:30:40.864 INFO:teuthology.orchestra.run.smithi148.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T07:30:40.884 INFO:teuthology.orchestra.run.smithi039.stdout:(101/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-09-21T07:30:40.909 INFO:teuthology.orchestra.run.smithi039.stdout:(102/157): socat-1.7.4.1-6.el9.x86_64.rpm 866 kB/s | 304 kB 00:00 2024-09-21T07:30:40.935 INFO:teuthology.orchestra.run.smithi039.stdout:(103/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 263 kB/s | 64 kB 00:00 2024-09-21T07:30:40.960 INFO:teuthology.orchestra.run.smithi039.stdout:(104/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 597 kB/s | 45 kB 00:00 2024-09-21T07:30:41.007 INFO:teuthology.orchestra.run.smithi148.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 548 kB/s | 79 kB 00:00 2024-09-21T07:30:41.018 INFO:teuthology.orchestra.run.smithi039.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-21T07:30:41.039 INFO:teuthology.orchestra.run.smithi148.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 1.8 MB/s | 58 kB 00:00 2024-09-21T07:30:41.052 INFO:teuthology.orchestra.run.smithi039.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 9.1 MB/s | 308 kB 00:00 2024-09-21T07:30:41.077 INFO:teuthology.orchestra.run.smithi039.stdout:(107/157): lua-devel-5.4.4-4.el9.x86_64.rpm 157 kB/s | 22 kB 00:00 2024-09-21T07:30:41.080 INFO:teuthology.orchestra.run.smithi148.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 3.4 MB/s | 1.0 MB 00:00 2024-09-21T07:30:41.111 INFO:teuthology.orchestra.run.smithi039.stdout:(108/157): grpc-data-1.46.7-10.el9.noarch.rpm 333 kB/s | 19 kB 00:00 2024-09-21T07:30:41.128 INFO:teuthology.orchestra.run.smithi039.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.7 MB/s | 26 kB 00:00 2024-09-21T07:30:41.139 INFO:teuthology.orchestra.run.smithi148.stdout:(134/157): libarrow-9.0.0-11.el9.x86_64.rpm 4.5 MB/s | 4.4 MB 00:00 2024-09-21T07:30:41.145 INFO:teuthology.orchestra.run.smithi039.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 2.8 MB/s | 49 kB 00:00 2024-09-21T07:30:41.165 INFO:teuthology.orchestra.run.smithi148.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2024-09-21T07:30:41.170 INFO:teuthology.orchestra.run.smithi039.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-09-21T07:30:41.190 INFO:teuthology.orchestra.run.smithi148.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-21T07:30:41.204 INFO:teuthology.orchestra.run.smithi039.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 4.4 MB/s | 151 kB 00:00 2024-09-21T07:30:41.232 INFO:teuthology.orchestra.run.smithi148.stdout:(137/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 977 kB/s | 90 kB 00:00 2024-09-21T07:30:41.237 INFO:teuthology.orchestra.run.smithi039.stdout:(113/157): protobuf-compiler-3.14.0-13.el9.x86_ 2.8 MB/s | 863 kB 00:00 2024-09-21T07:30:41.248 INFO:teuthology.orchestra.run.smithi148.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 526 kB/s | 31 kB 00:00 2024-09-21T07:30:41.274 INFO:teuthology.orchestra.run.smithi148.stdout:(139/157): python3-pecan-1.4.2-3.el9.noarch.rpm 1.1 MB/s | 272 kB 00:00 2024-09-21T07:30:41.291 INFO:teuthology.orchestra.run.smithi148.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-09-21T07:30:41.308 INFO:teuthology.orchestra.run.smithi148.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 1.0 MB/s | 36 kB 00:00 2024-09-21T07:30:41.321 INFO:teuthology.orchestra.run.smithi039.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-09-21T07:30:41.333 INFO:teuthology.orchestra.run.smithi148.stdout:(142/157): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-09-21T07:30:41.346 INFO:teuthology.orchestra.run.smithi039.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-21T07:30:41.358 INFO:teuthology.orchestra.run.smithi148.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 1.1 MB/s | 74 kB 00:00 2024-09-21T07:30:41.371 INFO:teuthology.orchestra.run.smithi039.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-21T07:30:41.384 INFO:teuthology.orchestra.run.smithi148.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-09-21T07:30:41.397 INFO:teuthology.orchestra.run.smithi039.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 569 kB/s | 14 kB 00:00 2024-09-21T07:30:41.417 INFO:teuthology.orchestra.run.smithi148.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-21T07:30:41.431 INFO:teuthology.orchestra.run.smithi039.stdout:(118/157): python3-cheroot-8.6.0-4.el9.noarch.r 4.9 MB/s | 172 kB 00:00 2024-09-21T07:30:41.442 INFO:teuthology.orchestra.run.smithi148.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 341 kB/s | 20 kB 00:00 2024-09-21T07:30:41.456 INFO:teuthology.orchestra.run.smithi039.stdout:(119/157): python3-asyncssh-2.13.2-5.el9.noarch 2.4 MB/s | 548 kB 00:00 2024-09-21T07:30:41.459 INFO:teuthology.orchestra.run.smithi148.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-21T07:30:41.485 INFO:teuthology.orchestra.run.smithi148.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-21T07:30:41.490 INFO:teuthology.orchestra.run.smithi039.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-21T07:30:41.524 INFO:teuthology.orchestra.run.smithi039.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-21T07:30:41.550 INFO:teuthology.orchestra.run.smithi039.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 3.6 MB/s | 220 kB 00:00 2024-09-21T07:30:41.552 INFO:teuthology.orchestra.run.smithi148.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-21T07:30:41.575 INFO:teuthology.orchestra.run.smithi039.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-21T07:30:41.600 INFO:teuthology.orchestra.run.smithi039.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 431 kB/s | 11 kB 00:00 2024-09-21T07:30:41.625 INFO:teuthology.orchestra.run.smithi039.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 708 kB/s | 18 kB 00:00 2024-09-21T07:30:41.650 INFO:teuthology.orchestra.run.smithi039.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 932 kB/s | 23 kB 00:00 2024-09-21T07:30:41.676 INFO:teuthology.orchestra.run.smithi039.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 780 kB/s | 19 kB 00:00 2024-09-21T07:30:41.701 INFO:teuthology.orchestra.run.smithi039.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 786 kB/s | 20 kB 00:00 2024-09-21T07:30:41.744 INFO:teuthology.orchestra.run.smithi148.stdout:(150/157): librbd1-19.3.0-5120.g18432218.el9.x8 12 MB/s | 3.2 MB 00:00 2024-09-21T07:30:41.785 INFO:teuthology.orchestra.run.smithi039.stdout:(129/157): libarrow-9.0.0-11.el9.x86_64.rpm 6.2 MB/s | 4.4 MB 00:00 2024-09-21T07:30:41.803 INFO:teuthology.orchestra.run.smithi148.stdout:(151/157): librados2-19.3.0-5120.g18432218.el9. 9.9 MB/s | 3.4 MB 00:00 2024-09-21T07:30:41.814 INFO:teuthology.orchestra.run.smithi039.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T07:30:41.827 INFO:teuthology.orchestra.run.smithi039.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 4.6 MB/s | 79 kB 00:00 2024-09-21T07:30:41.869 INFO:teuthology.orchestra.run.smithi039.stdout:(132/157): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-09-21T07:30:41.894 INFO:teuthology.orchestra.run.smithi039.stdout:(133/157): python3-natsort-7.1.1-5.el9.noarch.r 866 kB/s | 58 kB 00:00 2024-09-21T07:30:41.919 INFO:teuthology.orchestra.run.smithi039.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 661 kB/s | 16 kB 00:00 2024-09-21T07:30:41.936 INFO:teuthology.orchestra.run.smithi148.stdout:(152/157): NetworkManager-1.51.0-1.el9.x86_64.r 6.0 MB/s | 2.3 MB 00:00 2024-09-21T07:30:41.945 INFO:teuthology.orchestra.run.smithi039.stdout:(135/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-21T07:30:41.962 INFO:teuthology.orchestra.run.smithi148.stdout:(153/157): NetworkManager-team-1.51.0-1.el9.x86 258 kB/s | 41 kB 00:00 2024-09-21T07:30:41.971 INFO:teuthology.orchestra.run.smithi039.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-21T07:30:41.995 INFO:teuthology.orchestra.run.smithi148.stdout:(154/157): NetworkManager-tui-1.51.0-1.el9.x86_ 4.1 MB/s | 249 kB 00:00 2024-09-21T07:30:42.012 INFO:teuthology.orchestra.run.smithi039.stdout:(137/157): python3-grpcio-1.46.7-10.el9.x86_64. 4.2 MB/s | 2.0 MB 00:00 2024-09-21T07:30:42.037 INFO:teuthology.orchestra.run.smithi039.stdout:(138/157): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-09-21T07:30:42.054 INFO:teuthology.orchestra.run.smithi148.stdout:(155/157): NetworkManager-libnm-1.51.0-1.el9.x8 5.8 MB/s | 1.8 MB 00:00 2024-09-21T07:30:42.062 INFO:teuthology.orchestra.run.smithi039.stdout:(139/157): python3-repoze-lru-0.7-16.el9.noarch 334 kB/s | 31 kB 00:00 2024-09-21T07:30:42.088 INFO:teuthology.orchestra.run.smithi039.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-21T07:30:42.113 INFO:teuthology.orchestra.run.smithi039.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-09-21T07:30:42.138 INFO:teuthology.orchestra.run.smithi039.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-21T07:30:42.163 INFO:teuthology.orchestra.run.smithi039.stdout:(143/157): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-09-21T07:30:42.189 INFO:teuthology.orchestra.run.smithi039.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-21T07:30:42.199 INFO:teuthology.orchestra.run.smithi148.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 5.7 MB/s | 1.4 MB 00:00 2024-09-21T07:30:42.214 INFO:teuthology.orchestra.run.smithi039.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-21T07:30:42.245 INFO:teuthology.orchestra.run.smithi148.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 9.6 MB/s | 2.4 MB 00:00 2024-09-21T07:30:42.250 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-21T07:30:42.251 INFO:teuthology.orchestra.run.smithi148.stdout:Total 22 MB/s | 228 MB 00:10 2024-09-21T07:30:42.255 INFO:teuthology.orchestra.run.smithi039.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 298 kB/s | 20 kB 00:00 2024-09-21T07:30:42.283 INFO:teuthology.orchestra.run.smithi039.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-09-21T07:30:42.306 INFO:teuthology.orchestra.run.smithi039.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-09-21T07:30:42.375 INFO:teuthology.orchestra.run.smithi039.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-21T07:30:42.541 INFO:teuthology.orchestra.run.smithi039.stdout:(150/157): librados2-19.3.0-5120.g18432218.el9. 13 MB/s | 3.4 MB 00:00 2024-09-21T07:30:42.683 INFO:teuthology.orchestra.run.smithi039.stdout:(151/157): librbd1-19.3.0-5120.g18432218.el9.x8 8.4 MB/s | 3.2 MB 00:00 2024-09-21T07:30:42.799 INFO:teuthology.orchestra.run.smithi039.stdout:(152/157): NetworkManager-libnm-1.51.0-1.el9.x8 7.0 MB/s | 1.8 MB 00:00 2024-09-21T07:30:42.824 INFO:teuthology.orchestra.run.smithi039.stdout:(153/157): NetworkManager-team-1.51.0-1.el9.x86 288 kB/s | 41 kB 00:00 2024-09-21T07:30:42.858 INFO:teuthology.orchestra.run.smithi039.stdout:(154/157): NetworkManager-tui-1.51.0-1.el9.x86_ 4.2 MB/s | 249 kB 00:00 2024-09-21T07:30:42.925 INFO:teuthology.orchestra.run.smithi039.stdout:(155/157): NetworkManager-1.51.0-1.el9.x86_64.r 4.3 MB/s | 2.3 MB 00:00 2024-09-21T07:30:43.067 INFO:teuthology.orchestra.run.smithi039.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 5.6 MB/s | 1.4 MB 00:00 2024-09-21T07:30:43.117 INFO:teuthology.orchestra.run.smithi039.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 9.3 MB/s | 2.4 MB 00:00 2024-09-21T07:30:43.123 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-09-21T07:30:43.123 INFO:teuthology.orchestra.run.smithi039.stdout:Total 20 MB/s | 228 MB 00:11 2024-09-21T07:30:44.310 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-21T07:30:44.494 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-21T07:30:44.494 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-21T07:30:45.245 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-09-21T07:30:45.444 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-09-21T07:30:45.445 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-09-21T07:30:46.804 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-21T07:30:46.804 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-21T07:30:47.971 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-09-21T07:30:47.971 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-09-21T07:30:53.564 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-21T07:30:53.742 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T07:30:53.937 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T07:30:54.110 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T07:30:54.162 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T07:30:54.185 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T07:30:54.251 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T07:30:54.638 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T07:30:54.772 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T07:30:54.928 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T07:30:54.952 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T07:30:55.048 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T07:30:55.077 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-09-21T07:30:55.104 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T07:30:55.140 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T07:30:55.214 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T07:30:55.269 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T07:30:55.349 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T07:30:55.357 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T07:30:55.400 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T07:30:55.414 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T07:30:55.431 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T07:30:55.466 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T07:30:55.496 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T07:30:55.582 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T07:30:55.608 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T07:30:55.698 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T07:30:55.726 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T07:30:55.797 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T07:30:55.906 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T07:30:56.003 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T07:30:56.023 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T07:30:56.177 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T07:30:56.189 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T07:30:56.208 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T07:30:56.305 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T07:30:56.338 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T07:30:56.378 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T07:30:56.416 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T07:30:56.537 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T07:30:56.604 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T07:30:56.626 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T07:30:56.670 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T07:30:56.696 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T07:30:56.826 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T07:30:56.864 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T07:30:56.911 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T07:30:56.946 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T07:30:56.973 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T07:30:57.017 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T07:30:57.064 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T07:30:57.121 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T07:30:57.168 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T07:30:57.232 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T07:30:57.250 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T07:30:57.303 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T07:30:57.343 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T07:30:57.411 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T07:30:57.436 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T07:30:57.485 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T07:30:57.519 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T07:30:57.563 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T07:30:57.606 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T07:30:57.663 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T07:30:57.678 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T07:30:57.710 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T07:30:57.773 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T07:30:57.779 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T07:30:57.873 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T07:30:57.910 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T07:30:57.954 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T07:30:57.982 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T07:30:58.066 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T07:30:58.077 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T07:30:58.126 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T07:30:58.220 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T07:30:58.225 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T07:30:58.266 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T07:30:58.307 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T07:30:58.392 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T07:30:58.395 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T07:30:58.452 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T07:30:58.471 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T07:30:58.490 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T07:30:58.545 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T07:30:58.592 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T07:30:58.625 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T07:30:58.716 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T07:30:58.804 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T07:30:58.837 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T07:30:58.889 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T07:30:58.982 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T07:30:59.016 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T07:30:59.061 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T07:30:59.096 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T07:30:59.109 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T07:30:59.189 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T07:30:59.212 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T07:30:59.250 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T07:30:59.257 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T07:30:59.309 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T07:30:59.350 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T07:30:59.381 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T07:30:59.434 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T07:30:59.515 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T07:30:59.579 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T07:30:59.609 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T07:30:59.712 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T07:30:59.818 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T07:30:59.890 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T07:30:59.974 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T07:31:00.039 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T07:31:00.084 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T07:31:00.150 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T07:31:00.191 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T07:31:00.237 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T07:31:00.249 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T07:31:00.287 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T07:31:00.347 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T07:31:00.389 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T07:31:00.438 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T07:31:00.457 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T07:31:00.479 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T07:31:00.503 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T07:31:00.533 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T07:31:00.546 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T07:31:00.663 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T07:31:00.712 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T07:31:00.714 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T07:31:00.766 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T07:31:00.801 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T07:31:00.825 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T07:31:00.947 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T07:31:01.125 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T07:31:01.141 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T07:31:01.175 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T07:31:01.224 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T07:31:01.263 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T07:31:01.268 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T07:31:01.303 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T07:31:01.344 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T07:31:01.359 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T07:31:01.413 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T07:31:01.434 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T07:31:01.448 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T07:31:01.479 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T07:31:01.481 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T07:31:01.515 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T07:31:01.525 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T07:31:01.557 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T07:31:01.574 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T07:31:01.614 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T07:31:01.655 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T07:31:01.660 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T07:31:01.716 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T07:31:01.741 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T07:31:01.761 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T07:31:01.882 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T07:31:01.922 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T07:31:01.949 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T07:31:01.977 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T07:31:02.020 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T07:31:02.150 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T07:31:02.201 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T07:31:02.253 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T07:31:02.282 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T07:31:02.306 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T07:31:02.422 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T07:31:02.590 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T07:31:02.640 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T07:31:02.687 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T07:31:02.747 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T07:31:02.811 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T07:31:02.852 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T07:31:02.857 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T07:31:02.894 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T07:31:02.910 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T07:31:02.938 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T07:31:02.968 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T07:31:02.971 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T07:31:03.015 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T07:31:03.047 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T07:31:03.088 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T07:31:03.142 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T07:31:03.151 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T07:31:03.198 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T07:31:03.203 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T07:31:03.234 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T07:31:03.299 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T07:31:03.345 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T07:31:03.362 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T07:31:03.423 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T07:31:03.585 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T07:31:03.617 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T07:31:03.713 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T07:31:04.357 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T07:31:04.415 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T07:31:04.449 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T07:31:04.537 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T07:31:04.567 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T07:31:04.625 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T07:31:04.679 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T07:31:04.689 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T07:31:04.813 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T07:31:04.851 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T07:31:05.094 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T07:31:05.131 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T07:31:05.229 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T07:31:06.082 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T07:31:06.250 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T07:31:06.263 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T07:31:06.380 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T07:31:06.416 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T07:31:06.449 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T07:31:06.809 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T07:31:06.877 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T07:31:06.981 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T07:31:07.060 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T07:31:07.449 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T07:31:07.906 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T07:31:08.003 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T07:31:08.004 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T07:31:08.038 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T07:31:08.040 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T07:31:08.073 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T07:31:08.129 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T07:31:08.157 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T07:31:08.463 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T07:31:08.818 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T07:31:08.939 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T07:31:08.974 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T07:31:09.024 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T07:31:09.612 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T07:31:09.654 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T07:31:09.729 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T07:31:09.756 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T07:31:10.664 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T07:31:10.705 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T07:31:11.373 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T07:31:11.425 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T07:31:11.576 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T07:31:11.576 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T07:31:11.576 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T07:31:11.576 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:31:11.634 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T07:31:12.337 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T07:31:12.377 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T07:31:13.076 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T07:31:13.122 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T07:31:13.248 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T07:31:13.248 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T07:31:13.248 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T07:31:13.248 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:31:13.290 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T07:31:24.124 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T07:31:24.125 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /sys 2024-09-21T07:31:24.125 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /proc 2024-09-21T07:31:24.125 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /mnt 2024-09-21T07:31:24.125 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /var/tmp 2024-09-21T07:31:24.125 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /home 2024-09-21T07:31:24.125 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /root 2024-09-21T07:31:24.125 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /tmp 2024-09-21T07:31:24.125 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:31:24.346 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T07:31:24.387 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T07:31:24.387 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:31:24.388 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T07:31:24.388 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:31:26.082 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T07:31:26.140 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T07:31:26.140 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:31:26.140 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T07:31:26.140 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T07:31:26.140 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T07:31:26.140 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:31:26.383 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T07:31:26.433 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T07:31:26.433 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:31:26.433 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T07:31:26.433 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T07:31:26.433 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T07:31:26.434 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:31:26.499 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T07:31:26.499 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /sys 2024-09-21T07:31:26.499 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /proc 2024-09-21T07:31:26.499 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /mnt 2024-09-21T07:31:26.500 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /var/tmp 2024-09-21T07:31:26.500 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /home 2024-09-21T07:31:26.500 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /root 2024-09-21T07:31:26.500 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /tmp 2024-09-21T07:31:26.500 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:31:26.719 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T07:31:26.756 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T07:31:26.756 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:31:26.756 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T07:31:26.756 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:31:26.967 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T07:31:27.018 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T07:31:27.018 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:31:27.018 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T07:31:27.018 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T07:31:27.018 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T07:31:27.018 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:31:27.151 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T07:31:27.211 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T07:31:27.274 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T07:31:27.509 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T07:31:27.571 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T07:31:27.680 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T07:31:27.790 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T07:31:27.962 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T07:31:28.012 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T07:31:28.055 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T07:31:28.077 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T07:31:28.538 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T07:31:28.573 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T07:31:28.597 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T07:31:28.597 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:31:28.597 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T07:31:28.597 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T07:31:28.597 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T07:31:28.597 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:31:28.614 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T07:31:28.652 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T07:31:28.734 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T07:31:28.766 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T07:31:28.855 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T07:31:28.906 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T07:31:28.906 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:31:28.907 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T07:31:28.907 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T07:31:28.907 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T07:31:28.907 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:31:29.456 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T07:31:29.508 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T07:31:29.508 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:31:29.508 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T07:31:29.508 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T07:31:29.509 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T07:31:29.509 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:31:29.648 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T07:31:29.710 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T07:31:29.783 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T07:31:30.021 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T07:31:30.085 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T07:31:30.176 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T07:31:30.294 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T07:31:30.473 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T07:31:30.526 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T07:31:30.566 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T07:31:30.599 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T07:31:31.173 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T07:31:31.207 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T07:31:31.250 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T07:31:31.333 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T07:31:31.363 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T07:31:32.415 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T07:31:32.480 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T07:31:33.577 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T07:31:33.613 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T07:31:33.764 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T07:31:33.974 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T07:31:34.007 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T07:31:34.059 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T07:31:34.059 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:31:34.059 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T07:31:34.059 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T07:31:34.059 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T07:31:34.059 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:31:34.116 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T07:31:34.145 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T07:31:34.259 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T07:31:34.331 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T07:31:35.244 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T07:31:35.319 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T07:31:35.391 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T07:31:35.419 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T07:31:35.470 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T07:31:35.470 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:31:35.471 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T07:31:35.471 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T07:31:35.471 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T07:31:35.471 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:31:35.519 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T07:31:35.569 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T07:31:35.570 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:31:35.570 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T07:31:35.570 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:31:35.628 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T07:31:35.679 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T07:31:35.679 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:31:35.679 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T07:31:35.680 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T07:31:35.680 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T07:31:35.680 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:31:36.037 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T07:31:36.088 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T07:31:36.088 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:31:36.089 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T07:31:36.089 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T07:31:36.089 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T07:31:36.089 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:31:36.470 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T07:31:36.502 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T07:31:36.665 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T07:31:36.874 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T07:31:36.904 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T07:31:36.957 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T07:31:36.957 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:31:36.957 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T07:31:36.957 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T07:31:36.957 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T07:31:36.957 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:31:37.013 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T07:31:37.040 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T07:31:37.155 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T07:31:37.218 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T07:31:38.330 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T07:31:38.366 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T07:31:38.419 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T07:31:38.419 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:31:38.419 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T07:31:38.419 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T07:31:38.419 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T07:31:38.419 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:31:38.475 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T07:31:38.525 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T07:31:38.526 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:31:38.526 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T07:31:38.526 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:31:38.584 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T07:31:38.636 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T07:31:38.636 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:31:38.636 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T07:31:38.636 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T07:31:38.636 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T07:31:38.636 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:31:39.018 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T07:31:39.069 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T07:31:39.070 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:31:39.070 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T07:31:39.070 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T07:31:39.070 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T07:31:39.070 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:31:40.459 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T07:31:40.501 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T07:31:40.552 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T07:31:40.581 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T07:31:40.616 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T07:31:40.881 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T07:31:40.929 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T07:31:40.959 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T07:31:40.990 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T07:31:41.067 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T07:31:41.745 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T07:31:41.798 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T07:31:41.841 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T07:31:43.135 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T07:31:43.239 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T07:31:43.297 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T07:31:43.427 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T07:31:43.458 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T07:31:43.483 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T07:31:43.519 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T07:31:43.519 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T07:31:43.610 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T07:31:43.611 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T07:31:43.707 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T07:31:43.728 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T07:31:43.752 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T07:31:43.757 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T07:31:43.768 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T07:31:43.768 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T07:31:43.809 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T07:31:43.845 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T07:31:43.880 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T07:31:44.015 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T07:31:44.015 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T07:31:44.131 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T07:31:44.155 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T07:31:44.208 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T07:31:44.211 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T07:31:44.239 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T07:31:44.273 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T07:31:44.315 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T07:31:44.435 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T07:31:44.435 INFO:teuthology.orchestra.run.smithi148.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-21T07:31:44.435 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:31:45.123 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T07:31:45.171 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T07:31:45.213 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T07:31:46.612 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T07:31:46.720 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T07:31:46.775 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T07:31:46.894 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T07:31:46.939 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T07:31:46.964 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T07:31:47.000 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T07:31:47.001 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T07:31:47.100 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T07:31:47.101 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T07:31:47.209 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T07:31:47.233 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T07:31:47.249 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T07:31:47.249 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T07:31:47.502 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T07:31:47.502 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T07:31:47.612 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T07:31:47.689 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T07:31:47.897 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T07:31:47.898 INFO:teuthology.orchestra.run.smithi039.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-21T07:31:47.898 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:31:50.651 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T07:31:50.651 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T07:31:50.651 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T07:31:50.651 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T07:31:50.651 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T07:31:50.652 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T07:31:50.652 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T07:31:50.652 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T07:31:50.652 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T07:31:50.652 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T07:31:50.652 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T07:31:50.652 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T07:31:50.652 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T07:31:50.652 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T07:31:50.652 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T07:31:50.652 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T07:31:50.652 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T07:31:50.652 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T07:31:50.652 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T07:31:50.653 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T07:31:50.653 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T07:31:50.653 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T07:31:50.653 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T07:31:50.653 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T07:31:50.653 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T07:31:50.653 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T07:31:50.664 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T07:31:50.665 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T07:31:50.665 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T07:31:50.665 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T07:31:50.665 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T07:31:50.665 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T07:31:50.665 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T07:31:50.665 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T07:31:50.665 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T07:31:50.665 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T07:31:50.665 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T07:31:50.665 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T07:31:50.665 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T07:31:50.665 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T07:31:50.665 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T07:31:50.666 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T07:31:50.666 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T07:31:50.666 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T07:31:50.666 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T07:31:50.666 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T07:31:50.666 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T07:31:50.666 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T07:31:50.666 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T07:31:50.666 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T07:31:50.666 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T07:31:50.666 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T07:31:50.666 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T07:31:50.666 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T07:31:50.666 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T07:31:50.667 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T07:31:50.667 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T07:31:50.667 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T07:31:50.667 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T07:31:50.667 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T07:31:50.667 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T07:31:50.667 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T07:31:50.667 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T07:31:50.667 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T07:31:50.667 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T07:31:50.667 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T07:31:50.667 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T07:31:50.667 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T07:31:50.667 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T07:31:50.668 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T07:31:50.668 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T07:31:50.668 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T07:31:50.668 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T07:31:50.668 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T07:31:50.668 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T07:31:50.668 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T07:31:50.668 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T07:31:50.668 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T07:31:50.668 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T07:31:50.668 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T07:31:50.668 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T07:31:50.668 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T07:31:50.668 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T07:31:50.668 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T07:31:50.669 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T07:31:50.669 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T07:31:50.669 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T07:31:50.669 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T07:31:50.669 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T07:31:50.669 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T07:31:50.669 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T07:31:50.669 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T07:31:50.669 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T07:31:50.669 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T07:31:50.669 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T07:31:50.669 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T07:31:50.669 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T07:31:50.669 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T07:31:50.669 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T07:31:50.670 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T07:31:50.670 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T07:31:50.670 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T07:31:50.670 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T07:31:50.670 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T07:31:50.670 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T07:31:50.670 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T07:31:50.670 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T07:31:50.670 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T07:31:50.670 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T07:31:50.670 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T07:31:50.670 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T07:31:50.670 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T07:31:50.671 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T07:31:50.671 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T07:31:50.671 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T07:31:50.671 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T07:31:50.671 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T07:31:50.671 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T07:31:50.671 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T07:31:50.671 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T07:31:50.671 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T07:31:50.671 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T07:31:50.671 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T07:31:50.671 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T07:31:50.671 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T07:31:50.671 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T07:31:50.672 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T07:31:50.672 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T07:31:50.672 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T07:31:50.672 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T07:31:50.672 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T07:31:50.672 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T07:31:50.672 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T07:31:50.672 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T07:31:50.672 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T07:31:50.672 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T07:31:50.672 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T07:31:50.672 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T07:31:50.672 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T07:31:50.673 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T07:31:50.673 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T07:31:50.673 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T07:31:50.673 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T07:31:50.673 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T07:31:50.673 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T07:31:50.673 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T07:31:50.673 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T07:31:50.673 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T07:31:50.673 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T07:31:50.673 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T07:31:50.673 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T07:31:50.673 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T07:31:50.674 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T07:31:50.674 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T07:31:50.674 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T07:31:50.674 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T07:31:50.674 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T07:31:50.674 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T07:31:50.674 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T07:31:50.674 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T07:31:50.674 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T07:31:50.674 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T07:31:50.674 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T07:31:50.674 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T07:31:52.925 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T07:31:52.925 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:31:52.926 INFO:teuthology.orchestra.run.smithi148.stdout:Upgraded: 2024-09-21T07:31:52.926 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T07:31:52.926 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T07:31:52.926 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T07:31:52.926 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T07:31:52.926 INFO:teuthology.orchestra.run.smithi148.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:52.926 INFO:teuthology.orchestra.run.smithi148.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:52.926 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T07:31:52.927 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T07:31:52.927 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-21T07:31:52.927 INFO:teuthology.orchestra.run.smithi148.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T07:31:52.927 INFO:teuthology.orchestra.run.smithi148.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T07:31:52.927 INFO:teuthology.orchestra.run.smithi148.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T07:31:52.927 INFO:teuthology.orchestra.run.smithi148.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T07:31:52.927 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:52.927 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:52.927 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:52.927 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:52.927 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:31:52.927 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:52.927 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:52.928 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:52.928 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:31:52.928 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:31:52.928 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:31:52.928 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:31:52.928 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:31:52.928 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:52.928 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:52.928 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:31:52.928 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:52.928 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:52.928 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:52.929 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:31:52.929 INFO:teuthology.orchestra.run.smithi148.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:31:52.929 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:52.929 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:52.929 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:31:52.929 INFO:teuthology.orchestra.run.smithi148.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T07:31:52.929 INFO:teuthology.orchestra.run.smithi148.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T07:31:52.929 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T07:31:52.929 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T07:31:52.929 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T07:31:52.929 INFO:teuthology.orchestra.run.smithi148.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T07:31:52.929 INFO:teuthology.orchestra.run.smithi148.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T07:31:52.929 INFO:teuthology.orchestra.run.smithi148.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T07:31:52.930 INFO:teuthology.orchestra.run.smithi148.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T07:31:52.930 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T07:31:52.930 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T07:31:52.930 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:52.930 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:52.930 INFO:teuthology.orchestra.run.smithi148.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:52.930 INFO:teuthology.orchestra.run.smithi148.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T07:31:52.930 INFO:teuthology.orchestra.run.smithi148.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T07:31:52.930 INFO:teuthology.orchestra.run.smithi148.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T07:31:52.930 INFO:teuthology.orchestra.run.smithi148.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T07:31:52.930 INFO:teuthology.orchestra.run.smithi148.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T07:31:52.930 INFO:teuthology.orchestra.run.smithi148.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T07:31:52.930 INFO:teuthology.orchestra.run.smithi148.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:52.931 INFO:teuthology.orchestra.run.smithi148.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:52.931 INFO:teuthology.orchestra.run.smithi148.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T07:31:52.931 INFO:teuthology.orchestra.run.smithi148.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:52.931 INFO:teuthology.orchestra.run.smithi148.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T07:31:52.931 INFO:teuthology.orchestra.run.smithi148.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T07:31:52.931 INFO:teuthology.orchestra.run.smithi148.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T07:31:52.931 INFO:teuthology.orchestra.run.smithi148.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T07:31:52.931 INFO:teuthology.orchestra.run.smithi148.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T07:31:52.931 INFO:teuthology.orchestra.run.smithi148.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T07:31:52.932 INFO:teuthology.orchestra.run.smithi148.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T07:31:52.932 INFO:teuthology.orchestra.run.smithi148.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T07:31:52.932 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T07:31:52.932 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T07:31:52.932 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T07:31:52.933 INFO:teuthology.orchestra.run.smithi148.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T07:31:52.933 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T07:31:52.933 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T07:31:52.933 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T07:31:52.933 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T07:31:52.933 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T07:31:52.933 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T07:31:52.934 INFO:teuthology.orchestra.run.smithi148.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T07:31:52.934 INFO:teuthology.orchestra.run.smithi148.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T07:31:52.934 INFO:teuthology.orchestra.run.smithi148.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T07:31:52.934 INFO:teuthology.orchestra.run.smithi148.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T07:31:52.934 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T07:31:52.934 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:52.934 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:52.935 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:52.935 INFO:teuthology.orchestra.run.smithi148.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T07:31:52.935 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T07:31:52.935 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T07:31:52.935 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T07:31:52.935 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T07:31:52.936 INFO:teuthology.orchestra.run.smithi148.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T07:31:52.936 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T07:31:52.936 INFO:teuthology.orchestra.run.smithi148.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T07:31:52.936 INFO:teuthology.orchestra.run.smithi148.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T07:31:52.936 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T07:31:52.936 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T07:31:52.937 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T07:31:52.937 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T07:31:52.937 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T07:31:52.937 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T07:31:52.937 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T07:31:52.937 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T07:31:52.938 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T07:31:52.938 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T07:31:52.938 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T07:31:52.938 INFO:teuthology.orchestra.run.smithi148.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T07:31:52.938 INFO:teuthology.orchestra.run.smithi148.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T07:31:52.938 INFO:teuthology.orchestra.run.smithi148.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T07:31:52.939 INFO:teuthology.orchestra.run.smithi148.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T07:31:52.939 INFO:teuthology.orchestra.run.smithi148.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T07:31:52.939 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T07:31:52.939 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T07:31:52.939 INFO:teuthology.orchestra.run.smithi148.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T07:31:52.939 INFO:teuthology.orchestra.run.smithi148.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T07:31:52.940 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T07:31:52.940 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T07:31:52.940 INFO:teuthology.orchestra.run.smithi148.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T07:31:52.940 INFO:teuthology.orchestra.run.smithi148.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T07:31:52.940 INFO:teuthology.orchestra.run.smithi148.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T07:31:52.940 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T07:31:52.941 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T07:31:52.941 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T07:31:52.941 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T07:31:52.941 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T07:31:52.941 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T07:31:52.941 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:52.942 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:52.942 INFO:teuthology.orchestra.run.smithi148.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T07:31:52.942 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T07:31:52.942 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T07:31:52.942 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:52.942 INFO:teuthology.orchestra.run.smithi148.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T07:31:52.943 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T07:31:52.943 INFO:teuthology.orchestra.run.smithi148.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T07:31:52.943 INFO:teuthology.orchestra.run.smithi148.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T07:31:52.943 INFO:teuthology.orchestra.run.smithi148.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T07:31:52.943 INFO:teuthology.orchestra.run.smithi148.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T07:31:52.943 INFO:teuthology.orchestra.run.smithi148.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T07:31:52.944 INFO:teuthology.orchestra.run.smithi148.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T07:31:52.944 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T07:31:52.944 INFO:teuthology.orchestra.run.smithi148.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T07:31:52.944 INFO:teuthology.orchestra.run.smithi148.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T07:31:52.944 INFO:teuthology.orchestra.run.smithi148.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T07:31:52.944 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T07:31:52.945 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T07:31:52.945 INFO:teuthology.orchestra.run.smithi148.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T07:31:52.945 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:52.945 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:52.945 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:52.945 INFO:teuthology.orchestra.run.smithi148.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T07:31:52.945 INFO:teuthology.orchestra.run.smithi148.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T07:31:52.946 INFO:teuthology.orchestra.run.smithi148.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T07:31:52.946 INFO:teuthology.orchestra.run.smithi148.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T07:31:52.946 INFO:teuthology.orchestra.run.smithi148.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T07:31:52.946 INFO:teuthology.orchestra.run.smithi148.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T07:31:52.946 INFO:teuthology.orchestra.run.smithi148.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T07:31:52.946 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:31:52.947 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-21T07:31:53.615 DEBUG:teuthology.parallel:result is None 2024-09-21T07:31:54.224 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T07:31:54.224 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T07:31:54.224 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T07:31:54.224 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T07:31:54.224 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T07:31:54.224 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T07:31:54.224 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T07:31:54.225 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T07:31:54.225 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T07:31:54.225 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T07:31:54.225 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T07:31:54.225 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T07:31:54.225 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T07:31:54.225 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T07:31:54.225 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T07:31:54.225 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T07:31:54.225 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T07:31:54.225 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T07:31:54.225 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T07:31:54.225 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T07:31:54.225 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T07:31:54.226 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T07:31:54.226 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T07:31:54.226 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T07:31:54.226 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T07:31:54.226 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T07:31:54.226 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T07:31:54.226 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T07:31:54.226 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T07:31:54.226 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T07:31:54.226 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T07:31:54.226 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T07:31:54.226 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T07:31:54.226 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T07:31:54.227 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T07:31:54.227 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T07:31:54.227 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T07:31:54.227 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T07:31:54.227 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T07:31:54.227 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T07:31:54.227 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T07:31:54.227 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T07:31:54.227 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T07:31:54.227 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T07:31:54.227 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T07:31:54.228 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T07:31:54.228 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T07:31:54.228 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T07:31:54.228 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T07:31:54.228 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T07:31:54.228 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T07:31:54.228 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T07:31:54.228 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T07:31:54.228 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T07:31:54.228 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T07:31:54.228 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T07:31:54.228 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T07:31:54.228 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T07:31:54.228 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T07:31:54.228 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T07:31:54.229 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T07:31:54.229 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T07:31:54.229 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T07:31:54.229 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T07:31:54.229 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T07:31:54.229 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T07:31:54.229 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T07:31:54.229 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T07:31:54.229 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T07:31:54.229 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T07:31:54.229 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T07:31:54.230 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T07:31:54.230 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T07:31:54.230 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T07:31:54.230 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T07:31:54.230 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T07:31:54.230 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T07:31:54.230 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T07:31:54.230 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T07:31:54.230 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T07:31:54.230 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T07:31:54.230 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T07:31:54.230 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T07:31:54.230 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T07:31:54.230 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T07:31:54.231 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T07:31:54.231 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T07:31:54.231 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T07:31:54.231 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T07:31:54.231 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T07:31:54.231 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T07:31:54.231 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T07:31:54.231 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T07:31:54.231 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T07:31:54.231 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T07:31:54.231 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T07:31:54.231 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T07:31:54.231 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T07:31:54.231 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T07:31:54.232 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T07:31:54.232 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T07:31:54.232 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T07:31:54.232 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T07:31:54.232 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T07:31:54.232 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T07:31:54.232 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T07:31:54.232 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T07:31:54.232 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T07:31:54.232 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T07:31:54.232 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T07:31:54.232 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T07:31:54.232 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T07:31:54.232 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T07:31:54.233 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T07:31:54.233 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T07:31:54.233 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T07:31:54.233 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T07:31:54.233 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T07:31:54.233 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T07:31:54.233 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T07:31:54.233 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T07:31:54.233 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T07:31:54.233 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T07:31:54.233 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T07:31:54.233 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T07:31:54.233 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T07:31:54.233 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T07:31:54.233 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T07:31:54.234 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T07:31:54.234 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T07:31:54.234 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T07:31:54.234 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T07:31:54.234 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T07:31:54.234 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T07:31:54.234 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T07:31:54.234 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T07:31:54.234 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T07:31:54.234 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T07:31:54.234 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T07:31:54.234 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T07:31:54.235 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T07:31:54.235 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T07:31:54.235 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T07:31:54.235 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T07:31:54.235 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T07:31:54.235 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T07:31:54.235 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T07:31:54.235 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T07:31:54.235 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T07:31:54.235 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T07:31:54.235 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T07:31:54.235 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T07:31:54.235 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T07:31:54.236 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T07:31:54.236 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T07:31:54.236 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T07:31:54.236 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T07:31:54.236 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T07:31:54.236 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T07:31:54.236 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T07:31:54.236 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T07:31:54.236 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T07:31:54.236 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T07:31:54.236 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T07:31:56.567 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T07:31:56.567 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:31:56.567 INFO:teuthology.orchestra.run.smithi039.stdout:Upgraded: 2024-09-21T07:31:56.567 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T07:31:56.567 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T07:31:56.567 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T07:31:56.567 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T07:31:56.567 INFO:teuthology.orchestra.run.smithi039.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:56.568 INFO:teuthology.orchestra.run.smithi039.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:56.568 INFO:teuthology.orchestra.run.smithi039.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T07:31:56.568 INFO:teuthology.orchestra.run.smithi039.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T07:31:56.568 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-09-21T07:31:56.568 INFO:teuthology.orchestra.run.smithi039.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T07:31:56.568 INFO:teuthology.orchestra.run.smithi039.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T07:31:56.568 INFO:teuthology.orchestra.run.smithi039.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T07:31:56.568 INFO:teuthology.orchestra.run.smithi039.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T07:31:56.568 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:56.568 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:56.568 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:56.568 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:56.568 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:31:56.568 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:56.569 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:56.569 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:56.569 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:31:56.569 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:31:56.569 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:31:56.569 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:31:56.569 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:31:56.569 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:56.569 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:56.569 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:31:56.569 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:56.569 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:56.569 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:56.569 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:31:56.570 INFO:teuthology.orchestra.run.smithi039.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:31:56.570 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:56.570 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:56.570 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:31:56.570 INFO:teuthology.orchestra.run.smithi039.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T07:31:56.570 INFO:teuthology.orchestra.run.smithi039.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T07:31:56.570 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T07:31:56.570 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T07:31:56.570 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T07:31:56.570 INFO:teuthology.orchestra.run.smithi039.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T07:31:56.570 INFO:teuthology.orchestra.run.smithi039.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T07:31:56.570 INFO:teuthology.orchestra.run.smithi039.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T07:31:56.570 INFO:teuthology.orchestra.run.smithi039.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T07:31:56.570 INFO:teuthology.orchestra.run.smithi039.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T07:31:56.571 INFO:teuthology.orchestra.run.smithi039.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T07:31:56.571 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:56.571 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:56.571 INFO:teuthology.orchestra.run.smithi039.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:56.571 INFO:teuthology.orchestra.run.smithi039.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T07:31:56.571 INFO:teuthology.orchestra.run.smithi039.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T07:31:56.571 INFO:teuthology.orchestra.run.smithi039.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T07:31:56.571 INFO:teuthology.orchestra.run.smithi039.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T07:31:56.571 INFO:teuthology.orchestra.run.smithi039.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T07:31:56.571 INFO:teuthology.orchestra.run.smithi039.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T07:31:56.571 INFO:teuthology.orchestra.run.smithi039.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:56.571 INFO:teuthology.orchestra.run.smithi039.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:56.571 INFO:teuthology.orchestra.run.smithi039.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T07:31:56.572 INFO:teuthology.orchestra.run.smithi039.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:56.572 INFO:teuthology.orchestra.run.smithi039.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T07:31:56.572 INFO:teuthology.orchestra.run.smithi039.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T07:31:56.572 INFO:teuthology.orchestra.run.smithi039.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T07:31:56.572 INFO:teuthology.orchestra.run.smithi039.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T07:31:56.572 INFO:teuthology.orchestra.run.smithi039.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T07:31:56.572 INFO:teuthology.orchestra.run.smithi039.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T07:31:56.572 INFO:teuthology.orchestra.run.smithi039.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T07:31:56.572 INFO:teuthology.orchestra.run.smithi039.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T07:31:56.572 INFO:teuthology.orchestra.run.smithi039.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T07:31:56.572 INFO:teuthology.orchestra.run.smithi039.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T07:31:56.572 INFO:teuthology.orchestra.run.smithi039.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T07:31:56.572 INFO:teuthology.orchestra.run.smithi039.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T07:31:56.572 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T07:31:56.572 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T07:31:56.573 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T07:31:56.573 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T07:31:56.573 INFO:teuthology.orchestra.run.smithi039.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T07:31:56.573 INFO:teuthology.orchestra.run.smithi039.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T07:31:56.573 INFO:teuthology.orchestra.run.smithi039.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T07:31:56.573 INFO:teuthology.orchestra.run.smithi039.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T07:31:56.573 INFO:teuthology.orchestra.run.smithi039.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T07:31:56.573 INFO:teuthology.orchestra.run.smithi039.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T07:31:56.573 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T07:31:56.573 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:56.573 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:56.573 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:56.573 INFO:teuthology.orchestra.run.smithi039.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T07:31:56.573 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T07:31:56.573 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T07:31:56.574 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T07:31:56.574 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T07:31:56.574 INFO:teuthology.orchestra.run.smithi039.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T07:31:56.574 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T07:31:56.574 INFO:teuthology.orchestra.run.smithi039.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T07:31:56.574 INFO:teuthology.orchestra.run.smithi039.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T07:31:56.574 INFO:teuthology.orchestra.run.smithi039.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T07:31:56.574 INFO:teuthology.orchestra.run.smithi039.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T07:31:56.574 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T07:31:56.574 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T07:31:56.574 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T07:31:56.574 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T07:31:56.574 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T07:31:56.574 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T07:31:56.575 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T07:31:56.575 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T07:31:56.575 INFO:teuthology.orchestra.run.smithi039.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T07:31:56.575 INFO:teuthology.orchestra.run.smithi039.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T07:31:56.575 INFO:teuthology.orchestra.run.smithi039.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T07:31:56.575 INFO:teuthology.orchestra.run.smithi039.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T07:31:56.575 INFO:teuthology.orchestra.run.smithi039.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T07:31:56.575 INFO:teuthology.orchestra.run.smithi039.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T07:31:56.575 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T07:31:56.575 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T07:31:56.575 INFO:teuthology.orchestra.run.smithi039.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T07:31:56.575 INFO:teuthology.orchestra.run.smithi039.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T07:31:56.575 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T07:31:56.575 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T07:31:56.575 INFO:teuthology.orchestra.run.smithi039.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T07:31:56.576 INFO:teuthology.orchestra.run.smithi039.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T07:31:56.576 INFO:teuthology.orchestra.run.smithi039.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T07:31:56.576 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T07:31:56.576 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T07:31:56.576 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T07:31:56.576 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T07:31:56.576 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T07:31:56.576 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T07:31:56.576 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:56.576 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:56.576 INFO:teuthology.orchestra.run.smithi039.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T07:31:56.576 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T07:31:56.576 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T07:31:56.576 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:56.577 INFO:teuthology.orchestra.run.smithi039.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T07:31:56.577 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T07:31:56.577 INFO:teuthology.orchestra.run.smithi039.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T07:31:56.577 INFO:teuthology.orchestra.run.smithi039.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T07:31:56.577 INFO:teuthology.orchestra.run.smithi039.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T07:31:56.577 INFO:teuthology.orchestra.run.smithi039.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T07:31:56.577 INFO:teuthology.orchestra.run.smithi039.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T07:31:56.577 INFO:teuthology.orchestra.run.smithi039.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T07:31:56.577 INFO:teuthology.orchestra.run.smithi039.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T07:31:56.577 INFO:teuthology.orchestra.run.smithi039.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T07:31:56.577 INFO:teuthology.orchestra.run.smithi039.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T07:31:56.577 INFO:teuthology.orchestra.run.smithi039.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T07:31:56.577 INFO:teuthology.orchestra.run.smithi039.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T07:31:56.577 INFO:teuthology.orchestra.run.smithi039.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T07:31:56.577 INFO:teuthology.orchestra.run.smithi039.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T07:31:56.577 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:56.578 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:56.578 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:31:56.578 INFO:teuthology.orchestra.run.smithi039.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T07:31:56.578 INFO:teuthology.orchestra.run.smithi039.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T07:31:56.578 INFO:teuthology.orchestra.run.smithi039.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T07:31:56.578 INFO:teuthology.orchestra.run.smithi039.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T07:31:56.578 INFO:teuthology.orchestra.run.smithi039.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T07:31:56.578 INFO:teuthology.orchestra.run.smithi039.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T07:31:56.578 INFO:teuthology.orchestra.run.smithi039.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T07:31:56.578 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:31:56.579 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-09-21T07:31:57.170 DEBUG:teuthology.parallel:result is None 2024-09-21T07:31:57.170 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T07:31:57.312 DEBUG:teuthology.orchestra.run.smithi039:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T07:31:57.358 INFO:teuthology.orchestra.run.smithi039.stdout:19.3.0-5120.g18432218.el9 2024-09-21T07:31:57.359 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T07:31:57.359 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T07:31:57.361 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T07:31:57.502 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T07:31:57.554 INFO:teuthology.orchestra.run.smithi148.stdout:19.3.0-5120.g18432218.el9 2024-09-21T07:31:57.554 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T07:31:57.555 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T07:31:57.556 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T07:31:57.557 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:31:57.557 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T07:31:57.618 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:31:57.618 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T07:31:57.677 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T07:31:57.677 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:31:57.677 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T07:31:57.735 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T07:31:57.834 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:31:57.835 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T07:31:57.892 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T07:31:57.988 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T07:31:57.988 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:31:57.988 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T07:31:58.047 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T07:31:58.148 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:31:58.148 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T07:31:58.205 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T07:31:58.301 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T07:31:58.301 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:31:58.301 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T07:31:58.363 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T07:31:58.462 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:31:58.463 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T07:31:58.518 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T07:31:58.614 INFO:teuthology.run_tasks:Running task ceph... 2024-09-21T07:31:58.691 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-21T07:31:58.691 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 777 /var/log/ceph 2024-09-21T07:31:58.695 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 777 /var/log/ceph 2024-09-21T07:31:58.754 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-21T07:31:58.754 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T07:31:58.759 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T07:31:58.815 INFO:tasks.ceph:Creating extra log directories... 2024-09-21T07:31:58.815 DEBUG:teuthology.orchestra.run.smithi039:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T07:31:58.819 DEBUG:teuthology.orchestra.run.smithi148:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T07:31:58.881 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-21T07:31:58.881 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-21T07:31:58.881 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-21T07:31:58.881 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:31:58.881 DEBUG:teuthology.orchestra.run.smithi039:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-21T07:31:58.920 DEBUG:teuthology.orchestra.run.smithi039:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-21T07:31:59.103 DEBUG:teuthology.orchestra.run.smithi039:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-21T07:31:59.159 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:31:59.159 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-21T07:31:59.194 DEBUG:teuthology.orchestra.run.smithi148:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-21T07:31:59.363 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-21T07:31:59.423 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-21T07:31:59.423 INFO:tasks.ceph:config {'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'evicting unresponsive client', 'RECENT_CRASH'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-21T07:31:59.423 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913602', 'branch': 'wip-pdonnell-testing-20240920.212106-debug', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/snapshots}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7913602', 'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20240920.212106-debug'}, 'ceph': {'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'evicting unresponsive client', 'RECENT_CRASH'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'check-counter': {'dry_run': True}, 'install': {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}}, 'kclient': {'syntax': 'v2'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240920.212106-debug', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 90, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 7864, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'sleep_before_teardown': 0, 'subset': '67/512', 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20240920.212106-debug', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi039.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE/444ear+/QNL2dzlGKlS6MtvL7iVEZnYbJfsrfKdpvTUU0FPzjwxrgA6zk8lGwMcqoardajhSGzIQxKKKzZxE=', 'smithi148.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGTY8Z8iIfpfOzg9VGQxWzpaUnnGwoUVPqRfeKLhaZfAuri1kEyYYnjeRZbta8xMl46Zk2Iwx2ZMrS7DfhUxV4s='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}}, {'ceph': {'create_rbd_pool': False, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'evicting unresponsive client', 'RECENT_CRASH'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'cluster': 'ceph'}}, {'kclient': None}, {'exec': {'mon.a': ['ceph config set mgr mgr/crash/warn_recent_interval 0']}}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_snapshots']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-20_23:48:22', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2902'} 2024-09-21T07:31:59.424 DEBUG:teuthology.orchestra.run.smithi039:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T07:31:59.428 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T07:31:59.463 DEBUG:teuthology.orchestra.run.smithi039:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T07:31:59.503 DEBUG:teuthology.orchestra.run.smithi148:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T07:31:59.558 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:31:59.559 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T07:31:59.593 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T07:31:59.594 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_1 2024-09-21T07:31:59.669 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T07:31:59.669 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T07:31:59.669 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 6h/6d Inode: 631 Links: 1 2024-09-21T07:31:59.670 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T07:31:59.670 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T07:31:59.670 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-09-21 07:31:48.258592347 +0000 2024-09-21T07:31:59.670 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-09-21 07:29:46.450328213 +0000 2024-09-21T07:31:59.670 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-09-21 07:29:46.450328213 +0000 2024-09-21T07:31:59.670 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: 2024-09-21 07:29:46.450328213 +0000 2024-09-21T07:31:59.670 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T07:31:59.762 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-09-21T07:31:59.762 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-09-21T07:31:59.763 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000438497 s, 1.2 MB/s 2024-09-21T07:31:59.765 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T07:31:59.802 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_2 2024-09-21T07:31:59.878 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T07:31:59.878 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T07:31:59.878 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-09-21T07:31:59.878 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T07:31:59.878 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T07:31:59.878 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-09-21 07:31:48.258592347 +0000 2024-09-21T07:31:59.878 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-09-21 07:29:46.900319556 +0000 2024-09-21T07:31:59.879 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-09-21 07:29:46.900319556 +0000 2024-09-21T07:31:59.879 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: 2024-09-21 07:29:46.900319556 +0000 2024-09-21T07:31:59.879 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T07:31:59.973 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-09-21T07:31:59.973 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-09-21T07:31:59.973 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000393653 s, 1.3 MB/s 2024-09-21T07:31:59.976 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T07:32:00.014 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_3 2024-09-21T07:32:00.089 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T07:32:00.089 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T07:32:00.089 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-09-21T07:32:00.089 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T07:32:00.089 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T07:32:00.089 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-09-21 07:31:48.259592327 +0000 2024-09-21T07:32:00.089 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-09-21 07:29:47.332311244 +0000 2024-09-21T07:32:00.089 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-09-21 07:29:47.332311244 +0000 2024-09-21T07:32:00.090 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: 2024-09-21 07:29:47.332311244 +0000 2024-09-21T07:32:00.090 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T07:32:00.184 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-09-21T07:32:00.184 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-09-21T07:32:00.184 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000389105 s, 1.3 MB/s 2024-09-21T07:32:00.186 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T07:32:00.222 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_4 2024-09-21T07:32:00.297 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T07:32:00.298 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T07:32:00.298 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-09-21T07:32:00.298 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T07:32:00.298 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T07:32:00.298 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-09-21 07:31:48.260592308 +0000 2024-09-21T07:32:00.298 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-09-21 07:29:47.808302086 +0000 2024-09-21T07:32:00.298 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-09-21 07:29:47.808302086 +0000 2024-09-21T07:32:00.298 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: 2024-09-21 07:29:47.808302086 +0000 2024-09-21T07:32:00.298 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T07:32:00.390 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-09-21T07:32:00.390 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-09-21T07:32:00.390 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000339847 s, 1.5 MB/s 2024-09-21T07:32:00.393 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T07:32:00.429 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T07:32:00.429 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:32:00.429 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T07:32:00.468 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T07:32:00.469 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_1 2024-09-21T07:32:00.547 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T07:32:00.547 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T07:32:00.547 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 6h/6d Inode: 639 Links: 1 2024-09-21T07:32:00.547 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T07:32:00.547 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T07:32:00.547 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-09-21 07:31:44.795000530 +0000 2024-09-21T07:32:00.547 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-09-21 07:29:43.376608855 +0000 2024-09-21T07:32:00.547 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-09-21 07:29:43.376608855 +0000 2024-09-21T07:32:00.547 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-09-21 07:29:43.376608855 +0000 2024-09-21T07:32:00.548 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T07:32:00.641 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-09-21T07:32:00.642 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-09-21T07:32:00.642 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000290711 s, 1.8 MB/s 2024-09-21T07:32:00.644 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T07:32:00.682 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_2 2024-09-21T07:32:00.757 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T07:32:00.757 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T07:32:00.757 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 6h/6d Inode: 656 Links: 1 2024-09-21T07:32:00.757 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T07:32:00.758 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T07:32:00.758 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-09-21 07:31:44.795000530 +0000 2024-09-21T07:32:00.758 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-09-21 07:29:43.870597318 +0000 2024-09-21T07:32:00.758 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-09-21 07:29:43.870597318 +0000 2024-09-21T07:32:00.758 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-09-21 07:29:43.870597318 +0000 2024-09-21T07:32:00.758 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T07:32:00.850 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-09-21T07:32:00.851 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-09-21T07:32:00.851 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000278333 s, 1.8 MB/s 2024-09-21T07:32:00.854 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T07:32:00.891 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_3 2024-09-21T07:32:00.966 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T07:32:00.966 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T07:32:00.966 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 6h/6d Inode: 677 Links: 1 2024-09-21T07:32:00.966 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T07:32:00.966 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T07:32:00.966 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-09-21 07:31:44.796000506 +0000 2024-09-21T07:32:00.967 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-09-21 07:29:44.349586130 +0000 2024-09-21T07:32:00.967 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-09-21 07:29:44.349586130 +0000 2024-09-21T07:32:00.967 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-09-21 07:29:44.349586130 +0000 2024-09-21T07:32:00.967 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T07:32:01.059 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-09-21T07:32:01.059 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-09-21T07:32:01.059 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000622201 s, 823 kB/s 2024-09-21T07:32:01.062 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T07:32:01.098 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_4 2024-09-21T07:32:01.172 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T07:32:01.173 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T07:32:01.173 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 6h/6d Inode: 694 Links: 1 2024-09-21T07:32:01.173 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T07:32:01.173 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T07:32:01.173 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-09-21 07:31:44.797000483 +0000 2024-09-21T07:32:01.173 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-09-21 07:29:44.778576111 +0000 2024-09-21T07:32:01.173 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-09-21 07:29:44.778576111 +0000 2024-09-21T07:32:01.173 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-09-21 07:29:44.778576111 +0000 2024-09-21T07:32:01.173 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T07:32:01.265 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-09-21T07:32:01.266 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-09-21T07:32:01.266 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000284381 s, 1.8 MB/s 2024-09-21T07:32:01.268 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T07:32:01.304 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T07:32:01.304 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi039.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi148.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-21T07:32:01.304 INFO:tasks.ceph:Generating config... 2024-09-21T07:32:01.306 INFO:tasks.ceph:[client] pid file = 2024-09-21T07:32:01.306 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-09-21T07:32:01.306 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-21T07:32:01.306 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-21T07:32:01.306 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-21T07:32:01.306 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-21T07:32:01.306 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-21T07:32:01.306 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-21T07:32:01.306 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-21T07:32:01.306 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-21T07:32:01.306 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-21T07:32:01.307 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-21T07:32:01.307 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-21T07:32:01.307 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-21T07:32:01.307 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-21T07:32:01.307 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-21T07:32:01.307 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-21T07:32:01.307 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-21T07:32:01.307 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-21T07:32:01.307 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-21T07:32:01.307 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-21T07:32:01.307 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-21T07:32:01.307 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-21T07:32:01.307 INFO:tasks.ceph:Setting up mon.a... 2024-09-21T07:32:01.308 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-21T07:32:01.470 INFO:teuthology.orchestra.run.smithi039.stdout:creating /etc/ceph/ceph.keyring 2024-09-21T07:32:01.483 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-21T07:32:01.639 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T07:32:01.694 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.39'), ('mon.b', '172.21.15.148'), ('mon.c', '[v2:172.21.15.148:3301,v1:172.21.15.148:6790]')] 2024-09-21T07:32:01.695 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.39,172.21.15.148,[v2:172.21.15.148:3301,v1:172.21.15.148:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'pid file': ''}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}} 2024-09-21T07:32:01.695 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:32:01.696 DEBUG:teuthology.orchestra.run.smithi039:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-21T07:32:01.729 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.39 --add b 172.21.15.148 --addv c '[v2:172.21.15.148:3301,v1:172.21.15.148:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:32:01.913 INFO:teuthology.orchestra.run.smithi039.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:32:01.913 INFO:teuthology.orchestra.run.smithi039.stdout:monmaptool: generated fsid 7ab9b04a-f88d-49aa-a493-e5c076637ffb 2024-09-21T07:32:01.913 INFO:teuthology.orchestra.run.smithi039.stdout:setting min_mon_release = quincy 2024-09-21T07:32:01.913 INFO:teuthology.orchestra.run.smithi039.stdout:epoch 0 2024-09-21T07:32:01.913 INFO:teuthology.orchestra.run.smithi039.stdout:fsid 7ab9b04a-f88d-49aa-a493-e5c076637ffb 2024-09-21T07:32:01.914 INFO:teuthology.orchestra.run.smithi039.stdout:last_changed 2024-09-21T07:32:01.912664+0000 2024-09-21T07:32:01.914 INFO:teuthology.orchestra.run.smithi039.stdout:created 2024-09-21T07:32:01.912664+0000 2024-09-21T07:32:01.914 INFO:teuthology.orchestra.run.smithi039.stdout:min_mon_release 17 (quincy) 2024-09-21T07:32:01.914 INFO:teuthology.orchestra.run.smithi039.stdout:election_strategy: 1 2024-09-21T07:32:01.914 INFO:teuthology.orchestra.run.smithi039.stdout:0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-09-21T07:32:01.914 INFO:teuthology.orchestra.run.smithi039.stdout:1: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.b 2024-09-21T07:32:01.914 INFO:teuthology.orchestra.run.smithi039.stdout:2: [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] mon.c 2024-09-21T07:32:01.914 INFO:teuthology.orchestra.run.smithi039.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-21T07:32:01.922 DEBUG:teuthology.orchestra.run.smithi039:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-21T07:32:01.956 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 7ab9b04a-f88d-49aa-a493-e5c076637ffb... 2024-09-21T07:32:01.957 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T07:32:02.000 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T07:32:02.094 INFO:teuthology.orchestra.run.smithi039.stdout:[global] 2024-09-21T07:32:02.095 INFO:teuthology.orchestra.run.smithi039.stdout: chdir = "" 2024-09-21T07:32:02.095 INFO:teuthology.orchestra.run.smithi039.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T07:32:02.095 INFO:teuthology.orchestra.run.smithi039.stdout: auth supported = cephx 2024-09-21T07:32:02.095 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:32:02.095 INFO:teuthology.orchestra.run.smithi039.stdout: filestore xattr use omap = true 2024-09-21T07:32:02.095 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:32:02.095 INFO:teuthology.orchestra.run.smithi039.stdout: mon clock drift allowed = 1.000 2024-09-21T07:32:02.095 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:32:02.095 INFO:teuthology.orchestra.run.smithi039.stdout: osd crush chooseleaf type = 0 2024-09-21T07:32:02.095 INFO:teuthology.orchestra.run.smithi039.stdout: auth debug = true 2024-09-21T07:32:02.095 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:32:02.096 INFO:teuthology.orchestra.run.smithi039.stdout: ms die on old message = true 2024-09-21T07:32:02.096 INFO:teuthology.orchestra.run.smithi039.stdout: ms die on bug = true 2024-09-21T07:32:02.096 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:32:02.096 INFO:teuthology.orchestra.run.smithi039.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T07:32:02.096 INFO:teuthology.orchestra.run.smithi039.stdout: mon pg warn max object skew = 0 2024-09-21T07:32:02.096 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:32:02.096 INFO:teuthology.orchestra.run.smithi039.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T07:32:02.096 INFO:teuthology.orchestra.run.smithi039.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T07:32:02.096 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:32:02.096 INFO:teuthology.orchestra.run.smithi039.stdout: osd pool default size = 2 2024-09-21T07:32:02.096 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:32:02.096 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd allow primary affinity = true 2024-09-21T07:32:02.096 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd allow pg remap = true 2024-09-21T07:32:02.097 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on legacy crush tunables = false 2024-09-21T07:32:02.097 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on crush straw calc version zero = false 2024-09-21T07:32:02.097 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on no sortbitwise = false 2024-09-21T07:32:02.097 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on osd down out interval zero = false 2024-09-21T07:32:02.097 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on too few osds = false 2024-09-21T07:32:02.097 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T07:32:02.097 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T07:32:02.097 INFO:teuthology.orchestra.run.smithi039.stdout: mon_allow_pool_size_one = true 2024-09-21T07:32:02.097 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:32:02.097 INFO:teuthology.orchestra.run.smithi039.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T07:32:02.097 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:32:02.097 INFO:teuthology.orchestra.run.smithi039.stdout: osd default data pool replay window = 5 2024-09-21T07:32:02.098 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:32:02.098 INFO:teuthology.orchestra.run.smithi039.stdout: mon allow pool delete = true 2024-09-21T07:32:02.098 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:32:02.098 INFO:teuthology.orchestra.run.smithi039.stdout: mon cluster log file level = debug 2024-09-21T07:32:02.098 INFO:teuthology.orchestra.run.smithi039.stdout: debug asserts on shutdown = true 2024-09-21T07:32:02.098 INFO:teuthology.orchestra.run.smithi039.stdout: mon health detail to clog = false 2024-09-21T07:32:02.098 INFO:teuthology.orchestra.run.smithi039.stdout: mon host = "172.21.15.39,172.21.15.148,[v2:172.21.15.148:3301,v1:172.21.15.148:6790]" 2024-09-21T07:32:02.098 INFO:teuthology.orchestra.run.smithi039.stdout: ms die on skipped message = False 2024-09-21T07:32:02.098 INFO:teuthology.orchestra.run.smithi039.stdout: fsid = 7ab9b04a-f88d-49aa-a493-e5c076637ffb 2024-09-21T07:32:02.098 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:32:02.098 INFO:teuthology.orchestra.run.smithi039.stdout:[osd] 2024-09-21T07:32:02.099 INFO:teuthology.orchestra.run.smithi039.stdout: osd journal size = 100 2024-09-21T07:32:02.099 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:32:02.099 INFO:teuthology.orchestra.run.smithi039.stdout: osd scrub load threshold = 5.0 2024-09-21T07:32:02.099 INFO:teuthology.orchestra.run.smithi039.stdout: osd scrub max interval = 600 2024-09-21T07:32:02.099 INFO:teuthology.orchestra.run.smithi039.stdout: osd mclock profile = high_recovery_ops 2024-09-21T07:32:02.099 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:32:02.099 INFO:teuthology.orchestra.run.smithi039.stdout: osd recover clone overlap = true 2024-09-21T07:32:02.099 INFO:teuthology.orchestra.run.smithi039.stdout: osd recovery max chunk = 1048576 2024-09-21T07:32:02.099 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:32:02.099 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug shutdown = true 2024-09-21T07:32:02.099 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug op order = true 2024-09-21T07:32:02.100 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug verify stray on activate = true 2024-09-21T07:32:02.100 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug trim objects = true 2024-09-21T07:32:02.100 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:32:02.100 INFO:teuthology.orchestra.run.smithi039.stdout: osd open classes on start = true 2024-09-21T07:32:02.100 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug pg log writeout = true 2024-09-21T07:32:02.100 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:32:02.100 INFO:teuthology.orchestra.run.smithi039.stdout: osd deep scrub update digest min age = 30 2024-09-21T07:32:02.100 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:32:02.100 INFO:teuthology.orchestra.run.smithi039.stdout: osd map max advance = 10 2024-09-21T07:32:02.100 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:32:02.100 INFO:teuthology.orchestra.run.smithi039.stdout: journal zero on create = true 2024-09-21T07:32:02.100 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:32:02.101 INFO:teuthology.orchestra.run.smithi039.stdout: filestore ondisk finisher threads = 3 2024-09-21T07:32:02.101 INFO:teuthology.orchestra.run.smithi039.stdout: filestore apply finisher threads = 3 2024-09-21T07:32:02.101 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:32:02.101 INFO:teuthology.orchestra.run.smithi039.stdout: bdev debug aio = true 2024-09-21T07:32:02.101 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug misdirected ops = true 2024-09-21T07:32:02.101 INFO:teuthology.orchestra.run.smithi039.stdout: bdev async discard = True 2024-09-21T07:32:02.101 INFO:teuthology.orchestra.run.smithi039.stdout: bdev enable discard = True 2024-09-21T07:32:02.101 INFO:teuthology.orchestra.run.smithi039.stdout: bluestore allocator = bitmap 2024-09-21T07:32:02.101 INFO:teuthology.orchestra.run.smithi039.stdout: bluestore block size = 96636764160 2024-09-21T07:32:02.101 INFO:teuthology.orchestra.run.smithi039.stdout: bluestore fsck on mount = True 2024-09-21T07:32:02.102 INFO:teuthology.orchestra.run.smithi039.stdout: debug bluefs = 1/20 2024-09-21T07:32:02.102 INFO:teuthology.orchestra.run.smithi039.stdout: debug bluestore = 1/20 2024-09-21T07:32:02.102 INFO:teuthology.orchestra.run.smithi039.stdout: debug ms = 1 2024-09-21T07:32:02.102 INFO:teuthology.orchestra.run.smithi039.stdout: debug osd = 20 2024-09-21T07:32:02.102 INFO:teuthology.orchestra.run.smithi039.stdout: debug rocksdb = 4/10 2024-09-21T07:32:02.102 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T07:32:02.102 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd full ratio = 0.9 2024-09-21T07:32:02.102 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd nearfull ratio = 0.8 2024-09-21T07:32:02.102 INFO:teuthology.orchestra.run.smithi039.stdout: osd failsafe full ratio = 0.95 2024-09-21T07:32:02.103 INFO:teuthology.orchestra.run.smithi039.stdout: osd objectstore = bluestore 2024-09-21T07:32:02.103 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:32:02.103 INFO:teuthology.orchestra.run.smithi039.stdout:[mgr] 2024-09-21T07:32:02.103 INFO:teuthology.orchestra.run.smithi039.stdout: debug ms = 1 2024-09-21T07:32:02.103 INFO:teuthology.orchestra.run.smithi039.stdout: debug mgr = 20 2024-09-21T07:32:02.103 INFO:teuthology.orchestra.run.smithi039.stdout: debug mon = 20 2024-09-21T07:32:02.103 INFO:teuthology.orchestra.run.smithi039.stdout: debug auth = 20 2024-09-21T07:32:02.103 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min pgs per osd = 4 2024-09-21T07:32:02.103 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min bytes per osd = 10 2024-09-21T07:32:02.103 INFO:teuthology.orchestra.run.smithi039.stdout: mgr/telemetry/nag = false 2024-09-21T07:32:02.104 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:32:02.104 INFO:teuthology.orchestra.run.smithi039.stdout:[mon] 2024-09-21T07:32:02.104 INFO:teuthology.orchestra.run.smithi039.stdout: debug ms = 1 2024-09-21T07:32:02.104 INFO:teuthology.orchestra.run.smithi039.stdout: debug mon = 20 2024-09-21T07:32:02.104 INFO:teuthology.orchestra.run.smithi039.stdout: debug paxos = 20 2024-09-21T07:32:02.104 INFO:teuthology.orchestra.run.smithi039.stdout: debug auth = 20 2024-09-21T07:32:02.104 INFO:teuthology.orchestra.run.smithi039.stdout: mon data avail warn = 5 2024-09-21T07:32:02.104 INFO:teuthology.orchestra.run.smithi039.stdout: mon mgr mkfs grace = 240 2024-09-21T07:32:02.104 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min pgs per osd = 4 2024-09-21T07:32:02.104 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd reporter subtree level = osd 2024-09-21T07:32:02.104 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd prime pg temp = true 2024-09-21T07:32:02.104 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min bytes per osd = 10 2024-09-21T07:32:02.105 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:32:02.105 INFO:teuthology.orchestra.run.smithi039.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T07:32:02.105 INFO:teuthology.orchestra.run.smithi039.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T07:32:02.105 INFO:teuthology.orchestra.run.smithi039.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T07:32:02.105 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:32:02.105 INFO:teuthology.orchestra.run.smithi039.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T07:32:02.105 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T07:32:02.105 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T07:32:02.105 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:32:02.105 INFO:teuthology.orchestra.run.smithi039.stdout: # 1m isn't quite enough 2024-09-21T07:32:02.105 INFO:teuthology.orchestra.run.smithi039.stdout: mon_down_mkfs_grace = 2m 2024-09-21T07:32:02.105 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:32:02.106 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_filestore_osds = false 2024-09-21T07:32:02.106 INFO:teuthology.orchestra.run.smithi039.stdout: mon down mkfs grace = 300 2024-09-21T07:32:02.106 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:32:02.106 INFO:teuthology.orchestra.run.smithi039.stdout:[client] 2024-09-21T07:32:02.106 INFO:teuthology.orchestra.run.smithi039.stdout: rgw cache enabled = true 2024-09-21T07:32:02.106 INFO:teuthology.orchestra.run.smithi039.stdout: rgw enable ops log = true 2024-09-21T07:32:02.106 INFO:teuthology.orchestra.run.smithi039.stdout: rgw enable usage log = true 2024-09-21T07:32:02.106 INFO:teuthology.orchestra.run.smithi039.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T07:32:02.106 INFO:teuthology.orchestra.run.smithi039.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T07:32:02.106 INFO:teuthology.orchestra.run.smithi039.stdout: pid file = "" 2024-09-21T07:32:02.106 INFO:teuthology.orchestra.run.smithi039.stdout:[mon.a] 2024-09-21T07:32:02.106 INFO:teuthology.orchestra.run.smithi039.stdout:[mon.b] 2024-09-21T07:32:02.106 INFO:teuthology.orchestra.run.smithi039.stdout:[mon.c] 2024-09-21T07:32:02.107 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.a] 2024-09-21T07:32:02.107 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.c] 2024-09-21T07:32:02.107 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.b] 2024-09-21T07:32:02.107 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.d] 2024-09-21T07:32:02.107 INFO:teuthology.orchestra.run.smithi148.stdout:[global] 2024-09-21T07:32:02.107 INFO:teuthology.orchestra.run.smithi148.stdout: chdir = "" 2024-09-21T07:32:02.107 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T07:32:02.108 INFO:teuthology.orchestra.run.smithi148.stdout: auth supported = cephx 2024-09-21T07:32:02.108 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:32:02.108 INFO:teuthology.orchestra.run.smithi148.stdout: filestore xattr use omap = true 2024-09-21T07:32:02.108 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:32:02.108 INFO:teuthology.orchestra.run.smithi148.stdout: mon clock drift allowed = 1.000 2024-09-21T07:32:02.108 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:32:02.108 INFO:teuthology.orchestra.run.smithi148.stdout: osd crush chooseleaf type = 0 2024-09-21T07:32:02.108 INFO:teuthology.orchestra.run.smithi148.stdout: auth debug = true 2024-09-21T07:32:02.108 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:32:02.108 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on old message = true 2024-09-21T07:32:02.108 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on bug = true 2024-09-21T07:32:02.108 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:32:02.108 INFO:teuthology.orchestra.run.smithi148.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T07:32:02.109 INFO:teuthology.orchestra.run.smithi148.stdout: mon pg warn max object skew = 0 2024-09-21T07:32:02.109 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:32:02.109 INFO:teuthology.orchestra.run.smithi148.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T07:32:02.109 INFO:teuthology.orchestra.run.smithi148.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T07:32:02.109 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:32:02.109 INFO:teuthology.orchestra.run.smithi148.stdout: osd pool default size = 2 2024-09-21T07:32:02.109 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:32:02.109 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd allow primary affinity = true 2024-09-21T07:32:02.109 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd allow pg remap = true 2024-09-21T07:32:02.109 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on legacy crush tunables = false 2024-09-21T07:32:02.109 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on crush straw calc version zero = false 2024-09-21T07:32:02.109 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on no sortbitwise = false 2024-09-21T07:32:02.110 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on osd down out interval zero = false 2024-09-21T07:32:02.110 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on too few osds = false 2024-09-21T07:32:02.110 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T07:32:02.110 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T07:32:02.110 INFO:teuthology.orchestra.run.smithi148.stdout: mon_allow_pool_size_one = true 2024-09-21T07:32:02.110 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:32:02.110 INFO:teuthology.orchestra.run.smithi148.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T07:32:02.110 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:32:02.110 INFO:teuthology.orchestra.run.smithi148.stdout: osd default data pool replay window = 5 2024-09-21T07:32:02.110 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:32:02.110 INFO:teuthology.orchestra.run.smithi148.stdout: mon allow pool delete = true 2024-09-21T07:32:02.110 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:32:02.111 INFO:teuthology.orchestra.run.smithi148.stdout: mon cluster log file level = debug 2024-09-21T07:32:02.111 INFO:teuthology.orchestra.run.smithi148.stdout: debug asserts on shutdown = true 2024-09-21T07:32:02.111 INFO:teuthology.orchestra.run.smithi148.stdout: mon health detail to clog = false 2024-09-21T07:32:02.111 INFO:teuthology.orchestra.run.smithi148.stdout: mon host = "172.21.15.39,172.21.15.148,[v2:172.21.15.148:3301,v1:172.21.15.148:6790]" 2024-09-21T07:32:02.111 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on skipped message = False 2024-09-21T07:32:02.111 INFO:teuthology.orchestra.run.smithi148.stdout: fsid = 7ab9b04a-f88d-49aa-a493-e5c076637ffb 2024-09-21T07:32:02.111 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:32:02.111 INFO:teuthology.orchestra.run.smithi148.stdout:[osd] 2024-09-21T07:32:02.111 INFO:teuthology.orchestra.run.smithi148.stdout: osd journal size = 100 2024-09-21T07:32:02.111 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:32:02.111 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub load threshold = 5.0 2024-09-21T07:32:02.111 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub max interval = 600 2024-09-21T07:32:02.111 INFO:teuthology.orchestra.run.smithi148.stdout: osd mclock profile = high_recovery_ops 2024-09-21T07:32:02.112 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:32:02.112 INFO:teuthology.orchestra.run.smithi148.stdout: osd recover clone overlap = true 2024-09-21T07:32:02.112 INFO:teuthology.orchestra.run.smithi148.stdout: osd recovery max chunk = 1048576 2024-09-21T07:32:02.112 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:32:02.112 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug shutdown = true 2024-09-21T07:32:02.112 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug op order = true 2024-09-21T07:32:02.112 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug verify stray on activate = true 2024-09-21T07:32:02.112 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug trim objects = true 2024-09-21T07:32:02.112 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:32:02.112 INFO:teuthology.orchestra.run.smithi148.stdout: osd open classes on start = true 2024-09-21T07:32:02.112 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug pg log writeout = true 2024-09-21T07:32:02.112 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:32:02.112 INFO:teuthology.orchestra.run.smithi148.stdout: osd deep scrub update digest min age = 30 2024-09-21T07:32:02.113 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:32:02.113 INFO:teuthology.orchestra.run.smithi148.stdout: osd map max advance = 10 2024-09-21T07:32:02.113 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:32:02.113 INFO:teuthology.orchestra.run.smithi148.stdout: journal zero on create = true 2024-09-21T07:32:02.113 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:32:02.113 INFO:teuthology.orchestra.run.smithi148.stdout: filestore ondisk finisher threads = 3 2024-09-21T07:32:02.113 INFO:teuthology.orchestra.run.smithi148.stdout: filestore apply finisher threads = 3 2024-09-21T07:32:02.113 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:32:02.113 INFO:teuthology.orchestra.run.smithi148.stdout: bdev debug aio = true 2024-09-21T07:32:02.113 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug misdirected ops = true 2024-09-21T07:32:02.113 INFO:teuthology.orchestra.run.smithi148.stdout: bdev async discard = True 2024-09-21T07:32:02.113 INFO:teuthology.orchestra.run.smithi148.stdout: bdev enable discard = True 2024-09-21T07:32:02.114 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore allocator = bitmap 2024-09-21T07:32:02.114 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore block size = 96636764160 2024-09-21T07:32:02.114 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore fsck on mount = True 2024-09-21T07:32:02.114 INFO:teuthology.orchestra.run.smithi148.stdout: debug bluefs = 1/20 2024-09-21T07:32:02.114 INFO:teuthology.orchestra.run.smithi148.stdout: debug bluestore = 1/20 2024-09-21T07:32:02.114 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-09-21T07:32:02.114 INFO:teuthology.orchestra.run.smithi148.stdout: debug osd = 20 2024-09-21T07:32:02.114 INFO:teuthology.orchestra.run.smithi148.stdout: debug rocksdb = 4/10 2024-09-21T07:32:02.114 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T07:32:02.114 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd full ratio = 0.9 2024-09-21T07:32:02.114 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd nearfull ratio = 0.8 2024-09-21T07:32:02.114 INFO:teuthology.orchestra.run.smithi148.stdout: osd failsafe full ratio = 0.95 2024-09-21T07:32:02.115 INFO:teuthology.orchestra.run.smithi148.stdout: osd objectstore = bluestore 2024-09-21T07:32:02.115 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:32:02.115 INFO:teuthology.orchestra.run.smithi148.stdout:[mgr] 2024-09-21T07:32:02.115 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-09-21T07:32:02.115 INFO:teuthology.orchestra.run.smithi148.stdout: debug mgr = 20 2024-09-21T07:32:02.115 INFO:teuthology.orchestra.run.smithi148.stdout: debug mon = 20 2024-09-21T07:32:02.115 INFO:teuthology.orchestra.run.smithi148.stdout: debug auth = 20 2024-09-21T07:32:02.115 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min pgs per osd = 4 2024-09-21T07:32:02.115 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min bytes per osd = 10 2024-09-21T07:32:02.115 INFO:teuthology.orchestra.run.smithi148.stdout: mgr/telemetry/nag = false 2024-09-21T07:32:02.115 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:32:02.115 INFO:teuthology.orchestra.run.smithi148.stdout:[mon] 2024-09-21T07:32:02.115 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-09-21T07:32:02.116 INFO:teuthology.orchestra.run.smithi148.stdout: debug mon = 20 2024-09-21T07:32:02.116 INFO:teuthology.orchestra.run.smithi148.stdout: debug paxos = 20 2024-09-21T07:32:02.116 INFO:teuthology.orchestra.run.smithi148.stdout: debug auth = 20 2024-09-21T07:32:02.116 INFO:teuthology.orchestra.run.smithi148.stdout: mon data avail warn = 5 2024-09-21T07:32:02.116 INFO:teuthology.orchestra.run.smithi148.stdout: mon mgr mkfs grace = 240 2024-09-21T07:32:02.116 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min pgs per osd = 4 2024-09-21T07:32:02.116 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd reporter subtree level = osd 2024-09-21T07:32:02.116 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd prime pg temp = true 2024-09-21T07:32:02.116 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min bytes per osd = 10 2024-09-21T07:32:02.116 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:32:02.116 INFO:teuthology.orchestra.run.smithi148.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T07:32:02.116 INFO:teuthology.orchestra.run.smithi148.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T07:32:02.116 INFO:teuthology.orchestra.run.smithi148.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T07:32:02.117 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:32:02.117 INFO:teuthology.orchestra.run.smithi148.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T07:32:02.117 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T07:32:02.117 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T07:32:02.117 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:32:02.117 INFO:teuthology.orchestra.run.smithi148.stdout: # 1m isn't quite enough 2024-09-21T07:32:02.117 INFO:teuthology.orchestra.run.smithi148.stdout: mon_down_mkfs_grace = 2m 2024-09-21T07:32:02.117 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:32:02.117 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_filestore_osds = false 2024-09-21T07:32:02.117 INFO:teuthology.orchestra.run.smithi148.stdout: mon down mkfs grace = 300 2024-09-21T07:32:02.117 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:32:02.117 INFO:teuthology.orchestra.run.smithi148.stdout:[client] 2024-09-21T07:32:02.117 INFO:teuthology.orchestra.run.smithi148.stdout: rgw cache enabled = true 2024-09-21T07:32:02.118 INFO:teuthology.orchestra.run.smithi148.stdout: rgw enable ops log = true 2024-09-21T07:32:02.118 INFO:teuthology.orchestra.run.smithi148.stdout: rgw enable usage log = true 2024-09-21T07:32:02.118 INFO:teuthology.orchestra.run.smithi148.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T07:32:02.118 INFO:teuthology.orchestra.run.smithi148.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T07:32:02.118 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = "" 2024-09-21T07:32:02.118 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.a] 2024-09-21T07:32:02.118 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.b] 2024-09-21T07:32:02.118 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.c] 2024-09-21T07:32:02.118 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.a] 2024-09-21T07:32:02.118 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.c] 2024-09-21T07:32:02.118 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.b] 2024-09-21T07:32:02.118 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.d] 2024-09-21T07:32:02.131 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-21T07:32:02.131 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-21T07:32:02.300 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-21T07:32:02.300 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:32:02.300 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-21T07:32:02.335 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:32:02.335 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-21T07:32:02.409 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T07:32:02.409 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:32:02.409 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T07:32:02.409 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T07:32:02.527 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:32:02.527 DEBUG:teuthology.orchestra.run.smithi039:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:32:02.602 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi148.front.sepia.ceph.com 2024-09-21T07:32:02.602 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:32:02.603 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T07:32:02.603 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T07:32:02.681 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:32:02.681 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:32:02.756 INFO:tasks.ceph:Setting up mon nodes... 2024-09-21T07:32:02.756 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-21T07:32:02.757 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-21T07:32:02.925 INFO:teuthology.orchestra.run.smithi039.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-21T07:32:02.937 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-21T07:32:03.110 INFO:teuthology.orchestra.run.smithi148.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-21T07:32:03.124 INFO:tasks.ceph:Setting up mds nodes... 2024-09-21T07:32:03.124 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-21T07:32:03.294 INFO:teuthology.orchestra.run.smithi039.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-21T07:32:03.305 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-21T07:32:03.361 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-21T07:32:03.528 INFO:teuthology.orchestra.run.smithi039.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-21T07:32:03.540 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-21T07:32:03.595 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-21T07:32:03.766 INFO:teuthology.orchestra.run.smithi148.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-21T07:32:03.777 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-21T07:32:03.834 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-21T07:32:04.000 INFO:teuthology.orchestra.run.smithi148.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-21T07:32:04.012 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-21T07:32:04.068 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-21T07:32:04.068 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T07:32:04.218 INFO:teuthology.orchestra.run.smithi039.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-21T07:32:04.251 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-21T07:32:04.397 INFO:teuthology.orchestra.run.smithi039.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-21T07:32:04.430 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-21T07:32:04.573 INFO:teuthology.orchestra.run.smithi148.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-21T07:32:04.606 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-21T07:32:04.750 INFO:teuthology.orchestra.run.smithi148.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-21T07:32:04.783 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-21T07:32:04.783 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi039.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi148.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-21T07:32:04.783 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-21T07:32:04.840 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T07:32:04.840 INFO:tasks.ceph:role: osd.0 2024-09-21T07:32:04.840 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T07:32:04.840 DEBUG:teuthology.orchestra.run.smithi039:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-21T07:32:04.922 INFO:teuthology.orchestra.run.smithi039.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T07:32:04.923 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T07:32:04.923 INFO:teuthology.orchestra.run.smithi039.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T07:32:04.923 INFO:teuthology.orchestra.run.smithi039.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T07:32:04.923 INFO:teuthology.orchestra.run.smithi039.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T07:32:04.923 INFO:teuthology.orchestra.run.smithi039.stdout: = sunit=0 swidth=0 blks 2024-09-21T07:32:04.923 INFO:teuthology.orchestra.run.smithi039.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T07:32:04.923 INFO:teuthology.orchestra.run.smithi039.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T07:32:04.923 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T07:32:04.923 INFO:teuthology.orchestra.run.smithi039.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T07:32:06.194 INFO:teuthology.orchestra.run.smithi039.stdout:Discarding blocks...Done. 2024-09-21T07:32:06.198 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi039.front.sepia.ceph.com -o noatime 2024-09-21T07:32:06.198 DEBUG:teuthology.orchestra.run.smithi039:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-21T07:32:06.285 DEBUG:teuthology.orchestra.run.smithi039:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-21T07:32:06.349 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-21T07:32:06.405 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T07:32:06.405 INFO:tasks.ceph:role: osd.1 2024-09-21T07:32:06.405 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T07:32:06.405 DEBUG:teuthology.orchestra.run.smithi039:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-21T07:32:06.490 INFO:teuthology.orchestra.run.smithi039.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T07:32:06.490 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T07:32:06.490 INFO:teuthology.orchestra.run.smithi039.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T07:32:06.490 INFO:teuthology.orchestra.run.smithi039.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T07:32:06.491 INFO:teuthology.orchestra.run.smithi039.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T07:32:06.491 INFO:teuthology.orchestra.run.smithi039.stdout: = sunit=0 swidth=0 blks 2024-09-21T07:32:06.491 INFO:teuthology.orchestra.run.smithi039.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T07:32:06.491 INFO:teuthology.orchestra.run.smithi039.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T07:32:06.491 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T07:32:06.491 INFO:teuthology.orchestra.run.smithi039.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T07:32:07.763 INFO:teuthology.orchestra.run.smithi039.stdout:Discarding blocks...Done. 2024-09-21T07:32:07.768 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi039.front.sepia.ceph.com -o noatime 2024-09-21T07:32:07.768 DEBUG:teuthology.orchestra.run.smithi039:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-21T07:32:07.856 DEBUG:teuthology.orchestra.run.smithi039:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-21T07:32:07.920 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-21T07:32:07.976 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T07:32:07.976 INFO:tasks.ceph:role: osd.2 2024-09-21T07:32:07.976 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T07:32:07.977 DEBUG:teuthology.orchestra.run.smithi039:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-21T07:32:08.059 INFO:teuthology.orchestra.run.smithi039.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T07:32:08.060 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T07:32:08.060 INFO:teuthology.orchestra.run.smithi039.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T07:32:08.060 INFO:teuthology.orchestra.run.smithi039.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T07:32:08.060 INFO:teuthology.orchestra.run.smithi039.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T07:32:08.060 INFO:teuthology.orchestra.run.smithi039.stdout: = sunit=0 swidth=0 blks 2024-09-21T07:32:08.060 INFO:teuthology.orchestra.run.smithi039.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T07:32:08.060 INFO:teuthology.orchestra.run.smithi039.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T07:32:08.060 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T07:32:08.060 INFO:teuthology.orchestra.run.smithi039.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T07:32:09.347 INFO:teuthology.orchestra.run.smithi039.stdout:Discarding blocks...Done. 2024-09-21T07:32:09.351 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi039.front.sepia.ceph.com -o noatime 2024-09-21T07:32:09.351 DEBUG:teuthology.orchestra.run.smithi039:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-21T07:32:09.436 DEBUG:teuthology.orchestra.run.smithi039:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-21T07:32:09.499 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-21T07:32:09.555 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T07:32:09.555 INFO:tasks.ceph:role: osd.3 2024-09-21T07:32:09.555 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T07:32:09.555 DEBUG:teuthology.orchestra.run.smithi039:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-21T07:32:09.638 INFO:teuthology.orchestra.run.smithi039.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T07:32:09.639 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T07:32:09.639 INFO:teuthology.orchestra.run.smithi039.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T07:32:09.639 INFO:teuthology.orchestra.run.smithi039.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T07:32:09.639 INFO:teuthology.orchestra.run.smithi039.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T07:32:09.639 INFO:teuthology.orchestra.run.smithi039.stdout: = sunit=0 swidth=0 blks 2024-09-21T07:32:09.639 INFO:teuthology.orchestra.run.smithi039.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T07:32:09.639 INFO:teuthology.orchestra.run.smithi039.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T07:32:09.639 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T07:32:09.639 INFO:teuthology.orchestra.run.smithi039.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T07:32:10.911 INFO:teuthology.orchestra.run.smithi039.stdout:Discarding blocks...Done. 2024-09-21T07:32:10.915 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi039.front.sepia.ceph.com -o noatime 2024-09-21T07:32:10.916 DEBUG:teuthology.orchestra.run.smithi039:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-21T07:32:11.006 DEBUG:teuthology.orchestra.run.smithi039:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-21T07:32:11.070 DEBUG:teuthology.orchestra.run.smithi039:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:32:11.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:11.238+0000 7f721fa1d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-21T07:32:11.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:11.239+0000 7f721fa1d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-21T07:32:11.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:11.239+0000 7f721fa1d640 -1 bdev(0x55d81b179900 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-21T07:32:11.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:11.239+0000 7f721fa1d640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-21T07:32:14.766 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-21T07:32:14.824 DEBUG:teuthology.orchestra.run.smithi039:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:32:14.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:14.981+0000 7f7f869b9640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-21T07:32:14.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:14.982+0000 7f7f869b9640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-21T07:32:14.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:14.982+0000 7f7f869b9640 -1 bdev(0x5593466a9900 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-21T07:32:14.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:14.982+0000 7f7f869b9640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-21T07:32:18.034 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-21T07:32:18.089 DEBUG:teuthology.orchestra.run.smithi039:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:32:18.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:18.246+0000 7f8695c28640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-21T07:32:18.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:18.246+0000 7f8695c28640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-21T07:32:18.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:18.246+0000 7f8695c28640 -1 bdev(0x55bd1c455900 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-21T07:32:18.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:18.247+0000 7f8695c28640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-21T07:32:21.307 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-21T07:32:21.366 DEBUG:teuthology.orchestra.run.smithi039:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:32:21.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:21.522+0000 7f60ead4b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-21T07:32:21.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:21.522+0000 7f60ead4b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-21T07:32:21.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:21.523+0000 7f60ead4b640 -1 bdev(0x5588f3591900 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-21T07:32:21.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:21.523+0000 7f60ead4b640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-21T07:32:24.554 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-21T07:32:24.612 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-21T07:32:24.674 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T07:32:24.674 INFO:tasks.ceph:role: osd.4 2024-09-21T07:32:24.675 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi148.front.sepia.ceph.com 2024-09-21T07:32:24.675 DEBUG:teuthology.orchestra.run.smithi148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-21T07:32:24.740 INFO:teuthology.orchestra.run.smithi148.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T07:32:24.740 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T07:32:24.740 INFO:teuthology.orchestra.run.smithi148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T07:32:24.740 INFO:teuthology.orchestra.run.smithi148.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T07:32:24.740 INFO:teuthology.orchestra.run.smithi148.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T07:32:24.740 INFO:teuthology.orchestra.run.smithi148.stdout: = sunit=0 swidth=0 blks 2024-09-21T07:32:24.740 INFO:teuthology.orchestra.run.smithi148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T07:32:24.740 INFO:teuthology.orchestra.run.smithi148.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T07:32:24.740 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T07:32:24.741 INFO:teuthology.orchestra.run.smithi148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T07:32:25.154 INFO:teuthology.orchestra.run.smithi148.stdout:Discarding blocks...Done. 2024-09-21T07:32:25.158 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi148.front.sepia.ceph.com -o noatime 2024-09-21T07:32:25.158 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-21T07:32:25.287 DEBUG:teuthology.orchestra.run.smithi148:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-21T07:32:25.355 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-21T07:32:25.410 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T07:32:25.410 INFO:tasks.ceph:role: osd.5 2024-09-21T07:32:25.411 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi148.front.sepia.ceph.com 2024-09-21T07:32:25.411 DEBUG:teuthology.orchestra.run.smithi148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-21T07:32:25.474 INFO:teuthology.orchestra.run.smithi148.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T07:32:25.475 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T07:32:25.475 INFO:teuthology.orchestra.run.smithi148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T07:32:25.475 INFO:teuthology.orchestra.run.smithi148.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T07:32:25.475 INFO:teuthology.orchestra.run.smithi148.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T07:32:25.475 INFO:teuthology.orchestra.run.smithi148.stdout: = sunit=0 swidth=0 blks 2024-09-21T07:32:25.475 INFO:teuthology.orchestra.run.smithi148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T07:32:25.475 INFO:teuthology.orchestra.run.smithi148.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T07:32:25.475 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T07:32:25.475 INFO:teuthology.orchestra.run.smithi148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T07:32:25.831 INFO:teuthology.orchestra.run.smithi148.stdout:Discarding blocks...Done. 2024-09-21T07:32:25.835 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi148.front.sepia.ceph.com -o noatime 2024-09-21T07:32:25.835 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-21T07:32:25.920 DEBUG:teuthology.orchestra.run.smithi148:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-21T07:32:25.982 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-21T07:32:26.046 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T07:32:26.046 INFO:tasks.ceph:role: osd.6 2024-09-21T07:32:26.046 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi148.front.sepia.ceph.com 2024-09-21T07:32:26.046 DEBUG:teuthology.orchestra.run.smithi148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-21T07:32:26.110 INFO:teuthology.orchestra.run.smithi148.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T07:32:26.110 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T07:32:26.110 INFO:teuthology.orchestra.run.smithi148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T07:32:26.110 INFO:teuthology.orchestra.run.smithi148.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T07:32:26.110 INFO:teuthology.orchestra.run.smithi148.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T07:32:26.110 INFO:teuthology.orchestra.run.smithi148.stdout: = sunit=0 swidth=0 blks 2024-09-21T07:32:26.110 INFO:teuthology.orchestra.run.smithi148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T07:32:26.111 INFO:teuthology.orchestra.run.smithi148.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T07:32:26.111 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T07:32:26.111 INFO:teuthology.orchestra.run.smithi148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T07:32:26.496 INFO:teuthology.orchestra.run.smithi148.stdout:Discarding blocks...Done. 2024-09-21T07:32:26.499 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi148.front.sepia.ceph.com -o noatime 2024-09-21T07:32:26.500 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-21T07:32:26.600 DEBUG:teuthology.orchestra.run.smithi148:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-21T07:32:26.662 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-21T07:32:26.717 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T07:32:26.717 INFO:tasks.ceph:role: osd.7 2024-09-21T07:32:26.717 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi148.front.sepia.ceph.com 2024-09-21T07:32:26.718 DEBUG:teuthology.orchestra.run.smithi148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-21T07:32:26.780 INFO:teuthology.orchestra.run.smithi148.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T07:32:26.780 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T07:32:26.781 INFO:teuthology.orchestra.run.smithi148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T07:32:26.781 INFO:teuthology.orchestra.run.smithi148.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T07:32:26.781 INFO:teuthology.orchestra.run.smithi148.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T07:32:26.781 INFO:teuthology.orchestra.run.smithi148.stdout: = sunit=0 swidth=0 blks 2024-09-21T07:32:26.781 INFO:teuthology.orchestra.run.smithi148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T07:32:26.781 INFO:teuthology.orchestra.run.smithi148.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T07:32:26.781 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T07:32:26.781 INFO:teuthology.orchestra.run.smithi148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T07:32:27.201 INFO:teuthology.orchestra.run.smithi148.stdout:Discarding blocks...Done. 2024-09-21T07:32:27.206 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi148.front.sepia.ceph.com -o noatime 2024-09-21T07:32:27.206 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-21T07:32:27.340 DEBUG:teuthology.orchestra.run.smithi148:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-21T07:32:27.404 DEBUG:teuthology.orchestra.run.smithi148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:32:27.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:27.573+0000 7f005fc0b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-21T07:32:27.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:27.578+0000 7f005fc0b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-21T07:32:27.580 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:27.578+0000 7f005fc0b640 -1 bdev(0x55c3c5143900 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-21T07:32:27.581 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:27.578+0000 7f005fc0b640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-21T07:32:29.425 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:32:29.430 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:32:30.643 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-21T07:32:30.703 DEBUG:teuthology.orchestra.run.smithi148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:32:30.858 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:30.856+0000 7f5e2cfa8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-21T07:32:30.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:30.857+0000 7f5e2cfa8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-21T07:32:30.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:30.857+0000 7f5e2cfa8640 -1 bdev(0x56059235b900 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-21T07:32:30.859 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:30.857+0000 7f5e2cfa8640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-21T07:32:33.916 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-21T07:32:33.974 DEBUG:teuthology.orchestra.run.smithi148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:32:34.132 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:34.130+0000 7f92cccdc640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-21T07:32:34.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:34.131+0000 7f92cccdc640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-21T07:32:34.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:34.131+0000 7f92cccdc640 -1 bdev(0x556c7fd0d900 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-21T07:32:34.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:34.131+0000 7f92cccdc640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-21T07:32:37.140 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-21T07:32:37.196 DEBUG:teuthology.orchestra.run.smithi148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:32:37.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:37.348+0000 7fd020418640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-21T07:32:37.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:37.348+0000 7fd020418640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-21T07:32:37.350 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:37.348+0000 7fd020418640 -1 bdev(0x5592256d9900 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-21T07:32:37.351 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:37.349+0000 7fd020418640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-21T07:32:40.388 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-21T07:32:40.446 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-21T07:32:40.446 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:32:40.446 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-21T07:32:40.509 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:32:40.509 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-21T07:32:40.608 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:32:40.608 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-21T07:32:40.704 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:32:40.704 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-21T07:32:40.801 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:32:40.801 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-21T07:32:40.896 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:32:40.896 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-21T07:32:40.992 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:32:40.992 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-21T07:32:41.087 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:32:41.114 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-21T07:32:41.175 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:32:41.175 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-21T07:32:41.274 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:32:41.274 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-21T07:32:41.370 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:32:41.370 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-21T07:32:41.465 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:32:41.466 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-21T07:32:41.561 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:32:41.561 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-21T07:32:41.657 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:32:41.657 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-21T07:32:41.752 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:32:41.752 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-21T07:32:41.788 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:32:41.788 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-21T07:32:41.864 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:32:41.864 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-21T07:32:41.899 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:32:41.899 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-21T07:32:41.973 INFO:tasks.ceph:Adding keys to all mons... 2024-09-21T07:32:41.973 DEBUG:teuthology.orchestra.run.smithi039:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-21T07:32:41.978 DEBUG:teuthology.orchestra.run.smithi148:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-21T07:32:42.026 INFO:teuthology.orchestra.run.smithi039.stdout:[mgr.y] 2024-09-21T07:32:42.026 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQBydu5mr+8aNxAA7TFYNms2WsXUfa4ITTJOHw== 2024-09-21T07:32:42.027 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.a] 2024-09-21T07:32:42.027 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQBzdu5m+P6EERAABQqmlkrocSi1+hwc73Adrg== 2024-09-21T07:32:42.027 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.c] 2024-09-21T07:32:42.027 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQBzdu5msbx6HxAALsF8ORvIoJpIAFZ5uxmOEw== 2024-09-21T07:32:42.027 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.0] 2024-09-21T07:32:42.027 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQB7du5msw81DhAAmVimqN4x9K3YpT0nbovppQ== 2024-09-21T07:32:42.027 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.1] 2024-09-21T07:32:42.027 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQB+du5mdnCIOhAAtVB4ThehQwb0zRriyKcaxA== 2024-09-21T07:32:42.027 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.2] 2024-09-21T07:32:42.027 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQCCdu5mTfWxDhAAadJCbPN5xBx/f9L0v2yTdg== 2024-09-21T07:32:42.027 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.3] 2024-09-21T07:32:42.027 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQCFdu5mL1E2HxAAyOhY97uUXEw5L1CcDeONVA== 2024-09-21T07:32:42.027 INFO:teuthology.orchestra.run.smithi039.stdout:[mgr.x] 2024-09-21T07:32:42.027 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQBzdu5mA2GRBhAAkyCKdgH6laHy8xNt080dXg== 2024-09-21T07:32:42.028 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.b] 2024-09-21T07:32:42.028 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQBzdu5m5/GeLRAAfE8fr5dbitQa89rNK9pZ2Q== 2024-09-21T07:32:42.028 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.d] 2024-09-21T07:32:42.028 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQB0du5mIrcBABAApKAB/7FkZ5glNTxlbkIsBw== 2024-09-21T07:32:42.028 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.4] 2024-09-21T07:32:42.028 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQCLdu5m6PY/IhAAgvcTgm+6uWA7lIG4tcTIEg== 2024-09-21T07:32:42.028 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.5] 2024-09-21T07:32:42.028 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQCOdu5mE/wlMxAA3JBg4xH+N/PPC1Uv2ZVrYQ== 2024-09-21T07:32:42.028 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.6] 2024-09-21T07:32:42.028 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQCSdu5mC3TgBxAAwsArjeIdFcApY3/1OJffEg== 2024-09-21T07:32:42.028 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.7] 2024-09-21T07:32:42.028 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQCVdu5mSWHWFBAA9qPHWRdsvrB74inrp1lj4Q== 2024-09-21T07:32:42.028 INFO:teuthology.orchestra.run.smithi039.stdout:[client.0] 2024-09-21T07:32:42.028 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQB0du5mGfb1DBAA/aSMKR5or3xy53rB/iUjpA== 2024-09-21T07:32:42.028 INFO:teuthology.orchestra.run.smithi039.stdout:[client.2] 2024-09-21T07:32:42.029 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQB0du5mSu2oFxAAgjy2m/wWKh6xaoh+YDoU1Q== 2024-09-21T07:32:42.029 INFO:teuthology.orchestra.run.smithi039.stdout:[client.1] 2024-09-21T07:32:42.029 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQB0du5mEkciIhAAid+bxJEr4P7vMnZ42PAy5w== 2024-09-21T07:32:42.029 INFO:teuthology.orchestra.run.smithi039.stdout:[client.3] 2024-09-21T07:32:42.029 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQB0du5mCHy0LBAAZneF2jUWgfuyGehZFbOxDg== 2024-09-21T07:32:42.065 INFO:teuthology.orchestra.run.smithi148.stdout:[mgr.y] 2024-09-21T07:32:42.066 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQBydu5mr+8aNxAA7TFYNms2WsXUfa4ITTJOHw== 2024-09-21T07:32:42.066 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.a] 2024-09-21T07:32:42.066 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQBzdu5m+P6EERAABQqmlkrocSi1+hwc73Adrg== 2024-09-21T07:32:42.066 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.c] 2024-09-21T07:32:42.066 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQBzdu5msbx6HxAALsF8ORvIoJpIAFZ5uxmOEw== 2024-09-21T07:32:42.066 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.0] 2024-09-21T07:32:42.066 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQB7du5msw81DhAAmVimqN4x9K3YpT0nbovppQ== 2024-09-21T07:32:42.066 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.1] 2024-09-21T07:32:42.066 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQB+du5mdnCIOhAAtVB4ThehQwb0zRriyKcaxA== 2024-09-21T07:32:42.066 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.2] 2024-09-21T07:32:42.066 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCCdu5mTfWxDhAAadJCbPN5xBx/f9L0v2yTdg== 2024-09-21T07:32:42.066 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.3] 2024-09-21T07:32:42.066 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCFdu5mL1E2HxAAyOhY97uUXEw5L1CcDeONVA== 2024-09-21T07:32:42.066 INFO:teuthology.orchestra.run.smithi148.stdout:[mgr.x] 2024-09-21T07:32:42.067 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQBzdu5mA2GRBhAAkyCKdgH6laHy8xNt080dXg== 2024-09-21T07:32:42.067 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.b] 2024-09-21T07:32:42.067 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQBzdu5m5/GeLRAAfE8fr5dbitQa89rNK9pZ2Q== 2024-09-21T07:32:42.067 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.d] 2024-09-21T07:32:42.067 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQB0du5mIrcBABAApKAB/7FkZ5glNTxlbkIsBw== 2024-09-21T07:32:42.067 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.4] 2024-09-21T07:32:42.067 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCLdu5m6PY/IhAAgvcTgm+6uWA7lIG4tcTIEg== 2024-09-21T07:32:42.067 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.5] 2024-09-21T07:32:42.067 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCOdu5mE/wlMxAA3JBg4xH+N/PPC1Uv2ZVrYQ== 2024-09-21T07:32:42.067 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.6] 2024-09-21T07:32:42.067 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCSdu5mC3TgBxAAwsArjeIdFcApY3/1OJffEg== 2024-09-21T07:32:42.067 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.7] 2024-09-21T07:32:42.067 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCVdu5mSWHWFBAA9qPHWRdsvrB74inrp1lj4Q== 2024-09-21T07:32:42.067 INFO:teuthology.orchestra.run.smithi148.stdout:[client.0] 2024-09-21T07:32:42.068 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQB0du5mGfb1DBAA/aSMKR5or3xy53rB/iUjpA== 2024-09-21T07:32:42.068 INFO:teuthology.orchestra.run.smithi148.stdout:[client.2] 2024-09-21T07:32:42.068 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQB0du5mSu2oFxAAgjy2m/wWKh6xaoh+YDoU1Q== 2024-09-21T07:32:42.068 INFO:teuthology.orchestra.run.smithi148.stdout:[client.1] 2024-09-21T07:32:42.068 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQB0du5mEkciIhAAid+bxJEr4P7vMnZ42PAy5w== 2024-09-21T07:32:42.068 INFO:teuthology.orchestra.run.smithi148.stdout:[client.3] 2024-09-21T07:32:42.068 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQB0du5mCHy0LBAAZneF2jUWgfuyGehZFbOxDg== 2024-09-21T07:32:42.069 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T07:32:42.073 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T07:32:42.271 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T07:32:42.275 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T07:32:42.440 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T07:32:42.444 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T07:32:42.607 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:32:42.611 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:32:42.773 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:32:42.777 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:32:42.942 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:32:42.946 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:32:43.110 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:32:43.114 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:32:43.276 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T07:32:43.280 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T07:32:43.442 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T07:32:43.445 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T07:32:43.608 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T07:32:43.612 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T07:32:43.778 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:32:43.781 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:32:43.942 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:32:43.946 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:32:44.109 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:32:44.113 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:32:44.276 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:32:44.280 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:32:44.441 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T07:32:44.445 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T07:32:44.609 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T07:32:44.613 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T07:32:44.777 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T07:32:44.781 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T07:32:44.944 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T07:32:44.947 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T07:32:45.114 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-21T07:32:45.114 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-21T07:32:45.170 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T07:32:45.366 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-21T07:32:45.422 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-21T07:32:45.478 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T07:32:45.654 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-21T07:32:45.710 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-21T07:32:45.765 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T07:32:45.940 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-21T07:32:45.996 DEBUG:teuthology.orchestra.run.smithi039:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:32:46.000 DEBUG:teuthology.orchestra.run.smithi148:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:32:46.034 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-21T07:32:46.035 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-21T07:32:46.035 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-21T07:32:46.077 INFO:tasks.ceph.mon.a:Started 2024-09-21T07:32:46.078 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-21T07:32:46.078 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-21T07:32:46.081 INFO:tasks.ceph.mon.b:Started 2024-09-21T07:32:46.082 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-21T07:32:46.082 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-21T07:32:46.085 INFO:tasks.ceph.mon.c:Started 2024-09-21T07:32:46.085 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-21T07:32:46.086 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-21T07:32:46.086 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-21T07:32:46.090 INFO:tasks.ceph.mgr.y:Started 2024-09-21T07:32:46.090 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-21T07:32:46.090 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-21T07:32:46.094 INFO:tasks.ceph.mgr.x:Started 2024-09-21T07:32:46.094 INFO:tasks.ceph:setting config [client] client_mount_timeout = 600 2024-09-21T07:32:46.095 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph config set client client_mount_timeout 600 2024-09-21T07:32:46.099 INFO:tasks.ceph:setting config [client] debug_client = 20 2024-09-21T07:32:46.099 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph config set client debug_client 20 2024-09-21T07:32:46.144 INFO:tasks.ceph:setting config [client] debug_ms = 1 2024-09-21T07:32:46.145 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph config set client debug_ms 1 2024-09-21T07:32:46.191 INFO:tasks.ceph:setting config [client] rados_mon_op_timeout = 900 2024-09-21T07:32:46.191 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph config set client rados_mon_op_timeout 900 2024-09-21T07:32:46.239 INFO:tasks.ceph:setting config [client] rados_osd_op_timeout = 900 2024-09-21T07:32:46.239 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph config set client rados_osd_op_timeout 900 2024-09-21T07:32:46.294 INFO:tasks.ceph:setting config [mds] debug_mds = 20 2024-09-21T07:32:46.294 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph config set mds debug_mds 20 2024-09-21T07:32:46.351 INFO:tasks.ceph:setting config [mds] debug_mds_balancer = 20 2024-09-21T07:32:46.351 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph config set mds debug_mds_balancer 20 2024-09-21T07:32:46.399 INFO:tasks.ceph:setting config [mds] debug_ms = 2 2024-09-21T07:32:46.399 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph config set mds debug_ms 2 2024-09-21T07:32:46.450 INFO:tasks.ceph:setting config [mds] mds_debug_frag = True 2024-09-21T07:32:46.450 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph config set mds mds_debug_frag True 2024-09-21T07:32:46.503 INFO:tasks.ceph:setting config [mds] mds_debug_scatterstat = True 2024-09-21T07:32:46.503 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph config set mds mds_debug_scatterstat True 2024-09-21T07:32:46.561 INFO:tasks.ceph:setting config [mds] mds_op_complaint_time = 180 2024-09-21T07:32:46.561 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph config set mds mds_op_complaint_time 180 2024-09-21T07:32:46.617 INFO:tasks.ceph:setting config [mds] mds_verify_scatter = True 2024-09-21T07:32:46.617 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph config set mds mds_verify_scatter True 2024-09-21T07:32:46.680 INFO:tasks.ceph:setting config [mds] osd_op_complaint_time = 180 2024-09-21T07:32:46.680 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph config set mds osd_op_complaint_time 180 2024-09-21T07:32:46.720 INFO:tasks.ceph:setting config [mds] rados_mon_op_timeout = 900 2024-09-21T07:32:46.721 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph config set mds rados_mon_op_timeout 900 2024-09-21T07:32:46.781 INFO:tasks.ceph:setting config [mds] rados_osd_op_timeout = 900 2024-09-21T07:32:46.781 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph config set mds rados_osd_op_timeout 900 2024-09-21T07:32:46.834 INFO:tasks.ceph:setting config [mgr] client_mount_timeout = 30 2024-09-21T07:32:46.835 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph config set mgr client_mount_timeout 30 2024-09-21T07:32:46.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.858+0000 7f8007e00640 1 -- 172.21.15.39:0/3895939405 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f80080cc6c0 con 0x7f80080d0bb0 2024-09-21T07:32:46.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.858+0000 7f7ffea00640 1 -- 172.21.15.39:0/3895939405 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7ff8044070 con 0x7f80080d0bb0 2024-09-21T07:32:46.883 INFO:tasks.ceph:setting config [mgr] debug_client = 20 2024-09-21T07:32:46.883 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph config set mgr debug_client 20 2024-09-21T07:32:46.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.876+0000 7f8007e00640 1 -- 172.21.15.39:0/3895939405 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80080cab10 con 0x7f80080d0bb0 2024-09-21T07:32:46.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.883+0000 7f7ffea00640 1 -- 172.21.15.39:0/3895939405 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7ff8046450 con 0x7f80080d0bb0 2024-09-21T07:32:46.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.886+0000 7f7ffea00640 1 -- 172.21.15.39:0/3895939405 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f7ff8046a40 con 0x7f80080d0bb0 2024-09-21T07:32:46.903 INFO:tasks.ceph:setting config [mgr] debug_mgr = 20 2024-09-21T07:32:46.903 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph config set mgr debug_mgr 20 2024-09-21T07:32:46.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.904+0000 7f70e1e00640 1 -- 172.21.15.39:0/3613068140 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f70dc15ea80 msgr2=0x7f70dc162f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:46.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.904+0000 7f70d9600640 1 -- 172.21.15.39:0/3613068140 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70cc035020 con 0x7f70dc15ea80 2024-09-21T07:32:46.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.904+0000 7f70e1e00640 1 --2- 172.21.15.39:0/3613068140 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f70dc15ea80 0x7f70dc162f20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f70dc072de0 tx=0x7f70cc02a5a0 comp rx=0 tx=0).stop 2024-09-21T07:32:46.924 INFO:tasks.ceph:setting config [mgr] debug_ms = 1 2024-09-21T07:32:46.924 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph config set mgr debug_ms 1 2024-09-21T07:32:46.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.914+0000 7f70e1e00640 1 -- 172.21.15.39:0/3613068140 shutdown_connections 2024-09-21T07:32:46.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.914+0000 7f70e1e00640 1 --2- 172.21.15.39:0/3613068140 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f70dc072e30 0x7f70dc073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:46.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.914+0000 7f70e1e00640 1 --2- 172.21.15.39:0/3613068140 >> v2:172.21.15.148:3300/0 conn(0x7f70dc15f650 0x7f70dc15fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:46.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.914+0000 7f70e1e00640 1 --2- 172.21.15.39:0/3613068140 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f70dc15ea80 0x7f70dc162f20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:46.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.914+0000 7f70e1e00640 1 -- 172.21.15.39:0/3613068140 >> 172.21.15.39:0/3613068140 conn(0x7f70dc06d410 msgr2=0x7f70dc0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:46.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.915+0000 7f70e1e00640 1 -- 172.21.15.39:0/3613068140 shutdown_connections 2024-09-21T07:32:46.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.915+0000 7f70e1e00640 1 -- 172.21.15.39:0/3613068140 wait complete. 2024-09-21T07:32:46.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.933+0000 7f1378000640 1 -- 172.21.15.39:0/4038368092 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f137006ddf0 con 0x7f13580099b0 2024-09-21T07:32:46.956 INFO:tasks.ceph:setting config [mgr] mon_warn_on_pool_no_app = False 2024-09-21T07:32:46.956 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph config set mgr mon_warn_on_pool_no_app False 2024-09-21T07:32:46.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.941+0000 7f4866a00640 1 -- 172.21.15.39:0/402454408 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f4848000e60 con 0x7f486c009ba0 2024-09-21T07:32:46.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.927+0000 7f70e1e00640 1 Processor -- start 2024-09-21T07:32:46.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.932+0000 7f70e1e00640 1 -- start start 2024-09-21T07:32:46.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.938+0000 7f70e1e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f70dc072e30 0x7f70dc205520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:46.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.942+0000 7f70daa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f70dc072e30 0x7f70dc205520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:46.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.942+0000 7f70daa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f70dc072e30 0x7f70dc205520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54794/0 (socket says 172.21.15.39:54794) 2024-09-21T07:32:46.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.942+0000 7f70daa00640 1 -- 172.21.15.39:0/3657657688 learned_addr learned my addr 172.21.15.39:0/3657657688 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:32:46.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.942+0000 7f70e1e00640 1 --2- 172.21.15.39:0/3657657688 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f70dc15ea80 0x7f70dc09cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:46.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.943+0000 7f70e1e00640 1 --2- 172.21.15.39:0/3657657688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70dc15f650 0x7f70dc205c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:46.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.943+0000 7f70da000640 1 --2- 172.21.15.39:0/3657657688 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f70dc15ea80 0x7f70dc09cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:46.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.946+0000 7f70e1e00640 1 -- 172.21.15.39:0/3657657688 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc074cc0 con 0x7f70dc072e30 2024-09-21T07:32:46.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.946+0000 7f70e1e00640 1 -- 172.21.15.39:0/3657657688 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc0b4570 con 0x7f70dc15f650 2024-09-21T07:32:46.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.946+0000 7f70e1e00640 1 -- 172.21.15.39:0/3657657688 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f70dc0cf5b0 con 0x7f70dc15ea80 2024-09-21T07:32:46.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.947+0000 7f70daa00640 1 -- 172.21.15.39:0/3657657688 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f70dc15ea80 msgr2=0x7f70dc09cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:46.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.947+0000 7f70daa00640 1 --2- 172.21.15.39:0/3657657688 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f70dc15ea80 0x7f70dc09cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:46.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.947+0000 7f70daa00640 1 -- 172.21.15.39:0/3657657688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70dc15f650 msgr2=0x7f70dc205c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T07:32:46.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.947+0000 7f70daa00640 1 --2- 172.21.15.39:0/3657657688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70dc15f650 0x7f70dc205c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:46.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.947+0000 7f70daa00640 1 -- 172.21.15.39:0/3657657688 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70dc09afa0 con 0x7f70dc072e30 2024-09-21T07:32:46.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.948+0000 7f70daa00640 1 --2- 172.21.15.39:0/3657657688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f70dc072e30 0x7f70dc205520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f70cc02ab60 tx=0x7f70cc003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:32:46.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.949+0000 7f70d3400640 1 -- 172.21.15.39:0/3657657688 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70cc03e070 con 0x7f70dc072e30 2024-09-21T07:32:46.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.949+0000 7f70d3400640 1 -- 172.21.15.39:0/3657657688 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(3 keys) ==== 79+0+0 (secure 0 0 0) 0x7f70cc004bb0 con 0x7f70dc072e30 2024-09-21T07:32:46.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.949+0000 7f70d3400640 1 -- 172.21.15.39:0/3657657688 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70cc035040 con 0x7f70dc072e30 2024-09-21T07:32:46.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.949+0000 7f11ee200640 1 -- 172.21.15.39:0/925935805 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11e80b9540 msgr2=0x7f11e80d4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:46.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.949+0000 7f11ee200640 1 --2- 172.21.15.39:0/925935805 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11e80b9540 0x7f11e80d4ee0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f11d40098e0 tx=0x7f11d402a5f0 comp rx=0 tx=0).stop 2024-09-21T07:32:46.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.951+0000 7f11ee200640 1 -- 172.21.15.39:0/925935805 shutdown_connections 2024-09-21T07:32:46.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.951+0000 7f11ee200640 1 --2- 172.21.15.39:0/925935805 >> v2:172.21.15.39:3300/0 conn(0x7f11e80d0bb0 0x7f11e80d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:46.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.952+0000 7f11ee200640 1 --2- 172.21.15.39:0/925935805 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11e80b9540 0x7f11e80d4ee0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:46.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.952+0000 7f11ee200640 1 -- 172.21.15.39:0/925935805 >> 172.21.15.39:0/925935805 conn(0x7f11e809d680 msgr2=0x7f11e80d4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:46.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.954+0000 7f11ee200640 1 -- 172.21.15.39:0/925935805 shutdown_connections 2024-09-21T07:32:46.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.953+0000 7f4871600640 1 -- 172.21.15.39:0/402454408 <== mon.0 v2:172.21.15.39:3300/0 8 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f4860004870 con 0x7f486c009ba0 2024-09-21T07:32:46.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.954+0000 7f7ffea00640 1 -- 172.21.15.39:0/3895939405 <== mon.2 v2:172.21.15.148:3301/0 7 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f7ff8046c40 con 0x7f80080d0bb0 2024-09-21T07:32:46.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.949+0000 7f70e1e00640 1 -- 172.21.15.39:0/3657657688 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70dc09b280 con 0x7f70dc072e30 2024-09-21T07:32:46.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.949+0000 7f70e1e00640 1 -- 172.21.15.39:0/3657657688 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70dc09b840 con 0x7f70dc072e30 2024-09-21T07:32:46.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.954+0000 7f11ee200640 1 -- 172.21.15.39:0/925935805 wait complete. 2024-09-21T07:32:46.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.952+0000 7f136d600640 1 -- 172.21.15.39:0/4038368092 <== mon.2 v2:172.21.15.148:3301/0 8 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f135c00fb20 con 0x7f13580099b0 2024-09-21T07:32:46.975 INFO:tasks.ceph:setting config [mon] mon_op_complaint_time = 120 2024-09-21T07:32:46.975 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph config set mon mon_op_complaint_time 120 2024-09-21T07:32:46.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.959+0000 7f136d600640 1 -- 172.21.15.39:0/4038368092 <== mon.2 v2:172.21.15.148:3301/0 9 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v5)=0 v5) ==== 123+0+0 (secure 0 0 0) 0x7f1370160fb0 con 0x7f13580099b0 2024-09-21T07:32:46.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.955+0000 7f70d3400640 1 -- 172.21.15.39:0/3657657688 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f70cc005df0 con 0x7f70dc072e30 2024-09-21T07:32:46.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.966+0000 7f1378000640 1 -- 172.21.15.39:0/4038368092 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f13580099b0 msgr2=0x7f1370146300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:46.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.966+0000 7f1378000640 1 --2- 172.21.15.39:0/4038368092 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f13580099b0 0x7f1370146300 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f135c00c940 tx=0x7f135c00ce00 comp rx=0 tx=0).stop 2024-09-21T07:32:46.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.966+0000 7f1378000640 1 -- 172.21.15.39:0/4038368092 shutdown_connections 2024-09-21T07:32:46.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.966+0000 7f1378000640 1 --2- 172.21.15.39:0/4038368092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1370160660 0x7f1370160a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:46.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.966+0000 7f1378000640 1 --2- 172.21.15.39:0/4038368092 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1370147bd0 0x7f1370143a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:46.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.966+0000 7f1378000640 1 --2- 172.21.15.39:0/4038368092 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f13580099b0 0x7f1370146300 secure :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f135c00c940 tx=0x7f135c00ce00 comp rx=0 tx=0).stop 2024-09-21T07:32:46.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.966+0000 7f1378000640 1 -- 172.21.15.39:0/4038368092 >> 172.21.15.39:0/4038368092 conn(0x7f137006d5d0 msgr2=0x7f137015ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:46.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.969+0000 7f1378000640 1 -- 172.21.15.39:0/4038368092 shutdown_connections 2024-09-21T07:32:46.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.969+0000 7f1378000640 1 -- 172.21.15.39:0/4038368092 wait complete. 2024-09-21T07:32:46.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.968+0000 7f70e1e00640 1 -- 172.21.15.39:0/3657657688 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70b4005290 con 0x7f70dc072e30 2024-09-21T07:32:46.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.974+0000 7f11ee200640 1 Processor -- start 2024-09-21T07:32:46.994 INFO:tasks.ceph:setting config [osd] osd_op_complaint_time = 180 2024-09-21T07:32:46.994 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph config set osd osd_op_complaint_time 180 2024-09-21T07:32:46.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.980+0000 7f4871600640 1 -- 172.21.15.39:0/402454408 <== mon.0 v2:172.21.15.39:3300/0 9 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v6)=0 v6) ==== 123+0+0 (secure 0 0 0) 0x7f4860004b20 con 0x7f486c009ba0 2024-09-21T07:32:46.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.980+0000 7f4871600640 1 -- 172.21.15.39:0/402454408 <== mon.0 v2:172.21.15.39:3300/0 10 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4860004870 con 0x7f486c009ba0 2024-09-21T07:32:46.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.967+0000 7f7ffea00640 1 -- 172.21.15.39:0/3895939405 <== mon.2 v2:172.21.15.148:3301/0 8 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ff80318a0 con 0x7f80080d0bb0 2024-09-21T07:32:46.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.990+0000 7f70d3400640 1 -- 172.21.15.39:0/3657657688 <== mon.0 v2:172.21.15.39:3300/0 5 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f70cc041070 con 0x7f70dc072e30 2024-09-21T07:32:46.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.990+0000 7f70d3400640 1 -- 172.21.15.39:0/3657657688 <== mon.0 v2:172.21.15.39:3300/0 6 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f70cc043ba0 con 0x7f70dc072e30 2024-09-21T07:32:46.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.990+0000 7f70d3400640 1 -- 172.21.15.39:0/3657657688 <== mon.0 v2:172.21.15.39:3300/0 7 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70cc031400 con 0x7f70dc072e30 2024-09-21T07:32:46.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.981+0000 7f11ee200640 1 -- start start 2024-09-21T07:32:46.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.983+0000 7f11ee200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11e80b3660 0x7f11e81f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:46.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.994+0000 7f4866a00640 1 -- 172.21.15.39:0/402454408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f486c009ba0 msgr2=0x7f486c1788a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:46.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.994+0000 7f4866a00640 1 --2- 172.21.15.39:0/402454408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f486c009ba0 0x7f486c1788a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4860009760 tx=0x7f4860009c20 comp rx=0 tx=0).stop 2024-09-21T07:32:46.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.994+0000 7f4866a00640 1 -- 172.21.15.39:0/402454408 shutdown_connections 2024-09-21T07:32:46.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.994+0000 7f4866a00640 1 --2- 172.21.15.39:0/402454408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f486c0de4a0 0x7f486c178de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:46.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.994+0000 7f4866a00640 1 --2- 172.21.15.39:0/402454408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f486c009ba0 0x7f486c1788a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:46.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.994+0000 7f4866a00640 1 --2- 172.21.15.39:0/402454408 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f486c005c60 0x7f486c046c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:46.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.994+0000 7f4866a00640 1 -- 172.21.15.39:0/402454408 >> 172.21.15.39:0/402454408 conn(0x7f486c051420 msgr2=0x7f486c038410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:46.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.994+0000 7f4866a00640 1 -- 172.21.15.39:0/402454408 shutdown_connections 2024-09-21T07:32:46.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.994+0000 7f4866a00640 1 -- 172.21.15.39:0/402454408 wait complete. 2024-09-21T07:32:46.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.994+0000 7f70d3400640 1 -- 172.21.15.39:0/3657657688 <== mon.0 v2:172.21.15.39:3300/0 8 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f70cc048030 con 0x7f70dc072e30 2024-09-21T07:32:47.006 DEBUG:tasks.ceph:set 22 configs 2024-09-21T07:32:47.006 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi039') 2024-09-21T07:32:47.007 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi039') 2024-09-21T07:32:47.007 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi039') 2024-09-21T07:32:47.007 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi039') 2024-09-21T07:32:47.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.993+0000 7f11ee200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11e80b9540 0x7f11e8157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.993+0000 7f11ee200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11e80d0bb0 0x7f11e815a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:46.996+0000 7f11e7e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11e80b9540 0x7f11e8157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:47.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.002+0000 7f11ed800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11e80d0bb0 0x7f11e815a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:47.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.002+0000 7f11ed800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11e80d0bb0 0x7f11e815a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54802/0 (socket says 172.21.15.39:54802) 2024-09-21T07:32:47.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.002+0000 7f11e7e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11e80b9540 0x7f11e8157d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.39:37534/0 (socket says 172.21.15.39:37534) 2024-09-21T07:32:47.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.002+0000 7f11ee200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f11e80b3ca0 con 0x7f11e80d0bb0 2024-09-21T07:32:47.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.002+0000 7f11ee200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f11e80d3420 con 0x7f11e80b3660 2024-09-21T07:32:47.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.002+0000 7f11ee200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f11e80b3e20 con 0x7f11e80b9540 2024-09-21T07:32:47.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.002+0000 7f11ed800640 1 -- 172.21.15.39:0/3774700333 learned_addr learned my addr 172.21.15.39:0/3774700333 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:32:47.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.002+0000 7f11ece00640 1 --2- 172.21.15.39:0/3774700333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11e80b3660 0x7f11e81f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:47.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.003+0000 7f11ece00640 1 -- 172.21.15.39:0/3774700333 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11e80b9540 msgr2=0x7f11e8157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.003+0000 7f11ece00640 1 --2- 172.21.15.39:0/3774700333 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11e80b9540 0x7f11e8157d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.003+0000 7f11ece00640 1 -- 172.21.15.39:0/3774700333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11e80d0bb0 msgr2=0x7f11e815a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.003+0000 7f11ece00640 1 --2- 172.21.15.39:0/3774700333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11e80d0bb0 0x7f11e815a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.003+0000 7f11ece00640 1 -- 172.21.15.39:0/3774700333 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e81f5c00 con 0x7f11e80b3660 2024-09-21T07:32:47.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.015+0000 7f11ece00640 1 --2- 172.21.15.39:0/3774700333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11e80b3660 0x7f11e81f56c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f11d4031000 tx=0x7f11d4033f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:32:47.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.019+0000 7f11e4c00640 1 -- 172.21.15.39:0/3774700333 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d4005cf0 con 0x7f11e80b3660 2024-09-21T07:32:47.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.019+0000 7f11e4c00640 1 -- 172.21.15.39:0/3774700333 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11d4004030 con 0x7f11e80b3660 2024-09-21T07:32:47.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.027+0000 7f11e4c00640 1 -- 172.21.15.39:0/3774700333 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d4004480 con 0x7f11e80b3660 2024-09-21T07:32:47.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.027+0000 7f11ee200640 1 -- 172.21.15.39:0/3774700333 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11e80b2210 con 0x7f11e80b3660 2024-09-21T07:32:47.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.027+0000 7f11ee200640 1 -- 172.21.15.39:0/3774700333 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11e81f4ec0 con 0x7f11e80b3660 2024-09-21T07:32:47.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.033+0000 7f11ee200640 1 -- 172.21.15.39:0/3774700333 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11e80d16e0 con 0x7f11e80b3660 2024-09-21T07:32:47.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.040+0000 7f11e4c00640 1 -- 172.21.15.39:0/3774700333 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f11d4004b30 con 0x7f11e80b3660 2024-09-21T07:32:47.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.040+0000 7f11e4c00640 1 -- 172.21.15.39:0/3774700333 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f11d4031b80 con 0x7f11e80b3660 2024-09-21T07:32:47.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.057+0000 7f11e4c00640 1 -- 172.21.15.39:0/3774700333 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f11e80d16e0 con 0x7f11e80b3660 2024-09-21T07:32:47.074 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi039') 2024-09-21T07:32:47.080 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi039') 2024-09-21T07:32:47.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.135+0000 7fcb62800640 1 -- 172.21.15.39:0/3415453658 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcb5c15da00 msgr2=0x7fcb5c0b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.139+0000 7fcb62800640 1 --2- 172.21.15.39:0/3415453658 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcb5c15da00 0x7fcb5c0b3940 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcb48005fd0 tx=0x7fcb4802a610 comp rx=0 tx=0).stop 2024-09-21T07:32:47.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.147+0000 7fcb62800640 1 -- 172.21.15.39:0/3415453658 shutdown_connections 2024-09-21T07:32:47.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.147+0000 7fcb62800640 1 --2- 172.21.15.39:0/3415453658 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcb5c15da00 0x7fcb5c0b3940 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.147+0000 7fcb62800640 1 --2- 172.21.15.39:0/3415453658 >> v2:172.21.15.39:3300/0 conn(0x7fcb5c15b970 0x7fcb5c15bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.147+0000 7fcb62800640 1 -- 172.21.15.39:0/3415453658 >> 172.21.15.39:0/3415453658 conn(0x7fcb5c074220 msgr2=0x7fcb5c074630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:47.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.151+0000 7fcb62800640 1 -- 172.21.15.39:0/3415453658 shutdown_connections 2024-09-21T07:32:47.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.151+0000 7fcb62800640 1 -- 172.21.15.39:0/3415453658 wait complete. 2024-09-21T07:32:47.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.169+0000 7fcb62800640 1 Processor -- start 2024-09-21T07:32:47.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.170+0000 7fcb62800640 1 -- start start 2024-09-21T07:32:47.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.185+0000 7fcb62800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb5c15b970 0x7fcb5c14a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.185+0000 7fcb62800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcb5c15da00 0x7fcb5c14e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.185+0000 7fcb62800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb5c15de10 0x7fcb5c14e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.185+0000 7fcb5b400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb5c15b970 0x7fcb5c14a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:47.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.186+0000 7fcb5b400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb5c15b970 0x7fcb5c14a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.39:40286/0 (socket says 172.21.15.39:40286) 2024-09-21T07:32:47.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.186+0000 7fcb62800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c159130 con 0x7fcb5c15de10 2024-09-21T07:32:47.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.186+0000 7fcb62800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c1592b0 con 0x7fcb5c15b970 2024-09-21T07:32:47.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.186+0000 7fcb62800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fcb5c1f5bb0 con 0x7fcb5c15da00 2024-09-21T07:32:47.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.186+0000 7fcb5b400640 1 -- 172.21.15.39:0/2971336290 learned_addr learned my addr 172.21.15.39:0/2971336290 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:32:47.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.186+0000 7fcb5aa00640 1 --2- 172.21.15.39:0/2971336290 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcb5c15da00 0x7fcb5c14e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:47.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.186+0000 7fcb61200640 1 --2- 172.21.15.39:0/2971336290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb5c15de10 0x7fcb5c14e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:47.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.187+0000 7fcb5b400640 1 -- 172.21.15.39:0/2971336290 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcb5c15da00 msgr2=0x7fcb5c14e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.188+0000 7fcb5b400640 1 --2- 172.21.15.39:0/2971336290 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcb5c15da00 0x7fcb5c14e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.188+0000 7fcb5b400640 1 -- 172.21.15.39:0/2971336290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb5c15de10 msgr2=0x7fcb5c14e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.188+0000 7fcb5b400640 1 --2- 172.21.15.39:0/2971336290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb5c15de10 0x7fcb5c14e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.188+0000 7fcb5b400640 1 -- 172.21.15.39:0/2971336290 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb5c1f5e90 con 0x7fcb5c15b970 2024-09-21T07:32:47.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.188+0000 7fcb61200640 1 --2- 172.21.15.39:0/2971336290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb5c15de10 0x7fcb5c14e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:32:47.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.189+0000 7fcb5b400640 1 --2- 172.21.15.39:0/2971336290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb5c15b970 0x7fcb5c14a930 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000b410 tx=0x7fcb5000b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:32:47.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.189+0000 7fcb5aa00640 1 --2- 172.21.15.39:0/2971336290 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcb5c15da00 0x7fcb5c14e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:32:47.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.189+0000 7fcb4f400640 1 -- 172.21.15.39:0/2971336290 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb50015020 con 0x7fcb5c15b970 2024-09-21T07:32:47.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.190+0000 7fcb4f400640 1 -- 172.21.15.39:0/2971336290 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb50011920 con 0x7fcb5c15b970 2024-09-21T07:32:47.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.190+0000 7fcb62800640 1 -- 172.21.15.39:0/2971336290 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb5c0b0f50 con 0x7fcb5c15b970 2024-09-21T07:32:47.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.195+0000 7fcb62800640 1 -- 172.21.15.39:0/2971336290 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb5c14b210 con 0x7fcb5c15b970 2024-09-21T07:32:47.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.197+0000 7fcb4cc00640 1 -- 172.21.15.39:0/2971336290 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb5c0b0a80 con 0x7fcb5c15b970 2024-09-21T07:32:47.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.201+0000 7fcb4f400640 1 -- 172.21.15.39:0/2971336290 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb50011d70 con 0x7fcb5c15b970 2024-09-21T07:32:47.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.201+0000 7fcb4f400640 1 -- 172.21.15.39:0/2971336290 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcb50018020 con 0x7fcb5c15b970 2024-09-21T07:32:47.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.202+0000 7fcb4f400640 1 -- 172.21.15.39:0/2971336290 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcb50019b70 con 0x7fcb5c15b970 2024-09-21T07:32:47.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.206+0000 7fcb4f400640 1 -- 172.21.15.39:0/2971336290 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fcb5001f020 con 0x7fcb5c15b970 2024-09-21T07:32:47.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.238+0000 7f8007e00640 1 -- 172.21.15.39:0/3895939405 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f7fdc0052f0 con 0x7f80080d0bb0 2024-09-21T07:32:47.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.262+0000 7f7ffea00640 1 -- 172.21.15.39:0/3895939405 <== mon.2 v2:172.21.15.148:3301/0 9 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v7)=0 v7) ==== 108+0+0 (secure 0 0 0) 0x7f7ff8031a90 con 0x7f80080d0bb0 2024-09-21T07:32:47.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.272+0000 7f7ff3e00640 1 -- 172.21.15.39:0/3895939405 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80080d0bb0 msgr2=0x7f800815f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.272+0000 7f7ff3e00640 1 --2- 172.21.15.39:0/3895939405 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80080d0bb0 0x7f800815f320 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8033040 tx=0x7f7ff80046c0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.292+0000 7fbc55e00640 1 -- 172.21.15.39:0/3185772872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc5015ca30 msgr2=0x7fbc5015ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.292+0000 7f7ff3e00640 1 -- 172.21.15.39:0/3895939405 shutdown_connections 2024-09-21T07:32:47.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.292+0000 7f7ff3e00640 1 --2- 172.21.15.39:0/3895939405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80080d4ee0 0x7f800815f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.292+0000 7f7ff3e00640 1 --2- 172.21.15.39:0/3895939405 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80080d0bb0 0x7f800815f320 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.292+0000 7f7ff3e00640 1 --2- 172.21.15.39:0/3895939405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80080b9540 0x7f80081537d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.292+0000 7f7ff3e00640 1 -- 172.21.15.39:0/3895939405 >> 172.21.15.39:0/3895939405 conn(0x7f800809d680 msgr2=0x7f80080d2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:47.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.292+0000 7fbc55e00640 1 --2- 172.21.15.39:0/3185772872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc5015ca30 0x7fbc5015ce40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c005fd0 tx=0x7fbc3c02a570 comp rx=0 tx=0).stop 2024-09-21T07:32:47.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.292+0000 7f7ff3e00640 1 -- 172.21.15.39:0/3895939405 shutdown_connections 2024-09-21T07:32:47.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.292+0000 7f7ff3e00640 1 -- 172.21.15.39:0/3895939405 wait complete. 2024-09-21T07:32:47.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.296+0000 7fbc55e00640 1 -- 172.21.15.39:0/3185772872 shutdown_connections 2024-09-21T07:32:47.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.296+0000 7fbc55e00640 1 --2- 172.21.15.39:0/3185772872 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc500d0340 0x7fbc500d0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.296+0000 7fbc55e00640 1 --2- 172.21.15.39:0/3185772872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc5015ca30 0x7fbc5015ce40 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.296+0000 7fbc55e00640 1 -- 172.21.15.39:0/3185772872 >> 172.21.15.39:0/3185772872 conn(0x7fbc50070b20 msgr2=0x7fbc50070f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:47.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.297+0000 7fbc55e00640 1 -- 172.21.15.39:0/3185772872 shutdown_connections 2024-09-21T07:32:47.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.297+0000 7fbc55e00640 1 -- 172.21.15.39:0/3185772872 wait complete. 2024-09-21T07:32:47.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.312+0000 7fbc55e00640 1 Processor -- start 2024-09-21T07:32:47.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.312+0000 7fbc55e00640 1 -- start start 2024-09-21T07:32:47.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.319+0000 7fbc55e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc5006f9f0 0x7fbc501f6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.319+0000 7fbc55e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc500d0340 0x7fbc501f6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.320+0000 7fbc55e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc5015ca30 0x7fbc5015f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.320+0000 7fbc55e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbc500ccc50 con 0x7fbc500d0340 2024-09-21T07:32:47.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.320+0000 7fbc55e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbc500ccad0 con 0x7fbc5006f9f0 2024-09-21T07:32:47.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.320+0000 7fbc55e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fbc500ccdd0 con 0x7fbc5015ca30 2024-09-21T07:32:47.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.325+0000 7fbc4fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc500d0340 0x7fbc501f6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:47.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.325+0000 7fbc4fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc500d0340 0x7fbc501f6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54836/0 (socket says 172.21.15.39:54836) 2024-09-21T07:32:47.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.325+0000 7fbc4fe00640 1 -- 172.21.15.39:0/2310902452 learned_addr learned my addr 172.21.15.39:0/2310902452 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:32:47.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.326+0000 7fbc4fe00640 1 -- 172.21.15.39:0/2310902452 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc5015ca30 msgr2=0x7fbc5015f200 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T07:32:47.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.326+0000 7fbc4fe00640 1 --2- 172.21.15.39:0/2310902452 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc5015ca30 0x7fbc5015f200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.326+0000 7fbc4fe00640 1 -- 172.21.15.39:0/2310902452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc5006f9f0 msgr2=0x7fbc501f6940 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T07:32:47.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.326+0000 7fbc4fe00640 1 --2- 172.21.15.39:0/2310902452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc5006f9f0 0x7fbc501f6940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.326+0000 7fbc4fe00640 1 -- 172.21.15.39:0/2310902452 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc500a9490 con 0x7fbc500d0340 2024-09-21T07:32:47.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.327+0000 7fbc4fe00640 1 --2- 172.21.15.39:0/2310902452 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc500d0340 0x7fbc501f6e80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c009850 tx=0x7fbc3c0042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:32:47.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.328+0000 7fbc4cc00640 1 -- 172.21.15.39:0/2310902452 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c03db40 con 0x7fbc500d0340 2024-09-21T07:32:47.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.328+0000 7fbc55e00640 1 -- 172.21.15.39:0/2310902452 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc500a9730 con 0x7fbc500d0340 2024-09-21T07:32:47.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.329+0000 7fbc4cc00640 1 -- 172.21.15.39:0/2310902452 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc3c03ddd0 con 0x7fbc500d0340 2024-09-21T07:32:47.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.329+0000 7fbc4cc00640 1 -- 172.21.15.39:0/2310902452 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c03db40 con 0x7fbc500d0340 2024-09-21T07:32:47.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.334+0000 7fbc55e00640 1 -- 172.21.15.39:0/2310902452 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc500a9cf0 con 0x7fbc500d0340 2024-09-21T07:32:47.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.334+0000 7fbc4cc00640 1 -- 172.21.15.39:0/2310902452 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbc3c048810 con 0x7fbc500d0340 2024-09-21T07:32:47.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.335+0000 7fbc55e00640 1 -- 172.21.15.39:0/2310902452 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc500cd2f0 con 0x7fbc500d0340 2024-09-21T07:32:47.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.347+0000 7fbc4cc00640 1 -- 172.21.15.39:0/2310902452 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbc3c031d00 con 0x7fbc500d0340 2024-09-21T07:32:47.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.348+0000 7fbc4cc00640 1 -- 172.21.15.39:0/2310902452 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fbc3c057080 con 0x7fbc500d0340 2024-09-21T07:32:47.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.350+0000 7f9a92200640 1 -- 172.21.15.39:0/4081922823 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a8c0b0af0 msgr2=0x7f9a8c0b3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.350+0000 7f9a92200640 1 --2- 172.21.15.39:0/4081922823 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a8c0b0af0 0x7f9a8c0b3ad0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9a7800b850 tx=0x7f9a78005f90 comp rx=0 tx=0).stop 2024-09-21T07:32:47.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.350+0000 7f9a92200640 1 -- 172.21.15.39:0/4081922823 shutdown_connections 2024-09-21T07:32:47.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.350+0000 7f9a92200640 1 --2- 172.21.15.39:0/4081922823 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a8c0b0af0 0x7f9a8c0b3ad0 secure :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9a7800b850 tx=0x7f9a78005f90 comp rx=0 tx=0).stop 2024-09-21T07:32:47.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.350+0000 7f9a92200640 1 --2- 172.21.15.39:0/4081922823 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a8c0afc00 0x7f9a8c15aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.350+0000 7f9a92200640 1 -- 172.21.15.39:0/4081922823 >> 172.21.15.39:0/4081922823 conn(0x7f9a8c074220 msgr2=0x7f9a8c074630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:47.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.351+0000 7f9a92200640 1 -- 172.21.15.39:0/4081922823 shutdown_connections 2024-09-21T07:32:47.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.351+0000 7f9a92200640 1 -- 172.21.15.39:0/4081922823 wait complete. 2024-09-21T07:32:47.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.359+0000 7f9a92200640 1 Processor -- start 2024-09-21T07:32:47.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.360+0000 7f9a92200640 1 -- start start 2024-09-21T07:32:47.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.370+0000 7f9a92200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a8c0afc00 0x7f9a8c143470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.370+0000 7f9a92200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a8c0b4010 0x7f9a8c15bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.370+0000 7f9a92200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a8c14b550 0x7f9a8c15c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.371+0000 7f9a92200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8c0aeae0 con 0x7f9a8c0afc00 2024-09-21T07:32:47.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.371+0000 7f9a8b400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a8c14b550 0x7f9a8c15c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:47.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.371+0000 7f9a8b400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a8c14b550 0x7f9a8c15c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.39:40296/0 (socket says 172.21.15.39:40296) 2024-09-21T07:32:47.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.371+0000 7f9a8b400640 1 -- 172.21.15.39:0/2774809512 learned_addr learned my addr 172.21.15.39:0/2774809512 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:32:47.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.371+0000 7f9a92200640 1 -- 172.21.15.39:0/2774809512 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8c074f50 con 0x7f9a8c14b550 2024-09-21T07:32:47.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.371+0000 7f9a8a000640 1 --2- 172.21.15.39:0/2774809512 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a8c0b4010 0x7f9a8c15bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:47.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.371+0000 7f9a92200640 1 -- 172.21.15.39:0/2774809512 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f9a8c0aec60 con 0x7f9a8c0b4010 2024-09-21T07:32:47.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.372+0000 7f9a8a000640 1 -- 172.21.15.39:0/2774809512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a8c14b550 msgr2=0x7f9a8c15c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.373+0000 7f9a8a000640 1 --2- 172.21.15.39:0/2774809512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a8c14b550 0x7f9a8c15c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.373+0000 7f9a8a000640 1 -- 172.21.15.39:0/2774809512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a8c0afc00 msgr2=0x7f9a8c143470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.373+0000 7f9a8a000640 1 --2- 172.21.15.39:0/2774809512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a8c0afc00 0x7f9a8c143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.373+0000 7f9a8a000640 1 -- 172.21.15.39:0/2774809512 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a8c14eda0 con 0x7f9a8c0b4010 2024-09-21T07:32:47.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.373+0000 7f9a8b400640 1 --2- 172.21.15.39:0/2774809512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a8c14b550 0x7f9a8c15c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:32:47.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.374+0000 7f9a8a000640 1 --2- 172.21.15.39:0/2774809512 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a8c0b4010 0x7f9a8c15bf70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c0752a0 tx=0x7f9a78032c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:32:47.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.376+0000 7f9a7f400640 1 -- 172.21.15.39:0/2774809512 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a78041030 con 0x7f9a8c0b4010 2024-09-21T07:32:47.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.377+0000 7f9a7f400640 1 -- 172.21.15.39:0/2774809512 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a78032e20 con 0x7f9a8c0b4010 2024-09-21T07:32:47.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.377+0000 7f9a92200640 1 -- 172.21.15.39:0/2774809512 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a8c14f080 con 0x7f9a8c0b4010 2024-09-21T07:32:47.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.377+0000 7f9a92200640 1 -- 172.21.15.39:0/2774809512 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a8c1439b0 con 0x7f9a8c0b4010 2024-09-21T07:32:47.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.385+0000 7f9a7f400640 1 -- 172.21.15.39:0/2774809512 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a78005020 con 0x7f9a8c0b4010 2024-09-21T07:32:47.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.385+0000 7f9a7f400640 1 -- 172.21.15.39:0/2774809512 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9a78044070 con 0x7f9a8c0b4010 2024-09-21T07:32:47.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.385+0000 7f9a7f400640 1 -- 172.21.15.39:0/2774809512 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9a78004ae0 con 0x7f9a8c0b4010 2024-09-21T07:32:47.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.385+0000 7f9a92200640 1 -- 172.21.15.39:0/2774809512 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a68005290 con 0x7f9a8c0b4010 2024-09-21T07:32:47.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.390+0000 7f9a7f400640 1 -- 172.21.15.39:0/2774809512 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f9a78003800 con 0x7f9a8c0b4010 2024-09-21T07:32:47.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.393+0000 7f70e1e00640 1 -- 172.21.15.39:0/3657657688 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7f70b4005850 con 0x7f70dc072e30 2024-09-21T07:32:47.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.402+0000 7f11ee200640 1 -- 172.21.15.39:0/3774700333 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f11e80d4ee0 con 0x7f11e80b3660 2024-09-21T07:32:47.418 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi039') 2024-09-21T07:32:47.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.430+0000 7f70d3400640 1 -- 172.21.15.39:0/3657657688 <== mon.0 v2:172.21.15.39:3300/0 9 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v8)=0 v8) ==== 117+0+0 (secure 0 0 0) 0x7f70cc003e20 con 0x7f70dc072e30 2024-09-21T07:32:47.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.434+0000 7f70d0c00640 1 -- 172.21.15.39:0/3657657688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f70dc072e30 msgr2=0x7f70dc205520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.434+0000 7f70d0c00640 1 --2- 172.21.15.39:0/3657657688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f70dc072e30 0x7f70dc205520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f70cc02ab60 tx=0x7f70cc003c30 comp rx=0 tx=0).stop 2024-09-21T07:32:47.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.443+0000 7f70d0c00640 1 -- 172.21.15.39:0/3657657688 shutdown_connections 2024-09-21T07:32:47.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.443+0000 7f70d0c00640 1 --2- 172.21.15.39:0/3657657688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70dc15f650 0x7f70dc205c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.443+0000 7f70d0c00640 1 --2- 172.21.15.39:0/3657657688 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f70dc15ea80 0x7f70dc09cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.443+0000 7f70d0c00640 1 --2- 172.21.15.39:0/3657657688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f70dc072e30 0x7f70dc205520 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.443+0000 7f70d0c00640 1 -- 172.21.15.39:0/3657657688 >> 172.21.15.39:0/3657657688 conn(0x7f70dc06d410 msgr2=0x7f70dc06f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:47.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.445+0000 7f1dc2c00640 1 -- 172.21.15.39:0/3826935296 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dbc0b1020 msgr2=0x7f1dbc0b1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.445+0000 7f1dc2c00640 1 --2- 172.21.15.39:0/3826935296 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dbc0b1020 0x7f1dbc0b1430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1db40099b0 tx=0x7f1db402a620 comp rx=0 tx=0).stop 2024-09-21T07:32:47.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.450+0000 7f70d0c00640 1 -- 172.21.15.39:0/3657657688 shutdown_connections 2024-09-21T07:32:47.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.450+0000 7f70d0c00640 1 -- 172.21.15.39:0/3657657688 wait complete. 2024-09-21T07:32:47.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.453+0000 7f1dc2c00640 1 -- 172.21.15.39:0/3826935296 shutdown_connections 2024-09-21T07:32:47.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.453+0000 7f1dc2c00640 1 --2- 172.21.15.39:0/3826935296 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dbc0b1020 0x7f1dbc0b1430 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.453+0000 7f1dc2c00640 1 --2- 172.21.15.39:0/3826935296 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1dbc0d0db0 0x7f1dbc0ad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.453+0000 7f1dc2c00640 1 -- 172.21.15.39:0/3826935296 >> 172.21.15.39:0/3826935296 conn(0x7f1dbc074220 msgr2=0x7f1dbc074630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:47.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.455+0000 7f11e4c00640 1 -- 172.21.15.39:0/3774700333 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v9)=0 v9) ==== 106+0+0 (secure 0 0 0) 0x7f11d403f070 con 0x7f11e80b3660 2024-09-21T07:32:47.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.456+0000 7f11ee200640 1 -- 172.21.15.39:0/3774700333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11e80b3660 msgr2=0x7f11e81f56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.456+0000 7f11ee200640 1 --2- 172.21.15.39:0/3774700333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11e80b3660 0x7f11e81f56c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f11d4031000 tx=0x7f11d4033f90 comp rx=0 tx=0).stop 2024-09-21T07:32:47.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.458+0000 7f1dc2c00640 1 -- 172.21.15.39:0/3826935296 shutdown_connections 2024-09-21T07:32:47.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.458+0000 7f1dc2c00640 1 -- 172.21.15.39:0/3826935296 wait complete. 2024-09-21T07:32:47.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.457+0000 7f11ee200640 1 -- 172.21.15.39:0/3774700333 shutdown_connections 2024-09-21T07:32:47.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.457+0000 7f11ee200640 1 --2- 172.21.15.39:0/3774700333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11e80d0bb0 0x7f11e815a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.457+0000 7f11ee200640 1 --2- 172.21.15.39:0/3774700333 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11e80b9540 0x7f11e8157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.457+0000 7f11ee200640 1 --2- 172.21.15.39:0/3774700333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11e80b3660 0x7f11e81f56c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.457+0000 7f11ee200640 1 -- 172.21.15.39:0/3774700333 >> 172.21.15.39:0/3774700333 conn(0x7f11e809d680 msgr2=0x7f11e806f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:47.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.457+0000 7f11ee200640 1 -- 172.21.15.39:0/3774700333 shutdown_connections 2024-09-21T07:32:47.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.457+0000 7f11ee200640 1 -- 172.21.15.39:0/3774700333 wait complete. 2024-09-21T07:32:47.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.478+0000 7f1dc2c00640 1 Processor -- start 2024-09-21T07:32:47.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.478+0000 7f1dc2c00640 1 -- start start 2024-09-21T07:32:47.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.492+0000 7f1dc2c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1dbc074d40 0x7f1dbc14a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.496+0000 7f1dc2c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1dbc0b1020 0x7f1dbc14c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.496+0000 7f1dc2c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dbc0d0db0 0x7f1dbc14ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.496+0000 7f1dc2c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc073760 con 0x7f1dbc0b1020 2024-09-21T07:32:47.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.496+0000 7f1dc2c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc0b1b80 con 0x7f1dbc0d0db0 2024-09-21T07:32:47.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.496+0000 7f1dc2c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f1dbc162450 con 0x7f1dbc074d40 2024-09-21T07:32:47.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.498+0000 7f1dbbe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dbc0d0db0 0x7f1dbc14ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:47.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.498+0000 7f1dbbe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dbc0d0db0 0x7f1dbc14ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.39:40312/0 (socket says 172.21.15.39:40312) 2024-09-21T07:32:47.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.499+0000 7f1dbbe00640 1 -- 172.21.15.39:0/186420665 learned_addr learned my addr 172.21.15.39:0/186420665 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:32:47.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.505+0000 7f1dbbe00640 1 -- 172.21.15.39:0/186420665 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1dbc074d40 msgr2=0x7f1dbc14a7a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T07:32:47.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.505+0000 7f1dbbe00640 1 --2- 172.21.15.39:0/186420665 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1dbc074d40 0x7f1dbc14a7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.505+0000 7f1dbbe00640 1 -- 172.21.15.39:0/186420665 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1dbc0b1020 msgr2=0x7f1dbc14c550 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T07:32:47.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.509+0000 7f1dbaa00640 1 --2- 172.21.15.39:0/186420665 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1dbc0b1020 0x7f1dbc14c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:47.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.509+0000 7f1dbbe00640 1 --2- 172.21.15.39:0/186420665 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1dbc0b1020 0x7f1dbc14c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.509+0000 7f1dbbe00640 1 -- 172.21.15.39:0/186420665 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dbc14ace0 con 0x7f1dbc0d0db0 2024-09-21T07:32:47.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.514+0000 7f1dbbe00640 1 --2- 172.21.15.39:0/186420665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dbc0d0db0 0x7f1dbc14ca90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1db4009ad0 tx=0x7f1db40024a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:32:47.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.519+0000 7f1dafe00640 1 -- 172.21.15.39:0/186420665 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db4032d10 con 0x7f1dbc0d0db0 2024-09-21T07:32:47.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.519+0000 7f1dafe00640 1 -- 172.21.15.39:0/186420665 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1db4040070 con 0x7f1dbc0d0db0 2024-09-21T07:32:47.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.519+0000 7f1dafe00640 1 -- 172.21.15.39:0/186420665 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db403b660 con 0x7f1dbc0d0db0 2024-09-21T07:32:47.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.526+0000 7f1dc2c00640 1 -- 172.21.15.39:0/186420665 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dbc1fabe0 con 0x7f1dbc0d0db0 2024-09-21T07:32:47.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.526+0000 7f1dafe00640 1 -- 172.21.15.39:0/186420665 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1db403a070 con 0x7f1dbc0d0db0 2024-09-21T07:32:47.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.524+0000 7fab1a600640 1 -- 172.21.15.39:0/2467534450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faafc003b40 msgr2=0x7faafc003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.524+0000 7fab1a600640 1 --2- 172.21.15.39:0/2467534450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faafc003b40 0x7faafc003f70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fab08009ce0 tx=0x7faafc005830 comp rx=0 tx=0).stop 2024-09-21T07:32:47.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.527+0000 7fab1a600640 1 -- 172.21.15.39:0/2467534450 shutdown_connections 2024-09-21T07:32:47.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.527+0000 7fab1a600640 1 --2- 172.21.15.39:0/2467534450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faafc003b40 0x7faafc003f70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.527+0000 7fab1a600640 1 -- 172.21.15.39:0/2467534450 >> 172.21.15.39:0/2467534450 conn(0x7fab0c01db80 msgr2=0x7fab0c039840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:47.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.528+0000 7fab1a600640 1 -- 172.21.15.39:0/2467534450 shutdown_connections 2024-09-21T07:32:47.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.528+0000 7fab1a600640 1 -- 172.21.15.39:0/2467534450 wait complete. 2024-09-21T07:32:47.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.530+0000 7f1dc2c00640 1 -- 172.21.15.39:0/186420665 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dbc14b1a0 con 0x7f1dbc0d0db0 2024-09-21T07:32:47.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.536+0000 7f1dad600640 1 -- 172.21.15.39:0/186420665 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d90005290 con 0x7f1dbc0d0db0 2024-09-21T07:32:47.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.536+0000 7f1dafe00640 1 -- 172.21.15.39:0/186420665 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1db4044530 con 0x7f1dbc0d0db0 2024-09-21T07:32:47.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.541+0000 7f1dafe00640 1 -- 172.21.15.39:0/186420665 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f1dbc14ace0 con 0x7f1dbc0d0db0 2024-09-21T07:32:47.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.540+0000 7fab1a600640 1 Processor -- start 2024-09-21T07:32:47.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.541+0000 7fab1a600640 1 -- start start 2024-09-21T07:32:47.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.559+0000 7fab1a600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faafc003b40 0x7fab0c0dfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.559+0000 7fab1a600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab0c0e0110 0x7fab0c0d9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.559+0000 7fab1a600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab0c0dae40 0x7fab0c0da230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.560+0000 7fab1a600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c0ddbb0 con 0x7fab0c0dae40 2024-09-21T07:32:47.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.560+0000 7fab1a600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c035e20 con 0x7fab0c0e0110 2024-09-21T07:32:47.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.560+0000 7fab1a600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fab0c17ec30 con 0x7faafc003b40 2024-09-21T07:32:47.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.561+0000 7fab19000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab0c0dae40 0x7fab0c0da230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:47.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.561+0000 7fab19000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab0c0dae40 0x7fab0c0da230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54876/0 (socket says 172.21.15.39:54876) 2024-09-21T07:32:47.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.561+0000 7fab19000640 1 -- 172.21.15.39:0/1475568403 learned_addr learned my addr 172.21.15.39:0/1475568403 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:32:47.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.565+0000 7fab19000640 1 -- 172.21.15.39:0/1475568403 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faafc003b40 msgr2=0x7fab0c0dfbd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T07:32:47.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.565+0000 7fab19000640 1 --2- 172.21.15.39:0/1475568403 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faafc003b40 0x7fab0c0dfbd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.565+0000 7fab19000640 1 -- 172.21.15.39:0/1475568403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab0c0e0110 msgr2=0x7fab0c0d9cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T07:32:47.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.565+0000 7fab19000640 1 --2- 172.21.15.39:0/1475568403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab0c0e0110 0x7fab0c0d9cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.565+0000 7fab19000640 1 -- 172.21.15.39:0/1475568403 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab0c0e2a10 con 0x7fab0c0dae40 2024-09-21T07:32:47.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.566+0000 7fab19000640 1 --2- 172.21.15.39:0/1475568403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab0c0dae40 0x7fab0c0da230 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fab14051230 tx=0x7fab14064940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:32:47.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.567+0000 7fab07400640 1 -- 172.21.15.39:0/1475568403 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab14069250 con 0x7fab0c0dae40 2024-09-21T07:32:47.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.567+0000 7fab1a600640 1 -- 172.21.15.39:0/1475568403 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab0c17e760 con 0x7fab0c0dae40 2024-09-21T07:32:47.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.567+0000 7fab1a600640 1 -- 172.21.15.39:0/1475568403 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab0c0e2f60 con 0x7fab0c0dae40 2024-09-21T07:32:47.575 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi039') 2024-09-21T07:32:47.575 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi039') 2024-09-21T07:32:47.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.571+0000 7fab07400640 1 -- 172.21.15.39:0/1475568403 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fab14092070 con 0x7fab0c0dae40 2024-09-21T07:32:47.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.571+0000 7fab07400640 1 -- 172.21.15.39:0/1475568403 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab14069630 con 0x7fab0c0dae40 2024-09-21T07:32:47.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.574+0000 7fab1a600640 1 -- 172.21.15.39:0/1475568403 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaf0005290 con 0x7fab0c0dae40 2024-09-21T07:32:47.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.576+0000 7fab07400640 1 -- 172.21.15.39:0/1475568403 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fab140697d0 con 0x7fab0c0dae40 2024-09-21T07:32:47.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.578+0000 7fab07400640 1 -- 172.21.15.39:0/1475568403 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fab140a03d0 con 0x7fab0c0dae40 2024-09-21T07:32:47.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.583+0000 7fcb4cc00640 1 -- 172.21.15.39:0/2971336290 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7fcb5c0b41f0 con 0x7fcb5c15b970 2024-09-21T07:32:47.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.583+0000 7fab07400640 1 -- 172.21.15.39:0/1475568403 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fab14097460 con 0x7fab0c0dae40 2024-09-21T07:32:47.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.598+0000 7fcb4f400640 1 -- 172.21.15.39:0/2971336290 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v10)=0 v10) ==== 115+0+0 (secure 0 0 0) 0x7fcb50019d70 con 0x7fcb5c15b970 2024-09-21T07:32:47.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.612+0000 7fcb4cc00640 1 -- 172.21.15.39:0/2971336290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb5c15b970 msgr2=0x7fcb5c14a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.612+0000 7fcb4cc00640 1 --2- 172.21.15.39:0/2971336290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb5c15b970 0x7fcb5c14a930 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000b410 tx=0x7fcb5000b8d0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.614+0000 7fcb4cc00640 1 -- 172.21.15.39:0/2971336290 shutdown_connections 2024-09-21T07:32:47.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.614+0000 7fcb4cc00640 1 --2- 172.21.15.39:0/2971336290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb5c15de10 0x7fcb5c14e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.614+0000 7fcb4cc00640 1 --2- 172.21.15.39:0/2971336290 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcb5c15da00 0x7fcb5c14e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.614+0000 7fcb4cc00640 1 --2- 172.21.15.39:0/2971336290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb5c15b970 0x7fcb5c14a930 secure :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000b410 tx=0x7fcb5000b8d0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.614+0000 7fcb4cc00640 1 -- 172.21.15.39:0/2971336290 >> 172.21.15.39:0/2971336290 conn(0x7fcb5c074220 msgr2=0x7fcb5c154770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:47.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.617+0000 7fcb4cc00640 1 -- 172.21.15.39:0/2971336290 shutdown_connections 2024-09-21T07:32:47.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.617+0000 7fcb4cc00640 1 -- 172.21.15.39:0/2971336290 wait complete. 2024-09-21T07:32:47.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.629+0000 7fd313a00640 1 -- 172.21.15.39:0/1673095953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd304056dd0 msgr2=0x7fd304057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.629+0000 7fd313a00640 1 --2- 172.21.15.39:0/1673095953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd304056dd0 0x7fd304057cb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd300009830 tx=0x7fd30002a340 comp rx=0 tx=0).stop 2024-09-21T07:32:47.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.631+0000 7fd313a00640 1 -- 172.21.15.39:0/1673095953 shutdown_connections 2024-09-21T07:32:47.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.633+0000 7fd313a00640 1 --2- 172.21.15.39:0/1673095953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd304056dd0 0x7fd304057cb0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.633+0000 7fd313a00640 1 -- 172.21.15.39:0/1673095953 >> 172.21.15.39:0/1673095953 conn(0x7fd30403a620 msgr2=0x7fd30403aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:47.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.633+0000 7fd313a00640 1 -- 172.21.15.39:0/1673095953 shutdown_connections 2024-09-21T07:32:47.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.633+0000 7fd313a00640 1 -- 172.21.15.39:0/1673095953 wait complete. 2024-09-21T07:32:47.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.636+0000 7fb43cc00640 1 -- 172.21.15.39:0/1825455855 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb41c0051c0 msgr2=0x7fb41c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.636+0000 7fb43cc00640 1 --2- 172.21.15.39:0/1825455855 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb41c0051c0 0x7fb41c00b9c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb43816b8d0 tx=0x7fb41c005830 comp rx=0 tx=0).stop 2024-09-21T07:32:47.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.643+0000 7fb437e00640 1 -- 172.21.15.39:0/1825455855 reap_dead start 2024-09-21T07:32:47.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.643+0000 7fb43cc00640 1 -- 172.21.15.39:0/1825455855 shutdown_connections 2024-09-21T07:32:47.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.643+0000 7fb43cc00640 1 -- 172.21.15.39:0/1825455855 >> 172.21.15.39:0/1825455855 conn(0x7fb43806d5d0 msgr2=0x7fb43806ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:47.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.644+0000 7fd313a00640 1 Processor -- start 2024-09-21T07:32:47.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.650+0000 7fb43cc00640 1 -- 172.21.15.39:0/1825455855 shutdown_connections 2024-09-21T07:32:47.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.644+0000 7fd313a00640 1 -- start start 2024-09-21T07:32:47.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.650+0000 7fb43cc00640 1 -- 172.21.15.39:0/1825455855 wait complete. 2024-09-21T07:32:47.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.651+0000 7fbc55e00640 1 -- 172.21.15.39:0/2310902452 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7fbc50145560 con 0x7fbc500d0340 2024-09-21T07:32:47.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.658+0000 7fd313a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd304054390 0x7fd3040dd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.658+0000 7fd313a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd304056dd0 0x7fd3040dba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.659+0000 7fd313a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd3040e2f70 0x7fd3040dbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.659+0000 7fd313a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd304007450 con 0x7fd304054390 2024-09-21T07:32:47.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.659+0000 7fd313a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd3040072d0 con 0x7fd304056dd0 2024-09-21T07:32:47.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.659+0000 7fd313a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd3040075d0 con 0x7fd3040e2f70 2024-09-21T07:32:47.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.660+0000 7fb43cc00640 1 Processor -- start 2024-09-21T07:32:47.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.661+0000 7fd311000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd304054390 0x7fd3040dd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:47.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.661+0000 7fd311000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd304054390 0x7fd3040dd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54922/0 (socket says 172.21.15.39:54922) 2024-09-21T07:32:47.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.661+0000 7fd311000640 1 -- 172.21.15.39:0/2305791737 learned_addr learned my addr 172.21.15.39:0/2305791737 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:32:47.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.661+0000 7fb43cc00640 1 -- start start 2024-09-21T07:32:47.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.666+0000 7fbc4cc00640 1 -- 172.21.15.39:0/2310902452 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v11)=0 v11) ==== 122+0+0 (secure 0 0 0) 0x7fbc3c031440 con 0x7fbc500d0340 2024-09-21T07:32:47.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.662+0000 7fd311000640 1 -- 172.21.15.39:0/2305791737 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd3040e2f70 msgr2=0x7fd3040dbf60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T07:32:47.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.664+0000 7fd30be00640 1 --2- 172.21.15.39:0/2305791737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd304056dd0 0x7fd3040dba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:47.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.664+0000 7fd311a00640 1 --2- 172.21.15.39:0/2305791737 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd3040e2f70 0x7fd3040dbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:47.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.668+0000 7fd311000640 1 --2- 172.21.15.39:0/2305791737 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd3040e2f70 0x7fd3040dbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.668+0000 7fd311000640 1 -- 172.21.15.39:0/2305791737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd304056dd0 msgr2=0x7fd3040dba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.668+0000 7fd311000640 1 --2- 172.21.15.39:0/2305791737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd304056dd0 0x7fd3040dba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.669+0000 7fd311000640 1 -- 172.21.15.39:0/2305791737 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd30404f5d0 con 0x7fd304054390 2024-09-21T07:32:47.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.669+0000 7fd30be00640 1 --2- 172.21.15.39:0/2305791737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd304056dd0 0x7fd3040dba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:32:47.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.669+0000 7fd311a00640 1 --2- 172.21.15.39:0/2305791737 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd3040e2f70 0x7fd3040dbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:32:47.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.670+0000 7fd311000640 1 --2- 172.21.15.39:0/2305791737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd304054390 0x7fd3040dd230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd300009780 tx=0x7fd300033f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:32:47.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.673+0000 7fbc55e00640 1 -- 172.21.15.39:0/2310902452 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc500d0340 msgr2=0x7fbc501f6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.673+0000 7fbc55e00640 1 --2- 172.21.15.39:0/2310902452 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc500d0340 0x7fbc501f6e80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c009850 tx=0x7fbc3c0042e0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.673+0000 7fbc55e00640 1 -- 172.21.15.39:0/2310902452 shutdown_connections 2024-09-21T07:32:47.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.673+0000 7fbc55e00640 1 --2- 172.21.15.39:0/2310902452 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc5015ca30 0x7fbc5015f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.673+0000 7fbc55e00640 1 --2- 172.21.15.39:0/2310902452 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc500d0340 0x7fbc501f6e80 secure :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c009850 tx=0x7fbc3c0042e0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.673+0000 7fbc55e00640 1 --2- 172.21.15.39:0/2310902452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc5006f9f0 0x7fbc501f6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.673+0000 7fbc55e00640 1 -- 172.21.15.39:0/2310902452 >> 172.21.15.39:0/2310902452 conn(0x7fbc50070b20 msgr2=0x7fbc50144cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:47.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.674+0000 7fbc55e00640 1 -- 172.21.15.39:0/2310902452 shutdown_connections 2024-09-21T07:32:47.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.675+0000 7fb43cc00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4380b5a60 0x7fb4381665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.675+0000 7fd308c00640 1 -- 172.21.15.39:0/2305791737 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd300037040 con 0x7fd304054390 2024-09-21T07:32:47.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.674+0000 7fbc55e00640 1 -- 172.21.15.39:0/2310902452 wait complete. 2024-09-21T07:32:47.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.675+0000 7fd308c00640 1 -- 172.21.15.39:0/2305791737 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd300042070 con 0x7fd304054390 2024-09-21T07:32:47.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.676+0000 7fd313a00640 1 -- 172.21.15.39:0/2305791737 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd30404cee0 con 0x7fd304054390 2024-09-21T07:32:47.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.676+0000 7fd313a00640 1 -- 172.21.15.39:0/2305791737 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd30404d4a0 con 0x7fd304054390 2024-09-21T07:32:47.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.675+0000 7f9a92200640 1 -- 172.21.15.39:0/2774809512 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f9a68005580 con 0x7f9a8c0b4010 2024-09-21T07:32:47.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.675+0000 7fb43cc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb438205d40 0x7fb4380cf460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.679+0000 7fb43cc00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4380cf9a0 0x7fb4380b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.679+0000 7fb43cc00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb4380cc630 con 0x7fb438205d40 2024-09-21T07:32:47.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.679+0000 7fb43cc00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb4380cb580 con 0x7fb4380b5a60 2024-09-21T07:32:47.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.679+0000 7fb437e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4380cf9a0 0x7fb4380b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:47.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.679+0000 7fb437e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4380cf9a0 0x7fb4380b2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.39:37634/0 (socket says 172.21.15.39:37634) 2024-09-21T07:32:47.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.679+0000 7fb437e00640 1 -- 172.21.15.39:0/3585518072 learned_addr learned my addr 172.21.15.39:0/3585518072 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:32:47.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.680+0000 7fb43cc00640 1 -- 172.21.15.39:0/3585518072 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb4380cc7b0 con 0x7fb4380cf9a0 2024-09-21T07:32:47.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.680+0000 7fb437400640 1 --2- 172.21.15.39:0/3585518072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4380b5a60 0x7fb4381665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:47.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.680+0000 7fb437e00640 1 -- 172.21.15.39:0/3585518072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4380b5a60 msgr2=0x7fb4381665c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.680+0000 7fb437e00640 1 --2- 172.21.15.39:0/3585518072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4380b5a60 0x7fb4381665c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.680+0000 7fb437e00640 1 -- 172.21.15.39:0/3585518072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb438205d40 msgr2=0x7fb4380cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.680+0000 7fb437e00640 1 --2- 172.21.15.39:0/3585518072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb438205d40 0x7fb4380cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.680+0000 7fb437e00640 1 -- 172.21.15.39:0/3585518072 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4380cfdb0 con 0x7fb4380cf9a0 2024-09-21T07:32:47.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.681+0000 7fd308c00640 1 -- 172.21.15.39:0/2305791737 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd300003940 con 0x7fd304054390 2024-09-21T07:32:47.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.680+0000 7fb437400640 1 --2- 172.21.15.39:0/3585518072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4380b5a60 0x7fb4381665c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:32:47.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.685+0000 7fb437e00640 1 --2- 172.21.15.39:0/3585518072 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4380cf9a0 0x7fb4380b2f20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb4300334c0 tx=0x7fb430004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:32:47.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.685+0000 7fb42f400640 1 -- 172.21.15.39:0/3585518072 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43004f070 con 0x7fb4380cf9a0 2024-09-21T07:32:47.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.685+0000 7fb43cc00640 1 -- 172.21.15.39:0/3585518072 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb438201f50 con 0x7fb4380cf9a0 2024-09-21T07:32:47.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.686+0000 7fb43cc00640 1 -- 172.21.15.39:0/3585518072 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb438202510 con 0x7fb4380cf9a0 2024-09-21T07:32:47.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.686+0000 7fb42f400640 1 -- 172.21.15.39:0/3585518072 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb430003b70 con 0x7fb4380cf9a0 2024-09-21T07:32:47.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.689+0000 7fb42f400640 1 -- 172.21.15.39:0/3585518072 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb430009b50 con 0x7fb4380cf9a0 2024-09-21T07:32:47.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.691+0000 7f9a7f400640 1 -- 172.21.15.39:0/2774809512 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v12)=0 v12) ==== 121+0+0 (secure 0 0 0) 0x7f9a78003800 con 0x7f9a8c0b4010 2024-09-21T07:32:47.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.691+0000 7fd308c00640 1 -- 172.21.15.39:0/2305791737 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd300033020 con 0x7fd304054390 2024-09-21T07:32:47.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.691+0000 7fd308c00640 1 -- 172.21.15.39:0/2305791737 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd300032ab0 con 0x7fd304054390 2024-09-21T07:32:47.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.691+0000 7fd313a00640 1 -- 172.21.15.39:0/2305791737 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2e8005290 con 0x7fd304054390 2024-09-21T07:32:47.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.695+0000 7f9a92200640 1 -- 172.21.15.39:0/2774809512 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a8c0b4010 msgr2=0x7f9a8c15bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.695+0000 7f9a92200640 1 --2- 172.21.15.39:0/2774809512 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a8c0b4010 0x7f9a8c15bf70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c0752a0 tx=0x7f9a78032c30 comp rx=0 tx=0).stop 2024-09-21T07:32:47.701 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi039') 2024-09-21T07:32:47.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.699+0000 7fb42f400640 1 -- 172.21.15.39:0/3585518072 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb430052070 con 0x7fb4380cf9a0 2024-09-21T07:32:47.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.699+0000 7fb42f400640 1 -- 172.21.15.39:0/3585518072 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb430045990 con 0x7fb4380cf9a0 2024-09-21T07:32:47.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.700+0000 7fb43cc00640 1 -- 172.21.15.39:0/3585518072 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb410005290 con 0x7fb4380cf9a0 2024-09-21T07:32:47.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.703+0000 7f9a92200640 1 -- 172.21.15.39:0/2774809512 shutdown_connections 2024-09-21T07:32:47.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.703+0000 7f9a92200640 1 --2- 172.21.15.39:0/2774809512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a8c14b550 0x7f9a8c15c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.703+0000 7f9a92200640 1 --2- 172.21.15.39:0/2774809512 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a8c0b4010 0x7f9a8c15bf70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.703+0000 7f9a92200640 1 --2- 172.21.15.39:0/2774809512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a8c0afc00 0x7f9a8c143470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.703+0000 7f9a92200640 1 -- 172.21.15.39:0/2774809512 >> 172.21.15.39:0/2774809512 conn(0x7f9a8c074220 msgr2=0x7f9a8c156530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:47.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.703+0000 7fd308c00640 1 -- 172.21.15.39:0/2305791737 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fd30004b060 con 0x7fd304054390 2024-09-21T07:32:47.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.703+0000 7f9a92200640 1 -- 172.21.15.39:0/2774809512 shutdown_connections 2024-09-21T07:32:47.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.703+0000 7f9a92200640 1 -- 172.21.15.39:0/2774809512 wait complete. 2024-09-21T07:32:47.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.715+0000 7fb42f400640 1 -- 172.21.15.39:0/3585518072 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fb430059020 con 0x7fb4380cf9a0 2024-09-21T07:32:47.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.715+0000 7fbb49e00640 1 -- 172.21.15.39:0/2041673106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb4406ee70 msgr2=0x7fbb4406f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.715+0000 7fbb49e00640 1 --2- 172.21.15.39:0/2041673106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb4406ee70 0x7fbb4406f280 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbb44075240 tx=0x7fbb3002b020 comp rx=0 tx=0).stop 2024-09-21T07:32:47.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.715+0000 7fbb49e00640 1 -- 172.21.15.39:0/2041673106 shutdown_connections 2024-09-21T07:32:47.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.715+0000 7fbb49e00640 1 --2- 172.21.15.39:0/2041673106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb4406ee70 0x7fbb4406f280 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.715+0000 7fbb49e00640 1 --2- 172.21.15.39:0/2041673106 >> v2:172.21.15.39:3300/0 conn(0x7fbb440b3620 0x7fbb440b3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.715+0000 7fbb49e00640 1 -- 172.21.15.39:0/2041673106 >> 172.21.15.39:0/2041673106 conn(0x7fbb44070b20 msgr2=0x7fbb44070f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:47.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.716+0000 7fbb49e00640 1 -- 172.21.15.39:0/2041673106 shutdown_connections 2024-09-21T07:32:47.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.716+0000 7fbb49e00640 1 -- 172.21.15.39:0/2041673106 wait complete. 2024-09-21T07:32:47.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.721+0000 7fbb49e00640 1 Processor -- start 2024-09-21T07:32:47.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.721+0000 7fbb49e00640 1 -- start start 2024-09-21T07:32:47.724 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi039') 2024-09-21T07:32:47.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.725+0000 7fbb49e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbb4406ee70 0x7fbb440b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.727+0000 7fbb49e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb440b3620 0x7fbb441536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.728+0000 7fbb48a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbb4406ee70 0x7fbb440b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:47.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.728+0000 7fbb48a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbb4406ee70 0x7fbb440b18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.39:37646/0 (socket says 172.21.15.39:37646) 2024-09-21T07:32:47.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.728+0000 7fbb48a00640 1 -- 172.21.15.39:0/3837181708 learned_addr learned my addr 172.21.15.39:0/3837181708 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:32:47.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.728+0000 7fbb49e00640 1 --2- 172.21.15.39:0/3837181708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb440cf950 0x7fbb441f17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.729+0000 7fbb49e00640 1 -- 172.21.15.39:0/3837181708 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbb440d04c0 con 0x7fbb440cf950 2024-09-21T07:32:47.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.729+0000 7fbb43e00640 1 --2- 172.21.15.39:0/3837181708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb440b3620 0x7fbb441536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:47.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.729+0000 7fbb49400640 1 --2- 172.21.15.39:0/3837181708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb440cf950 0x7fbb441f17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:47.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.729+0000 7fbb49e00640 1 -- 172.21.15.39:0/3837181708 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbb440d0340 con 0x7fbb440b3620 2024-09-21T07:32:47.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.729+0000 7fbb49e00640 1 -- 172.21.15.39:0/3837181708 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fbb440b3f70 con 0x7fbb4406ee70 2024-09-21T07:32:47.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.731+0000 7fbb48a00640 1 -- 172.21.15.39:0/3837181708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb440b3620 msgr2=0x7fbb441536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.731+0000 7fbb48a00640 1 --2- 172.21.15.39:0/3837181708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb440b3620 0x7fbb441536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.732+0000 7fbb48a00640 1 -- 172.21.15.39:0/3837181708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb440cf950 msgr2=0x7fbb441f17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.735+0000 7fbb48a00640 1 --2- 172.21.15.39:0/3837181708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb440cf950 0x7fbb441f17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.735+0000 7fbb48a00640 1 -- 172.21.15.39:0/3837181708 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb441f5670 con 0x7fbb4406ee70 2024-09-21T07:32:47.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.736+0000 7fbb43e00640 1 --2- 172.21.15.39:0/3837181708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb440b3620 0x7fbb441536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:32:47.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.736+0000 7fbb49400640 1 --2- 172.21.15.39:0/3837181708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb440cf950 0x7fbb441f17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:32:47.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.736+0000 7fbb48a00640 1 --2- 172.21.15.39:0/3837181708 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbb4406ee70 0x7fbb440b18f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbb3400c4d0 tx=0x7fbb3400c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:32:47.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.740+0000 7fbb40c00640 1 -- 172.21.15.39:0/3837181708 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb34016020 con 0x7fbb4406ee70 2024-09-21T07:32:47.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.740+0000 7fbb40c00640 1 -- 172.21.15.39:0/3837181708 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb340129e0 con 0x7fbb4406ee70 2024-09-21T07:32:47.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.740+0000 7fbb49e00640 1 -- 172.21.15.39:0/3837181708 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb441f4dd0 con 0x7fbb4406ee70 2024-09-21T07:32:47.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.740+0000 7fbb49e00640 1 -- 172.21.15.39:0/3837181708 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb441f5390 con 0x7fbb4406ee70 2024-09-21T07:32:47.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.742+0000 7fbb3a000640 1 -- 172.21.15.39:0/3837181708 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb1c005290 con 0x7fbb4406ee70 2024-09-21T07:32:47.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.743+0000 7fbb40c00640 1 -- 172.21.15.39:0/3837181708 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb34012e30 con 0x7fbb4406ee70 2024-09-21T07:32:47.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.744+0000 7fbb40c00640 1 -- 172.21.15.39:0/3837181708 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbb34019020 con 0x7fbb4406ee70 2024-09-21T07:32:47.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.744+0000 7fbb40c00640 1 -- 172.21.15.39:0/3837181708 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbb3401a8e0 con 0x7fbb4406ee70 2024-09-21T07:32:47.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.747+0000 7fbb40c00640 1 -- 172.21.15.39:0/3837181708 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fbb34020020 con 0x7fbb4406ee70 2024-09-21T07:32:47.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.750+0000 7f529ec00640 1 -- 172.21.15.39:0/2530741888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f527c004800 msgr2=0x7f527c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.750+0000 7f529ec00640 1 --2- 172.21.15.39:0/2530741888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f527c004800 0x7f527c004c10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5284003f00 tx=0x7f527c005830 comp rx=0 tx=0).stop 2024-09-21T07:32:47.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.750+0000 7f529ec00640 1 -- 172.21.15.39:0/2530741888 shutdown_connections 2024-09-21T07:32:47.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.750+0000 7f529ec00640 1 --2- 172.21.15.39:0/2530741888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f527c004800 0x7f527c004c10 secure :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5284003f00 tx=0x7f527c005830 comp rx=0 tx=0).stop 2024-09-21T07:32:47.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.750+0000 7f529ec00640 1 -- 172.21.15.39:0/2530741888 >> 172.21.15.39:0/2530741888 conn(0x7f529806d410 msgr2=0x7f52980b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:47.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.754+0000 7f529ec00640 1 -- 172.21.15.39:0/2530741888 shutdown_connections 2024-09-21T07:32:47.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.755+0000 7f529ec00640 1 -- 172.21.15.39:0/2530741888 wait complete. 2024-09-21T07:32:47.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.755+0000 7f3b39800640 1 -- 172.21.15.39:0/663853757 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b18004800 msgr2=0x7f3b18004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.755+0000 7f3b39800640 1 --2- 172.21.15.39:0/663853757 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b18004800 0x7f3b18004c10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3b20001eb0 tx=0x7f3b18005830 comp rx=0 tx=0).stop 2024-09-21T07:32:47.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.757+0000 7f3b39800640 1 -- 172.21.15.39:0/663853757 shutdown_connections 2024-09-21T07:32:47.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.757+0000 7f3b39800640 1 --2- 172.21.15.39:0/663853757 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b18004800 0x7f3b18004c10 secure :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3b20001eb0 tx=0x7f3b18005830 comp rx=0 tx=0).stop 2024-09-21T07:32:47.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.757+0000 7f3b39800640 1 -- 172.21.15.39:0/663853757 >> 172.21.15.39:0/663853757 conn(0x7f3b3406d410 msgr2=0x7f3b340b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:47.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.759+0000 7f3b39800640 1 -- 172.21.15.39:0/663853757 shutdown_connections 2024-09-21T07:32:47.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.759+0000 7f3b39800640 1 -- 172.21.15.39:0/663853757 wait complete. 2024-09-21T07:32:47.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.759+0000 7f529ec00640 1 Processor -- start 2024-09-21T07:32:47.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.760+0000 7f529ec00640 1 -- start start 2024-09-21T07:32:47.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.767+0000 7f3b39800640 1 Processor -- start 2024-09-21T07:32:47.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.767+0000 7f3b39800640 1 -- start start 2024-09-21T07:32:47.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.768+0000 7f529ec00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52981f8a30 0x7f52980acf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.769+0000 7f529ec00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52981f8e40 0x7f52981479d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.771+0000 7f529ec00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5298147f10 0x7f529814a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.771+0000 7f5297e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52981f8a30 0x7f52980acf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:47.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.771+0000 7f5297e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52981f8a30 0x7f52980acf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.39:40398/0 (socket says 172.21.15.39:40398) 2024-09-21T07:32:47.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.771+0000 7f529ec00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5298162f20 con 0x7f5298147f10 2024-09-21T07:32:47.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.772+0000 7f529ec00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f52980b4570 con 0x7f52981f8a30 2024-09-21T07:32:47.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.772+0000 7f529ec00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f52981630a0 con 0x7f52981f8e40 2024-09-21T07:32:47.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.773+0000 7f5297400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52981f8e40 0x7f52981479d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:47.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.773+0000 7f5297400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52981f8e40 0x7f52981479d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.39:37694/0 (socket says 172.21.15.39:37694) 2024-09-21T07:32:47.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.773+0000 7f5297400640 1 -- 172.21.15.39:0/302955748 learned_addr learned my addr 172.21.15.39:0/302955748 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:32:47.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.773+0000 7f529cc00640 1 --2- 172.21.15.39:0/302955748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5298147f10 0x7f529814a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:47.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.774+0000 7f3b39800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b34144a20 0x7f3b341f6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.774+0000 7f3b39800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b340aea30 0x7f3b341442d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.774+0000 7f3b33e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b34144a20 0x7f3b341f6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:47.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.774+0000 7f3b39800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b340b1740 0x7f3b340b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.774+0000 7f3b33e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b34144a20 0x7f3b341f6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.39:37702/0 (socket says 172.21.15.39:37702) 2024-09-21T07:32:47.777 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi039') 2024-09-21T07:32:47.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.774+0000 7f3b33400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b340aea30 0x7f3b341442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:47.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.775+0000 7f3b33e00640 1 -- 172.21.15.39:0/3038123208 learned_addr learned my addr 172.21.15.39:0/3038123208 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:32:47.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.775+0000 7f3b38e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b340b1740 0x7f3b340b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:47.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.775+0000 7f3b39800640 1 -- 172.21.15.39:0/3038123208 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3b340ccc10 con 0x7f3b340b1740 2024-09-21T07:32:47.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.775+0000 7f3b39800640 1 -- 172.21.15.39:0/3038123208 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3b340cca90 con 0x7f3b340aea30 2024-09-21T07:32:47.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.775+0000 7f3b39800640 1 -- 172.21.15.39:0/3038123208 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f3b340ccd90 con 0x7f3b34144a20 2024-09-21T07:32:47.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.775+0000 7f5297e00640 1 -- 172.21.15.39:0/302955748 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52981f8e40 msgr2=0x7f52981479d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.775+0000 7f5297e00640 1 --2- 172.21.15.39:0/302955748 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52981f8e40 0x7f52981479d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.775+0000 7f5297e00640 1 -- 172.21.15.39:0/302955748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5298147f10 msgr2=0x7f529814a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.775+0000 7f5297e00640 1 --2- 172.21.15.39:0/302955748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5298147f10 0x7f529814a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.775+0000 7f5297e00640 1 -- 172.21.15.39:0/302955748 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5298148320 con 0x7f52981f8a30 2024-09-21T07:32:47.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.776+0000 7f5297400640 1 --2- 172.21.15.39:0/302955748 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52981f8e40 0x7f52981479d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:32:47.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.777+0000 7f5297e00640 1 --2- 172.21.15.39:0/302955748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52981f8a30 0x7f52980acf90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f528800bee0 tx=0x7f52880039c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:32:47.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.777+0000 7f3b33e00640 1 -- 172.21.15.39:0/3038123208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b340aea30 msgr2=0x7f3b341442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.777+0000 7f3b33e00640 1 --2- 172.21.15.39:0/3038123208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b340aea30 0x7f3b341442d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.777+0000 7f3b33e00640 1 -- 172.21.15.39:0/3038123208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b340b1740 msgr2=0x7f3b340b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.777+0000 7f3b33e00640 1 --2- 172.21.15.39:0/3038123208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b340b1740 0x7f3b340b1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.777+0000 7f528fe00640 1 -- 172.21.15.39:0/302955748 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5288005220 con 0x7f52981f8a30 2024-09-21T07:32:47.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.777+0000 7f528fe00640 1 -- 172.21.15.39:0/302955748 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52880053c0 con 0x7f52981f8a30 2024-09-21T07:32:47.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.777+0000 7f528fe00640 1 -- 172.21.15.39:0/302955748 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52880047a0 con 0x7f52981f8a30 2024-09-21T07:32:47.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.778+0000 7f529ec00640 1 -- 172.21.15.39:0/302955748 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52980ad8e0 con 0x7f52981f8a30 2024-09-21T07:32:47.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.778+0000 7f529ec00640 1 -- 172.21.15.39:0/302955748 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52980adea0 con 0x7f52981f8a30 2024-09-21T07:32:47.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.780+0000 7f3b88200640 1 -- 172.21.15.39:0/1810460746 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b80074c60 msgr2=0x7f3b800ccde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.780+0000 7f3b88200640 1 --2- 172.21.15.39:0/1810460746 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b80074c60 0x7f3b800ccde0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3b78005fd0 tx=0x7f3b7802a850 comp rx=0 tx=0).stop 2024-09-21T07:32:47.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.780+0000 7f3b88200640 1 -- 172.21.15.39:0/1810460746 shutdown_connections 2024-09-21T07:32:47.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.780+0000 7f3b88200640 1 --2- 172.21.15.39:0/1810460746 >> v2:172.21.15.148:3300/0 conn(0x7f3b8015fab0 0x7f3b8015c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.780+0000 7f3b88200640 1 --2- 172.21.15.39:0/1810460746 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b80074c60 0x7f3b800ccde0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.780+0000 7f3b88200640 1 -- 172.21.15.39:0/1810460746 >> 172.21.15.39:0/1810460746 conn(0x7f3b800b20a0 msgr2=0x7f3b800b24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:47.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.781+0000 7f3b33e00640 1 -- 172.21.15.39:0/3038123208 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b340b2090 con 0x7f3b34144a20 2024-09-21T07:32:47.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.781+0000 7f3b88200640 1 -- 172.21.15.39:0/1810460746 shutdown_connections 2024-09-21T07:32:47.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.778+0000 7fb687200640 1 -- 172.21.15.39:0/1867520646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb66c004d40 msgr2=0x7fb66c005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.778+0000 7fb687200640 1 --2- 172.21.15.39:0/1867520646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb66c004d40 0x7fb66c005150 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb688074e00 tx=0x7fb66c005830 comp rx=0 tx=0).stop 2024-09-21T07:32:47.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.780+0000 7f528d600640 1 -- 172.21.15.39:0/302955748 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f526c005290 con 0x7f52981f8a30 2024-09-21T07:32:47.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.781+0000 7f528fe00640 1 -- 172.21.15.39:0/302955748 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5288004cd0 con 0x7f52981f8a30 2024-09-21T07:32:47.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.781+0000 7f528fe00640 1 -- 172.21.15.39:0/302955748 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f528801aa70 con 0x7f52981f8a30 2024-09-21T07:32:47.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.782+0000 7fb686800640 1 -- 172.21.15.39:0/1867520646 reap_dead start 2024-09-21T07:32:47.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.782+0000 7fb687200640 1 -- 172.21.15.39:0/1867520646 shutdown_connections 2024-09-21T07:32:47.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.782+0000 7fb687200640 1 -- 172.21.15.39:0/1867520646 >> 172.21.15.39:0/1867520646 conn(0x7fb6880b3be0 msgr2=0x7fb6880b3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:47.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.782+0000 7f3b33e00640 1 --2- 172.21.15.39:0/3038123208 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b34144a20 0x7f3b341f6700 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3b24011780 tx=0x7f3b24011c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:32:47.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.782+0000 7fb687200640 1 -- 172.21.15.39:0/1867520646 shutdown_connections 2024-09-21T07:32:47.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.782+0000 7f3b88200640 1 -- 172.21.15.39:0/1810460746 wait complete. 2024-09-21T07:32:47.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.786+0000 7f528fe00640 1 -- 172.21.15.39:0/302955748 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f5288028020 con 0x7f52981f8a30 2024-09-21T07:32:47.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.786+0000 7f3b2be00640 1 -- 172.21.15.39:0/3038123208 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b24004440 con 0x7f3b34144a20 2024-09-21T07:32:47.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.786+0000 7f3b2be00640 1 -- 172.21.15.39:0/3038123208 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b24011e90 con 0x7f3b34144a20 2024-09-21T07:32:47.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.786+0000 7f3b39800640 1 -- 172.21.15.39:0/3038123208 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b34144810 con 0x7f3b34144a20 2024-09-21T07:32:47.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.786+0000 7f3b39800640 1 -- 172.21.15.39:0/3038123208 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b340af480 con 0x7f3b34144a20 2024-09-21T07:32:47.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.787+0000 7f3b2be00640 1 -- 172.21.15.39:0/3038123208 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2400fba0 con 0x7f3b34144a20 2024-09-21T07:32:47.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.788+0000 7f3b88200640 1 Processor -- start 2024-09-21T07:32:47.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.788+0000 7f3b88200640 1 -- start start 2024-09-21T07:32:47.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.790+0000 7f3b2be00640 1 -- 172.21.15.39:0/3038123208 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3b24005020 con 0x7f3b34144a20 2024-09-21T07:32:47.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.791+0000 7f1dad600640 1 -- 172.21.15.39:0/186420665 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f1d90005850 con 0x7f1dbc0d0db0 2024-09-21T07:32:47.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.790+0000 7f3b29600640 1 -- 172.21.15.39:0/3038123208 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b08005290 con 0x7f3b34144a20 2024-09-21T07:32:47.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.782+0000 7fb687200640 1 -- 172.21.15.39:0/1867520646 wait complete. 2024-09-21T07:32:47.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.792+0000 7f3b88200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b80074c60 0x7f3b800c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.792+0000 7fb687200640 1 Processor -- start 2024-09-21T07:32:47.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.792+0000 7f3b88200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b800cd320 0x7f3b801f2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.792+0000 7fb687200640 1 -- start start 2024-09-21T07:32:47.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.793+0000 7f3b2be00640 1 -- 172.21.15.39:0/3038123208 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3b24003930 con 0x7f3b34144a20 2024-09-21T07:32:47.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.793+0000 7f3b88200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b8015fab0 0x7f3b800b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.793+0000 7f3b85800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b80074c60 0x7f3b800c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:47.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.793+0000 7f3b85800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b80074c60 0x7f3b800c97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.39:40416/0 (socket says 172.21.15.39:40416) 2024-09-21T07:32:47.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.793+0000 7f3b84e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b800cd320 0x7f3b801f2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:47.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.793+0000 7f3b84e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b800cd320 0x7f3b801f2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.39:37730/0 (socket says 172.21.15.39:37730) 2024-09-21T07:32:47.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.794+0000 7f3b86200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b8015fab0 0x7f3b800b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:47.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.794+0000 7f3b86200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b8015fab0 0x7f3b800b05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55038/0 (socket says 172.21.15.39:55038) 2024-09-21T07:32:47.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.794+0000 7f3b85800640 1 -- 172.21.15.39:0/1651754844 learned_addr learned my addr 172.21.15.39:0/1651754844 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:32:47.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.794+0000 7f3b88200640 1 -- 172.21.15.39:0/1651754844 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3b800cac40 con 0x7f3b8015fab0 2024-09-21T07:32:47.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.794+0000 7f3b88200640 1 -- 172.21.15.39:0/1651754844 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3b800caac0 con 0x7f3b80074c60 2024-09-21T07:32:47.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.795+0000 7f3b2be00640 1 -- 172.21.15.39:0/3038123208 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f3b24028070 con 0x7f3b34144a20 2024-09-21T07:32:47.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.794+0000 7f3b88200640 1 -- 172.21.15.39:0/1651754844 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f3b800cadc0 con 0x7f3b800cd320 2024-09-21T07:32:47.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.795+0000 7f3b86200640 1 -- 172.21.15.39:0/1651754844 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b800cd320 msgr2=0x7f3b801f2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.795+0000 7f3b86200640 1 --2- 172.21.15.39:0/1651754844 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b800cd320 0x7f3b801f2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.795+0000 7f3b86200640 1 -- 172.21.15.39:0/1651754844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b80074c60 msgr2=0x7f3b800c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.795+0000 7f3b86200640 1 --2- 172.21.15.39:0/1651754844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b80074c60 0x7f3b800c97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.795+0000 7f3b86200640 1 -- 172.21.15.39:0/1651754844 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b800b43c0 con 0x7f3b8015fab0 2024-09-21T07:32:47.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.796+0000 7f3b85800640 1 --2- 172.21.15.39:0/1651754844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b80074c60 0x7f3b800c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:32:47.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.796+0000 7fb687200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb68809e7b0 0x7fb68809bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.797+0000 7f3b86200640 1 --2- 172.21.15.39:0/1651754844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b8015fab0 0x7f3b800b05c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c0098a0 tx=0x7f3b6c009d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:32:47.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.797+0000 7fb685e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb68809e7b0 0x7fb68809bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:47.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.797+0000 7fb685e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb68809e7b0 0x7fb68809bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.39:40420/0 (socket says 172.21.15.39:40420) 2024-09-21T07:32:47.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.797+0000 7f3b7d600640 1 -- 172.21.15.39:0/1651754844 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c005220 con 0x7f3b8015fab0 2024-09-21T07:32:47.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.797+0000 7fb687200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb6881ffbb0 0x7fb688205eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.798+0000 7f3b88200640 1 -- 172.21.15.39:0/1651754844 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b800b46a0 con 0x7f3b8015fab0 2024-09-21T07:32:47.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.798+0000 7f3b7d600640 1 -- 172.21.15.39:0/1651754844 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b6c0053c0 con 0x7f3b8015fab0 2024-09-21T07:32:47.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.798+0000 7f3b7d600640 1 -- 172.21.15.39:0/1651754844 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c005220 con 0x7f3b8015fab0 2024-09-21T07:32:47.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.798+0000 7fb67d400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb6881ffbb0 0x7fb688205eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:47.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.798+0000 7fb67d400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb6881ffbb0 0x7fb688205eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.39:37734/0 (socket says 172.21.15.39:37734) 2024-09-21T07:32:47.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.798+0000 7fb685e00640 1 -- 172.21.15.39:0/2821954915 learned_addr learned my addr 172.21.15.39:0/2821954915 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:32:47.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.798+0000 7fb687200640 1 --2- 172.21.15.39:0/2821954915 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6881fffc0 0x7fb6882063f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.799+0000 7fb687200640 1 -- 172.21.15.39:0/2821954915 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb68809b1d0 con 0x7fb6881fffc0 2024-09-21T07:32:47.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.799+0000 7fb687200640 1 -- 172.21.15.39:0/2821954915 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb68809b050 con 0x7fb68809e7b0 2024-09-21T07:32:47.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.799+0000 7fb687200640 1 -- 172.21.15.39:0/2821954915 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb6880d06d0 con 0x7fb6881ffbb0 2024-09-21T07:32:47.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.800+0000 7f3b88200640 1 -- 172.21.15.39:0/1651754844 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b800b4c60 con 0x7f3b8015fab0 2024-09-21T07:32:47.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.801+0000 7fb67d400640 1 -- 172.21.15.39:0/2821954915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb68809e7b0 msgr2=0x7fb68809bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.801+0000 7fb67d400640 1 --2- 172.21.15.39:0/2821954915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb68809e7b0 0x7fb68809bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.801+0000 7f3b7d600640 1 -- 172.21.15.39:0/1651754844 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3b6c017070 con 0x7f3b8015fab0 2024-09-21T07:32:47.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.801+0000 7fb686800640 1 --2- 172.21.15.39:0/2821954915 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6881fffc0 0x7fb6882063f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:47.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.801+0000 7fb67d400640 1 -- 172.21.15.39:0/2821954915 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6881fffc0 msgr2=0x7fb6882063f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.801+0000 7fb67d400640 1 --2- 172.21.15.39:0/2821954915 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6881fffc0 0x7fb6882063f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.801+0000 7fb67d400640 1 -- 172.21.15.39:0/2821954915 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb68809ede0 con 0x7fb6881ffbb0 2024-09-21T07:32:47.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.802+0000 7fb686800640 1 --2- 172.21.15.39:0/2821954915 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6881fffc0 0x7fb6882063f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:32:47.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.802+0000 7fb685e00640 1 --2- 172.21.15.39:0/2821954915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb68809e7b0 0x7fb68809bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:32:47.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.802+0000 7fb67d400640 1 --2- 172.21.15.39:0/2821954915 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb6881ffbb0 0x7fb688205eb0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb670001eb0 tx=0x7fb670003aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:32:47.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.803+0000 7f1dafe00640 1 -- 172.21.15.39:0/186420665 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v13)=0 v13) ==== 121+0+0 (secure 0 0 0) 0x7f1dbc14ace0 con 0x7f1dbc0d0db0 2024-09-21T07:32:47.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.803+0000 7fb67ea00640 1 -- 172.21.15.39:0/2821954915 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb670015070 con 0x7fb6881ffbb0 2024-09-21T07:32:47.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.804+0000 7fb687200640 1 -- 172.21.15.39:0/2821954915 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb68809f0c0 con 0x7fb6881ffbb0 2024-09-21T07:32:47.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.804+0000 7fb67ea00640 1 -- 172.21.15.39:0/2821954915 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb67000f9e0 con 0x7fb6881ffbb0 2024-09-21T07:32:47.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.804+0000 7fb687200640 1 -- 172.21.15.39:0/2821954915 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb68809f680 con 0x7fb6881ffbb0 2024-09-21T07:32:47.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.804+0000 7fb67ea00640 1 -- 172.21.15.39:0/2821954915 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb670015070 con 0x7fb6881ffbb0 2024-09-21T07:32:47.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.803+0000 7f383aa00640 1 -- 172.21.15.39:0/2186314056 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c001a70 con 0x7f383c06f720 2024-09-21T07:32:47.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.805+0000 7f3842400640 1 -- 172.21.15.39:0/2186314056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f383c06f720 msgr2=0x7f383c0cfae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.805+0000 7f3842400640 1 --2- 172.21.15.39:0/2186314056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f383c06f720 0x7f383c0cfae0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3824009830 tx=0x7f382402a3b0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.805+0000 7f3842400640 1 -- 172.21.15.39:0/2186314056 shutdown_connections 2024-09-21T07:32:47.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.805+0000 7f3842400640 1 --2- 172.21.15.39:0/2186314056 >> v2:172.21.15.39:3300/0 conn(0x7f383c15a7d0 0x7f383c0cc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.806+0000 7fb67ea00640 1 -- 172.21.15.39:0/2821954915 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb67001b460 con 0x7fb6881ffbb0 2024-09-21T07:32:47.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.806+0000 7fb67ea00640 1 -- 172.21.15.39:0/2821954915 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb670011de0 con 0x7fb6881ffbb0 2024-09-21T07:32:47.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.807+0000 7f1dad600640 1 -- 172.21.15.39:0/186420665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dbc0d0db0 msgr2=0x7f1dbc14ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.807+0000 7f1dad600640 1 --2- 172.21.15.39:0/186420665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dbc0d0db0 0x7f1dbc14ca90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1db4009ad0 tx=0x7f1db40024a0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.805+0000 7f3842400640 1 --2- 172.21.15.39:0/2186314056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f383c06f720 0x7f383c0cfae0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.807+0000 7f3842400640 1 -- 172.21.15.39:0/2186314056 >> 172.21.15.39:0/2186314056 conn(0x7f383c06d7d0 msgr2=0x7f383c0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:47.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.807+0000 7f1dad600640 1 -- 172.21.15.39:0/186420665 shutdown_connections 2024-09-21T07:32:47.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.807+0000 7f1dad600640 1 --2- 172.21.15.39:0/186420665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dbc0d0db0 0x7f1dbc14ca90 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.807+0000 7f1dad600640 1 --2- 172.21.15.39:0/186420665 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1dbc0b1020 0x7f1dbc14c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.807+0000 7f1dad600640 1 --2- 172.21.15.39:0/186420665 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1dbc074d40 0x7f1dbc14a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.807+0000 7f1dad600640 1 -- 172.21.15.39:0/186420665 >> 172.21.15.39:0/186420665 conn(0x7f1dbc074220 msgr2=0x7f1dbc06f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:47.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.807+0000 7f3842400640 1 -- 172.21.15.39:0/2186314056 shutdown_connections 2024-09-21T07:32:47.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.807+0000 7f3842400640 1 -- 172.21.15.39:0/2186314056 wait complete. 2024-09-21T07:32:47.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.807+0000 7f1dad600640 1 -- 172.21.15.39:0/186420665 shutdown_connections 2024-09-21T07:32:47.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.808+0000 7f1dad600640 1 -- 172.21.15.39:0/186420665 wait complete. 2024-09-21T07:32:47.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.807+0000 7f3b88200640 1 -- 172.21.15.39:0/1651754844 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b800cefe0 con 0x7f3b8015fab0 2024-09-21T07:32:47.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.810+0000 7fb687200640 1 -- 172.21.15.39:0/2821954915 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb68809b790 con 0x7fb6881ffbb0 2024-09-21T07:32:47.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.814+0000 7f3842400640 1 Processor -- start 2024-09-21T07:32:47.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.815+0000 7f3842400640 1 -- start start 2024-09-21T07:32:47.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.816+0000 7fb67ea00640 1 -- 172.21.15.39:0/2821954915 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fb68809b790 con 0x7fb6881ffbb0 2024-09-21T07:32:47.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.817+0000 7f3842400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f383c06f720 0x7f383c09b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.817+0000 7f3842400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f383c070f20 0x7f383c147bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.817+0000 7f3842400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f383c15a7d0 0x7f383c158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.817+0000 7f3842400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f383c0cd030 con 0x7f383c070f20 2024-09-21T07:32:47.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.817+0000 7f3842400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f383c0cceb0 con 0x7f383c15a7d0 2024-09-21T07:32:47.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.817+0000 7f3842400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f383c0cd1b0 con 0x7f383c06f720 2024-09-21T07:32:47.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.818+0000 7f383b400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f383c06f720 0x7f383c09b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:47.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.818+0000 7f3832a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f383c070f20 0x7f383c147bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:47.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.818+0000 7f3832a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f383c070f20 0x7f383c147bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55062/0 (socket says 172.21.15.39:55062) 2024-09-21T07:32:47.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.818+0000 7f383be00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f383c15a7d0 0x7f383c158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:47.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.818+0000 7f3832a00640 1 -- 172.21.15.39:0/2560228455 learned_addr learned my addr 172.21.15.39:0/2560228455 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:32:47.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.819+0000 7f383be00640 1 -- 172.21.15.39:0/2560228455 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f383c06f720 msgr2=0x7f383c09b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.819+0000 7f383be00640 1 --2- 172.21.15.39:0/2560228455 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f383c06f720 0x7f383c09b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.819+0000 7f383be00640 1 -- 172.21.15.39:0/2560228455 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f383c070f20 msgr2=0x7f383c147bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.819+0000 7f383be00640 1 --2- 172.21.15.39:0/2560228455 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f383c070f20 0x7f383c147bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.819+0000 7f383be00640 1 -- 172.21.15.39:0/2560228455 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f383c148110 con 0x7f383c15a7d0 2024-09-21T07:32:47.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.819+0000 7f3832a00640 1 --2- 172.21.15.39:0/2560228455 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f383c070f20 0x7f383c147bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:32:47.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.819+0000 7f383b400640 1 --2- 172.21.15.39:0/2560228455 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f383c06f720 0x7f383c09b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:32:47.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.824+0000 7f383be00640 1 --2- 172.21.15.39:0/2560228455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f383c15a7d0 0x7f383c158e40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f383400d6b0 tx=0x7f383400db70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:32:47.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.824+0000 7f3b7d600640 1 -- 172.21.15.39:0/1651754844 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3b6c004df0 con 0x7f3b8015fab0 2024-09-21T07:32:47.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.825+0000 7f3833e00640 1 -- 172.21.15.39:0/2560228455 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3834017070 con 0x7f383c15a7d0 2024-09-21T07:32:47.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.826+0000 7f3833e00640 1 -- 172.21.15.39:0/2560228455 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3834004310 con 0x7f383c15a7d0 2024-09-21T07:32:47.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.826+0000 7f3833e00640 1 -- 172.21.15.39:0/2560228455 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3834017070 con 0x7f383c15a7d0 2024-09-21T07:32:47.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.826+0000 7f3b7d600640 1 -- 172.21.15.39:0/1651754844 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f3b6c0077a0 con 0x7f3b8015fab0 2024-09-21T07:32:47.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.826+0000 7f3842400640 1 -- 172.21.15.39:0/2560228455 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f383c155d20 con 0x7f383c15a7d0 2024-09-21T07:32:47.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.827+0000 7f3842400640 1 -- 172.21.15.39:0/2560228455 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f383c09aad0 con 0x7f383c15a7d0 2024-09-21T07:32:47.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.827+0000 7f3833e00640 1 -- 172.21.15.39:0/2560228455 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f38340053b0 con 0x7f383c15a7d0 2024-09-21T07:32:47.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.829+0000 7f3833e00640 1 -- 172.21.15.39:0/2560228455 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3834004cb0 con 0x7f383c15a7d0 2024-09-21T07:32:47.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.829+0000 7f3842400640 1 -- 172.21.15.39:0/2560228455 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f383c0cfae0 con 0x7f383c15a7d0 2024-09-21T07:32:47.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.832+0000 7f3833e00640 1 -- 172.21.15.39:0/2560228455 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f3834020020 con 0x7f383c15a7d0 2024-09-21T07:32:47.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.861+0000 7fab1a600640 1 -- 172.21.15.39:0/1475568403 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7faaf0005580 con 0x7fab0c0dae40 2024-09-21T07:32:47.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.869+0000 7fab07400640 1 -- 172.21.15.39:0/1475568403 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v14)=0 v14) ==== 119+0+0 (secure 0 0 0) 0x7fab14097640 con 0x7fab0c0dae40 2024-09-21T07:32:47.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.870+0000 7fab1a600640 1 -- 172.21.15.39:0/1475568403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab0c0dae40 msgr2=0x7fab0c0da230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.871+0000 7fab1a600640 1 --2- 172.21.15.39:0/1475568403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab0c0dae40 0x7fab0c0da230 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fab14051230 tx=0x7fab14064940 comp rx=0 tx=0).stop 2024-09-21T07:32:47.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.871+0000 7fab1a600640 1 -- 172.21.15.39:0/1475568403 shutdown_connections 2024-09-21T07:32:47.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.871+0000 7fab1a600640 1 --2- 172.21.15.39:0/1475568403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab0c0dae40 0x7fab0c0da230 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.871+0000 7fab1a600640 1 --2- 172.21.15.39:0/1475568403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab0c0e0110 0x7fab0c0d9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.871+0000 7fab1a600640 1 --2- 172.21.15.39:0/1475568403 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faafc003b40 0x7fab0c0dfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.874+0000 7fab1a600640 1 -- 172.21.15.39:0/1475568403 >> 172.21.15.39:0/1475568403 conn(0x7fab0c01db80 msgr2=0x7fab0c00a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:47.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.875+0000 7fab1a600640 1 -- 172.21.15.39:0/1475568403 shutdown_connections 2024-09-21T07:32:47.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.875+0000 7fab1a600640 1 -- 172.21.15.39:0/1475568403 wait complete. 2024-09-21T07:32:47.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.877+0000 7fd352000640 1 -- 172.21.15.39:0/2912951863 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd34c0cc480 msgr2=0x7fd34c166110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.877+0000 7fd352000640 1 --2- 172.21.15.39:0/2912951863 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd34c0cc480 0x7fd34c166110 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd344009be0 tx=0x7fd34402a760 comp rx=0 tx=0).stop 2024-09-21T07:32:47.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.877+0000 7fd352000640 1 -- 172.21.15.39:0/2912951863 shutdown_connections 2024-09-21T07:32:47.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.878+0000 7fd352000640 1 --2- 172.21.15.39:0/2912951863 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd34c0cc480 0x7fd34c166110 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.878+0000 7fd352000640 1 -- 172.21.15.39:0/2912951863 >> 172.21.15.39:0/2912951863 conn(0x7fd34c06d5d0 msgr2=0x7fd34c06ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:47.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.878+0000 7fd352000640 1 -- 172.21.15.39:0/2912951863 shutdown_connections 2024-09-21T07:32:47.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.878+0000 7fd352000640 1 -- 172.21.15.39:0/2912951863 wait complete. 2024-09-21T07:32:47.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.884+0000 7fd352000640 1 Processor -- start 2024-09-21T07:32:47.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.884+0000 7fd352000640 1 -- start start 2024-09-21T07:32:47.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.883+0000 7fb43cc00640 1 -- 172.21.15.39:0/3585518072 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7fb410005580 con 0x7fb4380cf9a0 2024-09-21T07:32:47.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.889+0000 7fd313a00640 1 -- 172.21.15.39:0/2305791737 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=debug_mgr}] v 0) -- 0x7fd2e8005580 con 0x7fd304054390 2024-09-21T07:32:47.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.893+0000 7fd352000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd34c0cedd0 0x7fd34c152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.894+0000 7fd352000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd34c0cf6a0 0x7fd34c144ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.893+0000 7fb42f400640 1 -- 172.21.15.39:0/3585518072 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v15)=0 v15) ==== 120+0+0 (secure 0 0 0) 0x7fb4300046c0 con 0x7fb4380cf9a0 2024-09-21T07:32:47.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.894+0000 7fd352000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd34c14d540 0x7fd34c0b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:47.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.894+0000 7fd352000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c1669e0 con 0x7fd34c14d540 2024-09-21T07:32:47.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.894+0000 7fd352000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c166860 con 0x7fd34c0cf6a0 2024-09-21T07:32:47.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.894+0000 7fd352000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd34c166b60 con 0x7fd34c0cedd0 2024-09-21T07:32:47.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.894+0000 7fd34be00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd34c0cf6a0 0x7fd34c144ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:47.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.896+0000 7fd34be00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd34c0cf6a0 0x7fd34c144ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.39:40438/0 (socket says 172.21.15.39:40438) 2024-09-21T07:32:47.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.896+0000 7fd34be00640 1 -- 172.21.15.39:0/1484985022 learned_addr learned my addr 172.21.15.39:0/1484985022 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:32:47.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.897+0000 7fb43cc00640 1 -- 172.21.15.39:0/3585518072 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4380cf9a0 msgr2=0x7fb4380b2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.897+0000 7fb43cc00640 1 --2- 172.21.15.39:0/3585518072 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4380cf9a0 0x7fb4380b2f20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb4300334c0 tx=0x7fb430004ab0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.897+0000 7fb43cc00640 1 -- 172.21.15.39:0/3585518072 shutdown_connections 2024-09-21T07:32:47.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.897+0000 7fb43cc00640 1 --2- 172.21.15.39:0/3585518072 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4380cf9a0 0x7fb4380b2f20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.897+0000 7fb43cc00640 1 --2- 172.21.15.39:0/3585518072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb438205d40 0x7fb4380cf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.897+0000 7fb43cc00640 1 --2- 172.21.15.39:0/3585518072 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4380b5a60 0x7fb4381665c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.897+0000 7fb43cc00640 1 -- 172.21.15.39:0/3585518072 >> 172.21.15.39:0/3585518072 conn(0x7fb43806d5d0 msgr2=0x7fb43816ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:47.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.897+0000 7fd350c00640 1 --2- 172.21.15.39:0/1484985022 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd34c0cedd0 0x7fd34c152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:47.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.897+0000 7fd34be00640 1 -- 172.21.15.39:0/1484985022 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd34c0cedd0 msgr2=0x7fd34c152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.897+0000 7fd34be00640 1 --2- 172.21.15.39:0/1484985022 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd34c0cedd0 0x7fd34c152cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.897+0000 7fd34be00640 1 -- 172.21.15.39:0/1484985022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd34c14d540 msgr2=0x7fd34c0b1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.897+0000 7fd34be00640 1 --2- 172.21.15.39:0/1484985022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd34c14d540 0x7fd34c0b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.897+0000 7fd34be00640 1 -- 172.21.15.39:0/1484985022 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd34c145400 con 0x7fd34c0cf6a0 2024-09-21T07:32:47.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.897+0000 7fb43cc00640 1 -- 172.21.15.39:0/3585518072 shutdown_connections 2024-09-21T07:32:47.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.898+0000 7fb43cc00640 1 -- 172.21.15.39:0/3585518072 wait complete. 2024-09-21T07:32:47.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.897+0000 7fd34be00640 1 --2- 172.21.15.39:0/1484985022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd34c0cf6a0 0x7fd34c144ec0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd33800c400 tx=0x7fd33800c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:32:47.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.899+0000 7fd350c00640 1 --2- 172.21.15.39:0/1484985022 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd34c0cedd0 0x7fd34c152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:32:47.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.899+0000 7fd308c00640 1 -- 172.21.15.39:0/2305791737 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mgr}]=0 v16)=0 v16) ==== 108+0+0 (secure 0 0 0) 0x7fd300032d00 con 0x7fd304054390 2024-09-21T07:32:47.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.900+0000 7fd348c00640 1 -- 172.21.15.39:0/1484985022 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd338011790 con 0x7fd34c0cf6a0 2024-09-21T07:32:47.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.900+0000 7fd352000640 1 -- 172.21.15.39:0/1484985022 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd34c0ca810 con 0x7fd34c0cf6a0 2024-09-21T07:32:47.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.900+0000 7fd352000640 1 -- 172.21.15.39:0/1484985022 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd34c14c1a0 con 0x7fd34c0cf6a0 2024-09-21T07:32:47.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.900+0000 7fd348c00640 1 -- 172.21.15.39:0/1484985022 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd338017070 con 0x7fd34c0cf6a0 2024-09-21T07:32:47.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.901+0000 7fd313a00640 1 -- 172.21.15.39:0/2305791737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd304054390 msgr2=0x7fd3040dd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.902+0000 7fd313a00640 1 --2- 172.21.15.39:0/2305791737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd304054390 0x7fd3040dd230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd300009780 tx=0x7fd300033f90 comp rx=0 tx=0).stop 2024-09-21T07:32:47.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.903+0000 7fd348c00640 1 -- 172.21.15.39:0/1484985022 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd338011790 con 0x7fd34c0cf6a0 2024-09-21T07:32:47.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.903+0000 7fd313a00640 1 -- 172.21.15.39:0/2305791737 shutdown_connections 2024-09-21T07:32:47.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.903+0000 7fd313a00640 1 --2- 172.21.15.39:0/2305791737 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd3040e2f70 0x7fd3040dbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.903+0000 7fd313a00640 1 --2- 172.21.15.39:0/2305791737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd304056dd0 0x7fd3040dba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.903+0000 7fd313a00640 1 --2- 172.21.15.39:0/2305791737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd304054390 0x7fd3040dd230 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.903+0000 7fd313a00640 1 -- 172.21.15.39:0/2305791737 >> 172.21.15.39:0/2305791737 conn(0x7fd30403a620 msgr2=0x7fd304057a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:47.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.903+0000 7fd348c00640 1 -- 172.21.15.39:0/1484985022 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd338011b20 con 0x7fd34c0cf6a0 2024-09-21T07:32:47.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.903+0000 7fd348c00640 1 -- 172.21.15.39:0/1484985022 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd338012470 con 0x7fd34c0cf6a0 2024-09-21T07:32:47.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.905+0000 7fd313a00640 1 -- 172.21.15.39:0/2305791737 shutdown_connections 2024-09-21T07:32:47.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.905+0000 7fd313a00640 1 -- 172.21.15.39:0/2305791737 wait complete. 2024-09-21T07:32:47.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.911+0000 7fd352000640 1 -- 172.21.15.39:0/1484985022 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd328005290 con 0x7fd34c0cf6a0 2024-09-21T07:32:47.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.916+0000 7fd348c00640 1 -- 172.21.15.39:0/1484985022 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fd34c145400 con 0x7fd34c0cf6a0 2024-09-21T07:32:47.930 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi039') 2024-09-21T07:32:47.930 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi039') 2024-09-21T07:32:47.962 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi039') 2024-09-21T07:32:47.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.960+0000 7f3b29600640 1 -- 172.21.15.39:0/3038123208 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7f3b08005850 con 0x7f3b34144a20 2024-09-21T07:32:47.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.967+0000 7fbb3a000640 1 -- 172.21.15.39:0/3837181708 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7fbb1c005850 con 0x7fbb4406ee70 2024-09-21T07:32:47.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.971+0000 7f3b2be00640 1 -- 172.21.15.39:0/3038123208 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v17)=0 v17) ==== 119+0+0 (secure 0 0 0) 0x7f3b340b2090 con 0x7f3b34144a20 2024-09-21T07:32:47.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.973+0000 7f3b29600640 1 -- 172.21.15.39:0/3038123208 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b34144a20 msgr2=0x7f3b341f6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.973+0000 7f3b29600640 1 --2- 172.21.15.39:0/3038123208 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b34144a20 0x7f3b341f6700 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3b24011780 tx=0x7f3b24011c40 comp rx=0 tx=0).stop 2024-09-21T07:32:47.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.974+0000 7f3b29600640 1 -- 172.21.15.39:0/3038123208 shutdown_connections 2024-09-21T07:32:47.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.974+0000 7f3b88200640 1 -- 172.21.15.39:0/1651754844 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f3b800cb2e0 con 0x7f3b8015fab0 2024-09-21T07:32:47.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.974+0000 7f3b29600640 1 --2- 172.21.15.39:0/3038123208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b340b1740 0x7f3b340b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.974+0000 7f3b29600640 1 --2- 172.21.15.39:0/3038123208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b340aea30 0x7f3b341442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.974+0000 7f3b29600640 1 --2- 172.21.15.39:0/3038123208 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b34144a20 0x7f3b341f6700 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.974+0000 7f3b29600640 1 -- 172.21.15.39:0/3038123208 >> 172.21.15.39:0/3038123208 conn(0x7f3b3406d410 msgr2=0x7f3b3415fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:47.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.974+0000 7f3b29600640 1 -- 172.21.15.39:0/3038123208 shutdown_connections 2024-09-21T07:32:47.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.975+0000 7f3b29600640 1 -- 172.21.15.39:0/3038123208 wait complete. 2024-09-21T07:32:47.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.980+0000 7fbb40c00640 1 -- 172.21.15.39:0/3837181708 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v18)=0 v18) ==== 120+0+0 (secure 0 0 0) 0x7fbb4406f7c0 con 0x7fbb4406ee70 2024-09-21T07:32:47.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.982+0000 7fbb3a000640 1 -- 172.21.15.39:0/3837181708 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbb4406ee70 msgr2=0x7fbb440b18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.982+0000 7fbb3a000640 1 --2- 172.21.15.39:0/3837181708 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbb4406ee70 0x7fbb440b18f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbb3400c4d0 tx=0x7fbb3400c990 comp rx=0 tx=0).stop 2024-09-21T07:32:47.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.983+0000 7f3b7d600640 1 -- 172.21.15.39:0/1651754844 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v19)=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7f3b6c007980 con 0x7f3b8015fab0 2024-09-21T07:32:47.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.985+0000 7f3b88200640 1 -- 172.21.15.39:0/1651754844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b8015fab0 msgr2=0x7f3b800b05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:47.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.985+0000 7f3b88200640 1 --2- 172.21.15.39:0/1651754844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b8015fab0 0x7f3b800b05c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c0098a0 tx=0x7f3b6c009d60 comp rx=0 tx=0).stop 2024-09-21T07:32:47.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.986+0000 7f3b88200640 1 -- 172.21.15.39:0/1651754844 shutdown_connections 2024-09-21T07:32:47.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.986+0000 7f3b88200640 1 --2- 172.21.15.39:0/1651754844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b8015fab0 0x7f3b800b05c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.986+0000 7f3b88200640 1 --2- 172.21.15.39:0/1651754844 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b800cd320 0x7f3b801f2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.986+0000 7f3b88200640 1 --2- 172.21.15.39:0/1651754844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b80074c60 0x7f3b800c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.986+0000 7f3b88200640 1 -- 172.21.15.39:0/1651754844 >> 172.21.15.39:0/1651754844 conn(0x7f3b800b20a0 msgr2=0x7f3b800b1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:47.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.987+0000 7fbb3a000640 1 -- 172.21.15.39:0/3837181708 shutdown_connections 2024-09-21T07:32:47.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.987+0000 7f3b88200640 1 -- 172.21.15.39:0/1651754844 shutdown_connections 2024-09-21T07:32:47.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.987+0000 7fbb3a000640 1 --2- 172.21.15.39:0/3837181708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb440cf950 0x7fbb441f17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.987+0000 7fbb3a000640 1 --2- 172.21.15.39:0/3837181708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb440b3620 0x7fbb441536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.987+0000 7fbb3a000640 1 --2- 172.21.15.39:0/3837181708 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbb4406ee70 0x7fbb440b18f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:47.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.987+0000 7fbb3a000640 1 -- 172.21.15.39:0/3837181708 >> 172.21.15.39:0/3837181708 conn(0x7fbb44070b20 msgr2=0x7fbb4415c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:47.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.988+0000 7fbb3a000640 1 -- 172.21.15.39:0/3837181708 shutdown_connections 2024-09-21T07:32:47.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.988+0000 7fbb3a000640 1 -- 172.21.15.39:0/3837181708 wait complete. 2024-09-21T07:32:47.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.987+0000 7f3b88200640 1 -- 172.21.15.39:0/1651754844 wait complete. 2024-09-21T07:32:47.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:47.996+0000 7fb687200640 1 -- 172.21.15.39:0/2821954915 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7fb68806a5e0 con 0x7fb6881ffbb0 2024-09-21T07:32:48.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.006+0000 7fb67ea00640 1 -- 172.21.15.39:0/2821954915 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v20)=0 v20) ==== 111+0+0 (secure 0 0 0) 0x7fb68806a5e0 con 0x7fb6881ffbb0 2024-09-21T07:32:48.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.008+0000 7fb687200640 1 -- 172.21.15.39:0/2821954915 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb6881ffbb0 msgr2=0x7fb688205eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:48.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.008+0000 7fb687200640 1 --2- 172.21.15.39:0/2821954915 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb6881ffbb0 0x7fb688205eb0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb670001eb0 tx=0x7fb670003aa0 comp rx=0 tx=0).stop 2024-09-21T07:32:48.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.008+0000 7fb687200640 1 -- 172.21.15.39:0/2821954915 shutdown_connections 2024-09-21T07:32:48.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.008+0000 7fb687200640 1 --2- 172.21.15.39:0/2821954915 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6881fffc0 0x7fb6882063f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:48.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.008+0000 7fb687200640 1 --2- 172.21.15.39:0/2821954915 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb6881ffbb0 0x7fb688205eb0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:48.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.008+0000 7fb687200640 1 --2- 172.21.15.39:0/2821954915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb68809e7b0 0x7fb68809bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:48.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.008+0000 7fb687200640 1 -- 172.21.15.39:0/2821954915 >> 172.21.15.39:0/2821954915 conn(0x7fb6880b3be0 msgr2=0x7fb6880b0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:48.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.009+0000 7fb687200640 1 -- 172.21.15.39:0/2821954915 shutdown_connections 2024-09-21T07:32:48.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.009+0000 7fb687200640 1 -- 172.21.15.39:0/2821954915 wait complete. 2024-09-21T07:32:48.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.011+0000 7f3842400640 1 -- 172.21.15.39:0/2560228455 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f383c06f010 con 0x7f383c15a7d0 2024-09-21T07:32:48.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.016+0000 7f528d600640 1 -- 172.21.15.39:0/302955748 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7f526c005580 con 0x7f52981f8a30 2024-09-21T07:32:48.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.019+0000 7f3833e00640 1 -- 172.21.15.39:0/2560228455 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v21)=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7f383c148110 con 0x7f383c15a7d0 2024-09-21T07:32:48.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.021+0000 7f3842400640 1 -- 172.21.15.39:0/2560228455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f383c15a7d0 msgr2=0x7f383c158e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:48.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.021+0000 7f3842400640 1 --2- 172.21.15.39:0/2560228455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f383c15a7d0 0x7f383c158e40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f383400d6b0 tx=0x7f383400db70 comp rx=0 tx=0).stop 2024-09-21T07:32:48.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.021+0000 7f3842400640 1 -- 172.21.15.39:0/2560228455 shutdown_connections 2024-09-21T07:32:48.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.021+0000 7f3842400640 1 --2- 172.21.15.39:0/2560228455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f383c15a7d0 0x7f383c158e40 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:48.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.021+0000 7f3842400640 1 --2- 172.21.15.39:0/2560228455 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f383c070f20 0x7f383c147bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:48.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.021+0000 7f3842400640 1 --2- 172.21.15.39:0/2560228455 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f383c06f720 0x7f383c09b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:48.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.021+0000 7f3842400640 1 -- 172.21.15.39:0/2560228455 >> 172.21.15.39:0/2560228455 conn(0x7f383c06d7d0 msgr2=0x7f383c0acf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:48.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.021+0000 7f3842400640 1 -- 172.21.15.39:0/2560228455 shutdown_connections 2024-09-21T07:32:48.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.021+0000 7f3842400640 1 -- 172.21.15.39:0/2560228455 wait complete. 2024-09-21T07:32:48.026 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'client_mount_timeout', '30'], check_status=True, hostname='smithi039') 2024-09-21T07:32:48.027 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_client', '20'], check_status=True, hostname='smithi039') 2024-09-21T07:32:48.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.027+0000 7f528fe00640 1 -- 172.21.15.39:0/302955748 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v22)=0 v22) ==== 125+0+0 (secure 0 0 0) 0x7f528801acc0 con 0x7f52981f8a30 2024-09-21T07:32:48.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.028+0000 7f528d600640 1 -- 172.21.15.39:0/302955748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52981f8a30 msgr2=0x7f52980acf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:48.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.028+0000 7f528d600640 1 --2- 172.21.15.39:0/302955748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52981f8a30 0x7f52980acf90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f528800bee0 tx=0x7f52880039c0 comp rx=0 tx=0).stop 2024-09-21T07:32:48.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.028+0000 7f528d600640 1 -- 172.21.15.39:0/302955748 shutdown_connections 2024-09-21T07:32:48.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.028+0000 7f528d600640 1 --2- 172.21.15.39:0/302955748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5298147f10 0x7f529814a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:48.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.028+0000 7f528d600640 1 --2- 172.21.15.39:0/302955748 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52981f8e40 0x7f52981479d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:48.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.028+0000 7f528d600640 1 --2- 172.21.15.39:0/302955748 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52981f8a30 0x7f52980acf90 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:48.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.029+0000 7f528d600640 1 -- 172.21.15.39:0/302955748 >> 172.21.15.39:0/302955748 conn(0x7f529806d410 msgr2=0x7f5298160550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:48.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.029+0000 7f528d600640 1 -- 172.21.15.39:0/302955748 shutdown_connections 2024-09-21T07:32:48.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.029+0000 7f528d600640 1 -- 172.21.15.39:0/302955748 wait complete. 2024-09-21T07:32:48.041 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_mgr', '20'], check_status=True, hostname='smithi039') 2024-09-21T07:32:48.041 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_ms', '1'], check_status=True, hostname='smithi039') 2024-09-21T07:32:48.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.051+0000 7fd352000640 1 -- 172.21.15.39:0/1484985022 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7fd328005580 con 0x7fd34c0cf6a0 2024-09-21T07:32:48.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.057+0000 7fd348c00640 1 -- 172.21.15.39:0/1484985022 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v23)=0 v23) ==== 106+0+0 (secure 0 0 0) 0x7fd34c145400 con 0x7fd34c0cf6a0 2024-09-21T07:32:48.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.059+0000 7fd352000640 1 -- 172.21.15.39:0/1484985022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd34c0cf6a0 msgr2=0x7fd34c144ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:48.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.059+0000 7fd352000640 1 --2- 172.21.15.39:0/1484985022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd34c0cf6a0 0x7fd34c144ec0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd33800c400 tx=0x7fd33800c8c0 comp rx=0 tx=0).stop 2024-09-21T07:32:48.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.060+0000 7fd352000640 1 -- 172.21.15.39:0/1484985022 shutdown_connections 2024-09-21T07:32:48.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.060+0000 7fd352000640 1 --2- 172.21.15.39:0/1484985022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd34c14d540 0x7fd34c0b1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:48.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.060+0000 7fd352000640 1 --2- 172.21.15.39:0/1484985022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd34c0cf6a0 0x7fd34c144ec0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:48.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.060+0000 7fd352000640 1 --2- 172.21.15.39:0/1484985022 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd34c0cedd0 0x7fd34c152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:48.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.060+0000 7fd352000640 1 -- 172.21.15.39:0/1484985022 >> 172.21.15.39:0/1484985022 conn(0x7fd34c06d5d0 msgr2=0x7fd34c1519d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:48.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.060+0000 7fd352000640 1 -- 172.21.15.39:0/1484985022 shutdown_connections 2024-09-21T07:32:48.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.060+0000 7fd352000640 1 -- 172.21.15.39:0/1484985022 wait complete. 2024-09-21T07:32:48.090 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi039') 2024-09-21T07:32:48.090 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi039') 2024-09-21T07:32:48.090 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi039') 2024-09-21T07:32:48.090 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph config dump 2024-09-21T07:32:48.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.347+0000 7f1617200640 1 -- 172.21.15.39:0/2587221360 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16100706b0 msgr2=0x7f16100735b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:48.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.347+0000 7f1617200640 1 --2- 172.21.15.39:0/2587221360 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16100706b0 0x7f16100735b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f16100c71a0 tx=0x7f160802aac0 comp rx=0 tx=0).stop 2024-09-21T07:32:48.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.348+0000 7f1617200640 1 -- 172.21.15.39:0/2587221360 shutdown_connections 2024-09-21T07:32:48.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.348+0000 7f1617200640 1 --2- 172.21.15.39:0/2587221360 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16100706b0 0x7f16100735b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:48.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.348+0000 7f1617200640 1 --2- 172.21.15.39:0/2587221360 >> v2:172.21.15.148:3300/0 conn(0x7f16100ca800 0x7f16100cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:48.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.348+0000 7f1617200640 1 -- 172.21.15.39:0/2587221360 >> 172.21.15.39:0/2587221360 conn(0x7f16100b1cb0 msgr2=0x7f16100b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:48.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.348+0000 7f1617200640 1 -- 172.21.15.39:0/2587221360 shutdown_connections 2024-09-21T07:32:48.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.348+0000 7f1617200640 1 -- 172.21.15.39:0/2587221360 wait complete. 2024-09-21T07:32:48.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.350+0000 7f1617200640 1 Processor -- start 2024-09-21T07:32:48.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.350+0000 7f1617200640 1 -- start start 2024-09-21T07:32:48.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.351+0000 7f1617200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16100ca800 0x7f161014a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:48.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.351+0000 7f1617200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16100cad30 0x7f161014d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:48.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.351+0000 7f1617200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f161014db90 0x7f161015df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:48.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.351+0000 7f1617200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1610073ef0 con 0x7f16100cad30 2024-09-21T07:32:48.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.351+0000 7f1617200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1610073d70 con 0x7f16100ca800 2024-09-21T07:32:48.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.351+0000 7f1617200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f1610074070 con 0x7f161014db90 2024-09-21T07:32:48.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.352+0000 7f160f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16100cad30 0x7f161014d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:48.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.352+0000 7f160f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16100cad30 0x7f161014d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55108/0 (socket says 172.21.15.39:55108) 2024-09-21T07:32:48.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.352+0000 7f160f400640 1 -- 172.21.15.39:0/1576677215 learned_addr learned my addr 172.21.15.39:0/1576677215 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:32:48.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.352+0000 7f1615200640 1 --2- 172.21.15.39:0/1576677215 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f161014db90 0x7f161015df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:48.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.352+0000 7f160fe00640 1 --2- 172.21.15.39:0/1576677215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16100ca800 0x7f161014a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:48.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.352+0000 7f1615200640 1 -- 172.21.15.39:0/1576677215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16100ca800 msgr2=0x7f161014a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:48.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.352+0000 7f1615200640 1 --2- 172.21.15.39:0/1576677215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16100ca800 0x7f161014a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:48.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.352+0000 7f1615200640 1 -- 172.21.15.39:0/1576677215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16100cad30 msgr2=0x7f161014d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:48.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.352+0000 7f1615200640 1 --2- 172.21.15.39:0/1576677215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16100cad30 0x7f161014d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:48.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.352+0000 7f1615200640 1 -- 172.21.15.39:0/1576677215 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f161014ada0 con 0x7f161014db90 2024-09-21T07:32:48.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.352+0000 7f160f400640 1 --2- 172.21.15.39:0/1576677215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16100cad30 0x7f161014d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:32:48.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.353+0000 7f1615200640 1 --2- 172.21.15.39:0/1576677215 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f161014db90 0x7f161015df70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f16100ce1b0 tx=0x7f1608003ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:32:48.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.353+0000 7f1607e00640 1 -- 172.21.15.39:0/1576677215 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1608004060 con 0x7f161014db90 2024-09-21T07:32:48.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.353+0000 7f1607e00640 1 -- 172.21.15.39:0/1576677215 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1608004200 con 0x7f161014db90 2024-09-21T07:32:48.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.353+0000 7f1617200640 1 -- 172.21.15.39:0/1576677215 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16101f8b50 con 0x7f161014db90 2024-09-21T07:32:48.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.353+0000 7f1607e00640 1 -- 172.21.15.39:0/1576677215 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1608004610 con 0x7f161014db90 2024-09-21T07:32:48.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.353+0000 7f1617200640 1 -- 172.21.15.39:0/1576677215 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f16101f9110 con 0x7f161014db90 2024-09-21T07:32:48.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.353+0000 7f1607e00640 1 -- 172.21.15.39:0/1576677215 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1608049430 con 0x7f161014db90 2024-09-21T07:32:48.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.354+0000 7f1607e00640 1 -- 172.21.15.39:0/1576677215 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1608040760 con 0x7f161014db90 2024-09-21T07:32:48.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.354+0000 7f1617200640 1 -- 172.21.15.39:0/1576677215 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1610074600 con 0x7f161014db90 2024-09-21T07:32:48.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.358+0000 7f1607e00640 1 -- 172.21.15.39:0/1576677215 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f1608032030 con 0x7f161014db90 2024-09-21T07:32:48.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.422+0000 7f1617200640 1 -- 172.21.15.39:0/1576677215 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f161014e0e0 con 0x7f161014db90 2024-09-21T07:32:48.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.423+0000 7f1607e00640 1 -- 172.21.15.39:0/1576677215 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v23) ==== 59+0+1472 (secure 0 0 0) 0x7f1608009850 con 0x7f161014db90 2024-09-21T07:32:48.425 INFO:teuthology.orchestra.run.smithi039.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T07:32:48.425 INFO:teuthology.orchestra.run.smithi039.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T07:32:48.426 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced client_mount_timeout 30 2024-09-21T07:32:48.426 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_client 20/20 2024-09-21T07:32:48.426 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_mgr 20/20 2024-09-21T07:32:48.426 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_ms 1/1 2024-09-21T07:32:48.426 INFO:teuthology.orchestra.run.smithi039.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T07:32:48.426 INFO:teuthology.orchestra.run.smithi039.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T07:32:48.426 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_mds 20/20 2024-09-21T07:32:48.426 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T07:32:48.426 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_ms 2/2 2024-09-21T07:32:48.426 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_debug_frag true 2024-09-21T07:32:48.426 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_debug_scatterstat true 2024-09-21T07:32:48.427 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T07:32:48.427 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_verify_scatter true 2024-09-21T07:32:48.427 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T07:32:48.427 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T07:32:48.427 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T07:32:48.427 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced client_mount_timeout 600 2024-09-21T07:32:48.427 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced debug_client 20/20 2024-09-21T07:32:48.427 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced debug_ms 1/1 2024-09-21T07:32:48.427 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T07:32:48.427 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T07:32:48.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.424+0000 7f1617200640 1 -- 172.21.15.39:0/1576677215 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f161014db90 msgr2=0x7f161015df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:48.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.424+0000 7f1617200640 1 --2- 172.21.15.39:0/1576677215 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f161014db90 0x7f161015df70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f16100ce1b0 tx=0x7f1608003ce0 comp rx=0 tx=0).stop 2024-09-21T07:32:48.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.424+0000 7f1617200640 1 -- 172.21.15.39:0/1576677215 shutdown_connections 2024-09-21T07:32:48.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.424+0000 7f1617200640 1 --2- 172.21.15.39:0/1576677215 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f161014db90 0x7f161015df70 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:48.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.424+0000 7f1617200640 1 --2- 172.21.15.39:0/1576677215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16100cad30 0x7f161014d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:48.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.424+0000 7f1617200640 1 --2- 172.21.15.39:0/1576677215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16100ca800 0x7f161014a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:48.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.424+0000 7f1617200640 1 -- 172.21.15.39:0/1576677215 >> 172.21.15.39:0/1576677215 conn(0x7f16100b1cb0 msgr2=0x7f161015b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:48.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.425+0000 7f1617200640 1 -- 172.21.15.39:0/1576677215 shutdown_connections 2024-09-21T07:32:48.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.425+0000 7f1617200640 1 -- 172.21.15.39:0/1576677215 wait complete. 2024-09-21T07:32:48.450 INFO:tasks.ceph:Setting crush tunables to default 2024-09-21T07:32:48.450 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph osd crush tunables default 2024-09-21T07:32:48.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.670+0000 7fcead800640 1 -- 172.21.15.39:0/2719265847 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcea8070440 msgr2=0x7fcea80b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:48.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.670+0000 7fcead800640 1 --2- 172.21.15.39:0/2719265847 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcea8070440 0x7fcea80b3090 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcea0005fd0 tx=0x7fcea002a8b0 comp rx=0 tx=0).stop 2024-09-21T07:32:48.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.670+0000 7fcead800640 1 -- 172.21.15.39:0/2719265847 shutdown_connections 2024-09-21T07:32:48.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.670+0000 7fcead800640 1 --2- 172.21.15.39:0/2719265847 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcea8070440 0x7fcea80b3090 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:48.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.670+0000 7fcead800640 1 --2- 172.21.15.39:0/2719265847 >> v2:172.21.15.148:3300/0 conn(0x7fcea8073170 0x7fcea815c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:48.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.670+0000 7fcead800640 1 -- 172.21.15.39:0/2719265847 >> 172.21.15.39:0/2719265847 conn(0x7fcea80b1cb0 msgr2=0x7fcea80b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:48.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.670+0000 7fcead800640 1 -- 172.21.15.39:0/2719265847 shutdown_connections 2024-09-21T07:32:48.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.670+0000 7fcead800640 1 -- 172.21.15.39:0/2719265847 wait complete. 2024-09-21T07:32:48.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.672+0000 7fcead800640 1 Processor -- start 2024-09-21T07:32:48.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.672+0000 7fcead800640 1 -- start start 2024-09-21T07:32:48.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.674+0000 7fcead800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcea8070030 0x7fcea815ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:48.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.674+0000 7fcead800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcea8070440 0x7fcea815f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:48.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.674+0000 7fcead800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcea8073170 0x7fcea815f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:48.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.674+0000 7fcead800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcea80b3b00 con 0x7fcea8073170 2024-09-21T07:32:48.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.674+0000 7fcead800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcea80b3980 con 0x7fcea8070030 2024-09-21T07:32:48.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.674+0000 7fcead800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fcea80b3c80 con 0x7fcea8070440 2024-09-21T07:32:48.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.674+0000 7fcea7400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcea8073170 0x7fcea815f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:48.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.674+0000 7fcea6a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcea8070030 0x7fcea815ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:48.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.674+0000 7fcea6a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcea8070030 0x7fcea815ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.39:40468/0 (socket says 172.21.15.39:40468) 2024-09-21T07:32:48.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.674+0000 7fcea6000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcea8070440 0x7fcea815f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:48.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.674+0000 7fcea6a00640 1 -- 172.21.15.39:0/4177229976 learned_addr learned my addr 172.21.15.39:0/4177229976 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:32:48.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.675+0000 7fcea6000640 1 -- 172.21.15.39:0/4177229976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcea8070030 msgr2=0x7fcea815ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:48.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.675+0000 7fcea6000640 1 --2- 172.21.15.39:0/4177229976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcea8070030 0x7fcea815ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:48.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.675+0000 7fcea6000640 1 -- 172.21.15.39:0/4177229976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcea8073170 msgr2=0x7fcea815f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:48.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.675+0000 7fcea6000640 1 --2- 172.21.15.39:0/4177229976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcea8073170 0x7fcea815f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:48.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.675+0000 7fcea6000640 1 -- 172.21.15.39:0/4177229976 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcea815dc40 con 0x7fcea8070440 2024-09-21T07:32:48.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.675+0000 7fcea6000640 1 --2- 172.21.15.39:0/4177229976 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcea8070440 0x7fcea815f3b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fce9400c3c0 tx=0x7fce9400c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:32:48.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.675+0000 7fce9ea00640 1 -- 172.21.15.39:0/4177229976 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce94011790 con 0x7fcea8070440 2024-09-21T07:32:48.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.675+0000 7fce9ea00640 1 -- 172.21.15.39:0/4177229976 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce94017070 con 0x7fcea8070440 2024-09-21T07:32:48.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.675+0000 7fce9ea00640 1 -- 172.21.15.39:0/4177229976 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce94011ce0 con 0x7fcea8070440 2024-09-21T07:32:48.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.675+0000 7fcead800640 1 -- 172.21.15.39:0/4177229976 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcea80c82b0 con 0x7fcea8070440 2024-09-21T07:32:48.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.675+0000 7fcead800640 1 -- 172.21.15.39:0/4177229976 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcea80cb360 con 0x7fcea8070440 2024-09-21T07:32:48.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.676+0000 7fce9ea00640 1 -- 172.21.15.39:0/4177229976 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fce9401c460 con 0x7fcea8070440 2024-09-21T07:32:48.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.676+0000 7fce9ea00640 1 -- 172.21.15.39:0/4177229976 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fce94012780 con 0x7fcea8070440 2024-09-21T07:32:48.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.676+0000 7fcead800640 1 -- 172.21.15.39:0/4177229976 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcea80b48f0 con 0x7fcea8070440 2024-09-21T07:32:48.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.679+0000 7fce9ea00640 1 -- 172.21.15.39:0/4177229976 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fce94011e80 con 0x7fcea8070440 2024-09-21T07:32:48.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.748+0000 7fcead800640 1 -- 172.21.15.39:0/4177229976 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fcea80c3860 con 0x7fcea8070440 2024-09-21T07:32:48.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.753+0000 7fce9ea00640 1 -- 172.21.15.39:0/4177229976 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fce94012980 con 0x7fcea8070440 2024-09-21T07:32:48.755 INFO:teuthology.orchestra.run.smithi039.stderr:adjusted tunables profile to default 2024-09-21T07:32:48.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.754+0000 7fcead800640 1 -- 172.21.15.39:0/4177229976 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcea8070440 msgr2=0x7fcea815f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:48.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.754+0000 7fcead800640 1 --2- 172.21.15.39:0/4177229976 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcea8070440 0x7fcea815f3b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fce9400c3c0 tx=0x7fce9400c880 comp rx=0 tx=0).stop 2024-09-21T07:32:48.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.754+0000 7fcead800640 1 -- 172.21.15.39:0/4177229976 shutdown_connections 2024-09-21T07:32:48.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.754+0000 7fcead800640 1 --2- 172.21.15.39:0/4177229976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcea8073170 0x7fcea815f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:48.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.754+0000 7fcead800640 1 --2- 172.21.15.39:0/4177229976 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcea8070440 0x7fcea815f3b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:48.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.754+0000 7fcead800640 1 --2- 172.21.15.39:0/4177229976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcea8070030 0x7fcea815ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:48.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.754+0000 7fcead800640 1 -- 172.21.15.39:0/4177229976 >> 172.21.15.39:0/4177229976 conn(0x7fcea80b1cb0 msgr2=0x7fcea80c9f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:48.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.755+0000 7fcead800640 1 -- 172.21.15.39:0/4177229976 shutdown_connections 2024-09-21T07:32:48.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:48.755+0000 7fcead800640 1 -- 172.21.15.39:0/4177229976 wait complete. 2024-09-21T07:32:48.777 INFO:tasks.ceph:check_enable_crimson: False 2024-09-21T07:32:48.777 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-21T07:32:48.778 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:32:48.778 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-21T07:32:48.833 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:32:48.833 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-21T07:32:48.928 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:32:48.928 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-21T07:32:49.024 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:32:49.025 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-21T07:32:49.121 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:32:49.121 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-21T07:32:49.178 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:32:49.178 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-21T07:32:49.273 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:32:49.274 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-21T07:32:49.371 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:32:49.371 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-21T07:32:49.468 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph osd new 0218dc78-8164-452b-8728-359d39705d5d 0 2024-09-21T07:32:49.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.722+0000 7f0090a00640 1 -- 172.21.15.148:0/4284513446 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f008c0cefd0 msgr2=0x7f008c0cf3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:49.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.722+0000 7f0090a00640 1 --2- 172.21.15.148:0/4284513446 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f008c0cefd0 0x7f008c0cf3e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0074009830 tx=0x7f007402a510 comp rx=0 tx=0).stop 2024-09-21T07:32:49.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.722+0000 7f0090a00640 1 -- 172.21.15.148:0/4284513446 shutdown_connections 2024-09-21T07:32:49.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.722+0000 7f0090a00640 1 --2- 172.21.15.148:0/4284513446 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f008c0cefd0 0x7f008c0cf3e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:49.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.722+0000 7f0090a00640 1 --2- 172.21.15.148:0/4284513446 >> v2:172.21.15.39:3300/0 conn(0x7f008c0b42f0 0x7f008c0b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:49.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.722+0000 7f0090a00640 1 -- 172.21.15.148:0/4284513446 >> 172.21.15.148:0/4284513446 conn(0x7f008c09d680 msgr2=0x7f008c0d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:49.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.722+0000 7f0090a00640 1 -- 172.21.15.148:0/4284513446 shutdown_connections 2024-09-21T07:32:49.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.722+0000 7f0090a00640 1 -- 172.21.15.148:0/4284513446 wait complete. 2024-09-21T07:32:49.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.724+0000 7f0090a00640 1 Processor -- start 2024-09-21T07:32:49.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.724+0000 7f0090a00640 1 -- start start 2024-09-21T07:32:49.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.726+0000 7f0090a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f008c0b42f0 0x7f008c1654c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:49.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.726+0000 7f0090a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f008c0b96b0 0x7f008c09e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:49.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.726+0000 7f0090a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f008c0cefd0 0x7f008c1f8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:49.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.726+0000 7f0090a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f008c0b4ec0 con 0x7f008c0b42f0 2024-09-21T07:32:49.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.726+0000 7f0090a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f008c0b4d20 con 0x7f008c0b96b0 2024-09-21T07:32:49.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.726+0000 7f0090a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f008c0b5040 con 0x7f008c0cefd0 2024-09-21T07:32:49.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.726+0000 7f0083e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f008c0b96b0 0x7f008c09e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:49.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.726+0000 7f008be00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f008c0cefd0 0x7f008c1f8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:49.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.726+0000 7f008b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f008c0b42f0 0x7f008c1654c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:49.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.726+0000 7f008b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f008c0b42f0 0x7f008c1654c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.148:42558/0 (socket says 172.21.15.148:42558) 2024-09-21T07:32:49.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.726+0000 7f0083e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f008c0b96b0 0x7f008c09e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39790/0 (socket says 172.21.15.148:39790) 2024-09-21T07:32:49.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.726+0000 7f008b400640 1 -- 172.21.15.148:0/902530817 learned_addr learned my addr 172.21.15.148:0/902530817 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-09-21T07:32:49.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.727+0000 7f008be00640 1 -- 172.21.15.148:0/902530817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f008c0b96b0 msgr2=0x7f008c09e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:49.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.727+0000 7f008be00640 1 --2- 172.21.15.148:0/902530817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f008c0b96b0 0x7f008c09e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:49.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.727+0000 7f008be00640 1 -- 172.21.15.148:0/902530817 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f008c0b42f0 msgr2=0x7f008c1654c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:49.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.727+0000 7f008be00640 1 --2- 172.21.15.148:0/902530817 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f008c0b42f0 0x7f008c1654c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:49.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.727+0000 7f008be00640 1 -- 172.21.15.148:0/902530817 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f008c165d10 con 0x7f008c0cefd0 2024-09-21T07:32:49.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.727+0000 7f008b400640 1 --2- 172.21.15.148:0/902530817 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f008c0b42f0 0x7f008c1654c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:32:49.730 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.727+0000 7f008be00640 1 --2- 172.21.15.148:0/902530817 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f008c0cefd0 0x7f008c1f8650 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f008400e9b0 tx=0x7f008400ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:32:49.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.727+0000 7f0083400640 1 -- 172.21.15.148:0/902530817 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f008400cd80 con 0x7f008c0cefd0 2024-09-21T07:32:49.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.727+0000 7f0083400640 1 -- 172.21.15.148:0/902530817 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0084004b20 con 0x7f008c0cefd0 2024-09-21T07:32:49.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.727+0000 7f0090a00640 1 -- 172.21.15.148:0/902530817 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f008c165ff0 con 0x7f008c0cefd0 2024-09-21T07:32:49.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.727+0000 7f0083400640 1 -- 172.21.15.148:0/902530817 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0084005020 con 0x7f008c0cefd0 2024-09-21T07:32:49.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.727+0000 7f0090a00640 1 -- 172.21.15.148:0/902530817 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f008c1f8f00 con 0x7f008c0cefd0 2024-09-21T07:32:49.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.728+0000 7f0083400640 1 -- 172.21.15.148:0/902530817 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0084005550 con 0x7f008c0cefd0 2024-09-21T07:32:49.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.728+0000 7f0083400640 1 -- 172.21.15.148:0/902530817 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f0084010770 con 0x7f008c0cefd0 2024-09-21T07:32:49.731 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.729+0000 7f0090a00640 1 -- 172.21.15.148:0/902530817 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f008c0d1310 con 0x7f008c0cefd0 2024-09-21T07:32:49.732 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.731+0000 7f0083400640 1 -- 172.21.15.148:0/902530817 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f008c0d1310 con 0x7f008c0cefd0 2024-09-21T07:32:49.806 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.805+0000 7f0090a00640 1 -- 172.21.15.148:0/902530817 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0218dc78-8164-452b-8728-359d39705d5d", "id": 0} v 0) -- 0x7f008c0b4700 con 0x7f008c0cefd0 2024-09-21T07:32:49.811 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.810+0000 7f0083400640 1 -- 172.21.15.148:0/902530817 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0218dc78-8164-452b-8728-359d39705d5d", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f0084026020 con 0x7f008c0cefd0 2024-09-21T07:32:49.811 INFO:teuthology.orchestra.run.smithi148.stdout:0 2024-09-21T07:32:49.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.811+0000 7f0090a00640 1 -- 172.21.15.148:0/902530817 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f008c0cefd0 msgr2=0x7f008c1f8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:49.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.811+0000 7f0090a00640 1 --2- 172.21.15.148:0/902530817 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f008c0cefd0 0x7f008c1f8650 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f008400e9b0 tx=0x7f008400ee70 comp rx=0 tx=0).stop 2024-09-21T07:32:49.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.811+0000 7f0090a00640 1 -- 172.21.15.148:0/902530817 shutdown_connections 2024-09-21T07:32:49.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.811+0000 7f0090a00640 1 --2- 172.21.15.148:0/902530817 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f008c0cefd0 0x7f008c1f8650 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:49.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.811+0000 7f0090a00640 1 --2- 172.21.15.148:0/902530817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f008c0b96b0 0x7f008c09e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:49.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.811+0000 7f0090a00640 1 --2- 172.21.15.148:0/902530817 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f008c0b42f0 0x7f008c1654c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:49.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.811+0000 7f0090a00640 1 -- 172.21.15.148:0/902530817 >> 172.21.15.148:0/902530817 conn(0x7f008c09d680 msgr2=0x7f008c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:49.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.811+0000 7f0090a00640 1 -- 172.21.15.148:0/902530817 shutdown_connections 2024-09-21T07:32:49.813 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:49.811+0000 7f0090a00640 1 -- 172.21.15.148:0/902530817 wait complete. 2024-09-21T07:32:49.833 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph osd new 3c181f18-28e5-40a2-91e1-494f088472a4 1 2024-09-21T07:32:50.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.049+0000 7f89dea00640 1 -- 172.21.15.148:0/1868490111 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d80b4d90 msgr2=0x7f89d80d1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:50.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.049+0000 7f89dea00640 1 --2- 172.21.15.148:0/1868490111 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d80b4d90 0x7f89d80d1100 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f89d0005fd0 tx=0x7f89d002a770 comp rx=0 tx=0).stop 2024-09-21T07:32:50.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.049+0000 7f89dea00640 1 -- 172.21.15.148:0/1868490111 shutdown_connections 2024-09-21T07:32:50.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.049+0000 7f89dea00640 1 --2- 172.21.15.148:0/1868490111 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d80b4d90 0x7f89d80d1100 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:50.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.049+0000 7f89dea00640 1 -- 172.21.15.148:0/1868490111 >> 172.21.15.148:0/1868490111 conn(0x7f89d809d6b0 msgr2=0x7f89d80d4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:50.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.049+0000 7f89dea00640 1 -- 172.21.15.148:0/1868490111 shutdown_connections 2024-09-21T07:32:50.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.049+0000 7f89dea00640 1 -- 172.21.15.148:0/1868490111 wait complete. 2024-09-21T07:32:50.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.051+0000 7f89dea00640 1 Processor -- start 2024-09-21T07:32:50.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.052+0000 7f89dea00640 1 -- start start 2024-09-21T07:32:50.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.053+0000 7f89dea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d80b4d90 0x7f89d815e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:50.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.053+0000 7f89dea00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f89d80caf80 0x7f89d815ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:50.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.053+0000 7f89dea00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89d80cffa0 0x7f89d815bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:50.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.053+0000 7f89dea00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f89d80b7c60 con 0x7f89d80b4d90 2024-09-21T07:32:50.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.053+0000 7f89dea00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f89d80b7ae0 con 0x7f89d80cffa0 2024-09-21T07:32:50.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.053+0000 7f89dea00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f89d80b7de0 con 0x7f89d80caf80 2024-09-21T07:32:50.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.053+0000 7f89de000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89d80cffa0 0x7f89d815bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:50.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.053+0000 7f89dcc00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f89d80caf80 0x7f89d815ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:50.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.053+0000 7f89dd600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d80b4d90 0x7f89d815e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:50.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.053+0000 7f89de000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89d80cffa0 0x7f89d815bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39796/0 (socket says 172.21.15.148:39796) 2024-09-21T07:32:50.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.053+0000 7f89de000640 1 -- 172.21.15.148:0/13547543 learned_addr learned my addr 172.21.15.148:0/13547543 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-09-21T07:32:50.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.054+0000 7f89de000640 1 -- 172.21.15.148:0/13547543 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f89d80caf80 msgr2=0x7f89d815ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:50.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.054+0000 7f89de000640 1 --2- 172.21.15.148:0/13547543 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f89d80caf80 0x7f89d815ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:50.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.054+0000 7f89de000640 1 -- 172.21.15.148:0/13547543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d80b4d90 msgr2=0x7f89d815e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:50.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.054+0000 7f89de000640 1 --2- 172.21.15.148:0/13547543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d80b4d90 0x7f89d815e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:50.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.054+0000 7f89de000640 1 -- 172.21.15.148:0/13547543 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d815ea70 con 0x7f89d80cffa0 2024-09-21T07:32:50.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.054+0000 7f89dcc00640 1 --2- 172.21.15.148:0/13547543 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f89d80caf80 0x7f89d815ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:32:50.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.054+0000 7f89de000640 1 --2- 172.21.15.148:0/13547543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89d80cffa0 0x7f89d815bfb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f89d0034040 tx=0x7f89d0032d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:32:50.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.054+0000 7f89d5600640 1 -- 172.21.15.148:0/13547543 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d0041020 con 0x7f89d80cffa0 2024-09-21T07:32:50.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.054+0000 7f89d5600640 1 -- 172.21.15.148:0/13547543 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89d0004350 con 0x7f89d80cffa0 2024-09-21T07:32:50.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.054+0000 7f89dea00640 1 -- 172.21.15.148:0/13547543 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89d81f55c0 con 0x7f89d80cffa0 2024-09-21T07:32:50.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.055+0000 7f89d5600640 1 -- 172.21.15.148:0/13547543 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d0036040 con 0x7f89d80cffa0 2024-09-21T07:32:50.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.055+0000 7f89dea00640 1 -- 172.21.15.148:0/13547543 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89d81f5b80 con 0x7f89d80cffa0 2024-09-21T07:32:50.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.055+0000 7f89d5600640 1 -- 172.21.15.148:0/13547543 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f89d0044020 con 0x7f89d80cffa0 2024-09-21T07:32:50.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.055+0000 7f89d5600640 1 -- 172.21.15.148:0/13547543 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f89d003cb00 con 0x7f89d80cffa0 2024-09-21T07:32:50.058 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.056+0000 7f89dea00640 1 -- 172.21.15.148:0/13547543 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89d808e3b0 con 0x7f89d80cffa0 2024-09-21T07:32:50.059 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.058+0000 7f89d5600640 1 -- 172.21.15.148:0/13547543 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f89d808e3b0 con 0x7f89d80cffa0 2024-09-21T07:32:50.130 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.129+0000 7f89dea00640 1 -- 172.21.15.148:0/13547543 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3c181f18-28e5-40a2-91e1-494f088472a4", "id": 1} v 0) -- 0x7f89d80b71c0 con 0x7f89d80cffa0 2024-09-21T07:32:50.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.134+0000 7f89d5600640 1 -- 172.21.15.148:0/13547543 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3c181f18-28e5-40a2-91e1-494f088472a4", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f89d004b070 con 0x7f89d80cffa0 2024-09-21T07:32:50.135 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-09-21T07:32:50.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.135+0000 7f89dea00640 1 -- 172.21.15.148:0/13547543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89d80cffa0 msgr2=0x7f89d815bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:50.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.135+0000 7f89dea00640 1 --2- 172.21.15.148:0/13547543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89d80cffa0 0x7f89d815bfb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f89d0034040 tx=0x7f89d0032d10 comp rx=0 tx=0).stop 2024-09-21T07:32:50.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.135+0000 7f89dea00640 1 -- 172.21.15.148:0/13547543 shutdown_connections 2024-09-21T07:32:50.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.135+0000 7f89dea00640 1 --2- 172.21.15.148:0/13547543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89d80cffa0 0x7f89d815bfb0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:50.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.135+0000 7f89dea00640 1 --2- 172.21.15.148:0/13547543 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f89d80caf80 0x7f89d815ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:50.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.135+0000 7f89dea00640 1 --2- 172.21.15.148:0/13547543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d80b4d90 0x7f89d815e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:50.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.135+0000 7f89dea00640 1 -- 172.21.15.148:0/13547543 >> 172.21.15.148:0/13547543 conn(0x7f89d809d6b0 msgr2=0x7f89d80cf1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:50.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.136+0000 7f89dea00640 1 -- 172.21.15.148:0/13547543 shutdown_connections 2024-09-21T07:32:50.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.136+0000 7f89dea00640 1 -- 172.21.15.148:0/13547543 wait complete. 2024-09-21T07:32:50.157 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph osd new 950170d9-946a-4f47-84c8-bece5afd5417 2 2024-09-21T07:32:50.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.377+0000 7f05a6000640 1 -- 172.21.15.148:0/4289972641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a00b96d0 msgr2=0x7f05a00cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:50.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.377+0000 7f05a6000640 1 --2- 172.21.15.148:0/4289972641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a00b96d0 0x7f05a00cef70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f05a008cfd0 tx=0x7f059402a560 comp rx=0 tx=0).stop 2024-09-21T07:32:50.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.377+0000 7f05a6000640 1 -- 172.21.15.148:0/4289972641 shutdown_connections 2024-09-21T07:32:50.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.377+0000 7f05a6000640 1 --2- 172.21.15.148:0/4289972641 >> v2:172.21.15.39:3300/0 conn(0x7f05a00d1ae0 0x7f05a00b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:50.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.377+0000 7f05a6000640 1 --2- 172.21.15.148:0/4289972641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a00b96d0 0x7f05a00cef70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:50.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.377+0000 7f05a6000640 1 -- 172.21.15.148:0/4289972641 >> 172.21.15.148:0/4289972641 conn(0x7f05a009d660 msgr2=0x7f05a00d43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:50.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.377+0000 7f05a6000640 1 -- 172.21.15.148:0/4289972641 shutdown_connections 2024-09-21T07:32:50.379 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.377+0000 7f05a6000640 1 -- 172.21.15.148:0/4289972641 wait complete. 2024-09-21T07:32:50.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.379+0000 7f05a6000640 1 Processor -- start 2024-09-21T07:32:50.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.379+0000 7f05a6000640 1 -- start start 2024-09-21T07:32:50.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.381+0000 7f05a6000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a00b96d0 0x7f05a0166ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:50.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.381+0000 7f05a6000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f05a00cf4b0 0x7f05a00b2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:50.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.381+0000 7f05a6000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05a00d1ae0 0x7f05a01f5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:50.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.381+0000 7f05a6000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f05a00b4970 con 0x7f05a00d1ae0 2024-09-21T07:32:50.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.381+0000 7f05a6000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f05a00b47f0 con 0x7f05a00b96d0 2024-09-21T07:32:50.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.381+0000 7f05a6000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f05a00b4af0 con 0x7f05a00cf4b0 2024-09-21T07:32:50.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.381+0000 7f05a4c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a00b96d0 0x7f05a0166ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:50.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.381+0000 7f05a4c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a00b96d0 0x7f05a0166ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39826/0 (socket says 172.21.15.148:39826) 2024-09-21T07:32:50.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.381+0000 7f05a4c00640 1 -- 172.21.15.148:0/2699858756 learned_addr learned my addr 172.21.15.148:0/2699858756 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-09-21T07:32:50.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.381+0000 7f059fe00640 1 --2- 172.21.15.148:0/2699858756 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f05a00cf4b0 0x7f05a00b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:50.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.381+0000 7f05a5600640 1 --2- 172.21.15.148:0/2699858756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05a00d1ae0 0x7f05a01f5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:50.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.381+0000 7f05a4c00640 1 -- 172.21.15.148:0/2699858756 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f05a00cf4b0 msgr2=0x7f05a00b2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:50.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.381+0000 7f05a4c00640 1 --2- 172.21.15.148:0/2699858756 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f05a00cf4b0 0x7f05a00b2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:50.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.381+0000 7f05a4c00640 1 -- 172.21.15.148:0/2699858756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05a00d1ae0 msgr2=0x7f05a01f5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:50.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.381+0000 7f05a4c00640 1 --2- 172.21.15.148:0/2699858756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05a00d1ae0 0x7f05a01f5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:50.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.381+0000 7f05a4c00640 1 -- 172.21.15.148:0/2699858756 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a01f5970 con 0x7f05a00b96d0 2024-09-21T07:32:50.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.382+0000 7f05a4c00640 1 --2- 172.21.15.148:0/2699858756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a00b96d0 0x7f05a0166ba0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0594034040 tx=0x7f0594003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:32:50.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.382+0000 7f059cc00640 1 -- 172.21.15.148:0/2699858756 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059403e030 con 0x7f05a00b96d0 2024-09-21T07:32:50.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.382+0000 7f059cc00640 1 -- 172.21.15.148:0/2699858756 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0594003d10 con 0x7f05a00b96d0 2024-09-21T07:32:50.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.382+0000 7f05a6000640 1 -- 172.21.15.148:0/2699858756 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05a01f94e0 con 0x7f05a00b96d0 2024-09-21T07:32:50.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.382+0000 7f059cc00640 1 -- 172.21.15.148:0/2699858756 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0594035040 con 0x7f05a00b96d0 2024-09-21T07:32:50.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.382+0000 7f05a6000640 1 -- 172.21.15.148:0/2699858756 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05a01f9aa0 con 0x7f05a00b96d0 2024-09-21T07:32:50.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.382+0000 7f059cc00640 1 -- 172.21.15.148:0/2699858756 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0594041030 con 0x7f05a00b96d0 2024-09-21T07:32:50.385 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.382+0000 7f059cc00640 1 -- 172.21.15.148:0/2699858756 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f0594043820 con 0x7f05a00b96d0 2024-09-21T07:32:50.386 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.383+0000 7f05a6000640 1 -- 172.21.15.148:0/2699858756 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05a01f5780 con 0x7f05a00b96d0 2024-09-21T07:32:50.387 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.386+0000 7f059cc00640 1 -- 172.21.15.148:0/2699858756 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f0594043410 con 0x7f05a00b96d0 2024-09-21T07:32:50.461 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.460+0000 7f05a6000640 1 -- 172.21.15.148:0/2699858756 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd new", "uuid": "950170d9-946a-4f47-84c8-bece5afd5417", "id": 2} v 0) -- 0x7f05a00cef70 con 0x7f05a00b96d0 2024-09-21T07:32:50.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.464+0000 7f059cc00640 1 -- 172.21.15.148:0/2699858756 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "950170d9-946a-4f47-84c8-bece5afd5417", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f0594004d50 con 0x7f05a00b96d0 2024-09-21T07:32:50.466 INFO:teuthology.orchestra.run.smithi148.stdout:2 2024-09-21T07:32:50.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.466+0000 7f05a6000640 1 -- 172.21.15.148:0/2699858756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a00b96d0 msgr2=0x7f05a0166ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:50.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.466+0000 7f05a6000640 1 --2- 172.21.15.148:0/2699858756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a00b96d0 0x7f05a0166ba0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0594034040 tx=0x7f0594003820 comp rx=0 tx=0).stop 2024-09-21T07:32:50.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.466+0000 7f05a6000640 1 -- 172.21.15.148:0/2699858756 shutdown_connections 2024-09-21T07:32:50.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.466+0000 7f05a6000640 1 --2- 172.21.15.148:0/2699858756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05a00d1ae0 0x7f05a01f5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:50.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.466+0000 7f05a6000640 1 --2- 172.21.15.148:0/2699858756 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f05a00cf4b0 0x7f05a00b2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:50.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.466+0000 7f05a6000640 1 --2- 172.21.15.148:0/2699858756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a00b96d0 0x7f05a0166ba0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:50.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.466+0000 7f05a6000640 1 -- 172.21.15.148:0/2699858756 >> 172.21.15.148:0/2699858756 conn(0x7f05a009d660 msgr2=0x7f05a00b8a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:50.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.466+0000 7f05a6000640 1 -- 172.21.15.148:0/2699858756 shutdown_connections 2024-09-21T07:32:50.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.466+0000 7f05a6000640 1 -- 172.21.15.148:0/2699858756 wait complete. 2024-09-21T07:32:50.487 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph osd new a8319049-b790-44d1-9cfb-ab2c2cc16953 3 2024-09-21T07:32:50.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.706+0000 7f97c0a00640 1 -- 172.21.15.148:0/1406096558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97bc0b4d90 msgr2=0x7f97bc0d1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:50.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.706+0000 7f97c0a00640 1 --2- 172.21.15.148:0/1406096558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97bc0b4d90 0x7f97bc0d1100 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f97b4005fd0 tx=0x7f97b402aaa0 comp rx=0 tx=0).stop 2024-09-21T07:32:50.708 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.707+0000 7f97c0a00640 1 -- 172.21.15.148:0/1406096558 shutdown_connections 2024-09-21T07:32:50.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.707+0000 7f97c0a00640 1 --2- 172.21.15.148:0/1406096558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97bc0b4d90 0x7f97bc0d1100 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:50.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.707+0000 7f97c0a00640 1 -- 172.21.15.148:0/1406096558 >> 172.21.15.148:0/1406096558 conn(0x7f97bc09d6b0 msgr2=0x7f97bc0d4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:50.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.707+0000 7f97c0a00640 1 -- 172.21.15.148:0/1406096558 shutdown_connections 2024-09-21T07:32:50.709 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.707+0000 7f97c0a00640 1 -- 172.21.15.148:0/1406096558 wait complete. 2024-09-21T07:32:50.711 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.710+0000 7f97c0a00640 1 Processor -- start 2024-09-21T07:32:50.712 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.711+0000 7f97c0a00640 1 -- start start 2024-09-21T07:32:50.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.713+0000 7f97c0a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97bc0b4d90 0x7f97bc161b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:50.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.713+0000 7f97c0a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97bc0caf80 0x7f97bc0b8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:50.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.713+0000 7f97c0a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f97bc0cffa0 0x7f97bc0b8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:50.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.713+0000 7f97c0a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc0b7cf0 con 0x7f97bc0b4d90 2024-09-21T07:32:50.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.713+0000 7f97c0a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc0b7b70 con 0x7f97bc0caf80 2024-09-21T07:32:50.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.713+0000 7f97c0a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f97bc0b7e70 con 0x7f97bc0cffa0 2024-09-21T07:32:50.715 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.713+0000 7f97bb200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f97bc0cffa0 0x7f97bc0b8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:50.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.713+0000 7f97ba800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97bc0b4d90 0x7f97bc161b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:50.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.713+0000 7f97b9e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97bc0caf80 0x7f97bc0b8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:50.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.713+0000 7f97ba800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97bc0b4d90 0x7f97bc161b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.148:42590/0 (socket says 172.21.15.148:42590) 2024-09-21T07:32:50.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.713+0000 7f97bb200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f97bc0cffa0 0x7f97bc0b8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.148:56232/0 (socket says 172.21.15.148:56232) 2024-09-21T07:32:50.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.713+0000 7f97ba800640 1 -- 172.21.15.148:0/453731712 learned_addr learned my addr 172.21.15.148:0/453731712 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-09-21T07:32:50.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.714+0000 7f97bb200640 1 -- 172.21.15.148:0/453731712 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97bc0caf80 msgr2=0x7f97bc0b8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:50.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.714+0000 7f97bb200640 1 --2- 172.21.15.148:0/453731712 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97bc0caf80 0x7f97bc0b8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:50.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.714+0000 7f97bb200640 1 -- 172.21.15.148:0/453731712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97bc0b4d90 msgr2=0x7f97bc161b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:50.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.714+0000 7f97bb200640 1 --2- 172.21.15.148:0/453731712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97bc0b4d90 0x7f97bc161b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:50.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.714+0000 7f97bb200640 1 -- 172.21.15.148:0/453731712 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97bc167450 con 0x7f97bc0cffa0 2024-09-21T07:32:50.716 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.714+0000 7f97b9e00640 1 --2- 172.21.15.148:0/453731712 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97bc0caf80 0x7f97bc0b8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:32:50.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.714+0000 7f97bb200640 1 --2- 172.21.15.148:0/453731712 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f97bc0cffa0 0x7f97bc0b8650 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f97bc0ba230 tx=0x7f97b4003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:32:50.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.715+0000 7f97b2a00640 1 -- 172.21.15.148:0/453731712 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b4044070 con 0x7f97bc0cffa0 2024-09-21T07:32:50.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.715+0000 7f97b2a00640 1 -- 172.21.15.148:0/453731712 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f97b4004250 con 0x7f97bc0cffa0 2024-09-21T07:32:50.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.715+0000 7f97c0a00640 1 -- 172.21.15.148:0/453731712 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97bc169a30 con 0x7f97bc0cffa0 2024-09-21T07:32:50.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.715+0000 7f97b2a00640 1 -- 172.21.15.148:0/453731712 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b40046a0 con 0x7f97bc0cffa0 2024-09-21T07:32:50.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.715+0000 7f97c0a00640 1 -- 172.21.15.148:0/453731712 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f97bc0b0fa0 con 0x7f97bc0cffa0 2024-09-21T07:32:50.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.715+0000 7f97b2a00640 1 -- 172.21.15.148:0/453731712 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f97b4047070 con 0x7f97bc0cffa0 2024-09-21T07:32:50.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.715+0000 7f97b2a00640 1 -- 172.21.15.148:0/453731712 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f97b403f8a0 con 0x7f97bc0cffa0 2024-09-21T07:32:50.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.716+0000 7f97c0a00640 1 -- 172.21.15.148:0/453731712 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97bc0b71c0 con 0x7f97bc0cffa0 2024-09-21T07:32:50.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.718+0000 7f97b2a00640 1 -- 172.21.15.148:0/453731712 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f97b404e070 con 0x7f97bc0cffa0 2024-09-21T07:32:50.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.791+0000 7f97c0a00640 1 -- 172.21.15.148:0/453731712 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a8319049-b790-44d1-9cfb-ab2c2cc16953", "id": 3} v 0) -- 0x7f97bc0cd760 con 0x7f97bc0cffa0 2024-09-21T07:32:50.801 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.800+0000 7f97b2a00640 1 -- 172.21.15.148:0/453731712 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a8319049-b790-44d1-9cfb-ab2c2cc16953", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f97b4032030 con 0x7f97bc0cffa0 2024-09-21T07:32:50.801 INFO:teuthology.orchestra.run.smithi148.stdout:3 2024-09-21T07:32:50.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.801+0000 7f97c0a00640 1 -- 172.21.15.148:0/453731712 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f97bc0cffa0 msgr2=0x7f97bc0b8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:50.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.801+0000 7f97c0a00640 1 --2- 172.21.15.148:0/453731712 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f97bc0cffa0 0x7f97bc0b8650 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f97bc0ba230 tx=0x7f97b4003f10 comp rx=0 tx=0).stop 2024-09-21T07:32:50.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.801+0000 7f97c0a00640 1 -- 172.21.15.148:0/453731712 shutdown_connections 2024-09-21T07:32:50.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.801+0000 7f97c0a00640 1 --2- 172.21.15.148:0/453731712 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f97bc0cffa0 0x7f97bc0b8650 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:50.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.801+0000 7f97c0a00640 1 --2- 172.21.15.148:0/453731712 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97bc0caf80 0x7f97bc0b8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:50.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.801+0000 7f97c0a00640 1 --2- 172.21.15.148:0/453731712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97bc0b4d90 0x7f97bc161b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:50.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.801+0000 7f97c0a00640 1 -- 172.21.15.148:0/453731712 >> 172.21.15.148:0/453731712 conn(0x7f97bc09d6b0 msgr2=0x7f97bc0cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:50.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.801+0000 7f97c0a00640 1 -- 172.21.15.148:0/453731712 shutdown_connections 2024-09-21T07:32:50.803 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:50.801+0000 7f97c0a00640 1 -- 172.21.15.148:0/453731712 wait complete. 2024-09-21T07:32:50.822 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph osd new 17c4fdd3-3d30-4e97-8319-7084cead1295 4 2024-09-21T07:32:51.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.041+0000 7f5e43200640 1 -- 172.21.15.148:0/4176291585 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5e440d1320 msgr2=0x7f5e440cefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:51.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.041+0000 7f5e43200640 1 --2- 172.21.15.148:0/4176291585 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5e440d1320 0x7f5e440cefd0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5e30009830 tx=0x7f5e3002a510 comp rx=0 tx=0).stop 2024-09-21T07:32:51.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.041+0000 7f5e43200640 1 -- 172.21.15.148:0/4176291585 shutdown_connections 2024-09-21T07:32:51.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.041+0000 7f5e43200640 1 --2- 172.21.15.148:0/4176291585 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5e440d1320 0x7f5e440cefd0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:51.043 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.041+0000 7f5e43200640 1 --2- 172.21.15.148:0/4176291585 >> v2:172.21.15.39:3300/0 conn(0x7f5e440b96b0 0x7f5e440d0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:51.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.041+0000 7f5e43200640 1 -- 172.21.15.148:0/4176291585 >> 172.21.15.148:0/4176291585 conn(0x7f5e4409d680 msgr2=0x7f5e440d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:51.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.041+0000 7f5e43200640 1 -- 172.21.15.148:0/4176291585 shutdown_connections 2024-09-21T07:32:51.044 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.042+0000 7f5e43200640 1 -- 172.21.15.148:0/4176291585 wait complete. 2024-09-21T07:32:51.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.044+0000 7f5e43200640 1 Processor -- start 2024-09-21T07:32:51.045 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.044+0000 7f5e43200640 1 -- start start 2024-09-21T07:32:51.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.045+0000 7f5e43200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e440b96b0 0x7f5e441573b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:51.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.045+0000 7f5e43200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5e440cf510 0x7f5e440b2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:51.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.045+0000 7f5e43200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e440d1320 0x7f5e44167160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:51.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.045+0000 7f5e43200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5e440b4a00 con 0x7f5e440b96b0 2024-09-21T07:32:51.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.045+0000 7f5e43200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5e440b50d0 con 0x7f5e440d1320 2024-09-21T07:32:51.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.045+0000 7f5e43200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f5e440b4b80 con 0x7f5e440cf510 2024-09-21T07:32:51.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.045+0000 7f5e41400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5e440cf510 0x7f5e440b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:51.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.046+0000 7f5e41400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5e440cf510 0x7f5e440b2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.148:56246/0 (socket says 172.21.15.148:56246) 2024-09-21T07:32:51.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.046+0000 7f5e41e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e440b96b0 0x7f5e441573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:51.047 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.046+0000 7f5e41400640 1 -- 172.21.15.148:0/709208977 learned_addr learned my addr 172.21.15.148:0/709208977 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-09-21T07:32:51.048 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.046+0000 7f5e42800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e440d1320 0x7f5e44167160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:51.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.046+0000 7f5e41400640 1 -- 172.21.15.148:0/709208977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e440d1320 msgr2=0x7f5e44167160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:51.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.046+0000 7f5e41400640 1 --2- 172.21.15.148:0/709208977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e440d1320 0x7f5e44167160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:51.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.046+0000 7f5e41400640 1 -- 172.21.15.148:0/709208977 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e440b96b0 msgr2=0x7f5e441573b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:51.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.046+0000 7f5e41400640 1 --2- 172.21.15.148:0/709208977 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e440b96b0 0x7f5e441573b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:51.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.046+0000 7f5e41400640 1 -- 172.21.15.148:0/709208977 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e441578f0 con 0x7f5e440cf510 2024-09-21T07:32:51.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.046+0000 7f5e42800640 1 --2- 172.21.15.148:0/709208977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e440d1320 0x7f5e44167160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:32:51.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.046+0000 7f5e41400640 1 --2- 172.21.15.148:0/709208977 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5e440cf510 0x7f5e440b2590 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5e3002ab00 tx=0x7f5e30033c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:32:51.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.047+0000 7f5e3a000640 1 -- 172.21.15.148:0/709208977 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e30005c90 con 0x7f5e440cf510 2024-09-21T07:32:51.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.047+0000 7f5e3a000640 1 -- 172.21.15.148:0/709208977 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e30031e50 con 0x7f5e440cf510 2024-09-21T07:32:51.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.047+0000 7f5e43200640 1 -- 172.21.15.148:0/709208977 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e441f8ba0 con 0x7f5e440cf510 2024-09-21T07:32:51.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.047+0000 7f5e3a000640 1 -- 172.21.15.148:0/709208977 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e30005260 con 0x7f5e440cf510 2024-09-21T07:32:51.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.047+0000 7f5e43200640 1 -- 172.21.15.148:0/709208977 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e441f9130 con 0x7f5e440cf510 2024-09-21T07:32:51.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.047+0000 7f5e3a000640 1 -- 172.21.15.148:0/709208977 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5e300047a0 con 0x7f5e440cf510 2024-09-21T07:32:51.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.047+0000 7f5e3a000640 1 -- 172.21.15.148:0/709208977 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f5e300035f0 con 0x7f5e440cf510 2024-09-21T07:32:51.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.048+0000 7f5e43200640 1 -- 172.21.15.148:0/709208977 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e440d0de0 con 0x7f5e440cf510 2024-09-21T07:32:51.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.050+0000 7f5e3a000640 1 -- 172.21.15.148:0/709208977 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f5e441578f0 con 0x7f5e440cf510 2024-09-21T07:32:51.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.123+0000 7f5e43200640 1 -- 172.21.15.148:0/709208977 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "osd new", "uuid": "17c4fdd3-3d30-4e97-8319-7084cead1295", "id": 4} v 0) -- 0x7f5e440b2e20 con 0x7f5e440cf510 2024-09-21T07:32:51.133 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.132+0000 7f5e3a000640 1 -- 172.21.15.148:0/709208977 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "17c4fdd3-3d30-4e97-8319-7084cead1295", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f5e3003f030 con 0x7f5e440cf510 2024-09-21T07:32:51.133 INFO:teuthology.orchestra.run.smithi148.stdout:4 2024-09-21T07:32:51.134 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.133+0000 7f5e43200640 1 -- 172.21.15.148:0/709208977 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5e440cf510 msgr2=0x7f5e440b2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:51.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.133+0000 7f5e43200640 1 --2- 172.21.15.148:0/709208977 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5e440cf510 0x7f5e440b2590 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5e3002ab00 tx=0x7f5e30033c20 comp rx=0 tx=0).stop 2024-09-21T07:32:51.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.133+0000 7f5e43200640 1 -- 172.21.15.148:0/709208977 shutdown_connections 2024-09-21T07:32:51.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.133+0000 7f5e43200640 1 --2- 172.21.15.148:0/709208977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e440d1320 0x7f5e44167160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:51.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.133+0000 7f5e43200640 1 --2- 172.21.15.148:0/709208977 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5e440cf510 0x7f5e440b2590 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:51.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.133+0000 7f5e43200640 1 --2- 172.21.15.148:0/709208977 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e440b96b0 0x7f5e441573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:51.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.133+0000 7f5e43200640 1 -- 172.21.15.148:0/709208977 >> 172.21.15.148:0/709208977 conn(0x7f5e4409d680 msgr2=0x7f5e440b71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:51.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.133+0000 7f5e43200640 1 -- 172.21.15.148:0/709208977 shutdown_connections 2024-09-21T07:32:51.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.133+0000 7f5e43200640 1 -- 172.21.15.148:0/709208977 wait complete. 2024-09-21T07:32:51.155 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph osd new e46b3734-6d7b-4d66-8934-2799d12ba363 5 2024-09-21T07:32:51.375 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.374+0000 7fa3f9a00640 1 -- 172.21.15.148:0/2523902536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3f4145720 msgr2=0x7fa3f40cf390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:51.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.374+0000 7fa3f9a00640 1 --2- 172.21.15.148:0/2523902536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3f4145720 0x7fa3f40cf390 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa3e40098e0 tx=0x7fa3e402a5d0 comp rx=0 tx=0).stop 2024-09-21T07:32:51.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.374+0000 7fa3f9a00640 1 -- 172.21.15.148:0/2523902536 shutdown_connections 2024-09-21T07:32:51.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.374+0000 7fa3f9a00640 1 --2- 172.21.15.148:0/2523902536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3f4145720 0x7fa3f40cf390 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:51.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.374+0000 7fa3f9a00640 1 -- 172.21.15.148:0/2523902536 >> 172.21.15.148:0/2523902536 conn(0x7fa3f40b1bb0 msgr2=0x7fa3f40b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:51.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.374+0000 7fa3f9a00640 1 -- 172.21.15.148:0/2523902536 shutdown_connections 2024-09-21T07:32:51.376 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.374+0000 7fa3f9a00640 1 -- 172.21.15.148:0/2523902536 wait complete. 2024-09-21T07:32:51.377 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.376+0000 7fa3f9a00640 1 Processor -- start 2024-09-21T07:32:51.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.376+0000 7fa3f9a00640 1 -- start start 2024-09-21T07:32:51.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.378+0000 7fa3f9a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa3f40749f0 0x7fa3f4141bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:51.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.378+0000 7fa3f9a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3f41429f0 0x7fa3f41f9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:51.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.378+0000 7fa3f9a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3f4145720 0x7fa3f40aea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:51.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.378+0000 7fa3f9a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f40b3e90 con 0x7fa3f41429f0 2024-09-21T07:32:51.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.378+0000 7fa3f9a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f40b3d10 con 0x7fa3f4145720 2024-09-21T07:32:51.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.378+0000 7fa3f9a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa3f40b4010 con 0x7fa3f40749f0 2024-09-21T07:32:51.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.378+0000 7fa3f3e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa3f40749f0 0x7fa3f4141bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:51.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.378+0000 7fa3f3e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa3f40749f0 0x7fa3f4141bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.148:56260/0 (socket says 172.21.15.148:56260) 2024-09-21T07:32:51.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.378+0000 7fa3f3e00640 1 -- 172.21.15.148:0/4038815792 learned_addr learned my addr 172.21.15.148:0/4038815792 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-09-21T07:32:51.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.378+0000 7fa3f9000640 1 --2- 172.21.15.148:0/4038815792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3f4145720 0x7fa3f40aea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:51.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.378+0000 7fa3ebe00640 1 --2- 172.21.15.148:0/4038815792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3f41429f0 0x7fa3f41f9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:51.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.379+0000 7fa3f9000640 1 -- 172.21.15.148:0/4038815792 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa3f40749f0 msgr2=0x7fa3f4141bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:51.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.379+0000 7fa3f9000640 1 --2- 172.21.15.148:0/4038815792 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa3f40749f0 0x7fa3f4141bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:51.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.379+0000 7fa3f9000640 1 -- 172.21.15.148:0/4038815792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3f41429f0 msgr2=0x7fa3f41f9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:51.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.379+0000 7fa3f9000640 1 --2- 172.21.15.148:0/4038815792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3f41429f0 0x7fa3f41f9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:51.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.379+0000 7fa3f9000640 1 -- 172.21.15.148:0/4038815792 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f41f9980 con 0x7fa3f4145720 2024-09-21T07:32:51.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.379+0000 7fa3f3e00640 1 --2- 172.21.15.148:0/4038815792 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa3f40749f0 0x7fa3f4141bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:32:51.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.379+0000 7fa3f9000640 1 --2- 172.21.15.148:0/4038815792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3f4145720 0x7fa3f40aea20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec00c730 tx=0x7fa3ec00cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:32:51.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.379+0000 7fa3f0c00640 1 -- 172.21.15.148:0/4038815792 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec003b70 con 0x7fa3f4145720 2024-09-21T07:32:51.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.379+0000 7fa3f0c00640 1 -- 172.21.15.148:0/4038815792 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3ec003d10 con 0x7fa3f4145720 2024-09-21T07:32:51.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.379+0000 7fa3f9a00640 1 -- 172.21.15.148:0/4038815792 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3f415ec40 con 0x7fa3f4145720 2024-09-21T07:32:51.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.379+0000 7fa3f0c00640 1 -- 172.21.15.148:0/4038815792 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec005330 con 0x7fa3f4145720 2024-09-21T07:32:51.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.379+0000 7fa3f9a00640 1 -- 172.21.15.148:0/4038815792 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3f415c7b0 con 0x7fa3f4145720 2024-09-21T07:32:51.382 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.379+0000 7fa3f0c00640 1 -- 172.21.15.148:0/4038815792 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa3ec0054d0 con 0x7fa3f4145720 2024-09-21T07:32:51.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.380+0000 7fa3f0c00640 1 -- 172.21.15.148:0/4038815792 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fa3ec0057c0 con 0x7fa3f4145720 2024-09-21T07:32:51.383 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.380+0000 7fa3f9a00640 1 -- 172.21.15.148:0/4038815792 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3f40cf390 con 0x7fa3f4145720 2024-09-21T07:32:51.384 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.383+0000 7fa3f0c00640 1 -- 172.21.15.148:0/4038815792 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fa3f41f9980 con 0x7fa3f4145720 2024-09-21T07:32:51.458 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.457+0000 7fa3f9a00640 1 -- 172.21.15.148:0/4038815792 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e46b3734-6d7b-4d66-8934-2799d12ba363", "id": 5} v 0) -- 0x7fa3f4144200 con 0x7fa3f4145720 2024-09-21T07:32:51.466 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.465+0000 7fa3f0c00640 1 -- 172.21.15.148:0/4038815792 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e46b3734-6d7b-4d66-8934-2799d12ba363", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fa3ec00a640 con 0x7fa3f4145720 2024-09-21T07:32:51.466 INFO:teuthology.orchestra.run.smithi148.stdout:5 2024-09-21T07:32:51.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.466+0000 7fa3f9a00640 1 -- 172.21.15.148:0/4038815792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3f4145720 msgr2=0x7fa3f40aea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:51.467 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.466+0000 7fa3f9a00640 1 --2- 172.21.15.148:0/4038815792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3f4145720 0x7fa3f40aea20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec00c730 tx=0x7fa3ec00cbf0 comp rx=0 tx=0).stop 2024-09-21T07:32:51.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.466+0000 7fa3f9a00640 1 -- 172.21.15.148:0/4038815792 shutdown_connections 2024-09-21T07:32:51.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.466+0000 7fa3f9a00640 1 --2- 172.21.15.148:0/4038815792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3f4145720 0x7fa3f40aea20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:51.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.466+0000 7fa3f9a00640 1 --2- 172.21.15.148:0/4038815792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3f41429f0 0x7fa3f41f9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:51.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.466+0000 7fa3f9a00640 1 --2- 172.21.15.148:0/4038815792 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa3f40749f0 0x7fa3f4141bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:51.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.466+0000 7fa3f9a00640 1 -- 172.21.15.148:0/4038815792 >> 172.21.15.148:0/4038815792 conn(0x7fa3f40b1bb0 msgr2=0x7fa3f4144a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:51.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.466+0000 7fa3f9a00640 1 -- 172.21.15.148:0/4038815792 shutdown_connections 2024-09-21T07:32:51.468 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.466+0000 7fa3f9a00640 1 -- 172.21.15.148:0/4038815792 wait complete. 2024-09-21T07:32:51.489 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph osd new 68bfcd21-68a0-450f-8646-dca1877516cc 6 2024-09-21T07:32:51.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.715+0000 7f9410400640 1 -- 172.21.15.148:0/2941454598 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f94080704b0 msgr2=0x7f94080b2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:51.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.715+0000 7f9410400640 1 --2- 172.21.15.148:0/2941454598 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f94080704b0 0x7f94080b2fb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9408074570 tx=0x7f93f402a440 comp rx=0 tx=0).stop 2024-09-21T07:32:51.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.716+0000 7f9410400640 1 -- 172.21.15.148:0/2941454598 shutdown_connections 2024-09-21T07:32:51.717 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.716+0000 7f9410400640 1 --2- 172.21.15.148:0/2941454598 >> v2:172.21.15.148:3300/0 conn(0x7f94080ca7b0 0x7f94080cabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:51.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.716+0000 7f9410400640 1 --2- 172.21.15.148:0/2941454598 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f94080704b0 0x7f94080b2fb0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:51.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.716+0000 7f9410400640 1 --2- 172.21.15.148:0/2941454598 >> v2:172.21.15.39:3300/0 conn(0x7f94080cc640 0x7f94080b56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:51.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.716+0000 7f9410400640 1 -- 172.21.15.148:0/2941454598 >> 172.21.15.148:0/2941454598 conn(0x7f94080d0160 msgr2=0x7f94080d0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:51.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.716+0000 7f9410400640 1 -- 172.21.15.148:0/2941454598 shutdown_connections 2024-09-21T07:32:51.718 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.716+0000 7f9410400640 1 -- 172.21.15.148:0/2941454598 wait complete. 2024-09-21T07:32:51.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.718+0000 7f9410400640 1 Processor -- start 2024-09-21T07:32:51.719 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.718+0000 7f9410400640 1 -- start start 2024-09-21T07:32:51.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.719+0000 7f9410400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94080704b0 0x7f94080b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:51.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.719+0000 7f9410400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f94080ca7b0 0x7f940815a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:51.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.719+0000 7f9410400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94080cc640 0x7f94080c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:51.722 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.719+0000 7f9410400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f94080cb4f0 con 0x7f94080cc640 2024-09-21T07:32:51.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.719+0000 7f9410400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f94080cb370 con 0x7f94080704b0 2024-09-21T07:32:51.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.719+0000 7f9410400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f94080cb670 con 0x7f94080ca7b0 2024-09-21T07:32:51.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.720+0000 7f940da00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94080704b0 0x7f94080b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:51.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.720+0000 7f940da00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94080704b0 0x7f94080b1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39954/0 (socket says 172.21.15.148:39954) 2024-09-21T07:32:51.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.720+0000 7f940da00640 1 -- 172.21.15.148:0/1477036512 learned_addr learned my addr 172.21.15.148:0/1477036512 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-09-21T07:32:51.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.720+0000 7f940d000640 1 --2- 172.21.15.148:0/1477036512 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f94080ca7b0 0x7f940815a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:51.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.720+0000 7f940e400640 1 --2- 172.21.15.148:0/1477036512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94080cc640 0x7f94080c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:51.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.720+0000 7f940d000640 1 -- 172.21.15.148:0/1477036512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94080704b0 msgr2=0x7f94080b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:51.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.720+0000 7f940d000640 1 --2- 172.21.15.148:0/1477036512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94080704b0 0x7f94080b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:51.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.720+0000 7f940d000640 1 -- 172.21.15.148:0/1477036512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94080cc640 msgr2=0x7f94080c9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:51.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.720+0000 7f940d000640 1 --2- 172.21.15.148:0/1477036512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94080cc640 0x7f94080c9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:51.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.720+0000 7f940d000640 1 -- 172.21.15.148:0/1477036512 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9408148240 con 0x7f94080ca7b0 2024-09-21T07:32:51.723 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.720+0000 7f940e400640 1 --2- 172.21.15.148:0/1477036512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94080cc640 0x7f94080c9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:32:51.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.720+0000 7f940da00640 1 --2- 172.21.15.148:0/1477036512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94080704b0 0x7f94080b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:32:51.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.720+0000 7f940d000640 1 --2- 172.21.15.148:0/1477036512 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f94080ca7b0 0x7f940815a3c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f93f4031000 tx=0x7f93f40038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:32:51.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.721+0000 7f9405600640 1 -- 172.21.15.148:0/1477036512 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f403e030 con 0x7f94080ca7b0 2024-09-21T07:32:51.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.721+0000 7f9405600640 1 -- 172.21.15.148:0/1477036512 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93f4003ca0 con 0x7f94080ca7b0 2024-09-21T07:32:51.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.721+0000 7f9410400640 1 -- 172.21.15.148:0/1477036512 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9408148520 con 0x7f94080ca7b0 2024-09-21T07:32:51.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.721+0000 7f9405600640 1 -- 172.21.15.148:0/1477036512 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f4035040 con 0x7f94080ca7b0 2024-09-21T07:32:51.724 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.721+0000 7f9410400640 1 -- 172.21.15.148:0/1477036512 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9408157800 con 0x7f94080ca7b0 2024-09-21T07:32:51.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.722+0000 7f9410400640 1 -- 172.21.15.148:0/1477036512 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94080cbc00 con 0x7f94080ca7b0 2024-09-21T07:32:51.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.722+0000 7f9405600640 1 -- 172.21.15.148:0/1477036512 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 2) ==== 50742+0+0 (secure 0 0 0) 0x7f93f4041030 con 0x7f94080ca7b0 2024-09-21T07:32:51.725 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.723+0000 7f9405600640 1 -- 172.21.15.148:0/1477036512 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f93f4072bd0 con 0x7f94080ca7b0 2024-09-21T07:32:51.726 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.725+0000 7f9405600640 1 -- 172.21.15.148:0/1477036512 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f93f403fd00 con 0x7f94080ca7b0 2024-09-21T07:32:51.797 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.796+0000 7f9410400640 1 -- 172.21.15.148:0/1477036512 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "osd new", "uuid": "68bfcd21-68a0-450f-8646-dca1877516cc", "id": 6} v 0) -- 0x7f9408145600 con 0x7f94080ca7b0 2024-09-21T07:32:51.807 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.805+0000 7f9405600640 1 -- 172.21.15.148:0/1477036512 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "68bfcd21-68a0-450f-8646-dca1877516cc", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f93f4004d30 con 0x7f94080ca7b0 2024-09-21T07:32:51.807 INFO:teuthology.orchestra.run.smithi148.stdout:6 2024-09-21T07:32:51.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.807+0000 7f9410400640 1 -- 172.21.15.148:0/1477036512 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f94080ca7b0 msgr2=0x7f940815a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:51.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.807+0000 7f9410400640 1 --2- 172.21.15.148:0/1477036512 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f94080ca7b0 0x7f940815a3c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f93f4031000 tx=0x7f93f40038a0 comp rx=0 tx=0).stop 2024-09-21T07:32:51.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.807+0000 7f9410400640 1 -- 172.21.15.148:0/1477036512 shutdown_connections 2024-09-21T07:32:51.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.807+0000 7f9410400640 1 --2- 172.21.15.148:0/1477036512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94080cc640 0x7f94080c9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:51.808 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.807+0000 7f9410400640 1 --2- 172.21.15.148:0/1477036512 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f94080ca7b0 0x7f940815a3c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:51.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.807+0000 7f9410400640 1 --2- 172.21.15.148:0/1477036512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94080704b0 0x7f94080b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:51.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.807+0000 7f9410400640 1 -- 172.21.15.148:0/1477036512 >> 172.21.15.148:0/1477036512 conn(0x7f94080d0160 msgr2=0x7f94080b4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:51.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.807+0000 7f9410400640 1 -- 172.21.15.148:0/1477036512 shutdown_connections 2024-09-21T07:32:51.809 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:51.807+0000 7f9410400640 1 -- 172.21.15.148:0/1477036512 wait complete. 2024-09-21T07:32:51.831 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster ceph osd new bb4c236d-8a7b-45a9-8cef-303a8b1adb7c 7 2024-09-21T07:32:52.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.047+0000 7efe34600640 1 -- 172.21.15.148:0/2251913393 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe2c0ccb50 msgr2=0x7efe2c0ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:52.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.047+0000 7efe34600640 1 --2- 172.21.15.148:0/2251913393 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe2c0ccb50 0x7efe2c0ccf60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efe24005fd0 tx=0x7efe2402aa90 comp rx=0 tx=0).stop 2024-09-21T07:32:52.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.047+0000 7efe34600640 1 -- 172.21.15.148:0/2251913393 shutdown_connections 2024-09-21T07:32:52.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.047+0000 7efe34600640 1 --2- 172.21.15.148:0/2251913393 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe2c0ccb50 0x7efe2c0ccf60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:52.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.047+0000 7efe34600640 1 --2- 172.21.15.148:0/2251913393 >> v2:172.21.15.39:3300/0 conn(0x7efe2c0af3c0 0x7efe2c0b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:52.049 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.047+0000 7efe34600640 1 -- 172.21.15.148:0/2251913393 >> 172.21.15.148:0/2251913393 conn(0x7efe2c06d7d0 msgr2=0x7efe2c0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:52.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.047+0000 7efe34600640 1 -- 172.21.15.148:0/2251913393 shutdown_connections 2024-09-21T07:32:52.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.048+0000 7efe34600640 1 -- 172.21.15.148:0/2251913393 wait complete. 2024-09-21T07:32:52.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.050+0000 7efe34600640 1 Processor -- start 2024-09-21T07:32:52.051 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.050+0000 7efe34600640 1 -- start start 2024-09-21T07:32:52.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.051+0000 7efe34600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe2c0af3c0 0x7efe2c15a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:52.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.051+0000 7efe34600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe2c0cc1e0 0x7efe2c09d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:52.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.051+0000 7efe34600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe2c0ccb50 0x7efe2c1e9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:52.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.051+0000 7efe34600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efe2c153520 con 0x7efe2c0cc1e0 2024-09-21T07:32:52.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.051+0000 7efe34600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efe2c1533a0 con 0x7efe2c0af3c0 2024-09-21T07:32:52.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.051+0000 7efe34600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7efe2c1536a0 con 0x7efe2c0ccb50 2024-09-21T07:32:52.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.051+0000 7efe31c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe2c0af3c0 0x7efe2c15a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:52.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.052+0000 7efe32600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe2c0ccb50 0x7efe2c1e9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:52.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.052+0000 7efe31c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe2c0af3c0 0x7efe2c15a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39958/0 (socket says 172.21.15.148:39958) 2024-09-21T07:32:52.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.052+0000 7efe31c00640 1 -- 172.21.15.148:0/632650782 learned_addr learned my addr 172.21.15.148:0/632650782 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-09-21T07:32:52.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.052+0000 7efe31200640 1 --2- 172.21.15.148:0/632650782 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe2c0cc1e0 0x7efe2c09d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:52.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.052+0000 7efe32600640 1 -- 172.21.15.148:0/632650782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe2c0af3c0 msgr2=0x7efe2c15a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:52.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.052+0000 7efe32600640 1 --2- 172.21.15.148:0/632650782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe2c0af3c0 0x7efe2c15a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:52.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.052+0000 7efe32600640 1 -- 172.21.15.148:0/632650782 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe2c0cc1e0 msgr2=0x7efe2c09d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:52.054 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.052+0000 7efe32600640 1 --2- 172.21.15.148:0/632650782 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe2c0cc1e0 0x7efe2c09d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:52.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.052+0000 7efe32600640 1 -- 172.21.15.148:0/632650782 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe2c1fa2b0 con 0x7efe2c0ccb50 2024-09-21T07:32:52.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.052+0000 7efe31c00640 1 --2- 172.21.15.148:0/632650782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe2c0af3c0 0x7efe2c15a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:32:52.055 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.052+0000 7efe32600640 1 --2- 172.21.15.148:0/632650782 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe2c0ccb50 0x7efe2c1e9310 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efe2c15a9f0 tx=0x7efe24003ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:32:52.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.053+0000 7efe29600640 1 -- 172.21.15.148:0/632650782 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe24044070 con 0x7efe2c0ccb50 2024-09-21T07:32:52.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.053+0000 7efe29600640 1 -- 172.21.15.148:0/632650782 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe24003e50 con 0x7efe2c0ccb50 2024-09-21T07:32:52.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.053+0000 7efe29600640 1 -- 172.21.15.148:0/632650782 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe240044b0 con 0x7efe2c0ccb50 2024-09-21T07:32:52.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.053+0000 7efe34600640 1 -- 172.21.15.148:0/632650782 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe2c1fa590 con 0x7efe2c0ccb50 2024-09-21T07:32:52.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.053+0000 7efe34600640 1 -- 172.21.15.148:0/632650782 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7efe2c1fab50 con 0x7efe2c0ccb50 2024-09-21T07:32:52.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.054+0000 7efe34600640 1 -- 172.21.15.148:0/632650782 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe2c152dc0 con 0x7efe2c0ccb50 2024-09-21T07:32:52.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.054+0000 7efe29600640 1 -- 172.21.15.148:0/632650782 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 2) ==== 50742+0+0 (secure 0 0 0) 0x7efe24047070 con 0x7efe2c0ccb50 2024-09-21T07:32:52.056 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.054+0000 7efe29600640 1 -- 172.21.15.148:0/632650782 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7efe240731c0 con 0x7efe2c0ccb50 2024-09-21T07:32:52.057 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.056+0000 7efe29600640 1 -- 172.21.15.148:0/632650782 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7efe24032030 con 0x7efe2c0ccb50 2024-09-21T07:32:52.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.127+0000 7efe34600640 1 -- 172.21.15.148:0/632650782 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "osd new", "uuid": "bb4c236d-8a7b-45a9-8cef-303a8b1adb7c", "id": 7} v 0) -- 0x7efe2c153b50 con 0x7efe2c0ccb50 2024-09-21T07:32:52.135 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.134+0000 7efe29600640 1 -- 172.21.15.148:0/632650782 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bb4c236d-8a7b-45a9-8cef-303a8b1adb7c", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7efe2402d3c0 con 0x7efe2c0ccb50 2024-09-21T07:32:52.135 INFO:teuthology.orchestra.run.smithi148.stdout:7 2024-09-21T07:32:52.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.135+0000 7efe34600640 1 -- 172.21.15.148:0/632650782 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe2c0ccb50 msgr2=0x7efe2c1e9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:52.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.135+0000 7efe34600640 1 --2- 172.21.15.148:0/632650782 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe2c0ccb50 0x7efe2c1e9310 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efe2c15a9f0 tx=0x7efe24003ce0 comp rx=0 tx=0).stop 2024-09-21T07:32:52.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.135+0000 7efe34600640 1 -- 172.21.15.148:0/632650782 shutdown_connections 2024-09-21T07:32:52.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.135+0000 7efe34600640 1 --2- 172.21.15.148:0/632650782 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe2c0ccb50 0x7efe2c1e9310 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:52.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.135+0000 7efe34600640 1 --2- 172.21.15.148:0/632650782 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe2c0cc1e0 0x7efe2c09d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:52.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.135+0000 7efe34600640 1 --2- 172.21.15.148:0/632650782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe2c0af3c0 0x7efe2c15a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:52.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.135+0000 7efe34600640 1 -- 172.21.15.148:0/632650782 >> 172.21.15.148:0/632650782 conn(0x7efe2c06d7d0 msgr2=0x7efe2c0b08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:52.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.136+0000 7efe34600640 1 -- 172.21.15.148:0/632650782 shutdown_connections 2024-09-21T07:32:52.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:32:52.136+0000 7efe34600640 1 -- 172.21.15.148:0/632650782 wait complete. 2024-09-21T07:32:52.160 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-21T07:32:52.160 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-21T07:32:52.164 INFO:tasks.ceph.osd.0:Started 2024-09-21T07:32:52.164 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-21T07:32:52.164 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-21T07:32:52.169 INFO:tasks.ceph.osd.1:Started 2024-09-21T07:32:52.169 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-21T07:32:52.169 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-21T07:32:52.174 INFO:tasks.ceph.osd.2:Started 2024-09-21T07:32:52.174 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-21T07:32:52.174 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-21T07:32:52.178 INFO:tasks.ceph.osd.3:Started 2024-09-21T07:32:52.179 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-21T07:32:52.179 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-21T07:32:52.183 INFO:tasks.ceph.osd.4:Started 2024-09-21T07:32:52.183 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-21T07:32:52.183 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-21T07:32:52.186 INFO:tasks.ceph.osd.5:Started 2024-09-21T07:32:52.186 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-21T07:32:52.187 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-21T07:32:52.190 INFO:tasks.ceph.osd.6:Started 2024-09-21T07:32:52.191 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-21T07:32:52.191 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-21T07:32:52.194 INFO:tasks.ceph.osd.7:Started 2024-09-21T07:32:52.195 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:32:52.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.521+0000 7fdf12800640 1 -- 172.21.15.39:0/3621957491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf0c0b4570 msgr2=0x7fdf0c162be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:52.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.521+0000 7fdf12800640 1 --2- 172.21.15.39:0/3621957491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf0c0b4570 0x7fdf0c162be0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdf04005fd0 tx=0x7fdf0402a840 comp rx=0 tx=0).stop 2024-09-21T07:32:52.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.521+0000 7fdf12800640 1 -- 172.21.15.39:0/3621957491 shutdown_connections 2024-09-21T07:32:52.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.521+0000 7fdf12800640 1 --2- 172.21.15.39:0/3621957491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf0c0b4570 0x7fdf0c162be0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:52.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.521+0000 7fdf12800640 1 -- 172.21.15.39:0/3621957491 >> 172.21.15.39:0/3621957491 conn(0x7fdf0c0cdcc0 msgr2=0x7fdf0c0ce0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:52.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.521+0000 7fdf12800640 1 -- 172.21.15.39:0/3621957491 shutdown_connections 2024-09-21T07:32:52.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.521+0000 7fdf12800640 1 -- 172.21.15.39:0/3621957491 wait complete. 2024-09-21T07:32:52.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.523+0000 7fdf12800640 1 Processor -- start 2024-09-21T07:32:52.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.524+0000 7fdf12800640 1 -- start start 2024-09-21T07:32:52.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.525+0000 7fdf12800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf0c0b4570 0x7fdf0c0cb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:52.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.525+0000 7fdf12800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf0c0b2290 0x7fdf0c160350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:52.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.525+0000 7fdf12800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf0c1f8ba0 0x7fdf0c1f8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:52.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.525+0000 7fdf12800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c0b4280 con 0x7fdf0c0b2290 2024-09-21T07:32:52.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.525+0000 7fdf12800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c0b4100 con 0x7fdf0c0b4570 2024-09-21T07:32:52.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.525+0000 7fdf12800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fdf0c163120 con 0x7fdf0c1f8ba0 2024-09-21T07:32:52.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.525+0000 7fdf0b400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf0c0b4570 0x7fdf0c0cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:52.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.525+0000 7fdf0b400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf0c0b4570 0x7fdf0c0cb870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.39:35468/0 (socket says 172.21.15.39:35468) 2024-09-21T07:32:52.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.525+0000 7fdf0b400640 1 -- 172.21.15.39:0/1400396887 learned_addr learned my addr 172.21.15.39:0/1400396887 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:32:52.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.525+0000 7fdf0aa00640 1 --2- 172.21.15.39:0/1400396887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf0c0b2290 0x7fdf0c160350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:52.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.525+0000 7fdf0be00640 1 --2- 172.21.15.39:0/1400396887 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf0c1f8ba0 0x7fdf0c1f8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:52.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.526+0000 7fdf0b400640 1 -- 172.21.15.39:0/1400396887 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf0c1f8ba0 msgr2=0x7fdf0c1f8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:52.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.526+0000 7fdf0b400640 1 --2- 172.21.15.39:0/1400396887 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf0c1f8ba0 0x7fdf0c1f8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:52.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.526+0000 7fdf0b400640 1 -- 172.21.15.39:0/1400396887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf0c0b2290 msgr2=0x7fdf0c160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:52.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.526+0000 7fdf0b400640 1 --2- 172.21.15.39:0/1400396887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf0c0b2290 0x7fdf0c160350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:52.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.526+0000 7fdf0b400640 1 -- 172.21.15.39:0/1400396887 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf0c15f590 con 0x7fdf0c0b4570 2024-09-21T07:32:52.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.526+0000 7fdf0b400640 1 --2- 172.21.15.39:0/1400396887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf0c0b4570 0x7fdf0c0cb870 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdefc00c450 tx=0x7fdefc00c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:32:52.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.526+0000 7fdf03400640 1 -- 172.21.15.39:0/1400396887 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdefc017070 con 0x7fdf0c0b4570 2024-09-21T07:32:52.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.526+0000 7fdf03400640 1 -- 172.21.15.39:0/1400396887 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdefc0035f0 con 0x7fdf0c0b4570 2024-09-21T07:32:52.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.526+0000 7fdf03400640 1 -- 172.21.15.39:0/1400396887 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdefc003a40 con 0x7fdf0c0b4570 2024-09-21T07:32:52.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.527+0000 7fdf12800640 1 -- 172.21.15.39:0/1400396887 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf0c15f870 con 0x7fdf0c0b4570 2024-09-21T07:32:52.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.527+0000 7fdf12800640 1 -- 172.21.15.39:0/1400396887 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf0c15fe30 con 0x7fdf0c0b4570 2024-09-21T07:32:52.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.528+0000 7fdf03400640 1 -- 172.21.15.39:0/1400396887 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 2) ==== 50742+0+0 (secure 0 0 0) 0x7fdefc01a070 con 0x7fdf0c0b4570 2024-09-21T07:32:52.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.528+0000 7fdf12800640 1 -- 172.21.15.39:0/1400396887 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf0c162040 con 0x7fdf0c0b4570 2024-09-21T07:32:52.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.528+0000 7fdf03400640 1 -- 172.21.15.39:0/1400396887 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fdefc051770 con 0x7fdf0c0b4570 2024-09-21T07:32:52.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.530+0000 7fdf03400640 1 -- 172.21.15.39:0/1400396887 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mgrmap(e 3) ==== 50869+0+0 (secure 0 0 0) 0x7fdefc01b8d0 con 0x7fdf0c0b4570 2024-09-21T07:32:52.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.530+0000 7fdf03400640 1 --2- 172.21.15.39:0/1400396887 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdeec03faf0 0x7fdeec041f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:52.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.531+0000 7fdf0aa00640 1 --2- 172.21.15.39:0/1400396887 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdeec03faf0 0x7fdeec041f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:52.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.532+0000 7fdf0aa00640 1 --2- 172.21.15.39:0/1400396887 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdeec03faf0 0x7fdeec041f10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c1f6500 tx=0x7fdef8009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:32:52.535 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-21T07:32:52.534+0000 7f5ab2600640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-21T07:32:52.540 INFO:tasks.ceph.mgr.x.smithi148.stderr:2024-09-21T07:32:52.538+0000 7f5ab2600640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-21T07:32:52.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.539+0000 7fdf03400640 1 -- 172.21.15.39:0/1400396887 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdefc019de0 con 0x7fdf0c0b4570 2024-09-21T07:32:52.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.735+0000 7fdf12800640 1 -- 172.21.15.39:0/1400396887 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf0c0cea20 con 0x7fdf0c0b4570 2024-09-21T07:32:52.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.736+0000 7fdf03400640 1 -- 172.21.15.39:0/1400396887 <== mon.1 v2:172.21.15.148:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fdefc011de0 con 0x7fdf0c0b4570 2024-09-21T07:32:52.738 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:32:52.739 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":10,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:32:52.128828+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:32:52.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.739+0000 7fdf12800640 1 -- 172.21.15.39:0/1400396887 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdeec03faf0 msgr2=0x7fdeec041f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:52.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.739+0000 7fdf12800640 1 --2- 172.21.15.39:0/1400396887 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdeec03faf0 0x7fdeec041f10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c1f6500 tx=0x7fdef8009040 comp rx=0 tx=0).stop 2024-09-21T07:32:52.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.739+0000 7fdf12800640 1 -- 172.21.15.39:0/1400396887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf0c0b4570 msgr2=0x7fdf0c0cb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:52.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.739+0000 7fdf12800640 1 --2- 172.21.15.39:0/1400396887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf0c0b4570 0x7fdf0c0cb870 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdefc00c450 tx=0x7fdefc00c910 comp rx=0 tx=0).stop 2024-09-21T07:32:52.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.739+0000 7fdf12800640 1 -- 172.21.15.39:0/1400396887 shutdown_connections 2024-09-21T07:32:52.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.739+0000 7fdf12800640 1 --2- 172.21.15.39:0/1400396887 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdeec03faf0 0x7fdeec041f10 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:52.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.739+0000 7fdf12800640 1 --2- 172.21.15.39:0/1400396887 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf0c1f8ba0 0x7fdf0c1f8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:52.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.739+0000 7fdf12800640 1 --2- 172.21.15.39:0/1400396887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf0c0b2290 0x7fdf0c160350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:52.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.739+0000 7fdf12800640 1 --2- 172.21.15.39:0/1400396887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf0c0b4570 0x7fdf0c0cb870 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:52.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.739+0000 7fdf12800640 1 -- 172.21.15.39:0/1400396887 >> 172.21.15.39:0/1400396887 conn(0x7fdf0c0cdcc0 msgr2=0x7fdf0c0d0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:52.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.739+0000 7fdf12800640 1 -- 172.21.15.39:0/1400396887 shutdown_connections 2024-09-21T07:32:52.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:32:52.739+0000 7fdf12800640 1 -- 172.21.15.39:0/1400396887 wait complete. 2024-09-21T07:32:52.762 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-21T07:32:52.762 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-21T07:32:53.568 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T07:32:53.774 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:53.772+0000 7f94db000640 1 -- 172.21.15.39:0/1806753339 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94d40ccee0 msgr2=0x7f94d40d20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:53.774 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:53.772+0000 7f94db000640 1 --2- 172.21.15.39:0/1806753339 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94d40ccee0 0x7f94d40d20a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f94c0005fd0 tx=0x7f94c002a5d0 comp rx=0 tx=0).stop 2024-09-21T07:32:53.775 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:53.772+0000 7f94db000640 1 -- 172.21.15.39:0/1806753339 shutdown_connections 2024-09-21T07:32:53.775 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:53.772+0000 7f94db000640 1 --2- 172.21.15.39:0/1806753339 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94d40ccee0 0x7f94d40d20a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:53.775 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:53.772+0000 7f94db000640 1 -- 172.21.15.39:0/1806753339 >> 172.21.15.39:0/1806753339 conn(0x7f94d406d200 msgr2=0x7f94d406d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:53.775 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:53.772+0000 7f94db000640 1 -- 172.21.15.39:0/1806753339 shutdown_connections 2024-09-21T07:32:53.775 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:53.772+0000 7f94db000640 1 -- 172.21.15.39:0/1806753339 wait complete. 2024-09-21T07:32:53.776 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:53.775+0000 7f94db000640 1 Processor -- start 2024-09-21T07:32:53.776 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:53.775+0000 7f94db000640 1 -- start start 2024-09-21T07:32:53.780 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:53.776+0000 7f94db000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94d40d25e0 0x7f94d40cb360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:53.780 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:53.776+0000 7f94db000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94d4153670 0x7f94d40c4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:53.780 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:53.776+0000 7f94db000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f94d40cb8a0 0x7f94d40c4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:53.780 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:53.776+0000 7f94db000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f94d40d0d50 con 0x7f94d4153670 2024-09-21T07:32:53.780 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:53.776+0000 7f94db000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f94d40d0bd0 con 0x7f94d40d25e0 2024-09-21T07:32:53.780 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:53.776+0000 7f94db000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f94d40d0ed0 con 0x7f94d40cb8a0 2024-09-21T07:32:53.780 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:53.776+0000 7f94d9200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94d4153670 0x7f94d40c4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:53.780 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:53.776+0000 7f94d9200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94d4153670 0x7f94d40c4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36038/0 (socket says 172.21.15.39:36038) 2024-09-21T07:32:53.780 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:53.776+0000 7f94d9200640 1 -- 172.21.15.39:0/805031059 learned_addr learned my addr 172.21.15.39:0/805031059 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:32:53.780 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:53.776+0000 7f94d9c00640 1 --2- 172.21.15.39:0/805031059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94d40d25e0 0x7f94d40cb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:53.780 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:53.777+0000 7f94da600640 1 --2- 172.21.15.39:0/805031059 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f94d40cb8a0 0x7f94d40c4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:53.780 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:53.777+0000 7f94d9200640 1 -- 172.21.15.39:0/805031059 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f94d40cb8a0 msgr2=0x7f94d40c4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:53.781 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:53.777+0000 7f94d9200640 1 --2- 172.21.15.39:0/805031059 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f94d40cb8a0 0x7f94d40c4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:53.781 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:53.777+0000 7f94d9200640 1 -- 172.21.15.39:0/805031059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94d40d25e0 msgr2=0x7f94d40cb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:53.781 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:53.777+0000 7f94d9200640 1 --2- 172.21.15.39:0/805031059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94d40d25e0 0x7f94d40cb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:53.781 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:53.777+0000 7f94d9200640 1 -- 172.21.15.39:0/805031059 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94d41f46e0 con 0x7f94d4153670 2024-09-21T07:32:53.781 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:53.777+0000 7f94d9200640 1 --2- 172.21.15.39:0/805031059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94d4153670 0x7f94d40c4940 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f94c0006010 tx=0x7f94c00038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:32:53.781 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:53.777+0000 7f94d1600640 1 -- 172.21.15.39:0/805031059 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c003e030 con 0x7f94d4153670 2024-09-21T07:32:53.781 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:53.777+0000 7f94d1600640 1 -- 172.21.15.39:0/805031059 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f94c0003ca0 con 0x7f94d4153670 2024-09-21T07:32:53.781 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:53.778+0000 7f94d1600640 1 -- 172.21.15.39:0/805031059 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c0035040 con 0x7f94d4153670 2024-09-21T07:32:53.781 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:53.778+0000 7f94db000640 1 -- 172.21.15.39:0/805031059 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94d41f49c0 con 0x7f94d4153670 2024-09-21T07:32:53.781 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:53.778+0000 7f94db000640 1 -- 172.21.15.39:0/805031059 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94d40b1070 con 0x7f94d4153670 2024-09-21T07:32:53.782 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:53.779+0000 7f94db000640 1 -- 172.21.15.39:0/805031059 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94d40d20a0 con 0x7f94d4153670 2024-09-21T07:32:53.782 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:53.780+0000 7f94d1600640 1 -- 172.21.15.39:0/805031059 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 4) ==== 101385+0+0 (secure 0 0 0) 0x7f94c0041030 con 0x7f94d4153670 2024-09-21T07:32:53.782 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:53.781+0000 7f94d1600640 1 --2- 172.21.15.39:0/805031059 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f94b4078b00 0x7f94b407b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:32:53.783 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:53.781+0000 7f94d1600640 1 -- 172.21.15.39:0/805031059 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f94c00ba230 con 0x7f94d4153670 2024-09-21T07:32:53.783 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:53.781+0000 7f94d9c00640 1 --2- 172.21.15.39:0/805031059 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f94b4078b00 0x7f94b407b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:32:53.784 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:53.782+0000 7f94d9c00640 1 --2- 172.21.15.39:0/805031059 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f94b4078b00 0x7f94b407b010 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f94c8001720 tx=0x7f94c8009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:32:53.787 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:53.786+0000 7f94d1600640 1 -- 172.21.15.39:0/805031059 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f94c00830e0 con 0x7f94d4153670 2024-09-21T07:32:53.798 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-21T07:32:53.796+0000 7f8446d41640 -1 Falling back to public interface 2024-09-21T07:32:53.810 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-21T07:32:53.809+0000 7ffadd20f640 -1 Falling back to public interface 2024-09-21T07:32:53.836 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-21T07:32:53.834+0000 7fe534018640 -1 Falling back to public interface 2024-09-21T07:32:53.851 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-21T07:32:53.849+0000 7fc50f014640 -1 Falling back to public interface 2024-09-21T07:32:53.860 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-21T07:32:53.858+0000 7f586780d640 -1 Falling back to public interface 2024-09-21T07:32:53.873 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-21T07:32:53.871+0000 7fece3e18640 -1 Falling back to public interface 2024-09-21T07:32:54.005 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:54.003+0000 7f94db000640 1 -- 172.21.15.39:0/805031059 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f94d40b4990 con 0x7f94d4153670 2024-09-21T07:32:54.006 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:54.004+0000 7f94d1600640 1 -- 172.21.15.39:0/805031059 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f94c0087b10 con 0x7f94d4153670 2024-09-21T07:32:54.006 INFO:teuthology.misc.health.smithi039.stdout: 2024-09-21T07:32:54.006 INFO:teuthology.misc.health.smithi039.stdout:{"epoch":10,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:32:52.128828+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:32:54.009 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:54.007+0000 7f94db000640 1 -- 172.21.15.39:0/805031059 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f94b4078b00 msgr2=0x7f94b407b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:54.009 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:54.007+0000 7f94db000640 1 --2- 172.21.15.39:0/805031059 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f94b4078b00 0x7f94b407b010 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f94c8001720 tx=0x7f94c8009040 comp rx=0 tx=0).stop 2024-09-21T07:32:54.009 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:54.007+0000 7f94db000640 1 -- 172.21.15.39:0/805031059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94d4153670 msgr2=0x7f94d40c4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:32:54.009 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:54.007+0000 7f94db000640 1 --2- 172.21.15.39:0/805031059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94d4153670 0x7f94d40c4940 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f94c0006010 tx=0x7f94c00038a0 comp rx=0 tx=0).stop 2024-09-21T07:32:54.009 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:54.007+0000 7f94db000640 1 -- 172.21.15.39:0/805031059 shutdown_connections 2024-09-21T07:32:54.009 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:54.007+0000 7f94db000640 1 --2- 172.21.15.39:0/805031059 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f94b4078b00 0x7f94b407b010 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:54.010 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:54.007+0000 7f94db000640 1 --2- 172.21.15.39:0/805031059 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f94d40cb8a0 0x7f94d40c4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:54.010 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:54.007+0000 7f94db000640 1 --2- 172.21.15.39:0/805031059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94d4153670 0x7f94d40c4940 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:54.010 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:54.007+0000 7f94db000640 1 --2- 172.21.15.39:0/805031059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94d40d25e0 0x7f94d40cb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:32:54.010 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:54.007+0000 7f94db000640 1 -- 172.21.15.39:0/805031059 >> 172.21.15.39:0/805031059 conn(0x7f94d406d200 msgr2=0x7f94d40c5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:32:54.010 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:54.007+0000 7f94db000640 1 -- 172.21.15.39:0/805031059 shutdown_connections 2024-09-21T07:32:54.010 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:32:54.007+0000 7f94db000640 1 -- 172.21.15.39:0/805031059 wait complete. 2024-09-21T07:32:54.010 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-21T07:32:54.009+0000 7fbcb50a5640 -1 Falling back to public interface 2024-09-21T07:32:54.030 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-21T07:32:54.062 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-21T07:32:54.061+0000 7f7e0f20f640 -1 Falling back to public interface 2024-09-21T07:32:57.467 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-21T07:32:57.465+0000 7f8446d41640 -1 osd.3 0 log_to_monitors true 2024-09-21T07:32:57.510 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-21T07:32:57.509+0000 7ffadd20f640 -1 osd.4 0 log_to_monitors true 2024-09-21T07:32:57.862 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-21T07:32:57.860+0000 7fece3e18640 -1 osd.1 0 log_to_monitors true 2024-09-21T07:32:58.019 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-21T07:32:58.017+0000 7fc50f014640 -1 osd.0 0 log_to_monitors true 2024-09-21T07:32:58.069 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-21T07:32:58.068+0000 7fe534018640 -1 osd.6 0 log_to_monitors true 2024-09-21T07:32:58.110 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-21T07:32:58.108+0000 7f586780d640 -1 osd.2 0 log_to_monitors true 2024-09-21T07:32:58.208 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-21T07:32:58.207+0000 7fbcb50a5640 -1 osd.7 0 log_to_monitors true 2024-09-21T07:32:58.479 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-21T07:32:58.478+0000 7f7e0f20f640 -1 osd.5 0 log_to_monitors true 2024-09-21T07:32:59.603 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:32:59.607 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:32:59.780 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-21T07:32:59.779+0000 7ffad8400640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T07:32:59.882 INFO:tasks.ceph.osd.4.smithi148.stderr:2024-09-21T07:32:59.881+0000 7ffacda00640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T07:32:59.933 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-21T07:32:59.931+0000 7f8441c00640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T07:33:00.046 INFO:tasks.ceph.osd.3.smithi039.stderr:2024-09-21T07:33:00.042+0000 7f8437c00640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T07:33:00.837 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T07:33:01.181 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.179+0000 7f8d63e00640 1 -- 172.21.15.39:0/977142360 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d6415c6f0 msgr2=0x7f8d640d0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:01.182 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.179+0000 7f8d63e00640 1 --2- 172.21.15.39:0/977142360 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d6415c6f0 0x7f8d640d0650 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8d640c7f00 tx=0x7f8d5402a4b0 comp rx=0 tx=0).stop 2024-09-21T07:33:01.182 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.179+0000 7f8d63e00640 1 -- 172.21.15.39:0/977142360 shutdown_connections 2024-09-21T07:33:01.182 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.179+0000 7f8d63e00640 1 --2- 172.21.15.39:0/977142360 >> v2:172.21.15.39:3300/0 conn(0x7f8d640c7430 0x7f8d640c7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:01.182 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.179+0000 7f8d63e00640 1 --2- 172.21.15.39:0/977142360 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d6415c6f0 0x7f8d640d0650 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:01.182 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.180+0000 7f8d63e00640 1 -- 172.21.15.39:0/977142360 >> 172.21.15.39:0/977142360 conn(0x7f8d640b3dc0 msgr2=0x7f8d640b41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:01.182 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.180+0000 7f8d63e00640 1 -- 172.21.15.39:0/977142360 shutdown_connections 2024-09-21T07:33:01.183 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.180+0000 7f8d63e00640 1 -- 172.21.15.39:0/977142360 wait complete. 2024-09-21T07:33:01.185 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.183+0000 7f8d63e00640 1 Processor -- start 2024-09-21T07:33:01.185 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.184+0000 7f8d63e00640 1 -- start start 2024-09-21T07:33:01.188 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.186+0000 7f8d63e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d640c7430 0x7f8d64148b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:01.188 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.187+0000 7f8d62a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d640c7430 0x7f8d64148b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:01.189 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.187+0000 7f8d62a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d640c7430 0x7f8d64148b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36100/0 (socket says 172.21.15.39:36100) 2024-09-21T07:33:01.189 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.187+0000 7f8d63e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d640cdeb0 0x7f8d64156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:01.189 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.187+0000 7f8d63e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d6415c6f0 0x7f8d64157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:01.189 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.187+0000 7f8d62a00640 1 -- 172.21.15.39:0/2497180084 learned_addr learned my addr 172.21.15.39:0/2497180084 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:01.190 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.187+0000 7f8d63e00640 1 -- 172.21.15.39:0/2497180084 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8d640cec60 con 0x7f8d640c7430 2024-09-21T07:33:01.190 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.187+0000 7f8d63e00640 1 -- 172.21.15.39:0/2497180084 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8d640ceae0 con 0x7f8d6415c6f0 2024-09-21T07:33:01.190 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.187+0000 7f8d63e00640 1 -- 172.21.15.39:0/2497180084 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f8d640cede0 con 0x7f8d640cdeb0 2024-09-21T07:33:01.190 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.187+0000 7f8d62000640 1 --2- 172.21.15.39:0/2497180084 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d640cdeb0 0x7f8d64156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:01.190 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.187+0000 7f8d63400640 1 --2- 172.21.15.39:0/2497180084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d6415c6f0 0x7f8d64157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:01.191 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.188+0000 7f8d62a00640 1 -- 172.21.15.39:0/2497180084 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d640cdeb0 msgr2=0x7f8d64156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:01.191 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.188+0000 7f8d62a00640 1 --2- 172.21.15.39:0/2497180084 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d640cdeb0 0x7f8d64156b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:01.191 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.188+0000 7f8d62a00640 1 -- 172.21.15.39:0/2497180084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d6415c6f0 msgr2=0x7f8d64157090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:01.191 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.188+0000 7f8d62a00640 1 --2- 172.21.15.39:0/2497180084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d6415c6f0 0x7f8d64157090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:01.191 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.188+0000 7f8d62a00640 1 -- 172.21.15.39:0/2497180084 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d640b6780 con 0x7f8d640c7430 2024-09-21T07:33:01.191 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.189+0000 7f8d62a00640 1 --2- 172.21.15.39:0/2497180084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d640c7430 0x7f8d64148b90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8d54000c00 tx=0x7f8d54003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:01.191 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.190+0000 7f8d5aa00640 1 -- 172.21.15.39:0/2497180084 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5403e030 con 0x7f8d640c7430 2024-09-21T07:33:01.192 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.190+0000 7f8d5aa00640 1 -- 172.21.15.39:0/2497180084 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d54003c20 con 0x7f8d640c7430 2024-09-21T07:33:01.192 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.190+0000 7f8d5aa00640 1 -- 172.21.15.39:0/2497180084 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d54035040 con 0x7f8d640c7430 2024-09-21T07:33:01.192 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.190+0000 7f8d63e00640 1 -- 172.21.15.39:0/2497180084 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d640b6a60 con 0x7f8d640c7430 2024-09-21T07:33:01.192 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.190+0000 7f8d63e00640 1 -- 172.21.15.39:0/2497180084 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d640b7020 con 0x7f8d640c7430 2024-09-21T07:33:01.194 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.192+0000 7f8d63e00640 1 -- 172.21.15.39:0/2497180084 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d640cf2c0 con 0x7f8d640c7430 2024-09-21T07:33:01.196 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.194+0000 7f8d5aa00640 1 -- 172.21.15.39:0/2497180084 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 4) ==== 101385+0+0 (secure 0 0 0) 0x7f8d54041030 con 0x7f8d640c7430 2024-09-21T07:33:01.196 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.195+0000 7f8d5aa00640 1 --2- 172.21.15.39:0/2497180084 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8d44078b80 0x7f8d4407b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:01.197 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.195+0000 7f8d5aa00640 1 -- 172.21.15.39:0/2497180084 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2791+0+0 (secure 0 0 0) 0x7f8d540ba920 con 0x7f8d640c7430 2024-09-21T07:33:01.198 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.196+0000 7f8d62000640 1 --2- 172.21.15.39:0/2497180084 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8d44078b80 0x7f8d4407b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:01.199 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.197+0000 7f8d62000640 1 --2- 172.21.15.39:0/2497180084 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8d44078b80 0x7f8d4407b090 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8d641f6990 tx=0x7f8d50007e30 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:01.206 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.204+0000 7f8d5aa00640 1 -- 172.21.15.39:0/2497180084 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8d54083450 con 0x7f8d640c7430 2024-09-21T07:33:01.436 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-21T07:33:01.434+0000 7fe52e800640 -1 osd.6 0 waiting for initial osdmap 2024-09-21T07:33:01.443 INFO:tasks.ceph.osd.6.smithi148.stderr:2024-09-21T07:33:01.441+0000 7fe524800640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T07:33:01.446 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-21T07:33:01.445+0000 7fbcb0a00640 -1 osd.7 0 waiting for initial osdmap 2024-09-21T07:33:01.452 INFO:tasks.ceph.osd.7.smithi148.stderr:2024-09-21T07:33:01.451+0000 7fbca6000640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T07:33:01.492 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-21T07:33:01.490+0000 7f7e0a600640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T07:33:01.497 INFO:tasks.ceph.osd.5.smithi148.stderr:2024-09-21T07:33:01.496+0000 7f7dffc00640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T07:33:01.595 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.593+0000 7f8d63e00640 1 -- 172.21.15.39:0/2497180084 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8d6415ad60 con 0x7f8d640c7430 2024-09-21T07:33:01.596 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.594+0000 7f8d5aa00640 1 -- 172.21.15.39:0/2497180084 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+8041 (secure 0 0 0) 0x7f8d54087e80 con 0x7f8d640c7430 2024-09-21T07:33:01.596 INFO:teuthology.misc.health.smithi039.stdout: 2024-09-21T07:33:01.596 INFO:teuthology.misc.health.smithi039.stdout:{"epoch":14,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:33:00.903914+0000","last_up_change":"2024-09-21T07:33:00.903914+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:33:01.599 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.597+0000 7f8d63e00640 1 -- 172.21.15.39:0/2497180084 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8d44078b80 msgr2=0x7f8d4407b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:01.600 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.597+0000 7f8d63e00640 1 --2- 172.21.15.39:0/2497180084 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8d44078b80 0x7f8d4407b090 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8d641f6990 tx=0x7f8d50007e30 comp rx=0 tx=0).stop 2024-09-21T07:33:01.600 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.597+0000 7f8d63e00640 1 -- 172.21.15.39:0/2497180084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d640c7430 msgr2=0x7f8d64148b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:01.600 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.597+0000 7f8d63e00640 1 --2- 172.21.15.39:0/2497180084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d640c7430 0x7f8d64148b90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8d54000c00 tx=0x7f8d54003820 comp rx=0 tx=0).stop 2024-09-21T07:33:01.600 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.598+0000 7f8d63e00640 1 -- 172.21.15.39:0/2497180084 shutdown_connections 2024-09-21T07:33:01.600 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.598+0000 7f8d63e00640 1 --2- 172.21.15.39:0/2497180084 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8d44078b80 0x7f8d4407b090 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:01.600 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.598+0000 7f8d63e00640 1 --2- 172.21.15.39:0/2497180084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d6415c6f0 0x7f8d64157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:01.600 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.598+0000 7f8d63e00640 1 --2- 172.21.15.39:0/2497180084 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d640cdeb0 0x7f8d64156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:01.600 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.598+0000 7f8d63e00640 1 --2- 172.21.15.39:0/2497180084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d640c7430 0x7f8d64148b90 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:01.600 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.598+0000 7f8d63e00640 1 -- 172.21.15.39:0/2497180084 >> 172.21.15.39:0/2497180084 conn(0x7f8d640b3dc0 msgr2=0x7f8d640b3650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:01.601 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.598+0000 7f8d63e00640 1 -- 172.21.15.39:0/2497180084 shutdown_connections 2024-09-21T07:33:01.601 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:01.598+0000 7f8d63e00640 1 -- 172.21.15.39:0/2497180084 wait complete. 2024-09-21T07:33:01.637 DEBUG:teuthology.misc:2 of 8 OSDs are up 2024-09-21T07:33:01.727 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-21T07:33:01.725+0000 7fecde800640 -1 osd.1 0 waiting for initial osdmap 2024-09-21T07:33:01.736 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-21T07:33:01.734+0000 7fecd4800640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T07:33:01.753 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-21T07:33:01.751+0000 7f5862000640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T07:33:01.764 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-21T07:33:01.762+0000 7f5858000640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T07:33:01.782 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-21T07:33:01.780+0000 7fc509800640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T07:33:01.792 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-21T07:33:01.790+0000 7fc4ff800640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T07:33:08.443 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T07:33:08.663 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.661+0000 7fd141e00640 1 -- 172.21.15.39:0/3611789269 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd120005150 msgr2=0x7fd12000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:08.663 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.661+0000 7fd141e00640 1 --2- 172.21.15.39:0/3611789269 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd120005150 0x7fd12000b9c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd13400e3d0 tx=0x7fd120005830 comp rx=0 tx=0).stop 2024-09-21T07:33:08.663 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.661+0000 7fd13b400640 1 -- 172.21.15.39:0/3611789269 reap_dead start 2024-09-21T07:33:08.663 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.661+0000 7fd141e00640 1 -- 172.21.15.39:0/3611789269 shutdown_connections 2024-09-21T07:33:08.663 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.661+0000 7fd141e00640 1 -- 172.21.15.39:0/3611789269 >> 172.21.15.39:0/3611789269 conn(0x7fd13c0d0a30 msgr2=0x7fd13c0b1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:08.663 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.661+0000 7fd141e00640 1 -- 172.21.15.39:0/3611789269 shutdown_connections 2024-09-21T07:33:08.663 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.662+0000 7fd141e00640 1 -- 172.21.15.39:0/3611789269 wait complete. 2024-09-21T07:33:08.664 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.664+0000 7fd141e00640 1 Processor -- start 2024-09-21T07:33:08.665 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.664+0000 7fd141e00640 1 -- start start 2024-09-21T07:33:08.667 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.665+0000 7fd141e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd120005150 0x7fd13c154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:08.667 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.665+0000 7fd141e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd13c0998e0 0x7fd13c159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:08.667 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.665+0000 7fd141e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd13c158cb0 0x7fd13c1590c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:08.667 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.665+0000 7fd141e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c0cf4f0 con 0x7fd13c158cb0 2024-09-21T07:33:08.667 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.665+0000 7fd141e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c071330 con 0x7fd120005150 2024-09-21T07:33:08.667 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.665+0000 7fd141e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd13c0cf670 con 0x7fd13c0998e0 2024-09-21T07:33:08.667 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.665+0000 7fd13b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd13c158cb0 0x7fd13c1590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:08.668 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.666+0000 7fd13b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd13c158cb0 0x7fd13c1590c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47144/0 (socket says 172.21.15.39:47144) 2024-09-21T07:33:08.668 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.666+0000 7fd13b400640 1 -- 172.21.15.39:0/555051423 learned_addr learned my addr 172.21.15.39:0/555051423 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:08.668 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.666+0000 7fd13aa00640 1 --2- 172.21.15.39:0/555051423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd120005150 0x7fd13c154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:08.668 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.666+0000 7fd13a000640 1 --2- 172.21.15.39:0/555051423 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd13c0998e0 0x7fd13c159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:08.668 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.666+0000 7fd13b400640 1 -- 172.21.15.39:0/555051423 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd13c0998e0 msgr2=0x7fd13c159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:08.668 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.666+0000 7fd13b400640 1 --2- 172.21.15.39:0/555051423 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd13c0998e0 0x7fd13c159c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:08.668 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.666+0000 7fd13b400640 1 -- 172.21.15.39:0/555051423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd120005150 msgr2=0x7fd13c154a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:08.668 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.666+0000 7fd13b400640 1 --2- 172.21.15.39:0/555051423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd120005150 0x7fd13c154a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:08.668 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.666+0000 7fd13b400640 1 -- 172.21.15.39:0/555051423 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd13c099390 con 0x7fd13c158cb0 2024-09-21T07:33:08.669 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.666+0000 7fd13b400640 1 --2- 172.21.15.39:0/555051423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd13c158cb0 0x7fd13c1590c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd134002230 tx=0x7fd134005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:08.669 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.667+0000 7fd133400640 1 -- 172.21.15.39:0/555051423 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c009880 con 0x7fd13c158cb0 2024-09-21T07:33:08.669 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.667+0000 7fd133400640 1 -- 172.21.15.39:0/555051423 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd134032400 con 0x7fd13c158cb0 2024-09-21T07:33:08.669 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.667+0000 7fd141e00640 1 -- 172.21.15.39:0/555051423 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd13c0adad0 con 0x7fd13c158cb0 2024-09-21T07:33:08.669 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.667+0000 7fd133400640 1 -- 172.21.15.39:0/555051423 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd134004d80 con 0x7fd13c158cb0 2024-09-21T07:33:08.670 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.667+0000 7fd141e00640 1 -- 172.21.15.39:0/555051423 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd13c0ad5b0 con 0x7fd13c158cb0 2024-09-21T07:33:08.670 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.668+0000 7fd141e00640 1 -- 172.21.15.39:0/555051423 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd13c0ca940 con 0x7fd13c158cb0 2024-09-21T07:33:08.670 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.669+0000 7fd133400640 1 -- 172.21.15.39:0/555051423 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd134046020 con 0x7fd13c158cb0 2024-09-21T07:33:08.670 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.670+0000 7fd133400640 1 --2- 172.21.15.39:0/555051423 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd11c078b90 0x7fd11c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:08.671 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.670+0000 7fd133400640 1 -- 172.21.15.39:0/555051423 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fd1340bbc50 con 0x7fd13c158cb0 2024-09-21T07:33:08.671 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.670+0000 7fd13aa00640 1 --2- 172.21.15.39:0/555051423 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd11c078b90 0x7fd11c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:08.672 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.671+0000 7fd13aa00640 1 --2- 172.21.15.39:0/555051423 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd11c078b90 0x7fd11c07b0a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd12c0096f0 tx=0x7fd12c005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:08.676 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.675+0000 7fd133400640 1 -- 172.21.15.39:0/555051423 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd134083d80 con 0x7fd13c158cb0 2024-09-21T07:33:08.876 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.875+0000 7fd141e00640 1 -- 172.21.15.39:0/555051423 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd13c1537c0 con 0x7fd13c158cb0 2024-09-21T07:33:08.877 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.876+0000 7fd133400640 1 -- 172.21.15.39:0/555051423 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12896 (secure 0 0 0) 0x7fd1340887b0 con 0x7fd13c158cb0 2024-09-21T07:33:08.877 INFO:teuthology.misc.health.smithi039.stdout: 2024-09-21T07:33:08.877 INFO:teuthology.misc.health.smithi039.stdout:{"epoch":17,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:33:04.446956+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:33:08.880 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.878+0000 7fd141e00640 1 -- 172.21.15.39:0/555051423 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd11c078b90 msgr2=0x7fd11c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:08.880 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.878+0000 7fd141e00640 1 --2- 172.21.15.39:0/555051423 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd11c078b90 0x7fd11c07b0a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd12c0096f0 tx=0x7fd12c005f90 comp rx=0 tx=0).stop 2024-09-21T07:33:08.880 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.878+0000 7fd141e00640 1 -- 172.21.15.39:0/555051423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd13c158cb0 msgr2=0x7fd13c1590c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:08.880 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.878+0000 7fd141e00640 1 --2- 172.21.15.39:0/555051423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd13c158cb0 0x7fd13c1590c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd134002230 tx=0x7fd134005c70 comp rx=0 tx=0).stop 2024-09-21T07:33:08.880 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.879+0000 7fd141e00640 1 -- 172.21.15.39:0/555051423 shutdown_connections 2024-09-21T07:33:08.880 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.879+0000 7fd141e00640 1 --2- 172.21.15.39:0/555051423 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd11c078b90 0x7fd11c07b0a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:08.880 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.879+0000 7fd141e00640 1 --2- 172.21.15.39:0/555051423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd13c158cb0 0x7fd13c1590c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:08.880 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.879+0000 7fd141e00640 1 --2- 172.21.15.39:0/555051423 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd13c0998e0 0x7fd13c159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:08.880 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.879+0000 7fd141e00640 1 --2- 172.21.15.39:0/555051423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd120005150 0x7fd13c154a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:08.880 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.879+0000 7fd141e00640 1 -- 172.21.15.39:0/555051423 >> 172.21.15.39:0/555051423 conn(0x7fd13c0d0a30 msgr2=0x7fd13c0af6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:08.880 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.879+0000 7fd141e00640 1 -- 172.21.15.39:0/555051423 shutdown_connections 2024-09-21T07:33:08.881 INFO:teuthology.misc.health.smithi039.stderr:2024-09-21T07:33:08.879+0000 7fd141e00640 1 -- 172.21.15.39:0/555051423 wait complete. 2024-09-21T07:33:08.898 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-21T07:33:08.898 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-21T07:33:08.898 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T07:33:08.898 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T07:33:08.902 INFO:tasks.ceph.mds.a:Started 2024-09-21T07:33:08.902 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T07:33:08.903 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T07:33:08.906 INFO:tasks.ceph.mds.c:Started 2024-09-21T07:33:08.906 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T07:33:08.906 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T07:33:08.911 INFO:tasks.ceph.mds.b:Started 2024-09-21T07:33:08.911 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T07:33:08.911 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T07:33:08.915 INFO:tasks.ceph.mds.d:Started 2024-09-21T07:33:08.915 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-21T07:33:08.915 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:33:09.100 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-21T07:33:09.104 INFO:tasks.ceph.mds.c.smithi039.stdout:starting mds.c at 2024-09-21T07:33:09.118 INFO:tasks.ceph.mds.d.smithi148.stdout:starting mds.d at 2024-09-21T07:33:09.118 INFO:tasks.ceph.mds.b.smithi148.stdout:starting mds.b at 2024-09-21T07:33:09.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.151+0000 7f0562c00640 1 -- 172.21.15.39:0/656821041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f055c0d0e50 msgr2=0x7f055c0d1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:09.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.151+0000 7f0562c00640 1 --2- 172.21.15.39:0/656821041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f055c0d0e50 0x7f055c0d1260 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0554005fd0 tx=0x7f055402a980 comp rx=0 tx=0).stop 2024-09-21T07:33:09.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.151+0000 7f0562c00640 1 -- 172.21.15.39:0/656821041 shutdown_connections 2024-09-21T07:33:09.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.151+0000 7f0562c00640 1 --2- 172.21.15.39:0/656821041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f055c0d0e50 0x7f055c0d1260 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:09.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.151+0000 7f0562c00640 1 --2- 172.21.15.39:0/656821041 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f055c06e0a0 0x7f055c0b97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:09.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.151+0000 7f0562c00640 1 -- 172.21.15.39:0/656821041 >> 172.21.15.39:0/656821041 conn(0x7f055c09d840 msgr2=0x7f055c08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:09.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.151+0000 7f0562c00640 1 -- 172.21.15.39:0/656821041 shutdown_connections 2024-09-21T07:33:09.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.151+0000 7f0562c00640 1 -- 172.21.15.39:0/656821041 wait complete. 2024-09-21T07:33:09.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.154+0000 7f0562c00640 1 Processor -- start 2024-09-21T07:33:09.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.154+0000 7f0562c00640 1 -- start start 2024-09-21T07:33:09.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.155+0000 7f0562c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f055c06e0a0 0x7f055c1f95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:09.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.155+0000 7f0562c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f055c06f6e0 0x7f055c1eac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:09.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.155+0000 7f0562c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f055c0d0e50 0x7f055c0ca330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:09.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.155+0000 7f0562c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f055c0cfea0 con 0x7f055c06e0a0 2024-09-21T07:33:09.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.155+0000 7f0562c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f055c0cfd20 con 0x7f055c0d0e50 2024-09-21T07:33:09.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.155+0000 7f0562c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f055c0d0020 con 0x7f055c06f6e0 2024-09-21T07:33:09.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.155+0000 7f055be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f055c06e0a0 0x7f055c1f95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:09.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.155+0000 7f055be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f055c06e0a0 0x7f055c1f95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47182/0 (socket says 172.21.15.39:47182) 2024-09-21T07:33:09.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.155+0000 7f055be00640 1 -- 172.21.15.39:0/91195392 learned_addr learned my addr 172.21.15.39:0/91195392 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:09.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.156+0000 7f0553e00640 1 --2- 172.21.15.39:0/91195392 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f055c06f6e0 0x7f055c1eac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:09.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.156+0000 7f055be00640 1 -- 172.21.15.39:0/91195392 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f055c06f6e0 msgr2=0x7f055c1eac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:09.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.156+0000 7f055be00640 1 --2- 172.21.15.39:0/91195392 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f055c06f6e0 0x7f055c1eac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:09.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.156+0000 7f055be00640 1 -- 172.21.15.39:0/91195392 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f055c0d0e50 msgr2=0x7f055c0ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:09.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.156+0000 7f055be00640 1 --2- 172.21.15.39:0/91195392 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f055c0d0e50 0x7f055c0ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:09.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.156+0000 7f055be00640 1 -- 172.21.15.39:0/91195392 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055c1f57e0 con 0x7f055c06e0a0 2024-09-21T07:33:09.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.156+0000 7f055be00640 1 --2- 172.21.15.39:0/91195392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f055c06e0a0 0x7f055c1f95f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f054c00e9f0 tx=0x7f054c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:09.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.156+0000 7f0558c00640 1 -- 172.21.15.39:0/91195392 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c00ccc0 con 0x7f055c06e0a0 2024-09-21T07:33:09.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.156+0000 7f0558c00640 1 -- 172.21.15.39:0/91195392 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f054c010430 con 0x7f055c06e0a0 2024-09-21T07:33:09.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.156+0000 7f0558c00640 1 -- 172.21.15.39:0/91195392 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c010880 con 0x7f055c06e0a0 2024-09-21T07:33:09.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.156+0000 7f0562c00640 1 -- 172.21.15.39:0/91195392 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f055c1f5ac0 con 0x7f055c06e0a0 2024-09-21T07:33:09.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.157+0000 7f0562c00640 1 -- 172.21.15.39:0/91195392 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f055c1f6080 con 0x7f055c06e0a0 2024-09-21T07:33:09.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.158+0000 7f0562c00640 1 -- 172.21.15.39:0/91195392 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f055c0d0490 con 0x7f055c06e0a0 2024-09-21T07:33:09.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.159+0000 7f0558c00640 1 -- 172.21.15.39:0/91195392 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f054c010a20 con 0x7f055c06e0a0 2024-09-21T07:33:09.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.160+0000 7f0558c00640 1 --2- 172.21.15.39:0/91195392 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0538078bc0 0x7f053807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:09.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.160+0000 7f0558c00640 1 -- 172.21.15.39:0/91195392 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f054c09b570 con 0x7f055c06e0a0 2024-09-21T07:33:09.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.160+0000 7f0553e00640 1 --2- 172.21.15.39:0/91195392 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0538078bc0 0x7f053807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:09.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.161+0000 7f0553e00640 1 --2- 172.21.15.39:0/91195392 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0538078bc0 0x7f053807b0d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f055c0cea50 tx=0x7f0544009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:09.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.164+0000 7f0558c00640 1 -- 172.21.15.39:0/91195392 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f054c063d00 con 0x7f055c06e0a0 2024-09-21T07:33:09.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.399+0000 7f0562c00640 1 -- 172.21.15.39:0/91195392 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f055c1eb550 con 0x7f055c06e0a0 2024-09-21T07:33:09.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.399+0000 7f0558c00640 1 -- 172.21.15.39:0/91195392 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1431 (secure 0 0 0) 0x7f054c063600 con 0x7f055c06e0a0 2024-09-21T07:33:09.401 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:33:09.401 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":2,"btime":"2024-09-21T07:33:09:108070+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4378,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/414851924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":414851924},{"type":"v1","addr":"172.21.15.39:6833","nonce":414851924}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T07:33:09.401 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 2 2024-09-21T07:33:09.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.402+0000 7f0562c00640 1 -- 172.21.15.39:0/91195392 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0538078bc0 msgr2=0x7f053807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:09.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.402+0000 7f0562c00640 1 --2- 172.21.15.39:0/91195392 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0538078bc0 0x7f053807b0d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f055c0cea50 tx=0x7f0544009040 comp rx=0 tx=0).stop 2024-09-21T07:33:09.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.402+0000 7f0562c00640 1 -- 172.21.15.39:0/91195392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f055c06e0a0 msgr2=0x7f055c1f95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:09.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.402+0000 7f0562c00640 1 --2- 172.21.15.39:0/91195392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f055c06e0a0 0x7f055c1f95f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f054c00e9f0 tx=0x7f054c00eeb0 comp rx=0 tx=0).stop 2024-09-21T07:33:09.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.402+0000 7f0562c00640 1 -- 172.21.15.39:0/91195392 shutdown_connections 2024-09-21T07:33:09.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.402+0000 7f0562c00640 1 --2- 172.21.15.39:0/91195392 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0538078bc0 0x7f053807b0d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:09.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.402+0000 7f0562c00640 1 --2- 172.21.15.39:0/91195392 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f055c0d0e50 0x7f055c0ca330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:09.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.402+0000 7f0562c00640 1 --2- 172.21.15.39:0/91195392 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f055c06f6e0 0x7f055c1eac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:09.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.402+0000 7f0562c00640 1 --2- 172.21.15.39:0/91195392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f055c06e0a0 0x7f055c1f95f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:09.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.402+0000 7f0562c00640 1 -- 172.21.15.39:0/91195392 >> 172.21.15.39:0/91195392 conn(0x7f055c09d840 msgr2=0x7f055c0b10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:09.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.402+0000 7f0562c00640 1 -- 172.21.15.39:0/91195392 shutdown_connections 2024-09-21T07:33:09.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:09.402+0000 7f0562c00640 1 -- 172.21.15.39:0/91195392 wait complete. 2024-09-21T07:33:11.424 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:33:11.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.660+0000 7fcf4ec00640 1 -- 172.21.15.39:0/1146545421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf30004dd0 msgr2=0x7fcf300052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:11.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.660+0000 7fcf4ec00640 1 --2- 172.21.15.39:0/1146545421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf30004dd0 0x7fcf300052a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcf40005fd0 tx=0x7fcf30005830 comp rx=0 tx=0).stop 2024-09-21T07:33:11.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.660+0000 7fcf4e200640 1 -- 172.21.15.39:0/1146545421 reap_dead start 2024-09-21T07:33:11.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.660+0000 7fcf4ec00640 1 -- 172.21.15.39:0/1146545421 shutdown_connections 2024-09-21T07:33:11.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.660+0000 7fcf4ec00640 1 -- 172.21.15.39:0/1146545421 >> 172.21.15.39:0/1146545421 conn(0x7fcf4809d680 msgr2=0x7fcf480d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:11.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.660+0000 7fcf4ec00640 1 -- 172.21.15.39:0/1146545421 shutdown_connections 2024-09-21T07:33:11.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.660+0000 7fcf4ec00640 1 -- 172.21.15.39:0/1146545421 wait complete. 2024-09-21T07:33:11.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.662+0000 7fcf4ec00640 1 Processor -- start 2024-09-21T07:33:11.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.662+0000 7fcf4ec00640 1 -- start start 2024-09-21T07:33:11.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.664+0000 7fcf4ec00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf30004dd0 0x7fcf481f9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:11.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.664+0000 7fcf4ec00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf481f95a0 0x7fcf481f99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:11.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.664+0000 7fcf4ec00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcf480ccbb0 0x7fcf480ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:11.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.664+0000 7fcf4ec00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcf480b9e10 con 0x7fcf481f95a0 2024-09-21T07:33:11.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.664+0000 7fcf4ec00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcf480b50d0 con 0x7fcf30004dd0 2024-09-21T07:33:11.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.664+0000 7fcf4ec00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fcf480cf3e0 con 0x7fcf480ccbb0 2024-09-21T07:33:11.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.664+0000 7fcf4ce00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf481f95a0 0x7fcf481f99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:11.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.664+0000 7fcf4ce00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf481f95a0 0x7fcf481f99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47192/0 (socket says 172.21.15.39:47192) 2024-09-21T07:33:11.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.664+0000 7fcf4ce00640 1 -- 172.21.15.39:0/3580865656 learned_addr learned my addr 172.21.15.39:0/3580865656 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:11.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.664+0000 7fcf4d800640 1 --2- 172.21.15.39:0/3580865656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf30004dd0 0x7fcf481f9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:11.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.664+0000 7fcf4e200640 1 --2- 172.21.15.39:0/3580865656 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcf480ccbb0 0x7fcf480ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:11.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.665+0000 7fcf4ce00640 1 -- 172.21.15.39:0/3580865656 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcf480ccbb0 msgr2=0x7fcf480ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:11.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.665+0000 7fcf4ce00640 1 --2- 172.21.15.39:0/3580865656 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcf480ccbb0 0x7fcf480ccfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:11.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.665+0000 7fcf4ce00640 1 -- 172.21.15.39:0/3580865656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf30004dd0 msgr2=0x7fcf481f9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:11.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.665+0000 7fcf4ce00640 1 --2- 172.21.15.39:0/3580865656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf30004dd0 0x7fcf481f9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:11.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.665+0000 7fcf4ce00640 1 -- 172.21.15.39:0/3580865656 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf480cc6d0 con 0x7fcf481f95a0 2024-09-21T07:33:11.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.665+0000 7fcf4e200640 1 --2- 172.21.15.39:0/3580865656 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcf480ccbb0 0x7fcf480ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:33:11.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.665+0000 7fcf4ce00640 1 --2- 172.21.15.39:0/3580865656 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf481f95a0 0x7fcf481f99b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcf34003e70 tx=0x7fcf34001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:11.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.665+0000 7fcf45600640 1 -- 172.21.15.39:0/3580865656 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf34011700 con 0x7fcf481f95a0 2024-09-21T07:33:11.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.665+0000 7fcf45600640 1 -- 172.21.15.39:0/3580865656 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf34016020 con 0x7fcf481f95a0 2024-09-21T07:33:11.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.665+0000 7fcf45600640 1 -- 172.21.15.39:0/3580865656 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf34011b40 con 0x7fcf481f95a0 2024-09-21T07:33:11.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.665+0000 7fcf4ec00640 1 -- 172.21.15.39:0/3580865656 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf480a0740 con 0x7fcf481f95a0 2024-09-21T07:33:11.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.665+0000 7fcf4ec00640 1 -- 172.21.15.39:0/3580865656 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf480a0d00 con 0x7fcf481f95a0 2024-09-21T07:33:11.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.666+0000 7fcf4ec00640 1 -- 172.21.15.39:0/3580865656 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf480a04b0 con 0x7fcf481f95a0 2024-09-21T07:33:11.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.668+0000 7fcf45600640 1 -- 172.21.15.39:0/3580865656 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcf34011ce0 con 0x7fcf481f95a0 2024-09-21T07:33:11.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.669+0000 7fcf45600640 1 --2- 172.21.15.39:0/3580865656 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcf28078b90 0x7fcf2807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:11.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.669+0000 7fcf45600640 1 -- 172.21.15.39:0/3580865656 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fcf3409b6d0 con 0x7fcf481f95a0 2024-09-21T07:33:11.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.669+0000 7fcf4d800640 1 --2- 172.21.15.39:0/3580865656 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcf28078b90 0x7fcf2807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:11.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.670+0000 7fcf4d800640 1 --2- 172.21.15.39:0/3580865656 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcf28078b90 0x7fcf2807b0a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c006180 tx=0x7fcf3c001e30 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:11.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.674+0000 7fcf45600640 1 -- 172.21.15.39:0/3580865656 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcf34020020 con 0x7fcf481f95a0 2024-09-21T07:33:11.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.908+0000 7fcf4ec00640 1 -- 172.21.15.39:0/3580865656 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf4806f8d0 con 0x7fcf481f95a0 2024-09-21T07:33:11.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.909+0000 7fcf45600640 1 -- 172.21.15.39:0/3580865656 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3909 (secure 0 0 0) 0x7fcf4806f8d0 con 0x7fcf481f95a0 2024-09-21T07:33:11.911 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:33:11.911 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":3,"btime":"2024-09-21T07:33:10:109948+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4325,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1939850940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1939850940},{"type":"v1","addr":"172.21.15.39:6835","nonce":1939850940}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4328,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3712021428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3712021428},{"type":"v1","addr":"172.21.15.148:6836","nonce":3712021428}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4331,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3708353709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3708353709},{"type":"v1","addr":"172.21.15.148:6837","nonce":3708353709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4378,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/414851924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":414851924},{"type":"v1","addr":"172.21.15.39:6833","nonce":414851924}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T07:33:11.911 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 3 2024-09-21T07:33:11.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.912+0000 7fcf4ec00640 1 -- 172.21.15.39:0/3580865656 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcf28078b90 msgr2=0x7fcf2807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:11.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.912+0000 7fcf4ec00640 1 --2- 172.21.15.39:0/3580865656 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcf28078b90 0x7fcf2807b0a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c006180 tx=0x7fcf3c001e30 comp rx=0 tx=0).stop 2024-09-21T07:33:11.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.912+0000 7fcf4ec00640 1 -- 172.21.15.39:0/3580865656 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf481f95a0 msgr2=0x7fcf481f99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:11.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.912+0000 7fcf4ec00640 1 --2- 172.21.15.39:0/3580865656 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf481f95a0 0x7fcf481f99b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcf34003e70 tx=0x7fcf34001800 comp rx=0 tx=0).stop 2024-09-21T07:33:11.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.912+0000 7fcf4ec00640 1 -- 172.21.15.39:0/3580865656 shutdown_connections 2024-09-21T07:33:11.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.912+0000 7fcf4ec00640 1 --2- 172.21.15.39:0/3580865656 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcf28078b90 0x7fcf2807b0a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:11.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.912+0000 7fcf4ec00640 1 --2- 172.21.15.39:0/3580865656 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcf480ccbb0 0x7fcf480ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:11.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.912+0000 7fcf4ec00640 1 --2- 172.21.15.39:0/3580865656 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf481f95a0 0x7fcf481f99b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:11.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.912+0000 7fcf4ec00640 1 --2- 172.21.15.39:0/3580865656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf30004dd0 0x7fcf481f9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:11.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.912+0000 7fcf4ec00640 1 -- 172.21.15.39:0/3580865656 >> 172.21.15.39:0/3580865656 conn(0x7fcf4809d680 msgr2=0x7fcf4806de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:11.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.912+0000 7fcf4ec00640 1 -- 172.21.15.39:0/3580865656 shutdown_connections 2024-09-21T07:33:11.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:11.912+0000 7fcf4ec00640 1 -- 172.21.15.39:0/3580865656 wait complete. 2024-09-21T07:33:11.936 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T07:33:12.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.161+0000 7fd997000640 1 -- 172.21.15.39:0/1219666709 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd990070030 msgr2=0x7fd990074640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:12.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.161+0000 7fd997000640 1 --2- 172.21.15.39:0/1219666709 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd990070030 0x7fd990074640 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd97c005fd0 tx=0x7fd97c02a540 comp rx=0 tx=0).stop 2024-09-21T07:33:12.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.161+0000 7fd997000640 1 -- 172.21.15.39:0/1219666709 shutdown_connections 2024-09-21T07:33:12.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.161+0000 7fd997000640 1 --2- 172.21.15.39:0/1219666709 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd990070030 0x7fd990074640 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:12.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.161+0000 7fd997000640 1 --2- 172.21.15.39:0/1219666709 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd990073170 0x7fd99015c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:12.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.161+0000 7fd997000640 1 -- 172.21.15.39:0/1219666709 >> 172.21.15.39:0/1219666709 conn(0x7fd9900b1cb0 msgr2=0x7fd9900b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:12.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.161+0000 7fd997000640 1 -- 172.21.15.39:0/1219666709 shutdown_connections 2024-09-21T07:33:12.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.161+0000 7fd997000640 1 -- 172.21.15.39:0/1219666709 wait complete. 2024-09-21T07:33:12.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.163+0000 7fd997000640 1 Processor -- start 2024-09-21T07:33:12.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.163+0000 7fd997000640 1 -- start start 2024-09-21T07:33:12.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.165+0000 7fd997000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd990070030 0x7fd9900cbff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:12.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.165+0000 7fd997000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd990070440 0x7fd99015e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:12.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.165+0000 7fd997000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd990073170 0x7fd99015e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:12.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.165+0000 7fd997000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd99006f5a0 con 0x7fd990073170 2024-09-21T07:33:12.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.165+0000 7fd997000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd990074f80 con 0x7fd990070440 2024-09-21T07:33:12.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.165+0000 7fd997000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd99006f720 con 0x7fd990070030 2024-09-21T07:33:12.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.165+0000 7fd995000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd990073170 0x7fd99015e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:12.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.165+0000 7fd995000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd990073170 0x7fd99015e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60338/0 (socket says 172.21.15.39:60338) 2024-09-21T07:33:12.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.165+0000 7fd995000640 1 -- 172.21.15.39:0/98840782 learned_addr learned my addr 172.21.15.39:0/98840782 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:12.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.165+0000 7fd98fe00640 1 --2- 172.21.15.39:0/98840782 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd990070030 0x7fd9900cbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:12.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.165+0000 7fd98f400640 1 --2- 172.21.15.39:0/98840782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd990070440 0x7fd99015e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:12.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.165+0000 7fd995000640 1 -- 172.21.15.39:0/98840782 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd990070030 msgr2=0x7fd9900cbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:12.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.165+0000 7fd995000640 1 --2- 172.21.15.39:0/98840782 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd990070030 0x7fd9900cbff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:12.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.165+0000 7fd995000640 1 -- 172.21.15.39:0/98840782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd990070440 msgr2=0x7fd99015e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:12.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.165+0000 7fd995000640 1 --2- 172.21.15.39:0/98840782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd990070440 0x7fd99015e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:12.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.166+0000 7fd995000640 1 -- 172.21.15.39:0/98840782 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9901f6570 con 0x7fd990073170 2024-09-21T07:33:12.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.166+0000 7fd98f400640 1 --2- 172.21.15.39:0/98840782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd990070440 0x7fd99015e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:33:12.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.166+0000 7fd995000640 1 --2- 172.21.15.39:0/98840782 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd990073170 0x7fd99015e850 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd98800c730 tx=0x7fd98800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:12.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.166+0000 7fd987e00640 1 -- 172.21.15.39:0/98840782 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd988003b70 con 0x7fd990073170 2024-09-21T07:33:12.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.166+0000 7fd987e00640 1 -- 172.21.15.39:0/98840782 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd988003d30 con 0x7fd990073170 2024-09-21T07:33:12.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.166+0000 7fd987e00640 1 -- 172.21.15.39:0/98840782 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9880052d0 con 0x7fd990073170 2024-09-21T07:33:12.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.166+0000 7fd997000640 1 -- 172.21.15.39:0/98840782 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd990150c60 con 0x7fd990073170 2024-09-21T07:33:12.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.166+0000 7fd997000640 1 -- 172.21.15.39:0/98840782 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd990151220 con 0x7fd990073170 2024-09-21T07:33:12.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.167+0000 7fd997000640 1 -- 172.21.15.39:0/98840782 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd99014ea80 con 0x7fd990073170 2024-09-21T07:33:12.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.169+0000 7fd987e00640 1 -- 172.21.15.39:0/98840782 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd988005680 con 0x7fd990073170 2024-09-21T07:33:12.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.170+0000 7fd987e00640 1 --2- 172.21.15.39:0/98840782 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd970078b90 0x7fd97007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:12.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.170+0000 7fd987e00640 1 -- 172.21.15.39:0/98840782 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fd98809a8e0 con 0x7fd990073170 2024-09-21T07:33:12.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.170+0000 7fd98fe00640 1 --2- 172.21.15.39:0/98840782 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd970078b90 0x7fd97007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:12.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.171+0000 7fd98fe00640 1 --2- 172.21.15.39:0/98840782 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd970078b90 0x7fd97007b0a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd980009b30 tx=0x7fd980005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:12.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.175+0000 7fd987e00640 1 -- 172.21.15.39:0/98840782 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd988063060 con 0x7fd990073170 2024-09-21T07:33:12.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.372+0000 7fd997000640 1 -- 172.21.15.39:0/98840782 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fd9900cc5b0 con 0x7fd990073170 2024-09-21T07:33:12.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.372+0000 7fd987e00640 1 -- 172.21.15.39:0/98840782 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7fd988062960 con 0x7fd990073170 2024-09-21T07:33:12.375 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:33:12.375 INFO:teuthology.orchestra.run.smithi039.stdout:[ 2024-09-21T07:33:12.375 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:33:12.375 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 1, 2024-09-21T07:33:12.375 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": ".mgr" 2024-09-21T07:33:12.375 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:33:12.375 INFO:teuthology.orchestra.run.smithi039.stdout:] 2024-09-21T07:33:12.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.375+0000 7fd997000640 1 -- 172.21.15.39:0/98840782 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd970078b90 msgr2=0x7fd97007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:12.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.375+0000 7fd997000640 1 --2- 172.21.15.39:0/98840782 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd970078b90 0x7fd97007b0a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd980009b30 tx=0x7fd980005f90 comp rx=0 tx=0).stop 2024-09-21T07:33:12.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.375+0000 7fd997000640 1 -- 172.21.15.39:0/98840782 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd990073170 msgr2=0x7fd99015e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:12.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.375+0000 7fd997000640 1 --2- 172.21.15.39:0/98840782 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd990073170 0x7fd99015e850 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd98800c730 tx=0x7fd98800cbf0 comp rx=0 tx=0).stop 2024-09-21T07:33:12.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.375+0000 7fd997000640 1 -- 172.21.15.39:0/98840782 shutdown_connections 2024-09-21T07:33:12.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.375+0000 7fd997000640 1 --2- 172.21.15.39:0/98840782 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd970078b90 0x7fd97007b0a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:12.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.375+0000 7fd997000640 1 --2- 172.21.15.39:0/98840782 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd990073170 0x7fd99015e850 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:12.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.375+0000 7fd997000640 1 --2- 172.21.15.39:0/98840782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd990070440 0x7fd99015e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:12.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.375+0000 7fd997000640 1 --2- 172.21.15.39:0/98840782 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd990070030 0x7fd9900cbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:12.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.375+0000 7fd997000640 1 -- 172.21.15.39:0/98840782 >> 172.21.15.39:0/98840782 conn(0x7fd9900b1cb0 msgr2=0x7fd99015cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:12.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.375+0000 7fd997000640 1 -- 172.21.15.39:0/98840782 shutdown_connections 2024-09-21T07:33:12.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.375+0000 7fd997000640 1 -- 172.21.15.39:0/98840782 wait complete. 2024-09-21T07:33:12.401 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T07:33:12.401 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T07:33:12.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.631+0000 7f384f200640 1 -- 172.21.15.39:0/2848252791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3834004d40 msgr2=0x7f3834005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:12.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.631+0000 7f384f200640 1 --2- 172.21.15.39:0/2848252791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3834004d40 0x7f3834005150 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3848009ed0 tx=0x7f3834005830 comp rx=0 tx=0).stop 2024-09-21T07:33:12.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.631+0000 7f384e800640 1 -- 172.21.15.39:0/2848252791 reap_dead start 2024-09-21T07:33:12.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.631+0000 7f384f200640 1 -- 172.21.15.39:0/2848252791 shutdown_connections 2024-09-21T07:33:12.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.631+0000 7f384f200640 1 -- 172.21.15.39:0/2848252791 >> 172.21.15.39:0/2848252791 conn(0x7f385006ee70 msgr2=0x7f385006f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:12.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.631+0000 7f384f200640 1 -- 172.21.15.39:0/2848252791 shutdown_connections 2024-09-21T07:33:12.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.631+0000 7f384f200640 1 -- 172.21.15.39:0/2848252791 wait complete. 2024-09-21T07:33:12.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.633+0000 7f384f200640 1 Processor -- start 2024-09-21T07:33:12.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.633+0000 7f384f200640 1 -- start start 2024-09-21T07:33:12.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.635+0000 7f384f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3834004d40 0x7f3850155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:12.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.635+0000 7f384f200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f385015b7d0 0x7f38501f4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:12.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.635+0000 7f384f200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38501f4a60 0x7f38501f6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:12.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.635+0000 7f384f200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f38500ceda0 con 0x7f3834004d40 2024-09-21T07:33:12.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.635+0000 7f384f200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f38500b5980 con 0x7f38501f4a60 2024-09-21T07:33:12.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.635+0000 7f384f200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f38500cef20 con 0x7f385015b7d0 2024-09-21T07:33:12.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.635+0000 7f384de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3834004d40 0x7f3850155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:12.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.635+0000 7f384de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3834004d40 0x7f3850155d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60362/0 (socket says 172.21.15.39:60362) 2024-09-21T07:33:12.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.635+0000 7f384de00640 1 -- 172.21.15.39:0/1223296179 learned_addr learned my addr 172.21.15.39:0/1223296179 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:12.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.635+0000 7f3847e00640 1 --2- 172.21.15.39:0/1223296179 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f385015b7d0 0x7f38501f4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:12.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.635+0000 7f384e800640 1 --2- 172.21.15.39:0/1223296179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38501f4a60 0x7f38501f6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:12.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.635+0000 7f384de00640 1 -- 172.21.15.39:0/1223296179 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f385015b7d0 msgr2=0x7f38501f4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:12.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.635+0000 7f384de00640 1 --2- 172.21.15.39:0/1223296179 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f385015b7d0 0x7f38501f4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:12.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.635+0000 7f384de00640 1 -- 172.21.15.39:0/1223296179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38501f4a60 msgr2=0x7f38501f6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:12.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.635+0000 7f384de00640 1 --2- 172.21.15.39:0/1223296179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38501f4a60 0x7f38501f6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:12.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.635+0000 7f384de00640 1 -- 172.21.15.39:0/1223296179 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f385015a730 con 0x7f3834004d40 2024-09-21T07:33:12.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.636+0000 7f384de00640 1 --2- 172.21.15.39:0/1223296179 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3834004d40 0x7f3850155d90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3840011b40 tx=0x7f3840009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:12.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.636+0000 7f3846000640 1 -- 172.21.15.39:0/1223296179 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384000fd10 con 0x7f3834004d40 2024-09-21T07:33:12.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.636+0000 7f3846000640 1 -- 172.21.15.39:0/1223296179 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3840004020 con 0x7f3834004d40 2024-09-21T07:33:12.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.636+0000 7f384f200640 1 -- 172.21.15.39:0/1223296179 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f385015aa10 con 0x7f3834004d40 2024-09-21T07:33:12.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.636+0000 7f3846000640 1 -- 172.21.15.39:0/1223296179 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3840004470 con 0x7f3834004d40 2024-09-21T07:33:12.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.636+0000 7f384f200640 1 -- 172.21.15.39:0/1223296179 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f385015afd0 con 0x7f3834004d40 2024-09-21T07:33:12.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.637+0000 7f384f200640 1 -- 172.21.15.39:0/1223296179 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3850073590 con 0x7f3834004d40 2024-09-21T07:33:12.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.638+0000 7f3846000640 1 -- 172.21.15.39:0/1223296179 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f38400035f0 con 0x7f3834004d40 2024-09-21T07:33:12.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.639+0000 7f3846000640 1 --2- 172.21.15.39:0/1223296179 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f382c078b90 0x7f382c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:12.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.639+0000 7f3846000640 1 -- 172.21.15.39:0/1223296179 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f384009b4e0 con 0x7f3834004d40 2024-09-21T07:33:12.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.640+0000 7f3847e00640 1 --2- 172.21.15.39:0/1223296179 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f382c078b90 0x7f382c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:12.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.641+0000 7f3847e00640 1 --2- 172.21.15.39:0/1223296179 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f382c078b90 0x7f382c07b0a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3850051080 tx=0x7f3838006ce0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:12.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.644+0000 7f3846000640 1 -- 172.21.15.39:0/1223296179 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3840063610 con 0x7f3834004d40 2024-09-21T07:33:12.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:12.841+0000 7f384f200640 1 -- 172.21.15.39:0/1223296179 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f385015e230 con 0x7f3834004d40 2024-09-21T07:33:13.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.133+0000 7f3846000640 1 -- 172.21.15.39:0/1223296179 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7f3840068040 con 0x7f3834004d40 2024-09-21T07:33:13.135 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' created 2024-09-21T07:33:13.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.136+0000 7f384f200640 1 -- 172.21.15.39:0/1223296179 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f382c078b90 msgr2=0x7f382c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:13.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.136+0000 7f384f200640 1 --2- 172.21.15.39:0/1223296179 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f382c078b90 0x7f382c07b0a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3850051080 tx=0x7f3838006ce0 comp rx=0 tx=0).stop 2024-09-21T07:33:13.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.136+0000 7f384f200640 1 -- 172.21.15.39:0/1223296179 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3834004d40 msgr2=0x7f3850155d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:13.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.136+0000 7f384f200640 1 --2- 172.21.15.39:0/1223296179 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3834004d40 0x7f3850155d90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3840011b40 tx=0x7f3840009f10 comp rx=0 tx=0).stop 2024-09-21T07:33:13.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.136+0000 7f384f200640 1 -- 172.21.15.39:0/1223296179 shutdown_connections 2024-09-21T07:33:13.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.136+0000 7f384f200640 1 --2- 172.21.15.39:0/1223296179 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f382c078b90 0x7f382c07b0a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:13.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.136+0000 7f384f200640 1 --2- 172.21.15.39:0/1223296179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38501f4a60 0x7f38501f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:13.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.136+0000 7f384f200640 1 --2- 172.21.15.39:0/1223296179 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f385015b7d0 0x7f38501f4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:13.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.136+0000 7f384f200640 1 --2- 172.21.15.39:0/1223296179 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3834004d40 0x7f3850155d90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:13.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.136+0000 7f384f200640 1 -- 172.21.15.39:0/1223296179 >> 172.21.15.39:0/1223296179 conn(0x7f385006ee70 msgr2=0x7f3850158360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:13.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.136+0000 7f384f200640 1 -- 172.21.15.39:0/1223296179 shutdown_connections 2024-09-21T07:33:13.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.136+0000 7f384f200640 1 -- 172.21.15.39:0/1223296179 wait complete. 2024-09-21T07:33:13.167 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T07:33:13.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.398+0000 7f716b200640 1 -- 172.21.15.39:0/1647071266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f716c0d0dc0 msgr2=0x7f716c0b1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:13.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.398+0000 7f716b200640 1 --2- 172.21.15.39:0/1647071266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f716c0d0dc0 0x7f716c0b1a20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7164005fd0 tx=0x7f716402aa90 comp rx=0 tx=0).stop 2024-09-21T07:33:13.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.398+0000 7f716b200640 1 -- 172.21.15.39:0/1647071266 shutdown_connections 2024-09-21T07:33:13.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.398+0000 7f716b200640 1 --2- 172.21.15.39:0/1647071266 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f716c0b7580 0x7f716c0d0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:13.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.398+0000 7f716b200640 1 --2- 172.21.15.39:0/1647071266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f716c0d0dc0 0x7f716c0b1a20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:13.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.398+0000 7f716b200640 1 -- 172.21.15.39:0/1647071266 >> 172.21.15.39:0/1647071266 conn(0x7f716c0ceba0 msgr2=0x7f716c0cefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:13.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.398+0000 7f716b200640 1 -- 172.21.15.39:0/1647071266 shutdown_connections 2024-09-21T07:33:13.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.398+0000 7f716b200640 1 -- 172.21.15.39:0/1647071266 wait complete. 2024-09-21T07:33:13.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.400+0000 7f716b200640 1 Processor -- start 2024-09-21T07:33:13.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.400+0000 7f716b200640 1 -- start start 2024-09-21T07:33:13.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.402+0000 7f716b200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f716c0b7580 0x7f716c163bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:13.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.402+0000 7f716b200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f716c0ce730 0x7f716c15f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:13.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.402+0000 7f716b200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f716c0d0dc0 0x7f716c166100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:13.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.402+0000 7f716b200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f716c0b22f0 con 0x7f716c0d0dc0 2024-09-21T07:33:13.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.402+0000 7f716b200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f716c0b2170 con 0x7f716c0ce730 2024-09-21T07:33:13.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.402+0000 7f716b200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f716c0b2470 con 0x7f716c0b7580 2024-09-21T07:33:13.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.402+0000 7f716a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f716c0d0dc0 0x7f716c166100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:13.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.402+0000 7f716a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f716c0d0dc0 0x7f716c166100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60376/0 (socket says 172.21.15.39:60376) 2024-09-21T07:33:13.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.402+0000 7f716a800640 1 -- 172.21.15.39:0/2866620138 learned_addr learned my addr 172.21.15.39:0/2866620138 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:13.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.402+0000 7f7163e00640 1 --2- 172.21.15.39:0/2866620138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f716c0ce730 0x7f716c15f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:13.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.402+0000 7f7169e00640 1 --2- 172.21.15.39:0/2866620138 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f716c0b7580 0x7f716c163bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:13.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.402+0000 7f716a800640 1 -- 172.21.15.39:0/2866620138 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f716c0b7580 msgr2=0x7f716c163bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:13.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.402+0000 7f716a800640 1 --2- 172.21.15.39:0/2866620138 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f716c0b7580 0x7f716c163bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:13.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.402+0000 7f716a800640 1 -- 172.21.15.39:0/2866620138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f716c0ce730 msgr2=0x7f716c15f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:13.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.402+0000 7f716a800640 1 --2- 172.21.15.39:0/2866620138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f716c0ce730 0x7f716c15f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:13.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.402+0000 7f716a800640 1 -- 172.21.15.39:0/2866620138 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f716c164470 con 0x7f716c0d0dc0 2024-09-21T07:33:13.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.403+0000 7f716a800640 1 --2- 172.21.15.39:0/2866620138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f716c0d0dc0 0x7f716c166100 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7164009d90 tx=0x7f7164003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:13.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.403+0000 7f7162000640 1 -- 172.21.15.39:0/2866620138 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7164044070 con 0x7f716c0d0dc0 2024-09-21T07:33:13.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.403+0000 7f7162000640 1 -- 172.21.15.39:0/2866620138 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f716402e7f0 con 0x7f716c0d0dc0 2024-09-21T07:33:13.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.403+0000 7f716b200640 1 -- 172.21.15.39:0/2866620138 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f716c164750 con 0x7f716c0d0dc0 2024-09-21T07:33:13.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.403+0000 7f7162000640 1 -- 172.21.15.39:0/2866620138 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71640044b0 con 0x7f716c0d0dc0 2024-09-21T07:33:13.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.403+0000 7f716b200640 1 -- 172.21.15.39:0/2866620138 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f716c1f6e10 con 0x7f716c0d0dc0 2024-09-21T07:33:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.404+0000 7f716b200640 1 -- 172.21.15.39:0/2866620138 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f716c0d0600 con 0x7f716c0d0dc0 2024-09-21T07:33:13.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.407+0000 7f7162000640 1 -- 172.21.15.39:0/2866620138 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7164047070 con 0x7f716c0d0dc0 2024-09-21T07:33:13.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.408+0000 7f7162000640 1 --2- 172.21.15.39:0/2866620138 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7148078b90 0x7f714807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:13.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.408+0000 7f7162000640 1 -- 172.21.15.39:0/2866620138 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7f71640bbb20 con 0x7f716c0d0dc0 2024-09-21T07:33:13.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.408+0000 7f7169e00640 1 --2- 172.21.15.39:0/2866620138 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7148078b90 0x7f714807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:13.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.409+0000 7f7169e00640 1 --2- 172.21.15.39:0/2866620138 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7148078b90 0x7f714807b0a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f715c007fd0 tx=0x7f715c009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:13.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.412+0000 7f7162000640 1 -- 172.21.15.39:0/2866620138 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f716c0d0600 con 0x7f716c0d0dc0 2024-09-21T07:33:13.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:13.610+0000 7f716b200640 1 -- 172.21.15.39:0/2866620138 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f716c0d18b0 con 0x7f716c0d0dc0 2024-09-21T07:33:14.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.143+0000 7f7162000640 1 -- 172.21.15.39:0/2866620138 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) ==== 170+0+0 (secure 0 0 0) 0x7f716c0d18b0 con 0x7f716c0d0dc0 2024-09-21T07:33:14.146 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' created 2024-09-21T07:33:14.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.146+0000 7f716b200640 1 -- 172.21.15.39:0/2866620138 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7148078b90 msgr2=0x7f714807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:14.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.146+0000 7f716b200640 1 --2- 172.21.15.39:0/2866620138 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7148078b90 0x7f714807b0a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f715c007fd0 tx=0x7f715c009040 comp rx=0 tx=0).stop 2024-09-21T07:33:14.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.146+0000 7f716b200640 1 -- 172.21.15.39:0/2866620138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f716c0d0dc0 msgr2=0x7f716c166100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:14.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.146+0000 7f716b200640 1 --2- 172.21.15.39:0/2866620138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f716c0d0dc0 0x7f716c166100 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7164009d90 tx=0x7f7164003f10 comp rx=0 tx=0).stop 2024-09-21T07:33:14.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.146+0000 7f716b200640 1 -- 172.21.15.39:0/2866620138 shutdown_connections 2024-09-21T07:33:14.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.146+0000 7f716b200640 1 --2- 172.21.15.39:0/2866620138 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7148078b90 0x7f714807b0a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:14.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.146+0000 7f716b200640 1 --2- 172.21.15.39:0/2866620138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f716c0d0dc0 0x7f716c166100 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:14.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.146+0000 7f716b200640 1 --2- 172.21.15.39:0/2866620138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f716c0ce730 0x7f716c15f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:14.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.146+0000 7f716b200640 1 --2- 172.21.15.39:0/2866620138 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f716c0b7580 0x7f716c163bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:14.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.146+0000 7f716b200640 1 -- 172.21.15.39:0/2866620138 >> 172.21.15.39:0/2866620138 conn(0x7f716c0ceba0 msgr2=0x7f716c0cefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:14.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.146+0000 7f716b200640 1 -- 172.21.15.39:0/2866620138 shutdown_connections 2024-09-21T07:33:14.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.146+0000 7f716b200640 1 -- 172.21.15.39:0/2866620138 wait complete. 2024-09-21T07:33:14.192 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T07:33:14.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.420+0000 7fbfa0600640 1 -- 172.21.15.39:0/1042608123 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf9815d8a0 msgr2=0x7fbf980b40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:14.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.420+0000 7fbfa0600640 1 --2- 172.21.15.39:0/1042608123 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf9815d8a0 0x7fbf980b40b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbf98071d60 tx=0x7fbf8402a530 comp rx=0 tx=0).stop 2024-09-21T07:33:14.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.420+0000 7fbfa0600640 1 -- 172.21.15.39:0/1042608123 shutdown_connections 2024-09-21T07:33:14.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.420+0000 7fbfa0600640 1 --2- 172.21.15.39:0/1042608123 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf9815d8a0 0x7fbf980b40b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:14.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.420+0000 7fbfa0600640 1 --2- 172.21.15.39:0/1042608123 >> v2:172.21.15.148:3300/0 conn(0x7fbf980cc1e0 0x7fbf980af3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:14.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.420+0000 7fbfa0600640 1 -- 172.21.15.39:0/1042608123 >> 172.21.15.39:0/1042608123 conn(0x7fbf9806d7d0 msgr2=0x7fbf980b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:14.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.420+0000 7fbfa0600640 1 -- 172.21.15.39:0/1042608123 shutdown_connections 2024-09-21T07:33:14.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.421+0000 7fbfa0600640 1 -- 172.21.15.39:0/1042608123 wait complete. 2024-09-21T07:33:14.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.423+0000 7fbfa0600640 1 Processor -- start 2024-09-21T07:33:14.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.423+0000 7fbfa0600640 1 -- start start 2024-09-21T07:33:14.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.424+0000 7fbfa0600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf980cc1e0 0x7fbf98152350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:14.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.424+0000 7fbfa0600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf980ccb50 0x7fbf9815ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:14.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.424+0000 7fbfa0600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbf9815d8a0 0x7fbf98153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:14.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.424+0000 7fbfa0600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbf98153520 con 0x7fbf980ccb50 2024-09-21T07:33:14.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.424+0000 7fbfa0600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbf981533a0 con 0x7fbf980cc1e0 2024-09-21T07:33:14.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.424+0000 7fbfa0600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fbf981536a0 con 0x7fbf9815d8a0 2024-09-21T07:33:14.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.425+0000 7fbf9d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf980ccb50 0x7fbf9815ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:14.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.425+0000 7fbf9d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf980ccb50 0x7fbf9815ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60388/0 (socket says 172.21.15.39:60388) 2024-09-21T07:33:14.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.425+0000 7fbf9d200640 1 -- 172.21.15.39:0/887633551 learned_addr learned my addr 172.21.15.39:0/887633551 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:14.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.425+0000 7fbf9dc00640 1 --2- 172.21.15.39:0/887633551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf980cc1e0 0x7fbf98152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:14.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.425+0000 7fbf9e600640 1 --2- 172.21.15.39:0/887633551 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbf9815d8a0 0x7fbf98153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:14.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.425+0000 7fbf9dc00640 1 -- 172.21.15.39:0/887633551 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbf9815d8a0 msgr2=0x7fbf98153d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:14.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.425+0000 7fbf9dc00640 1 --2- 172.21.15.39:0/887633551 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbf9815d8a0 0x7fbf98153d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:14.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.425+0000 7fbf9dc00640 1 -- 172.21.15.39:0/887633551 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf980ccb50 msgr2=0x7fbf9815ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:14.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.425+0000 7fbf9dc00640 1 --2- 172.21.15.39:0/887633551 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf980ccb50 0x7fbf9815ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:14.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.425+0000 7fbf9dc00640 1 -- 172.21.15.39:0/887633551 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf9815def0 con 0x7fbf980cc1e0 2024-09-21T07:33:14.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.425+0000 7fbf9d200640 1 --2- 172.21.15.39:0/887633551 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf980ccb50 0x7fbf9815ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:33:14.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.425+0000 7fbf9dc00640 1 --2- 172.21.15.39:0/887633551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf980cc1e0 0x7fbf98152350 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c00e700 tx=0x7fbf8c00ebc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:14.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.426+0000 7fbf95600640 1 -- 172.21.15.39:0/887633551 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c00cab0 con 0x7fbf980cc1e0 2024-09-21T07:33:14.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.426+0000 7fbf95600640 1 -- 172.21.15.39:0/887633551 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf8c00cd40 con 0x7fbf980cc1e0 2024-09-21T07:33:14.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.426+0000 7fbfa0600640 1 -- 172.21.15.39:0/887633551 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf9815e190 con 0x7fbf980cc1e0 2024-09-21T07:33:14.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.426+0000 7fbf95600640 1 -- 172.21.15.39:0/887633551 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c010690 con 0x7fbf980cc1e0 2024-09-21T07:33:14.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.426+0000 7fbfa0600640 1 -- 172.21.15.39:0/887633551 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf9815e750 con 0x7fbf980cc1e0 2024-09-21T07:33:14.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.427+0000 7fbfa0600640 1 -- 172.21.15.39:0/887633551 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf98153b50 con 0x7fbf980cc1e0 2024-09-21T07:33:14.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.428+0000 7fbf95600640 1 -- 172.21.15.39:0/887633551 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbf8c010830 con 0x7fbf980cc1e0 2024-09-21T07:33:14.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.429+0000 7fbf95600640 1 --2- 172.21.15.39:0/887633551 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbf78078bc0 0x7fbf7807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:14.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.429+0000 7fbf95600640 1 -- 172.21.15.39:0/887633551 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7fbf8c09b520 con 0x7fbf980cc1e0 2024-09-21T07:33:14.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.429+0000 7fbf9d200640 1 --2- 172.21.15.39:0/887633551 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbf78078bc0 0x7fbf7807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:14.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.430+0000 7fbf9d200640 1 --2- 172.21.15.39:0/887633551 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbf78078bc0 0x7fbf7807b0d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbf981ea1f0 tx=0x7fbf84004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:14.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.435+0000 7fbf95600640 1 -- 172.21.15.39:0/887633551 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbf8c063360 con 0x7fbf980cc1e0 2024-09-21T07:33:14.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:14.672+0000 7fbfa0600640 1 -- 172.21.15.39:0/887633551 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fbf980cdae0 con 0x7fbf980cc1e0 2024-09-21T07:33:15.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.152+0000 7fbf95600640 1 -- 172.21.15.39:0/887633551 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7fbf8c067d90 con 0x7fbf980cc1e0 2024-09-21T07:33:15.155 INFO:teuthology.orchestra.run.smithi039.stderr:new fs with metadata pool 2 and data pool 3 2024-09-21T07:33:15.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.159+0000 7fbfa0600640 1 -- 172.21.15.39:0/887633551 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbf78078bc0 msgr2=0x7fbf7807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:15.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.159+0000 7fbfa0600640 1 --2- 172.21.15.39:0/887633551 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbf78078bc0 0x7fbf7807b0d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbf981ea1f0 tx=0x7fbf84004730 comp rx=0 tx=0).stop 2024-09-21T07:33:15.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.159+0000 7fbfa0600640 1 -- 172.21.15.39:0/887633551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf980cc1e0 msgr2=0x7fbf98152350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:15.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.159+0000 7fbfa0600640 1 --2- 172.21.15.39:0/887633551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf980cc1e0 0x7fbf98152350 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c00e700 tx=0x7fbf8c00ebc0 comp rx=0 tx=0).stop 2024-09-21T07:33:15.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.159+0000 7fbfa0600640 1 -- 172.21.15.39:0/887633551 shutdown_connections 2024-09-21T07:33:15.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.159+0000 7fbfa0600640 1 --2- 172.21.15.39:0/887633551 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbf78078bc0 0x7fbf7807b0d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:15.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.159+0000 7fbfa0600640 1 --2- 172.21.15.39:0/887633551 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbf9815d8a0 0x7fbf98153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:15.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.159+0000 7fbfa0600640 1 --2- 172.21.15.39:0/887633551 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf980ccb50 0x7fbf9815ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:15.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.159+0000 7fbfa0600640 1 --2- 172.21.15.39:0/887633551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf980cc1e0 0x7fbf98152350 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:15.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.159+0000 7fbfa0600640 1 -- 172.21.15.39:0/887633551 >> 172.21.15.39:0/887633551 conn(0x7fbf9806d7d0 msgr2=0x7fbf980b1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:15.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.159+0000 7fbfa0600640 1 -- 172.21.15.39:0/887633551 shutdown_connections 2024-09-21T07:33:15.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.159+0000 7fbfa0600640 1 -- 172.21.15.39:0/887633551 wait complete. 2024-09-21T07:33:15.205 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:33:15.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.446+0000 7f8720c00640 1 -- 172.21.15.39:0/68784471 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f871c0af780 msgr2=0x7f871c0b39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:15.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.446+0000 7f8720c00640 1 --2- 172.21.15.39:0/68784471 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f871c0af780 0x7f871c0b39c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f870c005fd0 tx=0x7f870c02a5d0 comp rx=0 tx=0).stop 2024-09-21T07:33:15.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.446+0000 7f8720c00640 1 -- 172.21.15.39:0/68784471 shutdown_connections 2024-09-21T07:33:15.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.446+0000 7f8720c00640 1 --2- 172.21.15.39:0/68784471 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f871c072ff0 0x7f871c073400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:15.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.446+0000 7f8720c00640 1 --2- 172.21.15.39:0/68784471 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f871c0af780 0x7f871c0b39c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:15.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.446+0000 7f8720c00640 1 -- 172.21.15.39:0/68784471 >> 172.21.15.39:0/68784471 conn(0x7f871c06d7d0 msgr2=0x7f871c0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:15.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.446+0000 7f8720c00640 1 -- 172.21.15.39:0/68784471 shutdown_connections 2024-09-21T07:33:15.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.446+0000 7f8720c00640 1 -- 172.21.15.39:0/68784471 wait complete. 2024-09-21T07:33:15.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.449+0000 7f8720c00640 1 Processor -- start 2024-09-21T07:33:15.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.449+0000 7f8720c00640 1 -- start start 2024-09-21T07:33:15.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.450+0000 7f8720c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f871c072ff0 0x7f871c14e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:15.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.450+0000 7f8720c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f871c0af780 0x7f871c1e8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:15.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.450+0000 7f8720c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f871c0cd3e0 0x7f871c1e93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:15.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.450+0000 7f8720c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f871c073d10 con 0x7f871c072ff0 2024-09-21T07:33:15.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.450+0000 7f8720c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f871c073b90 con 0x7f871c0cd3e0 2024-09-21T07:33:15.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.450+0000 7f8720c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f871c15f930 con 0x7f871c0af780 2024-09-21T07:33:15.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.450+0000 7f8719e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f871c072ff0 0x7f871c14e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:15.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.450+0000 7f8719e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f871c072ff0 0x7f871c14e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60416/0 (socket says 172.21.15.39:60416) 2024-09-21T07:33:15.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.450+0000 7f8719e00640 1 -- 172.21.15.39:0/1323632907 learned_addr learned my addr 172.21.15.39:0/1323632907 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:15.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.451+0000 7f871a800640 1 --2- 172.21.15.39:0/1323632907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f871c0cd3e0 0x7f871c1e93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:15.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.451+0000 7f8713e00640 1 --2- 172.21.15.39:0/1323632907 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f871c0af780 0x7f871c1e8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:15.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.451+0000 7f8719e00640 1 -- 172.21.15.39:0/1323632907 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f871c0af780 msgr2=0x7f871c1e8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:15.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.451+0000 7f8719e00640 1 --2- 172.21.15.39:0/1323632907 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f871c0af780 0x7f871c1e8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:15.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.451+0000 7f8719e00640 1 -- 172.21.15.39:0/1323632907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f871c0cd3e0 msgr2=0x7f871c1e93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:15.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.451+0000 7f8719e00640 1 --2- 172.21.15.39:0/1323632907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f871c0cd3e0 0x7f871c1e93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:15.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.451+0000 7f8719e00640 1 -- 172.21.15.39:0/1323632907 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f871c14eda0 con 0x7f871c072ff0 2024-09-21T07:33:15.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.451+0000 7f8713e00640 1 --2- 172.21.15.39:0/1323632907 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f871c0af780 0x7f871c1e8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:33:15.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.451+0000 7f8719e00640 1 --2- 172.21.15.39:0/1323632907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f871c072ff0 0x7f871c14e860 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f870c034040 tx=0x7f870c0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:15.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.452+0000 7f8712000640 1 -- 172.21.15.39:0/1323632907 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870c03e030 con 0x7f871c072ff0 2024-09-21T07:33:15.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.452+0000 7f8712000640 1 -- 172.21.15.39:0/1323632907 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f870c003ae0 con 0x7f871c072ff0 2024-09-21T07:33:15.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.452+0000 7f8720c00640 1 -- 172.21.15.39:0/1323632907 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f871c1f9470 con 0x7f871c072ff0 2024-09-21T07:33:15.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.452+0000 7f8712000640 1 -- 172.21.15.39:0/1323632907 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870c035040 con 0x7f871c072ff0 2024-09-21T07:33:15.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.452+0000 7f8720c00640 1 -- 172.21.15.39:0/1323632907 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f871c1f9a30 con 0x7f871c072ff0 2024-09-21T07:33:15.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.453+0000 7f8720c00640 1 -- 172.21.15.39:0/1323632907 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f871c1f9c70 con 0x7f871c072ff0 2024-09-21T07:33:15.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.454+0000 7f8712000640 1 -- 172.21.15.39:0/1323632907 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f870c041030 con 0x7f871c072ff0 2024-09-21T07:33:15.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.455+0000 7f8712000640 1 --2- 172.21.15.39:0/1323632907 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f86f8078b90 0x7f86f807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:15.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.455+0000 7f8712000640 1 -- 172.21.15.39:0/1323632907 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f870c0bb670 con 0x7f871c072ff0 2024-09-21T07:33:15.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.455+0000 7f8713e00640 1 --2- 172.21.15.39:0/1323632907 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f86f8078b90 0x7f86f807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:15.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.456+0000 7f8713e00640 1 --2- 172.21.15.39:0/1323632907 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f86f8078b90 0x7f86f807b0a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f871c15d9b0 tx=0x7f8704009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:15.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.460+0000 7f8712000640 1 -- 172.21.15.39:0/1323632907 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f871c1f9c70 con 0x7f871c072ff0 2024-09-21T07:33:15.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.658+0000 7f8720c00640 1 -- 172.21.15.39:0/1323632907 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f871c073400 con 0x7f871c072ff0 2024-09-21T07:33:15.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.659+0000 7f8712000640 1 -- 172.21.15.39:0/1323632907 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16678 (secure 0 0 0) 0x7f870c0833c0 con 0x7f871c072ff0 2024-09-21T07:33:15.662 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:33:15.662 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":20,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:33:14.674603+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:33:12.854113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:33:13.622611+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:33:15.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.662+0000 7f8720c00640 1 -- 172.21.15.39:0/1323632907 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f86f8078b90 msgr2=0x7f86f807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:15.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.662+0000 7f8720c00640 1 --2- 172.21.15.39:0/1323632907 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f86f8078b90 0x7f86f807b0a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f871c15d9b0 tx=0x7f8704009040 comp rx=0 tx=0).stop 2024-09-21T07:33:15.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.662+0000 7f8720c00640 1 -- 172.21.15.39:0/1323632907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f871c072ff0 msgr2=0x7f871c14e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:15.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.662+0000 7f8720c00640 1 --2- 172.21.15.39:0/1323632907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f871c072ff0 0x7f871c14e860 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f870c034040 tx=0x7f870c0035f0 comp rx=0 tx=0).stop 2024-09-21T07:33:15.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.662+0000 7f8720c00640 1 -- 172.21.15.39:0/1323632907 shutdown_connections 2024-09-21T07:33:15.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.662+0000 7f8720c00640 1 --2- 172.21.15.39:0/1323632907 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f86f8078b90 0x7f86f807b0a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:15.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.662+0000 7f8720c00640 1 --2- 172.21.15.39:0/1323632907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f871c0cd3e0 0x7f871c1e93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:15.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.662+0000 7f8720c00640 1 --2- 172.21.15.39:0/1323632907 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f871c0af780 0x7f871c1e8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:15.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.662+0000 7f8720c00640 1 --2- 172.21.15.39:0/1323632907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f871c072ff0 0x7f871c14e860 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:15.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.662+0000 7f8720c00640 1 -- 172.21.15.39:0/1323632907 >> 172.21.15.39:0/1323632907 conn(0x7f871c06d7d0 msgr2=0x7f871c0b00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:15.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.662+0000 7f8720c00640 1 -- 172.21.15.39:0/1323632907 shutdown_connections 2024-09-21T07:33:15.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.662+0000 7f8720c00640 1 -- 172.21.15.39:0/1323632907 wait complete. 2024-09-21T07:33:15.688 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:33:15.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.913+0000 7fef7aa00640 1 -- 172.21.15.39:0/2788948301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef5c003a50 msgr2=0x7fef5c003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:15.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.913+0000 7fef7aa00640 1 --2- 172.21.15.39:0/2788948301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef5c003a50 0x7fef5c003ec0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fef680016c0 tx=0x7fef5c005830 comp rx=0 tx=0).stop 2024-09-21T07:33:15.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.913+0000 7fef7aa00640 1 -- 172.21.15.39:0/2788948301 shutdown_connections 2024-09-21T07:33:15.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.913+0000 7fef7aa00640 1 --2- 172.21.15.39:0/2788948301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef5c003a50 0x7fef5c003ec0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:15.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.913+0000 7fef7aa00640 1 -- 172.21.15.39:0/2788948301 >> 172.21.15.39:0/2788948301 conn(0x7fef7406d410 msgr2=0x7fef740b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:15.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.913+0000 7fef7aa00640 1 -- 172.21.15.39:0/2788948301 shutdown_connections 2024-09-21T07:33:15.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.913+0000 7fef7aa00640 1 -- 172.21.15.39:0/2788948301 wait complete. 2024-09-21T07:33:15.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.915+0000 7fef7aa00640 1 Processor -- start 2024-09-21T07:33:15.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.915+0000 7fef7aa00640 1 -- start start 2024-09-21T07:33:15.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.917+0000 7fef7aa00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fef5c003a50 0x7fef74159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:15.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.917+0000 7fef7aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef7413c7d0 0x7fef74153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:15.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.917+0000 7fef7aa00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef74154020 0x7fef74142240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:15.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.917+0000 7fef7aa00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fef740ccbc0 con 0x7fef7413c7d0 2024-09-21T07:33:15.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.917+0000 7fef7aa00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fef740cca40 con 0x7fef74154020 2024-09-21T07:33:15.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.917+0000 7fef7aa00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fef740ccd40 con 0x7fef5c003a50 2024-09-21T07:33:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.917+0000 7fef78c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef7413c7d0 0x7fef74153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.917+0000 7fef78c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef7413c7d0 0x7fef74153ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60428/0 (socket says 172.21.15.39:60428) 2024-09-21T07:33:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.917+0000 7fef78c00640 1 -- 172.21.15.39:0/1111436469 learned_addr learned my addr 172.21.15.39:0/1111436469 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.917+0000 7fef79600640 1 --2- 172.21.15.39:0/1111436469 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fef5c003a50 0x7fef74159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.917+0000 7fef7a000640 1 --2- 172.21.15.39:0/1111436469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef74154020 0x7fef74142240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.918+0000 7fef78c00640 1 -- 172.21.15.39:0/1111436469 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fef5c003a50 msgr2=0x7fef74159340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.918+0000 7fef78c00640 1 --2- 172.21.15.39:0/1111436469 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fef5c003a50 0x7fef74159340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.918+0000 7fef78c00640 1 -- 172.21.15.39:0/1111436469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef74154020 msgr2=0x7fef74142240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.918+0000 7fef78c00640 1 --2- 172.21.15.39:0/1111436469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef74154020 0x7fef74142240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.918+0000 7fef78c00640 1 -- 172.21.15.39:0/1111436469 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef74158b80 con 0x7fef7413c7d0 2024-09-21T07:33:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.918+0000 7fef7a000640 1 --2- 172.21.15.39:0/1111436469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef74154020 0x7fef74142240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:33:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.918+0000 7fef78c00640 1 --2- 172.21.15.39:0/1111436469 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef7413c7d0 0x7fef74153ae0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fef6000e590 tx=0x7fef6000ea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.918+0000 7fef71600640 1 -- 172.21.15.39:0/1111436469 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef60015070 con 0x7fef7413c7d0 2024-09-21T07:33:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.918+0000 7fef71600640 1 -- 172.21.15.39:0/1111436469 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef6000caa0 con 0x7fef7413c7d0 2024-09-21T07:33:15.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.918+0000 7fef71600640 1 -- 172.21.15.39:0/1111436469 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef60011460 con 0x7fef7413c7d0 2024-09-21T07:33:15.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.918+0000 7fef7aa00640 1 -- 172.21.15.39:0/1111436469 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef741f2f70 con 0x7fef7413c7d0 2024-09-21T07:33:15.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.918+0000 7fef7aa00640 1 -- 172.21.15.39:0/1111436469 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef74156640 con 0x7fef7413c7d0 2024-09-21T07:33:15.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.919+0000 7fef7aa00640 1 -- 172.21.15.39:0/1111436469 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef741436c0 con 0x7fef7413c7d0 2024-09-21T07:33:15.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.921+0000 7fef71600640 1 -- 172.21.15.39:0/1111436469 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fef60018070 con 0x7fef7413c7d0 2024-09-21T07:33:15.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.922+0000 7fef71600640 1 --2- 172.21.15.39:0/1111436469 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fef54078b90 0x7fef5407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:15.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.922+0000 7fef71600640 1 -- 172.21.15.39:0/1111436469 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fef6009b070 con 0x7fef7413c7d0 2024-09-21T07:33:15.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.922+0000 7fef79600640 1 --2- 172.21.15.39:0/1111436469 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fef54078b90 0x7fef5407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:15.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.923+0000 7fef79600640 1 --2- 172.21.15.39:0/1111436469 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fef54078b90 0x7fef5407b0a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fef68032000 tx=0x7fef68001190 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:15.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:15.927+0000 7fef71600640 1 -- 172.21.15.39:0/1111436469 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fef60062e50 con 0x7fef7413c7d0 2024-09-21T07:33:16.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.125+0000 7fef7aa00640 1 -- 172.21.15.39:0/1111436469 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fef7406f0c0 con 0x7fef7413c7d0 2024-09-21T07:33:16.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.126+0000 7fef71600640 1 -- 172.21.15.39:0/1111436469 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16678 (secure 0 0 0) 0x7fef60067880 con 0x7fef7413c7d0 2024-09-21T07:33:16.128 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:33:16.128 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":20,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:33:14.674603+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:33:12.854113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:33:13.622611+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:33:16.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.128+0000 7fef7aa00640 1 -- 172.21.15.39:0/1111436469 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fef54078b90 msgr2=0x7fef5407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:16.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.128+0000 7fef7aa00640 1 --2- 172.21.15.39:0/1111436469 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fef54078b90 0x7fef5407b0a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fef68032000 tx=0x7fef68001190 comp rx=0 tx=0).stop 2024-09-21T07:33:16.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.128+0000 7fef7aa00640 1 -- 172.21.15.39:0/1111436469 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef7413c7d0 msgr2=0x7fef74153ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:16.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.128+0000 7fef7aa00640 1 --2- 172.21.15.39:0/1111436469 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef7413c7d0 0x7fef74153ae0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fef6000e590 tx=0x7fef6000ea50 comp rx=0 tx=0).stop 2024-09-21T07:33:16.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.128+0000 7fef7aa00640 1 -- 172.21.15.39:0/1111436469 shutdown_connections 2024-09-21T07:33:16.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.129+0000 7fef7aa00640 1 --2- 172.21.15.39:0/1111436469 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fef54078b90 0x7fef5407b0a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:16.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.129+0000 7fef7aa00640 1 --2- 172.21.15.39:0/1111436469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef74154020 0x7fef74142240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:16.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.129+0000 7fef7aa00640 1 --2- 172.21.15.39:0/1111436469 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef7413c7d0 0x7fef74153ae0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:16.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.129+0000 7fef7aa00640 1 --2- 172.21.15.39:0/1111436469 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fef5c003a50 0x7fef74159340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:16.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.129+0000 7fef7aa00640 1 -- 172.21.15.39:0/1111436469 >> 172.21.15.39:0/1111436469 conn(0x7fef7406d410 msgr2=0x7fef740b1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:16.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.129+0000 7fef7aa00640 1 -- 172.21.15.39:0/1111436469 shutdown_connections 2024-09-21T07:33:16.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.129+0000 7fef7aa00640 1 -- 172.21.15.39:0/1111436469 wait complete. 2024-09-21T07:33:16.153 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T07:33:16.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.385+0000 7fe310200640 1 -- 172.21.15.39:0/2093452685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3080b5fa0 msgr2=0x7fe3080b63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:16.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.385+0000 7fe310200640 1 --2- 172.21.15.39:0/2093452685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3080b5fa0 0x7fe3080b63b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe30808d960 tx=0x7fe30002aac0 comp rx=0 tx=0).stop 2024-09-21T07:33:16.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.385+0000 7fe310200640 1 -- 172.21.15.39:0/2093452685 shutdown_connections 2024-09-21T07:33:16.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.385+0000 7fe310200640 1 --2- 172.21.15.39:0/2093452685 >> v2:172.21.15.148:3300/0 conn(0x7fe3080b9cf0 0x7fe3080d31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:16.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.385+0000 7fe310200640 1 --2- 172.21.15.39:0/2093452685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3080b5fa0 0x7fe3080b63b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:16.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.385+0000 7fe310200640 1 -- 172.21.15.39:0/2093452685 >> 172.21.15.39:0/2093452685 conn(0x7fe30809d830 msgr2=0x7fe3080d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:16.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.385+0000 7fe310200640 1 -- 172.21.15.39:0/2093452685 shutdown_connections 2024-09-21T07:33:16.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.385+0000 7fe310200640 1 -- 172.21.15.39:0/2093452685 wait complete. 2024-09-21T07:33:16.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.387+0000 7fe310200640 1 Processor -- start 2024-09-21T07:33:16.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.387+0000 7fe310200640 1 -- start start 2024-09-21T07:33:16.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.388+0000 7fe310200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3080b5fa0 0x7fe3081f5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:16.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.388+0000 7fe310200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3080b9cf0 0x7fe3080a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:16.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.389+0000 7fe310200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3080d4fd0 0x7fe3080c9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:16.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.389+0000 7fe310200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe30806e230 con 0x7fe3080b5fa0 2024-09-21T07:33:16.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.389+0000 7fe310200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe30806e0b0 con 0x7fe3080b9cf0 2024-09-21T07:33:16.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.389+0000 7fe310200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe30806e3b0 con 0x7fe3080d4fd0 2024-09-21T07:33:16.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.389+0000 7fe30d800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3080b5fa0 0x7fe3081f5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:16.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.389+0000 7fe30d800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3080b5fa0 0x7fe3081f5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60452/0 (socket says 172.21.15.39:60452) 2024-09-21T07:33:16.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.389+0000 7fe30d800640 1 -- 172.21.15.39:0/34038638 learned_addr learned my addr 172.21.15.39:0/34038638 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:16.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.389+0000 7fe30ce00640 1 --2- 172.21.15.39:0/34038638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3080b9cf0 0x7fe3080a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:16.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.389+0000 7fe30e200640 1 --2- 172.21.15.39:0/34038638 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3080d4fd0 0x7fe3080c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:16.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.389+0000 7fe30d800640 1 -- 172.21.15.39:0/34038638 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3080d4fd0 msgr2=0x7fe3080c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:16.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.389+0000 7fe30d800640 1 --2- 172.21.15.39:0/34038638 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3080d4fd0 0x7fe3080c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:16.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.389+0000 7fe30d800640 1 -- 172.21.15.39:0/34038638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3080b9cf0 msgr2=0x7fe3080a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:16.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.389+0000 7fe30d800640 1 --2- 172.21.15.39:0/34038638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3080b9cf0 0x7fe3080a1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:16.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.389+0000 7fe30d800640 1 -- 172.21.15.39:0/34038638 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3080c82a0 con 0x7fe3080b5fa0 2024-09-21T07:33:16.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.390+0000 7fe30d800640 1 --2- 172.21.15.39:0/34038638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3080b5fa0 0x7fe3081f5540 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc00c390 tx=0x7fe2fc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:16.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.390+0000 7fe305600640 1 -- 172.21.15.39:0/34038638 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc017070 con 0x7fe3080b5fa0 2024-09-21T07:33:16.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.390+0000 7fe305600640 1 -- 172.21.15.39:0/34038638 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe2fc0053c0 con 0x7fe3080b5fa0 2024-09-21T07:33:16.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.390+0000 7fe305600640 1 -- 172.21.15.39:0/34038638 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc003880 con 0x7fe3080b5fa0 2024-09-21T07:33:16.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.390+0000 7fe310200640 1 -- 172.21.15.39:0/34038638 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3080c8580 con 0x7fe3080b5fa0 2024-09-21T07:33:16.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.390+0000 7fe310200640 1 -- 172.21.15.39:0/34038638 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3080c8b40 con 0x7fe3080b5fa0 2024-09-21T07:33:16.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.391+0000 7fe310200640 1 -- 172.21.15.39:0/34038638 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe30806e860 con 0x7fe3080b5fa0 2024-09-21T07:33:16.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.392+0000 7fe305600640 1 -- 172.21.15.39:0/34038638 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe2fc01a070 con 0x7fe3080b5fa0 2024-09-21T07:33:16.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.393+0000 7fe305600640 1 --2- 172.21.15.39:0/34038638 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe2e8078b90 0x7fe2e807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:16.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.393+0000 7fe305600640 1 -- 172.21.15.39:0/34038638 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe2fc09a0a0 con 0x7fe3080b5fa0 2024-09-21T07:33:16.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.394+0000 7fe30ce00640 1 --2- 172.21.15.39:0/34038638 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe2e8078b90 0x7fe2e807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:16.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.394+0000 7fe30ce00640 1 --2- 172.21.15.39:0/34038638 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe2e8078b90 0x7fe2e807b0a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe30815f1f0 tx=0x7fe2f4005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:16.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.398+0000 7fe305600640 1 -- 172.21.15.39:0/34038638 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe3080b68f0 con 0x7fe3080b5fa0 2024-09-21T07:33:16.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:16.636+0000 7fe310200640 1 -- 172.21.15.39:0/34038638 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe3080b7480 con 0x7fe3080b5fa0 2024-09-21T07:33:17.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.178+0000 7fe305600640 1 -- 172.21.15.39:0/34038638 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7fe2fc061e80 con 0x7fe3080b5fa0 2024-09-21T07:33:17.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.182+0000 7fe310200640 1 -- 172.21.15.39:0/34038638 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe2e8078b90 msgr2=0x7fe2e807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:17.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.182+0000 7fe310200640 1 --2- 172.21.15.39:0/34038638 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe2e8078b90 0x7fe2e807b0a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe30815f1f0 tx=0x7fe2f4005f90 comp rx=0 tx=0).stop 2024-09-21T07:33:17.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.182+0000 7fe310200640 1 -- 172.21.15.39:0/34038638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3080b5fa0 msgr2=0x7fe3081f5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:17.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.182+0000 7fe310200640 1 --2- 172.21.15.39:0/34038638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3080b5fa0 0x7fe3081f5540 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc00c390 tx=0x7fe2fc00c850 comp rx=0 tx=0).stop 2024-09-21T07:33:17.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.182+0000 7fe310200640 1 -- 172.21.15.39:0/34038638 shutdown_connections 2024-09-21T07:33:17.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.182+0000 7fe310200640 1 --2- 172.21.15.39:0/34038638 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe2e8078b90 0x7fe2e807b0a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:17.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.182+0000 7fe310200640 1 --2- 172.21.15.39:0/34038638 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3080d4fd0 0x7fe3080c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:17.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.182+0000 7fe310200640 1 --2- 172.21.15.39:0/34038638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3080b9cf0 0x7fe3080a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:17.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.182+0000 7fe310200640 1 --2- 172.21.15.39:0/34038638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3080b5fa0 0x7fe3081f5540 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:17.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.182+0000 7fe310200640 1 -- 172.21.15.39:0/34038638 >> 172.21.15.39:0/34038638 conn(0x7fe30809d830 msgr2=0x7fe3080b79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:17.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.182+0000 7fe310200640 1 -- 172.21.15.39:0/34038638 shutdown_connections 2024-09-21T07:33:17.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.182+0000 7fe310200640 1 -- 172.21.15.39:0/34038638 wait complete. 2024-09-21T07:33:17.209 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T07:33:17.209 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T07:33:17.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.439+0000 7f7eb5a00640 1 -- 172.21.15.39:0/2792741514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e94004800 msgr2=0x7f7e94004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:17.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.439+0000 7f7eb5a00640 1 --2- 172.21.15.39:0/2792741514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e94004800 0x7f7e94004c10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7e98009b50 tx=0x7f7e94005830 comp rx=0 tx=0).stop 2024-09-21T07:33:17.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.439+0000 7f7eb5000640 1 -- 172.21.15.39:0/2792741514 reap_dead start 2024-09-21T07:33:17.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.439+0000 7f7eb5a00640 1 -- 172.21.15.39:0/2792741514 shutdown_connections 2024-09-21T07:33:17.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.439+0000 7f7eb5a00640 1 -- 172.21.15.39:0/2792741514 >> 172.21.15.39:0/2792741514 conn(0x7f7eb00b3be0 msgr2=0x7f7eb00b3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:17.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.439+0000 7f7eb5a00640 1 -- 172.21.15.39:0/2792741514 shutdown_connections 2024-09-21T07:33:17.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.439+0000 7f7eb5a00640 1 -- 172.21.15.39:0/2792741514 wait complete. 2024-09-21T07:33:17.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.441+0000 7f7eb5a00640 1 Processor -- start 2024-09-21T07:33:17.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.442+0000 7f7eb5a00640 1 -- start start 2024-09-21T07:33:17.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.443+0000 7f7eb5a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7e94004800 0x7f7eb01f68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:17.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.443+0000 7f7eb5a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7eb015cee0 0x7f7eb015d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:17.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.443+0000 7f7eb5a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7eb0071de0 0x7f7eb00721f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:17.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.443+0000 7f7eb5a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb00b4a90 con 0x7f7eb0071de0 2024-09-21T07:33:17.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.443+0000 7f7eb5a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb00b4910 con 0x7f7eb015cee0 2024-09-21T07:33:17.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.443+0000 7f7eb5a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f7eb00d06d0 con 0x7f7e94004800 2024-09-21T07:33:17.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.443+0000 7f7eb5000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7eb0071de0 0x7f7eb00721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:17.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.443+0000 7f7eb5000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7eb0071de0 0x7f7eb00721f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60462/0 (socket says 172.21.15.39:60462) 2024-09-21T07:33:17.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.443+0000 7f7eb5000640 1 -- 172.21.15.39:0/2585951802 learned_addr learned my addr 172.21.15.39:0/2585951802 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:17.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.443+0000 7f7eafe00640 1 --2- 172.21.15.39:0/2585951802 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7e94004800 0x7f7eb01f68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:17.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.443+0000 7f7ea7400640 1 --2- 172.21.15.39:0/2585951802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7eb015cee0 0x7f7eb015d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:17.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.444+0000 7f7eb5000640 1 -- 172.21.15.39:0/2585951802 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7e94004800 msgr2=0x7f7eb01f68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:17.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.444+0000 7f7eb5000640 1 --2- 172.21.15.39:0/2585951802 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7e94004800 0x7f7eb01f68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:17.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.444+0000 7f7eb5000640 1 -- 172.21.15.39:0/2585951802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7eb015cee0 msgr2=0x7f7eb015d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:17.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.444+0000 7f7eb5000640 1 --2- 172.21.15.39:0/2585951802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7eb015cee0 0x7f7eb015d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:17.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.444+0000 7f7eb5000640 1 -- 172.21.15.39:0/2585951802 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb0072730 con 0x7f7eb0071de0 2024-09-21T07:33:17.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.444+0000 7f7eb5000640 1 --2- 172.21.15.39:0/2585951802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7eb0071de0 0x7f7eb00721f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7eb00cc6f0 tx=0x7f7ea8006720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:17.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.444+0000 7f7eacc00640 1 -- 172.21.15.39:0/2585951802 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea800fa80 con 0x7f7eb0071de0 2024-09-21T07:33:17.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.444+0000 7f7eacc00640 1 -- 172.21.15.39:0/2585951802 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ea800fc20 con 0x7f7eb0071de0 2024-09-21T07:33:17.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.444+0000 7f7eacc00640 1 -- 172.21.15.39:0/2585951802 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea8011430 con 0x7f7eb0071de0 2024-09-21T07:33:17.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.444+0000 7f7eb5a00640 1 -- 172.21.15.39:0/2585951802 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eb01f2d60 con 0x7f7eb0071de0 2024-09-21T07:33:17.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.444+0000 7f7eb5a00640 1 -- 172.21.15.39:0/2585951802 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eb01f3320 con 0x7f7eb0071de0 2024-09-21T07:33:17.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.445+0000 7f7eb5a00640 1 -- 172.21.15.39:0/2585951802 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eb00b4f40 con 0x7f7eb0071de0 2024-09-21T07:33:17.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.447+0000 7f7eacc00640 1 -- 172.21.15.39:0/2585951802 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7ea80115d0 con 0x7f7eb0071de0 2024-09-21T07:33:17.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.447+0000 7f7eacc00640 1 --2- 172.21.15.39:0/2585951802 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7e8c078bc0 0x7f7e8c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:17.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.448+0000 7f7eacc00640 1 -- 172.21.15.39:0/2585951802 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7ea809b5e0 con 0x7f7eb0071de0 2024-09-21T07:33:17.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.448+0000 7f7eafe00640 1 --2- 172.21.15.39:0/2585951802 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7e8c078bc0 0x7f7e8c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:17.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.449+0000 7f7eafe00640 1 --2- 172.21.15.39:0/2585951802 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7e8c078bc0 0x7f7e8c07b0d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7ea00058e0 tx=0x7f7ea0005f30 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:17.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.452+0000 7f7eacc00640 1 -- 172.21.15.39:0/2585951802 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7eb0072730 con 0x7f7eb0071de0 2024-09-21T07:33:17.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:17.686+0000 7f7eb5a00640 1 -- 172.21.15.39:0/2585951802 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f7eb00ce8e0 con 0x7f7eb0071de0 2024-09-21T07:33:18.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.187+0000 7f7eacc00640 1 -- 172.21.15.39:0/2585951802 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7f7ea80633c0 con 0x7f7eb0071de0 2024-09-21T07:33:18.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.190+0000 7f7eb5a00640 1 -- 172.21.15.39:0/2585951802 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7e8c078bc0 msgr2=0x7f7e8c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:18.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.190+0000 7f7eb5a00640 1 --2- 172.21.15.39:0/2585951802 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7e8c078bc0 0x7f7e8c07b0d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7ea00058e0 tx=0x7f7ea0005f30 comp rx=0 tx=0).stop 2024-09-21T07:33:18.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.190+0000 7f7eb5a00640 1 -- 172.21.15.39:0/2585951802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7eb0071de0 msgr2=0x7f7eb00721f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:18.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.190+0000 7f7eb5a00640 1 --2- 172.21.15.39:0/2585951802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7eb0071de0 0x7f7eb00721f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7eb00cc6f0 tx=0x7f7ea8006720 comp rx=0 tx=0).stop 2024-09-21T07:33:18.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.190+0000 7f7eb5a00640 1 -- 172.21.15.39:0/2585951802 shutdown_connections 2024-09-21T07:33:18.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.190+0000 7f7eb5a00640 1 --2- 172.21.15.39:0/2585951802 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7e8c078bc0 0x7f7e8c07b0d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:18.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.190+0000 7f7eb5a00640 1 --2- 172.21.15.39:0/2585951802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7eb0071de0 0x7f7eb00721f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:18.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.190+0000 7f7eb5a00640 1 --2- 172.21.15.39:0/2585951802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7eb015cee0 0x7f7eb015d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:18.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.190+0000 7f7eb5a00640 1 --2- 172.21.15.39:0/2585951802 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7e94004800 0x7f7eb01f68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:18.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.190+0000 7f7eb5a00640 1 -- 172.21.15.39:0/2585951802 >> 172.21.15.39:0/2585951802 conn(0x7f7eb00b3be0 msgr2=0x7f7eb00b6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:18.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.190+0000 7f7eb5a00640 1 -- 172.21.15.39:0/2585951802 shutdown_connections 2024-09-21T07:33:18.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.190+0000 7f7eb5a00640 1 -- 172.21.15.39:0/2585951802 wait complete. 2024-09-21T07:33:18.215 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T07:33:18.215 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:33:18.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.440+0000 7f7bfb600640 1 -- 172.21.15.39:0/685729268 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bf40706b0 msgr2=0x7f7bf40735b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:18.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.440+0000 7f7bfb600640 1 --2- 172.21.15.39:0/685729268 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bf40706b0 0x7f7bf40735b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7bec005fd0 tx=0x7f7bec02aa90 comp rx=0 tx=0).stop 2024-09-21T07:33:18.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.440+0000 7f7bfb600640 1 -- 172.21.15.39:0/685729268 shutdown_connections 2024-09-21T07:33:18.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.440+0000 7f7bfb600640 1 --2- 172.21.15.39:0/685729268 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bf40706b0 0x7f7bf40735b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:18.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.440+0000 7f7bfb600640 1 --2- 172.21.15.39:0/685729268 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7bf40ca800 0x7f7bf40cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:18.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.440+0000 7f7bfb600640 1 -- 172.21.15.39:0/685729268 >> 172.21.15.39:0/685729268 conn(0x7f7bf40b1cb0 msgr2=0x7f7bf40b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:18.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.440+0000 7f7bfb600640 1 -- 172.21.15.39:0/685729268 shutdown_connections 2024-09-21T07:33:18.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.440+0000 7f7bfb600640 1 -- 172.21.15.39:0/685729268 wait complete. 2024-09-21T07:33:18.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.442+0000 7f7bfb600640 1 Processor -- start 2024-09-21T07:33:18.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.442+0000 7f7bfb600640 1 -- start start 2024-09-21T07:33:18.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.444+0000 7f7bfb600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bf40ca800 0x7f7bf415b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:18.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.444+0000 7f7bfb600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bf40cad30 0x7f7bf41e87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:18.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.444+0000 7f7bfb600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7bf41e8d20 0x7f7bf415daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:18.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.444+0000 7f7bfb600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf4073f60 con 0x7f7bf40cad30 2024-09-21T07:33:18.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.444+0000 7f7bfb600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf4073de0 con 0x7f7bf40ca800 2024-09-21T07:33:18.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.444+0000 7f7bfb600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f7bf40740e0 con 0x7f7bf41e8d20 2024-09-21T07:33:18.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.444+0000 7f7bebe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bf40cad30 0x7f7bf41e87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:18.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.444+0000 7f7bebe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bf40cad30 0x7f7bf41e87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60480/0 (socket says 172.21.15.39:60480) 2024-09-21T07:33:18.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.444+0000 7f7bebe00640 1 -- 172.21.15.39:0/4143894111 learned_addr learned my addr 172.21.15.39:0/4143894111 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:18.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.444+0000 7f7bf8c00640 1 --2- 172.21.15.39:0/4143894111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bf40ca800 0x7f7bf415b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:18.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.444+0000 7f7bf9600640 1 --2- 172.21.15.39:0/4143894111 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7bf41e8d20 0x7f7bf415daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:18.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.445+0000 7f7bebe00640 1 -- 172.21.15.39:0/4143894111 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7bf41e8d20 msgr2=0x7f7bf415daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:18.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.445+0000 7f7bebe00640 1 --2- 172.21.15.39:0/4143894111 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7bf41e8d20 0x7f7bf415daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:18.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.445+0000 7f7bebe00640 1 -- 172.21.15.39:0/4143894111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bf40ca800 msgr2=0x7f7bf415b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:18.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.445+0000 7f7bebe00640 1 --2- 172.21.15.39:0/4143894111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bf40ca800 0x7f7bf415b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:18.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.445+0000 7f7bebe00640 1 -- 172.21.15.39:0/4143894111 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf40c8860 con 0x7f7bf40cad30 2024-09-21T07:33:18.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.445+0000 7f7bebe00640 1 --2- 172.21.15.39:0/4143894111 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bf40cad30 0x7f7bf41e87e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00c3c0 tx=0x7f7bdc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:18.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.445+0000 7f7bf1600640 1 -- 172.21.15.39:0/4143894111 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc017070 con 0x7f7bf40cad30 2024-09-21T07:33:18.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.445+0000 7f7bf1600640 1 -- 172.21.15.39:0/4143894111 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7bdc0035f0 con 0x7f7bf40cad30 2024-09-21T07:33:18.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.445+0000 7f7bf1600640 1 -- 172.21.15.39:0/4143894111 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc003a80 con 0x7f7bf40cad30 2024-09-21T07:33:18.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.445+0000 7f7bfb600640 1 -- 172.21.15.39:0/4143894111 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bf41f8890 con 0x7f7bf40cad30 2024-09-21T07:33:18.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.445+0000 7f7bfb600640 1 -- 172.21.15.39:0/4143894111 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bf41f8e50 con 0x7f7bf40cad30 2024-09-21T07:33:18.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.446+0000 7f7bfb600640 1 -- 172.21.15.39:0/4143894111 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bf4074600 con 0x7f7bf40cad30 2024-09-21T07:33:18.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.448+0000 7f7bf1600640 1 -- 172.21.15.39:0/4143894111 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7bdc01a070 con 0x7f7bf40cad30 2024-09-21T07:33:18.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.449+0000 7f7bf1600640 1 --2- 172.21.15.39:0/4143894111 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7bd0078b90 0x7f7bd007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:18.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.449+0000 7f7bf1600640 1 -- 172.21.15.39:0/4143894111 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7bdc099f20 con 0x7f7bf40cad30 2024-09-21T07:33:18.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.449+0000 7f7bf8c00640 1 --2- 172.21.15.39:0/4143894111 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7bd0078b90 0x7f7bd007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:18.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.450+0000 7f7bf8c00640 1 --2- 172.21.15.39:0/4143894111 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7bd0078b90 0x7f7bd007b0a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7bf40cb250 tx=0x7f7be4005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:18.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.453+0000 7f7bf1600640 1 -- 172.21.15.39:0/4143894111 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7bdc061d90 con 0x7f7bf40cad30 2024-09-21T07:33:18.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.687+0000 7f7bfb600640 1 -- 172.21.15.39:0/4143894111 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7bf40b1700 con 0x7f7bf40cad30 2024-09-21T07:33:18.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.688+0000 7f7bf1600640 1 -- 172.21.15.39:0/4143894111 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5220 (secure 0 0 0) 0x7f7bdc0667c0 con 0x7f7bf40cad30 2024-09-21T07:33:18.690 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:33:18.690 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":8,"btime":"2024-09-21T07:33:18:179932+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4325,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1939850940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1939850940},{"type":"v1","addr":"172.21.15.39:6835","nonce":1939850940}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4328,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3712021428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3712021428},{"type":"v1","addr":"172.21.15.148:6836","nonce":3712021428}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4331,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3708353709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3708353709},{"type":"v1","addr":"172.21.15.148:6837","nonce":3708353709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:33:14.674552+0000","modified":"2024-09-21T07:33:17.688290+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4378},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4378":{"gid":4378,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6833/414851924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":414851924},{"type":"v1","addr":"172.21.15.39:6833","nonce":414851924}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4378,"qdb_cluster":[4378]},"id":1}]} 2024-09-21T07:33:18.690 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 8 2024-09-21T07:33:18.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.690+0000 7f7bfb600640 1 -- 172.21.15.39:0/4143894111 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7bd0078b90 msgr2=0x7f7bd007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:18.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.690+0000 7f7bfb600640 1 --2- 172.21.15.39:0/4143894111 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7bd0078b90 0x7f7bd007b0a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7bf40cb250 tx=0x7f7be4005f90 comp rx=0 tx=0).stop 2024-09-21T07:33:18.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.690+0000 7f7bfb600640 1 -- 172.21.15.39:0/4143894111 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bf40cad30 msgr2=0x7f7bf41e87e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:18.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.690+0000 7f7bfb600640 1 --2- 172.21.15.39:0/4143894111 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bf40cad30 0x7f7bf41e87e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00c3c0 tx=0x7f7bdc00c880 comp rx=0 tx=0).stop 2024-09-21T07:33:18.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.690+0000 7f7bfb600640 1 -- 172.21.15.39:0/4143894111 shutdown_connections 2024-09-21T07:33:18.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.690+0000 7f7bfb600640 1 --2- 172.21.15.39:0/4143894111 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7bd0078b90 0x7f7bd007b0a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:18.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.690+0000 7f7bfb600640 1 --2- 172.21.15.39:0/4143894111 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7bf41e8d20 0x7f7bf415daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:18.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.690+0000 7f7bfb600640 1 --2- 172.21.15.39:0/4143894111 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bf40cad30 0x7f7bf41e87e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:18.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.690+0000 7f7bfb600640 1 --2- 172.21.15.39:0/4143894111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bf40ca800 0x7f7bf415b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:18.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.690+0000 7f7bfb600640 1 -- 172.21.15.39:0/4143894111 >> 172.21.15.39:0/4143894111 conn(0x7f7bf40b1cb0 msgr2=0x7f7bf40b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:18.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.690+0000 7f7bfb600640 1 -- 172.21.15.39:0/4143894111 shutdown_connections 2024-09-21T07:33:18.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.690+0000 7f7bfb600640 1 -- 172.21.15.39:0/4143894111 wait complete. 2024-09-21T07:33:18.713 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:33:18.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.936+0000 7f25d1e00640 1 -- 172.21.15.39:0/1733917698 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25cc0cc1e0 msgr2=0x7f25cc0af3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:18.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.936+0000 7f25d1e00640 1 --2- 172.21.15.39:0/1733917698 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25cc0cc1e0 0x7f25cc0af3c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f25cc0cd710 tx=0x7f25bc02a560 comp rx=0 tx=0).stop 2024-09-21T07:33:18.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.936+0000 7f25d1e00640 1 -- 172.21.15.39:0/1733917698 shutdown_connections 2024-09-21T07:33:18.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.936+0000 7f25d1e00640 1 --2- 172.21.15.39:0/1733917698 >> v2:172.21.15.148:3300/0 conn(0x7f25cc0ccb50 0x7f25cc0ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:18.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.936+0000 7f25d1e00640 1 --2- 172.21.15.39:0/1733917698 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f25cc15d8a0 0x7f25cc0b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:18.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.936+0000 7f25d1e00640 1 --2- 172.21.15.39:0/1733917698 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25cc0cc1e0 0x7f25cc0af3c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:18.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.936+0000 7f25d1e00640 1 -- 172.21.15.39:0/1733917698 >> 172.21.15.39:0/1733917698 conn(0x7f25cc06d7d0 msgr2=0x7f25cc0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:18.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.936+0000 7f25d1e00640 1 -- 172.21.15.39:0/1733917698 shutdown_connections 2024-09-21T07:33:18.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.937+0000 7f25d1e00640 1 -- 172.21.15.39:0/1733917698 wait complete. 2024-09-21T07:33:18.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.939+0000 7f25d1e00640 1 Processor -- start 2024-09-21T07:33:18.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.939+0000 7f25d1e00640 1 -- start start 2024-09-21T07:33:18.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.940+0000 7f25d1e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25cc0cc1e0 0x7f25cc1f9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:18.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.940+0000 7f25d1e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25cc0ccb50 0x7f25cc155c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:18.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.940+0000 7f25d1e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f25cc15d8a0 0x7f25cc09aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:18.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.940+0000 7f25d1e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f25cc0b4770 con 0x7f25cc0cc1e0 2024-09-21T07:33:18.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.940+0000 7f25d1e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f25cc0b45f0 con 0x7f25cc0ccb50 2024-09-21T07:33:18.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.940+0000 7f25d1e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f25cc1533a0 con 0x7f25cc15d8a0 2024-09-21T07:33:18.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.941+0000 7f25caa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25cc0cc1e0 0x7f25cc1f9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:18.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.941+0000 7f25caa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25cc0cc1e0 0x7f25cc1f9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60500/0 (socket says 172.21.15.39:60500) 2024-09-21T07:33:18.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.941+0000 7f25caa00640 1 -- 172.21.15.39:0/3414808999 learned_addr learned my addr 172.21.15.39:0/3414808999 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:18.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.941+0000 7f25cb400640 1 --2- 172.21.15.39:0/3414808999 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f25cc15d8a0 0x7f25cc09aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:18.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.941+0000 7f25ca000640 1 --2- 172.21.15.39:0/3414808999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25cc0ccb50 0x7f25cc155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:18.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.941+0000 7f25caa00640 1 -- 172.21.15.39:0/3414808999 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f25cc15d8a0 msgr2=0x7f25cc09aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:18.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.941+0000 7f25caa00640 1 --2- 172.21.15.39:0/3414808999 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f25cc15d8a0 0x7f25cc09aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:18.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.941+0000 7f25caa00640 1 -- 172.21.15.39:0/3414808999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25cc0ccb50 msgr2=0x7f25cc155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:18.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.941+0000 7f25caa00640 1 --2- 172.21.15.39:0/3414808999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25cc0ccb50 0x7f25cc155c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:18.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.941+0000 7f25caa00640 1 -- 172.21.15.39:0/3414808999 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25cc09d0f0 con 0x7f25cc0cc1e0 2024-09-21T07:33:18.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.941+0000 7f25cb400640 1 --2- 172.21.15.39:0/3414808999 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f25cc15d8a0 0x7f25cc09aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:33:18.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.941+0000 7f25ca000640 1 --2- 172.21.15.39:0/3414808999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25cc0ccb50 0x7f25cc155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:33:18.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.941+0000 7f25caa00640 1 --2- 172.21.15.39:0/3414808999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25cc0cc1e0 0x7f25cc1f9930 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f25bc0099c0 tx=0x7f25bc0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:18.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.942+0000 7f25c3400640 1 -- 172.21.15.39:0/3414808999 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25bc03e030 con 0x7f25cc0cc1e0 2024-09-21T07:33:18.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.942+0000 7f25c3400640 1 -- 172.21.15.39:0/3414808999 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25bc003db0 con 0x7f25cc0cc1e0 2024-09-21T07:33:18.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.942+0000 7f25c3400640 1 -- 172.21.15.39:0/3414808999 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25bc035040 con 0x7f25cc0cc1e0 2024-09-21T07:33:18.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.942+0000 7f25d1e00640 1 -- 172.21.15.39:0/3414808999 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25cc09d3d0 con 0x7f25cc0cc1e0 2024-09-21T07:33:18.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.942+0000 7f25d1e00640 1 -- 172.21.15.39:0/3414808999 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25cc09d990 con 0x7f25cc0cc1e0 2024-09-21T07:33:18.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.943+0000 7f25d1e00640 1 -- 172.21.15.39:0/3414808999 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25cc0ccf60 con 0x7f25cc0cc1e0 2024-09-21T07:33:18.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.944+0000 7f25c3400640 1 -- 172.21.15.39:0/3414808999 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f25bc041030 con 0x7f25cc0cc1e0 2024-09-21T07:33:18.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.945+0000 7f25c3400640 1 --2- 172.21.15.39:0/3414808999 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f25ac078b90 0x7f25ac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:18.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.945+0000 7f25c3400640 1 -- 172.21.15.39:0/3414808999 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f25bc0bb7b0 con 0x7f25cc0cc1e0 2024-09-21T07:33:18.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.946+0000 7f25ca000640 1 --2- 172.21.15.39:0/3414808999 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f25ac078b90 0x7f25ac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:18.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.946+0000 7f25ca000640 1 --2- 172.21.15.39:0/3414808999 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f25ac078b90 0x7f25ac07b0a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f25cc0c9420 tx=0x7f25b8005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:18.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:18.950+0000 7f25c3400640 1 -- 172.21.15.39:0/3414808999 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f25bc083be0 con 0x7f25cc0cc1e0 2024-09-21T07:33:19.123 INFO:tasks.ceph.mds.a.smithi039.stderr:2024-09-21T07:33:19.121+0000 7fb55e200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:33:19.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.150+0000 7f25d1e00640 1 -- 172.21.15.39:0/3414808999 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f25cc0cdac0 con 0x7f25cc0cc1e0 2024-09-21T07:33:19.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.151+0000 7f25c3400640 1 -- 172.21.15.39:0/3414808999 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16665 (secure 0 0 0) 0x7f25bc0834e0 con 0x7f25cc0cc1e0 2024-09-21T07:33:19.153 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:33:19.154 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":21,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:33:16.159267+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:33:12.854113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:33:13.622611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:33:19.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.154+0000 7f25d1e00640 1 -- 172.21.15.39:0/3414808999 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f25ac078b90 msgr2=0x7f25ac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:19.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.154+0000 7f25d1e00640 1 --2- 172.21.15.39:0/3414808999 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f25ac078b90 0x7f25ac07b0a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f25cc0c9420 tx=0x7f25b8005f90 comp rx=0 tx=0).stop 2024-09-21T07:33:19.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.154+0000 7f25d1e00640 1 -- 172.21.15.39:0/3414808999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25cc0cc1e0 msgr2=0x7f25cc1f9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:19.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.154+0000 7f25d1e00640 1 --2- 172.21.15.39:0/3414808999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25cc0cc1e0 0x7f25cc1f9930 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f25bc0099c0 tx=0x7f25bc0038a0 comp rx=0 tx=0).stop 2024-09-21T07:33:19.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.154+0000 7f25d1e00640 1 -- 172.21.15.39:0/3414808999 shutdown_connections 2024-09-21T07:33:19.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.154+0000 7f25d1e00640 1 --2- 172.21.15.39:0/3414808999 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f25ac078b90 0x7f25ac07b0a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:19.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.154+0000 7f25d1e00640 1 --2- 172.21.15.39:0/3414808999 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f25cc15d8a0 0x7f25cc09aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:19.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.154+0000 7f25d1e00640 1 --2- 172.21.15.39:0/3414808999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25cc0ccb50 0x7f25cc155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:19.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.154+0000 7f25d1e00640 1 --2- 172.21.15.39:0/3414808999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25cc0cc1e0 0x7f25cc1f9930 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:19.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.154+0000 7f25d1e00640 1 -- 172.21.15.39:0/3414808999 >> 172.21.15.39:0/3414808999 conn(0x7f25cc06d7d0 msgr2=0x7f25cc0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:19.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.154+0000 7f25d1e00640 1 -- 172.21.15.39:0/3414808999 shutdown_connections 2024-09-21T07:33:19.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.154+0000 7f25d1e00640 1 -- 172.21.15.39:0/3414808999 wait complete. 2024-09-21T07:33:19.180 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T07:33:19.181 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:33:19.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.406+0000 7f92cfe00640 1 -- 172.21.15.39:0/967624638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92d00b5470 msgr2=0x7f92d00b9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:19.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.406+0000 7f92cfe00640 1 --2- 172.21.15.39:0/967624638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92d00b5470 0x7f92d00b9100 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f92c0009830 tx=0x7f92c002a510 comp rx=0 tx=0).stop 2024-09-21T07:33:19.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.406+0000 7f92cfe00640 1 -- 172.21.15.39:0/967624638 shutdown_connections 2024-09-21T07:33:19.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.406+0000 7f92cfe00640 1 --2- 172.21.15.39:0/967624638 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92d00b9640 0x7f92d00d2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:19.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.406+0000 7f92cfe00640 1 --2- 172.21.15.39:0/967624638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92d00b5470 0x7f92d00b9100 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:19.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.406+0000 7f92cfe00640 1 -- 172.21.15.39:0/967624638 >> 172.21.15.39:0/967624638 conn(0x7f92d009d6a0 msgr2=0x7f92d00d4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:19.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.406+0000 7f92cfe00640 1 -- 172.21.15.39:0/967624638 shutdown_connections 2024-09-21T07:33:19.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.406+0000 7f92cfe00640 1 -- 172.21.15.39:0/967624638 wait complete. 2024-09-21T07:33:19.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.408+0000 7f92cfe00640 1 Processor -- start 2024-09-21T07:33:19.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.409+0000 7f92cfe00640 1 -- start start 2024-09-21T07:33:19.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.410+0000 7f92cfe00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92d00b5470 0x7f92d01513c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:19.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.410+0000 7f92cfe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92d00b9640 0x7f92d01f5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:19.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.410+0000 7f92cfe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92d00d2650 0x7f92d00cd670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:19.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.410+0000 7f92cfe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f92d00b4160 con 0x7f92d00d2650 2024-09-21T07:33:19.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.410+0000 7f92cfe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f92d00b8730 con 0x7f92d00b9640 2024-09-21T07:33:19.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.410+0000 7f92cfe00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f92d00b42e0 con 0x7f92d00b5470 2024-09-21T07:33:19.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.411+0000 7f92cf400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92d00d2650 0x7f92d00cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:19.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.411+0000 7f92c7e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92d00b9640 0x7f92d01f5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:19.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.411+0000 7f92cea00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92d00b5470 0x7f92d01513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:19.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.411+0000 7f92c7e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92d00b9640 0x7f92d01f5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.39:51608/0 (socket says 172.21.15.39:51608) 2024-09-21T07:33:19.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.411+0000 7f92cf400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92d00d2650 0x7f92d00cd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60514/0 (socket says 172.21.15.39:60514) 2024-09-21T07:33:19.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.411+0000 7f92cea00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92d00b5470 0x7f92d01513c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.39:60814/0 (socket says 172.21.15.39:60814) 2024-09-21T07:33:19.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.411+0000 7f92c7e00640 1 -- 172.21.15.39:0/1677974758 learned_addr learned my addr 172.21.15.39:0/1677974758 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:19.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.411+0000 7f92c7e00640 1 -- 172.21.15.39:0/1677974758 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92d00b5470 msgr2=0x7f92d01513c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:19.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.411+0000 7f92c7e00640 1 --2- 172.21.15.39:0/1677974758 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92d00b5470 0x7f92d01513c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:19.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.411+0000 7f92c7e00640 1 -- 172.21.15.39:0/1677974758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92d00d2650 msgr2=0x7f92d00cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:19.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.411+0000 7f92c7e00640 1 --2- 172.21.15.39:0/1677974758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92d00d2650 0x7f92d00cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:19.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.411+0000 7f92c7e00640 1 -- 172.21.15.39:0/1677974758 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92d01f6040 con 0x7f92d00b9640 2024-09-21T07:33:19.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.411+0000 7f92cf400640 1 --2- 172.21.15.39:0/1677974758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92d00d2650 0x7f92d00cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:33:19.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.411+0000 7f92c7e00640 1 --2- 172.21.15.39:0/1677974758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92d00b9640 0x7f92d01f5b00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f92b800ea20 tx=0x7f92b800eee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:19.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.412+0000 7f92c6a00640 1 -- 172.21.15.39:0/1677974758 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b800ce40 con 0x7f92d00b9640 2024-09-21T07:33:19.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.412+0000 7f92c6a00640 1 -- 172.21.15.39:0/1677974758 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f92b8004020 con 0x7f92d00b9640 2024-09-21T07:33:19.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.412+0000 7f92cfe00640 1 -- 172.21.15.39:0/1677974758 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92d00cc5d0 con 0x7f92d00b9640 2024-09-21T07:33:19.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.412+0000 7f92c6a00640 1 -- 172.21.15.39:0/1677974758 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b8004470 con 0x7f92d00b9640 2024-09-21T07:33:19.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.412+0000 7f92cfe00640 1 -- 172.21.15.39:0/1677974758 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92d00ccb90 con 0x7f92d00b9640 2024-09-21T07:33:19.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.413+0000 7f92cfe00640 1 -- 172.21.15.39:0/1677974758 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92d00b4800 con 0x7f92d00b9640 2024-09-21T07:33:19.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.414+0000 7f92c6a00640 1 -- 172.21.15.39:0/1677974758 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f92b803bd10 con 0x7f92d00b9640 2024-09-21T07:33:19.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.415+0000 7f92c6a00640 1 --2- 172.21.15.39:0/1677974758 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f92ac078b90 0x7f92ac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:19.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.415+0000 7f92c6a00640 1 -- 172.21.15.39:0/1677974758 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f92b809b6c0 con 0x7f92d00b9640 2024-09-21T07:33:19.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.416+0000 7f92cea00640 1 --2- 172.21.15.39:0/1677974758 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f92ac078b90 0x7f92ac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:19.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.417+0000 7f92cea00640 1 --2- 172.21.15.39:0/1677974758 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f92ac078b90 0x7f92ac07b0a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f92c0034000 tx=0x7f92c0004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:19.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.419+0000 7f92c6a00640 1 -- 172.21.15.39:0/1677974758 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f92b8063620 con 0x7f92d00b9640 2024-09-21T07:33:19.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.615+0000 7f92cfe00640 1 -- 172.21.15.39:0/1677974758 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f92d0160e90 con 0x7f92ac078b90 2024-09-21T07:33:19.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.621+0000 7f92c6a00640 1 -- 172.21.15.39:0/1677974758 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251940 (secure 0 0 0) 0x7f92d0160e90 con 0x7f92ac078b90 2024-09-21T07:33:19.626 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:33:19.627 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:33:19.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.625+0000 7f92cfe00640 1 -- 172.21.15.39:0/1677974758 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f92ac078b90 msgr2=0x7f92ac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:19.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.625+0000 7f92cfe00640 1 --2- 172.21.15.39:0/1677974758 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f92ac078b90 0x7f92ac07b0a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f92c0034000 tx=0x7f92c0004730 comp rx=0 tx=0).stop 2024-09-21T07:33:19.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.625+0000 7f92cfe00640 1 -- 172.21.15.39:0/1677974758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92d00b9640 msgr2=0x7f92d01f5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:19.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.625+0000 7f92cfe00640 1 --2- 172.21.15.39:0/1677974758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92d00b9640 0x7f92d01f5b00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f92b800ea20 tx=0x7f92b800eee0 comp rx=0 tx=0).stop 2024-09-21T07:33:19.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.625+0000 7f92cfe00640 1 -- 172.21.15.39:0/1677974758 shutdown_connections 2024-09-21T07:33:19.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.625+0000 7f92cfe00640 1 --2- 172.21.15.39:0/1677974758 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f92ac078b90 0x7f92ac07b0a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:19.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.625+0000 7f92cfe00640 1 --2- 172.21.15.39:0/1677974758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92d00d2650 0x7f92d00cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:19.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.625+0000 7f92cfe00640 1 --2- 172.21.15.39:0/1677974758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92d00b9640 0x7f92d01f5b00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:19.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.625+0000 7f92cfe00640 1 --2- 172.21.15.39:0/1677974758 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92d00b5470 0x7f92d01513c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:19.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.625+0000 7f92cfe00640 1 -- 172.21.15.39:0/1677974758 >> 172.21.15.39:0/1677974758 conn(0x7f92d009d6a0 msgr2=0x7f92d00b2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:19.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.626+0000 7f92cfe00640 1 -- 172.21.15.39:0/1677974758 shutdown_connections 2024-09-21T07:33:19.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.626+0000 7f92cfe00640 1 -- 172.21.15.39:0/1677974758 wait complete. 2024-09-21T07:33:19.648 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-09-21T07:33:19.533815+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":166,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235708,"kb_used_data":2832,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739012,"statfs":{"total":773094113280,"available":772852748288,"internally_reserved":0,"allocated":2899968,"data_stored":1750452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001523"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274756+0000","last_change":"2024-09-21T07:33:15.176955+0000","last_active":"2024-09-21T07:33:16.274756+0000","last_peered":"2024-09-21T07:33:16.274756+0000","last_clean":"2024-09-21T07:33:16.274756+0000","last_became_active":"2024-09-21T07:33:15.176632+0000","last_became_peered":"2024-09-21T07:33:15.176632+0000","last_unstale":"2024-09-21T07:33:16.274756+0000","last_undegraded":"2024-09-21T07:33:16.274756+0000","last_fullsized":"2024-09-21T07:33:16.274756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:10:51.829331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171764+0000","last_change":"2024-09-21T07:33:15.184382+0000","last_active":"2024-09-21T07:33:16.171764+0000","last_peered":"2024-09-21T07:33:16.171764+0000","last_clean":"2024-09-21T07:33:16.171764+0000","last_became_active":"2024-09-21T07:33:15.184072+0000","last_became_peered":"2024-09-21T07:33:15.184072+0000","last_unstale":"2024-09-21T07:33:16.171764+0000","last_undegraded":"2024-09-21T07:33:16.171764+0000","last_fullsized":"2024-09-21T07:33:16.171764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:08.677357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273864+0000","last_change":"2024-09-21T07:33:15.186660+0000","last_active":"2024-09-21T07:33:16.273864+0000","last_peered":"2024-09-21T07:33:16.273864+0000","last_clean":"2024-09-21T07:33:16.273864+0000","last_became_active":"2024-09-21T07:33:15.185832+0000","last_became_peered":"2024-09-21T07:33:15.185832+0000","last_unstale":"2024-09-21T07:33:16.273864+0000","last_undegraded":"2024-09-21T07:33:16.273864+0000","last_fullsized":"2024-09-21T07:33:16.273864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:08.677357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.170390+0000","last_change":"2024-09-21T07:33:15.193591+0000","last_active":"2024-09-21T07:33:16.170390+0000","last_peered":"2024-09-21T07:33:16.170390+0000","last_clean":"2024-09-21T07:33:16.170390+0000","last_became_active":"2024-09-21T07:33:15.191327+0000","last_became_peered":"2024-09-21T07:33:15.191327+0000","last_unstale":"2024-09-21T07:33:16.170390+0000","last_undegraded":"2024-09-21T07:33:16.170390+0000","last_fullsized":"2024-09-21T07:33:16.170390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:00:15.919300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272836+0000","last_change":"2024-09-21T07:33:15.187754+0000","last_active":"2024-09-21T07:33:16.272836+0000","last_peered":"2024-09-21T07:33:16.272836+0000","last_clean":"2024-09-21T07:33:16.272836+0000","last_became_active":"2024-09-21T07:33:15.187381+0000","last_became_peered":"2024-09-21T07:33:15.187381+0000","last_unstale":"2024-09-21T07:33:16.272836+0000","last_undegraded":"2024-09-21T07:33:16.272836+0000","last_fullsized":"2024-09-21T07:33:16.272836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:19:59.491735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.169414+0000","last_change":"2024-09-21T07:33:15.180176+0000","last_active":"2024-09-21T07:33:16.169414+0000","last_peered":"2024-09-21T07:33:16.169414+0000","last_clean":"2024-09-21T07:33:16.169414+0000","last_became_active":"2024-09-21T07:33:15.179828+0000","last_became_peered":"2024-09-21T07:33:15.179828+0000","last_unstale":"2024-09-21T07:33:16.169414+0000","last_undegraded":"2024-09-21T07:33:16.169414+0000","last_fullsized":"2024-09-21T07:33:16.169414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:40:52.015841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274813+0000","last_change":"2024-09-21T07:33:15.183358+0000","last_active":"2024-09-21T07:33:16.274813+0000","last_peered":"2024-09-21T07:33:16.274813+0000","last_clean":"2024-09-21T07:33:16.274813+0000","last_became_active":"2024-09-21T07:33:15.182969+0000","last_became_peered":"2024-09-21T07:33:15.182969+0000","last_unstale":"2024-09-21T07:33:16.274813+0000","last_undegraded":"2024-09-21T07:33:16.274813+0000","last_fullsized":"2024-09-21T07:33:16.274813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:41:44.441220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272174+0000","last_change":"2024-09-21T07:33:15.173445+0000","last_active":"2024-09-21T07:33:16.272174+0000","last_peered":"2024-09-21T07:33:16.272174+0000","last_clean":"2024-09-21T07:33:16.272174+0000","last_became_active":"2024-09-21T07:33:15.172833+0000","last_became_peered":"2024-09-21T07:33:15.172833+0000","last_unstale":"2024-09-21T07:33:16.272174+0000","last_undegraded":"2024-09-21T07:33:16.272174+0000","last_fullsized":"2024-09-21T07:33:16.272174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:24:46.139397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274983+0000","last_change":"2024-09-21T07:33:15.194854+0000","last_active":"2024-09-21T07:33:16.274983+0000","last_peered":"2024-09-21T07:33:16.274983+0000","last_clean":"2024-09-21T07:33:16.274983+0000","last_became_active":"2024-09-21T07:33:15.194423+0000","last_became_peered":"2024-09-21T07:33:15.194423+0000","last_unstale":"2024-09-21T07:33:16.274983+0000","last_undegraded":"2024-09-21T07:33:16.274983+0000","last_fullsized":"2024-09-21T07:33:16.274983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:14.196020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274097+0000","last_change":"2024-09-21T07:33:15.176651+0000","last_active":"2024-09-21T07:33:16.274097+0000","last_peered":"2024-09-21T07:33:16.274097+0000","last_clean":"2024-09-21T07:33:16.274097+0000","last_became_active":"2024-09-21T07:33:15.176222+0000","last_became_peered":"2024-09-21T07:33:15.176222+0000","last_unstale":"2024-09-21T07:33:16.274097+0000","last_undegraded":"2024-09-21T07:33:16.274097+0000","last_fullsized":"2024-09-21T07:33:16.274097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:47.494973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274249+0000","last_change":"2024-09-21T07:33:15.170250+0000","last_active":"2024-09-21T07:33:16.274249+0000","last_peered":"2024-09-21T07:33:16.274249+0000","last_clean":"2024-09-21T07:33:16.274249+0000","last_became_active":"2024-09-21T07:33:15.166192+0000","last_became_peered":"2024-09-21T07:33:15.166192+0000","last_unstale":"2024-09-21T07:33:16.274249+0000","last_undegraded":"2024-09-21T07:33:16.274249+0000","last_fullsized":"2024-09-21T07:33:16.274249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:48:05.009446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274541+0000","last_change":"2024-09-21T07:33:15.180001+0000","last_active":"2024-09-21T07:33:16.274541+0000","last_peered":"2024-09-21T07:33:16.274541+0000","last_clean":"2024-09-21T07:33:16.274541+0000","last_became_active":"2024-09-21T07:33:15.179679+0000","last_became_peered":"2024-09-21T07:33:15.179679+0000","last_unstale":"2024-09-21T07:33:16.274541+0000","last_undegraded":"2024-09-21T07:33:16.274541+0000","last_fullsized":"2024-09-21T07:33:16.274541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:05:31.417897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272616+0000","last_change":"2024-09-21T07:33:15.179831+0000","last_active":"2024-09-21T07:33:16.272616+0000","last_peered":"2024-09-21T07:33:16.272616+0000","last_clean":"2024-09-21T07:33:16.272616+0000","last_became_active":"2024-09-21T07:33:15.179382+0000","last_became_peered":"2024-09-21T07:33:15.179382+0000","last_unstale":"2024-09-21T07:33:16.272616+0000","last_undegraded":"2024-09-21T07:33:16.272616+0000","last_fullsized":"2024-09-21T07:33:16.272616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:26:20.407200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272116+0000","last_change":"2024-09-21T07:33:15.174462+0000","last_active":"2024-09-21T07:33:16.272116+0000","last_peered":"2024-09-21T07:33:16.272116+0000","last_clean":"2024-09-21T07:33:16.272116+0000","last_became_active":"2024-09-21T07:33:15.174135+0000","last_became_peered":"2024-09-21T07:33:15.174135+0000","last_unstale":"2024-09-21T07:33:16.272116+0000","last_undegraded":"2024-09-21T07:33:16.272116+0000","last_fullsized":"2024-09-21T07:33:16.272116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:53.282103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171346+0000","last_change":"2024-09-21T07:33:15.184280+0000","last_active":"2024-09-21T07:33:16.171346+0000","last_peered":"2024-09-21T07:33:16.171346+0000","last_clean":"2024-09-21T07:33:16.171346+0000","last_became_active":"2024-09-21T07:33:15.183935+0000","last_became_peered":"2024-09-21T07:33:15.183935+0000","last_unstale":"2024-09-21T07:33:16.171346+0000","last_undegraded":"2024-09-21T07:33:16.171346+0000","last_fullsized":"2024-09-21T07:33:16.171346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:17:23.362898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.169177+0000","last_change":"2024-09-21T07:33:15.178501+0000","last_active":"2024-09-21T07:33:16.169177+0000","last_peered":"2024-09-21T07:33:16.169177+0000","last_clean":"2024-09-21T07:33:16.169177+0000","last_became_active":"2024-09-21T07:33:15.178157+0000","last_became_peered":"2024-09-21T07:33:15.178157+0000","last_unstale":"2024-09-21T07:33:16.169177+0000","last_undegraded":"2024-09-21T07:33:16.169177+0000","last_fullsized":"2024-09-21T07:33:16.169177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:47.494973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.170990+0000","last_change":"2024-09-21T07:33:15.194890+0000","last_active":"2024-09-21T07:33:16.170990+0000","last_peered":"2024-09-21T07:33:16.170990+0000","last_clean":"2024-09-21T07:33:16.170990+0000","last_became_active":"2024-09-21T07:33:15.194618+0000","last_became_peered":"2024-09-21T07:33:15.194618+0000","last_unstale":"2024-09-21T07:33:16.170990+0000","last_undegraded":"2024-09-21T07:33:16.170990+0000","last_fullsized":"2024-09-21T07:33:16.170990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:42:25.647850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.173528+0000","last_change":"2024-09-21T07:33:15.177265+0000","last_active":"2024-09-21T07:33:16.173528+0000","last_peered":"2024-09-21T07:33:16.173528+0000","last_clean":"2024-09-21T07:33:16.173528+0000","last_became_active":"2024-09-21T07:33:15.176835+0000","last_became_peered":"2024-09-21T07:33:15.176835+0000","last_unstale":"2024-09-21T07:33:16.173528+0000","last_undegraded":"2024-09-21T07:33:16.173528+0000","last_fullsized":"2024-09-21T07:33:16.173528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:26:27.046080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273611+0000","last_change":"2024-09-21T07:33:15.187455+0000","last_active":"2024-09-21T07:33:16.273611+0000","last_peered":"2024-09-21T07:33:16.273611+0000","last_clean":"2024-09-21T07:33:16.273611+0000","last_became_active":"2024-09-21T07:33:15.187097+0000","last_became_peered":"2024-09-21T07:33:15.187097+0000","last_unstale":"2024-09-21T07:33:16.273611+0000","last_undegraded":"2024-09-21T07:33:16.273611+0000","last_fullsized":"2024-09-21T07:33:16.273611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:12:27.933810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273575+0000","last_change":"2024-09-21T07:33:15.196207+0000","last_active":"2024-09-21T07:33:16.273575+0000","last_peered":"2024-09-21T07:33:16.273575+0000","last_clean":"2024-09-21T07:33:16.273575+0000","last_became_active":"2024-09-21T07:33:15.195889+0000","last_became_peered":"2024-09-21T07:33:15.195889+0000","last_unstale":"2024-09-21T07:33:16.273575+0000","last_undegraded":"2024-09-21T07:33:16.273575+0000","last_fullsized":"2024-09-21T07:33:16.273575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:23.420344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272226+0000","last_change":"2024-09-21T07:33:15.182205+0000","last_active":"2024-09-21T07:33:16.272226+0000","last_peered":"2024-09-21T07:33:16.272226+0000","last_clean":"2024-09-21T07:33:16.272226+0000","last_became_active":"2024-09-21T07:33:15.181790+0000","last_became_peered":"2024-09-21T07:33:15.181790+0000","last_unstale":"2024-09-21T07:33:16.272226+0000","last_undegraded":"2024-09-21T07:33:16.272226+0000","last_fullsized":"2024-09-21T07:33:16.272226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:09.053804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.169665+0000","last_change":"2024-09-21T07:33:15.172738+0000","last_active":"2024-09-21T07:33:16.169665+0000","last_peered":"2024-09-21T07:33:16.169665+0000","last_clean":"2024-09-21T07:33:16.169665+0000","last_became_active":"2024-09-21T07:33:15.172454+0000","last_became_peered":"2024-09-21T07:33:15.172454+0000","last_unstale":"2024-09-21T07:33:16.169665+0000","last_undegraded":"2024-09-21T07:33:16.169665+0000","last_fullsized":"2024-09-21T07:33:16.169665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:26:46.580701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272525+0000","last_change":"2024-09-21T07:33:15.185561+0000","last_active":"2024-09-21T07:33:16.272525+0000","last_peered":"2024-09-21T07:33:16.272525+0000","last_clean":"2024-09-21T07:33:16.272525+0000","last_became_active":"2024-09-21T07:33:15.185004+0000","last_became_peered":"2024-09-21T07:33:15.185004+0000","last_unstale":"2024-09-21T07:33:16.272525+0000","last_undegraded":"2024-09-21T07:33:16.272525+0000","last_fullsized":"2024-09-21T07:33:16.272525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:25:13.620607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.169268+0000","last_change":"2024-09-21T07:33:15.164795+0000","last_active":"2024-09-21T07:33:16.169268+0000","last_peered":"2024-09-21T07:33:16.169268+0000","last_clean":"2024-09-21T07:33:16.169268+0000","last_became_active":"2024-09-21T07:33:15.164419+0000","last_became_peered":"2024-09-21T07:33:15.164419+0000","last_unstale":"2024-09-21T07:33:16.169268+0000","last_undegraded":"2024-09-21T07:33:16.169268+0000","last_fullsized":"2024-09-21T07:33:16.169268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:48:05.009446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273570+0000","last_change":"2024-09-21T07:33:15.183825+0000","last_active":"2024-09-21T07:33:16.273570+0000","last_peered":"2024-09-21T07:33:16.273570+0000","last_clean":"2024-09-21T07:33:16.273570+0000","last_became_active":"2024-09-21T07:33:15.183503+0000","last_became_peered":"2024-09-21T07:33:15.183503+0000","last_unstale":"2024-09-21T07:33:16.273570+0000","last_undegraded":"2024-09-21T07:33:16.273570+0000","last_fullsized":"2024-09-21T07:33:16.273570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:12.378254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273927+0000","last_change":"2024-09-21T07:33:15.177070+0000","last_active":"2024-09-21T07:33:16.273927+0000","last_peered":"2024-09-21T07:33:16.273927+0000","last_clean":"2024-09-21T07:33:16.273927+0000","last_became_active":"2024-09-21T07:33:15.176706+0000","last_became_peered":"2024-09-21T07:33:15.176706+0000","last_unstale":"2024-09-21T07:33:16.273927+0000","last_undegraded":"2024-09-21T07:33:16.273927+0000","last_fullsized":"2024-09-21T07:33:16.273927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:40:41.309353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171642+0000","last_change":"2024-09-21T07:33:15.174783+0000","last_active":"2024-09-21T07:33:16.171642+0000","last_peered":"2024-09-21T07:33:16.171642+0000","last_clean":"2024-09-21T07:33:16.171642+0000","last_became_active":"2024-09-21T07:33:15.173581+0000","last_became_peered":"2024-09-21T07:33:15.173581+0000","last_unstale":"2024-09-21T07:33:16.171642+0000","last_undegraded":"2024-09-21T07:33:16.171642+0000","last_fullsized":"2024-09-21T07:33:16.171642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:43:34.665544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273063+0000","last_change":"2024-09-21T07:33:15.185421+0000","last_active":"2024-09-21T07:33:16.273063+0000","last_peered":"2024-09-21T07:33:16.273063+0000","last_clean":"2024-09-21T07:33:16.273063+0000","last_became_active":"2024-09-21T07:33:15.185131+0000","last_became_peered":"2024-09-21T07:33:15.185131+0000","last_unstale":"2024-09-21T07:33:16.273063+0000","last_undegraded":"2024-09-21T07:33:16.273063+0000","last_fullsized":"2024-09-21T07:33:16.273063+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:29.320339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273245+0000","last_change":"2024-09-21T07:33:15.178626+0000","last_active":"2024-09-21T07:33:16.273245+0000","last_peered":"2024-09-21T07:33:16.273245+0000","last_clean":"2024-09-21T07:33:16.273245+0000","last_became_active":"2024-09-21T07:33:15.178344+0000","last_became_peered":"2024-09-21T07:33:15.178344+0000","last_unstale":"2024-09-21T07:33:16.273245+0000","last_undegraded":"2024-09-21T07:33:16.273245+0000","last_fullsized":"2024-09-21T07:33:16.273245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:17:23.362898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.169955+0000","last_change":"2024-09-21T07:33:15.187835+0000","last_active":"2024-09-21T07:33:16.169955+0000","last_peered":"2024-09-21T07:33:16.169955+0000","last_clean":"2024-09-21T07:33:16.169955+0000","last_became_active":"2024-09-21T07:33:15.185993+0000","last_became_peered":"2024-09-21T07:33:15.185993+0000","last_unstale":"2024-09-21T07:33:16.169955+0000","last_undegraded":"2024-09-21T07:33:16.169955+0000","last_fullsized":"2024-09-21T07:33:16.169955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:17.480483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.173171+0000","last_change":"2024-09-21T07:33:15.163142+0000","last_active":"2024-09-21T07:33:16.173171+0000","last_peered":"2024-09-21T07:33:16.173171+0000","last_clean":"2024-09-21T07:33:16.173171+0000","last_became_active":"2024-09-21T07:33:15.162044+0000","last_became_peered":"2024-09-21T07:33:15.162044+0000","last_unstale":"2024-09-21T07:33:16.173171+0000","last_undegraded":"2024-09-21T07:33:16.173171+0000","last_fullsized":"2024-09-21T07:33:16.173171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:48:44.995481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272735+0000","last_change":"2024-09-21T07:33:15.188257+0000","last_active":"2024-09-21T07:33:16.272735+0000","last_peered":"2024-09-21T07:33:16.272735+0000","last_clean":"2024-09-21T07:33:16.272735+0000","last_became_active":"2024-09-21T07:33:15.184871+0000","last_became_peered":"2024-09-21T07:33:15.184871+0000","last_unstale":"2024-09-21T07:33:16.272735+0000","last_undegraded":"2024-09-21T07:33:16.272735+0000","last_fullsized":"2024-09-21T07:33:16.272735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:31:35.062821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274086+0000","last_change":"2024-09-21T07:33:15.178648+0000","last_active":"2024-09-21T07:33:16.274086+0000","last_peered":"2024-09-21T07:33:16.274086+0000","last_clean":"2024-09-21T07:33:16.274086+0000","last_became_active":"2024-09-21T07:33:15.178260+0000","last_became_peered":"2024-09-21T07:33:15.178260+0000","last_unstale":"2024-09-21T07:33:16.274086+0000","last_undegraded":"2024-09-21T07:33:16.274086+0000","last_fullsized":"2024-09-21T07:33:16.274086+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:36.198560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272837+0000","last_change":"2024-09-21T07:33:14.173209+0000","last_active":"2024-09-21T07:33:16.272837+0000","last_peered":"2024-09-21T07:33:16.272837+0000","last_clean":"2024-09-21T07:33:16.272837+0000","last_became_active":"2024-09-21T07:33:14.172936+0000","last_became_peered":"2024-09-21T07:33:14.172936+0000","last_unstale":"2024-09-21T07:33:16.272837+0000","last_undegraded":"2024-09-21T07:33:16.272837+0000","last_fullsized":"2024-09-21T07:33:16.272837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:03.284858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273336+0000","last_change":"2024-09-21T07:33:15.191981+0000","last_active":"2024-09-21T07:33:16.273336+0000","last_peered":"2024-09-21T07:33:16.273336+0000","last_clean":"2024-09-21T07:33:16.273336+0000","last_became_active":"2024-09-21T07:33:15.188976+0000","last_became_peered":"2024-09-21T07:33:15.188976+0000","last_unstale":"2024-09-21T07:33:16.273336+0000","last_undegraded":"2024-09-21T07:33:16.273336+0000","last_fullsized":"2024-09-21T07:33:16.273336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:34.842293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273636+0000","last_change":"2024-09-21T07:33:14.169344+0000","last_active":"2024-09-21T07:33:16.273636+0000","last_peered":"2024-09-21T07:33:16.273636+0000","last_clean":"2024-09-21T07:33:16.273636+0000","last_became_active":"2024-09-21T07:33:14.169015+0000","last_became_peered":"2024-09-21T07:33:14.169015+0000","last_unstale":"2024-09-21T07:33:16.273636+0000","last_undegraded":"2024-09-21T07:33:16.273636+0000","last_fullsized":"2024-09-21T07:33:16.273636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:10.925173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272616+0000","last_change":"2024-09-21T07:33:14.163956+0000","last_active":"2024-09-21T07:33:16.272616+0000","last_peered":"2024-09-21T07:33:16.272616+0000","last_clean":"2024-09-21T07:33:16.272616+0000","last_became_active":"2024-09-21T07:33:14.163698+0000","last_became_peered":"2024-09-21T07:33:14.163698+0000","last_unstale":"2024-09-21T07:33:16.272616+0000","last_undegraded":"2024-09-21T07:33:16.272616+0000","last_fullsized":"2024-09-21T07:33:16.272616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:32.172082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272523+0000","last_change":"2024-09-21T07:33:15.184364+0000","last_active":"2024-09-21T07:33:16.272523+0000","last_peered":"2024-09-21T07:33:16.272523+0000","last_clean":"2024-09-21T07:33:16.272523+0000","last_became_active":"2024-09-21T07:33:15.184042+0000","last_became_peered":"2024-09-21T07:33:15.184042+0000","last_unstale":"2024-09-21T07:33:16.272523+0000","last_undegraded":"2024-09-21T07:33:16.272523+0000","last_fullsized":"2024-09-21T07:33:16.272523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:39:05.189714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.169283+0000","last_change":"2024-09-21T07:33:14.160602+0000","last_active":"2024-09-21T07:33:16.169283+0000","last_peered":"2024-09-21T07:33:16.169283+0000","last_clean":"2024-09-21T07:33:16.169283+0000","last_became_active":"2024-09-21T07:33:14.159110+0000","last_became_peered":"2024-09-21T07:33:14.159110+0000","last_unstale":"2024-09-21T07:33:16.169283+0000","last_undegraded":"2024-09-21T07:33:16.169283+0000","last_fullsized":"2024-09-21T07:33:16.169283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:10.925173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171398+0000","last_change":"2024-09-21T07:33:15.178332+0000","last_active":"2024-09-21T07:33:16.171398+0000","last_peered":"2024-09-21T07:33:16.171398+0000","last_clean":"2024-09-21T07:33:16.171398+0000","last_became_active":"2024-09-21T07:33:15.177793+0000","last_became_peered":"2024-09-21T07:33:15.177793+0000","last_unstale":"2024-09-21T07:33:16.171398+0000","last_undegraded":"2024-09-21T07:33:16.171398+0000","last_fullsized":"2024-09-21T07:33:16.171398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:23.420344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273829+0000","last_change":"2024-09-21T07:33:14.171262+0000","last_active":"2024-09-21T07:33:16.273829+0000","last_peered":"2024-09-21T07:33:16.273829+0000","last_clean":"2024-09-21T07:33:16.273829+0000","last_became_active":"2024-09-21T07:33:14.170882+0000","last_became_peered":"2024-09-21T07:33:14.170882+0000","last_unstale":"2024-09-21T07:33:16.273829+0000","last_undegraded":"2024-09-21T07:33:16.273829+0000","last_fullsized":"2024-09-21T07:33:16.273829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:34.304149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.170713+0000","last_change":"2024-09-21T07:33:15.173639+0000","last_active":"2024-09-21T07:33:16.170713+0000","last_peered":"2024-09-21T07:33:16.170713+0000","last_clean":"2024-09-21T07:33:16.170713+0000","last_became_active":"2024-09-21T07:33:15.173275+0000","last_became_peered":"2024-09-21T07:33:15.173275+0000","last_unstale":"2024-09-21T07:33:16.170713+0000","last_undegraded":"2024-09-21T07:33:16.170713+0000","last_fullsized":"2024-09-21T07:33:16.170713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:12:30.901535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.170872+0000","last_change":"2024-09-21T07:33:14.170282+0000","last_active":"2024-09-21T07:33:16.170872+0000","last_peered":"2024-09-21T07:33:16.170872+0000","last_clean":"2024-09-21T07:33:16.170872+0000","last_became_active":"2024-09-21T07:33:14.169935+0000","last_became_peered":"2024-09-21T07:33:14.169935+0000","last_unstale":"2024-09-21T07:33:16.170872+0000","last_undegraded":"2024-09-21T07:33:16.170872+0000","last_fullsized":"2024-09-21T07:33:16.170872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:06.477956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273555+0000","last_change":"2024-09-21T07:33:15.171671+0000","last_active":"2024-09-21T07:33:16.273555+0000","last_peered":"2024-09-21T07:33:16.273555+0000","last_clean":"2024-09-21T07:33:16.273555+0000","last_became_active":"2024-09-21T07:33:15.171327+0000","last_became_peered":"2024-09-21T07:33:15.171327+0000","last_unstale":"2024-09-21T07:33:16.273555+0000","last_undegraded":"2024-09-21T07:33:16.273555+0000","last_fullsized":"2024-09-21T07:33:16.273555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:43:34.665544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171766+0000","last_change":"2024-09-21T07:33:14.166941+0000","last_active":"2024-09-21T07:33:16.171766+0000","last_peered":"2024-09-21T07:33:16.171766+0000","last_clean":"2024-09-21T07:33:16.171766+0000","last_became_active":"2024-09-21T07:33:14.165770+0000","last_became_peered":"2024-09-21T07:33:14.165770+0000","last_unstale":"2024-09-21T07:33:16.171766+0000","last_undegraded":"2024-09-21T07:33:16.171766+0000","last_fullsized":"2024-09-21T07:33:16.171766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:51.025109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274800+0000","last_change":"2024-09-21T07:33:15.181837+0000","last_active":"2024-09-21T07:33:16.274800+0000","last_peered":"2024-09-21T07:33:16.274800+0000","last_clean":"2024-09-21T07:33:16.274800+0000","last_became_active":"2024-09-21T07:33:15.181358+0000","last_became_peered":"2024-09-21T07:33:15.181358+0000","last_unstale":"2024-09-21T07:33:16.274800+0000","last_undegraded":"2024-09-21T07:33:16.274800+0000","last_fullsized":"2024-09-21T07:33:16.274800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:56.567663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273547+0000","last_change":"2024-09-21T07:33:14.172709+0000","last_active":"2024-09-21T07:33:16.273547+0000","last_peered":"2024-09-21T07:33:16.273547+0000","last_clean":"2024-09-21T07:33:16.273547+0000","last_became_active":"2024-09-21T07:33:14.172316+0000","last_became_peered":"2024-09-21T07:33:14.172316+0000","last_unstale":"2024-09-21T07:33:16.273547+0000","last_undegraded":"2024-09-21T07:33:16.273547+0000","last_fullsized":"2024-09-21T07:33:16.273547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:06.477956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272937+0000","last_change":"2024-09-21T07:33:15.179712+0000","last_active":"2024-09-21T07:33:16.272937+0000","last_peered":"2024-09-21T07:33:16.272937+0000","last_clean":"2024-09-21T07:33:16.272937+0000","last_became_active":"2024-09-21T07:33:15.179368+0000","last_became_peered":"2024-09-21T07:33:15.179368+0000","last_unstale":"2024-09-21T07:33:16.272937+0000","last_undegraded":"2024-09-21T07:33:16.272937+0000","last_fullsized":"2024-09-21T07:33:16.272937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:14.765726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273376+0000","last_change":"2024-09-21T07:33:14.180872+0000","last_active":"2024-09-21T07:33:16.273376+0000","last_peered":"2024-09-21T07:33:16.273376+0000","last_clean":"2024-09-21T07:33:16.273376+0000","last_became_active":"2024-09-21T07:33:14.179682+0000","last_became_peered":"2024-09-21T07:33:14.179682+0000","last_unstale":"2024-09-21T07:33:16.273376+0000","last_undegraded":"2024-09-21T07:33:16.273376+0000","last_fullsized":"2024-09-21T07:33:16.273376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:35:06.212701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272328+0000","last_change":"2024-09-21T07:33:15.177488+0000","last_active":"2024-09-21T07:33:16.272328+0000","last_peered":"2024-09-21T07:33:16.272328+0000","last_clean":"2024-09-21T07:33:16.272328+0000","last_became_active":"2024-09-21T07:33:15.176270+0000","last_became_peered":"2024-09-21T07:33:15.176270+0000","last_unstale":"2024-09-21T07:33:16.272328+0000","last_undegraded":"2024-09-21T07:33:16.272328+0000","last_fullsized":"2024-09-21T07:33:16.272328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:47.805013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.169798+0000","last_change":"2024-09-21T07:33:14.178744+0000","last_active":"2024-09-21T07:33:16.169798+0000","last_peered":"2024-09-21T07:33:16.169798+0000","last_clean":"2024-09-21T07:33:16.169798+0000","last_became_active":"2024-09-21T07:33:14.177786+0000","last_became_peered":"2024-09-21T07:33:14.177786+0000","last_unstale":"2024-09-21T07:33:16.169798+0000","last_undegraded":"2024-09-21T07:33:16.169798+0000","last_fullsized":"2024-09-21T07:33:16.169798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:34.304149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273092+0000","last_change":"2024-09-21T07:33:15.191036+0000","last_active":"2024-09-21T07:33:16.273092+0000","last_peered":"2024-09-21T07:33:16.273092+0000","last_clean":"2024-09-21T07:33:16.273092+0000","last_became_active":"2024-09-21T07:33:15.190631+0000","last_became_peered":"2024-09-21T07:33:15.190631+0000","last_unstale":"2024-09-21T07:33:16.273092+0000","last_undegraded":"2024-09-21T07:33:16.273092+0000","last_fullsized":"2024-09-21T07:33:16.273092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:39:17.752820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273048+0000","last_change":"2024-09-21T07:33:14.170962+0000","last_active":"2024-09-21T07:33:16.273048+0000","last_peered":"2024-09-21T07:33:16.273048+0000","last_clean":"2024-09-21T07:33:16.273048+0000","last_became_active":"2024-09-21T07:33:14.169996+0000","last_became_peered":"2024-09-21T07:33:14.169996+0000","last_unstale":"2024-09-21T07:33:16.273048+0000","last_undegraded":"2024-09-21T07:33:16.273048+0000","last_fullsized":"2024-09-21T07:33:16.273048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:45.280842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272830+0000","last_change":"2024-09-21T07:33:15.190018+0000","last_active":"2024-09-21T07:33:16.272830+0000","last_peered":"2024-09-21T07:33:16.272830+0000","last_clean":"2024-09-21T07:33:16.272830+0000","last_became_active":"2024-09-21T07:33:15.185361+0000","last_became_peered":"2024-09-21T07:33:15.185361+0000","last_unstale":"2024-09-21T07:33:16.272830+0000","last_undegraded":"2024-09-21T07:33:16.272830+0000","last_fullsized":"2024-09-21T07:33:16.272830+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:35.723678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.173087+0000","last_change":"2024-09-21T07:33:14.171085+0000","last_active":"2024-09-21T07:33:16.173087+0000","last_peered":"2024-09-21T07:33:16.173087+0000","last_clean":"2024-09-21T07:33:16.173087+0000","last_became_active":"2024-09-21T07:33:14.170838+0000","last_became_peered":"2024-09-21T07:33:14.170838+0000","last_unstale":"2024-09-21T07:33:16.173087+0000","last_undegraded":"2024-09-21T07:33:16.173087+0000","last_fullsized":"2024-09-21T07:33:16.173087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:06.477956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.275049+0000","last_change":"2024-09-21T07:33:15.181599+0000","last_active":"2024-09-21T07:33:16.275049+0000","last_peered":"2024-09-21T07:33:16.275049+0000","last_clean":"2024-09-21T07:33:16.275049+0000","last_became_active":"2024-09-21T07:33:15.181044+0000","last_became_peered":"2024-09-21T07:33:15.181044+0000","last_unstale":"2024-09-21T07:33:16.275049+0000","last_undegraded":"2024-09-21T07:33:16.275049+0000","last_fullsized":"2024-09-21T07:33:16.275049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:39.660131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273625+0000","last_change":"2024-09-21T07:33:14.180061+0000","last_active":"2024-09-21T07:33:16.273625+0000","last_peered":"2024-09-21T07:33:16.273625+0000","last_clean":"2024-09-21T07:33:16.273625+0000","last_became_active":"2024-09-21T07:33:14.179053+0000","last_became_peered":"2024-09-21T07:33:14.179053+0000","last_unstale":"2024-09-21T07:33:16.273625+0000","last_undegraded":"2024-09-21T07:33:16.273625+0000","last_fullsized":"2024-09-21T07:33:16.273625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:46.482047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274323+0000","last_change":"2024-09-21T07:33:15.187133+0000","last_active":"2024-09-21T07:33:16.274323+0000","last_peered":"2024-09-21T07:33:16.274323+0000","last_clean":"2024-09-21T07:33:16.274323+0000","last_became_active":"2024-09-21T07:33:15.186773+0000","last_became_peered":"2024-09-21T07:33:15.186773+0000","last_unstale":"2024-09-21T07:33:16.274323+0000","last_undegraded":"2024-09-21T07:33:16.274323+0000","last_fullsized":"2024-09-21T07:33:16.274323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:51.977863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.173649+0000","last_change":"2024-09-21T07:33:14.163096+0000","last_active":"2024-09-21T07:33:16.173649+0000","last_peered":"2024-09-21T07:33:16.173649+0000","last_clean":"2024-09-21T07:33:16.173649+0000","last_became_active":"2024-09-21T07:33:14.162512+0000","last_became_peered":"2024-09-21T07:33:14.162512+0000","last_unstale":"2024-09-21T07:33:16.173649+0000","last_undegraded":"2024-09-21T07:33:16.173649+0000","last_fullsized":"2024-09-21T07:33:16.173649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:10.925173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.275012+0000","last_change":"2024-09-21T07:33:15.184770+0000","last_active":"2024-09-21T07:33:16.275012+0000","last_peered":"2024-09-21T07:33:16.275012+0000","last_clean":"2024-09-21T07:33:16.275012+0000","last_became_active":"2024-09-21T07:33:15.184167+0000","last_became_peered":"2024-09-21T07:33:15.184167+0000","last_unstale":"2024-09-21T07:33:16.275012+0000","last_undegraded":"2024-09-21T07:33:16.275012+0000","last_fullsized":"2024-09-21T07:33:16.275012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:10.342501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171867+0000","last_change":"2024-09-21T07:33:14.170661+0000","last_active":"2024-09-21T07:33:16.171867+0000","last_peered":"2024-09-21T07:33:16.171867+0000","last_clean":"2024-09-21T07:33:16.171867+0000","last_became_active":"2024-09-21T07:33:14.170408+0000","last_became_peered":"2024-09-21T07:33:14.170408+0000","last_unstale":"2024-09-21T07:33:16.171867+0000","last_undegraded":"2024-09-21T07:33:16.171867+0000","last_fullsized":"2024-09-21T07:33:16.171867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:43:00.990750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.170680+0000","last_change":"2024-09-21T07:33:15.165673+0000","last_active":"2024-09-21T07:33:16.170680+0000","last_peered":"2024-09-21T07:33:16.170680+0000","last_clean":"2024-09-21T07:33:16.170680+0000","last_became_active":"2024-09-21T07:33:15.164392+0000","last_became_peered":"2024-09-21T07:33:15.164392+0000","last_unstale":"2024-09-21T07:33:16.170680+0000","last_undegraded":"2024-09-21T07:33:16.170680+0000","last_fullsized":"2024-09-21T07:33:16.170680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:19:11.398812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.173330+0000","last_change":"2024-09-21T07:33:14.166709+0000","last_active":"2024-09-21T07:33:16.173330+0000","last_peered":"2024-09-21T07:33:16.173330+0000","last_clean":"2024-09-21T07:33:16.173330+0000","last_became_active":"2024-09-21T07:33:14.166314+0000","last_became_peered":"2024-09-21T07:33:14.166314+0000","last_unstale":"2024-09-21T07:33:16.173330+0000","last_undegraded":"2024-09-21T07:33:16.173330+0000","last_fullsized":"2024-09-21T07:33:16.173330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:19.577738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273229+0000","last_change":"2024-09-21T07:33:15.195086+0000","last_active":"2024-09-21T07:33:16.273229+0000","last_peered":"2024-09-21T07:33:16.273229+0000","last_clean":"2024-09-21T07:33:16.273229+0000","last_became_active":"2024-09-21T07:33:15.194432+0000","last_became_peered":"2024-09-21T07:33:15.194432+0000","last_unstale":"2024-09-21T07:33:16.273229+0000","last_undegraded":"2024-09-21T07:33:16.273229+0000","last_fullsized":"2024-09-21T07:33:16.273229+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:51.754458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272609+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:33:16.272609+0000","last_peered":"2024-09-21T07:33:16.272609+0000","last_clean":"2024-09-21T07:33:16.272609+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:33:16.272609+0000","last_undegraded":"2024-09-21T07:33:16.272609+0000","last_fullsized":"2024-09-21T07:33:16.272609+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.170062+0000","last_change":"2024-09-21T07:33:14.167299+0000","last_active":"2024-09-21T07:33:16.170062+0000","last_peered":"2024-09-21T07:33:16.170062+0000","last_clean":"2024-09-21T07:33:16.170062+0000","last_became_active":"2024-09-21T07:33:14.165495+0000","last_became_peered":"2024-09-21T07:33:14.165495+0000","last_unstale":"2024-09-21T07:33:16.170062+0000","last_undegraded":"2024-09-21T07:33:16.170062+0000","last_fullsized":"2024-09-21T07:33:16.170062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:45.280842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272850+0000","last_change":"2024-09-21T07:33:15.193199+0000","last_active":"2024-09-21T07:33:16.272850+0000","last_peered":"2024-09-21T07:33:16.272850+0000","last_clean":"2024-09-21T07:33:16.272850+0000","last_became_active":"2024-09-21T07:33:15.192865+0000","last_became_peered":"2024-09-21T07:33:15.192865+0000","last_unstale":"2024-09-21T07:33:16.272850+0000","last_undegraded":"2024-09-21T07:33:16.272850+0000","last_fullsized":"2024-09-21T07:33:16.272850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:18.112050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272503+0000","last_change":"2024-09-21T07:33:14.182542+0000","last_active":"2024-09-21T07:33:16.272503+0000","last_peered":"2024-09-21T07:33:16.272503+0000","last_clean":"2024-09-21T07:33:16.272503+0000","last_became_active":"2024-09-21T07:33:14.182190+0000","last_became_peered":"2024-09-21T07:33:14.182190+0000","last_unstale":"2024-09-21T07:33:16.272503+0000","last_undegraded":"2024-09-21T07:33:16.272503+0000","last_fullsized":"2024-09-21T07:33:16.272503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:26:26.032819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272052+0000","last_change":"2024-09-21T07:33:15.185457+0000","last_active":"2024-09-21T07:33:16.272052+0000","last_peered":"2024-09-21T07:33:16.272052+0000","last_clean":"2024-09-21T07:33:16.272052+0000","last_became_active":"2024-09-21T07:33:15.185135+0000","last_became_peered":"2024-09-21T07:33:15.185135+0000","last_unstale":"2024-09-21T07:33:16.272052+0000","last_undegraded":"2024-09-21T07:33:16.272052+0000","last_fullsized":"2024-09-21T07:33:16.272052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:13:07.781038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.173288+0000","last_change":"2024-09-21T07:33:14.167137+0000","last_active":"2024-09-21T07:33:16.173288+0000","last_peered":"2024-09-21T07:33:16.173288+0000","last_clean":"2024-09-21T07:33:16.173288+0000","last_became_active":"2024-09-21T07:33:14.166565+0000","last_became_peered":"2024-09-21T07:33:14.166565+0000","last_unstale":"2024-09-21T07:33:16.173288+0000","last_undegraded":"2024-09-21T07:33:16.173288+0000","last_fullsized":"2024-09-21T07:33:16.173288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:45.280842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171392+0000","last_change":"2024-09-21T07:33:15.170020+0000","last_active":"2024-09-21T07:33:16.171392+0000","last_peered":"2024-09-21T07:33:16.171392+0000","last_clean":"2024-09-21T07:33:16.171392+0000","last_became_active":"2024-09-21T07:33:15.169603+0000","last_became_peered":"2024-09-21T07:33:15.169603+0000","last_unstale":"2024-09-21T07:33:16.171392+0000","last_undegraded":"2024-09-21T07:33:16.171392+0000","last_fullsized":"2024-09-21T07:33:16.171392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:42.179630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.169972+0000","last_change":"2024-09-21T07:33:14.171611+0000","last_active":"2024-09-21T07:33:16.169972+0000","last_peered":"2024-09-21T07:33:16.169972+0000","last_clean":"2024-09-21T07:33:16.169972+0000","last_became_active":"2024-09-21T07:33:14.171272+0000","last_became_peered":"2024-09-21T07:33:14.171272+0000","last_unstale":"2024-09-21T07:33:16.169972+0000","last_undegraded":"2024-09-21T07:33:16.169972+0000","last_fullsized":"2024-09-21T07:33:16.169972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:37.723838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274839+0000","last_change":"2024-09-21T07:33:15.184034+0000","last_active":"2024-09-21T07:33:16.274839+0000","last_peered":"2024-09-21T07:33:16.274839+0000","last_clean":"2024-09-21T07:33:16.274839+0000","last_became_active":"2024-09-21T07:33:15.183802+0000","last_became_peered":"2024-09-21T07:33:15.183802+0000","last_unstale":"2024-09-21T07:33:16.274839+0000","last_undegraded":"2024-09-21T07:33:16.274839+0000","last_fullsized":"2024-09-21T07:33:16.274839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:52:13.038200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171745+0000","last_change":"2024-09-21T07:33:14.179927+0000","last_active":"2024-09-21T07:33:16.171745+0000","last_peered":"2024-09-21T07:33:16.171745+0000","last_clean":"2024-09-21T07:33:16.171745+0000","last_became_active":"2024-09-21T07:33:14.179480+0000","last_became_peered":"2024-09-21T07:33:14.179480+0000","last_unstale":"2024-09-21T07:33:16.171745+0000","last_undegraded":"2024-09-21T07:33:16.171745+0000","last_fullsized":"2024-09-21T07:33:16.171745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:20:56.907479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.170757+0000","last_change":"2024-09-21T07:33:15.170345+0000","last_active":"2024-09-21T07:33:16.170757+0000","last_peered":"2024-09-21T07:33:16.170757+0000","last_clean":"2024-09-21T07:33:16.170757+0000","last_became_active":"2024-09-21T07:33:15.169978+0000","last_became_peered":"2024-09-21T07:33:15.169978+0000","last_unstale":"2024-09-21T07:33:16.170757+0000","last_undegraded":"2024-09-21T07:33:16.170757+0000","last_fullsized":"2024-09-21T07:33:16.170757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:22.852589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273738+0000","last_change":"2024-09-21T07:33:14.186976+0000","last_active":"2024-09-21T07:33:16.273738+0000","last_peered":"2024-09-21T07:33:16.273738+0000","last_clean":"2024-09-21T07:33:16.273738+0000","last_became_active":"2024-09-21T07:33:14.186584+0000","last_became_peered":"2024-09-21T07:33:14.186584+0000","last_unstale":"2024-09-21T07:33:16.273738+0000","last_undegraded":"2024-09-21T07:33:16.273738+0000","last_fullsized":"2024-09-21T07:33:16.273738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:48.956923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171626+0000","last_change":"2024-09-21T07:33:15.168130+0000","last_active":"2024-09-21T07:33:16.171626+0000","last_peered":"2024-09-21T07:33:16.171626+0000","last_clean":"2024-09-21T07:33:16.171626+0000","last_became_active":"2024-09-21T07:33:15.167822+0000","last_became_peered":"2024-09-21T07:33:15.167822+0000","last_unstale":"2024-09-21T07:33:16.171626+0000","last_undegraded":"2024-09-21T07:33:16.171626+0000","last_fullsized":"2024-09-21T07:33:16.171626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:21.223732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:17.161466+0000","last_change":"2024-09-21T07:33:14.170061+0000","last_active":"2024-09-21T07:33:17.161466+0000","last_peered":"2024-09-21T07:33:17.161466+0000","last_clean":"2024-09-21T07:33:17.161466+0000","last_became_active":"2024-09-21T07:33:14.169738+0000","last_became_peered":"2024-09-21T07:33:14.169738+0000","last_unstale":"2024-09-21T07:33:17.161466+0000","last_undegraded":"2024-09-21T07:33:17.161466+0000","last_fullsized":"2024-09-21T07:33:17.161466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:30.472062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273688+0000","last_change":"2024-09-21T07:33:15.179567+0000","last_active":"2024-09-21T07:33:16.273688+0000","last_peered":"2024-09-21T07:33:16.273688+0000","last_clean":"2024-09-21T07:33:16.273688+0000","last_became_active":"2024-09-21T07:33:15.179203+0000","last_became_peered":"2024-09-21T07:33:15.179203+0000","last_unstale":"2024-09-21T07:33:16.273688+0000","last_undegraded":"2024-09-21T07:33:16.273688+0000","last_fullsized":"2024-09-21T07:33:16.273688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:11:16.190546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273883+0000","last_change":"2024-09-21T07:33:14.178218+0000","last_active":"2024-09-21T07:33:16.273883+0000","last_peered":"2024-09-21T07:33:16.273883+0000","last_clean":"2024-09-21T07:33:16.273883+0000","last_became_active":"2024-09-21T07:33:14.177879+0000","last_became_peered":"2024-09-21T07:33:14.177879+0000","last_unstale":"2024-09-21T07:33:16.273883+0000","last_undegraded":"2024-09-21T07:33:16.273883+0000","last_fullsized":"2024-09-21T07:33:16.273883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:37.723838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273327+0000","last_change":"2024-09-21T07:33:15.174382+0000","last_active":"2024-09-21T07:33:16.273327+0000","last_peered":"2024-09-21T07:33:16.273327+0000","last_clean":"2024-09-21T07:33:16.273327+0000","last_became_active":"2024-09-21T07:33:15.174058+0000","last_became_peered":"2024-09-21T07:33:15.174058+0000","last_unstale":"2024-09-21T07:33:16.273327+0000","last_undegraded":"2024-09-21T07:33:16.273327+0000","last_fullsized":"2024-09-21T07:33:16.273327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:42.179630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273075+0000","last_change":"2024-09-21T07:33:14.186745+0000","last_active":"2024-09-21T07:33:16.273075+0000","last_peered":"2024-09-21T07:33:16.273075+0000","last_clean":"2024-09-21T07:33:16.273075+0000","last_became_active":"2024-09-21T07:33:14.186394+0000","last_became_peered":"2024-09-21T07:33:14.186394+0000","last_unstale":"2024-09-21T07:33:16.273075+0000","last_undegraded":"2024-09-21T07:33:16.273075+0000","last_fullsized":"2024-09-21T07:33:16.273075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:48:43.982220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274193+0000","last_change":"2024-09-21T07:33:15.183374+0000","last_active":"2024-09-21T07:33:16.274193+0000","last_peered":"2024-09-21T07:33:16.274193+0000","last_clean":"2024-09-21T07:33:16.274193+0000","last_became_active":"2024-09-21T07:33:15.182688+0000","last_became_peered":"2024-09-21T07:33:15.182688+0000","last_unstale":"2024-09-21T07:33:16.274193+0000","last_undegraded":"2024-09-21T07:33:16.274193+0000","last_fullsized":"2024-09-21T07:33:16.274193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:50.289066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171527+0000","last_change":"2024-09-21T07:33:14.176760+0000","last_active":"2024-09-21T07:33:16.171527+0000","last_peered":"2024-09-21T07:33:16.171527+0000","last_clean":"2024-09-21T07:33:16.171527+0000","last_became_active":"2024-09-21T07:33:14.176465+0000","last_became_peered":"2024-09-21T07:33:14.176465+0000","last_unstale":"2024-09-21T07:33:16.171527+0000","last_undegraded":"2024-09-21T07:33:16.171527+0000","last_fullsized":"2024-09-21T07:33:16.171527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:41:25.774466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272765+0000","last_change":"2024-09-21T07:33:15.173534+0000","last_active":"2024-09-21T07:33:16.272765+0000","last_peered":"2024-09-21T07:33:16.272765+0000","last_clean":"2024-09-21T07:33:16.272765+0000","last_became_active":"2024-09-21T07:33:15.173251+0000","last_became_peered":"2024-09-21T07:33:15.173251+0000","last_unstale":"2024-09-21T07:33:16.272765+0000","last_undegraded":"2024-09-21T07:33:16.272765+0000","last_fullsized":"2024-09-21T07:33:16.272765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:26.738282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171462+0000","last_change":"2024-09-21T07:33:14.171832+0000","last_active":"2024-09-21T07:33:16.171462+0000","last_peered":"2024-09-21T07:33:16.171462+0000","last_clean":"2024-09-21T07:33:16.171462+0000","last_became_active":"2024-09-21T07:33:14.171528+0000","last_became_peered":"2024-09-21T07:33:14.171528+0000","last_unstale":"2024-09-21T07:33:16.171462+0000","last_undegraded":"2024-09-21T07:33:16.171462+0000","last_fullsized":"2024-09-21T07:33:16.171462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:24:12.251487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272756+0000","last_change":"2024-09-21T07:33:15.178517+0000","last_active":"2024-09-21T07:33:16.272756+0000","last_peered":"2024-09-21T07:33:16.272756+0000","last_clean":"2024-09-21T07:33:16.272756+0000","last_became_active":"2024-09-21T07:33:15.178031+0000","last_became_peered":"2024-09-21T07:33:15.178031+0000","last_unstale":"2024-09-21T07:33:16.272756+0000","last_undegraded":"2024-09-21T07:33:16.272756+0000","last_fullsized":"2024-09-21T07:33:16.272756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:44.158584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273321+0000","last_change":"2024-09-21T07:33:14.177385+0000","last_active":"2024-09-21T07:33:16.273321+0000","last_peered":"2024-09-21T07:33:16.273321+0000","last_clean":"2024-09-21T07:33:16.273321+0000","last_became_active":"2024-09-21T07:33:14.176365+0000","last_became_peered":"2024-09-21T07:33:14.176365+0000","last_unstale":"2024-09-21T07:33:16.273321+0000","last_undegraded":"2024-09-21T07:33:16.273321+0000","last_fullsized":"2024-09-21T07:33:16.273321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:10.407008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273445+0000","last_change":"2024-09-21T07:33:15.190071+0000","last_active":"2024-09-21T07:33:16.273445+0000","last_peered":"2024-09-21T07:33:16.273445+0000","last_clean":"2024-09-21T07:33:16.273445+0000","last_became_active":"2024-09-21T07:33:15.189778+0000","last_became_peered":"2024-09-21T07:33:15.189778+0000","last_unstale":"2024-09-21T07:33:16.273445+0000","last_undegraded":"2024-09-21T07:33:16.273445+0000","last_fullsized":"2024-09-21T07:33:16.273445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:02.683216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.170600+0000","last_change":"2024-09-21T07:33:15.183143+0000","last_active":"2024-09-21T07:33:16.170600+0000","last_peered":"2024-09-21T07:33:16.170600+0000","last_clean":"2024-09-21T07:33:16.170600+0000","last_became_active":"2024-09-21T07:33:15.182803+0000","last_became_peered":"2024-09-21T07:33:15.182803+0000","last_unstale":"2024-09-21T07:33:16.170600+0000","last_undegraded":"2024-09-21T07:33:16.170600+0000","last_fullsized":"2024-09-21T07:33:16.170600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:35.308422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273005+0000","last_change":"2024-09-21T07:33:14.181290+0000","last_active":"2024-09-21T07:33:16.273005+0000","last_peered":"2024-09-21T07:33:16.273005+0000","last_clean":"2024-09-21T07:33:16.273005+0000","last_became_active":"2024-09-21T07:33:14.180919+0000","last_became_peered":"2024-09-21T07:33:14.180919+0000","last_unstale":"2024-09-21T07:33:16.273005+0000","last_undegraded":"2024-09-21T07:33:16.273005+0000","last_fullsized":"2024-09-21T07:33:16.273005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:37.723838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.170380+0000","last_change":"2024-09-21T07:33:15.194137+0000","last_active":"2024-09-21T07:33:16.170380+0000","last_peered":"2024-09-21T07:33:16.170380+0000","last_clean":"2024-09-21T07:33:16.170380+0000","last_became_active":"2024-09-21T07:33:15.193903+0000","last_became_peered":"2024-09-21T07:33:15.193903+0000","last_unstale":"2024-09-21T07:33:16.170380+0000","last_undegraded":"2024-09-21T07:33:16.170380+0000","last_fullsized":"2024-09-21T07:33:16.170380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:39:25.870595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273368+0000","last_change":"2024-09-21T07:33:14.178482+0000","last_active":"2024-09-21T07:33:16.273368+0000","last_peered":"2024-09-21T07:33:16.273368+0000","last_clean":"2024-09-21T07:33:16.273368+0000","last_became_active":"2024-09-21T07:33:14.178256+0000","last_became_peered":"2024-09-21T07:33:14.178256+0000","last_unstale":"2024-09-21T07:33:16.273368+0000","last_undegraded":"2024-09-21T07:33:16.273368+0000","last_fullsized":"2024-09-21T07:33:16.273368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:05.003971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273533+0000","last_change":"2024-09-21T07:33:15.177351+0000","last_active":"2024-09-21T07:33:16.273533+0000","last_peered":"2024-09-21T07:33:16.273533+0000","last_clean":"2024-09-21T07:33:16.273533+0000","last_became_active":"2024-09-21T07:33:15.172454+0000","last_became_peered":"2024-09-21T07:33:15.172454+0000","last_unstale":"2024-09-21T07:33:16.273533+0000","last_undegraded":"2024-09-21T07:33:16.273533+0000","last_fullsized":"2024-09-21T07:33:16.273533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:21.223732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273101+0000","last_change":"2024-09-21T07:33:14.178706+0000","last_active":"2024-09-21T07:33:16.273101+0000","last_peered":"2024-09-21T07:33:16.273101+0000","last_clean":"2024-09-21T07:33:16.273101+0000","last_became_active":"2024-09-21T07:33:14.178398+0000","last_became_peered":"2024-09-21T07:33:14.178398+0000","last_unstale":"2024-09-21T07:33:16.273101+0000","last_undegraded":"2024-09-21T07:33:16.273101+0000","last_fullsized":"2024-09-21T07:33:16.273101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:39.076808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273018+0000","last_change":"2024-09-21T07:33:15.181010+0000","last_active":"2024-09-21T07:33:16.273018+0000","last_peered":"2024-09-21T07:33:16.273018+0000","last_clean":"2024-09-21T07:33:16.273018+0000","last_became_active":"2024-09-21T07:33:15.179699+0000","last_became_peered":"2024-09-21T07:33:15.179699+0000","last_unstale":"2024-09-21T07:33:16.273018+0000","last_undegraded":"2024-09-21T07:33:16.273018+0000","last_fullsized":"2024-09-21T07:33:16.273018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:01.852516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273687+0000","last_change":"2024-09-21T07:33:14.177313+0000","last_active":"2024-09-21T07:33:16.273687+0000","last_peered":"2024-09-21T07:33:16.273687+0000","last_clean":"2024-09-21T07:33:16.273687+0000","last_became_active":"2024-09-21T07:33:14.176867+0000","last_became_peered":"2024-09-21T07:33:14.176867+0000","last_unstale":"2024-09-21T07:33:16.273687+0000","last_undegraded":"2024-09-21T07:33:16.273687+0000","last_fullsized":"2024-09-21T07:33:16.273687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:01.421862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":806912,"data_stored":663137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":73799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":72277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":208896,"data_stored":72187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":72347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":815104,"data_stored":663193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":66013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":208896,"data_stored":67499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:33:19.652 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:33:19.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.914+0000 7f21f5e00640 1 -- 172.21.15.39:0/848979672 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f21f0073170 msgr2=0x7f21f015c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:19.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.914+0000 7f21f5e00640 1 --2- 172.21.15.39:0/848979672 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f21f0073170 0x7f21f015c780 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f21e0005fd0 tx=0x7f21e002a540 comp rx=0 tx=0).stop 2024-09-21T07:33:19.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.914+0000 7f21f5e00640 1 -- 172.21.15.39:0/848979672 shutdown_connections 2024-09-21T07:33:19.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.914+0000 7f21f5e00640 1 --2- 172.21.15.39:0/848979672 >> v2:172.21.15.148:3300/0 conn(0x7f21f0070030 0x7f21f0074640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:19.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.914+0000 7f21f5e00640 1 --2- 172.21.15.39:0/848979672 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f21f0073170 0x7f21f015c780 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:19.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.914+0000 7f21f5e00640 1 -- 172.21.15.39:0/848979672 >> 172.21.15.39:0/848979672 conn(0x7f21f00b1cb0 msgr2=0x7f21f00b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:19.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.914+0000 7f21f5e00640 1 -- 172.21.15.39:0/848979672 shutdown_connections 2024-09-21T07:33:19.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.914+0000 7f21f5e00640 1 -- 172.21.15.39:0/848979672 wait complete. 2024-09-21T07:33:19.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.916+0000 7f21f5e00640 1 Processor -- start 2024-09-21T07:33:19.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.916+0000 7f21f5e00640 1 -- start start 2024-09-21T07:33:19.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.918+0000 7f21f5e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f21f0070030 0x7f21f01f65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:19.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.918+0000 7f21f5e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21f0070440 0x7f21f014b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:19.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.918+0000 7f21f5e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21f0073170 0x7f21f01f9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:19.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.918+0000 7f21f5e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f21f006f5a0 con 0x7f21f0070440 2024-09-21T07:33:19.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.918+0000 7f21f5e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f21f0074f80 con 0x7f21f0073170 2024-09-21T07:33:19.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.918+0000 7f21f5e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f21f006f720 con 0x7f21f0070030 2024-09-21T07:33:19.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.918+0000 7f21e7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21f0070440 0x7f21f014b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:19.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.918+0000 7f21eea00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f21f0070030 0x7f21f01f65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:19.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.918+0000 7f21e7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21f0070440 0x7f21f014b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60526/0 (socket says 172.21.15.39:60526) 2024-09-21T07:33:19.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.918+0000 7f21e7e00640 1 -- 172.21.15.39:0/187653540 learned_addr learned my addr 172.21.15.39:0/187653540 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:19.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.918+0000 7f21ef400640 1 --2- 172.21.15.39:0/187653540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21f0073170 0x7f21f01f9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:19.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.919+0000 7f21e7e00640 1 -- 172.21.15.39:0/187653540 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f21f0070030 msgr2=0x7f21f01f65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:19.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.919+0000 7f21e7e00640 1 --2- 172.21.15.39:0/187653540 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f21f0070030 0x7f21f01f65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:19.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.919+0000 7f21e7e00640 1 -- 172.21.15.39:0/187653540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21f0073170 msgr2=0x7f21f01f9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:19.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.919+0000 7f21e7e00640 1 --2- 172.21.15.39:0/187653540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21f0073170 0x7f21f01f9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:19.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.919+0000 7f21e7e00640 1 -- 172.21.15.39:0/187653540 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f00c9ad0 con 0x7f21f0070440 2024-09-21T07:33:19.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.919+0000 7f21e7e00640 1 --2- 172.21.15.39:0/187653540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21f0070440 0x7f21f014b130 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f21d800e9f0 tx=0x7f21d800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:19.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.919+0000 7f21e7400640 1 -- 172.21.15.39:0/187653540 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d800ccc0 con 0x7f21f0070440 2024-09-21T07:33:19.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.919+0000 7f21e7400640 1 -- 172.21.15.39:0/187653540 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21d8004020 con 0x7f21f0070440 2024-09-21T07:33:19.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.919+0000 7f21f5e00640 1 -- 172.21.15.39:0/187653540 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21f01f9bc0 con 0x7f21f0070440 2024-09-21T07:33:19.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.919+0000 7f21e7400640 1 -- 172.21.15.39:0/187653540 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d800ce60 con 0x7f21f0070440 2024-09-21T07:33:19.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.919+0000 7f21f5e00640 1 -- 172.21.15.39:0/187653540 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21f01f5db0 con 0x7f21f0070440 2024-09-21T07:33:19.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.920+0000 7f21f5e00640 1 -- 172.21.15.39:0/187653540 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21f01f9930 con 0x7f21f0070440 2024-09-21T07:33:19.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.922+0000 7f21e7400640 1 -- 172.21.15.39:0/187653540 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f21d80035f0 con 0x7f21f0070440 2024-09-21T07:33:19.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.923+0000 7f21e7400640 1 --2- 172.21.15.39:0/187653540 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f21cc078b90 0x7f21cc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:19.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.923+0000 7f21e7400640 1 -- 172.21.15.39:0/187653540 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f21d809b750 con 0x7f21f0070440 2024-09-21T07:33:19.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.923+0000 7f21eea00640 1 --2- 172.21.15.39:0/187653540 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f21cc078b90 0x7f21cc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:19.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.924+0000 7f21eea00640 1 --2- 172.21.15.39:0/187653540 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f21cc078b90 0x7f21cc07b0a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f21e0033000 tx=0x7f21e0004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:19.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:19.927+0000 7f21e7400640 1 -- 172.21.15.39:0/187653540 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f21d80634a0 con 0x7f21f0070440 2024-09-21T07:33:20.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.119+0000 7f21f5e00640 1 -- 172.21.15.39:0/187653540 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f21f006fc40 con 0x7f21cc078b90 2024-09-21T07:33:20.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.125+0000 7f21e7400640 1 -- 172.21.15.39:0/187653540 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251940 (secure 0 0 0) 0x7f21f006fc40 con 0x7f21cc078b90 2024-09-21T07:33:20.129 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:33:20.131 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:33:20.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.129+0000 7f21f5e00640 1 -- 172.21.15.39:0/187653540 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f21cc078b90 msgr2=0x7f21cc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:20.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.129+0000 7f21f5e00640 1 --2- 172.21.15.39:0/187653540 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f21cc078b90 0x7f21cc07b0a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f21e0033000 tx=0x7f21e0004730 comp rx=0 tx=0).stop 2024-09-21T07:33:20.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.129+0000 7f21f5e00640 1 -- 172.21.15.39:0/187653540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21f0070440 msgr2=0x7f21f014b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:20.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.129+0000 7f21f5e00640 1 --2- 172.21.15.39:0/187653540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21f0070440 0x7f21f014b130 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f21d800e9f0 tx=0x7f21d800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:33:20.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.129+0000 7f21f5e00640 1 -- 172.21.15.39:0/187653540 shutdown_connections 2024-09-21T07:33:20.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.129+0000 7f21f5e00640 1 --2- 172.21.15.39:0/187653540 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f21cc078b90 0x7f21cc07b0a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:20.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.129+0000 7f21f5e00640 1 --2- 172.21.15.39:0/187653540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21f0073170 0x7f21f01f9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:20.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.129+0000 7f21f5e00640 1 --2- 172.21.15.39:0/187653540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21f0070440 0x7f21f014b130 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:20.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.129+0000 7f21f5e00640 1 --2- 172.21.15.39:0/187653540 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f21f0070030 0x7f21f01f65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:20.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.129+0000 7f21f5e00640 1 -- 172.21.15.39:0/187653540 >> 172.21.15.39:0/187653540 conn(0x7f21f00b1cb0 msgr2=0x7f21f00ae9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:20.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.129+0000 7f21f5e00640 1 -- 172.21.15.39:0/187653540 shutdown_connections 2024-09-21T07:33:20.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.129+0000 7f21f5e00640 1 -- 172.21.15.39:0/187653540 wait complete. 2024-09-21T07:33:20.154 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-09-21T07:33:19.533815+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":166,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235708,"kb_used_data":2832,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739012,"statfs":{"total":773094113280,"available":772852748288,"internally_reserved":0,"allocated":2899968,"data_stored":1750452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001523"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274756+0000","last_change":"2024-09-21T07:33:15.176955+0000","last_active":"2024-09-21T07:33:16.274756+0000","last_peered":"2024-09-21T07:33:16.274756+0000","last_clean":"2024-09-21T07:33:16.274756+0000","last_became_active":"2024-09-21T07:33:15.176632+0000","last_became_peered":"2024-09-21T07:33:15.176632+0000","last_unstale":"2024-09-21T07:33:16.274756+0000","last_undegraded":"2024-09-21T07:33:16.274756+0000","last_fullsized":"2024-09-21T07:33:16.274756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:10:51.829331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171764+0000","last_change":"2024-09-21T07:33:15.184382+0000","last_active":"2024-09-21T07:33:16.171764+0000","last_peered":"2024-09-21T07:33:16.171764+0000","last_clean":"2024-09-21T07:33:16.171764+0000","last_became_active":"2024-09-21T07:33:15.184072+0000","last_became_peered":"2024-09-21T07:33:15.184072+0000","last_unstale":"2024-09-21T07:33:16.171764+0000","last_undegraded":"2024-09-21T07:33:16.171764+0000","last_fullsized":"2024-09-21T07:33:16.171764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:08.677357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273864+0000","last_change":"2024-09-21T07:33:15.186660+0000","last_active":"2024-09-21T07:33:16.273864+0000","last_peered":"2024-09-21T07:33:16.273864+0000","last_clean":"2024-09-21T07:33:16.273864+0000","last_became_active":"2024-09-21T07:33:15.185832+0000","last_became_peered":"2024-09-21T07:33:15.185832+0000","last_unstale":"2024-09-21T07:33:16.273864+0000","last_undegraded":"2024-09-21T07:33:16.273864+0000","last_fullsized":"2024-09-21T07:33:16.273864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:08.677357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.170390+0000","last_change":"2024-09-21T07:33:15.193591+0000","last_active":"2024-09-21T07:33:16.170390+0000","last_peered":"2024-09-21T07:33:16.170390+0000","last_clean":"2024-09-21T07:33:16.170390+0000","last_became_active":"2024-09-21T07:33:15.191327+0000","last_became_peered":"2024-09-21T07:33:15.191327+0000","last_unstale":"2024-09-21T07:33:16.170390+0000","last_undegraded":"2024-09-21T07:33:16.170390+0000","last_fullsized":"2024-09-21T07:33:16.170390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:00:15.919300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272836+0000","last_change":"2024-09-21T07:33:15.187754+0000","last_active":"2024-09-21T07:33:16.272836+0000","last_peered":"2024-09-21T07:33:16.272836+0000","last_clean":"2024-09-21T07:33:16.272836+0000","last_became_active":"2024-09-21T07:33:15.187381+0000","last_became_peered":"2024-09-21T07:33:15.187381+0000","last_unstale":"2024-09-21T07:33:16.272836+0000","last_undegraded":"2024-09-21T07:33:16.272836+0000","last_fullsized":"2024-09-21T07:33:16.272836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:19:59.491735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.169414+0000","last_change":"2024-09-21T07:33:15.180176+0000","last_active":"2024-09-21T07:33:16.169414+0000","last_peered":"2024-09-21T07:33:16.169414+0000","last_clean":"2024-09-21T07:33:16.169414+0000","last_became_active":"2024-09-21T07:33:15.179828+0000","last_became_peered":"2024-09-21T07:33:15.179828+0000","last_unstale":"2024-09-21T07:33:16.169414+0000","last_undegraded":"2024-09-21T07:33:16.169414+0000","last_fullsized":"2024-09-21T07:33:16.169414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:40:52.015841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274813+0000","last_change":"2024-09-21T07:33:15.183358+0000","last_active":"2024-09-21T07:33:16.274813+0000","last_peered":"2024-09-21T07:33:16.274813+0000","last_clean":"2024-09-21T07:33:16.274813+0000","last_became_active":"2024-09-21T07:33:15.182969+0000","last_became_peered":"2024-09-21T07:33:15.182969+0000","last_unstale":"2024-09-21T07:33:16.274813+0000","last_undegraded":"2024-09-21T07:33:16.274813+0000","last_fullsized":"2024-09-21T07:33:16.274813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:41:44.441220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272174+0000","last_change":"2024-09-21T07:33:15.173445+0000","last_active":"2024-09-21T07:33:16.272174+0000","last_peered":"2024-09-21T07:33:16.272174+0000","last_clean":"2024-09-21T07:33:16.272174+0000","last_became_active":"2024-09-21T07:33:15.172833+0000","last_became_peered":"2024-09-21T07:33:15.172833+0000","last_unstale":"2024-09-21T07:33:16.272174+0000","last_undegraded":"2024-09-21T07:33:16.272174+0000","last_fullsized":"2024-09-21T07:33:16.272174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:24:46.139397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274983+0000","last_change":"2024-09-21T07:33:15.194854+0000","last_active":"2024-09-21T07:33:16.274983+0000","last_peered":"2024-09-21T07:33:16.274983+0000","last_clean":"2024-09-21T07:33:16.274983+0000","last_became_active":"2024-09-21T07:33:15.194423+0000","last_became_peered":"2024-09-21T07:33:15.194423+0000","last_unstale":"2024-09-21T07:33:16.274983+0000","last_undegraded":"2024-09-21T07:33:16.274983+0000","last_fullsized":"2024-09-21T07:33:16.274983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:14.196020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274097+0000","last_change":"2024-09-21T07:33:15.176651+0000","last_active":"2024-09-21T07:33:16.274097+0000","last_peered":"2024-09-21T07:33:16.274097+0000","last_clean":"2024-09-21T07:33:16.274097+0000","last_became_active":"2024-09-21T07:33:15.176222+0000","last_became_peered":"2024-09-21T07:33:15.176222+0000","last_unstale":"2024-09-21T07:33:16.274097+0000","last_undegraded":"2024-09-21T07:33:16.274097+0000","last_fullsized":"2024-09-21T07:33:16.274097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:47.494973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274249+0000","last_change":"2024-09-21T07:33:15.170250+0000","last_active":"2024-09-21T07:33:16.274249+0000","last_peered":"2024-09-21T07:33:16.274249+0000","last_clean":"2024-09-21T07:33:16.274249+0000","last_became_active":"2024-09-21T07:33:15.166192+0000","last_became_peered":"2024-09-21T07:33:15.166192+0000","last_unstale":"2024-09-21T07:33:16.274249+0000","last_undegraded":"2024-09-21T07:33:16.274249+0000","last_fullsized":"2024-09-21T07:33:16.274249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:48:05.009446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274541+0000","last_change":"2024-09-21T07:33:15.180001+0000","last_active":"2024-09-21T07:33:16.274541+0000","last_peered":"2024-09-21T07:33:16.274541+0000","last_clean":"2024-09-21T07:33:16.274541+0000","last_became_active":"2024-09-21T07:33:15.179679+0000","last_became_peered":"2024-09-21T07:33:15.179679+0000","last_unstale":"2024-09-21T07:33:16.274541+0000","last_undegraded":"2024-09-21T07:33:16.274541+0000","last_fullsized":"2024-09-21T07:33:16.274541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:05:31.417897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272616+0000","last_change":"2024-09-21T07:33:15.179831+0000","last_active":"2024-09-21T07:33:16.272616+0000","last_peered":"2024-09-21T07:33:16.272616+0000","last_clean":"2024-09-21T07:33:16.272616+0000","last_became_active":"2024-09-21T07:33:15.179382+0000","last_became_peered":"2024-09-21T07:33:15.179382+0000","last_unstale":"2024-09-21T07:33:16.272616+0000","last_undegraded":"2024-09-21T07:33:16.272616+0000","last_fullsized":"2024-09-21T07:33:16.272616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:26:20.407200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272116+0000","last_change":"2024-09-21T07:33:15.174462+0000","last_active":"2024-09-21T07:33:16.272116+0000","last_peered":"2024-09-21T07:33:16.272116+0000","last_clean":"2024-09-21T07:33:16.272116+0000","last_became_active":"2024-09-21T07:33:15.174135+0000","last_became_peered":"2024-09-21T07:33:15.174135+0000","last_unstale":"2024-09-21T07:33:16.272116+0000","last_undegraded":"2024-09-21T07:33:16.272116+0000","last_fullsized":"2024-09-21T07:33:16.272116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:53.282103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171346+0000","last_change":"2024-09-21T07:33:15.184280+0000","last_active":"2024-09-21T07:33:16.171346+0000","last_peered":"2024-09-21T07:33:16.171346+0000","last_clean":"2024-09-21T07:33:16.171346+0000","last_became_active":"2024-09-21T07:33:15.183935+0000","last_became_peered":"2024-09-21T07:33:15.183935+0000","last_unstale":"2024-09-21T07:33:16.171346+0000","last_undegraded":"2024-09-21T07:33:16.171346+0000","last_fullsized":"2024-09-21T07:33:16.171346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:17:23.362898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.169177+0000","last_change":"2024-09-21T07:33:15.178501+0000","last_active":"2024-09-21T07:33:16.169177+0000","last_peered":"2024-09-21T07:33:16.169177+0000","last_clean":"2024-09-21T07:33:16.169177+0000","last_became_active":"2024-09-21T07:33:15.178157+0000","last_became_peered":"2024-09-21T07:33:15.178157+0000","last_unstale":"2024-09-21T07:33:16.169177+0000","last_undegraded":"2024-09-21T07:33:16.169177+0000","last_fullsized":"2024-09-21T07:33:16.169177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:47.494973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.170990+0000","last_change":"2024-09-21T07:33:15.194890+0000","last_active":"2024-09-21T07:33:16.170990+0000","last_peered":"2024-09-21T07:33:16.170990+0000","last_clean":"2024-09-21T07:33:16.170990+0000","last_became_active":"2024-09-21T07:33:15.194618+0000","last_became_peered":"2024-09-21T07:33:15.194618+0000","last_unstale":"2024-09-21T07:33:16.170990+0000","last_undegraded":"2024-09-21T07:33:16.170990+0000","last_fullsized":"2024-09-21T07:33:16.170990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:42:25.647850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.173528+0000","last_change":"2024-09-21T07:33:15.177265+0000","last_active":"2024-09-21T07:33:16.173528+0000","last_peered":"2024-09-21T07:33:16.173528+0000","last_clean":"2024-09-21T07:33:16.173528+0000","last_became_active":"2024-09-21T07:33:15.176835+0000","last_became_peered":"2024-09-21T07:33:15.176835+0000","last_unstale":"2024-09-21T07:33:16.173528+0000","last_undegraded":"2024-09-21T07:33:16.173528+0000","last_fullsized":"2024-09-21T07:33:16.173528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:26:27.046080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273611+0000","last_change":"2024-09-21T07:33:15.187455+0000","last_active":"2024-09-21T07:33:16.273611+0000","last_peered":"2024-09-21T07:33:16.273611+0000","last_clean":"2024-09-21T07:33:16.273611+0000","last_became_active":"2024-09-21T07:33:15.187097+0000","last_became_peered":"2024-09-21T07:33:15.187097+0000","last_unstale":"2024-09-21T07:33:16.273611+0000","last_undegraded":"2024-09-21T07:33:16.273611+0000","last_fullsized":"2024-09-21T07:33:16.273611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:12:27.933810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273575+0000","last_change":"2024-09-21T07:33:15.196207+0000","last_active":"2024-09-21T07:33:16.273575+0000","last_peered":"2024-09-21T07:33:16.273575+0000","last_clean":"2024-09-21T07:33:16.273575+0000","last_became_active":"2024-09-21T07:33:15.195889+0000","last_became_peered":"2024-09-21T07:33:15.195889+0000","last_unstale":"2024-09-21T07:33:16.273575+0000","last_undegraded":"2024-09-21T07:33:16.273575+0000","last_fullsized":"2024-09-21T07:33:16.273575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:23.420344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272226+0000","last_change":"2024-09-21T07:33:15.182205+0000","last_active":"2024-09-21T07:33:16.272226+0000","last_peered":"2024-09-21T07:33:16.272226+0000","last_clean":"2024-09-21T07:33:16.272226+0000","last_became_active":"2024-09-21T07:33:15.181790+0000","last_became_peered":"2024-09-21T07:33:15.181790+0000","last_unstale":"2024-09-21T07:33:16.272226+0000","last_undegraded":"2024-09-21T07:33:16.272226+0000","last_fullsized":"2024-09-21T07:33:16.272226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:09.053804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.169665+0000","last_change":"2024-09-21T07:33:15.172738+0000","last_active":"2024-09-21T07:33:16.169665+0000","last_peered":"2024-09-21T07:33:16.169665+0000","last_clean":"2024-09-21T07:33:16.169665+0000","last_became_active":"2024-09-21T07:33:15.172454+0000","last_became_peered":"2024-09-21T07:33:15.172454+0000","last_unstale":"2024-09-21T07:33:16.169665+0000","last_undegraded":"2024-09-21T07:33:16.169665+0000","last_fullsized":"2024-09-21T07:33:16.169665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:26:46.580701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272525+0000","last_change":"2024-09-21T07:33:15.185561+0000","last_active":"2024-09-21T07:33:16.272525+0000","last_peered":"2024-09-21T07:33:16.272525+0000","last_clean":"2024-09-21T07:33:16.272525+0000","last_became_active":"2024-09-21T07:33:15.185004+0000","last_became_peered":"2024-09-21T07:33:15.185004+0000","last_unstale":"2024-09-21T07:33:16.272525+0000","last_undegraded":"2024-09-21T07:33:16.272525+0000","last_fullsized":"2024-09-21T07:33:16.272525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:25:13.620607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.169268+0000","last_change":"2024-09-21T07:33:15.164795+0000","last_active":"2024-09-21T07:33:16.169268+0000","last_peered":"2024-09-21T07:33:16.169268+0000","last_clean":"2024-09-21T07:33:16.169268+0000","last_became_active":"2024-09-21T07:33:15.164419+0000","last_became_peered":"2024-09-21T07:33:15.164419+0000","last_unstale":"2024-09-21T07:33:16.169268+0000","last_undegraded":"2024-09-21T07:33:16.169268+0000","last_fullsized":"2024-09-21T07:33:16.169268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:48:05.009446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273570+0000","last_change":"2024-09-21T07:33:15.183825+0000","last_active":"2024-09-21T07:33:16.273570+0000","last_peered":"2024-09-21T07:33:16.273570+0000","last_clean":"2024-09-21T07:33:16.273570+0000","last_became_active":"2024-09-21T07:33:15.183503+0000","last_became_peered":"2024-09-21T07:33:15.183503+0000","last_unstale":"2024-09-21T07:33:16.273570+0000","last_undegraded":"2024-09-21T07:33:16.273570+0000","last_fullsized":"2024-09-21T07:33:16.273570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:12.378254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273927+0000","last_change":"2024-09-21T07:33:15.177070+0000","last_active":"2024-09-21T07:33:16.273927+0000","last_peered":"2024-09-21T07:33:16.273927+0000","last_clean":"2024-09-21T07:33:16.273927+0000","last_became_active":"2024-09-21T07:33:15.176706+0000","last_became_peered":"2024-09-21T07:33:15.176706+0000","last_unstale":"2024-09-21T07:33:16.273927+0000","last_undegraded":"2024-09-21T07:33:16.273927+0000","last_fullsized":"2024-09-21T07:33:16.273927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:40:41.309353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171642+0000","last_change":"2024-09-21T07:33:15.174783+0000","last_active":"2024-09-21T07:33:16.171642+0000","last_peered":"2024-09-21T07:33:16.171642+0000","last_clean":"2024-09-21T07:33:16.171642+0000","last_became_active":"2024-09-21T07:33:15.173581+0000","last_became_peered":"2024-09-21T07:33:15.173581+0000","last_unstale":"2024-09-21T07:33:16.171642+0000","last_undegraded":"2024-09-21T07:33:16.171642+0000","last_fullsized":"2024-09-21T07:33:16.171642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:43:34.665544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273063+0000","last_change":"2024-09-21T07:33:15.185421+0000","last_active":"2024-09-21T07:33:16.273063+0000","last_peered":"2024-09-21T07:33:16.273063+0000","last_clean":"2024-09-21T07:33:16.273063+0000","last_became_active":"2024-09-21T07:33:15.185131+0000","last_became_peered":"2024-09-21T07:33:15.185131+0000","last_unstale":"2024-09-21T07:33:16.273063+0000","last_undegraded":"2024-09-21T07:33:16.273063+0000","last_fullsized":"2024-09-21T07:33:16.273063+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:29.320339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273245+0000","last_change":"2024-09-21T07:33:15.178626+0000","last_active":"2024-09-21T07:33:16.273245+0000","last_peered":"2024-09-21T07:33:16.273245+0000","last_clean":"2024-09-21T07:33:16.273245+0000","last_became_active":"2024-09-21T07:33:15.178344+0000","last_became_peered":"2024-09-21T07:33:15.178344+0000","last_unstale":"2024-09-21T07:33:16.273245+0000","last_undegraded":"2024-09-21T07:33:16.273245+0000","last_fullsized":"2024-09-21T07:33:16.273245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:17:23.362898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.169955+0000","last_change":"2024-09-21T07:33:15.187835+0000","last_active":"2024-09-21T07:33:16.169955+0000","last_peered":"2024-09-21T07:33:16.169955+0000","last_clean":"2024-09-21T07:33:16.169955+0000","last_became_active":"2024-09-21T07:33:15.185993+0000","last_became_peered":"2024-09-21T07:33:15.185993+0000","last_unstale":"2024-09-21T07:33:16.169955+0000","last_undegraded":"2024-09-21T07:33:16.169955+0000","last_fullsized":"2024-09-21T07:33:16.169955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:17.480483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.173171+0000","last_change":"2024-09-21T07:33:15.163142+0000","last_active":"2024-09-21T07:33:16.173171+0000","last_peered":"2024-09-21T07:33:16.173171+0000","last_clean":"2024-09-21T07:33:16.173171+0000","last_became_active":"2024-09-21T07:33:15.162044+0000","last_became_peered":"2024-09-21T07:33:15.162044+0000","last_unstale":"2024-09-21T07:33:16.173171+0000","last_undegraded":"2024-09-21T07:33:16.173171+0000","last_fullsized":"2024-09-21T07:33:16.173171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:48:44.995481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272735+0000","last_change":"2024-09-21T07:33:15.188257+0000","last_active":"2024-09-21T07:33:16.272735+0000","last_peered":"2024-09-21T07:33:16.272735+0000","last_clean":"2024-09-21T07:33:16.272735+0000","last_became_active":"2024-09-21T07:33:15.184871+0000","last_became_peered":"2024-09-21T07:33:15.184871+0000","last_unstale":"2024-09-21T07:33:16.272735+0000","last_undegraded":"2024-09-21T07:33:16.272735+0000","last_fullsized":"2024-09-21T07:33:16.272735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:31:35.062821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274086+0000","last_change":"2024-09-21T07:33:15.178648+0000","last_active":"2024-09-21T07:33:16.274086+0000","last_peered":"2024-09-21T07:33:16.274086+0000","last_clean":"2024-09-21T07:33:16.274086+0000","last_became_active":"2024-09-21T07:33:15.178260+0000","last_became_peered":"2024-09-21T07:33:15.178260+0000","last_unstale":"2024-09-21T07:33:16.274086+0000","last_undegraded":"2024-09-21T07:33:16.274086+0000","last_fullsized":"2024-09-21T07:33:16.274086+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:36.198560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272837+0000","last_change":"2024-09-21T07:33:14.173209+0000","last_active":"2024-09-21T07:33:16.272837+0000","last_peered":"2024-09-21T07:33:16.272837+0000","last_clean":"2024-09-21T07:33:16.272837+0000","last_became_active":"2024-09-21T07:33:14.172936+0000","last_became_peered":"2024-09-21T07:33:14.172936+0000","last_unstale":"2024-09-21T07:33:16.272837+0000","last_undegraded":"2024-09-21T07:33:16.272837+0000","last_fullsized":"2024-09-21T07:33:16.272837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:03.284858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273336+0000","last_change":"2024-09-21T07:33:15.191981+0000","last_active":"2024-09-21T07:33:16.273336+0000","last_peered":"2024-09-21T07:33:16.273336+0000","last_clean":"2024-09-21T07:33:16.273336+0000","last_became_active":"2024-09-21T07:33:15.188976+0000","last_became_peered":"2024-09-21T07:33:15.188976+0000","last_unstale":"2024-09-21T07:33:16.273336+0000","last_undegraded":"2024-09-21T07:33:16.273336+0000","last_fullsized":"2024-09-21T07:33:16.273336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:34.842293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273636+0000","last_change":"2024-09-21T07:33:14.169344+0000","last_active":"2024-09-21T07:33:16.273636+0000","last_peered":"2024-09-21T07:33:16.273636+0000","last_clean":"2024-09-21T07:33:16.273636+0000","last_became_active":"2024-09-21T07:33:14.169015+0000","last_became_peered":"2024-09-21T07:33:14.169015+0000","last_unstale":"2024-09-21T07:33:16.273636+0000","last_undegraded":"2024-09-21T07:33:16.273636+0000","last_fullsized":"2024-09-21T07:33:16.273636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:10.925173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272616+0000","last_change":"2024-09-21T07:33:14.163956+0000","last_active":"2024-09-21T07:33:16.272616+0000","last_peered":"2024-09-21T07:33:16.272616+0000","last_clean":"2024-09-21T07:33:16.272616+0000","last_became_active":"2024-09-21T07:33:14.163698+0000","last_became_peered":"2024-09-21T07:33:14.163698+0000","last_unstale":"2024-09-21T07:33:16.272616+0000","last_undegraded":"2024-09-21T07:33:16.272616+0000","last_fullsized":"2024-09-21T07:33:16.272616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:32.172082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272523+0000","last_change":"2024-09-21T07:33:15.184364+0000","last_active":"2024-09-21T07:33:16.272523+0000","last_peered":"2024-09-21T07:33:16.272523+0000","last_clean":"2024-09-21T07:33:16.272523+0000","last_became_active":"2024-09-21T07:33:15.184042+0000","last_became_peered":"2024-09-21T07:33:15.184042+0000","last_unstale":"2024-09-21T07:33:16.272523+0000","last_undegraded":"2024-09-21T07:33:16.272523+0000","last_fullsized":"2024-09-21T07:33:16.272523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:39:05.189714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.169283+0000","last_change":"2024-09-21T07:33:14.160602+0000","last_active":"2024-09-21T07:33:16.169283+0000","last_peered":"2024-09-21T07:33:16.169283+0000","last_clean":"2024-09-21T07:33:16.169283+0000","last_became_active":"2024-09-21T07:33:14.159110+0000","last_became_peered":"2024-09-21T07:33:14.159110+0000","last_unstale":"2024-09-21T07:33:16.169283+0000","last_undegraded":"2024-09-21T07:33:16.169283+0000","last_fullsized":"2024-09-21T07:33:16.169283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:10.925173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171398+0000","last_change":"2024-09-21T07:33:15.178332+0000","last_active":"2024-09-21T07:33:16.171398+0000","last_peered":"2024-09-21T07:33:16.171398+0000","last_clean":"2024-09-21T07:33:16.171398+0000","last_became_active":"2024-09-21T07:33:15.177793+0000","last_became_peered":"2024-09-21T07:33:15.177793+0000","last_unstale":"2024-09-21T07:33:16.171398+0000","last_undegraded":"2024-09-21T07:33:16.171398+0000","last_fullsized":"2024-09-21T07:33:16.171398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:23.420344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273829+0000","last_change":"2024-09-21T07:33:14.171262+0000","last_active":"2024-09-21T07:33:16.273829+0000","last_peered":"2024-09-21T07:33:16.273829+0000","last_clean":"2024-09-21T07:33:16.273829+0000","last_became_active":"2024-09-21T07:33:14.170882+0000","last_became_peered":"2024-09-21T07:33:14.170882+0000","last_unstale":"2024-09-21T07:33:16.273829+0000","last_undegraded":"2024-09-21T07:33:16.273829+0000","last_fullsized":"2024-09-21T07:33:16.273829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:34.304149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.170713+0000","last_change":"2024-09-21T07:33:15.173639+0000","last_active":"2024-09-21T07:33:16.170713+0000","last_peered":"2024-09-21T07:33:16.170713+0000","last_clean":"2024-09-21T07:33:16.170713+0000","last_became_active":"2024-09-21T07:33:15.173275+0000","last_became_peered":"2024-09-21T07:33:15.173275+0000","last_unstale":"2024-09-21T07:33:16.170713+0000","last_undegraded":"2024-09-21T07:33:16.170713+0000","last_fullsized":"2024-09-21T07:33:16.170713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:12:30.901535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.170872+0000","last_change":"2024-09-21T07:33:14.170282+0000","last_active":"2024-09-21T07:33:16.170872+0000","last_peered":"2024-09-21T07:33:16.170872+0000","last_clean":"2024-09-21T07:33:16.170872+0000","last_became_active":"2024-09-21T07:33:14.169935+0000","last_became_peered":"2024-09-21T07:33:14.169935+0000","last_unstale":"2024-09-21T07:33:16.170872+0000","last_undegraded":"2024-09-21T07:33:16.170872+0000","last_fullsized":"2024-09-21T07:33:16.170872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:06.477956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273555+0000","last_change":"2024-09-21T07:33:15.171671+0000","last_active":"2024-09-21T07:33:16.273555+0000","last_peered":"2024-09-21T07:33:16.273555+0000","last_clean":"2024-09-21T07:33:16.273555+0000","last_became_active":"2024-09-21T07:33:15.171327+0000","last_became_peered":"2024-09-21T07:33:15.171327+0000","last_unstale":"2024-09-21T07:33:16.273555+0000","last_undegraded":"2024-09-21T07:33:16.273555+0000","last_fullsized":"2024-09-21T07:33:16.273555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:43:34.665544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171766+0000","last_change":"2024-09-21T07:33:14.166941+0000","last_active":"2024-09-21T07:33:16.171766+0000","last_peered":"2024-09-21T07:33:16.171766+0000","last_clean":"2024-09-21T07:33:16.171766+0000","last_became_active":"2024-09-21T07:33:14.165770+0000","last_became_peered":"2024-09-21T07:33:14.165770+0000","last_unstale":"2024-09-21T07:33:16.171766+0000","last_undegraded":"2024-09-21T07:33:16.171766+0000","last_fullsized":"2024-09-21T07:33:16.171766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:51.025109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274800+0000","last_change":"2024-09-21T07:33:15.181837+0000","last_active":"2024-09-21T07:33:16.274800+0000","last_peered":"2024-09-21T07:33:16.274800+0000","last_clean":"2024-09-21T07:33:16.274800+0000","last_became_active":"2024-09-21T07:33:15.181358+0000","last_became_peered":"2024-09-21T07:33:15.181358+0000","last_unstale":"2024-09-21T07:33:16.274800+0000","last_undegraded":"2024-09-21T07:33:16.274800+0000","last_fullsized":"2024-09-21T07:33:16.274800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:56.567663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273547+0000","last_change":"2024-09-21T07:33:14.172709+0000","last_active":"2024-09-21T07:33:16.273547+0000","last_peered":"2024-09-21T07:33:16.273547+0000","last_clean":"2024-09-21T07:33:16.273547+0000","last_became_active":"2024-09-21T07:33:14.172316+0000","last_became_peered":"2024-09-21T07:33:14.172316+0000","last_unstale":"2024-09-21T07:33:16.273547+0000","last_undegraded":"2024-09-21T07:33:16.273547+0000","last_fullsized":"2024-09-21T07:33:16.273547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:06.477956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272937+0000","last_change":"2024-09-21T07:33:15.179712+0000","last_active":"2024-09-21T07:33:16.272937+0000","last_peered":"2024-09-21T07:33:16.272937+0000","last_clean":"2024-09-21T07:33:16.272937+0000","last_became_active":"2024-09-21T07:33:15.179368+0000","last_became_peered":"2024-09-21T07:33:15.179368+0000","last_unstale":"2024-09-21T07:33:16.272937+0000","last_undegraded":"2024-09-21T07:33:16.272937+0000","last_fullsized":"2024-09-21T07:33:16.272937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:14.765726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273376+0000","last_change":"2024-09-21T07:33:14.180872+0000","last_active":"2024-09-21T07:33:16.273376+0000","last_peered":"2024-09-21T07:33:16.273376+0000","last_clean":"2024-09-21T07:33:16.273376+0000","last_became_active":"2024-09-21T07:33:14.179682+0000","last_became_peered":"2024-09-21T07:33:14.179682+0000","last_unstale":"2024-09-21T07:33:16.273376+0000","last_undegraded":"2024-09-21T07:33:16.273376+0000","last_fullsized":"2024-09-21T07:33:16.273376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:35:06.212701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272328+0000","last_change":"2024-09-21T07:33:15.177488+0000","last_active":"2024-09-21T07:33:16.272328+0000","last_peered":"2024-09-21T07:33:16.272328+0000","last_clean":"2024-09-21T07:33:16.272328+0000","last_became_active":"2024-09-21T07:33:15.176270+0000","last_became_peered":"2024-09-21T07:33:15.176270+0000","last_unstale":"2024-09-21T07:33:16.272328+0000","last_undegraded":"2024-09-21T07:33:16.272328+0000","last_fullsized":"2024-09-21T07:33:16.272328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:47.805013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.169798+0000","last_change":"2024-09-21T07:33:14.178744+0000","last_active":"2024-09-21T07:33:16.169798+0000","last_peered":"2024-09-21T07:33:16.169798+0000","last_clean":"2024-09-21T07:33:16.169798+0000","last_became_active":"2024-09-21T07:33:14.177786+0000","last_became_peered":"2024-09-21T07:33:14.177786+0000","last_unstale":"2024-09-21T07:33:16.169798+0000","last_undegraded":"2024-09-21T07:33:16.169798+0000","last_fullsized":"2024-09-21T07:33:16.169798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:34.304149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273092+0000","last_change":"2024-09-21T07:33:15.191036+0000","last_active":"2024-09-21T07:33:16.273092+0000","last_peered":"2024-09-21T07:33:16.273092+0000","last_clean":"2024-09-21T07:33:16.273092+0000","last_became_active":"2024-09-21T07:33:15.190631+0000","last_became_peered":"2024-09-21T07:33:15.190631+0000","last_unstale":"2024-09-21T07:33:16.273092+0000","last_undegraded":"2024-09-21T07:33:16.273092+0000","last_fullsized":"2024-09-21T07:33:16.273092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:39:17.752820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273048+0000","last_change":"2024-09-21T07:33:14.170962+0000","last_active":"2024-09-21T07:33:16.273048+0000","last_peered":"2024-09-21T07:33:16.273048+0000","last_clean":"2024-09-21T07:33:16.273048+0000","last_became_active":"2024-09-21T07:33:14.169996+0000","last_became_peered":"2024-09-21T07:33:14.169996+0000","last_unstale":"2024-09-21T07:33:16.273048+0000","last_undegraded":"2024-09-21T07:33:16.273048+0000","last_fullsized":"2024-09-21T07:33:16.273048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:45.280842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272830+0000","last_change":"2024-09-21T07:33:15.190018+0000","last_active":"2024-09-21T07:33:16.272830+0000","last_peered":"2024-09-21T07:33:16.272830+0000","last_clean":"2024-09-21T07:33:16.272830+0000","last_became_active":"2024-09-21T07:33:15.185361+0000","last_became_peered":"2024-09-21T07:33:15.185361+0000","last_unstale":"2024-09-21T07:33:16.272830+0000","last_undegraded":"2024-09-21T07:33:16.272830+0000","last_fullsized":"2024-09-21T07:33:16.272830+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:35.723678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.173087+0000","last_change":"2024-09-21T07:33:14.171085+0000","last_active":"2024-09-21T07:33:16.173087+0000","last_peered":"2024-09-21T07:33:16.173087+0000","last_clean":"2024-09-21T07:33:16.173087+0000","last_became_active":"2024-09-21T07:33:14.170838+0000","last_became_peered":"2024-09-21T07:33:14.170838+0000","last_unstale":"2024-09-21T07:33:16.173087+0000","last_undegraded":"2024-09-21T07:33:16.173087+0000","last_fullsized":"2024-09-21T07:33:16.173087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:06.477956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.275049+0000","last_change":"2024-09-21T07:33:15.181599+0000","last_active":"2024-09-21T07:33:16.275049+0000","last_peered":"2024-09-21T07:33:16.275049+0000","last_clean":"2024-09-21T07:33:16.275049+0000","last_became_active":"2024-09-21T07:33:15.181044+0000","last_became_peered":"2024-09-21T07:33:15.181044+0000","last_unstale":"2024-09-21T07:33:16.275049+0000","last_undegraded":"2024-09-21T07:33:16.275049+0000","last_fullsized":"2024-09-21T07:33:16.275049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:39.660131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273625+0000","last_change":"2024-09-21T07:33:14.180061+0000","last_active":"2024-09-21T07:33:16.273625+0000","last_peered":"2024-09-21T07:33:16.273625+0000","last_clean":"2024-09-21T07:33:16.273625+0000","last_became_active":"2024-09-21T07:33:14.179053+0000","last_became_peered":"2024-09-21T07:33:14.179053+0000","last_unstale":"2024-09-21T07:33:16.273625+0000","last_undegraded":"2024-09-21T07:33:16.273625+0000","last_fullsized":"2024-09-21T07:33:16.273625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:46.482047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274323+0000","last_change":"2024-09-21T07:33:15.187133+0000","last_active":"2024-09-21T07:33:16.274323+0000","last_peered":"2024-09-21T07:33:16.274323+0000","last_clean":"2024-09-21T07:33:16.274323+0000","last_became_active":"2024-09-21T07:33:15.186773+0000","last_became_peered":"2024-09-21T07:33:15.186773+0000","last_unstale":"2024-09-21T07:33:16.274323+0000","last_undegraded":"2024-09-21T07:33:16.274323+0000","last_fullsized":"2024-09-21T07:33:16.274323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:51.977863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.173649+0000","last_change":"2024-09-21T07:33:14.163096+0000","last_active":"2024-09-21T07:33:16.173649+0000","last_peered":"2024-09-21T07:33:16.173649+0000","last_clean":"2024-09-21T07:33:16.173649+0000","last_became_active":"2024-09-21T07:33:14.162512+0000","last_became_peered":"2024-09-21T07:33:14.162512+0000","last_unstale":"2024-09-21T07:33:16.173649+0000","last_undegraded":"2024-09-21T07:33:16.173649+0000","last_fullsized":"2024-09-21T07:33:16.173649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:10.925173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.275012+0000","last_change":"2024-09-21T07:33:15.184770+0000","last_active":"2024-09-21T07:33:16.275012+0000","last_peered":"2024-09-21T07:33:16.275012+0000","last_clean":"2024-09-21T07:33:16.275012+0000","last_became_active":"2024-09-21T07:33:15.184167+0000","last_became_peered":"2024-09-21T07:33:15.184167+0000","last_unstale":"2024-09-21T07:33:16.275012+0000","last_undegraded":"2024-09-21T07:33:16.275012+0000","last_fullsized":"2024-09-21T07:33:16.275012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:10.342501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171867+0000","last_change":"2024-09-21T07:33:14.170661+0000","last_active":"2024-09-21T07:33:16.171867+0000","last_peered":"2024-09-21T07:33:16.171867+0000","last_clean":"2024-09-21T07:33:16.171867+0000","last_became_active":"2024-09-21T07:33:14.170408+0000","last_became_peered":"2024-09-21T07:33:14.170408+0000","last_unstale":"2024-09-21T07:33:16.171867+0000","last_undegraded":"2024-09-21T07:33:16.171867+0000","last_fullsized":"2024-09-21T07:33:16.171867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:43:00.990750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.170680+0000","last_change":"2024-09-21T07:33:15.165673+0000","last_active":"2024-09-21T07:33:16.170680+0000","last_peered":"2024-09-21T07:33:16.170680+0000","last_clean":"2024-09-21T07:33:16.170680+0000","last_became_active":"2024-09-21T07:33:15.164392+0000","last_became_peered":"2024-09-21T07:33:15.164392+0000","last_unstale":"2024-09-21T07:33:16.170680+0000","last_undegraded":"2024-09-21T07:33:16.170680+0000","last_fullsized":"2024-09-21T07:33:16.170680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:19:11.398812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.173330+0000","last_change":"2024-09-21T07:33:14.166709+0000","last_active":"2024-09-21T07:33:16.173330+0000","last_peered":"2024-09-21T07:33:16.173330+0000","last_clean":"2024-09-21T07:33:16.173330+0000","last_became_active":"2024-09-21T07:33:14.166314+0000","last_became_peered":"2024-09-21T07:33:14.166314+0000","last_unstale":"2024-09-21T07:33:16.173330+0000","last_undegraded":"2024-09-21T07:33:16.173330+0000","last_fullsized":"2024-09-21T07:33:16.173330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:19.577738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273229+0000","last_change":"2024-09-21T07:33:15.195086+0000","last_active":"2024-09-21T07:33:16.273229+0000","last_peered":"2024-09-21T07:33:16.273229+0000","last_clean":"2024-09-21T07:33:16.273229+0000","last_became_active":"2024-09-21T07:33:15.194432+0000","last_became_peered":"2024-09-21T07:33:15.194432+0000","last_unstale":"2024-09-21T07:33:16.273229+0000","last_undegraded":"2024-09-21T07:33:16.273229+0000","last_fullsized":"2024-09-21T07:33:16.273229+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:51.754458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272609+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:33:16.272609+0000","last_peered":"2024-09-21T07:33:16.272609+0000","last_clean":"2024-09-21T07:33:16.272609+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:33:16.272609+0000","last_undegraded":"2024-09-21T07:33:16.272609+0000","last_fullsized":"2024-09-21T07:33:16.272609+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.170062+0000","last_change":"2024-09-21T07:33:14.167299+0000","last_active":"2024-09-21T07:33:16.170062+0000","last_peered":"2024-09-21T07:33:16.170062+0000","last_clean":"2024-09-21T07:33:16.170062+0000","last_became_active":"2024-09-21T07:33:14.165495+0000","last_became_peered":"2024-09-21T07:33:14.165495+0000","last_unstale":"2024-09-21T07:33:16.170062+0000","last_undegraded":"2024-09-21T07:33:16.170062+0000","last_fullsized":"2024-09-21T07:33:16.170062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:45.280842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272850+0000","last_change":"2024-09-21T07:33:15.193199+0000","last_active":"2024-09-21T07:33:16.272850+0000","last_peered":"2024-09-21T07:33:16.272850+0000","last_clean":"2024-09-21T07:33:16.272850+0000","last_became_active":"2024-09-21T07:33:15.192865+0000","last_became_peered":"2024-09-21T07:33:15.192865+0000","last_unstale":"2024-09-21T07:33:16.272850+0000","last_undegraded":"2024-09-21T07:33:16.272850+0000","last_fullsized":"2024-09-21T07:33:16.272850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:18.112050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272503+0000","last_change":"2024-09-21T07:33:14.182542+0000","last_active":"2024-09-21T07:33:16.272503+0000","last_peered":"2024-09-21T07:33:16.272503+0000","last_clean":"2024-09-21T07:33:16.272503+0000","last_became_active":"2024-09-21T07:33:14.182190+0000","last_became_peered":"2024-09-21T07:33:14.182190+0000","last_unstale":"2024-09-21T07:33:16.272503+0000","last_undegraded":"2024-09-21T07:33:16.272503+0000","last_fullsized":"2024-09-21T07:33:16.272503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:26:26.032819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272052+0000","last_change":"2024-09-21T07:33:15.185457+0000","last_active":"2024-09-21T07:33:16.272052+0000","last_peered":"2024-09-21T07:33:16.272052+0000","last_clean":"2024-09-21T07:33:16.272052+0000","last_became_active":"2024-09-21T07:33:15.185135+0000","last_became_peered":"2024-09-21T07:33:15.185135+0000","last_unstale":"2024-09-21T07:33:16.272052+0000","last_undegraded":"2024-09-21T07:33:16.272052+0000","last_fullsized":"2024-09-21T07:33:16.272052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:13:07.781038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.173288+0000","last_change":"2024-09-21T07:33:14.167137+0000","last_active":"2024-09-21T07:33:16.173288+0000","last_peered":"2024-09-21T07:33:16.173288+0000","last_clean":"2024-09-21T07:33:16.173288+0000","last_became_active":"2024-09-21T07:33:14.166565+0000","last_became_peered":"2024-09-21T07:33:14.166565+0000","last_unstale":"2024-09-21T07:33:16.173288+0000","last_undegraded":"2024-09-21T07:33:16.173288+0000","last_fullsized":"2024-09-21T07:33:16.173288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:45.280842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171392+0000","last_change":"2024-09-21T07:33:15.170020+0000","last_active":"2024-09-21T07:33:16.171392+0000","last_peered":"2024-09-21T07:33:16.171392+0000","last_clean":"2024-09-21T07:33:16.171392+0000","last_became_active":"2024-09-21T07:33:15.169603+0000","last_became_peered":"2024-09-21T07:33:15.169603+0000","last_unstale":"2024-09-21T07:33:16.171392+0000","last_undegraded":"2024-09-21T07:33:16.171392+0000","last_fullsized":"2024-09-21T07:33:16.171392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:42.179630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.169972+0000","last_change":"2024-09-21T07:33:14.171611+0000","last_active":"2024-09-21T07:33:16.169972+0000","last_peered":"2024-09-21T07:33:16.169972+0000","last_clean":"2024-09-21T07:33:16.169972+0000","last_became_active":"2024-09-21T07:33:14.171272+0000","last_became_peered":"2024-09-21T07:33:14.171272+0000","last_unstale":"2024-09-21T07:33:16.169972+0000","last_undegraded":"2024-09-21T07:33:16.169972+0000","last_fullsized":"2024-09-21T07:33:16.169972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:37.723838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274839+0000","last_change":"2024-09-21T07:33:15.184034+0000","last_active":"2024-09-21T07:33:16.274839+0000","last_peered":"2024-09-21T07:33:16.274839+0000","last_clean":"2024-09-21T07:33:16.274839+0000","last_became_active":"2024-09-21T07:33:15.183802+0000","last_became_peered":"2024-09-21T07:33:15.183802+0000","last_unstale":"2024-09-21T07:33:16.274839+0000","last_undegraded":"2024-09-21T07:33:16.274839+0000","last_fullsized":"2024-09-21T07:33:16.274839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:52:13.038200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171745+0000","last_change":"2024-09-21T07:33:14.179927+0000","last_active":"2024-09-21T07:33:16.171745+0000","last_peered":"2024-09-21T07:33:16.171745+0000","last_clean":"2024-09-21T07:33:16.171745+0000","last_became_active":"2024-09-21T07:33:14.179480+0000","last_became_peered":"2024-09-21T07:33:14.179480+0000","last_unstale":"2024-09-21T07:33:16.171745+0000","last_undegraded":"2024-09-21T07:33:16.171745+0000","last_fullsized":"2024-09-21T07:33:16.171745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:20:56.907479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.170757+0000","last_change":"2024-09-21T07:33:15.170345+0000","last_active":"2024-09-21T07:33:16.170757+0000","last_peered":"2024-09-21T07:33:16.170757+0000","last_clean":"2024-09-21T07:33:16.170757+0000","last_became_active":"2024-09-21T07:33:15.169978+0000","last_became_peered":"2024-09-21T07:33:15.169978+0000","last_unstale":"2024-09-21T07:33:16.170757+0000","last_undegraded":"2024-09-21T07:33:16.170757+0000","last_fullsized":"2024-09-21T07:33:16.170757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:22.852589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273738+0000","last_change":"2024-09-21T07:33:14.186976+0000","last_active":"2024-09-21T07:33:16.273738+0000","last_peered":"2024-09-21T07:33:16.273738+0000","last_clean":"2024-09-21T07:33:16.273738+0000","last_became_active":"2024-09-21T07:33:14.186584+0000","last_became_peered":"2024-09-21T07:33:14.186584+0000","last_unstale":"2024-09-21T07:33:16.273738+0000","last_undegraded":"2024-09-21T07:33:16.273738+0000","last_fullsized":"2024-09-21T07:33:16.273738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:48.956923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171626+0000","last_change":"2024-09-21T07:33:15.168130+0000","last_active":"2024-09-21T07:33:16.171626+0000","last_peered":"2024-09-21T07:33:16.171626+0000","last_clean":"2024-09-21T07:33:16.171626+0000","last_became_active":"2024-09-21T07:33:15.167822+0000","last_became_peered":"2024-09-21T07:33:15.167822+0000","last_unstale":"2024-09-21T07:33:16.171626+0000","last_undegraded":"2024-09-21T07:33:16.171626+0000","last_fullsized":"2024-09-21T07:33:16.171626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:21.223732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:17.161466+0000","last_change":"2024-09-21T07:33:14.170061+0000","last_active":"2024-09-21T07:33:17.161466+0000","last_peered":"2024-09-21T07:33:17.161466+0000","last_clean":"2024-09-21T07:33:17.161466+0000","last_became_active":"2024-09-21T07:33:14.169738+0000","last_became_peered":"2024-09-21T07:33:14.169738+0000","last_unstale":"2024-09-21T07:33:17.161466+0000","last_undegraded":"2024-09-21T07:33:17.161466+0000","last_fullsized":"2024-09-21T07:33:17.161466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:30.472062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273688+0000","last_change":"2024-09-21T07:33:15.179567+0000","last_active":"2024-09-21T07:33:16.273688+0000","last_peered":"2024-09-21T07:33:16.273688+0000","last_clean":"2024-09-21T07:33:16.273688+0000","last_became_active":"2024-09-21T07:33:15.179203+0000","last_became_peered":"2024-09-21T07:33:15.179203+0000","last_unstale":"2024-09-21T07:33:16.273688+0000","last_undegraded":"2024-09-21T07:33:16.273688+0000","last_fullsized":"2024-09-21T07:33:16.273688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:11:16.190546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273883+0000","last_change":"2024-09-21T07:33:14.178218+0000","last_active":"2024-09-21T07:33:16.273883+0000","last_peered":"2024-09-21T07:33:16.273883+0000","last_clean":"2024-09-21T07:33:16.273883+0000","last_became_active":"2024-09-21T07:33:14.177879+0000","last_became_peered":"2024-09-21T07:33:14.177879+0000","last_unstale":"2024-09-21T07:33:16.273883+0000","last_undegraded":"2024-09-21T07:33:16.273883+0000","last_fullsized":"2024-09-21T07:33:16.273883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:37.723838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273327+0000","last_change":"2024-09-21T07:33:15.174382+0000","last_active":"2024-09-21T07:33:16.273327+0000","last_peered":"2024-09-21T07:33:16.273327+0000","last_clean":"2024-09-21T07:33:16.273327+0000","last_became_active":"2024-09-21T07:33:15.174058+0000","last_became_peered":"2024-09-21T07:33:15.174058+0000","last_unstale":"2024-09-21T07:33:16.273327+0000","last_undegraded":"2024-09-21T07:33:16.273327+0000","last_fullsized":"2024-09-21T07:33:16.273327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:42.179630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273075+0000","last_change":"2024-09-21T07:33:14.186745+0000","last_active":"2024-09-21T07:33:16.273075+0000","last_peered":"2024-09-21T07:33:16.273075+0000","last_clean":"2024-09-21T07:33:16.273075+0000","last_became_active":"2024-09-21T07:33:14.186394+0000","last_became_peered":"2024-09-21T07:33:14.186394+0000","last_unstale":"2024-09-21T07:33:16.273075+0000","last_undegraded":"2024-09-21T07:33:16.273075+0000","last_fullsized":"2024-09-21T07:33:16.273075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:48:43.982220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274193+0000","last_change":"2024-09-21T07:33:15.183374+0000","last_active":"2024-09-21T07:33:16.274193+0000","last_peered":"2024-09-21T07:33:16.274193+0000","last_clean":"2024-09-21T07:33:16.274193+0000","last_became_active":"2024-09-21T07:33:15.182688+0000","last_became_peered":"2024-09-21T07:33:15.182688+0000","last_unstale":"2024-09-21T07:33:16.274193+0000","last_undegraded":"2024-09-21T07:33:16.274193+0000","last_fullsized":"2024-09-21T07:33:16.274193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:50.289066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171527+0000","last_change":"2024-09-21T07:33:14.176760+0000","last_active":"2024-09-21T07:33:16.171527+0000","last_peered":"2024-09-21T07:33:16.171527+0000","last_clean":"2024-09-21T07:33:16.171527+0000","last_became_active":"2024-09-21T07:33:14.176465+0000","last_became_peered":"2024-09-21T07:33:14.176465+0000","last_unstale":"2024-09-21T07:33:16.171527+0000","last_undegraded":"2024-09-21T07:33:16.171527+0000","last_fullsized":"2024-09-21T07:33:16.171527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:41:25.774466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272765+0000","last_change":"2024-09-21T07:33:15.173534+0000","last_active":"2024-09-21T07:33:16.272765+0000","last_peered":"2024-09-21T07:33:16.272765+0000","last_clean":"2024-09-21T07:33:16.272765+0000","last_became_active":"2024-09-21T07:33:15.173251+0000","last_became_peered":"2024-09-21T07:33:15.173251+0000","last_unstale":"2024-09-21T07:33:16.272765+0000","last_undegraded":"2024-09-21T07:33:16.272765+0000","last_fullsized":"2024-09-21T07:33:16.272765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:26.738282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171462+0000","last_change":"2024-09-21T07:33:14.171832+0000","last_active":"2024-09-21T07:33:16.171462+0000","last_peered":"2024-09-21T07:33:16.171462+0000","last_clean":"2024-09-21T07:33:16.171462+0000","last_became_active":"2024-09-21T07:33:14.171528+0000","last_became_peered":"2024-09-21T07:33:14.171528+0000","last_unstale":"2024-09-21T07:33:16.171462+0000","last_undegraded":"2024-09-21T07:33:16.171462+0000","last_fullsized":"2024-09-21T07:33:16.171462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:24:12.251487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272756+0000","last_change":"2024-09-21T07:33:15.178517+0000","last_active":"2024-09-21T07:33:16.272756+0000","last_peered":"2024-09-21T07:33:16.272756+0000","last_clean":"2024-09-21T07:33:16.272756+0000","last_became_active":"2024-09-21T07:33:15.178031+0000","last_became_peered":"2024-09-21T07:33:15.178031+0000","last_unstale":"2024-09-21T07:33:16.272756+0000","last_undegraded":"2024-09-21T07:33:16.272756+0000","last_fullsized":"2024-09-21T07:33:16.272756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:44.158584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273321+0000","last_change":"2024-09-21T07:33:14.177385+0000","last_active":"2024-09-21T07:33:16.273321+0000","last_peered":"2024-09-21T07:33:16.273321+0000","last_clean":"2024-09-21T07:33:16.273321+0000","last_became_active":"2024-09-21T07:33:14.176365+0000","last_became_peered":"2024-09-21T07:33:14.176365+0000","last_unstale":"2024-09-21T07:33:16.273321+0000","last_undegraded":"2024-09-21T07:33:16.273321+0000","last_fullsized":"2024-09-21T07:33:16.273321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:10.407008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273445+0000","last_change":"2024-09-21T07:33:15.190071+0000","last_active":"2024-09-21T07:33:16.273445+0000","last_peered":"2024-09-21T07:33:16.273445+0000","last_clean":"2024-09-21T07:33:16.273445+0000","last_became_active":"2024-09-21T07:33:15.189778+0000","last_became_peered":"2024-09-21T07:33:15.189778+0000","last_unstale":"2024-09-21T07:33:16.273445+0000","last_undegraded":"2024-09-21T07:33:16.273445+0000","last_fullsized":"2024-09-21T07:33:16.273445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:02.683216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.170600+0000","last_change":"2024-09-21T07:33:15.183143+0000","last_active":"2024-09-21T07:33:16.170600+0000","last_peered":"2024-09-21T07:33:16.170600+0000","last_clean":"2024-09-21T07:33:16.170600+0000","last_became_active":"2024-09-21T07:33:15.182803+0000","last_became_peered":"2024-09-21T07:33:15.182803+0000","last_unstale":"2024-09-21T07:33:16.170600+0000","last_undegraded":"2024-09-21T07:33:16.170600+0000","last_fullsized":"2024-09-21T07:33:16.170600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:35.308422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273005+0000","last_change":"2024-09-21T07:33:14.181290+0000","last_active":"2024-09-21T07:33:16.273005+0000","last_peered":"2024-09-21T07:33:16.273005+0000","last_clean":"2024-09-21T07:33:16.273005+0000","last_became_active":"2024-09-21T07:33:14.180919+0000","last_became_peered":"2024-09-21T07:33:14.180919+0000","last_unstale":"2024-09-21T07:33:16.273005+0000","last_undegraded":"2024-09-21T07:33:16.273005+0000","last_fullsized":"2024-09-21T07:33:16.273005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:37.723838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.170380+0000","last_change":"2024-09-21T07:33:15.194137+0000","last_active":"2024-09-21T07:33:16.170380+0000","last_peered":"2024-09-21T07:33:16.170380+0000","last_clean":"2024-09-21T07:33:16.170380+0000","last_became_active":"2024-09-21T07:33:15.193903+0000","last_became_peered":"2024-09-21T07:33:15.193903+0000","last_unstale":"2024-09-21T07:33:16.170380+0000","last_undegraded":"2024-09-21T07:33:16.170380+0000","last_fullsized":"2024-09-21T07:33:16.170380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:39:25.870595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273368+0000","last_change":"2024-09-21T07:33:14.178482+0000","last_active":"2024-09-21T07:33:16.273368+0000","last_peered":"2024-09-21T07:33:16.273368+0000","last_clean":"2024-09-21T07:33:16.273368+0000","last_became_active":"2024-09-21T07:33:14.178256+0000","last_became_peered":"2024-09-21T07:33:14.178256+0000","last_unstale":"2024-09-21T07:33:16.273368+0000","last_undegraded":"2024-09-21T07:33:16.273368+0000","last_fullsized":"2024-09-21T07:33:16.273368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:05.003971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273533+0000","last_change":"2024-09-21T07:33:15.177351+0000","last_active":"2024-09-21T07:33:16.273533+0000","last_peered":"2024-09-21T07:33:16.273533+0000","last_clean":"2024-09-21T07:33:16.273533+0000","last_became_active":"2024-09-21T07:33:15.172454+0000","last_became_peered":"2024-09-21T07:33:15.172454+0000","last_unstale":"2024-09-21T07:33:16.273533+0000","last_undegraded":"2024-09-21T07:33:16.273533+0000","last_fullsized":"2024-09-21T07:33:16.273533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:21.223732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273101+0000","last_change":"2024-09-21T07:33:14.178706+0000","last_active":"2024-09-21T07:33:16.273101+0000","last_peered":"2024-09-21T07:33:16.273101+0000","last_clean":"2024-09-21T07:33:16.273101+0000","last_became_active":"2024-09-21T07:33:14.178398+0000","last_became_peered":"2024-09-21T07:33:14.178398+0000","last_unstale":"2024-09-21T07:33:16.273101+0000","last_undegraded":"2024-09-21T07:33:16.273101+0000","last_fullsized":"2024-09-21T07:33:16.273101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:39.076808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273018+0000","last_change":"2024-09-21T07:33:15.181010+0000","last_active":"2024-09-21T07:33:16.273018+0000","last_peered":"2024-09-21T07:33:16.273018+0000","last_clean":"2024-09-21T07:33:16.273018+0000","last_became_active":"2024-09-21T07:33:15.179699+0000","last_became_peered":"2024-09-21T07:33:15.179699+0000","last_unstale":"2024-09-21T07:33:16.273018+0000","last_undegraded":"2024-09-21T07:33:16.273018+0000","last_fullsized":"2024-09-21T07:33:16.273018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:01.852516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273687+0000","last_change":"2024-09-21T07:33:14.177313+0000","last_active":"2024-09-21T07:33:16.273687+0000","last_peered":"2024-09-21T07:33:16.273687+0000","last_clean":"2024-09-21T07:33:16.273687+0000","last_became_active":"2024-09-21T07:33:14.176867+0000","last_became_peered":"2024-09-21T07:33:14.176867+0000","last_unstale":"2024-09-21T07:33:16.273687+0000","last_undegraded":"2024-09-21T07:33:16.273687+0000","last_fullsized":"2024-09-21T07:33:16.273687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:01.421862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":806912,"data_stored":663137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":73799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":72277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":208896,"data_stored":72187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":72347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":815104,"data_stored":663193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":66013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":208896,"data_stored":67499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:33:20.158 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T07:33:20.158 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-21T07:33:20.159 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-21T07:33:20.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.419+0000 7f8a4ba00640 1 -- 172.21.15.39:0/2617210121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a44070440 msgr2=0x7f8a44074b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:20.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.419+0000 7f8a4ba00640 1 --2- 172.21.15.39:0/2617210121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a44070440 0x7f8a44074b80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c005fd0 tx=0x7f8a3c02a8b0 comp rx=0 tx=0).stop 2024-09-21T07:33:20.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.419+0000 7f8a4ba00640 1 -- 172.21.15.39:0/2617210121 shutdown_connections 2024-09-21T07:33:20.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.419+0000 7f8a4ba00640 1 --2- 172.21.15.39:0/2617210121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a44070440 0x7f8a44074b80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:20.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.419+0000 7f8a4ba00640 1 --2- 172.21.15.39:0/2617210121 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a44070030 0x7f8a44074640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:20.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.419+0000 7f8a4ba00640 1 -- 172.21.15.39:0/2617210121 >> 172.21.15.39:0/2617210121 conn(0x7f8a440b1cb0 msgr2=0x7f8a440b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:20.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.419+0000 7f8a4ba00640 1 -- 172.21.15.39:0/2617210121 shutdown_connections 2024-09-21T07:33:20.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.419+0000 7f8a4ba00640 1 -- 172.21.15.39:0/2617210121 wait complete. 2024-09-21T07:33:20.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.421+0000 7f8a4ba00640 1 Processor -- start 2024-09-21T07:33:20.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.422+0000 7f8a4ba00640 1 -- start start 2024-09-21T07:33:20.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.423+0000 7f8a4ba00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a44070030 0x7f8a440ce020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:20.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.423+0000 7f8a4ba00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a44070440 0x7f8a4415a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:20.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.423+0000 7f8a4ba00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a44073170 0x7f8a441f3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:20.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.423+0000 7f8a4ba00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8a440b3a90 con 0x7f8a44070030 2024-09-21T07:33:20.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.423+0000 7f8a4ba00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8a440b3910 con 0x7f8a44070440 2024-09-21T07:33:20.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.423+0000 7f8a4ba00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f8a440b3c10 con 0x7f8a44073170 2024-09-21T07:33:20.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.423+0000 7f8a49000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a44070030 0x7f8a440ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:20.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.423+0000 7f8a49000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a44070030 0x7f8a440ce020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60534/0 (socket says 172.21.15.39:60534) 2024-09-21T07:33:20.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.423+0000 7f8a49000640 1 -- 172.21.15.39:0/3142822858 learned_addr learned my addr 172.21.15.39:0/3142822858 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:20.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.423+0000 7f8a49a00640 1 --2- 172.21.15.39:0/3142822858 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a44073170 0x7f8a441f3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:20.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.423+0000 7f8a43e00640 1 --2- 172.21.15.39:0/3142822858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a44070440 0x7f8a4415a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:20.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.424+0000 7f8a49000640 1 -- 172.21.15.39:0/3142822858 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a44073170 msgr2=0x7f8a441f3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:20.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.424+0000 7f8a49000640 1 --2- 172.21.15.39:0/3142822858 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a44073170 0x7f8a441f3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:20.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.424+0000 7f8a49000640 1 -- 172.21.15.39:0/3142822858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a44070440 msgr2=0x7f8a4415a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:20.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.424+0000 7f8a49000640 1 --2- 172.21.15.39:0/3142822858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a44070440 0x7f8a4415a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:20.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.424+0000 7f8a49000640 1 -- 172.21.15.39:0/3142822858 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a441f7270 con 0x7f8a44070030 2024-09-21T07:33:20.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.424+0000 7f8a43e00640 1 --2- 172.21.15.39:0/3142822858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a44070440 0x7f8a4415a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:33:20.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.424+0000 7f8a49000640 1 --2- 172.21.15.39:0/3142822858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a44070030 0x7f8a440ce020 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8a3800c3c0 tx=0x7f8a3800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:20.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.424+0000 7f8a40c00640 1 -- 172.21.15.39:0/3142822858 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a38017070 con 0x7f8a44070030 2024-09-21T07:33:20.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.424+0000 7f8a40c00640 1 -- 172.21.15.39:0/3142822858 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a380035f0 con 0x7f8a44070030 2024-09-21T07:33:20.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.424+0000 7f8a4ba00640 1 -- 172.21.15.39:0/3142822858 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a441f7550 con 0x7f8a44070030 2024-09-21T07:33:20.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.424+0000 7f8a40c00640 1 -- 172.21.15.39:0/3142822858 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a38003a80 con 0x7f8a44070030 2024-09-21T07:33:20.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.424+0000 7f8a4ba00640 1 -- 172.21.15.39:0/3142822858 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a440d0440 con 0x7f8a44070030 2024-09-21T07:33:20.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.425+0000 7f8a4ba00640 1 -- 172.21.15.39:0/3142822858 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a440b4910 con 0x7f8a44070030 2024-09-21T07:33:20.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.427+0000 7f8a40c00640 1 -- 172.21.15.39:0/3142822858 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8a3801a070 con 0x7f8a44070030 2024-09-21T07:33:20.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.427+0000 7f8a40c00640 1 --2- 172.21.15.39:0/3142822858 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8a24078b90 0x7f8a2407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:20.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.428+0000 7f8a40c00640 1 -- 172.21.15.39:0/3142822858 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8a38099f90 con 0x7f8a44070030 2024-09-21T07:33:20.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.428+0000 7f8a43e00640 1 --2- 172.21.15.39:0/3142822858 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8a24078b90 0x7f8a2407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:20.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.429+0000 7f8a43e00640 1 --2- 172.21.15.39:0/3142822858 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8a24078b90 0x7f8a2407b0a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8a440d0200 tx=0x7f8a30005f10 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:20.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.432+0000 7f8a40c00640 1 -- 172.21.15.39:0/3142822858 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8a38061d70 con 0x7f8a44070030 2024-09-21T07:33:20.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.630+0000 7f8a4ba00640 1 -- 172.21.15.39:0/3142822858 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f8a4415bc00 con 0x7f8a44070030 2024-09-21T07:33:20.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.631+0000 7f8a40c00640 1 -- 172.21.15.39:0/3142822858 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v31) ==== 86+0+160 (secure 0 0 0) 0x7f8a380667a0 con 0x7f8a44070030 2024-09-21T07:33:20.633 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:33:20.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.633+0000 7f8a4ba00640 1 -- 172.21.15.39:0/3142822858 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8a24078b90 msgr2=0x7f8a2407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:20.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.633+0000 7f8a4ba00640 1 --2- 172.21.15.39:0/3142822858 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8a24078b90 0x7f8a2407b0a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8a440d0200 tx=0x7f8a30005f10 comp rx=0 tx=0).stop 2024-09-21T07:33:20.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.633+0000 7f8a4ba00640 1 -- 172.21.15.39:0/3142822858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a44070030 msgr2=0x7f8a440ce020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:20.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.633+0000 7f8a4ba00640 1 --2- 172.21.15.39:0/3142822858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a44070030 0x7f8a440ce020 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8a3800c3c0 tx=0x7f8a3800c880 comp rx=0 tx=0).stop 2024-09-21T07:33:20.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.633+0000 7f8a4ba00640 1 -- 172.21.15.39:0/3142822858 shutdown_connections 2024-09-21T07:33:20.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.633+0000 7f8a4ba00640 1 --2- 172.21.15.39:0/3142822858 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8a24078b90 0x7f8a2407b0a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:20.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.633+0000 7f8a4ba00640 1 --2- 172.21.15.39:0/3142822858 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a44073170 0x7f8a441f3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:20.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.633+0000 7f8a4ba00640 1 --2- 172.21.15.39:0/3142822858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a44070440 0x7f8a4415a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:20.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.633+0000 7f8a4ba00640 1 --2- 172.21.15.39:0/3142822858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a44070030 0x7f8a440ce020 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:20.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.633+0000 7f8a4ba00640 1 -- 172.21.15.39:0/3142822858 >> 172.21.15.39:0/3142822858 conn(0x7f8a440b1cb0 msgr2=0x7f8a4415eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:20.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.633+0000 7f8a4ba00640 1 -- 172.21.15.39:0/3142822858 shutdown_connections 2024-09-21T07:33:20.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.633+0000 7f8a4ba00640 1 -- 172.21.15.39:0/3142822858 wait complete. 2024-09-21T07:33:20.656 INFO:teuthology.orchestra.run.smithi039.stdout:[{"version":31,"timestamp":"2024-09-21T07:33:01.782181+0000","name":"","changes":[{"name":"osd.0/osd_mclock_max_capacity_iops_ssd","new_value":"3421.881759"}]}] 2024-09-21T07:33:20.656 INFO:tasks.ceph_manager:config epoch is 31 2024-09-21T07:33:20.657 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T07:33:20.657 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-21T07:33:20.657 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-21T07:33:20.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.880+0000 7fae2bc00640 1 -- 172.21.15.39:0/3315048658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae2415c6f0 msgr2=0x7fae240ceb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:20.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.880+0000 7fae2bc00640 1 --2- 172.21.15.39:0/3315048658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae2415c6f0 0x7fae240ceb50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fae240ce3e0 tx=0x7fae1002a560 comp rx=0 tx=0).stop 2024-09-21T07:33:20.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.880+0000 7fae2bc00640 1 -- 172.21.15.39:0/3315048658 shutdown_connections 2024-09-21T07:33:20.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.880+0000 7fae2bc00640 1 --2- 172.21.15.39:0/3315048658 >> v2:172.21.15.148:3300/0 conn(0x7fae240ccba0 0x7fae240ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:20.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.880+0000 7fae2bc00640 1 --2- 172.21.15.39:0/3315048658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae2415c6f0 0x7fae240ceb50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:20.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.880+0000 7fae2bc00640 1 --2- 172.21.15.39:0/3315048658 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae240d1d40 0x7fae240b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:20.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.880+0000 7fae2bc00640 1 -- 172.21.15.39:0/3315048658 >> 172.21.15.39:0/3315048658 conn(0x7fae240b3f10 msgr2=0x7fae240b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:20.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.880+0000 7fae2bc00640 1 -- 172.21.15.39:0/3315048658 shutdown_connections 2024-09-21T07:33:20.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.881+0000 7fae2bc00640 1 -- 172.21.15.39:0/3315048658 wait complete. 2024-09-21T07:33:20.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.883+0000 7fae2bc00640 1 Processor -- start 2024-09-21T07:33:20.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.883+0000 7fae2bc00640 1 -- start start 2024-09-21T07:33:20.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.884+0000 7fae2bc00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae240ccba0 0x7fae2406fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:20.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.884+0000 7fae2bc00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae240d1d40 0x7fae24148d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:20.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.884+0000 7fae2bc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae2415c6f0 0x7fae2406e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:20.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.884+0000 7fae2bc00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fae240d11f0 con 0x7fae2415c6f0 2024-09-21T07:33:20.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.884+0000 7fae2bc00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fae240d1070 con 0x7fae240d1d40 2024-09-21T07:33:20.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.884+0000 7fae2bc00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fae240d1370 con 0x7fae240ccba0 2024-09-21T07:33:20.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.885+0000 7fae29c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae2415c6f0 0x7fae2406e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:20.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.885+0000 7fae29200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae240ccba0 0x7fae2406fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:20.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.885+0000 7fae29c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae2415c6f0 0x7fae2406e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60562/0 (socket says 172.21.15.39:60562) 2024-09-21T07:33:20.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.885+0000 7fae29c00640 1 -- 172.21.15.39:0/3608219863 learned_addr learned my addr 172.21.15.39:0/3608219863 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:20.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.885+0000 7fae23e00640 1 --2- 172.21.15.39:0/3608219863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae240d1d40 0x7fae24148d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:20.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.885+0000 7fae29c00640 1 -- 172.21.15.39:0/3608219863 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae240ccba0 msgr2=0x7fae2406fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:20.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.885+0000 7fae29c00640 1 --2- 172.21.15.39:0/3608219863 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae240ccba0 0x7fae2406fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:20.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.885+0000 7fae29c00640 1 -- 172.21.15.39:0/3608219863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae240d1d40 msgr2=0x7fae24148d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:20.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.885+0000 7fae29c00640 1 --2- 172.21.15.39:0/3608219863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae240d1d40 0x7fae24148d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:20.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.885+0000 7fae29c00640 1 -- 172.21.15.39:0/3608219863 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae24149280 con 0x7fae2415c6f0 2024-09-21T07:33:20.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.885+0000 7fae29c00640 1 --2- 172.21.15.39:0/3608219863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae2415c6f0 0x7fae2406e820 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fae1c00e990 tx=0x7fae1c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:20.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.886+0000 7fae20c00640 1 -- 172.21.15.39:0/3608219863 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1c004d20 con 0x7fae2415c6f0 2024-09-21T07:33:20.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.886+0000 7fae20c00640 1 -- 172.21.15.39:0/3608219863 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae1c00caa0 con 0x7fae2415c6f0 2024-09-21T07:33:20.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.886+0000 7fae2bc00640 1 -- 172.21.15.39:0/3608219863 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae240a9540 con 0x7fae2415c6f0 2024-09-21T07:33:20.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.886+0000 7fae20c00640 1 -- 172.21.15.39:0/3608219863 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1c005020 con 0x7fae2415c6f0 2024-09-21T07:33:20.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.886+0000 7fae2bc00640 1 -- 172.21.15.39:0/3608219863 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae2406e020 con 0x7fae2415c6f0 2024-09-21T07:33:20.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.887+0000 7fae2bc00640 1 -- 172.21.15.39:0/3608219863 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae240d1900 con 0x7fae2415c6f0 2024-09-21T07:33:20.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.888+0000 7fae20c00640 1 -- 172.21.15.39:0/3608219863 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fae1c0051c0 con 0x7fae2415c6f0 2024-09-21T07:33:20.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.890+0000 7fae20c00640 1 --2- 172.21.15.39:0/3608219863 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae04078b90 0x7fae0407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:20.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.890+0000 7fae20c00640 1 -- 172.21.15.39:0/3608219863 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fae1c09b4f0 con 0x7fae2415c6f0 2024-09-21T07:33:20.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.890+0000 7fae29200640 1 --2- 172.21.15.39:0/3608219863 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae04078b90 0x7fae0407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:20.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.891+0000 7fae29200640 1 --2- 172.21.15.39:0/3608219863 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae04078b90 0x7fae0407b0a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fae18004630 tx=0x7fae1800b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:20.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:20.894+0000 7fae20c00640 1 -- 172.21.15.39:0/3608219863 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fae1c016030 con 0x7fae2415c6f0 2024-09-21T07:33:21.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.137+0000 7fae2bc00640 1 -- 172.21.15.39:0/3608219863 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fae240b5210 con 0x7fae2415c6f0 2024-09-21T07:33:21.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.141+0000 7fae20c00640 1 -- 172.21.15.39:0/3608219863 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+194717 (secure 0 0 0) 0x7fae1c0632d0 con 0x7fae2415c6f0 2024-09-21T07:33:21.145 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:33:21.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.144+0000 7fae2bc00640 1 -- 172.21.15.39:0/3608219863 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae04078b90 msgr2=0x7fae0407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:21.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.144+0000 7fae2bc00640 1 --2- 172.21.15.39:0/3608219863 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae04078b90 0x7fae0407b0a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fae18004630 tx=0x7fae1800b2e0 comp rx=0 tx=0).stop 2024-09-21T07:33:21.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.144+0000 7fae2bc00640 1 -- 172.21.15.39:0/3608219863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae2415c6f0 msgr2=0x7fae2406e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:21.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.144+0000 7fae2bc00640 1 --2- 172.21.15.39:0/3608219863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae2415c6f0 0x7fae2406e820 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fae1c00e990 tx=0x7fae1c00ee50 comp rx=0 tx=0).stop 2024-09-21T07:33:21.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.144+0000 7fae2bc00640 1 -- 172.21.15.39:0/3608219863 shutdown_connections 2024-09-21T07:33:21.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.144+0000 7fae2bc00640 1 --2- 172.21.15.39:0/3608219863 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae04078b90 0x7fae0407b0a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:21.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.144+0000 7fae2bc00640 1 --2- 172.21.15.39:0/3608219863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae2415c6f0 0x7fae2406e820 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:21.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.144+0000 7fae2bc00640 1 --2- 172.21.15.39:0/3608219863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae240d1d40 0x7fae24148d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:21.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.144+0000 7fae2bc00640 1 --2- 172.21.15.39:0/3608219863 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae240ccba0 0x7fae2406fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:21.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.144+0000 7fae2bc00640 1 -- 172.21.15.39:0/3608219863 >> 172.21.15.39:0/3608219863 conn(0x7fae240b3f10 msgr2=0x7fae240d22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:21.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.144+0000 7fae2bc00640 1 -- 172.21.15.39:0/3608219863 shutdown_connections 2024-09-21T07:33:21.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.144+0000 7fae2bc00640 1 -- 172.21.15.39:0/3608219863 wait complete. 2024-09-21T07:33:21.168 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":5,"flags":0,"active_gid":4115,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":896204996},{"type":"v1","addr":"172.21.15.148:6801","nonce":896204996}]},"active_addr":"172.21.15.148:6801/896204996","active_change":"2024-09-21T07:32:51.519843+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4121,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":358942158}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":3259316685}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":982113022}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":4006317108}]}]} 2024-09-21T07:33:21.173 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-21T07:33:21.173 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-21T07:33:21.173 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:33:21.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.396+0000 7fadb1600640 1 -- 172.21.15.39:0/24343248 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fadac0b5a20 msgr2=0x7fadac0b3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:21.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.396+0000 7fadb1600640 1 --2- 172.21.15.39:0/24343248 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fadac0b5a20 0x7fadac0b3b50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fad94009860 tx=0x7fad9402a5a0 comp rx=0 tx=0).stop 2024-09-21T07:33:21.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.396+0000 7fadb1600640 1 -- 172.21.15.39:0/24343248 shutdown_connections 2024-09-21T07:33:21.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.396+0000 7fadb1600640 1 --2- 172.21.15.39:0/24343248 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fadac0b5a20 0x7fadac0b3b50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:21.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.396+0000 7fadb1600640 1 -- 172.21.15.39:0/24343248 >> 172.21.15.39:0/24343248 conn(0x7fadac09d6a0 msgr2=0x7fadac0d43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:21.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.396+0000 7fadb1600640 1 -- 172.21.15.39:0/24343248 shutdown_connections 2024-09-21T07:33:21.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.396+0000 7fadb1600640 1 -- 172.21.15.39:0/24343248 wait complete. 2024-09-21T07:33:21.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.398+0000 7fadb1600640 1 Processor -- start 2024-09-21T07:33:21.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.398+0000 7fadb1600640 1 -- start start 2024-09-21T07:33:21.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.400+0000 7fadb1600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadac0b5a20 0x7fadac1f6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:21.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.400+0000 7fadb1600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fadac162470 0x7fadac15b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:21.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.400+0000 7fadb1600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadac163110 0x7fadac0d3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:21.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.400+0000 7fadb1600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fadac0cffa0 con 0x7fadac163110 2024-09-21T07:33:21.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.400+0000 7fadb1600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fadac0cfe20 con 0x7fadac0b5a20 2024-09-21T07:33:21.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.400+0000 7fadb1600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fadac0d0120 con 0x7fadac162470 2024-09-21T07:33:21.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.400+0000 7fadb0c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadac163110 0x7fadac0d3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:21.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.400+0000 7fadb0c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadac163110 0x7fadac0d3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60576/0 (socket says 172.21.15.39:60576) 2024-09-21T07:33:21.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.400+0000 7fadb0c00640 1 -- 172.21.15.39:0/2131824795 learned_addr learned my addr 172.21.15.39:0/2131824795 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:21.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.400+0000 7fadabe00640 1 --2- 172.21.15.39:0/2131824795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadac0b5a20 0x7fadac1f6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:21.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.400+0000 7fada3400640 1 --2- 172.21.15.39:0/2131824795 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fadac162470 0x7fadac15b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:21.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.401+0000 7fadb0c00640 1 -- 172.21.15.39:0/2131824795 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fadac162470 msgr2=0x7fadac15b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:21.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.401+0000 7fadb0c00640 1 --2- 172.21.15.39:0/2131824795 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fadac162470 0x7fadac15b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:21.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.401+0000 7fadb0c00640 1 -- 172.21.15.39:0/2131824795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadac0b5a20 msgr2=0x7fadac1f6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:21.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.401+0000 7fadb0c00640 1 --2- 172.21.15.39:0/2131824795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadac0b5a20 0x7fadac1f6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:21.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.401+0000 7fadb0c00640 1 -- 172.21.15.39:0/2131824795 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadac1f54b0 con 0x7fadac163110 2024-09-21T07:33:21.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.401+0000 7fadb0c00640 1 --2- 172.21.15.39:0/2131824795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadac163110 0x7fadac0d3fe0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fada400cac0 tx=0x7fada400cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:21.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.401+0000 7fada8c00640 1 -- 172.21.15.39:0/2131824795 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada4016020 con 0x7fadac163110 2024-09-21T07:33:21.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.401+0000 7fada8c00640 1 -- 172.21.15.39:0/2131824795 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fada400ace0 con 0x7fadac163110 2024-09-21T07:33:21.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.401+0000 7fada8c00640 1 -- 172.21.15.39:0/2131824795 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada4012680 con 0x7fadac163110 2024-09-21T07:33:21.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.401+0000 7fadb1600640 1 -- 172.21.15.39:0/2131824795 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadac15c020 con 0x7fadac163110 2024-09-21T07:33:21.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.401+0000 7fadb1600640 1 -- 172.21.15.39:0/2131824795 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadac15bd40 con 0x7fadac163110 2024-09-21T07:33:21.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.402+0000 7fadb1600640 1 -- 172.21.15.39:0/2131824795 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadac0b3b50 con 0x7fadac163110 2024-09-21T07:33:21.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.404+0000 7fada8c00640 1 -- 172.21.15.39:0/2131824795 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fada4019020 con 0x7fadac163110 2024-09-21T07:33:21.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.405+0000 7fada8c00640 1 --2- 172.21.15.39:0/2131824795 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fad88078bc0 0x7fad8807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:21.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.405+0000 7fada8c00640 1 -- 172.21.15.39:0/2131824795 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fada409b260 con 0x7fadac163110 2024-09-21T07:33:21.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.406+0000 7fadabe00640 1 --2- 172.21.15.39:0/2131824795 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fad88078bc0 0x7fad8807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:21.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.406+0000 7fadabe00640 1 --2- 172.21.15.39:0/2131824795 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fad88078bc0 0x7fad8807b0d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fad9c007fd0 tx=0x7fad9c009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:21.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.410+0000 7fada8c00640 1 -- 172.21.15.39:0/2131824795 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fada4020020 con 0x7fadac163110 2024-09-21T07:33:21.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.608+0000 7fadb1600640 1 -- 172.21.15.39:0/2131824795 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fadac08e450 con 0x7fadac163110 2024-09-21T07:33:21.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.609+0000 7fada8c00640 1 -- 172.21.15.39:0/2131824795 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16665 (secure 0 0 0) 0x7fadac15c020 con 0x7fadac163110 2024-09-21T07:33:21.611 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:33:21.611 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":21,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:33:16.159267+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:33:12.854113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:33:13.622611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:33:21.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.611+0000 7fadb1600640 1 -- 172.21.15.39:0/2131824795 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fad88078bc0 msgr2=0x7fad8807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:21.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.611+0000 7fadb1600640 1 --2- 172.21.15.39:0/2131824795 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fad88078bc0 0x7fad8807b0d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fad9c007fd0 tx=0x7fad9c009040 comp rx=0 tx=0).stop 2024-09-21T07:33:21.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.611+0000 7fadb1600640 1 -- 172.21.15.39:0/2131824795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadac163110 msgr2=0x7fadac0d3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:21.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.611+0000 7fadb1600640 1 --2- 172.21.15.39:0/2131824795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadac163110 0x7fadac0d3fe0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fada400cac0 tx=0x7fada400cf80 comp rx=0 tx=0).stop 2024-09-21T07:33:21.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.611+0000 7fadb1600640 1 -- 172.21.15.39:0/2131824795 shutdown_connections 2024-09-21T07:33:21.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.612+0000 7fadb1600640 1 --2- 172.21.15.39:0/2131824795 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fad88078bc0 0x7fad8807b0d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:21.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.612+0000 7fadb1600640 1 --2- 172.21.15.39:0/2131824795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadac163110 0x7fadac0d3fe0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:21.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.612+0000 7fadb1600640 1 --2- 172.21.15.39:0/2131824795 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fadac162470 0x7fadac15b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:21.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.612+0000 7fadb1600640 1 --2- 172.21.15.39:0/2131824795 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadac0b5a20 0x7fadac1f6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:21.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.612+0000 7fadb1600640 1 -- 172.21.15.39:0/2131824795 >> 172.21.15.39:0/2131824795 conn(0x7fadac09d6a0 msgr2=0x7fadac068d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:21.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.612+0000 7fadb1600640 1 -- 172.21.15.39:0/2131824795 shutdown_connections 2024-09-21T07:33:21.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.612+0000 7fadb1600640 1 -- 172.21.15.39:0/2131824795 wait complete. 2024-09-21T07:33:21.638 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-21T07:33:21.638 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:33:21.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.862+0000 7fe9fc800640 1 -- 172.21.15.39:0/583354259 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9f40706b0 msgr2=0x7fe9f40cba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:21.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.862+0000 7fe9fc800640 1 --2- 172.21.15.39:0/583354259 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9f40706b0 0x7fe9f40cba00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4005fd0 tx=0x7fe9e402a5d0 comp rx=0 tx=0).stop 2024-09-21T07:33:21.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.862+0000 7fe9fc800640 1 -- 172.21.15.39:0/583354259 shutdown_connections 2024-09-21T07:33:21.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.862+0000 7fe9fc800640 1 --2- 172.21.15.39:0/583354259 >> v2:172.21.15.148:3300/0 conn(0x7fe9f40cad30 0x7fe9f40735b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:21.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.862+0000 7fe9fc800640 1 --2- 172.21.15.39:0/583354259 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9f40706b0 0x7fe9f40cba00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:21.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.862+0000 7fe9fc800640 1 -- 172.21.15.39:0/583354259 >> 172.21.15.39:0/583354259 conn(0x7fe9f40b1cb0 msgr2=0x7fe9f40b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:21.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.862+0000 7fe9fc800640 1 -- 172.21.15.39:0/583354259 shutdown_connections 2024-09-21T07:33:21.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.862+0000 7fe9fc800640 1 -- 172.21.15.39:0/583354259 wait complete. 2024-09-21T07:33:21.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.864+0000 7fe9fc800640 1 Processor -- start 2024-09-21T07:33:21.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.864+0000 7fe9fc800640 1 -- start start 2024-09-21T07:33:21.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.866+0000 7fe9fc800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9f40ca800 0x7fe9f4159750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:21.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.866+0000 7fe9fc800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9f40cad30 0x7fe9f40ae090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:21.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.866+0000 7fe9fc800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9f40ca2f0 0x7fe9f40ae5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:21.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.866+0000 7fe9fc800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f4073f00 con 0x7fe9f40ca2f0 2024-09-21T07:33:21.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.866+0000 7fe9fc800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f4073d80 con 0x7fe9f40cad30 2024-09-21T07:33:21.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.866+0000 7fe9fc800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe9f4074080 con 0x7fe9f40ca800 2024-09-21T07:33:21.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.866+0000 7fe9fa800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9f40ca2f0 0x7fe9f40ae5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:21.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.866+0000 7fe9fa800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9f40ca2f0 0x7fe9f40ae5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60578/0 (socket says 172.21.15.39:60578) 2024-09-21T07:33:21.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.866+0000 7fe9fa800640 1 -- 172.21.15.39:0/150183285 learned_addr learned my addr 172.21.15.39:0/150183285 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:21.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.866+0000 7fe9f9400640 1 --2- 172.21.15.39:0/150183285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9f40cad30 0x7fe9f40ae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:21.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.866+0000 7fe9f9e00640 1 --2- 172.21.15.39:0/150183285 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9f40ca800 0x7fe9f4159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:21.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.866+0000 7fe9fa800640 1 -- 172.21.15.39:0/150183285 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9f40ca800 msgr2=0x7fe9f4159750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:21.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.866+0000 7fe9fa800640 1 --2- 172.21.15.39:0/150183285 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9f40ca800 0x7fe9f4159750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:21.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.866+0000 7fe9fa800640 1 -- 172.21.15.39:0/150183285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9f40cad30 msgr2=0x7fe9f40ae090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:21.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.867+0000 7fe9fa800640 1 --2- 172.21.15.39:0/150183285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9f40cad30 0x7fe9f40ae090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:21.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.867+0000 7fe9fa800640 1 -- 172.21.15.39:0/150183285 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9f40c7660 con 0x7fe9f40ca2f0 2024-09-21T07:33:21.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.867+0000 7fe9fa800640 1 --2- 172.21.15.39:0/150183285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9f40ca2f0 0x7fe9f40ae5d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe9f000ec20 tx=0x7fe9f000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:21.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.867+0000 7fe9ee000640 1 -- 172.21.15.39:0/150183285 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0010630 con 0x7fe9f40ca2f0 2024-09-21T07:33:21.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.867+0000 7fe9ee000640 1 -- 172.21.15.39:0/150183285 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe9f00107d0 con 0x7fe9f40ca2f0 2024-09-21T07:33:21.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.867+0000 7fe9fc800640 1 -- 172.21.15.39:0/150183285 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9f41f33b0 con 0x7fe9f40ca2f0 2024-09-21T07:33:21.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.867+0000 7fe9ee000640 1 -- 172.21.15.39:0/150183285 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0010c20 con 0x7fe9f40ca2f0 2024-09-21T07:33:21.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.867+0000 7fe9fc800640 1 -- 172.21.15.39:0/150183285 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9f4159c90 con 0x7fe9f40ca2f0 2024-09-21T07:33:21.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.868+0000 7fe9fc800640 1 -- 172.21.15.39:0/150183285 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9f41f31c0 con 0x7fe9f40ca2f0 2024-09-21T07:33:21.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.870+0000 7fe9ee000640 1 -- 172.21.15.39:0/150183285 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe9f0021440 con 0x7fe9f40ca2f0 2024-09-21T07:33:21.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.870+0000 7fe9ee000640 1 --2- 172.21.15.39:0/150183285 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe9d8078b90 0x7fe9d807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:21.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.871+0000 7fe9ee000640 1 -- 172.21.15.39:0/150183285 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe9f009baf0 con 0x7fe9f40ca2f0 2024-09-21T07:33:21.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.871+0000 7fe9f9e00640 1 --2- 172.21.15.39:0/150183285 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe9d8078b90 0x7fe9d807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:21.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.872+0000 7fe9f9e00640 1 --2- 172.21.15.39:0/150183285 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe9d8078b90 0x7fe9d807b0a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe9f40cb250 tx=0x7fe9e8009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:21.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:21.875+0000 7fe9ee000640 1 -- 172.21.15.39:0/150183285 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe9f0063960 con 0x7fe9f40ca2f0 2024-09-21T07:33:22.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.071+0000 7fe9fc800640 1 -- 172.21.15.39:0/150183285 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe9f40c73f0 con 0x7fe9f40ca2f0 2024-09-21T07:33:22.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.073+0000 7fe9ee000640 1 -- 172.21.15.39:0/150183285 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16665 (secure 0 0 0) 0x7fe9f0068390 con 0x7fe9f40ca2f0 2024-09-21T07:33:22.075 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:33:22.075 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":21,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:33:16.159267+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:33:12.854113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:33:13.622611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:33:22.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.075+0000 7fe9fc800640 1 -- 172.21.15.39:0/150183285 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe9d8078b90 msgr2=0x7fe9d807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.075+0000 7fe9fc800640 1 --2- 172.21.15.39:0/150183285 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe9d8078b90 0x7fe9d807b0a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe9f40cb250 tx=0x7fe9e8009040 comp rx=0 tx=0).stop 2024-09-21T07:33:22.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.075+0000 7fe9fc800640 1 -- 172.21.15.39:0/150183285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9f40ca2f0 msgr2=0x7fe9f40ae5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.075+0000 7fe9fc800640 1 --2- 172.21.15.39:0/150183285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9f40ca2f0 0x7fe9f40ae5d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe9f000ec20 tx=0x7fe9f000c6a0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.075+0000 7fe9fc800640 1 -- 172.21.15.39:0/150183285 shutdown_connections 2024-09-21T07:33:22.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.075+0000 7fe9fc800640 1 --2- 172.21.15.39:0/150183285 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe9d8078b90 0x7fe9d807b0a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.075+0000 7fe9fc800640 1 --2- 172.21.15.39:0/150183285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9f40ca2f0 0x7fe9f40ae5d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.075+0000 7fe9fc800640 1 --2- 172.21.15.39:0/150183285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9f40cad30 0x7fe9f40ae090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.075+0000 7fe9fc800640 1 --2- 172.21.15.39:0/150183285 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9f40ca800 0x7fe9f4159750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.075+0000 7fe9fc800640 1 -- 172.21.15.39:0/150183285 >> 172.21.15.39:0/150183285 conn(0x7fe9f40b1cb0 msgr2=0x7fe9f415b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:22.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.075+0000 7fe9fc800640 1 -- 172.21.15.39:0/150183285 shutdown_connections 2024-09-21T07:33:22.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.075+0000 7fe9fc800640 1 -- 172.21.15.39:0/150183285 wait complete. 2024-09-21T07:33:22.099 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-21T07:33:22.099 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-21T07:33:22.100 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-21T07:33:22.100 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-21T07:33:22.100 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-21T07:33:22.101 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-21T07:33:22.101 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-21T07:33:22.101 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-21T07:33:22.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.511+0000 7f460ce00640 1 -- 172.21.15.39:0/1110206515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45ec004800 msgr2=0x7f45ec004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.511+0000 7f460ce00640 1 --2- 172.21.15.39:0/1110206515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45ec004800 0x7f45ec004c10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f45f800c3c0 tx=0x7f45ec005830 comp rx=0 tx=0).stop 2024-09-21T07:33:22.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.511+0000 7f460ce00640 1 -- 172.21.15.39:0/1110206515 shutdown_connections 2024-09-21T07:33:22.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.511+0000 7f460ce00640 1 --2- 172.21.15.39:0/1110206515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45ec004800 0x7f45ec004c10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.511+0000 7f460ce00640 1 -- 172.21.15.39:0/1110206515 >> 172.21.15.39:0/1110206515 conn(0x7f46080b1cb0 msgr2=0x7f46080b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:22.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.511+0000 7fdfb6c00640 1 -- 172.21.15.39:0/3504939639 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdfb00d0340 msgr2=0x7fdfb00d0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.511+0000 7fdfb6c00640 1 --2- 172.21.15.39:0/3504939639 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdfb00d0340 0x7fdfb00d0750 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8005fd0 tx=0x7fdfa802aa10 comp rx=0 tx=0).stop 2024-09-21T07:33:22.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.511+0000 7f059e600640 1 -- 172.21.15.39:0/3933686508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0598074950 msgr2=0x7f05981622a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.511+0000 7f059e600640 1 --2- 172.21.15.39:0/3933686508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0598074950 0x7f05981622a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0588009830 tx=0x7f058802a530 comp rx=0 tx=0).stop 2024-09-21T07:33:22.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.512+0000 7fdfb6c00640 1 -- 172.21.15.39:0/3504939639 shutdown_connections 2024-09-21T07:33:22.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.512+0000 7fdfb6c00640 1 --2- 172.21.15.39:0/3504939639 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdfb00d0340 0x7fdfb00d0750 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.512+0000 7fdfb6c00640 1 --2- 172.21.15.39:0/3504939639 >> v2:172.21.15.148:3300/0 conn(0x7fdfb015ca30 0x7fdfb015ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.512+0000 7fdfb6c00640 1 -- 172.21.15.39:0/3504939639 >> 172.21.15.39:0/3504939639 conn(0x7fdfb0070b20 msgr2=0x7fdfb0070f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:22.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.512+0000 7fdfb6c00640 1 -- 172.21.15.39:0/3504939639 shutdown_connections 2024-09-21T07:33:22.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.512+0000 7fdfb6c00640 1 -- 172.21.15.39:0/3504939639 wait complete. 2024-09-21T07:33:22.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.513+0000 7f460ce00640 1 -- 172.21.15.39:0/1110206515 shutdown_connections 2024-09-21T07:33:22.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.513+0000 7f460ce00640 1 -- 172.21.15.39:0/1110206515 wait complete. 2024-09-21T07:33:22.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.513+0000 7f059e600640 1 -- 172.21.15.39:0/3933686508 shutdown_connections 2024-09-21T07:33:22.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.513+0000 7f059e600640 1 --2- 172.21.15.39:0/3933686508 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f059815e610 0x7f05980b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.513+0000 7f059e600640 1 --2- 172.21.15.39:0/3933686508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0598074950 0x7f05981622a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.513+0000 7f059e600640 1 -- 172.21.15.39:0/3933686508 >> 172.21.15.39:0/3933686508 conn(0x7f05980d0000 msgr2=0x7f05980d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:22.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.514+0000 7f059e600640 1 -- 172.21.15.39:0/3933686508 shutdown_connections 2024-09-21T07:33:22.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.514+0000 7f059e600640 1 -- 172.21.15.39:0/3933686508 wait complete. 2024-09-21T07:33:22.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.516+0000 7f5975c00640 1 -- 172.21.15.39:0/2154980548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5968036b10 msgr2=0x7f5968005c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.516+0000 7f5975c00640 1 --2- 172.21.15.39:0/2154980548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5968036b10 0x7f5968005c60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f596800ca70 tx=0x7f596402a540 comp rx=0 tx=0).stop 2024-09-21T07:33:22.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.516+0000 7f5975c00640 1 -- 172.21.15.39:0/2154980548 shutdown_connections 2024-09-21T07:33:22.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.516+0000 7f5975c00640 1 --2- 172.21.15.39:0/2154980548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5968036b10 0x7f5968005c60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.516+0000 7f5975c00640 1 --2- 172.21.15.39:0/2154980548 >> v2:172.21.15.148:3300/0 conn(0x7f596800bc30 0x7f59680df180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.516+0000 7f5975c00640 1 -- 172.21.15.39:0/2154980548 >> 172.21.15.39:0/2154980548 conn(0x7f5968051420 msgr2=0x7f5968051860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:22.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.516+0000 7f5975c00640 1 -- 172.21.15.39:0/2154980548 shutdown_connections 2024-09-21T07:33:22.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.517+0000 7f5975c00640 1 -- 172.21.15.39:0/2154980548 wait complete. 2024-09-21T07:33:22.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.517+0000 7f460ce00640 1 Processor -- start 2024-09-21T07:33:22.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.517+0000 7f460ce00640 1 -- start start 2024-09-21T07:33:22.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.517+0000 7fdfb6c00640 1 Processor -- start 2024-09-21T07:33:22.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.518+0000 7fdfb6c00640 1 -- start start 2024-09-21T07:33:22.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.518+0000 7f059e600640 1 Processor -- start 2024-09-21T07:33:22.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.518+0000 7f059e600640 1 -- start start 2024-09-21T07:33:22.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.519+0000 7f460ce00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45ec004800 0x7f460815d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:22.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.520+0000 7f460ce00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4608154a50 0x7f4608159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:22.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.520+0000 7f460ce00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f460815a220 0x7f46080c3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:22.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.520+0000 7f460ce00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f46080cf010 con 0x7f4608154a50 2024-09-21T07:33:22.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.520+0000 7f460ce00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f46080706c0 con 0x7f45ec004800 2024-09-21T07:33:22.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.520+0000 7f460ce00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f46080cf190 con 0x7f460815a220 2024-09-21T07:33:22.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.520+0000 7f5975c00640 1 Processor -- start 2024-09-21T07:33:22.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.520+0000 7f5975c00640 1 -- start start 2024-09-21T07:33:22.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.520+0000 7f4605e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45ec004800 0x7f460815d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:22.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.520+0000 7f4605e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45ec004800 0x7f460815d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.39:58894/0 (socket says 172.21.15.39:58894) 2024-09-21T07:33:22.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.520+0000 7f4605e00640 1 -- 172.21.15.39:0/199935886 learned_addr learned my addr 172.21.15.39:0/199935886 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:22.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.520+0000 7f4606800640 1 --2- 172.21.15.39:0/199935886 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f460815a220 0x7f46080c3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:22.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.521+0000 7f4605400640 1 --2- 172.21.15.39:0/199935886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4608154a50 0x7f4608159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:22.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.521+0000 7f059e600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0598074950 0x7f05981fa830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:22.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.521+0000 7f059e600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f05980b3b70 0x7f0598149080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:22.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.521+0000 7f059e600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f059815e610 0x7f0598160ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:22.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.521+0000 7f4605e00640 1 -- 172.21.15.39:0/199935886 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f460815a220 msgr2=0x7f46080c3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.521+0000 7f4605e00640 1 --2- 172.21.15.39:0/199935886 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f460815a220 0x7f46080c3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.521+0000 7f4605e00640 1 -- 172.21.15.39:0/199935886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4608154a50 msgr2=0x7f4608159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.521+0000 7f4605e00640 1 --2- 172.21.15.39:0/199935886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4608154a50 0x7f4608159ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.521+0000 7f4605e00640 1 -- 172.21.15.39:0/199935886 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f460815a630 con 0x7f45ec004800 2024-09-21T07:33:22.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.521+0000 7f059e600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0598073f10 con 0x7f059815e610 2024-09-21T07:33:22.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.521+0000 7f059e600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0598073d90 con 0x7f0598074950 2024-09-21T07:33:22.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.521+0000 7f059e600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f0598074090 con 0x7f05980b3b70 2024-09-21T07:33:22.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.522+0000 7f0597400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0598074950 0x7f05981fa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:22.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.522+0000 7f0597400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0598074950 0x7f05981fa830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.39:58908/0 (socket says 172.21.15.39:58908) 2024-09-21T07:33:22.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.522+0000 7f0597400640 1 -- 172.21.15.39:0/2629231421 learned_addr learned my addr 172.21.15.39:0/2629231421 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:22.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.522+0000 7fdfb6c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdfb006f9f0 0x7fdfb00b2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:22.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.522+0000 7fdfb6c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfb00d0340 0x7fdfb01fd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:22.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.522+0000 7fdfb6c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfb015ca30 0x7fdfb01fd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:22.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.522+0000 7f4605400640 1 --2- 172.21.15.39:0/199935886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4608154a50 0x7f4608159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:33:22.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.522+0000 7f0597e00640 1 --2- 172.21.15.39:0/2629231421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f059815e610 0x7f0598160ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:22.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.522+0000 7f0596a00640 1 --2- 172.21.15.39:0/2629231421 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f05980b3b70 0x7f0598149080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:22.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.522+0000 7fdfb6c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb00ccc50 con 0x7fdfb00d0340 2024-09-21T07:33:22.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.522+0000 7fdfb6c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb00ccad0 con 0x7fdfb015ca30 2024-09-21T07:33:22.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.522+0000 7fdfb6c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fdfb00ccdd0 con 0x7fdfb006f9f0 2024-09-21T07:33:22.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.522+0000 7fdfb4e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfb00d0340 0x7fdfb01fd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:22.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.523+0000 7f4605e00640 1 --2- 172.21.15.39:0/199935886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45ec004800 0x7f460815d710 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f45f000d730 tx=0x7f45f000dbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:22.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.523+0000 7fdfb6200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfb015ca30 0x7fdfb01fd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:22.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.523+0000 7fdfb5800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdfb006f9f0 0x7fdfb00b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:22.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.523+0000 7fdfb6200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfb015ca30 0x7fdfb01fd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.39:58910/0 (socket says 172.21.15.39:58910) 2024-09-21T07:33:22.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.523+0000 7fdfb6200640 1 -- 172.21.15.39:0/2434118133 learned_addr learned my addr 172.21.15.39:0/2434118133 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:22.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.523+0000 7f0597400640 1 -- 172.21.15.39:0/2629231421 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f05980b3b70 msgr2=0x7f0598149080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.523+0000 7f0597400640 1 --2- 172.21.15.39:0/2629231421 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f05980b3b70 0x7f0598149080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.523+0000 7fdfb4e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfb00d0340 0x7fdfb01fd460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59950/0 (socket says 172.21.15.39:59950) 2024-09-21T07:33:22.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.523+0000 7f0597400640 1 -- 172.21.15.39:0/2629231421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f059815e610 msgr2=0x7f0598160ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.523+0000 7f5975c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5968009ba0 0x7f59680dd820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:22.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.523+0000 7f5975c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f596800bc30 0x7f59680ed6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:22.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.523+0000 7f5975c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5968036b10 0x7f59680edc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:22.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.523+0000 7f5975c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f59680069f0 con 0x7f5968009ba0 2024-09-21T07:33:22.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.523+0000 7f5975c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5968006870 con 0x7f596800bc30 2024-09-21T07:33:22.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.523+0000 7f5975c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f5968006b70 con 0x7f5968036b10 2024-09-21T07:33:22.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.523+0000 7f0597400640 1 --2- 172.21.15.39:0/2629231421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f059815e610 0x7f0598160ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.523+0000 7f596fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5968009ba0 0x7f59680dd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:22.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.523+0000 7f596fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5968009ba0 0x7f59680dd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59960/0 (socket says 172.21.15.39:59960) 2024-09-21T07:33:22.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.523+0000 7f596fe00640 1 -- 172.21.15.39:0/13597896 learned_addr learned my addr 172.21.15.39:0/13597896 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:22.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.524+0000 7f5975200640 1 --2- 172.21.15.39:0/13597896 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5968036b10 0x7f59680edc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:22.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.524+0000 7f45fe000640 1 -- 172.21.15.39:0/199935886 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f0017070 con 0x7f45ec004800 2024-09-21T07:33:22.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.523+0000 7f0597400640 1 -- 172.21.15.39:0/2629231421 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05981fad70 con 0x7f0598074950 2024-09-21T07:33:22.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.523+0000 7f0597e00640 1 --2- 172.21.15.39:0/2629231421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f059815e610 0x7f0598160ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:33:22.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.524+0000 7f596f400640 1 --2- 172.21.15.39:0/13597896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f596800bc30 0x7f59680ed6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:22.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.524+0000 7fdfb4e00640 1 -- 172.21.15.39:0/2434118133 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdfb006f9f0 msgr2=0x7fdfb00b2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.524+0000 7f460ce00640 1 -- 172.21.15.39:0/199935886 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f460815dc50 con 0x7f45ec004800 2024-09-21T07:33:22.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.524+0000 7f45fe000640 1 -- 172.21.15.39:0/199935886 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45f0009d40 con 0x7f45ec004800 2024-09-21T07:33:22.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.524+0000 7f460ce00640 1 -- 172.21.15.39:0/199935886 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f460809ad10 con 0x7f45ec004800 2024-09-21T07:33:22.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.524+0000 7fdfb4e00640 1 --2- 172.21.15.39:0/2434118133 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdfb006f9f0 0x7fdfb00b2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.524+0000 7fdfb4e00640 1 -- 172.21.15.39:0/2434118133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfb015ca30 msgr2=0x7fdfb01fd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.524+0000 7fdfb4e00640 1 --2- 172.21.15.39:0/2434118133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfb015ca30 0x7fdfb01fd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.524+0000 7fdfb4e00640 1 -- 172.21.15.39:0/2434118133 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb01fafa0 con 0x7fdfb00d0340 2024-09-21T07:33:22.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.524+0000 7fdfb5800640 1 --2- 172.21.15.39:0/2434118133 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdfb006f9f0 0x7fdfb00b2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:33:22.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.524+0000 7f0597400640 1 --2- 172.21.15.39:0/2629231421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0598074950 0x7f05981fa830 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0588000c00 tx=0x7f0588033ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:22.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.525+0000 7fdfb4e00640 1 --2- 172.21.15.39:0/2434118133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfb00d0340 0x7fdfb01fd460 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c00d720 tx=0x7fdf9c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:22.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.525+0000 7fdfad600640 1 -- 172.21.15.39:0/2434118133 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c017070 con 0x7fdfb00d0340 2024-09-21T07:33:22.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.525+0000 7fdfb6c00640 1 -- 172.21.15.39:0/2434118133 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfb013f1e0 con 0x7fdfb00d0340 2024-09-21T07:33:22.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.525+0000 7f058f400640 1 -- 172.21.15.39:0/2629231421 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0588031d60 con 0x7f0598074950 2024-09-21T07:33:22.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.526+0000 7fdfb6c00640 1 -- 172.21.15.39:0/2434118133 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfb00aff80 con 0x7fdfb00d0340 2024-09-21T07:33:22.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.526+0000 7f460ce00640 1 -- 172.21.15.39:0/199935886 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f45e0001030 con 0x7f45ec004800 2024-09-21T07:33:22.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.526+0000 7f45fe000640 1 -- 172.21.15.39:0/199935886 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f00106c0 con 0x7f45ec004800 2024-09-21T07:33:22.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.526+0000 7f058f400640 1 -- 172.21.15.39:0/2629231421 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05880330d0 con 0x7f0598074950 2024-09-21T07:33:22.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.526+0000 7f596f400640 1 -- 172.21.15.39:0/13597896 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5968036b10 msgr2=0x7f59680edc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.527+0000 7fdfb6c00640 1 -- 172.21.15.39:0/2434118133 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdfb0074b80 con 0x7fdfb00d0340 2024-09-21T07:33:22.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.527+0000 7fdfad600640 1 -- 172.21.15.39:0/2434118133 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfb0074b80 con 0x7fdfb00d0340 2024-09-21T07:33:22.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.526+0000 7f059e600640 1 -- 172.21.15.39:0/2629231421 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05980a9a70 con 0x7f0598074950 2024-09-21T07:33:22.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.526+0000 7f059e600640 1 -- 172.21.15.39:0/2629231421 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05981fb4a0 con 0x7f0598074950 2024-09-21T07:33:22.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.527+0000 7f058f400640 1 -- 172.21.15.39:0/2629231421 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0588033500 con 0x7f0598074950 2024-09-21T07:33:22.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.527+0000 7f596f400640 1 --2- 172.21.15.39:0/13597896 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5968036b10 0x7f59680edc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.527+0000 7f596f400640 1 -- 172.21.15.39:0/13597896 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5968009ba0 msgr2=0x7f59680dd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.527+0000 7f596f400640 1 --2- 172.21.15.39:0/13597896 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5968009ba0 0x7f59680dd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.527+0000 7f596f400640 1 -- 172.21.15.39:0/13597896 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59680390f0 con 0x7f596800bc30 2024-09-21T07:33:22.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.528+0000 7f5975200640 1 --2- 172.21.15.39:0/13597896 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5968036b10 0x7f59680edc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:33:22.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.529+0000 7f596fe00640 1 --2- 172.21.15.39:0/13597896 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5968009ba0 0x7f59680dd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:33:22.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.527+0000 7fdfad600640 1 -- 172.21.15.39:0/2434118133 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c021060 con 0x7fdfb00d0340 2024-09-21T07:33:22.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.530+0000 7f45fe000640 1 -- 172.21.15.39:0/199935886 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f45f001a070 con 0x7f45ec004800 2024-09-21T07:33:22.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.529+0000 7f596f400640 1 --2- 172.21.15.39:0/13597896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f596800bc30 0x7f59680ed6e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f595c00c3c0 tx=0x7f595c00c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:22.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.530+0000 7f5963e00640 1 -- 172.21.15.39:0/13597896 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c011790 con 0x7f596800bc30 2024-09-21T07:33:22.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.530+0000 7f5963e00640 1 -- 172.21.15.39:0/13597896 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f595c017070 con 0x7f596800bc30 2024-09-21T07:33:22.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.530+0000 7f5963e00640 1 -- 172.21.15.39:0/13597896 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c011cc0 con 0x7f596800bc30 2024-09-21T07:33:22.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.530+0000 7f2d51e00640 1 -- 172.21.15.39:0/2503300862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d4c0b3c90 msgr2=0x7f2d4c161e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.530+0000 7f2d51e00640 1 --2- 172.21.15.39:0/2503300862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d4c0b3c90 0x7f2d4c161e10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2d44009830 tx=0x7f2d4402a510 comp rx=0 tx=0).stop 2024-09-21T07:33:22.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.530+0000 7f5975c00640 1 -- 172.21.15.39:0/13597896 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59680efcc0 con 0x7f596800bc30 2024-09-21T07:33:22.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.531+0000 7f5975c00640 1 -- 172.21.15.39:0/13597896 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5968022190 con 0x7f596800bc30 2024-09-21T07:33:22.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.531+0000 7f2d51e00640 1 -- 172.21.15.39:0/2503300862 shutdown_connections 2024-09-21T07:33:22.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.531+0000 7f2d51e00640 1 --2- 172.21.15.39:0/2503300862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d4c0b3c90 0x7f2d4c161e10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.531+0000 7f2d51e00640 1 -- 172.21.15.39:0/2503300862 >> 172.21.15.39:0/2503300862 conn(0x7f2d4c08bfc0 msgr2=0x7f2d4c09d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:22.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.531+0000 7f7285800640 1 -- 172.21.15.39:0/3948693107 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72800b2cb0 msgr2=0x7f72800b30c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.531+0000 7f7285800640 1 --2- 172.21.15.39:0/3948693107 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72800b2cb0 0x7f72800b30c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7280075570 tx=0x7f727802a4f0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.531+0000 7f2d51e00640 1 -- 172.21.15.39:0/2503300862 shutdown_connections 2024-09-21T07:33:22.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.530+0000 7f059e600640 1 -- 172.21.15.39:0/2629231421 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f05981fb880 con 0x7f0598074950 2024-09-21T07:33:22.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.532+0000 7f45fe000640 1 --2- 172.21.15.39:0/199935886 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f45e4078bc0 0x7f45e407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:22.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.532+0000 7f058f400640 1 -- 172.21.15.39:0/2629231421 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f05880035f0 con 0x7f0598074950 2024-09-21T07:33:22.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.532+0000 7f5975c00640 1 -- 172.21.15.39:0/13597896 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f59680efae0 con 0x7f596800bc30 2024-09-21T07:33:22.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.532+0000 7f7285800640 1 -- 172.21.15.39:0/3948693107 shutdown_connections 2024-09-21T07:33:22.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.532+0000 7f7285800640 1 --2- 172.21.15.39:0/3948693107 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72800b2cb0 0x7f72800b30c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.532+0000 7f7285800640 1 -- 172.21.15.39:0/3948693107 >> 172.21.15.39:0/3948693107 conn(0x7f72800cdcb0 msgr2=0x7f72800ce0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:22.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.533+0000 7fdfad600640 1 -- 172.21.15.39:0/2434118133 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdf9c01a070 con 0x7fdfb00d0340 2024-09-21T07:33:22.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.533+0000 7f7285800640 1 -- 172.21.15.39:0/3948693107 shutdown_connections 2024-09-21T07:33:22.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.533+0000 7f45fe000640 1 -- 172.21.15.39:0/199935886 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f45f009b2e0 con 0x7f45ec004800 2024-09-21T07:33:22.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.533+0000 7f7285800640 1 -- 172.21.15.39:0/3948693107 wait complete. 2024-09-21T07:33:22.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.533+0000 7f45fe000640 1 --2- 172.21.15.39:0/199935886 >> [v2:172.21.15.39:6808/4245419123,v1:172.21.15.39:6809/4245419123] conn(0x7f45e4080db0 0x7f45e4083200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:22.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.533+0000 7f45fe000640 1 -- 172.21.15.39:0/199935886 --> [v2:172.21.15.39:6808/4245419123,v1:172.21.15.39:6809/4245419123] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f45f0010e50 con 0x7f45e4080db0 2024-09-21T07:33:22.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.533+0000 7f45fe000640 1 -- 172.21.15.39:0/199935886 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f45f009b6d0 con 0x7f45ec004800 2024-09-21T07:33:22.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.532+0000 7f2d51e00640 1 -- 172.21.15.39:0/2503300862 wait complete. 2024-09-21T07:33:22.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.534+0000 7fa7c6400640 1 -- 172.21.15.39:0/3927263561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7c006e170 msgr2=0x7fa7c00b3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.534+0000 7fa7c6400640 1 --2- 172.21.15.39:0/3927263561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7c006e170 0x7fa7c00b3540 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0009860 tx=0x7fa7b002a4b0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.534+0000 7f058f400640 1 --2- 172.21.15.39:0/2629231421 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0574078bc0 0x7f057407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:22.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.534+0000 7f058f400640 1 -- 172.21.15.39:0/2629231421 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f05880bc1f0 con 0x7f0598074950 2024-09-21T07:33:22.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.534+0000 7fdfad600640 1 --2- 172.21.15.39:0/2434118133 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdf90078bc0 0x7fdf9007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:22.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.534+0000 7f058f400640 1 --2- 172.21.15.39:0/2629231421 >> [v2:172.21.15.39:6824/1628627569,v1:172.21.15.39:6825/1628627569] conn(0x7f0574080db0 0x7f0574083200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:22.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.534+0000 7f058f400640 1 -- 172.21.15.39:0/2629231421 --> [v2:172.21.15.39:6824/1628627569,v1:172.21.15.39:6825/1628627569] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f05880317a0 con 0x7f0574080db0 2024-09-21T07:33:22.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.534+0000 7f058f400640 1 -- 172.21.15.39:0/2629231421 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f05880c3050 con 0x7f0598074950 2024-09-21T07:33:22.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.534+0000 7f0596a00640 1 --2- 172.21.15.39:0/2629231421 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0574078bc0 0x7f057407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:22.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.534+0000 7fa7c6400640 1 -- 172.21.15.39:0/3927263561 shutdown_connections 2024-09-21T07:33:22.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.534+0000 7fdfad600640 1 -- 172.21.15.39:0/2434118133 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fdf9c09b070 con 0x7fdfb00d0340 2024-09-21T07:33:22.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.534+0000 7fa7c6400640 1 --2- 172.21.15.39:0/3927263561 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7c006f4b0 0x7fa7c00d3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.535+0000 7fa7c6400640 1 --2- 172.21.15.39:0/3927263561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7c006e170 0x7fa7c00b3540 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.535+0000 7fa7c6400640 1 -- 172.21.15.39:0/3927263561 >> 172.21.15.39:0/3927263561 conn(0x7fa7c008bfc0 msgr2=0x7fa7c009d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:22.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.535+0000 7fdfb5800640 1 --2- 172.21.15.39:0/2434118133 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdf90078bc0 0x7fdf9007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:22.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.535+0000 7fdfad600640 1 --2- 172.21.15.39:0/2434118133 >> [v2:172.21.15.39:6800/3213199177,v1:172.21.15.39:6801/3213199177] conn(0x7fdf90080e90 0x7fdf900832e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:22.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.535+0000 7fdfad600640 1 -- 172.21.15.39:0/2434118133 --> [v2:172.21.15.39:6800/3213199177,v1:172.21.15.39:6801/3213199177] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdf900839b0 con 0x7fdf90080e90 2024-09-21T07:33:22.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.535+0000 7fdfad600640 1 -- 172.21.15.39:0/2434118133 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdf9c09b460 con 0x7fdfb00d0340 2024-09-21T07:33:22.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.534+0000 7f4605400640 1 --2- 172.21.15.39:0/199935886 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f45e4078bc0 0x7f45e407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:22.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.535+0000 7f4606800640 1 --2- 172.21.15.39:0/199935886 >> [v2:172.21.15.39:6808/4245419123,v1:172.21.15.39:6809/4245419123] conn(0x7f45e4080db0 0x7f45e4083200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:22.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.536+0000 7f0597e00640 1 --2- 172.21.15.39:0/2629231421 >> [v2:172.21.15.39:6824/1628627569,v1:172.21.15.39:6825/1628627569] conn(0x7f0574080db0 0x7f0574083200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:22.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.536+0000 7f5b15c00640 1 -- 172.21.15.39:0/529418391 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5af4003a50 msgr2=0x7f5af4003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.536+0000 7f0596a00640 1 --2- 172.21.15.39:0/2629231421 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0574078bc0 0x7f057407b0d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f059815cee0 tx=0x7f0580009060 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:22.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.536+0000 7fdfb5800640 1 --2- 172.21.15.39:0/2434118133 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdf90078bc0 0x7fdf9007b0d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0000f60 tx=0x7fdfa0009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:22.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.536+0000 7fa7c6400640 1 -- 172.21.15.39:0/3927263561 shutdown_connections 2024-09-21T07:33:22.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.536+0000 7fa7c6400640 1 -- 172.21.15.39:0/3927263561 wait complete. 2024-09-21T07:33:22.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.537+0000 7f4605400640 1 --2- 172.21.15.39:0/199935886 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f45e4078bc0 0x7f45e407b0d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f45f8033000 tx=0x7f45f802d000 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:22.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.535+0000 7f5963e00640 1 -- 172.21.15.39:0/13597896 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f595c01c460 con 0x7f596800bc30 2024-09-21T07:33:22.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.536+0000 7f5b15c00640 1 --2- 172.21.15.39:0/529418391 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5af4003a50 0x7f5af4003ec0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009920 tx=0x7f5af4005830 comp rx=0 tx=0).stop 2024-09-21T07:33:22.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.536+0000 7f5b15c00640 1 -- 172.21.15.39:0/529418391 shutdown_connections 2024-09-21T07:33:22.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.536+0000 7f5b15c00640 1 --2- 172.21.15.39:0/529418391 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5af4003a50 0x7f5af4003ec0 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009920 tx=0x7f5af4005830 comp rx=0 tx=0).stop 2024-09-21T07:33:22.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.536+0000 7f5b15c00640 1 -- 172.21.15.39:0/529418391 >> 172.21.15.39:0/529418391 conn(0x7f5b1009d680 msgr2=0x7f5b100d4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:22.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.537+0000 7f2d51e00640 1 Processor -- start 2024-09-21T07:33:22.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.537+0000 7fdfb6200640 1 --2- 172.21.15.39:0/2434118133 >> [v2:172.21.15.39:6800/3213199177,v1:172.21.15.39:6801/3213199177] conn(0x7fdf90080e90 0x7fdf900832e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:22.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.537+0000 7f7285800640 1 Processor -- start 2024-09-21T07:33:22.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.537+0000 7f7285800640 1 -- start start 2024-09-21T07:33:22.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.537+0000 7f5b15c00640 1 -- 172.21.15.39:0/529418391 shutdown_connections 2024-09-21T07:33:22.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.537+0000 7f5b15c00640 1 -- 172.21.15.39:0/529418391 wait complete. 2024-09-21T07:33:22.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.537+0000 7f2d51e00640 1 -- start start 2024-09-21T07:33:22.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.538+0000 7fdfb6200640 1 --2- 172.21.15.39:0/2434118133 >> [v2:172.21.15.39:6800/3213199177,v1:172.21.15.39:6801/3213199177] conn(0x7fdf90080e90 0x7fdf900832e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:22.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.538+0000 7f5963e00640 1 --2- 172.21.15.39:0/13597896 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5950078c90 0x7f595007b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:22.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.538+0000 7f5963e00640 1 -- 172.21.15.39:0/13597896 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f595c09a820 con 0x7f596800bc30 2024-09-21T07:33:22.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.538+0000 7f5963e00640 1 --2- 172.21.15.39:0/13597896 >> [v2:172.21.15.39:6816/3897193606,v1:172.21.15.39:6817/3897193606] conn(0x7f5950080f60 0x7f59500833b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:22.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.538+0000 7f5963e00640 1 -- 172.21.15.39:0/13597896 --> [v2:172.21.15.39:6816/3897193606,v1:172.21.15.39:6817/3897193606] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5950083a80 con 0x7f5950080f60 2024-09-21T07:33:22.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.538+0000 7f5963e00640 1 -- 172.21.15.39:0/13597896 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f595c0a1050 con 0x7f596800bc30 2024-09-21T07:33:22.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.539+0000 7f4606800640 1 --2- 172.21.15.39:0/199935886 >> [v2:172.21.15.39:6808/4245419123,v1:172.21.15.39:6809/4245419123] conn(0x7f45e4080db0 0x7f45e4083200 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:22.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.539+0000 7f596fe00640 1 --2- 172.21.15.39:0/13597896 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5950078c90 0x7f595007b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:22.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.539+0000 7f5975200640 1 --2- 172.21.15.39:0/13597896 >> [v2:172.21.15.39:6816/3897193606,v1:172.21.15.39:6817/3897193606] conn(0x7f5950080f60 0x7f59500833b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:22.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.540+0000 7f2d51e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d4c0b2c10 0x7f2d4c15e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:22.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.540+0000 7f0597e00640 1 --2- 172.21.15.39:0/2629231421 >> [v2:172.21.15.39:6824/1628627569,v1:172.21.15.39:6825/1628627569] conn(0x7f0574080db0 0x7f0574083200 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:22.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.540+0000 7f7285800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72800b2cb0 0x7f728015ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:22.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.540+0000 7f7285800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72800cf130 0x7f7280148520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:22.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.540+0000 7f7285800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f728015b8f0 0x7f72800b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:22.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.540+0000 7f7285800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f72800ae440 con 0x7f728015b8f0 2024-09-21T07:33:22.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.540+0000 7f7285800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f72800ae2c0 con 0x7f72800b2cb0 2024-09-21T07:33:22.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.540+0000 7f7285800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f72800ae5c0 con 0x7f72800cf130 2024-09-21T07:33:22.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.540+0000 7f2d51e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d4c0b3c90 0x7f2d4c0d19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:22.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.540+0000 7fa7c6400640 1 Processor -- start 2024-09-21T07:33:22.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.540+0000 7fdfad600640 1 -- 172.21.15.39:0/2434118133 <== osd.3 v2:172.21.15.39:6800/3213199177 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdf900839b0 con 0x7fdf90080e90 2024-09-21T07:33:22.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.541+0000 7f5b15c00640 1 Processor -- start 2024-09-21T07:33:22.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.541+0000 7f5b15c00640 1 -- start start 2024-09-21T07:33:22.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.540+0000 7f2d50a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d4c0b2c10 0x7f2d4c15e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:22.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.541+0000 7f2d50a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d4c0b2c10 0x7f2d4c15e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59984/0 (socket says 172.21.15.39:59984) 2024-09-21T07:33:22.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.541+0000 7f5975200640 1 --2- 172.21.15.39:0/13597896 >> [v2:172.21.15.39:6816/3897193606,v1:172.21.15.39:6817/3897193606] conn(0x7f5950080f60 0x7f59500833b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:22.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.541+0000 7f727f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f728015b8f0 0x7f72800b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:22.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.541+0000 7f727f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f728015b8f0 0x7f72800b51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60000/0 (socket says 172.21.15.39:60000) 2024-09-21T07:33:22.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.541+0000 7f727f400640 1 -- 172.21.15.39:0/2832958790 learned_addr learned my addr 172.21.15.39:0/2832958790 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:22.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.541+0000 7f727ea00640 1 --2- 172.21.15.39:0/2832958790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72800b2cb0 0x7f728015ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:22.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.541+0000 7f596fe00640 1 --2- 172.21.15.39:0/13597896 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5950078c90 0x7f595007b1a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f597006bc20 tx=0x7f597008e040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:22.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.541+0000 7f058f400640 1 -- 172.21.15.39:0/2629231421 <== osd.1 v2:172.21.15.39:6824/1628627569 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f05880317a0 con 0x7f0574080db0 2024-09-21T07:33:22.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.541+0000 7fa7c6400640 1 -- start start 2024-09-21T07:33:22.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.541+0000 7f7276000640 1 --2- 172.21.15.39:0/2832958790 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72800cf130 0x7f7280148520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:22.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.541+0000 7f2d50a00640 1 -- 172.21.15.39:0/3970659647 learned_addr learned my addr 172.21.15.39:0/3970659647 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:22.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.542+0000 7f2d51e00640 1 --2- 172.21.15.39:0/3970659647 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d4c0d4390 0x7f2d4c0cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:22.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.542+0000 7f5963e00640 1 -- 172.21.15.39:0/13597896 <== osd.2 v2:172.21.15.39:6816/3897193606 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5950083a80 con 0x7f5950080f60 2024-09-21T07:33:22.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.542+0000 7f2d4be00640 1 --2- 172.21.15.39:0/3970659647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d4c0b3c90 0x7f2d4c0d19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:22.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.542+0000 7f7276000640 1 -- 172.21.15.39:0/2832958790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72800b2cb0 msgr2=0x7f728015ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.542+0000 7f7276000640 1 --2- 172.21.15.39:0/2832958790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72800b2cb0 0x7f728015ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.542+0000 7f7276000640 1 -- 172.21.15.39:0/2832958790 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f728015b8f0 msgr2=0x7f72800b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.542+0000 7f7276000640 1 --2- 172.21.15.39:0/2832958790 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f728015b8f0 0x7f72800b51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.542+0000 7f7276000640 1 -- 172.21.15.39:0/2832958790 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7280152a50 con 0x7f72800cf130 2024-09-21T07:33:22.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.543+0000 7f2d51400640 1 --2- 172.21.15.39:0/3970659647 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d4c0d4390 0x7f2d4c0cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:22.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.543+0000 7f7276000640 1 --2- 172.21.15.39:0/2832958790 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72800cf130 0x7f7280148520 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f726800b3e0 tx=0x7f726800b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:22.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.543+0000 7f2d51e00640 1 -- 172.21.15.39:0/3970659647 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c0b5460 con 0x7f2d4c0b2c10 2024-09-21T07:33:22.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.542+0000 7f45fe000640 1 -- 172.21.15.39:0/199935886 <== osd.0 v2:172.21.15.39:6808/4245419123 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f45f0010e50 con 0x7f45e4080db0 2024-09-21T07:33:22.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.543+0000 7f2d51e00640 1 -- 172.21.15.39:0/3970659647 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c0b52e0 con 0x7f2d4c0b3c90 2024-09-21T07:33:22.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.543+0000 7f2d51e00640 1 -- 172.21.15.39:0/3970659647 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f2d4c0b55e0 con 0x7f2d4c0d4390 2024-09-21T07:33:22.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.543+0000 7f5b15c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b100cded0 0x7f5b100b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:22.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.543+0000 7f5b15c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b100ce2e0 0x7f5b10161a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:22.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.543+0000 7f5b15c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b100b2500 0x7f5b10161f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:22.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.544+0000 7f5b15c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5b100d5270 con 0x7f5b100b2500 2024-09-21T07:33:22.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.544+0000 7f5b15c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5b100d50f0 con 0x7f5b100cded0 2024-09-21T07:33:22.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.544+0000 7f5b15c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f5b100d0fc0 con 0x7f5b100ce2e0 2024-09-21T07:33:22.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.544+0000 7f7277400640 1 -- 172.21.15.39:0/2832958790 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7268015020 con 0x7f72800cf130 2024-09-21T07:33:22.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.544+0000 7f7277400640 1 -- 172.21.15.39:0/2832958790 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7268011920 con 0x7f72800cf130 2024-09-21T07:33:22.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.544+0000 7f7277400640 1 -- 172.21.15.39:0/2832958790 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7268011d50 con 0x7f72800cf130 2024-09-21T07:33:22.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.544+0000 7f7285800640 1 -- 172.21.15.39:0/2832958790 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7280152d30 con 0x7f72800cf130 2024-09-21T07:33:22.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.544+0000 7f7285800640 1 -- 172.21.15.39:0/2832958790 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f72801532f0 con 0x7f72800cf130 2024-09-21T07:33:22.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.544+0000 7f5b0f400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b100ce2e0 0x7f5b10161a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:22.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.544+0000 7f5b15200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b100b2500 0x7f5b10161f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:22.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.544+0000 7f5b0f400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b100ce2e0 0x7f5b10161a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.39:40856/0 (socket says 172.21.15.39:40856) 2024-09-21T07:33:22.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.544+0000 7f5b0f400640 1 -- 172.21.15.39:0/326258085 learned_addr learned my addr 172.21.15.39:0/326258085 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:22.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.545+0000 7f5b15200640 1 -- 172.21.15.39:0/326258085 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b100ce2e0 msgr2=0x7f5b10161a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.545+0000 7f2d50a00640 1 -- 172.21.15.39:0/3970659647 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d4c0d4390 msgr2=0x7f2d4c0cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.545+0000 7f5b15200640 1 --2- 172.21.15.39:0/326258085 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b100ce2e0 0x7f5b10161a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.545+0000 7f5b15200640 1 -- 172.21.15.39:0/326258085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b100cded0 msgr2=0x7f5b100b1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.545+0000 7f5b15200640 1 --2- 172.21.15.39:0/326258085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b100cded0 0x7f5b100b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.545+0000 7f5b15200640 1 -- 172.21.15.39:0/326258085 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b100b29f0 con 0x7f5b100b2500 2024-09-21T07:33:22.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.546+0000 7f2d50a00640 1 --2- 172.21.15.39:0/3970659647 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d4c0d4390 0x7f2d4c0cfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.546+0000 7f2d50a00640 1 -- 172.21.15.39:0/3970659647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d4c0b3c90 msgr2=0x7f2d4c0d19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.546+0000 7f2d50a00640 1 --2- 172.21.15.39:0/3970659647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d4c0b3c90 0x7f2d4c0d19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.546+0000 7f2d50a00640 1 -- 172.21.15.39:0/3970659647 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d4c0cdf50 con 0x7f2d4c0b2c10 2024-09-21T07:33:22.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.546+0000 7f2d51400640 1 --2- 172.21.15.39:0/3970659647 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d4c0d4390 0x7f2d4c0cfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:33:22.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.546+0000 7f2d4be00640 1 --2- 172.21.15.39:0/3970659647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d4c0b3c90 0x7f2d4c0d19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:33:22.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.547+0000 7f2d50a00640 1 --2- 172.21.15.39:0/3970659647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d4c0b2c10 0x7f2d4c15e890 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2d44000c00 tx=0x7f2d44033ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:22.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.547+0000 7f5b15200640 1 --2- 172.21.15.39:0/326258085 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b100b2500 0x7f5b10161f80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5b101638c0 tx=0x7f5b0800ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:22.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.547+0000 7f2d48c00640 1 -- 172.21.15.39:0/3970659647 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d44031d60 con 0x7f2d4c0b2c10 2024-09-21T07:33:22.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.547+0000 7f2d48c00640 1 -- 172.21.15.39:0/3970659647 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d44033070 con 0x7f2d4c0b2c10 2024-09-21T07:33:22.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.547+0000 7f2d51e00640 1 -- 172.21.15.39:0/3970659647 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d4c0ce230 con 0x7f2d4c0b2c10 2024-09-21T07:33:22.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.547+0000 7f2d48c00640 1 -- 172.21.15.39:0/3970659647 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d440334c0 con 0x7f2d4c0b2c10 2024-09-21T07:33:22.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.547+0000 7f2d51e00640 1 -- 172.21.15.39:0/3970659647 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d4c1fad40 con 0x7f2d4c0b2c10 2024-09-21T07:33:22.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.547+0000 7fa7c6400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7c006f4b0 0x7fa7c00c8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:22.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.548+0000 7fa7c6400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7c00d3870 0x7fa7c014e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:22.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.547+0000 7f7285800640 1 -- 172.21.15.39:0/2832958790 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7258001030 con 0x7f72800cf130 2024-09-21T07:33:22.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.548+0000 7f5b07e00640 1 -- 172.21.15.39:0/326258085 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0800c9a0 con 0x7f5b100b2500 2024-09-21T07:33:22.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.548+0000 7f7277400640 1 -- 172.21.15.39:0/2832958790 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7268018020 con 0x7f72800cf130 2024-09-21T07:33:22.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.549+0000 7fa7c6400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7c015f790 0x7fa7c00cd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:22.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.549+0000 7fa7c6400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c00d0e60 con 0x7fa7c00d3870 2024-09-21T07:33:22.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.549+0000 7fa7c6400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c00d0ce0 con 0x7fa7c015f790 2024-09-21T07:33:22.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.549+0000 7fa7c6400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa7c00d0fe0 con 0x7fa7c006f4b0 2024-09-21T07:33:22.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.548+0000 7f5b15c00640 1 -- 172.21.15.39:0/326258085 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b101f91e0 con 0x7f5b100b2500 2024-09-21T07:33:22.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.548+0000 7f5b15c00640 1 -- 172.21.15.39:0/326258085 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b101f97a0 con 0x7f5b100b2500 2024-09-21T07:33:22.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.549+0000 7fa7bea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7c00d3870 0x7fa7c014e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:22.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.549+0000 7fa7bea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7c00d3870 0x7fa7c014e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60008/0 (socket says 172.21.15.39:60008) 2024-09-21T07:33:22.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.549+0000 7fa7bea00640 1 -- 172.21.15.39:0/2238880528 learned_addr learned my addr 172.21.15.39:0/2238880528 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:22.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.550+0000 7fa7bf400640 1 --2- 172.21.15.39:0/2238880528 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7c006f4b0 0x7fa7c00c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:22.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.550+0000 7f7277400640 1 --2- 172.21.15.39:0/2832958790 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f725c078b90 0x7f725c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:22.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.551+0000 7fa7bfe00640 1 --2- 172.21.15.39:0/2238880528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7c015f790 0x7fa7c00cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:22.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.551+0000 7f7277400640 1 -- 172.21.15.39:0/2832958790 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f726809aae0 con 0x7f72800cf130 2024-09-21T07:33:22.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.549+0000 7f5b07e00640 1 -- 172.21.15.39:0/326258085 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b08018070 con 0x7f5b100b2500 2024-09-21T07:33:22.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.549+0000 7f5b07e00640 1 -- 172.21.15.39:0/326258085 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b08011460 con 0x7f5b100b2500 2024-09-21T07:33:22.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.550+0000 7f5b15c00640 1 -- 172.21.15.39:0/326258085 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5ae8001030 con 0x7f5b100b2500 2024-09-21T07:33:22.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.551+0000 7f7277400640 1 --2- 172.21.15.39:0/2832958790 >> [v2:172.21.15.148:6826/1837190792,v1:172.21.15.148:6827/1837190792] conn(0x7f725c080e60 0x7f725c0832b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:22.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.551+0000 7f727ea00640 1 --2- 172.21.15.39:0/2832958790 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f725c078b90 0x7f725c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:22.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.551+0000 7f7277400640 1 -- 172.21.15.39:0/2832958790 --> [v2:172.21.15.148:6826/1837190792,v1:172.21.15.148:6827/1837190792] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f726800de50 con 0x7f725c080e60 2024-09-21T07:33:22.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.551+0000 7f7277400640 1 -- 172.21.15.39:0/2832958790 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f726809f2a0 con 0x7f72800cf130 2024-09-21T07:33:22.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.551+0000 7fa7bf400640 1 -- 172.21.15.39:0/2238880528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7c015f790 msgr2=0x7fa7c00cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.552+0000 7f727f400640 1 --2- 172.21.15.39:0/2832958790 >> [v2:172.21.15.148:6826/1837190792,v1:172.21.15.148:6827/1837190792] conn(0x7f725c080e60 0x7f725c0832b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:22.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.551+0000 7fa7bf400640 1 --2- 172.21.15.39:0/2238880528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7c015f790 0x7fa7c00cd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.551+0000 7fa7bf400640 1 -- 172.21.15.39:0/2238880528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7c00d3870 msgr2=0x7fa7c014e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.551+0000 7fa7bf400640 1 --2- 172.21.15.39:0/2238880528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7c00d3870 0x7fa7c014e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.551+0000 7fa7bf400640 1 -- 172.21.15.39:0/2238880528 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c014ebf0 con 0x7fa7c006f4b0 2024-09-21T07:33:22.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.551+0000 7fa7bea00640 1 --2- 172.21.15.39:0/2238880528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7c00d3870 0x7fa7c014e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:33:22.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.550+0000 7f2d51e00640 1 -- 172.21.15.39:0/3970659647 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2d4c0b5b70 con 0x7f2d4c0b2c10 2024-09-21T07:33:22.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.552+0000 7f727f400640 1 --2- 172.21.15.39:0/2832958790 >> [v2:172.21.15.148:6826/1837190792,v1:172.21.15.148:6827/1837190792] conn(0x7f725c080e60 0x7f725c0832b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:22.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.553+0000 7fa7bf400640 1 --2- 172.21.15.39:0/2238880528 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7c006f4b0 0x7fa7c00c8430 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0031000 tx=0x7fa7b0032b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:22.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.554+0000 7f7277400640 1 -- 172.21.15.39:0/2832958790 <== osd.5 v2:172.21.15.148:6826/1837190792 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f726800de50 con 0x7f725c080e60 2024-09-21T07:33:22.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.554+0000 7fa7b7400640 1 -- 172.21.15.39:0/2238880528 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0040070 con 0x7fa7c006f4b0 2024-09-21T07:33:22.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.554+0000 7fa7b7400640 1 -- 172.21.15.39:0/2238880528 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa7b0003900 con 0x7fa7c006f4b0 2024-09-21T07:33:22.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.554+0000 7f727ea00640 1 --2- 172.21.15.39:0/2832958790 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f725c078b90 0x7f725c07b0a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7270001c10 tx=0x7f7270001ba0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:22.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.555+0000 7f5b07e00640 1 -- 172.21.15.39:0/326258085 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5b0801c3c0 con 0x7f5b100b2500 2024-09-21T07:33:22.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.557+0000 7f5b07e00640 1 --2- 172.21.15.39:0/326258085 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5af0078b90 0x7f5af007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:22.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.557+0000 7f5b07e00640 1 -- 172.21.15.39:0/326258085 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5b0809b4d0 con 0x7f5b100b2500 2024-09-21T07:33:22.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.557+0000 7f5b0fe00640 1 --2- 172.21.15.39:0/326258085 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5af0078b90 0x7f5af007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:22.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.557+0000 7f5b07e00640 1 --2- 172.21.15.39:0/326258085 >> [v2:172.21.15.148:6810/65974579,v1:172.21.15.148:6811/65974579] conn(0x7f5af0080e00 0x7f5af0083250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:22.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.557+0000 7f5b07e00640 1 -- 172.21.15.39:0/326258085 --> [v2:172.21.15.148:6810/65974579,v1:172.21.15.148:6811/65974579] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5af0083920 con 0x7f5af0080e00 2024-09-21T07:33:22.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.557+0000 7f5b07e00640 1 -- 172.21.15.39:0/326258085 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5b0809b8c0 con 0x7f5b100b2500 2024-09-21T07:33:22.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.555+0000 7f2d48c00640 1 -- 172.21.15.39:0/3970659647 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2d44033660 con 0x7f2d4c0b2c10 2024-09-21T07:33:22.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.557+0000 7f2d48c00640 1 --2- 172.21.15.39:0/3970659647 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2d28078b90 0x7f2d2807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:22.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.557+0000 7f2d48c00640 1 -- 172.21.15.39:0/3970659647 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2d440bc160 con 0x7f2d4c0b2c10 2024-09-21T07:33:22.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.557+0000 7f2d48c00640 1 --2- 172.21.15.39:0/3970659647 >> [v2:172.21.15.148:6818/1438255756,v1:172.21.15.148:6819/1438255756] conn(0x7f2d28080d80 0x7f2d280831d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:22.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.557+0000 7f2d48c00640 1 -- 172.21.15.39:0/3970659647 --> [v2:172.21.15.148:6818/1438255756,v1:172.21.15.148:6819/1438255756] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2d44046e50 con 0x7f2d28080d80 2024-09-21T07:33:22.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.554+0000 7fa7c6400640 1 -- 172.21.15.39:0/2238880528 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7c00cc860 con 0x7fa7c006f4b0 2024-09-21T07:33:22.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.554+0000 7fa7c6400640 1 -- 172.21.15.39:0/2238880528 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7c00cce20 con 0x7fa7c006f4b0 2024-09-21T07:33:22.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.557+0000 7f5b0f400640 1 --2- 172.21.15.39:0/326258085 >> [v2:172.21.15.148:6810/65974579,v1:172.21.15.148:6811/65974579] conn(0x7f5af0080e00 0x7f5af0083250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:22.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.554+0000 7fa7b7400640 1 -- 172.21.15.39:0/2238880528 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0003d50 con 0x7fa7c006f4b0 2024-09-21T07:33:22.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.555+0000 7fa7c6400640 1 -- 172.21.15.39:0/2238880528 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa798001030 con 0x7fa7c006f4b0 2024-09-21T07:33:22.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.558+0000 7f2d4be00640 1 --2- 172.21.15.39:0/3970659647 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2d28078b90 0x7f2d2807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:22.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.558+0000 7f2d51400640 1 --2- 172.21.15.39:0/3970659647 >> [v2:172.21.15.148:6818/1438255756,v1:172.21.15.148:6819/1438255756] conn(0x7f2d28080d80 0x7f2d280831d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:22.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.558+0000 7f5b0f400640 1 --2- 172.21.15.39:0/326258085 >> [v2:172.21.15.148:6810/65974579,v1:172.21.15.148:6811/65974579] conn(0x7f5af0080e00 0x7f5af0083250 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:22.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.559+0000 7f5b0fe00640 1 --2- 172.21.15.39:0/326258085 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5af0078b90 0x7f5af007b0a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5b00034000 tx=0x7f5b00004820 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:22.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.558+0000 7f2d48c00640 1 -- 172.21.15.39:0/3970659647 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2d440c3050 con 0x7f2d4c0b2c10 2024-09-21T07:33:22.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.559+0000 7f2d4be00640 1 --2- 172.21.15.39:0/3970659647 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2d28078b90 0x7f2d2807b0a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0a23b0 tx=0x7f2d3400d040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:22.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.559+0000 7fa7b7400640 1 -- 172.21.15.39:0/2238880528 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa7b0039040 con 0x7fa7c006f4b0 2024-09-21T07:33:22.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.559+0000 7f5b07e00640 1 -- 172.21.15.39:0/326258085 <== osd.6 v2:172.21.15.148:6810/65974579 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5af0083920 con 0x7f5af0080e00 2024-09-21T07:33:22.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.561+0000 7fa7b7400640 1 --2- 172.21.15.39:0/2238880528 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa7a0078c60 0x7fa7a007b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:22.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.561+0000 7fa7b7400640 1 -- 172.21.15.39:0/2238880528 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa7b006b050 con 0x7fa7c006f4b0 2024-09-21T07:33:22.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.561+0000 7fa7b7400640 1 --2- 172.21.15.39:0/2238880528 >> [v2:172.21.15.148:6802/1437509971,v1:172.21.15.148:6803/1437509971] conn(0x7fa7a0080f30 0x7fa7a0083380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:22.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.561+0000 7fa7b7400640 1 -- 172.21.15.39:0/2238880528 --> [v2:172.21.15.148:6802/1437509971,v1:172.21.15.148:6803/1437509971] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa7b0041ec0 con 0x7fa7a0080f30 2024-09-21T07:33:22.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.561+0000 7fa7b7400640 1 -- 172.21.15.39:0/2238880528 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa7b00bc6d0 con 0x7fa7c006f4b0 2024-09-21T07:33:22.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.561+0000 7f2d51400640 1 --2- 172.21.15.39:0/3970659647 >> [v2:172.21.15.148:6818/1438255756,v1:172.21.15.148:6819/1438255756] conn(0x7f2d28080d80 0x7f2d280831d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:22.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.562+0000 7fa7bfe00640 1 --2- 172.21.15.39:0/2238880528 >> [v2:172.21.15.148:6802/1437509971,v1:172.21.15.148:6803/1437509971] conn(0x7fa7a0080f30 0x7fa7a0083380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:22.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.562+0000 7fa7bea00640 1 --2- 172.21.15.39:0/2238880528 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa7a0078c60 0x7fa7a007b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:22.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.563+0000 7fa7bfe00640 1 --2- 172.21.15.39:0/2238880528 >> [v2:172.21.15.148:6802/1437509971,v1:172.21.15.148:6803/1437509971] conn(0x7fa7a0080f30 0x7fa7a0083380 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:22.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.563+0000 7f2d48c00640 1 -- 172.21.15.39:0/3970659647 <== osd.7 v2:172.21.15.148:6818/1438255756 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2d44046e50 con 0x7f2d28080d80 2024-09-21T07:33:22.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.563+0000 7fdfb6c00640 1 -- 172.21.15.39:0/2434118133 --> [v2:172.21.15.39:6800/3213199177,v1:172.21.15.39:6801/3213199177] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdfb013ef90 con 0x7fdf90080e90 2024-09-21T07:33:22.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.564+0000 7fa7bea00640 1 --2- 172.21.15.39:0/2238880528 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa7a0078c60 0x7fa7a007b170 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa7c00b11c0 tx=0x7fa7ac005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:22.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.564+0000 7fa7b7400640 1 -- 172.21.15.39:0/2238880528 <== osd.4 v2:172.21.15.148:6802/1437509971 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa7b0041ec0 con 0x7fa7a0080f30 2024-09-21T07:33:22.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.564+0000 7fdfad600640 1 -- 172.21.15.39:0/2434118133 <== osd.3 v2:172.21.15.39:6800/3213199177 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fdfb013ef90 con 0x7fdf90080e90 2024-09-21T07:33:22.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.565+0000 7fdfb6c00640 1 -- 172.21.15.39:0/2434118133 >> [v2:172.21.15.39:6800/3213199177,v1:172.21.15.39:6801/3213199177] conn(0x7fdf90080e90 msgr2=0x7fdf900832e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.565+0000 7fdfb6c00640 1 --2- 172.21.15.39:0/2434118133 >> [v2:172.21.15.39:6800/3213199177,v1:172.21.15.39:6801/3213199177] conn(0x7fdf90080e90 0x7fdf900832e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.565+0000 7fdfb6c00640 1 -- 172.21.15.39:0/2434118133 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdf90078bc0 msgr2=0x7fdf9007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.565+0000 7fdfb6c00640 1 --2- 172.21.15.39:0/2434118133 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdf90078bc0 0x7fdf9007b0d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0000f60 tx=0x7fdfa0009040 comp rx=0 tx=0).stop 2024-09-21T07:33:22.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.565+0000 7fdfb6c00640 1 -- 172.21.15.39:0/2434118133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfb00d0340 msgr2=0x7fdfb01fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.565+0000 7fdfb6c00640 1 --2- 172.21.15.39:0/2434118133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfb00d0340 0x7fdfb01fd460 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c00d720 tx=0x7fdf9c00dbe0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.565+0000 7fdfb6200640 1 -- 172.21.15.39:0/2434118133 reap_dead start 2024-09-21T07:33:22.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.565+0000 7fdfb6c00640 1 -- 172.21.15.39:0/2434118133 shutdown_connections 2024-09-21T07:33:22.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.565+0000 7fdfb6c00640 1 -- 172.21.15.39:0/2434118133 >> 172.21.15.39:0/2434118133 conn(0x7fdfb0070b20 msgr2=0x7fdfb015f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:22.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.568+0000 7fdfb6c00640 1 -- 172.21.15.39:0/2434118133 shutdown_connections 2024-09-21T07:33:22.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.568+0000 7fdfb6c00640 1 -- 172.21.15.39:0/2434118133 wait complete. 2024-09-21T07:33:22.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.569+0000 7f5975c00640 1 -- 172.21.15.39:0/13597896 --> [v2:172.21.15.39:6816/3897193606,v1:172.21.15.39:6817/3897193606] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f596800a3f0 con 0x7f5950080f60 2024-09-21T07:33:22.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.569+0000 7f5963e00640 1 -- 172.21.15.39:0/13597896 <== osd.2 v2:172.21.15.39:6816/3897193606 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f596800a3f0 con 0x7f5950080f60 2024-09-21T07:33:22.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.570+0000 7f5961600640 1 -- 172.21.15.39:0/13597896 >> [v2:172.21.15.39:6816/3897193606,v1:172.21.15.39:6817/3897193606] conn(0x7f5950080f60 msgr2=0x7f59500833b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.570+0000 7f5961600640 1 --2- 172.21.15.39:0/13597896 >> [v2:172.21.15.39:6816/3897193606,v1:172.21.15.39:6817/3897193606] conn(0x7f5950080f60 0x7f59500833b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.571+0000 7f5961600640 1 -- 172.21.15.39:0/13597896 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5950078c90 msgr2=0x7f595007b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.571+0000 7f5961600640 1 --2- 172.21.15.39:0/13597896 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5950078c90 0x7f595007b1a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f597006bc20 tx=0x7f597008e040 comp rx=0 tx=0).stop 2024-09-21T07:33:22.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.571+0000 7f5961600640 1 -- 172.21.15.39:0/13597896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f596800bc30 msgr2=0x7f59680ed6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.571+0000 7f5961600640 1 --2- 172.21.15.39:0/13597896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f596800bc30 0x7f59680ed6e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f595c00c3c0 tx=0x7f595c00c880 comp rx=0 tx=0).stop 2024-09-21T07:33:22.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.571+0000 7f5975200640 1 -- 172.21.15.39:0/13597896 reap_dead start 2024-09-21T07:33:22.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.571+0000 7f5961600640 1 -- 172.21.15.39:0/13597896 shutdown_connections 2024-09-21T07:33:22.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.571+0000 7f5961600640 1 -- 172.21.15.39:0/13597896 >> 172.21.15.39:0/13597896 conn(0x7f5968051420 msgr2=0x7f5968034a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:22.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.574+0000 7f5961600640 1 -- 172.21.15.39:0/13597896 shutdown_connections 2024-09-21T07:33:22.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.574+0000 7f5961600640 1 -- 172.21.15.39:0/13597896 wait complete. 2024-09-21T07:33:22.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.579+0000 7f059e600640 1 -- 172.21.15.39:0/2629231421 --> [v2:172.21.15.39:6824/1628627569,v1:172.21.15.39:6825/1628627569] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f05981622a0 con 0x7f0574080db0 2024-09-21T07:33:22.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.579+0000 7f460ce00640 1 -- 172.21.15.39:0/199935886 --> [v2:172.21.15.39:6808/4245419123,v1:172.21.15.39:6809/4245419123] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f45e0002dd0 con 0x7f45e4080db0 2024-09-21T07:33:22.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.580+0000 7f058f400640 1 -- 172.21.15.39:0/2629231421 <== osd.1 v2:172.21.15.39:6824/1628627569 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f05981622a0 con 0x7f0574080db0 2024-09-21T07:33:22.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.580+0000 7f45fe000640 1 -- 172.21.15.39:0/199935886 <== osd.0 v2:172.21.15.39:6808/4245419123 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f45e0002dd0 con 0x7f45e4080db0 2024-09-21T07:33:22.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.581+0000 7f059e600640 1 -- 172.21.15.39:0/2629231421 >> [v2:172.21.15.39:6824/1628627569,v1:172.21.15.39:6825/1628627569] conn(0x7f0574080db0 msgr2=0x7f0574083200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.581+0000 7f059e600640 1 --2- 172.21.15.39:0/2629231421 >> [v2:172.21.15.39:6824/1628627569,v1:172.21.15.39:6825/1628627569] conn(0x7f0574080db0 0x7f0574083200 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.581+0000 7f460ce00640 1 -- 172.21.15.39:0/199935886 >> [v2:172.21.15.39:6808/4245419123,v1:172.21.15.39:6809/4245419123] conn(0x7f45e4080db0 msgr2=0x7f45e4083200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.581+0000 7f460ce00640 1 --2- 172.21.15.39:0/199935886 >> [v2:172.21.15.39:6808/4245419123,v1:172.21.15.39:6809/4245419123] conn(0x7f45e4080db0 0x7f45e4083200 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.581+0000 7f460ce00640 1 -- 172.21.15.39:0/199935886 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f45e4078bc0 msgr2=0x7f45e407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.581+0000 7f460ce00640 1 --2- 172.21.15.39:0/199935886 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f45e4078bc0 0x7f45e407b0d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f45f8033000 tx=0x7f45f802d000 comp rx=0 tx=0).stop 2024-09-21T07:33:22.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.581+0000 7f460ce00640 1 -- 172.21.15.39:0/199935886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45ec004800 msgr2=0x7f460815d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.581+0000 7f460ce00640 1 --2- 172.21.15.39:0/199935886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45ec004800 0x7f460815d710 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f45f000d730 tx=0x7f45f000dbf0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.582+0000 7f4606800640 1 -- 172.21.15.39:0/199935886 reap_dead start 2024-09-21T07:33:22.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.582+0000 7f460ce00640 1 -- 172.21.15.39:0/199935886 shutdown_connections 2024-09-21T07:33:22.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.582+0000 7f460ce00640 1 -- 172.21.15.39:0/199935886 >> 172.21.15.39:0/199935886 conn(0x7f46080b1cb0 msgr2=0x7f46080b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:22.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.582+0000 7f059e600640 1 -- 172.21.15.39:0/2629231421 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0574078bc0 msgr2=0x7f057407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.582+0000 7f059e600640 1 --2- 172.21.15.39:0/2629231421 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0574078bc0 0x7f057407b0d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f059815cee0 tx=0x7f0580009060 comp rx=0 tx=0).stop 2024-09-21T07:33:22.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.582+0000 7f059e600640 1 -- 172.21.15.39:0/2629231421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0598074950 msgr2=0x7f05981fa830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.582+0000 7f059e600640 1 --2- 172.21.15.39:0/2629231421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0598074950 0x7f05981fa830 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0588000c00 tx=0x7f0588033ce0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.582+0000 7f0597e00640 1 -- 172.21.15.39:0/2629231421 reap_dead start 2024-09-21T07:33:22.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.583+0000 7f460ce00640 1 -- 172.21.15.39:0/199935886 shutdown_connections 2024-09-21T07:33:22.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.583+0000 7f059e600640 1 -- 172.21.15.39:0/2629231421 shutdown_connections 2024-09-21T07:33:22.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.583+0000 7f059e600640 1 -- 172.21.15.39:0/2629231421 >> 172.21.15.39:0/2629231421 conn(0x7f05980d0000 msgr2=0x7f0598070a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:22.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.583+0000 7f460ce00640 1 -- 172.21.15.39:0/199935886 wait complete. 2024-09-21T07:33:22.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.583+0000 7f059e600640 1 -- 172.21.15.39:0/2629231421 shutdown_connections 2024-09-21T07:33:22.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.583+0000 7f059e600640 1 -- 172.21.15.39:0/2629231421 wait complete. 2024-09-21T07:33:22.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.589+0000 7f7285800640 1 -- 172.21.15.39:0/2832958790 --> [v2:172.21.15.148:6826/1837190792,v1:172.21.15.148:6827/1837190792] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7258002dd0 con 0x7f725c080e60 2024-09-21T07:33:22.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.590+0000 7f7277400640 1 -- 172.21.15.39:0/2832958790 <== osd.5 v2:172.21.15.148:6826/1837190792 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7258002dd0 con 0x7f725c080e60 2024-09-21T07:33:22.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.591+0000 7f5b15c00640 1 -- 172.21.15.39:0/326258085 --> [v2:172.21.15.148:6810/65974579,v1:172.21.15.148:6811/65974579] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5ae8002dd0 con 0x7f5af0080e00 2024-09-21T07:33:22.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.591+0000 7f726fe00640 1 -- 172.21.15.39:0/2832958790 >> [v2:172.21.15.148:6826/1837190792,v1:172.21.15.148:6827/1837190792] conn(0x7f725c080e60 msgr2=0x7f725c0832b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.591+0000 7f726fe00640 1 --2- 172.21.15.39:0/2832958790 >> [v2:172.21.15.148:6826/1837190792,v1:172.21.15.148:6827/1837190792] conn(0x7f725c080e60 0x7f725c0832b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.591+0000 7f726fe00640 1 -- 172.21.15.39:0/2832958790 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f725c078b90 msgr2=0x7f725c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.592+0000 7f726fe00640 1 --2- 172.21.15.39:0/2832958790 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f725c078b90 0x7f725c07b0a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7270001c10 tx=0x7f7270001ba0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.592+0000 7f726fe00640 1 -- 172.21.15.39:0/2832958790 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72800cf130 msgr2=0x7f7280148520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.592+0000 7f726fe00640 1 --2- 172.21.15.39:0/2832958790 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72800cf130 0x7f7280148520 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f726800b3e0 tx=0x7f726800b8a0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.592+0000 7f727f400640 1 -- 172.21.15.39:0/2832958790 reap_dead start 2024-09-21T07:33:22.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.592+0000 7f726fe00640 1 -- 172.21.15.39:0/2832958790 shutdown_connections 2024-09-21T07:33:22.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.592+0000 7f726fe00640 1 -- 172.21.15.39:0/2832958790 >> 172.21.15.39:0/2832958790 conn(0x7f72800cdcb0 msgr2=0x7f72800b2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:22.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.593+0000 7f5b07e00640 1 -- 172.21.15.39:0/326258085 <== osd.6 v2:172.21.15.148:6810/65974579 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5ae8002dd0 con 0x7f5af0080e00 2024-09-21T07:33:22.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.593+0000 7f2d51e00640 1 -- 172.21.15.39:0/3970659647 --> [v2:172.21.15.148:6818/1438255756,v1:172.21.15.148:6819/1438255756] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2d4c06ece0 con 0x7f2d28080d80 2024-09-21T07:33:22.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.593+0000 7f5b15c00640 1 -- 172.21.15.39:0/326258085 >> [v2:172.21.15.148:6810/65974579,v1:172.21.15.148:6811/65974579] conn(0x7f5af0080e00 msgr2=0x7f5af0083250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.593+0000 7f5b15c00640 1 --2- 172.21.15.39:0/326258085 >> [v2:172.21.15.148:6810/65974579,v1:172.21.15.148:6811/65974579] conn(0x7f5af0080e00 0x7f5af0083250 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.593+0000 7f2d48c00640 1 -- 172.21.15.39:0/3970659647 <== osd.7 v2:172.21.15.148:6818/1438255756 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2d4c06ece0 con 0x7f2d28080d80 2024-09-21T07:33:22.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.594+0000 7f5b15c00640 1 -- 172.21.15.39:0/326258085 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5af0078b90 msgr2=0x7f5af007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.593+0000 7f726fe00640 1 -- 172.21.15.39:0/2832958790 shutdown_connections 2024-09-21T07:33:22.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.594+0000 7f726fe00640 1 -- 172.21.15.39:0/2832958790 wait complete. 2024-09-21T07:33:22.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.594+0000 7f5b15c00640 1 --2- 172.21.15.39:0/326258085 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5af0078b90 0x7f5af007b0a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5b00034000 tx=0x7f5b00004820 comp rx=0 tx=0).stop 2024-09-21T07:33:22.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.594+0000 7f2d51e00640 1 -- 172.21.15.39:0/3970659647 >> [v2:172.21.15.148:6818/1438255756,v1:172.21.15.148:6819/1438255756] conn(0x7f2d28080d80 msgr2=0x7f2d280831d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.594+0000 7f2d51e00640 1 --2- 172.21.15.39:0/3970659647 >> [v2:172.21.15.148:6818/1438255756,v1:172.21.15.148:6819/1438255756] conn(0x7f2d28080d80 0x7f2d280831d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.594+0000 7f5b15c00640 1 -- 172.21.15.39:0/326258085 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b100b2500 msgr2=0x7f5b10161f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.594+0000 7f5b15c00640 1 --2- 172.21.15.39:0/326258085 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b100b2500 0x7f5b10161f80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5b101638c0 tx=0x7f5b0800ed10 comp rx=0 tx=0).stop 2024-09-21T07:33:22.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.594+0000 7f5b15200640 1 -- 172.21.15.39:0/326258085 reap_dead start 2024-09-21T07:33:22.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.594+0000 7f5b15c00640 1 -- 172.21.15.39:0/326258085 shutdown_connections 2024-09-21T07:33:22.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.594+0000 7f5b15c00640 1 -- 172.21.15.39:0/326258085 >> 172.21.15.39:0/326258085 conn(0x7f5b1009d680 msgr2=0x7f5b100b7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:22.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.594+0000 7f5b15c00640 1 -- 172.21.15.39:0/326258085 shutdown_connections 2024-09-21T07:33:22.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.594+0000 7f5b15c00640 1 -- 172.21.15.39:0/326258085 wait complete. 2024-09-21T07:33:22.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.594+0000 7f2d51e00640 1 -- 172.21.15.39:0/3970659647 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2d28078b90 msgr2=0x7f2d2807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.594+0000 7f2d51e00640 1 --2- 172.21.15.39:0/3970659647 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2d28078b90 0x7f2d2807b0a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0a23b0 tx=0x7f2d3400d040 comp rx=0 tx=0).stop 2024-09-21T07:33:22.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.595+0000 7f2d51e00640 1 -- 172.21.15.39:0/3970659647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d4c0b2c10 msgr2=0x7f2d4c15e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.595+0000 7f2d51e00640 1 --2- 172.21.15.39:0/3970659647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d4c0b2c10 0x7f2d4c15e890 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2d44000c00 tx=0x7f2d44033ce0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.595+0000 7f2d51400640 1 -- 172.21.15.39:0/3970659647 reap_dead start 2024-09-21T07:33:22.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.595+0000 7f2d51e00640 1 -- 172.21.15.39:0/3970659647 shutdown_connections 2024-09-21T07:33:22.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.595+0000 7f2d51e00640 1 -- 172.21.15.39:0/3970659647 >> 172.21.15.39:0/3970659647 conn(0x7f2d4c08bfc0 msgr2=0x7f2d4c06dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:22.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.595+0000 7f2d51e00640 1 -- 172.21.15.39:0/3970659647 shutdown_connections 2024-09-21T07:33:22.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.595+0000 7f2d51e00640 1 -- 172.21.15.39:0/3970659647 wait complete. 2024-09-21T07:33:22.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.597+0000 7fa7c6400640 1 -- 172.21.15.39:0/2238880528 --> [v2:172.21.15.148:6802/1437509971,v1:172.21.15.148:6803/1437509971] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa798002dd0 con 0x7fa7a0080f30 2024-09-21T07:33:22.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.598+0000 7fa7b7400640 1 -- 172.21.15.39:0/2238880528 <== osd.4 v2:172.21.15.148:6802/1437509971 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa798002dd0 con 0x7fa7a0080f30 2024-09-21T07:33:22.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.599+0000 7fa7c6400640 1 -- 172.21.15.39:0/2238880528 >> [v2:172.21.15.148:6802/1437509971,v1:172.21.15.148:6803/1437509971] conn(0x7fa7a0080f30 msgr2=0x7fa7a0083380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.599+0000 7fa7c6400640 1 --2- 172.21.15.39:0/2238880528 >> [v2:172.21.15.148:6802/1437509971,v1:172.21.15.148:6803/1437509971] conn(0x7fa7a0080f30 0x7fa7a0083380 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:22.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.599+0000 7fa7c6400640 1 -- 172.21.15.39:0/2238880528 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa7a0078c60 msgr2=0x7fa7a007b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.599+0000 7fa7c6400640 1 --2- 172.21.15.39:0/2238880528 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa7a0078c60 0x7fa7a007b170 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa7c00b11c0 tx=0x7fa7ac005f90 comp rx=0 tx=0).stop 2024-09-21T07:33:22.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.599+0000 7fa7c6400640 1 -- 172.21.15.39:0/2238880528 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7c006f4b0 msgr2=0x7fa7c00c8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:22.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.599+0000 7fa7c6400640 1 --2- 172.21.15.39:0/2238880528 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7c006f4b0 0x7fa7c00c8430 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0031000 tx=0x7fa7b0032b90 comp rx=0 tx=0).stop 2024-09-21T07:33:22.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.599+0000 7fa7bfe00640 1 -- 172.21.15.39:0/2238880528 reap_dead start 2024-09-21T07:33:22.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.600+0000 7fa7c6400640 1 -- 172.21.15.39:0/2238880528 shutdown_connections 2024-09-21T07:33:22.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.600+0000 7fa7c6400640 1 -- 172.21.15.39:0/2238880528 >> 172.21.15.39:0/2238880528 conn(0x7fa7c008bfc0 msgr2=0x7fa7c00b2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:22.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.602+0000 7fa7c6400640 1 -- 172.21.15.39:0/2238880528 shutdown_connections 2024-09-21T07:33:22.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:22.602+0000 7fa7c6400640 1 -- 172.21.15.39:0/2238880528 wait complete. 2024-09-21T07:33:22.607 INFO:teuthology.orchestra.run.smithi039.stdout:60129542151 2024-09-21T07:33:22.607 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T07:33:22.614 INFO:teuthology.orchestra.run.smithi039.stdout:64424509448 2024-09-21T07:33:22.614 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T07:33:22.625 INFO:teuthology.orchestra.run.smithi039.stdout:64424509449 2024-09-21T07:33:22.625 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T07:33:22.626 INFO:teuthology.orchestra.run.smithi039.stdout:64424509449 2024-09-21T07:33:22.627 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T07:33:22.631 INFO:teuthology.orchestra.run.smithi039.stdout:64424509447 2024-09-21T07:33:22.631 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-21T07:33:22.635 INFO:teuthology.orchestra.run.smithi039.stdout:64424509447 2024-09-21T07:33:22.635 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T07:33:22.639 INFO:teuthology.orchestra.run.smithi039.stdout:64424509449 2024-09-21T07:33:22.639 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-21T07:33:22.642 INFO:teuthology.orchestra.run.smithi039.stdout:60129542152 2024-09-21T07:33:22.642 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T07:33:23.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.016+0000 7fcb21000640 1 -- 172.21.15.39:0/4203270489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb1c162480 msgr2=0x7fcb1c162890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:23.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.016+0000 7fcb21000640 1 --2- 172.21.15.39:0/4203270489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb1c162480 0x7fcb1c162890 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009860 tx=0x7fcb0802a510 comp rx=0 tx=0).stop 2024-09-21T07:33:23.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.018+0000 7fcb21000640 1 -- 172.21.15.39:0/4203270489 shutdown_connections 2024-09-21T07:33:23.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.018+0000 7fcb21000640 1 --2- 172.21.15.39:0/4203270489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb1c162480 0x7fcb1c162890 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.018+0000 7fcb21000640 1 --2- 172.21.15.39:0/4203270489 >> v2:172.21.15.148:3300/0 conn(0x7fcb1c0b3b30 0x7fcb1c0b57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.018+0000 7fcb21000640 1 -- 172.21.15.39:0/4203270489 >> 172.21.15.39:0/4203270489 conn(0x7fcb1c09d680 msgr2=0x7fcb1c0d4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:23.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.018+0000 7fcb21000640 1 -- 172.21.15.39:0/4203270489 shutdown_connections 2024-09-21T07:33:23.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.020+0000 7fcb21000640 1 -- 172.21.15.39:0/4203270489 wait complete. 2024-09-21T07:33:23.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.027+0000 7fcb21000640 1 Processor -- start 2024-09-21T07:33:23.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.027+0000 7fcb21000640 1 -- start start 2024-09-21T07:33:23.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.030+0000 7fcb21000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcb1c0b3b30 0x7fcb1c1fa750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:23.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.030+0000 7fcb21000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb1c162480 0x7fcb1c160b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:23.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.030+0000 7fcb19e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcb1c0b3b30 0x7fcb1c1fa750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:23.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.031+0000 7fcb19e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcb1c0b3b30 0x7fcb1c1fa750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.39:40896/0 (socket says 172.21.15.39:40896) 2024-09-21T07:33:23.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.031+0000 7fcb19e00640 1 -- 172.21.15.39:0/633531708 learned_addr learned my addr 172.21.15.39:0/633531708 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:23.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.031+0000 7fcb19400640 1 --2- 172.21.15.39:0/633531708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb1c162480 0x7fcb1c160b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:23.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.031+0000 7fcb21000640 1 --2- 172.21.15.39:0/633531708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb1c1630b0 0x7fcb1c0c4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:23.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.031+0000 7fd6b6c00640 1 -- 172.21.15.39:0/2898199198 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd6980051a0 msgr2=0x7fd69800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:23.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.031+0000 7fd6b6c00640 1 --2- 172.21.15.39:0/2898199198 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd6980051a0 0x7fd69800b9c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd6b00521d0 tx=0x7fd698005830 comp rx=0 tx=0).stop 2024-09-21T07:33:23.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.031+0000 7fd6b6200640 1 -- 172.21.15.39:0/2898199198 reap_dead start 2024-09-21T07:33:23.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.032+0000 7fd6b6c00640 1 -- 172.21.15.39:0/2898199198 shutdown_connections 2024-09-21T07:33:23.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.032+0000 7fcb21000640 1 -- 172.21.15.39:0/633531708 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c0cfbb0 con 0x7fcb1c1630b0 2024-09-21T07:33:23.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.032+0000 7fd6b6c00640 1 -- 172.21.15.39:0/2898199198 >> 172.21.15.39:0/2898199198 conn(0x7fd6a8051420 msgr2=0x7fd6a8051860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:23.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.032+0000 7fcb1a800640 1 --2- 172.21.15.39:0/633531708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb1c1630b0 0x7fcb1c0c4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:23.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.032+0000 7fcb21000640 1 -- 172.21.15.39:0/633531708 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c0cfa30 con 0x7fcb1c162480 2024-09-21T07:33:23.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.032+0000 7fcb21000640 1 -- 172.21.15.39:0/633531708 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fcb1c0cfd30 con 0x7fcb1c0b3b30 2024-09-21T07:33:23.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.033+0000 7fd6b6c00640 1 -- 172.21.15.39:0/2898199198 shutdown_connections 2024-09-21T07:33:23.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.034+0000 7fd6b6c00640 1 -- 172.21.15.39:0/2898199198 wait complete. 2024-09-21T07:33:23.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.035+0000 7fcb1a800640 1 -- 172.21.15.39:0/633531708 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcb1c0b3b30 msgr2=0x7fcb1c1fa750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:23.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.035+0000 7fcb1a800640 1 --2- 172.21.15.39:0/633531708 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcb1c0b3b30 0x7fcb1c1fa750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.035+0000 7fcb1a800640 1 -- 172.21.15.39:0/633531708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb1c162480 msgr2=0x7fcb1c160b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:23.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.035+0000 7fcb1a800640 1 --2- 172.21.15.39:0/633531708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb1c162480 0x7fcb1c160b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.035+0000 7fcb1a800640 1 -- 172.21.15.39:0/633531708 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb1c161090 con 0x7fcb1c1630b0 2024-09-21T07:33:23.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.036+0000 7fcb1a800640 1 --2- 172.21.15.39:0/633531708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb1c1630b0 0x7fcb1c0c4ea0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c08e9e0 tx=0x7fcb08033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:23.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.036+0000 7f42eb200640 1 -- 172.21.15.39:0/1391819560 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42e4070f20 msgr2=0x7f42e40cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:23.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.036+0000 7f42eb200640 1 --2- 172.21.15.39:0/1391819560 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42e4070f20 0x7f42e40cf010 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f42d4009890 tx=0x7f42d402c6c0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.037+0000 7f42eb200640 1 -- 172.21.15.39:0/1391819560 shutdown_connections 2024-09-21T07:33:23.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.037+0000 7f42eb200640 1 --2- 172.21.15.39:0/1391819560 >> v2:172.21.15.148:3300/0 conn(0x7f42e40cf660 0x7f42e40ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.037+0000 7f42eb200640 1 --2- 172.21.15.39:0/1391819560 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42e4070f20 0x7f42e40cf010 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.037+0000 7f42eb200640 1 -- 172.21.15.39:0/1391819560 >> 172.21.15.39:0/1391819560 conn(0x7f42e40b1cb0 msgr2=0x7f42e40b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:23.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.037+0000 7f42eb200640 1 -- 172.21.15.39:0/1391819560 shutdown_connections 2024-09-21T07:33:23.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.037+0000 7fd6b6c00640 1 Processor -- start 2024-09-21T07:33:23.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.037+0000 7f42eb200640 1 -- 172.21.15.39:0/1391819560 wait complete. 2024-09-21T07:33:23.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.038+0000 7fd6b6c00640 1 -- start start 2024-09-21T07:33:23.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.039+0000 7fd6b6c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6980051a0 0x7fd6a8038d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:23.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.039+0000 7fd6b6c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd6a80392a0 0x7fd6a801fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:23.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.039+0000 7fd6b6c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6a8020420 0x7fd6a801e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:23.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.040+0000 7fcb12000640 1 -- 172.21.15.39:0/633531708 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb08031b50 con 0x7fcb1c1630b0 2024-09-21T07:33:23.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.040+0000 7fcb12000640 1 -- 172.21.15.39:0/633531708 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb08031de0 con 0x7fcb1c1630b0 2024-09-21T07:33:23.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.040+0000 7fcb12000640 1 -- 172.21.15.39:0/633531708 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb08033330 con 0x7fcb1c1630b0 2024-09-21T07:33:23.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.040+0000 7fd6b6200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6a8020420 0x7fd6a801e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:23.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.040+0000 7fd6b6200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6a8020420 0x7fd6a801e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60068/0 (socket says 172.21.15.39:60068) 2024-09-21T07:33:23.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.040+0000 7fd6b5800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6980051a0 0x7fd6a8038d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:23.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.040+0000 7fd6b5800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6980051a0 0x7fd6a8038d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.39:59072/0 (socket says 172.21.15.39:59072) 2024-09-21T07:33:23.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.041+0000 7f42eb200640 1 Processor -- start 2024-09-21T07:33:23.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.042+0000 7f42eb200640 1 -- start start 2024-09-21T07:33:23.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.041+0000 7f6fff200640 1 -- 172.21.15.39:0/1202173543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ff80d4230 msgr2=0x7f6ff80d4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:23.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.041+0000 7f6fff200640 1 --2- 172.21.15.39:0/1202173543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ff80d4230 0x7f6ff80d4640 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4005fd0 tx=0x7f6fe402a540 comp rx=0 tx=0).stop 2024-09-21T07:33:23.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.041+0000 7fcb21000640 1 -- 172.21.15.39:0/633531708 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb1c0a2000 con 0x7fcb1c1630b0 2024-09-21T07:33:23.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.040+0000 7fd6b4e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd6a80392a0 0x7fd6a801fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:23.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.042+0000 7f6fff200640 1 -- 172.21.15.39:0/1202173543 shutdown_connections 2024-09-21T07:33:23.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.042+0000 7f6fff200640 1 --2- 172.21.15.39:0/1202173543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ff80d4230 0x7f6ff80d4640 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.042+0000 7f6fff200640 1 --2- 172.21.15.39:0/1202173543 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6ff80cfbc0 0x7f6ff80cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.042+0000 7f6fff200640 1 -- 172.21.15.39:0/1202173543 >> 172.21.15.39:0/1202173543 conn(0x7f6ff808bfc0 msgr2=0x7f6ff809d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:23.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.042+0000 7fd6b6200640 1 -- 172.21.15.39:0/2698409775 learned_addr learned my addr 172.21.15.39:0/2698409775 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:23.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.042+0000 7fd6b6c00640 1 -- 172.21.15.39:0/2698409775 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a8036cc0 con 0x7fd6a8020420 2024-09-21T07:33:23.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.042+0000 7fd6b6c00640 1 -- 172.21.15.39:0/2698409775 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a8036b40 con 0x7fd6980051a0 2024-09-21T07:33:23.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.042+0000 7fd6b6c00640 1 -- 172.21.15.39:0/2698409775 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd6a800bc30 con 0x7fd6a80392a0 2024-09-21T07:33:23.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.043+0000 7f6fff200640 1 -- 172.21.15.39:0/1202173543 shutdown_connections 2024-09-21T07:33:23.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.043+0000 7fd6b5800640 1 -- 172.21.15.39:0/2698409775 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd6a80392a0 msgr2=0x7fd6a801fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:23.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.043+0000 7fd6b5800640 1 --2- 172.21.15.39:0/2698409775 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd6a80392a0 0x7fd6a801fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.044+0000 7fd6b5800640 1 -- 172.21.15.39:0/2698409775 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6a8020420 msgr2=0x7fd6a801e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:23.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.044+0000 7fd6b5800640 1 --2- 172.21.15.39:0/2698409775 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6a8020420 0x7fd6a801e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.044+0000 7fd6b5800640 1 -- 172.21.15.39:0/2698409775 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a8020a80 con 0x7fd6980051a0 2024-09-21T07:33:23.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.044+0000 7fd6b6200640 1 --2- 172.21.15.39:0/2698409775 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6a8020420 0x7fd6a801e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:33:23.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.043+0000 7f6fff200640 1 -- 172.21.15.39:0/1202173543 wait complete. 2024-09-21T07:33:23.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.043+0000 7f8442200640 1 -- 172.21.15.39:0/3411285793 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84240039b0 msgr2=0x7f8424003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:23.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.043+0000 7f8442200640 1 --2- 172.21.15.39:0/3411285793 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84240039b0 0x7f8424003de0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f843000c720 tx=0x7f8424005830 comp rx=0 tx=0).stop 2024-09-21T07:33:23.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.042+0000 7fcb21000640 1 -- 172.21.15.39:0/633531708 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb1c09f060 con 0x7fcb1c1630b0 2024-09-21T07:33:23.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.044+0000 7fcb21000640 1 -- 172.21.15.39:0/633531708 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb1c0d0250 con 0x7fcb1c1630b0 2024-09-21T07:33:23.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.045+0000 7f42eb200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42e4070f20 0x7f42e4159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:23.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.045+0000 7fd6b5800640 1 --2- 172.21.15.39:0/2698409775 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6980051a0 0x7fd6a8038d60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4000fb0 tx=0x7fd6a40039c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:23.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.045+0000 7f42eb200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42e40cf660 0x7f42e41f43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:23.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.046+0000 7f42e3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42e4070f20 0x7f42e4159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:23.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.046+0000 7f42e3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42e4070f20 0x7f42e4159dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60080/0 (socket says 172.21.15.39:60080) 2024-09-21T07:33:23.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.046+0000 7f42e3e00640 1 -- 172.21.15.39:0/1106160591 learned_addr learned my addr 172.21.15.39:0/1106160591 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:23.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.047+0000 7f42eb200640 1 --2- 172.21.15.39:0/1106160591 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42e40cfb90 0x7f42e41f4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:23.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.047+0000 7f42eb200640 1 -- 172.21.15.39:0/1106160591 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f42e40cd1f0 con 0x7f42e4070f20 2024-09-21T07:33:23.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.047+0000 7f42dbe00640 1 --2- 172.21.15.39:0/1106160591 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42e40cf660 0x7f42e41f43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:23.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.047+0000 7f42eb200640 1 -- 172.21.15.39:0/1106160591 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f42e40cd070 con 0x7f42e40cfb90 2024-09-21T07:33:23.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.047+0000 7f42eb200640 1 -- 172.21.15.39:0/1106160591 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f42e40cd370 con 0x7f42e40cf660 2024-09-21T07:33:23.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.047+0000 7f8441800640 1 -- 172.21.15.39:0/3411285793 reap_dead start 2024-09-21T07:33:23.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.047+0000 7f42e9200640 1 --2- 172.21.15.39:0/1106160591 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42e40cfb90 0x7f42e41f4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:23.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.047+0000 7f8442200640 1 -- 172.21.15.39:0/3411285793 shutdown_connections 2024-09-21T07:33:23.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.047+0000 7fd6ad600640 1 -- 172.21.15.39:0/2698409775 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a4015070 con 0x7fd6980051a0 2024-09-21T07:33:23.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.047+0000 7f8442200640 1 -- 172.21.15.39:0/3411285793 >> 172.21.15.39:0/3411285793 conn(0x7f843c06d870 msgr2=0x7f843c06ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:23.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.048+0000 7f8442200640 1 -- 172.21.15.39:0/3411285793 shutdown_connections 2024-09-21T07:33:23.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.048+0000 7f8442200640 1 -- 172.21.15.39:0/3411285793 wait complete. 2024-09-21T07:33:23.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.048+0000 7fd6ad600640 1 -- 172.21.15.39:0/2698409775 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6a4011900 con 0x7fd6980051a0 2024-09-21T07:33:23.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.048+0000 7fd6ad600640 1 -- 172.21.15.39:0/2698409775 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a4011d30 con 0x7fd6980051a0 2024-09-21T07:33:23.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.048+0000 7fd6b6c00640 1 -- 172.21.15.39:0/2698409775 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6a80e0370 con 0x7fd6980051a0 2024-09-21T07:33:23.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.048+0000 7fd6b6c00640 1 -- 172.21.15.39:0/2698409775 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6a80e0930 con 0x7fd6980051a0 2024-09-21T07:33:23.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.048+0000 7f42e9200640 1 -- 172.21.15.39:0/1106160591 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42e40cf660 msgr2=0x7f42e41f43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:23.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.048+0000 7fcb12000640 1 -- 172.21.15.39:0/633531708 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcb080334d0 con 0x7fcb1c1630b0 2024-09-21T07:33:23.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.048+0000 7f42e9200640 1 --2- 172.21.15.39:0/1106160591 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42e40cf660 0x7f42e41f43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.048+0000 7f6fff200640 1 Processor -- start 2024-09-21T07:33:23.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.049+0000 7fa175600640 1 -- 172.21.15.39:0/1245337357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa17006e170 msgr2=0x7fa1700b3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:23.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.049+0000 7fa175600640 1 --2- 172.21.15.39:0/1245337357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa17006e170 0x7fa1700b3540 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa1700ba090 tx=0x7fa15802a4b0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.048+0000 7f42e9200640 1 -- 172.21.15.39:0/1106160591 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42e4070f20 msgr2=0x7f42e4159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:23.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.048+0000 7f42dbe00640 1 --2- 172.21.15.39:0/1106160591 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42e40cf660 0x7f42e41f43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:33:23.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.048+0000 7f42e9200640 1 --2- 172.21.15.39:0/1106160591 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42e4070f20 0x7f42e4159dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.048+0000 7f42e9200640 1 -- 172.21.15.39:0/1106160591 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e409dc20 con 0x7f42e40cfb90 2024-09-21T07:33:23.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.048+0000 7f42e3e00640 1 --2- 172.21.15.39:0/1106160591 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42e4070f20 0x7f42e4159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:33:23.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.049+0000 7f42e9200640 1 --2- 172.21.15.39:0/1106160591 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42e40cfb90 0x7f42e41f4920 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f42dc00c730 tx=0x7f42dc00cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:23.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.050+0000 7f42e0c00640 1 -- 172.21.15.39:0/1106160591 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc003b70 con 0x7f42e40cfb90 2024-09-21T07:33:23.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.050+0000 7f42e0c00640 1 -- 172.21.15.39:0/1106160591 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f42dc003d10 con 0x7f42e40cfb90 2024-09-21T07:33:23.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.050+0000 7f42e0c00640 1 -- 172.21.15.39:0/1106160591 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc0052d0 con 0x7f42e40cfb90 2024-09-21T07:33:23.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.050+0000 7fcb12000640 1 --2- 172.21.15.39:0/633531708 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcafc078bc0 0x7fcafc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:23.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.051+0000 7fcb12000640 1 -- 172.21.15.39:0/633531708 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fcb080bc490 con 0x7fcb1c1630b0 2024-09-21T07:33:23.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.051+0000 7fa175600640 1 -- 172.21.15.39:0/1245337357 shutdown_connections 2024-09-21T07:33:23.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.051+0000 7fa175600640 1 --2- 172.21.15.39:0/1245337357 >> v2:172.21.15.39:3300/0 conn(0x7fa1700ceb40 0x7fa1700cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.051+0000 7fa175600640 1 --2- 172.21.15.39:0/1245337357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa17006e170 0x7fa1700b3540 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.051+0000 7fa175600640 1 -- 172.21.15.39:0/1245337357 >> 172.21.15.39:0/1245337357 conn(0x7fa17008bfc0 msgr2=0x7fa17009d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:23.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.049+0000 7f6fff200640 1 -- start start 2024-09-21T07:33:23.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.051+0000 7f42eb200640 1 -- 172.21.15.39:0/1106160591 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42e409ad60 con 0x7f42e40cfb90 2024-09-21T07:33:23.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.052+0000 7fa175600640 1 -- 172.21.15.39:0/1245337357 shutdown_connections 2024-09-21T07:33:23.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.052+0000 7fcb19e00640 1 --2- 172.21.15.39:0/633531708 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcafc078bc0 0x7fcafc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:23.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.052+0000 7fa175600640 1 -- 172.21.15.39:0/1245337357 wait complete. 2024-09-21T07:33:23.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.051+0000 7f42eb200640 1 -- 172.21.15.39:0/1106160591 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42e409d660 con 0x7f42e40cfb90 2024-09-21T07:33:23.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.052+0000 7fd6ad600640 1 -- 172.21.15.39:0/2698409775 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd6a4018070 con 0x7fd6980051a0 2024-09-21T07:33:23.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.052+0000 7f6fff200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ff80b3910 0x7f6ff80734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:23.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.053+0000 7f6fff200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6ff80cfbc0 0x7f6ff80caae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:23.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.053+0000 7f8442200640 1 Processor -- start 2024-09-21T07:33:23.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.053+0000 7fcb19e00640 1 --2- 172.21.15.39:0/633531708 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcafc078bc0 0x7fcafc07b0d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcb140059a0 tx=0x7fcb14009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:23.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.053+0000 7f42eb200640 1 -- 172.21.15.39:0/1106160591 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42e40cf010 con 0x7f42e40cfb90 2024-09-21T07:33:23.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.053+0000 7f6fff200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ff80d4230 0x7f6ff815ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:23.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.053+0000 7f8442200640 1 -- start start 2024-09-21T07:33:23.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.054+0000 7f6fff200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff80b5c00 con 0x7f6ff80b3910 2024-09-21T07:33:23.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.054+0000 7fd6b6c00640 1 -- 172.21.15.39:0/2698409775 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6a800a3f0 con 0x7fd6980051a0 2024-09-21T07:33:23.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.054+0000 7f6ff7400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6ff80cfbc0 0x7f6ff80caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:23.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.054+0000 7f6fff200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff80b5a80 con 0x7f6ff80d4230 2024-09-21T07:33:23.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.054+0000 7f6fff200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f6ff80d4b80 con 0x7f6ff80cfbc0 2024-09-21T07:33:23.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.054+0000 7fd6ad600640 1 --2- 172.21.15.39:0/2698409775 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd690078b60 0x7fd69007b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:23.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.054+0000 7fd6ad600640 1 -- 172.21.15.39:0/2698409775 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd6a409ace0 con 0x7fd6980051a0 2024-09-21T07:33:23.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.054+0000 7f6ff7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ff80b3910 0x7f6ff80734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:23.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.054+0000 7f6ff7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ff80b3910 0x7f6ff80734b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60100/0 (socket says 172.21.15.39:60100) 2024-09-21T07:33:23.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.054+0000 7f6ff7e00640 1 -- 172.21.15.39:0/3777912717 learned_addr learned my addr 172.21.15.39:0/3777912717 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:23.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.054+0000 7f6ffd200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ff80d4230 0x7f6ff815ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:23.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.055+0000 7fd6b4e00640 1 --2- 172.21.15.39:0/2698409775 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd690078b60 0x7fd69007b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:23.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.055+0000 7f42e0c00640 1 -- 172.21.15.39:0/1106160591 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f42dc005470 con 0x7f42e40cfb90 2024-09-21T07:33:23.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.055+0000 7f6ffd200640 1 -- 172.21.15.39:0/3777912717 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6ff80cfbc0 msgr2=0x7f6ff80caae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:23.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.055+0000 7f6ffd200640 1 --2- 172.21.15.39:0/3777912717 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6ff80cfbc0 0x7f6ff80caae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.055+0000 7f6ffd200640 1 -- 172.21.15.39:0/3777912717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ff80b3910 msgr2=0x7f6ff80734b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:23.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.055+0000 7f6ffd200640 1 --2- 172.21.15.39:0/3777912717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ff80b3910 0x7f6ff80734b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.055+0000 7f6ffd200640 1 -- 172.21.15.39:0/3777912717 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ff80708c0 con 0x7f6ff80d4230 2024-09-21T07:33:23.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.056+0000 7f6ffd200640 1 --2- 172.21.15.39:0/3777912717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ff80d4230 0x7f6ff815ccf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4031000 tx=0x7f6fe4003800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:23.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.056+0000 7fa175600640 1 Processor -- start 2024-09-21T07:33:23.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.056+0000 7fa175600640 1 -- start start 2024-09-21T07:33:23.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.056+0000 7f42e0c00640 1 --2- 172.21.15.39:0/1106160591 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f42c0078be0 0x7f42c007b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:23.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.056+0000 7f42e0c00640 1 -- 172.21.15.39:0/1106160591 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f42dc09a5a0 con 0x7f42e40cfb90 2024-09-21T07:33:23.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.056+0000 7f8442200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84240039b0 0x7f843c0cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:23.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.057+0000 7fd6b4e00640 1 --2- 172.21.15.39:0/2698409775 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd690078b60 0x7fd69007b070 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd6a818a910 tx=0x7fd69c00d360 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:23.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.057+0000 7f8442200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f843c0ccdf0 0x7f843c1412d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:23.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.057+0000 7f8440e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84240039b0 0x7f843c0cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:23.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.057+0000 7f42e3e00640 1 --2- 172.21.15.39:0/1106160591 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f42c0078be0 0x7f42c007b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:23.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.057+0000 7fcb12000640 1 -- 172.21.15.39:0/633531708 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcb08052080 con 0x7fcb1c1630b0 2024-09-21T07:33:23.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.057+0000 7f8440e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84240039b0 0x7f843c0cc8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.39:59110/0 (socket says 172.21.15.39:59110) 2024-09-21T07:33:23.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.057+0000 7f8442200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f843c147a10 0x7f843c141810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:23.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.058+0000 7f42e3e00640 1 --2- 172.21.15.39:0/1106160591 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f42c0078be0 0x7f42c007b0f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f42d4009890 tx=0x7f42d4004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:23.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.058+0000 7f8440e00640 1 -- 172.21.15.39:0/3022467020 learned_addr learned my addr 172.21.15.39:0/3022467020 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:23.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.058+0000 7f8442200640 1 -- 172.21.15.39:0/3022467020 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f843c0734f0 con 0x7f843c147a10 2024-09-21T07:33:23.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.058+0000 7f8442200640 1 -- 172.21.15.39:0/3022467020 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f843c073370 con 0x7f84240039b0 2024-09-21T07:33:23.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.058+0000 7f8442200640 1 -- 172.21.15.39:0/3022467020 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f843c0ca710 con 0x7f843c0ccdf0 2024-09-21T07:33:23.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.058+0000 7f843be00640 1 --2- 172.21.15.39:0/3022467020 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f843c0ccdf0 0x7f843c1412d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:23.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.058+0000 7f8441800640 1 --2- 172.21.15.39:0/3022467020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f843c147a10 0x7f843c141810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:23.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.058+0000 7ff4f8800640 1 -- 172.21.15.39:0/565478214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4f0073170 msgr2=0x7ff4f015c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:23.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.058+0000 7ff4f8800640 1 --2- 172.21.15.39:0/565478214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4f0073170 0x7ff4f015c780 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec005fd0 tx=0x7ff4ec02a970 comp rx=0 tx=0).stop 2024-09-21T07:33:23.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.058+0000 7f74d9800640 1 -- 172.21.15.39:0/3529361460 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74d40ceb40 msgr2=0x7f74d40cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:23.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.058+0000 7f8440e00640 1 -- 172.21.15.39:0/3022467020 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f843c0ccdf0 msgr2=0x7f843c1412d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:23.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.058+0000 7f8440e00640 1 --2- 172.21.15.39:0/3022467020 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f843c0ccdf0 0x7f843c1412d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.058+0000 7f8440e00640 1 -- 172.21.15.39:0/3022467020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f843c147a10 msgr2=0x7f843c141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:23.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.058+0000 7ff4f8800640 1 -- 172.21.15.39:0/565478214 shutdown_connections 2024-09-21T07:33:23.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.058+0000 7ff4f8800640 1 --2- 172.21.15.39:0/565478214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4f0073170 0x7ff4f015c780 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.058+0000 7ff4f8800640 1 -- 172.21.15.39:0/565478214 >> 172.21.15.39:0/565478214 conn(0x7ff4f00b1cb0 msgr2=0x7ff4f00b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:23.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.058+0000 7f74d9800640 1 --2- 172.21.15.39:0/3529361460 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74d40ceb40 0x7f74d40cef50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f74cc005fd0 tx=0x7f74cc02a8a0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.058+0000 7f74d9800640 1 -- 172.21.15.39:0/3529361460 shutdown_connections 2024-09-21T07:33:23.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.058+0000 7f74d9800640 1 --2- 172.21.15.39:0/3529361460 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74d40ceb40 0x7f74d40cef50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.058+0000 7f74d9800640 1 -- 172.21.15.39:0/3529361460 >> 172.21.15.39:0/3529361460 conn(0x7f74d408bfc0 msgr2=0x7f74d409d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:23.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.059+0000 7f74d9800640 1 -- 172.21.15.39:0/3529361460 shutdown_connections 2024-09-21T07:33:23.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.059+0000 7f8440e00640 1 --2- 172.21.15.39:0/3022467020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f843c147a10 0x7f843c141810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.059+0000 7f8440e00640 1 -- 172.21.15.39:0/3022467020 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f843c0b2b20 con 0x7f84240039b0 2024-09-21T07:33:23.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.059+0000 7f8441800640 1 --2- 172.21.15.39:0/3022467020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f843c147a10 0x7f843c141810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:33:23.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.059+0000 7ff4f8800640 1 -- 172.21.15.39:0/565478214 shutdown_connections 2024-09-21T07:33:23.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.059+0000 7f8440e00640 1 --2- 172.21.15.39:0/3022467020 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84240039b0 0x7f843c0cc8b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8430005fd0 tx=0x7f843000c340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:23.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.059+0000 7ff4f8800640 1 -- 172.21.15.39:0/565478214 wait complete. 2024-09-21T07:33:23.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.059+0000 7fa175600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa17006f4b0 0x7fa170160ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:23.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.059+0000 7fa175600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1700ceb40 0x7fa170161400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:23.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.059+0000 7f74d9800640 1 -- 172.21.15.39:0/3529361460 wait complete. 2024-09-21T07:33:23.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.059+0000 7fa175600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1701fb1c0 0x7fa1701fb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:23.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.059+0000 7fa175600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa1700d0e60 con 0x7fa1700ceb40 2024-09-21T07:33:23.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.059+0000 7fa175600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa1700d0ce0 con 0x7fa1701fb1c0 2024-09-21T07:33:23.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.059+0000 7fa175600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa1700d0fe0 con 0x7fa17006f4b0 2024-09-21T07:33:23.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.060+0000 7fa16ea00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa17006f4b0 0x7fa170160ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:23.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.060+0000 7fa16ea00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa17006f4b0 0x7fa170160ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.39:40974/0 (socket says 172.21.15.39:40974) 2024-09-21T07:33:23.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.060+0000 7fa16ea00640 1 -- 172.21.15.39:0/2934075726 learned_addr learned my addr 172.21.15.39:0/2934075726 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:23.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.060+0000 7fa16f400640 1 --2- 172.21.15.39:0/2934075726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1701fb1c0 0x7fa1701fb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:23.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.060+0000 7fa166000640 1 --2- 172.21.15.39:0/2934075726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1700ceb40 0x7fa170161400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:23.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.060+0000 7fa16ea00640 1 -- 172.21.15.39:0/2934075726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1701fb1c0 msgr2=0x7fa1701fb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:23.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.060+0000 7fa16ea00640 1 --2- 172.21.15.39:0/2934075726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1701fb1c0 0x7fa1701fb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.060+0000 7fa16ea00640 1 -- 172.21.15.39:0/2934075726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1700ceb40 msgr2=0x7fa170161400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:23.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.060+0000 7fa16ea00640 1 --2- 172.21.15.39:0/2934075726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1700ceb40 0x7fa170161400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.060+0000 7fa16ea00640 1 -- 172.21.15.39:0/2934075726 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1701fc0a0 con 0x7fa17006f4b0 2024-09-21T07:33:23.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.061+0000 7fa166000640 1 --2- 172.21.15.39:0/2934075726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1700ceb40 0x7fa170161400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:33:23.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.061+0000 7fa16ea00640 1 --2- 172.21.15.39:0/2934075726 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa17006f4b0 0x7fa170160ec0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa158034040 tx=0x7fa158003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:23.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.061+0000 7fd6ad600640 1 -- 172.21.15.39:0/2698409775 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd6a8020a80 con 0x7fd6980051a0 2024-09-21T07:33:23.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.061+0000 7ff4f8800640 1 Processor -- start 2024-09-21T07:33:23.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.062+0000 7ff4f8800640 1 -- start start 2024-09-21T07:33:23.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.062+0000 7fa167400640 1 -- 172.21.15.39:0/2934075726 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15803e030 con 0x7fa17006f4b0 2024-09-21T07:33:23.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.062+0000 7fa167400640 1 -- 172.21.15.39:0/2934075726 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1580047a0 con 0x7fa17006f4b0 2024-09-21T07:33:23.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.062+0000 7fa167400640 1 -- 172.21.15.39:0/2934075726 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa158035040 con 0x7fa17006f4b0 2024-09-21T07:33:23.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.062+0000 7f74d9800640 1 Processor -- start 2024-09-21T07:33:23.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.062+0000 7fa175600640 1 -- 172.21.15.39:0/2934075726 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1701fc380 con 0x7fa17006f4b0 2024-09-21T07:33:23.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.062+0000 7fa175600640 1 -- 172.21.15.39:0/2934075726 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1701fc940 con 0x7fa17006f4b0 2024-09-21T07:33:23.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.062+0000 7f74d9800640 1 -- start start 2024-09-21T07:33:23.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.064+0000 7ff4f8800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4f0070030 0x7ff4f01e94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:23.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.064+0000 7fa15fe00640 1 -- 172.21.15.39:0/2934075726 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1700d1600 con 0x7fa17006f4b0 2024-09-21T07:33:23.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.064+0000 7ff4f8800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4f0070440 0x7ff4f01f5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:23.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.064+0000 7ff4f8800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4f0073170 0x7ff4f015aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:23.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.064+0000 7ff4f8800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f00b3cd0 con 0x7ff4f0070030 2024-09-21T07:33:23.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.064+0000 7ff4f8800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f00b3b50 con 0x7ff4f0073170 2024-09-21T07:33:23.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.064+0000 7ff4f8800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7ff4f00b3e50 con 0x7ff4f0070440 2024-09-21T07:33:23.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.064+0000 7ff4f5400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4f0070440 0x7ff4f01f5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:23.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.064+0000 7ff4f6800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4f0073170 0x7ff4f015aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:23.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.064+0000 7ff4f5400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4f0070440 0x7ff4f01f5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.39:40986/0 (socket says 172.21.15.39:40986) 2024-09-21T07:33:23.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.064+0000 7ff4f5400640 1 -- 172.21.15.39:0/3805520136 learned_addr learned my addr 172.21.15.39:0/3805520136 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:23.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.065+0000 7f74d9800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74d40b3540 0x7f74d40cae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:23.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.065+0000 7f74d9800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f74d40ceb40 0x7f74d415fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:23.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.064+0000 7ff4f5e00640 1 --2- 172.21.15.39:0/3805520136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4f0070030 0x7ff4f01e94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:23.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.065+0000 7ff4f5400640 1 -- 172.21.15.39:0/3805520136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4f0073170 msgr2=0x7ff4f015aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:23.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.065+0000 7ff4f5400640 1 --2- 172.21.15.39:0/3805520136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4f0073170 0x7ff4f015aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.065+0000 7ff4f5400640 1 -- 172.21.15.39:0/3805520136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4f0070030 msgr2=0x7ff4f01e94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:23.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.065+0000 7ff4f5400640 1 --2- 172.21.15.39:0/3805520136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4f0070030 0x7ff4f01e94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.065+0000 7ff4f5400640 1 -- 172.21.15.39:0/3805520136 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f01e9a20 con 0x7ff4f0070440 2024-09-21T07:33:23.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.065+0000 7ff4f6800640 1 --2- 172.21.15.39:0/3805520136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4f0073170 0x7ff4f015aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:33:23.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.065+0000 7ff4f5e00640 1 --2- 172.21.15.39:0/3805520136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4f0070030 0x7ff4f01e94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:33:23.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.065+0000 7f74d9800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74d40c85e0 0x7f74d415b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:23.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.065+0000 7f74d9800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f74d40d1070 con 0x7f74d40c85e0 2024-09-21T07:33:23.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.065+0000 7f74d9800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f74d40d0ef0 con 0x7f74d40b3540 2024-09-21T07:33:23.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.065+0000 7f74d9800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f74d40d11f0 con 0x7f74d40ceb40 2024-09-21T07:33:23.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.065+0000 7f74d2000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f74d40ceb40 0x7f74d415fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:23.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.065+0000 7f74d2000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f74d40ceb40 0x7f74d415fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.39:40992/0 (socket says 172.21.15.39:40992) 2024-09-21T07:33:23.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.065+0000 7f74d2000640 1 -- 172.21.15.39:0/749266464 learned_addr learned my addr 172.21.15.39:0/749266464 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:23.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.065+0000 7f74d2a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74d40b3540 0x7f74d40cae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:23.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.066+0000 7f6fefe00640 1 -- 172.21.15.39:0/3777912717 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe403e030 con 0x7f6ff80d4230 2024-09-21T07:33:23.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.066+0000 7ff4f5400640 1 --2- 172.21.15.39:0/3805520136 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4f0070440 0x7ff4f01f5560 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff4e400b3f0 tx=0x7ff4e400b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:23.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.066+0000 7f74d2000640 1 -- 172.21.15.39:0/749266464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74d40b3540 msgr2=0x7f74d40cae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:23.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.066+0000 7f74d2000640 1 --2- 172.21.15.39:0/749266464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74d40b3540 0x7f74d40cae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.066+0000 7f74d2000640 1 -- 172.21.15.39:0/749266464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74d40c85e0 msgr2=0x7f74d415b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:23.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.066+0000 7f74d3400640 1 --2- 172.21.15.39:0/749266464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74d40c85e0 0x7f74d415b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:23.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.066+0000 7f6fff200640 1 -- 172.21.15.39:0/3777912717 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ff80705e0 con 0x7f6ff80d4230 2024-09-21T07:33:23.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.066+0000 7f6fefe00640 1 -- 172.21.15.39:0/3777912717 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fe4003db0 con 0x7f6ff80d4230 2024-09-21T07:33:23.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.066+0000 7f6fefe00640 1 -- 172.21.15.39:0/3777912717 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe4035040 con 0x7f6ff80d4230 2024-09-21T07:33:23.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.066+0000 7f6fff200640 1 -- 172.21.15.39:0/3777912717 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ff8072cb0 con 0x7f6ff80d4230 2024-09-21T07:33:23.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.066+0000 7f74d2000640 1 --2- 172.21.15.39:0/749266464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74d40c85e0 0x7f74d415b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.066+0000 7f74d2000640 1 -- 172.21.15.39:0/749266464 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d415c2c0 con 0x7f74d40ceb40 2024-09-21T07:33:23.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.066+0000 7f74d3400640 1 --2- 172.21.15.39:0/749266464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74d40c85e0 0x7f74d415b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:33:23.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.067+0000 7f42e0c00640 1 -- 172.21.15.39:0/1106160591 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f42dc020060 con 0x7f42e40cfb90 2024-09-21T07:33:23.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.067+0000 7f8438c00640 1 -- 172.21.15.39:0/3022467020 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430043070 con 0x7f84240039b0 2024-09-21T07:33:23.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.067+0000 7f8438c00640 1 -- 172.21.15.39:0/3022467020 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f843002bba0 con 0x7f84240039b0 2024-09-21T07:33:23.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.067+0000 7f8438c00640 1 -- 172.21.15.39:0/3022467020 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430006030 con 0x7f84240039b0 2024-09-21T07:33:23.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.067+0000 7f74d2000640 1 --2- 172.21.15.39:0/749266464 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f74d40ceb40 0x7f74d415fe70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f74c000c3f0 tx=0x7f74c000c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:23.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.067+0000 7fa167400640 1 -- 172.21.15.39:0/2934075726 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa158041030 con 0x7fa17006f4b0 2024-09-21T07:33:23.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.067+0000 7ff4ea000640 1 -- 172.21.15.39:0/3805520136 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4015020 con 0x7ff4f0070440 2024-09-21T07:33:23.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.067+0000 7ff4ea000640 1 -- 172.21.15.39:0/3805520136 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4e4011920 con 0x7ff4f0070440 2024-09-21T07:33:23.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.067+0000 7ff4ea000640 1 -- 172.21.15.39:0/3805520136 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4011d70 con 0x7ff4f0070440 2024-09-21T07:33:23.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.067+0000 7ff4f8800640 1 -- 172.21.15.39:0/3805520136 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4f00cf3e0 con 0x7ff4f0070440 2024-09-21T07:33:23.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.067+0000 7f8442200640 1 -- 172.21.15.39:0/3022467020 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f843c14c720 con 0x7f84240039b0 2024-09-21T07:33:23.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.067+0000 7f8442200640 1 -- 172.21.15.39:0/3022467020 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f843c0b2840 con 0x7f84240039b0 2024-09-21T07:33:23.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.068+0000 7ff4f8800640 1 -- 172.21.15.39:0/3805520136 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4f01ea210 con 0x7ff4f0070440 2024-09-21T07:33:23.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.068+0000 7fa167400640 1 --2- 172.21.15.39:0/2934075726 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa14c078be0 0x7fa14c07b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:23.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.069+0000 7fa167400640 1 -- 172.21.15.39:0/2934075726 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa1580bb5a0 con 0x7fa17006f4b0 2024-09-21T07:33:23.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.067+0000 7f6fff200640 1 -- 172.21.15.39:0/3777912717 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ff80cffd0 con 0x7f6ff80d4230 2024-09-21T07:33:23.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.069+0000 7ff4f8800640 1 -- 172.21.15.39:0/3805520136 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4f01e9ca0 con 0x7ff4f0070440 2024-09-21T07:33:23.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.069+0000 7f8442200640 1 -- 172.21.15.39:0/3022467020 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f843c15a490 con 0x7f84240039b0 2024-09-21T07:33:23.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.070+0000 7fa166000640 1 --2- 172.21.15.39:0/2934075726 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa14c078be0 0x7fa14c07b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:23.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.070+0000 7fa166000640 1 --2- 172.21.15.39:0/2934075726 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa14c078be0 0x7fa14c07b0f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa160006fd0 tx=0x7fa160008040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:23.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.070+0000 7f6fefe00640 1 -- 172.21.15.39:0/3777912717 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6fe4041030 con 0x7f6ff80d4230 2024-09-21T07:33:23.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.071+0000 7f6fefe00640 1 --2- 172.21.15.39:0/3777912717 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6fd8078bc0 0x7f6fd807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:23.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.071+0000 7f6fefe00640 1 -- 172.21.15.39:0/3777912717 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6fe40bb7f0 con 0x7f6ff80d4230 2024-09-21T07:33:23.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.072+0000 7f6ff7e00640 1 --2- 172.21.15.39:0/3777912717 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6fd8078bc0 0x7f6fd807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:23.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.072+0000 7f6ff7e00640 1 --2- 172.21.15.39:0/3777912717 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6fd8078bc0 0x7f6fd807b0d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8004f40 tx=0x7f6fe8007ef0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:23.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.073+0000 7f6fefe00640 1 -- 172.21.15.39:0/3777912717 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6ff80708c0 con 0x7f6ff80d4230 2024-09-21T07:33:23.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.073+0000 7f74caa00640 1 -- 172.21.15.39:0/749266464 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c0017070 con 0x7f74d40ceb40 2024-09-21T07:33:23.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.074+0000 7f74d9800640 1 -- 172.21.15.39:0/749266464 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74d415c560 con 0x7f74d40ceb40 2024-09-21T07:33:23.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.074+0000 7f74d9800640 1 -- 172.21.15.39:0/749266464 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f74d40c92f0 con 0x7f74d40ceb40 2024-09-21T07:33:23.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.074+0000 7fa167400640 1 -- 172.21.15.39:0/2934075726 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa158083370 con 0x7fa17006f4b0 2024-09-21T07:33:23.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.074+0000 7f74caa00640 1 -- 172.21.15.39:0/749266464 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f74c00053e0 con 0x7f74d40ceb40 2024-09-21T07:33:23.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.074+0000 7f74caa00640 1 -- 172.21.15.39:0/749266464 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c0003870 con 0x7f74d40ceb40 2024-09-21T07:33:23.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.076+0000 7f74d9800640 1 -- 172.21.15.39:0/749266464 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74ac005290 con 0x7f74d40ceb40 2024-09-21T07:33:23.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.078+0000 7ff4ea000640 1 -- 172.21.15.39:0/3805520136 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff4e4018020 con 0x7ff4f0070440 2024-09-21T07:33:23.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.077+0000 7f8438c00640 1 -- 172.21.15.39:0/3022467020 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f84300061d0 con 0x7f84240039b0 2024-09-21T07:33:23.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.079+0000 7f8438c00640 1 --2- 172.21.15.39:0/3022467020 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f841c078bc0 0x7f841c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:23.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.079+0000 7f8438c00640 1 -- 172.21.15.39:0/3022467020 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f84300bb340 con 0x7f84240039b0 2024-09-21T07:33:23.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.079+0000 7f843be00640 1 --2- 172.21.15.39:0/3022467020 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f841c078bc0 0x7f841c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:23.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.080+0000 7f843be00640 1 --2- 172.21.15.39:0/3022467020 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f841c078bc0 0x7f841c07b0d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f843c0b2280 tx=0x7f8428006cf0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:23.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.080+0000 7f8438c00640 1 -- 172.21.15.39:0/3022467020 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f84300831a0 con 0x7f84240039b0 2024-09-21T07:33:23.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.081+0000 7ff4ea000640 1 --2- 172.21.15.39:0/3805520136 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff4d4078c10 0x7ff4d407b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:23.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.081+0000 7ff4ea000640 1 -- 172.21.15.39:0/3805520136 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff4e409b110 con 0x7ff4f0070440 2024-09-21T07:33:23.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.081+0000 7ff4ea000640 1 -- 172.21.15.39:0/3805520136 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff4e401f020 con 0x7ff4f0070440 2024-09-21T07:33:23.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.083+0000 7ff4f5e00640 1 --2- 172.21.15.39:0/3805520136 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff4d4078c10 0x7ff4d407b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:23.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.085+0000 7f74caa00640 1 -- 172.21.15.39:0/749266464 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f74c001a070 con 0x7f74d40ceb40 2024-09-21T07:33:23.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.087+0000 7f74caa00640 1 --2- 172.21.15.39:0/749266464 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f74b4078b90 0x7f74b407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:23.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.087+0000 7f74caa00640 1 -- 172.21.15.39:0/749266464 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f74c009a330 con 0x7f74d40ceb40 2024-09-21T07:33:23.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.088+0000 7ff4f5e00640 1 --2- 172.21.15.39:0/3805520136 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff4d4078c10 0x7ff4d407b120 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec035000 tx=0x7ff4ec004ab0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:23.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.088+0000 7f74caa00640 1 -- 172.21.15.39:0/749266464 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f74c0021080 con 0x7f74d40ceb40 2024-09-21T07:33:23.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.092+0000 7f74d2a00640 1 --2- 172.21.15.39:0/749266464 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f74b4078b90 0x7f74b407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:23.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.093+0000 7f74d2a00640 1 --2- 172.21.15.39:0/749266464 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f74b4078b90 0x7f74b407b0a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f74c40046b0 tx=0x7f74c4009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:23.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.506+0000 7fcb21000640 1 -- 172.21.15.39:0/633531708 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fcb1c0d16b0 con 0x7fcb1c1630b0 2024-09-21T07:33:23.511 INFO:teuthology.orchestra.run.smithi039.stdout:60129542149 2024-09-21T07:33:23.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.507+0000 7fcb12000640 1 -- 172.21.15.39:0/633531708 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcb0803f030 con 0x7fcb1c1630b0 2024-09-21T07:33:23.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.512+0000 7fcb21000640 1 -- 172.21.15.39:0/633531708 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcafc078bc0 msgr2=0x7fcafc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:23.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.512+0000 7fcb21000640 1 --2- 172.21.15.39:0/633531708 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcafc078bc0 0x7fcafc07b0d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcb140059a0 tx=0x7fcb14009040 comp rx=0 tx=0).stop 2024-09-21T07:33:23.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.512+0000 7fcb21000640 1 -- 172.21.15.39:0/633531708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb1c1630b0 msgr2=0x7fcb1c0c4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:23.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.512+0000 7fcb21000640 1 --2- 172.21.15.39:0/633531708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb1c1630b0 0x7fcb1c0c4ea0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c08e9e0 tx=0x7fcb08033ba0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.513+0000 7fcb21000640 1 -- 172.21.15.39:0/633531708 shutdown_connections 2024-09-21T07:33:23.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.513+0000 7fcb21000640 1 --2- 172.21.15.39:0/633531708 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcafc078bc0 0x7fcafc07b0d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.513+0000 7fcb21000640 1 --2- 172.21.15.39:0/633531708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb1c1630b0 0x7fcb1c0c4ea0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.513+0000 7fcb21000640 1 --2- 172.21.15.39:0/633531708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb1c162480 0x7fcb1c160b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.513+0000 7fcb21000640 1 --2- 172.21.15.39:0/633531708 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcb1c0b3b30 0x7fcb1c1fa750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.513+0000 7fcb21000640 1 -- 172.21.15.39:0/633531708 >> 172.21.15.39:0/633531708 conn(0x7fcb1c09d680 msgr2=0x7fcb1c06e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:23.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.514+0000 7fcb21000640 1 -- 172.21.15.39:0/633531708 shutdown_connections 2024-09-21T07:33:23.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.514+0000 7fcb21000640 1 -- 172.21.15.39:0/633531708 wait complete. 2024-09-21T07:33:23.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.520+0000 7fd6b6c00640 1 -- 172.21.15.39:0/2698409775 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fd6a8051b90 con 0x7fd6980051a0 2024-09-21T07:33:23.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.521+0000 7fd6ad600640 1 -- 172.21.15.39:0/2698409775 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd6a8006070 con 0x7fd6980051a0 2024-09-21T07:33:23.527 INFO:teuthology.orchestra.run.smithi039.stdout:64424509446 2024-09-21T07:33:23.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.525+0000 7fd6b6c00640 1 -- 172.21.15.39:0/2698409775 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd690078b60 msgr2=0x7fd69007b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:23.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.525+0000 7fd6b6c00640 1 --2- 172.21.15.39:0/2698409775 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd690078b60 0x7fd69007b070 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd6a818a910 tx=0x7fd69c00d360 comp rx=0 tx=0).stop 2024-09-21T07:33:23.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.525+0000 7fd6b6c00640 1 -- 172.21.15.39:0/2698409775 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6980051a0 msgr2=0x7fd6a8038d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:23.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.525+0000 7fd6b6c00640 1 --2- 172.21.15.39:0/2698409775 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6980051a0 0x7fd6a8038d60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4000fb0 tx=0x7fd6a40039c0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.526+0000 7fd6b6c00640 1 -- 172.21.15.39:0/2698409775 shutdown_connections 2024-09-21T07:33:23.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.526+0000 7fd6b6c00640 1 --2- 172.21.15.39:0/2698409775 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd690078b60 0x7fd69007b070 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.526+0000 7fd6b6c00640 1 --2- 172.21.15.39:0/2698409775 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6a8020420 0x7fd6a801e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.526+0000 7fd6b6c00640 1 --2- 172.21.15.39:0/2698409775 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd6a80392a0 0x7fd6a801fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.526+0000 7fd6b6c00640 1 --2- 172.21.15.39:0/2698409775 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6980051a0 0x7fd6a8038d60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.526+0000 7fd6b6c00640 1 -- 172.21.15.39:0/2698409775 >> 172.21.15.39:0/2698409775 conn(0x7fd6a8051420 msgr2=0x7fd6a8053ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:23.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.526+0000 7fd6b6c00640 1 -- 172.21.15.39:0/2698409775 shutdown_connections 2024-09-21T07:33:23.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.526+0000 7fd6b6c00640 1 -- 172.21.15.39:0/2698409775 wait complete. 2024-09-21T07:33:23.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.531+0000 7f42eb200640 1 -- 172.21.15.39:0/1106160591 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f42e406ddf0 con 0x7f42e40cfb90 2024-09-21T07:33:23.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.531+0000 7fa15fe00640 1 -- 172.21.15.39:0/2934075726 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fa17006efa0 con 0x7fa17006f4b0 2024-09-21T07:33:23.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.532+0000 7fa167400640 1 -- 172.21.15.39:0/2934075726 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa158087da0 con 0x7fa17006f4b0 2024-09-21T07:33:23.537 INFO:teuthology.orchestra.run.smithi039.stdout:64424509447 2024-09-21T07:33:23.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.532+0000 7f42e0c00640 1 -- 172.21.15.39:0/1106160591 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f42dc06b020 con 0x7f42e40cfb90 2024-09-21T07:33:23.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.535+0000 7f6fff200640 1 -- 172.21.15.39:0/3777912717 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f6ff80b4230 con 0x7f6ff80d4230 2024-09-21T07:33:23.538 INFO:teuthology.orchestra.run.smithi039.stdout:64424509445 2024-09-21T07:33:23.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.536+0000 7f6fefe00640 1 -- 172.21.15.39:0/3777912717 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6fe40835c0 con 0x7f6ff80d4230 2024-09-21T07:33:23.539 INFO:teuthology.orchestra.run.smithi039.stdout:64424509445 2024-09-21T07:33:23.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.536+0000 7fa15fe00640 1 -- 172.21.15.39:0/2934075726 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa14c078be0 msgr2=0x7fa14c07b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:23.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.536+0000 7fa15fe00640 1 --2- 172.21.15.39:0/2934075726 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa14c078be0 0x7fa14c07b0f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa160006fd0 tx=0x7fa160008040 comp rx=0 tx=0).stop 2024-09-21T07:33:23.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.537+0000 7fa15fe00640 1 -- 172.21.15.39:0/2934075726 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa17006f4b0 msgr2=0x7fa170160ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:23.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.537+0000 7fa15fe00640 1 --2- 172.21.15.39:0/2934075726 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa17006f4b0 0x7fa170160ec0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa158034040 tx=0x7fa158003b90 comp rx=0 tx=0).stop 2024-09-21T07:33:23.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.537+0000 7fa15fe00640 1 -- 172.21.15.39:0/2934075726 shutdown_connections 2024-09-21T07:33:23.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.537+0000 7fa15fe00640 1 --2- 172.21.15.39:0/2934075726 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa14c078be0 0x7fa14c07b0f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.537+0000 7fa15fe00640 1 --2- 172.21.15.39:0/2934075726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1701fb1c0 0x7fa1701fb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.537+0000 7fa15fe00640 1 --2- 172.21.15.39:0/2934075726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1700ceb40 0x7fa170161400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.537+0000 7fa15fe00640 1 --2- 172.21.15.39:0/2934075726 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa17006f4b0 0x7fa170160ec0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.538+0000 7f8442200640 1 -- 172.21.15.39:0/3022467020 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f843c0d07c0 con 0x7f84240039b0 2024-09-21T07:33:23.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.537+0000 7fa15fe00640 1 -- 172.21.15.39:0/2934075726 >> 172.21.15.39:0/2934075726 conn(0x7fa17008bfc0 msgr2=0x7fa1700b2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:23.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.538+0000 7f42eb200640 1 -- 172.21.15.39:0/1106160591 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f42c0078be0 msgr2=0x7f42c007b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:23.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.538+0000 7f42eb200640 1 --2- 172.21.15.39:0/1106160591 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f42c0078be0 0x7f42c007b0f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f42d4009890 tx=0x7f42d4004730 comp rx=0 tx=0).stop 2024-09-21T07:33:23.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.538+0000 7f42eb200640 1 -- 172.21.15.39:0/1106160591 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42e40cfb90 msgr2=0x7f42e41f4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:23.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.538+0000 7f42eb200640 1 --2- 172.21.15.39:0/1106160591 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42e40cfb90 0x7f42e41f4920 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f42dc00c730 tx=0x7f42dc00cbf0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.538+0000 7f42eb200640 1 -- 172.21.15.39:0/1106160591 shutdown_connections 2024-09-21T07:33:23.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.538+0000 7f42eb200640 1 --2- 172.21.15.39:0/1106160591 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f42c0078be0 0x7f42c007b0f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.538+0000 7f42eb200640 1 --2- 172.21.15.39:0/1106160591 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42e40cfb90 0x7f42e41f4920 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.538+0000 7f42eb200640 1 --2- 172.21.15.39:0/1106160591 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42e40cf660 0x7f42e41f43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.538+0000 7f42eb200640 1 --2- 172.21.15.39:0/1106160591 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42e4070f20 0x7f42e4159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.538+0000 7f42eb200640 1 -- 172.21.15.39:0/1106160591 >> 172.21.15.39:0/1106160591 conn(0x7f42e40b1cb0 msgr2=0x7f42e40b13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:23.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.538+0000 7fa15fe00640 1 -- 172.21.15.39:0/2934075726 shutdown_connections 2024-09-21T07:33:23.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.538+0000 7fa15fe00640 1 -- 172.21.15.39:0/2934075726 wait complete. 2024-09-21T07:33:23.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.538+0000 7f8438c00640 1 -- 172.21.15.39:0/3022467020 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8430087bd0 con 0x7f84240039b0 2024-09-21T07:33:23.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.539+0000 7f42eb200640 1 -- 172.21.15.39:0/1106160591 shutdown_connections 2024-09-21T07:33:23.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.539+0000 7f42eb200640 1 -- 172.21.15.39:0/1106160591 wait complete. 2024-09-21T07:33:23.544 INFO:teuthology.orchestra.run.smithi039.stdout:64424509447 2024-09-21T07:33:23.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.540+0000 7f6fff200640 1 -- 172.21.15.39:0/3777912717 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6fd8078bc0 msgr2=0x7f6fd807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:23.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.540+0000 7f6fff200640 1 --2- 172.21.15.39:0/3777912717 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6fd8078bc0 0x7f6fd807b0d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8004f40 tx=0x7f6fe8007ef0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.540+0000 7f6fff200640 1 -- 172.21.15.39:0/3777912717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ff80d4230 msgr2=0x7f6ff815ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:23.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.540+0000 7f6fff200640 1 --2- 172.21.15.39:0/3777912717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ff80d4230 0x7f6ff815ccf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4031000 tx=0x7f6fe4003800 comp rx=0 tx=0).stop 2024-09-21T07:33:23.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.540+0000 7f6fff200640 1 -- 172.21.15.39:0/3777912717 shutdown_connections 2024-09-21T07:33:23.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.540+0000 7f6fff200640 1 --2- 172.21.15.39:0/3777912717 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6fd8078bc0 0x7f6fd807b0d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.540+0000 7f6fff200640 1 --2- 172.21.15.39:0/3777912717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ff80d4230 0x7f6ff815ccf0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.540+0000 7f6fff200640 1 --2- 172.21.15.39:0/3777912717 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6ff80cfbc0 0x7f6ff80caae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.540+0000 7f6fff200640 1 --2- 172.21.15.39:0/3777912717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ff80b3910 0x7f6ff80734b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.540+0000 7f6fff200640 1 -- 172.21.15.39:0/3777912717 >> 172.21.15.39:0/3777912717 conn(0x7f6ff808bfc0 msgr2=0x7f6ff80b9420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:23.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.541+0000 7f6fff200640 1 -- 172.21.15.39:0/3777912717 shutdown_connections 2024-09-21T07:33:23.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.541+0000 7f6fff200640 1 -- 172.21.15.39:0/3777912717 wait complete. 2024-09-21T07:33:23.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.543+0000 7f8442200640 1 -- 172.21.15.39:0/3022467020 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f841c078bc0 msgr2=0x7f841c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:23.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.543+0000 7ff4f8800640 1 -- 172.21.15.39:0/3805520136 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7ff4f015c780 con 0x7ff4f0070440 2024-09-21T07:33:23.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.543+0000 7f8442200640 1 --2- 172.21.15.39:0/3022467020 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f841c078bc0 0x7f841c07b0d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f843c0b2280 tx=0x7f8428006cf0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.543+0000 7f8442200640 1 -- 172.21.15.39:0/3022467020 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84240039b0 msgr2=0x7f843c0cc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:23.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.543+0000 7f8442200640 1 --2- 172.21.15.39:0/3022467020 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84240039b0 0x7f843c0cc8b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8430005fd0 tx=0x7f843000c340 comp rx=0 tx=0).stop 2024-09-21T07:33:23.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.543+0000 7ff4ea000640 1 -- 172.21.15.39:0/3805520136 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff4e40635c0 con 0x7ff4f0070440 2024-09-21T07:33:23.547 INFO:teuthology.orchestra.run.smithi039.stdout:64424509447 2024-09-21T07:33:23.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.544+0000 7f8442200640 1 -- 172.21.15.39:0/3022467020 shutdown_connections 2024-09-21T07:33:23.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.544+0000 7f8442200640 1 --2- 172.21.15.39:0/3022467020 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f841c078bc0 0x7f841c07b0d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.544+0000 7f8442200640 1 --2- 172.21.15.39:0/3022467020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f843c147a10 0x7f843c141810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.544+0000 7f8442200640 1 --2- 172.21.15.39:0/3022467020 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f843c0ccdf0 0x7f843c1412d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.544+0000 7f8442200640 1 --2- 172.21.15.39:0/3022467020 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84240039b0 0x7f843c0cc8b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.545+0000 7f8442200640 1 -- 172.21.15.39:0/3022467020 >> 172.21.15.39:0/3022467020 conn(0x7f843c06d870 msgr2=0x7f843c06ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:23.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.545+0000 7f8442200640 1 -- 172.21.15.39:0/3022467020 shutdown_connections 2024-09-21T07:33:23.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.545+0000 7f8442200640 1 -- 172.21.15.39:0/3022467020 wait complete. 2024-09-21T07:33:23.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.548+0000 7ff4f8800640 1 -- 172.21.15.39:0/3805520136 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff4d4078c10 msgr2=0x7ff4d407b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:23.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.548+0000 7ff4f8800640 1 --2- 172.21.15.39:0/3805520136 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff4d4078c10 0x7ff4d407b120 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec035000 tx=0x7ff4ec004ab0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.548+0000 7ff4f8800640 1 -- 172.21.15.39:0/3805520136 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4f0070440 msgr2=0x7ff4f01f5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:23.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.548+0000 7ff4f8800640 1 --2- 172.21.15.39:0/3805520136 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4f0070440 0x7ff4f01f5560 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff4e400b3f0 tx=0x7ff4e400b8b0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.549+0000 7ff4f8800640 1 -- 172.21.15.39:0/3805520136 shutdown_connections 2024-09-21T07:33:23.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.549+0000 7ff4f8800640 1 --2- 172.21.15.39:0/3805520136 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff4d4078c10 0x7ff4d407b120 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.549+0000 7ff4f8800640 1 --2- 172.21.15.39:0/3805520136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4f0073170 0x7ff4f015aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.549+0000 7ff4f8800640 1 --2- 172.21.15.39:0/3805520136 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4f0070440 0x7ff4f01f5560 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.549+0000 7ff4f8800640 1 --2- 172.21.15.39:0/3805520136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4f0070030 0x7ff4f01e94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.549+0000 7ff4f8800640 1 -- 172.21.15.39:0/3805520136 >> 172.21.15.39:0/3805520136 conn(0x7ff4f00b1cb0 msgr2=0x7ff4f00ad9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:23.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.549+0000 7ff4f8800640 1 -- 172.21.15.39:0/3805520136 shutdown_connections 2024-09-21T07:33:23.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.549+0000 7ff4f8800640 1 -- 172.21.15.39:0/3805520136 wait complete. 2024-09-21T07:33:23.554 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542149 for osd.3 2024-09-21T07:33:23.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.552+0000 7f74d9800640 1 -- 172.21.15.39:0/749266464 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f74ac005850 con 0x7f74d40ceb40 2024-09-21T07:33:23.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.553+0000 7f74caa00640 1 -- 172.21.15.39:0/749266464 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f74c0062810 con 0x7f74d40ceb40 2024-09-21T07:33:23.557 INFO:teuthology.orchestra.run.smithi039.stdout:60129542150 2024-09-21T07:33:23.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.558+0000 7f74d9800640 1 -- 172.21.15.39:0/749266464 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f74b4078b90 msgr2=0x7f74b407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:23.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.558+0000 7f74d9800640 1 --2- 172.21.15.39:0/749266464 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f74b4078b90 0x7f74b407b0a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f74c40046b0 tx=0x7f74c4009040 comp rx=0 tx=0).stop 2024-09-21T07:33:23.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.558+0000 7f74d9800640 1 -- 172.21.15.39:0/749266464 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f74d40ceb40 msgr2=0x7f74d415fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:23.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.558+0000 7f74d9800640 1 --2- 172.21.15.39:0/749266464 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f74d40ceb40 0x7f74d415fe70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f74c000c3f0 tx=0x7f74c000c8b0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.559+0000 7f74d9800640 1 -- 172.21.15.39:0/749266464 shutdown_connections 2024-09-21T07:33:23.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.559+0000 7f74d9800640 1 --2- 172.21.15.39:0/749266464 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f74b4078b90 0x7f74b407b0a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.559+0000 7f74d9800640 1 --2- 172.21.15.39:0/749266464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74d40c85e0 0x7f74d415b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.559+0000 7f74d9800640 1 --2- 172.21.15.39:0/749266464 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f74d40ceb40 0x7f74d415fe70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.559+0000 7f74d9800640 1 --2- 172.21.15.39:0/749266464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f74d40b3540 0x7f74d40cae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:23.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.559+0000 7f74d9800640 1 -- 172.21.15.39:0/749266464 >> 172.21.15.39:0/749266464 conn(0x7f74d408bfc0 msgr2=0x7f74d406f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:23.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.559+0000 7f74d9800640 1 -- 172.21.15.39:0/749266464 shutdown_connections 2024-09-21T07:33:23.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:23.559+0000 7f74d9800640 1 -- 172.21.15.39:0/749266464 wait complete. 2024-09-21T07:33:23.564 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509446 for osd.2 2024-09-21T07:33:23.581 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509445 for osd.5 2024-09-21T07:33:23.582 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509447 for osd.0 2024-09-21T07:33:23.582 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509445 for osd.7 2024-09-21T07:33:23.586 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509447 for osd.1 2024-09-21T07:33:23.589 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509447 for osd.6 2024-09-21T07:33:23.591 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542150 for osd.4 2024-09-21T07:33:24.555 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T07:33:24.565 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T07:33:24.582 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T07:33:24.582 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T07:33:24.584 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-21T07:33:24.587 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T07:33:24.590 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-21T07:33:24.592 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T07:33:24.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:24.932+0000 7fe1eaa00640 1 -- 172.21.15.39:0/3424010970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1dc007540 msgr2=0x7fe1dc0e31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:24.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:24.932+0000 7fe1eaa00640 1 --2- 172.21.15.39:0/3424010970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1dc007540 0x7fe1dc0e31f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc007230 tx=0x7fe1e40a6650 comp rx=0 tx=0).stop 2024-09-21T07:33:24.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:24.932+0000 7fe1eaa00640 1 -- 172.21.15.39:0/3424010970 shutdown_connections 2024-09-21T07:33:24.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:24.933+0000 7fe1eaa00640 1 --2- 172.21.15.39:0/3424010970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1dc007540 0x7fe1dc0e31f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:24.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:24.933+0000 7fe1eaa00640 1 -- 172.21.15.39:0/3424010970 >> 172.21.15.39:0/3424010970 conn(0x7fe1dc03a670 msgr2=0x7fe1dc03aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:24.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:24.933+0000 7fe1eaa00640 1 -- 172.21.15.39:0/3424010970 shutdown_connections 2024-09-21T07:33:24.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:24.933+0000 7fe1eaa00640 1 -- 172.21.15.39:0/3424010970 wait complete. 2024-09-21T07:33:24.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:24.938+0000 7fe1eaa00640 1 Processor -- start 2024-09-21T07:33:24.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:24.939+0000 7fe1eaa00640 1 -- start start 2024-09-21T07:33:24.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:24.942+0000 7fe1eaa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1dc007540 0x7fe1dc0e70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:24.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:24.943+0000 7fe1eaa00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1dc053850 0x7fe1dc0e5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:24.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:24.943+0000 7fe1eaa00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1dc054180 0x7fe1dc0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:24.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:24.943+0000 7fe1eaa00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe1dc03cee0 con 0x7fe1dc007540 2024-09-21T07:33:24.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:24.943+0000 7fe1eaa00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe1dc03cd60 con 0x7fe1dc053850 2024-09-21T07:33:24.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:24.943+0000 7fe1eaa00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe1dc03d060 con 0x7fe1dc054180 2024-09-21T07:33:24.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:24.944+0000 7fe1e3400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1dc053850 0x7fe1dc0e5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:24.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:24.944+0000 7fe1e3400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1dc053850 0x7fe1dc0e5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.39:59148/0 (socket says 172.21.15.39:59148) 2024-09-21T07:33:24.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:24.944+0000 7fe1e3400640 1 -- 172.21.15.39:0/3211394759 learned_addr learned my addr 172.21.15.39:0/3211394759 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:24.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:24.944+0000 7fe1e3400640 1 -- 172.21.15.39:0/3211394759 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1dc054180 msgr2=0x7fe1dc0cfad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T07:33:24.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:24.944+0000 7fe1e3400640 1 --2- 172.21.15.39:0/3211394759 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1dc054180 0x7fe1dc0cfad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:24.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:24.944+0000 7fe1e3400640 1 -- 172.21.15.39:0/3211394759 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1dc007540 msgr2=0x7fe1dc0e70f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T07:33:24.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:24.945+0000 7fe1e3e00640 1 --2- 172.21.15.39:0/3211394759 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1dc007540 0x7fe1dc0e70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:24.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:24.945+0000 7fe1e3400640 1 --2- 172.21.15.39:0/3211394759 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1dc007540 0x7fe1dc0e70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:24.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:24.945+0000 7fe1e3400640 1 -- 172.21.15.39:0/3211394759 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1dc17a640 con 0x7fe1dc053850 2024-09-21T07:33:24.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:24.946+0000 7fe1e3400640 1 --2- 172.21.15.39:0/3211394759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1dc053850 0x7fe1dc0e5360 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe1e40759c0 tx=0x7fe1e40a6b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:24.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:24.946+0000 7fe1d7e00640 1 -- 172.21.15.39:0/3211394759 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e40bb070 con 0x7fe1dc053850 2024-09-21T07:33:24.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:24.947+0000 7fe1d7e00640 1 -- 172.21.15.39:0/3211394759 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1e40b7910 con 0x7fe1dc053850 2024-09-21T07:33:24.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:24.947+0000 7fe1d7e00640 1 -- 172.21.15.39:0/3211394759 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e40b7cc0 con 0x7fe1dc053850 2024-09-21T07:33:24.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:24.947+0000 7fe1eaa00640 1 -- 172.21.15.39:0/3211394759 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1dc17a920 con 0x7fe1dc053850 2024-09-21T07:33:24.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:24.947+0000 7fe1eaa00640 1 -- 172.21.15.39:0/3211394759 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1dc17aee0 con 0x7fe1dc053850 2024-09-21T07:33:24.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:24.951+0000 7fe1eaa00640 1 -- 172.21.15.39:0/3211394759 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1c0005290 con 0x7fe1dc053850 2024-09-21T07:33:24.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:24.953+0000 7fe1d7e00640 1 -- 172.21.15.39:0/3211394759 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe1e40ad640 con 0x7fe1dc053850 2024-09-21T07:33:24.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:24.955+0000 7fe1d7e00640 1 --2- 172.21.15.39:0/3211394759 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe1c4078b90 0x7fe1c407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:24.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:24.956+0000 7fe1d7e00640 1 -- 172.21.15.39:0/3211394759 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe1e4137ae0 con 0x7fe1dc053850 2024-09-21T07:33:24.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:24.957+0000 7fe1e3e00640 1 --2- 172.21.15.39:0/3211394759 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe1c4078b90 0x7fe1c407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:24.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:24.958+0000 7fe1e3e00640 1 --2- 172.21.15.39:0/3211394759 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe1c4078b90 0x7fe1c407b0a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0001590 tx=0x7fe1d0009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:24.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:24.959+0000 7fe1d7e00640 1 -- 172.21.15.39:0/3211394759 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe1e40ffe80 con 0x7fe1dc053850 2024-09-21T07:33:25.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.016+0000 7f9eb6a00640 1 -- 172.21.15.39:0/637541711 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb0031bb0 con 0x7f9eb80d2ed0 2024-09-21T07:33:25.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.016+0000 7efe1f200640 1 -- 172.21.15.39:0/2204788066 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe04004800 msgr2=0x7efe04004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:25.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.016+0000 7efe1f200640 1 --2- 172.21.15.39:0/2204788066 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe04004800 0x7efe04004c10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efe1000c7a0 tx=0x7efe04005830 comp rx=0 tx=0).stop 2024-09-21T07:33:25.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.017+0000 7f9ebec00640 1 -- 172.21.15.39:0/637541711 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9eb80d2ed0 msgr2=0x7f9eb80d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:25.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.017+0000 7f9ebec00640 1 --2- 172.21.15.39:0/637541711 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9eb80d2ed0 0x7f9eb80d1a80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0009820 tx=0x7f9eb002a510 comp rx=0 tx=0).stop 2024-09-21T07:33:25.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.017+0000 7f9ebec00640 1 -- 172.21.15.39:0/637541711 shutdown_connections 2024-09-21T07:33:25.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.017+0000 7f9ebec00640 1 --2- 172.21.15.39:0/637541711 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9eb80d2ed0 0x7f9eb80d1a80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.017+0000 7f9ebec00640 1 -- 172.21.15.39:0/637541711 >> 172.21.15.39:0/637541711 conn(0x7f9eb808bfc0 msgr2=0x7f9eb809d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:25.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.017+0000 7f9ebec00640 1 -- 172.21.15.39:0/637541711 shutdown_connections 2024-09-21T07:33:25.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.017+0000 7efe1f200640 1 -- 172.21.15.39:0/2204788066 shutdown_connections 2024-09-21T07:33:25.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.017+0000 7efe1f200640 1 --2- 172.21.15.39:0/2204788066 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe04004800 0x7efe04004c10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.017+0000 7efe1f200640 1 -- 172.21.15.39:0/2204788066 >> 172.21.15.39:0/2204788066 conn(0x7efe1809d6a0 msgr2=0x7efe180d43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:25.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.017+0000 7efe1f200640 1 -- 172.21.15.39:0/2204788066 shutdown_connections 2024-09-21T07:33:25.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.018+0000 7f9ebec00640 1 -- 172.21.15.39:0/637541711 wait complete. 2024-09-21T07:33:25.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.018+0000 7efe1f200640 1 -- 172.21.15.39:0/2204788066 wait complete. 2024-09-21T07:33:25.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.019+0000 7f39d8c00640 1 -- 172.21.15.39:0/3378548472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39b8005150 msgr2=0x7f39b800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:25.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.019+0000 7f39d8c00640 1 --2- 172.21.15.39:0/3378548472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39b8005150 0x7f39b800b9c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f39d406a350 tx=0x7f39b8005830 comp rx=0 tx=0).stop 2024-09-21T07:33:25.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.019+0000 7f39d2800640 1 -- 172.21.15.39:0/3378548472 reap_dead start 2024-09-21T07:33:25.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.020+0000 7f39d8c00640 1 -- 172.21.15.39:0/3378548472 shutdown_connections 2024-09-21T07:33:25.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.020+0000 7f39d8c00640 1 -- 172.21.15.39:0/3378548472 >> 172.21.15.39:0/3378548472 conn(0x7f39cc051410 msgr2=0x7f39cc051850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:25.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.020+0000 7efc56a00640 1 -- 172.21.15.39:0/2591852943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc500c9950 msgr2=0x7efc50144b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:25.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.020+0000 7efc56a00640 1 --2- 172.21.15.39:0/2591852943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc500c9950 0x7efc50144b50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efc40005fd0 tx=0x7efc4002a670 comp rx=0 tx=0).stop 2024-09-21T07:33:25.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.020+0000 7f39d8c00640 1 -- 172.21.15.39:0/3378548472 shutdown_connections 2024-09-21T07:33:25.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.021+0000 7efc56a00640 1 -- 172.21.15.39:0/2591852943 shutdown_connections 2024-09-21T07:33:25.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.021+0000 7efe1f200640 1 Processor -- start 2024-09-21T07:33:25.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.021+0000 7efc56a00640 1 --2- 172.21.15.39:0/2591852943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc500c9950 0x7efc50144b50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.021+0000 7efe1f200640 1 -- start start 2024-09-21T07:33:25.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.021+0000 7efc56a00640 1 -- 172.21.15.39:0/2591852943 >> 172.21.15.39:0/2591852943 conn(0x7efc5006d4a0 msgr2=0x7efc5006ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:25.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.022+0000 7f9ebec00640 1 Processor -- start 2024-09-21T07:33:25.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.021+0000 7f39d8c00640 1 -- 172.21.15.39:0/3378548472 wait complete. 2024-09-21T07:33:25.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.022+0000 7efc56a00640 1 -- 172.21.15.39:0/2591852943 shutdown_connections 2024-09-21T07:33:25.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.023+0000 7efc56a00640 1 -- 172.21.15.39:0/2591852943 wait complete. 2024-09-21T07:33:25.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.023+0000 7f9ebec00640 1 -- start start 2024-09-21T07:33:25.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.026+0000 7f9871e00640 1 -- 172.21.15.39:0/1403589106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f986c073160 msgr2=0x7f986c0cfcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:25.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.026+0000 7f9ebec00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9eb80b9b70 0x7f9eb8155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:25.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.026+0000 7f9ebec00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9eb80cf7a0 0x7f9eb8160710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:25.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.026+0000 7f9ebec00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9eb80cb350 0x7f9eb8160c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:25.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.026+0000 7f9ebec00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb8162f70 con 0x7f9eb80b9b70 2024-09-21T07:33:25.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.026+0000 7f9ebec00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb8162df0 con 0x7f9eb80cf7a0 2024-09-21T07:33:25.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.026+0000 7f9ebec00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f9eb81630f0 con 0x7f9eb80cb350 2024-09-21T07:33:25.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.026+0000 7f9871e00640 1 --2- 172.21.15.39:0/1403589106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f986c073160 0x7f986c0cfcb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9858005fd0 tx=0x7f985802a150 comp rx=0 tx=0).stop 2024-09-21T07:33:25.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.026+0000 7f9871e00640 1 -- 172.21.15.39:0/1403589106 shutdown_connections 2024-09-21T07:33:25.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.026+0000 7efe1f200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe04004800 0x7efe180ca120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:25.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.026+0000 7f9871e00640 1 --2- 172.21.15.39:0/1403589106 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f986c0d01f0 0x7f986c0ca710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.027+0000 7efe1f200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe181f89e0 0x7efe181f8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:25.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.026+0000 7f9871e00640 1 --2- 172.21.15.39:0/1403589106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f986c073160 0x7f986c0cfcb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.026+0000 7f9871e00640 1 -- 172.21.15.39:0/1403589106 >> 172.21.15.39:0/1403589106 conn(0x7f986c06d870 msgr2=0x7f986c06ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:25.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.027+0000 7efe1f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe180a1780 0x7efe180a1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:25.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.026+0000 7efc56a00640 1 Processor -- start 2024-09-21T07:33:25.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.027+0000 7efc56a00640 1 -- start start 2024-09-21T07:33:25.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.027+0000 7efe1f200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efe180b3b50 con 0x7efe180a1780 2024-09-21T07:33:25.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.027+0000 7f9871e00640 1 -- 172.21.15.39:0/1403589106 shutdown_connections 2024-09-21T07:33:25.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.027+0000 7ff187200640 1 -- 172.21.15.39:0/1346962117 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff16c004800 msgr2=0x7ff16c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:25.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.027+0000 7ff187200640 1 --2- 172.21.15.39:0/1346962117 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff16c004800 0x7ff16c004c10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff174009b50 tx=0x7ff16c005830 comp rx=0 tx=0).stop 2024-09-21T07:33:25.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.027+0000 7efe1f200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efe18162ba0 con 0x7efe181f89e0 2024-09-21T07:33:25.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.027+0000 7f39d8c00640 1 Processor -- start 2024-09-21T07:33:25.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.027+0000 7efe1f200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7efe180b3cd0 con 0x7efe04004800 2024-09-21T07:33:25.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.027+0000 7efe1e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe180a1780 0x7efe180a1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:25.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.027+0000 7efe1de00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe04004800 0x7efe180ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:25.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.027+0000 7efe1e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe180a1780 0x7efe180a1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60192/0 (socket says 172.21.15.39:60192) 2024-09-21T07:33:25.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.027+0000 7efe1e800640 1 -- 172.21.15.39:0/3013391307 learned_addr learned my addr 172.21.15.39:0/3013391307 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:25.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.027+0000 7efe1d400640 1 --2- 172.21.15.39:0/3013391307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe181f89e0 0x7efe181f8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:25.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.028+0000 7ff187200640 1 -- 172.21.15.39:0/1346962117 shutdown_connections 2024-09-21T07:33:25.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.028+0000 7ff187200640 1 --2- 172.21.15.39:0/1346962117 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff16c004800 0x7ff16c004c10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.028+0000 7ff187200640 1 -- 172.21.15.39:0/1346962117 >> 172.21.15.39:0/1346962117 conn(0x7ff18806d410 msgr2=0x7ff1880b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:25.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.027+0000 7f9eb7400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9eb80cf7a0 0x7f9eb8160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:25.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.027+0000 7f9eb7400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9eb80cf7a0 0x7f9eb8160710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.39:59252/0 (socket says 172.21.15.39:59252) 2024-09-21T07:33:25.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.027+0000 7f9eb7400640 1 -- 172.21.15.39:0/3096063931 learned_addr learned my addr 172.21.15.39:0/3096063931 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:25.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.028+0000 7f9eb7e00640 1 --2- 172.21.15.39:0/3096063931 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9eb80b9b70 0x7f9eb8155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:25.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.028+0000 7f39d8c00640 1 -- start start 2024-09-21T07:33:25.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.028+0000 7f9871e00640 1 -- 172.21.15.39:0/1403589106 wait complete. 2024-09-21T07:33:25.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.028+0000 7f9ebcc00640 1 --2- 172.21.15.39:0/3096063931 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9eb80cb350 0x7f9eb8160c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:25.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.028+0000 7efe1e800640 1 -- 172.21.15.39:0/3013391307 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe04004800 msgr2=0x7efe180ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:25.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.028+0000 7efe1e800640 1 --2- 172.21.15.39:0/3013391307 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe04004800 0x7efe180ca120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.028+0000 7efe1e800640 1 -- 172.21.15.39:0/3013391307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe181f89e0 msgr2=0x7efe181f8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:25.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.028+0000 7efe1e800640 1 --2- 172.21.15.39:0/3013391307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe181f89e0 0x7efe181f8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.029+0000 7efe1e800640 1 -- 172.21.15.39:0/3013391307 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe180a20d0 con 0x7efe180a1780 2024-09-21T07:33:25.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.029+0000 7f9eb7400640 1 -- 172.21.15.39:0/3096063931 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9eb80cb350 msgr2=0x7f9eb8160c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:25.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.029+0000 7ff187200640 1 -- 172.21.15.39:0/1346962117 shutdown_connections 2024-09-21T07:33:25.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.029+0000 7efe1de00640 1 --2- 172.21.15.39:0/3013391307 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe04004800 0x7efe180ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:33:25.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.029+0000 7efe1e800640 1 --2- 172.21.15.39:0/3013391307 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe180a1780 0x7efe180a1b90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efe180d3fb0 tx=0x7efe08010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:25.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.029+0000 7efc56a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc500c9950 0x7efc5013e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:25.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.029+0000 7efc56a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc500ccec0 0x7efc50159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:25.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.029+0000 7efc56a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc500ce230 0x7efc500b4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:25.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.029+0000 7efc56a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efc50162b60 con 0x7efc500ccec0 2024-09-21T07:33:25.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.029+0000 7efc56a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efc501629e0 con 0x7efc500ce230 2024-09-21T07:33:25.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.029+0000 7efc56a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7efc50162ce0 con 0x7efc500c9950 2024-09-21T07:33:25.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.029+0000 7f9eb7400640 1 --2- 172.21.15.39:0/3096063931 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9eb80cb350 0x7f9eb8160c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.029+0000 7f9eb7400640 1 -- 172.21.15.39:0/3096063931 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9eb80b9b70 msgr2=0x7f9eb8155cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:25.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.029+0000 7f9eb7400640 1 --2- 172.21.15.39:0/3096063931 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9eb80b9b70 0x7f9eb8155cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.029+0000 7f9eb7400640 1 -- 172.21.15.39:0/3096063931 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb80c8340 con 0x7f9eb80cf7a0 2024-09-21T07:33:25.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.029+0000 7ff187200640 1 -- 172.21.15.39:0/1346962117 wait complete. 2024-09-21T07:33:25.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.030+0000 7f9eb7e00640 1 --2- 172.21.15.39:0/3096063931 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9eb80b9b70 0x7f9eb8155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:33:25.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.030+0000 7f9ebcc00640 1 --2- 172.21.15.39:0/3096063931 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9eb80cb350 0x7f9eb8160c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:33:25.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.030+0000 7efe0e000640 1 -- 172.21.15.39:0/3013391307 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe08004440 con 0x7efe180a1780 2024-09-21T07:33:25.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.030+0000 7efc4fe00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc500c9950 0x7efc5013e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:25.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.030+0000 7efc4f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc500ccec0 0x7efc50159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:25.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.030+0000 7efc4f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc500ccec0 0x7efc50159390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60206/0 (socket says 172.21.15.39:60206) 2024-09-21T07:33:25.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.030+0000 7efc4f400640 1 -- 172.21.15.39:0/2266108071 learned_addr learned my addr 172.21.15.39:0/2266108071 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:25.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.030+0000 7f4ca7200640 1 -- 172.21.15.39:0/2083849538 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4c88003a00 msgr2=0x7f4c88003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:25.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.030+0000 7efe1f200640 1 -- 172.21.15.39:0/3013391307 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe1809f9e0 con 0x7efe180a1780 2024-09-21T07:33:25.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.030+0000 7efe1f200640 1 -- 172.21.15.39:0/3013391307 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe1809ffa0 con 0x7efe180a1780 2024-09-21T07:33:25.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.030+0000 7f4ca7200640 1 --2- 172.21.15.39:0/2083849538 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4c88003a00 0x7f4c88003e30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4c9400c310 tx=0x7f4c88005830 comp rx=0 tx=0).stop 2024-09-21T07:33:25.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.030+0000 7efe0e000640 1 -- 172.21.15.39:0/3013391307 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe0801a070 con 0x7efe180a1780 2024-09-21T07:33:25.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.031+0000 7f4ca7200640 1 -- 172.21.15.39:0/2083849538 shutdown_connections 2024-09-21T07:33:25.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.031+0000 7f4ca7200640 1 --2- 172.21.15.39:0/2083849538 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4c88003a00 0x7f4c88003e30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.031+0000 7f4ca7200640 1 -- 172.21.15.39:0/2083849538 >> 172.21.15.39:0/2083849538 conn(0x7f4ca006d410 msgr2=0x7f4ca00b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:25.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.030+0000 7efc54a00640 1 --2- 172.21.15.39:0/2266108071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc500ce230 0x7efc500b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:25.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.031+0000 7efc4f400640 1 -- 172.21.15.39:0/2266108071 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc500c9950 msgr2=0x7efc5013e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:25.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.031+0000 7efc4f400640 1 --2- 172.21.15.39:0/2266108071 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc500c9950 0x7efc5013e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.031+0000 7efc4f400640 1 -- 172.21.15.39:0/2266108071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc500ce230 msgr2=0x7efc500b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:25.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.031+0000 7f4ca7200640 1 -- 172.21.15.39:0/2083849538 shutdown_connections 2024-09-21T07:33:25.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.030+0000 7f9eb7400640 1 --2- 172.21.15.39:0/3096063931 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9eb80cf7a0 0x7f9eb8160710 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9ea800ba80 tx=0x7f9ea800bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:25.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.031+0000 7f9eafe00640 1 -- 172.21.15.39:0/3096063931 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea800cba0 con 0x7f9eb80cf7a0 2024-09-21T07:33:25.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.031+0000 7f9eafe00640 1 -- 172.21.15.39:0/3096063931 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ea800ce50 con 0x7f9eb80cf7a0 2024-09-21T07:33:25.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.031+0000 7f9eafe00640 1 -- 172.21.15.39:0/3096063931 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8003bd0 con 0x7f9eb80cf7a0 2024-09-21T07:33:25.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.031+0000 7f9ebec00640 1 -- 172.21.15.39:0/3096063931 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eb80c8620 con 0x7f9eb80cf7a0 2024-09-21T07:33:25.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.031+0000 7f9ebec00640 1 -- 172.21.15.39:0/3096063931 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eb80c8900 con 0x7f9eb80cf7a0 2024-09-21T07:33:25.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.032+0000 7efe0e000640 1 -- 172.21.15.39:0/3013391307 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe08004440 con 0x7efe180a1780 2024-09-21T07:33:25.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.032+0000 7f9871e00640 1 Processor -- start 2024-09-21T07:33:25.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.032+0000 7efc4f400640 1 --2- 172.21.15.39:0/2266108071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc500ce230 0x7efc500b4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.032+0000 7efc4f400640 1 -- 172.21.15.39:0/2266108071 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc501fae00 con 0x7efc500ccec0 2024-09-21T07:33:25.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.032+0000 7efc4fe00640 1 --2- 172.21.15.39:0/2266108071 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc500c9950 0x7efc5013e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:33:25.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.033+0000 7efe1f200640 1 -- 172.21.15.39:0/3013391307 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdf8005290 con 0x7efe180a1780 2024-09-21T07:33:25.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.033+0000 7efc4f400640 1 --2- 172.21.15.39:0/2266108071 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc500ccec0 0x7efc50159390 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00c400 tx=0x7efc3c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:25.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.033+0000 7efc54a00640 1 --2- 172.21.15.39:0/2266108071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc500ce230 0x7efc500b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:33:25.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.033+0000 7f39d8c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39b8005150 0x7f39cc0204c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:25.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.031+0000 7f4ca7200640 1 -- 172.21.15.39:0/2083849538 wait complete. 2024-09-21T07:33:25.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.033+0000 7f39d8c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39cc04e1c0 0x7f39cc01f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:25.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.033+0000 7f39d8c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39cc04e5d0 0x7f39cc178260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:25.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.034+0000 7f39d8c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f39cc009ff0 con 0x7f39cc04e1c0 2024-09-21T07:33:25.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.034+0000 7f39d8c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f39cc009e70 con 0x7f39b8005150 2024-09-21T07:33:25.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.034+0000 7f39d8c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f39cc007080 con 0x7f39cc04e5d0 2024-09-21T07:33:25.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.034+0000 7f39d1e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39b8005150 0x7f39cc0204c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:25.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.034+0000 7f39d2800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39cc04e5d0 0x7f39cc178260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:25.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.034+0000 7ff187200640 1 Processor -- start 2024-09-21T07:33:25.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.034+0000 7ff187200640 1 -- start start 2024-09-21T07:33:25.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.034+0000 7f39d1e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39b8005150 0x7f39cc0204c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.39:59280/0 (socket says 172.21.15.39:59280) 2024-09-21T07:33:25.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.034+0000 7f39d2800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39cc04e5d0 0x7f39cc178260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.39:41070/0 (socket says 172.21.15.39:41070) 2024-09-21T07:33:25.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.034+0000 7f39d1e00640 1 -- 172.21.15.39:0/3682238474 learned_addr learned my addr 172.21.15.39:0/3682238474 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:25.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.034+0000 7f39d1400640 1 --2- 172.21.15.39:0/3682238474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39cc04e1c0 0x7f39cc01f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:25.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.034+0000 7f39d1e00640 1 -- 172.21.15.39:0/3682238474 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39cc04e5d0 msgr2=0x7f39cc178260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:25.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.034+0000 7f39d1e00640 1 --2- 172.21.15.39:0/3682238474 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39cc04e5d0 0x7f39cc178260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.034+0000 7f39d1e00640 1 -- 172.21.15.39:0/3682238474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39cc04e1c0 msgr2=0x7f39cc01f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:25.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.034+0000 7f39d1e00640 1 --2- 172.21.15.39:0/3682238474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39cc04e1c0 0x7f39cc01f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.034+0000 7f39d1e00640 1 -- 172.21.15.39:0/3682238474 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39cc01e580 con 0x7f39b8005150 2024-09-21T07:33:25.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.032+0000 7f9871e00640 1 -- start start 2024-09-21T07:33:25.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.035+0000 7f9871e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f986c073160 0x7f986c155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:25.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.035+0000 7f9871e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f986c0d01f0 0x7f986c1f48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:25.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.035+0000 7f9871e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f986c15e210 0x7f986c15bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:25.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.035+0000 7f4ca7200640 1 Processor -- start 2024-09-21T07:33:25.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.035+0000 7efc47e00640 1 -- 172.21.15.39:0/2266108071 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c017070 con 0x7efc500ccec0 2024-09-21T07:33:25.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.035+0000 7efc47e00640 1 -- 172.21.15.39:0/2266108071 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc3c0053c0 con 0x7efc500ccec0 2024-09-21T07:33:25.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.035+0000 7efc47e00640 1 -- 172.21.15.39:0/2266108071 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c003850 con 0x7efc500ccec0 2024-09-21T07:33:25.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.035+0000 7f4ca7200640 1 -- start start 2024-09-21T07:33:25.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.036+0000 7efc56a00640 1 -- 172.21.15.39:0/2266108071 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc5015edf0 con 0x7efc500ccec0 2024-09-21T07:33:25.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.036+0000 7efc56a00640 1 -- 172.21.15.39:0/2266108071 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc5015f3b0 con 0x7efc500ccec0 2024-09-21T07:33:25.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.035+0000 7f9870a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f986c073160 0x7f986c155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:25.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.035+0000 7f9870a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f986c073160 0x7f986c155aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.39:41084/0 (socket says 172.21.15.39:41084) 2024-09-21T07:33:25.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.035+0000 7f9870a00640 1 -- 172.21.15.39:0/1960573816 learned_addr learned my addr 172.21.15.39:0/1960573816 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:25.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.036+0000 7f986be00640 1 --2- 172.21.15.39:0/1960573816 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f986c0d01f0 0x7f986c1f48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:25.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.036+0000 7f9871e00640 1 -- 172.21.15.39:0/1960573816 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f986c0cac50 con 0x7f986c0d01f0 2024-09-21T07:33:25.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.035+0000 7f9ebec00640 1 -- 172.21.15.39:0/3096063931 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e90005290 con 0x7f9eb80cf7a0 2024-09-21T07:33:25.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.036+0000 7f9eafe00640 1 -- 172.21.15.39:0/3096063931 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9ea8014440 con 0x7f9eb80cf7a0 2024-09-21T07:33:25.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.037+0000 7efe0e000640 1 -- 172.21.15.39:0/3013391307 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7efe0800e940 con 0x7efe180a1780 2024-09-21T07:33:25.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.036+0000 7f9871e00640 1 -- 172.21.15.39:0/1960573816 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f986c070fa0 con 0x7f986c15e210 2024-09-21T07:33:25.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.036+0000 7f9871e00640 1 -- 172.21.15.39:0/1960573816 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f986c0cadd0 con 0x7f986c073160 2024-09-21T07:33:25.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.037+0000 7ff187200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff16c004800 0x7ff188142140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:25.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.038+0000 7ff187200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff188157290 0x7ff18815e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:25.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.038+0000 7f9eafe00640 1 --2- 172.21.15.39:0/3096063931 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9e98078bc0 0x7f9e9807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:25.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.038+0000 7f9871400640 1 --2- 172.21.15.39:0/1960573816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f986c15e210 0x7f986c15bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:25.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.038+0000 7ff187200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff188142880 0x7ff18815e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:25.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.038+0000 7ff187200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff1880cce30 con 0x7ff16c004800 2024-09-21T07:33:25.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.038+0000 7ff187200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff1880cccb0 con 0x7ff188157290 2024-09-21T07:33:25.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.038+0000 7ff187200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7ff18806ee70 con 0x7ff188142880 2024-09-21T07:33:25.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.038+0000 7f39d1e00640 1 --2- 172.21.15.39:0/3682238474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39b8005150 0x7f39cc0204c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f39c4009840 tx=0x7f39c4009d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:25.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.038+0000 7ff185400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff188157290 0x7ff18815e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:25.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.038+0000 7ff185400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff188157290 0x7ff18815e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.39:59298/0 (socket says 172.21.15.39:59298) 2024-09-21T07:33:25.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.038+0000 7f4ca7200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4c88003a00 0x7f4ca0146cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:25.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.038+0000 7ff185400640 1 -- 172.21.15.39:0/4056764972 learned_addr learned my addr 172.21.15.39:0/4056764972 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:25.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.038+0000 7f9eafe00640 1 -- 172.21.15.39:0/3096063931 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9ea809ab30 con 0x7f9eb80cf7a0 2024-09-21T07:33:25.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.038+0000 7f4ca7200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ca00cf970 0x7f4ca014c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:25.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.038+0000 7f4ca7200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ca01499b0 0x7f4ca014caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:25.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.038+0000 7f4ca7200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca01517e0 con 0x7f4ca01499b0 2024-09-21T07:33:25.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.038+0000 7f4ca7200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca0151660 con 0x7f4ca00cf970 2024-09-21T07:33:25.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.038+0000 7f4ca7200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f4ca0151960 con 0x7f4c88003a00 2024-09-21T07:33:25.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.039+0000 7efe0e000640 1 --2- 172.21.15.39:0/3013391307 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efdfc078f90 0x7efdfc07b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:25.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.038+0000 7ff185e00640 1 --2- 172.21.15.39:0/4056764972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff16c004800 0x7ff188142140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:25.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.039+0000 7ff186800640 1 --2- 172.21.15.39:0/4056764972 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff188142880 0x7ff18815e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:25.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.039+0000 7efe0e000640 1 -- 172.21.15.39:0/3013391307 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7efe08004240 con 0x7efe180a1780 2024-09-21T07:33:25.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.039+0000 7efc45600640 1 -- 172.21.15.39:0/2266108071 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc50144b50 con 0x7efc500ccec0 2024-09-21T07:33:25.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.039+0000 7f9eb7e00640 1 --2- 172.21.15.39:0/3096063931 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9e98078bc0 0x7f9e9807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:25.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.039+0000 7f9870a00640 1 -- 172.21.15.39:0/1960573816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f986c15e210 msgr2=0x7f986c15bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:25.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.039+0000 7f9870a00640 1 --2- 172.21.15.39:0/1960573816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f986c15e210 0x7f986c15bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.039+0000 7f9870a00640 1 -- 172.21.15.39:0/1960573816 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f986c0d01f0 msgr2=0x7f986c1f48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:25.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.039+0000 7f9870a00640 1 --2- 172.21.15.39:0/1960573816 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f986c0d01f0 0x7f986c1f48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.039+0000 7f9870a00640 1 -- 172.21.15.39:0/1960573816 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f986c1f55b0 con 0x7f986c073160 2024-09-21T07:33:25.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.039+0000 7f4ca5e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4c88003a00 0x7f4ca0146cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:25.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.039+0000 7f4ca5e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4c88003a00 0x7f4ca0146cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.39:41102/0 (socket says 172.21.15.39:41102) 2024-09-21T07:33:25.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.039+0000 7f4ca5e00640 1 -- 172.21.15.39:0/2742817405 learned_addr learned my addr 172.21.15.39:0/2742817405 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:25.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.039+0000 7f4ca6800640 1 --2- 172.21.15.39:0/2742817405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ca01499b0 0x7f4ca014caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:25.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.039+0000 7ff185400640 1 -- 172.21.15.39:0/4056764972 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff188142880 msgr2=0x7ff18815e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:25.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.039+0000 7ff185400640 1 --2- 172.21.15.39:0/4056764972 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff188142880 0x7ff18815e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.039+0000 7ff185400640 1 -- 172.21.15.39:0/4056764972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff16c004800 msgr2=0x7ff188142140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:25.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.039+0000 7ff185400640 1 --2- 172.21.15.39:0/4056764972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff16c004800 0x7ff188142140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.039+0000 7ff185400640 1 -- 172.21.15.39:0/4056764972 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff188142680 con 0x7ff188157290 2024-09-21T07:33:25.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.039+0000 7ff185e00640 1 --2- 172.21.15.39:0/4056764972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff16c004800 0x7ff188142140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:33:25.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.039+0000 7f4ca5400640 1 --2- 172.21.15.39:0/2742817405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ca00cf970 0x7f4ca014c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:25.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.040+0000 7f9870a00640 1 --2- 172.21.15.39:0/1960573816 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f986c073160 0x7f986c155aa0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f985c00c730 tx=0x7f985c00cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:25.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.040+0000 7f4ca5e00640 1 -- 172.21.15.39:0/2742817405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ca00cf970 msgr2=0x7f4ca014c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:25.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.040+0000 7f4ca5e00640 1 --2- 172.21.15.39:0/2742817405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ca00cf970 0x7f4ca014c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.040+0000 7f4ca5e00640 1 -- 172.21.15.39:0/2742817405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ca01499b0 msgr2=0x7f4ca014caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:25.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.040+0000 7f4ca5e00640 1 --2- 172.21.15.39:0/2742817405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ca01499b0 0x7f4ca014caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.040+0000 7f4ca5e00640 1 -- 172.21.15.39:0/2742817405 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca0147210 con 0x7f4c88003a00 2024-09-21T07:33:25.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.040+0000 7ff185400640 1 --2- 172.21.15.39:0/4056764972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff188157290 0x7ff18815e0a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff174009a70 tx=0x7ff174033670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:25.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.040+0000 7f9868c00640 1 -- 172.21.15.39:0/1960573816 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c0052d0 con 0x7f986c073160 2024-09-21T07:33:25.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.040+0000 7f9871e00640 1 -- 172.21.15.39:0/1960573816 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f986c1f14c0 con 0x7f986c073160 2024-09-21T07:33:25.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.040+0000 7f9868c00640 1 -- 172.21.15.39:0/1960573816 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f985c005470 con 0x7f986c073160 2024-09-21T07:33:25.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.040+0000 7f9871e00640 1 -- 172.21.15.39:0/1960573816 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f986c1f1a80 con 0x7f986c073160 2024-09-21T07:33:25.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.040+0000 7f9868c00640 1 -- 172.21.15.39:0/1960573816 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c0052d0 con 0x7f986c073160 2024-09-21T07:33:25.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.040+0000 7f4ca5e00640 1 --2- 172.21.15.39:0/2742817405 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4c88003a00 0x7f4ca0146cd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4c9400c310 tx=0x7f4c94004be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:25.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.040+0000 7efe1de00640 1 --2- 172.21.15.39:0/3013391307 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efdfc078f90 0x7efdfc07b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:25.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.041+0000 7f4c9a000640 1 -- 172.21.15.39:0/2742817405 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c94006c70 con 0x7f4c88003a00 2024-09-21T07:33:25.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.041+0000 7f4ca7200640 1 -- 172.21.15.39:0/2742817405 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ca014b070 con 0x7f4c88003a00 2024-09-21T07:33:25.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.041+0000 7f4ca7200640 1 -- 172.21.15.39:0/2742817405 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ca014b630 con 0x7f4c88003a00 2024-09-21T07:33:25.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.041+0000 7f4c9a000640 1 -- 172.21.15.39:0/2742817405 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c94006420 con 0x7f4c88003a00 2024-09-21T07:33:25.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.041+0000 7f4c9a000640 1 -- 172.21.15.39:0/2742817405 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c94006c70 con 0x7f4c88003a00 2024-09-21T07:33:25.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.041+0000 7efe1de00640 1 --2- 172.21.15.39:0/3013391307 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efdfc078f90 0x7efdfc07b4a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efe140058e0 tx=0x7efe14005f30 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:25.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.041+0000 7f9eb7e00640 1 --2- 172.21.15.39:0/3096063931 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9e98078bc0 0x7f9e9807b0d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0034000 tx=0x7f9eb0004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:25.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.042+0000 7f9871e00640 1 -- 172.21.15.39:0/1960573816 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f986c0cfcb0 con 0x7f986c073160 2024-09-21T07:33:25.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.042+0000 7f39ca000640 1 -- 172.21.15.39:0/3682238474 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c401c020 con 0x7f39b8005150 2024-09-21T07:33:25.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.042+0000 7f39ca000640 1 -- 172.21.15.39:0/3682238474 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39c40079b0 con 0x7f39b8005150 2024-09-21T07:33:25.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.043+0000 7f39d8c00640 1 -- 172.21.15.39:0/3682238474 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39cc01e860 con 0x7f39b8005150 2024-09-21T07:33:25.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.043+0000 7f39d8c00640 1 -- 172.21.15.39:0/3682238474 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39cc01ee20 con 0x7f39b8005150 2024-09-21T07:33:25.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.043+0000 7f39ca000640 1 -- 172.21.15.39:0/3682238474 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c4007dc0 con 0x7f39b8005150 2024-09-21T07:33:25.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.043+0000 7f4ca7200640 1 -- 172.21.15.39:0/2742817405 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ca014ae60 con 0x7f4c88003a00 2024-09-21T07:33:25.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.044+0000 7f9eafe00640 1 -- 172.21.15.39:0/3096063931 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9ea80628c0 con 0x7f9eb80cf7a0 2024-09-21T07:33:25.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.044+0000 7ff17e000640 1 -- 172.21.15.39:0/4056764972 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff174038070 con 0x7ff188157290 2024-09-21T07:33:25.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.044+0000 7ff17e000640 1 -- 172.21.15.39:0/4056764972 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff174042070 con 0x7ff188157290 2024-09-21T07:33:25.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.044+0000 7ff17e000640 1 -- 172.21.15.39:0/4056764972 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17403d440 con 0x7ff188157290 2024-09-21T07:33:25.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.045+0000 7efe0e000640 1 -- 172.21.15.39:0/3013391307 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7efe08063450 con 0x7efe180a1780 2024-09-21T07:33:25.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.046+0000 7f9868c00640 1 -- 172.21.15.39:0/1960573816 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f985c00a640 con 0x7f986c073160 2024-09-21T07:33:25.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.046+0000 7f39d8c00640 1 -- 172.21.15.39:0/3682238474 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39cc00a7b0 con 0x7f39b8005150 2024-09-21T07:33:25.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.045+0000 7f4c9a000640 1 -- 172.21.15.39:0/2742817405 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4c94001430 con 0x7f4c88003a00 2024-09-21T07:33:25.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.046+0000 7f4c9a000640 1 --2- 172.21.15.39:0/2742817405 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4c84078d90 0x7f4c8407b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:25.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.047+0000 7f4c9a000640 1 -- 172.21.15.39:0/2742817405 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4c940ba690 con 0x7f4c88003a00 2024-09-21T07:33:25.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.047+0000 7f4ca5400640 1 --2- 172.21.15.39:0/2742817405 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4c84078d90 0x7f4c8407b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:25.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.045+0000 7ff187200640 1 -- 172.21.15.39:0/4056764972 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff188157830 con 0x7ff188157290 2024-09-21T07:33:25.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.046+0000 7ff187200640 1 -- 172.21.15.39:0/4056764972 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff188159fb0 con 0x7ff188157290 2024-09-21T07:33:25.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.047+0000 7f9868c00640 1 --2- 172.21.15.39:0/1960573816 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f984c078fc0 0x7f984c07b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:25.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.048+0000 7f9868c00640 1 -- 172.21.15.39:0/1960573816 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f985c0050d0 con 0x7f986c073160 2024-09-21T07:33:25.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.048+0000 7f986be00640 1 --2- 172.21.15.39:0/1960573816 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f984c078fc0 0x7f984c07b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:25.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.048+0000 7f4ca5400640 1 --2- 172.21.15.39:0/2742817405 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4c84078d90 0x7f4c8407b2a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4ca014d190 tx=0x7f4c90006b10 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:25.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.049+0000 7f9868c00640 1 -- 172.21.15.39:0/1960573816 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f986c0d0600 con 0x7f986c073160 2024-09-21T07:33:25.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.047+0000 7f39ca000640 1 -- 172.21.15.39:0/3682238474 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f39c4004830 con 0x7f39b8005150 2024-09-21T07:33:25.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.049+0000 7f39ca000640 1 --2- 172.21.15.39:0/3682238474 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f39b4078be0 0x7f39b407b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:25.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.049+0000 7f986be00640 1 --2- 172.21.15.39:0/1960573816 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f984c078fc0 0x7f984c07b4d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9858032000 tx=0x7f9858004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:25.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.049+0000 7ff17e000640 1 -- 172.21.15.39:0/4056764972 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff17403d5e0 con 0x7ff188157290 2024-09-21T07:33:25.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.050+0000 7f39ca000640 1 -- 172.21.15.39:0/3682238474 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f39c409d090 con 0x7f39b8005150 2024-09-21T07:33:25.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.050+0000 7f39d1400640 1 --2- 172.21.15.39:0/3682238474 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f39b4078be0 0x7f39b407b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:25.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.051+0000 7ff17e000640 1 --2- 172.21.15.39:0/4056764972 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff168078be0 0x7ff16807b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:25.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.052+0000 7ff17e000640 1 -- 172.21.15.39:0/4056764972 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff17407f550 con 0x7ff188157290 2024-09-21T07:33:25.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.052+0000 7ff185e00640 1 --2- 172.21.15.39:0/4056764972 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff168078be0 0x7ff16807b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:25.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.052+0000 7f39d1400640 1 --2- 172.21.15.39:0/3682238474 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f39b4078be0 0x7f39b407b0f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f39cc1721a0 tx=0x7f39c00067d0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:25.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.052+0000 7efc47e00640 1 -- 172.21.15.39:0/2266108071 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7efc3c01a070 con 0x7efc500ccec0 2024-09-21T07:33:25.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.052+0000 7ff187200640 1 -- 172.21.15.39:0/4056764972 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff18806f380 con 0x7ff188157290 2024-09-21T07:33:25.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.053+0000 7ff185e00640 1 --2- 172.21.15.39:0/4056764972 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff168078be0 0x7ff16807b0f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff18806a350 tx=0x7ff17800b5c0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:25.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.054+0000 7f4c9a000640 1 -- 172.21.15.39:0/2742817405 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4c94030030 con 0x7f4c88003a00 2024-09-21T07:33:25.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.054+0000 7efc47e00640 1 --2- 172.21.15.39:0/2266108071 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efc30078c60 0x7efc3007b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:25.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.054+0000 7efc47e00640 1 -- 172.21.15.39:0/2266108071 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7efc3c09a5c0 con 0x7efc500ccec0 2024-09-21T07:33:25.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.055+0000 7efc4fe00640 1 --2- 172.21.15.39:0/2266108071 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efc30078c60 0x7efc3007b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:25.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.055+0000 7efc47e00640 1 -- 172.21.15.39:0/2266108071 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7efc3c021020 con 0x7efc500ccec0 2024-09-21T07:33:25.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.056+0000 7efc4fe00640 1 --2- 172.21.15.39:0/2266108071 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efc30078c60 0x7efc3007b170 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efc40033000 tx=0x7efc40004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:25.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.058+0000 7f39ca000640 1 -- 172.21.15.39:0/3682238474 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f39c4026020 con 0x7f39b8005150 2024-09-21T07:33:25.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.062+0000 7ff17e000640 1 -- 172.21.15.39:0/4056764972 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff17404b020 con 0x7ff188157290 2024-09-21T07:33:25.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.417+0000 7fe1eaa00640 1 -- 172.21.15.39:0/3211394759 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fe1c0005580 con 0x7fe1dc053850 2024-09-21T07:33:25.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.418+0000 7fe1d7e00640 1 -- 172.21.15.39:0/3211394759 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe1e40ff780 con 0x7fe1dc053850 2024-09-21T07:33:25.421 INFO:teuthology.orchestra.run.smithi039.stdout:60129542151 2024-09-21T07:33:25.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.422+0000 7fe1eaa00640 1 -- 172.21.15.39:0/3211394759 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe1c4078b90 msgr2=0x7fe1c407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:25.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.422+0000 7fe1eaa00640 1 --2- 172.21.15.39:0/3211394759 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe1c4078b90 0x7fe1c407b0a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0001590 tx=0x7fe1d0009040 comp rx=0 tx=0).stop 2024-09-21T07:33:25.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.423+0000 7fe1eaa00640 1 -- 172.21.15.39:0/3211394759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1dc053850 msgr2=0x7fe1dc0e5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:25.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.423+0000 7fe1eaa00640 1 --2- 172.21.15.39:0/3211394759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1dc053850 0x7fe1dc0e5360 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe1e40759c0 tx=0x7fe1e40a6b50 comp rx=0 tx=0).stop 2024-09-21T07:33:25.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.423+0000 7fe1eaa00640 1 -- 172.21.15.39:0/3211394759 shutdown_connections 2024-09-21T07:33:25.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.423+0000 7fe1eaa00640 1 --2- 172.21.15.39:0/3211394759 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe1c4078b90 0x7fe1c407b0a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.423+0000 7fe1eaa00640 1 --2- 172.21.15.39:0/3211394759 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1dc054180 0x7fe1dc0cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.424+0000 7fe1eaa00640 1 --2- 172.21.15.39:0/3211394759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1dc053850 0x7fe1dc0e5360 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.424+0000 7fe1eaa00640 1 --2- 172.21.15.39:0/3211394759 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1dc007540 0x7fe1dc0e70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.424+0000 7fe1eaa00640 1 -- 172.21.15.39:0/3211394759 >> 172.21.15.39:0/3211394759 conn(0x7fe1dc03a670 msgr2=0x7fe1dc004a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:25.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.424+0000 7fe1eaa00640 1 -- 172.21.15.39:0/3211394759 shutdown_connections 2024-09-21T07:33:25.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.424+0000 7fe1eaa00640 1 -- 172.21.15.39:0/3211394759 wait complete. 2024-09-21T07:33:25.464 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.3 2024-09-21T07:33:25.465 DEBUG:teuthology.parallel:result is None 2024-09-21T07:33:25.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.478+0000 7f39d8c00640 1 -- 172.21.15.39:0/3682238474 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f39cc01de00 con 0x7f39b8005150 2024-09-21T07:33:25.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.479+0000 7f39ca000640 1 -- 172.21.15.39:0/3682238474 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f39c4066f20 con 0x7f39b8005150 2024-09-21T07:33:25.481 INFO:teuthology.orchestra.run.smithi039.stdout:64424509449 2024-09-21T07:33:25.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.481+0000 7f39d8c00640 1 -- 172.21.15.39:0/3682238474 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f39b4078be0 msgr2=0x7f39b407b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:25.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.481+0000 7f39d8c00640 1 --2- 172.21.15.39:0/3682238474 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f39b4078be0 0x7f39b407b0f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f39cc1721a0 tx=0x7f39c00067d0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.481+0000 7f39d8c00640 1 -- 172.21.15.39:0/3682238474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39b8005150 msgr2=0x7f39cc0204c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:25.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.481+0000 7f39d8c00640 1 --2- 172.21.15.39:0/3682238474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39b8005150 0x7f39cc0204c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f39c4009840 tx=0x7f39c4009d00 comp rx=0 tx=0).stop 2024-09-21T07:33:25.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.482+0000 7f39d8c00640 1 -- 172.21.15.39:0/3682238474 shutdown_connections 2024-09-21T07:33:25.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.482+0000 7f39d8c00640 1 --2- 172.21.15.39:0/3682238474 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f39b4078be0 0x7f39b407b0f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.482+0000 7f39d8c00640 1 --2- 172.21.15.39:0/3682238474 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39cc04e5d0 0x7f39cc178260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.482+0000 7f39d8c00640 1 --2- 172.21.15.39:0/3682238474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39cc04e1c0 0x7f39cc01f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.482+0000 7f39d8c00640 1 --2- 172.21.15.39:0/3682238474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39b8005150 0x7f39cc0204c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.482+0000 7f39d8c00640 1 -- 172.21.15.39:0/3682238474 >> 172.21.15.39:0/3682238474 conn(0x7f39cc051410 msgr2=0x7f39cc006910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:25.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.482+0000 7f39d8c00640 1 -- 172.21.15.39:0/3682238474 shutdown_connections 2024-09-21T07:33:25.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.482+0000 7f39d8c00640 1 -- 172.21.15.39:0/3682238474 wait complete. 2024-09-21T07:33:25.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.499+0000 7f9ebec00640 1 -- 172.21.15.39:0/3096063931 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f9e90005580 con 0x7f9eb80cf7a0 2024-09-21T07:33:25.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.500+0000 7f9eafe00640 1 -- 172.21.15.39:0/3096063931 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9ea80672f0 con 0x7f9eb80cf7a0 2024-09-21T07:33:25.502 INFO:teuthology.orchestra.run.smithi039.stdout:64424509448 2024-09-21T07:33:25.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.502+0000 7f9871e00640 1 -- 172.21.15.39:0/1960573816 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f986c1f1280 con 0x7f986c073160 2024-09-21T07:33:25.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.502+0000 7f9868c00640 1 -- 172.21.15.39:0/1960573816 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f985c062a40 con 0x7f986c073160 2024-09-21T07:33:25.504 INFO:teuthology.orchestra.run.smithi039.stdout:64424509449 2024-09-21T07:33:25.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.504+0000 7f9ebec00640 1 -- 172.21.15.39:0/3096063931 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9e98078bc0 msgr2=0x7f9e9807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:25.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.504+0000 7f9ebec00640 1 --2- 172.21.15.39:0/3096063931 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9e98078bc0 0x7f9e9807b0d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0034000 tx=0x7f9eb0004730 comp rx=0 tx=0).stop 2024-09-21T07:33:25.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.504+0000 7f9ebec00640 1 -- 172.21.15.39:0/3096063931 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9eb80cf7a0 msgr2=0x7f9eb8160710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:25.507 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.1 2024-09-21T07:33:25.507 DEBUG:teuthology.parallel:result is None 2024-09-21T07:33:25.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.504+0000 7f9ebec00640 1 --2- 172.21.15.39:0/3096063931 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9eb80cf7a0 0x7f9eb8160710 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9ea800ba80 tx=0x7f9ea800bf40 comp rx=0 tx=0).stop 2024-09-21T07:33:25.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.505+0000 7f9ebec00640 1 -- 172.21.15.39:0/3096063931 shutdown_connections 2024-09-21T07:33:25.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.505+0000 7f9ebec00640 1 --2- 172.21.15.39:0/3096063931 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9e98078bc0 0x7f9e9807b0d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.505+0000 7f9ebec00640 1 --2- 172.21.15.39:0/3096063931 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9eb80cb350 0x7f9eb8160c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.505+0000 7f9ebec00640 1 --2- 172.21.15.39:0/3096063931 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9eb80cf7a0 0x7f9eb8160710 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.505+0000 7f9ebec00640 1 --2- 172.21.15.39:0/3096063931 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9eb80b9b70 0x7f9eb8155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.505+0000 7f9ebec00640 1 -- 172.21.15.39:0/3096063931 >> 172.21.15.39:0/3096063931 conn(0x7f9eb808bfc0 msgr2=0x7f9eb80d3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:25.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.505+0000 7f9ebec00640 1 -- 172.21.15.39:0/3096063931 shutdown_connections 2024-09-21T07:33:25.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.505+0000 7f9ebec00640 1 -- 172.21.15.39:0/3096063931 wait complete. 2024-09-21T07:33:25.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.507+0000 7f9871e00640 1 -- 172.21.15.39:0/1960573816 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f984c078fc0 msgr2=0x7f984c07b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:25.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.507+0000 7f9871e00640 1 --2- 172.21.15.39:0/1960573816 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f984c078fc0 0x7f984c07b4d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9858032000 tx=0x7f9858004730 comp rx=0 tx=0).stop 2024-09-21T07:33:25.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.507+0000 7f9871e00640 1 -- 172.21.15.39:0/1960573816 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f986c073160 msgr2=0x7f986c155aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:25.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.507+0000 7f9871e00640 1 --2- 172.21.15.39:0/1960573816 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f986c073160 0x7f986c155aa0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f985c00c730 tx=0x7f985c00cbf0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.507+0000 7f9871e00640 1 -- 172.21.15.39:0/1960573816 shutdown_connections 2024-09-21T07:33:25.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.507+0000 7f9871e00640 1 --2- 172.21.15.39:0/1960573816 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f984c078fc0 0x7f984c07b4d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.507+0000 7f9871e00640 1 --2- 172.21.15.39:0/1960573816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f986c15e210 0x7f986c15bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.507+0000 7f9871e00640 1 --2- 172.21.15.39:0/1960573816 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f986c0d01f0 0x7f986c1f48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.507+0000 7f9871e00640 1 --2- 172.21.15.39:0/1960573816 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f986c073160 0x7f986c155aa0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.507+0000 7f9871e00640 1 -- 172.21.15.39:0/1960573816 >> 172.21.15.39:0/1960573816 conn(0x7f986c06d870 msgr2=0x7f986c0b1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:25.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.507+0000 7f9871e00640 1 -- 172.21.15.39:0/1960573816 shutdown_connections 2024-09-21T07:33:25.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.507+0000 7f9871e00640 1 -- 172.21.15.39:0/1960573816 wait complete. 2024-09-21T07:33:25.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.508+0000 7efe1f200640 1 -- 172.21.15.39:0/3013391307 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7efdf8005850 con 0x7efe180a1780 2024-09-21T07:33:25.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.509+0000 7efe0e000640 1 -- 172.21.15.39:0/3013391307 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efe08067e80 con 0x7efe180a1780 2024-09-21T07:33:25.511 INFO:teuthology.orchestra.run.smithi039.stdout:64424509447 2024-09-21T07:33:25.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.513+0000 7efe1f200640 1 -- 172.21.15.39:0/3013391307 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efdfc078f90 msgr2=0x7efdfc07b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:25.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.513+0000 7efe1f200640 1 --2- 172.21.15.39:0/3013391307 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efdfc078f90 0x7efdfc07b4a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efe140058e0 tx=0x7efe14005f30 comp rx=0 tx=0).stop 2024-09-21T07:33:25.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.513+0000 7efe1f200640 1 -- 172.21.15.39:0/3013391307 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe180a1780 msgr2=0x7efe180a1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:25.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.513+0000 7efe1f200640 1 --2- 172.21.15.39:0/3013391307 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe180a1780 0x7efe180a1b90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efe180d3fb0 tx=0x7efe08010da0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.514+0000 7efe1f200640 1 -- 172.21.15.39:0/3013391307 shutdown_connections 2024-09-21T07:33:25.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.514+0000 7efe1f200640 1 --2- 172.21.15.39:0/3013391307 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efdfc078f90 0x7efdfc07b4a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.514+0000 7efe1f200640 1 --2- 172.21.15.39:0/3013391307 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe180a1780 0x7efe180a1b90 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.514+0000 7efe1f200640 1 --2- 172.21.15.39:0/3013391307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe181f89e0 0x7efe181f8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.514+0000 7efe1f200640 1 --2- 172.21.15.39:0/3013391307 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe04004800 0x7efe180ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.514+0000 7efe1f200640 1 -- 172.21.15.39:0/3013391307 >> 172.21.15.39:0/3013391307 conn(0x7efe1809d6a0 msgr2=0x7efe1806db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:25.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.514+0000 7f4ca7200640 1 -- 172.21.15.39:0/2742817405 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f4ca006ef50 con 0x7f4c88003a00 2024-09-21T07:33:25.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.514+0000 7efe1f200640 1 -- 172.21.15.39:0/3013391307 shutdown_connections 2024-09-21T07:33:25.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.514+0000 7efe1f200640 1 -- 172.21.15.39:0/3013391307 wait complete. 2024-09-21T07:33:25.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.514+0000 7f4c9a000640 1 -- 172.21.15.39:0/2742817405 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4c94082a30 con 0x7f4c88003a00 2024-09-21T07:33:25.517 INFO:teuthology.orchestra.run.smithi039.stdout:64424509449 2024-09-21T07:33:25.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.515+0000 7ff187200640 1 -- 172.21.15.39:0/4056764972 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7ff18806fb50 con 0x7ff188157290 2024-09-21T07:33:25.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.515+0000 7ff17e000640 1 -- 172.21.15.39:0/4056764972 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff188142680 con 0x7ff188157290 2024-09-21T07:33:25.518 INFO:teuthology.orchestra.run.smithi039.stdout:60129542152 2024-09-21T07:33:25.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.516+0000 7efc45600640 1 -- 172.21.15.39:0/2266108071 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7efc5013e7b0 con 0x7efc500ccec0 2024-09-21T07:33:25.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.516+0000 7efc47e00640 1 -- 172.21.15.39:0/2266108071 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efc3c0629e0 con 0x7efc500ccec0 2024-09-21T07:33:25.520 INFO:teuthology.orchestra.run.smithi039.stdout:64424509447 2024-09-21T07:33:25.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.518+0000 7f4ca7200640 1 -- 172.21.15.39:0/2742817405 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4c84078d90 msgr2=0x7f4c8407b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:25.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.518+0000 7f4ca7200640 1 --2- 172.21.15.39:0/2742817405 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4c84078d90 0x7f4c8407b2a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4ca014d190 tx=0x7f4c90006b10 comp rx=0 tx=0).stop 2024-09-21T07:33:25.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.518+0000 7f4ca7200640 1 -- 172.21.15.39:0/2742817405 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4c88003a00 msgr2=0x7f4ca0146cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:25.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.518+0000 7f4ca7200640 1 --2- 172.21.15.39:0/2742817405 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4c88003a00 0x7f4ca0146cd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4c9400c310 tx=0x7f4c94004be0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.519+0000 7f4ca7200640 1 -- 172.21.15.39:0/2742817405 shutdown_connections 2024-09-21T07:33:25.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.519+0000 7f4ca7200640 1 --2- 172.21.15.39:0/2742817405 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4c84078d90 0x7f4c8407b2a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.519+0000 7f4ca7200640 1 --2- 172.21.15.39:0/2742817405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ca01499b0 0x7f4ca014caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.519+0000 7f4ca7200640 1 --2- 172.21.15.39:0/2742817405 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ca00cf970 0x7f4ca014c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.519+0000 7f4ca7200640 1 --2- 172.21.15.39:0/2742817405 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4c88003a00 0x7f4ca0146cd0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.519+0000 7f4ca7200640 1 -- 172.21.15.39:0/2742817405 >> 172.21.15.39:0/2742817405 conn(0x7f4ca006d410 msgr2=0x7f4ca015e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:25.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.519+0000 7f4ca7200640 1 -- 172.21.15.39:0/2742817405 shutdown_connections 2024-09-21T07:33:25.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.519+0000 7f4ca7200640 1 -- 172.21.15.39:0/2742817405 wait complete. 2024-09-21T07:33:25.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.519+0000 7ff187200640 1 -- 172.21.15.39:0/4056764972 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff168078be0 msgr2=0x7ff16807b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:25.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.519+0000 7ff187200640 1 --2- 172.21.15.39:0/4056764972 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff168078be0 0x7ff16807b0f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff18806a350 tx=0x7ff17800b5c0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.519+0000 7ff187200640 1 -- 172.21.15.39:0/4056764972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff188157290 msgr2=0x7ff18815e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:25.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.519+0000 7ff187200640 1 --2- 172.21.15.39:0/4056764972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff188157290 0x7ff18815e0a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff174009a70 tx=0x7ff174033670 comp rx=0 tx=0).stop 2024-09-21T07:33:25.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.520+0000 7ff187200640 1 -- 172.21.15.39:0/4056764972 shutdown_connections 2024-09-21T07:33:25.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.520+0000 7ff187200640 1 --2- 172.21.15.39:0/4056764972 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff168078be0 0x7ff16807b0f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.520+0000 7ff187200640 1 --2- 172.21.15.39:0/4056764972 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff188142880 0x7ff18815e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.520+0000 7ff187200640 1 --2- 172.21.15.39:0/4056764972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff188157290 0x7ff18815e0a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.520+0000 7ff187200640 1 --2- 172.21.15.39:0/4056764972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff16c004800 0x7ff188142140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.520+0000 7ff187200640 1 -- 172.21.15.39:0/4056764972 >> 172.21.15.39:0/4056764972 conn(0x7ff18806d410 msgr2=0x7ff1880d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:25.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.520+0000 7ff187200640 1 -- 172.21.15.39:0/4056764972 shutdown_connections 2024-09-21T07:33:25.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.520+0000 7ff187200640 1 -- 172.21.15.39:0/4056764972 wait complete. 2024-09-21T07:33:25.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.522+0000 7efc45600640 1 -- 172.21.15.39:0/2266108071 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efc30078c60 msgr2=0x7efc3007b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:25.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.522+0000 7efc45600640 1 --2- 172.21.15.39:0/2266108071 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efc30078c60 0x7efc3007b170 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efc40033000 tx=0x7efc40004730 comp rx=0 tx=0).stop 2024-09-21T07:33:25.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.522+0000 7efc45600640 1 -- 172.21.15.39:0/2266108071 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc500ccec0 msgr2=0x7efc50159390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:25.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.522+0000 7efc45600640 1 --2- 172.21.15.39:0/2266108071 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc500ccec0 0x7efc50159390 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00c400 tx=0x7efc3c00c8c0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.523+0000 7efc45600640 1 -- 172.21.15.39:0/2266108071 shutdown_connections 2024-09-21T07:33:25.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.523+0000 7efc45600640 1 --2- 172.21.15.39:0/2266108071 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efc30078c60 0x7efc3007b170 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.523+0000 7efc45600640 1 --2- 172.21.15.39:0/2266108071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc500ce230 0x7efc500b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.523+0000 7efc45600640 1 --2- 172.21.15.39:0/2266108071 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc500ccec0 0x7efc50159390 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.523+0000 7efc45600640 1 --2- 172.21.15.39:0/2266108071 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc500c9950 0x7efc5013e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.523+0000 7efc45600640 1 -- 172.21.15.39:0/2266108071 >> 172.21.15.39:0/2266108071 conn(0x7efc5006d4a0 msgr2=0x7efc500747c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:25.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.523+0000 7efc45600640 1 -- 172.21.15.39:0/2266108071 shutdown_connections 2024-09-21T07:33:25.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.523+0000 7efc45600640 1 -- 172.21.15.39:0/2266108071 wait complete. 2024-09-21T07:33:25.541 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.2 2024-09-21T07:33:25.541 DEBUG:teuthology.parallel:result is None 2024-09-21T07:33:25.544 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.0 2024-09-21T07:33:25.544 DEBUG:teuthology.parallel:result is None 2024-09-21T07:33:25.549 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.6 2024-09-21T07:33:25.549 DEBUG:teuthology.parallel:result is None 2024-09-21T07:33:25.552 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.4 2024-09-21T07:33:25.552 DEBUG:teuthology.parallel:result is None 2024-09-21T07:33:25.555 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.5 2024-09-21T07:33:25.555 DEBUG:teuthology.parallel:result is None 2024-09-21T07:33:25.560 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.7 2024-09-21T07:33:25.560 DEBUG:teuthology.parallel:result is None 2024-09-21T07:33:25.560 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T07:33:25.560 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:33:25.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.823+0000 7fc05ca00640 1 -- 172.21.15.39:0/1264328185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0580cc1e0 msgr2=0x7fc0580af3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:25.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.823+0000 7fc05ca00640 1 --2- 172.21.15.39:0/1264328185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0580cc1e0 0x7fc0580af3c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc048005fd0 tx=0x7fc04802a5d0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.823+0000 7fc05ca00640 1 -- 172.21.15.39:0/1264328185 shutdown_connections 2024-09-21T07:33:25.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.823+0000 7fc05ca00640 1 --2- 172.21.15.39:0/1264328185 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc05815d8a0 0x7fc0580b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.823+0000 7fc05ca00640 1 --2- 172.21.15.39:0/1264328185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0580cc1e0 0x7fc0580af3c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.823+0000 7fc05ca00640 1 -- 172.21.15.39:0/1264328185 >> 172.21.15.39:0/1264328185 conn(0x7fc05806d7d0 msgr2=0x7fc0580b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:25.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.823+0000 7fc05ca00640 1 -- 172.21.15.39:0/1264328185 shutdown_connections 2024-09-21T07:33:25.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.823+0000 7fc05ca00640 1 -- 172.21.15.39:0/1264328185 wait complete. 2024-09-21T07:33:25.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.826+0000 7fc05ca00640 1 Processor -- start 2024-09-21T07:33:25.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.826+0000 7fc05ca00640 1 -- start start 2024-09-21T07:33:25.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.827+0000 7fc05ca00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0580cc1e0 0x7fc058149710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:25.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.827+0000 7fc05ca00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0580ccb50 0x7fc05815c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:25.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.827+0000 7fc05ca00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc05815d8a0 0x7fc0581e9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:25.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.827+0000 7fc05ca00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc058153520 con 0x7fc0580cc1e0 2024-09-21T07:33:25.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.827+0000 7fc05ca00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc0581533a0 con 0x7fc0580ccb50 2024-09-21T07:33:25.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.827+0000 7fc05ca00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc0581536a0 con 0x7fc05815d8a0 2024-09-21T07:33:25.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.828+0000 7fc055e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0580cc1e0 0x7fc058149710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:25.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.828+0000 7fc055e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0580cc1e0 0x7fc058149710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60260/0 (socket says 172.21.15.39:60260) 2024-09-21T07:33:25.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.828+0000 7fc055e00640 1 -- 172.21.15.39:0/2166192680 learned_addr learned my addr 172.21.15.39:0/2166192680 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:25.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.828+0000 7fc056800640 1 --2- 172.21.15.39:0/2166192680 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc05815d8a0 0x7fc0581e9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:25.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.828+0000 7fc04fe00640 1 --2- 172.21.15.39:0/2166192680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0580ccb50 0x7fc05815c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:25.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.828+0000 7fc055e00640 1 -- 172.21.15.39:0/2166192680 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc05815d8a0 msgr2=0x7fc0581e9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:25.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.828+0000 7fc055e00640 1 --2- 172.21.15.39:0/2166192680 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc05815d8a0 0x7fc0581e9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.828+0000 7fc055e00640 1 -- 172.21.15.39:0/2166192680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0580ccb50 msgr2=0x7fc05815c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:25.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.828+0000 7fc055e00640 1 --2- 172.21.15.39:0/2166192680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0580ccb50 0x7fc05815c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:25.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.828+0000 7fc055e00640 1 -- 172.21.15.39:0/2166192680 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc05809abd0 con 0x7fc0580cc1e0 2024-09-21T07:33:25.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.828+0000 7fc04fe00640 1 --2- 172.21.15.39:0/2166192680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0580ccb50 0x7fc05815c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:33:25.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.828+0000 7fc055e00640 1 --2- 172.21.15.39:0/2166192680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0580cc1e0 0x7fc058149710 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc048034040 tx=0x7fc0480038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:25.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.829+0000 7fc04e000640 1 -- 172.21.15.39:0/2166192680 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04803e030 con 0x7fc0580cc1e0 2024-09-21T07:33:25.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.829+0000 7fc04e000640 1 -- 172.21.15.39:0/2166192680 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc048003ca0 con 0x7fc0580cc1e0 2024-09-21T07:33:25.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.829+0000 7fc04e000640 1 -- 172.21.15.39:0/2166192680 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc048035040 con 0x7fc0580cc1e0 2024-09-21T07:33:25.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.829+0000 7fc05ca00640 1 -- 172.21.15.39:0/2166192680 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc05809aeb0 con 0x7fc0580cc1e0 2024-09-21T07:33:25.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.829+0000 7fc05ca00640 1 -- 172.21.15.39:0/2166192680 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0580732e0 con 0x7fc0580cc1e0 2024-09-21T07:33:25.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.830+0000 7fc05ca00640 1 -- 172.21.15.39:0/2166192680 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc058153b50 con 0x7fc0580cc1e0 2024-09-21T07:33:25.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.831+0000 7fc04e000640 1 -- 172.21.15.39:0/2166192680 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc048041030 con 0x7fc0580cc1e0 2024-09-21T07:33:25.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.832+0000 7fc04e000640 1 --2- 172.21.15.39:0/2166192680 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc034078b90 0x7fc03407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:25.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.832+0000 7fc04e000640 1 -- 172.21.15.39:0/2166192680 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc0480bb430 con 0x7fc0580cc1e0 2024-09-21T07:33:25.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.832+0000 7fc04fe00640 1 --2- 172.21.15.39:0/2166192680 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc034078b90 0x7fc03407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:25.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.833+0000 7fc04fe00640 1 --2- 172.21.15.39:0/2166192680 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc034078b90 0x7fc03407b0a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc040005fd0 tx=0x7fc040005ea0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:25.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:25.837+0000 7fc04e000640 1 -- 172.21.15.39:0/2166192680 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc048083180 con 0x7fc0580cc1e0 2024-09-21T07:33:26.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.031+0000 7fc05ca00640 1 -- 172.21.15.39:0/2166192680 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc058073580 con 0x7fc034078b90 2024-09-21T07:33:26.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.036+0000 7fc04e000640 1 -- 172.21.15.39:0/2166192680 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251826 (secure 0 0 0) 0x7fc058073580 con 0x7fc034078b90 2024-09-21T07:33:26.041 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:33:26.042 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:33:26.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.040+0000 7fc05ca00640 1 -- 172.21.15.39:0/2166192680 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc034078b90 msgr2=0x7fc03407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:26.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.040+0000 7fc05ca00640 1 --2- 172.21.15.39:0/2166192680 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc034078b90 0x7fc03407b0a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc040005fd0 tx=0x7fc040005ea0 comp rx=0 tx=0).stop 2024-09-21T07:33:26.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.040+0000 7fc05ca00640 1 -- 172.21.15.39:0/2166192680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0580cc1e0 msgr2=0x7fc058149710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:26.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.040+0000 7fc05ca00640 1 --2- 172.21.15.39:0/2166192680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0580cc1e0 0x7fc058149710 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc048034040 tx=0x7fc0480038a0 comp rx=0 tx=0).stop 2024-09-21T07:33:26.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.040+0000 7fc05ca00640 1 -- 172.21.15.39:0/2166192680 shutdown_connections 2024-09-21T07:33:26.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.040+0000 7fc05ca00640 1 --2- 172.21.15.39:0/2166192680 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc034078b90 0x7fc03407b0a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:26.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.040+0000 7fc05ca00640 1 --2- 172.21.15.39:0/2166192680 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc05815d8a0 0x7fc0581e9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:26.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.040+0000 7fc05ca00640 1 --2- 172.21.15.39:0/2166192680 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0580ccb50 0x7fc05815c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:26.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.040+0000 7fc05ca00640 1 --2- 172.21.15.39:0/2166192680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0580cc1e0 0x7fc058149710 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:26.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.040+0000 7fc05ca00640 1 -- 172.21.15.39:0/2166192680 >> 172.21.15.39:0/2166192680 conn(0x7fc05806d7d0 msgr2=0x7fc0580b1450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:26.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.040+0000 7fc05ca00640 1 -- 172.21.15.39:0/2166192680 shutdown_connections 2024-09-21T07:33:26.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.040+0000 7fc05ca00640 1 -- 172.21.15.39:0/2166192680 wait complete. 2024-09-21T07:33:26.067 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-21T07:33:25.535563+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235740,"kb_used_data":2856,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738980,"statfs":{"total":773094113280,"available":772852715520,"internally_reserved":0,"allocated":2924544,"data_stored":1763052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002845"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274756+0000","last_change":"2024-09-21T07:33:15.176955+0000","last_active":"2024-09-21T07:33:16.274756+0000","last_peered":"2024-09-21T07:33:16.274756+0000","last_clean":"2024-09-21T07:33:16.274756+0000","last_became_active":"2024-09-21T07:33:15.176632+0000","last_became_peered":"2024-09-21T07:33:15.176632+0000","last_unstale":"2024-09-21T07:33:16.274756+0000","last_undegraded":"2024-09-21T07:33:16.274756+0000","last_fullsized":"2024-09-21T07:33:16.274756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:10:51.829331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171764+0000","last_change":"2024-09-21T07:33:15.184382+0000","last_active":"2024-09-21T07:33:16.171764+0000","last_peered":"2024-09-21T07:33:16.171764+0000","last_clean":"2024-09-21T07:33:16.171764+0000","last_became_active":"2024-09-21T07:33:15.184072+0000","last_became_peered":"2024-09-21T07:33:15.184072+0000","last_unstale":"2024-09-21T07:33:16.171764+0000","last_undegraded":"2024-09-21T07:33:16.171764+0000","last_fullsized":"2024-09-21T07:33:16.171764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:08.677357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273864+0000","last_change":"2024-09-21T07:33:15.186660+0000","last_active":"2024-09-21T07:33:16.273864+0000","last_peered":"2024-09-21T07:33:16.273864+0000","last_clean":"2024-09-21T07:33:16.273864+0000","last_became_active":"2024-09-21T07:33:15.185832+0000","last_became_peered":"2024-09-21T07:33:15.185832+0000","last_unstale":"2024-09-21T07:33:16.273864+0000","last_undegraded":"2024-09-21T07:33:16.273864+0000","last_fullsized":"2024-09-21T07:33:16.273864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:08.677357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.170390+0000","last_change":"2024-09-21T07:33:15.193591+0000","last_active":"2024-09-21T07:33:16.170390+0000","last_peered":"2024-09-21T07:33:16.170390+0000","last_clean":"2024-09-21T07:33:16.170390+0000","last_became_active":"2024-09-21T07:33:15.191327+0000","last_became_peered":"2024-09-21T07:33:15.191327+0000","last_unstale":"2024-09-21T07:33:16.170390+0000","last_undegraded":"2024-09-21T07:33:16.170390+0000","last_fullsized":"2024-09-21T07:33:16.170390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:00:15.919300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272836+0000","last_change":"2024-09-21T07:33:15.187754+0000","last_active":"2024-09-21T07:33:16.272836+0000","last_peered":"2024-09-21T07:33:16.272836+0000","last_clean":"2024-09-21T07:33:16.272836+0000","last_became_active":"2024-09-21T07:33:15.187381+0000","last_became_peered":"2024-09-21T07:33:15.187381+0000","last_unstale":"2024-09-21T07:33:16.272836+0000","last_undegraded":"2024-09-21T07:33:16.272836+0000","last_fullsized":"2024-09-21T07:33:16.272836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:19:59.491735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.169414+0000","last_change":"2024-09-21T07:33:15.180176+0000","last_active":"2024-09-21T07:33:16.169414+0000","last_peered":"2024-09-21T07:33:16.169414+0000","last_clean":"2024-09-21T07:33:16.169414+0000","last_became_active":"2024-09-21T07:33:15.179828+0000","last_became_peered":"2024-09-21T07:33:15.179828+0000","last_unstale":"2024-09-21T07:33:16.169414+0000","last_undegraded":"2024-09-21T07:33:16.169414+0000","last_fullsized":"2024-09-21T07:33:16.169414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:40:52.015841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274813+0000","last_change":"2024-09-21T07:33:15.183358+0000","last_active":"2024-09-21T07:33:16.274813+0000","last_peered":"2024-09-21T07:33:16.274813+0000","last_clean":"2024-09-21T07:33:16.274813+0000","last_became_active":"2024-09-21T07:33:15.182969+0000","last_became_peered":"2024-09-21T07:33:15.182969+0000","last_unstale":"2024-09-21T07:33:16.274813+0000","last_undegraded":"2024-09-21T07:33:16.274813+0000","last_fullsized":"2024-09-21T07:33:16.274813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:41:44.441220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272174+0000","last_change":"2024-09-21T07:33:15.173445+0000","last_active":"2024-09-21T07:33:16.272174+0000","last_peered":"2024-09-21T07:33:16.272174+0000","last_clean":"2024-09-21T07:33:16.272174+0000","last_became_active":"2024-09-21T07:33:15.172833+0000","last_became_peered":"2024-09-21T07:33:15.172833+0000","last_unstale":"2024-09-21T07:33:16.272174+0000","last_undegraded":"2024-09-21T07:33:16.272174+0000","last_fullsized":"2024-09-21T07:33:16.272174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:24:46.139397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274983+0000","last_change":"2024-09-21T07:33:15.194854+0000","last_active":"2024-09-21T07:33:16.274983+0000","last_peered":"2024-09-21T07:33:16.274983+0000","last_clean":"2024-09-21T07:33:16.274983+0000","last_became_active":"2024-09-21T07:33:15.194423+0000","last_became_peered":"2024-09-21T07:33:15.194423+0000","last_unstale":"2024-09-21T07:33:16.274983+0000","last_undegraded":"2024-09-21T07:33:16.274983+0000","last_fullsized":"2024-09-21T07:33:16.274983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:14.196020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274097+0000","last_change":"2024-09-21T07:33:15.176651+0000","last_active":"2024-09-21T07:33:16.274097+0000","last_peered":"2024-09-21T07:33:16.274097+0000","last_clean":"2024-09-21T07:33:16.274097+0000","last_became_active":"2024-09-21T07:33:15.176222+0000","last_became_peered":"2024-09-21T07:33:15.176222+0000","last_unstale":"2024-09-21T07:33:16.274097+0000","last_undegraded":"2024-09-21T07:33:16.274097+0000","last_fullsized":"2024-09-21T07:33:16.274097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:47.494973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274249+0000","last_change":"2024-09-21T07:33:15.170250+0000","last_active":"2024-09-21T07:33:16.274249+0000","last_peered":"2024-09-21T07:33:16.274249+0000","last_clean":"2024-09-21T07:33:16.274249+0000","last_became_active":"2024-09-21T07:33:15.166192+0000","last_became_peered":"2024-09-21T07:33:15.166192+0000","last_unstale":"2024-09-21T07:33:16.274249+0000","last_undegraded":"2024-09-21T07:33:16.274249+0000","last_fullsized":"2024-09-21T07:33:16.274249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:48:05.009446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274541+0000","last_change":"2024-09-21T07:33:15.180001+0000","last_active":"2024-09-21T07:33:16.274541+0000","last_peered":"2024-09-21T07:33:16.274541+0000","last_clean":"2024-09-21T07:33:16.274541+0000","last_became_active":"2024-09-21T07:33:15.179679+0000","last_became_peered":"2024-09-21T07:33:15.179679+0000","last_unstale":"2024-09-21T07:33:16.274541+0000","last_undegraded":"2024-09-21T07:33:16.274541+0000","last_fullsized":"2024-09-21T07:33:16.274541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:05:31.417897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272616+0000","last_change":"2024-09-21T07:33:15.179831+0000","last_active":"2024-09-21T07:33:16.272616+0000","last_peered":"2024-09-21T07:33:16.272616+0000","last_clean":"2024-09-21T07:33:16.272616+0000","last_became_active":"2024-09-21T07:33:15.179382+0000","last_became_peered":"2024-09-21T07:33:15.179382+0000","last_unstale":"2024-09-21T07:33:16.272616+0000","last_undegraded":"2024-09-21T07:33:16.272616+0000","last_fullsized":"2024-09-21T07:33:16.272616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:26:20.407200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272116+0000","last_change":"2024-09-21T07:33:15.174462+0000","last_active":"2024-09-21T07:33:16.272116+0000","last_peered":"2024-09-21T07:33:16.272116+0000","last_clean":"2024-09-21T07:33:16.272116+0000","last_became_active":"2024-09-21T07:33:15.174135+0000","last_became_peered":"2024-09-21T07:33:15.174135+0000","last_unstale":"2024-09-21T07:33:16.272116+0000","last_undegraded":"2024-09-21T07:33:16.272116+0000","last_fullsized":"2024-09-21T07:33:16.272116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:53.282103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171346+0000","last_change":"2024-09-21T07:33:15.184280+0000","last_active":"2024-09-21T07:33:16.171346+0000","last_peered":"2024-09-21T07:33:16.171346+0000","last_clean":"2024-09-21T07:33:16.171346+0000","last_became_active":"2024-09-21T07:33:15.183935+0000","last_became_peered":"2024-09-21T07:33:15.183935+0000","last_unstale":"2024-09-21T07:33:16.171346+0000","last_undegraded":"2024-09-21T07:33:16.171346+0000","last_fullsized":"2024-09-21T07:33:16.171346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:17:23.362898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.169177+0000","last_change":"2024-09-21T07:33:15.178501+0000","last_active":"2024-09-21T07:33:16.169177+0000","last_peered":"2024-09-21T07:33:16.169177+0000","last_clean":"2024-09-21T07:33:16.169177+0000","last_became_active":"2024-09-21T07:33:15.178157+0000","last_became_peered":"2024-09-21T07:33:15.178157+0000","last_unstale":"2024-09-21T07:33:16.169177+0000","last_undegraded":"2024-09-21T07:33:16.169177+0000","last_fullsized":"2024-09-21T07:33:16.169177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:47.494973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.170990+0000","last_change":"2024-09-21T07:33:15.194890+0000","last_active":"2024-09-21T07:33:16.170990+0000","last_peered":"2024-09-21T07:33:16.170990+0000","last_clean":"2024-09-21T07:33:16.170990+0000","last_became_active":"2024-09-21T07:33:15.194618+0000","last_became_peered":"2024-09-21T07:33:15.194618+0000","last_unstale":"2024-09-21T07:33:16.170990+0000","last_undegraded":"2024-09-21T07:33:16.170990+0000","last_fullsized":"2024-09-21T07:33:16.170990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:42:25.647850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.173528+0000","last_change":"2024-09-21T07:33:15.177265+0000","last_active":"2024-09-21T07:33:16.173528+0000","last_peered":"2024-09-21T07:33:16.173528+0000","last_clean":"2024-09-21T07:33:16.173528+0000","last_became_active":"2024-09-21T07:33:15.176835+0000","last_became_peered":"2024-09-21T07:33:15.176835+0000","last_unstale":"2024-09-21T07:33:16.173528+0000","last_undegraded":"2024-09-21T07:33:16.173528+0000","last_fullsized":"2024-09-21T07:33:16.173528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:26:27.046080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273611+0000","last_change":"2024-09-21T07:33:15.187455+0000","last_active":"2024-09-21T07:33:16.273611+0000","last_peered":"2024-09-21T07:33:16.273611+0000","last_clean":"2024-09-21T07:33:16.273611+0000","last_became_active":"2024-09-21T07:33:15.187097+0000","last_became_peered":"2024-09-21T07:33:15.187097+0000","last_unstale":"2024-09-21T07:33:16.273611+0000","last_undegraded":"2024-09-21T07:33:16.273611+0000","last_fullsized":"2024-09-21T07:33:16.273611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:12:27.933810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273575+0000","last_change":"2024-09-21T07:33:15.196207+0000","last_active":"2024-09-21T07:33:16.273575+0000","last_peered":"2024-09-21T07:33:16.273575+0000","last_clean":"2024-09-21T07:33:16.273575+0000","last_became_active":"2024-09-21T07:33:15.195889+0000","last_became_peered":"2024-09-21T07:33:15.195889+0000","last_unstale":"2024-09-21T07:33:16.273575+0000","last_undegraded":"2024-09-21T07:33:16.273575+0000","last_fullsized":"2024-09-21T07:33:16.273575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:23.420344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272226+0000","last_change":"2024-09-21T07:33:15.182205+0000","last_active":"2024-09-21T07:33:16.272226+0000","last_peered":"2024-09-21T07:33:16.272226+0000","last_clean":"2024-09-21T07:33:16.272226+0000","last_became_active":"2024-09-21T07:33:15.181790+0000","last_became_peered":"2024-09-21T07:33:15.181790+0000","last_unstale":"2024-09-21T07:33:16.272226+0000","last_undegraded":"2024-09-21T07:33:16.272226+0000","last_fullsized":"2024-09-21T07:33:16.272226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:09.053804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.169665+0000","last_change":"2024-09-21T07:33:15.172738+0000","last_active":"2024-09-21T07:33:16.169665+0000","last_peered":"2024-09-21T07:33:16.169665+0000","last_clean":"2024-09-21T07:33:16.169665+0000","last_became_active":"2024-09-21T07:33:15.172454+0000","last_became_peered":"2024-09-21T07:33:15.172454+0000","last_unstale":"2024-09-21T07:33:16.169665+0000","last_undegraded":"2024-09-21T07:33:16.169665+0000","last_fullsized":"2024-09-21T07:33:16.169665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:26:46.580701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272525+0000","last_change":"2024-09-21T07:33:15.185561+0000","last_active":"2024-09-21T07:33:16.272525+0000","last_peered":"2024-09-21T07:33:16.272525+0000","last_clean":"2024-09-21T07:33:16.272525+0000","last_became_active":"2024-09-21T07:33:15.185004+0000","last_became_peered":"2024-09-21T07:33:15.185004+0000","last_unstale":"2024-09-21T07:33:16.272525+0000","last_undegraded":"2024-09-21T07:33:16.272525+0000","last_fullsized":"2024-09-21T07:33:16.272525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:25:13.620607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.169268+0000","last_change":"2024-09-21T07:33:15.164795+0000","last_active":"2024-09-21T07:33:16.169268+0000","last_peered":"2024-09-21T07:33:16.169268+0000","last_clean":"2024-09-21T07:33:16.169268+0000","last_became_active":"2024-09-21T07:33:15.164419+0000","last_became_peered":"2024-09-21T07:33:15.164419+0000","last_unstale":"2024-09-21T07:33:16.169268+0000","last_undegraded":"2024-09-21T07:33:16.169268+0000","last_fullsized":"2024-09-21T07:33:16.169268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:48:05.009446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273570+0000","last_change":"2024-09-21T07:33:15.183825+0000","last_active":"2024-09-21T07:33:16.273570+0000","last_peered":"2024-09-21T07:33:16.273570+0000","last_clean":"2024-09-21T07:33:16.273570+0000","last_became_active":"2024-09-21T07:33:15.183503+0000","last_became_peered":"2024-09-21T07:33:15.183503+0000","last_unstale":"2024-09-21T07:33:16.273570+0000","last_undegraded":"2024-09-21T07:33:16.273570+0000","last_fullsized":"2024-09-21T07:33:16.273570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:12.378254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273927+0000","last_change":"2024-09-21T07:33:15.177070+0000","last_active":"2024-09-21T07:33:16.273927+0000","last_peered":"2024-09-21T07:33:16.273927+0000","last_clean":"2024-09-21T07:33:16.273927+0000","last_became_active":"2024-09-21T07:33:15.176706+0000","last_became_peered":"2024-09-21T07:33:15.176706+0000","last_unstale":"2024-09-21T07:33:16.273927+0000","last_undegraded":"2024-09-21T07:33:16.273927+0000","last_fullsized":"2024-09-21T07:33:16.273927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:40:41.309353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171642+0000","last_change":"2024-09-21T07:33:15.174783+0000","last_active":"2024-09-21T07:33:16.171642+0000","last_peered":"2024-09-21T07:33:16.171642+0000","last_clean":"2024-09-21T07:33:16.171642+0000","last_became_active":"2024-09-21T07:33:15.173581+0000","last_became_peered":"2024-09-21T07:33:15.173581+0000","last_unstale":"2024-09-21T07:33:16.171642+0000","last_undegraded":"2024-09-21T07:33:16.171642+0000","last_fullsized":"2024-09-21T07:33:16.171642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:43:34.665544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273063+0000","last_change":"2024-09-21T07:33:15.185421+0000","last_active":"2024-09-21T07:33:16.273063+0000","last_peered":"2024-09-21T07:33:16.273063+0000","last_clean":"2024-09-21T07:33:16.273063+0000","last_became_active":"2024-09-21T07:33:15.185131+0000","last_became_peered":"2024-09-21T07:33:15.185131+0000","last_unstale":"2024-09-21T07:33:16.273063+0000","last_undegraded":"2024-09-21T07:33:16.273063+0000","last_fullsized":"2024-09-21T07:33:16.273063+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:29.320339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273245+0000","last_change":"2024-09-21T07:33:15.178626+0000","last_active":"2024-09-21T07:33:16.273245+0000","last_peered":"2024-09-21T07:33:16.273245+0000","last_clean":"2024-09-21T07:33:16.273245+0000","last_became_active":"2024-09-21T07:33:15.178344+0000","last_became_peered":"2024-09-21T07:33:15.178344+0000","last_unstale":"2024-09-21T07:33:16.273245+0000","last_undegraded":"2024-09-21T07:33:16.273245+0000","last_fullsized":"2024-09-21T07:33:16.273245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:17:23.362898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.169955+0000","last_change":"2024-09-21T07:33:15.187835+0000","last_active":"2024-09-21T07:33:16.169955+0000","last_peered":"2024-09-21T07:33:16.169955+0000","last_clean":"2024-09-21T07:33:16.169955+0000","last_became_active":"2024-09-21T07:33:15.185993+0000","last_became_peered":"2024-09-21T07:33:15.185993+0000","last_unstale":"2024-09-21T07:33:16.169955+0000","last_undegraded":"2024-09-21T07:33:16.169955+0000","last_fullsized":"2024-09-21T07:33:16.169955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:17.480483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.173171+0000","last_change":"2024-09-21T07:33:15.163142+0000","last_active":"2024-09-21T07:33:16.173171+0000","last_peered":"2024-09-21T07:33:16.173171+0000","last_clean":"2024-09-21T07:33:16.173171+0000","last_became_active":"2024-09-21T07:33:15.162044+0000","last_became_peered":"2024-09-21T07:33:15.162044+0000","last_unstale":"2024-09-21T07:33:16.173171+0000","last_undegraded":"2024-09-21T07:33:16.173171+0000","last_fullsized":"2024-09-21T07:33:16.173171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:48:44.995481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272735+0000","last_change":"2024-09-21T07:33:15.188257+0000","last_active":"2024-09-21T07:33:16.272735+0000","last_peered":"2024-09-21T07:33:16.272735+0000","last_clean":"2024-09-21T07:33:16.272735+0000","last_became_active":"2024-09-21T07:33:15.184871+0000","last_became_peered":"2024-09-21T07:33:15.184871+0000","last_unstale":"2024-09-21T07:33:16.272735+0000","last_undegraded":"2024-09-21T07:33:16.272735+0000","last_fullsized":"2024-09-21T07:33:16.272735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:31:35.062821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274086+0000","last_change":"2024-09-21T07:33:15.178648+0000","last_active":"2024-09-21T07:33:16.274086+0000","last_peered":"2024-09-21T07:33:16.274086+0000","last_clean":"2024-09-21T07:33:16.274086+0000","last_became_active":"2024-09-21T07:33:15.178260+0000","last_became_peered":"2024-09-21T07:33:15.178260+0000","last_unstale":"2024-09-21T07:33:16.274086+0000","last_undegraded":"2024-09-21T07:33:16.274086+0000","last_fullsized":"2024-09-21T07:33:16.274086+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:36.198560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272837+0000","last_change":"2024-09-21T07:33:14.173209+0000","last_active":"2024-09-21T07:33:16.272837+0000","last_peered":"2024-09-21T07:33:16.272837+0000","last_clean":"2024-09-21T07:33:16.272837+0000","last_became_active":"2024-09-21T07:33:14.172936+0000","last_became_peered":"2024-09-21T07:33:14.172936+0000","last_unstale":"2024-09-21T07:33:16.272837+0000","last_undegraded":"2024-09-21T07:33:16.272837+0000","last_fullsized":"2024-09-21T07:33:16.272837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:03.284858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273336+0000","last_change":"2024-09-21T07:33:15.191981+0000","last_active":"2024-09-21T07:33:16.273336+0000","last_peered":"2024-09-21T07:33:16.273336+0000","last_clean":"2024-09-21T07:33:16.273336+0000","last_became_active":"2024-09-21T07:33:15.188976+0000","last_became_peered":"2024-09-21T07:33:15.188976+0000","last_unstale":"2024-09-21T07:33:16.273336+0000","last_undegraded":"2024-09-21T07:33:16.273336+0000","last_fullsized":"2024-09-21T07:33:16.273336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:34.842293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273636+0000","last_change":"2024-09-21T07:33:14.169344+0000","last_active":"2024-09-21T07:33:16.273636+0000","last_peered":"2024-09-21T07:33:16.273636+0000","last_clean":"2024-09-21T07:33:16.273636+0000","last_became_active":"2024-09-21T07:33:14.169015+0000","last_became_peered":"2024-09-21T07:33:14.169015+0000","last_unstale":"2024-09-21T07:33:16.273636+0000","last_undegraded":"2024-09-21T07:33:16.273636+0000","last_fullsized":"2024-09-21T07:33:16.273636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:10.925173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272616+0000","last_change":"2024-09-21T07:33:14.163956+0000","last_active":"2024-09-21T07:33:16.272616+0000","last_peered":"2024-09-21T07:33:16.272616+0000","last_clean":"2024-09-21T07:33:16.272616+0000","last_became_active":"2024-09-21T07:33:14.163698+0000","last_became_peered":"2024-09-21T07:33:14.163698+0000","last_unstale":"2024-09-21T07:33:16.272616+0000","last_undegraded":"2024-09-21T07:33:16.272616+0000","last_fullsized":"2024-09-21T07:33:16.272616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:32.172082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272523+0000","last_change":"2024-09-21T07:33:15.184364+0000","last_active":"2024-09-21T07:33:16.272523+0000","last_peered":"2024-09-21T07:33:16.272523+0000","last_clean":"2024-09-21T07:33:16.272523+0000","last_became_active":"2024-09-21T07:33:15.184042+0000","last_became_peered":"2024-09-21T07:33:15.184042+0000","last_unstale":"2024-09-21T07:33:16.272523+0000","last_undegraded":"2024-09-21T07:33:16.272523+0000","last_fullsized":"2024-09-21T07:33:16.272523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:39:05.189714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.169283+0000","last_change":"2024-09-21T07:33:14.160602+0000","last_active":"2024-09-21T07:33:16.169283+0000","last_peered":"2024-09-21T07:33:16.169283+0000","last_clean":"2024-09-21T07:33:16.169283+0000","last_became_active":"2024-09-21T07:33:14.159110+0000","last_became_peered":"2024-09-21T07:33:14.159110+0000","last_unstale":"2024-09-21T07:33:16.169283+0000","last_undegraded":"2024-09-21T07:33:16.169283+0000","last_fullsized":"2024-09-21T07:33:16.169283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:10.925173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171398+0000","last_change":"2024-09-21T07:33:15.178332+0000","last_active":"2024-09-21T07:33:16.171398+0000","last_peered":"2024-09-21T07:33:16.171398+0000","last_clean":"2024-09-21T07:33:16.171398+0000","last_became_active":"2024-09-21T07:33:15.177793+0000","last_became_peered":"2024-09-21T07:33:15.177793+0000","last_unstale":"2024-09-21T07:33:16.171398+0000","last_undegraded":"2024-09-21T07:33:16.171398+0000","last_fullsized":"2024-09-21T07:33:16.171398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:23.420344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273829+0000","last_change":"2024-09-21T07:33:14.171262+0000","last_active":"2024-09-21T07:33:16.273829+0000","last_peered":"2024-09-21T07:33:16.273829+0000","last_clean":"2024-09-21T07:33:16.273829+0000","last_became_active":"2024-09-21T07:33:14.170882+0000","last_became_peered":"2024-09-21T07:33:14.170882+0000","last_unstale":"2024-09-21T07:33:16.273829+0000","last_undegraded":"2024-09-21T07:33:16.273829+0000","last_fullsized":"2024-09-21T07:33:16.273829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:34.304149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.170713+0000","last_change":"2024-09-21T07:33:15.173639+0000","last_active":"2024-09-21T07:33:16.170713+0000","last_peered":"2024-09-21T07:33:16.170713+0000","last_clean":"2024-09-21T07:33:16.170713+0000","last_became_active":"2024-09-21T07:33:15.173275+0000","last_became_peered":"2024-09-21T07:33:15.173275+0000","last_unstale":"2024-09-21T07:33:16.170713+0000","last_undegraded":"2024-09-21T07:33:16.170713+0000","last_fullsized":"2024-09-21T07:33:16.170713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:12:30.901535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.170872+0000","last_change":"2024-09-21T07:33:14.170282+0000","last_active":"2024-09-21T07:33:16.170872+0000","last_peered":"2024-09-21T07:33:16.170872+0000","last_clean":"2024-09-21T07:33:16.170872+0000","last_became_active":"2024-09-21T07:33:14.169935+0000","last_became_peered":"2024-09-21T07:33:14.169935+0000","last_unstale":"2024-09-21T07:33:16.170872+0000","last_undegraded":"2024-09-21T07:33:16.170872+0000","last_fullsized":"2024-09-21T07:33:16.170872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:06.477956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273555+0000","last_change":"2024-09-21T07:33:15.171671+0000","last_active":"2024-09-21T07:33:16.273555+0000","last_peered":"2024-09-21T07:33:16.273555+0000","last_clean":"2024-09-21T07:33:16.273555+0000","last_became_active":"2024-09-21T07:33:15.171327+0000","last_became_peered":"2024-09-21T07:33:15.171327+0000","last_unstale":"2024-09-21T07:33:16.273555+0000","last_undegraded":"2024-09-21T07:33:16.273555+0000","last_fullsized":"2024-09-21T07:33:16.273555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:43:34.665544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171766+0000","last_change":"2024-09-21T07:33:14.166941+0000","last_active":"2024-09-21T07:33:16.171766+0000","last_peered":"2024-09-21T07:33:16.171766+0000","last_clean":"2024-09-21T07:33:16.171766+0000","last_became_active":"2024-09-21T07:33:14.165770+0000","last_became_peered":"2024-09-21T07:33:14.165770+0000","last_unstale":"2024-09-21T07:33:16.171766+0000","last_undegraded":"2024-09-21T07:33:16.171766+0000","last_fullsized":"2024-09-21T07:33:16.171766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:51.025109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274800+0000","last_change":"2024-09-21T07:33:15.181837+0000","last_active":"2024-09-21T07:33:16.274800+0000","last_peered":"2024-09-21T07:33:16.274800+0000","last_clean":"2024-09-21T07:33:16.274800+0000","last_became_active":"2024-09-21T07:33:15.181358+0000","last_became_peered":"2024-09-21T07:33:15.181358+0000","last_unstale":"2024-09-21T07:33:16.274800+0000","last_undegraded":"2024-09-21T07:33:16.274800+0000","last_fullsized":"2024-09-21T07:33:16.274800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:56.567663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273547+0000","last_change":"2024-09-21T07:33:14.172709+0000","last_active":"2024-09-21T07:33:16.273547+0000","last_peered":"2024-09-21T07:33:16.273547+0000","last_clean":"2024-09-21T07:33:16.273547+0000","last_became_active":"2024-09-21T07:33:14.172316+0000","last_became_peered":"2024-09-21T07:33:14.172316+0000","last_unstale":"2024-09-21T07:33:16.273547+0000","last_undegraded":"2024-09-21T07:33:16.273547+0000","last_fullsized":"2024-09-21T07:33:16.273547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:06.477956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272937+0000","last_change":"2024-09-21T07:33:15.179712+0000","last_active":"2024-09-21T07:33:16.272937+0000","last_peered":"2024-09-21T07:33:16.272937+0000","last_clean":"2024-09-21T07:33:16.272937+0000","last_became_active":"2024-09-21T07:33:15.179368+0000","last_became_peered":"2024-09-21T07:33:15.179368+0000","last_unstale":"2024-09-21T07:33:16.272937+0000","last_undegraded":"2024-09-21T07:33:16.272937+0000","last_fullsized":"2024-09-21T07:33:16.272937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:14.765726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273376+0000","last_change":"2024-09-21T07:33:14.180872+0000","last_active":"2024-09-21T07:33:16.273376+0000","last_peered":"2024-09-21T07:33:16.273376+0000","last_clean":"2024-09-21T07:33:16.273376+0000","last_became_active":"2024-09-21T07:33:14.179682+0000","last_became_peered":"2024-09-21T07:33:14.179682+0000","last_unstale":"2024-09-21T07:33:16.273376+0000","last_undegraded":"2024-09-21T07:33:16.273376+0000","last_fullsized":"2024-09-21T07:33:16.273376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:35:06.212701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272328+0000","last_change":"2024-09-21T07:33:15.177488+0000","last_active":"2024-09-21T07:33:16.272328+0000","last_peered":"2024-09-21T07:33:16.272328+0000","last_clean":"2024-09-21T07:33:16.272328+0000","last_became_active":"2024-09-21T07:33:15.176270+0000","last_became_peered":"2024-09-21T07:33:15.176270+0000","last_unstale":"2024-09-21T07:33:16.272328+0000","last_undegraded":"2024-09-21T07:33:16.272328+0000","last_fullsized":"2024-09-21T07:33:16.272328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:47.805013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.169798+0000","last_change":"2024-09-21T07:33:14.178744+0000","last_active":"2024-09-21T07:33:16.169798+0000","last_peered":"2024-09-21T07:33:16.169798+0000","last_clean":"2024-09-21T07:33:16.169798+0000","last_became_active":"2024-09-21T07:33:14.177786+0000","last_became_peered":"2024-09-21T07:33:14.177786+0000","last_unstale":"2024-09-21T07:33:16.169798+0000","last_undegraded":"2024-09-21T07:33:16.169798+0000","last_fullsized":"2024-09-21T07:33:16.169798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:34.304149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273092+0000","last_change":"2024-09-21T07:33:15.191036+0000","last_active":"2024-09-21T07:33:16.273092+0000","last_peered":"2024-09-21T07:33:16.273092+0000","last_clean":"2024-09-21T07:33:16.273092+0000","last_became_active":"2024-09-21T07:33:15.190631+0000","last_became_peered":"2024-09-21T07:33:15.190631+0000","last_unstale":"2024-09-21T07:33:16.273092+0000","last_undegraded":"2024-09-21T07:33:16.273092+0000","last_fullsized":"2024-09-21T07:33:16.273092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:39:17.752820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273048+0000","last_change":"2024-09-21T07:33:14.170962+0000","last_active":"2024-09-21T07:33:16.273048+0000","last_peered":"2024-09-21T07:33:16.273048+0000","last_clean":"2024-09-21T07:33:16.273048+0000","last_became_active":"2024-09-21T07:33:14.169996+0000","last_became_peered":"2024-09-21T07:33:14.169996+0000","last_unstale":"2024-09-21T07:33:16.273048+0000","last_undegraded":"2024-09-21T07:33:16.273048+0000","last_fullsized":"2024-09-21T07:33:16.273048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:45.280842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272830+0000","last_change":"2024-09-21T07:33:15.190018+0000","last_active":"2024-09-21T07:33:16.272830+0000","last_peered":"2024-09-21T07:33:16.272830+0000","last_clean":"2024-09-21T07:33:16.272830+0000","last_became_active":"2024-09-21T07:33:15.185361+0000","last_became_peered":"2024-09-21T07:33:15.185361+0000","last_unstale":"2024-09-21T07:33:16.272830+0000","last_undegraded":"2024-09-21T07:33:16.272830+0000","last_fullsized":"2024-09-21T07:33:16.272830+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:35.723678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.173087+0000","last_change":"2024-09-21T07:33:14.171085+0000","last_active":"2024-09-21T07:33:16.173087+0000","last_peered":"2024-09-21T07:33:16.173087+0000","last_clean":"2024-09-21T07:33:16.173087+0000","last_became_active":"2024-09-21T07:33:14.170838+0000","last_became_peered":"2024-09-21T07:33:14.170838+0000","last_unstale":"2024-09-21T07:33:16.173087+0000","last_undegraded":"2024-09-21T07:33:16.173087+0000","last_fullsized":"2024-09-21T07:33:16.173087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:06.477956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.275049+0000","last_change":"2024-09-21T07:33:15.181599+0000","last_active":"2024-09-21T07:33:16.275049+0000","last_peered":"2024-09-21T07:33:16.275049+0000","last_clean":"2024-09-21T07:33:16.275049+0000","last_became_active":"2024-09-21T07:33:15.181044+0000","last_became_peered":"2024-09-21T07:33:15.181044+0000","last_unstale":"2024-09-21T07:33:16.275049+0000","last_undegraded":"2024-09-21T07:33:16.275049+0000","last_fullsized":"2024-09-21T07:33:16.275049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:39.660131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273625+0000","last_change":"2024-09-21T07:33:14.180061+0000","last_active":"2024-09-21T07:33:16.273625+0000","last_peered":"2024-09-21T07:33:16.273625+0000","last_clean":"2024-09-21T07:33:16.273625+0000","last_became_active":"2024-09-21T07:33:14.179053+0000","last_became_peered":"2024-09-21T07:33:14.179053+0000","last_unstale":"2024-09-21T07:33:16.273625+0000","last_undegraded":"2024-09-21T07:33:16.273625+0000","last_fullsized":"2024-09-21T07:33:16.273625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:46.482047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274323+0000","last_change":"2024-09-21T07:33:15.187133+0000","last_active":"2024-09-21T07:33:16.274323+0000","last_peered":"2024-09-21T07:33:16.274323+0000","last_clean":"2024-09-21T07:33:16.274323+0000","last_became_active":"2024-09-21T07:33:15.186773+0000","last_became_peered":"2024-09-21T07:33:15.186773+0000","last_unstale":"2024-09-21T07:33:16.274323+0000","last_undegraded":"2024-09-21T07:33:16.274323+0000","last_fullsized":"2024-09-21T07:33:16.274323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:51.977863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.173649+0000","last_change":"2024-09-21T07:33:14.163096+0000","last_active":"2024-09-21T07:33:16.173649+0000","last_peered":"2024-09-21T07:33:16.173649+0000","last_clean":"2024-09-21T07:33:16.173649+0000","last_became_active":"2024-09-21T07:33:14.162512+0000","last_became_peered":"2024-09-21T07:33:14.162512+0000","last_unstale":"2024-09-21T07:33:16.173649+0000","last_undegraded":"2024-09-21T07:33:16.173649+0000","last_fullsized":"2024-09-21T07:33:16.173649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:10.925173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.275012+0000","last_change":"2024-09-21T07:33:15.184770+0000","last_active":"2024-09-21T07:33:16.275012+0000","last_peered":"2024-09-21T07:33:16.275012+0000","last_clean":"2024-09-21T07:33:16.275012+0000","last_became_active":"2024-09-21T07:33:15.184167+0000","last_became_peered":"2024-09-21T07:33:15.184167+0000","last_unstale":"2024-09-21T07:33:16.275012+0000","last_undegraded":"2024-09-21T07:33:16.275012+0000","last_fullsized":"2024-09-21T07:33:16.275012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:10.342501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171867+0000","last_change":"2024-09-21T07:33:14.170661+0000","last_active":"2024-09-21T07:33:16.171867+0000","last_peered":"2024-09-21T07:33:16.171867+0000","last_clean":"2024-09-21T07:33:16.171867+0000","last_became_active":"2024-09-21T07:33:14.170408+0000","last_became_peered":"2024-09-21T07:33:14.170408+0000","last_unstale":"2024-09-21T07:33:16.171867+0000","last_undegraded":"2024-09-21T07:33:16.171867+0000","last_fullsized":"2024-09-21T07:33:16.171867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:43:00.990750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.170680+0000","last_change":"2024-09-21T07:33:15.165673+0000","last_active":"2024-09-21T07:33:16.170680+0000","last_peered":"2024-09-21T07:33:16.170680+0000","last_clean":"2024-09-21T07:33:16.170680+0000","last_became_active":"2024-09-21T07:33:15.164392+0000","last_became_peered":"2024-09-21T07:33:15.164392+0000","last_unstale":"2024-09-21T07:33:16.170680+0000","last_undegraded":"2024-09-21T07:33:16.170680+0000","last_fullsized":"2024-09-21T07:33:16.170680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:19:11.398812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.173330+0000","last_change":"2024-09-21T07:33:14.166709+0000","last_active":"2024-09-21T07:33:16.173330+0000","last_peered":"2024-09-21T07:33:16.173330+0000","last_clean":"2024-09-21T07:33:16.173330+0000","last_became_active":"2024-09-21T07:33:14.166314+0000","last_became_peered":"2024-09-21T07:33:14.166314+0000","last_unstale":"2024-09-21T07:33:16.173330+0000","last_undegraded":"2024-09-21T07:33:16.173330+0000","last_fullsized":"2024-09-21T07:33:16.173330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:19.577738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273229+0000","last_change":"2024-09-21T07:33:15.195086+0000","last_active":"2024-09-21T07:33:16.273229+0000","last_peered":"2024-09-21T07:33:16.273229+0000","last_clean":"2024-09-21T07:33:16.273229+0000","last_became_active":"2024-09-21T07:33:15.194432+0000","last_became_peered":"2024-09-21T07:33:15.194432+0000","last_unstale":"2024-09-21T07:33:16.273229+0000","last_undegraded":"2024-09-21T07:33:16.273229+0000","last_fullsized":"2024-09-21T07:33:16.273229+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:51.754458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272609+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:33:16.272609+0000","last_peered":"2024-09-21T07:33:16.272609+0000","last_clean":"2024-09-21T07:33:16.272609+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:33:16.272609+0000","last_undegraded":"2024-09-21T07:33:16.272609+0000","last_fullsized":"2024-09-21T07:33:16.272609+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.170062+0000","last_change":"2024-09-21T07:33:14.167299+0000","last_active":"2024-09-21T07:33:16.170062+0000","last_peered":"2024-09-21T07:33:16.170062+0000","last_clean":"2024-09-21T07:33:16.170062+0000","last_became_active":"2024-09-21T07:33:14.165495+0000","last_became_peered":"2024-09-21T07:33:14.165495+0000","last_unstale":"2024-09-21T07:33:16.170062+0000","last_undegraded":"2024-09-21T07:33:16.170062+0000","last_fullsized":"2024-09-21T07:33:16.170062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:45.280842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272850+0000","last_change":"2024-09-21T07:33:15.193199+0000","last_active":"2024-09-21T07:33:16.272850+0000","last_peered":"2024-09-21T07:33:16.272850+0000","last_clean":"2024-09-21T07:33:16.272850+0000","last_became_active":"2024-09-21T07:33:15.192865+0000","last_became_peered":"2024-09-21T07:33:15.192865+0000","last_unstale":"2024-09-21T07:33:16.272850+0000","last_undegraded":"2024-09-21T07:33:16.272850+0000","last_fullsized":"2024-09-21T07:33:16.272850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:18.112050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272503+0000","last_change":"2024-09-21T07:33:14.182542+0000","last_active":"2024-09-21T07:33:16.272503+0000","last_peered":"2024-09-21T07:33:16.272503+0000","last_clean":"2024-09-21T07:33:16.272503+0000","last_became_active":"2024-09-21T07:33:14.182190+0000","last_became_peered":"2024-09-21T07:33:14.182190+0000","last_unstale":"2024-09-21T07:33:16.272503+0000","last_undegraded":"2024-09-21T07:33:16.272503+0000","last_fullsized":"2024-09-21T07:33:16.272503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:26:26.032819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272052+0000","last_change":"2024-09-21T07:33:15.185457+0000","last_active":"2024-09-21T07:33:16.272052+0000","last_peered":"2024-09-21T07:33:16.272052+0000","last_clean":"2024-09-21T07:33:16.272052+0000","last_became_active":"2024-09-21T07:33:15.185135+0000","last_became_peered":"2024-09-21T07:33:15.185135+0000","last_unstale":"2024-09-21T07:33:16.272052+0000","last_undegraded":"2024-09-21T07:33:16.272052+0000","last_fullsized":"2024-09-21T07:33:16.272052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:13:07.781038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.173288+0000","last_change":"2024-09-21T07:33:14.167137+0000","last_active":"2024-09-21T07:33:16.173288+0000","last_peered":"2024-09-21T07:33:16.173288+0000","last_clean":"2024-09-21T07:33:16.173288+0000","last_became_active":"2024-09-21T07:33:14.166565+0000","last_became_peered":"2024-09-21T07:33:14.166565+0000","last_unstale":"2024-09-21T07:33:16.173288+0000","last_undegraded":"2024-09-21T07:33:16.173288+0000","last_fullsized":"2024-09-21T07:33:16.173288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:45.280842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171392+0000","last_change":"2024-09-21T07:33:15.170020+0000","last_active":"2024-09-21T07:33:16.171392+0000","last_peered":"2024-09-21T07:33:16.171392+0000","last_clean":"2024-09-21T07:33:16.171392+0000","last_became_active":"2024-09-21T07:33:15.169603+0000","last_became_peered":"2024-09-21T07:33:15.169603+0000","last_unstale":"2024-09-21T07:33:16.171392+0000","last_undegraded":"2024-09-21T07:33:16.171392+0000","last_fullsized":"2024-09-21T07:33:16.171392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:42.179630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.169972+0000","last_change":"2024-09-21T07:33:14.171611+0000","last_active":"2024-09-21T07:33:16.169972+0000","last_peered":"2024-09-21T07:33:16.169972+0000","last_clean":"2024-09-21T07:33:16.169972+0000","last_became_active":"2024-09-21T07:33:14.171272+0000","last_became_peered":"2024-09-21T07:33:14.171272+0000","last_unstale":"2024-09-21T07:33:16.169972+0000","last_undegraded":"2024-09-21T07:33:16.169972+0000","last_fullsized":"2024-09-21T07:33:16.169972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:37.723838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274839+0000","last_change":"2024-09-21T07:33:15.184034+0000","last_active":"2024-09-21T07:33:16.274839+0000","last_peered":"2024-09-21T07:33:16.274839+0000","last_clean":"2024-09-21T07:33:16.274839+0000","last_became_active":"2024-09-21T07:33:15.183802+0000","last_became_peered":"2024-09-21T07:33:15.183802+0000","last_unstale":"2024-09-21T07:33:16.274839+0000","last_undegraded":"2024-09-21T07:33:16.274839+0000","last_fullsized":"2024-09-21T07:33:16.274839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:52:13.038200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171745+0000","last_change":"2024-09-21T07:33:14.179927+0000","last_active":"2024-09-21T07:33:16.171745+0000","last_peered":"2024-09-21T07:33:16.171745+0000","last_clean":"2024-09-21T07:33:16.171745+0000","last_became_active":"2024-09-21T07:33:14.179480+0000","last_became_peered":"2024-09-21T07:33:14.179480+0000","last_unstale":"2024-09-21T07:33:16.171745+0000","last_undegraded":"2024-09-21T07:33:16.171745+0000","last_fullsized":"2024-09-21T07:33:16.171745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:20:56.907479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.170757+0000","last_change":"2024-09-21T07:33:15.170345+0000","last_active":"2024-09-21T07:33:16.170757+0000","last_peered":"2024-09-21T07:33:16.170757+0000","last_clean":"2024-09-21T07:33:16.170757+0000","last_became_active":"2024-09-21T07:33:15.169978+0000","last_became_peered":"2024-09-21T07:33:15.169978+0000","last_unstale":"2024-09-21T07:33:16.170757+0000","last_undegraded":"2024-09-21T07:33:16.170757+0000","last_fullsized":"2024-09-21T07:33:16.170757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:22.852589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273738+0000","last_change":"2024-09-21T07:33:14.186976+0000","last_active":"2024-09-21T07:33:16.273738+0000","last_peered":"2024-09-21T07:33:16.273738+0000","last_clean":"2024-09-21T07:33:16.273738+0000","last_became_active":"2024-09-21T07:33:14.186584+0000","last_became_peered":"2024-09-21T07:33:14.186584+0000","last_unstale":"2024-09-21T07:33:16.273738+0000","last_undegraded":"2024-09-21T07:33:16.273738+0000","last_fullsized":"2024-09-21T07:33:16.273738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:48.956923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171626+0000","last_change":"2024-09-21T07:33:15.168130+0000","last_active":"2024-09-21T07:33:16.171626+0000","last_peered":"2024-09-21T07:33:16.171626+0000","last_clean":"2024-09-21T07:33:16.171626+0000","last_became_active":"2024-09-21T07:33:15.167822+0000","last_became_peered":"2024-09-21T07:33:15.167822+0000","last_unstale":"2024-09-21T07:33:16.171626+0000","last_undegraded":"2024-09-21T07:33:16.171626+0000","last_fullsized":"2024-09-21T07:33:16.171626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:21.223732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:17.161466+0000","last_change":"2024-09-21T07:33:14.170061+0000","last_active":"2024-09-21T07:33:17.161466+0000","last_peered":"2024-09-21T07:33:17.161466+0000","last_clean":"2024-09-21T07:33:17.161466+0000","last_became_active":"2024-09-21T07:33:14.169738+0000","last_became_peered":"2024-09-21T07:33:14.169738+0000","last_unstale":"2024-09-21T07:33:17.161466+0000","last_undegraded":"2024-09-21T07:33:17.161466+0000","last_fullsized":"2024-09-21T07:33:17.161466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:30.472062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273688+0000","last_change":"2024-09-21T07:33:15.179567+0000","last_active":"2024-09-21T07:33:16.273688+0000","last_peered":"2024-09-21T07:33:16.273688+0000","last_clean":"2024-09-21T07:33:16.273688+0000","last_became_active":"2024-09-21T07:33:15.179203+0000","last_became_peered":"2024-09-21T07:33:15.179203+0000","last_unstale":"2024-09-21T07:33:16.273688+0000","last_undegraded":"2024-09-21T07:33:16.273688+0000","last_fullsized":"2024-09-21T07:33:16.273688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:11:16.190546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273883+0000","last_change":"2024-09-21T07:33:14.178218+0000","last_active":"2024-09-21T07:33:16.273883+0000","last_peered":"2024-09-21T07:33:16.273883+0000","last_clean":"2024-09-21T07:33:16.273883+0000","last_became_active":"2024-09-21T07:33:14.177879+0000","last_became_peered":"2024-09-21T07:33:14.177879+0000","last_unstale":"2024-09-21T07:33:16.273883+0000","last_undegraded":"2024-09-21T07:33:16.273883+0000","last_fullsized":"2024-09-21T07:33:16.273883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:37.723838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273327+0000","last_change":"2024-09-21T07:33:15.174382+0000","last_active":"2024-09-21T07:33:16.273327+0000","last_peered":"2024-09-21T07:33:16.273327+0000","last_clean":"2024-09-21T07:33:16.273327+0000","last_became_active":"2024-09-21T07:33:15.174058+0000","last_became_peered":"2024-09-21T07:33:15.174058+0000","last_unstale":"2024-09-21T07:33:16.273327+0000","last_undegraded":"2024-09-21T07:33:16.273327+0000","last_fullsized":"2024-09-21T07:33:16.273327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:42.179630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273075+0000","last_change":"2024-09-21T07:33:14.186745+0000","last_active":"2024-09-21T07:33:16.273075+0000","last_peered":"2024-09-21T07:33:16.273075+0000","last_clean":"2024-09-21T07:33:16.273075+0000","last_became_active":"2024-09-21T07:33:14.186394+0000","last_became_peered":"2024-09-21T07:33:14.186394+0000","last_unstale":"2024-09-21T07:33:16.273075+0000","last_undegraded":"2024-09-21T07:33:16.273075+0000","last_fullsized":"2024-09-21T07:33:16.273075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:48:43.982220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274193+0000","last_change":"2024-09-21T07:33:15.183374+0000","last_active":"2024-09-21T07:33:16.274193+0000","last_peered":"2024-09-21T07:33:16.274193+0000","last_clean":"2024-09-21T07:33:16.274193+0000","last_became_active":"2024-09-21T07:33:15.182688+0000","last_became_peered":"2024-09-21T07:33:15.182688+0000","last_unstale":"2024-09-21T07:33:16.274193+0000","last_undegraded":"2024-09-21T07:33:16.274193+0000","last_fullsized":"2024-09-21T07:33:16.274193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:50.289066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171527+0000","last_change":"2024-09-21T07:33:14.176760+0000","last_active":"2024-09-21T07:33:16.171527+0000","last_peered":"2024-09-21T07:33:16.171527+0000","last_clean":"2024-09-21T07:33:16.171527+0000","last_became_active":"2024-09-21T07:33:14.176465+0000","last_became_peered":"2024-09-21T07:33:14.176465+0000","last_unstale":"2024-09-21T07:33:16.171527+0000","last_undegraded":"2024-09-21T07:33:16.171527+0000","last_fullsized":"2024-09-21T07:33:16.171527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:41:25.774466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272765+0000","last_change":"2024-09-21T07:33:15.173534+0000","last_active":"2024-09-21T07:33:16.272765+0000","last_peered":"2024-09-21T07:33:16.272765+0000","last_clean":"2024-09-21T07:33:16.272765+0000","last_became_active":"2024-09-21T07:33:15.173251+0000","last_became_peered":"2024-09-21T07:33:15.173251+0000","last_unstale":"2024-09-21T07:33:16.272765+0000","last_undegraded":"2024-09-21T07:33:16.272765+0000","last_fullsized":"2024-09-21T07:33:16.272765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:26.738282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171462+0000","last_change":"2024-09-21T07:33:14.171832+0000","last_active":"2024-09-21T07:33:16.171462+0000","last_peered":"2024-09-21T07:33:16.171462+0000","last_clean":"2024-09-21T07:33:16.171462+0000","last_became_active":"2024-09-21T07:33:14.171528+0000","last_became_peered":"2024-09-21T07:33:14.171528+0000","last_unstale":"2024-09-21T07:33:16.171462+0000","last_undegraded":"2024-09-21T07:33:16.171462+0000","last_fullsized":"2024-09-21T07:33:16.171462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:24:12.251487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272756+0000","last_change":"2024-09-21T07:33:15.178517+0000","last_active":"2024-09-21T07:33:16.272756+0000","last_peered":"2024-09-21T07:33:16.272756+0000","last_clean":"2024-09-21T07:33:16.272756+0000","last_became_active":"2024-09-21T07:33:15.178031+0000","last_became_peered":"2024-09-21T07:33:15.178031+0000","last_unstale":"2024-09-21T07:33:16.272756+0000","last_undegraded":"2024-09-21T07:33:16.272756+0000","last_fullsized":"2024-09-21T07:33:16.272756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:44.158584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273321+0000","last_change":"2024-09-21T07:33:14.177385+0000","last_active":"2024-09-21T07:33:16.273321+0000","last_peered":"2024-09-21T07:33:16.273321+0000","last_clean":"2024-09-21T07:33:16.273321+0000","last_became_active":"2024-09-21T07:33:14.176365+0000","last_became_peered":"2024-09-21T07:33:14.176365+0000","last_unstale":"2024-09-21T07:33:16.273321+0000","last_undegraded":"2024-09-21T07:33:16.273321+0000","last_fullsized":"2024-09-21T07:33:16.273321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:10.407008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273445+0000","last_change":"2024-09-21T07:33:15.190071+0000","last_active":"2024-09-21T07:33:16.273445+0000","last_peered":"2024-09-21T07:33:16.273445+0000","last_clean":"2024-09-21T07:33:16.273445+0000","last_became_active":"2024-09-21T07:33:15.189778+0000","last_became_peered":"2024-09-21T07:33:15.189778+0000","last_unstale":"2024-09-21T07:33:16.273445+0000","last_undegraded":"2024-09-21T07:33:16.273445+0000","last_fullsized":"2024-09-21T07:33:16.273445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:02.683216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.170600+0000","last_change":"2024-09-21T07:33:15.183143+0000","last_active":"2024-09-21T07:33:16.170600+0000","last_peered":"2024-09-21T07:33:16.170600+0000","last_clean":"2024-09-21T07:33:16.170600+0000","last_became_active":"2024-09-21T07:33:15.182803+0000","last_became_peered":"2024-09-21T07:33:15.182803+0000","last_unstale":"2024-09-21T07:33:16.170600+0000","last_undegraded":"2024-09-21T07:33:16.170600+0000","last_fullsized":"2024-09-21T07:33:16.170600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:35.308422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273005+0000","last_change":"2024-09-21T07:33:14.181290+0000","last_active":"2024-09-21T07:33:16.273005+0000","last_peered":"2024-09-21T07:33:16.273005+0000","last_clean":"2024-09-21T07:33:16.273005+0000","last_became_active":"2024-09-21T07:33:14.180919+0000","last_became_peered":"2024-09-21T07:33:14.180919+0000","last_unstale":"2024-09-21T07:33:16.273005+0000","last_undegraded":"2024-09-21T07:33:16.273005+0000","last_fullsized":"2024-09-21T07:33:16.273005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:37.723838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.170380+0000","last_change":"2024-09-21T07:33:15.194137+0000","last_active":"2024-09-21T07:33:16.170380+0000","last_peered":"2024-09-21T07:33:16.170380+0000","last_clean":"2024-09-21T07:33:16.170380+0000","last_became_active":"2024-09-21T07:33:15.193903+0000","last_became_peered":"2024-09-21T07:33:15.193903+0000","last_unstale":"2024-09-21T07:33:16.170380+0000","last_undegraded":"2024-09-21T07:33:16.170380+0000","last_fullsized":"2024-09-21T07:33:16.170380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:39:25.870595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273368+0000","last_change":"2024-09-21T07:33:14.178482+0000","last_active":"2024-09-21T07:33:16.273368+0000","last_peered":"2024-09-21T07:33:16.273368+0000","last_clean":"2024-09-21T07:33:16.273368+0000","last_became_active":"2024-09-21T07:33:14.178256+0000","last_became_peered":"2024-09-21T07:33:14.178256+0000","last_unstale":"2024-09-21T07:33:16.273368+0000","last_undegraded":"2024-09-21T07:33:16.273368+0000","last_fullsized":"2024-09-21T07:33:16.273368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:05.003971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273533+0000","last_change":"2024-09-21T07:33:15.177351+0000","last_active":"2024-09-21T07:33:16.273533+0000","last_peered":"2024-09-21T07:33:16.273533+0000","last_clean":"2024-09-21T07:33:16.273533+0000","last_became_active":"2024-09-21T07:33:15.172454+0000","last_became_peered":"2024-09-21T07:33:15.172454+0000","last_unstale":"2024-09-21T07:33:16.273533+0000","last_undegraded":"2024-09-21T07:33:16.273533+0000","last_fullsized":"2024-09-21T07:33:16.273533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:21.223732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273101+0000","last_change":"2024-09-21T07:33:14.178706+0000","last_active":"2024-09-21T07:33:16.273101+0000","last_peered":"2024-09-21T07:33:16.273101+0000","last_clean":"2024-09-21T07:33:16.273101+0000","last_became_active":"2024-09-21T07:33:14.178398+0000","last_became_peered":"2024-09-21T07:33:14.178398+0000","last_unstale":"2024-09-21T07:33:16.273101+0000","last_undegraded":"2024-09-21T07:33:16.273101+0000","last_fullsized":"2024-09-21T07:33:16.273101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:39.076808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273018+0000","last_change":"2024-09-21T07:33:15.181010+0000","last_active":"2024-09-21T07:33:16.273018+0000","last_peered":"2024-09-21T07:33:16.273018+0000","last_clean":"2024-09-21T07:33:16.273018+0000","last_became_active":"2024-09-21T07:33:15.179699+0000","last_became_peered":"2024-09-21T07:33:15.179699+0000","last_unstale":"2024-09-21T07:33:16.273018+0000","last_undegraded":"2024-09-21T07:33:16.273018+0000","last_fullsized":"2024-09-21T07:33:16.273018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:01.852516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273687+0000","last_change":"2024-09-21T07:33:14.177313+0000","last_active":"2024-09-21T07:33:16.273687+0000","last_peered":"2024-09-21T07:33:16.273687+0000","last_clean":"2024-09-21T07:33:16.273687+0000","last_became_active":"2024-09-21T07:33:14.176867+0000","last_became_peered":"2024-09-21T07:33:14.176867+0000","last_unstale":"2024-09-21T07:33:16.273687+0000","last_undegraded":"2024-09-21T07:33:16.273687+0000","last_fullsized":"2024-09-21T07:33:16.273687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:01.421862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":806912,"data_stored":663137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509449,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":73799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":72277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509448,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":208896,"data_stored":72187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509449,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":72347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509449,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":815104,"data_stored":663193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542152,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":72313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":221184,"data_stored":73799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:33:26.070 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:33:26.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.330+0000 7f7a87400640 1 -- 172.21.15.39:0/1774676268 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a800ce020 msgr2=0x7f7a800ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:26.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.330+0000 7f7a87400640 1 --2- 172.21.15.39:0/1774676268 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a800ce020 0x7f7a800ce430 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7a78005fd0 tx=0x7f7a7802aaa0 comp rx=0 tx=0).stop 2024-09-21T07:33:26.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.330+0000 7f7a87400640 1 -- 172.21.15.39:0/1774676268 shutdown_connections 2024-09-21T07:33:26.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.330+0000 7f7a87400640 1 --2- 172.21.15.39:0/1774676268 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a800cd240 0x7f7a800cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:26.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.330+0000 7f7a87400640 1 --2- 172.21.15.39:0/1774676268 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a800ce020 0x7f7a800ce430 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:26.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.330+0000 7f7a87400640 1 -- 172.21.15.39:0/1774676268 >> 172.21.15.39:0/1774676268 conn(0x7f7a800b1cb0 msgr2=0x7f7a800b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:26.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.330+0000 7f7a87400640 1 -- 172.21.15.39:0/1774676268 shutdown_connections 2024-09-21T07:33:26.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.331+0000 7f7a87400640 1 -- 172.21.15.39:0/1774676268 wait complete. 2024-09-21T07:33:26.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.333+0000 7f7a87400640 1 Processor -- start 2024-09-21T07:33:26.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.333+0000 7f7a87400640 1 -- start start 2024-09-21T07:33:26.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.334+0000 7f7a87400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a800ce020 0x7f7a8015b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:26.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.334+0000 7f7a87400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a800cf6f0 0x7f7a8009c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:26.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.334+0000 7f7a87400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a80157370 0x7f7a8009c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:26.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.334+0000 7f7a87400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7a800701b0 con 0x7f7a800cf6f0 2024-09-21T07:33:26.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.334+0000 7f7a87400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7a800cf3d0 con 0x7f7a800ce020 2024-09-21T07:33:26.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.334+0000 7f7a87400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f7a80070330 con 0x7f7a80157370 2024-09-21T07:33:26.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.334+0000 7f7a7fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a800cf6f0 0x7f7a8009c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:26.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.335+0000 7f7a7fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a800cf6f0 0x7f7a8009c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60278/0 (socket says 172.21.15.39:60278) 2024-09-21T07:33:26.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.335+0000 7f7a7fe00640 1 -- 172.21.15.39:0/1541820062 learned_addr learned my addr 172.21.15.39:0/1541820062 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:26.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.335+0000 7f7a84a00640 1 --2- 172.21.15.39:0/1541820062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a800ce020 0x7f7a8015b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:26.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.335+0000 7f7a85400640 1 --2- 172.21.15.39:0/1541820062 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a80157370 0x7f7a8009c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:26.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.335+0000 7f7a7fe00640 1 -- 172.21.15.39:0/1541820062 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a80157370 msgr2=0x7f7a8009c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:26.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.335+0000 7f7a7fe00640 1 --2- 172.21.15.39:0/1541820062 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a80157370 0x7f7a8009c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:26.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.335+0000 7f7a7fe00640 1 -- 172.21.15.39:0/1541820062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a800ce020 msgr2=0x7f7a8015b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:26.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.335+0000 7f7a7fe00640 1 --2- 172.21.15.39:0/1541820062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a800ce020 0x7f7a8015b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:26.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.335+0000 7f7a7fe00640 1 -- 172.21.15.39:0/1541820062 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8009d3c0 con 0x7f7a800cf6f0 2024-09-21T07:33:26.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.335+0000 7f7a7fe00640 1 --2- 172.21.15.39:0/1541820062 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a800cf6f0 0x7f7a8009c2c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c00d700 tx=0x7f7a6c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:26.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.335+0000 7f7a7cc00640 1 -- 172.21.15.39:0/1541820062 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c017070 con 0x7f7a800cf6f0 2024-09-21T07:33:26.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.336+0000 7f7a7cc00640 1 -- 172.21.15.39:0/1541820062 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a6c009b10 con 0x7f7a800cf6f0 2024-09-21T07:33:26.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.336+0000 7f7a7cc00640 1 -- 172.21.15.39:0/1541820062 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c010460 con 0x7f7a800cf6f0 2024-09-21T07:33:26.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.336+0000 7f7a87400640 1 -- 172.21.15.39:0/1541820062 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a8009d6a0 con 0x7f7a800cf6f0 2024-09-21T07:33:26.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.336+0000 7f7a87400640 1 -- 172.21.15.39:0/1541820062 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a8015bc40 con 0x7f7a800cf6f0 2024-09-21T07:33:26.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.337+0000 7f7a87400640 1 -- 172.21.15.39:0/1541820062 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a800cec30 con 0x7f7a800cf6f0 2024-09-21T07:33:26.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.338+0000 7f7a7cc00640 1 -- 172.21.15.39:0/1541820062 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7a6c01a070 con 0x7f7a800cf6f0 2024-09-21T07:33:26.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.339+0000 7f7a7cc00640 1 --2- 172.21.15.39:0/1541820062 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7a60078bc0 0x7f7a6007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:26.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.339+0000 7f7a7cc00640 1 -- 172.21.15.39:0/1541820062 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7a6c09ae30 con 0x7f7a800cf6f0 2024-09-21T07:33:26.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.339+0000 7f7a84a00640 1 --2- 172.21.15.39:0/1541820062 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7a60078bc0 0x7f7a6007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:26.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.340+0000 7f7a84a00640 1 --2- 172.21.15.39:0/1541820062 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7a60078bc0 0x7f7a6007b0d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7a80071290 tx=0x7f7a74009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:26.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.344+0000 7f7a7cc00640 1 -- 172.21.15.39:0/1541820062 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7a6c062b80 con 0x7f7a800cf6f0 2024-09-21T07:33:26.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.539+0000 7f7a87400640 1 -- 172.21.15.39:0/1541820062 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7a8009d870 con 0x7f7a60078bc0 2024-09-21T07:33:26.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.544+0000 7f7a7cc00640 1 -- 172.21.15.39:0/1541820062 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251826 (secure 0 0 0) 0x7f7a8009d870 con 0x7f7a60078bc0 2024-09-21T07:33:26.549 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:33:26.550 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:33:26.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.548+0000 7f7a87400640 1 -- 172.21.15.39:0/1541820062 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7a60078bc0 msgr2=0x7f7a6007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:26.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.548+0000 7f7a87400640 1 --2- 172.21.15.39:0/1541820062 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7a60078bc0 0x7f7a6007b0d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7a80071290 tx=0x7f7a74009040 comp rx=0 tx=0).stop 2024-09-21T07:33:26.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.548+0000 7f7a87400640 1 -- 172.21.15.39:0/1541820062 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a800cf6f0 msgr2=0x7f7a8009c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:26.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.548+0000 7f7a87400640 1 --2- 172.21.15.39:0/1541820062 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a800cf6f0 0x7f7a8009c2c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c00d700 tx=0x7f7a6c00dbc0 comp rx=0 tx=0).stop 2024-09-21T07:33:26.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.548+0000 7f7a87400640 1 -- 172.21.15.39:0/1541820062 shutdown_connections 2024-09-21T07:33:26.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.548+0000 7f7a87400640 1 --2- 172.21.15.39:0/1541820062 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7a60078bc0 0x7f7a6007b0d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:26.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.548+0000 7f7a87400640 1 --2- 172.21.15.39:0/1541820062 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7a80157370 0x7f7a8009c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:26.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.548+0000 7f7a87400640 1 --2- 172.21.15.39:0/1541820062 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a800cf6f0 0x7f7a8009c2c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:26.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.548+0000 7f7a87400640 1 --2- 172.21.15.39:0/1541820062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a800ce020 0x7f7a8015b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:26.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.548+0000 7f7a87400640 1 -- 172.21.15.39:0/1541820062 >> 172.21.15.39:0/1541820062 conn(0x7f7a800b1cb0 msgr2=0x7f7a8009daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:26.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.549+0000 7f7a87400640 1 -- 172.21.15.39:0/1541820062 shutdown_connections 2024-09-21T07:33:26.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.549+0000 7f7a87400640 1 -- 172.21.15.39:0/1541820062 wait complete. 2024-09-21T07:33:26.571 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-21T07:33:25.535563+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235740,"kb_used_data":2856,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738980,"statfs":{"total":773094113280,"available":772852715520,"internally_reserved":0,"allocated":2924544,"data_stored":1763052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002845"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274756+0000","last_change":"2024-09-21T07:33:15.176955+0000","last_active":"2024-09-21T07:33:16.274756+0000","last_peered":"2024-09-21T07:33:16.274756+0000","last_clean":"2024-09-21T07:33:16.274756+0000","last_became_active":"2024-09-21T07:33:15.176632+0000","last_became_peered":"2024-09-21T07:33:15.176632+0000","last_unstale":"2024-09-21T07:33:16.274756+0000","last_undegraded":"2024-09-21T07:33:16.274756+0000","last_fullsized":"2024-09-21T07:33:16.274756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:10:51.829331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171764+0000","last_change":"2024-09-21T07:33:15.184382+0000","last_active":"2024-09-21T07:33:16.171764+0000","last_peered":"2024-09-21T07:33:16.171764+0000","last_clean":"2024-09-21T07:33:16.171764+0000","last_became_active":"2024-09-21T07:33:15.184072+0000","last_became_peered":"2024-09-21T07:33:15.184072+0000","last_unstale":"2024-09-21T07:33:16.171764+0000","last_undegraded":"2024-09-21T07:33:16.171764+0000","last_fullsized":"2024-09-21T07:33:16.171764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:08.677357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273864+0000","last_change":"2024-09-21T07:33:15.186660+0000","last_active":"2024-09-21T07:33:16.273864+0000","last_peered":"2024-09-21T07:33:16.273864+0000","last_clean":"2024-09-21T07:33:16.273864+0000","last_became_active":"2024-09-21T07:33:15.185832+0000","last_became_peered":"2024-09-21T07:33:15.185832+0000","last_unstale":"2024-09-21T07:33:16.273864+0000","last_undegraded":"2024-09-21T07:33:16.273864+0000","last_fullsized":"2024-09-21T07:33:16.273864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:08.677357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.170390+0000","last_change":"2024-09-21T07:33:15.193591+0000","last_active":"2024-09-21T07:33:16.170390+0000","last_peered":"2024-09-21T07:33:16.170390+0000","last_clean":"2024-09-21T07:33:16.170390+0000","last_became_active":"2024-09-21T07:33:15.191327+0000","last_became_peered":"2024-09-21T07:33:15.191327+0000","last_unstale":"2024-09-21T07:33:16.170390+0000","last_undegraded":"2024-09-21T07:33:16.170390+0000","last_fullsized":"2024-09-21T07:33:16.170390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:00:15.919300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272836+0000","last_change":"2024-09-21T07:33:15.187754+0000","last_active":"2024-09-21T07:33:16.272836+0000","last_peered":"2024-09-21T07:33:16.272836+0000","last_clean":"2024-09-21T07:33:16.272836+0000","last_became_active":"2024-09-21T07:33:15.187381+0000","last_became_peered":"2024-09-21T07:33:15.187381+0000","last_unstale":"2024-09-21T07:33:16.272836+0000","last_undegraded":"2024-09-21T07:33:16.272836+0000","last_fullsized":"2024-09-21T07:33:16.272836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:19:59.491735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.169414+0000","last_change":"2024-09-21T07:33:15.180176+0000","last_active":"2024-09-21T07:33:16.169414+0000","last_peered":"2024-09-21T07:33:16.169414+0000","last_clean":"2024-09-21T07:33:16.169414+0000","last_became_active":"2024-09-21T07:33:15.179828+0000","last_became_peered":"2024-09-21T07:33:15.179828+0000","last_unstale":"2024-09-21T07:33:16.169414+0000","last_undegraded":"2024-09-21T07:33:16.169414+0000","last_fullsized":"2024-09-21T07:33:16.169414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:40:52.015841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274813+0000","last_change":"2024-09-21T07:33:15.183358+0000","last_active":"2024-09-21T07:33:16.274813+0000","last_peered":"2024-09-21T07:33:16.274813+0000","last_clean":"2024-09-21T07:33:16.274813+0000","last_became_active":"2024-09-21T07:33:15.182969+0000","last_became_peered":"2024-09-21T07:33:15.182969+0000","last_unstale":"2024-09-21T07:33:16.274813+0000","last_undegraded":"2024-09-21T07:33:16.274813+0000","last_fullsized":"2024-09-21T07:33:16.274813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:41:44.441220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272174+0000","last_change":"2024-09-21T07:33:15.173445+0000","last_active":"2024-09-21T07:33:16.272174+0000","last_peered":"2024-09-21T07:33:16.272174+0000","last_clean":"2024-09-21T07:33:16.272174+0000","last_became_active":"2024-09-21T07:33:15.172833+0000","last_became_peered":"2024-09-21T07:33:15.172833+0000","last_unstale":"2024-09-21T07:33:16.272174+0000","last_undegraded":"2024-09-21T07:33:16.272174+0000","last_fullsized":"2024-09-21T07:33:16.272174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:24:46.139397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274983+0000","last_change":"2024-09-21T07:33:15.194854+0000","last_active":"2024-09-21T07:33:16.274983+0000","last_peered":"2024-09-21T07:33:16.274983+0000","last_clean":"2024-09-21T07:33:16.274983+0000","last_became_active":"2024-09-21T07:33:15.194423+0000","last_became_peered":"2024-09-21T07:33:15.194423+0000","last_unstale":"2024-09-21T07:33:16.274983+0000","last_undegraded":"2024-09-21T07:33:16.274983+0000","last_fullsized":"2024-09-21T07:33:16.274983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:14.196020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274097+0000","last_change":"2024-09-21T07:33:15.176651+0000","last_active":"2024-09-21T07:33:16.274097+0000","last_peered":"2024-09-21T07:33:16.274097+0000","last_clean":"2024-09-21T07:33:16.274097+0000","last_became_active":"2024-09-21T07:33:15.176222+0000","last_became_peered":"2024-09-21T07:33:15.176222+0000","last_unstale":"2024-09-21T07:33:16.274097+0000","last_undegraded":"2024-09-21T07:33:16.274097+0000","last_fullsized":"2024-09-21T07:33:16.274097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:47.494973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274249+0000","last_change":"2024-09-21T07:33:15.170250+0000","last_active":"2024-09-21T07:33:16.274249+0000","last_peered":"2024-09-21T07:33:16.274249+0000","last_clean":"2024-09-21T07:33:16.274249+0000","last_became_active":"2024-09-21T07:33:15.166192+0000","last_became_peered":"2024-09-21T07:33:15.166192+0000","last_unstale":"2024-09-21T07:33:16.274249+0000","last_undegraded":"2024-09-21T07:33:16.274249+0000","last_fullsized":"2024-09-21T07:33:16.274249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:48:05.009446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274541+0000","last_change":"2024-09-21T07:33:15.180001+0000","last_active":"2024-09-21T07:33:16.274541+0000","last_peered":"2024-09-21T07:33:16.274541+0000","last_clean":"2024-09-21T07:33:16.274541+0000","last_became_active":"2024-09-21T07:33:15.179679+0000","last_became_peered":"2024-09-21T07:33:15.179679+0000","last_unstale":"2024-09-21T07:33:16.274541+0000","last_undegraded":"2024-09-21T07:33:16.274541+0000","last_fullsized":"2024-09-21T07:33:16.274541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:05:31.417897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272616+0000","last_change":"2024-09-21T07:33:15.179831+0000","last_active":"2024-09-21T07:33:16.272616+0000","last_peered":"2024-09-21T07:33:16.272616+0000","last_clean":"2024-09-21T07:33:16.272616+0000","last_became_active":"2024-09-21T07:33:15.179382+0000","last_became_peered":"2024-09-21T07:33:15.179382+0000","last_unstale":"2024-09-21T07:33:16.272616+0000","last_undegraded":"2024-09-21T07:33:16.272616+0000","last_fullsized":"2024-09-21T07:33:16.272616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:26:20.407200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272116+0000","last_change":"2024-09-21T07:33:15.174462+0000","last_active":"2024-09-21T07:33:16.272116+0000","last_peered":"2024-09-21T07:33:16.272116+0000","last_clean":"2024-09-21T07:33:16.272116+0000","last_became_active":"2024-09-21T07:33:15.174135+0000","last_became_peered":"2024-09-21T07:33:15.174135+0000","last_unstale":"2024-09-21T07:33:16.272116+0000","last_undegraded":"2024-09-21T07:33:16.272116+0000","last_fullsized":"2024-09-21T07:33:16.272116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:53.282103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171346+0000","last_change":"2024-09-21T07:33:15.184280+0000","last_active":"2024-09-21T07:33:16.171346+0000","last_peered":"2024-09-21T07:33:16.171346+0000","last_clean":"2024-09-21T07:33:16.171346+0000","last_became_active":"2024-09-21T07:33:15.183935+0000","last_became_peered":"2024-09-21T07:33:15.183935+0000","last_unstale":"2024-09-21T07:33:16.171346+0000","last_undegraded":"2024-09-21T07:33:16.171346+0000","last_fullsized":"2024-09-21T07:33:16.171346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:17:23.362898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.169177+0000","last_change":"2024-09-21T07:33:15.178501+0000","last_active":"2024-09-21T07:33:16.169177+0000","last_peered":"2024-09-21T07:33:16.169177+0000","last_clean":"2024-09-21T07:33:16.169177+0000","last_became_active":"2024-09-21T07:33:15.178157+0000","last_became_peered":"2024-09-21T07:33:15.178157+0000","last_unstale":"2024-09-21T07:33:16.169177+0000","last_undegraded":"2024-09-21T07:33:16.169177+0000","last_fullsized":"2024-09-21T07:33:16.169177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:47.494973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.170990+0000","last_change":"2024-09-21T07:33:15.194890+0000","last_active":"2024-09-21T07:33:16.170990+0000","last_peered":"2024-09-21T07:33:16.170990+0000","last_clean":"2024-09-21T07:33:16.170990+0000","last_became_active":"2024-09-21T07:33:15.194618+0000","last_became_peered":"2024-09-21T07:33:15.194618+0000","last_unstale":"2024-09-21T07:33:16.170990+0000","last_undegraded":"2024-09-21T07:33:16.170990+0000","last_fullsized":"2024-09-21T07:33:16.170990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:42:25.647850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.173528+0000","last_change":"2024-09-21T07:33:15.177265+0000","last_active":"2024-09-21T07:33:16.173528+0000","last_peered":"2024-09-21T07:33:16.173528+0000","last_clean":"2024-09-21T07:33:16.173528+0000","last_became_active":"2024-09-21T07:33:15.176835+0000","last_became_peered":"2024-09-21T07:33:15.176835+0000","last_unstale":"2024-09-21T07:33:16.173528+0000","last_undegraded":"2024-09-21T07:33:16.173528+0000","last_fullsized":"2024-09-21T07:33:16.173528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:26:27.046080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273611+0000","last_change":"2024-09-21T07:33:15.187455+0000","last_active":"2024-09-21T07:33:16.273611+0000","last_peered":"2024-09-21T07:33:16.273611+0000","last_clean":"2024-09-21T07:33:16.273611+0000","last_became_active":"2024-09-21T07:33:15.187097+0000","last_became_peered":"2024-09-21T07:33:15.187097+0000","last_unstale":"2024-09-21T07:33:16.273611+0000","last_undegraded":"2024-09-21T07:33:16.273611+0000","last_fullsized":"2024-09-21T07:33:16.273611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:12:27.933810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273575+0000","last_change":"2024-09-21T07:33:15.196207+0000","last_active":"2024-09-21T07:33:16.273575+0000","last_peered":"2024-09-21T07:33:16.273575+0000","last_clean":"2024-09-21T07:33:16.273575+0000","last_became_active":"2024-09-21T07:33:15.195889+0000","last_became_peered":"2024-09-21T07:33:15.195889+0000","last_unstale":"2024-09-21T07:33:16.273575+0000","last_undegraded":"2024-09-21T07:33:16.273575+0000","last_fullsized":"2024-09-21T07:33:16.273575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:23.420344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272226+0000","last_change":"2024-09-21T07:33:15.182205+0000","last_active":"2024-09-21T07:33:16.272226+0000","last_peered":"2024-09-21T07:33:16.272226+0000","last_clean":"2024-09-21T07:33:16.272226+0000","last_became_active":"2024-09-21T07:33:15.181790+0000","last_became_peered":"2024-09-21T07:33:15.181790+0000","last_unstale":"2024-09-21T07:33:16.272226+0000","last_undegraded":"2024-09-21T07:33:16.272226+0000","last_fullsized":"2024-09-21T07:33:16.272226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:09.053804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.169665+0000","last_change":"2024-09-21T07:33:15.172738+0000","last_active":"2024-09-21T07:33:16.169665+0000","last_peered":"2024-09-21T07:33:16.169665+0000","last_clean":"2024-09-21T07:33:16.169665+0000","last_became_active":"2024-09-21T07:33:15.172454+0000","last_became_peered":"2024-09-21T07:33:15.172454+0000","last_unstale":"2024-09-21T07:33:16.169665+0000","last_undegraded":"2024-09-21T07:33:16.169665+0000","last_fullsized":"2024-09-21T07:33:16.169665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:26:46.580701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272525+0000","last_change":"2024-09-21T07:33:15.185561+0000","last_active":"2024-09-21T07:33:16.272525+0000","last_peered":"2024-09-21T07:33:16.272525+0000","last_clean":"2024-09-21T07:33:16.272525+0000","last_became_active":"2024-09-21T07:33:15.185004+0000","last_became_peered":"2024-09-21T07:33:15.185004+0000","last_unstale":"2024-09-21T07:33:16.272525+0000","last_undegraded":"2024-09-21T07:33:16.272525+0000","last_fullsized":"2024-09-21T07:33:16.272525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:25:13.620607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.169268+0000","last_change":"2024-09-21T07:33:15.164795+0000","last_active":"2024-09-21T07:33:16.169268+0000","last_peered":"2024-09-21T07:33:16.169268+0000","last_clean":"2024-09-21T07:33:16.169268+0000","last_became_active":"2024-09-21T07:33:15.164419+0000","last_became_peered":"2024-09-21T07:33:15.164419+0000","last_unstale":"2024-09-21T07:33:16.169268+0000","last_undegraded":"2024-09-21T07:33:16.169268+0000","last_fullsized":"2024-09-21T07:33:16.169268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:48:05.009446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273570+0000","last_change":"2024-09-21T07:33:15.183825+0000","last_active":"2024-09-21T07:33:16.273570+0000","last_peered":"2024-09-21T07:33:16.273570+0000","last_clean":"2024-09-21T07:33:16.273570+0000","last_became_active":"2024-09-21T07:33:15.183503+0000","last_became_peered":"2024-09-21T07:33:15.183503+0000","last_unstale":"2024-09-21T07:33:16.273570+0000","last_undegraded":"2024-09-21T07:33:16.273570+0000","last_fullsized":"2024-09-21T07:33:16.273570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:12.378254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273927+0000","last_change":"2024-09-21T07:33:15.177070+0000","last_active":"2024-09-21T07:33:16.273927+0000","last_peered":"2024-09-21T07:33:16.273927+0000","last_clean":"2024-09-21T07:33:16.273927+0000","last_became_active":"2024-09-21T07:33:15.176706+0000","last_became_peered":"2024-09-21T07:33:15.176706+0000","last_unstale":"2024-09-21T07:33:16.273927+0000","last_undegraded":"2024-09-21T07:33:16.273927+0000","last_fullsized":"2024-09-21T07:33:16.273927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:40:41.309353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171642+0000","last_change":"2024-09-21T07:33:15.174783+0000","last_active":"2024-09-21T07:33:16.171642+0000","last_peered":"2024-09-21T07:33:16.171642+0000","last_clean":"2024-09-21T07:33:16.171642+0000","last_became_active":"2024-09-21T07:33:15.173581+0000","last_became_peered":"2024-09-21T07:33:15.173581+0000","last_unstale":"2024-09-21T07:33:16.171642+0000","last_undegraded":"2024-09-21T07:33:16.171642+0000","last_fullsized":"2024-09-21T07:33:16.171642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:43:34.665544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273063+0000","last_change":"2024-09-21T07:33:15.185421+0000","last_active":"2024-09-21T07:33:16.273063+0000","last_peered":"2024-09-21T07:33:16.273063+0000","last_clean":"2024-09-21T07:33:16.273063+0000","last_became_active":"2024-09-21T07:33:15.185131+0000","last_became_peered":"2024-09-21T07:33:15.185131+0000","last_unstale":"2024-09-21T07:33:16.273063+0000","last_undegraded":"2024-09-21T07:33:16.273063+0000","last_fullsized":"2024-09-21T07:33:16.273063+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:29.320339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273245+0000","last_change":"2024-09-21T07:33:15.178626+0000","last_active":"2024-09-21T07:33:16.273245+0000","last_peered":"2024-09-21T07:33:16.273245+0000","last_clean":"2024-09-21T07:33:16.273245+0000","last_became_active":"2024-09-21T07:33:15.178344+0000","last_became_peered":"2024-09-21T07:33:15.178344+0000","last_unstale":"2024-09-21T07:33:16.273245+0000","last_undegraded":"2024-09-21T07:33:16.273245+0000","last_fullsized":"2024-09-21T07:33:16.273245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:17:23.362898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.169955+0000","last_change":"2024-09-21T07:33:15.187835+0000","last_active":"2024-09-21T07:33:16.169955+0000","last_peered":"2024-09-21T07:33:16.169955+0000","last_clean":"2024-09-21T07:33:16.169955+0000","last_became_active":"2024-09-21T07:33:15.185993+0000","last_became_peered":"2024-09-21T07:33:15.185993+0000","last_unstale":"2024-09-21T07:33:16.169955+0000","last_undegraded":"2024-09-21T07:33:16.169955+0000","last_fullsized":"2024-09-21T07:33:16.169955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:17.480483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.173171+0000","last_change":"2024-09-21T07:33:15.163142+0000","last_active":"2024-09-21T07:33:16.173171+0000","last_peered":"2024-09-21T07:33:16.173171+0000","last_clean":"2024-09-21T07:33:16.173171+0000","last_became_active":"2024-09-21T07:33:15.162044+0000","last_became_peered":"2024-09-21T07:33:15.162044+0000","last_unstale":"2024-09-21T07:33:16.173171+0000","last_undegraded":"2024-09-21T07:33:16.173171+0000","last_fullsized":"2024-09-21T07:33:16.173171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:48:44.995481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272735+0000","last_change":"2024-09-21T07:33:15.188257+0000","last_active":"2024-09-21T07:33:16.272735+0000","last_peered":"2024-09-21T07:33:16.272735+0000","last_clean":"2024-09-21T07:33:16.272735+0000","last_became_active":"2024-09-21T07:33:15.184871+0000","last_became_peered":"2024-09-21T07:33:15.184871+0000","last_unstale":"2024-09-21T07:33:16.272735+0000","last_undegraded":"2024-09-21T07:33:16.272735+0000","last_fullsized":"2024-09-21T07:33:16.272735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:31:35.062821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274086+0000","last_change":"2024-09-21T07:33:15.178648+0000","last_active":"2024-09-21T07:33:16.274086+0000","last_peered":"2024-09-21T07:33:16.274086+0000","last_clean":"2024-09-21T07:33:16.274086+0000","last_became_active":"2024-09-21T07:33:15.178260+0000","last_became_peered":"2024-09-21T07:33:15.178260+0000","last_unstale":"2024-09-21T07:33:16.274086+0000","last_undegraded":"2024-09-21T07:33:16.274086+0000","last_fullsized":"2024-09-21T07:33:16.274086+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:36.198560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272837+0000","last_change":"2024-09-21T07:33:14.173209+0000","last_active":"2024-09-21T07:33:16.272837+0000","last_peered":"2024-09-21T07:33:16.272837+0000","last_clean":"2024-09-21T07:33:16.272837+0000","last_became_active":"2024-09-21T07:33:14.172936+0000","last_became_peered":"2024-09-21T07:33:14.172936+0000","last_unstale":"2024-09-21T07:33:16.272837+0000","last_undegraded":"2024-09-21T07:33:16.272837+0000","last_fullsized":"2024-09-21T07:33:16.272837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:03.284858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273336+0000","last_change":"2024-09-21T07:33:15.191981+0000","last_active":"2024-09-21T07:33:16.273336+0000","last_peered":"2024-09-21T07:33:16.273336+0000","last_clean":"2024-09-21T07:33:16.273336+0000","last_became_active":"2024-09-21T07:33:15.188976+0000","last_became_peered":"2024-09-21T07:33:15.188976+0000","last_unstale":"2024-09-21T07:33:16.273336+0000","last_undegraded":"2024-09-21T07:33:16.273336+0000","last_fullsized":"2024-09-21T07:33:16.273336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:34.842293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273636+0000","last_change":"2024-09-21T07:33:14.169344+0000","last_active":"2024-09-21T07:33:16.273636+0000","last_peered":"2024-09-21T07:33:16.273636+0000","last_clean":"2024-09-21T07:33:16.273636+0000","last_became_active":"2024-09-21T07:33:14.169015+0000","last_became_peered":"2024-09-21T07:33:14.169015+0000","last_unstale":"2024-09-21T07:33:16.273636+0000","last_undegraded":"2024-09-21T07:33:16.273636+0000","last_fullsized":"2024-09-21T07:33:16.273636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:10.925173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272616+0000","last_change":"2024-09-21T07:33:14.163956+0000","last_active":"2024-09-21T07:33:16.272616+0000","last_peered":"2024-09-21T07:33:16.272616+0000","last_clean":"2024-09-21T07:33:16.272616+0000","last_became_active":"2024-09-21T07:33:14.163698+0000","last_became_peered":"2024-09-21T07:33:14.163698+0000","last_unstale":"2024-09-21T07:33:16.272616+0000","last_undegraded":"2024-09-21T07:33:16.272616+0000","last_fullsized":"2024-09-21T07:33:16.272616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:32.172082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272523+0000","last_change":"2024-09-21T07:33:15.184364+0000","last_active":"2024-09-21T07:33:16.272523+0000","last_peered":"2024-09-21T07:33:16.272523+0000","last_clean":"2024-09-21T07:33:16.272523+0000","last_became_active":"2024-09-21T07:33:15.184042+0000","last_became_peered":"2024-09-21T07:33:15.184042+0000","last_unstale":"2024-09-21T07:33:16.272523+0000","last_undegraded":"2024-09-21T07:33:16.272523+0000","last_fullsized":"2024-09-21T07:33:16.272523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:39:05.189714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.169283+0000","last_change":"2024-09-21T07:33:14.160602+0000","last_active":"2024-09-21T07:33:16.169283+0000","last_peered":"2024-09-21T07:33:16.169283+0000","last_clean":"2024-09-21T07:33:16.169283+0000","last_became_active":"2024-09-21T07:33:14.159110+0000","last_became_peered":"2024-09-21T07:33:14.159110+0000","last_unstale":"2024-09-21T07:33:16.169283+0000","last_undegraded":"2024-09-21T07:33:16.169283+0000","last_fullsized":"2024-09-21T07:33:16.169283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:10.925173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171398+0000","last_change":"2024-09-21T07:33:15.178332+0000","last_active":"2024-09-21T07:33:16.171398+0000","last_peered":"2024-09-21T07:33:16.171398+0000","last_clean":"2024-09-21T07:33:16.171398+0000","last_became_active":"2024-09-21T07:33:15.177793+0000","last_became_peered":"2024-09-21T07:33:15.177793+0000","last_unstale":"2024-09-21T07:33:16.171398+0000","last_undegraded":"2024-09-21T07:33:16.171398+0000","last_fullsized":"2024-09-21T07:33:16.171398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:23.420344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273829+0000","last_change":"2024-09-21T07:33:14.171262+0000","last_active":"2024-09-21T07:33:16.273829+0000","last_peered":"2024-09-21T07:33:16.273829+0000","last_clean":"2024-09-21T07:33:16.273829+0000","last_became_active":"2024-09-21T07:33:14.170882+0000","last_became_peered":"2024-09-21T07:33:14.170882+0000","last_unstale":"2024-09-21T07:33:16.273829+0000","last_undegraded":"2024-09-21T07:33:16.273829+0000","last_fullsized":"2024-09-21T07:33:16.273829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:34.304149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.170713+0000","last_change":"2024-09-21T07:33:15.173639+0000","last_active":"2024-09-21T07:33:16.170713+0000","last_peered":"2024-09-21T07:33:16.170713+0000","last_clean":"2024-09-21T07:33:16.170713+0000","last_became_active":"2024-09-21T07:33:15.173275+0000","last_became_peered":"2024-09-21T07:33:15.173275+0000","last_unstale":"2024-09-21T07:33:16.170713+0000","last_undegraded":"2024-09-21T07:33:16.170713+0000","last_fullsized":"2024-09-21T07:33:16.170713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:12:30.901535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.170872+0000","last_change":"2024-09-21T07:33:14.170282+0000","last_active":"2024-09-21T07:33:16.170872+0000","last_peered":"2024-09-21T07:33:16.170872+0000","last_clean":"2024-09-21T07:33:16.170872+0000","last_became_active":"2024-09-21T07:33:14.169935+0000","last_became_peered":"2024-09-21T07:33:14.169935+0000","last_unstale":"2024-09-21T07:33:16.170872+0000","last_undegraded":"2024-09-21T07:33:16.170872+0000","last_fullsized":"2024-09-21T07:33:16.170872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:06.477956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273555+0000","last_change":"2024-09-21T07:33:15.171671+0000","last_active":"2024-09-21T07:33:16.273555+0000","last_peered":"2024-09-21T07:33:16.273555+0000","last_clean":"2024-09-21T07:33:16.273555+0000","last_became_active":"2024-09-21T07:33:15.171327+0000","last_became_peered":"2024-09-21T07:33:15.171327+0000","last_unstale":"2024-09-21T07:33:16.273555+0000","last_undegraded":"2024-09-21T07:33:16.273555+0000","last_fullsized":"2024-09-21T07:33:16.273555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:43:34.665544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171766+0000","last_change":"2024-09-21T07:33:14.166941+0000","last_active":"2024-09-21T07:33:16.171766+0000","last_peered":"2024-09-21T07:33:16.171766+0000","last_clean":"2024-09-21T07:33:16.171766+0000","last_became_active":"2024-09-21T07:33:14.165770+0000","last_became_peered":"2024-09-21T07:33:14.165770+0000","last_unstale":"2024-09-21T07:33:16.171766+0000","last_undegraded":"2024-09-21T07:33:16.171766+0000","last_fullsized":"2024-09-21T07:33:16.171766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:51.025109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274800+0000","last_change":"2024-09-21T07:33:15.181837+0000","last_active":"2024-09-21T07:33:16.274800+0000","last_peered":"2024-09-21T07:33:16.274800+0000","last_clean":"2024-09-21T07:33:16.274800+0000","last_became_active":"2024-09-21T07:33:15.181358+0000","last_became_peered":"2024-09-21T07:33:15.181358+0000","last_unstale":"2024-09-21T07:33:16.274800+0000","last_undegraded":"2024-09-21T07:33:16.274800+0000","last_fullsized":"2024-09-21T07:33:16.274800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:56.567663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273547+0000","last_change":"2024-09-21T07:33:14.172709+0000","last_active":"2024-09-21T07:33:16.273547+0000","last_peered":"2024-09-21T07:33:16.273547+0000","last_clean":"2024-09-21T07:33:16.273547+0000","last_became_active":"2024-09-21T07:33:14.172316+0000","last_became_peered":"2024-09-21T07:33:14.172316+0000","last_unstale":"2024-09-21T07:33:16.273547+0000","last_undegraded":"2024-09-21T07:33:16.273547+0000","last_fullsized":"2024-09-21T07:33:16.273547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:06.477956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272937+0000","last_change":"2024-09-21T07:33:15.179712+0000","last_active":"2024-09-21T07:33:16.272937+0000","last_peered":"2024-09-21T07:33:16.272937+0000","last_clean":"2024-09-21T07:33:16.272937+0000","last_became_active":"2024-09-21T07:33:15.179368+0000","last_became_peered":"2024-09-21T07:33:15.179368+0000","last_unstale":"2024-09-21T07:33:16.272937+0000","last_undegraded":"2024-09-21T07:33:16.272937+0000","last_fullsized":"2024-09-21T07:33:16.272937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:14.765726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273376+0000","last_change":"2024-09-21T07:33:14.180872+0000","last_active":"2024-09-21T07:33:16.273376+0000","last_peered":"2024-09-21T07:33:16.273376+0000","last_clean":"2024-09-21T07:33:16.273376+0000","last_became_active":"2024-09-21T07:33:14.179682+0000","last_became_peered":"2024-09-21T07:33:14.179682+0000","last_unstale":"2024-09-21T07:33:16.273376+0000","last_undegraded":"2024-09-21T07:33:16.273376+0000","last_fullsized":"2024-09-21T07:33:16.273376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:35:06.212701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272328+0000","last_change":"2024-09-21T07:33:15.177488+0000","last_active":"2024-09-21T07:33:16.272328+0000","last_peered":"2024-09-21T07:33:16.272328+0000","last_clean":"2024-09-21T07:33:16.272328+0000","last_became_active":"2024-09-21T07:33:15.176270+0000","last_became_peered":"2024-09-21T07:33:15.176270+0000","last_unstale":"2024-09-21T07:33:16.272328+0000","last_undegraded":"2024-09-21T07:33:16.272328+0000","last_fullsized":"2024-09-21T07:33:16.272328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:47.805013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.169798+0000","last_change":"2024-09-21T07:33:14.178744+0000","last_active":"2024-09-21T07:33:16.169798+0000","last_peered":"2024-09-21T07:33:16.169798+0000","last_clean":"2024-09-21T07:33:16.169798+0000","last_became_active":"2024-09-21T07:33:14.177786+0000","last_became_peered":"2024-09-21T07:33:14.177786+0000","last_unstale":"2024-09-21T07:33:16.169798+0000","last_undegraded":"2024-09-21T07:33:16.169798+0000","last_fullsized":"2024-09-21T07:33:16.169798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:34.304149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273092+0000","last_change":"2024-09-21T07:33:15.191036+0000","last_active":"2024-09-21T07:33:16.273092+0000","last_peered":"2024-09-21T07:33:16.273092+0000","last_clean":"2024-09-21T07:33:16.273092+0000","last_became_active":"2024-09-21T07:33:15.190631+0000","last_became_peered":"2024-09-21T07:33:15.190631+0000","last_unstale":"2024-09-21T07:33:16.273092+0000","last_undegraded":"2024-09-21T07:33:16.273092+0000","last_fullsized":"2024-09-21T07:33:16.273092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:39:17.752820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273048+0000","last_change":"2024-09-21T07:33:14.170962+0000","last_active":"2024-09-21T07:33:16.273048+0000","last_peered":"2024-09-21T07:33:16.273048+0000","last_clean":"2024-09-21T07:33:16.273048+0000","last_became_active":"2024-09-21T07:33:14.169996+0000","last_became_peered":"2024-09-21T07:33:14.169996+0000","last_unstale":"2024-09-21T07:33:16.273048+0000","last_undegraded":"2024-09-21T07:33:16.273048+0000","last_fullsized":"2024-09-21T07:33:16.273048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:45.280842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272830+0000","last_change":"2024-09-21T07:33:15.190018+0000","last_active":"2024-09-21T07:33:16.272830+0000","last_peered":"2024-09-21T07:33:16.272830+0000","last_clean":"2024-09-21T07:33:16.272830+0000","last_became_active":"2024-09-21T07:33:15.185361+0000","last_became_peered":"2024-09-21T07:33:15.185361+0000","last_unstale":"2024-09-21T07:33:16.272830+0000","last_undegraded":"2024-09-21T07:33:16.272830+0000","last_fullsized":"2024-09-21T07:33:16.272830+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:35.723678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.173087+0000","last_change":"2024-09-21T07:33:14.171085+0000","last_active":"2024-09-21T07:33:16.173087+0000","last_peered":"2024-09-21T07:33:16.173087+0000","last_clean":"2024-09-21T07:33:16.173087+0000","last_became_active":"2024-09-21T07:33:14.170838+0000","last_became_peered":"2024-09-21T07:33:14.170838+0000","last_unstale":"2024-09-21T07:33:16.173087+0000","last_undegraded":"2024-09-21T07:33:16.173087+0000","last_fullsized":"2024-09-21T07:33:16.173087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:06.477956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.275049+0000","last_change":"2024-09-21T07:33:15.181599+0000","last_active":"2024-09-21T07:33:16.275049+0000","last_peered":"2024-09-21T07:33:16.275049+0000","last_clean":"2024-09-21T07:33:16.275049+0000","last_became_active":"2024-09-21T07:33:15.181044+0000","last_became_peered":"2024-09-21T07:33:15.181044+0000","last_unstale":"2024-09-21T07:33:16.275049+0000","last_undegraded":"2024-09-21T07:33:16.275049+0000","last_fullsized":"2024-09-21T07:33:16.275049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:39.660131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273625+0000","last_change":"2024-09-21T07:33:14.180061+0000","last_active":"2024-09-21T07:33:16.273625+0000","last_peered":"2024-09-21T07:33:16.273625+0000","last_clean":"2024-09-21T07:33:16.273625+0000","last_became_active":"2024-09-21T07:33:14.179053+0000","last_became_peered":"2024-09-21T07:33:14.179053+0000","last_unstale":"2024-09-21T07:33:16.273625+0000","last_undegraded":"2024-09-21T07:33:16.273625+0000","last_fullsized":"2024-09-21T07:33:16.273625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:46.482047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274323+0000","last_change":"2024-09-21T07:33:15.187133+0000","last_active":"2024-09-21T07:33:16.274323+0000","last_peered":"2024-09-21T07:33:16.274323+0000","last_clean":"2024-09-21T07:33:16.274323+0000","last_became_active":"2024-09-21T07:33:15.186773+0000","last_became_peered":"2024-09-21T07:33:15.186773+0000","last_unstale":"2024-09-21T07:33:16.274323+0000","last_undegraded":"2024-09-21T07:33:16.274323+0000","last_fullsized":"2024-09-21T07:33:16.274323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:51.977863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.173649+0000","last_change":"2024-09-21T07:33:14.163096+0000","last_active":"2024-09-21T07:33:16.173649+0000","last_peered":"2024-09-21T07:33:16.173649+0000","last_clean":"2024-09-21T07:33:16.173649+0000","last_became_active":"2024-09-21T07:33:14.162512+0000","last_became_peered":"2024-09-21T07:33:14.162512+0000","last_unstale":"2024-09-21T07:33:16.173649+0000","last_undegraded":"2024-09-21T07:33:16.173649+0000","last_fullsized":"2024-09-21T07:33:16.173649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:10.925173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.275012+0000","last_change":"2024-09-21T07:33:15.184770+0000","last_active":"2024-09-21T07:33:16.275012+0000","last_peered":"2024-09-21T07:33:16.275012+0000","last_clean":"2024-09-21T07:33:16.275012+0000","last_became_active":"2024-09-21T07:33:15.184167+0000","last_became_peered":"2024-09-21T07:33:15.184167+0000","last_unstale":"2024-09-21T07:33:16.275012+0000","last_undegraded":"2024-09-21T07:33:16.275012+0000","last_fullsized":"2024-09-21T07:33:16.275012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:10.342501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171867+0000","last_change":"2024-09-21T07:33:14.170661+0000","last_active":"2024-09-21T07:33:16.171867+0000","last_peered":"2024-09-21T07:33:16.171867+0000","last_clean":"2024-09-21T07:33:16.171867+0000","last_became_active":"2024-09-21T07:33:14.170408+0000","last_became_peered":"2024-09-21T07:33:14.170408+0000","last_unstale":"2024-09-21T07:33:16.171867+0000","last_undegraded":"2024-09-21T07:33:16.171867+0000","last_fullsized":"2024-09-21T07:33:16.171867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:43:00.990750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.170680+0000","last_change":"2024-09-21T07:33:15.165673+0000","last_active":"2024-09-21T07:33:16.170680+0000","last_peered":"2024-09-21T07:33:16.170680+0000","last_clean":"2024-09-21T07:33:16.170680+0000","last_became_active":"2024-09-21T07:33:15.164392+0000","last_became_peered":"2024-09-21T07:33:15.164392+0000","last_unstale":"2024-09-21T07:33:16.170680+0000","last_undegraded":"2024-09-21T07:33:16.170680+0000","last_fullsized":"2024-09-21T07:33:16.170680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:19:11.398812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.173330+0000","last_change":"2024-09-21T07:33:14.166709+0000","last_active":"2024-09-21T07:33:16.173330+0000","last_peered":"2024-09-21T07:33:16.173330+0000","last_clean":"2024-09-21T07:33:16.173330+0000","last_became_active":"2024-09-21T07:33:14.166314+0000","last_became_peered":"2024-09-21T07:33:14.166314+0000","last_unstale":"2024-09-21T07:33:16.173330+0000","last_undegraded":"2024-09-21T07:33:16.173330+0000","last_fullsized":"2024-09-21T07:33:16.173330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:19.577738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273229+0000","last_change":"2024-09-21T07:33:15.195086+0000","last_active":"2024-09-21T07:33:16.273229+0000","last_peered":"2024-09-21T07:33:16.273229+0000","last_clean":"2024-09-21T07:33:16.273229+0000","last_became_active":"2024-09-21T07:33:15.194432+0000","last_became_peered":"2024-09-21T07:33:15.194432+0000","last_unstale":"2024-09-21T07:33:16.273229+0000","last_undegraded":"2024-09-21T07:33:16.273229+0000","last_fullsized":"2024-09-21T07:33:16.273229+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:51.754458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272609+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:33:16.272609+0000","last_peered":"2024-09-21T07:33:16.272609+0000","last_clean":"2024-09-21T07:33:16.272609+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:33:16.272609+0000","last_undegraded":"2024-09-21T07:33:16.272609+0000","last_fullsized":"2024-09-21T07:33:16.272609+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.170062+0000","last_change":"2024-09-21T07:33:14.167299+0000","last_active":"2024-09-21T07:33:16.170062+0000","last_peered":"2024-09-21T07:33:16.170062+0000","last_clean":"2024-09-21T07:33:16.170062+0000","last_became_active":"2024-09-21T07:33:14.165495+0000","last_became_peered":"2024-09-21T07:33:14.165495+0000","last_unstale":"2024-09-21T07:33:16.170062+0000","last_undegraded":"2024-09-21T07:33:16.170062+0000","last_fullsized":"2024-09-21T07:33:16.170062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:45.280842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272850+0000","last_change":"2024-09-21T07:33:15.193199+0000","last_active":"2024-09-21T07:33:16.272850+0000","last_peered":"2024-09-21T07:33:16.272850+0000","last_clean":"2024-09-21T07:33:16.272850+0000","last_became_active":"2024-09-21T07:33:15.192865+0000","last_became_peered":"2024-09-21T07:33:15.192865+0000","last_unstale":"2024-09-21T07:33:16.272850+0000","last_undegraded":"2024-09-21T07:33:16.272850+0000","last_fullsized":"2024-09-21T07:33:16.272850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:18.112050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272503+0000","last_change":"2024-09-21T07:33:14.182542+0000","last_active":"2024-09-21T07:33:16.272503+0000","last_peered":"2024-09-21T07:33:16.272503+0000","last_clean":"2024-09-21T07:33:16.272503+0000","last_became_active":"2024-09-21T07:33:14.182190+0000","last_became_peered":"2024-09-21T07:33:14.182190+0000","last_unstale":"2024-09-21T07:33:16.272503+0000","last_undegraded":"2024-09-21T07:33:16.272503+0000","last_fullsized":"2024-09-21T07:33:16.272503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:26:26.032819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272052+0000","last_change":"2024-09-21T07:33:15.185457+0000","last_active":"2024-09-21T07:33:16.272052+0000","last_peered":"2024-09-21T07:33:16.272052+0000","last_clean":"2024-09-21T07:33:16.272052+0000","last_became_active":"2024-09-21T07:33:15.185135+0000","last_became_peered":"2024-09-21T07:33:15.185135+0000","last_unstale":"2024-09-21T07:33:16.272052+0000","last_undegraded":"2024-09-21T07:33:16.272052+0000","last_fullsized":"2024-09-21T07:33:16.272052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:13:07.781038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.173288+0000","last_change":"2024-09-21T07:33:14.167137+0000","last_active":"2024-09-21T07:33:16.173288+0000","last_peered":"2024-09-21T07:33:16.173288+0000","last_clean":"2024-09-21T07:33:16.173288+0000","last_became_active":"2024-09-21T07:33:14.166565+0000","last_became_peered":"2024-09-21T07:33:14.166565+0000","last_unstale":"2024-09-21T07:33:16.173288+0000","last_undegraded":"2024-09-21T07:33:16.173288+0000","last_fullsized":"2024-09-21T07:33:16.173288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:45.280842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171392+0000","last_change":"2024-09-21T07:33:15.170020+0000","last_active":"2024-09-21T07:33:16.171392+0000","last_peered":"2024-09-21T07:33:16.171392+0000","last_clean":"2024-09-21T07:33:16.171392+0000","last_became_active":"2024-09-21T07:33:15.169603+0000","last_became_peered":"2024-09-21T07:33:15.169603+0000","last_unstale":"2024-09-21T07:33:16.171392+0000","last_undegraded":"2024-09-21T07:33:16.171392+0000","last_fullsized":"2024-09-21T07:33:16.171392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:42.179630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.169972+0000","last_change":"2024-09-21T07:33:14.171611+0000","last_active":"2024-09-21T07:33:16.169972+0000","last_peered":"2024-09-21T07:33:16.169972+0000","last_clean":"2024-09-21T07:33:16.169972+0000","last_became_active":"2024-09-21T07:33:14.171272+0000","last_became_peered":"2024-09-21T07:33:14.171272+0000","last_unstale":"2024-09-21T07:33:16.169972+0000","last_undegraded":"2024-09-21T07:33:16.169972+0000","last_fullsized":"2024-09-21T07:33:16.169972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:37.723838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274839+0000","last_change":"2024-09-21T07:33:15.184034+0000","last_active":"2024-09-21T07:33:16.274839+0000","last_peered":"2024-09-21T07:33:16.274839+0000","last_clean":"2024-09-21T07:33:16.274839+0000","last_became_active":"2024-09-21T07:33:15.183802+0000","last_became_peered":"2024-09-21T07:33:15.183802+0000","last_unstale":"2024-09-21T07:33:16.274839+0000","last_undegraded":"2024-09-21T07:33:16.274839+0000","last_fullsized":"2024-09-21T07:33:16.274839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:52:13.038200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171745+0000","last_change":"2024-09-21T07:33:14.179927+0000","last_active":"2024-09-21T07:33:16.171745+0000","last_peered":"2024-09-21T07:33:16.171745+0000","last_clean":"2024-09-21T07:33:16.171745+0000","last_became_active":"2024-09-21T07:33:14.179480+0000","last_became_peered":"2024-09-21T07:33:14.179480+0000","last_unstale":"2024-09-21T07:33:16.171745+0000","last_undegraded":"2024-09-21T07:33:16.171745+0000","last_fullsized":"2024-09-21T07:33:16.171745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:20:56.907479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.170757+0000","last_change":"2024-09-21T07:33:15.170345+0000","last_active":"2024-09-21T07:33:16.170757+0000","last_peered":"2024-09-21T07:33:16.170757+0000","last_clean":"2024-09-21T07:33:16.170757+0000","last_became_active":"2024-09-21T07:33:15.169978+0000","last_became_peered":"2024-09-21T07:33:15.169978+0000","last_unstale":"2024-09-21T07:33:16.170757+0000","last_undegraded":"2024-09-21T07:33:16.170757+0000","last_fullsized":"2024-09-21T07:33:16.170757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:22.852589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273738+0000","last_change":"2024-09-21T07:33:14.186976+0000","last_active":"2024-09-21T07:33:16.273738+0000","last_peered":"2024-09-21T07:33:16.273738+0000","last_clean":"2024-09-21T07:33:16.273738+0000","last_became_active":"2024-09-21T07:33:14.186584+0000","last_became_peered":"2024-09-21T07:33:14.186584+0000","last_unstale":"2024-09-21T07:33:16.273738+0000","last_undegraded":"2024-09-21T07:33:16.273738+0000","last_fullsized":"2024-09-21T07:33:16.273738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:48.956923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171626+0000","last_change":"2024-09-21T07:33:15.168130+0000","last_active":"2024-09-21T07:33:16.171626+0000","last_peered":"2024-09-21T07:33:16.171626+0000","last_clean":"2024-09-21T07:33:16.171626+0000","last_became_active":"2024-09-21T07:33:15.167822+0000","last_became_peered":"2024-09-21T07:33:15.167822+0000","last_unstale":"2024-09-21T07:33:16.171626+0000","last_undegraded":"2024-09-21T07:33:16.171626+0000","last_fullsized":"2024-09-21T07:33:16.171626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:21.223732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:17.161466+0000","last_change":"2024-09-21T07:33:14.170061+0000","last_active":"2024-09-21T07:33:17.161466+0000","last_peered":"2024-09-21T07:33:17.161466+0000","last_clean":"2024-09-21T07:33:17.161466+0000","last_became_active":"2024-09-21T07:33:14.169738+0000","last_became_peered":"2024-09-21T07:33:14.169738+0000","last_unstale":"2024-09-21T07:33:17.161466+0000","last_undegraded":"2024-09-21T07:33:17.161466+0000","last_fullsized":"2024-09-21T07:33:17.161466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:30.472062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273688+0000","last_change":"2024-09-21T07:33:15.179567+0000","last_active":"2024-09-21T07:33:16.273688+0000","last_peered":"2024-09-21T07:33:16.273688+0000","last_clean":"2024-09-21T07:33:16.273688+0000","last_became_active":"2024-09-21T07:33:15.179203+0000","last_became_peered":"2024-09-21T07:33:15.179203+0000","last_unstale":"2024-09-21T07:33:16.273688+0000","last_undegraded":"2024-09-21T07:33:16.273688+0000","last_fullsized":"2024-09-21T07:33:16.273688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:11:16.190546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273883+0000","last_change":"2024-09-21T07:33:14.178218+0000","last_active":"2024-09-21T07:33:16.273883+0000","last_peered":"2024-09-21T07:33:16.273883+0000","last_clean":"2024-09-21T07:33:16.273883+0000","last_became_active":"2024-09-21T07:33:14.177879+0000","last_became_peered":"2024-09-21T07:33:14.177879+0000","last_unstale":"2024-09-21T07:33:16.273883+0000","last_undegraded":"2024-09-21T07:33:16.273883+0000","last_fullsized":"2024-09-21T07:33:16.273883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:37.723838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273327+0000","last_change":"2024-09-21T07:33:15.174382+0000","last_active":"2024-09-21T07:33:16.273327+0000","last_peered":"2024-09-21T07:33:16.273327+0000","last_clean":"2024-09-21T07:33:16.273327+0000","last_became_active":"2024-09-21T07:33:15.174058+0000","last_became_peered":"2024-09-21T07:33:15.174058+0000","last_unstale":"2024-09-21T07:33:16.273327+0000","last_undegraded":"2024-09-21T07:33:16.273327+0000","last_fullsized":"2024-09-21T07:33:16.273327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:42.179630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273075+0000","last_change":"2024-09-21T07:33:14.186745+0000","last_active":"2024-09-21T07:33:16.273075+0000","last_peered":"2024-09-21T07:33:16.273075+0000","last_clean":"2024-09-21T07:33:16.273075+0000","last_became_active":"2024-09-21T07:33:14.186394+0000","last_became_peered":"2024-09-21T07:33:14.186394+0000","last_unstale":"2024-09-21T07:33:16.273075+0000","last_undegraded":"2024-09-21T07:33:16.273075+0000","last_fullsized":"2024-09-21T07:33:16.273075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:48:43.982220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.274193+0000","last_change":"2024-09-21T07:33:15.183374+0000","last_active":"2024-09-21T07:33:16.274193+0000","last_peered":"2024-09-21T07:33:16.274193+0000","last_clean":"2024-09-21T07:33:16.274193+0000","last_became_active":"2024-09-21T07:33:15.182688+0000","last_became_peered":"2024-09-21T07:33:15.182688+0000","last_unstale":"2024-09-21T07:33:16.274193+0000","last_undegraded":"2024-09-21T07:33:16.274193+0000","last_fullsized":"2024-09-21T07:33:16.274193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:50.289066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171527+0000","last_change":"2024-09-21T07:33:14.176760+0000","last_active":"2024-09-21T07:33:16.171527+0000","last_peered":"2024-09-21T07:33:16.171527+0000","last_clean":"2024-09-21T07:33:16.171527+0000","last_became_active":"2024-09-21T07:33:14.176465+0000","last_became_peered":"2024-09-21T07:33:14.176465+0000","last_unstale":"2024-09-21T07:33:16.171527+0000","last_undegraded":"2024-09-21T07:33:16.171527+0000","last_fullsized":"2024-09-21T07:33:16.171527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:41:25.774466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272765+0000","last_change":"2024-09-21T07:33:15.173534+0000","last_active":"2024-09-21T07:33:16.272765+0000","last_peered":"2024-09-21T07:33:16.272765+0000","last_clean":"2024-09-21T07:33:16.272765+0000","last_became_active":"2024-09-21T07:33:15.173251+0000","last_became_peered":"2024-09-21T07:33:15.173251+0000","last_unstale":"2024-09-21T07:33:16.272765+0000","last_undegraded":"2024-09-21T07:33:16.272765+0000","last_fullsized":"2024-09-21T07:33:16.272765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:26.738282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.171462+0000","last_change":"2024-09-21T07:33:14.171832+0000","last_active":"2024-09-21T07:33:16.171462+0000","last_peered":"2024-09-21T07:33:16.171462+0000","last_clean":"2024-09-21T07:33:16.171462+0000","last_became_active":"2024-09-21T07:33:14.171528+0000","last_became_peered":"2024-09-21T07:33:14.171528+0000","last_unstale":"2024-09-21T07:33:16.171462+0000","last_undegraded":"2024-09-21T07:33:16.171462+0000","last_fullsized":"2024-09-21T07:33:16.171462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:24:12.251487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.272756+0000","last_change":"2024-09-21T07:33:15.178517+0000","last_active":"2024-09-21T07:33:16.272756+0000","last_peered":"2024-09-21T07:33:16.272756+0000","last_clean":"2024-09-21T07:33:16.272756+0000","last_became_active":"2024-09-21T07:33:15.178031+0000","last_became_peered":"2024-09-21T07:33:15.178031+0000","last_unstale":"2024-09-21T07:33:16.272756+0000","last_undegraded":"2024-09-21T07:33:16.272756+0000","last_fullsized":"2024-09-21T07:33:16.272756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:44.158584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273321+0000","last_change":"2024-09-21T07:33:14.177385+0000","last_active":"2024-09-21T07:33:16.273321+0000","last_peered":"2024-09-21T07:33:16.273321+0000","last_clean":"2024-09-21T07:33:16.273321+0000","last_became_active":"2024-09-21T07:33:14.176365+0000","last_became_peered":"2024-09-21T07:33:14.176365+0000","last_unstale":"2024-09-21T07:33:16.273321+0000","last_undegraded":"2024-09-21T07:33:16.273321+0000","last_fullsized":"2024-09-21T07:33:16.273321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:10.407008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273445+0000","last_change":"2024-09-21T07:33:15.190071+0000","last_active":"2024-09-21T07:33:16.273445+0000","last_peered":"2024-09-21T07:33:16.273445+0000","last_clean":"2024-09-21T07:33:16.273445+0000","last_became_active":"2024-09-21T07:33:15.189778+0000","last_became_peered":"2024-09-21T07:33:15.189778+0000","last_unstale":"2024-09-21T07:33:16.273445+0000","last_undegraded":"2024-09-21T07:33:16.273445+0000","last_fullsized":"2024-09-21T07:33:16.273445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:02.683216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.170600+0000","last_change":"2024-09-21T07:33:15.183143+0000","last_active":"2024-09-21T07:33:16.170600+0000","last_peered":"2024-09-21T07:33:16.170600+0000","last_clean":"2024-09-21T07:33:16.170600+0000","last_became_active":"2024-09-21T07:33:15.182803+0000","last_became_peered":"2024-09-21T07:33:15.182803+0000","last_unstale":"2024-09-21T07:33:16.170600+0000","last_undegraded":"2024-09-21T07:33:16.170600+0000","last_fullsized":"2024-09-21T07:33:16.170600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:35.308422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273005+0000","last_change":"2024-09-21T07:33:14.181290+0000","last_active":"2024-09-21T07:33:16.273005+0000","last_peered":"2024-09-21T07:33:16.273005+0000","last_clean":"2024-09-21T07:33:16.273005+0000","last_became_active":"2024-09-21T07:33:14.180919+0000","last_became_peered":"2024-09-21T07:33:14.180919+0000","last_unstale":"2024-09-21T07:33:16.273005+0000","last_undegraded":"2024-09-21T07:33:16.273005+0000","last_fullsized":"2024-09-21T07:33:16.273005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:37.723838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.170380+0000","last_change":"2024-09-21T07:33:15.194137+0000","last_active":"2024-09-21T07:33:16.170380+0000","last_peered":"2024-09-21T07:33:16.170380+0000","last_clean":"2024-09-21T07:33:16.170380+0000","last_became_active":"2024-09-21T07:33:15.193903+0000","last_became_peered":"2024-09-21T07:33:15.193903+0000","last_unstale":"2024-09-21T07:33:16.170380+0000","last_undegraded":"2024-09-21T07:33:16.170380+0000","last_fullsized":"2024-09-21T07:33:16.170380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:39:25.870595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273368+0000","last_change":"2024-09-21T07:33:14.178482+0000","last_active":"2024-09-21T07:33:16.273368+0000","last_peered":"2024-09-21T07:33:16.273368+0000","last_clean":"2024-09-21T07:33:16.273368+0000","last_became_active":"2024-09-21T07:33:14.178256+0000","last_became_peered":"2024-09-21T07:33:14.178256+0000","last_unstale":"2024-09-21T07:33:16.273368+0000","last_undegraded":"2024-09-21T07:33:16.273368+0000","last_fullsized":"2024-09-21T07:33:16.273368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:05.003971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273533+0000","last_change":"2024-09-21T07:33:15.177351+0000","last_active":"2024-09-21T07:33:16.273533+0000","last_peered":"2024-09-21T07:33:16.273533+0000","last_clean":"2024-09-21T07:33:16.273533+0000","last_became_active":"2024-09-21T07:33:15.172454+0000","last_became_peered":"2024-09-21T07:33:15.172454+0000","last_unstale":"2024-09-21T07:33:16.273533+0000","last_undegraded":"2024-09-21T07:33:16.273533+0000","last_fullsized":"2024-09-21T07:33:16.273533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:21.223732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273101+0000","last_change":"2024-09-21T07:33:14.178706+0000","last_active":"2024-09-21T07:33:16.273101+0000","last_peered":"2024-09-21T07:33:16.273101+0000","last_clean":"2024-09-21T07:33:16.273101+0000","last_became_active":"2024-09-21T07:33:14.178398+0000","last_became_peered":"2024-09-21T07:33:14.178398+0000","last_unstale":"2024-09-21T07:33:16.273101+0000","last_undegraded":"2024-09-21T07:33:16.273101+0000","last_fullsized":"2024-09-21T07:33:16.273101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:39.076808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273018+0000","last_change":"2024-09-21T07:33:15.181010+0000","last_active":"2024-09-21T07:33:16.273018+0000","last_peered":"2024-09-21T07:33:16.273018+0000","last_clean":"2024-09-21T07:33:16.273018+0000","last_became_active":"2024-09-21T07:33:15.179699+0000","last_became_peered":"2024-09-21T07:33:15.179699+0000","last_unstale":"2024-09-21T07:33:16.273018+0000","last_undegraded":"2024-09-21T07:33:16.273018+0000","last_fullsized":"2024-09-21T07:33:16.273018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:14.135026+0000","last_clean_scrub_stamp":"2024-09-21T07:33:14.135026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:01.852516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T07:33:16.273687+0000","last_change":"2024-09-21T07:33:14.177313+0000","last_active":"2024-09-21T07:33:16.273687+0000","last_peered":"2024-09-21T07:33:16.273687+0000","last_clean":"2024-09-21T07:33:16.273687+0000","last_became_active":"2024-09-21T07:33:14.176867+0000","last_became_peered":"2024-09-21T07:33:14.176867+0000","last_unstale":"2024-09-21T07:33:16.273687+0000","last_undegraded":"2024-09-21T07:33:16.273687+0000","last_fullsized":"2024-09-21T07:33:16.273687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:13.121765+0000","last_clean_scrub_stamp":"2024-09-21T07:33:13.121765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:01.421862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":806912,"data_stored":663137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509449,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":73799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":72277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509448,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":208896,"data_stored":72187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509449,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":72347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509449,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":815104,"data_stored":663193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542152,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":72313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":221184,"data_stored":73799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:33:26.575 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T07:33:26.575 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T07:33:26.575 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-21T07:33:26.576 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-21T07:33:26.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.838+0000 7fec7b200640 1 -- 172.21.15.39:0/214174972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec60003c30 msgr2=0x7fec60004080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:26.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.838+0000 7fec7b200640 1 --2- 172.21.15.39:0/214174972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec60003c30 0x7fec60004080 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fec6c00c2e0 tx=0x7fec60005830 comp rx=0 tx=0).stop 2024-09-21T07:33:26.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.838+0000 7fec7b200640 1 -- 172.21.15.39:0/214174972 shutdown_connections 2024-09-21T07:33:26.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.838+0000 7fec7b200640 1 --2- 172.21.15.39:0/214174972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec60003c30 0x7fec60004080 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:26.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.838+0000 7fec7b200640 1 -- 172.21.15.39:0/214174972 >> 172.21.15.39:0/214174972 conn(0x7fec7c08bfc0 msgr2=0x7fec7c09d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:26.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.838+0000 7fec7b200640 1 -- 172.21.15.39:0/214174972 shutdown_connections 2024-09-21T07:33:26.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.838+0000 7fec7b200640 1 -- 172.21.15.39:0/214174972 wait complete. 2024-09-21T07:33:26.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.840+0000 7fec7b200640 1 Processor -- start 2024-09-21T07:33:26.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.840+0000 7fec7b200640 1 -- start start 2024-09-21T07:33:26.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.842+0000 7fec7b200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec60003c30 0x7fec7c0d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:26.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.842+0000 7fec7b200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec7c157a50 0x7fec7c1571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:26.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.842+0000 7fec7b200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec7c1608d0 0x7fec7c160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:26.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.842+0000 7fec7b200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c161e10 con 0x7fec60003c30 2024-09-21T07:33:26.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.842+0000 7fec7b200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c0d46b0 con 0x7fec7c157a50 2024-09-21T07:33:26.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.842+0000 7fec7b200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fec7c161f90 con 0x7fec7c1608d0 2024-09-21T07:33:26.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.842+0000 7fec79e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec60003c30 0x7fec7c0d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:26.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.842+0000 7fec79e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec60003c30 0x7fec7c0d0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60306/0 (socket says 172.21.15.39:60306) 2024-09-21T07:33:26.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.842+0000 7fec79e00640 1 -- 172.21.15.39:0/3643264798 learned_addr learned my addr 172.21.15.39:0/3643264798 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:26.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.842+0000 7fec79400640 1 --2- 172.21.15.39:0/3643264798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec7c157a50 0x7fec7c1571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:26.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.842+0000 7fec7a800640 1 --2- 172.21.15.39:0/3643264798 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec7c1608d0 0x7fec7c160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:26.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.842+0000 7fec79e00640 1 -- 172.21.15.39:0/3643264798 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec7c1608d0 msgr2=0x7fec7c160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:26.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.842+0000 7fec79e00640 1 --2- 172.21.15.39:0/3643264798 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec7c1608d0 0x7fec7c160ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:26.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.842+0000 7fec79e00640 1 -- 172.21.15.39:0/3643264798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec7c157a50 msgr2=0x7fec7c1571b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:26.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.842+0000 7fec79e00640 1 --2- 172.21.15.39:0/3643264798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec7c157a50 0x7fec7c1571b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:26.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.842+0000 7fec79e00640 1 -- 172.21.15.39:0/3643264798 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec7c161220 con 0x7fec60003c30 2024-09-21T07:33:26.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.842+0000 7fec79400640 1 --2- 172.21.15.39:0/3643264798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec7c157a50 0x7fec7c1571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:33:26.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.842+0000 7fec7a800640 1 --2- 172.21.15.39:0/3643264798 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec7c1608d0 0x7fec7c160ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:33:26.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.843+0000 7fec79e00640 1 --2- 172.21.15.39:0/3643264798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec60003c30 0x7fec7c0d0cf0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fec6c000c00 tx=0x7fec6c001220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:26.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.843+0000 7fec72000640 1 -- 172.21.15.39:0/3643264798 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6c003810 con 0x7fec60003c30 2024-09-21T07:33:26.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.843+0000 7fec72000640 1 -- 172.21.15.39:0/3643264798 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec6c003ac0 con 0x7fec60003c30 2024-09-21T07:33:26.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.843+0000 7fec72000640 1 -- 172.21.15.39:0/3643264798 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6c02e460 con 0x7fec60003c30 2024-09-21T07:33:26.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.843+0000 7fec7b200640 1 -- 172.21.15.39:0/3643264798 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec7c0d1230 con 0x7fec60003c30 2024-09-21T07:33:26.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.843+0000 7fec7b200640 1 -- 172.21.15.39:0/3643264798 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec7c1f5b60 con 0x7fec60003c30 2024-09-21T07:33:26.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.844+0000 7fec7b200640 1 -- 172.21.15.39:0/3643264798 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec7c161850 con 0x7fec60003c30 2024-09-21T07:33:26.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.846+0000 7fec72000640 1 -- 172.21.15.39:0/3643264798 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fec6c035020 con 0x7fec60003c30 2024-09-21T07:33:26.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.847+0000 7fec72000640 1 --2- 172.21.15.39:0/3643264798 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fec5c078b90 0x7fec5c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:26.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.847+0000 7fec72000640 1 -- 172.21.15.39:0/3643264798 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fec6c0bb100 con 0x7fec60003c30 2024-09-21T07:33:26.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.848+0000 7fec79400640 1 --2- 172.21.15.39:0/3643264798 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fec5c078b90 0x7fec5c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:26.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.848+0000 7fec79400640 1 --2- 172.21.15.39:0/3643264798 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fec5c078b90 0x7fec5c07b0a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fec7c0cb170 tx=0x7fec68005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:26.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:26.852+0000 7fec72000640 1 -- 172.21.15.39:0/3643264798 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fec7c161850 con 0x7fec60003c30 2024-09-21T07:33:27.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.092+0000 7fec7b200640 1 -- 172.21.15.39:0/3643264798 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fec7c0b23c0 con 0x7fec60003c30 2024-09-21T07:33:27.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.092+0000 7fec72000640 1 -- 172.21.15.39:0/3643264798 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fec6c047030 con 0x7fec60003c30 2024-09-21T07:33:27.094 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:33:27.094 INFO:teuthology.orchestra.run.smithi039.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T07:33:27.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.094+0000 7fec7b200640 1 -- 172.21.15.39:0/3643264798 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fec5c078b90 msgr2=0x7fec5c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:27.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.094+0000 7fec7b200640 1 --2- 172.21.15.39:0/3643264798 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fec5c078b90 0x7fec5c07b0a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fec7c0cb170 tx=0x7fec68005f90 comp rx=0 tx=0).stop 2024-09-21T07:33:27.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.094+0000 7fec7b200640 1 -- 172.21.15.39:0/3643264798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec60003c30 msgr2=0x7fec7c0d0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:27.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.094+0000 7fec7b200640 1 --2- 172.21.15.39:0/3643264798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec60003c30 0x7fec7c0d0cf0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fec6c000c00 tx=0x7fec6c001220 comp rx=0 tx=0).stop 2024-09-21T07:33:27.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.095+0000 7fec7b200640 1 -- 172.21.15.39:0/3643264798 shutdown_connections 2024-09-21T07:33:27.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.095+0000 7fec7b200640 1 --2- 172.21.15.39:0/3643264798 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fec5c078b90 0x7fec5c07b0a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:27.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.095+0000 7fec7b200640 1 --2- 172.21.15.39:0/3643264798 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec7c1608d0 0x7fec7c160ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:27.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.095+0000 7fec7b200640 1 --2- 172.21.15.39:0/3643264798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec7c157a50 0x7fec7c1571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:27.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.095+0000 7fec7b200640 1 --2- 172.21.15.39:0/3643264798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec60003c30 0x7fec7c0d0cf0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:27.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.095+0000 7fec7b200640 1 -- 172.21.15.39:0/3643264798 >> 172.21.15.39:0/3643264798 conn(0x7fec7c08bfc0 msgr2=0x7fec7c0b5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:27.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.095+0000 7fec7b200640 1 -- 172.21.15.39:0/3643264798 shutdown_connections 2024-09-21T07:33:27.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.095+0000 7fec7b200640 1 -- 172.21.15.39:0/3643264798 wait complete. 2024-09-21T07:33:27.122 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-21T07:33:27.122 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:33:27.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.345+0000 7f9786800640 1 -- 172.21.15.39:0/2268670529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9780070440 msgr2=0x7f9780074b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:27.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.345+0000 7f9786800640 1 --2- 172.21.15.39:0/2268670529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9780070440 0x7f9780074b80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9778005fd0 tx=0x7f977802aa90 comp rx=0 tx=0).stop 2024-09-21T07:33:27.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.345+0000 7f9786800640 1 -- 172.21.15.39:0/2268670529 shutdown_connections 2024-09-21T07:33:27.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.345+0000 7f9786800640 1 --2- 172.21.15.39:0/2268670529 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9780070030 0x7f9780074640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:27.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.345+0000 7f9786800640 1 --2- 172.21.15.39:0/2268670529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9780070440 0x7f9780074b80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:27.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.345+0000 7f9786800640 1 -- 172.21.15.39:0/2268670529 >> 172.21.15.39:0/2268670529 conn(0x7f97800b1cb0 msgr2=0x7f97800b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:27.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.345+0000 7f9786800640 1 -- 172.21.15.39:0/2268670529 shutdown_connections 2024-09-21T07:33:27.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.345+0000 7f9786800640 1 -- 172.21.15.39:0/2268670529 wait complete. 2024-09-21T07:33:27.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.347+0000 7f9786800640 1 Processor -- start 2024-09-21T07:33:27.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.347+0000 7f9786800640 1 -- start start 2024-09-21T07:33:27.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.349+0000 7f9786800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9780070030 0x7f978009b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:27.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.349+0000 7f9786800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9780070440 0x7f978014e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:27.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.349+0000 7f9786800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9780073170 0x7f97801e94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:27.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.349+0000 7f9786800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f97800b3a40 con 0x7f9780070440 2024-09-21T07:33:27.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.349+0000 7f9786800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f97800b38c0 con 0x7f9780073170 2024-09-21T07:33:27.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.349+0000 7f9786800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f97800b3bc0 con 0x7f9780070030 2024-09-21T07:33:27.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.349+0000 7f977ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9780070440 0x7f978014e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:27.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.349+0000 7f977ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9780070440 0x7f978014e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60334/0 (socket says 172.21.15.39:60334) 2024-09-21T07:33:27.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.349+0000 7f977ea00640 1 -- 172.21.15.39:0/13018397 learned_addr learned my addr 172.21.15.39:0/13018397 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:27.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.349+0000 7f977fe00640 1 --2- 172.21.15.39:0/13018397 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9780073170 0x7f97801e94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:27.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.349+0000 7f977f400640 1 --2- 172.21.15.39:0/13018397 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9780070030 0x7f978009b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:27.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.350+0000 7f977ea00640 1 -- 172.21.15.39:0/13018397 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9780070030 msgr2=0x7f978009b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:27.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.350+0000 7f977ea00640 1 --2- 172.21.15.39:0/13018397 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9780070030 0x7f978009b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:27.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.350+0000 7f977ea00640 1 -- 172.21.15.39:0/13018397 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9780073170 msgr2=0x7f97801e94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:27.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.350+0000 7f977ea00640 1 --2- 172.21.15.39:0/13018397 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9780073170 0x7f97801e94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:27.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.350+0000 7f977ea00640 1 -- 172.21.15.39:0/13018397 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f978014e9e0 con 0x7f9780070440 2024-09-21T07:33:27.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.350+0000 7f977ea00640 1 --2- 172.21.15.39:0/13018397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9780070440 0x7f978014e4a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f976c00d720 tx=0x7f976c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:27.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.350+0000 7f9777400640 1 -- 172.21.15.39:0/13018397 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c009c30 con 0x7f9780070440 2024-09-21T07:33:27.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.350+0000 7f9777400640 1 -- 172.21.15.39:0/13018397 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f976c00de30 con 0x7f9780070440 2024-09-21T07:33:27.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.350+0000 7f9777400640 1 -- 172.21.15.39:0/13018397 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c004290 con 0x7f9780070440 2024-09-21T07:33:27.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.350+0000 7f9786800640 1 -- 172.21.15.39:0/13018397 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f978009dbc0 con 0x7f9780070440 2024-09-21T07:33:27.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.350+0000 7f9786800640 1 -- 172.21.15.39:0/13018397 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f978009e180 con 0x7f9780070440 2024-09-21T07:33:27.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.352+0000 7f9786800640 1 -- 172.21.15.39:0/13018397 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97800b48a0 con 0x7f9780070440 2024-09-21T07:33:27.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.353+0000 7f9777400640 1 -- 172.21.15.39:0/13018397 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f976c0035f0 con 0x7f9780070440 2024-09-21T07:33:27.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.354+0000 7f9777400640 1 --2- 172.21.15.39:0/13018397 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9760078b90 0x7f976007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:27.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.354+0000 7f9777400640 1 -- 172.21.15.39:0/13018397 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f976c09aea0 con 0x7f9780070440 2024-09-21T07:33:27.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.354+0000 7f977f400640 1 --2- 172.21.15.39:0/13018397 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9760078b90 0x7f976007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:27.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.355+0000 7f977f400640 1 --2- 172.21.15.39:0/13018397 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9760078b90 0x7f976007b0a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9770001720 tx=0x7f9770009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:27.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.358+0000 7f9777400640 1 -- 172.21.15.39:0/13018397 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f976c017020 con 0x7f9780070440 2024-09-21T07:33:27.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.593+0000 7f9786800640 1 -- 172.21.15.39:0/13018397 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97800c3860 con 0x7f9780070440 2024-09-21T07:33:27.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.594+0000 7f9777400640 1 -- 172.21.15.39:0/13018397 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5220 (secure 0 0 0) 0x7f976c062c90 con 0x7f9780070440 2024-09-21T07:33:27.596 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:33:27.596 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":8,"btime":"2024-09-21T07:33:18:179932+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4325,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1939850940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1939850940},{"type":"v1","addr":"172.21.15.39:6835","nonce":1939850940}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4328,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3712021428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3712021428},{"type":"v1","addr":"172.21.15.148:6836","nonce":3712021428}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4331,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3708353709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3708353709},{"type":"v1","addr":"172.21.15.148:6837","nonce":3708353709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:33:14.674552+0000","modified":"2024-09-21T07:33:17.688290+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4378},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4378":{"gid":4378,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6833/414851924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":414851924},{"type":"v1","addr":"172.21.15.39:6833","nonce":414851924}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4378,"qdb_cluster":[4378]},"id":1}]} 2024-09-21T07:33:27.596 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 8 2024-09-21T07:33:27.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.596+0000 7f9786800640 1 -- 172.21.15.39:0/13018397 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9760078b90 msgr2=0x7f976007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:27.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.596+0000 7f9786800640 1 --2- 172.21.15.39:0/13018397 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9760078b90 0x7f976007b0a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9770001720 tx=0x7f9770009040 comp rx=0 tx=0).stop 2024-09-21T07:33:27.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.596+0000 7f9786800640 1 -- 172.21.15.39:0/13018397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9780070440 msgr2=0x7f978014e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:27.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.596+0000 7f9786800640 1 --2- 172.21.15.39:0/13018397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9780070440 0x7f978014e4a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f976c00d720 tx=0x7f976c00dbe0 comp rx=0 tx=0).stop 2024-09-21T07:33:27.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.596+0000 7f9786800640 1 -- 172.21.15.39:0/13018397 shutdown_connections 2024-09-21T07:33:27.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.596+0000 7f9786800640 1 --2- 172.21.15.39:0/13018397 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9760078b90 0x7f976007b0a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:27.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.596+0000 7f9786800640 1 --2- 172.21.15.39:0/13018397 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9780073170 0x7f97801e94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:27.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.596+0000 7f9786800640 1 --2- 172.21.15.39:0/13018397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9780070440 0x7f978014e4a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:27.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.596+0000 7f9786800640 1 --2- 172.21.15.39:0/13018397 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9780070030 0x7f978009b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:27.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.596+0000 7f9786800640 1 -- 172.21.15.39:0/13018397 >> 172.21.15.39:0/13018397 conn(0x7f97800b1cb0 msgr2=0x7f978015f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:27.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.596+0000 7f9786800640 1 -- 172.21.15.39:0/13018397 shutdown_connections 2024-09-21T07:33:27.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.596+0000 7f9786800640 1 -- 172.21.15.39:0/13018397 wait complete. 2024-09-21T07:33:27.619 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:33:27.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.843+0000 7fb396400640 1 -- 172.21.15.39:0/2947931028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb390070440 msgr2=0x7fb390074b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:27.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.843+0000 7fb396400640 1 --2- 172.21.15.39:0/2947931028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb390070440 0x7fb390074b80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb388005fd0 tx=0x7fb38802aa90 comp rx=0 tx=0).stop 2024-09-21T07:33:27.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.843+0000 7fb396400640 1 -- 172.21.15.39:0/2947931028 shutdown_connections 2024-09-21T07:33:27.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.843+0000 7fb396400640 1 --2- 172.21.15.39:0/2947931028 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb390070030 0x7fb390074640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:27.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.843+0000 7fb396400640 1 --2- 172.21.15.39:0/2947931028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb390070440 0x7fb390074b80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:27.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.843+0000 7fb396400640 1 -- 172.21.15.39:0/2947931028 >> 172.21.15.39:0/2947931028 conn(0x7fb3900b1cb0 msgr2=0x7fb3900b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:27.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.843+0000 7fb396400640 1 -- 172.21.15.39:0/2947931028 shutdown_connections 2024-09-21T07:33:27.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.843+0000 7fb396400640 1 -- 172.21.15.39:0/2947931028 wait complete. 2024-09-21T07:33:27.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.845+0000 7fb396400640 1 Processor -- start 2024-09-21T07:33:27.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.845+0000 7fb396400640 1 -- start start 2024-09-21T07:33:27.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.847+0000 7fb396400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb390070030 0x7fb39014e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:27.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.847+0000 7fb396400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb390070440 0x7fb3900cea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:27.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.847+0000 7fb396400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb390073170 0x7fb39015ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:27.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.847+0000 7fb396400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb3900b3ae0 con 0x7fb390073170 2024-09-21T07:33:27.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.847+0000 7fb396400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb3900b3960 con 0x7fb390070440 2024-09-21T07:33:27.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.847+0000 7fb396400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb3900b3c60 con 0x7fb390070030 2024-09-21T07:33:27.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.847+0000 7fb38fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb390073170 0x7fb39015ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:27.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.847+0000 7fb38fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb390073170 0x7fb39015ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60364/0 (socket says 172.21.15.39:60364) 2024-09-21T07:33:27.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.847+0000 7fb38fe00640 1 -- 172.21.15.39:0/2476932663 learned_addr learned my addr 172.21.15.39:0/2476932663 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:27.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.847+0000 7fb387e00640 1 --2- 172.21.15.39:0/2476932663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb390070440 0x7fb3900cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:27.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.847+0000 7fb38f400640 1 --2- 172.21.15.39:0/2476932663 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb390070030 0x7fb39014e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:27.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.848+0000 7fb38fe00640 1 -- 172.21.15.39:0/2476932663 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb390070030 msgr2=0x7fb39014e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:27.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.848+0000 7fb38fe00640 1 --2- 172.21.15.39:0/2476932663 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb390070030 0x7fb39014e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:27.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.848+0000 7fb38fe00640 1 -- 172.21.15.39:0/2476932663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb390070440 msgr2=0x7fb3900cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:27.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.848+0000 7fb38fe00640 1 --2- 172.21.15.39:0/2476932663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb390070440 0x7fb3900cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:27.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.848+0000 7fb38fe00640 1 -- 172.21.15.39:0/2476932663 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3900cef70 con 0x7fb390073170 2024-09-21T07:33:27.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.848+0000 7fb38fe00640 1 --2- 172.21.15.39:0/2476932663 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb390073170 0x7fb39015ee90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb388006010 tx=0x7fb388003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:27.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.848+0000 7fb387400640 1 -- 172.21.15.39:0/2476932663 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb388044070 con 0x7fb390073170 2024-09-21T07:33:27.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.848+0000 7fb387400640 1 -- 172.21.15.39:0/2476932663 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb388004040 con 0x7fb390073170 2024-09-21T07:33:27.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.848+0000 7fb387400640 1 -- 172.21.15.39:0/2476932663 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb388004430 con 0x7fb390073170 2024-09-21T07:33:27.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.848+0000 7fb396400640 1 -- 172.21.15.39:0/2476932663 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3900cfee0 con 0x7fb390073170 2024-09-21T07:33:27.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.848+0000 7fb396400640 1 -- 172.21.15.39:0/2476932663 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3900d04a0 con 0x7fb390073170 2024-09-21T07:33:27.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.849+0000 7fb396400640 1 -- 172.21.15.39:0/2476932663 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3900cfc50 con 0x7fb390073170 2024-09-21T07:33:27.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.851+0000 7fb387400640 1 -- 172.21.15.39:0/2476932663 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb388047070 con 0x7fb390073170 2024-09-21T07:33:27.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.852+0000 7fb387400640 1 --2- 172.21.15.39:0/2476932663 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb36c078bc0 0x7fb36c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:27.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.852+0000 7fb387400640 1 -- 172.21.15.39:0/2476932663 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb3880bba90 con 0x7fb390073170 2024-09-21T07:33:27.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.852+0000 7fb38f400640 1 --2- 172.21.15.39:0/2476932663 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb36c078bc0 0x7fb36c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:27.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.853+0000 7fb38f400640 1 --2- 172.21.15.39:0/2476932663 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb36c078bc0 0x7fb36c07b0d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb380001720 tx=0x7fb380009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:27.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:27.856+0000 7fb387400640 1 -- 172.21.15.39:0/2476932663 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb388032030 con 0x7fb390073170 2024-09-21T07:33:28.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.093+0000 7fb396400640 1 -- 172.21.15.39:0/2476932663 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb3900b4180 con 0x7fb390073170 2024-09-21T07:33:28.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.094+0000 7fb387400640 1 -- 172.21.15.39:0/2476932663 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5220 (secure 0 0 0) 0x7fb388083910 con 0x7fb390073170 2024-09-21T07:33:28.096 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:33:28.096 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":8,"btime":"2024-09-21T07:33:18:179932+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4325,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1939850940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1939850940},{"type":"v1","addr":"172.21.15.39:6835","nonce":1939850940}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4328,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3712021428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3712021428},{"type":"v1","addr":"172.21.15.148:6836","nonce":3712021428}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4331,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3708353709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3708353709},{"type":"v1","addr":"172.21.15.148:6837","nonce":3708353709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:33:14.674552+0000","modified":"2024-09-21T07:33:17.688290+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4378},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4378":{"gid":4378,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6833/414851924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":414851924},{"type":"v1","addr":"172.21.15.39:6833","nonce":414851924}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4378,"qdb_cluster":[4378]},"id":1}]} 2024-09-21T07:33:28.096 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 8 2024-09-21T07:33:28.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.096+0000 7fb396400640 1 -- 172.21.15.39:0/2476932663 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb36c078bc0 msgr2=0x7fb36c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:28.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.096+0000 7fb396400640 1 --2- 172.21.15.39:0/2476932663 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb36c078bc0 0x7fb36c07b0d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb380001720 tx=0x7fb380009040 comp rx=0 tx=0).stop 2024-09-21T07:33:28.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.096+0000 7fb396400640 1 -- 172.21.15.39:0/2476932663 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb390073170 msgr2=0x7fb39015ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:28.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.096+0000 7fb396400640 1 --2- 172.21.15.39:0/2476932663 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb390073170 0x7fb39015ee90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb388006010 tx=0x7fb388003ce0 comp rx=0 tx=0).stop 2024-09-21T07:33:28.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.096+0000 7fb396400640 1 -- 172.21.15.39:0/2476932663 shutdown_connections 2024-09-21T07:33:28.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.096+0000 7fb396400640 1 --2- 172.21.15.39:0/2476932663 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb36c078bc0 0x7fb36c07b0d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:28.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.096+0000 7fb396400640 1 --2- 172.21.15.39:0/2476932663 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb390073170 0x7fb39015ee90 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:28.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.096+0000 7fb396400640 1 --2- 172.21.15.39:0/2476932663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb390070440 0x7fb3900cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:28.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.096+0000 7fb396400640 1 --2- 172.21.15.39:0/2476932663 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb390070030 0x7fb39014e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:28.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.096+0000 7fb396400640 1 -- 172.21.15.39:0/2476932663 >> 172.21.15.39:0/2476932663 conn(0x7fb3900b1cb0 msgr2=0x7fb3900ad600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:28.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.096+0000 7fb396400640 1 -- 172.21.15.39:0/2476932663 shutdown_connections 2024-09-21T07:33:28.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.096+0000 7fb396400640 1 -- 172.21.15.39:0/2476932663 wait complete. 2024-09-21T07:33:28.123 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:33:28.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.347+0000 7fb1ef200640 1 -- 172.21.15.39:0/2960725571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1d4003a30 msgr2=0x7fb1d4003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:28.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.347+0000 7fb1ef200640 1 --2- 172.21.15.39:0/2960725571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1d4003a30 0x7fb1d4003ea0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb1e000c310 tx=0x7fb1d4005830 comp rx=0 tx=0).stop 2024-09-21T07:33:28.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.347+0000 7fb1ee800640 1 -- 172.21.15.39:0/2960725571 reap_dead start 2024-09-21T07:33:28.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.347+0000 7fb1ef200640 1 -- 172.21.15.39:0/2960725571 shutdown_connections 2024-09-21T07:33:28.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.347+0000 7fb1ef200640 1 -- 172.21.15.39:0/2960725571 >> 172.21.15.39:0/2960725571 conn(0x7fb1f009d6a0 msgr2=0x7fb1f00d4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:28.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.348+0000 7fb1ef200640 1 -- 172.21.15.39:0/2960725571 shutdown_connections 2024-09-21T07:33:28.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.348+0000 7fb1ef200640 1 -- 172.21.15.39:0/2960725571 wait complete. 2024-09-21T07:33:28.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.350+0000 7fb1ef200640 1 Processor -- start 2024-09-21T07:33:28.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.350+0000 7fb1ef200640 1 -- start start 2024-09-21T07:33:28.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.351+0000 7fb1ef200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1d4003a30 0x7fb1f01525f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:28.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.351+0000 7fb1ef200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f00a1710 0x7fb1f0151cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:28.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.351+0000 7fb1ef200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb1f00cd800 0x7fb1f00cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:28.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.351+0000 7fb1ef200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f00d2a50 con 0x7fb1f00a1710 2024-09-21T07:33:28.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.351+0000 7fb1ef200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f00d28d0 con 0x7fb1d4003a30 2024-09-21T07:33:28.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.351+0000 7fb1ef200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb1f00b7f80 con 0x7fb1f00cd800 2024-09-21T07:33:28.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.352+0000 7fb1ed400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f00a1710 0x7fb1f0151cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:28.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.352+0000 7fb1ed400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f00a1710 0x7fb1f0151cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60384/0 (socket says 172.21.15.39:60384) 2024-09-21T07:33:28.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.352+0000 7fb1ed400640 1 -- 172.21.15.39:0/1492222360 learned_addr learned my addr 172.21.15.39:0/1492222360 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:28.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.352+0000 7fb1ede00640 1 --2- 172.21.15.39:0/1492222360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1d4003a30 0x7fb1f01525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:28.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.352+0000 7fb1ee800640 1 --2- 172.21.15.39:0/1492222360 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb1f00cd800 0x7fb1f00cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:28.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.352+0000 7fb1ed400640 1 -- 172.21.15.39:0/1492222360 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb1f00cd800 msgr2=0x7fb1f00cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:28.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.352+0000 7fb1ed400640 1 --2- 172.21.15.39:0/1492222360 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb1f00cd800 0x7fb1f00cdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:28.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.352+0000 7fb1ed400640 1 -- 172.21.15.39:0/1492222360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1d4003a30 msgr2=0x7fb1f01525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:28.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.352+0000 7fb1ed400640 1 --2- 172.21.15.39:0/1492222360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1d4003a30 0x7fb1f01525f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:28.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.352+0000 7fb1ed400640 1 -- 172.21.15.39:0/1492222360 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f00cc760 con 0x7fb1f00a1710 2024-09-21T07:33:28.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.352+0000 7fb1ede00640 1 --2- 172.21.15.39:0/1492222360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1d4003a30 0x7fb1f01525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:33:28.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.352+0000 7fb1ed400640 1 --2- 172.21.15.39:0/1492222360 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f00a1710 0x7fb1f0151cf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc00fa30 tx=0x7fb1dc00fef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:28.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.353+0000 7fb1e6000640 1 -- 172.21.15.39:0/1492222360 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc00bd70 con 0x7fb1f00a1710 2024-09-21T07:33:28.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.353+0000 7fb1e6000640 1 -- 172.21.15.39:0/1492222360 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1dc011440 con 0x7fb1f00a1710 2024-09-21T07:33:28.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.353+0000 7fb1e6000640 1 -- 172.21.15.39:0/1492222360 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc011870 con 0x7fb1f00a1710 2024-09-21T07:33:28.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.353+0000 7fb1ef200640 1 -- 172.21.15.39:0/1492222360 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1f00cca40 con 0x7fb1f00a1710 2024-09-21T07:33:28.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.353+0000 7fb1ef200640 1 -- 172.21.15.39:0/1492222360 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1f00cd000 con 0x7fb1f00a1710 2024-09-21T07:33:28.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.354+0000 7fb1ef200640 1 -- 172.21.15.39:0/1492222360 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1f00b7d00 con 0x7fb1f00a1710 2024-09-21T07:33:28.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.355+0000 7fb1e6000640 1 -- 172.21.15.39:0/1492222360 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb1dc019020 con 0x7fb1f00a1710 2024-09-21T07:33:28.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.356+0000 7fb1e6000640 1 --2- 172.21.15.39:0/1492222360 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb1d0078bc0 0x7fb1d007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:28.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.356+0000 7fb1e6000640 1 -- 172.21.15.39:0/1492222360 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb1dc09b770 con 0x7fb1f00a1710 2024-09-21T07:33:28.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.356+0000 7fb1ede00640 1 --2- 172.21.15.39:0/1492222360 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb1d0078bc0 0x7fb1d007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:28.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.357+0000 7fb1ede00640 1 --2- 172.21.15.39:0/1492222360 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb1d0078bc0 0x7fb1d007b0d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0033000 tx=0x7fb1e00019f0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:28.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.360+0000 7fb1e6000640 1 -- 172.21.15.39:0/1492222360 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb1f00b7d00 con 0x7fb1f00a1710 2024-09-21T07:33:28.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.559+0000 7fb1ef200640 1 -- 172.21.15.39:0/1492222360 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb1f00b84e0 con 0x7fb1f00a1710 2024-09-21T07:33:28.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.561+0000 7fb1e6000640 1 -- 172.21.15.39:0/1492222360 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16665 (secure 0 0 0) 0x7fb1dc063bb0 con 0x7fb1f00a1710 2024-09-21T07:33:28.563 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:33:28.563 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":21,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:33:16.159267+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:33:12.854113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:33:13.622611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:33:28.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.563+0000 7fb1ef200640 1 -- 172.21.15.39:0/1492222360 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb1d0078bc0 msgr2=0x7fb1d007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:28.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.563+0000 7fb1ef200640 1 --2- 172.21.15.39:0/1492222360 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb1d0078bc0 0x7fb1d007b0d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0033000 tx=0x7fb1e00019f0 comp rx=0 tx=0).stop 2024-09-21T07:33:28.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.563+0000 7fb1ef200640 1 -- 172.21.15.39:0/1492222360 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f00a1710 msgr2=0x7fb1f0151cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:28.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.563+0000 7fb1ef200640 1 --2- 172.21.15.39:0/1492222360 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f00a1710 0x7fb1f0151cf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc00fa30 tx=0x7fb1dc00fef0 comp rx=0 tx=0).stop 2024-09-21T07:33:28.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.563+0000 7fb1ef200640 1 -- 172.21.15.39:0/1492222360 shutdown_connections 2024-09-21T07:33:28.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.563+0000 7fb1ef200640 1 --2- 172.21.15.39:0/1492222360 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb1d0078bc0 0x7fb1d007b0d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:28.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.563+0000 7fb1ef200640 1 --2- 172.21.15.39:0/1492222360 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb1f00cd800 0x7fb1f00cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:28.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.563+0000 7fb1ef200640 1 --2- 172.21.15.39:0/1492222360 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f00a1710 0x7fb1f0151cf0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:28.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.563+0000 7fb1ef200640 1 --2- 172.21.15.39:0/1492222360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1d4003a30 0x7fb1f01525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:28.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.563+0000 7fb1ef200640 1 -- 172.21.15.39:0/1492222360 >> 172.21.15.39:0/1492222360 conn(0x7fb1f009d6a0 msgr2=0x7fb1f006f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:28.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.564+0000 7fb1ef200640 1 -- 172.21.15.39:0/1492222360 shutdown_connections 2024-09-21T07:33:28.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.564+0000 7fb1ef200640 1 -- 172.21.15.39:0/1492222360 wait complete. 2024-09-21T07:33:28.588 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:33:28.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.812+0000 7f497f800640 1 -- 172.21.15.39:0/2265359707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49780af3c0 msgr2=0x7f49780b40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:28.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.812+0000 7f497f800640 1 --2- 172.21.15.39:0/2265359707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49780af3c0 0x7f49780b40b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4964009830 tx=0x7f496402a5c0 comp rx=0 tx=0).stop 2024-09-21T07:33:28.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.812+0000 7f497f800640 1 -- 172.21.15.39:0/2265359707 shutdown_connections 2024-09-21T07:33:28.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.812+0000 7f497f800640 1 --2- 172.21.15.39:0/2265359707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49780af3c0 0x7f49780b40b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:28.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.812+0000 7f497f800640 1 -- 172.21.15.39:0/2265359707 >> 172.21.15.39:0/2265359707 conn(0x7f497806d7d0 msgr2=0x7f49780b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:28.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.812+0000 7f497f800640 1 -- 172.21.15.39:0/2265359707 shutdown_connections 2024-09-21T07:33:28.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.812+0000 7f497f800640 1 -- 172.21.15.39:0/2265359707 wait complete. 2024-09-21T07:33:28.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.815+0000 7f497f800640 1 Processor -- start 2024-09-21T07:33:28.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.815+0000 7f497f800640 1 -- start start 2024-09-21T07:33:28.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.816+0000 7f497f800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49780af3c0 0x7f49781e77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:28.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.816+0000 7f497f800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49780cc1e0 0x7f4978145980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:28.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.816+0000 7f497f800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49780ccb50 0x7f49781f3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:28.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.816+0000 7f497f800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4978153520 con 0x7f49780ccb50 2024-09-21T07:33:28.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.816+0000 7f497f800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f49781533a0 con 0x7f49780cc1e0 2024-09-21T07:33:28.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.816+0000 7f497f800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f49781536a0 con 0x7f49780af3c0 2024-09-21T07:33:28.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.816+0000 7f497d800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49780ccb50 0x7f49781f3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:28.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.816+0000 7f497d800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49780ccb50 0x7f49781f3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60390/0 (socket says 172.21.15.39:60390) 2024-09-21T07:33:28.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.816+0000 7f497d800640 1 -- 172.21.15.39:0/3687028760 learned_addr learned my addr 172.21.15.39:0/3687028760 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:28.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.817+0000 7f497ce00640 1 --2- 172.21.15.39:0/3687028760 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49780af3c0 0x7f49781e77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:28.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.817+0000 7f4977e00640 1 --2- 172.21.15.39:0/3687028760 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49780cc1e0 0x7f4978145980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:28.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.817+0000 7f497d800640 1 -- 172.21.15.39:0/3687028760 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49780af3c0 msgr2=0x7f49781e77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:28.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.817+0000 7f497d800640 1 --2- 172.21.15.39:0/3687028760 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49780af3c0 0x7f49781e77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:28.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.817+0000 7f497d800640 1 -- 172.21.15.39:0/3687028760 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49780cc1e0 msgr2=0x7f4978145980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:28.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.817+0000 7f497d800640 1 --2- 172.21.15.39:0/3687028760 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49780cc1e0 0x7f4978145980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:28.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.817+0000 7f497d800640 1 -- 172.21.15.39:0/3687028760 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49781e7ce0 con 0x7f49780ccb50 2024-09-21T07:33:28.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.817+0000 7f497d800640 1 --2- 172.21.15.39:0/3687028760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49780ccb50 0x7f49781f3b00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f497000cb10 tx=0x7f497000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:28.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.817+0000 7f4974c00640 1 -- 172.21.15.39:0/3687028760 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4970016020 con 0x7f49780ccb50 2024-09-21T07:33:28.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.817+0000 7f4974c00640 1 -- 172.21.15.39:0/3687028760 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f497000ad80 con 0x7f49780ccb50 2024-09-21T07:33:28.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.817+0000 7f4974c00640 1 -- 172.21.15.39:0/3687028760 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4970012680 con 0x7f49780ccb50 2024-09-21T07:33:28.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.817+0000 7f497f800640 1 -- 172.21.15.39:0/3687028760 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f497809c750 con 0x7f49780ccb50 2024-09-21T07:33:28.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.818+0000 7f497f800640 1 -- 172.21.15.39:0/3687028760 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f497809cd10 con 0x7f49780ccb50 2024-09-21T07:33:28.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.819+0000 7f497f800640 1 -- 172.21.15.39:0/3687028760 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4978153b50 con 0x7f49780ccb50 2024-09-21T07:33:28.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.820+0000 7f4974c00640 1 -- 172.21.15.39:0/3687028760 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4970019020 con 0x7f49780ccb50 2024-09-21T07:33:28.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.821+0000 7f4974c00640 1 --2- 172.21.15.39:0/3687028760 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4958078b90 0x7f495807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:28.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.821+0000 7f4974c00640 1 -- 172.21.15.39:0/3687028760 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f497009b300 con 0x7f49780ccb50 2024-09-21T07:33:28.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.821+0000 7f497ce00640 1 --2- 172.21.15.39:0/3687028760 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4958078b90 0x7f495807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:28.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.822+0000 7f497ce00640 1 --2- 172.21.15.39:0/3687028760 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4958078b90 0x7f495807b0a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f496c001120 tx=0x7f496c009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:28.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:28.825+0000 7f4974c00640 1 -- 172.21.15.39:0/3687028760 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4970020020 con 0x7f49780ccb50 2024-09-21T07:33:29.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.060+0000 7f497f800640 1 -- 172.21.15.39:0/3687028760 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f497815ba30 con 0x7f49780ccb50 2024-09-21T07:33:29.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.060+0000 7f4974c00640 1 -- 172.21.15.39:0/3687028760 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5220 (secure 0 0 0) 0x7f4970063050 con 0x7f49780ccb50 2024-09-21T07:33:29.062 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:33:29.063 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":8,"btime":"2024-09-21T07:33:18:179932+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4325,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1939850940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1939850940},{"type":"v1","addr":"172.21.15.39:6835","nonce":1939850940}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4328,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3712021428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3712021428},{"type":"v1","addr":"172.21.15.148:6836","nonce":3712021428}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4331,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3708353709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3708353709},{"type":"v1","addr":"172.21.15.148:6837","nonce":3708353709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:33:14.674552+0000","modified":"2024-09-21T07:33:17.688290+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4378},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4378":{"gid":4378,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6833/414851924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":414851924},{"type":"v1","addr":"172.21.15.39:6833","nonce":414851924}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4378,"qdb_cluster":[4378]},"id":1}]} 2024-09-21T07:33:29.063 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 8 2024-09-21T07:33:29.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.062+0000 7f497f800640 1 -- 172.21.15.39:0/3687028760 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4958078b90 msgr2=0x7f495807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:29.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.063+0000 7f497f800640 1 --2- 172.21.15.39:0/3687028760 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4958078b90 0x7f495807b0a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f496c001120 tx=0x7f496c009040 comp rx=0 tx=0).stop 2024-09-21T07:33:29.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.063+0000 7f497f800640 1 -- 172.21.15.39:0/3687028760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49780ccb50 msgr2=0x7f49781f3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:29.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.063+0000 7f497f800640 1 --2- 172.21.15.39:0/3687028760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49780ccb50 0x7f49781f3b00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f497000cb10 tx=0x7f497000a060 comp rx=0 tx=0).stop 2024-09-21T07:33:29.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.063+0000 7f497f800640 1 -- 172.21.15.39:0/3687028760 shutdown_connections 2024-09-21T07:33:29.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.063+0000 7f497f800640 1 --2- 172.21.15.39:0/3687028760 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4958078b90 0x7f495807b0a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:29.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.063+0000 7f497f800640 1 --2- 172.21.15.39:0/3687028760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49780ccb50 0x7f49781f3b00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:29.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.063+0000 7f497f800640 1 --2- 172.21.15.39:0/3687028760 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49780cc1e0 0x7f4978145980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:29.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.063+0000 7f497f800640 1 --2- 172.21.15.39:0/3687028760 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49780af3c0 0x7f49781e77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:29.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.063+0000 7f497f800640 1 -- 172.21.15.39:0/3687028760 >> 172.21.15.39:0/3687028760 conn(0x7f497806d7d0 msgr2=0x7f49780d0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:29.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.063+0000 7f497f800640 1 -- 172.21.15.39:0/3687028760 shutdown_connections 2024-09-21T07:33:29.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.063+0000 7f497f800640 1 -- 172.21.15.39:0/3687028760 wait complete. 2024-09-21T07:33:29.488 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T07:33:14.674552+0000', 'modified': '2024-09-21T07:33:17.688290+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4378}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4378': {'gid': 4378, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.39:6833/414851924', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 414851924}, {'type': 'v1', 'addr': '172.21.15.39:6833', 'nonce': 414851924}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4378, 'qdb_cluster': [4378]} 2024-09-21T07:33:29.489 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T07:33:29.489 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T07:33:29.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.715+0000 7f26dd800640 1 -- 172.21.15.39:0/3657673087 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26d8162440 msgr2=0x7f26d8162850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:29.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.715+0000 7f26dd800640 1 --2- 172.21.15.39:0/3657673087 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26d8162440 0x7f26d8162850 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f26d80b6f10 tx=0x7f26c002a4b0 comp rx=0 tx=0).stop 2024-09-21T07:33:29.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.715+0000 7f26dd800640 1 -- 172.21.15.39:0/3657673087 shutdown_connections 2024-09-21T07:33:29.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.715+0000 7f26dd800640 1 --2- 172.21.15.39:0/3657673087 >> v2:172.21.15.148:3300/0 conn(0x7f26d80b8fd0 0x7f26d80b93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:29.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.715+0000 7f26dd800640 1 --2- 172.21.15.39:0/3657673087 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26d8162440 0x7f26d8162850 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:29.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.715+0000 7f26dd800640 1 --2- 172.21.15.39:0/3657673087 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f26d80b9bd0 0x7f26d80b71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:29.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.715+0000 7f26dd800640 1 -- 172.21.15.39:0/3657673087 >> 172.21.15.39:0/3657673087 conn(0x7f26d809d6c0 msgr2=0x7f26d80d4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:29.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.715+0000 7f26dd800640 1 -- 172.21.15.39:0/3657673087 shutdown_connections 2024-09-21T07:33:29.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.715+0000 7f26dd800640 1 -- 172.21.15.39:0/3657673087 wait complete. 2024-09-21T07:33:29.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.717+0000 7f26dd800640 1 Processor -- start 2024-09-21T07:33:29.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.717+0000 7f26dd800640 1 -- start start 2024-09-21T07:33:29.721 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:33:29.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.719+0000 7f26dd800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f26d80b8fd0 0x7f26d8150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:29.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.719+0000 7f26dd800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26d80b9bd0 0x7f26d81611f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:29.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.719+0000 7f26dd800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26d8162440 0x7f26d8161730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:29.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.719+0000 7f26dd800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f26d80b5450 con 0x7f26d80b9bd0 2024-09-21T07:33:29.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.719+0000 7f26dd800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f26d80b52d0 con 0x7f26d8162440 2024-09-21T07:33:29.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.719+0000 7f26dd800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f26d80b55d0 con 0x7f26d80b8fd0 2024-09-21T07:33:29.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.719+0000 7f26cf400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26d80b9bd0 0x7f26d81611f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:29.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.719+0000 7f26cf400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26d80b9bd0 0x7f26d81611f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60404/0 (socket says 172.21.15.39:60404) 2024-09-21T07:33:29.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.719+0000 7f26d7e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f26d80b8fd0 0x7f26d8150ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:29.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.719+0000 7f26cf400640 1 -- 172.21.15.39:0/686183674 learned_addr learned my addr 172.21.15.39:0/686183674 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:29.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.719+0000 7f26dce00640 1 --2- 172.21.15.39:0/686183674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26d8162440 0x7f26d8161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:29.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.720+0000 7f26cf400640 1 -- 172.21.15.39:0/686183674 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f26d80b8fd0 msgr2=0x7f26d8150ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:29.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.720+0000 7f26cf400640 1 --2- 172.21.15.39:0/686183674 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f26d80b8fd0 0x7f26d8150ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:29.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.720+0000 7f26cf400640 1 -- 172.21.15.39:0/686183674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26d8162440 msgr2=0x7f26d8161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:29.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.720+0000 7f26cf400640 1 --2- 172.21.15.39:0/686183674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26d8162440 0x7f26d8161730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:29.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.720+0000 7f26cf400640 1 -- 172.21.15.39:0/686183674 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d81f66b0 con 0x7f26d80b9bd0 2024-09-21T07:33:29.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.720+0000 7f26cf400640 1 --2- 172.21.15.39:0/686183674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26d80b9bd0 0x7f26d81611f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f26c00099d0 tx=0x7f26c00052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:29.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.720+0000 7f26d4c00640 1 -- 172.21.15.39:0/686183674 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c0005cf0 con 0x7f26d80b9bd0 2024-09-21T07:33:29.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.720+0000 7f26d4c00640 1 -- 172.21.15.39:0/686183674 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f26c0031b50 con 0x7f26d80b9bd0 2024-09-21T07:33:29.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.720+0000 7f26d4c00640 1 -- 172.21.15.39:0/686183674 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c00330d0 con 0x7f26d80b9bd0 2024-09-21T07:33:29.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.720+0000 7f26dd800640 1 -- 172.21.15.39:0/686183674 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26d8153a10 con 0x7f26d80b9bd0 2024-09-21T07:33:29.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.720+0000 7f26dd800640 1 -- 172.21.15.39:0/686183674 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26d81f9af0 con 0x7f26d80b9bd0 2024-09-21T07:33:29.725 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:33:29.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.723+0000 7f26d4c00640 1 -- 172.21.15.39:0/686183674 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f26c0033270 con 0x7f26d80b9bd0 2024-09-21T07:33:29.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.725+0000 7f26d4c00640 1 --2- 172.21.15.39:0/686183674 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f26b4078bc0 0x7f26b407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:29.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.725+0000 7f26d4c00640 1 -- 172.21.15.39:0/686183674 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f26c00bbfe0 con 0x7f26d80b9bd0 2024-09-21T07:33:29.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.725+0000 7f26d7e00640 1 --2- 172.21.15.39:0/686183674 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f26b4078bc0 0x7f26b407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:29.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.725+0000 7f26e5ea1740 1 Processor -- start 2024-09-21T07:33:29.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.725+0000 7f26e5ea1740 1 -- start start 2024-09-21T07:33:29.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.726+0000 7f26d7e00640 1 --2- 172.21.15.39:0/686183674 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f26b4078bc0 0x7f26b407b0d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f26c8004510 tx=0x7f26c800b390 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:29.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.727+0000 7f26e5ea1740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559035557270 0x559035927b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:29.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.727+0000 7f26e5ea1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55903552e3b0 0x5590359280a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:29.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.727+0000 7f26e5ea1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559035536590 0x5590358f8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:29.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.727+0000 7f26e5ea1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559035881430 con 0x55903552e3b0 2024-09-21T07:33:29.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.727+0000 7f26e5ea1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5590358715d0 con 0x559035536590 2024-09-21T07:33:29.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.727+0000 7f26e5ea1740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x559035860a00 con 0x559035557270 2024-09-21T07:33:29.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.727+0000 7f26cf400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55903552e3b0 0x5590359280a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:29.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.727+0000 7f26cf400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55903552e3b0 0x5590359280a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60418/0 (socket says 172.21.15.39:60418) 2024-09-21T07:33:29.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.727+0000 7f26cf400640 1 -- 172.21.15.39:0/532436076 learned_addr learned my addr 172.21.15.39:0/532436076 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:29.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.727+0000 7f26d7e00640 1 --2- 172.21.15.39:0/532436076 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559035557270 0x559035927b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:29.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.727+0000 7f26dce00640 1 --2- 172.21.15.39:0/532436076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559035536590 0x5590358f8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:29.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.727+0000 7f26cf400640 1 -- 172.21.15.39:0/532436076 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559035557270 msgr2=0x559035927b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:33:29.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.727+0000 7f26cf400640 1 --2- 172.21.15.39:0/532436076 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559035557270 0x559035927b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:29.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.727+0000 7f26cf400640 1 -- 172.21.15.39:0/532436076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559035536590 msgr2=0x5590358f8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:33:29.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.727+0000 7f26cf400640 1 --2- 172.21.15.39:0/532436076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559035536590 0x5590358f8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:29.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.727+0000 7f26cf400640 1 -- 172.21.15.39:0/532436076 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559035829520 con 0x55903552e3b0 2024-09-21T07:33:29.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.728+0000 7f26cf400640 1 --2- 172.21.15.39:0/532436076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55903552e3b0 0x5590359280a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f26c002aaa0 tx=0x7f26c0083c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c45217dfc1f59ef8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:29.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.728+0000 7f26ccc00640 1 -- 172.21.15.39:0/532436076 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c0048040 con 0x55903552e3b0 2024-09-21T07:33:29.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.728+0000 7f26ccc00640 1 -- 172.21.15.39:0/532436076 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f26c0083d30 con 0x55903552e3b0 2024-09-21T07:33:29.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.728+0000 7f26ccc00640 1 -- 172.21.15.39:0/532436076 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c00aa0d0 con 0x55903552e3b0 2024-09-21T07:33:29.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.729+0000 7f26e5ea1740 1 -- 172.21.15.39:0/532436076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55903552e3b0 msgr2=0x5590359280a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:29.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.729+0000 7f26e5ea1740 1 --2- 172.21.15.39:0/532436076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55903552e3b0 0x5590359280a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f26c002aaa0 tx=0x7f26c0083c30 comp rx=0 tx=0).stop 2024-09-21T07:33:29.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.729+0000 7f26e5ea1740 1 -- 172.21.15.39:0/532436076 shutdown_connections 2024-09-21T07:33:29.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.729+0000 7f26e5ea1740 1 --2- 172.21.15.39:0/532436076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559035536590 0x5590358f8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:29.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.729+0000 7f26e5ea1740 1 --2- 172.21.15.39:0/532436076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55903552e3b0 0x5590359280a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:29.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.729+0000 7f26e5ea1740 1 --2- 172.21.15.39:0/532436076 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559035557270 0x559035927b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:29.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.729+0000 7f26e5ea1740 1 -- 172.21.15.39:0/532436076 >> 172.21.15.39:0/532436076 conn(0x55903554b3c0 msgr2=0x5590358fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:29.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.729+0000 7f26e5ea1740 1 -- 172.21.15.39:0/532436076 shutdown_connections 2024-09-21T07:33:29.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.729+0000 7f26e5ea1740 1 -- 172.21.15.39:0/532436076 wait complete. 2024-09-21T07:33:29.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.730+0000 7f26e5ea1740 1 -- start start 2024-09-21T07:33:29.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.730+0000 7f26e5ea1740 1 Processor -- start 2024-09-21T07:33:29.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.732+0000 7f26e5ea1740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55903552e3b0 0x559035920da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:29.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.732+0000 7f26e5ea1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559035536590 0x5590359518a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:29.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.732+0000 7f26e5ea1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559035557270 0x55903591e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:29.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.732+0000 7f26e5ea1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559035807330 con 0x559035557270 2024-09-21T07:33:29.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.732+0000 7f26e5ea1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559035864810 con 0x559035536590 2024-09-21T07:33:29.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.732+0000 7f26e5ea1740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x5590355d1620 con 0x55903552e3b0 2024-09-21T07:33:29.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.732+0000 7f26e5ea1740 10 client.0 _ll_register_callbacks cb 0x55903580c420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:33:29.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.732+0000 7f26dce00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559035557270 0x55903591e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:29.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.732+0000 7f26d7e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55903552e3b0 0x559035920da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:29.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.732+0000 7f26cf400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559035536590 0x5590359518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:29.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.732+0000 7f26dce00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559035557270 0x55903591e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60426/0 (socket says 172.21.15.39:60426) 2024-09-21T07:33:29.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.732+0000 7f26dce00640 1 -- 172.21.15.39:0/377451798 learned_addr learned my addr 172.21.15.39:0/377451798 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:29.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.732+0000 7f26dce00640 1 -- 172.21.15.39:0/377451798 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55903552e3b0 msgr2=0x559035920da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:33:29.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.732+0000 7f26dce00640 1 --2- 172.21.15.39:0/377451798 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55903552e3b0 0x559035920da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:29.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.732+0000 7f26dce00640 1 -- 172.21.15.39:0/377451798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559035536590 msgr2=0x5590359518a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:33:29.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.732+0000 7f26dce00640 1 --2- 172.21.15.39:0/377451798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559035536590 0x5590359518a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:29.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.732+0000 7f26dce00640 1 -- 172.21.15.39:0/377451798 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590355c2850 con 0x559035557270 2024-09-21T07:33:29.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.733+0000 7f26dce00640 1 --2- 172.21.15.39:0/377451798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559035557270 0x55903591e230 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f26d0011a30 tx=0x7f26d0011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce083783c2d099b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:29.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.733+0000 7f26c5600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:33:29.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.733+0000 7f26c5600640 1 -- 172.21.15.39:0/377451798 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d000fcf0 con 0x559035557270 2024-09-21T07:33:29.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.733+0000 7f26e5ea1740 1 -- 172.21.15.39:0/377451798 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26c0083d30 con 0x559035557270 2024-09-21T07:33:29.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.733+0000 7f26c5600640 1 -- 172.21.15.39:0/377451798 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f26d0004b20 con 0x559035557270 2024-09-21T07:33:29.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.733+0000 7f26c5600640 1 -- 172.21.15.39:0/377451798 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f26d0024050 con 0x559035557270 2024-09-21T07:33:29.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.733+0000 7f26e5ea1740 10 client.4692 fetch_fsmap learned FSMap version 8 2024-09-21T07:33:29.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.733+0000 7f26e5ea1740 1 -- 172.21.15.39:0/377451798 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x5590356396b0 con 0x559035557270 2024-09-21T07:33:29.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.734+0000 7f26c5600640 1 -- 172.21.15.39:0/377451798 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f26c0083d30 con 0x559035557270 2024-09-21T07:33:29.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.734+0000 7f26c5600640 1 -- 172.21.15.39:0/377451798 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7f26d0027020 con 0x559035557270 2024-09-21T07:33:29.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.734+0000 7f26e5ea1740 10 client.4692 fetch_fsmap finished waiting for FSMap version 8 2024-09-21T07:33:29.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.734+0000 7f26e5ea1740 10 client.4692 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:33:29.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.734+0000 7f26e5ea1740 20 client.4692 populate_metadata read hostname 'smithi039' 2024-09-21T07:33:29.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.734+0000 7f26e5ea1740 1 --2- 172.21.15.39:0/377451798 >> [v2:172.21.15.39:6832/414851924,v1:172.21.15.39:6833/414851924] conn(0x559035515e40 0x55903593aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:29.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.734+0000 7f26e5ea1740 4 client.4692 mds_command: new command op to 4378 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:33:29.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.734+0000 7f26e5ea1740 1 -- 172.21.15.39:0/377451798 --> [v2:172.21.15.39:6832/414851924,v1:172.21.15.39:6833/414851924] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559035873520 con 0x559035515e40 2024-09-21T07:33:29.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.734+0000 7f26d7e00640 1 --2- 172.21.15.39:0/377451798 >> [v2:172.21.15.39:6832/414851924,v1:172.21.15.39:6833/414851924] conn(0x559035515e40 0x55903593aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:29.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.735+0000 7f26d7e00640 1 --2- 172.21.15.39:0/377451798 >> [v2:172.21.15.39:6832/414851924,v1:172.21.15.39:6833/414851924] conn(0x559035515e40 0x55903593aae0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12fbb21cc8e6a882 server_cookie=695d694741e177d8 in_seq=0 out_seq=0 2024-09-21T07:33:29.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.735+0000 7f26c5600640 10 client.4692 ms_handle_connect on v2:172.21.15.39:6832/414851924 2024-09-21T07:33:29.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.736+0000 7f26c5600640 1 -- 172.21.15.39:0/377451798 <== mds.0 v2:172.21.15.39:6832/414851924 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559035873520 con 0x559035515e40 2024-09-21T07:33:29.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.736+0000 7f26c5600640 10 client.4692 handle_command_reply: tid=0 2024-09-21T07:33:29.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.736+0000 7f26e5ea1740 1 client.4692 shutdown 2024-09-21T07:33:29.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.736+0000 7f26e5ea1740 1 -- 172.21.15.39:0/377451798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559035557270 msgr2=0x55903591e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:29.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.736+0000 7f26e5ea1740 1 --2- 172.21.15.39:0/377451798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559035557270 0x55903591e230 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f26d0011a30 tx=0x7f26d0011ef0 comp rx=0 tx=0).stop 2024-09-21T07:33:29.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.736+0000 7f26e5ea1740 1 -- 172.21.15.39:0/377451798 shutdown_connections 2024-09-21T07:33:29.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.736+0000 7f26e5ea1740 1 --2- 172.21.15.39:0/377451798 >> [v2:172.21.15.39:6832/414851924,v1:172.21.15.39:6833/414851924] conn(0x559035515e40 0x55903593aae0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:29.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.736+0000 7f26e5ea1740 1 --2- 172.21.15.39:0/377451798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559035557270 0x55903591e230 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:29.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.736+0000 7f26e5ea1740 1 --2- 172.21.15.39:0/377451798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559035536590 0x5590359518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:29.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.736+0000 7f26e5ea1740 1 --2- 172.21.15.39:0/377451798 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55903552e3b0 0x559035920da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:29.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.736+0000 7f26e5ea1740 1 -- 172.21.15.39:0/377451798 >> 172.21.15.39:0/377451798 conn(0x55903554b3c0 msgr2=0x5590358f7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:29.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.736+0000 7f26c5600640 0 client.4692 ms_handle_reset on v2:172.21.15.39:6832/414851924 2024-09-21T07:33:29.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.736+0000 7f26e5ea1740 1 -- 172.21.15.39:0/377451798 shutdown_connections 2024-09-21T07:33:29.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.736+0000 7f26e5ea1740 1 -- 172.21.15.39:0/377451798 wait complete. 2024-09-21T07:33:29.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.736+0000 7f26e5ea1740 20 client.4692 trim_cache size 0 max 16384 2024-09-21T07:33:29.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.756+0000 7f26e5ea1740 1 Processor -- start 2024-09-21T07:33:29.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.756+0000 7f26e5ea1740 1 -- start start 2024-09-21T07:33:29.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.757+0000 7f26e5ea1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55903554b3c0 0x5590358f9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:29.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.757+0000 7f26e5ea1740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55903552e3b0 0x559035966c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:29.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.757+0000 7f26e5ea1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559035536590 0x5590359671c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:29.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.757+0000 7f26e5ea1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f26d8201f10 con 0x559035536590 2024-09-21T07:33:29.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.757+0000 7f26e5ea1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f26d80c9d20 con 0x55903554b3c0 2024-09-21T07:33:29.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.757+0000 7f26e5ea1740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f26d80c9080 con 0x55903552e3b0 2024-09-21T07:33:29.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.758+0000 7f26dce00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559035536590 0x5590359671c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:29.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.758+0000 7f26cf400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55903552e3b0 0x559035966c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:29.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.758+0000 7f26d7e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55903554b3c0 0x5590358f9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:29.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.758+0000 7f26cf400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55903552e3b0 0x559035966c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.39:41284/0 (socket says 172.21.15.39:41284) 2024-09-21T07:33:29.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.758+0000 7f26cf400640 1 -- 172.21.15.39:0/1490589980 learned_addr learned my addr 172.21.15.39:0/1490589980 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:29.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.758+0000 7f26dce00640 1 -- 172.21.15.39:0/1490589980 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55903552e3b0 msgr2=0x559035966c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:33:29.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.758+0000 7f26dce00640 1 --2- 172.21.15.39:0/1490589980 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55903552e3b0 0x559035966c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:29.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.758+0000 7f26dce00640 1 -- 172.21.15.39:0/1490589980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55903554b3c0 msgr2=0x5590358f9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:33:29.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.758+0000 7f26dce00640 1 --2- 172.21.15.39:0/1490589980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55903554b3c0 0x5590358f9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:29.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.758+0000 7f26dce00640 1 -- 172.21.15.39:0/1490589980 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590358fe840 con 0x559035536590 2024-09-21T07:33:29.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.758+0000 7f26dce00640 1 --2- 172.21.15.39:0/1490589980 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559035536590 0x5590359671c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f26d0006010 tx=0x7f26d0011510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1684692b441801d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:29.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.759+0000 7f26c7e00640 1 -- 172.21.15.39:0/1490589980 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d0004b20 con 0x559035536590 2024-09-21T07:33:29.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.759+0000 7f26c7e00640 1 -- 172.21.15.39:0/1490589980 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f26d0003aa0 con 0x559035536590 2024-09-21T07:33:29.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.759+0000 7f26c7e00640 1 -- 172.21.15.39:0/1490589980 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d0003cf0 con 0x559035536590 2024-09-21T07:33:29.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.760+0000 7f26e5ea1740 1 -- 172.21.15.39:0/1490589980 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559035536590 msgr2=0x5590359671c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:29.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.760+0000 7f26e5ea1740 1 --2- 172.21.15.39:0/1490589980 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559035536590 0x5590359671c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f26d0006010 tx=0x7f26d0011510 comp rx=0 tx=0).stop 2024-09-21T07:33:29.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.760+0000 7f26e5ea1740 1 -- 172.21.15.39:0/1490589980 shutdown_connections 2024-09-21T07:33:29.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.760+0000 7f26e5ea1740 1 --2- 172.21.15.39:0/1490589980 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559035536590 0x5590359671c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:29.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.760+0000 7f26e5ea1740 1 --2- 172.21.15.39:0/1490589980 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55903552e3b0 0x559035966c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:29.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.760+0000 7f26e5ea1740 1 --2- 172.21.15.39:0/1490589980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55903554b3c0 0x5590358f9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:29.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.760+0000 7f26e5ea1740 1 -- 172.21.15.39:0/1490589980 >> 172.21.15.39:0/1490589980 conn(0x5590359435b0 msgr2=0x55903591d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:29.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.760+0000 7f26e5ea1740 1 -- 172.21.15.39:0/1490589980 shutdown_connections 2024-09-21T07:33:29.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.760+0000 7f26e5ea1740 1 -- 172.21.15.39:0/1490589980 wait complete. 2024-09-21T07:33:29.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.761+0000 7f26e5ea1740 1 -- start start 2024-09-21T07:33:29.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.761+0000 7f26e5ea1740 1 Processor -- start 2024-09-21T07:33:29.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.762+0000 7f26e5ea1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55903552e3b0 0x559035926bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:29.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.762+0000 7f26e5ea1740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559035536590 0x5590359b9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:29.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.762+0000 7f26e5ea1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55903554b3c0 0x5590359c3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:29.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.762+0000 7f26e5ea1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5590359241c0 con 0x55903554b3c0 2024-09-21T07:33:29.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.762+0000 7f26e5ea1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f26d80c96d0 con 0x55903552e3b0 2024-09-21T07:33:29.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.762+0000 7f26e5ea1740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55903593c080 con 0x559035536590 2024-09-21T07:33:29.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.762+0000 7f26e5ea1740 10 client.0 _ll_register_callbacks cb 0x7f26d0011610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:33:29.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.763+0000 7f26dce00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55903554b3c0 0x5590359c3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:29.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.763+0000 7f26d7e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55903552e3b0 0x559035926bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:29.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.763+0000 7f26d7e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55903552e3b0 0x559035926bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.39:59460/0 (socket says 172.21.15.39:59460) 2024-09-21T07:33:29.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.763+0000 7f26cf400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559035536590 0x5590359b9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:29.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.763+0000 7f26d7e00640 1 -- 172.21.15.39:0/3073453262 learned_addr learned my addr 172.21.15.39:0/3073453262 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:29.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.763+0000 7f26cf400640 1 -- 172.21.15.39:0/3073453262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55903552e3b0 msgr2=0x559035926bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:33:29.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.763+0000 7f26cf400640 1 --2- 172.21.15.39:0/3073453262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55903552e3b0 0x559035926bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:29.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.763+0000 7f26cf400640 1 -- 172.21.15.39:0/3073453262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55903554b3c0 msgr2=0x5590359c3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:33:29.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.763+0000 7f26cf400640 1 --2- 172.21.15.39:0/3073453262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55903554b3c0 0x5590359c3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:29.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.763+0000 7f26cf400640 1 -- 172.21.15.39:0/3073453262 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) -- 0x55903591dfb0 con 0x559035536590 2024-09-21T07:33:29.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.763+0000 7f26cf400640 1 --2- 172.21.15.39:0/3073453262 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559035536590 0x5590359b9540 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f26c002aaa0 tx=0x7f26c0033770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e825d3c676746b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:29.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.763+0000 7f26c5600640 10 client.0 ms_handle_connect on v2:172.21.15.148:3301/0 2024-09-21T07:33:29.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.764+0000 7f26c5600640 1 -- 172.21.15.39:0/3073453262 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c0048040 con 0x559035536590 2024-09-21T07:33:29.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.764+0000 7f26c5600640 1 -- 172.21.15.39:0/3073453262 <== mon.2 v2:172.21.15.148:3301/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f26c00841a0 con 0x559035536590 2024-09-21T07:33:29.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.764+0000 7f26e5ea1740 1 -- 172.21.15.39:0/3073453262 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26d0003aa0 con 0x559035536590 2024-09-21T07:33:29.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.764+0000 7f26c5600640 1 -- 172.21.15.39:0/3073453262 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f26c00aa0d0 con 0x559035536590 2024-09-21T07:33:29.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.764+0000 7f26e5ea1740 10 client.4565 fetch_fsmap learned FSMap version 8 2024-09-21T07:33:29.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.764+0000 7f26e5ea1740 1 -- 172.21.15.39:0/3073453262 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x559035915de0 con 0x559035536590 2024-09-21T07:33:29.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.764+0000 7f26c5600640 1 -- 172.21.15.39:0/3073453262 <== mon.2 v2:172.21.15.148:3301/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f26d0003aa0 con 0x559035536590 2024-09-21T07:33:29.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.764+0000 7f26c5600640 1 -- 172.21.15.39:0/3073453262 <== mon.2 v2:172.21.15.148:3301/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7f26c008d210 con 0x559035536590 2024-09-21T07:33:29.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.764+0000 7f26e5ea1740 10 client.4565 fetch_fsmap finished waiting for FSMap version 8 2024-09-21T07:33:29.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.764+0000 7f26e5ea1740 10 client.4565 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:33:29.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.764+0000 7f26e5ea1740 20 client.4565 populate_metadata read hostname 'smithi039' 2024-09-21T07:33:29.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.764+0000 7f26e5ea1740 1 --2- 172.21.15.39:0/3073453262 >> [v2:172.21.15.39:6832/414851924,v1:172.21.15.39:6833/414851924] conn(0x5590354eb280 0x559035919090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:29.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.764+0000 7f26e5ea1740 4 client.4565 mds_command: new command op to 4378 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:33:29.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.764+0000 7f26e5ea1740 1 -- 172.21.15.39:0/3073453262 --> [v2:172.21.15.39:6832/414851924,v1:172.21.15.39:6833/414851924] -- command(tid 0: {"prefix": "status"}) -- 0x55903586efd0 con 0x5590354eb280 2024-09-21T07:33:29.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.765+0000 7f26d7e00640 1 --2- 172.21.15.39:0/3073453262 >> [v2:172.21.15.39:6832/414851924,v1:172.21.15.39:6833/414851924] conn(0x5590354eb280 0x559035919090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:29.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.765+0000 7f26d7e00640 1 --2- 172.21.15.39:0/3073453262 >> [v2:172.21.15.39:6832/414851924,v1:172.21.15.39:6833/414851924] conn(0x5590354eb280 0x559035919090 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3283f99622de9e79 server_cookie=361362188ccd9f59 in_seq=0 out_seq=0 2024-09-21T07:33:29.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.765+0000 7f26c5600640 10 client.4565 ms_handle_connect on v2:172.21.15.39:6832/414851924 2024-09-21T07:33:29.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.765+0000 7f26c5600640 1 -- 172.21.15.39:0/3073453262 <== mds.0 v2:172.21.15.39:6832/414851924 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55903586efd0 con 0x5590354eb280 2024-09-21T07:33:29.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.765+0000 7f26c5600640 10 client.4565 handle_command_reply: tid=0 2024-09-21T07:33:29.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.765+0000 7f26e5ea1740 1 client.4565 shutdown 2024-09-21T07:33:29.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.766+0000 7f26e5ea1740 1 -- 172.21.15.39:0/3073453262 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559035536590 msgr2=0x5590359b9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:29.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.766+0000 7f26e5ea1740 1 --2- 172.21.15.39:0/3073453262 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559035536590 0x5590359b9540 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f26c002aaa0 tx=0x7f26c0033770 comp rx=0 tx=0).stop 2024-09-21T07:33:29.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.766+0000 7f26e5ea1740 1 -- 172.21.15.39:0/3073453262 shutdown_connections 2024-09-21T07:33:29.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.766+0000 7f26e5ea1740 1 --2- 172.21.15.39:0/3073453262 >> [v2:172.21.15.39:6832/414851924,v1:172.21.15.39:6833/414851924] conn(0x5590354eb280 0x559035919090 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:29.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.766+0000 7f26e5ea1740 1 --2- 172.21.15.39:0/3073453262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55903554b3c0 0x5590359c3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:29.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.766+0000 7f26e5ea1740 1 --2- 172.21.15.39:0/3073453262 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559035536590 0x5590359b9540 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:29.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.766+0000 7f26e5ea1740 1 --2- 172.21.15.39:0/3073453262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55903552e3b0 0x559035926bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:29.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.766+0000 7f26c5600640 0 client.4565 ms_handle_reset on v2:172.21.15.39:6832/414851924 2024-09-21T07:33:29.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.766+0000 7f26e5ea1740 1 -- 172.21.15.39:0/3073453262 >> 172.21.15.39:0/3073453262 conn(0x5590359435b0 msgr2=0x55903591d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:29.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.766+0000 7f26e5ea1740 1 -- 172.21.15.39:0/3073453262 shutdown_connections 2024-09-21T07:33:29.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.766+0000 7f26e5ea1740 1 -- 172.21.15.39:0/3073453262 wait complete. 2024-09-21T07:33:29.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.766+0000 7f26e5ea1740 20 client.4565 trim_cache size 0 max 16384 2024-09-21T07:33:29.771 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:33:29.771 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:33:29.771 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-09-21T07:33:29.771 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 4378, 2024-09-21T07:33:29.771 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:33:29.772 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:33:29.772 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:33:29.772 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 14.618303492000001, 2024-09-21T07:33:29.772 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 8, 2024-09-21T07:33:29.772 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 21, 2024-09-21T07:33:29.772 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 21, 2024-09-21T07:33:29.772 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 20.662889263 2024-09-21T07:33:29.772 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:33:29.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.767+0000 7f26dd800640 1 -- 172.21.15.39:0/686183674 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f26b4078bc0 msgr2=0x7f26b407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:29.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.767+0000 7f26dd800640 1 --2- 172.21.15.39:0/686183674 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f26b4078bc0 0x7f26b407b0d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f26c8004510 tx=0x7f26c800b390 comp rx=0 tx=0).stop 2024-09-21T07:33:29.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.767+0000 7f26dd800640 1 -- 172.21.15.39:0/686183674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26d80b9bd0 msgr2=0x7f26d81611f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:29.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.767+0000 7f26dd800640 1 --2- 172.21.15.39:0/686183674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26d80b9bd0 0x7f26d81611f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f26c00099d0 tx=0x7f26c00052f0 comp rx=0 tx=0).stop 2024-09-21T07:33:29.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.767+0000 7f26dd800640 1 -- 172.21.15.39:0/686183674 shutdown_connections 2024-09-21T07:33:29.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.767+0000 7f26dd800640 1 --2- 172.21.15.39:0/686183674 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f26b4078bc0 0x7f26b407b0d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:29.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.767+0000 7f26dd800640 1 --2- 172.21.15.39:0/686183674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26d8162440 0x7f26d8161730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:29.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.767+0000 7f26dd800640 1 --2- 172.21.15.39:0/686183674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26d80b9bd0 0x7f26d81611f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:29.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.767+0000 7f26dd800640 1 --2- 172.21.15.39:0/686183674 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f26d80b8fd0 0x7f26d8150ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:29.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.767+0000 7f26dd800640 1 -- 172.21.15.39:0/686183674 >> 172.21.15.39:0/686183674 conn(0x7f26d809d6c0 msgr2=0x7f26d80d2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:29.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.767+0000 7f26dd800640 1 -- 172.21.15.39:0/686183674 shutdown_connections 2024-09-21T07:33:29.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:29.767+0000 7f26dd800640 1 -- 172.21.15.39:0/686183674 wait complete. 2024-09-21T07:33:29.792 INFO:teuthology.run_tasks:Running task kclient... 2024-09-21T07:33:29.815 INFO:tasks.kclient:Mounting kernel clients... 2024-09-21T07:33:29.815 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-21T07:33:29.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:29.815 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-09-21T07:33:29.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:29.901 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link delete ceph-brx 2024-09-21T07:33:29.999 INFO:teuthology.orchestra.run.smithi039.stderr:Cannot find device "ceph-brx" 2024-09-21T07:33:30.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:33:30.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:30.003 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-09-21T07:33:30.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:30.038 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link delete ceph-brx 2024-09-21T07:33:30.136 INFO:teuthology.orchestra.run.smithi039.stderr:Cannot find device "ceph-brx" 2024-09-21T07:33:30.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:33:30.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:30.139 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-09-21T07:33:30.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:30.180 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link delete ceph-brx 2024-09-21T07:33:30.282 INFO:teuthology.orchestra.run.smithi148.stderr:Cannot find device "ceph-brx" 2024-09-21T07:33:30.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:33:30.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:30.285 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-09-21T07:33:30.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:30.321 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link delete ceph-brx 2024-09-21T07:33:30.420 INFO:teuthology.orchestra.run.smithi148.stderr:Cannot find device "ceph-brx" 2024-09-21T07:33:30.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:33:30.423 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-09-21T07:33:30.423 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T07:33:30.423 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:33:30.423 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:33:30.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.648+0000 7f2846e00640 1 -- 172.21.15.39:0/3709807068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2824003fd0 msgr2=0x7f2824004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:30.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.648+0000 7f2846e00640 1 --2- 172.21.15.39:0/3709807068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2824003fd0 0x7f2824004800 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f282c00c260 tx=0x7f2824005830 comp rx=0 tx=0).stop 2024-09-21T07:33:30.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.648+0000 7f2844e00640 1 -- 172.21.15.39:0/3709807068 reap_dead start 2024-09-21T07:33:30.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.648+0000 7f2846e00640 1 -- 172.21.15.39:0/3709807068 shutdown_connections 2024-09-21T07:33:30.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.648+0000 7f2846e00640 1 -- 172.21.15.39:0/3709807068 >> 172.21.15.39:0/3709807068 conn(0x7f28400b1cb0 msgr2=0x7f28400b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:30.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.648+0000 7f2846e00640 1 -- 172.21.15.39:0/3709807068 shutdown_connections 2024-09-21T07:33:30.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.649+0000 7f2846e00640 1 -- 172.21.15.39:0/3709807068 wait complete. 2024-09-21T07:33:30.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.651+0000 7f2846e00640 1 Processor -- start 2024-09-21T07:33:30.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.651+0000 7f2846e00640 1 -- start start 2024-09-21T07:33:30.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.652+0000 7f2846e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2824003fd0 0x7f284009c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:30.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.652+0000 7f2846e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28400ca800 0x7f28400d0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:30.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.652+0000 7f2846e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28400d05d0 0x7f28401f8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:30.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.652+0000 7f2846e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f28400710a0 con 0x7f28400ca800 2024-09-21T07:33:30.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.652+0000 7f2846e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2840070f20 con 0x7f28400d05d0 2024-09-21T07:33:30.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.652+0000 7f2846e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f28400cf010 con 0x7f2824003fd0 2024-09-21T07:33:30.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.653+0000 7f283f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28400ca800 0x7f28400d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:30.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.653+0000 7f283f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28400ca800 0x7f28400d0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60482/0 (socket says 172.21.15.39:60482) 2024-09-21T07:33:30.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.653+0000 7f283f400640 1 -- 172.21.15.39:0/2880845706 learned_addr learned my addr 172.21.15.39:0/2880845706 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:30.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.653+0000 7f2844e00640 1 --2- 172.21.15.39:0/2880845706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28400d05d0 0x7f28401f8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:30.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.653+0000 7f283fe00640 1 --2- 172.21.15.39:0/2880845706 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2824003fd0 0x7f284009c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:30.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.653+0000 7f283f400640 1 -- 172.21.15.39:0/2880845706 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2824003fd0 msgr2=0x7f284009c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:30.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.653+0000 7f283f400640 1 --2- 172.21.15.39:0/2880845706 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2824003fd0 0x7f284009c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:30.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.653+0000 7f283f400640 1 -- 172.21.15.39:0/2880845706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28400d05d0 msgr2=0x7f28401f8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:30.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.653+0000 7f283f400640 1 --2- 172.21.15.39:0/2880845706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28400d05d0 0x7f28401f8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:30.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.653+0000 7f283f400640 1 -- 172.21.15.39:0/2880845706 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28400d09e0 con 0x7f28400ca800 2024-09-21T07:33:30.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.653+0000 7f2844e00640 1 --2- 172.21.15.39:0/2880845706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28400d05d0 0x7f28401f8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:33:30.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.653+0000 7f283f400640 1 --2- 172.21.15.39:0/2880845706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28400ca800 0x7f28400d0090 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f282c00c380 tx=0x7f282c001d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:30.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.654+0000 7f2837e00640 1 -- 172.21.15.39:0/2880845706 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282c031040 con 0x7f28400ca800 2024-09-21T07:33:30.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.654+0000 7f2837e00640 1 -- 172.21.15.39:0/2880845706 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f282c0035f0 con 0x7f28400ca800 2024-09-21T07:33:30.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.654+0000 7f2846e00640 1 -- 172.21.15.39:0/2880845706 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2840147090 con 0x7f28400ca800 2024-09-21T07:33:30.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.654+0000 7f2837e00640 1 -- 172.21.15.39:0/2880845706 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282c003a40 con 0x7f28400ca800 2024-09-21T07:33:30.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.654+0000 7f2846e00640 1 -- 172.21.15.39:0/2880845706 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2840147650 con 0x7f28400ca800 2024-09-21T07:33:30.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.655+0000 7f2846e00640 1 -- 172.21.15.39:0/2880845706 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28400b3430 con 0x7f28400ca800 2024-09-21T07:33:30.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.656+0000 7f2837e00640 1 -- 172.21.15.39:0/2880845706 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f282c034070 con 0x7f28400ca800 2024-09-21T07:33:30.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.657+0000 7f2837e00640 1 --2- 172.21.15.39:0/2880845706 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2820078b90 0x7f282007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:30.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.657+0000 7f2837e00640 1 -- 172.21.15.39:0/2880845706 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f282c0baf60 con 0x7f28400ca800 2024-09-21T07:33:30.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.657+0000 7f283fe00640 1 --2- 172.21.15.39:0/2880845706 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2820078b90 0x7f282007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:30.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.658+0000 7f283fe00640 1 --2- 172.21.15.39:0/2880845706 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2820078b90 0x7f282007b0a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f28400c3830 tx=0x7f283000c040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:30.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.662+0000 7f2837e00640 1 -- 172.21.15.39:0/2880845706 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f28400d09e0 con 0x7f28400ca800 2024-09-21T07:33:30.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.898+0000 7f2846e00640 1 -- 172.21.15.39:0/2880845706 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f28400b3150 con 0x7f28400ca800 2024-09-21T07:33:30.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.899+0000 7f2837e00640 1 -- 172.21.15.39:0/2880845706 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f282c0833a0 con 0x7f28400ca800 2024-09-21T07:33:30.901 INFO:teuthology.orchestra.run.smithi039.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:33:30.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.901+0000 7f2846e00640 1 -- 172.21.15.39:0/2880845706 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2820078b90 msgr2=0x7f282007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:30.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.901+0000 7f2846e00640 1 --2- 172.21.15.39:0/2880845706 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2820078b90 0x7f282007b0a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f28400c3830 tx=0x7f283000c040 comp rx=0 tx=0).stop 2024-09-21T07:33:30.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.901+0000 7f2846e00640 1 -- 172.21.15.39:0/2880845706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28400ca800 msgr2=0x7f28400d0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:30.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.902+0000 7f2846e00640 1 --2- 172.21.15.39:0/2880845706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28400ca800 0x7f28400d0090 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f282c00c380 tx=0x7f282c001d20 comp rx=0 tx=0).stop 2024-09-21T07:33:30.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.902+0000 7f2846e00640 1 -- 172.21.15.39:0/2880845706 shutdown_connections 2024-09-21T07:33:30.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.902+0000 7f2846e00640 1 --2- 172.21.15.39:0/2880845706 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2820078b90 0x7f282007b0a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:30.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.902+0000 7f2846e00640 1 --2- 172.21.15.39:0/2880845706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28400d05d0 0x7f28401f8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:30.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.902+0000 7f2846e00640 1 --2- 172.21.15.39:0/2880845706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28400ca800 0x7f28400d0090 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:30.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.902+0000 7f2846e00640 1 --2- 172.21.15.39:0/2880845706 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2824003fd0 0x7f284009c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:30.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.902+0000 7f2846e00640 1 -- 172.21.15.39:0/2880845706 >> 172.21.15.39:0/2880845706 conn(0x7f28400b1cb0 msgr2=0x7f28400b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:30.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.902+0000 7f2846e00640 1 -- 172.21.15.39:0/2880845706 shutdown_connections 2024-09-21T07:33:30.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:30.902+0000 7f2846e00640 1 -- 172.21.15.39:0/2880845706 wait complete. 2024-09-21T07:33:30.927 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T07:33:30.927 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:33:30.927 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi039.front.sepia.ceph.com 2024-09-21T07:33:30.927 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T07:33:30.927 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:33:30.927 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T07:33:30.927 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T07:33:30.927 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:33:30.927 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:33:30.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:30.928 DEBUG:teuthology.orchestra.run.smithi039:> ip addr 2024-09-21T07:33:30.963 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:33:30.963 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:33:30.963 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:33:30.963 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:33:30.963 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2024-09-21T07:33:30.964 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:33:30.964 INFO:teuthology.orchestra.run.smithi039.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:33:30.964 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:33:30.964 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f0 2024-09-21T07:33:30.964 INFO:teuthology.orchestra.run.smithi039.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:33:30.964 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c7 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:33:30.964 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f1 2024-09-21T07:33:30.964 INFO:teuthology.orchestra.run.smithi039.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:33:30.964 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:70 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:33:30.964 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f0 2024-09-21T07:33:30.964 INFO:teuthology.orchestra.run.smithi039.stdout: inet 172.21.15.39/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T07:33:30.964 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft 42708sec preferred_lft 42708sec 2024-09-21T07:33:30.964 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec4:7aff:febd:1370/64 scope link 2024-09-21T07:33:30.964 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:33:30.965 INFO:teuthology.orchestra.run.smithi039.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:33:30.965 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:71 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:33:30.965 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f1 2024-09-21T07:33:30.965 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T07:33:30.965 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:33:30.965 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-21T07:33:30.965 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link add name ceph-brx type bridge 2024-09-21T07:33:30.965 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip addr flush dev ceph-brx 2024-09-21T07:33:30.966 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set ceph-brx up 2024-09-21T07:33:30.966 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T07:33:30.966 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-21T07:33:31.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:33:31.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:33:31.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:31.410 DEBUG:teuthology.orchestra.run.smithi039:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:33:31.464 INFO:teuthology.orchestra.run.smithi039.stdout:1 2024-09-21T07:33:31.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:31.468 DEBUG:teuthology.orchestra.run.smithi039:> route 2024-09-21T07:33:31.511 INFO:teuthology.orchestra.run.smithi039.stdout:Kernel IP routing table 2024-09-21T07:33:31.511 INFO:teuthology.orchestra.run.smithi039.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:33:31.511 INFO:teuthology.orchestra.run.smithi039.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T07:33:31.511 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T07:33:31.511 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:33:31.511 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:33:31.512 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-21T07:33:31.512 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T07:33:31.512 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T07:33:31.512 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T07:33:31.512 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-21T07:33:31.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:33:32.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:33:32.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:32.159 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-09-21T07:33:32.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:32.195 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list-id 2024-09-21T07:33:32.271 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:33:32.271 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-21T07:33:32.271 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:33:32.271 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-21T07:33:32.271 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-21T07:33:32.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:33:32.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:33:32.449 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-21T07:33:32.449 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:33:32.449 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-21T07:33:32.450 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-21T07:33:32.450 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T07:33:32.450 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-21T07:33:32.450 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-21T07:33:32.450 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-21T07:33:32.450 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-21T07:33:32.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:33:32.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:33:32.691 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:33:32.691 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-21T07:33:32.691 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-09-21T07:33:32.691 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T07:33:32.691 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-21T07:33:32.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:33:32.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:33:32.860 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:33:32.860 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T07:33:32.895 INFO:teuthology.orchestra.run.smithi039.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:33:32.895 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:33:32.895 DEBUG:teuthology.orchestra.run.smithi039:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:33:32.970 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-21T07:33:32.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:32.970 DEBUG:teuthology.orchestra.run.smithi039:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:33:33.186 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:33:33.187 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:33:33.777 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:33:33.777 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:33:33.777 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: resolved to: "172.21.15.39:3300,172.21.15.148:3300,172.21.15.148:3301" 2024-09-21T07:33:33.777 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: trying mount with new device syntax: 0@7ab9b04a-f88d-49aa-a493-e5c076637ffb.cephfs=/ 2024-09-21T07:33:33.778 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.39:3300/172.21.15.148:3300/172.21.15.148:3301" will pass to kernel 2024-09-21T07:33:33.779 INFO:teuthology.orchestra.run.smithi039.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T07:33:33.779 INFO:teuthology.orchestra.run.smithi039.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:33:33.779 INFO:teuthology.orchestra.run.smithi039.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:33:33.779 INFO:teuthology.orchestra.run.smithi039.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:33:33.779 INFO:teuthology.orchestra.run.smithi039.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:33:33.782 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:33:33.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:33.783 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:33:33.843 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:33:33.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:33 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:33:33.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:33:33.942 INFO:teuthology.orchestra.run.smithi039.stdout:client4574 2024-09-21T07:33:33.943 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:33:33.943 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:33:33.978 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:33:34.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:33:34.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:33:34.115 INFO:teuthology.orchestra.run.smithi039.stdout:client4574 2024-09-21T07:33:34.116 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:33:34.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:33:34.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:33:34.205 INFO:teuthology.orchestra.run.smithi039.stdout:client4574 2024-09-21T07:33:34.205 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/7ab9b04a-f88d-49aa-a493-e5c076637ffb.client4574/status') 2024-09-21T07:33:34.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:33:34.305 INFO:teuthology.orchestra.run.smithi039.stdout:instance: client.4574 (3)172.21.15.39:0/751946809 2024-09-21T07:33:34.305 INFO:teuthology.orchestra.run.smithi039.stdout:blocklisted: false 2024-09-21T07:33:34.305 INFO:teuthology.orchestra.run.smithi039.stderr:0+1 records in 2024-09-21T07:33:34.305 INFO:teuthology.orchestra.run.smithi039.stderr:0+1 records out 2024-09-21T07:33:34.305 INFO:teuthology.orchestra.run.smithi039.stderr:69 bytes copied, 5.9748e-05 s, 1.2 MB/s 2024-09-21T07:33:34.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:33:34.316 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-09-21T07:33:34.316 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T07:33:34.316 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-21T07:33:34.316 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:33:34.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.540+0000 7fe290a00640 1 -- 172.21.15.39:0/2361299902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe28c0ccb50 msgr2=0x7fe28c0ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:34.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.540+0000 7fe290a00640 1 --2- 172.21.15.39:0/2361299902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe28c0ccb50 0x7fe28c0ccf60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe284005fd0 tx=0x7fe28402a8b0 comp rx=0 tx=0).stop 2024-09-21T07:33:34.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.540+0000 7fe290a00640 1 -- 172.21.15.39:0/2361299902 shutdown_connections 2024-09-21T07:33:34.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.540+0000 7fe290a00640 1 --2- 172.21.15.39:0/2361299902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe28c0ccb50 0x7fe28c0ccf60 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:34.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.540+0000 7fe290a00640 1 --2- 172.21.15.39:0/2361299902 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe28c0af3c0 0x7fe28c0b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:34.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.540+0000 7fe290a00640 1 -- 172.21.15.39:0/2361299902 >> 172.21.15.39:0/2361299902 conn(0x7fe28c06d7d0 msgr2=0x7fe28c0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:34.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.540+0000 7fe290a00640 1 -- 172.21.15.39:0/2361299902 shutdown_connections 2024-09-21T07:33:34.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.540+0000 7fe290a00640 1 -- 172.21.15.39:0/2361299902 wait complete. 2024-09-21T07:33:34.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.543+0000 7fe290a00640 1 Processor -- start 2024-09-21T07:33:34.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.543+0000 7fe290a00640 1 -- start start 2024-09-21T07:33:34.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.545+0000 7fe290a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe28c0af3c0 0x7fe28c09aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:34.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.545+0000 7fe290a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe28c0cc1e0 0x7fe28c15a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:34.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.545+0000 7fe290a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe28c0ccb50 0x7fe28c144ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:34.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.545+0000 7fe290a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c153520 con 0x7fe28c0cc1e0 2024-09-21T07:33:34.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.545+0000 7fe290a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c1533a0 con 0x7fe28c0ccb50 2024-09-21T07:33:34.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.545+0000 7fe290a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe28c1536a0 con 0x7fe28c0af3c0 2024-09-21T07:33:34.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.545+0000 7fe283e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe28c0cc1e0 0x7fe28c15a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:34.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.545+0000 7fe283e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe28c0cc1e0 0x7fe28c15a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36368/0 (socket says 172.21.15.39:36368) 2024-09-21T07:33:34.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.545+0000 7fe283e00640 1 -- 172.21.15.39:0/4210298744 learned_addr learned my addr 172.21.15.39:0/4210298744 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:34.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.545+0000 7fe289e00640 1 --2- 172.21.15.39:0/4210298744 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe28c0af3c0 0x7fe28c09aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:34.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.545+0000 7fe28a800640 1 --2- 172.21.15.39:0/4210298744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe28c0ccb50 0x7fe28c144ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:34.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.546+0000 7fe283e00640 1 -- 172.21.15.39:0/4210298744 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe28c0af3c0 msgr2=0x7fe28c09aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:34.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.546+0000 7fe283e00640 1 --2- 172.21.15.39:0/4210298744 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe28c0af3c0 0x7fe28c09aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:34.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.546+0000 7fe283e00640 1 -- 172.21.15.39:0/4210298744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe28c0ccb50 msgr2=0x7fe28c144ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:34.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.546+0000 7fe283e00640 1 --2- 172.21.15.39:0/4210298744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe28c0ccb50 0x7fe28c144ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:34.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.546+0000 7fe283e00640 1 -- 172.21.15.39:0/4210298744 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe28c09d120 con 0x7fe28c0cc1e0 2024-09-21T07:33:34.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.546+0000 7fe283e00640 1 --2- 172.21.15.39:0/4210298744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe28c0cc1e0 0x7fe28c15a0e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe27400e7c0 tx=0x7fe27400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:34.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.546+0000 7fe282000640 1 -- 172.21.15.39:0/4210298744 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27400ccc0 con 0x7fe28c0cc1e0 2024-09-21T07:33:34.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.546+0000 7fe282000640 1 -- 172.21.15.39:0/4210298744 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe274010430 con 0x7fe28c0cc1e0 2024-09-21T07:33:34.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.546+0000 7fe282000640 1 -- 172.21.15.39:0/4210298744 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe274010860 con 0x7fe28c0cc1e0 2024-09-21T07:33:34.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.546+0000 7fe290a00640 1 -- 172.21.15.39:0/4210298744 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe28c09d400 con 0x7fe28c0cc1e0 2024-09-21T07:33:34.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.546+0000 7fe290a00640 1 -- 172.21.15.39:0/4210298744 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe28c09d9c0 con 0x7fe28c0cc1e0 2024-09-21T07:33:34.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.547+0000 7fe290a00640 1 -- 172.21.15.39:0/4210298744 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe28c153bc0 con 0x7fe28c0cc1e0 2024-09-21T07:33:34.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.549+0000 7fe282000640 1 -- 172.21.15.39:0/4210298744 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe274010a00 con 0x7fe28c0cc1e0 2024-09-21T07:33:34.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.550+0000 7fe282000640 1 --2- 172.21.15.39:0/4210298744 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe268078bc0 0x7fe26807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:34.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.550+0000 7fe282000640 1 -- 172.21.15.39:0/4210298744 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe27409b5c0 con 0x7fe28c0cc1e0 2024-09-21T07:33:34.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.550+0000 7fe289e00640 1 --2- 172.21.15.39:0/4210298744 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe268078bc0 0x7fe26807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:34.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.551+0000 7fe289e00640 1 --2- 172.21.15.39:0/4210298744 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe268078bc0 0x7fe26807b0d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe27c000f60 tx=0x7fe27c009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:34.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.555+0000 7fe282000640 1 -- 172.21.15.39:0/4210298744 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe274063440 con 0x7fe28c0cc1e0 2024-09-21T07:33:34.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.795+0000 7fe290a00640 1 -- 172.21.15.39:0/4210298744 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe28c09aff0 con 0x7fe28c0cc1e0 2024-09-21T07:33:34.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.795+0000 7fe282000640 1 -- 172.21.15.39:0/4210298744 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fe274067e70 con 0x7fe28c0cc1e0 2024-09-21T07:33:34.797 INFO:teuthology.orchestra.run.smithi039.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:33:34.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.798+0000 7fe290a00640 1 -- 172.21.15.39:0/4210298744 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe268078bc0 msgr2=0x7fe26807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:34.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.798+0000 7fe290a00640 1 --2- 172.21.15.39:0/4210298744 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe268078bc0 0x7fe26807b0d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe27c000f60 tx=0x7fe27c009040 comp rx=0 tx=0).stop 2024-09-21T07:33:34.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.798+0000 7fe290a00640 1 -- 172.21.15.39:0/4210298744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe28c0cc1e0 msgr2=0x7fe28c15a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:34.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.798+0000 7fe290a00640 1 --2- 172.21.15.39:0/4210298744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe28c0cc1e0 0x7fe28c15a0e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe27400e7c0 tx=0x7fe27400ec80 comp rx=0 tx=0).stop 2024-09-21T07:33:34.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.798+0000 7fe290a00640 1 -- 172.21.15.39:0/4210298744 shutdown_connections 2024-09-21T07:33:34.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.798+0000 7fe290a00640 1 --2- 172.21.15.39:0/4210298744 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe268078bc0 0x7fe26807b0d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:34.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.798+0000 7fe290a00640 1 --2- 172.21.15.39:0/4210298744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe28c0ccb50 0x7fe28c144ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:34.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.798+0000 7fe290a00640 1 --2- 172.21.15.39:0/4210298744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe28c0cc1e0 0x7fe28c15a0e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:34.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.798+0000 7fe290a00640 1 --2- 172.21.15.39:0/4210298744 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe28c0af3c0 0x7fe28c09aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:34.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.798+0000 7fe290a00640 1 -- 172.21.15.39:0/4210298744 >> 172.21.15.39:0/4210298744 conn(0x7fe28c06d7d0 msgr2=0x7fe28c0ce790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:34.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.798+0000 7fe290a00640 1 -- 172.21.15.39:0/4210298744 shutdown_connections 2024-09-21T07:33:34.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:34.798+0000 7fe290a00640 1 -- 172.21.15.39:0/4210298744 wait complete. 2024-09-21T07:33:34.821 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T07:33:34.821 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:33:34.821 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi039.front.sepia.ceph.com 2024-09-21T07:33:34.821 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-21T07:33:34.821 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-21T07:33:34.821 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T07:33:34.821 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T07:33:34.821 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:33:34.821 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-21T07:33:34.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:34.821 DEBUG:teuthology.orchestra.run.smithi039:> ip addr 2024-09-21T07:33:34.857 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:33:34.857 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:33:34.858 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:33:34.858 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:33:34.858 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2024-09-21T07:33:34.858 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:33:34.858 INFO:teuthology.orchestra.run.smithi039.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:33:34.858 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:33:34.858 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f0 2024-09-21T07:33:34.858 INFO:teuthology.orchestra.run.smithi039.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:33:34.858 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c7 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:33:34.858 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f1 2024-09-21T07:33:34.858 INFO:teuthology.orchestra.run.smithi039.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:33:34.859 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:70 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:33:34.859 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f0 2024-09-21T07:33:34.859 INFO:teuthology.orchestra.run.smithi039.stdout: inet 172.21.15.39/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T07:33:34.859 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft 42704sec preferred_lft 42704sec 2024-09-21T07:33:34.859 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec4:7aff:febd:1370/64 scope link 2024-09-21T07:33:34.859 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:33:34.859 INFO:teuthology.orchestra.run.smithi039.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:33:34.859 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:71 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:33:34.859 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f1 2024-09-21T07:33:34.859 INFO:teuthology.orchestra.run.smithi039.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:33:34.860 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff 2024-09-21T07:33:34.860 INFO:teuthology.orchestra.run.smithi039.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:33:34.860 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:33:34.860 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::b079:ddff:fe0e:6333/64 scope link 2024-09-21T07:33:34.860 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:33:34.860 INFO:teuthology.orchestra.run.smithi039.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:33:34.860 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:33:34.860 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::2ca7:98ff:fe8a:d0ed/64 scope link 2024-09-21T07:33:34.860 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:33:34.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:34.861 DEBUG:teuthology.orchestra.run.smithi039:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:33:34.951 INFO:teuthology.orchestra.run.smithi039.stdout:1 2024-09-21T07:33:34.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:34.954 DEBUG:teuthology.orchestra.run.smithi039:> route 2024-09-21T07:33:34.990 INFO:teuthology.orchestra.run.smithi039.stdout:Kernel IP routing table 2024-09-21T07:33:34.990 INFO:teuthology.orchestra.run.smithi039.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:33:34.990 INFO:teuthology.orchestra.run.smithi039.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T07:33:34.990 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T07:33:34.990 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:33:34.991 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:33:34.991 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-21T07:33:34.991 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T07:33:34.991 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T07:33:34.991 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T07:33:34.991 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-21T07:33:35.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:33:35.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:33:35.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:35.188 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-09-21T07:33:35.227 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:33:35.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:35.227 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list-id 2024-09-21T07:33:35.302 INFO:teuthology.orchestra.run.smithi039.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-21T07:33:35.302 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:33:35.302 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-21T07:33:35.302 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:33:35.302 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-21T07:33:35.303 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-21T07:33:35.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:33:35.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:33:35.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:35.478 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-21T07:33:35.535 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:33:35.535 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:33:35.535 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:33:35.535 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:33:35.535 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2024-09-21T07:33:35.535 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:33:35.535 INFO:teuthology.orchestra.run.smithi039.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:33:35.535 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether c2:f5:ae:ee:39:7b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-21T07:33:35.535 INFO:teuthology.orchestra.run.smithi039.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-21T07:33:35.535 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:33:35.535 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::c0f5:aeff:feee:397b/64 scope link 2024-09-21T07:33:35.536 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:33:35.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:35.538 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-21T07:33:35.593 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:33:35.593 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:33:35.593 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:33:35.593 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:33:35.593 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2024-09-21T07:33:35.593 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:33:35.593 INFO:teuthology.orchestra.run.smithi039.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:33:35.594 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether c2:f5:ae:ee:39:7b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-21T07:33:35.594 INFO:teuthology.orchestra.run.smithi039.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-21T07:33:35.594 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:33:35.594 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::c0f5:aeff:feee:397b/64 scope link 2024-09-21T07:33:35.594 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:33:35.596 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-21T07:33:35.596 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:33:35.596 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-21T07:33:35.596 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-21T07:33:35.596 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-21T07:33:35.596 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-21T07:33:35.596 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-21T07:33:35.596 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-21T07:33:35.596 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-21T07:33:35.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:33:35.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:33:35.813 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:33:35.814 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-21T07:33:35.814 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.1 up 2024-09-21T07:33:35.814 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set dev brx.1 master ceph-brx 2024-09-21T07:33:35.814 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-21T07:33:35.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:33:35.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:33:35.980 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:33:35.980 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-21T07:33:36.015 INFO:teuthology.orchestra.run.smithi039.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-21T07:33:36.016 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:33:36.016 DEBUG:teuthology.orchestra.run.smithi039:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-21T07:33:36.091 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-09-21T07:33:36.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:36.092 DEBUG:teuthology.orchestra.run.smithi039:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:33:36.287 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:33:36.288 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:33:36.323 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:33:36.323 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:33:36.323 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: resolved to: "172.21.15.39:3300,172.21.15.148:3300,172.21.15.148:3301" 2024-09-21T07:33:36.323 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: trying mount with new device syntax: 2@7ab9b04a-f88d-49aa-a493-e5c076637ffb.cephfs=/ 2024-09-21T07:33:36.324 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.39:3300/172.21.15.148:3300/172.21.15.148:3301" will pass to kernel 2024-09-21T07:33:36.325 INFO:teuthology.orchestra.run.smithi039.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-09-21T07:33:36.325 INFO:teuthology.orchestra.run.smithi039.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:33:36.325 INFO:teuthology.orchestra.run.smithi039.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:33:36.325 INFO:teuthology.orchestra.run.smithi039.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:33:36.325 INFO:teuthology.orchestra.run.smithi039.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:33:36.328 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:33:36.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:36.328 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-21T07:33:36.383 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:33:36.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:36 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:33:36.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:33:36.472 INFO:teuthology.orchestra.run.smithi039.stdout:client4583 2024-09-21T07:33:36.473 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:33:36.473 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:33:36.508 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:33:36.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:36 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:33:36.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:33:36.639 INFO:teuthology.orchestra.run.smithi039.stdout:client4583 2024-09-21T07:33:36.639 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:33:36.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:36 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:33:36.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:33:36.728 INFO:teuthology.orchestra.run.smithi039.stdout:client4583 2024-09-21T07:33:36.728 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/7ab9b04a-f88d-49aa-a493-e5c076637ffb.client4583/status') 2024-09-21T07:33:36.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:33:36.828 INFO:teuthology.orchestra.run.smithi039.stdout:instance: client.4583 (3)172.21.15.39:0/3269640952 2024-09-21T07:33:36.828 INFO:teuthology.orchestra.run.smithi039.stdout:blocklisted: false 2024-09-21T07:33:36.828 INFO:teuthology.orchestra.run.smithi039.stderr:0+1 records in 2024-09-21T07:33:36.828 INFO:teuthology.orchestra.run.smithi039.stderr:0+1 records out 2024-09-21T07:33:36.828 INFO:teuthology.orchestra.run.smithi039.stderr:70 bytes copied, 5.8177e-05 s, 1.2 MB/s 2024-09-21T07:33:36.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:33:36.838 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-09-21T07:33:36.838 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T07:33:36.838 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T07:33:36.838 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:33:37.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.062+0000 7fa555a00640 1 -- 172.21.15.39:0/2047623513 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5500cefd0 msgr2=0x7fa5500cf3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:37.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.062+0000 7fa555a00640 1 --2- 172.21.15.39:0/2047623513 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5500cefd0 0x7fa5500cf3e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa538005fd0 tx=0x7fa53802a540 comp rx=0 tx=0).stop 2024-09-21T07:33:37.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.062+0000 7fa555a00640 1 -- 172.21.15.39:0/2047623513 shutdown_connections 2024-09-21T07:33:37.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.062+0000 7fa555a00640 1 --2- 172.21.15.39:0/2047623513 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5500cefd0 0x7fa5500cf3e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:37.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.062+0000 7fa555a00640 1 -- 172.21.15.39:0/2047623513 >> 172.21.15.39:0/2047623513 conn(0x7fa55009d680 msgr2=0x7fa5500d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:37.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.062+0000 7fa555a00640 1 -- 172.21.15.39:0/2047623513 shutdown_connections 2024-09-21T07:33:37.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.062+0000 7fa555a00640 1 -- 172.21.15.39:0/2047623513 wait complete. 2024-09-21T07:33:37.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.064+0000 7fa555a00640 1 Processor -- start 2024-09-21T07:33:37.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.064+0000 7fa555a00640 1 -- start start 2024-09-21T07:33:37.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.066+0000 7fa555a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5500b42f0 0x7fa5501f6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:37.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.066+0000 7fa555a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5500b96b0 0x7fa55015e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:37.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.066+0000 7fa555a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa5500cefd0 0x7fa5501f8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:37.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.066+0000 7fa555a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa5500b5040 con 0x7fa5500b96b0 2024-09-21T07:33:37.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.066+0000 7fa555a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa5500b4ec0 con 0x7fa5500b42f0 2024-09-21T07:33:37.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.066+0000 7fa555a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa55016a0a0 con 0x7fa5500cefd0 2024-09-21T07:33:37.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.066+0000 7fa547e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5500b96b0 0x7fa55015e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:37.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.066+0000 7fa547e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5500b96b0 0x7fa55015e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36396/0 (socket says 172.21.15.39:36396) 2024-09-21T07:33:37.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.066+0000 7fa547e00640 1 -- 172.21.15.39:0/1494683626 learned_addr learned my addr 172.21.15.39:0/1494683626 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:37.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.066+0000 7fa555000640 1 --2- 172.21.15.39:0/1494683626 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa5500cefd0 0x7fa5501f8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:37.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.066+0000 7fa54fe00640 1 --2- 172.21.15.39:0/1494683626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5500b42f0 0x7fa5501f6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:37.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.066+0000 7fa547e00640 1 -- 172.21.15.39:0/1494683626 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa5500cefd0 msgr2=0x7fa5501f8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:37.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.066+0000 7fa547e00640 1 --2- 172.21.15.39:0/1494683626 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa5500cefd0 0x7fa5501f8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:37.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.066+0000 7fa547e00640 1 -- 172.21.15.39:0/1494683626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5500b42f0 msgr2=0x7fa5501f6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:37.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.066+0000 7fa547e00640 1 --2- 172.21.15.39:0/1494683626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5500b42f0 0x7fa5501f6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:37.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.066+0000 7fa547e00640 1 -- 172.21.15.39:0/1494683626 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5501f6680 con 0x7fa5500b96b0 2024-09-21T07:33:37.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.067+0000 7fa555000640 1 --2- 172.21.15.39:0/1494683626 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa5500cefd0 0x7fa5501f8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:33:37.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.067+0000 7fa547e00640 1 --2- 172.21.15.39:0/1494683626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5500b96b0 0x7fa55015e880 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa538006010 tx=0x7fa538003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:37.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.067+0000 7fa54cc00640 1 -- 172.21.15.39:0/1494683626 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53803e030 con 0x7fa5500b96b0 2024-09-21T07:33:37.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.067+0000 7fa54cc00640 1 -- 172.21.15.39:0/1494683626 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa5380047a0 con 0x7fa5500b96b0 2024-09-21T07:33:37.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.067+0000 7fa555a00640 1 -- 172.21.15.39:0/1494683626 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa550168940 con 0x7fa5500b96b0 2024-09-21T07:33:37.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.067+0000 7fa54cc00640 1 -- 172.21.15.39:0/1494683626 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa538035040 con 0x7fa5500b96b0 2024-09-21T07:33:37.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.067+0000 7fa555a00640 1 -- 172.21.15.39:0/1494683626 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa550168be0 con 0x7fa5500b96b0 2024-09-21T07:33:37.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.068+0000 7fa555a00640 1 -- 172.21.15.39:0/1494683626 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5500cf3e0 con 0x7fa5500b96b0 2024-09-21T07:33:37.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.069+0000 7fa54cc00640 1 -- 172.21.15.39:0/1494683626 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa538041030 con 0x7fa5500b96b0 2024-09-21T07:33:37.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.070+0000 7fa54cc00640 1 --2- 172.21.15.39:0/1494683626 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa52c078bc0 0x7fa52c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:37.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.071+0000 7fa54cc00640 1 -- 172.21.15.39:0/1494683626 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa5380bb6b0 con 0x7fa5500b96b0 2024-09-21T07:33:37.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.071+0000 7fa54fe00640 1 --2- 172.21.15.39:0/1494683626 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa52c078bc0 0x7fa52c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:37.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.072+0000 7fa54fe00640 1 --2- 172.21.15.39:0/1494683626 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa52c078bc0 0x7fa52c07b0d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa5500b5780 tx=0x7fa540009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:37.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.076+0000 7fa54cc00640 1 -- 172.21.15.39:0/1494683626 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa5501f6680 con 0x7fa5500b96b0 2024-09-21T07:33:37.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.315+0000 7fa555a00640 1 -- 172.21.15.39:0/1494683626 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa5500d0860 con 0x7fa5500b96b0 2024-09-21T07:33:37.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.316+0000 7fa54cc00640 1 -- 172.21.15.39:0/1494683626 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fa53808c020 con 0x7fa5500b96b0 2024-09-21T07:33:37.318 INFO:teuthology.orchestra.run.smithi039.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:33:37.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.318+0000 7fa555a00640 1 -- 172.21.15.39:0/1494683626 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa52c078bc0 msgr2=0x7fa52c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:37.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.318+0000 7fa555a00640 1 --2- 172.21.15.39:0/1494683626 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa52c078bc0 0x7fa52c07b0d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa5500b5780 tx=0x7fa540009040 comp rx=0 tx=0).stop 2024-09-21T07:33:37.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.318+0000 7fa555a00640 1 -- 172.21.15.39:0/1494683626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5500b96b0 msgr2=0x7fa55015e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:37.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.318+0000 7fa555a00640 1 --2- 172.21.15.39:0/1494683626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5500b96b0 0x7fa55015e880 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa538006010 tx=0x7fa538003970 comp rx=0 tx=0).stop 2024-09-21T07:33:37.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.318+0000 7fa555a00640 1 -- 172.21.15.39:0/1494683626 shutdown_connections 2024-09-21T07:33:37.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.318+0000 7fa555a00640 1 --2- 172.21.15.39:0/1494683626 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa52c078bc0 0x7fa52c07b0d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:37.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.318+0000 7fa555a00640 1 --2- 172.21.15.39:0/1494683626 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa5500cefd0 0x7fa5501f8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:37.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.318+0000 7fa555a00640 1 --2- 172.21.15.39:0/1494683626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5500b96b0 0x7fa55015e880 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:37.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.318+0000 7fa555a00640 1 --2- 172.21.15.39:0/1494683626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5500b42f0 0x7fa5501f6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:37.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.318+0000 7fa555a00640 1 -- 172.21.15.39:0/1494683626 >> 172.21.15.39:0/1494683626 conn(0x7fa55009d680 msgr2=0x7fa5500b8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:37.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.318+0000 7fa555a00640 1 -- 172.21.15.39:0/1494683626 shutdown_connections 2024-09-21T07:33:37.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:37.318+0000 7fa555a00640 1 -- 172.21.15.39:0/1494683626 wait complete. 2024-09-21T07:33:37.342 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T07:33:37.342 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:33:37.343 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2024-09-21T07:33:37.343 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T07:33:37.343 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T07:33:37.343 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T07:33:37.343 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T07:33:37.343 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:33:37.343 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T07:33:37.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:37.343 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2024-09-21T07:33:37.383 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:33:37.383 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:33:37.383 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:33:37.383 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:33:37.383 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-09-21T07:33:37.383 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:33:37.383 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:33:37.383 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:33:37.383 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2024-09-21T07:33:37.383 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:33:37.383 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:33:37.383 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2024-09-21T07:33:37.384 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:33:37.384 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:33:37.384 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:33:37.384 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:33:37.384 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T07:33:37.384 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42668sec preferred_lft 42668sec 2024-09-21T07:33:37.384 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::ec4:7aff:fe88:7257/64 scope link noprefixroute 2024-09-21T07:33:37.384 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:33:37.385 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T07:33:37.385 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:33:37.385 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-09-21T07:33:37.385 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link add name ceph-brx type bridge 2024-09-21T07:33:37.385 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip addr flush dev ceph-brx 2024-09-21T07:33:37.385 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set ceph-brx up 2024-09-21T07:33:37.385 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T07:33:37.386 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-09-21T07:33:37.508 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:33:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:33:37.805 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:33:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:33:37.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:37.814 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:33:37.872 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-09-21T07:33:37.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:37.875 DEBUG:teuthology.orchestra.run.smithi148:> route 2024-09-21T07:33:37.922 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2024-09-21T07:33:37.922 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:33:37.922 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T07:33:37.923 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T07:33:37.923 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:33:37.923 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:33:37.923 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-09-21T07:33:37.923 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T07:33:37.923 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T07:33:37.923 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T07:33:37.923 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-09-21T07:33:38.001 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:33:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:33:38.543 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:33:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:33:38.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:38.550 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-09-21T07:33:38.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:38.586 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list-id 2024-09-21T07:33:38.662 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:33:38.662 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-09-21T07:33:38.662 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T07:33:38.662 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-21T07:33:38.663 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-09-21T07:33:38.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:33:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:33:38.833 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:33:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:33:38.843 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-21T07:33:38.844 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:33:38.844 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-09-21T07:33:38.844 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-21T07:33:38.844 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T07:33:38.845 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-21T07:33:38.846 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-21T07:33:38.846 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-21T07:33:38.846 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-09-21T07:33:38.931 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:33:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:33:39.096 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:33:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:33:39.105 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:33:39.106 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-09-21T07:33:39.106 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.0 up 2024-09-21T07:33:39.106 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T07:33:39.106 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-09-21T07:33:39.197 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:33:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:33:39.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:33:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:33:39.286 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:33:39.286 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T07:33:39.322 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T07:33:39.323 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:33:39.323 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T07:33:39.399 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-21T07:33:39.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:39.399 DEBUG:teuthology.orchestra.run.smithi148:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:33:39.608 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:33:39.608 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:33:40.283 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:33:40.283 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:33:40.283 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.39:3300,172.21.15.148:3300,172.21.15.148:3301" 2024-09-21T07:33:40.283 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with new device syntax: 1@7ab9b04a-f88d-49aa-a493-e5c076637ffb.cephfs=/ 2024-09-21T07:33:40.283 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.39:3300/172.21.15.148:3300/172.21.15.148:3301" will pass to kernel 2024-09-21T07:33:40.285 INFO:teuthology.orchestra.run.smithi148.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-21T07:33:40.285 INFO:teuthology.orchestra.run.smithi148.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:33:40.285 INFO:teuthology.orchestra.run.smithi148.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:33:40.285 INFO:teuthology.orchestra.run.smithi148.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:33:40.285 INFO:teuthology.orchestra.run.smithi148.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:33:40.288 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:33:40.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:40.289 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T07:33:40.345 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:33:40.425 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:33:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:33:40.430 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:33:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:33:40.438 INFO:teuthology.orchestra.run.smithi148.stdout:client4589 2024-09-21T07:33:40.438 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:33:40.438 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:33:40.474 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:33:40.592 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:33:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:33:40.595 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:33:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:33:40.603 INFO:teuthology.orchestra.run.smithi148.stdout:client4589 2024-09-21T07:33:40.604 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:33:40.681 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:33:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:33:40.684 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:33:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:33:40.692 INFO:teuthology.orchestra.run.smithi148.stdout:client4589 2024-09-21T07:33:40.692 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/7ab9b04a-f88d-49aa-a493-e5c076637ffb.client4589/status') 2024-09-21T07:33:40.769 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:33:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:33:40.789 INFO:teuthology.orchestra.run.smithi148.stdout:instance: client.4589 (3)192.168.0.1:0/2492199524 2024-09-21T07:33:40.789 INFO:teuthology.orchestra.run.smithi148.stdout:blocklisted: false 2024-09-21T07:33:40.790 INFO:teuthology.orchestra.run.smithi148.stderr:0+1 records in 2024-09-21T07:33:40.790 INFO:teuthology.orchestra.run.smithi148.stderr:0+1 records out 2024-09-21T07:33:40.790 INFO:teuthology.orchestra.run.smithi148.stderr:69 bytes copied, 6.9019e-05 s, 1000 kB/s 2024-09-21T07:33:40.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:33:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:33:40.800 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-09-21T07:33:40.800 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T07:33:40.800 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-21T07:33:40.800 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:33:41.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.037+0000 7faec0800640 1 -- 172.21.15.39:0/1527286211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeb80ccb50 msgr2=0x7faeb80ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:41.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.037+0000 7faec0800640 1 --2- 172.21.15.39:0/1527286211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeb80ccb50 0x7faeb80ccf60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7faeb8070f20 tx=0x7faeb402aa30 comp rx=0 tx=0).stop 2024-09-21T07:33:41.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.037+0000 7faec0800640 1 -- 172.21.15.39:0/1527286211 shutdown_connections 2024-09-21T07:33:41.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.037+0000 7faec0800640 1 --2- 172.21.15.39:0/1527286211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeb80ccb50 0x7faeb80ccf60 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:41.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.037+0000 7faec0800640 1 --2- 172.21.15.39:0/1527286211 >> v2:172.21.15.148:3300/0 conn(0x7faeb815d8a0 0x7faeb80b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:41.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.037+0000 7faec0800640 1 --2- 172.21.15.39:0/1527286211 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faeb80cc1e0 0x7faeb80af3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:41.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.037+0000 7faec0800640 1 -- 172.21.15.39:0/1527286211 >> 172.21.15.39:0/1527286211 conn(0x7faeb806d7d0 msgr2=0x7faeb80b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:41.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.037+0000 7faec0800640 1 -- 172.21.15.39:0/1527286211 shutdown_connections 2024-09-21T07:33:41.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.037+0000 7faec0800640 1 -- 172.21.15.39:0/1527286211 wait complete. 2024-09-21T07:33:41.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.039+0000 7faec0800640 1 Processor -- start 2024-09-21T07:33:41.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.040+0000 7faec0800640 1 -- start start 2024-09-21T07:33:41.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.041+0000 7faec0800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faeb80cc1e0 0x7faeb81f8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:41.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.041+0000 7faec0800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faeb80ccb50 0x7faeb806f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:41.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.041+0000 7faec0800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeb815d8a0 0x7faeb81442b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:41.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.041+0000 7faec0800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7faeb80b4770 con 0x7faeb815d8a0 2024-09-21T07:33:41.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.041+0000 7faec0800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7faeb80b45f0 con 0x7faeb80ccb50 2024-09-21T07:33:41.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.041+0000 7faec0800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7faeb81533a0 con 0x7faeb80cc1e0 2024-09-21T07:33:41.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.041+0000 7faebe800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeb815d8a0 0x7faeb81442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:41.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.041+0000 7faebe800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeb815d8a0 0x7faeb81442b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36406/0 (socket says 172.21.15.39:36406) 2024-09-21T07:33:41.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.041+0000 7faebe800640 1 -- 172.21.15.39:0/2127368769 learned_addr learned my addr 172.21.15.39:0/2127368769 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:41.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.042+0000 7faebd400640 1 --2- 172.21.15.39:0/2127368769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faeb80ccb50 0x7faeb806f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:41.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.042+0000 7faebde00640 1 --2- 172.21.15.39:0/2127368769 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faeb80cc1e0 0x7faeb81f8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:41.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.042+0000 7faebe800640 1 -- 172.21.15.39:0/2127368769 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faeb80cc1e0 msgr2=0x7faeb81f8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:41.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.042+0000 7faebe800640 1 --2- 172.21.15.39:0/2127368769 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faeb80cc1e0 0x7faeb81f8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:41.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.042+0000 7faebe800640 1 -- 172.21.15.39:0/2127368769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faeb80ccb50 msgr2=0x7faeb806f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:41.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.042+0000 7faebe800640 1 --2- 172.21.15.39:0/2127368769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faeb80ccb50 0x7faeb806f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:41.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.042+0000 7faebe800640 1 -- 172.21.15.39:0/2127368769 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeb81f97e0 con 0x7faeb815d8a0 2024-09-21T07:33:41.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.042+0000 7faebe800640 1 --2- 172.21.15.39:0/2127368769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeb815d8a0 0x7faeb81442b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faeb4009d60 tx=0x7faeb4003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:41.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.043+0000 7faeb2000640 1 -- 172.21.15.39:0/2127368769 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb4033050 con 0x7faeb815d8a0 2024-09-21T07:33:41.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.043+0000 7faeb2000640 1 -- 172.21.15.39:0/2127368769 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faeb4004450 con 0x7faeb815d8a0 2024-09-21T07:33:41.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.043+0000 7faec0800640 1 -- 172.21.15.39:0/2127368769 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeb81f9ac0 con 0x7faeb815d8a0 2024-09-21T07:33:41.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.043+0000 7faeb2000640 1 -- 172.21.15.39:0/2127368769 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb4038450 con 0x7faeb815d8a0 2024-09-21T07:33:41.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.043+0000 7faec0800640 1 -- 172.21.15.39:0/2127368769 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeb815e9c0 con 0x7faeb815d8a0 2024-09-21T07:33:41.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.044+0000 7faec0800640 1 -- 172.21.15.39:0/2127368769 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faeb81f9550 con 0x7faeb815d8a0 2024-09-21T07:33:41.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.045+0000 7faeb2000640 1 -- 172.21.15.39:0/2127368769 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7faeb4036070 con 0x7faeb815d8a0 2024-09-21T07:33:41.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.046+0000 7faeb2000640 1 --2- 172.21.15.39:0/2127368769 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae9c078bc0 0x7fae9c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:41.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.046+0000 7faeb2000640 1 -- 172.21.15.39:0/2127368769 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7faeb40bc020 con 0x7faeb815d8a0 2024-09-21T07:33:41.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.046+0000 7faebde00640 1 --2- 172.21.15.39:0/2127368769 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae9c078bc0 0x7fae9c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:41.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.047+0000 7faebde00640 1 --2- 172.21.15.39:0/2127368769 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae9c078bc0 0x7fae9c07b0d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7faeac009ab0 tx=0x7faeac005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:41.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.050+0000 7faeb2000640 1 -- 172.21.15.39:0/2127368769 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7faeb4083e80 con 0x7faeb815d8a0 2024-09-21T07:33:41.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.290+0000 7faec0800640 1 -- 172.21.15.39:0/2127368769 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7faeb80ccf60 con 0x7faeb815d8a0 2024-09-21T07:33:41.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.294+0000 7faeb2000640 1 -- 172.21.15.39:0/2127368769 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7faeb40888b0 con 0x7faeb815d8a0 2024-09-21T07:33:41.296 INFO:teuthology.orchestra.run.smithi039.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:33:41.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.296+0000 7faec0800640 1 -- 172.21.15.39:0/2127368769 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae9c078bc0 msgr2=0x7fae9c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:41.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.296+0000 7faec0800640 1 --2- 172.21.15.39:0/2127368769 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae9c078bc0 0x7fae9c07b0d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7faeac009ab0 tx=0x7faeac005f90 comp rx=0 tx=0).stop 2024-09-21T07:33:41.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.296+0000 7faec0800640 1 -- 172.21.15.39:0/2127368769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeb815d8a0 msgr2=0x7faeb81442b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:41.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.296+0000 7faec0800640 1 --2- 172.21.15.39:0/2127368769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeb815d8a0 0x7faeb81442b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faeb4009d60 tx=0x7faeb4003f10 comp rx=0 tx=0).stop 2024-09-21T07:33:41.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.296+0000 7faec0800640 1 -- 172.21.15.39:0/2127368769 shutdown_connections 2024-09-21T07:33:41.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.296+0000 7faec0800640 1 --2- 172.21.15.39:0/2127368769 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae9c078bc0 0x7fae9c07b0d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:41.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.296+0000 7faec0800640 1 --2- 172.21.15.39:0/2127368769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faeb815d8a0 0x7faeb81442b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:41.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.297+0000 7faec0800640 1 --2- 172.21.15.39:0/2127368769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faeb80ccb50 0x7faeb806f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:41.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.297+0000 7faec0800640 1 --2- 172.21.15.39:0/2127368769 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faeb80cc1e0 0x7faeb81f8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:41.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.297+0000 7faec0800640 1 -- 172.21.15.39:0/2127368769 >> 172.21.15.39:0/2127368769 conn(0x7faeb806d7d0 msgr2=0x7faeb806fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:41.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.297+0000 7faec0800640 1 -- 172.21.15.39:0/2127368769 shutdown_connections 2024-09-21T07:33:41.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:41.297+0000 7faec0800640 1 -- 172.21.15.39:0/2127368769 wait complete. 2024-09-21T07:33:41.322 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T07:33:41.322 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:33:41.322 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi148.front.sepia.ceph.com 2024-09-21T07:33:41.322 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-21T07:33:41.323 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-21T07:33:41.323 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T07:33:41.323 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T07:33:41.323 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:33:41.323 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-21T07:33:41.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:41.323 DEBUG:teuthology.orchestra.run.smithi148:> ip addr 2024-09-21T07:33:41.359 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:33:41.360 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:33:41.360 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:33:41.360 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:33:41.360 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-09-21T07:33:41.360 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:33:41.360 INFO:teuthology.orchestra.run.smithi148.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:33:41.360 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a4 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:33:41.360 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f0 2024-09-21T07:33:41.360 INFO:teuthology.orchestra.run.smithi148.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:33:41.360 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:da:6a:a5 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:33:41.360 INFO:teuthology.orchestra.run.smithi148.stdout: altname enp7s0f1 2024-09-21T07:33:41.360 INFO:teuthology.orchestra.run.smithi148.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:33:41.360 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:56 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:33:41.360 INFO:teuthology.orchestra.run.smithi148.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:33:41.361 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether 0c:c4:7a:88:72:57 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:33:41.361 INFO:teuthology.orchestra.run.smithi148.stdout: inet 172.21.15.148/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T07:33:41.361 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft 42664sec preferred_lft 42664sec 2024-09-21T07:33:41.361 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::ec4:7aff:fe88:7257/64 scope link noprefixroute 2024-09-21T07:33:41.361 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:33:41.361 INFO:teuthology.orchestra.run.smithi148.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:33:41.361 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether ce:20:a1:16:2a:4f brd ff:ff:ff:ff:ff:ff 2024-09-21T07:33:41.361 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:33:41.361 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:33:41.361 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::7cb1:3eff:fee8:eaa/64 scope link 2024-09-21T07:33:41.361 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:33:41.361 INFO:teuthology.orchestra.run.smithi148.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:33:41.361 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether ce:20:a1:16:2a:4f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T07:33:41.361 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::cc20:a1ff:fe16:2a4f/64 scope link 2024-09-21T07:33:41.362 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:33:41.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:41.363 DEBUG:teuthology.orchestra.run.smithi148:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:33:41.457 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-09-21T07:33:41.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:41.460 DEBUG:teuthology.orchestra.run.smithi148:> route 2024-09-21T07:33:41.496 INFO:teuthology.orchestra.run.smithi148.stdout:Kernel IP routing table 2024-09-21T07:33:41.496 INFO:teuthology.orchestra.run.smithi148.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:33:41.497 INFO:teuthology.orchestra.run.smithi148.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T07:33:41.497 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T07:33:41.497 INFO:teuthology.orchestra.run.smithi148.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:33:41.497 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:33:41.497 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-09-21T07:33:41.497 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T07:33:41.497 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T07:33:41.497 DEBUG:teuthology.orchestra.run.smithi148:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T07:33:41.497 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-09-21T07:33:41.615 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:33:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:33:41.685 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:33:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:33:41.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:41.694 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-09-21T07:33:41.728 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T07:33:41.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:41.730 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list-id 2024-09-21T07:33:41.804 INFO:teuthology.orchestra.run.smithi148.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-21T07:33:41.805 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:33:41.805 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-09-21T07:33:41.806 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T07:33:41.806 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-21T07:33:41.806 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-09-21T07:33:41.922 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:33:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:33:41.972 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:33:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:33:41.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:41.980 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-21T07:33:42.036 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:33:42.036 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:33:42.036 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:33:42.036 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:33:42.036 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-09-21T07:33:42.036 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:33:42.036 INFO:teuthology.orchestra.run.smithi148.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:33:42.036 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether e2:b8:0f:38:2a:0e brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-21T07:33:42.037 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-21T07:33:42.037 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:33:42.037 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::e0b8:fff:fe38:2a0e/64 scope link 2024-09-21T07:33:42.037 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:33:42.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:42.039 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-21T07:33:42.094 INFO:teuthology.orchestra.run.smithi148.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:33:42.094 INFO:teuthology.orchestra.run.smithi148.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:33:42.095 INFO:teuthology.orchestra.run.smithi148.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:33:42.095 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:33:42.095 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 ::1/128 scope host 2024-09-21T07:33:42.095 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:33:42.095 INFO:teuthology.orchestra.run.smithi148.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:33:42.095 INFO:teuthology.orchestra.run.smithi148.stdout: link/ether e2:b8:0f:38:2a:0e brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-21T07:33:42.095 INFO:teuthology.orchestra.run.smithi148.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-21T07:33:42.095 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:33:42.095 INFO:teuthology.orchestra.run.smithi148.stdout: inet6 fe80::e0b8:fff:fe38:2a0e/64 scope link 2024-09-21T07:33:42.095 INFO:teuthology.orchestra.run.smithi148.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:33:42.097 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-21T07:33:42.097 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:33:42.097 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-09-21T07:33:42.097 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-21T07:33:42.098 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-21T07:33:42.098 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-21T07:33:42.098 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-21T07:33:42.098 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-21T07:33:42.098 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-09-21T07:33:42.176 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:33:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:33:42.307 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:33:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:33:42.315 DEBUG:teuthology.orchestra.run.smithi148:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:33:42.316 DEBUG:teuthology.orchestra.run.smithi148:> set -e 2024-09-21T07:33:42.316 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set brx.1 up 2024-09-21T07:33:42.316 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link set dev brx.1 master ceph-brx 2024-09-21T07:33:42.316 DEBUG:teuthology.orchestra.run.smithi148:> ') 2024-09-21T07:33:42.394 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:33:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:33:42.474 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:33:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:33:42.482 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:33:42.483 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-21T07:33:42.518 INFO:teuthology.orchestra.run.smithi148.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-21T07:33:42.518 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:33:42.518 DEBUG:teuthology.orchestra.run.smithi148:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-21T07:33:42.593 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-09-21T07:33:42.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:42.593 DEBUG:teuthology.orchestra.run.smithi148:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:33:42.787 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:33:42.787 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:33:42.822 INFO:teuthology.orchestra.run.smithi148.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:33:42.823 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:33:42.823 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: resolved to: "172.21.15.39:3300,172.21.15.148:3300,172.21.15.148:3301" 2024-09-21T07:33:42.823 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: trying mount with new device syntax: 3@7ab9b04a-f88d-49aa-a493-e5c076637ffb.cephfs=/ 2024-09-21T07:33:42.823 INFO:teuthology.orchestra.run.smithi148.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.39:3300/172.21.15.148:3300/172.21.15.148:3301" will pass to kernel 2024-09-21T07:33:42.824 INFO:teuthology.orchestra.run.smithi148.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-09-21T07:33:42.824 INFO:teuthology.orchestra.run.smithi148.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:33:42.824 INFO:teuthology.orchestra.run.smithi148.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:33:42.824 INFO:teuthology.orchestra.run.smithi148.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:33:42.824 INFO:teuthology.orchestra.run.smithi148.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:33:42.827 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:33:42.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:42.828 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-21T07:33:42.883 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:33:42.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:33:42 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:33:42.965 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:33:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:33:42.972 INFO:teuthology.orchestra.run.smithi148.stdout:client4731 2024-09-21T07:33:42.973 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:33:42.973 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:33:43.008 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:33:43.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:33:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:33:43.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:33:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:33:43.136 INFO:teuthology.orchestra.run.smithi148.stdout:client4731 2024-09-21T07:33:43.137 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:33:43.215 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:33:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:33:43.218 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:33:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:33:43.226 INFO:teuthology.orchestra.run.smithi148.stdout:client4731 2024-09-21T07:33:43.226 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/7ab9b04a-f88d-49aa-a493-e5c076637ffb.client4731/status') 2024-09-21T07:33:43.304 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:33:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:33:43.326 INFO:teuthology.orchestra.run.smithi148.stdout:instance: client.4731 (3)172.21.15.148:0/2867641806 2024-09-21T07:33:43.326 INFO:teuthology.orchestra.run.smithi148.stdout:blocklisted: false 2024-09-21T07:33:43.326 INFO:teuthology.orchestra.run.smithi148.stderr:0+1 records in 2024-09-21T07:33:43.326 INFO:teuthology.orchestra.run.smithi148.stderr:0+1 records out 2024-09-21T07:33:43.326 INFO:teuthology.orchestra.run.smithi148.stderr:71 bytes copied, 5.8458e-05 s, 1.2 MB/s 2024-09-21T07:33:43.328 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-21T07:33:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:33:43.337 INFO:teuthology.run_tasks:Running task exec... 2024-09-21T07:33:43.345 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T07:33:43.345 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T07:33:43.345 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/crash/warn_recent_interval 0' 2024-09-21T07:33:43.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.579+0000 7f2abaa00640 1 -- 172.21.15.39:0/2543725274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ab4156910 msgr2=0x7f2ab406f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:43.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.579+0000 7f2abaa00640 1 --2- 172.21.15.39:0/2543725274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ab4156910 0x7f2ab406f3f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2ab40aeb90 tx=0x7f2aa802a560 comp rx=0 tx=0).stop 2024-09-21T07:33:43.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.579+0000 7f2abaa00640 1 -- 172.21.15.39:0/2543725274 shutdown_connections 2024-09-21T07:33:43.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.579+0000 7f2abaa00640 1 --2- 172.21.15.39:0/2543725274 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ab40acf60 0x7f2ab40ad370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:43.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.579+0000 7f2abaa00640 1 --2- 172.21.15.39:0/2543725274 >> v2:172.21.15.148:3300/0 conn(0x7f2ab40af050 0x7f2ab40af460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:43.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.579+0000 7f2abaa00640 1 --2- 172.21.15.39:0/2543725274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ab4156910 0x7f2ab406f3f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:43.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.579+0000 7f2abaa00640 1 -- 172.21.15.39:0/2543725274 >> 172.21.15.39:0/2543725274 conn(0x7f2ab40b1c80 msgr2=0x7f2ab40b2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:43.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.579+0000 7f2abaa00640 1 -- 172.21.15.39:0/2543725274 shutdown_connections 2024-09-21T07:33:43.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.579+0000 7f2abaa00640 1 -- 172.21.15.39:0/2543725274 wait complete. 2024-09-21T07:33:43.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.582+0000 7f2abaa00640 1 Processor -- start 2024-09-21T07:33:43.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.582+0000 7f2abaa00640 1 -- start start 2024-09-21T07:33:43.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.583+0000 7f2abaa00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ab40acf60 0x7f2ab415bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:43.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.583+0000 7f2abaa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ab40af050 0x7f2ab4157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:43.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.583+0000 7f2abaa00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ab4156910 0x7f2ab415aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:43.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.583+0000 7f2abaa00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab4074030 con 0x7f2ab40af050 2024-09-21T07:33:43.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.583+0000 7f2abaa00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab4073eb0 con 0x7f2ab4156910 2024-09-21T07:33:43.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.583+0000 7f2abaa00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f2ab40741b0 con 0x7f2ab40acf60 2024-09-21T07:33:43.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.583+0000 7f2ab8c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ab40af050 0x7f2ab4157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:43.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.583+0000 7f2ab8c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ab40af050 0x7f2ab4157e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35350/0 (socket says 172.21.15.39:35350) 2024-09-21T07:33:43.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.583+0000 7f2ab8c00640 1 -- 172.21.15.39:0/3656707767 learned_addr learned my addr 172.21.15.39:0/3656707767 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:43.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.584+0000 7f2ab9600640 1 --2- 172.21.15.39:0/3656707767 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ab40acf60 0x7f2ab415bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:43.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.584+0000 7f2aba000640 1 --2- 172.21.15.39:0/3656707767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ab4156910 0x7f2ab415aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:43.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.584+0000 7f2ab8c00640 1 -- 172.21.15.39:0/3656707767 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ab40acf60 msgr2=0x7f2ab415bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:43.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.584+0000 7f2ab8c00640 1 --2- 172.21.15.39:0/3656707767 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ab40acf60 0x7f2ab415bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:43.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.584+0000 7f2ab8c00640 1 -- 172.21.15.39:0/3656707767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ab4156910 msgr2=0x7f2ab415aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:43.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.584+0000 7f2ab8c00640 1 --2- 172.21.15.39:0/3656707767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ab4156910 0x7f2ab415aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:43.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.584+0000 7f2ab8c00640 1 -- 172.21.15.39:0/3656707767 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab41f4300 con 0x7f2ab40af050 2024-09-21T07:33:43.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.584+0000 7f2ab8c00640 1 --2- 172.21.15.39:0/3656707767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ab40af050 0x7f2ab4157e80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2aa000d770 tx=0x7f2aa000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:43.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.584+0000 7f2ab1600640 1 -- 172.21.15.39:0/3656707767 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa0017070 con 0x7f2ab40af050 2024-09-21T07:33:43.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.585+0000 7f2ab1600640 1 -- 172.21.15.39:0/3656707767 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2aa0009d20 con 0x7f2ab40af050 2024-09-21T07:33:43.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.585+0000 7f2abaa00640 1 -- 172.21.15.39:0/3656707767 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ab41570c0 con 0x7f2ab40af050 2024-09-21T07:33:43.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.585+0000 7f2ab1600640 1 -- 172.21.15.39:0/3656707767 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa0004280 con 0x7f2ab40af050 2024-09-21T07:33:43.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.585+0000 7f2abaa00640 1 -- 172.21.15.39:0/3656707767 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ab4157680 con 0x7f2ab40af050 2024-09-21T07:33:43.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.586+0000 7f2abaa00640 1 -- 172.21.15.39:0/3656707767 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ab40717b0 con 0x7f2ab40af050 2024-09-21T07:33:43.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.587+0000 7f2ab1600640 1 -- 172.21.15.39:0/3656707767 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2aa0005020 con 0x7f2ab40af050 2024-09-21T07:33:43.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.588+0000 7f2ab1600640 1 --2- 172.21.15.39:0/3656707767 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2a94078bc0 0x7f2a9407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:43.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.588+0000 7f2ab1600640 1 -- 172.21.15.39:0/3656707767 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2aa009af50 con 0x7f2ab40af050 2024-09-21T07:33:43.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.589+0000 7f2ab9600640 1 --2- 172.21.15.39:0/3656707767 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2a94078bc0 0x7f2a9407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:43.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.589+0000 7f2ab9600640 1 --2- 172.21.15.39:0/3656707767 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2a94078bc0 0x7f2a9407b0d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8033000 tx=0x7f2aa8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:43.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.593+0000 7f2ab1600640 1 -- 172.21.15.39:0/3656707767 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2ab40717b0 con 0x7f2ab40af050 2024-09-21T07:33:43.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.789+0000 7f2abaa00640 1 -- 172.21.15.39:0/3656707767 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mgr/crash/warn_recent_interval}] v 0) -- 0x7f2ab4074700 con 0x7f2ab40af050 2024-09-21T07:33:43.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.797+0000 7f2ab1600640 1 -- 172.21.15.39:0/3656707767 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/crash/warn_recent_interval}]=0 v32)=0 v32) ==== 128+0+0 (secure 0 0 0) 0x7f2aa0020020 con 0x7f2ab40af050 2024-09-21T07:33:43.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.799+0000 7f2abaa00640 1 -- 172.21.15.39:0/3656707767 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2a94078bc0 msgr2=0x7f2a9407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:43.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.799+0000 7f2abaa00640 1 --2- 172.21.15.39:0/3656707767 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2a94078bc0 0x7f2a9407b0d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8033000 tx=0x7f2aa8004730 comp rx=0 tx=0).stop 2024-09-21T07:33:43.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.799+0000 7f2abaa00640 1 -- 172.21.15.39:0/3656707767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ab40af050 msgr2=0x7f2ab4157e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:43.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.799+0000 7f2abaa00640 1 --2- 172.21.15.39:0/3656707767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ab40af050 0x7f2ab4157e80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2aa000d770 tx=0x7f2aa000dc30 comp rx=0 tx=0).stop 2024-09-21T07:33:43.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.799+0000 7f2abaa00640 1 -- 172.21.15.39:0/3656707767 shutdown_connections 2024-09-21T07:33:43.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.799+0000 7f2abaa00640 1 --2- 172.21.15.39:0/3656707767 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2a94078bc0 0x7f2a9407b0d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:43.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.799+0000 7f2abaa00640 1 --2- 172.21.15.39:0/3656707767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ab4156910 0x7f2ab415aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:43.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.799+0000 7f2abaa00640 1 --2- 172.21.15.39:0/3656707767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ab40af050 0x7f2ab4157e80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:43.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.799+0000 7f2abaa00640 1 --2- 172.21.15.39:0/3656707767 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ab40acf60 0x7f2ab415bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:43.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.799+0000 7f2abaa00640 1 -- 172.21.15.39:0/3656707767 >> 172.21.15.39:0/3656707767 conn(0x7f2ab40b1c80 msgr2=0x7f2ab415efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:43.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.799+0000 7f2abaa00640 1 -- 172.21.15.39:0/3656707767 shutdown_connections 2024-09-21T07:33:43.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:43.800+0000 7f2abaa00640 1 -- 172.21.15.39:0/3656707767 wait complete. 2024-09-21T07:33:43.825 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-21T07:33:45.459 INFO:tasks.cephfs_test_runner:Starting test: test_attaching_pools_with_snaps_to_fs_fails (tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools) Test that attempt to attach pool with snapshots to an fs fails 2024-09-21T07:33:45.459 INFO:tasks.cephfs_test_runner:test_attaching_pools_with_snaps_to_fs_fails (tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools) 2024-09-21T07:33:45.460 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:33:45.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.695+0000 7f5b72c00640 1 -- 172.21.15.39:0/3826247644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b54004800 msgr2=0x7f5b54004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:45.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.695+0000 7f5b72c00640 1 --2- 172.21.15.39:0/3826247644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b54004800 0x7f5b54004c10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5b58009d00 tx=0x7f5b54005830 comp rx=0 tx=0).stop 2024-09-21T07:33:45.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.695+0000 7f5b72c00640 1 -- 172.21.15.39:0/3826247644 shutdown_connections 2024-09-21T07:33:45.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.695+0000 7f5b72c00640 1 --2- 172.21.15.39:0/3826247644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b54004800 0x7f5b54004c10 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:45.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.695+0000 7f5b72c00640 1 -- 172.21.15.39:0/3826247644 >> 172.21.15.39:0/3826247644 conn(0x7f5b6c06d870 msgr2=0x7f5b6c06ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:45.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.695+0000 7f5b72c00640 1 -- 172.21.15.39:0/3826247644 shutdown_connections 2024-09-21T07:33:45.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.695+0000 7f5b72c00640 1 -- 172.21.15.39:0/3826247644 wait complete. 2024-09-21T07:33:45.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.698+0000 7f5b72c00640 1 Processor -- start 2024-09-21T07:33:45.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.698+0000 7f5b72c00640 1 -- start start 2024-09-21T07:33:45.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.699+0000 7f5b72c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b54004800 0x7f5b6c13ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:45.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.699+0000 7f5b72c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b6c0c7a00 0x7f5b6c0c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:45.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.699+0000 7f5b72c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b6c0c98d0 0x7f5b6c0c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:45.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.699+0000 7f5b72c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c153690 con 0x7f5b6c0c98d0 2024-09-21T07:33:45.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.699+0000 7f5b72c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c153510 con 0x7f5b6c0c7a00 2024-09-21T07:33:45.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.699+0000 7f5b72c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f5b6c153810 con 0x7f5b54004800 2024-09-21T07:33:45.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.700+0000 7f5b72200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b6c0c98d0 0x7f5b6c0c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:45.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.700+0000 7f5b72200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b6c0c98d0 0x7f5b6c0c9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35368/0 (socket says 172.21.15.39:35368) 2024-09-21T07:33:45.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.700+0000 7f5b72200640 1 -- 172.21.15.39:0/3383985047 learned_addr learned my addr 172.21.15.39:0/3383985047 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:45.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.700+0000 7f5b70e00640 1 --2- 172.21.15.39:0/3383985047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b6c0c7a00 0x7f5b6c0c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:45.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.700+0000 7f5b71800640 1 --2- 172.21.15.39:0/3383985047 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b54004800 0x7f5b6c13ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:45.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.700+0000 7f5b72200640 1 -- 172.21.15.39:0/3383985047 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b54004800 msgr2=0x7f5b6c13ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:45.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.700+0000 7f5b72200640 1 --2- 172.21.15.39:0/3383985047 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b54004800 0x7f5b6c13ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:45.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.700+0000 7f5b72200640 1 -- 172.21.15.39:0/3383985047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b6c0c7a00 msgr2=0x7f5b6c0c7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:45.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.700+0000 7f5b72200640 1 --2- 172.21.15.39:0/3383985047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b6c0c7a00 0x7f5b6c0c7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:45.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.700+0000 7f5b72200640 1 -- 172.21.15.39:0/3383985047 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b6c0ca220 con 0x7f5b6c0c98d0 2024-09-21T07:33:45.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.700+0000 7f5b72200640 1 --2- 172.21.15.39:0/3383985047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b6c0c98d0 0x7f5b6c0c9ce0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c0752d0 tx=0x7f5b64006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:45.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.701+0000 7f5b69600640 1 -- 172.21.15.39:0/3383985047 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b64011700 con 0x7f5b6c0c98d0 2024-09-21T07:33:45.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.701+0000 7f5b69600640 1 -- 172.21.15.39:0/3383985047 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b640118c0 con 0x7f5b6c0c98d0 2024-09-21T07:33:45.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.701+0000 7f5b69600640 1 -- 172.21.15.39:0/3383985047 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b64011d10 con 0x7f5b6c0c98d0 2024-09-21T07:33:45.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.701+0000 7f5b72c00640 1 -- 172.21.15.39:0/3383985047 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b6c1f3090 con 0x7f5b6c0c98d0 2024-09-21T07:33:45.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.701+0000 7f5b72c00640 1 -- 172.21.15.39:0/3383985047 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b6c1f3630 con 0x7f5b6c0c98d0 2024-09-21T07:33:45.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.702+0000 7f5b72c00640 1 -- 172.21.15.39:0/3383985047 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b6c0b5930 con 0x7f5b6c0c98d0 2024-09-21T07:33:45.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.703+0000 7f5b69600640 1 -- 172.21.15.39:0/3383985047 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5b6401a020 con 0x7f5b6c0c98d0 2024-09-21T07:33:45.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.704+0000 7f5b69600640 1 --2- 172.21.15.39:0/3383985047 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5b4c078b90 0x7f5b4c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:45.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.704+0000 7f5b69600640 1 -- 172.21.15.39:0/3383985047 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5b6409b090 con 0x7f5b6c0c98d0 2024-09-21T07:33:45.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.705+0000 7f5b71800640 1 --2- 172.21.15.39:0/3383985047 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5b4c078b90 0x7f5b4c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:45.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.705+0000 7f5b71800640 1 --2- 172.21.15.39:0/3383985047 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5b4c078b90 0x7f5b4c07b0a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5b60000c00 tx=0x7f5b60006b10 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:45.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.709+0000 7f5b69600640 1 -- 172.21.15.39:0/3383985047 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5b6c0ca220 con 0x7f5b6c0c98d0 2024-09-21T07:33:45.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.906+0000 7f5b72c00640 1 -- 172.21.15.39:0/3383985047 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b6c0ccfb0 con 0x7f5b6c0c98d0 2024-09-21T07:33:45.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.908+0000 7f5b69600640 1 -- 172.21.15.39:0/3383985047 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16665 (secure 0 0 0) 0x7f5b64063430 con 0x7f5b6c0c98d0 2024-09-21T07:33:45.910 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:33:45.910 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":21,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:33:16.159267+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:33:12.854113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:33:13.622611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:33:45.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.910+0000 7f5b72c00640 1 -- 172.21.15.39:0/3383985047 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5b4c078b90 msgr2=0x7f5b4c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:45.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.910+0000 7f5b72c00640 1 --2- 172.21.15.39:0/3383985047 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5b4c078b90 0x7f5b4c07b0a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5b60000c00 tx=0x7f5b60006b10 comp rx=0 tx=0).stop 2024-09-21T07:33:45.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.910+0000 7f5b72c00640 1 -- 172.21.15.39:0/3383985047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b6c0c98d0 msgr2=0x7f5b6c0c9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:45.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.910+0000 7f5b72c00640 1 --2- 172.21.15.39:0/3383985047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b6c0c98d0 0x7f5b6c0c9ce0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c0752d0 tx=0x7f5b64006430 comp rx=0 tx=0).stop 2024-09-21T07:33:45.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.910+0000 7f5b72c00640 1 -- 172.21.15.39:0/3383985047 shutdown_connections 2024-09-21T07:33:45.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.910+0000 7f5b72c00640 1 --2- 172.21.15.39:0/3383985047 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5b4c078b90 0x7f5b4c07b0a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:45.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.910+0000 7f5b72c00640 1 --2- 172.21.15.39:0/3383985047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b6c0c98d0 0x7f5b6c0c9ce0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:45.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.910+0000 7f5b72c00640 1 --2- 172.21.15.39:0/3383985047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b6c0c7a00 0x7f5b6c0c7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:45.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.910+0000 7f5b72c00640 1 --2- 172.21.15.39:0/3383985047 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b54004800 0x7f5b6c13ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:45.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.910+0000 7f5b72c00640 1 -- 172.21.15.39:0/3383985047 >> 172.21.15.39:0/3383985047 conn(0x7f5b6c06d870 msgr2=0x7f5b6c15d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:45.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.910+0000 7f5b72c00640 1 -- 172.21.15.39:0/3383985047 shutdown_connections 2024-09-21T07:33:45.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:45.910+0000 7f5b72c00640 1 -- 172.21.15.39:0/3383985047 wait complete. 2024-09-21T07:33:45.935 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T07:33:01.569433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T07:33:12.854113+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T07:33:13.622611+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T07:33:45.935 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T07:33:46.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.163+0000 7fd713c00640 1 -- 172.21.15.39:0/1341216815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd70c15d8a0 msgr2=0x7fd70c0b40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:46.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.163+0000 7fd713c00640 1 --2- 172.21.15.39:0/1341216815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd70c15d8a0 0x7fd70c0b40b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd70c0cd710 tx=0x7fd6f802a560 comp rx=0 tx=0).stop 2024-09-21T07:33:46.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.163+0000 7fd713c00640 1 -- 172.21.15.39:0/1341216815 shutdown_connections 2024-09-21T07:33:46.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.163+0000 7fd713c00640 1 --2- 172.21.15.39:0/1341216815 >> v2:172.21.15.148:3300/0 conn(0x7fd70c0ccb50 0x7fd70c0ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:46.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.163+0000 7fd713c00640 1 --2- 172.21.15.39:0/1341216815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd70c15d8a0 0x7fd70c0b40b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:46.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.163+0000 7fd713c00640 1 --2- 172.21.15.39:0/1341216815 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd70c0cc1e0 0x7fd70c0af3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:46.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.163+0000 7fd713c00640 1 -- 172.21.15.39:0/1341216815 >> 172.21.15.39:0/1341216815 conn(0x7fd70c06d7d0 msgr2=0x7fd70c0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:46.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.164+0000 7fd713c00640 1 -- 172.21.15.39:0/1341216815 shutdown_connections 2024-09-21T07:33:46.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.164+0000 7fd713c00640 1 -- 172.21.15.39:0/1341216815 wait complete. 2024-09-21T07:33:46.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.166+0000 7fd713c00640 1 Processor -- start 2024-09-21T07:33:46.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.166+0000 7fd713c00640 1 -- start start 2024-09-21T07:33:46.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.168+0000 7fd713c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd70c0cc1e0 0x7fd70c1f6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:46.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.168+0000 7fd713c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd70c0ccb50 0x7fd70c09d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:46.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.168+0000 7fd713c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd70c15d8a0 0x7fd70c0ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:46.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.168+0000 7fd713c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd70c0b4770 con 0x7fd70c0ccb50 2024-09-21T07:33:46.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.168+0000 7fd713c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd70c0b45f0 con 0x7fd70c0cc1e0 2024-09-21T07:33:46.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.168+0000 7fd713c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd70c1533a0 con 0x7fd70c15d8a0 2024-09-21T07:33:46.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.168+0000 7fd70be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd70c0ccb50 0x7fd70c09d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:46.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.168+0000 7fd70be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd70c0ccb50 0x7fd70c09d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35386/0 (socket says 172.21.15.39:35386) 2024-09-21T07:33:46.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.168+0000 7fd70be00640 1 -- 172.21.15.39:0/2365872054 learned_addr learned my addr 172.21.15.39:0/2365872054 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:46.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.168+0000 7fd711200640 1 --2- 172.21.15.39:0/2365872054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd70c0cc1e0 0x7fd70c1f6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:46.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.168+0000 7fd711c00640 1 --2- 172.21.15.39:0/2365872054 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd70c15d8a0 0x7fd70c0ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:46.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.168+0000 7fd70be00640 1 -- 172.21.15.39:0/2365872054 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd70c15d8a0 msgr2=0x7fd70c0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:46.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.168+0000 7fd70be00640 1 --2- 172.21.15.39:0/2365872054 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd70c15d8a0 0x7fd70c0ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:46.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.168+0000 7fd70be00640 1 -- 172.21.15.39:0/2365872054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd70c0cc1e0 msgr2=0x7fd70c1f6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:46.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.168+0000 7fd70be00640 1 --2- 172.21.15.39:0/2365872054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd70c0cc1e0 0x7fd70c1f6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:46.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.168+0000 7fd70be00640 1 -- 172.21.15.39:0/2365872054 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd70c0aea40 con 0x7fd70c0ccb50 2024-09-21T07:33:46.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.169+0000 7fd70be00640 1 --2- 172.21.15.39:0/2365872054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd70c0ccb50 0x7fd70c09d5b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8000c00 tx=0x7fd6f8003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:46.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.169+0000 7fd708c00640 1 -- 172.21.15.39:0/2365872054 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f803e030 con 0x7fd70c0ccb50 2024-09-21T07:33:46.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.169+0000 7fd708c00640 1 -- 172.21.15.39:0/2365872054 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6f8003e00 con 0x7fd70c0ccb50 2024-09-21T07:33:46.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.169+0000 7fd713c00640 1 -- 172.21.15.39:0/2365872054 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd70c09abd0 con 0x7fd70c0ccb50 2024-09-21T07:33:46.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.169+0000 7fd708c00640 1 -- 172.21.15.39:0/2365872054 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8035040 con 0x7fd70c0ccb50 2024-09-21T07:33:46.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.169+0000 7fd713c00640 1 -- 172.21.15.39:0/2365872054 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd70c09ae70 con 0x7fd70c0ccb50 2024-09-21T07:33:46.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.171+0000 7fd713c00640 1 -- 172.21.15.39:0/2365872054 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd70c0ccf60 con 0x7fd70c0ccb50 2024-09-21T07:33:46.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.172+0000 7fd708c00640 1 -- 172.21.15.39:0/2365872054 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd6f8041030 con 0x7fd70c0ccb50 2024-09-21T07:33:46.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.173+0000 7fd708c00640 1 --2- 172.21.15.39:0/2365872054 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd6ec078b90 0x7fd6ec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:46.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.173+0000 7fd708c00640 1 -- 172.21.15.39:0/2365872054 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd6f80bb4e0 con 0x7fd70c0ccb50 2024-09-21T07:33:46.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.173+0000 7fd711200640 1 --2- 172.21.15.39:0/2365872054 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd6ec078b90 0x7fd6ec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:46.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.174+0000 7fd711200640 1 --2- 172.21.15.39:0/2365872054 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd6ec078b90 0x7fd6ec07b0a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd700005fd0 tx=0x7fd70000b260 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:46.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.178+0000 7fd708c00640 1 -- 172.21.15.39:0/2365872054 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd6f8083360 con 0x7fd70c0ccb50 2024-09-21T07:33:46.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.375+0000 7fd713c00640 1 -- 172.21.15.39:0/2365872054 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd70c09ded0 con 0x7fd70c0ccb50 2024-09-21T07:33:46.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.376+0000 7fd708c00640 1 -- 172.21.15.39:0/2365872054 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7fd6f8087d90 con 0x7fd70c0ccb50 2024-09-21T07:33:46.378 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-09-21T07:33:46.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.378+0000 7fd713c00640 1 -- 172.21.15.39:0/2365872054 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd6ec078b90 msgr2=0x7fd6ec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:46.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.378+0000 7fd713c00640 1 --2- 172.21.15.39:0/2365872054 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd6ec078b90 0x7fd6ec07b0a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd700005fd0 tx=0x7fd70000b260 comp rx=0 tx=0).stop 2024-09-21T07:33:46.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.378+0000 7fd713c00640 1 -- 172.21.15.39:0/2365872054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd70c0ccb50 msgr2=0x7fd70c09d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:46.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.378+0000 7fd713c00640 1 --2- 172.21.15.39:0/2365872054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd70c0ccb50 0x7fd70c09d5b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8000c00 tx=0x7fd6f8003910 comp rx=0 tx=0).stop 2024-09-21T07:33:46.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.379+0000 7fd713c00640 1 -- 172.21.15.39:0/2365872054 shutdown_connections 2024-09-21T07:33:46.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.379+0000 7fd713c00640 1 --2- 172.21.15.39:0/2365872054 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd6ec078b90 0x7fd6ec07b0a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:46.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.379+0000 7fd713c00640 1 --2- 172.21.15.39:0/2365872054 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd70c15d8a0 0x7fd70c0ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:46.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.379+0000 7fd713c00640 1 --2- 172.21.15.39:0/2365872054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd70c0ccb50 0x7fd70c09d5b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:46.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.379+0000 7fd713c00640 1 --2- 172.21.15.39:0/2365872054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd70c0cc1e0 0x7fd70c1f6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:46.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.379+0000 7fd713c00640 1 -- 172.21.15.39:0/2365872054 >> 172.21.15.39:0/2365872054 conn(0x7fd70c06d7d0 msgr2=0x7fd70c06f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:46.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.379+0000 7fd713c00640 1 -- 172.21.15.39:0/2365872054 shutdown_connections 2024-09-21T07:33:46.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.379+0000 7fd713c00640 1 -- 172.21.15.39:0/2365872054 wait complete. 2024-09-21T07:33:46.406 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T07:33:46.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.634+0000 7fc1ab200640 1 -- 172.21.15.39:0/3576065804 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1ac1654a0 msgr2=0x7fc1ac1678c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:46.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.634+0000 7fc1ab200640 1 --2- 172.21.15.39:0/3576065804 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1ac1654a0 0x7fc1ac1678c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4005fd0 tx=0x7fc1a402a820 comp rx=0 tx=0).stop 2024-09-21T07:33:46.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.634+0000 7fc1ab200640 1 -- 172.21.15.39:0/3576065804 shutdown_connections 2024-09-21T07:33:46.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.634+0000 7fc1ab200640 1 --2- 172.21.15.39:0/3576065804 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1ac1654a0 0x7fc1ac1678c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:46.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.634+0000 7fc1ab200640 1 -- 172.21.15.39:0/3576065804 >> 172.21.15.39:0/3576065804 conn(0x7fc1ac0cebf0 msgr2=0x7fc1ac0cf000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:46.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.634+0000 7fc1ab200640 1 -- 172.21.15.39:0/3576065804 shutdown_connections 2024-09-21T07:33:46.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.634+0000 7fc1ab200640 1 -- 172.21.15.39:0/3576065804 wait complete. 2024-09-21T07:33:46.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.636+0000 7fc1ab200640 1 Processor -- start 2024-09-21T07:33:46.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.636+0000 7fc1ab200640 1 -- start start 2024-09-21T07:33:46.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.638+0000 7fc1ab200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1ac0b72a0 0x7fc1ac0b5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:46.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.638+0000 7fc1ab200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1ac0d0550 0x7fc1ac15f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:46.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.638+0000 7fc1ab200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc1ac1654a0 0x7fc1ac15f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:46.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.638+0000 7fc1ab200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ac0b7b00 con 0x7fc1ac0b72a0 2024-09-21T07:33:46.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.638+0000 7fc1ab200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ac0b76b0 con 0x7fc1ac0d0550 2024-09-21T07:33:46.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.638+0000 7fc1ab200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc1ac168010 con 0x7fc1ac1654a0 2024-09-21T07:33:46.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.638+0000 7fc1a9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1ac0b72a0 0x7fc1ac0b5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:46.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.638+0000 7fc1a9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1ac0b72a0 0x7fc1ac0b5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35404/0 (socket says 172.21.15.39:35404) 2024-09-21T07:33:46.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.638+0000 7fc1a9e00640 1 -- 172.21.15.39:0/1013262558 learned_addr learned my addr 172.21.15.39:0/1013262558 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:46.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.638+0000 7fc1aa800640 1 --2- 172.21.15.39:0/1013262558 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc1ac1654a0 0x7fc1ac15f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:46.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.638+0000 7fc1a1400640 1 --2- 172.21.15.39:0/1013262558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1ac0d0550 0x7fc1ac15f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:46.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.639+0000 7fc1a9e00640 1 -- 172.21.15.39:0/1013262558 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc1ac1654a0 msgr2=0x7fc1ac15f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:46.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.639+0000 7fc1a9e00640 1 --2- 172.21.15.39:0/1013262558 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc1ac1654a0 0x7fc1ac15f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:46.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.639+0000 7fc1a9e00640 1 -- 172.21.15.39:0/1013262558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1ac0d0550 msgr2=0x7fc1ac15f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:46.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.639+0000 7fc1a9e00640 1 --2- 172.21.15.39:0/1013262558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1ac0d0550 0x7fc1ac15f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:46.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.639+0000 7fc1a9e00640 1 -- 172.21.15.39:0/1013262558 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1ac15e320 con 0x7fc1ac0b72a0 2024-09-21T07:33:46.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.639+0000 7fc1aa800640 1 --2- 172.21.15.39:0/1013262558 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc1ac1654a0 0x7fc1ac15f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:33:46.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.639+0000 7fc1a9e00640 1 --2- 172.21.15.39:0/1013262558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1ac0b72a0 0x7fc1ac0b5d70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00c3c0 tx=0x7fc19c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:46.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.639+0000 7fc1a2a00640 1 -- 172.21.15.39:0/1013262558 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c017070 con 0x7fc1ac0b72a0 2024-09-21T07:33:46.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.639+0000 7fc1a2a00640 1 -- 172.21.15.39:0/1013262558 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc19c0053c0 con 0x7fc1ac0b72a0 2024-09-21T07:33:46.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.639+0000 7fc1ab200640 1 -- 172.21.15.39:0/1013262558 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1ac15e600 con 0x7fc1ac0b72a0 2024-09-21T07:33:46.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.639+0000 7fc1a2a00640 1 -- 172.21.15.39:0/1013262558 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c003880 con 0x7fc1ac0b72a0 2024-09-21T07:33:46.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.639+0000 7fc1ab200640 1 -- 172.21.15.39:0/1013262558 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1ac1f76b0 con 0x7fc1ac0b72a0 2024-09-21T07:33:46.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.640+0000 7fc1ab200640 1 -- 172.21.15.39:0/1013262558 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1ac1f7910 con 0x7fc1ac0b72a0 2024-09-21T07:33:46.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.642+0000 7fc1a2a00640 1 -- 172.21.15.39:0/1013262558 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc19c01a070 con 0x7fc1ac0b72a0 2024-09-21T07:33:46.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.643+0000 7fc1a2a00640 1 --2- 172.21.15.39:0/1013262558 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc188078b90 0x7fc18807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:46.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.643+0000 7fc1a2a00640 1 -- 172.21.15.39:0/1013262558 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc19c09a100 con 0x7fc1ac0b72a0 2024-09-21T07:33:46.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.643+0000 7fc1a1400640 1 --2- 172.21.15.39:0/1013262558 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc188078b90 0x7fc18807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:46.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.644+0000 7fc1a1400640 1 --2- 172.21.15.39:0/1013262558 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc188078b90 0x7fc18807b0a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac0b4c00 tx=0x7fc194009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:46.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.647+0000 7fc1a2a00640 1 -- 172.21.15.39:0/1013262558 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc19c0624a0 con 0x7fc1ac0b72a0 2024-09-21T07:33:46.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.842+0000 7fc1ab200640 1 -- 172.21.15.39:0/1013262558 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc1ac0b23b0 con 0x7fc1ac0b72a0 2024-09-21T07:33:46.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.843+0000 7fc1a2a00640 1 -- 172.21.15.39:0/1013262558 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7fc19c061da0 con 0x7fc1ac0b72a0 2024-09-21T07:33:46.845 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-21T07:33:46.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.845+0000 7fc1ab200640 1 -- 172.21.15.39:0/1013262558 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc188078b90 msgr2=0x7fc18807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:46.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.845+0000 7fc1ab200640 1 --2- 172.21.15.39:0/1013262558 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc188078b90 0x7fc18807b0a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac0b4c00 tx=0x7fc194009040 comp rx=0 tx=0).stop 2024-09-21T07:33:46.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.845+0000 7fc1ab200640 1 -- 172.21.15.39:0/1013262558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1ac0b72a0 msgr2=0x7fc1ac0b5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:46.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.845+0000 7fc1ab200640 1 --2- 172.21.15.39:0/1013262558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1ac0b72a0 0x7fc1ac0b5d70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00c3c0 tx=0x7fc19c00c880 comp rx=0 tx=0).stop 2024-09-21T07:33:46.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.845+0000 7fc1ab200640 1 -- 172.21.15.39:0/1013262558 shutdown_connections 2024-09-21T07:33:46.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.845+0000 7fc1ab200640 1 --2- 172.21.15.39:0/1013262558 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc188078b90 0x7fc18807b0a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:46.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.845+0000 7fc1ab200640 1 --2- 172.21.15.39:0/1013262558 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc1ac1654a0 0x7fc1ac15f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:46.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.845+0000 7fc1ab200640 1 --2- 172.21.15.39:0/1013262558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1ac0d0550 0x7fc1ac15f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:46.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.845+0000 7fc1ab200640 1 --2- 172.21.15.39:0/1013262558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1ac0b72a0 0x7fc1ac0b5d70 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:46.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.845+0000 7fc1ab200640 1 -- 172.21.15.39:0/1013262558 >> 172.21.15.39:0/1013262558 conn(0x7fc1ac0cebf0 msgr2=0x7fc1ac09b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:46.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.845+0000 7fc1ab200640 1 -- 172.21.15.39:0/1013262558 shutdown_connections 2024-09-21T07:33:46.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:46.845+0000 7fc1ab200640 1 -- 172.21.15.39:0/1013262558 wait complete. 2024-09-21T07:33:46.870 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T07:33:47.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.100+0000 7fb5e2800640 1 -- 172.21.15.39:0/3664607136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5dc0cd050 msgr2=0x7fb5dc15f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:47.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.100+0000 7fb5e2800640 1 --2- 172.21.15.39:0/3664607136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5dc0cd050 0x7fb5dc15f410 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc06f6b0 tx=0x7fb5d402a9e0 comp rx=0 tx=0).stop 2024-09-21T07:33:47.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.100+0000 7fb5e2800640 1 -- 172.21.15.39:0/3664607136 shutdown_connections 2024-09-21T07:33:47.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.100+0000 7fb5e2800640 1 --2- 172.21.15.39:0/3664607136 >> v2:172.21.15.148:3300/0 conn(0x7fb5dc06f2a0 0x7fb5dc0ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:47.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.100+0000 7fb5e2800640 1 --2- 172.21.15.39:0/3664607136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5dc0cd050 0x7fb5dc15f410 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:47.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.100+0000 7fb5e2800640 1 -- 172.21.15.39:0/3664607136 >> 172.21.15.39:0/3664607136 conn(0x7fb5dc06d7d0 msgr2=0x7fb5dc0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:47.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.100+0000 7fb5e2800640 1 -- 172.21.15.39:0/3664607136 shutdown_connections 2024-09-21T07:33:47.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.100+0000 7fb5e2800640 1 -- 172.21.15.39:0/3664607136 wait complete. 2024-09-21T07:33:47.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.102+0000 7fb5e2800640 1 Processor -- start 2024-09-21T07:33:47.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.102+0000 7fb5e2800640 1 -- start start 2024-09-21T07:33:47.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.104+0000 7fb5e2800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5dc06ee90 0x7fb5dc0b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:47.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.104+0000 7fb5e2800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5dc06f2a0 0x7fb5dc14ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:47.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.104+0000 7fb5e2800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5dc0cd050 0x7fb5dc15e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:47.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.104+0000 7fb5e2800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb5dc162730 con 0x7fb5dc06f2a0 2024-09-21T07:33:47.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.104+0000 7fb5e2800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb5dc1625b0 con 0x7fb5dc0cd050 2024-09-21T07:33:47.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.104+0000 7fb5e2800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb5dc15f950 con 0x7fb5dc06ee90 2024-09-21T07:33:47.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.104+0000 7fb5d3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5dc06f2a0 0x7fb5dc14ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:47.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.104+0000 7fb5d3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5dc06f2a0 0x7fb5dc14ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35430/0 (socket says 172.21.15.39:35430) 2024-09-21T07:33:47.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.104+0000 7fb5d3e00640 1 -- 172.21.15.39:0/3635854446 learned_addr learned my addr 172.21.15.39:0/3635854446 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:47.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.104+0000 7fb5dbe00640 1 --2- 172.21.15.39:0/3635854446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5dc0cd050 0x7fb5dc15e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:47.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.104+0000 7fb5db400640 1 --2- 172.21.15.39:0/3635854446 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5dc06ee90 0x7fb5dc0b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:47.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.105+0000 7fb5d3e00640 1 -- 172.21.15.39:0/3635854446 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5dc06ee90 msgr2=0x7fb5dc0b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:47.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.105+0000 7fb5d3e00640 1 --2- 172.21.15.39:0/3635854446 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5dc06ee90 0x7fb5dc0b13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:47.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.105+0000 7fb5d3e00640 1 -- 172.21.15.39:0/3635854446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5dc0cd050 msgr2=0x7fb5dc15e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:47.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.105+0000 7fb5d3e00640 1 --2- 172.21.15.39:0/3635854446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5dc0cd050 0x7fb5dc15e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:47.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.105+0000 7fb5d3e00640 1 -- 172.21.15.39:0/3635854446 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5dc14f180 con 0x7fb5dc06f2a0 2024-09-21T07:33:47.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.105+0000 7fb5d3e00640 1 --2- 172.21.15.39:0/3635854446 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5dc06f2a0 0x7fb5dc14ec40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb5c400d6d0 tx=0x7fb5c400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:47.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.105+0000 7fb5d3400640 1 -- 172.21.15.39:0/3635854446 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c4017070 con 0x7fb5dc06f2a0 2024-09-21T07:33:47.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.105+0000 7fb5d3400640 1 -- 172.21.15.39:0/3635854446 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5c4005330 con 0x7fb5dc06f2a0 2024-09-21T07:33:47.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.105+0000 7fb5d3400640 1 -- 172.21.15.39:0/3635854446 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c4009820 con 0x7fb5dc06f2a0 2024-09-21T07:33:47.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.105+0000 7fb5e2800640 1 -- 172.21.15.39:0/3635854446 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5dc09aa30 con 0x7fb5dc06f2a0 2024-09-21T07:33:47.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.105+0000 7fb5e2800640 1 -- 172.21.15.39:0/3635854446 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5dc09aff0 con 0x7fb5dc06f2a0 2024-09-21T07:33:47.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.106+0000 7fb5e2800640 1 -- 172.21.15.39:0/3635854446 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5dc0750f0 con 0x7fb5dc06f2a0 2024-09-21T07:33:47.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.108+0000 7fb5d3400640 1 -- 172.21.15.39:0/3635854446 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb5c4009bd0 con 0x7fb5dc06f2a0 2024-09-21T07:33:47.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.108+0000 7fb5d3400640 1 --2- 172.21.15.39:0/3635854446 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb5b8078b90 0x7fb5b807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:47.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.109+0000 7fb5d3400640 1 -- 172.21.15.39:0/3635854446 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb5c409b070 con 0x7fb5dc06f2a0 2024-09-21T07:33:47.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.109+0000 7fb5db400640 1 --2- 172.21.15.39:0/3635854446 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb5b8078b90 0x7fb5b807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:47.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.110+0000 7fb5db400640 1 --2- 172.21.15.39:0/3635854446 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb5b8078b90 0x7fb5b807b0a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc006fd0 tx=0x7fb5cc009060 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:47.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.113+0000 7fb5d3400640 1 -- 172.21.15.39:0/3635854446 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb5dc14f180 con 0x7fb5dc06f2a0 2024-09-21T07:33:47.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.312+0000 7fb5e2800640 1 -- 172.21.15.39:0/3635854446 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb5dc160780 con 0x7fb5dc06f2a0 2024-09-21T07:33:47.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.313+0000 7fb5d3400640 1 -- 172.21.15.39:0/3635854446 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7fb5c4062dc0 con 0x7fb5dc06f2a0 2024-09-21T07:33:47.315 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-09-21T07:33:47.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.315+0000 7fb5e2800640 1 -- 172.21.15.39:0/3635854446 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb5b8078b90 msgr2=0x7fb5b807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:47.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.315+0000 7fb5e2800640 1 --2- 172.21.15.39:0/3635854446 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb5b8078b90 0x7fb5b807b0a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc006fd0 tx=0x7fb5cc009060 comp rx=0 tx=0).stop 2024-09-21T07:33:47.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.315+0000 7fb5e2800640 1 -- 172.21.15.39:0/3635854446 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5dc06f2a0 msgr2=0x7fb5dc14ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:47.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.315+0000 7fb5e2800640 1 --2- 172.21.15.39:0/3635854446 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5dc06f2a0 0x7fb5dc14ec40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb5c400d6d0 tx=0x7fb5c400db90 comp rx=0 tx=0).stop 2024-09-21T07:33:47.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.315+0000 7fb5e2800640 1 -- 172.21.15.39:0/3635854446 shutdown_connections 2024-09-21T07:33:47.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.315+0000 7fb5e2800640 1 --2- 172.21.15.39:0/3635854446 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb5b8078b90 0x7fb5b807b0a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:47.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.315+0000 7fb5e2800640 1 --2- 172.21.15.39:0/3635854446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5dc0cd050 0x7fb5dc15e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:47.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.315+0000 7fb5e2800640 1 --2- 172.21.15.39:0/3635854446 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5dc06f2a0 0x7fb5dc14ec40 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:47.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.315+0000 7fb5e2800640 1 --2- 172.21.15.39:0/3635854446 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5dc06ee90 0x7fb5dc0b13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:47.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.315+0000 7fb5e2800640 1 -- 172.21.15.39:0/3635854446 >> 172.21.15.39:0/3635854446 conn(0x7fb5dc06d7d0 msgr2=0x7fb5dc15bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:47.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.315+0000 7fb5e2800640 1 -- 172.21.15.39:0/3635854446 shutdown_connections 2024-09-21T07:33:47.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.315+0000 7fb5e2800640 1 -- 172.21.15.39:0/3635854446 wait complete. 2024-09-21T07:33:47.339 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_attaching_pools_with_snaps_to_fs_fails' 2024-09-21T07:33:47.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.567+0000 7efee7600640 1 -- 172.21.15.39:0/1807697778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efee00cf6f0 msgr2=0x7efee00cc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:47.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.567+0000 7efee7600640 1 --2- 172.21.15.39:0/1807697778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efee00cf6f0 0x7efee00cc4c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7efee00cf170 tx=0x7efed402a560 comp rx=0 tx=0).stop 2024-09-21T07:33:47.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.567+0000 7efee7600640 1 -- 172.21.15.39:0/1807697778 shutdown_connections 2024-09-21T07:33:47.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.567+0000 7efee7600640 1 --2- 172.21.15.39:0/1807697778 >> v2:172.21.15.148:3300/0 conn(0x7efee00ccb30 0x7efee00ccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:47.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.567+0000 7efee7600640 1 --2- 172.21.15.39:0/1807697778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efee00cf6f0 0x7efee00cc4c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:47.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.567+0000 7efee7600640 1 -- 172.21.15.39:0/1807697778 >> 172.21.15.39:0/1807697778 conn(0x7efee00b1cb0 msgr2=0x7efee00b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:47.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.567+0000 7efee7600640 1 -- 172.21.15.39:0/1807697778 shutdown_connections 2024-09-21T07:33:47.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.567+0000 7efee7600640 1 -- 172.21.15.39:0/1807697778 wait complete. 2024-09-21T07:33:47.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.569+0000 7efee7600640 1 Processor -- start 2024-09-21T07:33:47.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.569+0000 7efee7600640 1 -- start start 2024-09-21T07:33:47.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.571+0000 7efee7600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efee00ccb30 0x7efee00d0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:47.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.571+0000 7efee7600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efee00ce020 0x7efee015e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:47.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.571+0000 7efee7600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efee00cf6f0 0x7efee015e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:47.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.571+0000 7efee7600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efee00cecc0 con 0x7efee00ce020 2024-09-21T07:33:47.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.571+0000 7efee7600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efee00ceb40 con 0x7efee00ccb30 2024-09-21T07:33:47.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.571+0000 7efee7600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7efee00cf1c0 con 0x7efee00cf6f0 2024-09-21T07:33:47.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.571+0000 7efedfe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efee00ce020 0x7efee015e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:47.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.571+0000 7efee4c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efee00ccb30 0x7efee00d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:47.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.571+0000 7efedfe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efee00ce020 0x7efee015e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35460/0 (socket says 172.21.15.39:35460) 2024-09-21T07:33:47.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.571+0000 7efedfe00640 1 -- 172.21.15.39:0/3528006007 learned_addr learned my addr 172.21.15.39:0/3528006007 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:47.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.571+0000 7efee5600640 1 --2- 172.21.15.39:0/3528006007 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efee00cf6f0 0x7efee015e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:47.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.572+0000 7efedfe00640 1 -- 172.21.15.39:0/3528006007 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efee00cf6f0 msgr2=0x7efee015e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:47.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.572+0000 7efedfe00640 1 --2- 172.21.15.39:0/3528006007 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efee00cf6f0 0x7efee015e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:47.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.572+0000 7efedfe00640 1 -- 172.21.15.39:0/3528006007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efee00ccb30 msgr2=0x7efee00d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:47.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.572+0000 7efedfe00640 1 --2- 172.21.15.39:0/3528006007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efee00ccb30 0x7efee00d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:47.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.572+0000 7efedfe00640 1 -- 172.21.15.39:0/3528006007 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee01f6210 con 0x7efee00ce020 2024-09-21T07:33:47.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.572+0000 7efedfe00640 1 --2- 172.21.15.39:0/3528006007 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efee00ce020 0x7efee015e300 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efecc00c970 tx=0x7efecc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:47.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.572+0000 7efedcc00640 1 -- 172.21.15.39:0/3528006007 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc007bb0 con 0x7efee00ce020 2024-09-21T07:33:47.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.572+0000 7efedcc00640 1 -- 172.21.15.39:0/3528006007 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efecc007e40 con 0x7efee00ce020 2024-09-21T07:33:47.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.572+0000 7efedcc00640 1 -- 172.21.15.39:0/3528006007 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc00f690 con 0x7efee00ce020 2024-09-21T07:33:47.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.572+0000 7efee7600640 1 -- 172.21.15.39:0/3528006007 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee0161bd0 con 0x7efee00ce020 2024-09-21T07:33:47.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.573+0000 7efee7600640 1 -- 172.21.15.39:0/3528006007 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee009bb40 con 0x7efee00ce020 2024-09-21T07:33:47.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.574+0000 7efee7600640 1 -- 172.21.15.39:0/3528006007 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efee0161a00 con 0x7efee00ce020 2024-09-21T07:33:47.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.575+0000 7efedcc00640 1 -- 172.21.15.39:0/3528006007 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7efecc00f830 con 0x7efee00ce020 2024-09-21T07:33:47.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.576+0000 7efedcc00640 1 --2- 172.21.15.39:0/3528006007 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efec0078bc0 0x7efec007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:47.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.577+0000 7efedcc00640 1 -- 172.21.15.39:0/3528006007 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7efecc09b4a0 con 0x7efee00ce020 2024-09-21T07:33:47.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.577+0000 7efee4c00640 1 --2- 172.21.15.39:0/3528006007 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efec0078bc0 0x7efec007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:47.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.578+0000 7efee4c00640 1 --2- 172.21.15.39:0/3528006007 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efec0078bc0 0x7efec007b0d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efed4033000 tx=0x7efed4004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:47.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.581+0000 7efedcc00640 1 -- 172.21.15.39:0/3528006007 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7efecc014030 con 0x7efee00ce020 2024-09-21T07:33:47.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:47.851+0000 7efee7600640 1 -- 172.21.15.39:0/3528006007 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_attaching_pools_with_snaps_to_fs_fails"]} v 0) -- 0x7efee0070290 con 0x7efee00ce020 2024-09-21T07:33:48.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:48.825+0000 7efedcc00640 1 -- 172.21.15.39:0/3528006007 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_attaching_pools_with_snaps_to_fs_fails"]}]=0 v51) ==== 176+0+0 (secure 0 0 0) 0x7efecc0638d0 con 0x7efee00ce020 2024-09-21T07:33:48.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:48.828+0000 7efee7600640 1 -- 172.21.15.39:0/3528006007 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efec0078bc0 msgr2=0x7efec007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:48.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:48.828+0000 7efee7600640 1 --2- 172.21.15.39:0/3528006007 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efec0078bc0 0x7efec007b0d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efed4033000 tx=0x7efed4004730 comp rx=0 tx=0).stop 2024-09-21T07:33:48.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:48.828+0000 7efee7600640 1 -- 172.21.15.39:0/3528006007 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efee00ce020 msgr2=0x7efee015e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:48.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:48.828+0000 7efee7600640 1 --2- 172.21.15.39:0/3528006007 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efee00ce020 0x7efee015e300 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efecc00c970 tx=0x7efecc00ce30 comp rx=0 tx=0).stop 2024-09-21T07:33:48.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:48.828+0000 7efee7600640 1 -- 172.21.15.39:0/3528006007 shutdown_connections 2024-09-21T07:33:48.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:48.828+0000 7efee7600640 1 --2- 172.21.15.39:0/3528006007 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efec0078bc0 0x7efec007b0d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:48.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:48.828+0000 7efee7600640 1 --2- 172.21.15.39:0/3528006007 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efee00cf6f0 0x7efee015e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:48.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:48.828+0000 7efee7600640 1 --2- 172.21.15.39:0/3528006007 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efee00ce020 0x7efee015e300 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:48.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:48.828+0000 7efee7600640 1 --2- 172.21.15.39:0/3528006007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efee00ccb30 0x7efee00d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:48.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:48.828+0000 7efee7600640 1 -- 172.21.15.39:0/3528006007 >> 172.21.15.39:0/3528006007 conn(0x7efee00b1cb0 msgr2=0x7efee00ce430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:48.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:48.828+0000 7efee7600640 1 -- 172.21.15.39:0/3528006007 shutdown_connections 2024-09-21T07:33:48.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:48.828+0000 7efee7600640 1 -- 172.21.15.39:0/3528006007 wait complete. 2024-09-21T07:33:48.859 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T07:33:49.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.090+0000 7f447a800640 1 -- 172.21.15.39:0/2746526100 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4458004d40 msgr2=0x7f4458005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:49.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.090+0000 7f447a800640 1 --2- 172.21.15.39:0/2746526100 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4458004d40 0x7f4458005150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f446c005fd0 tx=0x7f4458005830 comp rx=0 tx=0).stop 2024-09-21T07:33:49.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.090+0000 7f4473e00640 1 -- 172.21.15.39:0/2746526100 reap_dead start 2024-09-21T07:33:49.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.090+0000 7f447a800640 1 -- 172.21.15.39:0/2746526100 shutdown_connections 2024-09-21T07:33:49.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.090+0000 7f447a800640 1 -- 172.21.15.39:0/2746526100 >> 172.21.15.39:0/2746526100 conn(0x7f44740b1cb0 msgr2=0x7f44740b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:49.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.091+0000 7f447a800640 1 -- 172.21.15.39:0/2746526100 shutdown_connections 2024-09-21T07:33:49.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.091+0000 7f447a800640 1 -- 172.21.15.39:0/2746526100 wait complete. 2024-09-21T07:33:49.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.093+0000 7f447a800640 1 Processor -- start 2024-09-21T07:33:49.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.093+0000 7f447a800640 1 -- start start 2024-09-21T07:33:49.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.094+0000 7f447a800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4458004d40 0x7f447415b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:49.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.094+0000 7f447a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44740c9d70 0x7f44740ca180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:49.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.094+0000 7f447a800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f447415d790 0x7f447415dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:49.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.094+0000 7f447a800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f44740b3420 con 0x7f44740c9d70 2024-09-21T07:33:49.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.094+0000 7f447a800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f44740b32a0 con 0x7f4458004d40 2024-09-21T07:33:49.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.094+0000 7f447a800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f44740b35a0 con 0x7f447415d790 2024-09-21T07:33:49.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.095+0000 7f4472a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44740c9d70 0x7f44740ca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:49.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.095+0000 7f4472a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44740c9d70 0x7f44740ca180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35480/0 (socket says 172.21.15.39:35480) 2024-09-21T07:33:49.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.095+0000 7f4472a00640 1 -- 172.21.15.39:0/2949543499 learned_addr learned my addr 172.21.15.39:0/2949543499 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:49.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.095+0000 7f4472a00640 1 -- 172.21.15.39:0/2949543499 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f447415d790 msgr2=0x7f447415dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:49.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.095+0000 7f4472a00640 1 --2- 172.21.15.39:0/2949543499 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f447415d790 0x7f447415dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:49.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.095+0000 7f4472a00640 1 -- 172.21.15.39:0/2949543499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4458004d40 msgr2=0x7f447415b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:49.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.095+0000 7f4472a00640 1 --2- 172.21.15.39:0/2949543499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4458004d40 0x7f447415b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:49.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.095+0000 7f4472a00640 1 -- 172.21.15.39:0/2949543499 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447415e0e0 con 0x7f44740c9d70 2024-09-21T07:33:49.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.095+0000 7f4472a00640 1 --2- 172.21.15.39:0/2949543499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44740c9d70 0x7f44740ca180 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f44600065c0 tx=0x7f44600066b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:49.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.096+0000 7f446b400640 1 -- 172.21.15.39:0/2949543499 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4460015070 con 0x7f44740c9d70 2024-09-21T07:33:49.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.096+0000 7f446b400640 1 -- 172.21.15.39:0/2949543499 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4460011900 con 0x7f44740c9d70 2024-09-21T07:33:49.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.096+0000 7f446b400640 1 -- 172.21.15.39:0/2949543499 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4460011d50 con 0x7f44740c9d70 2024-09-21T07:33:49.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.096+0000 7f447a800640 1 -- 172.21.15.39:0/2949543499 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44740ca9c0 con 0x7f44740c9d70 2024-09-21T07:33:49.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.096+0000 7f447a800640 1 -- 172.21.15.39:0/2949543499 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44741f59e0 con 0x7f44740c9d70 2024-09-21T07:33:49.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.097+0000 7f447a800640 1 -- 172.21.15.39:0/2949543499 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44740ca730 con 0x7f44740c9d70 2024-09-21T07:33:49.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.098+0000 7f446b400640 1 -- 172.21.15.39:0/2949543499 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4460018070 con 0x7f44740c9d70 2024-09-21T07:33:49.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.100+0000 7f446b400640 1 --2- 172.21.15.39:0/2949543499 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4454078b90 0x7f445407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:49.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.100+0000 7f446b400640 1 -- 172.21.15.39:0/2949543499 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f446009bb90 con 0x7f44740c9d70 2024-09-21T07:33:49.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.100+0000 7f4473400640 1 --2- 172.21.15.39:0/2949543499 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4454078b90 0x7f445407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:49.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.101+0000 7f4473400640 1 --2- 172.21.15.39:0/2949543499 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4454078b90 0x7f445407b0a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4464009bc0 tx=0x7f4464009680 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:49.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.104+0000 7f446b400640 1 -- 172.21.15.39:0/2949543499 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f447415e0e0 con 0x7f44740c9d70 2024-09-21T07:33:49.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.301+0000 7f447a800640 1 -- 172.21.15.39:0/2949543499 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f447415c780 con 0x7f44740c9d70 2024-09-21T07:33:49.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.302+0000 7f446b400640 1 -- 172.21.15.39:0/2949543499 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v32) ==== 59+0+2368 (secure 0 0 0) 0x7f4460063fc0 con 0x7f44740c9d70 2024-09-21T07:33:49.304 INFO:teuthology.orchestra.run.smithi039.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T07:33:49.304 INFO:teuthology.orchestra.run.smithi039.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T07:33:49.304 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced client_mount_timeout 30 2024-09-21T07:33:49.304 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_client 20/20 2024-09-21T07:33:49.304 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_mgr 20/20 2024-09-21T07:33:49.304 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_ms 1/1 2024-09-21T07:33:49.305 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced mgr/crash/warn_recent_interval 0 2024-09-21T07:33:49.305 INFO:teuthology.orchestra.run.smithi039.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T07:33:49.305 INFO:teuthology.orchestra.run.smithi039.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T07:33:49.305 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3421.881759 2024-09-21T07:33:49.305 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3520.252978 2024-09-21T07:33:49.305 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3511.183316 2024-09-21T07:33:49.305 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3829.631847 2024-09-21T07:33:49.305 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 5474.545872 2024-09-21T07:33:49.305 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4013.628559 2024-09-21T07:33:49.305 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4230.187288 2024-09-21T07:33:49.305 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4148.983107 2024-09-21T07:33:49.305 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_mds 20/20 2024-09-21T07:33:49.305 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T07:33:49.306 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_ms 2/2 2024-09-21T07:33:49.306 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_debug_frag true 2024-09-21T07:33:49.306 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_debug_scatterstat true 2024-09-21T07:33:49.306 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T07:33:49.306 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_verify_scatter true 2024-09-21T07:33:49.306 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T07:33:49.306 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T07:33:49.306 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T07:33:49.306 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced client_mount_timeout 600 2024-09-21T07:33:49.306 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced debug_client 20/20 2024-09-21T07:33:49.306 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced debug_ms 1/1 2024-09-21T07:33:49.307 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T07:33:49.307 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T07:33:49.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.304+0000 7f447a800640 1 -- 172.21.15.39:0/2949543499 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4454078b90 msgr2=0x7f445407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:49.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.304+0000 7f447a800640 1 --2- 172.21.15.39:0/2949543499 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4454078b90 0x7f445407b0a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4464009bc0 tx=0x7f4464009680 comp rx=0 tx=0).stop 2024-09-21T07:33:49.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.304+0000 7f447a800640 1 -- 172.21.15.39:0/2949543499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44740c9d70 msgr2=0x7f44740ca180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:49.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.304+0000 7f447a800640 1 --2- 172.21.15.39:0/2949543499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44740c9d70 0x7f44740ca180 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f44600065c0 tx=0x7f44600066b0 comp rx=0 tx=0).stop 2024-09-21T07:33:49.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.304+0000 7f447a800640 1 -- 172.21.15.39:0/2949543499 shutdown_connections 2024-09-21T07:33:49.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.304+0000 7f447a800640 1 --2- 172.21.15.39:0/2949543499 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4454078b90 0x7f445407b0a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:49.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.304+0000 7f447a800640 1 --2- 172.21.15.39:0/2949543499 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f447415d790 0x7f447415dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:49.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.304+0000 7f447a800640 1 --2- 172.21.15.39:0/2949543499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44740c9d70 0x7f44740ca180 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:49.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.304+0000 7f447a800640 1 --2- 172.21.15.39:0/2949543499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4458004d40 0x7f447415b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:49.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.304+0000 7f447a800640 1 -- 172.21.15.39:0/2949543499 >> 172.21.15.39:0/2949543499 conn(0x7f44740b1cb0 msgr2=0x7f44740b4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:49.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.305+0000 7f447a800640 1 -- 172.21.15.39:0/2949543499 shutdown_connections 2024-09-21T07:33:49.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.305+0000 7f447a800640 1 -- 172.21.15.39:0/2949543499 wait complete. 2024-09-21T07:33:49.328 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T07:33:49.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.555+0000 7fb030e00640 1 -- 172.21.15.39:0/219978709 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb02c0cffa0 msgr2=0x7fb02c0d09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:49.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.555+0000 7fb030e00640 1 --2- 172.21.15.39:0/219978709 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb02c0cffa0 0x7fb02c0d09f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb02c0ceca0 tx=0x7fb01402a4d0 comp rx=0 tx=0).stop 2024-09-21T07:33:49.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.555+0000 7fb030e00640 1 -- 172.21.15.39:0/219978709 shutdown_connections 2024-09-21T07:33:49.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.555+0000 7fb030e00640 1 --2- 172.21.15.39:0/219978709 >> v2:172.21.15.148:3300/0 conn(0x7fb02c0d0f30 0x7fb02c0b71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:49.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.555+0000 7fb030e00640 1 --2- 172.21.15.39:0/219978709 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb02c0cffa0 0x7fb02c0d09f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:49.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.555+0000 7fb030e00640 1 --2- 172.21.15.39:0/219978709 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb02c0caf80 0x7fb02c0b4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:49.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.555+0000 7fb030e00640 1 -- 172.21.15.39:0/219978709 >> 172.21.15.39:0/219978709 conn(0x7fb02c09d6b0 msgr2=0x7fb02c0d4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:49.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.555+0000 7fb030e00640 1 -- 172.21.15.39:0/219978709 shutdown_connections 2024-09-21T07:33:49.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.555+0000 7fb030e00640 1 -- 172.21.15.39:0/219978709 wait complete. 2024-09-21T07:33:49.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.557+0000 7fb030e00640 1 Processor -- start 2024-09-21T07:33:49.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.558+0000 7fb030e00640 1 -- start start 2024-09-21T07:33:49.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.559+0000 7fb030e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb02c0caf80 0x7fb02c0b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:49.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.559+0000 7fb030e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb02c0cffa0 0x7fb02c1f8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:49.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.559+0000 7fb030e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb02c0d0f30 0x7fb02c0cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:49.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.559+0000 7fb030e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c0b7880 con 0x7fb02c0d0f30 2024-09-21T07:33:49.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.559+0000 7fb030e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c0b7700 con 0x7fb02c0caf80 2024-09-21T07:33:49.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.559+0000 7fb030e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb02c0b7a00 con 0x7fb02c0cffa0 2024-09-21T07:33:49.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.560+0000 7fb02b400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb02c0caf80 0x7fb02c0b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:49.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.560+0000 7fb02be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb02c0d0f30 0x7fb02c0cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:49.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.560+0000 7fb023e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb02c0cffa0 0x7fb02c1f8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:49.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.560+0000 7fb02b400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb02c0caf80 0x7fb02c0b48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.39:56570/0 (socket says 172.21.15.39:56570) 2024-09-21T07:33:49.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.560+0000 7fb02b400640 1 -- 172.21.15.39:0/3723606160 learned_addr learned my addr 172.21.15.39:0/3723606160 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:49.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.560+0000 7fb02be00640 1 -- 172.21.15.39:0/3723606160 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb02c0cffa0 msgr2=0x7fb02c1f8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:49.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.560+0000 7fb02be00640 1 --2- 172.21.15.39:0/3723606160 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb02c0cffa0 0x7fb02c1f8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:49.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.560+0000 7fb02be00640 1 -- 172.21.15.39:0/3723606160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb02c0caf80 msgr2=0x7fb02c0b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:49.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.560+0000 7fb02be00640 1 --2- 172.21.15.39:0/3723606160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb02c0caf80 0x7fb02c0b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:49.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.560+0000 7fb02be00640 1 -- 172.21.15.39:0/3723606160 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb02c1f9940 con 0x7fb02c0d0f30 2024-09-21T07:33:49.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.560+0000 7fb023e00640 1 --2- 172.21.15.39:0/3723606160 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb02c0cffa0 0x7fb02c1f8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:33:49.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.560+0000 7fb02be00640 1 --2- 172.21.15.39:0/3723606160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb02c0d0f30 0x7fb02c0cc570 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb02400e960 tx=0x7fb02400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:49.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.561+0000 7fb023400640 1 -- 172.21.15.39:0/3723606160 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb024004d20 con 0x7fb02c0d0f30 2024-09-21T07:33:49.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.561+0000 7fb023400640 1 -- 172.21.15.39:0/3723606160 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb02400ca60 con 0x7fb02c0d0f30 2024-09-21T07:33:49.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.561+0000 7fb023400640 1 -- 172.21.15.39:0/3723606160 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb024005020 con 0x7fb02c0d0f30 2024-09-21T07:33:49.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.561+0000 7fb030e00640 1 -- 172.21.15.39:0/3723606160 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb02c1f6980 con 0x7fb02c0d0f30 2024-09-21T07:33:49.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.561+0000 7fb030e00640 1 -- 172.21.15.39:0/3723606160 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb02c1f6f40 con 0x7fb02c0d0f30 2024-09-21T07:33:49.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.562+0000 7fb030e00640 1 -- 172.21.15.39:0/3723606160 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb02c1f96b0 con 0x7fb02c0d0f30 2024-09-21T07:33:49.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.563+0000 7fb023400640 1 -- 172.21.15.39:0/3723606160 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb0240051c0 con 0x7fb02c0d0f30 2024-09-21T07:33:49.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.564+0000 7fb023400640 1 --2- 172.21.15.39:0/3723606160 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb008078b90 0x7fb00807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:49.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.564+0000 7fb023400640 1 -- 172.21.15.39:0/3723606160 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb02409b590 con 0x7fb02c0d0f30 2024-09-21T07:33:49.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.565+0000 7fb02b400640 1 --2- 172.21.15.39:0/3723606160 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb008078b90 0x7fb00807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:49.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.565+0000 7fb02b400640 1 --2- 172.21.15.39:0/3723606160 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb008078b90 0x7fb00807b0a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb01c0058c0 tx=0x7fb01c00b410 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:49.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.569+0000 7fb023400640 1 -- 172.21.15.39:0/3723606160 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb024016030 con 0x7fb02c0d0f30 2024-09-21T07:33:49.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.767+0000 7fb030e00640 1 -- 172.21.15.39:0/3723606160 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb02c0b88e0 con 0x7fb02c0d0f30 2024-09-21T07:33:49.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.771+0000 7fb023400640 1 -- 172.21.15.39:0/3723606160 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v33)=0 v33) ==== 122+0+0 (secure 0 0 0) 0x7fb0240632e0 con 0x7fb02c0d0f30 2024-09-21T07:33:49.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.773+0000 7fb030e00640 1 -- 172.21.15.39:0/3723606160 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb008078b90 msgr2=0x7fb00807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:49.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.773+0000 7fb030e00640 1 --2- 172.21.15.39:0/3723606160 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb008078b90 0x7fb00807b0a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb01c0058c0 tx=0x7fb01c00b410 comp rx=0 tx=0).stop 2024-09-21T07:33:49.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.773+0000 7fb030e00640 1 -- 172.21.15.39:0/3723606160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb02c0d0f30 msgr2=0x7fb02c0cc570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:49.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.773+0000 7fb030e00640 1 --2- 172.21.15.39:0/3723606160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb02c0d0f30 0x7fb02c0cc570 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb02400e960 tx=0x7fb02400ed60 comp rx=0 tx=0).stop 2024-09-21T07:33:49.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.774+0000 7fb030e00640 1 -- 172.21.15.39:0/3723606160 shutdown_connections 2024-09-21T07:33:49.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.774+0000 7fb030e00640 1 --2- 172.21.15.39:0/3723606160 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb008078b90 0x7fb00807b0a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:49.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.774+0000 7fb030e00640 1 --2- 172.21.15.39:0/3723606160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb02c0d0f30 0x7fb02c0cc570 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:49.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.774+0000 7fb030e00640 1 --2- 172.21.15.39:0/3723606160 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb02c0cffa0 0x7fb02c1f8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:49.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.774+0000 7fb030e00640 1 --2- 172.21.15.39:0/3723606160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb02c0caf80 0x7fb02c0b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:49.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.774+0000 7fb030e00640 1 -- 172.21.15.39:0/3723606160 >> 172.21.15.39:0/3723606160 conn(0x7fb02c09d6b0 msgr2=0x7fb02c1e6830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:49.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.774+0000 7fb030e00640 1 -- 172.21.15.39:0/3723606160 shutdown_connections 2024-09-21T07:33:49.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:49.774+0000 7fb030e00640 1 -- 172.21.15.39:0/3723606160 wait complete. 2024-09-21T07:33:49.797 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:33:49.798 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:33:49.898 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:33:49.993 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:33:49.994 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:33:49.994 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:33:50.029 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:33:50.029 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:33:50.105 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:33:50.105 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:33:50.180 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:33:50.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.443+0000 7f6390a00640 1 -- 172.21.15.39:0/3662136753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f638c0ccb50 msgr2=0x7f638c0ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:50.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.443+0000 7f6390a00640 1 --2- 172.21.15.39:0/3662136753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f638c0ccb50 0x7f638c0ccf60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f638c072bc0 tx=0x7f638402aa30 comp rx=0 tx=0).stop 2024-09-21T07:33:50.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.443+0000 7f6390a00640 1 -- 172.21.15.39:0/3662136753 shutdown_connections 2024-09-21T07:33:50.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.443+0000 7f6390a00640 1 --2- 172.21.15.39:0/3662136753 >> v2:172.21.15.148:3300/0 conn(0x7f638c0af3c0 0x7f638c0b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:50.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.443+0000 7f6390a00640 1 --2- 172.21.15.39:0/3662136753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f638c0ccb50 0x7f638c0ccf60 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:50.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.443+0000 7f6390a00640 1 -- 172.21.15.39:0/3662136753 >> 172.21.15.39:0/3662136753 conn(0x7f638c06d7d0 msgr2=0x7f638c0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:50.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.443+0000 7f6390a00640 1 -- 172.21.15.39:0/3662136753 shutdown_connections 2024-09-21T07:33:50.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.443+0000 7f6390a00640 1 -- 172.21.15.39:0/3662136753 wait complete. 2024-09-21T07:33:50.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.446+0000 7f6390a00640 1 Processor -- start 2024-09-21T07:33:50.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.446+0000 7f6390a00640 1 -- start start 2024-09-21T07:33:50.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.447+0000 7f6390a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f638c0af3c0 0x7f638c1f73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:50.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.447+0000 7f6390a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f638c0cc1e0 0x7f638c1f3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:50.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.447+0000 7f6390a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f638c0ccb50 0x7f638c1442b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:50.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.447+0000 7f6390a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f638c153520 con 0x7f638c0ccb50 2024-09-21T07:33:50.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.447+0000 7f6390a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f638c1533a0 con 0x7f638c0af3c0 2024-09-21T07:33:50.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.447+0000 7f6390a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f638c1536a0 con 0x7f638c0cc1e0 2024-09-21T07:33:50.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.448+0000 7f638a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f638c0ccb50 0x7f638c1442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:50.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.448+0000 7f638a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f638c0ccb50 0x7f638c1442b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35516/0 (socket says 172.21.15.39:35516) 2024-09-21T07:33:50.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.448+0000 7f638a800640 1 -- 172.21.15.39:0/1945667113 learned_addr learned my addr 172.21.15.39:0/1945667113 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:50.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.448+0000 7f6381400640 1 --2- 172.21.15.39:0/1945667113 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f638c0cc1e0 0x7f638c1f3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:50.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.448+0000 7f6389e00640 1 --2- 172.21.15.39:0/1945667113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f638c0af3c0 0x7f638c1f73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:50.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.448+0000 7f638a800640 1 -- 172.21.15.39:0/1945667113 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f638c0cc1e0 msgr2=0x7f638c1f3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:50.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.448+0000 7f638a800640 1 --2- 172.21.15.39:0/1945667113 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f638c0cc1e0 0x7f638c1f3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:50.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.448+0000 7f638a800640 1 -- 172.21.15.39:0/1945667113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f638c0af3c0 msgr2=0x7f638c1f73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:50.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.448+0000 7f638a800640 1 --2- 172.21.15.39:0/1945667113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f638c0af3c0 0x7f638c1f73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:50.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.448+0000 7f638a800640 1 -- 172.21.15.39:0/1945667113 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f638c1f4ba0 con 0x7f638c0ccb50 2024-09-21T07:33:50.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.448+0000 7f6389e00640 1 --2- 172.21.15.39:0/1945667113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f638c0af3c0 0x7f638c1f73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:33:50.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.448+0000 7f638a800640 1 --2- 172.21.15.39:0/1945667113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f638c0ccb50 0x7f638c1442b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6384009d60 tx=0x7f6384003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:50.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.449+0000 7f6382a00640 1 -- 172.21.15.39:0/1945667113 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6384033050 con 0x7f638c0ccb50 2024-09-21T07:33:50.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.449+0000 7f6382a00640 1 -- 172.21.15.39:0/1945667113 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6384004550 con 0x7f638c0ccb50 2024-09-21T07:33:50.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.449+0000 7f6382a00640 1 -- 172.21.15.39:0/1945667113 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6384038450 con 0x7f638c0ccb50 2024-09-21T07:33:50.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.449+0000 7f6390a00640 1 -- 172.21.15.39:0/1945667113 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f638c1f48c0 con 0x7f638c0ccb50 2024-09-21T07:33:50.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.449+0000 7f6390a00640 1 -- 172.21.15.39:0/1945667113 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f638c09b0c0 con 0x7f638c0ccb50 2024-09-21T07:33:50.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.450+0000 7f6390a00640 1 -- 172.21.15.39:0/1945667113 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f638c1f4630 con 0x7f638c0ccb50 2024-09-21T07:33:50.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.451+0000 7f6382a00640 1 -- 172.21.15.39:0/1945667113 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6384036070 con 0x7f638c0ccb50 2024-09-21T07:33:50.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.452+0000 7f6382a00640 1 --2- 172.21.15.39:0/1945667113 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6368078b90 0x7f636807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:50.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.452+0000 7f6382a00640 1 -- 172.21.15.39:0/1945667113 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f63840bc310 con 0x7f638c0ccb50 2024-09-21T07:33:50.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.452+0000 7f6389e00640 1 --2- 172.21.15.39:0/1945667113 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6368078b90 0x7f636807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:50.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.453+0000 7f6389e00640 1 --2- 172.21.15.39:0/1945667113 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6368078b90 0x7f636807b0a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f637c006fd0 tx=0x7f637c008040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:50.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.457+0000 7f6382a00640 1 -- 172.21.15.39:0/1945667113 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f638c0cd4a0 con 0x7f638c0ccb50 2024-09-21T07:33:50.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.665+0000 7f6390a00640 1 -- 172.21.15.39:0/1945667113 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f638c070850 con 0x7f638c0ccb50 2024-09-21T07:33:50.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.666+0000 7f6382a00640 1 -- 172.21.15.39:0/1945667113 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7f638c070850 con 0x7f638c0ccb50 2024-09-21T07:33:50.668 INFO:teuthology.orchestra.run.smithi039.stderr:listed 0 entries 2024-09-21T07:33:50.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.669+0000 7f6390a00640 1 -- 172.21.15.39:0/1945667113 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6368078b90 msgr2=0x7f636807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:50.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.669+0000 7f6390a00640 1 --2- 172.21.15.39:0/1945667113 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6368078b90 0x7f636807b0a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f637c006fd0 tx=0x7f637c008040 comp rx=0 tx=0).stop 2024-09-21T07:33:50.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.669+0000 7f6390a00640 1 -- 172.21.15.39:0/1945667113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f638c0ccb50 msgr2=0x7f638c1442b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:50.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.669+0000 7f6390a00640 1 --2- 172.21.15.39:0/1945667113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f638c0ccb50 0x7f638c1442b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6384009d60 tx=0x7f6384003f10 comp rx=0 tx=0).stop 2024-09-21T07:33:50.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.669+0000 7f6390a00640 1 -- 172.21.15.39:0/1945667113 shutdown_connections 2024-09-21T07:33:50.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.669+0000 7f6390a00640 1 --2- 172.21.15.39:0/1945667113 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6368078b90 0x7f636807b0a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:50.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.669+0000 7f6390a00640 1 --2- 172.21.15.39:0/1945667113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f638c0ccb50 0x7f638c1442b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:50.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.669+0000 7f6390a00640 1 --2- 172.21.15.39:0/1945667113 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f638c0cc1e0 0x7f638c1f3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:50.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.669+0000 7f6390a00640 1 --2- 172.21.15.39:0/1945667113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f638c0af3c0 0x7f638c1f73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:50.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.669+0000 7f6390a00640 1 -- 172.21.15.39:0/1945667113 >> 172.21.15.39:0/1945667113 conn(0x7f638c06d7d0 msgr2=0x7f638c06f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:50.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.669+0000 7f6390a00640 1 -- 172.21.15.39:0/1945667113 shutdown_connections 2024-09-21T07:33:50.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:50.669+0000 7f6390a00640 1 -- 172.21.15.39:0/1945667113 wait complete. 2024-09-21T07:33:50.695 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T07:33:50.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:50.695 DEBUG:teuthology.orchestra.run.smithi039:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-21T07:33:50.759 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T07:33:50.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:50.759 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:33:50.804 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:33:50.804 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:33:50.845 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:33:50.845 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:33:50.923 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:33:50.924 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:33:51.001 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:33:51.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.230+0000 7fe102200640 1 -- 172.21.15.39:0/381046592 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0fc0cd0e0 msgr2=0x7fe0fc0d0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:51.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.230+0000 7fe102200640 1 --2- 172.21.15.39:0/381046592 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0fc0cd0e0 0x7fe0fc0d0880 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8009830 tx=0x7fe0e802a510 comp rx=0 tx=0).stop 2024-09-21T07:33:51.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.230+0000 7fe102200640 1 -- 172.21.15.39:0/381046592 shutdown_connections 2024-09-21T07:33:51.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.230+0000 7fe102200640 1 --2- 172.21.15.39:0/381046592 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0fc0cd0e0 0x7fe0fc0d0880 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:51.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.230+0000 7fe102200640 1 --2- 172.21.15.39:0/381046592 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe0fc0ce730 0x7fe0fc0ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:51.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.230+0000 7fe102200640 1 -- 172.21.15.39:0/381046592 >> 172.21.15.39:0/381046592 conn(0x7fe0fc0ceba0 msgr2=0x7fe0fc0cefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:51.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.230+0000 7fe102200640 1 -- 172.21.15.39:0/381046592 shutdown_connections 2024-09-21T07:33:51.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.230+0000 7fe102200640 1 -- 172.21.15.39:0/381046592 wait complete. 2024-09-21T07:33:51.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.232+0000 7fe102200640 1 Processor -- start 2024-09-21T07:33:51.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.232+0000 7fe102200640 1 -- start start 2024-09-21T07:33:51.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.233+0000 7fe102200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0fc0cd0e0 0x7fe0fc165ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:51.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.234+0000 7fe102200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe0fc0ce730 0x7fe0fc164820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:51.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.234+0000 7fe102200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0fc0d0dc0 0x7fe0fc164d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:51.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.234+0000 7fe102200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc0b2130 con 0x7fe0fc0cd0e0 2024-09-21T07:33:51.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.234+0000 7fe102200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc0b22b0 con 0x7fe0fc0d0dc0 2024-09-21T07:33:51.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.234+0000 7fe102200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe0fc0b2430 con 0x7fe0fc0ce730 2024-09-21T07:33:51.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.234+0000 7fe100e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0fc0cd0e0 0x7fe0fc165ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:51.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.234+0000 7fe100e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0fc0cd0e0 0x7fe0fc165ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35534/0 (socket says 172.21.15.39:35534) 2024-09-21T07:33:51.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.234+0000 7fe100e00640 1 -- 172.21.15.39:0/3024234063 learned_addr learned my addr 172.21.15.39:0/3024234063 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:51.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.234+0000 7fe0fbe00640 1 --2- 172.21.15.39:0/3024234063 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe0fc0ce730 0x7fe0fc164820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:51.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.234+0000 7fe101800640 1 --2- 172.21.15.39:0/3024234063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0fc0d0dc0 0x7fe0fc164d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:51.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.234+0000 7fe100e00640 1 -- 172.21.15.39:0/3024234063 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe0fc0ce730 msgr2=0x7fe0fc164820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:51.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.234+0000 7fe100e00640 1 --2- 172.21.15.39:0/3024234063 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe0fc0ce730 0x7fe0fc164820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:51.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.234+0000 7fe100e00640 1 -- 172.21.15.39:0/3024234063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0fc0d0dc0 msgr2=0x7fe0fc164d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:51.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.234+0000 7fe100e00640 1 --2- 172.21.15.39:0/3024234063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0fc0d0dc0 0x7fe0fc164d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:51.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.234+0000 7fe100e00640 1 -- 172.21.15.39:0/3024234063 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0fc166750 con 0x7fe0fc0cd0e0 2024-09-21T07:33:51.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.234+0000 7fe0fbe00640 1 --2- 172.21.15.39:0/3024234063 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe0fc0ce730 0x7fe0fc164820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:33:51.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.234+0000 7fe101800640 1 --2- 172.21.15.39:0/3024234063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0fc0d0dc0 0x7fe0fc164d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:33:51.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.235+0000 7fe100e00640 1 --2- 172.21.15.39:0/3024234063 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0fc0cd0e0 0x7fe0fc165ce0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe0f000e7f0 tx=0x7fe0f000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:51.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.235+0000 7fe0f8c00640 1 -- 172.21.15.39:0/3024234063 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f000ccc0 con 0x7fe0fc0cd0e0 2024-09-21T07:33:51.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.235+0000 7fe0f8c00640 1 -- 172.21.15.39:0/3024234063 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe0f0010430 con 0x7fe0fc0cd0e0 2024-09-21T07:33:51.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.235+0000 7fe0f8c00640 1 -- 172.21.15.39:0/3024234063 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0010840 con 0x7fe0fc0cd0e0 2024-09-21T07:33:51.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.235+0000 7fe102200640 1 -- 172.21.15.39:0/3024234063 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0fc0c9300 con 0x7fe0fc0cd0e0 2024-09-21T07:33:51.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.235+0000 7fe102200640 1 -- 172.21.15.39:0/3024234063 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0fc0cb9e0 con 0x7fe0fc0cd0e0 2024-09-21T07:33:51.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.236+0000 7fe102200640 1 -- 172.21.15.39:0/3024234063 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0fc0d0880 con 0x7fe0fc0cd0e0 2024-09-21T07:33:51.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.237+0000 7fe0f8c00640 1 -- 172.21.15.39:0/3024234063 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe0f00109e0 con 0x7fe0fc0cd0e0 2024-09-21T07:33:51.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.238+0000 7fe0f8c00640 1 --2- 172.21.15.39:0/3024234063 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe0dc078b90 0x7fe0dc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:51.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.238+0000 7fe0f8c00640 1 -- 172.21.15.39:0/3024234063 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe0f009b8a0 con 0x7fe0fc0cd0e0 2024-09-21T07:33:51.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.239+0000 7fe0fbe00640 1 --2- 172.21.15.39:0/3024234063 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe0dc078b90 0x7fe0dc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:51.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.240+0000 7fe0fbe00640 1 --2- 172.21.15.39:0/3024234063 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe0dc078b90 0x7fe0dc07b0a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8034000 tx=0x7fe0e8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:51.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.243+0000 7fe0f8c00640 1 -- 172.21.15.39:0/3024234063 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe0f0063c40 con 0x7fe0fc0cd0e0 2024-09-21T07:33:51.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.454+0000 7fe102200640 1 -- 172.21.15.39:0/3024234063 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe0fc0af3f0 con 0x7fe0fc0cd0e0 2024-09-21T07:33:51.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.454+0000 7fe0f8c00640 1 -- 172.21.15.39:0/3024234063 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7fe0f0063540 con 0x7fe0fc0cd0e0 2024-09-21T07:33:51.456 INFO:teuthology.orchestra.run.smithi039.stderr:listed 0 entries 2024-09-21T07:33:51.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.457+0000 7fe102200640 1 -- 172.21.15.39:0/3024234063 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe0dc078b90 msgr2=0x7fe0dc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:51.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.457+0000 7fe102200640 1 --2- 172.21.15.39:0/3024234063 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe0dc078b90 0x7fe0dc07b0a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8034000 tx=0x7fe0e8004730 comp rx=0 tx=0).stop 2024-09-21T07:33:51.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.457+0000 7fe102200640 1 -- 172.21.15.39:0/3024234063 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0fc0cd0e0 msgr2=0x7fe0fc165ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:51.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.457+0000 7fe102200640 1 --2- 172.21.15.39:0/3024234063 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0fc0cd0e0 0x7fe0fc165ce0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe0f000e7f0 tx=0x7fe0f000ecb0 comp rx=0 tx=0).stop 2024-09-21T07:33:51.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.457+0000 7fe102200640 1 -- 172.21.15.39:0/3024234063 shutdown_connections 2024-09-21T07:33:51.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.457+0000 7fe102200640 1 --2- 172.21.15.39:0/3024234063 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe0dc078b90 0x7fe0dc07b0a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:51.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.457+0000 7fe102200640 1 --2- 172.21.15.39:0/3024234063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0fc0d0dc0 0x7fe0fc164d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:51.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.457+0000 7fe102200640 1 --2- 172.21.15.39:0/3024234063 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe0fc0ce730 0x7fe0fc164820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:51.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.457+0000 7fe102200640 1 --2- 172.21.15.39:0/3024234063 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0fc0cd0e0 0x7fe0fc165ce0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:51.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.457+0000 7fe102200640 1 -- 172.21.15.39:0/3024234063 >> 172.21.15.39:0/3024234063 conn(0x7fe0fc0ceba0 msgr2=0x7fe0fc072290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:51.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.457+0000 7fe102200640 1 -- 172.21.15.39:0/3024234063 shutdown_connections 2024-09-21T07:33:51.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:51.457+0000 7fe102200640 1 -- 172.21.15.39:0/3024234063 wait complete. 2024-09-21T07:33:51.484 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-21T07:33:51.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:51.484 DEBUG:teuthology.orchestra.run.smithi148:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-09-21T07:33:51.554 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2024-09-21T07:33:51.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:51.555 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T07:33:51.603 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:33:51.603 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:33:51.639 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:33:51.639 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:33:51.715 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:33:51.715 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:33:51.790 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:33:52.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.062+0000 7fc78ee00640 1 -- 172.21.15.39:0/1156379257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7880706b0 msgr2=0x7fc7880cba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:52.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.062+0000 7fc78ee00640 1 --2- 172.21.15.39:0/1156379257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7880706b0 0x7fc7880cba00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc788070dc0 tx=0x7fc77002a540 comp rx=0 tx=0).stop 2024-09-21T07:33:52.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.062+0000 7fc78ee00640 1 -- 172.21.15.39:0/1156379257 shutdown_connections 2024-09-21T07:33:52.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.062+0000 7fc78ee00640 1 --2- 172.21.15.39:0/1156379257 >> v2:172.21.15.148:3300/0 conn(0x7fc7880cad30 0x7fc7880735b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:52.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.062+0000 7fc78ee00640 1 --2- 172.21.15.39:0/1156379257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7880706b0 0x7fc7880cba00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:52.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.062+0000 7fc78ee00640 1 -- 172.21.15.39:0/1156379257 >> 172.21.15.39:0/1156379257 conn(0x7fc7880b1cb0 msgr2=0x7fc7880b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:52.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.062+0000 7fc78ee00640 1 -- 172.21.15.39:0/1156379257 shutdown_connections 2024-09-21T07:33:52.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.062+0000 7fc78ee00640 1 -- 172.21.15.39:0/1156379257 wait complete. 2024-09-21T07:33:52.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.064+0000 7fc78ee00640 1 Processor -- start 2024-09-21T07:33:52.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.064+0000 7fc78ee00640 1 -- start start 2024-09-21T07:33:52.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.066+0000 7fc78ee00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7880ca800 0x7fc7881f6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:52.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.066+0000 7fc78ee00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc7880cad30 0x7fc78815df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:52.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.066+0000 7fc78ee00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7881f6d80 0x7fc78815e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:52.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.066+0000 7fc78ee00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc788073ec0 con 0x7fc7881f6d80 2024-09-21T07:33:52.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.066+0000 7fc78ee00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc788073d40 con 0x7fc7880ca800 2024-09-21T07:33:52.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.066+0000 7fc78ee00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc788074040 con 0x7fc7880cad30 2024-09-21T07:33:52.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.066+0000 7fc78ce00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7881f6d80 0x7fc78815e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:52.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.066+0000 7fc78ce00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7881f6d80 0x7fc78815e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51848/0 (socket says 172.21.15.39:51848) 2024-09-21T07:33:52.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.066+0000 7fc78ce00640 1 -- 172.21.15.39:0/3765568749 learned_addr learned my addr 172.21.15.39:0/3765568749 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:52.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.066+0000 7fc787e00640 1 --2- 172.21.15.39:0/3765568749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7880ca800 0x7fc7881f6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:52.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.066+0000 7fc77f400640 1 --2- 172.21.15.39:0/3765568749 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc7880cad30 0x7fc78815df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:52.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.066+0000 7fc78ce00640 1 -- 172.21.15.39:0/3765568749 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc7880cad30 msgr2=0x7fc78815df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:52.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.066+0000 7fc78ce00640 1 --2- 172.21.15.39:0/3765568749 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc7880cad30 0x7fc78815df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:52.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.066+0000 7fc78ce00640 1 -- 172.21.15.39:0/3765568749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7880ca800 msgr2=0x7fc7881f6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:52.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.066+0000 7fc78ce00640 1 --2- 172.21.15.39:0/3765568749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7880ca800 0x7fc7881f6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:52.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.066+0000 7fc78ce00640 1 -- 172.21.15.39:0/3765568749 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7880c5070 con 0x7fc7881f6d80 2024-09-21T07:33:52.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.066+0000 7fc787e00640 1 --2- 172.21.15.39:0/3765568749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7880ca800 0x7fc7881f6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:33:52.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.067+0000 7fc78ce00640 1 --2- 172.21.15.39:0/3765568749 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7881f6d80 0x7fc78815e440 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc78000d690 tx=0x7fc78000db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:52.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.067+0000 7fc784c00640 1 -- 172.21.15.39:0/3765568749 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc780017070 con 0x7fc7881f6d80 2024-09-21T07:33:52.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.067+0000 7fc784c00640 1 -- 172.21.15.39:0/3765568749 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc780005310 con 0x7fc7881f6d80 2024-09-21T07:33:52.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.067+0000 7fc78ee00640 1 -- 172.21.15.39:0/3765568749 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7880c5350 con 0x7fc7881f6d80 2024-09-21T07:33:52.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.067+0000 7fc784c00640 1 -- 172.21.15.39:0/3765568749 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7800097c0 con 0x7fc7881f6d80 2024-09-21T07:33:52.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.067+0000 7fc78ee00640 1 -- 172.21.15.39:0/3765568749 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7881f59d0 con 0x7fc7881f6d80 2024-09-21T07:33:52.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.068+0000 7fc78ee00640 1 -- 172.21.15.39:0/3765568749 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7880c4de0 con 0x7fc7881f6d80 2024-09-21T07:33:52.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.069+0000 7fc784c00640 1 -- 172.21.15.39:0/3765568749 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc780009960 con 0x7fc7881f6d80 2024-09-21T07:33:52.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.070+0000 7fc784c00640 1 --2- 172.21.15.39:0/3765568749 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc764078bc0 0x7fc76407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:52.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.070+0000 7fc784c00640 1 -- 172.21.15.39:0/3765568749 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5970+0+0 (secure 0 0 0) 0x7fc78009aec0 con 0x7fc7881f6d80 2024-09-21T07:33:52.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.071+0000 7fc787e00640 1 --2- 172.21.15.39:0/3765568749 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc764078bc0 0x7fc76407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:52.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.072+0000 7fc787e00640 1 --2- 172.21.15.39:0/3765568749 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc764078bc0 0x7fc76407b0d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc7880cb250 tx=0x7fc778009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:52.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.075+0000 7fc784c00640 1 -- 172.21.15.39:0/3765568749 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc7800631e0 con 0x7fc7881f6d80 2024-09-21T07:33:52.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.284+0000 7fc78ee00640 1 -- 172.21.15.39:0/3765568749 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc7880745d0 con 0x7fc7881f6d80 2024-09-21T07:33:52.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.285+0000 7fc784c00640 1 -- 172.21.15.39:0/3765568749 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fc780062ae0 con 0x7fc7881f6d80 2024-09-21T07:33:52.287 INFO:teuthology.orchestra.run.smithi039.stderr:listed 0 entries 2024-09-21T07:33:52.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.287+0000 7fc78ee00640 1 -- 172.21.15.39:0/3765568749 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc764078bc0 msgr2=0x7fc76407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:52.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.287+0000 7fc78ee00640 1 --2- 172.21.15.39:0/3765568749 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc764078bc0 0x7fc76407b0d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc7880cb250 tx=0x7fc778009040 comp rx=0 tx=0).stop 2024-09-21T07:33:52.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.287+0000 7fc78ee00640 1 -- 172.21.15.39:0/3765568749 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7881f6d80 msgr2=0x7fc78815e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:52.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.287+0000 7fc78ee00640 1 --2- 172.21.15.39:0/3765568749 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7881f6d80 0x7fc78815e440 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc78000d690 tx=0x7fc78000db50 comp rx=0 tx=0).stop 2024-09-21T07:33:52.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.287+0000 7fc78ee00640 1 -- 172.21.15.39:0/3765568749 shutdown_connections 2024-09-21T07:33:52.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.287+0000 7fc78ee00640 1 --2- 172.21.15.39:0/3765568749 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc764078bc0 0x7fc76407b0d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:52.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.287+0000 7fc78ee00640 1 --2- 172.21.15.39:0/3765568749 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7881f6d80 0x7fc78815e440 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:52.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.287+0000 7fc78ee00640 1 --2- 172.21.15.39:0/3765568749 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc7880cad30 0x7fc78815df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:52.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.287+0000 7fc78ee00640 1 --2- 172.21.15.39:0/3765568749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7880ca800 0x7fc7881f6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:52.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.287+0000 7fc78ee00640 1 -- 172.21.15.39:0/3765568749 >> 172.21.15.39:0/3765568749 conn(0x7fc7880b1cb0 msgr2=0x7fc7880b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:52.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.287+0000 7fc78ee00640 1 -- 172.21.15.39:0/3765568749 shutdown_connections 2024-09-21T07:33:52.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.287+0000 7fc78ee00640 1 -- 172.21.15.39:0/3765568749 wait complete. 2024-09-21T07:33:52.314 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-09-21T07:33:52.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:52.314 DEBUG:teuthology.orchestra.run.smithi039:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-09-21T07:33:52.374 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T07:33:52.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:52.374 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-21T07:33:52.410 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:33:52.411 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:33:52.451 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:33:52.451 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:33:52.529 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:33:52.529 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:33:52.607 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:33:52.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.834+0000 7f3286a00640 1 -- 172.21.15.39:0/3538315485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3264005280 msgr2=0x7f326400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:52.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.834+0000 7f3286a00640 1 --2- 172.21.15.39:0/3538315485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3264005280 0x7f326400b9c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3278005fd0 tx=0x7f3264005830 comp rx=0 tx=0).stop 2024-09-21T07:33:52.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.834+0000 7f3286a00640 1 -- 172.21.15.39:0/3538315485 shutdown_connections 2024-09-21T07:33:52.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.834+0000 7f3286a00640 1 --2- 172.21.15.39:0/3538315485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3264005280 0x7f326400b9c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:52.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.834+0000 7f3286a00640 1 -- 172.21.15.39:0/3538315485 >> 172.21.15.39:0/3538315485 conn(0x7f32800b1cb0 msgr2=0x7f32800b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:52.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.834+0000 7f3286a00640 1 -- 172.21.15.39:0/3538315485 shutdown_connections 2024-09-21T07:33:52.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.834+0000 7f3286a00640 1 -- 172.21.15.39:0/3538315485 wait complete. 2024-09-21T07:33:52.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.837+0000 7f3286a00640 1 Processor -- start 2024-09-21T07:33:52.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.837+0000 7f3286a00640 1 -- start start 2024-09-21T07:33:52.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.838+0000 7f3286a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3264005280 0x7f32801e6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:52.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.838+0000 7f3286a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32801f46f0 0x7f32801f7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:52.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.838+0000 7f3286a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32801e6fe0 0x7f32801e73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:52.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.838+0000 7f3286a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3280074640 con 0x7f3264005280 2024-09-21T07:33:52.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.838+0000 7f3286a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f328015cc00 con 0x7f32801e6fe0 2024-09-21T07:33:52.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.838+0000 7f3286a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f32800747c0 con 0x7f32801f46f0 2024-09-21T07:33:52.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.839+0000 7f327fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3264005280 0x7f32801e6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:52.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.839+0000 7f327fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3264005280 0x7f32801e6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51866/0 (socket says 172.21.15.39:51866) 2024-09-21T07:33:52.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.839+0000 7f327fe00640 1 -- 172.21.15.39:0/2806286584 learned_addr learned my addr 172.21.15.39:0/2806286584 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:52.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.839+0000 7f3284a00640 1 --2- 172.21.15.39:0/2806286584 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32801e6fe0 0x7f32801e73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:52.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.839+0000 7f3277e00640 1 --2- 172.21.15.39:0/2806286584 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32801f46f0 0x7f32801f7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:52.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.839+0000 7f327fe00640 1 -- 172.21.15.39:0/2806286584 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32801f46f0 msgr2=0x7f32801f7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:52.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.839+0000 7f327fe00640 1 --2- 172.21.15.39:0/2806286584 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32801f46f0 0x7f32801f7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:52.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.839+0000 7f327fe00640 1 -- 172.21.15.39:0/2806286584 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32801e6fe0 msgr2=0x7f32801e73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:52.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.839+0000 7f327fe00640 1 --2- 172.21.15.39:0/2806286584 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32801e6fe0 0x7f32801e73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:52.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.839+0000 7f327fe00640 1 -- 172.21.15.39:0/2806286584 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32801e7930 con 0x7f3264005280 2024-09-21T07:33:52.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.839+0000 7f3277e00640 1 --2- 172.21.15.39:0/2806286584 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32801f46f0 0x7f32801f7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:33:52.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.839+0000 7f3284a00640 1 --2- 172.21.15.39:0/2806286584 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32801e6fe0 0x7f32801e73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:33:52.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.839+0000 7f327fe00640 1 --2- 172.21.15.39:0/2806286584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3264005280 0x7f32801e6460 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3270003d30 tx=0x7f3270001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:52.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.840+0000 7f327cc00640 1 -- 172.21.15.39:0/2806286584 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3270011640 con 0x7f3264005280 2024-09-21T07:33:52.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.840+0000 7f327cc00640 1 -- 172.21.15.39:0/2806286584 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3270016020 con 0x7f3264005280 2024-09-21T07:33:52.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.840+0000 7f327cc00640 1 -- 172.21.15.39:0/2806286584 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3270011a60 con 0x7f3264005280 2024-09-21T07:33:52.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.840+0000 7f3286a00640 1 -- 172.21.15.39:0/2806286584 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32801f7c40 con 0x7f3264005280 2024-09-21T07:33:52.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.840+0000 7f3286a00640 1 -- 172.21.15.39:0/2806286584 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32801f3ef0 con 0x7f3264005280 2024-09-21T07:33:52.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.841+0000 7f3286a00640 1 -- 172.21.15.39:0/2806286584 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3280074d20 con 0x7f3264005280 2024-09-21T07:33:52.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.842+0000 7f327cc00640 1 -- 172.21.15.39:0/2806286584 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3270011c00 con 0x7f3264005280 2024-09-21T07:33:52.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.843+0000 7f327cc00640 1 --2- 172.21.15.39:0/2806286584 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f325c078b90 0x7f325c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:52.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.843+0000 7f327cc00640 1 -- 172.21.15.39:0/2806286584 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5970+0+0 (secure 0 0 0) 0x7f327009ba50 con 0x7f3264005280 2024-09-21T07:33:52.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.844+0000 7f3277e00640 1 --2- 172.21.15.39:0/2806286584 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f325c078b90 0x7f325c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:52.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.845+0000 7f3277e00640 1 --2- 172.21.15.39:0/2806286584 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f325c078b90 0x7f325c07b0a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3268001eb0 tx=0x7f32680011b0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:52.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:52.848+0000 7f327cc00640 1 -- 172.21.15.39:0/2806286584 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3270020020 con 0x7f3264005280 2024-09-21T07:33:53.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.065+0000 7f3286a00640 1 -- 172.21.15.39:0/2806286584 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f32800b3e80 con 0x7f3264005280 2024-09-21T07:33:53.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.066+0000 7f327cc00640 1 -- 172.21.15.39:0/2806286584 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f32801e7930 con 0x7f3264005280 2024-09-21T07:33:53.068 INFO:teuthology.orchestra.run.smithi039.stderr:listed 0 entries 2024-09-21T07:33:53.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.068+0000 7f3286a00640 1 -- 172.21.15.39:0/2806286584 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f325c078b90 msgr2=0x7f325c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:53.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.068+0000 7f3286a00640 1 --2- 172.21.15.39:0/2806286584 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f325c078b90 0x7f325c07b0a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3268001eb0 tx=0x7f32680011b0 comp rx=0 tx=0).stop 2024-09-21T07:33:53.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.068+0000 7f3286a00640 1 -- 172.21.15.39:0/2806286584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3264005280 msgr2=0x7f32801e6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:53.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.068+0000 7f3286a00640 1 --2- 172.21.15.39:0/2806286584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3264005280 0x7f32801e6460 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3270003d30 tx=0x7f3270001800 comp rx=0 tx=0).stop 2024-09-21T07:33:53.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.069+0000 7f3286a00640 1 -- 172.21.15.39:0/2806286584 shutdown_connections 2024-09-21T07:33:53.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.069+0000 7f3286a00640 1 --2- 172.21.15.39:0/2806286584 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f325c078b90 0x7f325c07b0a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:53.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.069+0000 7f3286a00640 1 --2- 172.21.15.39:0/2806286584 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32801e6fe0 0x7f32801e73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:53.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.069+0000 7f3286a00640 1 --2- 172.21.15.39:0/2806286584 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32801f46f0 0x7f32801f7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:53.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.069+0000 7f3286a00640 1 --2- 172.21.15.39:0/2806286584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3264005280 0x7f32801e6460 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:53.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.069+0000 7f3286a00640 1 -- 172.21.15.39:0/2806286584 >> 172.21.15.39:0/2806286584 conn(0x7f32800b1cb0 msgr2=0x7f32800afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:53.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.069+0000 7f3286a00640 1 -- 172.21.15.39:0/2806286584 shutdown_connections 2024-09-21T07:33:53.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.069+0000 7f3286a00640 1 -- 172.21.15.39:0/2806286584 wait complete. 2024-09-21T07:33:53.095 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-09-21T07:33:53.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:53.095 DEBUG:teuthology.orchestra.run.smithi148:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-09-21T07:33:53.162 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi148.front.sepia.ceph.com 2024-09-21T07:33:53.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:33:53.162 DEBUG:teuthology.orchestra.run.smithi148:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-21T07:33:53.199 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T07:33:53.200 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:33:53.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.424+0000 7f53dda00640 1 -- 172.21.15.39:0/1970693824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53d80cc1e0 msgr2=0x7f53d80af3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:53.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.424+0000 7f53dda00640 1 --2- 172.21.15.39:0/1970693824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53d80cc1e0 0x7f53d80af3c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f53d8072bc0 tx=0x7f53c802a560 comp rx=0 tx=0).stop 2024-09-21T07:33:53.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.425+0000 7f53dda00640 1 -- 172.21.15.39:0/1970693824 shutdown_connections 2024-09-21T07:33:53.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.425+0000 7f53dda00640 1 --2- 172.21.15.39:0/1970693824 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f53d80ccb50 0x7f53d80ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:53.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.425+0000 7f53dda00640 1 --2- 172.21.15.39:0/1970693824 >> v2:172.21.15.148:3300/0 conn(0x7f53d815d8a0 0x7f53d80b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:53.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.425+0000 7f53dda00640 1 --2- 172.21.15.39:0/1970693824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53d80cc1e0 0x7f53d80af3c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:53.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.425+0000 7f53dda00640 1 -- 172.21.15.39:0/1970693824 >> 172.21.15.39:0/1970693824 conn(0x7f53d806d7d0 msgr2=0x7f53d80b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:53.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.425+0000 7f53dda00640 1 -- 172.21.15.39:0/1970693824 shutdown_connections 2024-09-21T07:33:53.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.425+0000 7f53dda00640 1 -- 172.21.15.39:0/1970693824 wait complete. 2024-09-21T07:33:53.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.427+0000 7f53dda00640 1 Processor -- start 2024-09-21T07:33:53.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.427+0000 7f53dda00640 1 -- start start 2024-09-21T07:33:53.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.428+0000 7f53dda00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53d80cc1e0 0x7f53d8159b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:53.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.428+0000 7f53dda00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f53d80ccb50 0x7f53d815bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:53.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.428+0000 7f53dda00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53d815d8a0 0x7f53d815c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:53.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.428+0000 7f53dda00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f53d80b4770 con 0x7f53d815d8a0 2024-09-21T07:33:53.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.428+0000 7f53dda00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f53d80b45f0 con 0x7f53d80cc1e0 2024-09-21T07:33:53.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.428+0000 7f53dda00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f53d81533a0 con 0x7f53d80ccb50 2024-09-21T07:33:53.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.429+0000 7f53d7400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53d815d8a0 0x7f53d815c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:53.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.429+0000 7f53d7400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53d815d8a0 0x7f53d815c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51898/0 (socket says 172.21.15.39:51898) 2024-09-21T07:33:53.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.429+0000 7f53d7400640 1 -- 172.21.15.39:0/3263366753 learned_addr learned my addr 172.21.15.39:0/3263366753 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:53.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.429+0000 7f53d6a00640 1 --2- 172.21.15.39:0/3263366753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53d80cc1e0 0x7f53d8159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:53.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.429+0000 7f53d6000640 1 --2- 172.21.15.39:0/3263366753 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f53d80ccb50 0x7f53d815bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:53.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.429+0000 7f53d7400640 1 -- 172.21.15.39:0/3263366753 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f53d80ccb50 msgr2=0x7f53d815bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:53.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.429+0000 7f53d7400640 1 --2- 172.21.15.39:0/3263366753 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f53d80ccb50 0x7f53d815bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:53.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.429+0000 7f53d7400640 1 -- 172.21.15.39:0/3263366753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53d80cc1e0 msgr2=0x7f53d8159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:53.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.429+0000 7f53d7400640 1 --2- 172.21.15.39:0/3263366753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53d80cc1e0 0x7f53d8159b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:53.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.429+0000 7f53d7400640 1 -- 172.21.15.39:0/3263366753 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d81f93d0 con 0x7f53d815d8a0 2024-09-21T07:33:53.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.430+0000 7f53d7400640 1 --2- 172.21.15.39:0/3263366753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53d815d8a0 0x7f53d815c0e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f53d000e9c0 tx=0x7f53d000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:53.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.430+0000 7f53cea00640 1 -- 172.21.15.39:0/3263366753 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d0004d20 con 0x7f53d815d8a0 2024-09-21T07:33:53.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.430+0000 7f53cea00640 1 -- 172.21.15.39:0/3263366753 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53d000cb20 con 0x7f53d815d8a0 2024-09-21T07:33:53.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.430+0000 7f53dda00640 1 -- 172.21.15.39:0/3263366753 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53d81f96b0 con 0x7f53d815d8a0 2024-09-21T07:33:53.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.430+0000 7f53cea00640 1 -- 172.21.15.39:0/3263366753 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d0005020 con 0x7f53d815d8a0 2024-09-21T07:33:53.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.430+0000 7f53dda00640 1 -- 172.21.15.39:0/3263366753 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53d81f9c70 con 0x7f53d815d8a0 2024-09-21T07:33:53.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.431+0000 7f53dda00640 1 -- 172.21.15.39:0/3263366753 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53d81f9140 con 0x7f53d815d8a0 2024-09-21T07:33:53.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.432+0000 7f53cea00640 1 -- 172.21.15.39:0/3263366753 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f53d00051c0 con 0x7f53d815d8a0 2024-09-21T07:33:53.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.433+0000 7f53cea00640 1 --2- 172.21.15.39:0/3263366753 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f53b8078b90 0x7f53b807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:53.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.433+0000 7f53cea00640 1 -- 172.21.15.39:0/3263366753 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5970+0+0 (secure 0 0 0) 0x7f53d009b950 con 0x7f53d815d8a0 2024-09-21T07:33:53.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.434+0000 7f53d6a00640 1 --2- 172.21.15.39:0/3263366753 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f53b8078b90 0x7f53b807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:53.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.435+0000 7f53d6a00640 1 --2- 172.21.15.39:0/3263366753 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f53b8078b90 0x7f53b807b0a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f53c8033000 tx=0x7f53c8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:53.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.438+0000 7f53cea00640 1 -- 172.21.15.39:0/3263366753 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f53d0016030 con 0x7f53d815d8a0 2024-09-21T07:33:53.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.671+0000 7f53dda00640 1 -- 172.21.15.39:0/3263366753 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53d80ccf60 con 0x7f53d815d8a0 2024-09-21T07:33:53.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.671+0000 7f53cea00640 1 -- 172.21.15.39:0/3263366753 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5220 (secure 0 0 0) 0x7f53d0063620 con 0x7f53d815d8a0 2024-09-21T07:33:53.673 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:33:53.673 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":8,"btime":"2024-09-21T07:33:18:179932+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4325,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1939850940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1939850940},{"type":"v1","addr":"172.21.15.39:6835","nonce":1939850940}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4328,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3712021428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3712021428},{"type":"v1","addr":"172.21.15.148:6836","nonce":3712021428}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4331,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3708353709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3708353709},{"type":"v1","addr":"172.21.15.148:6837","nonce":3708353709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:33:14.674552+0000","modified":"2024-09-21T07:33:17.688290+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4378},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4378":{"gid":4378,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6833/414851924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":414851924},{"type":"v1","addr":"172.21.15.39:6833","nonce":414851924}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4378,"qdb_cluster":[4378]},"id":1}]} 2024-09-21T07:33:53.674 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 8 2024-09-21T07:33:53.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.673+0000 7f53dda00640 1 -- 172.21.15.39:0/3263366753 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f53b8078b90 msgr2=0x7f53b807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:53.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.673+0000 7f53dda00640 1 --2- 172.21.15.39:0/3263366753 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f53b8078b90 0x7f53b807b0a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f53c8033000 tx=0x7f53c8004730 comp rx=0 tx=0).stop 2024-09-21T07:33:53.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.673+0000 7f53dda00640 1 -- 172.21.15.39:0/3263366753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53d815d8a0 msgr2=0x7f53d815c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:53.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.673+0000 7f53dda00640 1 --2- 172.21.15.39:0/3263366753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53d815d8a0 0x7f53d815c0e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f53d000e9c0 tx=0x7f53d000ee80 comp rx=0 tx=0).stop 2024-09-21T07:33:53.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.674+0000 7f53dda00640 1 -- 172.21.15.39:0/3263366753 shutdown_connections 2024-09-21T07:33:53.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.674+0000 7f53dda00640 1 --2- 172.21.15.39:0/3263366753 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f53b8078b90 0x7f53b807b0a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:53.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.674+0000 7f53dda00640 1 --2- 172.21.15.39:0/3263366753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53d815d8a0 0x7f53d815c0e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:53.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.674+0000 7f53dda00640 1 --2- 172.21.15.39:0/3263366753 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f53d80ccb50 0x7f53d815bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:53.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.674+0000 7f53dda00640 1 --2- 172.21.15.39:0/3263366753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53d80cc1e0 0x7f53d8159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:53.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.674+0000 7f53dda00640 1 -- 172.21.15.39:0/3263366753 >> 172.21.15.39:0/3263366753 conn(0x7f53d806d7d0 msgr2=0x7f53d8074610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:53.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.674+0000 7f53dda00640 1 -- 172.21.15.39:0/3263366753 shutdown_connections 2024-09-21T07:33:53.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.674+0000 7f53dda00640 1 -- 172.21.15.39:0/3263366753 wait complete. 2024-09-21T07:33:53.702 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:33:53.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.928+0000 7fbae6200640 1 -- 172.21.15.39:0/159940959 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbae006f580 msgr2=0x7fbae015d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:53.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.928+0000 7fbae6200640 1 --2- 172.21.15.39:0/159940959 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbae006f580 0x7fbae015d700 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbad8005fd0 tx=0x7fbad802aa10 comp rx=0 tx=0).stop 2024-09-21T07:33:53.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.928+0000 7fbae6200640 1 -- 172.21.15.39:0/159940959 shutdown_connections 2024-09-21T07:33:53.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.928+0000 7fbae6200640 1 --2- 172.21.15.39:0/159940959 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbae006f580 0x7fbae015d700 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:53.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.928+0000 7fbae6200640 1 --2- 172.21.15.39:0/159940959 >> v2:172.21.15.148:3300/0 conn(0x7fbae015ca50 0x7fbae00cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:53.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.928+0000 7fbae6200640 1 -- 172.21.15.39:0/159940959 >> 172.21.15.39:0/159940959 conn(0x7fbae006d410 msgr2=0x7fbae00b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:53.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.928+0000 7fbae6200640 1 -- 172.21.15.39:0/159940959 shutdown_connections 2024-09-21T07:33:53.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.928+0000 7fbae6200640 1 -- 172.21.15.39:0/159940959 wait complete. 2024-09-21T07:33:53.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.930+0000 7fbae6200640 1 Processor -- start 2024-09-21T07:33:53.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.930+0000 7fbae6200640 1 -- start start 2024-09-21T07:33:53.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.932+0000 7fbae6200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbae00cf540 0x7fbae01f8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:53.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.932+0000 7fbae6200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbae015ca50 0x7fbae0131070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:53.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.932+0000 7fbae6200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbae0144c70 0x7fbae0155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:53.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.932+0000 7fbae6200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbae015ddc0 con 0x7fbae0144c70 2024-09-21T07:33:53.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.932+0000 7fbae6200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbae015dc40 con 0x7fbae015ca50 2024-09-21T07:33:53.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.932+0000 7fbae6200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fbae015df40 con 0x7fbae00cf540 2024-09-21T07:33:53.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.932+0000 7fbae5800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbae0144c70 0x7fbae0155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:53.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.932+0000 7fbae5800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbae0144c70 0x7fbae0155ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51910/0 (socket says 172.21.15.39:51910) 2024-09-21T07:33:53.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.932+0000 7fbae5800640 1 -- 172.21.15.39:0/743387876 learned_addr learned my addr 172.21.15.39:0/743387876 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:53.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.932+0000 7fbae4e00640 1 --2- 172.21.15.39:0/743387876 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbae00cf540 0x7fbae01f8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:53.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.933+0000 7fbad7e00640 1 --2- 172.21.15.39:0/743387876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbae015ca50 0x7fbae0131070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:53.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.933+0000 7fbae5800640 1 -- 172.21.15.39:0/743387876 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbae00cf540 msgr2=0x7fbae01f8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:53.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.933+0000 7fbae5800640 1 --2- 172.21.15.39:0/743387876 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbae00cf540 0x7fbae01f8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:53.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.933+0000 7fbae5800640 1 -- 172.21.15.39:0/743387876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbae015ca50 msgr2=0x7fbae0131070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:53.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.933+0000 7fbae5800640 1 --2- 172.21.15.39:0/743387876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbae015ca50 0x7fbae0131070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:53.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.933+0000 7fbae5800640 1 -- 172.21.15.39:0/743387876 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae01f9480 con 0x7fbae0144c70 2024-09-21T07:33:53.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.933+0000 7fbae5800640 1 --2- 172.21.15.39:0/743387876 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbae0144c70 0x7fbae0155ec0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbad8006010 tx=0x7fbad8003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:53.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.933+0000 7fbadd600640 1 -- 172.21.15.39:0/743387876 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8046070 con 0x7fbae0144c70 2024-09-21T07:33:53.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.934+0000 7fbae6200640 1 -- 172.21.15.39:0/743387876 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbae01315b0 con 0x7fbae0144c70 2024-09-21T07:33:53.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.934+0000 7fbadd600640 1 -- 172.21.15.39:0/743387876 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbad8004540 con 0x7fbae0144c70 2024-09-21T07:33:53.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.934+0000 7fbadd600640 1 -- 172.21.15.39:0/743387876 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8033070 con 0x7fbae0144c70 2024-09-21T07:33:53.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.934+0000 7fbae6200640 1 -- 172.21.15.39:0/743387876 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbae0154e20 con 0x7fbae0144c70 2024-09-21T07:33:53.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.935+0000 7fbae6200640 1 -- 172.21.15.39:0/743387876 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbae015e3f0 con 0x7fbae0144c70 2024-09-21T07:33:53.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.936+0000 7fbadd600640 1 -- 172.21.15.39:0/743387876 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbad8049070 con 0x7fbae0144c70 2024-09-21T07:33:53.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.939+0000 7fbadd600640 1 --2- 172.21.15.39:0/743387876 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbabc078b90 0x7fbabc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:53.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.939+0000 7fbadd600640 1 -- 172.21.15.39:0/743387876 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5970+0+0 (secure 0 0 0) 0x7fbad80bbe10 con 0x7fbae0144c70 2024-09-21T07:33:53.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.939+0000 7fbae4e00640 1 --2- 172.21.15.39:0/743387876 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbabc078b90 0x7fbabc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:53.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.939+0000 7fbae4e00640 1 --2- 172.21.15.39:0/743387876 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbabc078b90 0x7fbabc07b0a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbad00095f0 tx=0x7fbad0005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:53.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:53.942+0000 7fbadd600640 1 -- 172.21.15.39:0/743387876 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbae01f9480 con 0x7fbae0144c70 2024-09-21T07:33:54.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.178+0000 7fbae6200640 1 -- 172.21.15.39:0/743387876 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbae006ddf0 con 0x7fbae0144c70 2024-09-21T07:33:54.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.178+0000 7fbadd600640 1 -- 172.21.15.39:0/743387876 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5220 (secure 0 0 0) 0x7fbad803e070 con 0x7fbae0144c70 2024-09-21T07:33:54.180 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:33:54.181 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":8,"btime":"2024-09-21T07:33:18:179932+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4325,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1939850940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1939850940},{"type":"v1","addr":"172.21.15.39:6835","nonce":1939850940}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4328,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3712021428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3712021428},{"type":"v1","addr":"172.21.15.148:6836","nonce":3712021428}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4331,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3708353709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3708353709},{"type":"v1","addr":"172.21.15.148:6837","nonce":3708353709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:33:14.674552+0000","modified":"2024-09-21T07:33:17.688290+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4378},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4378":{"gid":4378,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6833/414851924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":414851924},{"type":"v1","addr":"172.21.15.39:6833","nonce":414851924}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4378,"qdb_cluster":[4378]},"id":1}]} 2024-09-21T07:33:54.181 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 8 2024-09-21T07:33:54.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.181+0000 7fbae6200640 1 -- 172.21.15.39:0/743387876 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbabc078b90 msgr2=0x7fbabc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:54.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.181+0000 7fbae6200640 1 --2- 172.21.15.39:0/743387876 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbabc078b90 0x7fbabc07b0a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbad00095f0 tx=0x7fbad0005f90 comp rx=0 tx=0).stop 2024-09-21T07:33:54.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.181+0000 7fbae6200640 1 -- 172.21.15.39:0/743387876 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbae0144c70 msgr2=0x7fbae0155ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:54.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.181+0000 7fbae6200640 1 --2- 172.21.15.39:0/743387876 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbae0144c70 0x7fbae0155ec0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbad8006010 tx=0x7fbad8003f10 comp rx=0 tx=0).stop 2024-09-21T07:33:54.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.181+0000 7fbae6200640 1 -- 172.21.15.39:0/743387876 shutdown_connections 2024-09-21T07:33:54.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.181+0000 7fbae6200640 1 --2- 172.21.15.39:0/743387876 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbabc078b90 0x7fbabc07b0a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:54.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.181+0000 7fbae6200640 1 --2- 172.21.15.39:0/743387876 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbae0144c70 0x7fbae0155ec0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:54.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.181+0000 7fbae6200640 1 --2- 172.21.15.39:0/743387876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbae015ca50 0x7fbae0131070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:54.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.181+0000 7fbae6200640 1 --2- 172.21.15.39:0/743387876 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbae00cf540 0x7fbae01f8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:54.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.181+0000 7fbae6200640 1 -- 172.21.15.39:0/743387876 >> 172.21.15.39:0/743387876 conn(0x7fbae006d410 msgr2=0x7fbae00b1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:54.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.181+0000 7fbae6200640 1 -- 172.21.15.39:0/743387876 shutdown_connections 2024-09-21T07:33:54.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.181+0000 7fbae6200640 1 -- 172.21.15.39:0/743387876 wait complete. 2024-09-21T07:33:54.205 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:33:54.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.433+0000 7f1107200640 1 -- 172.21.15.39:0/384568654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11000b71a0 msgr2=0x7f11000b75b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:54.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.433+0000 7f1107200640 1 --2- 172.21.15.39:0/384568654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11000b71a0 0x7f11000b75b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1100076cc0 tx=0x7f10f002a5f0 comp rx=0 tx=0).stop 2024-09-21T07:33:54.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.433+0000 7f1107200640 1 -- 172.21.15.39:0/384568654 shutdown_connections 2024-09-21T07:33:54.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.433+0000 7f1107200640 1 --2- 172.21.15.39:0/384568654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11000b71a0 0x7f11000b75b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:54.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.433+0000 7f1107200640 1 --2- 172.21.15.39:0/384568654 >> v2:172.21.15.148:3300/0 conn(0x7f11000b5520 0x7f11000cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:54.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.433+0000 7f1107200640 1 -- 172.21.15.39:0/384568654 >> 172.21.15.39:0/384568654 conn(0x7f110009d680 msgr2=0x7f11000d4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:54.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.433+0000 7f1107200640 1 -- 172.21.15.39:0/384568654 shutdown_connections 2024-09-21T07:33:54.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.433+0000 7f1107200640 1 -- 172.21.15.39:0/384568654 wait complete. 2024-09-21T07:33:54.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.435+0000 7f1107200640 1 Processor -- start 2024-09-21T07:33:54.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.435+0000 7f1107200640 1 -- start start 2024-09-21T07:33:54.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.438+0000 7f1107200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11000b5520 0x7f11001610d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:54.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.438+0000 7f1107200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11000b71a0 0x7f1100152db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:54.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.438+0000 7f1107200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11000d2250 0x7f11001570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:54.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.438+0000 7f1107200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f11000d0d50 con 0x7f11000b71a0 2024-09-21T07:33:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.438+0000 7f1107200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f11000d2f80 con 0x7f11000d2250 2024-09-21T07:33:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.438+0000 7f1107200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f11000d0ed0 con 0x7f11000b5520 2024-09-21T07:33:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.438+0000 7f1105400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11000b71a0 0x7f1100152db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.438+0000 7f1105400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11000b71a0 0x7f1100152db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51932/0 (socket says 172.21.15.39:51932) 2024-09-21T07:33:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.438+0000 7f1105400640 1 -- 172.21.15.39:0/2530054262 learned_addr learned my addr 172.21.15.39:0/2530054262 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.438+0000 7f1106800640 1 --2- 172.21.15.39:0/2530054262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11000d2250 0x7f11001570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.438+0000 7f1105e00640 1 --2- 172.21.15.39:0/2530054262 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11000b5520 0x7f11001610d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.438+0000 7f1105400640 1 -- 172.21.15.39:0/2530054262 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11000b5520 msgr2=0x7f11001610d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.438+0000 7f1105400640 1 --2- 172.21.15.39:0/2530054262 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11000b5520 0x7f11001610d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.438+0000 7f1105400640 1 -- 172.21.15.39:0/2530054262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11000d2250 msgr2=0x7f11001570d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.438+0000 7f1105400640 1 --2- 172.21.15.39:0/2530054262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11000d2250 0x7f11001570d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.438+0000 7f1105400640 1 -- 172.21.15.39:0/2530054262 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1100161a20 con 0x7f11000b71a0 2024-09-21T07:33:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.438+0000 7f1106800640 1 --2- 172.21.15.39:0/2530054262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11000d2250 0x7f11001570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:33:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.439+0000 7f1105400640 1 --2- 172.21.15.39:0/2530054262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11000b71a0 0x7f1100152db0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f10f0000c00 tx=0x7f10f0003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.439+0000 7f10fa000640 1 -- 172.21.15.39:0/2530054262 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f003e030 con 0x7f11000b71a0 2024-09-21T07:33:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.439+0000 7f10fa000640 1 -- 172.21.15.39:0/2530054262 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10f0003e00 con 0x7f11000b71a0 2024-09-21T07:33:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.439+0000 7f1107200640 1 -- 172.21.15.39:0/2530054262 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11001592f0 con 0x7f11000b71a0 2024-09-21T07:33:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.439+0000 7f10fa000640 1 -- 172.21.15.39:0/2530054262 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f0035040 con 0x7f11000b71a0 2024-09-21T07:33:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.439+0000 7f1107200640 1 -- 172.21.15.39:0/2530054262 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11000ca560 con 0x7f11000b71a0 2024-09-21T07:33:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.440+0000 7f1107200640 1 -- 172.21.15.39:0/2530054262 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1100159110 con 0x7f11000b71a0 2024-09-21T07:33:54.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.442+0000 7f10fa000640 1 -- 172.21.15.39:0/2530054262 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f10f0041030 con 0x7f11000b71a0 2024-09-21T07:33:54.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.442+0000 7f10fa000640 1 --2- 172.21.15.39:0/2530054262 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f10e4078b90 0x7f10e407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:54.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.442+0000 7f10fa000640 1 -- 172.21.15.39:0/2530054262 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5970+0+0 (secure 0 0 0) 0x7f10f00bb6f0 con 0x7f11000b71a0 2024-09-21T07:33:54.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.442+0000 7f1105e00640 1 --2- 172.21.15.39:0/2530054262 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f10e4078b90 0x7f10e407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:54.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.444+0000 7f1105e00640 1 --2- 172.21.15.39:0/2530054262 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f10e4078b90 0x7f10e407b0a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f10f400bb30 tx=0x7f10f400b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:54.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.448+0000 7f10fa000640 1 -- 172.21.15.39:0/2530054262 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1100159110 con 0x7f11000b71a0 2024-09-21T07:33:54.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.644+0000 7f1107200640 1 -- 172.21.15.39:0/2530054262 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f11000cfd70 con 0x7f11000b71a0 2024-09-21T07:33:54.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.645+0000 7f10fa000640 1 -- 172.21.15.39:0/2530054262 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+17091 (secure 0 0 0) 0x7f10f00832b0 con 0x7f11000b71a0 2024-09-21T07:33:54.646 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:33:54.646 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":23,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:33:52.850044+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:33:12.854113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:33:13.622611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.b","mappings":[{"from":0,"to":6}]},{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.13","mappings":[{"from":4,"to":6}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.2d","mappings":[{"from":7,"to":1}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:33:54.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.647+0000 7f1107200640 1 -- 172.21.15.39:0/2530054262 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f10e4078b90 msgr2=0x7f10e407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:54.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.647+0000 7f1107200640 1 --2- 172.21.15.39:0/2530054262 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f10e4078b90 0x7f10e407b0a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f10f400bb30 tx=0x7f10f400b2e0 comp rx=0 tx=0).stop 2024-09-21T07:33:54.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.647+0000 7f1107200640 1 -- 172.21.15.39:0/2530054262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11000b71a0 msgr2=0x7f1100152db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:54.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.647+0000 7f1107200640 1 --2- 172.21.15.39:0/2530054262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11000b71a0 0x7f1100152db0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f10f0000c00 tx=0x7f10f0003910 comp rx=0 tx=0).stop 2024-09-21T07:33:54.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.647+0000 7f1107200640 1 -- 172.21.15.39:0/2530054262 shutdown_connections 2024-09-21T07:33:54.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.647+0000 7f1107200640 1 --2- 172.21.15.39:0/2530054262 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f10e4078b90 0x7f10e407b0a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:54.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.647+0000 7f1107200640 1 --2- 172.21.15.39:0/2530054262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f11000d2250 0x7f11001570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:54.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.647+0000 7f1107200640 1 --2- 172.21.15.39:0/2530054262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11000b71a0 0x7f1100152db0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:54.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.647+0000 7f1107200640 1 --2- 172.21.15.39:0/2530054262 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11000b5520 0x7f11001610d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:54.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.647+0000 7f1107200640 1 -- 172.21.15.39:0/2530054262 >> 172.21.15.39:0/2530054262 conn(0x7f110009d680 msgr2=0x7f11000d2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:54.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.647+0000 7f1107200640 1 -- 172.21.15.39:0/2530054262 shutdown_connections 2024-09-21T07:33:54.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.647+0000 7f1107200640 1 -- 172.21.15.39:0/2530054262 wait complete. 2024-09-21T07:33:54.674 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T07:33:54.675 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:33:54.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.901+0000 7f99d5800640 1 -- 172.21.15.39:0/2605599604 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99d00d1320 msgr2=0x7f99d00cefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:54.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.901+0000 7f99d5800640 1 --2- 172.21.15.39:0/2605599604 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99d00d1320 0x7f99d00cefd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f99bc005fd0 tx=0x7f99bc02a5d0 comp rx=0 tx=0).stop 2024-09-21T07:33:54.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.901+0000 7f99d5800640 1 -- 172.21.15.39:0/2605599604 shutdown_connections 2024-09-21T07:33:54.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.901+0000 7f99d5800640 1 --2- 172.21.15.39:0/2605599604 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99d00d1320 0x7f99d00cefd0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:54.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.901+0000 7f99d5800640 1 --2- 172.21.15.39:0/2605599604 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99d00b96b0 0x7f99d00d0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:54.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.901+0000 7f99d5800640 1 -- 172.21.15.39:0/2605599604 >> 172.21.15.39:0/2605599604 conn(0x7f99d009d680 msgr2=0x7f99d00d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:54.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.901+0000 7f99d5800640 1 -- 172.21.15.39:0/2605599604 shutdown_connections 2024-09-21T07:33:54.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.901+0000 7f99d5800640 1 -- 172.21.15.39:0/2605599604 wait complete. 2024-09-21T07:33:54.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.903+0000 7f99d5800640 1 Processor -- start 2024-09-21T07:33:54.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.903+0000 7f99d5800640 1 -- start start 2024-09-21T07:33:54.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.905+0000 7f99d5800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99d00b96b0 0x7f99d01676d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:54.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.905+0000 7f99d5800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99d00cf510 0x7f99d0167c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:54.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.905+0000 7f99d5800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99d00d1320 0x7f99d0163740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:54.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.905+0000 7f99d5800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f99d00b4a00 con 0x7f99d00cf510 2024-09-21T07:33:54.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.905+0000 7f99d5800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f99d00b50d0 con 0x7f99d00d1320 2024-09-21T07:33:54.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.905+0000 7f99d5800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f99d00b4b80 con 0x7f99d00b96b0 2024-09-21T07:33:54.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.905+0000 7f99cf400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99d00cf510 0x7f99d0167c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:54.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.905+0000 7f99cf400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99d00cf510 0x7f99d0167c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51946/0 (socket says 172.21.15.39:51946) 2024-09-21T07:33:54.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.905+0000 7f99cf400640 1 -- 172.21.15.39:0/3779115170 learned_addr learned my addr 172.21.15.39:0/3779115170 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:54.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.905+0000 7f99cfe00640 1 --2- 172.21.15.39:0/3779115170 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99d00b96b0 0x7f99d01676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:54.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.905+0000 7f99d4e00640 1 --2- 172.21.15.39:0/3779115170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99d00d1320 0x7f99d0163740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:54.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.906+0000 7f99cf400640 1 -- 172.21.15.39:0/3779115170 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99d00b96b0 msgr2=0x7f99d01676d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:54.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.906+0000 7f99cf400640 1 --2- 172.21.15.39:0/3779115170 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99d00b96b0 0x7f99d01676d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:54.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.906+0000 7f99cf400640 1 -- 172.21.15.39:0/3779115170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99d00d1320 msgr2=0x7f99d0163740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:54.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.906+0000 7f99cf400640 1 --2- 172.21.15.39:0/3779115170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99d00d1320 0x7f99d0163740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:54.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.906+0000 7f99cf400640 1 -- 172.21.15.39:0/3779115170 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99d015c2d0 con 0x7f99d00cf510 2024-09-21T07:33:54.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.906+0000 7f99cf400640 1 --2- 172.21.15.39:0/3779115170 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99d00cf510 0x7f99d0167c10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f99bc034040 tx=0x7f99bc0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:54.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.906+0000 7f99c7e00640 1 -- 172.21.15.39:0/3779115170 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc03e030 con 0x7f99d00cf510 2024-09-21T07:33:54.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.906+0000 7f99c7e00640 1 -- 172.21.15.39:0/3779115170 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f99bc003ca0 con 0x7f99d00cf510 2024-09-21T07:33:54.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.906+0000 7f99c7e00640 1 -- 172.21.15.39:0/3779115170 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc035040 con 0x7f99d00cf510 2024-09-21T07:33:54.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.906+0000 7f99d5800640 1 -- 172.21.15.39:0/3779115170 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99d0161760 con 0x7f99d00cf510 2024-09-21T07:33:54.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.906+0000 7f99d5800640 1 -- 172.21.15.39:0/3779115170 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99d0168210 con 0x7f99d00cf510 2024-09-21T07:33:54.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.907+0000 7f99d5800640 1 -- 172.21.15.39:0/3779115170 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99d00d0de0 con 0x7f99d00cf510 2024-09-21T07:33:54.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.909+0000 7f99c7e00640 1 -- 172.21.15.39:0/3779115170 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f99bc041030 con 0x7f99d00cf510 2024-09-21T07:33:54.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.910+0000 7f99c7e00640 1 --2- 172.21.15.39:0/3779115170 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f99b0078bc0 0x7f99b007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:54.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.910+0000 7f99c7e00640 1 -- 172.21.15.39:0/3779115170 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5970+0+0 (secure 0 0 0) 0x7f99bc0bb470 con 0x7f99d00cf510 2024-09-21T07:33:54.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.910+0000 7f99cfe00640 1 --2- 172.21.15.39:0/3779115170 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f99b0078bc0 0x7f99b007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:54.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.911+0000 7f99cfe00640 1 --2- 172.21.15.39:0/3779115170 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f99b0078bc0 0x7f99b007b0d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f99c000bb30 tx=0x7f99c000b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:54.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:54.914+0000 7f99c7e00640 1 -- 172.21.15.39:0/3779115170 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f99bc0830b0 con 0x7f99d00cf510 2024-09-21T07:33:55.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.150+0000 7f99d5800640 1 -- 172.21.15.39:0/3779115170 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99d0162d70 con 0x7f99d00cf510 2024-09-21T07:33:55.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.151+0000 7f99c7e00640 1 -- 172.21.15.39:0/3779115170 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5220 (secure 0 0 0) 0x7f99bc087ae0 con 0x7f99d00cf510 2024-09-21T07:33:55.152 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:33:55.152 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":8,"btime":"2024-09-21T07:33:18:179932+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4325,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1939850940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1939850940},{"type":"v1","addr":"172.21.15.39:6835","nonce":1939850940}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4328,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3712021428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3712021428},{"type":"v1","addr":"172.21.15.148:6836","nonce":3712021428}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4331,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3708353709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3708353709},{"type":"v1","addr":"172.21.15.148:6837","nonce":3708353709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:33:14.674552+0000","modified":"2024-09-21T07:33:17.688290+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4378},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4378":{"gid":4378,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6833/414851924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":414851924},{"type":"v1","addr":"172.21.15.39:6833","nonce":414851924}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4378,"qdb_cluster":[4378]},"id":1}]} 2024-09-21T07:33:55.152 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 8 2024-09-21T07:33:55.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.153+0000 7f99d5800640 1 -- 172.21.15.39:0/3779115170 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f99b0078bc0 msgr2=0x7f99b007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:55.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.153+0000 7f99d5800640 1 --2- 172.21.15.39:0/3779115170 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f99b0078bc0 0x7f99b007b0d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f99c000bb30 tx=0x7f99c000b2e0 comp rx=0 tx=0).stop 2024-09-21T07:33:55.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.153+0000 7f99d5800640 1 -- 172.21.15.39:0/3779115170 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99d00cf510 msgr2=0x7f99d0167c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:55.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.153+0000 7f99d5800640 1 --2- 172.21.15.39:0/3779115170 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99d00cf510 0x7f99d0167c10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f99bc034040 tx=0x7f99bc0038a0 comp rx=0 tx=0).stop 2024-09-21T07:33:55.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.153+0000 7f99d5800640 1 -- 172.21.15.39:0/3779115170 shutdown_connections 2024-09-21T07:33:55.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.153+0000 7f99d5800640 1 --2- 172.21.15.39:0/3779115170 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f99b0078bc0 0x7f99b007b0d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:55.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.153+0000 7f99d5800640 1 --2- 172.21.15.39:0/3779115170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99d00d1320 0x7f99d0163740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:55.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.153+0000 7f99d5800640 1 --2- 172.21.15.39:0/3779115170 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99d00cf510 0x7f99d0167c10 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:55.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.153+0000 7f99d5800640 1 --2- 172.21.15.39:0/3779115170 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99d00b96b0 0x7f99d01676d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:55.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.153+0000 7f99d5800640 1 -- 172.21.15.39:0/3779115170 >> 172.21.15.39:0/3779115170 conn(0x7f99d009d680 msgr2=0x7f99d00b3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:55.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.153+0000 7f99d5800640 1 -- 172.21.15.39:0/3779115170 shutdown_connections 2024-09-21T07:33:55.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.153+0000 7f99d5800640 1 -- 172.21.15.39:0/3779115170 wait complete. 2024-09-21T07:33:55.177 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:33:55.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.404+0000 7f1d69400640 1 -- 172.21.15.39:0/151702883 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d64070030 msgr2=0x7f1d64074640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:55.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.404+0000 7f1d69400640 1 --2- 172.21.15.39:0/151702883 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d64070030 0x7f1d64074640 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1d50005fd0 tx=0x7f1d5002a5d0 comp rx=0 tx=0).stop 2024-09-21T07:33:55.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.404+0000 7f1d69400640 1 -- 172.21.15.39:0/151702883 shutdown_connections 2024-09-21T07:33:55.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.404+0000 7f1d69400640 1 --2- 172.21.15.39:0/151702883 >> v2:172.21.15.148:3300/0 conn(0x7f1d64070440 0x7f1d64074b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:55.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.404+0000 7f1d69400640 1 --2- 172.21.15.39:0/151702883 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d64070030 0x7f1d64074640 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:55.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.404+0000 7f1d69400640 1 -- 172.21.15.39:0/151702883 >> 172.21.15.39:0/151702883 conn(0x7f1d640b1cb0 msgr2=0x7f1d640b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:55.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.404+0000 7f1d69400640 1 -- 172.21.15.39:0/151702883 shutdown_connections 2024-09-21T07:33:55.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.404+0000 7f1d69400640 1 -- 172.21.15.39:0/151702883 wait complete. 2024-09-21T07:33:55.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.406+0000 7f1d69400640 1 Processor -- start 2024-09-21T07:33:55.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.407+0000 7f1d69400640 1 -- start start 2024-09-21T07:33:55.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.408+0000 7f1d69400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d64070030 0x7f1d6414b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:55.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.408+0000 7f1d69400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d64070440 0x7f1d6414d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:55.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.408+0000 7f1d69400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d64073170 0x7f1d6414dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:55.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.408+0000 7f1d69400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1d640b3a90 con 0x7f1d64070030 2024-09-21T07:33:55.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.408+0000 7f1d69400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1d640b3910 con 0x7f1d64073170 2024-09-21T07:33:55.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.408+0000 7f1d69400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f1d640b3c10 con 0x7f1d64070440 2024-09-21T07:33:55.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.408+0000 7f1d62800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d64070030 0x7f1d6414b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:55.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.408+0000 7f1d62800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d64070030 0x7f1d6414b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51954/0 (socket says 172.21.15.39:51954) 2024-09-21T07:33:55.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.408+0000 7f1d62800640 1 -- 172.21.15.39:0/67699665 learned_addr learned my addr 172.21.15.39:0/67699665 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:55.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.408+0000 7f1d61e00640 1 --2- 172.21.15.39:0/67699665 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d64070440 0x7f1d6414d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:55.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.408+0000 7f1d63200640 1 --2- 172.21.15.39:0/67699665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d64073170 0x7f1d6414dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:55.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.409+0000 7f1d62800640 1 -- 172.21.15.39:0/67699665 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d64070440 msgr2=0x7f1d6414d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:55.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.409+0000 7f1d62800640 1 --2- 172.21.15.39:0/67699665 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d64070440 0x7f1d6414d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:55.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.409+0000 7f1d62800640 1 -- 172.21.15.39:0/67699665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d64073170 msgr2=0x7f1d6414dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:55.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.409+0000 7f1d62800640 1 --2- 172.21.15.39:0/67699665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d64073170 0x7f1d6414dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:55.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.409+0000 7f1d62800640 1 -- 172.21.15.39:0/67699665 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d6414b970 con 0x7f1d64070030 2024-09-21T07:33:55.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.409+0000 7f1d62800640 1 --2- 172.21.15.39:0/67699665 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d64070030 0x7f1d6414b190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1d5400c3d0 tx=0x7f1d5400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:55.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.409+0000 7f1d5aa00640 1 -- 172.21.15.39:0/67699665 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d54017070 con 0x7f1d64070030 2024-09-21T07:33:55.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.409+0000 7f1d5aa00640 1 -- 172.21.15.39:0/67699665 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d540053c0 con 0x7f1d64070030 2024-09-21T07:33:55.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.409+0000 7f1d5aa00640 1 -- 172.21.15.39:0/67699665 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d54003850 con 0x7f1d64070030 2024-09-21T07:33:55.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.409+0000 7f1d69400640 1 -- 172.21.15.39:0/67699665 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d6414e530 con 0x7f1d64070030 2024-09-21T07:33:55.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.409+0000 7f1d69400640 1 -- 172.21.15.39:0/67699665 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d6414d490 con 0x7f1d64070030 2024-09-21T07:33:55.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.411+0000 7f1d69400640 1 -- 172.21.15.39:0/67699665 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d640b4130 con 0x7f1d64070030 2024-09-21T07:33:55.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.412+0000 7f1d5aa00640 1 -- 172.21.15.39:0/67699665 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1d5401a070 con 0x7f1d64070030 2024-09-21T07:33:55.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.413+0000 7f1d5aa00640 1 --2- 172.21.15.39:0/67699665 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1d44078bc0 0x7f1d4407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:55.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.413+0000 7f1d5aa00640 1 -- 172.21.15.39:0/67699665 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5970+0+0 (secure 0 0 0) 0x7f1d5409a120 con 0x7f1d64070030 2024-09-21T07:33:55.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.414+0000 7f1d61e00640 1 --2- 172.21.15.39:0/67699665 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1d44078bc0 0x7f1d4407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:55.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.415+0000 7f1d61e00640 1 --2- 172.21.15.39:0/67699665 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1d44078bc0 0x7f1d4407b0d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1d50033000 tx=0x7f1d50004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:55.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.418+0000 7f1d5aa00640 1 -- 172.21.15.39:0/67699665 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1d54061df0 con 0x7f1d64070030 2024-09-21T07:33:55.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.653+0000 7f1d69400640 1 -- 172.21.15.39:0/67699665 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d640c3860 con 0x7f1d64070030 2024-09-21T07:33:55.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.653+0000 7f1d5aa00640 1 -- 172.21.15.39:0/67699665 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5220 (secure 0 0 0) 0x7f1d54066820 con 0x7f1d64070030 2024-09-21T07:33:55.654 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:33:55.654 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":8,"btime":"2024-09-21T07:33:18:179932+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4325,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1939850940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1939850940},{"type":"v1","addr":"172.21.15.39:6835","nonce":1939850940}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4328,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3712021428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3712021428},{"type":"v1","addr":"172.21.15.148:6836","nonce":3712021428}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4331,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3708353709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3708353709},{"type":"v1","addr":"172.21.15.148:6837","nonce":3708353709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:33:14.674552+0000","modified":"2024-09-21T07:33:17.688290+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4378},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4378":{"gid":4378,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6833/414851924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":414851924},{"type":"v1","addr":"172.21.15.39:6833","nonce":414851924}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4378,"qdb_cluster":[4378]},"id":1}]} 2024-09-21T07:33:55.655 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 8 2024-09-21T07:33:55.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.656+0000 7f1d69400640 1 -- 172.21.15.39:0/67699665 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1d44078bc0 msgr2=0x7f1d4407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:55.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.656+0000 7f1d69400640 1 --2- 172.21.15.39:0/67699665 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1d44078bc0 0x7f1d4407b0d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1d50033000 tx=0x7f1d50004730 comp rx=0 tx=0).stop 2024-09-21T07:33:55.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.656+0000 7f1d69400640 1 -- 172.21.15.39:0/67699665 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d64070030 msgr2=0x7f1d6414b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:55.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.656+0000 7f1d69400640 1 --2- 172.21.15.39:0/67699665 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d64070030 0x7f1d6414b190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1d5400c3d0 tx=0x7f1d5400c890 comp rx=0 tx=0).stop 2024-09-21T07:33:55.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.656+0000 7f1d69400640 1 -- 172.21.15.39:0/67699665 shutdown_connections 2024-09-21T07:33:55.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.656+0000 7f1d69400640 1 --2- 172.21.15.39:0/67699665 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1d44078bc0 0x7f1d4407b0d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:55.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.656+0000 7f1d69400640 1 --2- 172.21.15.39:0/67699665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d64073170 0x7f1d6414dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:55.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.656+0000 7f1d69400640 1 --2- 172.21.15.39:0/67699665 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d64070440 0x7f1d6414d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:55.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.656+0000 7f1d69400640 1 --2- 172.21.15.39:0/67699665 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d64070030 0x7f1d6414b190 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:55.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.656+0000 7f1d69400640 1 -- 172.21.15.39:0/67699665 >> 172.21.15.39:0/67699665 conn(0x7f1d640b1cb0 msgr2=0x7f1d6415ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:55.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.656+0000 7f1d69400640 1 -- 172.21.15.39:0/67699665 shutdown_connections 2024-09-21T07:33:55.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.656+0000 7f1d69400640 1 -- 172.21.15.39:0/67699665 wait complete. 2024-09-21T07:33:55.677 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:33:55.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.905+0000 7f28c2800640 1 -- 172.21.15.39:0/1168335989 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28bc0ccb50 msgr2=0x7f28bc0ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:55.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.905+0000 7f28c2800640 1 --2- 172.21.15.39:0/1168335989 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28bc0ccb50 0x7f28bc0ccf60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f28b4005fd0 tx=0x7f28b402aaa0 comp rx=0 tx=0).stop 2024-09-21T07:33:55.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.905+0000 7f28c2800640 1 -- 172.21.15.39:0/1168335989 shutdown_connections 2024-09-21T07:33:55.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.905+0000 7f28c2800640 1 --2- 172.21.15.39:0/1168335989 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28bc0ccb50 0x7f28bc0ccf60 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:55.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.905+0000 7f28c2800640 1 --2- 172.21.15.39:0/1168335989 >> v2:172.21.15.148:3300/0 conn(0x7f28bc0af3c0 0x7f28bc0b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:55.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.905+0000 7f28c2800640 1 -- 172.21.15.39:0/1168335989 >> 172.21.15.39:0/1168335989 conn(0x7f28bc06d7d0 msgr2=0x7f28bc0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:55.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.905+0000 7f28c2800640 1 -- 172.21.15.39:0/1168335989 shutdown_connections 2024-09-21T07:33:55.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.905+0000 7f28c2800640 1 -- 172.21.15.39:0/1168335989 wait complete. 2024-09-21T07:33:55.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.907+0000 7f28c2800640 1 Processor -- start 2024-09-21T07:33:55.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.907+0000 7f28c2800640 1 -- start start 2024-09-21T07:33:55.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.909+0000 7f28c2800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28bc0af3c0 0x7f28bc146c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:55.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.909+0000 7f28c2800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28bc0cc1e0 0x7f28bc158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:55.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.909+0000 7f28c2800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28bc0ccb50 0x7f28bc06ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:55.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.909+0000 7f28c2800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f28bc153520 con 0x7f28bc0af3c0 2024-09-21T07:33:55.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.909+0000 7f28c2800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f28bc1533a0 con 0x7f28bc0cc1e0 2024-09-21T07:33:55.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.909+0000 7f28c2800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f28bc1536a0 con 0x7f28bc0ccb50 2024-09-21T07:33:55.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.909+0000 7f28bb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28bc0af3c0 0x7f28bc146c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:55.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.909+0000 7f28bb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28bc0af3c0 0x7f28bc146c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51960/0 (socket says 172.21.15.39:51960) 2024-09-21T07:33:55.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.909+0000 7f28bb400640 1 -- 172.21.15.39:0/1240233243 learned_addr learned my addr 172.21.15.39:0/1240233243 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:55.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.909+0000 7f28baa00640 1 --2- 172.21.15.39:0/1240233243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28bc0cc1e0 0x7f28bc158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:55.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.909+0000 7f28bbe00640 1 --2- 172.21.15.39:0/1240233243 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28bc0ccb50 0x7f28bc06ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:55.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.909+0000 7f28bb400640 1 -- 172.21.15.39:0/1240233243 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28bc0ccb50 msgr2=0x7f28bc06ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:55.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.909+0000 7f28bb400640 1 --2- 172.21.15.39:0/1240233243 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28bc0ccb50 0x7f28bc06ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:55.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.909+0000 7f28bb400640 1 -- 172.21.15.39:0/1240233243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28bc0cc1e0 msgr2=0x7f28bc158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:55.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.910+0000 7f28bb400640 1 --2- 172.21.15.39:0/1240233243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28bc0cc1e0 0x7f28bc158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:55.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.910+0000 7f28bb400640 1 -- 172.21.15.39:0/1240233243 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28bc158d90 con 0x7f28bc0af3c0 2024-09-21T07:33:55.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.910+0000 7f28bb400640 1 --2- 172.21.15.39:0/1240233243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28bc0af3c0 0x7f28bc146c30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f28ac00c3d0 tx=0x7f28ac00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:55.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.910+0000 7f28b3400640 1 -- 172.21.15.39:0/1240233243 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ac017070 con 0x7f28bc0af3c0 2024-09-21T07:33:55.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.910+0000 7f28b3400640 1 -- 172.21.15.39:0/1240233243 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f28ac0053c0 con 0x7f28bc0af3c0 2024-09-21T07:33:55.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.910+0000 7f28c2800640 1 -- 172.21.15.39:0/1240233243 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28bc1f91e0 con 0x7f28bc0af3c0 2024-09-21T07:33:55.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.910+0000 7f28b3400640 1 -- 172.21.15.39:0/1240233243 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ac003850 con 0x7f28bc0af3c0 2024-09-21T07:33:55.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.910+0000 7f28c2800640 1 -- 172.21.15.39:0/1240233243 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28bc1f97a0 con 0x7f28bc0af3c0 2024-09-21T07:33:55.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.911+0000 7f28c2800640 1 -- 172.21.15.39:0/1240233243 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28bc153b50 con 0x7f28bc0af3c0 2024-09-21T07:33:55.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.913+0000 7f28b3400640 1 -- 172.21.15.39:0/1240233243 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f28ac01a070 con 0x7f28bc0af3c0 2024-09-21T07:33:55.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.914+0000 7f28b3400640 1 --2- 172.21.15.39:0/1240233243 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f289c078b90 0x7f289c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:55.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.914+0000 7f28b3400640 1 -- 172.21.15.39:0/1240233243 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5970+0+0 (secure 0 0 0) 0x7f28ac099ff0 con 0x7f28bc0af3c0 2024-09-21T07:33:55.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.914+0000 7f28baa00640 1 --2- 172.21.15.39:0/1240233243 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f289c078b90 0x7f289c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:55.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.915+0000 7f28baa00640 1 --2- 172.21.15.39:0/1240233243 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f289c078b90 0x7f289c07b0a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f28bc14a7a0 tx=0x7f28a8005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:55.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:55.918+0000 7f28b3400640 1 -- 172.21.15.39:0/1240233243 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f28bc0cd4a0 con 0x7f28bc0af3c0 2024-09-21T07:33:56.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.118+0000 7f28c2800640 1 -- 172.21.15.39:0/1240233243 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f28bc070bf0 con 0x7f28bc0af3c0 2024-09-21T07:33:56.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.120+0000 7f28b3400640 1 -- 172.21.15.39:0/1240233243 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+17091 (secure 0 0 0) 0x7f28ac061d90 con 0x7f28bc0af3c0 2024-09-21T07:33:56.121 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:33:56.121 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":23,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:33:52.850044+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:33:12.854113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:33:13.622611+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.b","mappings":[{"from":0,"to":6}]},{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.11","mappings":[{"from":7,"to":2}]},{"pgid":"3.13","mappings":[{"from":4,"to":6}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]},{"pgid":"3.2d","mappings":[{"from":7,"to":1}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:33:56.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.122+0000 7f28c2800640 1 -- 172.21.15.39:0/1240233243 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f289c078b90 msgr2=0x7f289c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:56.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.122+0000 7f28c2800640 1 --2- 172.21.15.39:0/1240233243 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f289c078b90 0x7f289c07b0a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f28bc14a7a0 tx=0x7f28a8005f90 comp rx=0 tx=0).stop 2024-09-21T07:33:56.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.122+0000 7f28c2800640 1 -- 172.21.15.39:0/1240233243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28bc0af3c0 msgr2=0x7f28bc146c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:56.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.122+0000 7f28c2800640 1 --2- 172.21.15.39:0/1240233243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28bc0af3c0 0x7f28bc146c30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f28ac00c3d0 tx=0x7f28ac00c890 comp rx=0 tx=0).stop 2024-09-21T07:33:56.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.122+0000 7f28c2800640 1 -- 172.21.15.39:0/1240233243 shutdown_connections 2024-09-21T07:33:56.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.122+0000 7f28c2800640 1 --2- 172.21.15.39:0/1240233243 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f289c078b90 0x7f289c07b0a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:56.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.122+0000 7f28c2800640 1 --2- 172.21.15.39:0/1240233243 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28bc0ccb50 0x7f28bc06ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:56.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.122+0000 7f28c2800640 1 --2- 172.21.15.39:0/1240233243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28bc0cc1e0 0x7f28bc158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:56.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.122+0000 7f28c2800640 1 --2- 172.21.15.39:0/1240233243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28bc0af3c0 0x7f28bc146c30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:56.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.122+0000 7f28c2800640 1 -- 172.21.15.39:0/1240233243 >> 172.21.15.39:0/1240233243 conn(0x7f28bc06d7d0 msgr2=0x7f28bc15fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:56.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.122+0000 7f28c2800640 1 -- 172.21.15.39:0/1240233243 shutdown_connections 2024-09-21T07:33:56.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.122+0000 7f28c2800640 1 -- 172.21.15.39:0/1240233243 wait complete. 2024-09-21T07:33:56.144 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T07:33:56.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.372+0000 7ff9dfc00640 1 -- 172.21.15.39:0/2842889075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9c0004800 msgr2=0x7ff9c0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:56.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.372+0000 7ff9dfc00640 1 --2- 172.21.15.39:0/2842889075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9c0004800 0x7ff9c0004c10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff9c400c290 tx=0x7ff9c0005830 comp rx=0 tx=0).stop 2024-09-21T07:33:56.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.372+0000 7ff9dfc00640 1 -- 172.21.15.39:0/2842889075 shutdown_connections 2024-09-21T07:33:56.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.372+0000 7ff9dfc00640 1 --2- 172.21.15.39:0/2842889075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9c0004800 0x7ff9c0004c10 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:56.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.372+0000 7ff9dfc00640 1 -- 172.21.15.39:0/2842889075 >> 172.21.15.39:0/2842889075 conn(0x7ff9d806d550 msgr2=0x7ff9d8071150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:56.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.372+0000 7ff9dfc00640 1 -- 172.21.15.39:0/2842889075 shutdown_connections 2024-09-21T07:33:56.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.373+0000 7ff9dfc00640 1 -- 172.21.15.39:0/2842889075 wait complete. 2024-09-21T07:33:56.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.375+0000 7ff9dfc00640 1 Processor -- start 2024-09-21T07:33:56.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.375+0000 7ff9dfc00640 1 -- start start 2024-09-21T07:33:56.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.376+0000 7ff9dfc00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9c0004800 0x7ff9d81f3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:56.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.376+0000 7ff9dfc00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff9d81f71b0 0x7ff9d80c7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:56.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.376+0000 7ff9dfc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9d80c5e10 0x7ff9d80c6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:56.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.377+0000 7ff9dfc00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d815d620 con 0x7ff9d80c5e10 2024-09-21T07:33:56.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.377+0000 7ff9dfc00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d815d4a0 con 0x7ff9c0004800 2024-09-21T07:33:56.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.377+0000 7ff9dfc00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7ff9d80d1f20 con 0x7ff9d81f71b0 2024-09-21T07:33:56.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.377+0000 7ff9ddc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9d80c5e10 0x7ff9d80c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:56.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.377+0000 7ff9ddc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9d80c5e10 0x7ff9d80c6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51978/0 (socket says 172.21.15.39:51978) 2024-09-21T07:33:56.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.377+0000 7ff9ddc00640 1 -- 172.21.15.39:0/1952481550 learned_addr learned my addr 172.21.15.39:0/1952481550 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:56.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.377+0000 7ff9d7e00640 1 --2- 172.21.15.39:0/1952481550 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff9d81f71b0 0x7ff9d80c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:56.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.377+0000 7ff9dd200640 1 --2- 172.21.15.39:0/1952481550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9c0004800 0x7ff9d81f3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:56.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.377+0000 7ff9ddc00640 1 -- 172.21.15.39:0/1952481550 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff9d81f71b0 msgr2=0x7ff9d80c7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:56.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.377+0000 7ff9ddc00640 1 --2- 172.21.15.39:0/1952481550 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff9d81f71b0 0x7ff9d80c7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:56.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.377+0000 7ff9ddc00640 1 -- 172.21.15.39:0/1952481550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9c0004800 msgr2=0x7ff9d81f3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:56.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.377+0000 7ff9ddc00640 1 --2- 172.21.15.39:0/1952481550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9c0004800 0x7ff9d81f3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:56.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.377+0000 7ff9ddc00640 1 -- 172.21.15.39:0/1952481550 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d80b3c40 con 0x7ff9d80c5e10 2024-09-21T07:33:56.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.377+0000 7ff9dd200640 1 --2- 172.21.15.39:0/1952481550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9c0004800 0x7ff9d81f3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:33:56.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.378+0000 7ff9ddc00640 1 --2- 172.21.15.39:0/1952481550 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9d80c5e10 0x7ff9d80c6220 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0005fd0 tx=0x7ff9d0010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:56.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.378+0000 7ff9d4c00640 1 -- 172.21.15.39:0/1952481550 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d000eea0 con 0x7ff9d80c5e10 2024-09-21T07:33:56.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.378+0000 7ff9d4c00640 1 -- 172.21.15.39:0/1952481550 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9d0015070 con 0x7ff9d80c5e10 2024-09-21T07:33:56.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.378+0000 7ff9d4c00640 1 -- 172.21.15.39:0/1952481550 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d0003970 con 0x7ff9d80c5e10 2024-09-21T07:33:56.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.378+0000 7ff9dfc00640 1 -- 172.21.15.39:0/1952481550 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9d80b3f20 con 0x7ff9d80c5e10 2024-09-21T07:33:56.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.378+0000 7ff9dfc00640 1 -- 172.21.15.39:0/1952481550 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9d81f35a0 con 0x7ff9d80c5e10 2024-09-21T07:33:56.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.379+0000 7ff9dfc00640 1 -- 172.21.15.39:0/1952481550 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9d80b39b0 con 0x7ff9d80c5e10 2024-09-21T07:33:56.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.380+0000 7ff9d4c00640 1 -- 172.21.15.39:0/1952481550 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff9d0003b10 con 0x7ff9d80c5e10 2024-09-21T07:33:56.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.381+0000 7ff9d4c00640 1 --2- 172.21.15.39:0/1952481550 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff9b8078b90 0x7ff9b807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:56.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.381+0000 7ff9d4c00640 1 -- 172.21.15.39:0/1952481550 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5970+0+0 (secure 0 0 0) 0x7ff9d009b9d0 con 0x7ff9d80c5e10 2024-09-21T07:33:56.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.382+0000 7ff9dd200640 1 --2- 172.21.15.39:0/1952481550 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff9b8078b90 0x7ff9b807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:56.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.382+0000 7ff9dd200640 1 --2- 172.21.15.39:0/1952481550 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff9b8078b90 0x7ff9b807b0a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc000c00 tx=0x7ff9cc006ac0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:56.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.386+0000 7ff9d4c00640 1 -- 172.21.15.39:0/1952481550 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff9d00636a0 con 0x7ff9d80c5e10 2024-09-21T07:33:56.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.623+0000 7ff9dfc00640 1 -- 172.21.15.39:0/1952481550 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff9d80ce3d0 con 0x7ff9d80c5e10 2024-09-21T07:33:56.625 INFO:tasks.ceph.mon.a.smithi039.stderr:2024-09-21T07:33:56.624+0000 7fabab600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T07:33:56.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.627+0000 7ff9d4c00640 1 -- 172.21.15.39:0/1952481550 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7ff9d80ce3d0 con 0x7ff9d80c5e10 2024-09-21T07:33:56.628 INFO:teuthology.orchestra.run.smithi039.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T07:33:56.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.630+0000 7ff9dfc00640 1 -- 172.21.15.39:0/1952481550 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff9b8078b90 msgr2=0x7ff9b807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:56.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.630+0000 7ff9dfc00640 1 --2- 172.21.15.39:0/1952481550 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff9b8078b90 0x7ff9b807b0a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc000c00 tx=0x7ff9cc006ac0 comp rx=0 tx=0).stop 2024-09-21T07:33:56.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.630+0000 7ff9dfc00640 1 -- 172.21.15.39:0/1952481550 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9d80c5e10 msgr2=0x7ff9d80c6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:56.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.630+0000 7ff9dfc00640 1 --2- 172.21.15.39:0/1952481550 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9d80c5e10 0x7ff9d80c6220 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0005fd0 tx=0x7ff9d0010f40 comp rx=0 tx=0).stop 2024-09-21T07:33:56.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.631+0000 7ff9dfc00640 1 -- 172.21.15.39:0/1952481550 shutdown_connections 2024-09-21T07:33:56.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.631+0000 7ff9dfc00640 1 --2- 172.21.15.39:0/1952481550 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff9b8078b90 0x7ff9b807b0a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:56.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.631+0000 7ff9dfc00640 1 --2- 172.21.15.39:0/1952481550 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9d80c5e10 0x7ff9d80c6220 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:56.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.631+0000 7ff9dfc00640 1 --2- 172.21.15.39:0/1952481550 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff9d81f71b0 0x7ff9d80c7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:56.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.631+0000 7ff9dfc00640 1 --2- 172.21.15.39:0/1952481550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9c0004800 0x7ff9d81f3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:56.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.631+0000 7ff9dfc00640 1 -- 172.21.15.39:0/1952481550 >> 172.21.15.39:0/1952481550 conn(0x7ff9d806d550 msgr2=0x7ff9d814f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:56.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.631+0000 7ff9dfc00640 1 -- 172.21.15.39:0/1952481550 shutdown_connections 2024-09-21T07:33:56.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.631+0000 7ff9dfc00640 1 -- 172.21.15.39:0/1952481550 wait complete. 2024-09-21T07:33:56.642 INFO:tasks.ceph.mds.a.smithi039.stderr: -5890> 2024-09-21T07:33:19.121+0000 7fb55e200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:33:56.659 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T07:33:56.768 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-21T07:33:56.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.895+0000 7fd0a7200640 1 -- 172.21.15.39:0/3048501867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0a80b6e20 msgr2=0x7fd0a80b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:56.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.895+0000 7fd0a7200640 1 --2- 172.21.15.39:0/3048501867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0a80b6e20 0x7fd0a80b4f40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8074900 tx=0x7fd09002a540 comp rx=0 tx=0).stop 2024-09-21T07:33:56.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.895+0000 7fd0a7200640 1 -- 172.21.15.39:0/3048501867 shutdown_connections 2024-09-21T07:33:56.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.895+0000 7fd0a7200640 1 --2- 172.21.15.39:0/3048501867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0a80b6e20 0x7fd0a80b4f40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:56.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.895+0000 7fd0a7200640 1 --2- 172.21.15.39:0/3048501867 >> v2:172.21.15.148:3300/0 conn(0x7fd0a80cd860 0x7fd0a80b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:56.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.895+0000 7fd0a7200640 1 -- 172.21.15.39:0/3048501867 >> 172.21.15.39:0/3048501867 conn(0x7fd0a80b3be0 msgr2=0x7fd0a80b3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:56.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.895+0000 7fd0a7200640 1 -- 172.21.15.39:0/3048501867 shutdown_connections 2024-09-21T07:33:56.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.895+0000 7fd0a7200640 1 -- 172.21.15.39:0/3048501867 wait complete. 2024-09-21T07:33:56.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.897+0000 7fd0a7200640 1 Processor -- start 2024-09-21T07:33:56.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.897+0000 7fd0a7200640 1 -- start start 2024-09-21T07:33:56.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.899+0000 7fd0a7200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd0a809aa30 0x7fd0a8159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:56.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.899+0000 7fd0a7200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0a80b6e20 0x7fd0a81f5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:56.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.899+0000 7fd0a7200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0a80cd860 0x7fd0a81f9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:56.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.899+0000 7fd0a7200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a80d0a60 con 0x7fd0a80b6e20 2024-09-21T07:33:56.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.899+0000 7fd0a7200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a80d08e0 con 0x7fd0a80cd860 2024-09-21T07:33:56.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.899+0000 7fd0a7200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd0a80d0be0 con 0x7fd0a809aa30 2024-09-21T07:33:56.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.899+0000 7fd09fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0a80b6e20 0x7fd0a81f5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:56.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.899+0000 7fd09fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0a80b6e20 0x7fd0a81f5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52028/0 (socket says 172.21.15.39:52028) 2024-09-21T07:33:56.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.899+0000 7fd09fe00640 1 -- 172.21.15.39:0/295188983 learned_addr learned my addr 172.21.15.39:0/295188983 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:56.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.899+0000 7fd0a5e00640 1 --2- 172.21.15.39:0/295188983 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd0a809aa30 0x7fd0a8159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:56.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.899+0000 7fd0a6800640 1 --2- 172.21.15.39:0/295188983 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0a80cd860 0x7fd0a81f9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:56.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.900+0000 7fd09fe00640 1 -- 172.21.15.39:0/295188983 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd0a809aa30 msgr2=0x7fd0a8159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:56.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.900+0000 7fd09fe00640 1 --2- 172.21.15.39:0/295188983 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd0a809aa30 0x7fd0a8159b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:56.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.900+0000 7fd09fe00640 1 -- 172.21.15.39:0/295188983 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0a80cd860 msgr2=0x7fd0a81f9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:56.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.900+0000 7fd09fe00640 1 --2- 172.21.15.39:0/295188983 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0a80cd860 0x7fd0a81f9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:56.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.900+0000 7fd09fe00640 1 -- 172.21.15.39:0/295188983 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a81f5480 con 0x7fd0a80b6e20 2024-09-21T07:33:56.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.900+0000 7fd0a6800640 1 --2- 172.21.15.39:0/295188983 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0a80cd860 0x7fd0a81f9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:33:56.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.900+0000 7fd09fe00640 1 --2- 172.21.15.39:0/295188983 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0a80b6e20 0x7fd0a81f5f00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd09002ab30 tx=0x7fd090033c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:56.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.900+0000 7fd09e000640 1 -- 172.21.15.39:0/295188983 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd090031d60 con 0x7fd0a80b6e20 2024-09-21T07:33:56.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.900+0000 7fd09e000640 1 -- 172.21.15.39:0/295188983 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd090005020 con 0x7fd0a80b6e20 2024-09-21T07:33:56.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.900+0000 7fd0a7200640 1 -- 172.21.15.39:0/295188983 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a81f99c0 con 0x7fd0a80b6e20 2024-09-21T07:33:56.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.900+0000 7fd09e000640 1 -- 172.21.15.39:0/295188983 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd090005470 con 0x7fd0a80b6e20 2024-09-21T07:33:56.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.901+0000 7fd0a7200640 1 -- 172.21.15.39:0/295188983 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a815af80 con 0x7fd0a80b6e20 2024-09-21T07:33:56.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.902+0000 7fd0a7200640 1 -- 172.21.15.39:0/295188983 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0a80d1090 con 0x7fd0a80b6e20 2024-09-21T07:33:56.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.903+0000 7fd09e000640 1 -- 172.21.15.39:0/295188983 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd0900047a0 con 0x7fd0a80b6e20 2024-09-21T07:33:56.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.904+0000 7fd09e000640 1 --2- 172.21.15.39:0/295188983 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd084078b90 0x7fd08407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:56.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.904+0000 7fd09e000640 1 -- 172.21.15.39:0/295188983 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6056+0+0 (secure 0 0 0) 0x7fd0900bc140 con 0x7fd0a80b6e20 2024-09-21T07:33:56.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.904+0000 7fd0a5e00640 1 --2- 172.21.15.39:0/295188983 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd084078b90 0x7fd08407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:56.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.905+0000 7fd0a5e00640 1 --2- 172.21.15.39:0/295188983 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd084078b90 0x7fd08407b0a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd09800bb30 tx=0x7fd09800b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:56.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:56.908+0000 7fd09e000640 1 -- 172.21.15.39:0/295188983 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd0a81f99c0 con 0x7fd0a80b6e20 2024-09-21T07:33:57.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.147+0000 7fd0a7200640 1 -- 172.21.15.39:0/295188983 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fd0a80ce8e0 con 0x7fd0a80b6e20 2024-09-21T07:33:57.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.629+0000 7fd09e000640 1 -- 172.21.15.39:0/295188983 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7fd0a81f5480 con 0x7fd0a80b6e20 2024-09-21T07:33:57.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.632+0000 7fd0a7200640 1 -- 172.21.15.39:0/295188983 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd084078b90 msgr2=0x7fd08407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:57.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.632+0000 7fd0a7200640 1 --2- 172.21.15.39:0/295188983 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd084078b90 0x7fd08407b0a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd09800bb30 tx=0x7fd09800b2e0 comp rx=0 tx=0).stop 2024-09-21T07:33:57.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.632+0000 7fd0a7200640 1 -- 172.21.15.39:0/295188983 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0a80b6e20 msgr2=0x7fd0a81f5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:57.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.632+0000 7fd0a7200640 1 --2- 172.21.15.39:0/295188983 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0a80b6e20 0x7fd0a81f5f00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd09002ab30 tx=0x7fd090033c90 comp rx=0 tx=0).stop 2024-09-21T07:33:57.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.633+0000 7fd0a7200640 1 -- 172.21.15.39:0/295188983 shutdown_connections 2024-09-21T07:33:57.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.633+0000 7fd0a7200640 1 --2- 172.21.15.39:0/295188983 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd084078b90 0x7fd08407b0a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:57.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.633+0000 7fd0a7200640 1 --2- 172.21.15.39:0/295188983 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0a80cd860 0x7fd0a81f9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:57.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.633+0000 7fd0a7200640 1 --2- 172.21.15.39:0/295188983 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0a80b6e20 0x7fd0a81f5f00 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:57.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.633+0000 7fd0a7200640 1 --2- 172.21.15.39:0/295188983 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd0a809aa30 0x7fd0a8159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:57.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.633+0000 7fd0a7200640 1 -- 172.21.15.39:0/295188983 >> 172.21.15.39:0/295188983 conn(0x7fd0a80b3be0 msgr2=0x7fd0a80b2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:57.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.633+0000 7fd0a7200640 1 -- 172.21.15.39:0/295188983 shutdown_connections 2024-09-21T07:33:57.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.633+0000 7fd0a7200640 1 -- 172.21.15.39:0/295188983 wait complete. 2024-09-21T07:33:57.656 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T07:33:57.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.887+0000 7f1012a00640 1 -- 172.21.15.39:0/817220972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f100c0d07d0 msgr2=0x7f100c073f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:57.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.887+0000 7f1012a00640 1 --2- 172.21.15.39:0/817220972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f100c0d07d0 0x7f100c073f70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f100c0cc700 tx=0x7f100402aac0 comp rx=0 tx=0).stop 2024-09-21T07:33:57.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.887+0000 7f1012a00640 1 -- 172.21.15.39:0/817220972 shutdown_connections 2024-09-21T07:33:57.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.887+0000 7f1012a00640 1 --2- 172.21.15.39:0/817220972 >> v2:172.21.15.148:3300/0 conn(0x7f100c142b80 0x7f100c0d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:57.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.887+0000 7f1012a00640 1 --2- 172.21.15.39:0/817220972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f100c0d07d0 0x7f100c073f70 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:57.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.887+0000 7f1012a00640 1 -- 172.21.15.39:0/817220972 >> 172.21.15.39:0/817220972 conn(0x7f100c06cfc0 msgr2=0x7f100c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:57.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.887+0000 7f1012a00640 1 -- 172.21.15.39:0/817220972 shutdown_connections 2024-09-21T07:33:57.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.887+0000 7f1012a00640 1 -- 172.21.15.39:0/817220972 wait complete. 2024-09-21T07:33:57.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.889+0000 7f1012a00640 1 Processor -- start 2024-09-21T07:33:57.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.889+0000 7f1012a00640 1 -- start start 2024-09-21T07:33:57.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.891+0000 7f1012a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f100c06f730 0x7f100c0b4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:57.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.891+0000 7f1012a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f100c0d07d0 0x7f100c1387e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:57.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.891+0000 7f1012a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f100c142b80 0x7f100c0aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:57.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.891+0000 7f1012a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f100c0748c0 con 0x7f100c06f730 2024-09-21T07:33:57.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.891+0000 7f1012a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f100c074740 con 0x7f100c142b80 2024-09-21T07:33:57.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.891+0000 7f1012a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f100c074a40 con 0x7f100c0d07d0 2024-09-21T07:33:57.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.891+0000 7f1011600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f100c06f730 0x7f100c0b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:57.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.891+0000 7f1011600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f100c06f730 0x7f100c0b4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52042/0 (socket says 172.21.15.39:52042) 2024-09-21T07:33:57.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.891+0000 7f1011600640 1 -- 172.21.15.39:0/3260993432 learned_addr learned my addr 172.21.15.39:0/3260993432 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:57.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.891+0000 7f1010c00640 1 --2- 172.21.15.39:0/3260993432 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f100c0d07d0 0x7f100c1387e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:57.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.891+0000 7f1012000640 1 --2- 172.21.15.39:0/3260993432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f100c142b80 0x7f100c0aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:57.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.891+0000 7f1011600640 1 -- 172.21.15.39:0/3260993432 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f100c0d07d0 msgr2=0x7f100c1387e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:57.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.891+0000 7f1011600640 1 --2- 172.21.15.39:0/3260993432 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f100c0d07d0 0x7f100c1387e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:57.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.891+0000 7f1011600640 1 -- 172.21.15.39:0/3260993432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f100c142b80 msgr2=0x7f100c0aebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:57.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.891+0000 7f1011600640 1 --2- 172.21.15.39:0/3260993432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f100c142b80 0x7f100c0aebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:57.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.891+0000 7f1011600640 1 -- 172.21.15.39:0/3260993432 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f100c0b3820 con 0x7f100c06f730 2024-09-21T07:33:57.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.892+0000 7f1011600640 1 --2- 172.21.15.39:0/3260993432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f100c06f730 0x7f100c0b4100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f100000c3a0 tx=0x7f100000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:57.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.892+0000 7f1009600640 1 -- 172.21.15.39:0/3260993432 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10000052d0 con 0x7f100c06f730 2024-09-21T07:33:57.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.892+0000 7f1009600640 1 -- 172.21.15.39:0/3260993432 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1000007020 con 0x7f100c06f730 2024-09-21T07:33:57.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.892+0000 7f1009600640 1 -- 172.21.15.39:0/3260993432 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1000007470 con 0x7f100c06f730 2024-09-21T07:33:57.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.892+0000 7f1012a00640 1 -- 172.21.15.39:0/3260993432 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f100c13ee70 con 0x7f100c06f730 2024-09-21T07:33:57.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.892+0000 7f1012a00640 1 -- 172.21.15.39:0/3260993432 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f100c0b2820 con 0x7f100c06f730 2024-09-21T07:33:57.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.893+0000 7f1012a00640 1 -- 172.21.15.39:0/3260993432 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f100c074f60 con 0x7f100c06f730 2024-09-21T07:33:57.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.894+0000 7f1009600640 1 -- 172.21.15.39:0/3260993432 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f10000067d0 con 0x7f100c06f730 2024-09-21T07:33:57.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.895+0000 7f1009600640 1 --2- 172.21.15.39:0/3260993432 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0fec078b90 0x7f0fec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:57.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.896+0000 7f1009600640 1 -- 172.21.15.39:0/3260993432 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6056+0+0 (secure 0 0 0) 0x7f100009a350 con 0x7f100c06f730 2024-09-21T07:33:57.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.896+0000 7f1010c00640 1 --2- 172.21.15.39:0/3260993432 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0fec078b90 0x7f0fec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:57.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.897+0000 7f1010c00640 1 --2- 172.21.15.39:0/3260993432 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0fec078b90 0x7f0fec07b0a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f100c15efb0 tx=0x7f0ff8005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:57.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:57.900+0000 7f1009600640 1 -- 172.21.15.39:0/3260993432 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f100c0d0be0 con 0x7f100c06f730 2024-09-21T07:33:58.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:58.098+0000 7f1012a00640 1 -- 172.21.15.39:0/3260993432 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f100c145b20 con 0x7f100c06f730 2024-09-21T07:33:58.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:58.896+0000 7f1009600640 1 -- 172.21.15.39:0/3260993432 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) ==== 181+0+0 (secure 0 0 0) 0x7f1000061fd0 con 0x7f100c06f730 2024-09-21T07:33:58.897 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' removed 2024-09-21T07:33:58.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:58.900+0000 7f1012a00640 1 -- 172.21.15.39:0/3260993432 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0fec078b90 msgr2=0x7f0fec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:58.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:58.900+0000 7f1012a00640 1 --2- 172.21.15.39:0/3260993432 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0fec078b90 0x7f0fec07b0a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f100c15efb0 tx=0x7f0ff8005f90 comp rx=0 tx=0).stop 2024-09-21T07:33:58.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:58.900+0000 7f1012a00640 1 -- 172.21.15.39:0/3260993432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f100c06f730 msgr2=0x7f100c0b4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:58.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:58.900+0000 7f1012a00640 1 --2- 172.21.15.39:0/3260993432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f100c06f730 0x7f100c0b4100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f100000c3a0 tx=0x7f100000c860 comp rx=0 tx=0).stop 2024-09-21T07:33:58.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:58.901+0000 7f1012a00640 1 -- 172.21.15.39:0/3260993432 shutdown_connections 2024-09-21T07:33:58.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:58.901+0000 7f1012a00640 1 --2- 172.21.15.39:0/3260993432 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0fec078b90 0x7f0fec07b0a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:58.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:58.901+0000 7f1012a00640 1 --2- 172.21.15.39:0/3260993432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f100c142b80 0x7f100c0aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:58.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:58.901+0000 7f1012a00640 1 --2- 172.21.15.39:0/3260993432 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f100c0d07d0 0x7f100c1387e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:58.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:58.901+0000 7f1012a00640 1 --2- 172.21.15.39:0/3260993432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f100c06f730 0x7f100c0b4100 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:58.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:58.901+0000 7f1012a00640 1 -- 172.21.15.39:0/3260993432 >> 172.21.15.39:0/3260993432 conn(0x7f100c06cfc0 msgr2=0x7f100c0b4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:58.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:58.901+0000 7f1012a00640 1 -- 172.21.15.39:0/3260993432 shutdown_connections 2024-09-21T07:33:58.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:58.901+0000 7f1012a00640 1 -- 172.21.15.39:0/3260993432 wait complete. 2024-09-21T07:33:58.929 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T07:33:59.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.163+0000 7f0946000640 1 -- 172.21.15.39:0/1534615386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09400b5310 msgr2=0x7f09400b5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:59.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.163+0000 7f0946000640 1 --2- 172.21.15.39:0/1534615386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09400b5310 0x7f09400b5720 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0938005fd0 tx=0x7f093802a820 comp rx=0 tx=0).stop 2024-09-21T07:33:59.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.163+0000 7f0946000640 1 -- 172.21.15.39:0/1534615386 shutdown_connections 2024-09-21T07:33:59.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.163+0000 7f0946000640 1 --2- 172.21.15.39:0/1534615386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09400b5310 0x7f09400b5720 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:59.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.163+0000 7f0946000640 1 --2- 172.21.15.39:0/1534615386 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0940073590 0x7f094015d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:59.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.163+0000 7f0946000640 1 -- 172.21.15.39:0/1534615386 >> 172.21.15.39:0/1534615386 conn(0x7f094006ee70 msgr2=0x7f094006f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:59.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.163+0000 7f0946000640 1 -- 172.21.15.39:0/1534615386 shutdown_connections 2024-09-21T07:33:59.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.163+0000 7f0946000640 1 -- 172.21.15.39:0/1534615386 wait complete. 2024-09-21T07:33:59.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.166+0000 7f0946000640 1 Processor -- start 2024-09-21T07:33:59.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.166+0000 7f0946000640 1 -- start start 2024-09-21T07:33:59.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.167+0000 7f0946000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0940073590 0x7f09400b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:59.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.167+0000 7f0946000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0940074d30 0x7f094015a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:59.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.167+0000 7f0946000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09400b5310 0x7f094015aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:59.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.167+0000 7f0946000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f09400cf1b0 con 0x7f0940073590 2024-09-21T07:33:59.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.167+0000 7f0946000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f09400cf030 con 0x7f09400b5310 2024-09-21T07:33:59.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.167+0000 7f0946000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f09400cf330 con 0x7f0940074d30 2024-09-21T07:33:59.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.167+0000 7f0944c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0940073590 0x7f09400b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:59.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.168+0000 7f0944c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0940073590 0x7f09400b34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52056/0 (socket says 172.21.15.39:52056) 2024-09-21T07:33:59.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.168+0000 7f0944c00640 1 -- 172.21.15.39:0/3544106845 learned_addr learned my addr 172.21.15.39:0/3544106845 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:33:59.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.168+0000 7f0937e00640 1 --2- 172.21.15.39:0/3544106845 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0940074d30 0x7f094015a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:59.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.168+0000 7f0945600640 1 --2- 172.21.15.39:0/3544106845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09400b5310 0x7f094015aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:59.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.168+0000 7f0944c00640 1 -- 172.21.15.39:0/3544106845 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0940074d30 msgr2=0x7f094015a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:59.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.168+0000 7f0944c00640 1 --2- 172.21.15.39:0/3544106845 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0940074d30 0x7f094015a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:59.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.168+0000 7f0944c00640 1 -- 172.21.15.39:0/3544106845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09400b5310 msgr2=0x7f094015aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:59.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.168+0000 7f0944c00640 1 --2- 172.21.15.39:0/3544106845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09400b5310 0x7f094015aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:59.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.168+0000 7f0944c00640 1 -- 172.21.15.39:0/3544106845 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0940159bd0 con 0x7f0940073590 2024-09-21T07:33:59.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.168+0000 7f0944c00640 1 --2- 172.21.15.39:0/3544106845 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0940073590 0x7f09400b34b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f093000c3c0 tx=0x7f093000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:59.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.169+0000 7f093d600640 1 -- 172.21.15.39:0/3544106845 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0930017070 con 0x7f0940073590 2024-09-21T07:33:59.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.169+0000 7f093d600640 1 -- 172.21.15.39:0/3544106845 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09300035f0 con 0x7f0940073590 2024-09-21T07:33:59.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.169+0000 7f093d600640 1 -- 172.21.15.39:0/3544106845 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0930003a80 con 0x7f0940073590 2024-09-21T07:33:59.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.169+0000 7f0946000640 1 -- 172.21.15.39:0/3544106845 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0940159eb0 con 0x7f0940073590 2024-09-21T07:33:59.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.169+0000 7f0946000640 1 -- 172.21.15.39:0/3544106845 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f094015b410 con 0x7f0940073590 2024-09-21T07:33:59.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.170+0000 7f0946000640 1 -- 172.21.15.39:0/3544106845 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09400b3a40 con 0x7f0940073590 2024-09-21T07:33:59.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.171+0000 7f093d600640 1 -- 172.21.15.39:0/3544106845 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f093001a070 con 0x7f0940073590 2024-09-21T07:33:59.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.172+0000 7f093d600640 1 --2- 172.21.15.39:0/3544106845 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f091c078b90 0x7f091c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:33:59.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.172+0000 7f093d600640 1 -- 172.21.15.39:0/3544106845 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5618+0+0 (secure 0 0 0) 0x7f0930099ff0 con 0x7f0940073590 2024-09-21T07:33:59.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.172+0000 7f0937e00640 1 --2- 172.21.15.39:0/3544106845 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f091c078b90 0x7f091c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:33:59.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.173+0000 7f0937e00640 1 --2- 172.21.15.39:0/3544106845 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f091c078b90 0x7f091c07b0a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f094015bbe0 tx=0x7f092800b390 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:33:59.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.177+0000 7f093d600640 1 -- 172.21.15.39:0/3544106845 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f09400b5c60 con 0x7f0940073590 2024-09-21T07:33:59.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.374+0000 7f0946000640 1 -- 172.21.15.39:0/3544106845 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f094015d680 con 0x7f0940073590 2024-09-21T07:33:59.826 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:33:59.832 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:33:59.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.902+0000 7f093d600640 1 -- 172.21.15.39:0/3544106845 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) ==== 169+0+0 (secure 0 0 0) 0x7f094015d680 con 0x7f0940073590 2024-09-21T07:33:59.903 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' removed 2024-09-21T07:33:59.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.904+0000 7f0946000640 1 -- 172.21.15.39:0/3544106845 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f091c078b90 msgr2=0x7f091c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:59.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.904+0000 7f0946000640 1 --2- 172.21.15.39:0/3544106845 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f091c078b90 0x7f091c07b0a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f094015bbe0 tx=0x7f092800b390 comp rx=0 tx=0).stop 2024-09-21T07:33:59.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.904+0000 7f0946000640 1 -- 172.21.15.39:0/3544106845 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0940073590 msgr2=0x7f09400b34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:33:59.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.904+0000 7f0946000640 1 --2- 172.21.15.39:0/3544106845 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0940073590 0x7f09400b34b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f093000c3c0 tx=0x7f093000c880 comp rx=0 tx=0).stop 2024-09-21T07:33:59.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.904+0000 7f0946000640 1 -- 172.21.15.39:0/3544106845 shutdown_connections 2024-09-21T07:33:59.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.904+0000 7f0946000640 1 --2- 172.21.15.39:0/3544106845 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f091c078b90 0x7f091c07b0a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:59.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.904+0000 7f0946000640 1 --2- 172.21.15.39:0/3544106845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09400b5310 0x7f094015aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:59.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.904+0000 7f0946000640 1 --2- 172.21.15.39:0/3544106845 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0940074d30 0x7f094015a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:59.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.904+0000 7f0946000640 1 --2- 172.21.15.39:0/3544106845 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0940073590 0x7f09400b34b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:33:59.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.904+0000 7f0946000640 1 -- 172.21.15.39:0/3544106845 >> 172.21.15.39:0/3544106845 conn(0x7f094006ee70 msgr2=0x7f0940144280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:33:59.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.905+0000 7f0946000640 1 -- 172.21.15.39:0/3544106845 shutdown_connections 2024-09-21T07:33:59.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:33:59.905+0000 7f0946000640 1 -- 172.21.15.39:0/3544106845 wait complete. 2024-09-21T07:33:59.934 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T07:33:59.934 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T07:33:59.934 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T07:33:59.934 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:33:59.935 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T07:33:59.935 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T07:33:59.935 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T07:33:59.935 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:33:59.935 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T07:33:59.935 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T07:33:59.935 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T07:33:59.935 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:33:59.936 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T07:33:59.936 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T07:33:59.936 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T07:33:59.936 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:33:59.959 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T07:33:59.960 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T07:33:59.961 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T07:33:59.961 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T07:34:00.006 INFO:tasks.ceph.mds.b:Started 2024-09-21T07:34:00.006 DEBUG:teuthology.parallel:result is None 2024-09-21T07:34:00.012 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T07:34:00.012 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T07:34:00.013 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T07:34:00.013 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T07:34:00.013 INFO:tasks.ceph.mds.d:Started 2024-09-21T07:34:00.014 DEBUG:teuthology.parallel:result is None 2024-09-21T07:34:00.059 INFO:tasks.ceph.mds.c:Started 2024-09-21T07:34:00.059 DEBUG:teuthology.parallel:result is None 2024-09-21T07:34:00.060 INFO:tasks.ceph.mds.a:Started 2024-09-21T07:34:00.061 DEBUG:teuthology.parallel:result is None 2024-09-21T07:34:00.061 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T07:34:00.202 INFO:tasks.ceph.mds.b.smithi148.stdout:starting mds.b at 2024-09-21T07:34:00.210 INFO:tasks.ceph.mds.d.smithi148.stdout:starting mds.d at 2024-09-21T07:34:00.258 INFO:tasks.ceph.mds.c.smithi039.stdout:starting mds.c at 2024-09-21T07:34:00.258 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-21T07:34:00.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.296+0000 7f5774000640 1 -- 172.21.15.39:0/26420897 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f576c15d840 msgr2=0x7f576c0b7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:00.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.296+0000 7f5774000640 1 --2- 172.21.15.39:0/26420897 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f576c15d840 0x7f576c0b7490 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5764005fd0 tx=0x7f576402a840 comp rx=0 tx=0).stop 2024-09-21T07:34:00.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.296+0000 7f5774000640 1 -- 172.21.15.39:0/26420897 shutdown_connections 2024-09-21T07:34:00.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.296+0000 7f5774000640 1 --2- 172.21.15.39:0/26420897 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f576c15d840 0x7f576c0b7490 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:00.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.296+0000 7f5774000640 1 -- 172.21.15.39:0/26420897 >> 172.21.15.39:0/26420897 conn(0x7f576c06d550 msgr2=0x7f576c071150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:00.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.296+0000 7f5774000640 1 -- 172.21.15.39:0/26420897 shutdown_connections 2024-09-21T07:34:00.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.296+0000 7f5774000640 1 -- 172.21.15.39:0/26420897 wait complete. 2024-09-21T07:34:00.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.298+0000 7f5774000640 1 Processor -- start 2024-09-21T07:34:00.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.299+0000 7f5774000640 1 -- start start 2024-09-21T07:34:00.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.300+0000 7f5774000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f576c1570e0 0x7f576c06def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:00.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.300+0000 7f5774000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f576c15d840 0x7f576c14d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:00.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.300+0000 7f5774000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f576c154eb0 0x7f576c1552c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:00.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.300+0000 7f5774000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f576c0b7d60 con 0x7f576c154eb0 2024-09-21T07:34:00.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.300+0000 7f5774000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f576c0b7be0 con 0x7f576c15d840 2024-09-21T07:34:00.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.300+0000 7f5774000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f576c0b7ee0 con 0x7f576c1570e0 2024-09-21T07:34:00.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.300+0000 7f5772000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f576c154eb0 0x7f576c1552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:00.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.300+0000 7f5772000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f576c154eb0 0x7f576c1552c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52108/0 (socket says 172.21.15.39:52108) 2024-09-21T07:34:00.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.300+0000 7f5772000640 1 -- 172.21.15.39:0/3987307746 learned_addr learned my addr 172.21.15.39:0/3987307746 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:00.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.301+0000 7f5770c00640 1 --2- 172.21.15.39:0/3987307746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f576c15d840 0x7f576c14d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:00.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.301+0000 7f5771600640 1 --2- 172.21.15.39:0/3987307746 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f576c1570e0 0x7f576c06def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:00.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.301+0000 7f5772000640 1 -- 172.21.15.39:0/3987307746 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f576c1570e0 msgr2=0x7f576c06def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:00.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.301+0000 7f5772000640 1 --2- 172.21.15.39:0/3987307746 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f576c1570e0 0x7f576c06def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:00.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.301+0000 7f5772000640 1 -- 172.21.15.39:0/3987307746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f576c15d840 msgr2=0x7f576c14d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:00.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.301+0000 7f5772000640 1 --2- 172.21.15.39:0/3987307746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f576c15d840 0x7f576c14d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:00.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.301+0000 7f5772000640 1 -- 172.21.15.39:0/3987307746 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f576c155800 con 0x7f576c154eb0 2024-09-21T07:34:00.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.301+0000 7f5770c00640 1 --2- 172.21.15.39:0/3987307746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f576c15d840 0x7f576c14d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:34:00.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.301+0000 7f5772000640 1 --2- 172.21.15.39:0/3987307746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f576c154eb0 0x7f576c1552c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5764034040 tx=0x7f5764003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:00.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.301+0000 7f5769600640 1 -- 172.21.15.39:0/3987307746 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5764040070 con 0x7f576c154eb0 2024-09-21T07:34:00.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.301+0000 7f5769600640 1 -- 172.21.15.39:0/3987307746 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5764004540 con 0x7f576c154eb0 2024-09-21T07:34:00.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.301+0000 7f5774000640 1 -- 172.21.15.39:0/3987307746 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f576c1f6b90 con 0x7f576c154eb0 2024-09-21T07:34:00.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.301+0000 7f5769600640 1 -- 172.21.15.39:0/3987307746 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576403c420 con 0x7f576c154eb0 2024-09-21T07:34:00.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.302+0000 7f5774000640 1 -- 172.21.15.39:0/3987307746 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f576c1f5dd0 con 0x7f576c154eb0 2024-09-21T07:34:00.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.303+0000 7f5774000640 1 -- 172.21.15.39:0/3987307746 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f576c0c7bf0 con 0x7f576c154eb0 2024-09-21T07:34:00.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.304+0000 7f5769600640 1 -- 172.21.15.39:0/3987307746 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5764043070 con 0x7f576c154eb0 2024-09-21T07:34:00.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.305+0000 7f5769600640 1 --2- 172.21.15.39:0/3987307746 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f574c078b90 0x7f574c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:00.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.305+0000 7f5769600640 1 -- 172.21.15.39:0/3987307746 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 4943+0+0 (secure 0 0 0) 0x7f57640bbfb0 con 0x7f576c154eb0 2024-09-21T07:34:00.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.305+0000 7f5771600640 1 --2- 172.21.15.39:0/3987307746 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f574c078b90 0x7f574c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:00.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.306+0000 7f5771600640 1 --2- 172.21.15.39:0/3987307746 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f574c078b90 0x7f574c07b0a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5760007fd0 tx=0x7f5760009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:00.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.309+0000 7f5769600640 1 -- 172.21.15.39:0/3987307746 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f576c155800 con 0x7f576c154eb0 2024-09-21T07:34:00.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.516+0000 7f5774000640 1 -- 172.21.15.39:0/3987307746 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f576c0b7490 con 0x7f576c154eb0 2024-09-21T07:34:00.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.923+0000 7f5769600640 1 -- 172.21.15.39:0/3987307746 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v28) ==== 97+0+0 (secure 0 0 0) 0x7f5764084650 con 0x7f576c154eb0 2024-09-21T07:34:00.924 INFO:teuthology.orchestra.run.smithi039.stderr: removed all blocklist entries 2024-09-21T07:34:00.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.926+0000 7f5774000640 1 -- 172.21.15.39:0/3987307746 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f574c078b90 msgr2=0x7f574c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:00.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.926+0000 7f5774000640 1 --2- 172.21.15.39:0/3987307746 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f574c078b90 0x7f574c07b0a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5760007fd0 tx=0x7f5760009040 comp rx=0 tx=0).stop 2024-09-21T07:34:00.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.926+0000 7f5774000640 1 -- 172.21.15.39:0/3987307746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f576c154eb0 msgr2=0x7f576c1552c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:00.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.926+0000 7f5774000640 1 --2- 172.21.15.39:0/3987307746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f576c154eb0 0x7f576c1552c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5764034040 tx=0x7f5764003d60 comp rx=0 tx=0).stop 2024-09-21T07:34:00.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.926+0000 7f5774000640 1 -- 172.21.15.39:0/3987307746 shutdown_connections 2024-09-21T07:34:00.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.926+0000 7f5774000640 1 --2- 172.21.15.39:0/3987307746 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f574c078b90 0x7f574c07b0a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:00.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.926+0000 7f5774000640 1 --2- 172.21.15.39:0/3987307746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f576c154eb0 0x7f576c1552c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:00.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.926+0000 7f5774000640 1 --2- 172.21.15.39:0/3987307746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f576c15d840 0x7f576c14d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:00.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.926+0000 7f5774000640 1 --2- 172.21.15.39:0/3987307746 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f576c1570e0 0x7f576c06def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:00.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.926+0000 7f5774000640 1 -- 172.21.15.39:0/3987307746 >> 172.21.15.39:0/3987307746 conn(0x7f576c06d550 msgr2=0x7f576c15fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:00.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.926+0000 7f5774000640 1 -- 172.21.15.39:0/3987307746 shutdown_connections 2024-09-21T07:34:00.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:00.926+0000 7f5774000640 1 -- 172.21.15.39:0/3987307746 wait complete. 2024-09-21T07:34:00.955 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T07:34:01.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.184+0000 7efc17e00640 1 -- 172.21.15.39:0/3668421211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc180af700 msgr2=0x7efc180caa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:01.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.184+0000 7efc17e00640 1 --2- 172.21.15.39:0/3668421211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc180af700 0x7efc180caa80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efc08009830 tx=0x7efc0802a510 comp rx=0 tx=0).stop 2024-09-21T07:34:01.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.184+0000 7efc17e00640 1 -- 172.21.15.39:0/3668421211 shutdown_connections 2024-09-21T07:34:01.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.184+0000 7efc17e00640 1 --2- 172.21.15.39:0/3668421211 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc180735d0 0x7efc18153510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:01.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.184+0000 7efc17e00640 1 --2- 172.21.15.39:0/3668421211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc180af700 0x7efc180caa80 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:01.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.184+0000 7efc17e00640 1 -- 172.21.15.39:0/3668421211 >> 172.21.15.39:0/3668421211 conn(0x7efc1806d870 msgr2=0x7efc1806ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:01.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.184+0000 7efc17e00640 1 -- 172.21.15.39:0/3668421211 shutdown_connections 2024-09-21T07:34:01.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.184+0000 7efc17e00640 1 -- 172.21.15.39:0/3668421211 wait complete. 2024-09-21T07:34:01.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.186+0000 7efc17e00640 1 Processor -- start 2024-09-21T07:34:01.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.186+0000 7efc17e00640 1 -- start start 2024-09-21T07:34:01.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.188+0000 7efc17e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc180735d0 0x7efc1813e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:01.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.188+0000 7efc17e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc180af700 0x7efc1815f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:01.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.188+0000 7efc17e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc180b54d0 0x7efc1815f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:01.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.188+0000 7efc17e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efc1815d2b0 con 0x7efc180735d0 2024-09-21T07:34:01.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.188+0000 7efc17e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efc1815d130 con 0x7efc180af700 2024-09-21T07:34:01.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.188+0000 7efc17e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7efc1815d430 con 0x7efc180b54d0 2024-09-21T07:34:01.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.188+0000 7efc16a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc180735d0 0x7efc1813e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:01.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.188+0000 7efc16a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc180735d0 0x7efc1813e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52112/0 (socket says 172.21.15.39:52112) 2024-09-21T07:34:01.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.188+0000 7efc16a00640 1 -- 172.21.15.39:0/2136883276 learned_addr learned my addr 172.21.15.39:0/2136883276 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:01.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.189+0000 7efc17400640 1 --2- 172.21.15.39:0/2136883276 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc180b54d0 0x7efc1815f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:01.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.189+0000 7efc0fe00640 1 --2- 172.21.15.39:0/2136883276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc180af700 0x7efc1815f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:01.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.189+0000 7efc16a00640 1 -- 172.21.15.39:0/2136883276 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc180b54d0 msgr2=0x7efc1815f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:01.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.189+0000 7efc16a00640 1 --2- 172.21.15.39:0/2136883276 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc180b54d0 0x7efc1815f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:01.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.189+0000 7efc16a00640 1 -- 172.21.15.39:0/2136883276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc180af700 msgr2=0x7efc1815f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:01.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.189+0000 7efc16a00640 1 --2- 172.21.15.39:0/2136883276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc180af700 0x7efc1815f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:01.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.189+0000 7efc16a00640 1 -- 172.21.15.39:0/2136883276 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc18141230 con 0x7efc180735d0 2024-09-21T07:34:01.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.189+0000 7efc16a00640 1 --2- 172.21.15.39:0/2136883276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc180735d0 0x7efc1813e480 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7efc08009950 tx=0x7efc08033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:01.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.190+0000 7efc0ea00640 1 -- 172.21.15.39:0/2136883276 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc08031d60 con 0x7efc180735d0 2024-09-21T07:34:01.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.190+0000 7efc0ea00640 1 -- 172.21.15.39:0/2136883276 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc08033e10 con 0x7efc180735d0 2024-09-21T07:34:01.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.190+0000 7efc17e00640 1 -- 172.21.15.39:0/2136883276 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc18141510 con 0x7efc180735d0 2024-09-21T07:34:01.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.190+0000 7efc0ea00640 1 -- 172.21.15.39:0/2136883276 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc08033310 con 0x7efc180735d0 2024-09-21T07:34:01.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.190+0000 7efc17e00640 1 -- 172.21.15.39:0/2136883276 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc18141ad0 con 0x7efc180735d0 2024-09-21T07:34:01.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.191+0000 7efc17e00640 1 -- 172.21.15.39:0/2136883276 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc1815ea60 con 0x7efc180735d0 2024-09-21T07:34:01.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.192+0000 7efc0ea00640 1 -- 172.21.15.39:0/2136883276 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7efc080334b0 con 0x7efc180735d0 2024-09-21T07:34:01.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.193+0000 7efc0ea00640 1 --2- 172.21.15.39:0/2136883276 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efbf4078bc0 0x7efbf407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:01.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.193+0000 7efc0ea00640 1 -- 172.21.15.39:0/2136883276 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7efc080bbd00 con 0x7efc180735d0 2024-09-21T07:34:01.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.194+0000 7efc0fe00640 1 --2- 172.21.15.39:0/2136883276 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efbf4078bc0 0x7efbf407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:01.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.194+0000 7efc0fe00640 1 --2- 172.21.15.39:0/2136883276 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efbf4078bc0 0x7efbf407b0d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efc000095f0 tx=0x7efc00005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:01.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.202+0000 7efc0ea00640 1 -- 172.21.15.39:0/2136883276 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7efc0803f030 con 0x7efc180735d0 2024-09-21T07:34:01.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.476+0000 7efc17e00640 1 -- 172.21.15.39:0/2136883276 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7efc1815c740 con 0x7efc180735d0 2024-09-21T07:34:01.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.477+0000 7efc0ea00640 1 -- 172.21.15.39:0/2136883276 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) ==== 80+0+6645 (secure 0 0 0) 0x7efc08083e40 con 0x7efc180735d0 2024-09-21T07:34:01.478 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:34:01.478 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:34:01.478 INFO:teuthology.orchestra.run.smithi039.stdout: "auth_dump": [ 2024-09-21T07:34:01.478 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:01.478 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.a", 2024-09-21T07:34:01.478 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5m+P6EERAABQqmlkrocSi1+hwc73Adrg==", 2024-09-21T07:34:01.478 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:01.478 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:34:01.478 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:34:01.478 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:34:01.479 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:34:01.479 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:01.479 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:01.479 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:01.479 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.b", 2024-09-21T07:34:01.479 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5m5/GeLRAAfE8fr5dbitQa89rNK9pZ2Q==", 2024-09-21T07:34:01.479 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:01.479 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:34:01.479 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:34:01.479 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:34:01.479 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:34:01.479 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:01.479 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:01.479 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:01.480 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.c", 2024-09-21T07:34:01.480 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5msbx6HxAALsF8ORvIoJpIAFZ5uxmOEw==", 2024-09-21T07:34:01.480 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:01.480 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:34:01.480 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:34:01.480 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:34:01.480 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:34:01.480 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:01.480 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:01.480 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:01.480 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.d", 2024-09-21T07:34:01.480 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mIrcBABAApKAB/7FkZ5glNTxlbkIsBw==", 2024-09-21T07:34:01.480 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:01.480 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:34:01.480 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:34:01.481 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:34:01.481 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:34:01.481 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:01.481 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:01.481 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:01.481 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.0", 2024-09-21T07:34:01.481 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB7du5msw81DhAAmVimqN4x9K3YpT0nbovppQ==", 2024-09-21T07:34:01.481 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:01.481 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:34:01.481 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:34:01.481 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:34:01.481 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:01.481 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:01.481 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:01.481 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.1", 2024-09-21T07:34:01.482 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB+du5mdnCIOhAAtVB4ThehQwb0zRriyKcaxA==", 2024-09-21T07:34:01.482 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:01.482 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:34:01.482 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:34:01.482 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:34:01.482 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:01.482 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:01.482 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:01.482 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.2", 2024-09-21T07:34:01.482 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCCdu5mTfWxDhAAadJCbPN5xBx/f9L0v2yTdg==", 2024-09-21T07:34:01.482 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:01.482 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:34:01.482 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:34:01.482 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:34:01.483 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:01.483 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:01.483 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:01.483 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.3", 2024-09-21T07:34:01.483 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCFdu5mL1E2HxAAyOhY97uUXEw5L1CcDeONVA==", 2024-09-21T07:34:01.483 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:01.483 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:34:01.483 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:34:01.483 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:34:01.483 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:01.483 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:01.483 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:01.483 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.4", 2024-09-21T07:34:01.484 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCLdu5m6PY/IhAAgvcTgm+6uWA7lIG4tcTIEg==", 2024-09-21T07:34:01.484 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:01.484 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:34:01.484 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:34:01.484 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:34:01.484 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:01.484 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:01.484 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:01.484 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.5", 2024-09-21T07:34:01.484 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCOdu5mE/wlMxAA3JBg4xH+N/PPC1Uv2ZVrYQ==", 2024-09-21T07:34:01.484 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:01.485 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:34:01.485 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:34:01.485 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:34:01.485 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:01.485 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:01.485 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:01.485 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.6", 2024-09-21T07:34:01.485 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCSdu5mC3TgBxAAwsArjeIdFcApY3/1OJffEg==", 2024-09-21T07:34:01.485 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:01.485 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:34:01.485 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:34:01.485 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:34:01.485 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:01.485 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:01.486 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:01.486 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.7", 2024-09-21T07:34:01.486 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCVdu5mSWHWFBAA9qPHWRdsvrB74inrp1lj4Q==", 2024-09-21T07:34:01.486 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:01.486 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:34:01.486 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:34:01.486 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:34:01.486 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:01.486 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:01.486 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:01.486 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.0", 2024-09-21T07:34:01.486 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mGfb1DBAA/aSMKR5or3xy53rB/iUjpA==", 2024-09-21T07:34:01.486 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:01.486 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:34:01.487 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow r", 2024-09-21T07:34:01.487 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow rw", 2024-09-21T07:34:01.487 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rwx" 2024-09-21T07:34:01.487 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:01.487 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:01.487 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:01.487 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.1", 2024-09-21T07:34:01.487 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mEkciIhAAid+bxJEr4P7vMnZ42PAy5w==", 2024-09-21T07:34:01.487 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:01.487 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:34:01.487 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow r", 2024-09-21T07:34:01.487 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow rw", 2024-09-21T07:34:01.487 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rwx" 2024-09-21T07:34:01.487 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:01.488 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:01.488 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:01.488 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.2", 2024-09-21T07:34:01.488 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mSu2oFxAAgjy2m/wWKh6xaoh+YDoU1Q==", 2024-09-21T07:34:01.488 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:01.488 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:34:01.488 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow r", 2024-09-21T07:34:01.488 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow rw", 2024-09-21T07:34:01.488 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rwx" 2024-09-21T07:34:01.488 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:01.488 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:01.488 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:01.488 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.3", 2024-09-21T07:34:01.488 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mCHy0LBAAZneF2jUWgfuyGehZFbOxDg==", 2024-09-21T07:34:01.488 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:01.489 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:34:01.489 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow r", 2024-09-21T07:34:01.489 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow rw", 2024-09-21T07:34:01.489 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rwx" 2024-09-21T07:34:01.489 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:01.489 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:01.489 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:01.489 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.admin", 2024-09-21T07:34:01.489 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBydu5m9LInERAAAlDpqb94f2Qb9VYJJwTBcA==", 2024-09-21T07:34:01.489 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:01.489 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow *", 2024-09-21T07:34:01.489 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:34:01.489 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:34:01.489 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:34:01.490 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:01.490 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:01.490 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:01.490 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.bootstrap-mds", 2024-09-21T07:34:01.490 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCedu5mZiRnFRAAh2ULjjKJxB8qvQeqivYxkA==", 2024-09-21T07:34:01.490 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:01.490 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile bootstrap-mds" 2024-09-21T07:34:01.490 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:01.490 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:01.490 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:01.490 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.bootstrap-mgr", 2024-09-21T07:34:01.490 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCedu5mIjhnFRAAUVf/iDS0KcxiF1DuMz8XsQ==", 2024-09-21T07:34:01.490 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:01.490 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-21T07:34:01.490 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:01.491 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:01.491 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:01.491 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.bootstrap-osd", 2024-09-21T07:34:01.491 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCedu5mNEpnFRAAyraLQk3z2nQNmqjFKOAy0w==", 2024-09-21T07:34:01.491 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:01.491 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile bootstrap-osd" 2024-09-21T07:34:01.491 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:01.491 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:01.491 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:01.491 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.bootstrap-rbd", 2024-09-21T07:34:01.491 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCedu5m8ltnFRAATQH0C5sbB8uG88aJSkEZPw==", 2024-09-21T07:34:01.491 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:01.491 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-21T07:34:01.491 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:01.492 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:01.492 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:01.492 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-21T07:34:01.492 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCedu5mXG5nFRAAn1u839tatNEqwbHCVUDKPg==", 2024-09-21T07:34:01.492 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:01.492 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-21T07:34:01.492 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:01.492 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:01.492 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:01.492 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.bootstrap-rgw", 2024-09-21T07:34:01.492 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCedu5mY4BnFRAAz2Rk1YvPtMFidEme/XhuJQ==", 2024-09-21T07:34:01.492 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:01.492 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-21T07:34:01.492 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:01.492 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:01.493 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:01.493 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mgr.x", 2024-09-21T07:34:01.493 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5mA2GRBhAAkyCKdgH6laHy8xNt080dXg==", 2024-09-21T07:34:01.493 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:01.493 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow *", 2024-09-21T07:34:01.493 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile mgr", 2024-09-21T07:34:01.493 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:34:01.493 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:01.493 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:01.493 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:01.493 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mgr.y", 2024-09-21T07:34:01.493 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBydu5mr+8aNxAA7TFYNms2WsXUfa4ITTJOHw==", 2024-09-21T07:34:01.493 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:01.493 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow *", 2024-09-21T07:34:01.494 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile mgr", 2024-09-21T07:34:01.494 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:34:01.494 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:01.494 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:01.494 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-21T07:34:01.494 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:34:01.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.479+0000 7efc17e00640 1 -- 172.21.15.39:0/2136883276 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efbf4078bc0 msgr2=0x7efbf407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:01.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.479+0000 7efc17e00640 1 --2- 172.21.15.39:0/2136883276 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efbf4078bc0 0x7efbf407b0d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efc000095f0 tx=0x7efc00005f90 comp rx=0 tx=0).stop 2024-09-21T07:34:01.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.479+0000 7efc17e00640 1 -- 172.21.15.39:0/2136883276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc180735d0 msgr2=0x7efc1813e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:01.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.479+0000 7efc17e00640 1 --2- 172.21.15.39:0/2136883276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc180735d0 0x7efc1813e480 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7efc08009950 tx=0x7efc08033c20 comp rx=0 tx=0).stop 2024-09-21T07:34:01.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.480+0000 7efc17e00640 1 -- 172.21.15.39:0/2136883276 shutdown_connections 2024-09-21T07:34:01.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.480+0000 7efc17e00640 1 --2- 172.21.15.39:0/2136883276 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efbf4078bc0 0x7efbf407b0d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:01.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.480+0000 7efc17e00640 1 --2- 172.21.15.39:0/2136883276 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc180b54d0 0x7efc1815f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:01.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.480+0000 7efc17e00640 1 --2- 172.21.15.39:0/2136883276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc180af700 0x7efc1815f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:01.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.480+0000 7efc17e00640 1 --2- 172.21.15.39:0/2136883276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc180735d0 0x7efc1813e480 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:01.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.480+0000 7efc17e00640 1 -- 172.21.15.39:0/2136883276 >> 172.21.15.39:0/2136883276 conn(0x7efc1806d870 msgr2=0x7efc180ce3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:01.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.480+0000 7efc17e00640 1 -- 172.21.15.39:0/2136883276 shutdown_connections 2024-09-21T07:34:01.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.480+0000 7efc17e00640 1 -- 172.21.15.39:0/2136883276 wait complete. 2024-09-21T07:34:01.503 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-21T07:34:01.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.732+0000 7fd77fe00640 1 -- 172.21.15.39:0/41533991 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd760005280 msgr2=0x7fd76000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:01.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.732+0000 7fd77fe00640 1 --2- 172.21.15.39:0/41533991 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd760005280 0x7fd76000b9c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd770001f30 tx=0x7fd760005830 comp rx=0 tx=0).stop 2024-09-21T07:34:01.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.733+0000 7fd77fe00640 1 -- 172.21.15.39:0/41533991 shutdown_connections 2024-09-21T07:34:01.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.733+0000 7fd77fe00640 1 --2- 172.21.15.39:0/41533991 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd760005280 0x7fd76000b9c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:01.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.733+0000 7fd77fe00640 1 -- 172.21.15.39:0/41533991 >> 172.21.15.39:0/41533991 conn(0x7fd77806d7d0 msgr2=0x7fd7780b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:01.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.733+0000 7fd77fe00640 1 -- 172.21.15.39:0/41533991 shutdown_connections 2024-09-21T07:34:01.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.733+0000 7fd77fe00640 1 -- 172.21.15.39:0/41533991 wait complete. 2024-09-21T07:34:01.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.735+0000 7fd77fe00640 1 Processor -- start 2024-09-21T07:34:01.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.735+0000 7fd77fe00640 1 -- start start 2024-09-21T07:34:01.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.737+0000 7fd77fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd760005280 0x7fd7780aeab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:01.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.737+0000 7fd77fe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7781f8ae0 0x7fd7780ad640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:01.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.737+0000 7fd77fe00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd778072e30 0x7fd7781f5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:01.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.737+0000 7fd77fe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd77815dc30 con 0x7fd760005280 2024-09-21T07:34:01.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.737+0000 7fd77fe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd77815dab0 con 0x7fd7781f8ae0 2024-09-21T07:34:01.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.737+0000 7fd77fe00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd77815ddb0 con 0x7fd778072e30 2024-09-21T07:34:01.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.737+0000 7fd77d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd760005280 0x7fd7780aeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:01.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.737+0000 7fd77ca00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7781f8ae0 0x7fd7780ad640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:01.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.737+0000 7fd77ca00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7781f8ae0 0x7fd7780ad640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.39:47622/0 (socket says 172.21.15.39:47622) 2024-09-21T07:34:01.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.737+0000 7fd77ca00640 1 -- 172.21.15.39:0/301485162 learned_addr learned my addr 172.21.15.39:0/301485162 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:01.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.737+0000 7fd77de00640 1 --2- 172.21.15.39:0/301485162 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd778072e30 0x7fd7781f5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:01.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.738+0000 7fd77d400640 1 -- 172.21.15.39:0/301485162 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd778072e30 msgr2=0x7fd7781f5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:01.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.738+0000 7fd77d400640 1 --2- 172.21.15.39:0/301485162 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd778072e30 0x7fd7781f5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:01.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.738+0000 7fd77d400640 1 -- 172.21.15.39:0/301485162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7781f8ae0 msgr2=0x7fd7780ad640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:01.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.738+0000 7fd77d400640 1 --2- 172.21.15.39:0/301485162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7781f8ae0 0x7fd7780ad640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:01.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.738+0000 7fd77d400640 1 -- 172.21.15.39:0/301485162 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7781f92c0 con 0x7fd760005280 2024-09-21T07:34:01.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.738+0000 7fd77d400640 1 --2- 172.21.15.39:0/301485162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd760005280 0x7fd7780aeab0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd76c006600 tx=0x7fd76c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:01.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.738+0000 7fd775600640 1 -- 172.21.15.39:0/301485162 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76c015070 con 0x7fd760005280 2024-09-21T07:34:01.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.738+0000 7fd775600640 1 -- 172.21.15.39:0/301485162 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd76c011840 con 0x7fd760005280 2024-09-21T07:34:01.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.738+0000 7fd775600640 1 -- 172.21.15.39:0/301485162 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76c011c50 con 0x7fd760005280 2024-09-21T07:34:01.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.738+0000 7fd77fe00640 1 -- 172.21.15.39:0/301485162 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7781f95a0 con 0x7fd760005280 2024-09-21T07:34:01.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.738+0000 7fd77fe00640 1 -- 172.21.15.39:0/301485162 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7781f5820 con 0x7fd760005280 2024-09-21T07:34:01.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.740+0000 7fd77fe00640 1 -- 172.21.15.39:0/301485162 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7780cd250 con 0x7fd760005280 2024-09-21T07:34:01.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.741+0000 7fd775600640 1 -- 172.21.15.39:0/301485162 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd76c018070 con 0x7fd760005280 2024-09-21T07:34:01.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.742+0000 7fd775600640 1 --2- 172.21.15.39:0/301485162 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd758078bc0 0x7fd75807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:01.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.742+0000 7fd775600640 1 -- 172.21.15.39:0/301485162 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7fd76c09b620 con 0x7fd760005280 2024-09-21T07:34:01.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.742+0000 7fd77ca00640 1 --2- 172.21.15.39:0/301485162 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd758078bc0 0x7fd75807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:01.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.743+0000 7fd77ca00640 1 --2- 172.21.15.39:0/301485162 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd758078bc0 0x7fd75807b0d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd77815a8e0 tx=0x7fd764005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:01.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:01.746+0000 7fd775600640 1 -- 172.21.15.39:0/301485162 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd76c063770 con 0x7fd760005280 2024-09-21T07:34:02.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.022+0000 7fd77fe00640 1 -- 172.21.15.39:0/301485162 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fd7780d0d50 con 0x7fd760005280 2024-09-21T07:34:02.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.025+0000 7fd775600640 1 -- 172.21.15.39:0/301485162 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) ==== 90+0+0 (secure 0 0 0) 0x7fd76c0681a0 con 0x7fd760005280 2024-09-21T07:34:02.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.028+0000 7fd77fe00640 1 -- 172.21.15.39:0/301485162 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd758078bc0 msgr2=0x7fd75807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:02.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.028+0000 7fd77fe00640 1 --2- 172.21.15.39:0/301485162 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd758078bc0 0x7fd75807b0d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd77815a8e0 tx=0x7fd764005f90 comp rx=0 tx=0).stop 2024-09-21T07:34:02.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.028+0000 7fd77fe00640 1 -- 172.21.15.39:0/301485162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd760005280 msgr2=0x7fd7780aeab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:02.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.028+0000 7fd77fe00640 1 --2- 172.21.15.39:0/301485162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd760005280 0x7fd7780aeab0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd76c006600 tx=0x7fd76c0039c0 comp rx=0 tx=0).stop 2024-09-21T07:34:02.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.028+0000 7fd77fe00640 1 -- 172.21.15.39:0/301485162 shutdown_connections 2024-09-21T07:34:02.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.028+0000 7fd77fe00640 1 --2- 172.21.15.39:0/301485162 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd758078bc0 0x7fd75807b0d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:02.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.028+0000 7fd77fe00640 1 --2- 172.21.15.39:0/301485162 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd778072e30 0x7fd7781f5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:02.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.028+0000 7fd77fe00640 1 --2- 172.21.15.39:0/301485162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7781f8ae0 0x7fd7780ad640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:02.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.028+0000 7fd77fe00640 1 --2- 172.21.15.39:0/301485162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd760005280 0x7fd7780aeab0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:02.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.028+0000 7fd77fe00640 1 -- 172.21.15.39:0/301485162 >> 172.21.15.39:0/301485162 conn(0x7fd77806d7d0 msgr2=0x7fd7780b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:02.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.028+0000 7fd77fe00640 1 -- 172.21.15.39:0/301485162 shutdown_connections 2024-09-21T07:34:02.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.028+0000 7fd77fe00640 1 -- 172.21.15.39:0/301485162 wait complete. 2024-09-21T07:34:02.052 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-21T07:34:02.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.280+0000 7f796b800640 1 -- 172.21.15.39:0/3758931978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7964070030 msgr2=0x7f7964074640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:02.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.280+0000 7f796b800640 1 --2- 172.21.15.39:0/3758931978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7964070030 0x7f7964074640 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7950005fd0 tx=0x7f795002a670 comp rx=0 tx=0).stop 2024-09-21T07:34:02.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.280+0000 7f796b800640 1 -- 172.21.15.39:0/3758931978 shutdown_connections 2024-09-21T07:34:02.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.280+0000 7f796b800640 1 --2- 172.21.15.39:0/3758931978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7964070030 0x7f7964074640 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:02.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.280+0000 7f796b800640 1 -- 172.21.15.39:0/3758931978 >> 172.21.15.39:0/3758931978 conn(0x7f79640b1cb0 msgr2=0x7f79640b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:02.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.280+0000 7f796b800640 1 -- 172.21.15.39:0/3758931978 shutdown_connections 2024-09-21T07:34:02.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.280+0000 7f796b800640 1 -- 172.21.15.39:0/3758931978 wait complete. 2024-09-21T07:34:02.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.282+0000 7f796b800640 1 Processor -- start 2024-09-21T07:34:02.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.282+0000 7f796b800640 1 -- start start 2024-09-21T07:34:02.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.284+0000 7f796b800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7964070030 0x7f796409aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:02.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.284+0000 7f796b800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7964070440 0x7f796415ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:02.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.284+0000 7f796b800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7964073170 0x7f79641e94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:02.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.284+0000 7f796b800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f79640b3c40 con 0x7f7964070030 2024-09-21T07:34:02.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.284+0000 7f796b800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f79640b3ac0 con 0x7f7964070440 2024-09-21T07:34:02.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.284+0000 7f796b800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f79640b3dc0 con 0x7f7964073170 2024-09-21T07:34:02.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.284+0000 7f7968e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7964070030 0x7f796409aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:02.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.284+0000 7f7968e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7964070030 0x7f796409aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46200/0 (socket says 172.21.15.39:46200) 2024-09-21T07:34:02.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.284+0000 7f7968e00640 1 -- 172.21.15.39:0/2221024240 learned_addr learned my addr 172.21.15.39:0/2221024240 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:02.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.284+0000 7f7963e00640 1 --2- 172.21.15.39:0/2221024240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7964070440 0x7f796415ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:02.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.284+0000 7f7969800640 1 --2- 172.21.15.39:0/2221024240 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7964073170 0x7f79641e94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:02.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.285+0000 7f7968e00640 1 -- 172.21.15.39:0/2221024240 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7964073170 msgr2=0x7f79641e94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:02.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.285+0000 7f7968e00640 1 --2- 172.21.15.39:0/2221024240 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7964073170 0x7f79641e94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:02.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.285+0000 7f7968e00640 1 -- 172.21.15.39:0/2221024240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7964070440 msgr2=0x7f796415ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:02.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.285+0000 7f7968e00640 1 --2- 172.21.15.39:0/2221024240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7964070440 0x7f796415ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:02.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.285+0000 7f7968e00640 1 -- 172.21.15.39:0/2221024240 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79641f9e20 con 0x7f7964070030 2024-09-21T07:34:02.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.285+0000 7f7969800640 1 --2- 172.21.15.39:0/2221024240 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7964073170 0x7f79641e94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:34:02.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.285+0000 7f7968e00640 1 --2- 172.21.15.39:0/2221024240 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7964070030 0x7f796409aa30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f795800c430 tx=0x7f795800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:02.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.285+0000 7f7960c00640 1 -- 172.21.15.39:0/2221024240 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7958017070 con 0x7f7964070030 2024-09-21T07:34:02.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.285+0000 7f7960c00640 1 -- 172.21.15.39:0/2221024240 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f79580053c0 con 0x7f7964070030 2024-09-21T07:34:02.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.285+0000 7f796b800640 1 -- 172.21.15.39:0/2221024240 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79641f92a0 con 0x7f7964070030 2024-09-21T07:34:02.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.285+0000 7f7960c00640 1 -- 172.21.15.39:0/2221024240 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7958003850 con 0x7f7964070030 2024-09-21T07:34:02.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.285+0000 7f796b800640 1 -- 172.21.15.39:0/2221024240 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79641f9860 con 0x7f7964070030 2024-09-21T07:34:02.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.286+0000 7f796b800640 1 -- 172.21.15.39:0/2221024240 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79640b42e0 con 0x7f7964070030 2024-09-21T07:34:02.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.288+0000 7f7960c00640 1 -- 172.21.15.39:0/2221024240 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f795801a070 con 0x7f7964070030 2024-09-21T07:34:02.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.289+0000 7f7960c00640 1 --2- 172.21.15.39:0/2221024240 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7944078bc0 0x7f794407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:02.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.290+0000 7f7960c00640 1 -- 172.21.15.39:0/2221024240 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f7958099cd0 con 0x7f7964070030 2024-09-21T07:34:02.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.290+0000 7f7963e00640 1 --2- 172.21.15.39:0/2221024240 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7944078bc0 0x7f794407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:02.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.290+0000 7f7963e00640 1 --2- 172.21.15.39:0/2221024240 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7944078bc0 0x7f794407b0d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7950033000 tx=0x7f7950004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:02.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.294+0000 7f7960c00640 1 -- 172.21.15.39:0/2221024240 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7958061e00 con 0x7f7964070030 2024-09-21T07:34:02.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.563+0000 7f796b800640 1 -- 172.21.15.39:0/2221024240 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f79640c3860 con 0x7f7964070030 2024-09-21T07:34:02.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.566+0000 7f7960c00640 1 -- 172.21.15.39:0/2221024240 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) ==== 90+0+0 (secure 0 0 0) 0x7f7958066830 con 0x7f7964070030 2024-09-21T07:34:02.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.568+0000 7f796b800640 1 -- 172.21.15.39:0/2221024240 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7944078bc0 msgr2=0x7f794407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:02.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.568+0000 7f796b800640 1 --2- 172.21.15.39:0/2221024240 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7944078bc0 0x7f794407b0d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7950033000 tx=0x7f7950004730 comp rx=0 tx=0).stop 2024-09-21T07:34:02.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.568+0000 7f796b800640 1 -- 172.21.15.39:0/2221024240 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7964070030 msgr2=0x7f796409aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:02.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.568+0000 7f796b800640 1 --2- 172.21.15.39:0/2221024240 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7964070030 0x7f796409aa30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f795800c430 tx=0x7f795800c8f0 comp rx=0 tx=0).stop 2024-09-21T07:34:02.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.569+0000 7f796b800640 1 -- 172.21.15.39:0/2221024240 shutdown_connections 2024-09-21T07:34:02.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.569+0000 7f796b800640 1 --2- 172.21.15.39:0/2221024240 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7944078bc0 0x7f794407b0d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:02.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.569+0000 7f796b800640 1 --2- 172.21.15.39:0/2221024240 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7964073170 0x7f79641e94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:02.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.569+0000 7f796b800640 1 --2- 172.21.15.39:0/2221024240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7964070440 0x7f796415ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:02.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.569+0000 7f796b800640 1 --2- 172.21.15.39:0/2221024240 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7964070030 0x7f796409aa30 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:02.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.569+0000 7f796b800640 1 -- 172.21.15.39:0/2221024240 >> 172.21.15.39:0/2221024240 conn(0x7f79640b1cb0 msgr2=0x7f79640ae280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:02.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.569+0000 7f796b800640 1 -- 172.21.15.39:0/2221024240 shutdown_connections 2024-09-21T07:34:02.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.569+0000 7f796b800640 1 -- 172.21.15.39:0/2221024240 wait complete. 2024-09-21T07:34:02.591 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-21T07:34:02.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.818+0000 7fb745000640 1 -- 172.21.15.39:0/1113722590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb740070030 msgr2=0x7fb740074640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:02.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.818+0000 7fb745000640 1 --2- 172.21.15.39:0/1113722590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb740070030 0x7fb740074640 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb728009830 tx=0x7fb72802a5c0 comp rx=0 tx=0).stop 2024-09-21T07:34:02.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.818+0000 7fb745000640 1 -- 172.21.15.39:0/1113722590 shutdown_connections 2024-09-21T07:34:02.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.818+0000 7fb745000640 1 --2- 172.21.15.39:0/1113722590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb740070030 0x7fb740074640 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:02.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.818+0000 7fb745000640 1 -- 172.21.15.39:0/1113722590 >> 172.21.15.39:0/1113722590 conn(0x7fb7400b1cb0 msgr2=0x7fb7400b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:02.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.819+0000 7fb745000640 1 -- 172.21.15.39:0/1113722590 shutdown_connections 2024-09-21T07:34:02.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.819+0000 7fb745000640 1 -- 172.21.15.39:0/1113722590 wait complete. 2024-09-21T07:34:02.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.821+0000 7fb745000640 1 Processor -- start 2024-09-21T07:34:02.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.821+0000 7fb745000640 1 -- start start 2024-09-21T07:34:02.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.822+0000 7fb745000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb740070030 0x7fb740149390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:02.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.822+0000 7fb745000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb740070440 0x7fb7400cbb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:02.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.822+0000 7fb745000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb740073170 0x7fb7401e94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:02.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.822+0000 7fb745000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb7400b3c40 con 0x7fb740070440 2024-09-21T07:34:02.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.822+0000 7fb745000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb7400b3ac0 con 0x7fb740073170 2024-09-21T07:34:02.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.822+0000 7fb745000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb7400b3dc0 con 0x7fb740070030 2024-09-21T07:34:02.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.823+0000 7fb737e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb740070440 0x7fb7400cbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:02.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.823+0000 7fb737e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb740070440 0x7fb7400cbb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46220/0 (socket says 172.21.15.39:46220) 2024-09-21T07:34:02.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.823+0000 7fb737e00640 1 -- 172.21.15.39:0/4169610625 learned_addr learned my addr 172.21.15.39:0/4169610625 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:02.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.823+0000 7fb73e800640 1 --2- 172.21.15.39:0/4169610625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb740073170 0x7fb7401e94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:02.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.823+0000 7fb73de00640 1 --2- 172.21.15.39:0/4169610625 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb740070030 0x7fb740149390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:02.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.823+0000 7fb737e00640 1 -- 172.21.15.39:0/4169610625 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb740070030 msgr2=0x7fb740149390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:02.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.823+0000 7fb737e00640 1 --2- 172.21.15.39:0/4169610625 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb740070030 0x7fb740149390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:02.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.823+0000 7fb737e00640 1 -- 172.21.15.39:0/4169610625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb740073170 msgr2=0x7fb7401e94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:02.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.823+0000 7fb737e00640 1 --2- 172.21.15.39:0/4169610625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb740073170 0x7fb7401e94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:02.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.823+0000 7fb737e00640 1 -- 172.21.15.39:0/4169610625 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7400cb6d0 con 0x7fb740070440 2024-09-21T07:34:02.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.823+0000 7fb737e00640 1 --2- 172.21.15.39:0/4169610625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb740070440 0x7fb7400cbb00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb728009950 tx=0x7fb728033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:02.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.824+0000 7fb736000640 1 -- 172.21.15.39:0/4169610625 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb728005c90 con 0x7fb740070440 2024-09-21T07:34:02.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.824+0000 7fb736000640 1 -- 172.21.15.39:0/4169610625 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb728031e50 con 0x7fb740070440 2024-09-21T07:34:02.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.824+0000 7fb745000640 1 -- 172.21.15.39:0/4169610625 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7401f6970 con 0x7fb740070440 2024-09-21T07:34:02.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.824+0000 7fb736000640 1 -- 172.21.15.39:0/4169610625 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb728005260 con 0x7fb740070440 2024-09-21T07:34:02.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.824+0000 7fb745000640 1 -- 172.21.15.39:0/4169610625 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7400cabf0 con 0x7fb740070440 2024-09-21T07:34:02.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.825+0000 7fb745000640 1 -- 172.21.15.39:0/4169610625 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7400cc5a0 con 0x7fb740070440 2024-09-21T07:34:02.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.826+0000 7fb736000640 1 -- 172.21.15.39:0/4169610625 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb728005400 con 0x7fb740070440 2024-09-21T07:34:02.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.827+0000 7fb736000640 1 --2- 172.21.15.39:0/4169610625 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb71c078bc0 0x7fb71c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:02.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.827+0000 7fb736000640 1 -- 172.21.15.39:0/4169610625 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7fb7280bbaa0 con 0x7fb740070440 2024-09-21T07:34:02.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.828+0000 7fb73de00640 1 --2- 172.21.15.39:0/4169610625 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb71c078bc0 0x7fb71c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:02.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.828+0000 7fb73de00640 1 --2- 172.21.15.39:0/4169610625 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb71c078bc0 0x7fb71c07b0d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb730001610 tx=0x7fb730009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:02.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:02.831+0000 7fb736000640 1 -- 172.21.15.39:0/4169610625 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb72803f030 con 0x7fb740070440 2024-09-21T07:34:03.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.100+0000 7fb745000640 1 -- 172.21.15.39:0/4169610625 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fb7400b42e0 con 0x7fb740070440 2024-09-21T07:34:03.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.107+0000 7fb736000640 1 -- 172.21.15.39:0/4169610625 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7fb7280842c0 con 0x7fb740070440 2024-09-21T07:34:03.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.110+0000 7fb745000640 1 -- 172.21.15.39:0/4169610625 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb71c078bc0 msgr2=0x7fb71c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:03.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.110+0000 7fb745000640 1 --2- 172.21.15.39:0/4169610625 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb71c078bc0 0x7fb71c07b0d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb730001610 tx=0x7fb730009040 comp rx=0 tx=0).stop 2024-09-21T07:34:03.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.110+0000 7fb745000640 1 -- 172.21.15.39:0/4169610625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb740070440 msgr2=0x7fb7400cbb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:03.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.110+0000 7fb745000640 1 --2- 172.21.15.39:0/4169610625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb740070440 0x7fb7400cbb00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb728009950 tx=0x7fb728033c20 comp rx=0 tx=0).stop 2024-09-21T07:34:03.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.110+0000 7fb745000640 1 -- 172.21.15.39:0/4169610625 shutdown_connections 2024-09-21T07:34:03.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.110+0000 7fb745000640 1 --2- 172.21.15.39:0/4169610625 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb71c078bc0 0x7fb71c07b0d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:03.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.110+0000 7fb745000640 1 --2- 172.21.15.39:0/4169610625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb740073170 0x7fb7401e94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:03.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.110+0000 7fb745000640 1 --2- 172.21.15.39:0/4169610625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb740070440 0x7fb7400cbb00 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:03.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.110+0000 7fb745000640 1 --2- 172.21.15.39:0/4169610625 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb740070030 0x7fb740149390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:03.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.110+0000 7fb745000640 1 -- 172.21.15.39:0/4169610625 >> 172.21.15.39:0/4169610625 conn(0x7fb7400b1cb0 msgr2=0x7fb74015f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:03.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.110+0000 7fb745000640 1 -- 172.21.15.39:0/4169610625 shutdown_connections 2024-09-21T07:34:03.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.110+0000 7fb745000640 1 -- 172.21.15.39:0/4169610625 wait complete. 2024-09-21T07:34:03.135 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-21T07:34:03.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.362+0000 7f0bff200640 1 -- 172.21.15.39:0/2050006885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0be4003a30 msgr2=0x7f0be4003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:03.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.362+0000 7f0bff200640 1 --2- 172.21.15.39:0/2050006885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0be4003a30 0x7f0be4003ea0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0bf00016c0 tx=0x7f0be4005830 comp rx=0 tx=0).stop 2024-09-21T07:34:03.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.362+0000 7f0bfe800640 1 -- 172.21.15.39:0/2050006885 reap_dead start 2024-09-21T07:34:03.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.363+0000 7f0bff200640 1 -- 172.21.15.39:0/2050006885 shutdown_connections 2024-09-21T07:34:03.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.363+0000 7f0bff200640 1 -- 172.21.15.39:0/2050006885 >> 172.21.15.39:0/2050006885 conn(0x7f0c0006d410 msgr2=0x7f0c000b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:03.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.363+0000 7f0bff200640 1 -- 172.21.15.39:0/2050006885 shutdown_connections 2024-09-21T07:34:03.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.363+0000 7f0bff200640 1 -- 172.21.15.39:0/2050006885 wait complete. 2024-09-21T07:34:03.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.365+0000 7f0bff200640 1 Processor -- start 2024-09-21T07:34:03.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.365+0000 7f0bff200640 1 -- start start 2024-09-21T07:34:03.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.366+0000 7f0bff200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0be4003a30 0x7f0c00159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:03.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.367+0000 7f0bff200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c00140b10 0x7f0c00152ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:03.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.367+0000 7f0bff200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c001e8ec0 0x7f0c001e92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:03.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.367+0000 7f0bff200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0006ee70 con 0x7f0c00140b10 2024-09-21T07:34:03.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.367+0000 7f0bff200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0015e500 con 0x7f0be4003a30 2024-09-21T07:34:03.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.367+0000 7f0bff200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f0c0006eff0 con 0x7f0c001e8ec0 2024-09-21T07:34:03.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.367+0000 7f0bf7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c00140b10 0x7f0c00152ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:03.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.367+0000 7f0bf7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c00140b10 0x7f0c00152ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46230/0 (socket says 172.21.15.39:46230) 2024-09-21T07:34:03.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.367+0000 7f0bf7e00640 1 -- 172.21.15.39:0/3335523523 learned_addr learned my addr 172.21.15.39:0/3335523523 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:03.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.367+0000 7f0bfde00640 1 --2- 172.21.15.39:0/3335523523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0be4003a30 0x7f0c00159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:03.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.367+0000 7f0bfe800640 1 --2- 172.21.15.39:0/3335523523 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c001e8ec0 0x7f0c001e92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:03.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.367+0000 7f0bf7e00640 1 -- 172.21.15.39:0/3335523523 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c001e8ec0 msgr2=0x7f0c001e92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:03.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.367+0000 7f0bf7e00640 1 --2- 172.21.15.39:0/3335523523 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c001e8ec0 0x7f0c001e92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:03.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.367+0000 7f0bf7e00640 1 -- 172.21.15.39:0/3335523523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0be4003a30 msgr2=0x7f0c00159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:03.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.367+0000 7f0bf7e00640 1 --2- 172.21.15.39:0/3335523523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0be4003a30 0x7f0c00159fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:03.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.367+0000 7f0bf7e00640 1 -- 172.21.15.39:0/3335523523 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c001e9810 con 0x7f0c00140b10 2024-09-21T07:34:03.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.368+0000 7f0bf7e00640 1 --2- 172.21.15.39:0/3335523523 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c00140b10 0x7f0c00152ae0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0be80118b0 tx=0x7f0be8011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:03.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.368+0000 7f0bf6000640 1 -- 172.21.15.39:0/3335523523 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be800f9a0 con 0x7f0c00140b10 2024-09-21T07:34:03.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.368+0000 7f0bf6000640 1 -- 172.21.15.39:0/3335523523 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0be800fb40 con 0x7f0c00140b10 2024-09-21T07:34:03.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.368+0000 7f0bf6000640 1 -- 172.21.15.39:0/3335523523 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be8005020 con 0x7f0c00140b10 2024-09-21T07:34:03.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.368+0000 7f0bff200640 1 -- 172.21.15.39:0/3335523523 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c001f56b0 con 0x7f0c00140b10 2024-09-21T07:34:03.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.368+0000 7f0bff200640 1 -- 172.21.15.39:0/3335523523 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c00153020 con 0x7f0c00140b10 2024-09-21T07:34:03.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.369+0000 7f0bff200640 1 -- 172.21.15.39:0/3335523523 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c0006f580 con 0x7f0c00140b10 2024-09-21T07:34:03.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.370+0000 7f0bf6000640 1 -- 172.21.15.39:0/3335523523 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0be80051c0 con 0x7f0c00140b10 2024-09-21T07:34:03.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.372+0000 7f0bf6000640 1 --2- 172.21.15.39:0/3335523523 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0bdc078bc0 0x7f0bdc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:03.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.372+0000 7f0bf6000640 1 -- 172.21.15.39:0/3335523523 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f0be809b5c0 con 0x7f0c00140b10 2024-09-21T07:34:03.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.372+0000 7f0bfde00640 1 --2- 172.21.15.39:0/3335523523 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0bdc078bc0 0x7f0bdc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:03.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.373+0000 7f0bfde00640 1 --2- 172.21.15.39:0/3335523523 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0bdc078bc0 0x7f0bdc07b0d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0032000 tx=0x7f0bf0001190 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:03.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.377+0000 7f0bf6000640 1 -- 172.21.15.39:0/3335523523 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0c001e9810 con 0x7f0c00140b10 2024-09-21T07:34:03.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.652+0000 7f0bff200640 1 -- 172.21.15.39:0/3335523523 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f0c0015f400 con 0x7f0c00140b10 2024-09-21T07:34:03.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.655+0000 7f0bf6000640 1 -- 172.21.15.39:0/3335523523 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f0be8063de0 con 0x7f0c00140b10 2024-09-21T07:34:03.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.658+0000 7f0bff200640 1 -- 172.21.15.39:0/3335523523 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0bdc078bc0 msgr2=0x7f0bdc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:03.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.658+0000 7f0bff200640 1 --2- 172.21.15.39:0/3335523523 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0bdc078bc0 0x7f0bdc07b0d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0032000 tx=0x7f0bf0001190 comp rx=0 tx=0).stop 2024-09-21T07:34:03.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.658+0000 7f0bff200640 1 -- 172.21.15.39:0/3335523523 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c00140b10 msgr2=0x7f0c00152ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:03.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.658+0000 7f0bff200640 1 --2- 172.21.15.39:0/3335523523 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c00140b10 0x7f0c00152ae0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0be80118b0 tx=0x7f0be8011d70 comp rx=0 tx=0).stop 2024-09-21T07:34:03.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.658+0000 7f0bff200640 1 -- 172.21.15.39:0/3335523523 shutdown_connections 2024-09-21T07:34:03.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.658+0000 7f0bff200640 1 --2- 172.21.15.39:0/3335523523 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0bdc078bc0 0x7f0bdc07b0d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:03.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.658+0000 7f0bff200640 1 --2- 172.21.15.39:0/3335523523 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c001e8ec0 0x7f0c001e92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:03.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.658+0000 7f0bff200640 1 --2- 172.21.15.39:0/3335523523 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c00140b10 0x7f0c00152ae0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:03.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.658+0000 7f0bff200640 1 --2- 172.21.15.39:0/3335523523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0be4003a30 0x7f0c00159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:03.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.658+0000 7f0bff200640 1 -- 172.21.15.39:0/3335523523 >> 172.21.15.39:0/3335523523 conn(0x7f0c0006d410 msgr2=0x7f0c00073ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:03.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.658+0000 7f0bff200640 1 -- 172.21.15.39:0/3335523523 shutdown_connections 2024-09-21T07:34:03.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.658+0000 7f0bff200640 1 -- 172.21.15.39:0/3335523523 wait complete. 2024-09-21T07:34:03.680 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-21T07:34:03.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.905+0000 7fdca0600640 1 -- 172.21.15.39:0/572159049 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc980af3c0 msgr2=0x7fdc980b40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:03.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.905+0000 7fdca0600640 1 --2- 172.21.15.39:0/572159049 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc980af3c0 0x7fdc980b40b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdc84009830 tx=0x7fdc8402a520 comp rx=0 tx=0).stop 2024-09-21T07:34:03.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.905+0000 7fdca0600640 1 -- 172.21.15.39:0/572159049 shutdown_connections 2024-09-21T07:34:03.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.905+0000 7fdca0600640 1 --2- 172.21.15.39:0/572159049 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdc980ccb50 0x7fdc980ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:03.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.905+0000 7fdca0600640 1 --2- 172.21.15.39:0/572159049 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc980af3c0 0x7fdc980b40b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:03.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.905+0000 7fdca0600640 1 -- 172.21.15.39:0/572159049 >> 172.21.15.39:0/572159049 conn(0x7fdc9806d7d0 msgr2=0x7fdc980b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:03.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.905+0000 7fdca0600640 1 -- 172.21.15.39:0/572159049 shutdown_connections 2024-09-21T07:34:03.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.905+0000 7fdca0600640 1 -- 172.21.15.39:0/572159049 wait complete. 2024-09-21T07:34:03.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.907+0000 7fdca0600640 1 Processor -- start 2024-09-21T07:34:03.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.907+0000 7fdca0600640 1 -- start start 2024-09-21T07:34:03.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.909+0000 7fdca0600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc980af3c0 0x7fdc9815f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:03.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.909+0000 7fdca0600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc980cc1e0 0x7fdc981e76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:03.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.909+0000 7fdca0600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdc980ccb50 0x7fdc9815e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:03.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.909+0000 7fdca0600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdc98153520 con 0x7fdc980cc1e0 2024-09-21T07:34:03.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.909+0000 7fdca0600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdc981533a0 con 0x7fdc980af3c0 2024-09-21T07:34:03.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.909+0000 7fdca0600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fdc981536a0 con 0x7fdc980ccb50 2024-09-21T07:34:03.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.909+0000 7fdc9d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc980cc1e0 0x7fdc981e76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:03.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.909+0000 7fdc9d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc980cc1e0 0x7fdc981e76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46246/0 (socket says 172.21.15.39:46246) 2024-09-21T07:34:03.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.909+0000 7fdc9d200640 1 -- 172.21.15.39:0/547157918 learned_addr learned my addr 172.21.15.39:0/547157918 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:03.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.909+0000 7fdc9dc00640 1 --2- 172.21.15.39:0/547157918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc980af3c0 0x7fdc9815f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:03.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.909+0000 7fdc9e600640 1 --2- 172.21.15.39:0/547157918 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdc980ccb50 0x7fdc9815e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:03.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.910+0000 7fdc9d200640 1 -- 172.21.15.39:0/547157918 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdc980ccb50 msgr2=0x7fdc9815e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:03.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.910+0000 7fdc9d200640 1 --2- 172.21.15.39:0/547157918 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdc980ccb50 0x7fdc9815e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:03.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.910+0000 7fdc9d200640 1 -- 172.21.15.39:0/547157918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc980af3c0 msgr2=0x7fdc9815f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:03.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.910+0000 7fdc9d200640 1 --2- 172.21.15.39:0/547157918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc980af3c0 0x7fdc9815f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:03.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.910+0000 7fdc9d200640 1 -- 172.21.15.39:0/547157918 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc9813fb70 con 0x7fdc980cc1e0 2024-09-21T07:34:03.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.910+0000 7fdc9d200640 1 --2- 172.21.15.39:0/547157918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc980cc1e0 0x7fdc981e76d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdc8402ab10 tx=0x7fdc84033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:03.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.910+0000 7fdc95600640 1 -- 172.21.15.39:0/547157918 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc84005c90 con 0x7fdc980cc1e0 2024-09-21T07:34:03.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.910+0000 7fdc95600640 1 -- 172.21.15.39:0/547157918 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc84031e50 con 0x7fdc980cc1e0 2024-09-21T07:34:03.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.910+0000 7fdca0600640 1 -- 172.21.15.39:0/547157918 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc9813fe50 con 0x7fdc980cc1e0 2024-09-21T07:34:03.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.910+0000 7fdc95600640 1 -- 172.21.15.39:0/547157918 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc84005260 con 0x7fdc980cc1e0 2024-09-21T07:34:03.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.910+0000 7fdca0600640 1 -- 172.21.15.39:0/547157918 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc98140410 con 0x7fdc980cc1e0 2024-09-21T07:34:03.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.911+0000 7fdca0600640 1 -- 172.21.15.39:0/547157918 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc98153b50 con 0x7fdc980cc1e0 2024-09-21T07:34:03.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.913+0000 7fdc95600640 1 -- 172.21.15.39:0/547157918 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdc84005400 con 0x7fdc980cc1e0 2024-09-21T07:34:03.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.913+0000 7fdc95600640 1 --2- 172.21.15.39:0/547157918 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdc78078bc0 0x7fdc7807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:03.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.914+0000 7fdc95600640 1 -- 172.21.15.39:0/547157918 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7fdc840bbbd0 con 0x7fdc980cc1e0 2024-09-21T07:34:03.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.914+0000 7fdc9dc00640 1 --2- 172.21.15.39:0/547157918 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdc78078bc0 0x7fdc7807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:03.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.915+0000 7fdc9dc00640 1 --2- 172.21.15.39:0/547157918 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdc78078bc0 0x7fdc7807b0d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c0013f0 tx=0x7fdc8c009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:03.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:03.918+0000 7fdc95600640 1 -- 172.21.15.39:0/547157918 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdc8403f030 con 0x7fdc980cc1e0 2024-09-21T07:34:04.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.187+0000 7fdca0600640 1 -- 172.21.15.39:0/547157918 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7fdc98070c30 con 0x7fdc980cc1e0 2024-09-21T07:34:04.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.190+0000 7fdc95600640 1 -- 172.21.15.39:0/547157918 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) ==== 97+0+0 (secure 0 0 0) 0x7fdc840842c0 con 0x7fdc980cc1e0 2024-09-21T07:34:04.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.193+0000 7fdca0600640 1 -- 172.21.15.39:0/547157918 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdc78078bc0 msgr2=0x7fdc7807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:04.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.193+0000 7fdca0600640 1 --2- 172.21.15.39:0/547157918 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdc78078bc0 0x7fdc7807b0d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c0013f0 tx=0x7fdc8c009040 comp rx=0 tx=0).stop 2024-09-21T07:34:04.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.193+0000 7fdca0600640 1 -- 172.21.15.39:0/547157918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc980cc1e0 msgr2=0x7fdc981e76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:04.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.193+0000 7fdca0600640 1 --2- 172.21.15.39:0/547157918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc980cc1e0 0x7fdc981e76d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdc8402ab10 tx=0x7fdc84033c20 comp rx=0 tx=0).stop 2024-09-21T07:34:04.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.193+0000 7fdca0600640 1 -- 172.21.15.39:0/547157918 shutdown_connections 2024-09-21T07:34:04.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.193+0000 7fdca0600640 1 --2- 172.21.15.39:0/547157918 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdc78078bc0 0x7fdc7807b0d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:04.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.193+0000 7fdca0600640 1 --2- 172.21.15.39:0/547157918 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdc980ccb50 0x7fdc9815e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:04.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.193+0000 7fdca0600640 1 --2- 172.21.15.39:0/547157918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc980cc1e0 0x7fdc981e76d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:04.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.193+0000 7fdca0600640 1 --2- 172.21.15.39:0/547157918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc980af3c0 0x7fdc9815f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:04.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.193+0000 7fdca0600640 1 -- 172.21.15.39:0/547157918 >> 172.21.15.39:0/547157918 conn(0x7fdc9806d7d0 msgr2=0x7fdc980afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:04.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.193+0000 7fdca0600640 1 -- 172.21.15.39:0/547157918 shutdown_connections 2024-09-21T07:34:04.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.193+0000 7fdca0600640 1 -- 172.21.15.39:0/547157918 wait complete. 2024-09-21T07:34:04.216 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-21T07:34:04.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.441+0000 7f06f0a00640 1 -- 172.21.15.39:0/2620602454 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06d0004800 msgr2=0x7f06d0004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:04.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.441+0000 7f06f0a00640 1 --2- 172.21.15.39:0/2620602454 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06d0004800 0x7f06d0004c50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f06d4001830 tx=0x7f06d0005830 comp rx=0 tx=0).stop 2024-09-21T07:34:04.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.441+0000 7f06ebe00640 1 -- 172.21.15.39:0/2620602454 reap_dead start 2024-09-21T07:34:04.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.442+0000 7f06f0a00640 1 -- 172.21.15.39:0/2620602454 shutdown_connections 2024-09-21T07:34:04.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.442+0000 7f06f0a00640 1 -- 172.21.15.39:0/2620602454 >> 172.21.15.39:0/2620602454 conn(0x7f06ec06d870 msgr2=0x7f06ec06ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:04.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.442+0000 7f06f0a00640 1 -- 172.21.15.39:0/2620602454 shutdown_connections 2024-09-21T07:34:04.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.442+0000 7f06f0a00640 1 -- 172.21.15.39:0/2620602454 wait complete. 2024-09-21T07:34:04.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.444+0000 7f06f0a00640 1 Processor -- start 2024-09-21T07:34:04.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.444+0000 7f06f0a00640 1 -- start start 2024-09-21T07:34:04.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.445+0000 7f06f0a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06d0004800 0x7f06ec155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:04.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.445+0000 7f06f0a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f06ec0b13c0 0x7f06ec0b17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:04.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.445+0000 7f06f0a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06ec0b1d10 0x7f06ec1f6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:04.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.446+0000 7f06f0a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec1655f0 con 0x7f06d0004800 2024-09-21T07:34:04.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.446+0000 7f06f0a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec165470 con 0x7f06ec0b1d10 2024-09-21T07:34:04.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.446+0000 7f06f0a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f06ec165770 con 0x7f06ec0b13c0 2024-09-21T07:34:04.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.446+0000 7f06eb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06d0004800 0x7f06ec155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:04.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.446+0000 7f06eb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06d0004800 0x7f06ec155ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46254/0 (socket says 172.21.15.39:46254) 2024-09-21T07:34:04.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.446+0000 7f06eb400640 1 -- 172.21.15.39:0/2718037647 learned_addr learned my addr 172.21.15.39:0/2718037647 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:04.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.446+0000 7f06ebe00640 1 --2- 172.21.15.39:0/2718037647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06ec0b1d10 0x7f06ec1f6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:04.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.446+0000 7f06e3e00640 1 --2- 172.21.15.39:0/2718037647 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f06ec0b13c0 0x7f06ec0b17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:04.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.446+0000 7f06eb400640 1 -- 172.21.15.39:0/2718037647 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f06ec0b13c0 msgr2=0x7f06ec0b17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:04.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.446+0000 7f06eb400640 1 --2- 172.21.15.39:0/2718037647 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f06ec0b13c0 0x7f06ec0b17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:04.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.446+0000 7f06eb400640 1 -- 172.21.15.39:0/2718037647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06ec0b1d10 msgr2=0x7f06ec1f6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:04.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.446+0000 7f06eb400640 1 --2- 172.21.15.39:0/2718037647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06ec0b1d10 0x7f06ec1f6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:04.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.446+0000 7f06eb400640 1 -- 172.21.15.39:0/2718037647 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06ec14ce90 con 0x7f06d0004800 2024-09-21T07:34:04.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.446+0000 7f06ebe00640 1 --2- 172.21.15.39:0/2718037647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06ec0b1d10 0x7f06ec1f6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:34:04.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.446+0000 7f06e3e00640 1 --2- 172.21.15.39:0/2718037647 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f06ec0b13c0 0x7f06ec0b17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:34:04.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.447+0000 7f06eb400640 1 --2- 172.21.15.39:0/2718037647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06d0004800 0x7f06ec155ed0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f06dc006630 tx=0x7f06dc0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:04.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.447+0000 7f06e3400640 1 -- 172.21.15.39:0/2718037647 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06dc015070 con 0x7f06d0004800 2024-09-21T07:34:04.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.447+0000 7f06e3400640 1 -- 172.21.15.39:0/2718037647 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06dc011900 con 0x7f06d0004800 2024-09-21T07:34:04.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.447+0000 7f06f0a00640 1 -- 172.21.15.39:0/2718037647 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06ec14e750 con 0x7f06d0004800 2024-09-21T07:34:04.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.447+0000 7f06e3400640 1 -- 172.21.15.39:0/2718037647 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06dc011d50 con 0x7f06d0004800 2024-09-21T07:34:04.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.447+0000 7f06f0a00640 1 -- 172.21.15.39:0/2718037647 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06ec0b0900 con 0x7f06d0004800 2024-09-21T07:34:04.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.448+0000 7f06f0a00640 1 -- 172.21.15.39:0/2718037647 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06ec15cda0 con 0x7f06d0004800 2024-09-21T07:34:04.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.449+0000 7f06e3400640 1 -- 172.21.15.39:0/2718037647 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f06dc018070 con 0x7f06d0004800 2024-09-21T07:34:04.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.450+0000 7f06e3400640 1 --2- 172.21.15.39:0/2718037647 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f06c8078b90 0x7f06c807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:04.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.450+0000 7f06e3400640 1 -- 172.21.15.39:0/2718037647 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f06dc09b820 con 0x7f06d0004800 2024-09-21T07:34:04.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.451+0000 7f06e3e00640 1 --2- 172.21.15.39:0/2718037647 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f06c8078b90 0x7f06c807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:04.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.451+0000 7f06e3e00640 1 --2- 172.21.15.39:0/2718037647 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f06c8078b90 0x7f06c807b0a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f06d4032000 tx=0x7f06d4001360 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:04.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.454+0000 7f06e3400640 1 -- 172.21.15.39:0/2718037647 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f06dc063fc0 con 0x7f06d0004800 2024-09-21T07:34:04.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.726+0000 7f06f0a00640 1 -- 172.21.15.39:0/2718037647 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f06ec14e310 con 0x7f06d0004800 2024-09-21T07:34:04.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.730+0000 7f06e3400640 1 -- 172.21.15.39:0/2718037647 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f06dc0638c0 con 0x7f06d0004800 2024-09-21T07:34:04.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.732+0000 7f06f0a00640 1 -- 172.21.15.39:0/2718037647 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f06c8078b90 msgr2=0x7f06c807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:04.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.732+0000 7f06f0a00640 1 --2- 172.21.15.39:0/2718037647 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f06c8078b90 0x7f06c807b0a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f06d4032000 tx=0x7f06d4001360 comp rx=0 tx=0).stop 2024-09-21T07:34:04.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.732+0000 7f06f0a00640 1 -- 172.21.15.39:0/2718037647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06d0004800 msgr2=0x7f06ec155ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:04.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.732+0000 7f06f0a00640 1 --2- 172.21.15.39:0/2718037647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06d0004800 0x7f06ec155ed0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f06dc006630 tx=0x7f06dc0039c0 comp rx=0 tx=0).stop 2024-09-21T07:34:04.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.732+0000 7f06f0a00640 1 -- 172.21.15.39:0/2718037647 shutdown_connections 2024-09-21T07:34:04.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.732+0000 7f06f0a00640 1 --2- 172.21.15.39:0/2718037647 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f06c8078b90 0x7f06c807b0a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:04.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.732+0000 7f06f0a00640 1 --2- 172.21.15.39:0/2718037647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06ec0b1d10 0x7f06ec1f6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:04.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.732+0000 7f06f0a00640 1 --2- 172.21.15.39:0/2718037647 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f06ec0b13c0 0x7f06ec0b17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:04.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.732+0000 7f06f0a00640 1 --2- 172.21.15.39:0/2718037647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06d0004800 0x7f06ec155ed0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:04.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.732+0000 7f06f0a00640 1 -- 172.21.15.39:0/2718037647 >> 172.21.15.39:0/2718037647 conn(0x7f06ec06d870 msgr2=0x7f06ec0ca580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:04.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.732+0000 7f06f0a00640 1 -- 172.21.15.39:0/2718037647 shutdown_connections 2024-09-21T07:34:04.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.732+0000 7f06f0a00640 1 -- 172.21.15.39:0/2718037647 wait complete. 2024-09-21T07:34:04.756 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T07:34:04.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.978+0000 7f80b8e00640 1 -- 172.21.15.39:0/2849886038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80b4073170 msgr2=0x7f80b415c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:04.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.978+0000 7f80b8e00640 1 --2- 172.21.15.39:0/2849886038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80b4073170 0x7f80b415c780 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f80b40710f0 tx=0x7f80a402a560 comp rx=0 tx=0).stop 2024-09-21T07:34:04.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.979+0000 7f80b8e00640 1 -- 172.21.15.39:0/2849886038 shutdown_connections 2024-09-21T07:34:04.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.979+0000 7f80b8e00640 1 --2- 172.21.15.39:0/2849886038 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80b4070440 0x7f80b4074b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:04.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.979+0000 7f80b8e00640 1 --2- 172.21.15.39:0/2849886038 >> v2:172.21.15.148:3300/0 conn(0x7f80b4070030 0x7f80b4074640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:04.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.979+0000 7f80b8e00640 1 --2- 172.21.15.39:0/2849886038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80b4073170 0x7f80b415c780 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:04.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.979+0000 7f80b8e00640 1 -- 172.21.15.39:0/2849886038 >> 172.21.15.39:0/2849886038 conn(0x7f80b40b1cb0 msgr2=0x7f80b40b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:04.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.979+0000 7f80b8e00640 1 -- 172.21.15.39:0/2849886038 shutdown_connections 2024-09-21T07:34:04.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.979+0000 7f80b8e00640 1 -- 172.21.15.39:0/2849886038 wait complete. 2024-09-21T07:34:04.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.981+0000 7f80b8e00640 1 Processor -- start 2024-09-21T07:34:04.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.981+0000 7f80b8e00640 1 -- start start 2024-09-21T07:34:04.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.983+0000 7f80b8e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80b4070030 0x7f80b414ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:04.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.983+0000 7f80b8e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80b4070440 0x7f80b41f42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:04.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.983+0000 7f80b8e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80b4073170 0x7f80b40d0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:04.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.983+0000 7f80b8e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f80b406f630 con 0x7f80b4073170 2024-09-21T07:34:04.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.983+0000 7f80b8e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f80b406f4b0 con 0x7f80b4070030 2024-09-21T07:34:04.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.983+0000 7f80b8e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f80b406f7b0 con 0x7f80b4070440 2024-09-21T07:34:04.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.983+0000 7f80b2800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80b4073170 0x7f80b40d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:04.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.983+0000 7f80b2800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80b4073170 0x7f80b40d0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46264/0 (socket says 172.21.15.39:46264) 2024-09-21T07:34:04.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.983+0000 7f80b2800640 1 -- 172.21.15.39:0/3496182571 learned_addr learned my addr 172.21.15.39:0/3496182571 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:04.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.983+0000 7f80b1400640 1 --2- 172.21.15.39:0/3496182571 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80b4070440 0x7f80b41f42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:04.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.983+0000 7f80b1e00640 1 --2- 172.21.15.39:0/3496182571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80b4070030 0x7f80b414ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:04.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.983+0000 7f80b2800640 1 -- 172.21.15.39:0/3496182571 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80b4070440 msgr2=0x7f80b41f42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:04.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.983+0000 7f80b2800640 1 --2- 172.21.15.39:0/3496182571 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80b4070440 0x7f80b41f42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:04.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.983+0000 7f80b2800640 1 -- 172.21.15.39:0/3496182571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80b4070030 msgr2=0x7f80b414ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:04.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.983+0000 7f80b2800640 1 --2- 172.21.15.39:0/3496182571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80b4070030 0x7f80b414ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:04.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.983+0000 7f80b2800640 1 -- 172.21.15.39:0/3496182571 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b41f4830 con 0x7f80b4073170 2024-09-21T07:34:04.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.984+0000 7f80b2800640 1 --2- 172.21.15.39:0/3496182571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80b4073170 0x7f80b40d0440 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f80ac00cad0 tx=0x7f80ac00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:04.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.984+0000 7f80aa000640 1 -- 172.21.15.39:0/3496182571 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac00ae20 con 0x7f80b4073170 2024-09-21T07:34:04.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.984+0000 7f80aa000640 1 -- 172.21.15.39:0/3496182571 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80ac00f540 con 0x7f80b4073170 2024-09-21T07:34:04.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.984+0000 7f80aa000640 1 -- 172.21.15.39:0/3496182571 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac00f990 con 0x7f80b4073170 2024-09-21T07:34:04.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.984+0000 7f80b8e00640 1 -- 172.21.15.39:0/3496182571 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80b40d0980 con 0x7f80b4073170 2024-09-21T07:34:04.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.984+0000 7f80b8e00640 1 -- 172.21.15.39:0/3496182571 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80b40cf680 con 0x7f80b4073170 2024-09-21T07:34:04.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.985+0000 7f80b8e00640 1 -- 172.21.15.39:0/3496182571 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80b415c780 con 0x7f80b4073170 2024-09-21T07:34:04.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.987+0000 7f80aa000640 1 -- 172.21.15.39:0/3496182571 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f80ac00fb30 con 0x7f80b4073170 2024-09-21T07:34:04.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.988+0000 7f80aa000640 1 --2- 172.21.15.39:0/3496182571 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8094078b90 0x7f809407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:04.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.988+0000 7f80aa000640 1 -- 172.21.15.39:0/3496182571 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f80ac09b3f0 con 0x7f80b4073170 2024-09-21T07:34:04.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.988+0000 7f80b1e00640 1 --2- 172.21.15.39:0/3496182571 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8094078b90 0x7f809407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:04.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.989+0000 7f80b1e00640 1 --2- 172.21.15.39:0/3496182571 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8094078b90 0x7f809407b0a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f80a4033000 tx=0x7f80a4004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:04.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:04.992+0000 7f80aa000640 1 -- 172.21.15.39:0/3496182571 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f80b41f4830 con 0x7f80b4073170 2024-09-21T07:34:05.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.190+0000 7f80b8e00640 1 -- 172.21.15.39:0/3496182571 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f80b40c3860 con 0x7f80b4073170 2024-09-21T07:34:05.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.190+0000 7f80aa000640 1 -- 172.21.15.39:0/3496182571 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) ==== 84+0+65 (secure 0 0 0) 0x7f80ac014030 con 0x7f80b4073170 2024-09-21T07:34:05.193 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:34:05.193 INFO:teuthology.orchestra.run.smithi039.stdout:[ 2024-09-21T07:34:05.193 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:05.193 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 1, 2024-09-21T07:34:05.193 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": ".mgr" 2024-09-21T07:34:05.193 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:05.193 INFO:teuthology.orchestra.run.smithi039.stdout:] 2024-09-21T07:34:05.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.193+0000 7f80b8e00640 1 -- 172.21.15.39:0/3496182571 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8094078b90 msgr2=0x7f809407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:05.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.193+0000 7f80b8e00640 1 --2- 172.21.15.39:0/3496182571 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8094078b90 0x7f809407b0a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f80a4033000 tx=0x7f80a4004730 comp rx=0 tx=0).stop 2024-09-21T07:34:05.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.193+0000 7f80b8e00640 1 -- 172.21.15.39:0/3496182571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80b4073170 msgr2=0x7f80b40d0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:05.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.193+0000 7f80b8e00640 1 --2- 172.21.15.39:0/3496182571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80b4073170 0x7f80b40d0440 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f80ac00cad0 tx=0x7f80ac00cf90 comp rx=0 tx=0).stop 2024-09-21T07:34:05.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.193+0000 7f80b8e00640 1 -- 172.21.15.39:0/3496182571 shutdown_connections 2024-09-21T07:34:05.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.193+0000 7f80b8e00640 1 --2- 172.21.15.39:0/3496182571 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8094078b90 0x7f809407b0a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:05.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.193+0000 7f80b8e00640 1 --2- 172.21.15.39:0/3496182571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80b4073170 0x7f80b40d0440 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:05.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.193+0000 7f80b8e00640 1 --2- 172.21.15.39:0/3496182571 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80b4070440 0x7f80b41f42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:05.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.193+0000 7f80b8e00640 1 --2- 172.21.15.39:0/3496182571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80b4070030 0x7f80b414ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:05.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.193+0000 7f80b8e00640 1 -- 172.21.15.39:0/3496182571 >> 172.21.15.39:0/3496182571 conn(0x7f80b40b1cb0 msgr2=0x7f80b40ae810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:05.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.193+0000 7f80b8e00640 1 -- 172.21.15.39:0/3496182571 shutdown_connections 2024-09-21T07:34:05.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.193+0000 7f80b8e00640 1 -- 172.21.15.39:0/3496182571 wait complete. 2024-09-21T07:34:05.216 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T07:34:05.216 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T07:34:05.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.445+0000 7fd5d1400640 1 -- 172.21.15.39:0/2556254652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5b0003a70 msgr2=0x7fd5b0003ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:05.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.445+0000 7fd5d1400640 1 --2- 172.21.15.39:0/2556254652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5b0003a70 0x7fd5b0003ee0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc00c2c0 tx=0x7fd5b0005830 comp rx=0 tx=0).stop 2024-09-21T07:34:05.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.445+0000 7fd5d0a00640 1 -- 172.21.15.39:0/2556254652 reap_dead start 2024-09-21T07:34:05.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.445+0000 7fd5d1400640 1 -- 172.21.15.39:0/2556254652 shutdown_connections 2024-09-21T07:34:05.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.445+0000 7fd5d1400640 1 -- 172.21.15.39:0/2556254652 >> 172.21.15.39:0/2556254652 conn(0x7fd5cc06cfc0 msgr2=0x7fd5cc06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:05.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.445+0000 7fd5d1400640 1 -- 172.21.15.39:0/2556254652 shutdown_connections 2024-09-21T07:34:05.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.445+0000 7fd5d1400640 1 -- 172.21.15.39:0/2556254652 wait complete. 2024-09-21T07:34:05.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.447+0000 7fd5d1400640 1 Processor -- start 2024-09-21T07:34:05.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.448+0000 7fd5d1400640 1 -- start start 2024-09-21T07:34:05.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.449+0000 7fd5d1400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5b0003a70 0x7fd5cc158ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:05.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.449+0000 7fd5d1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5cc13f0b0 0x7fd5cc157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:05.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.449+0000 7fd5d1400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5cc159920 0x7fd5cc159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:05.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.449+0000 7fd5d1400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd5cc160510 con 0x7fd5cc13f0b0 2024-09-21T07:34:05.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.449+0000 7fd5d1400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd5cc160390 con 0x7fd5b0003a70 2024-09-21T07:34:05.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.449+0000 7fd5d1400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd5cc160cf0 con 0x7fd5cc159920 2024-09-21T07:34:05.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.449+0000 7fd5c3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5cc13f0b0 0x7fd5cc157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:05.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.449+0000 7fd5c3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5cc13f0b0 0x7fd5cc157b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46284/0 (socket says 172.21.15.39:46284) 2024-09-21T07:34:05.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.449+0000 7fd5c3e00640 1 -- 172.21.15.39:0/866253259 learned_addr learned my addr 172.21.15.39:0/866253259 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:05.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.449+0000 7fd5cbe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5b0003a70 0x7fd5cc158ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:05.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.449+0000 7fd5d0a00640 1 --2- 172.21.15.39:0/866253259 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5cc159920 0x7fd5cc159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:05.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.450+0000 7fd5c3e00640 1 -- 172.21.15.39:0/866253259 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5cc159920 msgr2=0x7fd5cc159d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:05.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.450+0000 7fd5c3e00640 1 --2- 172.21.15.39:0/866253259 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5cc159920 0x7fd5cc159d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:05.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.450+0000 7fd5c3e00640 1 -- 172.21.15.39:0/866253259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5b0003a70 msgr2=0x7fd5cc158ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:05.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.450+0000 7fd5c3e00640 1 --2- 172.21.15.39:0/866253259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5b0003a70 0x7fd5cc158ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:05.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.450+0000 7fd5c3e00640 1 -- 172.21.15.39:0/866253259 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5cc142060 con 0x7fd5cc13f0b0 2024-09-21T07:34:05.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.450+0000 7fd5c3e00640 1 --2- 172.21.15.39:0/866253259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5cc13f0b0 0x7fd5cc157b00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4011ae0 tx=0x7fd5b4009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:05.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.450+0000 7fd5c8c00640 1 -- 172.21.15.39:0/866253259 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b401e070 con 0x7fd5cc13f0b0 2024-09-21T07:34:05.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.450+0000 7fd5c8c00640 1 -- 172.21.15.39:0/866253259 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd5b400fe70 con 0x7fd5cc13f0b0 2024-09-21T07:34:05.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.450+0000 7fd5c8c00640 1 -- 172.21.15.39:0/866253259 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b4004290 con 0x7fd5cc13f0b0 2024-09-21T07:34:05.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.450+0000 7fd5d1400640 1 -- 172.21.15.39:0/866253259 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5cc158400 con 0x7fd5cc13f0b0 2024-09-21T07:34:05.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.451+0000 7fd5d1400640 1 -- 172.21.15.39:0/866253259 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5cc158040 con 0x7fd5cc13f0b0 2024-09-21T07:34:05.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.451+0000 7fd5d1400640 1 -- 172.21.15.39:0/866253259 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5cc141db0 con 0x7fd5cc13f0b0 2024-09-21T07:34:05.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.453+0000 7fd5c8c00640 1 -- 172.21.15.39:0/866253259 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd5b4021070 con 0x7fd5cc13f0b0 2024-09-21T07:34:05.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.454+0000 7fd5c8c00640 1 --2- 172.21.15.39:0/866253259 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd5a8078bc0 0x7fd5a807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:05.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.454+0000 7fd5c8c00640 1 -- 172.21.15.39:0/866253259 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7fd5b409b540 con 0x7fd5cc13f0b0 2024-09-21T07:34:05.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.454+0000 7fd5cbe00640 1 --2- 172.21.15.39:0/866253259 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd5a8078bc0 0x7fd5a807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:05.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.455+0000 7fd5cbe00640 1 --2- 172.21.15.39:0/866253259 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd5a8078bc0 0x7fd5a807b0d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc033000 tx=0x7fd5bc038040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:05.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.458+0000 7fd5c8c00640 1 -- 172.21.15.39:0/866253259 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd5b40636a0 con 0x7fd5cc13f0b0 2024-09-21T07:34:05.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:05.658+0000 7fd5d1400640 1 -- 172.21.15.39:0/866253259 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fd5cc0cec70 con 0x7fd5cc13f0b0 2024-09-21T07:34:06.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.120+0000 7fd5c8c00640 1 -- 172.21.15.39:0/866253259 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v31) ==== 138+0+0 (secure 0 0 0) 0x7fd5b40680d0 con 0x7fd5cc13f0b0 2024-09-21T07:34:06.122 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' created 2024-09-21T07:34:06.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.123+0000 7fd5d1400640 1 -- 172.21.15.39:0/866253259 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd5a8078bc0 msgr2=0x7fd5a807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:06.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.123+0000 7fd5d1400640 1 --2- 172.21.15.39:0/866253259 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd5a8078bc0 0x7fd5a807b0d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc033000 tx=0x7fd5bc038040 comp rx=0 tx=0).stop 2024-09-21T07:34:06.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.123+0000 7fd5d1400640 1 -- 172.21.15.39:0/866253259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5cc13f0b0 msgr2=0x7fd5cc157b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:06.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.123+0000 7fd5d1400640 1 --2- 172.21.15.39:0/866253259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5cc13f0b0 0x7fd5cc157b00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4011ae0 tx=0x7fd5b4009e90 comp rx=0 tx=0).stop 2024-09-21T07:34:06.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.123+0000 7fd5d1400640 1 -- 172.21.15.39:0/866253259 shutdown_connections 2024-09-21T07:34:06.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.123+0000 7fd5d1400640 1 --2- 172.21.15.39:0/866253259 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd5a8078bc0 0x7fd5a807b0d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:06.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.123+0000 7fd5d1400640 1 --2- 172.21.15.39:0/866253259 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5cc159920 0x7fd5cc159d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:06.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.123+0000 7fd5d1400640 1 --2- 172.21.15.39:0/866253259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5cc13f0b0 0x7fd5cc157b00 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:06.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.123+0000 7fd5d1400640 1 --2- 172.21.15.39:0/866253259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5b0003a70 0x7fd5cc158ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:06.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.123+0000 7fd5d1400640 1 -- 172.21.15.39:0/866253259 >> 172.21.15.39:0/866253259 conn(0x7fd5cc06cfc0 msgr2=0x7fd5cc0b38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:06.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.124+0000 7fd5d1400640 1 -- 172.21.15.39:0/866253259 shutdown_connections 2024-09-21T07:34:06.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.124+0000 7fd5d1400640 1 -- 172.21.15.39:0/866253259 wait complete. 2024-09-21T07:34:06.154 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T07:34:06.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.387+0000 7f5c7d200640 1 -- 172.21.15.39:0/2104976170 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c7815d8a0 msgr2=0x7f5c780b40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:06.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.387+0000 7f5c7d200640 1 --2- 172.21.15.39:0/2104976170 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c7815d8a0 0x7f5c780b40b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5c60005fd0 tx=0x7f5c6002a5d0 comp rx=0 tx=0).stop 2024-09-21T07:34:06.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.387+0000 7f5c7d200640 1 -- 172.21.15.39:0/2104976170 shutdown_connections 2024-09-21T07:34:06.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.387+0000 7f5c7d200640 1 --2- 172.21.15.39:0/2104976170 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c7815d8a0 0x7f5c780b40b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:06.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.387+0000 7f5c7d200640 1 --2- 172.21.15.39:0/2104976170 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c780cc1e0 0x7f5c780af3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:06.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.387+0000 7f5c7d200640 1 -- 172.21.15.39:0/2104976170 >> 172.21.15.39:0/2104976170 conn(0x7f5c7806d7d0 msgr2=0x7f5c780b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:06.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.387+0000 7f5c7d200640 1 -- 172.21.15.39:0/2104976170 shutdown_connections 2024-09-21T07:34:06.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.387+0000 7f5c7d200640 1 -- 172.21.15.39:0/2104976170 wait complete. 2024-09-21T07:34:06.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.389+0000 7f5c7d200640 1 Processor -- start 2024-09-21T07:34:06.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.389+0000 7f5c7d200640 1 -- start start 2024-09-21T07:34:06.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.391+0000 7f5c7d200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c780cc1e0 0x7f5c780ad360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:06.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.391+0000 7f5c7d200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c780ccb50 0x7f5c781f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:06.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.391+0000 7f5c7d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c7815d8a0 0x7f5c78073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:06.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.391+0000 7f5c7d200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5c78153520 con 0x7f5c7815d8a0 2024-09-21T07:34:06.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.391+0000 7f5c7d200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5c781533a0 con 0x7f5c780ccb50 2024-09-21T07:34:06.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.391+0000 7f5c7d200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f5c781536a0 con 0x7f5c780cc1e0 2024-09-21T07:34:06.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.391+0000 7f5c76800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c7815d8a0 0x7f5c78073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:06.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.391+0000 7f5c76800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c7815d8a0 0x7f5c78073f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46296/0 (socket says 172.21.15.39:46296) 2024-09-21T07:34:06.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.391+0000 7f5c76800640 1 -- 172.21.15.39:0/1839276332 learned_addr learned my addr 172.21.15.39:0/1839276332 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:06.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.392+0000 7f5c6d400640 1 --2- 172.21.15.39:0/1839276332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c780ccb50 0x7f5c781f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:06.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.392+0000 7f5c75e00640 1 --2- 172.21.15.39:0/1839276332 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c780cc1e0 0x7f5c780ad360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:06.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.392+0000 7f5c76800640 1 -- 172.21.15.39:0/1839276332 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c780cc1e0 msgr2=0x7f5c780ad360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:06.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.392+0000 7f5c76800640 1 --2- 172.21.15.39:0/1839276332 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c780cc1e0 0x7f5c780ad360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:06.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.392+0000 7f5c76800640 1 -- 172.21.15.39:0/1839276332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c780ccb50 msgr2=0x7f5c781f56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:06.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.392+0000 7f5c76800640 1 --2- 172.21.15.39:0/1839276332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c780ccb50 0x7f5c781f56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:06.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.392+0000 7f5c76800640 1 -- 172.21.15.39:0/1839276332 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c7809baa0 con 0x7f5c7815d8a0 2024-09-21T07:34:06.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.392+0000 7f5c75e00640 1 --2- 172.21.15.39:0/1839276332 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c780cc1e0 0x7f5c780ad360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:34:06.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.392+0000 7f5c76800640 1 --2- 172.21.15.39:0/1839276332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c7815d8a0 0x7f5c78073f00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5c70002230 tx=0x7f5c7000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:06.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.392+0000 7f5c6ea00640 1 -- 172.21.15.39:0/1839276332 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c70003b70 con 0x7f5c7815d8a0 2024-09-21T07:34:06.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.392+0000 7f5c6ea00640 1 -- 172.21.15.39:0/1839276332 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c70003d10 con 0x7f5c7815d8a0 2024-09-21T07:34:06.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.392+0000 7f5c6ea00640 1 -- 172.21.15.39:0/1839276332 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c70005330 con 0x7f5c7815d8a0 2024-09-21T07:34:06.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.392+0000 7f5c7d200640 1 -- 172.21.15.39:0/1839276332 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c7809b4e0 con 0x7f5c7815d8a0 2024-09-21T07:34:06.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.393+0000 7f5c7d200640 1 -- 172.21.15.39:0/1839276332 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c78156a90 con 0x7f5c7815d8a0 2024-09-21T07:34:06.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.394+0000 7f5c7d200640 1 -- 172.21.15.39:0/1839276332 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c78156d70 con 0x7f5c7815d8a0 2024-09-21T07:34:06.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.395+0000 7f5c6ea00640 1 -- 172.21.15.39:0/1839276332 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5c700056e0 con 0x7f5c7815d8a0 2024-09-21T07:34:06.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.397+0000 7f5c6ea00640 1 --2- 172.21.15.39:0/1839276332 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5c54078b90 0x7f5c5407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:06.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.397+0000 7f5c6ea00640 1 -- 172.21.15.39:0/1839276332 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5227+0+0 (secure 0 0 0) 0x7f5c7009b5f0 con 0x7f5c7815d8a0 2024-09-21T07:34:06.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.397+0000 7f5c75e00640 1 --2- 172.21.15.39:0/1839276332 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5c54078b90 0x7f5c5407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:06.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.398+0000 7f5c75e00640 1 --2- 172.21.15.39:0/1839276332 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5c54078b90 0x7f5c5407b0a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009ab0 tx=0x7f5c68005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:06.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.401+0000 7f5c6ea00640 1 -- 172.21.15.39:0/1839276332 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5c7809baa0 con 0x7f5c7815d8a0 2024-09-21T07:34:06.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:06.600+0000 7f5c7d200640 1 -- 172.21.15.39:0/1839276332 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f5c780af5e0 con 0x7f5c7815d8a0 2024-09-21T07:34:07.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.123+0000 7f5c6ea00640 1 -- 172.21.15.39:0/1839276332 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v32) ==== 170+0+0 (secure 0 0 0) 0x7f5c780af5e0 con 0x7f5c7815d8a0 2024-09-21T07:34:07.125 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' created 2024-09-21T07:34:07.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.127+0000 7f5c7d200640 1 -- 172.21.15.39:0/1839276332 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5c54078b90 msgr2=0x7f5c5407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:07.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.127+0000 7f5c7d200640 1 --2- 172.21.15.39:0/1839276332 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5c54078b90 0x7f5c5407b0a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009ab0 tx=0x7f5c68005f90 comp rx=0 tx=0).stop 2024-09-21T07:34:07.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.127+0000 7f5c7d200640 1 -- 172.21.15.39:0/1839276332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c7815d8a0 msgr2=0x7f5c78073f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:07.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.127+0000 7f5c7d200640 1 --2- 172.21.15.39:0/1839276332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c7815d8a0 0x7f5c78073f00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5c70002230 tx=0x7f5c7000cb90 comp rx=0 tx=0).stop 2024-09-21T07:34:07.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.129+0000 7f5c7d200640 1 -- 172.21.15.39:0/1839276332 shutdown_connections 2024-09-21T07:34:07.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.129+0000 7f5c7d200640 1 --2- 172.21.15.39:0/1839276332 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5c54078b90 0x7f5c5407b0a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:07.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.129+0000 7f5c7d200640 1 --2- 172.21.15.39:0/1839276332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c7815d8a0 0x7f5c78073f00 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:07.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.129+0000 7f5c7d200640 1 --2- 172.21.15.39:0/1839276332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c780ccb50 0x7f5c781f56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:07.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.129+0000 7f5c7d200640 1 --2- 172.21.15.39:0/1839276332 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c780cc1e0 0x7f5c780ad360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:07.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.129+0000 7f5c7d200640 1 -- 172.21.15.39:0/1839276332 >> 172.21.15.39:0/1839276332 conn(0x7f5c7806d7d0 msgr2=0x7f5c780b1450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:07.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.130+0000 7f5c7d200640 1 -- 172.21.15.39:0/1839276332 shutdown_connections 2024-09-21T07:34:07.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.130+0000 7f5c7d200640 1 -- 172.21.15.39:0/1839276332 wait complete. 2024-09-21T07:34:07.171 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T07:34:07.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.398+0000 7fc07f400640 1 -- 172.21.15.39:0/1627155736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0780cd3e0 msgr2=0x7fc078070180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:07.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.398+0000 7fc07f400640 1 --2- 172.21.15.39:0/1627155736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0780cd3e0 0x7fc078070180 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc064005fd0 tx=0x7fc06402a5d0 comp rx=0 tx=0).stop 2024-09-21T07:34:07.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.398+0000 7fc07f400640 1 -- 172.21.15.39:0/1627155736 shutdown_connections 2024-09-21T07:34:07.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.398+0000 7fc07f400640 1 --2- 172.21.15.39:0/1627155736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0780cd3e0 0x7fc078070180 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:07.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.398+0000 7fc07f400640 1 --2- 172.21.15.39:0/1627155736 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc0780af780 0x7fc0780b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:07.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.399+0000 7fc07f400640 1 -- 172.21.15.39:0/1627155736 >> 172.21.15.39:0/1627155736 conn(0x7fc07806d7d0 msgr2=0x7fc0780b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:07.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.399+0000 7fc07f400640 1 -- 172.21.15.39:0/1627155736 shutdown_connections 2024-09-21T07:34:07.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.399+0000 7fc07f400640 1 -- 172.21.15.39:0/1627155736 wait complete. 2024-09-21T07:34:07.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.401+0000 7fc07f400640 1 Processor -- start 2024-09-21T07:34:07.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.401+0000 7fc07f400640 1 -- start start 2024-09-21T07:34:07.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.402+0000 7fc07f400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc0780706c0 0x7fc0780c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:07.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.402+0000 7fc07f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0780af780 0x7fc07814bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:07.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.402+0000 7fc07f400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0780cd3e0 0x7fc07815cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:07.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.402+0000 7fc07f400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc078073b80 con 0x7fc0780af780 2024-09-21T07:34:07.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.402+0000 7fc07f400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc078073a00 con 0x7fc0780cd3e0 2024-09-21T07:34:07.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.402+0000 7fc07f400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc07814d190 con 0x7fc0780706c0 2024-09-21T07:34:07.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.403+0000 7fc077e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0780af780 0x7fc07814bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:07.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.403+0000 7fc077e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0780af780 0x7fc07814bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46314/0 (socket says 172.21.15.39:46314) 2024-09-21T07:34:07.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.403+0000 7fc077e00640 1 -- 172.21.15.39:0/1252631330 learned_addr learned my addr 172.21.15.39:0/1252631330 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:07.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.403+0000 7fc07ca00640 1 --2- 172.21.15.39:0/1252631330 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc0780706c0 0x7fc0780c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:07.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.403+0000 7fc07d400640 1 --2- 172.21.15.39:0/1252631330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0780cd3e0 0x7fc07815cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:07.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.403+0000 7fc077e00640 1 -- 172.21.15.39:0/1252631330 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc0780706c0 msgr2=0x7fc0780c4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:07.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.403+0000 7fc077e00640 1 --2- 172.21.15.39:0/1252631330 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc0780706c0 0x7fc0780c4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:07.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.403+0000 7fc077e00640 1 -- 172.21.15.39:0/1252631330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0780cd3e0 msgr2=0x7fc07815cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:07.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.403+0000 7fc077e00640 1 --2- 172.21.15.39:0/1252631330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0780cd3e0 0x7fc07815cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:07.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.403+0000 7fc077e00640 1 -- 172.21.15.39:0/1252631330 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc07814d3e0 con 0x7fc0780af780 2024-09-21T07:34:07.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.403+0000 7fc07d400640 1 --2- 172.21.15.39:0/1252631330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0780cd3e0 0x7fc07815cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:34:07.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.403+0000 7fc07ca00640 1 --2- 172.21.15.39:0/1252631330 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc0780706c0 0x7fc0780c4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:34:07.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.403+0000 7fc077e00640 1 --2- 172.21.15.39:0/1252631330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0780af780 0x7fc07814bc10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc064034040 tx=0x7fc0640035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:07.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.404+0000 7fc074c00640 1 -- 172.21.15.39:0/1252631330 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06403e030 con 0x7fc0780af780 2024-09-21T07:34:07.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.404+0000 7fc074c00640 1 -- 172.21.15.39:0/1252631330 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc064003ae0 con 0x7fc0780af780 2024-09-21T07:34:07.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.404+0000 7fc074c00640 1 -- 172.21.15.39:0/1252631330 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc064035040 con 0x7fc0780af780 2024-09-21T07:34:07.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.404+0000 7fc07f400640 1 -- 172.21.15.39:0/1252631330 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc07814d6c0 con 0x7fc0780af780 2024-09-21T07:34:07.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.404+0000 7fc07f400640 1 -- 172.21.15.39:0/1252631330 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc07814dc80 con 0x7fc0780af780 2024-09-21T07:34:07.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.405+0000 7fc07f400640 1 -- 172.21.15.39:0/1252631330 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0780b39c0 con 0x7fc0780af780 2024-09-21T07:34:07.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.407+0000 7fc074c00640 1 -- 172.21.15.39:0/1252631330 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc064041030 con 0x7fc0780af780 2024-09-21T07:34:07.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.408+0000 7fc074c00640 1 --2- 172.21.15.39:0/1252631330 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc058078b90 0x7fc05807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:07.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.408+0000 7fc074c00640 1 -- 172.21.15.39:0/1252631330 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5609+0+0 (secure 0 0 0) 0x7fc0640bb740 con 0x7fc0780af780 2024-09-21T07:34:07.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.408+0000 7fc07ca00640 1 --2- 172.21.15.39:0/1252631330 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc058078b90 0x7fc05807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:07.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.409+0000 7fc07ca00640 1 --2- 172.21.15.39:0/1252631330 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc058078b90 0x7fc05807b0a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc06c009f20 tx=0x7fc06c005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:07.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.412+0000 7fc074c00640 1 -- 172.21.15.39:0/1252631330 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc064083570 con 0x7fc0780af780 2024-09-21T07:34:07.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:07.650+0000 7fc07f400640 1 -- 172.21.15.39:0/1252631330 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fc0780b1e30 con 0x7fc0780af780 2024-09-21T07:34:08.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.149+0000 7fc074c00640 1 -- 172.21.15.39:0/1252631330 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7fc064087fa0 con 0x7fc0780af780 2024-09-21T07:34:08.152 INFO:teuthology.orchestra.run.smithi039.stderr:new fs with metadata pool 4 and data pool 5 2024-09-21T07:34:08.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.155+0000 7fc07f400640 1 -- 172.21.15.39:0/1252631330 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc058078b90 msgr2=0x7fc05807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:08.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.155+0000 7fc07f400640 1 --2- 172.21.15.39:0/1252631330 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc058078b90 0x7fc05807b0a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc06c009f20 tx=0x7fc06c005f90 comp rx=0 tx=0).stop 2024-09-21T07:34:08.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.155+0000 7fc07f400640 1 -- 172.21.15.39:0/1252631330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0780af780 msgr2=0x7fc07814bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:08.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.155+0000 7fc07f400640 1 --2- 172.21.15.39:0/1252631330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0780af780 0x7fc07814bc10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc064034040 tx=0x7fc0640035f0 comp rx=0 tx=0).stop 2024-09-21T07:34:08.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.156+0000 7fc07f400640 1 -- 172.21.15.39:0/1252631330 shutdown_connections 2024-09-21T07:34:08.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.156+0000 7fc07f400640 1 --2- 172.21.15.39:0/1252631330 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc058078b90 0x7fc05807b0a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:08.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.156+0000 7fc07f400640 1 --2- 172.21.15.39:0/1252631330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0780cd3e0 0x7fc07815cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:08.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.156+0000 7fc07f400640 1 --2- 172.21.15.39:0/1252631330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0780af780 0x7fc07814bc10 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:08.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.156+0000 7fc07f400640 1 --2- 172.21.15.39:0/1252631330 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc0780706c0 0x7fc0780c4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:08.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.156+0000 7fc07f400640 1 -- 172.21.15.39:0/1252631330 >> 172.21.15.39:0/1252631330 conn(0x7fc07806d7d0 msgr2=0x7fc0780ce320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:08.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.158+0000 7fc07f400640 1 -- 172.21.15.39:0/1252631330 shutdown_connections 2024-09-21T07:34:08.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.158+0000 7fc07f400640 1 -- 172.21.15.39:0/1252631330 wait complete. 2024-09-21T07:34:08.204 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:34:08.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.431+0000 7f92eb200640 1 -- 172.21.15.39:0/384916331 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92ec144ce0 msgr2=0x7f92ec0cbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:08.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.431+0000 7f92eb200640 1 --2- 172.21.15.39:0/384916331 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92ec144ce0 0x7f92ec0cbcd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f92ec073b10 tx=0x7f92dc02a5f0 comp rx=0 tx=0).stop 2024-09-21T07:34:08.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.431+0000 7f92eb200640 1 -- 172.21.15.39:0/384916331 shutdown_connections 2024-09-21T07:34:08.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.431+0000 7f92eb200640 1 --2- 172.21.15.39:0/384916331 >> v2:172.21.15.148:3300/0 conn(0x7f92ec15fb10 0x7f92ec0cfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:08.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.431+0000 7f92eb200640 1 --2- 172.21.15.39:0/384916331 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92ec144ce0 0x7f92ec0cbcd0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:08.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.431+0000 7f92eb200640 1 -- 172.21.15.39:0/384916331 >> 172.21.15.39:0/384916331 conn(0x7f92ec06d870 msgr2=0x7f92ec06ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:08.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.431+0000 7f92eb200640 1 -- 172.21.15.39:0/384916331 shutdown_connections 2024-09-21T07:34:08.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.431+0000 7f92eb200640 1 -- 172.21.15.39:0/384916331 wait complete. 2024-09-21T07:34:08.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.433+0000 7f92eb200640 1 Processor -- start 2024-09-21T07:34:08.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.433+0000 7f92eb200640 1 -- start start 2024-09-21T07:34:08.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.435+0000 7f92eb200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92ec0d0270 0x7f92ec1f9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:08.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.435+0000 7f92eb200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92ec144ce0 0x7f92ec0c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:08.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.435+0000 7f92eb200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92ec15fb10 0x7f92ec13f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:08.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.435+0000 7f92eb200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f92ec074fc0 con 0x7f92ec15fb10 2024-09-21T07:34:08.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.435+0000 7f92eb200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f92ec0ce230 con 0x7f92ec0d0270 2024-09-21T07:34:08.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.435+0000 7f92eb200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f92ec13cd90 con 0x7f92ec144ce0 2024-09-21T07:34:08.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.435+0000 7f92ea800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92ec15fb10 0x7f92ec13f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:08.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.435+0000 7f92ea800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92ec15fb10 0x7f92ec13f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46338/0 (socket says 172.21.15.39:46338) 2024-09-21T07:34:08.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.435+0000 7f92ea800640 1 -- 172.21.15.39:0/1653032251 learned_addr learned my addr 172.21.15.39:0/1653032251 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:08.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.435+0000 7f92e9e00640 1 --2- 172.21.15.39:0/1653032251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92ec0d0270 0x7f92ec1f9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:08.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.435+0000 7f92e3e00640 1 --2- 172.21.15.39:0/1653032251 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92ec144ce0 0x7f92ec0c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:08.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.435+0000 7f92ea800640 1 -- 172.21.15.39:0/1653032251 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92ec144ce0 msgr2=0x7f92ec0c9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:08.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.435+0000 7f92ea800640 1 --2- 172.21.15.39:0/1653032251 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92ec144ce0 0x7f92ec0c9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:08.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.435+0000 7f92ea800640 1 -- 172.21.15.39:0/1653032251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92ec0d0270 msgr2=0x7f92ec1f9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:08.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.435+0000 7f92ea800640 1 --2- 172.21.15.39:0/1653032251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92ec0d0270 0x7f92ec1f9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:08.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.435+0000 7f92ea800640 1 -- 172.21.15.39:0/1653032251 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92ec13d220 con 0x7f92ec15fb10 2024-09-21T07:34:08.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.435+0000 7f92e3e00640 1 --2- 172.21.15.39:0/1653032251 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92ec144ce0 0x7f92ec0c9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:34:08.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.436+0000 7f92ea800640 1 --2- 172.21.15.39:0/1653032251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92ec15fb10 0x7f92ec13f090 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f92e400c730 tx=0x7f92e400cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:08.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.436+0000 7f92e2000640 1 -- 172.21.15.39:0/1653032251 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e4003b70 con 0x7f92ec15fb10 2024-09-21T07:34:08.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.436+0000 7f92e2000640 1 -- 172.21.15.39:0/1653032251 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f92e4003d10 con 0x7f92ec15fb10 2024-09-21T07:34:08.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.436+0000 7f92eb200640 1 -- 172.21.15.39:0/1653032251 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92ec13d500 con 0x7f92ec15fb10 2024-09-21T07:34:08.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.436+0000 7f92e2000640 1 -- 172.21.15.39:0/1653032251 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e4005330 con 0x7f92ec15fb10 2024-09-21T07:34:08.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.436+0000 7f92eb200640 1 -- 172.21.15.39:0/1653032251 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92ec1f9060 con 0x7f92ec15fb10 2024-09-21T07:34:08.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.437+0000 7f92eb200640 1 -- 172.21.15.39:0/1653032251 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92ec1f9b80 con 0x7f92ec15fb10 2024-09-21T07:34:08.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.439+0000 7f92e2000640 1 -- 172.21.15.39:0/1653032251 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f92e40054d0 con 0x7f92ec15fb10 2024-09-21T07:34:08.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.439+0000 7f92e2000640 1 --2- 172.21.15.39:0/1653032251 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f92c8078bc0 0x7f92c807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:08.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.440+0000 7f92e2000640 1 -- 172.21.15.39:0/1653032251 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f92e409aab0 con 0x7f92ec15fb10 2024-09-21T07:34:08.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.440+0000 7f92e9e00640 1 --2- 172.21.15.39:0/1653032251 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f92c8078bc0 0x7f92c807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:08.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.441+0000 7f92e9e00640 1 --2- 172.21.15.39:0/1653032251 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f92c8078bc0 0x7f92c807b0d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f92dc033000 tx=0x7f92dc004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:08.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.444+0000 7f92e2000640 1 -- 172.21.15.39:0/1653032251 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f92e4062e50 con 0x7f92ec15fb10 2024-09-21T07:34:08.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.641+0000 7f92eb200640 1 -- 172.21.15.39:0/1653032251 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f92ec0ce750 con 0x7f92ec15fb10 2024-09-21T07:34:08.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.643+0000 7f92e2000640 1 -- 172.21.15.39:0/1653032251 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16742 (secure 0 0 0) 0x7f92e4062750 con 0x7f92ec15fb10 2024-09-21T07:34:08.645 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:34:08.645 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":33,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:34:07.651715+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:34:05.671267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-21T07:34:06.613642+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:34:08.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.645+0000 7f92eb200640 1 -- 172.21.15.39:0/1653032251 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f92c8078bc0 msgr2=0x7f92c807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:08.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.645+0000 7f92eb200640 1 --2- 172.21.15.39:0/1653032251 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f92c8078bc0 0x7f92c807b0d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f92dc033000 tx=0x7f92dc004730 comp rx=0 tx=0).stop 2024-09-21T07:34:08.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.645+0000 7f92eb200640 1 -- 172.21.15.39:0/1653032251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92ec15fb10 msgr2=0x7f92ec13f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:08.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.645+0000 7f92eb200640 1 --2- 172.21.15.39:0/1653032251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92ec15fb10 0x7f92ec13f090 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f92e400c730 tx=0x7f92e400cb30 comp rx=0 tx=0).stop 2024-09-21T07:34:08.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.645+0000 7f92eb200640 1 -- 172.21.15.39:0/1653032251 shutdown_connections 2024-09-21T07:34:08.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.645+0000 7f92eb200640 1 --2- 172.21.15.39:0/1653032251 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f92c8078bc0 0x7f92c807b0d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:08.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.645+0000 7f92eb200640 1 --2- 172.21.15.39:0/1653032251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92ec15fb10 0x7f92ec13f090 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:08.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.645+0000 7f92eb200640 1 --2- 172.21.15.39:0/1653032251 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92ec144ce0 0x7f92ec0c9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:08.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.645+0000 7f92eb200640 1 --2- 172.21.15.39:0/1653032251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92ec0d0270 0x7f92ec1f9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:08.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.645+0000 7f92eb200640 1 -- 172.21.15.39:0/1653032251 >> 172.21.15.39:0/1653032251 conn(0x7f92ec06d870 msgr2=0x7f92ec15eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:08.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.645+0000 7f92eb200640 1 -- 172.21.15.39:0/1653032251 shutdown_connections 2024-09-21T07:34:08.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.645+0000 7f92eb200640 1 -- 172.21.15.39:0/1653032251 wait complete. 2024-09-21T07:34:08.672 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:34:08.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.896+0000 7f92f6600640 1 -- 172.21.15.39:0/2109876744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92f00b42f0 msgr2=0x7f92f00b4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:08.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.896+0000 7f92f6600640 1 --2- 172.21.15.39:0/2109876744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92f00b42f0 0x7f92f00b4700 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f92e8005fd0 tx=0x7f92e802aa90 comp rx=0 tx=0).stop 2024-09-21T07:34:08.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.896+0000 7f92f6600640 1 -- 172.21.15.39:0/2109876744 shutdown_connections 2024-09-21T07:34:08.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.896+0000 7f92f6600640 1 --2- 172.21.15.39:0/2109876744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92f00b42f0 0x7f92f00b4700 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:08.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.896+0000 7f92f6600640 1 --2- 172.21.15.39:0/2109876744 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92f00b96b0 0x7f92f00d0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:08.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.896+0000 7f92f6600640 1 -- 172.21.15.39:0/2109876744 >> 172.21.15.39:0/2109876744 conn(0x7f92f009d680 msgr2=0x7f92f00d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:08.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.896+0000 7f92f6600640 1 -- 172.21.15.39:0/2109876744 shutdown_connections 2024-09-21T07:34:08.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.896+0000 7f92f6600640 1 -- 172.21.15.39:0/2109876744 wait complete. 2024-09-21T07:34:08.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.898+0000 7f92f6600640 1 Processor -- start 2024-09-21T07:34:08.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.898+0000 7f92f6600640 1 -- start start 2024-09-21T07:34:08.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.900+0000 7f92f6600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92f00b42f0 0x7f92f015aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:08.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.900+0000 7f92f6600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92f00b96b0 0x7f92f0157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:08.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.900+0000 7f92f6600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92f00d1320 0x7f92f01610f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:08.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.900+0000 7f92f6600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f92f00b5040 con 0x7f92f00d1320 2024-09-21T07:34:08.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.900+0000 7f92f6600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f92f00b4ec0 con 0x7f92f00b96b0 2024-09-21T07:34:08.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.900+0000 7f92f6600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f92f0169090 con 0x7f92f00b42f0 2024-09-21T07:34:08.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.900+0000 7f92f5c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92f00d1320 0x7f92f01610f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:08.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.900+0000 7f92f5c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92f00d1320 0x7f92f01610f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46364/0 (socket says 172.21.15.39:46364) 2024-09-21T07:34:08.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.900+0000 7f92f5c00640 1 -- 172.21.15.39:0/1449791276 learned_addr learned my addr 172.21.15.39:0/1449791276 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:08.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.900+0000 7f92f5200640 1 --2- 172.21.15.39:0/1449791276 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92f00b42f0 0x7f92f015aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:08.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.900+0000 7f92f5c00640 1 -- 172.21.15.39:0/1449791276 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92f00b42f0 msgr2=0x7f92f015aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:08.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.900+0000 7f92f5c00640 1 --2- 172.21.15.39:0/1449791276 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92f00b42f0 0x7f92f015aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:08.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.900+0000 7f92e7e00640 1 --2- 172.21.15.39:0/1449791276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92f00b96b0 0x7f92f0157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:08.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.900+0000 7f92f5c00640 1 -- 172.21.15.39:0/1449791276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92f00b96b0 msgr2=0x7f92f0157d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:08.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.900+0000 7f92f5c00640 1 --2- 172.21.15.39:0/1449791276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92f00b96b0 0x7f92f0157d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:08.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.900+0000 7f92f5c00640 1 -- 172.21.15.39:0/1449791276 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f015b310 con 0x7f92f00d1320 2024-09-21T07:34:08.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.900+0000 7f92f5200640 1 --2- 172.21.15.39:0/1449791276 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92f00b42f0 0x7f92f015aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:34:08.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.900+0000 7f92e7e00640 1 --2- 172.21.15.39:0/1449791276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92f00b96b0 0x7f92f0157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:34:08.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.901+0000 7f92f5c00640 1 --2- 172.21.15.39:0/1449791276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92f00d1320 0x7f92f01610f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f92e8006010 tx=0x7f92e8003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:08.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.901+0000 7f92ed600640 1 -- 172.21.15.39:0/1449791276 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e8044070 con 0x7f92f00d1320 2024-09-21T07:34:08.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.901+0000 7f92ed600640 1 -- 172.21.15.39:0/1449791276 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f92e8004040 con 0x7f92f00d1320 2024-09-21T07:34:08.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.901+0000 7f92f6600640 1 -- 172.21.15.39:0/1449791276 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92f015b5f0 con 0x7f92f00d1320 2024-09-21T07:34:08.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.901+0000 7f92ed600640 1 -- 172.21.15.39:0/1449791276 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e8004430 con 0x7f92f00d1320 2024-09-21T07:34:08.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.901+0000 7f92f6600640 1 -- 172.21.15.39:0/1449791276 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92f0166d20 con 0x7f92f00d1320 2024-09-21T07:34:08.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.902+0000 7f92f6600640 1 -- 172.21.15.39:0/1449791276 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92f00d3650 con 0x7f92f00d1320 2024-09-21T07:34:08.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.903+0000 7f92ed600640 1 -- 172.21.15.39:0/1449791276 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f92e8047070 con 0x7f92f00d1320 2024-09-21T07:34:08.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.905+0000 7f92ed600640 1 --2- 172.21.15.39:0/1449791276 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f92cc078b90 0x7f92cc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:08.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.905+0000 7f92ed600640 1 -- 172.21.15.39:0/1449791276 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f92e80bbbc0 con 0x7f92f00d1320 2024-09-21T07:34:08.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.905+0000 7f92f5200640 1 --2- 172.21.15.39:0/1449791276 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f92cc078b90 0x7f92cc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:08.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.906+0000 7f92f5200640 1 --2- 172.21.15.39:0/1449791276 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f92cc078b90 0x7f92cc07b0a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f92e0009aa0 tx=0x7f92e0005f30 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:08.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:08.910+0000 7f92ed600640 1 -- 172.21.15.39:0/1449791276 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f92f00d3650 con 0x7f92f00d1320 2024-09-21T07:34:09.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.105+0000 7f92f6600640 1 -- 172.21.15.39:0/1449791276 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f92f00b4700 con 0x7f92f00d1320 2024-09-21T07:34:09.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.107+0000 7f92ed600640 1 -- 172.21.15.39:0/1449791276 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16742 (secure 0 0 0) 0x7f92e8032030 con 0x7f92f00d1320 2024-09-21T07:34:09.109 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:34:09.109 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":33,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:34:07.651715+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:34:05.671267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-21T07:34:06.613642+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:34:09.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.109+0000 7f92f6600640 1 -- 172.21.15.39:0/1449791276 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f92cc078b90 msgr2=0x7f92cc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:09.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.109+0000 7f92f6600640 1 --2- 172.21.15.39:0/1449791276 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f92cc078b90 0x7f92cc07b0a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f92e0009aa0 tx=0x7f92e0005f30 comp rx=0 tx=0).stop 2024-09-21T07:34:09.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.109+0000 7f92f6600640 1 -- 172.21.15.39:0/1449791276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92f00d1320 msgr2=0x7f92f01610f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:09.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.109+0000 7f92f6600640 1 --2- 172.21.15.39:0/1449791276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92f00d1320 0x7f92f01610f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f92e8006010 tx=0x7f92e8003ce0 comp rx=0 tx=0).stop 2024-09-21T07:34:09.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.109+0000 7f92f6600640 1 -- 172.21.15.39:0/1449791276 shutdown_connections 2024-09-21T07:34:09.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.109+0000 7f92f6600640 1 --2- 172.21.15.39:0/1449791276 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f92cc078b90 0x7f92cc07b0a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:09.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.109+0000 7f92f6600640 1 --2- 172.21.15.39:0/1449791276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92f00d1320 0x7f92f01610f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:09.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.109+0000 7f92f6600640 1 --2- 172.21.15.39:0/1449791276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92f00b96b0 0x7f92f0157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:09.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.109+0000 7f92f6600640 1 --2- 172.21.15.39:0/1449791276 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92f00b42f0 0x7f92f015aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:09.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.109+0000 7f92f6600640 1 -- 172.21.15.39:0/1449791276 >> 172.21.15.39:0/1449791276 conn(0x7f92f009d680 msgr2=0x7f92f00ce380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:09.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.109+0000 7f92f6600640 1 -- 172.21.15.39:0/1449791276 shutdown_connections 2024-09-21T07:34:09.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.109+0000 7f92f6600640 1 -- 172.21.15.39:0/1449791276 wait complete. 2024-09-21T07:34:09.136 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T07:34:09.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.368+0000 7ff9c3e00640 1 -- 172.21.15.39:0/1272026743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9c40cf540 msgr2=0x7ff9c406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:09.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.368+0000 7ff9c3e00640 1 --2- 172.21.15.39:0/1272026743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9c40cf540 0x7ff9c406ee70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac005fd0 tx=0x7ff9ac02a540 comp rx=0 tx=0).stop 2024-09-21T07:34:09.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.368+0000 7ff9c3e00640 1 -- 172.21.15.39:0/1272026743 shutdown_connections 2024-09-21T07:34:09.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.368+0000 7ff9c3e00640 1 --2- 172.21.15.39:0/1272026743 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff9c40cf950 0x7ff9c406f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:09.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.368+0000 7ff9c3e00640 1 --2- 172.21.15.39:0/1272026743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9c40cf540 0x7ff9c406ee70 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:09.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.368+0000 7ff9c3e00640 1 -- 172.21.15.39:0/1272026743 >> 172.21.15.39:0/1272026743 conn(0x7ff9c406d410 msgr2=0x7ff9c40b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:09.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.368+0000 7ff9c3e00640 1 -- 172.21.15.39:0/1272026743 shutdown_connections 2024-09-21T07:34:09.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.368+0000 7ff9c3e00640 1 -- 172.21.15.39:0/1272026743 wait complete. 2024-09-21T07:34:09.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.370+0000 7ff9c3e00640 1 Processor -- start 2024-09-21T07:34:09.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.370+0000 7ff9c3e00640 1 -- start start 2024-09-21T07:34:09.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.372+0000 7ff9c3e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9c40cf540 0x7ff9c41f93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:09.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.372+0000 7ff9c3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9c40cf950 0x7ff9c413c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:09.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.372+0000 7ff9c3e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff9c415ca50 0x7ff9c4073c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:09.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.372+0000 7ff9c3e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c415ddd0 con 0x7ff9c40cf950 2024-09-21T07:34:09.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.372+0000 7ff9c3e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c415dc50 con 0x7ff9c40cf540 2024-09-21T07:34:09.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.372+0000 7ff9c3e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7ff9c415df50 con 0x7ff9c415ca50 2024-09-21T07:34:09.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.372+0000 7ff9bbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9c40cf950 0x7ff9c413c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:09.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.372+0000 7ff9bbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9c40cf950 0x7ff9c413c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46376/0 (socket says 172.21.15.39:46376) 2024-09-21T07:34:09.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.372+0000 7ff9bbe00640 1 -- 172.21.15.39:0/873465825 learned_addr learned my addr 172.21.15.39:0/873465825 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:09.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.372+0000 7ff9c2a00640 1 --2- 172.21.15.39:0/873465825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9c40cf540 0x7ff9c41f93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:09.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.372+0000 7ff9c3400640 1 --2- 172.21.15.39:0/873465825 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff9c415ca50 0x7ff9c4073c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:09.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.373+0000 7ff9bbe00640 1 -- 172.21.15.39:0/873465825 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff9c415ca50 msgr2=0x7ff9c4073c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:09.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.373+0000 7ff9bbe00640 1 --2- 172.21.15.39:0/873465825 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff9c415ca50 0x7ff9c4073c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:09.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.373+0000 7ff9bbe00640 1 -- 172.21.15.39:0/873465825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9c40cf540 msgr2=0x7ff9c41f93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:09.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.373+0000 7ff9bbe00640 1 --2- 172.21.15.39:0/873465825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9c40cf540 0x7ff9c41f93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:09.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.373+0000 7ff9bbe00640 1 -- 172.21.15.39:0/873465825 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c40b1ba0 con 0x7ff9c40cf950 2024-09-21T07:34:09.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.373+0000 7ff9bbe00640 1 --2- 172.21.15.39:0/873465825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9c40cf950 0x7ff9c413c980 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac034040 tx=0x7ff9ac003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:09.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.373+0000 7ff9bb400640 1 -- 172.21.15.39:0/873465825 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac03e030 con 0x7ff9c40cf950 2024-09-21T07:34:09.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.373+0000 7ff9bb400640 1 -- 172.21.15.39:0/873465825 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9ac0047a0 con 0x7ff9c40cf950 2024-09-21T07:34:09.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.373+0000 7ff9c3e00640 1 -- 172.21.15.39:0/873465825 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9c40b0c10 con 0x7ff9c40cf950 2024-09-21T07:34:09.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.373+0000 7ff9bb400640 1 -- 172.21.15.39:0/873465825 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac035040 con 0x7ff9c40cf950 2024-09-21T07:34:09.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.373+0000 7ff9c3e00640 1 -- 172.21.15.39:0/873465825 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9c40b11d0 con 0x7ff9c40cf950 2024-09-21T07:34:09.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.374+0000 7ff9c3e00640 1 -- 172.21.15.39:0/873465825 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9c40b1910 con 0x7ff9c40cf950 2024-09-21T07:34:09.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.376+0000 7ff9bb400640 1 -- 172.21.15.39:0/873465825 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff9ac041030 con 0x7ff9c40cf950 2024-09-21T07:34:09.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.377+0000 7ff9bb400640 1 --2- 172.21.15.39:0/873465825 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff9a0078b90 0x7ff9a007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:09.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.377+0000 7ff9bb400640 1 -- 172.21.15.39:0/873465825 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7ff9ac0bb5c0 con 0x7ff9c40cf950 2024-09-21T07:34:09.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.377+0000 7ff9c2a00640 1 --2- 172.21.15.39:0/873465825 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff9a0078b90 0x7ff9a007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:09.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.378+0000 7ff9c2a00640 1 --2- 172.21.15.39:0/873465825 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff9a0078b90 0x7ff9a007b0a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4000f60 tx=0x7ff9b4009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:09.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.381+0000 7ff9bb400640 1 -- 172.21.15.39:0/873465825 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff9ac083310 con 0x7ff9c40cf950 2024-09-21T07:34:09.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:09.618+0000 7ff9c3e00640 1 -- 172.21.15.39:0/873465825 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff9c415ebc0 con 0x7ff9c40cf950 2024-09-21T07:34:10.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.168+0000 7ff9bb400640 1 -- 172.21.15.39:0/873465825 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7ff9ac087d40 con 0x7ff9c40cf950 2024-09-21T07:34:10.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.170+0000 7ff9c3e00640 1 -- 172.21.15.39:0/873465825 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff9a0078b90 msgr2=0x7ff9a007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:10.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.170+0000 7ff9c3e00640 1 --2- 172.21.15.39:0/873465825 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff9a0078b90 0x7ff9a007b0a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4000f60 tx=0x7ff9b4009040 comp rx=0 tx=0).stop 2024-09-21T07:34:10.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.170+0000 7ff9c3e00640 1 -- 172.21.15.39:0/873465825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9c40cf950 msgr2=0x7ff9c413c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:10.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.170+0000 7ff9c3e00640 1 --2- 172.21.15.39:0/873465825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9c40cf950 0x7ff9c413c980 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac034040 tx=0x7ff9ac003970 comp rx=0 tx=0).stop 2024-09-21T07:34:10.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.171+0000 7ff9c3e00640 1 -- 172.21.15.39:0/873465825 shutdown_connections 2024-09-21T07:34:10.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.171+0000 7ff9c3e00640 1 --2- 172.21.15.39:0/873465825 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff9a0078b90 0x7ff9a007b0a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:10.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.171+0000 7ff9c3e00640 1 --2- 172.21.15.39:0/873465825 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff9c415ca50 0x7ff9c4073c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:10.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.171+0000 7ff9c3e00640 1 --2- 172.21.15.39:0/873465825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9c40cf950 0x7ff9c413c980 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:10.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.171+0000 7ff9c3e00640 1 --2- 172.21.15.39:0/873465825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9c40cf540 0x7ff9c41f93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:10.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.171+0000 7ff9c3e00640 1 -- 172.21.15.39:0/873465825 >> 172.21.15.39:0/873465825 conn(0x7ff9c406d410 msgr2=0x7ff9c40af2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:10.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.171+0000 7ff9c3e00640 1 -- 172.21.15.39:0/873465825 shutdown_connections 2024-09-21T07:34:10.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.171+0000 7ff9c3e00640 1 -- 172.21.15.39:0/873465825 wait complete. 2024-09-21T07:34:10.196 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T07:34:10.196 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T07:34:10.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.425+0000 7faf20c00640 1 -- 172.21.15.39:0/1734350058 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf1c0cc1e0 msgr2=0x7faf1c0af3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:10.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.425+0000 7faf20c00640 1 --2- 172.21.15.39:0/1734350058 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf1c0cc1e0 0x7faf1c0af3c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7faf0c005fd0 tx=0x7faf0c02a540 comp rx=0 tx=0).stop 2024-09-21T07:34:10.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.426+0000 7faf20c00640 1 -- 172.21.15.39:0/1734350058 shutdown_connections 2024-09-21T07:34:10.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.426+0000 7faf20c00640 1 --2- 172.21.15.39:0/1734350058 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faf1c15d8a0 0x7faf1c0b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:10.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.426+0000 7faf20c00640 1 --2- 172.21.15.39:0/1734350058 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf1c0cc1e0 0x7faf1c0af3c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:10.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.426+0000 7faf20c00640 1 -- 172.21.15.39:0/1734350058 >> 172.21.15.39:0/1734350058 conn(0x7faf1c06d7d0 msgr2=0x7faf1c0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:10.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.426+0000 7faf20c00640 1 -- 172.21.15.39:0/1734350058 shutdown_connections 2024-09-21T07:34:10.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.426+0000 7faf20c00640 1 -- 172.21.15.39:0/1734350058 wait complete. 2024-09-21T07:34:10.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.428+0000 7faf20c00640 1 Processor -- start 2024-09-21T07:34:10.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.428+0000 7faf20c00640 1 -- start start 2024-09-21T07:34:10.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.429+0000 7faf20c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faf1c0cc1e0 0x7faf1c0acf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:10.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.429+0000 7faf20c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf1c0ccb50 0x7faf1c09cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:10.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.429+0000 7faf20c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf1c15d8a0 0x7faf1c0adb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:10.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.429+0000 7faf20c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7faf1c153520 con 0x7faf1c15d8a0 2024-09-21T07:34:10.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.429+0000 7faf20c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7faf1c1533a0 con 0x7faf1c0ccb50 2024-09-21T07:34:10.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.429+0000 7faf20c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7faf1c1536a0 con 0x7faf1c0cc1e0 2024-09-21T07:34:10.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.430+0000 7faf1a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf1c15d8a0 0x7faf1c0adb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:10.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.430+0000 7faf1a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf1c15d8a0 0x7faf1c0adb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46408/0 (socket says 172.21.15.39:46408) 2024-09-21T07:34:10.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.430+0000 7faf1a800640 1 -- 172.21.15.39:0/1756665989 learned_addr learned my addr 172.21.15.39:0/1756665989 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:10.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.430+0000 7faf19e00640 1 --2- 172.21.15.39:0/1756665989 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faf1c0cc1e0 0x7faf1c0acf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:10.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.430+0000 7faf11400640 1 --2- 172.21.15.39:0/1756665989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf1c0ccb50 0x7faf1c09cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:10.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.430+0000 7faf1a800640 1 -- 172.21.15.39:0/1756665989 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faf1c0cc1e0 msgr2=0x7faf1c0acf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:10.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.430+0000 7faf1a800640 1 --2- 172.21.15.39:0/1756665989 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faf1c0cc1e0 0x7faf1c0acf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:10.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.430+0000 7faf1a800640 1 -- 172.21.15.39:0/1756665989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf1c0ccb50 msgr2=0x7faf1c09cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:10.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.430+0000 7faf1a800640 1 --2- 172.21.15.39:0/1756665989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf1c0ccb50 0x7faf1c09cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:10.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.430+0000 7faf1a800640 1 -- 172.21.15.39:0/1756665989 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf1c09d080 con 0x7faf1c15d8a0 2024-09-21T07:34:10.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.431+0000 7faf1a800640 1 --2- 172.21.15.39:0/1756665989 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf1c15d8a0 0x7faf1c0adb70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7faf1400ba10 tx=0x7faf1400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:10.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.431+0000 7faf12a00640 1 -- 172.21.15.39:0/1756665989 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf14019070 con 0x7faf1c15d8a0 2024-09-21T07:34:10.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.431+0000 7faf12a00640 1 -- 172.21.15.39:0/1756665989 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf1400cc30 con 0x7faf1c15d8a0 2024-09-21T07:34:10.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.431+0000 7faf12a00640 1 -- 172.21.15.39:0/1756665989 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf140146a0 con 0x7faf1c15d8a0 2024-09-21T07:34:10.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.431+0000 7faf20c00640 1 -- 172.21.15.39:0/1756665989 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf1c1f7700 con 0x7faf1c15d8a0 2024-09-21T07:34:10.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.431+0000 7faf20c00640 1 -- 172.21.15.39:0/1756665989 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf1c1f79e0 con 0x7faf1c15d8a0 2024-09-21T07:34:10.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.432+0000 7faf20c00640 1 -- 172.21.15.39:0/1756665989 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf1c0b33f0 con 0x7faf1c15d8a0 2024-09-21T07:34:10.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.433+0000 7faf12a00640 1 -- 172.21.15.39:0/1756665989 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7faf1401c070 con 0x7faf1c15d8a0 2024-09-21T07:34:10.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.434+0000 7faf12a00640 1 --2- 172.21.15.39:0/1756665989 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faef8078b90 0x7faef807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:10.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.434+0000 7faf12a00640 1 -- 172.21.15.39:0/1756665989 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7faf1409aa80 con 0x7faf1c15d8a0 2024-09-21T07:34:10.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.435+0000 7faf19e00640 1 --2- 172.21.15.39:0/1756665989 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faef8078b90 0x7faef807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:10.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.435+0000 7faf19e00640 1 --2- 172.21.15.39:0/1756665989 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faef8078b90 0x7faef807b0a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7faf0c033000 tx=0x7faf0c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:10.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.439+0000 7faf12a00640 1 -- 172.21.15.39:0/1756665989 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7faf1c0b33f0 con 0x7faf1c15d8a0 2024-09-21T07:34:10.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:10.673+0000 7faf20c00640 1 -- 172.21.15.39:0/1756665989 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7faf1c09dc80 con 0x7faf1c15d8a0 2024-09-21T07:34:11.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.181+0000 7faf12a00640 1 -- 172.21.15.39:0/1756665989 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7faf1c09dc80 con 0x7faf1c15d8a0 2024-09-21T07:34:11.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.184+0000 7faf20c00640 1 -- 172.21.15.39:0/1756665989 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faef8078b90 msgr2=0x7faef807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:11.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.184+0000 7faf20c00640 1 --2- 172.21.15.39:0/1756665989 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faef8078b90 0x7faef807b0a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7faf0c033000 tx=0x7faf0c004730 comp rx=0 tx=0).stop 2024-09-21T07:34:11.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.184+0000 7faf20c00640 1 -- 172.21.15.39:0/1756665989 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf1c15d8a0 msgr2=0x7faf1c0adb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:11.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.184+0000 7faf20c00640 1 --2- 172.21.15.39:0/1756665989 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf1c15d8a0 0x7faf1c0adb70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7faf1400ba10 tx=0x7faf1400bed0 comp rx=0 tx=0).stop 2024-09-21T07:34:11.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.184+0000 7faf20c00640 1 -- 172.21.15.39:0/1756665989 shutdown_connections 2024-09-21T07:34:11.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.184+0000 7faf20c00640 1 --2- 172.21.15.39:0/1756665989 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faef8078b90 0x7faef807b0a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:11.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.184+0000 7faf20c00640 1 --2- 172.21.15.39:0/1756665989 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf1c15d8a0 0x7faf1c0adb70 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:11.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.184+0000 7faf20c00640 1 --2- 172.21.15.39:0/1756665989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf1c0ccb50 0x7faf1c09cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:11.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.184+0000 7faf20c00640 1 --2- 172.21.15.39:0/1756665989 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faf1c0cc1e0 0x7faf1c0acf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:11.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.184+0000 7faf20c00640 1 -- 172.21.15.39:0/1756665989 >> 172.21.15.39:0/1756665989 conn(0x7faf1c06d7d0 msgr2=0x7faf1c1527d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:11.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.185+0000 7faf20c00640 1 -- 172.21.15.39:0/1756665989 shutdown_connections 2024-09-21T07:34:11.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.185+0000 7faf20c00640 1 -- 172.21.15.39:0/1756665989 wait complete. 2024-09-21T07:34:11.214 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T07:34:11.214 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:34:11.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.440+0000 7f0c9f200640 1 -- 172.21.15.39:0/2277516936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c980b9bd0 msgr2=0x7f0c980b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:11.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.440+0000 7f0c9f200640 1 --2- 172.21.15.39:0/2277516936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c980b9bd0 0x7f0c980b71a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c005fd0 tx=0x7f0c8c02a540 comp rx=0 tx=0).stop 2024-09-21T07:34:11.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.440+0000 7f0c9f200640 1 -- 172.21.15.39:0/2277516936 shutdown_connections 2024-09-21T07:34:11.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.440+0000 7f0c9f200640 1 --2- 172.21.15.39:0/2277516936 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c980b8fd0 0x7f0c980b93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:11.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.440+0000 7f0c9f200640 1 --2- 172.21.15.39:0/2277516936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c980b9bd0 0x7f0c980b71a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:11.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.440+0000 7f0c9f200640 1 -- 172.21.15.39:0/2277516936 >> 172.21.15.39:0/2277516936 conn(0x7f0c9809d6c0 msgr2=0x7f0c980d4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:11.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.440+0000 7f0c9f200640 1 -- 172.21.15.39:0/2277516936 shutdown_connections 2024-09-21T07:34:11.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.440+0000 7f0c9f200640 1 -- 172.21.15.39:0/2277516936 wait complete. 2024-09-21T07:34:11.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.442+0000 7f0c9f200640 1 Processor -- start 2024-09-21T07:34:11.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.442+0000 7f0c9f200640 1 -- start start 2024-09-21T07:34:11.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.444+0000 7f0c9f200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c980b8fd0 0x7f0c98159a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:11.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.444+0000 7f0c9f200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c980b9bd0 0x7f0c980c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:11.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.444+0000 7f0c9f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c98162440 0x7f0c980d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:11.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.444+0000 7f0c9f200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0c980b5360 con 0x7f0c98162440 2024-09-21T07:34:11.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.444+0000 7f0c9f200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0c980b51e0 con 0x7f0c980b8fd0 2024-09-21T07:34:11.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.444+0000 7f0c9f200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f0c980b54e0 con 0x7f0c980b9bd0 2024-09-21T07:34:11.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.444+0000 7f0c9e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c98162440 0x7f0c980d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:11.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.444+0000 7f0c9e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c98162440 0x7f0c980d02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46424/0 (socket says 172.21.15.39:46424) 2024-09-21T07:34:11.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.444+0000 7f0c9e800640 1 -- 172.21.15.39:0/1497425189 learned_addr learned my addr 172.21.15.39:0/1497425189 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:11.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.444+0000 7f0c9de00640 1 --2- 172.21.15.39:0/1497425189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c980b8fd0 0x7f0c98159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:11.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.444+0000 7f0c9d400640 1 --2- 172.21.15.39:0/1497425189 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c980b9bd0 0x7f0c980c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:11.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.445+0000 7f0c9e800640 1 -- 172.21.15.39:0/1497425189 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c980b9bd0 msgr2=0x7f0c980c7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:11.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.445+0000 7f0c9e800640 1 --2- 172.21.15.39:0/1497425189 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c980b9bd0 0x7f0c980c7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:11.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.445+0000 7f0c9e800640 1 -- 172.21.15.39:0/1497425189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c980b8fd0 msgr2=0x7f0c98159a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:11.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.445+0000 7f0c9e800640 1 --2- 172.21.15.39:0/1497425189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c980b8fd0 0x7f0c98159a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:11.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.445+0000 7f0c9e800640 1 -- 172.21.15.39:0/1497425189 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c980c83c0 con 0x7f0c98162440 2024-09-21T07:34:11.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.445+0000 7f0c9d400640 1 --2- 172.21.15.39:0/1497425189 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c980b9bd0 0x7f0c980c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:34:11.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.445+0000 7f0c9e800640 1 --2- 172.21.15.39:0/1497425189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c98162440 0x7f0c980d02d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0c9400ea40 tx=0x7f0c9400ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:11.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.445+0000 7f0c92000640 1 -- 172.21.15.39:0/1497425189 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9400ce40 con 0x7f0c98162440 2024-09-21T07:34:11.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.445+0000 7f0c92000640 1 -- 172.21.15.39:0/1497425189 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c94004c10 con 0x7f0c98162440 2024-09-21T07:34:11.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.445+0000 7f0c9f200640 1 -- 172.21.15.39:0/1497425189 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c980c8e00 con 0x7f0c98162440 2024-09-21T07:34:11.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.445+0000 7f0c92000640 1 -- 172.21.15.39:0/1497425189 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c94005020 con 0x7f0c98162440 2024-09-21T07:34:11.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.445+0000 7f0c9f200640 1 -- 172.21.15.39:0/1497425189 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c980c8b20 con 0x7f0c98162440 2024-09-21T07:34:11.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.447+0000 7f0c9f200640 1 -- 172.21.15.39:0/1497425189 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c98161d10 con 0x7f0c98162440 2024-09-21T07:34:11.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.448+0000 7f0c92000640 1 -- 172.21.15.39:0/1497425189 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0c940051c0 con 0x7f0c98162440 2024-09-21T07:34:11.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.449+0000 7f0c92000640 1 --2- 172.21.15.39:0/1497425189 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0c7c078bc0 0x7f0c7c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:11.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.449+0000 7f0c92000640 1 -- 172.21.15.39:0/1497425189 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f0c9409bbd0 con 0x7f0c98162440 2024-09-21T07:34:11.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.449+0000 7f0c9de00640 1 --2- 172.21.15.39:0/1497425189 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0c7c078bc0 0x7f0c7c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:11.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.450+0000 7f0c9de00640 1 --2- 172.21.15.39:0/1497425189 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0c7c078bc0 0x7f0c7c07b0d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c033000 tx=0x7f0c8c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:11.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.453+0000 7f0c92000640 1 -- 172.21.15.39:0/1497425189 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0c98161d10 con 0x7f0c98162440 2024-09-21T07:34:11.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.692+0000 7f0c9f200640 1 -- 172.21.15.39:0/1497425189 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c980b5a00 con 0x7f0c98162440 2024-09-21T07:34:11.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.692+0000 7f0c92000640 1 -- 172.21.15.39:0/1497425189 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5229 (secure 0 0 0) 0x7f0c94063f70 con 0x7f0c98162440 2024-09-21T07:34:11.694 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:34:11.694 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":17,"btime":"2024-09-21T07:34:11:169975+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2438395645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2438395645},{"type":"v1","addr":"172.21.15.148:6837","nonce":2438395645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4706,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3668947757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3668947757},{"type":"v1","addr":"172.21.15.39:6835","nonce":3668947757}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4870,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2800764560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2800764560},{"type":"v1","addr":"172.21.15.148:6835","nonce":2800764560}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:34:07.651669+0000","modified":"2024-09-21T07:34:10.675070+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4888":{"gid":4888,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/3524592265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3524592265},{"type":"v1","addr":"172.21.15.39:6834","nonce":3524592265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4888,"qdb_cluster":[4888]},"id":2}]} 2024-09-21T07:34:11.695 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 17 2024-09-21T07:34:11.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.695+0000 7f0c9f200640 1 -- 172.21.15.39:0/1497425189 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0c7c078bc0 msgr2=0x7f0c7c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:11.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.695+0000 7f0c9f200640 1 --2- 172.21.15.39:0/1497425189 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0c7c078bc0 0x7f0c7c07b0d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c033000 tx=0x7f0c8c004730 comp rx=0 tx=0).stop 2024-09-21T07:34:11.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.695+0000 7f0c9f200640 1 -- 172.21.15.39:0/1497425189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c98162440 msgr2=0x7f0c980d02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:11.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.695+0000 7f0c9f200640 1 --2- 172.21.15.39:0/1497425189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c98162440 0x7f0c980d02d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0c9400ea40 tx=0x7f0c9400ef00 comp rx=0 tx=0).stop 2024-09-21T07:34:11.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.695+0000 7f0c9f200640 1 -- 172.21.15.39:0/1497425189 shutdown_connections 2024-09-21T07:34:11.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.695+0000 7f0c9f200640 1 --2- 172.21.15.39:0/1497425189 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0c7c078bc0 0x7f0c7c07b0d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:11.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.695+0000 7f0c9f200640 1 --2- 172.21.15.39:0/1497425189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c98162440 0x7f0c980d02d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:11.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.695+0000 7f0c9f200640 1 --2- 172.21.15.39:0/1497425189 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c980b9bd0 0x7f0c980c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:11.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.695+0000 7f0c9f200640 1 --2- 172.21.15.39:0/1497425189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c980b8fd0 0x7f0c98159a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:11.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.695+0000 7f0c9f200640 1 -- 172.21.15.39:0/1497425189 >> 172.21.15.39:0/1497425189 conn(0x7f0c9809d6c0 msgr2=0x7f0c98161600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:11.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.695+0000 7f0c9f200640 1 -- 172.21.15.39:0/1497425189 shutdown_connections 2024-09-21T07:34:11.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.695+0000 7f0c9f200640 1 -- 172.21.15.39:0/1497425189 wait complete. 2024-09-21T07:34:11.719 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:34:11.933 INFO:tasks.ceph.mds.a.smithi039.stderr:2024-09-21T07:34:11.931+0000 7f28e3e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:34:11.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.948+0000 7efc5f400640 1 -- 172.21.15.39:0/5093385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc40003a00 msgr2=0x7efc40003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:11.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.948+0000 7efc5f400640 1 --2- 172.21.15.39:0/5093385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc40003a00 0x7efc40003e70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7efc4c009b50 tx=0x7efc40005830 comp rx=0 tx=0).stop 2024-09-21T07:34:11.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.948+0000 7efc5d400640 1 -- 172.21.15.39:0/5093385 reap_dead start 2024-09-21T07:34:11.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.948+0000 7efc5f400640 1 -- 172.21.15.39:0/5093385 shutdown_connections 2024-09-21T07:34:11.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.948+0000 7efc5f400640 1 -- 172.21.15.39:0/5093385 >> 172.21.15.39:0/5093385 conn(0x7efc580cdcc0 msgr2=0x7efc580ce0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:11.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.948+0000 7efc5f400640 1 -- 172.21.15.39:0/5093385 shutdown_connections 2024-09-21T07:34:11.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.948+0000 7efc5f400640 1 -- 172.21.15.39:0/5093385 wait complete. 2024-09-21T07:34:11.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.951+0000 7efc5f400640 1 Processor -- start 2024-09-21T07:34:11.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.951+0000 7efc5f400640 1 -- start start 2024-09-21T07:34:11.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.952+0000 7efc5f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc40003a00 0x7efc581f50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:11.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.952+0000 7efc5f400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc5815a9f0 0x7efc580bba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:11.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.952+0000 7efc5f400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc580bbf40 0x7efc580b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:11.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.952+0000 7efc5f400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efc580b38e0 con 0x7efc40003a00 2024-09-21T07:34:11.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.952+0000 7efc5f400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efc580b3760 con 0x7efc5815a9f0 2024-09-21T07:34:11.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.952+0000 7efc5f400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7efc580b1b00 con 0x7efc580bbf40 2024-09-21T07:34:11.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.953+0000 7efc5ca00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc40003a00 0x7efc581f50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:11.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.953+0000 7efc5ca00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc40003a00 0x7efc581f50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46444/0 (socket says 172.21.15.39:46444) 2024-09-21T07:34:11.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.953+0000 7efc5ca00640 1 -- 172.21.15.39:0/125376598 learned_addr learned my addr 172.21.15.39:0/125376598 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:11.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.953+0000 7efc5d400640 1 --2- 172.21.15.39:0/125376598 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc580bbf40 0x7efc580b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:11.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.953+0000 7efc57e00640 1 --2- 172.21.15.39:0/125376598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc5815a9f0 0x7efc580bba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:11.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.953+0000 7efc5ca00640 1 -- 172.21.15.39:0/125376598 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc580bbf40 msgr2=0x7efc580b0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:11.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.953+0000 7efc5ca00640 1 --2- 172.21.15.39:0/125376598 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc580bbf40 0x7efc580b0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:11.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.953+0000 7efc5ca00640 1 -- 172.21.15.39:0/125376598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc5815a9f0 msgr2=0x7efc580bba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:11.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.953+0000 7efc5ca00640 1 --2- 172.21.15.39:0/125376598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc5815a9f0 0x7efc580bba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:11.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.953+0000 7efc5ca00640 1 -- 172.21.15.39:0/125376598 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc580bc350 con 0x7efc40003a00 2024-09-21T07:34:11.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.953+0000 7efc5ca00640 1 --2- 172.21.15.39:0/125376598 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc40003a00 0x7efc581f50b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efc4c036040 tx=0x7efc4c033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:11.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.954+0000 7efc54c00640 1 -- 172.21.15.39:0/125376598 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4c038050 con 0x7efc40003a00 2024-09-21T07:34:11.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.954+0000 7efc54c00640 1 -- 172.21.15.39:0/125376598 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc4c039040 con 0x7efc40003a00 2024-09-21T07:34:11.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.954+0000 7efc5f400640 1 -- 172.21.15.39:0/125376598 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc5809e400 con 0x7efc40003a00 2024-09-21T07:34:11.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.954+0000 7efc54c00640 1 -- 172.21.15.39:0/125376598 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4c033c50 con 0x7efc40003a00 2024-09-21T07:34:11.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.954+0000 7efc5f400640 1 -- 172.21.15.39:0/125376598 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc5809c7e0 con 0x7efc40003a00 2024-09-21T07:34:11.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.955+0000 7efc5f400640 1 -- 172.21.15.39:0/125376598 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc580702d0 con 0x7efc40003a00 2024-09-21T07:34:11.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.956+0000 7efc54c00640 1 -- 172.21.15.39:0/125376598 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7efc4c046020 con 0x7efc40003a00 2024-09-21T07:34:11.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.957+0000 7efc54c00640 1 --2- 172.21.15.39:0/125376598 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efc38078b90 0x7efc3807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:11.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.957+0000 7efc54c00640 1 -- 172.21.15.39:0/125376598 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7efc4c0bb710 con 0x7efc40003a00 2024-09-21T07:34:11.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.958+0000 7efc57e00640 1 --2- 172.21.15.39:0/125376598 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efc38078b90 0x7efc3807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:11.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.959+0000 7efc57e00640 1 --2- 172.21.15.39:0/125376598 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efc38078b90 0x7efc3807b0a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efc580b0100 tx=0x7efc44005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:11.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:11.962+0000 7efc54c00640 1 -- 172.21.15.39:0/125376598 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7efc4c04d020 con 0x7efc40003a00 2024-09-21T07:34:12.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.162+0000 7efc5f400640 1 -- 172.21.15.39:0/125376598 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc5806ef00 con 0x7efc40003a00 2024-09-21T07:34:12.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.164+0000 7efc54c00640 1 -- 172.21.15.39:0/125376598 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16729 (secure 0 0 0) 0x7efc580bc350 con 0x7efc40003a00 2024-09-21T07:34:12.166 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:34:12.166 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":34,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:34:09.149842+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:34:05.671267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-21T07:34:06.613642+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:34:12.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.166+0000 7efc5f400640 1 -- 172.21.15.39:0/125376598 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efc38078b90 msgr2=0x7efc3807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:12.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.166+0000 7efc5f400640 1 --2- 172.21.15.39:0/125376598 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efc38078b90 0x7efc3807b0a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efc580b0100 tx=0x7efc44005f90 comp rx=0 tx=0).stop 2024-09-21T07:34:12.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.166+0000 7efc5f400640 1 -- 172.21.15.39:0/125376598 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc40003a00 msgr2=0x7efc581f50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:12.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.166+0000 7efc5f400640 1 --2- 172.21.15.39:0/125376598 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc40003a00 0x7efc581f50b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efc4c036040 tx=0x7efc4c033670 comp rx=0 tx=0).stop 2024-09-21T07:34:12.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.166+0000 7efc5f400640 1 -- 172.21.15.39:0/125376598 shutdown_connections 2024-09-21T07:34:12.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.166+0000 7efc5f400640 1 --2- 172.21.15.39:0/125376598 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efc38078b90 0x7efc3807b0a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:12.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.166+0000 7efc5f400640 1 --2- 172.21.15.39:0/125376598 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc580bbf40 0x7efc580b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:12.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.166+0000 7efc5f400640 1 --2- 172.21.15.39:0/125376598 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc5815a9f0 0x7efc580bba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:12.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.166+0000 7efc5f400640 1 --2- 172.21.15.39:0/125376598 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc40003a00 0x7efc581f50b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:12.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.166+0000 7efc5f400640 1 -- 172.21.15.39:0/125376598 >> 172.21.15.39:0/125376598 conn(0x7efc580cdcc0 msgr2=0x7efc5806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:12.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.166+0000 7efc5f400640 1 -- 172.21.15.39:0/125376598 shutdown_connections 2024-09-21T07:34:12.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.166+0000 7efc5f400640 1 -- 172.21.15.39:0/125376598 wait complete. 2024-09-21T07:34:12.193 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T07:34:12.193 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:34:12.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.418+0000 7fb9ab200640 1 -- 172.21.15.39:0/3903943391 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9ac0b54d0 msgr2=0x7fb9ac0b58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:12.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.418+0000 7fb9ab200640 1 --2- 172.21.15.39:0/3903943391 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9ac0b54d0 0x7fb9ac0b58e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4005fd0 tx=0x7fb9a402aa10 comp rx=0 tx=0).stop 2024-09-21T07:34:12.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.418+0000 7fb9ab200640 1 -- 172.21.15.39:0/3903943391 shutdown_connections 2024-09-21T07:34:12.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.418+0000 7fb9ab200640 1 --2- 172.21.15.39:0/3903943391 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb9ac0735d0 0x7fb9ac0caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:12.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.418+0000 7fb9ab200640 1 --2- 172.21.15.39:0/3903943391 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9ac0b54d0 0x7fb9ac0b58e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:12.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.418+0000 7fb9ab200640 1 -- 172.21.15.39:0/3903943391 >> 172.21.15.39:0/3903943391 conn(0x7fb9ac06d870 msgr2=0x7fb9ac06ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:12.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.418+0000 7fb9ab200640 1 -- 172.21.15.39:0/3903943391 shutdown_connections 2024-09-21T07:34:12.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.418+0000 7fb9ab200640 1 -- 172.21.15.39:0/3903943391 wait complete. 2024-09-21T07:34:12.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.420+0000 7fb9ab200640 1 Processor -- start 2024-09-21T07:34:12.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.420+0000 7fb9ab200640 1 -- start start 2024-09-21T07:34:12.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.422+0000 7fb9ab200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9ac0735d0 0x7fb9ac0ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:12.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.422+0000 7fb9ab200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9ac0af700 0x7fb9ac147a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:12.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.422+0000 7fb9ab200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb9ac0b54d0 0x7fb9ac0b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:12.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.422+0000 7fb9ab200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb9ac15d2b0 con 0x7fb9ac0af700 2024-09-21T07:34:12.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.422+0000 7fb9ab200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb9ac15d130 con 0x7fb9ac0735d0 2024-09-21T07:34:12.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.422+0000 7fb9ab200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb9ac15d430 con 0x7fb9ac0b54d0 2024-09-21T07:34:12.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.422+0000 7fb9a3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9ac0af700 0x7fb9ac147a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:12.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.422+0000 7fb9a3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9ac0af700 0x7fb9ac147a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41398/0 (socket says 172.21.15.39:41398) 2024-09-21T07:34:12.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.422+0000 7fb9a3e00640 1 -- 172.21.15.39:0/1499030408 learned_addr learned my addr 172.21.15.39:0/1499030408 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:12.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.422+0000 7fb9aa800640 1 --2- 172.21.15.39:0/1499030408 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb9ac0b54d0 0x7fb9ac0b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:12.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.422+0000 7fb9a9e00640 1 --2- 172.21.15.39:0/1499030408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9ac0735d0 0x7fb9ac0ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:12.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.423+0000 7fb9a3e00640 1 -- 172.21.15.39:0/1499030408 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb9ac0b54d0 msgr2=0x7fb9ac0b0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:12.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.423+0000 7fb9a3e00640 1 --2- 172.21.15.39:0/1499030408 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb9ac0b54d0 0x7fb9ac0b0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:12.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.423+0000 7fb9a3e00640 1 -- 172.21.15.39:0/1499030408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9ac0735d0 msgr2=0x7fb9ac0ccd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:12.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.423+0000 7fb9a3e00640 1 --2- 172.21.15.39:0/1499030408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9ac0735d0 0x7fb9ac0ccd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:12.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.423+0000 7fb9a3e00640 1 -- 172.21.15.39:0/1499030408 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9ac0cc2e0 con 0x7fb9ac0af700 2024-09-21T07:34:12.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.423+0000 7fb9a3e00640 1 --2- 172.21.15.39:0/1499030408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9ac0af700 0x7fb9ac147a60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb99400c9a0 tx=0x7fb99400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:12.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.423+0000 7fb9a2000640 1 -- 172.21.15.39:0/1499030408 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb994007bb0 con 0x7fb9ac0af700 2024-09-21T07:34:12.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.423+0000 7fb9a2000640 1 -- 172.21.15.39:0/1499030408 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb994007e40 con 0x7fb9ac0af700 2024-09-21T07:34:12.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.423+0000 7fb9a2000640 1 -- 172.21.15.39:0/1499030408 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99400f6b0 con 0x7fb9ac0af700 2024-09-21T07:34:12.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.423+0000 7fb9ab200640 1 -- 172.21.15.39:0/1499030408 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9ac147fa0 con 0x7fb9ac0af700 2024-09-21T07:34:12.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.423+0000 7fb9ab200640 1 -- 172.21.15.39:0/1499030408 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9ac0cc580 con 0x7fb9ac0af700 2024-09-21T07:34:12.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.424+0000 7fb9ab200640 1 -- 172.21.15.39:0/1499030408 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9ac051040 con 0x7fb9ac0af700 2024-09-21T07:34:12.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.426+0000 7fb9a2000640 1 -- 172.21.15.39:0/1499030408 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb99400f850 con 0x7fb9ac0af700 2024-09-21T07:34:12.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.427+0000 7fb9a2000640 1 --2- 172.21.15.39:0/1499030408 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb988078b90 0x7fb98807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:12.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.427+0000 7fb9a2000640 1 -- 172.21.15.39:0/1499030408 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fb99409b2e0 con 0x7fb9ac0af700 2024-09-21T07:34:12.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.427+0000 7fb9a9e00640 1 --2- 172.21.15.39:0/1499030408 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb988078b90 0x7fb98807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:12.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.428+0000 7fb9a9e00640 1 --2- 172.21.15.39:0/1499030408 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb988078b90 0x7fb98807b0a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb99c001720 tx=0x7fb99c009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:12.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.431+0000 7fb9a2000640 1 -- 172.21.15.39:0/1499030408 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb9ac051040 con 0x7fb9ac0af700 2024-09-21T07:34:12.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.625+0000 7fb9ab200640 1 -- 172.21.15.39:0/1499030408 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb9ac15d8e0 con 0x7fb988078b90 2024-09-21T07:34:12.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.631+0000 7fb9a2000640 1 -- 172.21.15.39:0/1499030408 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248165 (secure 0 0 0) 0x7fb9ac15d8e0 con 0x7fb988078b90 2024-09-21T07:34:12.636 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:34:12.637 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:34:12.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.636+0000 7fb9ab200640 1 -- 172.21.15.39:0/1499030408 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb988078b90 msgr2=0x7fb98807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:12.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.636+0000 7fb9ab200640 1 --2- 172.21.15.39:0/1499030408 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb988078b90 0x7fb98807b0a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb99c001720 tx=0x7fb99c009040 comp rx=0 tx=0).stop 2024-09-21T07:34:12.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.636+0000 7fb9ab200640 1 -- 172.21.15.39:0/1499030408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9ac0af700 msgr2=0x7fb9ac147a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:12.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.636+0000 7fb9ab200640 1 --2- 172.21.15.39:0/1499030408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9ac0af700 0x7fb9ac147a60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb99400c9a0 tx=0x7fb99400ce60 comp rx=0 tx=0).stop 2024-09-21T07:34:12.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.636+0000 7fb9ab200640 1 -- 172.21.15.39:0/1499030408 shutdown_connections 2024-09-21T07:34:12.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.636+0000 7fb9ab200640 1 --2- 172.21.15.39:0/1499030408 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb988078b90 0x7fb98807b0a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:12.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.636+0000 7fb9ab200640 1 --2- 172.21.15.39:0/1499030408 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb9ac0b54d0 0x7fb9ac0b0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:12.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.636+0000 7fb9ab200640 1 --2- 172.21.15.39:0/1499030408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9ac0af700 0x7fb9ac147a60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:12.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.636+0000 7fb9ab200640 1 --2- 172.21.15.39:0/1499030408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9ac0735d0 0x7fb9ac0ccd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:12.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.636+0000 7fb9ab200640 1 -- 172.21.15.39:0/1499030408 >> 172.21.15.39:0/1499030408 conn(0x7fb9ac06d870 msgr2=0x7fb9ac0b4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:12.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.636+0000 7fb9ab200640 1 -- 172.21.15.39:0/1499030408 shutdown_connections 2024-09-21T07:34:12.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.636+0000 7fb9ab200640 1 -- 172.21.15.39:0/1499030408 wait complete. 2024-09-21T07:34:12.659 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-09-21T07:34:11.547441+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592084,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":129,"num_write_kb":1639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":95,"ondisk_log_size":95,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236800,"kb_used_data":3936,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737920,"statfs":{"total":773094113280,"available":772851630080,"internally_reserved":0,"allocated":4030464,"data_stored":2268516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1716,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.424480"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.136358+0000","last_change":"2024-09-21T07:34:07.136358+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.136358+0000","last_undegraded":"2024-09-21T07:34:07.136358+0000","last_fullsized":"2024-09-21T07:34:07.136358+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496812+0000","last_change":"2024-09-21T07:34:08.163227+0000","last_active":"2024-09-21T07:34:09.496812+0000","last_peered":"2024-09-21T07:34:09.496812+0000","last_clean":"2024-09-21T07:34:09.496812+0000","last_became_active":"2024-09-21T07:34:08.162915+0000","last_became_peered":"2024-09-21T07:34:08.162915+0000","last_unstale":"2024-09-21T07:34:09.496812+0000","last_undegraded":"2024-09-21T07:34:09.496812+0000","last_fullsized":"2024-09-21T07:34:09.496812+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:16.292830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.129770+0000","last_change":"2024-09-21T07:34:07.129770+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.129770+0000","last_undegraded":"2024-09-21T07:34:07.129770+0000","last_fullsized":"2024-09-21T07:34:07.129770+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.144427+0000","last_change":"2024-09-21T07:34:07.144427+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.144427+0000","last_undegraded":"2024-09-21T07:34:07.144427+0000","last_fullsized":"2024-09-21T07:34:07.144427+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.130027+0000","last_change":"2024-09-21T07:34:07.130027+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.130027+0000","last_undegraded":"2024-09-21T07:34:07.130027+0000","last_fullsized":"2024-09-21T07:34:07.130027+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496422+0000","last_change":"2024-09-21T07:34:08.161341+0000","last_active":"2024-09-21T07:34:09.496422+0000","last_peered":"2024-09-21T07:34:09.496422+0000","last_clean":"2024-09-21T07:34:09.496422+0000","last_became_active":"2024-09-21T07:34:08.161037+0000","last_became_peered":"2024-09-21T07:34:08.161037+0000","last_unstale":"2024-09-21T07:34:09.496422+0000","last_undegraded":"2024-09-21T07:34:09.496422+0000","last_fullsized":"2024-09-21T07:34:09.496422+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:52.361740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.142031+0000","last_change":"2024-09-21T07:34:07.142031+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.142031+0000","last_undegraded":"2024-09-21T07:34:07.142031+0000","last_fullsized":"2024-09-21T07:34:07.142031+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.147161+0000","last_change":"2024-09-21T07:34:07.147161+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.147161+0000","last_undegraded":"2024-09-21T07:34:07.147161+0000","last_fullsized":"2024-09-21T07:34:07.147161+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497858+0000","last_change":"2024-09-21T07:34:08.170445+0000","last_active":"2024-09-21T07:34:09.497858+0000","last_peered":"2024-09-21T07:34:09.497858+0000","last_clean":"2024-09-21T07:34:09.497858+0000","last_became_active":"2024-09-21T07:34:08.170115+0000","last_became_peered":"2024-09-21T07:34:08.170115+0000","last_unstale":"2024-09-21T07:34:09.497858+0000","last_undegraded":"2024-09-21T07:34:09.497858+0000","last_fullsized":"2024-09-21T07:34:09.497858+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:32.224498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.140904+0000","last_change":"2024-09-21T07:34:07.140904+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.140904+0000","last_undegraded":"2024-09-21T07:34:07.140904+0000","last_fullsized":"2024-09-21T07:34:07.140904+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.147060+0000","last_change":"2024-09-21T07:34:07.147060+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.147060+0000","last_undegraded":"2024-09-21T07:34:07.147060+0000","last_fullsized":"2024-09-21T07:34:07.147060+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.131743+0000","last_change":"2024-09-21T07:34:07.131743+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.131743+0000","last_undegraded":"2024-09-21T07:34:07.131743+0000","last_fullsized":"2024-09-21T07:34:07.131743+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.129452+0000","last_change":"2024-09-21T07:34:07.129452+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.129452+0000","last_undegraded":"2024-09-21T07:34:07.129452+0000","last_fullsized":"2024-09-21T07:34:07.129452+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496432+0000","last_change":"2024-09-21T07:34:08.167950+0000","last_active":"2024-09-21T07:34:09.496432+0000","last_peered":"2024-09-21T07:34:09.496432+0000","last_clean":"2024-09-21T07:34:09.496432+0000","last_became_active":"2024-09-21T07:34:08.167687+0000","last_became_peered":"2024-09-21T07:34:08.167687+0000","last_unstale":"2024-09-21T07:34:09.496432+0000","last_undegraded":"2024-09-21T07:34:09.496432+0000","last_fullsized":"2024-09-21T07:34:09.496432+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:44:01.238051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.130177+0000","last_change":"2024-09-21T07:34:07.130177+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.130177+0000","last_undegraded":"2024-09-21T07:34:07.130177+0000","last_fullsized":"2024-09-21T07:34:07.130177+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.140164+0000","last_change":"2024-09-21T07:34:07.140164+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.140164+0000","last_undegraded":"2024-09-21T07:34:07.140164+0000","last_fullsized":"2024-09-21T07:34:07.140164+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.150046+0000","last_change":"2024-09-21T07:34:07.150046+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.150046+0000","last_undegraded":"2024-09-21T07:34:07.150046+0000","last_fullsized":"2024-09-21T07:34:07.150046+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496674+0000","last_change":"2024-09-21T07:34:08.153003+0000","last_active":"2024-09-21T07:34:09.496674+0000","last_peered":"2024-09-21T07:34:09.496674+0000","last_clean":"2024-09-21T07:34:09.496674+0000","last_became_active":"2024-09-21T07:34:08.150882+0000","last_became_peered":"2024-09-21T07:34:08.150882+0000","last_unstale":"2024-09-21T07:34:09.496674+0000","last_undegraded":"2024-09-21T07:34:09.496674+0000","last_fullsized":"2024-09-21T07:34:09.496674+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:38.611907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496759+0000","last_change":"2024-09-21T07:34:08.158999+0000","last_active":"2024-09-21T07:34:09.496759+0000","last_peered":"2024-09-21T07:34:09.496759+0000","last_clean":"2024-09-21T07:34:09.496759+0000","last_became_active":"2024-09-21T07:34:08.158712+0000","last_became_peered":"2024-09-21T07:34:08.158712+0000","last_unstale":"2024-09-21T07:34:09.496759+0000","last_undegraded":"2024-09-21T07:34:09.496759+0000","last_fullsized":"2024-09-21T07:34:09.496759+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:34:04.145536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.129482+0000","last_change":"2024-09-21T07:34:07.129482+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.129482+0000","last_undegraded":"2024-09-21T07:34:07.129482+0000","last_fullsized":"2024-09-21T07:34:07.129482+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496796+0000","last_change":"2024-09-21T07:34:08.168578+0000","last_active":"2024-09-21T07:34:09.496796+0000","last_peered":"2024-09-21T07:34:09.496796+0000","last_clean":"2024-09-21T07:34:09.496796+0000","last_became_active":"2024-09-21T07:34:08.168235+0000","last_became_peered":"2024-09-21T07:34:08.168235+0000","last_unstale":"2024-09-21T07:34:09.496796+0000","last_undegraded":"2024-09-21T07:34:09.496796+0000","last_fullsized":"2024-09-21T07:34:09.496796+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:18.132264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496889+0000","last_change":"2024-09-21T07:34:08.159192+0000","last_active":"2024-09-21T07:34:09.496889+0000","last_peered":"2024-09-21T07:34:09.496889+0000","last_clean":"2024-09-21T07:34:09.496889+0000","last_became_active":"2024-09-21T07:34:08.158860+0000","last_became_peered":"2024-09-21T07:34:08.158860+0000","last_unstale":"2024-09-21T07:34:09.496889+0000","last_undegraded":"2024-09-21T07:34:09.496889+0000","last_fullsized":"2024-09-21T07:34:09.496889+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:48.287489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.132277+0000","last_change":"2024-09-21T07:34:07.132277+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.132277+0000","last_undegraded":"2024-09-21T07:34:07.132277+0000","last_fullsized":"2024-09-21T07:34:07.132277+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.129770+0000","last_change":"2024-09-21T07:34:07.129770+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.129770+0000","last_undegraded":"2024-09-21T07:34:07.129770+0000","last_fullsized":"2024-09-21T07:34:07.129770+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.135980+0000","last_change":"2024-09-21T07:34:07.135980+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.135980+0000","last_undegraded":"2024-09-21T07:34:07.135980+0000","last_fullsized":"2024-09-21T07:34:07.135980+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496941+0000","last_change":"2024-09-21T07:34:08.158447+0000","last_active":"2024-09-21T07:34:09.496941+0000","last_peered":"2024-09-21T07:34:09.496941+0000","last_clean":"2024-09-21T07:34:09.496941+0000","last_became_active":"2024-09-21T07:34:08.158138+0000","last_became_peered":"2024-09-21T07:34:08.158138+0000","last_unstale":"2024-09-21T07:34:09.496941+0000","last_undegraded":"2024-09-21T07:34:09.496941+0000","last_fullsized":"2024-09-21T07:34:09.496941+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:05.155134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.136992+0000","last_change":"2024-09-21T07:34:07.136992+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.136992+0000","last_undegraded":"2024-09-21T07:34:07.136992+0000","last_fullsized":"2024-09-21T07:34:07.136992+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.129413+0000","last_change":"2024-09-21T07:34:07.129413+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.129413+0000","last_undegraded":"2024-09-21T07:34:07.129413+0000","last_fullsized":"2024-09-21T07:34:07.129413+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497087+0000","last_change":"2024-09-21T07:34:08.153475+0000","last_active":"2024-09-21T07:34:09.497087+0000","last_peered":"2024-09-21T07:34:09.497087+0000","last_clean":"2024-09-21T07:34:09.497087+0000","last_became_active":"2024-09-21T07:34:08.152735+0000","last_became_peered":"2024-09-21T07:34:08.152735+0000","last_unstale":"2024-09-21T07:34:09.497087+0000","last_undegraded":"2024-09-21T07:34:09.497087+0000","last_fullsized":"2024-09-21T07:34:09.497087+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:42.395864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.147613+0000","last_change":"2024-09-21T07:34:07.147613+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.147613+0000","last_undegraded":"2024-09-21T07:34:07.147613+0000","last_fullsized":"2024-09-21T07:34:07.147613+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.136369+0000","last_change":"2024-09-21T07:34:07.136369+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.136369+0000","last_undegraded":"2024-09-21T07:34:07.136369+0000","last_fullsized":"2024-09-21T07:34:07.136369+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.146626+0000","last_change":"2024-09-21T07:34:07.146626+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.146626+0000","last_undegraded":"2024-09-21T07:34:07.146626+0000","last_fullsized":"2024-09-21T07:34:07.146626+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.131763+0000","last_change":"2024-09-21T07:34:07.131763+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.131763+0000","last_undegraded":"2024-09-21T07:34:07.131763+0000","last_fullsized":"2024-09-21T07:34:07.131763+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T07:34:07.159481+0000","last_change":"2024-09-21T07:34:07.155982+0000","last_active":"2024-09-21T07:34:07.159481+0000","last_peered":"2024-09-21T07:34:07.159481+0000","last_clean":"2024-09-21T07:34:07.159481+0000","last_became_active":"2024-09-21T07:34:07.155500+0000","last_became_peered":"2024-09-21T07:34:07.155500+0000","last_unstale":"2024-09-21T07:34:07.159481+0000","last_undegraded":"2024-09-21T07:34:07.159481+0000","last_fullsized":"2024-09-21T07:34:07.159481+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:43:17.626826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.135832+0000","last_change":"2024-09-21T07:34:07.135832+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.135832+0000","last_undegraded":"2024-09-21T07:34:07.135832+0000","last_fullsized":"2024-09-21T07:34:07.135832+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T07:34:07.154433+0000","last_change":"2024-09-21T07:34:07.148229+0000","last_active":"2024-09-21T07:34:07.154433+0000","last_peered":"2024-09-21T07:34:07.154433+0000","last_clean":"2024-09-21T07:34:07.154433+0000","last_became_active":"2024-09-21T07:34:07.147920+0000","last_became_peered":"2024-09-21T07:34:07.147920+0000","last_unstale":"2024-09-21T07:34:07.154433+0000","last_undegraded":"2024-09-21T07:34:07.154433+0000","last_fullsized":"2024-09-21T07:34:07.154433+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:43:17.626826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497703+0000","last_change":"2024-09-21T07:34:08.162272+0000","last_active":"2024-09-21T07:34:09.497703+0000","last_peered":"2024-09-21T07:34:09.497703+0000","last_clean":"2024-09-21T07:34:09.497703+0000","last_became_active":"2024-09-21T07:34:08.162031+0000","last_became_peered":"2024-09-21T07:34:08.162031+0000","last_unstale":"2024-09-21T07:34:09.497703+0000","last_undegraded":"2024-09-21T07:34:09.497703+0000","last_fullsized":"2024-09-21T07:34:09.497703+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:48.649163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496548+0000","last_change":"2024-09-21T07:34:07.162816+0000","last_active":"2024-09-21T07:34:09.496548+0000","last_peered":"2024-09-21T07:34:09.496548+0000","last_clean":"2024-09-21T07:34:09.496548+0000","last_became_active":"2024-09-21T07:34:07.162478+0000","last_became_peered":"2024-09-21T07:34:07.162478+0000","last_unstale":"2024-09-21T07:34:09.496548+0000","last_undegraded":"2024-09-21T07:34:09.496548+0000","last_fullsized":"2024-09-21T07:34:09.496548+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:18:03.659966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.136432+0000","last_change":"2024-09-21T07:34:07.136432+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.136432+0000","last_undegraded":"2024-09-21T07:34:07.136432+0000","last_fullsized":"2024-09-21T07:34:07.136432+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T07:34:07.159878+0000","last_change":"2024-09-21T07:34:07.151195+0000","last_active":"2024-09-21T07:34:07.159878+0000","last_peered":"2024-09-21T07:34:07.159878+0000","last_clean":"2024-09-21T07:34:07.159878+0000","last_became_active":"2024-09-21T07:34:07.150871+0000","last_became_peered":"2024-09-21T07:34:07.150871+0000","last_unstale":"2024-09-21T07:34:07.159878+0000","last_undegraded":"2024-09-21T07:34:07.159878+0000","last_fullsized":"2024-09-21T07:34:07.159878+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:41:43.994817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.143895+0000","last_change":"2024-09-21T07:34:07.143895+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.143895+0000","last_undegraded":"2024-09-21T07:34:07.143895+0000","last_fullsized":"2024-09-21T07:34:07.143895+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497041+0000","last_change":"2024-09-21T07:34:07.154783+0000","last_active":"2024-09-21T07:34:09.497041+0000","last_peered":"2024-09-21T07:34:09.497041+0000","last_clean":"2024-09-21T07:34:09.497041+0000","last_became_active":"2024-09-21T07:34:07.154378+0000","last_became_peered":"2024-09-21T07:34:07.154378+0000","last_unstale":"2024-09-21T07:34:09.497041+0000","last_undegraded":"2024-09-21T07:34:09.497041+0000","last_fullsized":"2024-09-21T07:34:09.497041+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:16.041455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.131554+0000","last_change":"2024-09-21T07:34:07.131554+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.131554+0000","last_undegraded":"2024-09-21T07:34:07.131554+0000","last_fullsized":"2024-09-21T07:34:07.131554+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496886+0000","last_change":"2024-09-21T07:34:07.153288+0000","last_active":"2024-09-21T07:34:09.496886+0000","last_peered":"2024-09-21T07:34:09.496886+0000","last_clean":"2024-09-21T07:34:09.496886+0000","last_became_active":"2024-09-21T07:34:07.152949+0000","last_became_peered":"2024-09-21T07:34:07.152949+0000","last_unstale":"2024-09-21T07:34:09.496886+0000","last_undegraded":"2024-09-21T07:34:09.496886+0000","last_fullsized":"2024-09-21T07:34:09.496886+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:54:43.804810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496478+0000","last_change":"2024-09-21T07:34:08.166316+0000","last_active":"2024-09-21T07:34:09.496478+0000","last_peered":"2024-09-21T07:34:09.496478+0000","last_clean":"2024-09-21T07:34:09.496478+0000","last_became_active":"2024-09-21T07:34:08.165006+0000","last_became_peered":"2024-09-21T07:34:08.165006+0000","last_unstale":"2024-09-21T07:34:09.496478+0000","last_undegraded":"2024-09-21T07:34:09.496478+0000","last_fullsized":"2024-09-21T07:34:09.496478+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:27.528708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496037+0000","last_change":"2024-09-21T07:34:07.141466+0000","last_active":"2024-09-21T07:34:09.496037+0000","last_peered":"2024-09-21T07:34:09.496037+0000","last_clean":"2024-09-21T07:34:09.496037+0000","last_became_active":"2024-09-21T07:34:07.141003+0000","last_became_peered":"2024-09-21T07:34:07.141003+0000","last_unstale":"2024-09-21T07:34:09.496037+0000","last_undegraded":"2024-09-21T07:34:09.496037+0000","last_fullsized":"2024-09-21T07:34:09.496037+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:52:33.957055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.137582+0000","last_change":"2024-09-21T07:34:07.137582+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.137582+0000","last_undegraded":"2024-09-21T07:34:07.137582+0000","last_fullsized":"2024-09-21T07:34:07.137582+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496450+0000","last_change":"2024-09-21T07:34:07.142126+0000","last_active":"2024-09-21T07:34:09.496450+0000","last_peered":"2024-09-21T07:34:09.496450+0000","last_clean":"2024-09-21T07:34:09.496450+0000","last_became_active":"2024-09-21T07:34:07.141713+0000","last_became_peered":"2024-09-21T07:34:07.141713+0000","last_unstale":"2024-09-21T07:34:09.496450+0000","last_undegraded":"2024-09-21T07:34:09.496450+0000","last_fullsized":"2024-09-21T07:34:09.496450+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:58.789677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.129621+0000","last_change":"2024-09-21T07:34:07.129621+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.129621+0000","last_undegraded":"2024-09-21T07:34:07.129621+0000","last_fullsized":"2024-09-21T07:34:07.129621+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496527+0000","last_change":"2024-09-21T07:34:07.154435+0000","last_active":"2024-09-21T07:34:09.496527+0000","last_peered":"2024-09-21T07:34:09.496527+0000","last_clean":"2024-09-21T07:34:09.496527+0000","last_became_active":"2024-09-21T07:34:07.154127+0000","last_became_peered":"2024-09-21T07:34:07.154127+0000","last_unstale":"2024-09-21T07:34:09.496527+0000","last_undegraded":"2024-09-21T07:34:09.496527+0000","last_fullsized":"2024-09-21T07:34:09.496527+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:11:21.678594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496615+0000","last_change":"2024-09-21T07:34:08.170301+0000","last_active":"2024-09-21T07:34:09.496615+0000","last_peered":"2024-09-21T07:34:09.496615+0000","last_clean":"2024-09-21T07:34:09.496615+0000","last_became_active":"2024-09-21T07:34:08.169925+0000","last_became_peered":"2024-09-21T07:34:08.169925+0000","last_unstale":"2024-09-21T07:34:09.496615+0000","last_undegraded":"2024-09-21T07:34:09.496615+0000","last_fullsized":"2024-09-21T07:34:09.496615+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:31.073725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T07:34:07.159973+0000","last_change":"2024-09-21T07:34:07.152298+0000","last_active":"2024-09-21T07:34:07.159973+0000","last_peered":"2024-09-21T07:34:07.159973+0000","last_clean":"2024-09-21T07:34:07.159973+0000","last_became_active":"2024-09-21T07:34:07.151921+0000","last_became_peered":"2024-09-21T07:34:07.151921+0000","last_unstale":"2024-09-21T07:34:07.159973+0000","last_undegraded":"2024-09-21T07:34:07.159973+0000","last_fullsized":"2024-09-21T07:34:07.159973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:59:04.464801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496907+0000","last_change":"2024-09-21T07:34:08.160957+0000","last_active":"2024-09-21T07:34:09.496907+0000","last_peered":"2024-09-21T07:34:09.496907+0000","last_clean":"2024-09-21T07:34:09.496907+0000","last_became_active":"2024-09-21T07:34:08.160601+0000","last_became_peered":"2024-09-21T07:34:08.160601+0000","last_unstale":"2024-09-21T07:34:09.496907+0000","last_undegraded":"2024-09-21T07:34:09.496907+0000","last_fullsized":"2024-09-21T07:34:09.496907+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:19:16.261092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T07:34:07.157782+0000","last_change":"2024-09-21T07:34:07.153099+0000","last_active":"2024-09-21T07:34:07.157782+0000","last_peered":"2024-09-21T07:34:07.157782+0000","last_clean":"2024-09-21T07:34:07.157782+0000","last_became_active":"2024-09-21T07:34:07.152790+0000","last_became_peered":"2024-09-21T07:34:07.152790+0000","last_unstale":"2024-09-21T07:34:07.157782+0000","last_undegraded":"2024-09-21T07:34:07.157782+0000","last_fullsized":"2024-09-21T07:34:07.157782+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:57:12.586864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.131283+0000","last_change":"2024-09-21T07:34:07.131283+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.131283+0000","last_undegraded":"2024-09-21T07:34:07.131283+0000","last_fullsized":"2024-09-21T07:34:07.131283+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":21,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496721+0000","last_change":"2024-09-21T07:34:07.162976+0000","last_active":"2024-09-21T07:34:09.496721+0000","last_peered":"2024-09-21T07:34:09.496721+0000","last_clean":"2024-09-21T07:34:09.496721+0000","last_became_active":"2024-09-21T07:34:07.162698+0000","last_became_peered":"2024-09-21T07:34:07.162698+0000","last_unstale":"2024-09-21T07:34:09.496721+0000","last_undegraded":"2024-09-21T07:34:09.496721+0000","last_fullsized":"2024-09-21T07:34:09.496721+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:01:04.212051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.135901+0000","last_change":"2024-09-21T07:34:07.135901+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.135901+0000","last_undegraded":"2024-09-21T07:34:07.135901+0000","last_fullsized":"2024-09-21T07:34:07.135901+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497188+0000","last_change":"2024-09-21T07:34:07.150104+0000","last_active":"2024-09-21T07:34:09.497188+0000","last_peered":"2024-09-21T07:34:09.497188+0000","last_clean":"2024-09-21T07:34:09.497188+0000","last_became_active":"2024-09-21T07:34:07.149757+0000","last_became_peered":"2024-09-21T07:34:07.149757+0000","last_unstale":"2024-09-21T07:34:09.497188+0000","last_undegraded":"2024-09-21T07:34:09.497188+0000","last_fullsized":"2024-09-21T07:34:09.497188+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:17.255537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496697+0000","last_change":"2024-09-21T07:34:08.166843+0000","last_active":"2024-09-21T07:34:09.496697+0000","last_peered":"2024-09-21T07:34:09.496697+0000","last_clean":"2024-09-21T07:34:09.496697+0000","last_became_active":"2024-09-21T07:34:08.166516+0000","last_became_peered":"2024-09-21T07:34:08.166516+0000","last_unstale":"2024-09-21T07:34:09.496697+0000","last_undegraded":"2024-09-21T07:34:09.496697+0000","last_fullsized":"2024-09-21T07:34:09.496697+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:39.634059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T07:34:07.162417+0000","last_change":"2024-09-21T07:34:07.160820+0000","last_active":"2024-09-21T07:34:07.162417+0000","last_peered":"2024-09-21T07:34:07.162417+0000","last_clean":"2024-09-21T07:34:07.162417+0000","last_became_active":"2024-09-21T07:34:07.160364+0000","last_became_peered":"2024-09-21T07:34:07.160364+0000","last_unstale":"2024-09-21T07:34:07.162417+0000","last_undegraded":"2024-09-21T07:34:07.162417+0000","last_fullsized":"2024-09-21T07:34:07.162417+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:20:00.284603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.143090+0000","last_change":"2024-09-21T07:34:07.143090+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.143090+0000","last_undegraded":"2024-09-21T07:34:07.143090+0000","last_fullsized":"2024-09-21T07:34:07.143090+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496142+0000","last_change":"2024-09-21T07:34:07.153626+0000","last_active":"2024-09-21T07:34:09.496142+0000","last_peered":"2024-09-21T07:34:09.496142+0000","last_clean":"2024-09-21T07:34:09.496142+0000","last_became_active":"2024-09-21T07:34:07.153334+0000","last_became_peered":"2024-09-21T07:34:07.153334+0000","last_unstale":"2024-09-21T07:34:09.496142+0000","last_undegraded":"2024-09-21T07:34:09.496142+0000","last_fullsized":"2024-09-21T07:34:09.496142+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:48.214805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.135638+0000","last_change":"2024-09-21T07:34:07.135638+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.135638+0000","last_undegraded":"2024-09-21T07:34:07.135638+0000","last_fullsized":"2024-09-21T07:34:07.135638+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496343+0000","last_change":"2024-09-21T07:34:07.161149+0000","last_active":"2024-09-21T07:34:09.496343+0000","last_peered":"2024-09-21T07:34:09.496343+0000","last_clean":"2024-09-21T07:34:09.496343+0000","last_became_active":"2024-09-21T07:34:07.160882+0000","last_became_peered":"2024-09-21T07:34:07.160882+0000","last_unstale":"2024-09-21T07:34:09.496343+0000","last_undegraded":"2024-09-21T07:34:09.496343+0000","last_fullsized":"2024-09-21T07:34:09.496343+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:55.427089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497420+0000","last_change":"2024-09-21T07:34:08.167375+0000","last_active":"2024-09-21T07:34:09.497420+0000","last_peered":"2024-09-21T07:34:09.497420+0000","last_clean":"2024-09-21T07:34:09.497420+0000","last_became_active":"2024-09-21T07:34:08.165919+0000","last_became_peered":"2024-09-21T07:34:08.165919+0000","last_unstale":"2024-09-21T07:34:09.497420+0000","last_undegraded":"2024-09-21T07:34:09.497420+0000","last_fullsized":"2024-09-21T07:34:09.497420+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:30.388748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T07:34:07.161732+0000","last_change":"2024-09-21T07:34:07.154981+0000","last_active":"2024-09-21T07:34:07.161732+0000","last_peered":"2024-09-21T07:34:07.161732+0000","last_clean":"2024-09-21T07:34:07.161732+0000","last_became_active":"2024-09-21T07:34:07.154706+0000","last_became_peered":"2024-09-21T07:34:07.154706+0000","last_unstale":"2024-09-21T07:34:07.161732+0000","last_undegraded":"2024-09-21T07:34:07.161732+0000","last_fullsized":"2024-09-21T07:34:07.161732+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:00.242190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.131452+0000","last_change":"2024-09-21T07:34:07.131452+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.131452+0000","last_undegraded":"2024-09-21T07:34:07.131452+0000","last_fullsized":"2024-09-21T07:34:07.131452+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497519+0000","last_change":"2024-09-21T07:34:07.149293+0000","last_active":"2024-09-21T07:34:09.497519+0000","last_peered":"2024-09-21T07:34:09.497519+0000","last_clean":"2024-09-21T07:34:09.497519+0000","last_became_active":"2024-09-21T07:34:07.148938+0000","last_became_peered":"2024-09-21T07:34:07.148938+0000","last_unstale":"2024-09-21T07:34:09.497519+0000","last_undegraded":"2024-09-21T07:34:09.497519+0000","last_fullsized":"2024-09-21T07:34:09.497519+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:59.434905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.137799+0000","last_change":"2024-09-21T07:34:07.137799+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.137799+0000","last_undegraded":"2024-09-21T07:34:07.137799+0000","last_fullsized":"2024-09-21T07:34:07.137799+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497479+0000","last_change":"2024-09-21T07:34:07.148308+0000","last_active":"2024-09-21T07:34:09.497479+0000","last_peered":"2024-09-21T07:34:09.497479+0000","last_clean":"2024-09-21T07:34:09.497479+0000","last_became_active":"2024-09-21T07:34:07.148001+0000","last_became_peered":"2024-09-21T07:34:07.148001+0000","last_unstale":"2024-09-21T07:34:09.497479+0000","last_undegraded":"2024-09-21T07:34:09.497479+0000","last_fullsized":"2024-09-21T07:34:09.497479+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:54:32.939044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.129855+0000","last_change":"2024-09-21T07:34:07.129855+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.129855+0000","last_undegraded":"2024-09-21T07:34:07.129855+0000","last_fullsized":"2024-09-21T07:34:07.129855+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497624+0000","last_change":"2024-09-21T07:34:07.152996+0000","last_active":"2024-09-21T07:34:09.497624+0000","last_peered":"2024-09-21T07:34:09.497624+0000","last_clean":"2024-09-21T07:34:09.497624+0000","last_became_active":"2024-09-21T07:34:07.152627+0000","last_became_peered":"2024-09-21T07:34:07.152627+0000","last_unstale":"2024-09-21T07:34:09.497624+0000","last_undegraded":"2024-09-21T07:34:09.497624+0000","last_fullsized":"2024-09-21T07:34:09.497624+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:24:42.920690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.128256+0000","last_change":"2024-09-21T07:34:07.128256+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.128256+0000","last_undegraded":"2024-09-21T07:34:07.128256+0000","last_fullsized":"2024-09-21T07:34:07.128256+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T07:34:07.159940+0000","last_change":"2024-09-21T07:34:07.151886+0000","last_active":"2024-09-21T07:34:07.159940+0000","last_peered":"2024-09-21T07:34:07.159940+0000","last_clean":"2024-09-21T07:34:07.159940+0000","last_became_active":"2024-09-21T07:34:07.150347+0000","last_became_peered":"2024-09-21T07:34:07.150347+0000","last_unstale":"2024-09-21T07:34:07.159940+0000","last_undegraded":"2024-09-21T07:34:07.159940+0000","last_fullsized":"2024-09-21T07:34:07.159940+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:53.036159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.128809+0000","last_change":"2024-09-21T07:34:07.128809+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.128809+0000","last_undegraded":"2024-09-21T07:34:07.128809+0000","last_fullsized":"2024-09-21T07:34:07.128809+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T07:34:07.162084+0000","last_change":"2024-09-21T07:34:07.161337+0000","last_active":"2024-09-21T07:34:07.162084+0000","last_peered":"2024-09-21T07:34:07.162084+0000","last_clean":"2024-09-21T07:34:07.162084+0000","last_became_active":"2024-09-21T07:34:07.161055+0000","last_became_peered":"2024-09-21T07:34:07.161055+0000","last_unstale":"2024-09-21T07:34:07.162084+0000","last_undegraded":"2024-09-21T07:34:07.162084+0000","last_fullsized":"2024-09-21T07:34:07.162084+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:53.036159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.128188+0000","last_change":"2024-09-21T07:34:07.128188+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.128188+0000","last_undegraded":"2024-09-21T07:34:07.128188+0000","last_fullsized":"2024-09-21T07:34:07.128188+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497320+0000","last_change":"2024-09-21T07:34:07.148155+0000","last_active":"2024-09-21T07:34:09.497320+0000","last_peered":"2024-09-21T07:34:09.497320+0000","last_clean":"2024-09-21T07:34:09.497320+0000","last_became_active":"2024-09-21T07:34:07.147855+0000","last_became_peered":"2024-09-21T07:34:07.147855+0000","last_unstale":"2024-09-21T07:34:09.497320+0000","last_undegraded":"2024-09-21T07:34:09.497320+0000","last_fullsized":"2024-09-21T07:34:09.497320+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:41:10.667777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.142621+0000","last_change":"2024-09-21T07:34:07.142621+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.142621+0000","last_undegraded":"2024-09-21T07:34:07.142621+0000","last_fullsized":"2024-09-21T07:34:07.142621+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T07:34:07.160117+0000","last_change":"2024-09-21T07:34:07.156707+0000","last_active":"2024-09-21T07:34:07.160117+0000","last_peered":"2024-09-21T07:34:07.160117+0000","last_clean":"2024-09-21T07:34:07.160117+0000","last_became_active":"2024-09-21T07:34:07.156326+0000","last_became_peered":"2024-09-21T07:34:07.156326+0000","last_unstale":"2024-09-21T07:34:07.160117+0000","last_undegraded":"2024-09-21T07:34:07.160117+0000","last_fullsized":"2024-09-21T07:34:07.160117+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:19.737233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.130004+0000","last_change":"2024-09-21T07:34:07.130004+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.130004+0000","last_undegraded":"2024-09-21T07:34:07.130004+0000","last_fullsized":"2024-09-21T07:34:07.130004+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T07:34:07.161433+0000","last_change":"2024-09-21T07:34:07.153252+0000","last_active":"2024-09-21T07:34:07.161433+0000","last_peered":"2024-09-21T07:34:07.161433+0000","last_clean":"2024-09-21T07:34:07.161433+0000","last_became_active":"2024-09-21T07:34:07.152932+0000","last_became_peered":"2024-09-21T07:34:07.152932+0000","last_unstale":"2024-09-21T07:34:07.161433+0000","last_undegraded":"2024-09-21T07:34:07.161433+0000","last_fullsized":"2024-09-21T07:34:07.161433+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:31.639108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.130642+0000","last_change":"2024-09-21T07:34:07.130642+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.130642+0000","last_undegraded":"2024-09-21T07:34:07.130642+0000","last_fullsized":"2024-09-21T07:34:07.130642+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T07:34:07.163218+0000","last_change":"2024-09-21T07:34:07.161285+0000","last_active":"2024-09-21T07:34:07.163218+0000","last_peered":"2024-09-21T07:34:07.163218+0000","last_clean":"2024-09-21T07:34:07.163218+0000","last_became_active":"2024-09-21T07:34:07.160925+0000","last_became_peered":"2024-09-21T07:34:07.160925+0000","last_unstale":"2024-09-21T07:34:07.163218+0000","last_undegraded":"2024-09-21T07:34:07.163218+0000","last_fullsized":"2024-09-21T07:34:07.163218+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:26.882263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.132803+0000","last_change":"2024-09-21T07:34:07.132803+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.132803+0000","last_undegraded":"2024-09-21T07:34:07.132803+0000","last_fullsized":"2024-09-21T07:34:07.132803+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T07:34:07.161782+0000","last_change":"2024-09-21T07:34:07.151263+0000","last_active":"2024-09-21T07:34:07.161782+0000","last_peered":"2024-09-21T07:34:07.161782+0000","last_clean":"2024-09-21T07:34:07.161782+0000","last_became_active":"2024-09-21T07:34:07.150981+0000","last_became_peered":"2024-09-21T07:34:07.150981+0000","last_unstale":"2024-09-21T07:34:07.161782+0000","last_undegraded":"2024-09-21T07:34:07.161782+0000","last_fullsized":"2024-09-21T07:34:07.161782+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:06.174996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.135692+0000","last_change":"2024-09-21T07:34:07.135692+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.135692+0000","last_undegraded":"2024-09-21T07:34:07.135692+0000","last_fullsized":"2024-09-21T07:34:07.135692+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T07:34:07.149962+0000","last_change":"2024-09-21T07:34:07.139885+0000","last_active":"2024-09-21T07:34:07.149962+0000","last_peered":"2024-09-21T07:34:07.149962+0000","last_clean":"2024-09-21T07:34:07.149962+0000","last_became_active":"2024-09-21T07:34:07.139560+0000","last_became_peered":"2024-09-21T07:34:07.139560+0000","last_unstale":"2024-09-21T07:34:07.149962+0000","last_undegraded":"2024-09-21T07:34:07.149962+0000","last_fullsized":"2024-09-21T07:34:07.149962+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:28:14.831565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.132414+0000","last_change":"2024-09-21T07:34:07.132414+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.132414+0000","last_undegraded":"2024-09-21T07:34:07.132414+0000","last_fullsized":"2024-09-21T07:34:07.132414+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T07:34:07.161575+0000","last_change":"2024-09-21T07:34:07.152465+0000","last_active":"2024-09-21T07:34:07.161575+0000","last_peered":"2024-09-21T07:34:07.161575+0000","last_clean":"2024-09-21T07:34:07.161575+0000","last_became_active":"2024-09-21T07:34:07.152065+0000","last_became_peered":"2024-09-21T07:34:07.152065+0000","last_unstale":"2024-09-21T07:34:07.161575+0000","last_undegraded":"2024-09-21T07:34:07.161575+0000","last_fullsized":"2024-09-21T07:34:07.161575+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:02.321477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496179+0000","last_change":"2024-09-21T07:34:08.150846+0000","last_active":"2024-09-21T07:34:09.496179+0000","last_peered":"2024-09-21T07:34:09.496179+0000","last_clean":"2024-09-21T07:34:09.496179+0000","last_became_active":"2024-09-21T07:34:08.150534+0000","last_became_peered":"2024-09-21T07:34:08.150534+0000","last_unstale":"2024-09-21T07:34:09.496179+0000","last_undegraded":"2024-09-21T07:34:09.496179+0000","last_fullsized":"2024-09-21T07:34:09.496179+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:26:49.717420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497461+0000","last_change":"2024-09-21T07:34:07.147358+0000","last_active":"2024-09-21T07:34:09.497461+0000","last_peered":"2024-09-21T07:34:09.497461+0000","last_clean":"2024-09-21T07:34:09.497461+0000","last_became_active":"2024-09-21T07:34:07.147036+0000","last_became_peered":"2024-09-21T07:34:07.147036+0000","last_unstale":"2024-09-21T07:34:09.497461+0000","last_undegraded":"2024-09-21T07:34:09.497461+0000","last_fullsized":"2024-09-21T07:34:09.497461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:04:22.493944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497584+0000","last_change":"2024-09-21T07:34:08.159510+0000","last_active":"2024-09-21T07:34:09.497584+0000","last_peered":"2024-09-21T07:34:09.497584+0000","last_clean":"2024-09-21T07:34:09.497584+0000","last_became_active":"2024-09-21T07:34:08.159191+0000","last_became_peered":"2024-09-21T07:34:08.159191+0000","last_unstale":"2024-09-21T07:34:09.497584+0000","last_undegraded":"2024-09-21T07:34:09.497584+0000","last_fullsized":"2024-09-21T07:34:09.497584+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:27.319525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496085+0000","last_change":"2024-09-21T07:34:07.155987+0000","last_active":"2024-09-21T07:34:09.496085+0000","last_peered":"2024-09-21T07:34:09.496085+0000","last_clean":"2024-09-21T07:34:09.496085+0000","last_became_active":"2024-09-21T07:34:07.155735+0000","last_became_peered":"2024-09-21T07:34:07.155735+0000","last_unstale":"2024-09-21T07:34:09.496085+0000","last_undegraded":"2024-09-21T07:34:09.496085+0000","last_fullsized":"2024-09-21T07:34:09.496085+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:23:21.954791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.143410+0000","last_change":"2024-09-21T07:34:07.143410+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.143410+0000","last_undegraded":"2024-09-21T07:34:07.143410+0000","last_fullsized":"2024-09-21T07:34:07.143410+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T07:34:07.159470+0000","last_change":"2024-09-21T07:34:07.155156+0000","last_active":"2024-09-21T07:34:07.159470+0000","last_peered":"2024-09-21T07:34:07.159470+0000","last_clean":"2024-09-21T07:34:07.159470+0000","last_became_active":"2024-09-21T07:34:07.154829+0000","last_became_peered":"2024-09-21T07:34:07.154829+0000","last_unstale":"2024-09-21T07:34:07.159470+0000","last_undegraded":"2024-09-21T07:34:07.159470+0000","last_fullsized":"2024-09-21T07:34:07.159470+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:28:14.831565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":131,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T07:34:07.154589+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:34:07.154589+0000","last_peered":"2024-09-21T07:34:07.154589+0000","last_clean":"2024-09-21T07:34:07.154589+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:34:07.154589+0000","last_undegraded":"2024-09-21T07:34:07.154589+0000","last_fullsized":"2024-09-21T07:34:07.154589+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1716,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23,"ondisk_log_size":23,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509456,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":933888,"data_stored":719721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":339968,"data_stored":129353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":352256,"data_stored":135620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509457,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31532,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340308,"statfs":{"total":96636764160,"available":96604475392,"internally_reserved":0,"allocated":352256,"data_stored":135620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31520,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340320,"statfs":{"total":96636764160,"available":96604487680,"internally_reserved":0,"allocated":339968,"data_stored":129353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509458,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":933888,"data_stored":719721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542162,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":385024,"data_stored":148799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542161,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":393216,"data_stored":150329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:34:12.664 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:34:12.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.923+0000 7fae67e00640 1 -- 172.21.15.39:0/3649242054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae68074040 msgr2=0x7fae6815c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:12.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.923+0000 7fae67e00640 1 --2- 172.21.15.39:0/3649242054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae68074040 0x7fae6815c340 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fae50005fd0 tx=0x7fae5002a540 comp rx=0 tx=0).stop 2024-09-21T07:34:12.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.923+0000 7fae67e00640 1 -- 172.21.15.39:0/3649242054 shutdown_connections 2024-09-21T07:34:12.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.923+0000 7fae67e00640 1 --2- 172.21.15.39:0/3649242054 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae680b4690 0x7fae680b4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:12.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.923+0000 7fae67e00640 1 --2- 172.21.15.39:0/3649242054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae68074040 0x7fae6815c340 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:12.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.923+0000 7fae67e00640 1 -- 172.21.15.39:0/3649242054 >> 172.21.15.39:0/3649242054 conn(0x7fae680cd6c0 msgr2=0x7fae680cdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:12.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.923+0000 7fae67e00640 1 -- 172.21.15.39:0/3649242054 shutdown_connections 2024-09-21T07:34:12.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.923+0000 7fae67e00640 1 -- 172.21.15.39:0/3649242054 wait complete. 2024-09-21T07:34:12.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.925+0000 7fae67e00640 1 Processor -- start 2024-09-21T07:34:12.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.925+0000 7fae67e00640 1 -- start start 2024-09-21T07:34:12.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.927+0000 7fae67e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae680b4690 0x7fae68152320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:12.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.927+0000 7fae67e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae680cea10 0x7fae681f8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:12.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.927+0000 7fae67e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae681f8bc0 0x7fae680ada50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:12.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.927+0000 7fae67e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fae680b5370 con 0x7fae680cea10 2024-09-21T07:34:12.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.927+0000 7fae67e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fae680b51f0 con 0x7fae681f8bc0 2024-09-21T07:34:12.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.927+0000 7fae67e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fae680b54f0 con 0x7fae680b4690 2024-09-21T07:34:12.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.927+0000 7fae5e000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae680cea10 0x7fae681f8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:12.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.927+0000 7fae5e000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae680cea10 0x7fae681f8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41414/0 (socket says 172.21.15.39:41414) 2024-09-21T07:34:12.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.927+0000 7fae5e000640 1 -- 172.21.15.39:0/2208728571 learned_addr learned my addr 172.21.15.39:0/2208728571 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:12.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.927+0000 7fae66a00640 1 --2- 172.21.15.39:0/2208728571 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae680b4690 0x7fae68152320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:12.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.927+0000 7fae67400640 1 --2- 172.21.15.39:0/2208728571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae681f8bc0 0x7fae680ada50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:12.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.928+0000 7fae5e000640 1 -- 172.21.15.39:0/2208728571 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae680b4690 msgr2=0x7fae68152320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:12.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.928+0000 7fae5e000640 1 --2- 172.21.15.39:0/2208728571 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae680b4690 0x7fae68152320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:12.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.928+0000 7fae5e000640 1 -- 172.21.15.39:0/2208728571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae681f8bc0 msgr2=0x7fae680ada50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:12.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.928+0000 7fae5e000640 1 --2- 172.21.15.39:0/2208728571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae681f8bc0 0x7fae680ada50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:12.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.928+0000 7fae5e000640 1 -- 172.21.15.39:0/2208728571 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae680b17b0 con 0x7fae680cea10 2024-09-21T07:34:12.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.928+0000 7fae5e000640 1 --2- 172.21.15.39:0/2208728571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae680cea10 0x7fae681f8680 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fae50034040 tx=0x7fae500038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:12.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.928+0000 7fae5f400640 1 -- 172.21.15.39:0/2208728571 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5003e030 con 0x7fae680cea10 2024-09-21T07:34:12.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.928+0000 7fae5f400640 1 -- 172.21.15.39:0/2208728571 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae50003d90 con 0x7fae680cea10 2024-09-21T07:34:12.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.928+0000 7fae5f400640 1 -- 172.21.15.39:0/2208728571 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae50035040 con 0x7fae680cea10 2024-09-21T07:34:12.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.928+0000 7fae67e00640 1 -- 172.21.15.39:0/2208728571 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae680b1a90 con 0x7fae680cea10 2024-09-21T07:34:12.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.928+0000 7fae67e00640 1 -- 172.21.15.39:0/2208728571 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae680b2050 con 0x7fae680cea10 2024-09-21T07:34:12.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.930+0000 7fae67e00640 1 -- 172.21.15.39:0/2208728571 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae68155750 con 0x7fae680cea10 2024-09-21T07:34:12.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.931+0000 7fae5f400640 1 -- 172.21.15.39:0/2208728571 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fae50041030 con 0x7fae680cea10 2024-09-21T07:34:12.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.932+0000 7fae5f400640 1 --2- 172.21.15.39:0/2208728571 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae44078bc0 0x7fae4407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:12.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.932+0000 7fae5f400640 1 -- 172.21.15.39:0/2208728571 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fae500bb360 con 0x7fae680cea10 2024-09-21T07:34:12.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.932+0000 7fae66a00640 1 --2- 172.21.15.39:0/2208728571 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae44078bc0 0x7fae4407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:12.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.933+0000 7fae66a00640 1 --2- 172.21.15.39:0/2208728571 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae44078bc0 0x7fae4407b0d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fae58000f60 tx=0x7fae58009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:12.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:12.936+0000 7fae5f400640 1 -- 172.21.15.39:0/2208728571 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fae50083140 con 0x7fae680cea10 2024-09-21T07:34:13.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.129+0000 7fae67e00640 1 -- 172.21.15.39:0/2208728571 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fae681f93d0 con 0x7fae44078bc0 2024-09-21T07:34:13.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.134+0000 7fae5f400640 1 -- 172.21.15.39:0/2208728571 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248165 (secure 0 0 0) 0x7fae681f93d0 con 0x7fae44078bc0 2024-09-21T07:34:13.139 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:34:13.140 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:34:13.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.138+0000 7fae67e00640 1 -- 172.21.15.39:0/2208728571 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae44078bc0 msgr2=0x7fae4407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:13.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.138+0000 7fae67e00640 1 --2- 172.21.15.39:0/2208728571 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae44078bc0 0x7fae4407b0d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fae58000f60 tx=0x7fae58009040 comp rx=0 tx=0).stop 2024-09-21T07:34:13.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.138+0000 7fae67e00640 1 -- 172.21.15.39:0/2208728571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae680cea10 msgr2=0x7fae681f8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:13.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.138+0000 7fae67e00640 1 --2- 172.21.15.39:0/2208728571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae680cea10 0x7fae681f8680 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fae50034040 tx=0x7fae500038a0 comp rx=0 tx=0).stop 2024-09-21T07:34:13.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.138+0000 7fae67e00640 1 -- 172.21.15.39:0/2208728571 shutdown_connections 2024-09-21T07:34:13.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.138+0000 7fae67e00640 1 --2- 172.21.15.39:0/2208728571 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae44078bc0 0x7fae4407b0d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:13.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.138+0000 7fae67e00640 1 --2- 172.21.15.39:0/2208728571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae681f8bc0 0x7fae680ada50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:13.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.138+0000 7fae67e00640 1 --2- 172.21.15.39:0/2208728571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae680cea10 0x7fae681f8680 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:13.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.138+0000 7fae67e00640 1 --2- 172.21.15.39:0/2208728571 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae680b4690 0x7fae68152320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:13.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.138+0000 7fae67e00640 1 -- 172.21.15.39:0/2208728571 >> 172.21.15.39:0/2208728571 conn(0x7fae680cd6c0 msgr2=0x7fae680b59c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:13.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.138+0000 7fae67e00640 1 -- 172.21.15.39:0/2208728571 shutdown_connections 2024-09-21T07:34:13.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.138+0000 7fae67e00640 1 -- 172.21.15.39:0/2208728571 wait complete. 2024-09-21T07:34:13.162 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-09-21T07:34:11.547441+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592084,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":129,"num_write_kb":1639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":95,"ondisk_log_size":95,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236800,"kb_used_data":3936,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737920,"statfs":{"total":773094113280,"available":772851630080,"internally_reserved":0,"allocated":4030464,"data_stored":2268516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1716,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.424480"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.136358+0000","last_change":"2024-09-21T07:34:07.136358+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.136358+0000","last_undegraded":"2024-09-21T07:34:07.136358+0000","last_fullsized":"2024-09-21T07:34:07.136358+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496812+0000","last_change":"2024-09-21T07:34:08.163227+0000","last_active":"2024-09-21T07:34:09.496812+0000","last_peered":"2024-09-21T07:34:09.496812+0000","last_clean":"2024-09-21T07:34:09.496812+0000","last_became_active":"2024-09-21T07:34:08.162915+0000","last_became_peered":"2024-09-21T07:34:08.162915+0000","last_unstale":"2024-09-21T07:34:09.496812+0000","last_undegraded":"2024-09-21T07:34:09.496812+0000","last_fullsized":"2024-09-21T07:34:09.496812+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:16.292830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.129770+0000","last_change":"2024-09-21T07:34:07.129770+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.129770+0000","last_undegraded":"2024-09-21T07:34:07.129770+0000","last_fullsized":"2024-09-21T07:34:07.129770+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.144427+0000","last_change":"2024-09-21T07:34:07.144427+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.144427+0000","last_undegraded":"2024-09-21T07:34:07.144427+0000","last_fullsized":"2024-09-21T07:34:07.144427+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.130027+0000","last_change":"2024-09-21T07:34:07.130027+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.130027+0000","last_undegraded":"2024-09-21T07:34:07.130027+0000","last_fullsized":"2024-09-21T07:34:07.130027+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496422+0000","last_change":"2024-09-21T07:34:08.161341+0000","last_active":"2024-09-21T07:34:09.496422+0000","last_peered":"2024-09-21T07:34:09.496422+0000","last_clean":"2024-09-21T07:34:09.496422+0000","last_became_active":"2024-09-21T07:34:08.161037+0000","last_became_peered":"2024-09-21T07:34:08.161037+0000","last_unstale":"2024-09-21T07:34:09.496422+0000","last_undegraded":"2024-09-21T07:34:09.496422+0000","last_fullsized":"2024-09-21T07:34:09.496422+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:52.361740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.142031+0000","last_change":"2024-09-21T07:34:07.142031+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.142031+0000","last_undegraded":"2024-09-21T07:34:07.142031+0000","last_fullsized":"2024-09-21T07:34:07.142031+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.147161+0000","last_change":"2024-09-21T07:34:07.147161+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.147161+0000","last_undegraded":"2024-09-21T07:34:07.147161+0000","last_fullsized":"2024-09-21T07:34:07.147161+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497858+0000","last_change":"2024-09-21T07:34:08.170445+0000","last_active":"2024-09-21T07:34:09.497858+0000","last_peered":"2024-09-21T07:34:09.497858+0000","last_clean":"2024-09-21T07:34:09.497858+0000","last_became_active":"2024-09-21T07:34:08.170115+0000","last_became_peered":"2024-09-21T07:34:08.170115+0000","last_unstale":"2024-09-21T07:34:09.497858+0000","last_undegraded":"2024-09-21T07:34:09.497858+0000","last_fullsized":"2024-09-21T07:34:09.497858+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:32.224498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.140904+0000","last_change":"2024-09-21T07:34:07.140904+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.140904+0000","last_undegraded":"2024-09-21T07:34:07.140904+0000","last_fullsized":"2024-09-21T07:34:07.140904+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.147060+0000","last_change":"2024-09-21T07:34:07.147060+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.147060+0000","last_undegraded":"2024-09-21T07:34:07.147060+0000","last_fullsized":"2024-09-21T07:34:07.147060+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.131743+0000","last_change":"2024-09-21T07:34:07.131743+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.131743+0000","last_undegraded":"2024-09-21T07:34:07.131743+0000","last_fullsized":"2024-09-21T07:34:07.131743+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.129452+0000","last_change":"2024-09-21T07:34:07.129452+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.129452+0000","last_undegraded":"2024-09-21T07:34:07.129452+0000","last_fullsized":"2024-09-21T07:34:07.129452+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496432+0000","last_change":"2024-09-21T07:34:08.167950+0000","last_active":"2024-09-21T07:34:09.496432+0000","last_peered":"2024-09-21T07:34:09.496432+0000","last_clean":"2024-09-21T07:34:09.496432+0000","last_became_active":"2024-09-21T07:34:08.167687+0000","last_became_peered":"2024-09-21T07:34:08.167687+0000","last_unstale":"2024-09-21T07:34:09.496432+0000","last_undegraded":"2024-09-21T07:34:09.496432+0000","last_fullsized":"2024-09-21T07:34:09.496432+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:44:01.238051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.130177+0000","last_change":"2024-09-21T07:34:07.130177+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.130177+0000","last_undegraded":"2024-09-21T07:34:07.130177+0000","last_fullsized":"2024-09-21T07:34:07.130177+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.140164+0000","last_change":"2024-09-21T07:34:07.140164+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.140164+0000","last_undegraded":"2024-09-21T07:34:07.140164+0000","last_fullsized":"2024-09-21T07:34:07.140164+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.150046+0000","last_change":"2024-09-21T07:34:07.150046+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.150046+0000","last_undegraded":"2024-09-21T07:34:07.150046+0000","last_fullsized":"2024-09-21T07:34:07.150046+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496674+0000","last_change":"2024-09-21T07:34:08.153003+0000","last_active":"2024-09-21T07:34:09.496674+0000","last_peered":"2024-09-21T07:34:09.496674+0000","last_clean":"2024-09-21T07:34:09.496674+0000","last_became_active":"2024-09-21T07:34:08.150882+0000","last_became_peered":"2024-09-21T07:34:08.150882+0000","last_unstale":"2024-09-21T07:34:09.496674+0000","last_undegraded":"2024-09-21T07:34:09.496674+0000","last_fullsized":"2024-09-21T07:34:09.496674+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:38.611907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496759+0000","last_change":"2024-09-21T07:34:08.158999+0000","last_active":"2024-09-21T07:34:09.496759+0000","last_peered":"2024-09-21T07:34:09.496759+0000","last_clean":"2024-09-21T07:34:09.496759+0000","last_became_active":"2024-09-21T07:34:08.158712+0000","last_became_peered":"2024-09-21T07:34:08.158712+0000","last_unstale":"2024-09-21T07:34:09.496759+0000","last_undegraded":"2024-09-21T07:34:09.496759+0000","last_fullsized":"2024-09-21T07:34:09.496759+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:34:04.145536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.129482+0000","last_change":"2024-09-21T07:34:07.129482+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.129482+0000","last_undegraded":"2024-09-21T07:34:07.129482+0000","last_fullsized":"2024-09-21T07:34:07.129482+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496796+0000","last_change":"2024-09-21T07:34:08.168578+0000","last_active":"2024-09-21T07:34:09.496796+0000","last_peered":"2024-09-21T07:34:09.496796+0000","last_clean":"2024-09-21T07:34:09.496796+0000","last_became_active":"2024-09-21T07:34:08.168235+0000","last_became_peered":"2024-09-21T07:34:08.168235+0000","last_unstale":"2024-09-21T07:34:09.496796+0000","last_undegraded":"2024-09-21T07:34:09.496796+0000","last_fullsized":"2024-09-21T07:34:09.496796+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:18.132264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496889+0000","last_change":"2024-09-21T07:34:08.159192+0000","last_active":"2024-09-21T07:34:09.496889+0000","last_peered":"2024-09-21T07:34:09.496889+0000","last_clean":"2024-09-21T07:34:09.496889+0000","last_became_active":"2024-09-21T07:34:08.158860+0000","last_became_peered":"2024-09-21T07:34:08.158860+0000","last_unstale":"2024-09-21T07:34:09.496889+0000","last_undegraded":"2024-09-21T07:34:09.496889+0000","last_fullsized":"2024-09-21T07:34:09.496889+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:48.287489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.132277+0000","last_change":"2024-09-21T07:34:07.132277+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.132277+0000","last_undegraded":"2024-09-21T07:34:07.132277+0000","last_fullsized":"2024-09-21T07:34:07.132277+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.129770+0000","last_change":"2024-09-21T07:34:07.129770+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.129770+0000","last_undegraded":"2024-09-21T07:34:07.129770+0000","last_fullsized":"2024-09-21T07:34:07.129770+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.135980+0000","last_change":"2024-09-21T07:34:07.135980+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.135980+0000","last_undegraded":"2024-09-21T07:34:07.135980+0000","last_fullsized":"2024-09-21T07:34:07.135980+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496941+0000","last_change":"2024-09-21T07:34:08.158447+0000","last_active":"2024-09-21T07:34:09.496941+0000","last_peered":"2024-09-21T07:34:09.496941+0000","last_clean":"2024-09-21T07:34:09.496941+0000","last_became_active":"2024-09-21T07:34:08.158138+0000","last_became_peered":"2024-09-21T07:34:08.158138+0000","last_unstale":"2024-09-21T07:34:09.496941+0000","last_undegraded":"2024-09-21T07:34:09.496941+0000","last_fullsized":"2024-09-21T07:34:09.496941+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:05.155134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.136992+0000","last_change":"2024-09-21T07:34:07.136992+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.136992+0000","last_undegraded":"2024-09-21T07:34:07.136992+0000","last_fullsized":"2024-09-21T07:34:07.136992+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.129413+0000","last_change":"2024-09-21T07:34:07.129413+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.129413+0000","last_undegraded":"2024-09-21T07:34:07.129413+0000","last_fullsized":"2024-09-21T07:34:07.129413+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497087+0000","last_change":"2024-09-21T07:34:08.153475+0000","last_active":"2024-09-21T07:34:09.497087+0000","last_peered":"2024-09-21T07:34:09.497087+0000","last_clean":"2024-09-21T07:34:09.497087+0000","last_became_active":"2024-09-21T07:34:08.152735+0000","last_became_peered":"2024-09-21T07:34:08.152735+0000","last_unstale":"2024-09-21T07:34:09.497087+0000","last_undegraded":"2024-09-21T07:34:09.497087+0000","last_fullsized":"2024-09-21T07:34:09.497087+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:42.395864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.147613+0000","last_change":"2024-09-21T07:34:07.147613+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.147613+0000","last_undegraded":"2024-09-21T07:34:07.147613+0000","last_fullsized":"2024-09-21T07:34:07.147613+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.136369+0000","last_change":"2024-09-21T07:34:07.136369+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.136369+0000","last_undegraded":"2024-09-21T07:34:07.136369+0000","last_fullsized":"2024-09-21T07:34:07.136369+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.146626+0000","last_change":"2024-09-21T07:34:07.146626+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.146626+0000","last_undegraded":"2024-09-21T07:34:07.146626+0000","last_fullsized":"2024-09-21T07:34:07.146626+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.131763+0000","last_change":"2024-09-21T07:34:07.131763+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.131763+0000","last_undegraded":"2024-09-21T07:34:07.131763+0000","last_fullsized":"2024-09-21T07:34:07.131763+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T07:34:07.159481+0000","last_change":"2024-09-21T07:34:07.155982+0000","last_active":"2024-09-21T07:34:07.159481+0000","last_peered":"2024-09-21T07:34:07.159481+0000","last_clean":"2024-09-21T07:34:07.159481+0000","last_became_active":"2024-09-21T07:34:07.155500+0000","last_became_peered":"2024-09-21T07:34:07.155500+0000","last_unstale":"2024-09-21T07:34:07.159481+0000","last_undegraded":"2024-09-21T07:34:07.159481+0000","last_fullsized":"2024-09-21T07:34:07.159481+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:43:17.626826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.135832+0000","last_change":"2024-09-21T07:34:07.135832+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.135832+0000","last_undegraded":"2024-09-21T07:34:07.135832+0000","last_fullsized":"2024-09-21T07:34:07.135832+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T07:34:07.154433+0000","last_change":"2024-09-21T07:34:07.148229+0000","last_active":"2024-09-21T07:34:07.154433+0000","last_peered":"2024-09-21T07:34:07.154433+0000","last_clean":"2024-09-21T07:34:07.154433+0000","last_became_active":"2024-09-21T07:34:07.147920+0000","last_became_peered":"2024-09-21T07:34:07.147920+0000","last_unstale":"2024-09-21T07:34:07.154433+0000","last_undegraded":"2024-09-21T07:34:07.154433+0000","last_fullsized":"2024-09-21T07:34:07.154433+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:43:17.626826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497703+0000","last_change":"2024-09-21T07:34:08.162272+0000","last_active":"2024-09-21T07:34:09.497703+0000","last_peered":"2024-09-21T07:34:09.497703+0000","last_clean":"2024-09-21T07:34:09.497703+0000","last_became_active":"2024-09-21T07:34:08.162031+0000","last_became_peered":"2024-09-21T07:34:08.162031+0000","last_unstale":"2024-09-21T07:34:09.497703+0000","last_undegraded":"2024-09-21T07:34:09.497703+0000","last_fullsized":"2024-09-21T07:34:09.497703+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:48.649163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496548+0000","last_change":"2024-09-21T07:34:07.162816+0000","last_active":"2024-09-21T07:34:09.496548+0000","last_peered":"2024-09-21T07:34:09.496548+0000","last_clean":"2024-09-21T07:34:09.496548+0000","last_became_active":"2024-09-21T07:34:07.162478+0000","last_became_peered":"2024-09-21T07:34:07.162478+0000","last_unstale":"2024-09-21T07:34:09.496548+0000","last_undegraded":"2024-09-21T07:34:09.496548+0000","last_fullsized":"2024-09-21T07:34:09.496548+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:18:03.659966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.136432+0000","last_change":"2024-09-21T07:34:07.136432+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.136432+0000","last_undegraded":"2024-09-21T07:34:07.136432+0000","last_fullsized":"2024-09-21T07:34:07.136432+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T07:34:07.159878+0000","last_change":"2024-09-21T07:34:07.151195+0000","last_active":"2024-09-21T07:34:07.159878+0000","last_peered":"2024-09-21T07:34:07.159878+0000","last_clean":"2024-09-21T07:34:07.159878+0000","last_became_active":"2024-09-21T07:34:07.150871+0000","last_became_peered":"2024-09-21T07:34:07.150871+0000","last_unstale":"2024-09-21T07:34:07.159878+0000","last_undegraded":"2024-09-21T07:34:07.159878+0000","last_fullsized":"2024-09-21T07:34:07.159878+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:41:43.994817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.143895+0000","last_change":"2024-09-21T07:34:07.143895+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.143895+0000","last_undegraded":"2024-09-21T07:34:07.143895+0000","last_fullsized":"2024-09-21T07:34:07.143895+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497041+0000","last_change":"2024-09-21T07:34:07.154783+0000","last_active":"2024-09-21T07:34:09.497041+0000","last_peered":"2024-09-21T07:34:09.497041+0000","last_clean":"2024-09-21T07:34:09.497041+0000","last_became_active":"2024-09-21T07:34:07.154378+0000","last_became_peered":"2024-09-21T07:34:07.154378+0000","last_unstale":"2024-09-21T07:34:09.497041+0000","last_undegraded":"2024-09-21T07:34:09.497041+0000","last_fullsized":"2024-09-21T07:34:09.497041+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:16.041455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.131554+0000","last_change":"2024-09-21T07:34:07.131554+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.131554+0000","last_undegraded":"2024-09-21T07:34:07.131554+0000","last_fullsized":"2024-09-21T07:34:07.131554+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496886+0000","last_change":"2024-09-21T07:34:07.153288+0000","last_active":"2024-09-21T07:34:09.496886+0000","last_peered":"2024-09-21T07:34:09.496886+0000","last_clean":"2024-09-21T07:34:09.496886+0000","last_became_active":"2024-09-21T07:34:07.152949+0000","last_became_peered":"2024-09-21T07:34:07.152949+0000","last_unstale":"2024-09-21T07:34:09.496886+0000","last_undegraded":"2024-09-21T07:34:09.496886+0000","last_fullsized":"2024-09-21T07:34:09.496886+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:54:43.804810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496478+0000","last_change":"2024-09-21T07:34:08.166316+0000","last_active":"2024-09-21T07:34:09.496478+0000","last_peered":"2024-09-21T07:34:09.496478+0000","last_clean":"2024-09-21T07:34:09.496478+0000","last_became_active":"2024-09-21T07:34:08.165006+0000","last_became_peered":"2024-09-21T07:34:08.165006+0000","last_unstale":"2024-09-21T07:34:09.496478+0000","last_undegraded":"2024-09-21T07:34:09.496478+0000","last_fullsized":"2024-09-21T07:34:09.496478+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:27.528708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496037+0000","last_change":"2024-09-21T07:34:07.141466+0000","last_active":"2024-09-21T07:34:09.496037+0000","last_peered":"2024-09-21T07:34:09.496037+0000","last_clean":"2024-09-21T07:34:09.496037+0000","last_became_active":"2024-09-21T07:34:07.141003+0000","last_became_peered":"2024-09-21T07:34:07.141003+0000","last_unstale":"2024-09-21T07:34:09.496037+0000","last_undegraded":"2024-09-21T07:34:09.496037+0000","last_fullsized":"2024-09-21T07:34:09.496037+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:52:33.957055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.137582+0000","last_change":"2024-09-21T07:34:07.137582+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.137582+0000","last_undegraded":"2024-09-21T07:34:07.137582+0000","last_fullsized":"2024-09-21T07:34:07.137582+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496450+0000","last_change":"2024-09-21T07:34:07.142126+0000","last_active":"2024-09-21T07:34:09.496450+0000","last_peered":"2024-09-21T07:34:09.496450+0000","last_clean":"2024-09-21T07:34:09.496450+0000","last_became_active":"2024-09-21T07:34:07.141713+0000","last_became_peered":"2024-09-21T07:34:07.141713+0000","last_unstale":"2024-09-21T07:34:09.496450+0000","last_undegraded":"2024-09-21T07:34:09.496450+0000","last_fullsized":"2024-09-21T07:34:09.496450+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:58.789677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.129621+0000","last_change":"2024-09-21T07:34:07.129621+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.129621+0000","last_undegraded":"2024-09-21T07:34:07.129621+0000","last_fullsized":"2024-09-21T07:34:07.129621+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496527+0000","last_change":"2024-09-21T07:34:07.154435+0000","last_active":"2024-09-21T07:34:09.496527+0000","last_peered":"2024-09-21T07:34:09.496527+0000","last_clean":"2024-09-21T07:34:09.496527+0000","last_became_active":"2024-09-21T07:34:07.154127+0000","last_became_peered":"2024-09-21T07:34:07.154127+0000","last_unstale":"2024-09-21T07:34:09.496527+0000","last_undegraded":"2024-09-21T07:34:09.496527+0000","last_fullsized":"2024-09-21T07:34:09.496527+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:11:21.678594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496615+0000","last_change":"2024-09-21T07:34:08.170301+0000","last_active":"2024-09-21T07:34:09.496615+0000","last_peered":"2024-09-21T07:34:09.496615+0000","last_clean":"2024-09-21T07:34:09.496615+0000","last_became_active":"2024-09-21T07:34:08.169925+0000","last_became_peered":"2024-09-21T07:34:08.169925+0000","last_unstale":"2024-09-21T07:34:09.496615+0000","last_undegraded":"2024-09-21T07:34:09.496615+0000","last_fullsized":"2024-09-21T07:34:09.496615+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:31.073725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T07:34:07.159973+0000","last_change":"2024-09-21T07:34:07.152298+0000","last_active":"2024-09-21T07:34:07.159973+0000","last_peered":"2024-09-21T07:34:07.159973+0000","last_clean":"2024-09-21T07:34:07.159973+0000","last_became_active":"2024-09-21T07:34:07.151921+0000","last_became_peered":"2024-09-21T07:34:07.151921+0000","last_unstale":"2024-09-21T07:34:07.159973+0000","last_undegraded":"2024-09-21T07:34:07.159973+0000","last_fullsized":"2024-09-21T07:34:07.159973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:59:04.464801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496907+0000","last_change":"2024-09-21T07:34:08.160957+0000","last_active":"2024-09-21T07:34:09.496907+0000","last_peered":"2024-09-21T07:34:09.496907+0000","last_clean":"2024-09-21T07:34:09.496907+0000","last_became_active":"2024-09-21T07:34:08.160601+0000","last_became_peered":"2024-09-21T07:34:08.160601+0000","last_unstale":"2024-09-21T07:34:09.496907+0000","last_undegraded":"2024-09-21T07:34:09.496907+0000","last_fullsized":"2024-09-21T07:34:09.496907+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:19:16.261092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T07:34:07.157782+0000","last_change":"2024-09-21T07:34:07.153099+0000","last_active":"2024-09-21T07:34:07.157782+0000","last_peered":"2024-09-21T07:34:07.157782+0000","last_clean":"2024-09-21T07:34:07.157782+0000","last_became_active":"2024-09-21T07:34:07.152790+0000","last_became_peered":"2024-09-21T07:34:07.152790+0000","last_unstale":"2024-09-21T07:34:07.157782+0000","last_undegraded":"2024-09-21T07:34:07.157782+0000","last_fullsized":"2024-09-21T07:34:07.157782+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:57:12.586864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.131283+0000","last_change":"2024-09-21T07:34:07.131283+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.131283+0000","last_undegraded":"2024-09-21T07:34:07.131283+0000","last_fullsized":"2024-09-21T07:34:07.131283+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":21,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496721+0000","last_change":"2024-09-21T07:34:07.162976+0000","last_active":"2024-09-21T07:34:09.496721+0000","last_peered":"2024-09-21T07:34:09.496721+0000","last_clean":"2024-09-21T07:34:09.496721+0000","last_became_active":"2024-09-21T07:34:07.162698+0000","last_became_peered":"2024-09-21T07:34:07.162698+0000","last_unstale":"2024-09-21T07:34:09.496721+0000","last_undegraded":"2024-09-21T07:34:09.496721+0000","last_fullsized":"2024-09-21T07:34:09.496721+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:01:04.212051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.135901+0000","last_change":"2024-09-21T07:34:07.135901+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.135901+0000","last_undegraded":"2024-09-21T07:34:07.135901+0000","last_fullsized":"2024-09-21T07:34:07.135901+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497188+0000","last_change":"2024-09-21T07:34:07.150104+0000","last_active":"2024-09-21T07:34:09.497188+0000","last_peered":"2024-09-21T07:34:09.497188+0000","last_clean":"2024-09-21T07:34:09.497188+0000","last_became_active":"2024-09-21T07:34:07.149757+0000","last_became_peered":"2024-09-21T07:34:07.149757+0000","last_unstale":"2024-09-21T07:34:09.497188+0000","last_undegraded":"2024-09-21T07:34:09.497188+0000","last_fullsized":"2024-09-21T07:34:09.497188+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:17.255537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496697+0000","last_change":"2024-09-21T07:34:08.166843+0000","last_active":"2024-09-21T07:34:09.496697+0000","last_peered":"2024-09-21T07:34:09.496697+0000","last_clean":"2024-09-21T07:34:09.496697+0000","last_became_active":"2024-09-21T07:34:08.166516+0000","last_became_peered":"2024-09-21T07:34:08.166516+0000","last_unstale":"2024-09-21T07:34:09.496697+0000","last_undegraded":"2024-09-21T07:34:09.496697+0000","last_fullsized":"2024-09-21T07:34:09.496697+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:39.634059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T07:34:07.162417+0000","last_change":"2024-09-21T07:34:07.160820+0000","last_active":"2024-09-21T07:34:07.162417+0000","last_peered":"2024-09-21T07:34:07.162417+0000","last_clean":"2024-09-21T07:34:07.162417+0000","last_became_active":"2024-09-21T07:34:07.160364+0000","last_became_peered":"2024-09-21T07:34:07.160364+0000","last_unstale":"2024-09-21T07:34:07.162417+0000","last_undegraded":"2024-09-21T07:34:07.162417+0000","last_fullsized":"2024-09-21T07:34:07.162417+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:20:00.284603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.143090+0000","last_change":"2024-09-21T07:34:07.143090+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.143090+0000","last_undegraded":"2024-09-21T07:34:07.143090+0000","last_fullsized":"2024-09-21T07:34:07.143090+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496142+0000","last_change":"2024-09-21T07:34:07.153626+0000","last_active":"2024-09-21T07:34:09.496142+0000","last_peered":"2024-09-21T07:34:09.496142+0000","last_clean":"2024-09-21T07:34:09.496142+0000","last_became_active":"2024-09-21T07:34:07.153334+0000","last_became_peered":"2024-09-21T07:34:07.153334+0000","last_unstale":"2024-09-21T07:34:09.496142+0000","last_undegraded":"2024-09-21T07:34:09.496142+0000","last_fullsized":"2024-09-21T07:34:09.496142+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:48.214805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.135638+0000","last_change":"2024-09-21T07:34:07.135638+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.135638+0000","last_undegraded":"2024-09-21T07:34:07.135638+0000","last_fullsized":"2024-09-21T07:34:07.135638+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496343+0000","last_change":"2024-09-21T07:34:07.161149+0000","last_active":"2024-09-21T07:34:09.496343+0000","last_peered":"2024-09-21T07:34:09.496343+0000","last_clean":"2024-09-21T07:34:09.496343+0000","last_became_active":"2024-09-21T07:34:07.160882+0000","last_became_peered":"2024-09-21T07:34:07.160882+0000","last_unstale":"2024-09-21T07:34:09.496343+0000","last_undegraded":"2024-09-21T07:34:09.496343+0000","last_fullsized":"2024-09-21T07:34:09.496343+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:55.427089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497420+0000","last_change":"2024-09-21T07:34:08.167375+0000","last_active":"2024-09-21T07:34:09.497420+0000","last_peered":"2024-09-21T07:34:09.497420+0000","last_clean":"2024-09-21T07:34:09.497420+0000","last_became_active":"2024-09-21T07:34:08.165919+0000","last_became_peered":"2024-09-21T07:34:08.165919+0000","last_unstale":"2024-09-21T07:34:09.497420+0000","last_undegraded":"2024-09-21T07:34:09.497420+0000","last_fullsized":"2024-09-21T07:34:09.497420+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:30.388748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T07:34:07.161732+0000","last_change":"2024-09-21T07:34:07.154981+0000","last_active":"2024-09-21T07:34:07.161732+0000","last_peered":"2024-09-21T07:34:07.161732+0000","last_clean":"2024-09-21T07:34:07.161732+0000","last_became_active":"2024-09-21T07:34:07.154706+0000","last_became_peered":"2024-09-21T07:34:07.154706+0000","last_unstale":"2024-09-21T07:34:07.161732+0000","last_undegraded":"2024-09-21T07:34:07.161732+0000","last_fullsized":"2024-09-21T07:34:07.161732+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:00.242190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.131452+0000","last_change":"2024-09-21T07:34:07.131452+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.131452+0000","last_undegraded":"2024-09-21T07:34:07.131452+0000","last_fullsized":"2024-09-21T07:34:07.131452+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497519+0000","last_change":"2024-09-21T07:34:07.149293+0000","last_active":"2024-09-21T07:34:09.497519+0000","last_peered":"2024-09-21T07:34:09.497519+0000","last_clean":"2024-09-21T07:34:09.497519+0000","last_became_active":"2024-09-21T07:34:07.148938+0000","last_became_peered":"2024-09-21T07:34:07.148938+0000","last_unstale":"2024-09-21T07:34:09.497519+0000","last_undegraded":"2024-09-21T07:34:09.497519+0000","last_fullsized":"2024-09-21T07:34:09.497519+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:59.434905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.137799+0000","last_change":"2024-09-21T07:34:07.137799+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.137799+0000","last_undegraded":"2024-09-21T07:34:07.137799+0000","last_fullsized":"2024-09-21T07:34:07.137799+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497479+0000","last_change":"2024-09-21T07:34:07.148308+0000","last_active":"2024-09-21T07:34:09.497479+0000","last_peered":"2024-09-21T07:34:09.497479+0000","last_clean":"2024-09-21T07:34:09.497479+0000","last_became_active":"2024-09-21T07:34:07.148001+0000","last_became_peered":"2024-09-21T07:34:07.148001+0000","last_unstale":"2024-09-21T07:34:09.497479+0000","last_undegraded":"2024-09-21T07:34:09.497479+0000","last_fullsized":"2024-09-21T07:34:09.497479+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:54:32.939044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.129855+0000","last_change":"2024-09-21T07:34:07.129855+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.129855+0000","last_undegraded":"2024-09-21T07:34:07.129855+0000","last_fullsized":"2024-09-21T07:34:07.129855+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497624+0000","last_change":"2024-09-21T07:34:07.152996+0000","last_active":"2024-09-21T07:34:09.497624+0000","last_peered":"2024-09-21T07:34:09.497624+0000","last_clean":"2024-09-21T07:34:09.497624+0000","last_became_active":"2024-09-21T07:34:07.152627+0000","last_became_peered":"2024-09-21T07:34:07.152627+0000","last_unstale":"2024-09-21T07:34:09.497624+0000","last_undegraded":"2024-09-21T07:34:09.497624+0000","last_fullsized":"2024-09-21T07:34:09.497624+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:24:42.920690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.128256+0000","last_change":"2024-09-21T07:34:07.128256+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.128256+0000","last_undegraded":"2024-09-21T07:34:07.128256+0000","last_fullsized":"2024-09-21T07:34:07.128256+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T07:34:07.159940+0000","last_change":"2024-09-21T07:34:07.151886+0000","last_active":"2024-09-21T07:34:07.159940+0000","last_peered":"2024-09-21T07:34:07.159940+0000","last_clean":"2024-09-21T07:34:07.159940+0000","last_became_active":"2024-09-21T07:34:07.150347+0000","last_became_peered":"2024-09-21T07:34:07.150347+0000","last_unstale":"2024-09-21T07:34:07.159940+0000","last_undegraded":"2024-09-21T07:34:07.159940+0000","last_fullsized":"2024-09-21T07:34:07.159940+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:53.036159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.128809+0000","last_change":"2024-09-21T07:34:07.128809+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.128809+0000","last_undegraded":"2024-09-21T07:34:07.128809+0000","last_fullsized":"2024-09-21T07:34:07.128809+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T07:34:07.162084+0000","last_change":"2024-09-21T07:34:07.161337+0000","last_active":"2024-09-21T07:34:07.162084+0000","last_peered":"2024-09-21T07:34:07.162084+0000","last_clean":"2024-09-21T07:34:07.162084+0000","last_became_active":"2024-09-21T07:34:07.161055+0000","last_became_peered":"2024-09-21T07:34:07.161055+0000","last_unstale":"2024-09-21T07:34:07.162084+0000","last_undegraded":"2024-09-21T07:34:07.162084+0000","last_fullsized":"2024-09-21T07:34:07.162084+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:53.036159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.128188+0000","last_change":"2024-09-21T07:34:07.128188+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.128188+0000","last_undegraded":"2024-09-21T07:34:07.128188+0000","last_fullsized":"2024-09-21T07:34:07.128188+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497320+0000","last_change":"2024-09-21T07:34:07.148155+0000","last_active":"2024-09-21T07:34:09.497320+0000","last_peered":"2024-09-21T07:34:09.497320+0000","last_clean":"2024-09-21T07:34:09.497320+0000","last_became_active":"2024-09-21T07:34:07.147855+0000","last_became_peered":"2024-09-21T07:34:07.147855+0000","last_unstale":"2024-09-21T07:34:09.497320+0000","last_undegraded":"2024-09-21T07:34:09.497320+0000","last_fullsized":"2024-09-21T07:34:09.497320+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:41:10.667777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.142621+0000","last_change":"2024-09-21T07:34:07.142621+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.142621+0000","last_undegraded":"2024-09-21T07:34:07.142621+0000","last_fullsized":"2024-09-21T07:34:07.142621+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T07:34:07.160117+0000","last_change":"2024-09-21T07:34:07.156707+0000","last_active":"2024-09-21T07:34:07.160117+0000","last_peered":"2024-09-21T07:34:07.160117+0000","last_clean":"2024-09-21T07:34:07.160117+0000","last_became_active":"2024-09-21T07:34:07.156326+0000","last_became_peered":"2024-09-21T07:34:07.156326+0000","last_unstale":"2024-09-21T07:34:07.160117+0000","last_undegraded":"2024-09-21T07:34:07.160117+0000","last_fullsized":"2024-09-21T07:34:07.160117+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:19.737233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.130004+0000","last_change":"2024-09-21T07:34:07.130004+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.130004+0000","last_undegraded":"2024-09-21T07:34:07.130004+0000","last_fullsized":"2024-09-21T07:34:07.130004+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T07:34:07.161433+0000","last_change":"2024-09-21T07:34:07.153252+0000","last_active":"2024-09-21T07:34:07.161433+0000","last_peered":"2024-09-21T07:34:07.161433+0000","last_clean":"2024-09-21T07:34:07.161433+0000","last_became_active":"2024-09-21T07:34:07.152932+0000","last_became_peered":"2024-09-21T07:34:07.152932+0000","last_unstale":"2024-09-21T07:34:07.161433+0000","last_undegraded":"2024-09-21T07:34:07.161433+0000","last_fullsized":"2024-09-21T07:34:07.161433+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:31.639108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.130642+0000","last_change":"2024-09-21T07:34:07.130642+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.130642+0000","last_undegraded":"2024-09-21T07:34:07.130642+0000","last_fullsized":"2024-09-21T07:34:07.130642+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T07:34:07.163218+0000","last_change":"2024-09-21T07:34:07.161285+0000","last_active":"2024-09-21T07:34:07.163218+0000","last_peered":"2024-09-21T07:34:07.163218+0000","last_clean":"2024-09-21T07:34:07.163218+0000","last_became_active":"2024-09-21T07:34:07.160925+0000","last_became_peered":"2024-09-21T07:34:07.160925+0000","last_unstale":"2024-09-21T07:34:07.163218+0000","last_undegraded":"2024-09-21T07:34:07.163218+0000","last_fullsized":"2024-09-21T07:34:07.163218+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:26.882263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.132803+0000","last_change":"2024-09-21T07:34:07.132803+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.132803+0000","last_undegraded":"2024-09-21T07:34:07.132803+0000","last_fullsized":"2024-09-21T07:34:07.132803+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T07:34:07.161782+0000","last_change":"2024-09-21T07:34:07.151263+0000","last_active":"2024-09-21T07:34:07.161782+0000","last_peered":"2024-09-21T07:34:07.161782+0000","last_clean":"2024-09-21T07:34:07.161782+0000","last_became_active":"2024-09-21T07:34:07.150981+0000","last_became_peered":"2024-09-21T07:34:07.150981+0000","last_unstale":"2024-09-21T07:34:07.161782+0000","last_undegraded":"2024-09-21T07:34:07.161782+0000","last_fullsized":"2024-09-21T07:34:07.161782+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:06.174996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.135692+0000","last_change":"2024-09-21T07:34:07.135692+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.135692+0000","last_undegraded":"2024-09-21T07:34:07.135692+0000","last_fullsized":"2024-09-21T07:34:07.135692+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T07:34:07.149962+0000","last_change":"2024-09-21T07:34:07.139885+0000","last_active":"2024-09-21T07:34:07.149962+0000","last_peered":"2024-09-21T07:34:07.149962+0000","last_clean":"2024-09-21T07:34:07.149962+0000","last_became_active":"2024-09-21T07:34:07.139560+0000","last_became_peered":"2024-09-21T07:34:07.139560+0000","last_unstale":"2024-09-21T07:34:07.149962+0000","last_undegraded":"2024-09-21T07:34:07.149962+0000","last_fullsized":"2024-09-21T07:34:07.149962+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:28:14.831565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.132414+0000","last_change":"2024-09-21T07:34:07.132414+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.132414+0000","last_undegraded":"2024-09-21T07:34:07.132414+0000","last_fullsized":"2024-09-21T07:34:07.132414+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T07:34:07.161575+0000","last_change":"2024-09-21T07:34:07.152465+0000","last_active":"2024-09-21T07:34:07.161575+0000","last_peered":"2024-09-21T07:34:07.161575+0000","last_clean":"2024-09-21T07:34:07.161575+0000","last_became_active":"2024-09-21T07:34:07.152065+0000","last_became_peered":"2024-09-21T07:34:07.152065+0000","last_unstale":"2024-09-21T07:34:07.161575+0000","last_undegraded":"2024-09-21T07:34:07.161575+0000","last_fullsized":"2024-09-21T07:34:07.161575+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:02.321477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496179+0000","last_change":"2024-09-21T07:34:08.150846+0000","last_active":"2024-09-21T07:34:09.496179+0000","last_peered":"2024-09-21T07:34:09.496179+0000","last_clean":"2024-09-21T07:34:09.496179+0000","last_became_active":"2024-09-21T07:34:08.150534+0000","last_became_peered":"2024-09-21T07:34:08.150534+0000","last_unstale":"2024-09-21T07:34:09.496179+0000","last_undegraded":"2024-09-21T07:34:09.496179+0000","last_fullsized":"2024-09-21T07:34:09.496179+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:26:49.717420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497461+0000","last_change":"2024-09-21T07:34:07.147358+0000","last_active":"2024-09-21T07:34:09.497461+0000","last_peered":"2024-09-21T07:34:09.497461+0000","last_clean":"2024-09-21T07:34:09.497461+0000","last_became_active":"2024-09-21T07:34:07.147036+0000","last_became_peered":"2024-09-21T07:34:07.147036+0000","last_unstale":"2024-09-21T07:34:09.497461+0000","last_undegraded":"2024-09-21T07:34:09.497461+0000","last_fullsized":"2024-09-21T07:34:09.497461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:04:22.493944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497584+0000","last_change":"2024-09-21T07:34:08.159510+0000","last_active":"2024-09-21T07:34:09.497584+0000","last_peered":"2024-09-21T07:34:09.497584+0000","last_clean":"2024-09-21T07:34:09.497584+0000","last_became_active":"2024-09-21T07:34:08.159191+0000","last_became_peered":"2024-09-21T07:34:08.159191+0000","last_unstale":"2024-09-21T07:34:09.497584+0000","last_undegraded":"2024-09-21T07:34:09.497584+0000","last_fullsized":"2024-09-21T07:34:09.497584+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:27.319525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496085+0000","last_change":"2024-09-21T07:34:07.155987+0000","last_active":"2024-09-21T07:34:09.496085+0000","last_peered":"2024-09-21T07:34:09.496085+0000","last_clean":"2024-09-21T07:34:09.496085+0000","last_became_active":"2024-09-21T07:34:07.155735+0000","last_became_peered":"2024-09-21T07:34:07.155735+0000","last_unstale":"2024-09-21T07:34:09.496085+0000","last_undegraded":"2024-09-21T07:34:09.496085+0000","last_fullsized":"2024-09-21T07:34:09.496085+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:23:21.954791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-21T07:34:07.143410+0000","last_change":"2024-09-21T07:34:07.143410+0000","last_active":"2024-09-21T07:34:07.121447+0000","last_peered":"2024-09-21T07:34:07.121447+0000","last_clean":"2024-09-21T07:34:07.121447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:07.143410+0000","last_undegraded":"2024-09-21T07:34:07.143410+0000","last_fullsized":"2024-09-21T07:34:07.143410+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T07:34:07.159470+0000","last_change":"2024-09-21T07:34:07.155156+0000","last_active":"2024-09-21T07:34:07.159470+0000","last_peered":"2024-09-21T07:34:07.159470+0000","last_clean":"2024-09-21T07:34:07.159470+0000","last_became_active":"2024-09-21T07:34:07.154829+0000","last_became_peered":"2024-09-21T07:34:07.154829+0000","last_unstale":"2024-09-21T07:34:07.159470+0000","last_undegraded":"2024-09-21T07:34:07.159470+0000","last_fullsized":"2024-09-21T07:34:07.159470+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:28:14.831565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":131,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T07:34:07.154589+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:34:07.154589+0000","last_peered":"2024-09-21T07:34:07.154589+0000","last_clean":"2024-09-21T07:34:07.154589+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:34:07.154589+0000","last_undegraded":"2024-09-21T07:34:07.154589+0000","last_fullsized":"2024-09-21T07:34:07.154589+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1716,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23,"ondisk_log_size":23,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509456,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":933888,"data_stored":719721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":339968,"data_stored":129353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":352256,"data_stored":135620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509457,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31532,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340308,"statfs":{"total":96636764160,"available":96604475392,"internally_reserved":0,"allocated":352256,"data_stored":135620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31520,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340320,"statfs":{"total":96636764160,"available":96604487680,"internally_reserved":0,"allocated":339968,"data_stored":129353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509458,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":933888,"data_stored":719721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542162,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":385024,"data_stored":148799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542161,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":393216,"data_stored":150329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:34:13.165 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:34:13.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.426+0000 7fa016600640 1 -- 172.21.15.39:0/436068131 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0100cd240 msgr2=0x7fa0100b2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:13.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.426+0000 7fa016600640 1 --2- 172.21.15.39:0/436068131 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0100cd240 0x7fa0100b2fb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa008005fd0 tx=0x7fa00802a820 comp rx=0 tx=0).stop 2024-09-21T07:34:13.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.426+0000 7fa016600640 1 -- 172.21.15.39:0/436068131 shutdown_connections 2024-09-21T07:34:13.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.426+0000 7fa016600640 1 --2- 172.21.15.39:0/436068131 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0100cd240 0x7fa0100b2fb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:13.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.426+0000 7fa016600640 1 --2- 172.21.15.39:0/436068131 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa0100cf660 0x7fa0100cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:13.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.426+0000 7fa016600640 1 -- 172.21.15.39:0/436068131 >> 172.21.15.39:0/436068131 conn(0x7fa0100b1cb0 msgr2=0x7fa0100b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:13.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.426+0000 7fa016600640 1 -- 172.21.15.39:0/436068131 shutdown_connections 2024-09-21T07:34:13.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.427+0000 7fa016600640 1 -- 172.21.15.39:0/436068131 wait complete. 2024-09-21T07:34:13.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.429+0000 7fa016600640 1 Processor -- start 2024-09-21T07:34:13.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.429+0000 7fa016600640 1 -- start start 2024-09-21T07:34:13.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.430+0000 7fa016600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0100cf660 0x7fa0101f5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:13.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.430+0000 7fa016600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa0100cfb90 0x7fa0100c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:13.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.430+0000 7fa016600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa010158740 0x7fa01009b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:13.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.430+0000 7fa016600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa010070f20 con 0x7fa0100cf660 2024-09-21T07:34:13.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.430+0000 7fa016600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa010070660 con 0x7fa010158740 2024-09-21T07:34:13.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.430+0000 7fa016600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa0100710a0 con 0x7fa0100cfb90 2024-09-21T07:34:13.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.431+0000 7fa00f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0100cf660 0x7fa0101f5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:13.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.431+0000 7fa00f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0100cf660 0x7fa0101f5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41418/0 (socket says 172.21.15.39:41418) 2024-09-21T07:34:13.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.431+0000 7fa00f400640 1 -- 172.21.15.39:0/2593384218 learned_addr learned my addr 172.21.15.39:0/2593384218 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:13.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.431+0000 7fa00ea00640 1 --2- 172.21.15.39:0/2593384218 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa0100cfb90 0x7fa0100c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:13.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.431+0000 7fa00fe00640 1 --2- 172.21.15.39:0/2593384218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa010158740 0x7fa01009b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:13.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.431+0000 7fa00f400640 1 -- 172.21.15.39:0/2593384218 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa0100cfb90 msgr2=0x7fa0100c90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:13.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.431+0000 7fa00f400640 1 --2- 172.21.15.39:0/2593384218 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa0100cfb90 0x7fa0100c90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:13.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.431+0000 7fa00f400640 1 -- 172.21.15.39:0/2593384218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa010158740 msgr2=0x7fa01009b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:13.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.431+0000 7fa00f400640 1 --2- 172.21.15.39:0/2593384218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa010158740 0x7fa01009b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:13.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.431+0000 7fa00f400640 1 -- 172.21.15.39:0/2593384218 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0100c95f0 con 0x7fa0100cf660 2024-09-21T07:34:13.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.431+0000 7fa00f400640 1 --2- 172.21.15.39:0/2593384218 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0100cf660 0x7fa0101f5c60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa00000e9c0 tx=0x7fa00000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:13.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.432+0000 7fa007400640 1 -- 172.21.15.39:0/2593384218 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00000cd80 con 0x7fa0100cf660 2024-09-21T07:34:13.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.432+0000 7fa007400640 1 -- 172.21.15.39:0/2593384218 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa000010430 con 0x7fa0100cf660 2024-09-21T07:34:13.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.432+0000 7fa016600640 1 -- 172.21.15.39:0/2593384218 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa01009cd00 con 0x7fa0100cf660 2024-09-21T07:34:13.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.432+0000 7fa007400640 1 -- 172.21.15.39:0/2593384218 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa000010880 con 0x7fa0100cf660 2024-09-21T07:34:13.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.432+0000 7fa016600640 1 -- 172.21.15.39:0/2593384218 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa01009ab40 con 0x7fa0100cf660 2024-09-21T07:34:13.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.433+0000 7fa016600640 1 -- 172.21.15.39:0/2593384218 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0100b2fb0 con 0x7fa0100cf660 2024-09-21T07:34:13.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.434+0000 7fa007400640 1 -- 172.21.15.39:0/2593384218 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa00001f020 con 0x7fa0100cf660 2024-09-21T07:34:13.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.435+0000 7fa007400640 1 --2- 172.21.15.39:0/2593384218 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9ff0078b90 0x7f9ff007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:13.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.435+0000 7fa007400640 1 -- 172.21.15.39:0/2593384218 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fa00009b410 con 0x7fa0100cf660 2024-09-21T07:34:13.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.436+0000 7fa00ea00640 1 --2- 172.21.15.39:0/2593384218 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9ff0078b90 0x7f9ff007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:13.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.436+0000 7fa00ea00640 1 --2- 172.21.15.39:0/2593384218 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9ff0078b90 0x7f9ff007b0a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc007fd0 tx=0x7f9ffc009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:13.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.440+0000 7fa007400640 1 -- 172.21.15.39:0/2593384218 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa0000631f0 con 0x7fa0100cf660 2024-09-21T07:34:13.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.633+0000 7fa016600640 1 -- 172.21.15.39:0/2593384218 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa0100af170 con 0x7f9ff0078b90 2024-09-21T07:34:13.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.638+0000 7fa007400640 1 -- 172.21.15.39:0/2593384218 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251940 (secure 0 0 0) 0x7fa0100af170 con 0x7f9ff0078b90 2024-09-21T07:34:13.643 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:34:13.644 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:34:13.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.643+0000 7fa016600640 1 -- 172.21.15.39:0/2593384218 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9ff0078b90 msgr2=0x7f9ff007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:13.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.643+0000 7fa016600640 1 --2- 172.21.15.39:0/2593384218 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9ff0078b90 0x7f9ff007b0a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc007fd0 tx=0x7f9ffc009040 comp rx=0 tx=0).stop 2024-09-21T07:34:13.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.643+0000 7fa016600640 1 -- 172.21.15.39:0/2593384218 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0100cf660 msgr2=0x7fa0101f5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:13.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.643+0000 7fa016600640 1 --2- 172.21.15.39:0/2593384218 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0100cf660 0x7fa0101f5c60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa00000e9c0 tx=0x7fa00000edc0 comp rx=0 tx=0).stop 2024-09-21T07:34:13.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.643+0000 7fa016600640 1 -- 172.21.15.39:0/2593384218 shutdown_connections 2024-09-21T07:34:13.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.643+0000 7fa016600640 1 --2- 172.21.15.39:0/2593384218 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9ff0078b90 0x7f9ff007b0a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:13.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.643+0000 7fa016600640 1 --2- 172.21.15.39:0/2593384218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa010158740 0x7fa01009b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:13.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.643+0000 7fa016600640 1 --2- 172.21.15.39:0/2593384218 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa0100cfb90 0x7fa0100c90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:13.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.643+0000 7fa016600640 1 --2- 172.21.15.39:0/2593384218 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0100cf660 0x7fa0101f5c60 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:13.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.643+0000 7fa016600640 1 -- 172.21.15.39:0/2593384218 >> 172.21.15.39:0/2593384218 conn(0x7fa0100b1cb0 msgr2=0x7fa0101597f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:13.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.643+0000 7fa016600640 1 -- 172.21.15.39:0/2593384218 shutdown_connections 2024-09-21T07:34:13.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.643+0000 7fa016600640 1 -- 172.21.15.39:0/2593384218 wait complete. 2024-09-21T07:34:13.666 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2024-09-21T07:34:13.548171+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237064,"kb_used_data":4168,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737656,"statfs":{"total":773094113280,"available":772851359744,"internally_reserved":0,"allocated":4268032,"data_stored":2375116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.421904"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496910+0000","last_change":"2024-09-21T07:34:08.172048+0000","last_active":"2024-09-21T07:34:09.496910+0000","last_peered":"2024-09-21T07:34:09.496910+0000","last_clean":"2024-09-21T07:34:09.496910+0000","last_became_active":"2024-09-21T07:34:08.170885+0000","last_became_peered":"2024-09-21T07:34:08.170885+0000","last_unstale":"2024-09-21T07:34:09.496910+0000","last_undegraded":"2024-09-21T07:34:09.496910+0000","last_fullsized":"2024-09-21T07:34:09.496910+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:30.259079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496812+0000","last_change":"2024-09-21T07:34:08.163227+0000","last_active":"2024-09-21T07:34:09.496812+0000","last_peered":"2024-09-21T07:34:09.496812+0000","last_clean":"2024-09-21T07:34:09.496812+0000","last_became_active":"2024-09-21T07:34:08.162915+0000","last_became_peered":"2024-09-21T07:34:08.162915+0000","last_unstale":"2024-09-21T07:34:09.496812+0000","last_undegraded":"2024-09-21T07:34:09.496812+0000","last_fullsized":"2024-09-21T07:34:09.496812+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:16.292830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.165484+0000","last_change":"2024-09-21T07:34:08.161784+0000","last_active":"2024-09-21T07:34:09.165484+0000","last_peered":"2024-09-21T07:34:09.165484+0000","last_clean":"2024-09-21T07:34:09.165484+0000","last_became_active":"2024-09-21T07:34:08.161467+0000","last_became_peered":"2024-09-21T07:34:08.161467+0000","last_unstale":"2024-09-21T07:34:09.165484+0000","last_undegraded":"2024-09-21T07:34:09.165484+0000","last_fullsized":"2024-09-21T07:34:09.165484+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:14:22.162876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.180849+0000","last_change":"2024-09-21T07:34:08.174981+0000","last_active":"2024-09-21T07:34:09.180849+0000","last_peered":"2024-09-21T07:34:09.180849+0000","last_clean":"2024-09-21T07:34:09.180849+0000","last_became_active":"2024-09-21T07:34:08.174163+0000","last_became_peered":"2024-09-21T07:34:08.174163+0000","last_unstale":"2024-09-21T07:34:09.180849+0000","last_undegraded":"2024-09-21T07:34:09.180849+0000","last_fullsized":"2024-09-21T07:34:09.180849+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:08.608518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.163240+0000","last_change":"2024-09-21T07:34:08.170011+0000","last_active":"2024-09-21T07:34:09.163240+0000","last_peered":"2024-09-21T07:34:09.163240+0000","last_clean":"2024-09-21T07:34:09.163240+0000","last_became_active":"2024-09-21T07:34:08.169688+0000","last_became_peered":"2024-09-21T07:34:08.169688+0000","last_unstale":"2024-09-21T07:34:09.163240+0000","last_undegraded":"2024-09-21T07:34:09.163240+0000","last_fullsized":"2024-09-21T07:34:09.163240+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:59:05.472245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496422+0000","last_change":"2024-09-21T07:34:08.161341+0000","last_active":"2024-09-21T07:34:09.496422+0000","last_peered":"2024-09-21T07:34:09.496422+0000","last_clean":"2024-09-21T07:34:09.496422+0000","last_became_active":"2024-09-21T07:34:08.161037+0000","last_became_peered":"2024-09-21T07:34:08.161037+0000","last_unstale":"2024-09-21T07:34:09.496422+0000","last_undegraded":"2024-09-21T07:34:09.496422+0000","last_fullsized":"2024-09-21T07:34:09.496422+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:52.361740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.329349+0000","last_change":"2024-09-21T07:34:08.173122+0000","last_active":"2024-09-21T07:34:09.329349+0000","last_peered":"2024-09-21T07:34:09.329349+0000","last_clean":"2024-09-21T07:34:09.329349+0000","last_became_active":"2024-09-21T07:34:08.172780+0000","last_became_peered":"2024-09-21T07:34:08.172780+0000","last_unstale":"2024-09-21T07:34:09.329349+0000","last_undegraded":"2024-09-21T07:34:09.329349+0000","last_fullsized":"2024-09-21T07:34:09.329349+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:27.710069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.329093+0000","last_change":"2024-09-21T07:34:08.167193+0000","last_active":"2024-09-21T07:34:09.329093+0000","last_peered":"2024-09-21T07:34:09.329093+0000","last_clean":"2024-09-21T07:34:09.329093+0000","last_became_active":"2024-09-21T07:34:08.165230+0000","last_became_peered":"2024-09-21T07:34:08.165230+0000","last_unstale":"2024-09-21T07:34:09.329093+0000","last_undegraded":"2024-09-21T07:34:09.329093+0000","last_fullsized":"2024-09-21T07:34:09.329093+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:39.114639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497858+0000","last_change":"2024-09-21T07:34:08.170445+0000","last_active":"2024-09-21T07:34:09.497858+0000","last_peered":"2024-09-21T07:34:09.497858+0000","last_clean":"2024-09-21T07:34:09.497858+0000","last_became_active":"2024-09-21T07:34:08.170115+0000","last_became_peered":"2024-09-21T07:34:08.170115+0000","last_unstale":"2024-09-21T07:34:09.497858+0000","last_undegraded":"2024-09-21T07:34:09.497858+0000","last_fullsized":"2024-09-21T07:34:09.497858+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:32.224498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.181226+0000","last_change":"2024-09-21T07:34:08.162257+0000","last_active":"2024-09-21T07:34:09.181226+0000","last_peered":"2024-09-21T07:34:09.181226+0000","last_clean":"2024-09-21T07:34:09.181226+0000","last_became_active":"2024-09-21T07:34:08.161939+0000","last_became_peered":"2024-09-21T07:34:08.161939+0000","last_unstale":"2024-09-21T07:34:09.181226+0000","last_undegraded":"2024-09-21T07:34:09.181226+0000","last_fullsized":"2024-09-21T07:34:09.181226+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:32:50.694783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.329426+0000","last_change":"2024-09-21T07:34:08.177569+0000","last_active":"2024-09-21T07:34:09.329426+0000","last_peered":"2024-09-21T07:34:09.329426+0000","last_clean":"2024-09-21T07:34:09.329426+0000","last_became_active":"2024-09-21T07:34:08.177198+0000","last_became_peered":"2024-09-21T07:34:08.177198+0000","last_unstale":"2024-09-21T07:34:09.329426+0000","last_undegraded":"2024-09-21T07:34:09.329426+0000","last_fullsized":"2024-09-21T07:34:09.329426+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:50.102777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.165047+0000","last_change":"2024-09-21T07:34:08.155812+0000","last_active":"2024-09-21T07:34:09.165047+0000","last_peered":"2024-09-21T07:34:09.165047+0000","last_clean":"2024-09-21T07:34:09.165047+0000","last_became_active":"2024-09-21T07:34:08.155490+0000","last_became_peered":"2024-09-21T07:34:08.155490+0000","last_unstale":"2024-09-21T07:34:09.165047+0000","last_undegraded":"2024-09-21T07:34:09.165047+0000","last_fullsized":"2024-09-21T07:34:09.165047+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:13.730449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.163440+0000","last_change":"2024-09-21T07:34:08.174944+0000","last_active":"2024-09-21T07:34:09.163440+0000","last_peered":"2024-09-21T07:34:09.163440+0000","last_clean":"2024-09-21T07:34:09.163440+0000","last_became_active":"2024-09-21T07:34:08.173967+0000","last_became_peered":"2024-09-21T07:34:08.173967+0000","last_unstale":"2024-09-21T07:34:09.163440+0000","last_undegraded":"2024-09-21T07:34:09.163440+0000","last_fullsized":"2024-09-21T07:34:09.163440+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:55.791945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496432+0000","last_change":"2024-09-21T07:34:08.167950+0000","last_active":"2024-09-21T07:34:09.496432+0000","last_peered":"2024-09-21T07:34:09.496432+0000","last_clean":"2024-09-21T07:34:09.496432+0000","last_became_active":"2024-09-21T07:34:08.167687+0000","last_became_peered":"2024-09-21T07:34:08.167687+0000","last_unstale":"2024-09-21T07:34:09.496432+0000","last_undegraded":"2024-09-21T07:34:09.496432+0000","last_fullsized":"2024-09-21T07:34:09.496432+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:44:01.238051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.164897+0000","last_change":"2024-09-21T07:34:08.166495+0000","last_active":"2024-09-21T07:34:09.164897+0000","last_peered":"2024-09-21T07:34:09.164897+0000","last_clean":"2024-09-21T07:34:09.164897+0000","last_became_active":"2024-09-21T07:34:08.166104+0000","last_became_peered":"2024-09-21T07:34:08.166104+0000","last_unstale":"2024-09-21T07:34:09.164897+0000","last_undegraded":"2024-09-21T07:34:09.164897+0000","last_fullsized":"2024-09-21T07:34:09.164897+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:31:12.988796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.329881+0000","last_change":"2024-09-21T07:34:08.172013+0000","last_active":"2024-09-21T07:34:09.329881+0000","last_peered":"2024-09-21T07:34:09.329881+0000","last_clean":"2024-09-21T07:34:09.329881+0000","last_became_active":"2024-09-21T07:34:08.171497+0000","last_became_peered":"2024-09-21T07:34:08.171497+0000","last_unstale":"2024-09-21T07:34:09.329881+0000","last_undegraded":"2024-09-21T07:34:09.329881+0000","last_fullsized":"2024-09-21T07:34:09.329881+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:28.526013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.329782+0000","last_change":"2024-09-21T07:34:08.176187+0000","last_active":"2024-09-21T07:34:09.329782+0000","last_peered":"2024-09-21T07:34:09.329782+0000","last_clean":"2024-09-21T07:34:09.329782+0000","last_became_active":"2024-09-21T07:34:08.175895+0000","last_became_peered":"2024-09-21T07:34:08.175895+0000","last_unstale":"2024-09-21T07:34:09.329782+0000","last_undegraded":"2024-09-21T07:34:09.329782+0000","last_fullsized":"2024-09-21T07:34:09.329782+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:10.107127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496674+0000","last_change":"2024-09-21T07:34:08.153003+0000","last_active":"2024-09-21T07:34:09.496674+0000","last_peered":"2024-09-21T07:34:09.496674+0000","last_clean":"2024-09-21T07:34:09.496674+0000","last_became_active":"2024-09-21T07:34:08.150882+0000","last_became_peered":"2024-09-21T07:34:08.150882+0000","last_unstale":"2024-09-21T07:34:09.496674+0000","last_undegraded":"2024-09-21T07:34:09.496674+0000","last_fullsized":"2024-09-21T07:34:09.496674+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:38.611907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496759+0000","last_change":"2024-09-21T07:34:08.158999+0000","last_active":"2024-09-21T07:34:09.496759+0000","last_peered":"2024-09-21T07:34:09.496759+0000","last_clean":"2024-09-21T07:34:09.496759+0000","last_became_active":"2024-09-21T07:34:08.158712+0000","last_became_peered":"2024-09-21T07:34:08.158712+0000","last_unstale":"2024-09-21T07:34:09.496759+0000","last_undegraded":"2024-09-21T07:34:09.496759+0000","last_fullsized":"2024-09-21T07:34:09.496759+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:34:04.145536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.163438+0000","last_change":"2024-09-21T07:34:08.168400+0000","last_active":"2024-09-21T07:34:09.163438+0000","last_peered":"2024-09-21T07:34:09.163438+0000","last_clean":"2024-09-21T07:34:09.163438+0000","last_became_active":"2024-09-21T07:34:08.168071+0000","last_became_peered":"2024-09-21T07:34:08.168071+0000","last_unstale":"2024-09-21T07:34:09.163438+0000","last_undegraded":"2024-09-21T07:34:09.163438+0000","last_fullsized":"2024-09-21T07:34:09.163438+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:01.249634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496796+0000","last_change":"2024-09-21T07:34:08.168578+0000","last_active":"2024-09-21T07:34:09.496796+0000","last_peered":"2024-09-21T07:34:09.496796+0000","last_clean":"2024-09-21T07:34:09.496796+0000","last_became_active":"2024-09-21T07:34:08.168235+0000","last_became_peered":"2024-09-21T07:34:08.168235+0000","last_unstale":"2024-09-21T07:34:09.496796+0000","last_undegraded":"2024-09-21T07:34:09.496796+0000","last_fullsized":"2024-09-21T07:34:09.496796+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:18.132264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496889+0000","last_change":"2024-09-21T07:34:08.159192+0000","last_active":"2024-09-21T07:34:09.496889+0000","last_peered":"2024-09-21T07:34:09.496889+0000","last_clean":"2024-09-21T07:34:09.496889+0000","last_became_active":"2024-09-21T07:34:08.158860+0000","last_became_peered":"2024-09-21T07:34:08.158860+0000","last_unstale":"2024-09-21T07:34:09.496889+0000","last_undegraded":"2024-09-21T07:34:09.496889+0000","last_fullsized":"2024-09-21T07:34:09.496889+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:48.287489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.168895+0000","last_change":"2024-09-21T07:34:08.164892+0000","last_active":"2024-09-21T07:34:09.168895+0000","last_peered":"2024-09-21T07:34:09.168895+0000","last_clean":"2024-09-21T07:34:09.168895+0000","last_became_active":"2024-09-21T07:34:08.163504+0000","last_became_peered":"2024-09-21T07:34:08.163504+0000","last_unstale":"2024-09-21T07:34:09.168895+0000","last_undegraded":"2024-09-21T07:34:09.168895+0000","last_fullsized":"2024-09-21T07:34:09.168895+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:55.716857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.164789+0000","last_change":"2024-09-21T07:34:08.164618+0000","last_active":"2024-09-21T07:34:09.164789+0000","last_peered":"2024-09-21T07:34:09.164789+0000","last_clean":"2024-09-21T07:34:09.164789+0000","last_became_active":"2024-09-21T07:34:08.163696+0000","last_became_peered":"2024-09-21T07:34:08.163696+0000","last_unstale":"2024-09-21T07:34:09.164789+0000","last_undegraded":"2024-09-21T07:34:09.164789+0000","last_fullsized":"2024-09-21T07:34:09.164789+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:08:41.286205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.181440+0000","last_change":"2024-09-21T07:34:08.163763+0000","last_active":"2024-09-21T07:34:09.181440+0000","last_peered":"2024-09-21T07:34:09.181440+0000","last_clean":"2024-09-21T07:34:09.181440+0000","last_became_active":"2024-09-21T07:34:08.163455+0000","last_became_peered":"2024-09-21T07:34:08.163455+0000","last_unstale":"2024-09-21T07:34:09.181440+0000","last_undegraded":"2024-09-21T07:34:09.181440+0000","last_fullsized":"2024-09-21T07:34:09.181440+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:39.114639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496941+0000","last_change":"2024-09-21T07:34:08.158447+0000","last_active":"2024-09-21T07:34:09.496941+0000","last_peered":"2024-09-21T07:34:09.496941+0000","last_clean":"2024-09-21T07:34:09.496941+0000","last_became_active":"2024-09-21T07:34:08.158138+0000","last_became_peered":"2024-09-21T07:34:08.158138+0000","last_unstale":"2024-09-21T07:34:09.496941+0000","last_undegraded":"2024-09-21T07:34:09.496941+0000","last_fullsized":"2024-09-21T07:34:09.496941+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:05.155134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.181311+0000","last_change":"2024-09-21T07:34:08.172580+0000","last_active":"2024-09-21T07:34:09.181311+0000","last_peered":"2024-09-21T07:34:09.181311+0000","last_clean":"2024-09-21T07:34:09.181311+0000","last_became_active":"2024-09-21T07:34:08.172294+0000","last_became_peered":"2024-09-21T07:34:08.172294+0000","last_unstale":"2024-09-21T07:34:09.181311+0000","last_undegraded":"2024-09-21T07:34:09.181311+0000","last_fullsized":"2024-09-21T07:34:09.181311+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:21.245892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.165259+0000","last_change":"2024-09-21T07:34:08.169640+0000","last_active":"2024-09-21T07:34:09.165259+0000","last_peered":"2024-09-21T07:34:09.165259+0000","last_clean":"2024-09-21T07:34:09.165259+0000","last_became_active":"2024-09-21T07:34:08.169283+0000","last_became_peered":"2024-09-21T07:34:08.169283+0000","last_unstale":"2024-09-21T07:34:09.165259+0000","last_undegraded":"2024-09-21T07:34:09.165259+0000","last_fullsized":"2024-09-21T07:34:09.165259+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:07.122419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497087+0000","last_change":"2024-09-21T07:34:08.153475+0000","last_active":"2024-09-21T07:34:09.497087+0000","last_peered":"2024-09-21T07:34:09.497087+0000","last_clean":"2024-09-21T07:34:09.497087+0000","last_became_active":"2024-09-21T07:34:08.152735+0000","last_became_peered":"2024-09-21T07:34:08.152735+0000","last_unstale":"2024-09-21T07:34:09.497087+0000","last_undegraded":"2024-09-21T07:34:09.497087+0000","last_fullsized":"2024-09-21T07:34:09.497087+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:42.395864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.329350+0000","last_change":"2024-09-21T07:34:08.175295+0000","last_active":"2024-09-21T07:34:09.329350+0000","last_peered":"2024-09-21T07:34:09.329350+0000","last_clean":"2024-09-21T07:34:09.329350+0000","last_became_active":"2024-09-21T07:34:08.175002+0000","last_became_peered":"2024-09-21T07:34:08.175002+0000","last_unstale":"2024-09-21T07:34:09.329350+0000","last_undegraded":"2024-09-21T07:34:09.329350+0000","last_fullsized":"2024-09-21T07:34:09.329350+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:40.829187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496269+0000","last_change":"2024-09-21T07:34:08.163793+0000","last_active":"2024-09-21T07:34:09.496269+0000","last_peered":"2024-09-21T07:34:09.496269+0000","last_clean":"2024-09-21T07:34:09.496269+0000","last_became_active":"2024-09-21T07:34:08.163391+0000","last_became_peered":"2024-09-21T07:34:08.163391+0000","last_unstale":"2024-09-21T07:34:09.496269+0000","last_undegraded":"2024-09-21T07:34:09.496269+0000","last_fullsized":"2024-09-21T07:34:09.496269+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:55.152456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.329080+0000","last_change":"2024-09-21T07:34:08.173785+0000","last_active":"2024-09-21T07:34:09.329080+0000","last_peered":"2024-09-21T07:34:09.329080+0000","last_clean":"2024-09-21T07:34:09.329080+0000","last_became_active":"2024-09-21T07:34:08.173332+0000","last_became_peered":"2024-09-21T07:34:08.173332+0000","last_unstale":"2024-09-21T07:34:09.329080+0000","last_undegraded":"2024-09-21T07:34:09.329080+0000","last_fullsized":"2024-09-21T07:34:09.329080+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:13.730449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.169412+0000","last_change":"2024-09-21T07:34:08.166699+0000","last_active":"2024-09-21T07:34:09.169412+0000","last_peered":"2024-09-21T07:34:09.169412+0000","last_clean":"2024-09-21T07:34:09.169412+0000","last_became_active":"2024-09-21T07:34:08.161047+0000","last_became_peered":"2024-09-21T07:34:08.161047+0000","last_unstale":"2024-09-21T07:34:09.169412+0000","last_undegraded":"2024-09-21T07:34:09.169412+0000","last_fullsized":"2024-09-21T07:34:09.169412+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:34.740594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":19,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497099+0000","last_change":"2024-09-21T07:34:07.155982+0000","last_active":"2024-09-21T07:34:09.497099+0000","last_peered":"2024-09-21T07:34:09.497099+0000","last_clean":"2024-09-21T07:34:09.497099+0000","last_became_active":"2024-09-21T07:34:07.155500+0000","last_became_peered":"2024-09-21T07:34:07.155500+0000","last_unstale":"2024-09-21T07:34:09.497099+0000","last_undegraded":"2024-09-21T07:34:09.497099+0000","last_fullsized":"2024-09-21T07:34:09.497099+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:26.882263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497060+0000","last_change":"2024-09-21T07:34:08.178748+0000","last_active":"2024-09-21T07:34:09.497060+0000","last_peered":"2024-09-21T07:34:09.497060+0000","last_clean":"2024-09-21T07:34:09.497060+0000","last_became_active":"2024-09-21T07:34:08.177806+0000","last_became_peered":"2024-09-21T07:34:08.177806+0000","last_unstale":"2024-09-21T07:34:09.497060+0000","last_undegraded":"2024-09-21T07:34:09.497060+0000","last_fullsized":"2024-09-21T07:34:09.497060+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:47.458768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.169477+0000","last_change":"2024-09-21T07:34:07.148229+0000","last_active":"2024-09-21T07:34:09.169477+0000","last_peered":"2024-09-21T07:34:09.169477+0000","last_clean":"2024-09-21T07:34:09.169477+0000","last_became_active":"2024-09-21T07:34:07.147920+0000","last_became_peered":"2024-09-21T07:34:07.147920+0000","last_unstale":"2024-09-21T07:34:09.169477+0000","last_undegraded":"2024-09-21T07:34:09.169477+0000","last_fullsized":"2024-09-21T07:34:09.169477+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:09.410183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497703+0000","last_change":"2024-09-21T07:34:08.162272+0000","last_active":"2024-09-21T07:34:09.497703+0000","last_peered":"2024-09-21T07:34:09.497703+0000","last_clean":"2024-09-21T07:34:09.497703+0000","last_became_active":"2024-09-21T07:34:08.162031+0000","last_became_peered":"2024-09-21T07:34:08.162031+0000","last_unstale":"2024-09-21T07:34:09.497703+0000","last_undegraded":"2024-09-21T07:34:09.497703+0000","last_fullsized":"2024-09-21T07:34:09.497703+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:48.649163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496548+0000","last_change":"2024-09-21T07:34:07.162816+0000","last_active":"2024-09-21T07:34:09.496548+0000","last_peered":"2024-09-21T07:34:09.496548+0000","last_clean":"2024-09-21T07:34:09.496548+0000","last_became_active":"2024-09-21T07:34:07.162478+0000","last_became_peered":"2024-09-21T07:34:07.162478+0000","last_unstale":"2024-09-21T07:34:09.496548+0000","last_undegraded":"2024-09-21T07:34:09.496548+0000","last_fullsized":"2024-09-21T07:34:09.496548+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:18:03.659966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.181339+0000","last_change":"2024-09-21T07:34:08.172648+0000","last_active":"2024-09-21T07:34:09.181339+0000","last_peered":"2024-09-21T07:34:09.181339+0000","last_clean":"2024-09-21T07:34:09.181339+0000","last_became_active":"2024-09-21T07:34:08.172311+0000","last_became_peered":"2024-09-21T07:34:08.172311+0000","last_unstale":"2024-09-21T07:34:09.181339+0000","last_undegraded":"2024-09-21T07:34:09.181339+0000","last_fullsized":"2024-09-21T07:34:09.181339+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:25.699460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.163416+0000","last_change":"2024-09-21T07:34:07.151195+0000","last_active":"2024-09-21T07:34:09.163416+0000","last_peered":"2024-09-21T07:34:09.163416+0000","last_clean":"2024-09-21T07:34:09.163416+0000","last_became_active":"2024-09-21T07:34:07.150871+0000","last_became_peered":"2024-09-21T07:34:07.150871+0000","last_unstale":"2024-09-21T07:34:09.163416+0000","last_undegraded":"2024-09-21T07:34:09.163416+0000","last_fullsized":"2024-09-21T07:34:09.163416+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:27:38.559677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.329459+0000","last_change":"2024-09-21T07:34:08.174424+0000","last_active":"2024-09-21T07:34:09.329459+0000","last_peered":"2024-09-21T07:34:09.329459+0000","last_clean":"2024-09-21T07:34:09.329459+0000","last_became_active":"2024-09-21T07:34:08.174028+0000","last_became_peered":"2024-09-21T07:34:08.174028+0000","last_unstale":"2024-09-21T07:34:09.329459+0000","last_undegraded":"2024-09-21T07:34:09.329459+0000","last_fullsized":"2024-09-21T07:34:09.329459+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:31.524701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497041+0000","last_change":"2024-09-21T07:34:07.154783+0000","last_active":"2024-09-21T07:34:09.497041+0000","last_peered":"2024-09-21T07:34:09.497041+0000","last_clean":"2024-09-21T07:34:09.497041+0000","last_became_active":"2024-09-21T07:34:07.154378+0000","last_became_peered":"2024-09-21T07:34:07.154378+0000","last_unstale":"2024-09-21T07:34:09.497041+0000","last_undegraded":"2024-09-21T07:34:09.497041+0000","last_fullsized":"2024-09-21T07:34:09.497041+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:16.041455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.164907+0000","last_change":"2024-09-21T07:34:08.166752+0000","last_active":"2024-09-21T07:34:09.164907+0000","last_peered":"2024-09-21T07:34:09.164907+0000","last_clean":"2024-09-21T07:34:09.164907+0000","last_became_active":"2024-09-21T07:34:08.166477+0000","last_became_peered":"2024-09-21T07:34:08.166477+0000","last_unstale":"2024-09-21T07:34:09.164907+0000","last_undegraded":"2024-09-21T07:34:09.164907+0000","last_fullsized":"2024-09-21T07:34:09.164907+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:25.699460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496886+0000","last_change":"2024-09-21T07:34:07.153288+0000","last_active":"2024-09-21T07:34:09.496886+0000","last_peered":"2024-09-21T07:34:09.496886+0000","last_clean":"2024-09-21T07:34:09.496886+0000","last_became_active":"2024-09-21T07:34:07.152949+0000","last_became_peered":"2024-09-21T07:34:07.152949+0000","last_unstale":"2024-09-21T07:34:09.496886+0000","last_undegraded":"2024-09-21T07:34:09.496886+0000","last_fullsized":"2024-09-21T07:34:09.496886+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:54:43.804810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496478+0000","last_change":"2024-09-21T07:34:08.166316+0000","last_active":"2024-09-21T07:34:09.496478+0000","last_peered":"2024-09-21T07:34:09.496478+0000","last_clean":"2024-09-21T07:34:09.496478+0000","last_became_active":"2024-09-21T07:34:08.165006+0000","last_became_peered":"2024-09-21T07:34:08.165006+0000","last_unstale":"2024-09-21T07:34:09.496478+0000","last_undegraded":"2024-09-21T07:34:09.496478+0000","last_fullsized":"2024-09-21T07:34:09.496478+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:27.528708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496037+0000","last_change":"2024-09-21T07:34:07.141466+0000","last_active":"2024-09-21T07:34:09.496037+0000","last_peered":"2024-09-21T07:34:09.496037+0000","last_clean":"2024-09-21T07:34:09.496037+0000","last_became_active":"2024-09-21T07:34:07.141003+0000","last_became_peered":"2024-09-21T07:34:07.141003+0000","last_unstale":"2024-09-21T07:34:09.496037+0000","last_undegraded":"2024-09-21T07:34:09.496037+0000","last_fullsized":"2024-09-21T07:34:09.496037+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:52:33.957055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.180647+0000","last_change":"2024-09-21T07:34:08.163550+0000","last_active":"2024-09-21T07:34:09.180647+0000","last_peered":"2024-09-21T07:34:09.180647+0000","last_clean":"2024-09-21T07:34:09.180647+0000","last_became_active":"2024-09-21T07:34:08.163236+0000","last_became_peered":"2024-09-21T07:34:08.163236+0000","last_unstale":"2024-09-21T07:34:09.180647+0000","last_undegraded":"2024-09-21T07:34:09.180647+0000","last_fullsized":"2024-09-21T07:34:09.180647+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:56.729284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496450+0000","last_change":"2024-09-21T07:34:07.142126+0000","last_active":"2024-09-21T07:34:09.496450+0000","last_peered":"2024-09-21T07:34:09.496450+0000","last_clean":"2024-09-21T07:34:09.496450+0000","last_became_active":"2024-09-21T07:34:07.141713+0000","last_became_peered":"2024-09-21T07:34:07.141713+0000","last_unstale":"2024-09-21T07:34:09.496450+0000","last_undegraded":"2024-09-21T07:34:09.496450+0000","last_fullsized":"2024-09-21T07:34:09.496450+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:58.789677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.165516+0000","last_change":"2024-09-21T07:34:08.159275+0000","last_active":"2024-09-21T07:34:09.165516+0000","last_peered":"2024-09-21T07:34:09.165516+0000","last_clean":"2024-09-21T07:34:09.165516+0000","last_became_active":"2024-09-21T07:34:08.159005+0000","last_became_peered":"2024-09-21T07:34:08.159005+0000","last_unstale":"2024-09-21T07:34:09.165516+0000","last_undegraded":"2024-09-21T07:34:09.165516+0000","last_fullsized":"2024-09-21T07:34:09.165516+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:50.102777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496527+0000","last_change":"2024-09-21T07:34:07.154435+0000","last_active":"2024-09-21T07:34:09.496527+0000","last_peered":"2024-09-21T07:34:09.496527+0000","last_clean":"2024-09-21T07:34:09.496527+0000","last_became_active":"2024-09-21T07:34:07.154127+0000","last_became_peered":"2024-09-21T07:34:07.154127+0000","last_unstale":"2024-09-21T07:34:09.496527+0000","last_undegraded":"2024-09-21T07:34:09.496527+0000","last_fullsized":"2024-09-21T07:34:09.496527+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:11:21.678594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496615+0000","last_change":"2024-09-21T07:34:08.170301+0000","last_active":"2024-09-21T07:34:09.496615+0000","last_peered":"2024-09-21T07:34:09.496615+0000","last_clean":"2024-09-21T07:34:09.496615+0000","last_became_active":"2024-09-21T07:34:08.169925+0000","last_became_peered":"2024-09-21T07:34:08.169925+0000","last_unstale":"2024-09-21T07:34:09.496615+0000","last_undegraded":"2024-09-21T07:34:09.496615+0000","last_fullsized":"2024-09-21T07:34:09.496615+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:31.073725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.169543+0000","last_change":"2024-09-21T07:34:07.152298+0000","last_active":"2024-09-21T07:34:09.169543+0000","last_peered":"2024-09-21T07:34:09.169543+0000","last_clean":"2024-09-21T07:34:09.169543+0000","last_became_active":"2024-09-21T07:34:07.151921+0000","last_became_peered":"2024-09-21T07:34:07.151921+0000","last_unstale":"2024-09-21T07:34:09.169543+0000","last_undegraded":"2024-09-21T07:34:09.169543+0000","last_fullsized":"2024-09-21T07:34:09.169543+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:52.169625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496907+0000","last_change":"2024-09-21T07:34:08.160957+0000","last_active":"2024-09-21T07:34:09.496907+0000","last_peered":"2024-09-21T07:34:09.496907+0000","last_clean":"2024-09-21T07:34:09.496907+0000","last_became_active":"2024-09-21T07:34:08.160601+0000","last_became_peered":"2024-09-21T07:34:08.160601+0000","last_unstale":"2024-09-21T07:34:09.496907+0000","last_undegraded":"2024-09-21T07:34:09.496907+0000","last_fullsized":"2024-09-21T07:34:09.496907+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:19:16.261092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.165514+0000","last_change":"2024-09-21T07:34:07.153099+0000","last_active":"2024-09-21T07:34:09.165514+0000","last_peered":"2024-09-21T07:34:09.165514+0000","last_clean":"2024-09-21T07:34:09.165514+0000","last_became_active":"2024-09-21T07:34:07.152790+0000","last_became_peered":"2024-09-21T07:34:07.152790+0000","last_unstale":"2024-09-21T07:34:09.165514+0000","last_undegraded":"2024-09-21T07:34:09.165514+0000","last_fullsized":"2024-09-21T07:34:09.165514+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:51.990530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.165280+0000","last_change":"2024-09-21T07:34:08.165826+0000","last_active":"2024-09-21T07:34:09.165280+0000","last_peered":"2024-09-21T07:34:09.165280+0000","last_clean":"2024-09-21T07:34:09.165280+0000","last_became_active":"2024-09-21T07:34:08.160912+0000","last_became_peered":"2024-09-21T07:34:08.160912+0000","last_unstale":"2024-09-21T07:34:09.165280+0000","last_undegraded":"2024-09-21T07:34:09.165280+0000","last_fullsized":"2024-09-21T07:34:09.165280+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:27.710069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":21,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496721+0000","last_change":"2024-09-21T07:34:07.162976+0000","last_active":"2024-09-21T07:34:09.496721+0000","last_peered":"2024-09-21T07:34:09.496721+0000","last_clean":"2024-09-21T07:34:09.496721+0000","last_became_active":"2024-09-21T07:34:07.162698+0000","last_became_peered":"2024-09-21T07:34:07.162698+0000","last_unstale":"2024-09-21T07:34:09.496721+0000","last_undegraded":"2024-09-21T07:34:09.496721+0000","last_fullsized":"2024-09-21T07:34:09.496721+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:01:04.212051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.180755+0000","last_change":"2024-09-21T07:34:08.168839+0000","last_active":"2024-09-21T07:34:09.180755+0000","last_peered":"2024-09-21T07:34:09.180755+0000","last_clean":"2024-09-21T07:34:09.180755+0000","last_became_active":"2024-09-21T07:34:08.168183+0000","last_became_peered":"2024-09-21T07:34:08.168183+0000","last_unstale":"2024-09-21T07:34:09.180755+0000","last_undegraded":"2024-09-21T07:34:09.180755+0000","last_fullsized":"2024-09-21T07:34:09.180755+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:28.055160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497188+0000","last_change":"2024-09-21T07:34:07.150104+0000","last_active":"2024-09-21T07:34:09.497188+0000","last_peered":"2024-09-21T07:34:09.497188+0000","last_clean":"2024-09-21T07:34:09.497188+0000","last_became_active":"2024-09-21T07:34:07.149757+0000","last_became_peered":"2024-09-21T07:34:07.149757+0000","last_unstale":"2024-09-21T07:34:09.497188+0000","last_undegraded":"2024-09-21T07:34:09.497188+0000","last_fullsized":"2024-09-21T07:34:09.497188+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:17.255537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496697+0000","last_change":"2024-09-21T07:34:08.166843+0000","last_active":"2024-09-21T07:34:09.496697+0000","last_peered":"2024-09-21T07:34:09.496697+0000","last_clean":"2024-09-21T07:34:09.496697+0000","last_became_active":"2024-09-21T07:34:08.166516+0000","last_became_peered":"2024-09-21T07:34:08.166516+0000","last_unstale":"2024-09-21T07:34:09.496697+0000","last_undegraded":"2024-09-21T07:34:09.496697+0000","last_fullsized":"2024-09-21T07:34:09.496697+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:39.634059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.180719+0000","last_change":"2024-09-21T07:34:07.160820+0000","last_active":"2024-09-21T07:34:09.180719+0000","last_peered":"2024-09-21T07:34:09.180719+0000","last_clean":"2024-09-21T07:34:09.180719+0000","last_became_active":"2024-09-21T07:34:07.160364+0000","last_became_peered":"2024-09-21T07:34:07.160364+0000","last_unstale":"2024-09-21T07:34:09.180719+0000","last_undegraded":"2024-09-21T07:34:09.180719+0000","last_fullsized":"2024-09-21T07:34:09.180719+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:27.230155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.329827+0000","last_change":"2024-09-21T07:34:08.173868+0000","last_active":"2024-09-21T07:34:09.329827+0000","last_peered":"2024-09-21T07:34:09.329827+0000","last_clean":"2024-09-21T07:34:09.329827+0000","last_became_active":"2024-09-21T07:34:08.173552+0000","last_became_peered":"2024-09-21T07:34:08.173552+0000","last_unstale":"2024-09-21T07:34:09.329827+0000","last_undegraded":"2024-09-21T07:34:09.329827+0000","last_fullsized":"2024-09-21T07:34:09.329827+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:16.612188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496142+0000","last_change":"2024-09-21T07:34:07.153626+0000","last_active":"2024-09-21T07:34:09.496142+0000","last_peered":"2024-09-21T07:34:09.496142+0000","last_clean":"2024-09-21T07:34:09.496142+0000","last_became_active":"2024-09-21T07:34:07.153334+0000","last_became_peered":"2024-09-21T07:34:07.153334+0000","last_unstale":"2024-09-21T07:34:09.496142+0000","last_undegraded":"2024-09-21T07:34:09.496142+0000","last_fullsized":"2024-09-21T07:34:09.496142+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:48.214805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496501+0000","last_change":"2024-09-21T07:34:08.175374+0000","last_active":"2024-09-21T07:34:09.496501+0000","last_peered":"2024-09-21T07:34:09.496501+0000","last_clean":"2024-09-21T07:34:09.496501+0000","last_became_active":"2024-09-21T07:34:08.175065+0000","last_became_peered":"2024-09-21T07:34:08.175065+0000","last_unstale":"2024-09-21T07:34:09.496501+0000","last_undegraded":"2024-09-21T07:34:09.496501+0000","last_fullsized":"2024-09-21T07:34:09.496501+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:52.997974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496343+0000","last_change":"2024-09-21T07:34:07.161149+0000","last_active":"2024-09-21T07:34:09.496343+0000","last_peered":"2024-09-21T07:34:09.496343+0000","last_clean":"2024-09-21T07:34:09.496343+0000","last_became_active":"2024-09-21T07:34:07.160882+0000","last_became_peered":"2024-09-21T07:34:07.160882+0000","last_unstale":"2024-09-21T07:34:09.496343+0000","last_undegraded":"2024-09-21T07:34:09.496343+0000","last_fullsized":"2024-09-21T07:34:09.496343+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:55.427089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497420+0000","last_change":"2024-09-21T07:34:08.167375+0000","last_active":"2024-09-21T07:34:09.497420+0000","last_peered":"2024-09-21T07:34:09.497420+0000","last_clean":"2024-09-21T07:34:09.497420+0000","last_became_active":"2024-09-21T07:34:08.165919+0000","last_became_peered":"2024-09-21T07:34:08.165919+0000","last_unstale":"2024-09-21T07:34:09.497420+0000","last_undegraded":"2024-09-21T07:34:09.497420+0000","last_fullsized":"2024-09-21T07:34:09.497420+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:30.388748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.181509+0000","last_change":"2024-09-21T07:34:07.154981+0000","last_active":"2024-09-21T07:34:09.181509+0000","last_peered":"2024-09-21T07:34:09.181509+0000","last_clean":"2024-09-21T07:34:09.181509+0000","last_became_active":"2024-09-21T07:34:07.154706+0000","last_became_peered":"2024-09-21T07:34:07.154706+0000","last_unstale":"2024-09-21T07:34:09.181509+0000","last_undegraded":"2024-09-21T07:34:09.181509+0000","last_fullsized":"2024-09-21T07:34:09.181509+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:50.527238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.165808+0000","last_change":"2024-09-21T07:34:08.171328+0000","last_active":"2024-09-21T07:34:09.165808+0000","last_peered":"2024-09-21T07:34:09.165808+0000","last_clean":"2024-09-21T07:34:09.165808+0000","last_became_active":"2024-09-21T07:34:08.170351+0000","last_became_peered":"2024-09-21T07:34:08.170351+0000","last_unstale":"2024-09-21T07:34:09.165808+0000","last_undegraded":"2024-09-21T07:34:09.165808+0000","last_fullsized":"2024-09-21T07:34:09.165808+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:52.642170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497519+0000","last_change":"2024-09-21T07:34:07.149293+0000","last_active":"2024-09-21T07:34:09.497519+0000","last_peered":"2024-09-21T07:34:09.497519+0000","last_clean":"2024-09-21T07:34:09.497519+0000","last_became_active":"2024-09-21T07:34:07.148938+0000","last_became_peered":"2024-09-21T07:34:07.148938+0000","last_unstale":"2024-09-21T07:34:09.497519+0000","last_undegraded":"2024-09-21T07:34:09.497519+0000","last_fullsized":"2024-09-21T07:34:09.497519+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:59.434905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.180907+0000","last_change":"2024-09-21T07:34:08.177348+0000","last_active":"2024-09-21T07:34:09.180907+0000","last_peered":"2024-09-21T07:34:09.180907+0000","last_clean":"2024-09-21T07:34:09.180907+0000","last_became_active":"2024-09-21T07:34:08.174645+0000","last_became_peered":"2024-09-21T07:34:08.174645+0000","last_unstale":"2024-09-21T07:34:09.180907+0000","last_undegraded":"2024-09-21T07:34:09.180907+0000","last_fullsized":"2024-09-21T07:34:09.180907+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:06.515935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497479+0000","last_change":"2024-09-21T07:34:07.148308+0000","last_active":"2024-09-21T07:34:09.497479+0000","last_peered":"2024-09-21T07:34:09.497479+0000","last_clean":"2024-09-21T07:34:09.497479+0000","last_became_active":"2024-09-21T07:34:07.148001+0000","last_became_peered":"2024-09-21T07:34:07.148001+0000","last_unstale":"2024-09-21T07:34:09.497479+0000","last_undegraded":"2024-09-21T07:34:09.497479+0000","last_fullsized":"2024-09-21T07:34:09.497479+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:54:32.939044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.163433+0000","last_change":"2024-09-21T07:34:08.162638+0000","last_active":"2024-09-21T07:34:09.163433+0000","last_peered":"2024-09-21T07:34:09.163433+0000","last_clean":"2024-09-21T07:34:09.163433+0000","last_became_active":"2024-09-21T07:34:08.162237+0000","last_became_peered":"2024-09-21T07:34:08.162237+0000","last_unstale":"2024-09-21T07:34:09.163433+0000","last_undegraded":"2024-09-21T07:34:09.163433+0000","last_fullsized":"2024-09-21T07:34:09.163433+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:10.417627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497624+0000","last_change":"2024-09-21T07:34:07.152996+0000","last_active":"2024-09-21T07:34:09.497624+0000","last_peered":"2024-09-21T07:34:09.497624+0000","last_clean":"2024-09-21T07:34:09.497624+0000","last_became_active":"2024-09-21T07:34:07.152627+0000","last_became_peered":"2024-09-21T07:34:07.152627+0000","last_unstale":"2024-09-21T07:34:09.497624+0000","last_undegraded":"2024-09-21T07:34:09.497624+0000","last_fullsized":"2024-09-21T07:34:09.497624+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:24:42.920690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.163318+0000","last_change":"2024-09-21T07:34:08.173273+0000","last_active":"2024-09-21T07:34:09.163318+0000","last_peered":"2024-09-21T07:34:09.163318+0000","last_clean":"2024-09-21T07:34:09.163318+0000","last_became_active":"2024-09-21T07:34:08.172952+0000","last_became_peered":"2024-09-21T07:34:08.172952+0000","last_unstale":"2024-09-21T07:34:09.163318+0000","last_undegraded":"2024-09-21T07:34:09.163318+0000","last_fullsized":"2024-09-21T07:34:09.163318+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:47:39.799134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.169104+0000","last_change":"2024-09-21T07:34:07.151886+0000","last_active":"2024-09-21T07:34:09.169104+0000","last_peered":"2024-09-21T07:34:09.169104+0000","last_clean":"2024-09-21T07:34:09.169104+0000","last_became_active":"2024-09-21T07:34:07.150347+0000","last_became_peered":"2024-09-21T07:34:07.150347+0000","last_unstale":"2024-09-21T07:34:09.169104+0000","last_undegraded":"2024-09-21T07:34:09.169104+0000","last_fullsized":"2024-09-21T07:34:09.169104+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:27.230155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.163623+0000","last_change":"2024-09-21T07:34:08.174497+0000","last_active":"2024-09-21T07:34:09.163623+0000","last_peered":"2024-09-21T07:34:09.163623+0000","last_clean":"2024-09-21T07:34:09.163623+0000","last_became_active":"2024-09-21T07:34:08.173554+0000","last_became_peered":"2024-09-21T07:34:08.173554+0000","last_unstale":"2024-09-21T07:34:09.163623+0000","last_undegraded":"2024-09-21T07:34:09.163623+0000","last_fullsized":"2024-09-21T07:34:09.163623+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:18:16.768539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496838+0000","last_change":"2024-09-21T07:34:07.161337+0000","last_active":"2024-09-21T07:34:09.496838+0000","last_peered":"2024-09-21T07:34:09.496838+0000","last_clean":"2024-09-21T07:34:09.496838+0000","last_became_active":"2024-09-21T07:34:07.161055+0000","last_became_peered":"2024-09-21T07:34:07.161055+0000","last_unstale":"2024-09-21T07:34:09.496838+0000","last_undegraded":"2024-09-21T07:34:09.496838+0000","last_fullsized":"2024-09-21T07:34:09.496838+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:09.410183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.163185+0000","last_change":"2024-09-21T07:34:08.170546+0000","last_active":"2024-09-21T07:34:09.163185+0000","last_peered":"2024-09-21T07:34:09.163185+0000","last_clean":"2024-09-21T07:34:09.163185+0000","last_became_active":"2024-09-21T07:34:08.170249+0000","last_became_peered":"2024-09-21T07:34:08.170249+0000","last_unstale":"2024-09-21T07:34:09.163185+0000","last_undegraded":"2024-09-21T07:34:09.163185+0000","last_fullsized":"2024-09-21T07:34:09.163185+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:54.043603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497320+0000","last_change":"2024-09-21T07:34:07.148155+0000","last_active":"2024-09-21T07:34:09.497320+0000","last_peered":"2024-09-21T07:34:09.497320+0000","last_clean":"2024-09-21T07:34:09.497320+0000","last_became_active":"2024-09-21T07:34:07.147855+0000","last_became_peered":"2024-09-21T07:34:07.147855+0000","last_unstale":"2024-09-21T07:34:09.497320+0000","last_undegraded":"2024-09-21T07:34:09.497320+0000","last_fullsized":"2024-09-21T07:34:09.497320+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:41:10.667777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.329218+0000","last_change":"2024-09-21T07:34:08.165763+0000","last_active":"2024-09-21T07:34:09.329218+0000","last_peered":"2024-09-21T07:34:09.329218+0000","last_clean":"2024-09-21T07:34:09.329218+0000","last_became_active":"2024-09-21T07:34:08.163699+0000","last_became_peered":"2024-09-21T07:34:08.163699+0000","last_unstale":"2024-09-21T07:34:09.329218+0000","last_undegraded":"2024-09-21T07:34:09.329218+0000","last_fullsized":"2024-09-21T07:34:09.329218+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:14:22.162876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.165169+0000","last_change":"2024-09-21T07:34:07.156707+0000","last_active":"2024-09-21T07:34:09.165169+0000","last_peered":"2024-09-21T07:34:09.165169+0000","last_clean":"2024-09-21T07:34:09.165169+0000","last_became_active":"2024-09-21T07:34:07.156326+0000","last_became_peered":"2024-09-21T07:34:07.156326+0000","last_unstale":"2024-09-21T07:34:09.165169+0000","last_undegraded":"2024-09-21T07:34:09.165169+0000","last_fullsized":"2024-09-21T07:34:09.165169+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:54.145012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.163369+0000","last_change":"2024-09-21T07:34:08.166587+0000","last_active":"2024-09-21T07:34:09.163369+0000","last_peered":"2024-09-21T07:34:09.163369+0000","last_clean":"2024-09-21T07:34:09.163369+0000","last_became_active":"2024-09-21T07:34:08.166262+0000","last_became_peered":"2024-09-21T07:34:08.166262+0000","last_unstale":"2024-09-21T07:34:09.163369+0000","last_undegraded":"2024-09-21T07:34:09.163369+0000","last_fullsized":"2024-09-21T07:34:09.163369+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:27.889707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.181531+0000","last_change":"2024-09-21T07:34:07.153252+0000","last_active":"2024-09-21T07:34:09.181531+0000","last_peered":"2024-09-21T07:34:09.181531+0000","last_clean":"2024-09-21T07:34:09.181531+0000","last_became_active":"2024-09-21T07:34:07.152932+0000","last_became_peered":"2024-09-21T07:34:07.152932+0000","last_unstale":"2024-09-21T07:34:09.181531+0000","last_undegraded":"2024-09-21T07:34:09.181531+0000","last_fullsized":"2024-09-21T07:34:09.181531+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:24:14.371118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.165881+0000","last_change":"2024-09-21T07:34:08.171574+0000","last_active":"2024-09-21T07:34:09.165881+0000","last_peered":"2024-09-21T07:34:09.165881+0000","last_clean":"2024-09-21T07:34:09.165881+0000","last_became_active":"2024-09-21T07:34:08.170992+0000","last_became_peered":"2024-09-21T07:34:08.170992+0000","last_unstale":"2024-09-21T07:34:09.165881+0000","last_undegraded":"2024-09-21T07:34:09.165881+0000","last_fullsized":"2024-09-21T07:34:09.165881+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:31.524701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.169650+0000","last_change":"2024-09-21T07:34:07.161285+0000","last_active":"2024-09-21T07:34:09.169650+0000","last_peered":"2024-09-21T07:34:09.169650+0000","last_clean":"2024-09-21T07:34:09.169650+0000","last_became_active":"2024-09-21T07:34:07.160925+0000","last_became_peered":"2024-09-21T07:34:07.160925+0000","last_unstale":"2024-09-21T07:34:09.169650+0000","last_undegraded":"2024-09-21T07:34:09.169650+0000","last_fullsized":"2024-09-21T07:34:09.169650+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:54.784501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.169133+0000","last_change":"2024-09-21T07:34:08.160436+0000","last_active":"2024-09-21T07:34:09.169133+0000","last_peered":"2024-09-21T07:34:09.169133+0000","last_clean":"2024-09-21T07:34:09.169133+0000","last_became_active":"2024-09-21T07:34:08.159040+0000","last_became_peered":"2024-09-21T07:34:08.159040+0000","last_unstale":"2024-09-21T07:34:09.169133+0000","last_undegraded":"2024-09-21T07:34:09.169133+0000","last_fullsized":"2024-09-21T07:34:09.169133+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:30.259079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.181020+0000","last_change":"2024-09-21T07:34:07.151263+0000","last_active":"2024-09-21T07:34:09.181020+0000","last_peered":"2024-09-21T07:34:09.181020+0000","last_clean":"2024-09-21T07:34:09.181020+0000","last_became_active":"2024-09-21T07:34:07.150981+0000","last_became_peered":"2024-09-21T07:34:07.150981+0000","last_unstale":"2024-09-21T07:34:09.181020+0000","last_undegraded":"2024-09-21T07:34:09.181020+0000","last_fullsized":"2024-09-21T07:34:09.181020+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:26:19.106553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496799+0000","last_change":"2024-09-21T07:34:08.167728+0000","last_active":"2024-09-21T07:34:09.496799+0000","last_peered":"2024-09-21T07:34:09.496799+0000","last_clean":"2024-09-21T07:34:09.496799+0000","last_became_active":"2024-09-21T07:34:08.167409+0000","last_became_peered":"2024-09-21T07:34:08.167409+0000","last_unstale":"2024-09-21T07:34:09.496799+0000","last_undegraded":"2024-09-21T07:34:09.496799+0000","last_fullsized":"2024-09-21T07:34:09.496799+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:55.716857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.169272+0000","last_change":"2024-09-21T07:34:07.139885+0000","last_active":"2024-09-21T07:34:09.169272+0000","last_peered":"2024-09-21T07:34:09.169272+0000","last_clean":"2024-09-21T07:34:09.169272+0000","last_became_active":"2024-09-21T07:34:07.139560+0000","last_became_peered":"2024-09-21T07:34:07.139560+0000","last_unstale":"2024-09-21T07:34:09.169272+0000","last_undegraded":"2024-09-21T07:34:09.169272+0000","last_fullsized":"2024-09-21T07:34:09.169272+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:18:15.761095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.169169+0000","last_change":"2024-09-21T07:34:08.165778+0000","last_active":"2024-09-21T07:34:09.169169+0000","last_peered":"2024-09-21T07:34:09.169169+0000","last_clean":"2024-09-21T07:34:09.169169+0000","last_became_active":"2024-09-21T07:34:08.165481+0000","last_became_peered":"2024-09-21T07:34:08.165481+0000","last_unstale":"2024-09-21T07:34:09.169169+0000","last_undegraded":"2024-09-21T07:34:09.169169+0000","last_fullsized":"2024-09-21T07:34:09.169169+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:47.458768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.180835+0000","last_change":"2024-09-21T07:34:07.152465+0000","last_active":"2024-09-21T07:34:09.180835+0000","last_peered":"2024-09-21T07:34:09.180835+0000","last_clean":"2024-09-21T07:34:09.180835+0000","last_became_active":"2024-09-21T07:34:07.152065+0000","last_became_peered":"2024-09-21T07:34:07.152065+0000","last_unstale":"2024-09-21T07:34:09.180835+0000","last_undegraded":"2024-09-21T07:34:09.180835+0000","last_fullsized":"2024-09-21T07:34:09.180835+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:24:30.050440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496179+0000","last_change":"2024-09-21T07:34:08.150846+0000","last_active":"2024-09-21T07:34:09.496179+0000","last_peered":"2024-09-21T07:34:09.496179+0000","last_clean":"2024-09-21T07:34:09.496179+0000","last_became_active":"2024-09-21T07:34:08.150534+0000","last_became_peered":"2024-09-21T07:34:08.150534+0000","last_unstale":"2024-09-21T07:34:09.496179+0000","last_undegraded":"2024-09-21T07:34:09.496179+0000","last_fullsized":"2024-09-21T07:34:09.496179+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:26:49.717420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497461+0000","last_change":"2024-09-21T07:34:07.147358+0000","last_active":"2024-09-21T07:34:09.497461+0000","last_peered":"2024-09-21T07:34:09.497461+0000","last_clean":"2024-09-21T07:34:09.497461+0000","last_became_active":"2024-09-21T07:34:07.147036+0000","last_became_peered":"2024-09-21T07:34:07.147036+0000","last_unstale":"2024-09-21T07:34:09.497461+0000","last_undegraded":"2024-09-21T07:34:09.497461+0000","last_fullsized":"2024-09-21T07:34:09.497461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:04:22.493944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497584+0000","last_change":"2024-09-21T07:34:08.159510+0000","last_active":"2024-09-21T07:34:09.497584+0000","last_peered":"2024-09-21T07:34:09.497584+0000","last_clean":"2024-09-21T07:34:09.497584+0000","last_became_active":"2024-09-21T07:34:08.159191+0000","last_became_peered":"2024-09-21T07:34:08.159191+0000","last_unstale":"2024-09-21T07:34:09.497584+0000","last_undegraded":"2024-09-21T07:34:09.497584+0000","last_fullsized":"2024-09-21T07:34:09.497584+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:27.319525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496085+0000","last_change":"2024-09-21T07:34:07.155987+0000","last_active":"2024-09-21T07:34:09.496085+0000","last_peered":"2024-09-21T07:34:09.496085+0000","last_clean":"2024-09-21T07:34:09.496085+0000","last_became_active":"2024-09-21T07:34:07.155735+0000","last_became_peered":"2024-09-21T07:34:07.155735+0000","last_unstale":"2024-09-21T07:34:09.496085+0000","last_undegraded":"2024-09-21T07:34:09.496085+0000","last_fullsized":"2024-09-21T07:34:09.496085+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:23:21.954791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.329467+0000","last_change":"2024-09-21T07:34:08.167002+0000","last_active":"2024-09-21T07:34:09.329467+0000","last_peered":"2024-09-21T07:34:09.329467+0000","last_clean":"2024-09-21T07:34:09.329467+0000","last_became_active":"2024-09-21T07:34:08.166679+0000","last_became_peered":"2024-09-21T07:34:08.166679+0000","last_unstale":"2024-09-21T07:34:09.329467+0000","last_undegraded":"2024-09-21T07:34:09.329467+0000","last_fullsized":"2024-09-21T07:34:09.329467+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:52.642170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496567+0000","last_change":"2024-09-21T07:34:07.155156+0000","last_active":"2024-09-21T07:34:09.496567+0000","last_peered":"2024-09-21T07:34:09.496567+0000","last_clean":"2024-09-21T07:34:09.496567+0000","last_became_active":"2024-09-21T07:34:07.154829+0000","last_became_peered":"2024-09-21T07:34:07.154829+0000","last_unstale":"2024-09-21T07:34:09.496567+0000","last_undegraded":"2024-09-21T07:34:09.496567+0000","last_fullsized":"2024-09-21T07:34:09.496567+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:59:04.464801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":135,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.329996+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:34:09.329996+0000","last_peered":"2024-09-21T07:34:09.329996+0000","last_clean":"2024-09-21T07:34:09.329996+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:34:09.329996+0000","last_undegraded":"2024-09-21T07:34:09.329996+0000","last_fullsized":"2024-09-21T07:34:09.329996+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509457,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":974848,"data_stored":739153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509459,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":385024,"data_stored":149311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509457,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":385024,"data_stored":149367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509458,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31568,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340272,"statfs":{"total":96636764160,"available":96604438528,"internally_reserved":0,"allocated":380928,"data_stored":148765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509459,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":393216,"data_stored":150329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509459,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":970752,"data_stored":739063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542162,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":385024,"data_stored":148799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542161,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":393216,"data_stored":150329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:34:13.670 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T07:34:13.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.931+0000 7fc67b600640 1 -- 172.21.15.39:0/872972937 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc674070440 msgr2=0x7fc6740b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:13.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.931+0000 7fc67b600640 1 --2- 172.21.15.39:0/872972937 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc674070440 0x7fc6740b3090 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc66c005fd0 tx=0x7fc66c02a8b0 comp rx=0 tx=0).stop 2024-09-21T07:34:13.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.931+0000 7fc67b600640 1 -- 172.21.15.39:0/872972937 shutdown_connections 2024-09-21T07:34:13.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.931+0000 7fc67b600640 1 --2- 172.21.15.39:0/872972937 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc674070440 0x7fc6740b3090 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:13.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.931+0000 7fc67b600640 1 --2- 172.21.15.39:0/872972937 >> v2:172.21.15.148:3300/0 conn(0x7fc674073170 0x7fc67415c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:13.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.931+0000 7fc67b600640 1 -- 172.21.15.39:0/872972937 >> 172.21.15.39:0/872972937 conn(0x7fc6740b1cb0 msgr2=0x7fc6740b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:13.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.931+0000 7fc67b600640 1 -- 172.21.15.39:0/872972937 shutdown_connections 2024-09-21T07:34:13.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.931+0000 7fc67b600640 1 -- 172.21.15.39:0/872972937 wait complete. 2024-09-21T07:34:13.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.933+0000 7fc67b600640 1 Processor -- start 2024-09-21T07:34:13.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.933+0000 7fc67b600640 1 -- start start 2024-09-21T07:34:13.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.935+0000 7fc67b600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc674070030 0x7fc674157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:13.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.935+0000 7fc67b600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc674070440 0x7fc6740d0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:13.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.935+0000 7fc67b600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc674073170 0x7fc6740cf070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:13.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.935+0000 7fc67b600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc6740b3b00 con 0x7fc674070440 2024-09-21T07:34:13.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.935+0000 7fc67b600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc6740b3980 con 0x7fc674073170 2024-09-21T07:34:13.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.935+0000 7fc67b600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc6740b3c80 con 0x7fc674070030 2024-09-21T07:34:13.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.935+0000 7fc673e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc674070440 0x7fc6740d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:13.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.935+0000 7fc673e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc674070440 0x7fc6740d0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41428/0 (socket says 172.21.15.39:41428) 2024-09-21T07:34:13.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.935+0000 7fc673e00640 1 -- 172.21.15.39:0/2281945078 learned_addr learned my addr 172.21.15.39:0/2281945078 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:13.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.935+0000 7fc678c00640 1 --2- 172.21.15.39:0/2281945078 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc674070030 0x7fc674157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:13.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.935+0000 7fc679600640 1 --2- 172.21.15.39:0/2281945078 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc674073170 0x7fc6740cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:13.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.935+0000 7fc673e00640 1 -- 172.21.15.39:0/2281945078 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc674070030 msgr2=0x7fc674157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:13.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.935+0000 7fc673e00640 1 --2- 172.21.15.39:0/2281945078 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc674070030 0x7fc674157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:13.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.935+0000 7fc673e00640 1 -- 172.21.15.39:0/2281945078 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc674073170 msgr2=0x7fc6740cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:13.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.935+0000 7fc673e00640 1 --2- 172.21.15.39:0/2281945078 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc674073170 0x7fc6740cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:13.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.935+0000 7fc673e00640 1 -- 172.21.15.39:0/2281945078 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6741584b0 con 0x7fc674070440 2024-09-21T07:34:13.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.935+0000 7fc679600640 1 --2- 172.21.15.39:0/2281945078 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc674073170 0x7fc6740cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:34:13.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.936+0000 7fc673e00640 1 --2- 172.21.15.39:0/2281945078 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc674070440 0x7fc6740d0340 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc66000c400 tx=0x7fc66000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:13.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.936+0000 7fc670c00640 1 -- 172.21.15.39:0/2281945078 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc660011790 con 0x7fc674070440 2024-09-21T07:34:13.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.936+0000 7fc670c00640 1 -- 172.21.15.39:0/2281945078 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc660017070 con 0x7fc674070440 2024-09-21T07:34:13.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.936+0000 7fc670c00640 1 -- 172.21.15.39:0/2281945078 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc660011cc0 con 0x7fc674070440 2024-09-21T07:34:13.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.936+0000 7fc67b600640 1 -- 172.21.15.39:0/2281945078 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc674155630 con 0x7fc674070440 2024-09-21T07:34:13.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.936+0000 7fc67b600640 1 -- 172.21.15.39:0/2281945078 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6741f6950 con 0x7fc674070440 2024-09-21T07:34:13.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.937+0000 7fc67b600640 1 -- 172.21.15.39:0/2281945078 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6740b48f0 con 0x7fc674070440 2024-09-21T07:34:13.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.939+0000 7fc670c00640 1 -- 172.21.15.39:0/2281945078 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc66001c460 con 0x7fc674070440 2024-09-21T07:34:13.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.939+0000 7fc670c00640 1 --2- 172.21.15.39:0/2281945078 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc654078b90 0x7fc65407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:13.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.940+0000 7fc670c00640 1 -- 172.21.15.39:0/2281945078 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fc66009ab20 con 0x7fc674070440 2024-09-21T07:34:13.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.940+0000 7fc678c00640 1 --2- 172.21.15.39:0/2281945078 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc654078b90 0x7fc65407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:13.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.941+0000 7fc678c00640 1 --2- 172.21.15.39:0/2281945078 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc654078b90 0x7fc65407b0a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc668009b30 tx=0x7fc668005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:13.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:13.944+0000 7fc670c00640 1 -- 172.21.15.39:0/2281945078 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc660062f80 con 0x7fc674070440 2024-09-21T07:34:14.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.218+0000 7fc67b600640 1 -- 172.21.15.39:0/2281945078 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fc6740c3860 con 0x7fc674070440 2024-09-21T07:34:14.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.218+0000 7fc670c00640 1 -- 172.21.15.39:0/2281945078 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1602 (secure 0 0 0) 0x7fc660062880 con 0x7fc674070440 2024-09-21T07:34:14.221 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:34:14.221 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":87,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":34,"num_osds":8,"num_up_osds":8,"osd_up_since":1726903982,"num_in_osds":8,"osd_in_since":1726903972,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":242753536,"bytes_avail":772851359744,"bytes_total":773094113280,"write_bytes_sec":2232,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":17,"btime":"2024-09-21T07:34:11:169975+0000","id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":4888}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-21T07:34:03.545890+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-21T07:34:14.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.221+0000 7fc67b600640 1 -- 172.21.15.39:0/2281945078 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc654078b90 msgr2=0x7fc65407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:14.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.221+0000 7fc67b600640 1 --2- 172.21.15.39:0/2281945078 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc654078b90 0x7fc65407b0a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc668009b30 tx=0x7fc668005f90 comp rx=0 tx=0).stop 2024-09-21T07:34:14.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.221+0000 7fc67b600640 1 -- 172.21.15.39:0/2281945078 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc674070440 msgr2=0x7fc6740d0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:14.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.221+0000 7fc67b600640 1 --2- 172.21.15.39:0/2281945078 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc674070440 0x7fc6740d0340 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc66000c400 tx=0x7fc66000c8c0 comp rx=0 tx=0).stop 2024-09-21T07:34:14.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.221+0000 7fc67b600640 1 -- 172.21.15.39:0/2281945078 shutdown_connections 2024-09-21T07:34:14.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.221+0000 7fc67b600640 1 --2- 172.21.15.39:0/2281945078 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc654078b90 0x7fc65407b0a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:14.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.221+0000 7fc67b600640 1 --2- 172.21.15.39:0/2281945078 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc674073170 0x7fc6740cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:14.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.221+0000 7fc67b600640 1 --2- 172.21.15.39:0/2281945078 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc674070440 0x7fc6740d0340 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:14.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.221+0000 7fc67b600640 1 --2- 172.21.15.39:0/2281945078 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc674070030 0x7fc674157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:14.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.221+0000 7fc67b600640 1 -- 172.21.15.39:0/2281945078 >> 172.21.15.39:0/2281945078 conn(0x7fc6740b1cb0 msgr2=0x7fc6740b12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:14.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.221+0000 7fc67b600640 1 -- 172.21.15.39:0/2281945078 shutdown_connections 2024-09-21T07:34:14.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.221+0000 7fc67b600640 1 -- 172.21.15.39:0/2281945078 wait complete. 2024-09-21T07:34:14.247 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-21T07:34:14.247 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:34:14.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.472+0000 7fab19800640 1 -- 172.21.15.39:0/1495545495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab14142b00 msgr2=0x7fab1415ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:14.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.472+0000 7fab19800640 1 --2- 172.21.15.39:0/1495545495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab14142b00 0x7fab1415ca80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fab00005fd0 tx=0x7fab0002a670 comp rx=0 tx=0).stop 2024-09-21T07:34:14.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.472+0000 7fab19800640 1 -- 172.21.15.39:0/1495545495 shutdown_connections 2024-09-21T07:34:14.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.472+0000 7fab19800640 1 --2- 172.21.15.39:0/1495545495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab14142b00 0x7fab1415ca80 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:14.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.472+0000 7fab19800640 1 -- 172.21.15.39:0/1495545495 >> 172.21.15.39:0/1495545495 conn(0x7fab1406d410 msgr2=0x7fab140b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:14.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.472+0000 7fab19800640 1 -- 172.21.15.39:0/1495545495 shutdown_connections 2024-09-21T07:34:14.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.472+0000 7fab19800640 1 -- 172.21.15.39:0/1495545495 wait complete. 2024-09-21T07:34:14.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.474+0000 7fab19800640 1 Processor -- start 2024-09-21T07:34:14.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.474+0000 7fab19800640 1 -- start start 2024-09-21T07:34:14.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.476+0000 7fab19800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab140ced60 0x7fab140b4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:14.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.476+0000 7fab19800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab14142b00 0x7fab1413dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:14.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.476+0000 7fab19800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab141600c0 0x7fab14143ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:14.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.476+0000 7fab19800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fab1406fae0 con 0x7fab141600c0 2024-09-21T07:34:14.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.476+0000 7fab19800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fab1406f960 con 0x7fab140ced60 2024-09-21T07:34:14.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.476+0000 7fab19800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fab1406fc60 con 0x7fab14142b00 2024-09-21T07:34:14.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.476+0000 7fab18e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab141600c0 0x7fab14143ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:14.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.476+0000 7fab18e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab141600c0 0x7fab14143ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41454/0 (socket says 172.21.15.39:41454) 2024-09-21T07:34:14.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.476+0000 7fab18e00640 1 -- 172.21.15.39:0/205288305 learned_addr learned my addr 172.21.15.39:0/205288305 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:14.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.476+0000 7fab13e00640 1 --2- 172.21.15.39:0/205288305 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab140ced60 0x7fab140b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:14.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.476+0000 7fab13400640 1 --2- 172.21.15.39:0/205288305 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab14142b00 0x7fab1413dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:14.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.477+0000 7fab18e00640 1 -- 172.21.15.39:0/205288305 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab14142b00 msgr2=0x7fab1413dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:14.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.477+0000 7fab18e00640 1 --2- 172.21.15.39:0/205288305 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab14142b00 0x7fab1413dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:14.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.477+0000 7fab18e00640 1 -- 172.21.15.39:0/205288305 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab140ced60 msgr2=0x7fab140b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:14.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.477+0000 7fab18e00640 1 --2- 172.21.15.39:0/205288305 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab140ced60 0x7fab140b4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:14.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.477+0000 7fab18e00640 1 -- 172.21.15.39:0/205288305 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab14143230 con 0x7fab141600c0 2024-09-21T07:34:14.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.477+0000 7fab18e00640 1 --2- 172.21.15.39:0/205288305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab141600c0 0x7fab14143ff0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fab0c002230 tx=0x7fab0c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:14.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.477+0000 7fab0be00640 1 -- 172.21.15.39:0/205288305 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0c003b70 con 0x7fab141600c0 2024-09-21T07:34:14.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.477+0000 7fab0be00640 1 -- 172.21.15.39:0/205288305 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fab0c003d10 con 0x7fab141600c0 2024-09-21T07:34:14.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.477+0000 7fab0be00640 1 -- 172.21.15.39:0/205288305 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0c005330 con 0x7fab141600c0 2024-09-21T07:34:14.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.477+0000 7fab19800640 1 -- 172.21.15.39:0/205288305 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab14143510 con 0x7fab141600c0 2024-09-21T07:34:14.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.477+0000 7fab19800640 1 -- 172.21.15.39:0/205288305 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab14143ad0 con 0x7fab141600c0 2024-09-21T07:34:14.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.479+0000 7fab19800640 1 -- 172.21.15.39:0/205288305 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab1406a740 con 0x7fab141600c0 2024-09-21T07:34:14.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.480+0000 7fab0be00640 1 -- 172.21.15.39:0/205288305 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fab0c0056e0 con 0x7fab141600c0 2024-09-21T07:34:14.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.481+0000 7fab0be00640 1 --2- 172.21.15.39:0/205288305 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faaf4078bc0 0x7faaf407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:14.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.482+0000 7fab0be00640 1 -- 172.21.15.39:0/205288305 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fab0c09abc0 con 0x7fab141600c0 2024-09-21T07:34:14.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.482+0000 7fab13e00640 1 --2- 172.21.15.39:0/205288305 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faaf4078bc0 0x7faaf407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:14.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.483+0000 7fab13e00640 1 --2- 172.21.15.39:0/205288305 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faaf4078bc0 0x7faaf407b0d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fab04000f60 tx=0x7fab04009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:14.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.486+0000 7fab0be00640 1 -- 172.21.15.39:0/205288305 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fab0c0629a0 con 0x7fab141600c0 2024-09-21T07:34:14.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.681+0000 7fab19800640 1 -- 172.21.15.39:0/205288305 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fab1413e180 con 0x7faaf4078bc0 2024-09-21T07:34:14.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.686+0000 7fab0be00640 1 -- 172.21.15.39:0/205288305 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251940 (secure 0 0 0) 0x7fab04001cf0 con 0x7faaf4078bc0 2024-09-21T07:34:14.691 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:34:14.692 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:34:14.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.691+0000 7fab19800640 1 -- 172.21.15.39:0/205288305 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faaf4078bc0 msgr2=0x7faaf407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:14.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.691+0000 7fab19800640 1 --2- 172.21.15.39:0/205288305 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faaf4078bc0 0x7faaf407b0d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fab04000f60 tx=0x7fab04009040 comp rx=0 tx=0).stop 2024-09-21T07:34:14.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.691+0000 7fab19800640 1 -- 172.21.15.39:0/205288305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab141600c0 msgr2=0x7fab14143ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:14.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.691+0000 7fab19800640 1 --2- 172.21.15.39:0/205288305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab141600c0 0x7fab14143ff0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fab0c002230 tx=0x7fab0c00cb90 comp rx=0 tx=0).stop 2024-09-21T07:34:14.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.691+0000 7fab19800640 1 -- 172.21.15.39:0/205288305 shutdown_connections 2024-09-21T07:34:14.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.691+0000 7fab19800640 1 --2- 172.21.15.39:0/205288305 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faaf4078bc0 0x7faaf407b0d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:14.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.691+0000 7fab19800640 1 --2- 172.21.15.39:0/205288305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab141600c0 0x7fab14143ff0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:14.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.691+0000 7fab19800640 1 --2- 172.21.15.39:0/205288305 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab14142b00 0x7fab1413dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:14.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.691+0000 7fab19800640 1 --2- 172.21.15.39:0/205288305 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab140ced60 0x7fab140b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:14.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.691+0000 7fab19800640 1 -- 172.21.15.39:0/205288305 >> 172.21.15.39:0/205288305 conn(0x7fab1406d410 msgr2=0x7fab1415f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:14.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.691+0000 7fab19800640 1 -- 172.21.15.39:0/205288305 shutdown_connections 2024-09-21T07:34:14.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:14.691+0000 7fab19800640 1 -- 172.21.15.39:0/205288305 wait complete. 2024-09-21T07:34:14.715 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2024-09-21T07:34:13.548171+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237064,"kb_used_data":4168,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737656,"statfs":{"total":773094113280,"available":772851359744,"internally_reserved":0,"allocated":4268032,"data_stored":2375116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.421904"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496910+0000","last_change":"2024-09-21T07:34:08.172048+0000","last_active":"2024-09-21T07:34:09.496910+0000","last_peered":"2024-09-21T07:34:09.496910+0000","last_clean":"2024-09-21T07:34:09.496910+0000","last_became_active":"2024-09-21T07:34:08.170885+0000","last_became_peered":"2024-09-21T07:34:08.170885+0000","last_unstale":"2024-09-21T07:34:09.496910+0000","last_undegraded":"2024-09-21T07:34:09.496910+0000","last_fullsized":"2024-09-21T07:34:09.496910+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:30.259079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496812+0000","last_change":"2024-09-21T07:34:08.163227+0000","last_active":"2024-09-21T07:34:09.496812+0000","last_peered":"2024-09-21T07:34:09.496812+0000","last_clean":"2024-09-21T07:34:09.496812+0000","last_became_active":"2024-09-21T07:34:08.162915+0000","last_became_peered":"2024-09-21T07:34:08.162915+0000","last_unstale":"2024-09-21T07:34:09.496812+0000","last_undegraded":"2024-09-21T07:34:09.496812+0000","last_fullsized":"2024-09-21T07:34:09.496812+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:16.292830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.165484+0000","last_change":"2024-09-21T07:34:08.161784+0000","last_active":"2024-09-21T07:34:09.165484+0000","last_peered":"2024-09-21T07:34:09.165484+0000","last_clean":"2024-09-21T07:34:09.165484+0000","last_became_active":"2024-09-21T07:34:08.161467+0000","last_became_peered":"2024-09-21T07:34:08.161467+0000","last_unstale":"2024-09-21T07:34:09.165484+0000","last_undegraded":"2024-09-21T07:34:09.165484+0000","last_fullsized":"2024-09-21T07:34:09.165484+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:14:22.162876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.180849+0000","last_change":"2024-09-21T07:34:08.174981+0000","last_active":"2024-09-21T07:34:09.180849+0000","last_peered":"2024-09-21T07:34:09.180849+0000","last_clean":"2024-09-21T07:34:09.180849+0000","last_became_active":"2024-09-21T07:34:08.174163+0000","last_became_peered":"2024-09-21T07:34:08.174163+0000","last_unstale":"2024-09-21T07:34:09.180849+0000","last_undegraded":"2024-09-21T07:34:09.180849+0000","last_fullsized":"2024-09-21T07:34:09.180849+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:08.608518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.163240+0000","last_change":"2024-09-21T07:34:08.170011+0000","last_active":"2024-09-21T07:34:09.163240+0000","last_peered":"2024-09-21T07:34:09.163240+0000","last_clean":"2024-09-21T07:34:09.163240+0000","last_became_active":"2024-09-21T07:34:08.169688+0000","last_became_peered":"2024-09-21T07:34:08.169688+0000","last_unstale":"2024-09-21T07:34:09.163240+0000","last_undegraded":"2024-09-21T07:34:09.163240+0000","last_fullsized":"2024-09-21T07:34:09.163240+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:59:05.472245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496422+0000","last_change":"2024-09-21T07:34:08.161341+0000","last_active":"2024-09-21T07:34:09.496422+0000","last_peered":"2024-09-21T07:34:09.496422+0000","last_clean":"2024-09-21T07:34:09.496422+0000","last_became_active":"2024-09-21T07:34:08.161037+0000","last_became_peered":"2024-09-21T07:34:08.161037+0000","last_unstale":"2024-09-21T07:34:09.496422+0000","last_undegraded":"2024-09-21T07:34:09.496422+0000","last_fullsized":"2024-09-21T07:34:09.496422+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:52.361740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.329349+0000","last_change":"2024-09-21T07:34:08.173122+0000","last_active":"2024-09-21T07:34:09.329349+0000","last_peered":"2024-09-21T07:34:09.329349+0000","last_clean":"2024-09-21T07:34:09.329349+0000","last_became_active":"2024-09-21T07:34:08.172780+0000","last_became_peered":"2024-09-21T07:34:08.172780+0000","last_unstale":"2024-09-21T07:34:09.329349+0000","last_undegraded":"2024-09-21T07:34:09.329349+0000","last_fullsized":"2024-09-21T07:34:09.329349+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:27.710069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.329093+0000","last_change":"2024-09-21T07:34:08.167193+0000","last_active":"2024-09-21T07:34:09.329093+0000","last_peered":"2024-09-21T07:34:09.329093+0000","last_clean":"2024-09-21T07:34:09.329093+0000","last_became_active":"2024-09-21T07:34:08.165230+0000","last_became_peered":"2024-09-21T07:34:08.165230+0000","last_unstale":"2024-09-21T07:34:09.329093+0000","last_undegraded":"2024-09-21T07:34:09.329093+0000","last_fullsized":"2024-09-21T07:34:09.329093+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:39.114639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497858+0000","last_change":"2024-09-21T07:34:08.170445+0000","last_active":"2024-09-21T07:34:09.497858+0000","last_peered":"2024-09-21T07:34:09.497858+0000","last_clean":"2024-09-21T07:34:09.497858+0000","last_became_active":"2024-09-21T07:34:08.170115+0000","last_became_peered":"2024-09-21T07:34:08.170115+0000","last_unstale":"2024-09-21T07:34:09.497858+0000","last_undegraded":"2024-09-21T07:34:09.497858+0000","last_fullsized":"2024-09-21T07:34:09.497858+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:32.224498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.181226+0000","last_change":"2024-09-21T07:34:08.162257+0000","last_active":"2024-09-21T07:34:09.181226+0000","last_peered":"2024-09-21T07:34:09.181226+0000","last_clean":"2024-09-21T07:34:09.181226+0000","last_became_active":"2024-09-21T07:34:08.161939+0000","last_became_peered":"2024-09-21T07:34:08.161939+0000","last_unstale":"2024-09-21T07:34:09.181226+0000","last_undegraded":"2024-09-21T07:34:09.181226+0000","last_fullsized":"2024-09-21T07:34:09.181226+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:32:50.694783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.329426+0000","last_change":"2024-09-21T07:34:08.177569+0000","last_active":"2024-09-21T07:34:09.329426+0000","last_peered":"2024-09-21T07:34:09.329426+0000","last_clean":"2024-09-21T07:34:09.329426+0000","last_became_active":"2024-09-21T07:34:08.177198+0000","last_became_peered":"2024-09-21T07:34:08.177198+0000","last_unstale":"2024-09-21T07:34:09.329426+0000","last_undegraded":"2024-09-21T07:34:09.329426+0000","last_fullsized":"2024-09-21T07:34:09.329426+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:50.102777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.165047+0000","last_change":"2024-09-21T07:34:08.155812+0000","last_active":"2024-09-21T07:34:09.165047+0000","last_peered":"2024-09-21T07:34:09.165047+0000","last_clean":"2024-09-21T07:34:09.165047+0000","last_became_active":"2024-09-21T07:34:08.155490+0000","last_became_peered":"2024-09-21T07:34:08.155490+0000","last_unstale":"2024-09-21T07:34:09.165047+0000","last_undegraded":"2024-09-21T07:34:09.165047+0000","last_fullsized":"2024-09-21T07:34:09.165047+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:13.730449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.163440+0000","last_change":"2024-09-21T07:34:08.174944+0000","last_active":"2024-09-21T07:34:09.163440+0000","last_peered":"2024-09-21T07:34:09.163440+0000","last_clean":"2024-09-21T07:34:09.163440+0000","last_became_active":"2024-09-21T07:34:08.173967+0000","last_became_peered":"2024-09-21T07:34:08.173967+0000","last_unstale":"2024-09-21T07:34:09.163440+0000","last_undegraded":"2024-09-21T07:34:09.163440+0000","last_fullsized":"2024-09-21T07:34:09.163440+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:55.791945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496432+0000","last_change":"2024-09-21T07:34:08.167950+0000","last_active":"2024-09-21T07:34:09.496432+0000","last_peered":"2024-09-21T07:34:09.496432+0000","last_clean":"2024-09-21T07:34:09.496432+0000","last_became_active":"2024-09-21T07:34:08.167687+0000","last_became_peered":"2024-09-21T07:34:08.167687+0000","last_unstale":"2024-09-21T07:34:09.496432+0000","last_undegraded":"2024-09-21T07:34:09.496432+0000","last_fullsized":"2024-09-21T07:34:09.496432+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:44:01.238051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.164897+0000","last_change":"2024-09-21T07:34:08.166495+0000","last_active":"2024-09-21T07:34:09.164897+0000","last_peered":"2024-09-21T07:34:09.164897+0000","last_clean":"2024-09-21T07:34:09.164897+0000","last_became_active":"2024-09-21T07:34:08.166104+0000","last_became_peered":"2024-09-21T07:34:08.166104+0000","last_unstale":"2024-09-21T07:34:09.164897+0000","last_undegraded":"2024-09-21T07:34:09.164897+0000","last_fullsized":"2024-09-21T07:34:09.164897+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:31:12.988796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.329881+0000","last_change":"2024-09-21T07:34:08.172013+0000","last_active":"2024-09-21T07:34:09.329881+0000","last_peered":"2024-09-21T07:34:09.329881+0000","last_clean":"2024-09-21T07:34:09.329881+0000","last_became_active":"2024-09-21T07:34:08.171497+0000","last_became_peered":"2024-09-21T07:34:08.171497+0000","last_unstale":"2024-09-21T07:34:09.329881+0000","last_undegraded":"2024-09-21T07:34:09.329881+0000","last_fullsized":"2024-09-21T07:34:09.329881+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:28.526013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.329782+0000","last_change":"2024-09-21T07:34:08.176187+0000","last_active":"2024-09-21T07:34:09.329782+0000","last_peered":"2024-09-21T07:34:09.329782+0000","last_clean":"2024-09-21T07:34:09.329782+0000","last_became_active":"2024-09-21T07:34:08.175895+0000","last_became_peered":"2024-09-21T07:34:08.175895+0000","last_unstale":"2024-09-21T07:34:09.329782+0000","last_undegraded":"2024-09-21T07:34:09.329782+0000","last_fullsized":"2024-09-21T07:34:09.329782+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:10.107127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496674+0000","last_change":"2024-09-21T07:34:08.153003+0000","last_active":"2024-09-21T07:34:09.496674+0000","last_peered":"2024-09-21T07:34:09.496674+0000","last_clean":"2024-09-21T07:34:09.496674+0000","last_became_active":"2024-09-21T07:34:08.150882+0000","last_became_peered":"2024-09-21T07:34:08.150882+0000","last_unstale":"2024-09-21T07:34:09.496674+0000","last_undegraded":"2024-09-21T07:34:09.496674+0000","last_fullsized":"2024-09-21T07:34:09.496674+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:38.611907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496759+0000","last_change":"2024-09-21T07:34:08.158999+0000","last_active":"2024-09-21T07:34:09.496759+0000","last_peered":"2024-09-21T07:34:09.496759+0000","last_clean":"2024-09-21T07:34:09.496759+0000","last_became_active":"2024-09-21T07:34:08.158712+0000","last_became_peered":"2024-09-21T07:34:08.158712+0000","last_unstale":"2024-09-21T07:34:09.496759+0000","last_undegraded":"2024-09-21T07:34:09.496759+0000","last_fullsized":"2024-09-21T07:34:09.496759+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:34:04.145536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.163438+0000","last_change":"2024-09-21T07:34:08.168400+0000","last_active":"2024-09-21T07:34:09.163438+0000","last_peered":"2024-09-21T07:34:09.163438+0000","last_clean":"2024-09-21T07:34:09.163438+0000","last_became_active":"2024-09-21T07:34:08.168071+0000","last_became_peered":"2024-09-21T07:34:08.168071+0000","last_unstale":"2024-09-21T07:34:09.163438+0000","last_undegraded":"2024-09-21T07:34:09.163438+0000","last_fullsized":"2024-09-21T07:34:09.163438+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:01.249634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496796+0000","last_change":"2024-09-21T07:34:08.168578+0000","last_active":"2024-09-21T07:34:09.496796+0000","last_peered":"2024-09-21T07:34:09.496796+0000","last_clean":"2024-09-21T07:34:09.496796+0000","last_became_active":"2024-09-21T07:34:08.168235+0000","last_became_peered":"2024-09-21T07:34:08.168235+0000","last_unstale":"2024-09-21T07:34:09.496796+0000","last_undegraded":"2024-09-21T07:34:09.496796+0000","last_fullsized":"2024-09-21T07:34:09.496796+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:18.132264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496889+0000","last_change":"2024-09-21T07:34:08.159192+0000","last_active":"2024-09-21T07:34:09.496889+0000","last_peered":"2024-09-21T07:34:09.496889+0000","last_clean":"2024-09-21T07:34:09.496889+0000","last_became_active":"2024-09-21T07:34:08.158860+0000","last_became_peered":"2024-09-21T07:34:08.158860+0000","last_unstale":"2024-09-21T07:34:09.496889+0000","last_undegraded":"2024-09-21T07:34:09.496889+0000","last_fullsized":"2024-09-21T07:34:09.496889+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:48.287489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.168895+0000","last_change":"2024-09-21T07:34:08.164892+0000","last_active":"2024-09-21T07:34:09.168895+0000","last_peered":"2024-09-21T07:34:09.168895+0000","last_clean":"2024-09-21T07:34:09.168895+0000","last_became_active":"2024-09-21T07:34:08.163504+0000","last_became_peered":"2024-09-21T07:34:08.163504+0000","last_unstale":"2024-09-21T07:34:09.168895+0000","last_undegraded":"2024-09-21T07:34:09.168895+0000","last_fullsized":"2024-09-21T07:34:09.168895+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:55.716857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.164789+0000","last_change":"2024-09-21T07:34:08.164618+0000","last_active":"2024-09-21T07:34:09.164789+0000","last_peered":"2024-09-21T07:34:09.164789+0000","last_clean":"2024-09-21T07:34:09.164789+0000","last_became_active":"2024-09-21T07:34:08.163696+0000","last_became_peered":"2024-09-21T07:34:08.163696+0000","last_unstale":"2024-09-21T07:34:09.164789+0000","last_undegraded":"2024-09-21T07:34:09.164789+0000","last_fullsized":"2024-09-21T07:34:09.164789+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:08:41.286205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.181440+0000","last_change":"2024-09-21T07:34:08.163763+0000","last_active":"2024-09-21T07:34:09.181440+0000","last_peered":"2024-09-21T07:34:09.181440+0000","last_clean":"2024-09-21T07:34:09.181440+0000","last_became_active":"2024-09-21T07:34:08.163455+0000","last_became_peered":"2024-09-21T07:34:08.163455+0000","last_unstale":"2024-09-21T07:34:09.181440+0000","last_undegraded":"2024-09-21T07:34:09.181440+0000","last_fullsized":"2024-09-21T07:34:09.181440+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:39.114639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496941+0000","last_change":"2024-09-21T07:34:08.158447+0000","last_active":"2024-09-21T07:34:09.496941+0000","last_peered":"2024-09-21T07:34:09.496941+0000","last_clean":"2024-09-21T07:34:09.496941+0000","last_became_active":"2024-09-21T07:34:08.158138+0000","last_became_peered":"2024-09-21T07:34:08.158138+0000","last_unstale":"2024-09-21T07:34:09.496941+0000","last_undegraded":"2024-09-21T07:34:09.496941+0000","last_fullsized":"2024-09-21T07:34:09.496941+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:05.155134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.181311+0000","last_change":"2024-09-21T07:34:08.172580+0000","last_active":"2024-09-21T07:34:09.181311+0000","last_peered":"2024-09-21T07:34:09.181311+0000","last_clean":"2024-09-21T07:34:09.181311+0000","last_became_active":"2024-09-21T07:34:08.172294+0000","last_became_peered":"2024-09-21T07:34:08.172294+0000","last_unstale":"2024-09-21T07:34:09.181311+0000","last_undegraded":"2024-09-21T07:34:09.181311+0000","last_fullsized":"2024-09-21T07:34:09.181311+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:21.245892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.165259+0000","last_change":"2024-09-21T07:34:08.169640+0000","last_active":"2024-09-21T07:34:09.165259+0000","last_peered":"2024-09-21T07:34:09.165259+0000","last_clean":"2024-09-21T07:34:09.165259+0000","last_became_active":"2024-09-21T07:34:08.169283+0000","last_became_peered":"2024-09-21T07:34:08.169283+0000","last_unstale":"2024-09-21T07:34:09.165259+0000","last_undegraded":"2024-09-21T07:34:09.165259+0000","last_fullsized":"2024-09-21T07:34:09.165259+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:07.122419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497087+0000","last_change":"2024-09-21T07:34:08.153475+0000","last_active":"2024-09-21T07:34:09.497087+0000","last_peered":"2024-09-21T07:34:09.497087+0000","last_clean":"2024-09-21T07:34:09.497087+0000","last_became_active":"2024-09-21T07:34:08.152735+0000","last_became_peered":"2024-09-21T07:34:08.152735+0000","last_unstale":"2024-09-21T07:34:09.497087+0000","last_undegraded":"2024-09-21T07:34:09.497087+0000","last_fullsized":"2024-09-21T07:34:09.497087+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:42.395864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.329350+0000","last_change":"2024-09-21T07:34:08.175295+0000","last_active":"2024-09-21T07:34:09.329350+0000","last_peered":"2024-09-21T07:34:09.329350+0000","last_clean":"2024-09-21T07:34:09.329350+0000","last_became_active":"2024-09-21T07:34:08.175002+0000","last_became_peered":"2024-09-21T07:34:08.175002+0000","last_unstale":"2024-09-21T07:34:09.329350+0000","last_undegraded":"2024-09-21T07:34:09.329350+0000","last_fullsized":"2024-09-21T07:34:09.329350+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:40.829187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496269+0000","last_change":"2024-09-21T07:34:08.163793+0000","last_active":"2024-09-21T07:34:09.496269+0000","last_peered":"2024-09-21T07:34:09.496269+0000","last_clean":"2024-09-21T07:34:09.496269+0000","last_became_active":"2024-09-21T07:34:08.163391+0000","last_became_peered":"2024-09-21T07:34:08.163391+0000","last_unstale":"2024-09-21T07:34:09.496269+0000","last_undegraded":"2024-09-21T07:34:09.496269+0000","last_fullsized":"2024-09-21T07:34:09.496269+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:55.152456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.329080+0000","last_change":"2024-09-21T07:34:08.173785+0000","last_active":"2024-09-21T07:34:09.329080+0000","last_peered":"2024-09-21T07:34:09.329080+0000","last_clean":"2024-09-21T07:34:09.329080+0000","last_became_active":"2024-09-21T07:34:08.173332+0000","last_became_peered":"2024-09-21T07:34:08.173332+0000","last_unstale":"2024-09-21T07:34:09.329080+0000","last_undegraded":"2024-09-21T07:34:09.329080+0000","last_fullsized":"2024-09-21T07:34:09.329080+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:13.730449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.169412+0000","last_change":"2024-09-21T07:34:08.166699+0000","last_active":"2024-09-21T07:34:09.169412+0000","last_peered":"2024-09-21T07:34:09.169412+0000","last_clean":"2024-09-21T07:34:09.169412+0000","last_became_active":"2024-09-21T07:34:08.161047+0000","last_became_peered":"2024-09-21T07:34:08.161047+0000","last_unstale":"2024-09-21T07:34:09.169412+0000","last_undegraded":"2024-09-21T07:34:09.169412+0000","last_fullsized":"2024-09-21T07:34:09.169412+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:34.740594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":19,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497099+0000","last_change":"2024-09-21T07:34:07.155982+0000","last_active":"2024-09-21T07:34:09.497099+0000","last_peered":"2024-09-21T07:34:09.497099+0000","last_clean":"2024-09-21T07:34:09.497099+0000","last_became_active":"2024-09-21T07:34:07.155500+0000","last_became_peered":"2024-09-21T07:34:07.155500+0000","last_unstale":"2024-09-21T07:34:09.497099+0000","last_undegraded":"2024-09-21T07:34:09.497099+0000","last_fullsized":"2024-09-21T07:34:09.497099+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:26.882263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497060+0000","last_change":"2024-09-21T07:34:08.178748+0000","last_active":"2024-09-21T07:34:09.497060+0000","last_peered":"2024-09-21T07:34:09.497060+0000","last_clean":"2024-09-21T07:34:09.497060+0000","last_became_active":"2024-09-21T07:34:08.177806+0000","last_became_peered":"2024-09-21T07:34:08.177806+0000","last_unstale":"2024-09-21T07:34:09.497060+0000","last_undegraded":"2024-09-21T07:34:09.497060+0000","last_fullsized":"2024-09-21T07:34:09.497060+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:47.458768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.169477+0000","last_change":"2024-09-21T07:34:07.148229+0000","last_active":"2024-09-21T07:34:09.169477+0000","last_peered":"2024-09-21T07:34:09.169477+0000","last_clean":"2024-09-21T07:34:09.169477+0000","last_became_active":"2024-09-21T07:34:07.147920+0000","last_became_peered":"2024-09-21T07:34:07.147920+0000","last_unstale":"2024-09-21T07:34:09.169477+0000","last_undegraded":"2024-09-21T07:34:09.169477+0000","last_fullsized":"2024-09-21T07:34:09.169477+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:09.410183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497703+0000","last_change":"2024-09-21T07:34:08.162272+0000","last_active":"2024-09-21T07:34:09.497703+0000","last_peered":"2024-09-21T07:34:09.497703+0000","last_clean":"2024-09-21T07:34:09.497703+0000","last_became_active":"2024-09-21T07:34:08.162031+0000","last_became_peered":"2024-09-21T07:34:08.162031+0000","last_unstale":"2024-09-21T07:34:09.497703+0000","last_undegraded":"2024-09-21T07:34:09.497703+0000","last_fullsized":"2024-09-21T07:34:09.497703+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:48.649163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496548+0000","last_change":"2024-09-21T07:34:07.162816+0000","last_active":"2024-09-21T07:34:09.496548+0000","last_peered":"2024-09-21T07:34:09.496548+0000","last_clean":"2024-09-21T07:34:09.496548+0000","last_became_active":"2024-09-21T07:34:07.162478+0000","last_became_peered":"2024-09-21T07:34:07.162478+0000","last_unstale":"2024-09-21T07:34:09.496548+0000","last_undegraded":"2024-09-21T07:34:09.496548+0000","last_fullsized":"2024-09-21T07:34:09.496548+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:18:03.659966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.181339+0000","last_change":"2024-09-21T07:34:08.172648+0000","last_active":"2024-09-21T07:34:09.181339+0000","last_peered":"2024-09-21T07:34:09.181339+0000","last_clean":"2024-09-21T07:34:09.181339+0000","last_became_active":"2024-09-21T07:34:08.172311+0000","last_became_peered":"2024-09-21T07:34:08.172311+0000","last_unstale":"2024-09-21T07:34:09.181339+0000","last_undegraded":"2024-09-21T07:34:09.181339+0000","last_fullsized":"2024-09-21T07:34:09.181339+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:25.699460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.163416+0000","last_change":"2024-09-21T07:34:07.151195+0000","last_active":"2024-09-21T07:34:09.163416+0000","last_peered":"2024-09-21T07:34:09.163416+0000","last_clean":"2024-09-21T07:34:09.163416+0000","last_became_active":"2024-09-21T07:34:07.150871+0000","last_became_peered":"2024-09-21T07:34:07.150871+0000","last_unstale":"2024-09-21T07:34:09.163416+0000","last_undegraded":"2024-09-21T07:34:09.163416+0000","last_fullsized":"2024-09-21T07:34:09.163416+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:27:38.559677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.329459+0000","last_change":"2024-09-21T07:34:08.174424+0000","last_active":"2024-09-21T07:34:09.329459+0000","last_peered":"2024-09-21T07:34:09.329459+0000","last_clean":"2024-09-21T07:34:09.329459+0000","last_became_active":"2024-09-21T07:34:08.174028+0000","last_became_peered":"2024-09-21T07:34:08.174028+0000","last_unstale":"2024-09-21T07:34:09.329459+0000","last_undegraded":"2024-09-21T07:34:09.329459+0000","last_fullsized":"2024-09-21T07:34:09.329459+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:31.524701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497041+0000","last_change":"2024-09-21T07:34:07.154783+0000","last_active":"2024-09-21T07:34:09.497041+0000","last_peered":"2024-09-21T07:34:09.497041+0000","last_clean":"2024-09-21T07:34:09.497041+0000","last_became_active":"2024-09-21T07:34:07.154378+0000","last_became_peered":"2024-09-21T07:34:07.154378+0000","last_unstale":"2024-09-21T07:34:09.497041+0000","last_undegraded":"2024-09-21T07:34:09.497041+0000","last_fullsized":"2024-09-21T07:34:09.497041+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:16.041455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.164907+0000","last_change":"2024-09-21T07:34:08.166752+0000","last_active":"2024-09-21T07:34:09.164907+0000","last_peered":"2024-09-21T07:34:09.164907+0000","last_clean":"2024-09-21T07:34:09.164907+0000","last_became_active":"2024-09-21T07:34:08.166477+0000","last_became_peered":"2024-09-21T07:34:08.166477+0000","last_unstale":"2024-09-21T07:34:09.164907+0000","last_undegraded":"2024-09-21T07:34:09.164907+0000","last_fullsized":"2024-09-21T07:34:09.164907+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:25.699460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496886+0000","last_change":"2024-09-21T07:34:07.153288+0000","last_active":"2024-09-21T07:34:09.496886+0000","last_peered":"2024-09-21T07:34:09.496886+0000","last_clean":"2024-09-21T07:34:09.496886+0000","last_became_active":"2024-09-21T07:34:07.152949+0000","last_became_peered":"2024-09-21T07:34:07.152949+0000","last_unstale":"2024-09-21T07:34:09.496886+0000","last_undegraded":"2024-09-21T07:34:09.496886+0000","last_fullsized":"2024-09-21T07:34:09.496886+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:54:43.804810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496478+0000","last_change":"2024-09-21T07:34:08.166316+0000","last_active":"2024-09-21T07:34:09.496478+0000","last_peered":"2024-09-21T07:34:09.496478+0000","last_clean":"2024-09-21T07:34:09.496478+0000","last_became_active":"2024-09-21T07:34:08.165006+0000","last_became_peered":"2024-09-21T07:34:08.165006+0000","last_unstale":"2024-09-21T07:34:09.496478+0000","last_undegraded":"2024-09-21T07:34:09.496478+0000","last_fullsized":"2024-09-21T07:34:09.496478+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:27.528708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496037+0000","last_change":"2024-09-21T07:34:07.141466+0000","last_active":"2024-09-21T07:34:09.496037+0000","last_peered":"2024-09-21T07:34:09.496037+0000","last_clean":"2024-09-21T07:34:09.496037+0000","last_became_active":"2024-09-21T07:34:07.141003+0000","last_became_peered":"2024-09-21T07:34:07.141003+0000","last_unstale":"2024-09-21T07:34:09.496037+0000","last_undegraded":"2024-09-21T07:34:09.496037+0000","last_fullsized":"2024-09-21T07:34:09.496037+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:52:33.957055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.180647+0000","last_change":"2024-09-21T07:34:08.163550+0000","last_active":"2024-09-21T07:34:09.180647+0000","last_peered":"2024-09-21T07:34:09.180647+0000","last_clean":"2024-09-21T07:34:09.180647+0000","last_became_active":"2024-09-21T07:34:08.163236+0000","last_became_peered":"2024-09-21T07:34:08.163236+0000","last_unstale":"2024-09-21T07:34:09.180647+0000","last_undegraded":"2024-09-21T07:34:09.180647+0000","last_fullsized":"2024-09-21T07:34:09.180647+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:56.729284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496450+0000","last_change":"2024-09-21T07:34:07.142126+0000","last_active":"2024-09-21T07:34:09.496450+0000","last_peered":"2024-09-21T07:34:09.496450+0000","last_clean":"2024-09-21T07:34:09.496450+0000","last_became_active":"2024-09-21T07:34:07.141713+0000","last_became_peered":"2024-09-21T07:34:07.141713+0000","last_unstale":"2024-09-21T07:34:09.496450+0000","last_undegraded":"2024-09-21T07:34:09.496450+0000","last_fullsized":"2024-09-21T07:34:09.496450+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:58.789677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.165516+0000","last_change":"2024-09-21T07:34:08.159275+0000","last_active":"2024-09-21T07:34:09.165516+0000","last_peered":"2024-09-21T07:34:09.165516+0000","last_clean":"2024-09-21T07:34:09.165516+0000","last_became_active":"2024-09-21T07:34:08.159005+0000","last_became_peered":"2024-09-21T07:34:08.159005+0000","last_unstale":"2024-09-21T07:34:09.165516+0000","last_undegraded":"2024-09-21T07:34:09.165516+0000","last_fullsized":"2024-09-21T07:34:09.165516+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:50.102777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496527+0000","last_change":"2024-09-21T07:34:07.154435+0000","last_active":"2024-09-21T07:34:09.496527+0000","last_peered":"2024-09-21T07:34:09.496527+0000","last_clean":"2024-09-21T07:34:09.496527+0000","last_became_active":"2024-09-21T07:34:07.154127+0000","last_became_peered":"2024-09-21T07:34:07.154127+0000","last_unstale":"2024-09-21T07:34:09.496527+0000","last_undegraded":"2024-09-21T07:34:09.496527+0000","last_fullsized":"2024-09-21T07:34:09.496527+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:11:21.678594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496615+0000","last_change":"2024-09-21T07:34:08.170301+0000","last_active":"2024-09-21T07:34:09.496615+0000","last_peered":"2024-09-21T07:34:09.496615+0000","last_clean":"2024-09-21T07:34:09.496615+0000","last_became_active":"2024-09-21T07:34:08.169925+0000","last_became_peered":"2024-09-21T07:34:08.169925+0000","last_unstale":"2024-09-21T07:34:09.496615+0000","last_undegraded":"2024-09-21T07:34:09.496615+0000","last_fullsized":"2024-09-21T07:34:09.496615+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:31.073725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.169543+0000","last_change":"2024-09-21T07:34:07.152298+0000","last_active":"2024-09-21T07:34:09.169543+0000","last_peered":"2024-09-21T07:34:09.169543+0000","last_clean":"2024-09-21T07:34:09.169543+0000","last_became_active":"2024-09-21T07:34:07.151921+0000","last_became_peered":"2024-09-21T07:34:07.151921+0000","last_unstale":"2024-09-21T07:34:09.169543+0000","last_undegraded":"2024-09-21T07:34:09.169543+0000","last_fullsized":"2024-09-21T07:34:09.169543+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:52.169625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496907+0000","last_change":"2024-09-21T07:34:08.160957+0000","last_active":"2024-09-21T07:34:09.496907+0000","last_peered":"2024-09-21T07:34:09.496907+0000","last_clean":"2024-09-21T07:34:09.496907+0000","last_became_active":"2024-09-21T07:34:08.160601+0000","last_became_peered":"2024-09-21T07:34:08.160601+0000","last_unstale":"2024-09-21T07:34:09.496907+0000","last_undegraded":"2024-09-21T07:34:09.496907+0000","last_fullsized":"2024-09-21T07:34:09.496907+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:19:16.261092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.165514+0000","last_change":"2024-09-21T07:34:07.153099+0000","last_active":"2024-09-21T07:34:09.165514+0000","last_peered":"2024-09-21T07:34:09.165514+0000","last_clean":"2024-09-21T07:34:09.165514+0000","last_became_active":"2024-09-21T07:34:07.152790+0000","last_became_peered":"2024-09-21T07:34:07.152790+0000","last_unstale":"2024-09-21T07:34:09.165514+0000","last_undegraded":"2024-09-21T07:34:09.165514+0000","last_fullsized":"2024-09-21T07:34:09.165514+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:51.990530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.165280+0000","last_change":"2024-09-21T07:34:08.165826+0000","last_active":"2024-09-21T07:34:09.165280+0000","last_peered":"2024-09-21T07:34:09.165280+0000","last_clean":"2024-09-21T07:34:09.165280+0000","last_became_active":"2024-09-21T07:34:08.160912+0000","last_became_peered":"2024-09-21T07:34:08.160912+0000","last_unstale":"2024-09-21T07:34:09.165280+0000","last_undegraded":"2024-09-21T07:34:09.165280+0000","last_fullsized":"2024-09-21T07:34:09.165280+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:27.710069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":21,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496721+0000","last_change":"2024-09-21T07:34:07.162976+0000","last_active":"2024-09-21T07:34:09.496721+0000","last_peered":"2024-09-21T07:34:09.496721+0000","last_clean":"2024-09-21T07:34:09.496721+0000","last_became_active":"2024-09-21T07:34:07.162698+0000","last_became_peered":"2024-09-21T07:34:07.162698+0000","last_unstale":"2024-09-21T07:34:09.496721+0000","last_undegraded":"2024-09-21T07:34:09.496721+0000","last_fullsized":"2024-09-21T07:34:09.496721+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:01:04.212051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.180755+0000","last_change":"2024-09-21T07:34:08.168839+0000","last_active":"2024-09-21T07:34:09.180755+0000","last_peered":"2024-09-21T07:34:09.180755+0000","last_clean":"2024-09-21T07:34:09.180755+0000","last_became_active":"2024-09-21T07:34:08.168183+0000","last_became_peered":"2024-09-21T07:34:08.168183+0000","last_unstale":"2024-09-21T07:34:09.180755+0000","last_undegraded":"2024-09-21T07:34:09.180755+0000","last_fullsized":"2024-09-21T07:34:09.180755+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:28.055160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497188+0000","last_change":"2024-09-21T07:34:07.150104+0000","last_active":"2024-09-21T07:34:09.497188+0000","last_peered":"2024-09-21T07:34:09.497188+0000","last_clean":"2024-09-21T07:34:09.497188+0000","last_became_active":"2024-09-21T07:34:07.149757+0000","last_became_peered":"2024-09-21T07:34:07.149757+0000","last_unstale":"2024-09-21T07:34:09.497188+0000","last_undegraded":"2024-09-21T07:34:09.497188+0000","last_fullsized":"2024-09-21T07:34:09.497188+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:17.255537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496697+0000","last_change":"2024-09-21T07:34:08.166843+0000","last_active":"2024-09-21T07:34:09.496697+0000","last_peered":"2024-09-21T07:34:09.496697+0000","last_clean":"2024-09-21T07:34:09.496697+0000","last_became_active":"2024-09-21T07:34:08.166516+0000","last_became_peered":"2024-09-21T07:34:08.166516+0000","last_unstale":"2024-09-21T07:34:09.496697+0000","last_undegraded":"2024-09-21T07:34:09.496697+0000","last_fullsized":"2024-09-21T07:34:09.496697+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:39.634059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.180719+0000","last_change":"2024-09-21T07:34:07.160820+0000","last_active":"2024-09-21T07:34:09.180719+0000","last_peered":"2024-09-21T07:34:09.180719+0000","last_clean":"2024-09-21T07:34:09.180719+0000","last_became_active":"2024-09-21T07:34:07.160364+0000","last_became_peered":"2024-09-21T07:34:07.160364+0000","last_unstale":"2024-09-21T07:34:09.180719+0000","last_undegraded":"2024-09-21T07:34:09.180719+0000","last_fullsized":"2024-09-21T07:34:09.180719+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:27.230155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.329827+0000","last_change":"2024-09-21T07:34:08.173868+0000","last_active":"2024-09-21T07:34:09.329827+0000","last_peered":"2024-09-21T07:34:09.329827+0000","last_clean":"2024-09-21T07:34:09.329827+0000","last_became_active":"2024-09-21T07:34:08.173552+0000","last_became_peered":"2024-09-21T07:34:08.173552+0000","last_unstale":"2024-09-21T07:34:09.329827+0000","last_undegraded":"2024-09-21T07:34:09.329827+0000","last_fullsized":"2024-09-21T07:34:09.329827+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:16.612188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496142+0000","last_change":"2024-09-21T07:34:07.153626+0000","last_active":"2024-09-21T07:34:09.496142+0000","last_peered":"2024-09-21T07:34:09.496142+0000","last_clean":"2024-09-21T07:34:09.496142+0000","last_became_active":"2024-09-21T07:34:07.153334+0000","last_became_peered":"2024-09-21T07:34:07.153334+0000","last_unstale":"2024-09-21T07:34:09.496142+0000","last_undegraded":"2024-09-21T07:34:09.496142+0000","last_fullsized":"2024-09-21T07:34:09.496142+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:48.214805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496501+0000","last_change":"2024-09-21T07:34:08.175374+0000","last_active":"2024-09-21T07:34:09.496501+0000","last_peered":"2024-09-21T07:34:09.496501+0000","last_clean":"2024-09-21T07:34:09.496501+0000","last_became_active":"2024-09-21T07:34:08.175065+0000","last_became_peered":"2024-09-21T07:34:08.175065+0000","last_unstale":"2024-09-21T07:34:09.496501+0000","last_undegraded":"2024-09-21T07:34:09.496501+0000","last_fullsized":"2024-09-21T07:34:09.496501+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:52.997974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496343+0000","last_change":"2024-09-21T07:34:07.161149+0000","last_active":"2024-09-21T07:34:09.496343+0000","last_peered":"2024-09-21T07:34:09.496343+0000","last_clean":"2024-09-21T07:34:09.496343+0000","last_became_active":"2024-09-21T07:34:07.160882+0000","last_became_peered":"2024-09-21T07:34:07.160882+0000","last_unstale":"2024-09-21T07:34:09.496343+0000","last_undegraded":"2024-09-21T07:34:09.496343+0000","last_fullsized":"2024-09-21T07:34:09.496343+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:55.427089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497420+0000","last_change":"2024-09-21T07:34:08.167375+0000","last_active":"2024-09-21T07:34:09.497420+0000","last_peered":"2024-09-21T07:34:09.497420+0000","last_clean":"2024-09-21T07:34:09.497420+0000","last_became_active":"2024-09-21T07:34:08.165919+0000","last_became_peered":"2024-09-21T07:34:08.165919+0000","last_unstale":"2024-09-21T07:34:09.497420+0000","last_undegraded":"2024-09-21T07:34:09.497420+0000","last_fullsized":"2024-09-21T07:34:09.497420+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:30.388748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.181509+0000","last_change":"2024-09-21T07:34:07.154981+0000","last_active":"2024-09-21T07:34:09.181509+0000","last_peered":"2024-09-21T07:34:09.181509+0000","last_clean":"2024-09-21T07:34:09.181509+0000","last_became_active":"2024-09-21T07:34:07.154706+0000","last_became_peered":"2024-09-21T07:34:07.154706+0000","last_unstale":"2024-09-21T07:34:09.181509+0000","last_undegraded":"2024-09-21T07:34:09.181509+0000","last_fullsized":"2024-09-21T07:34:09.181509+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:50.527238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.165808+0000","last_change":"2024-09-21T07:34:08.171328+0000","last_active":"2024-09-21T07:34:09.165808+0000","last_peered":"2024-09-21T07:34:09.165808+0000","last_clean":"2024-09-21T07:34:09.165808+0000","last_became_active":"2024-09-21T07:34:08.170351+0000","last_became_peered":"2024-09-21T07:34:08.170351+0000","last_unstale":"2024-09-21T07:34:09.165808+0000","last_undegraded":"2024-09-21T07:34:09.165808+0000","last_fullsized":"2024-09-21T07:34:09.165808+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:52.642170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497519+0000","last_change":"2024-09-21T07:34:07.149293+0000","last_active":"2024-09-21T07:34:09.497519+0000","last_peered":"2024-09-21T07:34:09.497519+0000","last_clean":"2024-09-21T07:34:09.497519+0000","last_became_active":"2024-09-21T07:34:07.148938+0000","last_became_peered":"2024-09-21T07:34:07.148938+0000","last_unstale":"2024-09-21T07:34:09.497519+0000","last_undegraded":"2024-09-21T07:34:09.497519+0000","last_fullsized":"2024-09-21T07:34:09.497519+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:59.434905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.180907+0000","last_change":"2024-09-21T07:34:08.177348+0000","last_active":"2024-09-21T07:34:09.180907+0000","last_peered":"2024-09-21T07:34:09.180907+0000","last_clean":"2024-09-21T07:34:09.180907+0000","last_became_active":"2024-09-21T07:34:08.174645+0000","last_became_peered":"2024-09-21T07:34:08.174645+0000","last_unstale":"2024-09-21T07:34:09.180907+0000","last_undegraded":"2024-09-21T07:34:09.180907+0000","last_fullsized":"2024-09-21T07:34:09.180907+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:06.515935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497479+0000","last_change":"2024-09-21T07:34:07.148308+0000","last_active":"2024-09-21T07:34:09.497479+0000","last_peered":"2024-09-21T07:34:09.497479+0000","last_clean":"2024-09-21T07:34:09.497479+0000","last_became_active":"2024-09-21T07:34:07.148001+0000","last_became_peered":"2024-09-21T07:34:07.148001+0000","last_unstale":"2024-09-21T07:34:09.497479+0000","last_undegraded":"2024-09-21T07:34:09.497479+0000","last_fullsized":"2024-09-21T07:34:09.497479+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:54:32.939044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.163433+0000","last_change":"2024-09-21T07:34:08.162638+0000","last_active":"2024-09-21T07:34:09.163433+0000","last_peered":"2024-09-21T07:34:09.163433+0000","last_clean":"2024-09-21T07:34:09.163433+0000","last_became_active":"2024-09-21T07:34:08.162237+0000","last_became_peered":"2024-09-21T07:34:08.162237+0000","last_unstale":"2024-09-21T07:34:09.163433+0000","last_undegraded":"2024-09-21T07:34:09.163433+0000","last_fullsized":"2024-09-21T07:34:09.163433+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:10.417627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497624+0000","last_change":"2024-09-21T07:34:07.152996+0000","last_active":"2024-09-21T07:34:09.497624+0000","last_peered":"2024-09-21T07:34:09.497624+0000","last_clean":"2024-09-21T07:34:09.497624+0000","last_became_active":"2024-09-21T07:34:07.152627+0000","last_became_peered":"2024-09-21T07:34:07.152627+0000","last_unstale":"2024-09-21T07:34:09.497624+0000","last_undegraded":"2024-09-21T07:34:09.497624+0000","last_fullsized":"2024-09-21T07:34:09.497624+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:24:42.920690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.163318+0000","last_change":"2024-09-21T07:34:08.173273+0000","last_active":"2024-09-21T07:34:09.163318+0000","last_peered":"2024-09-21T07:34:09.163318+0000","last_clean":"2024-09-21T07:34:09.163318+0000","last_became_active":"2024-09-21T07:34:08.172952+0000","last_became_peered":"2024-09-21T07:34:08.172952+0000","last_unstale":"2024-09-21T07:34:09.163318+0000","last_undegraded":"2024-09-21T07:34:09.163318+0000","last_fullsized":"2024-09-21T07:34:09.163318+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:47:39.799134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.169104+0000","last_change":"2024-09-21T07:34:07.151886+0000","last_active":"2024-09-21T07:34:09.169104+0000","last_peered":"2024-09-21T07:34:09.169104+0000","last_clean":"2024-09-21T07:34:09.169104+0000","last_became_active":"2024-09-21T07:34:07.150347+0000","last_became_peered":"2024-09-21T07:34:07.150347+0000","last_unstale":"2024-09-21T07:34:09.169104+0000","last_undegraded":"2024-09-21T07:34:09.169104+0000","last_fullsized":"2024-09-21T07:34:09.169104+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:27.230155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.163623+0000","last_change":"2024-09-21T07:34:08.174497+0000","last_active":"2024-09-21T07:34:09.163623+0000","last_peered":"2024-09-21T07:34:09.163623+0000","last_clean":"2024-09-21T07:34:09.163623+0000","last_became_active":"2024-09-21T07:34:08.173554+0000","last_became_peered":"2024-09-21T07:34:08.173554+0000","last_unstale":"2024-09-21T07:34:09.163623+0000","last_undegraded":"2024-09-21T07:34:09.163623+0000","last_fullsized":"2024-09-21T07:34:09.163623+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:18:16.768539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496838+0000","last_change":"2024-09-21T07:34:07.161337+0000","last_active":"2024-09-21T07:34:09.496838+0000","last_peered":"2024-09-21T07:34:09.496838+0000","last_clean":"2024-09-21T07:34:09.496838+0000","last_became_active":"2024-09-21T07:34:07.161055+0000","last_became_peered":"2024-09-21T07:34:07.161055+0000","last_unstale":"2024-09-21T07:34:09.496838+0000","last_undegraded":"2024-09-21T07:34:09.496838+0000","last_fullsized":"2024-09-21T07:34:09.496838+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:09.410183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.163185+0000","last_change":"2024-09-21T07:34:08.170546+0000","last_active":"2024-09-21T07:34:09.163185+0000","last_peered":"2024-09-21T07:34:09.163185+0000","last_clean":"2024-09-21T07:34:09.163185+0000","last_became_active":"2024-09-21T07:34:08.170249+0000","last_became_peered":"2024-09-21T07:34:08.170249+0000","last_unstale":"2024-09-21T07:34:09.163185+0000","last_undegraded":"2024-09-21T07:34:09.163185+0000","last_fullsized":"2024-09-21T07:34:09.163185+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:54.043603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497320+0000","last_change":"2024-09-21T07:34:07.148155+0000","last_active":"2024-09-21T07:34:09.497320+0000","last_peered":"2024-09-21T07:34:09.497320+0000","last_clean":"2024-09-21T07:34:09.497320+0000","last_became_active":"2024-09-21T07:34:07.147855+0000","last_became_peered":"2024-09-21T07:34:07.147855+0000","last_unstale":"2024-09-21T07:34:09.497320+0000","last_undegraded":"2024-09-21T07:34:09.497320+0000","last_fullsized":"2024-09-21T07:34:09.497320+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:41:10.667777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.329218+0000","last_change":"2024-09-21T07:34:08.165763+0000","last_active":"2024-09-21T07:34:09.329218+0000","last_peered":"2024-09-21T07:34:09.329218+0000","last_clean":"2024-09-21T07:34:09.329218+0000","last_became_active":"2024-09-21T07:34:08.163699+0000","last_became_peered":"2024-09-21T07:34:08.163699+0000","last_unstale":"2024-09-21T07:34:09.329218+0000","last_undegraded":"2024-09-21T07:34:09.329218+0000","last_fullsized":"2024-09-21T07:34:09.329218+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:14:22.162876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.165169+0000","last_change":"2024-09-21T07:34:07.156707+0000","last_active":"2024-09-21T07:34:09.165169+0000","last_peered":"2024-09-21T07:34:09.165169+0000","last_clean":"2024-09-21T07:34:09.165169+0000","last_became_active":"2024-09-21T07:34:07.156326+0000","last_became_peered":"2024-09-21T07:34:07.156326+0000","last_unstale":"2024-09-21T07:34:09.165169+0000","last_undegraded":"2024-09-21T07:34:09.165169+0000","last_fullsized":"2024-09-21T07:34:09.165169+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:54.145012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.163369+0000","last_change":"2024-09-21T07:34:08.166587+0000","last_active":"2024-09-21T07:34:09.163369+0000","last_peered":"2024-09-21T07:34:09.163369+0000","last_clean":"2024-09-21T07:34:09.163369+0000","last_became_active":"2024-09-21T07:34:08.166262+0000","last_became_peered":"2024-09-21T07:34:08.166262+0000","last_unstale":"2024-09-21T07:34:09.163369+0000","last_undegraded":"2024-09-21T07:34:09.163369+0000","last_fullsized":"2024-09-21T07:34:09.163369+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:27.889707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.181531+0000","last_change":"2024-09-21T07:34:07.153252+0000","last_active":"2024-09-21T07:34:09.181531+0000","last_peered":"2024-09-21T07:34:09.181531+0000","last_clean":"2024-09-21T07:34:09.181531+0000","last_became_active":"2024-09-21T07:34:07.152932+0000","last_became_peered":"2024-09-21T07:34:07.152932+0000","last_unstale":"2024-09-21T07:34:09.181531+0000","last_undegraded":"2024-09-21T07:34:09.181531+0000","last_fullsized":"2024-09-21T07:34:09.181531+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:24:14.371118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.165881+0000","last_change":"2024-09-21T07:34:08.171574+0000","last_active":"2024-09-21T07:34:09.165881+0000","last_peered":"2024-09-21T07:34:09.165881+0000","last_clean":"2024-09-21T07:34:09.165881+0000","last_became_active":"2024-09-21T07:34:08.170992+0000","last_became_peered":"2024-09-21T07:34:08.170992+0000","last_unstale":"2024-09-21T07:34:09.165881+0000","last_undegraded":"2024-09-21T07:34:09.165881+0000","last_fullsized":"2024-09-21T07:34:09.165881+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:31.524701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.169650+0000","last_change":"2024-09-21T07:34:07.161285+0000","last_active":"2024-09-21T07:34:09.169650+0000","last_peered":"2024-09-21T07:34:09.169650+0000","last_clean":"2024-09-21T07:34:09.169650+0000","last_became_active":"2024-09-21T07:34:07.160925+0000","last_became_peered":"2024-09-21T07:34:07.160925+0000","last_unstale":"2024-09-21T07:34:09.169650+0000","last_undegraded":"2024-09-21T07:34:09.169650+0000","last_fullsized":"2024-09-21T07:34:09.169650+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:54.784501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.169133+0000","last_change":"2024-09-21T07:34:08.160436+0000","last_active":"2024-09-21T07:34:09.169133+0000","last_peered":"2024-09-21T07:34:09.169133+0000","last_clean":"2024-09-21T07:34:09.169133+0000","last_became_active":"2024-09-21T07:34:08.159040+0000","last_became_peered":"2024-09-21T07:34:08.159040+0000","last_unstale":"2024-09-21T07:34:09.169133+0000","last_undegraded":"2024-09-21T07:34:09.169133+0000","last_fullsized":"2024-09-21T07:34:09.169133+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:30.259079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.181020+0000","last_change":"2024-09-21T07:34:07.151263+0000","last_active":"2024-09-21T07:34:09.181020+0000","last_peered":"2024-09-21T07:34:09.181020+0000","last_clean":"2024-09-21T07:34:09.181020+0000","last_became_active":"2024-09-21T07:34:07.150981+0000","last_became_peered":"2024-09-21T07:34:07.150981+0000","last_unstale":"2024-09-21T07:34:09.181020+0000","last_undegraded":"2024-09-21T07:34:09.181020+0000","last_fullsized":"2024-09-21T07:34:09.181020+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:26:19.106553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496799+0000","last_change":"2024-09-21T07:34:08.167728+0000","last_active":"2024-09-21T07:34:09.496799+0000","last_peered":"2024-09-21T07:34:09.496799+0000","last_clean":"2024-09-21T07:34:09.496799+0000","last_became_active":"2024-09-21T07:34:08.167409+0000","last_became_peered":"2024-09-21T07:34:08.167409+0000","last_unstale":"2024-09-21T07:34:09.496799+0000","last_undegraded":"2024-09-21T07:34:09.496799+0000","last_fullsized":"2024-09-21T07:34:09.496799+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:55.716857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.169272+0000","last_change":"2024-09-21T07:34:07.139885+0000","last_active":"2024-09-21T07:34:09.169272+0000","last_peered":"2024-09-21T07:34:09.169272+0000","last_clean":"2024-09-21T07:34:09.169272+0000","last_became_active":"2024-09-21T07:34:07.139560+0000","last_became_peered":"2024-09-21T07:34:07.139560+0000","last_unstale":"2024-09-21T07:34:09.169272+0000","last_undegraded":"2024-09-21T07:34:09.169272+0000","last_fullsized":"2024-09-21T07:34:09.169272+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:18:15.761095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.169169+0000","last_change":"2024-09-21T07:34:08.165778+0000","last_active":"2024-09-21T07:34:09.169169+0000","last_peered":"2024-09-21T07:34:09.169169+0000","last_clean":"2024-09-21T07:34:09.169169+0000","last_became_active":"2024-09-21T07:34:08.165481+0000","last_became_peered":"2024-09-21T07:34:08.165481+0000","last_unstale":"2024-09-21T07:34:09.169169+0000","last_undegraded":"2024-09-21T07:34:09.169169+0000","last_fullsized":"2024-09-21T07:34:09.169169+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:47.458768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.180835+0000","last_change":"2024-09-21T07:34:07.152465+0000","last_active":"2024-09-21T07:34:09.180835+0000","last_peered":"2024-09-21T07:34:09.180835+0000","last_clean":"2024-09-21T07:34:09.180835+0000","last_became_active":"2024-09-21T07:34:07.152065+0000","last_became_peered":"2024-09-21T07:34:07.152065+0000","last_unstale":"2024-09-21T07:34:09.180835+0000","last_undegraded":"2024-09-21T07:34:09.180835+0000","last_fullsized":"2024-09-21T07:34:09.180835+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:24:30.050440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496179+0000","last_change":"2024-09-21T07:34:08.150846+0000","last_active":"2024-09-21T07:34:09.496179+0000","last_peered":"2024-09-21T07:34:09.496179+0000","last_clean":"2024-09-21T07:34:09.496179+0000","last_became_active":"2024-09-21T07:34:08.150534+0000","last_became_peered":"2024-09-21T07:34:08.150534+0000","last_unstale":"2024-09-21T07:34:09.496179+0000","last_undegraded":"2024-09-21T07:34:09.496179+0000","last_fullsized":"2024-09-21T07:34:09.496179+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:26:49.717420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497461+0000","last_change":"2024-09-21T07:34:07.147358+0000","last_active":"2024-09-21T07:34:09.497461+0000","last_peered":"2024-09-21T07:34:09.497461+0000","last_clean":"2024-09-21T07:34:09.497461+0000","last_became_active":"2024-09-21T07:34:07.147036+0000","last_became_peered":"2024-09-21T07:34:07.147036+0000","last_unstale":"2024-09-21T07:34:09.497461+0000","last_undegraded":"2024-09-21T07:34:09.497461+0000","last_fullsized":"2024-09-21T07:34:09.497461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:04:22.493944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497584+0000","last_change":"2024-09-21T07:34:08.159510+0000","last_active":"2024-09-21T07:34:09.497584+0000","last_peered":"2024-09-21T07:34:09.497584+0000","last_clean":"2024-09-21T07:34:09.497584+0000","last_became_active":"2024-09-21T07:34:08.159191+0000","last_became_peered":"2024-09-21T07:34:08.159191+0000","last_unstale":"2024-09-21T07:34:09.497584+0000","last_undegraded":"2024-09-21T07:34:09.497584+0000","last_fullsized":"2024-09-21T07:34:09.497584+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:27.319525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496085+0000","last_change":"2024-09-21T07:34:07.155987+0000","last_active":"2024-09-21T07:34:09.496085+0000","last_peered":"2024-09-21T07:34:09.496085+0000","last_clean":"2024-09-21T07:34:09.496085+0000","last_became_active":"2024-09-21T07:34:07.155735+0000","last_became_peered":"2024-09-21T07:34:07.155735+0000","last_unstale":"2024-09-21T07:34:09.496085+0000","last_undegraded":"2024-09-21T07:34:09.496085+0000","last_fullsized":"2024-09-21T07:34:09.496085+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:23:21.954791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.329467+0000","last_change":"2024-09-21T07:34:08.167002+0000","last_active":"2024-09-21T07:34:09.329467+0000","last_peered":"2024-09-21T07:34:09.329467+0000","last_clean":"2024-09-21T07:34:09.329467+0000","last_became_active":"2024-09-21T07:34:08.166679+0000","last_became_peered":"2024-09-21T07:34:08.166679+0000","last_unstale":"2024-09-21T07:34:09.329467+0000","last_undegraded":"2024-09-21T07:34:09.329467+0000","last_fullsized":"2024-09-21T07:34:09.329467+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:52.642170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496567+0000","last_change":"2024-09-21T07:34:07.155156+0000","last_active":"2024-09-21T07:34:09.496567+0000","last_peered":"2024-09-21T07:34:09.496567+0000","last_clean":"2024-09-21T07:34:09.496567+0000","last_became_active":"2024-09-21T07:34:07.154829+0000","last_became_peered":"2024-09-21T07:34:07.154829+0000","last_unstale":"2024-09-21T07:34:09.496567+0000","last_undegraded":"2024-09-21T07:34:09.496567+0000","last_fullsized":"2024-09-21T07:34:09.496567+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:59:04.464801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":135,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.329996+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:34:09.329996+0000","last_peered":"2024-09-21T07:34:09.329996+0000","last_clean":"2024-09-21T07:34:09.329996+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:34:09.329996+0000","last_undegraded":"2024-09-21T07:34:09.329996+0000","last_fullsized":"2024-09-21T07:34:09.329996+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509457,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":974848,"data_stored":739153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509459,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":385024,"data_stored":149311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509457,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":385024,"data_stored":149367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509458,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31568,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340272,"statfs":{"total":96636764160,"available":96604438528,"internally_reserved":0,"allocated":380928,"data_stored":148765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509459,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":393216,"data_stored":150329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509459,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":970752,"data_stored":739063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542162,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":385024,"data_stored":148799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542161,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":393216,"data_stored":150329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:34:17.720 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:34:17.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:17.955+0000 7f072ce00640 1 -- 172.21.15.39:0/3092721083 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0728070030 msgr2=0x7f0728074640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:17.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:17.955+0000 7f072ce00640 1 --2- 172.21.15.39:0/3092721083 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0728070030 0x7f0728074640 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f07280b4b40 tx=0x7f071402a5f0 comp rx=0 tx=0).stop 2024-09-21T07:34:17.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:17.956+0000 7f072ce00640 1 -- 172.21.15.39:0/3092721083 shutdown_connections 2024-09-21T07:34:17.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:17.956+0000 7f072ce00640 1 --2- 172.21.15.39:0/3092721083 >> v2:172.21.15.148:3300/0 conn(0x7f0728070440 0x7f0728074b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:17.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:17.956+0000 7f072ce00640 1 --2- 172.21.15.39:0/3092721083 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0728070030 0x7f0728074640 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:17.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:17.956+0000 7f072ce00640 1 -- 172.21.15.39:0/3092721083 >> 172.21.15.39:0/3092721083 conn(0x7f07280b1cb0 msgr2=0x7f07280b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:17.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:17.956+0000 7f072ce00640 1 -- 172.21.15.39:0/3092721083 shutdown_connections 2024-09-21T07:34:17.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:17.956+0000 7f072ce00640 1 -- 172.21.15.39:0/3092721083 wait complete. 2024-09-21T07:34:17.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:17.958+0000 7f072ce00640 1 Processor -- start 2024-09-21T07:34:17.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:17.958+0000 7f072ce00640 1 -- start start 2024-09-21T07:34:17.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:17.959+0000 7f072ce00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0728070030 0x7f072814c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:17.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:17.959+0000 7f072ce00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0728070440 0x7f07280cb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:17.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:17.959+0000 7f072ce00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0728073170 0x7f07281e94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:17.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:17.959+0000 7f072ce00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f07280b39f0 con 0x7f0728070440 2024-09-21T07:34:17.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:17.959+0000 7f072ce00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f07280b3870 con 0x7f0728073170 2024-09-21T07:34:17.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:17.959+0000 7f072ce00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f07280b3b70 con 0x7f0728070030 2024-09-21T07:34:17.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:17.960+0000 7f0725400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0728070440 0x7f07280cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:17.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:17.960+0000 7f0725400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0728070440 0x7f07280cb650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41474/0 (socket says 172.21.15.39:41474) 2024-09-21T07:34:17.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:17.960+0000 7f0725400640 1 -- 172.21.15.39:0/145875476 learned_addr learned my addr 172.21.15.39:0/145875476 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:17.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:17.960+0000 7f0726800640 1 --2- 172.21.15.39:0/145875476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0728073170 0x7f07281e94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:17.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:17.960+0000 7f0725e00640 1 --2- 172.21.15.39:0/145875476 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0728070030 0x7f072814c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:17.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:17.960+0000 7f0725400640 1 -- 172.21.15.39:0/145875476 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0728070030 msgr2=0x7f072814c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:17.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:17.960+0000 7f0725400640 1 --2- 172.21.15.39:0/145875476 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0728070030 0x7f072814c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:17.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:17.960+0000 7f0725400640 1 -- 172.21.15.39:0/145875476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0728073170 msgr2=0x7f07281e94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:17.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:17.960+0000 7f0725400640 1 --2- 172.21.15.39:0/145875476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0728073170 0x7f07281e94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:17.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:17.960+0000 7f0725400640 1 -- 172.21.15.39:0/145875476 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f072809c9d0 con 0x7f0728070440 2024-09-21T07:34:17.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:17.960+0000 7f0725e00640 1 --2- 172.21.15.39:0/145875476 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0728070030 0x7f072814c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:34:17.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:17.960+0000 7f0725400640 1 --2- 172.21.15.39:0/145875476 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0728070440 0x7f07280cb650 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0714034040 tx=0x7f0714003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:17.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:17.961+0000 7f071e000640 1 -- 172.21.15.39:0/145875476 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071403e030 con 0x7f0728070440 2024-09-21T07:34:17.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:17.961+0000 7f071e000640 1 -- 172.21.15.39:0/145875476 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0714003e00 con 0x7f0728070440 2024-09-21T07:34:17.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:17.961+0000 7f072ce00640 1 -- 172.21.15.39:0/145875476 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07280cc3c0 con 0x7f0728070440 2024-09-21T07:34:17.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:17.961+0000 7f071e000640 1 -- 172.21.15.39:0/145875476 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0714035040 con 0x7f0728070440 2024-09-21T07:34:17.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:17.961+0000 7f072ce00640 1 -- 172.21.15.39:0/145875476 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f072809ae90 con 0x7f0728070440 2024-09-21T07:34:17.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:17.962+0000 7f072ce00640 1 -- 172.21.15.39:0/145875476 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07280cc130 con 0x7f0728070440 2024-09-21T07:34:17.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:17.963+0000 7f071e000640 1 -- 172.21.15.39:0/145875476 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0714041030 con 0x7f0728070440 2024-09-21T07:34:17.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:17.965+0000 7f071e000640 1 --2- 172.21.15.39:0/145875476 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0708078b90 0x7f070807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:17.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:17.965+0000 7f071e000640 1 -- 172.21.15.39:0/145875476 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f07140bb530 con 0x7f0728070440 2024-09-21T07:34:17.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:17.965+0000 7f0725e00640 1 --2- 172.21.15.39:0/145875476 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0708078b90 0x7f070807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:17.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:17.966+0000 7f0725e00640 1 --2- 172.21.15.39:0/145875476 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0708078b90 0x7f070807b0a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0718000f60 tx=0x7f0718009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:17.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:17.969+0000 7f071e000640 1 -- 172.21.15.39:0/145875476 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0714083a00 con 0x7f0728070440 2024-09-21T07:34:18.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.164+0000 7f072ce00640 1 -- 172.21.15.39:0/145875476 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f07280b4090 con 0x7f0708078b90 2024-09-21T07:34:18.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.169+0000 7f071e000640 1 -- 172.21.15.39:0/145875476 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251880 (secure 0 0 0) 0x7f07280b4090 con 0x7f0708078b90 2024-09-21T07:34:18.174 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:34:18.175 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:34:18.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.173+0000 7f072ce00640 1 -- 172.21.15.39:0/145875476 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0708078b90 msgr2=0x7f070807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:18.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.173+0000 7f072ce00640 1 --2- 172.21.15.39:0/145875476 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0708078b90 0x7f070807b0a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0718000f60 tx=0x7f0718009040 comp rx=0 tx=0).stop 2024-09-21T07:34:18.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.173+0000 7f072ce00640 1 -- 172.21.15.39:0/145875476 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0728070440 msgr2=0x7f07280cb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:18.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.173+0000 7f072ce00640 1 --2- 172.21.15.39:0/145875476 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0728070440 0x7f07280cb650 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0714034040 tx=0x7f0714003910 comp rx=0 tx=0).stop 2024-09-21T07:34:18.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.173+0000 7f072ce00640 1 -- 172.21.15.39:0/145875476 shutdown_connections 2024-09-21T07:34:18.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.173+0000 7f072ce00640 1 --2- 172.21.15.39:0/145875476 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0708078b90 0x7f070807b0a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:18.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.173+0000 7f072ce00640 1 --2- 172.21.15.39:0/145875476 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0728073170 0x7f07281e94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:18.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.173+0000 7f072ce00640 1 --2- 172.21.15.39:0/145875476 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0728070440 0x7f07280cb650 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:18.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.173+0000 7f072ce00640 1 --2- 172.21.15.39:0/145875476 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0728070030 0x7f072814c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:18.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.173+0000 7f072ce00640 1 -- 172.21.15.39:0/145875476 >> 172.21.15.39:0/145875476 conn(0x7f07280b1cb0 msgr2=0x7f072815f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:18.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.174+0000 7f072ce00640 1 -- 172.21.15.39:0/145875476 shutdown_connections 2024-09-21T07:34:18.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.174+0000 7f072ce00640 1 -- 172.21.15.39:0/145875476 wait complete. 2024-09-21T07:34:18.197 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2024-09-21T07:34:17.549243+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237064,"kb_used_data":4168,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737656,"statfs":{"total":773094113280,"available":772851359744,"internally_reserved":0,"allocated":4268032,"data_stored":2375116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.405530"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496910+0000","last_change":"2024-09-21T07:34:08.172048+0000","last_active":"2024-09-21T07:34:09.496910+0000","last_peered":"2024-09-21T07:34:09.496910+0000","last_clean":"2024-09-21T07:34:09.496910+0000","last_became_active":"2024-09-21T07:34:08.170885+0000","last_became_peered":"2024-09-21T07:34:08.170885+0000","last_unstale":"2024-09-21T07:34:09.496910+0000","last_undegraded":"2024-09-21T07:34:09.496910+0000","last_fullsized":"2024-09-21T07:34:09.496910+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:30.259079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496812+0000","last_change":"2024-09-21T07:34:08.163227+0000","last_active":"2024-09-21T07:34:09.496812+0000","last_peered":"2024-09-21T07:34:09.496812+0000","last_clean":"2024-09-21T07:34:09.496812+0000","last_became_active":"2024-09-21T07:34:08.162915+0000","last_became_peered":"2024-09-21T07:34:08.162915+0000","last_unstale":"2024-09-21T07:34:09.496812+0000","last_undegraded":"2024-09-21T07:34:09.496812+0000","last_fullsized":"2024-09-21T07:34:09.496812+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:16.292830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.165484+0000","last_change":"2024-09-21T07:34:08.161784+0000","last_active":"2024-09-21T07:34:09.165484+0000","last_peered":"2024-09-21T07:34:09.165484+0000","last_clean":"2024-09-21T07:34:09.165484+0000","last_became_active":"2024-09-21T07:34:08.161467+0000","last_became_peered":"2024-09-21T07:34:08.161467+0000","last_unstale":"2024-09-21T07:34:09.165484+0000","last_undegraded":"2024-09-21T07:34:09.165484+0000","last_fullsized":"2024-09-21T07:34:09.165484+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:14:22.162876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.180849+0000","last_change":"2024-09-21T07:34:08.174981+0000","last_active":"2024-09-21T07:34:09.180849+0000","last_peered":"2024-09-21T07:34:09.180849+0000","last_clean":"2024-09-21T07:34:09.180849+0000","last_became_active":"2024-09-21T07:34:08.174163+0000","last_became_peered":"2024-09-21T07:34:08.174163+0000","last_unstale":"2024-09-21T07:34:09.180849+0000","last_undegraded":"2024-09-21T07:34:09.180849+0000","last_fullsized":"2024-09-21T07:34:09.180849+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:08.608518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.163240+0000","last_change":"2024-09-21T07:34:08.170011+0000","last_active":"2024-09-21T07:34:09.163240+0000","last_peered":"2024-09-21T07:34:09.163240+0000","last_clean":"2024-09-21T07:34:09.163240+0000","last_became_active":"2024-09-21T07:34:08.169688+0000","last_became_peered":"2024-09-21T07:34:08.169688+0000","last_unstale":"2024-09-21T07:34:09.163240+0000","last_undegraded":"2024-09-21T07:34:09.163240+0000","last_fullsized":"2024-09-21T07:34:09.163240+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:59:05.472245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496422+0000","last_change":"2024-09-21T07:34:08.161341+0000","last_active":"2024-09-21T07:34:09.496422+0000","last_peered":"2024-09-21T07:34:09.496422+0000","last_clean":"2024-09-21T07:34:09.496422+0000","last_became_active":"2024-09-21T07:34:08.161037+0000","last_became_peered":"2024-09-21T07:34:08.161037+0000","last_unstale":"2024-09-21T07:34:09.496422+0000","last_undegraded":"2024-09-21T07:34:09.496422+0000","last_fullsized":"2024-09-21T07:34:09.496422+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:52.361740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.329349+0000","last_change":"2024-09-21T07:34:08.173122+0000","last_active":"2024-09-21T07:34:09.329349+0000","last_peered":"2024-09-21T07:34:09.329349+0000","last_clean":"2024-09-21T07:34:09.329349+0000","last_became_active":"2024-09-21T07:34:08.172780+0000","last_became_peered":"2024-09-21T07:34:08.172780+0000","last_unstale":"2024-09-21T07:34:09.329349+0000","last_undegraded":"2024-09-21T07:34:09.329349+0000","last_fullsized":"2024-09-21T07:34:09.329349+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:27.710069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.329093+0000","last_change":"2024-09-21T07:34:08.167193+0000","last_active":"2024-09-21T07:34:09.329093+0000","last_peered":"2024-09-21T07:34:09.329093+0000","last_clean":"2024-09-21T07:34:09.329093+0000","last_became_active":"2024-09-21T07:34:08.165230+0000","last_became_peered":"2024-09-21T07:34:08.165230+0000","last_unstale":"2024-09-21T07:34:09.329093+0000","last_undegraded":"2024-09-21T07:34:09.329093+0000","last_fullsized":"2024-09-21T07:34:09.329093+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:39.114639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497858+0000","last_change":"2024-09-21T07:34:08.170445+0000","last_active":"2024-09-21T07:34:09.497858+0000","last_peered":"2024-09-21T07:34:09.497858+0000","last_clean":"2024-09-21T07:34:09.497858+0000","last_became_active":"2024-09-21T07:34:08.170115+0000","last_became_peered":"2024-09-21T07:34:08.170115+0000","last_unstale":"2024-09-21T07:34:09.497858+0000","last_undegraded":"2024-09-21T07:34:09.497858+0000","last_fullsized":"2024-09-21T07:34:09.497858+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:32.224498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.181226+0000","last_change":"2024-09-21T07:34:08.162257+0000","last_active":"2024-09-21T07:34:09.181226+0000","last_peered":"2024-09-21T07:34:09.181226+0000","last_clean":"2024-09-21T07:34:09.181226+0000","last_became_active":"2024-09-21T07:34:08.161939+0000","last_became_peered":"2024-09-21T07:34:08.161939+0000","last_unstale":"2024-09-21T07:34:09.181226+0000","last_undegraded":"2024-09-21T07:34:09.181226+0000","last_fullsized":"2024-09-21T07:34:09.181226+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:32:50.694783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.329426+0000","last_change":"2024-09-21T07:34:08.177569+0000","last_active":"2024-09-21T07:34:09.329426+0000","last_peered":"2024-09-21T07:34:09.329426+0000","last_clean":"2024-09-21T07:34:09.329426+0000","last_became_active":"2024-09-21T07:34:08.177198+0000","last_became_peered":"2024-09-21T07:34:08.177198+0000","last_unstale":"2024-09-21T07:34:09.329426+0000","last_undegraded":"2024-09-21T07:34:09.329426+0000","last_fullsized":"2024-09-21T07:34:09.329426+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:50.102777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.165047+0000","last_change":"2024-09-21T07:34:08.155812+0000","last_active":"2024-09-21T07:34:09.165047+0000","last_peered":"2024-09-21T07:34:09.165047+0000","last_clean":"2024-09-21T07:34:09.165047+0000","last_became_active":"2024-09-21T07:34:08.155490+0000","last_became_peered":"2024-09-21T07:34:08.155490+0000","last_unstale":"2024-09-21T07:34:09.165047+0000","last_undegraded":"2024-09-21T07:34:09.165047+0000","last_fullsized":"2024-09-21T07:34:09.165047+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:13.730449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.163440+0000","last_change":"2024-09-21T07:34:08.174944+0000","last_active":"2024-09-21T07:34:09.163440+0000","last_peered":"2024-09-21T07:34:09.163440+0000","last_clean":"2024-09-21T07:34:09.163440+0000","last_became_active":"2024-09-21T07:34:08.173967+0000","last_became_peered":"2024-09-21T07:34:08.173967+0000","last_unstale":"2024-09-21T07:34:09.163440+0000","last_undegraded":"2024-09-21T07:34:09.163440+0000","last_fullsized":"2024-09-21T07:34:09.163440+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:55.791945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496432+0000","last_change":"2024-09-21T07:34:08.167950+0000","last_active":"2024-09-21T07:34:09.496432+0000","last_peered":"2024-09-21T07:34:09.496432+0000","last_clean":"2024-09-21T07:34:09.496432+0000","last_became_active":"2024-09-21T07:34:08.167687+0000","last_became_peered":"2024-09-21T07:34:08.167687+0000","last_unstale":"2024-09-21T07:34:09.496432+0000","last_undegraded":"2024-09-21T07:34:09.496432+0000","last_fullsized":"2024-09-21T07:34:09.496432+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:44:01.238051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.164897+0000","last_change":"2024-09-21T07:34:08.166495+0000","last_active":"2024-09-21T07:34:09.164897+0000","last_peered":"2024-09-21T07:34:09.164897+0000","last_clean":"2024-09-21T07:34:09.164897+0000","last_became_active":"2024-09-21T07:34:08.166104+0000","last_became_peered":"2024-09-21T07:34:08.166104+0000","last_unstale":"2024-09-21T07:34:09.164897+0000","last_undegraded":"2024-09-21T07:34:09.164897+0000","last_fullsized":"2024-09-21T07:34:09.164897+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:31:12.988796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.329881+0000","last_change":"2024-09-21T07:34:08.172013+0000","last_active":"2024-09-21T07:34:09.329881+0000","last_peered":"2024-09-21T07:34:09.329881+0000","last_clean":"2024-09-21T07:34:09.329881+0000","last_became_active":"2024-09-21T07:34:08.171497+0000","last_became_peered":"2024-09-21T07:34:08.171497+0000","last_unstale":"2024-09-21T07:34:09.329881+0000","last_undegraded":"2024-09-21T07:34:09.329881+0000","last_fullsized":"2024-09-21T07:34:09.329881+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:28.526013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.329782+0000","last_change":"2024-09-21T07:34:08.176187+0000","last_active":"2024-09-21T07:34:09.329782+0000","last_peered":"2024-09-21T07:34:09.329782+0000","last_clean":"2024-09-21T07:34:09.329782+0000","last_became_active":"2024-09-21T07:34:08.175895+0000","last_became_peered":"2024-09-21T07:34:08.175895+0000","last_unstale":"2024-09-21T07:34:09.329782+0000","last_undegraded":"2024-09-21T07:34:09.329782+0000","last_fullsized":"2024-09-21T07:34:09.329782+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:10.107127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496674+0000","last_change":"2024-09-21T07:34:08.153003+0000","last_active":"2024-09-21T07:34:09.496674+0000","last_peered":"2024-09-21T07:34:09.496674+0000","last_clean":"2024-09-21T07:34:09.496674+0000","last_became_active":"2024-09-21T07:34:08.150882+0000","last_became_peered":"2024-09-21T07:34:08.150882+0000","last_unstale":"2024-09-21T07:34:09.496674+0000","last_undegraded":"2024-09-21T07:34:09.496674+0000","last_fullsized":"2024-09-21T07:34:09.496674+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:38.611907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496759+0000","last_change":"2024-09-21T07:34:08.158999+0000","last_active":"2024-09-21T07:34:09.496759+0000","last_peered":"2024-09-21T07:34:09.496759+0000","last_clean":"2024-09-21T07:34:09.496759+0000","last_became_active":"2024-09-21T07:34:08.158712+0000","last_became_peered":"2024-09-21T07:34:08.158712+0000","last_unstale":"2024-09-21T07:34:09.496759+0000","last_undegraded":"2024-09-21T07:34:09.496759+0000","last_fullsized":"2024-09-21T07:34:09.496759+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:34:04.145536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.163438+0000","last_change":"2024-09-21T07:34:08.168400+0000","last_active":"2024-09-21T07:34:09.163438+0000","last_peered":"2024-09-21T07:34:09.163438+0000","last_clean":"2024-09-21T07:34:09.163438+0000","last_became_active":"2024-09-21T07:34:08.168071+0000","last_became_peered":"2024-09-21T07:34:08.168071+0000","last_unstale":"2024-09-21T07:34:09.163438+0000","last_undegraded":"2024-09-21T07:34:09.163438+0000","last_fullsized":"2024-09-21T07:34:09.163438+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:01.249634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496796+0000","last_change":"2024-09-21T07:34:08.168578+0000","last_active":"2024-09-21T07:34:09.496796+0000","last_peered":"2024-09-21T07:34:09.496796+0000","last_clean":"2024-09-21T07:34:09.496796+0000","last_became_active":"2024-09-21T07:34:08.168235+0000","last_became_peered":"2024-09-21T07:34:08.168235+0000","last_unstale":"2024-09-21T07:34:09.496796+0000","last_undegraded":"2024-09-21T07:34:09.496796+0000","last_fullsized":"2024-09-21T07:34:09.496796+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:18.132264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496889+0000","last_change":"2024-09-21T07:34:08.159192+0000","last_active":"2024-09-21T07:34:09.496889+0000","last_peered":"2024-09-21T07:34:09.496889+0000","last_clean":"2024-09-21T07:34:09.496889+0000","last_became_active":"2024-09-21T07:34:08.158860+0000","last_became_peered":"2024-09-21T07:34:08.158860+0000","last_unstale":"2024-09-21T07:34:09.496889+0000","last_undegraded":"2024-09-21T07:34:09.496889+0000","last_fullsized":"2024-09-21T07:34:09.496889+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:48.287489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.168895+0000","last_change":"2024-09-21T07:34:08.164892+0000","last_active":"2024-09-21T07:34:09.168895+0000","last_peered":"2024-09-21T07:34:09.168895+0000","last_clean":"2024-09-21T07:34:09.168895+0000","last_became_active":"2024-09-21T07:34:08.163504+0000","last_became_peered":"2024-09-21T07:34:08.163504+0000","last_unstale":"2024-09-21T07:34:09.168895+0000","last_undegraded":"2024-09-21T07:34:09.168895+0000","last_fullsized":"2024-09-21T07:34:09.168895+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:55.716857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.164789+0000","last_change":"2024-09-21T07:34:08.164618+0000","last_active":"2024-09-21T07:34:09.164789+0000","last_peered":"2024-09-21T07:34:09.164789+0000","last_clean":"2024-09-21T07:34:09.164789+0000","last_became_active":"2024-09-21T07:34:08.163696+0000","last_became_peered":"2024-09-21T07:34:08.163696+0000","last_unstale":"2024-09-21T07:34:09.164789+0000","last_undegraded":"2024-09-21T07:34:09.164789+0000","last_fullsized":"2024-09-21T07:34:09.164789+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:08:41.286205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.181440+0000","last_change":"2024-09-21T07:34:08.163763+0000","last_active":"2024-09-21T07:34:09.181440+0000","last_peered":"2024-09-21T07:34:09.181440+0000","last_clean":"2024-09-21T07:34:09.181440+0000","last_became_active":"2024-09-21T07:34:08.163455+0000","last_became_peered":"2024-09-21T07:34:08.163455+0000","last_unstale":"2024-09-21T07:34:09.181440+0000","last_undegraded":"2024-09-21T07:34:09.181440+0000","last_fullsized":"2024-09-21T07:34:09.181440+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:39.114639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496941+0000","last_change":"2024-09-21T07:34:08.158447+0000","last_active":"2024-09-21T07:34:09.496941+0000","last_peered":"2024-09-21T07:34:09.496941+0000","last_clean":"2024-09-21T07:34:09.496941+0000","last_became_active":"2024-09-21T07:34:08.158138+0000","last_became_peered":"2024-09-21T07:34:08.158138+0000","last_unstale":"2024-09-21T07:34:09.496941+0000","last_undegraded":"2024-09-21T07:34:09.496941+0000","last_fullsized":"2024-09-21T07:34:09.496941+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:05.155134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.181311+0000","last_change":"2024-09-21T07:34:08.172580+0000","last_active":"2024-09-21T07:34:09.181311+0000","last_peered":"2024-09-21T07:34:09.181311+0000","last_clean":"2024-09-21T07:34:09.181311+0000","last_became_active":"2024-09-21T07:34:08.172294+0000","last_became_peered":"2024-09-21T07:34:08.172294+0000","last_unstale":"2024-09-21T07:34:09.181311+0000","last_undegraded":"2024-09-21T07:34:09.181311+0000","last_fullsized":"2024-09-21T07:34:09.181311+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:21.245892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.165259+0000","last_change":"2024-09-21T07:34:08.169640+0000","last_active":"2024-09-21T07:34:09.165259+0000","last_peered":"2024-09-21T07:34:09.165259+0000","last_clean":"2024-09-21T07:34:09.165259+0000","last_became_active":"2024-09-21T07:34:08.169283+0000","last_became_peered":"2024-09-21T07:34:08.169283+0000","last_unstale":"2024-09-21T07:34:09.165259+0000","last_undegraded":"2024-09-21T07:34:09.165259+0000","last_fullsized":"2024-09-21T07:34:09.165259+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:07.122419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497087+0000","last_change":"2024-09-21T07:34:08.153475+0000","last_active":"2024-09-21T07:34:09.497087+0000","last_peered":"2024-09-21T07:34:09.497087+0000","last_clean":"2024-09-21T07:34:09.497087+0000","last_became_active":"2024-09-21T07:34:08.152735+0000","last_became_peered":"2024-09-21T07:34:08.152735+0000","last_unstale":"2024-09-21T07:34:09.497087+0000","last_undegraded":"2024-09-21T07:34:09.497087+0000","last_fullsized":"2024-09-21T07:34:09.497087+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:42.395864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.329350+0000","last_change":"2024-09-21T07:34:08.175295+0000","last_active":"2024-09-21T07:34:09.329350+0000","last_peered":"2024-09-21T07:34:09.329350+0000","last_clean":"2024-09-21T07:34:09.329350+0000","last_became_active":"2024-09-21T07:34:08.175002+0000","last_became_peered":"2024-09-21T07:34:08.175002+0000","last_unstale":"2024-09-21T07:34:09.329350+0000","last_undegraded":"2024-09-21T07:34:09.329350+0000","last_fullsized":"2024-09-21T07:34:09.329350+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:40.829187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496269+0000","last_change":"2024-09-21T07:34:08.163793+0000","last_active":"2024-09-21T07:34:09.496269+0000","last_peered":"2024-09-21T07:34:09.496269+0000","last_clean":"2024-09-21T07:34:09.496269+0000","last_became_active":"2024-09-21T07:34:08.163391+0000","last_became_peered":"2024-09-21T07:34:08.163391+0000","last_unstale":"2024-09-21T07:34:09.496269+0000","last_undegraded":"2024-09-21T07:34:09.496269+0000","last_fullsized":"2024-09-21T07:34:09.496269+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:55.152456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.329080+0000","last_change":"2024-09-21T07:34:08.173785+0000","last_active":"2024-09-21T07:34:09.329080+0000","last_peered":"2024-09-21T07:34:09.329080+0000","last_clean":"2024-09-21T07:34:09.329080+0000","last_became_active":"2024-09-21T07:34:08.173332+0000","last_became_peered":"2024-09-21T07:34:08.173332+0000","last_unstale":"2024-09-21T07:34:09.329080+0000","last_undegraded":"2024-09-21T07:34:09.329080+0000","last_fullsized":"2024-09-21T07:34:09.329080+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:13.730449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.169412+0000","last_change":"2024-09-21T07:34:08.166699+0000","last_active":"2024-09-21T07:34:09.169412+0000","last_peered":"2024-09-21T07:34:09.169412+0000","last_clean":"2024-09-21T07:34:09.169412+0000","last_became_active":"2024-09-21T07:34:08.161047+0000","last_became_peered":"2024-09-21T07:34:08.161047+0000","last_unstale":"2024-09-21T07:34:09.169412+0000","last_undegraded":"2024-09-21T07:34:09.169412+0000","last_fullsized":"2024-09-21T07:34:09.169412+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:34.740594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":19,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497099+0000","last_change":"2024-09-21T07:34:07.155982+0000","last_active":"2024-09-21T07:34:09.497099+0000","last_peered":"2024-09-21T07:34:09.497099+0000","last_clean":"2024-09-21T07:34:09.497099+0000","last_became_active":"2024-09-21T07:34:07.155500+0000","last_became_peered":"2024-09-21T07:34:07.155500+0000","last_unstale":"2024-09-21T07:34:09.497099+0000","last_undegraded":"2024-09-21T07:34:09.497099+0000","last_fullsized":"2024-09-21T07:34:09.497099+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:26.882263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497060+0000","last_change":"2024-09-21T07:34:08.178748+0000","last_active":"2024-09-21T07:34:09.497060+0000","last_peered":"2024-09-21T07:34:09.497060+0000","last_clean":"2024-09-21T07:34:09.497060+0000","last_became_active":"2024-09-21T07:34:08.177806+0000","last_became_peered":"2024-09-21T07:34:08.177806+0000","last_unstale":"2024-09-21T07:34:09.497060+0000","last_undegraded":"2024-09-21T07:34:09.497060+0000","last_fullsized":"2024-09-21T07:34:09.497060+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:47.458768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.169477+0000","last_change":"2024-09-21T07:34:07.148229+0000","last_active":"2024-09-21T07:34:09.169477+0000","last_peered":"2024-09-21T07:34:09.169477+0000","last_clean":"2024-09-21T07:34:09.169477+0000","last_became_active":"2024-09-21T07:34:07.147920+0000","last_became_peered":"2024-09-21T07:34:07.147920+0000","last_unstale":"2024-09-21T07:34:09.169477+0000","last_undegraded":"2024-09-21T07:34:09.169477+0000","last_fullsized":"2024-09-21T07:34:09.169477+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:09.410183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497703+0000","last_change":"2024-09-21T07:34:08.162272+0000","last_active":"2024-09-21T07:34:09.497703+0000","last_peered":"2024-09-21T07:34:09.497703+0000","last_clean":"2024-09-21T07:34:09.497703+0000","last_became_active":"2024-09-21T07:34:08.162031+0000","last_became_peered":"2024-09-21T07:34:08.162031+0000","last_unstale":"2024-09-21T07:34:09.497703+0000","last_undegraded":"2024-09-21T07:34:09.497703+0000","last_fullsized":"2024-09-21T07:34:09.497703+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:48.649163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496548+0000","last_change":"2024-09-21T07:34:07.162816+0000","last_active":"2024-09-21T07:34:09.496548+0000","last_peered":"2024-09-21T07:34:09.496548+0000","last_clean":"2024-09-21T07:34:09.496548+0000","last_became_active":"2024-09-21T07:34:07.162478+0000","last_became_peered":"2024-09-21T07:34:07.162478+0000","last_unstale":"2024-09-21T07:34:09.496548+0000","last_undegraded":"2024-09-21T07:34:09.496548+0000","last_fullsized":"2024-09-21T07:34:09.496548+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:18:03.659966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.181339+0000","last_change":"2024-09-21T07:34:08.172648+0000","last_active":"2024-09-21T07:34:09.181339+0000","last_peered":"2024-09-21T07:34:09.181339+0000","last_clean":"2024-09-21T07:34:09.181339+0000","last_became_active":"2024-09-21T07:34:08.172311+0000","last_became_peered":"2024-09-21T07:34:08.172311+0000","last_unstale":"2024-09-21T07:34:09.181339+0000","last_undegraded":"2024-09-21T07:34:09.181339+0000","last_fullsized":"2024-09-21T07:34:09.181339+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:25.699460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.163416+0000","last_change":"2024-09-21T07:34:07.151195+0000","last_active":"2024-09-21T07:34:09.163416+0000","last_peered":"2024-09-21T07:34:09.163416+0000","last_clean":"2024-09-21T07:34:09.163416+0000","last_became_active":"2024-09-21T07:34:07.150871+0000","last_became_peered":"2024-09-21T07:34:07.150871+0000","last_unstale":"2024-09-21T07:34:09.163416+0000","last_undegraded":"2024-09-21T07:34:09.163416+0000","last_fullsized":"2024-09-21T07:34:09.163416+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:27:38.559677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.329459+0000","last_change":"2024-09-21T07:34:08.174424+0000","last_active":"2024-09-21T07:34:09.329459+0000","last_peered":"2024-09-21T07:34:09.329459+0000","last_clean":"2024-09-21T07:34:09.329459+0000","last_became_active":"2024-09-21T07:34:08.174028+0000","last_became_peered":"2024-09-21T07:34:08.174028+0000","last_unstale":"2024-09-21T07:34:09.329459+0000","last_undegraded":"2024-09-21T07:34:09.329459+0000","last_fullsized":"2024-09-21T07:34:09.329459+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:31.524701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497041+0000","last_change":"2024-09-21T07:34:07.154783+0000","last_active":"2024-09-21T07:34:09.497041+0000","last_peered":"2024-09-21T07:34:09.497041+0000","last_clean":"2024-09-21T07:34:09.497041+0000","last_became_active":"2024-09-21T07:34:07.154378+0000","last_became_peered":"2024-09-21T07:34:07.154378+0000","last_unstale":"2024-09-21T07:34:09.497041+0000","last_undegraded":"2024-09-21T07:34:09.497041+0000","last_fullsized":"2024-09-21T07:34:09.497041+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:16.041455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.164907+0000","last_change":"2024-09-21T07:34:08.166752+0000","last_active":"2024-09-21T07:34:09.164907+0000","last_peered":"2024-09-21T07:34:09.164907+0000","last_clean":"2024-09-21T07:34:09.164907+0000","last_became_active":"2024-09-21T07:34:08.166477+0000","last_became_peered":"2024-09-21T07:34:08.166477+0000","last_unstale":"2024-09-21T07:34:09.164907+0000","last_undegraded":"2024-09-21T07:34:09.164907+0000","last_fullsized":"2024-09-21T07:34:09.164907+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:25.699460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496886+0000","last_change":"2024-09-21T07:34:07.153288+0000","last_active":"2024-09-21T07:34:09.496886+0000","last_peered":"2024-09-21T07:34:09.496886+0000","last_clean":"2024-09-21T07:34:09.496886+0000","last_became_active":"2024-09-21T07:34:07.152949+0000","last_became_peered":"2024-09-21T07:34:07.152949+0000","last_unstale":"2024-09-21T07:34:09.496886+0000","last_undegraded":"2024-09-21T07:34:09.496886+0000","last_fullsized":"2024-09-21T07:34:09.496886+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:54:43.804810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496478+0000","last_change":"2024-09-21T07:34:08.166316+0000","last_active":"2024-09-21T07:34:09.496478+0000","last_peered":"2024-09-21T07:34:09.496478+0000","last_clean":"2024-09-21T07:34:09.496478+0000","last_became_active":"2024-09-21T07:34:08.165006+0000","last_became_peered":"2024-09-21T07:34:08.165006+0000","last_unstale":"2024-09-21T07:34:09.496478+0000","last_undegraded":"2024-09-21T07:34:09.496478+0000","last_fullsized":"2024-09-21T07:34:09.496478+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:27.528708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496037+0000","last_change":"2024-09-21T07:34:07.141466+0000","last_active":"2024-09-21T07:34:09.496037+0000","last_peered":"2024-09-21T07:34:09.496037+0000","last_clean":"2024-09-21T07:34:09.496037+0000","last_became_active":"2024-09-21T07:34:07.141003+0000","last_became_peered":"2024-09-21T07:34:07.141003+0000","last_unstale":"2024-09-21T07:34:09.496037+0000","last_undegraded":"2024-09-21T07:34:09.496037+0000","last_fullsized":"2024-09-21T07:34:09.496037+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:52:33.957055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.180647+0000","last_change":"2024-09-21T07:34:08.163550+0000","last_active":"2024-09-21T07:34:09.180647+0000","last_peered":"2024-09-21T07:34:09.180647+0000","last_clean":"2024-09-21T07:34:09.180647+0000","last_became_active":"2024-09-21T07:34:08.163236+0000","last_became_peered":"2024-09-21T07:34:08.163236+0000","last_unstale":"2024-09-21T07:34:09.180647+0000","last_undegraded":"2024-09-21T07:34:09.180647+0000","last_fullsized":"2024-09-21T07:34:09.180647+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:56.729284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496450+0000","last_change":"2024-09-21T07:34:07.142126+0000","last_active":"2024-09-21T07:34:09.496450+0000","last_peered":"2024-09-21T07:34:09.496450+0000","last_clean":"2024-09-21T07:34:09.496450+0000","last_became_active":"2024-09-21T07:34:07.141713+0000","last_became_peered":"2024-09-21T07:34:07.141713+0000","last_unstale":"2024-09-21T07:34:09.496450+0000","last_undegraded":"2024-09-21T07:34:09.496450+0000","last_fullsized":"2024-09-21T07:34:09.496450+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:58.789677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.165516+0000","last_change":"2024-09-21T07:34:08.159275+0000","last_active":"2024-09-21T07:34:09.165516+0000","last_peered":"2024-09-21T07:34:09.165516+0000","last_clean":"2024-09-21T07:34:09.165516+0000","last_became_active":"2024-09-21T07:34:08.159005+0000","last_became_peered":"2024-09-21T07:34:08.159005+0000","last_unstale":"2024-09-21T07:34:09.165516+0000","last_undegraded":"2024-09-21T07:34:09.165516+0000","last_fullsized":"2024-09-21T07:34:09.165516+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:50.102777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496527+0000","last_change":"2024-09-21T07:34:07.154435+0000","last_active":"2024-09-21T07:34:09.496527+0000","last_peered":"2024-09-21T07:34:09.496527+0000","last_clean":"2024-09-21T07:34:09.496527+0000","last_became_active":"2024-09-21T07:34:07.154127+0000","last_became_peered":"2024-09-21T07:34:07.154127+0000","last_unstale":"2024-09-21T07:34:09.496527+0000","last_undegraded":"2024-09-21T07:34:09.496527+0000","last_fullsized":"2024-09-21T07:34:09.496527+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:11:21.678594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496615+0000","last_change":"2024-09-21T07:34:08.170301+0000","last_active":"2024-09-21T07:34:09.496615+0000","last_peered":"2024-09-21T07:34:09.496615+0000","last_clean":"2024-09-21T07:34:09.496615+0000","last_became_active":"2024-09-21T07:34:08.169925+0000","last_became_peered":"2024-09-21T07:34:08.169925+0000","last_unstale":"2024-09-21T07:34:09.496615+0000","last_undegraded":"2024-09-21T07:34:09.496615+0000","last_fullsized":"2024-09-21T07:34:09.496615+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:31.073725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.169543+0000","last_change":"2024-09-21T07:34:07.152298+0000","last_active":"2024-09-21T07:34:09.169543+0000","last_peered":"2024-09-21T07:34:09.169543+0000","last_clean":"2024-09-21T07:34:09.169543+0000","last_became_active":"2024-09-21T07:34:07.151921+0000","last_became_peered":"2024-09-21T07:34:07.151921+0000","last_unstale":"2024-09-21T07:34:09.169543+0000","last_undegraded":"2024-09-21T07:34:09.169543+0000","last_fullsized":"2024-09-21T07:34:09.169543+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:52.169625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496907+0000","last_change":"2024-09-21T07:34:08.160957+0000","last_active":"2024-09-21T07:34:09.496907+0000","last_peered":"2024-09-21T07:34:09.496907+0000","last_clean":"2024-09-21T07:34:09.496907+0000","last_became_active":"2024-09-21T07:34:08.160601+0000","last_became_peered":"2024-09-21T07:34:08.160601+0000","last_unstale":"2024-09-21T07:34:09.496907+0000","last_undegraded":"2024-09-21T07:34:09.496907+0000","last_fullsized":"2024-09-21T07:34:09.496907+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:19:16.261092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.165514+0000","last_change":"2024-09-21T07:34:07.153099+0000","last_active":"2024-09-21T07:34:09.165514+0000","last_peered":"2024-09-21T07:34:09.165514+0000","last_clean":"2024-09-21T07:34:09.165514+0000","last_became_active":"2024-09-21T07:34:07.152790+0000","last_became_peered":"2024-09-21T07:34:07.152790+0000","last_unstale":"2024-09-21T07:34:09.165514+0000","last_undegraded":"2024-09-21T07:34:09.165514+0000","last_fullsized":"2024-09-21T07:34:09.165514+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:51.990530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.165280+0000","last_change":"2024-09-21T07:34:08.165826+0000","last_active":"2024-09-21T07:34:09.165280+0000","last_peered":"2024-09-21T07:34:09.165280+0000","last_clean":"2024-09-21T07:34:09.165280+0000","last_became_active":"2024-09-21T07:34:08.160912+0000","last_became_peered":"2024-09-21T07:34:08.160912+0000","last_unstale":"2024-09-21T07:34:09.165280+0000","last_undegraded":"2024-09-21T07:34:09.165280+0000","last_fullsized":"2024-09-21T07:34:09.165280+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:27.710069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":21,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496721+0000","last_change":"2024-09-21T07:34:07.162976+0000","last_active":"2024-09-21T07:34:09.496721+0000","last_peered":"2024-09-21T07:34:09.496721+0000","last_clean":"2024-09-21T07:34:09.496721+0000","last_became_active":"2024-09-21T07:34:07.162698+0000","last_became_peered":"2024-09-21T07:34:07.162698+0000","last_unstale":"2024-09-21T07:34:09.496721+0000","last_undegraded":"2024-09-21T07:34:09.496721+0000","last_fullsized":"2024-09-21T07:34:09.496721+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:01:04.212051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.180755+0000","last_change":"2024-09-21T07:34:08.168839+0000","last_active":"2024-09-21T07:34:09.180755+0000","last_peered":"2024-09-21T07:34:09.180755+0000","last_clean":"2024-09-21T07:34:09.180755+0000","last_became_active":"2024-09-21T07:34:08.168183+0000","last_became_peered":"2024-09-21T07:34:08.168183+0000","last_unstale":"2024-09-21T07:34:09.180755+0000","last_undegraded":"2024-09-21T07:34:09.180755+0000","last_fullsized":"2024-09-21T07:34:09.180755+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:28.055160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497188+0000","last_change":"2024-09-21T07:34:07.150104+0000","last_active":"2024-09-21T07:34:09.497188+0000","last_peered":"2024-09-21T07:34:09.497188+0000","last_clean":"2024-09-21T07:34:09.497188+0000","last_became_active":"2024-09-21T07:34:07.149757+0000","last_became_peered":"2024-09-21T07:34:07.149757+0000","last_unstale":"2024-09-21T07:34:09.497188+0000","last_undegraded":"2024-09-21T07:34:09.497188+0000","last_fullsized":"2024-09-21T07:34:09.497188+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:17.255537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496697+0000","last_change":"2024-09-21T07:34:08.166843+0000","last_active":"2024-09-21T07:34:09.496697+0000","last_peered":"2024-09-21T07:34:09.496697+0000","last_clean":"2024-09-21T07:34:09.496697+0000","last_became_active":"2024-09-21T07:34:08.166516+0000","last_became_peered":"2024-09-21T07:34:08.166516+0000","last_unstale":"2024-09-21T07:34:09.496697+0000","last_undegraded":"2024-09-21T07:34:09.496697+0000","last_fullsized":"2024-09-21T07:34:09.496697+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:39.634059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.180719+0000","last_change":"2024-09-21T07:34:07.160820+0000","last_active":"2024-09-21T07:34:09.180719+0000","last_peered":"2024-09-21T07:34:09.180719+0000","last_clean":"2024-09-21T07:34:09.180719+0000","last_became_active":"2024-09-21T07:34:07.160364+0000","last_became_peered":"2024-09-21T07:34:07.160364+0000","last_unstale":"2024-09-21T07:34:09.180719+0000","last_undegraded":"2024-09-21T07:34:09.180719+0000","last_fullsized":"2024-09-21T07:34:09.180719+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:27.230155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.329827+0000","last_change":"2024-09-21T07:34:08.173868+0000","last_active":"2024-09-21T07:34:09.329827+0000","last_peered":"2024-09-21T07:34:09.329827+0000","last_clean":"2024-09-21T07:34:09.329827+0000","last_became_active":"2024-09-21T07:34:08.173552+0000","last_became_peered":"2024-09-21T07:34:08.173552+0000","last_unstale":"2024-09-21T07:34:09.329827+0000","last_undegraded":"2024-09-21T07:34:09.329827+0000","last_fullsized":"2024-09-21T07:34:09.329827+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:16.612188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496142+0000","last_change":"2024-09-21T07:34:07.153626+0000","last_active":"2024-09-21T07:34:09.496142+0000","last_peered":"2024-09-21T07:34:09.496142+0000","last_clean":"2024-09-21T07:34:09.496142+0000","last_became_active":"2024-09-21T07:34:07.153334+0000","last_became_peered":"2024-09-21T07:34:07.153334+0000","last_unstale":"2024-09-21T07:34:09.496142+0000","last_undegraded":"2024-09-21T07:34:09.496142+0000","last_fullsized":"2024-09-21T07:34:09.496142+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:48.214805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496501+0000","last_change":"2024-09-21T07:34:08.175374+0000","last_active":"2024-09-21T07:34:09.496501+0000","last_peered":"2024-09-21T07:34:09.496501+0000","last_clean":"2024-09-21T07:34:09.496501+0000","last_became_active":"2024-09-21T07:34:08.175065+0000","last_became_peered":"2024-09-21T07:34:08.175065+0000","last_unstale":"2024-09-21T07:34:09.496501+0000","last_undegraded":"2024-09-21T07:34:09.496501+0000","last_fullsized":"2024-09-21T07:34:09.496501+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:52.997974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496343+0000","last_change":"2024-09-21T07:34:07.161149+0000","last_active":"2024-09-21T07:34:09.496343+0000","last_peered":"2024-09-21T07:34:09.496343+0000","last_clean":"2024-09-21T07:34:09.496343+0000","last_became_active":"2024-09-21T07:34:07.160882+0000","last_became_peered":"2024-09-21T07:34:07.160882+0000","last_unstale":"2024-09-21T07:34:09.496343+0000","last_undegraded":"2024-09-21T07:34:09.496343+0000","last_fullsized":"2024-09-21T07:34:09.496343+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:55.427089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497420+0000","last_change":"2024-09-21T07:34:08.167375+0000","last_active":"2024-09-21T07:34:09.497420+0000","last_peered":"2024-09-21T07:34:09.497420+0000","last_clean":"2024-09-21T07:34:09.497420+0000","last_became_active":"2024-09-21T07:34:08.165919+0000","last_became_peered":"2024-09-21T07:34:08.165919+0000","last_unstale":"2024-09-21T07:34:09.497420+0000","last_undegraded":"2024-09-21T07:34:09.497420+0000","last_fullsized":"2024-09-21T07:34:09.497420+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:30.388748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.181509+0000","last_change":"2024-09-21T07:34:07.154981+0000","last_active":"2024-09-21T07:34:09.181509+0000","last_peered":"2024-09-21T07:34:09.181509+0000","last_clean":"2024-09-21T07:34:09.181509+0000","last_became_active":"2024-09-21T07:34:07.154706+0000","last_became_peered":"2024-09-21T07:34:07.154706+0000","last_unstale":"2024-09-21T07:34:09.181509+0000","last_undegraded":"2024-09-21T07:34:09.181509+0000","last_fullsized":"2024-09-21T07:34:09.181509+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:50.527238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.165808+0000","last_change":"2024-09-21T07:34:08.171328+0000","last_active":"2024-09-21T07:34:09.165808+0000","last_peered":"2024-09-21T07:34:09.165808+0000","last_clean":"2024-09-21T07:34:09.165808+0000","last_became_active":"2024-09-21T07:34:08.170351+0000","last_became_peered":"2024-09-21T07:34:08.170351+0000","last_unstale":"2024-09-21T07:34:09.165808+0000","last_undegraded":"2024-09-21T07:34:09.165808+0000","last_fullsized":"2024-09-21T07:34:09.165808+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:52.642170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497519+0000","last_change":"2024-09-21T07:34:07.149293+0000","last_active":"2024-09-21T07:34:09.497519+0000","last_peered":"2024-09-21T07:34:09.497519+0000","last_clean":"2024-09-21T07:34:09.497519+0000","last_became_active":"2024-09-21T07:34:07.148938+0000","last_became_peered":"2024-09-21T07:34:07.148938+0000","last_unstale":"2024-09-21T07:34:09.497519+0000","last_undegraded":"2024-09-21T07:34:09.497519+0000","last_fullsized":"2024-09-21T07:34:09.497519+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:59.434905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.180907+0000","last_change":"2024-09-21T07:34:08.177348+0000","last_active":"2024-09-21T07:34:09.180907+0000","last_peered":"2024-09-21T07:34:09.180907+0000","last_clean":"2024-09-21T07:34:09.180907+0000","last_became_active":"2024-09-21T07:34:08.174645+0000","last_became_peered":"2024-09-21T07:34:08.174645+0000","last_unstale":"2024-09-21T07:34:09.180907+0000","last_undegraded":"2024-09-21T07:34:09.180907+0000","last_fullsized":"2024-09-21T07:34:09.180907+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:06.515935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497479+0000","last_change":"2024-09-21T07:34:07.148308+0000","last_active":"2024-09-21T07:34:09.497479+0000","last_peered":"2024-09-21T07:34:09.497479+0000","last_clean":"2024-09-21T07:34:09.497479+0000","last_became_active":"2024-09-21T07:34:07.148001+0000","last_became_peered":"2024-09-21T07:34:07.148001+0000","last_unstale":"2024-09-21T07:34:09.497479+0000","last_undegraded":"2024-09-21T07:34:09.497479+0000","last_fullsized":"2024-09-21T07:34:09.497479+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:54:32.939044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.163433+0000","last_change":"2024-09-21T07:34:08.162638+0000","last_active":"2024-09-21T07:34:09.163433+0000","last_peered":"2024-09-21T07:34:09.163433+0000","last_clean":"2024-09-21T07:34:09.163433+0000","last_became_active":"2024-09-21T07:34:08.162237+0000","last_became_peered":"2024-09-21T07:34:08.162237+0000","last_unstale":"2024-09-21T07:34:09.163433+0000","last_undegraded":"2024-09-21T07:34:09.163433+0000","last_fullsized":"2024-09-21T07:34:09.163433+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:10.417627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497624+0000","last_change":"2024-09-21T07:34:07.152996+0000","last_active":"2024-09-21T07:34:09.497624+0000","last_peered":"2024-09-21T07:34:09.497624+0000","last_clean":"2024-09-21T07:34:09.497624+0000","last_became_active":"2024-09-21T07:34:07.152627+0000","last_became_peered":"2024-09-21T07:34:07.152627+0000","last_unstale":"2024-09-21T07:34:09.497624+0000","last_undegraded":"2024-09-21T07:34:09.497624+0000","last_fullsized":"2024-09-21T07:34:09.497624+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:24:42.920690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.163318+0000","last_change":"2024-09-21T07:34:08.173273+0000","last_active":"2024-09-21T07:34:09.163318+0000","last_peered":"2024-09-21T07:34:09.163318+0000","last_clean":"2024-09-21T07:34:09.163318+0000","last_became_active":"2024-09-21T07:34:08.172952+0000","last_became_peered":"2024-09-21T07:34:08.172952+0000","last_unstale":"2024-09-21T07:34:09.163318+0000","last_undegraded":"2024-09-21T07:34:09.163318+0000","last_fullsized":"2024-09-21T07:34:09.163318+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:47:39.799134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.169104+0000","last_change":"2024-09-21T07:34:07.151886+0000","last_active":"2024-09-21T07:34:09.169104+0000","last_peered":"2024-09-21T07:34:09.169104+0000","last_clean":"2024-09-21T07:34:09.169104+0000","last_became_active":"2024-09-21T07:34:07.150347+0000","last_became_peered":"2024-09-21T07:34:07.150347+0000","last_unstale":"2024-09-21T07:34:09.169104+0000","last_undegraded":"2024-09-21T07:34:09.169104+0000","last_fullsized":"2024-09-21T07:34:09.169104+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:27.230155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.163623+0000","last_change":"2024-09-21T07:34:08.174497+0000","last_active":"2024-09-21T07:34:09.163623+0000","last_peered":"2024-09-21T07:34:09.163623+0000","last_clean":"2024-09-21T07:34:09.163623+0000","last_became_active":"2024-09-21T07:34:08.173554+0000","last_became_peered":"2024-09-21T07:34:08.173554+0000","last_unstale":"2024-09-21T07:34:09.163623+0000","last_undegraded":"2024-09-21T07:34:09.163623+0000","last_fullsized":"2024-09-21T07:34:09.163623+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:18:16.768539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496838+0000","last_change":"2024-09-21T07:34:07.161337+0000","last_active":"2024-09-21T07:34:09.496838+0000","last_peered":"2024-09-21T07:34:09.496838+0000","last_clean":"2024-09-21T07:34:09.496838+0000","last_became_active":"2024-09-21T07:34:07.161055+0000","last_became_peered":"2024-09-21T07:34:07.161055+0000","last_unstale":"2024-09-21T07:34:09.496838+0000","last_undegraded":"2024-09-21T07:34:09.496838+0000","last_fullsized":"2024-09-21T07:34:09.496838+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:09.410183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.163185+0000","last_change":"2024-09-21T07:34:08.170546+0000","last_active":"2024-09-21T07:34:09.163185+0000","last_peered":"2024-09-21T07:34:09.163185+0000","last_clean":"2024-09-21T07:34:09.163185+0000","last_became_active":"2024-09-21T07:34:08.170249+0000","last_became_peered":"2024-09-21T07:34:08.170249+0000","last_unstale":"2024-09-21T07:34:09.163185+0000","last_undegraded":"2024-09-21T07:34:09.163185+0000","last_fullsized":"2024-09-21T07:34:09.163185+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:54.043603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497320+0000","last_change":"2024-09-21T07:34:07.148155+0000","last_active":"2024-09-21T07:34:09.497320+0000","last_peered":"2024-09-21T07:34:09.497320+0000","last_clean":"2024-09-21T07:34:09.497320+0000","last_became_active":"2024-09-21T07:34:07.147855+0000","last_became_peered":"2024-09-21T07:34:07.147855+0000","last_unstale":"2024-09-21T07:34:09.497320+0000","last_undegraded":"2024-09-21T07:34:09.497320+0000","last_fullsized":"2024-09-21T07:34:09.497320+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:41:10.667777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.329218+0000","last_change":"2024-09-21T07:34:08.165763+0000","last_active":"2024-09-21T07:34:09.329218+0000","last_peered":"2024-09-21T07:34:09.329218+0000","last_clean":"2024-09-21T07:34:09.329218+0000","last_became_active":"2024-09-21T07:34:08.163699+0000","last_became_peered":"2024-09-21T07:34:08.163699+0000","last_unstale":"2024-09-21T07:34:09.329218+0000","last_undegraded":"2024-09-21T07:34:09.329218+0000","last_fullsized":"2024-09-21T07:34:09.329218+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:14:22.162876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.165169+0000","last_change":"2024-09-21T07:34:07.156707+0000","last_active":"2024-09-21T07:34:09.165169+0000","last_peered":"2024-09-21T07:34:09.165169+0000","last_clean":"2024-09-21T07:34:09.165169+0000","last_became_active":"2024-09-21T07:34:07.156326+0000","last_became_peered":"2024-09-21T07:34:07.156326+0000","last_unstale":"2024-09-21T07:34:09.165169+0000","last_undegraded":"2024-09-21T07:34:09.165169+0000","last_fullsized":"2024-09-21T07:34:09.165169+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:54.145012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.163369+0000","last_change":"2024-09-21T07:34:08.166587+0000","last_active":"2024-09-21T07:34:09.163369+0000","last_peered":"2024-09-21T07:34:09.163369+0000","last_clean":"2024-09-21T07:34:09.163369+0000","last_became_active":"2024-09-21T07:34:08.166262+0000","last_became_peered":"2024-09-21T07:34:08.166262+0000","last_unstale":"2024-09-21T07:34:09.163369+0000","last_undegraded":"2024-09-21T07:34:09.163369+0000","last_fullsized":"2024-09-21T07:34:09.163369+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:27.889707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.181531+0000","last_change":"2024-09-21T07:34:07.153252+0000","last_active":"2024-09-21T07:34:09.181531+0000","last_peered":"2024-09-21T07:34:09.181531+0000","last_clean":"2024-09-21T07:34:09.181531+0000","last_became_active":"2024-09-21T07:34:07.152932+0000","last_became_peered":"2024-09-21T07:34:07.152932+0000","last_unstale":"2024-09-21T07:34:09.181531+0000","last_undegraded":"2024-09-21T07:34:09.181531+0000","last_fullsized":"2024-09-21T07:34:09.181531+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:24:14.371118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.165881+0000","last_change":"2024-09-21T07:34:08.171574+0000","last_active":"2024-09-21T07:34:09.165881+0000","last_peered":"2024-09-21T07:34:09.165881+0000","last_clean":"2024-09-21T07:34:09.165881+0000","last_became_active":"2024-09-21T07:34:08.170992+0000","last_became_peered":"2024-09-21T07:34:08.170992+0000","last_unstale":"2024-09-21T07:34:09.165881+0000","last_undegraded":"2024-09-21T07:34:09.165881+0000","last_fullsized":"2024-09-21T07:34:09.165881+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:31.524701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.169650+0000","last_change":"2024-09-21T07:34:07.161285+0000","last_active":"2024-09-21T07:34:09.169650+0000","last_peered":"2024-09-21T07:34:09.169650+0000","last_clean":"2024-09-21T07:34:09.169650+0000","last_became_active":"2024-09-21T07:34:07.160925+0000","last_became_peered":"2024-09-21T07:34:07.160925+0000","last_unstale":"2024-09-21T07:34:09.169650+0000","last_undegraded":"2024-09-21T07:34:09.169650+0000","last_fullsized":"2024-09-21T07:34:09.169650+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:54.784501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.169133+0000","last_change":"2024-09-21T07:34:08.160436+0000","last_active":"2024-09-21T07:34:09.169133+0000","last_peered":"2024-09-21T07:34:09.169133+0000","last_clean":"2024-09-21T07:34:09.169133+0000","last_became_active":"2024-09-21T07:34:08.159040+0000","last_became_peered":"2024-09-21T07:34:08.159040+0000","last_unstale":"2024-09-21T07:34:09.169133+0000","last_undegraded":"2024-09-21T07:34:09.169133+0000","last_fullsized":"2024-09-21T07:34:09.169133+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:30.259079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.181020+0000","last_change":"2024-09-21T07:34:07.151263+0000","last_active":"2024-09-21T07:34:09.181020+0000","last_peered":"2024-09-21T07:34:09.181020+0000","last_clean":"2024-09-21T07:34:09.181020+0000","last_became_active":"2024-09-21T07:34:07.150981+0000","last_became_peered":"2024-09-21T07:34:07.150981+0000","last_unstale":"2024-09-21T07:34:09.181020+0000","last_undegraded":"2024-09-21T07:34:09.181020+0000","last_fullsized":"2024-09-21T07:34:09.181020+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:26:19.106553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496799+0000","last_change":"2024-09-21T07:34:08.167728+0000","last_active":"2024-09-21T07:34:09.496799+0000","last_peered":"2024-09-21T07:34:09.496799+0000","last_clean":"2024-09-21T07:34:09.496799+0000","last_became_active":"2024-09-21T07:34:08.167409+0000","last_became_peered":"2024-09-21T07:34:08.167409+0000","last_unstale":"2024-09-21T07:34:09.496799+0000","last_undegraded":"2024-09-21T07:34:09.496799+0000","last_fullsized":"2024-09-21T07:34:09.496799+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:55.716857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.169272+0000","last_change":"2024-09-21T07:34:07.139885+0000","last_active":"2024-09-21T07:34:09.169272+0000","last_peered":"2024-09-21T07:34:09.169272+0000","last_clean":"2024-09-21T07:34:09.169272+0000","last_became_active":"2024-09-21T07:34:07.139560+0000","last_became_peered":"2024-09-21T07:34:07.139560+0000","last_unstale":"2024-09-21T07:34:09.169272+0000","last_undegraded":"2024-09-21T07:34:09.169272+0000","last_fullsized":"2024-09-21T07:34:09.169272+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:18:15.761095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.169169+0000","last_change":"2024-09-21T07:34:08.165778+0000","last_active":"2024-09-21T07:34:09.169169+0000","last_peered":"2024-09-21T07:34:09.169169+0000","last_clean":"2024-09-21T07:34:09.169169+0000","last_became_active":"2024-09-21T07:34:08.165481+0000","last_became_peered":"2024-09-21T07:34:08.165481+0000","last_unstale":"2024-09-21T07:34:09.169169+0000","last_undegraded":"2024-09-21T07:34:09.169169+0000","last_fullsized":"2024-09-21T07:34:09.169169+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:47.458768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.180835+0000","last_change":"2024-09-21T07:34:07.152465+0000","last_active":"2024-09-21T07:34:09.180835+0000","last_peered":"2024-09-21T07:34:09.180835+0000","last_clean":"2024-09-21T07:34:09.180835+0000","last_became_active":"2024-09-21T07:34:07.152065+0000","last_became_peered":"2024-09-21T07:34:07.152065+0000","last_unstale":"2024-09-21T07:34:09.180835+0000","last_undegraded":"2024-09-21T07:34:09.180835+0000","last_fullsized":"2024-09-21T07:34:09.180835+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:24:30.050440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496179+0000","last_change":"2024-09-21T07:34:08.150846+0000","last_active":"2024-09-21T07:34:09.496179+0000","last_peered":"2024-09-21T07:34:09.496179+0000","last_clean":"2024-09-21T07:34:09.496179+0000","last_became_active":"2024-09-21T07:34:08.150534+0000","last_became_peered":"2024-09-21T07:34:08.150534+0000","last_unstale":"2024-09-21T07:34:09.496179+0000","last_undegraded":"2024-09-21T07:34:09.496179+0000","last_fullsized":"2024-09-21T07:34:09.496179+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:26:49.717420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497461+0000","last_change":"2024-09-21T07:34:07.147358+0000","last_active":"2024-09-21T07:34:09.497461+0000","last_peered":"2024-09-21T07:34:09.497461+0000","last_clean":"2024-09-21T07:34:09.497461+0000","last_became_active":"2024-09-21T07:34:07.147036+0000","last_became_peered":"2024-09-21T07:34:07.147036+0000","last_unstale":"2024-09-21T07:34:09.497461+0000","last_undegraded":"2024-09-21T07:34:09.497461+0000","last_fullsized":"2024-09-21T07:34:09.497461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:04:22.493944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.497584+0000","last_change":"2024-09-21T07:34:08.159510+0000","last_active":"2024-09-21T07:34:09.497584+0000","last_peered":"2024-09-21T07:34:09.497584+0000","last_clean":"2024-09-21T07:34:09.497584+0000","last_became_active":"2024-09-21T07:34:08.159191+0000","last_became_peered":"2024-09-21T07:34:08.159191+0000","last_unstale":"2024-09-21T07:34:09.497584+0000","last_undegraded":"2024-09-21T07:34:09.497584+0000","last_fullsized":"2024-09-21T07:34:09.497584+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:27.319525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496085+0000","last_change":"2024-09-21T07:34:07.155987+0000","last_active":"2024-09-21T07:34:09.496085+0000","last_peered":"2024-09-21T07:34:09.496085+0000","last_clean":"2024-09-21T07:34:09.496085+0000","last_became_active":"2024-09-21T07:34:07.155735+0000","last_became_peered":"2024-09-21T07:34:07.155735+0000","last_unstale":"2024-09-21T07:34:09.496085+0000","last_undegraded":"2024-09-21T07:34:09.496085+0000","last_fullsized":"2024-09-21T07:34:09.496085+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:23:21.954791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.329467+0000","last_change":"2024-09-21T07:34:08.167002+0000","last_active":"2024-09-21T07:34:09.329467+0000","last_peered":"2024-09-21T07:34:09.329467+0000","last_clean":"2024-09-21T07:34:09.329467+0000","last_became_active":"2024-09-21T07:34:08.166679+0000","last_became_peered":"2024-09-21T07:34:08.166679+0000","last_unstale":"2024-09-21T07:34:09.329467+0000","last_undegraded":"2024-09-21T07:34:09.329467+0000","last_fullsized":"2024-09-21T07:34:09.329467+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:07.121447+0000","last_clean_scrub_stamp":"2024-09-21T07:34:07.121447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:52.642170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.496567+0000","last_change":"2024-09-21T07:34:07.155156+0000","last_active":"2024-09-21T07:34:09.496567+0000","last_peered":"2024-09-21T07:34:09.496567+0000","last_clean":"2024-09-21T07:34:09.496567+0000","last_became_active":"2024-09-21T07:34:07.154829+0000","last_became_peered":"2024-09-21T07:34:07.154829+0000","last_unstale":"2024-09-21T07:34:09.496567+0000","last_undegraded":"2024-09-21T07:34:09.496567+0000","last_fullsized":"2024-09-21T07:34:09.496567+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:06.114003+0000","last_clean_scrub_stamp":"2024-09-21T07:34:06.114003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:59:04.464801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":135,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-21T07:34:09.329996+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:34:09.329996+0000","last_peered":"2024-09-21T07:34:09.329996+0000","last_clean":"2024-09-21T07:34:09.329996+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:34:09.329996+0000","last_undegraded":"2024-09-21T07:34:09.329996+0000","last_fullsized":"2024-09-21T07:34:09.329996+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":974848,"data_stored":739153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509459,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":385024,"data_stored":149311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509458,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":385024,"data_stored":149367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509459,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31568,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340272,"statfs":{"total":96636764160,"available":96604438528,"internally_reserved":0,"allocated":380928,"data_stored":148765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509459,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":393216,"data_stored":150329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509459,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":970752,"data_stored":739063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542163,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":385024,"data_stored":148799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542162,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":393216,"data_stored":150329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:34:18.202 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T07:34:18.202 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T07:34:18.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.469+0000 7f0145a00640 1 -- 172.21.15.39:0/2257510587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0140073170 msgr2=0x7f014015c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:18.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.469+0000 7f0145a00640 1 --2- 172.21.15.39:0/2257510587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0140073170 0x7f014015c780 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0130005fd0 tx=0x7f013002a540 comp rx=0 tx=0).stop 2024-09-21T07:34:18.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.470+0000 7f0145a00640 1 -- 172.21.15.39:0/2257510587 shutdown_connections 2024-09-21T07:34:18.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.470+0000 7f0145a00640 1 --2- 172.21.15.39:0/2257510587 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0140070030 0x7f0140074640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:18.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.470+0000 7f0145a00640 1 --2- 172.21.15.39:0/2257510587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0140073170 0x7f014015c780 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:18.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.470+0000 7f0145a00640 1 -- 172.21.15.39:0/2257510587 >> 172.21.15.39:0/2257510587 conn(0x7f01400b1cb0 msgr2=0x7f01400b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:18.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.470+0000 7f0145a00640 1 -- 172.21.15.39:0/2257510587 shutdown_connections 2024-09-21T07:34:18.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.470+0000 7f0145a00640 1 -- 172.21.15.39:0/2257510587 wait complete. 2024-09-21T07:34:18.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.472+0000 7f0145a00640 1 Processor -- start 2024-09-21T07:34:18.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.472+0000 7f0145a00640 1 -- start start 2024-09-21T07:34:18.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.473+0000 7f0145a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0140070030 0x7f014009dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:18.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.473+0000 7f0145a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0140070440 0x7f014014f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:18.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.473+0000 7f0145a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0140073170 0x7f0140159d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:18.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.474+0000 7f0145a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f014006f5a0 con 0x7f0140070440 2024-09-21T07:34:18.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.474+0000 7f0145a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0140074f80 con 0x7f0140073170 2024-09-21T07:34:18.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.474+0000 7f0145a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f014006f720 con 0x7f0140070030 2024-09-21T07:34:18.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.474+0000 7f013e000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0140070440 0x7f014014f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:18.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.474+0000 7f013e000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0140070440 0x7f014014f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41498/0 (socket says 172.21.15.39:41498) 2024-09-21T07:34:18.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.474+0000 7f013e000640 1 -- 172.21.15.39:0/2393915472 learned_addr learned my addr 172.21.15.39:0/2393915472 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:18.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.474+0000 7f013ea00640 1 --2- 172.21.15.39:0/2393915472 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0140070030 0x7f014009dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:18.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.474+0000 7f013f400640 1 --2- 172.21.15.39:0/2393915472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0140073170 0x7f0140159d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:18.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.474+0000 7f013e000640 1 -- 172.21.15.39:0/2393915472 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0140070030 msgr2=0x7f014009dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:18.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.474+0000 7f013e000640 1 --2- 172.21.15.39:0/2393915472 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0140070030 0x7f014009dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:18.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.474+0000 7f013e000640 1 -- 172.21.15.39:0/2393915472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0140073170 msgr2=0x7f0140159d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:18.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.474+0000 7f013e000640 1 --2- 172.21.15.39:0/2393915472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0140073170 0x7f0140159d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:18.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.474+0000 7f013e000640 1 -- 172.21.15.39:0/2393915472 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0140158990 con 0x7f0140070440 2024-09-21T07:34:18.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.474+0000 7f013f400640 1 --2- 172.21.15.39:0/2393915472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0140073170 0x7f0140159d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:34:18.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.474+0000 7f013e000640 1 --2- 172.21.15.39:0/2393915472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0140070440 0x7f014014f490 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f012c00e7c0 tx=0x7f012c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:18.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.475+0000 7f0136a00640 1 -- 172.21.15.39:0/2393915472 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c00ccc0 con 0x7f0140070440 2024-09-21T07:34:18.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.475+0000 7f0136a00640 1 -- 172.21.15.39:0/2393915472 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f012c00ce60 con 0x7f0140070440 2024-09-21T07:34:18.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.475+0000 7f0145a00640 1 -- 172.21.15.39:0/2393915472 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0140158c30 con 0x7f0140070440 2024-09-21T07:34:18.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.475+0000 7f0136a00640 1 -- 172.21.15.39:0/2393915472 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c005280 con 0x7f0140070440 2024-09-21T07:34:18.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.475+0000 7f0145a00640 1 -- 172.21.15.39:0/2393915472 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01401591f0 con 0x7f0140070440 2024-09-21T07:34:18.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.476+0000 7f0145a00640 1 -- 172.21.15.39:0/2393915472 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f014006fc40 con 0x7f0140070440 2024-09-21T07:34:18.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.477+0000 7f0136a00640 1 -- 172.21.15.39:0/2393915472 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f012c005420 con 0x7f0140070440 2024-09-21T07:34:18.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.479+0000 7f0136a00640 1 --2- 172.21.15.39:0/2393915472 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0120078b90 0x7f012007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:18.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.479+0000 7f0136a00640 1 -- 172.21.15.39:0/2393915472 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f012c09b760 con 0x7f0140070440 2024-09-21T07:34:18.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.479+0000 7f013ea00640 1 --2- 172.21.15.39:0/2393915472 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0120078b90 0x7f012007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:18.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.480+0000 7f013ea00640 1 --2- 172.21.15.39:0/2393915472 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0120078b90 0x7f012007b0a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0130033000 tx=0x7f0130004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:18.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.483+0000 7f0136a00640 1 -- 172.21.15.39:0/2393915472 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f012c063b00 con 0x7f0140070440 2024-09-21T07:34:18.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.751+0000 7f0145a00640 1 -- 172.21.15.39:0/2393915472 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f01400c3860 con 0x7f0140070440 2024-09-21T07:34:18.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.755+0000 7f0136a00640 1 -- 172.21.15.39:0/2393915472 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) ==== 190+0+147 (secure 0 0 0) 0x7f012c063400 con 0x7f0140070440 2024-09-21T07:34:18.757 INFO:teuthology.orchestra.run.smithi039.stdout:[client.0] 2024-09-21T07:34:18.758 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQB0du5mGfb1DBAA/aSMKR5or3xy53rB/iUjpA== 2024-09-21T07:34:18.758 INFO:teuthology.orchestra.run.smithi039.stdout: caps mds = "allow" 2024-09-21T07:34:18.758 INFO:teuthology.orchestra.run.smithi039.stdout: caps mon = "allow r" 2024-09-21T07:34:18.758 INFO:teuthology.orchestra.run.smithi039.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T07:34:18.758 INFO:teuthology.orchestra.run.smithi039.stderr:updated caps for client.0 2024-09-21T07:34:18.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.758+0000 7f0145a00640 1 -- 172.21.15.39:0/2393915472 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0120078b90 msgr2=0x7f012007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:18.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.758+0000 7f0145a00640 1 --2- 172.21.15.39:0/2393915472 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0120078b90 0x7f012007b0a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0130033000 tx=0x7f0130004730 comp rx=0 tx=0).stop 2024-09-21T07:34:18.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.758+0000 7f0145a00640 1 -- 172.21.15.39:0/2393915472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0140070440 msgr2=0x7f014014f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:18.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.758+0000 7f0145a00640 1 --2- 172.21.15.39:0/2393915472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0140070440 0x7f014014f490 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f012c00e7c0 tx=0x7f012c00ec80 comp rx=0 tx=0).stop 2024-09-21T07:34:18.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.758+0000 7f0145a00640 1 -- 172.21.15.39:0/2393915472 shutdown_connections 2024-09-21T07:34:18.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.758+0000 7f0145a00640 1 --2- 172.21.15.39:0/2393915472 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0120078b90 0x7f012007b0a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:18.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.758+0000 7f0145a00640 1 --2- 172.21.15.39:0/2393915472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0140073170 0x7f0140159d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:18.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.758+0000 7f0145a00640 1 --2- 172.21.15.39:0/2393915472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0140070440 0x7f014014f490 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:18.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.758+0000 7f0145a00640 1 --2- 172.21.15.39:0/2393915472 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0140070030 0x7f014009dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:18.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.758+0000 7f0145a00640 1 -- 172.21.15.39:0/2393915472 >> 172.21.15.39:0/2393915472 conn(0x7f01400b1cb0 msgr2=0x7f01400ad950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:18.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.758+0000 7f0145a00640 1 -- 172.21.15.39:0/2393915472 shutdown_connections 2024-09-21T07:34:18.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:18.758+0000 7f0145a00640 1 -- 172.21.15.39:0/2393915472 wait complete. 2024-09-21T07:34:18.781 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:34:19.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.005+0000 7f3f32000640 1 -- 172.21.15.39:0/3114393304 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f2c0b56f0 msgr2=0x7f3f2c0b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:19.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.005+0000 7f3f32000640 1 --2- 172.21.15.39:0/3114393304 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f2c0b56f0 0x7f3f2c0b4c10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c08d270 tx=0x7f3f2002a460 comp rx=0 tx=0).stop 2024-09-21T07:34:19.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.005+0000 7f3f32000640 1 -- 172.21.15.39:0/3114393304 shutdown_connections 2024-09-21T07:34:19.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.005+0000 7f3f32000640 1 --2- 172.21.15.39:0/3114393304 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3f2c0d0ed0 0x7f3f2c0d12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:19.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.005+0000 7f3f32000640 1 --2- 172.21.15.39:0/3114393304 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f2c0b56f0 0x7f3f2c0b4c10 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:19.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.005+0000 7f3f32000640 1 -- 172.21.15.39:0/3114393304 >> 172.21.15.39:0/3114393304 conn(0x7f3f2c09d620 msgr2=0x7f3f2c0d4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:19.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.005+0000 7f3f32000640 1 -- 172.21.15.39:0/3114393304 shutdown_connections 2024-09-21T07:34:19.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.005+0000 7f3f32000640 1 -- 172.21.15.39:0/3114393304 wait complete. 2024-09-21T07:34:19.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.007+0000 7f3f32000640 1 Processor -- start 2024-09-21T07:34:19.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.007+0000 7f3f32000640 1 -- start start 2024-09-21T07:34:19.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.008+0000 7f3f32000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f2c0b56f0 0x7f3f2c160270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:19.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.009+0000 7f3f32000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f2c0d0ed0 0x7f3f2c1594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:19.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.009+0000 7f3f32000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3f2c0d2270 0x7f3f2c15dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:19.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.009+0000 7f3f32000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2c0d2f10 con 0x7f3f2c0b56f0 2024-09-21T07:34:19.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.009+0000 7f3f32000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2c0d2d90 con 0x7f3f2c0d0ed0 2024-09-21T07:34:19.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.009+0000 7f3f32000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f3f2c0d3090 con 0x7f3f2c0d2270 2024-09-21T07:34:19.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.009+0000 7f3f30c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f2c0b56f0 0x7f3f2c160270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:19.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.009+0000 7f3f30c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f2c0b56f0 0x7f3f2c160270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41512/0 (socket says 172.21.15.39:41512) 2024-09-21T07:34:19.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.009+0000 7f3f30c00640 1 -- 172.21.15.39:0/762186716 learned_addr learned my addr 172.21.15.39:0/762186716 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:19.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.009+0000 7f3f2be00640 1 --2- 172.21.15.39:0/762186716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f2c0d0ed0 0x7f3f2c1594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:19.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.009+0000 7f3f31600640 1 --2- 172.21.15.39:0/762186716 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3f2c0d2270 0x7f3f2c15dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:19.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.009+0000 7f3f30c00640 1 -- 172.21.15.39:0/762186716 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3f2c0d2270 msgr2=0x7f3f2c15dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:19.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.009+0000 7f3f30c00640 1 --2- 172.21.15.39:0/762186716 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3f2c0d2270 0x7f3f2c15dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:19.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.009+0000 7f3f30c00640 1 -- 172.21.15.39:0/762186716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f2c0d0ed0 msgr2=0x7f3f2c1594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:19.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.009+0000 7f3f30c00640 1 --2- 172.21.15.39:0/762186716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f2c0d0ed0 0x7f3f2c1594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:19.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.009+0000 7f3f30c00640 1 -- 172.21.15.39:0/762186716 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f2c159cc0 con 0x7f3f2c0b56f0 2024-09-21T07:34:19.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.009+0000 7f3f31600640 1 --2- 172.21.15.39:0/762186716 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3f2c0d2270 0x7f3f2c15dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:34:19.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.010+0000 7f3f30c00640 1 --2- 172.21.15.39:0/762186716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f2c0b56f0 0x7f3f2c160270 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3f2002aa50 tx=0x7f3f20033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:19.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.010+0000 7f3f28c00640 1 -- 172.21.15.39:0/762186716 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f20031d60 con 0x7f3f2c0b56f0 2024-09-21T07:34:19.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.010+0000 7f3f28c00640 1 -- 172.21.15.39:0/762186716 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f20033e10 con 0x7f3f2c0b56f0 2024-09-21T07:34:19.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.010+0000 7f3f28c00640 1 -- 172.21.15.39:0/762186716 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f200332d0 con 0x7f3f2c0b56f0 2024-09-21T07:34:19.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.010+0000 7f3f32000640 1 -- 172.21.15.39:0/762186716 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f2c1fa110 con 0x7f3f2c0b56f0 2024-09-21T07:34:19.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.010+0000 7f3f32000640 1 -- 172.21.15.39:0/762186716 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f2c0b3d90 con 0x7f3f2c0b56f0 2024-09-21T07:34:19.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.011+0000 7f3f32000640 1 -- 172.21.15.39:0/762186716 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f2c0b4c10 con 0x7f3f2c0b56f0 2024-09-21T07:34:19.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.013+0000 7f3f28c00640 1 -- 172.21.15.39:0/762186716 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3f20033470 con 0x7f3f2c0b56f0 2024-09-21T07:34:19.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.013+0000 7f3f28c00640 1 --2- 172.21.15.39:0/762186716 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3f0c078bc0 0x7f3f0c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:19.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.014+0000 7f3f28c00640 1 -- 172.21.15.39:0/762186716 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f3f200bc190 con 0x7f3f2c0b56f0 2024-09-21T07:34:19.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.014+0000 7f3f2be00640 1 --2- 172.21.15.39:0/762186716 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3f0c078bc0 0x7f3f0c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:19.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.015+0000 7f3f2be00640 1 --2- 172.21.15.39:0/762186716 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3f0c078bc0 0x7f3f0c07b0d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c15b040 tx=0x7f3f1800b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:19.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.018+0000 7f3f28c00640 1 -- 172.21.15.39:0/762186716 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3f2003f030 con 0x7f3f2c0b56f0 2024-09-21T07:34:19.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.252+0000 7f3f32000640 1 -- 172.21.15.39:0/762186716 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f2c0d1b40 con 0x7f3f2c0b56f0 2024-09-21T07:34:19.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.253+0000 7f3f28c00640 1 -- 172.21.15.39:0/762186716 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5229 (secure 0 0 0) 0x7f3f20084530 con 0x7f3f2c0b56f0 2024-09-21T07:34:19.255 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:34:19.255 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":17,"btime":"2024-09-21T07:34:11:169975+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2438395645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2438395645},{"type":"v1","addr":"172.21.15.148:6837","nonce":2438395645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4706,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3668947757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3668947757},{"type":"v1","addr":"172.21.15.39:6835","nonce":3668947757}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4870,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2800764560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2800764560},{"type":"v1","addr":"172.21.15.148:6835","nonce":2800764560}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:34:07.651669+0000","modified":"2024-09-21T07:34:10.675070+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4888":{"gid":4888,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/3524592265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3524592265},{"type":"v1","addr":"172.21.15.39:6834","nonce":3524592265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4888,"qdb_cluster":[4888]},"id":2}]} 2024-09-21T07:34:19.255 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 17 2024-09-21T07:34:19.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.255+0000 7f3f32000640 1 -- 172.21.15.39:0/762186716 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3f0c078bc0 msgr2=0x7f3f0c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:19.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.255+0000 7f3f32000640 1 --2- 172.21.15.39:0/762186716 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3f0c078bc0 0x7f3f0c07b0d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c15b040 tx=0x7f3f1800b2e0 comp rx=0 tx=0).stop 2024-09-21T07:34:19.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.255+0000 7f3f32000640 1 -- 172.21.15.39:0/762186716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f2c0b56f0 msgr2=0x7f3f2c160270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:19.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.255+0000 7f3f32000640 1 --2- 172.21.15.39:0/762186716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f2c0b56f0 0x7f3f2c160270 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3f2002aa50 tx=0x7f3f20033c20 comp rx=0 tx=0).stop 2024-09-21T07:34:19.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.255+0000 7f3f32000640 1 -- 172.21.15.39:0/762186716 shutdown_connections 2024-09-21T07:34:19.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.255+0000 7f3f32000640 1 --2- 172.21.15.39:0/762186716 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3f0c078bc0 0x7f3f0c07b0d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:19.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.255+0000 7f3f32000640 1 --2- 172.21.15.39:0/762186716 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3f2c0d2270 0x7f3f2c15dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:19.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.255+0000 7f3f32000640 1 --2- 172.21.15.39:0/762186716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f2c0d0ed0 0x7f3f2c1594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:19.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.255+0000 7f3f32000640 1 --2- 172.21.15.39:0/762186716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f2c0b56f0 0x7f3f2c160270 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:19.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.255+0000 7f3f32000640 1 -- 172.21.15.39:0/762186716 >> 172.21.15.39:0/762186716 conn(0x7f3f2c09d620 msgr2=0x7f3f2c15f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:19.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.255+0000 7f3f32000640 1 -- 172.21.15.39:0/762186716 shutdown_connections 2024-09-21T07:34:19.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.255+0000 7f3f32000640 1 -- 172.21.15.39:0/762186716 wait complete. 2024-09-21T07:34:19.685 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T07:34:07.651669+0000', 'modified': '2024-09-21T07:34:10.675070+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4888}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4888': {'gid': 4888, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.39:6834/3524592265', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 3524592265}, {'type': 'v1', 'addr': '172.21.15.39:6834', 'nonce': 3524592265}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4888, 'qdb_cluster': [4888]} 2024-09-21T07:34:19.685 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T07:34:19.685 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T07:34:19.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.910+0000 7f03e1400640 1 -- 172.21.15.39:0/2193158381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03dc153780 msgr2=0x7f03dc0b5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:19.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.910+0000 7f03e1400640 1 --2- 172.21.15.39:0/2193158381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03dc153780 0x7f03dc0b5460 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f03cc009830 tx=0x7f03cc02a5c0 comp rx=0 tx=0).stop 2024-09-21T07:34:19.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.910+0000 7f03e1400640 1 -- 172.21.15.39:0/2193158381 shutdown_connections 2024-09-21T07:34:19.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.910+0000 7f03e1400640 1 --2- 172.21.15.39:0/2193158381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03dc153780 0x7f03dc0b5460 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:19.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.910+0000 7f03e1400640 1 -- 172.21.15.39:0/2193158381 >> 172.21.15.39:0/2193158381 conn(0x7f03dc06d870 msgr2=0x7f03dc06ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:19.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.910+0000 7f03e1400640 1 -- 172.21.15.39:0/2193158381 shutdown_connections 2024-09-21T07:34:19.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.910+0000 7f03e1400640 1 -- 172.21.15.39:0/2193158381 wait complete. 2024-09-21T07:34:19.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.912+0000 7f03e1400640 1 Processor -- start 2024-09-21T07:34:19.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.913+0000 7f03e1400640 1 -- start start 2024-09-21T07:34:19.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.914+0000 7f03e1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03dc0b59a0 0x7f03dc1582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:19.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.914+0000 7f03e1400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03dc153780 0x7f03dc149910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:19.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.914+0000 7f03e1400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03dc15d4b0 0x7f03dc1f6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:19.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.914+0000 7f03e1400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f03dc0ccf00 con 0x7f03dc0b59a0 2024-09-21T07:34:19.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.914+0000 7f03e1400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f03dc0ccd80 con 0x7f03dc153780 2024-09-21T07:34:19.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.914+0000 7f03e1400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f03dc0cd080 con 0x7f03dc15d4b0 2024-09-21T07:34:19.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.914+0000 7f03dbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03dc0b59a0 0x7f03dc1582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:19.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.914+0000 7f03dbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03dc0b59a0 0x7f03dc1582f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41526/0 (socket says 172.21.15.39:41526) 2024-09-21T07:34:19.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.914+0000 7f03dbe00640 1 -- 172.21.15.39:0/1216786809 learned_addr learned my addr 172.21.15.39:0/1216786809 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:19.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.914+0000 7f03d3e00640 1 --2- 172.21.15.39:0/1216786809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03dc153780 0x7f03dc149910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:19.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.915+0000 7f03e0a00640 1 --2- 172.21.15.39:0/1216786809 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03dc15d4b0 0x7f03dc1f6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:19.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.915+0000 7f03dbe00640 1 -- 172.21.15.39:0/1216786809 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03dc15d4b0 msgr2=0x7f03dc1f6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:19.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.915+0000 7f03dbe00640 1 --2- 172.21.15.39:0/1216786809 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03dc15d4b0 0x7f03dc1f6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:19.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.915+0000 7f03dbe00640 1 -- 172.21.15.39:0/1216786809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03dc153780 msgr2=0x7f03dc149910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:19.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.915+0000 7f03dbe00640 1 --2- 172.21.15.39:0/1216786809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03dc153780 0x7f03dc149910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:19.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.915+0000 7f03dbe00640 1 -- 172.21.15.39:0/1216786809 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03dc1569d0 con 0x7f03dc0b59a0 2024-09-21T07:34:19.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.915+0000 7f03e0a00640 1 --2- 172.21.15.39:0/1216786809 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03dc15d4b0 0x7f03dc1f6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:34:19.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.915+0000 7f03dbe00640 1 --2- 172.21.15.39:0/1216786809 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03dc0b59a0 0x7f03dc1582f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f03cc009950 tx=0x7f03cc033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:19.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.915+0000 7f03d8c00640 1 -- 172.21.15.39:0/1216786809 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03cc031d60 con 0x7f03dc0b59a0 2024-09-21T07:34:19.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.915+0000 7f03d8c00640 1 -- 172.21.15.39:0/1216786809 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f03cc033070 con 0x7f03dc0b59a0 2024-09-21T07:34:19.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.915+0000 7f03d8c00640 1 -- 172.21.15.39:0/1216786809 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03cc0334c0 con 0x7f03dc0b59a0 2024-09-21T07:34:19.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.915+0000 7f03e1400640 1 -- 172.21.15.39:0/1216786809 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03dc159ef0 con 0x7f03dc0b59a0 2024-09-21T07:34:19.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.915+0000 7f03e1400640 1 -- 172.21.15.39:0/1216786809 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03dc1f5c20 con 0x7f03dc0b59a0 2024-09-21T07:34:19.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.918+0000 7f03d8c00640 1 -- 172.21.15.39:0/1216786809 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f03cc033660 con 0x7f03dc0b59a0 2024-09-21T07:34:19.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.919+0000 7f03d8c00640 1 --2- 172.21.15.39:0/1216786809 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f03b8078b90 0x7f03b807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:19.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.919+0000 7f03d8c00640 1 -- 172.21.15.39:0/1216786809 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f03cc0bc370 con 0x7f03dc0b59a0 2024-09-21T07:34:19.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.919+0000 7f03e99e9740 1 Processor -- start 2024-09-21T07:34:19.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.919+0000 7f03e99e9740 1 -- start start 2024-09-21T07:34:19.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.920+0000 7f03d3e00640 1 --2- 172.21.15.39:0/1216786809 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f03b8078b90 0x7f03b807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:19.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.921+0000 7f03d3e00640 1 --2- 172.21.15.39:0/1216786809 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f03b8078b90 0x7f03b807b0a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f03c4000f60 tx=0x7f03c4009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:19.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.921+0000 7f03e99e9740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559d6bce6270 0x559d6c0b6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:19.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.921+0000 7f03e99e9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d6bcbd3b0 0x559d6c0b70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:19.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.921+0000 7f03e99e9740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d6bcc5590 0x559d6c087150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:19.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.921+0000 7f03e99e9740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559d6c010430 con 0x559d6bcc5590 2024-09-21T07:34:19.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.921+0000 7f03e99e9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559d6c0005d0 con 0x559d6bcbd3b0 2024-09-21T07:34:19.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.921+0000 7f03e99e9740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x559d6bfefa00 con 0x559d6bce6270 2024-09-21T07:34:19.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.921+0000 7f03e0a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d6bcc5590 0x559d6c087150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:19.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.921+0000 7f03e0a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d6bcc5590 0x559d6c087150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41538/0 (socket says 172.21.15.39:41538) 2024-09-21T07:34:19.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.921+0000 7f03e0a00640 1 -- 172.21.15.39:0/1894638648 learned_addr learned my addr 172.21.15.39:0/1894638648 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:19.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.922+0000 7f03dbe00640 1 --2- 172.21.15.39:0/1894638648 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559d6bce6270 0x559d6c0b6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:19.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.922+0000 7f03d3e00640 1 --2- 172.21.15.39:0/1894638648 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d6bcbd3b0 0x559d6c0b70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:19.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.922+0000 7f03e0a00640 1 -- 172.21.15.39:0/1894638648 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559d6bce6270 msgr2=0x559d6c0b6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:34:19.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.922+0000 7f03e0a00640 1 --2- 172.21.15.39:0/1894638648 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559d6bce6270 0x559d6c0b6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:19.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.922+0000 7f03e0a00640 1 -- 172.21.15.39:0/1894638648 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d6bcbd3b0 msgr2=0x559d6c0b70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:34:19.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.922+0000 7f03e0a00640 1 --2- 172.21.15.39:0/1894638648 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d6bcbd3b0 0x559d6c0b70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:19.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.922+0000 7f03e0a00640 1 -- 172.21.15.39:0/1894638648 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d6bfb8520 con 0x559d6bcc5590 2024-09-21T07:34:19.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.922+0000 7f03e0a00640 1 --2- 172.21.15.39:0/1894638648 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d6bcc5590 0x559d6c087150 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f03d4001df0 tx=0x7f03d400d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a370c6a1f45275d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:19.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.922+0000 7f03cbe00640 1 -- 172.21.15.39:0/1894638648 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d4018070 con 0x559d6bcc5590 2024-09-21T07:34:19.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.922+0000 7f03cbe00640 1 -- 172.21.15.39:0/1894638648 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f03d4003990 con 0x559d6bcc5590 2024-09-21T07:34:19.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.923+0000 7f03cbe00640 1 -- 172.21.15.39:0/1894638648 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d4003da0 con 0x559d6bcc5590 2024-09-21T07:34:19.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.923+0000 7f03e99e9740 1 -- 172.21.15.39:0/1894638648 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d6bcc5590 msgr2=0x559d6c087150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:19.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.923+0000 7f03e99e9740 1 --2- 172.21.15.39:0/1894638648 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d6bcc5590 0x559d6c087150 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f03d4001df0 tx=0x7f03d400d520 comp rx=0 tx=0).stop 2024-09-21T07:34:19.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.923+0000 7f03e99e9740 1 -- 172.21.15.39:0/1894638648 shutdown_connections 2024-09-21T07:34:19.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.923+0000 7f03e99e9740 1 --2- 172.21.15.39:0/1894638648 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d6bcc5590 0x559d6c087150 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:19.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.923+0000 7f03e99e9740 1 --2- 172.21.15.39:0/1894638648 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d6bcbd3b0 0x559d6c0b70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:19.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.923+0000 7f03e99e9740 1 --2- 172.21.15.39:0/1894638648 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559d6bce6270 0x559d6c0b6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:19.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.923+0000 7f03e99e9740 1 -- 172.21.15.39:0/1894638648 >> 172.21.15.39:0/1894638648 conn(0x559d6bcda3c0 msgr2=0x559d6c08cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:19.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.923+0000 7f03e99e9740 1 -- 172.21.15.39:0/1894638648 shutdown_connections 2024-09-21T07:34:19.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.923+0000 7f03e99e9740 1 -- 172.21.15.39:0/1894638648 wait complete. 2024-09-21T07:34:19.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.924+0000 7f03e99e9740 1 -- start start 2024-09-21T07:34:19.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.925+0000 7f03e99e9740 1 Processor -- start 2024-09-21T07:34:19.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.926+0000 7f03e99e9740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d6bcbd3b0 0x559d6c0cb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:19.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.926+0000 7f03e99e9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d6bcc5590 0x559d6c0e5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:19.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.926+0000 7f03e99e9740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559d6bce6270 0x559d6c0e5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:19.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.926+0000 7f03e99e9740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559d6bf96330 con 0x559d6bcbd3b0 2024-09-21T07:34:19.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.926+0000 7f03e99e9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559d6bff3810 con 0x559d6bcc5590 2024-09-21T07:34:19.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.926+0000 7f03e99e9740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x559d6bd60620 con 0x559d6bce6270 2024-09-21T07:34:19.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.926+0000 7f03e99e9740 10 client.0 _ll_register_callbacks cb 0x559d6bf9b420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:34:19.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.926+0000 7f03dbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d6bcbd3b0 0x559d6c0cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:19.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.927+0000 7f03dbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d6bcbd3b0 0x559d6c0cb680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41544/0 (socket says 172.21.15.39:41544) 2024-09-21T07:34:19.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.927+0000 7f03dbe00640 1 -- 172.21.15.39:0/3717556866 learned_addr learned my addr 172.21.15.39:0/3717556866 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:19.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.927+0000 7f03d3e00640 1 --2- 172.21.15.39:0/3717556866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d6bcc5590 0x559d6c0e5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:19.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.927+0000 7f03dbe00640 1 -- 172.21.15.39:0/3717556866 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559d6bce6270 msgr2=0x559d6c0e5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:34:19.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.927+0000 7f03e0a00640 1 --2- 172.21.15.39:0/3717556866 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559d6bce6270 0x559d6c0e5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:19.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.927+0000 7f03dbe00640 1 --2- 172.21.15.39:0/3717556866 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559d6bce6270 0x559d6c0e5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:19.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.927+0000 7f03dbe00640 1 -- 172.21.15.39:0/3717556866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d6bcc5590 msgr2=0x559d6c0e5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:34:19.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.927+0000 7f03dbe00640 1 --2- 172.21.15.39:0/3717556866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d6bcc5590 0x559d6c0e5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:19.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.927+0000 7f03dbe00640 1 -- 172.21.15.39:0/3717556866 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d6bd51850 con 0x559d6bcbd3b0 2024-09-21T07:34:19.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.927+0000 7f03d3e00640 1 --2- 172.21.15.39:0/3717556866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d6bcc5590 0x559d6c0e5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:34:19.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.927+0000 7f03e0a00640 1 --2- 172.21.15.39:0/3717556866 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559d6bce6270 0x559d6c0e5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:34:19.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.927+0000 7f03dbe00640 1 --2- 172.21.15.39:0/3717556866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d6bcbd3b0 0x559d6c0cb680 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f03cc0ba800 tx=0x7f03cc083b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a0cd8cf5aae78b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:19.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.927+0000 7f03c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:34:19.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.927+0000 7f03c8c00640 1 -- 172.21.15.39:0/3717556866 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03cc0aa3f0 con 0x559d6bcbd3b0 2024-09-21T07:34:19.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.928+0000 7f03e99e9740 1 -- 172.21.15.39:0/3717556866 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03d4003990 con 0x559d6bcbd3b0 2024-09-21T07:34:19.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.927+0000 7f03c8c00640 1 -- 172.21.15.39:0/3717556866 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f03cc084270 con 0x559d6bcbd3b0 2024-09-21T07:34:19.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.928+0000 7f03c8c00640 1 -- 172.21.15.39:0/3717556866 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f03cc088af0 con 0x559d6bcbd3b0 2024-09-21T07:34:19.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.928+0000 7f03e99e9740 10 client.5097 fetch_fsmap learned FSMap version 17 2024-09-21T07:34:19.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.928+0000 7f03e99e9740 1 -- 172.21.15.39:0/3717556866 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x559d6bdc86b0 con 0x559d6bcbd3b0 2024-09-21T07:34:19.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.928+0000 7f03c8c00640 1 -- 172.21.15.39:0/3717556866 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f03d4003990 con 0x559d6bcbd3b0 2024-09-21T07:34:19.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.928+0000 7f03c8c00640 1 -- 172.21.15.39:0/3717556866 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7f03cc04b030 con 0x559d6bcbd3b0 2024-09-21T07:34:19.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.928+0000 7f03e99e9740 10 client.5097 fetch_fsmap finished waiting for FSMap version 17 2024-09-21T07:34:19.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.928+0000 7f03e99e9740 10 client.5097 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:34:19.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.928+0000 7f03e99e9740 20 client.5097 populate_metadata read hostname 'smithi039' 2024-09-21T07:34:19.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.928+0000 7f03e99e9740 1 --2- 172.21.15.39:0/3717556866 >> [v2:172.21.15.39:6832/3524592265,v1:172.21.15.39:6834/3524592265] conn(0x559d6bca4e40 0x559d6c0cd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:19.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.928+0000 7f03e99e9740 4 client.5097 mds_command: new command op to 4888 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:34:19.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.928+0000 7f03e99e9740 1 -- 172.21.15.39:0/3717556866 --> [v2:172.21.15.39:6832/3524592265,v1:172.21.15.39:6834/3524592265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d6c002520 con 0x559d6bca4e40 2024-09-21T07:34:19.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.928+0000 7f03d3e00640 1 --2- 172.21.15.39:0/3717556866 >> [v2:172.21.15.39:6832/3524592265,v1:172.21.15.39:6834/3524592265] conn(0x559d6bca4e40 0x559d6c0cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:19.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.929+0000 7f03d3e00640 1 --2- 172.21.15.39:0/3717556866 >> [v2:172.21.15.39:6832/3524592265,v1:172.21.15.39:6834/3524592265] conn(0x559d6bca4e40 0x559d6c0cd040 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dca330de49316ea0 server_cookie=e205686e9a04756b in_seq=0 out_seq=0 2024-09-21T07:34:19.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.929+0000 7f03c8c00640 10 client.5097 ms_handle_connect on v2:172.21.15.39:6832/3524592265 2024-09-21T07:34:19.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.930+0000 7f03c8c00640 1 -- 172.21.15.39:0/3717556866 <== mds.0 v2:172.21.15.39:6832/3524592265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559d6c002520 con 0x559d6bca4e40 2024-09-21T07:34:19.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.930+0000 7f03c8c00640 10 client.5097 handle_command_reply: tid=0 2024-09-21T07:34:19.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.930+0000 7f03e99e9740 1 client.5097 shutdown 2024-09-21T07:34:19.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.930+0000 7f03e99e9740 1 -- 172.21.15.39:0/3717556866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d6bcbd3b0 msgr2=0x559d6c0cb680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:19.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.930+0000 7f03e99e9740 1 --2- 172.21.15.39:0/3717556866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d6bcbd3b0 0x559d6c0cb680 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f03cc0ba800 tx=0x7f03cc083b20 comp rx=0 tx=0).stop 2024-09-21T07:34:19.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.930+0000 7f03e99e9740 1 -- 172.21.15.39:0/3717556866 shutdown_connections 2024-09-21T07:34:19.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.930+0000 7f03e99e9740 1 --2- 172.21.15.39:0/3717556866 >> [v2:172.21.15.39:6832/3524592265,v1:172.21.15.39:6834/3524592265] conn(0x559d6bca4e40 0x559d6c0cd040 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:19.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.930+0000 7f03e99e9740 1 --2- 172.21.15.39:0/3717556866 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559d6bce6270 0x559d6c0e5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:19.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.930+0000 7f03e99e9740 1 --2- 172.21.15.39:0/3717556866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d6bcc5590 0x559d6c0e5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:19.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.930+0000 7f03c8c00640 0 client.5097 ms_handle_reset on v2:172.21.15.39:6832/3524592265 2024-09-21T07:34:19.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.930+0000 7f03e99e9740 1 --2- 172.21.15.39:0/3717556866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d6bcbd3b0 0x559d6c0cb680 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:19.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.930+0000 7f03e99e9740 1 -- 172.21.15.39:0/3717556866 >> 172.21.15.39:0/3717556866 conn(0x559d6bcda3c0 msgr2=0x559d6c0d46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:19.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.930+0000 7f03e99e9740 1 -- 172.21.15.39:0/3717556866 shutdown_connections 2024-09-21T07:34:19.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.931+0000 7f03e99e9740 1 -- 172.21.15.39:0/3717556866 wait complete. 2024-09-21T07:34:19.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.931+0000 7f03e99e9740 20 client.5097 trim_cache size 0 max 16384 2024-09-21T07:34:19.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.950+0000 7f03e99e9740 1 Processor -- start 2024-09-21T07:34:19.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.950+0000 7f03e99e9740 1 -- start start 2024-09-21T07:34:19.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.951+0000 7f03e99e9740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559d6bcbd3b0 0x559d6c0acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:19.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.951+0000 7f03e99e9740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d6bcc5590 0x559d6c0a0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:19.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.951+0000 7f03e99e9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d6bce6270 0x559d6c0a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:19.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.951+0000 7f03e99e9740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f03dc1fde00 con 0x559d6bcc5590 2024-09-21T07:34:19.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.951+0000 7f03e99e9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f03dc1f3690 con 0x559d6bce6270 2024-09-21T07:34:19.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.951+0000 7f03e99e9740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f03dc0cf060 con 0x559d6bcbd3b0 2024-09-21T07:34:19.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.952+0000 7f03d3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d6bcc5590 0x559d6c0a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:19.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.952+0000 7f03d3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d6bcc5590 0x559d6c0a0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41554/0 (socket says 172.21.15.39:41554) 2024-09-21T07:34:19.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.952+0000 7f03d3e00640 1 -- 172.21.15.39:0/3522175761 learned_addr learned my addr 172.21.15.39:0/3522175761 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:19.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.952+0000 7f03dbe00640 1 --2- 172.21.15.39:0/3522175761 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559d6bcbd3b0 0x559d6c0acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:19.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.952+0000 7f03e0a00640 1 --2- 172.21.15.39:0/3522175761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d6bce6270 0x559d6c0a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:19.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.952+0000 7f03d3e00640 1 -- 172.21.15.39:0/3522175761 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559d6bcbd3b0 msgr2=0x559d6c0acdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:34:19.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.952+0000 7f03d3e00640 1 --2- 172.21.15.39:0/3522175761 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559d6bcbd3b0 0x559d6c0acdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:19.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.952+0000 7f03d3e00640 1 -- 172.21.15.39:0/3522175761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d6bce6270 msgr2=0x559d6c0a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:34:19.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.952+0000 7f03d3e00640 1 --2- 172.21.15.39:0/3522175761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d6bce6270 0x559d6c0a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:19.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.952+0000 7f03d3e00640 1 -- 172.21.15.39:0/3522175761 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d6c08ddc0 con 0x559d6bcc5590 2024-09-21T07:34:19.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.952+0000 7f03d3e00640 1 --2- 172.21.15.39:0/3522175761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d6bcc5590 0x559d6c0a0890 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f03c4007c20 tx=0x7f03c4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec4702c4a3296f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:19.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.953+0000 7f03cb400640 1 -- 172.21.15.39:0/3522175761 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c40154f0 con 0x559d6bcc5590 2024-09-21T07:34:19.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.953+0000 7f03cb400640 1 -- 172.21.15.39:0/3522175761 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f03c402f070 con 0x559d6bcc5590 2024-09-21T07:34:19.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.953+0000 7f03cb400640 1 -- 172.21.15.39:0/3522175761 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c40157a0 con 0x559d6bcc5590 2024-09-21T07:34:19.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.953+0000 7f03e99e9740 1 -- 172.21.15.39:0/3522175761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d6bcc5590 msgr2=0x559d6c0a0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:19.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.953+0000 7f03e99e9740 1 --2- 172.21.15.39:0/3522175761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d6bcc5590 0x559d6c0a0890 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f03c4007c20 tx=0x7f03c4014000 comp rx=0 tx=0).stop 2024-09-21T07:34:19.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.953+0000 7f03e99e9740 1 -- 172.21.15.39:0/3522175761 shutdown_connections 2024-09-21T07:34:19.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.953+0000 7f03e99e9740 1 --2- 172.21.15.39:0/3522175761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d6bce6270 0x559d6c0a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:19.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.953+0000 7f03e99e9740 1 --2- 172.21.15.39:0/3522175761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d6bcc5590 0x559d6c0a0890 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:19.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.953+0000 7f03e99e9740 1 --2- 172.21.15.39:0/3522175761 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559d6bcbd3b0 0x559d6c0acdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:19.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.953+0000 7f03e99e9740 1 -- 172.21.15.39:0/3522175761 >> 172.21.15.39:0/3522175761 conn(0x559d6bcda3c0 msgr2=0x559d6c087050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:19.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.954+0000 7f03e99e9740 1 -- 172.21.15.39:0/3522175761 shutdown_connections 2024-09-21T07:34:19.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.954+0000 7f03e99e9740 1 -- 172.21.15.39:0/3522175761 wait complete. 2024-09-21T07:34:19.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.954+0000 7f03e99e9740 1 -- start start 2024-09-21T07:34:19.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.955+0000 7f03e99e9740 1 Processor -- start 2024-09-21T07:34:19.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.956+0000 7f03e99e9740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559d6bcbd3b0 0x559d6c085e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:19.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.956+0000 7f03e99e9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d6bcc5590 0x559d6c155510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:19.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.956+0000 7f03e99e9740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d6bce6270 0x559d6c0f81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:19.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.956+0000 7f03e99e9740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559d6bf96330 con 0x559d6bce6270 2024-09-21T07:34:19.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.956+0000 7f03e99e9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f03dc0cf6b0 con 0x559d6bcc5590 2024-09-21T07:34:19.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.956+0000 7f03e99e9740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x559d6c0cc650 con 0x559d6bcbd3b0 2024-09-21T07:34:19.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.956+0000 7f03e99e9740 10 client.0 _ll_register_callbacks cb 0x7f03cc083de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:34:19.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.956+0000 7f03e0a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d6bce6270 0x559d6c0f81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:19.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.956+0000 7f03e0a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d6bce6270 0x559d6c0f81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41570/0 (socket says 172.21.15.39:41570) 2024-09-21T07:34:19.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.956+0000 7f03e0a00640 1 -- 172.21.15.39:0/3068118676 learned_addr learned my addr 172.21.15.39:0/3068118676 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:19.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.956+0000 7f03dbe00640 1 --2- 172.21.15.39:0/3068118676 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559d6bcbd3b0 0x559d6c085e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:19.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.957+0000 7f03d3e00640 1 --2- 172.21.15.39:0/3068118676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d6bcc5590 0x559d6c155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:19.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.957+0000 7f03e0a00640 1 -- 172.21.15.39:0/3068118676 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559d6bcbd3b0 msgr2=0x559d6c085e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:34:19.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.957+0000 7f03e0a00640 1 --2- 172.21.15.39:0/3068118676 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559d6bcbd3b0 0x559d6c085e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:19.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.957+0000 7f03e0a00640 1 -- 172.21.15.39:0/3068118676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d6bcc5590 msgr2=0x559d6c155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:34:19.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.957+0000 7f03e0a00640 1 --2- 172.21.15.39:0/3068118676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d6bcc5590 0x559d6c155510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:19.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.957+0000 7f03e0a00640 1 -- 172.21.15.39:0/3068118676 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d6c08d840 con 0x559d6bce6270 2024-09-21T07:34:19.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.957+0000 7f03e0a00640 1 --2- 172.21.15.39:0/3068118676 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d6bce6270 0x559d6c0f81b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f03d4001dc0 tx=0x7f03d4003ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65db085c8699f45 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:19.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.957+0000 7f03c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:34:19.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.957+0000 7f03c8c00640 1 -- 172.21.15.39:0/3068118676 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d4022030 con 0x559d6bce6270 2024-09-21T07:34:19.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.958+0000 7f03c8c00640 1 -- 172.21.15.39:0/3068118676 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f03d400dbd0 con 0x559d6bce6270 2024-09-21T07:34:19.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.958+0000 7f03e99e9740 1 -- 172.21.15.39:0/3068118676 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03c402f070 con 0x559d6bce6270 2024-09-21T07:34:19.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.958+0000 7f03c8c00640 1 -- 172.21.15.39:0/3068118676 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f03d4018040 con 0x559d6bce6270 2024-09-21T07:34:19.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.958+0000 7f03e99e9740 10 client.5103 fetch_fsmap learned FSMap version 17 2024-09-21T07:34:19.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.958+0000 7f03e99e9740 1 -- 172.21.15.39:0/3068118676 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x559d6c0addc0 con 0x559d6bce6270 2024-09-21T07:34:19.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.958+0000 7f03c8c00640 1 -- 172.21.15.39:0/3068118676 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f03c402f070 con 0x559d6bce6270 2024-09-21T07:34:19.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.958+0000 7f03c8c00640 1 -- 172.21.15.39:0/3068118676 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7f03d4006b50 con 0x559d6bce6270 2024-09-21T07:34:19.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.958+0000 7f03e99e9740 10 client.5103 fetch_fsmap finished waiting for FSMap version 17 2024-09-21T07:34:19.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.958+0000 7f03e99e9740 10 client.5103 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:34:19.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.958+0000 7f03e99e9740 20 client.5103 populate_metadata read hostname 'smithi039' 2024-09-21T07:34:19.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.958+0000 7f03e99e9740 1 --2- 172.21.15.39:0/3068118676 >> [v2:172.21.15.39:6832/3524592265,v1:172.21.15.39:6834/3524592265] conn(0x559d6c0f8da0 0x559d6c159010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:19.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.958+0000 7f03e99e9740 4 client.5103 mds_command: new command op to 4888 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:34:19.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.958+0000 7f03e99e9740 1 -- 172.21.15.39:0/3068118676 --> [v2:172.21.15.39:6832/3524592265,v1:172.21.15.39:6834/3524592265] -- command(tid 0: {"prefix": "status"}) -- 0x559d6c0d21b0 con 0x559d6c0f8da0 2024-09-21T07:34:19.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.959+0000 7f03dbe00640 1 --2- 172.21.15.39:0/3068118676 >> [v2:172.21.15.39:6832/3524592265,v1:172.21.15.39:6834/3524592265] conn(0x559d6c0f8da0 0x559d6c159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:19.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.959+0000 7f03dbe00640 1 --2- 172.21.15.39:0/3068118676 >> [v2:172.21.15.39:6832/3524592265,v1:172.21.15.39:6834/3524592265] conn(0x559d6c0f8da0 0x559d6c159010 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c789b267242f214 server_cookie=ce77dd97850df891 in_seq=0 out_seq=0 2024-09-21T07:34:19.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.959+0000 7f03c8c00640 10 client.5103 ms_handle_connect on v2:172.21.15.39:6832/3524592265 2024-09-21T07:34:19.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.959+0000 7f03c8c00640 1 -- 172.21.15.39:0/3068118676 <== mds.0 v2:172.21.15.39:6832/3524592265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x559d6c0d21b0 con 0x559d6c0f8da0 2024-09-21T07:34:19.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.959+0000 7f03c8c00640 10 client.5103 handle_command_reply: tid=0 2024-09-21T07:34:19.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.959+0000 7f03e99e9740 1 client.5103 shutdown 2024-09-21T07:34:19.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.960+0000 7f03e99e9740 1 -- 172.21.15.39:0/3068118676 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d6bce6270 msgr2=0x559d6c0f81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:19.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.960+0000 7f03e99e9740 1 --2- 172.21.15.39:0/3068118676 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d6bce6270 0x559d6c0f81b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f03d4001dc0 tx=0x7f03d4003ed0 comp rx=0 tx=0).stop 2024-09-21T07:34:19.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.960+0000 7f03e99e9740 1 -- 172.21.15.39:0/3068118676 shutdown_connections 2024-09-21T07:34:19.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.960+0000 7f03e99e9740 1 --2- 172.21.15.39:0/3068118676 >> [v2:172.21.15.39:6832/3524592265,v1:172.21.15.39:6834/3524592265] conn(0x559d6c0f8da0 0x559d6c159010 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:19.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.960+0000 7f03e99e9740 1 --2- 172.21.15.39:0/3068118676 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d6bce6270 0x559d6c0f81b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:19.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.960+0000 7f03e99e9740 1 --2- 172.21.15.39:0/3068118676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559d6bcc5590 0x559d6c155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:19.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.960+0000 7f03e99e9740 1 --2- 172.21.15.39:0/3068118676 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559d6bcbd3b0 0x559d6c085e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:19.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.960+0000 7f03e99e9740 1 -- 172.21.15.39:0/3068118676 >> 172.21.15.39:0/3068118676 conn(0x559d6bcda3c0 msgr2=0x559d6c0a0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:19.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.960+0000 7f03c8c00640 0 client.5103 ms_handle_reset on v2:172.21.15.39:6832/3524592265 2024-09-21T07:34:19.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.960+0000 7f03e99e9740 1 -- 172.21.15.39:0/3068118676 shutdown_connections 2024-09-21T07:34:19.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.960+0000 7f03e99e9740 1 -- 172.21.15.39:0/3068118676 wait complete. 2024-09-21T07:34:19.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.960+0000 7f03e99e9740 20 client.5103 trim_cache size 0 max 16384 2024-09-21T07:34:19.965 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:34:19.965 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:34:19.965 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-09-21T07:34:19.965 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 4888, 2024-09-21T07:34:19.965 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:34:19.965 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:34:19.965 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:34:19.965 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 11.804859121, 2024-09-21T07:34:19.965 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 17, 2024-09-21T07:34:19.965 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 34, 2024-09-21T07:34:19.966 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 34, 2024-09-21T07:34:19.966 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 19.699798983000001 2024-09-21T07:34:19.966 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:34:19.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.961+0000 7f03e1400640 1 -- 172.21.15.39:0/1216786809 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f03b8078b90 msgr2=0x7f03b807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:19.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.961+0000 7f03e1400640 1 --2- 172.21.15.39:0/1216786809 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f03b8078b90 0x7f03b807b0a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f03c4000f60 tx=0x7f03c4009040 comp rx=0 tx=0).stop 2024-09-21T07:34:19.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.961+0000 7f03e1400640 1 -- 172.21.15.39:0/1216786809 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03dc0b59a0 msgr2=0x7f03dc1582f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:19.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.961+0000 7f03e1400640 1 --2- 172.21.15.39:0/1216786809 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03dc0b59a0 0x7f03dc1582f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f03cc009950 tx=0x7f03cc033d60 comp rx=0 tx=0).stop 2024-09-21T07:34:19.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.961+0000 7f03e1400640 1 -- 172.21.15.39:0/1216786809 shutdown_connections 2024-09-21T07:34:19.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.961+0000 7f03e1400640 1 --2- 172.21.15.39:0/1216786809 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f03b8078b90 0x7f03b807b0a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:19.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.961+0000 7f03e1400640 1 --2- 172.21.15.39:0/1216786809 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03dc15d4b0 0x7f03dc1f6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:19.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.961+0000 7f03e1400640 1 --2- 172.21.15.39:0/1216786809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03dc153780 0x7f03dc149910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:19.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.961+0000 7f03e1400640 1 --2- 172.21.15.39:0/1216786809 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03dc0b59a0 0x7f03dc1582f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:19.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.961+0000 7f03e1400640 1 -- 172.21.15.39:0/1216786809 >> 172.21.15.39:0/1216786809 conn(0x7f03dc06d870 msgr2=0x7f03dc0ca580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:19.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.961+0000 7f03e1400640 1 -- 172.21.15.39:0/1216786809 shutdown_connections 2024-09-21T07:34:19.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:19.961+0000 7f03e1400640 1 -- 172.21.15.39:0/1216786809 wait complete. 2024-09-21T07:34:19.987 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:34:20.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.210+0000 7f7d14400640 1 -- 172.21.15.39:0/4069222339 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d0c0706c0 msgr2=0x7f7d0c072ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:20.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.210+0000 7f7d14400640 1 --2- 172.21.15.39:0/4069222339 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d0c0706c0 0x7f7d0c072ff0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c0cffa0 tx=0x7f7d0402aa00 comp rx=0 tx=0).stop 2024-09-21T07:34:20.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.210+0000 7f7d14400640 1 -- 172.21.15.39:0/4069222339 shutdown_connections 2024-09-21T07:34:20.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.210+0000 7f7d14400640 1 --2- 172.21.15.39:0/4069222339 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d0c0706c0 0x7f7d0c072ff0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:20.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.210+0000 7f7d14400640 1 --2- 172.21.15.39:0/4069222339 >> v2:172.21.15.148:3300/0 conn(0x7f7d0c0cd3e0 0x7f7d0c070180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:20.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.210+0000 7f7d14400640 1 --2- 172.21.15.39:0/4069222339 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7d0c0af780 0x7f7d0c0b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:20.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.210+0000 7f7d14400640 1 -- 172.21.15.39:0/4069222339 >> 172.21.15.39:0/4069222339 conn(0x7f7d0c06d7d0 msgr2=0x7f7d0c0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:20.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.210+0000 7f7d14400640 1 -- 172.21.15.39:0/4069222339 shutdown_connections 2024-09-21T07:34:20.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.210+0000 7f7d14400640 1 -- 172.21.15.39:0/4069222339 wait complete. 2024-09-21T07:34:20.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.213+0000 7f7d14400640 1 Processor -- start 2024-09-21T07:34:20.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.213+0000 7f7d14400640 1 -- start start 2024-09-21T07:34:20.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.214+0000 7f7d14400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d0c0706c0 0x7f7d0c1472e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:20.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.214+0000 7f7d14400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7d0c0af780 0x7f7d0c0c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:20.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.214+0000 7f7d14400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d0c0cd3e0 0x7f7d0c0c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:20.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.214+0000 7f7d14400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0c073930 con 0x7f7d0c0706c0 2024-09-21T07:34:20.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.214+0000 7f7d14400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0c0737b0 con 0x7f7d0c0cd3e0 2024-09-21T07:34:20.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.214+0000 7f7d14400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f7d0c073ab0 con 0x7f7d0c0af780 2024-09-21T07:34:20.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.214+0000 7f7d11a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d0c0706c0 0x7f7d0c1472e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:20.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.214+0000 7f7d11a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d0c0706c0 0x7f7d0c1472e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41584/0 (socket says 172.21.15.39:41584) 2024-09-21T07:34:20.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.215+0000 7f7d11a00640 1 -- 172.21.15.39:0/3169810119 learned_addr learned my addr 172.21.15.39:0/3169810119 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:20.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.215+0000 7f7d11000640 1 --2- 172.21.15.39:0/3169810119 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7d0c0af780 0x7f7d0c0c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:20.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.215+0000 7f7d11a00640 1 -- 172.21.15.39:0/3169810119 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7d0c0af780 msgr2=0x7f7d0c0c9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:20.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.215+0000 7f7d11a00640 1 --2- 172.21.15.39:0/3169810119 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7d0c0af780 0x7f7d0c0c9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:20.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.215+0000 7f7d12400640 1 --2- 172.21.15.39:0/3169810119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d0c0cd3e0 0x7f7d0c0c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:20.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.215+0000 7f7d11a00640 1 -- 172.21.15.39:0/3169810119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d0c0cd3e0 msgr2=0x7f7d0c0c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:20.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.215+0000 7f7d11a00640 1 --2- 172.21.15.39:0/3169810119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d0c0cd3e0 0x7f7d0c0c98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:20.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.215+0000 7f7d11a00640 1 -- 172.21.15.39:0/3169810119 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0c09c660 con 0x7f7d0c0706c0 2024-09-21T07:34:20.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.215+0000 7f7d11000640 1 --2- 172.21.15.39:0/3169810119 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7d0c0af780 0x7f7d0c0c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:34:20.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.215+0000 7f7d11a00640 1 --2- 172.21.15.39:0/3169810119 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d0c0706c0 0x7f7d0c1472e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000d6d0 tx=0x7f7d0000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:20.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.215+0000 7f7d09600640 1 -- 172.21.15.39:0/3169810119 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d00017070 con 0x7f7d0c0706c0 2024-09-21T07:34:20.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.216+0000 7f7d09600640 1 -- 172.21.15.39:0/3169810119 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d00005330 con 0x7f7d0c0706c0 2024-09-21T07:34:20.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.216+0000 7f7d09600640 1 -- 172.21.15.39:0/3169810119 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d00009820 con 0x7f7d0c0706c0 2024-09-21T07:34:20.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.216+0000 7f7d14400640 1 -- 172.21.15.39:0/3169810119 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d0c09c8d0 con 0x7f7d0c0706c0 2024-09-21T07:34:20.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.216+0000 7f7d14400640 1 -- 172.21.15.39:0/3169810119 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d0c147820 con 0x7f7d0c0706c0 2024-09-21T07:34:20.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.217+0000 7f7d14400640 1 -- 172.21.15.39:0/3169810119 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d0c15eaa0 con 0x7f7d0c0706c0 2024-09-21T07:34:20.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.218+0000 7f7d09600640 1 -- 172.21.15.39:0/3169810119 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7d00009bd0 con 0x7f7d0c0706c0 2024-09-21T07:34:20.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.219+0000 7f7d09600640 1 --2- 172.21.15.39:0/3169810119 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7cec078bc0 0x7f7cec07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:20.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.219+0000 7f7d09600640 1 -- 172.21.15.39:0/3169810119 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f7d0009afb0 con 0x7f7d0c0706c0 2024-09-21T07:34:20.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.219+0000 7f7d11000640 1 --2- 172.21.15.39:0/3169810119 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7cec078bc0 0x7f7cec07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:20.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.220+0000 7f7d11000640 1 --2- 172.21.15.39:0/3169810119 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7cec078bc0 0x7f7cec07b0d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c1f90f0 tx=0x7f7cf8005d50 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:20.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.223+0000 7f7d09600640 1 -- 172.21.15.39:0/3169810119 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7d00063480 con 0x7f7d0c0706c0 2024-09-21T07:34:20.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.465+0000 7f7d14400640 1 -- 172.21.15.39:0/3169810119 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7d0c15e7c0 con 0x7f7d0c0706c0 2024-09-21T07:34:20.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.466+0000 7f7d09600640 1 -- 172.21.15.39:0/3169810119 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f7d00062d80 con 0x7f7d0c0706c0 2024-09-21T07:34:20.468 INFO:teuthology.orchestra.run.smithi039.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:34:20.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.468+0000 7f7d14400640 1 -- 172.21.15.39:0/3169810119 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7cec078bc0 msgr2=0x7f7cec07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:20.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.468+0000 7f7d14400640 1 --2- 172.21.15.39:0/3169810119 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7cec078bc0 0x7f7cec07b0d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c1f90f0 tx=0x7f7cf8005d50 comp rx=0 tx=0).stop 2024-09-21T07:34:20.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.468+0000 7f7d14400640 1 -- 172.21.15.39:0/3169810119 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d0c0706c0 msgr2=0x7f7d0c1472e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:20.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.468+0000 7f7d14400640 1 --2- 172.21.15.39:0/3169810119 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d0c0706c0 0x7f7d0c1472e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000d6d0 tx=0x7f7d0000db90 comp rx=0 tx=0).stop 2024-09-21T07:34:20.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.468+0000 7f7d14400640 1 -- 172.21.15.39:0/3169810119 shutdown_connections 2024-09-21T07:34:20.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.468+0000 7f7d14400640 1 --2- 172.21.15.39:0/3169810119 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7cec078bc0 0x7f7cec07b0d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:20.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.468+0000 7f7d14400640 1 --2- 172.21.15.39:0/3169810119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d0c0cd3e0 0x7f7d0c0c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:20.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.468+0000 7f7d14400640 1 --2- 172.21.15.39:0/3169810119 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7d0c0af780 0x7f7d0c0c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:20.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.468+0000 7f7d14400640 1 --2- 172.21.15.39:0/3169810119 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d0c0706c0 0x7f7d0c1472e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:20.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.468+0000 7f7d14400640 1 -- 172.21.15.39:0/3169810119 >> 172.21.15.39:0/3169810119 conn(0x7f7d0c06d7d0 msgr2=0x7f7d0c156f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:20.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.469+0000 7f7d14400640 1 -- 172.21.15.39:0/3169810119 shutdown_connections 2024-09-21T07:34:20.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20.469+0000 7f7d14400640 1 -- 172.21.15.39:0/3169810119 wait complete. 2024-09-21T07:34:20.492 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:34:20.492 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:34:20.492 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi039.front.sepia.ceph.com 2024-09-21T07:34:20.492 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T07:34:20.492 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:34:20.493 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:34:20.493 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:34:20.493 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:34:20.493 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:34:20.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:34:20.493 DEBUG:teuthology.orchestra.run.smithi039:> ip addr 2024-09-21T07:34:20.529 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:34:20.529 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:34:20.529 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:34:20.529 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:34:20.529 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2024-09-21T07:34:20.529 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:34:20.529 INFO:teuthology.orchestra.run.smithi039.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:34:20.529 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:34:20.529 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f0 2024-09-21T07:34:20.529 INFO:teuthology.orchestra.run.smithi039.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:34:20.530 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c7 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:34:20.530 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f1 2024-09-21T07:34:20.530 INFO:teuthology.orchestra.run.smithi039.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:34:20.530 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:70 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:34:20.530 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f0 2024-09-21T07:34:20.530 INFO:teuthology.orchestra.run.smithi039.stdout: inet 172.21.15.39/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T07:34:20.530 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft 42658sec preferred_lft 42658sec 2024-09-21T07:34:20.530 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec4:7aff:febd:1370/64 scope link 2024-09-21T07:34:20.530 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:34:20.530 INFO:teuthology.orchestra.run.smithi039.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:34:20.530 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:71 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:34:20.530 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f1 2024-09-21T07:34:20.530 INFO:teuthology.orchestra.run.smithi039.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:34:20.530 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff 2024-09-21T07:34:20.531 INFO:teuthology.orchestra.run.smithi039.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:34:20.531 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:34:20.531 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::b079:ddff:fe0e:6333/64 scope link 2024-09-21T07:34:20.531 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:34:20.531 INFO:teuthology.orchestra.run.smithi039.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:34:20.531 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:34:20.531 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::2ca7:98ff:fe8a:d0ed/64 scope link 2024-09-21T07:34:20.531 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:34:20.531 INFO:teuthology.orchestra.run.smithi039.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:34:20.531 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether ee:10:53:01:0c:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:34:20.531 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec10:53ff:fe01:c44/64 scope link 2024-09-21T07:34:20.531 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:34:20.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:34:20.532 DEBUG:teuthology.orchestra.run.smithi039:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:34:20.623 INFO:teuthology.orchestra.run.smithi039.stdout:1 2024-09-21T07:34:20.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:34:20.626 DEBUG:teuthology.orchestra.run.smithi039:> route 2024-09-21T07:34:20.662 INFO:teuthology.orchestra.run.smithi039.stdout:Kernel IP routing table 2024-09-21T07:34:20.662 INFO:teuthology.orchestra.run.smithi039.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:34:20.662 INFO:teuthology.orchestra.run.smithi039.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T07:34:20.662 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T07:34:20.662 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:34:20.663 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:34:20.663 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-21T07:34:20.663 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T07:34:20.663 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T07:34:20.663 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T07:34:20.663 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-21T07:34:20.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:34:20.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:34:20.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:34:20.861 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-09-21T07:34:20.897 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:34:20.897 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T07:34:20.897 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:34:20.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:34:20.898 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-09-21T07:34:20.994 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:34:20.994 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T07:34:21.030 INFO:teuthology.orchestra.run.smithi039.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:34:21.030 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:34:21.030 DEBUG:teuthology.orchestra.run.smithi039:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:34:21.105 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-21T07:34:21.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:34:21.105 DEBUG:teuthology.orchestra.run.smithi039:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:34:21.303 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:34:21.303 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:34:21.338 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:34:21.338 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:34:21.338 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: resolved to: "172.21.15.39:3300,172.21.15.148:3300,172.21.15.148:3301" 2024-09-21T07:34:21.338 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: trying mount with new device syntax: 0@7ab9b04a-f88d-49aa-a493-e5c076637ffb.cephfs=/ 2024-09-21T07:34:21.339 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.39:3300/172.21.15.148:3300/172.21.15.148:3301" will pass to kernel 2024-09-21T07:34:21.340 INFO:teuthology.orchestra.run.smithi039.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T07:34:21.340 INFO:teuthology.orchestra.run.smithi039.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:34:21.340 INFO:teuthology.orchestra.run.smithi039.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:34:21.340 INFO:teuthology.orchestra.run.smithi039.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:34:21.340 INFO:teuthology.orchestra.run.smithi039.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:34:21.343 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:34:21.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:34:21.343 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:34:21.400 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:34:21.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:34:21.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:34:21.498 INFO:teuthology.orchestra.run.smithi039.stdout:client5047 2024-09-21T07:34:21.498 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:34:21.499 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:34:21.534 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create snap-test-pool 2024-09-21T07:34:21.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:21.804+0000 7f4ff5800640 1 -- 172.21.15.39:0/1344398203 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4ff0073e50 msgr2=0x7f4ff0074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:21.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:21.804+0000 7f4ff5800640 1 --2- 172.21.15.39:0/1344398203 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4ff0073e50 0x7f4ff0074260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8009be0 tx=0x7f4fe802a890 comp rx=0 tx=0).stop 2024-09-21T07:34:21.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:21.804+0000 7f4ff5800640 1 -- 172.21.15.39:0/1344398203 shutdown_connections 2024-09-21T07:34:21.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:21.804+0000 7f4ff5800640 1 --2- 172.21.15.39:0/1344398203 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4ff0073e50 0x7f4ff0074260 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:21.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:21.804+0000 7f4ff5800640 1 -- 172.21.15.39:0/1344398203 >> 172.21.15.39:0/1344398203 conn(0x7f4ff00d15f0 msgr2=0x7f4ff0073740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:21.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:21.804+0000 7f4ff5800640 1 -- 172.21.15.39:0/1344398203 shutdown_connections 2024-09-21T07:34:21.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:21.805+0000 7f4ff5800640 1 -- 172.21.15.39:0/1344398203 wait complete. 2024-09-21T07:34:21.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:21.807+0000 7f4ff5800640 1 Processor -- start 2024-09-21T07:34:21.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:21.807+0000 7f4ff5800640 1 -- start start 2024-09-21T07:34:21.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:21.808+0000 7f4ff5800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4ff0073e50 0x7f4ff014dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:21.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:21.808+0000 7f4ff5800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ff00cfc00 0x7f4ff01f8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:21.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:21.808+0000 7f4ff5800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ff0153970 0x7f4ff01f92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:21.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:21.808+0000 7f4ff5800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff015e550 con 0x7f4ff0153970 2024-09-21T07:34:21.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:21.808+0000 7f4ff5800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff015e3d0 con 0x7f4ff00cfc00 2024-09-21T07:34:21.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:21.808+0000 7f4ff5800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f4ff015e6d0 con 0x7f4ff0073e50 2024-09-21T07:34:21.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:21.809+0000 7f4fef400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ff0153970 0x7f4ff01f92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:21.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:21.809+0000 7f4fef400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ff0153970 0x7f4ff01f92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41598/0 (socket says 172.21.15.39:41598) 2024-09-21T07:34:21.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:21.809+0000 7f4fef400640 1 -- 172.21.15.39:0/1675435532 learned_addr learned my addr 172.21.15.39:0/1675435532 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:21.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:21.809+0000 7f4fe6000640 1 --2- 172.21.15.39:0/1675435532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ff00cfc00 0x7f4ff01f8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:21.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:21.809+0000 7f4feea00640 1 --2- 172.21.15.39:0/1675435532 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4ff0073e50 0x7f4ff014dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:21.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:21.809+0000 7f4fef400640 1 -- 172.21.15.39:0/1675435532 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4ff0073e50 msgr2=0x7f4ff014dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:21.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:21.809+0000 7f4fef400640 1 --2- 172.21.15.39:0/1675435532 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4ff0073e50 0x7f4ff014dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:21.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:21.809+0000 7f4fef400640 1 -- 172.21.15.39:0/1675435532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ff00cfc00 msgr2=0x7f4ff01f8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:21.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:21.809+0000 7f4fef400640 1 --2- 172.21.15.39:0/1675435532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ff00cfc00 0x7f4ff01f8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:21.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:21.809+0000 7f4fef400640 1 -- 172.21.15.39:0/1675435532 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ff01f9be0 con 0x7f4ff0153970 2024-09-21T07:34:21.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:21.809+0000 7f4fef400640 1 --2- 172.21.15.39:0/1675435532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ff0153970 0x7f4ff01f92a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8009be0 tx=0x7f4fe8004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:21.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:21.810+0000 7f4fe7400640 1 -- 172.21.15.39:0/1675435532 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8041020 con 0x7f4ff0153970 2024-09-21T07:34:21.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:21.810+0000 7f4fe7400640 1 -- 172.21.15.39:0/1675435532 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4fe8004860 con 0x7f4ff0153970 2024-09-21T07:34:21.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:21.810+0000 7f4ff5800640 1 -- 172.21.15.39:0/1675435532 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ff00b4870 con 0x7f4ff0153970 2024-09-21T07:34:21.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:21.810+0000 7f4fe7400640 1 -- 172.21.15.39:0/1675435532 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8037040 con 0x7f4ff0153970 2024-09-21T07:34:21.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:21.810+0000 7f4ff5800640 1 -- 172.21.15.39:0/1675435532 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ff00b4e30 con 0x7f4ff0153970 2024-09-21T07:34:21.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:21.811+0000 7f4ff5800640 1 -- 172.21.15.39:0/1675435532 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ff006f900 con 0x7f4ff0153970 2024-09-21T07:34:21.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:21.812+0000 7f4fe7400640 1 -- 172.21.15.39:0/1675435532 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4fe8044020 con 0x7f4ff0153970 2024-09-21T07:34:21.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:21.813+0000 7f4fe7400640 1 --2- 172.21.15.39:0/1675435532 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4fcc078b90 0x7f4fcc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:21.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:21.813+0000 7f4fe7400640 1 -- 172.21.15.39:0/1675435532 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f4fe80bb5e0 con 0x7f4ff0153970 2024-09-21T07:34:21.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:21.814+0000 7f4feea00640 1 --2- 172.21.15.39:0/1675435532 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4fcc078b90 0x7f4fcc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:21.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:21.815+0000 7f4feea00640 1 --2- 172.21.15.39:0/1675435532 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4fcc078b90 0x7f4fcc07b0a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0000f60 tx=0x7f4fe0009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:21.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:21.818+0000 7f4fe7400640 1 -- 172.21.15.39:0/1675435532 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4fe804b030 con 0x7f4ff0153970 2024-09-21T07:34:22.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.019+0000 7f4ff5800640 1 -- 172.21.15.39:0/1675435532 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "snap-test-pool"} v 0) -- 0x7f4ff0074260 con 0x7f4ff0153970 2024-09-21T07:34:22.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.248+0000 7f4fe7400640 1 -- 172.21.15.39:0/1675435532 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "snap-test-pool"}]=0 pool 'snap-test-pool' created v35) ==== 118+0+0 (secure 0 0 0) 0x7f4fe80833d0 con 0x7f4ff0153970 2024-09-21T07:34:22.251 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'snap-test-pool' created 2024-09-21T07:34:22.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.251+0000 7f4ff5800640 1 -- 172.21.15.39:0/1675435532 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4fcc078b90 msgr2=0x7f4fcc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:22.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.251+0000 7f4ff5800640 1 --2- 172.21.15.39:0/1675435532 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4fcc078b90 0x7f4fcc07b0a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0000f60 tx=0x7f4fe0009040 comp rx=0 tx=0).stop 2024-09-21T07:34:22.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.251+0000 7f4ff5800640 1 -- 172.21.15.39:0/1675435532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ff0153970 msgr2=0x7f4ff01f92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:22.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.251+0000 7f4ff5800640 1 --2- 172.21.15.39:0/1675435532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ff0153970 0x7f4ff01f92a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8009be0 tx=0x7f4fe8004360 comp rx=0 tx=0).stop 2024-09-21T07:34:22.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.251+0000 7f4ff5800640 1 -- 172.21.15.39:0/1675435532 shutdown_connections 2024-09-21T07:34:22.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.251+0000 7f4ff5800640 1 --2- 172.21.15.39:0/1675435532 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4fcc078b90 0x7f4fcc07b0a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:22.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.251+0000 7f4ff5800640 1 --2- 172.21.15.39:0/1675435532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ff0153970 0x7f4ff01f92a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:22.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.251+0000 7f4ff5800640 1 --2- 172.21.15.39:0/1675435532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ff00cfc00 0x7f4ff01f8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:22.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.251+0000 7f4ff5800640 1 --2- 172.21.15.39:0/1675435532 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4ff0073e50 0x7f4ff014dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:22.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.251+0000 7f4ff5800640 1 -- 172.21.15.39:0/1675435532 >> 172.21.15.39:0/1675435532 conn(0x7f4ff00d15f0 msgr2=0x7f4ff00ce0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:22.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.252+0000 7f4ff5800640 1 -- 172.21.15.39:0/1675435532 shutdown_connections 2024-09-21T07:34:22.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.252+0000 7f4ff5800640 1 -- 172.21.15.39:0/1675435532 wait complete. 2024-09-21T07:34:22.285 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool snap-test-pool mksnap snap3 2024-09-21T07:34:22.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.425+0000 7f7daa7b6f40 1 Processor -- start 2024-09-21T07:34:22.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.425+0000 7f7daa7b6f40 1 -- start start 2024-09-21T07:34:22.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.426+0000 7f7daa7b6f40 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564b49b30cf0 0x564b49c8e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:22.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.426+0000 7f7daa7b6f40 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b49b74540 0x564b49c047e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:22.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.426+0000 7f7daa7b6f40 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b49b79600 0x564b49c04d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:22.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.426+0000 7f7daa7b6f40 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564b49add1d0 con 0x564b49b74540 2024-09-21T07:34:22.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.426+0000 7f7daa7b6f40 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x564b49ae9f90 con 0x564b49b79600 2024-09-21T07:34:22.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.426+0000 7f7daa7b6f40 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x564b49a7ec60 con 0x564b49b30cf0 2024-09-21T07:34:22.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.426+0000 7f7da3000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b49b74540 0x564b49c047e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:22.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.427+0000 7f7da3000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b49b74540 0x564b49c047e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52622/0 (socket says 172.21.15.39:52622) 2024-09-21T07:34:22.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.427+0000 7f7da3000640 1 -- 172.21.15.39:0/4211589526 learned_addr learned my addr 172.21.15.39:0/4211589526 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:22.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.427+0000 7f7da3a00640 1 --2- 172.21.15.39:0/4211589526 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564b49b30cf0 0x564b49c8e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:22.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.427+0000 7f7da4400640 1 --2- 172.21.15.39:0/4211589526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b49b79600 0x564b49c04d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:22.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.427+0000 7f7da3000640 1 -- 172.21.15.39:0/4211589526 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564b49b30cf0 msgr2=0x564b49c8e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:34:22.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.427+0000 7f7da3000640 1 --2- 172.21.15.39:0/4211589526 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564b49b30cf0 0x564b49c8e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:22.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.427+0000 7f7da3000640 1 -- 172.21.15.39:0/4211589526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b49b79600 msgr2=0x564b49c04d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:34:22.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.427+0000 7f7da3000640 1 --2- 172.21.15.39:0/4211589526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b49b79600 0x564b49c04d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:22.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.427+0000 7f7da3000640 1 -- 172.21.15.39:0/4211589526 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b49c06780 con 0x564b49b74540 2024-09-21T07:34:22.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.427+0000 7f7da3000640 1 --2- 172.21.15.39:0/4211589526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b49b74540 0x564b49c047e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7d90001e10 tx=0x7f7d9000b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc7e73c731f8a62 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:22.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.428+0000 7f7da1c00640 1 -- 172.21.15.39:0/4211589526 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d90016070 con 0x564b49b74540 2024-09-21T07:34:22.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.428+0000 7f7da1c00640 1 -- 172.21.15.39:0/4211589526 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d9000d7c0 con 0x564b49b74540 2024-09-21T07:34:22.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.428+0000 7f7da1c00640 1 -- 172.21.15.39:0/4211589526 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9000dc10 con 0x564b49b74540 2024-09-21T07:34:22.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.428+0000 7f7daa7b6f40 1 -- 172.21.15.39:0/4211589526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b49b74540 msgr2=0x564b49c047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:22.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.428+0000 7f7daa7b6f40 1 --2- 172.21.15.39:0/4211589526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b49b74540 0x564b49c047e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7d90001e10 tx=0x7f7d9000b650 comp rx=0 tx=0).stop 2024-09-21T07:34:22.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.428+0000 7f7daa7b6f40 1 -- 172.21.15.39:0/4211589526 shutdown_connections 2024-09-21T07:34:22.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.428+0000 7f7daa7b6f40 1 --2- 172.21.15.39:0/4211589526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b49b79600 0x564b49c04d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:22.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.428+0000 7f7daa7b6f40 1 --2- 172.21.15.39:0/4211589526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b49b74540 0x564b49c047e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:22.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.428+0000 7f7daa7b6f40 1 --2- 172.21.15.39:0/4211589526 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564b49b30cf0 0x564b49c8e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:22.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.428+0000 7f7daa7b6f40 1 -- 172.21.15.39:0/4211589526 >> 172.21.15.39:0/4211589526 conn(0x564b49a4fac0 msgr2=0x564b49c03c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:22.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.429+0000 7f7daa7b6f40 1 -- 172.21.15.39:0/4211589526 shutdown_connections 2024-09-21T07:34:22.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.429+0000 7f7daa7b6f40 1 -- 172.21.15.39:0/4211589526 wait complete. 2024-09-21T07:34:22.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.430+0000 7f7daa7b6f40 1 Processor -- start 2024-09-21T07:34:22.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.431+0000 7f7daa7b6f40 1 -- start start 2024-09-21T07:34:22.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.433+0000 7f7daa7b6f40 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b49b30cf0 0x564b49c9deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:22.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.433+0000 7f7daa7b6f40 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b49b74540 0x564b49bf8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:22.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.433+0000 7f7daa7b6f40 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564b49b79600 0x564b49c03a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:22.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.433+0000 7f7daa7b6f40 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564b49c01cd0 con 0x564b49b74540 2024-09-21T07:34:22.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.433+0000 7f7daa7b6f40 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x564b49b76eb0 con 0x564b49b30cf0 2024-09-21T07:34:22.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.433+0000 7f7daa7b6f40 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x564b49bf9420 con 0x564b49b79600 2024-09-21T07:34:22.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.433+0000 7f7da3000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b49b74540 0x564b49bf8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:22.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.433+0000 7f7da3000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b49b74540 0x564b49bf8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52634/0 (socket says 172.21.15.39:52634) 2024-09-21T07:34:22.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.433+0000 7f7da3000640 1 -- 172.21.15.39:0/2913644482 learned_addr learned my addr 172.21.15.39:0/2913644482 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:22.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.434+0000 7f7da3a00640 1 --2- 172.21.15.39:0/2913644482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b49b30cf0 0x564b49c9deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:22.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.434+0000 7f7da4400640 1 --2- 172.21.15.39:0/2913644482 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564b49b79600 0x564b49c03a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:22.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.434+0000 7f7da3000640 1 -- 172.21.15.39:0/2913644482 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564b49b79600 msgr2=0x564b49c03a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:22.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.434+0000 7f7da3000640 1 --2- 172.21.15.39:0/2913644482 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564b49b79600 0x564b49c03a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:22.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.434+0000 7f7da3000640 1 -- 172.21.15.39:0/2913644482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b49b30cf0 msgr2=0x564b49c9deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:22.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.434+0000 7f7da3000640 1 --2- 172.21.15.39:0/2913644482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b49b30cf0 0x564b49c9deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:22.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.434+0000 7f7da3000640 1 -- 172.21.15.39:0/2913644482 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b49bfa980 con 0x564b49b74540 2024-09-21T07:34:22.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.434+0000 7f7da3000640 1 --2- 172.21.15.39:0/2913644482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b49b74540 0x564b49bf8700 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7d90000c00 tx=0x7f7d9000dec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:22.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.435+0000 7f7d9be00640 1 -- 172.21.15.39:0/2913644482 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d90022070 con 0x564b49b74540 2024-09-21T07:34:22.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.435+0000 7f7d9be00640 1 -- 172.21.15.39:0/2913644482 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d90017040 con 0x564b49b74540 2024-09-21T07:34:22.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.435+0000 7f7daa7b6f40 1 -- 172.21.15.39:0/2913644482 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564b49bfac60 con 0x564b49b74540 2024-09-21T07:34:22.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.435+0000 7f7d9be00640 1 -- 172.21.15.39:0/2913644482 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d90016070 con 0x564b49b74540 2024-09-21T07:34:22.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.435+0000 7f7daa7b6f40 1 -- 172.21.15.39:0/2913644482 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b49bf9160 con 0x564b49b74540 2024-09-21T07:34:22.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.439+0000 7f7d9be00640 1 -- 172.21.15.39:0/2913644482 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7d90025070 con 0x564b49b74540 2024-09-21T07:34:22.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.440+0000 7f7d9be00640 1 --2- 172.21.15.39:0/2913644482 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7d84077570 0x7f7d84079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:22.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.440+0000 7f7d9be00640 1 -- 172.21.15.39:0/2913644482 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 6062+0+0 (secure 0 0 0) 0x7f7d9009a4d0 con 0x564b49b74540 2024-09-21T07:34:22.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.440+0000 7f7daa7b6f40 1 -- 172.21.15.39:0/2913644482 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- pool_op(create snap pool 6 tid 1 name snap3 v0) -- 0x564b49adfb20 con 0x564b49b74540 2024-09-21T07:34:22.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.440+0000 7f7da3a00640 1 --2- 172.21.15.39:0/2913644482 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7d84077570 0x7f7d84079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:22.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:22.441+0000 7f7da3a00640 1 --2- 172.21.15.39:0/2913644482 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7d84077570 0x7f7d84079a80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7d9400b6a0 tx=0x7f7d9400d040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:23.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.254+0000 7f7d9be00640 1 -- 172.21.15.39:0/2913644482 <== mon.0 v2:172.21.15.39:3300/0 6 ==== pool_op_reply(tid 1 (0) Success v36) ==== 43+0+0 (secure 0 0 0) 0x7f7d900621e0 con 0x564b49b74540 2024-09-21T07:34:23.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.254+0000 7f7d9be00640 1 -- 172.21.15.39:0/2913644482 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=36}) -- 0x7f7d8407fe00 con 0x564b49b74540 2024-09-21T07:34:23.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.256+0000 7f7d9be00640 1 -- 172.21.15.39:0/2913644482 <== mon.0 v2:172.21.15.39:3300/0 7 ==== osd_map(36..36 src has 1..36) ==== 710+0+0 (secure 0 0 0) 0x7f7d900690d0 con 0x564b49b74540 2024-09-21T07:34:23.258 INFO:teuthology.orchestra.run.smithi039.stdout:created pool snap-test-pool snap snap3 2024-09-21T07:34:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.256+0000 7f7daa7b6f40 1 -- 172.21.15.39:0/2913644482 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7d84077570 msgr2=0x7f7d84079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.256+0000 7f7daa7b6f40 1 --2- 172.21.15.39:0/2913644482 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7d84077570 0x7f7d84079a80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7d9400b6a0 tx=0x7f7d9400d040 comp rx=0 tx=0).stop 2024-09-21T07:34:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.257+0000 7f7daa7b6f40 1 -- 172.21.15.39:0/2913644482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b49b74540 msgr2=0x564b49bf8700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.257+0000 7f7daa7b6f40 1 --2- 172.21.15.39:0/2913644482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b49b74540 0x564b49bf8700 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7d90000c00 tx=0x7f7d9000dec0 comp rx=0 tx=0).stop 2024-09-21T07:34:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.257+0000 7f7daa7b6f40 1 -- 172.21.15.39:0/2913644482 shutdown_connections 2024-09-21T07:34:23.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.257+0000 7f7daa7b6f40 1 --2- 172.21.15.39:0/2913644482 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7d84077570 0x7f7d84079a80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:23.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.257+0000 7f7daa7b6f40 1 --2- 172.21.15.39:0/2913644482 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564b49b79600 0x564b49c03a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:23.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.257+0000 7f7daa7b6f40 1 --2- 172.21.15.39:0/2913644482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b49b74540 0x564b49bf8700 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:23.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.257+0000 7f7daa7b6f40 1 --2- 172.21.15.39:0/2913644482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564b49b30cf0 0x564b49c9deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:23.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.257+0000 7f7daa7b6f40 1 -- 172.21.15.39:0/2913644482 >> 172.21.15.39:0/2913644482 conn(0x564b49a4fac0 msgr2=0x564b49c04e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:23.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.257+0000 7f7daa7b6f40 1 -- 172.21.15.39:0/2913644482 shutdown_connections 2024-09-21T07:34:23.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.257+0000 7f7daa7b6f40 1 -- 172.21.15.39:0/2913644482 wait complete. 2024-09-21T07:34:23.277 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs snap-test-pool 2024-09-21T07:34:23.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.507+0000 7fd813200640 1 -- 172.21.15.39:0/3780664684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7f8004800 msgr2=0x7fd7f8004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:23.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.507+0000 7fd813200640 1 --2- 172.21.15.39:0/3780664684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7f8004800 0x7fd7f8004c10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00c260 tx=0x7fd7f8005830 comp rx=0 tx=0).stop 2024-09-21T07:34:23.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.507+0000 7fd812800640 1 -- 172.21.15.39:0/3780664684 reap_dead start 2024-09-21T07:34:23.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.507+0000 7fd813200640 1 -- 172.21.15.39:0/3780664684 shutdown_connections 2024-09-21T07:34:23.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.507+0000 7fd813200640 1 -- 172.21.15.39:0/3780664684 >> 172.21.15.39:0/3780664684 conn(0x7fd81406d410 msgr2=0x7fd8140b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:23.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.507+0000 7fd813200640 1 -- 172.21.15.39:0/3780664684 shutdown_connections 2024-09-21T07:34:23.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.507+0000 7fd813200640 1 -- 172.21.15.39:0/3780664684 wait complete. 2024-09-21T07:34:23.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.509+0000 7fd813200640 1 Processor -- start 2024-09-21T07:34:23.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.509+0000 7fd813200640 1 -- start start 2024-09-21T07:34:23.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.511+0000 7fd813200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7f8004800 0x7fd81413f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:23.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.511+0000 7fd813200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd8141570d0 0x7fd8141e8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:23.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.511+0000 7fd813200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8141f51c0 0x7fd8141e93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:23.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.511+0000 7fd813200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd8140cfb60 con 0x7fd8141f51c0 2024-09-21T07:34:23.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.511+0000 7fd813200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd81406f7f0 con 0x7fd7f8004800 2024-09-21T07:34:23.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.511+0000 7fd813200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd8141f5930 con 0x7fd8141570d0 2024-09-21T07:34:23.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.511+0000 7fd812800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8141f51c0 0x7fd8141e93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:23.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.511+0000 7fd812800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8141f51c0 0x7fd8141e93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52664/0 (socket says 172.21.15.39:52664) 2024-09-21T07:34:23.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.511+0000 7fd811e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7f8004800 0x7fd81413f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:23.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.511+0000 7fd812800640 1 -- 172.21.15.39:0/3040819916 learned_addr learned my addr 172.21.15.39:0/3040819916 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:23.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.511+0000 7fd80be00640 1 --2- 172.21.15.39:0/3040819916 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd8141570d0 0x7fd8141e8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:23.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.512+0000 7fd812800640 1 -- 172.21.15.39:0/3040819916 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd8141570d0 msgr2=0x7fd8141e8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:23.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.512+0000 7fd812800640 1 --2- 172.21.15.39:0/3040819916 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd8141570d0 0x7fd8141e8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:23.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.512+0000 7fd812800640 1 -- 172.21.15.39:0/3040819916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7f8004800 msgr2=0x7fd81413f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:23.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.512+0000 7fd812800640 1 --2- 172.21.15.39:0/3040819916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7f8004800 0x7fd81413f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:23.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.512+0000 7fd812800640 1 -- 172.21.15.39:0/3040819916 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8140b4570 con 0x7fd8141f51c0 2024-09-21T07:34:23.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.512+0000 7fd812800640 1 --2- 172.21.15.39:0/3040819916 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8141f51c0 0x7fd8141e93d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd81415a5b0 tx=0x7fd80c00f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:23.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.512+0000 7fd80a000640 1 -- 172.21.15.39:0/3040819916 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80c003970 con 0x7fd8141f51c0 2024-09-21T07:34:23.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.512+0000 7fd80a000640 1 -- 172.21.15.39:0/3040819916 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd80c003b10 con 0x7fd8141f51c0 2024-09-21T07:34:23.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.512+0000 7fd80a000640 1 -- 172.21.15.39:0/3040819916 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80c004030 con 0x7fd8141f51c0 2024-09-21T07:34:23.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.512+0000 7fd813200640 1 -- 172.21.15.39:0/3040819916 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8140b4850 con 0x7fd8141f51c0 2024-09-21T07:34:23.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.512+0000 7fd813200640 1 -- 172.21.15.39:0/3040819916 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8140b4e10 con 0x7fd8141f51c0 2024-09-21T07:34:23.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.513+0000 7fd813200640 1 -- 172.21.15.39:0/3040819916 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8140cc830 con 0x7fd8141f51c0 2024-09-21T07:34:23.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.515+0000 7fd80a000640 1 -- 172.21.15.39:0/3040819916 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd80c0041d0 con 0x7fd8141f51c0 2024-09-21T07:34:23.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.516+0000 7fd80a000640 1 --2- 172.21.15.39:0/3040819916 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd7f0078b90 0x7fd7f007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:23.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.516+0000 7fd80a000640 1 -- 172.21.15.39:0/3040819916 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7fd80c09b370 con 0x7fd8141f51c0 2024-09-21T07:34:23.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.516+0000 7fd811e00640 1 --2- 172.21.15.39:0/3040819916 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd7f0078b90 0x7fd7f007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.517+0000 7fd811e00640 1 --2- 172.21.15.39:0/3040819916 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd7f0078b90 0x7fd7f007b0a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd804005e00 tx=0x7fd804005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:23.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.521+0000 7fd80a000640 1 -- 172.21.15.39:0/3040819916 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd80c062f40 con 0x7fd8141f51c0 2024-09-21T07:34:23.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.765+0000 7fd813200640 1 -- 172.21.15.39:0/3040819916 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "snap-test-pool"} v 0) -- 0x7fd81406fb50 con 0x7fd8141f51c0 2024-09-21T07:34:23.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.766+0000 7fd80a000640 1 -- 172.21.15.39:0/3040819916 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "snap-test-pool"}]=-95 pool(6) already has mon-managed snaps; can't attach pool to fs v17) ==== 173+0+0 (secure 0 0 0) 0x7fd80c067970 con 0x7fd8141f51c0 2024-09-21T07:34:23.768 INFO:teuthology.orchestra.run.smithi039.stderr:Error ENOTSUP: pool(6) already has mon-managed snaps; can't attach pool to fs 2024-09-21T07:34:23.787 DEBUG:teuthology.orchestra.run:got remote process result: 95 2024-09-21T07:34:23.788 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool snap-test-pool rmsnap snap3 2024-09-21T07:34:23.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.930+0000 7f06cba73f40 1 Processor -- start 2024-09-21T07:34:23.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.930+0000 7f06cba73f40 1 -- start start 2024-09-21T07:34:23.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.932+0000 7f06cba73f40 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06ac003c40 0x55ff1db81600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:23.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.932+0000 7f06cba73f40 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ff1dae9bd0 0x55ff1da5faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:23.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.932+0000 7f06cba73f40 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ff1da5f4a0 0x55ff1da69d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:23.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.932+0000 7f06cba73f40 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ff1d9ca1d0 con 0x7f06ac003c40 2024-09-21T07:34:23.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.932+0000 7f06cba73f40 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ff1d9d6f90 con 0x55ff1da5f4a0 2024-09-21T07:34:23.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.932+0000 7f06cba73f40 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55ff1d96bc60 con 0x55ff1dae9bd0 2024-09-21T07:34:23.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.932+0000 7f06c4c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06ac003c40 0x55ff1db81600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:23.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.932+0000 7f06c4c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06ac003c40 0x55ff1db81600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52684/0 (socket says 172.21.15.39:52684) 2024-09-21T07:34:23.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.932+0000 7f06c4c00640 1 -- 172.21.15.39:0/3516118367 learned_addr learned my addr 172.21.15.39:0/3516118367 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:23.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.932+0000 7f06c5600640 1 --2- 172.21.15.39:0/3516118367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ff1da5f4a0 0x55ff1da69d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:23.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.932+0000 7f06bfe00640 1 --2- 172.21.15.39:0/3516118367 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ff1dae9bd0 0x55ff1da5faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:23.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.933+0000 7f06c4c00640 1 -- 172.21.15.39:0/3516118367 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ff1dae9bd0 msgr2=0x55ff1da5faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:34:23.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.933+0000 7f06c4c00640 1 --2- 172.21.15.39:0/3516118367 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ff1dae9bd0 0x55ff1da5faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:23.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.933+0000 7f06c4c00640 1 -- 172.21.15.39:0/3516118367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ff1da5f4a0 msgr2=0x55ff1da69d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:34:23.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.933+0000 7f06c4c00640 1 --2- 172.21.15.39:0/3516118367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ff1da5f4a0 0x55ff1da69d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:23.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.933+0000 7f06c4c00640 1 -- 172.21.15.39:0/3516118367 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff1daeda60 con 0x7f06ac003c40 2024-09-21T07:34:23.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.933+0000 7f06bfe00640 1 --2- 172.21.15.39:0/3516118367 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ff1dae9bd0 0x55ff1da5faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:34:23.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.933+0000 7f06c5600640 1 --2- 172.21.15.39:0/3516118367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ff1da5f4a0 0x55ff1da69d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:34:23.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.933+0000 7f06c4c00640 1 --2- 172.21.15.39:0/3516118367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06ac003c40 0x55ff1db81600 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f06b800c310 tx=0x7f06b800c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad0e1c1f3e7abca9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:23.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.933+0000 7f06bea00640 1 -- 172.21.15.39:0/3516118367 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b800f660 con 0x7f06ac003c40 2024-09-21T07:34:23.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.933+0000 7f06bea00640 1 -- 172.21.15.39:0/3516118367 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06b8014070 con 0x7f06ac003c40 2024-09-21T07:34:23.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.933+0000 7f06bea00640 1 -- 172.21.15.39:0/3516118367 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b800fb10 con 0x7f06ac003c40 2024-09-21T07:34:23.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.934+0000 7f06cba73f40 1 -- 172.21.15.39:0/3516118367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06ac003c40 msgr2=0x55ff1db81600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:23.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.934+0000 7f06cba73f40 1 --2- 172.21.15.39:0/3516118367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06ac003c40 0x55ff1db81600 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f06b800c310 tx=0x7f06b800c7d0 comp rx=0 tx=0).stop 2024-09-21T07:34:23.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.934+0000 7f06cba73f40 1 -- 172.21.15.39:0/3516118367 shutdown_connections 2024-09-21T07:34:23.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.934+0000 7f06cba73f40 1 --2- 172.21.15.39:0/3516118367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ff1da5f4a0 0x55ff1da69d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:23.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.934+0000 7f06cba73f40 1 --2- 172.21.15.39:0/3516118367 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ff1dae9bd0 0x55ff1da5faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:23.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.934+0000 7f06cba73f40 1 --2- 172.21.15.39:0/3516118367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06ac003c40 0x55ff1db81600 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:23.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.934+0000 7f06cba73f40 1 -- 172.21.15.39:0/3516118367 >> 172.21.15.39:0/3516118367 conn(0x55ff1d93cac0 msgr2=0x55ff1daf5dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:23.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.934+0000 7f06cba73f40 1 -- 172.21.15.39:0/3516118367 shutdown_connections 2024-09-21T07:34:23.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.934+0000 7f06cba73f40 1 -- 172.21.15.39:0/3516118367 wait complete. 2024-09-21T07:34:23.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.935+0000 7f06cba73f40 1 Processor -- start 2024-09-21T07:34:23.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.935+0000 7f06cba73f40 1 -- start start 2024-09-21T07:34:23.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.937+0000 7f06cba73f40 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ff1da5f4a0 0x55ff1dad0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:23.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.937+0000 7f06cba73f40 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ff1dae9bd0 0x55ff1daccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:23.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.937+0000 7f06cba73f40 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f06ac003c40 0x55ff1da667e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:23.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.937+0000 7f06cba73f40 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ff1dad80a0 con 0x55ff1da5f4a0 2024-09-21T07:34:23.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.937+0000 7f06cba73f40 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ff1da675e0 con 0x55ff1dae9bd0 2024-09-21T07:34:23.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.937+0000 7f06cba73f40 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55ff1daf6210 con 0x7f06ac003c40 2024-09-21T07:34:23.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.937+0000 7f06c4c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ff1da5f4a0 0x55ff1dad0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:23.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.937+0000 7f06c4c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ff1da5f4a0 0x55ff1dad0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52692/0 (socket says 172.21.15.39:52692) 2024-09-21T07:34:23.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.937+0000 7f06c4c00640 1 -- 172.21.15.39:0/84347045 learned_addr learned my addr 172.21.15.39:0/84347045 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:23.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.937+0000 7f06bfe00640 1 --2- 172.21.15.39:0/84347045 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ff1dae9bd0 0x55ff1daccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:23.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.937+0000 7f06c5600640 1 --2- 172.21.15.39:0/84347045 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f06ac003c40 0x55ff1da667e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:23.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.937+0000 7f06c4c00640 1 -- 172.21.15.39:0/84347045 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f06ac003c40 msgr2=0x55ff1da667e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:23.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.937+0000 7f06c4c00640 1 --2- 172.21.15.39:0/84347045 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f06ac003c40 0x55ff1da667e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:23.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.937+0000 7f06c4c00640 1 -- 172.21.15.39:0/84347045 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ff1dae9bd0 msgr2=0x55ff1daccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:23.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.937+0000 7f06c4c00640 1 --2- 172.21.15.39:0/84347045 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ff1dae9bd0 0x55ff1daccdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:23.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.937+0000 7f06c4c00640 1 -- 172.21.15.39:0/84347045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff1dacf0b0 con 0x55ff1da5f4a0 2024-09-21T07:34:23.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.937+0000 7f06c5600640 1 --2- 172.21.15.39:0/84347045 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f06ac003c40 0x55ff1da667e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:34:23.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.938+0000 7f06c4c00640 1 --2- 172.21.15.39:0/84347045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ff1da5f4a0 0x55ff1dad0150 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f06b800ccd0 tx=0x7f06b800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:23.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.938+0000 7f06bd600640 1 -- 172.21.15.39:0/84347045 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b80014e0 con 0x55ff1da5f4a0 2024-09-21T07:34:23.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.938+0000 7f06bd600640 1 -- 172.21.15.39:0/84347045 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06b8014040 con 0x55ff1da5f4a0 2024-09-21T07:34:23.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.938+0000 7f06bd600640 1 -- 172.21.15.39:0/84347045 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b8001a10 con 0x55ff1da5f4a0 2024-09-21T07:34:23.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.938+0000 7f06cba73f40 1 -- 172.21.15.39:0/84347045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ff1dacf390 con 0x55ff1da5f4a0 2024-09-21T07:34:23.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.938+0000 7f06cba73f40 1 -- 172.21.15.39:0/84347045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff1dacf950 con 0x55ff1da5f4a0 2024-09-21T07:34:23.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.940+0000 7f06bd600640 1 -- 172.21.15.39:0/84347045 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f06b8001bb0 con 0x55ff1da5f4a0 2024-09-21T07:34:23.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.941+0000 7f06bd600640 1 --2- 172.21.15.39:0/84347045 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f06a4077570 0x7f06a4079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:23.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.941+0000 7f06bd600640 1 -- 172.21.15.39:0/84347045 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7f06b809ba30 con 0x55ff1da5f4a0 2024-09-21T07:34:23.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.942+0000 7f06cba73f40 1 -- 172.21.15.39:0/84347045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- pool_op(delete snap pool 6 tid 1 name snap3 v0) -- 0x55ff1d9ccb20 con 0x55ff1da5f4a0 2024-09-21T07:34:23.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.942+0000 7f06bfe00640 1 --2- 172.21.15.39:0/84347045 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f06a4077570 0x7f06a4079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:23.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:23.943+0000 7f06bfe00640 1 --2- 172.21.15.39:0/84347045 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f06a4077570 0x7f06a4079a80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x55ff1daeda30 tx=0x7f06b000f040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:24.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.256+0000 7f06bd600640 1 -- 172.21.15.39:0/84347045 <== mon.0 v2:172.21.15.39:3300/0 6 ==== pool_op_reply(tid 1 (0) Success v37) ==== 43+0+0 (secure 0 0 0) 0x7f06b8063710 con 0x55ff1da5f4a0 2024-09-21T07:34:24.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.256+0000 7f06bd600640 1 -- 172.21.15.39:0/84347045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=37}) -- 0x7f06a407fe60 con 0x55ff1da5f4a0 2024-09-21T07:34:24.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.257+0000 7f06bd600640 1 -- 172.21.15.39:0/84347045 <== mon.0 v2:172.21.15.39:3300/0 7 ==== osd_map(37..37 src has 1..37) ==== 643+0+0 (secure 0 0 0) 0x7f06b8099e40 con 0x55ff1da5f4a0 2024-09-21T07:34:24.259 INFO:teuthology.orchestra.run.smithi039.stdout:removed pool snap-test-pool snap snap3 2024-09-21T07:34:24.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.258+0000 7f06cba73f40 1 -- 172.21.15.39:0/84347045 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f06a4077570 msgr2=0x7f06a4079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:24.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.258+0000 7f06cba73f40 1 --2- 172.21.15.39:0/84347045 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f06a4077570 0x7f06a4079a80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x55ff1daeda30 tx=0x7f06b000f040 comp rx=0 tx=0).stop 2024-09-21T07:34:24.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.258+0000 7f06cba73f40 1 -- 172.21.15.39:0/84347045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ff1da5f4a0 msgr2=0x55ff1dad0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:24.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.258+0000 7f06cba73f40 1 --2- 172.21.15.39:0/84347045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ff1da5f4a0 0x55ff1dad0150 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f06b800ccd0 tx=0x7f06b800f690 comp rx=0 tx=0).stop 2024-09-21T07:34:24.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.259+0000 7f06cba73f40 1 -- 172.21.15.39:0/84347045 shutdown_connections 2024-09-21T07:34:24.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.259+0000 7f06cba73f40 1 --2- 172.21.15.39:0/84347045 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f06a4077570 0x7f06a4079a80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:24.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.259+0000 7f06cba73f40 1 --2- 172.21.15.39:0/84347045 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f06ac003c40 0x55ff1da667e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:24.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.259+0000 7f06cba73f40 1 --2- 172.21.15.39:0/84347045 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ff1dae9bd0 0x55ff1daccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:24.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.259+0000 7f06cba73f40 1 --2- 172.21.15.39:0/84347045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ff1da5f4a0 0x55ff1dad0150 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:24.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.259+0000 7f06cba73f40 1 -- 172.21.15.39:0/84347045 >> 172.21.15.39:0/84347045 conn(0x55ff1d93cac0 msgr2=0x55ff1dad0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:24.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.259+0000 7f06cba73f40 1 -- 172.21.15.39:0/84347045 shutdown_connections 2024-09-21T07:34:24.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.259+0000 7f06cba73f40 1 -- 172.21.15.39:0/84347045 wait complete. 2024-09-21T07:34:24.273 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete snap-test-pool 2024-09-21T07:34:24.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.499+0000 7f4dad000640 1 -- 172.21.15.39:0/577168383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4da8070030 msgr2=0x7f4da8074640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:24.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.499+0000 7f4dad000640 1 --2- 172.21.15.39:0/577168383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4da8070030 0x7f4da8074640 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4d94005fd0 tx=0x7f4d9402a670 comp rx=0 tx=0).stop 2024-09-21T07:34:24.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.499+0000 7f4dad000640 1 -- 172.21.15.39:0/577168383 shutdown_connections 2024-09-21T07:34:24.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.499+0000 7f4dad000640 1 --2- 172.21.15.39:0/577168383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4da8070030 0x7f4da8074640 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:24.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.499+0000 7f4dad000640 1 -- 172.21.15.39:0/577168383 >> 172.21.15.39:0/577168383 conn(0x7f4da80b1cb0 msgr2=0x7f4da80b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:24.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.499+0000 7f4dad000640 1 -- 172.21.15.39:0/577168383 shutdown_connections 2024-09-21T07:34:24.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.500+0000 7f4dad000640 1 -- 172.21.15.39:0/577168383 wait complete. 2024-09-21T07:34:24.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.502+0000 7f4dad000640 1 Processor -- start 2024-09-21T07:34:24.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.502+0000 7f4dad000640 1 -- start start 2024-09-21T07:34:24.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.503+0000 7f4dad000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4da8070030 0x7f4da815b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:24.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.503+0000 7f4dad000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4da8070440 0x7f4da815e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:24.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.503+0000 7f4dad000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4da8073170 0x7f4da81ea330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:24.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.503+0000 7f4dad000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4da80b3c40 con 0x7f4da8073170 2024-09-21T07:34:24.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.503+0000 7f4dad000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4da80b3ac0 con 0x7f4da8070030 2024-09-21T07:34:24.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.503+0000 7f4dad000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f4da80b3dc0 con 0x7f4da8070440 2024-09-21T07:34:24.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.504+0000 7f4da6800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4da8073170 0x7f4da81ea330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:24.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.504+0000 7f4da5e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4da8070030 0x7f4da815b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:24.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.504+0000 7f4da6800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4da8073170 0x7f4da81ea330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52710/0 (socket says 172.21.15.39:52710) 2024-09-21T07:34:24.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.504+0000 7f4da6800640 1 -- 172.21.15.39:0/295609031 learned_addr learned my addr 172.21.15.39:0/295609031 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:24.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.504+0000 7f4da5400640 1 --2- 172.21.15.39:0/295609031 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4da8070440 0x7f4da815e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:24.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.504+0000 7f4da6800640 1 -- 172.21.15.39:0/295609031 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4da8070440 msgr2=0x7f4da815e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:24.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.504+0000 7f4da6800640 1 --2- 172.21.15.39:0/295609031 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4da8070440 0x7f4da815e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:24.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.504+0000 7f4da6800640 1 -- 172.21.15.39:0/295609031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4da8070030 msgr2=0x7f4da815b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:24.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.504+0000 7f4da6800640 1 --2- 172.21.15.39:0/295609031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4da8070030 0x7f4da815b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:24.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.504+0000 7f4da6800640 1 -- 172.21.15.39:0/295609031 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da80cebd0 con 0x7f4da8073170 2024-09-21T07:34:24.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.504+0000 7f4da6800640 1 --2- 172.21.15.39:0/295609031 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4da8073170 0x7f4da81ea330 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4da000c700 tx=0x7f4da000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:24.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.505+0000 7f4d9e000640 1 -- 172.21.15.39:0/295609031 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da0003b70 con 0x7f4da8073170 2024-09-21T07:34:24.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.505+0000 7f4d9e000640 1 -- 172.21.15.39:0/295609031 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4da0003d10 con 0x7f4da8073170 2024-09-21T07:34:24.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.505+0000 7f4dad000640 1 -- 172.21.15.39:0/295609031 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4da80cee70 con 0x7f4da8073170 2024-09-21T07:34:24.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.505+0000 7f4d9e000640 1 -- 172.21.15.39:0/295609031 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da000aab0 con 0x7f4da8073170 2024-09-21T07:34:24.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.505+0000 7f4dad000640 1 -- 172.21.15.39:0/295609031 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4da81f70a0 con 0x7f4da8073170 2024-09-21T07:34:24.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.506+0000 7f4dad000640 1 -- 172.21.15.39:0/295609031 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4da80d0220 con 0x7f4da8073170 2024-09-21T07:34:24.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.507+0000 7f4d9e000640 1 -- 172.21.15.39:0/295609031 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4da000ac50 con 0x7f4da8073170 2024-09-21T07:34:24.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.508+0000 7f4d9e000640 1 --2- 172.21.15.39:0/295609031 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4d88078bc0 0x7f4d8807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:24.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.508+0000 7f4d9e000640 1 -- 172.21.15.39:0/295609031 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6118+0+0 (secure 0 0 0) 0x7f4da009ab30 con 0x7f4da8073170 2024-09-21T07:34:24.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.509+0000 7f4da5e00640 1 --2- 172.21.15.39:0/295609031 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4d88078bc0 0x7f4d8807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:24.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.510+0000 7f4da5e00640 1 --2- 172.21.15.39:0/295609031 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4d88078bc0 0x7f4d8807b0d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4d98001720 tx=0x7f4d98009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:24.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.513+0000 7f4d9e000640 1 -- 172.21.15.39:0/295609031 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4da0062770 con 0x7f4da8073170 2024-09-21T07:34:24.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.712+0000 7f4dad000640 1 -- 172.21.15.39:0/295609031 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "snap-test-pool"} v 0) -- 0x7f4da80b42e0 con 0x7f4da8073170 2024-09-21T07:34:24.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:24.712+0000 7f4d9e000640 1 -- 172.21.15.39:0/295609031 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "snap-test-pool"}]=-1 WARNING: this will *PERMANENTLY DESTROY* all data stored in pool snap-test-pool. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. v37) ==== 296+0+0 (secure 0 0 0) 0x7f4da00671a0 con 0x7f4da8073170 2024-09-21T07:34:24.715 INFO:teuthology.orchestra.run.smithi039.stderr:Error EPERM: WARNING: this will *PERMANENTLY DESTROY* all data stored in pool snap-test-pool. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. 2024-09-21T07:34:24.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:34:24.732 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:34:24.732 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:34:24.821 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:34:24.917 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:34:24.917 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:34:24.917 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:34:24.918 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:34:24.953 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:34:24.953 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:34:25.029 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:34:25.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.297+0000 7f1674800640 1 -- 172.21.15.39:0/1867849007 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1654005250 msgr2=0x7f165400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:25.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.297+0000 7f1674800640 1 --2- 172.21.15.39:0/1867849007 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1654005250 0x7f165400b9c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f166800c610 tx=0x7f1654005830 comp rx=0 tx=0).stop 2024-09-21T07:34:25.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.297+0000 7f1674800640 1 -- 172.21.15.39:0/1867849007 shutdown_connections 2024-09-21T07:34:25.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.297+0000 7f1674800640 1 --2- 172.21.15.39:0/1867849007 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1654005250 0x7f165400b9c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:25.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.297+0000 7f1674800640 1 -- 172.21.15.39:0/1867849007 >> 172.21.15.39:0/1867849007 conn(0x7f166c06d7d0 msgr2=0x7f166c0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:25.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.297+0000 7f1674800640 1 -- 172.21.15.39:0/1867849007 shutdown_connections 2024-09-21T07:34:25.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.297+0000 7f1674800640 1 -- 172.21.15.39:0/1867849007 wait complete. 2024-09-21T07:34:25.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.300+0000 7f1674800640 1 Processor -- start 2024-09-21T07:34:25.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.300+0000 7f1674800640 1 -- start start 2024-09-21T07:34:25.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.301+0000 7f1674800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1654005250 0x7f166c0cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:25.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.301+0000 7f1674800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f166c156ff0 0x7f166c0adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:25.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.301+0000 7f1674800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f166c151000 0x7f166c0ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:25.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.301+0000 7f1674800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f166c0af750 con 0x7f166c156ff0 2024-09-21T07:34:25.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.301+0000 7f1674800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f166c0af5d0 con 0x7f1654005250 2024-09-21T07:34:25.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.301+0000 7f1674800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f166c0af8d0 con 0x7f166c151000 2024-09-21T07:34:25.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.302+0000 7f1671400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f166c156ff0 0x7f166c0adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:25.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.302+0000 7f1671400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f166c156ff0 0x7f166c0adfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52728/0 (socket says 172.21.15.39:52728) 2024-09-21T07:34:25.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.302+0000 7f1671400640 1 -- 172.21.15.39:0/884393984 learned_addr learned my addr 172.21.15.39:0/884393984 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:25.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.302+0000 7f1671e00640 1 --2- 172.21.15.39:0/884393984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1654005250 0x7f166c0cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:25.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.302+0000 7f1672800640 1 --2- 172.21.15.39:0/884393984 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f166c151000 0x7f166c0ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:25.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.302+0000 7f1671400640 1 -- 172.21.15.39:0/884393984 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f166c151000 msgr2=0x7f166c0ae520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:25.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.302+0000 7f1671400640 1 --2- 172.21.15.39:0/884393984 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f166c151000 0x7f166c0ae520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:25.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.302+0000 7f1671400640 1 -- 172.21.15.39:0/884393984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1654005250 msgr2=0x7f166c0cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:25.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.302+0000 7f1671400640 1 --2- 172.21.15.39:0/884393984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1654005250 0x7f166c0cb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:25.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.302+0000 7f1671400640 1 -- 172.21.15.39:0/884393984 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166c157560 con 0x7f166c156ff0 2024-09-21T07:34:25.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.303+0000 7f1671400640 1 --2- 172.21.15.39:0/884393984 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f166c156ff0 0x7f166c0adfe0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f165c004ed0 tx=0x7f165c001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:25.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.303+0000 7f1666000640 1 -- 172.21.15.39:0/884393984 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165c015020 con 0x7f166c156ff0 2024-09-21T07:34:25.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.303+0000 7f1666000640 1 -- 172.21.15.39:0/884393984 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f165c011960 con 0x7f166c156ff0 2024-09-21T07:34:25.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.303+0000 7f1666000640 1 -- 172.21.15.39:0/884393984 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165c011d90 con 0x7f166c156ff0 2024-09-21T07:34:25.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.303+0000 7f1674800640 1 -- 172.21.15.39:0/884393984 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f166c0cb900 con 0x7f166c156ff0 2024-09-21T07:34:25.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.303+0000 7f1674800640 1 -- 172.21.15.39:0/884393984 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f166c157ab0 con 0x7f166c156ff0 2024-09-21T07:34:25.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.304+0000 7f1674800640 1 -- 172.21.15.39:0/884393984 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f166c157cb0 con 0x7f166c156ff0 2024-09-21T07:34:25.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.305+0000 7f1666000640 1 -- 172.21.15.39:0/884393984 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f165c018020 con 0x7f166c156ff0 2024-09-21T07:34:25.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.307+0000 7f1666000640 1 --2- 172.21.15.39:0/884393984 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1650078bc0 0x7f165007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:25.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.307+0000 7f1666000640 1 -- 172.21.15.39:0/884393984 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6118+0+0 (secure 0 0 0) 0x7f165c09b8f0 con 0x7f166c156ff0 2024-09-21T07:34:25.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.307+0000 7f1671e00640 1 --2- 172.21.15.39:0/884393984 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1650078bc0 0x7f165007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:25.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.308+0000 7f1671e00640 1 --2- 172.21.15.39:0/884393984 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1650078bc0 0x7f165007b0d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1660009bd0 tx=0x7f1660005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:25.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.311+0000 7f1666000640 1 -- 172.21.15.39:0/884393984 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f165c01f020 con 0x7f166c156ff0 2024-09-21T07:34:25.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.518+0000 7f1674800640 1 -- 172.21.15.39:0/884393984 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f166c0b4290 con 0x7f166c156ff0 2024-09-21T07:34:25.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.518+0000 7f1666000640 1 -- 172.21.15.39:0/884393984 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v37) ==== 80+0+0 (secure 0 0 0) 0x7f165c063af0 con 0x7f166c156ff0 2024-09-21T07:34:25.520 INFO:teuthology.orchestra.run.smithi039.stderr:listed 0 entries 2024-09-21T07:34:25.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.520+0000 7f1674800640 1 -- 172.21.15.39:0/884393984 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1650078bc0 msgr2=0x7f165007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:25.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.520+0000 7f1674800640 1 --2- 172.21.15.39:0/884393984 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1650078bc0 0x7f165007b0d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1660009bd0 tx=0x7f1660005f90 comp rx=0 tx=0).stop 2024-09-21T07:34:25.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.520+0000 7f1674800640 1 -- 172.21.15.39:0/884393984 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f166c156ff0 msgr2=0x7f166c0adfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:25.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.520+0000 7f1674800640 1 --2- 172.21.15.39:0/884393984 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f166c156ff0 0x7f166c0adfe0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f165c004ed0 tx=0x7f165c001630 comp rx=0 tx=0).stop 2024-09-21T07:34:25.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.520+0000 7f1674800640 1 -- 172.21.15.39:0/884393984 shutdown_connections 2024-09-21T07:34:25.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.520+0000 7f1674800640 1 --2- 172.21.15.39:0/884393984 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1650078bc0 0x7f165007b0d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:25.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.520+0000 7f1674800640 1 --2- 172.21.15.39:0/884393984 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f166c151000 0x7f166c0ae520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:25.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.520+0000 7f1674800640 1 --2- 172.21.15.39:0/884393984 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f166c156ff0 0x7f166c0adfe0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:25.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.520+0000 7f1674800640 1 --2- 172.21.15.39:0/884393984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1654005250 0x7f166c0cb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:25.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.520+0000 7f1674800640 1 -- 172.21.15.39:0/884393984 >> 172.21.15.39:0/884393984 conn(0x7f166c06d7d0 msgr2=0x7f166c15dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:25.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.521+0000 7f1674800640 1 -- 172.21.15.39:0/884393984 shutdown_connections 2024-09-21T07:34:25.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.521+0000 7f1674800640 1 -- 172.21.15.39:0/884393984 wait complete. 2024-09-21T07:34:25.544 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T07:34:25.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:34:25.544 DEBUG:teuthology.orchestra.run.smithi039:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T07:34:25.607 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T07:34:25.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:34:25.608 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:34:25.643 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:34:25.643 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:34:25.644 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:34:25.684 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:34:25.684 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:34:25.685 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:34:25.685 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:34:25.720 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:34:25.720 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:34:25.720 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:34:25.720 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:34:25.763 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:34:25.763 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:34:25.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.988+0000 7fd39a200640 1 -- 172.21.15.39:0/1579280881 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd39415f660 msgr2=0x7fd3940b3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:25.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.988+0000 7fd39a200640 1 --2- 172.21.15.39:0/1579280881 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd39415f660 0x7fd3940b3570 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd38c005fd0 tx=0x7fd38c02aa10 comp rx=0 tx=0).stop 2024-09-21T07:34:25.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.988+0000 7fd39a200640 1 -- 172.21.15.39:0/1579280881 shutdown_connections 2024-09-21T07:34:25.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.988+0000 7fd39a200640 1 --2- 172.21.15.39:0/1579280881 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd39415f660 0x7fd3940b3570 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:25.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.988+0000 7fd39a200640 1 --2- 172.21.15.39:0/1579280881 >> v2:172.21.15.148:3300/0 conn(0x7fd39415fbd0 0x7fd394152360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:25.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.988+0000 7fd39a200640 1 -- 172.21.15.39:0/1579280881 >> 172.21.15.39:0/1579280881 conn(0x7fd39406d410 msgr2=0x7fd3940b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:25.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.988+0000 7fd39a200640 1 -- 172.21.15.39:0/1579280881 shutdown_connections 2024-09-21T07:34:25.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.988+0000 7fd39a200640 1 -- 172.21.15.39:0/1579280881 wait complete. 2024-09-21T07:34:25.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.990+0000 7fd39a200640 1 Processor -- start 2024-09-21T07:34:25.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.990+0000 7fd39a200640 1 -- start start 2024-09-21T07:34:25.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.992+0000 7fd39a200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd394072c30 0x7fd3941482c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:25.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.992+0000 7fd39a200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd394153530 0x7fd39414bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:25.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.992+0000 7fd39a200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd394148bb0 0x7fd39414c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:25.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.992+0000 7fd39a200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd3940b3fe0 con 0x7fd394072c30 2024-09-21T07:34:25.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.992+0000 7fd39a200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd3940b3e60 con 0x7fd394148bb0 2024-09-21T07:34:25.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.992+0000 7fd39a200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd3940b4160 con 0x7fd394153530 2024-09-21T07:34:25.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.992+0000 7fd398e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd394072c30 0x7fd3941482c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:25.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.992+0000 7fd398e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd394072c30 0x7fd3941482c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52744/0 (socket says 172.21.15.39:52744) 2024-09-21T07:34:25.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.992+0000 7fd398e00640 1 -- 172.21.15.39:0/172512372 learned_addr learned my addr 172.21.15.39:0/172512372 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:25.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.993+0000 7fd393e00640 1 --2- 172.21.15.39:0/172512372 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd394153530 0x7fd39414bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:25.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.993+0000 7fd399800640 1 --2- 172.21.15.39:0/172512372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd394148bb0 0x7fd39414c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:25.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.993+0000 7fd398e00640 1 -- 172.21.15.39:0/172512372 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd394153530 msgr2=0x7fd39414bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:25.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.993+0000 7fd398e00640 1 --2- 172.21.15.39:0/172512372 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd394153530 0x7fd39414bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:25.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.993+0000 7fd398e00640 1 -- 172.21.15.39:0/172512372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd394148bb0 msgr2=0x7fd39414c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:25.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.993+0000 7fd398e00640 1 --2- 172.21.15.39:0/172512372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd394148bb0 0x7fd39414c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:25.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.993+0000 7fd398e00640 1 -- 172.21.15.39:0/172512372 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3941f59c0 con 0x7fd394072c30 2024-09-21T07:34:25.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.993+0000 7fd399800640 1 --2- 172.21.15.39:0/172512372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd394148bb0 0x7fd39414c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:34:25.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.993+0000 7fd398e00640 1 --2- 172.21.15.39:0/172512372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd394072c30 0x7fd3941482c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd38800e790 tx=0x7fd38800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:25.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.993+0000 7fd390c00640 1 -- 172.21.15.39:0/172512372 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38800eea0 con 0x7fd394072c30 2024-09-21T07:34:25.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.993+0000 7fd390c00640 1 -- 172.21.15.39:0/172512372 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd38800cdb0 con 0x7fd394072c30 2024-09-21T07:34:25.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.993+0000 7fd39a200640 1 -- 172.21.15.39:0/172512372 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3941f5ca0 con 0x7fd394072c30 2024-09-21T07:34:25.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.993+0000 7fd390c00640 1 -- 172.21.15.39:0/172512372 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd388010690 con 0x7fd394072c30 2024-09-21T07:34:25.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.994+0000 7fd39a200640 1 -- 172.21.15.39:0/172512372 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3941f6260 con 0x7fd394072c30 2024-09-21T07:34:25.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.995+0000 7fd39a200640 1 -- 172.21.15.39:0/172512372 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3940ccd40 con 0x7fd394072c30 2024-09-21T07:34:25.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.996+0000 7fd390c00640 1 -- 172.21.15.39:0/172512372 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd388010830 con 0x7fd394072c30 2024-09-21T07:34:25.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.997+0000 7fd390c00640 1 --2- 172.21.15.39:0/172512372 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd374078b90 0x7fd37407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:25.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.998+0000 7fd390c00640 1 -- 172.21.15.39:0/172512372 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6118+0+0 (secure 0 0 0) 0x7fd38809b930 con 0x7fd394072c30 2024-09-21T07:34:26.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.998+0000 7fd393e00640 1 --2- 172.21.15.39:0/172512372 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd374078b90 0x7fd37407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:26.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:25.999+0000 7fd393e00640 1 --2- 172.21.15.39:0/172512372 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd374078b90 0x7fd37407b0a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd39414ba80 tx=0x7fd380009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:26.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.002+0000 7fd390c00640 1 -- 172.21.15.39:0/172512372 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd3940ccd40 con 0x7fd394072c30 2024-09-21T07:34:26.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.236+0000 7fd39a200640 1 -- 172.21.15.39:0/172512372 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3940b4680 con 0x7fd394072c30 2024-09-21T07:34:26.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.237+0000 7fd390c00640 1 -- 172.21.15.39:0/172512372 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5229 (secure 0 0 0) 0x7fd388063bd0 con 0x7fd394072c30 2024-09-21T07:34:26.239 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:34:26.239 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":17,"btime":"2024-09-21T07:34:11:169975+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2438395645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2438395645},{"type":"v1","addr":"172.21.15.148:6837","nonce":2438395645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4706,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3668947757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3668947757},{"type":"v1","addr":"172.21.15.39:6835","nonce":3668947757}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4870,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2800764560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2800764560},{"type":"v1","addr":"172.21.15.148:6835","nonce":2800764560}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:34:07.651669+0000","modified":"2024-09-21T07:34:10.675070+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4888":{"gid":4888,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/3524592265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3524592265},{"type":"v1","addr":"172.21.15.39:6834","nonce":3524592265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4888,"qdb_cluster":[4888]},"id":2}]} 2024-09-21T07:34:26.239 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 17 2024-09-21T07:34:26.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.239+0000 7fd39a200640 1 -- 172.21.15.39:0/172512372 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd374078b90 msgr2=0x7fd37407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:26.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.239+0000 7fd39a200640 1 --2- 172.21.15.39:0/172512372 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd374078b90 0x7fd37407b0a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd39414ba80 tx=0x7fd380009040 comp rx=0 tx=0).stop 2024-09-21T07:34:26.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.239+0000 7fd39a200640 1 -- 172.21.15.39:0/172512372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd394072c30 msgr2=0x7fd3941482c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:26.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.239+0000 7fd39a200640 1 --2- 172.21.15.39:0/172512372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd394072c30 0x7fd3941482c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd38800e790 tx=0x7fd38800ec50 comp rx=0 tx=0).stop 2024-09-21T07:34:26.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.239+0000 7fd39a200640 1 -- 172.21.15.39:0/172512372 shutdown_connections 2024-09-21T07:34:26.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.239+0000 7fd39a200640 1 --2- 172.21.15.39:0/172512372 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd374078b90 0x7fd37407b0a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:26.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.239+0000 7fd39a200640 1 --2- 172.21.15.39:0/172512372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd394148bb0 0x7fd39414c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:26.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.239+0000 7fd39a200640 1 --2- 172.21.15.39:0/172512372 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd394153530 0x7fd39414bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:26.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.239+0000 7fd39a200640 1 --2- 172.21.15.39:0/172512372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd394072c30 0x7fd3941482c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:26.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.239+0000 7fd39a200640 1 -- 172.21.15.39:0/172512372 >> 172.21.15.39:0/172512372 conn(0x7fd39406d410 msgr2=0x7fd39415adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:26.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.239+0000 7fd39a200640 1 -- 172.21.15.39:0/172512372 shutdown_connections 2024-09-21T07:34:26.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.239+0000 7fd39a200640 1 -- 172.21.15.39:0/172512372 wait complete. 2024-09-21T07:34:26.264 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:34:26.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.488+0000 7f4cfd000640 1 -- 172.21.15.39:0/1627982715 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cf80b59a0 msgr2=0x7f4cf815cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:26.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.488+0000 7f4cfd000640 1 --2- 172.21.15.39:0/1627982715 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cf80b59a0 0x7f4cf815cda0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0005fd0 tx=0x7f4ce002a530 comp rx=0 tx=0).stop 2024-09-21T07:34:26.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.488+0000 7f4cfd000640 1 -- 172.21.15.39:0/1627982715 shutdown_connections 2024-09-21T07:34:26.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.488+0000 7f4cfd000640 1 --2- 172.21.15.39:0/1627982715 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cf80b59a0 0x7f4cf815cda0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:26.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.488+0000 7f4cfd000640 1 --2- 172.21.15.39:0/1627982715 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4cf8153780 0x7f4cf80b5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:26.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.488+0000 7f4cfd000640 1 -- 172.21.15.39:0/1627982715 >> 172.21.15.39:0/1627982715 conn(0x7f4cf806d870 msgr2=0x7f4cf806ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:26.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.488+0000 7f4cfd000640 1 -- 172.21.15.39:0/1627982715 shutdown_connections 2024-09-21T07:34:26.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.488+0000 7f4cfd000640 1 -- 172.21.15.39:0/1627982715 wait complete. 2024-09-21T07:34:26.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.491+0000 7f4cfd000640 1 Processor -- start 2024-09-21T07:34:26.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.491+0000 7f4cfd000640 1 -- start start 2024-09-21T07:34:26.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.492+0000 7f4cfd000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4cf80b59a0 0x7f4cf8159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:26.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.492+0000 7f4cfd000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cf8153780 0x7f4cf81f5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:26.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.492+0000 7f4cfd000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cf815d2e0 0x7f4cf80c92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:26.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.492+0000 7f4cfd000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf8160800 con 0x7f4cf815d2e0 2024-09-21T07:34:26.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.492+0000 7f4cfd000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf80cd4f0 con 0x7f4cf8153780 2024-09-21T07:34:26.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.492+0000 7f4cfd000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f4cf8160980 con 0x7f4cf80b59a0 2024-09-21T07:34:26.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.493+0000 7f4cf7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cf815d2e0 0x7f4cf80c92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:26.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.493+0000 7f4cf7400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4cf80b59a0 0x7f4cf8159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:26.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.493+0000 7f4cf7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cf815d2e0 0x7f4cf80c92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52762/0 (socket says 172.21.15.39:52762) 2024-09-21T07:34:26.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.493+0000 7f4cf7e00640 1 -- 172.21.15.39:0/2876482910 learned_addr learned my addr 172.21.15.39:0/2876482910 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:26.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.493+0000 7f4cefe00640 1 --2- 172.21.15.39:0/2876482910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cf8153780 0x7f4cf81f5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:26.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.493+0000 7f4cf7e00640 1 -- 172.21.15.39:0/2876482910 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4cf80b59a0 msgr2=0x7f4cf8159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:26.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.493+0000 7f4cf7e00640 1 --2- 172.21.15.39:0/2876482910 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4cf80b59a0 0x7f4cf8159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:26.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.493+0000 7f4cf7e00640 1 -- 172.21.15.39:0/2876482910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cf8153780 msgr2=0x7f4cf81f5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:26.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.493+0000 7f4cf7e00640 1 --2- 172.21.15.39:0/2876482910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cf8153780 0x7f4cf81f5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:26.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.493+0000 7f4cf7e00640 1 -- 172.21.15.39:0/2876482910 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf81571d0 con 0x7f4cf815d2e0 2024-09-21T07:34:26.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.493+0000 7f4cf7e00640 1 --2- 172.21.15.39:0/2876482910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cf815d2e0 0x7f4cf80c92a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4cf000be90 tx=0x7f4cf000bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:26.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.494+0000 7f4cef400640 1 -- 172.21.15.39:0/2876482910 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0019070 con 0x7f4cf815d2e0 2024-09-21T07:34:26.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.494+0000 7f4cef400640 1 -- 172.21.15.39:0/2876482910 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4cf0014440 con 0x7f4cf815d2e0 2024-09-21T07:34:26.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.494+0000 7f4cef400640 1 -- 172.21.15.39:0/2876482910 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf00148d0 con 0x7f4cf815d2e0 2024-09-21T07:34:26.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.494+0000 7f4cfd000640 1 -- 172.21.15.39:0/2876482910 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cf81f6110 con 0x7f4cf815d2e0 2024-09-21T07:34:26.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.494+0000 7f4cfd000640 1 -- 172.21.15.39:0/2876482910 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cf81f66d0 con 0x7f4cf815d2e0 2024-09-21T07:34:26.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.495+0000 7f4cfd000640 1 -- 172.21.15.39:0/2876482910 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cf815cda0 con 0x7f4cf815d2e0 2024-09-21T07:34:26.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.496+0000 7f4cef400640 1 -- 172.21.15.39:0/2876482910 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4cf001c070 con 0x7f4cf815d2e0 2024-09-21T07:34:26.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.498+0000 7f4cef400640 1 --2- 172.21.15.39:0/2876482910 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4cd4078bc0 0x7f4cd407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:26.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.498+0000 7f4cef400640 1 -- 172.21.15.39:0/2876482910 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6118+0+0 (secure 0 0 0) 0x7f4cf009ae10 con 0x7f4cf815d2e0 2024-09-21T07:34:26.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.498+0000 7f4cf7400640 1 --2- 172.21.15.39:0/2876482910 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4cd4078bc0 0x7f4cd407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:26.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.499+0000 7f4cf7400640 1 --2- 172.21.15.39:0/2876482910 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4cd4078bc0 0x7f4cd407b0d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8004630 tx=0x7f4ce800b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:26.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.503+0000 7f4cef400640 1 -- 172.21.15.39:0/2876482910 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4cf009e030 con 0x7f4cf815d2e0 2024-09-21T07:34:26.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.742+0000 7f4cfd000640 1 -- 172.21.15.39:0/2876482910 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4cf8068fb0 con 0x7f4cf815d2e0 2024-09-21T07:34:26.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.742+0000 7f4cef400640 1 -- 172.21.15.39:0/2876482910 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5229 (secure 0 0 0) 0x7f4cf00629c0 con 0x7f4cf815d2e0 2024-09-21T07:34:26.745 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:34:26.745 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":17,"btime":"2024-09-21T07:34:11:169975+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2438395645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2438395645},{"type":"v1","addr":"172.21.15.148:6837","nonce":2438395645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4706,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3668947757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3668947757},{"type":"v1","addr":"172.21.15.39:6835","nonce":3668947757}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4870,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2800764560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2800764560},{"type":"v1","addr":"172.21.15.148:6835","nonce":2800764560}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:34:07.651669+0000","modified":"2024-09-21T07:34:10.675070+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4888":{"gid":4888,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/3524592265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3524592265},{"type":"v1","addr":"172.21.15.39:6834","nonce":3524592265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4888,"qdb_cluster":[4888]},"id":2}]} 2024-09-21T07:34:26.745 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 17 2024-09-21T07:34:26.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.745+0000 7f4cfd000640 1 -- 172.21.15.39:0/2876482910 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4cd4078bc0 msgr2=0x7f4cd407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:26.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.745+0000 7f4cfd000640 1 --2- 172.21.15.39:0/2876482910 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4cd4078bc0 0x7f4cd407b0d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8004630 tx=0x7f4ce800b2e0 comp rx=0 tx=0).stop 2024-09-21T07:34:26.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.745+0000 7f4cfd000640 1 -- 172.21.15.39:0/2876482910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cf815d2e0 msgr2=0x7f4cf80c92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:26.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.745+0000 7f4cfd000640 1 --2- 172.21.15.39:0/2876482910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cf815d2e0 0x7f4cf80c92a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4cf000be90 tx=0x7f4cf000bf80 comp rx=0 tx=0).stop 2024-09-21T07:34:26.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.745+0000 7f4cfd000640 1 -- 172.21.15.39:0/2876482910 shutdown_connections 2024-09-21T07:34:26.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.745+0000 7f4cfd000640 1 --2- 172.21.15.39:0/2876482910 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4cd4078bc0 0x7f4cd407b0d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:26.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.745+0000 7f4cfd000640 1 --2- 172.21.15.39:0/2876482910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4cf815d2e0 0x7f4cf80c92a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:26.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.745+0000 7f4cfd000640 1 --2- 172.21.15.39:0/2876482910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4cf8153780 0x7f4cf81f5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:26.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.745+0000 7f4cfd000640 1 --2- 172.21.15.39:0/2876482910 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4cf80b59a0 0x7f4cf8159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:26.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.745+0000 7f4cfd000640 1 -- 172.21.15.39:0/2876482910 >> 172.21.15.39:0/2876482910 conn(0x7f4cf806d870 msgr2=0x7f4cf815bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:26.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.745+0000 7f4cfd000640 1 -- 172.21.15.39:0/2876482910 shutdown_connections 2024-09-21T07:34:26.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.745+0000 7f4cfd000640 1 -- 172.21.15.39:0/2876482910 wait complete. 2024-09-21T07:34:26.769 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:34:26.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.994+0000 7fb4f6200640 1 -- 172.21.15.39:0/2143424355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4f00ce020 msgr2=0x7fb4f00ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:26.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.994+0000 7fb4f6200640 1 --2- 172.21.15.39:0/2143424355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4f00ce020 0x7fb4f00ce430 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0071d60 tx=0x7fb4e802aa30 comp rx=0 tx=0).stop 2024-09-21T07:34:26.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.994+0000 7fb4f6200640 1 -- 172.21.15.39:0/2143424355 shutdown_connections 2024-09-21T07:34:26.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.994+0000 7fb4f6200640 1 --2- 172.21.15.39:0/2143424355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4f00ce020 0x7fb4f00ce430 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:26.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.994+0000 7fb4f6200640 1 --2- 172.21.15.39:0/2143424355 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4f00ccb30 0x7fb4f00ccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:26.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.994+0000 7fb4f6200640 1 --2- 172.21.15.39:0/2143424355 >> v2:172.21.15.148:3300/0 conn(0x7fb4f00cf6f0 0x7fb4f00cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:26.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.994+0000 7fb4f6200640 1 -- 172.21.15.39:0/2143424355 >> 172.21.15.39:0/2143424355 conn(0x7fb4f00b1cb0 msgr2=0x7fb4f00b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:26.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.994+0000 7fb4f6200640 1 -- 172.21.15.39:0/2143424355 shutdown_connections 2024-09-21T07:34:26.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.994+0000 7fb4f6200640 1 -- 172.21.15.39:0/2143424355 wait complete. 2024-09-21T07:34:26.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.997+0000 7fb4f6200640 1 Processor -- start 2024-09-21T07:34:26.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.997+0000 7fb4f6200640 1 -- start start 2024-09-21T07:34:27.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.998+0000 7fb4f6200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4f00ccb30 0x7fb4f0161b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:27.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.998+0000 7fb4f6200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4f00ce020 0x7fb4f0154340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:27.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.998+0000 7fb4f6200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4f00cf6f0 0x7fb4f009cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:27.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.998+0000 7fb4f6200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f00cf340 con 0x7fb4f00cf6f0 2024-09-21T07:34:27.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.998+0000 7fb4f6200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f00cf1c0 con 0x7fb4f00ce020 2024-09-21T07:34:27.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.998+0000 7fb4f6200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb4f00701b0 con 0x7fb4f00ccb30 2024-09-21T07:34:27.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.999+0000 7fb4efe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4f00cf6f0 0x7fb4f009cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:27.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.999+0000 7fb4efe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4f00cf6f0 0x7fb4f009cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52778/0 (socket says 172.21.15.39:52778) 2024-09-21T07:34:27.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.999+0000 7fb4efe00640 1 -- 172.21.15.39:0/3971224763 learned_addr learned my addr 172.21.15.39:0/3971224763 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:27.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.999+0000 7fb4e7e00640 1 --2- 172.21.15.39:0/3971224763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4f00ce020 0x7fb4f0154340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:27.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.999+0000 7fb4ef400640 1 --2- 172.21.15.39:0/3971224763 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4f00ccb30 0x7fb4f0161b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:27.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.999+0000 7fb4efe00640 1 -- 172.21.15.39:0/3971224763 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4f00ccb30 msgr2=0x7fb4f0161b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:27.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.999+0000 7fb4efe00640 1 --2- 172.21.15.39:0/3971224763 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4f00ccb30 0x7fb4f0161b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:27.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.999+0000 7fb4efe00640 1 -- 172.21.15.39:0/3971224763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4f00ce020 msgr2=0x7fb4f0154340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:27.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.999+0000 7fb4efe00640 1 --2- 172.21.15.39:0/3971224763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4f00ce020 0x7fb4f0154340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:27.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.999+0000 7fb4efe00640 1 -- 172.21.15.39:0/3971224763 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4f009aa30 con 0x7fb4f00cf6f0 2024-09-21T07:34:27.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.999+0000 7fb4ef400640 1 --2- 172.21.15.39:0/3971224763 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4f00ccb30 0x7fb4f0161b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:34:27.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:26.999+0000 7fb4efe00640 1 --2- 172.21.15.39:0/3971224763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4f00cf6f0 0x7fb4f009cc90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb4e802b020 tx=0x7fb4e8003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:27.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.000+0000 7fb4e7400640 1 -- 172.21.15.39:0/3971224763 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8033050 con 0x7fb4f00cf6f0 2024-09-21T07:34:27.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.000+0000 7fb4e7400640 1 -- 172.21.15.39:0/3971224763 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb4e8004450 con 0x7fb4f00cf6f0 2024-09-21T07:34:27.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.000+0000 7fb4f6200640 1 -- 172.21.15.39:0/3971224763 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4f0160050 con 0x7fb4f00cf6f0 2024-09-21T07:34:27.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.000+0000 7fb4e7400640 1 -- 172.21.15.39:0/3971224763 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8038450 con 0x7fb4f00cf6f0 2024-09-21T07:34:27.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.000+0000 7fb4f6200640 1 -- 172.21.15.39:0/3971224763 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4f0160610 con 0x7fb4f00cf6f0 2024-09-21T07:34:27.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.001+0000 7fb4f6200640 1 -- 172.21.15.39:0/3971224763 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4f015fdc0 con 0x7fb4f00cf6f0 2024-09-21T07:34:27.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.002+0000 7fb4e7400640 1 -- 172.21.15.39:0/3971224763 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb4e8036070 con 0x7fb4f00cf6f0 2024-09-21T07:34:27.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.003+0000 7fb4e7400640 1 --2- 172.21.15.39:0/3971224763 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb4cc078b90 0x7fb4cc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:27.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.003+0000 7fb4e7400640 1 -- 172.21.15.39:0/3971224763 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6118+0+0 (secure 0 0 0) 0x7fb4e80bc440 con 0x7fb4f00cf6f0 2024-09-21T07:34:27.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.004+0000 7fb4ef400640 1 --2- 172.21.15.39:0/3971224763 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb4cc078b90 0x7fb4cc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:27.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.005+0000 7fb4ef400640 1 --2- 172.21.15.39:0/3971224763 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb4cc078b90 0x7fb4cc07b0a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0005fd0 tx=0x7fb4e000f360 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:27.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.007+0000 7fb4e7400640 1 -- 172.21.15.39:0/3971224763 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb4e80846d0 con 0x7fb4f00cf6f0 2024-09-21T07:34:27.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.207+0000 7fb4f6200640 1 -- 172.21.15.39:0/3971224763 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb4f00b1ad0 con 0x7fb4f00cf6f0 2024-09-21T07:34:27.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.208+0000 7fb4e7400640 1 -- 172.21.15.39:0/3971224763 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+18625 (secure 0 0 0) 0x7fb4f00b1ad0 con 0x7fb4f00cf6f0 2024-09-21T07:34:27.211 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:34:27.211 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":37,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:34:24.252034+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:34:05.671267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-21T07:34:06.613642+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":1,"length":1}]}],"new_removed_snaps":[{"pool":6,"snaps":[{"begin":1,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:34:27.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.211+0000 7fb4f6200640 1 -- 172.21.15.39:0/3971224763 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb4cc078b90 msgr2=0x7fb4cc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:27.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.211+0000 7fb4f6200640 1 --2- 172.21.15.39:0/3971224763 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb4cc078b90 0x7fb4cc07b0a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0005fd0 tx=0x7fb4e000f360 comp rx=0 tx=0).stop 2024-09-21T07:34:27.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.211+0000 7fb4f6200640 1 -- 172.21.15.39:0/3971224763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4f00cf6f0 msgr2=0x7fb4f009cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:27.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.211+0000 7fb4f6200640 1 --2- 172.21.15.39:0/3971224763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4f00cf6f0 0x7fb4f009cc90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb4e802b020 tx=0x7fb4e8003f10 comp rx=0 tx=0).stop 2024-09-21T07:34:27.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.211+0000 7fb4f6200640 1 -- 172.21.15.39:0/3971224763 shutdown_connections 2024-09-21T07:34:27.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.211+0000 7fb4f6200640 1 --2- 172.21.15.39:0/3971224763 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb4cc078b90 0x7fb4cc07b0a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:27.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.211+0000 7fb4f6200640 1 --2- 172.21.15.39:0/3971224763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4f00cf6f0 0x7fb4f009cc90 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:27.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.211+0000 7fb4f6200640 1 --2- 172.21.15.39:0/3971224763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4f00ce020 0x7fb4f0154340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:27.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.211+0000 7fb4f6200640 1 --2- 172.21.15.39:0/3971224763 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4f00ccb30 0x7fb4f0161b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:27.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.211+0000 7fb4f6200640 1 -- 172.21.15.39:0/3971224763 >> 172.21.15.39:0/3971224763 conn(0x7fb4f00b1cb0 msgr2=0x7fb4f015e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:27.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.211+0000 7fb4f6200640 1 -- 172.21.15.39:0/3971224763 shutdown_connections 2024-09-21T07:34:27.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.211+0000 7fb4f6200640 1 -- 172.21.15.39:0/3971224763 wait complete. 2024-09-21T07:34:27.238 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T07:34:27.238 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:34:27.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.463+0000 7f01e8c00640 1 -- 172.21.15.39:0/4099621266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01e40cf660 msgr2=0x7f01e40cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:27.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.463+0000 7f01e8c00640 1 --2- 172.21.15.39:0/4099621266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01e40cf660 0x7f01e40cf010 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f01e40ce460 tx=0x7f01d402a4d0 comp rx=0 tx=0).stop 2024-09-21T07:34:27.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.464+0000 7f01e8c00640 1 -- 172.21.15.39:0/4099621266 shutdown_connections 2024-09-21T07:34:27.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.464+0000 7f01e8c00640 1 --2- 172.21.15.39:0/4099621266 >> v2:172.21.15.148:3300/0 conn(0x7f01e40cd070 0x7f01e40b2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:27.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.464+0000 7f01e8c00640 1 --2- 172.21.15.39:0/4099621266 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f01e40cfb90 0x7f01e40ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:27.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.464+0000 7f01e8c00640 1 --2- 172.21.15.39:0/4099621266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01e40cf660 0x7f01e40cf010 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:27.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.464+0000 7f01e8c00640 1 -- 172.21.15.39:0/4099621266 >> 172.21.15.39:0/4099621266 conn(0x7f01e40b1cb0 msgr2=0x7f01e40b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:27.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.464+0000 7f01e8c00640 1 -- 172.21.15.39:0/4099621266 shutdown_connections 2024-09-21T07:34:27.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.464+0000 7f01e8c00640 1 -- 172.21.15.39:0/4099621266 wait complete. 2024-09-21T07:34:27.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.466+0000 7f01e8c00640 1 Processor -- start 2024-09-21T07:34:27.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.466+0000 7f01e8c00640 1 -- start start 2024-09-21T07:34:27.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.468+0000 7f01e8c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f01e40cd070 0x7f01e41f95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:27.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.468+0000 7f01e8c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01e40cf660 0x7f01e4152d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:27.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.468+0000 7f01e8c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01e40cfb90 0x7f01e4159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:27.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.468+0000 7f01e8c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f01e4070f20 con 0x7f01e40cfb90 2024-09-21T07:34:27.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.468+0000 7f01e8c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f01e40af810 con 0x7f01e40cf660 2024-09-21T07:34:27.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.468+0000 7f01e8c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f01e40710a0 con 0x7f01e40cd070 2024-09-21T07:34:27.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.468+0000 7f01e2800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01e40cfb90 0x7f01e4159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:27.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.468+0000 7f01e2800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01e40cfb90 0x7f01e4159f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52802/0 (socket says 172.21.15.39:52802) 2024-09-21T07:34:27.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.468+0000 7f01e2800640 1 -- 172.21.15.39:0/658684886 learned_addr learned my addr 172.21.15.39:0/658684886 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:27.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.468+0000 7f01e1e00640 1 --2- 172.21.15.39:0/658684886 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f01e40cd070 0x7f01e41f95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:27.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.468+0000 7f01e1400640 1 --2- 172.21.15.39:0/658684886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01e40cf660 0x7f01e4152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:27.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.468+0000 7f01e2800640 1 -- 172.21.15.39:0/658684886 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f01e40cd070 msgr2=0x7f01e41f95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:27.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.468+0000 7f01e2800640 1 --2- 172.21.15.39:0/658684886 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f01e40cd070 0x7f01e41f95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:27.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.468+0000 7f01e2800640 1 -- 172.21.15.39:0/658684886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01e40cf660 msgr2=0x7f01e4152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:27.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.468+0000 7f01e2800640 1 --2- 172.21.15.39:0/658684886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01e40cf660 0x7f01e4152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:27.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.468+0000 7f01e2800640 1 -- 172.21.15.39:0/658684886 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e41f9af0 con 0x7f01e40cfb90 2024-09-21T07:34:27.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.469+0000 7f01e2800640 1 --2- 172.21.15.39:0/658684886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01e40cfb90 0x7f01e4159f40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f01dc00ddc0 tx=0x7f01dc00b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:27.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.469+0000 7f01da000640 1 -- 172.21.15.39:0/658684886 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01dc004d20 con 0x7f01e40cfb90 2024-09-21T07:34:27.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.469+0000 7f01da000640 1 -- 172.21.15.39:0/658684886 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01dc00be40 con 0x7f01e40cfb90 2024-09-21T07:34:27.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.469+0000 7f01e8c00640 1 -- 172.21.15.39:0/658684886 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01e41f5920 con 0x7f01e40cfb90 2024-09-21T07:34:27.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.469+0000 7f01da000640 1 -- 172.21.15.39:0/658684886 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01dc00e8c0 con 0x7f01e40cfb90 2024-09-21T07:34:27.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.469+0000 7f01e8c00640 1 -- 172.21.15.39:0/658684886 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01e41f5ee0 con 0x7f01e40cfb90 2024-09-21T07:34:27.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.470+0000 7f01e8c00640 1 -- 172.21.15.39:0/658684886 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01e41f65a0 con 0x7f01e40cfb90 2024-09-21T07:34:27.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.471+0000 7f01da000640 1 -- 172.21.15.39:0/658684886 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f01dc00ea60 con 0x7f01e40cfb90 2024-09-21T07:34:27.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.473+0000 7f01da000640 1 --2- 172.21.15.39:0/658684886 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f01c4078b90 0x7f01c407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:27.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.473+0000 7f01da000640 1 -- 172.21.15.39:0/658684886 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6118+0+0 (secure 0 0 0) 0x7f01dc09b850 con 0x7f01e40cfb90 2024-09-21T07:34:27.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.473+0000 7f01e1e00640 1 --2- 172.21.15.39:0/658684886 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f01c4078b90 0x7f01c407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:27.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.474+0000 7f01e1e00640 1 --2- 172.21.15.39:0/658684886 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f01c4078b90 0x7f01c407b0a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f01d4033000 tx=0x7f01d4004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:27.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.478+0000 7f01da000640 1 -- 172.21.15.39:0/658684886 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f01e41f65a0 con 0x7f01e40cfb90 2024-09-21T07:34:27.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.714+0000 7f01e8c00640 1 -- 172.21.15.39:0/658684886 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01e40cf010 con 0x7f01e40cfb90 2024-09-21T07:34:27.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.714+0000 7f01da000640 1 -- 172.21.15.39:0/658684886 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5229 (secure 0 0 0) 0x7f01dc06c020 con 0x7f01e40cfb90 2024-09-21T07:34:27.717 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:34:27.717 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":17,"btime":"2024-09-21T07:34:11:169975+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2438395645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2438395645},{"type":"v1","addr":"172.21.15.148:6837","nonce":2438395645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4706,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3668947757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3668947757},{"type":"v1","addr":"172.21.15.39:6835","nonce":3668947757}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4870,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2800764560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2800764560},{"type":"v1","addr":"172.21.15.148:6835","nonce":2800764560}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:34:07.651669+0000","modified":"2024-09-21T07:34:10.675070+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4888":{"gid":4888,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/3524592265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3524592265},{"type":"v1","addr":"172.21.15.39:6834","nonce":3524592265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4888,"qdb_cluster":[4888]},"id":2}]} 2024-09-21T07:34:27.717 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 17 2024-09-21T07:34:27.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.717+0000 7f01e8c00640 1 -- 172.21.15.39:0/658684886 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f01c4078b90 msgr2=0x7f01c407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:27.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.717+0000 7f01e8c00640 1 --2- 172.21.15.39:0/658684886 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f01c4078b90 0x7f01c407b0a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f01d4033000 tx=0x7f01d4004730 comp rx=0 tx=0).stop 2024-09-21T07:34:27.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.717+0000 7f01e8c00640 1 -- 172.21.15.39:0/658684886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01e40cfb90 msgr2=0x7f01e4159f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:27.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.717+0000 7f01e8c00640 1 --2- 172.21.15.39:0/658684886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01e40cfb90 0x7f01e4159f40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f01dc00ddc0 tx=0x7f01dc00b730 comp rx=0 tx=0).stop 2024-09-21T07:34:27.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.717+0000 7f01e8c00640 1 -- 172.21.15.39:0/658684886 shutdown_connections 2024-09-21T07:34:27.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.717+0000 7f01e8c00640 1 --2- 172.21.15.39:0/658684886 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f01c4078b90 0x7f01c407b0a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:27.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.717+0000 7f01e8c00640 1 --2- 172.21.15.39:0/658684886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01e40cfb90 0x7f01e4159f40 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:27.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.717+0000 7f01e8c00640 1 --2- 172.21.15.39:0/658684886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01e40cf660 0x7f01e4152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:27.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.717+0000 7f01e8c00640 1 --2- 172.21.15.39:0/658684886 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f01e40cd070 0x7f01e41f95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:27.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.717+0000 7f01e8c00640 1 -- 172.21.15.39:0/658684886 >> 172.21.15.39:0/658684886 conn(0x7f01e40b1cb0 msgr2=0x7f01e40b14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:27.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.717+0000 7f01e8c00640 1 -- 172.21.15.39:0/658684886 shutdown_connections 2024-09-21T07:34:27.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.717+0000 7f01e8c00640 1 -- 172.21.15.39:0/658684886 wait complete. 2024-09-21T07:34:27.741 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:34:27.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.968+0000 7f685f600640 1 -- 172.21.15.39:0/3864154751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68580706c0 msgr2=0x7f6858072ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:27.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.968+0000 7f685f600640 1 --2- 172.21.15.39:0/3864154751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68580706c0 0x7f6858072ff0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6850005fd0 tx=0x7f685002aa10 comp rx=0 tx=0).stop 2024-09-21T07:34:27.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.968+0000 7f685f600640 1 -- 172.21.15.39:0/3864154751 shutdown_connections 2024-09-21T07:34:27.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.968+0000 7f685f600640 1 --2- 172.21.15.39:0/3864154751 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68580cd3e0 0x7f6858070180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:27.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.968+0000 7f685f600640 1 --2- 172.21.15.39:0/3864154751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68580706c0 0x7f6858072ff0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:27.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.968+0000 7f685f600640 1 -- 172.21.15.39:0/3864154751 >> 172.21.15.39:0/3864154751 conn(0x7f685806d7d0 msgr2=0x7f68580b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:27.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.968+0000 7f685f600640 1 -- 172.21.15.39:0/3864154751 shutdown_connections 2024-09-21T07:34:27.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.968+0000 7f685f600640 1 -- 172.21.15.39:0/3864154751 wait complete. 2024-09-21T07:34:27.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.970+0000 7f685f600640 1 Processor -- start 2024-09-21T07:34:27.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.970+0000 7f685f600640 1 -- start start 2024-09-21T07:34:27.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.972+0000 7f685f600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68580706c0 0x7f6858144c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:27.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.972+0000 7f685f600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68580af780 0x7f6858155550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:27.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.972+0000 7f685f600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68580cd3e0 0x7f6858155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:27.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.972+0000 7f685f600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6858073b70 con 0x7f68580706c0 2024-09-21T07:34:27.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.972+0000 7f685f600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f68580739f0 con 0x7f68580cd3e0 2024-09-21T07:34:27.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.972+0000 7f685f600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f6858073cf0 con 0x7f68580af780 2024-09-21T07:34:27.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.972+0000 7f685cc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68580706c0 0x7f6858144c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:27.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.972+0000 7f685cc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68580706c0 0x7f6858144c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52830/0 (socket says 172.21.15.39:52830) 2024-09-21T07:34:27.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.972+0000 7f685cc00640 1 -- 172.21.15.39:0/1328008417 learned_addr learned my addr 172.21.15.39:0/1328008417 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:27.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.972+0000 7f685d600640 1 --2- 172.21.15.39:0/1328008417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68580cd3e0 0x7f6858155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:27.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.972+0000 7f6857e00640 1 --2- 172.21.15.39:0/1328008417 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68580af780 0x7f6858155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:27.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.972+0000 7f685cc00640 1 -- 172.21.15.39:0/1328008417 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68580af780 msgr2=0x7f6858155550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:27.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.972+0000 7f685cc00640 1 --2- 172.21.15.39:0/1328008417 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68580af780 0x7f6858155550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:27.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.972+0000 7f685cc00640 1 -- 172.21.15.39:0/1328008417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68580cd3e0 msgr2=0x7f6858155a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:27.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.972+0000 7f685cc00640 1 --2- 172.21.15.39:0/1328008417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68580cd3e0 0x7f6858155a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:27.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.972+0000 7f685cc00640 1 -- 172.21.15.39:0/1328008417 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6858158d30 con 0x7f68580706c0 2024-09-21T07:34:27.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.972+0000 7f685d600640 1 --2- 172.21.15.39:0/1328008417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68580cd3e0 0x7f6858155a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:34:27.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.973+0000 7f685cc00640 1 --2- 172.21.15.39:0/1328008417 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68580706c0 0x7f6858144c10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f684c00c3a0 tx=0x7f684c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:27.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.973+0000 7f6854c00640 1 -- 172.21.15.39:0/1328008417 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684c017070 con 0x7f68580706c0 2024-09-21T07:34:27.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.973+0000 7f6854c00640 1 -- 172.21.15.39:0/1328008417 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f684c0053c0 con 0x7f68580706c0 2024-09-21T07:34:27.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.973+0000 7f6854c00640 1 -- 172.21.15.39:0/1328008417 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684c003850 con 0x7f68580706c0 2024-09-21T07:34:27.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.973+0000 7f685f600640 1 -- 172.21.15.39:0/1328008417 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6858159010 con 0x7f68580706c0 2024-09-21T07:34:27.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.973+0000 7f685f600640 1 -- 172.21.15.39:0/1328008417 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6858154d50 con 0x7f68580706c0 2024-09-21T07:34:27.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.974+0000 7f685f600640 1 -- 172.21.15.39:0/1328008417 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6858072ff0 con 0x7f68580706c0 2024-09-21T07:34:27.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.975+0000 7f6854c00640 1 -- 172.21.15.39:0/1328008417 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f684c01a070 con 0x7f68580706c0 2024-09-21T07:34:27.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.976+0000 7f6854c00640 1 --2- 172.21.15.39:0/1328008417 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6838078bc0 0x7f683807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:27.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.976+0000 7f6854c00640 1 -- 172.21.15.39:0/1328008417 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6118+0+0 (secure 0 0 0) 0x7f684c09a310 con 0x7f68580706c0 2024-09-21T07:34:27.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.977+0000 7f6857e00640 1 --2- 172.21.15.39:0/1328008417 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6838078bc0 0x7f683807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:27.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.977+0000 7f6857e00640 1 --2- 172.21.15.39:0/1328008417 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6838078bc0 0x7f683807b0d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f68581592e0 tx=0x7f6844005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:27.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:27.981+0000 7f6854c00640 1 -- 172.21.15.39:0/1328008417 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f684c062510 con 0x7f68580706c0 2024-09-21T07:34:28.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.222+0000 7f685f600640 1 -- 172.21.15.39:0/1328008417 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68580cf1b0 con 0x7f68580706c0 2024-09-21T07:34:28.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.223+0000 7f6854c00640 1 -- 172.21.15.39:0/1328008417 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5229 (secure 0 0 0) 0x7f684c061e10 con 0x7f68580706c0 2024-09-21T07:34:28.225 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:34:28.225 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":17,"btime":"2024-09-21T07:34:11:169975+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2438395645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2438395645},{"type":"v1","addr":"172.21.15.148:6837","nonce":2438395645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4706,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3668947757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3668947757},{"type":"v1","addr":"172.21.15.39:6835","nonce":3668947757}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4870,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2800764560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2800764560},{"type":"v1","addr":"172.21.15.148:6835","nonce":2800764560}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:34:07.651669+0000","modified":"2024-09-21T07:34:10.675070+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4888},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4888":{"gid":4888,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/3524592265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3524592265},{"type":"v1","addr":"172.21.15.39:6834","nonce":3524592265}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4888,"qdb_cluster":[4888]},"id":2}]} 2024-09-21T07:34:28.225 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 17 2024-09-21T07:34:28.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.226+0000 7f685f600640 1 -- 172.21.15.39:0/1328008417 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6838078bc0 msgr2=0x7f683807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:28.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.226+0000 7f685f600640 1 --2- 172.21.15.39:0/1328008417 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6838078bc0 0x7f683807b0d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f68581592e0 tx=0x7f6844005f90 comp rx=0 tx=0).stop 2024-09-21T07:34:28.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.226+0000 7f685f600640 1 -- 172.21.15.39:0/1328008417 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68580706c0 msgr2=0x7f6858144c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:28.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.226+0000 7f685f600640 1 --2- 172.21.15.39:0/1328008417 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68580706c0 0x7f6858144c10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f684c00c3a0 tx=0x7f684c00c860 comp rx=0 tx=0).stop 2024-09-21T07:34:28.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.226+0000 7f685f600640 1 -- 172.21.15.39:0/1328008417 shutdown_connections 2024-09-21T07:34:28.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.226+0000 7f685f600640 1 --2- 172.21.15.39:0/1328008417 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6838078bc0 0x7f683807b0d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:28.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.226+0000 7f685f600640 1 --2- 172.21.15.39:0/1328008417 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68580cd3e0 0x7f6858155a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:28.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.226+0000 7f685f600640 1 --2- 172.21.15.39:0/1328008417 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68580af780 0x7f6858155550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:28.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.226+0000 7f685f600640 1 --2- 172.21.15.39:0/1328008417 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68580706c0 0x7f6858144c10 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:28.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.226+0000 7f685f600640 1 -- 172.21.15.39:0/1328008417 >> 172.21.15.39:0/1328008417 conn(0x7f685806d7d0 msgr2=0x7f68580b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:28.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.226+0000 7f685f600640 1 -- 172.21.15.39:0/1328008417 shutdown_connections 2024-09-21T07:34:28.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.226+0000 7f685f600640 1 -- 172.21.15.39:0/1328008417 wait complete. 2024-09-21T07:34:28.253 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:34:28.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.480+0000 7f7f07200640 1 -- 172.21.15.39:0/1885282136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f08070720 msgr2=0x7f7f081556a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:28.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.480+0000 7f7f07200640 1 --2- 172.21.15.39:0/1885282136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f08070720 0x7f7f081556a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7f081591f0 tx=0x7f7ef002a680 comp rx=0 tx=0).stop 2024-09-21T07:34:28.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.480+0000 7f7f07200640 1 -- 172.21.15.39:0/1885282136 shutdown_connections 2024-09-21T07:34:28.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.480+0000 7f7f07200640 1 --2- 172.21.15.39:0/1885282136 >> v2:172.21.15.148:3300/0 conn(0x7f7f080b0140 0x7f7f08074cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:28.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.480+0000 7f7f07200640 1 --2- 172.21.15.39:0/1885282136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f08070720 0x7f7f081556a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:28.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.480+0000 7f7f07200640 1 -- 172.21.15.39:0/1885282136 >> 172.21.15.39:0/1885282136 conn(0x7f7f080b3090 msgr2=0x7f7f080b34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:28.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.480+0000 7f7f07200640 1 -- 172.21.15.39:0/1885282136 shutdown_connections 2024-09-21T07:34:28.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.480+0000 7f7f07200640 1 -- 172.21.15.39:0/1885282136 wait complete. 2024-09-21T07:34:28.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.482+0000 7f7f07200640 1 Processor -- start 2024-09-21T07:34:28.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.483+0000 7f7f07200640 1 -- start start 2024-09-21T07:34:28.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.484+0000 7f7f07200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f080b0140 0x7f7f080cfe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:28.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.484+0000 7f7f07200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f08162e80 0x7f7f081f5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:28.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.484+0000 7f7f07200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7f081f5b50 0x7f7f08157920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:28.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.484+0000 7f7f07200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0815f3b0 con 0x7f7f080b0140 2024-09-21T07:34:28.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.484+0000 7f7f07200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0815f230 con 0x7f7f08162e80 2024-09-21T07:34:28.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.484+0000 7f7f07200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f7f0815f530 con 0x7f7f081f5b50 2024-09-21T07:34:28.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.484+0000 7f7f05e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f080b0140 0x7f7f080cfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:28.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.484+0000 7f7f05e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f080b0140 0x7f7f080cfe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52848/0 (socket says 172.21.15.39:52848) 2024-09-21T07:34:28.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.484+0000 7f7f05e00640 1 -- 172.21.15.39:0/3197950855 learned_addr learned my addr 172.21.15.39:0/3197950855 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:28.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.484+0000 7f7efd400640 1 --2- 172.21.15.39:0/3197950855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f08162e80 0x7f7f081f5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:28.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.485+0000 7f7f06800640 1 --2- 172.21.15.39:0/3197950855 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7f081f5b50 0x7f7f08157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:28.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.485+0000 7f7f05e00640 1 -- 172.21.15.39:0/3197950855 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7f081f5b50 msgr2=0x7f7f08157920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:28.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.485+0000 7f7f05e00640 1 --2- 172.21.15.39:0/3197950855 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7f081f5b50 0x7f7f08157920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:28.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.485+0000 7f7f05e00640 1 -- 172.21.15.39:0/3197950855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f08162e80 msgr2=0x7f7f081f5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:28.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.485+0000 7f7f05e00640 1 --2- 172.21.15.39:0/3197950855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f08162e80 0x7f7f081f5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:28.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.485+0000 7f7f05e00640 1 -- 172.21.15.39:0/3197950855 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f08155ed0 con 0x7f7f080b0140 2024-09-21T07:34:28.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.485+0000 7f7f06800640 1 --2- 172.21.15.39:0/3197950855 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7f081f5b50 0x7f7f08157920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:34:28.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.485+0000 7f7f05e00640 1 --2- 172.21.15.39:0/3197950855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f080b0140 0x7f7f080cfe60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7ef800c3f0 tx=0x7f7ef800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:28.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.485+0000 7f7efea00640 1 -- 172.21.15.39:0/3197950855 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef8017070 con 0x7f7f080b0140 2024-09-21T07:34:28.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.485+0000 7f7efea00640 1 -- 172.21.15.39:0/3197950855 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ef80035f0 con 0x7f7f080b0140 2024-09-21T07:34:28.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.485+0000 7f7f07200640 1 -- 172.21.15.39:0/3197950855 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f081561b0 con 0x7f7f080b0140 2024-09-21T07:34:28.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.485+0000 7f7efea00640 1 -- 172.21.15.39:0/3197950855 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef8003a40 con 0x7f7f080b0140 2024-09-21T07:34:28.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.486+0000 7f7f07200640 1 -- 172.21.15.39:0/3197950855 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f08156770 con 0x7f7f080b0140 2024-09-21T07:34:28.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.487+0000 7f7f07200640 1 -- 172.21.15.39:0/3197950855 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f0815fa50 con 0x7f7f080b0140 2024-09-21T07:34:28.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.488+0000 7f7efea00640 1 -- 172.21.15.39:0/3197950855 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7ef801a070 con 0x7f7f080b0140 2024-09-21T07:34:28.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.489+0000 7f7efea00640 1 --2- 172.21.15.39:0/3197950855 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7ee4078b90 0x7f7ee407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:28.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.489+0000 7f7efea00640 1 -- 172.21.15.39:0/3197950855 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6118+0+0 (secure 0 0 0) 0x7f7ef809a090 con 0x7f7f080b0140 2024-09-21T07:34:28.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.489+0000 7f7efd400640 1 --2- 172.21.15.39:0/3197950855 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7ee4078b90 0x7f7ee407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:28.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.490+0000 7f7efd400640 1 --2- 172.21.15.39:0/3197950855 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7ee4078b90 0x7f7ee407b0a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0033000 tx=0x7f7ef0004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:28.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.493+0000 7f7efea00640 1 -- 172.21.15.39:0/3197950855 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7ef8061d70 con 0x7f7f080b0140 2024-09-21T07:34:28.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.692+0000 7f7f07200640 1 -- 172.21.15.39:0/3197950855 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7f080d03a0 con 0x7f7f080b0140 2024-09-21T07:34:28.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.694+0000 7f7efea00640 1 -- 172.21.15.39:0/3197950855 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+18625 (secure 0 0 0) 0x7f7ef80667a0 con 0x7f7f080b0140 2024-09-21T07:34:28.696 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:34:28.696 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":37,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:34:24.252034+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:34:05.671267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-21T07:34:06.613642+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":1,"length":1}]}],"new_removed_snaps":[{"pool":6,"snaps":[{"begin":1,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:34:28.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.696+0000 7f7f07200640 1 -- 172.21.15.39:0/3197950855 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7ee4078b90 msgr2=0x7f7ee407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:28.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.696+0000 7f7f07200640 1 --2- 172.21.15.39:0/3197950855 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7ee4078b90 0x7f7ee407b0a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0033000 tx=0x7f7ef0004730 comp rx=0 tx=0).stop 2024-09-21T07:34:28.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.696+0000 7f7f07200640 1 -- 172.21.15.39:0/3197950855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f080b0140 msgr2=0x7f7f080cfe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:28.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.696+0000 7f7f07200640 1 --2- 172.21.15.39:0/3197950855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f080b0140 0x7f7f080cfe60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7ef800c3f0 tx=0x7f7ef800c8b0 comp rx=0 tx=0).stop 2024-09-21T07:34:28.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.696+0000 7f7f07200640 1 -- 172.21.15.39:0/3197950855 shutdown_connections 2024-09-21T07:34:28.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.696+0000 7f7f07200640 1 --2- 172.21.15.39:0/3197950855 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7ee4078b90 0x7f7ee407b0a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:28.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.696+0000 7f7f07200640 1 --2- 172.21.15.39:0/3197950855 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7f081f5b50 0x7f7f08157920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:28.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.696+0000 7f7f07200640 1 --2- 172.21.15.39:0/3197950855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f08162e80 0x7f7f081f5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:28.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.696+0000 7f7f07200640 1 --2- 172.21.15.39:0/3197950855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f080b0140 0x7f7f080cfe60 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:28.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.696+0000 7f7f07200640 1 -- 172.21.15.39:0/3197950855 >> 172.21.15.39:0/3197950855 conn(0x7f7f080b3090 msgr2=0x7f7f080b1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:28.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.696+0000 7f7f07200640 1 -- 172.21.15.39:0/3197950855 shutdown_connections 2024-09-21T07:34:28.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.696+0000 7f7f07200640 1 -- 172.21.15.39:0/3197950855 wait complete. 2024-09-21T07:34:28.724 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T07:34:28.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.951+0000 7f3f16200640 1 -- 172.21.15.39:0/1235324501 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ef80043a0 msgr2=0x7f3ef80048c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:28.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.951+0000 7f3f16200640 1 --2- 172.21.15.39:0/1235324501 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ef80043a0 0x7f3ef80048c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3efc0017c0 tx=0x7f3ef8005830 comp rx=0 tx=0).stop 2024-09-21T07:34:28.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.951+0000 7f3f15800640 1 -- 172.21.15.39:0/1235324501 reap_dead start 2024-09-21T07:34:28.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.951+0000 7f3f16200640 1 -- 172.21.15.39:0/1235324501 shutdown_connections 2024-09-21T07:34:28.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.951+0000 7f3f16200640 1 -- 172.21.15.39:0/1235324501 >> 172.21.15.39:0/1235324501 conn(0x7f3f1006d410 msgr2=0x7f3f100b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:28.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.951+0000 7f3f16200640 1 -- 172.21.15.39:0/1235324501 shutdown_connections 2024-09-21T07:34:28.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.951+0000 7f3f16200640 1 -- 172.21.15.39:0/1235324501 wait complete. 2024-09-21T07:34:28.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.954+0000 7f3f16200640 1 Processor -- start 2024-09-21T07:34:28.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.954+0000 7f3f16200640 1 -- start start 2024-09-21T07:34:28.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.955+0000 7f3f16200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ef80043a0 0x7f3f101ef150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:28.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.955+0000 7f3f16200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3f101421f0 0x7f3f100b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:28.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.955+0000 7f3f16200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f101f8820 0x7f3f101f8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:28.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.955+0000 7f3f16200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1015d700 con 0x7f3ef80043a0 2024-09-21T07:34:28.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.955+0000 7f3f16200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3f100cfc70 con 0x7f3f101f8820 2024-09-21T07:34:28.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.955+0000 7f3f16200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f3f1015d880 con 0x7f3f101421f0 2024-09-21T07:34:28.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.955+0000 7f3f14e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ef80043a0 0x7f3f101ef150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:28.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.955+0000 7f3f14e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ef80043a0 0x7f3f101ef150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52864/0 (socket says 172.21.15.39:52864) 2024-09-21T07:34:28.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.955+0000 7f3f14e00640 1 -- 172.21.15.39:0/80467132 learned_addr learned my addr 172.21.15.39:0/80467132 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:28.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.956+0000 7f3f0fe00640 1 --2- 172.21.15.39:0/80467132 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3f101421f0 0x7f3f100b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:28.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.956+0000 7f3f15800640 1 --2- 172.21.15.39:0/80467132 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f101f8820 0x7f3f101f8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:28.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.956+0000 7f3f14e00640 1 -- 172.21.15.39:0/80467132 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3f101421f0 msgr2=0x7f3f100b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:28.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.956+0000 7f3f14e00640 1 --2- 172.21.15.39:0/80467132 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3f101421f0 0x7f3f100b3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:28.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.956+0000 7f3f14e00640 1 -- 172.21.15.39:0/80467132 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f101f8820 msgr2=0x7f3f101f8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:28.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.956+0000 7f3f14e00640 1 --2- 172.21.15.39:0/80467132 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f101f8820 0x7f3f101f8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:28.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.956+0000 7f3f14e00640 1 -- 172.21.15.39:0/80467132 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f101f9170 con 0x7f3ef80043a0 2024-09-21T07:34:28.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.956+0000 7f3f14e00640 1 --2- 172.21.15.39:0/80467132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ef80043a0 0x7f3f101ef150 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3f04006450 tx=0x7f3f040039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:28.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.956+0000 7f3f0cc00640 1 -- 172.21.15.39:0/80467132 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f04015070 con 0x7f3ef80043a0 2024-09-21T07:34:28.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.957+0000 7f3f0cc00640 1 -- 172.21.15.39:0/80467132 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f04011840 con 0x7f3ef80043a0 2024-09-21T07:34:28.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.957+0000 7f3f16200640 1 -- 172.21.15.39:0/80467132 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f100ada40 con 0x7f3ef80043a0 2024-09-21T07:34:28.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.957+0000 7f3f0cc00640 1 -- 172.21.15.39:0/80467132 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f04011c90 con 0x7f3ef80043a0 2024-09-21T07:34:28.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.957+0000 7f3f16200640 1 -- 172.21.15.39:0/80467132 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f100ae000 con 0x7f3ef80043a0 2024-09-21T07:34:28.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.958+0000 7f3f16200640 1 -- 172.21.15.39:0/80467132 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f1006ee70 con 0x7f3ef80043a0 2024-09-21T07:34:28.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.959+0000 7f3f0cc00640 1 -- 172.21.15.39:0/80467132 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3f04018070 con 0x7f3ef80043a0 2024-09-21T07:34:28.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.960+0000 7f3f0cc00640 1 --2- 172.21.15.39:0/80467132 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3ef0078bc0 0x7f3ef007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:28.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.960+0000 7f3f0cc00640 1 -- 172.21.15.39:0/80467132 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6118+0+0 (secure 0 0 0) 0x7f3f0409bb40 con 0x7f3ef80043a0 2024-09-21T07:34:28.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.960+0000 7f3f0fe00640 1 --2- 172.21.15.39:0/80467132 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3ef0078bc0 0x7f3ef007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:28.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.961+0000 7f3f0fe00640 1 --2- 172.21.15.39:0/80467132 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3ef0078bc0 0x7f3ef007b0d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3efc032000 tx=0x7f3efc02c000 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:28.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:28.965+0000 7f3f0cc00640 1 -- 172.21.15.39:0/80467132 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3f101f9170 con 0x7f3ef80043a0 2024-09-21T07:34:29.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.203+0000 7f3f16200640 1 -- 172.21.15.39:0/80467132 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f3f1006fb50 con 0x7f3ef80043a0 2024-09-21T07:34:29.206 INFO:tasks.ceph.mon.a.smithi039.stderr:2024-09-21T07:34:29.204+0000 7fabab600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T07:34:29.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.206+0000 7f3f0cc00640 1 -- 172.21.15.39:0/80467132 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) ==== 161+0+0 (secure 0 0 0) 0x7f3f04063700 con 0x7f3ef80043a0 2024-09-21T07:34:29.208 INFO:teuthology.orchestra.run.smithi039.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T07:34:29.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.209+0000 7f3f16200640 1 -- 172.21.15.39:0/80467132 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3ef0078bc0 msgr2=0x7f3ef007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:29.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.209+0000 7f3f16200640 1 --2- 172.21.15.39:0/80467132 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3ef0078bc0 0x7f3ef007b0d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3efc032000 tx=0x7f3efc02c000 comp rx=0 tx=0).stop 2024-09-21T07:34:29.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.209+0000 7f3f16200640 1 -- 172.21.15.39:0/80467132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ef80043a0 msgr2=0x7f3f101ef150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:29.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.209+0000 7f3f16200640 1 --2- 172.21.15.39:0/80467132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ef80043a0 0x7f3f101ef150 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3f04006450 tx=0x7f3f040039c0 comp rx=0 tx=0).stop 2024-09-21T07:34:29.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.210+0000 7f3f16200640 1 -- 172.21.15.39:0/80467132 shutdown_connections 2024-09-21T07:34:29.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.210+0000 7f3f16200640 1 --2- 172.21.15.39:0/80467132 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3ef0078bc0 0x7f3ef007b0d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:29.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.210+0000 7f3f16200640 1 --2- 172.21.15.39:0/80467132 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f101f8820 0x7f3f101f8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:29.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.210+0000 7f3f16200640 1 --2- 172.21.15.39:0/80467132 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3f101421f0 0x7f3f100b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:29.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.210+0000 7f3f16200640 1 --2- 172.21.15.39:0/80467132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ef80043a0 0x7f3f101ef150 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:29.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.210+0000 7f3f16200640 1 -- 172.21.15.39:0/80467132 >> 172.21.15.39:0/80467132 conn(0x7f3f1006d410 msgr2=0x7f3f101440d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:29.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.211+0000 7f3f16200640 1 -- 172.21.15.39:0/80467132 shutdown_connections 2024-09-21T07:34:29.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.211+0000 7f3f16200640 1 -- 172.21.15.39:0/80467132 wait complete. 2024-09-21T07:34:29.217 INFO:tasks.ceph.mds.a.smithi039.stderr: -2150> 2024-09-21T07:34:11.931+0000 7f28e3e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:34:29.236 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T07:34:29.340 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-21T07:34:29.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.468+0000 7f3685c00640 1 -- 172.21.15.39:0/4059563526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f368015eeb0 msgr2=0x7f368015f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:29.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.468+0000 7f3685c00640 1 --2- 172.21.15.39:0/4059563526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f368015eeb0 0x7f368015f2c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f36800744f0 tx=0x7f367802a980 comp rx=0 tx=0).stop 2024-09-21T07:34:29.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.468+0000 7f3685c00640 1 -- 172.21.15.39:0/4059563526 shutdown_connections 2024-09-21T07:34:29.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.468+0000 7f3685c00640 1 --2- 172.21.15.39:0/4059563526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f368015eeb0 0x7f368015f2c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:29.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.468+0000 7f3685c00640 1 --2- 172.21.15.39:0/4059563526 >> v2:172.21.15.148:3300/0 conn(0x7f3680073930 0x7f3680073d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:29.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.468+0000 7f3685c00640 1 --2- 172.21.15.39:0/4059563526 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36800cea10 0x7f368015c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:29.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.468+0000 7f3685c00640 1 -- 172.21.15.39:0/4059563526 >> 172.21.15.39:0/4059563526 conn(0x7f36800cd6c0 msgr2=0x7f36800cdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:29.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.468+0000 7f3685c00640 1 -- 172.21.15.39:0/4059563526 shutdown_connections 2024-09-21T07:34:29.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.468+0000 7f3685c00640 1 -- 172.21.15.39:0/4059563526 wait complete. 2024-09-21T07:34:29.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.470+0000 7f3685c00640 1 Processor -- start 2024-09-21T07:34:29.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.470+0000 7f3685c00640 1 -- start start 2024-09-21T07:34:29.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.472+0000 7f3685c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3680073930 0x7f368014e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:29.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.472+0000 7f3685c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36800cea10 0x7f36801662b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:29.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.472+0000 7f3685c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f368015eeb0 0x7f368006cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:29.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.472+0000 7f3685c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f368015f980 con 0x7f36800cea10 2024-09-21T07:34:29.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.472+0000 7f3685c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f368015f800 con 0x7f368015eeb0 2024-09-21T07:34:29.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.472+0000 7f3685c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f368015fb00 con 0x7f3680073930 2024-09-21T07:34:29.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.472+0000 7f367f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36800cea10 0x7f36801662b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:29.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.472+0000 7f367f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36800cea10 0x7f36801662b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52894/0 (socket says 172.21.15.39:52894) 2024-09-21T07:34:29.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.472+0000 7f367f400640 1 -- 172.21.15.39:0/25799570 learned_addr learned my addr 172.21.15.39:0/25799570 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:29.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.472+0000 7f3685200640 1 --2- 172.21.15.39:0/25799570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f368015eeb0 0x7f368006cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:29.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.472+0000 7f367fe00640 1 --2- 172.21.15.39:0/25799570 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3680073930 0x7f368014e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:29.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.473+0000 7f367f400640 1 -- 172.21.15.39:0/25799570 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3680073930 msgr2=0x7f368014e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:29.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.473+0000 7f367f400640 1 --2- 172.21.15.39:0/25799570 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3680073930 0x7f368014e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:29.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.473+0000 7f367f400640 1 -- 172.21.15.39:0/25799570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f368015eeb0 msgr2=0x7f368006cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:29.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.473+0000 7f367f400640 1 --2- 172.21.15.39:0/25799570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f368015eeb0 0x7f368006cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:29.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.473+0000 7f367f400640 1 -- 172.21.15.39:0/25799570 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f368014db20 con 0x7f36800cea10 2024-09-21T07:34:29.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.473+0000 7f367fe00640 1 --2- 172.21.15.39:0/25799570 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3680073930 0x7f368014e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:34:29.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.473+0000 7f367f400640 1 --2- 172.21.15.39:0/25799570 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36800cea10 0x7f36801662b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f366c00e740 tx=0x7f366c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:29.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.473+0000 7f3677e00640 1 -- 172.21.15.39:0/25799570 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366c00ccc0 con 0x7f36800cea10 2024-09-21T07:34:29.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.473+0000 7f3677e00640 1 -- 172.21.15.39:0/25799570 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f366c00ee50 con 0x7f36800cea10 2024-09-21T07:34:29.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.473+0000 7f3685c00640 1 -- 172.21.15.39:0/25799570 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f368014d4b0 con 0x7f36800cea10 2024-09-21T07:34:29.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.473+0000 7f3677e00640 1 -- 172.21.15.39:0/25799570 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366c010670 con 0x7f36800cea10 2024-09-21T07:34:29.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.473+0000 7f3685c00640 1 -- 172.21.15.39:0/25799570 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f368014eb60 con 0x7f36800cea10 2024-09-21T07:34:29.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.474+0000 7f3685c00640 1 -- 172.21.15.39:0/25799570 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f368015c340 con 0x7f36800cea10 2024-09-21T07:34:29.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.477+0000 7f3677e00640 1 -- 172.21.15.39:0/25799570 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f366c010810 con 0x7f36800cea10 2024-09-21T07:34:29.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.478+0000 7f3677e00640 1 --2- 172.21.15.39:0/25799570 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3660078b90 0x7f366007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:29.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.478+0000 7f3677e00640 1 -- 172.21.15.39:0/25799570 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6176+0+0 (secure 0 0 0) 0x7f366c09c0b0 con 0x7f36800cea10 2024-09-21T07:34:29.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.478+0000 7f367fe00640 1 --2- 172.21.15.39:0/25799570 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3660078b90 0x7f366007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:29.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.479+0000 7f367fe00640 1 --2- 172.21.15.39:0/25799570 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3660078b90 0x7f366007b0a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3680073310 tx=0x7f3670005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:29.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.480+0000 7f3677e00640 1 -- 172.21.15.39:0/25799570 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f366c063cd0 con 0x7f36800cea10 2024-09-21T07:34:29.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:29.722+0000 7f3685c00640 1 -- 172.21.15.39:0/25799570 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f368014d900 con 0x7f36800cea10 2024-09-21T07:34:29.929 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:34:29.933 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:34:30.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.294+0000 7f3677e00640 1 -- 172.21.15.39:0/25799570 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7f366c068700 con 0x7f36800cea10 2024-09-21T07:34:30.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.296+0000 7f3685c00640 1 -- 172.21.15.39:0/25799570 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3660078b90 msgr2=0x7f366007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:30.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.296+0000 7f3685c00640 1 --2- 172.21.15.39:0/25799570 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3660078b90 0x7f366007b0a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3680073310 tx=0x7f3670005f90 comp rx=0 tx=0).stop 2024-09-21T07:34:30.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.297+0000 7f3685c00640 1 -- 172.21.15.39:0/25799570 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36800cea10 msgr2=0x7f36801662b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:30.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.297+0000 7f3685c00640 1 --2- 172.21.15.39:0/25799570 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36800cea10 0x7f36801662b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f366c00e740 tx=0x7f366c00ec00 comp rx=0 tx=0).stop 2024-09-21T07:34:30.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.297+0000 7f3685c00640 1 -- 172.21.15.39:0/25799570 shutdown_connections 2024-09-21T07:34:30.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.297+0000 7f3685c00640 1 --2- 172.21.15.39:0/25799570 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3660078b90 0x7f366007b0a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:30.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.297+0000 7f3685c00640 1 --2- 172.21.15.39:0/25799570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f368015eeb0 0x7f368006cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:30.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.297+0000 7f3685c00640 1 --2- 172.21.15.39:0/25799570 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36800cea10 0x7f36801662b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:30.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.297+0000 7f3685c00640 1 --2- 172.21.15.39:0/25799570 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3680073930 0x7f368014e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:30.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.297+0000 7f3685c00640 1 -- 172.21.15.39:0/25799570 >> 172.21.15.39:0/25799570 conn(0x7f36800cd6c0 msgr2=0x7f36800b4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:30.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.297+0000 7f3685c00640 1 -- 172.21.15.39:0/25799570 shutdown_connections 2024-09-21T07:34:30.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.297+0000 7f3685c00640 1 -- 172.21.15.39:0/25799570 wait complete. 2024-09-21T07:34:30.323 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T07:34:30.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.551+0000 7f0c54600640 1 -- 172.21.15.39:0/3157239422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c4c06fe50 msgr2=0x7f0c4c0caf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:30.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.551+0000 7f0c54600640 1 --2- 172.21.15.39:0/3157239422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c4c06fe50 0x7f0c4c0caf30 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c0b2b90 tx=0x7f0c4402a820 comp rx=0 tx=0).stop 2024-09-21T07:34:30.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.551+0000 7f0c54600640 1 -- 172.21.15.39:0/3157239422 shutdown_connections 2024-09-21T07:34:30.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.551+0000 7f0c54600640 1 --2- 172.21.15.39:0/3157239422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c4c06fe50 0x7f0c4c0caf30 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:30.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.551+0000 7f0c54600640 1 --2- 172.21.15.39:0/3157239422 >> v2:172.21.15.148:3300/0 conn(0x7f0c4c0cc590 0x7f0c4c0cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:30.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.551+0000 7f0c54600640 1 -- 172.21.15.39:0/3157239422 >> 172.21.15.39:0/3157239422 conn(0x7f0c4c0b1c40 msgr2=0x7f0c4c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:30.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.551+0000 7f0c54600640 1 -- 172.21.15.39:0/3157239422 shutdown_connections 2024-09-21T07:34:30.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.551+0000 7f0c54600640 1 -- 172.21.15.39:0/3157239422 wait complete. 2024-09-21T07:34:30.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.553+0000 7f0c54600640 1 Processor -- start 2024-09-21T07:34:30.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.553+0000 7f0c54600640 1 -- start start 2024-09-21T07:34:30.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.555+0000 7f0c54600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c4c06fe50 0x7f0c4c15b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:30.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.555+0000 7f0c54600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c4c0cc590 0x7f0c4c1f82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:30.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.555+0000 7f0c54600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c4c0ccfb0 0x7f0c4c1f6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:30.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.555+0000 7f0c54600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4c0cb5f0 con 0x7f0c4c0ccfb0 2024-09-21T07:34:30.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.555+0000 7f0c54600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4c0cb470 con 0x7f0c4c06fe50 2024-09-21T07:34:30.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.555+0000 7f0c54600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f0c4c0cb770 con 0x7f0c4c0cc590 2024-09-21T07:34:30.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.555+0000 7f0c52600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c4c0ccfb0 0x7f0c4c1f6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:30.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.555+0000 7f0c52600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c4c0ccfb0 0x7f0c4c1f6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52910/0 (socket says 172.21.15.39:52910) 2024-09-21T07:34:30.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.555+0000 7f0c52600640 1 -- 172.21.15.39:0/3093137765 learned_addr learned my addr 172.21.15.39:0/3093137765 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:30.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.555+0000 7f0c51200640 1 --2- 172.21.15.39:0/3093137765 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c4c0cc590 0x7f0c4c1f82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:30.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.555+0000 7f0c51c00640 1 --2- 172.21.15.39:0/3093137765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c4c06fe50 0x7f0c4c15b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:30.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.556+0000 7f0c52600640 1 -- 172.21.15.39:0/3093137765 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c4c0cc590 msgr2=0x7f0c4c1f82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:30.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.556+0000 7f0c52600640 1 --2- 172.21.15.39:0/3093137765 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c4c0cc590 0x7f0c4c1f82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:30.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.556+0000 7f0c52600640 1 -- 172.21.15.39:0/3093137765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c4c06fe50 msgr2=0x7f0c4c15b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:30.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.556+0000 7f0c52600640 1 --2- 172.21.15.39:0/3093137765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c4c06fe50 0x7f0c4c15b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:30.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.556+0000 7f0c52600640 1 -- 172.21.15.39:0/3093137765 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c4c15bd80 con 0x7f0c4c0ccfb0 2024-09-21T07:34:30.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.556+0000 7f0c51c00640 1 --2- 172.21.15.39:0/3093137765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c4c06fe50 0x7f0c4c15b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:34:30.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.556+0000 7f0c52600640 1 --2- 172.21.15.39:0/3093137765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c4c0ccfb0 0x7f0c4c1f6660 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0c4402ae10 tx=0x7f0c44032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:30.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.556+0000 7f0c49600640 1 -- 172.21.15.39:0/3093137765 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c44037040 con 0x7f0c4c0ccfb0 2024-09-21T07:34:30.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.556+0000 7f0c49600640 1 -- 172.21.15.39:0/3093137765 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c44042070 con 0x7f0c4c0ccfb0 2024-09-21T07:34:30.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.556+0000 7f0c49600640 1 -- 172.21.15.39:0/3093137765 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c44003e60 con 0x7f0c4c0ccfb0 2024-09-21T07:34:30.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.556+0000 7f0c54600640 1 -- 172.21.15.39:0/3093137765 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c4c1f8820 con 0x7f0c4c0ccfb0 2024-09-21T07:34:30.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.556+0000 7f0c54600640 1 -- 172.21.15.39:0/3093137765 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c4c1f7520 con 0x7f0c4c0ccfb0 2024-09-21T07:34:30.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.557+0000 7f0c54600640 1 -- 172.21.15.39:0/3093137765 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c4c1f6fb0 con 0x7f0c4c0ccfb0 2024-09-21T07:34:30.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.559+0000 7f0c49600640 1 -- 172.21.15.39:0/3093137765 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0c440042a0 con 0x7f0c4c0ccfb0 2024-09-21T07:34:30.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.560+0000 7f0c49600640 1 --2- 172.21.15.39:0/3093137765 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0c2c078bc0 0x7f0c2c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:30.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.560+0000 7f0c49600640 1 -- 172.21.15.39:0/3093137765 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6176+0+0 (secure 0 0 0) 0x7f0c440bbd70 con 0x7f0c4c0ccfb0 2024-09-21T07:34:30.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.560+0000 7f0c51c00640 1 --2- 172.21.15.39:0/3093137765 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0c2c078bc0 0x7f0c2c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:30.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.561+0000 7f0c51c00640 1 --2- 172.21.15.39:0/3093137765 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0c2c078bc0 0x7f0c2c07b0d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0c40005fd0 tx=0x7f0c4000f360 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:30.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.565+0000 7f0c49600640 1 -- 172.21.15.39:0/3093137765 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0c4c15bd80 con 0x7f0c4c0ccfb0 2024-09-21T07:34:30.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:30.761+0000 7f0c54600640 1 -- 172.21.15.39:0/3093137765 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0c4c0b10d0 con 0x7f0c4c0ccfb0 2024-09-21T07:34:31.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.309+0000 7f0c49600640 1 -- 172.21.15.39:0/3093137765 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v39) ==== 181+0+0 (secure 0 0 0) 0x7f0c44083f60 con 0x7f0c4c0ccfb0 2024-09-21T07:34:31.312 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' removed 2024-09-21T07:34:31.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.312+0000 7f0c54600640 1 -- 172.21.15.39:0/3093137765 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0c2c078bc0 msgr2=0x7f0c2c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:31.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.312+0000 7f0c54600640 1 --2- 172.21.15.39:0/3093137765 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0c2c078bc0 0x7f0c2c07b0d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0c40005fd0 tx=0x7f0c4000f360 comp rx=0 tx=0).stop 2024-09-21T07:34:31.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.312+0000 7f0c54600640 1 -- 172.21.15.39:0/3093137765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c4c0ccfb0 msgr2=0x7f0c4c1f6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:31.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.312+0000 7f0c54600640 1 --2- 172.21.15.39:0/3093137765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c4c0ccfb0 0x7f0c4c1f6660 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0c4402ae10 tx=0x7f0c44032d10 comp rx=0 tx=0).stop 2024-09-21T07:34:31.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.312+0000 7f0c54600640 1 -- 172.21.15.39:0/3093137765 shutdown_connections 2024-09-21T07:34:31.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.312+0000 7f0c54600640 1 --2- 172.21.15.39:0/3093137765 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0c2c078bc0 0x7f0c2c07b0d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:31.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.312+0000 7f0c54600640 1 --2- 172.21.15.39:0/3093137765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c4c0ccfb0 0x7f0c4c1f6660 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:31.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.312+0000 7f0c54600640 1 --2- 172.21.15.39:0/3093137765 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c4c0cc590 0x7f0c4c1f82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:31.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.312+0000 7f0c54600640 1 --2- 172.21.15.39:0/3093137765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c4c06fe50 0x7f0c4c15b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:31.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.312+0000 7f0c54600640 1 -- 172.21.15.39:0/3093137765 >> 172.21.15.39:0/3093137765 conn(0x7f0c4c0b1c40 msgr2=0x7f0c4c074500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:31.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.312+0000 7f0c54600640 1 -- 172.21.15.39:0/3093137765 shutdown_connections 2024-09-21T07:34:31.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.312+0000 7f0c54600640 1 -- 172.21.15.39:0/3093137765 wait complete. 2024-09-21T07:34:31.340 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T07:34:31.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.572+0000 7f7440200640 1 -- 172.21.15.39:0/2182213529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74200053a0 msgr2=0x7f742000ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:31.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.572+0000 7f7440200640 1 --2- 172.21.15.39:0/2182213529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74200053a0 0x7f742000ba50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7430005fd0 tx=0x7f7420005830 comp rx=0 tx=0).stop 2024-09-21T07:34:31.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.572+0000 7f7440200640 1 -- 172.21.15.39:0/2182213529 shutdown_connections 2024-09-21T07:34:31.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.572+0000 7f7440200640 1 --2- 172.21.15.39:0/2182213529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74200053a0 0x7f742000ba50 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:31.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.572+0000 7f7440200640 1 -- 172.21.15.39:0/2182213529 >> 172.21.15.39:0/2182213529 conn(0x7f74380b1c40 msgr2=0x7f74380b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:31.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.573+0000 7f7440200640 1 -- 172.21.15.39:0/2182213529 shutdown_connections 2024-09-21T07:34:31.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.573+0000 7f7440200640 1 -- 172.21.15.39:0/2182213529 wait complete. 2024-09-21T07:34:31.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.575+0000 7f7440200640 1 Processor -- start 2024-09-21T07:34:31.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.575+0000 7f7440200640 1 -- start start 2024-09-21T07:34:31.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.576+0000 7f7440200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f74200053a0 0x7f74380d0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:31.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.576+0000 7f7440200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74380d0db0 0x7f74380c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:31.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.576+0000 7f7440200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f743815b920 0x7f743815c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:31.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.576+0000 7f7440200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f74380b30c0 con 0x7f74380d0db0 2024-09-21T07:34:31.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.576+0000 7f7440200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f74380b2f40 con 0x7f743815b920 2024-09-21T07:34:31.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.576+0000 7f7440200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f74380b3240 con 0x7f74200053a0 2024-09-21T07:34:31.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.577+0000 7f743ce00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74380d0db0 0x7f74380c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:31.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.577+0000 7f743ce00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74380d0db0 0x7f74380c8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52928/0 (socket says 172.21.15.39:52928) 2024-09-21T07:34:31.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.577+0000 7f743ce00640 1 -- 172.21.15.39:0/2983216058 learned_addr learned my addr 172.21.15.39:0/2983216058 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:31.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.577+0000 7f743d800640 1 --2- 172.21.15.39:0/2983216058 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f74200053a0 0x7f74380d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:31.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.577+0000 7f743e200640 1 --2- 172.21.15.39:0/2983216058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f743815b920 0x7f743815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:31.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.577+0000 7f743ce00640 1 -- 172.21.15.39:0/2983216058 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f74200053a0 msgr2=0x7f74380d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:31.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.577+0000 7f743ce00640 1 --2- 172.21.15.39:0/2983216058 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f74200053a0 0x7f74380d0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:31.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.577+0000 7f743ce00640 1 -- 172.21.15.39:0/2983216058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f743815b920 msgr2=0x7f743815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:31.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.577+0000 7f743ce00640 1 --2- 172.21.15.39:0/2983216058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f743815b920 0x7f743815c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:31.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.577+0000 7f743ce00640 1 -- 172.21.15.39:0/2983216058 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74381f8be0 con 0x7f74380d0db0 2024-09-21T07:34:31.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.577+0000 7f743ce00640 1 --2- 172.21.15.39:0/2983216058 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74380d0db0 0x7f74380c8bc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7424004d60 tx=0x7f7424001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:31.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.578+0000 7f7435600640 1 -- 172.21.15.39:0/2983216058 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74240107d0 con 0x7f74380d0db0 2024-09-21T07:34:31.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.578+0000 7f7435600640 1 -- 172.21.15.39:0/2983216058 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7424004e50 con 0x7f74380d0db0 2024-09-21T07:34:31.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.578+0000 7f7435600640 1 -- 172.21.15.39:0/2983216058 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7424010ca0 con 0x7f74380d0db0 2024-09-21T07:34:31.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.578+0000 7f7440200640 1 -- 172.21.15.39:0/2983216058 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74381f8ec0 con 0x7f74380d0db0 2024-09-21T07:34:31.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.578+0000 7f7440200640 1 -- 172.21.15.39:0/2983216058 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74381f9480 con 0x7f74380d0db0 2024-09-21T07:34:31.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.579+0000 7f7440200640 1 -- 172.21.15.39:0/2983216058 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74381f8950 con 0x7f74380d0db0 2024-09-21T07:34:31.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.580+0000 7f7435600640 1 -- 172.21.15.39:0/2983216058 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f742401c470 con 0x7f74380d0db0 2024-09-21T07:34:31.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.581+0000 7f7435600640 1 --2- 172.21.15.39:0/2983216058 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7418078b90 0x7f741807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:31.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.581+0000 7f7435600640 1 -- 172.21.15.39:0/2983216058 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5738+0+0 (secure 0 0 0) 0x7f742409b7a0 con 0x7f74380d0db0 2024-09-21T07:34:31.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.582+0000 7f743d800640 1 --2- 172.21.15.39:0/2983216058 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7418078b90 0x7f741807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:31.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.583+0000 7f743d800640 1 --2- 172.21.15.39:0/2983216058 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7418078b90 0x7f741807b0a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f742c001010 tx=0x7f742c0067a0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:31.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.586+0000 7f7435600640 1 -- 172.21.15.39:0/2983216058 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7424063b20 con 0x7f74380d0db0 2024-09-21T07:34:31.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:31.790+0000 7f7440200640 1 -- 172.21.15.39:0/2983216058 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7438072ff0 con 0x7f74380d0db0 2024-09-21T07:34:32.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.351+0000 7f7435600640 1 -- 172.21.15.39:0/2983216058 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v41) ==== 169+0+0 (secure 0 0 0) 0x7f7424063420 con 0x7f74380d0db0 2024-09-21T07:34:32.353 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' removed 2024-09-21T07:34:32.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.355+0000 7f7440200640 1 -- 172.21.15.39:0/2983216058 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7418078b90 msgr2=0x7f741807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:32.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.355+0000 7f7440200640 1 --2- 172.21.15.39:0/2983216058 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7418078b90 0x7f741807b0a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f742c001010 tx=0x7f742c0067a0 comp rx=0 tx=0).stop 2024-09-21T07:34:32.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.355+0000 7f7440200640 1 -- 172.21.15.39:0/2983216058 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74380d0db0 msgr2=0x7f74380c8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:32.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.355+0000 7f7440200640 1 --2- 172.21.15.39:0/2983216058 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74380d0db0 0x7f74380c8bc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7424004d60 tx=0x7f7424001630 comp rx=0 tx=0).stop 2024-09-21T07:34:32.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.356+0000 7f7440200640 1 -- 172.21.15.39:0/2983216058 shutdown_connections 2024-09-21T07:34:32.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.356+0000 7f7440200640 1 --2- 172.21.15.39:0/2983216058 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7418078b90 0x7f741807b0a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:32.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.356+0000 7f7440200640 1 --2- 172.21.15.39:0/2983216058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f743815b920 0x7f743815c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:32.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.356+0000 7f7440200640 1 --2- 172.21.15.39:0/2983216058 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74380d0db0 0x7f74380c8bc0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:32.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.356+0000 7f7440200640 1 --2- 172.21.15.39:0/2983216058 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f74200053a0 0x7f74380d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:32.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.356+0000 7f7440200640 1 -- 172.21.15.39:0/2983216058 >> 172.21.15.39:0/2983216058 conn(0x7f74380b1c40 msgr2=0x7f743806f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:32.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.357+0000 7f7440200640 1 -- 172.21.15.39:0/2983216058 shutdown_connections 2024-09-21T07:34:32.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.357+0000 7f7440200640 1 -- 172.21.15.39:0/2983216058 wait complete. 2024-09-21T07:34:32.386 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T07:34:32.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.615+0000 7fb127e00640 1 -- 172.21.15.39:0/16012279 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb10c004800 msgr2=0x7fb10c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:32.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.615+0000 7fb127e00640 1 --2- 172.21.15.39:0/16012279 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb10c004800 0x7fb10c004c10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb1100016c0 tx=0x7fb10c005830 comp rx=0 tx=0).stop 2024-09-21T07:34:32.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.615+0000 7fb127e00640 1 -- 172.21.15.39:0/16012279 shutdown_connections 2024-09-21T07:34:32.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.615+0000 7fb127e00640 1 --2- 172.21.15.39:0/16012279 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb10c004800 0x7fb10c004c10 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:32.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.615+0000 7fb127e00640 1 -- 172.21.15.39:0/16012279 >> 172.21.15.39:0/16012279 conn(0x7fb1280b3be0 msgr2=0x7fb1280b3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:32.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.615+0000 7fb127e00640 1 -- 172.21.15.39:0/16012279 shutdown_connections 2024-09-21T07:34:32.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.615+0000 7fb127e00640 1 -- 172.21.15.39:0/16012279 wait complete. 2024-09-21T07:34:32.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.618+0000 7fb127e00640 1 Processor -- start 2024-09-21T07:34:32.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.618+0000 7fb127e00640 1 -- start start 2024-09-21T07:34:32.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.619+0000 7fb127e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb10c004800 0x7fb12806f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:32.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.619+0000 7fb127e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb1280aedd0 0x7fb12814c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:32.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.619+0000 7fb127e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1280b07a0 0x7fb1280b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:32.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.619+0000 7fb127e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb1280b4f40 con 0x7fb10c004800 2024-09-21T07:34:32.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.619+0000 7fb127e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb1280b49c0 con 0x7fb1280b07a0 2024-09-21T07:34:32.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.619+0000 7fb127e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb1280b50c0 con 0x7fb1280aedd0 2024-09-21T07:34:32.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.619+0000 7fb126a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb10c004800 0x7fb12806f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:32.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.619+0000 7fb126a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb10c004800 0x7fb12806f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57396/0 (socket says 172.21.15.39:57396) 2024-09-21T07:34:32.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.619+0000 7fb126a00640 1 -- 172.21.15.39:0/976772536 learned_addr learned my addr 172.21.15.39:0/976772536 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:32.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.620+0000 7fb11fe00640 1 --2- 172.21.15.39:0/976772536 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb1280aedd0 0x7fb12814c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:32.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.620+0000 7fb127400640 1 --2- 172.21.15.39:0/976772536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1280b07a0 0x7fb1280b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:32.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.620+0000 7fb126a00640 1 -- 172.21.15.39:0/976772536 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb1280aedd0 msgr2=0x7fb12814c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:32.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.620+0000 7fb126a00640 1 --2- 172.21.15.39:0/976772536 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb1280aedd0 0x7fb12814c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:32.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.620+0000 7fb126a00640 1 -- 172.21.15.39:0/976772536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1280b07a0 msgr2=0x7fb1280b0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:32.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.620+0000 7fb126a00640 1 --2- 172.21.15.39:0/976772536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1280b07a0 0x7fb1280b0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:32.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.620+0000 7fb126a00640 1 -- 172.21.15.39:0/976772536 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1280b10f0 con 0x7fb10c004800 2024-09-21T07:34:32.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.620+0000 7fb126a00640 1 --2- 172.21.15.39:0/976772536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb10c004800 0x7fb12806f8f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb118001b50 tx=0x7fb1180039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:32.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.620+0000 7fb11ea00640 1 -- 172.21.15.39:0/976772536 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb118015070 con 0x7fb10c004800 2024-09-21T07:34:32.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.620+0000 7fb11ea00640 1 -- 172.21.15.39:0/976772536 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb118011900 con 0x7fb10c004800 2024-09-21T07:34:32.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.620+0000 7fb11ea00640 1 -- 172.21.15.39:0/976772536 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb118011d50 con 0x7fb10c004800 2024-09-21T07:34:32.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.620+0000 7fb127e00640 1 -- 172.21.15.39:0/976772536 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1281f87c0 con 0x7fb10c004800 2024-09-21T07:34:32.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.621+0000 7fb127e00640 1 -- 172.21.15.39:0/976772536 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1281f8d80 con 0x7fb10c004800 2024-09-21T07:34:32.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.622+0000 7fb127e00640 1 -- 172.21.15.39:0/976772536 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb12809aa30 con 0x7fb10c004800 2024-09-21T07:34:32.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.623+0000 7fb11ea00640 1 -- 172.21.15.39:0/976772536 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb118018070 con 0x7fb10c004800 2024-09-21T07:34:32.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.624+0000 7fb11ea00640 1 --2- 172.21.15.39:0/976772536 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb104078b90 0x7fb10407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:32.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.624+0000 7fb11ea00640 1 -- 172.21.15.39:0/976772536 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5296+0+0 (secure 0 0 0) 0x7fb11809b8c0 con 0x7fb10c004800 2024-09-21T07:34:32.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.624+0000 7fb11fe00640 1 --2- 172.21.15.39:0/976772536 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb104078b90 0x7fb10407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:32.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.625+0000 7fb11fe00640 1 --2- 172.21.15.39:0/976772536 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb104078b90 0x7fb10407b0a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb110032000 tx=0x7fb110001190 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:32.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.628+0000 7fb11ea00640 1 -- 172.21.15.39:0/976772536 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb1280b10f0 con 0x7fb10c004800 2024-09-21T07:34:32.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.827+0000 7fb127e00640 1 -- 172.21.15.39:0/976772536 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7fb1281e7ba0 con 0x7fb10c004800 2024-09-21T07:34:32.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.831+0000 7fb11ea00640 1 -- 172.21.15.39:0/976772536 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v34) ==== 71+0+0 (secure 0 0 0) 0x7fb118063e20 con 0x7fb10c004800 2024-09-21T07:34:32.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.834+0000 7fb127e00640 1 -- 172.21.15.39:0/976772536 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb104078b90 msgr2=0x7fb10407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:32.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.834+0000 7fb127e00640 1 --2- 172.21.15.39:0/976772536 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb104078b90 0x7fb10407b0a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb110032000 tx=0x7fb110001190 comp rx=0 tx=0).stop 2024-09-21T07:34:32.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.834+0000 7fb127e00640 1 -- 172.21.15.39:0/976772536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb10c004800 msgr2=0x7fb12806f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:32.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.834+0000 7fb127e00640 1 --2- 172.21.15.39:0/976772536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb10c004800 0x7fb12806f8f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb118001b50 tx=0x7fb1180039c0 comp rx=0 tx=0).stop 2024-09-21T07:34:32.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.834+0000 7fb127e00640 1 -- 172.21.15.39:0/976772536 shutdown_connections 2024-09-21T07:34:32.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.834+0000 7fb127e00640 1 --2- 172.21.15.39:0/976772536 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb104078b90 0x7fb10407b0a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:32.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.834+0000 7fb127e00640 1 --2- 172.21.15.39:0/976772536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb1280b07a0 0x7fb1280b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:32.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.834+0000 7fb127e00640 1 --2- 172.21.15.39:0/976772536 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb1280aedd0 0x7fb12814c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:32.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.834+0000 7fb127e00640 1 --2- 172.21.15.39:0/976772536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb10c004800 0x7fb12806f8f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:32.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.834+0000 7fb127e00640 1 -- 172.21.15.39:0/976772536 >> 172.21.15.39:0/976772536 conn(0x7fb1280b3be0 msgr2=0x7fb12815f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:32.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.834+0000 7fb127e00640 1 -- 172.21.15.39:0/976772536 shutdown_connections 2024-09-21T07:34:32.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:32.834+0000 7fb127e00640 1 -- 172.21.15.39:0/976772536 wait complete. 2024-09-21T07:34:32.862 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_attaching_pools_with_snaps_to_fs_fails' 2024-09-21T07:34:33.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:33.093+0000 7f5655c00640 1 -- 172.21.15.39:0/40978908 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5650070030 msgr2=0x7f5650074640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:33.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:33.093+0000 7f5655c00640 1 --2- 172.21.15.39:0/40978908 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5650070030 0x7f5650074640 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f563c009830 tx=0x7f563c02a5c0 comp rx=0 tx=0).stop 2024-09-21T07:34:33.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:33.094+0000 7f5655c00640 1 -- 172.21.15.39:0/40978908 shutdown_connections 2024-09-21T07:34:33.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:33.094+0000 7f5655c00640 1 --2- 172.21.15.39:0/40978908 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5650070030 0x7f5650074640 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:33.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:33.094+0000 7f5655c00640 1 -- 172.21.15.39:0/40978908 >> 172.21.15.39:0/40978908 conn(0x7f56500b1cb0 msgr2=0x7f56500b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:33.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:33.094+0000 7f5655c00640 1 -- 172.21.15.39:0/40978908 shutdown_connections 2024-09-21T07:34:33.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:33.094+0000 7f5655c00640 1 -- 172.21.15.39:0/40978908 wait complete. 2024-09-21T07:34:33.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:33.096+0000 7f5655c00640 1 Processor -- start 2024-09-21T07:34:33.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:33.096+0000 7f5655c00640 1 -- start start 2024-09-21T07:34:33.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:33.097+0000 7f5655c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5650070030 0x7f565014f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:33.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:33.098+0000 7f5655c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5650070440 0x7f56500cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:33.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:33.098+0000 7f5655c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5650073170 0x7f56500cb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:33.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:33.098+0000 7f5655c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f56500b3c40 con 0x7f5650073170 2024-09-21T07:34:33.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:33.098+0000 7f5655c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f56500b3ac0 con 0x7f5650070030 2024-09-21T07:34:33.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:33.098+0000 7f5655c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f56500b3dc0 con 0x7f5650070440 2024-09-21T07:34:33.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:33.098+0000 7f564f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5650073170 0x7f56500cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:33.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:33.098+0000 7f564f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5650073170 0x7f56500cb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57406/0 (socket says 172.21.15.39:57406) 2024-09-21T07:34:33.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:33.098+0000 7f564f400640 1 -- 172.21.15.39:0/1012173440 learned_addr learned my addr 172.21.15.39:0/1012173440 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:33.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:33.098+0000 7f564ea00640 1 --2- 172.21.15.39:0/1012173440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5650070030 0x7f565014f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:33.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:33.098+0000 7f564e000640 1 --2- 172.21.15.39:0/1012173440 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5650070440 0x7f56500cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:33.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:33.098+0000 7f564f400640 1 -- 172.21.15.39:0/1012173440 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5650070440 msgr2=0x7f56500cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:33.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:33.098+0000 7f564f400640 1 --2- 172.21.15.39:0/1012173440 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5650070440 0x7f56500cb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:33.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:33.098+0000 7f564f400640 1 -- 172.21.15.39:0/1012173440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5650070030 msgr2=0x7f565014f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:33.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:33.098+0000 7f564f400640 1 --2- 172.21.15.39:0/1012173440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5650070030 0x7f565014f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:33.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:33.098+0000 7f564f400640 1 -- 172.21.15.39:0/1012173440 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f565015a5d0 con 0x7f5650073170 2024-09-21T07:34:33.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:33.099+0000 7f564f400640 1 --2- 172.21.15.39:0/1012173440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5650073170 0x7f56500cb630 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f564800c760 tx=0x7f564800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:33.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:33.099+0000 7f5646a00640 1 -- 172.21.15.39:0/1012173440 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5648003da0 con 0x7f5650073170 2024-09-21T07:34:33.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:33.099+0000 7f5646a00640 1 -- 172.21.15.39:0/1012173440 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f564800ce10 con 0x7f5650073170 2024-09-21T07:34:33.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:33.099+0000 7f5646a00640 1 -- 172.21.15.39:0/1012173440 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56480052d0 con 0x7f5650073170 2024-09-21T07:34:33.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:33.099+0000 7f5655c00640 1 -- 172.21.15.39:0/1012173440 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f565015a8b0 con 0x7f5650073170 2024-09-21T07:34:33.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:33.099+0000 7f5655c00640 1 -- 172.21.15.39:0/1012173440 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f565015ae70 con 0x7f5650073170 2024-09-21T07:34:33.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:33.100+0000 7f5655c00640 1 -- 172.21.15.39:0/1012173440 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56500b4300 con 0x7f5650073170 2024-09-21T07:34:33.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:33.101+0000 7f5646a00640 1 -- 172.21.15.39:0/1012173440 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5648005470 con 0x7f5650073170 2024-09-21T07:34:33.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:33.103+0000 7f5646a00640 1 --2- 172.21.15.39:0/1012173440 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5630078b90 0x7f563007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:33.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:33.103+0000 7f5646a00640 1 -- 172.21.15.39:0/1012173440 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5296+0+0 (secure 0 0 0) 0x7f564809a810 con 0x7f5650073170 2024-09-21T07:34:33.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:33.103+0000 7f564ea00640 1 --2- 172.21.15.39:0/1012173440 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5630078b90 0x7f563007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:33.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:33.104+0000 7f564ea00640 1 --2- 172.21.15.39:0/1012173440 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5630078b90 0x7f563007b0a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5640000f60 tx=0x7f5640009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:33.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:33.107+0000 7f5646a00640 1 -- 172.21.15.39:0/1012173440 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f56480627b0 con 0x7f5650073170 2024-09-21T07:34:33.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:33.379+0000 7f5655c00640 1 -- 172.21.15.39:0/1012173440 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_attaching_pools_with_snaps_to_fs_fails"]} v 0) -- 0x7f5650073d20 con 0x7f5650073170 2024-09-21T07:34:34.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.359+0000 7f5646a00640 1 -- 172.21.15.39:0/1012173440 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_attaching_pools_with_snaps_to_fs_fails"]}]=0 v95) ==== 173+0+0 (secure 0 0 0) 0x7f56480671e0 con 0x7f5650073170 2024-09-21T07:34:34.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.362+0000 7f5655c00640 1 -- 172.21.15.39:0/1012173440 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5630078b90 msgr2=0x7f563007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:34.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.362+0000 7f5655c00640 1 --2- 172.21.15.39:0/1012173440 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5630078b90 0x7f563007b0a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5640000f60 tx=0x7f5640009040 comp rx=0 tx=0).stop 2024-09-21T07:34:34.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.362+0000 7f5655c00640 1 -- 172.21.15.39:0/1012173440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5650073170 msgr2=0x7f56500cb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:34.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.362+0000 7f5655c00640 1 --2- 172.21.15.39:0/1012173440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5650073170 0x7f56500cb630 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f564800c760 tx=0x7f564800cc20 comp rx=0 tx=0).stop 2024-09-21T07:34:34.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.363+0000 7f5655c00640 1 -- 172.21.15.39:0/1012173440 shutdown_connections 2024-09-21T07:34:34.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.363+0000 7f5655c00640 1 --2- 172.21.15.39:0/1012173440 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5630078b90 0x7f563007b0a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:34.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.363+0000 7f5655c00640 1 --2- 172.21.15.39:0/1012173440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5650073170 0x7f56500cb630 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:34.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.363+0000 7f5655c00640 1 --2- 172.21.15.39:0/1012173440 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5650070440 0x7f56500cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:34.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.363+0000 7f5655c00640 1 --2- 172.21.15.39:0/1012173440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5650070030 0x7f565014f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:34.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.363+0000 7f5655c00640 1 -- 172.21.15.39:0/1012173440 >> 172.21.15.39:0/1012173440 conn(0x7f56500b1cb0 msgr2=0x7f56500ada60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:34.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.363+0000 7f5655c00640 1 -- 172.21.15.39:0/1012173440 shutdown_connections 2024-09-21T07:34:34.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.363+0000 7f5655c00640 1 -- 172.21.15.39:0/1012173440 wait complete. 2024-09-21T07:34:34.393 INFO:tasks.cephfs_test_runner:Test that attempt to attach pool with snapshots to an fs fails ... ok 2024-09-21T07:34:34.394 INFO:tasks.cephfs_test_runner:Starting test: test_disallow_monitor_managed_snaps_for_fs_pools (tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools) Test that creation of monitor managed snaps fails for pools attached 2024-09-21T07:34:34.394 INFO:tasks.cephfs_test_runner:test_disallow_monitor_managed_snaps_for_fs_pools (tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools) 2024-09-21T07:34:34.394 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:34:34.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.622+0000 7f1f9f800640 1 -- 172.21.15.39:0/1917444442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f980cc1e0 msgr2=0x7f1f980af3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:34.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.622+0000 7f1f9f800640 1 --2- 172.21.15.39:0/1917444442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f980cc1e0 0x7f1f980af3c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c009830 tx=0x7f1f8c02a540 comp rx=0 tx=0).stop 2024-09-21T07:34:34.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.622+0000 7f1f9f800640 1 -- 172.21.15.39:0/1917444442 shutdown_connections 2024-09-21T07:34:34.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.622+0000 7f1f9f800640 1 --2- 172.21.15.39:0/1917444442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f980cc1e0 0x7f1f980af3c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:34.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.622+0000 7f1f9f800640 1 -- 172.21.15.39:0/1917444442 >> 172.21.15.39:0/1917444442 conn(0x7f1f9806d7d0 msgr2=0x7f1f980b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:34.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.622+0000 7f1f9f800640 1 -- 172.21.15.39:0/1917444442 shutdown_connections 2024-09-21T07:34:34.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.622+0000 7f1f9f800640 1 -- 172.21.15.39:0/1917444442 wait complete. 2024-09-21T07:34:34.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.624+0000 7f1f9f800640 1 Processor -- start 2024-09-21T07:34:34.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.624+0000 7f1f9f800640 1 -- start start 2024-09-21T07:34:34.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.626+0000 7f1f9f800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f980cc1e0 0x7f1f98148c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:34.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.626+0000 7f1f9f800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f980ccb50 0x7f1f9815b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:34.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.626+0000 7f1f9f800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f9815d8a0 0x7f1f9815be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:34.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.626+0000 7f1f9f800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1f98153520 con 0x7f1f9815d8a0 2024-09-21T07:34:34.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.626+0000 7f1f9f800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1f981533a0 con 0x7f1f980ccb50 2024-09-21T07:34:34.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.626+0000 7f1f9f800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f1f981536a0 con 0x7f1f980cc1e0 2024-09-21T07:34:34.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.626+0000 7f1f9d800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f9815d8a0 0x7f1f9815be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:34.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.626+0000 7f1f9d800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f9815d8a0 0x7f1f9815be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57408/0 (socket says 172.21.15.39:57408) 2024-09-21T07:34:34.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.626+0000 7f1f9d800640 1 -- 172.21.15.39:0/2360618073 learned_addr learned my addr 172.21.15.39:0/2360618073 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:34.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.626+0000 7f1f97e00640 1 --2- 172.21.15.39:0/2360618073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f980ccb50 0x7f1f9815b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:34.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.626+0000 7f1f9ce00640 1 --2- 172.21.15.39:0/2360618073 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f980cc1e0 0x7f1f98148c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:34.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.627+0000 7f1f9d800640 1 -- 172.21.15.39:0/2360618073 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f980cc1e0 msgr2=0x7f1f98148c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:34.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.627+0000 7f1f9d800640 1 --2- 172.21.15.39:0/2360618073 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f980cc1e0 0x7f1f98148c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:34.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.627+0000 7f1f9d800640 1 -- 172.21.15.39:0/2360618073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f980ccb50 msgr2=0x7f1f9815b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:34.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.627+0000 7f1f9d800640 1 --2- 172.21.15.39:0/2360618073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f980ccb50 0x7f1f9815b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:34.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.627+0000 7f1f9d800640 1 -- 172.21.15.39:0/2360618073 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f981f60c0 con 0x7f1f9815d8a0 2024-09-21T07:34:34.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.627+0000 7f1f9d800640 1 --2- 172.21.15.39:0/2360618073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f9815d8a0 0x7f1f9815be10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1f9000c700 tx=0x7f1f9000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:34.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.627+0000 7f1f94c00640 1 -- 172.21.15.39:0/2360618073 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f90003b70 con 0x7f1f9815d8a0 2024-09-21T07:34:34.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.627+0000 7f1f94c00640 1 -- 172.21.15.39:0/2360618073 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f90003d10 con 0x7f1f9815d8a0 2024-09-21T07:34:34.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.627+0000 7f1f9f800640 1 -- 172.21.15.39:0/2360618073 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f981f63a0 con 0x7f1f9815d8a0 2024-09-21T07:34:34.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.627+0000 7f1f94c00640 1 -- 172.21.15.39:0/2360618073 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9000aab0 con 0x7f1f9815d8a0 2024-09-21T07:34:34.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.627+0000 7f1f9f800640 1 -- 172.21.15.39:0/2360618073 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f981f6960 con 0x7f1f9815d8a0 2024-09-21T07:34:34.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.628+0000 7f1f9f800640 1 -- 172.21.15.39:0/2360618073 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f98153b50 con 0x7f1f9815d8a0 2024-09-21T07:34:34.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.630+0000 7f1f94c00640 1 -- 172.21.15.39:0/2360618073 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1f9000ac50 con 0x7f1f9815d8a0 2024-09-21T07:34:34.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.631+0000 7f1f94c00640 1 --2- 172.21.15.39:0/2360618073 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1f78078b90 0x7f1f7807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:34.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.631+0000 7f1f94c00640 1 -- 172.21.15.39:0/2360618073 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5296+0+0 (secure 0 0 0) 0x7f1f9009b320 con 0x7f1f9815d8a0 2024-09-21T07:34:34.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.631+0000 7f1f9ce00640 1 --2- 172.21.15.39:0/2360618073 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1f78078b90 0x7f1f7807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:34.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.632+0000 7f1f9ce00640 1 --2- 172.21.15.39:0/2360618073 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1f78078b90 0x7f1f7807b0a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c034000 tx=0x7f1f8c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:34.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.634+0000 7f1f94c00640 1 -- 172.21.15.39:0/2360618073 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1f90063350 con 0x7f1f9815d8a0 2024-09-21T07:34:34.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.834+0000 7f1f9f800640 1 -- 172.21.15.39:0/2360618073 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1f9815c840 con 0x7f1f9815d8a0 2024-09-21T07:34:34.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.835+0000 7f1f94c00640 1 -- 172.21.15.39:0/2360618073 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+14842 (secure 0 0 0) 0x7f1f90067d80 con 0x7f1f9815d8a0 2024-09-21T07:34:34.837 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:34:34.837 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":41,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:34:32.340796+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6834/3524592265":"2024-09-22T07:34:29.204874+0000","172.21.15.39:6832/3524592265":"2024-09-22T07:34:29.204874+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:34:34.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.837+0000 7f1f9f800640 1 -- 172.21.15.39:0/2360618073 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1f78078b90 msgr2=0x7f1f7807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:34.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.837+0000 7f1f9f800640 1 --2- 172.21.15.39:0/2360618073 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1f78078b90 0x7f1f7807b0a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c034000 tx=0x7f1f8c004730 comp rx=0 tx=0).stop 2024-09-21T07:34:34.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.837+0000 7f1f9f800640 1 -- 172.21.15.39:0/2360618073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f9815d8a0 msgr2=0x7f1f9815be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:34.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.837+0000 7f1f9f800640 1 --2- 172.21.15.39:0/2360618073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f9815d8a0 0x7f1f9815be10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1f9000c700 tx=0x7f1f9000cbc0 comp rx=0 tx=0).stop 2024-09-21T07:34:34.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.837+0000 7f1f9f800640 1 -- 172.21.15.39:0/2360618073 shutdown_connections 2024-09-21T07:34:34.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.837+0000 7f1f9f800640 1 --2- 172.21.15.39:0/2360618073 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1f78078b90 0x7f1f7807b0a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:34.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.837+0000 7f1f9f800640 1 --2- 172.21.15.39:0/2360618073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f9815d8a0 0x7f1f9815be10 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:34.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.837+0000 7f1f9f800640 1 --2- 172.21.15.39:0/2360618073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f980ccb50 0x7f1f9815b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:34.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.837+0000 7f1f9f800640 1 --2- 172.21.15.39:0/2360618073 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f980cc1e0 0x7f1f98148c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:34.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.838+0000 7f1f9f800640 1 -- 172.21.15.39:0/2360618073 >> 172.21.15.39:0/2360618073 conn(0x7f1f9806d7d0 msgr2=0x7f1f980cfe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:34.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.838+0000 7f1f9f800640 1 -- 172.21.15.39:0/2360618073 shutdown_connections 2024-09-21T07:34:34.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:34.838+0000 7f1f9f800640 1 -- 172.21.15.39:0/2360618073 wait complete. 2024-09-21T07:34:34.862 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T07:33:01.569433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'snap-test-pool', 'create_time': '2024-09-21T07:34:22.032185+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 1, 'snap_epoch': 37, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T07:34:34.862 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T07:34:35.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.089+0000 7ff08ea00640 1 -- 172.21.15.39:0/991903318 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0880caec0 msgr2=0x7ff0880704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:35.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.089+0000 7ff08ea00640 1 --2- 172.21.15.39:0/991903318 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0880caec0 0x7ff0880704b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff080005fd0 tx=0x7ff08002a820 comp rx=0 tx=0).stop 2024-09-21T07:34:35.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.089+0000 7ff08ea00640 1 -- 172.21.15.39:0/991903318 shutdown_connections 2024-09-21T07:34:35.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.089+0000 7ff08ea00640 1 --2- 172.21.15.39:0/991903318 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0880caec0 0x7ff0880704b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:35.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.089+0000 7ff08ea00640 1 -- 172.21.15.39:0/991903318 >> 172.21.15.39:0/991903318 conn(0x7ff0880d0160 msgr2=0x7ff0880d0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:35.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.089+0000 7ff08ea00640 1 -- 172.21.15.39:0/991903318 shutdown_connections 2024-09-21T07:34:35.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.089+0000 7ff08ea00640 1 -- 172.21.15.39:0/991903318 wait complete. 2024-09-21T07:34:35.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.091+0000 7ff08ea00640 1 Processor -- start 2024-09-21T07:34:35.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.091+0000 7ff08ea00640 1 -- start start 2024-09-21T07:34:35.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.093+0000 7ff08ea00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0880b56a0 0x7ff08809aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:35.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.093+0000 7ff08ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0880cc640 0x7ff08815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:35.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.093+0000 7ff08ea00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff0881f94a0 0x7ff0881f5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:35.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.093+0000 7ff08ea00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff0880cb7b0 con 0x7ff0880cc640 2024-09-21T07:34:35.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.093+0000 7ff08ea00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff0880cb630 con 0x7ff0880b56a0 2024-09-21T07:34:35.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.093+0000 7ff08ea00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7ff0880cb930 con 0x7ff0881f94a0 2024-09-21T07:34:35.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.093+0000 7ff07f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0880cc640 0x7ff08815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:35.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.093+0000 7ff07f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0880cc640 0x7ff08815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57424/0 (socket says 172.21.15.39:57424) 2024-09-21T07:34:35.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.093+0000 7ff07f400640 1 -- 172.21.15.39:0/919353176 learned_addr learned my addr 172.21.15.39:0/919353176 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:35.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.093+0000 7ff08ca00640 1 --2- 172.21.15.39:0/919353176 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff0881f94a0 0x7ff0881f5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:35.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.093+0000 7ff087e00640 1 --2- 172.21.15.39:0/919353176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0880b56a0 0x7ff08809aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:35.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.094+0000 7ff07f400640 1 -- 172.21.15.39:0/919353176 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff0881f94a0 msgr2=0x7ff0881f5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:35.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.094+0000 7ff07f400640 1 --2- 172.21.15.39:0/919353176 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff0881f94a0 0x7ff0881f5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:35.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.094+0000 7ff07f400640 1 -- 172.21.15.39:0/919353176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0880b56a0 msgr2=0x7ff08809aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:35.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.094+0000 7ff07f400640 1 --2- 172.21.15.39:0/919353176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0880b56a0 0x7ff08809aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:35.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.094+0000 7ff07f400640 1 -- 172.21.15.39:0/919353176 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff08809b5e0 con 0x7ff0880cc640 2024-09-21T07:34:35.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.094+0000 7ff087e00640 1 --2- 172.21.15.39:0/919353176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0880b56a0 0x7ff08809aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:34:35.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.094+0000 7ff07f400640 1 --2- 172.21.15.39:0/919353176 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0880cc640 0x7ff08815d5d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff07000c400 tx=0x7ff07000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:35.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.094+0000 7ff084c00640 1 -- 172.21.15.39:0/919353176 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff070017070 con 0x7ff0880cc640 2024-09-21T07:34:35.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.094+0000 7ff084c00640 1 -- 172.21.15.39:0/919353176 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff0700053c0 con 0x7ff0880cc640 2024-09-21T07:34:35.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.094+0000 7ff08ea00640 1 -- 172.21.15.39:0/919353176 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff08809b8c0 con 0x7ff0880cc640 2024-09-21T07:34:35.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.094+0000 7ff084c00640 1 -- 172.21.15.39:0/919353176 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff070003830 con 0x7ff0880cc640 2024-09-21T07:34:35.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.094+0000 7ff08ea00640 1 -- 172.21.15.39:0/919353176 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0881f5ed0 con 0x7ff0880cc640 2024-09-21T07:34:35.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.095+0000 7ff08ea00640 1 -- 172.21.15.39:0/919353176 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0880704b0 con 0x7ff0880cc640 2024-09-21T07:34:35.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.097+0000 7ff084c00640 1 -- 172.21.15.39:0/919353176 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff07001a070 con 0x7ff0880cc640 2024-09-21T07:34:35.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.097+0000 7ff084c00640 1 --2- 172.21.15.39:0/919353176 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff064078b90 0x7ff06407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:35.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.098+0000 7ff084c00640 1 -- 172.21.15.39:0/919353176 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5296+0+0 (secure 0 0 0) 0x7ff070099e10 con 0x7ff0880cc640 2024-09-21T07:34:35.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.098+0000 7ff087e00640 1 --2- 172.21.15.39:0/919353176 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff064078b90 0x7ff06407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:35.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.099+0000 7ff087e00640 1 --2- 172.21.15.39:0/919353176 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff064078b90 0x7ff06407b0a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff078000f60 tx=0x7ff078009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:35.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.102+0000 7ff084c00640 1 -- 172.21.15.39:0/919353176 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff07009d030 con 0x7ff0880cc640 2024-09-21T07:34:35.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.297+0000 7ff08ea00640 1 -- 172.21.15.39:0/919353176 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff0880cfaf0 con 0x7ff0880cc640 2024-09-21T07:34:35.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.297+0000 7ff084c00640 1 -- 172.21.15.39:0/919353176 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) ==== 93+0+10 (secure 0 0 0) 0x7ff0880b5ab0 con 0x7ff0880cc640 2024-09-21T07:34:35.299 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-09-21T07:34:35.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.300+0000 7ff08ea00640 1 -- 172.21.15.39:0/919353176 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff064078b90 msgr2=0x7ff06407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:35.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.300+0000 7ff08ea00640 1 --2- 172.21.15.39:0/919353176 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff064078b90 0x7ff06407b0a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff078000f60 tx=0x7ff078009040 comp rx=0 tx=0).stop 2024-09-21T07:34:35.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.300+0000 7ff08ea00640 1 -- 172.21.15.39:0/919353176 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0880cc640 msgr2=0x7ff08815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:35.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.300+0000 7ff08ea00640 1 --2- 172.21.15.39:0/919353176 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0880cc640 0x7ff08815d5d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff07000c400 tx=0x7ff07000c8c0 comp rx=0 tx=0).stop 2024-09-21T07:34:35.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.300+0000 7ff08ea00640 1 -- 172.21.15.39:0/919353176 shutdown_connections 2024-09-21T07:34:35.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.300+0000 7ff08ea00640 1 --2- 172.21.15.39:0/919353176 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff064078b90 0x7ff06407b0a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:35.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.300+0000 7ff08ea00640 1 --2- 172.21.15.39:0/919353176 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff0881f94a0 0x7ff0881f5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:35.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.300+0000 7ff08ea00640 1 --2- 172.21.15.39:0/919353176 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0880cc640 0x7ff08815d5d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:35.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.300+0000 7ff08ea00640 1 --2- 172.21.15.39:0/919353176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0880b56a0 0x7ff08809aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:35.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.300+0000 7ff08ea00640 1 -- 172.21.15.39:0/919353176 >> 172.21.15.39:0/919353176 conn(0x7ff0880d0160 msgr2=0x7ff0880ae2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:35.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.300+0000 7ff08ea00640 1 -- 172.21.15.39:0/919353176 shutdown_connections 2024-09-21T07:34:35.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.300+0000 7ff08ea00640 1 -- 172.21.15.39:0/919353176 wait complete. 2024-09-21T07:34:35.324 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2024-09-21T07:34:35.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.553+0000 7fb5e1a00640 1 -- 172.21.15.39:0/3439396324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5dc0ccb50 msgr2=0x7fb5dc0ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:35.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.553+0000 7fb5e1a00640 1 --2- 172.21.15.39:0/3439396324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5dc0ccb50 0x7fb5dc0ccf60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc072bc0 tx=0x7fb5d402a8b0 comp rx=0 tx=0).stop 2024-09-21T07:34:35.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.553+0000 7fb5e1a00640 1 -- 172.21.15.39:0/3439396324 shutdown_connections 2024-09-21T07:34:35.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.553+0000 7fb5e1a00640 1 --2- 172.21.15.39:0/3439396324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5dc0ccb50 0x7fb5dc0ccf60 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:35.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.553+0000 7fb5e1a00640 1 --2- 172.21.15.39:0/3439396324 >> v2:172.21.15.148:3300/0 conn(0x7fb5dc0af3c0 0x7fb5dc0b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:35.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.553+0000 7fb5e1a00640 1 -- 172.21.15.39:0/3439396324 >> 172.21.15.39:0/3439396324 conn(0x7fb5dc06d7d0 msgr2=0x7fb5dc0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:35.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.553+0000 7fb5e1a00640 1 -- 172.21.15.39:0/3439396324 shutdown_connections 2024-09-21T07:34:35.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.553+0000 7fb5e1a00640 1 -- 172.21.15.39:0/3439396324 wait complete. 2024-09-21T07:34:35.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.555+0000 7fb5e1a00640 1 Processor -- start 2024-09-21T07:34:35.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.556+0000 7fb5e1a00640 1 -- start start 2024-09-21T07:34:35.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.557+0000 7fb5e1a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5dc0af3c0 0x7fb5dc1f62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:35.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.557+0000 7fb5e1a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5dc0cc1e0 0x7fb5dc152350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:35.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.557+0000 7fb5e1a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5dc0ccb50 0x7fb5dc15aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:35.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.557+0000 7fb5e1a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb5dc1533a0 con 0x7fb5dc0af3c0 2024-09-21T07:34:35.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.557+0000 7fb5e1a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb5dc0af8e0 con 0x7fb5dc0cc1e0 2024-09-21T07:34:35.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.557+0000 7fb5e1a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb5dc153520 con 0x7fb5dc0ccb50 2024-09-21T07:34:35.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.557+0000 7fb5daa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5dc0af3c0 0x7fb5dc1f62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:35.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.557+0000 7fb5daa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5dc0af3c0 0x7fb5dc1f62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57434/0 (socket says 172.21.15.39:57434) 2024-09-21T07:34:35.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.557+0000 7fb5daa00640 1 -- 172.21.15.39:0/3641568442 learned_addr learned my addr 172.21.15.39:0/3641568442 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:35.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.557+0000 7fb5d3e00640 1 --2- 172.21.15.39:0/3641568442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5dc0cc1e0 0x7fb5dc152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:35.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.558+0000 7fb5db400640 1 --2- 172.21.15.39:0/3641568442 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5dc0ccb50 0x7fb5dc15aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:35.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.558+0000 7fb5daa00640 1 -- 172.21.15.39:0/3641568442 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5dc0ccb50 msgr2=0x7fb5dc15aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:35.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.558+0000 7fb5daa00640 1 --2- 172.21.15.39:0/3641568442 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5dc0ccb50 0x7fb5dc15aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:35.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.558+0000 7fb5daa00640 1 -- 172.21.15.39:0/3641568442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5dc0cc1e0 msgr2=0x7fb5dc152350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:35.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.558+0000 7fb5daa00640 1 --2- 172.21.15.39:0/3641568442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5dc0cc1e0 0x7fb5dc152350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:35.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.558+0000 7fb5daa00640 1 -- 172.21.15.39:0/3641568442 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5dc158ae0 con 0x7fb5dc0af3c0 2024-09-21T07:34:35.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.558+0000 7fb5db400640 1 --2- 172.21.15.39:0/3641568442 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5dc0ccb50 0x7fb5dc15aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:34:35.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.558+0000 7fb5daa00640 1 --2- 172.21.15.39:0/3641568442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5dc0af3c0 0x7fb5dc1f62f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc00c3c0 tx=0x7fb5cc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:35.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.558+0000 7fb5d2a00640 1 -- 172.21.15.39:0/3641568442 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc017070 con 0x7fb5dc0af3c0 2024-09-21T07:34:35.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.558+0000 7fb5d2a00640 1 -- 172.21.15.39:0/3641568442 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5cc0035f0 con 0x7fb5dc0af3c0 2024-09-21T07:34:35.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.558+0000 7fb5e1a00640 1 -- 172.21.15.39:0/3641568442 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5dc158dc0 con 0x7fb5dc0af3c0 2024-09-21T07:34:35.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.558+0000 7fb5d2a00640 1 -- 172.21.15.39:0/3641568442 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc003a40 con 0x7fb5dc0af3c0 2024-09-21T07:34:35.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.558+0000 7fb5e1a00640 1 -- 172.21.15.39:0/3641568442 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5dc159380 con 0x7fb5dc0af3c0 2024-09-21T07:34:35.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.560+0000 7fb5e1a00640 1 -- 172.21.15.39:0/3641568442 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5dc153a40 con 0x7fb5dc0af3c0 2024-09-21T07:34:35.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.561+0000 7fb5d2a00640 1 -- 172.21.15.39:0/3641568442 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb5cc01a070 con 0x7fb5dc0af3c0 2024-09-21T07:34:35.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.562+0000 7fb5d2a00640 1 --2- 172.21.15.39:0/3641568442 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb5b8078bc0 0x7fb5b807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:35.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.562+0000 7fb5d2a00640 1 -- 172.21.15.39:0/3641568442 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5296+0+0 (secure 0 0 0) 0x7fb5cc099ed0 con 0x7fb5dc0af3c0 2024-09-21T07:34:35.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.562+0000 7fb5d3e00640 1 --2- 172.21.15.39:0/3641568442 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb5b8078bc0 0x7fb5b807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:35.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.563+0000 7fb5d3e00640 1 --2- 172.21.15.39:0/3641568442 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb5b8078bc0 0x7fb5b807b0d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4005fd0 tx=0x7fb5c4005850 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:35.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.566+0000 7fb5d2a00640 1 -- 172.21.15.39:0/3641568442 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb5dc0cd4a0 con 0x7fb5dc0af3c0 2024-09-21T07:34:35.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.766+0000 7fb5e1a00640 1 -- 172.21.15.39:0/3641568442 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) -- 0x7fb5dc0cd170 con 0x7fb5dc0af3c0 2024-09-21T07:34:35.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.767+0000 7fb5d2a00640 1 -- 172.21.15.39:0/3641568442 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v41) ==== 103+0+11 (secure 0 0 0) 0x7fb5cc062430 con 0x7fb5dc0af3c0 2024-09-21T07:34:35.769 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-21T07:34:35.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.770+0000 7fb5e1a00640 1 -- 172.21.15.39:0/3641568442 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb5b8078bc0 msgr2=0x7fb5b807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:35.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.770+0000 7fb5e1a00640 1 --2- 172.21.15.39:0/3641568442 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb5b8078bc0 0x7fb5b807b0d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4005fd0 tx=0x7fb5c4005850 comp rx=0 tx=0).stop 2024-09-21T07:34:35.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.770+0000 7fb5e1a00640 1 -- 172.21.15.39:0/3641568442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5dc0af3c0 msgr2=0x7fb5dc1f62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:35.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.770+0000 7fb5e1a00640 1 --2- 172.21.15.39:0/3641568442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5dc0af3c0 0x7fb5dc1f62f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc00c3c0 tx=0x7fb5cc00c880 comp rx=0 tx=0).stop 2024-09-21T07:34:35.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.770+0000 7fb5e1a00640 1 -- 172.21.15.39:0/3641568442 shutdown_connections 2024-09-21T07:34:35.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.770+0000 7fb5e1a00640 1 --2- 172.21.15.39:0/3641568442 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb5b8078bc0 0x7fb5b807b0d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:35.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.770+0000 7fb5e1a00640 1 --2- 172.21.15.39:0/3641568442 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5dc0ccb50 0x7fb5dc15aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:35.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.770+0000 7fb5e1a00640 1 --2- 172.21.15.39:0/3641568442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5dc0cc1e0 0x7fb5dc152350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:35.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.770+0000 7fb5e1a00640 1 --2- 172.21.15.39:0/3641568442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5dc0af3c0 0x7fb5dc1f62f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:35.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.770+0000 7fb5e1a00640 1 -- 172.21.15.39:0/3641568442 >> 172.21.15.39:0/3641568442 conn(0x7fb5dc06d7d0 msgr2=0x7fb5dc0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:35.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.770+0000 7fb5e1a00640 1 -- 172.21.15.39:0/3641568442 shutdown_connections 2024-09-21T07:34:35.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:35.770+0000 7fb5e1a00640 1 -- 172.21.15.39:0/3641568442 wait complete. 2024-09-21T07:34:35.798 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_disallow_monitor_managed_snaps_for_fs_pools' 2024-09-21T07:34:36.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.023+0000 7fdc0ac00640 1 -- 172.21.15.39:0/1727344251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc04153970 msgr2=0x7fdc0415d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:36.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.023+0000 7fdc0ac00640 1 --2- 172.21.15.39:0/1727344251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc04153970 0x7fdc0415d810 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4005fd0 tx=0x7fdbf402a5d0 comp rx=0 tx=0).stop 2024-09-21T07:34:36.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.023+0000 7fdc0ac00640 1 -- 172.21.15.39:0/1727344251 shutdown_connections 2024-09-21T07:34:36.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.023+0000 7fdc0ac00640 1 --2- 172.21.15.39:0/1727344251 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdc040cfc00 0x7fdc040ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:36.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.023+0000 7fdc0ac00640 1 --2- 172.21.15.39:0/1727344251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc04153970 0x7fdc0415d810 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:36.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.023+0000 7fdc0ac00640 1 -- 172.21.15.39:0/1727344251 >> 172.21.15.39:0/1727344251 conn(0x7fdc040d15f0 msgr2=0x7fdc04073740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:36.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.023+0000 7fdc0ac00640 1 -- 172.21.15.39:0/1727344251 shutdown_connections 2024-09-21T07:34:36.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.023+0000 7fdc0ac00640 1 -- 172.21.15.39:0/1727344251 wait complete. 2024-09-21T07:34:36.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.025+0000 7fdc0ac00640 1 Processor -- start 2024-09-21T07:34:36.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.025+0000 7fdc0ac00640 1 -- start start 2024-09-21T07:34:36.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.027+0000 7fdc0ac00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc04073e50 0x7fdc04158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:36.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.027+0000 7fdc0ac00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdc040cfc00 0x7fdc041525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:36.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.027+0000 7fdc0ac00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc04153970 0x7fdc04070200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:36.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.027+0000 7fdc0ac00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0415e2e0 con 0x7fdc04153970 2024-09-21T07:34:36.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.027+0000 7fdc0ac00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0415e160 con 0x7fdc04073e50 2024-09-21T07:34:36.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.027+0000 7fdc0ac00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fdc0415e460 con 0x7fdc040cfc00 2024-09-21T07:34:36.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.027+0000 7fdc08c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc04153970 0x7fdc04070200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:36.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.027+0000 7fdc03e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc04073e50 0x7fdc04158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:36.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.027+0000 7fdc08c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc04153970 0x7fdc04070200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57444/0 (socket says 172.21.15.39:57444) 2024-09-21T07:34:36.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.027+0000 7fdc08c00640 1 -- 172.21.15.39:0/1589393105 learned_addr learned my addr 172.21.15.39:0/1589393105 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:36.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.027+0000 7fdc03400640 1 --2- 172.21.15.39:0/1589393105 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdc040cfc00 0x7fdc041525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:36.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.027+0000 7fdc08c00640 1 -- 172.21.15.39:0/1589393105 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdc040cfc00 msgr2=0x7fdc041525e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:36.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.027+0000 7fdc08c00640 1 --2- 172.21.15.39:0/1589393105 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdc040cfc00 0x7fdc041525e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:36.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.027+0000 7fdc08c00640 1 -- 172.21.15.39:0/1589393105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc04073e50 msgr2=0x7fdc04158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:36.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.027+0000 7fdc08c00640 1 --2- 172.21.15.39:0/1589393105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc04073e50 0x7fdc04158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:36.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.028+0000 7fdc08c00640 1 -- 172.21.15.39:0/1589393105 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc041f91c0 con 0x7fdc04153970 2024-09-21T07:34:36.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.028+0000 7fdc08c00640 1 --2- 172.21.15.39:0/1589393105 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc04153970 0x7fdc04070200 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc00c700 tx=0x7fdbfc00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:36.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.028+0000 7fdbfbe00640 1 -- 172.21.15.39:0/1589393105 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc003b70 con 0x7fdc04153970 2024-09-21T07:34:36.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.028+0000 7fdbfbe00640 1 -- 172.21.15.39:0/1589393105 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdbfc003d10 con 0x7fdc04153970 2024-09-21T07:34:36.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.028+0000 7fdc0ac00640 1 -- 172.21.15.39:0/1589393105 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc041f96c0 con 0x7fdc04153970 2024-09-21T07:34:36.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.028+0000 7fdbfbe00640 1 -- 172.21.15.39:0/1589393105 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc00aab0 con 0x7fdc04153970 2024-09-21T07:34:36.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.028+0000 7fdc0ac00640 1 -- 172.21.15.39:0/1589393105 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc041f9c80 con 0x7fdc04153970 2024-09-21T07:34:36.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.029+0000 7fdc0ac00640 1 -- 172.21.15.39:0/1589393105 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc0415e910 con 0x7fdc04153970 2024-09-21T07:34:36.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.031+0000 7fdbfbe00640 1 -- 172.21.15.39:0/1589393105 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdbfc00ac50 con 0x7fdc04153970 2024-09-21T07:34:36.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.032+0000 7fdbfbe00640 1 --2- 172.21.15.39:0/1589393105 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdbe4078b90 0x7fdbe407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:36.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.032+0000 7fdbfbe00640 1 -- 172.21.15.39:0/1589393105 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5296+0+0 (secure 0 0 0) 0x7fdbfc09a9f0 con 0x7fdc04153970 2024-09-21T07:34:36.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.032+0000 7fdc03e00640 1 --2- 172.21.15.39:0/1589393105 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdbe4078b90 0x7fdbe407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:36.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.033+0000 7fdc03e00640 1 --2- 172.21.15.39:0/1589393105 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdbe4078b90 0x7fdbe407b0a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4033000 tx=0x7fdbf4004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:36.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.035+0000 7fdbfbe00640 1 -- 172.21.15.39:0/1589393105 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdc0415cd90 con 0x7fdc04153970 2024-09-21T07:34:36.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.305+0000 7fdc0ac00640 1 -- 172.21.15.39:0/1589393105 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_disallow_monitor_managed_snaps_for_fs_pools"]} v 0) -- 0x7fdc0415d990 con 0x7fdc04153970 2024-09-21T07:34:36.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.367+0000 7fdbfbe00640 1 -- 172.21.15.39:0/1589393105 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_disallow_monitor_managed_snaps_for_fs_pools"]}]=0 v97) ==== 181+0+0 (secure 0 0 0) 0x7fdbfc062a40 con 0x7fdc04153970 2024-09-21T07:34:36.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.370+0000 7fdc0ac00640 1 -- 172.21.15.39:0/1589393105 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdbe4078b90 msgr2=0x7fdbe407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:36.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.370+0000 7fdc0ac00640 1 --2- 172.21.15.39:0/1589393105 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdbe4078b90 0x7fdbe407b0a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4033000 tx=0x7fdbf4004730 comp rx=0 tx=0).stop 2024-09-21T07:34:36.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.370+0000 7fdc0ac00640 1 -- 172.21.15.39:0/1589393105 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc04153970 msgr2=0x7fdc04070200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:36.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.370+0000 7fdc0ac00640 1 --2- 172.21.15.39:0/1589393105 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc04153970 0x7fdc04070200 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc00c700 tx=0x7fdbfc00cbc0 comp rx=0 tx=0).stop 2024-09-21T07:34:36.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.370+0000 7fdc0ac00640 1 -- 172.21.15.39:0/1589393105 shutdown_connections 2024-09-21T07:34:36.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.370+0000 7fdc0ac00640 1 --2- 172.21.15.39:0/1589393105 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdbe4078b90 0x7fdbe407b0a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:36.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.370+0000 7fdc0ac00640 1 --2- 172.21.15.39:0/1589393105 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc04153970 0x7fdc04070200 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:36.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.370+0000 7fdc0ac00640 1 --2- 172.21.15.39:0/1589393105 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdc040cfc00 0x7fdc041525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:36.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.370+0000 7fdc0ac00640 1 --2- 172.21.15.39:0/1589393105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc04073e50 0x7fdc04158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:36.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.370+0000 7fdc0ac00640 1 -- 172.21.15.39:0/1589393105 >> 172.21.15.39:0/1589393105 conn(0x7fdc040d15f0 msgr2=0x7fdc0415fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:36.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.370+0000 7fdc0ac00640 1 -- 172.21.15.39:0/1589393105 shutdown_connections 2024-09-21T07:34:36.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.370+0000 7fdc0ac00640 1 -- 172.21.15.39:0/1589393105 wait complete. 2024-09-21T07:34:36.397 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T07:34:36.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.621+0000 7f2845000640 1 -- 172.21.15.39:0/1405425920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28400706b0 msgr2=0x7f28400cba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:36.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.621+0000 7f2845000640 1 --2- 172.21.15.39:0/1405425920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28400706b0 0x7f28400cba00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2828009830 tx=0x7f282802a5b0 comp rx=0 tx=0).stop 2024-09-21T07:34:36.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.621+0000 7f2845000640 1 -- 172.21.15.39:0/1405425920 shutdown_connections 2024-09-21T07:34:36.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.621+0000 7f2845000640 1 --2- 172.21.15.39:0/1405425920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28400706b0 0x7f28400cba00 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:36.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.621+0000 7f2845000640 1 -- 172.21.15.39:0/1405425920 >> 172.21.15.39:0/1405425920 conn(0x7f28400b1cb0 msgr2=0x7f28400b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:36.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.621+0000 7f2845000640 1 -- 172.21.15.39:0/1405425920 shutdown_connections 2024-09-21T07:34:36.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.621+0000 7f2845000640 1 -- 172.21.15.39:0/1405425920 wait complete. 2024-09-21T07:34:36.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.623+0000 7f2845000640 1 Processor -- start 2024-09-21T07:34:36.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.623+0000 7f2845000640 1 -- start start 2024-09-21T07:34:36.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.624+0000 7f2845000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28400ca800 0x7f284009b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:36.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.624+0000 7f2845000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28400cad30 0x7f28401f8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:36.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.625+0000 7f2845000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f284015eb00 0x7f28401f64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:36.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.625+0000 7f2845000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2840074130 con 0x7f28400cad30 2024-09-21T07:34:36.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.625+0000 7f2845000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2840073fb0 con 0x7f284015eb00 2024-09-21T07:34:36.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.625+0000 7f2845000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f28400742b0 con 0x7f28400ca800 2024-09-21T07:34:36.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.625+0000 7f2837e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28400cad30 0x7f28401f8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:36.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.625+0000 7f2837e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28400cad30 0x7f28401f8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57460/0 (socket says 172.21.15.39:57460) 2024-09-21T07:34:36.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.625+0000 7f2837e00640 1 -- 172.21.15.39:0/1174239284 learned_addr learned my addr 172.21.15.39:0/1174239284 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:36.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.625+0000 7f283de00640 1 --2- 172.21.15.39:0/1174239284 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28400ca800 0x7f284009b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:36.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.625+0000 7f283e800640 1 --2- 172.21.15.39:0/1174239284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f284015eb00 0x7f28401f64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:36.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.625+0000 7f2837e00640 1 -- 172.21.15.39:0/1174239284 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28400ca800 msgr2=0x7f284009b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:36.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.625+0000 7f2837e00640 1 --2- 172.21.15.39:0/1174239284 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28400ca800 0x7f284009b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:36.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.625+0000 7f2837e00640 1 -- 172.21.15.39:0/1174239284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f284015eb00 msgr2=0x7f28401f64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:36.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.625+0000 7f2837e00640 1 --2- 172.21.15.39:0/1174239284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f284015eb00 0x7f28401f64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:36.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.625+0000 7f2837e00640 1 -- 172.21.15.39:0/1174239284 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f284009cf90 con 0x7f28400cad30 2024-09-21T07:34:36.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.625+0000 7f283de00640 1 --2- 172.21.15.39:0/1174239284 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28400ca800 0x7f284009b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:34:36.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.625+0000 7f283e800640 1 --2- 172.21.15.39:0/1174239284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f284015eb00 0x7f28401f64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:34:36.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.626+0000 7f2837e00640 1 --2- 172.21.15.39:0/1174239284 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28400cad30 0x7f28401f8230 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2828009950 tx=0x7f2828033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:36.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.626+0000 7f2836000640 1 -- 172.21.15.39:0/1174239284 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2828031d60 con 0x7f28400cad30 2024-09-21T07:34:36.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.626+0000 7f2836000640 1 -- 172.21.15.39:0/1174239284 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2828033070 con 0x7f28400cad30 2024-09-21T07:34:36.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.626+0000 7f2836000640 1 -- 172.21.15.39:0/1174239284 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28280334c0 con 0x7f28400cad30 2024-09-21T07:34:36.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.626+0000 7f2845000640 1 -- 172.21.15.39:0/1174239284 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f284009d270 con 0x7f28400cad30 2024-09-21T07:34:36.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.626+0000 7f2845000640 1 -- 172.21.15.39:0/1174239284 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f284009d830 con 0x7f28400cad30 2024-09-21T07:34:36.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.627+0000 7f2845000640 1 -- 172.21.15.39:0/1174239284 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28400747d0 con 0x7f28400cad30 2024-09-21T07:34:36.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.628+0000 7f2836000640 1 -- 172.21.15.39:0/1174239284 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2828033660 con 0x7f28400cad30 2024-09-21T07:34:36.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.629+0000 7f2836000640 1 --2- 172.21.15.39:0/1174239284 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f281c078b90 0x7f281c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:36.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.629+0000 7f2836000640 1 -- 172.21.15.39:0/1174239284 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5296+0+0 (secure 0 0 0) 0x7f28280bc150 con 0x7f28400cad30 2024-09-21T07:34:36.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.630+0000 7f283de00640 1 --2- 172.21.15.39:0/1174239284 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f281c078b90 0x7f281c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:36.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.631+0000 7f283de00640 1 --2- 172.21.15.39:0/1174239284 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f281c078b90 0x7f281c07b0a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2830007fd0 tx=0x7f2830009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:36.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.634+0000 7f2836000640 1 -- 172.21.15.39:0/1174239284 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f282803f030 con 0x7f28400cad30 2024-09-21T07:34:36.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.833+0000 7f2845000640 1 -- 172.21.15.39:0/1174239284 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f284006fa40 con 0x7f28400cad30 2024-09-21T07:34:36.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.834+0000 7f2836000640 1 -- 172.21.15.39:0/1174239284 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v34) ==== 59+0+2294 (secure 0 0 0) 0x7f28280840f0 con 0x7f28400cad30 2024-09-21T07:34:36.837 INFO:teuthology.orchestra.run.smithi039.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T07:34:36.837 INFO:teuthology.orchestra.run.smithi039.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T07:34:36.837 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced client_mount_timeout 30 2024-09-21T07:34:36.837 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_client 20/20 2024-09-21T07:34:36.837 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_mgr 20/20 2024-09-21T07:34:36.837 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_ms 1/1 2024-09-21T07:34:36.837 INFO:teuthology.orchestra.run.smithi039.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T07:34:36.838 INFO:teuthology.orchestra.run.smithi039.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T07:34:36.838 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3421.881759 2024-09-21T07:34:36.838 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3520.252978 2024-09-21T07:34:36.838 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3511.183316 2024-09-21T07:34:36.838 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3829.631847 2024-09-21T07:34:36.838 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 5474.545872 2024-09-21T07:34:36.838 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4013.628559 2024-09-21T07:34:36.838 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4230.187288 2024-09-21T07:34:36.839 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4148.983107 2024-09-21T07:34:36.839 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_mds 20/20 2024-09-21T07:34:36.839 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T07:34:36.839 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_ms 2/2 2024-09-21T07:34:36.839 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_debug_frag true 2024-09-21T07:34:36.839 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_debug_scatterstat true 2024-09-21T07:34:36.839 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T07:34:36.839 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_verify_scatter true 2024-09-21T07:34:36.840 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T07:34:36.840 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T07:34:36.840 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T07:34:36.840 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced client_mount_timeout 600 2024-09-21T07:34:36.840 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced debug_client 20/20 2024-09-21T07:34:36.840 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced debug_ms 1/1 2024-09-21T07:34:36.840 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T07:34:36.841 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T07:34:36.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.837+0000 7f2845000640 1 -- 172.21.15.39:0/1174239284 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f281c078b90 msgr2=0x7f281c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:36.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.837+0000 7f2845000640 1 --2- 172.21.15.39:0/1174239284 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f281c078b90 0x7f281c07b0a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2830007fd0 tx=0x7f2830009040 comp rx=0 tx=0).stop 2024-09-21T07:34:36.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.837+0000 7f2845000640 1 -- 172.21.15.39:0/1174239284 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28400cad30 msgr2=0x7f28401f8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:36.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.837+0000 7f2845000640 1 --2- 172.21.15.39:0/1174239284 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28400cad30 0x7f28401f8230 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2828009950 tx=0x7f2828033d60 comp rx=0 tx=0).stop 2024-09-21T07:34:36.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.837+0000 7f2845000640 1 -- 172.21.15.39:0/1174239284 shutdown_connections 2024-09-21T07:34:36.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.837+0000 7f2845000640 1 --2- 172.21.15.39:0/1174239284 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f281c078b90 0x7f281c07b0a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:36.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.837+0000 7f2845000640 1 --2- 172.21.15.39:0/1174239284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f284015eb00 0x7f28401f64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:36.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.837+0000 7f2845000640 1 --2- 172.21.15.39:0/1174239284 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28400cad30 0x7f28401f8230 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:36.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.837+0000 7f2845000640 1 --2- 172.21.15.39:0/1174239284 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28400ca800 0x7f284009b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:36.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.837+0000 7f2845000640 1 -- 172.21.15.39:0/1174239284 >> 172.21.15.39:0/1174239284 conn(0x7f28400b1cb0 msgr2=0x7f284009db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:36.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.837+0000 7f2845000640 1 -- 172.21.15.39:0/1174239284 shutdown_connections 2024-09-21T07:34:36.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:36.837+0000 7f2845000640 1 -- 172.21.15.39:0/1174239284 wait complete. 2024-09-21T07:34:36.864 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T07:34:37.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.094+0000 7f1c75600640 1 -- 172.21.15.39:0/2634844951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c7015ca50 msgr2=0x7f1c700cc830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:37.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.094+0000 7f1c75600640 1 --2- 172.21.15.39:0/2634844951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c7015ca50 0x7f1c700cc830 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1c60009830 tx=0x7f1c6002a480 comp rx=0 tx=0).stop 2024-09-21T07:34:37.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.094+0000 7f1c75600640 1 -- 172.21.15.39:0/2634844951 shutdown_connections 2024-09-21T07:34:37.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.094+0000 7f1c75600640 1 --2- 172.21.15.39:0/2634844951 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1c700cf540 0x7f1c7006ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:37.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.094+0000 7f1c75600640 1 --2- 172.21.15.39:0/2634844951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c7015ca50 0x7f1c700cc830 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:37.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.094+0000 7f1c75600640 1 -- 172.21.15.39:0/2634844951 >> 172.21.15.39:0/2634844951 conn(0x7f1c7006d410 msgr2=0x7f1c700b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:37.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.094+0000 7f1c75600640 1 -- 172.21.15.39:0/2634844951 shutdown_connections 2024-09-21T07:34:37.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.094+0000 7f1c75600640 1 -- 172.21.15.39:0/2634844951 wait complete. 2024-09-21T07:34:37.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.096+0000 7f1c75600640 1 Processor -- start 2024-09-21T07:34:37.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.097+0000 7f1c75600640 1 -- start start 2024-09-21T07:34:37.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.098+0000 7f1c75600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c700cf540 0x7f1c70156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:37.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.098+0000 7f1c75600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1c700cf950 0x7f1c701f32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:37.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.098+0000 7f1c75600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c7015ca50 0x7f1c7012f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:37.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.098+0000 7f1c75600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7015de10 con 0x7f1c7015ca50 2024-09-21T07:34:37.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.098+0000 7f1c75600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7015df90 con 0x7f1c700cf540 2024-09-21T07:34:37.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.098+0000 7f1c75600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f1c7015e110 con 0x7f1c700cf950 2024-09-21T07:34:37.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.098+0000 7f1c74c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c7015ca50 0x7f1c7012f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:37.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.098+0000 7f1c74c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c7015ca50 0x7f1c7012f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57478/0 (socket says 172.21.15.39:57478) 2024-09-21T07:34:37.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.098+0000 7f1c74c00640 1 -- 172.21.15.39:0/95834436 learned_addr learned my addr 172.21.15.39:0/95834436 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:37.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.098+0000 7f1c67400640 1 --2- 172.21.15.39:0/95834436 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1c700cf950 0x7f1c701f32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:37.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.098+0000 7f1c6fe00640 1 --2- 172.21.15.39:0/95834436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c700cf540 0x7f1c70156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:37.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.099+0000 7f1c74c00640 1 -- 172.21.15.39:0/95834436 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1c700cf950 msgr2=0x7f1c701f32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:37.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.099+0000 7f1c74c00640 1 --2- 172.21.15.39:0/95834436 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1c700cf950 0x7f1c701f32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:37.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.099+0000 7f1c74c00640 1 -- 172.21.15.39:0/95834436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c700cf540 msgr2=0x7f1c70156850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:37.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.099+0000 7f1c74c00640 1 --2- 172.21.15.39:0/95834436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c700cf540 0x7f1c70156850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:37.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.099+0000 7f1c74c00640 1 -- 172.21.15.39:0/95834436 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c70158780 con 0x7f1c7015ca50 2024-09-21T07:34:37.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.099+0000 7f1c74c00640 1 --2- 172.21.15.39:0/95834436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c7015ca50 0x7f1c7012f6d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1c6800c9f0 tx=0x7f1c6800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:37.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.099+0000 7f1c6cc00640 1 -- 172.21.15.39:0/95834436 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c68016020 con 0x7f1c7015ca50 2024-09-21T07:34:37.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.099+0000 7f1c6cc00640 1 -- 172.21.15.39:0/95834436 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c6800ac60 con 0x7f1c7015ca50 2024-09-21T07:34:37.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.099+0000 7f1c75600640 1 -- 172.21.15.39:0/95834436 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c70158a60 con 0x7f1c7015ca50 2024-09-21T07:34:37.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.099+0000 7f1c6cc00640 1 -- 172.21.15.39:0/95834436 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c68012680 con 0x7f1c7015ca50 2024-09-21T07:34:37.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.099+0000 7f1c75600640 1 -- 172.21.15.39:0/95834436 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c70159020 con 0x7f1c7015ca50 2024-09-21T07:34:37.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.100+0000 7f1c75600640 1 -- 172.21.15.39:0/95834436 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c701584f0 con 0x7f1c7015ca50 2024-09-21T07:34:37.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.102+0000 7f1c6cc00640 1 -- 172.21.15.39:0/95834436 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1c68019020 con 0x7f1c7015ca50 2024-09-21T07:34:37.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.103+0000 7f1c6cc00640 1 --2- 172.21.15.39:0/95834436 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1c4c078b90 0x7f1c4c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:37.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.103+0000 7f1c6cc00640 1 -- 172.21.15.39:0/95834436 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5296+0+0 (secure 0 0 0) 0x7f1c6809b060 con 0x7f1c7015ca50 2024-09-21T07:34:37.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.103+0000 7f1c6fe00640 1 --2- 172.21.15.39:0/95834436 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1c4c078b90 0x7f1c4c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:37.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.104+0000 7f1c6fe00640 1 --2- 172.21.15.39:0/95834436 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1c4c078b90 0x7f1c4c07b0a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1c60034000 tx=0x7f1c60004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:37.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.107+0000 7f1c6cc00640 1 -- 172.21.15.39:0/95834436 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1c68020020 con 0x7f1c7015ca50 2024-09-21T07:34:37.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.302+0000 7f1c75600640 1 -- 172.21.15.39:0/95834436 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f1c7006ee70 con 0x7f1c7015ca50 2024-09-21T07:34:37.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.307+0000 7f1c6cc00640 1 -- 172.21.15.39:0/95834436 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v35)=0 v35) ==== 122+0+0 (secure 0 0 0) 0x7f1c68062f70 con 0x7f1c7015ca50 2024-09-21T07:34:37.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.309+0000 7f1c75600640 1 -- 172.21.15.39:0/95834436 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1c4c078b90 msgr2=0x7f1c4c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:37.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.309+0000 7f1c75600640 1 --2- 172.21.15.39:0/95834436 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1c4c078b90 0x7f1c4c07b0a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1c60034000 tx=0x7f1c60004730 comp rx=0 tx=0).stop 2024-09-21T07:34:37.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.309+0000 7f1c75600640 1 -- 172.21.15.39:0/95834436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c7015ca50 msgr2=0x7f1c7012f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:37.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.309+0000 7f1c75600640 1 --2- 172.21.15.39:0/95834436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c7015ca50 0x7f1c7012f6d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1c6800c9f0 tx=0x7f1c6800ceb0 comp rx=0 tx=0).stop 2024-09-21T07:34:37.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.309+0000 7f1c75600640 1 -- 172.21.15.39:0/95834436 shutdown_connections 2024-09-21T07:34:37.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.309+0000 7f1c75600640 1 --2- 172.21.15.39:0/95834436 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1c4c078b90 0x7f1c4c07b0a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:37.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.309+0000 7f1c75600640 1 --2- 172.21.15.39:0/95834436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c7015ca50 0x7f1c7012f6d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:37.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.309+0000 7f1c75600640 1 --2- 172.21.15.39:0/95834436 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1c700cf950 0x7f1c701f32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:37.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.309+0000 7f1c75600640 1 --2- 172.21.15.39:0/95834436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c700cf540 0x7f1c70156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:37.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.309+0000 7f1c75600640 1 -- 172.21.15.39:0/95834436 >> 172.21.15.39:0/95834436 conn(0x7f1c7006d410 msgr2=0x7f1c701452b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:37.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.309+0000 7f1c75600640 1 -- 172.21.15.39:0/95834436 shutdown_connections 2024-09-21T07:34:37.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.310+0000 7f1c75600640 1 -- 172.21.15.39:0/95834436 wait complete. 2024-09-21T07:34:37.334 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:34:37.334 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:34:37.425 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:34:37.529 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:34:37.529 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:34:37.529 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:34:37.565 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:34:37.566 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:34:37.566 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:34:37.603 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:34:37.604 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:34:37.604 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:34:37.640 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:34:37.640 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:34:37.640 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:34:37.680 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:34:37.680 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T07:34:37.680 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:34:37.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.907+0000 7fb301a00640 1 -- 172.21.15.39:0/2603743353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2e0004800 msgr2=0x7fb2e0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:37.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.907+0000 7fb301a00640 1 --2- 172.21.15.39:0/2603743353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2e0004800 0x7fb2e0004c10 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009b50 tx=0x7fb2e0005830 comp rx=0 tx=0).stop 2024-09-21T07:34:37.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.907+0000 7fb301a00640 1 -- 172.21.15.39:0/2603743353 shutdown_connections 2024-09-21T07:34:37.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.907+0000 7fb301a00640 1 --2- 172.21.15.39:0/2603743353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2e0004800 0x7fb2e0004c10 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:37.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.907+0000 7fb301a00640 1 -- 172.21.15.39:0/2603743353 >> 172.21.15.39:0/2603743353 conn(0x7fb2fc06d870 msgr2=0x7fb2fc06ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:37.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.907+0000 7fb301a00640 1 -- 172.21.15.39:0/2603743353 shutdown_connections 2024-09-21T07:34:37.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.907+0000 7fb301a00640 1 -- 172.21.15.39:0/2603743353 wait complete. 2024-09-21T07:34:37.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.910+0000 7fb301a00640 1 Processor -- start 2024-09-21T07:34:37.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.910+0000 7fb301a00640 1 -- start start 2024-09-21T07:34:37.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.911+0000 7fb301a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2e0004800 0x7fb2fc0c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:37.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.911+0000 7fb301a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb2fc1f6370 0x7fb2fc1f3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:37.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.911+0000 7fb301a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2fc0c9af0 0x7fb2fc1f3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:37.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.911+0000 7fb301a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb2fc153690 con 0x7fb2e0004800 2024-09-21T07:34:37.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.911+0000 7fb301a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb2fc153510 con 0x7fb2fc0c9af0 2024-09-21T07:34:37.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.911+0000 7fb301a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb2fc153810 con 0x7fb2fc1f6370 2024-09-21T07:34:37.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.911+0000 7fb2fbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2e0004800 0x7fb2fc0c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:37.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.912+0000 7fb2fbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2e0004800 0x7fb2fc0c7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57490/0 (socket says 172.21.15.39:57490) 2024-09-21T07:34:37.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.912+0000 7fb2fbe00640 1 -- 172.21.15.39:0/2435933322 learned_addr learned my addr 172.21.15.39:0/2435933322 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:37.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.912+0000 7fb2f3e00640 1 --2- 172.21.15.39:0/2435933322 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb2fc1f6370 0x7fb2fc1f3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:37.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.912+0000 7fb301000640 1 --2- 172.21.15.39:0/2435933322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2fc0c9af0 0x7fb2fc1f3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:37.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.912+0000 7fb2fbe00640 1 -- 172.21.15.39:0/2435933322 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb2fc1f6370 msgr2=0x7fb2fc1f3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:37.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.912+0000 7fb2fbe00640 1 --2- 172.21.15.39:0/2435933322 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb2fc1f6370 0x7fb2fc1f3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:37.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.912+0000 7fb2fbe00640 1 -- 172.21.15.39:0/2435933322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2fc0c9af0 msgr2=0x7fb2fc1f3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:37.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.912+0000 7fb2fbe00640 1 --2- 172.21.15.39:0/2435933322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2fc0c9af0 0x7fb2fc1f3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:37.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.912+0000 7fb2fbe00640 1 -- 172.21.15.39:0/2435933322 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2fc1f43f0 con 0x7fb2e0004800 2024-09-21T07:34:37.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.912+0000 7fb2fbe00640 1 --2- 172.21.15.39:0/2435933322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2e0004800 0x7fb2fc0c7bc0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec000c00 tx=0x7fb2ec001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:37.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.913+0000 7fb2f8c00640 1 -- 172.21.15.39:0/2435933322 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec015020 con 0x7fb2e0004800 2024-09-21T07:34:37.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.913+0000 7fb2f8c00640 1 -- 172.21.15.39:0/2435933322 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb2ec0119a0 con 0x7fb2e0004800 2024-09-21T07:34:37.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.913+0000 7fb301a00640 1 -- 172.21.15.39:0/2435933322 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2fc1f2d80 con 0x7fb2e0004800 2024-09-21T07:34:37.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.913+0000 7fb2f8c00640 1 -- 172.21.15.39:0/2435933322 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec011df0 con 0x7fb2e0004800 2024-09-21T07:34:37.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.913+0000 7fb301a00640 1 -- 172.21.15.39:0/2435933322 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2fc1f6dc0 con 0x7fb2e0004800 2024-09-21T07:34:37.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.914+0000 7fb301a00640 1 -- 172.21.15.39:0/2435933322 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2fc0b59c0 con 0x7fb2e0004800 2024-09-21T07:34:37.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.915+0000 7fb2f8c00640 1 -- 172.21.15.39:0/2435933322 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb2ec018020 con 0x7fb2e0004800 2024-09-21T07:34:37.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.916+0000 7fb2f8c00640 1 --2- 172.21.15.39:0/2435933322 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb2d8078bc0 0x7fb2d807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:37.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.916+0000 7fb2f8c00640 1 -- 172.21.15.39:0/2435933322 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5296+0+0 (secure 0 0 0) 0x7fb2ec09b2c0 con 0x7fb2e0004800 2024-09-21T07:34:37.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.916+0000 7fb2f3e00640 1 --2- 172.21.15.39:0/2435933322 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb2d8078bc0 0x7fb2d807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:37.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.917+0000 7fb2f3e00640 1 --2- 172.21.15.39:0/2435933322 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb2d8078bc0 0x7fb2d807b0d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4035000 tx=0x7fb2e40046c0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:37.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:37.920+0000 7fb2f8c00640 1 -- 172.21.15.39:0/2435933322 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb2ec01f030 con 0x7fb2e0004800 2024-09-21T07:34:38.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.157+0000 7fb301a00640 1 -- 172.21.15.39:0/2435933322 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb2fc0cd060 con 0x7fb2e0004800 2024-09-21T07:34:38.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.158+0000 7fb2f8c00640 1 -- 172.21.15.39:0/2435933322 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+3914 (secure 0 0 0) 0x7fb2ec068e20 con 0x7fb2e0004800 2024-09-21T07:34:38.160 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:34:38.160 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":19,"btime":"2024-09-21T07:34:30:293455+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2438395645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2438395645},{"type":"v1","addr":"172.21.15.148:6837","nonce":2438395645}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4706,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3668947757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3668947757},{"type":"v1","addr":"172.21.15.39:6835","nonce":3668947757}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4870,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2800764560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2800764560},{"type":"v1","addr":"172.21.15.148:6835","nonce":2800764560}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/605789760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":605789760},{"type":"v1","addr":"172.21.15.39:6834","nonce":605789760}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[]} 2024-09-21T07:34:38.160 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 19 2024-09-21T07:34:38.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.160+0000 7fb301a00640 1 -- 172.21.15.39:0/2435933322 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb2d8078bc0 msgr2=0x7fb2d807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:38.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.160+0000 7fb301a00640 1 --2- 172.21.15.39:0/2435933322 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb2d8078bc0 0x7fb2d807b0d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4035000 tx=0x7fb2e40046c0 comp rx=0 tx=0).stop 2024-09-21T07:34:38.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.160+0000 7fb301a00640 1 -- 172.21.15.39:0/2435933322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2e0004800 msgr2=0x7fb2fc0c7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:38.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.160+0000 7fb301a00640 1 --2- 172.21.15.39:0/2435933322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2e0004800 0x7fb2fc0c7bc0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec000c00 tx=0x7fb2ec001800 comp rx=0 tx=0).stop 2024-09-21T07:34:38.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.160+0000 7fb301a00640 1 -- 172.21.15.39:0/2435933322 shutdown_connections 2024-09-21T07:34:38.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.160+0000 7fb301a00640 1 --2- 172.21.15.39:0/2435933322 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb2d8078bc0 0x7fb2d807b0d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:38.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.160+0000 7fb301a00640 1 --2- 172.21.15.39:0/2435933322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2fc0c9af0 0x7fb2fc1f3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:38.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.160+0000 7fb301a00640 1 --2- 172.21.15.39:0/2435933322 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb2fc1f6370 0x7fb2fc1f3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:38.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.160+0000 7fb301a00640 1 --2- 172.21.15.39:0/2435933322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2e0004800 0x7fb2fc0c7bc0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:38.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.160+0000 7fb301a00640 1 -- 172.21.15.39:0/2435933322 >> 172.21.15.39:0/2435933322 conn(0x7fb2fc06d870 msgr2=0x7fb2fc0cbeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:38.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.160+0000 7fb301a00640 1 -- 172.21.15.39:0/2435933322 shutdown_connections 2024-09-21T07:34:38.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.160+0000 7fb301a00640 1 -- 172.21.15.39:0/2435933322 wait complete. 2024-09-21T07:34:38.189 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T07:34:38.189 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T07:34:38.190 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T07:34:38.190 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:34:38.190 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T07:34:38.191 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T07:34:38.191 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T07:34:38.191 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:34:38.191 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T07:34:38.192 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T07:34:38.192 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T07:34:38.192 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:34:38.192 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T07:34:38.192 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T07:34:38.193 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T07:34:38.193 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:34:38.212 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T07:34:38.212 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T07:34:38.215 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T07:34:38.215 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T07:34:38.219 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T07:34:38.219 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T07:34:38.221 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T07:34:38.221 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T07:34:38.259 INFO:tasks.ceph.mds.a:Started 2024-09-21T07:34:38.260 DEBUG:teuthology.parallel:result is None 2024-09-21T07:34:38.262 INFO:tasks.ceph.mds.c:Started 2024-09-21T07:34:38.263 DEBUG:teuthology.parallel:result is None 2024-09-21T07:34:38.265 INFO:tasks.ceph.mds.b:Started 2024-09-21T07:34:38.265 DEBUG:teuthology.parallel:result is None 2024-09-21T07:34:38.267 INFO:tasks.ceph.mds.d:Started 2024-09-21T07:34:38.268 DEBUG:teuthology.parallel:result is None 2024-09-21T07:34:38.268 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T07:34:38.461 INFO:tasks.ceph.mds.c.smithi039.stdout:starting mds.c at 2024-09-21T07:34:38.461 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-21T07:34:38.465 INFO:tasks.ceph.mds.b.smithi148.stdout:starting mds.b at 2024-09-21T07:34:38.467 INFO:tasks.ceph.mds.d.smithi148.stdout:starting mds.d at 2024-09-21T07:34:38.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.506+0000 7fa69a600640 1 -- 172.21.15.39:0/2534460447 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6940b4a00 msgr2=0x7fa6940cf3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:38.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.506+0000 7fa69a600640 1 --2- 172.21.15.39:0/2534460447 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6940b4a00 0x7fa6940cf3e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa68c009be0 tx=0x7fa68c02a820 comp rx=0 tx=0).stop 2024-09-21T07:34:38.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.506+0000 7fa69a600640 1 -- 172.21.15.39:0/2534460447 shutdown_connections 2024-09-21T07:34:38.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.506+0000 7fa69a600640 1 --2- 172.21.15.39:0/2534460447 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6940b4a00 0x7fa6940cf3e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:38.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.506+0000 7fa69a600640 1 -- 172.21.15.39:0/2534460447 >> 172.21.15.39:0/2534460447 conn(0x7fa69409d680 msgr2=0x7fa6940d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:38.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.506+0000 7fa69a600640 1 -- 172.21.15.39:0/2534460447 shutdown_connections 2024-09-21T07:34:38.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.506+0000 7fa69a600640 1 -- 172.21.15.39:0/2534460447 wait complete. 2024-09-21T07:34:38.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.509+0000 7fa69a600640 1 Processor -- start 2024-09-21T07:34:38.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.509+0000 7fa69a600640 1 -- start start 2024-09-21T07:34:38.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.510+0000 7fa69a600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6940b4a00 0x7fa694166a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:38.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.510+0000 7fa69a600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6940b96b0 0x7fa69415f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:38.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.510+0000 7fa69a600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6940cefd0 0x7fa69415fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:38.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.510+0000 7fa69a600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa6940b5040 con 0x7fa6940b4a00 2024-09-21T07:34:38.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.510+0000 7fa69a600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa6940b4ec0 con 0x7fa6940cefd0 2024-09-21T07:34:38.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.510+0000 7fa69a600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa69416a0a0 con 0x7fa6940b96b0 2024-09-21T07:34:38.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.510+0000 7fa699200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6940b4a00 0x7fa694166a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:38.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.510+0000 7fa699200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6940b4a00 0x7fa694166a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57508/0 (socket says 172.21.15.39:57508) 2024-09-21T07:34:38.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.510+0000 7fa699200640 1 -- 172.21.15.39:0/289464240 learned_addr learned my addr 172.21.15.39:0/289464240 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:38.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.510+0000 7fa699c00640 1 --2- 172.21.15.39:0/289464240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6940cefd0 0x7fa69415fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:38.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.510+0000 7fa693e00640 1 --2- 172.21.15.39:0/289464240 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6940b96b0 0x7fa69415f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:38.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.511+0000 7fa699200640 1 -- 172.21.15.39:0/289464240 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6940b96b0 msgr2=0x7fa69415f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:38.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.511+0000 7fa699200640 1 --2- 172.21.15.39:0/289464240 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6940b96b0 0x7fa69415f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:38.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.511+0000 7fa699200640 1 -- 172.21.15.39:0/289464240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6940cefd0 msgr2=0x7fa69415fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:38.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.511+0000 7fa699200640 1 --2- 172.21.15.39:0/289464240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6940cefd0 0x7fa69415fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:38.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.511+0000 7fa699200640 1 -- 172.21.15.39:0/289464240 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa694168880 con 0x7fa6940b4a00 2024-09-21T07:34:38.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.511+0000 7fa699200640 1 --2- 172.21.15.39:0/289464240 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6940b4a00 0x7fa694166a30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa68800c460 tx=0x7fa68800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:38.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.511+0000 7fa690c00640 1 -- 172.21.15.39:0/289464240 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa688017070 con 0x7fa6940b4a00 2024-09-21T07:34:38.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.511+0000 7fa690c00640 1 -- 172.21.15.39:0/289464240 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa688005400 con 0x7fa6940b4a00 2024-09-21T07:34:38.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.511+0000 7fa69a600640 1 -- 172.21.15.39:0/289464240 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6941f8ee0 con 0x7fa6940b4a00 2024-09-21T07:34:38.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.511+0000 7fa690c00640 1 -- 172.21.15.39:0/289464240 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6880038a0 con 0x7fa6940b4a00 2024-09-21T07:34:38.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.512+0000 7fa69a600640 1 -- 172.21.15.39:0/289464240 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6941f94a0 con 0x7fa6940b4a00 2024-09-21T07:34:38.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.513+0000 7fa69a600640 1 -- 172.21.15.39:0/289464240 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa694168a50 con 0x7fa6940b4a00 2024-09-21T07:34:38.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.514+0000 7fa690c00640 1 -- 172.21.15.39:0/289464240 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa68801a070 con 0x7fa6940b4a00 2024-09-21T07:34:38.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.515+0000 7fa690c00640 1 --2- 172.21.15.39:0/289464240 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa674078b90 0x7fa67407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:38.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.515+0000 7fa690c00640 1 -- 172.21.15.39:0/289464240 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5296+0+0 (secure 0 0 0) 0x7fa688099fb0 con 0x7fa6940b4a00 2024-09-21T07:34:38.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.515+0000 7fa693e00640 1 --2- 172.21.15.39:0/289464240 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa674078b90 0x7fa67407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:38.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.516+0000 7fa693e00640 1 --2- 172.21.15.39:0/289464240 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa674078b90 0x7fa67407b0a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa6940b34b0 tx=0x7fa6800016d0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:38.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.519+0000 7fa690c00640 1 -- 172.21.15.39:0/289464240 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa688061ec0 con 0x7fa6940b4a00 2024-09-21T07:34:38.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:38.727+0000 7fa69a600640 1 -- 172.21.15.39:0/289464240 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fa6940cf3e0 con 0x7fa6940b4a00 2024-09-21T07:34:39.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.393+0000 7fa690c00640 1 -- 172.21.15.39:0/289464240 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v43) ==== 97+0+0 (secure 0 0 0) 0x7fa6880668f0 con 0x7fa6940b4a00 2024-09-21T07:34:39.395 INFO:teuthology.orchestra.run.smithi039.stderr: removed all blocklist entries 2024-09-21T07:34:39.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.396+0000 7fa69a600640 1 -- 172.21.15.39:0/289464240 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa674078b90 msgr2=0x7fa67407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:39.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.396+0000 7fa69a600640 1 --2- 172.21.15.39:0/289464240 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa674078b90 0x7fa67407b0a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa6940b34b0 tx=0x7fa6800016d0 comp rx=0 tx=0).stop 2024-09-21T07:34:39.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.396+0000 7fa69a600640 1 -- 172.21.15.39:0/289464240 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6940b4a00 msgr2=0x7fa694166a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:39.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.396+0000 7fa69a600640 1 --2- 172.21.15.39:0/289464240 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6940b4a00 0x7fa694166a30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa68800c460 tx=0x7fa68800c920 comp rx=0 tx=0).stop 2024-09-21T07:34:39.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.396+0000 7fa69a600640 1 -- 172.21.15.39:0/289464240 shutdown_connections 2024-09-21T07:34:39.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.396+0000 7fa69a600640 1 --2- 172.21.15.39:0/289464240 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa674078b90 0x7fa67407b0a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:39.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.396+0000 7fa69a600640 1 --2- 172.21.15.39:0/289464240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6940cefd0 0x7fa69415fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:39.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.396+0000 7fa69a600640 1 --2- 172.21.15.39:0/289464240 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6940b96b0 0x7fa69415f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:39.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.396+0000 7fa69a600640 1 --2- 172.21.15.39:0/289464240 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6940b4a00 0x7fa694166a30 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:39.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.396+0000 7fa69a600640 1 -- 172.21.15.39:0/289464240 >> 172.21.15.39:0/289464240 conn(0x7fa69409d680 msgr2=0x7fa69406e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:39.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.396+0000 7fa69a600640 1 -- 172.21.15.39:0/289464240 shutdown_connections 2024-09-21T07:34:39.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.396+0000 7fa69a600640 1 -- 172.21.15.39:0/289464240 wait complete. 2024-09-21T07:34:39.424 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T07:34:39.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.652+0000 7f6619800640 1 -- 172.21.15.39:0/2452233682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66140d3180 msgr2=0x7f66140b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:39.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.652+0000 7f6619800640 1 --2- 172.21.15.39:0/2452233682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66140d3180 0x7f66140b3d40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f661408d2c0 tx=0x7f660c02aa30 comp rx=0 tx=0).stop 2024-09-21T07:34:39.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.652+0000 7f6619800640 1 -- 172.21.15.39:0/2452233682 shutdown_connections 2024-09-21T07:34:39.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.652+0000 7f6619800640 1 --2- 172.21.15.39:0/2452233682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66140d3180 0x7f66140b3d40 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:39.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.652+0000 7f6619800640 1 --2- 172.21.15.39:0/2452233682 >> v2:172.21.15.148:3300/0 conn(0x7f66140b9730 0x7f66140b33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:39.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.652+0000 7f6619800640 1 -- 172.21.15.39:0/2452233682 >> 172.21.15.39:0/2452233682 conn(0x7f661409d670 msgr2=0x7f66140d4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:39.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.652+0000 7f6619800640 1 -- 172.21.15.39:0/2452233682 shutdown_connections 2024-09-21T07:34:39.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.652+0000 7f6619800640 1 -- 172.21.15.39:0/2452233682 wait complete. 2024-09-21T07:34:39.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.654+0000 7f6619800640 1 Processor -- start 2024-09-21T07:34:39.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.654+0000 7f6619800640 1 -- start start 2024-09-21T07:34:39.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.656+0000 7f6619800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66140b9730 0x7f66140c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:39.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.656+0000 7f6619800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66140d0b70 0x7f6614160300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:39.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.656+0000 7f6619800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f66140d3180 0x7f6614160840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:39.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.656+0000 7f6619800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f66140b4700 con 0x7f66140b9730 2024-09-21T07:34:39.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.656+0000 7f6619800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f66140b4580 con 0x7f66140d0b70 2024-09-21T07:34:39.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.656+0000 7f6619800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f66140b4880 con 0x7f66140d3180 2024-09-21T07:34:39.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.656+0000 7f6613e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66140b9730 0x7f66140c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:39.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.656+0000 7f6613e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66140b9730 0x7f66140c9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57522/0 (socket says 172.21.15.39:57522) 2024-09-21T07:34:39.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.656+0000 7f6613e00640 1 -- 172.21.15.39:0/109613634 learned_addr learned my addr 172.21.15.39:0/109613634 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:39.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.656+0000 7f660be00640 1 --2- 172.21.15.39:0/109613634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66140d0b70 0x7f6614160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:39.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.656+0000 7f6618e00640 1 --2- 172.21.15.39:0/109613634 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f66140d3180 0x7f6614160840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:39.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.657+0000 7f6613e00640 1 -- 172.21.15.39:0/109613634 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f66140d3180 msgr2=0x7f6614160840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:39.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.657+0000 7f6613e00640 1 --2- 172.21.15.39:0/109613634 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f66140d3180 0x7f6614160840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:39.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.657+0000 7f6613e00640 1 -- 172.21.15.39:0/109613634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66140d0b70 msgr2=0x7f6614160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:39.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.657+0000 7f6613e00640 1 --2- 172.21.15.39:0/109613634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66140d0b70 0x7f6614160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:39.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.657+0000 7f6613e00640 1 -- 172.21.15.39:0/109613634 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6614155bf0 con 0x7f66140b9730 2024-09-21T07:34:39.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.657+0000 7f6613e00640 1 --2- 172.21.15.39:0/109613634 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66140b9730 0x7f66140c9bd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f660400d700 tx=0x7f660400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:39.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.657+0000 7f6610c00640 1 -- 172.21.15.39:0/109613634 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6604017070 con 0x7f66140b9730 2024-09-21T07:34:39.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.657+0000 7f6610c00640 1 -- 172.21.15.39:0/109613634 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6604009d20 con 0x7f66140b9730 2024-09-21T07:34:39.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.657+0000 7f6610c00640 1 -- 172.21.15.39:0/109613634 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66040106c0 con 0x7f66140b9730 2024-09-21T07:34:39.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.657+0000 7f6619800640 1 -- 172.21.15.39:0/109613634 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66141e8aa0 con 0x7f66140b9730 2024-09-21T07:34:39.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.657+0000 7f6619800640 1 -- 172.21.15.39:0/109613634 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66141e84e0 con 0x7f66140b9730 2024-09-21T07:34:39.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.658+0000 7f6619800640 1 -- 172.21.15.39:0/109613634 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66140b4d30 con 0x7f66140b9730 2024-09-21T07:34:39.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.660+0000 7f6610c00640 1 -- 172.21.15.39:0/109613634 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f660401a070 con 0x7f66140b9730 2024-09-21T07:34:39.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.661+0000 7f6610c00640 1 --2- 172.21.15.39:0/109613634 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f65f0078b90 0x7f65f007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:39.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.661+0000 7f6610c00640 1 -- 172.21.15.39:0/109613634 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5210+0+0 (secure 0 0 0) 0x7f660409ace0 con 0x7f66140b9730 2024-09-21T07:34:39.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.661+0000 7f660be00640 1 --2- 172.21.15.39:0/109613634 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f65f0078b90 0x7f65f007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:39.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.662+0000 7f660be00640 1 --2- 172.21.15.39:0/109613634 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f65f0078b90 0x7f65f007b0a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f66140b58f0 tx=0x7f65fc009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:39.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.665+0000 7f6610c00640 1 -- 172.21.15.39:0/109613634 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6614155bf0 con 0x7f66140b9730 2024-09-21T07:34:39.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.938+0000 7f6619800640 1 -- 172.21.15.39:0/109613634 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f661406ef60 con 0x7f66140b9730 2024-09-21T07:34:39.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.938+0000 7f6610c00640 1 -- 172.21.15.39:0/109613634 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v9) ==== 80+0+5304 (secure 0 0 0) 0x7f6604062cb0 con 0x7f66140b9730 2024-09-21T07:34:39.940 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:34:39.941 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:34:39.941 INFO:teuthology.orchestra.run.smithi039.stdout: "auth_dump": [ 2024-09-21T07:34:39.941 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:39.941 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.a", 2024-09-21T07:34:39.941 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5m+P6EERAABQqmlkrocSi1+hwc73Adrg==", 2024-09-21T07:34:39.941 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:39.941 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:34:39.941 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:34:39.941 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:34:39.941 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:34:39.941 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:39.941 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:39.941 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:39.941 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.b", 2024-09-21T07:34:39.942 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5m5/GeLRAAfE8fr5dbitQa89rNK9pZ2Q==", 2024-09-21T07:34:39.942 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:39.942 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:34:39.942 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:34:39.942 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:34:39.942 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:34:39.942 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:39.942 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:39.942 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:39.942 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.c", 2024-09-21T07:34:39.942 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5msbx6HxAALsF8ORvIoJpIAFZ5uxmOEw==", 2024-09-21T07:34:39.942 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:39.942 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:34:39.943 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:34:39.943 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:34:39.943 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:34:39.943 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:39.943 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:39.943 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:39.943 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.d", 2024-09-21T07:34:39.943 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mIrcBABAApKAB/7FkZ5glNTxlbkIsBw==", 2024-09-21T07:34:39.943 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:39.943 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:34:39.943 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:34:39.943 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:34:39.943 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:34:39.943 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:39.944 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:39.944 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:39.944 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.0", 2024-09-21T07:34:39.944 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB7du5msw81DhAAmVimqN4x9K3YpT0nbovppQ==", 2024-09-21T07:34:39.944 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:39.944 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:34:39.944 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:34:39.944 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:34:39.944 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:39.944 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:39.944 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:39.944 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.1", 2024-09-21T07:34:39.944 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB+du5mdnCIOhAAtVB4ThehQwb0zRriyKcaxA==", 2024-09-21T07:34:39.944 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:39.944 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:34:39.945 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:34:39.945 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:34:39.945 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:39.945 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:39.945 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:39.945 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.2", 2024-09-21T07:34:39.945 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCCdu5mTfWxDhAAadJCbPN5xBx/f9L0v2yTdg==", 2024-09-21T07:34:39.945 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:39.945 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:34:39.945 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:34:39.945 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:34:39.945 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:39.945 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:39.945 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:39.945 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.3", 2024-09-21T07:34:39.946 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCFdu5mL1E2HxAAyOhY97uUXEw5L1CcDeONVA==", 2024-09-21T07:34:39.946 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:39.946 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:34:39.946 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:34:39.946 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:34:39.946 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:39.946 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:39.946 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:39.946 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.4", 2024-09-21T07:34:39.946 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCLdu5m6PY/IhAAgvcTgm+6uWA7lIG4tcTIEg==", 2024-09-21T07:34:39.946 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:39.946 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:34:39.946 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:34:39.946 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:34:39.947 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:39.947 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:39.947 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:39.947 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.5", 2024-09-21T07:34:39.947 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCOdu5mE/wlMxAA3JBg4xH+N/PPC1Uv2ZVrYQ==", 2024-09-21T07:34:39.947 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:39.947 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:34:39.947 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:34:39.947 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:34:39.947 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:39.947 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:39.947 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:39.947 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.6", 2024-09-21T07:34:39.947 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCSdu5mC3TgBxAAwsArjeIdFcApY3/1OJffEg==", 2024-09-21T07:34:39.948 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:39.948 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:34:39.948 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:34:39.948 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:34:39.948 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:39.948 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:39.948 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:39.948 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.7", 2024-09-21T07:34:39.948 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCVdu5mSWHWFBAA9qPHWRdsvrB74inrp1lj4Q==", 2024-09-21T07:34:39.948 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:39.948 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:34:39.948 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:34:39.948 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:34:39.948 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:39.948 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:39.949 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:39.949 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.0", 2024-09-21T07:34:39.949 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mGfb1DBAA/aSMKR5or3xy53rB/iUjpA==", 2024-09-21T07:34:39.949 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:39.949 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:34:39.949 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow r", 2024-09-21T07:34:39.949 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T07:34:39.949 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:39.949 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:39.949 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:39.949 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.1", 2024-09-21T07:34:39.949 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mEkciIhAAid+bxJEr4P7vMnZ42PAy5w==", 2024-09-21T07:34:39.949 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:39.949 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:34:39.950 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow r", 2024-09-21T07:34:39.950 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow rw", 2024-09-21T07:34:39.950 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rwx" 2024-09-21T07:34:39.950 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:39.950 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:39.950 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:39.950 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.2", 2024-09-21T07:34:39.950 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mSu2oFxAAgjy2m/wWKh6xaoh+YDoU1Q==", 2024-09-21T07:34:39.950 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:39.950 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:34:39.950 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow r", 2024-09-21T07:34:39.950 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow rw", 2024-09-21T07:34:39.950 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rwx" 2024-09-21T07:34:39.950 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:39.951 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:39.951 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:39.951 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.3", 2024-09-21T07:34:39.951 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mCHy0LBAAZneF2jUWgfuyGehZFbOxDg==", 2024-09-21T07:34:39.951 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:39.951 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:34:39.951 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow r", 2024-09-21T07:34:39.951 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow rw", 2024-09-21T07:34:39.951 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rwx" 2024-09-21T07:34:39.951 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:39.951 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:39.951 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:39.951 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.admin", 2024-09-21T07:34:39.951 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBydu5m9LInERAAAlDpqb94f2Qb9VYJJwTBcA==", 2024-09-21T07:34:39.951 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:39.952 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow *", 2024-09-21T07:34:39.952 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:34:39.952 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:34:39.952 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:34:39.952 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:39.952 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:39.952 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:39.952 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mgr.x", 2024-09-21T07:34:39.952 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5mA2GRBhAAkyCKdgH6laHy8xNt080dXg==", 2024-09-21T07:34:39.952 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:39.952 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow *", 2024-09-21T07:34:39.952 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile mgr", 2024-09-21T07:34:39.952 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:34:39.952 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:39.953 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:39.953 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:39.953 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mgr.y", 2024-09-21T07:34:39.953 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBydu5mr+8aNxAA7TFYNms2WsXUfa4ITTJOHw==", 2024-09-21T07:34:39.953 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:34:39.953 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow *", 2024-09-21T07:34:39.953 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile mgr", 2024-09-21T07:34:39.953 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:34:39.953 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:39.953 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:39.953 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-21T07:34:39.953 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:34:39.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.941+0000 7f6619800640 1 -- 172.21.15.39:0/109613634 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f65f0078b90 msgr2=0x7f65f007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:39.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.941+0000 7f6619800640 1 --2- 172.21.15.39:0/109613634 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f65f0078b90 0x7f65f007b0a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f66140b58f0 tx=0x7f65fc009040 comp rx=0 tx=0).stop 2024-09-21T07:34:39.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.941+0000 7f6619800640 1 -- 172.21.15.39:0/109613634 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66140b9730 msgr2=0x7f66140c9bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:39.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.941+0000 7f6619800640 1 --2- 172.21.15.39:0/109613634 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66140b9730 0x7f66140c9bd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f660400d700 tx=0x7f660400dbc0 comp rx=0 tx=0).stop 2024-09-21T07:34:39.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.941+0000 7f6619800640 1 -- 172.21.15.39:0/109613634 shutdown_connections 2024-09-21T07:34:39.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.941+0000 7f6619800640 1 --2- 172.21.15.39:0/109613634 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f65f0078b90 0x7f65f007b0a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:39.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.941+0000 7f6619800640 1 --2- 172.21.15.39:0/109613634 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f66140d3180 0x7f6614160840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:39.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.941+0000 7f6619800640 1 --2- 172.21.15.39:0/109613634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66140d0b70 0x7f6614160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:39.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.941+0000 7f6619800640 1 --2- 172.21.15.39:0/109613634 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66140b9730 0x7f66140c9bd0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:39.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.941+0000 7f6619800640 1 -- 172.21.15.39:0/109613634 >> 172.21.15.39:0/109613634 conn(0x7f661409d670 msgr2=0x7f661406f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:39.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.941+0000 7f6619800640 1 -- 172.21.15.39:0/109613634 shutdown_connections 2024-09-21T07:34:39.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:39.941+0000 7f6619800640 1 -- 172.21.15.39:0/109613634 wait complete. 2024-09-21T07:34:39.964 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T07:34:40.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.193+0000 7f15f0600640 1 -- 172.21.15.39:0/3446813103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15d0003c80 msgr2=0x7f15d00040d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:40.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.193+0000 7f15f0600640 1 --2- 172.21.15.39:0/3446813103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15d0003c80 0x7f15d00040d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f15dc009b50 tx=0x7f15d0005830 comp rx=0 tx=0).stop 2024-09-21T07:34:40.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.193+0000 7f15f0600640 1 -- 172.21.15.39:0/3446813103 shutdown_connections 2024-09-21T07:34:40.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.193+0000 7f15f0600640 1 --2- 172.21.15.39:0/3446813103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15d0003c80 0x7f15d00040d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:40.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.194+0000 7f15f0600640 1 -- 172.21.15.39:0/3446813103 >> 172.21.15.39:0/3446813103 conn(0x7f15e80b1cb0 msgr2=0x7f15e80b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:40.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.194+0000 7f15f0600640 1 -- 172.21.15.39:0/3446813103 shutdown_connections 2024-09-21T07:34:40.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.194+0000 7f15f0600640 1 -- 172.21.15.39:0/3446813103 wait complete. 2024-09-21T07:34:40.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.196+0000 7f15f0600640 1 Processor -- start 2024-09-21T07:34:40.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.196+0000 7f15f0600640 1 -- start start 2024-09-21T07:34:40.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.197+0000 7f15f0600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15d0003c80 0x7f15e80d0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:40.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.197+0000 7f15f0600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15e80d0a90 0x7f15e80cb540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:40.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.197+0000 7f15f0600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15e80cba80 0x7f15e80b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:40.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.197+0000 7f15f0600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f15e8074a20 con 0x7f15e80d0a90 2024-09-21T07:34:40.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.197+0000 7f15f0600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f15e80748a0 con 0x7f15d0003c80 2024-09-21T07:34:40.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.197+0000 7f15f0600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f15e8074ba0 con 0x7f15e80cba80 2024-09-21T07:34:40.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.198+0000 7f15ed200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15e80d0a90 0x7f15e80cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:40.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.198+0000 7f15ed200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15e80d0a90 0x7f15e80cb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57554/0 (socket says 172.21.15.39:57554) 2024-09-21T07:34:40.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.198+0000 7f15ed200640 1 -- 172.21.15.39:0/4185746265 learned_addr learned my addr 172.21.15.39:0/4185746265 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:40.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.198+0000 7f15edc00640 1 --2- 172.21.15.39:0/4185746265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15d0003c80 0x7f15e80d0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:40.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.198+0000 7f15ee600640 1 --2- 172.21.15.39:0/4185746265 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15e80cba80 0x7f15e80b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:40.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.198+0000 7f15ed200640 1 -- 172.21.15.39:0/4185746265 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15e80cba80 msgr2=0x7f15e80b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:40.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.198+0000 7f15ed200640 1 --2- 172.21.15.39:0/4185746265 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15e80cba80 0x7f15e80b0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:40.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.198+0000 7f15ed200640 1 -- 172.21.15.39:0/4185746265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15d0003c80 msgr2=0x7f15e80d0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:40.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.198+0000 7f15ed200640 1 --2- 172.21.15.39:0/4185746265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15d0003c80 0x7f15e80d0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:40.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.198+0000 7f15ed200640 1 -- 172.21.15.39:0/4185746265 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e81f56b0 con 0x7f15e80d0a90 2024-09-21T07:34:40.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.198+0000 7f15edc00640 1 --2- 172.21.15.39:0/4185746265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15d0003c80 0x7f15e80d0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:34:40.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.199+0000 7f15ed200640 1 --2- 172.21.15.39:0/4185746265 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15e80d0a90 0x7f15e80cb540 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f15d40063d0 tx=0x7f15d40039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:40.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.199+0000 7f15e5600640 1 -- 172.21.15.39:0/4185746265 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d4015070 con 0x7f15e80d0a90 2024-09-21T07:34:40.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.199+0000 7f15e5600640 1 -- 172.21.15.39:0/4185746265 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15d4011840 con 0x7f15e80d0a90 2024-09-21T07:34:40.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.199+0000 7f15e5600640 1 -- 172.21.15.39:0/4185746265 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d4011c90 con 0x7f15e80d0a90 2024-09-21T07:34:40.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.199+0000 7f15f0600640 1 -- 172.21.15.39:0/4185746265 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15e81f5990 con 0x7f15e80d0a90 2024-09-21T07:34:40.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.199+0000 7f15f0600640 1 -- 172.21.15.39:0/4185746265 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15e81f5f50 con 0x7f15e80d0a90 2024-09-21T07:34:40.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.200+0000 7f15f0600640 1 -- 172.21.15.39:0/4185746265 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15e80d0f10 con 0x7f15e80d0a90 2024-09-21T07:34:40.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.201+0000 7f15e5600640 1 -- 172.21.15.39:0/4185746265 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f15d4018070 con 0x7f15e80d0a90 2024-09-21T07:34:40.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.202+0000 7f15e5600640 1 --2- 172.21.15.39:0/4185746265 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f15c8078b90 0x7f15c807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:40.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.202+0000 7f15e5600640 1 -- 172.21.15.39:0/4185746265 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5210+0+0 (secure 0 0 0) 0x7f15d409b7b0 con 0x7f15e80d0a90 2024-09-21T07:34:40.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.203+0000 7f15edc00640 1 --2- 172.21.15.39:0/4185746265 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f15c8078b90 0x7f15c807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:40.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.204+0000 7f15edc00640 1 --2- 172.21.15.39:0/4185746265 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f15c8078b90 0x7f15c807b0a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f15dc035000 tx=0x7f15dc0046c0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:40.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.207+0000 7f15e5600640 1 -- 172.21.15.39:0/4185746265 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f15d4063dd0 con 0x7f15e80d0a90 2024-09-21T07:34:40.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.406+0000 7f15f0600640 1 -- 172.21.15.39:0/4185746265 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f15e80c3860 con 0x7f15e80d0a90 2024-09-21T07:34:40.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.406+0000 7f15e5600640 1 -- 172.21.15.39:0/4185746265 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v45) ==== 84+0+137 (secure 0 0 0) 0x7f15d40636d0 con 0x7f15e80d0a90 2024-09-21T07:34:40.409 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:34:40.409 INFO:teuthology.orchestra.run.smithi039.stdout:[ 2024-09-21T07:34:40.409 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:40.409 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 1, 2024-09-21T07:34:40.409 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": ".mgr" 2024-09-21T07:34:40.409 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:34:40.409 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:34:40.409 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 6, 2024-09-21T07:34:40.409 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": "snap-test-pool" 2024-09-21T07:34:40.409 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:34:40.409 INFO:teuthology.orchestra.run.smithi039.stdout:] 2024-09-21T07:34:40.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.409+0000 7f15f0600640 1 -- 172.21.15.39:0/4185746265 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f15c8078b90 msgr2=0x7f15c807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:40.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.409+0000 7f15f0600640 1 --2- 172.21.15.39:0/4185746265 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f15c8078b90 0x7f15c807b0a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f15dc035000 tx=0x7f15dc0046c0 comp rx=0 tx=0).stop 2024-09-21T07:34:40.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.409+0000 7f15f0600640 1 -- 172.21.15.39:0/4185746265 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15e80d0a90 msgr2=0x7f15e80cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:40.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.409+0000 7f15f0600640 1 --2- 172.21.15.39:0/4185746265 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15e80d0a90 0x7f15e80cb540 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f15d40063d0 tx=0x7f15d40039c0 comp rx=0 tx=0).stop 2024-09-21T07:34:40.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.409+0000 7f15f0600640 1 -- 172.21.15.39:0/4185746265 shutdown_connections 2024-09-21T07:34:40.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.409+0000 7f15f0600640 1 --2- 172.21.15.39:0/4185746265 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f15c8078b90 0x7f15c807b0a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:40.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.409+0000 7f15f0600640 1 --2- 172.21.15.39:0/4185746265 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15e80cba80 0x7f15e80b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:40.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.409+0000 7f15f0600640 1 --2- 172.21.15.39:0/4185746265 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15e80d0a90 0x7f15e80cb540 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:40.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.409+0000 7f15f0600640 1 --2- 172.21.15.39:0/4185746265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15d0003c80 0x7f15e80d0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:40.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.409+0000 7f15f0600640 1 -- 172.21.15.39:0/4185746265 >> 172.21.15.39:0/4185746265 conn(0x7f15e80b1cb0 msgr2=0x7f15e815eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:40.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.409+0000 7f15f0600640 1 -- 172.21.15.39:0/4185746265 shutdown_connections 2024-09-21T07:34:40.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.409+0000 7f15f0600640 1 -- 172.21.15.39:0/4185746265 wait complete. 2024-09-21T07:34:40.434 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T07:34:40.434 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T07:34:40.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.664+0000 7fc761200640 1 -- 172.21.15.39:0/2562671145 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc75c0d0440 msgr2=0x7fc75c0cd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:40.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.664+0000 7fc761200640 1 --2- 172.21.15.39:0/2562671145 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc75c0d0440 0x7fc75c0cd900 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc75c153bf0 tx=0x7fc74c02a600 comp rx=0 tx=0).stop 2024-09-21T07:34:40.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.664+0000 7fc761200640 1 -- 172.21.15.39:0/2562671145 shutdown_connections 2024-09-21T07:34:40.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.664+0000 7fc761200640 1 --2- 172.21.15.39:0/2562671145 >> v2:172.21.15.148:3300/0 conn(0x7fc75c074580 0x7fc75c15d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:40.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.664+0000 7fc761200640 1 --2- 172.21.15.39:0/2562671145 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc75c0d0440 0x7fc75c0cd900 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:40.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.664+0000 7fc761200640 1 -- 172.21.15.39:0/2562671145 >> 172.21.15.39:0/2562671145 conn(0x7fc75c0d1370 msgr2=0x7fc75c073760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:40.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.665+0000 7fc761200640 1 -- 172.21.15.39:0/2562671145 shutdown_connections 2024-09-21T07:34:40.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.665+0000 7fc761200640 1 -- 172.21.15.39:0/2562671145 wait complete. 2024-09-21T07:34:40.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.667+0000 7fc761200640 1 Processor -- start 2024-09-21T07:34:40.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.667+0000 7fc761200640 1 -- start start 2024-09-21T07:34:40.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.669+0000 7fc761200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc75c0af900 0x7fc75c151460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:40.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.669+0000 7fc761200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc75c0d0440 0x7fc75c1536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:40.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.669+0000 7fc761200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc75c1f6420 0x7fc75c151e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:40.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.669+0000 7fc761200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c15e080 con 0x7fc75c0d0440 2024-09-21T07:34:40.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.669+0000 7fc761200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc75c15df00 con 0x7fc75c0af900 2024-09-21T07:34:40.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.669+0000 7fc761200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc75c15e200 con 0x7fc75c1f6420 2024-09-21T07:34:40.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.669+0000 7fc759400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc75c0d0440 0x7fc75c1536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:40.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.669+0000 7fc759400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc75c0d0440 0x7fc75c1536d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57566/0 (socket says 172.21.15.39:57566) 2024-09-21T07:34:40.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.669+0000 7fc759400640 1 -- 172.21.15.39:0/3020164087 learned_addr learned my addr 172.21.15.39:0/3020164087 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:40.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.669+0000 7fc759e00640 1 --2- 172.21.15.39:0/3020164087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc75c0af900 0x7fc75c151460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:40.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.669+0000 7fc75a800640 1 --2- 172.21.15.39:0/3020164087 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc75c1f6420 0x7fc75c151e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:40.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.669+0000 7fc759400640 1 -- 172.21.15.39:0/3020164087 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc75c1f6420 msgr2=0x7fc75c151e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:40.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.669+0000 7fc759400640 1 --2- 172.21.15.39:0/3020164087 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc75c1f6420 0x7fc75c151e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:40.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.669+0000 7fc759400640 1 -- 172.21.15.39:0/3020164087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc75c0af900 msgr2=0x7fc75c151460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:40.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.669+0000 7fc759400640 1 --2- 172.21.15.39:0/3020164087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc75c0af900 0x7fc75c151460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:40.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.669+0000 7fc759400640 1 -- 172.21.15.39:0/3020164087 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc75c0aa510 con 0x7fc75c0d0440 2024-09-21T07:34:40.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.669+0000 7fc75a800640 1 --2- 172.21.15.39:0/3020164087 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc75c1f6420 0x7fc75c151e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:34:40.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.670+0000 7fc759400640 1 --2- 172.21.15.39:0/3020164087 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc75c0d0440 0x7fc75c1536d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc74800c370 tx=0x7fc74800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:40.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.670+0000 7fc752000640 1 -- 172.21.15.39:0/3020164087 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc748017070 con 0x7fc75c0d0440 2024-09-21T07:34:40.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.670+0000 7fc752000640 1 -- 172.21.15.39:0/3020164087 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7480053c0 con 0x7fc75c0d0440 2024-09-21T07:34:40.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.670+0000 7fc752000640 1 -- 172.21.15.39:0/3020164087 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc748003850 con 0x7fc75c0d0440 2024-09-21T07:34:40.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.670+0000 7fc761200640 1 -- 172.21.15.39:0/3020164087 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc75c150980 con 0x7fc75c0d0440 2024-09-21T07:34:40.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.670+0000 7fc761200640 1 -- 172.21.15.39:0/3020164087 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc75c152990 con 0x7fc75c0d0440 2024-09-21T07:34:40.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.671+0000 7fc761200640 1 -- 172.21.15.39:0/3020164087 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc75c15e790 con 0x7fc75c0d0440 2024-09-21T07:34:40.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.672+0000 7fc752000640 1 -- 172.21.15.39:0/3020164087 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc74801a070 con 0x7fc75c0d0440 2024-09-21T07:34:40.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.673+0000 7fc752000640 1 --2- 172.21.15.39:0/3020164087 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc73c078bc0 0x7fc73c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:40.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.673+0000 7fc752000640 1 -- 172.21.15.39:0/3020164087 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5210+0+0 (secure 0 0 0) 0x7fc748099e20 con 0x7fc75c0d0440 2024-09-21T07:34:40.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.674+0000 7fc759e00640 1 --2- 172.21.15.39:0/3020164087 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc73c078bc0 0x7fc73c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:40.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.674+0000 7fc759e00640 1 --2- 172.21.15.39:0/3020164087 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc73c078bc0 0x7fc73c07b0d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc74c033000 tx=0x7fc74c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:40.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.677+0000 7fc752000640 1 -- 172.21.15.39:0/3020164087 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc75c0cde40 con 0x7fc75c0d0440 2024-09-21T07:34:40.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:40.873+0000 7fc761200640 1 -- 172.21.15.39:0/3020164087 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc75c06e3d0 con 0x7fc75c0d0440 2024-09-21T07:34:41.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.415+0000 7fc752000640 1 -- 172.21.15.39:0/3020164087 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v46) ==== 138+0+0 (secure 0 0 0) 0x7fc7480624d0 con 0x7fc75c0d0440 2024-09-21T07:34:41.417 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' created 2024-09-21T07:34:41.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.417+0000 7fc761200640 1 -- 172.21.15.39:0/3020164087 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc73c078bc0 msgr2=0x7fc73c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:41.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.417+0000 7fc761200640 1 --2- 172.21.15.39:0/3020164087 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc73c078bc0 0x7fc73c07b0d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc74c033000 tx=0x7fc74c004730 comp rx=0 tx=0).stop 2024-09-21T07:34:41.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.417+0000 7fc761200640 1 -- 172.21.15.39:0/3020164087 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc75c0d0440 msgr2=0x7fc75c1536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:41.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.417+0000 7fc761200640 1 --2- 172.21.15.39:0/3020164087 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc75c0d0440 0x7fc75c1536d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc74800c370 tx=0x7fc74800c830 comp rx=0 tx=0).stop 2024-09-21T07:34:41.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.418+0000 7fc761200640 1 -- 172.21.15.39:0/3020164087 shutdown_connections 2024-09-21T07:34:41.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.418+0000 7fc761200640 1 --2- 172.21.15.39:0/3020164087 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc73c078bc0 0x7fc73c07b0d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:41.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.418+0000 7fc761200640 1 --2- 172.21.15.39:0/3020164087 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc75c1f6420 0x7fc75c151e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:41.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.418+0000 7fc761200640 1 --2- 172.21.15.39:0/3020164087 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc75c0d0440 0x7fc75c1536d0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:41.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.418+0000 7fc761200640 1 --2- 172.21.15.39:0/3020164087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc75c0af900 0x7fc75c151460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:41.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.418+0000 7fc761200640 1 -- 172.21.15.39:0/3020164087 >> 172.21.15.39:0/3020164087 conn(0x7fc75c0d1370 msgr2=0x7fc75c0b0430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:41.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.418+0000 7fc761200640 1 -- 172.21.15.39:0/3020164087 shutdown_connections 2024-09-21T07:34:41.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.418+0000 7fc761200640 1 -- 172.21.15.39:0/3020164087 wait complete. 2024-09-21T07:34:41.447 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T07:34:41.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.682+0000 7f3ce3e00640 1 -- 172.21.15.39:0/2222930423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ce406f580 msgr2=0x7f3ce415d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:41.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.682+0000 7f3ce3e00640 1 --2- 172.21.15.39:0/2222930423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ce406f580 0x7f3ce415d700 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc005fd0 tx=0x7f3cdc02a840 comp rx=0 tx=0).stop 2024-09-21T07:34:41.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.682+0000 7f3ce3e00640 1 -- 172.21.15.39:0/2222930423 shutdown_connections 2024-09-21T07:34:41.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.682+0000 7f3ce3e00640 1 --2- 172.21.15.39:0/2222930423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ce406f580 0x7f3ce415d700 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:41.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.682+0000 7f3ce3e00640 1 -- 172.21.15.39:0/2222930423 >> 172.21.15.39:0/2222930423 conn(0x7f3ce406d410 msgr2=0x7f3ce40b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:41.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.682+0000 7f3ce3e00640 1 -- 172.21.15.39:0/2222930423 shutdown_connections 2024-09-21T07:34:41.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.682+0000 7f3ce3e00640 1 -- 172.21.15.39:0/2222930423 wait complete. 2024-09-21T07:34:41.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.684+0000 7f3ce3e00640 1 Processor -- start 2024-09-21T07:34:41.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.684+0000 7f3ce3e00640 1 -- start start 2024-09-21T07:34:41.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.686+0000 7f3ce3e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ce40cf540 0x7f3ce41f5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:41.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.686+0000 7f3ce3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ce415ca50 0x7f3ce4143a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:41.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.686+0000 7f3ce3e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ce4155d30 0x7f3ce4156140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:41.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.686+0000 7f3ce3e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce415e010 con 0x7f3ce415ca50 2024-09-21T07:34:41.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.686+0000 7f3ce3e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce415de90 con 0x7f3ce40cf540 2024-09-21T07:34:41.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.686+0000 7f3ce3e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f3ce415e190 con 0x7f3ce4155d30 2024-09-21T07:34:41.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.686+0000 7f3ce2000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ce415ca50 0x7f3ce4143a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:41.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.686+0000 7f3ce2000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ce415ca50 0x7f3ce4143a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57574/0 (socket says 172.21.15.39:57574) 2024-09-21T07:34:41.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.686+0000 7f3ce2000640 1 -- 172.21.15.39:0/4245244163 learned_addr learned my addr 172.21.15.39:0/4245244163 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:41.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.686+0000 7f3ce2a00640 1 --2- 172.21.15.39:0/4245244163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ce40cf540 0x7f3ce41f5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:41.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.686+0000 7f3ce3400640 1 --2- 172.21.15.39:0/4245244163 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ce4155d30 0x7f3ce4156140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:41.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.687+0000 7f3ce2000640 1 -- 172.21.15.39:0/4245244163 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ce4155d30 msgr2=0x7f3ce4156140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:41.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.687+0000 7f3ce2000640 1 --2- 172.21.15.39:0/4245244163 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ce4155d30 0x7f3ce4156140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:41.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.687+0000 7f3ce2000640 1 -- 172.21.15.39:0/4245244163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ce40cf540 msgr2=0x7f3ce41f5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:41.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.687+0000 7f3ce2000640 1 --2- 172.21.15.39:0/4245244163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ce40cf540 0x7f3ce41f5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:41.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.687+0000 7f3ce2000640 1 -- 172.21.15.39:0/4245244163 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ce4156680 con 0x7f3ce415ca50 2024-09-21T07:34:41.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.687+0000 7f3ce2000640 1 --2- 172.21.15.39:0/4245244163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ce415ca50 0x7f3ce4143a80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3cd000c4d0 tx=0x7f3cd000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:41.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.688+0000 7f3cdaa00640 1 -- 172.21.15.39:0/4245244163 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd0017070 con 0x7f3ce415ca50 2024-09-21T07:34:41.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.688+0000 7f3ce3e00640 1 -- 172.21.15.39:0/4245244163 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ce41ebca0 con 0x7f3ce415ca50 2024-09-21T07:34:41.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.688+0000 7f3cdaa00640 1 -- 172.21.15.39:0/4245244163 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3cd00035f0 con 0x7f3ce415ca50 2024-09-21T07:34:41.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.688+0000 7f3cdaa00640 1 -- 172.21.15.39:0/4245244163 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd0003a00 con 0x7f3ce415ca50 2024-09-21T07:34:41.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.688+0000 7f3ce3e00640 1 -- 172.21.15.39:0/4245244163 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ce41ec260 con 0x7f3ce415ca50 2024-09-21T07:34:41.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.689+0000 7f3ce3e00640 1 -- 172.21.15.39:0/4245244163 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ce415d700 con 0x7f3ce415ca50 2024-09-21T07:34:41.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.690+0000 7f3cdaa00640 1 -- 172.21.15.39:0/4245244163 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3cd001a070 con 0x7f3ce415ca50 2024-09-21T07:34:41.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.692+0000 7f3cdaa00640 1 --2- 172.21.15.39:0/4245244163 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3cc4078bc0 0x7f3cc407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:41.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.692+0000 7f3cdaa00640 1 -- 172.21.15.39:0/4245244163 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5580+0+0 (secure 0 0 0) 0x7f3cd0099f90 con 0x7f3ce415ca50 2024-09-21T07:34:41.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.692+0000 7f3ce2a00640 1 --2- 172.21.15.39:0/4245244163 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3cc4078bc0 0x7f3cc407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:41.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.693+0000 7f3ce2a00640 1 --2- 172.21.15.39:0/4245244163 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3cc4078bc0 0x7f3cc407b0d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4001120 tx=0x7f3cd4009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:41.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.697+0000 7f3cdaa00640 1 -- 172.21.15.39:0/4245244163 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3ce4156680 con 0x7f3ce415ca50 2024-09-21T07:34:41.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:41.893+0000 7f3ce3e00640 1 -- 172.21.15.39:0/4245244163 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f3ce406fb50 con 0x7f3ce415ca50 2024-09-21T07:34:42.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.418+0000 7f3cdaa00640 1 -- 172.21.15.39:0/4245244163 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v47) ==== 170+0+0 (secure 0 0 0) 0x7f3cd0061e00 con 0x7f3ce415ca50 2024-09-21T07:34:42.420 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' created 2024-09-21T07:34:42.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.420+0000 7f3ce3e00640 1 -- 172.21.15.39:0/4245244163 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3cc4078bc0 msgr2=0x7f3cc407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:42.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.420+0000 7f3ce3e00640 1 --2- 172.21.15.39:0/4245244163 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3cc4078bc0 0x7f3cc407b0d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4001120 tx=0x7f3cd4009040 comp rx=0 tx=0).stop 2024-09-21T07:34:42.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.420+0000 7f3ce3e00640 1 -- 172.21.15.39:0/4245244163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ce415ca50 msgr2=0x7f3ce4143a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:42.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.420+0000 7f3ce3e00640 1 --2- 172.21.15.39:0/4245244163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ce415ca50 0x7f3ce4143a80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3cd000c4d0 tx=0x7f3cd000c990 comp rx=0 tx=0).stop 2024-09-21T07:34:42.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.421+0000 7f3ce3e00640 1 -- 172.21.15.39:0/4245244163 shutdown_connections 2024-09-21T07:34:42.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.421+0000 7f3ce3e00640 1 --2- 172.21.15.39:0/4245244163 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3cc4078bc0 0x7f3cc407b0d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:42.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.421+0000 7f3ce3e00640 1 --2- 172.21.15.39:0/4245244163 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ce4155d30 0x7f3ce4156140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:42.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.421+0000 7f3ce3e00640 1 --2- 172.21.15.39:0/4245244163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ce415ca50 0x7f3ce4143a80 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:42.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.421+0000 7f3ce3e00640 1 --2- 172.21.15.39:0/4245244163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ce40cf540 0x7f3ce41f5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:42.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.421+0000 7f3ce3e00640 1 -- 172.21.15.39:0/4245244163 >> 172.21.15.39:0/4245244163 conn(0x7f3ce406d410 msgr2=0x7f3ce415b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:42.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.421+0000 7f3ce3e00640 1 -- 172.21.15.39:0/4245244163 shutdown_connections 2024-09-21T07:34:42.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.421+0000 7f3ce3e00640 1 -- 172.21.15.39:0/4245244163 wait complete. 2024-09-21T07:34:42.467 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T07:34:42.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.698+0000 7fa77e000640 1 -- 172.21.15.39:0/2703624726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7780cc1e0 msgr2=0x7fa7780af3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:42.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.698+0000 7fa77e000640 1 --2- 172.21.15.39:0/2703624726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7780cc1e0 0x7fa7780af3c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa768009830 tx=0x7fa76802a510 comp rx=0 tx=0).stop 2024-09-21T07:34:42.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.698+0000 7fa77e000640 1 -- 172.21.15.39:0/2703624726 shutdown_connections 2024-09-21T07:34:42.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.698+0000 7fa77e000640 1 --2- 172.21.15.39:0/2703624726 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa77815d8a0 0x7fa7780b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:42.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.698+0000 7fa77e000640 1 --2- 172.21.15.39:0/2703624726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7780cc1e0 0x7fa7780af3c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:42.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.698+0000 7fa77e000640 1 -- 172.21.15.39:0/2703624726 >> 172.21.15.39:0/2703624726 conn(0x7fa77806d7d0 msgr2=0x7fa7780b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:42.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.698+0000 7fa77e000640 1 -- 172.21.15.39:0/2703624726 shutdown_connections 2024-09-21T07:34:42.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.698+0000 7fa77e000640 1 -- 172.21.15.39:0/2703624726 wait complete. 2024-09-21T07:34:42.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.700+0000 7fa77e000640 1 Processor -- start 2024-09-21T07:34:42.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.700+0000 7fa77e000640 1 -- start start 2024-09-21T07:34:42.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.702+0000 7fa77e000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7780cc1e0 0x7fa77809d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:42.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.702+0000 7fa77e000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7780ccb50 0x7fa7781f9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:42.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.702+0000 7fa77e000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa77815d8a0 0x7fa77814d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:42.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.702+0000 7fa77e000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa778153520 con 0x7fa7780cc1e0 2024-09-21T07:34:42.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.702+0000 7fa77e000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa7781533a0 con 0x7fa77815d8a0 2024-09-21T07:34:42.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.702+0000 7fa77e000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa7781536a0 con 0x7fa7780ccb50 2024-09-21T07:34:42.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.702+0000 7fa777400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7780cc1e0 0x7fa77809d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:42.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.702+0000 7fa777400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7780cc1e0 0x7fa77809d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57568/0 (socket says 172.21.15.39:57568) 2024-09-21T07:34:42.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.702+0000 7fa777400640 1 -- 172.21.15.39:0/1920819280 learned_addr learned my addr 172.21.15.39:0/1920819280 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:42.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.702+0000 7fa777400640 1 -- 172.21.15.39:0/1920819280 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7780ccb50 msgr2=0x7fa7781f9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:42.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.702+0000 7fa777400640 1 --2- 172.21.15.39:0/1920819280 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7780ccb50 0x7fa7781f9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:42.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.702+0000 7fa777400640 1 -- 172.21.15.39:0/1920819280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa77815d8a0 msgr2=0x7fa77814d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:42.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.702+0000 7fa777e00640 1 --2- 172.21.15.39:0/1920819280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa77815d8a0 0x7fa77814d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:42.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.702+0000 7fa777400640 1 --2- 172.21.15.39:0/1920819280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa77815d8a0 0x7fa77814d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:42.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.702+0000 7fa777400640 1 -- 172.21.15.39:0/1920819280 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7781fa0e0 con 0x7fa7780cc1e0 2024-09-21T07:34:42.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.702+0000 7fa777e00640 1 --2- 172.21.15.39:0/1920819280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa77815d8a0 0x7fa77814d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:34:42.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.703+0000 7fa777400640 1 --2- 172.21.15.39:0/1920819280 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7780cc1e0 0x7fa77809d560 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa768009950 tx=0x7fa768033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:42.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.703+0000 7fa76f400640 1 -- 172.21.15.39:0/1920819280 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa768005c90 con 0x7fa7780cc1e0 2024-09-21T07:34:42.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.703+0000 7fa76f400640 1 -- 172.21.15.39:0/1920819280 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa768031e50 con 0x7fa7780cc1e0 2024-09-21T07:34:42.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.703+0000 7fa77e000640 1 -- 172.21.15.39:0/1920819280 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa77815a780 con 0x7fa7780cc1e0 2024-09-21T07:34:42.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.703+0000 7fa76f400640 1 -- 172.21.15.39:0/1920819280 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa768005260 con 0x7fa7780cc1e0 2024-09-21T07:34:42.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.703+0000 7fa77e000640 1 -- 172.21.15.39:0/1920819280 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa77809b2f0 con 0x7fa7780cc1e0 2024-09-21T07:34:42.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.704+0000 7fa77e000640 1 -- 172.21.15.39:0/1920819280 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa778153b50 con 0x7fa7780cc1e0 2024-09-21T07:34:42.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.706+0000 7fa76f400640 1 -- 172.21.15.39:0/1920819280 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa768005400 con 0x7fa7780cc1e0 2024-09-21T07:34:42.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.707+0000 7fa76f400640 1 --2- 172.21.15.39:0/1920819280 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa754078b90 0x7fa75407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:42.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.707+0000 7fa76f400640 1 -- 172.21.15.39:0/1920819280 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5962+0+0 (secure 0 0 0) 0x7fa7680bc340 con 0x7fa7780cc1e0 2024-09-21T07:34:42.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.707+0000 7fa76fe00640 1 --2- 172.21.15.39:0/1920819280 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa754078b90 0x7fa75407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:42.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.708+0000 7fa76fe00640 1 --2- 172.21.15.39:0/1920819280 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa754078b90 0x7fa75407b0a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa760005d00 tx=0x7fa760005c50 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:42.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.711+0000 7fa76f400640 1 -- 172.21.15.39:0/1920819280 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa76803f030 con 0x7fa7780cc1e0 2024-09-21T07:34:42.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:42.954+0000 7fa77e000640 1 -- 172.21.15.39:0/1920819280 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa7780d09b0 con 0x7fa7780cc1e0 2024-09-21T07:34:43.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.437+0000 7fa76f400640 1 -- 172.21.15.39:0/1920819280 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v22) ==== 172+0+0 (secure 0 0 0) 0x7fa768084660 con 0x7fa7780cc1e0 2024-09-21T07:34:43.439 INFO:teuthology.orchestra.run.smithi039.stderr:new fs with metadata pool 7 and data pool 8 2024-09-21T07:34:43.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.442+0000 7fa77e000640 1 -- 172.21.15.39:0/1920819280 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa754078b90 msgr2=0x7fa75407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:43.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.442+0000 7fa77e000640 1 --2- 172.21.15.39:0/1920819280 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa754078b90 0x7fa75407b0a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa760005d00 tx=0x7fa760005c50 comp rx=0 tx=0).stop 2024-09-21T07:34:43.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.442+0000 7fa77e000640 1 -- 172.21.15.39:0/1920819280 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7780cc1e0 msgr2=0x7fa77809d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:43.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.442+0000 7fa77e000640 1 --2- 172.21.15.39:0/1920819280 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7780cc1e0 0x7fa77809d560 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa768009950 tx=0x7fa768033c20 comp rx=0 tx=0).stop 2024-09-21T07:34:43.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.444+0000 7fa77e000640 1 -- 172.21.15.39:0/1920819280 shutdown_connections 2024-09-21T07:34:43.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.444+0000 7fa77e000640 1 --2- 172.21.15.39:0/1920819280 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa754078b90 0x7fa75407b0a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:43.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.444+0000 7fa77e000640 1 --2- 172.21.15.39:0/1920819280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa77815d8a0 0x7fa77814d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:43.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.444+0000 7fa77e000640 1 --2- 172.21.15.39:0/1920819280 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7780ccb50 0x7fa7781f9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:43.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.444+0000 7fa77e000640 1 --2- 172.21.15.39:0/1920819280 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7780cc1e0 0x7fa77809d560 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:43.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.444+0000 7fa77e000640 1 -- 172.21.15.39:0/1920819280 >> 172.21.15.39:0/1920819280 conn(0x7fa77806d7d0 msgr2=0x7fa7780b1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:43.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.445+0000 7fa77e000640 1 -- 172.21.15.39:0/1920819280 shutdown_connections 2024-09-21T07:34:43.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.446+0000 7fa77e000640 1 -- 172.21.15.39:0/1920819280 wait complete. 2024-09-21T07:34:43.500 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:34:43.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.723+0000 7f6143200640 1 -- 172.21.15.39:0/102856789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6144073040 msgr2=0x7f614415f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:43.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.723+0000 7f6143200640 1 --2- 172.21.15.39:0/102856789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6144073040 0x7f614415f490 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f613c005fd0 tx=0x7f613c02aa90 comp rx=0 tx=0).stop 2024-09-21T07:34:43.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.724+0000 7f6143200640 1 -- 172.21.15.39:0/102856789 shutdown_connections 2024-09-21T07:34:43.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.724+0000 7f6143200640 1 --2- 172.21.15.39:0/102856789 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6144072c30 0x7f614415ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:43.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.724+0000 7f6143200640 1 --2- 172.21.15.39:0/102856789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6144073040 0x7f614415f490 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:43.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.724+0000 7f6143200640 1 -- 172.21.15.39:0/102856789 >> 172.21.15.39:0/102856789 conn(0x7f614406d410 msgr2=0x7f61440b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:43.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.724+0000 7f6143200640 1 -- 172.21.15.39:0/102856789 shutdown_connections 2024-09-21T07:34:43.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.724+0000 7f6143200640 1 -- 172.21.15.39:0/102856789 wait complete. 2024-09-21T07:34:43.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.726+0000 7f6143200640 1 Processor -- start 2024-09-21T07:34:43.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.726+0000 7f6143200640 1 -- start start 2024-09-21T07:34:43.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.728+0000 7f6143200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6144072c30 0x7f6144156000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:43.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.728+0000 7f6143200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6144073040 0x7f61441486f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:43.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.728+0000 7f6143200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61441f3640 0x7f614414eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:43.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.728+0000 7f6143200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f61440b3fd0 con 0x7f61441f3640 2024-09-21T07:34:43.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.728+0000 7f6143200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f61440b3e50 con 0x7f6144073040 2024-09-21T07:34:43.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.728+0000 7f6143200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f61440b4150 con 0x7f6144072c30 2024-09-21T07:34:43.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.728+0000 7f6142800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61441f3640 0x7f614414eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:43.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.728+0000 7f6142800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61441f3640 0x7f614414eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57592/0 (socket says 172.21.15.39:57592) 2024-09-21T07:34:43.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.728+0000 7f6142800640 1 -- 172.21.15.39:0/1987443682 learned_addr learned my addr 172.21.15.39:0/1987443682 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:43.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.728+0000 7f6141400640 1 --2- 172.21.15.39:0/1987443682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6144073040 0x7f61441486f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:43.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.728+0000 7f6141e00640 1 --2- 172.21.15.39:0/1987443682 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6144072c30 0x7f6144156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:43.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.728+0000 7f6142800640 1 -- 172.21.15.39:0/1987443682 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6144072c30 msgr2=0x7f6144156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:43.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.728+0000 7f6142800640 1 --2- 172.21.15.39:0/1987443682 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6144072c30 0x7f6144156000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:43.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.728+0000 7f6142800640 1 -- 172.21.15.39:0/1987443682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6144073040 msgr2=0x7f61441486f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:43.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.728+0000 7f6142800640 1 --2- 172.21.15.39:0/1987443682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6144073040 0x7f61441486f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:43.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.728+0000 7f6142800640 1 -- 172.21.15.39:0/1987443682 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6144156540 con 0x7f61441f3640 2024-09-21T07:34:43.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.728+0000 7f6141400640 1 --2- 172.21.15.39:0/1987443682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6144073040 0x7f61441486f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:34:43.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.729+0000 7f6142800640 1 --2- 172.21.15.39:0/1987443682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61441f3640 0x7f614414eb30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f613c009d90 tx=0x7f613c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:43.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.729+0000 7f613a000640 1 -- 172.21.15.39:0/1987443682 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613c044070 con 0x7f61441f3640 2024-09-21T07:34:43.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.729+0000 7f613a000640 1 -- 172.21.15.39:0/1987443682 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f613c004040 con 0x7f61441f3640 2024-09-21T07:34:43.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.729+0000 7f613a000640 1 -- 172.21.15.39:0/1987443682 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613c004430 con 0x7f61441f3640 2024-09-21T07:34:43.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.729+0000 7f6143200640 1 -- 172.21.15.39:0/1987443682 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61441f40e0 con 0x7f61441f3640 2024-09-21T07:34:43.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.729+0000 7f6143200640 1 -- 172.21.15.39:0/1987443682 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f614414e330 con 0x7f61441f3640 2024-09-21T07:34:43.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.730+0000 7f6143200640 1 -- 172.21.15.39:0/1987443682 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f614415fbd0 con 0x7f61441f3640 2024-09-21T07:34:43.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.732+0000 7f613a000640 1 -- 172.21.15.39:0/1987443682 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f613c047070 con 0x7f61441f3640 2024-09-21T07:34:43.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.733+0000 7f613a000640 1 --2- 172.21.15.39:0/1987443682 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6124078bc0 0x7f612407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:43.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.733+0000 7f613a000640 1 -- 172.21.15.39:0/1987443682 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6062+0+0 (secure 0 0 0) 0x7f613c0bbc40 con 0x7f61441f3640 2024-09-21T07:34:43.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.733+0000 7f6141e00640 1 --2- 172.21.15.39:0/1987443682 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6124078bc0 0x7f612407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:43.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.734+0000 7f6141e00640 1 --2- 172.21.15.39:0/1987443682 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6124078bc0 0x7f612407b0d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6134001640 tx=0x7f6134009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:43.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.737+0000 7f613a000640 1 -- 172.21.15.39:0/1987443682 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f613c032030 con 0x7f61441f3640 2024-09-21T07:34:43.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.934+0000 7f6143200640 1 -- 172.21.15.39:0/1987443682 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f61440750a0 con 0x7f61441f3640 2024-09-21T07:34:43.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.935+0000 7f613a000640 1 -- 172.21.15.39:0/1987443682 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+18552 (secure 0 0 0) 0x7f613c083fb0 con 0x7f61441f3640 2024-09-21T07:34:43.938 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:34:43.938 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":48,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:34:42.955798+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:34:40.886322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-21T07:34:41.906495+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:34:43.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.938+0000 7f6143200640 1 -- 172.21.15.39:0/1987443682 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6124078bc0 msgr2=0x7f612407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:43.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.938+0000 7f6143200640 1 --2- 172.21.15.39:0/1987443682 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6124078bc0 0x7f612407b0d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6134001640 tx=0x7f6134009040 comp rx=0 tx=0).stop 2024-09-21T07:34:43.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.938+0000 7f6143200640 1 -- 172.21.15.39:0/1987443682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61441f3640 msgr2=0x7f614414eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:43.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.938+0000 7f6143200640 1 --2- 172.21.15.39:0/1987443682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61441f3640 0x7f614414eb30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f613c009d90 tx=0x7f613c003ce0 comp rx=0 tx=0).stop 2024-09-21T07:34:43.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.938+0000 7f6143200640 1 -- 172.21.15.39:0/1987443682 shutdown_connections 2024-09-21T07:34:43.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.938+0000 7f6143200640 1 --2- 172.21.15.39:0/1987443682 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6124078bc0 0x7f612407b0d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:43.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.938+0000 7f6143200640 1 --2- 172.21.15.39:0/1987443682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61441f3640 0x7f614414eb30 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:43.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.938+0000 7f6143200640 1 --2- 172.21.15.39:0/1987443682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6144073040 0x7f61441486f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:43.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.938+0000 7f6143200640 1 --2- 172.21.15.39:0/1987443682 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6144072c30 0x7f6144156000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:43.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.938+0000 7f6143200640 1 -- 172.21.15.39:0/1987443682 >> 172.21.15.39:0/1987443682 conn(0x7f614406d410 msgr2=0x7f61440d08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:43.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.938+0000 7f6143200640 1 -- 172.21.15.39:0/1987443682 shutdown_connections 2024-09-21T07:34:43.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:43.938+0000 7f6143200640 1 -- 172.21.15.39:0/1987443682 wait complete. 2024-09-21T07:34:43.962 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:34:44.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.187+0000 7fdccec00640 1 -- 172.21.15.39:0/2127787652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcac003a50 msgr2=0x7fdcac003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:44.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.187+0000 7fdccec00640 1 --2- 172.21.15.39:0/2127787652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcac003a50 0x7fdcac003ec0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009b20 tx=0x7fdcac005830 comp rx=0 tx=0).stop 2024-09-21T07:34:44.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.187+0000 7fdccec00640 1 -- 172.21.15.39:0/2127787652 shutdown_connections 2024-09-21T07:34:44.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.187+0000 7fdccec00640 1 --2- 172.21.15.39:0/2127787652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcac003a50 0x7fdcac003ec0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:44.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.187+0000 7fdccec00640 1 -- 172.21.15.39:0/2127787652 >> 172.21.15.39:0/2127787652 conn(0x7fdcc80b1cb0 msgr2=0x7fdcc80b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:44.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.187+0000 7fdccec00640 1 -- 172.21.15.39:0/2127787652 shutdown_connections 2024-09-21T07:34:44.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.187+0000 7fdccec00640 1 -- 172.21.15.39:0/2127787652 wait complete. 2024-09-21T07:34:44.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.190+0000 7fdccec00640 1 Processor -- start 2024-09-21T07:34:44.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.190+0000 7fdccec00640 1 -- start start 2024-09-21T07:34:44.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.191+0000 7fdccec00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcac003a50 0x7fdcc809d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:44.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.191+0000 7fdccec00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdcc809aa30 0x7fdcc80ad1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:44.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.191+0000 7fdccec00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcc80ad730 0x7fdcc814e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:44.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.191+0000 7fdccec00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc80701b0 con 0x7fdcc80ad730 2024-09-21T07:34:44.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.191+0000 7fdccec00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc80cf3d0 con 0x7fdcac003a50 2024-09-21T07:34:44.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.191+0000 7fdccec00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fdcc8070330 con 0x7fdcc809aa30 2024-09-21T07:34:44.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.192+0000 7fdcccc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcc80ad730 0x7fdcc814e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:44.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.192+0000 7fdcccc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcc80ad730 0x7fdcc814e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57610/0 (socket says 172.21.15.39:57610) 2024-09-21T07:34:44.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.192+0000 7fdcccc00640 1 -- 172.21.15.39:0/2234273552 learned_addr learned my addr 172.21.15.39:0/2234273552 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:44.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.192+0000 7fdcc7e00640 1 --2- 172.21.15.39:0/2234273552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcac003a50 0x7fdcc809d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:44.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.192+0000 7fdcc7400640 1 --2- 172.21.15.39:0/2234273552 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdcc809aa30 0x7fdcc80ad1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:44.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.192+0000 7fdcccc00640 1 -- 172.21.15.39:0/2234273552 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdcc809aa30 msgr2=0x7fdcc80ad1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:44.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.192+0000 7fdcccc00640 1 --2- 172.21.15.39:0/2234273552 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdcc809aa30 0x7fdcc80ad1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:44.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.192+0000 7fdcccc00640 1 -- 172.21.15.39:0/2234273552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcac003a50 msgr2=0x7fdcc809d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:44.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.192+0000 7fdcccc00640 1 --2- 172.21.15.39:0/2234273552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcac003a50 0x7fdcc809d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:44.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.192+0000 7fdcccc00640 1 -- 172.21.15.39:0/2234273552 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc809c790 con 0x7fdcc80ad730 2024-09-21T07:34:44.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.192+0000 7fdcccc00640 1 --2- 172.21.15.39:0/2234273552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcc80ad730 0x7fdcc814e4d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8072be0 tx=0x7fdcc00019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:44.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.193+0000 7fdcbfe00640 1 -- 172.21.15.39:0/2234273552 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0017070 con 0x7fdcc80ad730 2024-09-21T07:34:44.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.193+0000 7fdcbfe00640 1 -- 172.21.15.39:0/2234273552 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdcc00108c0 con 0x7fdcc80ad730 2024-09-21T07:34:44.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.193+0000 7fdcbfe00640 1 -- 172.21.15.39:0/2234273552 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0010cf0 con 0x7fdcc80ad730 2024-09-21T07:34:44.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.193+0000 7fdccec00640 1 -- 172.21.15.39:0/2234273552 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc809ca70 con 0x7fdcc80ad730 2024-09-21T07:34:44.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.193+0000 7fdccec00640 1 -- 172.21.15.39:0/2234273552 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc809d030 con 0x7fdcc80ad730 2024-09-21T07:34:44.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.194+0000 7fdccec00640 1 -- 172.21.15.39:0/2234273552 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcc809c4f0 con 0x7fdcc80ad730 2024-09-21T07:34:44.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.195+0000 7fdcbfe00640 1 -- 172.21.15.39:0/2234273552 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdcc001a070 con 0x7fdcc80ad730 2024-09-21T07:34:44.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.196+0000 7fdcbfe00640 1 --2- 172.21.15.39:0/2234273552 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdca8078b90 0x7fdca807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:44.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.196+0000 7fdcbfe00640 1 -- 172.21.15.39:0/2234273552 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6062+0+0 (secure 0 0 0) 0x7fdcc009ac60 con 0x7fdcc80ad730 2024-09-21T07:34:44.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.197+0000 7fdcc7e00640 1 --2- 172.21.15.39:0/2234273552 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdca8078b90 0x7fdca807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:44.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.197+0000 7fdcc7e00640 1 --2- 172.21.15.39:0/2234273552 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdca8078b90 0x7fdca807b0a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8035000 tx=0x7fdcb80046c0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:44.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.201+0000 7fdcbfe00640 1 -- 172.21.15.39:0/2234273552 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdcc809c4f0 con 0x7fdcc80ad730 2024-09-21T07:34:44.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.398+0000 7fdccec00640 1 -- 172.21.15.39:0/2234273552 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdcc80cc740 con 0x7fdcc80ad730 2024-09-21T07:34:44.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.400+0000 7fdcbfe00640 1 -- 172.21.15.39:0/2234273552 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+18552 (secure 0 0 0) 0x7fdcc0062850 con 0x7fdcc80ad730 2024-09-21T07:34:44.402 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:34:44.403 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":48,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:34:42.955798+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:34:40.886322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-21T07:34:41.906495+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:34:44.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.402+0000 7fdccec00640 1 -- 172.21.15.39:0/2234273552 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdca8078b90 msgr2=0x7fdca807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:44.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.402+0000 7fdccec00640 1 --2- 172.21.15.39:0/2234273552 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdca8078b90 0x7fdca807b0a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8035000 tx=0x7fdcb80046c0 comp rx=0 tx=0).stop 2024-09-21T07:34:44.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.402+0000 7fdccec00640 1 -- 172.21.15.39:0/2234273552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcc80ad730 msgr2=0x7fdcc814e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:44.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.402+0000 7fdccec00640 1 --2- 172.21.15.39:0/2234273552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcc80ad730 0x7fdcc814e4d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8072be0 tx=0x7fdcc00019b0 comp rx=0 tx=0).stop 2024-09-21T07:34:44.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.402+0000 7fdccec00640 1 -- 172.21.15.39:0/2234273552 shutdown_connections 2024-09-21T07:34:44.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.402+0000 7fdccec00640 1 --2- 172.21.15.39:0/2234273552 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdca8078b90 0x7fdca807b0a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:44.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.402+0000 7fdccec00640 1 --2- 172.21.15.39:0/2234273552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcc80ad730 0x7fdcc814e4d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:44.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.402+0000 7fdccec00640 1 --2- 172.21.15.39:0/2234273552 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdcc809aa30 0x7fdcc80ad1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:44.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.402+0000 7fdccec00640 1 --2- 172.21.15.39:0/2234273552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcac003a50 0x7fdcc809d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:44.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.403+0000 7fdccec00640 1 -- 172.21.15.39:0/2234273552 >> 172.21.15.39:0/2234273552 conn(0x7fdcc80b1cb0 msgr2=0x7fdcc8073860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:44.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.403+0000 7fdccec00640 1 -- 172.21.15.39:0/2234273552 shutdown_connections 2024-09-21T07:34:44.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.403+0000 7fdccec00640 1 -- 172.21.15.39:0/2234273552 wait complete. 2024-09-21T07:34:44.427 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T07:34:44.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.658+0000 7f2347200640 1 -- 172.21.15.39:0/2770624491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f234815ca50 msgr2=0x7f23480cc830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:44.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.658+0000 7f2347200640 1 --2- 172.21.15.39:0/2770624491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f234815ca50 0x7f23480cc830 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f23480ce710 tx=0x7f233802a4d0 comp rx=0 tx=0).stop 2024-09-21T07:34:44.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.658+0000 7f2347200640 1 -- 172.21.15.39:0/2770624491 shutdown_connections 2024-09-21T07:34:44.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.658+0000 7f2347200640 1 --2- 172.21.15.39:0/2770624491 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f23480cf950 0x7f234806f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:44.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.658+0000 7f2347200640 1 --2- 172.21.15.39:0/2770624491 >> v2:172.21.15.148:3300/0 conn(0x7f23480cf540 0x7f234806ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:44.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.658+0000 7f2347200640 1 --2- 172.21.15.39:0/2770624491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f234815ca50 0x7f23480cc830 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:44.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.658+0000 7f2347200640 1 -- 172.21.15.39:0/2770624491 >> 172.21.15.39:0/2770624491 conn(0x7f234806d410 msgr2=0x7f23480b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:44.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.658+0000 7f2347200640 1 -- 172.21.15.39:0/2770624491 shutdown_connections 2024-09-21T07:34:44.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.658+0000 7f2347200640 1 -- 172.21.15.39:0/2770624491 wait complete. 2024-09-21T07:34:44.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.660+0000 7f2347200640 1 Processor -- start 2024-09-21T07:34:44.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.660+0000 7f2347200640 1 -- start start 2024-09-21T07:34:44.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.662+0000 7f2347200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23480cf540 0x7f2348155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:44.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.662+0000 7f2347200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f23480cf950 0x7f23481e6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:44.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.662+0000 7f2347200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f234815ca50 0x7f23481f6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:44.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.662+0000 7f2347200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f234815dbe0 con 0x7f23480cf540 2024-09-21T07:34:44.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.662+0000 7f2347200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f234815da60 con 0x7f234815ca50 2024-09-21T07:34:44.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.662+0000 7f2347200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f234815dd60 con 0x7f23480cf950 2024-09-21T07:34:44.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.662+0000 7f2345e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23480cf540 0x7f2348155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:44.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.662+0000 7f2345e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23480cf540 0x7f2348155df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57628/0 (socket says 172.21.15.39:57628) 2024-09-21T07:34:44.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.662+0000 7f2345e00640 1 -- 172.21.15.39:0/1438541336 learned_addr learned my addr 172.21.15.39:0/1438541336 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:44.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.662+0000 7f2346800640 1 --2- 172.21.15.39:0/1438541336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f234815ca50 0x7f23481f6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:44.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.662+0000 7f233fe00640 1 --2- 172.21.15.39:0/1438541336 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f23480cf950 0x7f23481e6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:44.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.663+0000 7f2345e00640 1 -- 172.21.15.39:0/1438541336 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f23480cf950 msgr2=0x7f23481e6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:44.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.663+0000 7f2345e00640 1 --2- 172.21.15.39:0/1438541336 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f23480cf950 0x7f23481e6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:44.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.663+0000 7f2345e00640 1 -- 172.21.15.39:0/1438541336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f234815ca50 msgr2=0x7f23481f6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:44.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.663+0000 7f2345e00640 1 --2- 172.21.15.39:0/1438541336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f234815ca50 0x7f23481f6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:44.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.663+0000 7f2345e00640 1 -- 172.21.15.39:0/1438541336 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2348156330 con 0x7f23480cf540 2024-09-21T07:34:44.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.663+0000 7f2346800640 1 --2- 172.21.15.39:0/1438541336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f234815ca50 0x7f23481f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:34:44.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.663+0000 7f2345e00640 1 --2- 172.21.15.39:0/1438541336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23480cf540 0x7f2348155df0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2338034040 tx=0x7f23380039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:44.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.663+0000 7f233e000640 1 -- 172.21.15.39:0/1438541336 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233803e030 con 0x7f23480cf540 2024-09-21T07:34:44.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.663+0000 7f233e000640 1 -- 172.21.15.39:0/1438541336 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23380047a0 con 0x7f23480cf540 2024-09-21T07:34:44.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.663+0000 7f2347200640 1 -- 172.21.15.39:0/1438541336 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2348154ad0 con 0x7f23480cf540 2024-09-21T07:34:44.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.663+0000 7f233e000640 1 -- 172.21.15.39:0/1438541336 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2338035040 con 0x7f23480cf540 2024-09-21T07:34:44.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.663+0000 7f2347200640 1 -- 172.21.15.39:0/1438541336 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2348155090 con 0x7f23480cf540 2024-09-21T07:34:44.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.664+0000 7f2347200640 1 -- 172.21.15.39:0/1438541336 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f234815e210 con 0x7f23480cf540 2024-09-21T07:34:44.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.666+0000 7f233e000640 1 -- 172.21.15.39:0/1438541336 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2338041030 con 0x7f23480cf540 2024-09-21T07:34:44.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.667+0000 7f233e000640 1 --2- 172.21.15.39:0/1438541336 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2324078b90 0x7f232407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:44.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.667+0000 7f233e000640 1 -- 172.21.15.39:0/1438541336 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6062+0+0 (secure 0 0 0) 0x7f23380bb860 con 0x7f23480cf540 2024-09-21T07:34:44.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.667+0000 7f233fe00640 1 --2- 172.21.15.39:0/1438541336 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2324078b90 0x7f232407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:44.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.668+0000 7f233fe00640 1 --2- 172.21.15.39:0/1438541336 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2324078b90 0x7f232407b0a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2330005fd0 tx=0x7f233000c460 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:44.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.671+0000 7f233e000640 1 -- 172.21.15.39:0/1438541336 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2338083aa0 con 0x7f23480cf540 2024-09-21T07:34:44.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:44.911+0000 7f2347200640 1 -- 172.21.15.39:0/1438541336 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f234806fb50 con 0x7f23480cf540 2024-09-21T07:34:45.399 INFO:tasks.ceph.mds.a.smithi039.stderr:2024-09-21T07:34:45.397+0000 7efea5c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:34:45.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.471+0000 7f233e000640 1 -- 172.21.15.39:0/1438541336 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7f23380833a0 con 0x7f23480cf540 2024-09-21T07:34:45.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.474+0000 7f2347200640 1 -- 172.21.15.39:0/1438541336 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2324078b90 msgr2=0x7f232407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:45.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.474+0000 7f2347200640 1 --2- 172.21.15.39:0/1438541336 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2324078b90 0x7f232407b0a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2330005fd0 tx=0x7f233000c460 comp rx=0 tx=0).stop 2024-09-21T07:34:45.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.474+0000 7f2347200640 1 -- 172.21.15.39:0/1438541336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23480cf540 msgr2=0x7f2348155df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:45.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.474+0000 7f2347200640 1 --2- 172.21.15.39:0/1438541336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23480cf540 0x7f2348155df0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2338034040 tx=0x7f23380039e0 comp rx=0 tx=0).stop 2024-09-21T07:34:45.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.474+0000 7f2347200640 1 -- 172.21.15.39:0/1438541336 shutdown_connections 2024-09-21T07:34:45.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.474+0000 7f2347200640 1 --2- 172.21.15.39:0/1438541336 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2324078b90 0x7f232407b0a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:45.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.474+0000 7f2347200640 1 --2- 172.21.15.39:0/1438541336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f234815ca50 0x7f23481f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:45.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.474+0000 7f2347200640 1 --2- 172.21.15.39:0/1438541336 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f23480cf950 0x7f23481e6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:45.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.474+0000 7f2347200640 1 --2- 172.21.15.39:0/1438541336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23480cf540 0x7f2348155df0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:45.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.474+0000 7f2347200640 1 -- 172.21.15.39:0/1438541336 >> 172.21.15.39:0/1438541336 conn(0x7f234806d410 msgr2=0x7f23480b1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:45.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.474+0000 7f2347200640 1 -- 172.21.15.39:0/1438541336 shutdown_connections 2024-09-21T07:34:45.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.475+0000 7f2347200640 1 -- 172.21.15.39:0/1438541336 wait complete. 2024-09-21T07:34:45.501 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T07:34:45.502 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T07:34:45.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.732+0000 7fdeb4c00640 1 -- 172.21.15.39:0/2313557145 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde94004d40 msgr2=0x7fde94005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:45.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.732+0000 7fdeb4c00640 1 --2- 172.21.15.39:0/2313557145 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde94004d40 0x7fde94005150 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdeb015a5b0 tx=0x7fde94005830 comp rx=0 tx=0).stop 2024-09-21T07:34:45.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.732+0000 7fdeafe00640 1 -- 172.21.15.39:0/2313557145 reap_dead start 2024-09-21T07:34:45.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.732+0000 7fdeb4c00640 1 -- 172.21.15.39:0/2313557145 shutdown_connections 2024-09-21T07:34:45.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.732+0000 7fdeb4c00640 1 -- 172.21.15.39:0/2313557145 >> 172.21.15.39:0/2313557145 conn(0x7fdeb006d410 msgr2=0x7fdeb00b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:45.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.733+0000 7fdeb4c00640 1 -- 172.21.15.39:0/2313557145 shutdown_connections 2024-09-21T07:34:45.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.733+0000 7fdeb4c00640 1 -- 172.21.15.39:0/2313557145 wait complete. 2024-09-21T07:34:45.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.735+0000 7fdeb4c00640 1 Processor -- start 2024-09-21T07:34:45.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.735+0000 7fdeb4c00640 1 -- start start 2024-09-21T07:34:45.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.736+0000 7fdeb4c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde94004d40 0x7fdeb0140300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:45.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.736+0000 7fdeb4c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdeb013ca10 0x7fdeb0140f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:45.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.737+0000 7fdeb4c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeb0141490 0x7fdeb013e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:45.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.737+0000 7fdeb4c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb015d880 con 0x7fde94004d40 2024-09-21T07:34:45.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.737+0000 7fdeb4c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb015d700 con 0x7fdeb0141490 2024-09-21T07:34:45.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.737+0000 7fdeb4c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fdeb015da00 con 0x7fdeb013ca10 2024-09-21T07:34:45.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.737+0000 7fdeaf400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde94004d40 0x7fdeb0140300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:45.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.737+0000 7fdeaf400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde94004d40 0x7fdeb0140300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57638/0 (socket says 172.21.15.39:57638) 2024-09-21T07:34:45.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.737+0000 7fdeaf400640 1 -- 172.21.15.39:0/3597269086 learned_addr learned my addr 172.21.15.39:0/3597269086 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:45.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.737+0000 7fdeafe00640 1 --2- 172.21.15.39:0/3597269086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeb0141490 0x7fdeb013e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:45.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.737+0000 7fdea7e00640 1 --2- 172.21.15.39:0/3597269086 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdeb013ca10 0x7fdeb0140f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:45.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.737+0000 7fdeaf400640 1 -- 172.21.15.39:0/3597269086 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdeb013ca10 msgr2=0x7fdeb0140f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:45.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.737+0000 7fdeaf400640 1 --2- 172.21.15.39:0/3597269086 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdeb013ca10 0x7fdeb0140f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:45.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.737+0000 7fdeaf400640 1 -- 172.21.15.39:0/3597269086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeb0141490 msgr2=0x7fdeb013e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:45.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.737+0000 7fdeaf400640 1 --2- 172.21.15.39:0/3597269086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeb0141490 0x7fdeb013e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:45.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.737+0000 7fdeaf400640 1 -- 172.21.15.39:0/3597269086 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeb01418a0 con 0x7fde94004d40 2024-09-21T07:34:45.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.738+0000 7fdeaf400640 1 --2- 172.21.15.39:0/3597269086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde94004d40 0x7fdeb0140300 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdea00066b0 tx=0x7fdea00039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:45.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.738+0000 7fdea7400640 1 -- 172.21.15.39:0/3597269086 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea0015070 con 0x7fde94004d40 2024-09-21T07:34:45.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.738+0000 7fdea7400640 1 -- 172.21.15.39:0/3597269086 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdea0011900 con 0x7fde94004d40 2024-09-21T07:34:45.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.738+0000 7fdea7400640 1 -- 172.21.15.39:0/3597269086 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea0011d30 con 0x7fde94004d40 2024-09-21T07:34:45.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.738+0000 7fdeb4c00640 1 -- 172.21.15.39:0/3597269086 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeb0142f60 con 0x7fde94004d40 2024-09-21T07:34:45.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.738+0000 7fdeb4c00640 1 -- 172.21.15.39:0/3597269086 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeb0143520 con 0x7fde94004d40 2024-09-21T07:34:45.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.739+0000 7fdeb4c00640 1 -- 172.21.15.39:0/3597269086 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdeb006ee70 con 0x7fde94004d40 2024-09-21T07:34:45.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.740+0000 7fdea7400640 1 -- 172.21.15.39:0/3597269086 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdea0010420 con 0x7fde94004d40 2024-09-21T07:34:45.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.741+0000 7fdea7400640 1 --2- 172.21.15.39:0/3597269086 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fde8c078b90 0x7fde8c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:45.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.741+0000 7fdea7400640 1 -- 172.21.15.39:0/3597269086 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6062+0+0 (secure 0 0 0) 0x7fdea009bc50 con 0x7fde94004d40 2024-09-21T07:34:45.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.742+0000 7fdea7e00640 1 --2- 172.21.15.39:0/3597269086 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fde8c078b90 0x7fde8c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:45.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.743+0000 7fdea7e00640 1 --2- 172.21.15.39:0/3597269086 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fde8c078b90 0x7fde8c07b0a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdeb015eac0 tx=0x7fde98005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:45.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.746+0000 7fdea7400640 1 -- 172.21.15.39:0/3597269086 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdea00a1020 con 0x7fde94004d40 2024-09-21T07:34:45.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:45.982+0000 7fdeb4c00640 1 -- 172.21.15.39:0/3597269086 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fdeb006fb50 con 0x7fde94004d40 2024-09-21T07:34:46.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.481+0000 7fdea7400640 1 -- 172.21.15.39:0/3597269086 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7fdeb01418a0 con 0x7fde94004d40 2024-09-21T07:34:46.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.483+0000 7fdeb4c00640 1 -- 172.21.15.39:0/3597269086 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fde8c078b90 msgr2=0x7fde8c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:46.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.483+0000 7fdeb4c00640 1 --2- 172.21.15.39:0/3597269086 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fde8c078b90 0x7fde8c07b0a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdeb015eac0 tx=0x7fde98005f90 comp rx=0 tx=0).stop 2024-09-21T07:34:46.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.483+0000 7fdeb4c00640 1 -- 172.21.15.39:0/3597269086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde94004d40 msgr2=0x7fdeb0140300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:46.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.483+0000 7fdeb4c00640 1 --2- 172.21.15.39:0/3597269086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde94004d40 0x7fdeb0140300 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdea00066b0 tx=0x7fdea00039c0 comp rx=0 tx=0).stop 2024-09-21T07:34:46.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.484+0000 7fdeb4c00640 1 -- 172.21.15.39:0/3597269086 shutdown_connections 2024-09-21T07:34:46.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.484+0000 7fdeb4c00640 1 --2- 172.21.15.39:0/3597269086 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fde8c078b90 0x7fde8c07b0a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:46.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.484+0000 7fdeb4c00640 1 --2- 172.21.15.39:0/3597269086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeb0141490 0x7fdeb013e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:46.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.484+0000 7fdeb4c00640 1 --2- 172.21.15.39:0/3597269086 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdeb013ca10 0x7fdeb0140f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:46.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.484+0000 7fdeb4c00640 1 --2- 172.21.15.39:0/3597269086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde94004d40 0x7fdeb0140300 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:46.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.484+0000 7fdeb4c00640 1 -- 172.21.15.39:0/3597269086 >> 172.21.15.39:0/3597269086 conn(0x7fdeb006d410 msgr2=0x7fdeb015df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:46.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.484+0000 7fdeb4c00640 1 -- 172.21.15.39:0/3597269086 shutdown_connections 2024-09-21T07:34:46.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.484+0000 7fdeb4c00640 1 -- 172.21.15.39:0/3597269086 wait complete. 2024-09-21T07:34:46.508 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T07:34:46.508 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:34:46.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.735+0000 7f37ef000640 1 -- 172.21.15.39:0/2714831612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37e8073170 msgr2=0x7f37e815c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:46.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.735+0000 7f37ef000640 1 --2- 172.21.15.39:0/2714831612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37e8073170 0x7f37e815c780 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f37e80b4b40 tx=0x7f37d802a560 comp rx=0 tx=0).stop 2024-09-21T07:34:46.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.735+0000 7f37ef000640 1 -- 172.21.15.39:0/2714831612 shutdown_connections 2024-09-21T07:34:46.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.735+0000 7f37ef000640 1 --2- 172.21.15.39:0/2714831612 >> v2:172.21.15.148:3300/0 conn(0x7f37e8070440 0x7f37e80b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:46.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.735+0000 7f37ef000640 1 --2- 172.21.15.39:0/2714831612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37e8073170 0x7f37e815c780 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:46.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.735+0000 7f37ef000640 1 -- 172.21.15.39:0/2714831612 >> 172.21.15.39:0/2714831612 conn(0x7f37e80b1cb0 msgr2=0x7f37e80b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:46.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.735+0000 7f37ef000640 1 -- 172.21.15.39:0/2714831612 shutdown_connections 2024-09-21T07:34:46.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.735+0000 7f37ef000640 1 -- 172.21.15.39:0/2714831612 wait complete. 2024-09-21T07:34:46.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.738+0000 7f37ef000640 1 Processor -- start 2024-09-21T07:34:46.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.738+0000 7f37ef000640 1 -- start start 2024-09-21T07:34:46.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.739+0000 7f37ef000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f37e8070030 0x7f37e80c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:46.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.739+0000 7f37ef000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37e8070440 0x7f37e815f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:46.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.739+0000 7f37ef000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37e8073170 0x7f37e81f96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:46.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.739+0000 7f37ef000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f37e80b3a40 con 0x7f37e8070440 2024-09-21T07:34:46.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.739+0000 7f37ef000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f37e80b38c0 con 0x7f37e8073170 2024-09-21T07:34:46.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.739+0000 7f37ef000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f37e80b3bc0 con 0x7f37e8070030 2024-09-21T07:34:46.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.739+0000 7f37dfe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37e8070440 0x7f37e815f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:46.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.740+0000 7f37dfe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37e8070440 0x7f37e815f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57646/0 (socket says 172.21.15.39:57646) 2024-09-21T07:34:46.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.740+0000 7f37dfe00640 1 -- 172.21.15.39:0/1864230878 learned_addr learned my addr 172.21.15.39:0/1864230878 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:46.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.740+0000 7f37ed000640 1 --2- 172.21.15.39:0/1864230878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37e8073170 0x7f37e81f96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:46.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.740+0000 7f37e7e00640 1 --2- 172.21.15.39:0/1864230878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f37e8070030 0x7f37e80c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:46.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.740+0000 7f37dfe00640 1 -- 172.21.15.39:0/1864230878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f37e8070030 msgr2=0x7f37e80c96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:46.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.740+0000 7f37dfe00640 1 --2- 172.21.15.39:0/1864230878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f37e8070030 0x7f37e80c96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:46.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.740+0000 7f37dfe00640 1 -- 172.21.15.39:0/1864230878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37e8073170 msgr2=0x7f37e81f96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:46.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.740+0000 7f37dfe00640 1 --2- 172.21.15.39:0/1864230878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37e8073170 0x7f37e81f96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:46.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.740+0000 7f37dfe00640 1 -- 172.21.15.39:0/1864230878 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e81f5eb0 con 0x7f37e8070440 2024-09-21T07:34:46.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.740+0000 7f37ed000640 1 --2- 172.21.15.39:0/1864230878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37e8073170 0x7f37e81f96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:34:46.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.740+0000 7f37e7e00640 1 --2- 172.21.15.39:0/1864230878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f37e8070030 0x7f37e80c96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:34:46.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.740+0000 7f37dfe00640 1 --2- 172.21.15.39:0/1864230878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37e8070440 0x7f37e815f520 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f37d000c390 tx=0x7f37d000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:46.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.740+0000 7f37e4c00640 1 -- 172.21.15.39:0/1864230878 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d0017070 con 0x7f37e8070440 2024-09-21T07:34:46.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.741+0000 7f37e4c00640 1 -- 172.21.15.39:0/1864230878 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f37d00035f0 con 0x7f37e8070440 2024-09-21T07:34:46.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.741+0000 7f37e4c00640 1 -- 172.21.15.39:0/1864230878 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d0003a20 con 0x7f37e8070440 2024-09-21T07:34:46.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.741+0000 7f37ef000640 1 -- 172.21.15.39:0/1864230878 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37e81f6190 con 0x7f37e8070440 2024-09-21T07:34:46.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.741+0000 7f37ef000640 1 -- 172.21.15.39:0/1864230878 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37e81f6750 con 0x7f37e8070440 2024-09-21T07:34:46.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.742+0000 7f37ef000640 1 -- 172.21.15.39:0/1864230878 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37e8051040 con 0x7f37e8070440 2024-09-21T07:34:46.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.743+0000 7f37e4c00640 1 -- 172.21.15.39:0/1864230878 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f37d001a070 con 0x7f37e8070440 2024-09-21T07:34:46.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.744+0000 7f37e4c00640 1 --2- 172.21.15.39:0/1864230878 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f37c4078bc0 0x7f37c407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:46.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.744+0000 7f37e4c00640 1 -- 172.21.15.39:0/1864230878 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6062+0+0 (secure 0 0 0) 0x7f37d009a210 con 0x7f37e8070440 2024-09-21T07:34:46.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.744+0000 7f37e7e00640 1 --2- 172.21.15.39:0/1864230878 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f37c4078bc0 0x7f37c407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:46.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.745+0000 7f37e7e00640 1 --2- 172.21.15.39:0/1864230878 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f37c4078bc0 0x7f37c407b0d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f37d8033000 tx=0x7f37d8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:46.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.749+0000 7f37e4c00640 1 -- 172.21.15.39:0/1864230878 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f37d0062450 con 0x7f37e8070440 2024-09-21T07:34:46.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.981+0000 7f37ef000640 1 -- 172.21.15.39:0/1864230878 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37e80c3860 con 0x7f37e8070440 2024-09-21T07:34:46.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.982+0000 7f37e4c00640 1 -- 172.21.15.39:0/1864230878 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5223 (secure 0 0 0) 0x7f37d0061d50 con 0x7f37e8070440 2024-09-21T07:34:46.983 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:34:46.983 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":26,"btime":"2024-09-21T07:34:46:472997+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4958,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/878081219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":878081219},{"type":"v1","addr":"172.21.15.148:6835","nonce":878081219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3340238606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3340238606},{"type":"v1","addr":"172.21.15.148:6837","nonce":3340238606}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5307,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/929230612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":929230612},{"type":"v1","addr":"172.21.15.39:6834","nonce":929230612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:34:42.955753+0000","modified":"2024-09-21T07:34:45.984044+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/1572835489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":1572835489},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572835489}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5310,"qdb_cluster":[5310]},"id":3}]} 2024-09-21T07:34:46.983 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 26 2024-09-21T07:34:46.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.984+0000 7f37ef000640 1 -- 172.21.15.39:0/1864230878 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f37c4078bc0 msgr2=0x7f37c407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:46.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.984+0000 7f37ef000640 1 --2- 172.21.15.39:0/1864230878 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f37c4078bc0 0x7f37c407b0d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f37d8033000 tx=0x7f37d8004730 comp rx=0 tx=0).stop 2024-09-21T07:34:46.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.984+0000 7f37ef000640 1 -- 172.21.15.39:0/1864230878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37e8070440 msgr2=0x7f37e815f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:46.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.984+0000 7f37ef000640 1 --2- 172.21.15.39:0/1864230878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37e8070440 0x7f37e815f520 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f37d000c390 tx=0x7f37d000c850 comp rx=0 tx=0).stop 2024-09-21T07:34:46.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.984+0000 7f37ef000640 1 -- 172.21.15.39:0/1864230878 shutdown_connections 2024-09-21T07:34:46.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.984+0000 7f37ef000640 1 --2- 172.21.15.39:0/1864230878 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f37c4078bc0 0x7f37c407b0d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:46.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.984+0000 7f37ef000640 1 --2- 172.21.15.39:0/1864230878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37e8073170 0x7f37e81f96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:46.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.984+0000 7f37ef000640 1 --2- 172.21.15.39:0/1864230878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37e8070440 0x7f37e815f520 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:46.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.984+0000 7f37ef000640 1 --2- 172.21.15.39:0/1864230878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f37e8070030 0x7f37e80c96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:46.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.984+0000 7f37ef000640 1 -- 172.21.15.39:0/1864230878 >> 172.21.15.39:0/1864230878 conn(0x7f37e80b1cb0 msgr2=0x7f37e80adb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:46.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.984+0000 7f37ef000640 1 -- 172.21.15.39:0/1864230878 shutdown_connections 2024-09-21T07:34:46.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:46.984+0000 7f37ef000640 1 -- 172.21.15.39:0/1864230878 wait complete. 2024-09-21T07:34:47.010 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:34:47.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.237+0000 7f24ef400640 1 -- 172.21.15.39:0/2309956152 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24d0004d40 msgr2=0x7f24d0005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:47.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.237+0000 7f24ef400640 1 --2- 172.21.15.39:0/2309956152 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24d0004d40 0x7f24d0005150 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f24e0005fd0 tx=0x7f24d0005830 comp rx=0 tx=0).stop 2024-09-21T07:34:47.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.237+0000 7f24ed400640 1 -- 172.21.15.39:0/2309956152 reap_dead start 2024-09-21T07:34:47.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.237+0000 7f24ef400640 1 -- 172.21.15.39:0/2309956152 shutdown_connections 2024-09-21T07:34:47.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.237+0000 7f24ef400640 1 -- 172.21.15.39:0/2309956152 >> 172.21.15.39:0/2309956152 conn(0x7f24e80b1cb0 msgr2=0x7f24e80b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:47.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.237+0000 7f24ef400640 1 -- 172.21.15.39:0/2309956152 shutdown_connections 2024-09-21T07:34:47.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.237+0000 7f24ef400640 1 -- 172.21.15.39:0/2309956152 wait complete. 2024-09-21T07:34:47.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.239+0000 7f24ef400640 1 Processor -- start 2024-09-21T07:34:47.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.240+0000 7f24ef400640 1 -- start start 2024-09-21T07:34:47.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.241+0000 7f24ef400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24d0004d40 0x7f24e809aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:47.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.241+0000 7f24ef400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f24e81e8d30 0x7f24e815e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:47.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.241+0000 7f24ef400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24e81f8430 0x7f24e81f8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:47.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.241+0000 7f24ef400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f24e8073940 con 0x7f24d0004d40 2024-09-21T07:34:47.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.241+0000 7f24ef400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f24e80737c0 con 0x7f24e81f8430 2024-09-21T07:34:47.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.241+0000 7f24ef400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f24e8073ac0 con 0x7f24e81e8d30 2024-09-21T07:34:47.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.241+0000 7f24eca00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24d0004d40 0x7f24e809aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:47.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.241+0000 7f24eca00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24d0004d40 0x7f24e809aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57666/0 (socket says 172.21.15.39:57666) 2024-09-21T07:34:47.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.241+0000 7f24eca00640 1 -- 172.21.15.39:0/379461664 learned_addr learned my addr 172.21.15.39:0/379461664 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:47.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.242+0000 7f24e7e00640 1 --2- 172.21.15.39:0/379461664 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f24e81e8d30 0x7f24e815e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:47.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.242+0000 7f24ed400640 1 --2- 172.21.15.39:0/379461664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24e81f8430 0x7f24e81f8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:47.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.242+0000 7f24eca00640 1 -- 172.21.15.39:0/379461664 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f24e81e8d30 msgr2=0x7f24e815e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:47.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.242+0000 7f24eca00640 1 --2- 172.21.15.39:0/379461664 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f24e81e8d30 0x7f24e815e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:47.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.242+0000 7f24eca00640 1 -- 172.21.15.39:0/379461664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24e81f8430 msgr2=0x7f24e81f8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:47.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.242+0000 7f24eca00640 1 --2- 172.21.15.39:0/379461664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24e81f8430 0x7f24e81f8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:47.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.242+0000 7f24eca00640 1 -- 172.21.15.39:0/379461664 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e815ed40 con 0x7f24d0004d40 2024-09-21T07:34:47.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.242+0000 7f24e7e00640 1 --2- 172.21.15.39:0/379461664 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f24e81e8d30 0x7f24e815e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:34:47.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.242+0000 7f24eca00640 1 --2- 172.21.15.39:0/379461664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24d0004d40 0x7f24e809aa30 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f24dc000fd0 tx=0x7f24dc00eab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:47.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.242+0000 7f24e4c00640 1 -- 172.21.15.39:0/379461664 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24dc015070 con 0x7f24d0004d40 2024-09-21T07:34:47.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.243+0000 7f24e4c00640 1 -- 172.21.15.39:0/379461664 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24dc00ca80 con 0x7f24d0004d40 2024-09-21T07:34:47.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.243+0000 7f24e4c00640 1 -- 172.21.15.39:0/379461664 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24dc011460 con 0x7f24d0004d40 2024-09-21T07:34:47.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.243+0000 7f24ef400640 1 -- 172.21.15.39:0/379461664 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24e81f8d80 con 0x7f24d0004d40 2024-09-21T07:34:47.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.243+0000 7f24ef400640 1 -- 172.21.15.39:0/379461664 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24e81e9140 con 0x7f24d0004d40 2024-09-21T07:34:47.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.244+0000 7f24ef400640 1 -- 172.21.15.39:0/379461664 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24e80cba00 con 0x7f24d0004d40 2024-09-21T07:34:47.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.245+0000 7f24e4c00640 1 -- 172.21.15.39:0/379461664 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f24dc018070 con 0x7f24d0004d40 2024-09-21T07:34:47.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.246+0000 7f24e4c00640 1 --2- 172.21.15.39:0/379461664 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f24c8078b90 0x7f24c807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:47.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.246+0000 7f24e4c00640 1 -- 172.21.15.39:0/379461664 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6062+0+0 (secure 0 0 0) 0x7f24dc09b460 con 0x7f24d0004d40 2024-09-21T07:34:47.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.246+0000 7f24e7e00640 1 --2- 172.21.15.39:0/379461664 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f24c8078b90 0x7f24c807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:47.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.247+0000 7f24e7e00640 1 --2- 172.21.15.39:0/379461664 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f24c8078b90 0x7f24c807b0a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f24e815b410 tx=0x7f24d4005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:47.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.251+0000 7f24e4c00640 1 -- 172.21.15.39:0/379461664 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f24e81f8d80 con 0x7f24d0004d40 2024-09-21T07:34:47.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.453+0000 7f24ef400640 1 -- 172.21.15.39:0/379461664 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f24e80ca4e0 con 0x7f24d0004d40 2024-09-21T07:34:47.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.454+0000 7f24e4c00640 1 -- 172.21.15.39:0/379461664 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) ==== 74+0+18539 (secure 0 0 0) 0x7f24dc063730 con 0x7f24d0004d40 2024-09-21T07:34:47.461 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:34:47.461 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":49,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:34:44.452364+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:34:40.886322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-21T07:34:41.906495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:34:47.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.457+0000 7f24ef400640 1 -- 172.21.15.39:0/379461664 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f24c8078b90 msgr2=0x7f24c807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:47.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.457+0000 7f24ef400640 1 --2- 172.21.15.39:0/379461664 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f24c8078b90 0x7f24c807b0a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f24e815b410 tx=0x7f24d4005f90 comp rx=0 tx=0).stop 2024-09-21T07:34:47.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.457+0000 7f24ef400640 1 -- 172.21.15.39:0/379461664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24d0004d40 msgr2=0x7f24e809aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:47.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.457+0000 7f24ef400640 1 --2- 172.21.15.39:0/379461664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24d0004d40 0x7f24e809aa30 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f24dc000fd0 tx=0x7f24dc00eab0 comp rx=0 tx=0).stop 2024-09-21T07:34:47.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.457+0000 7f24ef400640 1 -- 172.21.15.39:0/379461664 shutdown_connections 2024-09-21T07:34:47.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.457+0000 7f24ef400640 1 --2- 172.21.15.39:0/379461664 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f24c8078b90 0x7f24c807b0a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:47.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.457+0000 7f24ef400640 1 --2- 172.21.15.39:0/379461664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24e81f8430 0x7f24e81f8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:47.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.457+0000 7f24ef400640 1 --2- 172.21.15.39:0/379461664 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f24e81e8d30 0x7f24e815e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:47.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.457+0000 7f24ef400640 1 --2- 172.21.15.39:0/379461664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24d0004d40 0x7f24e809aa30 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:47.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.457+0000 7f24ef400640 1 -- 172.21.15.39:0/379461664 >> 172.21.15.39:0/379461664 conn(0x7f24e80b1cb0 msgr2=0x7f24e80b4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:47.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.457+0000 7f24ef400640 1 -- 172.21.15.39:0/379461664 shutdown_connections 2024-09-21T07:34:47.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.457+0000 7f24ef400640 1 -- 172.21.15.39:0/379461664 wait complete. 2024-09-21T07:34:47.481 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T07:34:47.481 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:34:47.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.710+0000 7eff8aa00640 1 -- 172.21.15.39:0/3636865990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff68004800 msgr2=0x7eff68004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:47.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.710+0000 7eff8aa00640 1 --2- 172.21.15.39:0/3636865990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff68004800 0x7eff68004c10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7eff70001800 tx=0x7eff68005830 comp rx=0 tx=0).stop 2024-09-21T07:34:47.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.710+0000 7eff88a00640 1 -- 172.21.15.39:0/3636865990 reap_dead start 2024-09-21T07:34:47.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.710+0000 7eff8aa00640 1 -- 172.21.15.39:0/3636865990 shutdown_connections 2024-09-21T07:34:47.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.710+0000 7eff8aa00640 1 -- 172.21.15.39:0/3636865990 >> 172.21.15.39:0/3636865990 conn(0x7eff840b1cb0 msgr2=0x7eff840b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:47.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.711+0000 7eff8aa00640 1 -- 172.21.15.39:0/3636865990 shutdown_connections 2024-09-21T07:34:47.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.711+0000 7eff8aa00640 1 -- 172.21.15.39:0/3636865990 wait complete. 2024-09-21T07:34:47.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.713+0000 7eff8aa00640 1 Processor -- start 2024-09-21T07:34:47.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.713+0000 7eff8aa00640 1 -- start start 2024-09-21T07:34:47.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.714+0000 7eff8aa00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff68004800 0x7eff8409d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:47.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.714+0000 7eff8aa00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7eff841562f0 0x7eff8415f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:47.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.714+0000 7eff8aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff84156700 0x7eff8415f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:47.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.714+0000 7eff8aa00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7eff841655c0 con 0x7eff84156700 2024-09-21T07:34:47.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.714+0000 7eff8aa00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7eff84165440 con 0x7eff68004800 2024-09-21T07:34:47.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.714+0000 7eff8aa00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7eff84165740 con 0x7eff841562f0 2024-09-21T07:34:47.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.715+0000 7eff88a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff84156700 0x7eff8415f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:47.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.715+0000 7eff88a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff84156700 0x7eff8415f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57686/0 (socket says 172.21.15.39:57686) 2024-09-21T07:34:47.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.715+0000 7eff88a00640 1 -- 172.21.15.39:0/4052929847 learned_addr learned my addr 172.21.15.39:0/4052929847 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:47.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.715+0000 7eff83400640 1 --2- 172.21.15.39:0/4052929847 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7eff841562f0 0x7eff8415f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:47.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.715+0000 7eff83e00640 1 --2- 172.21.15.39:0/4052929847 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff68004800 0x7eff8409d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:47.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.715+0000 7eff88a00640 1 -- 172.21.15.39:0/4052929847 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7eff841562f0 msgr2=0x7eff8415f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:47.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.715+0000 7eff88a00640 1 --2- 172.21.15.39:0/4052929847 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7eff841562f0 0x7eff8415f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:47.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.715+0000 7eff88a00640 1 -- 172.21.15.39:0/4052929847 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff68004800 msgr2=0x7eff8409d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:47.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.715+0000 7eff88a00640 1 --2- 172.21.15.39:0/4052929847 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff68004800 0x7eff8409d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:47.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.715+0000 7eff88a00640 1 -- 172.21.15.39:0/4052929847 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff841f5380 con 0x7eff84156700 2024-09-21T07:34:47.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.715+0000 7eff83400640 1 --2- 172.21.15.39:0/4052929847 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7eff841562f0 0x7eff8415f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:34:47.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.715+0000 7eff83e00640 1 --2- 172.21.15.39:0/4052929847 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff68004800 0x7eff8409d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:34:47.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.716+0000 7eff88a00640 1 --2- 172.21.15.39:0/4052929847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff84156700 0x7eff8415f620 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7eff840cd410 tx=0x7eff7c010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:47.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.716+0000 7eff7be00640 1 -- 172.21.15.39:0/4052929847 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c003970 con 0x7eff84156700 2024-09-21T07:34:47.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.716+0000 7eff7be00640 1 -- 172.21.15.39:0/4052929847 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff7c015070 con 0x7eff84156700 2024-09-21T07:34:47.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.716+0000 7eff8aa00640 1 -- 172.21.15.39:0/4052929847 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff8409dcf0 con 0x7eff84156700 2024-09-21T07:34:47.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.716+0000 7eff7be00640 1 -- 172.21.15.39:0/4052929847 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c004030 con 0x7eff84156700 2024-09-21T07:34:47.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.716+0000 7eff8aa00640 1 -- 172.21.15.39:0/4052929847 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff841f5620 con 0x7eff84156700 2024-09-21T07:34:47.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.717+0000 7eff8aa00640 1 -- 172.21.15.39:0/4052929847 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff84074640 con 0x7eff84156700 2024-09-21T07:34:47.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.718+0000 7eff7be00640 1 -- 172.21.15.39:0/4052929847 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7eff7c0041d0 con 0x7eff84156700 2024-09-21T07:34:47.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.719+0000 7eff7be00640 1 --2- 172.21.15.39:0/4052929847 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7eff64078bc0 0x7eff6407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:47.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.719+0000 7eff7be00640 1 -- 172.21.15.39:0/4052929847 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6062+0+0 (secure 0 0 0) 0x7eff7c09b900 con 0x7eff84156700 2024-09-21T07:34:47.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.720+0000 7eff83e00640 1 --2- 172.21.15.39:0/4052929847 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7eff64078bc0 0x7eff6407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:47.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.721+0000 7eff83e00640 1 --2- 172.21.15.39:0/4052929847 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7eff64078bc0 0x7eff6407b0d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7eff74001050 tx=0x7eff74001e10 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:47.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.724+0000 7eff7be00640 1 -- 172.21.15.39:0/4052929847 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7eff7c063c70 con 0x7eff84156700 2024-09-21T07:34:47.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.917+0000 7eff8aa00640 1 -- 172.21.15.39:0/4052929847 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7eff840c3860 con 0x7eff64078bc0 2024-09-21T07:34:47.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.925+0000 7eff7be00640 1 -- 172.21.15.39:0/4052929847 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332429 (secure 0 0 0) 0x7eff840c3860 con 0x7eff64078bc0 2024-09-21T07:34:47.927 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:34:47.932 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:34:47.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.929+0000 7eff8aa00640 1 -- 172.21.15.39:0/4052929847 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7eff64078bc0 msgr2=0x7eff6407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:47.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.929+0000 7eff8aa00640 1 --2- 172.21.15.39:0/4052929847 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7eff64078bc0 0x7eff6407b0d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7eff74001050 tx=0x7eff74001e10 comp rx=0 tx=0).stop 2024-09-21T07:34:47.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.929+0000 7eff8aa00640 1 -- 172.21.15.39:0/4052929847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff84156700 msgr2=0x7eff8415f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:47.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.929+0000 7eff8aa00640 1 --2- 172.21.15.39:0/4052929847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff84156700 0x7eff8415f620 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7eff840cd410 tx=0x7eff7c010f40 comp rx=0 tx=0).stop 2024-09-21T07:34:47.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.929+0000 7eff8aa00640 1 -- 172.21.15.39:0/4052929847 shutdown_connections 2024-09-21T07:34:47.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.929+0000 7eff8aa00640 1 --2- 172.21.15.39:0/4052929847 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7eff64078bc0 0x7eff6407b0d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:47.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.929+0000 7eff8aa00640 1 --2- 172.21.15.39:0/4052929847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff84156700 0x7eff8415f620 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:47.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.929+0000 7eff8aa00640 1 --2- 172.21.15.39:0/4052929847 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7eff841562f0 0x7eff8415f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:47.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.929+0000 7eff8aa00640 1 --2- 172.21.15.39:0/4052929847 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff68004800 0x7eff8409d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:47.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.929+0000 7eff8aa00640 1 -- 172.21.15.39:0/4052929847 >> 172.21.15.39:0/4052929847 conn(0x7eff840b1cb0 msgr2=0x7eff8415aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:47.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.929+0000 7eff8aa00640 1 -- 172.21.15.39:0/4052929847 shutdown_connections 2024-09-21T07:34:47.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:47.929+0000 7eff8aa00640 1 -- 172.21.15.39:0/4052929847 wait complete. 2024-09-21T07:34:47.955 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2024-09-21T07:34:47.556632+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592126,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":138,"num_write_kb":1640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":198,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":246848,"kb_used_data":5628,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754727872,"statfs":{"total":773094113280,"available":772841340928,"internally_reserved":0,"allocated":5763072,"data_stored":3099143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1758,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001569"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.466557+0000","last_change":"2024-09-21T07:34:43.473741+0000","last_active":"2024-09-21T07:34:44.466557+0000","last_peered":"2024-09-21T07:34:44.466557+0000","last_clean":"2024-09-21T07:34:44.466557+0000","last_became_active":"2024-09-21T07:34:43.473447+0000","last_became_peered":"2024-09-21T07:34:43.473447+0000","last_unstale":"2024-09-21T07:34:44.466557+0000","last_undegraded":"2024-09-21T07:34:44.466557+0000","last_fullsized":"2024-09-21T07:34:44.466557+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:00:23.404297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836408+0000","last_change":"2024-09-21T07:34:43.473362+0000","last_active":"2024-09-21T07:34:44.836408+0000","last_peered":"2024-09-21T07:34:44.836408+0000","last_clean":"2024-09-21T07:34:44.836408+0000","last_became_active":"2024-09-21T07:34:43.473083+0000","last_became_peered":"2024-09-21T07:34:43.473083+0000","last_unstale":"2024-09-21T07:34:44.836408+0000","last_undegraded":"2024-09-21T07:34:44.836408+0000","last_fullsized":"2024-09-21T07:34:44.836408+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:33.920441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834020+0000","last_change":"2024-09-21T07:34:43.481499+0000","last_active":"2024-09-21T07:34:44.834020+0000","last_peered":"2024-09-21T07:34:44.834020+0000","last_clean":"2024-09-21T07:34:44.834020+0000","last_became_active":"2024-09-21T07:34:43.478254+0000","last_became_peered":"2024-09-21T07:34:43.478254+0000","last_unstale":"2024-09-21T07:34:44.834020+0000","last_undegraded":"2024-09-21T07:34:44.834020+0000","last_fullsized":"2024-09-21T07:34:44.834020+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:03.654293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836742+0000","last_change":"2024-09-21T07:34:43.451459+0000","last_active":"2024-09-21T07:34:44.836742+0000","last_peered":"2024-09-21T07:34:44.836742+0000","last_clean":"2024-09-21T07:34:44.836742+0000","last_became_active":"2024-09-21T07:34:43.451121+0000","last_became_peered":"2024-09-21T07:34:43.451121+0000","last_unstale":"2024-09-21T07:34:44.836742+0000","last_undegraded":"2024-09-21T07:34:44.836742+0000","last_fullsized":"2024-09-21T07:34:44.836742+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:15.139818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834482+0000","last_change":"2024-09-21T07:34:43.455772+0000","last_active":"2024-09-21T07:34:44.834482+0000","last_peered":"2024-09-21T07:34:44.834482+0000","last_clean":"2024-09-21T07:34:44.834482+0000","last_became_active":"2024-09-21T07:34:43.454461+0000","last_became_peered":"2024-09-21T07:34:43.454461+0000","last_unstale":"2024-09-21T07:34:44.834482+0000","last_undegraded":"2024-09-21T07:34:44.834482+0000","last_fullsized":"2024-09-21T07:34:44.834482+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:03.456938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465593+0000","last_change":"2024-09-21T07:34:43.469533+0000","last_active":"2024-09-21T07:34:44.465593+0000","last_peered":"2024-09-21T07:34:44.465593+0000","last_clean":"2024-09-21T07:34:44.465593+0000","last_became_active":"2024-09-21T07:34:43.469336+0000","last_became_peered":"2024-09-21T07:34:43.469336+0000","last_unstale":"2024-09-21T07:34:44.465593+0000","last_undegraded":"2024-09-21T07:34:44.465593+0000","last_fullsized":"2024-09-21T07:34:44.465593+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:20.540854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833543+0000","last_change":"2024-09-21T07:34:43.483800+0000","last_active":"2024-09-21T07:34:44.833543+0000","last_peered":"2024-09-21T07:34:44.833543+0000","last_clean":"2024-09-21T07:34:44.833543+0000","last_became_active":"2024-09-21T07:34:43.483482+0000","last_became_peered":"2024-09-21T07:34:43.483482+0000","last_unstale":"2024-09-21T07:34:44.833543+0000","last_undegraded":"2024-09-21T07:34:44.833543+0000","last_fullsized":"2024-09-21T07:34:44.833543+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:20.152451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834422+0000","last_change":"2024-09-21T07:34:43.473653+0000","last_active":"2024-09-21T07:34:44.834422+0000","last_peered":"2024-09-21T07:34:44.834422+0000","last_clean":"2024-09-21T07:34:44.834422+0000","last_became_active":"2024-09-21T07:34:43.473348+0000","last_became_peered":"2024-09-21T07:34:43.473348+0000","last_unstale":"2024-09-21T07:34:44.834422+0000","last_undegraded":"2024-09-21T07:34:44.834422+0000","last_fullsized":"2024-09-21T07:34:44.834422+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:13.042429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835096+0000","last_change":"2024-09-21T07:34:43.483507+0000","last_active":"2024-09-21T07:34:44.835096+0000","last_peered":"2024-09-21T07:34:44.835096+0000","last_clean":"2024-09-21T07:34:44.835096+0000","last_became_active":"2024-09-21T07:34:43.482198+0000","last_became_peered":"2024-09-21T07:34:43.482198+0000","last_unstale":"2024-09-21T07:34:44.835096+0000","last_undegraded":"2024-09-21T07:34:44.835096+0000","last_fullsized":"2024-09-21T07:34:44.835096+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:45.957857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465790+0000","last_change":"2024-09-21T07:34:43.465258+0000","last_active":"2024-09-21T07:34:44.465790+0000","last_peered":"2024-09-21T07:34:44.465790+0000","last_clean":"2024-09-21T07:34:44.465790+0000","last_became_active":"2024-09-21T07:34:43.464922+0000","last_became_peered":"2024-09-21T07:34:43.464922+0000","last_unstale":"2024-09-21T07:34:44.465790+0000","last_undegraded":"2024-09-21T07:34:44.465790+0000","last_fullsized":"2024-09-21T07:34:44.465790+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:11.406345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467028+0000","last_change":"2024-09-21T07:34:43.449879+0000","last_active":"2024-09-21T07:34:44.467028+0000","last_peered":"2024-09-21T07:34:44.467028+0000","last_clean":"2024-09-21T07:34:44.467028+0000","last_became_active":"2024-09-21T07:34:43.448365+0000","last_became_peered":"2024-09-21T07:34:43.448365+0000","last_unstale":"2024-09-21T07:34:44.467028+0000","last_undegraded":"2024-09-21T07:34:44.467028+0000","last_fullsized":"2024-09-21T07:34:44.467028+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:29:00.993972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834222+0000","last_change":"2024-09-21T07:34:43.460167+0000","last_active":"2024-09-21T07:34:44.834222+0000","last_peered":"2024-09-21T07:34:44.834222+0000","last_clean":"2024-09-21T07:34:44.834222+0000","last_became_active":"2024-09-21T07:34:43.457296+0000","last_became_peered":"2024-09-21T07:34:43.457296+0000","last_unstale":"2024-09-21T07:34:44.834222+0000","last_undegraded":"2024-09-21T07:34:44.834222+0000","last_fullsized":"2024-09-21T07:34:44.834222+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:53:57.871505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.466129+0000","last_change":"2024-09-21T07:34:43.465519+0000","last_active":"2024-09-21T07:34:44.466129+0000","last_peered":"2024-09-21T07:34:44.466129+0000","last_clean":"2024-09-21T07:34:44.466129+0000","last_became_active":"2024-09-21T07:34:43.464802+0000","last_became_peered":"2024-09-21T07:34:43.464802+0000","last_unstale":"2024-09-21T07:34:44.466129+0000","last_undegraded":"2024-09-21T07:34:44.466129+0000","last_fullsized":"2024-09-21T07:34:44.466129+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:59.062262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834033+0000","last_change":"2024-09-21T07:34:43.478575+0000","last_active":"2024-09-21T07:34:44.834033+0000","last_peered":"2024-09-21T07:34:44.834033+0000","last_clean":"2024-09-21T07:34:44.834033+0000","last_became_active":"2024-09-21T07:34:43.478187+0000","last_became_peered":"2024-09-21T07:34:43.478187+0000","last_unstale":"2024-09-21T07:34:44.834033+0000","last_undegraded":"2024-09-21T07:34:44.834033+0000","last_fullsized":"2024-09-21T07:34:44.834033+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:44.866369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836848+0000","last_change":"2024-09-21T07:34:43.467093+0000","last_active":"2024-09-21T07:34:44.836848+0000","last_peered":"2024-09-21T07:34:44.836848+0000","last_clean":"2024-09-21T07:34:44.836848+0000","last_became_active":"2024-09-21T07:34:43.466773+0000","last_became_peered":"2024-09-21T07:34:43.466773+0000","last_unstale":"2024-09-21T07:34:44.836848+0000","last_undegraded":"2024-09-21T07:34:44.836848+0000","last_fullsized":"2024-09-21T07:34:44.836848+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:12:02.180192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465260+0000","last_change":"2024-09-21T07:34:43.462637+0000","last_active":"2024-09-21T07:34:44.465260+0000","last_peered":"2024-09-21T07:34:44.465260+0000","last_clean":"2024-09-21T07:34:44.465260+0000","last_became_active":"2024-09-21T07:34:43.462301+0000","last_became_peered":"2024-09-21T07:34:43.462301+0000","last_unstale":"2024-09-21T07:34:44.465260+0000","last_undegraded":"2024-09-21T07:34:44.465260+0000","last_fullsized":"2024-09-21T07:34:44.465260+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:42.168665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834898+0000","last_change":"2024-09-21T07:34:43.484342+0000","last_active":"2024-09-21T07:34:44.834898+0000","last_peered":"2024-09-21T07:34:44.834898+0000","last_clean":"2024-09-21T07:34:44.834898+0000","last_became_active":"2024-09-21T07:34:43.480452+0000","last_became_peered":"2024-09-21T07:34:43.480452+0000","last_unstale":"2024-09-21T07:34:44.834898+0000","last_undegraded":"2024-09-21T07:34:44.834898+0000","last_fullsized":"2024-09-21T07:34:44.834898+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:30.267103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836590+0000","last_change":"2024-09-21T07:34:43.472890+0000","last_active":"2024-09-21T07:34:44.836590+0000","last_peered":"2024-09-21T07:34:44.836590+0000","last_clean":"2024-09-21T07:34:44.836590+0000","last_became_active":"2024-09-21T07:34:43.472576+0000","last_became_peered":"2024-09-21T07:34:43.472576+0000","last_unstale":"2024-09-21T07:34:44.836590+0000","last_undegraded":"2024-09-21T07:34:44.836590+0000","last_fullsized":"2024-09-21T07:34:44.836590+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:24.719441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834303+0000","last_change":"2024-09-21T07:34:43.476774+0000","last_active":"2024-09-21T07:34:44.834303+0000","last_peered":"2024-09-21T07:34:44.834303+0000","last_clean":"2024-09-21T07:34:44.834303+0000","last_became_active":"2024-09-21T07:34:43.473147+0000","last_became_peered":"2024-09-21T07:34:43.473147+0000","last_unstale":"2024-09-21T07:34:44.834303+0000","last_undegraded":"2024-09-21T07:34:44.834303+0000","last_fullsized":"2024-09-21T07:34:44.834303+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:10.661941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-21T07:34:42.434570+0000","last_change":"2024-09-21T07:34:42.434570+0000","last_active":"2024-09-21T07:34:42.415959+0000","last_peered":"2024-09-21T07:34:42.415959+0000","last_clean":"2024-09-21T07:34:42.415959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:42.434570+0000","last_undegraded":"2024-09-21T07:34:42.434570+0000","last_fullsized":"2024-09-21T07:34:42.434570+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-21T07:34:42.434971+0000","last_change":"2024-09-21T07:34:42.434971+0000","last_active":"2024-09-21T07:34:42.415959+0000","last_peered":"2024-09-21T07:34:42.415959+0000","last_clean":"2024-09-21T07:34:42.415959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:42.434971+0000","last_undegraded":"2024-09-21T07:34:42.434971+0000","last_fullsized":"2024-09-21T07:34:42.434971+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834766+0000","last_change":"2024-09-21T07:34:43.483312+0000","last_active":"2024-09-21T07:34:44.834766+0000","last_peered":"2024-09-21T07:34:44.834766+0000","last_clean":"2024-09-21T07:34:44.834766+0000","last_became_active":"2024-09-21T07:34:43.483003+0000","last_became_peered":"2024-09-21T07:34:43.483003+0000","last_unstale":"2024-09-21T07:34:44.834766+0000","last_undegraded":"2024-09-21T07:34:44.834766+0000","last_fullsized":"2024-09-21T07:34:44.834766+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:33.782633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467544+0000","last_change":"2024-09-21T07:34:43.449273+0000","last_active":"2024-09-21T07:34:44.467544+0000","last_peered":"2024-09-21T07:34:44.467544+0000","last_clean":"2024-09-21T07:34:44.467544+0000","last_became_active":"2024-09-21T07:34:43.448944+0000","last_became_peered":"2024-09-21T07:34:43.448944+0000","last_unstale":"2024-09-21T07:34:44.467544+0000","last_undegraded":"2024-09-21T07:34:44.467544+0000","last_fullsized":"2024-09-21T07:34:44.467544+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:56.540403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833398+0000","last_change":"2024-09-21T07:34:43.479357+0000","last_active":"2024-09-21T07:34:44.833398+0000","last_peered":"2024-09-21T07:34:44.833398+0000","last_clean":"2024-09-21T07:34:44.833398+0000","last_became_active":"2024-09-21T07:34:43.478997+0000","last_became_peered":"2024-09-21T07:34:43.478997+0000","last_unstale":"2024-09-21T07:34:44.833398+0000","last_undegraded":"2024-09-21T07:34:44.833398+0000","last_fullsized":"2024-09-21T07:34:44.833398+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:54.644949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835486+0000","last_change":"2024-09-21T07:34:43.486869+0000","last_active":"2024-09-21T07:34:44.835486+0000","last_peered":"2024-09-21T07:34:44.835486+0000","last_clean":"2024-09-21T07:34:44.835486+0000","last_became_active":"2024-09-21T07:34:43.486351+0000","last_became_peered":"2024-09-21T07:34:43.486351+0000","last_unstale":"2024-09-21T07:34:44.835486+0000","last_undegraded":"2024-09-21T07:34:44.835486+0000","last_fullsized":"2024-09-21T07:34:44.835486+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:21:02.910848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833742+0000","last_change":"2024-09-21T07:34:43.463196+0000","last_active":"2024-09-21T07:34:44.833742+0000","last_peered":"2024-09-21T07:34:44.833742+0000","last_clean":"2024-09-21T07:34:44.833742+0000","last_became_active":"2024-09-21T07:34:43.458559+0000","last_became_peered":"2024-09-21T07:34:43.458559+0000","last_unstale":"2024-09-21T07:34:44.833742+0000","last_undegraded":"2024-09-21T07:34:44.833742+0000","last_fullsized":"2024-09-21T07:34:44.833742+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:37.233429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467280+0000","last_change":"2024-09-21T07:34:43.483549+0000","last_active":"2024-09-21T07:34:44.467280+0000","last_peered":"2024-09-21T07:34:44.467280+0000","last_clean":"2024-09-21T07:34:44.467280+0000","last_became_active":"2024-09-21T07:34:43.472653+0000","last_became_peered":"2024-09-21T07:34:43.472653+0000","last_unstale":"2024-09-21T07:34:44.467280+0000","last_undegraded":"2024-09-21T07:34:44.467280+0000","last_fullsized":"2024-09-21T07:34:44.467280+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:38.814916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-21T07:34:42.435611+0000","last_change":"2024-09-21T07:34:42.435611+0000","last_active":"2024-09-21T07:34:42.415959+0000","last_peered":"2024-09-21T07:34:42.415959+0000","last_clean":"2024-09-21T07:34:42.415959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:42.435611+0000","last_undegraded":"2024-09-21T07:34:42.435611+0000","last_fullsized":"2024-09-21T07:34:42.435611+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833338+0000","last_change":"2024-09-21T07:34:43.489357+0000","last_active":"2024-09-21T07:34:44.833338+0000","last_peered":"2024-09-21T07:34:44.833338+0000","last_clean":"2024-09-21T07:34:44.833338+0000","last_became_active":"2024-09-21T07:34:43.483224+0000","last_became_peered":"2024-09-21T07:34:43.483224+0000","last_unstale":"2024-09-21T07:34:44.833338+0000","last_undegraded":"2024-09-21T07:34:44.833338+0000","last_fullsized":"2024-09-21T07:34:44.833338+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:05.212534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835873+0000","last_change":"2024-09-21T07:34:43.460376+0000","last_active":"2024-09-21T07:34:44.835873+0000","last_peered":"2024-09-21T07:34:44.835873+0000","last_clean":"2024-09-21T07:34:44.835873+0000","last_became_active":"2024-09-21T07:34:43.458320+0000","last_became_peered":"2024-09-21T07:34:43.458320+0000","last_unstale":"2024-09-21T07:34:44.835873+0000","last_undegraded":"2024-09-21T07:34:44.835873+0000","last_fullsized":"2024-09-21T07:34:44.835873+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:35.190652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833745+0000","last_change":"2024-09-21T07:34:43.485349+0000","last_active":"2024-09-21T07:34:44.833745+0000","last_peered":"2024-09-21T07:34:44.833745+0000","last_clean":"2024-09-21T07:34:44.833745+0000","last_became_active":"2024-09-21T07:34:43.483623+0000","last_became_peered":"2024-09-21T07:34:43.483623+0000","last_unstale":"2024-09-21T07:34:44.833745+0000","last_undegraded":"2024-09-21T07:34:44.833745+0000","last_fullsized":"2024-09-21T07:34:44.833745+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:02.084719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467658+0000","last_change":"2024-09-21T07:34:43.471970+0000","last_active":"2024-09-21T07:34:44.467658+0000","last_peered":"2024-09-21T07:34:44.467658+0000","last_clean":"2024-09-21T07:34:44.467658+0000","last_became_active":"2024-09-21T07:34:43.465515+0000","last_became_peered":"2024-09-21T07:34:43.465515+0000","last_unstale":"2024-09-21T07:34:44.467658+0000","last_undegraded":"2024-09-21T07:34:44.467658+0000","last_fullsized":"2024-09-21T07:34:44.467658+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:59:53.819831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-21T07:34:42.435086+0000","last_change":"2024-09-21T07:34:42.435086+0000","last_active":"2024-09-21T07:34:42.415959+0000","last_peered":"2024-09-21T07:34:42.415959+0000","last_clean":"2024-09-21T07:34:42.415959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:42.435086+0000","last_undegraded":"2024-09-21T07:34:42.435086+0000","last_fullsized":"2024-09-21T07:34:42.435086+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"49'1","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834973+0000","last_change":"2024-09-21T07:34:42.447496+0000","last_active":"2024-09-21T07:34:44.834973+0000","last_peered":"2024-09-21T07:34:44.834973+0000","last_clean":"2024-09-21T07:34:44.834973+0000","last_became_active":"2024-09-21T07:34:42.447148+0000","last_became_peered":"2024-09-21T07:34:42.447148+0000","last_unstale":"2024-09-21T07:34:44.834973+0000","last_undegraded":"2024-09-21T07:34:44.834973+0000","last_fullsized":"2024-09-21T07:34:44.834973+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:42:06.750595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":163,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835508+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:34:44.835508+0000","last_peered":"2024-09-21T07:34:44.835508+0000","last_clean":"2024-09-21T07:34:44.835508+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:34:44.835508+0000","last_undegraded":"2024-09-21T07:34:44.835508+0000","last_fullsized":"2024-09-21T07:34:44.835508+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.468769+0000","last_change":"2024-09-21T07:34:24.539248+0000","last_active":"2024-09-21T07:34:44.468769+0000","last_peered":"2024-09-21T07:34:44.468769+0000","last_clean":"2024-09-21T07:34:44.468769+0000","last_became_active":"2024-09-21T07:34:23.273465+0000","last_became_peered":"2024-09-21T07:34:23.273465+0000","last_unstale":"2024-09-21T07:34:44.468769+0000","last_undegraded":"2024-09-21T07:34:44.468769+0000","last_fullsized":"2024-09-21T07:34:44.468769+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:55.517452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060627700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834170+0000","last_change":"2024-09-21T07:34:43.491609+0000","last_active":"2024-09-21T07:34:44.834170+0000","last_peered":"2024-09-21T07:34:44.834170+0000","last_clean":"2024-09-21T07:34:44.834170+0000","last_became_active":"2024-09-21T07:34:43.490138+0000","last_became_peered":"2024-09-21T07:34:43.490138+0000","last_unstale":"2024-09-21T07:34:44.834170+0000","last_undegraded":"2024-09-21T07:34:44.834170+0000","last_fullsized":"2024-09-21T07:34:44.834170+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:16.015947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833429+0000","last_change":"2024-09-21T07:34:42.448815+0000","last_active":"2024-09-21T07:34:44.833429+0000","last_peered":"2024-09-21T07:34:44.833429+0000","last_clean":"2024-09-21T07:34:44.833429+0000","last_became_active":"2024-09-21T07:34:42.448484+0000","last_became_peered":"2024-09-21T07:34:42.448484+0000","last_unstale":"2024-09-21T07:34:44.833429+0000","last_undegraded":"2024-09-21T07:34:44.833429+0000","last_fullsized":"2024-09-21T07:34:44.833429+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:15.522307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833978+0000","last_change":"2024-09-21T07:34:24.597274+0000","last_active":"2024-09-21T07:34:44.833978+0000","last_peered":"2024-09-21T07:34:44.833978+0000","last_clean":"2024-09-21T07:34:44.833978+0000","last_became_active":"2024-09-21T07:34:23.279560+0000","last_became_peered":"2024-09-21T07:34:23.279560+0000","last_unstale":"2024-09-21T07:34:44.833978+0000","last_undegraded":"2024-09-21T07:34:44.833978+0000","last_fullsized":"2024-09-21T07:34:44.833978+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:15.845265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059699799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.468039+0000","last_change":"2024-09-21T07:34:43.468683+0000","last_active":"2024-09-21T07:34:44.468039+0000","last_peered":"2024-09-21T07:34:44.468039+0000","last_clean":"2024-09-21T07:34:44.468039+0000","last_became_active":"2024-09-21T07:34:43.468377+0000","last_became_peered":"2024-09-21T07:34:43.468377+0000","last_unstale":"2024-09-21T07:34:44.468039+0000","last_undegraded":"2024-09-21T07:34:44.468039+0000","last_fullsized":"2024-09-21T07:34:44.468039+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:02.084719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834944+0000","last_change":"2024-09-21T07:34:42.442148+0000","last_active":"2024-09-21T07:34:44.834944+0000","last_peered":"2024-09-21T07:34:44.834944+0000","last_clean":"2024-09-21T07:34:44.834944+0000","last_became_active":"2024-09-21T07:34:42.441839+0000","last_became_peered":"2024-09-21T07:34:42.441839+0000","last_unstale":"2024-09-21T07:34:44.834944+0000","last_undegraded":"2024-09-21T07:34:44.834944+0000","last_fullsized":"2024-09-21T07:34:44.834944+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:35:57.065813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836188+0000","last_change":"2024-09-21T07:34:24.259906+0000","last_active":"2024-09-21T07:34:44.836188+0000","last_peered":"2024-09-21T07:34:44.836188+0000","last_clean":"2024-09-21T07:34:44.836188+0000","last_became_active":"2024-09-21T07:34:23.635282+0000","last_became_peered":"2024-09-21T07:34:23.635282+0000","last_unstale":"2024-09-21T07:34:44.836188+0000","last_undegraded":"2024-09-21T07:34:44.836188+0000","last_fullsized":"2024-09-21T07:34:44.836188+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:00.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001527664,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834033+0000","last_change":"2024-09-21T07:34:43.491274+0000","last_active":"2024-09-21T07:34:44.834033+0000","last_peered":"2024-09-21T07:34:44.834033+0000","last_clean":"2024-09-21T07:34:44.834033+0000","last_became_active":"2024-09-21T07:34:43.489063+0000","last_became_peered":"2024-09-21T07:34:43.489063+0000","last_unstale":"2024-09-21T07:34:44.834033+0000","last_undegraded":"2024-09-21T07:34:44.834033+0000","last_fullsized":"2024-09-21T07:34:44.834033+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:13.315568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833450+0000","last_change":"2024-09-21T07:34:42.437481+0000","last_active":"2024-09-21T07:34:44.833450+0000","last_peered":"2024-09-21T07:34:44.833450+0000","last_clean":"2024-09-21T07:34:44.833450+0000","last_became_active":"2024-09-21T07:34:42.437130+0000","last_became_peered":"2024-09-21T07:34:42.437130+0000","last_unstale":"2024-09-21T07:34:44.833450+0000","last_undegraded":"2024-09-21T07:34:44.833450+0000","last_fullsized":"2024-09-21T07:34:44.833450+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:55.611339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833360+0000","last_change":"2024-09-21T07:34:25.085454+0000","last_active":"2024-09-21T07:34:44.833360+0000","last_peered":"2024-09-21T07:34:44.833360+0000","last_clean":"2024-09-21T07:34:44.833360+0000","last_became_active":"2024-09-21T07:34:23.279205+0000","last_became_peered":"2024-09-21T07:34:23.279205+0000","last_unstale":"2024-09-21T07:34:44.833360+0000","last_undegraded":"2024-09-21T07:34:44.833360+0000","last_fullsized":"2024-09-21T07:34:44.833360+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:54.231666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095320599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835309+0000","last_change":"2024-09-21T07:34:43.457980+0000","last_active":"2024-09-21T07:34:44.835309+0000","last_peered":"2024-09-21T07:34:44.835309+0000","last_clean":"2024-09-21T07:34:44.835309+0000","last_became_active":"2024-09-21T07:34:43.457695+0000","last_became_peered":"2024-09-21T07:34:43.457695+0000","last_unstale":"2024-09-21T07:34:44.835309+0000","last_undegraded":"2024-09-21T07:34:44.835309+0000","last_fullsized":"2024-09-21T07:34:44.835309+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:01.492292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467870+0000","last_change":"2024-09-21T07:34:42.438107+0000","last_active":"2024-09-21T07:34:44.467870+0000","last_peered":"2024-09-21T07:34:44.467870+0000","last_clean":"2024-09-21T07:34:44.467870+0000","last_became_active":"2024-09-21T07:34:42.437794+0000","last_became_peered":"2024-09-21T07:34:42.437794+0000","last_unstale":"2024-09-21T07:34:44.467870+0000","last_undegraded":"2024-09-21T07:34:44.467870+0000","last_fullsized":"2024-09-21T07:34:44.467870+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:24.649223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465509+0000","last_change":"2024-09-21T07:34:25.028521+0000","last_active":"2024-09-21T07:34:44.465509+0000","last_peered":"2024-09-21T07:34:44.465509+0000","last_clean":"2024-09-21T07:34:44.465509+0000","last_became_active":"2024-09-21T07:34:23.635467+0000","last_became_peered":"2024-09-21T07:34:23.635467+0000","last_unstale":"2024-09-21T07:34:44.465509+0000","last_undegraded":"2024-09-21T07:34:44.465509+0000","last_fullsized":"2024-09-21T07:34:44.465509+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:33.729123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000660558,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467665+0000","last_change":"2024-09-21T07:34:43.465578+0000","last_active":"2024-09-21T07:34:44.467665+0000","last_peered":"2024-09-21T07:34:44.467665+0000","last_clean":"2024-09-21T07:34:44.467665+0000","last_became_active":"2024-09-21T07:34:43.465071+0000","last_became_peered":"2024-09-21T07:34:43.465071+0000","last_unstale":"2024-09-21T07:34:44.467665+0000","last_undegraded":"2024-09-21T07:34:44.467665+0000","last_fullsized":"2024-09-21T07:34:44.467665+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:20.152451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:34:42.453134+0000","last_change":"2024-09-21T07:34:42.445927+0000","last_active":"2024-09-21T07:34:42.453134+0000","last_peered":"2024-09-21T07:34:42.453134+0000","last_clean":"2024-09-21T07:34:42.453134+0000","last_became_active":"2024-09-21T07:34:42.445620+0000","last_became_peered":"2024-09-21T07:34:42.445620+0000","last_unstale":"2024-09-21T07:34:42.453134+0000","last_undegraded":"2024-09-21T07:34:42.453134+0000","last_fullsized":"2024-09-21T07:34:42.453134+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:19.691881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":40,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835533+0000","last_change":"2024-09-21T07:34:24.260046+0000","last_active":"2024-09-21T07:34:44.835533+0000","last_peered":"2024-09-21T07:34:44.835533+0000","last_clean":"2024-09-21T07:34:44.835533+0000","last_became_active":"2024-09-21T07:34:23.273431+0000","last_became_peered":"2024-09-21T07:34:23.273431+0000","last_unstale":"2024-09-21T07:34:44.835533+0000","last_undegraded":"2024-09-21T07:34:44.835533+0000","last_fullsized":"2024-09-21T07:34:44.835533+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:18.876057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019803310000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465631+0000","last_change":"2024-09-21T07:34:43.462550+0000","last_active":"2024-09-21T07:34:44.465631+0000","last_peered":"2024-09-21T07:34:44.465631+0000","last_clean":"2024-09-21T07:34:44.465631+0000","last_became_active":"2024-09-21T07:34:43.462210+0000","last_became_peered":"2024-09-21T07:34:43.462210+0000","last_unstale":"2024-09-21T07:34:44.465631+0000","last_undegraded":"2024-09-21T07:34:44.465631+0000","last_fullsized":"2024-09-21T07:34:44.465631+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:56.480314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836267+0000","last_change":"2024-09-21T07:34:42.452761+0000","last_active":"2024-09-21T07:34:44.836267+0000","last_peered":"2024-09-21T07:34:44.836267+0000","last_clean":"2024-09-21T07:34:44.836267+0000","last_became_active":"2024-09-21T07:34:42.452400+0000","last_became_peered":"2024-09-21T07:34:42.452400+0000","last_unstale":"2024-09-21T07:34:44.836267+0000","last_undegraded":"2024-09-21T07:34:44.836267+0000","last_fullsized":"2024-09-21T07:34:44.836267+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:19:05.278221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465673+0000","last_change":"2024-09-21T07:34:25.028640+0000","last_active":"2024-09-21T07:34:44.465673+0000","last_peered":"2024-09-21T07:34:44.465673+0000","last_clean":"2024-09-21T07:34:44.465673+0000","last_became_active":"2024-09-21T07:34:23.260950+0000","last_became_peered":"2024-09-21T07:34:23.260950+0000","last_unstale":"2024-09-21T07:34:44.465673+0000","last_undegraded":"2024-09-21T07:34:44.465673+0000","last_fullsized":"2024-09-21T07:34:44.465673+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:23:42.121398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055135099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833638+0000","last_change":"2024-09-21T07:34:43.453996+0000","last_active":"2024-09-21T07:34:44.833638+0000","last_peered":"2024-09-21T07:34:44.833638+0000","last_clean":"2024-09-21T07:34:44.833638+0000","last_became_active":"2024-09-21T07:34:43.453666+0000","last_became_peered":"2024-09-21T07:34:43.453666+0000","last_unstale":"2024-09-21T07:34:44.833638+0000","last_undegraded":"2024-09-21T07:34:44.833638+0000","last_fullsized":"2024-09-21T07:34:44.833638+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:31.445807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834426+0000","last_change":"2024-09-21T07:34:42.453822+0000","last_active":"2024-09-21T07:34:44.834426+0000","last_peered":"2024-09-21T07:34:44.834426+0000","last_clean":"2024-09-21T07:34:44.834426+0000","last_became_active":"2024-09-21T07:34:42.453499+0000","last_became_peered":"2024-09-21T07:34:42.453499+0000","last_unstale":"2024-09-21T07:34:44.834426+0000","last_undegraded":"2024-09-21T07:34:44.834426+0000","last_fullsized":"2024-09-21T07:34:44.834426+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:39:29.733953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836757+0000","last_change":"2024-09-21T07:34:24.260665+0000","last_active":"2024-09-21T07:34:44.836757+0000","last_peered":"2024-09-21T07:34:44.836757+0000","last_clean":"2024-09-21T07:34:44.836757+0000","last_became_active":"2024-09-21T07:34:23.635909+0000","last_became_peered":"2024-09-21T07:34:23.635909+0000","last_unstale":"2024-09-21T07:34:44.836757+0000","last_undegraded":"2024-09-21T07:34:44.836757+0000","last_fullsized":"2024-09-21T07:34:44.836757+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:44.536885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.468489+0000","last_change":"2024-09-21T07:34:43.469198+0000","last_active":"2024-09-21T07:34:44.468489+0000","last_peered":"2024-09-21T07:34:44.468489+0000","last_clean":"2024-09-21T07:34:44.468489+0000","last_became_active":"2024-09-21T07:34:43.467622+0000","last_became_peered":"2024-09-21T07:34:43.467622+0000","last_unstale":"2024-09-21T07:34:44.468489+0000","last_undegraded":"2024-09-21T07:34:44.468489+0000","last_fullsized":"2024-09-21T07:34:44.468489+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:44.866369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"48'2","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465974+0000","last_change":"2024-09-21T07:34:42.438031+0000","last_active":"2024-09-21T07:34:44.465974+0000","last_peered":"2024-09-21T07:34:44.465974+0000","last_clean":"2024-09-21T07:34:44.465974+0000","last_became_active":"2024-09-21T07:34:42.436986+0000","last_became_peered":"2024-09-21T07:34:42.436986+0000","last_unstale":"2024-09-21T07:34:44.465974+0000","last_undegraded":"2024-09-21T07:34:44.465974+0000","last_fullsized":"2024-09-21T07:34:44.465974+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:51.213865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467759+0000","last_change":"2024-09-21T07:34:24.538845+0000","last_active":"2024-09-21T07:34:44.467759+0000","last_peered":"2024-09-21T07:34:44.467759+0000","last_clean":"2024-09-21T07:34:44.467759+0000","last_became_active":"2024-09-21T07:34:23.636372+0000","last_became_peered":"2024-09-21T07:34:23.636372+0000","last_unstale":"2024-09-21T07:34:44.467759+0000","last_undegraded":"2024-09-21T07:34:44.467759+0000","last_fullsized":"2024-09-21T07:34:44.467759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:29.393813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841451,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-21T07:34:42.436882+0000","last_change":"2024-09-21T07:34:42.436882+0000","last_active":"2024-09-21T07:34:42.415959+0000","last_peered":"2024-09-21T07:34:42.415959+0000","last_clean":"2024-09-21T07:34:42.415959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:42.436882+0000","last_undegraded":"2024-09-21T07:34:42.436882+0000","last_fullsized":"2024-09-21T07:34:42.436882+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834840+0000","last_change":"2024-09-21T07:34:42.447519+0000","last_active":"2024-09-21T07:34:44.834840+0000","last_peered":"2024-09-21T07:34:44.834840+0000","last_clean":"2024-09-21T07:34:44.834840+0000","last_became_active":"2024-09-21T07:34:42.447129+0000","last_became_peered":"2024-09-21T07:34:42.447129+0000","last_unstale":"2024-09-21T07:34:44.834840+0000","last_undegraded":"2024-09-21T07:34:44.834840+0000","last_fullsized":"2024-09-21T07:34:44.834840+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:30.464417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":40,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833703+0000","last_change":"2024-09-21T07:34:24.259686+0000","last_active":"2024-09-21T07:34:44.833703+0000","last_peered":"2024-09-21T07:34:44.833703+0000","last_clean":"2024-09-21T07:34:44.833703+0000","last_became_active":"2024-09-21T07:34:23.274405+0000","last_became_peered":"2024-09-21T07:34:23.274405+0000","last_unstale":"2024-09-21T07:34:44.833703+0000","last_undegraded":"2024-09-21T07:34:44.833703+0000","last_fullsized":"2024-09-21T07:34:44.833703+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:26.304693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022009920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834760+0000","last_change":"2024-09-21T07:34:43.476482+0000","last_active":"2024-09-21T07:34:44.834760+0000","last_peered":"2024-09-21T07:34:44.834760+0000","last_clean":"2024-09-21T07:34:44.834760+0000","last_became_active":"2024-09-21T07:34:43.476189+0000","last_became_peered":"2024-09-21T07:34:43.476189+0000","last_unstale":"2024-09-21T07:34:44.834760+0000","last_undegraded":"2024-09-21T07:34:44.834760+0000","last_fullsized":"2024-09-21T07:34:44.834760+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:08:42.688308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833369+0000","last_change":"2024-09-21T07:34:42.456020+0000","last_active":"2024-09-21T07:34:44.833369+0000","last_peered":"2024-09-21T07:34:44.833369+0000","last_clean":"2024-09-21T07:34:44.833369+0000","last_became_active":"2024-09-21T07:34:42.455399+0000","last_became_peered":"2024-09-21T07:34:42.455399+0000","last_unstale":"2024-09-21T07:34:44.833369+0000","last_undegraded":"2024-09-21T07:34:44.833369+0000","last_fullsized":"2024-09-21T07:34:44.833369+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:57.601710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835457+0000","last_change":"2024-09-21T07:34:24.258754+0000","last_active":"2024-09-21T07:34:44.835457+0000","last_peered":"2024-09-21T07:34:44.835457+0000","last_clean":"2024-09-21T07:34:44.835457+0000","last_became_active":"2024-09-21T07:34:23.272657+0000","last_became_peered":"2024-09-21T07:34:23.272657+0000","last_unstale":"2024-09-21T07:34:44.835457+0000","last_undegraded":"2024-09-21T07:34:44.835457+0000","last_fullsized":"2024-09-21T07:34:44.835457+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:05.477099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067143200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.466771+0000","last_change":"2024-09-21T07:34:43.472869+0000","last_active":"2024-09-21T07:34:44.466771+0000","last_peered":"2024-09-21T07:34:44.466771+0000","last_clean":"2024-09-21T07:34:44.466771+0000","last_became_active":"2024-09-21T07:34:43.472514+0000","last_became_peered":"2024-09-21T07:34:43.472514+0000","last_unstale":"2024-09-21T07:34:44.466771+0000","last_undegraded":"2024-09-21T07:34:44.466771+0000","last_fullsized":"2024-09-21T07:34:44.466771+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:15:22.567958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.466329+0000","last_change":"2024-09-21T07:34:42.443208+0000","last_active":"2024-09-21T07:34:44.466329+0000","last_peered":"2024-09-21T07:34:44.466329+0000","last_clean":"2024-09-21T07:34:44.466329+0000","last_became_active":"2024-09-21T07:34:42.442891+0000","last_became_peered":"2024-09-21T07:34:42.442891+0000","last_unstale":"2024-09-21T07:34:44.466329+0000","last_undegraded":"2024-09-21T07:34:44.466329+0000","last_fullsized":"2024-09-21T07:34:44.466329+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:02.347260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":38,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833706+0000","last_change":"2024-09-21T07:34:25.085232+0000","last_active":"2024-09-21T07:34:44.833706+0000","last_peered":"2024-09-21T07:34:44.833706+0000","last_clean":"2024-09-21T07:34:44.833706+0000","last_became_active":"2024-09-21T07:34:23.277807+0000","last_became_peered":"2024-09-21T07:34:23.277807+0000","last_unstale":"2024-09-21T07:34:44.833706+0000","last_undegraded":"2024-09-21T07:34:44.833706+0000","last_fullsized":"2024-09-21T07:34:44.833706+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:36.362919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086619099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835299+0000","last_change":"2024-09-21T07:34:43.486460+0000","last_active":"2024-09-21T07:34:44.835299+0000","last_peered":"2024-09-21T07:34:44.835299+0000","last_clean":"2024-09-21T07:34:44.835299+0000","last_became_active":"2024-09-21T07:34:43.486148+0000","last_became_peered":"2024-09-21T07:34:43.486148+0000","last_unstale":"2024-09-21T07:34:44.835299+0000","last_undegraded":"2024-09-21T07:34:44.835299+0000","last_fullsized":"2024-09-21T07:34:44.835299+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:31.997216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"48'1","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835624+0000","last_change":"2024-09-21T07:34:42.450738+0000","last_active":"2024-09-21T07:34:44.835624+0000","last_peered":"2024-09-21T07:34:44.835624+0000","last_clean":"2024-09-21T07:34:44.835624+0000","last_became_active":"2024-09-21T07:34:42.450413+0000","last_became_peered":"2024-09-21T07:34:42.450413+0000","last_unstale":"2024-09-21T07:34:44.835624+0000","last_undegraded":"2024-09-21T07:34:44.835624+0000","last_fullsized":"2024-09-21T07:34:44.835624+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:48.569857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835043+0000","last_change":"2024-09-21T07:34:24.597672+0000","last_active":"2024-09-21T07:34:44.835043+0000","last_peered":"2024-09-21T07:34:44.835043+0000","last_clean":"2024-09-21T07:34:44.835043+0000","last_became_active":"2024-09-21T07:34:23.267770+0000","last_became_peered":"2024-09-21T07:34:23.267770+0000","last_unstale":"2024-09-21T07:34:44.835043+0000","last_undegraded":"2024-09-21T07:34:44.835043+0000","last_fullsized":"2024-09-21T07:34:44.835043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:51.629476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086378500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465282+0000","last_change":"2024-09-21T07:34:43.478411+0000","last_active":"2024-09-21T07:34:44.465282+0000","last_peered":"2024-09-21T07:34:44.465282+0000","last_clean":"2024-09-21T07:34:44.465282+0000","last_became_active":"2024-09-21T07:34:43.478106+0000","last_became_peered":"2024-09-21T07:34:43.478106+0000","last_unstale":"2024-09-21T07:34:44.465282+0000","last_undegraded":"2024-09-21T07:34:44.465282+0000","last_fullsized":"2024-09-21T07:34:44.465282+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:41.320870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"48'2","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467059+0000","last_change":"2024-09-21T07:34:42.449630+0000","last_active":"2024-09-21T07:34:44.467059+0000","last_peered":"2024-09-21T07:34:44.467059+0000","last_clean":"2024-09-21T07:34:44.467059+0000","last_became_active":"2024-09-21T07:34:42.448013+0000","last_became_peered":"2024-09-21T07:34:42.448013+0000","last_unstale":"2024-09-21T07:34:44.467059+0000","last_undegraded":"2024-09-21T07:34:44.467059+0000","last_fullsized":"2024-09-21T07:34:44.467059+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:42.900618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":38,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467361+0000","last_change":"2024-09-21T07:34:24.260181+0000","last_active":"2024-09-21T07:34:44.467361+0000","last_peered":"2024-09-21T07:34:44.467361+0000","last_clean":"2024-09-21T07:34:44.467361+0000","last_became_active":"2024-09-21T07:34:23.275580+0000","last_became_peered":"2024-09-21T07:34:23.275580+0000","last_unstale":"2024-09-21T07:34:44.467361+0000","last_undegraded":"2024-09-21T07:34:44.467361+0000","last_fullsized":"2024-09-21T07:34:44.467361+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:49.857622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014706980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834317+0000","last_change":"2024-09-21T07:34:43.479707+0000","last_active":"2024-09-21T07:34:44.834317+0000","last_peered":"2024-09-21T07:34:44.834317+0000","last_clean":"2024-09-21T07:34:44.834317+0000","last_became_active":"2024-09-21T07:34:43.479311+0000","last_became_peered":"2024-09-21T07:34:43.479311+0000","last_unstale":"2024-09-21T07:34:44.834317+0000","last_undegraded":"2024-09-21T07:34:44.834317+0000","last_fullsized":"2024-09-21T07:34:44.834317+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:03:54.452233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:34:42.460152+0000","last_change":"2024-09-21T07:34:42.455379+0000","last_active":"2024-09-21T07:34:42.460152+0000","last_peered":"2024-09-21T07:34:42.460152+0000","last_clean":"2024-09-21T07:34:42.460152+0000","last_became_active":"2024-09-21T07:34:42.455144+0000","last_became_peered":"2024-09-21T07:34:42.455144+0000","last_unstale":"2024-09-21T07:34:42.460152+0000","last_undegraded":"2024-09-21T07:34:42.460152+0000","last_fullsized":"2024-09-21T07:34:42.460152+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:31:47.280896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":40,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834497+0000","last_change":"2024-09-21T07:34:24.259112+0000","last_active":"2024-09-21T07:34:44.834497+0000","last_peered":"2024-09-21T07:34:44.834497+0000","last_clean":"2024-09-21T07:34:44.834497+0000","last_became_active":"2024-09-21T07:34:23.635444+0000","last_became_peered":"2024-09-21T07:34:23.635444+0000","last_unstale":"2024-09-21T07:34:44.834497+0000","last_undegraded":"2024-09-21T07:34:44.834497+0000","last_fullsized":"2024-09-21T07:34:44.834497+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:46.255385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001952636,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465803+0000","last_change":"2024-09-21T07:34:43.476982+0000","last_active":"2024-09-21T07:34:44.465803+0000","last_peered":"2024-09-21T07:34:44.465803+0000","last_clean":"2024-09-21T07:34:44.465803+0000","last_became_active":"2024-09-21T07:34:43.476670+0000","last_became_peered":"2024-09-21T07:34:43.476670+0000","last_unstale":"2024-09-21T07:34:44.465803+0000","last_undegraded":"2024-09-21T07:34:44.465803+0000","last_fullsized":"2024-09-21T07:34:44.465803+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:04:42.140238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:34:42.450602+0000","last_change":"2024-09-21T07:34:42.444915+0000","last_active":"2024-09-21T07:34:42.450602+0000","last_peered":"2024-09-21T07:34:42.450602+0000","last_clean":"2024-09-21T07:34:42.450602+0000","last_became_active":"2024-09-21T07:34:42.444506+0000","last_became_peered":"2024-09-21T07:34:42.444506+0000","last_unstale":"2024-09-21T07:34:42.450602+0000","last_undegraded":"2024-09-21T07:34:42.450602+0000","last_fullsized":"2024-09-21T07:34:42.450602+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:18.964200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.468302+0000","last_change":"2024-09-21T07:34:24.539152+0000","last_active":"2024-09-21T07:34:44.468302+0000","last_peered":"2024-09-21T07:34:44.468302+0000","last_clean":"2024-09-21T07:34:44.468302+0000","last_became_active":"2024-09-21T07:34:23.259573+0000","last_became_peered":"2024-09-21T07:34:23.259573+0000","last_unstale":"2024-09-21T07:34:44.468302+0000","last_undegraded":"2024-09-21T07:34:44.468302+0000","last_fullsized":"2024-09-21T07:34:44.468302+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:26.986900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058108400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.468175+0000","last_change":"2024-09-21T07:34:43.469411+0000","last_active":"2024-09-21T07:34:44.468175+0000","last_peered":"2024-09-21T07:34:44.468175+0000","last_clean":"2024-09-21T07:34:44.468175+0000","last_became_active":"2024-09-21T07:34:43.469098+0000","last_became_peered":"2024-09-21T07:34:43.469098+0000","last_unstale":"2024-09-21T07:34:44.468175+0000","last_undegraded":"2024-09-21T07:34:44.468175+0000","last_fullsized":"2024-09-21T07:34:44.468175+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:03.654293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"48'2","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834059+0000","last_change":"2024-09-21T07:34:42.444868+0000","last_active":"2024-09-21T07:34:44.834059+0000","last_peered":"2024-09-21T07:34:44.834059+0000","last_clean":"2024-09-21T07:34:44.834059+0000","last_became_active":"2024-09-21T07:34:42.444574+0000","last_became_peered":"2024-09-21T07:34:42.444574+0000","last_unstale":"2024-09-21T07:34:44.834059+0000","last_undegraded":"2024-09-21T07:34:44.834059+0000","last_fullsized":"2024-09-21T07:34:44.834059+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:14.168800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835810+0000","last_change":"2024-09-21T07:34:24.259902+0000","last_active":"2024-09-21T07:34:44.835810+0000","last_peered":"2024-09-21T07:34:44.835810+0000","last_clean":"2024-09-21T07:34:44.835810+0000","last_became_active":"2024-09-21T07:34:23.271902+0000","last_became_peered":"2024-09-21T07:34:23.271902+0000","last_unstale":"2024-09-21T07:34:44.835810+0000","last_undegraded":"2024-09-21T07:34:44.835810+0000","last_fullsized":"2024-09-21T07:34:44.835810+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:59.439525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001686358,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834649+0000","last_change":"2024-09-21T07:34:43.488076+0000","last_active":"2024-09-21T07:34:44.834649+0000","last_peered":"2024-09-21T07:34:44.834649+0000","last_clean":"2024-09-21T07:34:44.834649+0000","last_became_active":"2024-09-21T07:34:43.487739+0000","last_became_peered":"2024-09-21T07:34:43.487739+0000","last_unstale":"2024-09-21T07:34:44.834649+0000","last_undegraded":"2024-09-21T07:34:44.834649+0000","last_fullsized":"2024-09-21T07:34:44.834649+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:35:38.440936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:34:42.454137+0000","last_change":"2024-09-21T07:34:42.452085+0000","last_active":"2024-09-21T07:34:42.454137+0000","last_peered":"2024-09-21T07:34:42.454137+0000","last_clean":"2024-09-21T07:34:42.454137+0000","last_became_active":"2024-09-21T07:34:42.451768+0000","last_became_peered":"2024-09-21T07:34:42.451768+0000","last_unstale":"2024-09-21T07:34:42.454137+0000","last_undegraded":"2024-09-21T07:34:42.454137+0000","last_fullsized":"2024-09-21T07:34:42.454137+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:09:15.578304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":40,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833639+0000","last_change":"2024-09-21T07:34:24.259662+0000","last_active":"2024-09-21T07:34:44.833639+0000","last_peered":"2024-09-21T07:34:44.833639+0000","last_clean":"2024-09-21T07:34:44.833639+0000","last_became_active":"2024-09-21T07:34:23.278812+0000","last_became_peered":"2024-09-21T07:34:23.278812+0000","last_unstale":"2024-09-21T07:34:44.833639+0000","last_undegraded":"2024-09-21T07:34:44.833639+0000","last_fullsized":"2024-09-21T07:34:44.833639+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:49.829974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356463,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835746+0000","last_change":"2024-09-21T07:34:43.470322+0000","last_active":"2024-09-21T07:34:44.835746+0000","last_peered":"2024-09-21T07:34:44.835746+0000","last_clean":"2024-09-21T07:34:44.835746+0000","last_became_active":"2024-09-21T07:34:43.469955+0000","last_became_peered":"2024-09-21T07:34:43.469955+0000","last_unstale":"2024-09-21T07:34:44.835746+0000","last_undegraded":"2024-09-21T07:34:44.835746+0000","last_fullsized":"2024-09-21T07:34:44.835746+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:30.715538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835746+0000","last_change":"2024-09-21T07:34:42.441191+0000","last_active":"2024-09-21T07:34:44.835746+0000","last_peered":"2024-09-21T07:34:44.835746+0000","last_clean":"2024-09-21T07:34:44.835746+0000","last_became_active":"2024-09-21T07:34:42.440881+0000","last_became_peered":"2024-09-21T07:34:42.440881+0000","last_unstale":"2024-09-21T07:34:44.835746+0000","last_undegraded":"2024-09-21T07:34:44.835746+0000","last_fullsized":"2024-09-21T07:34:44.835746+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:45.066904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":38,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833256+0000","last_change":"2024-09-21T07:34:25.084861+0000","last_active":"2024-09-21T07:34:44.833256+0000","last_peered":"2024-09-21T07:34:44.833256+0000","last_clean":"2024-09-21T07:34:44.833256+0000","last_became_active":"2024-09-21T07:34:23.277708+0000","last_became_peered":"2024-09-21T07:34:23.277708+0000","last_unstale":"2024-09-21T07:34:44.833256+0000","last_undegraded":"2024-09-21T07:34:44.833256+0000","last_fullsized":"2024-09-21T07:34:44.833256+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:40.816488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000597566,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835256+0000","last_change":"2024-09-21T07:34:43.480401+0000","last_active":"2024-09-21T07:34:44.835256+0000","last_peered":"2024-09-21T07:34:44.835256+0000","last_clean":"2024-09-21T07:34:44.835256+0000","last_became_active":"2024-09-21T07:34:43.479988+0000","last_became_peered":"2024-09-21T07:34:43.479988+0000","last_unstale":"2024-09-21T07:34:44.835256+0000","last_undegraded":"2024-09-21T07:34:44.835256+0000","last_fullsized":"2024-09-21T07:34:44.835256+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:03.660689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"48'8","reported_seq":23,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834495+0000","last_change":"2024-09-21T07:34:42.449884+0000","last_active":"2024-09-21T07:34:44.834495+0000","last_peered":"2024-09-21T07:34:44.834495+0000","last_clean":"2024-09-21T07:34:44.834495+0000","last_became_active":"2024-09-21T07:34:42.448499+0000","last_became_peered":"2024-09-21T07:34:42.448499+0000","last_unstale":"2024-09-21T07:34:44.834495+0000","last_undegraded":"2024-09-21T07:34:44.834495+0000","last_fullsized":"2024-09-21T07:34:44.834495+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:58.240549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":40,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833217+0000","last_change":"2024-09-21T07:34:24.258709+0000","last_active":"2024-09-21T07:34:44.833217+0000","last_peered":"2024-09-21T07:34:44.833217+0000","last_clean":"2024-09-21T07:34:44.833217+0000","last_became_active":"2024-09-21T07:34:23.279042+0000","last_became_peered":"2024-09-21T07:34:23.279042+0000","last_unstale":"2024-09-21T07:34:44.833217+0000","last_undegraded":"2024-09-21T07:34:44.833217+0000","last_fullsized":"2024-09-21T07:34:44.833217+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:01.762117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019282100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833735+0000","last_change":"2024-09-21T07:34:43.466176+0000","last_active":"2024-09-21T07:34:44.833735+0000","last_peered":"2024-09-21T07:34:44.833735+0000","last_clean":"2024-09-21T07:34:44.833735+0000","last_became_active":"2024-09-21T07:34:43.464214+0000","last_became_peered":"2024-09-21T07:34:43.464214+0000","last_unstale":"2024-09-21T07:34:44.833735+0000","last_undegraded":"2024-09-21T07:34:44.833735+0000","last_fullsized":"2024-09-21T07:34:44.833735+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:22.282533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836035+0000","last_change":"2024-09-21T07:34:42.442308+0000","last_active":"2024-09-21T07:34:44.836035+0000","last_peered":"2024-09-21T07:34:44.836035+0000","last_clean":"2024-09-21T07:34:44.836035+0000","last_became_active":"2024-09-21T07:34:42.442018+0000","last_became_peered":"2024-09-21T07:34:42.442018+0000","last_unstale":"2024-09-21T07:34:44.836035+0000","last_undegraded":"2024-09-21T07:34:44.836035+0000","last_fullsized":"2024-09-21T07:34:44.836035+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:53.642537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":38,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835026+0000","last_change":"2024-09-21T07:34:25.085635+0000","last_active":"2024-09-21T07:34:44.835026+0000","last_peered":"2024-09-21T07:34:44.835026+0000","last_clean":"2024-09-21T07:34:44.835026+0000","last_became_active":"2024-09-21T07:34:23.273566+0000","last_became_peered":"2024-09-21T07:34:23.273566+0000","last_unstale":"2024-09-21T07:34:44.835026+0000","last_undegraded":"2024-09-21T07:34:44.835026+0000","last_fullsized":"2024-09-21T07:34:44.835026+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:14.142272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079897300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467666+0000","last_change":"2024-09-21T07:34:43.472808+0000","last_active":"2024-09-21T07:34:44.467666+0000","last_peered":"2024-09-21T07:34:44.467666+0000","last_clean":"2024-09-21T07:34:44.467666+0000","last_became_active":"2024-09-21T07:34:43.466752+0000","last_became_peered":"2024-09-21T07:34:43.466752+0000","last_unstale":"2024-09-21T07:34:44.467666+0000","last_undegraded":"2024-09-21T07:34:44.467666+0000","last_fullsized":"2024-09-21T07:34:44.467666+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:19:06.280633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833887+0000","last_change":"2024-09-21T07:34:42.447266+0000","last_active":"2024-09-21T07:34:44.833887+0000","last_peered":"2024-09-21T07:34:44.833887+0000","last_clean":"2024-09-21T07:34:44.833887+0000","last_became_active":"2024-09-21T07:34:42.446823+0000","last_became_peered":"2024-09-21T07:34:42.446823+0000","last_unstale":"2024-09-21T07:34:44.833887+0000","last_undegraded":"2024-09-21T07:34:44.833887+0000","last_fullsized":"2024-09-21T07:34:44.833887+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:16.281729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835166+0000","last_change":"2024-09-21T07:34:24.597926+0000","last_active":"2024-09-21T07:34:44.835166+0000","last_peered":"2024-09-21T07:34:44.835166+0000","last_clean":"2024-09-21T07:34:44.835166+0000","last_became_active":"2024-09-21T07:34:23.268926+0000","last_became_peered":"2024-09-21T07:34:23.268926+0000","last_unstale":"2024-09-21T07:34:44.835166+0000","last_undegraded":"2024-09-21T07:34:44.835166+0000","last_fullsized":"2024-09-21T07:34:44.835166+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:46.190752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101851,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834479+0000","last_change":"2024-09-21T07:34:43.481754+0000","last_active":"2024-09-21T07:34:44.834479+0000","last_peered":"2024-09-21T07:34:44.834479+0000","last_clean":"2024-09-21T07:34:44.834479+0000","last_became_active":"2024-09-21T07:34:43.481460+0000","last_became_peered":"2024-09-21T07:34:43.481460+0000","last_unstale":"2024-09-21T07:34:44.834479+0000","last_undegraded":"2024-09-21T07:34:44.834479+0000","last_fullsized":"2024-09-21T07:34:44.834479+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:17:36.213769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"48'3","reported_seq":18,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836119+0000","last_change":"2024-09-21T07:34:42.450507+0000","last_active":"2024-09-21T07:34:44.836119+0000","last_peered":"2024-09-21T07:34:44.836119+0000","last_clean":"2024-09-21T07:34:44.836119+0000","last_became_active":"2024-09-21T07:34:42.449613+0000","last_became_peered":"2024-09-21T07:34:42.449613+0000","last_unstale":"2024-09-21T07:34:44.836119+0000","last_undegraded":"2024-09-21T07:34:44.836119+0000","last_fullsized":"2024-09-21T07:34:44.836119+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:59:52.817419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":38,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.466765+0000","last_change":"2024-09-21T07:34:24.259794+0000","last_active":"2024-09-21T07:34:44.466765+0000","last_peered":"2024-09-21T07:34:44.466765+0000","last_clean":"2024-09-21T07:34:44.466765+0000","last_became_active":"2024-09-21T07:34:23.265679+0000","last_became_peered":"2024-09-21T07:34:23.265679+0000","last_unstale":"2024-09-21T07:34:44.466765+0000","last_undegraded":"2024-09-21T07:34:44.466765+0000","last_fullsized":"2024-09-21T07:34:44.466765+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:02.575796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001298518,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834659+0000","last_change":"2024-09-21T07:34:43.457653+0000","last_active":"2024-09-21T07:34:44.834659+0000","last_peered":"2024-09-21T07:34:44.834659+0000","last_clean":"2024-09-21T07:34:44.834659+0000","last_became_active":"2024-09-21T07:34:43.457347+0000","last_became_peered":"2024-09-21T07:34:43.457347+0000","last_unstale":"2024-09-21T07:34:44.834659+0000","last_undegraded":"2024-09-21T07:34:44.834659+0000","last_fullsized":"2024-09-21T07:34:44.834659+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:02:38.909862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"48'6","reported_seq":21,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834377+0000","last_change":"2024-09-21T07:34:42.449115+0000","last_active":"2024-09-21T07:34:44.834377+0000","last_peered":"2024-09-21T07:34:44.834377+0000","last_clean":"2024-09-21T07:34:44.834377+0000","last_became_active":"2024-09-21T07:34:42.448834+0000","last_became_peered":"2024-09-21T07:34:42.448834+0000","last_unstale":"2024-09-21T07:34:44.834377+0000","last_undegraded":"2024-09-21T07:34:44.834377+0000","last_fullsized":"2024-09-21T07:34:44.834377+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:49.800427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836377+0000","last_change":"2024-09-21T07:34:24.260330+0000","last_active":"2024-09-21T07:34:44.836377+0000","last_peered":"2024-09-21T07:34:44.836377+0000","last_clean":"2024-09-21T07:34:44.836377+0000","last_became_active":"2024-09-21T07:34:23.637210+0000","last_became_peered":"2024-09-21T07:34:23.637210+0000","last_unstale":"2024-09-21T07:34:44.836377+0000","last_undegraded":"2024-09-21T07:34:44.836377+0000","last_fullsized":"2024-09-21T07:34:44.836377+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.488451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001537093,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834186+0000","last_change":"2024-09-21T07:34:43.481741+0000","last_active":"2024-09-21T07:34:44.834186+0000","last_peered":"2024-09-21T07:34:44.834186+0000","last_clean":"2024-09-21T07:34:44.834186+0000","last_became_active":"2024-09-21T07:34:43.481368+0000","last_became_peered":"2024-09-21T07:34:43.481368+0000","last_unstale":"2024-09-21T07:34:44.834186+0000","last_undegraded":"2024-09-21T07:34:44.834186+0000","last_fullsized":"2024-09-21T07:34:44.834186+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:16.752080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"48'1","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833850+0000","last_change":"2024-09-21T07:34:42.455980+0000","last_active":"2024-09-21T07:34:44.833850+0000","last_peered":"2024-09-21T07:34:44.833850+0000","last_clean":"2024-09-21T07:34:44.833850+0000","last_became_active":"2024-09-21T07:34:42.455368+0000","last_became_peered":"2024-09-21T07:34:42.455368+0000","last_unstale":"2024-09-21T07:34:44.833850+0000","last_undegraded":"2024-09-21T07:34:44.833850+0000","last_fullsized":"2024-09-21T07:34:44.833850+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:07:34.444440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833846+0000","last_change":"2024-09-21T07:34:24.597286+0000","last_active":"2024-09-21T07:34:44.833846+0000","last_peered":"2024-09-21T07:34:44.833846+0000","last_clean":"2024-09-21T07:34:44.833846+0000","last_became_active":"2024-09-21T07:34:23.267133+0000","last_became_peered":"2024-09-21T07:34:23.267133+0000","last_unstale":"2024-09-21T07:34:44.833846+0000","last_undegraded":"2024-09-21T07:34:44.833846+0000","last_fullsized":"2024-09-21T07:34:44.833846+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:18.257010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836565+0000","last_change":"2024-09-21T07:34:43.476520+0000","last_active":"2024-09-21T07:34:44.836565+0000","last_peered":"2024-09-21T07:34:44.836565+0000","last_clean":"2024-09-21T07:34:44.836565+0000","last_became_active":"2024-09-21T07:34:43.476159+0000","last_became_peered":"2024-09-21T07:34:43.476159+0000","last_unstale":"2024-09-21T07:34:44.836565+0000","last_undegraded":"2024-09-21T07:34:44.836565+0000","last_fullsized":"2024-09-21T07:34:44.836565+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:27.525343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"48'1","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833695+0000","last_change":"2024-09-21T07:34:42.441002+0000","last_active":"2024-09-21T07:34:44.833695+0000","last_peered":"2024-09-21T07:34:44.833695+0000","last_clean":"2024-09-21T07:34:44.833695+0000","last_became_active":"2024-09-21T07:34:42.440702+0000","last_became_peered":"2024-09-21T07:34:42.440702+0000","last_unstale":"2024-09-21T07:34:44.833695+0000","last_undegraded":"2024-09-21T07:34:44.833695+0000","last_fullsized":"2024-09-21T07:34:44.833695+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:09.841490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":40,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833536+0000","last_change":"2024-09-21T07:34:24.258712+0000","last_active":"2024-09-21T07:34:44.833536+0000","last_peered":"2024-09-21T07:34:44.833536+0000","last_clean":"2024-09-21T07:34:44.833536+0000","last_became_active":"2024-09-21T07:34:23.274692+0000","last_became_peered":"2024-09-21T07:34:23.274692+0000","last_unstale":"2024-09-21T07:34:44.833536+0000","last_undegraded":"2024-09-21T07:34:44.833536+0000","last_fullsized":"2024-09-21T07:34:44.833536+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:43.279454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018051479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835466+0000","last_change":"2024-09-21T07:34:43.463236+0000","last_active":"2024-09-21T07:34:44.835466+0000","last_peered":"2024-09-21T07:34:44.835466+0000","last_clean":"2024-09-21T07:34:44.835466+0000","last_became_active":"2024-09-21T07:34:43.458917+0000","last_became_peered":"2024-09-21T07:34:43.458917+0000","last_unstale":"2024-09-21T07:34:44.835466+0000","last_undegraded":"2024-09-21T07:34:44.835466+0000","last_fullsized":"2024-09-21T07:34:44.835466+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:45:19.428476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:34:42.453151+0000","last_change":"2024-09-21T07:34:42.447422+0000","last_active":"2024-09-21T07:34:42.453151+0000","last_peered":"2024-09-21T07:34:42.453151+0000","last_clean":"2024-09-21T07:34:42.453151+0000","last_became_active":"2024-09-21T07:34:42.447132+0000","last_became_peered":"2024-09-21T07:34:42.447132+0000","last_unstale":"2024-09-21T07:34:42.453151+0000","last_undegraded":"2024-09-21T07:34:42.453151+0000","last_fullsized":"2024-09-21T07:34:42.453151+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:11.994352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":38,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833146+0000","last_change":"2024-09-21T07:34:25.084849+0000","last_active":"2024-09-21T07:34:44.833146+0000","last_peered":"2024-09-21T07:34:44.833146+0000","last_clean":"2024-09-21T07:34:44.833146+0000","last_became_active":"2024-09-21T07:34:23.277713+0000","last_became_peered":"2024-09-21T07:34:23.277713+0000","last_unstale":"2024-09-21T07:34:44.833146+0000","last_undegraded":"2024-09-21T07:34:44.833146+0000","last_fullsized":"2024-09-21T07:34:44.833146+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:05.811810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059079499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836091+0000","last_change":"2024-09-21T07:34:43.472697+0000","last_active":"2024-09-21T07:34:44.836091+0000","last_peered":"2024-09-21T07:34:44.836091+0000","last_clean":"2024-09-21T07:34:44.836091+0000","last_became_active":"2024-09-21T07:34:43.472276+0000","last_became_peered":"2024-09-21T07:34:43.472276+0000","last_unstale":"2024-09-21T07:34:44.836091+0000","last_undegraded":"2024-09-21T07:34:44.836091+0000","last_fullsized":"2024-09-21T07:34:44.836091+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:49.596333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833099+0000","last_change":"2024-09-21T07:34:42.453339+0000","last_active":"2024-09-21T07:34:44.833099+0000","last_peered":"2024-09-21T07:34:44.833099+0000","last_clean":"2024-09-21T07:34:44.833099+0000","last_became_active":"2024-09-21T07:34:42.452974+0000","last_became_peered":"2024-09-21T07:34:42.452974+0000","last_unstale":"2024-09-21T07:34:44.833099+0000","last_undegraded":"2024-09-21T07:34:44.833099+0000","last_fullsized":"2024-09-21T07:34:44.833099+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:22.788726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467897+0000","last_change":"2024-09-21T07:34:24.538610+0000","last_active":"2024-09-21T07:34:44.467897+0000","last_peered":"2024-09-21T07:34:44.467897+0000","last_clean":"2024-09-21T07:34:44.467897+0000","last_became_active":"2024-09-21T07:34:23.259700+0000","last_became_peered":"2024-09-21T07:34:23.259700+0000","last_unstale":"2024-09-21T07:34:44.467897+0000","last_undegraded":"2024-09-21T07:34:44.467897+0000","last_fullsized":"2024-09-21T07:34:44.467897+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:57.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070522399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834793+0000","last_change":"2024-09-21T07:34:43.474381+0000","last_active":"2024-09-21T07:34:44.834793+0000","last_peered":"2024-09-21T07:34:44.834793+0000","last_clean":"2024-09-21T07:34:44.834793+0000","last_became_active":"2024-09-21T07:34:43.473953+0000","last_became_peered":"2024-09-21T07:34:43.473953+0000","last_unstale":"2024-09-21T07:34:44.834793+0000","last_undegraded":"2024-09-21T07:34:44.834793+0000","last_fullsized":"2024-09-21T07:34:44.834793+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:31.362212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:34:42.450606+0000","last_change":"2024-09-21T07:34:42.444572+0000","last_active":"2024-09-21T07:34:42.450606+0000","last_peered":"2024-09-21T07:34:42.450606+0000","last_clean":"2024-09-21T07:34:42.450606+0000","last_became_active":"2024-09-21T07:34:42.444254+0000","last_became_peered":"2024-09-21T07:34:42.444254+0000","last_unstale":"2024-09-21T07:34:42.450606+0000","last_undegraded":"2024-09-21T07:34:42.450606+0000","last_fullsized":"2024-09-21T07:34:42.450606+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:08.176679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835079+0000","last_change":"2024-09-21T07:34:24.597597+0000","last_active":"2024-09-21T07:34:44.835079+0000","last_peered":"2024-09-21T07:34:44.835079+0000","last_clean":"2024-09-21T07:34:44.835079+0000","last_became_active":"2024-09-21T07:34:23.635204+0000","last_became_peered":"2024-09-21T07:34:23.635204+0000","last_unstale":"2024-09-21T07:34:44.835079+0000","last_undegraded":"2024-09-21T07:34:44.835079+0000","last_fullsized":"2024-09-21T07:34:44.835079+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:48.644789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081139199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467728+0000","last_change":"2024-09-21T07:34:43.460645+0000","last_active":"2024-09-21T07:34:44.467728+0000","last_peered":"2024-09-21T07:34:44.467728+0000","last_clean":"2024-09-21T07:34:44.467728+0000","last_became_active":"2024-09-21T07:34:43.460320+0000","last_became_peered":"2024-09-21T07:34:43.460320+0000","last_unstale":"2024-09-21T07:34:44.467728+0000","last_undegraded":"2024-09-21T07:34:44.467728+0000","last_fullsized":"2024-09-21T07:34:44.467728+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:54.644949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"48'1","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465954+0000","last_change":"2024-09-21T07:34:42.446762+0000","last_active":"2024-09-21T07:34:44.465954+0000","last_peered":"2024-09-21T07:34:44.465954+0000","last_clean":"2024-09-21T07:34:44.465954+0000","last_became_active":"2024-09-21T07:34:42.446320+0000","last_became_peered":"2024-09-21T07:34:42.446320+0000","last_unstale":"2024-09-21T07:34:44.465954+0000","last_undegraded":"2024-09-21T07:34:44.465954+0000","last_fullsized":"2024-09-21T07:34:44.465954+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:46.161972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":40,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835380+0000","last_change":"2024-09-21T07:34:24.259026+0000","last_active":"2024-09-21T07:34:44.835380+0000","last_peered":"2024-09-21T07:34:44.835380+0000","last_clean":"2024-09-21T07:34:44.835380+0000","last_became_active":"2024-09-21T07:34:23.635456+0000","last_became_peered":"2024-09-21T07:34:23.635456+0000","last_unstale":"2024-09-21T07:34:44.835380+0000","last_undegraded":"2024-09-21T07:34:44.835380+0000","last_fullsized":"2024-09-21T07:34:44.835380+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:09.592752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018654419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-21T07:34:42.434916+0000","last_change":"2024-09-21T07:34:42.434916+0000","last_active":"2024-09-21T07:34:42.415959+0000","last_peered":"2024-09-21T07:34:42.415959+0000","last_clean":"2024-09-21T07:34:42.415959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:42.434916+0000","last_undegraded":"2024-09-21T07:34:42.434916+0000","last_fullsized":"2024-09-21T07:34:42.434916+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"48'2","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467319+0000","last_change":"2024-09-21T07:34:42.443581+0000","last_active":"2024-09-21T07:34:44.467319+0000","last_peered":"2024-09-21T07:34:44.467319+0000","last_clean":"2024-09-21T07:34:44.467319+0000","last_became_active":"2024-09-21T07:34:42.443284+0000","last_became_peered":"2024-09-21T07:34:42.443284+0000","last_unstale":"2024-09-21T07:34:44.467319+0000","last_undegraded":"2024-09-21T07:34:44.467319+0000","last_fullsized":"2024-09-21T07:34:44.467319+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:59:14.692524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835966+0000","last_change":"2024-09-21T07:34:24.260300+0000","last_active":"2024-09-21T07:34:44.835966+0000","last_peered":"2024-09-21T07:34:44.835966+0000","last_clean":"2024-09-21T07:34:44.835966+0000","last_became_active":"2024-09-21T07:34:23.270128+0000","last_became_peered":"2024-09-21T07:34:23.270128+0000","last_unstale":"2024-09-21T07:34:44.835966+0000","last_undegraded":"2024-09-21T07:34:44.835966+0000","last_fullsized":"2024-09-21T07:34:44.835966+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:25.260931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00176651,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833363+0000","last_change":"2024-09-21T07:34:43.457202+0000","last_active":"2024-09-21T07:34:44.833363+0000","last_peered":"2024-09-21T07:34:44.833363+0000","last_clean":"2024-09-21T07:34:44.833363+0000","last_became_active":"2024-09-21T07:34:43.456851+0000","last_became_peered":"2024-09-21T07:34:43.456851+0000","last_unstale":"2024-09-21T07:34:44.833363+0000","last_undegraded":"2024-09-21T07:34:44.833363+0000","last_fullsized":"2024-09-21T07:34:44.833363+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:12.056767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"48'2","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.468447+0000","last_change":"2024-09-21T07:34:42.434252+0000","last_active":"2024-09-21T07:34:44.468447+0000","last_peered":"2024-09-21T07:34:44.468447+0000","last_clean":"2024-09-21T07:34:44.468447+0000","last_became_active":"2024-09-21T07:34:42.433936+0000","last_became_peered":"2024-09-21T07:34:42.433936+0000","last_unstale":"2024-09-21T07:34:44.468447+0000","last_undegraded":"2024-09-21T07:34:44.468447+0000","last_fullsized":"2024-09-21T07:34:44.468447+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:13.202719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467474+0000","last_change":"2024-09-21T07:34:25.028904+0000","last_active":"2024-09-21T07:34:44.467474+0000","last_peered":"2024-09-21T07:34:44.467474+0000","last_clean":"2024-09-21T07:34:44.467474+0000","last_became_active":"2024-09-21T07:34:23.260942+0000","last_became_peered":"2024-09-21T07:34:23.260942+0000","last_unstale":"2024-09-21T07:34:44.467474+0000","last_undegraded":"2024-09-21T07:34:44.467474+0000","last_fullsized":"2024-09-21T07:34:44.467474+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:22.709328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060553899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":1758,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509464,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32332,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339508,"statfs":{"total":96636764160,"available":96603656192,"internally_reserved":0,"allocated":1171456,"data_stored":832323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509465,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":532480,"data_stored":221422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509464,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":569344,"data_stored":241831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509465,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31764,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340076,"statfs":{"total":96636764160,"available":96604237824,"internally_reserved":0,"allocated":581632,"data_stored":243443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509466,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31756,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340084,"statfs":{"total":96636764160,"available":96604246016,"internally_reserved":0,"allocated":577536,"data_stored":241991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509466,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1171456,"data_stored":832255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542169,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31748,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340092,"statfs":{"total":96636764160,"available":96604254208,"internally_reserved":0,"allocated":573440,"data_stored":242413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542168,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31760,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340080,"statfs":{"total":96636764160,"available":96604241920,"internally_reserved":0,"allocated":585728,"data_stored":243465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:34:47.961 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:34:48.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.221+0000 7f6a03200640 1 -- 172.21.15.39:0/1811660916 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a040cf540 msgr2=0x7f6a0406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:48.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.221+0000 7f6a03200640 1 --2- 172.21.15.39:0/1811660916 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a040cf540 0x7f6a0406ee70 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f69ec005fd0 tx=0x7f69ec02a490 comp rx=0 tx=0).stop 2024-09-21T07:34:48.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.221+0000 7f6a03200640 1 -- 172.21.15.39:0/1811660916 shutdown_connections 2024-09-21T07:34:48.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.221+0000 7f6a03200640 1 --2- 172.21.15.39:0/1811660916 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a040cf950 0x7f6a0406f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:48.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.221+0000 7f6a03200640 1 --2- 172.21.15.39:0/1811660916 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a040cf540 0x7f6a0406ee70 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:48.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.221+0000 7f6a03200640 1 -- 172.21.15.39:0/1811660916 >> 172.21.15.39:0/1811660916 conn(0x7f6a0406d410 msgr2=0x7f6a040b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:48.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.221+0000 7f6a03200640 1 -- 172.21.15.39:0/1811660916 shutdown_connections 2024-09-21T07:34:48.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.222+0000 7f6a03200640 1 -- 172.21.15.39:0/1811660916 wait complete. 2024-09-21T07:34:48.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.224+0000 7f6a03200640 1 Processor -- start 2024-09-21T07:34:48.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.224+0000 7f6a03200640 1 -- start start 2024-09-21T07:34:48.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.225+0000 7f6a03200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a040cf540 0x7f6a0413e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:48.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.225+0000 7f6a03200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a040cf950 0x7f6a041f9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:48.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.225+0000 7f6a03200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a0415ca50 0x7f6a041f95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:48.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.225+0000 7f6a03200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0415dde0 con 0x7f6a0415ca50 2024-09-21T07:34:48.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.225+0000 7f6a03200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0415dc60 con 0x7f6a040cf540 2024-09-21T07:34:48.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.225+0000 7f6a03200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f6a0415df60 con 0x7f6a040cf950 2024-09-21T07:34:48.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.226+0000 7f6a02800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a0415ca50 0x7f6a041f95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:48.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.226+0000 7f6a02800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a0415ca50 0x7f6a041f95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57710/0 (socket says 172.21.15.39:57710) 2024-09-21T07:34:48.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.226+0000 7f6a02800640 1 -- 172.21.15.39:0/1259017719 learned_addr learned my addr 172.21.15.39:0/1259017719 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:48.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.226+0000 7f6a01e00640 1 --2- 172.21.15.39:0/1259017719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a040cf540 0x7f6a0413e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:48.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.226+0000 7f69fbe00640 1 --2- 172.21.15.39:0/1259017719 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a040cf950 0x7f6a041f9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:48.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.226+0000 7f6a02800640 1 -- 172.21.15.39:0/1259017719 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a040cf950 msgr2=0x7f6a041f9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:48.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.226+0000 7f6a02800640 1 --2- 172.21.15.39:0/1259017719 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a040cf950 0x7f6a041f9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:48.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.226+0000 7f6a02800640 1 -- 172.21.15.39:0/1259017719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a040cf540 msgr2=0x7f6a0413e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:48.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.226+0000 7f6a02800640 1 --2- 172.21.15.39:0/1259017719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a040cf540 0x7f6a0413e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:48.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.226+0000 7f6a02800640 1 -- 172.21.15.39:0/1259017719 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a04157a70 con 0x7f6a0415ca50 2024-09-21T07:34:48.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.226+0000 7f6a02800640 1 --2- 172.21.15.39:0/1259017719 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a0415ca50 0x7f6a041f95a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f69fc00cab0 tx=0x7f69fc00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:48.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.227+0000 7f69fa000640 1 -- 172.21.15.39:0/1259017719 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc00ae20 con 0x7f6a0415ca50 2024-09-21T07:34:48.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.227+0000 7f69fa000640 1 -- 172.21.15.39:0/1259017719 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69fc003a60 con 0x7f6a0415ca50 2024-09-21T07:34:48.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.227+0000 7f6a03200640 1 -- 172.21.15.39:0/1259017719 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a04156a00 con 0x7f6a0415ca50 2024-09-21T07:34:48.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.227+0000 7f69fa000640 1 -- 172.21.15.39:0/1259017719 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc004030 con 0x7f6a0415ca50 2024-09-21T07:34:48.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.227+0000 7f6a03200640 1 -- 172.21.15.39:0/1259017719 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a04157ff0 con 0x7f6a0415ca50 2024-09-21T07:34:48.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.228+0000 7f6a03200640 1 -- 172.21.15.39:0/1259017719 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a0415e410 con 0x7f6a0415ca50 2024-09-21T07:34:48.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.229+0000 7f69fa000640 1 -- 172.21.15.39:0/1259017719 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f69fc0041d0 con 0x7f6a0415ca50 2024-09-21T07:34:48.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.230+0000 7f69fa000640 1 --2- 172.21.15.39:0/1259017719 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f69e0078b90 0x7f69e007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:48.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.230+0000 7f69fa000640 1 -- 172.21.15.39:0/1259017719 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6062+0+0 (secure 0 0 0) 0x7f69fc09b720 con 0x7f6a0415ca50 2024-09-21T07:34:48.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.230+0000 7f6a01e00640 1 --2- 172.21.15.39:0/1259017719 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f69e0078b90 0x7f69e007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:48.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.231+0000 7f6a01e00640 1 --2- 172.21.15.39:0/1259017719 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f69e0078b90 0x7f69e007b0a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f69f4006fd0 tx=0x7f69f4009060 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:48.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.234+0000 7f69fa000640 1 -- 172.21.15.39:0/1259017719 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f69fc014070 con 0x7f6a0415ca50 2024-09-21T07:34:48.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.429+0000 7f6a03200640 1 -- 172.21.15.39:0/1259017719 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6a0415f1f0 con 0x7f69e0078b90 2024-09-21T07:34:48.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.436+0000 7f69fa000640 1 -- 172.21.15.39:0/1259017719 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332429 (secure 0 0 0) 0x7f6a0415f1f0 con 0x7f69e0078b90 2024-09-21T07:34:48.440 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:34:48.442 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:34:48.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.441+0000 7f6a03200640 1 -- 172.21.15.39:0/1259017719 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f69e0078b90 msgr2=0x7f69e007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:48.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.441+0000 7f6a03200640 1 --2- 172.21.15.39:0/1259017719 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f69e0078b90 0x7f69e007b0a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f69f4006fd0 tx=0x7f69f4009060 comp rx=0 tx=0).stop 2024-09-21T07:34:48.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.441+0000 7f6a03200640 1 -- 172.21.15.39:0/1259017719 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a0415ca50 msgr2=0x7f6a041f95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:48.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.441+0000 7f6a03200640 1 --2- 172.21.15.39:0/1259017719 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a0415ca50 0x7f6a041f95a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f69fc00cab0 tx=0x7f69fc00cf70 comp rx=0 tx=0).stop 2024-09-21T07:34:48.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.441+0000 7f6a03200640 1 -- 172.21.15.39:0/1259017719 shutdown_connections 2024-09-21T07:34:48.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.441+0000 7f6a03200640 1 --2- 172.21.15.39:0/1259017719 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f69e0078b90 0x7f69e007b0a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:48.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.441+0000 7f6a03200640 1 --2- 172.21.15.39:0/1259017719 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a0415ca50 0x7f6a041f95a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:48.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.441+0000 7f6a03200640 1 --2- 172.21.15.39:0/1259017719 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a040cf950 0x7f6a041f9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:48.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.441+0000 7f6a03200640 1 --2- 172.21.15.39:0/1259017719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a040cf540 0x7f6a0413e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:48.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.441+0000 7f6a03200640 1 -- 172.21.15.39:0/1259017719 >> 172.21.15.39:0/1259017719 conn(0x7f6a0406d410 msgr2=0x7f6a0415a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:48.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.441+0000 7f6a03200640 1 -- 172.21.15.39:0/1259017719 shutdown_connections 2024-09-21T07:34:48.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.441+0000 7f6a03200640 1 -- 172.21.15.39:0/1259017719 wait complete. 2024-09-21T07:34:48.463 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2024-09-21T07:34:47.556632+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592126,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":138,"num_write_kb":1640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":198,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":246848,"kb_used_data":5628,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754727872,"statfs":{"total":773094113280,"available":772841340928,"internally_reserved":0,"allocated":5763072,"data_stored":3099143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1758,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001569"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.466557+0000","last_change":"2024-09-21T07:34:43.473741+0000","last_active":"2024-09-21T07:34:44.466557+0000","last_peered":"2024-09-21T07:34:44.466557+0000","last_clean":"2024-09-21T07:34:44.466557+0000","last_became_active":"2024-09-21T07:34:43.473447+0000","last_became_peered":"2024-09-21T07:34:43.473447+0000","last_unstale":"2024-09-21T07:34:44.466557+0000","last_undegraded":"2024-09-21T07:34:44.466557+0000","last_fullsized":"2024-09-21T07:34:44.466557+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:00:23.404297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836408+0000","last_change":"2024-09-21T07:34:43.473362+0000","last_active":"2024-09-21T07:34:44.836408+0000","last_peered":"2024-09-21T07:34:44.836408+0000","last_clean":"2024-09-21T07:34:44.836408+0000","last_became_active":"2024-09-21T07:34:43.473083+0000","last_became_peered":"2024-09-21T07:34:43.473083+0000","last_unstale":"2024-09-21T07:34:44.836408+0000","last_undegraded":"2024-09-21T07:34:44.836408+0000","last_fullsized":"2024-09-21T07:34:44.836408+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:33.920441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834020+0000","last_change":"2024-09-21T07:34:43.481499+0000","last_active":"2024-09-21T07:34:44.834020+0000","last_peered":"2024-09-21T07:34:44.834020+0000","last_clean":"2024-09-21T07:34:44.834020+0000","last_became_active":"2024-09-21T07:34:43.478254+0000","last_became_peered":"2024-09-21T07:34:43.478254+0000","last_unstale":"2024-09-21T07:34:44.834020+0000","last_undegraded":"2024-09-21T07:34:44.834020+0000","last_fullsized":"2024-09-21T07:34:44.834020+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:03.654293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836742+0000","last_change":"2024-09-21T07:34:43.451459+0000","last_active":"2024-09-21T07:34:44.836742+0000","last_peered":"2024-09-21T07:34:44.836742+0000","last_clean":"2024-09-21T07:34:44.836742+0000","last_became_active":"2024-09-21T07:34:43.451121+0000","last_became_peered":"2024-09-21T07:34:43.451121+0000","last_unstale":"2024-09-21T07:34:44.836742+0000","last_undegraded":"2024-09-21T07:34:44.836742+0000","last_fullsized":"2024-09-21T07:34:44.836742+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:15.139818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834482+0000","last_change":"2024-09-21T07:34:43.455772+0000","last_active":"2024-09-21T07:34:44.834482+0000","last_peered":"2024-09-21T07:34:44.834482+0000","last_clean":"2024-09-21T07:34:44.834482+0000","last_became_active":"2024-09-21T07:34:43.454461+0000","last_became_peered":"2024-09-21T07:34:43.454461+0000","last_unstale":"2024-09-21T07:34:44.834482+0000","last_undegraded":"2024-09-21T07:34:44.834482+0000","last_fullsized":"2024-09-21T07:34:44.834482+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:03.456938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465593+0000","last_change":"2024-09-21T07:34:43.469533+0000","last_active":"2024-09-21T07:34:44.465593+0000","last_peered":"2024-09-21T07:34:44.465593+0000","last_clean":"2024-09-21T07:34:44.465593+0000","last_became_active":"2024-09-21T07:34:43.469336+0000","last_became_peered":"2024-09-21T07:34:43.469336+0000","last_unstale":"2024-09-21T07:34:44.465593+0000","last_undegraded":"2024-09-21T07:34:44.465593+0000","last_fullsized":"2024-09-21T07:34:44.465593+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:20.540854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833543+0000","last_change":"2024-09-21T07:34:43.483800+0000","last_active":"2024-09-21T07:34:44.833543+0000","last_peered":"2024-09-21T07:34:44.833543+0000","last_clean":"2024-09-21T07:34:44.833543+0000","last_became_active":"2024-09-21T07:34:43.483482+0000","last_became_peered":"2024-09-21T07:34:43.483482+0000","last_unstale":"2024-09-21T07:34:44.833543+0000","last_undegraded":"2024-09-21T07:34:44.833543+0000","last_fullsized":"2024-09-21T07:34:44.833543+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:20.152451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834422+0000","last_change":"2024-09-21T07:34:43.473653+0000","last_active":"2024-09-21T07:34:44.834422+0000","last_peered":"2024-09-21T07:34:44.834422+0000","last_clean":"2024-09-21T07:34:44.834422+0000","last_became_active":"2024-09-21T07:34:43.473348+0000","last_became_peered":"2024-09-21T07:34:43.473348+0000","last_unstale":"2024-09-21T07:34:44.834422+0000","last_undegraded":"2024-09-21T07:34:44.834422+0000","last_fullsized":"2024-09-21T07:34:44.834422+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:13.042429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835096+0000","last_change":"2024-09-21T07:34:43.483507+0000","last_active":"2024-09-21T07:34:44.835096+0000","last_peered":"2024-09-21T07:34:44.835096+0000","last_clean":"2024-09-21T07:34:44.835096+0000","last_became_active":"2024-09-21T07:34:43.482198+0000","last_became_peered":"2024-09-21T07:34:43.482198+0000","last_unstale":"2024-09-21T07:34:44.835096+0000","last_undegraded":"2024-09-21T07:34:44.835096+0000","last_fullsized":"2024-09-21T07:34:44.835096+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:45.957857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465790+0000","last_change":"2024-09-21T07:34:43.465258+0000","last_active":"2024-09-21T07:34:44.465790+0000","last_peered":"2024-09-21T07:34:44.465790+0000","last_clean":"2024-09-21T07:34:44.465790+0000","last_became_active":"2024-09-21T07:34:43.464922+0000","last_became_peered":"2024-09-21T07:34:43.464922+0000","last_unstale":"2024-09-21T07:34:44.465790+0000","last_undegraded":"2024-09-21T07:34:44.465790+0000","last_fullsized":"2024-09-21T07:34:44.465790+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:11.406345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467028+0000","last_change":"2024-09-21T07:34:43.449879+0000","last_active":"2024-09-21T07:34:44.467028+0000","last_peered":"2024-09-21T07:34:44.467028+0000","last_clean":"2024-09-21T07:34:44.467028+0000","last_became_active":"2024-09-21T07:34:43.448365+0000","last_became_peered":"2024-09-21T07:34:43.448365+0000","last_unstale":"2024-09-21T07:34:44.467028+0000","last_undegraded":"2024-09-21T07:34:44.467028+0000","last_fullsized":"2024-09-21T07:34:44.467028+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:29:00.993972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834222+0000","last_change":"2024-09-21T07:34:43.460167+0000","last_active":"2024-09-21T07:34:44.834222+0000","last_peered":"2024-09-21T07:34:44.834222+0000","last_clean":"2024-09-21T07:34:44.834222+0000","last_became_active":"2024-09-21T07:34:43.457296+0000","last_became_peered":"2024-09-21T07:34:43.457296+0000","last_unstale":"2024-09-21T07:34:44.834222+0000","last_undegraded":"2024-09-21T07:34:44.834222+0000","last_fullsized":"2024-09-21T07:34:44.834222+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:53:57.871505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.466129+0000","last_change":"2024-09-21T07:34:43.465519+0000","last_active":"2024-09-21T07:34:44.466129+0000","last_peered":"2024-09-21T07:34:44.466129+0000","last_clean":"2024-09-21T07:34:44.466129+0000","last_became_active":"2024-09-21T07:34:43.464802+0000","last_became_peered":"2024-09-21T07:34:43.464802+0000","last_unstale":"2024-09-21T07:34:44.466129+0000","last_undegraded":"2024-09-21T07:34:44.466129+0000","last_fullsized":"2024-09-21T07:34:44.466129+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:59.062262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834033+0000","last_change":"2024-09-21T07:34:43.478575+0000","last_active":"2024-09-21T07:34:44.834033+0000","last_peered":"2024-09-21T07:34:44.834033+0000","last_clean":"2024-09-21T07:34:44.834033+0000","last_became_active":"2024-09-21T07:34:43.478187+0000","last_became_peered":"2024-09-21T07:34:43.478187+0000","last_unstale":"2024-09-21T07:34:44.834033+0000","last_undegraded":"2024-09-21T07:34:44.834033+0000","last_fullsized":"2024-09-21T07:34:44.834033+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:44.866369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836848+0000","last_change":"2024-09-21T07:34:43.467093+0000","last_active":"2024-09-21T07:34:44.836848+0000","last_peered":"2024-09-21T07:34:44.836848+0000","last_clean":"2024-09-21T07:34:44.836848+0000","last_became_active":"2024-09-21T07:34:43.466773+0000","last_became_peered":"2024-09-21T07:34:43.466773+0000","last_unstale":"2024-09-21T07:34:44.836848+0000","last_undegraded":"2024-09-21T07:34:44.836848+0000","last_fullsized":"2024-09-21T07:34:44.836848+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:12:02.180192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465260+0000","last_change":"2024-09-21T07:34:43.462637+0000","last_active":"2024-09-21T07:34:44.465260+0000","last_peered":"2024-09-21T07:34:44.465260+0000","last_clean":"2024-09-21T07:34:44.465260+0000","last_became_active":"2024-09-21T07:34:43.462301+0000","last_became_peered":"2024-09-21T07:34:43.462301+0000","last_unstale":"2024-09-21T07:34:44.465260+0000","last_undegraded":"2024-09-21T07:34:44.465260+0000","last_fullsized":"2024-09-21T07:34:44.465260+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:42.168665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834898+0000","last_change":"2024-09-21T07:34:43.484342+0000","last_active":"2024-09-21T07:34:44.834898+0000","last_peered":"2024-09-21T07:34:44.834898+0000","last_clean":"2024-09-21T07:34:44.834898+0000","last_became_active":"2024-09-21T07:34:43.480452+0000","last_became_peered":"2024-09-21T07:34:43.480452+0000","last_unstale":"2024-09-21T07:34:44.834898+0000","last_undegraded":"2024-09-21T07:34:44.834898+0000","last_fullsized":"2024-09-21T07:34:44.834898+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:30.267103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836590+0000","last_change":"2024-09-21T07:34:43.472890+0000","last_active":"2024-09-21T07:34:44.836590+0000","last_peered":"2024-09-21T07:34:44.836590+0000","last_clean":"2024-09-21T07:34:44.836590+0000","last_became_active":"2024-09-21T07:34:43.472576+0000","last_became_peered":"2024-09-21T07:34:43.472576+0000","last_unstale":"2024-09-21T07:34:44.836590+0000","last_undegraded":"2024-09-21T07:34:44.836590+0000","last_fullsized":"2024-09-21T07:34:44.836590+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:24.719441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834303+0000","last_change":"2024-09-21T07:34:43.476774+0000","last_active":"2024-09-21T07:34:44.834303+0000","last_peered":"2024-09-21T07:34:44.834303+0000","last_clean":"2024-09-21T07:34:44.834303+0000","last_became_active":"2024-09-21T07:34:43.473147+0000","last_became_peered":"2024-09-21T07:34:43.473147+0000","last_unstale":"2024-09-21T07:34:44.834303+0000","last_undegraded":"2024-09-21T07:34:44.834303+0000","last_fullsized":"2024-09-21T07:34:44.834303+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:10.661941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-21T07:34:42.434570+0000","last_change":"2024-09-21T07:34:42.434570+0000","last_active":"2024-09-21T07:34:42.415959+0000","last_peered":"2024-09-21T07:34:42.415959+0000","last_clean":"2024-09-21T07:34:42.415959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:42.434570+0000","last_undegraded":"2024-09-21T07:34:42.434570+0000","last_fullsized":"2024-09-21T07:34:42.434570+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-21T07:34:42.434971+0000","last_change":"2024-09-21T07:34:42.434971+0000","last_active":"2024-09-21T07:34:42.415959+0000","last_peered":"2024-09-21T07:34:42.415959+0000","last_clean":"2024-09-21T07:34:42.415959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:42.434971+0000","last_undegraded":"2024-09-21T07:34:42.434971+0000","last_fullsized":"2024-09-21T07:34:42.434971+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834766+0000","last_change":"2024-09-21T07:34:43.483312+0000","last_active":"2024-09-21T07:34:44.834766+0000","last_peered":"2024-09-21T07:34:44.834766+0000","last_clean":"2024-09-21T07:34:44.834766+0000","last_became_active":"2024-09-21T07:34:43.483003+0000","last_became_peered":"2024-09-21T07:34:43.483003+0000","last_unstale":"2024-09-21T07:34:44.834766+0000","last_undegraded":"2024-09-21T07:34:44.834766+0000","last_fullsized":"2024-09-21T07:34:44.834766+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:33.782633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467544+0000","last_change":"2024-09-21T07:34:43.449273+0000","last_active":"2024-09-21T07:34:44.467544+0000","last_peered":"2024-09-21T07:34:44.467544+0000","last_clean":"2024-09-21T07:34:44.467544+0000","last_became_active":"2024-09-21T07:34:43.448944+0000","last_became_peered":"2024-09-21T07:34:43.448944+0000","last_unstale":"2024-09-21T07:34:44.467544+0000","last_undegraded":"2024-09-21T07:34:44.467544+0000","last_fullsized":"2024-09-21T07:34:44.467544+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:56.540403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833398+0000","last_change":"2024-09-21T07:34:43.479357+0000","last_active":"2024-09-21T07:34:44.833398+0000","last_peered":"2024-09-21T07:34:44.833398+0000","last_clean":"2024-09-21T07:34:44.833398+0000","last_became_active":"2024-09-21T07:34:43.478997+0000","last_became_peered":"2024-09-21T07:34:43.478997+0000","last_unstale":"2024-09-21T07:34:44.833398+0000","last_undegraded":"2024-09-21T07:34:44.833398+0000","last_fullsized":"2024-09-21T07:34:44.833398+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:54.644949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835486+0000","last_change":"2024-09-21T07:34:43.486869+0000","last_active":"2024-09-21T07:34:44.835486+0000","last_peered":"2024-09-21T07:34:44.835486+0000","last_clean":"2024-09-21T07:34:44.835486+0000","last_became_active":"2024-09-21T07:34:43.486351+0000","last_became_peered":"2024-09-21T07:34:43.486351+0000","last_unstale":"2024-09-21T07:34:44.835486+0000","last_undegraded":"2024-09-21T07:34:44.835486+0000","last_fullsized":"2024-09-21T07:34:44.835486+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:21:02.910848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833742+0000","last_change":"2024-09-21T07:34:43.463196+0000","last_active":"2024-09-21T07:34:44.833742+0000","last_peered":"2024-09-21T07:34:44.833742+0000","last_clean":"2024-09-21T07:34:44.833742+0000","last_became_active":"2024-09-21T07:34:43.458559+0000","last_became_peered":"2024-09-21T07:34:43.458559+0000","last_unstale":"2024-09-21T07:34:44.833742+0000","last_undegraded":"2024-09-21T07:34:44.833742+0000","last_fullsized":"2024-09-21T07:34:44.833742+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:37.233429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467280+0000","last_change":"2024-09-21T07:34:43.483549+0000","last_active":"2024-09-21T07:34:44.467280+0000","last_peered":"2024-09-21T07:34:44.467280+0000","last_clean":"2024-09-21T07:34:44.467280+0000","last_became_active":"2024-09-21T07:34:43.472653+0000","last_became_peered":"2024-09-21T07:34:43.472653+0000","last_unstale":"2024-09-21T07:34:44.467280+0000","last_undegraded":"2024-09-21T07:34:44.467280+0000","last_fullsized":"2024-09-21T07:34:44.467280+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:38.814916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-21T07:34:42.435611+0000","last_change":"2024-09-21T07:34:42.435611+0000","last_active":"2024-09-21T07:34:42.415959+0000","last_peered":"2024-09-21T07:34:42.415959+0000","last_clean":"2024-09-21T07:34:42.415959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:42.435611+0000","last_undegraded":"2024-09-21T07:34:42.435611+0000","last_fullsized":"2024-09-21T07:34:42.435611+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833338+0000","last_change":"2024-09-21T07:34:43.489357+0000","last_active":"2024-09-21T07:34:44.833338+0000","last_peered":"2024-09-21T07:34:44.833338+0000","last_clean":"2024-09-21T07:34:44.833338+0000","last_became_active":"2024-09-21T07:34:43.483224+0000","last_became_peered":"2024-09-21T07:34:43.483224+0000","last_unstale":"2024-09-21T07:34:44.833338+0000","last_undegraded":"2024-09-21T07:34:44.833338+0000","last_fullsized":"2024-09-21T07:34:44.833338+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:05.212534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835873+0000","last_change":"2024-09-21T07:34:43.460376+0000","last_active":"2024-09-21T07:34:44.835873+0000","last_peered":"2024-09-21T07:34:44.835873+0000","last_clean":"2024-09-21T07:34:44.835873+0000","last_became_active":"2024-09-21T07:34:43.458320+0000","last_became_peered":"2024-09-21T07:34:43.458320+0000","last_unstale":"2024-09-21T07:34:44.835873+0000","last_undegraded":"2024-09-21T07:34:44.835873+0000","last_fullsized":"2024-09-21T07:34:44.835873+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:35.190652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833745+0000","last_change":"2024-09-21T07:34:43.485349+0000","last_active":"2024-09-21T07:34:44.833745+0000","last_peered":"2024-09-21T07:34:44.833745+0000","last_clean":"2024-09-21T07:34:44.833745+0000","last_became_active":"2024-09-21T07:34:43.483623+0000","last_became_peered":"2024-09-21T07:34:43.483623+0000","last_unstale":"2024-09-21T07:34:44.833745+0000","last_undegraded":"2024-09-21T07:34:44.833745+0000","last_fullsized":"2024-09-21T07:34:44.833745+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:02.084719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467658+0000","last_change":"2024-09-21T07:34:43.471970+0000","last_active":"2024-09-21T07:34:44.467658+0000","last_peered":"2024-09-21T07:34:44.467658+0000","last_clean":"2024-09-21T07:34:44.467658+0000","last_became_active":"2024-09-21T07:34:43.465515+0000","last_became_peered":"2024-09-21T07:34:43.465515+0000","last_unstale":"2024-09-21T07:34:44.467658+0000","last_undegraded":"2024-09-21T07:34:44.467658+0000","last_fullsized":"2024-09-21T07:34:44.467658+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:59:53.819831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-21T07:34:42.435086+0000","last_change":"2024-09-21T07:34:42.435086+0000","last_active":"2024-09-21T07:34:42.415959+0000","last_peered":"2024-09-21T07:34:42.415959+0000","last_clean":"2024-09-21T07:34:42.415959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:42.435086+0000","last_undegraded":"2024-09-21T07:34:42.435086+0000","last_fullsized":"2024-09-21T07:34:42.435086+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"49'1","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834973+0000","last_change":"2024-09-21T07:34:42.447496+0000","last_active":"2024-09-21T07:34:44.834973+0000","last_peered":"2024-09-21T07:34:44.834973+0000","last_clean":"2024-09-21T07:34:44.834973+0000","last_became_active":"2024-09-21T07:34:42.447148+0000","last_became_peered":"2024-09-21T07:34:42.447148+0000","last_unstale":"2024-09-21T07:34:44.834973+0000","last_undegraded":"2024-09-21T07:34:44.834973+0000","last_fullsized":"2024-09-21T07:34:44.834973+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:42:06.750595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":163,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835508+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:34:44.835508+0000","last_peered":"2024-09-21T07:34:44.835508+0000","last_clean":"2024-09-21T07:34:44.835508+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:34:44.835508+0000","last_undegraded":"2024-09-21T07:34:44.835508+0000","last_fullsized":"2024-09-21T07:34:44.835508+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.468769+0000","last_change":"2024-09-21T07:34:24.539248+0000","last_active":"2024-09-21T07:34:44.468769+0000","last_peered":"2024-09-21T07:34:44.468769+0000","last_clean":"2024-09-21T07:34:44.468769+0000","last_became_active":"2024-09-21T07:34:23.273465+0000","last_became_peered":"2024-09-21T07:34:23.273465+0000","last_unstale":"2024-09-21T07:34:44.468769+0000","last_undegraded":"2024-09-21T07:34:44.468769+0000","last_fullsized":"2024-09-21T07:34:44.468769+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:55.517452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060627700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834170+0000","last_change":"2024-09-21T07:34:43.491609+0000","last_active":"2024-09-21T07:34:44.834170+0000","last_peered":"2024-09-21T07:34:44.834170+0000","last_clean":"2024-09-21T07:34:44.834170+0000","last_became_active":"2024-09-21T07:34:43.490138+0000","last_became_peered":"2024-09-21T07:34:43.490138+0000","last_unstale":"2024-09-21T07:34:44.834170+0000","last_undegraded":"2024-09-21T07:34:44.834170+0000","last_fullsized":"2024-09-21T07:34:44.834170+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:16.015947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833429+0000","last_change":"2024-09-21T07:34:42.448815+0000","last_active":"2024-09-21T07:34:44.833429+0000","last_peered":"2024-09-21T07:34:44.833429+0000","last_clean":"2024-09-21T07:34:44.833429+0000","last_became_active":"2024-09-21T07:34:42.448484+0000","last_became_peered":"2024-09-21T07:34:42.448484+0000","last_unstale":"2024-09-21T07:34:44.833429+0000","last_undegraded":"2024-09-21T07:34:44.833429+0000","last_fullsized":"2024-09-21T07:34:44.833429+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:15.522307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833978+0000","last_change":"2024-09-21T07:34:24.597274+0000","last_active":"2024-09-21T07:34:44.833978+0000","last_peered":"2024-09-21T07:34:44.833978+0000","last_clean":"2024-09-21T07:34:44.833978+0000","last_became_active":"2024-09-21T07:34:23.279560+0000","last_became_peered":"2024-09-21T07:34:23.279560+0000","last_unstale":"2024-09-21T07:34:44.833978+0000","last_undegraded":"2024-09-21T07:34:44.833978+0000","last_fullsized":"2024-09-21T07:34:44.833978+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:15.845265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059699799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.468039+0000","last_change":"2024-09-21T07:34:43.468683+0000","last_active":"2024-09-21T07:34:44.468039+0000","last_peered":"2024-09-21T07:34:44.468039+0000","last_clean":"2024-09-21T07:34:44.468039+0000","last_became_active":"2024-09-21T07:34:43.468377+0000","last_became_peered":"2024-09-21T07:34:43.468377+0000","last_unstale":"2024-09-21T07:34:44.468039+0000","last_undegraded":"2024-09-21T07:34:44.468039+0000","last_fullsized":"2024-09-21T07:34:44.468039+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:02.084719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834944+0000","last_change":"2024-09-21T07:34:42.442148+0000","last_active":"2024-09-21T07:34:44.834944+0000","last_peered":"2024-09-21T07:34:44.834944+0000","last_clean":"2024-09-21T07:34:44.834944+0000","last_became_active":"2024-09-21T07:34:42.441839+0000","last_became_peered":"2024-09-21T07:34:42.441839+0000","last_unstale":"2024-09-21T07:34:44.834944+0000","last_undegraded":"2024-09-21T07:34:44.834944+0000","last_fullsized":"2024-09-21T07:34:44.834944+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:35:57.065813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836188+0000","last_change":"2024-09-21T07:34:24.259906+0000","last_active":"2024-09-21T07:34:44.836188+0000","last_peered":"2024-09-21T07:34:44.836188+0000","last_clean":"2024-09-21T07:34:44.836188+0000","last_became_active":"2024-09-21T07:34:23.635282+0000","last_became_peered":"2024-09-21T07:34:23.635282+0000","last_unstale":"2024-09-21T07:34:44.836188+0000","last_undegraded":"2024-09-21T07:34:44.836188+0000","last_fullsized":"2024-09-21T07:34:44.836188+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:00.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001527664,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834033+0000","last_change":"2024-09-21T07:34:43.491274+0000","last_active":"2024-09-21T07:34:44.834033+0000","last_peered":"2024-09-21T07:34:44.834033+0000","last_clean":"2024-09-21T07:34:44.834033+0000","last_became_active":"2024-09-21T07:34:43.489063+0000","last_became_peered":"2024-09-21T07:34:43.489063+0000","last_unstale":"2024-09-21T07:34:44.834033+0000","last_undegraded":"2024-09-21T07:34:44.834033+0000","last_fullsized":"2024-09-21T07:34:44.834033+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:13.315568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833450+0000","last_change":"2024-09-21T07:34:42.437481+0000","last_active":"2024-09-21T07:34:44.833450+0000","last_peered":"2024-09-21T07:34:44.833450+0000","last_clean":"2024-09-21T07:34:44.833450+0000","last_became_active":"2024-09-21T07:34:42.437130+0000","last_became_peered":"2024-09-21T07:34:42.437130+0000","last_unstale":"2024-09-21T07:34:44.833450+0000","last_undegraded":"2024-09-21T07:34:44.833450+0000","last_fullsized":"2024-09-21T07:34:44.833450+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:55.611339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833360+0000","last_change":"2024-09-21T07:34:25.085454+0000","last_active":"2024-09-21T07:34:44.833360+0000","last_peered":"2024-09-21T07:34:44.833360+0000","last_clean":"2024-09-21T07:34:44.833360+0000","last_became_active":"2024-09-21T07:34:23.279205+0000","last_became_peered":"2024-09-21T07:34:23.279205+0000","last_unstale":"2024-09-21T07:34:44.833360+0000","last_undegraded":"2024-09-21T07:34:44.833360+0000","last_fullsized":"2024-09-21T07:34:44.833360+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:54.231666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095320599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835309+0000","last_change":"2024-09-21T07:34:43.457980+0000","last_active":"2024-09-21T07:34:44.835309+0000","last_peered":"2024-09-21T07:34:44.835309+0000","last_clean":"2024-09-21T07:34:44.835309+0000","last_became_active":"2024-09-21T07:34:43.457695+0000","last_became_peered":"2024-09-21T07:34:43.457695+0000","last_unstale":"2024-09-21T07:34:44.835309+0000","last_undegraded":"2024-09-21T07:34:44.835309+0000","last_fullsized":"2024-09-21T07:34:44.835309+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:01.492292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467870+0000","last_change":"2024-09-21T07:34:42.438107+0000","last_active":"2024-09-21T07:34:44.467870+0000","last_peered":"2024-09-21T07:34:44.467870+0000","last_clean":"2024-09-21T07:34:44.467870+0000","last_became_active":"2024-09-21T07:34:42.437794+0000","last_became_peered":"2024-09-21T07:34:42.437794+0000","last_unstale":"2024-09-21T07:34:44.467870+0000","last_undegraded":"2024-09-21T07:34:44.467870+0000","last_fullsized":"2024-09-21T07:34:44.467870+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:24.649223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465509+0000","last_change":"2024-09-21T07:34:25.028521+0000","last_active":"2024-09-21T07:34:44.465509+0000","last_peered":"2024-09-21T07:34:44.465509+0000","last_clean":"2024-09-21T07:34:44.465509+0000","last_became_active":"2024-09-21T07:34:23.635467+0000","last_became_peered":"2024-09-21T07:34:23.635467+0000","last_unstale":"2024-09-21T07:34:44.465509+0000","last_undegraded":"2024-09-21T07:34:44.465509+0000","last_fullsized":"2024-09-21T07:34:44.465509+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:33.729123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000660558,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467665+0000","last_change":"2024-09-21T07:34:43.465578+0000","last_active":"2024-09-21T07:34:44.467665+0000","last_peered":"2024-09-21T07:34:44.467665+0000","last_clean":"2024-09-21T07:34:44.467665+0000","last_became_active":"2024-09-21T07:34:43.465071+0000","last_became_peered":"2024-09-21T07:34:43.465071+0000","last_unstale":"2024-09-21T07:34:44.467665+0000","last_undegraded":"2024-09-21T07:34:44.467665+0000","last_fullsized":"2024-09-21T07:34:44.467665+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:20.152451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:34:42.453134+0000","last_change":"2024-09-21T07:34:42.445927+0000","last_active":"2024-09-21T07:34:42.453134+0000","last_peered":"2024-09-21T07:34:42.453134+0000","last_clean":"2024-09-21T07:34:42.453134+0000","last_became_active":"2024-09-21T07:34:42.445620+0000","last_became_peered":"2024-09-21T07:34:42.445620+0000","last_unstale":"2024-09-21T07:34:42.453134+0000","last_undegraded":"2024-09-21T07:34:42.453134+0000","last_fullsized":"2024-09-21T07:34:42.453134+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:19.691881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":40,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835533+0000","last_change":"2024-09-21T07:34:24.260046+0000","last_active":"2024-09-21T07:34:44.835533+0000","last_peered":"2024-09-21T07:34:44.835533+0000","last_clean":"2024-09-21T07:34:44.835533+0000","last_became_active":"2024-09-21T07:34:23.273431+0000","last_became_peered":"2024-09-21T07:34:23.273431+0000","last_unstale":"2024-09-21T07:34:44.835533+0000","last_undegraded":"2024-09-21T07:34:44.835533+0000","last_fullsized":"2024-09-21T07:34:44.835533+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:18.876057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019803310000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465631+0000","last_change":"2024-09-21T07:34:43.462550+0000","last_active":"2024-09-21T07:34:44.465631+0000","last_peered":"2024-09-21T07:34:44.465631+0000","last_clean":"2024-09-21T07:34:44.465631+0000","last_became_active":"2024-09-21T07:34:43.462210+0000","last_became_peered":"2024-09-21T07:34:43.462210+0000","last_unstale":"2024-09-21T07:34:44.465631+0000","last_undegraded":"2024-09-21T07:34:44.465631+0000","last_fullsized":"2024-09-21T07:34:44.465631+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:56.480314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836267+0000","last_change":"2024-09-21T07:34:42.452761+0000","last_active":"2024-09-21T07:34:44.836267+0000","last_peered":"2024-09-21T07:34:44.836267+0000","last_clean":"2024-09-21T07:34:44.836267+0000","last_became_active":"2024-09-21T07:34:42.452400+0000","last_became_peered":"2024-09-21T07:34:42.452400+0000","last_unstale":"2024-09-21T07:34:44.836267+0000","last_undegraded":"2024-09-21T07:34:44.836267+0000","last_fullsized":"2024-09-21T07:34:44.836267+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:19:05.278221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465673+0000","last_change":"2024-09-21T07:34:25.028640+0000","last_active":"2024-09-21T07:34:44.465673+0000","last_peered":"2024-09-21T07:34:44.465673+0000","last_clean":"2024-09-21T07:34:44.465673+0000","last_became_active":"2024-09-21T07:34:23.260950+0000","last_became_peered":"2024-09-21T07:34:23.260950+0000","last_unstale":"2024-09-21T07:34:44.465673+0000","last_undegraded":"2024-09-21T07:34:44.465673+0000","last_fullsized":"2024-09-21T07:34:44.465673+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:23:42.121398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055135099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833638+0000","last_change":"2024-09-21T07:34:43.453996+0000","last_active":"2024-09-21T07:34:44.833638+0000","last_peered":"2024-09-21T07:34:44.833638+0000","last_clean":"2024-09-21T07:34:44.833638+0000","last_became_active":"2024-09-21T07:34:43.453666+0000","last_became_peered":"2024-09-21T07:34:43.453666+0000","last_unstale":"2024-09-21T07:34:44.833638+0000","last_undegraded":"2024-09-21T07:34:44.833638+0000","last_fullsized":"2024-09-21T07:34:44.833638+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:31.445807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834426+0000","last_change":"2024-09-21T07:34:42.453822+0000","last_active":"2024-09-21T07:34:44.834426+0000","last_peered":"2024-09-21T07:34:44.834426+0000","last_clean":"2024-09-21T07:34:44.834426+0000","last_became_active":"2024-09-21T07:34:42.453499+0000","last_became_peered":"2024-09-21T07:34:42.453499+0000","last_unstale":"2024-09-21T07:34:44.834426+0000","last_undegraded":"2024-09-21T07:34:44.834426+0000","last_fullsized":"2024-09-21T07:34:44.834426+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:39:29.733953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836757+0000","last_change":"2024-09-21T07:34:24.260665+0000","last_active":"2024-09-21T07:34:44.836757+0000","last_peered":"2024-09-21T07:34:44.836757+0000","last_clean":"2024-09-21T07:34:44.836757+0000","last_became_active":"2024-09-21T07:34:23.635909+0000","last_became_peered":"2024-09-21T07:34:23.635909+0000","last_unstale":"2024-09-21T07:34:44.836757+0000","last_undegraded":"2024-09-21T07:34:44.836757+0000","last_fullsized":"2024-09-21T07:34:44.836757+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:44.536885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.468489+0000","last_change":"2024-09-21T07:34:43.469198+0000","last_active":"2024-09-21T07:34:44.468489+0000","last_peered":"2024-09-21T07:34:44.468489+0000","last_clean":"2024-09-21T07:34:44.468489+0000","last_became_active":"2024-09-21T07:34:43.467622+0000","last_became_peered":"2024-09-21T07:34:43.467622+0000","last_unstale":"2024-09-21T07:34:44.468489+0000","last_undegraded":"2024-09-21T07:34:44.468489+0000","last_fullsized":"2024-09-21T07:34:44.468489+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:44.866369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"48'2","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465974+0000","last_change":"2024-09-21T07:34:42.438031+0000","last_active":"2024-09-21T07:34:44.465974+0000","last_peered":"2024-09-21T07:34:44.465974+0000","last_clean":"2024-09-21T07:34:44.465974+0000","last_became_active":"2024-09-21T07:34:42.436986+0000","last_became_peered":"2024-09-21T07:34:42.436986+0000","last_unstale":"2024-09-21T07:34:44.465974+0000","last_undegraded":"2024-09-21T07:34:44.465974+0000","last_fullsized":"2024-09-21T07:34:44.465974+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:51.213865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467759+0000","last_change":"2024-09-21T07:34:24.538845+0000","last_active":"2024-09-21T07:34:44.467759+0000","last_peered":"2024-09-21T07:34:44.467759+0000","last_clean":"2024-09-21T07:34:44.467759+0000","last_became_active":"2024-09-21T07:34:23.636372+0000","last_became_peered":"2024-09-21T07:34:23.636372+0000","last_unstale":"2024-09-21T07:34:44.467759+0000","last_undegraded":"2024-09-21T07:34:44.467759+0000","last_fullsized":"2024-09-21T07:34:44.467759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:29.393813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841451,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-21T07:34:42.436882+0000","last_change":"2024-09-21T07:34:42.436882+0000","last_active":"2024-09-21T07:34:42.415959+0000","last_peered":"2024-09-21T07:34:42.415959+0000","last_clean":"2024-09-21T07:34:42.415959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:42.436882+0000","last_undegraded":"2024-09-21T07:34:42.436882+0000","last_fullsized":"2024-09-21T07:34:42.436882+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834840+0000","last_change":"2024-09-21T07:34:42.447519+0000","last_active":"2024-09-21T07:34:44.834840+0000","last_peered":"2024-09-21T07:34:44.834840+0000","last_clean":"2024-09-21T07:34:44.834840+0000","last_became_active":"2024-09-21T07:34:42.447129+0000","last_became_peered":"2024-09-21T07:34:42.447129+0000","last_unstale":"2024-09-21T07:34:44.834840+0000","last_undegraded":"2024-09-21T07:34:44.834840+0000","last_fullsized":"2024-09-21T07:34:44.834840+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:30.464417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":40,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833703+0000","last_change":"2024-09-21T07:34:24.259686+0000","last_active":"2024-09-21T07:34:44.833703+0000","last_peered":"2024-09-21T07:34:44.833703+0000","last_clean":"2024-09-21T07:34:44.833703+0000","last_became_active":"2024-09-21T07:34:23.274405+0000","last_became_peered":"2024-09-21T07:34:23.274405+0000","last_unstale":"2024-09-21T07:34:44.833703+0000","last_undegraded":"2024-09-21T07:34:44.833703+0000","last_fullsized":"2024-09-21T07:34:44.833703+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:26.304693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022009920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834760+0000","last_change":"2024-09-21T07:34:43.476482+0000","last_active":"2024-09-21T07:34:44.834760+0000","last_peered":"2024-09-21T07:34:44.834760+0000","last_clean":"2024-09-21T07:34:44.834760+0000","last_became_active":"2024-09-21T07:34:43.476189+0000","last_became_peered":"2024-09-21T07:34:43.476189+0000","last_unstale":"2024-09-21T07:34:44.834760+0000","last_undegraded":"2024-09-21T07:34:44.834760+0000","last_fullsized":"2024-09-21T07:34:44.834760+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:08:42.688308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833369+0000","last_change":"2024-09-21T07:34:42.456020+0000","last_active":"2024-09-21T07:34:44.833369+0000","last_peered":"2024-09-21T07:34:44.833369+0000","last_clean":"2024-09-21T07:34:44.833369+0000","last_became_active":"2024-09-21T07:34:42.455399+0000","last_became_peered":"2024-09-21T07:34:42.455399+0000","last_unstale":"2024-09-21T07:34:44.833369+0000","last_undegraded":"2024-09-21T07:34:44.833369+0000","last_fullsized":"2024-09-21T07:34:44.833369+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:57.601710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835457+0000","last_change":"2024-09-21T07:34:24.258754+0000","last_active":"2024-09-21T07:34:44.835457+0000","last_peered":"2024-09-21T07:34:44.835457+0000","last_clean":"2024-09-21T07:34:44.835457+0000","last_became_active":"2024-09-21T07:34:23.272657+0000","last_became_peered":"2024-09-21T07:34:23.272657+0000","last_unstale":"2024-09-21T07:34:44.835457+0000","last_undegraded":"2024-09-21T07:34:44.835457+0000","last_fullsized":"2024-09-21T07:34:44.835457+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:05.477099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067143200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.466771+0000","last_change":"2024-09-21T07:34:43.472869+0000","last_active":"2024-09-21T07:34:44.466771+0000","last_peered":"2024-09-21T07:34:44.466771+0000","last_clean":"2024-09-21T07:34:44.466771+0000","last_became_active":"2024-09-21T07:34:43.472514+0000","last_became_peered":"2024-09-21T07:34:43.472514+0000","last_unstale":"2024-09-21T07:34:44.466771+0000","last_undegraded":"2024-09-21T07:34:44.466771+0000","last_fullsized":"2024-09-21T07:34:44.466771+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:15:22.567958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.466329+0000","last_change":"2024-09-21T07:34:42.443208+0000","last_active":"2024-09-21T07:34:44.466329+0000","last_peered":"2024-09-21T07:34:44.466329+0000","last_clean":"2024-09-21T07:34:44.466329+0000","last_became_active":"2024-09-21T07:34:42.442891+0000","last_became_peered":"2024-09-21T07:34:42.442891+0000","last_unstale":"2024-09-21T07:34:44.466329+0000","last_undegraded":"2024-09-21T07:34:44.466329+0000","last_fullsized":"2024-09-21T07:34:44.466329+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:02.347260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":38,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833706+0000","last_change":"2024-09-21T07:34:25.085232+0000","last_active":"2024-09-21T07:34:44.833706+0000","last_peered":"2024-09-21T07:34:44.833706+0000","last_clean":"2024-09-21T07:34:44.833706+0000","last_became_active":"2024-09-21T07:34:23.277807+0000","last_became_peered":"2024-09-21T07:34:23.277807+0000","last_unstale":"2024-09-21T07:34:44.833706+0000","last_undegraded":"2024-09-21T07:34:44.833706+0000","last_fullsized":"2024-09-21T07:34:44.833706+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:36.362919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086619099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835299+0000","last_change":"2024-09-21T07:34:43.486460+0000","last_active":"2024-09-21T07:34:44.835299+0000","last_peered":"2024-09-21T07:34:44.835299+0000","last_clean":"2024-09-21T07:34:44.835299+0000","last_became_active":"2024-09-21T07:34:43.486148+0000","last_became_peered":"2024-09-21T07:34:43.486148+0000","last_unstale":"2024-09-21T07:34:44.835299+0000","last_undegraded":"2024-09-21T07:34:44.835299+0000","last_fullsized":"2024-09-21T07:34:44.835299+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:31.997216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"48'1","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835624+0000","last_change":"2024-09-21T07:34:42.450738+0000","last_active":"2024-09-21T07:34:44.835624+0000","last_peered":"2024-09-21T07:34:44.835624+0000","last_clean":"2024-09-21T07:34:44.835624+0000","last_became_active":"2024-09-21T07:34:42.450413+0000","last_became_peered":"2024-09-21T07:34:42.450413+0000","last_unstale":"2024-09-21T07:34:44.835624+0000","last_undegraded":"2024-09-21T07:34:44.835624+0000","last_fullsized":"2024-09-21T07:34:44.835624+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:48.569857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835043+0000","last_change":"2024-09-21T07:34:24.597672+0000","last_active":"2024-09-21T07:34:44.835043+0000","last_peered":"2024-09-21T07:34:44.835043+0000","last_clean":"2024-09-21T07:34:44.835043+0000","last_became_active":"2024-09-21T07:34:23.267770+0000","last_became_peered":"2024-09-21T07:34:23.267770+0000","last_unstale":"2024-09-21T07:34:44.835043+0000","last_undegraded":"2024-09-21T07:34:44.835043+0000","last_fullsized":"2024-09-21T07:34:44.835043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:51.629476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086378500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465282+0000","last_change":"2024-09-21T07:34:43.478411+0000","last_active":"2024-09-21T07:34:44.465282+0000","last_peered":"2024-09-21T07:34:44.465282+0000","last_clean":"2024-09-21T07:34:44.465282+0000","last_became_active":"2024-09-21T07:34:43.478106+0000","last_became_peered":"2024-09-21T07:34:43.478106+0000","last_unstale":"2024-09-21T07:34:44.465282+0000","last_undegraded":"2024-09-21T07:34:44.465282+0000","last_fullsized":"2024-09-21T07:34:44.465282+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:41.320870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"48'2","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467059+0000","last_change":"2024-09-21T07:34:42.449630+0000","last_active":"2024-09-21T07:34:44.467059+0000","last_peered":"2024-09-21T07:34:44.467059+0000","last_clean":"2024-09-21T07:34:44.467059+0000","last_became_active":"2024-09-21T07:34:42.448013+0000","last_became_peered":"2024-09-21T07:34:42.448013+0000","last_unstale":"2024-09-21T07:34:44.467059+0000","last_undegraded":"2024-09-21T07:34:44.467059+0000","last_fullsized":"2024-09-21T07:34:44.467059+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:42.900618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":38,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467361+0000","last_change":"2024-09-21T07:34:24.260181+0000","last_active":"2024-09-21T07:34:44.467361+0000","last_peered":"2024-09-21T07:34:44.467361+0000","last_clean":"2024-09-21T07:34:44.467361+0000","last_became_active":"2024-09-21T07:34:23.275580+0000","last_became_peered":"2024-09-21T07:34:23.275580+0000","last_unstale":"2024-09-21T07:34:44.467361+0000","last_undegraded":"2024-09-21T07:34:44.467361+0000","last_fullsized":"2024-09-21T07:34:44.467361+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:49.857622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014706980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834317+0000","last_change":"2024-09-21T07:34:43.479707+0000","last_active":"2024-09-21T07:34:44.834317+0000","last_peered":"2024-09-21T07:34:44.834317+0000","last_clean":"2024-09-21T07:34:44.834317+0000","last_became_active":"2024-09-21T07:34:43.479311+0000","last_became_peered":"2024-09-21T07:34:43.479311+0000","last_unstale":"2024-09-21T07:34:44.834317+0000","last_undegraded":"2024-09-21T07:34:44.834317+0000","last_fullsized":"2024-09-21T07:34:44.834317+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:03:54.452233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:34:42.460152+0000","last_change":"2024-09-21T07:34:42.455379+0000","last_active":"2024-09-21T07:34:42.460152+0000","last_peered":"2024-09-21T07:34:42.460152+0000","last_clean":"2024-09-21T07:34:42.460152+0000","last_became_active":"2024-09-21T07:34:42.455144+0000","last_became_peered":"2024-09-21T07:34:42.455144+0000","last_unstale":"2024-09-21T07:34:42.460152+0000","last_undegraded":"2024-09-21T07:34:42.460152+0000","last_fullsized":"2024-09-21T07:34:42.460152+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:31:47.280896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":40,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834497+0000","last_change":"2024-09-21T07:34:24.259112+0000","last_active":"2024-09-21T07:34:44.834497+0000","last_peered":"2024-09-21T07:34:44.834497+0000","last_clean":"2024-09-21T07:34:44.834497+0000","last_became_active":"2024-09-21T07:34:23.635444+0000","last_became_peered":"2024-09-21T07:34:23.635444+0000","last_unstale":"2024-09-21T07:34:44.834497+0000","last_undegraded":"2024-09-21T07:34:44.834497+0000","last_fullsized":"2024-09-21T07:34:44.834497+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:46.255385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001952636,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465803+0000","last_change":"2024-09-21T07:34:43.476982+0000","last_active":"2024-09-21T07:34:44.465803+0000","last_peered":"2024-09-21T07:34:44.465803+0000","last_clean":"2024-09-21T07:34:44.465803+0000","last_became_active":"2024-09-21T07:34:43.476670+0000","last_became_peered":"2024-09-21T07:34:43.476670+0000","last_unstale":"2024-09-21T07:34:44.465803+0000","last_undegraded":"2024-09-21T07:34:44.465803+0000","last_fullsized":"2024-09-21T07:34:44.465803+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:04:42.140238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:34:42.450602+0000","last_change":"2024-09-21T07:34:42.444915+0000","last_active":"2024-09-21T07:34:42.450602+0000","last_peered":"2024-09-21T07:34:42.450602+0000","last_clean":"2024-09-21T07:34:42.450602+0000","last_became_active":"2024-09-21T07:34:42.444506+0000","last_became_peered":"2024-09-21T07:34:42.444506+0000","last_unstale":"2024-09-21T07:34:42.450602+0000","last_undegraded":"2024-09-21T07:34:42.450602+0000","last_fullsized":"2024-09-21T07:34:42.450602+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:18.964200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.468302+0000","last_change":"2024-09-21T07:34:24.539152+0000","last_active":"2024-09-21T07:34:44.468302+0000","last_peered":"2024-09-21T07:34:44.468302+0000","last_clean":"2024-09-21T07:34:44.468302+0000","last_became_active":"2024-09-21T07:34:23.259573+0000","last_became_peered":"2024-09-21T07:34:23.259573+0000","last_unstale":"2024-09-21T07:34:44.468302+0000","last_undegraded":"2024-09-21T07:34:44.468302+0000","last_fullsized":"2024-09-21T07:34:44.468302+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:26.986900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058108400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.468175+0000","last_change":"2024-09-21T07:34:43.469411+0000","last_active":"2024-09-21T07:34:44.468175+0000","last_peered":"2024-09-21T07:34:44.468175+0000","last_clean":"2024-09-21T07:34:44.468175+0000","last_became_active":"2024-09-21T07:34:43.469098+0000","last_became_peered":"2024-09-21T07:34:43.469098+0000","last_unstale":"2024-09-21T07:34:44.468175+0000","last_undegraded":"2024-09-21T07:34:44.468175+0000","last_fullsized":"2024-09-21T07:34:44.468175+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:03.654293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"48'2","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834059+0000","last_change":"2024-09-21T07:34:42.444868+0000","last_active":"2024-09-21T07:34:44.834059+0000","last_peered":"2024-09-21T07:34:44.834059+0000","last_clean":"2024-09-21T07:34:44.834059+0000","last_became_active":"2024-09-21T07:34:42.444574+0000","last_became_peered":"2024-09-21T07:34:42.444574+0000","last_unstale":"2024-09-21T07:34:44.834059+0000","last_undegraded":"2024-09-21T07:34:44.834059+0000","last_fullsized":"2024-09-21T07:34:44.834059+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:14.168800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835810+0000","last_change":"2024-09-21T07:34:24.259902+0000","last_active":"2024-09-21T07:34:44.835810+0000","last_peered":"2024-09-21T07:34:44.835810+0000","last_clean":"2024-09-21T07:34:44.835810+0000","last_became_active":"2024-09-21T07:34:23.271902+0000","last_became_peered":"2024-09-21T07:34:23.271902+0000","last_unstale":"2024-09-21T07:34:44.835810+0000","last_undegraded":"2024-09-21T07:34:44.835810+0000","last_fullsized":"2024-09-21T07:34:44.835810+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:59.439525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001686358,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834649+0000","last_change":"2024-09-21T07:34:43.488076+0000","last_active":"2024-09-21T07:34:44.834649+0000","last_peered":"2024-09-21T07:34:44.834649+0000","last_clean":"2024-09-21T07:34:44.834649+0000","last_became_active":"2024-09-21T07:34:43.487739+0000","last_became_peered":"2024-09-21T07:34:43.487739+0000","last_unstale":"2024-09-21T07:34:44.834649+0000","last_undegraded":"2024-09-21T07:34:44.834649+0000","last_fullsized":"2024-09-21T07:34:44.834649+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:35:38.440936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:34:42.454137+0000","last_change":"2024-09-21T07:34:42.452085+0000","last_active":"2024-09-21T07:34:42.454137+0000","last_peered":"2024-09-21T07:34:42.454137+0000","last_clean":"2024-09-21T07:34:42.454137+0000","last_became_active":"2024-09-21T07:34:42.451768+0000","last_became_peered":"2024-09-21T07:34:42.451768+0000","last_unstale":"2024-09-21T07:34:42.454137+0000","last_undegraded":"2024-09-21T07:34:42.454137+0000","last_fullsized":"2024-09-21T07:34:42.454137+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:09:15.578304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":40,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833639+0000","last_change":"2024-09-21T07:34:24.259662+0000","last_active":"2024-09-21T07:34:44.833639+0000","last_peered":"2024-09-21T07:34:44.833639+0000","last_clean":"2024-09-21T07:34:44.833639+0000","last_became_active":"2024-09-21T07:34:23.278812+0000","last_became_peered":"2024-09-21T07:34:23.278812+0000","last_unstale":"2024-09-21T07:34:44.833639+0000","last_undegraded":"2024-09-21T07:34:44.833639+0000","last_fullsized":"2024-09-21T07:34:44.833639+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:49.829974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356463,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835746+0000","last_change":"2024-09-21T07:34:43.470322+0000","last_active":"2024-09-21T07:34:44.835746+0000","last_peered":"2024-09-21T07:34:44.835746+0000","last_clean":"2024-09-21T07:34:44.835746+0000","last_became_active":"2024-09-21T07:34:43.469955+0000","last_became_peered":"2024-09-21T07:34:43.469955+0000","last_unstale":"2024-09-21T07:34:44.835746+0000","last_undegraded":"2024-09-21T07:34:44.835746+0000","last_fullsized":"2024-09-21T07:34:44.835746+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:30.715538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835746+0000","last_change":"2024-09-21T07:34:42.441191+0000","last_active":"2024-09-21T07:34:44.835746+0000","last_peered":"2024-09-21T07:34:44.835746+0000","last_clean":"2024-09-21T07:34:44.835746+0000","last_became_active":"2024-09-21T07:34:42.440881+0000","last_became_peered":"2024-09-21T07:34:42.440881+0000","last_unstale":"2024-09-21T07:34:44.835746+0000","last_undegraded":"2024-09-21T07:34:44.835746+0000","last_fullsized":"2024-09-21T07:34:44.835746+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:45.066904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":38,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833256+0000","last_change":"2024-09-21T07:34:25.084861+0000","last_active":"2024-09-21T07:34:44.833256+0000","last_peered":"2024-09-21T07:34:44.833256+0000","last_clean":"2024-09-21T07:34:44.833256+0000","last_became_active":"2024-09-21T07:34:23.277708+0000","last_became_peered":"2024-09-21T07:34:23.277708+0000","last_unstale":"2024-09-21T07:34:44.833256+0000","last_undegraded":"2024-09-21T07:34:44.833256+0000","last_fullsized":"2024-09-21T07:34:44.833256+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:40.816488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000597566,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835256+0000","last_change":"2024-09-21T07:34:43.480401+0000","last_active":"2024-09-21T07:34:44.835256+0000","last_peered":"2024-09-21T07:34:44.835256+0000","last_clean":"2024-09-21T07:34:44.835256+0000","last_became_active":"2024-09-21T07:34:43.479988+0000","last_became_peered":"2024-09-21T07:34:43.479988+0000","last_unstale":"2024-09-21T07:34:44.835256+0000","last_undegraded":"2024-09-21T07:34:44.835256+0000","last_fullsized":"2024-09-21T07:34:44.835256+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:03.660689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"48'8","reported_seq":23,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834495+0000","last_change":"2024-09-21T07:34:42.449884+0000","last_active":"2024-09-21T07:34:44.834495+0000","last_peered":"2024-09-21T07:34:44.834495+0000","last_clean":"2024-09-21T07:34:44.834495+0000","last_became_active":"2024-09-21T07:34:42.448499+0000","last_became_peered":"2024-09-21T07:34:42.448499+0000","last_unstale":"2024-09-21T07:34:44.834495+0000","last_undegraded":"2024-09-21T07:34:44.834495+0000","last_fullsized":"2024-09-21T07:34:44.834495+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:58.240549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":40,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833217+0000","last_change":"2024-09-21T07:34:24.258709+0000","last_active":"2024-09-21T07:34:44.833217+0000","last_peered":"2024-09-21T07:34:44.833217+0000","last_clean":"2024-09-21T07:34:44.833217+0000","last_became_active":"2024-09-21T07:34:23.279042+0000","last_became_peered":"2024-09-21T07:34:23.279042+0000","last_unstale":"2024-09-21T07:34:44.833217+0000","last_undegraded":"2024-09-21T07:34:44.833217+0000","last_fullsized":"2024-09-21T07:34:44.833217+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:01.762117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019282100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833735+0000","last_change":"2024-09-21T07:34:43.466176+0000","last_active":"2024-09-21T07:34:44.833735+0000","last_peered":"2024-09-21T07:34:44.833735+0000","last_clean":"2024-09-21T07:34:44.833735+0000","last_became_active":"2024-09-21T07:34:43.464214+0000","last_became_peered":"2024-09-21T07:34:43.464214+0000","last_unstale":"2024-09-21T07:34:44.833735+0000","last_undegraded":"2024-09-21T07:34:44.833735+0000","last_fullsized":"2024-09-21T07:34:44.833735+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:22.282533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836035+0000","last_change":"2024-09-21T07:34:42.442308+0000","last_active":"2024-09-21T07:34:44.836035+0000","last_peered":"2024-09-21T07:34:44.836035+0000","last_clean":"2024-09-21T07:34:44.836035+0000","last_became_active":"2024-09-21T07:34:42.442018+0000","last_became_peered":"2024-09-21T07:34:42.442018+0000","last_unstale":"2024-09-21T07:34:44.836035+0000","last_undegraded":"2024-09-21T07:34:44.836035+0000","last_fullsized":"2024-09-21T07:34:44.836035+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:53.642537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":38,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835026+0000","last_change":"2024-09-21T07:34:25.085635+0000","last_active":"2024-09-21T07:34:44.835026+0000","last_peered":"2024-09-21T07:34:44.835026+0000","last_clean":"2024-09-21T07:34:44.835026+0000","last_became_active":"2024-09-21T07:34:23.273566+0000","last_became_peered":"2024-09-21T07:34:23.273566+0000","last_unstale":"2024-09-21T07:34:44.835026+0000","last_undegraded":"2024-09-21T07:34:44.835026+0000","last_fullsized":"2024-09-21T07:34:44.835026+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:14.142272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079897300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467666+0000","last_change":"2024-09-21T07:34:43.472808+0000","last_active":"2024-09-21T07:34:44.467666+0000","last_peered":"2024-09-21T07:34:44.467666+0000","last_clean":"2024-09-21T07:34:44.467666+0000","last_became_active":"2024-09-21T07:34:43.466752+0000","last_became_peered":"2024-09-21T07:34:43.466752+0000","last_unstale":"2024-09-21T07:34:44.467666+0000","last_undegraded":"2024-09-21T07:34:44.467666+0000","last_fullsized":"2024-09-21T07:34:44.467666+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:19:06.280633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833887+0000","last_change":"2024-09-21T07:34:42.447266+0000","last_active":"2024-09-21T07:34:44.833887+0000","last_peered":"2024-09-21T07:34:44.833887+0000","last_clean":"2024-09-21T07:34:44.833887+0000","last_became_active":"2024-09-21T07:34:42.446823+0000","last_became_peered":"2024-09-21T07:34:42.446823+0000","last_unstale":"2024-09-21T07:34:44.833887+0000","last_undegraded":"2024-09-21T07:34:44.833887+0000","last_fullsized":"2024-09-21T07:34:44.833887+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:16.281729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835166+0000","last_change":"2024-09-21T07:34:24.597926+0000","last_active":"2024-09-21T07:34:44.835166+0000","last_peered":"2024-09-21T07:34:44.835166+0000","last_clean":"2024-09-21T07:34:44.835166+0000","last_became_active":"2024-09-21T07:34:23.268926+0000","last_became_peered":"2024-09-21T07:34:23.268926+0000","last_unstale":"2024-09-21T07:34:44.835166+0000","last_undegraded":"2024-09-21T07:34:44.835166+0000","last_fullsized":"2024-09-21T07:34:44.835166+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:46.190752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101851,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834479+0000","last_change":"2024-09-21T07:34:43.481754+0000","last_active":"2024-09-21T07:34:44.834479+0000","last_peered":"2024-09-21T07:34:44.834479+0000","last_clean":"2024-09-21T07:34:44.834479+0000","last_became_active":"2024-09-21T07:34:43.481460+0000","last_became_peered":"2024-09-21T07:34:43.481460+0000","last_unstale":"2024-09-21T07:34:44.834479+0000","last_undegraded":"2024-09-21T07:34:44.834479+0000","last_fullsized":"2024-09-21T07:34:44.834479+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:17:36.213769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"48'3","reported_seq":18,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836119+0000","last_change":"2024-09-21T07:34:42.450507+0000","last_active":"2024-09-21T07:34:44.836119+0000","last_peered":"2024-09-21T07:34:44.836119+0000","last_clean":"2024-09-21T07:34:44.836119+0000","last_became_active":"2024-09-21T07:34:42.449613+0000","last_became_peered":"2024-09-21T07:34:42.449613+0000","last_unstale":"2024-09-21T07:34:44.836119+0000","last_undegraded":"2024-09-21T07:34:44.836119+0000","last_fullsized":"2024-09-21T07:34:44.836119+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:59:52.817419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":38,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.466765+0000","last_change":"2024-09-21T07:34:24.259794+0000","last_active":"2024-09-21T07:34:44.466765+0000","last_peered":"2024-09-21T07:34:44.466765+0000","last_clean":"2024-09-21T07:34:44.466765+0000","last_became_active":"2024-09-21T07:34:23.265679+0000","last_became_peered":"2024-09-21T07:34:23.265679+0000","last_unstale":"2024-09-21T07:34:44.466765+0000","last_undegraded":"2024-09-21T07:34:44.466765+0000","last_fullsized":"2024-09-21T07:34:44.466765+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:02.575796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001298518,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834659+0000","last_change":"2024-09-21T07:34:43.457653+0000","last_active":"2024-09-21T07:34:44.834659+0000","last_peered":"2024-09-21T07:34:44.834659+0000","last_clean":"2024-09-21T07:34:44.834659+0000","last_became_active":"2024-09-21T07:34:43.457347+0000","last_became_peered":"2024-09-21T07:34:43.457347+0000","last_unstale":"2024-09-21T07:34:44.834659+0000","last_undegraded":"2024-09-21T07:34:44.834659+0000","last_fullsized":"2024-09-21T07:34:44.834659+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:02:38.909862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"48'6","reported_seq":21,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834377+0000","last_change":"2024-09-21T07:34:42.449115+0000","last_active":"2024-09-21T07:34:44.834377+0000","last_peered":"2024-09-21T07:34:44.834377+0000","last_clean":"2024-09-21T07:34:44.834377+0000","last_became_active":"2024-09-21T07:34:42.448834+0000","last_became_peered":"2024-09-21T07:34:42.448834+0000","last_unstale":"2024-09-21T07:34:44.834377+0000","last_undegraded":"2024-09-21T07:34:44.834377+0000","last_fullsized":"2024-09-21T07:34:44.834377+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:49.800427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836377+0000","last_change":"2024-09-21T07:34:24.260330+0000","last_active":"2024-09-21T07:34:44.836377+0000","last_peered":"2024-09-21T07:34:44.836377+0000","last_clean":"2024-09-21T07:34:44.836377+0000","last_became_active":"2024-09-21T07:34:23.637210+0000","last_became_peered":"2024-09-21T07:34:23.637210+0000","last_unstale":"2024-09-21T07:34:44.836377+0000","last_undegraded":"2024-09-21T07:34:44.836377+0000","last_fullsized":"2024-09-21T07:34:44.836377+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.488451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001537093,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834186+0000","last_change":"2024-09-21T07:34:43.481741+0000","last_active":"2024-09-21T07:34:44.834186+0000","last_peered":"2024-09-21T07:34:44.834186+0000","last_clean":"2024-09-21T07:34:44.834186+0000","last_became_active":"2024-09-21T07:34:43.481368+0000","last_became_peered":"2024-09-21T07:34:43.481368+0000","last_unstale":"2024-09-21T07:34:44.834186+0000","last_undegraded":"2024-09-21T07:34:44.834186+0000","last_fullsized":"2024-09-21T07:34:44.834186+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:16.752080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"48'1","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833850+0000","last_change":"2024-09-21T07:34:42.455980+0000","last_active":"2024-09-21T07:34:44.833850+0000","last_peered":"2024-09-21T07:34:44.833850+0000","last_clean":"2024-09-21T07:34:44.833850+0000","last_became_active":"2024-09-21T07:34:42.455368+0000","last_became_peered":"2024-09-21T07:34:42.455368+0000","last_unstale":"2024-09-21T07:34:44.833850+0000","last_undegraded":"2024-09-21T07:34:44.833850+0000","last_fullsized":"2024-09-21T07:34:44.833850+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:07:34.444440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833846+0000","last_change":"2024-09-21T07:34:24.597286+0000","last_active":"2024-09-21T07:34:44.833846+0000","last_peered":"2024-09-21T07:34:44.833846+0000","last_clean":"2024-09-21T07:34:44.833846+0000","last_became_active":"2024-09-21T07:34:23.267133+0000","last_became_peered":"2024-09-21T07:34:23.267133+0000","last_unstale":"2024-09-21T07:34:44.833846+0000","last_undegraded":"2024-09-21T07:34:44.833846+0000","last_fullsized":"2024-09-21T07:34:44.833846+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:18.257010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836565+0000","last_change":"2024-09-21T07:34:43.476520+0000","last_active":"2024-09-21T07:34:44.836565+0000","last_peered":"2024-09-21T07:34:44.836565+0000","last_clean":"2024-09-21T07:34:44.836565+0000","last_became_active":"2024-09-21T07:34:43.476159+0000","last_became_peered":"2024-09-21T07:34:43.476159+0000","last_unstale":"2024-09-21T07:34:44.836565+0000","last_undegraded":"2024-09-21T07:34:44.836565+0000","last_fullsized":"2024-09-21T07:34:44.836565+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:27.525343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"48'1","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833695+0000","last_change":"2024-09-21T07:34:42.441002+0000","last_active":"2024-09-21T07:34:44.833695+0000","last_peered":"2024-09-21T07:34:44.833695+0000","last_clean":"2024-09-21T07:34:44.833695+0000","last_became_active":"2024-09-21T07:34:42.440702+0000","last_became_peered":"2024-09-21T07:34:42.440702+0000","last_unstale":"2024-09-21T07:34:44.833695+0000","last_undegraded":"2024-09-21T07:34:44.833695+0000","last_fullsized":"2024-09-21T07:34:44.833695+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:09.841490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":40,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833536+0000","last_change":"2024-09-21T07:34:24.258712+0000","last_active":"2024-09-21T07:34:44.833536+0000","last_peered":"2024-09-21T07:34:44.833536+0000","last_clean":"2024-09-21T07:34:44.833536+0000","last_became_active":"2024-09-21T07:34:23.274692+0000","last_became_peered":"2024-09-21T07:34:23.274692+0000","last_unstale":"2024-09-21T07:34:44.833536+0000","last_undegraded":"2024-09-21T07:34:44.833536+0000","last_fullsized":"2024-09-21T07:34:44.833536+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:43.279454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018051479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835466+0000","last_change":"2024-09-21T07:34:43.463236+0000","last_active":"2024-09-21T07:34:44.835466+0000","last_peered":"2024-09-21T07:34:44.835466+0000","last_clean":"2024-09-21T07:34:44.835466+0000","last_became_active":"2024-09-21T07:34:43.458917+0000","last_became_peered":"2024-09-21T07:34:43.458917+0000","last_unstale":"2024-09-21T07:34:44.835466+0000","last_undegraded":"2024-09-21T07:34:44.835466+0000","last_fullsized":"2024-09-21T07:34:44.835466+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:45:19.428476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:34:42.453151+0000","last_change":"2024-09-21T07:34:42.447422+0000","last_active":"2024-09-21T07:34:42.453151+0000","last_peered":"2024-09-21T07:34:42.453151+0000","last_clean":"2024-09-21T07:34:42.453151+0000","last_became_active":"2024-09-21T07:34:42.447132+0000","last_became_peered":"2024-09-21T07:34:42.447132+0000","last_unstale":"2024-09-21T07:34:42.453151+0000","last_undegraded":"2024-09-21T07:34:42.453151+0000","last_fullsized":"2024-09-21T07:34:42.453151+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:11.994352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":38,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833146+0000","last_change":"2024-09-21T07:34:25.084849+0000","last_active":"2024-09-21T07:34:44.833146+0000","last_peered":"2024-09-21T07:34:44.833146+0000","last_clean":"2024-09-21T07:34:44.833146+0000","last_became_active":"2024-09-21T07:34:23.277713+0000","last_became_peered":"2024-09-21T07:34:23.277713+0000","last_unstale":"2024-09-21T07:34:44.833146+0000","last_undegraded":"2024-09-21T07:34:44.833146+0000","last_fullsized":"2024-09-21T07:34:44.833146+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:05.811810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059079499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836091+0000","last_change":"2024-09-21T07:34:43.472697+0000","last_active":"2024-09-21T07:34:44.836091+0000","last_peered":"2024-09-21T07:34:44.836091+0000","last_clean":"2024-09-21T07:34:44.836091+0000","last_became_active":"2024-09-21T07:34:43.472276+0000","last_became_peered":"2024-09-21T07:34:43.472276+0000","last_unstale":"2024-09-21T07:34:44.836091+0000","last_undegraded":"2024-09-21T07:34:44.836091+0000","last_fullsized":"2024-09-21T07:34:44.836091+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:49.596333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833099+0000","last_change":"2024-09-21T07:34:42.453339+0000","last_active":"2024-09-21T07:34:44.833099+0000","last_peered":"2024-09-21T07:34:44.833099+0000","last_clean":"2024-09-21T07:34:44.833099+0000","last_became_active":"2024-09-21T07:34:42.452974+0000","last_became_peered":"2024-09-21T07:34:42.452974+0000","last_unstale":"2024-09-21T07:34:44.833099+0000","last_undegraded":"2024-09-21T07:34:44.833099+0000","last_fullsized":"2024-09-21T07:34:44.833099+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:22.788726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467897+0000","last_change":"2024-09-21T07:34:24.538610+0000","last_active":"2024-09-21T07:34:44.467897+0000","last_peered":"2024-09-21T07:34:44.467897+0000","last_clean":"2024-09-21T07:34:44.467897+0000","last_became_active":"2024-09-21T07:34:23.259700+0000","last_became_peered":"2024-09-21T07:34:23.259700+0000","last_unstale":"2024-09-21T07:34:44.467897+0000","last_undegraded":"2024-09-21T07:34:44.467897+0000","last_fullsized":"2024-09-21T07:34:44.467897+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:57.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070522399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834793+0000","last_change":"2024-09-21T07:34:43.474381+0000","last_active":"2024-09-21T07:34:44.834793+0000","last_peered":"2024-09-21T07:34:44.834793+0000","last_clean":"2024-09-21T07:34:44.834793+0000","last_became_active":"2024-09-21T07:34:43.473953+0000","last_became_peered":"2024-09-21T07:34:43.473953+0000","last_unstale":"2024-09-21T07:34:44.834793+0000","last_undegraded":"2024-09-21T07:34:44.834793+0000","last_fullsized":"2024-09-21T07:34:44.834793+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:31.362212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:34:42.450606+0000","last_change":"2024-09-21T07:34:42.444572+0000","last_active":"2024-09-21T07:34:42.450606+0000","last_peered":"2024-09-21T07:34:42.450606+0000","last_clean":"2024-09-21T07:34:42.450606+0000","last_became_active":"2024-09-21T07:34:42.444254+0000","last_became_peered":"2024-09-21T07:34:42.444254+0000","last_unstale":"2024-09-21T07:34:42.450606+0000","last_undegraded":"2024-09-21T07:34:42.450606+0000","last_fullsized":"2024-09-21T07:34:42.450606+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:08.176679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835079+0000","last_change":"2024-09-21T07:34:24.597597+0000","last_active":"2024-09-21T07:34:44.835079+0000","last_peered":"2024-09-21T07:34:44.835079+0000","last_clean":"2024-09-21T07:34:44.835079+0000","last_became_active":"2024-09-21T07:34:23.635204+0000","last_became_peered":"2024-09-21T07:34:23.635204+0000","last_unstale":"2024-09-21T07:34:44.835079+0000","last_undegraded":"2024-09-21T07:34:44.835079+0000","last_fullsized":"2024-09-21T07:34:44.835079+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:48.644789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081139199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467728+0000","last_change":"2024-09-21T07:34:43.460645+0000","last_active":"2024-09-21T07:34:44.467728+0000","last_peered":"2024-09-21T07:34:44.467728+0000","last_clean":"2024-09-21T07:34:44.467728+0000","last_became_active":"2024-09-21T07:34:43.460320+0000","last_became_peered":"2024-09-21T07:34:43.460320+0000","last_unstale":"2024-09-21T07:34:44.467728+0000","last_undegraded":"2024-09-21T07:34:44.467728+0000","last_fullsized":"2024-09-21T07:34:44.467728+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:54.644949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"48'1","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465954+0000","last_change":"2024-09-21T07:34:42.446762+0000","last_active":"2024-09-21T07:34:44.465954+0000","last_peered":"2024-09-21T07:34:44.465954+0000","last_clean":"2024-09-21T07:34:44.465954+0000","last_became_active":"2024-09-21T07:34:42.446320+0000","last_became_peered":"2024-09-21T07:34:42.446320+0000","last_unstale":"2024-09-21T07:34:44.465954+0000","last_undegraded":"2024-09-21T07:34:44.465954+0000","last_fullsized":"2024-09-21T07:34:44.465954+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:46.161972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":40,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835380+0000","last_change":"2024-09-21T07:34:24.259026+0000","last_active":"2024-09-21T07:34:44.835380+0000","last_peered":"2024-09-21T07:34:44.835380+0000","last_clean":"2024-09-21T07:34:44.835380+0000","last_became_active":"2024-09-21T07:34:23.635456+0000","last_became_peered":"2024-09-21T07:34:23.635456+0000","last_unstale":"2024-09-21T07:34:44.835380+0000","last_undegraded":"2024-09-21T07:34:44.835380+0000","last_fullsized":"2024-09-21T07:34:44.835380+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:09.592752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018654419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-21T07:34:42.434916+0000","last_change":"2024-09-21T07:34:42.434916+0000","last_active":"2024-09-21T07:34:42.415959+0000","last_peered":"2024-09-21T07:34:42.415959+0000","last_clean":"2024-09-21T07:34:42.415959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:42.434916+0000","last_undegraded":"2024-09-21T07:34:42.434916+0000","last_fullsized":"2024-09-21T07:34:42.434916+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"48'2","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467319+0000","last_change":"2024-09-21T07:34:42.443581+0000","last_active":"2024-09-21T07:34:44.467319+0000","last_peered":"2024-09-21T07:34:44.467319+0000","last_clean":"2024-09-21T07:34:44.467319+0000","last_became_active":"2024-09-21T07:34:42.443284+0000","last_became_peered":"2024-09-21T07:34:42.443284+0000","last_unstale":"2024-09-21T07:34:44.467319+0000","last_undegraded":"2024-09-21T07:34:44.467319+0000","last_fullsized":"2024-09-21T07:34:44.467319+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:59:14.692524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835966+0000","last_change":"2024-09-21T07:34:24.260300+0000","last_active":"2024-09-21T07:34:44.835966+0000","last_peered":"2024-09-21T07:34:44.835966+0000","last_clean":"2024-09-21T07:34:44.835966+0000","last_became_active":"2024-09-21T07:34:23.270128+0000","last_became_peered":"2024-09-21T07:34:23.270128+0000","last_unstale":"2024-09-21T07:34:44.835966+0000","last_undegraded":"2024-09-21T07:34:44.835966+0000","last_fullsized":"2024-09-21T07:34:44.835966+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:25.260931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00176651,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833363+0000","last_change":"2024-09-21T07:34:43.457202+0000","last_active":"2024-09-21T07:34:44.833363+0000","last_peered":"2024-09-21T07:34:44.833363+0000","last_clean":"2024-09-21T07:34:44.833363+0000","last_became_active":"2024-09-21T07:34:43.456851+0000","last_became_peered":"2024-09-21T07:34:43.456851+0000","last_unstale":"2024-09-21T07:34:44.833363+0000","last_undegraded":"2024-09-21T07:34:44.833363+0000","last_fullsized":"2024-09-21T07:34:44.833363+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:12.056767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"48'2","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.468447+0000","last_change":"2024-09-21T07:34:42.434252+0000","last_active":"2024-09-21T07:34:44.468447+0000","last_peered":"2024-09-21T07:34:44.468447+0000","last_clean":"2024-09-21T07:34:44.468447+0000","last_became_active":"2024-09-21T07:34:42.433936+0000","last_became_peered":"2024-09-21T07:34:42.433936+0000","last_unstale":"2024-09-21T07:34:44.468447+0000","last_undegraded":"2024-09-21T07:34:44.468447+0000","last_fullsized":"2024-09-21T07:34:44.468447+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:13.202719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467474+0000","last_change":"2024-09-21T07:34:25.028904+0000","last_active":"2024-09-21T07:34:44.467474+0000","last_peered":"2024-09-21T07:34:44.467474+0000","last_clean":"2024-09-21T07:34:44.467474+0000","last_became_active":"2024-09-21T07:34:23.260942+0000","last_became_peered":"2024-09-21T07:34:23.260942+0000","last_unstale":"2024-09-21T07:34:44.467474+0000","last_undegraded":"2024-09-21T07:34:44.467474+0000","last_fullsized":"2024-09-21T07:34:44.467474+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:22.709328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060553899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":1758,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509464,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32332,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339508,"statfs":{"total":96636764160,"available":96603656192,"internally_reserved":0,"allocated":1171456,"data_stored":832323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509465,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":532480,"data_stored":221422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509464,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":569344,"data_stored":241831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509465,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31764,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340076,"statfs":{"total":96636764160,"available":96604237824,"internally_reserved":0,"allocated":581632,"data_stored":243443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509466,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31756,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340084,"statfs":{"total":96636764160,"available":96604246016,"internally_reserved":0,"allocated":577536,"data_stored":241991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509466,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1171456,"data_stored":832255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542169,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31748,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340092,"statfs":{"total":96636764160,"available":96604254208,"internally_reserved":0,"allocated":573440,"data_stored":242413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542168,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31760,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340080,"statfs":{"total":96636764160,"available":96604241920,"internally_reserved":0,"allocated":585728,"data_stored":243465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:34:48.468 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:34:48.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.728+0000 7f5e6c400640 1 -- 172.21.15.39:0/268722305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e4c005350 msgr2=0x7f5e4c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:48.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.728+0000 7f5e6c400640 1 --2- 172.21.15.39:0/268722305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e4c005350 0x7f5e4c00b9c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c001d90 tx=0x7f5e4c005830 comp rx=0 tx=0).stop 2024-09-21T07:34:48.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.729+0000 7f5e6c400640 1 -- 172.21.15.39:0/268722305 shutdown_connections 2024-09-21T07:34:48.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.729+0000 7f5e6c400640 1 --2- 172.21.15.39:0/268722305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e4c005350 0x7f5e4c00b9c0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:48.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.729+0000 7f5e6c400640 1 -- 172.21.15.39:0/268722305 >> 172.21.15.39:0/268722305 conn(0x7f5e640b1cb0 msgr2=0x7f5e640b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:48.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.729+0000 7f5e6c400640 1 -- 172.21.15.39:0/268722305 shutdown_connections 2024-09-21T07:34:48.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.729+0000 7f5e6c400640 1 -- 172.21.15.39:0/268722305 wait complete. 2024-09-21T07:34:48.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.731+0000 7f5e6c400640 1 Processor -- start 2024-09-21T07:34:48.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.731+0000 7f5e6c400640 1 -- start start 2024-09-21T07:34:48.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.733+0000 7f5e6c400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e4c005350 0x7f5e64155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:48.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.733+0000 7f5e6c400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5e640c69d0 0x7f5e641f8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:48.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.733+0000 7f5e6c400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e641f8d40 0x7f5e6409bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:48.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.733+0000 7f5e6c400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5e640710a0 con 0x7f5e641f8d40 2024-09-21T07:34:48.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.733+0000 7f5e6c400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5e64070f20 con 0x7f5e4c005350 2024-09-21T07:34:48.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.733+0000 7f5e6c400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f5e640cf010 con 0x7f5e640c69d0 2024-09-21T07:34:48.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.733+0000 7f5e6a400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e641f8d40 0x7f5e6409bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:48.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.733+0000 7f5e6a400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e641f8d40 0x7f5e6409bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57736/0 (socket says 172.21.15.39:57736) 2024-09-21T07:34:48.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.733+0000 7f5e6a400640 1 -- 172.21.15.39:0/2731690423 learned_addr learned my addr 172.21.15.39:0/2731690423 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:48.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.733+0000 7f5e69000640 1 --2- 172.21.15.39:0/2731690423 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5e640c69d0 0x7f5e641f8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:48.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.733+0000 7f5e69a00640 1 --2- 172.21.15.39:0/2731690423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e4c005350 0x7f5e64155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:48.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.733+0000 7f5e6a400640 1 -- 172.21.15.39:0/2731690423 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5e640c69d0 msgr2=0x7f5e641f8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:48.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.733+0000 7f5e6a400640 1 --2- 172.21.15.39:0/2731690423 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5e640c69d0 0x7f5e641f8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:48.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.733+0000 7f5e6a400640 1 -- 172.21.15.39:0/2731690423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e4c005350 msgr2=0x7f5e64155c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:48.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.733+0000 7f5e6a400640 1 --2- 172.21.15.39:0/2731690423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e4c005350 0x7f5e64155c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:48.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.733+0000 7f5e6a400640 1 -- 172.21.15.39:0/2731690423 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e641561a0 con 0x7f5e641f8d40 2024-09-21T07:34:48.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.734+0000 7f5e6a400640 1 --2- 172.21.15.39:0/2731690423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e641f8d40 0x7f5e6409bdf0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c001eb0 tx=0x7f5e5c006d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:48.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.734+0000 7f5e61600640 1 -- 172.21.15.39:0/2731690423 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c003d80 con 0x7f5e641f8d40 2024-09-21T07:34:48.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.734+0000 7f5e61600640 1 -- 172.21.15.39:0/2731690423 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e5c005b80 con 0x7f5e641f8d40 2024-09-21T07:34:48.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.734+0000 7f5e61600640 1 -- 172.21.15.39:0/2731690423 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c005fb0 con 0x7f5e641f8d40 2024-09-21T07:34:48.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.734+0000 7f5e6c400640 1 -- 172.21.15.39:0/2731690423 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e641f9150 con 0x7f5e641f8d40 2024-09-21T07:34:48.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.734+0000 7f5e6c400640 1 -- 172.21.15.39:0/2731690423 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e6415a3d0 con 0x7f5e641f8d40 2024-09-21T07:34:48.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.735+0000 7f5e6c400640 1 -- 172.21.15.39:0/2731690423 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e64159e60 con 0x7f5e641f8d40 2024-09-21T07:34:48.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.736+0000 7f5e61600640 1 -- 172.21.15.39:0/2731690423 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5e5c005070 con 0x7f5e641f8d40 2024-09-21T07:34:48.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.737+0000 7f5e61600640 1 --2- 172.21.15.39:0/2731690423 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5e44078b90 0x7f5e4407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:48.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.737+0000 7f5e61600640 1 -- 172.21.15.39:0/2731690423 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6062+0+0 (secure 0 0 0) 0x7f5e5c0baf60 con 0x7f5e641f8d40 2024-09-21T07:34:48.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.738+0000 7f5e69a00640 1 --2- 172.21.15.39:0/2731690423 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5e44078b90 0x7f5e4407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:48.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.738+0000 7f5e69a00640 1 --2- 172.21.15.39:0/2731690423 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5e44078b90 0x7f5e4407b0a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5e58000fb0 tx=0x7f5e580067a0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:48.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.742+0000 7f5e61600640 1 -- 172.21.15.39:0/2731690423 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5e5c045020 con 0x7f5e641f8d40 2024-09-21T07:34:48.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.935+0000 7f5e6c400640 1 -- 172.21.15.39:0/2731690423 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5e640cd1c0 con 0x7f5e44078b90 2024-09-21T07:34:48.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.941+0000 7f5e61600640 1 -- 172.21.15.39:0/2731690423 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332429 (secure 0 0 0) 0x7f5e640cd1c0 con 0x7f5e44078b90 2024-09-21T07:34:48.946 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:34:48.948 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:34:48.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.946+0000 7f5e6c400640 1 -- 172.21.15.39:0/2731690423 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5e44078b90 msgr2=0x7f5e4407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:48.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.946+0000 7f5e6c400640 1 --2- 172.21.15.39:0/2731690423 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5e44078b90 0x7f5e4407b0a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5e58000fb0 tx=0x7f5e580067a0 comp rx=0 tx=0).stop 2024-09-21T07:34:48.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.946+0000 7f5e6c400640 1 -- 172.21.15.39:0/2731690423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e641f8d40 msgr2=0x7f5e6409bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:48.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.946+0000 7f5e6c400640 1 --2- 172.21.15.39:0/2731690423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e641f8d40 0x7f5e6409bdf0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c001eb0 tx=0x7f5e5c006d20 comp rx=0 tx=0).stop 2024-09-21T07:34:48.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.946+0000 7f5e6c400640 1 -- 172.21.15.39:0/2731690423 shutdown_connections 2024-09-21T07:34:48.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.946+0000 7f5e6c400640 1 --2- 172.21.15.39:0/2731690423 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5e44078b90 0x7f5e4407b0a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:48.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.946+0000 7f5e6c400640 1 --2- 172.21.15.39:0/2731690423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e641f8d40 0x7f5e6409bdf0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:48.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.946+0000 7f5e6c400640 1 --2- 172.21.15.39:0/2731690423 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5e640c69d0 0x7f5e641f8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:48.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.946+0000 7f5e6c400640 1 --2- 172.21.15.39:0/2731690423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e4c005350 0x7f5e64155c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:48.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.946+0000 7f5e6c400640 1 -- 172.21.15.39:0/2731690423 >> 172.21.15.39:0/2731690423 conn(0x7f5e640b1cb0 msgr2=0x7f5e640b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:48.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.946+0000 7f5e6c400640 1 -- 172.21.15.39:0/2731690423 shutdown_connections 2024-09-21T07:34:48.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:48.946+0000 7f5e6c400640 1 -- 172.21.15.39:0/2731690423 wait complete. 2024-09-21T07:34:48.971 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2024-09-21T07:34:47.556632+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592126,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":138,"num_write_kb":1640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":198,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":246848,"kb_used_data":5628,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754727872,"statfs":{"total":773094113280,"available":772841340928,"internally_reserved":0,"allocated":5763072,"data_stored":3099143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1758,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001569"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.466557+0000","last_change":"2024-09-21T07:34:43.473741+0000","last_active":"2024-09-21T07:34:44.466557+0000","last_peered":"2024-09-21T07:34:44.466557+0000","last_clean":"2024-09-21T07:34:44.466557+0000","last_became_active":"2024-09-21T07:34:43.473447+0000","last_became_peered":"2024-09-21T07:34:43.473447+0000","last_unstale":"2024-09-21T07:34:44.466557+0000","last_undegraded":"2024-09-21T07:34:44.466557+0000","last_fullsized":"2024-09-21T07:34:44.466557+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:00:23.404297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836408+0000","last_change":"2024-09-21T07:34:43.473362+0000","last_active":"2024-09-21T07:34:44.836408+0000","last_peered":"2024-09-21T07:34:44.836408+0000","last_clean":"2024-09-21T07:34:44.836408+0000","last_became_active":"2024-09-21T07:34:43.473083+0000","last_became_peered":"2024-09-21T07:34:43.473083+0000","last_unstale":"2024-09-21T07:34:44.836408+0000","last_undegraded":"2024-09-21T07:34:44.836408+0000","last_fullsized":"2024-09-21T07:34:44.836408+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:33.920441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834020+0000","last_change":"2024-09-21T07:34:43.481499+0000","last_active":"2024-09-21T07:34:44.834020+0000","last_peered":"2024-09-21T07:34:44.834020+0000","last_clean":"2024-09-21T07:34:44.834020+0000","last_became_active":"2024-09-21T07:34:43.478254+0000","last_became_peered":"2024-09-21T07:34:43.478254+0000","last_unstale":"2024-09-21T07:34:44.834020+0000","last_undegraded":"2024-09-21T07:34:44.834020+0000","last_fullsized":"2024-09-21T07:34:44.834020+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:03.654293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836742+0000","last_change":"2024-09-21T07:34:43.451459+0000","last_active":"2024-09-21T07:34:44.836742+0000","last_peered":"2024-09-21T07:34:44.836742+0000","last_clean":"2024-09-21T07:34:44.836742+0000","last_became_active":"2024-09-21T07:34:43.451121+0000","last_became_peered":"2024-09-21T07:34:43.451121+0000","last_unstale":"2024-09-21T07:34:44.836742+0000","last_undegraded":"2024-09-21T07:34:44.836742+0000","last_fullsized":"2024-09-21T07:34:44.836742+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:15.139818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834482+0000","last_change":"2024-09-21T07:34:43.455772+0000","last_active":"2024-09-21T07:34:44.834482+0000","last_peered":"2024-09-21T07:34:44.834482+0000","last_clean":"2024-09-21T07:34:44.834482+0000","last_became_active":"2024-09-21T07:34:43.454461+0000","last_became_peered":"2024-09-21T07:34:43.454461+0000","last_unstale":"2024-09-21T07:34:44.834482+0000","last_undegraded":"2024-09-21T07:34:44.834482+0000","last_fullsized":"2024-09-21T07:34:44.834482+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:03.456938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465593+0000","last_change":"2024-09-21T07:34:43.469533+0000","last_active":"2024-09-21T07:34:44.465593+0000","last_peered":"2024-09-21T07:34:44.465593+0000","last_clean":"2024-09-21T07:34:44.465593+0000","last_became_active":"2024-09-21T07:34:43.469336+0000","last_became_peered":"2024-09-21T07:34:43.469336+0000","last_unstale":"2024-09-21T07:34:44.465593+0000","last_undegraded":"2024-09-21T07:34:44.465593+0000","last_fullsized":"2024-09-21T07:34:44.465593+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:20.540854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833543+0000","last_change":"2024-09-21T07:34:43.483800+0000","last_active":"2024-09-21T07:34:44.833543+0000","last_peered":"2024-09-21T07:34:44.833543+0000","last_clean":"2024-09-21T07:34:44.833543+0000","last_became_active":"2024-09-21T07:34:43.483482+0000","last_became_peered":"2024-09-21T07:34:43.483482+0000","last_unstale":"2024-09-21T07:34:44.833543+0000","last_undegraded":"2024-09-21T07:34:44.833543+0000","last_fullsized":"2024-09-21T07:34:44.833543+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:20.152451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834422+0000","last_change":"2024-09-21T07:34:43.473653+0000","last_active":"2024-09-21T07:34:44.834422+0000","last_peered":"2024-09-21T07:34:44.834422+0000","last_clean":"2024-09-21T07:34:44.834422+0000","last_became_active":"2024-09-21T07:34:43.473348+0000","last_became_peered":"2024-09-21T07:34:43.473348+0000","last_unstale":"2024-09-21T07:34:44.834422+0000","last_undegraded":"2024-09-21T07:34:44.834422+0000","last_fullsized":"2024-09-21T07:34:44.834422+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:13.042429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835096+0000","last_change":"2024-09-21T07:34:43.483507+0000","last_active":"2024-09-21T07:34:44.835096+0000","last_peered":"2024-09-21T07:34:44.835096+0000","last_clean":"2024-09-21T07:34:44.835096+0000","last_became_active":"2024-09-21T07:34:43.482198+0000","last_became_peered":"2024-09-21T07:34:43.482198+0000","last_unstale":"2024-09-21T07:34:44.835096+0000","last_undegraded":"2024-09-21T07:34:44.835096+0000","last_fullsized":"2024-09-21T07:34:44.835096+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:45.957857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465790+0000","last_change":"2024-09-21T07:34:43.465258+0000","last_active":"2024-09-21T07:34:44.465790+0000","last_peered":"2024-09-21T07:34:44.465790+0000","last_clean":"2024-09-21T07:34:44.465790+0000","last_became_active":"2024-09-21T07:34:43.464922+0000","last_became_peered":"2024-09-21T07:34:43.464922+0000","last_unstale":"2024-09-21T07:34:44.465790+0000","last_undegraded":"2024-09-21T07:34:44.465790+0000","last_fullsized":"2024-09-21T07:34:44.465790+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:11.406345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467028+0000","last_change":"2024-09-21T07:34:43.449879+0000","last_active":"2024-09-21T07:34:44.467028+0000","last_peered":"2024-09-21T07:34:44.467028+0000","last_clean":"2024-09-21T07:34:44.467028+0000","last_became_active":"2024-09-21T07:34:43.448365+0000","last_became_peered":"2024-09-21T07:34:43.448365+0000","last_unstale":"2024-09-21T07:34:44.467028+0000","last_undegraded":"2024-09-21T07:34:44.467028+0000","last_fullsized":"2024-09-21T07:34:44.467028+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:29:00.993972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834222+0000","last_change":"2024-09-21T07:34:43.460167+0000","last_active":"2024-09-21T07:34:44.834222+0000","last_peered":"2024-09-21T07:34:44.834222+0000","last_clean":"2024-09-21T07:34:44.834222+0000","last_became_active":"2024-09-21T07:34:43.457296+0000","last_became_peered":"2024-09-21T07:34:43.457296+0000","last_unstale":"2024-09-21T07:34:44.834222+0000","last_undegraded":"2024-09-21T07:34:44.834222+0000","last_fullsized":"2024-09-21T07:34:44.834222+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:53:57.871505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.466129+0000","last_change":"2024-09-21T07:34:43.465519+0000","last_active":"2024-09-21T07:34:44.466129+0000","last_peered":"2024-09-21T07:34:44.466129+0000","last_clean":"2024-09-21T07:34:44.466129+0000","last_became_active":"2024-09-21T07:34:43.464802+0000","last_became_peered":"2024-09-21T07:34:43.464802+0000","last_unstale":"2024-09-21T07:34:44.466129+0000","last_undegraded":"2024-09-21T07:34:44.466129+0000","last_fullsized":"2024-09-21T07:34:44.466129+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:59.062262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834033+0000","last_change":"2024-09-21T07:34:43.478575+0000","last_active":"2024-09-21T07:34:44.834033+0000","last_peered":"2024-09-21T07:34:44.834033+0000","last_clean":"2024-09-21T07:34:44.834033+0000","last_became_active":"2024-09-21T07:34:43.478187+0000","last_became_peered":"2024-09-21T07:34:43.478187+0000","last_unstale":"2024-09-21T07:34:44.834033+0000","last_undegraded":"2024-09-21T07:34:44.834033+0000","last_fullsized":"2024-09-21T07:34:44.834033+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:44.866369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836848+0000","last_change":"2024-09-21T07:34:43.467093+0000","last_active":"2024-09-21T07:34:44.836848+0000","last_peered":"2024-09-21T07:34:44.836848+0000","last_clean":"2024-09-21T07:34:44.836848+0000","last_became_active":"2024-09-21T07:34:43.466773+0000","last_became_peered":"2024-09-21T07:34:43.466773+0000","last_unstale":"2024-09-21T07:34:44.836848+0000","last_undegraded":"2024-09-21T07:34:44.836848+0000","last_fullsized":"2024-09-21T07:34:44.836848+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:12:02.180192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465260+0000","last_change":"2024-09-21T07:34:43.462637+0000","last_active":"2024-09-21T07:34:44.465260+0000","last_peered":"2024-09-21T07:34:44.465260+0000","last_clean":"2024-09-21T07:34:44.465260+0000","last_became_active":"2024-09-21T07:34:43.462301+0000","last_became_peered":"2024-09-21T07:34:43.462301+0000","last_unstale":"2024-09-21T07:34:44.465260+0000","last_undegraded":"2024-09-21T07:34:44.465260+0000","last_fullsized":"2024-09-21T07:34:44.465260+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:42.168665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834898+0000","last_change":"2024-09-21T07:34:43.484342+0000","last_active":"2024-09-21T07:34:44.834898+0000","last_peered":"2024-09-21T07:34:44.834898+0000","last_clean":"2024-09-21T07:34:44.834898+0000","last_became_active":"2024-09-21T07:34:43.480452+0000","last_became_peered":"2024-09-21T07:34:43.480452+0000","last_unstale":"2024-09-21T07:34:44.834898+0000","last_undegraded":"2024-09-21T07:34:44.834898+0000","last_fullsized":"2024-09-21T07:34:44.834898+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:30.267103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836590+0000","last_change":"2024-09-21T07:34:43.472890+0000","last_active":"2024-09-21T07:34:44.836590+0000","last_peered":"2024-09-21T07:34:44.836590+0000","last_clean":"2024-09-21T07:34:44.836590+0000","last_became_active":"2024-09-21T07:34:43.472576+0000","last_became_peered":"2024-09-21T07:34:43.472576+0000","last_unstale":"2024-09-21T07:34:44.836590+0000","last_undegraded":"2024-09-21T07:34:44.836590+0000","last_fullsized":"2024-09-21T07:34:44.836590+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:24.719441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834303+0000","last_change":"2024-09-21T07:34:43.476774+0000","last_active":"2024-09-21T07:34:44.834303+0000","last_peered":"2024-09-21T07:34:44.834303+0000","last_clean":"2024-09-21T07:34:44.834303+0000","last_became_active":"2024-09-21T07:34:43.473147+0000","last_became_peered":"2024-09-21T07:34:43.473147+0000","last_unstale":"2024-09-21T07:34:44.834303+0000","last_undegraded":"2024-09-21T07:34:44.834303+0000","last_fullsized":"2024-09-21T07:34:44.834303+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:10.661941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-21T07:34:42.434570+0000","last_change":"2024-09-21T07:34:42.434570+0000","last_active":"2024-09-21T07:34:42.415959+0000","last_peered":"2024-09-21T07:34:42.415959+0000","last_clean":"2024-09-21T07:34:42.415959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:42.434570+0000","last_undegraded":"2024-09-21T07:34:42.434570+0000","last_fullsized":"2024-09-21T07:34:42.434570+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-21T07:34:42.434971+0000","last_change":"2024-09-21T07:34:42.434971+0000","last_active":"2024-09-21T07:34:42.415959+0000","last_peered":"2024-09-21T07:34:42.415959+0000","last_clean":"2024-09-21T07:34:42.415959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:42.434971+0000","last_undegraded":"2024-09-21T07:34:42.434971+0000","last_fullsized":"2024-09-21T07:34:42.434971+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834766+0000","last_change":"2024-09-21T07:34:43.483312+0000","last_active":"2024-09-21T07:34:44.834766+0000","last_peered":"2024-09-21T07:34:44.834766+0000","last_clean":"2024-09-21T07:34:44.834766+0000","last_became_active":"2024-09-21T07:34:43.483003+0000","last_became_peered":"2024-09-21T07:34:43.483003+0000","last_unstale":"2024-09-21T07:34:44.834766+0000","last_undegraded":"2024-09-21T07:34:44.834766+0000","last_fullsized":"2024-09-21T07:34:44.834766+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:33.782633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467544+0000","last_change":"2024-09-21T07:34:43.449273+0000","last_active":"2024-09-21T07:34:44.467544+0000","last_peered":"2024-09-21T07:34:44.467544+0000","last_clean":"2024-09-21T07:34:44.467544+0000","last_became_active":"2024-09-21T07:34:43.448944+0000","last_became_peered":"2024-09-21T07:34:43.448944+0000","last_unstale":"2024-09-21T07:34:44.467544+0000","last_undegraded":"2024-09-21T07:34:44.467544+0000","last_fullsized":"2024-09-21T07:34:44.467544+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:56.540403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833398+0000","last_change":"2024-09-21T07:34:43.479357+0000","last_active":"2024-09-21T07:34:44.833398+0000","last_peered":"2024-09-21T07:34:44.833398+0000","last_clean":"2024-09-21T07:34:44.833398+0000","last_became_active":"2024-09-21T07:34:43.478997+0000","last_became_peered":"2024-09-21T07:34:43.478997+0000","last_unstale":"2024-09-21T07:34:44.833398+0000","last_undegraded":"2024-09-21T07:34:44.833398+0000","last_fullsized":"2024-09-21T07:34:44.833398+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:54.644949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835486+0000","last_change":"2024-09-21T07:34:43.486869+0000","last_active":"2024-09-21T07:34:44.835486+0000","last_peered":"2024-09-21T07:34:44.835486+0000","last_clean":"2024-09-21T07:34:44.835486+0000","last_became_active":"2024-09-21T07:34:43.486351+0000","last_became_peered":"2024-09-21T07:34:43.486351+0000","last_unstale":"2024-09-21T07:34:44.835486+0000","last_undegraded":"2024-09-21T07:34:44.835486+0000","last_fullsized":"2024-09-21T07:34:44.835486+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:21:02.910848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833742+0000","last_change":"2024-09-21T07:34:43.463196+0000","last_active":"2024-09-21T07:34:44.833742+0000","last_peered":"2024-09-21T07:34:44.833742+0000","last_clean":"2024-09-21T07:34:44.833742+0000","last_became_active":"2024-09-21T07:34:43.458559+0000","last_became_peered":"2024-09-21T07:34:43.458559+0000","last_unstale":"2024-09-21T07:34:44.833742+0000","last_undegraded":"2024-09-21T07:34:44.833742+0000","last_fullsized":"2024-09-21T07:34:44.833742+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:37.233429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467280+0000","last_change":"2024-09-21T07:34:43.483549+0000","last_active":"2024-09-21T07:34:44.467280+0000","last_peered":"2024-09-21T07:34:44.467280+0000","last_clean":"2024-09-21T07:34:44.467280+0000","last_became_active":"2024-09-21T07:34:43.472653+0000","last_became_peered":"2024-09-21T07:34:43.472653+0000","last_unstale":"2024-09-21T07:34:44.467280+0000","last_undegraded":"2024-09-21T07:34:44.467280+0000","last_fullsized":"2024-09-21T07:34:44.467280+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:38.814916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-21T07:34:42.435611+0000","last_change":"2024-09-21T07:34:42.435611+0000","last_active":"2024-09-21T07:34:42.415959+0000","last_peered":"2024-09-21T07:34:42.415959+0000","last_clean":"2024-09-21T07:34:42.415959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:42.435611+0000","last_undegraded":"2024-09-21T07:34:42.435611+0000","last_fullsized":"2024-09-21T07:34:42.435611+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833338+0000","last_change":"2024-09-21T07:34:43.489357+0000","last_active":"2024-09-21T07:34:44.833338+0000","last_peered":"2024-09-21T07:34:44.833338+0000","last_clean":"2024-09-21T07:34:44.833338+0000","last_became_active":"2024-09-21T07:34:43.483224+0000","last_became_peered":"2024-09-21T07:34:43.483224+0000","last_unstale":"2024-09-21T07:34:44.833338+0000","last_undegraded":"2024-09-21T07:34:44.833338+0000","last_fullsized":"2024-09-21T07:34:44.833338+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:05.212534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835873+0000","last_change":"2024-09-21T07:34:43.460376+0000","last_active":"2024-09-21T07:34:44.835873+0000","last_peered":"2024-09-21T07:34:44.835873+0000","last_clean":"2024-09-21T07:34:44.835873+0000","last_became_active":"2024-09-21T07:34:43.458320+0000","last_became_peered":"2024-09-21T07:34:43.458320+0000","last_unstale":"2024-09-21T07:34:44.835873+0000","last_undegraded":"2024-09-21T07:34:44.835873+0000","last_fullsized":"2024-09-21T07:34:44.835873+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:35.190652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833745+0000","last_change":"2024-09-21T07:34:43.485349+0000","last_active":"2024-09-21T07:34:44.833745+0000","last_peered":"2024-09-21T07:34:44.833745+0000","last_clean":"2024-09-21T07:34:44.833745+0000","last_became_active":"2024-09-21T07:34:43.483623+0000","last_became_peered":"2024-09-21T07:34:43.483623+0000","last_unstale":"2024-09-21T07:34:44.833745+0000","last_undegraded":"2024-09-21T07:34:44.833745+0000","last_fullsized":"2024-09-21T07:34:44.833745+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:02.084719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467658+0000","last_change":"2024-09-21T07:34:43.471970+0000","last_active":"2024-09-21T07:34:44.467658+0000","last_peered":"2024-09-21T07:34:44.467658+0000","last_clean":"2024-09-21T07:34:44.467658+0000","last_became_active":"2024-09-21T07:34:43.465515+0000","last_became_peered":"2024-09-21T07:34:43.465515+0000","last_unstale":"2024-09-21T07:34:44.467658+0000","last_undegraded":"2024-09-21T07:34:44.467658+0000","last_fullsized":"2024-09-21T07:34:44.467658+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:59:53.819831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-21T07:34:42.435086+0000","last_change":"2024-09-21T07:34:42.435086+0000","last_active":"2024-09-21T07:34:42.415959+0000","last_peered":"2024-09-21T07:34:42.415959+0000","last_clean":"2024-09-21T07:34:42.415959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:42.435086+0000","last_undegraded":"2024-09-21T07:34:42.435086+0000","last_fullsized":"2024-09-21T07:34:42.435086+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"49'1","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834973+0000","last_change":"2024-09-21T07:34:42.447496+0000","last_active":"2024-09-21T07:34:44.834973+0000","last_peered":"2024-09-21T07:34:44.834973+0000","last_clean":"2024-09-21T07:34:44.834973+0000","last_became_active":"2024-09-21T07:34:42.447148+0000","last_became_peered":"2024-09-21T07:34:42.447148+0000","last_unstale":"2024-09-21T07:34:44.834973+0000","last_undegraded":"2024-09-21T07:34:44.834973+0000","last_fullsized":"2024-09-21T07:34:44.834973+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:42:06.750595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":163,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835508+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:34:44.835508+0000","last_peered":"2024-09-21T07:34:44.835508+0000","last_clean":"2024-09-21T07:34:44.835508+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:34:44.835508+0000","last_undegraded":"2024-09-21T07:34:44.835508+0000","last_fullsized":"2024-09-21T07:34:44.835508+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.468769+0000","last_change":"2024-09-21T07:34:24.539248+0000","last_active":"2024-09-21T07:34:44.468769+0000","last_peered":"2024-09-21T07:34:44.468769+0000","last_clean":"2024-09-21T07:34:44.468769+0000","last_became_active":"2024-09-21T07:34:23.273465+0000","last_became_peered":"2024-09-21T07:34:23.273465+0000","last_unstale":"2024-09-21T07:34:44.468769+0000","last_undegraded":"2024-09-21T07:34:44.468769+0000","last_fullsized":"2024-09-21T07:34:44.468769+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:55.517452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060627700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834170+0000","last_change":"2024-09-21T07:34:43.491609+0000","last_active":"2024-09-21T07:34:44.834170+0000","last_peered":"2024-09-21T07:34:44.834170+0000","last_clean":"2024-09-21T07:34:44.834170+0000","last_became_active":"2024-09-21T07:34:43.490138+0000","last_became_peered":"2024-09-21T07:34:43.490138+0000","last_unstale":"2024-09-21T07:34:44.834170+0000","last_undegraded":"2024-09-21T07:34:44.834170+0000","last_fullsized":"2024-09-21T07:34:44.834170+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:16.015947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833429+0000","last_change":"2024-09-21T07:34:42.448815+0000","last_active":"2024-09-21T07:34:44.833429+0000","last_peered":"2024-09-21T07:34:44.833429+0000","last_clean":"2024-09-21T07:34:44.833429+0000","last_became_active":"2024-09-21T07:34:42.448484+0000","last_became_peered":"2024-09-21T07:34:42.448484+0000","last_unstale":"2024-09-21T07:34:44.833429+0000","last_undegraded":"2024-09-21T07:34:44.833429+0000","last_fullsized":"2024-09-21T07:34:44.833429+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:15.522307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833978+0000","last_change":"2024-09-21T07:34:24.597274+0000","last_active":"2024-09-21T07:34:44.833978+0000","last_peered":"2024-09-21T07:34:44.833978+0000","last_clean":"2024-09-21T07:34:44.833978+0000","last_became_active":"2024-09-21T07:34:23.279560+0000","last_became_peered":"2024-09-21T07:34:23.279560+0000","last_unstale":"2024-09-21T07:34:44.833978+0000","last_undegraded":"2024-09-21T07:34:44.833978+0000","last_fullsized":"2024-09-21T07:34:44.833978+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:15.845265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059699799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.468039+0000","last_change":"2024-09-21T07:34:43.468683+0000","last_active":"2024-09-21T07:34:44.468039+0000","last_peered":"2024-09-21T07:34:44.468039+0000","last_clean":"2024-09-21T07:34:44.468039+0000","last_became_active":"2024-09-21T07:34:43.468377+0000","last_became_peered":"2024-09-21T07:34:43.468377+0000","last_unstale":"2024-09-21T07:34:44.468039+0000","last_undegraded":"2024-09-21T07:34:44.468039+0000","last_fullsized":"2024-09-21T07:34:44.468039+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:02.084719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834944+0000","last_change":"2024-09-21T07:34:42.442148+0000","last_active":"2024-09-21T07:34:44.834944+0000","last_peered":"2024-09-21T07:34:44.834944+0000","last_clean":"2024-09-21T07:34:44.834944+0000","last_became_active":"2024-09-21T07:34:42.441839+0000","last_became_peered":"2024-09-21T07:34:42.441839+0000","last_unstale":"2024-09-21T07:34:44.834944+0000","last_undegraded":"2024-09-21T07:34:44.834944+0000","last_fullsized":"2024-09-21T07:34:44.834944+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:35:57.065813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836188+0000","last_change":"2024-09-21T07:34:24.259906+0000","last_active":"2024-09-21T07:34:44.836188+0000","last_peered":"2024-09-21T07:34:44.836188+0000","last_clean":"2024-09-21T07:34:44.836188+0000","last_became_active":"2024-09-21T07:34:23.635282+0000","last_became_peered":"2024-09-21T07:34:23.635282+0000","last_unstale":"2024-09-21T07:34:44.836188+0000","last_undegraded":"2024-09-21T07:34:44.836188+0000","last_fullsized":"2024-09-21T07:34:44.836188+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:00.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001527664,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834033+0000","last_change":"2024-09-21T07:34:43.491274+0000","last_active":"2024-09-21T07:34:44.834033+0000","last_peered":"2024-09-21T07:34:44.834033+0000","last_clean":"2024-09-21T07:34:44.834033+0000","last_became_active":"2024-09-21T07:34:43.489063+0000","last_became_peered":"2024-09-21T07:34:43.489063+0000","last_unstale":"2024-09-21T07:34:44.834033+0000","last_undegraded":"2024-09-21T07:34:44.834033+0000","last_fullsized":"2024-09-21T07:34:44.834033+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:13.315568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833450+0000","last_change":"2024-09-21T07:34:42.437481+0000","last_active":"2024-09-21T07:34:44.833450+0000","last_peered":"2024-09-21T07:34:44.833450+0000","last_clean":"2024-09-21T07:34:44.833450+0000","last_became_active":"2024-09-21T07:34:42.437130+0000","last_became_peered":"2024-09-21T07:34:42.437130+0000","last_unstale":"2024-09-21T07:34:44.833450+0000","last_undegraded":"2024-09-21T07:34:44.833450+0000","last_fullsized":"2024-09-21T07:34:44.833450+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:55.611339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833360+0000","last_change":"2024-09-21T07:34:25.085454+0000","last_active":"2024-09-21T07:34:44.833360+0000","last_peered":"2024-09-21T07:34:44.833360+0000","last_clean":"2024-09-21T07:34:44.833360+0000","last_became_active":"2024-09-21T07:34:23.279205+0000","last_became_peered":"2024-09-21T07:34:23.279205+0000","last_unstale":"2024-09-21T07:34:44.833360+0000","last_undegraded":"2024-09-21T07:34:44.833360+0000","last_fullsized":"2024-09-21T07:34:44.833360+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:54.231666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095320599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835309+0000","last_change":"2024-09-21T07:34:43.457980+0000","last_active":"2024-09-21T07:34:44.835309+0000","last_peered":"2024-09-21T07:34:44.835309+0000","last_clean":"2024-09-21T07:34:44.835309+0000","last_became_active":"2024-09-21T07:34:43.457695+0000","last_became_peered":"2024-09-21T07:34:43.457695+0000","last_unstale":"2024-09-21T07:34:44.835309+0000","last_undegraded":"2024-09-21T07:34:44.835309+0000","last_fullsized":"2024-09-21T07:34:44.835309+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:01.492292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467870+0000","last_change":"2024-09-21T07:34:42.438107+0000","last_active":"2024-09-21T07:34:44.467870+0000","last_peered":"2024-09-21T07:34:44.467870+0000","last_clean":"2024-09-21T07:34:44.467870+0000","last_became_active":"2024-09-21T07:34:42.437794+0000","last_became_peered":"2024-09-21T07:34:42.437794+0000","last_unstale":"2024-09-21T07:34:44.467870+0000","last_undegraded":"2024-09-21T07:34:44.467870+0000","last_fullsized":"2024-09-21T07:34:44.467870+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:24.649223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465509+0000","last_change":"2024-09-21T07:34:25.028521+0000","last_active":"2024-09-21T07:34:44.465509+0000","last_peered":"2024-09-21T07:34:44.465509+0000","last_clean":"2024-09-21T07:34:44.465509+0000","last_became_active":"2024-09-21T07:34:23.635467+0000","last_became_peered":"2024-09-21T07:34:23.635467+0000","last_unstale":"2024-09-21T07:34:44.465509+0000","last_undegraded":"2024-09-21T07:34:44.465509+0000","last_fullsized":"2024-09-21T07:34:44.465509+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:33.729123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000660558,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467665+0000","last_change":"2024-09-21T07:34:43.465578+0000","last_active":"2024-09-21T07:34:44.467665+0000","last_peered":"2024-09-21T07:34:44.467665+0000","last_clean":"2024-09-21T07:34:44.467665+0000","last_became_active":"2024-09-21T07:34:43.465071+0000","last_became_peered":"2024-09-21T07:34:43.465071+0000","last_unstale":"2024-09-21T07:34:44.467665+0000","last_undegraded":"2024-09-21T07:34:44.467665+0000","last_fullsized":"2024-09-21T07:34:44.467665+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:20.152451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:34:42.453134+0000","last_change":"2024-09-21T07:34:42.445927+0000","last_active":"2024-09-21T07:34:42.453134+0000","last_peered":"2024-09-21T07:34:42.453134+0000","last_clean":"2024-09-21T07:34:42.453134+0000","last_became_active":"2024-09-21T07:34:42.445620+0000","last_became_peered":"2024-09-21T07:34:42.445620+0000","last_unstale":"2024-09-21T07:34:42.453134+0000","last_undegraded":"2024-09-21T07:34:42.453134+0000","last_fullsized":"2024-09-21T07:34:42.453134+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:19.691881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":40,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835533+0000","last_change":"2024-09-21T07:34:24.260046+0000","last_active":"2024-09-21T07:34:44.835533+0000","last_peered":"2024-09-21T07:34:44.835533+0000","last_clean":"2024-09-21T07:34:44.835533+0000","last_became_active":"2024-09-21T07:34:23.273431+0000","last_became_peered":"2024-09-21T07:34:23.273431+0000","last_unstale":"2024-09-21T07:34:44.835533+0000","last_undegraded":"2024-09-21T07:34:44.835533+0000","last_fullsized":"2024-09-21T07:34:44.835533+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:18.876057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019803310000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465631+0000","last_change":"2024-09-21T07:34:43.462550+0000","last_active":"2024-09-21T07:34:44.465631+0000","last_peered":"2024-09-21T07:34:44.465631+0000","last_clean":"2024-09-21T07:34:44.465631+0000","last_became_active":"2024-09-21T07:34:43.462210+0000","last_became_peered":"2024-09-21T07:34:43.462210+0000","last_unstale":"2024-09-21T07:34:44.465631+0000","last_undegraded":"2024-09-21T07:34:44.465631+0000","last_fullsized":"2024-09-21T07:34:44.465631+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:56.480314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836267+0000","last_change":"2024-09-21T07:34:42.452761+0000","last_active":"2024-09-21T07:34:44.836267+0000","last_peered":"2024-09-21T07:34:44.836267+0000","last_clean":"2024-09-21T07:34:44.836267+0000","last_became_active":"2024-09-21T07:34:42.452400+0000","last_became_peered":"2024-09-21T07:34:42.452400+0000","last_unstale":"2024-09-21T07:34:44.836267+0000","last_undegraded":"2024-09-21T07:34:44.836267+0000","last_fullsized":"2024-09-21T07:34:44.836267+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:19:05.278221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465673+0000","last_change":"2024-09-21T07:34:25.028640+0000","last_active":"2024-09-21T07:34:44.465673+0000","last_peered":"2024-09-21T07:34:44.465673+0000","last_clean":"2024-09-21T07:34:44.465673+0000","last_became_active":"2024-09-21T07:34:23.260950+0000","last_became_peered":"2024-09-21T07:34:23.260950+0000","last_unstale":"2024-09-21T07:34:44.465673+0000","last_undegraded":"2024-09-21T07:34:44.465673+0000","last_fullsized":"2024-09-21T07:34:44.465673+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:23:42.121398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055135099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833638+0000","last_change":"2024-09-21T07:34:43.453996+0000","last_active":"2024-09-21T07:34:44.833638+0000","last_peered":"2024-09-21T07:34:44.833638+0000","last_clean":"2024-09-21T07:34:44.833638+0000","last_became_active":"2024-09-21T07:34:43.453666+0000","last_became_peered":"2024-09-21T07:34:43.453666+0000","last_unstale":"2024-09-21T07:34:44.833638+0000","last_undegraded":"2024-09-21T07:34:44.833638+0000","last_fullsized":"2024-09-21T07:34:44.833638+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:31.445807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834426+0000","last_change":"2024-09-21T07:34:42.453822+0000","last_active":"2024-09-21T07:34:44.834426+0000","last_peered":"2024-09-21T07:34:44.834426+0000","last_clean":"2024-09-21T07:34:44.834426+0000","last_became_active":"2024-09-21T07:34:42.453499+0000","last_became_peered":"2024-09-21T07:34:42.453499+0000","last_unstale":"2024-09-21T07:34:44.834426+0000","last_undegraded":"2024-09-21T07:34:44.834426+0000","last_fullsized":"2024-09-21T07:34:44.834426+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:39:29.733953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836757+0000","last_change":"2024-09-21T07:34:24.260665+0000","last_active":"2024-09-21T07:34:44.836757+0000","last_peered":"2024-09-21T07:34:44.836757+0000","last_clean":"2024-09-21T07:34:44.836757+0000","last_became_active":"2024-09-21T07:34:23.635909+0000","last_became_peered":"2024-09-21T07:34:23.635909+0000","last_unstale":"2024-09-21T07:34:44.836757+0000","last_undegraded":"2024-09-21T07:34:44.836757+0000","last_fullsized":"2024-09-21T07:34:44.836757+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:44.536885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.468489+0000","last_change":"2024-09-21T07:34:43.469198+0000","last_active":"2024-09-21T07:34:44.468489+0000","last_peered":"2024-09-21T07:34:44.468489+0000","last_clean":"2024-09-21T07:34:44.468489+0000","last_became_active":"2024-09-21T07:34:43.467622+0000","last_became_peered":"2024-09-21T07:34:43.467622+0000","last_unstale":"2024-09-21T07:34:44.468489+0000","last_undegraded":"2024-09-21T07:34:44.468489+0000","last_fullsized":"2024-09-21T07:34:44.468489+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:44.866369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"48'2","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465974+0000","last_change":"2024-09-21T07:34:42.438031+0000","last_active":"2024-09-21T07:34:44.465974+0000","last_peered":"2024-09-21T07:34:44.465974+0000","last_clean":"2024-09-21T07:34:44.465974+0000","last_became_active":"2024-09-21T07:34:42.436986+0000","last_became_peered":"2024-09-21T07:34:42.436986+0000","last_unstale":"2024-09-21T07:34:44.465974+0000","last_undegraded":"2024-09-21T07:34:44.465974+0000","last_fullsized":"2024-09-21T07:34:44.465974+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:51.213865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467759+0000","last_change":"2024-09-21T07:34:24.538845+0000","last_active":"2024-09-21T07:34:44.467759+0000","last_peered":"2024-09-21T07:34:44.467759+0000","last_clean":"2024-09-21T07:34:44.467759+0000","last_became_active":"2024-09-21T07:34:23.636372+0000","last_became_peered":"2024-09-21T07:34:23.636372+0000","last_unstale":"2024-09-21T07:34:44.467759+0000","last_undegraded":"2024-09-21T07:34:44.467759+0000","last_fullsized":"2024-09-21T07:34:44.467759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:29.393813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841451,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-21T07:34:42.436882+0000","last_change":"2024-09-21T07:34:42.436882+0000","last_active":"2024-09-21T07:34:42.415959+0000","last_peered":"2024-09-21T07:34:42.415959+0000","last_clean":"2024-09-21T07:34:42.415959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:42.436882+0000","last_undegraded":"2024-09-21T07:34:42.436882+0000","last_fullsized":"2024-09-21T07:34:42.436882+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834840+0000","last_change":"2024-09-21T07:34:42.447519+0000","last_active":"2024-09-21T07:34:44.834840+0000","last_peered":"2024-09-21T07:34:44.834840+0000","last_clean":"2024-09-21T07:34:44.834840+0000","last_became_active":"2024-09-21T07:34:42.447129+0000","last_became_peered":"2024-09-21T07:34:42.447129+0000","last_unstale":"2024-09-21T07:34:44.834840+0000","last_undegraded":"2024-09-21T07:34:44.834840+0000","last_fullsized":"2024-09-21T07:34:44.834840+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:30.464417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":40,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833703+0000","last_change":"2024-09-21T07:34:24.259686+0000","last_active":"2024-09-21T07:34:44.833703+0000","last_peered":"2024-09-21T07:34:44.833703+0000","last_clean":"2024-09-21T07:34:44.833703+0000","last_became_active":"2024-09-21T07:34:23.274405+0000","last_became_peered":"2024-09-21T07:34:23.274405+0000","last_unstale":"2024-09-21T07:34:44.833703+0000","last_undegraded":"2024-09-21T07:34:44.833703+0000","last_fullsized":"2024-09-21T07:34:44.833703+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:26.304693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022009920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834760+0000","last_change":"2024-09-21T07:34:43.476482+0000","last_active":"2024-09-21T07:34:44.834760+0000","last_peered":"2024-09-21T07:34:44.834760+0000","last_clean":"2024-09-21T07:34:44.834760+0000","last_became_active":"2024-09-21T07:34:43.476189+0000","last_became_peered":"2024-09-21T07:34:43.476189+0000","last_unstale":"2024-09-21T07:34:44.834760+0000","last_undegraded":"2024-09-21T07:34:44.834760+0000","last_fullsized":"2024-09-21T07:34:44.834760+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:08:42.688308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833369+0000","last_change":"2024-09-21T07:34:42.456020+0000","last_active":"2024-09-21T07:34:44.833369+0000","last_peered":"2024-09-21T07:34:44.833369+0000","last_clean":"2024-09-21T07:34:44.833369+0000","last_became_active":"2024-09-21T07:34:42.455399+0000","last_became_peered":"2024-09-21T07:34:42.455399+0000","last_unstale":"2024-09-21T07:34:44.833369+0000","last_undegraded":"2024-09-21T07:34:44.833369+0000","last_fullsized":"2024-09-21T07:34:44.833369+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:57.601710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835457+0000","last_change":"2024-09-21T07:34:24.258754+0000","last_active":"2024-09-21T07:34:44.835457+0000","last_peered":"2024-09-21T07:34:44.835457+0000","last_clean":"2024-09-21T07:34:44.835457+0000","last_became_active":"2024-09-21T07:34:23.272657+0000","last_became_peered":"2024-09-21T07:34:23.272657+0000","last_unstale":"2024-09-21T07:34:44.835457+0000","last_undegraded":"2024-09-21T07:34:44.835457+0000","last_fullsized":"2024-09-21T07:34:44.835457+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:05.477099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067143200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.466771+0000","last_change":"2024-09-21T07:34:43.472869+0000","last_active":"2024-09-21T07:34:44.466771+0000","last_peered":"2024-09-21T07:34:44.466771+0000","last_clean":"2024-09-21T07:34:44.466771+0000","last_became_active":"2024-09-21T07:34:43.472514+0000","last_became_peered":"2024-09-21T07:34:43.472514+0000","last_unstale":"2024-09-21T07:34:44.466771+0000","last_undegraded":"2024-09-21T07:34:44.466771+0000","last_fullsized":"2024-09-21T07:34:44.466771+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:15:22.567958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.466329+0000","last_change":"2024-09-21T07:34:42.443208+0000","last_active":"2024-09-21T07:34:44.466329+0000","last_peered":"2024-09-21T07:34:44.466329+0000","last_clean":"2024-09-21T07:34:44.466329+0000","last_became_active":"2024-09-21T07:34:42.442891+0000","last_became_peered":"2024-09-21T07:34:42.442891+0000","last_unstale":"2024-09-21T07:34:44.466329+0000","last_undegraded":"2024-09-21T07:34:44.466329+0000","last_fullsized":"2024-09-21T07:34:44.466329+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:02.347260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":38,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833706+0000","last_change":"2024-09-21T07:34:25.085232+0000","last_active":"2024-09-21T07:34:44.833706+0000","last_peered":"2024-09-21T07:34:44.833706+0000","last_clean":"2024-09-21T07:34:44.833706+0000","last_became_active":"2024-09-21T07:34:23.277807+0000","last_became_peered":"2024-09-21T07:34:23.277807+0000","last_unstale":"2024-09-21T07:34:44.833706+0000","last_undegraded":"2024-09-21T07:34:44.833706+0000","last_fullsized":"2024-09-21T07:34:44.833706+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:36.362919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086619099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835299+0000","last_change":"2024-09-21T07:34:43.486460+0000","last_active":"2024-09-21T07:34:44.835299+0000","last_peered":"2024-09-21T07:34:44.835299+0000","last_clean":"2024-09-21T07:34:44.835299+0000","last_became_active":"2024-09-21T07:34:43.486148+0000","last_became_peered":"2024-09-21T07:34:43.486148+0000","last_unstale":"2024-09-21T07:34:44.835299+0000","last_undegraded":"2024-09-21T07:34:44.835299+0000","last_fullsized":"2024-09-21T07:34:44.835299+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:31.997216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"48'1","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835624+0000","last_change":"2024-09-21T07:34:42.450738+0000","last_active":"2024-09-21T07:34:44.835624+0000","last_peered":"2024-09-21T07:34:44.835624+0000","last_clean":"2024-09-21T07:34:44.835624+0000","last_became_active":"2024-09-21T07:34:42.450413+0000","last_became_peered":"2024-09-21T07:34:42.450413+0000","last_unstale":"2024-09-21T07:34:44.835624+0000","last_undegraded":"2024-09-21T07:34:44.835624+0000","last_fullsized":"2024-09-21T07:34:44.835624+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:48.569857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835043+0000","last_change":"2024-09-21T07:34:24.597672+0000","last_active":"2024-09-21T07:34:44.835043+0000","last_peered":"2024-09-21T07:34:44.835043+0000","last_clean":"2024-09-21T07:34:44.835043+0000","last_became_active":"2024-09-21T07:34:23.267770+0000","last_became_peered":"2024-09-21T07:34:23.267770+0000","last_unstale":"2024-09-21T07:34:44.835043+0000","last_undegraded":"2024-09-21T07:34:44.835043+0000","last_fullsized":"2024-09-21T07:34:44.835043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:51.629476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086378500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465282+0000","last_change":"2024-09-21T07:34:43.478411+0000","last_active":"2024-09-21T07:34:44.465282+0000","last_peered":"2024-09-21T07:34:44.465282+0000","last_clean":"2024-09-21T07:34:44.465282+0000","last_became_active":"2024-09-21T07:34:43.478106+0000","last_became_peered":"2024-09-21T07:34:43.478106+0000","last_unstale":"2024-09-21T07:34:44.465282+0000","last_undegraded":"2024-09-21T07:34:44.465282+0000","last_fullsized":"2024-09-21T07:34:44.465282+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:41.320870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"48'2","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467059+0000","last_change":"2024-09-21T07:34:42.449630+0000","last_active":"2024-09-21T07:34:44.467059+0000","last_peered":"2024-09-21T07:34:44.467059+0000","last_clean":"2024-09-21T07:34:44.467059+0000","last_became_active":"2024-09-21T07:34:42.448013+0000","last_became_peered":"2024-09-21T07:34:42.448013+0000","last_unstale":"2024-09-21T07:34:44.467059+0000","last_undegraded":"2024-09-21T07:34:44.467059+0000","last_fullsized":"2024-09-21T07:34:44.467059+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:42.900618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":38,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467361+0000","last_change":"2024-09-21T07:34:24.260181+0000","last_active":"2024-09-21T07:34:44.467361+0000","last_peered":"2024-09-21T07:34:44.467361+0000","last_clean":"2024-09-21T07:34:44.467361+0000","last_became_active":"2024-09-21T07:34:23.275580+0000","last_became_peered":"2024-09-21T07:34:23.275580+0000","last_unstale":"2024-09-21T07:34:44.467361+0000","last_undegraded":"2024-09-21T07:34:44.467361+0000","last_fullsized":"2024-09-21T07:34:44.467361+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:49.857622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014706980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834317+0000","last_change":"2024-09-21T07:34:43.479707+0000","last_active":"2024-09-21T07:34:44.834317+0000","last_peered":"2024-09-21T07:34:44.834317+0000","last_clean":"2024-09-21T07:34:44.834317+0000","last_became_active":"2024-09-21T07:34:43.479311+0000","last_became_peered":"2024-09-21T07:34:43.479311+0000","last_unstale":"2024-09-21T07:34:44.834317+0000","last_undegraded":"2024-09-21T07:34:44.834317+0000","last_fullsized":"2024-09-21T07:34:44.834317+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:03:54.452233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:34:42.460152+0000","last_change":"2024-09-21T07:34:42.455379+0000","last_active":"2024-09-21T07:34:42.460152+0000","last_peered":"2024-09-21T07:34:42.460152+0000","last_clean":"2024-09-21T07:34:42.460152+0000","last_became_active":"2024-09-21T07:34:42.455144+0000","last_became_peered":"2024-09-21T07:34:42.455144+0000","last_unstale":"2024-09-21T07:34:42.460152+0000","last_undegraded":"2024-09-21T07:34:42.460152+0000","last_fullsized":"2024-09-21T07:34:42.460152+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:31:47.280896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":40,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834497+0000","last_change":"2024-09-21T07:34:24.259112+0000","last_active":"2024-09-21T07:34:44.834497+0000","last_peered":"2024-09-21T07:34:44.834497+0000","last_clean":"2024-09-21T07:34:44.834497+0000","last_became_active":"2024-09-21T07:34:23.635444+0000","last_became_peered":"2024-09-21T07:34:23.635444+0000","last_unstale":"2024-09-21T07:34:44.834497+0000","last_undegraded":"2024-09-21T07:34:44.834497+0000","last_fullsized":"2024-09-21T07:34:44.834497+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:46.255385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001952636,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465803+0000","last_change":"2024-09-21T07:34:43.476982+0000","last_active":"2024-09-21T07:34:44.465803+0000","last_peered":"2024-09-21T07:34:44.465803+0000","last_clean":"2024-09-21T07:34:44.465803+0000","last_became_active":"2024-09-21T07:34:43.476670+0000","last_became_peered":"2024-09-21T07:34:43.476670+0000","last_unstale":"2024-09-21T07:34:44.465803+0000","last_undegraded":"2024-09-21T07:34:44.465803+0000","last_fullsized":"2024-09-21T07:34:44.465803+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:04:42.140238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:34:42.450602+0000","last_change":"2024-09-21T07:34:42.444915+0000","last_active":"2024-09-21T07:34:42.450602+0000","last_peered":"2024-09-21T07:34:42.450602+0000","last_clean":"2024-09-21T07:34:42.450602+0000","last_became_active":"2024-09-21T07:34:42.444506+0000","last_became_peered":"2024-09-21T07:34:42.444506+0000","last_unstale":"2024-09-21T07:34:42.450602+0000","last_undegraded":"2024-09-21T07:34:42.450602+0000","last_fullsized":"2024-09-21T07:34:42.450602+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:18.964200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.468302+0000","last_change":"2024-09-21T07:34:24.539152+0000","last_active":"2024-09-21T07:34:44.468302+0000","last_peered":"2024-09-21T07:34:44.468302+0000","last_clean":"2024-09-21T07:34:44.468302+0000","last_became_active":"2024-09-21T07:34:23.259573+0000","last_became_peered":"2024-09-21T07:34:23.259573+0000","last_unstale":"2024-09-21T07:34:44.468302+0000","last_undegraded":"2024-09-21T07:34:44.468302+0000","last_fullsized":"2024-09-21T07:34:44.468302+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:26.986900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058108400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.468175+0000","last_change":"2024-09-21T07:34:43.469411+0000","last_active":"2024-09-21T07:34:44.468175+0000","last_peered":"2024-09-21T07:34:44.468175+0000","last_clean":"2024-09-21T07:34:44.468175+0000","last_became_active":"2024-09-21T07:34:43.469098+0000","last_became_peered":"2024-09-21T07:34:43.469098+0000","last_unstale":"2024-09-21T07:34:44.468175+0000","last_undegraded":"2024-09-21T07:34:44.468175+0000","last_fullsized":"2024-09-21T07:34:44.468175+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:03.654293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"48'2","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834059+0000","last_change":"2024-09-21T07:34:42.444868+0000","last_active":"2024-09-21T07:34:44.834059+0000","last_peered":"2024-09-21T07:34:44.834059+0000","last_clean":"2024-09-21T07:34:44.834059+0000","last_became_active":"2024-09-21T07:34:42.444574+0000","last_became_peered":"2024-09-21T07:34:42.444574+0000","last_unstale":"2024-09-21T07:34:44.834059+0000","last_undegraded":"2024-09-21T07:34:44.834059+0000","last_fullsized":"2024-09-21T07:34:44.834059+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:14.168800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835810+0000","last_change":"2024-09-21T07:34:24.259902+0000","last_active":"2024-09-21T07:34:44.835810+0000","last_peered":"2024-09-21T07:34:44.835810+0000","last_clean":"2024-09-21T07:34:44.835810+0000","last_became_active":"2024-09-21T07:34:23.271902+0000","last_became_peered":"2024-09-21T07:34:23.271902+0000","last_unstale":"2024-09-21T07:34:44.835810+0000","last_undegraded":"2024-09-21T07:34:44.835810+0000","last_fullsized":"2024-09-21T07:34:44.835810+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:59.439525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001686358,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834649+0000","last_change":"2024-09-21T07:34:43.488076+0000","last_active":"2024-09-21T07:34:44.834649+0000","last_peered":"2024-09-21T07:34:44.834649+0000","last_clean":"2024-09-21T07:34:44.834649+0000","last_became_active":"2024-09-21T07:34:43.487739+0000","last_became_peered":"2024-09-21T07:34:43.487739+0000","last_unstale":"2024-09-21T07:34:44.834649+0000","last_undegraded":"2024-09-21T07:34:44.834649+0000","last_fullsized":"2024-09-21T07:34:44.834649+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:35:38.440936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:34:42.454137+0000","last_change":"2024-09-21T07:34:42.452085+0000","last_active":"2024-09-21T07:34:42.454137+0000","last_peered":"2024-09-21T07:34:42.454137+0000","last_clean":"2024-09-21T07:34:42.454137+0000","last_became_active":"2024-09-21T07:34:42.451768+0000","last_became_peered":"2024-09-21T07:34:42.451768+0000","last_unstale":"2024-09-21T07:34:42.454137+0000","last_undegraded":"2024-09-21T07:34:42.454137+0000","last_fullsized":"2024-09-21T07:34:42.454137+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:09:15.578304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":40,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833639+0000","last_change":"2024-09-21T07:34:24.259662+0000","last_active":"2024-09-21T07:34:44.833639+0000","last_peered":"2024-09-21T07:34:44.833639+0000","last_clean":"2024-09-21T07:34:44.833639+0000","last_became_active":"2024-09-21T07:34:23.278812+0000","last_became_peered":"2024-09-21T07:34:23.278812+0000","last_unstale":"2024-09-21T07:34:44.833639+0000","last_undegraded":"2024-09-21T07:34:44.833639+0000","last_fullsized":"2024-09-21T07:34:44.833639+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:49.829974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356463,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835746+0000","last_change":"2024-09-21T07:34:43.470322+0000","last_active":"2024-09-21T07:34:44.835746+0000","last_peered":"2024-09-21T07:34:44.835746+0000","last_clean":"2024-09-21T07:34:44.835746+0000","last_became_active":"2024-09-21T07:34:43.469955+0000","last_became_peered":"2024-09-21T07:34:43.469955+0000","last_unstale":"2024-09-21T07:34:44.835746+0000","last_undegraded":"2024-09-21T07:34:44.835746+0000","last_fullsized":"2024-09-21T07:34:44.835746+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:30.715538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835746+0000","last_change":"2024-09-21T07:34:42.441191+0000","last_active":"2024-09-21T07:34:44.835746+0000","last_peered":"2024-09-21T07:34:44.835746+0000","last_clean":"2024-09-21T07:34:44.835746+0000","last_became_active":"2024-09-21T07:34:42.440881+0000","last_became_peered":"2024-09-21T07:34:42.440881+0000","last_unstale":"2024-09-21T07:34:44.835746+0000","last_undegraded":"2024-09-21T07:34:44.835746+0000","last_fullsized":"2024-09-21T07:34:44.835746+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:45.066904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":38,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833256+0000","last_change":"2024-09-21T07:34:25.084861+0000","last_active":"2024-09-21T07:34:44.833256+0000","last_peered":"2024-09-21T07:34:44.833256+0000","last_clean":"2024-09-21T07:34:44.833256+0000","last_became_active":"2024-09-21T07:34:23.277708+0000","last_became_peered":"2024-09-21T07:34:23.277708+0000","last_unstale":"2024-09-21T07:34:44.833256+0000","last_undegraded":"2024-09-21T07:34:44.833256+0000","last_fullsized":"2024-09-21T07:34:44.833256+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:40.816488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000597566,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835256+0000","last_change":"2024-09-21T07:34:43.480401+0000","last_active":"2024-09-21T07:34:44.835256+0000","last_peered":"2024-09-21T07:34:44.835256+0000","last_clean":"2024-09-21T07:34:44.835256+0000","last_became_active":"2024-09-21T07:34:43.479988+0000","last_became_peered":"2024-09-21T07:34:43.479988+0000","last_unstale":"2024-09-21T07:34:44.835256+0000","last_undegraded":"2024-09-21T07:34:44.835256+0000","last_fullsized":"2024-09-21T07:34:44.835256+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:03.660689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"48'8","reported_seq":23,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834495+0000","last_change":"2024-09-21T07:34:42.449884+0000","last_active":"2024-09-21T07:34:44.834495+0000","last_peered":"2024-09-21T07:34:44.834495+0000","last_clean":"2024-09-21T07:34:44.834495+0000","last_became_active":"2024-09-21T07:34:42.448499+0000","last_became_peered":"2024-09-21T07:34:42.448499+0000","last_unstale":"2024-09-21T07:34:44.834495+0000","last_undegraded":"2024-09-21T07:34:44.834495+0000","last_fullsized":"2024-09-21T07:34:44.834495+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:58.240549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":40,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833217+0000","last_change":"2024-09-21T07:34:24.258709+0000","last_active":"2024-09-21T07:34:44.833217+0000","last_peered":"2024-09-21T07:34:44.833217+0000","last_clean":"2024-09-21T07:34:44.833217+0000","last_became_active":"2024-09-21T07:34:23.279042+0000","last_became_peered":"2024-09-21T07:34:23.279042+0000","last_unstale":"2024-09-21T07:34:44.833217+0000","last_undegraded":"2024-09-21T07:34:44.833217+0000","last_fullsized":"2024-09-21T07:34:44.833217+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:01.762117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019282100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833735+0000","last_change":"2024-09-21T07:34:43.466176+0000","last_active":"2024-09-21T07:34:44.833735+0000","last_peered":"2024-09-21T07:34:44.833735+0000","last_clean":"2024-09-21T07:34:44.833735+0000","last_became_active":"2024-09-21T07:34:43.464214+0000","last_became_peered":"2024-09-21T07:34:43.464214+0000","last_unstale":"2024-09-21T07:34:44.833735+0000","last_undegraded":"2024-09-21T07:34:44.833735+0000","last_fullsized":"2024-09-21T07:34:44.833735+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:22.282533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836035+0000","last_change":"2024-09-21T07:34:42.442308+0000","last_active":"2024-09-21T07:34:44.836035+0000","last_peered":"2024-09-21T07:34:44.836035+0000","last_clean":"2024-09-21T07:34:44.836035+0000","last_became_active":"2024-09-21T07:34:42.442018+0000","last_became_peered":"2024-09-21T07:34:42.442018+0000","last_unstale":"2024-09-21T07:34:44.836035+0000","last_undegraded":"2024-09-21T07:34:44.836035+0000","last_fullsized":"2024-09-21T07:34:44.836035+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:53.642537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":38,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835026+0000","last_change":"2024-09-21T07:34:25.085635+0000","last_active":"2024-09-21T07:34:44.835026+0000","last_peered":"2024-09-21T07:34:44.835026+0000","last_clean":"2024-09-21T07:34:44.835026+0000","last_became_active":"2024-09-21T07:34:23.273566+0000","last_became_peered":"2024-09-21T07:34:23.273566+0000","last_unstale":"2024-09-21T07:34:44.835026+0000","last_undegraded":"2024-09-21T07:34:44.835026+0000","last_fullsized":"2024-09-21T07:34:44.835026+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:14.142272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079897300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467666+0000","last_change":"2024-09-21T07:34:43.472808+0000","last_active":"2024-09-21T07:34:44.467666+0000","last_peered":"2024-09-21T07:34:44.467666+0000","last_clean":"2024-09-21T07:34:44.467666+0000","last_became_active":"2024-09-21T07:34:43.466752+0000","last_became_peered":"2024-09-21T07:34:43.466752+0000","last_unstale":"2024-09-21T07:34:44.467666+0000","last_undegraded":"2024-09-21T07:34:44.467666+0000","last_fullsized":"2024-09-21T07:34:44.467666+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:19:06.280633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833887+0000","last_change":"2024-09-21T07:34:42.447266+0000","last_active":"2024-09-21T07:34:44.833887+0000","last_peered":"2024-09-21T07:34:44.833887+0000","last_clean":"2024-09-21T07:34:44.833887+0000","last_became_active":"2024-09-21T07:34:42.446823+0000","last_became_peered":"2024-09-21T07:34:42.446823+0000","last_unstale":"2024-09-21T07:34:44.833887+0000","last_undegraded":"2024-09-21T07:34:44.833887+0000","last_fullsized":"2024-09-21T07:34:44.833887+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:16.281729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835166+0000","last_change":"2024-09-21T07:34:24.597926+0000","last_active":"2024-09-21T07:34:44.835166+0000","last_peered":"2024-09-21T07:34:44.835166+0000","last_clean":"2024-09-21T07:34:44.835166+0000","last_became_active":"2024-09-21T07:34:23.268926+0000","last_became_peered":"2024-09-21T07:34:23.268926+0000","last_unstale":"2024-09-21T07:34:44.835166+0000","last_undegraded":"2024-09-21T07:34:44.835166+0000","last_fullsized":"2024-09-21T07:34:44.835166+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:46.190752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101851,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834479+0000","last_change":"2024-09-21T07:34:43.481754+0000","last_active":"2024-09-21T07:34:44.834479+0000","last_peered":"2024-09-21T07:34:44.834479+0000","last_clean":"2024-09-21T07:34:44.834479+0000","last_became_active":"2024-09-21T07:34:43.481460+0000","last_became_peered":"2024-09-21T07:34:43.481460+0000","last_unstale":"2024-09-21T07:34:44.834479+0000","last_undegraded":"2024-09-21T07:34:44.834479+0000","last_fullsized":"2024-09-21T07:34:44.834479+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:17:36.213769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"48'3","reported_seq":18,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836119+0000","last_change":"2024-09-21T07:34:42.450507+0000","last_active":"2024-09-21T07:34:44.836119+0000","last_peered":"2024-09-21T07:34:44.836119+0000","last_clean":"2024-09-21T07:34:44.836119+0000","last_became_active":"2024-09-21T07:34:42.449613+0000","last_became_peered":"2024-09-21T07:34:42.449613+0000","last_unstale":"2024-09-21T07:34:44.836119+0000","last_undegraded":"2024-09-21T07:34:44.836119+0000","last_fullsized":"2024-09-21T07:34:44.836119+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:59:52.817419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":38,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.466765+0000","last_change":"2024-09-21T07:34:24.259794+0000","last_active":"2024-09-21T07:34:44.466765+0000","last_peered":"2024-09-21T07:34:44.466765+0000","last_clean":"2024-09-21T07:34:44.466765+0000","last_became_active":"2024-09-21T07:34:23.265679+0000","last_became_peered":"2024-09-21T07:34:23.265679+0000","last_unstale":"2024-09-21T07:34:44.466765+0000","last_undegraded":"2024-09-21T07:34:44.466765+0000","last_fullsized":"2024-09-21T07:34:44.466765+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:02.575796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001298518,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834659+0000","last_change":"2024-09-21T07:34:43.457653+0000","last_active":"2024-09-21T07:34:44.834659+0000","last_peered":"2024-09-21T07:34:44.834659+0000","last_clean":"2024-09-21T07:34:44.834659+0000","last_became_active":"2024-09-21T07:34:43.457347+0000","last_became_peered":"2024-09-21T07:34:43.457347+0000","last_unstale":"2024-09-21T07:34:44.834659+0000","last_undegraded":"2024-09-21T07:34:44.834659+0000","last_fullsized":"2024-09-21T07:34:44.834659+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:02:38.909862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"48'6","reported_seq":21,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834377+0000","last_change":"2024-09-21T07:34:42.449115+0000","last_active":"2024-09-21T07:34:44.834377+0000","last_peered":"2024-09-21T07:34:44.834377+0000","last_clean":"2024-09-21T07:34:44.834377+0000","last_became_active":"2024-09-21T07:34:42.448834+0000","last_became_peered":"2024-09-21T07:34:42.448834+0000","last_unstale":"2024-09-21T07:34:44.834377+0000","last_undegraded":"2024-09-21T07:34:44.834377+0000","last_fullsized":"2024-09-21T07:34:44.834377+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:49.800427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836377+0000","last_change":"2024-09-21T07:34:24.260330+0000","last_active":"2024-09-21T07:34:44.836377+0000","last_peered":"2024-09-21T07:34:44.836377+0000","last_clean":"2024-09-21T07:34:44.836377+0000","last_became_active":"2024-09-21T07:34:23.637210+0000","last_became_peered":"2024-09-21T07:34:23.637210+0000","last_unstale":"2024-09-21T07:34:44.836377+0000","last_undegraded":"2024-09-21T07:34:44.836377+0000","last_fullsized":"2024-09-21T07:34:44.836377+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.488451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001537093,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834186+0000","last_change":"2024-09-21T07:34:43.481741+0000","last_active":"2024-09-21T07:34:44.834186+0000","last_peered":"2024-09-21T07:34:44.834186+0000","last_clean":"2024-09-21T07:34:44.834186+0000","last_became_active":"2024-09-21T07:34:43.481368+0000","last_became_peered":"2024-09-21T07:34:43.481368+0000","last_unstale":"2024-09-21T07:34:44.834186+0000","last_undegraded":"2024-09-21T07:34:44.834186+0000","last_fullsized":"2024-09-21T07:34:44.834186+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:16.752080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"48'1","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833850+0000","last_change":"2024-09-21T07:34:42.455980+0000","last_active":"2024-09-21T07:34:44.833850+0000","last_peered":"2024-09-21T07:34:44.833850+0000","last_clean":"2024-09-21T07:34:44.833850+0000","last_became_active":"2024-09-21T07:34:42.455368+0000","last_became_peered":"2024-09-21T07:34:42.455368+0000","last_unstale":"2024-09-21T07:34:44.833850+0000","last_undegraded":"2024-09-21T07:34:44.833850+0000","last_fullsized":"2024-09-21T07:34:44.833850+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:07:34.444440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833846+0000","last_change":"2024-09-21T07:34:24.597286+0000","last_active":"2024-09-21T07:34:44.833846+0000","last_peered":"2024-09-21T07:34:44.833846+0000","last_clean":"2024-09-21T07:34:44.833846+0000","last_became_active":"2024-09-21T07:34:23.267133+0000","last_became_peered":"2024-09-21T07:34:23.267133+0000","last_unstale":"2024-09-21T07:34:44.833846+0000","last_undegraded":"2024-09-21T07:34:44.833846+0000","last_fullsized":"2024-09-21T07:34:44.833846+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:18.257010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836565+0000","last_change":"2024-09-21T07:34:43.476520+0000","last_active":"2024-09-21T07:34:44.836565+0000","last_peered":"2024-09-21T07:34:44.836565+0000","last_clean":"2024-09-21T07:34:44.836565+0000","last_became_active":"2024-09-21T07:34:43.476159+0000","last_became_peered":"2024-09-21T07:34:43.476159+0000","last_unstale":"2024-09-21T07:34:44.836565+0000","last_undegraded":"2024-09-21T07:34:44.836565+0000","last_fullsized":"2024-09-21T07:34:44.836565+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:27.525343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"48'1","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833695+0000","last_change":"2024-09-21T07:34:42.441002+0000","last_active":"2024-09-21T07:34:44.833695+0000","last_peered":"2024-09-21T07:34:44.833695+0000","last_clean":"2024-09-21T07:34:44.833695+0000","last_became_active":"2024-09-21T07:34:42.440702+0000","last_became_peered":"2024-09-21T07:34:42.440702+0000","last_unstale":"2024-09-21T07:34:44.833695+0000","last_undegraded":"2024-09-21T07:34:44.833695+0000","last_fullsized":"2024-09-21T07:34:44.833695+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:09.841490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":40,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833536+0000","last_change":"2024-09-21T07:34:24.258712+0000","last_active":"2024-09-21T07:34:44.833536+0000","last_peered":"2024-09-21T07:34:44.833536+0000","last_clean":"2024-09-21T07:34:44.833536+0000","last_became_active":"2024-09-21T07:34:23.274692+0000","last_became_peered":"2024-09-21T07:34:23.274692+0000","last_unstale":"2024-09-21T07:34:44.833536+0000","last_undegraded":"2024-09-21T07:34:44.833536+0000","last_fullsized":"2024-09-21T07:34:44.833536+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:43.279454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018051479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835466+0000","last_change":"2024-09-21T07:34:43.463236+0000","last_active":"2024-09-21T07:34:44.835466+0000","last_peered":"2024-09-21T07:34:44.835466+0000","last_clean":"2024-09-21T07:34:44.835466+0000","last_became_active":"2024-09-21T07:34:43.458917+0000","last_became_peered":"2024-09-21T07:34:43.458917+0000","last_unstale":"2024-09-21T07:34:44.835466+0000","last_undegraded":"2024-09-21T07:34:44.835466+0000","last_fullsized":"2024-09-21T07:34:44.835466+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:45:19.428476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:34:42.453151+0000","last_change":"2024-09-21T07:34:42.447422+0000","last_active":"2024-09-21T07:34:42.453151+0000","last_peered":"2024-09-21T07:34:42.453151+0000","last_clean":"2024-09-21T07:34:42.453151+0000","last_became_active":"2024-09-21T07:34:42.447132+0000","last_became_peered":"2024-09-21T07:34:42.447132+0000","last_unstale":"2024-09-21T07:34:42.453151+0000","last_undegraded":"2024-09-21T07:34:42.453151+0000","last_fullsized":"2024-09-21T07:34:42.453151+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:11.994352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":38,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833146+0000","last_change":"2024-09-21T07:34:25.084849+0000","last_active":"2024-09-21T07:34:44.833146+0000","last_peered":"2024-09-21T07:34:44.833146+0000","last_clean":"2024-09-21T07:34:44.833146+0000","last_became_active":"2024-09-21T07:34:23.277713+0000","last_became_peered":"2024-09-21T07:34:23.277713+0000","last_unstale":"2024-09-21T07:34:44.833146+0000","last_undegraded":"2024-09-21T07:34:44.833146+0000","last_fullsized":"2024-09-21T07:34:44.833146+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:05.811810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059079499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836091+0000","last_change":"2024-09-21T07:34:43.472697+0000","last_active":"2024-09-21T07:34:44.836091+0000","last_peered":"2024-09-21T07:34:44.836091+0000","last_clean":"2024-09-21T07:34:44.836091+0000","last_became_active":"2024-09-21T07:34:43.472276+0000","last_became_peered":"2024-09-21T07:34:43.472276+0000","last_unstale":"2024-09-21T07:34:44.836091+0000","last_undegraded":"2024-09-21T07:34:44.836091+0000","last_fullsized":"2024-09-21T07:34:44.836091+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:49.596333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833099+0000","last_change":"2024-09-21T07:34:42.453339+0000","last_active":"2024-09-21T07:34:44.833099+0000","last_peered":"2024-09-21T07:34:44.833099+0000","last_clean":"2024-09-21T07:34:44.833099+0000","last_became_active":"2024-09-21T07:34:42.452974+0000","last_became_peered":"2024-09-21T07:34:42.452974+0000","last_unstale":"2024-09-21T07:34:44.833099+0000","last_undegraded":"2024-09-21T07:34:44.833099+0000","last_fullsized":"2024-09-21T07:34:44.833099+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:22.788726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467897+0000","last_change":"2024-09-21T07:34:24.538610+0000","last_active":"2024-09-21T07:34:44.467897+0000","last_peered":"2024-09-21T07:34:44.467897+0000","last_clean":"2024-09-21T07:34:44.467897+0000","last_became_active":"2024-09-21T07:34:23.259700+0000","last_became_peered":"2024-09-21T07:34:23.259700+0000","last_unstale":"2024-09-21T07:34:44.467897+0000","last_undegraded":"2024-09-21T07:34:44.467897+0000","last_fullsized":"2024-09-21T07:34:44.467897+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:57.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070522399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834793+0000","last_change":"2024-09-21T07:34:43.474381+0000","last_active":"2024-09-21T07:34:44.834793+0000","last_peered":"2024-09-21T07:34:44.834793+0000","last_clean":"2024-09-21T07:34:44.834793+0000","last_became_active":"2024-09-21T07:34:43.473953+0000","last_became_peered":"2024-09-21T07:34:43.473953+0000","last_unstale":"2024-09-21T07:34:44.834793+0000","last_undegraded":"2024-09-21T07:34:44.834793+0000","last_fullsized":"2024-09-21T07:34:44.834793+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:31.362212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:34:42.450606+0000","last_change":"2024-09-21T07:34:42.444572+0000","last_active":"2024-09-21T07:34:42.450606+0000","last_peered":"2024-09-21T07:34:42.450606+0000","last_clean":"2024-09-21T07:34:42.450606+0000","last_became_active":"2024-09-21T07:34:42.444254+0000","last_became_peered":"2024-09-21T07:34:42.444254+0000","last_unstale":"2024-09-21T07:34:42.450606+0000","last_undegraded":"2024-09-21T07:34:42.450606+0000","last_fullsized":"2024-09-21T07:34:42.450606+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:08.176679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835079+0000","last_change":"2024-09-21T07:34:24.597597+0000","last_active":"2024-09-21T07:34:44.835079+0000","last_peered":"2024-09-21T07:34:44.835079+0000","last_clean":"2024-09-21T07:34:44.835079+0000","last_became_active":"2024-09-21T07:34:23.635204+0000","last_became_peered":"2024-09-21T07:34:23.635204+0000","last_unstale":"2024-09-21T07:34:44.835079+0000","last_undegraded":"2024-09-21T07:34:44.835079+0000","last_fullsized":"2024-09-21T07:34:44.835079+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:48.644789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081139199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467728+0000","last_change":"2024-09-21T07:34:43.460645+0000","last_active":"2024-09-21T07:34:44.467728+0000","last_peered":"2024-09-21T07:34:44.467728+0000","last_clean":"2024-09-21T07:34:44.467728+0000","last_became_active":"2024-09-21T07:34:43.460320+0000","last_became_peered":"2024-09-21T07:34:43.460320+0000","last_unstale":"2024-09-21T07:34:44.467728+0000","last_undegraded":"2024-09-21T07:34:44.467728+0000","last_fullsized":"2024-09-21T07:34:44.467728+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:54.644949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"48'1","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465954+0000","last_change":"2024-09-21T07:34:42.446762+0000","last_active":"2024-09-21T07:34:44.465954+0000","last_peered":"2024-09-21T07:34:44.465954+0000","last_clean":"2024-09-21T07:34:44.465954+0000","last_became_active":"2024-09-21T07:34:42.446320+0000","last_became_peered":"2024-09-21T07:34:42.446320+0000","last_unstale":"2024-09-21T07:34:44.465954+0000","last_undegraded":"2024-09-21T07:34:44.465954+0000","last_fullsized":"2024-09-21T07:34:44.465954+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:46.161972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":40,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835380+0000","last_change":"2024-09-21T07:34:24.259026+0000","last_active":"2024-09-21T07:34:44.835380+0000","last_peered":"2024-09-21T07:34:44.835380+0000","last_clean":"2024-09-21T07:34:44.835380+0000","last_became_active":"2024-09-21T07:34:23.635456+0000","last_became_peered":"2024-09-21T07:34:23.635456+0000","last_unstale":"2024-09-21T07:34:44.835380+0000","last_undegraded":"2024-09-21T07:34:44.835380+0000","last_fullsized":"2024-09-21T07:34:44.835380+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:09.592752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018654419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-21T07:34:42.434916+0000","last_change":"2024-09-21T07:34:42.434916+0000","last_active":"2024-09-21T07:34:42.415959+0000","last_peered":"2024-09-21T07:34:42.415959+0000","last_clean":"2024-09-21T07:34:42.415959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:34:42.434916+0000","last_undegraded":"2024-09-21T07:34:42.434916+0000","last_fullsized":"2024-09-21T07:34:42.434916+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"48'2","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467319+0000","last_change":"2024-09-21T07:34:42.443581+0000","last_active":"2024-09-21T07:34:44.467319+0000","last_peered":"2024-09-21T07:34:44.467319+0000","last_clean":"2024-09-21T07:34:44.467319+0000","last_became_active":"2024-09-21T07:34:42.443284+0000","last_became_peered":"2024-09-21T07:34:42.443284+0000","last_unstale":"2024-09-21T07:34:44.467319+0000","last_undegraded":"2024-09-21T07:34:44.467319+0000","last_fullsized":"2024-09-21T07:34:44.467319+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:59:14.692524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835966+0000","last_change":"2024-09-21T07:34:24.260300+0000","last_active":"2024-09-21T07:34:44.835966+0000","last_peered":"2024-09-21T07:34:44.835966+0000","last_clean":"2024-09-21T07:34:44.835966+0000","last_became_active":"2024-09-21T07:34:23.270128+0000","last_became_peered":"2024-09-21T07:34:23.270128+0000","last_unstale":"2024-09-21T07:34:44.835966+0000","last_undegraded":"2024-09-21T07:34:44.835966+0000","last_fullsized":"2024-09-21T07:34:44.835966+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:25.260931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00176651,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833363+0000","last_change":"2024-09-21T07:34:43.457202+0000","last_active":"2024-09-21T07:34:44.833363+0000","last_peered":"2024-09-21T07:34:44.833363+0000","last_clean":"2024-09-21T07:34:44.833363+0000","last_became_active":"2024-09-21T07:34:43.456851+0000","last_became_peered":"2024-09-21T07:34:43.456851+0000","last_unstale":"2024-09-21T07:34:44.833363+0000","last_undegraded":"2024-09-21T07:34:44.833363+0000","last_fullsized":"2024-09-21T07:34:44.833363+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:12.056767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"48'2","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.468447+0000","last_change":"2024-09-21T07:34:42.434252+0000","last_active":"2024-09-21T07:34:44.468447+0000","last_peered":"2024-09-21T07:34:44.468447+0000","last_clean":"2024-09-21T07:34:44.468447+0000","last_became_active":"2024-09-21T07:34:42.433936+0000","last_became_peered":"2024-09-21T07:34:42.433936+0000","last_unstale":"2024-09-21T07:34:44.468447+0000","last_undegraded":"2024-09-21T07:34:44.468447+0000","last_fullsized":"2024-09-21T07:34:44.468447+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:13.202719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467474+0000","last_change":"2024-09-21T07:34:25.028904+0000","last_active":"2024-09-21T07:34:44.467474+0000","last_peered":"2024-09-21T07:34:44.467474+0000","last_clean":"2024-09-21T07:34:44.467474+0000","last_became_active":"2024-09-21T07:34:23.260942+0000","last_became_peered":"2024-09-21T07:34:23.260942+0000","last_unstale":"2024-09-21T07:34:44.467474+0000","last_undegraded":"2024-09-21T07:34:44.467474+0000","last_fullsized":"2024-09-21T07:34:44.467474+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:22.709328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060553899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":1758,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509464,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32332,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339508,"statfs":{"total":96636764160,"available":96603656192,"internally_reserved":0,"allocated":1171456,"data_stored":832323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509465,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":532480,"data_stored":221422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509464,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":569344,"data_stored":241831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509465,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31764,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340076,"statfs":{"total":96636764160,"available":96604237824,"internally_reserved":0,"allocated":581632,"data_stored":243443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509466,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31756,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340084,"statfs":{"total":96636764160,"available":96604246016,"internally_reserved":0,"allocated":577536,"data_stored":241991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509466,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1171456,"data_stored":832255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542169,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31748,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340092,"statfs":{"total":96636764160,"available":96604254208,"internally_reserved":0,"allocated":573440,"data_stored":242413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542168,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31760,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340080,"statfs":{"total":96636764160,"available":96604241920,"internally_reserved":0,"allocated":585728,"data_stored":243465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:34:48.977 INFO:tasks.ceph.ceph_manager.ceph:PG 8.2c is not active+clean 2024-09-21T07:34:48.977 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 47, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:34:42.434570+0000', 'last_change': '2024-09-21T07:34:42.434570+0000', 'last_active': '2024-09-21T07:34:42.415959+0000', 'last_peered': '2024-09-21T07:34:42.415959+0000', 'last_clean': '2024-09-21T07:34:42.415959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:34:42.434570+0000', 'last_undegraded': '2024-09-21T07:34:42.434570+0000', 'last_fullsized': '2024-09-21T07:34:42.434570+0000', 'mapping_epoch': 47, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 47, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:34:42.415959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:34:42.415959+0000', 'last_clean_scrub_stamp': '2024-09-21T07:34:42.415959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T07:34:48.977 INFO:tasks.ceph.ceph_manager.ceph:PG 8.2b is not active+clean 2024-09-21T07:34:48.977 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 47, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:34:42.434971+0000', 'last_change': '2024-09-21T07:34:42.434971+0000', 'last_active': '2024-09-21T07:34:42.415959+0000', 'last_peered': '2024-09-21T07:34:42.415959+0000', 'last_clean': '2024-09-21T07:34:42.415959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:34:42.434971+0000', 'last_undegraded': '2024-09-21T07:34:42.434971+0000', 'last_fullsized': '2024-09-21T07:34:42.434971+0000', 'mapping_epoch': 47, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 47, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:34:42.415959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:34:42.415959+0000', 'last_clean_scrub_stamp': '2024-09-21T07:34:42.415959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T07:34:48.977 INFO:tasks.ceph.ceph_manager.ceph:PG 8.24 is not active+clean 2024-09-21T07:34:48.977 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 47, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:34:42.435611+0000', 'last_change': '2024-09-21T07:34:42.435611+0000', 'last_active': '2024-09-21T07:34:42.415959+0000', 'last_peered': '2024-09-21T07:34:42.415959+0000', 'last_clean': '2024-09-21T07:34:42.415959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:34:42.435611+0000', 'last_undegraded': '2024-09-21T07:34:42.435611+0000', 'last_fullsized': '2024-09-21T07:34:42.435611+0000', 'mapping_epoch': 47, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 47, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:34:42.415959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:34:42.415959+0000', 'last_clean_scrub_stamp': '2024-09-21T07:34:42.415959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T07:34:48.977 INFO:tasks.ceph.ceph_manager.ceph:PG 8.9 is not active+clean 2024-09-21T07:34:48.977 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 47, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:34:42.435086+0000', 'last_change': '2024-09-21T07:34:42.435086+0000', 'last_active': '2024-09-21T07:34:42.415959+0000', 'last_peered': '2024-09-21T07:34:42.415959+0000', 'last_clean': '2024-09-21T07:34:42.415959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:34:42.435086+0000', 'last_undegraded': '2024-09-21T07:34:42.435086+0000', 'last_fullsized': '2024-09-21T07:34:42.435086+0000', 'mapping_epoch': 47, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 47, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:34:42.415959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:34:42.415959+0000', 'last_clean_scrub_stamp': '2024-09-21T07:34:42.415959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T07:34:48.978 INFO:tasks.ceph.ceph_manager.ceph:PG 8.5 is not active+clean 2024-09-21T07:34:48.978 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 47, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:34:42.436882+0000', 'last_change': '2024-09-21T07:34:42.436882+0000', 'last_active': '2024-09-21T07:34:42.415959+0000', 'last_peered': '2024-09-21T07:34:42.415959+0000', 'last_clean': '2024-09-21T07:34:42.415959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:34:42.436882+0000', 'last_undegraded': '2024-09-21T07:34:42.436882+0000', 'last_fullsized': '2024-09-21T07:34:42.436882+0000', 'mapping_epoch': 47, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 47, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:34:42.415959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:34:42.415959+0000', 'last_clean_scrub_stamp': '2024-09-21T07:34:42.415959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T07:34:48.978 INFO:tasks.ceph.ceph_manager.ceph:PG 8.12 is not active+clean 2024-09-21T07:34:48.978 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 47, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:34:42.434916+0000', 'last_change': '2024-09-21T07:34:42.434916+0000', 'last_active': '2024-09-21T07:34:42.415959+0000', 'last_peered': '2024-09-21T07:34:42.415959+0000', 'last_clean': '2024-09-21T07:34:42.415959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:34:42.434916+0000', 'last_undegraded': '2024-09-21T07:34:42.434916+0000', 'last_fullsized': '2024-09-21T07:34:42.434916+0000', 'mapping_epoch': 47, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 47, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:34:42.415959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:34:42.415959+0000', 'last_clean_scrub_stamp': '2024-09-21T07:34:42.415959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T07:34:48.978 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T07:34:49.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.235+0000 7f7d82000640 1 -- 172.21.15.39:0/2177116653 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d7c0cc1e0 msgr2=0x7f7d7c0af3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:49.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.235+0000 7f7d82000640 1 --2- 172.21.15.39:0/2177116653 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d7c0cc1e0 0x7f7d7c0af3c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c072bc0 tx=0x7f7d6c02a560 comp rx=0 tx=0).stop 2024-09-21T07:34:49.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.236+0000 7f7d82000640 1 -- 172.21.15.39:0/2177116653 shutdown_connections 2024-09-21T07:34:49.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.236+0000 7f7d82000640 1 --2- 172.21.15.39:0/2177116653 >> v2:172.21.15.148:3300/0 conn(0x7f7d7c15d8a0 0x7f7d7c0b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:49.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.236+0000 7f7d82000640 1 --2- 172.21.15.39:0/2177116653 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d7c0cc1e0 0x7f7d7c0af3c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:49.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.236+0000 7f7d82000640 1 -- 172.21.15.39:0/2177116653 >> 172.21.15.39:0/2177116653 conn(0x7f7d7c06d7d0 msgr2=0x7f7d7c0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:49.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.236+0000 7f7d82000640 1 -- 172.21.15.39:0/2177116653 shutdown_connections 2024-09-21T07:34:49.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.236+0000 7f7d82000640 1 -- 172.21.15.39:0/2177116653 wait complete. 2024-09-21T07:34:49.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.238+0000 7f7d82000640 1 Processor -- start 2024-09-21T07:34:49.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.238+0000 7f7d82000640 1 -- start start 2024-09-21T07:34:49.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.239+0000 7f7d82000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d7c0cc1e0 0x7f7d7c1f5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:49.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.239+0000 7f7d82000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d7c0ccb50 0x7f7d7c0ae720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:49.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.239+0000 7f7d82000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7d7c15d8a0 0x7f7d7c073fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:49.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.240+0000 7f7d82000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c153520 con 0x7f7d7c0cc1e0 2024-09-21T07:34:49.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.240+0000 7f7d82000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c1533a0 con 0x7f7d7c0ccb50 2024-09-21T07:34:49.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.240+0000 7f7d82000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f7d7c1536a0 con 0x7f7d7c15d8a0 2024-09-21T07:34:49.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.240+0000 7f7d7b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d7c0cc1e0 0x7f7d7c1f5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:49.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.240+0000 7f7d7b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d7c0cc1e0 0x7f7d7c1f5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57758/0 (socket says 172.21.15.39:57758) 2024-09-21T07:34:49.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.240+0000 7f7d7b400640 1 -- 172.21.15.39:0/1100811770 learned_addr learned my addr 172.21.15.39:0/1100811770 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:49.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.240+0000 7f7d7b400640 1 -- 172.21.15.39:0/1100811770 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7d7c15d8a0 msgr2=0x7f7d7c073fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:49.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.240+0000 7f7d7b400640 1 --2- 172.21.15.39:0/1100811770 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7d7c15d8a0 0x7f7d7c073fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:49.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.240+0000 7f7d7b400640 1 -- 172.21.15.39:0/1100811770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d7c0ccb50 msgr2=0x7f7d7c0ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:49.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.240+0000 7f7d7b400640 1 --2- 172.21.15.39:0/1100811770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d7c0ccb50 0x7f7d7c0ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:49.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.240+0000 7f7d7b400640 1 -- 172.21.15.39:0/1100811770 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d7c1f9690 con 0x7f7d7c0cc1e0 2024-09-21T07:34:49.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.240+0000 7f7d7b400640 1 --2- 172.21.15.39:0/1100811770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d7c0cc1e0 0x7f7d7c1f5980 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c034040 tx=0x7f7d6c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:49.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.241+0000 7f7d73400640 1 -- 172.21.15.39:0/1100811770 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c03e030 con 0x7f7d7c0cc1e0 2024-09-21T07:34:49.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.241+0000 7f7d73400640 1 -- 172.21.15.39:0/1100811770 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d6c0047a0 con 0x7f7d7c0cc1e0 2024-09-21T07:34:49.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.241+0000 7f7d73400640 1 -- 172.21.15.39:0/1100811770 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c035040 con 0x7f7d7c0cc1e0 2024-09-21T07:34:49.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.241+0000 7f7d82000640 1 -- 172.21.15.39:0/1100811770 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d7c1f9970 con 0x7f7d7c0cc1e0 2024-09-21T07:34:49.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.241+0000 7f7d82000640 1 -- 172.21.15.39:0/1100811770 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d7c09c450 con 0x7f7d7c0cc1e0 2024-09-21T07:34:49.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.242+0000 7f7d82000640 1 -- 172.21.15.39:0/1100811770 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d7c153b50 con 0x7f7d7c0cc1e0 2024-09-21T07:34:49.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.243+0000 7f7d73400640 1 -- 172.21.15.39:0/1100811770 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7d6c041030 con 0x7f7d7c0cc1e0 2024-09-21T07:34:49.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.244+0000 7f7d73400640 1 --2- 172.21.15.39:0/1100811770 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7d5c078b90 0x7f7d5c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:49.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.244+0000 7f7d73400640 1 -- 172.21.15.39:0/1100811770 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6062+0+0 (secure 0 0 0) 0x7f7d6c0bb700 con 0x7f7d7c0cc1e0 2024-09-21T07:34:49.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.244+0000 7f7d7aa00640 1 --2- 172.21.15.39:0/1100811770 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7d5c078b90 0x7f7d5c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:49.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.245+0000 7f7d7aa00640 1 --2- 172.21.15.39:0/1100811770 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7d5c078b90 0x7f7d5c07b0a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c0d0fa0 tx=0x7f7d68005bd0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:49.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.248+0000 7f7d73400640 1 -- 172.21.15.39:0/1100811770 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7d6c083a60 con 0x7f7d7c0cc1e0 2024-09-21T07:34:49.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.520+0000 7f7d82000640 1 -- 172.21.15.39:0/1100811770 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f7d7c0b1270 con 0x7f7d7c0cc1e0 2024-09-21T07:34:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.521+0000 7f7d73400640 1 -- 172.21.15.39:0/1100811770 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1562 (secure 0 0 0) 0x7f7d6c083360 con 0x7f7d7c0cc1e0 2024-09-21T07:34:49.522 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:34:49.522 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":123,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":8,"num_up_osds":8,"osd_up_since":1726903982,"num_in_osds":8,"osd_in_since":1726903972,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":123},{"state_name":"creating+peering","count":6}],"num_pgs":129,"num_pools":4,"num_objects":20,"data_bytes":592126,"bytes_used":252772352,"bytes_avail":772841340928,"bytes_total":773094113280,"inactive_pgs_ratio":0.046511627733707428,"write_bytes_sec":2047,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":26,"btime":"2024-09-21T07:34:46:472997+0000","id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"a","status":"up:active","gid":5310}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-21T07:34:29.552326+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-21T07:34:49.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.524+0000 7f7d82000640 1 -- 172.21.15.39:0/1100811770 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7d5c078b90 msgr2=0x7f7d5c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:49.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.524+0000 7f7d82000640 1 --2- 172.21.15.39:0/1100811770 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7d5c078b90 0x7f7d5c07b0a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c0d0fa0 tx=0x7f7d68005bd0 comp rx=0 tx=0).stop 2024-09-21T07:34:49.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.524+0000 7f7d82000640 1 -- 172.21.15.39:0/1100811770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d7c0cc1e0 msgr2=0x7f7d7c1f5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:49.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.524+0000 7f7d82000640 1 --2- 172.21.15.39:0/1100811770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d7c0cc1e0 0x7f7d7c1f5980 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c034040 tx=0x7f7d6c0039e0 comp rx=0 tx=0).stop 2024-09-21T07:34:49.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.524+0000 7f7d82000640 1 -- 172.21.15.39:0/1100811770 shutdown_connections 2024-09-21T07:34:49.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.524+0000 7f7d82000640 1 --2- 172.21.15.39:0/1100811770 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7d5c078b90 0x7f7d5c07b0a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:49.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.524+0000 7f7d82000640 1 --2- 172.21.15.39:0/1100811770 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7d7c15d8a0 0x7f7d7c073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:49.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.524+0000 7f7d82000640 1 --2- 172.21.15.39:0/1100811770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d7c0ccb50 0x7f7d7c0ae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:49.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.524+0000 7f7d82000640 1 --2- 172.21.15.39:0/1100811770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d7c0cc1e0 0x7f7d7c1f5980 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:49.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.524+0000 7f7d82000640 1 -- 172.21.15.39:0/1100811770 >> 172.21.15.39:0/1100811770 conn(0x7f7d7c06d7d0 msgr2=0x7f7d7c0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:49.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.524+0000 7f7d82000640 1 -- 172.21.15.39:0/1100811770 shutdown_connections 2024-09-21T07:34:49.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.524+0000 7f7d82000640 1 -- 172.21.15.39:0/1100811770 wait complete. 2024-09-21T07:34:49.550 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-21T07:34:49.550 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:34:49.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.777+0000 7f73e7200640 1 -- 172.21.15.39:0/2158834375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73e815fbd0 msgr2=0x7f73e8152360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:49.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.777+0000 7f73e7200640 1 --2- 172.21.15.39:0/2158834375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73e815fbd0 0x7f73e8152360 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f73e80ccc60 tx=0x7f73d802a4a0 comp rx=0 tx=0).stop 2024-09-21T07:34:49.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.777+0000 7f73e7200640 1 -- 172.21.15.39:0/2158834375 shutdown_connections 2024-09-21T07:34:49.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.777+0000 7f73e7200640 1 --2- 172.21.15.39:0/2158834375 >> v2:172.21.15.148:3300/0 conn(0x7f73e815f660 0x7f73e80b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:49.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.777+0000 7f73e7200640 1 --2- 172.21.15.39:0/2158834375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73e815fbd0 0x7f73e8152360 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:49.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.777+0000 7f73e7200640 1 -- 172.21.15.39:0/2158834375 >> 172.21.15.39:0/2158834375 conn(0x7f73e806d410 msgr2=0x7f73e80b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:49.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.777+0000 7f73e7200640 1 -- 172.21.15.39:0/2158834375 shutdown_connections 2024-09-21T07:34:49.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.777+0000 7f73e7200640 1 -- 172.21.15.39:0/2158834375 wait complete. 2024-09-21T07:34:49.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.779+0000 7f73e7200640 1 Processor -- start 2024-09-21T07:34:49.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.779+0000 7f73e7200640 1 -- start start 2024-09-21T07:34:49.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.781+0000 7f73e7200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73e8072c30 0x7f73e81f8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:49.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.781+0000 7f73e7200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73e81484f0 0x7f73e814c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:49.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.781+0000 7f73e7200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f73e80d0120 0x7f73e80d0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:49.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.781+0000 7f73e7200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f73e80b3f50 con 0x7f73e8072c30 2024-09-21T07:34:49.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.781+0000 7f73e7200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f73e80b3dd0 con 0x7f73e81484f0 2024-09-21T07:34:49.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.781+0000 7f73e7200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f73e80b40d0 con 0x7f73e80d0120 2024-09-21T07:34:49.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.781+0000 7f73e5e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73e8072c30 0x7f73e81f8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:49.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.781+0000 7f73e5e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73e8072c30 0x7f73e81f8c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57772/0 (socket says 172.21.15.39:57772) 2024-09-21T07:34:49.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.781+0000 7f73e5e00640 1 -- 172.21.15.39:0/2525289681 learned_addr learned my addr 172.21.15.39:0/2525289681 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:49.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.781+0000 7f73e5400640 1 --2- 172.21.15.39:0/2525289681 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73e81484f0 0x7f73e814c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:49.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.781+0000 7f73e6800640 1 --2- 172.21.15.39:0/2525289681 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f73e80d0120 0x7f73e80d0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:49.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.782+0000 7f73e5e00640 1 -- 172.21.15.39:0/2525289681 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f73e80d0120 msgr2=0x7f73e80d0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:49.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.782+0000 7f73e5e00640 1 --2- 172.21.15.39:0/2525289681 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f73e80d0120 0x7f73e80d0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:49.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.782+0000 7f73e5e00640 1 -- 172.21.15.39:0/2525289681 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73e81484f0 msgr2=0x7f73e814c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:49.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.782+0000 7f73e5e00640 1 --2- 172.21.15.39:0/2525289681 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73e81484f0 0x7f73e814c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:49.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.782+0000 7f73e5e00640 1 -- 172.21.15.39:0/2525289681 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e80d0a70 con 0x7f73e8072c30 2024-09-21T07:34:49.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.782+0000 7f73e5e00640 1 --2- 172.21.15.39:0/2525289681 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73e8072c30 0x7f73e81f8c20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f73d8009780 tx=0x7f73d8033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:49.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.782+0000 7f73de000640 1 -- 172.21.15.39:0/2525289681 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d8031d60 con 0x7f73e8072c30 2024-09-21T07:34:49.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.782+0000 7f73de000640 1 -- 172.21.15.39:0/2525289681 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f73d8033e10 con 0x7f73e8072c30 2024-09-21T07:34:49.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.782+0000 7f73de000640 1 -- 172.21.15.39:0/2525289681 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d8005260 con 0x7f73e8072c30 2024-09-21T07:34:49.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.782+0000 7f73e7200640 1 -- 172.21.15.39:0/2525289681 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73e81f5610 con 0x7f73e8072c30 2024-09-21T07:34:49.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.783+0000 7f73e7200640 1 -- 172.21.15.39:0/2525289681 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73e815ad90 con 0x7f73e8072c30 2024-09-21T07:34:49.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.784+0000 7f73e7200640 1 -- 172.21.15.39:0/2525289681 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73e815fbd0 con 0x7f73e8072c30 2024-09-21T07:34:49.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.785+0000 7f73de000640 1 -- 172.21.15.39:0/2525289681 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f73d8005400 con 0x7f73e8072c30 2024-09-21T07:34:49.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.786+0000 7f73de000640 1 --2- 172.21.15.39:0/2525289681 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f73c8078b90 0x7f73c807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:49.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.787+0000 7f73de000640 1 -- 172.21.15.39:0/2525289681 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6062+0+0 (secure 0 0 0) 0x7f73d80bc1b0 con 0x7f73e8072c30 2024-09-21T07:34:49.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.787+0000 7f73e5400640 1 --2- 172.21.15.39:0/2525289681 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f73c8078b90 0x7f73c807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:49.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.788+0000 7f73e5400640 1 --2- 172.21.15.39:0/2525289681 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f73c8078b90 0x7f73c807b0a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f73e81f9c90 tx=0x7f73d4009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:49.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.791+0000 7f73de000640 1 -- 172.21.15.39:0/2525289681 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f73e80d0a70 con 0x7f73e8072c30 2024-09-21T07:34:49.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.985+0000 7f73e7200640 1 -- 172.21.15.39:0/2525289681 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f73e80cc2f0 con 0x7f73c8078b90 2024-09-21T07:34:49.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.991+0000 7f73de000640 1 -- 172.21.15.39:0/2525289681 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332927 (secure 0 0 0) 0x7f73e80cc2f0 con 0x7f73c8078b90 2024-09-21T07:34:49.998 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:34:50.000 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:34:50.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.996+0000 7f73e7200640 1 -- 172.21.15.39:0/2525289681 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f73c8078b90 msgr2=0x7f73c807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:50.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.996+0000 7f73e7200640 1 --2- 172.21.15.39:0/2525289681 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f73c8078b90 0x7f73c807b0a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f73e81f9c90 tx=0x7f73d4009040 comp rx=0 tx=0).stop 2024-09-21T07:34:50.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.996+0000 7f73e7200640 1 -- 172.21.15.39:0/2525289681 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73e8072c30 msgr2=0x7f73e81f8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:50.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.996+0000 7f73e7200640 1 --2- 172.21.15.39:0/2525289681 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73e8072c30 0x7f73e81f8c20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f73d8009780 tx=0x7f73d8033c20 comp rx=0 tx=0).stop 2024-09-21T07:34:50.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.996+0000 7f73e7200640 1 -- 172.21.15.39:0/2525289681 shutdown_connections 2024-09-21T07:34:50.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.996+0000 7f73e7200640 1 --2- 172.21.15.39:0/2525289681 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f73c8078b90 0x7f73c807b0a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:50.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.996+0000 7f73e7200640 1 --2- 172.21.15.39:0/2525289681 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f73e80d0120 0x7f73e80d0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:50.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.996+0000 7f73e7200640 1 --2- 172.21.15.39:0/2525289681 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73e81484f0 0x7f73e814c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:50.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.996+0000 7f73e7200640 1 --2- 172.21.15.39:0/2525289681 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73e8072c30 0x7f73e81f8c20 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:50.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.996+0000 7f73e7200640 1 -- 172.21.15.39:0/2525289681 >> 172.21.15.39:0/2525289681 conn(0x7f73e806d410 msgr2=0x7f73e8150820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:50.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.996+0000 7f73e7200640 1 -- 172.21.15.39:0/2525289681 shutdown_connections 2024-09-21T07:34:50.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:49.996+0000 7f73e7200640 1 -- 172.21.15.39:0/2525289681 wait complete. 2024-09-21T07:34:50.019 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":98,"stamp":"2024-09-21T07:34:49.557457+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":219,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":246888,"kb_used_data":5672,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754727832,"statfs":{"total":773094113280,"available":772841299968,"internally_reserved":0,"allocated":5808128,"data_stored":3120204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.136837"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.466557+0000","last_change":"2024-09-21T07:34:43.473741+0000","last_active":"2024-09-21T07:34:44.466557+0000","last_peered":"2024-09-21T07:34:44.466557+0000","last_clean":"2024-09-21T07:34:44.466557+0000","last_became_active":"2024-09-21T07:34:43.473447+0000","last_became_peered":"2024-09-21T07:34:43.473447+0000","last_unstale":"2024-09-21T07:34:44.466557+0000","last_undegraded":"2024-09-21T07:34:44.466557+0000","last_fullsized":"2024-09-21T07:34:44.466557+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:00:23.404297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836408+0000","last_change":"2024-09-21T07:34:43.473362+0000","last_active":"2024-09-21T07:34:44.836408+0000","last_peered":"2024-09-21T07:34:44.836408+0000","last_clean":"2024-09-21T07:34:44.836408+0000","last_became_active":"2024-09-21T07:34:43.473083+0000","last_became_peered":"2024-09-21T07:34:43.473083+0000","last_unstale":"2024-09-21T07:34:44.836408+0000","last_undegraded":"2024-09-21T07:34:44.836408+0000","last_fullsized":"2024-09-21T07:34:44.836408+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:33.920441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834020+0000","last_change":"2024-09-21T07:34:43.481499+0000","last_active":"2024-09-21T07:34:44.834020+0000","last_peered":"2024-09-21T07:34:44.834020+0000","last_clean":"2024-09-21T07:34:44.834020+0000","last_became_active":"2024-09-21T07:34:43.478254+0000","last_became_peered":"2024-09-21T07:34:43.478254+0000","last_unstale":"2024-09-21T07:34:44.834020+0000","last_undegraded":"2024-09-21T07:34:44.834020+0000","last_fullsized":"2024-09-21T07:34:44.834020+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:03.654293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836742+0000","last_change":"2024-09-21T07:34:43.451459+0000","last_active":"2024-09-21T07:34:44.836742+0000","last_peered":"2024-09-21T07:34:44.836742+0000","last_clean":"2024-09-21T07:34:44.836742+0000","last_became_active":"2024-09-21T07:34:43.451121+0000","last_became_peered":"2024-09-21T07:34:43.451121+0000","last_unstale":"2024-09-21T07:34:44.836742+0000","last_undegraded":"2024-09-21T07:34:44.836742+0000","last_fullsized":"2024-09-21T07:34:44.836742+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:15.139818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834482+0000","last_change":"2024-09-21T07:34:43.455772+0000","last_active":"2024-09-21T07:34:44.834482+0000","last_peered":"2024-09-21T07:34:44.834482+0000","last_clean":"2024-09-21T07:34:44.834482+0000","last_became_active":"2024-09-21T07:34:43.454461+0000","last_became_peered":"2024-09-21T07:34:43.454461+0000","last_unstale":"2024-09-21T07:34:44.834482+0000","last_undegraded":"2024-09-21T07:34:44.834482+0000","last_fullsized":"2024-09-21T07:34:44.834482+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:03.456938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465593+0000","last_change":"2024-09-21T07:34:43.469533+0000","last_active":"2024-09-21T07:34:44.465593+0000","last_peered":"2024-09-21T07:34:44.465593+0000","last_clean":"2024-09-21T07:34:44.465593+0000","last_became_active":"2024-09-21T07:34:43.469336+0000","last_became_peered":"2024-09-21T07:34:43.469336+0000","last_unstale":"2024-09-21T07:34:44.465593+0000","last_undegraded":"2024-09-21T07:34:44.465593+0000","last_fullsized":"2024-09-21T07:34:44.465593+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:20.540854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833543+0000","last_change":"2024-09-21T07:34:43.483800+0000","last_active":"2024-09-21T07:34:44.833543+0000","last_peered":"2024-09-21T07:34:44.833543+0000","last_clean":"2024-09-21T07:34:44.833543+0000","last_became_active":"2024-09-21T07:34:43.483482+0000","last_became_peered":"2024-09-21T07:34:43.483482+0000","last_unstale":"2024-09-21T07:34:44.833543+0000","last_undegraded":"2024-09-21T07:34:44.833543+0000","last_fullsized":"2024-09-21T07:34:44.833543+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:20.152451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834422+0000","last_change":"2024-09-21T07:34:43.473653+0000","last_active":"2024-09-21T07:34:44.834422+0000","last_peered":"2024-09-21T07:34:44.834422+0000","last_clean":"2024-09-21T07:34:44.834422+0000","last_became_active":"2024-09-21T07:34:43.473348+0000","last_became_peered":"2024-09-21T07:34:43.473348+0000","last_unstale":"2024-09-21T07:34:44.834422+0000","last_undegraded":"2024-09-21T07:34:44.834422+0000","last_fullsized":"2024-09-21T07:34:44.834422+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:13.042429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835096+0000","last_change":"2024-09-21T07:34:43.483507+0000","last_active":"2024-09-21T07:34:44.835096+0000","last_peered":"2024-09-21T07:34:44.835096+0000","last_clean":"2024-09-21T07:34:44.835096+0000","last_became_active":"2024-09-21T07:34:43.482198+0000","last_became_peered":"2024-09-21T07:34:43.482198+0000","last_unstale":"2024-09-21T07:34:44.835096+0000","last_undegraded":"2024-09-21T07:34:44.835096+0000","last_fullsized":"2024-09-21T07:34:44.835096+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:45.957857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465790+0000","last_change":"2024-09-21T07:34:43.465258+0000","last_active":"2024-09-21T07:34:44.465790+0000","last_peered":"2024-09-21T07:34:44.465790+0000","last_clean":"2024-09-21T07:34:44.465790+0000","last_became_active":"2024-09-21T07:34:43.464922+0000","last_became_peered":"2024-09-21T07:34:43.464922+0000","last_unstale":"2024-09-21T07:34:44.465790+0000","last_undegraded":"2024-09-21T07:34:44.465790+0000","last_fullsized":"2024-09-21T07:34:44.465790+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:11.406345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467028+0000","last_change":"2024-09-21T07:34:43.449879+0000","last_active":"2024-09-21T07:34:44.467028+0000","last_peered":"2024-09-21T07:34:44.467028+0000","last_clean":"2024-09-21T07:34:44.467028+0000","last_became_active":"2024-09-21T07:34:43.448365+0000","last_became_peered":"2024-09-21T07:34:43.448365+0000","last_unstale":"2024-09-21T07:34:44.467028+0000","last_undegraded":"2024-09-21T07:34:44.467028+0000","last_fullsized":"2024-09-21T07:34:44.467028+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:29:00.993972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834222+0000","last_change":"2024-09-21T07:34:43.460167+0000","last_active":"2024-09-21T07:34:44.834222+0000","last_peered":"2024-09-21T07:34:44.834222+0000","last_clean":"2024-09-21T07:34:44.834222+0000","last_became_active":"2024-09-21T07:34:43.457296+0000","last_became_peered":"2024-09-21T07:34:43.457296+0000","last_unstale":"2024-09-21T07:34:44.834222+0000","last_undegraded":"2024-09-21T07:34:44.834222+0000","last_fullsized":"2024-09-21T07:34:44.834222+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:53:57.871505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.466129+0000","last_change":"2024-09-21T07:34:43.465519+0000","last_active":"2024-09-21T07:34:44.466129+0000","last_peered":"2024-09-21T07:34:44.466129+0000","last_clean":"2024-09-21T07:34:44.466129+0000","last_became_active":"2024-09-21T07:34:43.464802+0000","last_became_peered":"2024-09-21T07:34:43.464802+0000","last_unstale":"2024-09-21T07:34:44.466129+0000","last_undegraded":"2024-09-21T07:34:44.466129+0000","last_fullsized":"2024-09-21T07:34:44.466129+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:59.062262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834033+0000","last_change":"2024-09-21T07:34:43.478575+0000","last_active":"2024-09-21T07:34:44.834033+0000","last_peered":"2024-09-21T07:34:44.834033+0000","last_clean":"2024-09-21T07:34:44.834033+0000","last_became_active":"2024-09-21T07:34:43.478187+0000","last_became_peered":"2024-09-21T07:34:43.478187+0000","last_unstale":"2024-09-21T07:34:44.834033+0000","last_undegraded":"2024-09-21T07:34:44.834033+0000","last_fullsized":"2024-09-21T07:34:44.834033+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:44.866369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836848+0000","last_change":"2024-09-21T07:34:43.467093+0000","last_active":"2024-09-21T07:34:44.836848+0000","last_peered":"2024-09-21T07:34:44.836848+0000","last_clean":"2024-09-21T07:34:44.836848+0000","last_became_active":"2024-09-21T07:34:43.466773+0000","last_became_peered":"2024-09-21T07:34:43.466773+0000","last_unstale":"2024-09-21T07:34:44.836848+0000","last_undegraded":"2024-09-21T07:34:44.836848+0000","last_fullsized":"2024-09-21T07:34:44.836848+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:12:02.180192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465260+0000","last_change":"2024-09-21T07:34:43.462637+0000","last_active":"2024-09-21T07:34:44.465260+0000","last_peered":"2024-09-21T07:34:44.465260+0000","last_clean":"2024-09-21T07:34:44.465260+0000","last_became_active":"2024-09-21T07:34:43.462301+0000","last_became_peered":"2024-09-21T07:34:43.462301+0000","last_unstale":"2024-09-21T07:34:44.465260+0000","last_undegraded":"2024-09-21T07:34:44.465260+0000","last_fullsized":"2024-09-21T07:34:44.465260+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:42.168665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834898+0000","last_change":"2024-09-21T07:34:43.484342+0000","last_active":"2024-09-21T07:34:44.834898+0000","last_peered":"2024-09-21T07:34:44.834898+0000","last_clean":"2024-09-21T07:34:44.834898+0000","last_became_active":"2024-09-21T07:34:43.480452+0000","last_became_peered":"2024-09-21T07:34:43.480452+0000","last_unstale":"2024-09-21T07:34:44.834898+0000","last_undegraded":"2024-09-21T07:34:44.834898+0000","last_fullsized":"2024-09-21T07:34:44.834898+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:30.267103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836590+0000","last_change":"2024-09-21T07:34:43.472890+0000","last_active":"2024-09-21T07:34:44.836590+0000","last_peered":"2024-09-21T07:34:44.836590+0000","last_clean":"2024-09-21T07:34:44.836590+0000","last_became_active":"2024-09-21T07:34:43.472576+0000","last_became_peered":"2024-09-21T07:34:43.472576+0000","last_unstale":"2024-09-21T07:34:44.836590+0000","last_undegraded":"2024-09-21T07:34:44.836590+0000","last_fullsized":"2024-09-21T07:34:44.836590+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:24.719441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834303+0000","last_change":"2024-09-21T07:34:43.476774+0000","last_active":"2024-09-21T07:34:44.834303+0000","last_peered":"2024-09-21T07:34:44.834303+0000","last_clean":"2024-09-21T07:34:44.834303+0000","last_became_active":"2024-09-21T07:34:43.473147+0000","last_became_peered":"2024-09-21T07:34:43.473147+0000","last_unstale":"2024-09-21T07:34:44.834303+0000","last_undegraded":"2024-09-21T07:34:44.834303+0000","last_fullsized":"2024-09-21T07:34:44.834303+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:10.661941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835069+0000","last_change":"2024-09-21T07:34:43.467366+0000","last_active":"2024-09-21T07:34:44.835069+0000","last_peered":"2024-09-21T07:34:44.835069+0000","last_clean":"2024-09-21T07:34:44.835069+0000","last_became_active":"2024-09-21T07:34:43.467032+0000","last_became_peered":"2024-09-21T07:34:43.467032+0000","last_unstale":"2024-09-21T07:34:44.835069+0000","last_undegraded":"2024-09-21T07:34:44.835069+0000","last_fullsized":"2024-09-21T07:34:44.835069+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:47.164384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833778+0000","last_change":"2024-09-21T07:34:43.466216+0000","last_active":"2024-09-21T07:34:44.833778+0000","last_peered":"2024-09-21T07:34:44.833778+0000","last_clean":"2024-09-21T07:34:44.833778+0000","last_became_active":"2024-09-21T07:34:43.465960+0000","last_became_peered":"2024-09-21T07:34:43.465960+0000","last_unstale":"2024-09-21T07:34:44.833778+0000","last_undegraded":"2024-09-21T07:34:44.833778+0000","last_fullsized":"2024-09-21T07:34:44.833778+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:24.439384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834766+0000","last_change":"2024-09-21T07:34:43.483312+0000","last_active":"2024-09-21T07:34:44.834766+0000","last_peered":"2024-09-21T07:34:44.834766+0000","last_clean":"2024-09-21T07:34:44.834766+0000","last_became_active":"2024-09-21T07:34:43.483003+0000","last_became_peered":"2024-09-21T07:34:43.483003+0000","last_unstale":"2024-09-21T07:34:44.834766+0000","last_undegraded":"2024-09-21T07:34:44.834766+0000","last_fullsized":"2024-09-21T07:34:44.834766+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:33.782633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467544+0000","last_change":"2024-09-21T07:34:43.449273+0000","last_active":"2024-09-21T07:34:44.467544+0000","last_peered":"2024-09-21T07:34:44.467544+0000","last_clean":"2024-09-21T07:34:44.467544+0000","last_became_active":"2024-09-21T07:34:43.448944+0000","last_became_peered":"2024-09-21T07:34:43.448944+0000","last_unstale":"2024-09-21T07:34:44.467544+0000","last_undegraded":"2024-09-21T07:34:44.467544+0000","last_fullsized":"2024-09-21T07:34:44.467544+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:56.540403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833398+0000","last_change":"2024-09-21T07:34:43.479357+0000","last_active":"2024-09-21T07:34:44.833398+0000","last_peered":"2024-09-21T07:34:44.833398+0000","last_clean":"2024-09-21T07:34:44.833398+0000","last_became_active":"2024-09-21T07:34:43.478997+0000","last_became_peered":"2024-09-21T07:34:43.478997+0000","last_unstale":"2024-09-21T07:34:44.833398+0000","last_undegraded":"2024-09-21T07:34:44.833398+0000","last_fullsized":"2024-09-21T07:34:44.833398+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:54.644949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835486+0000","last_change":"2024-09-21T07:34:43.486869+0000","last_active":"2024-09-21T07:34:44.835486+0000","last_peered":"2024-09-21T07:34:44.835486+0000","last_clean":"2024-09-21T07:34:44.835486+0000","last_became_active":"2024-09-21T07:34:43.486351+0000","last_became_peered":"2024-09-21T07:34:43.486351+0000","last_unstale":"2024-09-21T07:34:44.835486+0000","last_undegraded":"2024-09-21T07:34:44.835486+0000","last_fullsized":"2024-09-21T07:34:44.835486+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:21:02.910848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833742+0000","last_change":"2024-09-21T07:34:43.463196+0000","last_active":"2024-09-21T07:34:44.833742+0000","last_peered":"2024-09-21T07:34:44.833742+0000","last_clean":"2024-09-21T07:34:44.833742+0000","last_became_active":"2024-09-21T07:34:43.458559+0000","last_became_peered":"2024-09-21T07:34:43.458559+0000","last_unstale":"2024-09-21T07:34:44.833742+0000","last_undegraded":"2024-09-21T07:34:44.833742+0000","last_fullsized":"2024-09-21T07:34:44.833742+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:37.233429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467280+0000","last_change":"2024-09-21T07:34:43.483549+0000","last_active":"2024-09-21T07:34:44.467280+0000","last_peered":"2024-09-21T07:34:44.467280+0000","last_clean":"2024-09-21T07:34:44.467280+0000","last_became_active":"2024-09-21T07:34:43.472653+0000","last_became_peered":"2024-09-21T07:34:43.472653+0000","last_unstale":"2024-09-21T07:34:44.467280+0000","last_undegraded":"2024-09-21T07:34:44.467280+0000","last_fullsized":"2024-09-21T07:34:44.467280+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:38.814916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834808+0000","last_change":"2024-09-21T07:34:43.459996+0000","last_active":"2024-09-21T07:34:44.834808+0000","last_peered":"2024-09-21T07:34:44.834808+0000","last_clean":"2024-09-21T07:34:44.834808+0000","last_became_active":"2024-09-21T07:34:43.459068+0000","last_became_peered":"2024-09-21T07:34:43.459068+0000","last_unstale":"2024-09-21T07:34:44.834808+0000","last_undegraded":"2024-09-21T07:34:44.834808+0000","last_fullsized":"2024-09-21T07:34:44.834808+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:49.571297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833338+0000","last_change":"2024-09-21T07:34:43.489357+0000","last_active":"2024-09-21T07:34:44.833338+0000","last_peered":"2024-09-21T07:34:44.833338+0000","last_clean":"2024-09-21T07:34:44.833338+0000","last_became_active":"2024-09-21T07:34:43.483224+0000","last_became_peered":"2024-09-21T07:34:43.483224+0000","last_unstale":"2024-09-21T07:34:44.833338+0000","last_undegraded":"2024-09-21T07:34:44.833338+0000","last_fullsized":"2024-09-21T07:34:44.833338+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:05.212534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835873+0000","last_change":"2024-09-21T07:34:43.460376+0000","last_active":"2024-09-21T07:34:44.835873+0000","last_peered":"2024-09-21T07:34:44.835873+0000","last_clean":"2024-09-21T07:34:44.835873+0000","last_became_active":"2024-09-21T07:34:43.458320+0000","last_became_peered":"2024-09-21T07:34:43.458320+0000","last_unstale":"2024-09-21T07:34:44.835873+0000","last_undegraded":"2024-09-21T07:34:44.835873+0000","last_fullsized":"2024-09-21T07:34:44.835873+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:35.190652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833745+0000","last_change":"2024-09-21T07:34:43.485349+0000","last_active":"2024-09-21T07:34:44.833745+0000","last_peered":"2024-09-21T07:34:44.833745+0000","last_clean":"2024-09-21T07:34:44.833745+0000","last_became_active":"2024-09-21T07:34:43.483623+0000","last_became_peered":"2024-09-21T07:34:43.483623+0000","last_unstale":"2024-09-21T07:34:44.833745+0000","last_undegraded":"2024-09-21T07:34:44.833745+0000","last_fullsized":"2024-09-21T07:34:44.833745+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:02.084719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467658+0000","last_change":"2024-09-21T07:34:43.471970+0000","last_active":"2024-09-21T07:34:44.467658+0000","last_peered":"2024-09-21T07:34:44.467658+0000","last_clean":"2024-09-21T07:34:44.467658+0000","last_became_active":"2024-09-21T07:34:43.465515+0000","last_became_peered":"2024-09-21T07:34:43.465515+0000","last_unstale":"2024-09-21T07:34:44.467658+0000","last_undegraded":"2024-09-21T07:34:44.467658+0000","last_fullsized":"2024-09-21T07:34:44.467658+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:59:53.819831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834269+0000","last_change":"2024-09-21T07:34:43.466446+0000","last_active":"2024-09-21T07:34:44.834269+0000","last_peered":"2024-09-21T07:34:44.834269+0000","last_clean":"2024-09-21T07:34:44.834269+0000","last_became_active":"2024-09-21T07:34:43.466153+0000","last_became_peered":"2024-09-21T07:34:43.466153+0000","last_unstale":"2024-09-21T07:34:44.834269+0000","last_undegraded":"2024-09-21T07:34:44.834269+0000","last_fullsized":"2024-09-21T07:34:44.834269+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:17.974647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"49'1","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834973+0000","last_change":"2024-09-21T07:34:42.447496+0000","last_active":"2024-09-21T07:34:44.834973+0000","last_peered":"2024-09-21T07:34:44.834973+0000","last_clean":"2024-09-21T07:34:44.834973+0000","last_became_active":"2024-09-21T07:34:42.447148+0000","last_became_peered":"2024-09-21T07:34:42.447148+0000","last_unstale":"2024-09-21T07:34:44.834973+0000","last_undegraded":"2024-09-21T07:34:44.834973+0000","last_fullsized":"2024-09-21T07:34:44.834973+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:42:06.750595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":163,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835508+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:34:44.835508+0000","last_peered":"2024-09-21T07:34:44.835508+0000","last_clean":"2024-09-21T07:34:44.835508+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:34:44.835508+0000","last_undegraded":"2024-09-21T07:34:44.835508+0000","last_fullsized":"2024-09-21T07:34:44.835508+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.468769+0000","last_change":"2024-09-21T07:34:24.539248+0000","last_active":"2024-09-21T07:34:44.468769+0000","last_peered":"2024-09-21T07:34:44.468769+0000","last_clean":"2024-09-21T07:34:44.468769+0000","last_became_active":"2024-09-21T07:34:23.273465+0000","last_became_peered":"2024-09-21T07:34:23.273465+0000","last_unstale":"2024-09-21T07:34:44.468769+0000","last_undegraded":"2024-09-21T07:34:44.468769+0000","last_fullsized":"2024-09-21T07:34:44.468769+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:55.517452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060627700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834170+0000","last_change":"2024-09-21T07:34:43.491609+0000","last_active":"2024-09-21T07:34:44.834170+0000","last_peered":"2024-09-21T07:34:44.834170+0000","last_clean":"2024-09-21T07:34:44.834170+0000","last_became_active":"2024-09-21T07:34:43.490138+0000","last_became_peered":"2024-09-21T07:34:43.490138+0000","last_unstale":"2024-09-21T07:34:44.834170+0000","last_undegraded":"2024-09-21T07:34:44.834170+0000","last_fullsized":"2024-09-21T07:34:44.834170+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:16.015947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833429+0000","last_change":"2024-09-21T07:34:42.448815+0000","last_active":"2024-09-21T07:34:44.833429+0000","last_peered":"2024-09-21T07:34:44.833429+0000","last_clean":"2024-09-21T07:34:44.833429+0000","last_became_active":"2024-09-21T07:34:42.448484+0000","last_became_peered":"2024-09-21T07:34:42.448484+0000","last_unstale":"2024-09-21T07:34:44.833429+0000","last_undegraded":"2024-09-21T07:34:44.833429+0000","last_fullsized":"2024-09-21T07:34:44.833429+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:15.522307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833978+0000","last_change":"2024-09-21T07:34:24.597274+0000","last_active":"2024-09-21T07:34:44.833978+0000","last_peered":"2024-09-21T07:34:44.833978+0000","last_clean":"2024-09-21T07:34:44.833978+0000","last_became_active":"2024-09-21T07:34:23.279560+0000","last_became_peered":"2024-09-21T07:34:23.279560+0000","last_unstale":"2024-09-21T07:34:44.833978+0000","last_undegraded":"2024-09-21T07:34:44.833978+0000","last_fullsized":"2024-09-21T07:34:44.833978+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:15.845265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059699799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.468039+0000","last_change":"2024-09-21T07:34:43.468683+0000","last_active":"2024-09-21T07:34:44.468039+0000","last_peered":"2024-09-21T07:34:44.468039+0000","last_clean":"2024-09-21T07:34:44.468039+0000","last_became_active":"2024-09-21T07:34:43.468377+0000","last_became_peered":"2024-09-21T07:34:43.468377+0000","last_unstale":"2024-09-21T07:34:44.468039+0000","last_undegraded":"2024-09-21T07:34:44.468039+0000","last_fullsized":"2024-09-21T07:34:44.468039+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:02.084719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834944+0000","last_change":"2024-09-21T07:34:42.442148+0000","last_active":"2024-09-21T07:34:44.834944+0000","last_peered":"2024-09-21T07:34:44.834944+0000","last_clean":"2024-09-21T07:34:44.834944+0000","last_became_active":"2024-09-21T07:34:42.441839+0000","last_became_peered":"2024-09-21T07:34:42.441839+0000","last_unstale":"2024-09-21T07:34:44.834944+0000","last_undegraded":"2024-09-21T07:34:44.834944+0000","last_fullsized":"2024-09-21T07:34:44.834944+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:35:57.065813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836188+0000","last_change":"2024-09-21T07:34:24.259906+0000","last_active":"2024-09-21T07:34:44.836188+0000","last_peered":"2024-09-21T07:34:44.836188+0000","last_clean":"2024-09-21T07:34:44.836188+0000","last_became_active":"2024-09-21T07:34:23.635282+0000","last_became_peered":"2024-09-21T07:34:23.635282+0000","last_unstale":"2024-09-21T07:34:44.836188+0000","last_undegraded":"2024-09-21T07:34:44.836188+0000","last_fullsized":"2024-09-21T07:34:44.836188+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:00.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001527664,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834033+0000","last_change":"2024-09-21T07:34:43.491274+0000","last_active":"2024-09-21T07:34:44.834033+0000","last_peered":"2024-09-21T07:34:44.834033+0000","last_clean":"2024-09-21T07:34:44.834033+0000","last_became_active":"2024-09-21T07:34:43.489063+0000","last_became_peered":"2024-09-21T07:34:43.489063+0000","last_unstale":"2024-09-21T07:34:44.834033+0000","last_undegraded":"2024-09-21T07:34:44.834033+0000","last_fullsized":"2024-09-21T07:34:44.834033+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:13.315568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833450+0000","last_change":"2024-09-21T07:34:42.437481+0000","last_active":"2024-09-21T07:34:44.833450+0000","last_peered":"2024-09-21T07:34:44.833450+0000","last_clean":"2024-09-21T07:34:44.833450+0000","last_became_active":"2024-09-21T07:34:42.437130+0000","last_became_peered":"2024-09-21T07:34:42.437130+0000","last_unstale":"2024-09-21T07:34:44.833450+0000","last_undegraded":"2024-09-21T07:34:44.833450+0000","last_fullsized":"2024-09-21T07:34:44.833450+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:55.611339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833360+0000","last_change":"2024-09-21T07:34:25.085454+0000","last_active":"2024-09-21T07:34:44.833360+0000","last_peered":"2024-09-21T07:34:44.833360+0000","last_clean":"2024-09-21T07:34:44.833360+0000","last_became_active":"2024-09-21T07:34:23.279205+0000","last_became_peered":"2024-09-21T07:34:23.279205+0000","last_unstale":"2024-09-21T07:34:44.833360+0000","last_undegraded":"2024-09-21T07:34:44.833360+0000","last_fullsized":"2024-09-21T07:34:44.833360+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:54.231666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095320599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835309+0000","last_change":"2024-09-21T07:34:43.457980+0000","last_active":"2024-09-21T07:34:44.835309+0000","last_peered":"2024-09-21T07:34:44.835309+0000","last_clean":"2024-09-21T07:34:44.835309+0000","last_became_active":"2024-09-21T07:34:43.457695+0000","last_became_peered":"2024-09-21T07:34:43.457695+0000","last_unstale":"2024-09-21T07:34:44.835309+0000","last_undegraded":"2024-09-21T07:34:44.835309+0000","last_fullsized":"2024-09-21T07:34:44.835309+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:01.492292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467870+0000","last_change":"2024-09-21T07:34:42.438107+0000","last_active":"2024-09-21T07:34:44.467870+0000","last_peered":"2024-09-21T07:34:44.467870+0000","last_clean":"2024-09-21T07:34:44.467870+0000","last_became_active":"2024-09-21T07:34:42.437794+0000","last_became_peered":"2024-09-21T07:34:42.437794+0000","last_unstale":"2024-09-21T07:34:44.467870+0000","last_undegraded":"2024-09-21T07:34:44.467870+0000","last_fullsized":"2024-09-21T07:34:44.467870+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:24.649223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465509+0000","last_change":"2024-09-21T07:34:25.028521+0000","last_active":"2024-09-21T07:34:44.465509+0000","last_peered":"2024-09-21T07:34:44.465509+0000","last_clean":"2024-09-21T07:34:44.465509+0000","last_became_active":"2024-09-21T07:34:23.635467+0000","last_became_peered":"2024-09-21T07:34:23.635467+0000","last_unstale":"2024-09-21T07:34:44.465509+0000","last_undegraded":"2024-09-21T07:34:44.465509+0000","last_fullsized":"2024-09-21T07:34:44.465509+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:33.729123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000660558,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467665+0000","last_change":"2024-09-21T07:34:43.465578+0000","last_active":"2024-09-21T07:34:44.467665+0000","last_peered":"2024-09-21T07:34:44.467665+0000","last_clean":"2024-09-21T07:34:44.467665+0000","last_became_active":"2024-09-21T07:34:43.465071+0000","last_became_peered":"2024-09-21T07:34:43.465071+0000","last_unstale":"2024-09-21T07:34:44.467665+0000","last_undegraded":"2024-09-21T07:34:44.467665+0000","last_fullsized":"2024-09-21T07:34:44.467665+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:20.152451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"48'2","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834309+0000","last_change":"2024-09-21T07:34:42.445927+0000","last_active":"2024-09-21T07:34:44.834309+0000","last_peered":"2024-09-21T07:34:44.834309+0000","last_clean":"2024-09-21T07:34:44.834309+0000","last_became_active":"2024-09-21T07:34:42.445620+0000","last_became_peered":"2024-09-21T07:34:42.445620+0000","last_unstale":"2024-09-21T07:34:44.834309+0000","last_undegraded":"2024-09-21T07:34:44.834309+0000","last_fullsized":"2024-09-21T07:34:44.834309+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:48.022549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":40,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835533+0000","last_change":"2024-09-21T07:34:24.260046+0000","last_active":"2024-09-21T07:34:44.835533+0000","last_peered":"2024-09-21T07:34:44.835533+0000","last_clean":"2024-09-21T07:34:44.835533+0000","last_became_active":"2024-09-21T07:34:23.273431+0000","last_became_peered":"2024-09-21T07:34:23.273431+0000","last_unstale":"2024-09-21T07:34:44.835533+0000","last_undegraded":"2024-09-21T07:34:44.835533+0000","last_fullsized":"2024-09-21T07:34:44.835533+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:18.876057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019803310000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465631+0000","last_change":"2024-09-21T07:34:43.462550+0000","last_active":"2024-09-21T07:34:44.465631+0000","last_peered":"2024-09-21T07:34:44.465631+0000","last_clean":"2024-09-21T07:34:44.465631+0000","last_became_active":"2024-09-21T07:34:43.462210+0000","last_became_peered":"2024-09-21T07:34:43.462210+0000","last_unstale":"2024-09-21T07:34:44.465631+0000","last_undegraded":"2024-09-21T07:34:44.465631+0000","last_fullsized":"2024-09-21T07:34:44.465631+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:56.480314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836267+0000","last_change":"2024-09-21T07:34:42.452761+0000","last_active":"2024-09-21T07:34:44.836267+0000","last_peered":"2024-09-21T07:34:44.836267+0000","last_clean":"2024-09-21T07:34:44.836267+0000","last_became_active":"2024-09-21T07:34:42.452400+0000","last_became_peered":"2024-09-21T07:34:42.452400+0000","last_unstale":"2024-09-21T07:34:44.836267+0000","last_undegraded":"2024-09-21T07:34:44.836267+0000","last_fullsized":"2024-09-21T07:34:44.836267+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:19:05.278221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465673+0000","last_change":"2024-09-21T07:34:25.028640+0000","last_active":"2024-09-21T07:34:44.465673+0000","last_peered":"2024-09-21T07:34:44.465673+0000","last_clean":"2024-09-21T07:34:44.465673+0000","last_became_active":"2024-09-21T07:34:23.260950+0000","last_became_peered":"2024-09-21T07:34:23.260950+0000","last_unstale":"2024-09-21T07:34:44.465673+0000","last_undegraded":"2024-09-21T07:34:44.465673+0000","last_fullsized":"2024-09-21T07:34:44.465673+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:23:42.121398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055135099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833638+0000","last_change":"2024-09-21T07:34:43.453996+0000","last_active":"2024-09-21T07:34:44.833638+0000","last_peered":"2024-09-21T07:34:44.833638+0000","last_clean":"2024-09-21T07:34:44.833638+0000","last_became_active":"2024-09-21T07:34:43.453666+0000","last_became_peered":"2024-09-21T07:34:43.453666+0000","last_unstale":"2024-09-21T07:34:44.833638+0000","last_undegraded":"2024-09-21T07:34:44.833638+0000","last_fullsized":"2024-09-21T07:34:44.833638+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:31.445807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834426+0000","last_change":"2024-09-21T07:34:42.453822+0000","last_active":"2024-09-21T07:34:44.834426+0000","last_peered":"2024-09-21T07:34:44.834426+0000","last_clean":"2024-09-21T07:34:44.834426+0000","last_became_active":"2024-09-21T07:34:42.453499+0000","last_became_peered":"2024-09-21T07:34:42.453499+0000","last_unstale":"2024-09-21T07:34:44.834426+0000","last_undegraded":"2024-09-21T07:34:44.834426+0000","last_fullsized":"2024-09-21T07:34:44.834426+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:39:29.733953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836757+0000","last_change":"2024-09-21T07:34:24.260665+0000","last_active":"2024-09-21T07:34:44.836757+0000","last_peered":"2024-09-21T07:34:44.836757+0000","last_clean":"2024-09-21T07:34:44.836757+0000","last_became_active":"2024-09-21T07:34:23.635909+0000","last_became_peered":"2024-09-21T07:34:23.635909+0000","last_unstale":"2024-09-21T07:34:44.836757+0000","last_undegraded":"2024-09-21T07:34:44.836757+0000","last_fullsized":"2024-09-21T07:34:44.836757+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:44.536885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.468489+0000","last_change":"2024-09-21T07:34:43.469198+0000","last_active":"2024-09-21T07:34:44.468489+0000","last_peered":"2024-09-21T07:34:44.468489+0000","last_clean":"2024-09-21T07:34:44.468489+0000","last_became_active":"2024-09-21T07:34:43.467622+0000","last_became_peered":"2024-09-21T07:34:43.467622+0000","last_unstale":"2024-09-21T07:34:44.468489+0000","last_undegraded":"2024-09-21T07:34:44.468489+0000","last_fullsized":"2024-09-21T07:34:44.468489+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:44.866369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"48'2","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465974+0000","last_change":"2024-09-21T07:34:42.438031+0000","last_active":"2024-09-21T07:34:44.465974+0000","last_peered":"2024-09-21T07:34:44.465974+0000","last_clean":"2024-09-21T07:34:44.465974+0000","last_became_active":"2024-09-21T07:34:42.436986+0000","last_became_peered":"2024-09-21T07:34:42.436986+0000","last_unstale":"2024-09-21T07:34:44.465974+0000","last_undegraded":"2024-09-21T07:34:44.465974+0000","last_fullsized":"2024-09-21T07:34:44.465974+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:51.213865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467759+0000","last_change":"2024-09-21T07:34:24.538845+0000","last_active":"2024-09-21T07:34:44.467759+0000","last_peered":"2024-09-21T07:34:44.467759+0000","last_clean":"2024-09-21T07:34:44.467759+0000","last_became_active":"2024-09-21T07:34:23.636372+0000","last_became_peered":"2024-09-21T07:34:23.636372+0000","last_unstale":"2024-09-21T07:34:44.467759+0000","last_undegraded":"2024-09-21T07:34:44.467759+0000","last_fullsized":"2024-09-21T07:34:44.467759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:29.393813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841451,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833953+0000","last_change":"2024-09-21T07:34:43.465622+0000","last_active":"2024-09-21T07:34:44.833953+0000","last_peered":"2024-09-21T07:34:44.833953+0000","last_clean":"2024-09-21T07:34:44.833953+0000","last_became_active":"2024-09-21T07:34:43.465306+0000","last_became_peered":"2024-09-21T07:34:43.465306+0000","last_unstale":"2024-09-21T07:34:44.833953+0000","last_undegraded":"2024-09-21T07:34:44.833953+0000","last_fullsized":"2024-09-21T07:34:44.833953+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:29.579523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834840+0000","last_change":"2024-09-21T07:34:42.447519+0000","last_active":"2024-09-21T07:34:44.834840+0000","last_peered":"2024-09-21T07:34:44.834840+0000","last_clean":"2024-09-21T07:34:44.834840+0000","last_became_active":"2024-09-21T07:34:42.447129+0000","last_became_peered":"2024-09-21T07:34:42.447129+0000","last_unstale":"2024-09-21T07:34:44.834840+0000","last_undegraded":"2024-09-21T07:34:44.834840+0000","last_fullsized":"2024-09-21T07:34:44.834840+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:30.464417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":40,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833703+0000","last_change":"2024-09-21T07:34:24.259686+0000","last_active":"2024-09-21T07:34:44.833703+0000","last_peered":"2024-09-21T07:34:44.833703+0000","last_clean":"2024-09-21T07:34:44.833703+0000","last_became_active":"2024-09-21T07:34:23.274405+0000","last_became_peered":"2024-09-21T07:34:23.274405+0000","last_unstale":"2024-09-21T07:34:44.833703+0000","last_undegraded":"2024-09-21T07:34:44.833703+0000","last_fullsized":"2024-09-21T07:34:44.833703+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:26.304693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022009920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834760+0000","last_change":"2024-09-21T07:34:43.476482+0000","last_active":"2024-09-21T07:34:44.834760+0000","last_peered":"2024-09-21T07:34:44.834760+0000","last_clean":"2024-09-21T07:34:44.834760+0000","last_became_active":"2024-09-21T07:34:43.476189+0000","last_became_peered":"2024-09-21T07:34:43.476189+0000","last_unstale":"2024-09-21T07:34:44.834760+0000","last_undegraded":"2024-09-21T07:34:44.834760+0000","last_fullsized":"2024-09-21T07:34:44.834760+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:08:42.688308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833369+0000","last_change":"2024-09-21T07:34:42.456020+0000","last_active":"2024-09-21T07:34:44.833369+0000","last_peered":"2024-09-21T07:34:44.833369+0000","last_clean":"2024-09-21T07:34:44.833369+0000","last_became_active":"2024-09-21T07:34:42.455399+0000","last_became_peered":"2024-09-21T07:34:42.455399+0000","last_unstale":"2024-09-21T07:34:44.833369+0000","last_undegraded":"2024-09-21T07:34:44.833369+0000","last_fullsized":"2024-09-21T07:34:44.833369+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:57.601710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835457+0000","last_change":"2024-09-21T07:34:24.258754+0000","last_active":"2024-09-21T07:34:44.835457+0000","last_peered":"2024-09-21T07:34:44.835457+0000","last_clean":"2024-09-21T07:34:44.835457+0000","last_became_active":"2024-09-21T07:34:23.272657+0000","last_became_peered":"2024-09-21T07:34:23.272657+0000","last_unstale":"2024-09-21T07:34:44.835457+0000","last_undegraded":"2024-09-21T07:34:44.835457+0000","last_fullsized":"2024-09-21T07:34:44.835457+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:05.477099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067143200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.466771+0000","last_change":"2024-09-21T07:34:43.472869+0000","last_active":"2024-09-21T07:34:44.466771+0000","last_peered":"2024-09-21T07:34:44.466771+0000","last_clean":"2024-09-21T07:34:44.466771+0000","last_became_active":"2024-09-21T07:34:43.472514+0000","last_became_peered":"2024-09-21T07:34:43.472514+0000","last_unstale":"2024-09-21T07:34:44.466771+0000","last_undegraded":"2024-09-21T07:34:44.466771+0000","last_fullsized":"2024-09-21T07:34:44.466771+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:15:22.567958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.466329+0000","last_change":"2024-09-21T07:34:42.443208+0000","last_active":"2024-09-21T07:34:44.466329+0000","last_peered":"2024-09-21T07:34:44.466329+0000","last_clean":"2024-09-21T07:34:44.466329+0000","last_became_active":"2024-09-21T07:34:42.442891+0000","last_became_peered":"2024-09-21T07:34:42.442891+0000","last_unstale":"2024-09-21T07:34:44.466329+0000","last_undegraded":"2024-09-21T07:34:44.466329+0000","last_fullsized":"2024-09-21T07:34:44.466329+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:02.347260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":38,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833706+0000","last_change":"2024-09-21T07:34:25.085232+0000","last_active":"2024-09-21T07:34:44.833706+0000","last_peered":"2024-09-21T07:34:44.833706+0000","last_clean":"2024-09-21T07:34:44.833706+0000","last_became_active":"2024-09-21T07:34:23.277807+0000","last_became_peered":"2024-09-21T07:34:23.277807+0000","last_unstale":"2024-09-21T07:34:44.833706+0000","last_undegraded":"2024-09-21T07:34:44.833706+0000","last_fullsized":"2024-09-21T07:34:44.833706+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:36.362919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086619099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835299+0000","last_change":"2024-09-21T07:34:43.486460+0000","last_active":"2024-09-21T07:34:44.835299+0000","last_peered":"2024-09-21T07:34:44.835299+0000","last_clean":"2024-09-21T07:34:44.835299+0000","last_became_active":"2024-09-21T07:34:43.486148+0000","last_became_peered":"2024-09-21T07:34:43.486148+0000","last_unstale":"2024-09-21T07:34:44.835299+0000","last_undegraded":"2024-09-21T07:34:44.835299+0000","last_fullsized":"2024-09-21T07:34:44.835299+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:31.997216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"48'1","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835624+0000","last_change":"2024-09-21T07:34:42.450738+0000","last_active":"2024-09-21T07:34:44.835624+0000","last_peered":"2024-09-21T07:34:44.835624+0000","last_clean":"2024-09-21T07:34:44.835624+0000","last_became_active":"2024-09-21T07:34:42.450413+0000","last_became_peered":"2024-09-21T07:34:42.450413+0000","last_unstale":"2024-09-21T07:34:44.835624+0000","last_undegraded":"2024-09-21T07:34:44.835624+0000","last_fullsized":"2024-09-21T07:34:44.835624+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:48.569857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835043+0000","last_change":"2024-09-21T07:34:24.597672+0000","last_active":"2024-09-21T07:34:44.835043+0000","last_peered":"2024-09-21T07:34:44.835043+0000","last_clean":"2024-09-21T07:34:44.835043+0000","last_became_active":"2024-09-21T07:34:23.267770+0000","last_became_peered":"2024-09-21T07:34:23.267770+0000","last_unstale":"2024-09-21T07:34:44.835043+0000","last_undegraded":"2024-09-21T07:34:44.835043+0000","last_fullsized":"2024-09-21T07:34:44.835043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:51.629476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086378500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465282+0000","last_change":"2024-09-21T07:34:43.478411+0000","last_active":"2024-09-21T07:34:44.465282+0000","last_peered":"2024-09-21T07:34:44.465282+0000","last_clean":"2024-09-21T07:34:44.465282+0000","last_became_active":"2024-09-21T07:34:43.478106+0000","last_became_peered":"2024-09-21T07:34:43.478106+0000","last_unstale":"2024-09-21T07:34:44.465282+0000","last_undegraded":"2024-09-21T07:34:44.465282+0000","last_fullsized":"2024-09-21T07:34:44.465282+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:41.320870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"48'2","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467059+0000","last_change":"2024-09-21T07:34:42.449630+0000","last_active":"2024-09-21T07:34:44.467059+0000","last_peered":"2024-09-21T07:34:44.467059+0000","last_clean":"2024-09-21T07:34:44.467059+0000","last_became_active":"2024-09-21T07:34:42.448013+0000","last_became_peered":"2024-09-21T07:34:42.448013+0000","last_unstale":"2024-09-21T07:34:44.467059+0000","last_undegraded":"2024-09-21T07:34:44.467059+0000","last_fullsized":"2024-09-21T07:34:44.467059+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:42.900618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":38,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467361+0000","last_change":"2024-09-21T07:34:24.260181+0000","last_active":"2024-09-21T07:34:44.467361+0000","last_peered":"2024-09-21T07:34:44.467361+0000","last_clean":"2024-09-21T07:34:44.467361+0000","last_became_active":"2024-09-21T07:34:23.275580+0000","last_became_peered":"2024-09-21T07:34:23.275580+0000","last_unstale":"2024-09-21T07:34:44.467361+0000","last_undegraded":"2024-09-21T07:34:44.467361+0000","last_fullsized":"2024-09-21T07:34:44.467361+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:49.857622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014706980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834317+0000","last_change":"2024-09-21T07:34:43.479707+0000","last_active":"2024-09-21T07:34:44.834317+0000","last_peered":"2024-09-21T07:34:44.834317+0000","last_clean":"2024-09-21T07:34:44.834317+0000","last_became_active":"2024-09-21T07:34:43.479311+0000","last_became_peered":"2024-09-21T07:34:43.479311+0000","last_unstale":"2024-09-21T07:34:44.834317+0000","last_undegraded":"2024-09-21T07:34:44.834317+0000","last_fullsized":"2024-09-21T07:34:44.834317+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:03:54.452233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"48'1","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834163+0000","last_change":"2024-09-21T07:34:42.455379+0000","last_active":"2024-09-21T07:34:44.834163+0000","last_peered":"2024-09-21T07:34:44.834163+0000","last_clean":"2024-09-21T07:34:44.834163+0000","last_became_active":"2024-09-21T07:34:42.455144+0000","last_became_peered":"2024-09-21T07:34:42.455144+0000","last_unstale":"2024-09-21T07:34:44.834163+0000","last_undegraded":"2024-09-21T07:34:44.834163+0000","last_fullsized":"2024-09-21T07:34:44.834163+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:14:56.454976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":40,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834497+0000","last_change":"2024-09-21T07:34:24.259112+0000","last_active":"2024-09-21T07:34:44.834497+0000","last_peered":"2024-09-21T07:34:44.834497+0000","last_clean":"2024-09-21T07:34:44.834497+0000","last_became_active":"2024-09-21T07:34:23.635444+0000","last_became_peered":"2024-09-21T07:34:23.635444+0000","last_unstale":"2024-09-21T07:34:44.834497+0000","last_undegraded":"2024-09-21T07:34:44.834497+0000","last_fullsized":"2024-09-21T07:34:44.834497+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:46.255385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001952636,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465803+0000","last_change":"2024-09-21T07:34:43.476982+0000","last_active":"2024-09-21T07:34:44.465803+0000","last_peered":"2024-09-21T07:34:44.465803+0000","last_clean":"2024-09-21T07:34:44.465803+0000","last_became_active":"2024-09-21T07:34:43.476670+0000","last_became_peered":"2024-09-21T07:34:43.476670+0000","last_unstale":"2024-09-21T07:34:44.465803+0000","last_undegraded":"2024-09-21T07:34:44.465803+0000","last_fullsized":"2024-09-21T07:34:44.465803+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:04:42.140238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834385+0000","last_change":"2024-09-21T07:34:42.444915+0000","last_active":"2024-09-21T07:34:44.834385+0000","last_peered":"2024-09-21T07:34:44.834385+0000","last_clean":"2024-09-21T07:34:44.834385+0000","last_became_active":"2024-09-21T07:34:42.444506+0000","last_became_peered":"2024-09-21T07:34:42.444506+0000","last_unstale":"2024-09-21T07:34:44.834385+0000","last_undegraded":"2024-09-21T07:34:44.834385+0000","last_fullsized":"2024-09-21T07:34:44.834385+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:14:38.610112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.468302+0000","last_change":"2024-09-21T07:34:24.539152+0000","last_active":"2024-09-21T07:34:44.468302+0000","last_peered":"2024-09-21T07:34:44.468302+0000","last_clean":"2024-09-21T07:34:44.468302+0000","last_became_active":"2024-09-21T07:34:23.259573+0000","last_became_peered":"2024-09-21T07:34:23.259573+0000","last_unstale":"2024-09-21T07:34:44.468302+0000","last_undegraded":"2024-09-21T07:34:44.468302+0000","last_fullsized":"2024-09-21T07:34:44.468302+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:26.986900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058108400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.468175+0000","last_change":"2024-09-21T07:34:43.469411+0000","last_active":"2024-09-21T07:34:44.468175+0000","last_peered":"2024-09-21T07:34:44.468175+0000","last_clean":"2024-09-21T07:34:44.468175+0000","last_became_active":"2024-09-21T07:34:43.469098+0000","last_became_peered":"2024-09-21T07:34:43.469098+0000","last_unstale":"2024-09-21T07:34:44.468175+0000","last_undegraded":"2024-09-21T07:34:44.468175+0000","last_fullsized":"2024-09-21T07:34:44.468175+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:03.654293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"48'2","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834059+0000","last_change":"2024-09-21T07:34:42.444868+0000","last_active":"2024-09-21T07:34:44.834059+0000","last_peered":"2024-09-21T07:34:44.834059+0000","last_clean":"2024-09-21T07:34:44.834059+0000","last_became_active":"2024-09-21T07:34:42.444574+0000","last_became_peered":"2024-09-21T07:34:42.444574+0000","last_unstale":"2024-09-21T07:34:44.834059+0000","last_undegraded":"2024-09-21T07:34:44.834059+0000","last_fullsized":"2024-09-21T07:34:44.834059+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:14.168800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835810+0000","last_change":"2024-09-21T07:34:24.259902+0000","last_active":"2024-09-21T07:34:44.835810+0000","last_peered":"2024-09-21T07:34:44.835810+0000","last_clean":"2024-09-21T07:34:44.835810+0000","last_became_active":"2024-09-21T07:34:23.271902+0000","last_became_peered":"2024-09-21T07:34:23.271902+0000","last_unstale":"2024-09-21T07:34:44.835810+0000","last_undegraded":"2024-09-21T07:34:44.835810+0000","last_fullsized":"2024-09-21T07:34:44.835810+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:59.439525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001686358,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834649+0000","last_change":"2024-09-21T07:34:43.488076+0000","last_active":"2024-09-21T07:34:44.834649+0000","last_peered":"2024-09-21T07:34:44.834649+0000","last_clean":"2024-09-21T07:34:44.834649+0000","last_became_active":"2024-09-21T07:34:43.487739+0000","last_became_peered":"2024-09-21T07:34:43.487739+0000","last_unstale":"2024-09-21T07:34:44.834649+0000","last_undegraded":"2024-09-21T07:34:44.834649+0000","last_fullsized":"2024-09-21T07:34:44.834649+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:35:38.440936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834727+0000","last_change":"2024-09-21T07:34:42.452085+0000","last_active":"2024-09-21T07:34:44.834727+0000","last_peered":"2024-09-21T07:34:44.834727+0000","last_clean":"2024-09-21T07:34:44.834727+0000","last_became_active":"2024-09-21T07:34:42.451768+0000","last_became_peered":"2024-09-21T07:34:42.451768+0000","last_unstale":"2024-09-21T07:34:44.834727+0000","last_undegraded":"2024-09-21T07:34:44.834727+0000","last_fullsized":"2024-09-21T07:34:44.834727+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:41.414518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":40,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833639+0000","last_change":"2024-09-21T07:34:24.259662+0000","last_active":"2024-09-21T07:34:44.833639+0000","last_peered":"2024-09-21T07:34:44.833639+0000","last_clean":"2024-09-21T07:34:44.833639+0000","last_became_active":"2024-09-21T07:34:23.278812+0000","last_became_peered":"2024-09-21T07:34:23.278812+0000","last_unstale":"2024-09-21T07:34:44.833639+0000","last_undegraded":"2024-09-21T07:34:44.833639+0000","last_fullsized":"2024-09-21T07:34:44.833639+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:49.829974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356463,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835746+0000","last_change":"2024-09-21T07:34:43.470322+0000","last_active":"2024-09-21T07:34:44.835746+0000","last_peered":"2024-09-21T07:34:44.835746+0000","last_clean":"2024-09-21T07:34:44.835746+0000","last_became_active":"2024-09-21T07:34:43.469955+0000","last_became_peered":"2024-09-21T07:34:43.469955+0000","last_unstale":"2024-09-21T07:34:44.835746+0000","last_undegraded":"2024-09-21T07:34:44.835746+0000","last_fullsized":"2024-09-21T07:34:44.835746+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:30.715538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835746+0000","last_change":"2024-09-21T07:34:42.441191+0000","last_active":"2024-09-21T07:34:44.835746+0000","last_peered":"2024-09-21T07:34:44.835746+0000","last_clean":"2024-09-21T07:34:44.835746+0000","last_became_active":"2024-09-21T07:34:42.440881+0000","last_became_peered":"2024-09-21T07:34:42.440881+0000","last_unstale":"2024-09-21T07:34:44.835746+0000","last_undegraded":"2024-09-21T07:34:44.835746+0000","last_fullsized":"2024-09-21T07:34:44.835746+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:45.066904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":38,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833256+0000","last_change":"2024-09-21T07:34:25.084861+0000","last_active":"2024-09-21T07:34:44.833256+0000","last_peered":"2024-09-21T07:34:44.833256+0000","last_clean":"2024-09-21T07:34:44.833256+0000","last_became_active":"2024-09-21T07:34:23.277708+0000","last_became_peered":"2024-09-21T07:34:23.277708+0000","last_unstale":"2024-09-21T07:34:44.833256+0000","last_undegraded":"2024-09-21T07:34:44.833256+0000","last_fullsized":"2024-09-21T07:34:44.833256+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:40.816488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000597566,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835256+0000","last_change":"2024-09-21T07:34:43.480401+0000","last_active":"2024-09-21T07:34:44.835256+0000","last_peered":"2024-09-21T07:34:44.835256+0000","last_clean":"2024-09-21T07:34:44.835256+0000","last_became_active":"2024-09-21T07:34:43.479988+0000","last_became_peered":"2024-09-21T07:34:43.479988+0000","last_unstale":"2024-09-21T07:34:44.835256+0000","last_undegraded":"2024-09-21T07:34:44.835256+0000","last_fullsized":"2024-09-21T07:34:44.835256+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:03.660689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"48'8","reported_seq":23,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834495+0000","last_change":"2024-09-21T07:34:42.449884+0000","last_active":"2024-09-21T07:34:44.834495+0000","last_peered":"2024-09-21T07:34:44.834495+0000","last_clean":"2024-09-21T07:34:44.834495+0000","last_became_active":"2024-09-21T07:34:42.448499+0000","last_became_peered":"2024-09-21T07:34:42.448499+0000","last_unstale":"2024-09-21T07:34:44.834495+0000","last_undegraded":"2024-09-21T07:34:44.834495+0000","last_fullsized":"2024-09-21T07:34:44.834495+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:58.240549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":40,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833217+0000","last_change":"2024-09-21T07:34:24.258709+0000","last_active":"2024-09-21T07:34:44.833217+0000","last_peered":"2024-09-21T07:34:44.833217+0000","last_clean":"2024-09-21T07:34:44.833217+0000","last_became_active":"2024-09-21T07:34:23.279042+0000","last_became_peered":"2024-09-21T07:34:23.279042+0000","last_unstale":"2024-09-21T07:34:44.833217+0000","last_undegraded":"2024-09-21T07:34:44.833217+0000","last_fullsized":"2024-09-21T07:34:44.833217+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:01.762117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019282100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833735+0000","last_change":"2024-09-21T07:34:43.466176+0000","last_active":"2024-09-21T07:34:44.833735+0000","last_peered":"2024-09-21T07:34:44.833735+0000","last_clean":"2024-09-21T07:34:44.833735+0000","last_became_active":"2024-09-21T07:34:43.464214+0000","last_became_peered":"2024-09-21T07:34:43.464214+0000","last_unstale":"2024-09-21T07:34:44.833735+0000","last_undegraded":"2024-09-21T07:34:44.833735+0000","last_fullsized":"2024-09-21T07:34:44.833735+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:22.282533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836035+0000","last_change":"2024-09-21T07:34:42.442308+0000","last_active":"2024-09-21T07:34:44.836035+0000","last_peered":"2024-09-21T07:34:44.836035+0000","last_clean":"2024-09-21T07:34:44.836035+0000","last_became_active":"2024-09-21T07:34:42.442018+0000","last_became_peered":"2024-09-21T07:34:42.442018+0000","last_unstale":"2024-09-21T07:34:44.836035+0000","last_undegraded":"2024-09-21T07:34:44.836035+0000","last_fullsized":"2024-09-21T07:34:44.836035+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:53.642537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":38,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835026+0000","last_change":"2024-09-21T07:34:25.085635+0000","last_active":"2024-09-21T07:34:44.835026+0000","last_peered":"2024-09-21T07:34:44.835026+0000","last_clean":"2024-09-21T07:34:44.835026+0000","last_became_active":"2024-09-21T07:34:23.273566+0000","last_became_peered":"2024-09-21T07:34:23.273566+0000","last_unstale":"2024-09-21T07:34:44.835026+0000","last_undegraded":"2024-09-21T07:34:44.835026+0000","last_fullsized":"2024-09-21T07:34:44.835026+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:14.142272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079897300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467666+0000","last_change":"2024-09-21T07:34:43.472808+0000","last_active":"2024-09-21T07:34:44.467666+0000","last_peered":"2024-09-21T07:34:44.467666+0000","last_clean":"2024-09-21T07:34:44.467666+0000","last_became_active":"2024-09-21T07:34:43.466752+0000","last_became_peered":"2024-09-21T07:34:43.466752+0000","last_unstale":"2024-09-21T07:34:44.467666+0000","last_undegraded":"2024-09-21T07:34:44.467666+0000","last_fullsized":"2024-09-21T07:34:44.467666+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:19:06.280633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833887+0000","last_change":"2024-09-21T07:34:42.447266+0000","last_active":"2024-09-21T07:34:44.833887+0000","last_peered":"2024-09-21T07:34:44.833887+0000","last_clean":"2024-09-21T07:34:44.833887+0000","last_became_active":"2024-09-21T07:34:42.446823+0000","last_became_peered":"2024-09-21T07:34:42.446823+0000","last_unstale":"2024-09-21T07:34:44.833887+0000","last_undegraded":"2024-09-21T07:34:44.833887+0000","last_fullsized":"2024-09-21T07:34:44.833887+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:16.281729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835166+0000","last_change":"2024-09-21T07:34:24.597926+0000","last_active":"2024-09-21T07:34:44.835166+0000","last_peered":"2024-09-21T07:34:44.835166+0000","last_clean":"2024-09-21T07:34:44.835166+0000","last_became_active":"2024-09-21T07:34:23.268926+0000","last_became_peered":"2024-09-21T07:34:23.268926+0000","last_unstale":"2024-09-21T07:34:44.835166+0000","last_undegraded":"2024-09-21T07:34:44.835166+0000","last_fullsized":"2024-09-21T07:34:44.835166+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:46.190752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101851,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834479+0000","last_change":"2024-09-21T07:34:43.481754+0000","last_active":"2024-09-21T07:34:44.834479+0000","last_peered":"2024-09-21T07:34:44.834479+0000","last_clean":"2024-09-21T07:34:44.834479+0000","last_became_active":"2024-09-21T07:34:43.481460+0000","last_became_peered":"2024-09-21T07:34:43.481460+0000","last_unstale":"2024-09-21T07:34:44.834479+0000","last_undegraded":"2024-09-21T07:34:44.834479+0000","last_fullsized":"2024-09-21T07:34:44.834479+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:17:36.213769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"48'3","reported_seq":18,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836119+0000","last_change":"2024-09-21T07:34:42.450507+0000","last_active":"2024-09-21T07:34:44.836119+0000","last_peered":"2024-09-21T07:34:44.836119+0000","last_clean":"2024-09-21T07:34:44.836119+0000","last_became_active":"2024-09-21T07:34:42.449613+0000","last_became_peered":"2024-09-21T07:34:42.449613+0000","last_unstale":"2024-09-21T07:34:44.836119+0000","last_undegraded":"2024-09-21T07:34:44.836119+0000","last_fullsized":"2024-09-21T07:34:44.836119+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:59:52.817419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":38,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.466765+0000","last_change":"2024-09-21T07:34:24.259794+0000","last_active":"2024-09-21T07:34:44.466765+0000","last_peered":"2024-09-21T07:34:44.466765+0000","last_clean":"2024-09-21T07:34:44.466765+0000","last_became_active":"2024-09-21T07:34:23.265679+0000","last_became_peered":"2024-09-21T07:34:23.265679+0000","last_unstale":"2024-09-21T07:34:44.466765+0000","last_undegraded":"2024-09-21T07:34:44.466765+0000","last_fullsized":"2024-09-21T07:34:44.466765+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:02.575796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001298518,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834659+0000","last_change":"2024-09-21T07:34:43.457653+0000","last_active":"2024-09-21T07:34:44.834659+0000","last_peered":"2024-09-21T07:34:44.834659+0000","last_clean":"2024-09-21T07:34:44.834659+0000","last_became_active":"2024-09-21T07:34:43.457347+0000","last_became_peered":"2024-09-21T07:34:43.457347+0000","last_unstale":"2024-09-21T07:34:44.834659+0000","last_undegraded":"2024-09-21T07:34:44.834659+0000","last_fullsized":"2024-09-21T07:34:44.834659+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:02:38.909862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"48'6","reported_seq":21,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834377+0000","last_change":"2024-09-21T07:34:42.449115+0000","last_active":"2024-09-21T07:34:44.834377+0000","last_peered":"2024-09-21T07:34:44.834377+0000","last_clean":"2024-09-21T07:34:44.834377+0000","last_became_active":"2024-09-21T07:34:42.448834+0000","last_became_peered":"2024-09-21T07:34:42.448834+0000","last_unstale":"2024-09-21T07:34:44.834377+0000","last_undegraded":"2024-09-21T07:34:44.834377+0000","last_fullsized":"2024-09-21T07:34:44.834377+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:49.800427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836377+0000","last_change":"2024-09-21T07:34:24.260330+0000","last_active":"2024-09-21T07:34:44.836377+0000","last_peered":"2024-09-21T07:34:44.836377+0000","last_clean":"2024-09-21T07:34:44.836377+0000","last_became_active":"2024-09-21T07:34:23.637210+0000","last_became_peered":"2024-09-21T07:34:23.637210+0000","last_unstale":"2024-09-21T07:34:44.836377+0000","last_undegraded":"2024-09-21T07:34:44.836377+0000","last_fullsized":"2024-09-21T07:34:44.836377+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.488451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001537093,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834186+0000","last_change":"2024-09-21T07:34:43.481741+0000","last_active":"2024-09-21T07:34:44.834186+0000","last_peered":"2024-09-21T07:34:44.834186+0000","last_clean":"2024-09-21T07:34:44.834186+0000","last_became_active":"2024-09-21T07:34:43.481368+0000","last_became_peered":"2024-09-21T07:34:43.481368+0000","last_unstale":"2024-09-21T07:34:44.834186+0000","last_undegraded":"2024-09-21T07:34:44.834186+0000","last_fullsized":"2024-09-21T07:34:44.834186+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:16.752080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"48'1","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833850+0000","last_change":"2024-09-21T07:34:42.455980+0000","last_active":"2024-09-21T07:34:44.833850+0000","last_peered":"2024-09-21T07:34:44.833850+0000","last_clean":"2024-09-21T07:34:44.833850+0000","last_became_active":"2024-09-21T07:34:42.455368+0000","last_became_peered":"2024-09-21T07:34:42.455368+0000","last_unstale":"2024-09-21T07:34:44.833850+0000","last_undegraded":"2024-09-21T07:34:44.833850+0000","last_fullsized":"2024-09-21T07:34:44.833850+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:07:34.444440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833846+0000","last_change":"2024-09-21T07:34:24.597286+0000","last_active":"2024-09-21T07:34:44.833846+0000","last_peered":"2024-09-21T07:34:44.833846+0000","last_clean":"2024-09-21T07:34:44.833846+0000","last_became_active":"2024-09-21T07:34:23.267133+0000","last_became_peered":"2024-09-21T07:34:23.267133+0000","last_unstale":"2024-09-21T07:34:44.833846+0000","last_undegraded":"2024-09-21T07:34:44.833846+0000","last_fullsized":"2024-09-21T07:34:44.833846+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:18.257010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836565+0000","last_change":"2024-09-21T07:34:43.476520+0000","last_active":"2024-09-21T07:34:44.836565+0000","last_peered":"2024-09-21T07:34:44.836565+0000","last_clean":"2024-09-21T07:34:44.836565+0000","last_became_active":"2024-09-21T07:34:43.476159+0000","last_became_peered":"2024-09-21T07:34:43.476159+0000","last_unstale":"2024-09-21T07:34:44.836565+0000","last_undegraded":"2024-09-21T07:34:44.836565+0000","last_fullsized":"2024-09-21T07:34:44.836565+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:27.525343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"48'1","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833695+0000","last_change":"2024-09-21T07:34:42.441002+0000","last_active":"2024-09-21T07:34:44.833695+0000","last_peered":"2024-09-21T07:34:44.833695+0000","last_clean":"2024-09-21T07:34:44.833695+0000","last_became_active":"2024-09-21T07:34:42.440702+0000","last_became_peered":"2024-09-21T07:34:42.440702+0000","last_unstale":"2024-09-21T07:34:44.833695+0000","last_undegraded":"2024-09-21T07:34:44.833695+0000","last_fullsized":"2024-09-21T07:34:44.833695+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:09.841490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":40,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833536+0000","last_change":"2024-09-21T07:34:24.258712+0000","last_active":"2024-09-21T07:34:44.833536+0000","last_peered":"2024-09-21T07:34:44.833536+0000","last_clean":"2024-09-21T07:34:44.833536+0000","last_became_active":"2024-09-21T07:34:23.274692+0000","last_became_peered":"2024-09-21T07:34:23.274692+0000","last_unstale":"2024-09-21T07:34:44.833536+0000","last_undegraded":"2024-09-21T07:34:44.833536+0000","last_fullsized":"2024-09-21T07:34:44.833536+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:43.279454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018051479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835466+0000","last_change":"2024-09-21T07:34:43.463236+0000","last_active":"2024-09-21T07:34:44.835466+0000","last_peered":"2024-09-21T07:34:44.835466+0000","last_clean":"2024-09-21T07:34:44.835466+0000","last_became_active":"2024-09-21T07:34:43.458917+0000","last_became_peered":"2024-09-21T07:34:43.458917+0000","last_unstale":"2024-09-21T07:34:44.835466+0000","last_undegraded":"2024-09-21T07:34:44.835466+0000","last_fullsized":"2024-09-21T07:34:44.835466+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:45:19.428476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834561+0000","last_change":"2024-09-21T07:34:42.447422+0000","last_active":"2024-09-21T07:34:44.834561+0000","last_peered":"2024-09-21T07:34:44.834561+0000","last_clean":"2024-09-21T07:34:44.834561+0000","last_became_active":"2024-09-21T07:34:42.447132+0000","last_became_peered":"2024-09-21T07:34:42.447132+0000","last_unstale":"2024-09-21T07:34:44.834561+0000","last_undegraded":"2024-09-21T07:34:44.834561+0000","last_fullsized":"2024-09-21T07:34:44.834561+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:26:24.394876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":38,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833146+0000","last_change":"2024-09-21T07:34:25.084849+0000","last_active":"2024-09-21T07:34:44.833146+0000","last_peered":"2024-09-21T07:34:44.833146+0000","last_clean":"2024-09-21T07:34:44.833146+0000","last_became_active":"2024-09-21T07:34:23.277713+0000","last_became_peered":"2024-09-21T07:34:23.277713+0000","last_unstale":"2024-09-21T07:34:44.833146+0000","last_undegraded":"2024-09-21T07:34:44.833146+0000","last_fullsized":"2024-09-21T07:34:44.833146+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:05.811810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059079499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836091+0000","last_change":"2024-09-21T07:34:43.472697+0000","last_active":"2024-09-21T07:34:44.836091+0000","last_peered":"2024-09-21T07:34:44.836091+0000","last_clean":"2024-09-21T07:34:44.836091+0000","last_became_active":"2024-09-21T07:34:43.472276+0000","last_became_peered":"2024-09-21T07:34:43.472276+0000","last_unstale":"2024-09-21T07:34:44.836091+0000","last_undegraded":"2024-09-21T07:34:44.836091+0000","last_fullsized":"2024-09-21T07:34:44.836091+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:49.596333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833099+0000","last_change":"2024-09-21T07:34:42.453339+0000","last_active":"2024-09-21T07:34:44.833099+0000","last_peered":"2024-09-21T07:34:44.833099+0000","last_clean":"2024-09-21T07:34:44.833099+0000","last_became_active":"2024-09-21T07:34:42.452974+0000","last_became_peered":"2024-09-21T07:34:42.452974+0000","last_unstale":"2024-09-21T07:34:44.833099+0000","last_undegraded":"2024-09-21T07:34:44.833099+0000","last_fullsized":"2024-09-21T07:34:44.833099+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:22.788726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467897+0000","last_change":"2024-09-21T07:34:24.538610+0000","last_active":"2024-09-21T07:34:44.467897+0000","last_peered":"2024-09-21T07:34:44.467897+0000","last_clean":"2024-09-21T07:34:44.467897+0000","last_became_active":"2024-09-21T07:34:23.259700+0000","last_became_peered":"2024-09-21T07:34:23.259700+0000","last_unstale":"2024-09-21T07:34:44.467897+0000","last_undegraded":"2024-09-21T07:34:44.467897+0000","last_fullsized":"2024-09-21T07:34:44.467897+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:57.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070522399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834793+0000","last_change":"2024-09-21T07:34:43.474381+0000","last_active":"2024-09-21T07:34:44.834793+0000","last_peered":"2024-09-21T07:34:44.834793+0000","last_clean":"2024-09-21T07:34:44.834793+0000","last_became_active":"2024-09-21T07:34:43.473953+0000","last_became_peered":"2024-09-21T07:34:43.473953+0000","last_unstale":"2024-09-21T07:34:44.834793+0000","last_undegraded":"2024-09-21T07:34:44.834793+0000","last_fullsized":"2024-09-21T07:34:44.834793+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:31.362212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"48'4","reported_seq":19,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834187+0000","last_change":"2024-09-21T07:34:42.444572+0000","last_active":"2024-09-21T07:34:44.834187+0000","last_peered":"2024-09-21T07:34:44.834187+0000","last_clean":"2024-09-21T07:34:44.834187+0000","last_became_active":"2024-09-21T07:34:42.444254+0000","last_became_peered":"2024-09-21T07:34:42.444254+0000","last_unstale":"2024-09-21T07:34:44.834187+0000","last_undegraded":"2024-09-21T07:34:44.834187+0000","last_fullsized":"2024-09-21T07:34:44.834187+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:11:02.002169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835079+0000","last_change":"2024-09-21T07:34:24.597597+0000","last_active":"2024-09-21T07:34:44.835079+0000","last_peered":"2024-09-21T07:34:44.835079+0000","last_clean":"2024-09-21T07:34:44.835079+0000","last_became_active":"2024-09-21T07:34:23.635204+0000","last_became_peered":"2024-09-21T07:34:23.635204+0000","last_unstale":"2024-09-21T07:34:44.835079+0000","last_undegraded":"2024-09-21T07:34:44.835079+0000","last_fullsized":"2024-09-21T07:34:44.835079+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:48.644789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081139199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467728+0000","last_change":"2024-09-21T07:34:43.460645+0000","last_active":"2024-09-21T07:34:44.467728+0000","last_peered":"2024-09-21T07:34:44.467728+0000","last_clean":"2024-09-21T07:34:44.467728+0000","last_became_active":"2024-09-21T07:34:43.460320+0000","last_became_peered":"2024-09-21T07:34:43.460320+0000","last_unstale":"2024-09-21T07:34:44.467728+0000","last_undegraded":"2024-09-21T07:34:44.467728+0000","last_fullsized":"2024-09-21T07:34:44.467728+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:54.644949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"48'1","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465954+0000","last_change":"2024-09-21T07:34:42.446762+0000","last_active":"2024-09-21T07:34:44.465954+0000","last_peered":"2024-09-21T07:34:44.465954+0000","last_clean":"2024-09-21T07:34:44.465954+0000","last_became_active":"2024-09-21T07:34:42.446320+0000","last_became_peered":"2024-09-21T07:34:42.446320+0000","last_unstale":"2024-09-21T07:34:44.465954+0000","last_undegraded":"2024-09-21T07:34:44.465954+0000","last_fullsized":"2024-09-21T07:34:44.465954+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:46.161972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":40,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835380+0000","last_change":"2024-09-21T07:34:24.259026+0000","last_active":"2024-09-21T07:34:44.835380+0000","last_peered":"2024-09-21T07:34:44.835380+0000","last_clean":"2024-09-21T07:34:44.835380+0000","last_became_active":"2024-09-21T07:34:23.635456+0000","last_became_peered":"2024-09-21T07:34:23.635456+0000","last_unstale":"2024-09-21T07:34:44.835380+0000","last_undegraded":"2024-09-21T07:34:44.835380+0000","last_fullsized":"2024-09-21T07:34:44.835380+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:09.592752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018654419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833792+0000","last_change":"2024-09-21T07:34:43.456358+0000","last_active":"2024-09-21T07:34:44.833792+0000","last_peered":"2024-09-21T07:34:44.833792+0000","last_clean":"2024-09-21T07:34:44.833792+0000","last_became_active":"2024-09-21T07:34:43.456052+0000","last_became_peered":"2024-09-21T07:34:43.456052+0000","last_unstale":"2024-09-21T07:34:44.833792+0000","last_undegraded":"2024-09-21T07:34:44.833792+0000","last_fullsized":"2024-09-21T07:34:44.833792+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:07:17.182115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"48'2","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467319+0000","last_change":"2024-09-21T07:34:42.443581+0000","last_active":"2024-09-21T07:34:44.467319+0000","last_peered":"2024-09-21T07:34:44.467319+0000","last_clean":"2024-09-21T07:34:44.467319+0000","last_became_active":"2024-09-21T07:34:42.443284+0000","last_became_peered":"2024-09-21T07:34:42.443284+0000","last_unstale":"2024-09-21T07:34:44.467319+0000","last_undegraded":"2024-09-21T07:34:44.467319+0000","last_fullsized":"2024-09-21T07:34:44.467319+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:59:14.692524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835966+0000","last_change":"2024-09-21T07:34:24.260300+0000","last_active":"2024-09-21T07:34:44.835966+0000","last_peered":"2024-09-21T07:34:44.835966+0000","last_clean":"2024-09-21T07:34:44.835966+0000","last_became_active":"2024-09-21T07:34:23.270128+0000","last_became_peered":"2024-09-21T07:34:23.270128+0000","last_unstale":"2024-09-21T07:34:44.835966+0000","last_undegraded":"2024-09-21T07:34:44.835966+0000","last_fullsized":"2024-09-21T07:34:44.835966+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:25.260931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00176651,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833363+0000","last_change":"2024-09-21T07:34:43.457202+0000","last_active":"2024-09-21T07:34:44.833363+0000","last_peered":"2024-09-21T07:34:44.833363+0000","last_clean":"2024-09-21T07:34:44.833363+0000","last_became_active":"2024-09-21T07:34:43.456851+0000","last_became_peered":"2024-09-21T07:34:43.456851+0000","last_unstale":"2024-09-21T07:34:44.833363+0000","last_undegraded":"2024-09-21T07:34:44.833363+0000","last_fullsized":"2024-09-21T07:34:44.833363+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:12.056767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"48'2","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.468447+0000","last_change":"2024-09-21T07:34:42.434252+0000","last_active":"2024-09-21T07:34:44.468447+0000","last_peered":"2024-09-21T07:34:44.468447+0000","last_clean":"2024-09-21T07:34:44.468447+0000","last_became_active":"2024-09-21T07:34:42.433936+0000","last_became_peered":"2024-09-21T07:34:42.433936+0000","last_unstale":"2024-09-21T07:34:44.468447+0000","last_undegraded":"2024-09-21T07:34:44.468447+0000","last_fullsized":"2024-09-21T07:34:44.468447+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:13.202719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467474+0000","last_change":"2024-09-21T07:34:25.028904+0000","last_active":"2024-09-21T07:34:44.467474+0000","last_peered":"2024-09-21T07:34:44.467474+0000","last_clean":"2024-09-21T07:34:44.467474+0000","last_became_active":"2024-09-21T07:34:23.260942+0000","last_became_peered":"2024-09-21T07:34:23.260942+0000","last_unstale":"2024-09-21T07:34:44.467474+0000","last_undegraded":"2024-09-21T07:34:44.467474+0000","last_fullsized":"2024-09-21T07:34:44.467474+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:22.709328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060553899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509464,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32332,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339508,"statfs":{"total":96636764160,"available":96603656192,"internally_reserved":0,"allocated":1171456,"data_stored":832323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509466,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":577536,"data_stored":242483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509464,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":569344,"data_stored":241831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509465,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31764,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340076,"statfs":{"total":96636764160,"available":96604237824,"internally_reserved":0,"allocated":581632,"data_stored":243443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509466,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31756,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340084,"statfs":{"total":96636764160,"available":96604246016,"internally_reserved":0,"allocated":577536,"data_stored":241991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509466,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1171456,"data_stored":832255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542169,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31748,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340092,"statfs":{"total":96636764160,"available":96604254208,"internally_reserved":0,"allocated":573440,"data_stored":242413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542168,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31760,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340080,"statfs":{"total":96636764160,"available":96604241920,"internally_reserved":0,"allocated":585728,"data_stored":243465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:34:53.025 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:34:53.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.263+0000 7f12bf200640 1 -- 172.21.15.39:0/2445168400 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f12c00d2110 msgr2=0x7f12c00d2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:53.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.263+0000 7f12bf200640 1 --2- 172.21.15.39:0/2445168400 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f12c00d2110 0x7f12c00d2520 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009820 tx=0x7f12a802a510 comp rx=0 tx=0).stop 2024-09-21T07:34:53.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.263+0000 7f12bf200640 1 -- 172.21.15.39:0/2445168400 shutdown_connections 2024-09-21T07:34:53.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.263+0000 7f12bf200640 1 --2- 172.21.15.39:0/2445168400 >> v2:172.21.15.148:3300/0 conn(0x7f12c00b7460 0x7f12c00b7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:53.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.263+0000 7f12bf200640 1 --2- 172.21.15.39:0/2445168400 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f12c00d2110 0x7f12c00d2520 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:53.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.263+0000 7f12bf200640 1 -- 172.21.15.39:0/2445168400 >> 172.21.15.39:0/2445168400 conn(0x7f12c009d6a0 msgr2=0x7f12c00d4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:53.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.263+0000 7f12bf200640 1 -- 172.21.15.39:0/2445168400 shutdown_connections 2024-09-21T07:34:53.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.263+0000 7f12bf200640 1 -- 172.21.15.39:0/2445168400 wait complete. 2024-09-21T07:34:53.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.265+0000 7f12bf200640 1 Processor -- start 2024-09-21T07:34:53.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.265+0000 7f12bf200640 1 -- start start 2024-09-21T07:34:53.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.267+0000 7f12bf200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f12c00b5470 0x7f12c01515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:53.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.267+0000 7f12bf200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12c00b7460 0x7f12c0151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:53.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.267+0000 7f12bf200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12c00d2110 0x7f12c01523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:53.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.267+0000 7f12bf200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f12c00b8140 con 0x7f12c00b7460 2024-09-21T07:34:53.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.267+0000 7f12bf200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f12c00b7fc0 con 0x7f12c00d2110 2024-09-21T07:34:53.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.267+0000 7f12bf200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f12c00b82c0 con 0x7f12c00b5470 2024-09-21T07:34:53.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.267+0000 7f12b7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12c00b7460 0x7f12c0151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:53.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.267+0000 7f12b7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12c00b7460 0x7f12c0151ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57506/0 (socket says 172.21.15.39:57506) 2024-09-21T07:34:53.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.267+0000 7f12b7e00640 1 -- 172.21.15.39:0/363959724 learned_addr learned my addr 172.21.15.39:0/363959724 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:53.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.267+0000 7f12bde00640 1 --2- 172.21.15.39:0/363959724 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f12c00b5470 0x7f12c01515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:53.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.267+0000 7f12be800640 1 --2- 172.21.15.39:0/363959724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12c00d2110 0x7f12c01523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:53.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.268+0000 7f12b7e00640 1 -- 172.21.15.39:0/363959724 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f12c00b5470 msgr2=0x7f12c01515e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:53.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.268+0000 7f12b7e00640 1 --2- 172.21.15.39:0/363959724 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f12c00b5470 0x7f12c01515e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:53.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.268+0000 7f12b7e00640 1 -- 172.21.15.39:0/363959724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12c00d2110 msgr2=0x7f12c01523e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:53.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.268+0000 7f12b7e00640 1 --2- 172.21.15.39:0/363959724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12c00d2110 0x7f12c01523e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:53.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.268+0000 7f12b7e00640 1 -- 172.21.15.39:0/363959724 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12c01602a0 con 0x7f12c00b7460 2024-09-21T07:34:53.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.268+0000 7f12b7e00640 1 --2- 172.21.15.39:0/363959724 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12c00b7460 0x7f12c0151ea0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f12a8000c00 tx=0x7f12a8005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:53.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.268+0000 7f12b6000640 1 -- 172.21.15.39:0/363959724 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a803f070 con 0x7f12c00b7460 2024-09-21T07:34:53.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.268+0000 7f12b6000640 1 -- 172.21.15.39:0/363959724 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12a8004340 con 0x7f12c00b7460 2024-09-21T07:34:53.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.268+0000 7f12b6000640 1 -- 172.21.15.39:0/363959724 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a8033e60 con 0x7f12c00b7460 2024-09-21T07:34:53.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.268+0000 7f12bf200640 1 -- 172.21.15.39:0/363959724 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12c00cca30 con 0x7f12c00b7460 2024-09-21T07:34:53.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.269+0000 7f12bf200640 1 -- 172.21.15.39:0/363959724 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12c00ccff0 con 0x7f12c00b7460 2024-09-21T07:34:53.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.270+0000 7f12bf200640 1 -- 172.21.15.39:0/363959724 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12c00b7870 con 0x7f12c00b7460 2024-09-21T07:34:53.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.271+0000 7f12b6000640 1 -- 172.21.15.39:0/363959724 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f12a8042070 con 0x7f12c00b7460 2024-09-21T07:34:53.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.272+0000 7f12b6000640 1 --2- 172.21.15.39:0/363959724 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f129c078b90 0x7f129c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:53.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.272+0000 7f12b6000640 1 -- 172.21.15.39:0/363959724 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6120+0+0 (secure 0 0 0) 0x7f12a80bbd90 con 0x7f12c00b7460 2024-09-21T07:34:53.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.273+0000 7f12bde00640 1 --2- 172.21.15.39:0/363959724 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f129c078b90 0x7f129c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:53.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.274+0000 7f12bde00640 1 --2- 172.21.15.39:0/363959724 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f129c078b90 0x7f129c07b0a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f12b0007fd0 tx=0x7f12b0001370 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:53.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.277+0000 7f12b6000640 1 -- 172.21.15.39:0/363959724 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f12c01602a0 con 0x7f12c00b7460 2024-09-21T07:34:53.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.469+0000 7f12bf200640 1 -- 172.21.15.39:0/363959724 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f12c00d1b40 con 0x7f129c078b90 2024-09-21T07:34:53.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.477+0000 7f12b6000640 1 -- 172.21.15.39:0/363959724 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332904 (secure 0 0 0) 0x7f12c00d1b40 con 0x7f129c078b90 2024-09-21T07:34:53.483 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:34:53.484 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:34:53.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.481+0000 7f12bf200640 1 -- 172.21.15.39:0/363959724 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f129c078b90 msgr2=0x7f129c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:53.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.481+0000 7f12bf200640 1 --2- 172.21.15.39:0/363959724 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f129c078b90 0x7f129c07b0a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f12b0007fd0 tx=0x7f12b0001370 comp rx=0 tx=0).stop 2024-09-21T07:34:53.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.481+0000 7f12bf200640 1 -- 172.21.15.39:0/363959724 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12c00b7460 msgr2=0x7f12c0151ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:53.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.481+0000 7f12bf200640 1 --2- 172.21.15.39:0/363959724 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12c00b7460 0x7f12c0151ea0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f12a8000c00 tx=0x7f12a8005270 comp rx=0 tx=0).stop 2024-09-21T07:34:53.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.481+0000 7f12bf200640 1 -- 172.21.15.39:0/363959724 shutdown_connections 2024-09-21T07:34:53.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.481+0000 7f12bf200640 1 --2- 172.21.15.39:0/363959724 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f129c078b90 0x7f129c07b0a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:53.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.481+0000 7f12bf200640 1 --2- 172.21.15.39:0/363959724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12c00d2110 0x7f12c01523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:53.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.481+0000 7f12bf200640 1 --2- 172.21.15.39:0/363959724 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12c00b7460 0x7f12c0151ea0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:53.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.481+0000 7f12bf200640 1 --2- 172.21.15.39:0/363959724 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f12c00b5470 0x7f12c01515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:53.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.481+0000 7f12bf200640 1 -- 172.21.15.39:0/363959724 >> 172.21.15.39:0/363959724 conn(0x7f12c009d6a0 msgr2=0x7f12c00d4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:53.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.481+0000 7f12bf200640 1 -- 172.21.15.39:0/363959724 shutdown_connections 2024-09-21T07:34:53.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.481+0000 7f12bf200640 1 -- 172.21.15.39:0/363959724 wait complete. 2024-09-21T07:34:53.507 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2024-09-21T07:34:52.515236+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":258,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":246888,"kb_used_data":5672,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754727832,"statfs":{"total":773094113280,"available":772841299968,"internally_reserved":0,"allocated":5808128,"data_stored":3120204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.959533"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.466557+0000","last_change":"2024-09-21T07:34:43.473741+0000","last_active":"2024-09-21T07:34:44.466557+0000","last_peered":"2024-09-21T07:34:44.466557+0000","last_clean":"2024-09-21T07:34:44.466557+0000","last_became_active":"2024-09-21T07:34:43.473447+0000","last_became_peered":"2024-09-21T07:34:43.473447+0000","last_unstale":"2024-09-21T07:34:44.466557+0000","last_undegraded":"2024-09-21T07:34:44.466557+0000","last_fullsized":"2024-09-21T07:34:44.466557+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:00:23.404297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836408+0000","last_change":"2024-09-21T07:34:43.473362+0000","last_active":"2024-09-21T07:34:44.836408+0000","last_peered":"2024-09-21T07:34:44.836408+0000","last_clean":"2024-09-21T07:34:44.836408+0000","last_became_active":"2024-09-21T07:34:43.473083+0000","last_became_peered":"2024-09-21T07:34:43.473083+0000","last_unstale":"2024-09-21T07:34:44.836408+0000","last_undegraded":"2024-09-21T07:34:44.836408+0000","last_fullsized":"2024-09-21T07:34:44.836408+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:33.920441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834020+0000","last_change":"2024-09-21T07:34:43.481499+0000","last_active":"2024-09-21T07:34:44.834020+0000","last_peered":"2024-09-21T07:34:44.834020+0000","last_clean":"2024-09-21T07:34:44.834020+0000","last_became_active":"2024-09-21T07:34:43.478254+0000","last_became_peered":"2024-09-21T07:34:43.478254+0000","last_unstale":"2024-09-21T07:34:44.834020+0000","last_undegraded":"2024-09-21T07:34:44.834020+0000","last_fullsized":"2024-09-21T07:34:44.834020+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:03.654293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836742+0000","last_change":"2024-09-21T07:34:43.451459+0000","last_active":"2024-09-21T07:34:44.836742+0000","last_peered":"2024-09-21T07:34:44.836742+0000","last_clean":"2024-09-21T07:34:44.836742+0000","last_became_active":"2024-09-21T07:34:43.451121+0000","last_became_peered":"2024-09-21T07:34:43.451121+0000","last_unstale":"2024-09-21T07:34:44.836742+0000","last_undegraded":"2024-09-21T07:34:44.836742+0000","last_fullsized":"2024-09-21T07:34:44.836742+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:15.139818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834482+0000","last_change":"2024-09-21T07:34:43.455772+0000","last_active":"2024-09-21T07:34:44.834482+0000","last_peered":"2024-09-21T07:34:44.834482+0000","last_clean":"2024-09-21T07:34:44.834482+0000","last_became_active":"2024-09-21T07:34:43.454461+0000","last_became_peered":"2024-09-21T07:34:43.454461+0000","last_unstale":"2024-09-21T07:34:44.834482+0000","last_undegraded":"2024-09-21T07:34:44.834482+0000","last_fullsized":"2024-09-21T07:34:44.834482+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:03.456938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465593+0000","last_change":"2024-09-21T07:34:43.469533+0000","last_active":"2024-09-21T07:34:44.465593+0000","last_peered":"2024-09-21T07:34:44.465593+0000","last_clean":"2024-09-21T07:34:44.465593+0000","last_became_active":"2024-09-21T07:34:43.469336+0000","last_became_peered":"2024-09-21T07:34:43.469336+0000","last_unstale":"2024-09-21T07:34:44.465593+0000","last_undegraded":"2024-09-21T07:34:44.465593+0000","last_fullsized":"2024-09-21T07:34:44.465593+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:20.540854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833543+0000","last_change":"2024-09-21T07:34:43.483800+0000","last_active":"2024-09-21T07:34:44.833543+0000","last_peered":"2024-09-21T07:34:44.833543+0000","last_clean":"2024-09-21T07:34:44.833543+0000","last_became_active":"2024-09-21T07:34:43.483482+0000","last_became_peered":"2024-09-21T07:34:43.483482+0000","last_unstale":"2024-09-21T07:34:44.833543+0000","last_undegraded":"2024-09-21T07:34:44.833543+0000","last_fullsized":"2024-09-21T07:34:44.833543+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:20.152451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834422+0000","last_change":"2024-09-21T07:34:43.473653+0000","last_active":"2024-09-21T07:34:44.834422+0000","last_peered":"2024-09-21T07:34:44.834422+0000","last_clean":"2024-09-21T07:34:44.834422+0000","last_became_active":"2024-09-21T07:34:43.473348+0000","last_became_peered":"2024-09-21T07:34:43.473348+0000","last_unstale":"2024-09-21T07:34:44.834422+0000","last_undegraded":"2024-09-21T07:34:44.834422+0000","last_fullsized":"2024-09-21T07:34:44.834422+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:13.042429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835096+0000","last_change":"2024-09-21T07:34:43.483507+0000","last_active":"2024-09-21T07:34:44.835096+0000","last_peered":"2024-09-21T07:34:44.835096+0000","last_clean":"2024-09-21T07:34:44.835096+0000","last_became_active":"2024-09-21T07:34:43.482198+0000","last_became_peered":"2024-09-21T07:34:43.482198+0000","last_unstale":"2024-09-21T07:34:44.835096+0000","last_undegraded":"2024-09-21T07:34:44.835096+0000","last_fullsized":"2024-09-21T07:34:44.835096+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:45.957857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465790+0000","last_change":"2024-09-21T07:34:43.465258+0000","last_active":"2024-09-21T07:34:44.465790+0000","last_peered":"2024-09-21T07:34:44.465790+0000","last_clean":"2024-09-21T07:34:44.465790+0000","last_became_active":"2024-09-21T07:34:43.464922+0000","last_became_peered":"2024-09-21T07:34:43.464922+0000","last_unstale":"2024-09-21T07:34:44.465790+0000","last_undegraded":"2024-09-21T07:34:44.465790+0000","last_fullsized":"2024-09-21T07:34:44.465790+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:11.406345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467028+0000","last_change":"2024-09-21T07:34:43.449879+0000","last_active":"2024-09-21T07:34:44.467028+0000","last_peered":"2024-09-21T07:34:44.467028+0000","last_clean":"2024-09-21T07:34:44.467028+0000","last_became_active":"2024-09-21T07:34:43.448365+0000","last_became_peered":"2024-09-21T07:34:43.448365+0000","last_unstale":"2024-09-21T07:34:44.467028+0000","last_undegraded":"2024-09-21T07:34:44.467028+0000","last_fullsized":"2024-09-21T07:34:44.467028+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:29:00.993972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834222+0000","last_change":"2024-09-21T07:34:43.460167+0000","last_active":"2024-09-21T07:34:44.834222+0000","last_peered":"2024-09-21T07:34:44.834222+0000","last_clean":"2024-09-21T07:34:44.834222+0000","last_became_active":"2024-09-21T07:34:43.457296+0000","last_became_peered":"2024-09-21T07:34:43.457296+0000","last_unstale":"2024-09-21T07:34:44.834222+0000","last_undegraded":"2024-09-21T07:34:44.834222+0000","last_fullsized":"2024-09-21T07:34:44.834222+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:53:57.871505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.466129+0000","last_change":"2024-09-21T07:34:43.465519+0000","last_active":"2024-09-21T07:34:44.466129+0000","last_peered":"2024-09-21T07:34:44.466129+0000","last_clean":"2024-09-21T07:34:44.466129+0000","last_became_active":"2024-09-21T07:34:43.464802+0000","last_became_peered":"2024-09-21T07:34:43.464802+0000","last_unstale":"2024-09-21T07:34:44.466129+0000","last_undegraded":"2024-09-21T07:34:44.466129+0000","last_fullsized":"2024-09-21T07:34:44.466129+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:59.062262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834033+0000","last_change":"2024-09-21T07:34:43.478575+0000","last_active":"2024-09-21T07:34:44.834033+0000","last_peered":"2024-09-21T07:34:44.834033+0000","last_clean":"2024-09-21T07:34:44.834033+0000","last_became_active":"2024-09-21T07:34:43.478187+0000","last_became_peered":"2024-09-21T07:34:43.478187+0000","last_unstale":"2024-09-21T07:34:44.834033+0000","last_undegraded":"2024-09-21T07:34:44.834033+0000","last_fullsized":"2024-09-21T07:34:44.834033+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:44.866369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836848+0000","last_change":"2024-09-21T07:34:43.467093+0000","last_active":"2024-09-21T07:34:44.836848+0000","last_peered":"2024-09-21T07:34:44.836848+0000","last_clean":"2024-09-21T07:34:44.836848+0000","last_became_active":"2024-09-21T07:34:43.466773+0000","last_became_peered":"2024-09-21T07:34:43.466773+0000","last_unstale":"2024-09-21T07:34:44.836848+0000","last_undegraded":"2024-09-21T07:34:44.836848+0000","last_fullsized":"2024-09-21T07:34:44.836848+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:12:02.180192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465260+0000","last_change":"2024-09-21T07:34:43.462637+0000","last_active":"2024-09-21T07:34:44.465260+0000","last_peered":"2024-09-21T07:34:44.465260+0000","last_clean":"2024-09-21T07:34:44.465260+0000","last_became_active":"2024-09-21T07:34:43.462301+0000","last_became_peered":"2024-09-21T07:34:43.462301+0000","last_unstale":"2024-09-21T07:34:44.465260+0000","last_undegraded":"2024-09-21T07:34:44.465260+0000","last_fullsized":"2024-09-21T07:34:44.465260+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:42.168665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834898+0000","last_change":"2024-09-21T07:34:43.484342+0000","last_active":"2024-09-21T07:34:44.834898+0000","last_peered":"2024-09-21T07:34:44.834898+0000","last_clean":"2024-09-21T07:34:44.834898+0000","last_became_active":"2024-09-21T07:34:43.480452+0000","last_became_peered":"2024-09-21T07:34:43.480452+0000","last_unstale":"2024-09-21T07:34:44.834898+0000","last_undegraded":"2024-09-21T07:34:44.834898+0000","last_fullsized":"2024-09-21T07:34:44.834898+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:30.267103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836590+0000","last_change":"2024-09-21T07:34:43.472890+0000","last_active":"2024-09-21T07:34:44.836590+0000","last_peered":"2024-09-21T07:34:44.836590+0000","last_clean":"2024-09-21T07:34:44.836590+0000","last_became_active":"2024-09-21T07:34:43.472576+0000","last_became_peered":"2024-09-21T07:34:43.472576+0000","last_unstale":"2024-09-21T07:34:44.836590+0000","last_undegraded":"2024-09-21T07:34:44.836590+0000","last_fullsized":"2024-09-21T07:34:44.836590+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:24.719441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834303+0000","last_change":"2024-09-21T07:34:43.476774+0000","last_active":"2024-09-21T07:34:44.834303+0000","last_peered":"2024-09-21T07:34:44.834303+0000","last_clean":"2024-09-21T07:34:44.834303+0000","last_became_active":"2024-09-21T07:34:43.473147+0000","last_became_peered":"2024-09-21T07:34:43.473147+0000","last_unstale":"2024-09-21T07:34:44.834303+0000","last_undegraded":"2024-09-21T07:34:44.834303+0000","last_fullsized":"2024-09-21T07:34:44.834303+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:10.661941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835069+0000","last_change":"2024-09-21T07:34:43.467366+0000","last_active":"2024-09-21T07:34:44.835069+0000","last_peered":"2024-09-21T07:34:44.835069+0000","last_clean":"2024-09-21T07:34:44.835069+0000","last_became_active":"2024-09-21T07:34:43.467032+0000","last_became_peered":"2024-09-21T07:34:43.467032+0000","last_unstale":"2024-09-21T07:34:44.835069+0000","last_undegraded":"2024-09-21T07:34:44.835069+0000","last_fullsized":"2024-09-21T07:34:44.835069+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:47.164384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833778+0000","last_change":"2024-09-21T07:34:43.466216+0000","last_active":"2024-09-21T07:34:44.833778+0000","last_peered":"2024-09-21T07:34:44.833778+0000","last_clean":"2024-09-21T07:34:44.833778+0000","last_became_active":"2024-09-21T07:34:43.465960+0000","last_became_peered":"2024-09-21T07:34:43.465960+0000","last_unstale":"2024-09-21T07:34:44.833778+0000","last_undegraded":"2024-09-21T07:34:44.833778+0000","last_fullsized":"2024-09-21T07:34:44.833778+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:24.439384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834766+0000","last_change":"2024-09-21T07:34:43.483312+0000","last_active":"2024-09-21T07:34:44.834766+0000","last_peered":"2024-09-21T07:34:44.834766+0000","last_clean":"2024-09-21T07:34:44.834766+0000","last_became_active":"2024-09-21T07:34:43.483003+0000","last_became_peered":"2024-09-21T07:34:43.483003+0000","last_unstale":"2024-09-21T07:34:44.834766+0000","last_undegraded":"2024-09-21T07:34:44.834766+0000","last_fullsized":"2024-09-21T07:34:44.834766+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:33.782633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467544+0000","last_change":"2024-09-21T07:34:43.449273+0000","last_active":"2024-09-21T07:34:44.467544+0000","last_peered":"2024-09-21T07:34:44.467544+0000","last_clean":"2024-09-21T07:34:44.467544+0000","last_became_active":"2024-09-21T07:34:43.448944+0000","last_became_peered":"2024-09-21T07:34:43.448944+0000","last_unstale":"2024-09-21T07:34:44.467544+0000","last_undegraded":"2024-09-21T07:34:44.467544+0000","last_fullsized":"2024-09-21T07:34:44.467544+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:56.540403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833398+0000","last_change":"2024-09-21T07:34:43.479357+0000","last_active":"2024-09-21T07:34:44.833398+0000","last_peered":"2024-09-21T07:34:44.833398+0000","last_clean":"2024-09-21T07:34:44.833398+0000","last_became_active":"2024-09-21T07:34:43.478997+0000","last_became_peered":"2024-09-21T07:34:43.478997+0000","last_unstale":"2024-09-21T07:34:44.833398+0000","last_undegraded":"2024-09-21T07:34:44.833398+0000","last_fullsized":"2024-09-21T07:34:44.833398+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:54.644949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835486+0000","last_change":"2024-09-21T07:34:43.486869+0000","last_active":"2024-09-21T07:34:44.835486+0000","last_peered":"2024-09-21T07:34:44.835486+0000","last_clean":"2024-09-21T07:34:44.835486+0000","last_became_active":"2024-09-21T07:34:43.486351+0000","last_became_peered":"2024-09-21T07:34:43.486351+0000","last_unstale":"2024-09-21T07:34:44.835486+0000","last_undegraded":"2024-09-21T07:34:44.835486+0000","last_fullsized":"2024-09-21T07:34:44.835486+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:21:02.910848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833742+0000","last_change":"2024-09-21T07:34:43.463196+0000","last_active":"2024-09-21T07:34:44.833742+0000","last_peered":"2024-09-21T07:34:44.833742+0000","last_clean":"2024-09-21T07:34:44.833742+0000","last_became_active":"2024-09-21T07:34:43.458559+0000","last_became_peered":"2024-09-21T07:34:43.458559+0000","last_unstale":"2024-09-21T07:34:44.833742+0000","last_undegraded":"2024-09-21T07:34:44.833742+0000","last_fullsized":"2024-09-21T07:34:44.833742+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:37.233429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467280+0000","last_change":"2024-09-21T07:34:43.483549+0000","last_active":"2024-09-21T07:34:44.467280+0000","last_peered":"2024-09-21T07:34:44.467280+0000","last_clean":"2024-09-21T07:34:44.467280+0000","last_became_active":"2024-09-21T07:34:43.472653+0000","last_became_peered":"2024-09-21T07:34:43.472653+0000","last_unstale":"2024-09-21T07:34:44.467280+0000","last_undegraded":"2024-09-21T07:34:44.467280+0000","last_fullsized":"2024-09-21T07:34:44.467280+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:38.814916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834808+0000","last_change":"2024-09-21T07:34:43.459996+0000","last_active":"2024-09-21T07:34:44.834808+0000","last_peered":"2024-09-21T07:34:44.834808+0000","last_clean":"2024-09-21T07:34:44.834808+0000","last_became_active":"2024-09-21T07:34:43.459068+0000","last_became_peered":"2024-09-21T07:34:43.459068+0000","last_unstale":"2024-09-21T07:34:44.834808+0000","last_undegraded":"2024-09-21T07:34:44.834808+0000","last_fullsized":"2024-09-21T07:34:44.834808+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:49.571297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833338+0000","last_change":"2024-09-21T07:34:43.489357+0000","last_active":"2024-09-21T07:34:44.833338+0000","last_peered":"2024-09-21T07:34:44.833338+0000","last_clean":"2024-09-21T07:34:44.833338+0000","last_became_active":"2024-09-21T07:34:43.483224+0000","last_became_peered":"2024-09-21T07:34:43.483224+0000","last_unstale":"2024-09-21T07:34:44.833338+0000","last_undegraded":"2024-09-21T07:34:44.833338+0000","last_fullsized":"2024-09-21T07:34:44.833338+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:05.212534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835873+0000","last_change":"2024-09-21T07:34:43.460376+0000","last_active":"2024-09-21T07:34:44.835873+0000","last_peered":"2024-09-21T07:34:44.835873+0000","last_clean":"2024-09-21T07:34:44.835873+0000","last_became_active":"2024-09-21T07:34:43.458320+0000","last_became_peered":"2024-09-21T07:34:43.458320+0000","last_unstale":"2024-09-21T07:34:44.835873+0000","last_undegraded":"2024-09-21T07:34:44.835873+0000","last_fullsized":"2024-09-21T07:34:44.835873+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:35.190652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833745+0000","last_change":"2024-09-21T07:34:43.485349+0000","last_active":"2024-09-21T07:34:44.833745+0000","last_peered":"2024-09-21T07:34:44.833745+0000","last_clean":"2024-09-21T07:34:44.833745+0000","last_became_active":"2024-09-21T07:34:43.483623+0000","last_became_peered":"2024-09-21T07:34:43.483623+0000","last_unstale":"2024-09-21T07:34:44.833745+0000","last_undegraded":"2024-09-21T07:34:44.833745+0000","last_fullsized":"2024-09-21T07:34:44.833745+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:02.084719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467658+0000","last_change":"2024-09-21T07:34:43.471970+0000","last_active":"2024-09-21T07:34:44.467658+0000","last_peered":"2024-09-21T07:34:44.467658+0000","last_clean":"2024-09-21T07:34:44.467658+0000","last_became_active":"2024-09-21T07:34:43.465515+0000","last_became_peered":"2024-09-21T07:34:43.465515+0000","last_unstale":"2024-09-21T07:34:44.467658+0000","last_undegraded":"2024-09-21T07:34:44.467658+0000","last_fullsized":"2024-09-21T07:34:44.467658+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:59:53.819831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834269+0000","last_change":"2024-09-21T07:34:43.466446+0000","last_active":"2024-09-21T07:34:44.834269+0000","last_peered":"2024-09-21T07:34:44.834269+0000","last_clean":"2024-09-21T07:34:44.834269+0000","last_became_active":"2024-09-21T07:34:43.466153+0000","last_became_peered":"2024-09-21T07:34:43.466153+0000","last_unstale":"2024-09-21T07:34:44.834269+0000","last_undegraded":"2024-09-21T07:34:44.834269+0000","last_fullsized":"2024-09-21T07:34:44.834269+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:17.974647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"49'1","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834973+0000","last_change":"2024-09-21T07:34:42.447496+0000","last_active":"2024-09-21T07:34:44.834973+0000","last_peered":"2024-09-21T07:34:44.834973+0000","last_clean":"2024-09-21T07:34:44.834973+0000","last_became_active":"2024-09-21T07:34:42.447148+0000","last_became_peered":"2024-09-21T07:34:42.447148+0000","last_unstale":"2024-09-21T07:34:44.834973+0000","last_undegraded":"2024-09-21T07:34:44.834973+0000","last_fullsized":"2024-09-21T07:34:44.834973+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:42:06.750595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":163,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835508+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:34:44.835508+0000","last_peered":"2024-09-21T07:34:44.835508+0000","last_clean":"2024-09-21T07:34:44.835508+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:34:44.835508+0000","last_undegraded":"2024-09-21T07:34:44.835508+0000","last_fullsized":"2024-09-21T07:34:44.835508+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.468769+0000","last_change":"2024-09-21T07:34:24.539248+0000","last_active":"2024-09-21T07:34:44.468769+0000","last_peered":"2024-09-21T07:34:44.468769+0000","last_clean":"2024-09-21T07:34:44.468769+0000","last_became_active":"2024-09-21T07:34:23.273465+0000","last_became_peered":"2024-09-21T07:34:23.273465+0000","last_unstale":"2024-09-21T07:34:44.468769+0000","last_undegraded":"2024-09-21T07:34:44.468769+0000","last_fullsized":"2024-09-21T07:34:44.468769+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:55.517452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060627700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834170+0000","last_change":"2024-09-21T07:34:43.491609+0000","last_active":"2024-09-21T07:34:44.834170+0000","last_peered":"2024-09-21T07:34:44.834170+0000","last_clean":"2024-09-21T07:34:44.834170+0000","last_became_active":"2024-09-21T07:34:43.490138+0000","last_became_peered":"2024-09-21T07:34:43.490138+0000","last_unstale":"2024-09-21T07:34:44.834170+0000","last_undegraded":"2024-09-21T07:34:44.834170+0000","last_fullsized":"2024-09-21T07:34:44.834170+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:16.015947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833429+0000","last_change":"2024-09-21T07:34:42.448815+0000","last_active":"2024-09-21T07:34:44.833429+0000","last_peered":"2024-09-21T07:34:44.833429+0000","last_clean":"2024-09-21T07:34:44.833429+0000","last_became_active":"2024-09-21T07:34:42.448484+0000","last_became_peered":"2024-09-21T07:34:42.448484+0000","last_unstale":"2024-09-21T07:34:44.833429+0000","last_undegraded":"2024-09-21T07:34:44.833429+0000","last_fullsized":"2024-09-21T07:34:44.833429+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:15.522307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833978+0000","last_change":"2024-09-21T07:34:24.597274+0000","last_active":"2024-09-21T07:34:44.833978+0000","last_peered":"2024-09-21T07:34:44.833978+0000","last_clean":"2024-09-21T07:34:44.833978+0000","last_became_active":"2024-09-21T07:34:23.279560+0000","last_became_peered":"2024-09-21T07:34:23.279560+0000","last_unstale":"2024-09-21T07:34:44.833978+0000","last_undegraded":"2024-09-21T07:34:44.833978+0000","last_fullsized":"2024-09-21T07:34:44.833978+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:15.845265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059699799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.468039+0000","last_change":"2024-09-21T07:34:43.468683+0000","last_active":"2024-09-21T07:34:44.468039+0000","last_peered":"2024-09-21T07:34:44.468039+0000","last_clean":"2024-09-21T07:34:44.468039+0000","last_became_active":"2024-09-21T07:34:43.468377+0000","last_became_peered":"2024-09-21T07:34:43.468377+0000","last_unstale":"2024-09-21T07:34:44.468039+0000","last_undegraded":"2024-09-21T07:34:44.468039+0000","last_fullsized":"2024-09-21T07:34:44.468039+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:02.084719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834944+0000","last_change":"2024-09-21T07:34:42.442148+0000","last_active":"2024-09-21T07:34:44.834944+0000","last_peered":"2024-09-21T07:34:44.834944+0000","last_clean":"2024-09-21T07:34:44.834944+0000","last_became_active":"2024-09-21T07:34:42.441839+0000","last_became_peered":"2024-09-21T07:34:42.441839+0000","last_unstale":"2024-09-21T07:34:44.834944+0000","last_undegraded":"2024-09-21T07:34:44.834944+0000","last_fullsized":"2024-09-21T07:34:44.834944+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:35:57.065813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836188+0000","last_change":"2024-09-21T07:34:24.259906+0000","last_active":"2024-09-21T07:34:44.836188+0000","last_peered":"2024-09-21T07:34:44.836188+0000","last_clean":"2024-09-21T07:34:44.836188+0000","last_became_active":"2024-09-21T07:34:23.635282+0000","last_became_peered":"2024-09-21T07:34:23.635282+0000","last_unstale":"2024-09-21T07:34:44.836188+0000","last_undegraded":"2024-09-21T07:34:44.836188+0000","last_fullsized":"2024-09-21T07:34:44.836188+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:00.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001527664,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834033+0000","last_change":"2024-09-21T07:34:43.491274+0000","last_active":"2024-09-21T07:34:44.834033+0000","last_peered":"2024-09-21T07:34:44.834033+0000","last_clean":"2024-09-21T07:34:44.834033+0000","last_became_active":"2024-09-21T07:34:43.489063+0000","last_became_peered":"2024-09-21T07:34:43.489063+0000","last_unstale":"2024-09-21T07:34:44.834033+0000","last_undegraded":"2024-09-21T07:34:44.834033+0000","last_fullsized":"2024-09-21T07:34:44.834033+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:13.315568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833450+0000","last_change":"2024-09-21T07:34:42.437481+0000","last_active":"2024-09-21T07:34:44.833450+0000","last_peered":"2024-09-21T07:34:44.833450+0000","last_clean":"2024-09-21T07:34:44.833450+0000","last_became_active":"2024-09-21T07:34:42.437130+0000","last_became_peered":"2024-09-21T07:34:42.437130+0000","last_unstale":"2024-09-21T07:34:44.833450+0000","last_undegraded":"2024-09-21T07:34:44.833450+0000","last_fullsized":"2024-09-21T07:34:44.833450+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:55.611339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833360+0000","last_change":"2024-09-21T07:34:25.085454+0000","last_active":"2024-09-21T07:34:44.833360+0000","last_peered":"2024-09-21T07:34:44.833360+0000","last_clean":"2024-09-21T07:34:44.833360+0000","last_became_active":"2024-09-21T07:34:23.279205+0000","last_became_peered":"2024-09-21T07:34:23.279205+0000","last_unstale":"2024-09-21T07:34:44.833360+0000","last_undegraded":"2024-09-21T07:34:44.833360+0000","last_fullsized":"2024-09-21T07:34:44.833360+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:54.231666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095320599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835309+0000","last_change":"2024-09-21T07:34:43.457980+0000","last_active":"2024-09-21T07:34:44.835309+0000","last_peered":"2024-09-21T07:34:44.835309+0000","last_clean":"2024-09-21T07:34:44.835309+0000","last_became_active":"2024-09-21T07:34:43.457695+0000","last_became_peered":"2024-09-21T07:34:43.457695+0000","last_unstale":"2024-09-21T07:34:44.835309+0000","last_undegraded":"2024-09-21T07:34:44.835309+0000","last_fullsized":"2024-09-21T07:34:44.835309+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:01.492292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467870+0000","last_change":"2024-09-21T07:34:42.438107+0000","last_active":"2024-09-21T07:34:44.467870+0000","last_peered":"2024-09-21T07:34:44.467870+0000","last_clean":"2024-09-21T07:34:44.467870+0000","last_became_active":"2024-09-21T07:34:42.437794+0000","last_became_peered":"2024-09-21T07:34:42.437794+0000","last_unstale":"2024-09-21T07:34:44.467870+0000","last_undegraded":"2024-09-21T07:34:44.467870+0000","last_fullsized":"2024-09-21T07:34:44.467870+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:24.649223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465509+0000","last_change":"2024-09-21T07:34:25.028521+0000","last_active":"2024-09-21T07:34:44.465509+0000","last_peered":"2024-09-21T07:34:44.465509+0000","last_clean":"2024-09-21T07:34:44.465509+0000","last_became_active":"2024-09-21T07:34:23.635467+0000","last_became_peered":"2024-09-21T07:34:23.635467+0000","last_unstale":"2024-09-21T07:34:44.465509+0000","last_undegraded":"2024-09-21T07:34:44.465509+0000","last_fullsized":"2024-09-21T07:34:44.465509+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:33.729123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000660558,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467665+0000","last_change":"2024-09-21T07:34:43.465578+0000","last_active":"2024-09-21T07:34:44.467665+0000","last_peered":"2024-09-21T07:34:44.467665+0000","last_clean":"2024-09-21T07:34:44.467665+0000","last_became_active":"2024-09-21T07:34:43.465071+0000","last_became_peered":"2024-09-21T07:34:43.465071+0000","last_unstale":"2024-09-21T07:34:44.467665+0000","last_undegraded":"2024-09-21T07:34:44.467665+0000","last_fullsized":"2024-09-21T07:34:44.467665+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:20.152451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"48'2","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834309+0000","last_change":"2024-09-21T07:34:42.445927+0000","last_active":"2024-09-21T07:34:44.834309+0000","last_peered":"2024-09-21T07:34:44.834309+0000","last_clean":"2024-09-21T07:34:44.834309+0000","last_became_active":"2024-09-21T07:34:42.445620+0000","last_became_peered":"2024-09-21T07:34:42.445620+0000","last_unstale":"2024-09-21T07:34:44.834309+0000","last_undegraded":"2024-09-21T07:34:44.834309+0000","last_fullsized":"2024-09-21T07:34:44.834309+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:48.022549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":40,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835533+0000","last_change":"2024-09-21T07:34:24.260046+0000","last_active":"2024-09-21T07:34:44.835533+0000","last_peered":"2024-09-21T07:34:44.835533+0000","last_clean":"2024-09-21T07:34:44.835533+0000","last_became_active":"2024-09-21T07:34:23.273431+0000","last_became_peered":"2024-09-21T07:34:23.273431+0000","last_unstale":"2024-09-21T07:34:44.835533+0000","last_undegraded":"2024-09-21T07:34:44.835533+0000","last_fullsized":"2024-09-21T07:34:44.835533+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:18.876057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019803310000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465631+0000","last_change":"2024-09-21T07:34:43.462550+0000","last_active":"2024-09-21T07:34:44.465631+0000","last_peered":"2024-09-21T07:34:44.465631+0000","last_clean":"2024-09-21T07:34:44.465631+0000","last_became_active":"2024-09-21T07:34:43.462210+0000","last_became_peered":"2024-09-21T07:34:43.462210+0000","last_unstale":"2024-09-21T07:34:44.465631+0000","last_undegraded":"2024-09-21T07:34:44.465631+0000","last_fullsized":"2024-09-21T07:34:44.465631+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:56.480314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836267+0000","last_change":"2024-09-21T07:34:42.452761+0000","last_active":"2024-09-21T07:34:44.836267+0000","last_peered":"2024-09-21T07:34:44.836267+0000","last_clean":"2024-09-21T07:34:44.836267+0000","last_became_active":"2024-09-21T07:34:42.452400+0000","last_became_peered":"2024-09-21T07:34:42.452400+0000","last_unstale":"2024-09-21T07:34:44.836267+0000","last_undegraded":"2024-09-21T07:34:44.836267+0000","last_fullsized":"2024-09-21T07:34:44.836267+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:19:05.278221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465673+0000","last_change":"2024-09-21T07:34:25.028640+0000","last_active":"2024-09-21T07:34:44.465673+0000","last_peered":"2024-09-21T07:34:44.465673+0000","last_clean":"2024-09-21T07:34:44.465673+0000","last_became_active":"2024-09-21T07:34:23.260950+0000","last_became_peered":"2024-09-21T07:34:23.260950+0000","last_unstale":"2024-09-21T07:34:44.465673+0000","last_undegraded":"2024-09-21T07:34:44.465673+0000","last_fullsized":"2024-09-21T07:34:44.465673+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:23:42.121398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055135099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833638+0000","last_change":"2024-09-21T07:34:43.453996+0000","last_active":"2024-09-21T07:34:44.833638+0000","last_peered":"2024-09-21T07:34:44.833638+0000","last_clean":"2024-09-21T07:34:44.833638+0000","last_became_active":"2024-09-21T07:34:43.453666+0000","last_became_peered":"2024-09-21T07:34:43.453666+0000","last_unstale":"2024-09-21T07:34:44.833638+0000","last_undegraded":"2024-09-21T07:34:44.833638+0000","last_fullsized":"2024-09-21T07:34:44.833638+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:31.445807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834426+0000","last_change":"2024-09-21T07:34:42.453822+0000","last_active":"2024-09-21T07:34:44.834426+0000","last_peered":"2024-09-21T07:34:44.834426+0000","last_clean":"2024-09-21T07:34:44.834426+0000","last_became_active":"2024-09-21T07:34:42.453499+0000","last_became_peered":"2024-09-21T07:34:42.453499+0000","last_unstale":"2024-09-21T07:34:44.834426+0000","last_undegraded":"2024-09-21T07:34:44.834426+0000","last_fullsized":"2024-09-21T07:34:44.834426+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:39:29.733953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836757+0000","last_change":"2024-09-21T07:34:24.260665+0000","last_active":"2024-09-21T07:34:44.836757+0000","last_peered":"2024-09-21T07:34:44.836757+0000","last_clean":"2024-09-21T07:34:44.836757+0000","last_became_active":"2024-09-21T07:34:23.635909+0000","last_became_peered":"2024-09-21T07:34:23.635909+0000","last_unstale":"2024-09-21T07:34:44.836757+0000","last_undegraded":"2024-09-21T07:34:44.836757+0000","last_fullsized":"2024-09-21T07:34:44.836757+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:44.536885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.468489+0000","last_change":"2024-09-21T07:34:43.469198+0000","last_active":"2024-09-21T07:34:44.468489+0000","last_peered":"2024-09-21T07:34:44.468489+0000","last_clean":"2024-09-21T07:34:44.468489+0000","last_became_active":"2024-09-21T07:34:43.467622+0000","last_became_peered":"2024-09-21T07:34:43.467622+0000","last_unstale":"2024-09-21T07:34:44.468489+0000","last_undegraded":"2024-09-21T07:34:44.468489+0000","last_fullsized":"2024-09-21T07:34:44.468489+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:44.866369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"48'2","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465974+0000","last_change":"2024-09-21T07:34:42.438031+0000","last_active":"2024-09-21T07:34:44.465974+0000","last_peered":"2024-09-21T07:34:44.465974+0000","last_clean":"2024-09-21T07:34:44.465974+0000","last_became_active":"2024-09-21T07:34:42.436986+0000","last_became_peered":"2024-09-21T07:34:42.436986+0000","last_unstale":"2024-09-21T07:34:44.465974+0000","last_undegraded":"2024-09-21T07:34:44.465974+0000","last_fullsized":"2024-09-21T07:34:44.465974+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:51.213865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467759+0000","last_change":"2024-09-21T07:34:24.538845+0000","last_active":"2024-09-21T07:34:44.467759+0000","last_peered":"2024-09-21T07:34:44.467759+0000","last_clean":"2024-09-21T07:34:44.467759+0000","last_became_active":"2024-09-21T07:34:23.636372+0000","last_became_peered":"2024-09-21T07:34:23.636372+0000","last_unstale":"2024-09-21T07:34:44.467759+0000","last_undegraded":"2024-09-21T07:34:44.467759+0000","last_fullsized":"2024-09-21T07:34:44.467759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:29.393813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841451,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833953+0000","last_change":"2024-09-21T07:34:43.465622+0000","last_active":"2024-09-21T07:34:44.833953+0000","last_peered":"2024-09-21T07:34:44.833953+0000","last_clean":"2024-09-21T07:34:44.833953+0000","last_became_active":"2024-09-21T07:34:43.465306+0000","last_became_peered":"2024-09-21T07:34:43.465306+0000","last_unstale":"2024-09-21T07:34:44.833953+0000","last_undegraded":"2024-09-21T07:34:44.833953+0000","last_fullsized":"2024-09-21T07:34:44.833953+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:29.579523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834840+0000","last_change":"2024-09-21T07:34:42.447519+0000","last_active":"2024-09-21T07:34:44.834840+0000","last_peered":"2024-09-21T07:34:44.834840+0000","last_clean":"2024-09-21T07:34:44.834840+0000","last_became_active":"2024-09-21T07:34:42.447129+0000","last_became_peered":"2024-09-21T07:34:42.447129+0000","last_unstale":"2024-09-21T07:34:44.834840+0000","last_undegraded":"2024-09-21T07:34:44.834840+0000","last_fullsized":"2024-09-21T07:34:44.834840+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:30.464417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":40,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833703+0000","last_change":"2024-09-21T07:34:24.259686+0000","last_active":"2024-09-21T07:34:44.833703+0000","last_peered":"2024-09-21T07:34:44.833703+0000","last_clean":"2024-09-21T07:34:44.833703+0000","last_became_active":"2024-09-21T07:34:23.274405+0000","last_became_peered":"2024-09-21T07:34:23.274405+0000","last_unstale":"2024-09-21T07:34:44.833703+0000","last_undegraded":"2024-09-21T07:34:44.833703+0000","last_fullsized":"2024-09-21T07:34:44.833703+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:26.304693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022009920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834760+0000","last_change":"2024-09-21T07:34:43.476482+0000","last_active":"2024-09-21T07:34:44.834760+0000","last_peered":"2024-09-21T07:34:44.834760+0000","last_clean":"2024-09-21T07:34:44.834760+0000","last_became_active":"2024-09-21T07:34:43.476189+0000","last_became_peered":"2024-09-21T07:34:43.476189+0000","last_unstale":"2024-09-21T07:34:44.834760+0000","last_undegraded":"2024-09-21T07:34:44.834760+0000","last_fullsized":"2024-09-21T07:34:44.834760+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:08:42.688308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833369+0000","last_change":"2024-09-21T07:34:42.456020+0000","last_active":"2024-09-21T07:34:44.833369+0000","last_peered":"2024-09-21T07:34:44.833369+0000","last_clean":"2024-09-21T07:34:44.833369+0000","last_became_active":"2024-09-21T07:34:42.455399+0000","last_became_peered":"2024-09-21T07:34:42.455399+0000","last_unstale":"2024-09-21T07:34:44.833369+0000","last_undegraded":"2024-09-21T07:34:44.833369+0000","last_fullsized":"2024-09-21T07:34:44.833369+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:57.601710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835457+0000","last_change":"2024-09-21T07:34:24.258754+0000","last_active":"2024-09-21T07:34:44.835457+0000","last_peered":"2024-09-21T07:34:44.835457+0000","last_clean":"2024-09-21T07:34:44.835457+0000","last_became_active":"2024-09-21T07:34:23.272657+0000","last_became_peered":"2024-09-21T07:34:23.272657+0000","last_unstale":"2024-09-21T07:34:44.835457+0000","last_undegraded":"2024-09-21T07:34:44.835457+0000","last_fullsized":"2024-09-21T07:34:44.835457+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:05.477099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067143200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.466771+0000","last_change":"2024-09-21T07:34:43.472869+0000","last_active":"2024-09-21T07:34:44.466771+0000","last_peered":"2024-09-21T07:34:44.466771+0000","last_clean":"2024-09-21T07:34:44.466771+0000","last_became_active":"2024-09-21T07:34:43.472514+0000","last_became_peered":"2024-09-21T07:34:43.472514+0000","last_unstale":"2024-09-21T07:34:44.466771+0000","last_undegraded":"2024-09-21T07:34:44.466771+0000","last_fullsized":"2024-09-21T07:34:44.466771+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:15:22.567958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.466329+0000","last_change":"2024-09-21T07:34:42.443208+0000","last_active":"2024-09-21T07:34:44.466329+0000","last_peered":"2024-09-21T07:34:44.466329+0000","last_clean":"2024-09-21T07:34:44.466329+0000","last_became_active":"2024-09-21T07:34:42.442891+0000","last_became_peered":"2024-09-21T07:34:42.442891+0000","last_unstale":"2024-09-21T07:34:44.466329+0000","last_undegraded":"2024-09-21T07:34:44.466329+0000","last_fullsized":"2024-09-21T07:34:44.466329+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:02.347260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":38,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833706+0000","last_change":"2024-09-21T07:34:25.085232+0000","last_active":"2024-09-21T07:34:44.833706+0000","last_peered":"2024-09-21T07:34:44.833706+0000","last_clean":"2024-09-21T07:34:44.833706+0000","last_became_active":"2024-09-21T07:34:23.277807+0000","last_became_peered":"2024-09-21T07:34:23.277807+0000","last_unstale":"2024-09-21T07:34:44.833706+0000","last_undegraded":"2024-09-21T07:34:44.833706+0000","last_fullsized":"2024-09-21T07:34:44.833706+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:36.362919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086619099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835299+0000","last_change":"2024-09-21T07:34:43.486460+0000","last_active":"2024-09-21T07:34:44.835299+0000","last_peered":"2024-09-21T07:34:44.835299+0000","last_clean":"2024-09-21T07:34:44.835299+0000","last_became_active":"2024-09-21T07:34:43.486148+0000","last_became_peered":"2024-09-21T07:34:43.486148+0000","last_unstale":"2024-09-21T07:34:44.835299+0000","last_undegraded":"2024-09-21T07:34:44.835299+0000","last_fullsized":"2024-09-21T07:34:44.835299+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:31.997216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"48'1","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835624+0000","last_change":"2024-09-21T07:34:42.450738+0000","last_active":"2024-09-21T07:34:44.835624+0000","last_peered":"2024-09-21T07:34:44.835624+0000","last_clean":"2024-09-21T07:34:44.835624+0000","last_became_active":"2024-09-21T07:34:42.450413+0000","last_became_peered":"2024-09-21T07:34:42.450413+0000","last_unstale":"2024-09-21T07:34:44.835624+0000","last_undegraded":"2024-09-21T07:34:44.835624+0000","last_fullsized":"2024-09-21T07:34:44.835624+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:48.569857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835043+0000","last_change":"2024-09-21T07:34:24.597672+0000","last_active":"2024-09-21T07:34:44.835043+0000","last_peered":"2024-09-21T07:34:44.835043+0000","last_clean":"2024-09-21T07:34:44.835043+0000","last_became_active":"2024-09-21T07:34:23.267770+0000","last_became_peered":"2024-09-21T07:34:23.267770+0000","last_unstale":"2024-09-21T07:34:44.835043+0000","last_undegraded":"2024-09-21T07:34:44.835043+0000","last_fullsized":"2024-09-21T07:34:44.835043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:51.629476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086378500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465282+0000","last_change":"2024-09-21T07:34:43.478411+0000","last_active":"2024-09-21T07:34:44.465282+0000","last_peered":"2024-09-21T07:34:44.465282+0000","last_clean":"2024-09-21T07:34:44.465282+0000","last_became_active":"2024-09-21T07:34:43.478106+0000","last_became_peered":"2024-09-21T07:34:43.478106+0000","last_unstale":"2024-09-21T07:34:44.465282+0000","last_undegraded":"2024-09-21T07:34:44.465282+0000","last_fullsized":"2024-09-21T07:34:44.465282+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:41.320870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"48'2","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467059+0000","last_change":"2024-09-21T07:34:42.449630+0000","last_active":"2024-09-21T07:34:44.467059+0000","last_peered":"2024-09-21T07:34:44.467059+0000","last_clean":"2024-09-21T07:34:44.467059+0000","last_became_active":"2024-09-21T07:34:42.448013+0000","last_became_peered":"2024-09-21T07:34:42.448013+0000","last_unstale":"2024-09-21T07:34:44.467059+0000","last_undegraded":"2024-09-21T07:34:44.467059+0000","last_fullsized":"2024-09-21T07:34:44.467059+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:42.900618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":38,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467361+0000","last_change":"2024-09-21T07:34:24.260181+0000","last_active":"2024-09-21T07:34:44.467361+0000","last_peered":"2024-09-21T07:34:44.467361+0000","last_clean":"2024-09-21T07:34:44.467361+0000","last_became_active":"2024-09-21T07:34:23.275580+0000","last_became_peered":"2024-09-21T07:34:23.275580+0000","last_unstale":"2024-09-21T07:34:44.467361+0000","last_undegraded":"2024-09-21T07:34:44.467361+0000","last_fullsized":"2024-09-21T07:34:44.467361+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:49.857622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014706980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834317+0000","last_change":"2024-09-21T07:34:43.479707+0000","last_active":"2024-09-21T07:34:44.834317+0000","last_peered":"2024-09-21T07:34:44.834317+0000","last_clean":"2024-09-21T07:34:44.834317+0000","last_became_active":"2024-09-21T07:34:43.479311+0000","last_became_peered":"2024-09-21T07:34:43.479311+0000","last_unstale":"2024-09-21T07:34:44.834317+0000","last_undegraded":"2024-09-21T07:34:44.834317+0000","last_fullsized":"2024-09-21T07:34:44.834317+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:03:54.452233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"48'1","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834163+0000","last_change":"2024-09-21T07:34:42.455379+0000","last_active":"2024-09-21T07:34:44.834163+0000","last_peered":"2024-09-21T07:34:44.834163+0000","last_clean":"2024-09-21T07:34:44.834163+0000","last_became_active":"2024-09-21T07:34:42.455144+0000","last_became_peered":"2024-09-21T07:34:42.455144+0000","last_unstale":"2024-09-21T07:34:44.834163+0000","last_undegraded":"2024-09-21T07:34:44.834163+0000","last_fullsized":"2024-09-21T07:34:44.834163+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:14:56.454976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":40,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834497+0000","last_change":"2024-09-21T07:34:24.259112+0000","last_active":"2024-09-21T07:34:44.834497+0000","last_peered":"2024-09-21T07:34:44.834497+0000","last_clean":"2024-09-21T07:34:44.834497+0000","last_became_active":"2024-09-21T07:34:23.635444+0000","last_became_peered":"2024-09-21T07:34:23.635444+0000","last_unstale":"2024-09-21T07:34:44.834497+0000","last_undegraded":"2024-09-21T07:34:44.834497+0000","last_fullsized":"2024-09-21T07:34:44.834497+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:46.255385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001952636,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465803+0000","last_change":"2024-09-21T07:34:43.476982+0000","last_active":"2024-09-21T07:34:44.465803+0000","last_peered":"2024-09-21T07:34:44.465803+0000","last_clean":"2024-09-21T07:34:44.465803+0000","last_became_active":"2024-09-21T07:34:43.476670+0000","last_became_peered":"2024-09-21T07:34:43.476670+0000","last_unstale":"2024-09-21T07:34:44.465803+0000","last_undegraded":"2024-09-21T07:34:44.465803+0000","last_fullsized":"2024-09-21T07:34:44.465803+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:04:42.140238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834385+0000","last_change":"2024-09-21T07:34:42.444915+0000","last_active":"2024-09-21T07:34:44.834385+0000","last_peered":"2024-09-21T07:34:44.834385+0000","last_clean":"2024-09-21T07:34:44.834385+0000","last_became_active":"2024-09-21T07:34:42.444506+0000","last_became_peered":"2024-09-21T07:34:42.444506+0000","last_unstale":"2024-09-21T07:34:44.834385+0000","last_undegraded":"2024-09-21T07:34:44.834385+0000","last_fullsized":"2024-09-21T07:34:44.834385+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:14:38.610112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.468302+0000","last_change":"2024-09-21T07:34:24.539152+0000","last_active":"2024-09-21T07:34:44.468302+0000","last_peered":"2024-09-21T07:34:44.468302+0000","last_clean":"2024-09-21T07:34:44.468302+0000","last_became_active":"2024-09-21T07:34:23.259573+0000","last_became_peered":"2024-09-21T07:34:23.259573+0000","last_unstale":"2024-09-21T07:34:44.468302+0000","last_undegraded":"2024-09-21T07:34:44.468302+0000","last_fullsized":"2024-09-21T07:34:44.468302+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:26.986900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058108400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.468175+0000","last_change":"2024-09-21T07:34:43.469411+0000","last_active":"2024-09-21T07:34:44.468175+0000","last_peered":"2024-09-21T07:34:44.468175+0000","last_clean":"2024-09-21T07:34:44.468175+0000","last_became_active":"2024-09-21T07:34:43.469098+0000","last_became_peered":"2024-09-21T07:34:43.469098+0000","last_unstale":"2024-09-21T07:34:44.468175+0000","last_undegraded":"2024-09-21T07:34:44.468175+0000","last_fullsized":"2024-09-21T07:34:44.468175+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:03.654293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"48'2","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834059+0000","last_change":"2024-09-21T07:34:42.444868+0000","last_active":"2024-09-21T07:34:44.834059+0000","last_peered":"2024-09-21T07:34:44.834059+0000","last_clean":"2024-09-21T07:34:44.834059+0000","last_became_active":"2024-09-21T07:34:42.444574+0000","last_became_peered":"2024-09-21T07:34:42.444574+0000","last_unstale":"2024-09-21T07:34:44.834059+0000","last_undegraded":"2024-09-21T07:34:44.834059+0000","last_fullsized":"2024-09-21T07:34:44.834059+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:14.168800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835810+0000","last_change":"2024-09-21T07:34:24.259902+0000","last_active":"2024-09-21T07:34:44.835810+0000","last_peered":"2024-09-21T07:34:44.835810+0000","last_clean":"2024-09-21T07:34:44.835810+0000","last_became_active":"2024-09-21T07:34:23.271902+0000","last_became_peered":"2024-09-21T07:34:23.271902+0000","last_unstale":"2024-09-21T07:34:44.835810+0000","last_undegraded":"2024-09-21T07:34:44.835810+0000","last_fullsized":"2024-09-21T07:34:44.835810+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:59.439525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001686358,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834649+0000","last_change":"2024-09-21T07:34:43.488076+0000","last_active":"2024-09-21T07:34:44.834649+0000","last_peered":"2024-09-21T07:34:44.834649+0000","last_clean":"2024-09-21T07:34:44.834649+0000","last_became_active":"2024-09-21T07:34:43.487739+0000","last_became_peered":"2024-09-21T07:34:43.487739+0000","last_unstale":"2024-09-21T07:34:44.834649+0000","last_undegraded":"2024-09-21T07:34:44.834649+0000","last_fullsized":"2024-09-21T07:34:44.834649+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:35:38.440936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834727+0000","last_change":"2024-09-21T07:34:42.452085+0000","last_active":"2024-09-21T07:34:44.834727+0000","last_peered":"2024-09-21T07:34:44.834727+0000","last_clean":"2024-09-21T07:34:44.834727+0000","last_became_active":"2024-09-21T07:34:42.451768+0000","last_became_peered":"2024-09-21T07:34:42.451768+0000","last_unstale":"2024-09-21T07:34:44.834727+0000","last_undegraded":"2024-09-21T07:34:44.834727+0000","last_fullsized":"2024-09-21T07:34:44.834727+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:41.414518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":40,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833639+0000","last_change":"2024-09-21T07:34:24.259662+0000","last_active":"2024-09-21T07:34:44.833639+0000","last_peered":"2024-09-21T07:34:44.833639+0000","last_clean":"2024-09-21T07:34:44.833639+0000","last_became_active":"2024-09-21T07:34:23.278812+0000","last_became_peered":"2024-09-21T07:34:23.278812+0000","last_unstale":"2024-09-21T07:34:44.833639+0000","last_undegraded":"2024-09-21T07:34:44.833639+0000","last_fullsized":"2024-09-21T07:34:44.833639+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:49.829974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356463,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835746+0000","last_change":"2024-09-21T07:34:43.470322+0000","last_active":"2024-09-21T07:34:44.835746+0000","last_peered":"2024-09-21T07:34:44.835746+0000","last_clean":"2024-09-21T07:34:44.835746+0000","last_became_active":"2024-09-21T07:34:43.469955+0000","last_became_peered":"2024-09-21T07:34:43.469955+0000","last_unstale":"2024-09-21T07:34:44.835746+0000","last_undegraded":"2024-09-21T07:34:44.835746+0000","last_fullsized":"2024-09-21T07:34:44.835746+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:30.715538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835746+0000","last_change":"2024-09-21T07:34:42.441191+0000","last_active":"2024-09-21T07:34:44.835746+0000","last_peered":"2024-09-21T07:34:44.835746+0000","last_clean":"2024-09-21T07:34:44.835746+0000","last_became_active":"2024-09-21T07:34:42.440881+0000","last_became_peered":"2024-09-21T07:34:42.440881+0000","last_unstale":"2024-09-21T07:34:44.835746+0000","last_undegraded":"2024-09-21T07:34:44.835746+0000","last_fullsized":"2024-09-21T07:34:44.835746+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:45.066904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":38,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833256+0000","last_change":"2024-09-21T07:34:25.084861+0000","last_active":"2024-09-21T07:34:44.833256+0000","last_peered":"2024-09-21T07:34:44.833256+0000","last_clean":"2024-09-21T07:34:44.833256+0000","last_became_active":"2024-09-21T07:34:23.277708+0000","last_became_peered":"2024-09-21T07:34:23.277708+0000","last_unstale":"2024-09-21T07:34:44.833256+0000","last_undegraded":"2024-09-21T07:34:44.833256+0000","last_fullsized":"2024-09-21T07:34:44.833256+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:40.816488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000597566,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835256+0000","last_change":"2024-09-21T07:34:43.480401+0000","last_active":"2024-09-21T07:34:44.835256+0000","last_peered":"2024-09-21T07:34:44.835256+0000","last_clean":"2024-09-21T07:34:44.835256+0000","last_became_active":"2024-09-21T07:34:43.479988+0000","last_became_peered":"2024-09-21T07:34:43.479988+0000","last_unstale":"2024-09-21T07:34:44.835256+0000","last_undegraded":"2024-09-21T07:34:44.835256+0000","last_fullsized":"2024-09-21T07:34:44.835256+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:03.660689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"48'8","reported_seq":23,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834495+0000","last_change":"2024-09-21T07:34:42.449884+0000","last_active":"2024-09-21T07:34:44.834495+0000","last_peered":"2024-09-21T07:34:44.834495+0000","last_clean":"2024-09-21T07:34:44.834495+0000","last_became_active":"2024-09-21T07:34:42.448499+0000","last_became_peered":"2024-09-21T07:34:42.448499+0000","last_unstale":"2024-09-21T07:34:44.834495+0000","last_undegraded":"2024-09-21T07:34:44.834495+0000","last_fullsized":"2024-09-21T07:34:44.834495+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:58.240549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":40,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833217+0000","last_change":"2024-09-21T07:34:24.258709+0000","last_active":"2024-09-21T07:34:44.833217+0000","last_peered":"2024-09-21T07:34:44.833217+0000","last_clean":"2024-09-21T07:34:44.833217+0000","last_became_active":"2024-09-21T07:34:23.279042+0000","last_became_peered":"2024-09-21T07:34:23.279042+0000","last_unstale":"2024-09-21T07:34:44.833217+0000","last_undegraded":"2024-09-21T07:34:44.833217+0000","last_fullsized":"2024-09-21T07:34:44.833217+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:01.762117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019282100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833735+0000","last_change":"2024-09-21T07:34:43.466176+0000","last_active":"2024-09-21T07:34:44.833735+0000","last_peered":"2024-09-21T07:34:44.833735+0000","last_clean":"2024-09-21T07:34:44.833735+0000","last_became_active":"2024-09-21T07:34:43.464214+0000","last_became_peered":"2024-09-21T07:34:43.464214+0000","last_unstale":"2024-09-21T07:34:44.833735+0000","last_undegraded":"2024-09-21T07:34:44.833735+0000","last_fullsized":"2024-09-21T07:34:44.833735+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:22.282533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836035+0000","last_change":"2024-09-21T07:34:42.442308+0000","last_active":"2024-09-21T07:34:44.836035+0000","last_peered":"2024-09-21T07:34:44.836035+0000","last_clean":"2024-09-21T07:34:44.836035+0000","last_became_active":"2024-09-21T07:34:42.442018+0000","last_became_peered":"2024-09-21T07:34:42.442018+0000","last_unstale":"2024-09-21T07:34:44.836035+0000","last_undegraded":"2024-09-21T07:34:44.836035+0000","last_fullsized":"2024-09-21T07:34:44.836035+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:53.642537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":38,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835026+0000","last_change":"2024-09-21T07:34:25.085635+0000","last_active":"2024-09-21T07:34:44.835026+0000","last_peered":"2024-09-21T07:34:44.835026+0000","last_clean":"2024-09-21T07:34:44.835026+0000","last_became_active":"2024-09-21T07:34:23.273566+0000","last_became_peered":"2024-09-21T07:34:23.273566+0000","last_unstale":"2024-09-21T07:34:44.835026+0000","last_undegraded":"2024-09-21T07:34:44.835026+0000","last_fullsized":"2024-09-21T07:34:44.835026+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:14.142272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079897300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467666+0000","last_change":"2024-09-21T07:34:43.472808+0000","last_active":"2024-09-21T07:34:44.467666+0000","last_peered":"2024-09-21T07:34:44.467666+0000","last_clean":"2024-09-21T07:34:44.467666+0000","last_became_active":"2024-09-21T07:34:43.466752+0000","last_became_peered":"2024-09-21T07:34:43.466752+0000","last_unstale":"2024-09-21T07:34:44.467666+0000","last_undegraded":"2024-09-21T07:34:44.467666+0000","last_fullsized":"2024-09-21T07:34:44.467666+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:19:06.280633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833887+0000","last_change":"2024-09-21T07:34:42.447266+0000","last_active":"2024-09-21T07:34:44.833887+0000","last_peered":"2024-09-21T07:34:44.833887+0000","last_clean":"2024-09-21T07:34:44.833887+0000","last_became_active":"2024-09-21T07:34:42.446823+0000","last_became_peered":"2024-09-21T07:34:42.446823+0000","last_unstale":"2024-09-21T07:34:44.833887+0000","last_undegraded":"2024-09-21T07:34:44.833887+0000","last_fullsized":"2024-09-21T07:34:44.833887+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:16.281729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835166+0000","last_change":"2024-09-21T07:34:24.597926+0000","last_active":"2024-09-21T07:34:44.835166+0000","last_peered":"2024-09-21T07:34:44.835166+0000","last_clean":"2024-09-21T07:34:44.835166+0000","last_became_active":"2024-09-21T07:34:23.268926+0000","last_became_peered":"2024-09-21T07:34:23.268926+0000","last_unstale":"2024-09-21T07:34:44.835166+0000","last_undegraded":"2024-09-21T07:34:44.835166+0000","last_fullsized":"2024-09-21T07:34:44.835166+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:46.190752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101851,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834479+0000","last_change":"2024-09-21T07:34:43.481754+0000","last_active":"2024-09-21T07:34:44.834479+0000","last_peered":"2024-09-21T07:34:44.834479+0000","last_clean":"2024-09-21T07:34:44.834479+0000","last_became_active":"2024-09-21T07:34:43.481460+0000","last_became_peered":"2024-09-21T07:34:43.481460+0000","last_unstale":"2024-09-21T07:34:44.834479+0000","last_undegraded":"2024-09-21T07:34:44.834479+0000","last_fullsized":"2024-09-21T07:34:44.834479+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:17:36.213769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"48'3","reported_seq":18,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836119+0000","last_change":"2024-09-21T07:34:42.450507+0000","last_active":"2024-09-21T07:34:44.836119+0000","last_peered":"2024-09-21T07:34:44.836119+0000","last_clean":"2024-09-21T07:34:44.836119+0000","last_became_active":"2024-09-21T07:34:42.449613+0000","last_became_peered":"2024-09-21T07:34:42.449613+0000","last_unstale":"2024-09-21T07:34:44.836119+0000","last_undegraded":"2024-09-21T07:34:44.836119+0000","last_fullsized":"2024-09-21T07:34:44.836119+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:59:52.817419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":38,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.466765+0000","last_change":"2024-09-21T07:34:24.259794+0000","last_active":"2024-09-21T07:34:44.466765+0000","last_peered":"2024-09-21T07:34:44.466765+0000","last_clean":"2024-09-21T07:34:44.466765+0000","last_became_active":"2024-09-21T07:34:23.265679+0000","last_became_peered":"2024-09-21T07:34:23.265679+0000","last_unstale":"2024-09-21T07:34:44.466765+0000","last_undegraded":"2024-09-21T07:34:44.466765+0000","last_fullsized":"2024-09-21T07:34:44.466765+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:02.575796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001298518,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834659+0000","last_change":"2024-09-21T07:34:43.457653+0000","last_active":"2024-09-21T07:34:44.834659+0000","last_peered":"2024-09-21T07:34:44.834659+0000","last_clean":"2024-09-21T07:34:44.834659+0000","last_became_active":"2024-09-21T07:34:43.457347+0000","last_became_peered":"2024-09-21T07:34:43.457347+0000","last_unstale":"2024-09-21T07:34:44.834659+0000","last_undegraded":"2024-09-21T07:34:44.834659+0000","last_fullsized":"2024-09-21T07:34:44.834659+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:02:38.909862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"48'6","reported_seq":21,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834377+0000","last_change":"2024-09-21T07:34:42.449115+0000","last_active":"2024-09-21T07:34:44.834377+0000","last_peered":"2024-09-21T07:34:44.834377+0000","last_clean":"2024-09-21T07:34:44.834377+0000","last_became_active":"2024-09-21T07:34:42.448834+0000","last_became_peered":"2024-09-21T07:34:42.448834+0000","last_unstale":"2024-09-21T07:34:44.834377+0000","last_undegraded":"2024-09-21T07:34:44.834377+0000","last_fullsized":"2024-09-21T07:34:44.834377+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:49.800427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836377+0000","last_change":"2024-09-21T07:34:24.260330+0000","last_active":"2024-09-21T07:34:44.836377+0000","last_peered":"2024-09-21T07:34:44.836377+0000","last_clean":"2024-09-21T07:34:44.836377+0000","last_became_active":"2024-09-21T07:34:23.637210+0000","last_became_peered":"2024-09-21T07:34:23.637210+0000","last_unstale":"2024-09-21T07:34:44.836377+0000","last_undegraded":"2024-09-21T07:34:44.836377+0000","last_fullsized":"2024-09-21T07:34:44.836377+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.488451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001537093,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834186+0000","last_change":"2024-09-21T07:34:43.481741+0000","last_active":"2024-09-21T07:34:44.834186+0000","last_peered":"2024-09-21T07:34:44.834186+0000","last_clean":"2024-09-21T07:34:44.834186+0000","last_became_active":"2024-09-21T07:34:43.481368+0000","last_became_peered":"2024-09-21T07:34:43.481368+0000","last_unstale":"2024-09-21T07:34:44.834186+0000","last_undegraded":"2024-09-21T07:34:44.834186+0000","last_fullsized":"2024-09-21T07:34:44.834186+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:16.752080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"48'1","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833850+0000","last_change":"2024-09-21T07:34:42.455980+0000","last_active":"2024-09-21T07:34:44.833850+0000","last_peered":"2024-09-21T07:34:44.833850+0000","last_clean":"2024-09-21T07:34:44.833850+0000","last_became_active":"2024-09-21T07:34:42.455368+0000","last_became_peered":"2024-09-21T07:34:42.455368+0000","last_unstale":"2024-09-21T07:34:44.833850+0000","last_undegraded":"2024-09-21T07:34:44.833850+0000","last_fullsized":"2024-09-21T07:34:44.833850+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:07:34.444440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833846+0000","last_change":"2024-09-21T07:34:24.597286+0000","last_active":"2024-09-21T07:34:44.833846+0000","last_peered":"2024-09-21T07:34:44.833846+0000","last_clean":"2024-09-21T07:34:44.833846+0000","last_became_active":"2024-09-21T07:34:23.267133+0000","last_became_peered":"2024-09-21T07:34:23.267133+0000","last_unstale":"2024-09-21T07:34:44.833846+0000","last_undegraded":"2024-09-21T07:34:44.833846+0000","last_fullsized":"2024-09-21T07:34:44.833846+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:18.257010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836565+0000","last_change":"2024-09-21T07:34:43.476520+0000","last_active":"2024-09-21T07:34:44.836565+0000","last_peered":"2024-09-21T07:34:44.836565+0000","last_clean":"2024-09-21T07:34:44.836565+0000","last_became_active":"2024-09-21T07:34:43.476159+0000","last_became_peered":"2024-09-21T07:34:43.476159+0000","last_unstale":"2024-09-21T07:34:44.836565+0000","last_undegraded":"2024-09-21T07:34:44.836565+0000","last_fullsized":"2024-09-21T07:34:44.836565+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:27.525343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"48'1","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833695+0000","last_change":"2024-09-21T07:34:42.441002+0000","last_active":"2024-09-21T07:34:44.833695+0000","last_peered":"2024-09-21T07:34:44.833695+0000","last_clean":"2024-09-21T07:34:44.833695+0000","last_became_active":"2024-09-21T07:34:42.440702+0000","last_became_peered":"2024-09-21T07:34:42.440702+0000","last_unstale":"2024-09-21T07:34:44.833695+0000","last_undegraded":"2024-09-21T07:34:44.833695+0000","last_fullsized":"2024-09-21T07:34:44.833695+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:09.841490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":40,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833536+0000","last_change":"2024-09-21T07:34:24.258712+0000","last_active":"2024-09-21T07:34:44.833536+0000","last_peered":"2024-09-21T07:34:44.833536+0000","last_clean":"2024-09-21T07:34:44.833536+0000","last_became_active":"2024-09-21T07:34:23.274692+0000","last_became_peered":"2024-09-21T07:34:23.274692+0000","last_unstale":"2024-09-21T07:34:44.833536+0000","last_undegraded":"2024-09-21T07:34:44.833536+0000","last_fullsized":"2024-09-21T07:34:44.833536+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:43.279454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018051479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835466+0000","last_change":"2024-09-21T07:34:43.463236+0000","last_active":"2024-09-21T07:34:44.835466+0000","last_peered":"2024-09-21T07:34:44.835466+0000","last_clean":"2024-09-21T07:34:44.835466+0000","last_became_active":"2024-09-21T07:34:43.458917+0000","last_became_peered":"2024-09-21T07:34:43.458917+0000","last_unstale":"2024-09-21T07:34:44.835466+0000","last_undegraded":"2024-09-21T07:34:44.835466+0000","last_fullsized":"2024-09-21T07:34:44.835466+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:45:19.428476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834561+0000","last_change":"2024-09-21T07:34:42.447422+0000","last_active":"2024-09-21T07:34:44.834561+0000","last_peered":"2024-09-21T07:34:44.834561+0000","last_clean":"2024-09-21T07:34:44.834561+0000","last_became_active":"2024-09-21T07:34:42.447132+0000","last_became_peered":"2024-09-21T07:34:42.447132+0000","last_unstale":"2024-09-21T07:34:44.834561+0000","last_undegraded":"2024-09-21T07:34:44.834561+0000","last_fullsized":"2024-09-21T07:34:44.834561+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:26:24.394876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":38,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833146+0000","last_change":"2024-09-21T07:34:25.084849+0000","last_active":"2024-09-21T07:34:44.833146+0000","last_peered":"2024-09-21T07:34:44.833146+0000","last_clean":"2024-09-21T07:34:44.833146+0000","last_became_active":"2024-09-21T07:34:23.277713+0000","last_became_peered":"2024-09-21T07:34:23.277713+0000","last_unstale":"2024-09-21T07:34:44.833146+0000","last_undegraded":"2024-09-21T07:34:44.833146+0000","last_fullsized":"2024-09-21T07:34:44.833146+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:05.811810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059079499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.836091+0000","last_change":"2024-09-21T07:34:43.472697+0000","last_active":"2024-09-21T07:34:44.836091+0000","last_peered":"2024-09-21T07:34:44.836091+0000","last_clean":"2024-09-21T07:34:44.836091+0000","last_became_active":"2024-09-21T07:34:43.472276+0000","last_became_peered":"2024-09-21T07:34:43.472276+0000","last_unstale":"2024-09-21T07:34:44.836091+0000","last_undegraded":"2024-09-21T07:34:44.836091+0000","last_fullsized":"2024-09-21T07:34:44.836091+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:49.596333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833099+0000","last_change":"2024-09-21T07:34:42.453339+0000","last_active":"2024-09-21T07:34:44.833099+0000","last_peered":"2024-09-21T07:34:44.833099+0000","last_clean":"2024-09-21T07:34:44.833099+0000","last_became_active":"2024-09-21T07:34:42.452974+0000","last_became_peered":"2024-09-21T07:34:42.452974+0000","last_unstale":"2024-09-21T07:34:44.833099+0000","last_undegraded":"2024-09-21T07:34:44.833099+0000","last_fullsized":"2024-09-21T07:34:44.833099+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:22.788726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467897+0000","last_change":"2024-09-21T07:34:24.538610+0000","last_active":"2024-09-21T07:34:44.467897+0000","last_peered":"2024-09-21T07:34:44.467897+0000","last_clean":"2024-09-21T07:34:44.467897+0000","last_became_active":"2024-09-21T07:34:23.259700+0000","last_became_peered":"2024-09-21T07:34:23.259700+0000","last_unstale":"2024-09-21T07:34:44.467897+0000","last_undegraded":"2024-09-21T07:34:44.467897+0000","last_fullsized":"2024-09-21T07:34:44.467897+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:57.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070522399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834793+0000","last_change":"2024-09-21T07:34:43.474381+0000","last_active":"2024-09-21T07:34:44.834793+0000","last_peered":"2024-09-21T07:34:44.834793+0000","last_clean":"2024-09-21T07:34:44.834793+0000","last_became_active":"2024-09-21T07:34:43.473953+0000","last_became_peered":"2024-09-21T07:34:43.473953+0000","last_unstale":"2024-09-21T07:34:44.834793+0000","last_undegraded":"2024-09-21T07:34:44.834793+0000","last_fullsized":"2024-09-21T07:34:44.834793+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:31.362212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"48'4","reported_seq":19,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.834187+0000","last_change":"2024-09-21T07:34:42.444572+0000","last_active":"2024-09-21T07:34:44.834187+0000","last_peered":"2024-09-21T07:34:44.834187+0000","last_clean":"2024-09-21T07:34:44.834187+0000","last_became_active":"2024-09-21T07:34:42.444254+0000","last_became_peered":"2024-09-21T07:34:42.444254+0000","last_unstale":"2024-09-21T07:34:44.834187+0000","last_undegraded":"2024-09-21T07:34:44.834187+0000","last_fullsized":"2024-09-21T07:34:44.834187+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:11:02.002169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835079+0000","last_change":"2024-09-21T07:34:24.597597+0000","last_active":"2024-09-21T07:34:44.835079+0000","last_peered":"2024-09-21T07:34:44.835079+0000","last_clean":"2024-09-21T07:34:44.835079+0000","last_became_active":"2024-09-21T07:34:23.635204+0000","last_became_peered":"2024-09-21T07:34:23.635204+0000","last_unstale":"2024-09-21T07:34:44.835079+0000","last_undegraded":"2024-09-21T07:34:44.835079+0000","last_fullsized":"2024-09-21T07:34:44.835079+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:48.644789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081139199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467728+0000","last_change":"2024-09-21T07:34:43.460645+0000","last_active":"2024-09-21T07:34:44.467728+0000","last_peered":"2024-09-21T07:34:44.467728+0000","last_clean":"2024-09-21T07:34:44.467728+0000","last_became_active":"2024-09-21T07:34:43.460320+0000","last_became_peered":"2024-09-21T07:34:43.460320+0000","last_unstale":"2024-09-21T07:34:44.467728+0000","last_undegraded":"2024-09-21T07:34:44.467728+0000","last_fullsized":"2024-09-21T07:34:44.467728+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:54.644949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"48'1","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.465954+0000","last_change":"2024-09-21T07:34:42.446762+0000","last_active":"2024-09-21T07:34:44.465954+0000","last_peered":"2024-09-21T07:34:44.465954+0000","last_clean":"2024-09-21T07:34:44.465954+0000","last_became_active":"2024-09-21T07:34:42.446320+0000","last_became_peered":"2024-09-21T07:34:42.446320+0000","last_unstale":"2024-09-21T07:34:44.465954+0000","last_undegraded":"2024-09-21T07:34:44.465954+0000","last_fullsized":"2024-09-21T07:34:44.465954+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:46.161972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":40,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835380+0000","last_change":"2024-09-21T07:34:24.259026+0000","last_active":"2024-09-21T07:34:44.835380+0000","last_peered":"2024-09-21T07:34:44.835380+0000","last_clean":"2024-09-21T07:34:44.835380+0000","last_became_active":"2024-09-21T07:34:23.635456+0000","last_became_peered":"2024-09-21T07:34:23.635456+0000","last_unstale":"2024-09-21T07:34:44.835380+0000","last_undegraded":"2024-09-21T07:34:44.835380+0000","last_fullsized":"2024-09-21T07:34:44.835380+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:09.592752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018654419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833792+0000","last_change":"2024-09-21T07:34:43.456358+0000","last_active":"2024-09-21T07:34:44.833792+0000","last_peered":"2024-09-21T07:34:44.833792+0000","last_clean":"2024-09-21T07:34:44.833792+0000","last_became_active":"2024-09-21T07:34:43.456052+0000","last_became_peered":"2024-09-21T07:34:43.456052+0000","last_unstale":"2024-09-21T07:34:44.833792+0000","last_undegraded":"2024-09-21T07:34:44.833792+0000","last_fullsized":"2024-09-21T07:34:44.833792+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:07:17.182115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"48'2","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467319+0000","last_change":"2024-09-21T07:34:42.443581+0000","last_active":"2024-09-21T07:34:44.467319+0000","last_peered":"2024-09-21T07:34:44.467319+0000","last_clean":"2024-09-21T07:34:44.467319+0000","last_became_active":"2024-09-21T07:34:42.443284+0000","last_became_peered":"2024-09-21T07:34:42.443284+0000","last_unstale":"2024-09-21T07:34:44.467319+0000","last_undegraded":"2024-09-21T07:34:44.467319+0000","last_fullsized":"2024-09-21T07:34:44.467319+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:59:14.692524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.835966+0000","last_change":"2024-09-21T07:34:24.260300+0000","last_active":"2024-09-21T07:34:44.835966+0000","last_peered":"2024-09-21T07:34:44.835966+0000","last_clean":"2024-09-21T07:34:44.835966+0000","last_became_active":"2024-09-21T07:34:23.270128+0000","last_became_peered":"2024-09-21T07:34:23.270128+0000","last_unstale":"2024-09-21T07:34:44.835966+0000","last_undegraded":"2024-09-21T07:34:44.835966+0000","last_fullsized":"2024-09-21T07:34:44.835966+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:25.260931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00176651,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.833363+0000","last_change":"2024-09-21T07:34:43.457202+0000","last_active":"2024-09-21T07:34:44.833363+0000","last_peered":"2024-09-21T07:34:44.833363+0000","last_clean":"2024-09-21T07:34:44.833363+0000","last_became_active":"2024-09-21T07:34:43.456851+0000","last_became_peered":"2024-09-21T07:34:43.456851+0000","last_unstale":"2024-09-21T07:34:44.833363+0000","last_undegraded":"2024-09-21T07:34:44.833363+0000","last_fullsized":"2024-09-21T07:34:44.833363+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:42.415959+0000","last_clean_scrub_stamp":"2024-09-21T07:34:42.415959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:12.056767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"48'2","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.468447+0000","last_change":"2024-09-21T07:34:42.434252+0000","last_active":"2024-09-21T07:34:44.468447+0000","last_peered":"2024-09-21T07:34:44.468447+0000","last_clean":"2024-09-21T07:34:44.468447+0000","last_became_active":"2024-09-21T07:34:42.433936+0000","last_became_peered":"2024-09-21T07:34:42.433936+0000","last_unstale":"2024-09-21T07:34:44.468447+0000","last_undegraded":"2024-09-21T07:34:44.468447+0000","last_fullsized":"2024-09-21T07:34:44.468447+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:41.413546+0000","last_clean_scrub_stamp":"2024-09-21T07:34:41.413546+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:13.202719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-21T07:34:44.467474+0000","last_change":"2024-09-21T07:34:25.028904+0000","last_active":"2024-09-21T07:34:44.467474+0000","last_peered":"2024-09-21T07:34:44.467474+0000","last_clean":"2024-09-21T07:34:44.467474+0000","last_became_active":"2024-09-21T07:34:23.260942+0000","last_became_peered":"2024-09-21T07:34:23.260942+0000","last_unstale":"2024-09-21T07:34:44.467474+0000","last_undegraded":"2024-09-21T07:34:44.467474+0000","last_fullsized":"2024-09-21T07:34:44.467474+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:22.709328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060553899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509464,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32332,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339508,"statfs":{"total":96636764160,"available":96603656192,"internally_reserved":0,"allocated":1171456,"data_stored":832323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509466,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":577536,"data_stored":242483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509464,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":569344,"data_stored":241831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509465,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31764,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340076,"statfs":{"total":96636764160,"available":96604237824,"internally_reserved":0,"allocated":581632,"data_stored":243443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509466,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31756,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340084,"statfs":{"total":96636764160,"available":96604246016,"internally_reserved":0,"allocated":577536,"data_stored":241991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509466,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1171456,"data_stored":832255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542170,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31748,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340092,"statfs":{"total":96636764160,"available":96604254208,"internally_reserved":0,"allocated":573440,"data_stored":242413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542169,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31760,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340080,"statfs":{"total":96636764160,"available":96604241920,"internally_reserved":0,"allocated":585728,"data_stored":243465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:34:53.512 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T07:34:53.512 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T07:34:53.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.781+0000 7f62e5600640 1 -- 172.21.15.39:0/3633813353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62e015c9e0 msgr2=0x7f62e015cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:53.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.781+0000 7f62e5600640 1 --2- 172.21.15.39:0/3633813353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62e015c9e0 0x7f62e015cdf0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009830 tx=0x7f62cc02a510 comp rx=0 tx=0).stop 2024-09-21T07:34:53.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.782+0000 7f62e5600640 1 -- 172.21.15.39:0/3633813353 shutdown_connections 2024-09-21T07:34:53.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.782+0000 7f62e5600640 1 --2- 172.21.15.39:0/3633813353 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f62e006ff60 0x7f62e0070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:53.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.782+0000 7f62e5600640 1 --2- 172.21.15.39:0/3633813353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62e015c9e0 0x7f62e015cdf0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:53.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.782+0000 7f62e5600640 1 -- 172.21.15.39:0/3633813353 >> 172.21.15.39:0/3633813353 conn(0x7f62e00b3090 msgr2=0x7f62e00b34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:53.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.782+0000 7f62e5600640 1 -- 172.21.15.39:0/3633813353 shutdown_connections 2024-09-21T07:34:53.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.782+0000 7f62e5600640 1 -- 172.21.15.39:0/3633813353 wait complete. 2024-09-21T07:34:53.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.784+0000 7f62e5600640 1 Processor -- start 2024-09-21T07:34:53.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.784+0000 7f62e5600640 1 -- start start 2024-09-21T07:34:53.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.785+0000 7f62e5600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62e006ff60 0x7f62e0143dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:53.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.785+0000 7f62e5600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f62e00d08b0 0x7f62e0142ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:53.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.785+0000 7f62e5600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62e015c9e0 0x7f62e0132750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:53.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.785+0000 7f62e5600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f62e00cddb0 con 0x7f62e006ff60 2024-09-21T07:34:53.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.785+0000 7f62e5600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f62e00cdc30 con 0x7f62e015c9e0 2024-09-21T07:34:53.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.785+0000 7f62e5600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f62e00cdf30 con 0x7f62e00d08b0 2024-09-21T07:34:53.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.786+0000 7f62dfe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62e006ff60 0x7f62e0143dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:53.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.786+0000 7f62dfe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62e006ff60 0x7f62e0143dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57516/0 (socket says 172.21.15.39:57516) 2024-09-21T07:34:53.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.786+0000 7f62dfe00640 1 -- 172.21.15.39:0/1839420846 learned_addr learned my addr 172.21.15.39:0/1839420846 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:53.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.786+0000 7f62df400640 1 --2- 172.21.15.39:0/1839420846 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f62e00d08b0 0x7f62e0142ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:53.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.786+0000 7f62e4c00640 1 --2- 172.21.15.39:0/1839420846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62e015c9e0 0x7f62e0132750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:53.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.786+0000 7f62dfe00640 1 -- 172.21.15.39:0/1839420846 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f62e00d08b0 msgr2=0x7f62e0142ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:53.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.786+0000 7f62dfe00640 1 --2- 172.21.15.39:0/1839420846 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f62e00d08b0 0x7f62e0142ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:53.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.786+0000 7f62dfe00640 1 -- 172.21.15.39:0/1839420846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62e015c9e0 msgr2=0x7f62e0132750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:53.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.786+0000 7f62dfe00640 1 --2- 172.21.15.39:0/1839420846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62e015c9e0 0x7f62e0132750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:53.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.786+0000 7f62dfe00640 1 -- 172.21.15.39:0/1839420846 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e0142270 con 0x7f62e006ff60 2024-09-21T07:34:53.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.786+0000 7f62dfe00640 1 --2- 172.21.15.39:0/1839420846 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62e006ff60 0x7f62e0143dd0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f62d000c7a0 tx=0x7f62d000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:53.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.787+0000 7f62d7e00640 1 -- 172.21.15.39:0/1839420846 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d0017070 con 0x7f62e006ff60 2024-09-21T07:34:53.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.787+0000 7f62d7e00640 1 -- 172.21.15.39:0/1839420846 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f62d0005070 con 0x7f62e006ff60 2024-09-21T07:34:53.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.787+0000 7f62d7e00640 1 -- 172.21.15.39:0/1839420846 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d00054a0 con 0x7f62e006ff60 2024-09-21T07:34:53.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.787+0000 7f62e5600640 1 -- 172.21.15.39:0/1839420846 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62e01432f0 con 0x7f62e006ff60 2024-09-21T07:34:53.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.787+0000 7f62e5600640 1 -- 172.21.15.39:0/1839420846 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62e0142ff0 con 0x7f62e006ff60 2024-09-21T07:34:53.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.788+0000 7f62e5600640 1 -- 172.21.15.39:0/1839420846 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62e00ce450 con 0x7f62e006ff60 2024-09-21T07:34:53.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.789+0000 7f62d7e00640 1 -- 172.21.15.39:0/1839420846 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f62d001a070 con 0x7f62e006ff60 2024-09-21T07:34:53.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.790+0000 7f62d7e00640 1 --2- 172.21.15.39:0/1839420846 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f62c0078b90 0x7f62c007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:53.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.790+0000 7f62d7e00640 1 -- 172.21.15.39:0/1839420846 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6120+0+0 (secure 0 0 0) 0x7f62d009a4e0 con 0x7f62e006ff60 2024-09-21T07:34:53.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.791+0000 7f62df400640 1 --2- 172.21.15.39:0/1839420846 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f62c0078b90 0x7f62c007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:53.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.792+0000 7f62df400640 1 --2- 172.21.15.39:0/1839420846 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f62c0078b90 0x7f62c007b0a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f62cc034000 tx=0x7f62cc004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:53.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:53.795+0000 7f62d7e00640 1 -- 172.21.15.39:0/1839420846 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f62e00708b0 con 0x7f62e006ff60 2024-09-21T07:34:54.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.067+0000 7f62e5600640 1 -- 172.21.15.39:0/1839420846 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f62e0071f90 con 0x7f62e006ff60 2024-09-21T07:34:54.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.070+0000 7f62d7e00640 1 -- 172.21.15.39:0/1839420846 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) ==== 190+0+147 (secure 0 0 0) 0x7f62d0062130 con 0x7f62e006ff60 2024-09-21T07:34:54.071 INFO:teuthology.orchestra.run.smithi039.stdout:[client.0] 2024-09-21T07:34:54.071 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQB0du5mGfb1DBAA/aSMKR5or3xy53rB/iUjpA== 2024-09-21T07:34:54.071 INFO:teuthology.orchestra.run.smithi039.stdout: caps mds = "allow" 2024-09-21T07:34:54.071 INFO:teuthology.orchestra.run.smithi039.stdout: caps mon = "allow r" 2024-09-21T07:34:54.072 INFO:teuthology.orchestra.run.smithi039.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T07:34:54.072 INFO:teuthology.orchestra.run.smithi039.stderr:updated caps for client.0 2024-09-21T07:34:54.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.073+0000 7f62e5600640 1 -- 172.21.15.39:0/1839420846 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f62c0078b90 msgr2=0x7f62c007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:54.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.073+0000 7f62e5600640 1 --2- 172.21.15.39:0/1839420846 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f62c0078b90 0x7f62c007b0a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f62cc034000 tx=0x7f62cc004730 comp rx=0 tx=0).stop 2024-09-21T07:34:54.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.074+0000 7f62e5600640 1 -- 172.21.15.39:0/1839420846 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62e006ff60 msgr2=0x7f62e0143dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:54.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.074+0000 7f62e5600640 1 --2- 172.21.15.39:0/1839420846 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62e006ff60 0x7f62e0143dd0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f62d000c7a0 tx=0x7f62d000cc60 comp rx=0 tx=0).stop 2024-09-21T07:34:54.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.074+0000 7f62e5600640 1 -- 172.21.15.39:0/1839420846 shutdown_connections 2024-09-21T07:34:54.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.074+0000 7f62e5600640 1 --2- 172.21.15.39:0/1839420846 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f62c0078b90 0x7f62c007b0a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:54.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.074+0000 7f62e5600640 1 --2- 172.21.15.39:0/1839420846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62e015c9e0 0x7f62e0132750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:54.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.074+0000 7f62e5600640 1 --2- 172.21.15.39:0/1839420846 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f62e00d08b0 0x7f62e0142ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:54.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.074+0000 7f62e5600640 1 --2- 172.21.15.39:0/1839420846 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62e006ff60 0x7f62e0143dd0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:54.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.074+0000 7f62e5600640 1 -- 172.21.15.39:0/1839420846 >> 172.21.15.39:0/1839420846 conn(0x7f62e00b3090 msgr2=0x7f62e006d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:54.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.074+0000 7f62e5600640 1 -- 172.21.15.39:0/1839420846 shutdown_connections 2024-09-21T07:34:54.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.074+0000 7f62e5600640 1 -- 172.21.15.39:0/1839420846 wait complete. 2024-09-21T07:34:54.099 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:34:54.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.326+0000 7fa9c2e00640 1 -- 172.21.15.39:0/582438867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9bc0ccb50 msgr2=0x7fa9bc0ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:54.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.326+0000 7fa9c2e00640 1 --2- 172.21.15.39:0/582438867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9bc0ccb50 0x7fa9bc0ccf60 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc071d60 tx=0x7fa9b402aac0 comp rx=0 tx=0).stop 2024-09-21T07:34:54.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.326+0000 7fa9c2e00640 1 -- 172.21.15.39:0/582438867 shutdown_connections 2024-09-21T07:34:54.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.326+0000 7fa9c2e00640 1 --2- 172.21.15.39:0/582438867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9bc0ccb50 0x7fa9bc0ccf60 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:54.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.326+0000 7fa9c2e00640 1 --2- 172.21.15.39:0/582438867 >> v2:172.21.15.148:3300/0 conn(0x7fa9bc0cc1e0 0x7fa9bc0af3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:54.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.326+0000 7fa9c2e00640 1 -- 172.21.15.39:0/582438867 >> 172.21.15.39:0/582438867 conn(0x7fa9bc06d7d0 msgr2=0x7fa9bc0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:54.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.326+0000 7fa9c2e00640 1 -- 172.21.15.39:0/582438867 shutdown_connections 2024-09-21T07:34:54.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.326+0000 7fa9c2e00640 1 -- 172.21.15.39:0/582438867 wait complete. 2024-09-21T07:34:54.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.329+0000 7fa9c2e00640 1 Processor -- start 2024-09-21T07:34:54.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.329+0000 7fa9c2e00640 1 -- start start 2024-09-21T07:34:54.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.330+0000 7fa9c2e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa9bc0cc1e0 0x7fa9bc156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:54.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.330+0000 7fa9c2e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9bc0ccb50 0x7fa9bc1f4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:54.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.330+0000 7fa9c2e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9bc15d8a0 0x7fa9bc158da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:54.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.330+0000 7fa9c2e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa9bc1533a0 con 0x7fa9bc0ccb50 2024-09-21T07:34:54.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.330+0000 7fa9c2e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa9bc153520 con 0x7fa9bc15d8a0 2024-09-21T07:34:54.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.330+0000 7fa9c2e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa9bc1536a0 con 0x7fa9bc0cc1e0 2024-09-21T07:34:54.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.331+0000 7fa9b3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9bc0ccb50 0x7fa9bc1f4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:54.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.331+0000 7fa9b3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9bc0ccb50 0x7fa9bc1f4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57534/0 (socket says 172.21.15.39:57534) 2024-09-21T07:34:54.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.331+0000 7fa9b3e00640 1 -- 172.21.15.39:0/2105432691 learned_addr learned my addr 172.21.15.39:0/2105432691 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:54.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.331+0000 7fa9bbe00640 1 --2- 172.21.15.39:0/2105432691 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa9bc0cc1e0 0x7fa9bc156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:54.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.331+0000 7fa9c0e00640 1 --2- 172.21.15.39:0/2105432691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9bc15d8a0 0x7fa9bc158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:54.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.331+0000 7fa9b3e00640 1 -- 172.21.15.39:0/2105432691 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa9bc0cc1e0 msgr2=0x7fa9bc156e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:54.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.331+0000 7fa9b3e00640 1 --2- 172.21.15.39:0/2105432691 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa9bc0cc1e0 0x7fa9bc156e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:54.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.331+0000 7fa9b3e00640 1 -- 172.21.15.39:0/2105432691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9bc15d8a0 msgr2=0x7fa9bc158da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:54.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.331+0000 7fa9b3e00640 1 --2- 172.21.15.39:0/2105432691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9bc15d8a0 0x7fa9bc158da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:54.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.331+0000 7fa9b3e00640 1 -- 172.21.15.39:0/2105432691 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9bc09cef0 con 0x7fa9bc0ccb50 2024-09-21T07:34:54.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.332+0000 7fa9b3e00640 1 --2- 172.21.15.39:0/2105432691 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9bc0ccb50 0x7fa9bc1f4710 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa9a400c3d0 tx=0x7fa9a400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:54.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.332+0000 7fa9b8c00640 1 -- 172.21.15.39:0/2105432691 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4017070 con 0x7fa9bc0ccb50 2024-09-21T07:34:54.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.332+0000 7fa9b8c00640 1 -- 172.21.15.39:0/2105432691 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9a40053c0 con 0x7fa9bc0ccb50 2024-09-21T07:34:54.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.332+0000 7fa9c2e00640 1 -- 172.21.15.39:0/2105432691 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9bc09d3f0 con 0x7fa9bc0ccb50 2024-09-21T07:34:54.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.332+0000 7fa9b8c00640 1 -- 172.21.15.39:0/2105432691 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4003850 con 0x7fa9bc0ccb50 2024-09-21T07:34:54.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.332+0000 7fa9c2e00640 1 -- 172.21.15.39:0/2105432691 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9bc1f7770 con 0x7fa9bc0ccb50 2024-09-21T07:34:54.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.333+0000 7fa9c2e00640 1 -- 172.21.15.39:0/2105432691 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9bc09d160 con 0x7fa9bc0ccb50 2024-09-21T07:34:54.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.334+0000 7fa9b8c00640 1 -- 172.21.15.39:0/2105432691 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa9a401a070 con 0x7fa9bc0ccb50 2024-09-21T07:34:54.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.335+0000 7fa9b8c00640 1 --2- 172.21.15.39:0/2105432691 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa998078b90 0x7fa99807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:54.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.335+0000 7fa9b8c00640 1 -- 172.21.15.39:0/2105432691 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6120+0+0 (secure 0 0 0) 0x7fa9a409a290 con 0x7fa9bc0ccb50 2024-09-21T07:34:54.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.336+0000 7fa9bbe00640 1 --2- 172.21.15.39:0/2105432691 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa998078b90 0x7fa99807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:54.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.337+0000 7fa9bbe00640 1 --2- 172.21.15.39:0/2105432691 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa998078b90 0x7fa99807b0a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac004630 tx=0x7fa9ac00b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:54.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.340+0000 7fa9b8c00640 1 -- 172.21.15.39:0/2105432691 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa9bc0cd4a0 con 0x7fa9bc0ccb50 2024-09-21T07:34:54.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.574+0000 7fa9c2e00640 1 -- 172.21.15.39:0/2105432691 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9bc06a5e0 con 0x7fa9bc0ccb50 2024-09-21T07:34:54.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.575+0000 7fa9b8c00640 1 -- 172.21.15.39:0/2105432691 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5223 (secure 0 0 0) 0x7fa9bc06a5e0 con 0x7fa9bc0ccb50 2024-09-21T07:34:54.576 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:34:54.576 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":26,"btime":"2024-09-21T07:34:46:472997+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4958,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/878081219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":878081219},{"type":"v1","addr":"172.21.15.148:6835","nonce":878081219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3340238606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3340238606},{"type":"v1","addr":"172.21.15.148:6837","nonce":3340238606}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5307,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/929230612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":929230612},{"type":"v1","addr":"172.21.15.39:6834","nonce":929230612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:34:42.955753+0000","modified":"2024-09-21T07:34:45.984044+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/1572835489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":1572835489},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572835489}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5310,"qdb_cluster":[5310]},"id":3}]} 2024-09-21T07:34:54.576 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 26 2024-09-21T07:34:54.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.577+0000 7fa9c2e00640 1 -- 172.21.15.39:0/2105432691 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa998078b90 msgr2=0x7fa99807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:54.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.577+0000 7fa9c2e00640 1 --2- 172.21.15.39:0/2105432691 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa998078b90 0x7fa99807b0a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac004630 tx=0x7fa9ac00b2e0 comp rx=0 tx=0).stop 2024-09-21T07:34:54.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.577+0000 7fa9c2e00640 1 -- 172.21.15.39:0/2105432691 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9bc0ccb50 msgr2=0x7fa9bc1f4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:54.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.577+0000 7fa9c2e00640 1 --2- 172.21.15.39:0/2105432691 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9bc0ccb50 0x7fa9bc1f4710 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa9a400c3d0 tx=0x7fa9a400c890 comp rx=0 tx=0).stop 2024-09-21T07:34:54.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.577+0000 7fa9c2e00640 1 -- 172.21.15.39:0/2105432691 shutdown_connections 2024-09-21T07:34:54.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.577+0000 7fa9c2e00640 1 --2- 172.21.15.39:0/2105432691 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa998078b90 0x7fa99807b0a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:54.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.577+0000 7fa9c2e00640 1 --2- 172.21.15.39:0/2105432691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9bc15d8a0 0x7fa9bc158da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:54.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.577+0000 7fa9c2e00640 1 --2- 172.21.15.39:0/2105432691 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9bc0ccb50 0x7fa9bc1f4710 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:54.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.577+0000 7fa9c2e00640 1 --2- 172.21.15.39:0/2105432691 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa9bc0cc1e0 0x7fa9bc156e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:54.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.577+0000 7fa9c2e00640 1 -- 172.21.15.39:0/2105432691 >> 172.21.15.39:0/2105432691 conn(0x7fa9bc06d7d0 msgr2=0x7fa9bc0b4070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:54.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.577+0000 7fa9c2e00640 1 -- 172.21.15.39:0/2105432691 shutdown_connections 2024-09-21T07:34:54.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:54.577+0000 7fa9c2e00640 1 -- 172.21.15.39:0/2105432691 wait complete. 2024-09-21T07:34:55.005 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T07:34:42.955753+0000', 'modified': '2024-09-21T07:34:45.984044+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5310}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5310': {'gid': 5310, 'name': 'a', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.39:6835/1572835489', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 1572835489}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 1572835489}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5310, 'qdb_cluster': [5310]} 2024-09-21T07:34:55.005 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T07:34:55.005 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T07:34:55.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.234+0000 7fefbb400640 1 -- 172.21.15.39:0/993131668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef9c003a50 msgr2=0x7fef9c003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:55.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.234+0000 7fefbb400640 1 --2- 172.21.15.39:0/993131668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef9c003a50 0x7fef9c003ec0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fefa8009b50 tx=0x7fef9c005830 comp rx=0 tx=0).stop 2024-09-21T07:34:55.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.234+0000 7fefbb400640 1 -- 172.21.15.39:0/993131668 shutdown_connections 2024-09-21T07:34:55.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.234+0000 7fefbb400640 1 --2- 172.21.15.39:0/993131668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef9c003a50 0x7fef9c003ec0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:55.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.234+0000 7fefbb400640 1 -- 172.21.15.39:0/993131668 >> 172.21.15.39:0/993131668 conn(0x7fefb40b1cb0 msgr2=0x7fefb40b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:55.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.234+0000 7fefbb400640 1 -- 172.21.15.39:0/993131668 shutdown_connections 2024-09-21T07:34:55.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.234+0000 7fefbb400640 1 -- 172.21.15.39:0/993131668 wait complete. 2024-09-21T07:34:55.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.237+0000 7fefbb400640 1 Processor -- start 2024-09-21T07:34:55.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.237+0000 7fefbb400640 1 -- start start 2024-09-21T07:34:55.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.238+0000 7fefbb400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef9c003a50 0x7fefb409c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:55.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.238+0000 7fefbb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefb4153430 0x7fefb40caff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:55.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.238+0000 7fefbb400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefb40cb530 0x7fefb41f8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:55.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.238+0000 7fefbb400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fefb4074a20 con 0x7fefb4153430 2024-09-21T07:34:55.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.238+0000 7fefbb400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fefb40748a0 con 0x7fef9c003a50 2024-09-21T07:34:55.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.238+0000 7fefbb400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fefb4074ba0 con 0x7fefb40cb530 2024-09-21T07:34:55.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.238+0000 7fefb3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefb4153430 0x7fefb40caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:55.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.238+0000 7fefb3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefb4153430 0x7fefb40caff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57550/0 (socket says 172.21.15.39:57550) 2024-09-21T07:34:55.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.238+0000 7fefb3e00640 1 -- 172.21.15.39:0/3108694838 learned_addr learned my addr 172.21.15.39:0/3108694838 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:55.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.239+0000 7fefb9400640 1 --2- 172.21.15.39:0/3108694838 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefb40cb530 0x7fefb41f8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:55.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.239+0000 7fefb8a00640 1 --2- 172.21.15.39:0/3108694838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef9c003a50 0x7fefb409c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:55.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.239+0000 7fefb3e00640 1 -- 172.21.15.39:0/3108694838 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefb40cb530 msgr2=0x7fefb41f8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:55.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.239+0000 7fefb3e00640 1 --2- 172.21.15.39:0/3108694838 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefb40cb530 0x7fefb41f8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:55.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.239+0000 7fefb3e00640 1 -- 172.21.15.39:0/3108694838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef9c003a50 msgr2=0x7fefb409c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:55.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.239+0000 7fefb3e00640 1 --2- 172.21.15.39:0/3108694838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef9c003a50 0x7fefb409c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:55.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.239+0000 7fefb3e00640 1 -- 172.21.15.39:0/3108694838 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb40cb940 con 0x7fefb4153430 2024-09-21T07:34:55.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.239+0000 7fefb9400640 1 --2- 172.21.15.39:0/3108694838 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefb40cb530 0x7fefb41f8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:34:55.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.239+0000 7fefb8a00640 1 --2- 172.21.15.39:0/3108694838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef9c003a50 0x7fefb409c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:34:55.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.239+0000 7fefb3e00640 1 --2- 172.21.15.39:0/3108694838 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefb4153430 0x7fefb40caff0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fefa0004ed0 tx=0x7fefa0001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:55.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.239+0000 7fefb0c00640 1 -- 172.21.15.39:0/3108694838 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa0015020 con 0x7fefb4153430 2024-09-21T07:34:55.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.240+0000 7fefb0c00640 1 -- 172.21.15.39:0/3108694838 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fefa0011960 con 0x7fefb4153430 2024-09-21T07:34:55.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.240+0000 7fefbb400640 1 -- 172.21.15.39:0/3108694838 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefb409cf50 con 0x7fefb4153430 2024-09-21T07:34:55.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.240+0000 7fefb0c00640 1 -- 172.21.15.39:0/3108694838 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa0011d70 con 0x7fefb4153430 2024-09-21T07:34:55.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.240+0000 7fefbb400640 1 -- 172.21.15.39:0/3108694838 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefb40c82e0 con 0x7fefb4153430 2024-09-21T07:34:55.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.242+0000 7fefb0c00640 1 -- 172.21.15.39:0/3108694838 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fefa0018020 con 0x7fefb4153430 2024-09-21T07:34:55.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.243+0000 7fefb0c00640 1 --2- 172.21.15.39:0/3108694838 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fef94078b90 0x7fef9407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:55.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.243+0000 7fefb0c00640 1 -- 172.21.15.39:0/3108694838 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6120+0+0 (secure 0 0 0) 0x7fefa009ba80 con 0x7fefb4153430 2024-09-21T07:34:55.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.244+0000 7fefc2419740 1 Processor -- start 2024-09-21T07:34:55.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.244+0000 7fefb8a00640 1 --2- 172.21.15.39:0/3108694838 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fef94078b90 0x7fef9407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:55.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.244+0000 7fefc2419740 1 -- start start 2024-09-21T07:34:55.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.244+0000 7fefb8a00640 1 --2- 172.21.15.39:0/3108694838 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fef94078b90 0x7fef9407b0a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fefa8035000 tx=0x7fefa80046c0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:55.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.245+0000 7fefc2419740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555803987270 0x555803d57970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:55.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.245+0000 7fefc2419740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55580395e3b0 0x555803d57eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:55.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.245+0000 7fefc2419740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555803966590 0x555803d27f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:55.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.245+0000 7fefc2419740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x555803cb1430 con 0x555803966590 2024-09-21T07:34:55.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.245+0000 7fefc2419740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x555803ca15d0 con 0x55580395e3b0 2024-09-21T07:34:55.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.245+0000 7fefc2419740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x555803c90a00 con 0x555803987270 2024-09-21T07:34:55.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.246+0000 7fefb9400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555803966590 0x555803d27f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:55.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.246+0000 7fefb9400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555803966590 0x555803d27f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57560/0 (socket says 172.21.15.39:57560) 2024-09-21T07:34:55.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.246+0000 7fefb9400640 1 -- 172.21.15.39:0/4219538798 learned_addr learned my addr 172.21.15.39:0/4219538798 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:55.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.246+0000 7fefb8a00640 1 --2- 172.21.15.39:0/4219538798 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555803987270 0x555803d57970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:55.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.246+0000 7fefb3e00640 1 --2- 172.21.15.39:0/4219538798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55580395e3b0 0x555803d57eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:55.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.246+0000 7fefb9400640 1 -- 172.21.15.39:0/4219538798 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555803987270 msgr2=0x555803d57970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:34:55.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.246+0000 7fefb9400640 1 --2- 172.21.15.39:0/4219538798 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555803987270 0x555803d57970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:55.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.246+0000 7fefb9400640 1 -- 172.21.15.39:0/4219538798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55580395e3b0 msgr2=0x555803d57eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:34:55.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.246+0000 7fefb9400640 1 --2- 172.21.15.39:0/4219538798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55580395e3b0 0x555803d57eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:55.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.246+0000 7fefb9400640 1 -- 172.21.15.39:0/4219538798 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555803c59520 con 0x555803966590 2024-09-21T07:34:55.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.246+0000 7fefb3e00640 1 --2- 172.21.15.39:0/4219538798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55580395e3b0 0x555803d57eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:34:55.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.246+0000 7fefb9400640 1 --2- 172.21.15.39:0/4219538798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555803966590 0x555803d27f60 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fefb40b58f0 tx=0x7fefac004ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdaefc025f8df693 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:55.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.247+0000 7fefa5200640 1 -- 172.21.15.39:0/4219538798 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefac00fe30 con 0x555803966590 2024-09-21T07:34:55.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.247+0000 7fefa5200640 1 -- 172.21.15.39:0/4219538798 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fefac018070 con 0x555803966590 2024-09-21T07:34:55.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.247+0000 7fefa5200640 1 -- 172.21.15.39:0/4219538798 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefac0117c0 con 0x555803966590 2024-09-21T07:34:55.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.247+0000 7fefc2419740 1 -- 172.21.15.39:0/4219538798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555803966590 msgr2=0x555803d27f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:55.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.247+0000 7fefc2419740 1 --2- 172.21.15.39:0/4219538798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555803966590 0x555803d27f60 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fefb40b58f0 tx=0x7fefac004ec0 comp rx=0 tx=0).stop 2024-09-21T07:34:55.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.248+0000 7fefc2419740 1 -- 172.21.15.39:0/4219538798 shutdown_connections 2024-09-21T07:34:55.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.248+0000 7fefc2419740 1 --2- 172.21.15.39:0/4219538798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555803966590 0x555803d27f60 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:55.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.248+0000 7fefc2419740 1 --2- 172.21.15.39:0/4219538798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55580395e3b0 0x555803d57eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:55.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.248+0000 7fefc2419740 1 --2- 172.21.15.39:0/4219538798 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555803987270 0x555803d57970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:55.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.248+0000 7fefc2419740 1 -- 172.21.15.39:0/4219538798 >> 172.21.15.39:0/4219538798 conn(0x55580397b3c0 msgr2=0x555803d2d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:55.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.248+0000 7fefc2419740 1 -- 172.21.15.39:0/4219538798 shutdown_connections 2024-09-21T07:34:55.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.248+0000 7fefc2419740 1 -- 172.21.15.39:0/4219538798 wait complete. 2024-09-21T07:34:55.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.249+0000 7fefc2419740 1 -- start start 2024-09-21T07:34:55.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.249+0000 7fefc2419740 1 Processor -- start 2024-09-21T07:34:55.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.250+0000 7fefc2419740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55580395e3b0 0x555803d73170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:55.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.251+0000 7fefc2419740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555803966590 0x555803d85500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:55.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.251+0000 7fefc2419740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555803987270 0x555803d540c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:55.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.251+0000 7fefc2419740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x555803c37330 con 0x555803966590 2024-09-21T07:34:55.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.251+0000 7fefc2419740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x555803c94810 con 0x555803987270 2024-09-21T07:34:55.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.251+0000 7fefc2419740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x555803a01620 con 0x55580395e3b0 2024-09-21T07:34:55.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.251+0000 7fefc2419740 10 client.0 _ll_register_callbacks cb 0x555803c3c420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:34:55.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.251+0000 7fefb3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555803966590 0x555803d85500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:55.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.251+0000 7fefb3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555803966590 0x555803d85500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57570/0 (socket says 172.21.15.39:57570) 2024-09-21T07:34:55.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.251+0000 7fefb3e00640 1 -- 172.21.15.39:0/1510723686 learned_addr learned my addr 172.21.15.39:0/1510723686 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:55.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.251+0000 7fefb9400640 1 --2- 172.21.15.39:0/1510723686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555803987270 0x555803d540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:55.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.251+0000 7fefb8a00640 1 --2- 172.21.15.39:0/1510723686 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55580395e3b0 0x555803d73170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:55.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.251+0000 7fefb3e00640 1 -- 172.21.15.39:0/1510723686 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55580395e3b0 msgr2=0x555803d73170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:34:55.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.251+0000 7fefb3e00640 1 --2- 172.21.15.39:0/1510723686 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55580395e3b0 0x555803d73170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:55.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.251+0000 7fefb3e00640 1 -- 172.21.15.39:0/1510723686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555803987270 msgr2=0x555803d540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:34:55.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.251+0000 7fefb3e00640 1 --2- 172.21.15.39:0/1510723686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555803987270 0x555803d540c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:55.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.251+0000 7fefb3e00640 1 -- 172.21.15.39:0/1510723686 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558039f2850 con 0x555803966590 2024-09-21T07:34:55.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.252+0000 7fefb3e00640 1 --2- 172.21.15.39:0/1510723686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555803966590 0x555803d85500 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fefa0099d70 tx=0x7fefa005fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd16e401ff4e5170 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:55.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.252+0000 7fef99600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:34:55.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.252+0000 7fef99600640 1 -- 172.21.15.39:0/1510723686 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa004e020 con 0x555803966590 2024-09-21T07:34:55.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.252+0000 7fefc2419740 1 -- 172.21.15.39:0/1510723686 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefac018070 con 0x555803966590 2024-09-21T07:34:55.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.252+0000 7fef99600640 1 -- 172.21.15.39:0/1510723686 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(51..51 src has 1..51) ==== 6120+0+0 (secure 0 0 0) 0x7fefa00629f0 con 0x555803966590 2024-09-21T07:34:55.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.252+0000 7fef99600640 1 -- 172.21.15.39:0/1510723686 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fefa0082050 con 0x555803966590 2024-09-21T07:34:55.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.252+0000 7fefc2419740 10 client.5460 fetch_fsmap learned FSMap version 26 2024-09-21T07:34:55.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.252+0000 7fefc2419740 1 -- 172.21.15.39:0/1510723686 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x555803a696b0 con 0x555803966590 2024-09-21T07:34:55.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.252+0000 7fef99600640 1 -- 172.21.15.39:0/1510723686 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fefac018070 con 0x555803966590 2024-09-21T07:34:55.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.252+0000 7fef99600640 1 -- 172.21.15.39:0/1510723686 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) ==== 3502+0+0 (secure 0 0 0) 0x7fefa0085080 con 0x555803966590 2024-09-21T07:34:55.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.252+0000 7fefc2419740 10 client.5460 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T07:34:55.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.252+0000 7fefc2419740 10 client.5460 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:34:55.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.252+0000 7fefc2419740 20 client.5460 populate_metadata read hostname 'smithi039' 2024-09-21T07:34:55.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.252+0000 7fefc2419740 1 --2- 172.21.15.39:0/1510723686 >> [v2:172.21.15.39:6833/1572835489,v1:172.21.15.39:6835/1572835489] conn(0x555803945e40 0x555803d81f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:55.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.252+0000 7fefc2419740 4 client.5460 mds_command: new command op to 5310 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:34:55.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.252+0000 7fefc2419740 1 -- 172.21.15.39:0/1510723686 --> [v2:172.21.15.39:6833/1572835489,v1:172.21.15.39:6835/1572835489] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555803ca3520 con 0x555803945e40 2024-09-21T07:34:55.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.253+0000 7fefb8a00640 1 --2- 172.21.15.39:0/1510723686 >> [v2:172.21.15.39:6833/1572835489,v1:172.21.15.39:6835/1572835489] conn(0x555803945e40 0x555803d81f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:55.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.253+0000 7fefb8a00640 1 --2- 172.21.15.39:0/1510723686 >> [v2:172.21.15.39:6833/1572835489,v1:172.21.15.39:6835/1572835489] conn(0x555803945e40 0x555803d81f00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b1c940ef606d6cc server_cookie=d8c04a238d890795 in_seq=0 out_seq=0 2024-09-21T07:34:55.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.253+0000 7fef99600640 10 client.5460 ms_handle_connect on v2:172.21.15.39:6833/1572835489 2024-09-21T07:34:55.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.254+0000 7fef99600640 1 -- 172.21.15.39:0/1510723686 <== mds.0 v2:172.21.15.39:6833/1572835489 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555803ca3520 con 0x555803945e40 2024-09-21T07:34:55.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.254+0000 7fef99600640 10 client.5460 handle_command_reply: tid=0 2024-09-21T07:34:55.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.254+0000 7fefc2419740 1 client.5460 shutdown 2024-09-21T07:34:55.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.254+0000 7fefc2419740 1 -- 172.21.15.39:0/1510723686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555803966590 msgr2=0x555803d85500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:55.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.254+0000 7fefc2419740 1 --2- 172.21.15.39:0/1510723686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555803966590 0x555803d85500 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fefa0099d70 tx=0x7fefa005fce0 comp rx=0 tx=0).stop 2024-09-21T07:34:55.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.255+0000 7fefc2419740 1 -- 172.21.15.39:0/1510723686 shutdown_connections 2024-09-21T07:34:55.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.255+0000 7fefc2419740 1 --2- 172.21.15.39:0/1510723686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555803987270 0x555803d540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:55.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.255+0000 7fefc2419740 1 --2- 172.21.15.39:0/1510723686 >> [v2:172.21.15.39:6833/1572835489,v1:172.21.15.39:6835/1572835489] conn(0x555803945e40 0x555803d81f00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:55.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.255+0000 7fefc2419740 1 --2- 172.21.15.39:0/1510723686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555803966590 0x555803d85500 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:55.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.255+0000 7fefc2419740 1 --2- 172.21.15.39:0/1510723686 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55580395e3b0 0x555803d73170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:55.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.255+0000 7fef99600640 0 client.5460 ms_handle_reset on v2:172.21.15.39:6833/1572835489 2024-09-21T07:34:55.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.255+0000 7fefc2419740 1 -- 172.21.15.39:0/1510723686 >> 172.21.15.39:0/1510723686 conn(0x55580397b3c0 msgr2=0x555803d755b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:55.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.255+0000 7fefc2419740 1 -- 172.21.15.39:0/1510723686 shutdown_connections 2024-09-21T07:34:55.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.255+0000 7fefc2419740 1 -- 172.21.15.39:0/1510723686 wait complete. 2024-09-21T07:34:55.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.255+0000 7fefc2419740 20 client.5460 trim_cache size 0 max 16384 2024-09-21T07:34:55.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.276+0000 7fefc2419740 1 Processor -- start 2024-09-21T07:34:55.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.276+0000 7fefc2419740 1 -- start start 2024-09-21T07:34:55.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.277+0000 7fefc2419740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55580395e3b0 0x555803d73bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:55.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.277+0000 7fefc2419740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555803966590 0x555803d74130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:55.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.277+0000 7fefc2419740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555803987270 0x555803d957d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:55.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.277+0000 7fefc2419740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fefb4202480 con 0x555803987270 2024-09-21T07:34:55.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.277+0000 7fefc2419740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fefb40c9850 con 0x555803966590 2024-09-21T07:34:55.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.277+0000 7fefc2419740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fefb4201e00 con 0x55580395e3b0 2024-09-21T07:34:55.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.277+0000 7fefb9400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555803987270 0x555803d957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:55.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.278+0000 7fefb9400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555803987270 0x555803d957d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57576/0 (socket says 172.21.15.39:57576) 2024-09-21T07:34:55.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.278+0000 7fefb9400640 1 -- 172.21.15.39:0/2043642056 learned_addr learned my addr 172.21.15.39:0/2043642056 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:55.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.278+0000 7fefb8a00640 1 --2- 172.21.15.39:0/2043642056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55580395e3b0 0x555803d73bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:55.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.278+0000 7fefb3e00640 1 --2- 172.21.15.39:0/2043642056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555803966590 0x555803d74130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:55.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.278+0000 7fefb9400640 1 -- 172.21.15.39:0/2043642056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55580395e3b0 msgr2=0x555803d73bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:34:55.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.278+0000 7fefb9400640 1 --2- 172.21.15.39:0/2043642056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55580395e3b0 0x555803d73bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:55.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.278+0000 7fefb9400640 1 -- 172.21.15.39:0/2043642056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555803966590 msgr2=0x555803d74130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:34:55.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.278+0000 7fefb9400640 1 --2- 172.21.15.39:0/2043642056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555803966590 0x555803d74130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:55.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.278+0000 7fefb9400640 1 -- 172.21.15.39:0/2043642056 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555803d2e650 con 0x555803987270 2024-09-21T07:34:55.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.278+0000 7fefb9400640 1 --2- 172.21.15.39:0/2043642056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555803987270 0x555803d957d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fefac001a40 tx=0x7fefb40ccd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=730f932cbad25047 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:55.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.279+0000 7fef9be00640 1 -- 172.21.15.39:0/2043642056 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefac011520 con 0x555803987270 2024-09-21T07:34:55.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.279+0000 7fef9be00640 1 -- 172.21.15.39:0/2043642056 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fefac018070 con 0x555803987270 2024-09-21T07:34:55.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.279+0000 7fef9be00640 1 -- 172.21.15.39:0/2043642056 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefac011a70 con 0x555803987270 2024-09-21T07:34:55.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.280+0000 7fefc2419740 1 -- 172.21.15.39:0/2043642056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555803987270 msgr2=0x555803d957d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:55.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.280+0000 7fefc2419740 1 --2- 172.21.15.39:0/2043642056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555803987270 0x555803d957d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fefac001a40 tx=0x7fefb40ccd00 comp rx=0 tx=0).stop 2024-09-21T07:34:55.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.280+0000 7fefc2419740 1 -- 172.21.15.39:0/2043642056 shutdown_connections 2024-09-21T07:34:55.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.280+0000 7fefc2419740 1 --2- 172.21.15.39:0/2043642056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555803987270 0x555803d957d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:55.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.280+0000 7fefc2419740 1 --2- 172.21.15.39:0/2043642056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555803966590 0x555803d74130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:55.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.280+0000 7fefc2419740 1 --2- 172.21.15.39:0/2043642056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55580395e3b0 0x555803d73bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:55.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.280+0000 7fefc2419740 1 -- 172.21.15.39:0/2043642056 >> 172.21.15.39:0/2043642056 conn(0x55580397b3c0 msgr2=0x555803d28c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:55.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.280+0000 7fefc2419740 1 -- 172.21.15.39:0/2043642056 shutdown_connections 2024-09-21T07:34:55.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.280+0000 7fefc2419740 1 -- 172.21.15.39:0/2043642056 wait complete. 2024-09-21T07:34:55.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.281+0000 7fefc2419740 1 -- start start 2024-09-21T07:34:55.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.281+0000 7fefc2419740 1 Processor -- start 2024-09-21T07:34:55.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.283+0000 7fefc2419740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55580395e3b0 0x555803df5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:55.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.283+0000 7fefc2419740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555803966590 0x555803d576d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:55.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.283+0000 7fefc2419740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555803987270 0x555803df1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:55.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.283+0000 7fefc2419740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x555803d6d750 con 0x555803987270 2024-09-21T07:34:55.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.283+0000 7fefc2419740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x555803d50b60 con 0x555803966590 2024-09-21T07:34:55.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.283+0000 7fefc2419740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x555803d4dde0 con 0x55580395e3b0 2024-09-21T07:34:55.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.283+0000 7fefc2419740 10 client.0 _ll_register_callbacks cb 0x7fefa0063350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:34:55.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.283+0000 7fefb9400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555803987270 0x555803df1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:55.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.283+0000 7fefb9400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555803987270 0x555803df1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57590/0 (socket says 172.21.15.39:57590) 2024-09-21T07:34:55.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.283+0000 7fefb9400640 1 -- 172.21.15.39:0/1232668538 learned_addr learned my addr 172.21.15.39:0/1232668538 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:55.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.283+0000 7fefb8a00640 1 --2- 172.21.15.39:0/1232668538 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55580395e3b0 0x555803df5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:55.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.283+0000 7fefb3e00640 1 --2- 172.21.15.39:0/1232668538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555803966590 0x555803d576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:55.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.283+0000 7fefb9400640 1 -- 172.21.15.39:0/1232668538 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55580395e3b0 msgr2=0x555803df5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:34:55.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.283+0000 7fefb9400640 1 --2- 172.21.15.39:0/1232668538 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55580395e3b0 0x555803df5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:55.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.283+0000 7fefb9400640 1 -- 172.21.15.39:0/1232668538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555803966590 msgr2=0x555803d576d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:34:55.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.283+0000 7fefb9400640 1 --2- 172.21.15.39:0/1232668538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555803966590 0x555803d576d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:55.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.283+0000 7fefb9400640 1 -- 172.21.15.39:0/1232668538 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555803d57c10 con 0x555803987270 2024-09-21T07:34:55.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.283+0000 7fefb8a00640 1 --2- 172.21.15.39:0/1232668538 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55580395e3b0 0x555803df5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:34:55.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.283+0000 7fefb3e00640 1 --2- 172.21.15.39:0/1232668538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555803966590 0x555803d576d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:34:55.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.284+0000 7fefb9400640 1 --2- 172.21.15.39:0/1232668538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555803987270 0x555803df1550 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fefac002080 tx=0x7fefac011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d001b0deff7f32 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:55.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.284+0000 7fef99600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:34:55.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.284+0000 7fef99600640 1 -- 172.21.15.39:0/1232668538 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefac011520 con 0x555803987270 2024-09-21T07:34:55.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.284+0000 7fef99600640 1 -- 172.21.15.39:0/1232668538 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(51..51 src has 1..51) ==== 6120+0+0 (secure 0 0 0) 0x7fefac014880 con 0x555803987270 2024-09-21T07:34:55.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.284+0000 7fefc2419740 1 -- 172.21.15.39:0/1232668538 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefac018070 con 0x555803987270 2024-09-21T07:34:55.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.284+0000 7fef99600640 1 -- 172.21.15.39:0/1232668538 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fefac001890 con 0x555803987270 2024-09-21T07:34:55.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.284+0000 7fefc2419740 10 client.5466 fetch_fsmap learned FSMap version 26 2024-09-21T07:34:55.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.284+0000 7fefc2419740 1 -- 172.21.15.39:0/1232668538 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x555803d96560 con 0x555803987270 2024-09-21T07:34:55.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.285+0000 7fef99600640 1 -- 172.21.15.39:0/1232668538 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fefac018070 con 0x555803987270 2024-09-21T07:34:55.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.285+0000 7fef99600640 1 -- 172.21.15.39:0/1232668538 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 26) ==== 3502+0+0 (secure 0 0 0) 0x7fefac014b70 con 0x555803987270 2024-09-21T07:34:55.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.285+0000 7fefc2419740 10 client.5466 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T07:34:55.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.285+0000 7fefc2419740 10 client.5466 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:34:55.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.285+0000 7fefc2419740 20 client.5466 populate_metadata read hostname 'smithi039' 2024-09-21T07:34:55.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.285+0000 7fefc2419740 1 --2- 172.21.15.39:0/1232668538 >> [v2:172.21.15.39:6833/1572835489,v1:172.21.15.39:6835/1572835489] conn(0x555803d2d8b0 0x555803d2dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:55.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.285+0000 7fefc2419740 4 client.5466 mds_command: new command op to 5310 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:34:55.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.285+0000 7fefc2419740 1 -- 172.21.15.39:0/1232668538 --> [v2:172.21.15.39:6833/1572835489,v1:172.21.15.39:6835/1572835489] -- command(tid 0: {"prefix": "status"}) -- 0x555803d2cfb0 con 0x555803d2d8b0 2024-09-21T07:34:55.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.285+0000 7fefb8a00640 1 --2- 172.21.15.39:0/1232668538 >> [v2:172.21.15.39:6833/1572835489,v1:172.21.15.39:6835/1572835489] conn(0x555803d2d8b0 0x555803d2dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:55.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.286+0000 7fefb8a00640 1 --2- 172.21.15.39:0/1232668538 >> [v2:172.21.15.39:6833/1572835489,v1:172.21.15.39:6835/1572835489] conn(0x555803d2d8b0 0x555803d2dcc0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1ae15d0917f56ed server_cookie=38cd18f95f14746 in_seq=0 out_seq=0 2024-09-21T07:34:55.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.286+0000 7fef99600640 10 client.5466 ms_handle_connect on v2:172.21.15.39:6833/1572835489 2024-09-21T07:34:55.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.286+0000 7fef99600640 1 -- 172.21.15.39:0/1232668538 <== mds.0 v2:172.21.15.39:6833/1572835489 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x555803d2cfb0 con 0x555803d2d8b0 2024-09-21T07:34:55.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.286+0000 7fef99600640 10 client.5466 handle_command_reply: tid=0 2024-09-21T07:34:55.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.286+0000 7fefc2419740 1 client.5466 shutdown 2024-09-21T07:34:55.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.286+0000 7fefc2419740 1 -- 172.21.15.39:0/1232668538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555803987270 msgr2=0x555803df1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:55.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.286+0000 7fefc2419740 1 --2- 172.21.15.39:0/1232668538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555803987270 0x555803df1550 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fefac002080 tx=0x7fefac011b90 comp rx=0 tx=0).stop 2024-09-21T07:34:55.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.286+0000 7fefc2419740 1 -- 172.21.15.39:0/1232668538 shutdown_connections 2024-09-21T07:34:55.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.286+0000 7fefc2419740 1 --2- 172.21.15.39:0/1232668538 >> [v2:172.21.15.39:6833/1572835489,v1:172.21.15.39:6835/1572835489] conn(0x555803d2d8b0 0x555803d2dcc0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:55.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.286+0000 7fefc2419740 1 --2- 172.21.15.39:0/1232668538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555803987270 0x555803df1550 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:55.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.286+0000 7fefc2419740 1 --2- 172.21.15.39:0/1232668538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555803966590 0x555803d576d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:55.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.286+0000 7fefc2419740 1 --2- 172.21.15.39:0/1232668538 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55580395e3b0 0x555803df5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:55.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.286+0000 7fefc2419740 1 -- 172.21.15.39:0/1232668538 >> 172.21.15.39:0/1232668538 conn(0x55580397b3c0 msgr2=0x555803d28c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:55.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.286+0000 7fef99600640 0 client.5466 ms_handle_reset on v2:172.21.15.39:6833/1572835489 2024-09-21T07:34:55.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.286+0000 7fefc2419740 1 -- 172.21.15.39:0/1232668538 shutdown_connections 2024-09-21T07:34:55.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.286+0000 7fefc2419740 1 -- 172.21.15.39:0/1232668538 wait complete. 2024-09-21T07:34:55.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.286+0000 7fefc2419740 20 client.5466 trim_cache size 0 max 16384 2024-09-21T07:34:55.292 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:34:55.292 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:34:55.292 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-09-21T07:34:55.292 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 5310, 2024-09-21T07:34:55.292 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:34:55.292 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:34:55.292 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:34:55.293 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 11.848088854, 2024-09-21T07:34:55.293 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 26, 2024-09-21T07:34:55.293 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 51, 2024-09-21T07:34:55.293 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 49, 2024-09-21T07:34:55.293 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 16.820855094999999 2024-09-21T07:34:55.293 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:34:55.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.287+0000 7fefbb400640 1 -- 172.21.15.39:0/3108694838 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fef94078b90 msgr2=0x7fef9407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:55.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.287+0000 7fefbb400640 1 --2- 172.21.15.39:0/3108694838 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fef94078b90 0x7fef9407b0a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fefa8035000 tx=0x7fefa80046c0 comp rx=0 tx=0).stop 2024-09-21T07:34:55.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.287+0000 7fefbb400640 1 -- 172.21.15.39:0/3108694838 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefb4153430 msgr2=0x7fefb40caff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:55.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.287+0000 7fefbb400640 1 --2- 172.21.15.39:0/3108694838 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefb4153430 0x7fefb40caff0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fefa0004ed0 tx=0x7fefa0001630 comp rx=0 tx=0).stop 2024-09-21T07:34:55.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.287+0000 7fefbb400640 1 -- 172.21.15.39:0/3108694838 shutdown_connections 2024-09-21T07:34:55.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.287+0000 7fefbb400640 1 --2- 172.21.15.39:0/3108694838 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fef94078b90 0x7fef9407b0a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:55.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.287+0000 7fefbb400640 1 --2- 172.21.15.39:0/3108694838 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefb40cb530 0x7fefb41f8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:55.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.287+0000 7fefbb400640 1 --2- 172.21.15.39:0/3108694838 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefb4153430 0x7fefb40caff0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:55.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.287+0000 7fefbb400640 1 --2- 172.21.15.39:0/3108694838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef9c003a50 0x7fefb409c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:55.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.287+0000 7fefbb400640 1 -- 172.21.15.39:0/3108694838 >> 172.21.15.39:0/3108694838 conn(0x7fefb40b1cb0 msgr2=0x7fefb40aede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:55.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.287+0000 7fefbb400640 1 -- 172.21.15.39:0/3108694838 shutdown_connections 2024-09-21T07:34:55.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.287+0000 7fefbb400640 1 -- 172.21.15.39:0/3108694838 wait complete. 2024-09-21T07:34:55.313 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:34:55.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.540+0000 7f29cf200640 1 -- 172.21.15.39:0/3677939208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29b4004800 msgr2=0x7f29b4004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:55.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.540+0000 7f29cf200640 1 --2- 172.21.15.39:0/3677939208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29b4004800 0x7f29b4004c10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f29bc00c310 tx=0x7f29b4005830 comp rx=0 tx=0).stop 2024-09-21T07:34:55.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.540+0000 7f29cf200640 1 -- 172.21.15.39:0/3677939208 shutdown_connections 2024-09-21T07:34:55.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.540+0000 7f29cf200640 1 --2- 172.21.15.39:0/3677939208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29b4004800 0x7f29b4004c10 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:55.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.540+0000 7f29cf200640 1 -- 172.21.15.39:0/3677939208 >> 172.21.15.39:0/3677939208 conn(0x7f29d009d6a0 msgr2=0x7f29d00d4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:55.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.540+0000 7f29cf200640 1 -- 172.21.15.39:0/3677939208 shutdown_connections 2024-09-21T07:34:55.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.540+0000 7f29cf200640 1 -- 172.21.15.39:0/3677939208 wait complete. 2024-09-21T07:34:55.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.542+0000 7f29cf200640 1 Processor -- start 2024-09-21T07:34:55.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.542+0000 7f29cf200640 1 -- start start 2024-09-21T07:34:55.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.544+0000 7f29cf200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29b4004800 0x7f29d01f5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:55.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.544+0000 7f29cf200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29d01597e0 0x7f29d015d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:55.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.544+0000 7f29cf200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f29d0159bf0 0x7f29d015db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:55.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.544+0000 7f29cf200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f29d00d2290 con 0x7f29d01597e0 2024-09-21T07:34:55.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.544+0000 7f29cf200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f29d00d2110 con 0x7f29b4004800 2024-09-21T07:34:55.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.544+0000 7f29cf200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f29d00d2410 con 0x7f29d0159bf0 2024-09-21T07:34:55.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.544+0000 7f29cd400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29d01597e0 0x7f29d015d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:55.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.544+0000 7f29cd400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29d01597e0 0x7f29d015d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57598/0 (socket says 172.21.15.39:57598) 2024-09-21T07:34:55.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.544+0000 7f29cd400640 1 -- 172.21.15.39:0/4215353186 learned_addr learned my addr 172.21.15.39:0/4215353186 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:55.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.545+0000 7f29ce800640 1 --2- 172.21.15.39:0/4215353186 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f29d0159bf0 0x7f29d015db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:55.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.545+0000 7f29cde00640 1 --2- 172.21.15.39:0/4215353186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29b4004800 0x7f29d01f5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:55.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.545+0000 7f29cd400640 1 -- 172.21.15.39:0/4215353186 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f29d0159bf0 msgr2=0x7f29d015db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:55.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.545+0000 7f29cd400640 1 --2- 172.21.15.39:0/4215353186 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f29d0159bf0 0x7f29d015db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:55.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.545+0000 7f29cd400640 1 -- 172.21.15.39:0/4215353186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29b4004800 msgr2=0x7f29d01f5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:55.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.545+0000 7f29cd400640 1 --2- 172.21.15.39:0/4215353186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29b4004800 0x7f29d01f5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:55.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.545+0000 7f29cd400640 1 -- 172.21.15.39:0/4215353186 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d015fe60 con 0x7f29d01597e0 2024-09-21T07:34:55.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.545+0000 7f29ce800640 1 --2- 172.21.15.39:0/4215353186 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f29d0159bf0 0x7f29d015db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:34:55.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.545+0000 7f29cde00640 1 --2- 172.21.15.39:0/4215353186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29b4004800 0x7f29d01f5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:34:55.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.545+0000 7f29cd400640 1 --2- 172.21.15.39:0/4215353186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29d01597e0 0x7f29d015d5e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f29bc034040 tx=0x7f29bc006380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:55.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.545+0000 7f29c6000640 1 -- 172.21.15.39:0/4215353186 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29bc031050 con 0x7f29d01597e0 2024-09-21T07:34:55.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.545+0000 7f29c6000640 1 -- 172.21.15.39:0/4215353186 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f29bc0038e0 con 0x7f29d01597e0 2024-09-21T07:34:55.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.545+0000 7f29cf200640 1 -- 172.21.15.39:0/4215353186 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29d0160140 con 0x7f29d01597e0 2024-09-21T07:34:55.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.545+0000 7f29c6000640 1 -- 172.21.15.39:0/4215353186 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29bc003cf0 con 0x7f29d01597e0 2024-09-21T07:34:55.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.545+0000 7f29cf200640 1 -- 172.21.15.39:0/4215353186 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29d0160700 con 0x7f29d01597e0 2024-09-21T07:34:55.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.547+0000 7f29cf200640 1 -- 172.21.15.39:0/4215353186 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29d015fbd0 con 0x7f29d01597e0 2024-09-21T07:34:55.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.548+0000 7f29c6000640 1 -- 172.21.15.39:0/4215353186 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f29bc02d460 con 0x7f29d01597e0 2024-09-21T07:34:55.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.549+0000 7f29c6000640 1 --2- 172.21.15.39:0/4215353186 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f29b0078bc0 0x7f29b007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:55.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.550+0000 7f29c6000640 1 -- 172.21.15.39:0/4215353186 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6120+0+0 (secure 0 0 0) 0x7f29bc0bb690 con 0x7f29d01597e0 2024-09-21T07:34:55.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.550+0000 7f29cde00640 1 --2- 172.21.15.39:0/4215353186 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f29b0078bc0 0x7f29b007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:55.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.551+0000 7f29cde00640 1 --2- 172.21.15.39:0/4215353186 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f29b0078bc0 0x7f29b007b0d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f29d00521d0 tx=0x7f29c0005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:55.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.554+0000 7f29c6000640 1 -- 172.21.15.39:0/4215353186 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f29bc046060 con 0x7f29d01597e0 2024-09-21T07:34:55.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.792+0000 7f29cf200640 1 -- 172.21.15.39:0/4215353186 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f29d00ca5d0 con 0x7f29d01597e0 2024-09-21T07:34:55.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.793+0000 7f29c6000640 1 -- 172.21.15.39:0/4215353186 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7f29bc0839c0 con 0x7f29d01597e0 2024-09-21T07:34:55.795 INFO:teuthology.orchestra.run.smithi039.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:34:55.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.795+0000 7f29cf200640 1 -- 172.21.15.39:0/4215353186 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f29b0078bc0 msgr2=0x7f29b007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:55.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.795+0000 7f29cf200640 1 --2- 172.21.15.39:0/4215353186 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f29b0078bc0 0x7f29b007b0d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f29d00521d0 tx=0x7f29c0005f90 comp rx=0 tx=0).stop 2024-09-21T07:34:55.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.795+0000 7f29cf200640 1 -- 172.21.15.39:0/4215353186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29d01597e0 msgr2=0x7f29d015d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:55.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.795+0000 7f29cf200640 1 --2- 172.21.15.39:0/4215353186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29d01597e0 0x7f29d015d5e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f29bc034040 tx=0x7f29bc006380 comp rx=0 tx=0).stop 2024-09-21T07:34:55.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.795+0000 7f29cf200640 1 -- 172.21.15.39:0/4215353186 shutdown_connections 2024-09-21T07:34:55.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.795+0000 7f29cf200640 1 --2- 172.21.15.39:0/4215353186 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f29b0078bc0 0x7f29b007b0d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:55.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.795+0000 7f29cf200640 1 --2- 172.21.15.39:0/4215353186 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f29d0159bf0 0x7f29d015db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:55.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.795+0000 7f29cf200640 1 --2- 172.21.15.39:0/4215353186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29d01597e0 0x7f29d015d5e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:55.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.795+0000 7f29cf200640 1 --2- 172.21.15.39:0/4215353186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29b4004800 0x7f29d01f5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:55.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.795+0000 7f29cf200640 1 -- 172.21.15.39:0/4215353186 >> 172.21.15.39:0/4215353186 conn(0x7f29d009d6a0 msgr2=0x7f29d006df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:55.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.795+0000 7f29cf200640 1 -- 172.21.15.39:0/4215353186 shutdown_connections 2024-09-21T07:34:55.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:55.795+0000 7f29cf200640 1 -- 172.21.15.39:0/4215353186 wait complete. 2024-09-21T07:34:55.820 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:34:55.821 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:34:55.821 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi039.front.sepia.ceph.com 2024-09-21T07:34:55.821 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T07:34:55.821 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:34:55.821 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:34:55.821 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:34:55.821 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:34:55.821 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:34:55.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:34:55.822 DEBUG:teuthology.orchestra.run.smithi039:> ip addr 2024-09-21T07:34:55.858 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:34:55.858 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:34:55.858 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:34:55.858 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:34:55.858 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2024-09-21T07:34:55.858 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:34:55.858 INFO:teuthology.orchestra.run.smithi039.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:34:55.858 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:34:55.858 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f0 2024-09-21T07:34:55.858 INFO:teuthology.orchestra.run.smithi039.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:34:55.859 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c7 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:34:55.859 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f1 2024-09-21T07:34:55.859 INFO:teuthology.orchestra.run.smithi039.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:34:55.859 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:70 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:34:55.859 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f0 2024-09-21T07:34:55.859 INFO:teuthology.orchestra.run.smithi039.stdout: inet 172.21.15.39/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T07:34:55.859 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft 42623sec preferred_lft 42623sec 2024-09-21T07:34:55.859 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec4:7aff:febd:1370/64 scope link 2024-09-21T07:34:55.859 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:34:55.860 INFO:teuthology.orchestra.run.smithi039.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:34:55.860 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:71 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:34:55.860 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f1 2024-09-21T07:34:55.860 INFO:teuthology.orchestra.run.smithi039.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:34:55.860 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff 2024-09-21T07:34:55.860 INFO:teuthology.orchestra.run.smithi039.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:34:55.860 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:34:55.860 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::b079:ddff:fe0e:6333/64 scope link 2024-09-21T07:34:55.860 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:34:55.860 INFO:teuthology.orchestra.run.smithi039.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:34:55.861 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:34:55.861 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::2ca7:98ff:fe8a:d0ed/64 scope link 2024-09-21T07:34:55.861 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:34:55.861 INFO:teuthology.orchestra.run.smithi039.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:34:55.861 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether ee:10:53:01:0c:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:34:55.861 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec10:53ff:fe01:c44/64 scope link 2024-09-21T07:34:55.861 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:34:55.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:34:55.862 DEBUG:teuthology.orchestra.run.smithi039:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:34:55.952 INFO:teuthology.orchestra.run.smithi039.stdout:1 2024-09-21T07:34:55.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:34:55.955 DEBUG:teuthology.orchestra.run.smithi039:> route 2024-09-21T07:34:55.991 INFO:teuthology.orchestra.run.smithi039.stdout:Kernel IP routing table 2024-09-21T07:34:55.991 INFO:teuthology.orchestra.run.smithi039.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:34:55.992 INFO:teuthology.orchestra.run.smithi039.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T07:34:55.992 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T07:34:55.992 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:34:55.992 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:34:55.992 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-21T07:34:55.992 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T07:34:55.992 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T07:34:55.992 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T07:34:55.992 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-21T07:34:56.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:34:56.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:34:56.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:34:56.188 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-09-21T07:34:56.224 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:34:56.224 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T07:34:56.224 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:34:56.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:34:56.224 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-09-21T07:34:56.319 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:34:56.319 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T07:34:56.354 INFO:teuthology.orchestra.run.smithi039.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:34:56.355 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:34:56.355 DEBUG:teuthology.orchestra.run.smithi039:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:34:56.430 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-21T07:34:56.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:34:56.430 DEBUG:teuthology.orchestra.run.smithi039:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:34:56.629 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:34:56.629 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:34:56.667 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:34:56.667 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:34:56.667 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: resolved to: "172.21.15.39:3300,172.21.15.148:3300,172.21.15.148:3301" 2024-09-21T07:34:56.667 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: trying mount with new device syntax: 0@7ab9b04a-f88d-49aa-a493-e5c076637ffb.cephfs=/ 2024-09-21T07:34:56.667 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.39:3300/172.21.15.148:3300/172.21.15.148:3301" will pass to kernel 2024-09-21T07:34:56.668 INFO:teuthology.orchestra.run.smithi039.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T07:34:56.669 INFO:teuthology.orchestra.run.smithi039.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:34:56.669 INFO:teuthology.orchestra.run.smithi039.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:34:56.669 INFO:teuthology.orchestra.run.smithi039.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:34:56.669 INFO:teuthology.orchestra.run.smithi039.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:34:56.671 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:34:56.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:34:56.672 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:34:56.728 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:34:56.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:34:56.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:34:56.825 INFO:teuthology.orchestra.run.smithi039.stdout:client5475 2024-09-21T07:34:56.825 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:34:56.825 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:34:56.861 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data mksnap snap1 2024-09-21T07:34:57.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.043+0000 7f2e9ba1ff40 1 Processor -- start 2024-09-21T07:34:57.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.043+0000 7f2e9ba1ff40 1 -- start start 2024-09-21T07:34:57.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.044+0000 7f2e9ba1ff40 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e7c003630 0x558a17464f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:57.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.044+0000 7f2e9ba1ff40 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558a1747efe0 0x558a1747f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:57.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.044+0000 7f2e9ba1ff40 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a1745d420 0x558a17482830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:57.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.044+0000 7f2e9ba1ff40 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558a172fac60 con 0x558a1745d420 2024-09-21T07:34:57.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.044+0000 7f2e9ba1ff40 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558a173591d0 con 0x7f2e7c003630 2024-09-21T07:34:57.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.044+0000 7f2e9ba1ff40 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x558a17481060 con 0x558a1747efe0 2024-09-21T07:34:57.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.044+0000 7f2e95600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a1745d420 0x558a17482830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:57.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.044+0000 7f2e95600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a1745d420 0x558a17482830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57616/0 (socket says 172.21.15.39:57616) 2024-09-21T07:34:57.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.044+0000 7f2e95600640 1 -- 172.21.15.39:0/1430618912 learned_addr learned my addr 172.21.15.39:0/1430618912 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:57.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.045+0000 7f2e94c00640 1 --2- 172.21.15.39:0/1430618912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e7c003630 0x558a17464f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:57.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.045+0000 7f2e8fe00640 1 --2- 172.21.15.39:0/1430618912 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558a1747efe0 0x558a1747f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:57.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.045+0000 7f2e95600640 1 -- 172.21.15.39:0/1430618912 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558a1747efe0 msgr2=0x558a1747f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:34:57.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.045+0000 7f2e95600640 1 --2- 172.21.15.39:0/1430618912 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558a1747efe0 0x558a1747f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:57.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.045+0000 7f2e95600640 1 -- 172.21.15.39:0/1430618912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e7c003630 msgr2=0x558a17464f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:34:57.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.045+0000 7f2e95600640 1 --2- 172.21.15.39:0/1430618912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e7c003630 0x558a17464f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:57.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.045+0000 7f2e95600640 1 -- 172.21.15.39:0/1430618912 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a173f5a40 con 0x558a1745d420 2024-09-21T07:34:57.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.045+0000 7f2e94c00640 1 --2- 172.21.15.39:0/1430618912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e7c003630 0x558a17464f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:34:57.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.045+0000 7f2e95600640 1 --2- 172.21.15.39:0/1430618912 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a1745d420 0x558a17482830 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2e90057040 tx=0x7f2e900033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e23038720eec2d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:57.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.045+0000 7f2e8ea00640 1 -- 172.21.15.39:0/1430618912 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e90064030 con 0x558a1745d420 2024-09-21T07:34:57.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.045+0000 7f2e8ea00640 1 -- 172.21.15.39:0/1430618912 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e900557e0 con 0x558a1745d420 2024-09-21T07:34:57.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.045+0000 7f2e8ea00640 1 -- 172.21.15.39:0/1430618912 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e90055c10 con 0x558a1745d420 2024-09-21T07:34:57.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.046+0000 7f2e9ba1ff40 1 -- 172.21.15.39:0/1430618912 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a1745d420 msgr2=0x558a17482830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:57.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.046+0000 7f2e9ba1ff40 1 --2- 172.21.15.39:0/1430618912 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a1745d420 0x558a17482830 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2e90057040 tx=0x7f2e900033a0 comp rx=0 tx=0).stop 2024-09-21T07:34:57.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.046+0000 7f2e9ba1ff40 1 -- 172.21.15.39:0/1430618912 shutdown_connections 2024-09-21T07:34:57.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.046+0000 7f2e9ba1ff40 1 --2- 172.21.15.39:0/1430618912 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a1745d420 0x558a17482830 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:57.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.046+0000 7f2e9ba1ff40 1 --2- 172.21.15.39:0/1430618912 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558a1747efe0 0x558a1747f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:57.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.046+0000 7f2e9ba1ff40 1 --2- 172.21.15.39:0/1430618912 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2e7c003630 0x558a17464f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:57.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.046+0000 7f2e9ba1ff40 1 -- 172.21.15.39:0/1430618912 >> 172.21.15.39:0/1430618912 conn(0x558a172cbac0 msgr2=0x558a17479a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:57.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.046+0000 7f2e9ba1ff40 1 -- 172.21.15.39:0/1430618912 shutdown_connections 2024-09-21T07:34:57.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.046+0000 7f2e9ba1ff40 1 -- 172.21.15.39:0/1430618912 wait complete. 2024-09-21T07:34:57.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.047+0000 7f2e9ba1ff40 1 Processor -- start 2024-09-21T07:34:57.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.047+0000 7f2e9ba1ff40 1 -- start start 2024-09-21T07:34:57.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.049+0000 7f2e9ba1ff40 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a1745d420 0x558a1745bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:57.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.049+0000 7f2e9ba1ff40 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e7c003630 0x558a17482830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:57.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.049+0000 7f2e9ba1ff40 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558a17462ca0 0x558a17482d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:57.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.049+0000 7f2e9ba1ff40 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558a173f4f70 con 0x7f2e7c003630 2024-09-21T07:34:57.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.049+0000 7f2e9ba1ff40 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558a1745e1a0 con 0x558a1745d420 2024-09-21T07:34:57.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.049+0000 7f2e9ba1ff40 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x558a173f4680 con 0x558a17462ca0 2024-09-21T07:34:57.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.049+0000 7f2e8fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e7c003630 0x558a17482830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:57.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.049+0000 7f2e8fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e7c003630 0x558a17482830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57622/0 (socket says 172.21.15.39:57622) 2024-09-21T07:34:57.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.049+0000 7f2e8fe00640 1 -- 172.21.15.39:0/872814567 learned_addr learned my addr 172.21.15.39:0/872814567 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:57.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.049+0000 7f2e94c00640 1 --2- 172.21.15.39:0/872814567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a1745d420 0x558a1745bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:57.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.049+0000 7f2e8fe00640 1 -- 172.21.15.39:0/872814567 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558a17462ca0 msgr2=0x558a17482d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:57.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.049+0000 7f2e95600640 1 --2- 172.21.15.39:0/872814567 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558a17462ca0 0x558a17482d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:57.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.050+0000 7f2e8fe00640 1 --2- 172.21.15.39:0/872814567 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558a17462ca0 0x558a17482d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:57.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.050+0000 7f2e8fe00640 1 -- 172.21.15.39:0/872814567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a1745d420 msgr2=0x558a1745bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:57.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.050+0000 7f2e8fe00640 1 --2- 172.21.15.39:0/872814567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a1745d420 0x558a1745bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:57.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.050+0000 7f2e8fe00640 1 -- 172.21.15.39:0/872814567 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a1745c2b0 con 0x7f2e7c003630 2024-09-21T07:34:57.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.050+0000 7f2e94c00640 1 --2- 172.21.15.39:0/872814567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a1745d420 0x558a1745bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:34:57.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.050+0000 7f2e95600640 1 --2- 172.21.15.39:0/872814567 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558a17462ca0 0x558a17482d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:34:57.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.050+0000 7f2e8fe00640 1 --2- 172.21.15.39:0/872814567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e7c003630 0x558a17482830 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2e8000a6e0 tx=0x7f2e8000aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:57.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.050+0000 7f2e8d600640 1 -- 172.21.15.39:0/872814567 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e80009ba0 con 0x7f2e7c003630 2024-09-21T07:34:57.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.050+0000 7f2e8d600640 1 -- 172.21.15.39:0/872814567 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e8001b070 con 0x7f2e7c003630 2024-09-21T07:34:57.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.050+0000 7f2e9ba1ff40 1 -- 172.21.15.39:0/872814567 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558a175194f0 con 0x7f2e7c003630 2024-09-21T07:34:57.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.050+0000 7f2e8d600640 1 -- 172.21.15.39:0/872814567 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e80016470 con 0x7f2e7c003630 2024-09-21T07:34:57.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.050+0000 7f2e9ba1ff40 1 -- 172.21.15.39:0/872814567 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a17519ab0 con 0x7f2e7c003630 2024-09-21T07:34:57.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.052+0000 7f2e8d600640 1 -- 172.21.15.39:0/872814567 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2e80016610 con 0x7f2e7c003630 2024-09-21T07:34:57.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.053+0000 7f2e8d600640 1 --2- 172.21.15.39:0/872814567 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2e74077570 0x7f2e74079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:57.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.053+0000 7f2e8d600640 1 -- 172.21.15.39:0/872814567 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6120+0+0 (secure 0 0 0) 0x7f2e8009b120 con 0x7f2e7c003630 2024-09-21T07:34:57.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.054+0000 7f2e9ba1ff40 1 -- 172.21.15.39:0/872814567 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- pool_op(create snap pool 8 tid 1 name snap1 v0) -- 0x558a1735bb20 con 0x7f2e7c003630 2024-09-21T07:34:57.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.054+0000 7f2e8d600640 1 -- 172.21.15.39:0/872814567 <== mon.0 v2:172.21.15.39:3300/0 6 ==== pool_op_reply(tid 1 (95) Operation not supported v51) ==== 43+0+0 (secure 0 0 0) 0x7f2e80012040 con 0x7f2e7c003630 2024-09-21T07:34:57.057 INFO:teuthology.orchestra.run.smithi039.stderr:error creating pool 2024-09-21T07:34:57.054+0000 7f2e94c00640 1 --2- 172.21.15.39:0/872814567 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2e74077570 0x7f2e74079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:57.057 INFO:teuthology.orchestra.run.smithi039.stderr:cephfs_data snapshot snap1: (95) Operation not supported 2024-09-21T07:34:57.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.054+0000 7f2e9ba1ff40 1 -- 172.21.15.39:0/872814567 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2e74077570 msgr2=0x7f2e74079a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:57.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.054+0000 7f2e9ba1ff40 1 --2- 172.21.15.39:0/872814567 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2e74077570 0x7f2e74079a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:57.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.054+0000 7f2e9ba1ff40 1 -- 172.21.15.39:0/872814567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e7c003630 msgr2=0x558a17482830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:57.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.054+0000 7f2e9ba1ff40 1 --2- 172.21.15.39:0/872814567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e7c003630 0x558a17482830 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2e8000a6e0 tx=0x7f2e8000aba0 comp rx=0 tx=0).stop 2024-09-21T07:34:57.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.054+0000 7f2e9ba1ff40 1 -- 172.21.15.39:0/872814567 shutdown_connections 2024-09-21T07:34:57.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.054+0000 7f2e9ba1ff40 1 --2- 172.21.15.39:0/872814567 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2e74077570 0x7f2e74079a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:57.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.054+0000 7f2e9ba1ff40 1 --2- 172.21.15.39:0/872814567 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558a17462ca0 0x558a17482d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:57.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.054+0000 7f2e9ba1ff40 1 --2- 172.21.15.39:0/872814567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e7c003630 0x558a17482830 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:57.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.054+0000 7f2e9ba1ff40 1 --2- 172.21.15.39:0/872814567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a1745d420 0x558a1745bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:57.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.054+0000 7f2e9ba1ff40 1 -- 172.21.15.39:0/872814567 >> 172.21.15.39:0/872814567 conn(0x558a172cbac0 msgr2=0x558a173f2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:57.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.054+0000 7f2e9ba1ff40 1 -- 172.21.15.39:0/872814567 shutdown_connections 2024-09-21T07:34:57.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.054+0000 7f2e9ba1ff40 1 -- 172.21.15.39:0/872814567 wait complete. 2024-09-21T07:34:57.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:34:57.070 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata mksnap snap2 2024-09-21T07:34:57.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.211+0000 7f362f881f40 1 Processor -- start 2024-09-21T07:34:57.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.211+0000 7f362f881f40 1 -- start start 2024-09-21T07:34:57.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.212+0000 7f362f881f40 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e5e1af4a0 0x556e5e275ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:57.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.212+0000 7f362f881f40 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e5e279e30 0x556e5e1d96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:57.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.212+0000 7f362f881f40 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x556e5e286e40 0x556e5e1d9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:57.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.212+0000 7f362f881f40 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556e5e0fec60 con 0x556e5e279e30 2024-09-21T07:34:57.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.212+0000 7f362f881f40 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556e5e15d1d0 con 0x556e5e1af4a0 2024-09-21T07:34:57.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.212+0000 7f362f881f40 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x556e5e1afd70 con 0x556e5e286e40 2024-09-21T07:34:57.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.212+0000 7f3623e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e5e279e30 0x556e5e1d96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:57.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.212+0000 7f3623e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e5e279e30 0x556e5e1d96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57632/0 (socket says 172.21.15.39:57632) 2024-09-21T07:34:57.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.212+0000 7f3623e00640 1 -- 172.21.15.39:0/2513348155 learned_addr learned my addr 172.21.15.39:0/2513348155 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:57.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.212+0000 7f3629400640 1 --2- 172.21.15.39:0/2513348155 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x556e5e286e40 0x556e5e1d9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:57.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.212+0000 7f3628a00640 1 --2- 172.21.15.39:0/2513348155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e5e1af4a0 0x556e5e275ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:57.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.213+0000 7f3623e00640 1 -- 172.21.15.39:0/2513348155 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x556e5e286e40 msgr2=0x556e5e1d9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:34:57.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.213+0000 7f3623e00640 1 --2- 172.21.15.39:0/2513348155 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x556e5e286e40 0x556e5e1d9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:57.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.213+0000 7f3623e00640 1 -- 172.21.15.39:0/2513348155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e5e1af4a0 msgr2=0x556e5e275ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:34:57.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.213+0000 7f3623e00640 1 --2- 172.21.15.39:0/2513348155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e5e1af4a0 0x556e5e275ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:57.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.213+0000 7f3623e00640 1 -- 172.21.15.39:0/2513348155 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e5e279150 con 0x556e5e279e30 2024-09-21T07:34:57.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.213+0000 7f3629400640 1 --2- 172.21.15.39:0/2513348155 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x556e5e286e40 0x556e5e1d9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:34:57.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.213+0000 7f3628a00640 1 --2- 172.21.15.39:0/2513348155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e5e1af4a0 0x556e5e275ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:34:57.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.213+0000 7f3623e00640 1 --2- 172.21.15.39:0/2513348155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e5e279e30 0x556e5e1d96d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3614025e60 tx=0x7f3614002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed773d2fa8cac3f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:57.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.213+0000 7f3622a00640 1 -- 172.21.15.39:0/2513348155 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361404d8c0 con 0x556e5e279e30 2024-09-21T07:34:57.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.213+0000 7f3622a00640 1 -- 172.21.15.39:0/2513348155 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f361404dd70 con 0x556e5e279e30 2024-09-21T07:34:57.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.213+0000 7f3622a00640 1 -- 172.21.15.39:0/2513348155 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36140566d0 con 0x556e5e279e30 2024-09-21T07:34:57.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.214+0000 7f362f881f40 1 -- 172.21.15.39:0/2513348155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e5e279e30 msgr2=0x556e5e1d96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:57.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.214+0000 7f362f881f40 1 --2- 172.21.15.39:0/2513348155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e5e279e30 0x556e5e1d96d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3614025e60 tx=0x7f3614002fb0 comp rx=0 tx=0).stop 2024-09-21T07:34:57.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.214+0000 7f362f881f40 1 -- 172.21.15.39:0/2513348155 shutdown_connections 2024-09-21T07:34:57.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.214+0000 7f362f881f40 1 --2- 172.21.15.39:0/2513348155 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x556e5e286e40 0x556e5e1d9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:57.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.214+0000 7f362f881f40 1 --2- 172.21.15.39:0/2513348155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e5e279e30 0x556e5e1d96d0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:57.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.214+0000 7f362f881f40 1 --2- 172.21.15.39:0/2513348155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e5e1af4a0 0x556e5e275ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:57.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.214+0000 7f362f881f40 1 -- 172.21.15.39:0/2513348155 >> 172.21.15.39:0/2513348155 conn(0x556e5e0cfac0 msgr2=0x556e5e1db480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:57.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.214+0000 7f362f881f40 1 -- 172.21.15.39:0/2513348155 shutdown_connections 2024-09-21T07:34:57.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.214+0000 7f362f881f40 1 -- 172.21.15.39:0/2513348155 wait complete. 2024-09-21T07:34:57.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.215+0000 7f362f881f40 1 Processor -- start 2024-09-21T07:34:57.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.215+0000 7f362f881f40 1 -- start start 2024-09-21T07:34:57.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.217+0000 7f362f881f40 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e5e1af4a0 0x556e5e264120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:57.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.217+0000 7f362f881f40 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e5e279e30 0x556e5e1add00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:57.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.217+0000 7f362f881f40 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x556e5e286e40 0x556e5e268ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:57.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.217+0000 7f362f881f40 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556e5e27d020 con 0x556e5e279e30 2024-09-21T07:34:57.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.217+0000 7f362f881f40 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556e5e1da6c0 con 0x556e5e1af4a0 2024-09-21T07:34:57.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.217+0000 7f362f881f40 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x556e5e1dd1f0 con 0x556e5e286e40 2024-09-21T07:34:57.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.217+0000 7f3623e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e5e279e30 0x556e5e1add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:57.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.217+0000 7f3623e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e5e279e30 0x556e5e1add00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57648/0 (socket says 172.21.15.39:57648) 2024-09-21T07:34:57.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.217+0000 7f3623e00640 1 -- 172.21.15.39:0/276570155 learned_addr learned my addr 172.21.15.39:0/276570155 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:57.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.217+0000 7f3628a00640 1 --2- 172.21.15.39:0/276570155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e5e1af4a0 0x556e5e264120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:57.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.217+0000 7f3629400640 1 --2- 172.21.15.39:0/276570155 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x556e5e286e40 0x556e5e268ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:57.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.217+0000 7f3623e00640 1 -- 172.21.15.39:0/276570155 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x556e5e286e40 msgr2=0x556e5e268ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:57.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.217+0000 7f3623e00640 1 --2- 172.21.15.39:0/276570155 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x556e5e286e40 0x556e5e268ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:57.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.217+0000 7f3623e00640 1 -- 172.21.15.39:0/276570155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e5e1af4a0 msgr2=0x556e5e264120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:57.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.217+0000 7f3623e00640 1 --2- 172.21.15.39:0/276570155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e5e1af4a0 0x556e5e264120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:57.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.217+0000 7f3623e00640 1 -- 172.21.15.39:0/276570155 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e5e2689e0 con 0x556e5e279e30 2024-09-21T07:34:57.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.218+0000 7f3623e00640 1 --2- 172.21.15.39:0/276570155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e5e279e30 0x556e5e1add00 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3614046560 tx=0x7f3614057660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:57.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.218+0000 7f3621600640 1 -- 172.21.15.39:0/276570155 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3614057910 con 0x556e5e279e30 2024-09-21T07:34:57.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.218+0000 7f3621600640 1 -- 172.21.15.39:0/276570155 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3614070070 con 0x556e5e279e30 2024-09-21T07:34:57.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.218+0000 7f3621600640 1 -- 172.21.15.39:0/276570155 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3614057e60 con 0x556e5e279e30 2024-09-21T07:34:57.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.218+0000 7f362f881f40 1 -- 172.21.15.39:0/276570155 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556e5e275840 con 0x556e5e279e30 2024-09-21T07:34:57.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.218+0000 7f362f881f40 1 -- 172.21.15.39:0/276570155 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e5e277bb0 con 0x556e5e279e30 2024-09-21T07:34:57.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.220+0000 7f3621600640 1 -- 172.21.15.39:0/276570155 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f361406a070 con 0x556e5e279e30 2024-09-21T07:34:57.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.221+0000 7f3621600640 1 --2- 172.21.15.39:0/276570155 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3608077570 0x7f3608079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:57.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.221+0000 7f3621600640 1 -- 172.21.15.39:0/276570155 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6120+0+0 (secure 0 0 0) 0x7f36140df330 con 0x556e5e279e30 2024-09-21T07:34:57.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.221+0000 7f362f881f40 1 -- 172.21.15.39:0/276570155 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- pool_op(create snap pool 7 tid 1 name snap2 v0) -- 0x556e5e15fb20 con 0x556e5e279e30 2024-09-21T07:34:57.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.222+0000 7f3628a00640 1 --2- 172.21.15.39:0/276570155 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3608077570 0x7f3608079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:57.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.222+0000 7f3621600640 1 -- 172.21.15.39:0/276570155 <== mon.0 v2:172.21.15.39:3300/0 6 ==== pool_op_reply(tid 1 (95) Operation not supported v51) ==== 43+0+0 (secure 0 0 0) 0x7f3614018f70 con 0x556e5e279e30 2024-09-21T07:34:57.225 INFO:teuthology.orchestra.run.smithi039.stderr:error creating pool cephfs_metadata snapshot snap2: (95) Operation not supported 2024-09-21T07:34:57.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.222+0000 7f362f881f40 1 -- 172.21.15.39:0/276570155 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3608077570 msgr2=0x7f3608079a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:57.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.222+0000 7f362f881f40 1 --2- 172.21.15.39:0/276570155 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3608077570 0x7f3608079a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:57.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.222+0000 7f362f881f40 1 -- 172.21.15.39:0/276570155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e5e279e30 msgr2=0x556e5e1add00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:57.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.222+0000 7f362f881f40 1 --2- 172.21.15.39:0/276570155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e5e279e30 0x556e5e1add00 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3614046560 tx=0x7f3614057660 comp rx=0 tx=0).stop 2024-09-21T07:34:57.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.222+0000 7f362f881f40 1 -- 172.21.15.39:0/276570155 shutdown_connections 2024-09-21T07:34:57.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.222+0000 7f362f881f40 1 --2- 172.21.15.39:0/276570155 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3608077570 0x7f3608079a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:57.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.222+0000 7f362f881f40 1 --2- 172.21.15.39:0/276570155 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x556e5e286e40 0x556e5e268ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:57.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.222+0000 7f362f881f40 1 --2- 172.21.15.39:0/276570155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e5e279e30 0x556e5e1add00 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:57.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.222+0000 7f362f881f40 1 --2- 172.21.15.39:0/276570155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e5e1af4a0 0x556e5e264120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:57.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.222+0000 7f362f881f40 1 -- 172.21.15.39:0/276570155 >> 172.21.15.39:0/276570155 conn(0x556e5e0cfac0 msgr2=0x556e5e27a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:57.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.222+0000 7f362f881f40 1 -- 172.21.15.39:0/276570155 shutdown_connections 2024-09-21T07:34:57.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.222+0000 7f362f881f40 1 -- 172.21.15.39:0/276570155 wait complete. 2024-09-21T07:34:57.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:34:57.239 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool mksnap cephfs_data snap3 2024-09-21T07:34:57.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.468+0000 7fcd56400640 1 -- 172.21.15.39:0/947854935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd500b6e20 msgr2=0x7fcd500b4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:57.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.468+0000 7fcd56400640 1 --2- 172.21.15.39:0/947854935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd500b6e20 0x7fcd500b4700 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c009830 tx=0x7fcd3c02a5c0 comp rx=0 tx=0).stop 2024-09-21T07:34:57.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.468+0000 7fcd56400640 1 -- 172.21.15.39:0/947854935 shutdown_connections 2024-09-21T07:34:57.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.468+0000 7fcd56400640 1 --2- 172.21.15.39:0/947854935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd500b6e20 0x7fcd500b4700 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:57.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.468+0000 7fcd56400640 1 -- 172.21.15.39:0/947854935 >> 172.21.15.39:0/947854935 conn(0x7fcd500b3be0 msgr2=0x7fcd500b3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:57.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.468+0000 7fcd56400640 1 -- 172.21.15.39:0/947854935 shutdown_connections 2024-09-21T07:34:57.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.468+0000 7fcd56400640 1 -- 172.21.15.39:0/947854935 wait complete. 2024-09-21T07:34:57.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.471+0000 7fcd56400640 1 Processor -- start 2024-09-21T07:34:57.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.471+0000 7fcd56400640 1 -- start start 2024-09-21T07:34:57.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.472+0000 7fcd56400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd5009aa30 0x7fcd50151520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:57.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.472+0000 7fcd56400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd500b6e20 0x7fcd501f5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:57.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.472+0000 7fcd56400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcd500cd860 0x7fcd501e5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:57.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.472+0000 7fcd56400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcd500d0b70 con 0x7fcd5009aa30 2024-09-21T07:34:57.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.472+0000 7fcd56400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcd500d09f0 con 0x7fcd500b6e20 2024-09-21T07:34:57.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.472+0000 7fcd56400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fcd500d0cf0 con 0x7fcd500cd860 2024-09-21T07:34:57.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.473+0000 7fcd55000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd5009aa30 0x7fcd50151520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:57.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.473+0000 7fcd55000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd5009aa30 0x7fcd50151520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57654/0 (socket says 172.21.15.39:57654) 2024-09-21T07:34:57.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.473+0000 7fcd55000640 1 -- 172.21.15.39:0/4244185253 learned_addr learned my addr 172.21.15.39:0/4244185253 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:57.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.473+0000 7fcd55a00640 1 --2- 172.21.15.39:0/4244185253 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcd500cd860 0x7fcd501e5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:57.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.473+0000 7fcd4fe00640 1 --2- 172.21.15.39:0/4244185253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd500b6e20 0x7fcd501f5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:57.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.473+0000 7fcd55000640 1 -- 172.21.15.39:0/4244185253 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcd500cd860 msgr2=0x7fcd501e5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:57.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.473+0000 7fcd55000640 1 --2- 172.21.15.39:0/4244185253 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcd500cd860 0x7fcd501e5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:57.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.473+0000 7fcd55000640 1 -- 172.21.15.39:0/4244185253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd500b6e20 msgr2=0x7fcd501f5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:57.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.473+0000 7fcd55000640 1 --2- 172.21.15.39:0/4244185253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd500b6e20 0x7fcd501f5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:57.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.473+0000 7fcd55000640 1 -- 172.21.15.39:0/4244185253 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd501f64e0 con 0x7fcd5009aa30 2024-09-21T07:34:57.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.474+0000 7fcd55000640 1 --2- 172.21.15.39:0/4244185253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd5009aa30 0x7fcd50151520 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fcd4400c430 tx=0x7fcd4400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:57.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.474+0000 7fcd4cc00640 1 -- 172.21.15.39:0/4244185253 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd44017070 con 0x7fcd5009aa30 2024-09-21T07:34:57.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.474+0000 7fcd4cc00640 1 -- 172.21.15.39:0/4244185253 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd440053c0 con 0x7fcd5009aa30 2024-09-21T07:34:57.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.474+0000 7fcd56400640 1 -- 172.21.15.39:0/4244185253 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd5009e830 con 0x7fcd5009aa30 2024-09-21T07:34:57.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.474+0000 7fcd4cc00640 1 -- 172.21.15.39:0/4244185253 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd440038a0 con 0x7fcd5009aa30 2024-09-21T07:34:57.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.474+0000 7fcd56400640 1 -- 172.21.15.39:0/4244185253 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd5009edf0 con 0x7fcd5009aa30 2024-09-21T07:34:57.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.475+0000 7fcd56400640 1 -- 172.21.15.39:0/4244185253 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd5009e640 con 0x7fcd5009aa30 2024-09-21T07:34:57.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.476+0000 7fcd4cc00640 1 -- 172.21.15.39:0/4244185253 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcd4401a070 con 0x7fcd5009aa30 2024-09-21T07:34:57.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.477+0000 7fcd4cc00640 1 --2- 172.21.15.39:0/4244185253 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcd30078b90 0x7fcd3007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:57.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.477+0000 7fcd4cc00640 1 -- 172.21.15.39:0/4244185253 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6120+0+0 (secure 0 0 0) 0x7fcd4409a210 con 0x7fcd5009aa30 2024-09-21T07:34:57.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.478+0000 7fcd4fe00640 1 --2- 172.21.15.39:0/4244185253 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcd30078b90 0x7fcd3007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:57.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.478+0000 7fcd4fe00640 1 --2- 172.21.15.39:0/4244185253 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcd30078b90 0x7fcd3007b0a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c034000 tx=0x7fcd3c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:57.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.482+0000 7fcd4cc00640 1 -- 172.21.15.39:0/4244185253 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcd44062540 con 0x7fcd5009aa30 2024-09-21T07:34:57.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.679+0000 7fcd56400640 1 -- 172.21.15.39:0/4244185253 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool mksnap", "pool": "cephfs_data", "snap": "snap3"} v 0) -- 0x7fcd500b4700 con 0x7fcd5009aa30 2024-09-21T07:34:57.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.680+0000 7fcd4cc00640 1 -- 172.21.15.39:0/4244185253 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool mksnap", "pool": "cephfs_data", "snap": "snap3"}]=-95 (95) Operation not supported v51) ==== 131+0+0 (secure 0 0 0) 0x7fcd44061e40 con 0x7fcd5009aa30 2024-09-21T07:34:57.682 INFO:teuthology.orchestra.run.smithi039.stderr:Error ENOTSUP: (95) Operation not supported 2024-09-21T07:34:57.701 DEBUG:teuthology.orchestra.run:got remote process result: 95 2024-09-21T07:34:57.702 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool mksnap cephfs_metadata snap4 2024-09-21T07:34:57.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.929+0000 7f245e000640 1 -- 172.21.15.39:0/3873934149 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f245815c580 msgr2=0x7f24580d0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:57.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.929+0000 7f245e000640 1 --2- 172.21.15.39:0/3873934149 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f245815c580 0x7f24580d0800 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f244c009830 tx=0x7f244c02a480 comp rx=0 tx=0).stop 2024-09-21T07:34:57.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.929+0000 7f245e000640 1 -- 172.21.15.39:0/3873934149 shutdown_connections 2024-09-21T07:34:57.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.929+0000 7f245e000640 1 --2- 172.21.15.39:0/3873934149 >> v2:172.21.15.148:3300/0 conn(0x7f2458072ca0 0x7f24580730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:57.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.929+0000 7f245e000640 1 --2- 172.21.15.39:0/3873934149 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f245815c580 0x7f24580d0800 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:57.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.929+0000 7f245e000640 1 -- 172.21.15.39:0/3873934149 >> 172.21.15.39:0/3873934149 conn(0x7f245806ee50 msgr2=0x7f245806f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:57.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.929+0000 7f245e000640 1 -- 172.21.15.39:0/3873934149 shutdown_connections 2024-09-21T07:34:57.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.929+0000 7f245e000640 1 -- 172.21.15.39:0/3873934149 wait complete. 2024-09-21T07:34:57.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.931+0000 7f245e000640 1 Processor -- start 2024-09-21T07:34:57.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.931+0000 7f245e000640 1 -- start start 2024-09-21T07:34:57.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.933+0000 7f245e000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2458072ca0 0x7f245813f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:57.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.933+0000 7f245e000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24580735f0 0x7f245815dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:57.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.933+0000 7f245e000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f245815c580 0x7f245815e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:57.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.933+0000 7f245e000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f24581608d0 con 0x7f24580735f0 2024-09-21T07:34:57.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.933+0000 7f245e000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2458160750 con 0x7f245815c580 2024-09-21T07:34:57.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.933+0000 7f245e000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f2458160a50 con 0x7f2458072ca0 2024-09-21T07:34:57.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.933+0000 7f2457e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24580735f0 0x7f245815dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:57.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.933+0000 7f245cc00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2458072ca0 0x7f245813f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:57.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.933+0000 7f245cc00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2458072ca0 0x7f245813f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.39:33104/0 (socket says 172.21.15.39:33104) 2024-09-21T07:34:57.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.933+0000 7f2457e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24580735f0 0x7f245815dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57668/0 (socket says 172.21.15.39:57668) 2024-09-21T07:34:57.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.933+0000 7f245cc00640 1 -- 172.21.15.39:0/1134407383 learned_addr learned my addr 172.21.15.39:0/1134407383 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:57.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.933+0000 7f245d600640 1 --2- 172.21.15.39:0/1134407383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f245815c580 0x7f245815e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:57.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.934+0000 7f2457e00640 1 -- 172.21.15.39:0/1134407383 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2458072ca0 msgr2=0x7f245813f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:57.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.934+0000 7f2457e00640 1 --2- 172.21.15.39:0/1134407383 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2458072ca0 0x7f245813f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:57.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.934+0000 7f2457e00640 1 -- 172.21.15.39:0/1134407383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f245815c580 msgr2=0x7f245815e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:57.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.934+0000 7f2457e00640 1 --2- 172.21.15.39:0/1134407383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f245815c580 0x7f245815e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:57.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.934+0000 7f2457e00640 1 -- 172.21.15.39:0/1134407383 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f245815eb80 con 0x7f24580735f0 2024-09-21T07:34:57.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.934+0000 7f2457e00640 1 --2- 172.21.15.39:0/1134407383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24580735f0 0x7f245815dad0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f245815ffa0 tx=0x7f244400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:57.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.934+0000 7f2454c00640 1 -- 172.21.15.39:0/1134407383 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244400cc40 con 0x7f24580735f0 2024-09-21T07:34:57.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.934+0000 7f2454c00640 1 -- 172.21.15.39:0/1134407383 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f244400cde0 con 0x7f24580735f0 2024-09-21T07:34:57.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.934+0000 7f2454c00640 1 -- 172.21.15.39:0/1134407383 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2444004280 con 0x7f24580735f0 2024-09-21T07:34:57.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.934+0000 7f245e000640 1 -- 172.21.15.39:0/1134407383 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f245815ee60 con 0x7f24580735f0 2024-09-21T07:34:57.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.935+0000 7f245e000640 1 -- 172.21.15.39:0/1134407383 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f245815f420 con 0x7f24580735f0 2024-09-21T07:34:57.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.935+0000 7f245e000640 1 -- 172.21.15.39:0/1134407383 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24580730b0 con 0x7f24580735f0 2024-09-21T07:34:57.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.937+0000 7f2454c00640 1 -- 172.21.15.39:0/1134407383 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f24440035f0 con 0x7f24580735f0 2024-09-21T07:34:57.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.938+0000 7f2454c00640 1 --2- 172.21.15.39:0/1134407383 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2438078bc0 0x7f243807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:57.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.938+0000 7f2454c00640 1 -- 172.21.15.39:0/1134407383 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6120+0+0 (secure 0 0 0) 0x7f244409b5d0 con 0x7f24580735f0 2024-09-21T07:34:57.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.938+0000 7f245cc00640 1 --2- 172.21.15.39:0/1134407383 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2438078bc0 0x7f243807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:57.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.939+0000 7f245cc00640 1 --2- 172.21.15.39:0/1134407383 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2438078bc0 0x7f243807b0d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f244c005eb0 tx=0x7f244c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:57.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:57.942+0000 7f2454c00640 1 -- 172.21.15.39:0/1134407383 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2444063210 con 0x7f24580735f0 2024-09-21T07:34:58.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.141+0000 7f245e000640 1 -- 172.21.15.39:0/1134407383 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool mksnap", "pool": "cephfs_metadata", "snap": "snap4"} v 0) -- 0x7f245813f600 con 0x7f24580735f0 2024-09-21T07:34:58.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.142+0000 7f2454c00640 1 -- 172.21.15.39:0/1134407383 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool mksnap", "pool": "cephfs_metadata", "snap": "snap4"}]=-95 (95) Operation not supported v51) ==== 135+0+0 (secure 0 0 0) 0x7f2444067c40 con 0x7f24580735f0 2024-09-21T07:34:58.144 INFO:teuthology.orchestra.run.smithi039.stderr:Error ENOTSUP: (95) Operation not supported 2024-09-21T07:34:58.163 DEBUG:teuthology.orchestra.run:got remote process result: 95 2024-09-21T07:34:58.164 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:34:58.164 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:34:58.268 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:34:58.364 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:34:58.364 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:34:58.365 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:34:58.365 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:34:58.401 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:34:58.401 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:34:58.478 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:34:58.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.746+0000 7fda3ee00640 1 -- 172.21.15.39:0/1670604666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda1c003a30 msgr2=0x7fda1c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:58.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.746+0000 7fda3ee00640 1 --2- 172.21.15.39:0/1670604666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda1c003a30 0x7fda1c003ea0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fda28006fd0 tx=0x7fda1c005830 comp rx=0 tx=0).stop 2024-09-21T07:34:58.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.746+0000 7fda3ce00640 1 -- 172.21.15.39:0/1670604666 reap_dead start 2024-09-21T07:34:58.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.746+0000 7fda3ee00640 1 -- 172.21.15.39:0/1670604666 shutdown_connections 2024-09-21T07:34:58.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.746+0000 7fda3ee00640 1 -- 172.21.15.39:0/1670604666 >> 172.21.15.39:0/1670604666 conn(0x7fda3806d7d0 msgr2=0x7fda380b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:58.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.746+0000 7fda3ee00640 1 -- 172.21.15.39:0/1670604666 shutdown_connections 2024-09-21T07:34:58.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.746+0000 7fda3ee00640 1 -- 172.21.15.39:0/1670604666 wait complete. 2024-09-21T07:34:58.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.749+0000 7fda3ee00640 1 Processor -- start 2024-09-21T07:34:58.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.749+0000 7fda3ee00640 1 -- start start 2024-09-21T07:34:58.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.750+0000 7fda3ee00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda1c003a30 0x7fda380c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:58.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.750+0000 7fda3ee00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda380d0380 0x7fda380d0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:58.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.750+0000 7fda3ee00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda3814e290 0x7fda3814e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:58.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.750+0000 7fda3ee00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fda3815fca0 con 0x7fda380d0380 2024-09-21T07:34:58.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.750+0000 7fda3ee00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fda3815fb20 con 0x7fda1c003a30 2024-09-21T07:34:58.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.750+0000 7fda3ee00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fda3815fe20 con 0x7fda3814e290 2024-09-21T07:34:58.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.751+0000 7fda37400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda380d0380 0x7fda380d0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:58.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.751+0000 7fda37400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda380d0380 0x7fda380d0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57696/0 (socket says 172.21.15.39:57696) 2024-09-21T07:34:58.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.751+0000 7fda37400640 1 -- 172.21.15.39:0/2438315779 learned_addr learned my addr 172.21.15.39:0/2438315779 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:58.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.751+0000 7fda3ce00640 1 --2- 172.21.15.39:0/2438315779 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda3814e290 0x7fda3814e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:58.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.751+0000 7fda37e00640 1 --2- 172.21.15.39:0/2438315779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda1c003a30 0x7fda380c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:58.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.751+0000 7fda37400640 1 -- 172.21.15.39:0/2438315779 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda3814e290 msgr2=0x7fda3814e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:58.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.751+0000 7fda37400640 1 --2- 172.21.15.39:0/2438315779 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda3814e290 0x7fda3814e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:58.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.751+0000 7fda37400640 1 -- 172.21.15.39:0/2438315779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda1c003a30 msgr2=0x7fda380c9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:58.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.751+0000 7fda37400640 1 --2- 172.21.15.39:0/2438315779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda1c003a30 0x7fda380c9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:58.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.751+0000 7fda37400640 1 -- 172.21.15.39:0/2438315779 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda3814f720 con 0x7fda380d0380 2024-09-21T07:34:58.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.751+0000 7fda3ce00640 1 --2- 172.21.15.39:0/2438315779 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda3814e290 0x7fda3814e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:34:58.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.751+0000 7fda37400640 1 --2- 172.21.15.39:0/2438315779 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda380d0380 0x7fda380d0790 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fda24011ae0 tx=0x7fda24009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:58.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.752+0000 7fda2fe00640 1 -- 172.21.15.39:0/2438315779 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2401e070 con 0x7fda380d0380 2024-09-21T07:34:58.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.752+0000 7fda2fe00640 1 -- 172.21.15.39:0/2438315779 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda2400fe70 con 0x7fda380d0380 2024-09-21T07:34:58.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.752+0000 7fda2fe00640 1 -- 172.21.15.39:0/2438315779 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda24004290 con 0x7fda380d0380 2024-09-21T07:34:58.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.752+0000 7fda3ee00640 1 -- 172.21.15.39:0/2438315779 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda381fac40 con 0x7fda380d0380 2024-09-21T07:34:58.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.752+0000 7fda3ee00640 1 -- 172.21.15.39:0/2438315779 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda380cb1f0 con 0x7fda380d0380 2024-09-21T07:34:58.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.753+0000 7fda3ee00640 1 -- 172.21.15.39:0/2438315779 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda381fa9b0 con 0x7fda380d0380 2024-09-21T07:34:58.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.754+0000 7fda2fe00640 1 -- 172.21.15.39:0/2438315779 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fda24021070 con 0x7fda380d0380 2024-09-21T07:34:58.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.755+0000 7fda2fe00640 1 --2- 172.21.15.39:0/2438315779 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fda18078bc0 0x7fda1807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:58.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.755+0000 7fda2fe00640 1 -- 172.21.15.39:0/2438315779 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6120+0+0 (secure 0 0 0) 0x7fda2409b900 con 0x7fda380d0380 2024-09-21T07:34:58.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.756+0000 7fda37e00640 1 --2- 172.21.15.39:0/2438315779 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fda18078bc0 0x7fda1807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:58.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.757+0000 7fda37e00640 1 --2- 172.21.15.39:0/2438315779 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fda18078bc0 0x7fda1807b0d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fda28033000 tx=0x7fda28001a10 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:58.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.759+0000 7fda2fe00640 1 -- 172.21.15.39:0/2438315779 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fda3814f720 con 0x7fda380d0380 2024-09-21T07:34:58.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.972+0000 7fda3ee00640 1 -- 172.21.15.39:0/2438315779 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fda38076820 con 0x7fda380d0380 2024-09-21T07:34:58.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.973+0000 7fda2fe00640 1 -- 172.21.15.39:0/2438315779 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v51) ==== 80+0+0 (secure 0 0 0) 0x7fda38076820 con 0x7fda380d0380 2024-09-21T07:34:58.975 INFO:teuthology.orchestra.run.smithi039.stderr:listed 0 entries 2024-09-21T07:34:58.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.975+0000 7fda3ee00640 1 -- 172.21.15.39:0/2438315779 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fda18078bc0 msgr2=0x7fda1807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:58.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.975+0000 7fda3ee00640 1 --2- 172.21.15.39:0/2438315779 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fda18078bc0 0x7fda1807b0d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fda28033000 tx=0x7fda28001a10 comp rx=0 tx=0).stop 2024-09-21T07:34:58.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.975+0000 7fda3ee00640 1 -- 172.21.15.39:0/2438315779 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda380d0380 msgr2=0x7fda380d0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:58.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.975+0000 7fda3ee00640 1 --2- 172.21.15.39:0/2438315779 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda380d0380 0x7fda380d0790 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fda24011ae0 tx=0x7fda24009e90 comp rx=0 tx=0).stop 2024-09-21T07:34:58.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.975+0000 7fda3ee00640 1 -- 172.21.15.39:0/2438315779 shutdown_connections 2024-09-21T07:34:58.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.975+0000 7fda3ee00640 1 --2- 172.21.15.39:0/2438315779 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fda18078bc0 0x7fda1807b0d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:58.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.975+0000 7fda3ee00640 1 --2- 172.21.15.39:0/2438315779 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda3814e290 0x7fda3814e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:58.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.975+0000 7fda3ee00640 1 --2- 172.21.15.39:0/2438315779 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda380d0380 0x7fda380d0790 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:58.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.975+0000 7fda3ee00640 1 --2- 172.21.15.39:0/2438315779 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda1c003a30 0x7fda380c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:58.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.975+0000 7fda3ee00640 1 -- 172.21.15.39:0/2438315779 >> 172.21.15.39:0/2438315779 conn(0x7fda3806d7d0 msgr2=0x7fda3815cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:58.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.976+0000 7fda3ee00640 1 -- 172.21.15.39:0/2438315779 shutdown_connections 2024-09-21T07:34:58.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:58.976+0000 7fda3ee00640 1 -- 172.21.15.39:0/2438315779 wait complete. 2024-09-21T07:34:59.002 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T07:34:59.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:34:59.002 DEBUG:teuthology.orchestra.run.smithi039:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T07:34:59.064 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T07:34:59.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:34:59.065 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:34:59.100 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:34:59.100 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:34:59.101 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:34:59.141 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:34:59.141 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:34:59.141 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:34:59.141 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:34:59.176 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:34:59.176 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:34:59.177 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:34:59.177 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:34:59.219 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:34:59.219 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:34:59.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.444+0000 7fce67a00640 1 -- 172.21.15.39:0/806274176 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce600ce020 msgr2=0x7fce600ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:59.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.444+0000 7fce67a00640 1 --2- 172.21.15.39:0/806274176 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce600ce020 0x7fce600ce430 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fce58005fd0 tx=0x7fce5802aa00 comp rx=0 tx=0).stop 2024-09-21T07:34:59.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.444+0000 7fce67a00640 1 -- 172.21.15.39:0/806274176 shutdown_connections 2024-09-21T07:34:59.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.444+0000 7fce67a00640 1 --2- 172.21.15.39:0/806274176 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce600ce020 0x7fce600ce430 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:59.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.444+0000 7fce67a00640 1 --2- 172.21.15.39:0/806274176 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fce600cf6f0 0x7fce600cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:59.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.444+0000 7fce67a00640 1 -- 172.21.15.39:0/806274176 >> 172.21.15.39:0/806274176 conn(0x7fce600b1cb0 msgr2=0x7fce600b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:59.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.445+0000 7fce67a00640 1 -- 172.21.15.39:0/806274176 shutdown_connections 2024-09-21T07:34:59.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.445+0000 7fce67a00640 1 -- 172.21.15.39:0/806274176 wait complete. 2024-09-21T07:34:59.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.447+0000 7fce67a00640 1 Processor -- start 2024-09-21T07:34:59.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.447+0000 7fce67a00640 1 -- start start 2024-09-21T07:34:59.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.448+0000 7fce67a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce600ccb30 0x7fce60161870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:59.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.448+0000 7fce67a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fce600ce020 0x7fce601544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:59.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.448+0000 7fce67a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce600cf6f0 0x7fce600ae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:59.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.448+0000 7fce67a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fce600701b0 con 0x7fce600cf6f0 2024-09-21T07:34:59.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.448+0000 7fce67a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fce600cf3d0 con 0x7fce600ccb30 2024-09-21T07:34:59.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.448+0000 7fce67a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fce60070330 con 0x7fce600ce020 2024-09-21T07:34:59.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.449+0000 7fce65a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce600cf6f0 0x7fce600ae6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:59.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.449+0000 7fce65a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce600cf6f0 0x7fce600ae6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57720/0 (socket says 172.21.15.39:57720) 2024-09-21T07:34:59.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.449+0000 7fce65a00640 1 -- 172.21.15.39:0/822473 learned_addr learned my addr 172.21.15.39:0/822473 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:59.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.449+0000 7fce5fe00640 1 --2- 172.21.15.39:0/822473 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fce600ce020 0x7fce601544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:59.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.449+0000 7fce65000640 1 --2- 172.21.15.39:0/822473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce600ccb30 0x7fce60161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:59.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.449+0000 7fce65a00640 1 -- 172.21.15.39:0/822473 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fce600ce020 msgr2=0x7fce601544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:59.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.449+0000 7fce65a00640 1 --2- 172.21.15.39:0/822473 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fce600ce020 0x7fce601544e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:59.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.449+0000 7fce65a00640 1 -- 172.21.15.39:0/822473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce600ccb30 msgr2=0x7fce60161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:59.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.449+0000 7fce65a00640 1 --2- 172.21.15.39:0/822473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce600ccb30 0x7fce60161870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:59.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.449+0000 7fce65a00640 1 -- 172.21.15.39:0/822473 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce601f91f0 con 0x7fce600cf6f0 2024-09-21T07:34:59.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.449+0000 7fce65a00640 1 --2- 172.21.15.39:0/822473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce600cf6f0 0x7fce600ae6b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fce58009d90 tx=0x7fce58003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.450+0000 7fce5cc00640 1 -- 172.21.15.39:0/822473 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce58033050 con 0x7fce600cf6f0 2024-09-21T07:34:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.450+0000 7fce5cc00640 1 -- 172.21.15.39:0/822473 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce58004450 con 0x7fce600cf6f0 2024-09-21T07:34:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.450+0000 7fce67a00640 1 -- 172.21.15.39:0/822473 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce601f94d0 con 0x7fce600cf6f0 2024-09-21T07:34:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.450+0000 7fce5cc00640 1 -- 172.21.15.39:0/822473 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce58038450 con 0x7fce600cf6f0 2024-09-21T07:34:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.450+0000 7fce67a00640 1 -- 172.21.15.39:0/822473 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce601f9a90 con 0x7fce600cf6f0 2024-09-21T07:34:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.451+0000 7fce67a00640 1 -- 172.21.15.39:0/822473 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce600cebf0 con 0x7fce600cf6f0 2024-09-21T07:34:59.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.452+0000 7fce5cc00640 1 -- 172.21.15.39:0/822473 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fce58036070 con 0x7fce600cf6f0 2024-09-21T07:34:59.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.453+0000 7fce5cc00640 1 --2- 172.21.15.39:0/822473 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fce40078b90 0x7fce4007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:59.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.453+0000 7fce5cc00640 1 -- 172.21.15.39:0/822473 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6120+0+0 (secure 0 0 0) 0x7fce580bc380 con 0x7fce600cf6f0 2024-09-21T07:34:59.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.454+0000 7fce65000640 1 --2- 172.21.15.39:0/822473 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fce40078b90 0x7fce4007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:59.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.455+0000 7fce65000640 1 --2- 172.21.15.39:0/822473 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fce40078b90 0x7fce4007b0a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fce60071290 tx=0x7fce5400a2d0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:59.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.458+0000 7fce5cc00640 1 -- 172.21.15.39:0/822473 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fce58083f30 con 0x7fce600cf6f0 2024-09-21T07:34:59.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.696+0000 7fce67a00640 1 -- 172.21.15.39:0/822473 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce6015eb10 con 0x7fce600cf6f0 2024-09-21T07:34:59.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.696+0000 7fce5cc00640 1 -- 172.21.15.39:0/822473 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5223 (secure 0 0 0) 0x7fce58087480 con 0x7fce600cf6f0 2024-09-21T07:34:59.698 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:34:59.699 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":26,"btime":"2024-09-21T07:34:46:472997+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4958,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/878081219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":878081219},{"type":"v1","addr":"172.21.15.148:6835","nonce":878081219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3340238606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3340238606},{"type":"v1","addr":"172.21.15.148:6837","nonce":3340238606}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5307,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/929230612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":929230612},{"type":"v1","addr":"172.21.15.39:6834","nonce":929230612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:34:42.955753+0000","modified":"2024-09-21T07:34:45.984044+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/1572835489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":1572835489},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572835489}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5310,"qdb_cluster":[5310]},"id":3}]} 2024-09-21T07:34:59.699 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 26 2024-09-21T07:34:59.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.699+0000 7fce67a00640 1 -- 172.21.15.39:0/822473 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fce40078b90 msgr2=0x7fce4007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:59.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.699+0000 7fce67a00640 1 --2- 172.21.15.39:0/822473 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fce40078b90 0x7fce4007b0a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fce60071290 tx=0x7fce5400a2d0 comp rx=0 tx=0).stop 2024-09-21T07:34:59.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.699+0000 7fce67a00640 1 -- 172.21.15.39:0/822473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce600cf6f0 msgr2=0x7fce600ae6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:59.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.699+0000 7fce67a00640 1 --2- 172.21.15.39:0/822473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce600cf6f0 0x7fce600ae6b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fce58009d90 tx=0x7fce58003f10 comp rx=0 tx=0).stop 2024-09-21T07:34:59.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.699+0000 7fce67a00640 1 -- 172.21.15.39:0/822473 shutdown_connections 2024-09-21T07:34:59.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.699+0000 7fce67a00640 1 --2- 172.21.15.39:0/822473 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fce40078b90 0x7fce4007b0a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:59.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.699+0000 7fce67a00640 1 --2- 172.21.15.39:0/822473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce600cf6f0 0x7fce600ae6b0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:59.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.699+0000 7fce67a00640 1 --2- 172.21.15.39:0/822473 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fce600ce020 0x7fce601544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:59.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.699+0000 7fce67a00640 1 --2- 172.21.15.39:0/822473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce600ccb30 0x7fce60161870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:59.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.699+0000 7fce67a00640 1 -- 172.21.15.39:0/822473 >> 172.21.15.39:0/822473 conn(0x7fce600b1cb0 msgr2=0x7fce600d00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:59.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.699+0000 7fce67a00640 1 -- 172.21.15.39:0/822473 shutdown_connections 2024-09-21T07:34:59.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.699+0000 7fce67a00640 1 -- 172.21.15.39:0/822473 wait complete. 2024-09-21T07:34:59.723 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:34:59.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.949+0000 7f5f94c00640 1 -- 172.21.15.39:0/1722293787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f9015f660 msgr2=0x7f5f901662e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:59.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.949+0000 7f5f94c00640 1 --2- 172.21.15.39:0/1722293787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f9015f660 0x7f5f901662e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5f88005fd0 tx=0x7f5f8802aaa0 comp rx=0 tx=0).stop 2024-09-21T07:34:59.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.949+0000 7f5f94c00640 1 -- 172.21.15.39:0/1722293787 shutdown_connections 2024-09-21T07:34:59.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.949+0000 7f5f94c00640 1 --2- 172.21.15.39:0/1722293787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f9015f660 0x7f5f901662e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:59.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.949+0000 7f5f94c00640 1 -- 172.21.15.39:0/1722293787 >> 172.21.15.39:0/1722293787 conn(0x7f5f9006d410 msgr2=0x7f5f900b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:34:59.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.949+0000 7f5f94c00640 1 -- 172.21.15.39:0/1722293787 shutdown_connections 2024-09-21T07:34:59.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.949+0000 7f5f94c00640 1 -- 172.21.15.39:0/1722293787 wait complete. 2024-09-21T07:34:59.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.951+0000 7f5f94c00640 1 Processor -- start 2024-09-21T07:34:59.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.951+0000 7f5f94c00640 1 -- start start 2024-09-21T07:34:59.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.953+0000 7f5f94c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f90072c30 0x7f5f90150320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:59.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.953+0000 7f5f94c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f901434e0 0x7f5f900cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:59.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.953+0000 7f5f94c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f900ce270 0x7f5f901f8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:59.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.953+0000 7f5f94c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5f90166bb0 con 0x7f5f901434e0 2024-09-21T07:34:59.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.953+0000 7f5f94c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5f90166a30 con 0x7f5f90072c30 2024-09-21T07:34:59.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.953+0000 7f5f94c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f5f90166d30 con 0x7f5f900ce270 2024-09-21T07:34:59.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.953+0000 7f5f87e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f901434e0 0x7f5f900cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:59.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.953+0000 7f5f87e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f901434e0 0x7f5f900cdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57734/0 (socket says 172.21.15.39:57734) 2024-09-21T07:34:59.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.953+0000 7f5f87e00640 1 -- 172.21.15.39:0/1255119653 learned_addr learned my addr 172.21.15.39:0/1255119653 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:34:59.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.953+0000 7f5f8fe00640 1 --2- 172.21.15.39:0/1255119653 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f900ce270 0x7f5f901f8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:59.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.953+0000 7f5f8f400640 1 --2- 172.21.15.39:0/1255119653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f90072c30 0x7f5f90150320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:59.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.953+0000 7f5f87e00640 1 -- 172.21.15.39:0/1255119653 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f900ce270 msgr2=0x7f5f901f8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:59.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.953+0000 7f5f87e00640 1 --2- 172.21.15.39:0/1255119653 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f900ce270 0x7f5f901f8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:59.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.953+0000 7f5f87e00640 1 -- 172.21.15.39:0/1255119653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f90072c30 msgr2=0x7f5f90150320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:34:59.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.953+0000 7f5f87e00640 1 --2- 172.21.15.39:0/1255119653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f90072c30 0x7f5f90150320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:34:59.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.954+0000 7f5f87e00640 1 -- 172.21.15.39:0/1255119653 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f900ce680 con 0x7f5f901434e0 2024-09-21T07:34:59.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.954+0000 7f5f87e00640 1 --2- 172.21.15.39:0/1255119653 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f901434e0 0x7f5f900cdd30 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5f7800c400 tx=0x7f5f7800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:59.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.954+0000 7f5f87400640 1 -- 172.21.15.39:0/1255119653 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f78017070 con 0x7f5f901434e0 2024-09-21T07:34:59.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.954+0000 7f5f87400640 1 -- 172.21.15.39:0/1255119653 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f780053c0 con 0x7f5f901434e0 2024-09-21T07:34:59.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.954+0000 7f5f94c00640 1 -- 172.21.15.39:0/1255119653 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f900d0c70 con 0x7f5f901434e0 2024-09-21T07:34:59.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.954+0000 7f5f87400640 1 -- 172.21.15.39:0/1255119653 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f78003830 con 0x7f5f901434e0 2024-09-21T07:34:59.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.954+0000 7f5f94c00640 1 -- 172.21.15.39:0/1255119653 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f9014e4e0 con 0x7f5f901434e0 2024-09-21T07:34:59.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.955+0000 7f5f94c00640 1 -- 172.21.15.39:0/1255119653 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f9015fbd0 con 0x7f5f901434e0 2024-09-21T07:34:59.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.957+0000 7f5f87400640 1 -- 172.21.15.39:0/1255119653 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5f7801a070 con 0x7f5f901434e0 2024-09-21T07:34:59.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.958+0000 7f5f87400640 1 --2- 172.21.15.39:0/1255119653 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5f6c078b90 0x7f5f6c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:34:59.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.958+0000 7f5f87400640 1 -- 172.21.15.39:0/1255119653 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6120+0+0 (secure 0 0 0) 0x7f5f7809a370 con 0x7f5f901434e0 2024-09-21T07:34:59.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.958+0000 7f5f8f400640 1 --2- 172.21.15.39:0/1255119653 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5f6c078b90 0x7f5f6c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:34:59.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.959+0000 7f5f8f400640 1 --2- 172.21.15.39:0/1255119653 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5f6c078b90 0x7f5f6c07b0a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5f80000f60 tx=0x7f5f80009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:34:59.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:34:59.962+0000 7f5f87400640 1 -- 172.21.15.39:0/1255119653 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5f900ce680 con 0x7f5f901434e0 2024-09-21T07:35:00.025 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:35:00.029 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:35:00.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.199+0000 7f5f94c00640 1 -- 172.21.15.39:0/1255119653 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f900750a0 con 0x7f5f901434e0 2024-09-21T07:35:00.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.199+0000 7f5f87400640 1 -- 172.21.15.39:0/1255119653 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5223 (secure 0 0 0) 0x7f5f78062600 con 0x7f5f901434e0 2024-09-21T07:35:00.201 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:35:00.201 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":26,"btime":"2024-09-21T07:34:46:472997+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4958,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/878081219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":878081219},{"type":"v1","addr":"172.21.15.148:6835","nonce":878081219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3340238606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3340238606},{"type":"v1","addr":"172.21.15.148:6837","nonce":3340238606}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5307,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/929230612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":929230612},{"type":"v1","addr":"172.21.15.39:6834","nonce":929230612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:34:42.955753+0000","modified":"2024-09-21T07:34:45.984044+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/1572835489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":1572835489},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572835489}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5310,"qdb_cluster":[5310]},"id":3}]} 2024-09-21T07:35:00.202 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 26 2024-09-21T07:35:00.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.201+0000 7f5f94c00640 1 -- 172.21.15.39:0/1255119653 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5f6c078b90 msgr2=0x7f5f6c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:00.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.201+0000 7f5f94c00640 1 --2- 172.21.15.39:0/1255119653 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5f6c078b90 0x7f5f6c07b0a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5f80000f60 tx=0x7f5f80009040 comp rx=0 tx=0).stop 2024-09-21T07:35:00.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.201+0000 7f5f94c00640 1 -- 172.21.15.39:0/1255119653 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f901434e0 msgr2=0x7f5f900cdd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:00.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.201+0000 7f5f94c00640 1 --2- 172.21.15.39:0/1255119653 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f901434e0 0x7f5f900cdd30 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5f7800c400 tx=0x7f5f7800c8c0 comp rx=0 tx=0).stop 2024-09-21T07:35:00.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.202+0000 7f5f94c00640 1 -- 172.21.15.39:0/1255119653 shutdown_connections 2024-09-21T07:35:00.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.202+0000 7f5f94c00640 1 --2- 172.21.15.39:0/1255119653 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5f6c078b90 0x7f5f6c07b0a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:00.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.202+0000 7f5f94c00640 1 --2- 172.21.15.39:0/1255119653 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f900ce270 0x7f5f901f8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:00.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.202+0000 7f5f94c00640 1 --2- 172.21.15.39:0/1255119653 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f901434e0 0x7f5f900cdd30 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:00.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.202+0000 7f5f94c00640 1 --2- 172.21.15.39:0/1255119653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f90072c30 0x7f5f90150320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:00.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.202+0000 7f5f94c00640 1 -- 172.21.15.39:0/1255119653 >> 172.21.15.39:0/1255119653 conn(0x7f5f9006d410 msgr2=0x7f5f90151000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:00.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.202+0000 7f5f94c00640 1 -- 172.21.15.39:0/1255119653 shutdown_connections 2024-09-21T07:35:00.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.202+0000 7f5f94c00640 1 -- 172.21.15.39:0/1255119653 wait complete. 2024-09-21T07:35:00.228 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:35:00.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.454+0000 7f39f5000640 1 -- 172.21.15.39:0/4087237039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39f00b3b60 msgr2=0x7f39f00cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:00.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.454+0000 7f39f5000640 1 --2- 172.21.15.39:0/4087237039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39f00b3b60 0x7f39f00cfb50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f39f006d7c0 tx=0x7f39d802a530 comp rx=0 tx=0).stop 2024-09-21T07:35:00.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.454+0000 7f39f5000640 1 -- 172.21.15.39:0/4087237039 shutdown_connections 2024-09-21T07:35:00.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.454+0000 7f39f5000640 1 --2- 172.21.15.39:0/4087237039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39f00b3b60 0x7f39f00cfb50 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:00.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.454+0000 7f39f5000640 1 --2- 172.21.15.39:0/4087237039 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39f00d2af0 0x7f39f00b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:00.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.454+0000 7f39f5000640 1 -- 172.21.15.39:0/4087237039 >> 172.21.15.39:0/4087237039 conn(0x7f39f009d690 msgr2=0x7f39f00d4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:00.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.454+0000 7f39f5000640 1 -- 172.21.15.39:0/4087237039 shutdown_connections 2024-09-21T07:35:00.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.454+0000 7f39f5000640 1 -- 172.21.15.39:0/4087237039 wait complete. 2024-09-21T07:35:00.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.457+0000 7f39f5000640 1 Processor -- start 2024-09-21T07:35:00.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.457+0000 7f39f5000640 1 -- start start 2024-09-21T07:35:00.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.458+0000 7f39f5000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39f00b3b60 0x7f39f01f8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:00.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.458+0000 7f39f5000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39f00d0090 0x7f39f01f8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:00.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.458+0000 7f39f5000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39f00d2af0 0x7f39f00cce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:00.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.458+0000 7f39f5000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f39f00d0830 con 0x7f39f00b3b60 2024-09-21T07:35:00.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.458+0000 7f39f5000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f39f00d06b0 con 0x7f39f00d0090 2024-09-21T07:35:00.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.458+0000 7f39f5000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f39f00d13d0 con 0x7f39f00d2af0 2024-09-21T07:35:00.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.458+0000 7f39ef400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39f00b3b60 0x7f39f01f8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:00.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.458+0000 7f39ef400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39f00b3b60 0x7f39f01f8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57750/0 (socket says 172.21.15.39:57750) 2024-09-21T07:35:00.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.459+0000 7f39ef400640 1 -- 172.21.15.39:0/3594468271 learned_addr learned my addr 172.21.15.39:0/3594468271 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:00.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.459+0000 7f39e7e00640 1 --2- 172.21.15.39:0/3594468271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39f00d0090 0x7f39f01f8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:00.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.459+0000 7f39efe00640 1 --2- 172.21.15.39:0/3594468271 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39f00d2af0 0x7f39f00cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:00.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.459+0000 7f39ef400640 1 -- 172.21.15.39:0/3594468271 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39f00d2af0 msgr2=0x7f39f00cce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:00.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.459+0000 7f39ef400640 1 --2- 172.21.15.39:0/3594468271 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39f00d2af0 0x7f39f00cce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:00.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.459+0000 7f39ef400640 1 -- 172.21.15.39:0/3594468271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39f00d0090 msgr2=0x7f39f01f8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:00.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.459+0000 7f39ef400640 1 --2- 172.21.15.39:0/3594468271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39f00d0090 0x7f39f01f8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:00.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.459+0000 7f39ef400640 1 -- 172.21.15.39:0/3594468271 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39f01540b0 con 0x7f39f00b3b60 2024-09-21T07:35:00.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.459+0000 7f39e7e00640 1 --2- 172.21.15.39:0/3594468271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39f00d0090 0x7f39f01f8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:35:00.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.459+0000 7f39efe00640 1 --2- 172.21.15.39:0/3594468271 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39f00d2af0 0x7f39f00cce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:35:00.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.459+0000 7f39ef400640 1 --2- 172.21.15.39:0/3594468271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39f00b3b60 0x7f39f01f8970 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f39e000c9a0 tx=0x7f39e000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:00.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.460+0000 7f39e7400640 1 -- 172.21.15.39:0/3594468271 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e0007bb0 con 0x7f39f00b3b60 2024-09-21T07:35:00.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.460+0000 7f39e7400640 1 -- 172.21.15.39:0/3594468271 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39e0007e40 con 0x7f39f00b3b60 2024-09-21T07:35:00.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.460+0000 7f39f5000640 1 -- 172.21.15.39:0/3594468271 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39f00cc330 con 0x7f39f00b3b60 2024-09-21T07:35:00.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.460+0000 7f39e7400640 1 -- 172.21.15.39:0/3594468271 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e000f690 con 0x7f39f00b3b60 2024-09-21T07:35:00.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.460+0000 7f39f5000640 1 -- 172.21.15.39:0/3594468271 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39f00cc8f0 con 0x7f39f00b3b60 2024-09-21T07:35:00.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.461+0000 7f39f5000640 1 -- 172.21.15.39:0/3594468271 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39f00d18f0 con 0x7f39f00b3b60 2024-09-21T07:35:00.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.462+0000 7f39e7400640 1 -- 172.21.15.39:0/3594468271 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f39e000f830 con 0x7f39f00b3b60 2024-09-21T07:35:00.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.463+0000 7f39e7400640 1 --2- 172.21.15.39:0/3594468271 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f39cc078b90 0x7f39cc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:00.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.463+0000 7f39e7400640 1 -- 172.21.15.39:0/3594468271 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6120+0+0 (secure 0 0 0) 0x7f39e009b6b0 con 0x7f39f00b3b60 2024-09-21T07:35:00.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.464+0000 7f39e7e00640 1 --2- 172.21.15.39:0/3594468271 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f39cc078b90 0x7f39cc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:00.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.465+0000 7f39e7e00640 1 --2- 172.21.15.39:0/3594468271 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f39cc078b90 0x7f39cc07b0a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f39d8033000 tx=0x7f39d8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:00.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.468+0000 7f39e7400640 1 -- 172.21.15.39:0/3594468271 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f39e0014030 con 0x7f39f00b3b60 2024-09-21T07:35:00.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.665+0000 7f39f5000640 1 -- 172.21.15.39:0/3594468271 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f39f0068ea0 con 0x7f39f00b3b60 2024-09-21T07:35:00.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.666+0000 7f39e7400640 1 -- 172.21.15.39:0/3594468271 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+18635 (secure 0 0 0) 0x7f39e0063a50 con 0x7f39f00b3b60 2024-09-21T07:35:00.669 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:35:00.669 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":51,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:34:53.513338+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:34:40.886322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-21T07:34:41.906495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.2f","mappings":[{"from":0,"to":6}]},{"pgid":"8.3d","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:35:00.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.669+0000 7f39f5000640 1 -- 172.21.15.39:0/3594468271 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f39cc078b90 msgr2=0x7f39cc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:00.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.669+0000 7f39f5000640 1 --2- 172.21.15.39:0/3594468271 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f39cc078b90 0x7f39cc07b0a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f39d8033000 tx=0x7f39d8004730 comp rx=0 tx=0).stop 2024-09-21T07:35:00.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.669+0000 7f39f5000640 1 -- 172.21.15.39:0/3594468271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39f00b3b60 msgr2=0x7f39f01f8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:00.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.669+0000 7f39f5000640 1 --2- 172.21.15.39:0/3594468271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39f00b3b60 0x7f39f01f8970 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f39e000c9a0 tx=0x7f39e000ce60 comp rx=0 tx=0).stop 2024-09-21T07:35:00.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.669+0000 7f39f5000640 1 -- 172.21.15.39:0/3594468271 shutdown_connections 2024-09-21T07:35:00.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.669+0000 7f39f5000640 1 --2- 172.21.15.39:0/3594468271 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f39cc078b90 0x7f39cc07b0a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:00.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.669+0000 7f39f5000640 1 --2- 172.21.15.39:0/3594468271 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39f00d2af0 0x7f39f00cce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:00.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.669+0000 7f39f5000640 1 --2- 172.21.15.39:0/3594468271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39f00d0090 0x7f39f01f8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:00.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.669+0000 7f39f5000640 1 --2- 172.21.15.39:0/3594468271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39f00b3b60 0x7f39f01f8970 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:00.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.669+0000 7f39f5000640 1 -- 172.21.15.39:0/3594468271 >> 172.21.15.39:0/3594468271 conn(0x7f39f009d690 msgr2=0x7f39f00b9140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:00.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.669+0000 7f39f5000640 1 -- 172.21.15.39:0/3594468271 shutdown_connections 2024-09-21T07:35:00.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.669+0000 7f39f5000640 1 -- 172.21.15.39:0/3594468271 wait complete. 2024-09-21T07:35:00.693 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T07:35:00.693 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:35:00.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.916+0000 7f8167200640 1 -- 172.21.15.39:0/3041486622 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81681600c0 msgr2=0x7f8168142b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:00.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.916+0000 7f8167200640 1 --2- 172.21.15.39:0/3041486622 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81681600c0 0x7f8168142b00 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f81680cc690 tx=0x7f815802a5f0 comp rx=0 tx=0).stop 2024-09-21T07:35:00.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.916+0000 7f8167200640 1 -- 172.21.15.39:0/3041486622 shutdown_connections 2024-09-21T07:35:00.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.916+0000 7f8167200640 1 --2- 172.21.15.39:0/3041486622 >> v2:172.21.15.148:3300/0 conn(0x7f8168072c30 0x7f816815ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:00.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.916+0000 7f8167200640 1 --2- 172.21.15.39:0/3041486622 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81681600c0 0x7f8168142b00 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:00.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.916+0000 7f8167200640 1 -- 172.21.15.39:0/3041486622 >> 172.21.15.39:0/3041486622 conn(0x7f816806d410 msgr2=0x7f81680b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:00.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.916+0000 7f8167200640 1 -- 172.21.15.39:0/3041486622 shutdown_connections 2024-09-21T07:35:00.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.916+0000 7f8167200640 1 -- 172.21.15.39:0/3041486622 wait complete. 2024-09-21T07:35:00.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.918+0000 7f8167200640 1 Processor -- start 2024-09-21T07:35:00.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.918+0000 7f8167200640 1 -- start start 2024-09-21T07:35:00.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.920+0000 7f8167200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8168072c30 0x7f81681459e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:00.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.920+0000 7f8167200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81680ced60 0x7f816815bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:00.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.920+0000 7f8167200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f81681600c0 0x7f816815c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:00.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.920+0000 7f8167200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f816806f9c0 con 0x7f81680ced60 2024-09-21T07:35:00.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.920+0000 7f8167200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f816806f840 con 0x7f8168072c30 2024-09-21T07:35:00.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.920+0000 7f8167200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f816806fb40 con 0x7f81681600c0 2024-09-21T07:35:00.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.920+0000 7f815fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81680ced60 0x7f816815bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:00.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.920+0000 7f815fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81680ced60 0x7f816815bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57762/0 (socket says 172.21.15.39:57762) 2024-09-21T07:35:00.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.920+0000 7f815fe00640 1 -- 172.21.15.39:0/885063949 learned_addr learned my addr 172.21.15.39:0/885063949 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:00.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.920+0000 7f8165e00640 1 --2- 172.21.15.39:0/885063949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8168072c30 0x7f81681459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:00.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.920+0000 7f8166800640 1 --2- 172.21.15.39:0/885063949 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f81681600c0 0x7f816815c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:00.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.921+0000 7f815fe00640 1 -- 172.21.15.39:0/885063949 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f81681600c0 msgr2=0x7f816815c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:00.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.921+0000 7f815fe00640 1 --2- 172.21.15.39:0/885063949 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f81681600c0 0x7f816815c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:00.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.921+0000 7f815fe00640 1 -- 172.21.15.39:0/885063949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8168072c30 msgr2=0x7f81681459e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:00.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.921+0000 7f815fe00640 1 --2- 172.21.15.39:0/885063949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8168072c30 0x7f81681459e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:00.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.921+0000 7f815fe00640 1 -- 172.21.15.39:0/885063949 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81680b3f90 con 0x7f81680ced60 2024-09-21T07:35:00.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.921+0000 7f815fe00640 1 --2- 172.21.15.39:0/885063949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81680ced60 0x7f816815bbc0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f815000d6d0 tx=0x7f815000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:00.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.921+0000 7f815e000640 1 -- 172.21.15.39:0/885063949 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8150017070 con 0x7f81680ced60 2024-09-21T07:35:00.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.921+0000 7f815e000640 1 -- 172.21.15.39:0/885063949 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8150005330 con 0x7f81680ced60 2024-09-21T07:35:00.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.921+0000 7f815e000640 1 -- 172.21.15.39:0/885063949 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8150009820 con 0x7f81680ced60 2024-09-21T07:35:00.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.921+0000 7f8167200640 1 -- 172.21.15.39:0/885063949 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81680b4270 con 0x7f81680ced60 2024-09-21T07:35:00.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.922+0000 7f8167200640 1 -- 172.21.15.39:0/885063949 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81680b4830 con 0x7f81680ced60 2024-09-21T07:35:00.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.923+0000 7f8167200640 1 -- 172.21.15.39:0/885063949 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8168070060 con 0x7f81680ced60 2024-09-21T07:35:00.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.924+0000 7f815e000640 1 -- 172.21.15.39:0/885063949 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8150009bd0 con 0x7f81680ced60 2024-09-21T07:35:00.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.925+0000 7f815e000640 1 --2- 172.21.15.39:0/885063949 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8144078b90 0x7f814407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:00.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.925+0000 7f815e000640 1 -- 172.21.15.39:0/885063949 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6120+0+0 (secure 0 0 0) 0x7f815009b230 con 0x7f81680ced60 2024-09-21T07:35:00.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.925+0000 7f8165e00640 1 --2- 172.21.15.39:0/885063949 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8144078b90 0x7f814407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:00.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.926+0000 7f8165e00640 1 --2- 172.21.15.39:0/885063949 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8144078b90 0x7f814407b0a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8158033000 tx=0x7f8158004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:00.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:00.929+0000 7f815e000640 1 -- 172.21.15.39:0/885063949 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8150062de0 con 0x7f81680ced60 2024-09-21T07:35:01.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.162+0000 7f8167200640 1 -- 172.21.15.39:0/885063949 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81680cd6b0 con 0x7f81680ced60 2024-09-21T07:35:01.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.162+0000 7f815e000640 1 -- 172.21.15.39:0/885063949 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5223 (secure 0 0 0) 0x7f8150067810 con 0x7f81680ced60 2024-09-21T07:35:01.164 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:35:01.164 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":26,"btime":"2024-09-21T07:34:46:472997+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4958,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/878081219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":878081219},{"type":"v1","addr":"172.21.15.148:6835","nonce":878081219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3340238606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3340238606},{"type":"v1","addr":"172.21.15.148:6837","nonce":3340238606}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5307,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/929230612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":929230612},{"type":"v1","addr":"172.21.15.39:6834","nonce":929230612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:34:42.955753+0000","modified":"2024-09-21T07:34:45.984044+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/1572835489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":1572835489},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572835489}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5310,"qdb_cluster":[5310]},"id":3}]} 2024-09-21T07:35:01.165 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 26 2024-09-21T07:35:01.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.165+0000 7f8167200640 1 -- 172.21.15.39:0/885063949 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8144078b90 msgr2=0x7f814407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:01.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.165+0000 7f8167200640 1 --2- 172.21.15.39:0/885063949 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8144078b90 0x7f814407b0a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8158033000 tx=0x7f8158004730 comp rx=0 tx=0).stop 2024-09-21T07:35:01.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.165+0000 7f8167200640 1 -- 172.21.15.39:0/885063949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81680ced60 msgr2=0x7f816815bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:01.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.165+0000 7f8167200640 1 --2- 172.21.15.39:0/885063949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81680ced60 0x7f816815bbc0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f815000d6d0 tx=0x7f815000db90 comp rx=0 tx=0).stop 2024-09-21T07:35:01.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.165+0000 7f8167200640 1 -- 172.21.15.39:0/885063949 shutdown_connections 2024-09-21T07:35:01.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.165+0000 7f8167200640 1 --2- 172.21.15.39:0/885063949 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8144078b90 0x7f814407b0a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:01.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.165+0000 7f8167200640 1 --2- 172.21.15.39:0/885063949 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f81681600c0 0x7f816815c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:01.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.165+0000 7f8167200640 1 --2- 172.21.15.39:0/885063949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81680ced60 0x7f816815bbc0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:01.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.165+0000 7f8167200640 1 --2- 172.21.15.39:0/885063949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8168072c30 0x7f81681459e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:01.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.165+0000 7f8167200640 1 -- 172.21.15.39:0/885063949 >> 172.21.15.39:0/885063949 conn(0x7f816806d410 msgr2=0x7f81680b1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:01.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.165+0000 7f8167200640 1 -- 172.21.15.39:0/885063949 shutdown_connections 2024-09-21T07:35:01.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.165+0000 7f8167200640 1 -- 172.21.15.39:0/885063949 wait complete. 2024-09-21T07:35:01.190 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:35:01.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.414+0000 7f8d48c00640 1 -- 172.21.15.39:0/2566593732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d440cad30 msgr2=0x7f8d4406ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:01.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.414+0000 7f8d48c00640 1 --2- 172.21.15.39:0/2566593732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d440cad30 0x7f8d4406ffa0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c009830 tx=0x7f8d2c02a510 comp rx=0 tx=0).stop 2024-09-21T07:35:01.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.414+0000 7f8d48c00640 1 -- 172.21.15.39:0/2566593732 shutdown_connections 2024-09-21T07:35:01.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.414+0000 7f8d48c00640 1 --2- 172.21.15.39:0/2566593732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d440cad30 0x7f8d4406ffa0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:01.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.414+0000 7f8d48c00640 1 --2- 172.21.15.39:0/2566593732 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d440ca800 0x7f8d440cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:01.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.414+0000 7f8d48c00640 1 -- 172.21.15.39:0/2566593732 >> 172.21.15.39:0/2566593732 conn(0x7f8d440b1cb0 msgr2=0x7f8d440b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:01.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.414+0000 7f8d48c00640 1 -- 172.21.15.39:0/2566593732 shutdown_connections 2024-09-21T07:35:01.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.414+0000 7f8d48c00640 1 -- 172.21.15.39:0/2566593732 wait complete. 2024-09-21T07:35:01.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.417+0000 7f8d48c00640 1 Processor -- start 2024-09-21T07:35:01.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.417+0000 7f8d48c00640 1 -- start start 2024-09-21T07:35:01.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.418+0000 7f8d48c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d440704e0 0x7f8d44156140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:01.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.418+0000 7f8d48c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d440ca800 0x7f8d4409aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:01.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.418+0000 7f8d48c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d440cad30 0x7f8d440c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:01.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.418+0000 7f8d48c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8d440ccf40 con 0x7f8d440cad30 2024-09-21T07:35:01.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.418+0000 7f8d48c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8d440747d0 con 0x7f8d440ca800 2024-09-21T07:35:01.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.418+0000 7f8d48c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f8d440cd0c0 con 0x7f8d440704e0 2024-09-21T07:35:01.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.418+0000 7f8d42800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d440cad30 0x7f8d440c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:01.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.419+0000 7f8d42800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d440cad30 0x7f8d440c93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57792/0 (socket says 172.21.15.39:57792) 2024-09-21T07:35:01.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.419+0000 7f8d42800640 1 -- 172.21.15.39:0/1169942224 learned_addr learned my addr 172.21.15.39:0/1169942224 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:01.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.419+0000 7f8d39400640 1 --2- 172.21.15.39:0/1169942224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d440ca800 0x7f8d4409aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:01.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.419+0000 7f8d41e00640 1 --2- 172.21.15.39:0/1169942224 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d440704e0 0x7f8d44156140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:01.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.419+0000 7f8d42800640 1 -- 172.21.15.39:0/1169942224 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d440704e0 msgr2=0x7f8d44156140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:01.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.419+0000 7f8d42800640 1 --2- 172.21.15.39:0/1169942224 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d440704e0 0x7f8d44156140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:01.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.419+0000 7f8d42800640 1 -- 172.21.15.39:0/1169942224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d440ca800 msgr2=0x7f8d4409aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:01.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.419+0000 7f8d42800640 1 --2- 172.21.15.39:0/1169942224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d440ca800 0x7f8d4409aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:01.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.419+0000 7f8d42800640 1 -- 172.21.15.39:0/1169942224 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d4409b410 con 0x7f8d440cad30 2024-09-21T07:35:01.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.419+0000 7f8d42800640 1 --2- 172.21.15.39:0/1169942224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d440cad30 0x7f8d440c93d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c00c730 tx=0x7f8d3c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:01.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.420+0000 7f8d3aa00640 1 -- 172.21.15.39:0/1169942224 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c003b70 con 0x7f8d440cad30 2024-09-21T07:35:01.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.420+0000 7f8d3aa00640 1 -- 172.21.15.39:0/1169942224 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d3c003d30 con 0x7f8d440cad30 2024-09-21T07:35:01.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.420+0000 7f8d3aa00640 1 -- 172.21.15.39:0/1169942224 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c0052d0 con 0x7f8d440cad30 2024-09-21T07:35:01.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.420+0000 7f8d48c00640 1 -- 172.21.15.39:0/1169942224 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d4409b6f0 con 0x7f8d440cad30 2024-09-21T07:35:01.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.420+0000 7f8d48c00640 1 -- 172.21.15.39:0/1169942224 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d4409bcb0 con 0x7f8d440cad30 2024-09-21T07:35:01.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.421+0000 7f8d48c00640 1 -- 172.21.15.39:0/1169942224 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d4409b180 con 0x7f8d440cad30 2024-09-21T07:35:01.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.422+0000 7f8d3aa00640 1 -- 172.21.15.39:0/1169942224 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8d3c005470 con 0x7f8d440cad30 2024-09-21T07:35:01.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.423+0000 7f8d3aa00640 1 --2- 172.21.15.39:0/1169942224 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8d20078b90 0x7f8d2007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:01.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.423+0000 7f8d3aa00640 1 -- 172.21.15.39:0/1169942224 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6120+0+0 (secure 0 0 0) 0x7f8d3c09ab40 con 0x7f8d440cad30 2024-09-21T07:35:01.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.423+0000 7f8d41e00640 1 --2- 172.21.15.39:0/1169942224 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8d20078b90 0x7f8d2007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:01.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.424+0000 7f8d41e00640 1 --2- 172.21.15.39:0/1169942224 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8d20078b90 0x7f8d2007b0a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8d440cb250 tx=0x7f8d3400b390 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:01.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.428+0000 7f8d3aa00640 1 -- 172.21.15.39:0/1169942224 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8d440708f0 con 0x7f8d440cad30 2024-09-21T07:35:01.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.664+0000 7f8d48c00640 1 -- 172.21.15.39:0/1169942224 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d4415bc30 con 0x7f8d440cad30 2024-09-21T07:35:01.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.665+0000 7f8d3aa00640 1 -- 172.21.15.39:0/1169942224 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5223 (secure 0 0 0) 0x7f8d3c062780 con 0x7f8d440cad30 2024-09-21T07:35:01.667 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:35:01.667 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":26,"btime":"2024-09-21T07:34:46:472997+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4958,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/878081219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":878081219},{"type":"v1","addr":"172.21.15.148:6835","nonce":878081219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3340238606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3340238606},{"type":"v1","addr":"172.21.15.148:6837","nonce":3340238606}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5307,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/929230612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":929230612},{"type":"v1","addr":"172.21.15.39:6834","nonce":929230612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:34:42.955753+0000","modified":"2024-09-21T07:34:45.984044+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/1572835489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":1572835489},{"type":"v1","addr":"172.21.15.39:6835","nonce":1572835489}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5310,"qdb_cluster":[5310]},"id":3}]} 2024-09-21T07:35:01.668 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 26 2024-09-21T07:35:01.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.667+0000 7f8d48c00640 1 -- 172.21.15.39:0/1169942224 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8d20078b90 msgr2=0x7f8d2007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:01.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.667+0000 7f8d48c00640 1 --2- 172.21.15.39:0/1169942224 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8d20078b90 0x7f8d2007b0a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8d440cb250 tx=0x7f8d3400b390 comp rx=0 tx=0).stop 2024-09-21T07:35:01.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.667+0000 7f8d48c00640 1 -- 172.21.15.39:0/1169942224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d440cad30 msgr2=0x7f8d440c93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:01.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.667+0000 7f8d48c00640 1 --2- 172.21.15.39:0/1169942224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d440cad30 0x7f8d440c93d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c00c730 tx=0x7f8d3c00cbf0 comp rx=0 tx=0).stop 2024-09-21T07:35:01.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.667+0000 7f8d48c00640 1 -- 172.21.15.39:0/1169942224 shutdown_connections 2024-09-21T07:35:01.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.668+0000 7f8d48c00640 1 --2- 172.21.15.39:0/1169942224 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8d20078b90 0x7f8d2007b0a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:01.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.668+0000 7f8d48c00640 1 --2- 172.21.15.39:0/1169942224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d440cad30 0x7f8d440c93d0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:01.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.668+0000 7f8d48c00640 1 --2- 172.21.15.39:0/1169942224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d440ca800 0x7f8d4409aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:01.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.668+0000 7f8d48c00640 1 --2- 172.21.15.39:0/1169942224 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d440704e0 0x7f8d44156140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:01.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.668+0000 7f8d48c00640 1 -- 172.21.15.39:0/1169942224 >> 172.21.15.39:0/1169942224 conn(0x7f8d440b1cb0 msgr2=0x7f8d440aefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:01.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.668+0000 7f8d48c00640 1 -- 172.21.15.39:0/1169942224 shutdown_connections 2024-09-21T07:35:01.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.668+0000 7f8d48c00640 1 -- 172.21.15.39:0/1169942224 wait complete. 2024-09-21T07:35:01.691 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:35:01.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.915+0000 7f95b6a00640 1 -- 172.21.15.39:0/3803697246 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95b0073170 msgr2=0x7f95b015c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:01.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.915+0000 7f95b6a00640 1 --2- 172.21.15.39:0/3803697246 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95b0073170 0x7f95b015c780 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f95b00b4b40 tx=0x7f95a002a4d0 comp rx=0 tx=0).stop 2024-09-21T07:35:01.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.915+0000 7f95b6a00640 1 -- 172.21.15.39:0/3803697246 shutdown_connections 2024-09-21T07:35:01.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.915+0000 7f95b6a00640 1 --2- 172.21.15.39:0/3803697246 >> v2:172.21.15.148:3300/0 conn(0x7f95b0070440 0x7f95b00b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:01.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.915+0000 7f95b6a00640 1 --2- 172.21.15.39:0/3803697246 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95b0073170 0x7f95b015c780 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:01.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.915+0000 7f95b6a00640 1 -- 172.21.15.39:0/3803697246 >> 172.21.15.39:0/3803697246 conn(0x7f95b00b1cb0 msgr2=0x7f95b00b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:01.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.915+0000 7f95b6a00640 1 -- 172.21.15.39:0/3803697246 shutdown_connections 2024-09-21T07:35:01.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.915+0000 7f95b6a00640 1 -- 172.21.15.39:0/3803697246 wait complete. 2024-09-21T07:35:01.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.918+0000 7f95b6a00640 1 Processor -- start 2024-09-21T07:35:01.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.918+0000 7f95b6a00640 1 -- start start 2024-09-21T07:35:01.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.919+0000 7f95b6a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95b0070030 0x7f95b0157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:01.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.920+0000 7f95b6a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95b0070440 0x7f95b00cf240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:01.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.920+0000 7f95b6a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95b0073170 0x7f95b00ce020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:01.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.920+0000 7f95b6a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f95b00b3a40 con 0x7f95b0073170 2024-09-21T07:35:01.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.920+0000 7f95b6a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f95b00b38c0 con 0x7f95b0070030 2024-09-21T07:35:01.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.920+0000 7f95b6a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f95b00b3bc0 con 0x7f95b0070440 2024-09-21T07:35:01.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.920+0000 7f95b4a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95b0073170 0x7f95b00ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:01.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.920+0000 7f95b4a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95b0073170 0x7f95b00ce020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57798/0 (socket says 172.21.15.39:57798) 2024-09-21T07:35:01.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.920+0000 7f95b4a00640 1 -- 172.21.15.39:0/1684769686 learned_addr learned my addr 172.21.15.39:0/1684769686 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:01.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.920+0000 7f95afe00640 1 --2- 172.21.15.39:0/1684769686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95b0070030 0x7f95b0157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:01.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.920+0000 7f95a7e00640 1 --2- 172.21.15.39:0/1684769686 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95b0070440 0x7f95b00cf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:01.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.920+0000 7f95b4a00640 1 -- 172.21.15.39:0/1684769686 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95b0070440 msgr2=0x7f95b00cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:01.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.920+0000 7f95b4a00640 1 --2- 172.21.15.39:0/1684769686 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95b0070440 0x7f95b00cf240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:01.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.920+0000 7f95b4a00640 1 -- 172.21.15.39:0/1684769686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95b0070030 msgr2=0x7f95b0157cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:01.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.920+0000 7f95b4a00640 1 --2- 172.21.15.39:0/1684769686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95b0070030 0x7f95b0157cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:01.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.920+0000 7f95b4a00640 1 -- 172.21.15.39:0/1684769686 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95b00cfe20 con 0x7f95b0073170 2024-09-21T07:35:01.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.920+0000 7f95afe00640 1 --2- 172.21.15.39:0/1684769686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95b0070030 0x7f95b0157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:35:01.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.920+0000 7f95a7e00640 1 --2- 172.21.15.39:0/1684769686 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95b0070440 0x7f95b00cf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:35:01.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.921+0000 7f95b4a00640 1 --2- 172.21.15.39:0/1684769686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95b0073170 0x7f95b00ce020 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f95a800cad0 tx=0x7f95a800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:01.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.921+0000 7f95acc00640 1 -- 172.21.15.39:0/1684769686 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a800ae20 con 0x7f95b0073170 2024-09-21T07:35:01.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.921+0000 7f95acc00640 1 -- 172.21.15.39:0/1684769686 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95a800f540 con 0x7f95b0073170 2024-09-21T07:35:01.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.921+0000 7f95b6a00640 1 -- 172.21.15.39:0/1684769686 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95b00d0100 con 0x7f95b0073170 2024-09-21T07:35:01.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.921+0000 7f95acc00640 1 -- 172.21.15.39:0/1684769686 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a800f990 con 0x7f95b0073170 2024-09-21T07:35:01.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.921+0000 7f95b6a00640 1 -- 172.21.15.39:0/1684769686 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95b009af60 con 0x7f95b0073170 2024-09-21T07:35:01.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.922+0000 7f95b6a00640 1 -- 172.21.15.39:0/1684769686 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95b00b40e0 con 0x7f95b0073170 2024-09-21T07:35:01.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.923+0000 7f95acc00640 1 -- 172.21.15.39:0/1684769686 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f95a800fb30 con 0x7f95b0073170 2024-09-21T07:35:01.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.924+0000 7f95acc00640 1 --2- 172.21.15.39:0/1684769686 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f958c078bc0 0x7f958c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:01.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.925+0000 7f95acc00640 1 -- 172.21.15.39:0/1684769686 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6120+0+0 (secure 0 0 0) 0x7f95a809ba50 con 0x7f95b0073170 2024-09-21T07:35:01.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.925+0000 7f95afe00640 1 --2- 172.21.15.39:0/1684769686 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f958c078bc0 0x7f958c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:01.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.926+0000 7f95afe00640 1 --2- 172.21.15.39:0/1684769686 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f958c078bc0 0x7f958c07b0d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f95a0033000 tx=0x7f95a0004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:01.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:01.929+0000 7f95acc00640 1 -- 172.21.15.39:0/1684769686 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f95a8014030 con 0x7f95b0073170 2024-09-21T07:35:02.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.127+0000 7f95b6a00640 1 -- 172.21.15.39:0/1684769686 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f95b00c3860 con 0x7f95b0073170 2024-09-21T07:35:02.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.128+0000 7f95acc00640 1 -- 172.21.15.39:0/1684769686 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+18635 (secure 0 0 0) 0x7f95a800fe20 con 0x7f95b0073170 2024-09-21T07:35:02.131 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:35:02.131 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":51,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:34:53.513338+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:34:40.886322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-21T07:34:41.906495+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.2f","mappings":[{"from":0,"to":6}]},{"pgid":"8.3d","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:35:02.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.131+0000 7f95b6a00640 1 -- 172.21.15.39:0/1684769686 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f958c078bc0 msgr2=0x7f958c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:02.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.131+0000 7f95b6a00640 1 --2- 172.21.15.39:0/1684769686 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f958c078bc0 0x7f958c07b0d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f95a0033000 tx=0x7f95a0004730 comp rx=0 tx=0).stop 2024-09-21T07:35:02.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.131+0000 7f95b6a00640 1 -- 172.21.15.39:0/1684769686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95b0073170 msgr2=0x7f95b00ce020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:02.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.131+0000 7f95b6a00640 1 --2- 172.21.15.39:0/1684769686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95b0073170 0x7f95b00ce020 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f95a800cad0 tx=0x7f95a800cf90 comp rx=0 tx=0).stop 2024-09-21T07:35:02.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.131+0000 7f95b6a00640 1 -- 172.21.15.39:0/1684769686 shutdown_connections 2024-09-21T07:35:02.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.131+0000 7f95b6a00640 1 --2- 172.21.15.39:0/1684769686 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f958c078bc0 0x7f958c07b0d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:02.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.131+0000 7f95b6a00640 1 --2- 172.21.15.39:0/1684769686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95b0073170 0x7f95b00ce020 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:02.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.131+0000 7f95b6a00640 1 --2- 172.21.15.39:0/1684769686 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95b0070440 0x7f95b00cf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:02.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.131+0000 7f95b6a00640 1 --2- 172.21.15.39:0/1684769686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95b0070030 0x7f95b0157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:02.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.131+0000 7f95b6a00640 1 -- 172.21.15.39:0/1684769686 >> 172.21.15.39:0/1684769686 conn(0x7f95b00b1cb0 msgr2=0x7f95b009dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:02.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.131+0000 7f95b6a00640 1 -- 172.21.15.39:0/1684769686 shutdown_connections 2024-09-21T07:35:02.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.131+0000 7f95b6a00640 1 -- 172.21.15.39:0/1684769686 wait complete. 2024-09-21T07:35:02.157 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T07:35:02.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.383+0000 7f28e7200640 1 -- 172.21.15.39:0/651850103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28cc004800 msgr2=0x7f28cc004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:02.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.383+0000 7f28e7200640 1 --2- 172.21.15.39:0/651850103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28cc004800 0x7f28cc004c10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f28d0009b50 tx=0x7f28cc005830 comp rx=0 tx=0).stop 2024-09-21T07:35:02.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.383+0000 7f28e6800640 1 -- 172.21.15.39:0/651850103 reap_dead start 2024-09-21T07:35:02.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.383+0000 7f28e7200640 1 -- 172.21.15.39:0/651850103 shutdown_connections 2024-09-21T07:35:02.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.383+0000 7f28e7200640 1 -- 172.21.15.39:0/651850103 >> 172.21.15.39:0/651850103 conn(0x7f28e806d410 msgr2=0x7f28e80b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:02.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.383+0000 7f28e7200640 1 -- 172.21.15.39:0/651850103 shutdown_connections 2024-09-21T07:35:02.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.383+0000 7f28e7200640 1 -- 172.21.15.39:0/651850103 wait complete. 2024-09-21T07:35:02.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.386+0000 7f28e7200640 1 Processor -- start 2024-09-21T07:35:02.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.386+0000 7f28e7200640 1 -- start start 2024-09-21T07:35:02.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.387+0000 7f28e7200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28cc004800 0x7f28e81484f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:02.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.387+0000 7f28e7200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28e8148b10 0x7f28e81e8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:02.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.387+0000 7f28e7200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28e81e9440 0x7f28e81f89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:02.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.387+0000 7f28e7200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f28e80b39c0 con 0x7f28cc004800 2024-09-21T07:35:02.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.387+0000 7f28e7200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f28e80b3840 con 0x7f28e81e9440 2024-09-21T07:35:02.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.387+0000 7f28e7200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f28e80b3b40 con 0x7f28e8148b10 2024-09-21T07:35:02.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.387+0000 7f28e5e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28cc004800 0x7f28e81484f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:02.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.388+0000 7f28e5e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28cc004800 0x7f28e81484f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49264/0 (socket says 172.21.15.39:49264) 2024-09-21T07:35:02.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.388+0000 7f28e5e00640 1 -- 172.21.15.39:0/3315988294 learned_addr learned my addr 172.21.15.39:0/3315988294 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:02.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.388+0000 7f28dfe00640 1 --2- 172.21.15.39:0/3315988294 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28e8148b10 0x7f28e81e8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:02.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.388+0000 7f28e6800640 1 --2- 172.21.15.39:0/3315988294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28e81e9440 0x7f28e81f89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:02.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.388+0000 7f28e5e00640 1 -- 172.21.15.39:0/3315988294 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28e8148b10 msgr2=0x7f28e81e8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:02.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.388+0000 7f28e5e00640 1 --2- 172.21.15.39:0/3315988294 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28e8148b10 0x7f28e81e8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:02.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.388+0000 7f28e5e00640 1 -- 172.21.15.39:0/3315988294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28e81e9440 msgr2=0x7f28e81f89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:02.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.388+0000 7f28e5e00640 1 --2- 172.21.15.39:0/3315988294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28e81e9440 0x7f28e81f89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:02.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.388+0000 7f28e5e00640 1 -- 172.21.15.39:0/3315988294 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28e814bf30 con 0x7f28cc004800 2024-09-21T07:35:02.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.388+0000 7f28dfe00640 1 --2- 172.21.15.39:0/3315988294 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28e8148b10 0x7f28e81e8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:35:02.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.388+0000 7f28e5e00640 1 --2- 172.21.15.39:0/3315988294 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28cc004800 0x7f28e81484f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f28d800e5a0 tx=0x7f28d800ea60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:02.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.389+0000 7f28de000640 1 -- 172.21.15.39:0/3315988294 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d8015070 con 0x7f28cc004800 2024-09-21T07:35:02.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.389+0000 7f28de000640 1 -- 172.21.15.39:0/3315988294 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f28d8010980 con 0x7f28cc004800 2024-09-21T07:35:02.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.389+0000 7f28de000640 1 -- 172.21.15.39:0/3315988294 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d8010db0 con 0x7f28cc004800 2024-09-21T07:35:02.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.389+0000 7f28e7200640 1 -- 172.21.15.39:0/3315988294 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28e814aad0 con 0x7f28cc004800 2024-09-21T07:35:02.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.389+0000 7f28e7200640 1 -- 172.21.15.39:0/3315988294 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28e814a7f0 con 0x7f28cc004800 2024-09-21T07:35:02.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.390+0000 7f28e7200640 1 -- 172.21.15.39:0/3315988294 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28e815fbd0 con 0x7f28cc004800 2024-09-21T07:35:02.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.391+0000 7f28de000640 1 -- 172.21.15.39:0/3315988294 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f28d8018070 con 0x7f28cc004800 2024-09-21T07:35:02.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.393+0000 7f28de000640 1 --2- 172.21.15.39:0/3315988294 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f28c4078b90 0x7f28c407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:02.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.393+0000 7f28de000640 1 -- 172.21.15.39:0/3315988294 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6120+0+0 (secure 0 0 0) 0x7f28d809b330 con 0x7f28cc004800 2024-09-21T07:35:02.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.393+0000 7f28dfe00640 1 --2- 172.21.15.39:0/3315988294 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f28c4078b90 0x7f28c407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:02.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.394+0000 7f28dfe00640 1 --2- 172.21.15.39:0/3315988294 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f28c4078b90 0x7f28c407b0a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f28d0035000 tx=0x7f28d002d040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:02.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.397+0000 7f28de000640 1 -- 172.21.15.39:0/3315988294 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f28e814aad0 con 0x7f28cc004800 2024-09-21T07:35:02.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.634+0000 7f28e7200640 1 -- 172.21.15.39:0/3315988294 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f28e80750a0 con 0x7f28cc004800 2024-09-21T07:35:02.640 INFO:tasks.ceph.mon.a.smithi039.stderr:2024-09-21T07:35:02.638+0000 7fabab600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T07:35:02.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.644+0000 7f28de000640 1 -- 172.21.15.39:0/3315988294 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) ==== 161+0+0 (secure 0 0 0) 0x7f28e814bf30 con 0x7f28cc004800 2024-09-21T07:35:02.646 INFO:teuthology.orchestra.run.smithi039.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T07:35:02.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.646+0000 7f28e7200640 1 -- 172.21.15.39:0/3315988294 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f28c4078b90 msgr2=0x7f28c407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:02.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.646+0000 7f28e7200640 1 --2- 172.21.15.39:0/3315988294 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f28c4078b90 0x7f28c407b0a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f28d0035000 tx=0x7f28d002d040 comp rx=0 tx=0).stop 2024-09-21T07:35:02.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.646+0000 7f28e7200640 1 -- 172.21.15.39:0/3315988294 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28cc004800 msgr2=0x7f28e81484f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:02.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.646+0000 7f28e7200640 1 --2- 172.21.15.39:0/3315988294 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28cc004800 0x7f28e81484f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f28d800e5a0 tx=0x7f28d800ea60 comp rx=0 tx=0).stop 2024-09-21T07:35:02.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.646+0000 7f28e7200640 1 -- 172.21.15.39:0/3315988294 shutdown_connections 2024-09-21T07:35:02.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.646+0000 7f28e7200640 1 --2- 172.21.15.39:0/3315988294 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f28c4078b90 0x7f28c407b0a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:02.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.646+0000 7f28e7200640 1 --2- 172.21.15.39:0/3315988294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28e81e9440 0x7f28e81f89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:02.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.646+0000 7f28e7200640 1 --2- 172.21.15.39:0/3315988294 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28e8148b10 0x7f28e81e8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:02.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.646+0000 7f28e7200640 1 --2- 172.21.15.39:0/3315988294 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28cc004800 0x7f28e81484f0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:02.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.646+0000 7f28e7200640 1 -- 172.21.15.39:0/3315988294 >> 172.21.15.39:0/3315988294 conn(0x7f28e806d410 msgr2=0x7f28e806ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:02.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.646+0000 7f28e7200640 1 -- 172.21.15.39:0/3315988294 shutdown_connections 2024-09-21T07:35:02.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.646+0000 7f28e7200640 1 -- 172.21.15.39:0/3315988294 wait complete. 2024-09-21T07:35:02.656 INFO:tasks.ceph.mds.a.smithi039.stderr: -2171> 2024-09-21T07:34:45.397+0000 7efea5c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:35:02.672 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T07:35:02.778 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-21T07:35:02.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.902+0000 7fb238a00640 1 -- 172.21.15.39:0/1119986504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb234070030 msgr2=0x7fb234074640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:02.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.902+0000 7fb238a00640 1 --2- 172.21.15.39:0/1119986504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb234070030 0x7fb234074640 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb21c005fd0 tx=0x7fb21c02a5d0 comp rx=0 tx=0).stop 2024-09-21T07:35:02.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.902+0000 7fb238a00640 1 -- 172.21.15.39:0/1119986504 shutdown_connections 2024-09-21T07:35:02.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.902+0000 7fb238a00640 1 --2- 172.21.15.39:0/1119986504 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb234070440 0x7fb234074b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:02.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.902+0000 7fb238a00640 1 --2- 172.21.15.39:0/1119986504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb234070030 0x7fb234074640 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:02.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.903+0000 7fb238a00640 1 -- 172.21.15.39:0/1119986504 >> 172.21.15.39:0/1119986504 conn(0x7fb2340b1cb0 msgr2=0x7fb2340b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:02.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.903+0000 7fb238a00640 1 -- 172.21.15.39:0/1119986504 shutdown_connections 2024-09-21T07:35:02.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.903+0000 7fb238a00640 1 -- 172.21.15.39:0/1119986504 wait complete. 2024-09-21T07:35:02.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.905+0000 7fb238a00640 1 Processor -- start 2024-09-21T07:35:02.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.905+0000 7fb238a00640 1 -- start start 2024-09-21T07:35:02.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.907+0000 7fb238a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb234070030 0x7fb23415b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:02.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.907+0000 7fb238a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb234070440 0x7fb23409aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:02.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.907+0000 7fb238a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb234073170 0x7fb2341e94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:02.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.907+0000 7fb238a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb2340b3a40 con 0x7fb234070030 2024-09-21T07:35:02.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.907+0000 7fb238a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb2340b38c0 con 0x7fb234073170 2024-09-21T07:35:02.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.907+0000 7fb238a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb2340b3bc0 con 0x7fb234070440 2024-09-21T07:35:02.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.907+0000 7fb231e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb234070030 0x7fb23415b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:02.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.907+0000 7fb231e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb234070030 0x7fb23415b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49290/0 (socket says 172.21.15.39:49290) 2024-09-21T07:35:02.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.907+0000 7fb231e00640 1 -- 172.21.15.39:0/2171443616 learned_addr learned my addr 172.21.15.39:0/2171443616 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:02.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.907+0000 7fb22be00640 1 --2- 172.21.15.39:0/2171443616 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb234070440 0x7fb23409aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:02.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.907+0000 7fb232800640 1 --2- 172.21.15.39:0/2171443616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb234073170 0x7fb2341e94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:02.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.907+0000 7fb231e00640 1 -- 172.21.15.39:0/2171443616 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb234070440 msgr2=0x7fb23409aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:02.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.907+0000 7fb231e00640 1 --2- 172.21.15.39:0/2171443616 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb234070440 0x7fb23409aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:02.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.907+0000 7fb231e00640 1 -- 172.21.15.39:0/2171443616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb234073170 msgr2=0x7fb2341e94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:02.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.907+0000 7fb231e00640 1 --2- 172.21.15.39:0/2171443616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb234073170 0x7fb2341e94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:02.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.907+0000 7fb231e00640 1 -- 172.21.15.39:0/2171443616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb23409b6b0 con 0x7fb234070030 2024-09-21T07:35:02.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.908+0000 7fb231e00640 1 --2- 172.21.15.39:0/2171443616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb234070030 0x7fb23415b750 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb22400c3d0 tx=0x7fb22400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:02.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.908+0000 7fb22a000640 1 -- 172.21.15.39:0/2171443616 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb224017070 con 0x7fb234070030 2024-09-21T07:35:02.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.908+0000 7fb238a00640 1 -- 172.21.15.39:0/2171443616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb23409b990 con 0x7fb234070030 2024-09-21T07:35:02.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.908+0000 7fb22a000640 1 -- 172.21.15.39:0/2171443616 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb2240053c0 con 0x7fb234070030 2024-09-21T07:35:02.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.908+0000 7fb22a000640 1 -- 172.21.15.39:0/2171443616 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb224003850 con 0x7fb234070030 2024-09-21T07:35:02.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.908+0000 7fb238a00640 1 -- 172.21.15.39:0/2171443616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb23409bf50 con 0x7fb234070030 2024-09-21T07:35:02.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.909+0000 7fb238a00640 1 -- 172.21.15.39:0/2171443616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2340b40e0 con 0x7fb234070030 2024-09-21T07:35:02.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.911+0000 7fb22a000640 1 -- 172.21.15.39:0/2171443616 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb22401a070 con 0x7fb234070030 2024-09-21T07:35:02.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.912+0000 7fb22a000640 1 --2- 172.21.15.39:0/2171443616 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb210078b90 0x7fb21007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:02.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.912+0000 7fb22a000640 1 -- 172.21.15.39:0/2171443616 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6206+0+0 (secure 0 0 0) 0x7fb22409a2d0 con 0x7fb234070030 2024-09-21T07:35:02.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.913+0000 7fb22be00640 1 --2- 172.21.15.39:0/2171443616 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb210078b90 0x7fb21007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:02.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.913+0000 7fb22be00640 1 --2- 172.21.15.39:0/2171443616 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb210078b90 0x7fb21007b0a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb21c033000 tx=0x7fb21c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:02.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:02.917+0000 7fb22a000640 1 -- 172.21.15.39:0/2171443616 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb224061e30 con 0x7fb234070030 2024-09-21T07:35:03.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.152+0000 7fb238a00640 1 -- 172.21.15.39:0/2171443616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fb2340c3860 con 0x7fb234070030 2024-09-21T07:35:03.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.649+0000 7fb22a000640 1 -- 172.21.15.39:0/2171443616 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) ==== 104+0+0 (secure 0 0 0) 0x7fb224066860 con 0x7fb234070030 2024-09-21T07:35:03.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.651+0000 7fb238a00640 1 -- 172.21.15.39:0/2171443616 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb210078b90 msgr2=0x7fb21007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:03.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.651+0000 7fb238a00640 1 --2- 172.21.15.39:0/2171443616 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb210078b90 0x7fb21007b0a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb21c033000 tx=0x7fb21c004730 comp rx=0 tx=0).stop 2024-09-21T07:35:03.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.651+0000 7fb238a00640 1 -- 172.21.15.39:0/2171443616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb234070030 msgr2=0x7fb23415b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:03.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.651+0000 7fb238a00640 1 --2- 172.21.15.39:0/2171443616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb234070030 0x7fb23415b750 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb22400c3d0 tx=0x7fb22400c890 comp rx=0 tx=0).stop 2024-09-21T07:35:03.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.651+0000 7fb238a00640 1 -- 172.21.15.39:0/2171443616 shutdown_connections 2024-09-21T07:35:03.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.651+0000 7fb238a00640 1 --2- 172.21.15.39:0/2171443616 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb210078b90 0x7fb21007b0a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:03.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.652+0000 7fb238a00640 1 --2- 172.21.15.39:0/2171443616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb234073170 0x7fb2341e94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:03.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.652+0000 7fb238a00640 1 --2- 172.21.15.39:0/2171443616 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb234070440 0x7fb23409aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:03.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.652+0000 7fb238a00640 1 --2- 172.21.15.39:0/2171443616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb234070030 0x7fb23415b750 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:03.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.652+0000 7fb238a00640 1 -- 172.21.15.39:0/2171443616 >> 172.21.15.39:0/2171443616 conn(0x7fb2340b1cb0 msgr2=0x7fb2340ae630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:03.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.652+0000 7fb238a00640 1 -- 172.21.15.39:0/2171443616 shutdown_connections 2024-09-21T07:35:03.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.652+0000 7fb238a00640 1 -- 172.21.15.39:0/2171443616 wait complete. 2024-09-21T07:35:03.676 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T07:35:03.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.907+0000 7fd75ca00640 1 -- 172.21.15.39:0/2254536288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd75806fe50 msgr2=0x7fd758073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:03.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.907+0000 7fd75ca00640 1 --2- 172.21.15.39:0/2254536288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd75806fe50 0x7fd758073590 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd758071db0 tx=0x7fd75002a9a0 comp rx=0 tx=0).stop 2024-09-21T07:35:03.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.907+0000 7fd75ca00640 1 -- 172.21.15.39:0/2254536288 shutdown_connections 2024-09-21T07:35:03.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.907+0000 7fd75ca00640 1 --2- 172.21.15.39:0/2254536288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd75806fe50 0x7fd758073590 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:03.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.907+0000 7fd75ca00640 1 --2- 172.21.15.39:0/2254536288 >> v2:172.21.15.148:3300/0 conn(0x7fd7580748f0 0x7fd7580b2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:03.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.907+0000 7fd75ca00640 1 -- 172.21.15.39:0/2254536288 >> 172.21.15.39:0/2254536288 conn(0x7fd7580b1c40 msgr2=0x7fd7580b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:03.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.907+0000 7fd75ca00640 1 -- 172.21.15.39:0/2254536288 shutdown_connections 2024-09-21T07:35:03.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.907+0000 7fd75ca00640 1 -- 172.21.15.39:0/2254536288 wait complete. 2024-09-21T07:35:03.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.909+0000 7fd75ca00640 1 Processor -- start 2024-09-21T07:35:03.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.909+0000 7fd75ca00640 1 -- start start 2024-09-21T07:35:03.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.911+0000 7fd75ca00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd75806fe50 0x7fd7580c3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:03.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.911+0000 7fd75ca00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7580748f0 0x7fd75814f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:03.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.911+0000 7fd75ca00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7580b3480 0x7fd7580c4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:03.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.911+0000 7fd75ca00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd758073c50 con 0x7fd75806fe50 2024-09-21T07:35:03.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.911+0000 7fd75ca00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd758073ad0 con 0x7fd7580748f0 2024-09-21T07:35:03.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.911+0000 7fd75ca00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd758073dd0 con 0x7fd7580b3480 2024-09-21T07:35:03.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.911+0000 7fd755e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd75806fe50 0x7fd7580c3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:03.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.911+0000 7fd755e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd75806fe50 0x7fd7580c3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49312/0 (socket says 172.21.15.39:49312) 2024-09-21T07:35:03.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.911+0000 7fd755e00640 1 -- 172.21.15.39:0/1682389341 learned_addr learned my addr 172.21.15.39:0/1682389341 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:03.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.911+0000 7fd74d400640 1 --2- 172.21.15.39:0/1682389341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7580748f0 0x7fd75814f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:03.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.911+0000 7fd756800640 1 --2- 172.21.15.39:0/1682389341 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7580b3480 0x7fd7580c4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:03.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.911+0000 7fd755e00640 1 -- 172.21.15.39:0/1682389341 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7580b3480 msgr2=0x7fd7580c4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:03.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.911+0000 7fd755e00640 1 --2- 172.21.15.39:0/1682389341 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7580b3480 0x7fd7580c4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:03.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.911+0000 7fd755e00640 1 -- 172.21.15.39:0/1682389341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7580748f0 msgr2=0x7fd75814f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:03.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.911+0000 7fd755e00640 1 --2- 172.21.15.39:0/1682389341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7580748f0 0x7fd75814f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:03.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.911+0000 7fd755e00640 1 -- 172.21.15.39:0/1682389341 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd75815ceb0 con 0x7fd75806fe50 2024-09-21T07:35:03.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.912+0000 7fd755e00640 1 --2- 172.21.15.39:0/1682389341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd75806fe50 0x7fd7580c3ac0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd74800d7a0 tx=0x7fd74800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:03.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.912+0000 7fd74ea00640 1 -- 172.21.15.39:0/1682389341 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd748017070 con 0x7fd75806fe50 2024-09-21T07:35:03.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.912+0000 7fd74ea00640 1 -- 172.21.15.39:0/1682389341 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd748009d20 con 0x7fd75806fe50 2024-09-21T07:35:03.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.912+0000 7fd75ca00640 1 -- 172.21.15.39:0/1682389341 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd75815d190 con 0x7fd75806fe50 2024-09-21T07:35:03.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.912+0000 7fd74ea00640 1 -- 172.21.15.39:0/1682389341 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd748004280 con 0x7fd75806fe50 2024-09-21T07:35:03.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.912+0000 7fd75ca00640 1 -- 172.21.15.39:0/1682389341 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd75815d750 con 0x7fd75806fe50 2024-09-21T07:35:03.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.913+0000 7fd75ca00640 1 -- 172.21.15.39:0/1682389341 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7580b2f40 con 0x7fd75806fe50 2024-09-21T07:35:03.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.915+0000 7fd74ea00640 1 -- 172.21.15.39:0/1682389341 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd748005020 con 0x7fd75806fe50 2024-09-21T07:35:03.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.915+0000 7fd74ea00640 1 --2- 172.21.15.39:0/1682389341 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd734078bc0 0x7fd73407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:03.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.916+0000 7fd74ea00640 1 -- 172.21.15.39:0/1682389341 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6206+0+0 (secure 0 0 0) 0x7fd74809b3f0 con 0x7fd75806fe50 2024-09-21T07:35:03.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.916+0000 7fd74d400640 1 --2- 172.21.15.39:0/1682389341 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd734078bc0 0x7fd73407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:03.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.917+0000 7fd74d400640 1 --2- 172.21.15.39:0/1682389341 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd734078bc0 0x7fd73407b0d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd7581f6c30 tx=0x7fd740009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:03.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:03.920+0000 7fd74ea00640 1 -- 172.21.15.39:0/1682389341 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd748020020 con 0x7fd75806fe50 2024-09-21T07:35:04.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.116+0000 7fd75ca00640 1 -- 172.21.15.39:0/1682389341 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd7580ccd40 con 0x7fd75806fe50 2024-09-21T07:35:04.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.653+0000 7fd74ea00640 1 -- 172.21.15.39:0/1682389341 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v53) ==== 181+0+0 (secure 0 0 0) 0x7fd7480635a0 con 0x7fd75806fe50 2024-09-21T07:35:04.655 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' removed 2024-09-21T07:35:04.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.655+0000 7fd75ca00640 1 -- 172.21.15.39:0/1682389341 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd734078bc0 msgr2=0x7fd73407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:04.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.655+0000 7fd75ca00640 1 --2- 172.21.15.39:0/1682389341 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd734078bc0 0x7fd73407b0d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd7581f6c30 tx=0x7fd740009040 comp rx=0 tx=0).stop 2024-09-21T07:35:04.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.655+0000 7fd75ca00640 1 -- 172.21.15.39:0/1682389341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd75806fe50 msgr2=0x7fd7580c3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:04.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.655+0000 7fd75ca00640 1 --2- 172.21.15.39:0/1682389341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd75806fe50 0x7fd7580c3ac0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd74800d7a0 tx=0x7fd74800dc60 comp rx=0 tx=0).stop 2024-09-21T07:35:04.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.656+0000 7fd75ca00640 1 -- 172.21.15.39:0/1682389341 shutdown_connections 2024-09-21T07:35:04.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.656+0000 7fd75ca00640 1 --2- 172.21.15.39:0/1682389341 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd734078bc0 0x7fd73407b0d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:04.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.656+0000 7fd75ca00640 1 --2- 172.21.15.39:0/1682389341 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd7580b3480 0x7fd7580c4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:04.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.656+0000 7fd75ca00640 1 --2- 172.21.15.39:0/1682389341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7580748f0 0x7fd75814f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:04.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.656+0000 7fd75ca00640 1 --2- 172.21.15.39:0/1682389341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd75806fe50 0x7fd7580c3ac0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:04.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.656+0000 7fd75ca00640 1 -- 172.21.15.39:0/1682389341 >> 172.21.15.39:0/1682389341 conn(0x7fd7580b1c40 msgr2=0x7fd75815f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:04.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.656+0000 7fd75ca00640 1 -- 172.21.15.39:0/1682389341 shutdown_connections 2024-09-21T07:35:04.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.656+0000 7fd75ca00640 1 -- 172.21.15.39:0/1682389341 wait complete. 2024-09-21T07:35:04.685 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T07:35:04.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.916+0000 7fb3a0000640 1 -- 172.21.15.39:0/3392651609 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3800039e0 msgr2=0x7fb380003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:04.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.916+0000 7fb3a0000640 1 --2- 172.21.15.39:0/3392651609 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3800039e0 0x7fb380003e10 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb38c009b50 tx=0x7fb380005830 comp rx=0 tx=0).stop 2024-09-21T07:35:04.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.916+0000 7fb39e000640 1 -- 172.21.15.39:0/3392651609 reap_dead start 2024-09-21T07:35:04.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.916+0000 7fb3a0000640 1 -- 172.21.15.39:0/3392651609 shutdown_connections 2024-09-21T07:35:04.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.916+0000 7fb3a0000640 1 -- 172.21.15.39:0/3392651609 >> 172.21.15.39:0/3392651609 conn(0x7fb3980b1c40 msgr2=0x7fb3980b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:04.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.916+0000 7fb3a0000640 1 -- 172.21.15.39:0/3392651609 shutdown_connections 2024-09-21T07:35:04.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.916+0000 7fb3a0000640 1 -- 172.21.15.39:0/3392651609 wait complete. 2024-09-21T07:35:04.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.918+0000 7fb3a0000640 1 Processor -- start 2024-09-21T07:35:04.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.919+0000 7fb3a0000640 1 -- start start 2024-09-21T07:35:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.920+0000 7fb3a0000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb3800039e0 0x7fb3981f5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.920+0000 7fb3a0000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3981f94f0 0x7fb39815c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.920+0000 7fb3a0000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3980cdcc0 0x7fb3980ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.920+0000 7fb3a0000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb3980b30c0 con 0x7fb3981f94f0 2024-09-21T07:35:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.920+0000 7fb3a0000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb3980b2f40 con 0x7fb3980cdcc0 2024-09-21T07:35:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.920+0000 7fb3a0000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb3980b3240 con 0x7fb3800039e0 2024-09-21T07:35:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.920+0000 7fb39cc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3981f94f0 0x7fb39815c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.920+0000 7fb39cc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3981f94f0 0x7fb39815c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49336/0 (socket says 172.21.15.39:49336) 2024-09-21T07:35:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.920+0000 7fb39cc00640 1 -- 172.21.15.39:0/1771041870 learned_addr learned my addr 172.21.15.39:0/1771041870 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.920+0000 7fb39d600640 1 --2- 172.21.15.39:0/1771041870 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb3800039e0 0x7fb3981f5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.920+0000 7fb39e000640 1 --2- 172.21.15.39:0/1771041870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3980cdcc0 0x7fb3980ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:04.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.921+0000 7fb39cc00640 1 -- 172.21.15.39:0/1771041870 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb3800039e0 msgr2=0x7fb3981f5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:04.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.921+0000 7fb39cc00640 1 --2- 172.21.15.39:0/1771041870 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb3800039e0 0x7fb3981f5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:04.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.921+0000 7fb39cc00640 1 -- 172.21.15.39:0/1771041870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3980cdcc0 msgr2=0x7fb3980ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:04.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.921+0000 7fb39cc00640 1 --2- 172.21.15.39:0/1771041870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3980cdcc0 0x7fb3980ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:04.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.921+0000 7fb39cc00640 1 -- 172.21.15.39:0/1771041870 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3981f54f0 con 0x7fb3981f94f0 2024-09-21T07:35:04.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.921+0000 7fb39cc00640 1 --2- 172.21.15.39:0/1771041870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3981f94f0 0x7fb39815c490 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb384003f20 tx=0x7fb3840017d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:04.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.921+0000 7fb395600640 1 -- 172.21.15.39:0/1771041870 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb384015070 con 0x7fb3981f94f0 2024-09-21T07:35:04.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.921+0000 7fb395600640 1 -- 172.21.15.39:0/1771041870 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb384011900 con 0x7fb3981f94f0 2024-09-21T07:35:04.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.921+0000 7fb3a0000640 1 -- 172.21.15.39:0/1771041870 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3981e96c0 con 0x7fb3981f94f0 2024-09-21T07:35:04.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.921+0000 7fb395600640 1 -- 172.21.15.39:0/1771041870 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb384011d50 con 0x7fb3981f94f0 2024-09-21T07:35:04.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.921+0000 7fb3a0000640 1 -- 172.21.15.39:0/1771041870 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3981f57b0 con 0x7fb3981f94f0 2024-09-21T07:35:04.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.922+0000 7fb3a0000640 1 -- 172.21.15.39:0/1771041870 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb398073dc0 con 0x7fb3981f94f0 2024-09-21T07:35:04.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.924+0000 7fb395600640 1 -- 172.21.15.39:0/1771041870 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb384018070 con 0x7fb3981f94f0 2024-09-21T07:35:04.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.924+0000 7fb395600640 1 --2- 172.21.15.39:0/1771041870 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb378078bc0 0x7fb37807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:04.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.925+0000 7fb395600640 1 -- 172.21.15.39:0/1771041870 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5768+0+0 (secure 0 0 0) 0x7fb38409ba60 con 0x7fb3981f94f0 2024-09-21T07:35:04.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.925+0000 7fb39d600640 1 --2- 172.21.15.39:0/1771041870 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb378078bc0 0x7fb37807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:04.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.926+0000 7fb39d600640 1 --2- 172.21.15.39:0/1771041870 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb378078bc0 0x7fb37807b0d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb38c035000 tx=0x7fb38c02d040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:04.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:04.929+0000 7fb395600640 1 -- 172.21.15.39:0/1771041870 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb384063800 con 0x7fb3981f94f0 2024-09-21T07:35:05.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.124+0000 7fb3a0000640 1 -- 172.21.15.39:0/1771041870 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb3980730f0 con 0x7fb3981f94f0 2024-09-21T07:35:05.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.664+0000 7fb395600640 1 -- 172.21.15.39:0/1771041870 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v54) ==== 169+0+0 (secure 0 0 0) 0x7fb384068230 con 0x7fb3981f94f0 2024-09-21T07:35:05.666 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' removed 2024-09-21T07:35:05.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.667+0000 7fb3a0000640 1 -- 172.21.15.39:0/1771041870 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb378078bc0 msgr2=0x7fb37807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:05.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.667+0000 7fb3a0000640 1 --2- 172.21.15.39:0/1771041870 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb378078bc0 0x7fb37807b0d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb38c035000 tx=0x7fb38c02d040 comp rx=0 tx=0).stop 2024-09-21T07:35:05.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.667+0000 7fb3a0000640 1 -- 172.21.15.39:0/1771041870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3981f94f0 msgr2=0x7fb39815c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:05.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.667+0000 7fb3a0000640 1 --2- 172.21.15.39:0/1771041870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3981f94f0 0x7fb39815c490 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb384003f20 tx=0x7fb3840017d0 comp rx=0 tx=0).stop 2024-09-21T07:35:05.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.667+0000 7fb3a0000640 1 -- 172.21.15.39:0/1771041870 shutdown_connections 2024-09-21T07:35:05.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.667+0000 7fb3a0000640 1 --2- 172.21.15.39:0/1771041870 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb378078bc0 0x7fb37807b0d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:05.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.667+0000 7fb3a0000640 1 --2- 172.21.15.39:0/1771041870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3980cdcc0 0x7fb3980ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:05.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.667+0000 7fb3a0000640 1 --2- 172.21.15.39:0/1771041870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3981f94f0 0x7fb39815c490 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:05.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.667+0000 7fb3a0000640 1 --2- 172.21.15.39:0/1771041870 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb3800039e0 0x7fb3981f5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:05.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.667+0000 7fb3a0000640 1 -- 172.21.15.39:0/1771041870 >> 172.21.15.39:0/1771041870 conn(0x7fb3980b1c40 msgr2=0x7fb39815f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:05.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.667+0000 7fb3a0000640 1 -- 172.21.15.39:0/1771041870 shutdown_connections 2024-09-21T07:35:05.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.667+0000 7fb3a0000640 1 -- 172.21.15.39:0/1771041870 wait complete. 2024-09-21T07:35:05.697 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T07:35:05.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.923+0000 7f2cdbe00640 1 -- 172.21.15.39:0/1591658344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cdc0b3b10 msgr2=0x7f2cdc0afec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:05.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.923+0000 7f2cdbe00640 1 --2- 172.21.15.39:0/1591658344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cdc0b3b10 0x7f2cdc0afec0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc161030 tx=0x7f2ccc02a540 comp rx=0 tx=0).stop 2024-09-21T07:35:05.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.923+0000 7f2cdbe00640 1 -- 172.21.15.39:0/1591658344 shutdown_connections 2024-09-21T07:35:05.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.923+0000 7f2cdbe00640 1 --2- 172.21.15.39:0/1591658344 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2cdc153480 0x7f2cdc15ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:05.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.923+0000 7f2cdbe00640 1 --2- 172.21.15.39:0/1591658344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cdc0b3b10 0x7f2cdc0afec0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:05.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.923+0000 7f2cdbe00640 1 -- 172.21.15.39:0/1591658344 >> 172.21.15.39:0/1591658344 conn(0x7f2cdc0d0830 msgr2=0x7f2cdc0d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:05.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.923+0000 7f2cdbe00640 1 -- 172.21.15.39:0/1591658344 shutdown_connections 2024-09-21T07:35:05.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.923+0000 7f2cdbe00640 1 -- 172.21.15.39:0/1591658344 wait complete. 2024-09-21T07:35:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.926+0000 7f2cdbe00640 1 Processor -- start 2024-09-21T07:35:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.926+0000 7f2cdbe00640 1 -- start start 2024-09-21T07:35:05.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.927+0000 7f2cdbe00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2cdc0b3b10 0x7f2cdc1f7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:05.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.927+0000 7f2cdbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cdc153480 0x7f2cdc14ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:05.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.927+0000 7f2cdbe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cdc160500 0x7f2cdc1f2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:05.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.927+0000 7f2cdbe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2cdc15d210 con 0x7f2cdc153480 2024-09-21T07:35:05.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.927+0000 7f2cdbe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2cdc15d090 con 0x7f2cdc160500 2024-09-21T07:35:05.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.927+0000 7f2cdbe00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f2cdc15d390 con 0x7f2cdc0b3b10 2024-09-21T07:35:05.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.927+0000 7f2cd3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cdc153480 0x7f2cdc14ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:05.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.927+0000 7f2cd3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cdc153480 0x7f2cdc14ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49362/0 (socket says 172.21.15.39:49362) 2024-09-21T07:35:05.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.927+0000 7f2cd3e00640 1 -- 172.21.15.39:0/711509418 learned_addr learned my addr 172.21.15.39:0/711509418 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:05.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.928+0000 7f2cdaa00640 1 --2- 172.21.15.39:0/711509418 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2cdc0b3b10 0x7f2cdc1f7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:05.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.928+0000 7f2cdb400640 1 --2- 172.21.15.39:0/711509418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cdc160500 0x7f2cdc1f2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:05.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.928+0000 7f2cd3e00640 1 -- 172.21.15.39:0/711509418 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2cdc0b3b10 msgr2=0x7f2cdc1f7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:05.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.928+0000 7f2cd3e00640 1 --2- 172.21.15.39:0/711509418 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2cdc0b3b10 0x7f2cdc1f7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:05.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.928+0000 7f2cd3e00640 1 -- 172.21.15.39:0/711509418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cdc160500 msgr2=0x7f2cdc1f2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:05.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.928+0000 7f2cd3e00640 1 --2- 172.21.15.39:0/711509418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cdc160500 0x7f2cdc1f2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:05.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.928+0000 7f2cd3e00640 1 -- 172.21.15.39:0/711509418 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cdc139df0 con 0x7f2cdc153480 2024-09-21T07:35:05.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.928+0000 7f2cd3e00640 1 --2- 172.21.15.39:0/711509418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cdc153480 0x7f2cdc14ef20 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400e790 tx=0x7f2cc400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:05.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.928+0000 7f2cd2a00640 1 -- 172.21.15.39:0/711509418 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc400eea0 con 0x7f2cdc153480 2024-09-21T07:35:05.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.929+0000 7f2cd2a00640 1 -- 172.21.15.39:0/711509418 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2cc400cdb0 con 0x7f2cdc153480 2024-09-21T07:35:05.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.929+0000 7f2cd2a00640 1 -- 172.21.15.39:0/711509418 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4010690 con 0x7f2cdc153480 2024-09-21T07:35:05.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.929+0000 7f2cdbe00640 1 -- 172.21.15.39:0/711509418 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cdc13a0d0 con 0x7f2cdc153480 2024-09-21T07:35:05.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.929+0000 7f2cdbe00640 1 -- 172.21.15.39:0/711509418 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cdc13a690 con 0x7f2cdc153480 2024-09-21T07:35:05.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.930+0000 7f2cdbe00640 1 -- 172.21.15.39:0/711509418 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cdc15e9c0 con 0x7f2cdc153480 2024-09-21T07:35:05.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.931+0000 7f2cd2a00640 1 -- 172.21.15.39:0/711509418 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2cc4010830 con 0x7f2cdc153480 2024-09-21T07:35:05.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.932+0000 7f2cd2a00640 1 --2- 172.21.15.39:0/711509418 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2cb8078b90 0x7f2cb807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:05.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.932+0000 7f2cd2a00640 1 -- 172.21.15.39:0/711509418 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5296+0+0 (secure 0 0 0) 0x7f2cc409b530 con 0x7f2cdc153480 2024-09-21T07:35:05.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.933+0000 7f2cdaa00640 1 --2- 172.21.15.39:0/711509418 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2cb8078b90 0x7f2cb807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:05.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.934+0000 7f2cdaa00640 1 --2- 172.21.15.39:0/711509418 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2cb8078b90 0x7f2cb807b0a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc033000 tx=0x7f2ccc004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:05.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:05.936+0000 7f2cd2a00640 1 -- 172.21.15.39:0/711509418 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2cc40634d0 con 0x7f2cdc153480 2024-09-21T07:35:06.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.133+0000 7f2cdbe00640 1 -- 172.21.15.39:0/711509418 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f2cdc1f77b0 con 0x7f2cdc153480 2024-09-21T07:35:06.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.138+0000 7f2cd2a00640 1 -- 172.21.15.39:0/711509418 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v36) ==== 71+0+0 (secure 0 0 0) 0x7f2cc4067f00 con 0x7f2cdc153480 2024-09-21T07:35:06.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.140+0000 7f2cdbe00640 1 -- 172.21.15.39:0/711509418 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2cb8078b90 msgr2=0x7f2cb807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:06.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.140+0000 7f2cdbe00640 1 --2- 172.21.15.39:0/711509418 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2cb8078b90 0x7f2cb807b0a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc033000 tx=0x7f2ccc004730 comp rx=0 tx=0).stop 2024-09-21T07:35:06.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.140+0000 7f2cdbe00640 1 -- 172.21.15.39:0/711509418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cdc153480 msgr2=0x7f2cdc14ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:06.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.140+0000 7f2cdbe00640 1 --2- 172.21.15.39:0/711509418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cdc153480 0x7f2cdc14ef20 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400e790 tx=0x7f2cc400ec50 comp rx=0 tx=0).stop 2024-09-21T07:35:06.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.140+0000 7f2cdbe00640 1 -- 172.21.15.39:0/711509418 shutdown_connections 2024-09-21T07:35:06.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.140+0000 7f2cdbe00640 1 --2- 172.21.15.39:0/711509418 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2cb8078b90 0x7f2cb807b0a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:06.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.140+0000 7f2cdbe00640 1 --2- 172.21.15.39:0/711509418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cdc160500 0x7f2cdc1f2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:06.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.140+0000 7f2cdbe00640 1 --2- 172.21.15.39:0/711509418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cdc153480 0x7f2cdc14ef20 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:06.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.140+0000 7f2cdbe00640 1 --2- 172.21.15.39:0/711509418 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2cdc0b3b10 0x7f2cdc1f7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:06.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.140+0000 7f2cdbe00640 1 -- 172.21.15.39:0/711509418 >> 172.21.15.39:0/711509418 conn(0x7f2cdc0d0830 msgr2=0x7f2cdc06d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:06.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.140+0000 7f2cdbe00640 1 -- 172.21.15.39:0/711509418 shutdown_connections 2024-09-21T07:35:06.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.140+0000 7f2cdbe00640 1 -- 172.21.15.39:0/711509418 wait complete. 2024-09-21T07:35:06.166 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_disallow_monitor_managed_snaps_for_fs_pools' 2024-09-21T07:35:06.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.395+0000 7f5556e00640 1 -- 172.21.15.39:0/1125596476 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f555009aa30 msgr2=0x7f555009ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:06.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.395+0000 7f5556e00640 1 --2- 172.21.15.39:0/1125596476 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f555009aa30 0x7f555009ae40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5548005fd0 tx=0x7f554802a8b0 comp rx=0 tx=0).stop 2024-09-21T07:35:06.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.395+0000 7f5556e00640 1 -- 172.21.15.39:0/1125596476 shutdown_connections 2024-09-21T07:35:06.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.395+0000 7f5556e00640 1 --2- 172.21.15.39:0/1125596476 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f555009aa30 0x7f555009ae40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:06.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.395+0000 7f5556e00640 1 --2- 172.21.15.39:0/1125596476 >> v2:172.21.15.148:3300/0 conn(0x7f55500cd860 0x7f55500b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:06.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.395+0000 7f5556e00640 1 -- 172.21.15.39:0/1125596476 >> 172.21.15.39:0/1125596476 conn(0x7f55500b3be0 msgr2=0x7f55500b3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:06.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.395+0000 7f5556e00640 1 -- 172.21.15.39:0/1125596476 shutdown_connections 2024-09-21T07:35:06.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.395+0000 7f5556e00640 1 -- 172.21.15.39:0/1125596476 wait complete. 2024-09-21T07:35:06.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.397+0000 7f5556e00640 1 Processor -- start 2024-09-21T07:35:06.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.398+0000 7f5556e00640 1 -- start start 2024-09-21T07:35:06.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.399+0000 7f5556e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f555009aa30 0x7f55500cc150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:06.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.399+0000 7f5556e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55500b6e20 0x7f5550159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:06.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.399+0000 7f5556e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55500cd860 0x7f555006f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:06.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.399+0000 7f5556e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f55500d0a60 con 0x7f55500cd860 2024-09-21T07:35:06.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.399+0000 7f5556e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f55500d08e0 con 0x7f55500b6e20 2024-09-21T07:35:06.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.399+0000 7f5556e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f55500d0be0 con 0x7f555009aa30 2024-09-21T07:35:06.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.399+0000 7f5556400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55500cd860 0x7f555006f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:06.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.399+0000 7f5556400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55500cd860 0x7f555006f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49368/0 (socket says 172.21.15.39:49368) 2024-09-21T07:35:06.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.400+0000 7f5556400640 1 -- 172.21.15.39:0/234346486 learned_addr learned my addr 172.21.15.39:0/234346486 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:06.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.399+0000 7f5555a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f555009aa30 0x7f55500cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:06.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.400+0000 7f5555000640 1 --2- 172.21.15.39:0/234346486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55500b6e20 0x7f5550159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:06.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.400+0000 7f5556400640 1 -- 172.21.15.39:0/234346486 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f555009aa30 msgr2=0x7f55500cc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:06.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.400+0000 7f5556400640 1 --2- 172.21.15.39:0/234346486 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f555009aa30 0x7f55500cc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:06.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.400+0000 7f5556400640 1 -- 172.21.15.39:0/234346486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55500b6e20 msgr2=0x7f5550159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:06.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.400+0000 7f5556400640 1 --2- 172.21.15.39:0/234346486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55500b6e20 0x7f5550159190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:06.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.400+0000 7f5556400640 1 -- 172.21.15.39:0/234346486 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f555015d240 con 0x7f55500cd860 2024-09-21T07:35:06.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.400+0000 7f5556400640 1 --2- 172.21.15.39:0/234346486 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55500cd860 0x7f555006f090 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f554802aea0 tx=0x7f5548033f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:06.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.401+0000 7f554d600640 1 -- 172.21.15.39:0/234346486 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5548031990 con 0x7f55500cd860 2024-09-21T07:35:06.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.401+0000 7f554d600640 1 -- 172.21.15.39:0/234346486 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5548031c20 con 0x7f55500cd860 2024-09-21T07:35:06.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.401+0000 7f554d600640 1 -- 172.21.15.39:0/234346486 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55480330d0 con 0x7f55500cd860 2024-09-21T07:35:06.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.401+0000 7f5556e00640 1 -- 172.21.15.39:0/234346486 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f555015d740 con 0x7f55500cd860 2024-09-21T07:35:06.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.401+0000 7f5556e00640 1 -- 172.21.15.39:0/234346486 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f555006e890 con 0x7f55500cd860 2024-09-21T07:35:06.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.402+0000 7f5556e00640 1 -- 172.21.15.39:0/234346486 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f555015d4b0 con 0x7f55500cd860 2024-09-21T07:35:06.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.403+0000 7f554d600640 1 -- 172.21.15.39:0/234346486 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5548033270 con 0x7f55500cd860 2024-09-21T07:35:06.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.404+0000 7f554d600640 1 --2- 172.21.15.39:0/234346486 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5530078bc0 0x7f553007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:06.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.404+0000 7f554d600640 1 -- 172.21.15.39:0/234346486 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5296+0+0 (secure 0 0 0) 0x7f55480bbf40 con 0x7f55500cd860 2024-09-21T07:35:06.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.404+0000 7f5555a00640 1 --2- 172.21.15.39:0/234346486 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5530078bc0 0x7f553007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:06.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.405+0000 7f5555a00640 1 --2- 172.21.15.39:0/234346486 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5530078bc0 0x7f553007b0d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5544009590 tx=0x7f5544005f30 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:06.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.409+0000 7f554d600640 1 -- 172.21.15.39:0/234346486 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f554803f030 con 0x7f55500cd860 2024-09-21T07:35:06.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:06.685+0000 7f5556e00640 1 -- 172.21.15.39:0/234346486 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_disallow_monitor_managed_snaps_for_fs_pools"]} v 0) -- 0x7f55500cc690 con 0x7f55500cd860 2024-09-21T07:35:07.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.673+0000 7f554d600640 1 -- 172.21.15.39:0/234346486 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_disallow_monitor_managed_snaps_for_fs_pools"]}]=0 v126) ==== 178+0+0 (secure 0 0 0) 0x7f55480317b0 con 0x7f55500cd860 2024-09-21T07:35:07.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.676+0000 7f5556e00640 1 -- 172.21.15.39:0/234346486 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5530078bc0 msgr2=0x7f553007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:07.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.676+0000 7f5556e00640 1 --2- 172.21.15.39:0/234346486 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5530078bc0 0x7f553007b0d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5544009590 tx=0x7f5544005f30 comp rx=0 tx=0).stop 2024-09-21T07:35:07.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.676+0000 7f5556e00640 1 -- 172.21.15.39:0/234346486 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55500cd860 msgr2=0x7f555006f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:07.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.676+0000 7f5556e00640 1 --2- 172.21.15.39:0/234346486 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55500cd860 0x7f555006f090 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f554802aea0 tx=0x7f5548033f70 comp rx=0 tx=0).stop 2024-09-21T07:35:07.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.676+0000 7f5556e00640 1 -- 172.21.15.39:0/234346486 shutdown_connections 2024-09-21T07:35:07.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.676+0000 7f5556e00640 1 --2- 172.21.15.39:0/234346486 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5530078bc0 0x7f553007b0d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:07.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.676+0000 7f5556e00640 1 --2- 172.21.15.39:0/234346486 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55500cd860 0x7f555006f090 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:07.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.676+0000 7f5556e00640 1 --2- 172.21.15.39:0/234346486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55500b6e20 0x7f5550159190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:07.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.676+0000 7f5556e00640 1 --2- 172.21.15.39:0/234346486 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f555009aa30 0x7f55500cc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:07.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.676+0000 7f5556e00640 1 -- 172.21.15.39:0/234346486 >> 172.21.15.39:0/234346486 conn(0x7f55500b3be0 msgr2=0x7f55500b17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:07.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.677+0000 7f5556e00640 1 -- 172.21.15.39:0/234346486 shutdown_connections 2024-09-21T07:35:07.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.677+0000 7f5556e00640 1 -- 172.21.15.39:0/234346486 wait complete. 2024-09-21T07:35:07.705 INFO:tasks.cephfs_test_runner:Test that creation of monitor managed snaps fails for pools attached ... ok 2024-09-21T07:35:07.705 INFO:tasks.cephfs_test_runner:Starting test: test_using_pool_with_snap_fails_fs_creation (tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools) Test that using a pool with snaps for fs creation fails 2024-09-21T07:35:07.705 INFO:tasks.cephfs_test_runner:test_using_pool_with_snap_fails_fs_creation (tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools) 2024-09-21T07:35:07.705 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:35:07.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.935+0000 7f5394000640 1 -- 172.21.15.39:0/2876435605 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f538c0706c0 msgr2=0x7f538c072ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:07.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.935+0000 7f5394000640 1 --2- 172.21.15.39:0/2876435605 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f538c0706c0 0x7f538c072ff0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5384005fd0 tx=0x7f538402aa00 comp rx=0 tx=0).stop 2024-09-21T07:35:07.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.935+0000 7f5394000640 1 -- 172.21.15.39:0/2876435605 shutdown_connections 2024-09-21T07:35:07.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.935+0000 7f5394000640 1 --2- 172.21.15.39:0/2876435605 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f538c0706c0 0x7f538c072ff0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:07.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.935+0000 7f5394000640 1 --2- 172.21.15.39:0/2876435605 >> v2:172.21.15.148:3300/0 conn(0x7f538c0cd3e0 0x7f538c070180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:07.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.935+0000 7f5394000640 1 -- 172.21.15.39:0/2876435605 >> 172.21.15.39:0/2876435605 conn(0x7f538c06d7d0 msgr2=0x7f538c0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:07.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.935+0000 7f5394000640 1 -- 172.21.15.39:0/2876435605 shutdown_connections 2024-09-21T07:35:07.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.935+0000 7f5394000640 1 -- 172.21.15.39:0/2876435605 wait complete. 2024-09-21T07:35:07.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.938+0000 7f5394000640 1 Processor -- start 2024-09-21T07:35:07.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.938+0000 7f5394000640 1 -- start start 2024-09-21T07:35:07.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.939+0000 7f5394000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f538c0706c0 0x7f538c1599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:07.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.939+0000 7f5394000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f538c0af780 0x7f538c09c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:07.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.939+0000 7f5394000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f538c0cd3e0 0x7f538c09caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:07.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.939+0000 7f5394000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f538c073b70 con 0x7f538c0706c0 2024-09-21T07:35:07.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.939+0000 7f5394000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f538c0739f0 con 0x7f538c0cd3e0 2024-09-21T07:35:07.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.939+0000 7f5394000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f538c073cf0 con 0x7f538c0af780 2024-09-21T07:35:07.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.939+0000 7f5391600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f538c0706c0 0x7f538c1599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:07.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.940+0000 7f5391600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f538c0706c0 0x7f538c1599c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49384/0 (socket says 172.21.15.39:49384) 2024-09-21T07:35:07.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.940+0000 7f5391600640 1 -- 172.21.15.39:0/3388900328 learned_addr learned my addr 172.21.15.39:0/3388900328 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:07.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.940+0000 7f5392000640 1 --2- 172.21.15.39:0/3388900328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f538c0cd3e0 0x7f538c09caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:07.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.940+0000 7f5390c00640 1 --2- 172.21.15.39:0/3388900328 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f538c0af780 0x7f538c09c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:07.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.940+0000 7f5391600640 1 -- 172.21.15.39:0/3388900328 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f538c0af780 msgr2=0x7f538c09c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:07.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.940+0000 7f5391600640 1 --2- 172.21.15.39:0/3388900328 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f538c0af780 0x7f538c09c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:07.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.940+0000 7f5391600640 1 -- 172.21.15.39:0/3388900328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f538c0cd3e0 msgr2=0x7f538c09caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:07.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.940+0000 7f5391600640 1 --2- 172.21.15.39:0/3388900328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f538c0cd3e0 0x7f538c09caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:07.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.940+0000 7f5391600640 1 -- 172.21.15.39:0/3388900328 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f538c159f00 con 0x7f538c0706c0 2024-09-21T07:35:07.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.940+0000 7f5391600640 1 --2- 172.21.15.39:0/3388900328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f538c0706c0 0x7f538c1599c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f538000c3d0 tx=0x7f538000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:07.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.941+0000 7f5389600640 1 -- 172.21.15.39:0/3388900328 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5380017070 con 0x7f538c0706c0 2024-09-21T07:35:07.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.941+0000 7f5389600640 1 -- 172.21.15.39:0/3388900328 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53800053c0 con 0x7f538c0706c0 2024-09-21T07:35:07.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.941+0000 7f5394000640 1 -- 172.21.15.39:0/3388900328 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f538c1e7780 con 0x7f538c0706c0 2024-09-21T07:35:07.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.941+0000 7f5389600640 1 -- 172.21.15.39:0/3388900328 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5380003850 con 0x7f538c0706c0 2024-09-21T07:35:07.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.941+0000 7f5394000640 1 -- 172.21.15.39:0/3388900328 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f538c1f9030 con 0x7f538c0706c0 2024-09-21T07:35:07.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.942+0000 7f5394000640 1 -- 172.21.15.39:0/3388900328 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f538c1f8b50 con 0x7f538c0706c0 2024-09-21T07:35:07.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.943+0000 7f5389600640 1 -- 172.21.15.39:0/3388900328 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f538001a070 con 0x7f538c0706c0 2024-09-21T07:35:07.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.944+0000 7f5389600640 1 --2- 172.21.15.39:0/3388900328 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f536c078b90 0x7f536c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:07.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.944+0000 7f5389600640 1 -- 172.21.15.39:0/3388900328 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5296+0+0 (secure 0 0 0) 0x7f5380099fc0 con 0x7f538c0706c0 2024-09-21T07:35:07.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.944+0000 7f5390c00640 1 --2- 172.21.15.39:0/3388900328 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f536c078b90 0x7f536c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:07.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.945+0000 7f5390c00640 1 --2- 172.21.15.39:0/3388900328 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f536c078b90 0x7f536c07b0a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f538c15c5d0 tx=0x7f5378005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:07.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:07.948+0000 7f5389600640 1 -- 172.21.15.39:0/3388900328 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5380062520 con 0x7f538c0706c0 2024-09-21T07:35:08.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.146+0000 7f5394000640 1 -- 172.21.15.39:0/3388900328 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f538c072ff0 con 0x7f538c0706c0 2024-09-21T07:35:08.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.147+0000 7f5389600640 1 -- 172.21.15.39:0/3388900328 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+14842 (secure 0 0 0) 0x7f5380061e20 con 0x7f538c0706c0 2024-09-21T07:35:08.151 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:35:08.151 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":54,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:35:05.654737+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6835/1572835489":"2024-09-22T07:35:02.639571+0000","172.21.15.39:6833/1572835489":"2024-09-22T07:35:02.639571+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:35:08.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.150+0000 7f5394000640 1 -- 172.21.15.39:0/3388900328 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f536c078b90 msgr2=0x7f536c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:08.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.150+0000 7f5394000640 1 --2- 172.21.15.39:0/3388900328 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f536c078b90 0x7f536c07b0a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f538c15c5d0 tx=0x7f5378005f90 comp rx=0 tx=0).stop 2024-09-21T07:35:08.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.150+0000 7f5394000640 1 -- 172.21.15.39:0/3388900328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f538c0706c0 msgr2=0x7f538c1599c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:08.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.150+0000 7f5394000640 1 --2- 172.21.15.39:0/3388900328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f538c0706c0 0x7f538c1599c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f538000c3d0 tx=0x7f538000c890 comp rx=0 tx=0).stop 2024-09-21T07:35:08.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.150+0000 7f5394000640 1 -- 172.21.15.39:0/3388900328 shutdown_connections 2024-09-21T07:35:08.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.150+0000 7f5394000640 1 --2- 172.21.15.39:0/3388900328 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f536c078b90 0x7f536c07b0a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:08.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.150+0000 7f5394000640 1 --2- 172.21.15.39:0/3388900328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f538c0cd3e0 0x7f538c09caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:08.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.150+0000 7f5394000640 1 --2- 172.21.15.39:0/3388900328 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f538c0af780 0x7f538c09c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:08.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.150+0000 7f5394000640 1 --2- 172.21.15.39:0/3388900328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f538c0706c0 0x7f538c1599c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:08.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.150+0000 7f5394000640 1 -- 172.21.15.39:0/3388900328 >> 172.21.15.39:0/3388900328 conn(0x7f538c06d7d0 msgr2=0x7f538c0ce320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:08.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.150+0000 7f5394000640 1 -- 172.21.15.39:0/3388900328 shutdown_connections 2024-09-21T07:35:08.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.150+0000 7f5394000640 1 -- 172.21.15.39:0/3388900328 wait complete. 2024-09-21T07:35:08.178 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T07:33:01.569433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'snap-test-pool', 'create_time': '2024-09-21T07:34:22.032185+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 1, 'snap_epoch': 37, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T07:35:08.179 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T07:35:08.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.411+0000 7feb1a600640 1 -- 172.21.15.39:0/1655549771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb140704b0 msgr2=0x7feb140b2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:08.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.411+0000 7feb1a600640 1 --2- 172.21.15.39:0/1655549771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb140704b0 0x7feb140b2fb0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7feb140745a0 tx=0x7feb0002a440 comp rx=0 tx=0).stop 2024-09-21T07:35:08.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.411+0000 7feb1a600640 1 -- 172.21.15.39:0/1655549771 shutdown_connections 2024-09-21T07:35:08.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.411+0000 7feb1a600640 1 --2- 172.21.15.39:0/1655549771 >> v2:172.21.15.148:3300/0 conn(0x7feb140ca7b0 0x7feb140cabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:08.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.412+0000 7feb1a600640 1 --2- 172.21.15.39:0/1655549771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb140704b0 0x7feb140b2fb0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:08.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.412+0000 7feb1a600640 1 --2- 172.21.15.39:0/1655549771 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb140cc640 0x7feb140b56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:08.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.412+0000 7feb1a600640 1 -- 172.21.15.39:0/1655549771 >> 172.21.15.39:0/1655549771 conn(0x7feb140d0160 msgr2=0x7feb140d0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:08.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.412+0000 7feb1a600640 1 -- 172.21.15.39:0/1655549771 shutdown_connections 2024-09-21T07:35:08.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.412+0000 7feb1a600640 1 -- 172.21.15.39:0/1655549771 wait complete. 2024-09-21T07:35:08.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.414+0000 7feb1a600640 1 Processor -- start 2024-09-21T07:35:08.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.414+0000 7feb1a600640 1 -- start start 2024-09-21T07:35:08.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.416+0000 7feb1a600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb140704b0 0x7feb1414c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:08.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.416+0000 7feb1a600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb140ca7b0 0x7feb1414cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:08.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.416+0000 7feb1a600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb140cc640 0x7feb1414d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:08.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.416+0000 7feb1a600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7feb140cb4f0 con 0x7feb140704b0 2024-09-21T07:35:08.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.416+0000 7feb1a600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7feb140cb370 con 0x7feb140ca7b0 2024-09-21T07:35:08.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.416+0000 7feb1a600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7feb140cb670 con 0x7feb140cc640 2024-09-21T07:35:08.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.416+0000 7feb13400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb140704b0 0x7feb1414c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:08.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.416+0000 7feb13400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb140704b0 0x7feb1414c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49412/0 (socket says 172.21.15.39:49412) 2024-09-21T07:35:08.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.416+0000 7feb13400640 1 -- 172.21.15.39:0/3420375684 learned_addr learned my addr 172.21.15.39:0/3420375684 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:08.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.416+0000 7feb12a00640 1 --2- 172.21.15.39:0/3420375684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb140ca7b0 0x7feb1414cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:08.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.416+0000 7feb13e00640 1 --2- 172.21.15.39:0/3420375684 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb140cc640 0x7feb1414d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:08.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.416+0000 7feb13400640 1 -- 172.21.15.39:0/3420375684 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb140cc640 msgr2=0x7feb1414d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:08.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.416+0000 7feb13400640 1 --2- 172.21.15.39:0/3420375684 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb140cc640 0x7feb1414d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:08.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.416+0000 7feb13400640 1 -- 172.21.15.39:0/3420375684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb140ca7b0 msgr2=0x7feb1414cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:08.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.416+0000 7feb13400640 1 --2- 172.21.15.39:0/3420375684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb140ca7b0 0x7feb1414cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:08.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.416+0000 7feb13400640 1 -- 172.21.15.39:0/3420375684 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb1409c0a0 con 0x7feb140704b0 2024-09-21T07:35:08.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.416+0000 7feb13e00640 1 --2- 172.21.15.39:0/3420375684 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb140cc640 0x7feb1414d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:35:08.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.417+0000 7feb13400640 1 --2- 172.21.15.39:0/3420375684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb140704b0 0x7feb1414c170 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7feb0400d6d0 tx=0x7feb0400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:08.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.417+0000 7feb0b400640 1 -- 172.21.15.39:0/3420375684 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb04017070 con 0x7feb140704b0 2024-09-21T07:35:08.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.417+0000 7feb0b400640 1 -- 172.21.15.39:0/3420375684 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb04005330 con 0x7feb140704b0 2024-09-21T07:35:08.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.417+0000 7feb0b400640 1 -- 172.21.15.39:0/3420375684 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb04009820 con 0x7feb140704b0 2024-09-21T07:35:08.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.417+0000 7feb1a600640 1 -- 172.21.15.39:0/3420375684 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb1409c380 con 0x7feb140704b0 2024-09-21T07:35:08.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.417+0000 7feb1a600640 1 -- 172.21.15.39:0/3420375684 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb1409c940 con 0x7feb140704b0 2024-09-21T07:35:08.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.418+0000 7feb1a600640 1 -- 172.21.15.39:0/3420375684 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb140b4570 con 0x7feb140704b0 2024-09-21T07:35:08.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.419+0000 7feb0b400640 1 -- 172.21.15.39:0/3420375684 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7feb04009bd0 con 0x7feb140704b0 2024-09-21T07:35:08.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.420+0000 7feb0b400640 1 --2- 172.21.15.39:0/3420375684 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feaf4078b90 0x7feaf407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:08.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.421+0000 7feb0b400640 1 -- 172.21.15.39:0/3420375684 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5296+0+0 (secure 0 0 0) 0x7feb0409aee0 con 0x7feb140704b0 2024-09-21T07:35:08.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.421+0000 7feb12a00640 1 --2- 172.21.15.39:0/3420375684 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feaf4078b90 0x7feaf407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:08.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.422+0000 7feb12a00640 1 --2- 172.21.15.39:0/3420375684 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feaf4078b90 0x7feaf407b0a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7feb00033000 tx=0x7feb00004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:08.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.425+0000 7feb0b400640 1 -- 172.21.15.39:0/3420375684 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7feb040634d0 con 0x7feb140704b0 2024-09-21T07:35:08.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.618+0000 7feb1a600640 1 -- 172.21.15.39:0/3420375684 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7feb140b4290 con 0x7feb140704b0 2024-09-21T07:35:08.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.619+0000 7feb0b400640 1 -- 172.21.15.39:0/3420375684 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) ==== 93+0+10 (secure 0 0 0) 0x7feb04062dd0 con 0x7feb140704b0 2024-09-21T07:35:08.621 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-09-21T07:35:08.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.621+0000 7feb1a600640 1 -- 172.21.15.39:0/3420375684 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feaf4078b90 msgr2=0x7feaf407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:08.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.621+0000 7feb1a600640 1 --2- 172.21.15.39:0/3420375684 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feaf4078b90 0x7feaf407b0a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7feb00033000 tx=0x7feb00004730 comp rx=0 tx=0).stop 2024-09-21T07:35:08.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.621+0000 7feb1a600640 1 -- 172.21.15.39:0/3420375684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb140704b0 msgr2=0x7feb1414c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:08.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.621+0000 7feb1a600640 1 --2- 172.21.15.39:0/3420375684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb140704b0 0x7feb1414c170 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7feb0400d6d0 tx=0x7feb0400db90 comp rx=0 tx=0).stop 2024-09-21T07:35:08.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.621+0000 7feb1a600640 1 -- 172.21.15.39:0/3420375684 shutdown_connections 2024-09-21T07:35:08.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.621+0000 7feb1a600640 1 --2- 172.21.15.39:0/3420375684 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feaf4078b90 0x7feaf407b0a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:08.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.621+0000 7feb1a600640 1 --2- 172.21.15.39:0/3420375684 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb140cc640 0x7feb1414d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:08.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.621+0000 7feb1a600640 1 --2- 172.21.15.39:0/3420375684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb140ca7b0 0x7feb1414cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:08.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.621+0000 7feb1a600640 1 --2- 172.21.15.39:0/3420375684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb140704b0 0x7feb1414c170 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:08.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.621+0000 7feb1a600640 1 -- 172.21.15.39:0/3420375684 >> 172.21.15.39:0/3420375684 conn(0x7feb140d0160 msgr2=0x7feb140cf570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:08.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.621+0000 7feb1a600640 1 -- 172.21.15.39:0/3420375684 shutdown_connections 2024-09-21T07:35:08.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.621+0000 7feb1a600640 1 -- 172.21.15.39:0/3420375684 wait complete. 2024-09-21T07:35:08.645 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2024-09-21T07:35:08.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.879+0000 7f4b6de00640 1 -- 172.21.15.39:0/1227364757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b68073170 msgr2=0x7f4b6815c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:08.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.879+0000 7f4b6de00640 1 --2- 172.21.15.39:0/1227364757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b68073170 0x7f4b6815c780 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4b680710f0 tx=0x7f4b5802a560 comp rx=0 tx=0).stop 2024-09-21T07:35:08.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.879+0000 7f4b6de00640 1 -- 172.21.15.39:0/1227364757 shutdown_connections 2024-09-21T07:35:08.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.879+0000 7f4b6de00640 1 --2- 172.21.15.39:0/1227364757 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b68070440 0x7f4b68074b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:08.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.879+0000 7f4b6de00640 1 --2- 172.21.15.39:0/1227364757 >> v2:172.21.15.148:3300/0 conn(0x7f4b68070030 0x7f4b68074640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:08.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.879+0000 7f4b6de00640 1 --2- 172.21.15.39:0/1227364757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b68073170 0x7f4b6815c780 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:08.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.879+0000 7f4b6de00640 1 -- 172.21.15.39:0/1227364757 >> 172.21.15.39:0/1227364757 conn(0x7f4b680b1cb0 msgr2=0x7f4b680b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:08.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.879+0000 7f4b6de00640 1 -- 172.21.15.39:0/1227364757 shutdown_connections 2024-09-21T07:35:08.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.879+0000 7f4b6de00640 1 -- 172.21.15.39:0/1227364757 wait complete. 2024-09-21T07:35:08.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.881+0000 7f4b6de00640 1 Processor -- start 2024-09-21T07:35:08.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.882+0000 7f4b6de00640 1 -- start start 2024-09-21T07:35:08.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.883+0000 7f4b6de00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b68070030 0x7f4b680cea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:08.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.883+0000 7f4b6de00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b68070440 0x7f4b681534c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:08.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.883+0000 7f4b6de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b68073170 0x7f4b6815f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:08.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.883+0000 7f4b6de00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6806f630 con 0x7f4b68073170 2024-09-21T07:35:08.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.883+0000 7f4b6de00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6806f4b0 con 0x7f4b68070030 2024-09-21T07:35:08.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.883+0000 7f4b6de00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f4b6806f7b0 con 0x7f4b68070440 2024-09-21T07:35:08.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.883+0000 7f4b67400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b68073170 0x7f4b6815f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:08.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.883+0000 7f4b67400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b68073170 0x7f4b6815f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49424/0 (socket says 172.21.15.39:49424) 2024-09-21T07:35:08.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.883+0000 7f4b67400640 1 -- 172.21.15.39:0/756247373 learned_addr learned my addr 172.21.15.39:0/756247373 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:08.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.883+0000 7f4b66a00640 1 --2- 172.21.15.39:0/756247373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b68070030 0x7f4b680cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:08.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.883+0000 7f4b5fe00640 1 --2- 172.21.15.39:0/756247373 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b68070440 0x7f4b681534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:08.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.884+0000 7f4b67400640 1 -- 172.21.15.39:0/756247373 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b68070440 msgr2=0x7f4b681534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:08.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.884+0000 7f4b67400640 1 --2- 172.21.15.39:0/756247373 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b68070440 0x7f4b681534c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:08.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.884+0000 7f4b67400640 1 -- 172.21.15.39:0/756247373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b68070030 msgr2=0x7f4b680cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:08.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.884+0000 7f4b67400640 1 --2- 172.21.15.39:0/756247373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b68070030 0x7f4b680cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:08.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.884+0000 7f4b67400640 1 -- 172.21.15.39:0/756247373 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b6815eb10 con 0x7f4b68073170 2024-09-21T07:35:08.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.884+0000 7f4b67400640 1 --2- 172.21.15.39:0/756247373 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b68073170 0x7f4b6815f1e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f4b6000ea40 tx=0x7f4b6000ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:08.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.884+0000 7f4b5f400640 1 -- 172.21.15.39:0/756247373 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6000ce40 con 0x7f4b68073170 2024-09-21T07:35:08.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.884+0000 7f4b5f400640 1 -- 172.21.15.39:0/756247373 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b60004c10 con 0x7f4b68073170 2024-09-21T07:35:08.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.884+0000 7f4b5f400640 1 -- 172.21.15.39:0/756247373 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b60005020 con 0x7f4b68073170 2024-09-21T07:35:08.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.884+0000 7f4b6de00640 1 -- 172.21.15.39:0/756247373 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b6809b380 con 0x7f4b68073170 2024-09-21T07:35:08.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.885+0000 7f4b6de00640 1 -- 172.21.15.39:0/756247373 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b680cf580 con 0x7f4b68073170 2024-09-21T07:35:08.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.886+0000 7f4b6de00640 1 -- 172.21.15.39:0/756247373 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b6815c780 con 0x7f4b68073170 2024-09-21T07:35:08.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.887+0000 7f4b5f400640 1 -- 172.21.15.39:0/756247373 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4b600051c0 con 0x7f4b68073170 2024-09-21T07:35:08.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.888+0000 7f4b5f400640 1 --2- 172.21.15.39:0/756247373 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4b44078b90 0x7f4b4407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:08.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.888+0000 7f4b5f400640 1 -- 172.21.15.39:0/756247373 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5296+0+0 (secure 0 0 0) 0x7f4b6009b790 con 0x7f4b68073170 2024-09-21T07:35:08.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.888+0000 7f4b66a00640 1 --2- 172.21.15.39:0/756247373 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4b44078b90 0x7f4b4407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:08.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.889+0000 7f4b66a00640 1 --2- 172.21.15.39:0/756247373 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4b44078b90 0x7f4b4407b0a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4b58033000 tx=0x7f4b58004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:08.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:08.892+0000 7f4b5f400640 1 -- 172.21.15.39:0/756247373 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4b60063730 con 0x7f4b68073170 2024-09-21T07:35:09.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.089+0000 7f4b6de00640 1 -- 172.21.15.39:0/756247373 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) -- 0x7f4b680c3860 con 0x7f4b68073170 2024-09-21T07:35:09.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.090+0000 7f4b5f400640 1 -- 172.21.15.39:0/756247373 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v54) ==== 103+0+11 (secure 0 0 0) 0x7f4b60068160 con 0x7f4b68073170 2024-09-21T07:35:09.092 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-21T07:35:09.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.092+0000 7f4b6de00640 1 -- 172.21.15.39:0/756247373 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4b44078b90 msgr2=0x7f4b4407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:09.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.092+0000 7f4b6de00640 1 --2- 172.21.15.39:0/756247373 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4b44078b90 0x7f4b4407b0a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4b58033000 tx=0x7f4b58004730 comp rx=0 tx=0).stop 2024-09-21T07:35:09.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.092+0000 7f4b6de00640 1 -- 172.21.15.39:0/756247373 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b68073170 msgr2=0x7f4b6815f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:09.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.092+0000 7f4b6de00640 1 --2- 172.21.15.39:0/756247373 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b68073170 0x7f4b6815f1e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f4b6000ea40 tx=0x7f4b6000ef00 comp rx=0 tx=0).stop 2024-09-21T07:35:09.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.092+0000 7f4b6de00640 1 -- 172.21.15.39:0/756247373 shutdown_connections 2024-09-21T07:35:09.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.092+0000 7f4b6de00640 1 --2- 172.21.15.39:0/756247373 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4b44078b90 0x7f4b4407b0a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:09.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.092+0000 7f4b6de00640 1 --2- 172.21.15.39:0/756247373 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b68073170 0x7f4b6815f1e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:09.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.092+0000 7f4b6de00640 1 --2- 172.21.15.39:0/756247373 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b68070440 0x7f4b681534c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:09.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.092+0000 7f4b6de00640 1 --2- 172.21.15.39:0/756247373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b68070030 0x7f4b680cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:09.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.092+0000 7f4b6de00640 1 -- 172.21.15.39:0/756247373 >> 172.21.15.39:0/756247373 conn(0x7f4b680b1cb0 msgr2=0x7f4b6815a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:09.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.092+0000 7f4b6de00640 1 -- 172.21.15.39:0/756247373 shutdown_connections 2024-09-21T07:35:09.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.093+0000 7f4b6de00640 1 -- 172.21.15.39:0/756247373 wait complete. 2024-09-21T07:35:09.115 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_using_pool_with_snap_fails_fs_creation' 2024-09-21T07:35:09.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.340+0000 7f4b8e400640 1 -- 172.21.15.39:0/799358510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b88070440 msgr2=0x7f4b88074b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:09.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.340+0000 7f4b8e400640 1 --2- 172.21.15.39:0/799358510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b88070440 0x7f4b88074b80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4b80005fd0 tx=0x7f4b8002aa90 comp rx=0 tx=0).stop 2024-09-21T07:35:09.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.340+0000 7f4b8e400640 1 -- 172.21.15.39:0/799358510 shutdown_connections 2024-09-21T07:35:09.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.340+0000 7f4b8e400640 1 --2- 172.21.15.39:0/799358510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b88070440 0x7f4b88074b80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:09.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.340+0000 7f4b8e400640 1 --2- 172.21.15.39:0/799358510 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b88070030 0x7f4b88074640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:09.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.340+0000 7f4b8e400640 1 -- 172.21.15.39:0/799358510 >> 172.21.15.39:0/799358510 conn(0x7f4b880b1cb0 msgr2=0x7f4b880b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:09.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.340+0000 7f4b8e400640 1 -- 172.21.15.39:0/799358510 shutdown_connections 2024-09-21T07:35:09.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.340+0000 7f4b8e400640 1 -- 172.21.15.39:0/799358510 wait complete. 2024-09-21T07:35:09.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.342+0000 7f4b8e400640 1 Processor -- start 2024-09-21T07:35:09.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.342+0000 7f4b8e400640 1 -- start start 2024-09-21T07:35:09.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.344+0000 7f4b8e400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b88070030 0x7f4b8814fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:09.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.344+0000 7f4b8e400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b88070440 0x7f4b8809bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:09.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.344+0000 7f4b8e400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b88073170 0x7f4b8809caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:09.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.344+0000 7f4b8e400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4b880b3a40 con 0x7f4b88073170 2024-09-21T07:35:09.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.344+0000 7f4b8e400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4b880b38c0 con 0x7f4b88070440 2024-09-21T07:35:09.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.344+0000 7f4b8e400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f4b880b3bc0 con 0x7f4b88070030 2024-09-21T07:35:09.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.344+0000 7f4b87e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b88073170 0x7f4b8809caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:09.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.344+0000 7f4b87e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b88073170 0x7f4b8809caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49438/0 (socket says 172.21.15.39:49438) 2024-09-21T07:35:09.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.344+0000 7f4b87e00640 1 -- 172.21.15.39:0/3645232796 learned_addr learned my addr 172.21.15.39:0/3645232796 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:09.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.344+0000 7f4b87400640 1 --2- 172.21.15.39:0/3645232796 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b88070030 0x7f4b8814fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:09.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.344+0000 7f4b7ea00640 1 --2- 172.21.15.39:0/3645232796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b88070440 0x7f4b8809bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:09.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.345+0000 7f4b87e00640 1 -- 172.21.15.39:0/3645232796 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b88070030 msgr2=0x7f4b8814fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:09.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.345+0000 7f4b87e00640 1 --2- 172.21.15.39:0/3645232796 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b88070030 0x7f4b8814fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:09.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.345+0000 7f4b87e00640 1 -- 172.21.15.39:0/3645232796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b88070440 msgr2=0x7f4b8809bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:09.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.345+0000 7f4b87e00640 1 --2- 172.21.15.39:0/3645232796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b88070440 0x7f4b8809bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:09.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.345+0000 7f4b87e00640 1 -- 172.21.15.39:0/3645232796 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b88150420 con 0x7f4b88073170 2024-09-21T07:35:09.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.345+0000 7f4b87400640 1 --2- 172.21.15.39:0/3645232796 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b88070030 0x7f4b8814fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:35:09.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.345+0000 7f4b87e00640 1 --2- 172.21.15.39:0/3645232796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b88073170 0x7f4b8809caf0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f4b80009d90 tx=0x7f4b80003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:09.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.346+0000 7f4b7fe00640 1 -- 172.21.15.39:0/3645232796 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b80044070 con 0x7f4b88073170 2024-09-21T07:35:09.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.346+0000 7f4b7fe00640 1 -- 172.21.15.39:0/3645232796 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b80003e00 con 0x7f4b88073170 2024-09-21T07:35:09.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.346+0000 7f4b8e400640 1 -- 172.21.15.39:0/3645232796 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b8809ac80 con 0x7f4b88073170 2024-09-21T07:35:09.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.346+0000 7f4b7fe00640 1 -- 172.21.15.39:0/3645232796 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b800044b0 con 0x7f4b88073170 2024-09-21T07:35:09.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.346+0000 7f4b8e400640 1 -- 172.21.15.39:0/3645232796 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b8809b240 con 0x7f4b88073170 2024-09-21T07:35:09.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.347+0000 7f4b8e400640 1 -- 172.21.15.39:0/3645232796 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b880b40e0 con 0x7f4b88073170 2024-09-21T07:35:09.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.348+0000 7f4b7fe00640 1 -- 172.21.15.39:0/3645232796 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4b80047070 con 0x7f4b88073170 2024-09-21T07:35:09.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.350+0000 7f4b7fe00640 1 --2- 172.21.15.39:0/3645232796 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4b64078b90 0x7f4b6407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:09.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.350+0000 7f4b7fe00640 1 -- 172.21.15.39:0/3645232796 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5296+0+0 (secure 0 0 0) 0x7f4b800bba70 con 0x7f4b88073170 2024-09-21T07:35:09.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.350+0000 7f4b87400640 1 --2- 172.21.15.39:0/3645232796 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4b64078b90 0x7f4b6407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:09.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.351+0000 7f4b87400640 1 --2- 172.21.15.39:0/3645232796 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4b64078b90 0x7f4b6407b0a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4b78001b10 tx=0x7f4b78009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:09.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.354+0000 7f4b7fe00640 1 -- 172.21.15.39:0/3645232796 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4b80032030 con 0x7f4b88073170 2024-09-21T07:35:09.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.626+0000 7f4b8e400640 1 -- 172.21.15.39:0/3645232796 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_using_pool_with_snap_fails_fs_creation"]} v 0) -- 0x7f4b880c3860 con 0x7f4b88073170 2024-09-21T07:35:09.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.684+0000 7f4b7fe00640 1 -- 172.21.15.39:0/3645232796 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_using_pool_with_snap_fails_fs_creation"]}]=0 v128) ==== 176+0+0 (secure 0 0 0) 0x7f4b800098b0 con 0x7f4b88073170 2024-09-21T07:35:09.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.686+0000 7f4b8e400640 1 -- 172.21.15.39:0/3645232796 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4b64078b90 msgr2=0x7f4b6407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:09.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.686+0000 7f4b8e400640 1 --2- 172.21.15.39:0/3645232796 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4b64078b90 0x7f4b6407b0a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4b78001b10 tx=0x7f4b78009040 comp rx=0 tx=0).stop 2024-09-21T07:35:09.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.686+0000 7f4b8e400640 1 -- 172.21.15.39:0/3645232796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b88073170 msgr2=0x7f4b8809caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:09.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.686+0000 7f4b8e400640 1 --2- 172.21.15.39:0/3645232796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b88073170 0x7f4b8809caf0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f4b80009d90 tx=0x7f4b80003c10 comp rx=0 tx=0).stop 2024-09-21T07:35:09.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.687+0000 7f4b8e400640 1 -- 172.21.15.39:0/3645232796 shutdown_connections 2024-09-21T07:35:09.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.687+0000 7f4b8e400640 1 --2- 172.21.15.39:0/3645232796 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4b64078b90 0x7f4b6407b0a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:09.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.687+0000 7f4b8e400640 1 --2- 172.21.15.39:0/3645232796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b88073170 0x7f4b8809caf0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:09.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.687+0000 7f4b8e400640 1 --2- 172.21.15.39:0/3645232796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b88070440 0x7f4b8809bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:09.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.687+0000 7f4b8e400640 1 --2- 172.21.15.39:0/3645232796 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b88070030 0x7f4b8814fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:09.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.687+0000 7f4b8e400640 1 -- 172.21.15.39:0/3645232796 >> 172.21.15.39:0/3645232796 conn(0x7f4b880b1cb0 msgr2=0x7f4b880ae5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:09.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.687+0000 7f4b8e400640 1 -- 172.21.15.39:0/3645232796 shutdown_connections 2024-09-21T07:35:09.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.687+0000 7f4b8e400640 1 -- 172.21.15.39:0/3645232796 wait complete. 2024-09-21T07:35:09.714 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T07:35:09.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.940+0000 7fecf9600640 1 -- 172.21.15.39:0/313237150 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecf40d01f0 msgr2=0x7fecf40705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:09.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.940+0000 7fecf9600640 1 --2- 172.21.15.39:0/313237150 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecf40d01f0 0x7fecf40705a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fecf40737f0 tx=0x7fecdc02a5f0 comp rx=0 tx=0).stop 2024-09-21T07:35:09.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.940+0000 7fecf9600640 1 -- 172.21.15.39:0/313237150 shutdown_connections 2024-09-21T07:35:09.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.940+0000 7fecf9600640 1 --2- 172.21.15.39:0/313237150 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecf40d01f0 0x7fecf40705a0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:09.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.940+0000 7fecf9600640 1 --2- 172.21.15.39:0/313237150 >> v2:172.21.15.148:3300/0 conn(0x7fecf40cfde0 0x7fecf4070060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:09.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.940+0000 7fecf9600640 1 -- 172.21.15.39:0/313237150 >> 172.21.15.39:0/313237150 conn(0x7fecf40b3090 msgr2=0x7fecf40b34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:09.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.940+0000 7fecf9600640 1 -- 172.21.15.39:0/313237150 shutdown_connections 2024-09-21T07:35:09.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.940+0000 7fecf9600640 1 -- 172.21.15.39:0/313237150 wait complete. 2024-09-21T07:35:09.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.942+0000 7fecf9600640 1 Processor -- start 2024-09-21T07:35:09.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.943+0000 7fecf9600640 1 -- start start 2024-09-21T07:35:09.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.944+0000 7fecf9600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fecf40cfde0 0x7fecf41e5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:09.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.944+0000 7fecf9600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecf40d01f0 0x7fecf41453c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:09.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.944+0000 7fecf9600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecf41604f0 0x7fecf4158c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:09.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.944+0000 7fecf9600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fecf415d1b0 con 0x7fecf41604f0 2024-09-21T07:35:09.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.944+0000 7fecf9600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fecf415d030 con 0x7fecf40d01f0 2024-09-21T07:35:09.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.944+0000 7fecf9600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fecf415d330 con 0x7fecf40cfde0 2024-09-21T07:35:09.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.944+0000 7fecf3e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fecf40cfde0 0x7fecf41e5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:09.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.944+0000 7fecf3e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fecf40cfde0 0x7fecf41e5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.39:34514/0 (socket says 172.21.15.39:34514) 2024-09-21T07:35:09.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.944+0000 7fecf3e00640 1 -- 172.21.15.39:0/2395672675 learned_addr learned my addr 172.21.15.39:0/2395672675 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:09.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.944+0000 7fecf8c00640 1 --2- 172.21.15.39:0/2395672675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecf41604f0 0x7fecf4158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:09.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.944+0000 7fecebe00640 1 --2- 172.21.15.39:0/2395672675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecf40d01f0 0x7fecf41453c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:09.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.945+0000 7fecf3e00640 1 -- 172.21.15.39:0/2395672675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecf40d01f0 msgr2=0x7fecf41453c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:09.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.945+0000 7fecf3e00640 1 --2- 172.21.15.39:0/2395672675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecf40d01f0 0x7fecf41453c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:09.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.945+0000 7fecf3e00640 1 -- 172.21.15.39:0/2395672675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecf41604f0 msgr2=0x7fecf4158c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:09.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.945+0000 7fecf3e00640 1 --2- 172.21.15.39:0/2395672675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecf41604f0 0x7fecf4158c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:09.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.945+0000 7fecf3e00640 1 -- 172.21.15.39:0/2395672675 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf4146030 con 0x7fecf40cfde0 2024-09-21T07:35:09.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.945+0000 7fecf3e00640 1 --2- 172.21.15.39:0/2395672675 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fecf40cfde0 0x7fecf41e5e00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fece400e7c0 tx=0x7fece400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:09.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.946+0000 7fecf0c00640 1 -- 172.21.15.39:0/2395672675 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece400ccc0 con 0x7fecf40cfde0 2024-09-21T07:35:09.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.946+0000 7fecf0c00640 1 -- 172.21.15.39:0/2395672675 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fece400ce80 con 0x7fecf40cfde0 2024-09-21T07:35:09.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.946+0000 7fecf9600640 1 -- 172.21.15.39:0/2395672675 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecf41f73e0 con 0x7fecf40cfde0 2024-09-21T07:35:09.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.946+0000 7fecf0c00640 1 -- 172.21.15.39:0/2395672675 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece4005280 con 0x7fecf40cfde0 2024-09-21T07:35:09.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.946+0000 7fecf9600640 1 -- 172.21.15.39:0/2395672675 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fecf41f79a0 con 0x7fecf40cfde0 2024-09-21T07:35:09.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.947+0000 7fecf9600640 1 -- 172.21.15.39:0/2395672675 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecf41551a0 con 0x7fecf40cfde0 2024-09-21T07:35:09.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.948+0000 7fecf0c00640 1 -- 172.21.15.39:0/2395672675 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fece403bd10 con 0x7fecf40cfde0 2024-09-21T07:35:09.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.949+0000 7fecf0c00640 1 --2- 172.21.15.39:0/2395672675 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fecd0078b90 0x7fecd007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:09.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.949+0000 7fecf0c00640 1 -- 172.21.15.39:0/2395672675 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(54..54 src has 1..54) ==== 5296+0+0 (secure 0 0 0) 0x7fece409b5f0 con 0x7fecf40cfde0 2024-09-21T07:35:09.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.950+0000 7fecebe00640 1 --2- 172.21.15.39:0/2395672675 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fecd0078b90 0x7fecd007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:09.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.950+0000 7fecebe00640 1 --2- 172.21.15.39:0/2395672675 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fecd0078b90 0x7fecd007b0a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fecdc033000 tx=0x7fecdc004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:09.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:09.953+0000 7fecf0c00640 1 -- 172.21.15.39:0/2395672675 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fece40636a0 con 0x7fecf40cfde0 2024-09-21T07:35:10.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.149+0000 7fecf9600640 1 -- 172.21.15.39:0/2395672675 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fecf415d850 con 0x7fecf40cfde0 2024-09-21T07:35:10.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.150+0000 7fecf0c00640 1 -- 172.21.15.39:0/2395672675 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v36) ==== 59+0+2294 (secure 0 0 0) 0x7fece40680d0 con 0x7fecf40cfde0 2024-09-21T07:35:10.153 INFO:teuthology.orchestra.run.smithi039.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T07:35:10.153 INFO:teuthology.orchestra.run.smithi039.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T07:35:10.153 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced client_mount_timeout 30 2024-09-21T07:35:10.153 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_client 20/20 2024-09-21T07:35:10.153 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_mgr 20/20 2024-09-21T07:35:10.153 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_ms 1/1 2024-09-21T07:35:10.153 INFO:teuthology.orchestra.run.smithi039.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T07:35:10.153 INFO:teuthology.orchestra.run.smithi039.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T07:35:10.153 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3421.881759 2024-09-21T07:35:10.153 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3520.252978 2024-09-21T07:35:10.153 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3511.183316 2024-09-21T07:35:10.153 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3829.631847 2024-09-21T07:35:10.154 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 5474.545872 2024-09-21T07:35:10.154 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4013.628559 2024-09-21T07:35:10.154 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4230.187288 2024-09-21T07:35:10.154 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4148.983107 2024-09-21T07:35:10.154 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_mds 20/20 2024-09-21T07:35:10.154 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T07:35:10.154 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_ms 2/2 2024-09-21T07:35:10.154 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_debug_frag true 2024-09-21T07:35:10.154 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_debug_scatterstat true 2024-09-21T07:35:10.154 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T07:35:10.154 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_verify_scatter true 2024-09-21T07:35:10.154 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T07:35:10.154 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T07:35:10.154 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T07:35:10.155 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced client_mount_timeout 600 2024-09-21T07:35:10.155 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced debug_client 20/20 2024-09-21T07:35:10.155 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced debug_ms 1/1 2024-09-21T07:35:10.155 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T07:35:10.155 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T07:35:10.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.152+0000 7fecf9600640 1 -- 172.21.15.39:0/2395672675 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fecd0078b90 msgr2=0x7fecd007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:10.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.152+0000 7fecf9600640 1 --2- 172.21.15.39:0/2395672675 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fecd0078b90 0x7fecd007b0a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fecdc033000 tx=0x7fecdc004730 comp rx=0 tx=0).stop 2024-09-21T07:35:10.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.152+0000 7fecf9600640 1 -- 172.21.15.39:0/2395672675 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fecf40cfde0 msgr2=0x7fecf41e5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:10.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.152+0000 7fecf9600640 1 --2- 172.21.15.39:0/2395672675 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fecf40cfde0 0x7fecf41e5e00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fece400e7c0 tx=0x7fece400ec80 comp rx=0 tx=0).stop 2024-09-21T07:35:10.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.152+0000 7fecf9600640 1 -- 172.21.15.39:0/2395672675 shutdown_connections 2024-09-21T07:35:10.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.152+0000 7fecf9600640 1 --2- 172.21.15.39:0/2395672675 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fecd0078b90 0x7fecd007b0a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:10.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.152+0000 7fecf9600640 1 --2- 172.21.15.39:0/2395672675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecf41604f0 0x7fecf4158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:10.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.152+0000 7fecf9600640 1 --2- 172.21.15.39:0/2395672675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecf40d01f0 0x7fecf41453c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:10.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.152+0000 7fecf9600640 1 --2- 172.21.15.39:0/2395672675 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fecf40cfde0 0x7fecf41e5e00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:10.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.152+0000 7fecf9600640 1 -- 172.21.15.39:0/2395672675 >> 172.21.15.39:0/2395672675 conn(0x7fecf40b3090 msgr2=0x7fecf4160900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:10.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.153+0000 7fecf9600640 1 -- 172.21.15.39:0/2395672675 shutdown_connections 2024-09-21T07:35:10.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.153+0000 7fecf9600640 1 -- 172.21.15.39:0/2395672675 wait complete. 2024-09-21T07:35:10.180 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T07:35:10.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.407+0000 7fb66fe00640 1 -- 172.21.15.39:0/405244077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb67015fbd0 msgr2=0x7fb670152360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:10.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.407+0000 7fb66fe00640 1 --2- 172.21.15.39:0/405244077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb67015fbd0 0x7fb670152360 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb6700ccc60 tx=0x7fb66002a560 comp rx=0 tx=0).stop 2024-09-21T07:35:10.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.407+0000 7fb66fe00640 1 -- 172.21.15.39:0/405244077 shutdown_connections 2024-09-21T07:35:10.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.407+0000 7fb66fe00640 1 --2- 172.21.15.39:0/405244077 >> v2:172.21.15.148:3300/0 conn(0x7fb67015f660 0x7fb6700b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:10.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.407+0000 7fb66fe00640 1 --2- 172.21.15.39:0/405244077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb67015fbd0 0x7fb670152360 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:10.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.407+0000 7fb66fe00640 1 -- 172.21.15.39:0/405244077 >> 172.21.15.39:0/405244077 conn(0x7fb67006d410 msgr2=0x7fb6700b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:10.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.407+0000 7fb66fe00640 1 -- 172.21.15.39:0/405244077 shutdown_connections 2024-09-21T07:35:10.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.407+0000 7fb66fe00640 1 -- 172.21.15.39:0/405244077 wait complete. 2024-09-21T07:35:10.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.409+0000 7fb66fe00640 1 Processor -- start 2024-09-21T07:35:10.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.409+0000 7fb66fe00640 1 -- start start 2024-09-21T07:35:10.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.411+0000 7fb66fe00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb670072c30 0x7fb6700ca960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:10.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.411+0000 7fb66fe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6701488d0 0x7fb67015d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:10.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.411+0000 7fb66fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb670148ce0 0x7fb67015d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:10.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.411+0000 7fb66fe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb6700b3f50 con 0x7fb670148ce0 2024-09-21T07:35:10.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.411+0000 7fb66fe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb6700b3dd0 con 0x7fb6701488d0 2024-09-21T07:35:10.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.411+0000 7fb66fe00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb6700b40d0 con 0x7fb670072c30 2024-09-21T07:35:10.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.411+0000 7fb66f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb670148ce0 0x7fb67015d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:10.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.411+0000 7fb66f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb670148ce0 0x7fb67015d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49484/0 (socket says 172.21.15.39:49484) 2024-09-21T07:35:10.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.411+0000 7fb66f400640 1 -- 172.21.15.39:0/3990031513 learned_addr learned my addr 172.21.15.39:0/3990031513 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:10.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.411+0000 7fb66ea00640 1 --2- 172.21.15.39:0/3990031513 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb670072c30 0x7fb6700ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:10.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.411+0000 7fb66e000640 1 --2- 172.21.15.39:0/3990031513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6701488d0 0x7fb67015d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:10.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.412+0000 7fb66f400640 1 -- 172.21.15.39:0/3990031513 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb670072c30 msgr2=0x7fb6700ca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:10.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.412+0000 7fb66f400640 1 --2- 172.21.15.39:0/3990031513 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb670072c30 0x7fb6700ca960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:10.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.412+0000 7fb66f400640 1 -- 172.21.15.39:0/3990031513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6701488d0 msgr2=0x7fb67015d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:10.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.412+0000 7fb66f400640 1 --2- 172.21.15.39:0/3990031513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6701488d0 0x7fb67015d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:10.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.412+0000 7fb66f400640 1 -- 172.21.15.39:0/3990031513 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6701f9700 con 0x7fb670148ce0 2024-09-21T07:35:10.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.412+0000 7fb66f400640 1 --2- 172.21.15.39:0/3990031513 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb670148ce0 0x7fb67015d630 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb66800e9c0 tx=0x7fb66800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:10.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.412+0000 7fb666a00640 1 -- 172.21.15.39:0/3990031513 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb668004d20 con 0x7fb670148ce0 2024-09-21T07:35:10.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.412+0000 7fb666a00640 1 -- 172.21.15.39:0/3990031513 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb66800cb20 con 0x7fb670148ce0 2024-09-21T07:35:10.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.412+0000 7fb66fe00640 1 -- 172.21.15.39:0/3990031513 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6701f8940 con 0x7fb670148ce0 2024-09-21T07:35:10.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.412+0000 7fb666a00640 1 -- 172.21.15.39:0/3990031513 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb668005020 con 0x7fb670148ce0 2024-09-21T07:35:10.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.412+0000 7fb66fe00640 1 -- 172.21.15.39:0/3990031513 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6701f8f00 con 0x7fb670148ce0 2024-09-21T07:35:10.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.413+0000 7fb66fe00640 1 -- 172.21.15.39:0/3990031513 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6700cb2d0 con 0x7fb670148ce0 2024-09-21T07:35:10.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.415+0000 7fb666a00640 1 -- 172.21.15.39:0/3990031513 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb6680051c0 con 0x7fb670148ce0 2024-09-21T07:35:10.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.416+0000 7fb666a00640 1 --2- 172.21.15.39:0/3990031513 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb650078bc0 0x7fb65007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:10.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.416+0000 7fb666a00640 1 -- 172.21.15.39:0/3990031513 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5296+0+0 (secure 0 0 0) 0x7fb66809b440 con 0x7fb670148ce0 2024-09-21T07:35:10.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.416+0000 7fb66ea00640 1 --2- 172.21.15.39:0/3990031513 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb650078bc0 0x7fb65007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:10.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.417+0000 7fb66ea00640 1 --2- 172.21.15.39:0/3990031513 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb650078bc0 0x7fb65007b0d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb660033000 tx=0x7fb660004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:10.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.420+0000 7fb666a00640 1 -- 172.21.15.39:0/3990031513 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb6700cb2d0 con 0x7fb670148ce0 2024-09-21T07:35:10.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.613+0000 7fb66fe00640 1 -- 172.21.15.39:0/3990031513 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb67015fbd0 con 0x7fb670148ce0 2024-09-21T07:35:10.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.617+0000 7fb666a00640 1 -- 172.21.15.39:0/3990031513 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v37)=0 v37) ==== 122+0+0 (secure 0 0 0) 0x7fb668016030 con 0x7fb670148ce0 2024-09-21T07:35:10.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.620+0000 7fb66fe00640 1 -- 172.21.15.39:0/3990031513 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb650078bc0 msgr2=0x7fb65007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:10.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.620+0000 7fb66fe00640 1 --2- 172.21.15.39:0/3990031513 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb650078bc0 0x7fb65007b0d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb660033000 tx=0x7fb660004730 comp rx=0 tx=0).stop 2024-09-21T07:35:10.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.620+0000 7fb66fe00640 1 -- 172.21.15.39:0/3990031513 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb670148ce0 msgr2=0x7fb67015d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:10.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.620+0000 7fb66fe00640 1 --2- 172.21.15.39:0/3990031513 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb670148ce0 0x7fb67015d630 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb66800e9c0 tx=0x7fb66800ee80 comp rx=0 tx=0).stop 2024-09-21T07:35:10.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.620+0000 7fb66fe00640 1 -- 172.21.15.39:0/3990031513 shutdown_connections 2024-09-21T07:35:10.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.620+0000 7fb66fe00640 1 --2- 172.21.15.39:0/3990031513 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb650078bc0 0x7fb65007b0d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:10.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.620+0000 7fb66fe00640 1 --2- 172.21.15.39:0/3990031513 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb670148ce0 0x7fb67015d630 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:10.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.620+0000 7fb66fe00640 1 --2- 172.21.15.39:0/3990031513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6701488d0 0x7fb67015d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:10.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.620+0000 7fb66fe00640 1 --2- 172.21.15.39:0/3990031513 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb670072c30 0x7fb6700ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:10.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.620+0000 7fb66fe00640 1 -- 172.21.15.39:0/3990031513 >> 172.21.15.39:0/3990031513 conn(0x7fb67006d410 msgr2=0x7fb6701503d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:10.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.620+0000 7fb66fe00640 1 -- 172.21.15.39:0/3990031513 shutdown_connections 2024-09-21T07:35:10.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:10.620+0000 7fb66fe00640 1 -- 172.21.15.39:0/3990031513 wait complete. 2024-09-21T07:35:10.645 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:35:10.645 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:35:10.733 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:35:10.819 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:35:10.820 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:35:10.820 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:35:10.855 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:35:10.855 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:35:10.855 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:35:10.892 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:35:10.892 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:35:10.892 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:35:10.930 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:35:10.930 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:35:10.930 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:35:10.968 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:35:10.968 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T07:35:10.968 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:35:11.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.195+0000 7fe4ef400640 1 -- 172.21.15.39:0/3265367413 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe4e80ca800 msgr2=0x7fe4e80cba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:11.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.195+0000 7fe4ef400640 1 --2- 172.21.15.39:0/3265367413 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe4e80ca800 0x7fe4e80cba00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc009830 tx=0x7fe4dc02a510 comp rx=0 tx=0).stop 2024-09-21T07:35:11.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.195+0000 7fe4ef400640 1 -- 172.21.15.39:0/3265367413 shutdown_connections 2024-09-21T07:35:11.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.195+0000 7fe4ef400640 1 --2- 172.21.15.39:0/3265367413 >> v2:172.21.15.148:3300/0 conn(0x7fe4e80cad30 0x7fe4e806ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:11.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.195+0000 7fe4ef400640 1 --2- 172.21.15.39:0/3265367413 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe4e80ca800 0x7fe4e80cba00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:11.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.195+0000 7fe4ef400640 1 -- 172.21.15.39:0/3265367413 >> 172.21.15.39:0/3265367413 conn(0x7fe4e80b1cb0 msgr2=0x7fe4e80b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:11.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.196+0000 7fe4ef400640 1 -- 172.21.15.39:0/3265367413 shutdown_connections 2024-09-21T07:35:11.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.196+0000 7fe4ef400640 1 -- 172.21.15.39:0/3265367413 wait complete. 2024-09-21T07:35:11.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.198+0000 7fe4ef400640 1 Processor -- start 2024-09-21T07:35:11.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.198+0000 7fe4ef400640 1 -- start start 2024-09-21T07:35:11.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.199+0000 7fe4ef400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4e80704e0 0x7fe4e8157d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:11.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.200+0000 7fe4ef400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe4e80ca800 0x7fe4e81f9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:11.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.200+0000 7fe4ef400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4e80cad30 0x7fe4e81f9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:11.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.200+0000 7fe4ef400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e80ccf40 con 0x7fe4e80cad30 2024-09-21T07:35:11.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.200+0000 7fe4ef400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e80747d0 con 0x7fe4e80704e0 2024-09-21T07:35:11.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.200+0000 7fe4ef400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe4e80cd0c0 con 0x7fe4e80ca800 2024-09-21T07:35:11.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.200+0000 7fe4ed400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4e80cad30 0x7fe4e81f9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:11.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.200+0000 7fe4eca00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4e80704e0 0x7fe4e8157d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:11.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.200+0000 7fe4ed400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4e80cad30 0x7fe4e81f9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49490/0 (socket says 172.21.15.39:49490) 2024-09-21T07:35:11.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.200+0000 7fe4ed400640 1 -- 172.21.15.39:0/1209309035 learned_addr learned my addr 172.21.15.39:0/1209309035 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:11.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.200+0000 7fe4e7e00640 1 --2- 172.21.15.39:0/1209309035 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe4e80ca800 0x7fe4e81f9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:11.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.200+0000 7fe4ed400640 1 -- 172.21.15.39:0/1209309035 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe4e80ca800 msgr2=0x7fe4e81f9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:11.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.200+0000 7fe4ed400640 1 --2- 172.21.15.39:0/1209309035 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe4e80ca800 0x7fe4e81f9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:11.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.200+0000 7fe4ed400640 1 -- 172.21.15.39:0/1209309035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4e80704e0 msgr2=0x7fe4e8157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:11.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.200+0000 7fe4ed400640 1 --2- 172.21.15.39:0/1209309035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4e80704e0 0x7fe4e8157d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:11.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.200+0000 7fe4ed400640 1 -- 172.21.15.39:0/1209309035 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4e80c5c00 con 0x7fe4e80cad30 2024-09-21T07:35:11.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.201+0000 7fe4ed400640 1 --2- 172.21.15.39:0/1209309035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4e80cad30 0x7fe4e81f9580 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0001770 tx=0x7fe4e000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:11.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.201+0000 7fe4e4c00640 1 -- 172.21.15.39:0/1209309035 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0003b70 con 0x7fe4e80cad30 2024-09-21T07:35:11.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.201+0000 7fe4e4c00640 1 -- 172.21.15.39:0/1209309035 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4e0003d30 con 0x7fe4e80cad30 2024-09-21T07:35:11.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.201+0000 7fe4ef400640 1 -- 172.21.15.39:0/1209309035 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4e80c5ee0 con 0x7fe4e80cad30 2024-09-21T07:35:11.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.201+0000 7fe4e4c00640 1 -- 172.21.15.39:0/1209309035 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0005330 con 0x7fe4e80cad30 2024-09-21T07:35:11.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.201+0000 7fe4ef400640 1 -- 172.21.15.39:0/1209309035 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4e80c64a0 con 0x7fe4e80cad30 2024-09-21T07:35:11.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.202+0000 7fe4ef400640 1 -- 172.21.15.39:0/1209309035 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4e80cd650 con 0x7fe4e80cad30 2024-09-21T07:35:11.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.204+0000 7fe4e4c00640 1 -- 172.21.15.39:0/1209309035 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe4e00056e0 con 0x7fe4e80cad30 2024-09-21T07:35:11.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.205+0000 7fe4e4c00640 1 --2- 172.21.15.39:0/1209309035 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe4c8078b90 0x7fe4c807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:11.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.205+0000 7fe4e4c00640 1 -- 172.21.15.39:0/1209309035 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5296+0+0 (secure 0 0 0) 0x7fe4e009aa90 con 0x7fe4e80cad30 2024-09-21T07:35:11.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.205+0000 7fe4eca00640 1 --2- 172.21.15.39:0/1209309035 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe4c8078b90 0x7fe4c807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:11.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.206+0000 7fe4eca00640 1 --2- 172.21.15.39:0/1209309035 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe4c8078b90 0x7fe4c807b0a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc034000 tx=0x7fe4dc004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:11.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.209+0000 7fe4e4c00640 1 -- 172.21.15.39:0/1209309035 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe4e80708f0 con 0x7fe4e80cad30 2024-09-21T07:35:11.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.448+0000 7fe4ef400640 1 -- 172.21.15.39:0/1209309035 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4e8073d60 con 0x7fe4e80cad30 2024-09-21T07:35:11.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.448+0000 7fe4e4c00640 1 -- 172.21.15.39:0/1209309035 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+3908 (secure 0 0 0) 0x7fe4e00629a0 con 0x7fe4e80cad30 2024-09-21T07:35:11.450 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:35:11.450 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":28,"btime":"2024-09-21T07:35:03:645446+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4958,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/878081219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":878081219},{"type":"v1","addr":"172.21.15.148:6835","nonce":878081219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3340238606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3340238606},{"type":"v1","addr":"172.21.15.148:6837","nonce":3340238606}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5307,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/929230612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":929230612},{"type":"v1","addr":"172.21.15.39:6834","nonce":929230612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5562,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/367861258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":367861258},{"type":"v1","addr":"172.21.15.39:6835","nonce":367861258}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[]} 2024-09-21T07:35:11.451 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 28 2024-09-21T07:35:11.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.451+0000 7fe4ef400640 1 -- 172.21.15.39:0/1209309035 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe4c8078b90 msgr2=0x7fe4c807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:11.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.451+0000 7fe4ef400640 1 --2- 172.21.15.39:0/1209309035 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe4c8078b90 0x7fe4c807b0a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc034000 tx=0x7fe4dc004730 comp rx=0 tx=0).stop 2024-09-21T07:35:11.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.451+0000 7fe4ef400640 1 -- 172.21.15.39:0/1209309035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4e80cad30 msgr2=0x7fe4e81f9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:11.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.451+0000 7fe4ef400640 1 --2- 172.21.15.39:0/1209309035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4e80cad30 0x7fe4e81f9580 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0001770 tx=0x7fe4e000cb90 comp rx=0 tx=0).stop 2024-09-21T07:35:11.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.451+0000 7fe4ef400640 1 -- 172.21.15.39:0/1209309035 shutdown_connections 2024-09-21T07:35:11.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.451+0000 7fe4ef400640 1 --2- 172.21.15.39:0/1209309035 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe4c8078b90 0x7fe4c807b0a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:11.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.451+0000 7fe4ef400640 1 --2- 172.21.15.39:0/1209309035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4e80cad30 0x7fe4e81f9580 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:11.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.451+0000 7fe4ef400640 1 --2- 172.21.15.39:0/1209309035 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe4e80ca800 0x7fe4e81f9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:11.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.451+0000 7fe4ef400640 1 --2- 172.21.15.39:0/1209309035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4e80704e0 0x7fe4e8157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:11.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.451+0000 7fe4ef400640 1 -- 172.21.15.39:0/1209309035 >> 172.21.15.39:0/1209309035 conn(0x7fe4e80b1cb0 msgr2=0x7fe4e80b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:11.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.451+0000 7fe4ef400640 1 -- 172.21.15.39:0/1209309035 shutdown_connections 2024-09-21T07:35:11.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.451+0000 7fe4ef400640 1 -- 172.21.15.39:0/1209309035 wait complete. 2024-09-21T07:35:11.474 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T07:35:11.475 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T07:35:11.475 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T07:35:11.475 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:35:11.475 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T07:35:11.475 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T07:35:11.475 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T07:35:11.476 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:35:11.476 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T07:35:11.476 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T07:35:11.476 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T07:35:11.476 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:35:11.476 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T07:35:11.476 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T07:35:11.477 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T07:35:11.477 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:35:11.499 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T07:35:11.499 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T07:35:11.500 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T07:35:11.500 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T07:35:11.501 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T07:35:11.501 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T07:35:11.502 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T07:35:11.502 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T07:35:11.544 INFO:tasks.ceph.mds.c:Started 2024-09-21T07:35:11.544 DEBUG:teuthology.parallel:result is None 2024-09-21T07:35:11.545 INFO:tasks.ceph.mds.b:Started 2024-09-21T07:35:11.545 DEBUG:teuthology.parallel:result is None 2024-09-21T07:35:11.546 INFO:tasks.ceph.mds.a:Started 2024-09-21T07:35:11.546 DEBUG:teuthology.parallel:result is None 2024-09-21T07:35:11.547 INFO:tasks.ceph.mds.d:Started 2024-09-21T07:35:11.547 DEBUG:teuthology.parallel:result is None 2024-09-21T07:35:11.548 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T07:35:11.744 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-21T07:35:11.744 INFO:tasks.ceph.mds.c.smithi039.stdout:starting mds.c at 2024-09-21T07:35:11.745 INFO:tasks.ceph.mds.b.smithi148.stdout:starting mds.b at 2024-09-21T07:35:11.745 INFO:tasks.ceph.mds.d.smithi148.stdout:starting mds.d at 2024-09-21T07:35:11.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.786+0000 7f8182000640 1 -- 172.21.15.39:0/3765990024 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f817c162470 msgr2=0x7f817c162880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:11.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.786+0000 7f8182000640 1 --2- 172.21.15.39:0/3765990024 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f817c162470 0x7f817c162880 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8174005fd0 tx=0x7f817402a8b0 comp rx=0 tx=0).stop 2024-09-21T07:35:11.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.786+0000 7f8182000640 1 -- 172.21.15.39:0/3765990024 shutdown_connections 2024-09-21T07:35:11.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.786+0000 7f8182000640 1 --2- 172.21.15.39:0/3765990024 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f817c162470 0x7f817c162880 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:11.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.786+0000 7f8182000640 1 --2- 172.21.15.39:0/3765990024 >> v2:172.21.15.148:3300/0 conn(0x7f817c0b5a20 0x7f817c0b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:11.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.786+0000 7f8182000640 1 -- 172.21.15.39:0/3765990024 >> 172.21.15.39:0/3765990024 conn(0x7f817c09d6a0 msgr2=0x7f817c0d43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:11.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.786+0000 7f8182000640 1 -- 172.21.15.39:0/3765990024 shutdown_connections 2024-09-21T07:35:11.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.786+0000 7f8182000640 1 -- 172.21.15.39:0/3765990024 wait complete. 2024-09-21T07:35:11.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.788+0000 7f8182000640 1 Processor -- start 2024-09-21T07:35:11.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.789+0000 7f8182000640 1 -- start start 2024-09-21T07:35:11.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.790+0000 7f8182000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f817c0b5a20 0x7f817c14d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:11.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.790+0000 7f8182000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f817c162470 0x7f817c14fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:11.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.790+0000 7f8182000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f817c163110 0x7f817c150150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:11.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.790+0000 7f8182000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f817c0cfdb0 con 0x7f817c162470 2024-09-21T07:35:11.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.790+0000 7f8182000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f817c0cfc30 con 0x7f817c0b5a20 2024-09-21T07:35:11.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.790+0000 7f8182000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f817c0cff30 con 0x7f817c163110 2024-09-21T07:35:11.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.790+0000 7f817be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f817c162470 0x7f817c14fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:11.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.790+0000 7f817be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f817c162470 0x7f817c14fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49542/0 (socket says 172.21.15.39:49542) 2024-09-21T07:35:11.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.790+0000 7f817be00640 1 -- 172.21.15.39:0/2530946190 learned_addr learned my addr 172.21.15.39:0/2530946190 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:11.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.790+0000 7f8180c00640 1 --2- 172.21.15.39:0/2530946190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f817c0b5a20 0x7f817c14d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:11.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.791+0000 7f8181600640 1 --2- 172.21.15.39:0/2530946190 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f817c163110 0x7f817c150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:11.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.791+0000 7f817be00640 1 -- 172.21.15.39:0/2530946190 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f817c163110 msgr2=0x7f817c150150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:11.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.791+0000 7f817be00640 1 --2- 172.21.15.39:0/2530946190 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f817c163110 0x7f817c150150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:11.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.791+0000 7f817be00640 1 -- 172.21.15.39:0/2530946190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f817c0b5a20 msgr2=0x7f817c14d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:11.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.791+0000 7f817be00640 1 --2- 172.21.15.39:0/2530946190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f817c0b5a20 0x7f817c14d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:11.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.791+0000 7f817be00640 1 -- 172.21.15.39:0/2530946190 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f817c14d880 con 0x7f817c162470 2024-09-21T07:35:11.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.791+0000 7f817be00640 1 --2- 172.21.15.39:0/2530946190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f817c162470 0x7f817c14fc10 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f816800e9f0 tx=0x7f816800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:11.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.792+0000 7f8178c00640 1 -- 172.21.15.39:0/2530946190 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816800cd80 con 0x7f817c162470 2024-09-21T07:35:11.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.792+0000 7f8178c00640 1 -- 172.21.15.39:0/2530946190 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8168004020 con 0x7f817c162470 2024-09-21T07:35:11.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.792+0000 7f8182000640 1 -- 172.21.15.39:0/2530946190 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f817c14dc20 con 0x7f817c162470 2024-09-21T07:35:11.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.792+0000 7f8178c00640 1 -- 172.21.15.39:0/2530946190 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8168004470 con 0x7f817c162470 2024-09-21T07:35:11.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.792+0000 7f8182000640 1 -- 172.21.15.39:0/2530946190 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f817c14e1e0 con 0x7f817c162470 2024-09-21T07:35:11.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.793+0000 7f8182000640 1 -- 172.21.15.39:0/2530946190 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f817c0d0450 con 0x7f817c162470 2024-09-21T07:35:11.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.794+0000 7f8178c00640 1 -- 172.21.15.39:0/2530946190 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f81680035f0 con 0x7f817c162470 2024-09-21T07:35:11.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.795+0000 7f8178c00640 1 --2- 172.21.15.39:0/2530946190 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f815c078b90 0x7f815c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:11.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.795+0000 7f8178c00640 1 -- 172.21.15.39:0/2530946190 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5296+0+0 (secure 0 0 0) 0x7f816809b6a0 con 0x7f817c162470 2024-09-21T07:35:11.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.795+0000 7f8180c00640 1 --2- 172.21.15.39:0/2530946190 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f815c078b90 0x7f815c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:11.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.796+0000 7f8180c00640 1 --2- 172.21.15.39:0/2530946190 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f815c078b90 0x7f815c07b0a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8170001750 tx=0x7f8170009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:11.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:11.800+0000 7f8178c00640 1 -- 172.21.15.39:0/2530946190 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8168063530 con 0x7f817c162470 2024-09-21T07:35:12.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.010+0000 7f8182000640 1 -- 172.21.15.39:0/2530946190 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f817c161ed0 con 0x7f817c162470 2024-09-21T07:35:12.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.715+0000 7f8178c00640 1 -- 172.21.15.39:0/2530946190 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v56) ==== 97+0+0 (secure 0 0 0) 0x7f8168067f60 con 0x7f817c162470 2024-09-21T07:35:12.718 INFO:teuthology.orchestra.run.smithi039.stderr: removed all blocklist entries 2024-09-21T07:35:12.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.719+0000 7f8182000640 1 -- 172.21.15.39:0/2530946190 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f815c078b90 msgr2=0x7f815c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:12.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.719+0000 7f8182000640 1 --2- 172.21.15.39:0/2530946190 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f815c078b90 0x7f815c07b0a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8170001750 tx=0x7f8170009040 comp rx=0 tx=0).stop 2024-09-21T07:35:12.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.719+0000 7f8182000640 1 -- 172.21.15.39:0/2530946190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f817c162470 msgr2=0x7f817c14fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:12.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.719+0000 7f8182000640 1 --2- 172.21.15.39:0/2530946190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f817c162470 0x7f817c14fc10 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f816800e9f0 tx=0x7f816800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:35:12.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.719+0000 7f8182000640 1 -- 172.21.15.39:0/2530946190 shutdown_connections 2024-09-21T07:35:12.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.719+0000 7f8182000640 1 --2- 172.21.15.39:0/2530946190 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f815c078b90 0x7f815c07b0a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:12.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.719+0000 7f8182000640 1 --2- 172.21.15.39:0/2530946190 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f817c163110 0x7f817c150150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:12.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.719+0000 7f8182000640 1 --2- 172.21.15.39:0/2530946190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f817c162470 0x7f817c14fc10 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:12.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.719+0000 7f8182000640 1 --2- 172.21.15.39:0/2530946190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f817c0b5a20 0x7f817c14d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:12.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.719+0000 7f8182000640 1 -- 172.21.15.39:0/2530946190 >> 172.21.15.39:0/2530946190 conn(0x7f817c09d6a0 msgr2=0x7f817c06e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:12.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.719+0000 7f8182000640 1 -- 172.21.15.39:0/2530946190 shutdown_connections 2024-09-21T07:35:12.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.719+0000 7f8182000640 1 -- 172.21.15.39:0/2530946190 wait complete. 2024-09-21T07:35:12.752 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T07:35:12.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.980+0000 7f6286200640 1 -- 172.21.15.39:0/3452072670 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6280151640 msgr2=0x7f628006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:12.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.980+0000 7f6286200640 1 --2- 172.21.15.39:0/3452072670 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6280151640 0x7f628006ee70 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f62800b2c50 tx=0x7f626c02a540 comp rx=0 tx=0).stop 2024-09-21T07:35:12.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.980+0000 7f6286200640 1 -- 172.21.15.39:0/3452072670 shutdown_connections 2024-09-21T07:35:12.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.980+0000 7f6286200640 1 --2- 172.21.15.39:0/3452072670 >> v2:172.21.15.148:3300/0 conn(0x7f628006f3b0 0x7f62800cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:12.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.980+0000 7f6286200640 1 --2- 172.21.15.39:0/3452072670 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6280151640 0x7f628006ee70 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:12.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.980+0000 7f6286200640 1 --2- 172.21.15.39:0/3452072670 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f628015b8f0 0x7f6280073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:12.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.980+0000 7f6286200640 1 -- 172.21.15.39:0/3452072670 >> 172.21.15.39:0/3452072670 conn(0x7f628006d410 msgr2=0x7f62800b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:12.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.980+0000 7f6286200640 1 -- 172.21.15.39:0/3452072670 shutdown_connections 2024-09-21T07:35:12.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.980+0000 7f6286200640 1 -- 172.21.15.39:0/3452072670 wait complete. 2024-09-21T07:35:12.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.982+0000 7f6286200640 1 Processor -- start 2024-09-21T07:35:12.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.982+0000 7f6286200640 1 -- start start 2024-09-21T07:35:12.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.984+0000 7f6286200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f628006f3b0 0x7f628015d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:12.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.984+0000 7f6286200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6280151640 0x7f62801f5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:12.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.984+0000 7f6286200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f628015b8f0 0x7f628014ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:12.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.984+0000 7f6286200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f62800cf5c0 con 0x7f628015b8f0 2024-09-21T07:35:12.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.984+0000 7f6286200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6280070f70 con 0x7f6280151640 2024-09-21T07:35:12.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.984+0000 7f6286200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f62800cf740 con 0x7f628006f3b0 2024-09-21T07:35:12.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.984+0000 7f6285800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f628015b8f0 0x7f628014ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:12.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.984+0000 7f6285800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f628015b8f0 0x7f628014ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46446/0 (socket says 172.21.15.39:46446) 2024-09-21T07:35:12.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.984+0000 7f6285800640 1 -- 172.21.15.39:0/1169966214 learned_addr learned my addr 172.21.15.39:0/1169966214 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:12.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.984+0000 7f6284e00640 1 --2- 172.21.15.39:0/1169966214 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f628006f3b0 0x7f628015d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:12.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.984+0000 7f627fe00640 1 --2- 172.21.15.39:0/1169966214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6280151640 0x7f62801f5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:12.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.985+0000 7f6285800640 1 -- 172.21.15.39:0/1169966214 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f628006f3b0 msgr2=0x7f628015d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:12.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.985+0000 7f6285800640 1 --2- 172.21.15.39:0/1169966214 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f628006f3b0 0x7f628015d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:12.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.985+0000 7f6285800640 1 -- 172.21.15.39:0/1169966214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6280151640 msgr2=0x7f62801f5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:12.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.985+0000 7f6285800640 1 --2- 172.21.15.39:0/1169966214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6280151640 0x7f62801f5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:12.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.985+0000 7f6285800640 1 -- 172.21.15.39:0/1169966214 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f628015c530 con 0x7f628015b8f0 2024-09-21T07:35:12.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.985+0000 7f6285800640 1 --2- 172.21.15.39:0/1169966214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f628015b8f0 0x7f628014ed50 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f627800e8e0 tx=0x7f627800eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:12.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.985+0000 7f627cc00640 1 -- 172.21.15.39:0/1169966214 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6278004d20 con 0x7f628015b8f0 2024-09-21T07:35:12.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.985+0000 7f627cc00640 1 -- 172.21.15.39:0/1169966214 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f627800ca60 con 0x7f628015b8f0 2024-09-21T07:35:12.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.985+0000 7f627cc00640 1 -- 172.21.15.39:0/1169966214 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6278005020 con 0x7f628015b8f0 2024-09-21T07:35:12.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.985+0000 7f6286200640 1 -- 172.21.15.39:0/1169966214 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f628015c810 con 0x7f628015b8f0 2024-09-21T07:35:12.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.985+0000 7f6286200640 1 -- 172.21.15.39:0/1169966214 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62801f8940 con 0x7f628015b8f0 2024-09-21T07:35:12.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.986+0000 7f6286200640 1 -- 172.21.15.39:0/1169966214 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62800cfbf0 con 0x7f628015b8f0 2024-09-21T07:35:12.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.988+0000 7f627cc00640 1 -- 172.21.15.39:0/1169966214 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f62780051c0 con 0x7f628015b8f0 2024-09-21T07:35:12.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.988+0000 7f627cc00640 1 --2- 172.21.15.39:0/1169966214 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6260078bc0 0x7f626007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:12.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.989+0000 7f627cc00640 1 -- 172.21.15.39:0/1169966214 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5210+0+0 (secure 0 0 0) 0x7f627809b550 con 0x7f628015b8f0 2024-09-21T07:35:12.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.989+0000 7f6284e00640 1 --2- 172.21.15.39:0/1169966214 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6260078bc0 0x7f626007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:12.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.990+0000 7f6284e00640 1 --2- 172.21.15.39:0/1169966214 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6260078bc0 0x7f626007b0d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6274005fd0 tx=0x7f627400b3e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:12.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:12.993+0000 7f627cc00640 1 -- 172.21.15.39:0/1169966214 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6278016030 con 0x7f628015b8f0 2024-09-21T07:35:13.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.267+0000 7f6286200640 1 -- 172.21.15.39:0/1169966214 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f62800b4760 con 0x7f628015b8f0 2024-09-21T07:35:13.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.267+0000 7f627cc00640 1 -- 172.21.15.39:0/1169966214 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v10) ==== 80+0+5304 (secure 0 0 0) 0x7f6278063490 con 0x7f628015b8f0 2024-09-21T07:35:13.269 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:35:13.270 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:35:13.270 INFO:teuthology.orchestra.run.smithi039.stdout: "auth_dump": [ 2024-09-21T07:35:13.270 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:13.270 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.a", 2024-09-21T07:35:13.270 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5m+P6EERAABQqmlkrocSi1+hwc73Adrg==", 2024-09-21T07:35:13.270 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:35:13.270 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:35:13.270 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:35:13.270 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:35:13.270 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:35:13.270 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:13.270 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:35:13.271 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:13.271 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.b", 2024-09-21T07:35:13.271 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5m5/GeLRAAfE8fr5dbitQa89rNK9pZ2Q==", 2024-09-21T07:35:13.271 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:35:13.271 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:35:13.271 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:35:13.271 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:35:13.271 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:35:13.271 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:13.271 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:35:13.271 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:13.271 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.c", 2024-09-21T07:35:13.271 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5msbx6HxAALsF8ORvIoJpIAFZ5uxmOEw==", 2024-09-21T07:35:13.271 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:35:13.272 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:35:13.272 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:35:13.272 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:35:13.272 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:35:13.272 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:13.272 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:35:13.272 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:13.272 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.d", 2024-09-21T07:35:13.272 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mIrcBABAApKAB/7FkZ5glNTxlbkIsBw==", 2024-09-21T07:35:13.272 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:35:13.272 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:35:13.272 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:35:13.272 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:35:13.272 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:35:13.272 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:13.273 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:35:13.273 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:13.273 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.0", 2024-09-21T07:35:13.273 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB7du5msw81DhAAmVimqN4x9K3YpT0nbovppQ==", 2024-09-21T07:35:13.273 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:35:13.273 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:35:13.273 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:35:13.273 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:35:13.273 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:13.273 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:35:13.273 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:13.273 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.1", 2024-09-21T07:35:13.273 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB+du5mdnCIOhAAtVB4ThehQwb0zRriyKcaxA==", 2024-09-21T07:35:13.273 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:35:13.274 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:35:13.274 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:35:13.274 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:35:13.274 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:13.274 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:35:13.274 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:13.274 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.2", 2024-09-21T07:35:13.274 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCCdu5mTfWxDhAAadJCbPN5xBx/f9L0v2yTdg==", 2024-09-21T07:35:13.274 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:35:13.274 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:35:13.274 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:35:13.274 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:35:13.274 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:13.274 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:35:13.275 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:13.275 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.3", 2024-09-21T07:35:13.275 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCFdu5mL1E2HxAAyOhY97uUXEw5L1CcDeONVA==", 2024-09-21T07:35:13.275 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:35:13.275 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:35:13.275 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:35:13.275 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:35:13.275 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:13.275 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:35:13.275 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:13.275 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.4", 2024-09-21T07:35:13.275 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCLdu5m6PY/IhAAgvcTgm+6uWA7lIG4tcTIEg==", 2024-09-21T07:35:13.275 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:35:13.275 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:35:13.275 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:35:13.276 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:35:13.276 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:13.276 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:35:13.276 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:13.276 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.5", 2024-09-21T07:35:13.276 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCOdu5mE/wlMxAA3JBg4xH+N/PPC1Uv2ZVrYQ==", 2024-09-21T07:35:13.276 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:35:13.276 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:35:13.276 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:35:13.276 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:35:13.276 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:13.276 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:35:13.276 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:13.277 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.6", 2024-09-21T07:35:13.277 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCSdu5mC3TgBxAAwsArjeIdFcApY3/1OJffEg==", 2024-09-21T07:35:13.277 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:35:13.277 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:35:13.277 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:35:13.277 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:35:13.277 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:13.277 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:35:13.277 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:13.277 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.7", 2024-09-21T07:35:13.277 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCVdu5mSWHWFBAA9qPHWRdsvrB74inrp1lj4Q==", 2024-09-21T07:35:13.277 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:35:13.277 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:35:13.277 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:35:13.277 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:35:13.278 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:13.278 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:35:13.278 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:13.278 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.0", 2024-09-21T07:35:13.278 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mGfb1DBAA/aSMKR5or3xy53rB/iUjpA==", 2024-09-21T07:35:13.278 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:35:13.278 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:35:13.278 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow r", 2024-09-21T07:35:13.278 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T07:35:13.278 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:13.278 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:35:13.278 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:13.278 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.1", 2024-09-21T07:35:13.278 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mEkciIhAAid+bxJEr4P7vMnZ42PAy5w==", 2024-09-21T07:35:13.279 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:35:13.279 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:35:13.279 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow r", 2024-09-21T07:35:13.279 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow rw", 2024-09-21T07:35:13.279 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rwx" 2024-09-21T07:35:13.279 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:13.279 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:35:13.279 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:13.279 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.2", 2024-09-21T07:35:13.279 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mSu2oFxAAgjy2m/wWKh6xaoh+YDoU1Q==", 2024-09-21T07:35:13.279 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:35:13.279 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:35:13.279 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow r", 2024-09-21T07:35:13.279 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow rw", 2024-09-21T07:35:13.279 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rwx" 2024-09-21T07:35:13.280 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:13.280 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:35:13.280 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:13.280 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.3", 2024-09-21T07:35:13.280 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mCHy0LBAAZneF2jUWgfuyGehZFbOxDg==", 2024-09-21T07:35:13.280 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:35:13.280 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:35:13.280 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow r", 2024-09-21T07:35:13.280 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow rw", 2024-09-21T07:35:13.280 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rwx" 2024-09-21T07:35:13.280 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:13.280 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:35:13.280 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:13.280 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.admin", 2024-09-21T07:35:13.281 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBydu5m9LInERAAAlDpqb94f2Qb9VYJJwTBcA==", 2024-09-21T07:35:13.281 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:35:13.281 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow *", 2024-09-21T07:35:13.281 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:35:13.281 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:35:13.281 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:35:13.281 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:13.281 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:35:13.281 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:13.281 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mgr.x", 2024-09-21T07:35:13.281 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5mA2GRBhAAkyCKdgH6laHy8xNt080dXg==", 2024-09-21T07:35:13.281 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:35:13.281 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow *", 2024-09-21T07:35:13.281 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile mgr", 2024-09-21T07:35:13.282 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:35:13.282 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:13.282 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:35:13.282 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:13.282 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mgr.y", 2024-09-21T07:35:13.282 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBydu5mr+8aNxAA7TFYNms2WsXUfa4ITTJOHw==", 2024-09-21T07:35:13.282 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:35:13.282 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow *", 2024-09-21T07:35:13.282 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile mgr", 2024-09-21T07:35:13.282 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:35:13.282 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:13.282 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:13.282 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-21T07:35:13.282 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:35:13.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.270+0000 7f6286200640 1 -- 172.21.15.39:0/1169966214 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6260078bc0 msgr2=0x7f626007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:13.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.270+0000 7f6286200640 1 --2- 172.21.15.39:0/1169966214 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6260078bc0 0x7f626007b0d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6274005fd0 tx=0x7f627400b3e0 comp rx=0 tx=0).stop 2024-09-21T07:35:13.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.270+0000 7f6286200640 1 -- 172.21.15.39:0/1169966214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f628015b8f0 msgr2=0x7f628014ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:13.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.270+0000 7f6286200640 1 --2- 172.21.15.39:0/1169966214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f628015b8f0 0x7f628014ed50 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f627800e8e0 tx=0x7f627800eda0 comp rx=0 tx=0).stop 2024-09-21T07:35:13.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.270+0000 7f6286200640 1 -- 172.21.15.39:0/1169966214 shutdown_connections 2024-09-21T07:35:13.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.270+0000 7f6286200640 1 --2- 172.21.15.39:0/1169966214 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6260078bc0 0x7f626007b0d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:13.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.270+0000 7f6286200640 1 --2- 172.21.15.39:0/1169966214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f628015b8f0 0x7f628014ed50 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:13.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.270+0000 7f6286200640 1 --2- 172.21.15.39:0/1169966214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6280151640 0x7f62801f5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:13.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.270+0000 7f6286200640 1 --2- 172.21.15.39:0/1169966214 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f628006f3b0 0x7f628015d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:13.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.270+0000 7f6286200640 1 -- 172.21.15.39:0/1169966214 >> 172.21.15.39:0/1169966214 conn(0x7f628006d410 msgr2=0x7f62800cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:13.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.270+0000 7f6286200640 1 -- 172.21.15.39:0/1169966214 shutdown_connections 2024-09-21T07:35:13.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.270+0000 7f6286200640 1 -- 172.21.15.39:0/1169966214 wait complete. 2024-09-21T07:35:13.294 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T07:35:13.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.519+0000 7efda3e00640 1 -- 172.21.15.39:0/4134155943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd9c0cd3e0 msgr2=0x7efd9c070180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:13.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.519+0000 7efda3e00640 1 --2- 172.21.15.39:0/4134155943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd9c0cd3e0 0x7efd9c070180 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7efd88009830 tx=0x7efd8802a480 comp rx=0 tx=0).stop 2024-09-21T07:35:13.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.519+0000 7efda3e00640 1 -- 172.21.15.39:0/4134155943 shutdown_connections 2024-09-21T07:35:13.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.519+0000 7efda3e00640 1 --2- 172.21.15.39:0/4134155943 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efd9c0706c0 0x7efd9c072ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:13.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.519+0000 7efda3e00640 1 --2- 172.21.15.39:0/4134155943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd9c0cd3e0 0x7efd9c070180 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:13.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.519+0000 7efda3e00640 1 -- 172.21.15.39:0/4134155943 >> 172.21.15.39:0/4134155943 conn(0x7efd9c06d7d0 msgr2=0x7efd9c0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:13.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.519+0000 7efda3e00640 1 -- 172.21.15.39:0/4134155943 shutdown_connections 2024-09-21T07:35:13.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.519+0000 7efda3e00640 1 -- 172.21.15.39:0/4134155943 wait complete. 2024-09-21T07:35:13.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.522+0000 7efda3e00640 1 Processor -- start 2024-09-21T07:35:13.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.522+0000 7efda3e00640 1 -- start start 2024-09-21T07:35:13.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.523+0000 7efda3e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd9c0706c0 0x7efd9c15d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:13.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.523+0000 7efda3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd9c0af780 0x7efd9c1482f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:13.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.523+0000 7efda3e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efd9c0cd3e0 0x7efd9c148830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:13.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.523+0000 7efda3e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efd9c073b80 con 0x7efd9c0af780 2024-09-21T07:35:13.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.523+0000 7efda3e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efd9c073a00 con 0x7efd9c0706c0 2024-09-21T07:35:13.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.523+0000 7efda3e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7efd9c14ed70 con 0x7efd9c0cd3e0 2024-09-21T07:35:13.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.523+0000 7efda0a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd9c0af780 0x7efd9c1482f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:13.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.523+0000 7efda0a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd9c0af780 0x7efd9c1482f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46460/0 (socket says 172.21.15.39:46460) 2024-09-21T07:35:13.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.524+0000 7efda0a00640 1 -- 172.21.15.39:0/2106566768 learned_addr learned my addr 172.21.15.39:0/2106566768 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:13.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.524+0000 7efda1400640 1 --2- 172.21.15.39:0/2106566768 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd9c0706c0 0x7efd9c15d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:13.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.524+0000 7efda1e00640 1 --2- 172.21.15.39:0/2106566768 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efd9c0cd3e0 0x7efd9c148830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:13.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.524+0000 7efda0a00640 1 -- 172.21.15.39:0/2106566768 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efd9c0cd3e0 msgr2=0x7efd9c148830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:13.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.524+0000 7efda0a00640 1 --2- 172.21.15.39:0/2106566768 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efd9c0cd3e0 0x7efd9c148830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:13.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.524+0000 7efda0a00640 1 -- 172.21.15.39:0/2106566768 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd9c0706c0 msgr2=0x7efd9c15d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:13.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.524+0000 7efda0a00640 1 --2- 172.21.15.39:0/2106566768 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd9c0706c0 0x7efd9c15d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:13.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.524+0000 7efda0a00640 1 -- 172.21.15.39:0/2106566768 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd9c15d830 con 0x7efd9c0af780 2024-09-21T07:35:13.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.524+0000 7efda0a00640 1 --2- 172.21.15.39:0/2106566768 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd9c0af780 0x7efd9c1482f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7efd88009950 tx=0x7efd88033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:13.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.524+0000 7efd99600640 1 -- 172.21.15.39:0/2106566768 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd88005c90 con 0x7efd9c0af780 2024-09-21T07:35:13.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.525+0000 7efd99600640 1 -- 172.21.15.39:0/2106566768 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd88031e50 con 0x7efd9c0af780 2024-09-21T07:35:13.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.525+0000 7efd99600640 1 -- 172.21.15.39:0/2106566768 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd88005260 con 0x7efd9c0af780 2024-09-21T07:35:13.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.525+0000 7efda3e00640 1 -- 172.21.15.39:0/2106566768 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd9c1f9200 con 0x7efd9c0af780 2024-09-21T07:35:13.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.525+0000 7efda3e00640 1 -- 172.21.15.39:0/2106566768 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd9c1f97c0 con 0x7efd9c0af780 2024-09-21T07:35:13.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.526+0000 7efda3e00640 1 -- 172.21.15.39:0/2106566768 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd9c09dd10 con 0x7efd9c0af780 2024-09-21T07:35:13.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.527+0000 7efd99600640 1 -- 172.21.15.39:0/2106566768 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7efd88005400 con 0x7efd9c0af780 2024-09-21T07:35:13.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.528+0000 7efd99600640 1 --2- 172.21.15.39:0/2106566768 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efd7c078bc0 0x7efd7c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:13.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.528+0000 7efd99600640 1 -- 172.21.15.39:0/2106566768 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5210+0+0 (secure 0 0 0) 0x7efd880bbbd0 con 0x7efd9c0af780 2024-09-21T07:35:13.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.529+0000 7efda1400640 1 --2- 172.21.15.39:0/2106566768 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efd7c078bc0 0x7efd7c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:13.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.530+0000 7efda1400640 1 --2- 172.21.15.39:0/2106566768 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efd7c078bc0 0x7efd7c07b0d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7efd90000f60 tx=0x7efd90009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:13.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.533+0000 7efd99600640 1 -- 172.21.15.39:0/2106566768 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7efd9c09dd10 con 0x7efd9c0af780 2024-09-21T07:35:13.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.733+0000 7efda3e00640 1 -- 172.21.15.39:0/2106566768 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7efd9c072ff0 con 0x7efd9c0af780 2024-09-21T07:35:13.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.734+0000 7efd99600640 1 -- 172.21.15.39:0/2106566768 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v58) ==== 84+0+137 (secure 0 0 0) 0x7efd8803f030 con 0x7efd9c0af780 2024-09-21T07:35:13.736 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:35:13.736 INFO:teuthology.orchestra.run.smithi039.stdout:[ 2024-09-21T07:35:13.736 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:13.736 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 1, 2024-09-21T07:35:13.736 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": ".mgr" 2024-09-21T07:35:13.736 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:35:13.736 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:13.736 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 6, 2024-09-21T07:35:13.736 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": "snap-test-pool" 2024-09-21T07:35:13.736 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:13.736 INFO:teuthology.orchestra.run.smithi039.stdout:] 2024-09-21T07:35:13.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.736+0000 7efda3e00640 1 -- 172.21.15.39:0/2106566768 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efd7c078bc0 msgr2=0x7efd7c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:13.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.736+0000 7efda3e00640 1 --2- 172.21.15.39:0/2106566768 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efd7c078bc0 0x7efd7c07b0d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7efd90000f60 tx=0x7efd90009040 comp rx=0 tx=0).stop 2024-09-21T07:35:13.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.736+0000 7efda3e00640 1 -- 172.21.15.39:0/2106566768 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd9c0af780 msgr2=0x7efd9c1482f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:13.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.736+0000 7efda3e00640 1 --2- 172.21.15.39:0/2106566768 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd9c0af780 0x7efd9c1482f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7efd88009950 tx=0x7efd88033c20 comp rx=0 tx=0).stop 2024-09-21T07:35:13.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.737+0000 7efda3e00640 1 -- 172.21.15.39:0/2106566768 shutdown_connections 2024-09-21T07:35:13.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.737+0000 7efda3e00640 1 --2- 172.21.15.39:0/2106566768 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efd7c078bc0 0x7efd7c07b0d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:13.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.737+0000 7efda3e00640 1 --2- 172.21.15.39:0/2106566768 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efd9c0cd3e0 0x7efd9c148830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:13.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.737+0000 7efda3e00640 1 --2- 172.21.15.39:0/2106566768 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd9c0af780 0x7efd9c1482f0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:13.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.737+0000 7efda3e00640 1 --2- 172.21.15.39:0/2106566768 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd9c0706c0 0x7efd9c15d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:13.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.737+0000 7efda3e00640 1 -- 172.21.15.39:0/2106566768 >> 172.21.15.39:0/2106566768 conn(0x7efd9c06d7d0 msgr2=0x7efd9c09cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:13.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.737+0000 7efda3e00640 1 -- 172.21.15.39:0/2106566768 shutdown_connections 2024-09-21T07:35:13.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.737+0000 7efda3e00640 1 -- 172.21.15.39:0/2106566768 wait complete. 2024-09-21T07:35:13.761 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T07:35:13.761 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T07:35:13.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.990+0000 7f871b000640 1 -- 172.21.15.39:0/3189958827 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f871406e500 msgr2=0x7f871415cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:13.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.990+0000 7f871b000640 1 --2- 172.21.15.39:0/3189958827 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f871406e500 0x7f871415cc20 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8714073240 tx=0x7f870c02a980 comp rx=0 tx=0).stop 2024-09-21T07:35:13.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.990+0000 7f871b000640 1 -- 172.21.15.39:0/3189958827 shutdown_connections 2024-09-21T07:35:13.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.990+0000 7f871b000640 1 --2- 172.21.15.39:0/3189958827 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f871406e500 0x7f871415cc20 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:13.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.990+0000 7f871b000640 1 --2- 172.21.15.39:0/3189958827 >> v2:172.21.15.148:3300/0 conn(0x7f8714075fa0 0x7f87140afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:13.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.990+0000 7f871b000640 1 -- 172.21.15.39:0/3189958827 >> 172.21.15.39:0/3189958827 conn(0x7f8714073640 msgr2=0x7f8714073a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:13.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.990+0000 7f871b000640 1 -- 172.21.15.39:0/3189958827 shutdown_connections 2024-09-21T07:35:13.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.990+0000 7f871b000640 1 -- 172.21.15.39:0/3189958827 wait complete. 2024-09-21T07:35:13.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.992+0000 7f871b000640 1 Processor -- start 2024-09-21T07:35:13.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.993+0000 7f871b000640 1 -- start start 2024-09-21T07:35:13.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.994+0000 7f871b000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8714075fa0 0x7f8714157340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:13.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.994+0000 7f871b000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87140cdde0 0x7f87140aec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:13.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.994+0000 7f871b000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87141f64c0 0x7f87141f68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:13.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.994+0000 7f871b000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f871415d4f0 con 0x7f87140cdde0 2024-09-21T07:35:13.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.994+0000 7f871b000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f871415d370 con 0x7f87141f64c0 2024-09-21T07:35:13.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.994+0000 7f871b000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f871415d670 con 0x7f8714075fa0 2024-09-21T07:35:13.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.994+0000 7f8719200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87140cdde0 0x7f87140aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:13.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.994+0000 7f8719200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87140cdde0 0x7f87140aec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46488/0 (socket says 172.21.15.39:46488) 2024-09-21T07:35:13.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.994+0000 7f8719200640 1 -- 172.21.15.39:0/422971375 learned_addr learned my addr 172.21.15.39:0/422971375 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:13.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.995+0000 7f8719c00640 1 --2- 172.21.15.39:0/422971375 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8714075fa0 0x7f8714157340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:13.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.995+0000 7f871a600640 1 --2- 172.21.15.39:0/422971375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87141f64c0 0x7f87141f68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:13.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.995+0000 7f8719200640 1 -- 172.21.15.39:0/422971375 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8714075fa0 msgr2=0x7f8714157340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:13.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.995+0000 7f8719200640 1 --2- 172.21.15.39:0/422971375 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8714075fa0 0x7f8714157340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:13.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.995+0000 7f8719200640 1 -- 172.21.15.39:0/422971375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87141f64c0 msgr2=0x7f87141f68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:13.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.995+0000 7f8719200640 1 --2- 172.21.15.39:0/422971375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87141f64c0 0x7f87141f68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:13.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.995+0000 7f8719200640 1 -- 172.21.15.39:0/422971375 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8714157880 con 0x7f87140cdde0 2024-09-21T07:35:13.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.995+0000 7f8719200640 1 --2- 172.21.15.39:0/422971375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87140cdde0 0x7f87140aec90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8700009fd0 tx=0x7f870000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:13.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.995+0000 7f8711600640 1 -- 172.21.15.39:0/422971375 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8700015020 con 0x7f87140cdde0 2024-09-21T07:35:13.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.996+0000 7f8711600640 1 -- 172.21.15.39:0/422971375 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8700011920 con 0x7f87140cdde0 2024-09-21T07:35:13.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.996+0000 7f871b000640 1 -- 172.21.15.39:0/422971375 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8714157d90 con 0x7f87140cdde0 2024-09-21T07:35:13.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.996+0000 7f8711600640 1 -- 172.21.15.39:0/422971375 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8700011d50 con 0x7f87140cdde0 2024-09-21T07:35:13.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.996+0000 7f871b000640 1 -- 172.21.15.39:0/422971375 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8714158350 con 0x7f87140cdde0 2024-09-21T07:35:13.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.997+0000 7f871b000640 1 -- 172.21.15.39:0/422971375 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f871415dbd0 con 0x7f87140cdde0 2024-09-21T07:35:14.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.998+0000 7f8711600640 1 -- 172.21.15.39:0/422971375 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8700018020 con 0x7f87140cdde0 2024-09-21T07:35:14.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.999+0000 7f8711600640 1 --2- 172.21.15.39:0/422971375 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f86f4078b90 0x7f86f407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:14.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.999+0000 7f8711600640 1 -- 172.21.15.39:0/422971375 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5210+0+0 (secure 0 0 0) 0x7f870009a880 con 0x7f87140cdde0 2024-09-21T07:35:14.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:13.999+0000 7f8719c00640 1 --2- 172.21.15.39:0/422971375 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f86f4078b90 0x7f86f407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:14.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:14.000+0000 7f8719c00640 1 --2- 172.21.15.39:0/422971375 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f86f4078b90 0x7f86f407b0a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8708004630 tx=0x7f870800b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:14.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:14.003+0000 7f8711600640 1 -- 172.21.15.39:0/422971375 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f870001f020 con 0x7f87140cdde0 2024-09-21T07:35:14.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:14.200+0000 7f871b000640 1 -- 172.21.15.39:0/422971375 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f87141f6e10 con 0x7f87140cdde0 2024-09-21T07:35:14.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:14.734+0000 7f8711600640 1 -- 172.21.15.39:0/422971375 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v59) ==== 138+0+0 (secure 0 0 0) 0x7f87000628f0 con 0x7f87140cdde0 2024-09-21T07:35:14.736 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' created 2024-09-21T07:35:14.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:14.737+0000 7f871b000640 1 -- 172.21.15.39:0/422971375 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f86f4078b90 msgr2=0x7f86f407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:14.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:14.737+0000 7f871b000640 1 --2- 172.21.15.39:0/422971375 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f86f4078b90 0x7f86f407b0a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8708004630 tx=0x7f870800b2e0 comp rx=0 tx=0).stop 2024-09-21T07:35:14.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:14.737+0000 7f871b000640 1 -- 172.21.15.39:0/422971375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87140cdde0 msgr2=0x7f87140aec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:14.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:14.737+0000 7f871b000640 1 --2- 172.21.15.39:0/422971375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87140cdde0 0x7f87140aec90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8700009fd0 tx=0x7f870000b870 comp rx=0 tx=0).stop 2024-09-21T07:35:14.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:14.738+0000 7f871b000640 1 -- 172.21.15.39:0/422971375 shutdown_connections 2024-09-21T07:35:14.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:14.738+0000 7f871b000640 1 --2- 172.21.15.39:0/422971375 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f86f4078b90 0x7f86f407b0a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:14.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:14.738+0000 7f871b000640 1 --2- 172.21.15.39:0/422971375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87141f64c0 0x7f87141f68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:14.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:14.738+0000 7f871b000640 1 --2- 172.21.15.39:0/422971375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87140cdde0 0x7f87140aec90 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:14.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:14.738+0000 7f871b000640 1 --2- 172.21.15.39:0/422971375 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8714075fa0 0x7f8714157340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:14.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:14.738+0000 7f871b000640 1 -- 172.21.15.39:0/422971375 >> 172.21.15.39:0/422971375 conn(0x7f8714073640 msgr2=0x7f87141533b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:14.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:14.738+0000 7f871b000640 1 -- 172.21.15.39:0/422971375 shutdown_connections 2024-09-21T07:35:14.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:14.738+0000 7f871b000640 1 -- 172.21.15.39:0/422971375 wait complete. 2024-09-21T07:35:14.768 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T07:35:15.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.002+0000 7f4a73e00640 1 -- 172.21.15.39:0/625391103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a58005150 msgr2=0x7f4a5800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:15.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.002+0000 7f4a73e00640 1 --2- 172.21.15.39:0/625391103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a58005150 0x7f4a5800b9c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4a740b2610 tx=0x7f4a58005830 comp rx=0 tx=0).stop 2024-09-21T07:35:15.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.002+0000 7f4a73400640 1 -- 172.21.15.39:0/625391103 reap_dead start 2024-09-21T07:35:15.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.002+0000 7f4a73e00640 1 -- 172.21.15.39:0/625391103 shutdown_connections 2024-09-21T07:35:15.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.002+0000 7f4a73e00640 1 -- 172.21.15.39:0/625391103 >> 172.21.15.39:0/625391103 conn(0x7f4a7406d410 msgr2=0x7f4a740b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:15.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.002+0000 7f4a73e00640 1 -- 172.21.15.39:0/625391103 shutdown_connections 2024-09-21T07:35:15.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.002+0000 7f4a73e00640 1 -- 172.21.15.39:0/625391103 wait complete. 2024-09-21T07:35:15.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.004+0000 7f4a73e00640 1 Processor -- start 2024-09-21T07:35:15.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.005+0000 7f4a73e00640 1 -- start start 2024-09-21T07:35:15.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.006+0000 7f4a73e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a58005150 0x7f4a740b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:15.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.006+0000 7f4a73e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a74155540 0x7f4a740ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:15.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.006+0000 7f4a73e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a740adde0 0x7f4a74153ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:15.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.006+0000 7f4a73e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4a740ccbc0 con 0x7f4a74155540 2024-09-21T07:35:15.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.006+0000 7f4a73e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4a740cca40 con 0x7f4a740adde0 2024-09-21T07:35:15.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.006+0000 7f4a73e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f4a740ccd40 con 0x7f4a58005150 2024-09-21T07:35:15.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.006+0000 7f4a6be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a74155540 0x7f4a740ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:15.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.007+0000 7f4a6be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a74155540 0x7f4a740ad8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46496/0 (socket says 172.21.15.39:46496) 2024-09-21T07:35:15.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.007+0000 7f4a6be00640 1 -- 172.21.15.39:0/2939929962 learned_addr learned my addr 172.21.15.39:0/2939929962 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:15.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.007+0000 7f4a72a00640 1 --2- 172.21.15.39:0/2939929962 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a58005150 0x7f4a740b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:15.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.007+0000 7f4a73400640 1 --2- 172.21.15.39:0/2939929962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a740adde0 0x7f4a74153ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:15.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.007+0000 7f4a6be00640 1 -- 172.21.15.39:0/2939929962 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a58005150 msgr2=0x7f4a740b4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:15.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.007+0000 7f4a6be00640 1 --2- 172.21.15.39:0/2939929962 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a58005150 0x7f4a740b4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:15.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.007+0000 7f4a6be00640 1 -- 172.21.15.39:0/2939929962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a740adde0 msgr2=0x7f4a74153ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:15.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.007+0000 7f4a6be00640 1 --2- 172.21.15.39:0/2939929962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a740adde0 0x7f4a74153ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:15.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.007+0000 7f4a6be00640 1 -- 172.21.15.39:0/2939929962 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a74155a50 con 0x7f4a74155540 2024-09-21T07:35:15.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.007+0000 7f4a6be00640 1 --2- 172.21.15.39:0/2939929962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a74155540 0x7f4a740ad8a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c006680 tx=0x7f4a5c0066b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:15.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.008+0000 7f4a6aa00640 1 -- 172.21.15.39:0/2939929962 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c015070 con 0x7f4a74155540 2024-09-21T07:35:15.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.008+0000 7f4a6aa00640 1 -- 172.21.15.39:0/2939929962 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a5c011900 con 0x7f4a74155540 2024-09-21T07:35:15.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.008+0000 7f4a73e00640 1 -- 172.21.15.39:0/2939929962 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a74155d10 con 0x7f4a74155540 2024-09-21T07:35:15.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.008+0000 7f4a6aa00640 1 -- 172.21.15.39:0/2939929962 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c011d50 con 0x7f4a74155540 2024-09-21T07:35:15.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.008+0000 7f4a73e00640 1 -- 172.21.15.39:0/2939929962 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a740b4320 con 0x7f4a74155540 2024-09-21T07:35:15.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.009+0000 7f4a73e00640 1 -- 172.21.15.39:0/2939929962 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a7415d700 con 0x7f4a74155540 2024-09-21T07:35:15.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.010+0000 7f4a6aa00640 1 -- 172.21.15.39:0/2939929962 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4a5c018070 con 0x7f4a74155540 2024-09-21T07:35:15.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.011+0000 7f4a6aa00640 1 --2- 172.21.15.39:0/2939929962 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4a50078b90 0x7f4a5007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:15.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.011+0000 7f4a6aa00640 1 -- 172.21.15.39:0/2939929962 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5580+0+0 (secure 0 0 0) 0x7f4a5c09b950 con 0x7f4a74155540 2024-09-21T07:35:15.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.012+0000 7f4a72a00640 1 --2- 172.21.15.39:0/2939929962 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4a50078b90 0x7f4a5007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:15.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.012+0000 7f4a72a00640 1 --2- 172.21.15.39:0/2939929962 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4a50078b90 0x7f4a5007b0a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4a740b5220 tx=0x7f4a64005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:15.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.016+0000 7f4a6aa00640 1 -- 172.21.15.39:0/2939929962 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4a5c0637b0 con 0x7f4a74155540 2024-09-21T07:35:15.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.215+0000 7f4a73e00640 1 -- 172.21.15.39:0/2939929962 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f4a7406fb50 con 0x7f4a74155540 2024-09-21T07:35:15.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.742+0000 7f4a6aa00640 1 -- 172.21.15.39:0/2939929962 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v60) ==== 170+0+0 (secure 0 0 0) 0x7f4a5c0681e0 con 0x7f4a74155540 2024-09-21T07:35:15.745 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' created 2024-09-21T07:35:15.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.745+0000 7f4a73e00640 1 -- 172.21.15.39:0/2939929962 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4a50078b90 msgr2=0x7f4a5007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:15.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.745+0000 7f4a73e00640 1 --2- 172.21.15.39:0/2939929962 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4a50078b90 0x7f4a5007b0a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4a740b5220 tx=0x7f4a64005f90 comp rx=0 tx=0).stop 2024-09-21T07:35:15.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.745+0000 7f4a73e00640 1 -- 172.21.15.39:0/2939929962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a74155540 msgr2=0x7f4a740ad8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:15.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.745+0000 7f4a73e00640 1 --2- 172.21.15.39:0/2939929962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a74155540 0x7f4a740ad8a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c006680 tx=0x7f4a5c0066b0 comp rx=0 tx=0).stop 2024-09-21T07:35:15.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.745+0000 7f4a73e00640 1 -- 172.21.15.39:0/2939929962 shutdown_connections 2024-09-21T07:35:15.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.745+0000 7f4a73e00640 1 --2- 172.21.15.39:0/2939929962 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4a50078b90 0x7f4a5007b0a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:15.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.745+0000 7f4a73e00640 1 --2- 172.21.15.39:0/2939929962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a740adde0 0x7f4a74153ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:15.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.745+0000 7f4a73e00640 1 --2- 172.21.15.39:0/2939929962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a74155540 0x7f4a740ad8a0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:15.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.745+0000 7f4a73e00640 1 --2- 172.21.15.39:0/2939929962 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a58005150 0x7f4a740b4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:15.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.745+0000 7f4a73e00640 1 -- 172.21.15.39:0/2939929962 >> 172.21.15.39:0/2939929962 conn(0x7f4a7406d410 msgr2=0x7f4a74073780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:15.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.745+0000 7f4a73e00640 1 -- 172.21.15.39:0/2939929962 shutdown_connections 2024-09-21T07:35:15.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:15.745+0000 7f4a73e00640 1 -- 172.21.15.39:0/2939929962 wait complete. 2024-09-21T07:35:15.789 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T07:35:16.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.017+0000 7f64ed800640 1 -- 172.21.15.39:0/3351898984 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f64e8070030 msgr2=0x7f64e8074640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:16.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.017+0000 7f64ed800640 1 --2- 172.21.15.39:0/3351898984 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f64e8070030 0x7f64e8074640 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f64d4009830 tx=0x7f64d402a480 comp rx=0 tx=0).stop 2024-09-21T07:35:16.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.017+0000 7f64ed800640 1 -- 172.21.15.39:0/3351898984 shutdown_connections 2024-09-21T07:35:16.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.017+0000 7f64ed800640 1 --2- 172.21.15.39:0/3351898984 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f64e8070030 0x7f64e8074640 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:16.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.017+0000 7f64ed800640 1 --2- 172.21.15.39:0/3351898984 >> v2:172.21.15.148:3300/0 conn(0x7f64e8073170 0x7f64e815c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:16.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.017+0000 7f64ed800640 1 -- 172.21.15.39:0/3351898984 >> 172.21.15.39:0/3351898984 conn(0x7f64e80b1cb0 msgr2=0x7f64e80b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:16.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.017+0000 7f64ed800640 1 -- 172.21.15.39:0/3351898984 shutdown_connections 2024-09-21T07:35:16.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.017+0000 7f64ed800640 1 -- 172.21.15.39:0/3351898984 wait complete. 2024-09-21T07:35:16.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.019+0000 7f64ed800640 1 Processor -- start 2024-09-21T07:35:16.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.020+0000 7f64ed800640 1 -- start start 2024-09-21T07:35:16.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.021+0000 7f64ed800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64e8070030 0x7f64e80cc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:16.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.021+0000 7f64ed800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64e8070440 0x7f64e8151640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:16.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.021+0000 7f64ed800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f64e8073170 0x7f64e80ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:16.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.021+0000 7f64ed800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f64e806f5a0 con 0x7f64e8070440 2024-09-21T07:35:16.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.021+0000 7f64ed800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f64e8074f80 con 0x7f64e8070030 2024-09-21T07:35:16.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.021+0000 7f64ed800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f64e806f720 con 0x7f64e8073170 2024-09-21T07:35:16.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.021+0000 7f64e6000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64e8070440 0x7f64e8151640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:16.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.021+0000 7f64e6000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64e8070440 0x7f64e8151640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46506/0 (socket says 172.21.15.39:46506) 2024-09-21T07:35:16.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.021+0000 7f64e6000640 1 -- 172.21.15.39:0/2608057271 learned_addr learned my addr 172.21.15.39:0/2608057271 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:16.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.021+0000 7f64e6a00640 1 --2- 172.21.15.39:0/2608057271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64e8070030 0x7f64e80cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:16.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.022+0000 7f64e7400640 1 --2- 172.21.15.39:0/2608057271 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f64e8073170 0x7f64e80ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:16.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.022+0000 7f64e6000640 1 -- 172.21.15.39:0/2608057271 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f64e8073170 msgr2=0x7f64e80ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:16.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.022+0000 7f64e6000640 1 --2- 172.21.15.39:0/2608057271 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f64e8073170 0x7f64e80ca1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:16.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.022+0000 7f64e6000640 1 -- 172.21.15.39:0/2608057271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64e8070030 msgr2=0x7f64e80cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:16.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.022+0000 7f64e6000640 1 --2- 172.21.15.39:0/2608057271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64e8070030 0x7f64e80cc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:16.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.022+0000 7f64e6000640 1 -- 172.21.15.39:0/2608057271 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e809bbb0 con 0x7f64e8070440 2024-09-21T07:35:16.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.022+0000 7f64e6000640 1 --2- 172.21.15.39:0/2608057271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64e8070440 0x7f64e8151640 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f64d402aa70 tx=0x7f64d4033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:16.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.022+0000 7f64dea00640 1 -- 172.21.15.39:0/2608057271 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d4031d60 con 0x7f64e8070440 2024-09-21T07:35:16.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.023+0000 7f64dea00640 1 -- 172.21.15.39:0/2608057271 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64d4033e10 con 0x7f64e8070440 2024-09-21T07:35:16.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.023+0000 7f64ed800640 1 -- 172.21.15.39:0/2608057271 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64e809be90 con 0x7f64e8070440 2024-09-21T07:35:16.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.023+0000 7f64dea00640 1 -- 172.21.15.39:0/2608057271 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d4005260 con 0x7f64e8070440 2024-09-21T07:35:16.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.023+0000 7f64ed800640 1 -- 172.21.15.39:0/2608057271 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64e809c450 con 0x7f64e8070440 2024-09-21T07:35:16.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.024+0000 7f64ed800640 1 -- 172.21.15.39:0/2608057271 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64e806fc40 con 0x7f64e8070440 2024-09-21T07:35:16.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.025+0000 7f64dea00640 1 -- 172.21.15.39:0/2608057271 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f64d4005400 con 0x7f64e8070440 2024-09-21T07:35:16.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.026+0000 7f64dea00640 1 --2- 172.21.15.39:0/2608057271 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f64c8078b90 0x7f64c807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:16.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.026+0000 7f64dea00640 1 -- 172.21.15.39:0/2608057271 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5962+0+0 (secure 0 0 0) 0x7f64d40bbea0 con 0x7f64e8070440 2024-09-21T07:35:16.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.027+0000 7f64e6a00640 1 --2- 172.21.15.39:0/2608057271 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f64c8078b90 0x7f64c807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:16.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.028+0000 7f64e6a00640 1 --2- 172.21.15.39:0/2608057271 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f64c8078b90 0x7f64c807b0a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f64d8009530 tx=0x7f64d8009260 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:16.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.032+0000 7f64dea00640 1 -- 172.21.15.39:0/2608057271 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f64d403f030 con 0x7f64e8070440 2024-09-21T07:35:16.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.268+0000 7f64ed800640 1 -- 172.21.15.39:0/2608057271 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f64e80c3860 con 0x7f64e8070440 2024-09-21T07:35:16.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.748+0000 7f64dea00640 1 -- 172.21.15.39:0/2608057271 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v31) ==== 173+0+0 (secure 0 0 0) 0x7f64d4083b80 con 0x7f64e8070440 2024-09-21T07:35:16.750 INFO:teuthology.orchestra.run.smithi039.stderr:new fs with metadata pool 9 and data pool 10 2024-09-21T07:35:16.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.751+0000 7f64ed800640 1 -- 172.21.15.39:0/2608057271 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f64c8078b90 msgr2=0x7f64c807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:16.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.751+0000 7f64ed800640 1 --2- 172.21.15.39:0/2608057271 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f64c8078b90 0x7f64c807b0a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f64d8009530 tx=0x7f64d8009260 comp rx=0 tx=0).stop 2024-09-21T07:35:16.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.751+0000 7f64ed800640 1 -- 172.21.15.39:0/2608057271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64e8070440 msgr2=0x7f64e8151640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:16.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.751+0000 7f64ed800640 1 --2- 172.21.15.39:0/2608057271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64e8070440 0x7f64e8151640 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f64d402aa70 tx=0x7f64d4033c20 comp rx=0 tx=0).stop 2024-09-21T07:35:16.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.751+0000 7f64ed800640 1 -- 172.21.15.39:0/2608057271 shutdown_connections 2024-09-21T07:35:16.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.751+0000 7f64ed800640 1 --2- 172.21.15.39:0/2608057271 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f64c8078b90 0x7f64c807b0a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:16.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.751+0000 7f64ed800640 1 --2- 172.21.15.39:0/2608057271 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f64e8073170 0x7f64e80ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:16.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.751+0000 7f64ed800640 1 --2- 172.21.15.39:0/2608057271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64e8070440 0x7f64e8151640 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:16.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.751+0000 7f64ed800640 1 --2- 172.21.15.39:0/2608057271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64e8070030 0x7f64e80cc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:16.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.751+0000 7f64ed800640 1 -- 172.21.15.39:0/2608057271 >> 172.21.15.39:0/2608057271 conn(0x7f64e80b1cb0 msgr2=0x7f64e80d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:16.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.751+0000 7f64ed800640 1 -- 172.21.15.39:0/2608057271 shutdown_connections 2024-09-21T07:35:16.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:16.751+0000 7f64ed800640 1 -- 172.21.15.39:0/2608057271 wait complete. 2024-09-21T07:35:16.798 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:35:17.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.026+0000 7fc73da00640 1 -- 172.21.15.39:0/2310674030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7380cc1e0 msgr2=0x7fc7380af3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:17.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.026+0000 7fc73da00640 1 --2- 172.21.15.39:0/2310674030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7380cc1e0 0x7fc7380af3c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc728005fd0 tx=0x7fc72802aaa0 comp rx=0 tx=0).stop 2024-09-21T07:35:17.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.027+0000 7fc73da00640 1 -- 172.21.15.39:0/2310674030 shutdown_connections 2024-09-21T07:35:17.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.027+0000 7fc73da00640 1 --2- 172.21.15.39:0/2310674030 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc7380ccb50 0x7fc7380ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:17.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.027+0000 7fc73da00640 1 --2- 172.21.15.39:0/2310674030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7380cc1e0 0x7fc7380af3c0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:17.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.027+0000 7fc73da00640 1 -- 172.21.15.39:0/2310674030 >> 172.21.15.39:0/2310674030 conn(0x7fc73806d7d0 msgr2=0x7fc7380b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:17.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.027+0000 7fc73da00640 1 -- 172.21.15.39:0/2310674030 shutdown_connections 2024-09-21T07:35:17.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.027+0000 7fc73da00640 1 -- 172.21.15.39:0/2310674030 wait complete. 2024-09-21T07:35:17.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.029+0000 7fc73da00640 1 Processor -- start 2024-09-21T07:35:17.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.029+0000 7fc73da00640 1 -- start start 2024-09-21T07:35:17.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.031+0000 7fc73da00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc7380cc1e0 0x7fc738152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:17.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.031+0000 7fc73da00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7380ccb50 0x7fc738152db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:17.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.031+0000 7fc73da00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc73815d8a0 0x7fc7380ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:17.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.031+0000 7fc73da00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc7381533a0 con 0x7fc7380ccb50 2024-09-21T07:35:17.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.031+0000 7fc73da00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc73815d3e0 con 0x7fc73815d8a0 2024-09-21T07:35:17.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.031+0000 7fc73da00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc738153520 con 0x7fc7380cc1e0 2024-09-21T07:35:17.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.031+0000 7fc72e000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7380ccb50 0x7fc738152db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:17.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.031+0000 7fc72e000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7380ccb50 0x7fc738152db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46516/0 (socket says 172.21.15.39:46516) 2024-09-21T07:35:17.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.031+0000 7fc72e000640 1 -- 172.21.15.39:0/1423146469 learned_addr learned my addr 172.21.15.39:0/1423146469 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:17.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.031+0000 7fc736a00640 1 --2- 172.21.15.39:0/1423146469 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc7380cc1e0 0x7fc738152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:17.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.031+0000 7fc737400640 1 --2- 172.21.15.39:0/1423146469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc73815d8a0 0x7fc7380ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:17.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.031+0000 7fc72e000640 1 -- 172.21.15.39:0/1423146469 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc7380cc1e0 msgr2=0x7fc738152870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:17.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.031+0000 7fc72e000640 1 --2- 172.21.15.39:0/1423146469 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc7380cc1e0 0x7fc738152870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:17.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.031+0000 7fc72e000640 1 -- 172.21.15.39:0/1423146469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc73815d8a0 msgr2=0x7fc7380ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:17.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.031+0000 7fc72e000640 1 --2- 172.21.15.39:0/1423146469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc73815d8a0 0x7fc7380ca9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:17.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.031+0000 7fc72e000640 1 -- 172.21.15.39:0/1423146469 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7381f6e20 con 0x7fc7380ccb50 2024-09-21T07:35:17.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.032+0000 7fc72e000640 1 --2- 172.21.15.39:0/1423146469 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7380ccb50 0x7fc738152db0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc72000c3a0 tx=0x7fc72000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:17.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.032+0000 7fc72f400640 1 -- 172.21.15.39:0/1423146469 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc720017070 con 0x7fc7380ccb50 2024-09-21T07:35:17.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.032+0000 7fc72f400640 1 -- 172.21.15.39:0/1423146469 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7200053c0 con 0x7fc7380ccb50 2024-09-21T07:35:17.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.032+0000 7fc72f400640 1 -- 172.21.15.39:0/1423146469 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc720003850 con 0x7fc7380ccb50 2024-09-21T07:35:17.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.032+0000 7fc73da00640 1 -- 172.21.15.39:0/1423146469 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7380c9f30 con 0x7fc7380ccb50 2024-09-21T07:35:17.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.032+0000 7fc73da00640 1 -- 172.21.15.39:0/1423146469 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7380ca1d0 con 0x7fc7380ccb50 2024-09-21T07:35:17.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.033+0000 7fc73da00640 1 -- 172.21.15.39:0/1423146469 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7380ccf60 con 0x7fc7380ccb50 2024-09-21T07:35:17.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.035+0000 7fc72f400640 1 -- 172.21.15.39:0/1423146469 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc72001a070 con 0x7fc7380ccb50 2024-09-21T07:35:17.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.036+0000 7fc72f400640 1 --2- 172.21.15.39:0/1423146469 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc714078bc0 0x7fc71407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:17.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.036+0000 7fc72f400640 1 -- 172.21.15.39:0/1423146469 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6062+0+0 (secure 0 0 0) 0x7fc72009a210 con 0x7fc7380ccb50 2024-09-21T07:35:17.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.036+0000 7fc736a00640 1 --2- 172.21.15.39:0/1423146469 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc714078bc0 0x7fc71407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:17.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.037+0000 7fc736a00640 1 --2- 172.21.15.39:0/1423146469 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc714078bc0 0x7fc71407b0d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc728035000 tx=0x7fc728038040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:17.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.040+0000 7fc72f400640 1 -- 172.21.15.39:0/1423146469 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc7380cd4a0 con 0x7fc7380ccb50 2024-09-21T07:35:17.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.238+0000 7fc73da00640 1 -- 172.21.15.39:0/1423146469 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc738070b80 con 0x7fc7380ccb50 2024-09-21T07:35:17.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.240+0000 7fc72f400640 1 -- 172.21.15.39:0/1423146469 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) ==== 74+0+18494 (secure 0 0 0) 0x7fc720062580 con 0x7fc7380ccb50 2024-09-21T07:35:17.242 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:35:17.242 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":61,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:35:16.270387+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:35:14.214410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-21T07:35:15.228034+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:35:17.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.242+0000 7fc73da00640 1 -- 172.21.15.39:0/1423146469 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc714078bc0 msgr2=0x7fc71407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:17.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.242+0000 7fc73da00640 1 --2- 172.21.15.39:0/1423146469 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc714078bc0 0x7fc71407b0d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc728035000 tx=0x7fc728038040 comp rx=0 tx=0).stop 2024-09-21T07:35:17.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.242+0000 7fc73da00640 1 -- 172.21.15.39:0/1423146469 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7380ccb50 msgr2=0x7fc738152db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:17.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.242+0000 7fc73da00640 1 --2- 172.21.15.39:0/1423146469 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7380ccb50 0x7fc738152db0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc72000c3a0 tx=0x7fc72000c860 comp rx=0 tx=0).stop 2024-09-21T07:35:17.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.242+0000 7fc73da00640 1 -- 172.21.15.39:0/1423146469 shutdown_connections 2024-09-21T07:35:17.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.242+0000 7fc73da00640 1 --2- 172.21.15.39:0/1423146469 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc714078bc0 0x7fc71407b0d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:17.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.242+0000 7fc73da00640 1 --2- 172.21.15.39:0/1423146469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc73815d8a0 0x7fc7380ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:17.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.242+0000 7fc73da00640 1 --2- 172.21.15.39:0/1423146469 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7380ccb50 0x7fc738152db0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:17.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.242+0000 7fc73da00640 1 --2- 172.21.15.39:0/1423146469 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc7380cc1e0 0x7fc738152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:17.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.242+0000 7fc73da00640 1 -- 172.21.15.39:0/1423146469 >> 172.21.15.39:0/1423146469 conn(0x7fc73806d7d0 msgr2=0x7fc7380b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:17.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.242+0000 7fc73da00640 1 -- 172.21.15.39:0/1423146469 shutdown_connections 2024-09-21T07:35:17.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.242+0000 7fc73da00640 1 -- 172.21.15.39:0/1423146469 wait complete. 2024-09-21T07:35:17.266 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:35:17.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.488+0000 7f26ace00640 1 -- 172.21.15.39:0/1158497543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26a80706b0 msgr2=0x7f26a80cba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:17.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.488+0000 7f26ace00640 1 --2- 172.21.15.39:0/1158497543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26a80706b0 0x7f26a80cba00 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2690005fd0 tx=0x7f269002a5d0 comp rx=0 tx=0).stop 2024-09-21T07:35:17.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.488+0000 7f26ace00640 1 -- 172.21.15.39:0/1158497543 shutdown_connections 2024-09-21T07:35:17.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.488+0000 7f26ace00640 1 --2- 172.21.15.39:0/1158497543 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f26a80cad30 0x7f26a80735b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:17.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.488+0000 7f26ace00640 1 --2- 172.21.15.39:0/1158497543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26a80706b0 0x7f26a80cba00 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:17.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.488+0000 7f26ace00640 1 -- 172.21.15.39:0/1158497543 >> 172.21.15.39:0/1158497543 conn(0x7f26a80b1cb0 msgr2=0x7f26a80b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:17.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.488+0000 7f26ace00640 1 -- 172.21.15.39:0/1158497543 shutdown_connections 2024-09-21T07:35:17.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.488+0000 7f26ace00640 1 -- 172.21.15.39:0/1158497543 wait complete. 2024-09-21T07:35:17.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.491+0000 7f26ace00640 1 Processor -- start 2024-09-21T07:35:17.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.491+0000 7f26ace00640 1 -- start start 2024-09-21T07:35:17.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.492+0000 7f26ace00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26a80ca800 0x7f26a80b0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:17.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.492+0000 7f26ace00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f26a80cad30 0x7f26a80b1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:17.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.492+0000 7f26ace00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26a815a480 0x7f26a815a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:17.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.492+0000 7f26ace00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f26a8073f00 con 0x7f26a80ca800 2024-09-21T07:35:17.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.492+0000 7f26ace00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f26a8073d80 con 0x7f26a815a480 2024-09-21T07:35:17.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.492+0000 7f26ace00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f26a8074080 con 0x7f26a80cad30 2024-09-21T07:35:17.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.492+0000 7f26a5e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26a80ca800 0x7f26a80b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:17.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.492+0000 7f26a5e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26a80ca800 0x7f26a80b0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46536/0 (socket says 172.21.15.39:46536) 2024-09-21T07:35:17.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.493+0000 7f26a5e00640 1 -- 172.21.15.39:0/159245356 learned_addr learned my addr 172.21.15.39:0/159245356 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:17.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.493+0000 7f269d400640 1 --2- 172.21.15.39:0/159245356 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f26a80cad30 0x7f26a80b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:17.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.493+0000 7f26a6800640 1 --2- 172.21.15.39:0/159245356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26a815a480 0x7f26a815a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:17.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.493+0000 7f26a5e00640 1 -- 172.21.15.39:0/159245356 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f26a80cad30 msgr2=0x7f26a80b1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:17.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.493+0000 7f26a5e00640 1 --2- 172.21.15.39:0/159245356 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f26a80cad30 0x7f26a80b1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:17.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.493+0000 7f26a5e00640 1 -- 172.21.15.39:0/159245356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26a815a480 msgr2=0x7f26a815a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:17.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.493+0000 7f26a5e00640 1 --2- 172.21.15.39:0/159245356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26a815a480 0x7f26a815a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:17.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.493+0000 7f26a5e00640 1 -- 172.21.15.39:0/159245356 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a80b0860 con 0x7f26a80ca800 2024-09-21T07:35:17.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.493+0000 7f26a5e00640 1 --2- 172.21.15.39:0/159245356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26a80ca800 0x7f26a80b0320 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f269800c6c0 tx=0x7f269800cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:17.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.493+0000 7f269ea00640 1 -- 172.21.15.39:0/159245356 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2698016020 con 0x7f26a80ca800 2024-09-21T07:35:17.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.493+0000 7f269ea00640 1 -- 172.21.15.39:0/159245356 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2698006e40 con 0x7f26a80ca800 2024-09-21T07:35:17.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.493+0000 7f269ea00640 1 -- 172.21.15.39:0/159245356 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2698003bf0 con 0x7f26a80ca800 2024-09-21T07:35:17.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.494+0000 7f26ace00640 1 -- 172.21.15.39:0/159245356 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26a815add0 con 0x7f26a80ca800 2024-09-21T07:35:17.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.494+0000 7f26ace00640 1 -- 172.21.15.39:0/159245356 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26a81f9420 con 0x7f26a80ca800 2024-09-21T07:35:17.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.495+0000 7f26ace00640 1 -- 172.21.15.39:0/159245356 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26a80745e0 con 0x7f26a80ca800 2024-09-21T07:35:17.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.496+0000 7f269ea00640 1 -- 172.21.15.39:0/159245356 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f26980050d0 con 0x7f26a80ca800 2024-09-21T07:35:17.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.497+0000 7f269ea00640 1 --2- 172.21.15.39:0/159245356 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2684078b90 0x7f268407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:17.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.497+0000 7f269ea00640 1 -- 172.21.15.39:0/159245356 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6062+0+0 (secure 0 0 0) 0x7f269809a950 con 0x7f26a80ca800 2024-09-21T07:35:17.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.497+0000 7f269d400640 1 --2- 172.21.15.39:0/159245356 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2684078b90 0x7f268407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:17.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.498+0000 7f269d400640 1 --2- 172.21.15.39:0/159245356 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2684078b90 0x7f268407b0a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2690033000 tx=0x7f2690004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:17.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.502+0000 7f269ea00640 1 -- 172.21.15.39:0/159245356 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2698020020 con 0x7f26a80ca800 2024-09-21T07:35:17.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.700+0000 7f26ace00640 1 -- 172.21.15.39:0/159245356 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f26a80b20c0 con 0x7f26a80ca800 2024-09-21T07:35:17.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.702+0000 7f269ea00640 1 -- 172.21.15.39:0/159245356 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) ==== 74+0+18494 (secure 0 0 0) 0x7f26a815add0 con 0x7f26a80ca800 2024-09-21T07:35:17.704 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:35:17.704 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":61,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:35:16.270387+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:35:14.214410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-21T07:35:15.228034+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:35:17.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.704+0000 7f26ace00640 1 -- 172.21.15.39:0/159245356 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2684078b90 msgr2=0x7f268407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:17.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.704+0000 7f26ace00640 1 --2- 172.21.15.39:0/159245356 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2684078b90 0x7f268407b0a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2690033000 tx=0x7f2690004730 comp rx=0 tx=0).stop 2024-09-21T07:35:17.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.704+0000 7f26ace00640 1 -- 172.21.15.39:0/159245356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26a80ca800 msgr2=0x7f26a80b0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:17.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.704+0000 7f26ace00640 1 --2- 172.21.15.39:0/159245356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26a80ca800 0x7f26a80b0320 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f269800c6c0 tx=0x7f269800cb80 comp rx=0 tx=0).stop 2024-09-21T07:35:17.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.704+0000 7f26ace00640 1 -- 172.21.15.39:0/159245356 shutdown_connections 2024-09-21T07:35:17.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.704+0000 7f26ace00640 1 --2- 172.21.15.39:0/159245356 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2684078b90 0x7f268407b0a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:17.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.704+0000 7f26ace00640 1 --2- 172.21.15.39:0/159245356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26a815a480 0x7f26a815a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:17.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.704+0000 7f26ace00640 1 --2- 172.21.15.39:0/159245356 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f26a80cad30 0x7f26a80b1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:17.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.704+0000 7f26ace00640 1 --2- 172.21.15.39:0/159245356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26a80ca800 0x7f26a80b0320 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:17.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.704+0000 7f26ace00640 1 -- 172.21.15.39:0/159245356 >> 172.21.15.39:0/159245356 conn(0x7f26a80b1cb0 msgr2=0x7f26a815b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:17.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.704+0000 7f26ace00640 1 -- 172.21.15.39:0/159245356 shutdown_connections 2024-09-21T07:35:17.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.704+0000 7f26ace00640 1 -- 172.21.15.39:0/159245356 wait complete. 2024-09-21T07:35:17.728 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T07:35:17.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.958+0000 7f69c5200640 1 -- 172.21.15.39:0/4292394600 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69c0070030 msgr2=0x7f69c0074640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:17.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.958+0000 7f69c5200640 1 --2- 172.21.15.39:0/4292394600 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69c0070030 0x7f69c0074640 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f69c0070e70 tx=0x7f69a802a4d0 comp rx=0 tx=0).stop 2024-09-21T07:35:17.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.958+0000 7f69c5200640 1 -- 172.21.15.39:0/4292394600 shutdown_connections 2024-09-21T07:35:17.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.958+0000 7f69c5200640 1 --2- 172.21.15.39:0/4292394600 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69c0070030 0x7f69c0074640 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:17.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.958+0000 7f69c5200640 1 --2- 172.21.15.39:0/4292394600 >> v2:172.21.15.148:3300/0 conn(0x7f69c0073170 0x7f69c015c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:17.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.958+0000 7f69c5200640 1 -- 172.21.15.39:0/4292394600 >> 172.21.15.39:0/4292394600 conn(0x7f69c00b1cb0 msgr2=0x7f69c00b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:17.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.958+0000 7f69c5200640 1 -- 172.21.15.39:0/4292394600 shutdown_connections 2024-09-21T07:35:17.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.959+0000 7f69c5200640 1 -- 172.21.15.39:0/4292394600 wait complete. 2024-09-21T07:35:17.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.961+0000 7f69c5200640 1 Processor -- start 2024-09-21T07:35:17.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.961+0000 7f69c5200640 1 -- start start 2024-09-21T07:35:17.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.962+0000 7f69c5200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69c0070030 0x7f69c01f4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:17.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.962+0000 7f69c5200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f69c0070440 0x7f69c015ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:17.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.962+0000 7f69c5200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69c0073170 0x7f69c01f6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:17.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.962+0000 7f69c5200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f69c0075050 con 0x7f69c0070030 2024-09-21T07:35:17.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.962+0000 7f69c5200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f69c0074ed0 con 0x7f69c0073170 2024-09-21T07:35:17.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.962+0000 7f69c5200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f69c006f5a0 con 0x7f69c0070440 2024-09-21T07:35:17.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.962+0000 7f69bde00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69c0070030 0x7f69c01f4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:17.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.963+0000 7f69bde00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69c0070030 0x7f69c01f4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46568/0 (socket says 172.21.15.39:46568) 2024-09-21T07:35:17.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.963+0000 7f69bde00640 1 -- 172.21.15.39:0/3027011839 learned_addr learned my addr 172.21.15.39:0/3027011839 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:17.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.963+0000 7f69b5400640 1 --2- 172.21.15.39:0/3027011839 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f69c0070440 0x7f69c015ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:17.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.963+0000 7f69be800640 1 --2- 172.21.15.39:0/3027011839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69c0073170 0x7f69c01f6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:17.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.963+0000 7f69bde00640 1 -- 172.21.15.39:0/3027011839 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f69c0070440 msgr2=0x7f69c015ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:17.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.963+0000 7f69bde00640 1 --2- 172.21.15.39:0/3027011839 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f69c0070440 0x7f69c015ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:17.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.963+0000 7f69bde00640 1 -- 172.21.15.39:0/3027011839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69c0073170 msgr2=0x7f69c01f6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:17.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.963+0000 7f69bde00640 1 --2- 172.21.15.39:0/3027011839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69c0073170 0x7f69c01f6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:17.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.963+0000 7f69bde00640 1 -- 172.21.15.39:0/3027011839 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69c01f48d0 con 0x7f69c0070030 2024-09-21T07:35:17.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.963+0000 7f69b5400640 1 --2- 172.21.15.39:0/3027011839 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f69c0070440 0x7f69c015ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:35:17.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.963+0000 7f69be800640 1 --2- 172.21.15.39:0/3027011839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69c0073170 0x7f69c01f6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:35:17.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.963+0000 7f69bde00640 1 --2- 172.21.15.39:0/3027011839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69c0070030 0x7f69c01f4390 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f69b000d6d0 tx=0x7f69b000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:17.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.963+0000 7f69b6a00640 1 -- 172.21.15.39:0/3027011839 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b0017070 con 0x7f69c0070030 2024-09-21T07:35:17.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.964+0000 7f69b6a00640 1 -- 172.21.15.39:0/3027011839 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69b0005330 con 0x7f69c0070030 2024-09-21T07:35:17.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.964+0000 7f69c5200640 1 -- 172.21.15.39:0/3027011839 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69c00cea90 con 0x7f69c0070030 2024-09-21T07:35:17.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.964+0000 7f69b6a00640 1 -- 172.21.15.39:0/3027011839 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b0009820 con 0x7f69c0070030 2024-09-21T07:35:17.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.964+0000 7f69c5200640 1 -- 172.21.15.39:0/3027011839 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69c009af60 con 0x7f69c0070030 2024-09-21T07:35:17.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.965+0000 7f69c5200640 1 -- 172.21.15.39:0/3027011839 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69c006fac0 con 0x7f69c0070030 2024-09-21T07:35:17.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.966+0000 7f69b6a00640 1 -- 172.21.15.39:0/3027011839 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f69b0009bd0 con 0x7f69c0070030 2024-09-21T07:35:17.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.967+0000 7f69b6a00640 1 --2- 172.21.15.39:0/3027011839 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f699c078bc0 0x7f699c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:17.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.967+0000 7f69b6a00640 1 -- 172.21.15.39:0/3027011839 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6062+0+0 (secure 0 0 0) 0x7f69b009b240 con 0x7f69c0070030 2024-09-21T07:35:17.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.967+0000 7f69b5400640 1 --2- 172.21.15.39:0/3027011839 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f699c078bc0 0x7f699c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:17.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.969+0000 7f69b5400640 1 --2- 172.21.15.39:0/3027011839 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f699c078bc0 0x7f699c07b0d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f69a8033000 tx=0x7f69a8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:17.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:17.972+0000 7f69b6a00640 1 -- 172.21.15.39:0/3027011839 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f69c01f48d0 con 0x7f69c0070030 2024-09-21T07:35:18.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:18.206+0000 7f69c5200640 1 -- 172.21.15.39:0/3027011839 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f69c015bc00 con 0x7f69c0070030 2024-09-21T07:35:18.719 INFO:tasks.ceph.mds.a.smithi039.stderr:2024-09-21T07:35:18.718+0000 7ff61a800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:35:18.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:18.786+0000 7f69b6a00640 1 -- 172.21.15.39:0/3027011839 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v34) ==== 118+0+0 (secure 0 0 0) 0x7f69b0063480 con 0x7f69c0070030 2024-09-21T07:35:18.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:18.789+0000 7f69c5200640 1 -- 172.21.15.39:0/3027011839 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f699c078bc0 msgr2=0x7f699c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:18.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:18.789+0000 7f69c5200640 1 --2- 172.21.15.39:0/3027011839 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f699c078bc0 0x7f699c07b0d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f69a8033000 tx=0x7f69a8004730 comp rx=0 tx=0).stop 2024-09-21T07:35:18.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:18.789+0000 7f69c5200640 1 -- 172.21.15.39:0/3027011839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69c0070030 msgr2=0x7f69c01f4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:18.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:18.789+0000 7f69c5200640 1 --2- 172.21.15.39:0/3027011839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69c0070030 0x7f69c01f4390 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f69b000d6d0 tx=0x7f69b000db90 comp rx=0 tx=0).stop 2024-09-21T07:35:18.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:18.789+0000 7f69c5200640 1 -- 172.21.15.39:0/3027011839 shutdown_connections 2024-09-21T07:35:18.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:18.789+0000 7f69c5200640 1 --2- 172.21.15.39:0/3027011839 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f699c078bc0 0x7f699c07b0d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:18.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:18.789+0000 7f69c5200640 1 --2- 172.21.15.39:0/3027011839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69c0073170 0x7f69c01f6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:18.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:18.789+0000 7f69c5200640 1 --2- 172.21.15.39:0/3027011839 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f69c0070440 0x7f69c015ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:18.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:18.789+0000 7f69c5200640 1 --2- 172.21.15.39:0/3027011839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69c0070030 0x7f69c01f4390 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:18.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:18.789+0000 7f69c5200640 1 -- 172.21.15.39:0/3027011839 >> 172.21.15.39:0/3027011839 conn(0x7f69c00b1cb0 msgr2=0x7f69c00b1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:18.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:18.789+0000 7f69c5200640 1 -- 172.21.15.39:0/3027011839 shutdown_connections 2024-09-21T07:35:18.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:18.789+0000 7f69c5200640 1 -- 172.21.15.39:0/3027011839 wait complete. 2024-09-21T07:35:18.818 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T07:35:18.818 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T07:35:19.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.046+0000 7f9feb200640 1 -- 172.21.15.39:0/921144903 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fec15c580 msgr2=0x7f9fec0d0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:19.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.046+0000 7f9feb200640 1 --2- 172.21.15.39:0/921144903 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fec15c580 0x7f9fec0d0800 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc005fd0 tx=0x7f9fdc02a540 comp rx=0 tx=0).stop 2024-09-21T07:35:19.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.046+0000 7f9feb200640 1 -- 172.21.15.39:0/921144903 shutdown_connections 2024-09-21T07:35:19.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.046+0000 7f9feb200640 1 --2- 172.21.15.39:0/921144903 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fec15c580 0x7f9fec0d0800 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:19.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.046+0000 7f9feb200640 1 -- 172.21.15.39:0/921144903 >> 172.21.15.39:0/921144903 conn(0x7f9fec06ee50 msgr2=0x7f9fec06f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:19.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.046+0000 7f9feb200640 1 -- 172.21.15.39:0/921144903 shutdown_connections 2024-09-21T07:35:19.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.046+0000 7f9feb200640 1 -- 172.21.15.39:0/921144903 wait complete. 2024-09-21T07:35:19.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.049+0000 7f9feb200640 1 Processor -- start 2024-09-21T07:35:19.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.049+0000 7f9feb200640 1 -- start start 2024-09-21T07:35:19.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.050+0000 7f9feb200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fec072ca0 0x7f9fec159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:19.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.050+0000 7f9feb200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fec0737c0 0x7f9fec148b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:19.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.050+0000 7f9feb200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9fec15c580 0x7f9fec15db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:19.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.050+0000 7f9feb200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9fec0cb3e0 con 0x7f9fec072ca0 2024-09-21T07:35:19.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.050+0000 7f9feb200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9fec0cb260 con 0x7f9fec0737c0 2024-09-21T07:35:19.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.050+0000 7f9feb200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f9fec0cb560 con 0x7f9fec15c580 2024-09-21T07:35:19.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.050+0000 7f9fe9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fec072ca0 0x7f9fec159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:19.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.050+0000 7f9fe9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fec072ca0 0x7f9fec159290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46600/0 (socket says 172.21.15.39:46600) 2024-09-21T07:35:19.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.050+0000 7f9fe9e00640 1 -- 172.21.15.39:0/1376865852 learned_addr learned my addr 172.21.15.39:0/1376865852 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:19.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.051+0000 7f9fe1400640 1 --2- 172.21.15.39:0/1376865852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fec0737c0 0x7f9fec148b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:19.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.051+0000 7f9fea800640 1 --2- 172.21.15.39:0/1376865852 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9fec15c580 0x7f9fec15db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:19.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.051+0000 7f9fe9e00640 1 -- 172.21.15.39:0/1376865852 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9fec15c580 msgr2=0x7f9fec15db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:19.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.051+0000 7f9fe9e00640 1 --2- 172.21.15.39:0/1376865852 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9fec15c580 0x7f9fec15db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:19.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.051+0000 7f9fe9e00640 1 -- 172.21.15.39:0/1376865852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fec0737c0 msgr2=0x7f9fec148b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:19.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.051+0000 7f9fe9e00640 1 --2- 172.21.15.39:0/1376865852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fec0737c0 0x7f9fec148b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:19.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.051+0000 7f9fe9e00640 1 -- 172.21.15.39:0/1376865852 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fec1f40d0 con 0x7f9fec072ca0 2024-09-21T07:35:19.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.051+0000 7f9fea800640 1 --2- 172.21.15.39:0/1376865852 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9fec15c580 0x7f9fec15db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:35:19.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.051+0000 7f9fe9e00640 1 --2- 172.21.15.39:0/1376865852 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fec072ca0 0x7f9fec159290 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc009850 tx=0x7f9fdc0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:19.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.051+0000 7f9fe2a00640 1 -- 172.21.15.39:0/1376865852 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc03e030 con 0x7f9fec072ca0 2024-09-21T07:35:19.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.052+0000 7f9fe2a00640 1 -- 172.21.15.39:0/1376865852 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9fdc0047a0 con 0x7f9fec072ca0 2024-09-21T07:35:19.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.052+0000 7f9fe2a00640 1 -- 172.21.15.39:0/1376865852 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc035040 con 0x7f9fec072ca0 2024-09-21T07:35:19.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.052+0000 7f9feb200640 1 -- 172.21.15.39:0/1376865852 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fec15d080 con 0x7f9fec072ca0 2024-09-21T07:35:19.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.052+0000 7f9feb200640 1 -- 172.21.15.39:0/1376865852 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fec159860 con 0x7f9fec072ca0 2024-09-21T07:35:19.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.053+0000 7f9feb200640 1 -- 172.21.15.39:0/1376865852 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fec0730b0 con 0x7f9fec072ca0 2024-09-21T07:35:19.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.054+0000 7f9fe2a00640 1 -- 172.21.15.39:0/1376865852 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9fdc041030 con 0x7f9fec072ca0 2024-09-21T07:35:19.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.055+0000 7f9fe2a00640 1 --2- 172.21.15.39:0/1376865852 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9fc8078b90 0x7f9fc807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:19.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.055+0000 7f9fe2a00640 1 -- 172.21.15.39:0/1376865852 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6062+0+0 (secure 0 0 0) 0x7f9fdc0bb660 con 0x7f9fec072ca0 2024-09-21T07:35:19.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.056+0000 7f9fe1400640 1 --2- 172.21.15.39:0/1376865852 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9fc8078b90 0x7f9fc807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:19.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.056+0000 7f9fe1400640 1 --2- 172.21.15.39:0/1376865852 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9fc8078b90 0x7f9fc807b0a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4000f60 tx=0x7f9fd4009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:19.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.059+0000 7f9fe2a00640 1 -- 172.21.15.39:0/1376865852 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9fdc083370 con 0x7f9fec072ca0 2024-09-21T07:35:19.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.297+0000 7f9feb200640 1 -- 172.21.15.39:0/1376865852 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f9fec15cd30 con 0x7f9fec072ca0 2024-09-21T07:35:19.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.798+0000 7f9fe2a00640 1 -- 172.21.15.39:0/1376865852 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v35) ==== 122+0+0 (secure 0 0 0) 0x7f9fdc087da0 con 0x7f9fec072ca0 2024-09-21T07:35:19.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.800+0000 7f9feb200640 1 -- 172.21.15.39:0/1376865852 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9fc8078b90 msgr2=0x7f9fc807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:19.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.800+0000 7f9feb200640 1 --2- 172.21.15.39:0/1376865852 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9fc8078b90 0x7f9fc807b0a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4000f60 tx=0x7f9fd4009040 comp rx=0 tx=0).stop 2024-09-21T07:35:19.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.800+0000 7f9feb200640 1 -- 172.21.15.39:0/1376865852 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fec072ca0 msgr2=0x7f9fec159290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:19.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.800+0000 7f9feb200640 1 --2- 172.21.15.39:0/1376865852 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fec072ca0 0x7f9fec159290 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc009850 tx=0x7f9fdc0039e0 comp rx=0 tx=0).stop 2024-09-21T07:35:19.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.801+0000 7f9feb200640 1 -- 172.21.15.39:0/1376865852 shutdown_connections 2024-09-21T07:35:19.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.801+0000 7f9feb200640 1 --2- 172.21.15.39:0/1376865852 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9fc8078b90 0x7f9fc807b0a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:19.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.801+0000 7f9feb200640 1 --2- 172.21.15.39:0/1376865852 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9fec15c580 0x7f9fec15db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:19.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.801+0000 7f9feb200640 1 --2- 172.21.15.39:0/1376865852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9fec0737c0 0x7f9fec148b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:19.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.801+0000 7f9feb200640 1 --2- 172.21.15.39:0/1376865852 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fec072ca0 0x7f9fec159290 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:19.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.801+0000 7f9feb200640 1 -- 172.21.15.39:0/1376865852 >> 172.21.15.39:0/1376865852 conn(0x7f9fec06ee50 msgr2=0x7f9fec155fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:19.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.801+0000 7f9feb200640 1 -- 172.21.15.39:0/1376865852 shutdown_connections 2024-09-21T07:35:19.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:19.801+0000 7f9feb200640 1 -- 172.21.15.39:0/1376865852 wait complete. 2024-09-21T07:35:19.828 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T07:35:19.828 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:35:20.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.054+0000 7f9a17a00640 1 -- 172.21.15.39:0/1600946631 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99f8005250 msgr2=0x7f99f800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:20.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.054+0000 7f9a17a00640 1 --2- 172.21.15.39:0/1600946631 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99f8005250 0x7f99f800b9c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9a100cd410 tx=0x7f99f8005830 comp rx=0 tx=0).stop 2024-09-21T07:35:20.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.054+0000 7f9a17a00640 1 -- 172.21.15.39:0/1600946631 shutdown_connections 2024-09-21T07:35:20.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.054+0000 7f9a17a00640 1 --2- 172.21.15.39:0/1600946631 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99f8005250 0x7f99f800b9c0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:20.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.054+0000 7f9a17a00640 1 -- 172.21.15.39:0/1600946631 >> 172.21.15.39:0/1600946631 conn(0x7f9a100b1cb0 msgr2=0x7f9a100b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:20.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.055+0000 7f9a17a00640 1 -- 172.21.15.39:0/1600946631 shutdown_connections 2024-09-21T07:35:20.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.055+0000 7f9a17a00640 1 -- 172.21.15.39:0/1600946631 wait complete. 2024-09-21T07:35:20.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.057+0000 7f9a17a00640 1 Processor -- start 2024-09-21T07:35:20.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.057+0000 7f9a17a00640 1 -- start start 2024-09-21T07:35:20.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.058+0000 7f9a17a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99f8005250 0x7f9a100cb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:20.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.058+0000 7f9a17a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a101f9860 0x7f9a100cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:20.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.058+0000 7f9a17a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a10155f80 0x7f9a100c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:20.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.058+0000 7f9a17a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9a10074a80 con 0x7f9a101f9860 2024-09-21T07:35:20.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.058+0000 7f9a17a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9a10074900 con 0x7f9a10155f80 2024-09-21T07:35:20.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.058+0000 7f9a17a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f9a10074c00 con 0x7f99f8005250 2024-09-21T07:35:20.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.059+0000 7f9a0fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a101f9860 0x7f9a100cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:20.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.059+0000 7f9a0fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a101f9860 0x7f9a100cfc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46620/0 (socket says 172.21.15.39:46620) 2024-09-21T07:35:20.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.059+0000 7f9a0fe00640 1 -- 172.21.15.39:0/2910556455 learned_addr learned my addr 172.21.15.39:0/2910556455 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:20.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.059+0000 7f9a15a00640 1 --2- 172.21.15.39:0/2910556455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a10155f80 0x7f9a100c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:20.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.059+0000 7f9a15000640 1 --2- 172.21.15.39:0/2910556455 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99f8005250 0x7f9a100cb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:20.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.059+0000 7f9a0fe00640 1 -- 172.21.15.39:0/2910556455 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99f8005250 msgr2=0x7f9a100cb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:20.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.059+0000 7f9a0fe00640 1 --2- 172.21.15.39:0/2910556455 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99f8005250 0x7f9a100cb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:20.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.059+0000 7f9a0fe00640 1 -- 172.21.15.39:0/2910556455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a10155f80 msgr2=0x7f9a100c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:20.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.059+0000 7f9a0fe00640 1 --2- 172.21.15.39:0/2910556455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a10155f80 0x7f9a100c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:20.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.059+0000 7f9a0fe00640 1 -- 172.21.15.39:0/2910556455 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a100d0180 con 0x7f9a101f9860 2024-09-21T07:35:20.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.059+0000 7f9a0fe00640 1 --2- 172.21.15.39:0/2910556455 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a101f9860 0x7f9a100cfc40 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f99fc011770 tx=0x7f99fc011c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:20.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.060+0000 7f9a0cc00640 1 -- 172.21.15.39:0/2910556455 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc01f070 con 0x7f9a101f9860 2024-09-21T07:35:20.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.060+0000 7f9a0cc00640 1 -- 172.21.15.39:0/2910556455 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f99fc00fde0 con 0x7f9a101f9860 2024-09-21T07:35:20.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.060+0000 7f9a17a00640 1 -- 172.21.15.39:0/2910556455 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a100ca910 con 0x7f9a101f9860 2024-09-21T07:35:20.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.060+0000 7f9a0cc00640 1 -- 172.21.15.39:0/2910556455 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc0126b0 con 0x7f9a101f9860 2024-09-21T07:35:20.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.060+0000 7f9a17a00640 1 -- 172.21.15.39:0/2910556455 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a100caed0 con 0x7f9a101f9860 2024-09-21T07:35:20.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.061+0000 7f9a17a00640 1 -- 172.21.15.39:0/2910556455 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a100d0350 con 0x7f9a101f9860 2024-09-21T07:35:20.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.062+0000 7f9a0cc00640 1 -- 172.21.15.39:0/2910556455 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f99fc022070 con 0x7f9a101f9860 2024-09-21T07:35:20.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.063+0000 7f9a0cc00640 1 --2- 172.21.15.39:0/2910556455 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f99f0078b90 0x7f99f007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:20.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.063+0000 7f9a0cc00640 1 -- 172.21.15.39:0/2910556455 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6062+0+0 (secure 0 0 0) 0x7f99fc09b7e0 con 0x7f9a101f9860 2024-09-21T07:35:20.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.063+0000 7f9a15000640 1 --2- 172.21.15.39:0/2910556455 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f99f0078b90 0x7f99f007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:20.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.064+0000 7f9a15000640 1 --2- 172.21.15.39:0/2910556455 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f99f0078b90 0x7f99f007b0a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9a04006fd0 tx=0x7f9a04006d10 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:20.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.067+0000 7f9a0cc00640 1 -- 172.21.15.39:0/2910556455 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f99fc063460 con 0x7f9a101f9860 2024-09-21T07:35:20.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.303+0000 7f9a17a00640 1 -- 172.21.15.39:0/2910556455 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a100c3860 con 0x7f9a101f9860 2024-09-21T07:35:20.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.303+0000 7f9a0cc00640 1 -- 172.21.15.39:0/2910556455 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5224 (secure 0 0 0) 0x7f99fc067e90 con 0x7f9a101f9860 2024-09-21T07:35:20.305 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:35:20.306 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":35,"btime":"2024-09-21T07:35:19:788250+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5183,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2219254706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2219254706},{"type":"v1","addr":"172.21.15.148:6836","nonce":2219254706}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5189,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3512870782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3512870782},{"type":"v1","addr":"172.21.15.148:6837","nonce":3512870782}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5551,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/761690784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":761690784},{"type":"v1","addr":"172.21.15.39:6835","nonce":761690784}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:35:16.270342+0000","modified":"2024-09-21T07:35:19.299418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5646},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5646":{"gid":5646,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/252473586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":252473586},{"type":"v1","addr":"172.21.15.39:6834","nonce":252473586}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5646,"qdb_cluster":[5646]},"id":4}]} 2024-09-21T07:35:20.306 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 35 2024-09-21T07:35:20.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.306+0000 7f9a17a00640 1 -- 172.21.15.39:0/2910556455 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f99f0078b90 msgr2=0x7f99f007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:20.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.306+0000 7f9a17a00640 1 --2- 172.21.15.39:0/2910556455 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f99f0078b90 0x7f99f007b0a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9a04006fd0 tx=0x7f9a04006d10 comp rx=0 tx=0).stop 2024-09-21T07:35:20.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.306+0000 7f9a17a00640 1 -- 172.21.15.39:0/2910556455 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a101f9860 msgr2=0x7f9a100cfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:20.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.306+0000 7f9a17a00640 1 --2- 172.21.15.39:0/2910556455 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a101f9860 0x7f9a100cfc40 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f99fc011770 tx=0x7f99fc011c30 comp rx=0 tx=0).stop 2024-09-21T07:35:20.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.306+0000 7f9a17a00640 1 -- 172.21.15.39:0/2910556455 shutdown_connections 2024-09-21T07:35:20.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.306+0000 7f9a17a00640 1 --2- 172.21.15.39:0/2910556455 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f99f0078b90 0x7f99f007b0a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:20.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.306+0000 7f9a17a00640 1 --2- 172.21.15.39:0/2910556455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a10155f80 0x7f9a100c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:20.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.306+0000 7f9a17a00640 1 --2- 172.21.15.39:0/2910556455 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a101f9860 0x7f9a100cfc40 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:20.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.306+0000 7f9a17a00640 1 --2- 172.21.15.39:0/2910556455 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99f8005250 0x7f9a100cb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:20.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.306+0000 7f9a17a00640 1 -- 172.21.15.39:0/2910556455 >> 172.21.15.39:0/2910556455 conn(0x7f9a100b1cb0 msgr2=0x7f9a1015bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:20.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.306+0000 7f9a17a00640 1 -- 172.21.15.39:0/2910556455 shutdown_connections 2024-09-21T07:35:20.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.306+0000 7f9a17a00640 1 -- 172.21.15.39:0/2910556455 wait complete. 2024-09-21T07:35:20.330 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:35:20.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.555+0000 7f4478600640 1 -- 172.21.15.39:0/45033521 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4470070030 msgr2=0x7f4470074640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:20.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.555+0000 7f4478600640 1 --2- 172.21.15.39:0/45033521 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4470070030 0x7f4470074640 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f445c005fd0 tx=0x7f445c02a540 comp rx=0 tx=0).stop 2024-09-21T07:35:20.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.555+0000 7f4478600640 1 -- 172.21.15.39:0/45033521 shutdown_connections 2024-09-21T07:35:20.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.555+0000 7f4478600640 1 --2- 172.21.15.39:0/45033521 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4470070440 0x7f4470074b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:20.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.555+0000 7f4478600640 1 --2- 172.21.15.39:0/45033521 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4470070030 0x7f4470074640 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:20.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.555+0000 7f4478600640 1 -- 172.21.15.39:0/45033521 >> 172.21.15.39:0/45033521 conn(0x7f44700b1cb0 msgr2=0x7f44700b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:20.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.555+0000 7f4478600640 1 -- 172.21.15.39:0/45033521 shutdown_connections 2024-09-21T07:35:20.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.555+0000 7f4478600640 1 -- 172.21.15.39:0/45033521 wait complete. 2024-09-21T07:35:20.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.558+0000 7f4478600640 1 Processor -- start 2024-09-21T07:35:20.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.558+0000 7f4478600640 1 -- start start 2024-09-21T07:35:20.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.559+0000 7f4478600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4470070030 0x7f4470151280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:20.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.559+0000 7f4478600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4470070440 0x7f44700d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:20.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.559+0000 7f4478600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4470073170 0x7f44700d0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:20.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.559+0000 7f4478600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f44700b3a40 con 0x7f4470070030 2024-09-21T07:35:20.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.559+0000 7f4478600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f44700b38c0 con 0x7f4470073170 2024-09-21T07:35:20.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.559+0000 7f4478600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f44700b3bc0 con 0x7f4470070440 2024-09-21T07:35:20.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.559+0000 7f4475c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4470070030 0x7f4470151280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:20.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.559+0000 7f4475c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4470070030 0x7f4470151280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46632/0 (socket says 172.21.15.39:46632) 2024-09-21T07:35:20.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.559+0000 7f4475c00640 1 -- 172.21.15.39:0/2935084414 learned_addr learned my addr 172.21.15.39:0/2935084414 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:20.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.560+0000 7f4475200640 1 --2- 172.21.15.39:0/2935084414 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4470070440 0x7f44700d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:20.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.560+0000 7f4476600640 1 --2- 172.21.15.39:0/2935084414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4470073170 0x7f44700d0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:20.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.560+0000 7f4475c00640 1 -- 172.21.15.39:0/2935084414 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4470070440 msgr2=0x7f44700d01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:20.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.560+0000 7f4475c00640 1 --2- 172.21.15.39:0/2935084414 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4470070440 0x7f44700d01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:20.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.560+0000 7f4475c00640 1 -- 172.21.15.39:0/2935084414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4470073170 msgr2=0x7f44700d0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:20.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.560+0000 7f4475c00640 1 --2- 172.21.15.39:0/2935084414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4470073170 0x7f44700d0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:20.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.560+0000 7f4475c00640 1 -- 172.21.15.39:0/2935084414 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447015b200 con 0x7f4470070030 2024-09-21T07:35:20.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.560+0000 7f4475c00640 1 --2- 172.21.15.39:0/2935084414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4470070030 0x7f4470151280 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f446400b420 tx=0x7f446400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:20.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.561+0000 7f446d600640 1 -- 172.21.15.39:0/2935084414 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4464011830 con 0x7f4470070030 2024-09-21T07:35:20.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.561+0000 7f446d600640 1 -- 172.21.15.39:0/2935084414 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4464016070 con 0x7f4470070030 2024-09-21T07:35:20.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.561+0000 7f446d600640 1 -- 172.21.15.39:0/2935084414 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4464011d60 con 0x7f4470070030 2024-09-21T07:35:20.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.561+0000 7f4478600640 1 -- 172.21.15.39:0/2935084414 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4470158ad0 con 0x7f4470070030 2024-09-21T07:35:20.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.561+0000 7f4478600640 1 -- 172.21.15.39:0/2935084414 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44700cbb40 con 0x7f4470070030 2024-09-21T07:35:20.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.562+0000 7f4478600640 1 -- 172.21.15.39:0/2935084414 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44700b4100 con 0x7f4470070030 2024-09-21T07:35:20.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.563+0000 7f446d600640 1 -- 172.21.15.39:0/2935084414 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f446401b460 con 0x7f4470070030 2024-09-21T07:35:20.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.564+0000 7f446d600640 1 --2- 172.21.15.39:0/2935084414 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4450078b90 0x7f445007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:20.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.564+0000 7f446d600640 1 -- 172.21.15.39:0/2935084414 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6062+0+0 (secure 0 0 0) 0x7f446409a960 con 0x7f4470070030 2024-09-21T07:35:20.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.565+0000 7f4475200640 1 --2- 172.21.15.39:0/2935084414 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4450078b90 0x7f445007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:20.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.565+0000 7f4475200640 1 --2- 172.21.15.39:0/2935084414 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4450078b90 0x7f445007b0a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f445c033000 tx=0x7f445c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:20.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.569+0000 7f446d600640 1 -- 172.21.15.39:0/2935084414 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4464062710 con 0x7f4470070030 2024-09-21T07:35:20.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.767+0000 7f4478600640 1 -- 172.21.15.39:0/2935084414 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f44700c3860 con 0x7f4470070030 2024-09-21T07:35:20.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.768+0000 7f446d600640 1 -- 172.21.15.39:0/2935084414 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) ==== 74+0+18481 (secure 0 0 0) 0x7f4464067140 con 0x7f4470070030 2024-09-21T07:35:20.770 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:35:20.771 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":62,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:35:17.761061+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:35:14.214410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-21T07:35:15.228034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:35:20.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.770+0000 7f4478600640 1 -- 172.21.15.39:0/2935084414 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4450078b90 msgr2=0x7f445007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:20.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.770+0000 7f4478600640 1 --2- 172.21.15.39:0/2935084414 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4450078b90 0x7f445007b0a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f445c033000 tx=0x7f445c004730 comp rx=0 tx=0).stop 2024-09-21T07:35:20.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.770+0000 7f4478600640 1 -- 172.21.15.39:0/2935084414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4470070030 msgr2=0x7f4470151280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:20.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.770+0000 7f4478600640 1 --2- 172.21.15.39:0/2935084414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4470070030 0x7f4470151280 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f446400b420 tx=0x7f446400b8e0 comp rx=0 tx=0).stop 2024-09-21T07:35:20.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.771+0000 7f4478600640 1 -- 172.21.15.39:0/2935084414 shutdown_connections 2024-09-21T07:35:20.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.771+0000 7f4478600640 1 --2- 172.21.15.39:0/2935084414 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4450078b90 0x7f445007b0a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:20.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.771+0000 7f4478600640 1 --2- 172.21.15.39:0/2935084414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4470073170 0x7f44700d0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:20.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.771+0000 7f4478600640 1 --2- 172.21.15.39:0/2935084414 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4470070440 0x7f44700d01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:20.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.771+0000 7f4478600640 1 --2- 172.21.15.39:0/2935084414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4470070030 0x7f4470151280 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:20.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.771+0000 7f4478600640 1 -- 172.21.15.39:0/2935084414 >> 172.21.15.39:0/2935084414 conn(0x7f44700b1cb0 msgr2=0x7f44700c9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:20.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.771+0000 7f4478600640 1 -- 172.21.15.39:0/2935084414 shutdown_connections 2024-09-21T07:35:20.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:20.771+0000 7f4478600640 1 -- 172.21.15.39:0/2935084414 wait complete. 2024-09-21T07:35:20.799 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T07:35:20.799 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:35:21.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.026+0000 7fdeebe00640 1 -- 172.21.15.39:0/2610276836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fded0004800 msgr2=0x7fded0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:21.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.026+0000 7fdeebe00640 1 --2- 172.21.15.39:0/2610276836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fded0004800 0x7fded0004c10 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fded4009b50 tx=0x7fded0005830 comp rx=0 tx=0).stop 2024-09-21T07:35:21.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.026+0000 7fdeeb400640 1 -- 172.21.15.39:0/2610276836 reap_dead start 2024-09-21T07:35:21.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.026+0000 7fdeebe00640 1 -- 172.21.15.39:0/2610276836 shutdown_connections 2024-09-21T07:35:21.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.026+0000 7fdeebe00640 1 -- 172.21.15.39:0/2610276836 >> 172.21.15.39:0/2610276836 conn(0x7fdeec06d870 msgr2=0x7fdeec06ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:21.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.026+0000 7fdeebe00640 1 -- 172.21.15.39:0/2610276836 shutdown_connections 2024-09-21T07:35:21.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.027+0000 7fdeebe00640 1 -- 172.21.15.39:0/2610276836 wait complete. 2024-09-21T07:35:21.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.029+0000 7fdeebe00640 1 Processor -- start 2024-09-21T07:35:21.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.029+0000 7fdeebe00640 1 -- start start 2024-09-21T07:35:21.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.030+0000 7fdeebe00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fded0004800 0x7fdeec149980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:21.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.030+0000 7fdeebe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeec0b05a0 0x7fdeec15a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:21.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.031+0000 7fdeebe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdeec14eb00 0x7fdeec15a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:21.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.031+0000 7fdeebe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdeec073fa0 con 0x7fdeec14eb00 2024-09-21T07:35:21.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.031+0000 7fdeebe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdeec073e20 con 0x7fdeec0b05a0 2024-09-21T07:35:21.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.031+0000 7fdeebe00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fdeec074120 con 0x7fded0004800 2024-09-21T07:35:21.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.031+0000 7fdeeb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdeec14eb00 0x7fdeec15a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:21.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.031+0000 7fdeeb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdeec14eb00 0x7fdeec15a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46646/0 (socket says 172.21.15.39:46646) 2024-09-21T07:35:21.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.031+0000 7fdeeb400640 1 -- 172.21.15.39:0/1276306988 learned_addr learned my addr 172.21.15.39:0/1276306988 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:21.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.031+0000 7fdee3e00640 1 --2- 172.21.15.39:0/1276306988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeec0b05a0 0x7fdeec15a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:21.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.031+0000 7fdeeaa00640 1 --2- 172.21.15.39:0/1276306988 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fded0004800 0x7fdeec149980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:21.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.031+0000 7fdeeb400640 1 -- 172.21.15.39:0/1276306988 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fded0004800 msgr2=0x7fdeec149980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:21.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.031+0000 7fdeeb400640 1 --2- 172.21.15.39:0/1276306988 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fded0004800 0x7fdeec149980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:21.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.031+0000 7fdeeb400640 1 -- 172.21.15.39:0/1276306988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeec0b05a0 msgr2=0x7fdeec15a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:21.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.031+0000 7fdeeb400640 1 --2- 172.21.15.39:0/1276306988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeec0b05a0 0x7fdeec15a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:21.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.031+0000 7fdeeb400640 1 -- 172.21.15.39:0/1276306988 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeec0b1930 con 0x7fdeec14eb00 2024-09-21T07:35:21.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.032+0000 7fdeeb400640 1 --2- 172.21.15.39:0/1276306988 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdeec14eb00 0x7fdeec15a980 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fdee4005fd0 tx=0x7fdee4010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:21.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.032+0000 7fdee2a00640 1 -- 172.21.15.39:0/1276306988 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee4004b20 con 0x7fdeec14eb00 2024-09-21T07:35:21.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.032+0000 7fdee2a00640 1 -- 172.21.15.39:0/1276306988 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdee4015070 con 0x7fdeec14eb00 2024-09-21T07:35:21.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.032+0000 7fdee2a00640 1 -- 172.21.15.39:0/1276306988 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee4003970 con 0x7fdeec14eb00 2024-09-21T07:35:21.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.032+0000 7fdeebe00640 1 -- 172.21.15.39:0/1276306988 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeec152300 con 0x7fdeec14eb00 2024-09-21T07:35:21.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.032+0000 7fdeebe00640 1 -- 172.21.15.39:0/1276306988 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeec0b1410 con 0x7fdeec14eb00 2024-09-21T07:35:21.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.033+0000 7fdeebe00640 1 -- 172.21.15.39:0/1276306988 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdeec0b1200 con 0x7fdeec14eb00 2024-09-21T07:35:21.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.034+0000 7fdee2a00640 1 -- 172.21.15.39:0/1276306988 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdee4003b10 con 0x7fdeec14eb00 2024-09-21T07:35:21.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.036+0000 7fdee2a00640 1 --2- 172.21.15.39:0/1276306988 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdec8078b90 0x7fdec807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:21.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.036+0000 7fdee2a00640 1 -- 172.21.15.39:0/1276306988 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6062+0+0 (secure 0 0 0) 0x7fdee409b900 con 0x7fdeec14eb00 2024-09-21T07:35:21.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.036+0000 7fdeeaa00640 1 --2- 172.21.15.39:0/1276306988 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdec8078b90 0x7fdec807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:21.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.037+0000 7fdeeaa00640 1 --2- 172.21.15.39:0/1276306988 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdec8078b90 0x7fdec807b0a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdedc001220 tx=0x7fdedc001110 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:21.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.040+0000 7fdee2a00640 1 -- 172.21.15.39:0/1276306988 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdee40634f0 con 0x7fdeec14eb00 2024-09-21T07:35:21.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.232+0000 7fdeebe00640 1 -- 172.21.15.39:0/1276306988 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdeec1520b0 con 0x7fdec8078b90 2024-09-21T07:35:21.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.238+0000 7fdee2a00640 1 -- 172.21.15.39:0/1276306988 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+331760 (secure 0 0 0) 0x7fdeec1520b0 con 0x7fdec8078b90 2024-09-21T07:35:21.246 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:35:21.248 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:35:21.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.243+0000 7fdeebe00640 1 -- 172.21.15.39:0/1276306988 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdec8078b90 msgr2=0x7fdec807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:21.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.243+0000 7fdeebe00640 1 --2- 172.21.15.39:0/1276306988 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdec8078b90 0x7fdec807b0a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdedc001220 tx=0x7fdedc001110 comp rx=0 tx=0).stop 2024-09-21T07:35:21.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.243+0000 7fdeebe00640 1 -- 172.21.15.39:0/1276306988 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdeec14eb00 msgr2=0x7fdeec15a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:21.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.243+0000 7fdeebe00640 1 --2- 172.21.15.39:0/1276306988 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdeec14eb00 0x7fdeec15a980 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fdee4005fd0 tx=0x7fdee4010f40 comp rx=0 tx=0).stop 2024-09-21T07:35:21.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.243+0000 7fdeebe00640 1 -- 172.21.15.39:0/1276306988 shutdown_connections 2024-09-21T07:35:21.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.243+0000 7fdeebe00640 1 --2- 172.21.15.39:0/1276306988 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdec8078b90 0x7fdec807b0a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:21.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.243+0000 7fdeebe00640 1 --2- 172.21.15.39:0/1276306988 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdeec14eb00 0x7fdeec15a980 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:21.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.243+0000 7fdeebe00640 1 --2- 172.21.15.39:0/1276306988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeec0b05a0 0x7fdeec15a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:21.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.243+0000 7fdeebe00640 1 --2- 172.21.15.39:0/1276306988 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fded0004800 0x7fdeec149980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:21.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.243+0000 7fdeebe00640 1 -- 172.21.15.39:0/1276306988 >> 172.21.15.39:0/1276306988 conn(0x7fdeec06d870 msgr2=0x7fdeec06ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:21.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.243+0000 7fdeebe00640 1 -- 172.21.15.39:0/1276306988 shutdown_connections 2024-09-21T07:35:21.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.244+0000 7fdeebe00640 1 -- 172.21.15.39:0/1276306988 wait complete. 2024-09-21T07:35:21.268 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2024-09-21T07:35:19.565469+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":133,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":136,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":256360,"kb_used_data":6840,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754718360,"statfs":{"total":773094113280,"available":772831600640,"internally_reserved":0,"allocated":7004160,"data_stored":3689848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.825339"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.792646+0000","last_change":"2024-09-21T07:35:16.780659+0000","last_active":"2024-09-21T07:35:16.792646+0000","last_peered":"2024-09-21T07:35:16.792646+0000","last_clean":"2024-09-21T07:35:16.792646+0000","last_became_active":"2024-09-21T07:35:16.780313+0000","last_became_peered":"2024-09-21T07:35:16.780313+0000","last_unstale":"2024-09-21T07:35:16.792646+0000","last_undegraded":"2024-09-21T07:35:16.792646+0000","last_fullsized":"2024-09-21T07:35:16.792646+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:50.957994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.791129+0000","last_change":"2024-09-21T07:35:16.784420+0000","last_active":"2024-09-21T07:35:16.791129+0000","last_peered":"2024-09-21T07:35:16.791129+0000","last_clean":"2024-09-21T07:35:16.791129+0000","last_became_active":"2024-09-21T07:35:16.782894+0000","last_became_peered":"2024-09-21T07:35:16.782894+0000","last_unstale":"2024-09-21T07:35:16.791129+0000","last_undegraded":"2024-09-21T07:35:16.791129+0000","last_fullsized":"2024-09-21T07:35:16.791129+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:37.394072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.789556+0000","last_change":"2024-09-21T07:35:16.783387+0000","last_active":"2024-09-21T07:35:16.789556+0000","last_peered":"2024-09-21T07:35:16.789556+0000","last_clean":"2024-09-21T07:35:16.789556+0000","last_became_active":"2024-09-21T07:35:16.780787+0000","last_became_peered":"2024-09-21T07:35:16.780787+0000","last_unstale":"2024-09-21T07:35:16.789556+0000","last_undegraded":"2024-09-21T07:35:16.789556+0000","last_fullsized":"2024-09-21T07:35:16.789556+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:20.669159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-21T07:35:15.758922+0000","last_change":"2024-09-21T07:35:15.758922+0000","last_active":"2024-09-21T07:35:15.735669+0000","last_peered":"2024-09-21T07:35:15.735669+0000","last_clean":"2024-09-21T07:35:15.735669+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:35:15.758922+0000","last_undegraded":"2024-09-21T07:35:15.758922+0000","last_fullsized":"2024-09-21T07:35:15.758922+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.797715+0000","last_change":"2024-09-21T07:35:16.792626+0000","last_active":"2024-09-21T07:35:16.797715+0000","last_peered":"2024-09-21T07:35:16.797715+0000","last_clean":"2024-09-21T07:35:16.797715+0000","last_became_active":"2024-09-21T07:35:16.792285+0000","last_became_peered":"2024-09-21T07:35:16.792285+0000","last_unstale":"2024-09-21T07:35:16.797715+0000","last_undegraded":"2024-09-21T07:35:16.797715+0000","last_fullsized":"2024-09-21T07:35:16.797715+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:37.160118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.799578+0000","last_change":"2024-09-21T07:35:16.791910+0000","last_active":"2024-09-21T07:35:16.799578+0000","last_peered":"2024-09-21T07:35:16.799578+0000","last_clean":"2024-09-21T07:35:16.799578+0000","last_became_active":"2024-09-21T07:35:16.791615+0000","last_became_peered":"2024-09-21T07:35:16.791615+0000","last_unstale":"2024-09-21T07:35:16.799578+0000","last_undegraded":"2024-09-21T07:35:16.799578+0000","last_fullsized":"2024-09-21T07:35:16.799578+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:23.252899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.780555+0000","last_change":"2024-09-21T07:35:16.774639+0000","last_active":"2024-09-21T07:35:16.780555+0000","last_peered":"2024-09-21T07:35:16.780555+0000","last_clean":"2024-09-21T07:35:16.780555+0000","last_became_active":"2024-09-21T07:35:16.773837+0000","last_became_peered":"2024-09-21T07:35:16.773837+0000","last_unstale":"2024-09-21T07:35:16.780555+0000","last_undegraded":"2024-09-21T07:35:16.780555+0000","last_fullsized":"2024-09-21T07:35:16.780555+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:11:02.899233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.791964+0000","last_change":"2024-09-21T07:35:16.785293+0000","last_active":"2024-09-21T07:35:16.791964+0000","last_peered":"2024-09-21T07:35:16.791964+0000","last_clean":"2024-09-21T07:35:16.791964+0000","last_became_active":"2024-09-21T07:35:16.784803+0000","last_became_peered":"2024-09-21T07:35:16.784803+0000","last_unstale":"2024-09-21T07:35:16.791964+0000","last_undegraded":"2024-09-21T07:35:16.791964+0000","last_fullsized":"2024-09-21T07:35:16.791964+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:56.522045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.788943+0000","last_change":"2024-09-21T07:35:16.777289+0000","last_active":"2024-09-21T07:35:16.788943+0000","last_peered":"2024-09-21T07:35:16.788943+0000","last_clean":"2024-09-21T07:35:16.788943+0000","last_became_active":"2024-09-21T07:35:16.776434+0000","last_became_peered":"2024-09-21T07:35:16.776434+0000","last_unstale":"2024-09-21T07:35:16.788943+0000","last_undegraded":"2024-09-21T07:35:16.788943+0000","last_fullsized":"2024-09-21T07:35:16.788943+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:57.705627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.790050+0000","last_change":"2024-09-21T07:35:16.787398+0000","last_active":"2024-09-21T07:35:16.790050+0000","last_peered":"2024-09-21T07:35:16.790050+0000","last_clean":"2024-09-21T07:35:16.790050+0000","last_became_active":"2024-09-21T07:35:16.787011+0000","last_became_peered":"2024-09-21T07:35:16.787011+0000","last_unstale":"2024-09-21T07:35:16.790050+0000","last_undegraded":"2024-09-21T07:35:16.790050+0000","last_fullsized":"2024-09-21T07:35:16.790050+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:46.011177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-21T07:35:15.762353+0000","last_change":"2024-09-21T07:35:15.762353+0000","last_active":"2024-09-21T07:35:15.735669+0000","last_peered":"2024-09-21T07:35:15.735669+0000","last_clean":"2024-09-21T07:35:15.735669+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:35:15.762353+0000","last_undegraded":"2024-09-21T07:35:15.762353+0000","last_fullsized":"2024-09-21T07:35:15.762353+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.791813+0000","last_change":"2024-09-21T07:35:16.783310+0000","last_active":"2024-09-21T07:35:16.791813+0000","last_peered":"2024-09-21T07:35:16.791813+0000","last_clean":"2024-09-21T07:35:16.791813+0000","last_became_active":"2024-09-21T07:35:16.783000+0000","last_became_peered":"2024-09-21T07:35:16.783000+0000","last_unstale":"2024-09-21T07:35:16.791813+0000","last_undegraded":"2024-09-21T07:35:16.791813+0000","last_fullsized":"2024-09-21T07:35:16.791813+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:51.769069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.782698+0000","last_change":"2024-09-21T07:35:16.779103+0000","last_active":"2024-09-21T07:35:16.782698+0000","last_peered":"2024-09-21T07:35:16.782698+0000","last_clean":"2024-09-21T07:35:16.782698+0000","last_became_active":"2024-09-21T07:35:16.778797+0000","last_became_peered":"2024-09-21T07:35:16.778797+0000","last_unstale":"2024-09-21T07:35:16.782698+0000","last_undegraded":"2024-09-21T07:35:16.782698+0000","last_fullsized":"2024-09-21T07:35:16.782698+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:01.158445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.789575+0000","last_change":"2024-09-21T07:35:16.778487+0000","last_active":"2024-09-21T07:35:16.789575+0000","last_peered":"2024-09-21T07:35:16.789575+0000","last_clean":"2024-09-21T07:35:16.789575+0000","last_became_active":"2024-09-21T07:35:16.778114+0000","last_became_peered":"2024-09-21T07:35:16.778114+0000","last_unstale":"2024-09-21T07:35:16.789575+0000","last_undegraded":"2024-09-21T07:35:16.789575+0000","last_fullsized":"2024-09-21T07:35:16.789575+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:18.986538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.790206+0000","last_change":"2024-09-21T07:35:16.787501+0000","last_active":"2024-09-21T07:35:16.790206+0000","last_peered":"2024-09-21T07:35:16.790206+0000","last_clean":"2024-09-21T07:35:16.790206+0000","last_became_active":"2024-09-21T07:35:16.784848+0000","last_became_peered":"2024-09-21T07:35:16.784848+0000","last_unstale":"2024-09-21T07:35:16.790206+0000","last_undegraded":"2024-09-21T07:35:16.790206+0000","last_fullsized":"2024-09-21T07:35:16.790206+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:00.824167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.775312+0000","last_change":"2024-09-21T07:35:16.762563+0000","last_active":"2024-09-21T07:35:16.775312+0000","last_peered":"2024-09-21T07:35:16.775312+0000","last_clean":"2024-09-21T07:35:16.775312+0000","last_became_active":"2024-09-21T07:35:16.761910+0000","last_became_peered":"2024-09-21T07:35:16.761910+0000","last_unstale":"2024-09-21T07:35:16.775312+0000","last_undegraded":"2024-09-21T07:35:16.775312+0000","last_fullsized":"2024-09-21T07:35:16.775312+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:57.759094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.787316+0000","last_change":"2024-09-21T07:35:16.780085+0000","last_active":"2024-09-21T07:35:16.787316+0000","last_peered":"2024-09-21T07:35:16.787316+0000","last_clean":"2024-09-21T07:35:16.787316+0000","last_became_active":"2024-09-21T07:35:16.779774+0000","last_became_peered":"2024-09-21T07:35:16.779774+0000","last_unstale":"2024-09-21T07:35:16.787316+0000","last_undegraded":"2024-09-21T07:35:16.787316+0000","last_fullsized":"2024-09-21T07:35:16.787316+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:04:15.356743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-21T07:35:15.761429+0000","last_change":"2024-09-21T07:35:15.761429+0000","last_active":"2024-09-21T07:35:15.735669+0000","last_peered":"2024-09-21T07:35:15.735669+0000","last_clean":"2024-09-21T07:35:15.735669+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:35:15.761429+0000","last_undegraded":"2024-09-21T07:35:15.761429+0000","last_fullsized":"2024-09-21T07:35:15.761429+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.792282+0000","last_change":"2024-09-21T07:35:16.786945+0000","last_active":"2024-09-21T07:35:16.792282+0000","last_peered":"2024-09-21T07:35:16.792282+0000","last_clean":"2024-09-21T07:35:16.792282+0000","last_became_active":"2024-09-21T07:35:16.786605+0000","last_became_peered":"2024-09-21T07:35:16.786605+0000","last_unstale":"2024-09-21T07:35:16.792282+0000","last_undegraded":"2024-09-21T07:35:16.792282+0000","last_fullsized":"2024-09-21T07:35:16.792282+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:28:00.845053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-21T07:35:15.763567+0000","last_change":"2024-09-21T07:35:15.763567+0000","last_active":"2024-09-21T07:35:15.735669+0000","last_peered":"2024-09-21T07:35:15.735669+0000","last_clean":"2024-09-21T07:35:15.735669+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:35:15.763567+0000","last_undegraded":"2024-09-21T07:35:15.763567+0000","last_fullsized":"2024-09-21T07:35:15.763567+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.791882+0000","last_change":"2024-09-21T07:35:16.784386+0000","last_active":"2024-09-21T07:35:16.791882+0000","last_peered":"2024-09-21T07:35:16.791882+0000","last_clean":"2024-09-21T07:35:16.791882+0000","last_became_active":"2024-09-21T07:35:16.783981+0000","last_became_peered":"2024-09-21T07:35:16.783981+0000","last_unstale":"2024-09-21T07:35:16.791882+0000","last_undegraded":"2024-09-21T07:35:16.791882+0000","last_fullsized":"2024-09-21T07:35:16.791882+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:40.131273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-21T07:35:15.763243+0000","last_change":"2024-09-21T07:35:15.763243+0000","last_active":"2024-09-21T07:35:15.735669+0000","last_peered":"2024-09-21T07:35:15.735669+0000","last_clean":"2024-09-21T07:35:15.735669+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:35:15.763243+0000","last_undegraded":"2024-09-21T07:35:15.763243+0000","last_fullsized":"2024-09-21T07:35:15.763243+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-21T07:35:15.765074+0000","last_change":"2024-09-21T07:35:15.765074+0000","last_active":"2024-09-21T07:35:15.735669+0000","last_peered":"2024-09-21T07:35:15.735669+0000","last_clean":"2024-09-21T07:35:15.735669+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:35:15.765074+0000","last_undegraded":"2024-09-21T07:35:15.765074+0000","last_fullsized":"2024-09-21T07:35:15.765074+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.789141+0000","last_change":"2024-09-21T07:35:16.780348+0000","last_active":"2024-09-21T07:35:16.789141+0000","last_peered":"2024-09-21T07:35:16.789141+0000","last_clean":"2024-09-21T07:35:16.789141+0000","last_became_active":"2024-09-21T07:35:16.780006+0000","last_became_peered":"2024-09-21T07:35:16.780006+0000","last_unstale":"2024-09-21T07:35:16.789141+0000","last_undegraded":"2024-09-21T07:35:16.789141+0000","last_fullsized":"2024-09-21T07:35:16.789141+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:38.181721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.775059+0000","last_change":"2024-09-21T07:35:16.761639+0000","last_active":"2024-09-21T07:35:16.775059+0000","last_peered":"2024-09-21T07:35:16.775059+0000","last_clean":"2024-09-21T07:35:16.775059+0000","last_became_active":"2024-09-21T07:35:16.761231+0000","last_became_peered":"2024-09-21T07:35:16.761231+0000","last_unstale":"2024-09-21T07:35:16.775059+0000","last_undegraded":"2024-09-21T07:35:16.775059+0000","last_fullsized":"2024-09-21T07:35:16.775059+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:41.477998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.775504+0000","last_change":"2024-09-21T07:35:16.769703+0000","last_active":"2024-09-21T07:35:16.775504+0000","last_peered":"2024-09-21T07:35:16.775504+0000","last_clean":"2024-09-21T07:35:16.775504+0000","last_became_active":"2024-09-21T07:35:16.769306+0000","last_became_peered":"2024-09-21T07:35:16.769306+0000","last_unstale":"2024-09-21T07:35:16.775504+0000","last_undegraded":"2024-09-21T07:35:16.775504+0000","last_fullsized":"2024-09-21T07:35:16.775504+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:02.221815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-21T07:35:15.760207+0000","last_change":"2024-09-21T07:35:15.760207+0000","last_active":"2024-09-21T07:35:15.735669+0000","last_peered":"2024-09-21T07:35:15.735669+0000","last_clean":"2024-09-21T07:35:15.735669+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:35:15.760207+0000","last_undegraded":"2024-09-21T07:35:15.760207+0000","last_fullsized":"2024-09-21T07:35:15.760207+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.799668+0000","last_change":"2024-09-21T07:35:16.792511+0000","last_active":"2024-09-21T07:35:16.799668+0000","last_peered":"2024-09-21T07:35:16.799668+0000","last_clean":"2024-09-21T07:35:16.799668+0000","last_became_active":"2024-09-21T07:35:16.792095+0000","last_became_peered":"2024-09-21T07:35:16.792095+0000","last_unstale":"2024-09-21T07:35:16.799668+0000","last_undegraded":"2024-09-21T07:35:16.799668+0000","last_fullsized":"2024-09-21T07:35:16.799668+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:56.522045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.789095+0000","last_change":"2024-09-21T07:35:16.779904+0000","last_active":"2024-09-21T07:35:16.789095+0000","last_peered":"2024-09-21T07:35:16.789095+0000","last_clean":"2024-09-21T07:35:16.789095+0000","last_became_active":"2024-09-21T07:35:16.779548+0000","last_became_peered":"2024-09-21T07:35:16.779548+0000","last_unstale":"2024-09-21T07:35:16.789095+0000","last_undegraded":"2024-09-21T07:35:16.789095+0000","last_fullsized":"2024-09-21T07:35:16.789095+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:17.442893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.792032+0000","last_change":"2024-09-21T07:35:16.787033+0000","last_active":"2024-09-21T07:35:16.792032+0000","last_peered":"2024-09-21T07:35:16.792032+0000","last_clean":"2024-09-21T07:35:16.792032+0000","last_became_active":"2024-09-21T07:35:16.784639+0000","last_became_peered":"2024-09-21T07:35:16.784639+0000","last_unstale":"2024-09-21T07:35:16.792032+0000","last_undegraded":"2024-09-21T07:35:16.792032+0000","last_fullsized":"2024-09-21T07:35:16.792032+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:19:41.151420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-21T07:35:15.757448+0000","last_change":"2024-09-21T07:35:15.757448+0000","last_active":"2024-09-21T07:35:15.735669+0000","last_peered":"2024-09-21T07:35:15.735669+0000","last_clean":"2024-09-21T07:35:15.735669+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:35:15.757448+0000","last_undegraded":"2024-09-21T07:35:15.757448+0000","last_fullsized":"2024-09-21T07:35:15.757448+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.793719+0000","last_change":"2024-09-21T07:35:16.781495+0000","last_active":"2024-09-21T07:35:16.793719+0000","last_peered":"2024-09-21T07:35:16.793719+0000","last_clean":"2024-09-21T07:35:16.793719+0000","last_became_active":"2024-09-21T07:35:16.781096+0000","last_became_peered":"2024-09-21T07:35:16.781096+0000","last_unstale":"2024-09-21T07:35:16.793719+0000","last_undegraded":"2024-09-21T07:35:16.793719+0000","last_fullsized":"2024-09-21T07:35:16.793719+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:19:53.457207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.797687+0000","last_change":"2024-09-21T07:35:16.790402+0000","last_active":"2024-09-21T07:35:16.797687+0000","last_peered":"2024-09-21T07:35:16.797687+0000","last_clean":"2024-09-21T07:35:16.797687+0000","last_became_active":"2024-09-21T07:35:16.786935+0000","last_became_peered":"2024-09-21T07:35:16.786935+0000","last_unstale":"2024-09-21T07:35:16.797687+0000","last_undegraded":"2024-09-21T07:35:16.797687+0000","last_fullsized":"2024-09-21T07:35:16.797687+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:12:35.499903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:35:15.776056+0000","last_change":"2024-09-21T07:35:15.772382+0000","last_active":"2024-09-21T07:35:15.776056+0000","last_peered":"2024-09-21T07:35:15.776056+0000","last_clean":"2024-09-21T07:35:15.776056+0000","last_became_active":"2024-09-21T07:35:15.771916+0000","last_became_peered":"2024-09-21T07:35:15.771916+0000","last_unstale":"2024-09-21T07:35:15.776056+0000","last_undegraded":"2024-09-21T07:35:15.776056+0000","last_fullsized":"2024-09-21T07:35:15.776056+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:52.209917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":188,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.764544+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:35:16.764544+0000","last_peered":"2024-09-21T07:35:16.764544+0000","last_clean":"2024-09-21T07:35:16.764544+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:35:16.764544+0000","last_undegraded":"2024-09-21T07:35:16.764544+0000","last_fullsized":"2024-09-21T07:35:16.764544+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":62,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.755390+0000","last_change":"2024-09-21T07:34:24.539248+0000","last_active":"2024-09-21T07:35:16.755390+0000","last_peered":"2024-09-21T07:35:16.755390+0000","last_clean":"2024-09-21T07:35:16.755390+0000","last_became_active":"2024-09-21T07:34:23.273465+0000","last_became_peered":"2024-09-21T07:34:23.273465+0000","last_unstale":"2024-09-21T07:35:16.755390+0000","last_undegraded":"2024-09-21T07:35:16.755390+0000","last_fullsized":"2024-09-21T07:35:16.755390+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:55.517452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060627700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-21T07:35:15.760866+0000","last_change":"2024-09-21T07:35:15.760866+0000","last_active":"2024-09-21T07:35:15.735669+0000","last_peered":"2024-09-21T07:35:15.735669+0000","last_clean":"2024-09-21T07:35:15.735669+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:35:15.760866+0000","last_undegraded":"2024-09-21T07:35:15.760866+0000","last_fullsized":"2024-09-21T07:35:15.760866+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"61'1","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.795434+0000","last_change":"2024-09-21T07:35:15.777735+0000","last_active":"2024-09-21T07:35:16.795434+0000","last_peered":"2024-09-21T07:35:16.795434+0000","last_clean":"2024-09-21T07:35:16.795434+0000","last_became_active":"2024-09-21T07:35:15.777411+0000","last_became_peered":"2024-09-21T07:35:15.777411+0000","last_unstale":"2024-09-21T07:35:16.795434+0000","last_undegraded":"2024-09-21T07:35:16.795434+0000","last_fullsized":"2024-09-21T07:35:16.795434+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:51:55.267013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":57,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:35:15.769378+0000","last_change":"2024-09-21T07:34:24.597274+0000","last_active":"2024-09-21T07:35:15.769378+0000","last_peered":"2024-09-21T07:35:15.769378+0000","last_clean":"2024-09-21T07:35:15.769378+0000","last_became_active":"2024-09-21T07:34:23.279560+0000","last_became_peered":"2024-09-21T07:34:23.279560+0000","last_unstale":"2024-09-21T07:35:15.769378+0000","last_undegraded":"2024-09-21T07:35:15.769378+0000","last_fullsized":"2024-09-21T07:35:15.769378+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:15.845265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059699799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.788623+0000","last_change":"2024-09-21T07:35:16.778041+0000","last_active":"2024-09-21T07:35:16.788623+0000","last_peered":"2024-09-21T07:35:16.788623+0000","last_clean":"2024-09-21T07:35:16.788623+0000","last_became_active":"2024-09-21T07:35:16.777265+0000","last_became_peered":"2024-09-21T07:35:16.777265+0000","last_unstale":"2024-09-21T07:35:16.788623+0000","last_undegraded":"2024-09-21T07:35:16.788623+0000","last_fullsized":"2024-09-21T07:35:16.788623+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:00:49.777067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:35:15.775987+0000","last_change":"2024-09-21T07:35:15.771748+0000","last_active":"2024-09-21T07:35:15.775987+0000","last_peered":"2024-09-21T07:35:15.775987+0000","last_clean":"2024-09-21T07:35:15.775987+0000","last_became_active":"2024-09-21T07:35:15.771453+0000","last_became_peered":"2024-09-21T07:35:15.771453+0000","last_unstale":"2024-09-21T07:35:15.775987+0000","last_undegraded":"2024-09-21T07:35:15.775987+0000","last_fullsized":"2024-09-21T07:35:15.775987+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:37.809143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":63,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.764742+0000","last_change":"2024-09-21T07:34:24.259906+0000","last_active":"2024-09-21T07:35:16.764742+0000","last_peered":"2024-09-21T07:35:16.764742+0000","last_clean":"2024-09-21T07:35:16.764742+0000","last_became_active":"2024-09-21T07:34:23.635282+0000","last_became_peered":"2024-09-21T07:34:23.635282+0000","last_unstale":"2024-09-21T07:35:16.764742+0000","last_undegraded":"2024-09-21T07:35:16.764742+0000","last_fullsized":"2024-09-21T07:35:16.764742+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:00.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001527664,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.788032+0000","last_change":"2024-09-21T07:35:16.778542+0000","last_active":"2024-09-21T07:35:16.788032+0000","last_peered":"2024-09-21T07:35:16.788032+0000","last_clean":"2024-09-21T07:35:16.788032+0000","last_became_active":"2024-09-21T07:35:16.778202+0000","last_became_peered":"2024-09-21T07:35:16.778202+0000","last_unstale":"2024-09-21T07:35:16.788032+0000","last_undegraded":"2024-09-21T07:35:16.788032+0000","last_fullsized":"2024-09-21T07:35:16.788032+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:33.739027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"61'1","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.787864+0000","last_change":"2024-09-21T07:35:15.769004+0000","last_active":"2024-09-21T07:35:16.787864+0000","last_peered":"2024-09-21T07:35:16.787864+0000","last_clean":"2024-09-21T07:35:16.787864+0000","last_became_active":"2024-09-21T07:35:15.768705+0000","last_became_peered":"2024-09-21T07:35:15.768705+0000","last_unstale":"2024-09-21T07:35:16.787864+0000","last_undegraded":"2024-09-21T07:35:16.787864+0000","last_fullsized":"2024-09-21T07:35:16.787864+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:16.143968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":60,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.764496+0000","last_change":"2024-09-21T07:34:25.085454+0000","last_active":"2024-09-21T07:35:16.764496+0000","last_peered":"2024-09-21T07:35:16.764496+0000","last_clean":"2024-09-21T07:35:16.764496+0000","last_became_active":"2024-09-21T07:34:23.279205+0000","last_became_peered":"2024-09-21T07:34:23.279205+0000","last_unstale":"2024-09-21T07:35:16.764496+0000","last_undegraded":"2024-09-21T07:35:16.764496+0000","last_fullsized":"2024-09-21T07:35:16.764496+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:54.231666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095320599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.792070+0000","last_change":"2024-09-21T07:35:16.782202+0000","last_active":"2024-09-21T07:35:16.792070+0000","last_peered":"2024-09-21T07:35:16.792070+0000","last_clean":"2024-09-21T07:35:16.792070+0000","last_became_active":"2024-09-21T07:35:16.781868+0000","last_became_peered":"2024-09-21T07:35:16.781868+0000","last_unstale":"2024-09-21T07:35:16.792070+0000","last_undegraded":"2024-09-21T07:35:16.792070+0000","last_fullsized":"2024-09-21T07:35:16.792070+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:44:08.510363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.791376+0000","last_change":"2024-09-21T07:35:15.778418+0000","last_active":"2024-09-21T07:35:16.791376+0000","last_peered":"2024-09-21T07:35:16.791376+0000","last_clean":"2024-09-21T07:35:16.791376+0000","last_became_active":"2024-09-21T07:35:15.778001+0000","last_became_peered":"2024-09-21T07:35:15.778001+0000","last_unstale":"2024-09-21T07:35:16.791376+0000","last_undegraded":"2024-09-21T07:35:16.791376+0000","last_fullsized":"2024-09-21T07:35:16.791376+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:38.435523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":61,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.777782+0000","last_change":"2024-09-21T07:34:25.028521+0000","last_active":"2024-09-21T07:35:16.777782+0000","last_peered":"2024-09-21T07:35:16.777782+0000","last_clean":"2024-09-21T07:35:16.777782+0000","last_became_active":"2024-09-21T07:34:23.635467+0000","last_became_peered":"2024-09-21T07:34:23.635467+0000","last_unstale":"2024-09-21T07:35:16.777782+0000","last_undegraded":"2024-09-21T07:35:16.777782+0000","last_fullsized":"2024-09-21T07:35:16.777782+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:33.729123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000660558,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.789071+0000","last_change":"2024-09-21T07:35:16.781562+0000","last_active":"2024-09-21T07:35:16.789071+0000","last_peered":"2024-09-21T07:35:16.789071+0000","last_clean":"2024-09-21T07:35:16.789071+0000","last_became_active":"2024-09-21T07:35:16.780384+0000","last_became_peered":"2024-09-21T07:35:16.780384+0000","last_unstale":"2024-09-21T07:35:16.789071+0000","last_undegraded":"2024-09-21T07:35:16.789071+0000","last_fullsized":"2024-09-21T07:35:16.789071+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:57:16.255292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.768020+0000","last_change":"2024-09-21T07:35:15.771439+0000","last_active":"2024-09-21T07:35:16.768020+0000","last_peered":"2024-09-21T07:35:16.768020+0000","last_clean":"2024-09-21T07:35:16.768020+0000","last_became_active":"2024-09-21T07:35:15.771176+0000","last_became_peered":"2024-09-21T07:35:15.771176+0000","last_unstale":"2024-09-21T07:35:16.768020+0000","last_undegraded":"2024-09-21T07:35:16.768020+0000","last_fullsized":"2024-09-21T07:35:16.768020+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:30.194293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":62,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:35:15.758617+0000","last_change":"2024-09-21T07:34:24.260046+0000","last_active":"2024-09-21T07:35:15.758617+0000","last_peered":"2024-09-21T07:35:15.758617+0000","last_clean":"2024-09-21T07:35:15.758617+0000","last_became_active":"2024-09-21T07:34:23.273431+0000","last_became_peered":"2024-09-21T07:34:23.273431+0000","last_unstale":"2024-09-21T07:35:15.758617+0000","last_undegraded":"2024-09-21T07:35:15.758617+0000","last_fullsized":"2024-09-21T07:35:15.758617+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:18.876057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019803310000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.793747+0000","last_change":"2024-09-21T07:35:16.787188+0000","last_active":"2024-09-21T07:35:16.793747+0000","last_peered":"2024-09-21T07:35:16.793747+0000","last_clean":"2024-09-21T07:35:16.793747+0000","last_became_active":"2024-09-21T07:35:16.786877+0000","last_became_peered":"2024-09-21T07:35:16.786877+0000","last_unstale":"2024-09-21T07:35:16.793747+0000","last_undegraded":"2024-09-21T07:35:16.793747+0000","last_fullsized":"2024-09-21T07:35:16.793747+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:29:42.858624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.789751+0000","last_change":"2024-09-21T07:35:15.773125+0000","last_active":"2024-09-21T07:35:16.789751+0000","last_peered":"2024-09-21T07:35:16.789751+0000","last_clean":"2024-09-21T07:35:16.789751+0000","last_became_active":"2024-09-21T07:35:15.772316+0000","last_became_peered":"2024-09-21T07:35:15.772316+0000","last_unstale":"2024-09-21T07:35:16.789751+0000","last_undegraded":"2024-09-21T07:35:16.789751+0000","last_fullsized":"2024-09-21T07:35:16.789751+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:59.264954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":61,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.785693+0000","last_change":"2024-09-21T07:34:25.028640+0000","last_active":"2024-09-21T07:35:16.785693+0000","last_peered":"2024-09-21T07:35:16.785693+0000","last_clean":"2024-09-21T07:35:16.785693+0000","last_became_active":"2024-09-21T07:34:23.260950+0000","last_became_peered":"2024-09-21T07:34:23.260950+0000","last_unstale":"2024-09-21T07:35:16.785693+0000","last_undegraded":"2024-09-21T07:35:16.785693+0000","last_fullsized":"2024-09-21T07:35:16.785693+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:23:42.121398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055135099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.765442+0000","last_change":"2024-09-21T07:35:16.761498+0000","last_active":"2024-09-21T07:35:16.765442+0000","last_peered":"2024-09-21T07:35:16.765442+0000","last_clean":"2024-09-21T07:35:16.765442+0000","last_became_active":"2024-09-21T07:35:16.761105+0000","last_became_peered":"2024-09-21T07:35:16.761105+0000","last_unstale":"2024-09-21T07:35:16.765442+0000","last_undegraded":"2024-09-21T07:35:16.765442+0000","last_fullsized":"2024-09-21T07:35:16.765442+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:50.957994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.755285+0000","last_change":"2024-09-21T07:35:15.763918+0000","last_active":"2024-09-21T07:35:16.755285+0000","last_peered":"2024-09-21T07:35:16.755285+0000","last_clean":"2024-09-21T07:35:16.755285+0000","last_became_active":"2024-09-21T07:35:15.763596+0000","last_became_peered":"2024-09-21T07:35:15.763596+0000","last_unstale":"2024-09-21T07:35:16.755285+0000","last_undegraded":"2024-09-21T07:35:16.755285+0000","last_fullsized":"2024-09-21T07:35:16.755285+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:37.955816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":63,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.764202+0000","last_change":"2024-09-21T07:34:24.260665+0000","last_active":"2024-09-21T07:35:16.764202+0000","last_peered":"2024-09-21T07:35:16.764202+0000","last_clean":"2024-09-21T07:35:16.764202+0000","last_became_active":"2024-09-21T07:34:23.635909+0000","last_became_peered":"2024-09-21T07:34:23.635909+0000","last_unstale":"2024-09-21T07:35:16.764202+0000","last_undegraded":"2024-09-21T07:35:16.764202+0000","last_fullsized":"2024-09-21T07:35:16.764202+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:44.536885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.782220+0000","last_change":"2024-09-21T07:35:16.778786+0000","last_active":"2024-09-21T07:35:16.782220+0000","last_peered":"2024-09-21T07:35:16.782220+0000","last_clean":"2024-09-21T07:35:16.782220+0000","last_became_active":"2024-09-21T07:35:16.778450+0000","last_became_peered":"2024-09-21T07:35:16.778450+0000","last_unstale":"2024-09-21T07:35:16.782220+0000","last_undegraded":"2024-09-21T07:35:16.782220+0000","last_fullsized":"2024-09-21T07:35:16.782220+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:37.394072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.767888+0000","last_change":"2024-09-21T07:35:15.762424+0000","last_active":"2024-09-21T07:35:16.767888+0000","last_peered":"2024-09-21T07:35:16.767888+0000","last_clean":"2024-09-21T07:35:16.767888+0000","last_became_active":"2024-09-21T07:35:15.762109+0000","last_became_peered":"2024-09-21T07:35:15.762109+0000","last_unstale":"2024-09-21T07:35:16.767888+0000","last_undegraded":"2024-09-21T07:35:16.767888+0000","last_fullsized":"2024-09-21T07:35:16.767888+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:00.093906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":62,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.755314+0000","last_change":"2024-09-21T07:34:24.538845+0000","last_active":"2024-09-21T07:35:16.755314+0000","last_peered":"2024-09-21T07:35:16.755314+0000","last_clean":"2024-09-21T07:35:16.755314+0000","last_became_active":"2024-09-21T07:34:23.636372+0000","last_became_peered":"2024-09-21T07:34:23.636372+0000","last_unstale":"2024-09-21T07:35:16.755314+0000","last_undegraded":"2024-09-21T07:35:16.755314+0000","last_fullsized":"2024-09-21T07:35:16.755314+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:29.393813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841451,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.788577+0000","last_change":"2024-09-21T07:35:16.777683+0000","last_active":"2024-09-21T07:35:16.788577+0000","last_peered":"2024-09-21T07:35:16.788577+0000","last_clean":"2024-09-21T07:35:16.788577+0000","last_became_active":"2024-09-21T07:35:16.776971+0000","last_became_peered":"2024-09-21T07:35:16.776971+0000","last_unstale":"2024-09-21T07:35:16.788577+0000","last_undegraded":"2024-09-21T07:35:16.788577+0000","last_fullsized":"2024-09-21T07:35:16.788577+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:44:47.941274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.785884+0000","last_change":"2024-09-21T07:35:15.773061+0000","last_active":"2024-09-21T07:35:16.785884+0000","last_peered":"2024-09-21T07:35:16.785884+0000","last_clean":"2024-09-21T07:35:16.785884+0000","last_became_active":"2024-09-21T07:35:15.772705+0000","last_became_peered":"2024-09-21T07:35:15.772705+0000","last_unstale":"2024-09-21T07:35:16.785884+0000","last_undegraded":"2024-09-21T07:35:16.785884+0000","last_fullsized":"2024-09-21T07:35:16.785884+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:50.766243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":62,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:35:15.758011+0000","last_change":"2024-09-21T07:34:24.259686+0000","last_active":"2024-09-21T07:35:15.758011+0000","last_peered":"2024-09-21T07:35:15.758011+0000","last_clean":"2024-09-21T07:35:15.758011+0000","last_became_active":"2024-09-21T07:34:23.274405+0000","last_became_peered":"2024-09-21T07:34:23.274405+0000","last_unstale":"2024-09-21T07:35:15.758011+0000","last_undegraded":"2024-09-21T07:35:15.758011+0000","last_fullsized":"2024-09-21T07:35:15.758011+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:26.304693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022009920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.787809+0000","last_change":"2024-09-21T07:35:16.779064+0000","last_active":"2024-09-21T07:35:16.787809+0000","last_peered":"2024-09-21T07:35:16.787809+0000","last_clean":"2024-09-21T07:35:16.787809+0000","last_became_active":"2024-09-21T07:35:16.778764+0000","last_became_peered":"2024-09-21T07:35:16.778764+0000","last_unstale":"2024-09-21T07:35:16.787809+0000","last_undegraded":"2024-09-21T07:35:16.787809+0000","last_fullsized":"2024-09-21T07:35:16.787809+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:43.382121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.793246+0000","last_change":"2024-09-21T07:35:15.771995+0000","last_active":"2024-09-21T07:35:16.793246+0000","last_peered":"2024-09-21T07:35:16.793246+0000","last_clean":"2024-09-21T07:35:16.793246+0000","last_became_active":"2024-09-21T07:35:15.771691+0000","last_became_peered":"2024-09-21T07:35:15.771691+0000","last_unstale":"2024-09-21T07:35:16.793246+0000","last_undegraded":"2024-09-21T07:35:16.793246+0000","last_fullsized":"2024-09-21T07:35:16.793246+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:04:14.353917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":63,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.764414+0000","last_change":"2024-09-21T07:34:24.258754+0000","last_active":"2024-09-21T07:35:16.764414+0000","last_peered":"2024-09-21T07:35:16.764414+0000","last_clean":"2024-09-21T07:35:16.764414+0000","last_became_active":"2024-09-21T07:34:23.272657+0000","last_became_peered":"2024-09-21T07:34:23.272657+0000","last_unstale":"2024-09-21T07:35:16.764414+0000","last_undegraded":"2024-09-21T07:35:16.764414+0000","last_fullsized":"2024-09-21T07:35:16.764414+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:05.477099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067143200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.781770+0000","last_change":"2024-09-21T07:35:16.779289+0000","last_active":"2024-09-21T07:35:16.781770+0000","last_peered":"2024-09-21T07:35:16.781770+0000","last_clean":"2024-09-21T07:35:16.781770+0000","last_became_active":"2024-09-21T07:35:16.778146+0000","last_became_peered":"2024-09-21T07:35:16.778146+0000","last_unstale":"2024-09-21T07:35:16.781770+0000","last_undegraded":"2024-09-21T07:35:16.781770+0000","last_fullsized":"2024-09-21T07:35:16.781770+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:51.294357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.754996+0000","last_change":"2024-09-21T07:35:15.757150+0000","last_active":"2024-09-21T07:35:16.754996+0000","last_peered":"2024-09-21T07:35:16.754996+0000","last_clean":"2024-09-21T07:35:16.754996+0000","last_became_active":"2024-09-21T07:35:15.756816+0000","last_became_peered":"2024-09-21T07:35:15.756816+0000","last_unstale":"2024-09-21T07:35:16.754996+0000","last_undegraded":"2024-09-21T07:35:16.754996+0000","last_fullsized":"2024-09-21T07:35:16.754996+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:52.469665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":61,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.768191+0000","last_change":"2024-09-21T07:34:25.085232+0000","last_active":"2024-09-21T07:35:16.768191+0000","last_peered":"2024-09-21T07:35:16.768191+0000","last_clean":"2024-09-21T07:35:16.768191+0000","last_became_active":"2024-09-21T07:34:23.277807+0000","last_became_peered":"2024-09-21T07:34:23.277807+0000","last_unstale":"2024-09-21T07:35:16.768191+0000","last_undegraded":"2024-09-21T07:35:16.768191+0000","last_fullsized":"2024-09-21T07:35:16.768191+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:36.362919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086619099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.790466+0000","last_change":"2024-09-21T07:35:16.783210+0000","last_active":"2024-09-21T07:35:16.790466+0000","last_peered":"2024-09-21T07:35:16.790466+0000","last_clean":"2024-09-21T07:35:16.790466+0000","last_became_active":"2024-09-21T07:35:16.782534+0000","last_became_peered":"2024-09-21T07:35:16.782534+0000","last_unstale":"2024-09-21T07:35:16.790466+0000","last_undegraded":"2024-09-21T07:35:16.790466+0000","last_fullsized":"2024-09-21T07:35:16.790466+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:02.221815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:35:15.780612+0000","last_change":"2024-09-21T07:35:15.774096+0000","last_active":"2024-09-21T07:35:15.780612+0000","last_peered":"2024-09-21T07:35:15.780612+0000","last_clean":"2024-09-21T07:35:15.780612+0000","last_became_active":"2024-09-21T07:35:15.773756+0000","last_became_peered":"2024-09-21T07:35:15.773756+0000","last_unstale":"2024-09-21T07:35:15.780612+0000","last_undegraded":"2024-09-21T07:35:15.780612+0000","last_fullsized":"2024-09-21T07:35:15.780612+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:54.235109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":57,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:35:15.768942+0000","last_change":"2024-09-21T07:34:24.597672+0000","last_active":"2024-09-21T07:35:15.768942+0000","last_peered":"2024-09-21T07:35:15.768942+0000","last_clean":"2024-09-21T07:35:15.768942+0000","last_became_active":"2024-09-21T07:34:23.267770+0000","last_became_peered":"2024-09-21T07:34:23.267770+0000","last_unstale":"2024-09-21T07:35:15.768942+0000","last_undegraded":"2024-09-21T07:35:15.768942+0000","last_fullsized":"2024-09-21T07:35:15.768942+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:51.629476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086378500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.773367+0000","last_change":"2024-09-21T07:35:16.761921+0000","last_active":"2024-09-21T07:35:16.773367+0000","last_peered":"2024-09-21T07:35:16.773367+0000","last_clean":"2024-09-21T07:35:16.773367+0000","last_became_active":"2024-09-21T07:35:16.760976+0000","last_became_peered":"2024-09-21T07:35:16.760976+0000","last_unstale":"2024-09-21T07:35:16.773367+0000","last_undegraded":"2024-09-21T07:35:16.773367+0000","last_fullsized":"2024-09-21T07:35:16.773367+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:37.140236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.775942+0000","last_change":"2024-09-21T07:35:15.769225+0000","last_active":"2024-09-21T07:35:16.775942+0000","last_peered":"2024-09-21T07:35:16.775942+0000","last_clean":"2024-09-21T07:35:16.775942+0000","last_became_active":"2024-09-21T07:35:15.768922+0000","last_became_peered":"2024-09-21T07:35:15.768922+0000","last_unstale":"2024-09-21T07:35:16.775942+0000","last_undegraded":"2024-09-21T07:35:16.775942+0000","last_fullsized":"2024-09-21T07:35:16.775942+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:16.143968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":62,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.757342+0000","last_change":"2024-09-21T07:34:24.260181+0000","last_active":"2024-09-21T07:35:16.757342+0000","last_peered":"2024-09-21T07:35:16.757342+0000","last_clean":"2024-09-21T07:35:16.757342+0000","last_became_active":"2024-09-21T07:34:23.275580+0000","last_became_peered":"2024-09-21T07:34:23.275580+0000","last_unstale":"2024-09-21T07:35:16.757342+0000","last_undegraded":"2024-09-21T07:35:16.757342+0000","last_fullsized":"2024-09-21T07:35:16.757342+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:49.857622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014706980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.792227+0000","last_change":"2024-09-21T07:35:16.786379+0000","last_active":"2024-09-21T07:35:16.792227+0000","last_peered":"2024-09-21T07:35:16.792227+0000","last_clean":"2024-09-21T07:35:16.792227+0000","last_became_active":"2024-09-21T07:35:16.786061+0000","last_became_peered":"2024-09-21T07:35:16.786061+0000","last_unstale":"2024-09-21T07:35:16.792227+0000","last_undegraded":"2024-09-21T07:35:16.792227+0000","last_fullsized":"2024-09-21T07:35:16.792227+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:56:07.240152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.764137+0000","last_change":"2024-09-21T07:35:15.777167+0000","last_active":"2024-09-21T07:35:16.764137+0000","last_peered":"2024-09-21T07:35:16.764137+0000","last_clean":"2024-09-21T07:35:16.764137+0000","last_became_active":"2024-09-21T07:35:15.776916+0000","last_became_peered":"2024-09-21T07:35:15.776916+0000","last_unstale":"2024-09-21T07:35:16.764137+0000","last_undegraded":"2024-09-21T07:35:16.764137+0000","last_fullsized":"2024-09-21T07:35:16.764137+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:34.379866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":62,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:35:15.758108+0000","last_change":"2024-09-21T07:34:24.259112+0000","last_active":"2024-09-21T07:35:15.758108+0000","last_peered":"2024-09-21T07:35:15.758108+0000","last_clean":"2024-09-21T07:35:15.758108+0000","last_became_active":"2024-09-21T07:34:23.635444+0000","last_became_peered":"2024-09-21T07:34:23.635444+0000","last_unstale":"2024-09-21T07:35:15.758108+0000","last_undegraded":"2024-09-21T07:35:15.758108+0000","last_fullsized":"2024-09-21T07:35:15.758108+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:46.255385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001952636,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-21T07:35:15.755897+0000","last_change":"2024-09-21T07:35:15.755897+0000","last_active":"2024-09-21T07:35:15.735669+0000","last_peered":"2024-09-21T07:35:15.735669+0000","last_clean":"2024-09-21T07:35:15.735669+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:35:15.755897+0000","last_undegraded":"2024-09-21T07:35:15.755897+0000","last_fullsized":"2024-09-21T07:35:15.755897+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:35:15.773316+0000","last_change":"2024-09-21T07:35:15.768589+0000","last_active":"2024-09-21T07:35:15.773316+0000","last_peered":"2024-09-21T07:35:15.773316+0000","last_clean":"2024-09-21T07:35:15.773316+0000","last_became_active":"2024-09-21T07:35:15.767181+0000","last_became_peered":"2024-09-21T07:35:15.767181+0000","last_unstale":"2024-09-21T07:35:15.773316+0000","last_undegraded":"2024-09-21T07:35:15.773316+0000","last_fullsized":"2024-09-21T07:35:15.773316+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:03.655834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":62,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.754721+0000","last_change":"2024-09-21T07:34:24.539152+0000","last_active":"2024-09-21T07:35:16.754721+0000","last_peered":"2024-09-21T07:35:16.754721+0000","last_clean":"2024-09-21T07:35:16.754721+0000","last_became_active":"2024-09-21T07:34:23.259573+0000","last_became_peered":"2024-09-21T07:34:23.259573+0000","last_unstale":"2024-09-21T07:35:16.754721+0000","last_undegraded":"2024-09-21T07:35:16.754721+0000","last_fullsized":"2024-09-21T07:35:16.754721+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:26.986900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058108400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.799620+0000","last_change":"2024-09-21T07:35:16.790419+0000","last_active":"2024-09-21T07:35:16.799620+0000","last_peered":"2024-09-21T07:35:16.799620+0000","last_clean":"2024-09-21T07:35:16.799620+0000","last_became_active":"2024-09-21T07:35:16.790026+0000","last_became_peered":"2024-09-21T07:35:16.790026+0000","last_unstale":"2024-09-21T07:35:16.799620+0000","last_undegraded":"2024-09-21T07:35:16.799620+0000","last_fullsized":"2024-09-21T07:35:16.799620+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:01:03.904960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.792366+0000","last_change":"2024-09-21T07:35:15.762119+0000","last_active":"2024-09-21T07:35:16.792366+0000","last_peered":"2024-09-21T07:35:16.792366+0000","last_clean":"2024-09-21T07:35:16.792366+0000","last_became_active":"2024-09-21T07:35:15.761816+0000","last_became_peered":"2024-09-21T07:35:15.761816+0000","last_unstale":"2024-09-21T07:35:16.792366+0000","last_undegraded":"2024-09-21T07:35:16.792366+0000","last_fullsized":"2024-09-21T07:35:16.792366+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:55:20.866012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":63,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.762411+0000","last_change":"2024-09-21T07:34:24.259902+0000","last_active":"2024-09-21T07:35:16.762411+0000","last_peered":"2024-09-21T07:35:16.762411+0000","last_clean":"2024-09-21T07:35:16.762411+0000","last_became_active":"2024-09-21T07:34:23.271902+0000","last_became_peered":"2024-09-21T07:34:23.271902+0000","last_unstale":"2024-09-21T07:35:16.762411+0000","last_undegraded":"2024-09-21T07:35:16.762411+0000","last_fullsized":"2024-09-21T07:35:16.762411+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:59.439525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001686358,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.791866+0000","last_change":"2024-09-21T07:35:16.788234+0000","last_active":"2024-09-21T07:35:16.791866+0000","last_peered":"2024-09-21T07:35:16.791866+0000","last_clean":"2024-09-21T07:35:16.791866+0000","last_became_active":"2024-09-21T07:35:16.787916+0000","last_became_peered":"2024-09-21T07:35:16.787916+0000","last_unstale":"2024-09-21T07:35:16.791866+0000","last_undegraded":"2024-09-21T07:35:16.791866+0000","last_fullsized":"2024-09-21T07:35:16.791866+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:49.059613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.784022+0000","last_change":"2024-09-21T07:35:15.769948+0000","last_active":"2024-09-21T07:35:16.784022+0000","last_peered":"2024-09-21T07:35:16.784022+0000","last_clean":"2024-09-21T07:35:16.784022+0000","last_became_active":"2024-09-21T07:35:15.769514+0000","last_became_peered":"2024-09-21T07:35:15.769514+0000","last_unstale":"2024-09-21T07:35:16.784022+0000","last_undegraded":"2024-09-21T07:35:16.784022+0000","last_fullsized":"2024-09-21T07:35:16.784022+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:54.510608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":62,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:35:15.757941+0000","last_change":"2024-09-21T07:34:24.259662+0000","last_active":"2024-09-21T07:35:15.757941+0000","last_peered":"2024-09-21T07:35:15.757941+0000","last_clean":"2024-09-21T07:35:15.757941+0000","last_became_active":"2024-09-21T07:34:23.278812+0000","last_became_peered":"2024-09-21T07:34:23.278812+0000","last_unstale":"2024-09-21T07:35:15.757941+0000","last_undegraded":"2024-09-21T07:35:15.757941+0000","last_fullsized":"2024-09-21T07:35:15.757941+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:49.829974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356463,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.792258+0000","last_change":"2024-09-21T07:35:16.786498+0000","last_active":"2024-09-21T07:35:16.792258+0000","last_peered":"2024-09-21T07:35:16.792258+0000","last_clean":"2024-09-21T07:35:16.792258+0000","last_became_active":"2024-09-21T07:35:16.786050+0000","last_became_peered":"2024-09-21T07:35:16.786050+0000","last_unstale":"2024-09-21T07:35:16.792258+0000","last_undegraded":"2024-09-21T07:35:16.792258+0000","last_fullsized":"2024-09-21T07:35:16.792258+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:58.039151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"61'4","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.801341+0000","last_change":"2024-09-21T07:35:15.774334+0000","last_active":"2024-09-21T07:35:16.801341+0000","last_peered":"2024-09-21T07:35:16.801341+0000","last_clean":"2024-09-21T07:35:16.801341+0000","last_became_active":"2024-09-21T07:35:15.773656+0000","last_became_peered":"2024-09-21T07:35:15.773656+0000","last_unstale":"2024-09-21T07:35:16.801341+0000","last_undegraded":"2024-09-21T07:35:16.801341+0000","last_fullsized":"2024-09-21T07:35:16.801341+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:39.128447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":61,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.764415+0000","last_change":"2024-09-21T07:34:25.084861+0000","last_active":"2024-09-21T07:35:16.764415+0000","last_peered":"2024-09-21T07:35:16.764415+0000","last_clean":"2024-09-21T07:35:16.764415+0000","last_became_active":"2024-09-21T07:34:23.277708+0000","last_became_peered":"2024-09-21T07:34:23.277708+0000","last_unstale":"2024-09-21T07:35:16.764415+0000","last_undegraded":"2024-09-21T07:35:16.764415+0000","last_fullsized":"2024-09-21T07:35:16.764415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:40.816488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000597566,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-21T07:35:15.759707+0000","last_change":"2024-09-21T07:35:15.759707+0000","last_active":"2024-09-21T07:35:15.735669+0000","last_peered":"2024-09-21T07:35:15.735669+0000","last_clean":"2024-09-21T07:35:15.735669+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:35:15.759707+0000","last_undegraded":"2024-09-21T07:35:15.759707+0000","last_fullsized":"2024-09-21T07:35:15.759707+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"61'1","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.793088+0000","last_change":"2024-09-21T07:35:15.774203+0000","last_active":"2024-09-21T07:35:16.793088+0000","last_peered":"2024-09-21T07:35:16.793088+0000","last_clean":"2024-09-21T07:35:16.793088+0000","last_became_active":"2024-09-21T07:35:15.773853+0000","last_became_peered":"2024-09-21T07:35:15.773853+0000","last_unstale":"2024-09-21T07:35:16.793088+0000","last_undegraded":"2024-09-21T07:35:16.793088+0000","last_fullsized":"2024-09-21T07:35:16.793088+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:32.736201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":62,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:35:15.757857+0000","last_change":"2024-09-21T07:34:24.258709+0000","last_active":"2024-09-21T07:35:15.757857+0000","last_peered":"2024-09-21T07:35:15.757857+0000","last_clean":"2024-09-21T07:35:15.757857+0000","last_became_active":"2024-09-21T07:34:23.279042+0000","last_became_peered":"2024-09-21T07:34:23.279042+0000","last_unstale":"2024-09-21T07:35:15.757857+0000","last_undegraded":"2024-09-21T07:35:15.757857+0000","last_fullsized":"2024-09-21T07:35:15.757857+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:01.762117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019282100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.791522+0000","last_change":"2024-09-21T07:35:16.781881+0000","last_active":"2024-09-21T07:35:16.791522+0000","last_peered":"2024-09-21T07:35:16.791522+0000","last_clean":"2024-09-21T07:35:16.791522+0000","last_became_active":"2024-09-21T07:35:16.780834+0000","last_became_peered":"2024-09-21T07:35:16.780834+0000","last_unstale":"2024-09-21T07:35:16.791522+0000","last_undegraded":"2024-09-21T07:35:16.791522+0000","last_fullsized":"2024-09-21T07:35:16.791522+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:04.681922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:35:15.780638+0000","last_change":"2024-09-21T07:35:15.775359+0000","last_active":"2024-09-21T07:35:15.780638+0000","last_peered":"2024-09-21T07:35:15.780638+0000","last_clean":"2024-09-21T07:35:15.780638+0000","last_became_active":"2024-09-21T07:35:15.774998+0000","last_became_peered":"2024-09-21T07:35:15.774998+0000","last_unstale":"2024-09-21T07:35:15.780638+0000","last_undegraded":"2024-09-21T07:35:15.780638+0000","last_fullsized":"2024-09-21T07:35:15.780638+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:03.514687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":61,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.768313+0000","last_change":"2024-09-21T07:34:25.085635+0000","last_active":"2024-09-21T07:35:16.768313+0000","last_peered":"2024-09-21T07:35:16.768313+0000","last_clean":"2024-09-21T07:35:16.768313+0000","last_became_active":"2024-09-21T07:34:23.273566+0000","last_became_peered":"2024-09-21T07:34:23.273566+0000","last_unstale":"2024-09-21T07:35:16.768313+0000","last_undegraded":"2024-09-21T07:35:16.768313+0000","last_fullsized":"2024-09-21T07:35:16.768313+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:14.142272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079897300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-21T07:35:15.761381+0000","last_change":"2024-09-21T07:35:15.761381+0000","last_active":"2024-09-21T07:35:15.735669+0000","last_peered":"2024-09-21T07:35:15.735669+0000","last_clean":"2024-09-21T07:35:15.735669+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:35:15.761381+0000","last_undegraded":"2024-09-21T07:35:15.761381+0000","last_fullsized":"2024-09-21T07:35:15.761381+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.772198+0000","last_change":"2024-09-21T07:35:15.772141+0000","last_active":"2024-09-21T07:35:16.772198+0000","last_peered":"2024-09-21T07:35:16.772198+0000","last_clean":"2024-09-21T07:35:16.772198+0000","last_became_active":"2024-09-21T07:35:15.771819+0000","last_became_peered":"2024-09-21T07:35:15.771819+0000","last_unstale":"2024-09-21T07:35:16.772198+0000","last_undegraded":"2024-09-21T07:35:16.772198+0000","last_fullsized":"2024-09-21T07:35:16.772198+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:48.972425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":57,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:35:15.768883+0000","last_change":"2024-09-21T07:34:24.597926+0000","last_active":"2024-09-21T07:35:15.768883+0000","last_peered":"2024-09-21T07:35:15.768883+0000","last_clean":"2024-09-21T07:35:15.768883+0000","last_became_active":"2024-09-21T07:34:23.268926+0000","last_became_peered":"2024-09-21T07:34:23.268926+0000","last_unstale":"2024-09-21T07:35:15.768883+0000","last_undegraded":"2024-09-21T07:35:15.768883+0000","last_fullsized":"2024-09-21T07:35:15.768883+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:46.190752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101851,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.789907+0000","last_change":"2024-09-21T07:35:16.784521+0000","last_active":"2024-09-21T07:35:16.789907+0000","last_peered":"2024-09-21T07:35:16.789907+0000","last_clean":"2024-09-21T07:35:16.789907+0000","last_became_active":"2024-09-21T07:35:16.784211+0000","last_became_peered":"2024-09-21T07:35:16.784211+0000","last_unstale":"2024-09-21T07:35:16.789907+0000","last_undegraded":"2024-09-21T07:35:16.789907+0000","last_fullsized":"2024-09-21T07:35:16.789907+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:07:50.501826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:35:15.771620+0000","last_change":"2024-09-21T07:35:15.769243+0000","last_active":"2024-09-21T07:35:15.771620+0000","last_peered":"2024-09-21T07:35:15.771620+0000","last_clean":"2024-09-21T07:35:15.771620+0000","last_became_active":"2024-09-21T07:35:15.768915+0000","last_became_peered":"2024-09-21T07:35:15.768915+0000","last_unstale":"2024-09-21T07:35:15.771620+0000","last_undegraded":"2024-09-21T07:35:15.771620+0000","last_fullsized":"2024-09-21T07:35:15.771620+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:56:17.283581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":62,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.757280+0000","last_change":"2024-09-21T07:34:24.259794+0000","last_active":"2024-09-21T07:35:16.757280+0000","last_peered":"2024-09-21T07:35:16.757280+0000","last_clean":"2024-09-21T07:35:16.757280+0000","last_became_active":"2024-09-21T07:34:23.265679+0000","last_became_peered":"2024-09-21T07:34:23.265679+0000","last_unstale":"2024-09-21T07:35:16.757280+0000","last_undegraded":"2024-09-21T07:35:16.757280+0000","last_fullsized":"2024-09-21T07:35:16.757280+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:02.575796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001298518,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.789145+0000","last_change":"2024-09-21T07:35:16.783085+0000","last_active":"2024-09-21T07:35:16.789145+0000","last_peered":"2024-09-21T07:35:16.789145+0000","last_clean":"2024-09-21T07:35:16.789145+0000","last_became_active":"2024-09-21T07:35:16.782399+0000","last_became_peered":"2024-09-21T07:35:16.782399+0000","last_unstale":"2024-09-21T07:35:16.789145+0000","last_undegraded":"2024-09-21T07:35:16.789145+0000","last_fullsized":"2024-09-21T07:35:16.789145+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:41.477998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:35:15.770541+0000","last_change":"2024-09-21T07:35:15.766345+0000","last_active":"2024-09-21T07:35:15.770541+0000","last_peered":"2024-09-21T07:35:15.770541+0000","last_clean":"2024-09-21T07:35:15.770541+0000","last_became_active":"2024-09-21T07:35:15.766006+0000","last_became_peered":"2024-09-21T07:35:15.766006+0000","last_unstale":"2024-09-21T07:35:15.770541+0000","last_undegraded":"2024-09-21T07:35:15.770541+0000","last_fullsized":"2024-09-21T07:35:15.770541+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:25:53.460865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":63,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.763589+0000","last_change":"2024-09-21T07:34:24.260330+0000","last_active":"2024-09-21T07:35:16.763589+0000","last_peered":"2024-09-21T07:35:16.763589+0000","last_clean":"2024-09-21T07:35:16.763589+0000","last_became_active":"2024-09-21T07:34:23.637210+0000","last_became_peered":"2024-09-21T07:34:23.637210+0000","last_unstale":"2024-09-21T07:35:16.763589+0000","last_undegraded":"2024-09-21T07:35:16.763589+0000","last_fullsized":"2024-09-21T07:35:16.763589+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.488451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001537093,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.792367+0000","last_change":"2024-09-21T07:35:16.789414+0000","last_active":"2024-09-21T07:35:16.792367+0000","last_peered":"2024-09-21T07:35:16.792367+0000","last_clean":"2024-09-21T07:35:16.792367+0000","last_became_active":"2024-09-21T07:35:16.789051+0000","last_became_peered":"2024-09-21T07:35:16.789051+0000","last_unstale":"2024-09-21T07:35:16.792367+0000","last_undegraded":"2024-09-21T07:35:16.792367+0000","last_fullsized":"2024-09-21T07:35:16.792367+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:48.459529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"61'3","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.795293+0000","last_change":"2024-09-21T07:35:15.777864+0000","last_active":"2024-09-21T07:35:16.795293+0000","last_peered":"2024-09-21T07:35:16.795293+0000","last_clean":"2024-09-21T07:35:16.795293+0000","last_became_active":"2024-09-21T07:35:15.777554+0000","last_became_peered":"2024-09-21T07:35:15.777554+0000","last_unstale":"2024-09-21T07:35:16.795293+0000","last_undegraded":"2024-09-21T07:35:16.795293+0000","last_fullsized":"2024-09-21T07:35:16.795293+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:50.028374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":57,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:35:15.769268+0000","last_change":"2024-09-21T07:34:24.597286+0000","last_active":"2024-09-21T07:35:15.769268+0000","last_peered":"2024-09-21T07:35:15.769268+0000","last_clean":"2024-09-21T07:35:15.769268+0000","last_became_active":"2024-09-21T07:34:23.267133+0000","last_became_peered":"2024-09-21T07:34:23.267133+0000","last_unstale":"2024-09-21T07:35:15.769268+0000","last_undegraded":"2024-09-21T07:35:15.769268+0000","last_fullsized":"2024-09-21T07:35:15.769268+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:18.257010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-21T07:35:15.756101+0000","last_change":"2024-09-21T07:35:15.756101+0000","last_active":"2024-09-21T07:35:15.735669+0000","last_peered":"2024-09-21T07:35:15.735669+0000","last_clean":"2024-09-21T07:35:15.735669+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:35:15.756101+0000","last_undegraded":"2024-09-21T07:35:15.756101+0000","last_fullsized":"2024-09-21T07:35:15.756101+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"61'6","reported_seq":19,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.802587+0000","last_change":"2024-09-21T07:35:15.776385+0000","last_active":"2024-09-21T07:35:16.802587+0000","last_peered":"2024-09-21T07:35:16.802587+0000","last_clean":"2024-09-21T07:35:16.802587+0000","last_became_active":"2024-09-21T07:35:15.775925+0000","last_became_peered":"2024-09-21T07:35:15.775925+0000","last_unstale":"2024-09-21T07:35:16.802587+0000","last_undegraded":"2024-09-21T07:35:16.802587+0000","last_fullsized":"2024-09-21T07:35:16.802587+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:40.466844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":62,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:35:15.757876+0000","last_change":"2024-09-21T07:34:24.258712+0000","last_active":"2024-09-21T07:35:15.757876+0000","last_peered":"2024-09-21T07:35:15.757876+0000","last_clean":"2024-09-21T07:35:15.757876+0000","last_became_active":"2024-09-21T07:34:23.274692+0000","last_became_peered":"2024-09-21T07:34:23.274692+0000","last_unstale":"2024-09-21T07:35:15.757876+0000","last_undegraded":"2024-09-21T07:35:15.757876+0000","last_fullsized":"2024-09-21T07:35:15.757876+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:43.279454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018051479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.793825+0000","last_change":"2024-09-21T07:35:16.784678+0000","last_active":"2024-09-21T07:35:16.793825+0000","last_peered":"2024-09-21T07:35:16.793825+0000","last_clean":"2024-09-21T07:35:16.793825+0000","last_became_active":"2024-09-21T07:35:16.784376+0000","last_became_peered":"2024-09-21T07:35:16.784376+0000","last_unstale":"2024-09-21T07:35:16.793825+0000","last_undegraded":"2024-09-21T07:35:16.793825+0000","last_fullsized":"2024-09-21T07:35:16.793825+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:01.158445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.764645+0000","last_change":"2024-09-21T07:35:15.781466+0000","last_active":"2024-09-21T07:35:16.764645+0000","last_peered":"2024-09-21T07:35:16.764645+0000","last_clean":"2024-09-21T07:35:16.764645+0000","last_became_active":"2024-09-21T07:35:15.781232+0000","last_became_peered":"2024-09-21T07:35:15.781232+0000","last_unstale":"2024-09-21T07:35:16.764645+0000","last_undegraded":"2024-09-21T07:35:16.764645+0000","last_fullsized":"2024-09-21T07:35:16.764645+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:44.771518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":61,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.764378+0000","last_change":"2024-09-21T07:34:25.084849+0000","last_active":"2024-09-21T07:35:16.764378+0000","last_peered":"2024-09-21T07:35:16.764378+0000","last_clean":"2024-09-21T07:35:16.764378+0000","last_became_active":"2024-09-21T07:34:23.277713+0000","last_became_peered":"2024-09-21T07:34:23.277713+0000","last_unstale":"2024-09-21T07:35:16.764378+0000","last_undegraded":"2024-09-21T07:35:16.764378+0000","last_fullsized":"2024-09-21T07:35:16.764378+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:05.811810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059079499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-21T07:35:15.761367+0000","last_change":"2024-09-21T07:35:15.761367+0000","last_active":"2024-09-21T07:35:15.735669+0000","last_peered":"2024-09-21T07:35:15.735669+0000","last_clean":"2024-09-21T07:35:15.735669+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:35:15.761367+0000","last_undegraded":"2024-09-21T07:35:15.761367+0000","last_fullsized":"2024-09-21T07:35:15.761367+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:35:15.779379+0000","last_change":"2024-09-21T07:35:15.776840+0000","last_active":"2024-09-21T07:35:15.779379+0000","last_peered":"2024-09-21T07:35:15.779379+0000","last_clean":"2024-09-21T07:35:15.779379+0000","last_became_active":"2024-09-21T07:35:15.776512+0000","last_became_peered":"2024-09-21T07:35:15.776512+0000","last_unstale":"2024-09-21T07:35:15.779379+0000","last_undegraded":"2024-09-21T07:35:15.779379+0000","last_fullsized":"2024-09-21T07:35:15.779379+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:48.841603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":62,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.755710+0000","last_change":"2024-09-21T07:34:24.538610+0000","last_active":"2024-09-21T07:35:16.755710+0000","last_peered":"2024-09-21T07:35:16.755710+0000","last_clean":"2024-09-21T07:35:16.755710+0000","last_became_active":"2024-09-21T07:34:23.259700+0000","last_became_peered":"2024-09-21T07:34:23.259700+0000","last_unstale":"2024-09-21T07:35:16.755710+0000","last_undegraded":"2024-09-21T07:35:16.755710+0000","last_fullsized":"2024-09-21T07:35:16.755710+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:57.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070522399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.792114+0000","last_change":"2024-09-21T07:35:16.782226+0000","last_active":"2024-09-21T07:35:16.792114+0000","last_peered":"2024-09-21T07:35:16.792114+0000","last_clean":"2024-09-21T07:35:16.792114+0000","last_became_active":"2024-09-21T07:35:16.781232+0000","last_became_peered":"2024-09-21T07:35:16.781232+0000","last_unstale":"2024-09-21T07:35:16.792114+0000","last_undegraded":"2024-09-21T07:35:16.792114+0000","last_fullsized":"2024-09-21T07:35:16.792114+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:22.916044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.763992+0000","last_change":"2024-09-21T07:35:15.774672+0000","last_active":"2024-09-21T07:35:16.763992+0000","last_peered":"2024-09-21T07:35:16.763992+0000","last_clean":"2024-09-21T07:35:16.763992+0000","last_became_active":"2024-09-21T07:35:15.774260+0000","last_became_peered":"2024-09-21T07:35:15.774260+0000","last_unstale":"2024-09-21T07:35:16.763992+0000","last_undegraded":"2024-09-21T07:35:16.763992+0000","last_fullsized":"2024-09-21T07:35:16.763992+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:33.522625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":57,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:35:15.769061+0000","last_change":"2024-09-21T07:34:24.597597+0000","last_active":"2024-09-21T07:35:15.769061+0000","last_peered":"2024-09-21T07:35:15.769061+0000","last_clean":"2024-09-21T07:35:15.769061+0000","last_became_active":"2024-09-21T07:34:23.635204+0000","last_became_peered":"2024-09-21T07:34:23.635204+0000","last_unstale":"2024-09-21T07:35:15.769061+0000","last_undegraded":"2024-09-21T07:35:15.769061+0000","last_fullsized":"2024-09-21T07:35:15.769061+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:48.644789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081139199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.799763+0000","last_change":"2024-09-21T07:35:16.794557+0000","last_active":"2024-09-21T07:35:16.799763+0000","last_peered":"2024-09-21T07:35:16.799763+0000","last_clean":"2024-09-21T07:35:16.799763+0000","last_became_active":"2024-09-21T07:35:16.794227+0000","last_became_peered":"2024-09-21T07:35:16.794227+0000","last_unstale":"2024-09-21T07:35:16.799763+0000","last_undegraded":"2024-09-21T07:35:16.799763+0000","last_fullsized":"2024-09-21T07:35:16.799763+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:19:41.151420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:35:15.776304+0000","last_change":"2024-09-21T07:35:15.773449+0000","last_active":"2024-09-21T07:35:15.776304+0000","last_peered":"2024-09-21T07:35:15.776304+0000","last_clean":"2024-09-21T07:35:15.776304+0000","last_became_active":"2024-09-21T07:35:15.772416+0000","last_became_peered":"2024-09-21T07:35:15.772416+0000","last_unstale":"2024-09-21T07:35:15.776304+0000","last_undegraded":"2024-09-21T07:35:15.776304+0000","last_fullsized":"2024-09-21T07:35:15.776304+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:27:23.119723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":62,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:35:15.758746+0000","last_change":"2024-09-21T07:34:24.259026+0000","last_active":"2024-09-21T07:35:15.758746+0000","last_peered":"2024-09-21T07:35:15.758746+0000","last_clean":"2024-09-21T07:35:15.758746+0000","last_became_active":"2024-09-21T07:34:23.635456+0000","last_became_peered":"2024-09-21T07:34:23.635456+0000","last_unstale":"2024-09-21T07:35:15.758746+0000","last_undegraded":"2024-09-21T07:35:15.758746+0000","last_fullsized":"2024-09-21T07:35:15.758746+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:09.592752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018654419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-21T07:35:15.762283+0000","last_change":"2024-09-21T07:35:15.762283+0000","last_active":"2024-09-21T07:35:15.735669+0000","last_peered":"2024-09-21T07:35:15.735669+0000","last_clean":"2024-09-21T07:35:15.735669+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:35:15.762283+0000","last_undegraded":"2024-09-21T07:35:15.762283+0000","last_fullsized":"2024-09-21T07:35:15.762283+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.793555+0000","last_change":"2024-09-21T07:35:15.775733+0000","last_active":"2024-09-21T07:35:16.793555+0000","last_peered":"2024-09-21T07:35:16.793555+0000","last_clean":"2024-09-21T07:35:16.793555+0000","last_became_active":"2024-09-21T07:35:15.775407+0000","last_became_peered":"2024-09-21T07:35:15.775407+0000","last_unstale":"2024-09-21T07:35:16.793555+0000","last_undegraded":"2024-09-21T07:35:16.793555+0000","last_fullsized":"2024-09-21T07:35:16.793555+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:24:28.776627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":63,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.762721+0000","last_change":"2024-09-21T07:34:24.260300+0000","last_active":"2024-09-21T07:35:16.762721+0000","last_peered":"2024-09-21T07:35:16.762721+0000","last_clean":"2024-09-21T07:35:16.762721+0000","last_became_active":"2024-09-21T07:34:23.270128+0000","last_became_peered":"2024-09-21T07:34:23.270128+0000","last_unstale":"2024-09-21T07:35:16.762721+0000","last_undegraded":"2024-09-21T07:35:16.762721+0000","last_fullsized":"2024-09-21T07:35:16.762721+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:25.260931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00176651,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.787612+0000","last_change":"2024-09-21T07:35:16.779848+0000","last_active":"2024-09-21T07:35:16.787612+0000","last_peered":"2024-09-21T07:35:16.787612+0000","last_clean":"2024-09-21T07:35:16.787612+0000","last_became_active":"2024-09-21T07:35:16.779541+0000","last_became_peered":"2024-09-21T07:35:16.779541+0000","last_unstale":"2024-09-21T07:35:16.787612+0000","last_undegraded":"2024-09-21T07:35:16.787612+0000","last_fullsized":"2024-09-21T07:35:16.787612+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:41.469670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.755140+0000","last_change":"2024-09-21T07:35:15.763268+0000","last_active":"2024-09-21T07:35:16.755140+0000","last_peered":"2024-09-21T07:35:16.755140+0000","last_clean":"2024-09-21T07:35:16.755140+0000","last_became_active":"2024-09-21T07:35:15.762901+0000","last_became_peered":"2024-09-21T07:35:15.762901+0000","last_unstale":"2024-09-21T07:35:16.755140+0000","last_undegraded":"2024-09-21T07:35:16.755140+0000","last_fullsized":"2024-09-21T07:35:16.755140+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:45.928812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":61,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.791174+0000","last_change":"2024-09-21T07:34:25.028904+0000","last_active":"2024-09-21T07:35:16.791174+0000","last_peered":"2024-09-21T07:35:16.791174+0000","last_clean":"2024-09-21T07:35:16.791174+0000","last_became_active":"2024-09-21T07:34:23.260942+0000","last_became_peered":"2024-09-21T07:34:23.260942+0000","last_unstale":"2024-09-21T07:35:16.791174+0000","last_undegraded":"2024-09-21T07:35:16.791174+0000","last_fullsized":"2024-09-21T07:35:16.791174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:22.709328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060553899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":3208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28,"ondisk_log_size":28,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509470,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32500,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339340,"statfs":{"total":96636764160,"available":96603484160,"internally_reserved":0,"allocated":1335296,"data_stored":907055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509472,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31904,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339936,"statfs":{"total":96636764160,"available":96604094464,"internally_reserved":0,"allocated":729088,"data_stored":315945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509470,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31900,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339940,"statfs":{"total":96636764160,"available":96604098560,"internally_reserved":0,"allocated":729088,"data_stored":315933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509471,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31884,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339956,"statfs":{"total":96636764160,"available":96604114944,"internally_reserved":0,"allocated":712704,"data_stored":308783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509472,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31924,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339916,"statfs":{"total":96636764160,"available":96604073984,"internally_reserved":0,"allocated":753664,"data_stored":318287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509472,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32480,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1318912,"data_stored":906279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542175,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31884,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339956,"statfs":{"total":96636764160,"available":96604114944,"internally_reserved":0,"allocated":712704,"data_stored":308783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542174,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31884,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339956,"statfs":{"total":96636764160,"available":96604114944,"internally_reserved":0,"allocated":712704,"data_stored":308783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:35:21.274 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:35:21.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.533+0000 7f63bb200640 1 -- 172.21.15.39:0/2167091611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63bc0cfee0 msgr2=0x7f63bc0d02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:21.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.533+0000 7f63bb200640 1 --2- 172.21.15.39:0/2167091611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63bc0cfee0 0x7f63bc0d02f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f63bc0ca530 tx=0x7f63a802a540 comp rx=0 tx=0).stop 2024-09-21T07:35:21.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.533+0000 7f63bb200640 1 -- 172.21.15.39:0/2167091611 shutdown_connections 2024-09-21T07:35:21.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.533+0000 7f63bb200640 1 --2- 172.21.15.39:0/2167091611 >> v2:172.21.15.148:3300/0 conn(0x7f63bc073f50 0x7f63bc074360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:21.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.533+0000 7f63bb200640 1 --2- 172.21.15.39:0/2167091611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63bc0cfee0 0x7f63bc0d02f0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:21.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.533+0000 7f63bb200640 1 -- 172.21.15.39:0/2167091611 >> 172.21.15.39:0/2167091611 conn(0x7f63bc06d870 msgr2=0x7f63bc06ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:21.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.533+0000 7f63bb200640 1 -- 172.21.15.39:0/2167091611 shutdown_connections 2024-09-21T07:35:21.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.533+0000 7f63bb200640 1 -- 172.21.15.39:0/2167091611 wait complete. 2024-09-21T07:35:21.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.536+0000 7f63bb200640 1 Processor -- start 2024-09-21T07:35:21.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.536+0000 7f63bb200640 1 -- start start 2024-09-21T07:35:21.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.537+0000 7f63bb200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63bc073f50 0x7f63bc0c9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:21.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.537+0000 7f63bb200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63bc0cfee0 0x7f63bc15a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:21.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.537+0000 7f63bb200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f63bc144bc0 0x7f63bc140270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:21.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.537+0000 7f63bb200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f63bc074a20 con 0x7f63bc0cfee0 2024-09-21T07:35:21.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.537+0000 7f63bb200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f63bc0748a0 con 0x7f63bc073f50 2024-09-21T07:35:21.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.537+0000 7f63bb200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f63bc074ba0 con 0x7f63bc144bc0 2024-09-21T07:35:21.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.538+0000 7f63b9400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63bc0cfee0 0x7f63bc15a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:21.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.538+0000 7f63b9400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63bc0cfee0 0x7f63bc15a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46664/0 (socket says 172.21.15.39:46664) 2024-09-21T07:35:21.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.538+0000 7f63b9400640 1 -- 172.21.15.39:0/3578934744 learned_addr learned my addr 172.21.15.39:0/3578934744 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:21.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.538+0000 7f63ba800640 1 --2- 172.21.15.39:0/3578934744 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f63bc144bc0 0x7f63bc140270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:21.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.538+0000 7f63b9e00640 1 --2- 172.21.15.39:0/3578934744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63bc073f50 0x7f63bc0c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:21.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.538+0000 7f63b9400640 1 -- 172.21.15.39:0/3578934744 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f63bc144bc0 msgr2=0x7f63bc140270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:21.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.538+0000 7f63b9400640 1 --2- 172.21.15.39:0/3578934744 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f63bc144bc0 0x7f63bc140270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:21.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.538+0000 7f63b9400640 1 -- 172.21.15.39:0/3578934744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63bc073f50 msgr2=0x7f63bc0c9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:21.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.538+0000 7f63b9400640 1 --2- 172.21.15.39:0/3578934744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63bc073f50 0x7f63bc0c9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:21.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.538+0000 7f63b9400640 1 -- 172.21.15.39:0/3578934744 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63bc0c8280 con 0x7f63bc0cfee0 2024-09-21T07:35:21.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.538+0000 7f63b9400640 1 --2- 172.21.15.39:0/3578934744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63bc0cfee0 0x7f63bc15a080 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f63a8009920 tx=0x7f63a8033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:21.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.539+0000 7f63b2000640 1 -- 172.21.15.39:0/3578934744 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a8031d60 con 0x7f63bc0cfee0 2024-09-21T07:35:21.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.539+0000 7f63b2000640 1 -- 172.21.15.39:0/3578934744 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f63a8033e10 con 0x7f63bc0cfee0 2024-09-21T07:35:21.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.539+0000 7f63b2000640 1 -- 172.21.15.39:0/3578934744 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a8005260 con 0x7f63bc0cfee0 2024-09-21T07:35:21.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.539+0000 7f63bb200640 1 -- 172.21.15.39:0/3578934744 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63bc0c8520 con 0x7f63bc0cfee0 2024-09-21T07:35:21.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.539+0000 7f63bb200640 1 -- 172.21.15.39:0/3578934744 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63bc0c8ae0 con 0x7f63bc0cfee0 2024-09-21T07:35:21.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.540+0000 7f63bb200640 1 -- 172.21.15.39:0/3578934744 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63bc075060 con 0x7f63bc0cfee0 2024-09-21T07:35:21.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.541+0000 7f63b2000640 1 -- 172.21.15.39:0/3578934744 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f63a8005400 con 0x7f63bc0cfee0 2024-09-21T07:35:21.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.543+0000 7f63b2000640 1 --2- 172.21.15.39:0/3578934744 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f639c078b90 0x7f639c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:21.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.543+0000 7f63b2000640 1 -- 172.21.15.39:0/3578934744 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6062+0+0 (secure 0 0 0) 0x7f63a80bc1b0 con 0x7f63bc0cfee0 2024-09-21T07:35:21.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.543+0000 7f63b9e00640 1 --2- 172.21.15.39:0/3578934744 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f639c078b90 0x7f639c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:21.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.544+0000 7f63b9e00640 1 --2- 172.21.15.39:0/3578934744 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f639c078b90 0x7f639c07b0a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f63ac0015e0 tx=0x7f63ac009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:21.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.548+0000 7f63b2000640 1 -- 172.21.15.39:0/3578934744 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f63a803f030 con 0x7f63bc0cfee0 2024-09-21T07:35:21.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.743+0000 7f63bb200640 1 -- 172.21.15.39:0/3578934744 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f63bc15a8d0 con 0x7f639c078b90 2024-09-21T07:35:21.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.749+0000 7f63b2000640 1 -- 172.21.15.39:0/3578934744 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332990 (secure 0 0 0) 0x7f63bc15a8d0 con 0x7f639c078b90 2024-09-21T07:35:21.754 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:35:21.756 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:35:21.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.754+0000 7f63bb200640 1 -- 172.21.15.39:0/3578934744 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f639c078b90 msgr2=0x7f639c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:21.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.754+0000 7f63bb200640 1 --2- 172.21.15.39:0/3578934744 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f639c078b90 0x7f639c07b0a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f63ac0015e0 tx=0x7f63ac009040 comp rx=0 tx=0).stop 2024-09-21T07:35:21.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.754+0000 7f63bb200640 1 -- 172.21.15.39:0/3578934744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63bc0cfee0 msgr2=0x7f63bc15a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:21.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.754+0000 7f63bb200640 1 --2- 172.21.15.39:0/3578934744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63bc0cfee0 0x7f63bc15a080 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f63a8009920 tx=0x7f63a8033c20 comp rx=0 tx=0).stop 2024-09-21T07:35:21.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.754+0000 7f63bb200640 1 -- 172.21.15.39:0/3578934744 shutdown_connections 2024-09-21T07:35:21.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.754+0000 7f63bb200640 1 --2- 172.21.15.39:0/3578934744 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f639c078b90 0x7f639c07b0a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:21.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.754+0000 7f63bb200640 1 --2- 172.21.15.39:0/3578934744 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f63bc144bc0 0x7f63bc140270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:21.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.754+0000 7f63bb200640 1 --2- 172.21.15.39:0/3578934744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63bc0cfee0 0x7f63bc15a080 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:21.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.754+0000 7f63bb200640 1 --2- 172.21.15.39:0/3578934744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63bc073f50 0x7f63bc0c9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:21.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.754+0000 7f63bb200640 1 -- 172.21.15.39:0/3578934744 >> 172.21.15.39:0/3578934744 conn(0x7f63bc06d870 msgr2=0x7f63bc06ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:21.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.754+0000 7f63bb200640 1 -- 172.21.15.39:0/3578934744 shutdown_connections 2024-09-21T07:35:21.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:21.754+0000 7f63bb200640 1 -- 172.21.15.39:0/3578934744 wait complete. 2024-09-21T07:35:21.781 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2024-09-21T07:35:21.565970+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":178,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":256428,"kb_used_data":6900,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754718292,"statfs":{"total":773094113280,"available":772831531008,"internally_reserved":0,"allocated":7065600,"data_stored":3719022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001614"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.792646+0000","last_change":"2024-09-21T07:35:16.780659+0000","last_active":"2024-09-21T07:35:16.792646+0000","last_peered":"2024-09-21T07:35:16.792646+0000","last_clean":"2024-09-21T07:35:16.792646+0000","last_became_active":"2024-09-21T07:35:16.780313+0000","last_became_peered":"2024-09-21T07:35:16.780313+0000","last_unstale":"2024-09-21T07:35:16.792646+0000","last_undegraded":"2024-09-21T07:35:16.792646+0000","last_fullsized":"2024-09-21T07:35:16.792646+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:50.957994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.791129+0000","last_change":"2024-09-21T07:35:16.784420+0000","last_active":"2024-09-21T07:35:16.791129+0000","last_peered":"2024-09-21T07:35:16.791129+0000","last_clean":"2024-09-21T07:35:16.791129+0000","last_became_active":"2024-09-21T07:35:16.782894+0000","last_became_peered":"2024-09-21T07:35:16.782894+0000","last_unstale":"2024-09-21T07:35:16.791129+0000","last_undegraded":"2024-09-21T07:35:16.791129+0000","last_fullsized":"2024-09-21T07:35:16.791129+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:37.394072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.789556+0000","last_change":"2024-09-21T07:35:16.783387+0000","last_active":"2024-09-21T07:35:16.789556+0000","last_peered":"2024-09-21T07:35:16.789556+0000","last_clean":"2024-09-21T07:35:16.789556+0000","last_became_active":"2024-09-21T07:35:16.780787+0000","last_became_peered":"2024-09-21T07:35:16.780787+0000","last_unstale":"2024-09-21T07:35:16.789556+0000","last_undegraded":"2024-09-21T07:35:16.789556+0000","last_fullsized":"2024-09-21T07:35:16.789556+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:20.669159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T07:35:19.110392+0000","last_change":"2024-09-21T07:35:16.781580+0000","last_active":"2024-09-21T07:35:19.110392+0000","last_peered":"2024-09-21T07:35:19.110392+0000","last_clean":"2024-09-21T07:35:19.110392+0000","last_became_active":"2024-09-21T07:35:16.781258+0000","last_became_peered":"2024-09-21T07:35:16.781258+0000","last_unstale":"2024-09-21T07:35:19.110392+0000","last_undegraded":"2024-09-21T07:35:19.110392+0000","last_fullsized":"2024-09-21T07:35:19.110392+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:19.277567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.797715+0000","last_change":"2024-09-21T07:35:16.792626+0000","last_active":"2024-09-21T07:35:16.797715+0000","last_peered":"2024-09-21T07:35:16.797715+0000","last_clean":"2024-09-21T07:35:16.797715+0000","last_became_active":"2024-09-21T07:35:16.792285+0000","last_became_peered":"2024-09-21T07:35:16.792285+0000","last_unstale":"2024-09-21T07:35:16.797715+0000","last_undegraded":"2024-09-21T07:35:16.797715+0000","last_fullsized":"2024-09-21T07:35:16.797715+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:37.160118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.799578+0000","last_change":"2024-09-21T07:35:16.791910+0000","last_active":"2024-09-21T07:35:16.799578+0000","last_peered":"2024-09-21T07:35:16.799578+0000","last_clean":"2024-09-21T07:35:16.799578+0000","last_became_active":"2024-09-21T07:35:16.791615+0000","last_became_peered":"2024-09-21T07:35:16.791615+0000","last_unstale":"2024-09-21T07:35:16.799578+0000","last_undegraded":"2024-09-21T07:35:16.799578+0000","last_fullsized":"2024-09-21T07:35:16.799578+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:23.252899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.780555+0000","last_change":"2024-09-21T07:35:16.774639+0000","last_active":"2024-09-21T07:35:16.780555+0000","last_peered":"2024-09-21T07:35:16.780555+0000","last_clean":"2024-09-21T07:35:16.780555+0000","last_became_active":"2024-09-21T07:35:16.773837+0000","last_became_peered":"2024-09-21T07:35:16.773837+0000","last_unstale":"2024-09-21T07:35:16.780555+0000","last_undegraded":"2024-09-21T07:35:16.780555+0000","last_fullsized":"2024-09-21T07:35:16.780555+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:11:02.899233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.791964+0000","last_change":"2024-09-21T07:35:16.785293+0000","last_active":"2024-09-21T07:35:16.791964+0000","last_peered":"2024-09-21T07:35:16.791964+0000","last_clean":"2024-09-21T07:35:16.791964+0000","last_became_active":"2024-09-21T07:35:16.784803+0000","last_became_peered":"2024-09-21T07:35:16.784803+0000","last_unstale":"2024-09-21T07:35:16.791964+0000","last_undegraded":"2024-09-21T07:35:16.791964+0000","last_fullsized":"2024-09-21T07:35:16.791964+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:56.522045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.788943+0000","last_change":"2024-09-21T07:35:16.777289+0000","last_active":"2024-09-21T07:35:16.788943+0000","last_peered":"2024-09-21T07:35:16.788943+0000","last_clean":"2024-09-21T07:35:16.788943+0000","last_became_active":"2024-09-21T07:35:16.776434+0000","last_became_peered":"2024-09-21T07:35:16.776434+0000","last_unstale":"2024-09-21T07:35:16.788943+0000","last_undegraded":"2024-09-21T07:35:16.788943+0000","last_fullsized":"2024-09-21T07:35:16.788943+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:57.705627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.790050+0000","last_change":"2024-09-21T07:35:16.787398+0000","last_active":"2024-09-21T07:35:16.790050+0000","last_peered":"2024-09-21T07:35:16.790050+0000","last_clean":"2024-09-21T07:35:16.790050+0000","last_became_active":"2024-09-21T07:35:16.787011+0000","last_became_peered":"2024-09-21T07:35:16.787011+0000","last_unstale":"2024-09-21T07:35:16.790050+0000","last_undegraded":"2024-09-21T07:35:16.790050+0000","last_fullsized":"2024-09-21T07:35:16.790050+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:46.011177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T07:35:19.109848+0000","last_change":"2024-09-21T07:35:16.785716+0000","last_active":"2024-09-21T07:35:19.109848+0000","last_peered":"2024-09-21T07:35:19.109848+0000","last_clean":"2024-09-21T07:35:19.109848+0000","last_became_active":"2024-09-21T07:35:16.785355+0000","last_became_peered":"2024-09-21T07:35:16.785355+0000","last_unstale":"2024-09-21T07:35:19.109848+0000","last_undegraded":"2024-09-21T07:35:19.109848+0000","last_fullsized":"2024-09-21T07:35:19.109848+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:54:31.191216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.791813+0000","last_change":"2024-09-21T07:35:16.783310+0000","last_active":"2024-09-21T07:35:16.791813+0000","last_peered":"2024-09-21T07:35:16.791813+0000","last_clean":"2024-09-21T07:35:16.791813+0000","last_became_active":"2024-09-21T07:35:16.783000+0000","last_became_peered":"2024-09-21T07:35:16.783000+0000","last_unstale":"2024-09-21T07:35:16.791813+0000","last_undegraded":"2024-09-21T07:35:16.791813+0000","last_fullsized":"2024-09-21T07:35:16.791813+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:51.769069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.782698+0000","last_change":"2024-09-21T07:35:16.779103+0000","last_active":"2024-09-21T07:35:16.782698+0000","last_peered":"2024-09-21T07:35:16.782698+0000","last_clean":"2024-09-21T07:35:16.782698+0000","last_became_active":"2024-09-21T07:35:16.778797+0000","last_became_peered":"2024-09-21T07:35:16.778797+0000","last_unstale":"2024-09-21T07:35:16.782698+0000","last_undegraded":"2024-09-21T07:35:16.782698+0000","last_fullsized":"2024-09-21T07:35:16.782698+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:01.158445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.789575+0000","last_change":"2024-09-21T07:35:16.778487+0000","last_active":"2024-09-21T07:35:16.789575+0000","last_peered":"2024-09-21T07:35:16.789575+0000","last_clean":"2024-09-21T07:35:16.789575+0000","last_became_active":"2024-09-21T07:35:16.778114+0000","last_became_peered":"2024-09-21T07:35:16.778114+0000","last_unstale":"2024-09-21T07:35:16.789575+0000","last_undegraded":"2024-09-21T07:35:16.789575+0000","last_fullsized":"2024-09-21T07:35:16.789575+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:18.986538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.790206+0000","last_change":"2024-09-21T07:35:16.787501+0000","last_active":"2024-09-21T07:35:16.790206+0000","last_peered":"2024-09-21T07:35:16.790206+0000","last_clean":"2024-09-21T07:35:16.790206+0000","last_became_active":"2024-09-21T07:35:16.784848+0000","last_became_peered":"2024-09-21T07:35:16.784848+0000","last_unstale":"2024-09-21T07:35:16.790206+0000","last_undegraded":"2024-09-21T07:35:16.790206+0000","last_fullsized":"2024-09-21T07:35:16.790206+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:00.824167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.775312+0000","last_change":"2024-09-21T07:35:16.762563+0000","last_active":"2024-09-21T07:35:16.775312+0000","last_peered":"2024-09-21T07:35:16.775312+0000","last_clean":"2024-09-21T07:35:16.775312+0000","last_became_active":"2024-09-21T07:35:16.761910+0000","last_became_peered":"2024-09-21T07:35:16.761910+0000","last_unstale":"2024-09-21T07:35:16.775312+0000","last_undegraded":"2024-09-21T07:35:16.775312+0000","last_fullsized":"2024-09-21T07:35:16.775312+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:57.759094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.787316+0000","last_change":"2024-09-21T07:35:16.780085+0000","last_active":"2024-09-21T07:35:16.787316+0000","last_peered":"2024-09-21T07:35:16.787316+0000","last_clean":"2024-09-21T07:35:16.787316+0000","last_became_active":"2024-09-21T07:35:16.779774+0000","last_became_peered":"2024-09-21T07:35:16.779774+0000","last_unstale":"2024-09-21T07:35:16.787316+0000","last_undegraded":"2024-09-21T07:35:16.787316+0000","last_fullsized":"2024-09-21T07:35:16.787316+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:04:15.356743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T07:35:19.110490+0000","last_change":"2024-09-21T07:35:16.788676+0000","last_active":"2024-09-21T07:35:19.110490+0000","last_peered":"2024-09-21T07:35:19.110490+0000","last_clean":"2024-09-21T07:35:19.110490+0000","last_became_active":"2024-09-21T07:35:16.788265+0000","last_became_peered":"2024-09-21T07:35:16.788265+0000","last_unstale":"2024-09-21T07:35:19.110490+0000","last_undegraded":"2024-09-21T07:35:19.110490+0000","last_fullsized":"2024-09-21T07:35:19.110490+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:05.316926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.792282+0000","last_change":"2024-09-21T07:35:16.786945+0000","last_active":"2024-09-21T07:35:16.792282+0000","last_peered":"2024-09-21T07:35:16.792282+0000","last_clean":"2024-09-21T07:35:16.792282+0000","last_became_active":"2024-09-21T07:35:16.786605+0000","last_became_peered":"2024-09-21T07:35:16.786605+0000","last_unstale":"2024-09-21T07:35:16.792282+0000","last_undegraded":"2024-09-21T07:35:16.792282+0000","last_fullsized":"2024-09-21T07:35:16.792282+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:28:00.845053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T07:35:17.780644+0000","last_change":"2024-09-21T07:35:16.777197+0000","last_active":"2024-09-21T07:35:17.780644+0000","last_peered":"2024-09-21T07:35:17.780644+0000","last_clean":"2024-09-21T07:35:17.780644+0000","last_became_active":"2024-09-21T07:35:16.776868+0000","last_became_peered":"2024-09-21T07:35:16.776868+0000","last_unstale":"2024-09-21T07:35:17.780644+0000","last_undegraded":"2024-09-21T07:35:17.780644+0000","last_fullsized":"2024-09-21T07:35:17.780644+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:02.995938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.791882+0000","last_change":"2024-09-21T07:35:16.784386+0000","last_active":"2024-09-21T07:35:16.791882+0000","last_peered":"2024-09-21T07:35:16.791882+0000","last_clean":"2024-09-21T07:35:16.791882+0000","last_became_active":"2024-09-21T07:35:16.783981+0000","last_became_peered":"2024-09-21T07:35:16.783981+0000","last_unstale":"2024-09-21T07:35:16.791882+0000","last_undegraded":"2024-09-21T07:35:16.791882+0000","last_fullsized":"2024-09-21T07:35:16.791882+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:40.131273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T07:35:17.779556+0000","last_change":"2024-09-21T07:35:16.776391+0000","last_active":"2024-09-21T07:35:17.779556+0000","last_peered":"2024-09-21T07:35:17.779556+0000","last_clean":"2024-09-21T07:35:17.779556+0000","last_became_active":"2024-09-21T07:35:16.775909+0000","last_became_peered":"2024-09-21T07:35:16.775909+0000","last_unstale":"2024-09-21T07:35:17.779556+0000","last_undegraded":"2024-09-21T07:35:17.779556+0000","last_fullsized":"2024-09-21T07:35:17.779556+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:41:18.293479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T07:35:17.779824+0000","last_change":"2024-09-21T07:35:16.777214+0000","last_active":"2024-09-21T07:35:17.779824+0000","last_peered":"2024-09-21T07:35:17.779824+0000","last_clean":"2024-09-21T07:35:17.779824+0000","last_became_active":"2024-09-21T07:35:16.776911+0000","last_became_peered":"2024-09-21T07:35:16.776911+0000","last_unstale":"2024-09-21T07:35:17.779824+0000","last_undegraded":"2024-09-21T07:35:17.779824+0000","last_fullsized":"2024-09-21T07:35:17.779824+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:05.489425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.789141+0000","last_change":"2024-09-21T07:35:16.780348+0000","last_active":"2024-09-21T07:35:16.789141+0000","last_peered":"2024-09-21T07:35:16.789141+0000","last_clean":"2024-09-21T07:35:16.789141+0000","last_became_active":"2024-09-21T07:35:16.780006+0000","last_became_peered":"2024-09-21T07:35:16.780006+0000","last_unstale":"2024-09-21T07:35:16.789141+0000","last_undegraded":"2024-09-21T07:35:16.789141+0000","last_fullsized":"2024-09-21T07:35:16.789141+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:38.181721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.775059+0000","last_change":"2024-09-21T07:35:16.761639+0000","last_active":"2024-09-21T07:35:16.775059+0000","last_peered":"2024-09-21T07:35:16.775059+0000","last_clean":"2024-09-21T07:35:16.775059+0000","last_became_active":"2024-09-21T07:35:16.761231+0000","last_became_peered":"2024-09-21T07:35:16.761231+0000","last_unstale":"2024-09-21T07:35:16.775059+0000","last_undegraded":"2024-09-21T07:35:16.775059+0000","last_fullsized":"2024-09-21T07:35:16.775059+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:41.477998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.775504+0000","last_change":"2024-09-21T07:35:16.769703+0000","last_active":"2024-09-21T07:35:16.775504+0000","last_peered":"2024-09-21T07:35:16.775504+0000","last_clean":"2024-09-21T07:35:16.775504+0000","last_became_active":"2024-09-21T07:35:16.769306+0000","last_became_peered":"2024-09-21T07:35:16.769306+0000","last_unstale":"2024-09-21T07:35:16.775504+0000","last_undegraded":"2024-09-21T07:35:16.775504+0000","last_fullsized":"2024-09-21T07:35:16.775504+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:02.221815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T07:35:19.110128+0000","last_change":"2024-09-21T07:35:16.788542+0000","last_active":"2024-09-21T07:35:19.110128+0000","last_peered":"2024-09-21T07:35:19.110128+0000","last_clean":"2024-09-21T07:35:19.110128+0000","last_became_active":"2024-09-21T07:35:16.788207+0000","last_became_peered":"2024-09-21T07:35:16.788207+0000","last_unstale":"2024-09-21T07:35:19.110128+0000","last_undegraded":"2024-09-21T07:35:19.110128+0000","last_fullsized":"2024-09-21T07:35:19.110128+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:46.362140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.799668+0000","last_change":"2024-09-21T07:35:16.792511+0000","last_active":"2024-09-21T07:35:16.799668+0000","last_peered":"2024-09-21T07:35:16.799668+0000","last_clean":"2024-09-21T07:35:16.799668+0000","last_became_active":"2024-09-21T07:35:16.792095+0000","last_became_peered":"2024-09-21T07:35:16.792095+0000","last_unstale":"2024-09-21T07:35:16.799668+0000","last_undegraded":"2024-09-21T07:35:16.799668+0000","last_fullsized":"2024-09-21T07:35:16.799668+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:56.522045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.789095+0000","last_change":"2024-09-21T07:35:16.779904+0000","last_active":"2024-09-21T07:35:16.789095+0000","last_peered":"2024-09-21T07:35:16.789095+0000","last_clean":"2024-09-21T07:35:16.789095+0000","last_became_active":"2024-09-21T07:35:16.779548+0000","last_became_peered":"2024-09-21T07:35:16.779548+0000","last_unstale":"2024-09-21T07:35:16.789095+0000","last_undegraded":"2024-09-21T07:35:16.789095+0000","last_fullsized":"2024-09-21T07:35:16.789095+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:17.442893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.792032+0000","last_change":"2024-09-21T07:35:16.787033+0000","last_active":"2024-09-21T07:35:16.792032+0000","last_peered":"2024-09-21T07:35:16.792032+0000","last_clean":"2024-09-21T07:35:16.792032+0000","last_became_active":"2024-09-21T07:35:16.784639+0000","last_became_peered":"2024-09-21T07:35:16.784639+0000","last_unstale":"2024-09-21T07:35:16.792032+0000","last_undegraded":"2024-09-21T07:35:16.792032+0000","last_fullsized":"2024-09-21T07:35:16.792032+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:19:41.151420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T07:35:19.109445+0000","last_change":"2024-09-21T07:35:16.782138+0000","last_active":"2024-09-21T07:35:19.109445+0000","last_peered":"2024-09-21T07:35:19.109445+0000","last_clean":"2024-09-21T07:35:19.109445+0000","last_became_active":"2024-09-21T07:35:16.781705+0000","last_became_peered":"2024-09-21T07:35:16.781705+0000","last_unstale":"2024-09-21T07:35:19.109445+0000","last_undegraded":"2024-09-21T07:35:19.109445+0000","last_fullsized":"2024-09-21T07:35:19.109445+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:10.553140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.793719+0000","last_change":"2024-09-21T07:35:16.781495+0000","last_active":"2024-09-21T07:35:16.793719+0000","last_peered":"2024-09-21T07:35:16.793719+0000","last_clean":"2024-09-21T07:35:16.793719+0000","last_became_active":"2024-09-21T07:35:16.781096+0000","last_became_peered":"2024-09-21T07:35:16.781096+0000","last_unstale":"2024-09-21T07:35:16.793719+0000","last_undegraded":"2024-09-21T07:35:16.793719+0000","last_fullsized":"2024-09-21T07:35:16.793719+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:19:53.457207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.797687+0000","last_change":"2024-09-21T07:35:16.790402+0000","last_active":"2024-09-21T07:35:16.797687+0000","last_peered":"2024-09-21T07:35:16.797687+0000","last_clean":"2024-09-21T07:35:16.797687+0000","last_became_active":"2024-09-21T07:35:16.786935+0000","last_became_peered":"2024-09-21T07:35:16.786935+0000","last_unstale":"2024-09-21T07:35:16.797687+0000","last_undegraded":"2024-09-21T07:35:16.797687+0000","last_fullsized":"2024-09-21T07:35:16.797687+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:12:35.499903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T07:35:17.779281+0000","last_change":"2024-09-21T07:35:15.772382+0000","last_active":"2024-09-21T07:35:17.779281+0000","last_peered":"2024-09-21T07:35:17.779281+0000","last_clean":"2024-09-21T07:35:17.779281+0000","last_became_active":"2024-09-21T07:35:15.771916+0000","last_became_peered":"2024-09-21T07:35:15.771916+0000","last_unstale":"2024-09-21T07:35:17.779281+0000","last_undegraded":"2024-09-21T07:35:17.779281+0000","last_fullsized":"2024-09-21T07:35:17.779281+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:46.529511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":188,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.764544+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:35:16.764544+0000","last_peered":"2024-09-21T07:35:16.764544+0000","last_clean":"2024-09-21T07:35:16.764544+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:35:16.764544+0000","last_undegraded":"2024-09-21T07:35:16.764544+0000","last_fullsized":"2024-09-21T07:35:16.764544+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":62,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.755390+0000","last_change":"2024-09-21T07:34:24.539248+0000","last_active":"2024-09-21T07:35:16.755390+0000","last_peered":"2024-09-21T07:35:16.755390+0000","last_clean":"2024-09-21T07:35:16.755390+0000","last_became_active":"2024-09-21T07:34:23.273465+0000","last_became_peered":"2024-09-21T07:34:23.273465+0000","last_unstale":"2024-09-21T07:35:16.755390+0000","last_undegraded":"2024-09-21T07:35:16.755390+0000","last_fullsized":"2024-09-21T07:35:16.755390+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:55.517452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060627700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T07:35:19.110601+0000","last_change":"2024-09-21T07:35:16.798747+0000","last_active":"2024-09-21T07:35:19.110601+0000","last_peered":"2024-09-21T07:35:19.110601+0000","last_clean":"2024-09-21T07:35:19.110601+0000","last_became_active":"2024-09-21T07:35:16.798393+0000","last_became_peered":"2024-09-21T07:35:16.798393+0000","last_unstale":"2024-09-21T07:35:19.110601+0000","last_undegraded":"2024-09-21T07:35:19.110601+0000","last_fullsized":"2024-09-21T07:35:19.110601+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:34.812002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"61'1","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.795434+0000","last_change":"2024-09-21T07:35:15.777735+0000","last_active":"2024-09-21T07:35:16.795434+0000","last_peered":"2024-09-21T07:35:16.795434+0000","last_clean":"2024-09-21T07:35:16.795434+0000","last_became_active":"2024-09-21T07:35:15.777411+0000","last_became_peered":"2024-09-21T07:35:15.777411+0000","last_unstale":"2024-09-21T07:35:16.795434+0000","last_undegraded":"2024-09-21T07:35:16.795434+0000","last_fullsized":"2024-09-21T07:35:16.795434+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:51:55.267013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":61,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T07:35:19.109653+0000","last_change":"2024-09-21T07:34:24.597274+0000","last_active":"2024-09-21T07:35:19.109653+0000","last_peered":"2024-09-21T07:35:19.109653+0000","last_clean":"2024-09-21T07:35:19.109653+0000","last_became_active":"2024-09-21T07:34:23.279560+0000","last_became_peered":"2024-09-21T07:34:23.279560+0000","last_unstale":"2024-09-21T07:35:19.109653+0000","last_undegraded":"2024-09-21T07:35:19.109653+0000","last_fullsized":"2024-09-21T07:35:19.109653+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:15.845265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059699799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.788623+0000","last_change":"2024-09-21T07:35:16.778041+0000","last_active":"2024-09-21T07:35:16.788623+0000","last_peered":"2024-09-21T07:35:16.788623+0000","last_clean":"2024-09-21T07:35:16.788623+0000","last_became_active":"2024-09-21T07:35:16.777265+0000","last_became_peered":"2024-09-21T07:35:16.777265+0000","last_unstale":"2024-09-21T07:35:16.788623+0000","last_undegraded":"2024-09-21T07:35:16.788623+0000","last_fullsized":"2024-09-21T07:35:16.788623+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:00:49.777067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T07:35:17.780114+0000","last_change":"2024-09-21T07:35:15.771748+0000","last_active":"2024-09-21T07:35:17.780114+0000","last_peered":"2024-09-21T07:35:17.780114+0000","last_clean":"2024-09-21T07:35:17.780114+0000","last_became_active":"2024-09-21T07:35:15.771453+0000","last_became_peered":"2024-09-21T07:35:15.771453+0000","last_unstale":"2024-09-21T07:35:17.780114+0000","last_undegraded":"2024-09-21T07:35:17.780114+0000","last_fullsized":"2024-09-21T07:35:17.780114+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:20.453063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":63,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.764742+0000","last_change":"2024-09-21T07:34:24.259906+0000","last_active":"2024-09-21T07:35:16.764742+0000","last_peered":"2024-09-21T07:35:16.764742+0000","last_clean":"2024-09-21T07:35:16.764742+0000","last_became_active":"2024-09-21T07:34:23.635282+0000","last_became_peered":"2024-09-21T07:34:23.635282+0000","last_unstale":"2024-09-21T07:35:16.764742+0000","last_undegraded":"2024-09-21T07:35:16.764742+0000","last_fullsized":"2024-09-21T07:35:16.764742+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:00.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001527664,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.788032+0000","last_change":"2024-09-21T07:35:16.778542+0000","last_active":"2024-09-21T07:35:16.788032+0000","last_peered":"2024-09-21T07:35:16.788032+0000","last_clean":"2024-09-21T07:35:16.788032+0000","last_became_active":"2024-09-21T07:35:16.778202+0000","last_became_peered":"2024-09-21T07:35:16.778202+0000","last_unstale":"2024-09-21T07:35:16.788032+0000","last_undegraded":"2024-09-21T07:35:16.788032+0000","last_fullsized":"2024-09-21T07:35:16.788032+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:33.739027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"61'1","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.787864+0000","last_change":"2024-09-21T07:35:15.769004+0000","last_active":"2024-09-21T07:35:16.787864+0000","last_peered":"2024-09-21T07:35:16.787864+0000","last_clean":"2024-09-21T07:35:16.787864+0000","last_became_active":"2024-09-21T07:35:15.768705+0000","last_became_peered":"2024-09-21T07:35:15.768705+0000","last_unstale":"2024-09-21T07:35:16.787864+0000","last_undegraded":"2024-09-21T07:35:16.787864+0000","last_fullsized":"2024-09-21T07:35:16.787864+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:16.143968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":60,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.764496+0000","last_change":"2024-09-21T07:34:25.085454+0000","last_active":"2024-09-21T07:35:16.764496+0000","last_peered":"2024-09-21T07:35:16.764496+0000","last_clean":"2024-09-21T07:35:16.764496+0000","last_became_active":"2024-09-21T07:34:23.279205+0000","last_became_peered":"2024-09-21T07:34:23.279205+0000","last_unstale":"2024-09-21T07:35:16.764496+0000","last_undegraded":"2024-09-21T07:35:16.764496+0000","last_fullsized":"2024-09-21T07:35:16.764496+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:54.231666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095320599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.792070+0000","last_change":"2024-09-21T07:35:16.782202+0000","last_active":"2024-09-21T07:35:16.792070+0000","last_peered":"2024-09-21T07:35:16.792070+0000","last_clean":"2024-09-21T07:35:16.792070+0000","last_became_active":"2024-09-21T07:35:16.781868+0000","last_became_peered":"2024-09-21T07:35:16.781868+0000","last_unstale":"2024-09-21T07:35:16.792070+0000","last_undegraded":"2024-09-21T07:35:16.792070+0000","last_fullsized":"2024-09-21T07:35:16.792070+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:44:08.510363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.791376+0000","last_change":"2024-09-21T07:35:15.778418+0000","last_active":"2024-09-21T07:35:16.791376+0000","last_peered":"2024-09-21T07:35:16.791376+0000","last_clean":"2024-09-21T07:35:16.791376+0000","last_became_active":"2024-09-21T07:35:15.778001+0000","last_became_peered":"2024-09-21T07:35:15.778001+0000","last_unstale":"2024-09-21T07:35:16.791376+0000","last_undegraded":"2024-09-21T07:35:16.791376+0000","last_fullsized":"2024-09-21T07:35:16.791376+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:38.435523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":61,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.777782+0000","last_change":"2024-09-21T07:34:25.028521+0000","last_active":"2024-09-21T07:35:16.777782+0000","last_peered":"2024-09-21T07:35:16.777782+0000","last_clean":"2024-09-21T07:35:16.777782+0000","last_became_active":"2024-09-21T07:34:23.635467+0000","last_became_peered":"2024-09-21T07:34:23.635467+0000","last_unstale":"2024-09-21T07:35:16.777782+0000","last_undegraded":"2024-09-21T07:35:16.777782+0000","last_fullsized":"2024-09-21T07:35:16.777782+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:33.729123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000660558,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.789071+0000","last_change":"2024-09-21T07:35:16.781562+0000","last_active":"2024-09-21T07:35:16.789071+0000","last_peered":"2024-09-21T07:35:16.789071+0000","last_clean":"2024-09-21T07:35:16.789071+0000","last_became_active":"2024-09-21T07:35:16.780384+0000","last_became_peered":"2024-09-21T07:35:16.780384+0000","last_unstale":"2024-09-21T07:35:16.789071+0000","last_undegraded":"2024-09-21T07:35:16.789071+0000","last_fullsized":"2024-09-21T07:35:16.789071+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:57:16.255292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.768020+0000","last_change":"2024-09-21T07:35:15.771439+0000","last_active":"2024-09-21T07:35:16.768020+0000","last_peered":"2024-09-21T07:35:16.768020+0000","last_clean":"2024-09-21T07:35:16.768020+0000","last_became_active":"2024-09-21T07:35:15.771176+0000","last_became_peered":"2024-09-21T07:35:15.771176+0000","last_unstale":"2024-09-21T07:35:16.768020+0000","last_undegraded":"2024-09-21T07:35:16.768020+0000","last_fullsized":"2024-09-21T07:35:16.768020+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:30.194293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":66,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T07:35:17.780135+0000","last_change":"2024-09-21T07:34:24.260046+0000","last_active":"2024-09-21T07:35:17.780135+0000","last_peered":"2024-09-21T07:35:17.780135+0000","last_clean":"2024-09-21T07:35:17.780135+0000","last_became_active":"2024-09-21T07:34:23.273431+0000","last_became_peered":"2024-09-21T07:34:23.273431+0000","last_unstale":"2024-09-21T07:35:17.780135+0000","last_undegraded":"2024-09-21T07:35:17.780135+0000","last_fullsized":"2024-09-21T07:35:17.780135+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:18.876057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019803310000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.793747+0000","last_change":"2024-09-21T07:35:16.787188+0000","last_active":"2024-09-21T07:35:16.793747+0000","last_peered":"2024-09-21T07:35:16.793747+0000","last_clean":"2024-09-21T07:35:16.793747+0000","last_became_active":"2024-09-21T07:35:16.786877+0000","last_became_peered":"2024-09-21T07:35:16.786877+0000","last_unstale":"2024-09-21T07:35:16.793747+0000","last_undegraded":"2024-09-21T07:35:16.793747+0000","last_fullsized":"2024-09-21T07:35:16.793747+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:29:42.858624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.789751+0000","last_change":"2024-09-21T07:35:15.773125+0000","last_active":"2024-09-21T07:35:16.789751+0000","last_peered":"2024-09-21T07:35:16.789751+0000","last_clean":"2024-09-21T07:35:16.789751+0000","last_became_active":"2024-09-21T07:35:15.772316+0000","last_became_peered":"2024-09-21T07:35:15.772316+0000","last_unstale":"2024-09-21T07:35:16.789751+0000","last_undegraded":"2024-09-21T07:35:16.789751+0000","last_fullsized":"2024-09-21T07:35:16.789751+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:59.264954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":61,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.785693+0000","last_change":"2024-09-21T07:34:25.028640+0000","last_active":"2024-09-21T07:35:16.785693+0000","last_peered":"2024-09-21T07:35:16.785693+0000","last_clean":"2024-09-21T07:35:16.785693+0000","last_became_active":"2024-09-21T07:34:23.260950+0000","last_became_peered":"2024-09-21T07:34:23.260950+0000","last_unstale":"2024-09-21T07:35:16.785693+0000","last_undegraded":"2024-09-21T07:35:16.785693+0000","last_fullsized":"2024-09-21T07:35:16.785693+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:23:42.121398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055135099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.765442+0000","last_change":"2024-09-21T07:35:16.761498+0000","last_active":"2024-09-21T07:35:16.765442+0000","last_peered":"2024-09-21T07:35:16.765442+0000","last_clean":"2024-09-21T07:35:16.765442+0000","last_became_active":"2024-09-21T07:35:16.761105+0000","last_became_peered":"2024-09-21T07:35:16.761105+0000","last_unstale":"2024-09-21T07:35:16.765442+0000","last_undegraded":"2024-09-21T07:35:16.765442+0000","last_fullsized":"2024-09-21T07:35:16.765442+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:50.957994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.755285+0000","last_change":"2024-09-21T07:35:15.763918+0000","last_active":"2024-09-21T07:35:16.755285+0000","last_peered":"2024-09-21T07:35:16.755285+0000","last_clean":"2024-09-21T07:35:16.755285+0000","last_became_active":"2024-09-21T07:35:15.763596+0000","last_became_peered":"2024-09-21T07:35:15.763596+0000","last_unstale":"2024-09-21T07:35:16.755285+0000","last_undegraded":"2024-09-21T07:35:16.755285+0000","last_fullsized":"2024-09-21T07:35:16.755285+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:37.955816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":63,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.764202+0000","last_change":"2024-09-21T07:34:24.260665+0000","last_active":"2024-09-21T07:35:16.764202+0000","last_peered":"2024-09-21T07:35:16.764202+0000","last_clean":"2024-09-21T07:35:16.764202+0000","last_became_active":"2024-09-21T07:34:23.635909+0000","last_became_peered":"2024-09-21T07:34:23.635909+0000","last_unstale":"2024-09-21T07:35:16.764202+0000","last_undegraded":"2024-09-21T07:35:16.764202+0000","last_fullsized":"2024-09-21T07:35:16.764202+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:44.536885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.782220+0000","last_change":"2024-09-21T07:35:16.778786+0000","last_active":"2024-09-21T07:35:16.782220+0000","last_peered":"2024-09-21T07:35:16.782220+0000","last_clean":"2024-09-21T07:35:16.782220+0000","last_became_active":"2024-09-21T07:35:16.778450+0000","last_became_peered":"2024-09-21T07:35:16.778450+0000","last_unstale":"2024-09-21T07:35:16.782220+0000","last_undegraded":"2024-09-21T07:35:16.782220+0000","last_fullsized":"2024-09-21T07:35:16.782220+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:37.394072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.767888+0000","last_change":"2024-09-21T07:35:15.762424+0000","last_active":"2024-09-21T07:35:16.767888+0000","last_peered":"2024-09-21T07:35:16.767888+0000","last_clean":"2024-09-21T07:35:16.767888+0000","last_became_active":"2024-09-21T07:35:15.762109+0000","last_became_peered":"2024-09-21T07:35:15.762109+0000","last_unstale":"2024-09-21T07:35:16.767888+0000","last_undegraded":"2024-09-21T07:35:16.767888+0000","last_fullsized":"2024-09-21T07:35:16.767888+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:00.093906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":62,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.755314+0000","last_change":"2024-09-21T07:34:24.538845+0000","last_active":"2024-09-21T07:35:16.755314+0000","last_peered":"2024-09-21T07:35:16.755314+0000","last_clean":"2024-09-21T07:35:16.755314+0000","last_became_active":"2024-09-21T07:34:23.636372+0000","last_became_peered":"2024-09-21T07:34:23.636372+0000","last_unstale":"2024-09-21T07:35:16.755314+0000","last_undegraded":"2024-09-21T07:35:16.755314+0000","last_fullsized":"2024-09-21T07:35:16.755314+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:29.393813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841451,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.788577+0000","last_change":"2024-09-21T07:35:16.777683+0000","last_active":"2024-09-21T07:35:16.788577+0000","last_peered":"2024-09-21T07:35:16.788577+0000","last_clean":"2024-09-21T07:35:16.788577+0000","last_became_active":"2024-09-21T07:35:16.776971+0000","last_became_peered":"2024-09-21T07:35:16.776971+0000","last_unstale":"2024-09-21T07:35:16.788577+0000","last_undegraded":"2024-09-21T07:35:16.788577+0000","last_fullsized":"2024-09-21T07:35:16.788577+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:44:47.941274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.785884+0000","last_change":"2024-09-21T07:35:15.773061+0000","last_active":"2024-09-21T07:35:16.785884+0000","last_peered":"2024-09-21T07:35:16.785884+0000","last_clean":"2024-09-21T07:35:16.785884+0000","last_became_active":"2024-09-21T07:35:15.772705+0000","last_became_peered":"2024-09-21T07:35:15.772705+0000","last_unstale":"2024-09-21T07:35:16.785884+0000","last_undegraded":"2024-09-21T07:35:16.785884+0000","last_fullsized":"2024-09-21T07:35:16.785884+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:50.766243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":66,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T07:35:17.780483+0000","last_change":"2024-09-21T07:34:24.259686+0000","last_active":"2024-09-21T07:35:17.780483+0000","last_peered":"2024-09-21T07:35:17.780483+0000","last_clean":"2024-09-21T07:35:17.780483+0000","last_became_active":"2024-09-21T07:34:23.274405+0000","last_became_peered":"2024-09-21T07:34:23.274405+0000","last_unstale":"2024-09-21T07:35:17.780483+0000","last_undegraded":"2024-09-21T07:35:17.780483+0000","last_fullsized":"2024-09-21T07:35:17.780483+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:26.304693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022009920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.787809+0000","last_change":"2024-09-21T07:35:16.779064+0000","last_active":"2024-09-21T07:35:16.787809+0000","last_peered":"2024-09-21T07:35:16.787809+0000","last_clean":"2024-09-21T07:35:16.787809+0000","last_became_active":"2024-09-21T07:35:16.778764+0000","last_became_peered":"2024-09-21T07:35:16.778764+0000","last_unstale":"2024-09-21T07:35:16.787809+0000","last_undegraded":"2024-09-21T07:35:16.787809+0000","last_fullsized":"2024-09-21T07:35:16.787809+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:43.382121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.793246+0000","last_change":"2024-09-21T07:35:15.771995+0000","last_active":"2024-09-21T07:35:16.793246+0000","last_peered":"2024-09-21T07:35:16.793246+0000","last_clean":"2024-09-21T07:35:16.793246+0000","last_became_active":"2024-09-21T07:35:15.771691+0000","last_became_peered":"2024-09-21T07:35:15.771691+0000","last_unstale":"2024-09-21T07:35:16.793246+0000","last_undegraded":"2024-09-21T07:35:16.793246+0000","last_fullsized":"2024-09-21T07:35:16.793246+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:04:14.353917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":63,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.764414+0000","last_change":"2024-09-21T07:34:24.258754+0000","last_active":"2024-09-21T07:35:16.764414+0000","last_peered":"2024-09-21T07:35:16.764414+0000","last_clean":"2024-09-21T07:35:16.764414+0000","last_became_active":"2024-09-21T07:34:23.272657+0000","last_became_peered":"2024-09-21T07:34:23.272657+0000","last_unstale":"2024-09-21T07:35:16.764414+0000","last_undegraded":"2024-09-21T07:35:16.764414+0000","last_fullsized":"2024-09-21T07:35:16.764414+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:05.477099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067143200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.781770+0000","last_change":"2024-09-21T07:35:16.779289+0000","last_active":"2024-09-21T07:35:16.781770+0000","last_peered":"2024-09-21T07:35:16.781770+0000","last_clean":"2024-09-21T07:35:16.781770+0000","last_became_active":"2024-09-21T07:35:16.778146+0000","last_became_peered":"2024-09-21T07:35:16.778146+0000","last_unstale":"2024-09-21T07:35:16.781770+0000","last_undegraded":"2024-09-21T07:35:16.781770+0000","last_fullsized":"2024-09-21T07:35:16.781770+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:51.294357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.754996+0000","last_change":"2024-09-21T07:35:15.757150+0000","last_active":"2024-09-21T07:35:16.754996+0000","last_peered":"2024-09-21T07:35:16.754996+0000","last_clean":"2024-09-21T07:35:16.754996+0000","last_became_active":"2024-09-21T07:35:15.756816+0000","last_became_peered":"2024-09-21T07:35:15.756816+0000","last_unstale":"2024-09-21T07:35:16.754996+0000","last_undegraded":"2024-09-21T07:35:16.754996+0000","last_fullsized":"2024-09-21T07:35:16.754996+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:52.469665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":61,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.768191+0000","last_change":"2024-09-21T07:34:25.085232+0000","last_active":"2024-09-21T07:35:16.768191+0000","last_peered":"2024-09-21T07:35:16.768191+0000","last_clean":"2024-09-21T07:35:16.768191+0000","last_became_active":"2024-09-21T07:34:23.277807+0000","last_became_peered":"2024-09-21T07:34:23.277807+0000","last_unstale":"2024-09-21T07:35:16.768191+0000","last_undegraded":"2024-09-21T07:35:16.768191+0000","last_fullsized":"2024-09-21T07:35:16.768191+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:36.362919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086619099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.790466+0000","last_change":"2024-09-21T07:35:16.783210+0000","last_active":"2024-09-21T07:35:16.790466+0000","last_peered":"2024-09-21T07:35:16.790466+0000","last_clean":"2024-09-21T07:35:16.790466+0000","last_became_active":"2024-09-21T07:35:16.782534+0000","last_became_peered":"2024-09-21T07:35:16.782534+0000","last_unstale":"2024-09-21T07:35:16.790466+0000","last_undegraded":"2024-09-21T07:35:16.790466+0000","last_fullsized":"2024-09-21T07:35:16.790466+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:02.221815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T07:35:17.779585+0000","last_change":"2024-09-21T07:35:15.774096+0000","last_active":"2024-09-21T07:35:17.779585+0000","last_peered":"2024-09-21T07:35:17.779585+0000","last_clean":"2024-09-21T07:35:17.779585+0000","last_became_active":"2024-09-21T07:35:15.773756+0000","last_became_peered":"2024-09-21T07:35:15.773756+0000","last_unstale":"2024-09-21T07:35:17.779585+0000","last_undegraded":"2024-09-21T07:35:17.779585+0000","last_fullsized":"2024-09-21T07:35:17.779585+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:50.834792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":61,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T07:35:19.110857+0000","last_change":"2024-09-21T07:34:24.597672+0000","last_active":"2024-09-21T07:35:19.110857+0000","last_peered":"2024-09-21T07:35:19.110857+0000","last_clean":"2024-09-21T07:35:19.110857+0000","last_became_active":"2024-09-21T07:34:23.267770+0000","last_became_peered":"2024-09-21T07:34:23.267770+0000","last_unstale":"2024-09-21T07:35:19.110857+0000","last_undegraded":"2024-09-21T07:35:19.110857+0000","last_fullsized":"2024-09-21T07:35:19.110857+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:51.629476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086378500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.773367+0000","last_change":"2024-09-21T07:35:16.761921+0000","last_active":"2024-09-21T07:35:16.773367+0000","last_peered":"2024-09-21T07:35:16.773367+0000","last_clean":"2024-09-21T07:35:16.773367+0000","last_became_active":"2024-09-21T07:35:16.760976+0000","last_became_peered":"2024-09-21T07:35:16.760976+0000","last_unstale":"2024-09-21T07:35:16.773367+0000","last_undegraded":"2024-09-21T07:35:16.773367+0000","last_fullsized":"2024-09-21T07:35:16.773367+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:37.140236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.775942+0000","last_change":"2024-09-21T07:35:15.769225+0000","last_active":"2024-09-21T07:35:16.775942+0000","last_peered":"2024-09-21T07:35:16.775942+0000","last_clean":"2024-09-21T07:35:16.775942+0000","last_became_active":"2024-09-21T07:35:15.768922+0000","last_became_peered":"2024-09-21T07:35:15.768922+0000","last_unstale":"2024-09-21T07:35:16.775942+0000","last_undegraded":"2024-09-21T07:35:16.775942+0000","last_fullsized":"2024-09-21T07:35:16.775942+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:16.143968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":62,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.757342+0000","last_change":"2024-09-21T07:34:24.260181+0000","last_active":"2024-09-21T07:35:16.757342+0000","last_peered":"2024-09-21T07:35:16.757342+0000","last_clean":"2024-09-21T07:35:16.757342+0000","last_became_active":"2024-09-21T07:34:23.275580+0000","last_became_peered":"2024-09-21T07:34:23.275580+0000","last_unstale":"2024-09-21T07:35:16.757342+0000","last_undegraded":"2024-09-21T07:35:16.757342+0000","last_fullsized":"2024-09-21T07:35:16.757342+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:49.857622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014706980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.792227+0000","last_change":"2024-09-21T07:35:16.786379+0000","last_active":"2024-09-21T07:35:16.792227+0000","last_peered":"2024-09-21T07:35:16.792227+0000","last_clean":"2024-09-21T07:35:16.792227+0000","last_became_active":"2024-09-21T07:35:16.786061+0000","last_became_peered":"2024-09-21T07:35:16.786061+0000","last_unstale":"2024-09-21T07:35:16.792227+0000","last_undegraded":"2024-09-21T07:35:16.792227+0000","last_fullsized":"2024-09-21T07:35:16.792227+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:56:07.240152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.764137+0000","last_change":"2024-09-21T07:35:15.777167+0000","last_active":"2024-09-21T07:35:16.764137+0000","last_peered":"2024-09-21T07:35:16.764137+0000","last_clean":"2024-09-21T07:35:16.764137+0000","last_became_active":"2024-09-21T07:35:15.776916+0000","last_became_peered":"2024-09-21T07:35:15.776916+0000","last_unstale":"2024-09-21T07:35:16.764137+0000","last_undegraded":"2024-09-21T07:35:16.764137+0000","last_fullsized":"2024-09-21T07:35:16.764137+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:34.379866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":66,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T07:35:17.780246+0000","last_change":"2024-09-21T07:34:24.259112+0000","last_active":"2024-09-21T07:35:17.780246+0000","last_peered":"2024-09-21T07:35:17.780246+0000","last_clean":"2024-09-21T07:35:17.780246+0000","last_became_active":"2024-09-21T07:34:23.635444+0000","last_became_peered":"2024-09-21T07:34:23.635444+0000","last_unstale":"2024-09-21T07:35:17.780246+0000","last_undegraded":"2024-09-21T07:35:17.780246+0000","last_fullsized":"2024-09-21T07:35:17.780246+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:46.255385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001952636,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T07:35:19.109746+0000","last_change":"2024-09-21T07:35:16.795394+0000","last_active":"2024-09-21T07:35:19.109746+0000","last_peered":"2024-09-21T07:35:19.109746+0000","last_clean":"2024-09-21T07:35:19.109746+0000","last_became_active":"2024-09-21T07:35:16.795060+0000","last_became_peered":"2024-09-21T07:35:16.795060+0000","last_unstale":"2024-09-21T07:35:19.109746+0000","last_undegraded":"2024-09-21T07:35:19.109746+0000","last_fullsized":"2024-09-21T07:35:19.109746+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:04.035249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T07:35:17.779854+0000","last_change":"2024-09-21T07:35:15.768589+0000","last_active":"2024-09-21T07:35:17.779854+0000","last_peered":"2024-09-21T07:35:17.779854+0000","last_clean":"2024-09-21T07:35:17.779854+0000","last_became_active":"2024-09-21T07:35:15.767181+0000","last_became_peered":"2024-09-21T07:35:15.767181+0000","last_unstale":"2024-09-21T07:35:17.779854+0000","last_undegraded":"2024-09-21T07:35:17.779854+0000","last_fullsized":"2024-09-21T07:35:17.779854+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:48.666632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":62,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.754721+0000","last_change":"2024-09-21T07:34:24.539152+0000","last_active":"2024-09-21T07:35:16.754721+0000","last_peered":"2024-09-21T07:35:16.754721+0000","last_clean":"2024-09-21T07:35:16.754721+0000","last_became_active":"2024-09-21T07:34:23.259573+0000","last_became_peered":"2024-09-21T07:34:23.259573+0000","last_unstale":"2024-09-21T07:35:16.754721+0000","last_undegraded":"2024-09-21T07:35:16.754721+0000","last_fullsized":"2024-09-21T07:35:16.754721+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:26.986900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058108400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.799620+0000","last_change":"2024-09-21T07:35:16.790419+0000","last_active":"2024-09-21T07:35:16.799620+0000","last_peered":"2024-09-21T07:35:16.799620+0000","last_clean":"2024-09-21T07:35:16.799620+0000","last_became_active":"2024-09-21T07:35:16.790026+0000","last_became_peered":"2024-09-21T07:35:16.790026+0000","last_unstale":"2024-09-21T07:35:16.799620+0000","last_undegraded":"2024-09-21T07:35:16.799620+0000","last_fullsized":"2024-09-21T07:35:16.799620+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:01:03.904960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.792366+0000","last_change":"2024-09-21T07:35:15.762119+0000","last_active":"2024-09-21T07:35:16.792366+0000","last_peered":"2024-09-21T07:35:16.792366+0000","last_clean":"2024-09-21T07:35:16.792366+0000","last_became_active":"2024-09-21T07:35:15.761816+0000","last_became_peered":"2024-09-21T07:35:15.761816+0000","last_unstale":"2024-09-21T07:35:16.792366+0000","last_undegraded":"2024-09-21T07:35:16.792366+0000","last_fullsized":"2024-09-21T07:35:16.792366+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:55:20.866012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":63,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.762411+0000","last_change":"2024-09-21T07:34:24.259902+0000","last_active":"2024-09-21T07:35:16.762411+0000","last_peered":"2024-09-21T07:35:16.762411+0000","last_clean":"2024-09-21T07:35:16.762411+0000","last_became_active":"2024-09-21T07:34:23.271902+0000","last_became_peered":"2024-09-21T07:34:23.271902+0000","last_unstale":"2024-09-21T07:35:16.762411+0000","last_undegraded":"2024-09-21T07:35:16.762411+0000","last_fullsized":"2024-09-21T07:35:16.762411+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:59.439525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001686358,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.791866+0000","last_change":"2024-09-21T07:35:16.788234+0000","last_active":"2024-09-21T07:35:16.791866+0000","last_peered":"2024-09-21T07:35:16.791866+0000","last_clean":"2024-09-21T07:35:16.791866+0000","last_became_active":"2024-09-21T07:35:16.787916+0000","last_became_peered":"2024-09-21T07:35:16.787916+0000","last_unstale":"2024-09-21T07:35:16.791866+0000","last_undegraded":"2024-09-21T07:35:16.791866+0000","last_fullsized":"2024-09-21T07:35:16.791866+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:49.059613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.784022+0000","last_change":"2024-09-21T07:35:15.769948+0000","last_active":"2024-09-21T07:35:16.784022+0000","last_peered":"2024-09-21T07:35:16.784022+0000","last_clean":"2024-09-21T07:35:16.784022+0000","last_became_active":"2024-09-21T07:35:15.769514+0000","last_became_peered":"2024-09-21T07:35:15.769514+0000","last_unstale":"2024-09-21T07:35:16.784022+0000","last_undegraded":"2024-09-21T07:35:16.784022+0000","last_fullsized":"2024-09-21T07:35:16.784022+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:54.510608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":66,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T07:35:17.780420+0000","last_change":"2024-09-21T07:34:24.259662+0000","last_active":"2024-09-21T07:35:17.780420+0000","last_peered":"2024-09-21T07:35:17.780420+0000","last_clean":"2024-09-21T07:35:17.780420+0000","last_became_active":"2024-09-21T07:34:23.278812+0000","last_became_peered":"2024-09-21T07:34:23.278812+0000","last_unstale":"2024-09-21T07:35:17.780420+0000","last_undegraded":"2024-09-21T07:35:17.780420+0000","last_fullsized":"2024-09-21T07:35:17.780420+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:49.829974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356463,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.792258+0000","last_change":"2024-09-21T07:35:16.786498+0000","last_active":"2024-09-21T07:35:16.792258+0000","last_peered":"2024-09-21T07:35:16.792258+0000","last_clean":"2024-09-21T07:35:16.792258+0000","last_became_active":"2024-09-21T07:35:16.786050+0000","last_became_peered":"2024-09-21T07:35:16.786050+0000","last_unstale":"2024-09-21T07:35:16.792258+0000","last_undegraded":"2024-09-21T07:35:16.792258+0000","last_fullsized":"2024-09-21T07:35:16.792258+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:58.039151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"61'4","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.801341+0000","last_change":"2024-09-21T07:35:15.774334+0000","last_active":"2024-09-21T07:35:16.801341+0000","last_peered":"2024-09-21T07:35:16.801341+0000","last_clean":"2024-09-21T07:35:16.801341+0000","last_became_active":"2024-09-21T07:35:15.773656+0000","last_became_peered":"2024-09-21T07:35:15.773656+0000","last_unstale":"2024-09-21T07:35:16.801341+0000","last_undegraded":"2024-09-21T07:35:16.801341+0000","last_fullsized":"2024-09-21T07:35:16.801341+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:39.128447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":61,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.764415+0000","last_change":"2024-09-21T07:34:25.084861+0000","last_active":"2024-09-21T07:35:16.764415+0000","last_peered":"2024-09-21T07:35:16.764415+0000","last_clean":"2024-09-21T07:35:16.764415+0000","last_became_active":"2024-09-21T07:34:23.277708+0000","last_became_peered":"2024-09-21T07:34:23.277708+0000","last_unstale":"2024-09-21T07:35:16.764415+0000","last_undegraded":"2024-09-21T07:35:16.764415+0000","last_fullsized":"2024-09-21T07:35:16.764415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:40.816488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000597566,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T07:35:19.110393+0000","last_change":"2024-09-21T07:35:16.793393+0000","last_active":"2024-09-21T07:35:19.110393+0000","last_peered":"2024-09-21T07:35:19.110393+0000","last_clean":"2024-09-21T07:35:19.110393+0000","last_became_active":"2024-09-21T07:35:16.793049+0000","last_became_peered":"2024-09-21T07:35:16.793049+0000","last_unstale":"2024-09-21T07:35:19.110393+0000","last_undegraded":"2024-09-21T07:35:19.110393+0000","last_fullsized":"2024-09-21T07:35:19.110393+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:39:07.102343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"61'1","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.793088+0000","last_change":"2024-09-21T07:35:15.774203+0000","last_active":"2024-09-21T07:35:16.793088+0000","last_peered":"2024-09-21T07:35:16.793088+0000","last_clean":"2024-09-21T07:35:16.793088+0000","last_became_active":"2024-09-21T07:35:15.773853+0000","last_became_peered":"2024-09-21T07:35:15.773853+0000","last_unstale":"2024-09-21T07:35:16.793088+0000","last_undegraded":"2024-09-21T07:35:16.793088+0000","last_fullsized":"2024-09-21T07:35:16.793088+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:32.736201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":66,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T07:35:17.779689+0000","last_change":"2024-09-21T07:34:24.258709+0000","last_active":"2024-09-21T07:35:17.779689+0000","last_peered":"2024-09-21T07:35:17.779689+0000","last_clean":"2024-09-21T07:35:17.779689+0000","last_became_active":"2024-09-21T07:34:23.279042+0000","last_became_peered":"2024-09-21T07:34:23.279042+0000","last_unstale":"2024-09-21T07:35:17.779689+0000","last_undegraded":"2024-09-21T07:35:17.779689+0000","last_fullsized":"2024-09-21T07:35:17.779689+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:01.762117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019282100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.791522+0000","last_change":"2024-09-21T07:35:16.781881+0000","last_active":"2024-09-21T07:35:16.791522+0000","last_peered":"2024-09-21T07:35:16.791522+0000","last_clean":"2024-09-21T07:35:16.791522+0000","last_became_active":"2024-09-21T07:35:16.780834+0000","last_became_peered":"2024-09-21T07:35:16.780834+0000","last_unstale":"2024-09-21T07:35:16.791522+0000","last_undegraded":"2024-09-21T07:35:16.791522+0000","last_fullsized":"2024-09-21T07:35:16.791522+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:04.681922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T07:35:17.779519+0000","last_change":"2024-09-21T07:35:15.775359+0000","last_active":"2024-09-21T07:35:17.779519+0000","last_peered":"2024-09-21T07:35:17.779519+0000","last_clean":"2024-09-21T07:35:17.779519+0000","last_became_active":"2024-09-21T07:35:15.774998+0000","last_became_peered":"2024-09-21T07:35:15.774998+0000","last_unstale":"2024-09-21T07:35:17.779519+0000","last_undegraded":"2024-09-21T07:35:17.779519+0000","last_fullsized":"2024-09-21T07:35:17.779519+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:00:06.261901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":61,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.768313+0000","last_change":"2024-09-21T07:34:25.085635+0000","last_active":"2024-09-21T07:35:16.768313+0000","last_peered":"2024-09-21T07:35:16.768313+0000","last_clean":"2024-09-21T07:35:16.768313+0000","last_became_active":"2024-09-21T07:34:23.273566+0000","last_became_peered":"2024-09-21T07:34:23.273566+0000","last_unstale":"2024-09-21T07:35:16.768313+0000","last_undegraded":"2024-09-21T07:35:16.768313+0000","last_fullsized":"2024-09-21T07:35:16.768313+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:14.142272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079897300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T07:35:17.780202+0000","last_change":"2024-09-21T07:35:16.779260+0000","last_active":"2024-09-21T07:35:17.780202+0000","last_peered":"2024-09-21T07:35:17.780202+0000","last_clean":"2024-09-21T07:35:17.780202+0000","last_became_active":"2024-09-21T07:35:16.778922+0000","last_became_peered":"2024-09-21T07:35:16.778922+0000","last_unstale":"2024-09-21T07:35:17.780202+0000","last_undegraded":"2024-09-21T07:35:17.780202+0000","last_fullsized":"2024-09-21T07:35:17.780202+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:17.868057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.772198+0000","last_change":"2024-09-21T07:35:15.772141+0000","last_active":"2024-09-21T07:35:16.772198+0000","last_peered":"2024-09-21T07:35:16.772198+0000","last_clean":"2024-09-21T07:35:16.772198+0000","last_became_active":"2024-09-21T07:35:15.771819+0000","last_became_peered":"2024-09-21T07:35:15.771819+0000","last_unstale":"2024-09-21T07:35:16.772198+0000","last_undegraded":"2024-09-21T07:35:16.772198+0000","last_fullsized":"2024-09-21T07:35:16.772198+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:48.972425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":61,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T07:35:19.110751+0000","last_change":"2024-09-21T07:34:24.597926+0000","last_active":"2024-09-21T07:35:19.110751+0000","last_peered":"2024-09-21T07:35:19.110751+0000","last_clean":"2024-09-21T07:35:19.110751+0000","last_became_active":"2024-09-21T07:34:23.268926+0000","last_became_peered":"2024-09-21T07:34:23.268926+0000","last_unstale":"2024-09-21T07:35:19.110751+0000","last_undegraded":"2024-09-21T07:35:19.110751+0000","last_fullsized":"2024-09-21T07:35:19.110751+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:46.190752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101851,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.789907+0000","last_change":"2024-09-21T07:35:16.784521+0000","last_active":"2024-09-21T07:35:16.789907+0000","last_peered":"2024-09-21T07:35:16.789907+0000","last_clean":"2024-09-21T07:35:16.789907+0000","last_became_active":"2024-09-21T07:35:16.784211+0000","last_became_peered":"2024-09-21T07:35:16.784211+0000","last_unstale":"2024-09-21T07:35:16.789907+0000","last_undegraded":"2024-09-21T07:35:16.789907+0000","last_fullsized":"2024-09-21T07:35:16.789907+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:07:50.501826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"61'1","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T07:35:17.780238+0000","last_change":"2024-09-21T07:35:15.769243+0000","last_active":"2024-09-21T07:35:17.780238+0000","last_peered":"2024-09-21T07:35:17.780238+0000","last_clean":"2024-09-21T07:35:17.780238+0000","last_became_active":"2024-09-21T07:35:15.768915+0000","last_became_peered":"2024-09-21T07:35:15.768915+0000","last_unstale":"2024-09-21T07:35:17.780238+0000","last_undegraded":"2024-09-21T07:35:17.780238+0000","last_fullsized":"2024-09-21T07:35:17.780238+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:34:43.549603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":62,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.757280+0000","last_change":"2024-09-21T07:34:24.259794+0000","last_active":"2024-09-21T07:35:16.757280+0000","last_peered":"2024-09-21T07:35:16.757280+0000","last_clean":"2024-09-21T07:35:16.757280+0000","last_became_active":"2024-09-21T07:34:23.265679+0000","last_became_peered":"2024-09-21T07:34:23.265679+0000","last_unstale":"2024-09-21T07:35:16.757280+0000","last_undegraded":"2024-09-21T07:35:16.757280+0000","last_fullsized":"2024-09-21T07:35:16.757280+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:02.575796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001298518,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.789145+0000","last_change":"2024-09-21T07:35:16.783085+0000","last_active":"2024-09-21T07:35:16.789145+0000","last_peered":"2024-09-21T07:35:16.789145+0000","last_clean":"2024-09-21T07:35:16.789145+0000","last_became_active":"2024-09-21T07:35:16.782399+0000","last_became_peered":"2024-09-21T07:35:16.782399+0000","last_unstale":"2024-09-21T07:35:16.789145+0000","last_undegraded":"2024-09-21T07:35:16.789145+0000","last_fullsized":"2024-09-21T07:35:16.789145+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:41.477998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"61'1","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T07:35:17.779736+0000","last_change":"2024-09-21T07:35:15.766345+0000","last_active":"2024-09-21T07:35:17.779736+0000","last_peered":"2024-09-21T07:35:17.779736+0000","last_clean":"2024-09-21T07:35:17.779736+0000","last_became_active":"2024-09-21T07:35:15.766006+0000","last_became_peered":"2024-09-21T07:35:15.766006+0000","last_unstale":"2024-09-21T07:35:17.779736+0000","last_undegraded":"2024-09-21T07:35:17.779736+0000","last_fullsized":"2024-09-21T07:35:17.779736+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:01.494161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":63,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.763589+0000","last_change":"2024-09-21T07:34:24.260330+0000","last_active":"2024-09-21T07:35:16.763589+0000","last_peered":"2024-09-21T07:35:16.763589+0000","last_clean":"2024-09-21T07:35:16.763589+0000","last_became_active":"2024-09-21T07:34:23.637210+0000","last_became_peered":"2024-09-21T07:34:23.637210+0000","last_unstale":"2024-09-21T07:35:16.763589+0000","last_undegraded":"2024-09-21T07:35:16.763589+0000","last_fullsized":"2024-09-21T07:35:16.763589+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.488451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001537093,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.792367+0000","last_change":"2024-09-21T07:35:16.789414+0000","last_active":"2024-09-21T07:35:16.792367+0000","last_peered":"2024-09-21T07:35:16.792367+0000","last_clean":"2024-09-21T07:35:16.792367+0000","last_became_active":"2024-09-21T07:35:16.789051+0000","last_became_peered":"2024-09-21T07:35:16.789051+0000","last_unstale":"2024-09-21T07:35:16.792367+0000","last_undegraded":"2024-09-21T07:35:16.792367+0000","last_fullsized":"2024-09-21T07:35:16.792367+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:48.459529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"61'3","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.795293+0000","last_change":"2024-09-21T07:35:15.777864+0000","last_active":"2024-09-21T07:35:16.795293+0000","last_peered":"2024-09-21T07:35:16.795293+0000","last_clean":"2024-09-21T07:35:16.795293+0000","last_became_active":"2024-09-21T07:35:15.777554+0000","last_became_peered":"2024-09-21T07:35:15.777554+0000","last_unstale":"2024-09-21T07:35:16.795293+0000","last_undegraded":"2024-09-21T07:35:16.795293+0000","last_fullsized":"2024-09-21T07:35:16.795293+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:50.028374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":61,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T07:35:19.110210+0000","last_change":"2024-09-21T07:34:24.597286+0000","last_active":"2024-09-21T07:35:19.110210+0000","last_peered":"2024-09-21T07:35:19.110210+0000","last_clean":"2024-09-21T07:35:19.110210+0000","last_became_active":"2024-09-21T07:34:23.267133+0000","last_became_peered":"2024-09-21T07:34:23.267133+0000","last_unstale":"2024-09-21T07:35:19.110210+0000","last_undegraded":"2024-09-21T07:35:19.110210+0000","last_fullsized":"2024-09-21T07:35:19.110210+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:18.257010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T07:35:19.110807+0000","last_change":"2024-09-21T07:35:16.790768+0000","last_active":"2024-09-21T07:35:19.110807+0000","last_peered":"2024-09-21T07:35:19.110807+0000","last_clean":"2024-09-21T07:35:19.110807+0000","last_became_active":"2024-09-21T07:35:16.790447+0000","last_became_peered":"2024-09-21T07:35:16.790447+0000","last_unstale":"2024-09-21T07:35:19.110807+0000","last_undegraded":"2024-09-21T07:35:19.110807+0000","last_fullsized":"2024-09-21T07:35:19.110807+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:45:52.748187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"61'6","reported_seq":19,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.802587+0000","last_change":"2024-09-21T07:35:15.776385+0000","last_active":"2024-09-21T07:35:16.802587+0000","last_peered":"2024-09-21T07:35:16.802587+0000","last_clean":"2024-09-21T07:35:16.802587+0000","last_became_active":"2024-09-21T07:35:15.775925+0000","last_became_peered":"2024-09-21T07:35:15.775925+0000","last_unstale":"2024-09-21T07:35:16.802587+0000","last_undegraded":"2024-09-21T07:35:16.802587+0000","last_fullsized":"2024-09-21T07:35:16.802587+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:40.466844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":66,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T07:35:17.780377+0000","last_change":"2024-09-21T07:34:24.258712+0000","last_active":"2024-09-21T07:35:17.780377+0000","last_peered":"2024-09-21T07:35:17.780377+0000","last_clean":"2024-09-21T07:35:17.780377+0000","last_became_active":"2024-09-21T07:34:23.274692+0000","last_became_peered":"2024-09-21T07:34:23.274692+0000","last_unstale":"2024-09-21T07:35:17.780377+0000","last_undegraded":"2024-09-21T07:35:17.780377+0000","last_fullsized":"2024-09-21T07:35:17.780377+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:43.279454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018051479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.793825+0000","last_change":"2024-09-21T07:35:16.784678+0000","last_active":"2024-09-21T07:35:16.793825+0000","last_peered":"2024-09-21T07:35:16.793825+0000","last_clean":"2024-09-21T07:35:16.793825+0000","last_became_active":"2024-09-21T07:35:16.784376+0000","last_became_peered":"2024-09-21T07:35:16.784376+0000","last_unstale":"2024-09-21T07:35:16.793825+0000","last_undegraded":"2024-09-21T07:35:16.793825+0000","last_fullsized":"2024-09-21T07:35:16.793825+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:01.158445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.764645+0000","last_change":"2024-09-21T07:35:15.781466+0000","last_active":"2024-09-21T07:35:16.764645+0000","last_peered":"2024-09-21T07:35:16.764645+0000","last_clean":"2024-09-21T07:35:16.764645+0000","last_became_active":"2024-09-21T07:35:15.781232+0000","last_became_peered":"2024-09-21T07:35:15.781232+0000","last_unstale":"2024-09-21T07:35:16.764645+0000","last_undegraded":"2024-09-21T07:35:16.764645+0000","last_fullsized":"2024-09-21T07:35:16.764645+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:44.771518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":61,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.764378+0000","last_change":"2024-09-21T07:34:25.084849+0000","last_active":"2024-09-21T07:35:16.764378+0000","last_peered":"2024-09-21T07:35:16.764378+0000","last_clean":"2024-09-21T07:35:16.764378+0000","last_became_active":"2024-09-21T07:34:23.277713+0000","last_became_peered":"2024-09-21T07:34:23.277713+0000","last_unstale":"2024-09-21T07:35:16.764378+0000","last_undegraded":"2024-09-21T07:35:16.764378+0000","last_fullsized":"2024-09-21T07:35:16.764378+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:05.811810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059079499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T07:35:17.780191+0000","last_change":"2024-09-21T07:35:16.778192+0000","last_active":"2024-09-21T07:35:17.780191+0000","last_peered":"2024-09-21T07:35:17.780191+0000","last_clean":"2024-09-21T07:35:17.780191+0000","last_became_active":"2024-09-21T07:35:16.777872+0000","last_became_peered":"2024-09-21T07:35:16.777872+0000","last_unstale":"2024-09-21T07:35:17.780191+0000","last_undegraded":"2024-09-21T07:35:17.780191+0000","last_fullsized":"2024-09-21T07:35:17.780191+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:11.466900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T07:35:19.110707+0000","last_change":"2024-09-21T07:35:15.776840+0000","last_active":"2024-09-21T07:35:19.110707+0000","last_peered":"2024-09-21T07:35:19.110707+0000","last_clean":"2024-09-21T07:35:19.110707+0000","last_became_active":"2024-09-21T07:35:15.776512+0000","last_became_peered":"2024-09-21T07:35:15.776512+0000","last_unstale":"2024-09-21T07:35:19.110707+0000","last_undegraded":"2024-09-21T07:35:19.110707+0000","last_fullsized":"2024-09-21T07:35:19.110707+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:40.457227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":62,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.755710+0000","last_change":"2024-09-21T07:34:24.538610+0000","last_active":"2024-09-21T07:35:16.755710+0000","last_peered":"2024-09-21T07:35:16.755710+0000","last_clean":"2024-09-21T07:35:16.755710+0000","last_became_active":"2024-09-21T07:34:23.259700+0000","last_became_peered":"2024-09-21T07:34:23.259700+0000","last_unstale":"2024-09-21T07:35:16.755710+0000","last_undegraded":"2024-09-21T07:35:16.755710+0000","last_fullsized":"2024-09-21T07:35:16.755710+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:57.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070522399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.792114+0000","last_change":"2024-09-21T07:35:16.782226+0000","last_active":"2024-09-21T07:35:16.792114+0000","last_peered":"2024-09-21T07:35:16.792114+0000","last_clean":"2024-09-21T07:35:16.792114+0000","last_became_active":"2024-09-21T07:35:16.781232+0000","last_became_peered":"2024-09-21T07:35:16.781232+0000","last_unstale":"2024-09-21T07:35:16.792114+0000","last_undegraded":"2024-09-21T07:35:16.792114+0000","last_fullsized":"2024-09-21T07:35:16.792114+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:22.916044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.763992+0000","last_change":"2024-09-21T07:35:15.774672+0000","last_active":"2024-09-21T07:35:16.763992+0000","last_peered":"2024-09-21T07:35:16.763992+0000","last_clean":"2024-09-21T07:35:16.763992+0000","last_became_active":"2024-09-21T07:35:15.774260+0000","last_became_peered":"2024-09-21T07:35:15.774260+0000","last_unstale":"2024-09-21T07:35:16.763992+0000","last_undegraded":"2024-09-21T07:35:16.763992+0000","last_fullsized":"2024-09-21T07:35:16.763992+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:33.522625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":61,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T07:35:19.110859+0000","last_change":"2024-09-21T07:34:24.597597+0000","last_active":"2024-09-21T07:35:19.110859+0000","last_peered":"2024-09-21T07:35:19.110859+0000","last_clean":"2024-09-21T07:35:19.110859+0000","last_became_active":"2024-09-21T07:34:23.635204+0000","last_became_peered":"2024-09-21T07:34:23.635204+0000","last_unstale":"2024-09-21T07:35:19.110859+0000","last_undegraded":"2024-09-21T07:35:19.110859+0000","last_fullsized":"2024-09-21T07:35:19.110859+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:48.644789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081139199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.799763+0000","last_change":"2024-09-21T07:35:16.794557+0000","last_active":"2024-09-21T07:35:16.799763+0000","last_peered":"2024-09-21T07:35:16.799763+0000","last_clean":"2024-09-21T07:35:16.799763+0000","last_became_active":"2024-09-21T07:35:16.794227+0000","last_became_peered":"2024-09-21T07:35:16.794227+0000","last_unstale":"2024-09-21T07:35:16.799763+0000","last_undegraded":"2024-09-21T07:35:16.799763+0000","last_fullsized":"2024-09-21T07:35:16.799763+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:19:41.151420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"61'8","reported_seq":23,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T07:35:19.109585+0000","last_change":"2024-09-21T07:35:15.773449+0000","last_active":"2024-09-21T07:35:19.109585+0000","last_peered":"2024-09-21T07:35:19.109585+0000","last_clean":"2024-09-21T07:35:19.109585+0000","last_became_active":"2024-09-21T07:35:15.772416+0000","last_became_peered":"2024-09-21T07:35:15.772416+0000","last_unstale":"2024-09-21T07:35:19.109585+0000","last_undegraded":"2024-09-21T07:35:19.109585+0000","last_fullsized":"2024-09-21T07:35:19.109585+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:14.233271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":66,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T07:35:17.780468+0000","last_change":"2024-09-21T07:34:24.259026+0000","last_active":"2024-09-21T07:35:17.780468+0000","last_peered":"2024-09-21T07:35:17.780468+0000","last_clean":"2024-09-21T07:35:17.780468+0000","last_became_active":"2024-09-21T07:34:23.635456+0000","last_became_peered":"2024-09-21T07:34:23.635456+0000","last_unstale":"2024-09-21T07:35:17.780468+0000","last_undegraded":"2024-09-21T07:35:17.780468+0000","last_fullsized":"2024-09-21T07:35:17.780468+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:09.592752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018654419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-21T07:35:19.109334+0000","last_change":"2024-09-21T07:35:16.799264+0000","last_active":"2024-09-21T07:35:19.109334+0000","last_peered":"2024-09-21T07:35:19.109334+0000","last_clean":"2024-09-21T07:35:19.109334+0000","last_became_active":"2024-09-21T07:35:16.798365+0000","last_became_peered":"2024-09-21T07:35:16.798365+0000","last_unstale":"2024-09-21T07:35:19.109334+0000","last_undegraded":"2024-09-21T07:35:19.109334+0000","last_fullsized":"2024-09-21T07:35:19.109334+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:31:04.765517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.793555+0000","last_change":"2024-09-21T07:35:15.775733+0000","last_active":"2024-09-21T07:35:16.793555+0000","last_peered":"2024-09-21T07:35:16.793555+0000","last_clean":"2024-09-21T07:35:16.793555+0000","last_became_active":"2024-09-21T07:35:15.775407+0000","last_became_peered":"2024-09-21T07:35:15.775407+0000","last_unstale":"2024-09-21T07:35:16.793555+0000","last_undegraded":"2024-09-21T07:35:16.793555+0000","last_fullsized":"2024-09-21T07:35:16.793555+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:24:28.776627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":63,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.762721+0000","last_change":"2024-09-21T07:34:24.260300+0000","last_active":"2024-09-21T07:35:16.762721+0000","last_peered":"2024-09-21T07:35:16.762721+0000","last_clean":"2024-09-21T07:35:16.762721+0000","last_became_active":"2024-09-21T07:34:23.270128+0000","last_became_peered":"2024-09-21T07:34:23.270128+0000","last_unstale":"2024-09-21T07:35:16.762721+0000","last_undegraded":"2024-09-21T07:35:16.762721+0000","last_fullsized":"2024-09-21T07:35:16.762721+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:25.260931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00176651,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.787612+0000","last_change":"2024-09-21T07:35:16.779848+0000","last_active":"2024-09-21T07:35:16.787612+0000","last_peered":"2024-09-21T07:35:16.787612+0000","last_clean":"2024-09-21T07:35:16.787612+0000","last_became_active":"2024-09-21T07:35:16.779541+0000","last_became_peered":"2024-09-21T07:35:16.779541+0000","last_unstale":"2024-09-21T07:35:16.787612+0000","last_undegraded":"2024-09-21T07:35:16.787612+0000","last_fullsized":"2024-09-21T07:35:16.787612+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:15.735669+0000","last_clean_scrub_stamp":"2024-09-21T07:35:15.735669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:41.469670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.755140+0000","last_change":"2024-09-21T07:35:15.763268+0000","last_active":"2024-09-21T07:35:16.755140+0000","last_peered":"2024-09-21T07:35:16.755140+0000","last_clean":"2024-09-21T07:35:16.755140+0000","last_became_active":"2024-09-21T07:35:15.762901+0000","last_became_peered":"2024-09-21T07:35:15.762901+0000","last_unstale":"2024-09-21T07:35:16.755140+0000","last_undegraded":"2024-09-21T07:35:16.755140+0000","last_fullsized":"2024-09-21T07:35:16.755140+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:14.732843+0000","last_clean_scrub_stamp":"2024-09-21T07:35:14.732843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:45.928812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":61,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-21T07:35:16.791174+0000","last_change":"2024-09-21T07:34:25.028904+0000","last_active":"2024-09-21T07:35:16.791174+0000","last_peered":"2024-09-21T07:35:16.791174+0000","last_clean":"2024-09-21T07:35:16.791174+0000","last_became_active":"2024-09-21T07:34:23.260942+0000","last_became_peered":"2024-09-21T07:34:23.260942+0000","last_unstale":"2024-09-21T07:35:16.791174+0000","last_undegraded":"2024-09-21T07:35:16.791174+0000","last_fullsized":"2024-09-21T07:35:16.791174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:22.709328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060553899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509470,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32500,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339340,"statfs":{"total":96636764160,"available":96603484160,"internally_reserved":0,"allocated":1335296,"data_stored":907055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509472,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31904,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339936,"statfs":{"total":96636764160,"available":96604094464,"internally_reserved":0,"allocated":729088,"data_stored":315945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509470,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31900,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339940,"statfs":{"total":96636764160,"available":96604098560,"internally_reserved":0,"allocated":729088,"data_stored":315933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509471,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31884,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339956,"statfs":{"total":96636764160,"available":96604114944,"internally_reserved":0,"allocated":712704,"data_stored":308783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509472,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31924,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339916,"statfs":{"total":96636764160,"available":96604073984,"internally_reserved":0,"allocated":753664,"data_stored":318287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509472,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32480,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1318912,"data_stored":906279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542176,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31920,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339920,"statfs":{"total":96636764160,"available":96604078080,"internally_reserved":0,"allocated":749568,"data_stored":324176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542175,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31916,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339924,"statfs":{"total":96636764160,"available":96604082176,"internally_reserved":0,"allocated":737280,"data_stored":322564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:35:21.785 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T07:35:21.785 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T07:35:22.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.054+0000 7f175ca00640 1 -- 172.21.15.39:0/686821498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17580cbee0 msgr2=0x7f17581622f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:22.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.054+0000 7f175ca00640 1 --2- 172.21.15.39:0/686821498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17580cbee0 0x7f17581622f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1750005fd0 tx=0x7f175002aa90 comp rx=0 tx=0).stop 2024-09-21T07:35:22.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.054+0000 7f175ca00640 1 -- 172.21.15.39:0/686821498 shutdown_connections 2024-09-21T07:35:22.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.054+0000 7f175ca00640 1 --2- 172.21.15.39:0/686821498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17580cbee0 0x7f17581622f0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:22.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.054+0000 7f175ca00640 1 --2- 172.21.15.39:0/686821498 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f17580c9930 0x7f1758144b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:22.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.054+0000 7f175ca00640 1 -- 172.21.15.39:0/686821498 >> 172.21.15.39:0/686821498 conn(0x7f175806ee50 msgr2=0x7f175806f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:22.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.054+0000 7f175ca00640 1 -- 172.21.15.39:0/686821498 shutdown_connections 2024-09-21T07:35:22.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.054+0000 7f175ca00640 1 -- 172.21.15.39:0/686821498 wait complete. 2024-09-21T07:35:22.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.057+0000 7f175ca00640 1 Processor -- start 2024-09-21T07:35:22.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.057+0000 7f175ca00640 1 -- start start 2024-09-21T07:35:22.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.058+0000 7f175ca00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17580c9930 0x7f175815edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:22.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.058+0000 7f175ca00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17580cbee0 0x7f175815e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:22.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.058+0000 7f175ca00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f17580ce950 0x7f17580b50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:22.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.058+0000 7f175ca00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f17581629b0 con 0x7f17580c9930 2024-09-21T07:35:22.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.058+0000 7f175ca00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1758162830 con 0x7f17580cbee0 2024-09-21T07:35:22.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.058+0000 7f175ca00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f1758162b30 con 0x7f17580ce950 2024-09-21T07:35:22.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.058+0000 7f1757400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17580c9930 0x7f175815edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:22.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.059+0000 7f1757400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17580c9930 0x7f175815edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39366/0 (socket says 172.21.15.39:39366) 2024-09-21T07:35:22.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.059+0000 7f1757400640 1 -- 172.21.15.39:0/3764825397 learned_addr learned my addr 172.21.15.39:0/3764825397 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:22.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.059+0000 7f1757e00640 1 --2- 172.21.15.39:0/3764825397 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f17580ce950 0x7f17580b50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:22.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.059+0000 7f174ea00640 1 --2- 172.21.15.39:0/3764825397 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17580cbee0 0x7f175815e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:22.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.059+0000 7f1757400640 1 -- 172.21.15.39:0/3764825397 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f17580ce950 msgr2=0x7f17580b50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:22.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.059+0000 7f1757400640 1 --2- 172.21.15.39:0/3764825397 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f17580ce950 0x7f17580b50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:22.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.059+0000 7f1757400640 1 -- 172.21.15.39:0/3764825397 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17580cbee0 msgr2=0x7f175815e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:22.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.059+0000 7f1757400640 1 --2- 172.21.15.39:0/3764825397 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17580cbee0 0x7f175815e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:22.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.059+0000 7f1757400640 1 -- 172.21.15.39:0/3764825397 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f175815e7d0 con 0x7f17580c9930 2024-09-21T07:35:22.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.059+0000 7f1757e00640 1 --2- 172.21.15.39:0/3764825397 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f17580ce950 0x7f17580b50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:35:22.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.059+0000 7f1757400640 1 --2- 172.21.15.39:0/3764825397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17580c9930 0x7f175815edf0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f174800e7c0 tx=0x7f174800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:22.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.060+0000 7f174fe00640 1 -- 172.21.15.39:0/3764825397 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174800ccc0 con 0x7f17580c9930 2024-09-21T07:35:22.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.060+0000 7f175ca00640 1 -- 172.21.15.39:0/3764825397 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f175815f760 con 0x7f17580c9930 2024-09-21T07:35:22.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.060+0000 7f174fe00640 1 -- 172.21.15.39:0/3764825397 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1748010430 con 0x7f17580c9930 2024-09-21T07:35:22.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.060+0000 7f174fe00640 1 -- 172.21.15.39:0/3764825397 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1748010860 con 0x7f17580c9930 2024-09-21T07:35:22.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.060+0000 7f175ca00640 1 -- 172.21.15.39:0/3764825397 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f175815fd20 con 0x7f17580c9930 2024-09-21T07:35:22.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.061+0000 7f175ca00640 1 -- 172.21.15.39:0/3764825397 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1758163070 con 0x7f17580c9930 2024-09-21T07:35:22.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.062+0000 7f174fe00640 1 -- 172.21.15.39:0/3764825397 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1748010a00 con 0x7f17580c9930 2024-09-21T07:35:22.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.063+0000 7f174fe00640 1 --2- 172.21.15.39:0/3764825397 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1734078b90 0x7f173407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:22.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.063+0000 7f174fe00640 1 -- 172.21.15.39:0/3764825397 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6062+0+0 (secure 0 0 0) 0x7f174809b980 con 0x7f17580c9930 2024-09-21T07:35:22.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.064+0000 7f174ea00640 1 --2- 172.21.15.39:0/3764825397 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1734078b90 0x7f173407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:22.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.064+0000 7f174ea00640 1 --2- 172.21.15.39:0/3764825397 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1734078b90 0x7f173407b0a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1740007fd0 tx=0x7f1740009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:22.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.067+0000 7f174fe00640 1 -- 172.21.15.39:0/3764825397 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1748063be0 con 0x7f17580c9930 2024-09-21T07:35:22.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.335+0000 7f175ca00640 1 -- 172.21.15.39:0/3764825397 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f1758074ef0 con 0x7f17580c9930 2024-09-21T07:35:22.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.339+0000 7f174fe00640 1 -- 172.21.15.39:0/3764825397 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7f17480634e0 con 0x7f17580c9930 2024-09-21T07:35:22.341 INFO:teuthology.orchestra.run.smithi039.stdout:[client.0] 2024-09-21T07:35:22.341 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQB0du5mGfb1DBAA/aSMKR5or3xy53rB/iUjpA== 2024-09-21T07:35:22.341 INFO:teuthology.orchestra.run.smithi039.stdout: caps mds = "allow" 2024-09-21T07:35:22.341 INFO:teuthology.orchestra.run.smithi039.stdout: caps mon = "allow r" 2024-09-21T07:35:22.341 INFO:teuthology.orchestra.run.smithi039.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T07:35:22.341 INFO:teuthology.orchestra.run.smithi039.stderr:updated caps for client.0 2024-09-21T07:35:22.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.342+0000 7f175ca00640 1 -- 172.21.15.39:0/3764825397 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1734078b90 msgr2=0x7f173407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:22.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.342+0000 7f175ca00640 1 --2- 172.21.15.39:0/3764825397 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1734078b90 0x7f173407b0a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1740007fd0 tx=0x7f1740009040 comp rx=0 tx=0).stop 2024-09-21T07:35:22.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.342+0000 7f175ca00640 1 -- 172.21.15.39:0/3764825397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17580c9930 msgr2=0x7f175815edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:22.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.342+0000 7f175ca00640 1 --2- 172.21.15.39:0/3764825397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17580c9930 0x7f175815edf0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f174800e7c0 tx=0x7f174800ec80 comp rx=0 tx=0).stop 2024-09-21T07:35:22.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.342+0000 7f175ca00640 1 -- 172.21.15.39:0/3764825397 shutdown_connections 2024-09-21T07:35:22.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.342+0000 7f175ca00640 1 --2- 172.21.15.39:0/3764825397 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1734078b90 0x7f173407b0a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:22.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.342+0000 7f175ca00640 1 --2- 172.21.15.39:0/3764825397 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f17580ce950 0x7f17580b50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:22.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.342+0000 7f175ca00640 1 --2- 172.21.15.39:0/3764825397 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17580cbee0 0x7f175815e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:22.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.342+0000 7f175ca00640 1 --2- 172.21.15.39:0/3764825397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17580c9930 0x7f175815edf0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:22.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.342+0000 7f175ca00640 1 -- 172.21.15.39:0/3764825397 >> 172.21.15.39:0/3764825397 conn(0x7f175806ee50 msgr2=0x7f17580cfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:22.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.342+0000 7f175ca00640 1 -- 172.21.15.39:0/3764825397 shutdown_connections 2024-09-21T07:35:22.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.342+0000 7f175ca00640 1 -- 172.21.15.39:0/3764825397 wait complete. 2024-09-21T07:35:22.365 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:35:22.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.590+0000 7f9b53200640 1 -- 172.21.15.39:0/1219788310 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b5415ca50 msgr2=0x7f9b540cc830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:22.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.590+0000 7f9b53200640 1 --2- 172.21.15.39:0/1219788310 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b5415ca50 0x7f9b540cc830 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9b44005fd0 tx=0x7f9b4402a540 comp rx=0 tx=0).stop 2024-09-21T07:35:22.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.591+0000 7f9b53200640 1 -- 172.21.15.39:0/1219788310 shutdown_connections 2024-09-21T07:35:22.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.591+0000 7f9b53200640 1 --2- 172.21.15.39:0/1219788310 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9b540cf540 0x7f9b5406ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:22.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.591+0000 7f9b53200640 1 --2- 172.21.15.39:0/1219788310 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b5415ca50 0x7f9b540cc830 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:22.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.591+0000 7f9b53200640 1 -- 172.21.15.39:0/1219788310 >> 172.21.15.39:0/1219788310 conn(0x7f9b5406d410 msgr2=0x7f9b540b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:22.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.591+0000 7f9b53200640 1 -- 172.21.15.39:0/1219788310 shutdown_connections 2024-09-21T07:35:22.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.591+0000 7f9b53200640 1 -- 172.21.15.39:0/1219788310 wait complete. 2024-09-21T07:35:22.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.593+0000 7f9b53200640 1 Processor -- start 2024-09-21T07:35:22.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.593+0000 7f9b53200640 1 -- start start 2024-09-21T07:35:22.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.595+0000 7f9b53200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9b540cf540 0x7f9b54153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:22.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.595+0000 7f9b53200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b540cf950 0x7f9b541f5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:22.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.595+0000 7f9b53200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b5415ca50 0x7f9b540b42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:22.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.595+0000 7f9b53200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5415de10 con 0x7f9b540cf950 2024-09-21T07:35:22.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.595+0000 7f9b53200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5415df90 con 0x7f9b5415ca50 2024-09-21T07:35:22.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.595+0000 7f9b53200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f9b5415e110 con 0x7f9b540cf540 2024-09-21T07:35:22.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.595+0000 7f9b4be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b540cf950 0x7f9b541f5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:22.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.595+0000 7f9b4be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b540cf950 0x7f9b541f5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39378/0 (socket says 172.21.15.39:39378) 2024-09-21T07:35:22.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.595+0000 7f9b4be00640 1 -- 172.21.15.39:0/2548713431 learned_addr learned my addr 172.21.15.39:0/2548713431 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:22.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.595+0000 7f9b51e00640 1 --2- 172.21.15.39:0/2548713431 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9b540cf540 0x7f9b54153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:22.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.595+0000 7f9b52800640 1 --2- 172.21.15.39:0/2548713431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b5415ca50 0x7f9b540b42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:22.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.595+0000 7f9b4be00640 1 -- 172.21.15.39:0/2548713431 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9b540cf540 msgr2=0x7f9b54153ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:22.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.596+0000 7f9b4be00640 1 --2- 172.21.15.39:0/2548713431 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9b540cf540 0x7f9b54153ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:22.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.596+0000 7f9b4be00640 1 -- 172.21.15.39:0/2548713431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b5415ca50 msgr2=0x7f9b540b42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:22.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.596+0000 7f9b4be00640 1 --2- 172.21.15.39:0/2548713431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b5415ca50 0x7f9b540b42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:22.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.596+0000 7f9b4be00640 1 -- 172.21.15.39:0/2548713431 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b54157760 con 0x7f9b540cf950 2024-09-21T07:35:22.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.596+0000 7f9b4be00640 1 --2- 172.21.15.39:0/2548713431 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b540cf950 0x7f9b541f5c30 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00e6f0 tx=0x7f9b3c00ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:22.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.596+0000 7f9b4a000640 1 -- 172.21.15.39:0/2548713431 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c00cab0 con 0x7f9b540cf950 2024-09-21T07:35:22.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.596+0000 7f9b4a000640 1 -- 172.21.15.39:0/2548713431 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b3c00cc50 con 0x7f9b540cf950 2024-09-21T07:35:22.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.596+0000 7f9b4a000640 1 -- 172.21.15.39:0/2548713431 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c005280 con 0x7f9b540cf950 2024-09-21T07:35:22.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.596+0000 7f9b53200640 1 -- 172.21.15.39:0/2548713431 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b54157a40 con 0x7f9b540cf950 2024-09-21T07:35:22.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.596+0000 7f9b53200640 1 -- 172.21.15.39:0/2548713431 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b54158c70 con 0x7f9b540cf950 2024-09-21T07:35:22.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.597+0000 7f9b53200640 1 -- 172.21.15.39:0/2548713431 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b5406ee70 con 0x7f9b540cf950 2024-09-21T07:35:22.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.599+0000 7f9b4a000640 1 -- 172.21.15.39:0/2548713431 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9b3c005420 con 0x7f9b540cf950 2024-09-21T07:35:22.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.600+0000 7f9b4a000640 1 --2- 172.21.15.39:0/2548713431 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9b30078b90 0x7f9b3007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:22.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.600+0000 7f9b4a000640 1 -- 172.21.15.39:0/2548713431 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6062+0+0 (secure 0 0 0) 0x7f9b3c09b7d0 con 0x7f9b540cf950 2024-09-21T07:35:22.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.601+0000 7f9b51e00640 1 --2- 172.21.15.39:0/2548713431 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9b30078b90 0x7f9b3007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:22.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.601+0000 7f9b51e00640 1 --2- 172.21.15.39:0/2548713431 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9b30078b90 0x7f9b3007b0a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9b44033000 tx=0x7f9b44004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:22.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.605+0000 7f9b4a000640 1 -- 172.21.15.39:0/2548713431 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9b3c0633d0 con 0x7f9b540cf950 2024-09-21T07:35:22.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.840+0000 7f9b53200640 1 -- 172.21.15.39:0/2548713431 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b541574f0 con 0x7f9b540cf950 2024-09-21T07:35:22.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.841+0000 7f9b4a000640 1 -- 172.21.15.39:0/2548713431 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5224 (secure 0 0 0) 0x7f9b3c067e00 con 0x7f9b540cf950 2024-09-21T07:35:22.843 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:35:22.843 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":35,"btime":"2024-09-21T07:35:19:788250+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5183,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2219254706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2219254706},{"type":"v1","addr":"172.21.15.148:6836","nonce":2219254706}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5189,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3512870782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3512870782},{"type":"v1","addr":"172.21.15.148:6837","nonce":3512870782}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5551,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/761690784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":761690784},{"type":"v1","addr":"172.21.15.39:6835","nonce":761690784}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:35:16.270342+0000","modified":"2024-09-21T07:35:19.299418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5646},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5646":{"gid":5646,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/252473586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":252473586},{"type":"v1","addr":"172.21.15.39:6834","nonce":252473586}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5646,"qdb_cluster":[5646]},"id":4}]} 2024-09-21T07:35:22.844 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 35 2024-09-21T07:35:22.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.843+0000 7f9b53200640 1 -- 172.21.15.39:0/2548713431 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9b30078b90 msgr2=0x7f9b3007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:22.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.843+0000 7f9b53200640 1 --2- 172.21.15.39:0/2548713431 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9b30078b90 0x7f9b3007b0a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9b44033000 tx=0x7f9b44004730 comp rx=0 tx=0).stop 2024-09-21T07:35:22.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.843+0000 7f9b53200640 1 -- 172.21.15.39:0/2548713431 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b540cf950 msgr2=0x7f9b541f5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:22.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.843+0000 7f9b53200640 1 --2- 172.21.15.39:0/2548713431 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b540cf950 0x7f9b541f5c30 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00e6f0 tx=0x7f9b3c00ebb0 comp rx=0 tx=0).stop 2024-09-21T07:35:22.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.843+0000 7f9b53200640 1 -- 172.21.15.39:0/2548713431 shutdown_connections 2024-09-21T07:35:22.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.844+0000 7f9b53200640 1 --2- 172.21.15.39:0/2548713431 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9b30078b90 0x7f9b3007b0a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:22.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.844+0000 7f9b53200640 1 --2- 172.21.15.39:0/2548713431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b5415ca50 0x7f9b540b42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:22.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.844+0000 7f9b53200640 1 --2- 172.21.15.39:0/2548713431 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b540cf950 0x7f9b541f5c30 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:22.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.844+0000 7f9b53200640 1 --2- 172.21.15.39:0/2548713431 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9b540cf540 0x7f9b54153ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:22.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.844+0000 7f9b53200640 1 -- 172.21.15.39:0/2548713431 >> 172.21.15.39:0/2548713431 conn(0x7f9b5406d410 msgr2=0x7f9b5415ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:22.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.844+0000 7f9b53200640 1 -- 172.21.15.39:0/2548713431 shutdown_connections 2024-09-21T07:35:22.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:22.844+0000 7f9b53200640 1 -- 172.21.15.39:0/2548713431 wait complete. 2024-09-21T07:35:23.270 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T07:35:16.270342+0000', 'modified': '2024-09-21T07:35:19.299418+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5646}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5646': {'gid': 5646, 'name': 'a', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.39:6834/252473586', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 252473586}, {'type': 'v1', 'addr': '172.21.15.39:6834', 'nonce': 252473586}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5646, 'qdb_cluster': [5646]} 2024-09-21T07:35:23.270 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T07:35:23.270 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T07:35:23.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.497+0000 7fa83e800640 1 -- 172.21.15.39:0/3807295448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8380ccb50 msgr2=0x7fa8380ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:23.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.497+0000 7fa83e800640 1 --2- 172.21.15.39:0/3807295448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8380ccb50 0x7fa8380ccf60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa830005fd0 tx=0x7fa83002aaa0 comp rx=0 tx=0).stop 2024-09-21T07:35:23.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.497+0000 7fa83e800640 1 -- 172.21.15.39:0/3807295448 shutdown_connections 2024-09-21T07:35:23.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.497+0000 7fa83e800640 1 --2- 172.21.15.39:0/3807295448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8380ccb50 0x7fa8380ccf60 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:23.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.497+0000 7fa83e800640 1 --2- 172.21.15.39:0/3807295448 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8380cc1e0 0x7fa8380af3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:23.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.497+0000 7fa83e800640 1 -- 172.21.15.39:0/3807295448 >> 172.21.15.39:0/3807295448 conn(0x7fa83806d7d0 msgr2=0x7fa8380b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:23.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.497+0000 7fa83e800640 1 -- 172.21.15.39:0/3807295448 shutdown_connections 2024-09-21T07:35:23.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.497+0000 7fa83e800640 1 -- 172.21.15.39:0/3807295448 wait complete. 2024-09-21T07:35:23.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.499+0000 7fa83e800640 1 Processor -- start 2024-09-21T07:35:23.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.500+0000 7fa83e800640 1 -- start start 2024-09-21T07:35:23.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.501+0000 7fa83e800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8380cc1e0 0x7fa838153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:23.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.501+0000 7fa83e800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8380ccb50 0x7fa838157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:23.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.501+0000 7fa83e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa83815d8a0 0x7fa838159600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:23.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.501+0000 7fa83e800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa838153520 con 0x7fa83815d8a0 2024-09-21T07:35:23.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.501+0000 7fa83e800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa8381533a0 con 0x7fa8380ccb50 2024-09-21T07:35:23.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.501+0000 7fa83e800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa8381536a0 con 0x7fa8380cc1e0 2024-09-21T07:35:23.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.501+0000 7fa837e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa83815d8a0 0x7fa838159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:23.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.501+0000 7fa837e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa83815d8a0 0x7fa838159600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39394/0 (socket says 172.21.15.39:39394) 2024-09-21T07:35:23.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.501+0000 7fa837e00640 1 -- 172.21.15.39:0/2505067014 learned_addr learned my addr 172.21.15.39:0/2505067014 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:23.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.502+0000 7fa82fe00640 1 --2- 172.21.15.39:0/2505067014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8380ccb50 0x7fa838157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:23.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.502+0000 7fa837400640 1 --2- 172.21.15.39:0/2505067014 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8380cc1e0 0x7fa838153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:23.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.502+0000 7fa837e00640 1 -- 172.21.15.39:0/2505067014 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8380cc1e0 msgr2=0x7fa838153d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:23.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.502+0000 7fa837e00640 1 --2- 172.21.15.39:0/2505067014 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8380cc1e0 0x7fa838153d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:23.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.502+0000 7fa837e00640 1 -- 172.21.15.39:0/2505067014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8380ccb50 msgr2=0x7fa838157aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:23.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.502+0000 7fa837e00640 1 --2- 172.21.15.39:0/2505067014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8380ccb50 0x7fa838157aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:23.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.502+0000 7fa837e00640 1 -- 172.21.15.39:0/2505067014 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa838158560 con 0x7fa83815d8a0 2024-09-21T07:35:23.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.502+0000 7fa82fe00640 1 --2- 172.21.15.39:0/2505067014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8380ccb50 0x7fa838157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:35:23.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.502+0000 7fa837400640 1 --2- 172.21.15.39:0/2505067014 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8380cc1e0 0x7fa838153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:35:23.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.502+0000 7fa837e00640 1 --2- 172.21.15.39:0/2505067014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa83815d8a0 0x7fa838159600 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa830006010 tx=0x7fa830003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:23.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.502+0000 7fa82f400640 1 -- 172.21.15.39:0/2505067014 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa830044070 con 0x7fa83815d8a0 2024-09-21T07:35:23.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.502+0000 7fa82f400640 1 -- 172.21.15.39:0/2505067014 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa830004040 con 0x7fa83815d8a0 2024-09-21T07:35:23.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.502+0000 7fa82f400640 1 -- 172.21.15.39:0/2505067014 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa830004430 con 0x7fa83815d8a0 2024-09-21T07:35:23.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.503+0000 7fa83e800640 1 -- 172.21.15.39:0/2505067014 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa838158840 con 0x7fa83815d8a0 2024-09-21T07:35:23.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.503+0000 7fa83e800640 1 -- 172.21.15.39:0/2505067014 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa838158e00 con 0x7fa83815d8a0 2024-09-21T07:35:23.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.505+0000 7fa82f400640 1 -- 172.21.15.39:0/2505067014 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa830047070 con 0x7fa83815d8a0 2024-09-21T07:35:23.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.506+0000 7fa82f400640 1 --2- 172.21.15.39:0/2505067014 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa814078bc0 0x7fa81407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:23.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.506+0000 7fa82f400640 1 -- 172.21.15.39:0/2505067014 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6062+0+0 (secure 0 0 0) 0x7fa8300bbda0 con 0x7fa83815d8a0 2024-09-21T07:35:23.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.506+0000 7fa8457f7740 1 Processor -- start 2024-09-21T07:35:23.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.506+0000 7fa837400640 1 --2- 172.21.15.39:0/2505067014 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa814078bc0 0x7fa81407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:23.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.506+0000 7fa8457f7740 1 -- start start 2024-09-21T07:35:23.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.507+0000 7fa837400640 1 --2- 172.21.15.39:0/2505067014 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa814078bc0 0x7fa81407b0d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa828009b80 tx=0x7fa828005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:23.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.508+0000 7fa8457f7740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae4c48c270 0x55ae4c85c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:23.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.508+0000 7fa8457f7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4c4633b0 0x55ae4c85ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:23.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.508+0000 7fa8457f7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae4c46b590 0x55ae4c82cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:23.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.508+0000 7fa8457f7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ae4c7b6430 con 0x55ae4c4633b0 2024-09-21T07:35:23.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.508+0000 7fa8457f7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ae4c7a65d0 con 0x55ae4c46b590 2024-09-21T07:35:23.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.508+0000 7fa8457f7740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55ae4c795a00 con 0x55ae4c48c270 2024-09-21T07:35:23.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.508+0000 7fa82fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4c4633b0 0x55ae4c85ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:23.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.508+0000 7fa82fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4c4633b0 0x55ae4c85ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39404/0 (socket says 172.21.15.39:39404) 2024-09-21T07:35:23.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.508+0000 7fa82fe00640 1 -- 172.21.15.39:0/1363261862 learned_addr learned my addr 172.21.15.39:0/1363261862 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:23.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.508+0000 7fa837400640 1 --2- 172.21.15.39:0/1363261862 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae4c48c270 0x55ae4c85c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:23.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.508+0000 7fa837e00640 1 --2- 172.21.15.39:0/1363261862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae4c46b590 0x55ae4c82cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:23.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.509+0000 7fa82fe00640 1 -- 172.21.15.39:0/1363261862 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae4c48c270 msgr2=0x55ae4c85c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:35:23.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.509+0000 7fa82fe00640 1 --2- 172.21.15.39:0/1363261862 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae4c48c270 0x55ae4c85c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:23.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.509+0000 7fa82fe00640 1 -- 172.21.15.39:0/1363261862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae4c46b590 msgr2=0x55ae4c82cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:35:23.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.509+0000 7fa82fe00640 1 --2- 172.21.15.39:0/1363261862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae4c46b590 0x55ae4c82cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:23.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.509+0000 7fa82fe00640 1 -- 172.21.15.39:0/1363261862 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae4c75e520 con 0x55ae4c4633b0 2024-09-21T07:35:23.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.509+0000 7fa82fe00640 1 --2- 172.21.15.39:0/1363261862 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4c4633b0 0x55ae4c85ceb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa8200015a0 tx=0x7fa820001a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72156c3ef097447a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:23.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.509+0000 7fa827e00640 1 -- 172.21.15.39:0/1363261862 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa820019070 con 0x55ae4c4633b0 2024-09-21T07:35:23.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.509+0000 7fa827e00640 1 -- 172.21.15.39:0/1363261862 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8200050d0 con 0x55ae4c4633b0 2024-09-21T07:35:23.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.509+0000 7fa827e00640 1 -- 172.21.15.39:0/1363261862 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa820007030 con 0x55ae4c4633b0 2024-09-21T07:35:23.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.510+0000 7fa8457f7740 1 -- 172.21.15.39:0/1363261862 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4c4633b0 msgr2=0x55ae4c85ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:23.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.510+0000 7fa8457f7740 1 --2- 172.21.15.39:0/1363261862 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4c4633b0 0x55ae4c85ceb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa8200015a0 tx=0x7fa820001a60 comp rx=0 tx=0).stop 2024-09-21T07:35:23.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.510+0000 7fa8457f7740 1 -- 172.21.15.39:0/1363261862 shutdown_connections 2024-09-21T07:35:23.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.510+0000 7fa8457f7740 1 --2- 172.21.15.39:0/1363261862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae4c46b590 0x55ae4c82cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:23.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.510+0000 7fa8457f7740 1 --2- 172.21.15.39:0/1363261862 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4c4633b0 0x55ae4c85ceb0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:23.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.510+0000 7fa8457f7740 1 --2- 172.21.15.39:0/1363261862 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae4c48c270 0x55ae4c85c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:23.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.510+0000 7fa8457f7740 1 -- 172.21.15.39:0/1363261862 >> 172.21.15.39:0/1363261862 conn(0x55ae4c4803c0 msgr2=0x55ae4c832970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:23.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.510+0000 7fa8457f7740 1 -- 172.21.15.39:0/1363261862 shutdown_connections 2024-09-21T07:35:23.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.510+0000 7fa8457f7740 1 -- 172.21.15.39:0/1363261862 wait complete. 2024-09-21T07:35:23.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.511+0000 7fa8457f7740 1 -- start start 2024-09-21T07:35:23.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.511+0000 7fa8457f7740 1 Processor -- start 2024-09-21T07:35:23.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.513+0000 7fa8457f7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae4c4633b0 0x55ae4c85a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:23.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.513+0000 7fa8457f7740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae4c46b590 0x55ae4c843ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:23.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.513+0000 7fa8457f7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4c48c270 0x55ae4c86efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:23.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.513+0000 7fa8457f7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ae4c73c330 con 0x55ae4c48c270 2024-09-21T07:35:23.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.513+0000 7fa8457f7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ae4c799810 con 0x55ae4c4633b0 2024-09-21T07:35:23.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.513+0000 7fa8457f7740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55ae4c506620 con 0x55ae4c46b590 2024-09-21T07:35:23.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.513+0000 7fa8457f7740 10 client.0 _ll_register_callbacks cb 0x55ae4c741420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:35:23.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.513+0000 7fa837e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4c48c270 0x55ae4c86efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:23.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.513+0000 7fa837400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae4c4633b0 0x55ae4c85a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:23.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.513+0000 7fa82fe00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae4c46b590 0x55ae4c843ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:23.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.513+0000 7fa837e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4c48c270 0x55ae4c86efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39414/0 (socket says 172.21.15.39:39414) 2024-09-21T07:35:23.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.513+0000 7fa837e00640 1 -- 172.21.15.39:0/2064806134 learned_addr learned my addr 172.21.15.39:0/2064806134 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:23.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.514+0000 7fa837e00640 1 -- 172.21.15.39:0/2064806134 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae4c46b590 msgr2=0x55ae4c843ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:35:23.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.514+0000 7fa837e00640 1 --2- 172.21.15.39:0/2064806134 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae4c46b590 0x55ae4c843ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:23.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.514+0000 7fa837e00640 1 -- 172.21.15.39:0/2064806134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae4c4633b0 msgr2=0x55ae4c85a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:35:23.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.514+0000 7fa837e00640 1 --2- 172.21.15.39:0/2064806134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae4c4633b0 0x55ae4c85a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:23.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.514+0000 7fa837e00640 1 -- 172.21.15.39:0/2064806134 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae4c4f7850 con 0x55ae4c48c270 2024-09-21T07:35:23.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.514+0000 7fa837e00640 1 --2- 172.21.15.39:0/2064806134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4c48c270 0x55ae4c86efe0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa830083670 tx=0x7fa830083f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72907ac80c999392 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:23.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.514+0000 7fa824c00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:35:23.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.514+0000 7fa824c00640 1 -- 172.21.15.39:0/2064806134 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa830082dd0 con 0x55ae4c48c270 2024-09-21T07:35:23.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.514+0000 7fa824c00640 1 -- 172.21.15.39:0/2064806134 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(62..62 src has 1..62) ==== 6062+0+0 (secure 0 0 0) 0x7fa83007f2c0 con 0x55ae4c48c270 2024-09-21T07:35:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.514+0000 7fa8457f7740 1 -- 172.21.15.39:0/2064806134 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8200050d0 con 0x55ae4c48c270 2024-09-21T07:35:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.514+0000 7fa824c00640 1 -- 172.21.15.39:0/2064806134 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa8300a4050 con 0x55ae4c48c270 2024-09-21T07:35:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.515+0000 7fa8457f7740 10 client.5766 fetch_fsmap learned FSMap version 35 2024-09-21T07:35:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.515+0000 7fa8457f7740 1 -- 172.21.15.39:0/2064806134 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55ae4c56e6b0 con 0x55ae4c48c270 2024-09-21T07:35:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.515+0000 7fa824c00640 1 -- 172.21.15.39:0/2064806134 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8200050d0 con 0x55ae4c48c270 2024-09-21T07:35:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.515+0000 7fa824c00640 1 -- 172.21.15.39:0/2064806134 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 35) ==== 3502+0+0 (secure 0 0 0) 0x7fa83007f4c0 con 0x55ae4c48c270 2024-09-21T07:35:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.515+0000 7fa8457f7740 10 client.5766 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T07:35:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.515+0000 7fa8457f7740 10 client.5766 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:35:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.515+0000 7fa8457f7740 20 client.5766 populate_metadata read hostname 'smithi039' 2024-09-21T07:35:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.515+0000 7fa8457f7740 1 --2- 172.21.15.39:0/2064806134 >> [v2:172.21.15.39:6832/252473586,v1:172.21.15.39:6834/252473586] conn(0x55ae4c44ae40 0x55ae4c86fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.515+0000 7fa8457f7740 4 client.5766 mds_command: new command op to 5646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:35:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.515+0000 7fa8457f7740 1 -- 172.21.15.39:0/2064806134 --> [v2:172.21.15.39:6832/252473586,v1:172.21.15.39:6834/252473586] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae4c7a8520 con 0x55ae4c44ae40 2024-09-21T07:35:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.515+0000 7fa837400640 1 --2- 172.21.15.39:0/2064806134 >> [v2:172.21.15.39:6832/252473586,v1:172.21.15.39:6834/252473586] conn(0x55ae4c44ae40 0x55ae4c86fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.516+0000 7fa837400640 1 --2- 172.21.15.39:0/2064806134 >> [v2:172.21.15.39:6832/252473586,v1:172.21.15.39:6834/252473586] conn(0x55ae4c44ae40 0x55ae4c86fd30 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8ba7e86b4ab273c server_cookie=ec33472d8442a012 in_seq=0 out_seq=0 2024-09-21T07:35:23.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.516+0000 7fa824c00640 10 client.5766 ms_handle_connect on v2:172.21.15.39:6832/252473586 2024-09-21T07:35:23.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.517+0000 7fa824c00640 1 -- 172.21.15.39:0/2064806134 <== mds.0 v2:172.21.15.39:6832/252473586 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ae4c7a8520 con 0x55ae4c44ae40 2024-09-21T07:35:23.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.517+0000 7fa824c00640 10 client.5766 handle_command_reply: tid=0 2024-09-21T07:35:23.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.517+0000 7fa8457f7740 1 client.5766 shutdown 2024-09-21T07:35:23.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.517+0000 7fa8457f7740 1 -- 172.21.15.39:0/2064806134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4c48c270 msgr2=0x55ae4c86efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:23.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.517+0000 7fa8457f7740 1 --2- 172.21.15.39:0/2064806134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4c48c270 0x55ae4c86efe0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa830083670 tx=0x7fa830083f90 comp rx=0 tx=0).stop 2024-09-21T07:35:23.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.517+0000 7fa8457f7740 1 -- 172.21.15.39:0/2064806134 shutdown_connections 2024-09-21T07:35:23.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.517+0000 7fa8457f7740 1 --2- 172.21.15.39:0/2064806134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4c48c270 0x55ae4c86efe0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:23.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.517+0000 7fa8457f7740 1 --2- 172.21.15.39:0/2064806134 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae4c46b590 0x55ae4c843ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:23.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.517+0000 7fa8457f7740 1 --2- 172.21.15.39:0/2064806134 >> [v2:172.21.15.39:6832/252473586,v1:172.21.15.39:6834/252473586] conn(0x55ae4c44ae40 0x55ae4c86fd30 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:23.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.517+0000 7fa8457f7740 1 --2- 172.21.15.39:0/2064806134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae4c4633b0 0x55ae4c85a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:23.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.517+0000 7fa8457f7740 1 -- 172.21.15.39:0/2064806134 >> 172.21.15.39:0/2064806134 conn(0x55ae4c4803c0 msgr2=0x55ae4c87ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:23.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.517+0000 7fa824c00640 0 client.5766 ms_handle_reset on v2:172.21.15.39:6832/252473586 2024-09-21T07:35:23.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.517+0000 7fa8457f7740 1 -- 172.21.15.39:0/2064806134 shutdown_connections 2024-09-21T07:35:23.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.518+0000 7fa8457f7740 1 -- 172.21.15.39:0/2064806134 wait complete. 2024-09-21T07:35:23.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.518+0000 7fa8457f7740 20 client.5766 trim_cache size 0 max 16384 2024-09-21T07:35:23.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.537+0000 7fa8457f7740 1 Processor -- start 2024-09-21T07:35:23.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.537+0000 7fa8457f7740 1 -- start start 2024-09-21T07:35:23.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.538+0000 7fa8457f7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae4c4633b0 0x55ae4c82e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:23.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.538+0000 7fa8457f7740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae4c46b590 0x55ae4c86e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:23.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.538+0000 7fa8457f7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4c48c270 0x55ae4c86ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:23.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.538+0000 7fa8457f7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa838201f20 con 0x55ae4c48c270 2024-09-21T07:35:23.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.538+0000 7fa8457f7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa8380739a0 con 0x55ae4c4633b0 2024-09-21T07:35:23.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.538+0000 7fa8457f7740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa8382018d0 con 0x55ae4c46b590 2024-09-21T07:35:23.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.539+0000 7fa837e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4c48c270 0x55ae4c86ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:23.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.539+0000 7fa837e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4c48c270 0x55ae4c86ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39420/0 (socket says 172.21.15.39:39420) 2024-09-21T07:35:23.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.539+0000 7fa837e00640 1 -- 172.21.15.39:0/3474025141 learned_addr learned my addr 172.21.15.39:0/3474025141 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:23.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.539+0000 7fa837400640 1 --2- 172.21.15.39:0/3474025141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae4c4633b0 0x55ae4c82e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:23.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.539+0000 7fa82fe00640 1 --2- 172.21.15.39:0/3474025141 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae4c46b590 0x55ae4c86e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:23.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.539+0000 7fa837e00640 1 -- 172.21.15.39:0/3474025141 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae4c46b590 msgr2=0x55ae4c86e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:35:23.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.539+0000 7fa837e00640 1 --2- 172.21.15.39:0/3474025141 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae4c46b590 0x55ae4c86e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:23.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.539+0000 7fa837e00640 1 -- 172.21.15.39:0/3474025141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae4c4633b0 msgr2=0x55ae4c82e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:35:23.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.539+0000 7fa837e00640 1 --2- 172.21.15.39:0/3474025141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae4c4633b0 0x55ae4c82e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:23.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.539+0000 7fa837e00640 1 -- 172.21.15.39:0/3474025141 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae4c844240 con 0x55ae4c48c270 2024-09-21T07:35:23.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.539+0000 7fa837e00640 1 --2- 172.21.15.39:0/3474025141 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4c48c270 0x55ae4c86ef20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa830083900 tx=0x7fa8300099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adaf3802de527937 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:23.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.540+0000 7fa827400640 1 -- 172.21.15.39:0/3474025141 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa830082dd0 con 0x55ae4c48c270 2024-09-21T07:35:23.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.540+0000 7fa827400640 1 -- 172.21.15.39:0/3474025141 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa83008d300 con 0x55ae4c48c270 2024-09-21T07:35:23.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.540+0000 7fa827400640 1 -- 172.21.15.39:0/3474025141 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83007f520 con 0x55ae4c48c270 2024-09-21T07:35:23.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.540+0000 7fa8457f7740 1 -- 172.21.15.39:0/3474025141 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4c48c270 msgr2=0x55ae4c86ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:23.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.540+0000 7fa8457f7740 1 --2- 172.21.15.39:0/3474025141 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4c48c270 0x55ae4c86ef20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa830083900 tx=0x7fa8300099f0 comp rx=0 tx=0).stop 2024-09-21T07:35:23.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.540+0000 7fa8457f7740 1 -- 172.21.15.39:0/3474025141 shutdown_connections 2024-09-21T07:35:23.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.540+0000 7fa8457f7740 1 --2- 172.21.15.39:0/3474025141 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4c48c270 0x55ae4c86ef20 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:23.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.540+0000 7fa8457f7740 1 --2- 172.21.15.39:0/3474025141 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae4c46b590 0x55ae4c86e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:23.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.540+0000 7fa8457f7740 1 --2- 172.21.15.39:0/3474025141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae4c4633b0 0x55ae4c82e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:23.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.540+0000 7fa8457f7740 1 -- 172.21.15.39:0/3474025141 >> 172.21.15.39:0/3474025141 conn(0x55ae4c4803c0 msgr2=0x55ae4c87a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:23.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.541+0000 7fa8457f7740 1 -- 172.21.15.39:0/3474025141 shutdown_connections 2024-09-21T07:35:23.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.541+0000 7fa8457f7740 1 -- 172.21.15.39:0/3474025141 wait complete. 2024-09-21T07:35:23.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.541+0000 7fa8457f7740 1 -- start start 2024-09-21T07:35:23.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.542+0000 7fa8457f7740 1 Processor -- start 2024-09-21T07:35:23.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.543+0000 7fa8457f7740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae4c4633b0 0x55ae4c8f7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:23.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.543+0000 7fa8457f7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4c46b590 0x55ae4c86e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:23.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.543+0000 7fa8457f7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae4c48c270 0x55ae4c86ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:23.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.543+0000 7fa8457f7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ae4c8575d0 con 0x55ae4c46b590 2024-09-21T07:35:23.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.543+0000 7fa8457f7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ae4c86fad0 con 0x55ae4c48c270 2024-09-21T07:35:23.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.543+0000 7fa8457f7740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55ae4c857bf0 con 0x55ae4c4633b0 2024-09-21T07:35:23.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.543+0000 7fa8457f7740 10 client.0 _ll_register_callbacks cb 0x7fa830083740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:35:23.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.543+0000 7fa82fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4c46b590 0x55ae4c86e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:23.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.544+0000 7fa837e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae4c48c270 0x55ae4c86ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:23.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.544+0000 7fa837400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae4c4633b0 0x55ae4c8f7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:23.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.544+0000 7fa82fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4c46b590 0x55ae4c86e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39434/0 (socket says 172.21.15.39:39434) 2024-09-21T07:35:23.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.544+0000 7fa82fe00640 1 -- 172.21.15.39:0/3676781434 learned_addr learned my addr 172.21.15.39:0/3676781434 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:23.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.544+0000 7fa82fe00640 1 -- 172.21.15.39:0/3676781434 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae4c4633b0 msgr2=0x55ae4c8f7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:35:23.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.544+0000 7fa82fe00640 1 --2- 172.21.15.39:0/3676781434 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae4c4633b0 0x55ae4c8f7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:23.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.544+0000 7fa82fe00640 1 -- 172.21.15.39:0/3676781434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae4c48c270 msgr2=0x55ae4c86ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:35:23.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.544+0000 7fa82fe00640 1 --2- 172.21.15.39:0/3676781434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae4c48c270 0x55ae4c86ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:23.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.544+0000 7fa82fe00640 1 -- 172.21.15.39:0/3676781434 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae4c8338c0 con 0x55ae4c46b590 2024-09-21T07:35:23.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.544+0000 7fa82fe00640 1 --2- 172.21.15.39:0/3676781434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4c46b590 0x55ae4c86e9e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa820001570 tx=0x7fa820005300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b26ff294dd96298 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:23.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.544+0000 7fa824c00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:35:23.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.545+0000 7fa824c00640 1 -- 172.21.15.39:0/3676781434 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa820025060 con 0x55ae4c46b590 2024-09-21T07:35:23.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.545+0000 7fa824c00640 1 -- 172.21.15.39:0/3676781434 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(62..62 src has 1..62) ==== 6062+0+0 (secure 0 0 0) 0x7fa820007440 con 0x55ae4c46b590 2024-09-21T07:35:23.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.545+0000 7fa8457f7740 1 -- 172.21.15.39:0/3676781434 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa83008d300 con 0x55ae4c46b590 2024-09-21T07:35:23.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.545+0000 7fa824c00640 1 -- 172.21.15.39:0/3676781434 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa82002b050 con 0x55ae4c46b590 2024-09-21T07:35:23.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.545+0000 7fa8457f7740 10 client.5772 fetch_fsmap learned FSMap version 35 2024-09-21T07:35:23.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.545+0000 7fa8457f7740 1 -- 172.21.15.39:0/3676781434 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55ae4c8f7fd0 con 0x55ae4c46b590 2024-09-21T07:35:23.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.545+0000 7fa824c00640 1 -- 172.21.15.39:0/3676781434 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa83008d300 con 0x55ae4c46b590 2024-09-21T07:35:23.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.545+0000 7fa824c00640 1 -- 172.21.15.39:0/3676781434 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 35) ==== 3502+0+0 (secure 0 0 0) 0x7fa82002e020 con 0x55ae4c46b590 2024-09-21T07:35:23.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.545+0000 7fa8457f7740 10 client.5772 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T07:35:23.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.545+0000 7fa8457f7740 10 client.5772 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:35:23.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.545+0000 7fa8457f7740 20 client.5772 populate_metadata read hostname 'smithi039' 2024-09-21T07:35:23.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.545+0000 7fa8457f7740 1 --2- 172.21.15.39:0/3676781434 >> [v2:172.21.15.39:6832/252473586,v1:172.21.15.39:6834/252473586] conn(0x55ae4c888940 0x55ae4c888d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:23.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.546+0000 7fa8457f7740 4 client.5772 mds_command: new command op to 5646 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:35:23.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.546+0000 7fa8457f7740 1 -- 172.21.15.39:0/3676781434 --> [v2:172.21.15.39:6832/252473586,v1:172.21.15.39:6834/252473586] -- command(tid 0: {"prefix": "status"}) -- 0x55ae4c7a3fd0 con 0x55ae4c888940 2024-09-21T07:35:23.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.546+0000 7fa837400640 1 --2- 172.21.15.39:0/3676781434 >> [v2:172.21.15.39:6832/252473586,v1:172.21.15.39:6834/252473586] conn(0x55ae4c888940 0x55ae4c888d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:23.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.546+0000 7fa837400640 1 --2- 172.21.15.39:0/3676781434 >> [v2:172.21.15.39:6832/252473586,v1:172.21.15.39:6834/252473586] conn(0x55ae4c888940 0x55ae4c888d50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90b637bc1333b053 server_cookie=633c03e4e558bb95 in_seq=0 out_seq=0 2024-09-21T07:35:23.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.546+0000 7fa824c00640 10 client.5772 ms_handle_connect on v2:172.21.15.39:6832/252473586 2024-09-21T07:35:23.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.547+0000 7fa824c00640 1 -- 172.21.15.39:0/3676781434 <== mds.0 v2:172.21.15.39:6832/252473586 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55ae4c7a3fd0 con 0x55ae4c888940 2024-09-21T07:35:23.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.547+0000 7fa824c00640 10 client.5772 handle_command_reply: tid=0 2024-09-21T07:35:23.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.547+0000 7fa8457f7740 1 client.5772 shutdown 2024-09-21T07:35:23.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.547+0000 7fa8457f7740 1 -- 172.21.15.39:0/3676781434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4c46b590 msgr2=0x55ae4c86e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:23.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.547+0000 7fa8457f7740 1 --2- 172.21.15.39:0/3676781434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4c46b590 0x55ae4c86e9e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa820001570 tx=0x7fa820005300 comp rx=0 tx=0).stop 2024-09-21T07:35:23.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.547+0000 7fa8457f7740 1 -- 172.21.15.39:0/3676781434 shutdown_connections 2024-09-21T07:35:23.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.547+0000 7fa8457f7740 1 --2- 172.21.15.39:0/3676781434 >> [v2:172.21.15.39:6832/252473586,v1:172.21.15.39:6834/252473586] conn(0x55ae4c888940 0x55ae4c888d50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:23.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.547+0000 7fa8457f7740 1 --2- 172.21.15.39:0/3676781434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae4c48c270 0x55ae4c86ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:23.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.547+0000 7fa8457f7740 1 --2- 172.21.15.39:0/3676781434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4c46b590 0x55ae4c86e9e0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:23.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.547+0000 7fa824c00640 0 client.5772 ms_handle_reset on v2:172.21.15.39:6832/252473586 2024-09-21T07:35:23.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.547+0000 7fa8457f7740 1 --2- 172.21.15.39:0/3676781434 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae4c4633b0 0x55ae4c8f7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:23.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.547+0000 7fa8457f7740 1 -- 172.21.15.39:0/3676781434 >> 172.21.15.39:0/3676781434 conn(0x55ae4c4803c0 msgr2=0x55ae4c87a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:23.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.547+0000 7fa8457f7740 1 -- 172.21.15.39:0/3676781434 shutdown_connections 2024-09-21T07:35:23.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.547+0000 7fa8457f7740 1 -- 172.21.15.39:0/3676781434 wait complete. 2024-09-21T07:35:23.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.547+0000 7fa8457f7740 20 client.5772 trim_cache size 0 max 16384 2024-09-21T07:35:23.552 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:35:23.552 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:35:23.552 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-09-21T07:35:23.552 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 5646, 2024-09-21T07:35:23.552 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:35:23.552 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:35:23.552 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:35:23.552 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 6.7976520799999998, 2024-09-21T07:35:23.552 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 35, 2024-09-21T07:35:23.552 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 62, 2024-09-21T07:35:23.552 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 62, 2024-09-21T07:35:23.552 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 11.800877127 2024-09-21T07:35:23.552 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:35:23.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.548+0000 7fa83e800640 1 -- 172.21.15.39:0/2505067014 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa814078bc0 msgr2=0x7fa81407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:23.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.548+0000 7fa83e800640 1 --2- 172.21.15.39:0/2505067014 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa814078bc0 0x7fa81407b0d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa828009b80 tx=0x7fa828005f90 comp rx=0 tx=0).stop 2024-09-21T07:35:23.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.548+0000 7fa83e800640 1 -- 172.21.15.39:0/2505067014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa83815d8a0 msgr2=0x7fa838159600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:23.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.548+0000 7fa83e800640 1 --2- 172.21.15.39:0/2505067014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa83815d8a0 0x7fa838159600 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa830006010 tx=0x7fa830003ce0 comp rx=0 tx=0).stop 2024-09-21T07:35:23.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.548+0000 7fa83e800640 1 -- 172.21.15.39:0/2505067014 shutdown_connections 2024-09-21T07:35:23.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.548+0000 7fa83e800640 1 --2- 172.21.15.39:0/2505067014 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa814078bc0 0x7fa81407b0d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:23.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.548+0000 7fa83e800640 1 --2- 172.21.15.39:0/2505067014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa83815d8a0 0x7fa838159600 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:23.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.548+0000 7fa83e800640 1 --2- 172.21.15.39:0/2505067014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8380ccb50 0x7fa838157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:23.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.548+0000 7fa83e800640 1 --2- 172.21.15.39:0/2505067014 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8380cc1e0 0x7fa838153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:23.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.548+0000 7fa83e800640 1 -- 172.21.15.39:0/2505067014 >> 172.21.15.39:0/2505067014 conn(0x7fa83806d7d0 msgr2=0x7fa8380d0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:23.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.548+0000 7fa83e800640 1 -- 172.21.15.39:0/2505067014 shutdown_connections 2024-09-21T07:35:23.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.548+0000 7fa83e800640 1 -- 172.21.15.39:0/2505067014 wait complete. 2024-09-21T07:35:23.572 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:35:23.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.797+0000 7f36b7000640 1 -- 172.21.15.39:0/995781051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36b0070030 msgr2=0x7f36b0074640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:23.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.797+0000 7f36b7000640 1 --2- 172.21.15.39:0/995781051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36b0070030 0x7f36b0074640 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f36b0070e70 tx=0x7f369802a560 comp rx=0 tx=0).stop 2024-09-21T07:35:23.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.797+0000 7f36b7000640 1 -- 172.21.15.39:0/995781051 shutdown_connections 2024-09-21T07:35:23.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.797+0000 7f36b7000640 1 --2- 172.21.15.39:0/995781051 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36b0070440 0x7f36b0074b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:23.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.797+0000 7f36b7000640 1 --2- 172.21.15.39:0/995781051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36b0070030 0x7f36b0074640 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:23.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.797+0000 7f36b7000640 1 --2- 172.21.15.39:0/995781051 >> v2:172.21.15.148:3300/0 conn(0x7f36b0073170 0x7f36b015c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:23.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.797+0000 7f36b7000640 1 -- 172.21.15.39:0/995781051 >> 172.21.15.39:0/995781051 conn(0x7f36b00b1cb0 msgr2=0x7f36b00b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:23.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.797+0000 7f36b7000640 1 -- 172.21.15.39:0/995781051 shutdown_connections 2024-09-21T07:35:23.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.797+0000 7f36b7000640 1 -- 172.21.15.39:0/995781051 wait complete. 2024-09-21T07:35:23.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.800+0000 7f36b7000640 1 Processor -- start 2024-09-21T07:35:23.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.800+0000 7f36b7000640 1 -- start start 2024-09-21T07:35:23.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.801+0000 7f36b7000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36b0070030 0x7f36b0145e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:23.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.801+0000 7f36b7000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36b0070440 0x7f36b0149020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:23.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.801+0000 7f36b7000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36b0073170 0x7f36b015e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:23.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.801+0000 7f36b7000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f36b006f650 con 0x7f36b0070030 2024-09-21T07:35:23.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.801+0000 7f36b7000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f36b006f4d0 con 0x7f36b0073170 2024-09-21T07:35:23.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.801+0000 7f36b7000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f36b006f7d0 con 0x7f36b0070440 2024-09-21T07:35:23.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.801+0000 7f36afe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36b0070030 0x7f36b0145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:23.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.801+0000 7f36afe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36b0070030 0x7f36b0145e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39444/0 (socket says 172.21.15.39:39444) 2024-09-21T07:35:23.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.801+0000 7f36afe00640 1 -- 172.21.15.39:0/3591201242 learned_addr learned my addr 172.21.15.39:0/3591201242 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:23.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.802+0000 7f36a7e00640 1 --2- 172.21.15.39:0/3591201242 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36b0070440 0x7f36b0149020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:23.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.802+0000 7f36b5000640 1 --2- 172.21.15.39:0/3591201242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36b0073170 0x7f36b015e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:23.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.802+0000 7f36afe00640 1 -- 172.21.15.39:0/3591201242 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36b0070440 msgr2=0x7f36b0149020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:23.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.802+0000 7f36afe00640 1 --2- 172.21.15.39:0/3591201242 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36b0070440 0x7f36b0149020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:23.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.802+0000 7f36afe00640 1 -- 172.21.15.39:0/3591201242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36b0073170 msgr2=0x7f36b015e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:23.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.802+0000 7f36afe00640 1 --2- 172.21.15.39:0/3591201242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36b0073170 0x7f36b015e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:23.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.802+0000 7f36afe00640 1 -- 172.21.15.39:0/3591201242 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b0149560 con 0x7f36b0070030 2024-09-21T07:35:23.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.802+0000 7f36a7e00640 1 --2- 172.21.15.39:0/3591201242 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36b0070440 0x7f36b0149020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:35:23.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.802+0000 7f36afe00640 1 --2- 172.21.15.39:0/3591201242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36b0070030 0x7f36b0145e10 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f36a000b410 tx=0x7f36a000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:23.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.802+0000 7f36acc00640 1 -- 172.21.15.39:0/3591201242 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a0015020 con 0x7f36b0070030 2024-09-21T07:35:23.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.803+0000 7f36acc00640 1 -- 172.21.15.39:0/3591201242 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36a0011940 con 0x7f36b0070030 2024-09-21T07:35:23.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.803+0000 7f36acc00640 1 -- 172.21.15.39:0/3591201242 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a0011d90 con 0x7f36b0070030 2024-09-21T07:35:23.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.803+0000 7f36b7000640 1 -- 172.21.15.39:0/3591201242 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36b00d0510 con 0x7f36b0070030 2024-09-21T07:35:23.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.803+0000 7f36b7000640 1 -- 172.21.15.39:0/3591201242 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36b00d0ad0 con 0x7f36b0070030 2024-09-21T07:35:23.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.804+0000 7f36b7000640 1 -- 172.21.15.39:0/3591201242 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36b015c780 con 0x7f36b0070030 2024-09-21T07:35:23.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.805+0000 7f36acc00640 1 -- 172.21.15.39:0/3591201242 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f36a0018020 con 0x7f36b0070030 2024-09-21T07:35:23.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.806+0000 7f36acc00640 1 --2- 172.21.15.39:0/3591201242 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f368c078b90 0x7f368c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:23.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.806+0000 7f36acc00640 1 -- 172.21.15.39:0/3591201242 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6062+0+0 (secure 0 0 0) 0x7f36a009ae40 con 0x7f36b0070030 2024-09-21T07:35:23.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.807+0000 7f36a7e00640 1 --2- 172.21.15.39:0/3591201242 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f368c078b90 0x7f368c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.807+0000 7f36a7e00640 1 --2- 172.21.15.39:0/3591201242 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f368c078b90 0x7f368c07b0a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3698033000 tx=0x7f3698004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:23.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:23.811+0000 7f36acc00640 1 -- 172.21.15.39:0/3591201242 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f36a001f020 con 0x7f36b0070030 2024-09-21T07:35:24.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:24.045+0000 7f36b7000640 1 -- 172.21.15.39:0/3591201242 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f36b00ad780 con 0x7f36b0070030 2024-09-21T07:35:24.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:24.046+0000 7f36acc00640 1 -- 172.21.15.39:0/3591201242 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) ==== 53+0+73 (secure 0 0 0) 0x7f36a0063110 con 0x7f36b0070030 2024-09-21T07:35:24.048 INFO:teuthology.orchestra.run.smithi039.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:35:24.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:24.048+0000 7f36b7000640 1 -- 172.21.15.39:0/3591201242 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f368c078b90 msgr2=0x7f368c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:24.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:24.048+0000 7f36b7000640 1 --2- 172.21.15.39:0/3591201242 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f368c078b90 0x7f368c07b0a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3698033000 tx=0x7f3698004730 comp rx=0 tx=0).stop 2024-09-21T07:35:24.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:24.048+0000 7f36b7000640 1 -- 172.21.15.39:0/3591201242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36b0070030 msgr2=0x7f36b0145e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:24.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:24.048+0000 7f36b7000640 1 --2- 172.21.15.39:0/3591201242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36b0070030 0x7f36b0145e10 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f36a000b410 tx=0x7f36a000b8d0 comp rx=0 tx=0).stop 2024-09-21T07:35:24.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:24.048+0000 7f36b7000640 1 -- 172.21.15.39:0/3591201242 shutdown_connections 2024-09-21T07:35:24.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:24.048+0000 7f36b7000640 1 --2- 172.21.15.39:0/3591201242 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f368c078b90 0x7f368c07b0a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:24.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:24.048+0000 7f36b7000640 1 --2- 172.21.15.39:0/3591201242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36b0073170 0x7f36b015e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:24.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:24.048+0000 7f36b7000640 1 --2- 172.21.15.39:0/3591201242 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36b0070440 0x7f36b0149020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:24.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:24.048+0000 7f36b7000640 1 --2- 172.21.15.39:0/3591201242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36b0070030 0x7f36b0145e10 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:24.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:24.048+0000 7f36b7000640 1 -- 172.21.15.39:0/3591201242 >> 172.21.15.39:0/3591201242 conn(0x7f36b00b1cb0 msgr2=0x7f36b00ae5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:24.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:24.049+0000 7f36b7000640 1 -- 172.21.15.39:0/3591201242 shutdown_connections 2024-09-21T07:35:24.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:24.049+0000 7f36b7000640 1 -- 172.21.15.39:0/3591201242 wait complete. 2024-09-21T07:35:24.074 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:35:24.074 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:35:24.074 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi039.front.sepia.ceph.com 2024-09-21T07:35:24.074 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T07:35:24.074 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:35:24.074 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:35:24.074 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:35:24.074 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:35:24.074 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:35:24.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:35:24.075 DEBUG:teuthology.orchestra.run.smithi039:> ip addr 2024-09-21T07:35:24.110 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:35:24.110 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:35:24.110 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:35:24.110 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:35:24.110 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2024-09-21T07:35:24.111 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:35:24.111 INFO:teuthology.orchestra.run.smithi039.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:35:24.111 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:35:24.111 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f0 2024-09-21T07:35:24.111 INFO:teuthology.orchestra.run.smithi039.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:35:24.111 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c7 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:35:24.111 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f1 2024-09-21T07:35:24.111 INFO:teuthology.orchestra.run.smithi039.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:35:24.111 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:70 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:35:24.111 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f0 2024-09-21T07:35:24.111 INFO:teuthology.orchestra.run.smithi039.stdout: inet 172.21.15.39/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T07:35:24.111 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft 42595sec preferred_lft 42595sec 2024-09-21T07:35:24.112 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec4:7aff:febd:1370/64 scope link 2024-09-21T07:35:24.112 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:35:24.112 INFO:teuthology.orchestra.run.smithi039.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:35:24.112 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:71 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:35:24.112 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f1 2024-09-21T07:35:24.112 INFO:teuthology.orchestra.run.smithi039.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:35:24.112 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff 2024-09-21T07:35:24.112 INFO:teuthology.orchestra.run.smithi039.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:35:24.112 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:35:24.112 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::b079:ddff:fe0e:6333/64 scope link 2024-09-21T07:35:24.112 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:35:24.112 INFO:teuthology.orchestra.run.smithi039.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:35:24.113 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:35:24.113 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::2ca7:98ff:fe8a:d0ed/64 scope link 2024-09-21T07:35:24.113 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:35:24.113 INFO:teuthology.orchestra.run.smithi039.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:35:24.113 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether ee:10:53:01:0c:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:35:24.113 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec10:53ff:fe01:c44/64 scope link 2024-09-21T07:35:24.113 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:35:24.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:35:24.113 DEBUG:teuthology.orchestra.run.smithi039:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:35:24.204 INFO:teuthology.orchestra.run.smithi039.stdout:1 2024-09-21T07:35:24.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:35:24.207 DEBUG:teuthology.orchestra.run.smithi039:> route 2024-09-21T07:35:24.243 INFO:teuthology.orchestra.run.smithi039.stdout:Kernel IP routing table 2024-09-21T07:35:24.243 INFO:teuthology.orchestra.run.smithi039.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:35:24.243 INFO:teuthology.orchestra.run.smithi039.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T07:35:24.243 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T07:35:24.243 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:35:24.244 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:35:24.244 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-21T07:35:24.244 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T07:35:24.244 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T07:35:24.244 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T07:35:24.244 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-21T07:35:24.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:35:24.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:35:24.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:35:24.441 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-09-21T07:35:24.476 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:35:24.476 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T07:35:24.476 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:35:24.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:35:24.477 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-09-21T07:35:24.572 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:35:24.573 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T07:35:24.607 INFO:teuthology.orchestra.run.smithi039.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:35:24.607 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:35:24.608 DEBUG:teuthology.orchestra.run.smithi039:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:35:24.683 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-21T07:35:24.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:35:24.683 DEBUG:teuthology.orchestra.run.smithi039:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:35:24.882 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:35:24.882 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:35:24.919 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:35:24.919 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:35:24.919 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: resolved to: "172.21.15.39:3300,172.21.15.148:3300,172.21.15.148:3301" 2024-09-21T07:35:24.919 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: trying mount with new device syntax: 0@7ab9b04a-f88d-49aa-a493-e5c076637ffb.cephfs=/ 2024-09-21T07:35:24.919 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.39:3300/172.21.15.148:3300/172.21.15.148:3301" will pass to kernel 2024-09-21T07:35:24.921 INFO:teuthology.orchestra.run.smithi039.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T07:35:24.921 INFO:teuthology.orchestra.run.smithi039.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:35:24.921 INFO:teuthology.orchestra.run.smithi039.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:35:24.921 INFO:teuthology.orchestra.run.smithi039.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:35:24.921 INFO:teuthology.orchestra.run.smithi039.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:35:24.924 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:35:24.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:35:24.924 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:35:24.980 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:35:25.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:35:25.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:35:25.081 INFO:teuthology.orchestra.run.smithi039.stdout:client5674 2024-09-21T07:35:25.081 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:35:25.081 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:35:25.117 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create test_data_pool 2024-09-21T07:35:25.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.383+0000 7f1f68e00640 1 -- 172.21.15.39:0/2224424403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f64073e50 msgr2=0x7f1f64074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:25.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.383+0000 7f1f68e00640 1 --2- 172.21.15.39:0/2224424403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f64073e50 0x7f1f64074260 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1f6406e840 tx=0x7f1f5c02aa30 comp rx=0 tx=0).stop 2024-09-21T07:35:25.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.383+0000 7f1f68e00640 1 -- 172.21.15.39:0/2224424403 shutdown_connections 2024-09-21T07:35:25.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.383+0000 7f1f68e00640 1 --2- 172.21.15.39:0/2224424403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f64073e50 0x7f1f64074260 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:25.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.383+0000 7f1f68e00640 1 --2- 172.21.15.39:0/2224424403 >> v2:172.21.15.148:3300/0 conn(0x7f1f640cfc00 0x7f1f640ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:25.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.383+0000 7f1f68e00640 1 -- 172.21.15.39:0/2224424403 >> 172.21.15.39:0/2224424403 conn(0x7f1f640d15f0 msgr2=0x7f1f64073740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:25.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.384+0000 7f1f68e00640 1 -- 172.21.15.39:0/2224424403 shutdown_connections 2024-09-21T07:35:25.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.384+0000 7f1f68e00640 1 -- 172.21.15.39:0/2224424403 wait complete. 2024-09-21T07:35:25.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.386+0000 7f1f68e00640 1 Processor -- start 2024-09-21T07:35:25.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.386+0000 7f1f68e00640 1 -- start start 2024-09-21T07:35:25.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.387+0000 7f1f68e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f64073e50 0x7f1f640b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:25.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.387+0000 7f1f68e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f640cfc00 0x7f1f640b4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:25.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.387+0000 7f1f68e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f64153970 0x7f1f641f3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:25.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.387+0000 7f1f68e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6415e2e0 con 0x7f1f64073e50 2024-09-21T07:35:25.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.387+0000 7f1f68e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6415e160 con 0x7f1f640cfc00 2024-09-21T07:35:25.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.387+0000 7f1f68e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f1f6415e460 con 0x7f1f64153970 2024-09-21T07:35:25.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.388+0000 7f1f61e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f64073e50 0x7f1f640b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:25.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.388+0000 7f1f61e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f64073e50 0x7f1f640b3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39470/0 (socket says 172.21.15.39:39470) 2024-09-21T07:35:25.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.388+0000 7f1f61e00640 1 -- 172.21.15.39:0/319620579 learned_addr learned my addr 172.21.15.39:0/319620579 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:25.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.388+0000 7f1f61e00640 1 -- 172.21.15.39:0/319620579 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f64153970 msgr2=0x7f1f641f3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:25.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.388+0000 7f1f61e00640 1 --2- 172.21.15.39:0/319620579 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f64153970 0x7f1f641f3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:25.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.388+0000 7f1f61e00640 1 -- 172.21.15.39:0/319620579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f640cfc00 msgr2=0x7f1f640b4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:25.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.388+0000 7f1f61400640 1 --2- 172.21.15.39:0/319620579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f640cfc00 0x7f1f640b4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:25.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.388+0000 7f1f61e00640 1 --2- 172.21.15.39:0/319620579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f640cfc00 0x7f1f640b4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:25.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.388+0000 7f1f61e00640 1 -- 172.21.15.39:0/319620579 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f640b5070 con 0x7f1f64073e50 2024-09-21T07:35:25.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.388+0000 7f1f61400640 1 --2- 172.21.15.39:0/319620579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f640cfc00 0x7f1f640b4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:35:25.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.388+0000 7f1f61e00640 1 --2- 172.21.15.39:0/319620579 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f64073e50 0x7f1f640b3a60 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1f5400c3d0 tx=0x7f1f5400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:25.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.389+0000 7f1f5a000640 1 -- 172.21.15.39:0/319620579 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f54017070 con 0x7f1f64073e50 2024-09-21T07:35:25.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.389+0000 7f1f5a000640 1 -- 172.21.15.39:0/319620579 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f540053c0 con 0x7f1f64073e50 2024-09-21T07:35:25.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.389+0000 7f1f5a000640 1 -- 172.21.15.39:0/319620579 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f54003850 con 0x7f1f64073e50 2024-09-21T07:35:25.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.389+0000 7f1f68e00640 1 -- 172.21.15.39:0/319620579 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f641f3df0 con 0x7f1f64073e50 2024-09-21T07:35:25.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.389+0000 7f1f68e00640 1 -- 172.21.15.39:0/319620579 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f6414cf70 con 0x7f1f64073e50 2024-09-21T07:35:25.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.390+0000 7f1f68e00640 1 -- 172.21.15.39:0/319620579 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f6415e9e0 con 0x7f1f64073e50 2024-09-21T07:35:25.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.391+0000 7f1f5a000640 1 -- 172.21.15.39:0/319620579 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1f5401a070 con 0x7f1f64073e50 2024-09-21T07:35:25.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.392+0000 7f1f5a000640 1 --2- 172.21.15.39:0/319620579 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1f44078bc0 0x7f1f4407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:25.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.392+0000 7f1f5a000640 1 -- 172.21.15.39:0/319620579 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6062+0+0 (secure 0 0 0) 0x7f1f5409a220 con 0x7f1f64073e50 2024-09-21T07:35:25.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.393+0000 7f1f61400640 1 --2- 172.21.15.39:0/319620579 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1f44078bc0 0x7f1f4407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:25.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.394+0000 7f1f61400640 1 --2- 172.21.15.39:0/319620579 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1f44078bc0 0x7f1f4407b0d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1f6415bac0 tx=0x7f1f50008040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:25.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.397+0000 7f1f5a000640 1 -- 172.21.15.39:0/319620579 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1f6415cd90 con 0x7f1f64073e50 2024-09-21T07:35:25.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.591+0000 7f1f68e00640 1 -- 172.21.15.39:0/319620579 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "test_data_pool"} v 0) -- 0x7f1f64160ee0 con 0x7f1f64073e50 2024-09-21T07:35:25.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.843+0000 7f1f5a000640 1 -- 172.21.15.39:0/319620579 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "test_data_pool"}]=0 pool 'test_data_pool' created v63) ==== 118+0+0 (secure 0 0 0) 0x7f1f540624f0 con 0x7f1f64073e50 2024-09-21T07:35:25.846 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'test_data_pool' created 2024-09-21T07:35:25.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.846+0000 7f1f68e00640 1 -- 172.21.15.39:0/319620579 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1f44078bc0 msgr2=0x7f1f4407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:25.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.846+0000 7f1f68e00640 1 --2- 172.21.15.39:0/319620579 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1f44078bc0 0x7f1f4407b0d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1f6415bac0 tx=0x7f1f50008040 comp rx=0 tx=0).stop 2024-09-21T07:35:25.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.846+0000 7f1f68e00640 1 -- 172.21.15.39:0/319620579 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f64073e50 msgr2=0x7f1f640b3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:25.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.846+0000 7f1f68e00640 1 --2- 172.21.15.39:0/319620579 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f64073e50 0x7f1f640b3a60 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1f5400c3d0 tx=0x7f1f5400c890 comp rx=0 tx=0).stop 2024-09-21T07:35:25.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.847+0000 7f1f68e00640 1 -- 172.21.15.39:0/319620579 shutdown_connections 2024-09-21T07:35:25.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.847+0000 7f1f68e00640 1 --2- 172.21.15.39:0/319620579 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1f44078bc0 0x7f1f4407b0d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:25.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.847+0000 7f1f68e00640 1 --2- 172.21.15.39:0/319620579 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f64153970 0x7f1f641f3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:25.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.847+0000 7f1f68e00640 1 --2- 172.21.15.39:0/319620579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f640cfc00 0x7f1f640b4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:25.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.847+0000 7f1f68e00640 1 --2- 172.21.15.39:0/319620579 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f64073e50 0x7f1f640b3a60 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:25.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.847+0000 7f1f68e00640 1 -- 172.21.15.39:0/319620579 >> 172.21.15.39:0/319620579 conn(0x7f1f640d15f0 msgr2=0x7f1f64160310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:25.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.847+0000 7f1f68e00640 1 -- 172.21.15.39:0/319620579 shutdown_connections 2024-09-21T07:35:25.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:25.847+0000 7f1f68e00640 1 -- 172.21.15.39:0/319620579 wait complete. 2024-09-21T07:35:25.878 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create test_metadata_pool 2024-09-21T07:35:26.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.105+0000 7fe92c000640 1 -- 172.21.15.39:0/2076733609 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9240706b0 msgr2=0x7fe9240cba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:26.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.105+0000 7fe92c000640 1 --2- 172.21.15.39:0/2076733609 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9240706b0 0x7fe9240cba00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe910009860 tx=0x7fe91002a5a0 comp rx=0 tx=0).stop 2024-09-21T07:35:26.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.105+0000 7fe92c000640 1 -- 172.21.15.39:0/2076733609 shutdown_connections 2024-09-21T07:35:26.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.105+0000 7fe92c000640 1 --2- 172.21.15.39:0/2076733609 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9240706b0 0x7fe9240cba00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:26.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.105+0000 7fe92c000640 1 -- 172.21.15.39:0/2076733609 >> 172.21.15.39:0/2076733609 conn(0x7fe9240b1cb0 msgr2=0x7fe9240b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:26.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.105+0000 7fe92c000640 1 -- 172.21.15.39:0/2076733609 shutdown_connections 2024-09-21T07:35:26.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.105+0000 7fe92c000640 1 -- 172.21.15.39:0/2076733609 wait complete. 2024-09-21T07:35:26.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.108+0000 7fe92c000640 1 Processor -- start 2024-09-21T07:35:26.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.108+0000 7fe92c000640 1 -- start start 2024-09-21T07:35:26.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.109+0000 7fe92c000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9240ca800 0x7fe924153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:26.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.109+0000 7fe92c000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9240cad30 0x7fe924156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:26.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.109+0000 7fe92c000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe924156e80 0x7fe9240aed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:26.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.109+0000 7fe92c000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe924074190 con 0x7fe9240cad30 2024-09-21T07:35:26.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.109+0000 7fe92c000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe924074010 con 0x7fe9240ca800 2024-09-21T07:35:26.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.109+0000 7fe92c000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe924074310 con 0x7fe924156e80 2024-09-21T07:35:26.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.109+0000 7fe928c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9240cad30 0x7fe924156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:26.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.110+0000 7fe928c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9240cad30 0x7fe924156940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39480/0 (socket says 172.21.15.39:39480) 2024-09-21T07:35:26.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.110+0000 7fe928c00640 1 -- 172.21.15.39:0/3119138190 learned_addr learned my addr 172.21.15.39:0/3119138190 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:26.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.110+0000 7fe929600640 1 --2- 172.21.15.39:0/3119138190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9240ca800 0x7fe924153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:26.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.110+0000 7fe92a000640 1 --2- 172.21.15.39:0/3119138190 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe924156e80 0x7fe9240aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:26.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.110+0000 7fe928c00640 1 -- 172.21.15.39:0/3119138190 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe924156e80 msgr2=0x7fe9240aed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:26.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.110+0000 7fe928c00640 1 --2- 172.21.15.39:0/3119138190 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe924156e80 0x7fe9240aed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:26.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.110+0000 7fe928c00640 1 -- 172.21.15.39:0/3119138190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9240ca800 msgr2=0x7fe924153110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:26.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.110+0000 7fe928c00640 1 --2- 172.21.15.39:0/3119138190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9240ca800 0x7fe924153110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:26.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.110+0000 7fe928c00640 1 -- 172.21.15.39:0/3119138190 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9241541a0 con 0x7fe9240cad30 2024-09-21T07:35:26.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.110+0000 7fe92a000640 1 --2- 172.21.15.39:0/3119138190 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe924156e80 0x7fe9240aed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:35:26.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.110+0000 7fe928c00640 1 --2- 172.21.15.39:0/3119138190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9240cad30 0x7fe924156940 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe91002ab90 tx=0x7fe910033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:26.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.111+0000 7fe921600640 1 -- 172.21.15.39:0/3119138190 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe910031b50 con 0x7fe9240cad30 2024-09-21T07:35:26.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.111+0000 7fe921600640 1 -- 172.21.15.39:0/3119138190 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe910031de0 con 0x7fe9240cad30 2024-09-21T07:35:26.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.111+0000 7fe92c000640 1 -- 172.21.15.39:0/3119138190 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe92415e230 con 0x7fe9240cad30 2024-09-21T07:35:26.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.111+0000 7fe921600640 1 -- 172.21.15.39:0/3119138190 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe910033330 con 0x7fe9240cad30 2024-09-21T07:35:26.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.111+0000 7fe92c000640 1 -- 172.21.15.39:0/3119138190 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe92415e7f0 con 0x7fe9240cad30 2024-09-21T07:35:26.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.112+0000 7fe92c000640 1 -- 172.21.15.39:0/3119138190 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe924074800 con 0x7fe9240cad30 2024-09-21T07:35:26.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.114+0000 7fe921600640 1 -- 172.21.15.39:0/3119138190 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe9100334d0 con 0x7fe9240cad30 2024-09-21T07:35:26.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.115+0000 7fe921600640 1 --2- 172.21.15.39:0/3119138190 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe904078b90 0x7fe90407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:26.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.115+0000 7fe921600640 1 -- 172.21.15.39:0/3119138190 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6415+0+0 (secure 0 0 0) 0x7fe9100bc3c0 con 0x7fe9240cad30 2024-09-21T07:35:26.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.116+0000 7fe929600640 1 --2- 172.21.15.39:0/3119138190 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe904078b90 0x7fe90407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:26.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.116+0000 7fe929600640 1 --2- 172.21.15.39:0/3119138190 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe904078b90 0x7fe90407b0a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe918001120 tx=0x7fe918009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:26.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.119+0000 7fe921600640 1 -- 172.21.15.39:0/3119138190 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe91004a030 con 0x7fe9240cad30 2024-09-21T07:35:26.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.314+0000 7fe92c000640 1 -- 172.21.15.39:0/3119138190 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "test_metadata_pool"} v 0) -- 0x7fe9241573b0 con 0x7fe9240cad30 2024-09-21T07:35:26.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.847+0000 7fe921600640 1 -- 172.21.15.39:0/3119138190 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "test_metadata_pool"}]=0 pool 'test_metadata_pool' created v64) ==== 126+0+0 (secure 0 0 0) 0x7fe91003f030 con 0x7fe9240cad30 2024-09-21T07:35:26.849 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'test_metadata_pool' created 2024-09-21T07:35:26.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.849+0000 7fe92c000640 1 -- 172.21.15.39:0/3119138190 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe904078b90 msgr2=0x7fe90407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:26.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.849+0000 7fe92c000640 1 --2- 172.21.15.39:0/3119138190 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe904078b90 0x7fe90407b0a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe918001120 tx=0x7fe918009040 comp rx=0 tx=0).stop 2024-09-21T07:35:26.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.849+0000 7fe92c000640 1 -- 172.21.15.39:0/3119138190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9240cad30 msgr2=0x7fe924156940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:26.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.849+0000 7fe92c000640 1 --2- 172.21.15.39:0/3119138190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9240cad30 0x7fe924156940 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe91002ab90 tx=0x7fe910033ba0 comp rx=0 tx=0).stop 2024-09-21T07:35:26.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.849+0000 7fe92c000640 1 -- 172.21.15.39:0/3119138190 shutdown_connections 2024-09-21T07:35:26.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.849+0000 7fe92c000640 1 --2- 172.21.15.39:0/3119138190 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe904078b90 0x7fe90407b0a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:26.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.849+0000 7fe92c000640 1 --2- 172.21.15.39:0/3119138190 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe924156e80 0x7fe9240aed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:26.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.849+0000 7fe92c000640 1 --2- 172.21.15.39:0/3119138190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9240cad30 0x7fe924156940 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:26.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.850+0000 7fe92c000640 1 --2- 172.21.15.39:0/3119138190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9240ca800 0x7fe924153110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:26.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.850+0000 7fe92c000640 1 -- 172.21.15.39:0/3119138190 >> 172.21.15.39:0/3119138190 conn(0x7fe9240b1cb0 msgr2=0x7fe92415a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:26.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.850+0000 7fe92c000640 1 -- 172.21.15.39:0/3119138190 shutdown_connections 2024-09-21T07:35:26.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:26.850+0000 7fe92c000640 1 -- 172.21.15.39:0/3119138190 wait complete. 2024-09-21T07:35:26.893 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool test_data_pool mksnap snap4 2024-09-21T07:35:27.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.034+0000 7f836f528f40 1 Processor -- start 2024-09-21T07:35:27.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.034+0000 7f836f528f40 1 -- start start 2024-09-21T07:35:27.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.035+0000 7f836f528f40 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ec6d60d450 0x55ec6d692720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:27.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.035+0000 7f836f528f40 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec6d691510 0x55ec6d603f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:27.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.035+0000 7f836f528f40 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ec6d69ae70 0x55ec6d729e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:27.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.035+0000 7f836f528f40 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ec6d511c60 con 0x55ec6d69ae70 2024-09-21T07:35:27.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.035+0000 7f836f528f40 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ec6d5701d0 con 0x55ec6d691510 2024-09-21T07:35:27.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.035+0000 7f836f528f40 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55ec6d60d860 con 0x55ec6d60d450 2024-09-21T07:35:27.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.036+0000 7f8368600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ec6d69ae70 0x55ec6d729e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:27.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.036+0000 7f8368600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ec6d69ae70 0x55ec6d729e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39488/0 (socket says 172.21.15.39:39488) 2024-09-21T07:35:27.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.036+0000 7f8368600640 1 -- 172.21.15.39:0/1126810226 learned_addr learned my addr 172.21.15.39:0/1126810226 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:27.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.036+0000 7f8367200640 1 --2- 172.21.15.39:0/1126810226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec6d691510 0x55ec6d603f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:27.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.036+0000 7f8367c00640 1 --2- 172.21.15.39:0/1126810226 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ec6d60d450 0x55ec6d692720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:27.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.036+0000 7f8368600640 1 -- 172.21.15.39:0/1126810226 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ec6d60d450 msgr2=0x55ec6d692720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:35:27.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.036+0000 7f8368600640 1 --2- 172.21.15.39:0/1126810226 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ec6d60d450 0x55ec6d692720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:27.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.036+0000 7f8368600640 1 -- 172.21.15.39:0/1126810226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec6d691510 msgr2=0x55ec6d603f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:35:27.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.036+0000 7f8368600640 1 --2- 172.21.15.39:0/1126810226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec6d691510 0x55ec6d603f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:27.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.036+0000 7f8368600640 1 -- 172.21.15.39:0/1126810226 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec6d6749a0 con 0x55ec6d69ae70 2024-09-21T07:35:27.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.037+0000 7f8368600640 1 --2- 172.21.15.39:0/1126810226 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ec6d69ae70 0x55ec6d729e20 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8360057040 tx=0x7f8360003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=571f14f516b6cdc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:27.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.037+0000 7f8365e00640 1 -- 172.21.15.39:0/1126810226 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8360064030 con 0x55ec6d69ae70 2024-09-21T07:35:27.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.037+0000 7f8365e00640 1 -- 172.21.15.39:0/1126810226 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8360004020 con 0x55ec6d69ae70 2024-09-21T07:35:27.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.037+0000 7f8365e00640 1 -- 172.21.15.39:0/1126810226 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8360004270 con 0x55ec6d69ae70 2024-09-21T07:35:27.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.038+0000 7f836f528f40 1 -- 172.21.15.39:0/1126810226 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ec6d69ae70 msgr2=0x55ec6d729e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:27.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.038+0000 7f836f528f40 1 --2- 172.21.15.39:0/1126810226 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ec6d69ae70 0x55ec6d729e20 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8360057040 tx=0x7f8360003330 comp rx=0 tx=0).stop 2024-09-21T07:35:27.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.038+0000 7f836f528f40 1 -- 172.21.15.39:0/1126810226 shutdown_connections 2024-09-21T07:35:27.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.038+0000 7f836f528f40 1 --2- 172.21.15.39:0/1126810226 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ec6d69ae70 0x55ec6d729e20 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:27.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.038+0000 7f836f528f40 1 --2- 172.21.15.39:0/1126810226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec6d691510 0x55ec6d603f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:27.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.038+0000 7f836f528f40 1 --2- 172.21.15.39:0/1126810226 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ec6d60d450 0x55ec6d692720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:27.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.038+0000 7f836f528f40 1 -- 172.21.15.39:0/1126810226 >> 172.21.15.39:0/1126810226 conn(0x55ec6d4e2ac0 msgr2=0x55ec6d67cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:27.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.038+0000 7f836f528f40 1 -- 172.21.15.39:0/1126810226 shutdown_connections 2024-09-21T07:35:27.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.038+0000 7f836f528f40 1 -- 172.21.15.39:0/1126810226 wait complete. 2024-09-21T07:35:27.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.039+0000 7f836f528f40 1 Processor -- start 2024-09-21T07:35:27.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.039+0000 7f836f528f40 1 -- start start 2024-09-21T07:35:27.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.040+0000 7f836f528f40 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ec6d60d450 0x55ec6d608360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:27.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.040+0000 7f836f528f40 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec6d691510 0x55ec6d671cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:27.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.041+0000 7f836f528f40 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ec6d69ae70 0x55ec6d5c21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:27.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.041+0000 7f836f528f40 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ec6d67c7d0 con 0x55ec6d60d450 2024-09-21T07:35:27.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.041+0000 7f836f528f40 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ec6d5c2ea0 con 0x55ec6d691510 2024-09-21T07:35:27.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.041+0000 7f836f528f40 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55ec6d6943c0 con 0x55ec6d69ae70 2024-09-21T07:35:27.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.041+0000 7f8367c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ec6d60d450 0x55ec6d608360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:27.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.041+0000 7f8367200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec6d691510 0x55ec6d671cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:27.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.041+0000 7f8367c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ec6d60d450 0x55ec6d608360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39504/0 (socket says 172.21.15.39:39504) 2024-09-21T07:35:27.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.041+0000 7f8367c00640 1 -- 172.21.15.39:0/1149565985 learned_addr learned my addr 172.21.15.39:0/1149565985 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:27.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.041+0000 7f8368600640 1 --2- 172.21.15.39:0/1149565985 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ec6d69ae70 0x55ec6d5c21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:27.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.041+0000 7f8367c00640 1 -- 172.21.15.39:0/1149565985 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ec6d69ae70 msgr2=0x55ec6d5c21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:27.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.041+0000 7f8367c00640 1 --2- 172.21.15.39:0/1149565985 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ec6d69ae70 0x55ec6d5c21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:27.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.041+0000 7f8367c00640 1 -- 172.21.15.39:0/1149565985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec6d691510 msgr2=0x55ec6d671cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:27.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.041+0000 7f8367c00640 1 --2- 172.21.15.39:0/1149565985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec6d691510 0x55ec6d671cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:27.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.041+0000 7f8367c00640 1 -- 172.21.15.39:0/1149565985 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec6d673b70 con 0x55ec6d60d450 2024-09-21T07:35:27.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.042+0000 7f8367c00640 1 --2- 172.21.15.39:0/1149565985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ec6d60d450 0x55ec6d608360 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f835800a6c0 tx=0x7f835800ab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:27.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.042+0000 7f8364a00640 1 -- 172.21.15.39:0/1149565985 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835801c070 con 0x55ec6d60d450 2024-09-21T07:35:27.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.042+0000 7f8364a00640 1 -- 172.21.15.39:0/1149565985 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8358008da0 con 0x55ec6d60d450 2024-09-21T07:35:27.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.042+0000 7f836f528f40 1 -- 172.21.15.39:0/1149565985 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ec6d694610 con 0x55ec6d60d450 2024-09-21T07:35:27.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.042+0000 7f8364a00640 1 -- 172.21.15.39:0/1149565985 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83580176e0 con 0x55ec6d60d450 2024-09-21T07:35:27.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.042+0000 7f836f528f40 1 -- 172.21.15.39:0/1149565985 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec6d5c2be0 con 0x55ec6d60d450 2024-09-21T07:35:27.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.044+0000 7f8364a00640 1 -- 172.21.15.39:0/1149565985 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f835801f070 con 0x55ec6d60d450 2024-09-21T07:35:27.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.045+0000 7f8364a00640 1 --2- 172.21.15.39:0/1149565985 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f83480775a0 0x7f8348079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:27.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.045+0000 7f8364a00640 1 -- 172.21.15.39:0/1149565985 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6772+0+0 (secure 0 0 0) 0x7f835809a2d0 con 0x55ec6d60d450 2024-09-21T07:35:27.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.046+0000 7f836f528f40 1 -- 172.21.15.39:0/1149565985 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- pool_op(create snap pool 11 tid 1 name snap4 v0) -- 0x55ec6d572b20 con 0x55ec6d60d450 2024-09-21T07:35:27.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.046+0000 7f8367200640 1 --2- 172.21.15.39:0/1149565985 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f83480775a0 0x7f8348079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:27.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.047+0000 7f8367200640 1 --2- 172.21.15.39:0/1149565985 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f83480775a0 0x7f8348079ab0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x55ec6d5c7120 tx=0x7f8354001730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:27.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.856+0000 7f8364a00640 1 -- 172.21.15.39:0/1149565985 <== mon.0 v2:172.21.15.39:3300/0 6 ==== pool_op_reply(tid 1 (0) Success v65) ==== 43+0+0 (secure 0 0 0) 0x7f8358061c90 con 0x55ec6d60d450 2024-09-21T07:35:27.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.856+0000 7f8364a00640 1 -- 172.21.15.39:0/1149565985 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=65}) -- 0x7f8348081820 con 0x55ec6d60d450 2024-09-21T07:35:27.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.857+0000 7f8364a00640 1 -- 172.21.15.39:0/1149565985 <== mon.0 v2:172.21.15.39:3300/0 7 ==== osd_map(65..65 src has 1..65) ==== 718+0+0 (secure 0 0 0) 0x7f8358098440 con 0x55ec6d60d450 2024-09-21T07:35:27.859 INFO:teuthology.orchestra.run.smithi039.stdout:created pool test_data_pool snap snap4 2024-09-21T07:35:27.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.857+0000 7f836f528f40 1 -- 172.21.15.39:0/1149565985 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f83480775a0 msgr2=0x7f8348079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:27.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.857+0000 7f836f528f40 1 --2- 172.21.15.39:0/1149565985 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f83480775a0 0x7f8348079ab0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x55ec6d5c7120 tx=0x7f8354001730 comp rx=0 tx=0).stop 2024-09-21T07:35:27.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.857+0000 7f836f528f40 1 -- 172.21.15.39:0/1149565985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ec6d60d450 msgr2=0x55ec6d608360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:27.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.857+0000 7f836f528f40 1 --2- 172.21.15.39:0/1149565985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ec6d60d450 0x55ec6d608360 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f835800a6c0 tx=0x7f835800ab80 comp rx=0 tx=0).stop 2024-09-21T07:35:27.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.858+0000 7f836f528f40 1 -- 172.21.15.39:0/1149565985 shutdown_connections 2024-09-21T07:35:27.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.858+0000 7f836f528f40 1 --2- 172.21.15.39:0/1149565985 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f83480775a0 0x7f8348079ab0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:27.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.858+0000 7f836f528f40 1 --2- 172.21.15.39:0/1149565985 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ec6d69ae70 0x55ec6d5c21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:27.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.858+0000 7f836f528f40 1 --2- 172.21.15.39:0/1149565985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec6d691510 0x55ec6d671cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:27.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.858+0000 7f836f528f40 1 --2- 172.21.15.39:0/1149565985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ec6d60d450 0x55ec6d608360 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:27.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.858+0000 7f836f528f40 1 -- 172.21.15.39:0/1149565985 >> 172.21.15.39:0/1149565985 conn(0x55ec6d4e2ac0 msgr2=0x55ec6d603e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:27.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.859+0000 7f836f528f40 1 -- 172.21.15.39:0/1149565985 shutdown_connections 2024-09-21T07:35:27.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:27.859+0000 7f836f528f40 1 -- 172.21.15.39:0/1149565985 wait complete. 2024-09-21T07:35:27.888 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new testfs test_metadata_pool test_data_pool 2024-09-21T07:35:28.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.121+0000 7f051fa00640 1 -- 172.21.15.39:0/1557724997 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f05180af3c0 msgr2=0x7f05180b40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:28.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.121+0000 7f051fa00640 1 --2- 172.21.15.39:0/1557724997 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f05180af3c0 0x7f05180b40b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0500005fd0 tx=0x7f050002a5d0 comp rx=0 tx=0).stop 2024-09-21T07:35:28.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.121+0000 7f051fa00640 1 -- 172.21.15.39:0/1557724997 shutdown_connections 2024-09-21T07:35:28.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.121+0000 7f051fa00640 1 --2- 172.21.15.39:0/1557724997 >> v2:172.21.15.148:3300/0 conn(0x7f05180ccb50 0x7f05180ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:28.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.121+0000 7f051fa00640 1 --2- 172.21.15.39:0/1557724997 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f05180af3c0 0x7f05180b40b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:28.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.121+0000 7f051fa00640 1 -- 172.21.15.39:0/1557724997 >> 172.21.15.39:0/1557724997 conn(0x7f051806d7d0 msgr2=0x7f05180b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:28.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.121+0000 7f051fa00640 1 -- 172.21.15.39:0/1557724997 shutdown_connections 2024-09-21T07:35:28.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.121+0000 7f051fa00640 1 -- 172.21.15.39:0/1557724997 wait complete. 2024-09-21T07:35:28.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.124+0000 7f051fa00640 1 Processor -- start 2024-09-21T07:35:28.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.124+0000 7f051fa00640 1 -- start start 2024-09-21T07:35:28.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.125+0000 7f051fa00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f05180af3c0 0x7f051815fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:28.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.125+0000 7f051fa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05180cc1e0 0x7f0518152b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:28.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.125+0000 7f051fa00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05180ccb50 0x7f051809ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:28.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.125+0000 7f051fa00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0518153520 con 0x7f05180cc1e0 2024-09-21T07:35:28.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.125+0000 7f051fa00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f05181533a0 con 0x7f05180ccb50 2024-09-21T07:35:28.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.125+0000 7f051fa00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f05181536a0 con 0x7f05180af3c0 2024-09-21T07:35:28.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.125+0000 7f050fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05180cc1e0 0x7f0518152b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:28.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.126+0000 7f050fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05180cc1e0 0x7f0518152b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39518/0 (socket says 172.21.15.39:39518) 2024-09-21T07:35:28.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.126+0000 7f050fe00640 1 -- 172.21.15.39:0/4257657355 learned_addr learned my addr 172.21.15.39:0/4257657355 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:28.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.126+0000 7f051d000640 1 --2- 172.21.15.39:0/4257657355 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f05180af3c0 0x7f051815fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:28.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.126+0000 7f051da00640 1 --2- 172.21.15.39:0/4257657355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05180ccb50 0x7f051809ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:28.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.126+0000 7f050fe00640 1 -- 172.21.15.39:0/4257657355 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f05180af3c0 msgr2=0x7f051815fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:28.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.126+0000 7f050fe00640 1 --2- 172.21.15.39:0/4257657355 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f05180af3c0 0x7f051815fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:28.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.126+0000 7f050fe00640 1 -- 172.21.15.39:0/4257657355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05180ccb50 msgr2=0x7f051809ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:28.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.126+0000 7f050fe00640 1 --2- 172.21.15.39:0/4257657355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05180ccb50 0x7f051809ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:28.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.126+0000 7f050fe00640 1 -- 172.21.15.39:0/4257657355 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f051815ecf0 con 0x7f05180cc1e0 2024-09-21T07:35:28.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.126+0000 7f050fe00640 1 --2- 172.21.15.39:0/4257657355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05180cc1e0 0x7f0518152b80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0500034040 tx=0x7f05000035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:28.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.127+0000 7f0515600640 1 -- 172.21.15.39:0/4257657355 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050003e030 con 0x7f05180cc1e0 2024-09-21T07:35:28.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.127+0000 7f0515600640 1 -- 172.21.15.39:0/4257657355 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0500003ae0 con 0x7f05180cc1e0 2024-09-21T07:35:28.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.127+0000 7f0515600640 1 -- 172.21.15.39:0/4257657355 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0500035040 con 0x7f05180cc1e0 2024-09-21T07:35:28.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.127+0000 7f051fa00640 1 -- 172.21.15.39:0/4257657355 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f051815efd0 con 0x7f05180cc1e0 2024-09-21T07:35:28.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.127+0000 7f051fa00640 1 -- 172.21.15.39:0/4257657355 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f051815f590 con 0x7f05180cc1e0 2024-09-21T07:35:28.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.128+0000 7f051fa00640 1 -- 172.21.15.39:0/4257657355 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0518153bc0 con 0x7f05180cc1e0 2024-09-21T07:35:28.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.129+0000 7f0515600640 1 -- 172.21.15.39:0/4257657355 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0500041030 con 0x7f05180cc1e0 2024-09-21T07:35:28.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.130+0000 7f0515600640 1 --2- 172.21.15.39:0/4257657355 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f04f4078b90 0x7f04f407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:28.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.130+0000 7f0515600640 1 -- 172.21.15.39:0/4257657355 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6811+0+0 (secure 0 0 0) 0x7f05000bb830 con 0x7f05180cc1e0 2024-09-21T07:35:28.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.131+0000 7f051d000640 1 --2- 172.21.15.39:0/4257657355 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f04f4078b90 0x7f04f407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:28.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.132+0000 7f051d000640 1 --2- 172.21.15.39:0/4257657355 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f04f4078b90 0x7f04f407b0a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0508001720 tx=0x7f0508009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:28.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.134+0000 7f0515600640 1 -- 172.21.15.39:0/4257657355 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f05000833d0 con 0x7f05180cc1e0 2024-09-21T07:35:28.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.371+0000 7f051fa00640 1 -- 172.21.15.39:0/4257657355 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "testfs", "metadata": "test_metadata_pool", "data": "test_data_pool"} v 0) -- 0x7f0518070c30 con 0x7f05180cc1e0 2024-09-21T07:35:28.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.372+0000 7f0515600640 1 -- 172.21.15.39:0/4257657355 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "testfs", "metadata": "test_metadata_pool", "data": "test_data_pool"}]=-95 pool(11) already has mon-managed snaps; can't attach pool to fs v35) ==== 198+0+0 (secure 0 0 0) 0x7f0500087e00 con 0x7f05180cc1e0 2024-09-21T07:35:28.374 INFO:teuthology.orchestra.run.smithi039.stderr:Error ENOTSUP: pool(11) already has mon-managed snaps; can't attach pool to fs 2024-09-21T07:35:28.394 DEBUG:teuthology.orchestra.run:got remote process result: 95 2024-09-21T07:35:28.394 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool test_data_pool rmsnap snap4 2024-09-21T07:35:28.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.536+0000 7f5090068f40 1 Processor -- start 2024-09-21T07:35:28.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.536+0000 7f5090068f40 1 -- start start 2024-09-21T07:35:28.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.537+0000 7f5090068f40 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a0d2806ae0 0x55a0d28876c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:28.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.537+0000 7f5090068f40 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0d2807540 0x55a0d2809370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:28.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.537+0000 7f5090068f40 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a0d2899340 0x55a0d28098b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:28.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.537+0000 7f5090068f40 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a0d2807100 con 0x55a0d2899340 2024-09-21T07:35:28.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.537+0000 7f5090068f40 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a0d2711c60 con 0x55a0d2807540 2024-09-21T07:35:28.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.537+0000 7f5090068f40 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55a0d280cb40 con 0x55a0d2806ae0 2024-09-21T07:35:28.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.538+0000 7f5089c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a0d2899340 0x55a0d28098b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:28.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.538+0000 7f5089c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a0d2899340 0x55a0d28098b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39538/0 (socket says 172.21.15.39:39538) 2024-09-21T07:35:28.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.538+0000 7f5089c00640 1 -- 172.21.15.39:0/1523381039 learned_addr learned my addr 172.21.15.39:0/1523381039 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:28.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.538+0000 7f5089200640 1 --2- 172.21.15.39:0/1523381039 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a0d2806ae0 0x55a0d28876c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:28.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.538+0000 7f5083e00640 1 --2- 172.21.15.39:0/1523381039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0d2807540 0x55a0d2809370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:28.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.538+0000 7f5089c00640 1 -- 172.21.15.39:0/1523381039 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a0d2806ae0 msgr2=0x55a0d28876c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:35:28.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.538+0000 7f5089c00640 1 --2- 172.21.15.39:0/1523381039 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a0d2806ae0 0x55a0d28876c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:28.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.538+0000 7f5089c00640 1 -- 172.21.15.39:0/1523381039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0d2807540 msgr2=0x55a0d2809370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:35:28.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.538+0000 7f5089c00640 1 --2- 172.21.15.39:0/1523381039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0d2807540 0x55a0d2809370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:28.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.538+0000 7f5089c00640 1 -- 172.21.15.39:0/1523381039 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0d26e9f10 con 0x55a0d2899340 2024-09-21T07:35:28.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.538+0000 7f5089c00640 1 --2- 172.21.15.39:0/1523381039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a0d2899340 0x55a0d28098b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f508400d7c0 tx=0x7f508400dc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e5e11de2a57cfaf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:28.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.539+0000 7f5082a00640 1 -- 172.21.15.39:0/1523381039 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508400ba90 con 0x55a0d2899340 2024-09-21T07:35:28.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.539+0000 7f5082a00640 1 -- 172.21.15.39:0/1523381039 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f508400bd20 con 0x55a0d2899340 2024-09-21T07:35:28.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.539+0000 7f5082a00640 1 -- 172.21.15.39:0/1523381039 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508400f690 con 0x55a0d2899340 2024-09-21T07:35:28.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.539+0000 7f5090068f40 1 -- 172.21.15.39:0/1523381039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a0d2899340 msgr2=0x55a0d28098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:28.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.539+0000 7f5090068f40 1 --2- 172.21.15.39:0/1523381039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a0d2899340 0x55a0d28098b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f508400d7c0 tx=0x7f508400dc80 comp rx=0 tx=0).stop 2024-09-21T07:35:28.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.539+0000 7f5090068f40 1 -- 172.21.15.39:0/1523381039 shutdown_connections 2024-09-21T07:35:28.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.539+0000 7f5090068f40 1 --2- 172.21.15.39:0/1523381039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a0d2899340 0x55a0d28098b0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:28.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.539+0000 7f5090068f40 1 --2- 172.21.15.39:0/1523381039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0d2807540 0x55a0d2809370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:28.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.539+0000 7f5090068f40 1 --2- 172.21.15.39:0/1523381039 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a0d2806ae0 0x55a0d28876c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:28.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.539+0000 7f5090068f40 1 -- 172.21.15.39:0/1523381039 >> 172.21.15.39:0/1523381039 conn(0x55a0d26e2ac0 msgr2=0x55a0d289d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:28.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.539+0000 7f5090068f40 1 -- 172.21.15.39:0/1523381039 shutdown_connections 2024-09-21T07:35:28.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.539+0000 7f5090068f40 1 -- 172.21.15.39:0/1523381039 wait complete. 2024-09-21T07:35:28.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.540+0000 7f5090068f40 1 Processor -- start 2024-09-21T07:35:28.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.541+0000 7f5090068f40 1 -- start start 2024-09-21T07:35:28.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.542+0000 7f5090068f40 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0d2806ae0 0x55a0d27c19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:28.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.542+0000 7f5090068f40 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a0d2807540 0x55a0d289b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:28.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.542+0000 7f5090068f40 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a0d2899340 0x55a0d289b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:28.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.542+0000 7f5090068f40 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a0d2898170 con 0x55a0d2807540 2024-09-21T07:35:28.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.542+0000 7f5090068f40 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a0d2887c00 con 0x55a0d2806ae0 2024-09-21T07:35:28.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.542+0000 7f5090068f40 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55a0d2894fa0 con 0x55a0d2899340 2024-09-21T07:35:28.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.542+0000 7f5083e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a0d2807540 0x55a0d289b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:28.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.542+0000 7f5083e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a0d2807540 0x55a0d289b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39550/0 (socket says 172.21.15.39:39550) 2024-09-21T07:35:28.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.542+0000 7f5083e00640 1 -- 172.21.15.39:0/1143714395 learned_addr learned my addr 172.21.15.39:0/1143714395 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:28.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.543+0000 7f5089c00640 1 --2- 172.21.15.39:0/1143714395 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a0d2899340 0x55a0d289b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:28.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.543+0000 7f5089200640 1 --2- 172.21.15.39:0/1143714395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0d2806ae0 0x55a0d27c19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:28.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.543+0000 7f5083e00640 1 -- 172.21.15.39:0/1143714395 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a0d2899340 msgr2=0x55a0d289b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:28.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.543+0000 7f5083e00640 1 --2- 172.21.15.39:0/1143714395 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a0d2899340 0x55a0d289b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:28.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.543+0000 7f5083e00640 1 -- 172.21.15.39:0/1143714395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0d2806ae0 msgr2=0x55a0d27c19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:28.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.543+0000 7f5083e00640 1 --2- 172.21.15.39:0/1143714395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0d2806ae0 0x55a0d27c19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:28.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.543+0000 7f5083e00640 1 -- 172.21.15.39:0/1143714395 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0d2883e60 con 0x55a0d2807540 2024-09-21T07:35:28.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.543+0000 7f5089200640 1 --2- 172.21.15.39:0/1143714395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0d2806ae0 0x55a0d27c19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:35:28.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.543+0000 7f5089c00640 1 --2- 172.21.15.39:0/1143714395 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a0d2899340 0x55a0d289b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:35:28.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.543+0000 7f5083e00640 1 --2- 172.21.15.39:0/1143714395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a0d2807540 0x55a0d289b4a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f507400a680 tx=0x7f507400ab40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:28.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.543+0000 7f5081600640 1 -- 172.21.15.39:0/1143714395 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507401a070 con 0x55a0d2807540 2024-09-21T07:35:28.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.543+0000 7f5081600640 1 -- 172.21.15.39:0/1143714395 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5074008930 con 0x55a0d2807540 2024-09-21T07:35:28.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.543+0000 7f5090068f40 1 -- 172.21.15.39:0/1143714395 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a0d2899d60 con 0x55a0d2807540 2024-09-21T07:35:28.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.543+0000 7f5081600640 1 -- 172.21.15.39:0/1143714395 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5074008d40 con 0x55a0d2807540 2024-09-21T07:35:28.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.543+0000 7f5090068f40 1 -- 172.21.15.39:0/1143714395 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0d28843e0 con 0x55a0d2807540 2024-09-21T07:35:28.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.546+0000 7f5081600640 1 -- 172.21.15.39:0/1143714395 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f507401d070 con 0x55a0d2807540 2024-09-21T07:35:28.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.546+0000 7f5081600640 1 --2- 172.21.15.39:0/1143714395 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5068077570 0x7f5068079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:28.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.547+0000 7f5081600640 1 -- 172.21.15.39:0/1143714395 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6811+0+0 (secure 0 0 0) 0x7f5074099880 con 0x55a0d2807540 2024-09-21T07:35:28.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.547+0000 7f5090068f40 1 -- 172.21.15.39:0/1143714395 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- pool_op(delete snap pool 11 tid 1 name snap4 v0) -- 0x55a0d2772b20 con 0x55a0d2807540 2024-09-21T07:35:28.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.547+0000 7f5089200640 1 --2- 172.21.15.39:0/1143714395 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5068077570 0x7f5068079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:28.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.548+0000 7f5089200640 1 --2- 172.21.15.39:0/1143714395 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5068077570 0x7f5068079a80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f507c055000 tx=0x7f507c058040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:28.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.860+0000 7f5081600640 1 -- 172.21.15.39:0/1143714395 <== mon.0 v2:172.21.15.39:3300/0 6 ==== pool_op_reply(tid 1 (0) Success v66) ==== 43+0+0 (secure 0 0 0) 0x7f5074061340 con 0x55a0d2807540 2024-09-21T07:35:28.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.860+0000 7f5081600640 1 -- 172.21.15.39:0/1143714395 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=66}) -- 0x7f5068081850 con 0x55a0d2807540 2024-09-21T07:35:28.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.862+0000 7f5081600640 1 -- 172.21.15.39:0/1143714395 <== mon.0 v2:172.21.15.39:3300/0 7 ==== osd_map(66..66 src has 1..66) ==== 970+0+0 (secure 0 0 0) 0x7f5074097a70 con 0x55a0d2807540 2024-09-21T07:35:28.865 INFO:teuthology.orchestra.run.smithi039.stdout:removed pool test_data_pool snap snap4 2024-09-21T07:35:28.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.862+0000 7f5090068f40 1 -- 172.21.15.39:0/1143714395 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5068077570 msgr2=0x7f5068079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:28.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.862+0000 7f5090068f40 1 --2- 172.21.15.39:0/1143714395 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5068077570 0x7f5068079a80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f507c055000 tx=0x7f507c058040 comp rx=0 tx=0).stop 2024-09-21T07:35:28.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.862+0000 7f5090068f40 1 -- 172.21.15.39:0/1143714395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a0d2807540 msgr2=0x55a0d289b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:28.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.862+0000 7f5090068f40 1 --2- 172.21.15.39:0/1143714395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a0d2807540 0x55a0d289b4a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f507400a680 tx=0x7f507400ab40 comp rx=0 tx=0).stop 2024-09-21T07:35:28.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.862+0000 7f5090068f40 1 -- 172.21.15.39:0/1143714395 shutdown_connections 2024-09-21T07:35:28.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.862+0000 7f5090068f40 1 --2- 172.21.15.39:0/1143714395 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5068077570 0x7f5068079a80 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:28.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.862+0000 7f5090068f40 1 --2- 172.21.15.39:0/1143714395 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55a0d2899340 0x55a0d289b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:28.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.862+0000 7f5090068f40 1 --2- 172.21.15.39:0/1143714395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a0d2807540 0x55a0d289b4a0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:28.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.862+0000 7f5090068f40 1 --2- 172.21.15.39:0/1143714395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a0d2806ae0 0x55a0d27c19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:28.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.862+0000 7f5090068f40 1 -- 172.21.15.39:0/1143714395 >> 172.21.15.39:0/1143714395 conn(0x55a0d26e2ac0 msgr2=0x55a0d280a760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:28.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.863+0000 7f5090068f40 1 -- 172.21.15.39:0/1143714395 shutdown_connections 2024-09-21T07:35:28.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:28.863+0000 7f5090068f40 1 -- 172.21.15.39:0/1143714395 wait complete. 2024-09-21T07:35:28.880 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete test_data_pool 2024-09-21T07:35:29.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.109+0000 7f5544c00640 1 -- 172.21.15.39:0/1548103675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5524004d90 msgr2=0x7f5524005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:29.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.109+0000 7f5544c00640 1 --2- 172.21.15.39:0/1548103675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5524004d90 0x7f5524005220 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f55400b2610 tx=0x7f5524005830 comp rx=0 tx=0).stop 2024-09-21T07:35:29.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.109+0000 7f553fe00640 1 -- 172.21.15.39:0/1548103675 reap_dead start 2024-09-21T07:35:29.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.109+0000 7f5544c00640 1 -- 172.21.15.39:0/1548103675 shutdown_connections 2024-09-21T07:35:29.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.109+0000 7f5544c00640 1 -- 172.21.15.39:0/1548103675 >> 172.21.15.39:0/1548103675 conn(0x7f554006d410 msgr2=0x7f55400b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:29.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.109+0000 7f5544c00640 1 -- 172.21.15.39:0/1548103675 shutdown_connections 2024-09-21T07:35:29.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.109+0000 7f5544c00640 1 -- 172.21.15.39:0/1548103675 wait complete. 2024-09-21T07:35:29.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.111+0000 7f5544c00640 1 Processor -- start 2024-09-21T07:35:29.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.112+0000 7f5544c00640 1 -- start start 2024-09-21T07:35:29.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.113+0000 7f5544c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5524004d90 0x7f55400cac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:29.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.113+0000 7f5544c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55401428f0 0x7f55400c9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:29.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.113+0000 7f5544c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55400c9bb0 0x7f55401388f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:29.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.113+0000 7f5544c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f55400ccd00 con 0x7f55401428f0 2024-09-21T07:35:29.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.113+0000 7f5544c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f55400ccb80 con 0x7f55400c9bb0 2024-09-21T07:35:29.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.113+0000 7f5544c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f55400cce80 con 0x7f5524004d90 2024-09-21T07:35:29.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.113+0000 7f5537e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55401428f0 0x7f55400c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:29.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.113+0000 7f5537e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55401428f0 0x7f55400c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39568/0 (socket says 172.21.15.39:39568) 2024-09-21T07:35:29.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.113+0000 7f5537e00640 1 -- 172.21.15.39:0/1253350460 learned_addr learned my addr 172.21.15.39:0/1253350460 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:29.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.114+0000 7f553f400640 1 --2- 172.21.15.39:0/1253350460 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5524004d90 0x7f55400cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:29.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.114+0000 7f553fe00640 1 --2- 172.21.15.39:0/1253350460 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55400c9bb0 0x7f55401388f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:29.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.114+0000 7f5537e00640 1 -- 172.21.15.39:0/1253350460 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5524004d90 msgr2=0x7f55400cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:29.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.114+0000 7f5537e00640 1 --2- 172.21.15.39:0/1253350460 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5524004d90 0x7f55400cac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:29.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.114+0000 7f5537e00640 1 -- 172.21.15.39:0/1253350460 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55400c9bb0 msgr2=0x7f55401388f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:29.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.114+0000 7f5537e00640 1 --2- 172.21.15.39:0/1253350460 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55400c9bb0 0x7f55401388f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:29.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.114+0000 7f5537e00640 1 -- 172.21.15.39:0/1253350460 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55401f4f40 con 0x7f55401428f0 2024-09-21T07:35:29.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.114+0000 7f5537e00640 1 --2- 172.21.15.39:0/1253350460 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55401428f0 0x7f55400c9670 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f5528001050 tx=0x7f5528001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:29.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.115+0000 7f5537400640 1 -- 172.21.15.39:0/1253350460 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5528011640 con 0x7f55401428f0 2024-09-21T07:35:29.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.115+0000 7f5537400640 1 -- 172.21.15.39:0/1253350460 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5528016020 con 0x7f55401428f0 2024-09-21T07:35:29.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.115+0000 7f5544c00640 1 -- 172.21.15.39:0/1253350460 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5540153d30 con 0x7f55401428f0 2024-09-21T07:35:29.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.115+0000 7f5537400640 1 -- 172.21.15.39:0/1253350460 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5528011a80 con 0x7f55401428f0 2024-09-21T07:35:29.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.115+0000 7f5544c00640 1 -- 172.21.15.39:0/1253350460 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5540156400 con 0x7f55401428f0 2024-09-21T07:35:29.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.116+0000 7f5544c00640 1 -- 172.21.15.39:0/1253350460 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f554015e220 con 0x7f55401428f0 2024-09-21T07:35:29.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.117+0000 7f5537400640 1 -- 172.21.15.39:0/1253350460 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5528011c20 con 0x7f55401428f0 2024-09-21T07:35:29.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.118+0000 7f5537400640 1 --2- 172.21.15.39:0/1253350460 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f551c078bc0 0x7f551c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:29.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.118+0000 7f5537400640 1 -- 172.21.15.39:0/1253350460 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6828+0+0 (secure 0 0 0) 0x7f552809bda0 con 0x7f55401428f0 2024-09-21T07:35:29.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.119+0000 7f553f400640 1 --2- 172.21.15.39:0/1253350460 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f551c078bc0 0x7f551c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:29.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.119+0000 7f553f400640 1 --2- 172.21.15.39:0/1253350460 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f551c078bc0 0x7f551c07b0d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5530009c50 tx=0x7f5530005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:29.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.123+0000 7f5537400640 1 -- 172.21.15.39:0/1253350460 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5528020020 con 0x7f55401428f0 2024-09-21T07:35:29.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.318+0000 7f5544c00640 1 -- 172.21.15.39:0/1253350460 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "test_data_pool"} v 0) -- 0x7f554006fb50 con 0x7f55401428f0 2024-09-21T07:35:29.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.318+0000 7f5537400640 1 -- 172.21.15.39:0/1253350460 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "test_data_pool"}]=-1 WARNING: this will *PERMANENTLY DESTROY* all data stored in pool test_data_pool. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. v66) ==== 296+0+0 (secure 0 0 0) 0x7f5528063d70 con 0x7f55401428f0 2024-09-21T07:35:29.321 INFO:teuthology.orchestra.run.smithi039.stderr:Error EPERM: WARNING: this will *PERMANENTLY DESTROY* all data stored in pool test_data_pool. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. 2024-09-21T07:35:29.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:35:29.341 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete test_metadata_pool 2024-09-21T07:35:29.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.570+0000 7f0f37e00640 1 -- 172.21.15.39:0/1320570862 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0f380caf80 msgr2=0x7f0f380b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:29.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.570+0000 7f0f37e00640 1 --2- 172.21.15.39:0/1320570862 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0f380caf80 0x7f0f380b4d90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0f28009830 tx=0x7f0f2802a490 comp rx=0 tx=0).stop 2024-09-21T07:35:29.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.570+0000 7f0f37e00640 1 -- 172.21.15.39:0/1320570862 shutdown_connections 2024-09-21T07:35:29.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.570+0000 7f0f37e00640 1 --2- 172.21.15.39:0/1320570862 >> v2:172.21.15.148:3300/0 conn(0x7f0f380d1100 0x7f0f380b71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:29.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.570+0000 7f0f37e00640 1 --2- 172.21.15.39:0/1320570862 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0f380caf80 0x7f0f380b4d90 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:29.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.570+0000 7f0f37e00640 1 -- 172.21.15.39:0/1320570862 >> 172.21.15.39:0/1320570862 conn(0x7f0f3809d6b0 msgr2=0x7f0f380d4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:29.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.570+0000 7f0f37e00640 1 -- 172.21.15.39:0/1320570862 shutdown_connections 2024-09-21T07:35:29.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.570+0000 7f0f37e00640 1 -- 172.21.15.39:0/1320570862 wait complete. 2024-09-21T07:35:29.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.573+0000 7f0f37e00640 1 Processor -- start 2024-09-21T07:35:29.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.573+0000 7f0f37e00640 1 -- start start 2024-09-21T07:35:29.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.574+0000 7f0f37e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0f380caf80 0x7f0f3815c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:29.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.574+0000 7f0f37e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0f380cffa0 0x7f0f381f5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:29.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.574+0000 7f0f37e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f380d1100 0x7f0f3815ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:29.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.574+0000 7f0f37e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0f380b7ae0 con 0x7f0f380caf80 2024-09-21T07:35:29.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.574+0000 7f0f37e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0f380b7960 con 0x7f0f380d1100 2024-09-21T07:35:29.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.574+0000 7f0f37e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f0f380b7c60 con 0x7f0f380cffa0 2024-09-21T07:35:29.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.575+0000 7f0f36a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0f380caf80 0x7f0f3815c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:29.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.575+0000 7f0f36a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0f380caf80 0x7f0f3815c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39572/0 (socket says 172.21.15.39:39572) 2024-09-21T07:35:29.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.575+0000 7f0f36a00640 1 -- 172.21.15.39:0/2998030625 learned_addr learned my addr 172.21.15.39:0/2998030625 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:29.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.575+0000 7f0f37400640 1 --2- 172.21.15.39:0/2998030625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f380d1100 0x7f0f3815ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:29.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.575+0000 7f0f36000640 1 --2- 172.21.15.39:0/2998030625 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0f380cffa0 0x7f0f381f5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:29.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.575+0000 7f0f36a00640 1 -- 172.21.15.39:0/2998030625 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0f380cffa0 msgr2=0x7f0f381f5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:29.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.575+0000 7f0f36a00640 1 --2- 172.21.15.39:0/2998030625 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0f380cffa0 0x7f0f381f5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:29.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.575+0000 7f0f36a00640 1 -- 172.21.15.39:0/2998030625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f380d1100 msgr2=0x7f0f3815ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:29.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.575+0000 7f0f36a00640 1 --2- 172.21.15.39:0/2998030625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f380d1100 0x7f0f3815ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:29.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.575+0000 7f0f36a00640 1 -- 172.21.15.39:0/2998030625 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f381f6100 con 0x7f0f380caf80 2024-09-21T07:35:29.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.575+0000 7f0f36000640 1 --2- 172.21.15.39:0/2998030625 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0f380cffa0 0x7f0f381f5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:35:29.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.575+0000 7f0f36a00640 1 --2- 172.21.15.39:0/2998030625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0f380caf80 0x7f0f3815c800 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0f28000c00 tx=0x7f0f28033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:29.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.576+0000 7f0f2f400640 1 -- 172.21.15.39:0/2998030625 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f28005c90 con 0x7f0f380caf80 2024-09-21T07:35:29.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.576+0000 7f0f2f400640 1 -- 172.21.15.39:0/2998030625 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f28031e50 con 0x7f0f380caf80 2024-09-21T07:35:29.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.576+0000 7f0f2f400640 1 -- 172.21.15.39:0/2998030625 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f28033290 con 0x7f0f380caf80 2024-09-21T07:35:29.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.576+0000 7f0f37e00640 1 -- 172.21.15.39:0/2998030625 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f381f63e0 con 0x7f0f380caf80 2024-09-21T07:35:29.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.576+0000 7f0f37e00640 1 -- 172.21.15.39:0/2998030625 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f380cc2a0 con 0x7f0f380caf80 2024-09-21T07:35:29.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.577+0000 7f0f37e00640 1 -- 172.21.15.39:0/2998030625 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f381f6620 con 0x7f0f380caf80 2024-09-21T07:35:29.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.578+0000 7f0f2f400640 1 -- 172.21.15.39:0/2998030625 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0f28033430 con 0x7f0f380caf80 2024-09-21T07:35:29.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.579+0000 7f0f2f400640 1 --2- 172.21.15.39:0/2998030625 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0f18078b90 0x7f0f1807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:29.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.579+0000 7f0f2f400640 1 -- 172.21.15.39:0/2998030625 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6828+0+0 (secure 0 0 0) 0x7f0f280bc430 con 0x7f0f380caf80 2024-09-21T07:35:29.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.579+0000 7f0f36000640 1 --2- 172.21.15.39:0/2998030625 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0f18078b90 0x7f0f1807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:29.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.581+0000 7f0f36000640 1 --2- 172.21.15.39:0/2998030625 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0f18078b90 0x7f0f1807b0a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0f24001240 tx=0x7f0f24009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:29.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.584+0000 7f0f2f400640 1 -- 172.21.15.39:0/2998030625 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0f381f6620 con 0x7f0f380caf80 2024-09-21T07:35:29.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.778+0000 7f0f37e00640 1 -- 172.21.15.39:0/2998030625 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "test_metadata_pool"} v 0) -- 0x7f0f380b4d90 con 0x7f0f380caf80 2024-09-21T07:35:29.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:29.778+0000 7f0f2f400640 1 -- 172.21.15.39:0/2998030625 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "test_metadata_pool"}]=-1 WARNING: this will *PERMANENTLY DESTROY* all data stored in pool test_metadata_pool. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. v66) ==== 304+0+0 (secure 0 0 0) 0x7f0f2803f030 con 0x7f0f380caf80 2024-09-21T07:35:29.780 INFO:teuthology.orchestra.run.smithi039.stderr:Error EPERM: WARNING: this will *PERMANENTLY DESTROY* all data stored in pool test_metadata_pool. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. 2024-09-21T07:35:29.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:35:29.798 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:35:29.799 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:35:29.883 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:35:29.980 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:35:29.981 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:35:29.981 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:35:29.981 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:35:30.017 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:35:30.017 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:35:30.092 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:35:30.130 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:35:30.139 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:35:30.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.360+0000 7fb6f6e00640 1 -- 172.21.15.39:0/36271316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6d4004d40 msgr2=0x7fb6d4005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:30.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.360+0000 7fb6f6e00640 1 --2- 172.21.15.39:0/36271316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6d4004d40 0x7fb6d4005150 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8006fd0 tx=0x7fb6d4005830 comp rx=0 tx=0).stop 2024-09-21T07:35:30.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.360+0000 7fb6f4e00640 1 -- 172.21.15.39:0/36271316 reap_dead start 2024-09-21T07:35:30.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.360+0000 7fb6f6e00640 1 -- 172.21.15.39:0/36271316 shutdown_connections 2024-09-21T07:35:30.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.360+0000 7fb6f6e00640 1 -- 172.21.15.39:0/36271316 >> 172.21.15.39:0/36271316 conn(0x7fb6f00b1cb0 msgr2=0x7fb6f00b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:30.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.360+0000 7fb6f6e00640 1 -- 172.21.15.39:0/36271316 shutdown_connections 2024-09-21T07:35:30.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.360+0000 7fb6f6e00640 1 -- 172.21.15.39:0/36271316 wait complete. 2024-09-21T07:35:30.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.363+0000 7fb6f6e00640 1 Processor -- start 2024-09-21T07:35:30.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.363+0000 7fb6f6e00640 1 -- start start 2024-09-21T07:35:30.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.364+0000 7fb6f6e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6d4004d40 0x7fb6f01f66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:30.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.364+0000 7fb6f6e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb6f0157720 0x7fb6f0155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:30.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.364+0000 7fb6f6e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6f009d470 0x7fb6f01597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:30.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.364+0000 7fb6f6e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f006fa40 con 0x7fb6f009d470 2024-09-21T07:35:30.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.364+0000 7fb6f6e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f006f8c0 con 0x7fb6d4004d40 2024-09-21T07:35:30.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.364+0000 7fb6f6e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb6f006fbc0 con 0x7fb6f0157720 2024-09-21T07:35:30.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.365+0000 7fb6f4e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6f009d470 0x7fb6f01597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:30.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.365+0000 7fb6f4e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6f009d470 0x7fb6f01597f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39598/0 (socket says 172.21.15.39:39598) 2024-09-21T07:35:30.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.365+0000 7fb6f4e00640 1 -- 172.21.15.39:0/3986942214 learned_addr learned my addr 172.21.15.39:0/3986942214 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:30.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.365+0000 7fb6efe00640 1 --2- 172.21.15.39:0/3986942214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6d4004d40 0x7fb6f01f66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:30.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.365+0000 7fb6e7400640 1 --2- 172.21.15.39:0/3986942214 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb6f0157720 0x7fb6f0155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:30.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.365+0000 7fb6f4e00640 1 -- 172.21.15.39:0/3986942214 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb6f0157720 msgr2=0x7fb6f0155fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:30.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.365+0000 7fb6f4e00640 1 --2- 172.21.15.39:0/3986942214 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb6f0157720 0x7fb6f0155fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:30.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.365+0000 7fb6f4e00640 1 -- 172.21.15.39:0/3986942214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6d4004d40 msgr2=0x7fb6f01f66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:30.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.365+0000 7fb6f4e00640 1 --2- 172.21.15.39:0/3986942214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6d4004d40 0x7fb6f01f66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:30.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.365+0000 7fb6f4e00640 1 -- 172.21.15.39:0/3986942214 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f00d0a70 con 0x7fb6f009d470 2024-09-21T07:35:30.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.365+0000 7fb6f4e00640 1 --2- 172.21.15.39:0/3986942214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6f009d470 0x7fb6f01597f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800c6d0 tx=0x7fb6e8004bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:30.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.366+0000 7fb6ecc00640 1 -- 172.21.15.39:0/3986942214 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e802bbf0 con 0x7fb6f009d470 2024-09-21T07:35:30.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.366+0000 7fb6ecc00640 1 -- 172.21.15.39:0/3986942214 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb6e8006070 con 0x7fb6f009d470 2024-09-21T07:35:30.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.366+0000 7fb6ecc00640 1 -- 172.21.15.39:0/3986942214 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8006480 con 0x7fb6f009d470 2024-09-21T07:35:30.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.366+0000 7fb6f6e00640 1 -- 172.21.15.39:0/3986942214 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6f01f61d0 con 0x7fb6f009d470 2024-09-21T07:35:30.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.366+0000 7fb6f6e00640 1 -- 172.21.15.39:0/3986942214 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6f01592d0 con 0x7fb6f009d470 2024-09-21T07:35:30.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.367+0000 7fb6f6e00640 1 -- 172.21.15.39:0/3986942214 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6f00d0d10 con 0x7fb6f009d470 2024-09-21T07:35:30.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.368+0000 7fb6ecc00640 1 -- 172.21.15.39:0/3986942214 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb6e8006620 con 0x7fb6f009d470 2024-09-21T07:35:30.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.369+0000 7fb6ecc00640 1 --2- 172.21.15.39:0/3986942214 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb6cc078b90 0x7fb6cc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:30.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.369+0000 7fb6ecc00640 1 -- 172.21.15.39:0/3986942214 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6828+0+0 (secure 0 0 0) 0x7fb6e80bb590 con 0x7fb6f009d470 2024-09-21T07:35:30.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.370+0000 7fb6efe00640 1 --2- 172.21.15.39:0/3986942214 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb6cc078b90 0x7fb6cc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:30.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.371+0000 7fb6efe00640 1 --2- 172.21.15.39:0/3986942214 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb6cc078b90 0x7fb6cc07b0a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0009df0 tx=0x7fb6e0005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:30.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.374+0000 7fb6ecc00640 1 -- 172.21.15.39:0/3986942214 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb6e8083630 con 0x7fb6f009d470 2024-09-21T07:35:30.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.583+0000 7fb6f6e00640 1 -- 172.21.15.39:0/3986942214 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb6f0158de0 con 0x7fb6f009d470 2024-09-21T07:35:30.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.584+0000 7fb6ecc00640 1 -- 172.21.15.39:0/3986942214 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v66) ==== 80+0+0 (secure 0 0 0) 0x7fb6e8082f30 con 0x7fb6f009d470 2024-09-21T07:35:30.586 INFO:teuthology.orchestra.run.smithi039.stderr:listed 0 entries 2024-09-21T07:35:30.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.586+0000 7fb6f6e00640 1 -- 172.21.15.39:0/3986942214 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb6cc078b90 msgr2=0x7fb6cc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:30.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.586+0000 7fb6f6e00640 1 --2- 172.21.15.39:0/3986942214 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb6cc078b90 0x7fb6cc07b0a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0009df0 tx=0x7fb6e0005f90 comp rx=0 tx=0).stop 2024-09-21T07:35:30.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.586+0000 7fb6f6e00640 1 -- 172.21.15.39:0/3986942214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6f009d470 msgr2=0x7fb6f01597f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:30.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.586+0000 7fb6f6e00640 1 --2- 172.21.15.39:0/3986942214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6f009d470 0x7fb6f01597f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800c6d0 tx=0x7fb6e8004bb0 comp rx=0 tx=0).stop 2024-09-21T07:35:30.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.586+0000 7fb6f6e00640 1 -- 172.21.15.39:0/3986942214 shutdown_connections 2024-09-21T07:35:30.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.586+0000 7fb6f6e00640 1 --2- 172.21.15.39:0/3986942214 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb6cc078b90 0x7fb6cc07b0a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:30.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.586+0000 7fb6f6e00640 1 --2- 172.21.15.39:0/3986942214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6f009d470 0x7fb6f01597f0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:30.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.586+0000 7fb6f6e00640 1 --2- 172.21.15.39:0/3986942214 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb6f0157720 0x7fb6f0155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:30.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.586+0000 7fb6f6e00640 1 --2- 172.21.15.39:0/3986942214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6d4004d40 0x7fb6f01f66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:30.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.586+0000 7fb6f6e00640 1 -- 172.21.15.39:0/3986942214 >> 172.21.15.39:0/3986942214 conn(0x7fb6f00b1cb0 msgr2=0x7fb6f00ae510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:30.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.586+0000 7fb6f6e00640 1 -- 172.21.15.39:0/3986942214 shutdown_connections 2024-09-21T07:35:30.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:30.586+0000 7fb6f6e00640 1 -- 172.21.15.39:0/3986942214 wait complete. 2024-09-21T07:35:30.613 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T07:35:30.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:35:30.614 DEBUG:teuthology.orchestra.run.smithi039:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T07:35:30.676 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T07:35:30.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:35:30.677 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:35:30.712 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:35:30.712 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:35:30.713 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:35:30.751 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:35:30.751 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:35:30.751 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:35:30.751 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:35:30.788 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:35:30.788 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:35:30.788 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:35:30.788 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:35:30.827 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:35:30.828 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:35:31.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.054+0000 7f1779400640 1 -- 172.21.15.39:0/312395637 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17740d0260 msgr2=0x7f1774161340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:31.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.054+0000 7f1779400640 1 --2- 172.21.15.39:0/312395637 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17740d0260 0x7f1774161340 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f176c005fd0 tx=0x7f176c02aa90 comp rx=0 tx=0).stop 2024-09-21T07:35:31.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.054+0000 7f1779400640 1 -- 172.21.15.39:0/312395637 shutdown_connections 2024-09-21T07:35:31.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.054+0000 7f1779400640 1 --2- 172.21.15.39:0/312395637 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17740d0260 0x7f1774161340 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:31.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.054+0000 7f1779400640 1 --2- 172.21.15.39:0/312395637 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f17740d2af0 0x7f17740b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:31.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.054+0000 7f1779400640 1 -- 172.21.15.39:0/312395637 >> 172.21.15.39:0/312395637 conn(0x7f177409d690 msgr2=0x7f17740d4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:31.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.054+0000 7f1779400640 1 -- 172.21.15.39:0/312395637 shutdown_connections 2024-09-21T07:35:31.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.054+0000 7f1779400640 1 -- 172.21.15.39:0/312395637 wait complete. 2024-09-21T07:35:31.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.056+0000 7f1779400640 1 Processor -- start 2024-09-21T07:35:31.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.057+0000 7f1779400640 1 -- start start 2024-09-21T07:35:31.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.058+0000 7f1779400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17740b3b60 0x7f17740cce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:31.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.058+0000 7f1779400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17740d0260 0x7f177415aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:31.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.058+0000 7f1779400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f17740d2af0 0x7f17741f5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:31.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.058+0000 7f1779400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1774161880 con 0x7f17740b3b60 2024-09-21T07:35:31.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.058+0000 7f1779400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f17740d0880 con 0x7f17740d0260 2024-09-21T07:35:31.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.058+0000 7f1779400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f1774161a00 con 0x7f17740d2af0 2024-09-21T07:35:31.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.058+0000 7f1773e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17740b3b60 0x7f17740cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:31.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.058+0000 7f1773e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17740b3b60 0x7f17740cce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39616/0 (socket says 172.21.15.39:39616) 2024-09-21T07:35:31.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.058+0000 7f1773e00640 1 -- 172.21.15.39:0/937370180 learned_addr learned my addr 172.21.15.39:0/937370180 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:31.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.059+0000 7f176b400640 1 --2- 172.21.15.39:0/937370180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17740d0260 0x7f177415aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:31.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.059+0000 7f1773e00640 1 -- 172.21.15.39:0/937370180 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f17740d2af0 msgr2=0x7f17741f5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:31.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.059+0000 7f1773e00640 1 --2- 172.21.15.39:0/937370180 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f17740d2af0 0x7f17741f5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:31.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.059+0000 7f1773e00640 1 -- 172.21.15.39:0/937370180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17740d0260 msgr2=0x7f177415aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:31.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.059+0000 7f1773e00640 1 --2- 172.21.15.39:0/937370180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17740d0260 0x7f177415aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:31.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.059+0000 7f1773e00640 1 -- 172.21.15.39:0/937370180 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17741f6a10 con 0x7f17740b3b60 2024-09-21T07:35:31.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.059+0000 7f1773e00640 1 --2- 172.21.15.39:0/937370180 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17740b3b60 0x7f17740cce10 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f176400e790 tx=0x7f176400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:31.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.059+0000 7f1770c00640 1 -- 172.21.15.39:0/937370180 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176400eea0 con 0x7f17740b3b60 2024-09-21T07:35:31.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.059+0000 7f1770c00640 1 -- 172.21.15.39:0/937370180 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f176400cdb0 con 0x7f17740b3b60 2024-09-21T07:35:31.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.059+0000 7f1770c00640 1 -- 172.21.15.39:0/937370180 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1764010690 con 0x7f17740b3b60 2024-09-21T07:35:31.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.059+0000 7f1779400640 1 -- 172.21.15.39:0/937370180 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17740cc330 con 0x7f17740b3b60 2024-09-21T07:35:31.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.059+0000 7f1779400640 1 -- 172.21.15.39:0/937370180 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17740cc8f0 con 0x7f17740b3b60 2024-09-21T07:35:31.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.060+0000 7f1779400640 1 -- 172.21.15.39:0/937370180 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17740b5560 con 0x7f17740b3b60 2024-09-21T07:35:31.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.062+0000 7f1770c00640 1 -- 172.21.15.39:0/937370180 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1764010830 con 0x7f17740b3b60 2024-09-21T07:35:31.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.063+0000 7f1770c00640 1 --2- 172.21.15.39:0/937370180 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1750078b90 0x7f175007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:31.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.063+0000 7f1770c00640 1 -- 172.21.15.39:0/937370180 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6828+0+0 (secure 0 0 0) 0x7f176409bba0 con 0x7f17740b3b60 2024-09-21T07:35:31.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.063+0000 7f176b400640 1 --2- 172.21.15.39:0/937370180 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1750078b90 0x7f175007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:31.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.064+0000 7f176b400640 1 --2- 172.21.15.39:0/937370180 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1750078b90 0x7f175007b0a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f17740cf280 tx=0x7f175c009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:31.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.067+0000 7f1770c00640 1 -- 172.21.15.39:0/937370180 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1764063b70 con 0x7f17740b3b60 2024-09-21T07:35:31.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.301+0000 7f1779400640 1 -- 172.21.15.39:0/937370180 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17740ce7a0 con 0x7f17740b3b60 2024-09-21T07:35:31.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.302+0000 7f1770c00640 1 -- 172.21.15.39:0/937370180 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5224 (secure 0 0 0) 0x7f1764063470 con 0x7f17740b3b60 2024-09-21T07:35:31.304 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:35:31.304 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":35,"btime":"2024-09-21T07:35:19:788250+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5183,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2219254706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2219254706},{"type":"v1","addr":"172.21.15.148:6836","nonce":2219254706}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5189,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3512870782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3512870782},{"type":"v1","addr":"172.21.15.148:6837","nonce":3512870782}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5551,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/761690784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":761690784},{"type":"v1","addr":"172.21.15.39:6835","nonce":761690784}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:35:16.270342+0000","modified":"2024-09-21T07:35:19.299418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5646},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5646":{"gid":5646,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/252473586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":252473586},{"type":"v1","addr":"172.21.15.39:6834","nonce":252473586}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5646,"qdb_cluster":[5646]},"id":4}]} 2024-09-21T07:35:31.304 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 35 2024-09-21T07:35:31.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.305+0000 7f1779400640 1 -- 172.21.15.39:0/937370180 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1750078b90 msgr2=0x7f175007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:31.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.305+0000 7f1779400640 1 --2- 172.21.15.39:0/937370180 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1750078b90 0x7f175007b0a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f17740cf280 tx=0x7f175c009040 comp rx=0 tx=0).stop 2024-09-21T07:35:31.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.305+0000 7f1779400640 1 -- 172.21.15.39:0/937370180 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17740b3b60 msgr2=0x7f17740cce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:31.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.305+0000 7f1779400640 1 --2- 172.21.15.39:0/937370180 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17740b3b60 0x7f17740cce10 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f176400e790 tx=0x7f176400ec50 comp rx=0 tx=0).stop 2024-09-21T07:35:31.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.305+0000 7f1779400640 1 -- 172.21.15.39:0/937370180 shutdown_connections 2024-09-21T07:35:31.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.305+0000 7f1779400640 1 --2- 172.21.15.39:0/937370180 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1750078b90 0x7f175007b0a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:31.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.305+0000 7f1779400640 1 --2- 172.21.15.39:0/937370180 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f17740d2af0 0x7f17741f5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:31.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.305+0000 7f1779400640 1 --2- 172.21.15.39:0/937370180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17740d0260 0x7f177415aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:31.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.305+0000 7f1779400640 1 --2- 172.21.15.39:0/937370180 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17740b3b60 0x7f17740cce10 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:31.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.305+0000 7f1779400640 1 -- 172.21.15.39:0/937370180 >> 172.21.15.39:0/937370180 conn(0x7f177409d690 msgr2=0x7f17740d1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:31.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.305+0000 7f1779400640 1 -- 172.21.15.39:0/937370180 shutdown_connections 2024-09-21T07:35:31.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.305+0000 7f1779400640 1 -- 172.21.15.39:0/937370180 wait complete. 2024-09-21T07:35:31.333 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:35:31.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.557+0000 7f05d7a00640 1 -- 172.21.15.39:0/3029119296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05d0070030 msgr2=0x7f05d0074640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:31.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.557+0000 7f05d7a00640 1 --2- 172.21.15.39:0/3029119296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05d0070030 0x7f05d0074640 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f05bc005fd0 tx=0x7f05bc02a540 comp rx=0 tx=0).stop 2024-09-21T07:35:31.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.557+0000 7f05d7a00640 1 -- 172.21.15.39:0/3029119296 shutdown_connections 2024-09-21T07:35:31.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.558+0000 7f05d7a00640 1 --2- 172.21.15.39:0/3029119296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05d0070030 0x7f05d0074640 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:31.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.558+0000 7f05d7a00640 1 --2- 172.21.15.39:0/3029119296 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f05d0073170 0x7f05d015c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:31.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.558+0000 7f05d7a00640 1 -- 172.21.15.39:0/3029119296 >> 172.21.15.39:0/3029119296 conn(0x7f05d00b1cb0 msgr2=0x7f05d00b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:31.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.558+0000 7f05d7a00640 1 -- 172.21.15.39:0/3029119296 shutdown_connections 2024-09-21T07:35:31.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.558+0000 7f05d7a00640 1 -- 172.21.15.39:0/3029119296 wait complete. 2024-09-21T07:35:31.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.560+0000 7f05d7a00640 1 Processor -- start 2024-09-21T07:35:31.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.560+0000 7f05d7a00640 1 -- start start 2024-09-21T07:35:31.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.562+0000 7f05d7a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f05d0070030 0x7f05d015a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:31.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.562+0000 7f05d7a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05d0070440 0x7f05d009cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:31.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.562+0000 7f05d7a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05d0073170 0x7f05d009d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:31.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.562+0000 7f05d7a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f05d006f5a0 con 0x7f05d0073170 2024-09-21T07:35:31.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.562+0000 7f05d7a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f05d0074f80 con 0x7f05d0070440 2024-09-21T07:35:31.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.562+0000 7f05d7a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f05d006f720 con 0x7f05d0070030 2024-09-21T07:35:31.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.562+0000 7f05d5a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05d0073170 0x7f05d009d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:31.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.562+0000 7f05d5000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f05d0070030 0x7f05d015a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:31.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.562+0000 7f05cfe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05d0070440 0x7f05d009cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:31.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.562+0000 7f05d5a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05d0073170 0x7f05d009d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39646/0 (socket says 172.21.15.39:39646) 2024-09-21T07:35:31.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.562+0000 7f05d5a00640 1 -- 172.21.15.39:0/1404723100 learned_addr learned my addr 172.21.15.39:0/1404723100 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:31.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.563+0000 7f05d5000640 1 -- 172.21.15.39:0/1404723100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05d0070440 msgr2=0x7f05d009cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:31.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.563+0000 7f05d5000640 1 --2- 172.21.15.39:0/1404723100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05d0070440 0x7f05d009cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:31.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.563+0000 7f05d5000640 1 -- 172.21.15.39:0/1404723100 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05d0073170 msgr2=0x7f05d009d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:31.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.563+0000 7f05d5000640 1 --2- 172.21.15.39:0/1404723100 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05d0073170 0x7f05d009d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:31.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.563+0000 7f05d5000640 1 -- 172.21.15.39:0/1404723100 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05d00c3fd0 con 0x7f05d0070030 2024-09-21T07:35:31.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.563+0000 7f05d5a00640 1 --2- 172.21.15.39:0/1404723100 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05d0073170 0x7f05d009d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:35:31.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.563+0000 7f05d5000640 1 --2- 172.21.15.39:0/1404723100 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f05d0070030 0x7f05d015a760 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f05c400e6d0 tx=0x7f05c400eb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:31.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.563+0000 7f05ccc00640 1 -- 172.21.15.39:0/1404723100 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c400cab0 con 0x7f05d0070030 2024-09-21T07:35:31.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.563+0000 7f05ccc00640 1 -- 172.21.15.39:0/1404723100 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05c400cd40 con 0x7f05d0070030 2024-09-21T07:35:31.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.563+0000 7f05ccc00640 1 -- 172.21.15.39:0/1404723100 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c4010690 con 0x7f05d0070030 2024-09-21T07:35:31.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.563+0000 7f05d7a00640 1 -- 172.21.15.39:0/1404723100 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05d00c42b0 con 0x7f05d0070030 2024-09-21T07:35:31.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.563+0000 7f05d7a00640 1 -- 172.21.15.39:0/1404723100 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f05d00c4870 con 0x7f05d0070030 2024-09-21T07:35:31.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.564+0000 7f05d7a00640 1 -- 172.21.15.39:0/1404723100 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05d006fc40 con 0x7f05d0070030 2024-09-21T07:35:31.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.566+0000 7f05ccc00640 1 -- 172.21.15.39:0/1404723100 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f05c4010830 con 0x7f05d0070030 2024-09-21T07:35:31.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.567+0000 7f05ccc00640 1 --2- 172.21.15.39:0/1404723100 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f05b0078bc0 0x7f05b007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:31.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.567+0000 7f05ccc00640 1 -- 172.21.15.39:0/1404723100 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(66..66 src has 1..66) ==== 6828+0+0 (secure 0 0 0) 0x7f05c409bbd0 con 0x7f05d0070030 2024-09-21T07:35:31.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.567+0000 7f05cfe00640 1 --2- 172.21.15.39:0/1404723100 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f05b0078bc0 0x7f05b007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:31.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.568+0000 7f05cfe00640 1 --2- 172.21.15.39:0/1404723100 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f05b0078bc0 0x7f05b007b0d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f05bc033000 tx=0x7f05bc004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:31.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.571+0000 7f05ccc00640 1 -- 172.21.15.39:0/1404723100 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f05c40635e0 con 0x7f05d0070030 2024-09-21T07:35:31.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.809+0000 7f05d7a00640 1 -- 172.21.15.39:0/1404723100 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05d009e0f0 con 0x7f05d0070030 2024-09-21T07:35:31.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.810+0000 7f05ccc00640 1 -- 172.21.15.39:0/1404723100 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5224 (secure 0 0 0) 0x7f05c4068010 con 0x7f05d0070030 2024-09-21T07:35:31.812 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:35:31.812 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":35,"btime":"2024-09-21T07:35:19:788250+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5183,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2219254706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2219254706},{"type":"v1","addr":"172.21.15.148:6836","nonce":2219254706}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5189,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3512870782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3512870782},{"type":"v1","addr":"172.21.15.148:6837","nonce":3512870782}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5551,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/761690784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":761690784},{"type":"v1","addr":"172.21.15.39:6835","nonce":761690784}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:35:16.270342+0000","modified":"2024-09-21T07:35:19.299418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5646},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5646":{"gid":5646,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/252473586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":252473586},{"type":"v1","addr":"172.21.15.39:6834","nonce":252473586}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5646,"qdb_cluster":[5646]},"id":4}]} 2024-09-21T07:35:31.812 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 35 2024-09-21T07:35:31.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.812+0000 7f05d7a00640 1 -- 172.21.15.39:0/1404723100 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f05b0078bc0 msgr2=0x7f05b007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:31.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.812+0000 7f05d7a00640 1 --2- 172.21.15.39:0/1404723100 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f05b0078bc0 0x7f05b007b0d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f05bc033000 tx=0x7f05bc004730 comp rx=0 tx=0).stop 2024-09-21T07:35:31.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.812+0000 7f05d7a00640 1 -- 172.21.15.39:0/1404723100 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f05d0070030 msgr2=0x7f05d015a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:31.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.812+0000 7f05d7a00640 1 --2- 172.21.15.39:0/1404723100 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f05d0070030 0x7f05d015a760 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f05c400e6d0 tx=0x7f05c400eb90 comp rx=0 tx=0).stop 2024-09-21T07:35:31.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.812+0000 7f05d7a00640 1 -- 172.21.15.39:0/1404723100 shutdown_connections 2024-09-21T07:35:31.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.812+0000 7f05d7a00640 1 --2- 172.21.15.39:0/1404723100 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f05b0078bc0 0x7f05b007b0d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:31.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.812+0000 7f05d7a00640 1 --2- 172.21.15.39:0/1404723100 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05d0073170 0x7f05d009d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:31.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.812+0000 7f05d7a00640 1 --2- 172.21.15.39:0/1404723100 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05d0070440 0x7f05d009cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:31.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.812+0000 7f05d7a00640 1 --2- 172.21.15.39:0/1404723100 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f05d0070030 0x7f05d015a760 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:31.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.812+0000 7f05d7a00640 1 -- 172.21.15.39:0/1404723100 >> 172.21.15.39:0/1404723100 conn(0x7f05d00b1cb0 msgr2=0x7f05d00ae2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:31.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.812+0000 7f05d7a00640 1 -- 172.21.15.39:0/1404723100 shutdown_connections 2024-09-21T07:35:31.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:31.813+0000 7f05d7a00640 1 -- 172.21.15.39:0/1404723100 wait complete. 2024-09-21T07:35:31.840 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:35:32.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.064+0000 7f5cf2600640 1 -- 172.21.15.39:0/3902702666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cec15d8a0 msgr2=0x7f5cec0b40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:32.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.064+0000 7f5cf2600640 1 --2- 172.21.15.39:0/3902702666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cec15d8a0 0x7f5cec0b40b0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5cec071d60 tx=0x7f5cd802a560 comp rx=0 tx=0).stop 2024-09-21T07:35:32.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.065+0000 7f5cf2600640 1 -- 172.21.15.39:0/3902702666 shutdown_connections 2024-09-21T07:35:32.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.065+0000 7f5cf2600640 1 --2- 172.21.15.39:0/3902702666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cec15d8a0 0x7f5cec0b40b0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:32.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.065+0000 7f5cf2600640 1 --2- 172.21.15.39:0/3902702666 >> v2:172.21.15.148:3300/0 conn(0x7f5cec0cc1e0 0x7f5cec0af3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:32.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.065+0000 7f5cf2600640 1 -- 172.21.15.39:0/3902702666 >> 172.21.15.39:0/3902702666 conn(0x7f5cec06d7d0 msgr2=0x7f5cec0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:32.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.065+0000 7f5cf2600640 1 -- 172.21.15.39:0/3902702666 shutdown_connections 2024-09-21T07:35:32.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.065+0000 7f5cf2600640 1 -- 172.21.15.39:0/3902702666 wait complete. 2024-09-21T07:35:32.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.067+0000 7f5cf2600640 1 Processor -- start 2024-09-21T07:35:32.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.067+0000 7f5cf2600640 1 -- start start 2024-09-21T07:35:32.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.068+0000 7f5cf2600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cec0cc1e0 0x7f5cec148be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:32.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.069+0000 7f5cf2600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5cec0ccb50 0x7f5cec15b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:32.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.069+0000 7f5cf2600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cec15d8a0 0x7f5cec15be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:32.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.069+0000 7f5cf2600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5cec153520 con 0x7f5cec0cc1e0 2024-09-21T07:35:32.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.069+0000 7f5cf2600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5cec1533a0 con 0x7f5cec15d8a0 2024-09-21T07:35:32.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.069+0000 7f5cf2600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f5cec1536a0 con 0x7f5cec0ccb50 2024-09-21T07:35:32.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.069+0000 7f5ceb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cec0cc1e0 0x7f5cec148be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:32.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.069+0000 7f5ceb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cec0cc1e0 0x7f5cec148be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36054/0 (socket says 172.21.15.39:36054) 2024-09-21T07:35:32.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.069+0000 7f5ceb400640 1 -- 172.21.15.39:0/611302688 learned_addr learned my addr 172.21.15.39:0/611302688 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:32.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.069+0000 7f5cebe00640 1 --2- 172.21.15.39:0/611302688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cec15d8a0 0x7f5cec15be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:32.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.069+0000 7f5ceaa00640 1 --2- 172.21.15.39:0/611302688 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5cec0ccb50 0x7f5cec15b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:32.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.069+0000 7f5ceb400640 1 -- 172.21.15.39:0/611302688 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5cec0ccb50 msgr2=0x7f5cec15b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:32.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.069+0000 7f5ceb400640 1 --2- 172.21.15.39:0/611302688 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5cec0ccb50 0x7f5cec15b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:32.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.069+0000 7f5ceb400640 1 -- 172.21.15.39:0/611302688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cec15d8a0 msgr2=0x7f5cec15be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:32.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.069+0000 7f5ceb400640 1 --2- 172.21.15.39:0/611302688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cec15d8a0 0x7f5cec15be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:32.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.069+0000 7f5ceb400640 1 -- 172.21.15.39:0/611302688 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cec15e440 con 0x7f5cec0cc1e0 2024-09-21T07:35:32.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.069+0000 7f5ceaa00640 1 --2- 172.21.15.39:0/611302688 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5cec0ccb50 0x7f5cec15b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:35:32.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.069+0000 7f5cebe00640 1 --2- 172.21.15.39:0/611302688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cec15d8a0 0x7f5cec15be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:35:32.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.070+0000 7f5ceb400640 1 --2- 172.21.15.39:0/611302688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cec0cc1e0 0x7f5cec148be0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc00b440 tx=0x7f5cdc00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:32.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.070+0000 7f5ce3400640 1 -- 172.21.15.39:0/611302688 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc015020 con 0x7f5cec0cc1e0 2024-09-21T07:35:32.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.070+0000 7f5ce3400640 1 -- 172.21.15.39:0/611302688 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5cdc011980 con 0x7f5cec0cc1e0 2024-09-21T07:35:32.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.070+0000 7f5cf2600640 1 -- 172.21.15.39:0/611302688 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cec15e720 con 0x7f5cec0cc1e0 2024-09-21T07:35:32.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.070+0000 7f5ce3400640 1 -- 172.21.15.39:0/611302688 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc011dd0 con 0x7f5cec0cc1e0 2024-09-21T07:35:32.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.070+0000 7f5cf2600640 1 -- 172.21.15.39:0/611302688 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cec15ece0 con 0x7f5cec0cc1e0 2024-09-21T07:35:32.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.071+0000 7f5cf2600640 1 -- 172.21.15.39:0/611302688 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cec153b90 con 0x7f5cec0cc1e0 2024-09-21T07:35:32.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.072+0000 7f5ce3400640 1 -- 172.21.15.39:0/611302688 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5cdc018020 con 0x7f5cec0cc1e0 2024-09-21T07:35:32.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.073+0000 7f5ce3400640 1 --2- 172.21.15.39:0/611302688 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5ccc078b90 0x7f5ccc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:32.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.073+0000 7f5ce3400640 1 -- 172.21.15.39:0/611302688 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6828+0+0 (secure 0 0 0) 0x7f5cdc09b2f0 con 0x7f5cec0cc1e0 2024-09-21T07:35:32.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.074+0000 7f5ceaa00640 1 --2- 172.21.15.39:0/611302688 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5ccc078b90 0x7f5ccc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:32.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.074+0000 7f5ceaa00640 1 --2- 172.21.15.39:0/611302688 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5ccc078b90 0x7f5ccc07b0a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8033000 tx=0x7f5cd8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:32.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.078+0000 7f5ce3400640 1 -- 172.21.15.39:0/611302688 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5cdc01f020 con 0x7f5cec0cc1e0 2024-09-21T07:35:32.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.277+0000 7f5cf2600640 1 -- 172.21.15.39:0/611302688 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5cec0cd8d0 con 0x7f5cec0cc1e0 2024-09-21T07:35:32.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.279+0000 7f5ce3400640 1 -- 172.21.15.39:0/611302688 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+22174 (secure 0 0 0) 0x7f5cdc00d040 con 0x7f5cec0cc1e0 2024-09-21T07:35:32.281 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:35:32.281 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":66,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:35:28.850747+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:35:14.214410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-21T07:35:15.228034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":11,"snaps":[{"begin":1,"length":1}]}],"new_removed_snaps":[{"pool":11,"snaps":[{"begin":1,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:35:32.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.281+0000 7f5cf2600640 1 -- 172.21.15.39:0/611302688 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5ccc078b90 msgr2=0x7f5ccc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:32.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.281+0000 7f5cf2600640 1 --2- 172.21.15.39:0/611302688 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5ccc078b90 0x7f5ccc07b0a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8033000 tx=0x7f5cd8004730 comp rx=0 tx=0).stop 2024-09-21T07:35:32.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.281+0000 7f5cf2600640 1 -- 172.21.15.39:0/611302688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cec0cc1e0 msgr2=0x7f5cec148be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:32.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.281+0000 7f5cf2600640 1 --2- 172.21.15.39:0/611302688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cec0cc1e0 0x7f5cec148be0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc00b440 tx=0x7f5cdc00b900 comp rx=0 tx=0).stop 2024-09-21T07:35:32.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.282+0000 7f5cf2600640 1 -- 172.21.15.39:0/611302688 shutdown_connections 2024-09-21T07:35:32.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.282+0000 7f5cf2600640 1 --2- 172.21.15.39:0/611302688 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5ccc078b90 0x7f5ccc07b0a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:32.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.282+0000 7f5cf2600640 1 --2- 172.21.15.39:0/611302688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cec15d8a0 0x7f5cec15be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:32.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.282+0000 7f5cf2600640 1 --2- 172.21.15.39:0/611302688 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5cec0ccb50 0x7f5cec15b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:32.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.282+0000 7f5cf2600640 1 --2- 172.21.15.39:0/611302688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cec0cc1e0 0x7f5cec148be0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:32.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.282+0000 7f5cf2600640 1 -- 172.21.15.39:0/611302688 >> 172.21.15.39:0/611302688 conn(0x7f5cec06d7d0 msgr2=0x7f5cec0b0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:32.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.282+0000 7f5cf2600640 1 -- 172.21.15.39:0/611302688 shutdown_connections 2024-09-21T07:35:32.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.282+0000 7f5cf2600640 1 -- 172.21.15.39:0/611302688 wait complete. 2024-09-21T07:35:32.306 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T07:35:32.306 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:35:32.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.532+0000 7fe376400640 1 -- 172.21.15.39:0/1750305434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe370070030 msgr2=0x7fe370074640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:32.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.532+0000 7fe376400640 1 --2- 172.21.15.39:0/1750305434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe370070030 0x7fe370074640 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe358009830 tx=0x7fe35802a540 comp rx=0 tx=0).stop 2024-09-21T07:35:32.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.532+0000 7fe376400640 1 -- 172.21.15.39:0/1750305434 shutdown_connections 2024-09-21T07:35:32.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.532+0000 7fe376400640 1 --2- 172.21.15.39:0/1750305434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe370070030 0x7fe370074640 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:32.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.532+0000 7fe376400640 1 -- 172.21.15.39:0/1750305434 >> 172.21.15.39:0/1750305434 conn(0x7fe3700b1cb0 msgr2=0x7fe3700b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:32.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.532+0000 7fe376400640 1 -- 172.21.15.39:0/1750305434 shutdown_connections 2024-09-21T07:35:32.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.532+0000 7fe376400640 1 -- 172.21.15.39:0/1750305434 wait complete. 2024-09-21T07:35:32.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.534+0000 7fe376400640 1 Processor -- start 2024-09-21T07:35:32.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.534+0000 7fe376400640 1 -- start start 2024-09-21T07:35:32.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.536+0000 7fe376400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe370070030 0x7fe3700c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:32.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.536+0000 7fe376400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe370070440 0x7fe3700d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:32.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.536+0000 7fe376400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe370073170 0x7fe37015d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:32.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.536+0000 7fe376400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe3700b3c40 con 0x7fe370070030 2024-09-21T07:35:32.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.536+0000 7fe376400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe3700b3ac0 con 0x7fe370073170 2024-09-21T07:35:32.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.536+0000 7fe376400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe3700b3dc0 con 0x7fe370070440 2024-09-21T07:35:32.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.536+0000 7fe36f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe370070030 0x7fe3700c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:32.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.536+0000 7fe36f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe370070030 0x7fe3700c9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36064/0 (socket says 172.21.15.39:36064) 2024-09-21T07:35:32.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.536+0000 7fe36f400640 1 -- 172.21.15.39:0/2427669059 learned_addr learned my addr 172.21.15.39:0/2427669059 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:32.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.537+0000 7fe367e00640 1 --2- 172.21.15.39:0/2427669059 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe370070440 0x7fe3700d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:32.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.537+0000 7fe36fe00640 1 --2- 172.21.15.39:0/2427669059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe370073170 0x7fe37015d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:32.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.537+0000 7fe36f400640 1 -- 172.21.15.39:0/2427669059 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe370070440 msgr2=0x7fe3700d08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:32.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.537+0000 7fe36f400640 1 --2- 172.21.15.39:0/2427669059 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe370070440 0x7fe3700d08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:32.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.537+0000 7fe36f400640 1 -- 172.21.15.39:0/2427669059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe370073170 msgr2=0x7fe37015d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:32.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.537+0000 7fe36f400640 1 --2- 172.21.15.39:0/2427669059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe370073170 0x7fe37015d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:32.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.537+0000 7fe36f400640 1 -- 172.21.15.39:0/2427669059 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3700cc4b0 con 0x7fe370070030 2024-09-21T07:35:32.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.537+0000 7fe36f400640 1 --2- 172.21.15.39:0/2427669059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe370070030 0x7fe3700c9530 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe36000c400 tx=0x7fe36000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:32.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.537+0000 7fe367400640 1 -- 172.21.15.39:0/2427669059 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe360017070 con 0x7fe370070030 2024-09-21T07:35:32.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.538+0000 7fe367400640 1 -- 172.21.15.39:0/2427669059 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3600053c0 con 0x7fe370070030 2024-09-21T07:35:32.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.538+0000 7fe367400640 1 -- 172.21.15.39:0/2427669059 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe360003850 con 0x7fe370070030 2024-09-21T07:35:32.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.538+0000 7fe376400640 1 -- 172.21.15.39:0/2427669059 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3700cc1d0 con 0x7fe370070030 2024-09-21T07:35:32.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.538+0000 7fe376400640 1 -- 172.21.15.39:0/2427669059 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3701f92f0 con 0x7fe370070030 2024-09-21T07:35:32.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.539+0000 7fe376400640 1 -- 172.21.15.39:0/2427669059 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3700b4320 con 0x7fe370070030 2024-09-21T07:35:32.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.540+0000 7fe367400640 1 -- 172.21.15.39:0/2427669059 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe36001a070 con 0x7fe370070030 2024-09-21T07:35:32.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.542+0000 7fe367400640 1 --2- 172.21.15.39:0/2427669059 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe34c078b90 0x7fe34c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:32.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.542+0000 7fe367400640 1 -- 172.21.15.39:0/2427669059 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6828+0+0 (secure 0 0 0) 0x7fe36009a470 con 0x7fe370070030 2024-09-21T07:35:32.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.542+0000 7fe367e00640 1 --2- 172.21.15.39:0/2427669059 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe34c078b90 0x7fe34c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:32.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.543+0000 7fe367e00640 1 --2- 172.21.15.39:0/2427669059 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe34c078b90 0x7fe34c07b0a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe358034000 tx=0x7fe358004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:32.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.547+0000 7fe367400640 1 -- 172.21.15.39:0/2427669059 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe360061e80 con 0x7fe370070030 2024-09-21T07:35:32.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.787+0000 7fe376400640 1 -- 172.21.15.39:0/2427669059 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3700c3860 con 0x7fe370070030 2024-09-21T07:35:32.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.788+0000 7fe367400640 1 -- 172.21.15.39:0/2427669059 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5224 (secure 0 0 0) 0x7fe3600668b0 con 0x7fe370070030 2024-09-21T07:35:32.790 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:35:32.790 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":35,"btime":"2024-09-21T07:35:19:788250+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5183,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2219254706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2219254706},{"type":"v1","addr":"172.21.15.148:6836","nonce":2219254706}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5189,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3512870782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3512870782},{"type":"v1","addr":"172.21.15.148:6837","nonce":3512870782}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5551,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/761690784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":761690784},{"type":"v1","addr":"172.21.15.39:6835","nonce":761690784}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:35:16.270342+0000","modified":"2024-09-21T07:35:19.299418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5646},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5646":{"gid":5646,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/252473586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":252473586},{"type":"v1","addr":"172.21.15.39:6834","nonce":252473586}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5646,"qdb_cluster":[5646]},"id":4}]} 2024-09-21T07:35:32.790 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 35 2024-09-21T07:35:32.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.791+0000 7fe376400640 1 -- 172.21.15.39:0/2427669059 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe34c078b90 msgr2=0x7fe34c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:32.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.791+0000 7fe376400640 1 --2- 172.21.15.39:0/2427669059 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe34c078b90 0x7fe34c07b0a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe358034000 tx=0x7fe358004730 comp rx=0 tx=0).stop 2024-09-21T07:35:32.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.791+0000 7fe376400640 1 -- 172.21.15.39:0/2427669059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe370070030 msgr2=0x7fe3700c9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:32.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.791+0000 7fe376400640 1 --2- 172.21.15.39:0/2427669059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe370070030 0x7fe3700c9530 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe36000c400 tx=0x7fe36000c8c0 comp rx=0 tx=0).stop 2024-09-21T07:35:32.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.791+0000 7fe376400640 1 -- 172.21.15.39:0/2427669059 shutdown_connections 2024-09-21T07:35:32.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.791+0000 7fe376400640 1 --2- 172.21.15.39:0/2427669059 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe34c078b90 0x7fe34c07b0a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:32.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.791+0000 7fe376400640 1 --2- 172.21.15.39:0/2427669059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe370073170 0x7fe37015d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:32.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.791+0000 7fe376400640 1 --2- 172.21.15.39:0/2427669059 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe370070440 0x7fe3700d08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:32.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.791+0000 7fe376400640 1 --2- 172.21.15.39:0/2427669059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe370070030 0x7fe3700c9530 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:32.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.791+0000 7fe376400640 1 -- 172.21.15.39:0/2427669059 >> 172.21.15.39:0/2427669059 conn(0x7fe3700b1cb0 msgr2=0x7fe3700b09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:32.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.791+0000 7fe376400640 1 -- 172.21.15.39:0/2427669059 shutdown_connections 2024-09-21T07:35:32.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:32.791+0000 7fe376400640 1 -- 172.21.15.39:0/2427669059 wait complete. 2024-09-21T07:35:32.815 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:35:33.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.041+0000 7f59e3800640 1 -- 172.21.15.39:0/260057010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59c4005150 msgr2=0x7f59c400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:33.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.041+0000 7f59e3800640 1 --2- 172.21.15.39:0/260057010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59c4005150 0x7f59c400b9c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f59dc0afbb0 tx=0x7f59c4005830 comp rx=0 tx=0).stop 2024-09-21T07:35:33.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.041+0000 7f59e1800640 1 -- 172.21.15.39:0/260057010 reap_dead start 2024-09-21T07:35:33.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.042+0000 7f59e3800640 1 -- 172.21.15.39:0/260057010 shutdown_connections 2024-09-21T07:35:33.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.042+0000 7f59e3800640 1 -- 172.21.15.39:0/260057010 >> 172.21.15.39:0/260057010 conn(0x7f59dc06d7d0 msgr2=0x7f59dc0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:33.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.042+0000 7f59e3800640 1 -- 172.21.15.39:0/260057010 shutdown_connections 2024-09-21T07:35:33.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.042+0000 7f59e3800640 1 -- 172.21.15.39:0/260057010 wait complete. 2024-09-21T07:35:33.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.044+0000 7f59e3800640 1 Processor -- start 2024-09-21T07:35:33.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.044+0000 7f59e3800640 1 -- start start 2024-09-21T07:35:33.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.046+0000 7f59e3800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59c4005150 0x7f59dc14bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:33.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.046+0000 7f59e3800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59dc0acf90 0x7f59dc0ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:33.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.046+0000 7f59e3800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59dc1f56e0 0x7f59dc1f5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:33.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.046+0000 7f59e3800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f59dc15dcb0 con 0x7f59c4005150 2024-09-21T07:35:33.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.046+0000 7f59e3800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f59dc0ccf60 con 0x7f59dc1f56e0 2024-09-21T07:35:33.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.046+0000 7f59e3800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f59dc0adf20 con 0x7f59dc0acf90 2024-09-21T07:35:33.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.046+0000 7f59e0e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59c4005150 0x7f59dc14bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:33.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.046+0000 7f59e0e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59c4005150 0x7f59dc14bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36090/0 (socket says 172.21.15.39:36090) 2024-09-21T07:35:33.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.046+0000 7f59e0e00640 1 -- 172.21.15.39:0/588568911 learned_addr learned my addr 172.21.15.39:0/588568911 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:33.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.046+0000 7f59e0e00640 1 -- 172.21.15.39:0/588568911 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59dc0acf90 msgr2=0x7f59dc0ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:33.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.046+0000 7f59dbe00640 1 --2- 172.21.15.39:0/588568911 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59dc0acf90 0x7f59dc0ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:33.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.046+0000 7f59e0e00640 1 --2- 172.21.15.39:0/588568911 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59dc0acf90 0x7f59dc0ad3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:33.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.046+0000 7f59e1800640 1 --2- 172.21.15.39:0/588568911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59dc1f56e0 0x7f59dc1f5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:33.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.046+0000 7f59e0e00640 1 -- 172.21.15.39:0/588568911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59dc1f56e0 msgr2=0x7f59dc1f5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:33.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.046+0000 7f59e0e00640 1 --2- 172.21.15.39:0/588568911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59dc1f56e0 0x7f59dc1f5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:33.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.046+0000 7f59e0e00640 1 -- 172.21.15.39:0/588568911 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59dc14c1a0 con 0x7f59c4005150 2024-09-21T07:35:33.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.046+0000 7f59dbe00640 1 --2- 172.21.15.39:0/588568911 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59dc0acf90 0x7f59dc0ad3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:35:33.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.046+0000 7f59e1800640 1 --2- 172.21.15.39:0/588568911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59dc1f56e0 0x7f59dc1f5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:35:33.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.047+0000 7f59e0e00640 1 --2- 172.21.15.39:0/588568911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59c4005150 0x7f59dc14bc60 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f59d0011ae0 tx=0x7f59d0009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:33.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.047+0000 7f59d8c00640 1 -- 172.21.15.39:0/588568911 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d000fcb0 con 0x7f59c4005150 2024-09-21T07:35:33.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.047+0000 7f59d8c00640 1 -- 172.21.15.39:0/588568911 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f59d000fe50 con 0x7f59c4005150 2024-09-21T07:35:33.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.047+0000 7f59d8c00640 1 -- 172.21.15.39:0/588568911 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d0004280 con 0x7f59c4005150 2024-09-21T07:35:33.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.047+0000 7f59e3800640 1 -- 172.21.15.39:0/588568911 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59dc09cfb0 con 0x7f59c4005150 2024-09-21T07:35:33.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.047+0000 7f59e3800640 1 -- 172.21.15.39:0/588568911 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59dc09d570 con 0x7f59c4005150 2024-09-21T07:35:33.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.048+0000 7f59e3800640 1 -- 172.21.15.39:0/588568911 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59dc0af3c0 con 0x7f59c4005150 2024-09-21T07:35:33.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.049+0000 7f59d8c00640 1 -- 172.21.15.39:0/588568911 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f59d00035f0 con 0x7f59c4005150 2024-09-21T07:35:33.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.050+0000 7f59d8c00640 1 --2- 172.21.15.39:0/588568911 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f59bc078b90 0x7f59bc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:33.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.050+0000 7f59d8c00640 1 -- 172.21.15.39:0/588568911 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6828+0+0 (secure 0 0 0) 0x7f59d009bc80 con 0x7f59c4005150 2024-09-21T07:35:33.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.051+0000 7f59dbe00640 1 --2- 172.21.15.39:0/588568911 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f59bc078b90 0x7f59bc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:33.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.052+0000 7f59dbe00640 1 --2- 172.21.15.39:0/588568911 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f59bc078b90 0x7f59bc07b0a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f59c8001a30 tx=0x7f59c8001130 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:33.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.055+0000 7f59d8c00640 1 -- 172.21.15.39:0/588568911 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f59d0063cf0 con 0x7f59c4005150 2024-09-21T07:35:33.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.289+0000 7f59e3800640 1 -- 172.21.15.39:0/588568911 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59dc15c4a0 con 0x7f59c4005150 2024-09-21T07:35:33.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.289+0000 7f59d8c00640 1 -- 172.21.15.39:0/588568911 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5224 (secure 0 0 0) 0x7f59d00635f0 con 0x7f59c4005150 2024-09-21T07:35:33.291 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:35:33.292 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":35,"btime":"2024-09-21T07:35:19:788250+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5183,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2219254706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2219254706},{"type":"v1","addr":"172.21.15.148:6836","nonce":2219254706}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5189,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3512870782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3512870782},{"type":"v1","addr":"172.21.15.148:6837","nonce":3512870782}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5551,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/761690784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":761690784},{"type":"v1","addr":"172.21.15.39:6835","nonce":761690784}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:35:16.270342+0000","modified":"2024-09-21T07:35:19.299418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5646},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5646":{"gid":5646,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/252473586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":252473586},{"type":"v1","addr":"172.21.15.39:6834","nonce":252473586}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5646,"qdb_cluster":[5646]},"id":4}]} 2024-09-21T07:35:33.292 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 35 2024-09-21T07:35:33.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.292+0000 7f59e3800640 1 -- 172.21.15.39:0/588568911 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f59bc078b90 msgr2=0x7f59bc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:33.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.292+0000 7f59e3800640 1 --2- 172.21.15.39:0/588568911 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f59bc078b90 0x7f59bc07b0a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f59c8001a30 tx=0x7f59c8001130 comp rx=0 tx=0).stop 2024-09-21T07:35:33.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.292+0000 7f59e3800640 1 -- 172.21.15.39:0/588568911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59c4005150 msgr2=0x7f59dc14bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:33.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.292+0000 7f59e3800640 1 --2- 172.21.15.39:0/588568911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59c4005150 0x7f59dc14bc60 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f59d0011ae0 tx=0x7f59d0009f10 comp rx=0 tx=0).stop 2024-09-21T07:35:33.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.292+0000 7f59e3800640 1 -- 172.21.15.39:0/588568911 shutdown_connections 2024-09-21T07:35:33.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.292+0000 7f59e3800640 1 --2- 172.21.15.39:0/588568911 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f59bc078b90 0x7f59bc07b0a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:33.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.292+0000 7f59e3800640 1 --2- 172.21.15.39:0/588568911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59dc1f56e0 0x7f59dc1f5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:33.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.292+0000 7f59e3800640 1 --2- 172.21.15.39:0/588568911 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59dc0acf90 0x7f59dc0ad3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:33.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.292+0000 7f59e3800640 1 --2- 172.21.15.39:0/588568911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59c4005150 0x7f59dc14bc60 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:33.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.292+0000 7f59e3800640 1 -- 172.21.15.39:0/588568911 >> 172.21.15.39:0/588568911 conn(0x7f59dc06d7d0 msgr2=0x7f59dc15f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:33.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.292+0000 7f59e3800640 1 -- 172.21.15.39:0/588568911 shutdown_connections 2024-09-21T07:35:33.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.292+0000 7f59e3800640 1 -- 172.21.15.39:0/588568911 wait complete. 2024-09-21T07:35:33.319 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:35:33.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.544+0000 7f8692200640 1 -- 172.21.15.39:0/2862571529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8674003a00 msgr2=0x7f8674003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:33.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.544+0000 7f8692200640 1 --2- 172.21.15.39:0/2862571529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8674003a00 0x7f8674003e50 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f868000c770 tx=0x7f8674005830 comp rx=0 tx=0).stop 2024-09-21T07:35:33.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.544+0000 7f8692200640 1 -- 172.21.15.39:0/2862571529 shutdown_connections 2024-09-21T07:35:33.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.544+0000 7f8692200640 1 --2- 172.21.15.39:0/2862571529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8674003a00 0x7f8674003e50 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:33.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.544+0000 7f8692200640 1 -- 172.21.15.39:0/2862571529 >> 172.21.15.39:0/2862571529 conn(0x7f868c09d680 msgr2=0x7f868c0d4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:33.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.544+0000 7f8692200640 1 -- 172.21.15.39:0/2862571529 shutdown_connections 2024-09-21T07:35:33.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.544+0000 7f8692200640 1 -- 172.21.15.39:0/2862571529 wait complete. 2024-09-21T07:35:33.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.546+0000 7f8692200640 1 Processor -- start 2024-09-21T07:35:33.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.546+0000 7f8692200640 1 -- start start 2024-09-21T07:35:33.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.548+0000 7f8692200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8674003a00 0x7f868c1dd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:33.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.548+0000 7f8692200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f868c1f6070 0x7f868c1f8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:33.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.548+0000 7f8692200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f868c1f8eb0 0x7f868c1e50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:33.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.548+0000 7f8692200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f868c0cf3a0 con 0x7f8674003a00 2024-09-21T07:35:33.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.548+0000 7f8692200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f868c0cf220 con 0x7f868c1f6070 2024-09-21T07:35:33.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.548+0000 7f8692200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f868c0cf520 con 0x7f868c1f8eb0 2024-09-21T07:35:33.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.548+0000 7f8690e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8674003a00 0x7f868c1dd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:33.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.548+0000 7f8690e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8674003a00 0x7f868c1dd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36110/0 (socket says 172.21.15.39:36110) 2024-09-21T07:35:33.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.548+0000 7f8690e00640 1 -- 172.21.15.39:0/353723549 learned_addr learned my addr 172.21.15.39:0/353723549 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:33.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.548+0000 7f868be00640 1 --2- 172.21.15.39:0/353723549 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f868c1f6070 0x7f868c1f8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:33.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.548+0000 7f8691800640 1 --2- 172.21.15.39:0/353723549 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f868c1f8eb0 0x7f868c1e50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:33.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.549+0000 7f8690e00640 1 -- 172.21.15.39:0/353723549 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f868c1f8eb0 msgr2=0x7f868c1e50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:33.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.549+0000 7f8690e00640 1 --2- 172.21.15.39:0/353723549 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f868c1f8eb0 0x7f868c1e50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:33.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.549+0000 7f8690e00640 1 -- 172.21.15.39:0/353723549 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f868c1f6070 msgr2=0x7f868c1f8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:33.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.549+0000 7f8690e00640 1 --2- 172.21.15.39:0/353723549 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f868c1f6070 0x7f868c1f8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:33.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.549+0000 7f8690e00640 1 -- 172.21.15.39:0/353723549 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f868c1f9630 con 0x7f8674003a00 2024-09-21T07:35:33.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.549+0000 7f868be00640 1 --2- 172.21.15.39:0/353723549 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f868c1f6070 0x7f868c1f8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:35:33.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.549+0000 7f8690e00640 1 --2- 172.21.15.39:0/353723549 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8674003a00 0x7f868c1dd430 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f868002a840 tx=0x7f8680006270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:33.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.549+0000 7f8688c00640 1 -- 172.21.15.39:0/353723549 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8680004670 con 0x7f8674003a00 2024-09-21T07:35:33.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.549+0000 7f8688c00640 1 -- 172.21.15.39:0/353723549 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8680004810 con 0x7f8674003a00 2024-09-21T07:35:33.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.549+0000 7f8688c00640 1 -- 172.21.15.39:0/353723549 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8680004c40 con 0x7f8674003a00 2024-09-21T07:35:33.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.549+0000 7f8692200640 1 -- 172.21.15.39:0/353723549 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f868c1f5590 con 0x7f8674003a00 2024-09-21T07:35:33.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.550+0000 7f8692200640 1 -- 172.21.15.39:0/353723549 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f868c1dd970 con 0x7f8674003a00 2024-09-21T07:35:33.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.551+0000 7f8692200640 1 -- 172.21.15.39:0/353723549 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f868c1f5380 con 0x7f8674003a00 2024-09-21T07:35:33.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.552+0000 7f8688c00640 1 -- 172.21.15.39:0/353723549 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f868002c110 con 0x7f8674003a00 2024-09-21T07:35:33.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.553+0000 7f8688c00640 1 --2- 172.21.15.39:0/353723549 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f866c078bc0 0x7f866c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:33.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.553+0000 7f8688c00640 1 -- 172.21.15.39:0/353723549 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6828+0+0 (secure 0 0 0) 0x7f86800bad40 con 0x7f8674003a00 2024-09-21T07:35:33.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.553+0000 7f868be00640 1 --2- 172.21.15.39:0/353723549 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f866c078bc0 0x7f866c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:33.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.554+0000 7f868be00640 1 --2- 172.21.15.39:0/353723549 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f866c078bc0 0x7f866c07b0d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f868c1ef9f0 tx=0x7f8678006040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:33.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.557+0000 7f8688c00640 1 -- 172.21.15.39:0/353723549 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f86800826d0 con 0x7f8674003a00 2024-09-21T07:35:33.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.757+0000 7f8692200640 1 -- 172.21.15.39:0/353723549 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f868c0d0ec0 con 0x7f8674003a00 2024-09-21T07:35:33.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.758+0000 7f8688c00640 1 -- 172.21.15.39:0/353723549 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+22174 (secure 0 0 0) 0x7f8680087100 con 0x7f8674003a00 2024-09-21T07:35:33.761 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:35:33.761 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":66,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:35:28.850747+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:35:14.214410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-21T07:35:15.228034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":11,"snaps":[{"begin":1,"length":1}]}],"new_removed_snaps":[{"pool":11,"snaps":[{"begin":1,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:35:33.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.761+0000 7f8692200640 1 -- 172.21.15.39:0/353723549 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f866c078bc0 msgr2=0x7f866c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:33.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.761+0000 7f8692200640 1 --2- 172.21.15.39:0/353723549 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f866c078bc0 0x7f866c07b0d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f868c1ef9f0 tx=0x7f8678006040 comp rx=0 tx=0).stop 2024-09-21T07:35:33.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.761+0000 7f8692200640 1 -- 172.21.15.39:0/353723549 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8674003a00 msgr2=0x7f868c1dd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:33.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.761+0000 7f8692200640 1 --2- 172.21.15.39:0/353723549 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8674003a00 0x7f868c1dd430 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f868002a840 tx=0x7f8680006270 comp rx=0 tx=0).stop 2024-09-21T07:35:33.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.761+0000 7f8692200640 1 -- 172.21.15.39:0/353723549 shutdown_connections 2024-09-21T07:35:33.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.761+0000 7f8692200640 1 --2- 172.21.15.39:0/353723549 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f866c078bc0 0x7f866c07b0d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:33.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.761+0000 7f8692200640 1 --2- 172.21.15.39:0/353723549 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f868c1f8eb0 0x7f868c1e50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:33.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.761+0000 7f8692200640 1 --2- 172.21.15.39:0/353723549 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f868c1f6070 0x7f868c1f8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:33.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.761+0000 7f8692200640 1 --2- 172.21.15.39:0/353723549 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8674003a00 0x7f868c1dd430 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:33.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.761+0000 7f8692200640 1 -- 172.21.15.39:0/353723549 >> 172.21.15.39:0/353723549 conn(0x7f868c09d680 msgr2=0x7f868c1ea9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:33.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.761+0000 7f8692200640 1 -- 172.21.15.39:0/353723549 shutdown_connections 2024-09-21T07:35:33.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:33.761+0000 7f8692200640 1 -- 172.21.15.39:0/353723549 wait complete. 2024-09-21T07:35:33.786 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T07:35:34.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.012+0000 7fadd7a00640 1 -- 172.21.15.39:0/2818967441 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fadd0073170 msgr2=0x7fadd015c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:34.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.012+0000 7fadd7a00640 1 --2- 172.21.15.39:0/2818967441 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fadd0073170 0x7fadd015c780 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fadc4009860 tx=0x7fadc402a510 comp rx=0 tx=0).stop 2024-09-21T07:35:34.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.012+0000 7fadd7a00640 1 -- 172.21.15.39:0/2818967441 shutdown_connections 2024-09-21T07:35:34.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.012+0000 7fadd7a00640 1 --2- 172.21.15.39:0/2818967441 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fadd0073170 0x7fadd015c780 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:34.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.012+0000 7fadd7a00640 1 -- 172.21.15.39:0/2818967441 >> 172.21.15.39:0/2818967441 conn(0x7fadd00b1cb0 msgr2=0x7fadd00b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:34.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.012+0000 7fadd7a00640 1 -- 172.21.15.39:0/2818967441 shutdown_connections 2024-09-21T07:35:34.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.013+0000 7fadd7a00640 1 -- 172.21.15.39:0/2818967441 wait complete. 2024-09-21T07:35:34.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.015+0000 7fadd7a00640 1 Processor -- start 2024-09-21T07:35:34.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.015+0000 7fadd7a00640 1 -- start start 2024-09-21T07:35:34.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.016+0000 7fadd7a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fadd0070030 0x7fadd00ca770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:34.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.016+0000 7fadd7a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadd0070440 0x7fadd00cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:34.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.016+0000 7fadd7a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadd0073170 0x7fadd014e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:34.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.016+0000 7fadd7a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fadd00b3d20 con 0x7fadd0070440 2024-09-21T07:35:34.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.016+0000 7fadd7a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fadd00b3ba0 con 0x7fadd0073170 2024-09-21T07:35:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.016+0000 7fadd7a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fadd00b3ea0 con 0x7fadd0070030 2024-09-21T07:35:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.017+0000 7fadcfe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadd0070440 0x7fadd00cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.017+0000 7fadcfe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadd0070440 0x7fadd00cfa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36112/0 (socket says 172.21.15.39:36112) 2024-09-21T07:35:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.017+0000 7fadcfe00640 1 -- 172.21.15.39:0/2534102617 learned_addr learned my addr 172.21.15.39:0/2534102617 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.017+0000 7fadd5a00640 1 --2- 172.21.15.39:0/2534102617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadd0073170 0x7fadd014e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.017+0000 7fadd5000640 1 --2- 172.21.15.39:0/2534102617 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fadd0070030 0x7fadd00ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.017+0000 7fadcfe00640 1 -- 172.21.15.39:0/2534102617 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fadd0070030 msgr2=0x7fadd00ca770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.017+0000 7fadcfe00640 1 --2- 172.21.15.39:0/2534102617 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fadd0070030 0x7fadd00ca770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.017+0000 7fadcfe00640 1 -- 172.21.15.39:0/2534102617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadd0073170 msgr2=0x7fadd014e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.017+0000 7fadcfe00640 1 --2- 172.21.15.39:0/2534102617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadd0073170 0x7fadd014e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.017+0000 7fadcfe00640 1 -- 172.21.15.39:0/2534102617 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadd00d0bb0 con 0x7fadd0070440 2024-09-21T07:35:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.017+0000 7fadd5000640 1 --2- 172.21.15.39:0/2534102617 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fadd0070030 0x7fadd00ca770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:35:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.017+0000 7fadcfe00640 1 --2- 172.21.15.39:0/2534102617 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadd0070440 0x7fadd00cfa80 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fadbc00c390 tx=0x7fadbc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:34.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.018+0000 7fadccc00640 1 -- 172.21.15.39:0/2534102617 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadbc011790 con 0x7fadd0070440 2024-09-21T07:35:34.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.018+0000 7fadccc00640 1 -- 172.21.15.39:0/2534102617 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fadbc017070 con 0x7fadd0070440 2024-09-21T07:35:34.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.018+0000 7fadccc00640 1 -- 172.21.15.39:0/2534102617 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadbc011cc0 con 0x7fadd0070440 2024-09-21T07:35:34.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.018+0000 7fadd7a00640 1 -- 172.21.15.39:0/2534102617 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadd009aa30 con 0x7fadd0070440 2024-09-21T07:35:34.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.018+0000 7fadd7a00640 1 -- 172.21.15.39:0/2534102617 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadd009c900 con 0x7fadd0070440 2024-09-21T07:35:34.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.019+0000 7fadd7a00640 1 -- 172.21.15.39:0/2534102617 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadd00ad4c0 con 0x7fadd0070440 2024-09-21T07:35:34.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.020+0000 7fadccc00640 1 -- 172.21.15.39:0/2534102617 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fadbc01c460 con 0x7fadd0070440 2024-09-21T07:35:34.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.021+0000 7fadccc00640 1 --2- 172.21.15.39:0/2534102617 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fadb0078bc0 0x7fadb007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:34.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.021+0000 7fadccc00640 1 -- 172.21.15.39:0/2534102617 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6828+0+0 (secure 0 0 0) 0x7fadbc09aef0 con 0x7fadd0070440 2024-09-21T07:35:34.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.022+0000 7fadd5000640 1 --2- 172.21.15.39:0/2534102617 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fadb0078bc0 0x7fadb007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:34.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.023+0000 7fadd5000640 1 --2- 172.21.15.39:0/2534102617 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fadb0078bc0 0x7fadb007b0d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fadc4034000 tx=0x7fadc4004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:34.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.025+0000 7fadccc00640 1 -- 172.21.15.39:0/2534102617 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fadd00ad4c0 con 0x7fadd0070440 2024-09-21T07:35:34.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.260+0000 7fadd7a00640 1 -- 172.21.15.39:0/2534102617 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fadd015c780 con 0x7fadd0070440 2024-09-21T07:35:34.262 INFO:tasks.ceph.mon.a.smithi039.stderr:2024-09-21T07:35:34.261+0000 7fabab600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T07:35:34.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.262+0000 7fadccc00640 1 -- 172.21.15.39:0/2534102617 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v36) ==== 161+0+0 (secure 0 0 0) 0x7fadbc062ec0 con 0x7fadd0070440 2024-09-21T07:35:34.264 INFO:teuthology.orchestra.run.smithi039.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T07:35:34.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.266+0000 7fadd7a00640 1 -- 172.21.15.39:0/2534102617 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fadb0078bc0 msgr2=0x7fadb007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:34.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.266+0000 7fadd7a00640 1 --2- 172.21.15.39:0/2534102617 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fadb0078bc0 0x7fadb007b0d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fadc4034000 tx=0x7fadc4004730 comp rx=0 tx=0).stop 2024-09-21T07:35:34.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.266+0000 7fadd7a00640 1 -- 172.21.15.39:0/2534102617 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadd0070440 msgr2=0x7fadd00cfa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:34.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.266+0000 7fadd7a00640 1 --2- 172.21.15.39:0/2534102617 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadd0070440 0x7fadd00cfa80 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fadbc00c390 tx=0x7fadbc00c850 comp rx=0 tx=0).stop 2024-09-21T07:35:34.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.267+0000 7fadd7a00640 1 -- 172.21.15.39:0/2534102617 shutdown_connections 2024-09-21T07:35:34.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.267+0000 7fadd7a00640 1 --2- 172.21.15.39:0/2534102617 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fadb0078bc0 0x7fadb007b0d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:34.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.267+0000 7fadd7a00640 1 --2- 172.21.15.39:0/2534102617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fadd0073170 0x7fadd014e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:34.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.267+0000 7fadd7a00640 1 --2- 172.21.15.39:0/2534102617 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadd0070440 0x7fadd00cfa80 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:34.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.267+0000 7fadd7a00640 1 --2- 172.21.15.39:0/2534102617 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fadd0070030 0x7fadd00ca770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:34.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.267+0000 7fadd7a00640 1 -- 172.21.15.39:0/2534102617 >> 172.21.15.39:0/2534102617 conn(0x7fadd00b1cb0 msgr2=0x7fadd015ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:34.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.267+0000 7fadd7a00640 1 -- 172.21.15.39:0/2534102617 shutdown_connections 2024-09-21T07:35:34.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.267+0000 7fadd7a00640 1 -- 172.21.15.39:0/2534102617 wait complete. 2024-09-21T07:35:34.274 INFO:tasks.ceph.mds.a.smithi039.stderr: -2177> 2024-09-21T07:35:18.718+0000 7ff61a800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:35:34.291 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T07:35:34.401 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-21T07:35:34.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.525+0000 7fc109800640 1 -- 172.21.15.39:0/3721108018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc104070440 msgr2=0x7fc104074b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:34.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.525+0000 7fc109800640 1 --2- 172.21.15.39:0/3721108018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc104070440 0x7fc104074b80 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc104070e70 tx=0x7fc0fc02a980 comp rx=0 tx=0).stop 2024-09-21T07:35:34.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.525+0000 7fc109800640 1 -- 172.21.15.39:0/3721108018 shutdown_connections 2024-09-21T07:35:34.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.525+0000 7fc109800640 1 --2- 172.21.15.39:0/3721108018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc104070440 0x7fc104074b80 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:34.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.525+0000 7fc109800640 1 --2- 172.21.15.39:0/3721108018 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc104070030 0x7fc104074640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:34.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.525+0000 7fc109800640 1 --2- 172.21.15.39:0/3721108018 >> v2:172.21.15.148:3300/0 conn(0x7fc104073170 0x7fc10415c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:34.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.525+0000 7fc109800640 1 -- 172.21.15.39:0/3721108018 >> 172.21.15.39:0/3721108018 conn(0x7fc1040b1cb0 msgr2=0x7fc1040b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:34.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.525+0000 7fc109800640 1 -- 172.21.15.39:0/3721108018 shutdown_connections 2024-09-21T07:35:34.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.525+0000 7fc109800640 1 -- 172.21.15.39:0/3721108018 wait complete. 2024-09-21T07:35:34.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.527+0000 7fc109800640 1 Processor -- start 2024-09-21T07:35:34.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.527+0000 7fc109800640 1 -- start start 2024-09-21T07:35:34.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.529+0000 7fc109800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc104070030 0x7fc1041505c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:34.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.529+0000 7fc109800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc104070440 0x7fc1040cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:34.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.529+0000 7fc109800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc104073170 0x7fc1040cf830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:34.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.529+0000 7fc109800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc10406f690 con 0x7fc104070440 2024-09-21T07:35:34.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.529+0000 7fc109800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc10406f510 con 0x7fc104070030 2024-09-21T07:35:34.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.529+0000 7fc109800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc10406f810 con 0x7fc104073170 2024-09-21T07:35:34.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.529+0000 7fc0fa000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc104070440 0x7fc1040cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:34.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.529+0000 7fc0fa000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc104070440 0x7fc1040cbac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36150/0 (socket says 172.21.15.39:36150) 2024-09-21T07:35:34.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.529+0000 7fc0fa000640 1 -- 172.21.15.39:0/1184880466 learned_addr learned my addr 172.21.15.39:0/1184880466 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:34.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.529+0000 7fc102a00640 1 --2- 172.21.15.39:0/1184880466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc104070030 0x7fc1041505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:34.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.529+0000 7fc103400640 1 --2- 172.21.15.39:0/1184880466 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc104073170 0x7fc1040cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:34.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.529+0000 7fc0fa000640 1 -- 172.21.15.39:0/1184880466 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc104073170 msgr2=0x7fc1040cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:34.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.529+0000 7fc0fa000640 1 --2- 172.21.15.39:0/1184880466 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc104073170 0x7fc1040cf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:34.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.529+0000 7fc0fa000640 1 -- 172.21.15.39:0/1184880466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc104070030 msgr2=0x7fc1041505c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:34.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.529+0000 7fc0fa000640 1 --2- 172.21.15.39:0/1184880466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc104070030 0x7fc1041505c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:34.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.529+0000 7fc0fa000640 1 -- 172.21.15.39:0/1184880466 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc104150b00 con 0x7fc104070440 2024-09-21T07:35:34.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.530+0000 7fc103400640 1 --2- 172.21.15.39:0/1184880466 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc104073170 0x7fc1040cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:35:34.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.530+0000 7fc0fa000640 1 --2- 172.21.15.39:0/1184880466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc104070440 0x7fc1040cbac0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec00c970 tx=0x7fc0ec00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:34.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.530+0000 7fc0fb400640 1 -- 172.21.15.39:0/1184880466 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec007b50 con 0x7fc104070440 2024-09-21T07:35:34.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.530+0000 7fc0fb400640 1 -- 172.21.15.39:0/1184880466 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0ec007de0 con 0x7fc104070440 2024-09-21T07:35:34.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.530+0000 7fc0fb400640 1 -- 172.21.15.39:0/1184880466 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec00f6b0 con 0x7fc104070440 2024-09-21T07:35:34.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.530+0000 7fc109800640 1 -- 172.21.15.39:0/1184880466 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1041f94b0 con 0x7fc104070440 2024-09-21T07:35:34.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.530+0000 7fc109800640 1 -- 172.21.15.39:0/1184880466 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1041f9a70 con 0x7fc104070440 2024-09-21T07:35:34.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.531+0000 7fc109800640 1 -- 172.21.15.39:0/1184880466 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc10415c780 con 0x7fc104070440 2024-09-21T07:35:34.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.533+0000 7fc0fb400640 1 -- 172.21.15.39:0/1184880466 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc0ec00f850 con 0x7fc104070440 2024-09-21T07:35:34.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.534+0000 7fc0fb400640 1 --2- 172.21.15.39:0/1184880466 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc0e0078bc0 0x7fc0e007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:34.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.534+0000 7fc0fb400640 1 -- 172.21.15.39:0/1184880466 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6886+0+0 (secure 0 0 0) 0x7fc0ec09b780 con 0x7fc104070440 2024-09-21T07:35:34.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.534+0000 7fc102a00640 1 --2- 172.21.15.39:0/1184880466 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc0e0078bc0 0x7fc0e007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:34.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.535+0000 7fc102a00640 1 --2- 172.21.15.39:0/1184880466 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc0e0078bc0 0x7fc0e007b0d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4005fd0 tx=0x7fc0f400a250 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:34.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.538+0000 7fc0fb400640 1 -- 172.21.15.39:0/1184880466 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc104150b00 con 0x7fc104070440 2024-09-21T07:35:34.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:34.774+0000 7fc109800640 1 -- 172.21.15.39:0/1184880466 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc1040c97d0 con 0x7fc104070440 2024-09-21T07:35:35.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.269+0000 7fc0fb400640 1 -- 172.21.15.39:0/1184880466 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v37) ==== 104+0+0 (secure 0 0 0) 0x7fc0ec014030 con 0x7fc104070440 2024-09-21T07:35:35.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.271+0000 7fc109800640 1 -- 172.21.15.39:0/1184880466 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc0e0078bc0 msgr2=0x7fc0e007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:35.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.271+0000 7fc109800640 1 --2- 172.21.15.39:0/1184880466 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc0e0078bc0 0x7fc0e007b0d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4005fd0 tx=0x7fc0f400a250 comp rx=0 tx=0).stop 2024-09-21T07:35:35.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.271+0000 7fc109800640 1 -- 172.21.15.39:0/1184880466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc104070440 msgr2=0x7fc1040cbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:35.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.271+0000 7fc109800640 1 --2- 172.21.15.39:0/1184880466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc104070440 0x7fc1040cbac0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec00c970 tx=0x7fc0ec00ce30 comp rx=0 tx=0).stop 2024-09-21T07:35:35.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.271+0000 7fc109800640 1 -- 172.21.15.39:0/1184880466 shutdown_connections 2024-09-21T07:35:35.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.271+0000 7fc109800640 1 --2- 172.21.15.39:0/1184880466 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc0e0078bc0 0x7fc0e007b0d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:35.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.271+0000 7fc109800640 1 --2- 172.21.15.39:0/1184880466 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc104073170 0x7fc1040cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:35.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.271+0000 7fc109800640 1 --2- 172.21.15.39:0/1184880466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc104070440 0x7fc1040cbac0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:35.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.271+0000 7fc109800640 1 --2- 172.21.15.39:0/1184880466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc104070030 0x7fc1041505c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:35.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.271+0000 7fc109800640 1 -- 172.21.15.39:0/1184880466 >> 172.21.15.39:0/1184880466 conn(0x7fc1040b1cb0 msgr2=0x7fc1040ad330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:35.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.271+0000 7fc109800640 1 -- 172.21.15.39:0/1184880466 shutdown_connections 2024-09-21T07:35:35.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.271+0000 7fc109800640 1 -- 172.21.15.39:0/1184880466 wait complete. 2024-09-21T07:35:35.300 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T07:35:35.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.530+0000 7fd3a6200640 1 -- 172.21.15.39:0/1893881993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3a00cb4d0 msgr2=0x7fd3a0073820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:35.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.530+0000 7fd3a6200640 1 --2- 172.21.15.39:0/1893881993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3a00cb4d0 0x7fd3a0073820 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd388009830 tx=0x7fd38802a480 comp rx=0 tx=0).stop 2024-09-21T07:35:35.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.530+0000 7fd3a6200640 1 -- 172.21.15.39:0/1893881993 shutdown_connections 2024-09-21T07:35:35.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.530+0000 7fd3a6200640 1 --2- 172.21.15.39:0/1893881993 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd3a015fb80 0x7fd3a00acf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:35.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.530+0000 7fd3a6200640 1 --2- 172.21.15.39:0/1893881993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3a00cb4d0 0x7fd3a0073820 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:35.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.530+0000 7fd3a6200640 1 -- 172.21.15.39:0/1893881993 >> 172.21.15.39:0/1893881993 conn(0x7fd3a00d0130 msgr2=0x7fd3a00d0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:35.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.530+0000 7fd3a6200640 1 -- 172.21.15.39:0/1893881993 shutdown_connections 2024-09-21T07:35:35.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.530+0000 7fd3a6200640 1 -- 172.21.15.39:0/1893881993 wait complete. 2024-09-21T07:35:35.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.532+0000 7fd3a6200640 1 Processor -- start 2024-09-21T07:35:35.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.533+0000 7fd3a6200640 1 -- start start 2024-09-21T07:35:35.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.534+0000 7fd3a6200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd3a00cc560 0x7fd3a00b3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:35.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.534+0000 7fd3a6200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3a015fb80 0x7fd3a014c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:35.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.534+0000 7fd3a6200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3a014c8f0 0x7fd3a00cef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:35.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.534+0000 7fd3a6200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a00ad9b0 con 0x7fd3a014c8f0 2024-09-21T07:35:35.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.534+0000 7fd3a6200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a00ad830 con 0x7fd3a015fb80 2024-09-21T07:35:35.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.534+0000 7fd3a6200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd3a00adb30 con 0x7fd3a00cc560 2024-09-21T07:35:35.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.534+0000 7fd39fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3a014c8f0 0x7fd3a00cef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:35.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.534+0000 7fd39fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3a014c8f0 0x7fd3a00cef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36166/0 (socket says 172.21.15.39:36166) 2024-09-21T07:35:35.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.534+0000 7fd39fe00640 1 -- 172.21.15.39:0/2547037658 learned_addr learned my addr 172.21.15.39:0/2547037658 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:35.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.534+0000 7fd39f400640 1 --2- 172.21.15.39:0/2547037658 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd3a00cc560 0x7fd3a00b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:35.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.535+0000 7fd397e00640 1 --2- 172.21.15.39:0/2547037658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3a015fb80 0x7fd3a014c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:35.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.535+0000 7fd39fe00640 1 -- 172.21.15.39:0/2547037658 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd3a00cc560 msgr2=0x7fd3a00b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:35.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.535+0000 7fd39fe00640 1 --2- 172.21.15.39:0/2547037658 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd3a00cc560 0x7fd3a00b3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:35.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.535+0000 7fd39fe00640 1 -- 172.21.15.39:0/2547037658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3a015fb80 msgr2=0x7fd3a014c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:35.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.535+0000 7fd39fe00640 1 --2- 172.21.15.39:0/2547037658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3a015fb80 0x7fd3a014c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:35.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.535+0000 7fd39fe00640 1 -- 172.21.15.39:0/2547037658 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a00ce7b0 con 0x7fd3a014c8f0 2024-09-21T07:35:35.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.535+0000 7fd397e00640 1 --2- 172.21.15.39:0/2547037658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3a015fb80 0x7fd3a014c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:35:35.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.535+0000 7fd39fe00640 1 --2- 172.21.15.39:0/2547037658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3a014c8f0 0x7fd3a00cef40 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd39800d9d0 tx=0x7fd39800de90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:35.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.535+0000 7fd397400640 1 -- 172.21.15.39:0/2547037658 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd398017070 con 0x7fd3a014c8f0 2024-09-21T07:35:35.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.535+0000 7fd397400640 1 -- 172.21.15.39:0/2547037658 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd398003970 con 0x7fd3a014c8f0 2024-09-21T07:35:35.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.535+0000 7fd397400640 1 -- 172.21.15.39:0/2547037658 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39800be60 con 0x7fd3a014c8f0 2024-09-21T07:35:35.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.535+0000 7fd3a6200640 1 -- 172.21.15.39:0/2547037658 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3a014c5f0 con 0x7fd3a014c8f0 2024-09-21T07:35:35.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.536+0000 7fd3a6200640 1 -- 172.21.15.39:0/2547037658 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3a009ae40 con 0x7fd3a014c8f0 2024-09-21T07:35:35.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.537+0000 7fd3a6200640 1 -- 172.21.15.39:0/2547037658 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3a00b3bf0 con 0x7fd3a014c8f0 2024-09-21T07:35:35.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.538+0000 7fd397400640 1 -- 172.21.15.39:0/2547037658 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd398010460 con 0x7fd3a014c8f0 2024-09-21T07:35:35.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.539+0000 7fd397400640 1 --2- 172.21.15.39:0/2547037658 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd37c078b90 0x7fd37c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:35.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.539+0000 7fd397400640 1 -- 172.21.15.39:0/2547037658 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6886+0+0 (secure 0 0 0) 0x7fd39809b690 con 0x7fd3a014c8f0 2024-09-21T07:35:35.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.539+0000 7fd39f400640 1 --2- 172.21.15.39:0/2547037658 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd37c078b90 0x7fd37c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:35.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.540+0000 7fd39f400640 1 --2- 172.21.15.39:0/2547037658 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd37c078b90 0x7fd37c07b0a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd390000f60 tx=0x7fd390009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:35.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.543+0000 7fd397400640 1 -- 172.21.15.39:0/2547037658 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd398062f40 con 0x7fd3a014c8f0 2024-09-21T07:35:35.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:35.739+0000 7fd3a6200640 1 -- 172.21.15.39:0/2547037658 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd3a00ae050 con 0x7fd3a014c8f0 2024-09-21T07:35:36.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.281+0000 7fd397400640 1 -- 172.21.15.39:0/2547037658 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v68) ==== 181+0+0 (secure 0 0 0) 0x7fd398067970 con 0x7fd3a014c8f0 2024-09-21T07:35:36.283 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' removed 2024-09-21T07:35:36.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.284+0000 7fd3a6200640 1 -- 172.21.15.39:0/2547037658 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd37c078b90 msgr2=0x7fd37c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:36.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.284+0000 7fd3a6200640 1 --2- 172.21.15.39:0/2547037658 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd37c078b90 0x7fd37c07b0a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd390000f60 tx=0x7fd390009040 comp rx=0 tx=0).stop 2024-09-21T07:35:36.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.284+0000 7fd3a6200640 1 -- 172.21.15.39:0/2547037658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3a014c8f0 msgr2=0x7fd3a00cef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:36.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.284+0000 7fd3a6200640 1 --2- 172.21.15.39:0/2547037658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3a014c8f0 0x7fd3a00cef40 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd39800d9d0 tx=0x7fd39800de90 comp rx=0 tx=0).stop 2024-09-21T07:35:36.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.285+0000 7fd3a6200640 1 -- 172.21.15.39:0/2547037658 shutdown_connections 2024-09-21T07:35:36.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.285+0000 7fd3a6200640 1 --2- 172.21.15.39:0/2547037658 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd37c078b90 0x7fd37c07b0a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:36.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.285+0000 7fd3a6200640 1 --2- 172.21.15.39:0/2547037658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3a014c8f0 0x7fd3a00cef40 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:36.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.285+0000 7fd3a6200640 1 --2- 172.21.15.39:0/2547037658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3a015fb80 0x7fd3a014c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:36.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.285+0000 7fd3a6200640 1 --2- 172.21.15.39:0/2547037658 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd3a00cc560 0x7fd3a00b3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:36.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.285+0000 7fd3a6200640 1 -- 172.21.15.39:0/2547037658 >> 172.21.15.39:0/2547037658 conn(0x7fd3a00d0130 msgr2=0x7fd3a00cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:36.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.285+0000 7fd3a6200640 1 -- 172.21.15.39:0/2547037658 shutdown_connections 2024-09-21T07:35:36.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.285+0000 7fd3a6200640 1 -- 172.21.15.39:0/2547037658 wait complete. 2024-09-21T07:35:36.319 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T07:35:36.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.558+0000 7f5a80800640 1 -- 172.21.15.39:0/1612359385 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5a78070aa0 msgr2=0x7f5a60003a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:36.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.558+0000 7f5a80800640 1 --2- 172.21.15.39:0/1612359385 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5a78070aa0 0x7f5a60003a50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c009b50 tx=0x7f5a60005830 comp rx=0 tx=0).stop 2024-09-21T07:35:36.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.559+0000 7f5a7e800640 1 -- 172.21.15.39:0/1612359385 reap_dead start 2024-09-21T07:35:36.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.559+0000 7f5a80800640 1 -- 172.21.15.39:0/1612359385 shutdown_connections 2024-09-21T07:35:36.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.559+0000 7f5a80800640 1 -- 172.21.15.39:0/1612359385 >> 172.21.15.39:0/1612359385 conn(0x7f5a780b1c40 msgr2=0x7f5a780b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:36.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.559+0000 7f5a80800640 1 -- 172.21.15.39:0/1612359385 shutdown_connections 2024-09-21T07:35:36.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.559+0000 7f5a80800640 1 -- 172.21.15.39:0/1612359385 wait complete. 2024-09-21T07:35:36.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.561+0000 7f5a80800640 1 Processor -- start 2024-09-21T07:35:36.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.561+0000 7f5a80800640 1 -- start start 2024-09-21T07:35:36.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.563+0000 7f5a80800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5a78070aa0 0x7f5a7814cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:36.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.563+0000 7f5a80800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a780c9a20 0x7f5a7814a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:36.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.563+0000 7f5a80800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a7814a920 0x7f5a781f6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:36.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.563+0000 7f5a80800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5a780b49e0 con 0x7f5a7814a920 2024-09-21T07:35:36.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.563+0000 7f5a80800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5a780b4860 con 0x7f5a780c9a20 2024-09-21T07:35:36.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.563+0000 7f5a80800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f5a780cb5e0 con 0x7f5a78070aa0 2024-09-21T07:35:36.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.563+0000 7f5a7e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a7814a920 0x7f5a781f6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:36.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.563+0000 7f5a7e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a7814a920 0x7f5a781f6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36182/0 (socket says 172.21.15.39:36182) 2024-09-21T07:35:36.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.563+0000 7f5a7e800640 1 -- 172.21.15.39:0/2235653758 learned_addr learned my addr 172.21.15.39:0/2235653758 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:36.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.563+0000 7f5a7de00640 1 --2- 172.21.15.39:0/2235653758 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5a78070aa0 0x7f5a7814cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:36.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.563+0000 7f5a7e800640 1 -- 172.21.15.39:0/2235653758 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5a78070aa0 msgr2=0x7f5a7814cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:36.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.563+0000 7f5a7d400640 1 --2- 172.21.15.39:0/2235653758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a780c9a20 0x7f5a7814a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:36.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.563+0000 7f5a7e800640 1 --2- 172.21.15.39:0/2235653758 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5a78070aa0 0x7f5a7814cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:36.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.563+0000 7f5a7e800640 1 -- 172.21.15.39:0/2235653758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a780c9a20 msgr2=0x7f5a7814a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:36.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.563+0000 7f5a7e800640 1 --2- 172.21.15.39:0/2235653758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a780c9a20 0x7f5a7814a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:36.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.563+0000 7f5a7e800640 1 -- 172.21.15.39:0/2235653758 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a7814c510 con 0x7f5a7814a920 2024-09-21T07:35:36.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.563+0000 7f5a7de00640 1 --2- 172.21.15.39:0/2235653758 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5a78070aa0 0x7f5a7814cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:35:36.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.563+0000 7f5a7d400640 1 --2- 172.21.15.39:0/2235653758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a780c9a20 0x7f5a7814a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:35:36.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.564+0000 7f5a7e800640 1 --2- 172.21.15.39:0/2235653758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a7814a920 0x7f5a781f6ac0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f5a780cf350 tx=0x7f5a74006720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:36.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.564+0000 7f5a72000640 1 -- 172.21.15.39:0/2235653758 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7400fa20 con 0x7f5a7814a920 2024-09-21T07:35:36.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.564+0000 7f5a72000640 1 -- 172.21.15.39:0/2235653758 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a74018070 con 0x7f5a7814a920 2024-09-21T07:35:36.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.564+0000 7f5a72000640 1 -- 172.21.15.39:0/2235653758 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7400fea0 con 0x7f5a7814a920 2024-09-21T07:35:36.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.564+0000 7f5a80800640 1 -- 172.21.15.39:0/2235653758 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a781f7660 con 0x7f5a7814a920 2024-09-21T07:35:36.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.564+0000 7f5a80800640 1 -- 172.21.15.39:0/2235653758 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a780c9e30 con 0x7f5a7814a920 2024-09-21T07:35:36.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.565+0000 7f5a80800640 1 -- 172.21.15.39:0/2235653758 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a78074d10 con 0x7f5a7814a920 2024-09-21T07:35:36.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.567+0000 7f5a72000640 1 -- 172.21.15.39:0/2235653758 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5a7401d460 con 0x7f5a7814a920 2024-09-21T07:35:36.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.568+0000 7f5a72000640 1 --2- 172.21.15.39:0/2235653758 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5a5c078bc0 0x7f5a5c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:36.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.568+0000 7f5a72000640 1 -- 172.21.15.39:0/2235653758 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6448+0+0 (secure 0 0 0) 0x7f5a7409b7f0 con 0x7f5a7814a920 2024-09-21T07:35:36.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.568+0000 7f5a7de00640 1 --2- 172.21.15.39:0/2235653758 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5a5c078bc0 0x7f5a5c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:36.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.569+0000 7f5a7de00640 1 --2- 172.21.15.39:0/2235653758 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5a5c078bc0 0x7f5a5c07b0d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c035000 tx=0x7f5a6c0046c0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:36.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.573+0000 7f5a72000640 1 -- 172.21.15.39:0/2235653758 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5a78074d10 con 0x7f5a7814a920 2024-09-21T07:35:36.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:36.773+0000 7f5a80800640 1 -- 172.21.15.39:0/2235653758 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5a780b4150 con 0x7f5a7814a920 2024-09-21T07:35:37.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.353+0000 7f5a72000640 1 -- 172.21.15.39:0/2235653758 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v70) ==== 169+0+0 (secure 0 0 0) 0x7f5a780ca880 con 0x7f5a7814a920 2024-09-21T07:35:37.355 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' removed 2024-09-21T07:35:37.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.356+0000 7f5a80800640 1 -- 172.21.15.39:0/2235653758 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5a5c078bc0 msgr2=0x7f5a5c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:37.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.356+0000 7f5a80800640 1 --2- 172.21.15.39:0/2235653758 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5a5c078bc0 0x7f5a5c07b0d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c035000 tx=0x7f5a6c0046c0 comp rx=0 tx=0).stop 2024-09-21T07:35:37.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.356+0000 7f5a80800640 1 -- 172.21.15.39:0/2235653758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a7814a920 msgr2=0x7f5a781f6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:37.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.356+0000 7f5a80800640 1 --2- 172.21.15.39:0/2235653758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a7814a920 0x7f5a781f6ac0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f5a780cf350 tx=0x7f5a74006720 comp rx=0 tx=0).stop 2024-09-21T07:35:37.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.356+0000 7f5a80800640 1 -- 172.21.15.39:0/2235653758 shutdown_connections 2024-09-21T07:35:37.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.356+0000 7f5a80800640 1 --2- 172.21.15.39:0/2235653758 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5a5c078bc0 0x7f5a5c07b0d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:37.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.356+0000 7f5a80800640 1 --2- 172.21.15.39:0/2235653758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a7814a920 0x7f5a781f6ac0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:37.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.356+0000 7f5a80800640 1 --2- 172.21.15.39:0/2235653758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a780c9a20 0x7f5a7814a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:37.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.356+0000 7f5a80800640 1 --2- 172.21.15.39:0/2235653758 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5a78070aa0 0x7f5a7814cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:37.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.356+0000 7f5a80800640 1 -- 172.21.15.39:0/2235653758 >> 172.21.15.39:0/2235653758 conn(0x7f5a780b1c40 msgr2=0x7f5a780b0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:37.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.356+0000 7f5a80800640 1 -- 172.21.15.39:0/2235653758 shutdown_connections 2024-09-21T07:35:37.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.356+0000 7f5a80800640 1 -- 172.21.15.39:0/2235653758 wait complete. 2024-09-21T07:35:37.383 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T07:35:37.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.612+0000 7f1ba1800640 1 -- 172.21.15.39:0/1396595146 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b80003a30 msgr2=0x7f1b80003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:37.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.612+0000 7f1ba1800640 1 --2- 172.21.15.39:0/1396595146 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b80003a30 0x7f1b80003ea0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c006fd0 tx=0x7f1b80005830 comp rx=0 tx=0).stop 2024-09-21T07:35:37.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.612+0000 7f1b9b400640 1 -- 172.21.15.39:0/1396595146 reap_dead start 2024-09-21T07:35:37.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.612+0000 7f1ba1800640 1 -- 172.21.15.39:0/1396595146 shutdown_connections 2024-09-21T07:35:37.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.612+0000 7f1ba1800640 1 -- 172.21.15.39:0/1396595146 >> 172.21.15.39:0/1396595146 conn(0x7f1b9c06d7d0 msgr2=0x7f1b9c0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:37.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.612+0000 7f1ba1800640 1 -- 172.21.15.39:0/1396595146 shutdown_connections 2024-09-21T07:35:37.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.613+0000 7f1ba1800640 1 -- 172.21.15.39:0/1396595146 wait complete. 2024-09-21T07:35:37.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.615+0000 7f1ba1800640 1 Processor -- start 2024-09-21T07:35:37.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.615+0000 7f1ba1800640 1 -- start start 2024-09-21T07:35:37.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.616+0000 7f1ba1800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b80003a30 0x7f1b9c06f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:37.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.616+0000 7f1ba1800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1b9c09d780 0x7f1b9c06fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:37.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.616+0000 7f1ba1800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b9c1f5ab0 0x7f1b9c0734f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:37.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.616+0000 7f1ba1800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c0b4500 con 0x7f1b9c1f5ab0 2024-09-21T07:35:37.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.616+0000 7f1ba1800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c0b4380 con 0x7f1b80003a30 2024-09-21T07:35:37.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.616+0000 7f1ba1800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f1b9c0b4680 con 0x7f1b9c09d780 2024-09-21T07:35:37.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.617+0000 7f1b9b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b9c1f5ab0 0x7f1b9c0734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:37.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.617+0000 7f1b9b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b9c1f5ab0 0x7f1b9c0734f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36190/0 (socket says 172.21.15.39:36190) 2024-09-21T07:35:37.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.617+0000 7f1b9b400640 1 -- 172.21.15.39:0/1579941386 learned_addr learned my addr 172.21.15.39:0/1579941386 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:37.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.617+0000 7f1b9aa00640 1 --2- 172.21.15.39:0/1579941386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b80003a30 0x7f1b9c06f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:37.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.617+0000 7f1b9a000640 1 --2- 172.21.15.39:0/1579941386 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1b9c09d780 0x7f1b9c06fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:37.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.617+0000 7f1b9b400640 1 -- 172.21.15.39:0/1579941386 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1b9c09d780 msgr2=0x7f1b9c06fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:37.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.617+0000 7f1b9b400640 1 --2- 172.21.15.39:0/1579941386 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1b9c09d780 0x7f1b9c06fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:37.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.617+0000 7f1b9b400640 1 -- 172.21.15.39:0/1579941386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b80003a30 msgr2=0x7f1b9c06f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:37.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.617+0000 7f1b9b400640 1 --2- 172.21.15.39:0/1579941386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b80003a30 0x7f1b9c06f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:37.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.617+0000 7f1b9b400640 1 -- 172.21.15.39:0/1579941386 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b9c1511f0 con 0x7f1b9c1f5ab0 2024-09-21T07:35:37.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.617+0000 7f1b9a000640 1 --2- 172.21.15.39:0/1579941386 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1b9c09d780 0x7f1b9c06fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:35:37.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.618+0000 7f1b9b400640 1 --2- 172.21.15.39:0/1579941386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b9c1f5ab0 0x7f1b9c0734f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c070590 tx=0x7f1b940019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:37.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.618+0000 7f1b92a00640 1 -- 172.21.15.39:0/1579941386 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b940107d0 con 0x7f1b9c1f5ab0 2024-09-21T07:35:37.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.618+0000 7f1b92a00640 1 -- 172.21.15.39:0/1579941386 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b94016070 con 0x7f1b9c1f5ab0 2024-09-21T07:35:37.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.618+0000 7f1b92a00640 1 -- 172.21.15.39:0/1579941386 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b94010d00 con 0x7f1b9c1f5ab0 2024-09-21T07:35:37.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.618+0000 7f1ba1800640 1 -- 172.21.15.39:0/1579941386 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b9c1f97a0 con 0x7f1b9c1f5ab0 2024-09-21T07:35:37.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.618+0000 7f1ba1800640 1 -- 172.21.15.39:0/1579941386 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b9c1f9d60 con 0x7f1b9c1f5ab0 2024-09-21T07:35:37.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.619+0000 7f1ba1800640 1 -- 172.21.15.39:0/1579941386 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b9c1f9510 con 0x7f1b9c1f5ab0 2024-09-21T07:35:37.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.620+0000 7f1b92a00640 1 -- 172.21.15.39:0/1579941386 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1b9401b460 con 0x7f1b9c1f5ab0 2024-09-21T07:35:37.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.621+0000 7f1b92a00640 1 --2- 172.21.15.39:0/1579941386 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1b7c078bc0 0x7f1b7c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:37.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.621+0000 7f1b92a00640 1 -- 172.21.15.39:0/1579941386 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6006+0+0 (secure 0 0 0) 0x7f1b9409b630 con 0x7f1b9c1f5ab0 2024-09-21T07:35:37.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.622+0000 7f1b9aa00640 1 --2- 172.21.15.39:0/1579941386 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1b7c078bc0 0x7f1b7c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:37.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.622+0000 7f1b9aa00640 1 --2- 172.21.15.39:0/1579941386 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1b7c078bc0 0x7f1b7c07b0d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c033000 tx=0x7f1b8c038040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:37.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.626+0000 7f1b92a00640 1 -- 172.21.15.39:0/1579941386 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1b94063930 con 0x7f1b9c1f5ab0 2024-09-21T07:35:37.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.824+0000 7f1ba1800640 1 -- 172.21.15.39:0/1579941386 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f1b9c15d8a0 con 0x7f1b9c1f5ab0 2024-09-21T07:35:37.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.828+0000 7f1b92a00640 1 -- 172.21.15.39:0/1579941386 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v38) ==== 71+0+0 (secure 0 0 0) 0x7f1b94063230 con 0x7f1b9c1f5ab0 2024-09-21T07:35:37.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.831+0000 7f1ba1800640 1 -- 172.21.15.39:0/1579941386 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1b7c078bc0 msgr2=0x7f1b7c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:37.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.831+0000 7f1ba1800640 1 --2- 172.21.15.39:0/1579941386 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1b7c078bc0 0x7f1b7c07b0d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c033000 tx=0x7f1b8c038040 comp rx=0 tx=0).stop 2024-09-21T07:35:37.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.831+0000 7f1ba1800640 1 -- 172.21.15.39:0/1579941386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b9c1f5ab0 msgr2=0x7f1b9c0734f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:37.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.831+0000 7f1ba1800640 1 --2- 172.21.15.39:0/1579941386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b9c1f5ab0 0x7f1b9c0734f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c070590 tx=0x7f1b940019b0 comp rx=0 tx=0).stop 2024-09-21T07:35:37.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.831+0000 7f1ba1800640 1 -- 172.21.15.39:0/1579941386 shutdown_connections 2024-09-21T07:35:37.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.831+0000 7f1ba1800640 1 --2- 172.21.15.39:0/1579941386 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1b7c078bc0 0x7f1b7c07b0d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:37.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.831+0000 7f1ba1800640 1 --2- 172.21.15.39:0/1579941386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b9c1f5ab0 0x7f1b9c0734f0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:37.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.831+0000 7f1ba1800640 1 --2- 172.21.15.39:0/1579941386 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1b9c09d780 0x7f1b9c06fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:37.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.831+0000 7f1ba1800640 1 --2- 172.21.15.39:0/1579941386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b80003a30 0x7f1b9c06f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:37.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.831+0000 7f1ba1800640 1 -- 172.21.15.39:0/1579941386 >> 172.21.15.39:0/1579941386 conn(0x7f1b9c06d7d0 msgr2=0x7f1b9c06ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:37.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.831+0000 7f1ba1800640 1 -- 172.21.15.39:0/1579941386 shutdown_connections 2024-09-21T07:35:37.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:37.831+0000 7f1ba1800640 1 -- 172.21.15.39:0/1579941386 wait complete. 2024-09-21T07:35:37.858 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_using_pool_with_snap_fails_fs_creation' 2024-09-21T07:35:38.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.085+0000 7f7525000640 1 -- 172.21.15.39:0/3009011906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f752006ee70 msgr2=0x7f752006f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:38.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.085+0000 7f7525000640 1 --2- 172.21.15.39:0/3009011906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f752006ee70 0x7f752006f280 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f752006f970 tx=0x7f750c02a540 comp rx=0 tx=0).stop 2024-09-21T07:35:38.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.085+0000 7f7525000640 1 -- 172.21.15.39:0/3009011906 shutdown_connections 2024-09-21T07:35:38.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.085+0000 7f7525000640 1 --2- 172.21.15.39:0/3009011906 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7520162250 0x7f7520162660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:38.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.085+0000 7f7525000640 1 --2- 172.21.15.39:0/3009011906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f752006ee70 0x7f752006f280 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:38.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.085+0000 7f7525000640 1 --2- 172.21.15.39:0/3009011906 >> v2:172.21.15.148:3300/0 conn(0x7f7520158e60 0x7f752015f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:38.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.085+0000 7f7525000640 1 -- 172.21.15.39:0/3009011906 >> 172.21.15.39:0/3009011906 conn(0x7f752006d410 msgr2=0x7f75200b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:38.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.085+0000 7f7525000640 1 -- 172.21.15.39:0/3009011906 shutdown_connections 2024-09-21T07:35:38.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.085+0000 7f7525000640 1 -- 172.21.15.39:0/3009011906 wait complete. 2024-09-21T07:35:38.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.087+0000 7f7525000640 1 Processor -- start 2024-09-21T07:35:38.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.087+0000 7f7525000640 1 -- start start 2024-09-21T07:35:38.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.088+0000 7f7525000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f752006ee70 0x7f75200cc250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:38.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.089+0000 7f7525000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7520158e60 0x7f75201454a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:38.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.089+0000 7f7525000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7520162250 0x7f75200b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:38.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.089+0000 7f7525000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7520162d20 con 0x7f7520162250 2024-09-21T07:35:38.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.089+0000 7f7525000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7520162ba0 con 0x7f7520158e60 2024-09-21T07:35:38.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.089+0000 7f7525000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f7520162ea0 con 0x7f752006ee70 2024-09-21T07:35:38.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.089+0000 7f751fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7520162250 0x7f75200b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:38.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.089+0000 7f751fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7520162250 0x7f75200b0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36194/0 (socket says 172.21.15.39:36194) 2024-09-21T07:35:38.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.089+0000 7f751fe00640 1 -- 172.21.15.39:0/447750708 learned_addr learned my addr 172.21.15.39:0/447750708 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:38.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.089+0000 7f751ea00640 1 --2- 172.21.15.39:0/447750708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7520158e60 0x7f75201454a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:38.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.089+0000 7f751f400640 1 --2- 172.21.15.39:0/447750708 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f752006ee70 0x7f75200cc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:38.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.089+0000 7f751fe00640 1 -- 172.21.15.39:0/447750708 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f752006ee70 msgr2=0x7f75200cc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:38.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.089+0000 7f751fe00640 1 --2- 172.21.15.39:0/447750708 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f752006ee70 0x7f75200cc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:38.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.089+0000 7f751fe00640 1 -- 172.21.15.39:0/447750708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7520158e60 msgr2=0x7f75201454a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:38.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.089+0000 7f751fe00640 1 --2- 172.21.15.39:0/447750708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7520158e60 0x7f75201454a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:38.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.089+0000 7f751fe00640 1 -- 172.21.15.39:0/447750708 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75200ccb30 con 0x7f7520162250 2024-09-21T07:35:38.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.090+0000 7f751fe00640 1 --2- 172.21.15.39:0/447750708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7520162250 0x7f75200b0b50 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f751800c970 tx=0x7f751800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:38.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.090+0000 7f7517400640 1 -- 172.21.15.39:0/447750708 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7518004460 con 0x7f7520162250 2024-09-21T07:35:38.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.090+0000 7f7517400640 1 -- 172.21.15.39:0/447750708 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75180077a0 con 0x7f7520162250 2024-09-21T07:35:38.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.090+0000 7f7517400640 1 -- 172.21.15.39:0/447750708 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7518007bd0 con 0x7f7520162250 2024-09-21T07:35:38.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.090+0000 7f7525000640 1 -- 172.21.15.39:0/447750708 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75201f5d30 con 0x7f7520162250 2024-09-21T07:35:38.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.090+0000 7f7525000640 1 -- 172.21.15.39:0/447750708 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75200cce10 con 0x7f7520162250 2024-09-21T07:35:38.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.091+0000 7f7525000640 1 -- 172.21.15.39:0/447750708 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f752006faf0 con 0x7f7520162250 2024-09-21T07:35:38.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.093+0000 7f7517400640 1 -- 172.21.15.39:0/447750708 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f75180035f0 con 0x7f7520162250 2024-09-21T07:35:38.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.094+0000 7f7517400640 1 --2- 172.21.15.39:0/447750708 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7500078b90 0x7f750007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:38.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.094+0000 7f7517400640 1 -- 172.21.15.39:0/447750708 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6006+0+0 (secure 0 0 0) 0x7f751809b360 con 0x7f7520162250 2024-09-21T07:35:38.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.094+0000 7f751f400640 1 --2- 172.21.15.39:0/447750708 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7500078b90 0x7f750007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:38.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.095+0000 7f751f400640 1 --2- 172.21.15.39:0/447750708 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7500078b90 0x7f750007b0a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f75200b30a0 tx=0x7f7510004ab0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:38.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.099+0000 7f7517400640 1 -- 172.21.15.39:0/447750708 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f752006faf0 con 0x7f7520162250 2024-09-21T07:35:38.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:38.380+0000 7f7525000640 1 -- 172.21.15.39:0/447750708 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_using_pool_with_snap_fails_fs_creation"]} v 0) -- 0x7f752015f6e0 con 0x7f7520162250 2024-09-21T07:35:39.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.363+0000 7f7517400640 1 -- 172.21.15.39:0/447750708 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_using_pool_with_snap_fails_fs_creation"]}]=0 v157) ==== 173+0+0 (secure 0 0 0) 0x7f7518013030 con 0x7f7520162250 2024-09-21T07:35:39.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.366+0000 7f7525000640 1 -- 172.21.15.39:0/447750708 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7500078b90 msgr2=0x7f750007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:39.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.366+0000 7f7525000640 1 --2- 172.21.15.39:0/447750708 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7500078b90 0x7f750007b0a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f75200b30a0 tx=0x7f7510004ab0 comp rx=0 tx=0).stop 2024-09-21T07:35:39.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.366+0000 7f7525000640 1 -- 172.21.15.39:0/447750708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7520162250 msgr2=0x7f75200b0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:39.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.366+0000 7f7525000640 1 --2- 172.21.15.39:0/447750708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7520162250 0x7f75200b0b50 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f751800c970 tx=0x7f751800ce30 comp rx=0 tx=0).stop 2024-09-21T07:35:39.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.366+0000 7f7525000640 1 -- 172.21.15.39:0/447750708 shutdown_connections 2024-09-21T07:35:39.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.366+0000 7f7525000640 1 --2- 172.21.15.39:0/447750708 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7500078b90 0x7f750007b0a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:39.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.366+0000 7f7525000640 1 --2- 172.21.15.39:0/447750708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7520162250 0x7f75200b0b50 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:39.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.366+0000 7f7525000640 1 --2- 172.21.15.39:0/447750708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7520158e60 0x7f75201454a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:39.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.366+0000 7f7525000640 1 --2- 172.21.15.39:0/447750708 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f752006ee70 0x7f75200cc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:39.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.366+0000 7f7525000640 1 -- 172.21.15.39:0/447750708 >> 172.21.15.39:0/447750708 conn(0x7f752006d410 msgr2=0x7f752015d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:39.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.366+0000 7f7525000640 1 -- 172.21.15.39:0/447750708 shutdown_connections 2024-09-21T07:35:39.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.366+0000 7f7525000640 1 -- 172.21.15.39:0/447750708 wait complete. 2024-09-21T07:35:39.393 INFO:tasks.cephfs_test_runner:Test that using a pool with snaps for fs creation fails ... ok 2024-09-21T07:35:39.393 INFO:tasks.cephfs_test_runner:Starting test: test_allow_new_snaps_config (tasks.cephfs.test_snapshots.TestSnapshots) Check whether 'allow_new_snaps' setting works 2024-09-21T07:35:39.394 INFO:tasks.cephfs_test_runner:test_allow_new_snaps_config (tasks.cephfs.test_snapshots.TestSnapshots) 2024-09-21T07:35:39.394 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:35:39.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.624+0000 7f5d60200640 1 -- 172.21.15.39:0/863581240 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d580cc1e0 msgr2=0x7f5d580af3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:39.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.624+0000 7f5d60200640 1 --2- 172.21.15.39:0/863581240 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d580cc1e0 0x7f5d580af3c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c009830 tx=0x7f5d4c02a530 comp rx=0 tx=0).stop 2024-09-21T07:35:39.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.624+0000 7f5d60200640 1 -- 172.21.15.39:0/863581240 shutdown_connections 2024-09-21T07:35:39.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.624+0000 7f5d60200640 1 --2- 172.21.15.39:0/863581240 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5d5815d8a0 0x7f5d580b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:39.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.624+0000 7f5d60200640 1 --2- 172.21.15.39:0/863581240 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d580cc1e0 0x7f5d580af3c0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:39.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.624+0000 7f5d60200640 1 -- 172.21.15.39:0/863581240 >> 172.21.15.39:0/863581240 conn(0x7f5d5806d7d0 msgr2=0x7f5d580b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:39.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.625+0000 7f5d60200640 1 -- 172.21.15.39:0/863581240 shutdown_connections 2024-09-21T07:35:39.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.625+0000 7f5d60200640 1 -- 172.21.15.39:0/863581240 wait complete. 2024-09-21T07:35:39.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.627+0000 7f5d60200640 1 Processor -- start 2024-09-21T07:35:39.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.627+0000 7f5d60200640 1 -- start start 2024-09-21T07:35:39.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.628+0000 7f5d60200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5d580cc1e0 0x7f5d580ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:39.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.628+0000 7f5d60200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d580ccb50 0x7f5d5815de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:39.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.628+0000 7f5d60200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d5815d8a0 0x7f5d5815e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:39.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.628+0000 7f5d60200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5d58153520 con 0x7f5d580ccb50 2024-09-21T07:35:39.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.628+0000 7f5d60200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5d581533a0 con 0x7f5d5815d8a0 2024-09-21T07:35:39.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.628+0000 7f5d60200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f5d581536a0 con 0x7f5d580cc1e0 2024-09-21T07:35:39.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.629+0000 7f5d5ce00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d580ccb50 0x7f5d5815de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:39.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.629+0000 7f5d5ce00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d580ccb50 0x7f5d5815de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36224/0 (socket says 172.21.15.39:36224) 2024-09-21T07:35:39.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.629+0000 7f5d5ce00640 1 -- 172.21.15.39:0/3394260423 learned_addr learned my addr 172.21.15.39:0/3394260423 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:39.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.629+0000 7f5d5d800640 1 --2- 172.21.15.39:0/3394260423 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5d580cc1e0 0x7f5d580ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:39.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.629+0000 7f5d5e200640 1 --2- 172.21.15.39:0/3394260423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d5815d8a0 0x7f5d5815e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:39.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.629+0000 7f5d5ce00640 1 -- 172.21.15.39:0/3394260423 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5d580cc1e0 msgr2=0x7f5d580ae3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:39.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.629+0000 7f5d5ce00640 1 --2- 172.21.15.39:0/3394260423 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5d580cc1e0 0x7f5d580ae3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:39.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.629+0000 7f5d5ce00640 1 -- 172.21.15.39:0/3394260423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d5815d8a0 msgr2=0x7f5d5815e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:39.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.629+0000 7f5d5ce00640 1 --2- 172.21.15.39:0/3394260423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d5815d8a0 0x7f5d5815e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:39.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.629+0000 7f5d5ce00640 1 -- 172.21.15.39:0/3394260423 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d580ae8f0 con 0x7f5d580ccb50 2024-09-21T07:35:39.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.629+0000 7f5d5e200640 1 --2- 172.21.15.39:0/3394260423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d5815d8a0 0x7f5d5815e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:35:39.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.630+0000 7f5d5ce00640 1 --2- 172.21.15.39:0/3394260423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d580ccb50 0x7f5d5815de70 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400d770 tx=0x7f5d4400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:39.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.630+0000 7f5d55600640 1 -- 172.21.15.39:0/3394260423 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d44017070 con 0x7f5d580ccb50 2024-09-21T07:35:39.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.630+0000 7f5d55600640 1 -- 172.21.15.39:0/3394260423 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d44009d20 con 0x7f5d580ccb50 2024-09-21T07:35:39.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.630+0000 7f5d55600640 1 -- 172.21.15.39:0/3394260423 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d440106c0 con 0x7f5d580ccb50 2024-09-21T07:35:39.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.630+0000 7f5d60200640 1 -- 172.21.15.39:0/3394260423 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d58160000 con 0x7f5d580ccb50 2024-09-21T07:35:39.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.630+0000 7f5d60200640 1 -- 172.21.15.39:0/3394260423 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d581f68c0 con 0x7f5d580ccb50 2024-09-21T07:35:39.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.631+0000 7f5d60200640 1 -- 172.21.15.39:0/3394260423 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d58153b50 con 0x7f5d580ccb50 2024-09-21T07:35:39.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.632+0000 7f5d55600640 1 -- 172.21.15.39:0/3394260423 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5d4401a070 con 0x7f5d580ccb50 2024-09-21T07:35:39.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.633+0000 7f5d55600640 1 --2- 172.21.15.39:0/3394260423 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5d38078b90 0x7f5d3807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:39.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.634+0000 7f5d55600640 1 -- 172.21.15.39:0/3394260423 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6006+0+0 (secure 0 0 0) 0x7f5d4409afd0 con 0x7f5d580ccb50 2024-09-21T07:35:39.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.634+0000 7f5d5d800640 1 --2- 172.21.15.39:0/3394260423 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5d38078b90 0x7f5d3807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:39.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.635+0000 7f5d5d800640 1 --2- 172.21.15.39:0/3394260423 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5d38078b90 0x7f5d3807b0a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c034000 tx=0x7f5d4c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:39.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.638+0000 7f5d55600640 1 -- 172.21.15.39:0/3394260423 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5d580ae8f0 con 0x7f5d580ccb50 2024-09-21T07:35:39.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.834+0000 7f5d60200640 1 -- 172.21.15.39:0/3394260423 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5d580b1750 con 0x7f5d580ccb50 2024-09-21T07:35:39.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.835+0000 7f5d55600640 1 -- 172.21.15.39:0/3394260423 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+18446 (secure 0 0 0) 0x7f5d44062c80 con 0x7f5d580ccb50 2024-09-21T07:35:39.837 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:35:39.837 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":70,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:35:37.352575+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6834/252473586":"2024-09-22T07:35:34.261951+0000","172.21.15.39:6832/252473586":"2024-09-22T07:35:34.261951+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:35:39.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.838+0000 7f5d60200640 1 -- 172.21.15.39:0/3394260423 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5d38078b90 msgr2=0x7f5d3807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:39.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.838+0000 7f5d60200640 1 --2- 172.21.15.39:0/3394260423 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5d38078b90 0x7f5d3807b0a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c034000 tx=0x7f5d4c004730 comp rx=0 tx=0).stop 2024-09-21T07:35:39.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.838+0000 7f5d60200640 1 -- 172.21.15.39:0/3394260423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d580ccb50 msgr2=0x7f5d5815de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:39.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.838+0000 7f5d60200640 1 --2- 172.21.15.39:0/3394260423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d580ccb50 0x7f5d5815de70 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400d770 tx=0x7f5d4400dc30 comp rx=0 tx=0).stop 2024-09-21T07:35:39.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.838+0000 7f5d60200640 1 -- 172.21.15.39:0/3394260423 shutdown_connections 2024-09-21T07:35:39.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.838+0000 7f5d60200640 1 --2- 172.21.15.39:0/3394260423 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5d38078b90 0x7f5d3807b0a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:39.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.838+0000 7f5d60200640 1 --2- 172.21.15.39:0/3394260423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d5815d8a0 0x7f5d5815e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:39.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.838+0000 7f5d60200640 1 --2- 172.21.15.39:0/3394260423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d580ccb50 0x7f5d5815de70 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:39.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.838+0000 7f5d60200640 1 --2- 172.21.15.39:0/3394260423 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5d580cc1e0 0x7f5d580ae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:39.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.838+0000 7f5d60200640 1 -- 172.21.15.39:0/3394260423 >> 172.21.15.39:0/3394260423 conn(0x7f5d5806d7d0 msgr2=0x7f5d5806ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:39.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.838+0000 7f5d60200640 1 -- 172.21.15.39:0/3394260423 shutdown_connections 2024-09-21T07:35:39.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:39.838+0000 7f5d60200640 1 -- 172.21.15.39:0/3394260423 wait complete. 2024-09-21T07:35:39.865 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T07:33:01.569433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'snap-test-pool', 'create_time': '2024-09-21T07:34:22.032185+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 1, 'snap_epoch': 37, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'test_data_pool', 'create_time': '2024-09-21T07:35:25.604532+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 1, 'snap_epoch': 66, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_metadata_pool', 'create_time': '2024-09-21T07:35:26.327607+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T07:35:39.865 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T07:35:40.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.095+0000 7efd7a800640 1 -- 172.21.15.39:0/874591313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd740cf3e0 msgr2=0x7efd740b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:40.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.095+0000 7efd7a800640 1 --2- 172.21.15.39:0/874591313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd740cf3e0 0x7efd740b3150 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7efd740cfed0 tx=0x7efd6c02a820 comp rx=0 tx=0).stop 2024-09-21T07:35:40.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.095+0000 7efd7a800640 1 -- 172.21.15.39:0/874591313 shutdown_connections 2024-09-21T07:35:40.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.095+0000 7efd7a800640 1 --2- 172.21.15.39:0/874591313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd740cf3e0 0x7efd740b3150 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:40.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.095+0000 7efd7a800640 1 --2- 172.21.15.39:0/874591313 >> v2:172.21.15.148:3300/0 conn(0x7efd740749b0 0x7efd74142ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:40.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.095+0000 7efd7a800640 1 -- 172.21.15.39:0/874591313 >> 172.21.15.39:0/874591313 conn(0x7efd740b1bb0 msgr2=0x7efd740b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:40.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.095+0000 7efd7a800640 1 -- 172.21.15.39:0/874591313 shutdown_connections 2024-09-21T07:35:40.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.095+0000 7efd7a800640 1 -- 172.21.15.39:0/874591313 wait complete. 2024-09-21T07:35:40.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.097+0000 7efd7a800640 1 Processor -- start 2024-09-21T07:35:40.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.098+0000 7efd7a800640 1 -- start start 2024-09-21T07:35:40.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.099+0000 7efd7a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd740749b0 0x7efd7415e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:40.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.099+0000 7efd7a800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd740cf3e0 0x7efd7415ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:40.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.099+0000 7efd7a800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efd74145720 0x7efd741f2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:40.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.099+0000 7efd7a800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efd740b3b90 con 0x7efd740749b0 2024-09-21T07:35:40.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.099+0000 7efd7a800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efd740b3a10 con 0x7efd740cf3e0 2024-09-21T07:35:40.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.099+0000 7efd7a800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7efd740b3d10 con 0x7efd74145720 2024-09-21T07:35:40.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.099+0000 7efd79400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd740749b0 0x7efd7415e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:40.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.099+0000 7efd79400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd740749b0 0x7efd7415e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36252/0 (socket says 172.21.15.39:36252) 2024-09-21T07:35:40.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.099+0000 7efd79400640 1 -- 172.21.15.39:0/102852134 learned_addr learned my addr 172.21.15.39:0/102852134 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:40.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.099+0000 7efd79e00640 1 --2- 172.21.15.39:0/102852134 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efd74145720 0x7efd741f2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:40.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.099+0000 7efd78a00640 1 --2- 172.21.15.39:0/102852134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd740cf3e0 0x7efd7415ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:40.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.100+0000 7efd79400640 1 -- 172.21.15.39:0/102852134 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efd74145720 msgr2=0x7efd741f2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:40.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.100+0000 7efd79400640 1 --2- 172.21.15.39:0/102852134 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efd74145720 0x7efd741f2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:40.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.100+0000 7efd79400640 1 -- 172.21.15.39:0/102852134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd740cf3e0 msgr2=0x7efd7415ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:40.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.100+0000 7efd79400640 1 --2- 172.21.15.39:0/102852134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd740cf3e0 0x7efd7415ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:40.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.100+0000 7efd79400640 1 -- 172.21.15.39:0/102852134 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd741f5980 con 0x7efd740749b0 2024-09-21T07:35:40.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.100+0000 7efd79400640 1 --2- 172.21.15.39:0/102852134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd740749b0 0x7efd7415e330 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7efd6800c3c0 tx=0x7efd68001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:40.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.100+0000 7efd71600640 1 -- 172.21.15.39:0/102852134 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd68017070 con 0x7efd740749b0 2024-09-21T07:35:40.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.100+0000 7efd71600640 1 -- 172.21.15.39:0/102852134 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd680035f0 con 0x7efd740749b0 2024-09-21T07:35:40.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.100+0000 7efd71600640 1 -- 172.21.15.39:0/102852134 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd68003ac0 con 0x7efd740749b0 2024-09-21T07:35:40.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.100+0000 7efd7a800640 1 -- 172.21.15.39:0/102852134 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd741f5c60 con 0x7efd740749b0 2024-09-21T07:35:40.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.101+0000 7efd7a800640 1 -- 172.21.15.39:0/102852134 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd7415ef70 con 0x7efd740749b0 2024-09-21T07:35:40.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.102+0000 7efd7a800640 1 -- 172.21.15.39:0/102852134 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd74073020 con 0x7efd740749b0 2024-09-21T07:35:40.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.103+0000 7efd71600640 1 -- 172.21.15.39:0/102852134 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7efd6801a070 con 0x7efd740749b0 2024-09-21T07:35:40.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.104+0000 7efd71600640 1 --2- 172.21.15.39:0/102852134 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efd54078b90 0x7efd5407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:40.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.104+0000 7efd71600640 1 -- 172.21.15.39:0/102852134 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6006+0+0 (secure 0 0 0) 0x7efd68099f50 con 0x7efd740749b0 2024-09-21T07:35:40.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.104+0000 7efd78a00640 1 --2- 172.21.15.39:0/102852134 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efd54078b90 0x7efd5407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:40.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.105+0000 7efd78a00640 1 --2- 172.21.15.39:0/102852134 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efd54078b90 0x7efd5407b0a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efd740b19d0 tx=0x7efd6000b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:40.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.108+0000 7efd71600640 1 -- 172.21.15.39:0/102852134 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7efd74073020 con 0x7efd740749b0 2024-09-21T07:35:40.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.306+0000 7efd7a800640 1 -- 172.21.15.39:0/102852134 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efd740b4230 con 0x7efd740749b0 2024-09-21T07:35:40.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.306+0000 7efd71600640 1 -- 172.21.15.39:0/102852134 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) ==== 93+0+10 (secure 0 0 0) 0x7efd68061d30 con 0x7efd740749b0 2024-09-21T07:35:40.307 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-09-21T07:35:40.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.308+0000 7efd7a800640 1 -- 172.21.15.39:0/102852134 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efd54078b90 msgr2=0x7efd5407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:40.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.308+0000 7efd7a800640 1 --2- 172.21.15.39:0/102852134 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efd54078b90 0x7efd5407b0a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efd740b19d0 tx=0x7efd6000b2e0 comp rx=0 tx=0).stop 2024-09-21T07:35:40.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.308+0000 7efd7a800640 1 -- 172.21.15.39:0/102852134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd740749b0 msgr2=0x7efd7415e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:40.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.308+0000 7efd7a800640 1 --2- 172.21.15.39:0/102852134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd740749b0 0x7efd7415e330 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7efd6800c3c0 tx=0x7efd68001e70 comp rx=0 tx=0).stop 2024-09-21T07:35:40.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.308+0000 7efd7a800640 1 -- 172.21.15.39:0/102852134 shutdown_connections 2024-09-21T07:35:40.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.308+0000 7efd7a800640 1 --2- 172.21.15.39:0/102852134 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efd54078b90 0x7efd5407b0a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:40.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.308+0000 7efd7a800640 1 --2- 172.21.15.39:0/102852134 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efd74145720 0x7efd741f2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:40.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.308+0000 7efd7a800640 1 --2- 172.21.15.39:0/102852134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd740cf3e0 0x7efd7415ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:40.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.308+0000 7efd7a800640 1 --2- 172.21.15.39:0/102852134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd740749b0 0x7efd7415e330 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:40.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.308+0000 7efd7a800640 1 -- 172.21.15.39:0/102852134 >> 172.21.15.39:0/102852134 conn(0x7efd740b1bb0 msgr2=0x7efd74160a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:40.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.309+0000 7efd7a800640 1 -- 172.21.15.39:0/102852134 shutdown_connections 2024-09-21T07:35:40.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.309+0000 7efd7a800640 1 -- 172.21.15.39:0/102852134 wait complete. 2024-09-21T07:35:40.331 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2024-09-21T07:35:40.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.563+0000 7f0e5dc00640 1 -- 172.21.15.39:0/1837154808 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e3c004d40 msgr2=0x7f0e3c005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:40.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.563+0000 7f0e5dc00640 1 --2- 172.21.15.39:0/1837154808 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e3c004d40 0x7f0e3c005150 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f0e50005fd0 tx=0x7f0e3c005830 comp rx=0 tx=0).stop 2024-09-21T07:35:40.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.563+0000 7f0e57400640 1 -- 172.21.15.39:0/1837154808 reap_dead start 2024-09-21T07:35:40.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.563+0000 7f0e5dc00640 1 -- 172.21.15.39:0/1837154808 shutdown_connections 2024-09-21T07:35:40.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.563+0000 7f0e5dc00640 1 -- 172.21.15.39:0/1837154808 >> 172.21.15.39:0/1837154808 conn(0x7f0e5806d7d0 msgr2=0x7f0e580b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:40.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.564+0000 7f0e5dc00640 1 -- 172.21.15.39:0/1837154808 shutdown_connections 2024-09-21T07:35:40.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.564+0000 7f0e5dc00640 1 -- 172.21.15.39:0/1837154808 wait complete. 2024-09-21T07:35:40.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.566+0000 7f0e5dc00640 1 Processor -- start 2024-09-21T07:35:40.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.566+0000 7f0e5dc00640 1 -- start start 2024-09-21T07:35:40.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.567+0000 7f0e5dc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e3c004d40 0x7f0e581446c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:40.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.567+0000 7f0e5dc00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e581f6c50 0x7f0e581f7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:40.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.567+0000 7f0e5dc00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e581f75a0 0x7f0e58145460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:40.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.567+0000 7f0e5dc00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0e580b4230 con 0x7f0e3c004d40 2024-09-21T07:35:40.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.567+0000 7f0e5dc00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0e580b40b0 con 0x7f0e581f75a0 2024-09-21T07:35:40.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.567+0000 7f0e5dc00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f0e580b43b0 con 0x7f0e581f6c50 2024-09-21T07:35:40.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.568+0000 7f0e56a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e3c004d40 0x7f0e581446c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:40.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.568+0000 7f0e56a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e3c004d40 0x7f0e581446c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36266/0 (socket says 172.21.15.39:36266) 2024-09-21T07:35:40.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.568+0000 7f0e56a00640 1 -- 172.21.15.39:0/235758845 learned_addr learned my addr 172.21.15.39:0/235758845 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:40.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.568+0000 7f0e4fe00640 1 --2- 172.21.15.39:0/235758845 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e581f6c50 0x7f0e581f7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:40.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.568+0000 7f0e57400640 1 --2- 172.21.15.39:0/235758845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e581f75a0 0x7f0e58145460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:40.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.568+0000 7f0e56a00640 1 -- 172.21.15.39:0/235758845 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e581f6c50 msgr2=0x7f0e581f7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:40.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.568+0000 7f0e56a00640 1 --2- 172.21.15.39:0/235758845 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e581f6c50 0x7f0e581f7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:40.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.568+0000 7f0e56a00640 1 -- 172.21.15.39:0/235758845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e581f75a0 msgr2=0x7f0e58145460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:40.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.568+0000 7f0e56a00640 1 --2- 172.21.15.39:0/235758845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e581f75a0 0x7f0e58145460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:40.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.568+0000 7f0e56a00640 1 -- 172.21.15.39:0/235758845 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e5809bc30 con 0x7f0e3c004d40 2024-09-21T07:35:40.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.568+0000 7f0e56a00640 1 --2- 172.21.15.39:0/235758845 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e3c004d40 0x7f0e581446c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f0e48006f80 tx=0x7f0e480049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:40.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.569+0000 7f0e4ea00640 1 -- 172.21.15.39:0/235758845 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e48015070 con 0x7f0e3c004d40 2024-09-21T07:35:40.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.569+0000 7f0e4ea00640 1 -- 172.21.15.39:0/235758845 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e48010980 con 0x7f0e3c004d40 2024-09-21T07:35:40.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.569+0000 7f0e4ea00640 1 -- 172.21.15.39:0/235758845 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e48010db0 con 0x7f0e3c004d40 2024-09-21T07:35:40.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.569+0000 7f0e5dc00640 1 -- 172.21.15.39:0/235758845 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e5809bf10 con 0x7f0e3c004d40 2024-09-21T07:35:40.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.569+0000 7f0e5dc00640 1 -- 172.21.15.39:0/235758845 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e5809c4d0 con 0x7f0e3c004d40 2024-09-21T07:35:40.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.570+0000 7f0e5dc00640 1 -- 172.21.15.39:0/235758845 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e580ccf60 con 0x7f0e3c004d40 2024-09-21T07:35:40.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.571+0000 7f0e4ea00640 1 -- 172.21.15.39:0/235758845 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0e48018070 con 0x7f0e3c004d40 2024-09-21T07:35:40.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.572+0000 7f0e4ea00640 1 --2- 172.21.15.39:0/235758845 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0e34078b90 0x7f0e3407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:40.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.572+0000 7f0e4ea00640 1 -- 172.21.15.39:0/235758845 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6006+0+0 (secure 0 0 0) 0x7f0e4809b630 con 0x7f0e3c004d40 2024-09-21T07:35:40.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.573+0000 7f0e4fe00640 1 --2- 172.21.15.39:0/235758845 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0e34078b90 0x7f0e3407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:40.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.573+0000 7f0e4fe00640 1 --2- 172.21.15.39:0/235758845 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0e34078b90 0x7f0e3407b0a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0e5815a330 tx=0x7f0e40005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:40.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.577+0000 7f0e4ea00640 1 -- 172.21.15.39:0/235758845 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0e580cd4a0 con 0x7f0e3c004d40 2024-09-21T07:35:40.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.773+0000 7f0e5dc00640 1 -- 172.21.15.39:0/235758845 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) -- 0x7f0e58070220 con 0x7f0e3c004d40 2024-09-21T07:35:40.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.774+0000 7f0e4ea00640 1 -- 172.21.15.39:0/235758845 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v70) ==== 103+0+11 (secure 0 0 0) 0x7f0e480639d0 con 0x7f0e3c004d40 2024-09-21T07:35:40.775 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-21T07:35:40.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.777+0000 7f0e5dc00640 1 -- 172.21.15.39:0/235758845 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0e34078b90 msgr2=0x7f0e3407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:40.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.777+0000 7f0e5dc00640 1 --2- 172.21.15.39:0/235758845 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0e34078b90 0x7f0e3407b0a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0e5815a330 tx=0x7f0e40005f90 comp rx=0 tx=0).stop 2024-09-21T07:35:40.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.777+0000 7f0e5dc00640 1 -- 172.21.15.39:0/235758845 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e3c004d40 msgr2=0x7f0e581446c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:40.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.777+0000 7f0e5dc00640 1 --2- 172.21.15.39:0/235758845 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e3c004d40 0x7f0e581446c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f0e48006f80 tx=0x7f0e480049d0 comp rx=0 tx=0).stop 2024-09-21T07:35:40.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.777+0000 7f0e5dc00640 1 -- 172.21.15.39:0/235758845 shutdown_connections 2024-09-21T07:35:40.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.777+0000 7f0e5dc00640 1 --2- 172.21.15.39:0/235758845 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0e34078b90 0x7f0e3407b0a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:40.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.777+0000 7f0e5dc00640 1 --2- 172.21.15.39:0/235758845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e581f75a0 0x7f0e58145460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:40.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.777+0000 7f0e5dc00640 1 --2- 172.21.15.39:0/235758845 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e581f6c50 0x7f0e581f7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:40.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.777+0000 7f0e5dc00640 1 --2- 172.21.15.39:0/235758845 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e3c004d40 0x7f0e581446c0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:40.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.777+0000 7f0e5dc00640 1 -- 172.21.15.39:0/235758845 >> 172.21.15.39:0/235758845 conn(0x7f0e5806d7d0 msgr2=0x7f0e5815f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:40.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.777+0000 7f0e5dc00640 1 -- 172.21.15.39:0/235758845 shutdown_connections 2024-09-21T07:35:40.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:40.777+0000 7f0e5dc00640 1 -- 172.21.15.39:0/235758845 wait complete. 2024-09-21T07:35:40.801 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2024-09-21T07:35:41.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.028+0000 7fa851400640 1 -- 172.21.15.39:0/1911737006 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa84c0cbee0 msgr2=0x7fa84c169450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:41.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.028+0000 7fa851400640 1 --2- 172.21.15.39:0/1911737006 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa84c0cbee0 0x7fa84c169450 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa844009860 tx=0x7fa84402a510 comp rx=0 tx=0).stop 2024-09-21T07:35:41.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.028+0000 7fa851400640 1 -- 172.21.15.39:0/1911737006 shutdown_connections 2024-09-21T07:35:41.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.028+0000 7fa851400640 1 --2- 172.21.15.39:0/1911737006 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa84c0cbee0 0x7fa84c169450 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:41.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.028+0000 7fa851400640 1 -- 172.21.15.39:0/1911737006 >> 172.21.15.39:0/1911737006 conn(0x7fa84c06ee50 msgr2=0x7fa84c06f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:41.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.029+0000 7fa851400640 1 -- 172.21.15.39:0/1911737006 shutdown_connections 2024-09-21T07:35:41.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.029+0000 7fa851400640 1 -- 172.21.15.39:0/1911737006 wait complete. 2024-09-21T07:35:41.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.031+0000 7fa851400640 1 Processor -- start 2024-09-21T07:35:41.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.031+0000 7fa851400640 1 -- start start 2024-09-21T07:35:41.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.033+0000 7fa851400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa84c0c9930 0x7fa84c15e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:41.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.033+0000 7fa851400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa84c0cbee0 0x7fa84c13ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:41.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.033+0000 7fa851400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa84c144b00 0x7fa84c13b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:41.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.033+0000 7fa851400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c169ba0 con 0x7fa84c0cbee0 2024-09-21T07:35:41.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.033+0000 7fa851400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c0cc480 con 0x7fa84c0c9930 2024-09-21T07:35:41.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.033+0000 7fa851400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa84c169d20 con 0x7fa84c144b00 2024-09-21T07:35:41.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.033+0000 7fa843400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa84c0cbee0 0x7fa84c13ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:41.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.033+0000 7fa843400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa84c0cbee0 0x7fa84c13ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36276/0 (socket says 172.21.15.39:36276) 2024-09-21T07:35:41.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.033+0000 7fa843400640 1 -- 172.21.15.39:0/1845950165 learned_addr learned my addr 172.21.15.39:0/1845950165 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:41.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.033+0000 7fa850a00640 1 --2- 172.21.15.39:0/1845950165 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa84c144b00 0x7fa84c13b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:41.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.033+0000 7fa84be00640 1 --2- 172.21.15.39:0/1845950165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa84c0c9930 0x7fa84c15e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:41.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.034+0000 7fa843400640 1 -- 172.21.15.39:0/1845950165 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa84c144b00 msgr2=0x7fa84c13b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:41.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.034+0000 7fa843400640 1 --2- 172.21.15.39:0/1845950165 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa84c144b00 0x7fa84c13b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:41.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.034+0000 7fa843400640 1 -- 172.21.15.39:0/1845950165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa84c0c9930 msgr2=0x7fa84c15e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:41.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.034+0000 7fa843400640 1 --2- 172.21.15.39:0/1845950165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa84c0c9930 0x7fa84c15e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:41.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.034+0000 7fa843400640 1 -- 172.21.15.39:0/1845950165 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa84c1602a0 con 0x7fa84c0cbee0 2024-09-21T07:35:41.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.034+0000 7fa84be00640 1 --2- 172.21.15.39:0/1845950165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa84c0c9930 0x7fa84c15e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:35:41.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.034+0000 7fa843400640 1 --2- 172.21.15.39:0/1845950165 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa84c0cbee0 0x7fa84c13ab00 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa83400c3f0 tx=0x7fa83400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:41.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.034+0000 7fa848c00640 1 -- 172.21.15.39:0/1845950165 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa834017070 con 0x7fa84c0cbee0 2024-09-21T07:35:41.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.034+0000 7fa848c00640 1 -- 172.21.15.39:0/1845950165 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8340035f0 con 0x7fa84c0cbee0 2024-09-21T07:35:41.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.034+0000 7fa851400640 1 -- 172.21.15.39:0/1845950165 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa84c1f8c80 con 0x7fa84c0cbee0 2024-09-21T07:35:41.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.034+0000 7fa848c00640 1 -- 172.21.15.39:0/1845950165 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa834003a40 con 0x7fa84c0cbee0 2024-09-21T07:35:41.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.034+0000 7fa851400640 1 -- 172.21.15.39:0/1845950165 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa84c1f9240 con 0x7fa84c0cbee0 2024-09-21T07:35:41.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.035+0000 7fa851400640 1 -- 172.21.15.39:0/1845950165 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa84c0ce950 con 0x7fa84c0cbee0 2024-09-21T07:35:41.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.037+0000 7fa848c00640 1 -- 172.21.15.39:0/1845950165 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa83401a070 con 0x7fa84c0cbee0 2024-09-21T07:35:41.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.038+0000 7fa848c00640 1 --2- 172.21.15.39:0/1845950165 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa828078bc0 0x7fa82807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:41.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.039+0000 7fa848c00640 1 -- 172.21.15.39:0/1845950165 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6006+0+0 (secure 0 0 0) 0x7fa83409a230 con 0x7fa84c0cbee0 2024-09-21T07:35:41.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.039+0000 7fa84be00640 1 --2- 172.21.15.39:0/1845950165 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa828078bc0 0x7fa82807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:41.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.040+0000 7fa84be00640 1 --2- 172.21.15.39:0/1845950165 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa828078bc0 0x7fa82807b0d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa83c0015a0 tx=0x7fa83c009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:41.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.042+0000 7fa848c00640 1 -- 172.21.15.39:0/1845950165 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa8340624b0 con 0x7fa84c0cbee0 2024-09-21T07:35:41.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.237+0000 7fa851400640 1 -- 172.21.15.39:0/1845950165 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) -- 0x7fa84c074ef0 con 0x7fa84c0cbee0 2024-09-21T07:35:41.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.238+0000 7fa848c00640 1 -- 172.21.15.39:0/1845950165 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v70) ==== 103+0+11 (secure 0 0 0) 0x7fa834061db0 con 0x7fa84c0cbee0 2024-09-21T07:35:41.239 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-21T07:35:41.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.240+0000 7fa851400640 1 -- 172.21.15.39:0/1845950165 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa828078bc0 msgr2=0x7fa82807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:41.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.240+0000 7fa851400640 1 --2- 172.21.15.39:0/1845950165 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa828078bc0 0x7fa82807b0d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa83c0015a0 tx=0x7fa83c009040 comp rx=0 tx=0).stop 2024-09-21T07:35:41.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.240+0000 7fa851400640 1 -- 172.21.15.39:0/1845950165 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa84c0cbee0 msgr2=0x7fa84c13ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:41.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.240+0000 7fa851400640 1 --2- 172.21.15.39:0/1845950165 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa84c0cbee0 0x7fa84c13ab00 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa83400c3f0 tx=0x7fa83400c8b0 comp rx=0 tx=0).stop 2024-09-21T07:35:41.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.240+0000 7fa851400640 1 -- 172.21.15.39:0/1845950165 shutdown_connections 2024-09-21T07:35:41.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.240+0000 7fa851400640 1 --2- 172.21.15.39:0/1845950165 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa828078bc0 0x7fa82807b0d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:41.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.240+0000 7fa851400640 1 --2- 172.21.15.39:0/1845950165 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa84c144b00 0x7fa84c13b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:41.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.240+0000 7fa851400640 1 --2- 172.21.15.39:0/1845950165 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa84c0cbee0 0x7fa84c13ab00 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:41.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.240+0000 7fa851400640 1 --2- 172.21.15.39:0/1845950165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa84c0c9930 0x7fa84c15e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:41.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.240+0000 7fa851400640 1 -- 172.21.15.39:0/1845950165 >> 172.21.15.39:0/1845950165 conn(0x7fa84c06ee50 msgr2=0x7fa84c13d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:41.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.240+0000 7fa851400640 1 -- 172.21.15.39:0/1845950165 shutdown_connections 2024-09-21T07:35:41.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.240+0000 7fa851400640 1 -- 172.21.15.39:0/1845950165 wait complete. 2024-09-21T07:35:41.263 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2024-09-21T07:35:41.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.491+0000 7fae72800640 1 -- 172.21.15.39:0/2724324793 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae6c074c70 msgr2=0x7fae6c0702e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:41.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.491+0000 7fae72800640 1 --2- 172.21.15.39:0/2724324793 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae6c074c70 0x7fae6c0702e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fae64009be0 tx=0x7fae6402a890 comp rx=0 tx=0).stop 2024-09-21T07:35:41.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.491+0000 7fae72800640 1 -- 172.21.15.39:0/2724324793 shutdown_connections 2024-09-21T07:35:41.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.491+0000 7fae72800640 1 --2- 172.21.15.39:0/2724324793 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae6c074c70 0x7fae6c0702e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:41.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.491+0000 7fae72800640 1 -- 172.21.15.39:0/2724324793 >> 172.21.15.39:0/2724324793 conn(0x7fae6c0cdb20 msgr2=0x7fae6c0cdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:41.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.491+0000 7fae72800640 1 -- 172.21.15.39:0/2724324793 shutdown_connections 2024-09-21T07:35:41.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.491+0000 7fae72800640 1 -- 172.21.15.39:0/2724324793 wait complete. 2024-09-21T07:35:41.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.494+0000 7fae72800640 1 Processor -- start 2024-09-21T07:35:41.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.494+0000 7fae72800640 1 -- start start 2024-09-21T07:35:41.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.495+0000 7fae72800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae6c074c70 0x7fae6c0d0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:41.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.495+0000 7fae72800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae6c15d1e0 0x7fae6c155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:41.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.495+0000 7fae72800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae6c1559c0 0x7fae6c1541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:41.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.495+0000 7fae72800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c0cc690 con 0x7fae6c074c70 2024-09-21T07:35:41.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.495+0000 7fae72800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c0cc510 con 0x7fae6c1559c0 2024-09-21T07:35:41.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.495+0000 7fae72800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fae6c0cc810 con 0x7fae6c15d1e0 2024-09-21T07:35:41.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.495+0000 7fae71400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae6c074c70 0x7fae6c0d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:41.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.495+0000 7fae71400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae6c074c70 0x7fae6c0d0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36282/0 (socket says 172.21.15.39:36282) 2024-09-21T07:35:41.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.495+0000 7fae71400640 1 -- 172.21.15.39:0/1384273054 learned_addr learned my addr 172.21.15.39:0/1384273054 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:41.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.496+0000 7fae70a00640 1 --2- 172.21.15.39:0/1384273054 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae6c15d1e0 0x7fae6c155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:41.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.496+0000 7fae71e00640 1 --2- 172.21.15.39:0/1384273054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae6c1559c0 0x7fae6c1541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:41.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.496+0000 7fae71400640 1 -- 172.21.15.39:0/1384273054 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae6c15d1e0 msgr2=0x7fae6c155480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:41.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.496+0000 7fae71400640 1 --2- 172.21.15.39:0/1384273054 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae6c15d1e0 0x7fae6c155480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:41.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.496+0000 7fae71400640 1 -- 172.21.15.39:0/1384273054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae6c1559c0 msgr2=0x7fae6c1541d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:41.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.496+0000 7fae71400640 1 --2- 172.21.15.39:0/1384273054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae6c1559c0 0x7fae6c1541d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:41.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.496+0000 7fae71400640 1 -- 172.21.15.39:0/1384273054 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae6c1f5380 con 0x7fae6c074c70 2024-09-21T07:35:41.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.496+0000 7fae71400640 1 --2- 172.21.15.39:0/1384273054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae6c074c70 0x7fae6c0d0480 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fae60001eb0 tx=0x7fae6000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:41.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.496+0000 7fae69600640 1 -- 172.21.15.39:0/1384273054 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae60011790 con 0x7fae6c074c70 2024-09-21T07:35:41.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.497+0000 7fae69600640 1 -- 172.21.15.39:0/1384273054 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae60017070 con 0x7fae6c074c70 2024-09-21T07:35:41.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.497+0000 7fae72800640 1 -- 172.21.15.39:0/1384273054 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae6c1f5660 con 0x7fae6c074c70 2024-09-21T07:35:41.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.497+0000 7fae69600640 1 -- 172.21.15.39:0/1384273054 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae60011ce0 con 0x7fae6c074c70 2024-09-21T07:35:41.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.497+0000 7fae72800640 1 -- 172.21.15.39:0/1384273054 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae6c1f5080 con 0x7fae6c074c70 2024-09-21T07:35:41.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.498+0000 7fae72800640 1 -- 172.21.15.39:0/1384273054 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae6c0ccc90 con 0x7fae6c074c70 2024-09-21T07:35:41.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.499+0000 7fae69600640 1 -- 172.21.15.39:0/1384273054 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fae6001c460 con 0x7fae6c074c70 2024-09-21T07:35:41.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.501+0000 7fae69600640 1 --2- 172.21.15.39:0/1384273054 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae4c078b90 0x7fae4c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:41.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.501+0000 7fae69600640 1 -- 172.21.15.39:0/1384273054 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6006+0+0 (secure 0 0 0) 0x7fae6009abc0 con 0x7fae6c074c70 2024-09-21T07:35:41.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.501+0000 7fae70a00640 1 --2- 172.21.15.39:0/1384273054 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae4c078b90 0x7fae4c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:41.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.502+0000 7fae70a00640 1 --2- 172.21.15.39:0/1384273054 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae4c078b90 0x7fae4c07b0a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fae6c0ca550 tx=0x7fae58009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:41.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.505+0000 7fae69600640 1 -- 172.21.15.39:0/1384273054 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fae60062870 con 0x7fae6c074c70 2024-09-21T07:35:41.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.700+0000 7fae72800640 1 -- 172.21.15.39:0/1384273054 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) -- 0x7fae6c15fe90 con 0x7fae6c074c70 2024-09-21T07:35:41.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.701+0000 7fae69600640 1 -- 172.21.15.39:0/1384273054 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v70) ==== 107+0+11 (secure 0 0 0) 0x7fae600672a0 con 0x7fae6c074c70 2024-09-21T07:35:41.702 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-21T07:35:41.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.703+0000 7fae72800640 1 -- 172.21.15.39:0/1384273054 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae4c078b90 msgr2=0x7fae4c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:41.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.703+0000 7fae72800640 1 --2- 172.21.15.39:0/1384273054 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae4c078b90 0x7fae4c07b0a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fae6c0ca550 tx=0x7fae58009040 comp rx=0 tx=0).stop 2024-09-21T07:35:41.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.703+0000 7fae72800640 1 -- 172.21.15.39:0/1384273054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae6c074c70 msgr2=0x7fae6c0d0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:41.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.703+0000 7fae72800640 1 --2- 172.21.15.39:0/1384273054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae6c074c70 0x7fae6c0d0480 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fae60001eb0 tx=0x7fae6000c820 comp rx=0 tx=0).stop 2024-09-21T07:35:41.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.703+0000 7fae72800640 1 -- 172.21.15.39:0/1384273054 shutdown_connections 2024-09-21T07:35:41.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.703+0000 7fae72800640 1 --2- 172.21.15.39:0/1384273054 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae4c078b90 0x7fae4c07b0a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:41.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.703+0000 7fae72800640 1 --2- 172.21.15.39:0/1384273054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae6c1559c0 0x7fae6c1541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:41.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.703+0000 7fae72800640 1 --2- 172.21.15.39:0/1384273054 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae6c15d1e0 0x7fae6c155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:41.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.703+0000 7fae72800640 1 --2- 172.21.15.39:0/1384273054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae6c074c70 0x7fae6c0d0480 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:41.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.703+0000 7fae72800640 1 -- 172.21.15.39:0/1384273054 >> 172.21.15.39:0/1384273054 conn(0x7fae6c0cdb20 msgr2=0x7fae6c1565b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:41.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.703+0000 7fae72800640 1 -- 172.21.15.39:0/1384273054 shutdown_connections 2024-09-21T07:35:41.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.703+0000 7fae72800640 1 -- 172.21.15.39:0/1384273054 wait complete. 2024-09-21T07:35:41.726 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_allow_new_snaps_config' 2024-09-21T07:35:41.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.952+0000 7f8907000640 1 -- 172.21.15.39:0/320413308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89000cfb90 msgr2=0x7f89000cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:41.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.952+0000 7f8907000640 1 --2- 172.21.15.39:0/320413308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89000cfb90 0x7f89000cf010 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f88f8005fd0 tx=0x7f88f802bbe0 comp rx=0 tx=0).stop 2024-09-21T07:35:41.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.952+0000 7f8907000640 1 -- 172.21.15.39:0/320413308 shutdown_connections 2024-09-21T07:35:41.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.952+0000 7f8907000640 1 --2- 172.21.15.39:0/320413308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89000cfb90 0x7f89000cf010 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:41.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.952+0000 7f8907000640 1 -- 172.21.15.39:0/320413308 >> 172.21.15.39:0/320413308 conn(0x7f89000b1cb0 msgr2=0x7f89000b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:41.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.952+0000 7f8907000640 1 -- 172.21.15.39:0/320413308 shutdown_connections 2024-09-21T07:35:41.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.952+0000 7f8907000640 1 -- 172.21.15.39:0/320413308 wait complete. 2024-09-21T07:35:41.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.954+0000 7f8907000640 1 Processor -- start 2024-09-21T07:35:41.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.954+0000 7f8907000640 1 -- start start 2024-09-21T07:35:41.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.956+0000 7f8907000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89000cf660 0x7f8900157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:41.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.956+0000 7f8907000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89000cfb90 0x7f89000b4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:41.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.956+0000 7f8907000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f890009c420 0x7f89000b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:41.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.956+0000 7f8907000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f89000710a0 con 0x7f89000cf660 2024-09-21T07:35:41.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.956+0000 7f8907000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8900070f20 con 0x7f89000cfb90 2024-09-21T07:35:41.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.956+0000 7f8907000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f89000b36c0 con 0x7f890009c420 2024-09-21T07:35:41.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.956+0000 7f88ffe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89000cf660 0x7f8900157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:41.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.956+0000 7f88ffe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89000cf660 0x7f8900157350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36288/0 (socket says 172.21.15.39:36288) 2024-09-21T07:35:41.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.956+0000 7f88ffe00640 1 -- 172.21.15.39:0/2729157945 learned_addr learned my addr 172.21.15.39:0/2729157945 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:41.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.956+0000 7f8905000640 1 --2- 172.21.15.39:0/2729157945 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f890009c420 0x7f89000b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:41.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.956+0000 7f88ff400640 1 --2- 172.21.15.39:0/2729157945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89000cfb90 0x7f89000b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:41.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.956+0000 7f88ffe00640 1 -- 172.21.15.39:0/2729157945 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f890009c420 msgr2=0x7f89000b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:41.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.956+0000 7f88ffe00640 1 --2- 172.21.15.39:0/2729157945 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f890009c420 0x7f89000b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:41.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.956+0000 7f88ffe00640 1 -- 172.21.15.39:0/2729157945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89000cfb90 msgr2=0x7f89000b4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:41.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.956+0000 7f88ffe00640 1 --2- 172.21.15.39:0/2729157945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89000cfb90 0x7f89000b4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:41.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.956+0000 7f88ffe00640 1 -- 172.21.15.39:0/2729157945 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f890009bc90 con 0x7f89000cf660 2024-09-21T07:35:41.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.956+0000 7f8905000640 1 --2- 172.21.15.39:0/2729157945 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f890009c420 0x7f89000b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:35:41.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.956+0000 7f88ff400640 1 --2- 172.21.15.39:0/2729157945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89000cfb90 0x7f89000b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:35:41.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.957+0000 7f88ffe00640 1 --2- 172.21.15.39:0/2729157945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89000cf660 0x7f8900157350 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f88f000c400 tx=0x7f88f000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:41.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.957+0000 7f88f7e00640 1 -- 172.21.15.39:0/2729157945 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f0011790 con 0x7f89000cf660 2024-09-21T07:35:41.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.957+0000 7f88f7e00640 1 -- 172.21.15.39:0/2729157945 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88f0017070 con 0x7f89000cf660 2024-09-21T07:35:41.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.957+0000 7f8907000640 1 -- 172.21.15.39:0/2729157945 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f890009bf70 con 0x7f89000cf660 2024-09-21T07:35:41.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.957+0000 7f88f7e00640 1 -- 172.21.15.39:0/2729157945 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f0011cc0 con 0x7f89000cf660 2024-09-21T07:35:41.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.957+0000 7f8907000640 1 -- 172.21.15.39:0/2729157945 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89001e8ef0 con 0x7f89000cf660 2024-09-21T07:35:41.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.958+0000 7f8907000640 1 -- 172.21.15.39:0/2729157945 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89000b3b70 con 0x7f89000cf660 2024-09-21T07:35:41.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.959+0000 7f88f7e00640 1 -- 172.21.15.39:0/2729157945 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f88f001c460 con 0x7f89000cf660 2024-09-21T07:35:41.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.960+0000 7f88f7e00640 1 --2- 172.21.15.39:0/2729157945 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f88e0078b90 0x7f88e007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:41.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.960+0000 7f88f7e00640 1 -- 172.21.15.39:0/2729157945 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6006+0+0 (secure 0 0 0) 0x7f88f009ac40 con 0x7f89000cf660 2024-09-21T07:35:41.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.961+0000 7f88ff400640 1 --2- 172.21.15.39:0/2729157945 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f88e0078b90 0x7f88e007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:41.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.962+0000 7f88ff400640 1 --2- 172.21.15.39:0/2729157945 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f88e0078b90 0x7f88e007b0a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f89000af520 tx=0x7f88ec009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:41.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:41.965+0000 7f88f7e00640 1 -- 172.21.15.39:0/2729157945 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f88f0063010 con 0x7f89000cf660 2024-09-21T07:35:42.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.237+0000 7f8907000640 1 -- 172.21.15.39:0/2729157945 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_allow_new_snaps_config"]} v 0) -- 0x7f890009c140 con 0x7f89000cf660 2024-09-21T07:35:42.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.383+0000 7f88f7e00640 1 -- 172.21.15.39:0/2729157945 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_allow_new_snaps_config"]}]=0 v160) ==== 151+0+0 (secure 0 0 0) 0x7f88f0062910 con 0x7f89000cf660 2024-09-21T07:35:42.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.386+0000 7f8907000640 1 -- 172.21.15.39:0/2729157945 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f88e0078b90 msgr2=0x7f88e007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:42.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.386+0000 7f8907000640 1 --2- 172.21.15.39:0/2729157945 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f88e0078b90 0x7f88e007b0a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f89000af520 tx=0x7f88ec009040 comp rx=0 tx=0).stop 2024-09-21T07:35:42.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.386+0000 7f8907000640 1 -- 172.21.15.39:0/2729157945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89000cf660 msgr2=0x7f8900157350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:42.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.386+0000 7f8907000640 1 --2- 172.21.15.39:0/2729157945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89000cf660 0x7f8900157350 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f88f000c400 tx=0x7f88f000c8c0 comp rx=0 tx=0).stop 2024-09-21T07:35:42.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.386+0000 7f8907000640 1 -- 172.21.15.39:0/2729157945 shutdown_connections 2024-09-21T07:35:42.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.386+0000 7f8907000640 1 --2- 172.21.15.39:0/2729157945 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f88e0078b90 0x7f88e007b0a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:42.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.386+0000 7f8907000640 1 --2- 172.21.15.39:0/2729157945 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f890009c420 0x7f89000b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:42.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.386+0000 7f8907000640 1 --2- 172.21.15.39:0/2729157945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89000cfb90 0x7f89000b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:42.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.386+0000 7f8907000640 1 --2- 172.21.15.39:0/2729157945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89000cf660 0x7f8900157350 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:42.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.386+0000 7f8907000640 1 -- 172.21.15.39:0/2729157945 >> 172.21.15.39:0/2729157945 conn(0x7f89000b1cb0 msgr2=0x7f89000b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:42.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.386+0000 7f8907000640 1 -- 172.21.15.39:0/2729157945 shutdown_connections 2024-09-21T07:35:42.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.386+0000 7f8907000640 1 -- 172.21.15.39:0/2729157945 wait complete. 2024-09-21T07:35:42.410 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T07:35:42.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.636+0000 7fa78b200640 1 -- 172.21.15.39:0/1973680347 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa78c0d0500 msgr2=0x7fa78c0cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:42.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.636+0000 7fa78b200640 1 --2- 172.21.15.39:0/1973680347 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa78c0d0500 0x7fa78c0cecc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa77c009830 tx=0x7fa77c02a510 comp rx=0 tx=0).stop 2024-09-21T07:35:42.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.636+0000 7fa78b200640 1 -- 172.21.15.39:0/1973680347 shutdown_connections 2024-09-21T07:35:42.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.636+0000 7fa78b200640 1 --2- 172.21.15.39:0/1973680347 >> v2:172.21.15.148:3300/0 conn(0x7fa78c0cf200 0x7fa78c15ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:42.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.636+0000 7fa78b200640 1 --2- 172.21.15.39:0/1973680347 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa78c0d0500 0x7fa78c0cecc0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:42.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.636+0000 7fa78b200640 1 -- 172.21.15.39:0/1973680347 >> 172.21.15.39:0/1973680347 conn(0x7fa78c06d870 msgr2=0x7fa78c06ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:42.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.636+0000 7fa78b200640 1 -- 172.21.15.39:0/1973680347 shutdown_connections 2024-09-21T07:35:42.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.636+0000 7fa78b200640 1 -- 172.21.15.39:0/1973680347 wait complete. 2024-09-21T07:35:42.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.639+0000 7fa78b200640 1 Processor -- start 2024-09-21T07:35:42.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.639+0000 7fa78b200640 1 -- start start 2024-09-21T07:35:42.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.640+0000 7fa78b200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa78c0cf200 0x7fa78c143de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:42.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.640+0000 7fa78b200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa78c0d0500 0x7fa78c0c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:42.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.640+0000 7fa78b200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa78c15bf40 0x7fa78c1f3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:42.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.640+0000 7fa78b200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa78c0cea10 con 0x7fa78c0d0500 2024-09-21T07:35:42.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.640+0000 7fa78b200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa78c15c560 con 0x7fa78c15bf40 2024-09-21T07:35:42.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.640+0000 7fa78b200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa78c15f700 con 0x7fa78c0cf200 2024-09-21T07:35:42.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.640+0000 7fa789400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa78c0d0500 0x7fa78c0c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:42.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.641+0000 7fa789400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa78c0d0500 0x7fa78c0c8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56902/0 (socket says 172.21.15.39:56902) 2024-09-21T07:35:42.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.641+0000 7fa789400640 1 -- 172.21.15.39:0/2037718810 learned_addr learned my addr 172.21.15.39:0/2037718810 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:42.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.641+0000 7fa789e00640 1 --2- 172.21.15.39:0/2037718810 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa78c0cf200 0x7fa78c143de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:42.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.641+0000 7fa789400640 1 -- 172.21.15.39:0/2037718810 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa78c0cf200 msgr2=0x7fa78c143de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:42.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.641+0000 7fa789400640 1 --2- 172.21.15.39:0/2037718810 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa78c0cf200 0x7fa78c143de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:42.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.641+0000 7fa78a800640 1 --2- 172.21.15.39:0/2037718810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa78c15bf40 0x7fa78c1f3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:42.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.641+0000 7fa789400640 1 -- 172.21.15.39:0/2037718810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa78c15bf40 msgr2=0x7fa78c1f3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:42.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.641+0000 7fa789400640 1 --2- 172.21.15.39:0/2037718810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa78c15bf40 0x7fa78c1f3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:42.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.641+0000 7fa789400640 1 -- 172.21.15.39:0/2037718810 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa78c0c9be0 con 0x7fa78c0d0500 2024-09-21T07:35:42.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.641+0000 7fa78a800640 1 --2- 172.21.15.39:0/2037718810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa78c15bf40 0x7fa78c1f3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:35:42.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.641+0000 7fa789400640 1 --2- 172.21.15.39:0/2037718810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa78c0d0500 0x7fa78c0c8b70 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fa77800e790 tx=0x7fa77800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:42.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.642+0000 7fa782000640 1 -- 172.21.15.39:0/2037718810 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77800cac0 con 0x7fa78c0d0500 2024-09-21T07:35:42.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.642+0000 7fa78b200640 1 -- 172.21.15.39:0/2037718810 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa78c0c8090 con 0x7fa78c0d0500 2024-09-21T07:35:42.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.642+0000 7fa782000640 1 -- 172.21.15.39:0/2037718810 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa77800cc60 con 0x7fa78c0d0500 2024-09-21T07:35:42.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.642+0000 7fa782000640 1 -- 172.21.15.39:0/2037718810 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa778005280 con 0x7fa78c0d0500 2024-09-21T07:35:42.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.642+0000 7fa78b200640 1 -- 172.21.15.39:0/2037718810 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa78c0c8650 con 0x7fa78c0d0500 2024-09-21T07:35:42.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.643+0000 7fa78b200640 1 -- 172.21.15.39:0/2037718810 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa78c0c90b0 con 0x7fa78c0d0500 2024-09-21T07:35:42.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.644+0000 7fa782000640 1 -- 172.21.15.39:0/2037718810 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa778005420 con 0x7fa78c0d0500 2024-09-21T07:35:42.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.645+0000 7fa782000640 1 --2- 172.21.15.39:0/2037718810 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa76c078b90 0x7fa76c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:42.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.645+0000 7fa782000640 1 -- 172.21.15.39:0/2037718810 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6006+0+0 (secure 0 0 0) 0x7fa77809b7f0 con 0x7fa78c0d0500 2024-09-21T07:35:42.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.646+0000 7fa789e00640 1 --2- 172.21.15.39:0/2037718810 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa76c078b90 0x7fa76c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:42.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.646+0000 7fa789e00640 1 --2- 172.21.15.39:0/2037718810 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa76c078b90 0x7fa76c07b0a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa77c034000 tx=0x7fa77c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:42.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.650+0000 7fa782000640 1 -- 172.21.15.39:0/2037718810 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa778063a80 con 0x7fa78c0d0500 2024-09-21T07:35:42.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.848+0000 7fa78b200640 1 -- 172.21.15.39:0/2037718810 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fa78c15fbd0 con 0x7fa78c0d0500 2024-09-21T07:35:42.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.848+0000 7fa782000640 1 -- 172.21.15.39:0/2037718810 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v38) ==== 59+0+2294 (secure 0 0 0) 0x7fa778063380 con 0x7fa78c0d0500 2024-09-21T07:35:42.849 INFO:teuthology.orchestra.run.smithi039.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T07:35:42.849 INFO:teuthology.orchestra.run.smithi039.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T07:35:42.850 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced client_mount_timeout 30 2024-09-21T07:35:42.850 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_client 20/20 2024-09-21T07:35:42.850 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_mgr 20/20 2024-09-21T07:35:42.850 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_ms 1/1 2024-09-21T07:35:42.850 INFO:teuthology.orchestra.run.smithi039.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T07:35:42.850 INFO:teuthology.orchestra.run.smithi039.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T07:35:42.850 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3421.881759 2024-09-21T07:35:42.850 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3520.252978 2024-09-21T07:35:42.850 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3511.183316 2024-09-21T07:35:42.850 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3829.631847 2024-09-21T07:35:42.850 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 5474.545872 2024-09-21T07:35:42.850 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4013.628559 2024-09-21T07:35:42.850 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4230.187288 2024-09-21T07:35:42.851 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4148.983107 2024-09-21T07:35:42.851 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_mds 20/20 2024-09-21T07:35:42.851 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T07:35:42.851 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_ms 2/2 2024-09-21T07:35:42.851 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_debug_frag true 2024-09-21T07:35:42.851 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_debug_scatterstat true 2024-09-21T07:35:42.851 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T07:35:42.851 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_verify_scatter true 2024-09-21T07:35:42.851 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T07:35:42.851 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T07:35:42.851 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T07:35:42.851 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced client_mount_timeout 600 2024-09-21T07:35:42.851 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced debug_client 20/20 2024-09-21T07:35:42.851 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced debug_ms 1/1 2024-09-21T07:35:42.852 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T07:35:42.852 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T07:35:42.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.851+0000 7fa78b200640 1 -- 172.21.15.39:0/2037718810 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa76c078b90 msgr2=0x7fa76c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:42.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.851+0000 7fa78b200640 1 --2- 172.21.15.39:0/2037718810 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa76c078b90 0x7fa76c07b0a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa77c034000 tx=0x7fa77c004730 comp rx=0 tx=0).stop 2024-09-21T07:35:42.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.851+0000 7fa78b200640 1 -- 172.21.15.39:0/2037718810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa78c0d0500 msgr2=0x7fa78c0c8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:42.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.851+0000 7fa78b200640 1 --2- 172.21.15.39:0/2037718810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa78c0d0500 0x7fa78c0c8b70 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fa77800e790 tx=0x7fa77800ec50 comp rx=0 tx=0).stop 2024-09-21T07:35:42.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.851+0000 7fa78b200640 1 -- 172.21.15.39:0/2037718810 shutdown_connections 2024-09-21T07:35:42.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.851+0000 7fa78b200640 1 --2- 172.21.15.39:0/2037718810 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa76c078b90 0x7fa76c07b0a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:42.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.851+0000 7fa78b200640 1 --2- 172.21.15.39:0/2037718810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa78c15bf40 0x7fa78c1f3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:42.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.851+0000 7fa78b200640 1 --2- 172.21.15.39:0/2037718810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa78c0d0500 0x7fa78c0c8b70 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:42.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.851+0000 7fa78b200640 1 --2- 172.21.15.39:0/2037718810 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa78c0cf200 0x7fa78c143de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:42.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.851+0000 7fa78b200640 1 -- 172.21.15.39:0/2037718810 >> 172.21.15.39:0/2037718810 conn(0x7fa78c06d870 msgr2=0x7fa78c0b1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:42.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.851+0000 7fa78b200640 1 -- 172.21.15.39:0/2037718810 shutdown_connections 2024-09-21T07:35:42.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:42.851+0000 7fa78b200640 1 -- 172.21.15.39:0/2037718810 wait complete. 2024-09-21T07:35:42.874 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T07:35:43.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.103+0000 7fefad000640 1 -- 172.21.15.39:0/3362234402 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefa8070030 msgr2=0x7fefa8074640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:43.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.103+0000 7fefad000640 1 --2- 172.21.15.39:0/3362234402 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefa8070030 0x7fefa8074640 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fefa8070e70 tx=0x7fef9402a4d0 comp rx=0 tx=0).stop 2024-09-21T07:35:43.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.103+0000 7fefad000640 1 -- 172.21.15.39:0/3362234402 shutdown_connections 2024-09-21T07:35:43.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.103+0000 7fefad000640 1 --2- 172.21.15.39:0/3362234402 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefa8070030 0x7fefa8074640 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:43.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.103+0000 7fefad000640 1 --2- 172.21.15.39:0/3362234402 >> v2:172.21.15.148:3300/0 conn(0x7fefa8073170 0x7fefa815c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:43.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.103+0000 7fefad000640 1 -- 172.21.15.39:0/3362234402 >> 172.21.15.39:0/3362234402 conn(0x7fefa80b1cb0 msgr2=0x7fefa80b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:43.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.103+0000 7fefad000640 1 -- 172.21.15.39:0/3362234402 shutdown_connections 2024-09-21T07:35:43.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.103+0000 7fefad000640 1 -- 172.21.15.39:0/3362234402 wait complete. 2024-09-21T07:35:43.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.105+0000 7fefad000640 1 Processor -- start 2024-09-21T07:35:43.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.105+0000 7fefad000640 1 -- start start 2024-09-21T07:35:43.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.107+0000 7fefad000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefa8070030 0x7fefa81f68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:43.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.107+0000 7fefad000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefa8070440 0x7fefa80c9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:43.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.107+0000 7fefad000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefa8073170 0x7fefa80cb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:43.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.107+0000 7fefad000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fefa8075050 con 0x7fefa8070440 2024-09-21T07:35:43.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.107+0000 7fefad000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fefa8074ed0 con 0x7fefa8073170 2024-09-21T07:35:43.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.107+0000 7fefad000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fefa806f5a0 con 0x7fefa8070030 2024-09-21T07:35:43.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.107+0000 7fefa5400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefa8070440 0x7fefa80c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:43.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.107+0000 7fefa5400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefa8070440 0x7fefa80c9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56912/0 (socket says 172.21.15.39:56912) 2024-09-21T07:35:43.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.107+0000 7fefa5400640 1 -- 172.21.15.39:0/1279929428 learned_addr learned my addr 172.21.15.39:0/1279929428 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:43.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.107+0000 7fefa6800640 1 --2- 172.21.15.39:0/1279929428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefa8073170 0x7fefa80cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:43.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.107+0000 7fefa5e00640 1 --2- 172.21.15.39:0/1279929428 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefa8070030 0x7fefa81f68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:43.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.107+0000 7fefa5400640 1 -- 172.21.15.39:0/1279929428 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefa8070030 msgr2=0x7fefa81f68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:43.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.107+0000 7fefa5400640 1 --2- 172.21.15.39:0/1279929428 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefa8070030 0x7fefa81f68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:43.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.107+0000 7fefa5400640 1 -- 172.21.15.39:0/1279929428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefa8073170 msgr2=0x7fefa80cb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:43.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.107+0000 7fefa5400640 1 --2- 172.21.15.39:0/1279929428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefa8073170 0x7fefa80cb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:43.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.107+0000 7fefa5400640 1 -- 172.21.15.39:0/1279929428 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa80c8060 con 0x7fefa8070440 2024-09-21T07:35:43.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.107+0000 7fefa5e00640 1 --2- 172.21.15.39:0/1279929428 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefa8070030 0x7fefa81f68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:35:43.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.107+0000 7fefa6800640 1 --2- 172.21.15.39:0/1279929428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefa8073170 0x7fefa80cb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:35:43.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.108+0000 7fefa5400640 1 --2- 172.21.15.39:0/1279929428 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefa8070440 0x7fefa80c9100 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fef94000c00 tx=0x7fef94003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:43.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.108+0000 7fef9e000640 1 -- 172.21.15.39:0/1279929428 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9403e030 con 0x7fefa8070440 2024-09-21T07:35:43.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.108+0000 7fef9e000640 1 -- 172.21.15.39:0/1279929428 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef94003e00 con 0x7fefa8070440 2024-09-21T07:35:43.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.108+0000 7fef9e000640 1 -- 172.21.15.39:0/1279929428 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef94035040 con 0x7fefa8070440 2024-09-21T07:35:43.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.108+0000 7fefad000640 1 -- 172.21.15.39:0/1279929428 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefa80c8340 con 0x7fefa8070440 2024-09-21T07:35:43.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.108+0000 7fefad000640 1 -- 172.21.15.39:0/1279929428 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefa80c8900 con 0x7fefa8070440 2024-09-21T07:35:43.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.109+0000 7fefad000640 1 -- 172.21.15.39:0/1279929428 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefa806fac0 con 0x7fefa8070440 2024-09-21T07:35:43.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.110+0000 7fef9e000640 1 -- 172.21.15.39:0/1279929428 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fef94041030 con 0x7fefa8070440 2024-09-21T07:35:43.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.111+0000 7fef9e000640 1 --2- 172.21.15.39:0/1279929428 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fef88078b90 0x7fef8807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:43.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.112+0000 7fef9e000640 1 -- 172.21.15.39:0/1279929428 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6006+0+0 (secure 0 0 0) 0x7fef940bb730 con 0x7fefa8070440 2024-09-21T07:35:43.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.112+0000 7fefa5e00640 1 --2- 172.21.15.39:0/1279929428 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fef88078b90 0x7fef8807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:43.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.113+0000 7fefa5e00640 1 --2- 172.21.15.39:0/1279929428 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fef88078b90 0x7fef8807b0a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fef98005fd0 tx=0x7fef9800f360 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:43.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.116+0000 7fef9e000640 1 -- 172.21.15.39:0/1279929428 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fef940839a0 con 0x7fefa8070440 2024-09-21T07:35:43.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.312+0000 7fefad000640 1 -- 172.21.15.39:0/1279929428 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fefa80c3860 con 0x7fefa8070440 2024-09-21T07:35:43.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.316+0000 7fef9e000640 1 -- 172.21.15.39:0/1279929428 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v39)=0 v39) ==== 122+0+0 (secure 0 0 0) 0x7fef940832a0 con 0x7fefa8070440 2024-09-21T07:35:43.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.319+0000 7fefad000640 1 -- 172.21.15.39:0/1279929428 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fef88078b90 msgr2=0x7fef8807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:43.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.319+0000 7fefad000640 1 --2- 172.21.15.39:0/1279929428 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fef88078b90 0x7fef8807b0a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fef98005fd0 tx=0x7fef9800f360 comp rx=0 tx=0).stop 2024-09-21T07:35:43.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.319+0000 7fefad000640 1 -- 172.21.15.39:0/1279929428 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefa8070440 msgr2=0x7fefa80c9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:43.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.319+0000 7fefad000640 1 --2- 172.21.15.39:0/1279929428 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefa8070440 0x7fefa80c9100 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fef94000c00 tx=0x7fef94003910 comp rx=0 tx=0).stop 2024-09-21T07:35:43.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.319+0000 7fefad000640 1 -- 172.21.15.39:0/1279929428 shutdown_connections 2024-09-21T07:35:43.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.319+0000 7fefad000640 1 --2- 172.21.15.39:0/1279929428 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fef88078b90 0x7fef8807b0a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:43.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.319+0000 7fefad000640 1 --2- 172.21.15.39:0/1279929428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefa8073170 0x7fefa80cb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:43.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.319+0000 7fefad000640 1 --2- 172.21.15.39:0/1279929428 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefa8070440 0x7fefa80c9100 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:43.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.319+0000 7fefad000640 1 --2- 172.21.15.39:0/1279929428 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefa8070030 0x7fefa81f68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:43.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.319+0000 7fefad000640 1 -- 172.21.15.39:0/1279929428 >> 172.21.15.39:0/1279929428 conn(0x7fefa80b1cb0 msgr2=0x7fefa80b1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:43.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.319+0000 7fefad000640 1 -- 172.21.15.39:0/1279929428 shutdown_connections 2024-09-21T07:35:43.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.319+0000 7fefad000640 1 -- 172.21.15.39:0/1279929428 wait complete. 2024-09-21T07:35:43.346 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:35:43.346 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:35:43.440 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:35:43.547 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:35:43.547 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:35:43.547 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:35:43.583 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:35:43.583 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:35:43.584 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:35:43.620 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:35:43.620 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:35:43.621 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:35:43.659 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:35:43.659 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:35:43.659 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:35:43.696 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:35:43.697 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T07:35:43.697 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:35:43.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.926+0000 7f495ce00640 1 -- 172.21.15.39:0/3671695263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49580cfae0 msgr2=0x7f49580cc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:43.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.926+0000 7f495ce00640 1 --2- 172.21.15.39:0/3671695263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49580cfae0 0x7f49580cc970 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f4958070580 tx=0x7f494402a4d0 comp rx=0 tx=0).stop 2024-09-21T07:35:43.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.926+0000 7f495ce00640 1 -- 172.21.15.39:0/3671695263 shutdown_connections 2024-09-21T07:35:43.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.926+0000 7f495ce00640 1 --2- 172.21.15.39:0/3671695263 >> v2:172.21.15.148:3300/0 conn(0x7f49580cceb0 0x7f495806ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:43.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.926+0000 7f495ce00640 1 --2- 172.21.15.39:0/3671695263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49580cfae0 0x7f49580cc970 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:43.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.926+0000 7f495ce00640 1 -- 172.21.15.39:0/3671695263 >> 172.21.15.39:0/3671695263 conn(0x7f495806d7d0 msgr2=0x7f49580b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:43.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.927+0000 7f495ce00640 1 -- 172.21.15.39:0/3671695263 shutdown_connections 2024-09-21T07:35:43.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.927+0000 7f495ce00640 1 -- 172.21.15.39:0/3671695263 wait complete. 2024-09-21T07:35:43.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.929+0000 7f495ce00640 1 Processor -- start 2024-09-21T07:35:43.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.929+0000 7f495ce00640 1 -- start start 2024-09-21T07:35:43.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.930+0000 7f495ce00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49580cceb0 0x7f49580b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:43.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.930+0000 7f495ce00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49580cfae0 0x7f495814a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:43.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.930+0000 7f495ce00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f495815a9b0 0x7f495814ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:43.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.930+0000 7f495ce00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f49580cc760 con 0x7f495815a9b0 2024-09-21T07:35:43.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.930+0000 7f495ce00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f49580cc5e0 con 0x7f49580cceb0 2024-09-21T07:35:43.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.930+0000 7f495ce00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f4958070f20 con 0x7f49580cfae0 2024-09-21T07:35:43.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.931+0000 7f4956800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f495815a9b0 0x7f495814ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:43.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.931+0000 7f4956800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f495815a9b0 0x7f495814ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56930/0 (socket says 172.21.15.39:56930) 2024-09-21T07:35:43.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.931+0000 7f4955e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49580cceb0 0x7f49580b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:43.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.931+0000 7f4956800640 1 -- 172.21.15.39:0/3628993151 learned_addr learned my addr 172.21.15.39:0/3628993151 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:43.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.931+0000 7f4955400640 1 --2- 172.21.15.39:0/3628993151 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49580cfae0 0x7f495814a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:43.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.931+0000 7f4956800640 1 -- 172.21.15.39:0/3628993151 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49580cfae0 msgr2=0x7f495814a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:43.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.931+0000 7f4956800640 1 --2- 172.21.15.39:0/3628993151 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49580cfae0 0x7f495814a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:43.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.931+0000 7f4956800640 1 -- 172.21.15.39:0/3628993151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49580cceb0 msgr2=0x7f49580b0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:43.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.931+0000 7f4956800640 1 --2- 172.21.15.39:0/3628993151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49580cceb0 0x7f49580b0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:43.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.931+0000 7f4956800640 1 -- 172.21.15.39:0/3628993151 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4958158310 con 0x7f495815a9b0 2024-09-21T07:35:43.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.931+0000 7f4955e00640 1 --2- 172.21.15.39:0/3628993151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49580cceb0 0x7f49580b0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:35:43.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.931+0000 7f4956800640 1 --2- 172.21.15.39:0/3628993151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f495815a9b0 0x7f495814ad30 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f495000ea80 tx=0x7f495000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:43.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.932+0000 7f494e000640 1 -- 172.21.15.39:0/3628993151 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4950004b20 con 0x7f495815a9b0 2024-09-21T07:35:43.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.932+0000 7f494e000640 1 -- 172.21.15.39:0/3628993151 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4950004ce0 con 0x7f495815a9b0 2024-09-21T07:35:43.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.932+0000 7f495ce00640 1 -- 172.21.15.39:0/3628993151 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49581f6180 con 0x7f495815a9b0 2024-09-21T07:35:43.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.932+0000 7f494e000640 1 -- 172.21.15.39:0/3628993151 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4950005280 con 0x7f495815a9b0 2024-09-21T07:35:43.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.932+0000 7f495ce00640 1 -- 172.21.15.39:0/3628993151 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49581f6740 con 0x7f495815a9b0 2024-09-21T07:35:43.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.933+0000 7f495ce00640 1 -- 172.21.15.39:0/3628993151 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49580cef80 con 0x7f495815a9b0 2024-09-21T07:35:43.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.934+0000 7f494e000640 1 -- 172.21.15.39:0/3628993151 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4950005420 con 0x7f495815a9b0 2024-09-21T07:35:43.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.935+0000 7f494e000640 1 --2- 172.21.15.39:0/3628993151 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4938078bc0 0x7f493807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:43.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.935+0000 7f494e000640 1 -- 172.21.15.39:0/3628993151 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6006+0+0 (secure 0 0 0) 0x7f495009baa0 con 0x7f495815a9b0 2024-09-21T07:35:43.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.935+0000 7f4955e00640 1 --2- 172.21.15.39:0/3628993151 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4938078bc0 0x7f493807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:43.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.936+0000 7f4955e00640 1 --2- 172.21.15.39:0/3628993151 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4938078bc0 0x7f493807b0d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4948001b10 tx=0x7f4948009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:43.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:43.940+0000 7f494e000640 1 -- 172.21.15.39:0/3628993151 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4958158310 con 0x7f495815a9b0 2024-09-21T07:35:44.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.173+0000 7f495ce00640 1 -- 172.21.15.39:0/3628993151 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49581f5f10 con 0x7f495815a9b0 2024-09-21T07:35:44.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.174+0000 7f494e000640 1 -- 172.21.15.39:0/3628993151 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+3914 (secure 0 0 0) 0x7f49500636c0 con 0x7f495815a9b0 2024-09-21T07:35:44.175 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:35:44.175 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":37,"btime":"2024-09-21T07:35:35:264109+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5183,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2219254706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2219254706},{"type":"v1","addr":"172.21.15.148:6836","nonce":2219254706}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5189,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3512870782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3512870782},{"type":"v1","addr":"172.21.15.148:6837","nonce":3512870782}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5551,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/761690784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":761690784},{"type":"v1","addr":"172.21.15.39:6835","nonce":761690784}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5877,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/1586343601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1586343601},{"type":"v1","addr":"172.21.15.39:6834","nonce":1586343601}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[]} 2024-09-21T07:35:44.175 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 37 2024-09-21T07:35:44.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.176+0000 7f495ce00640 1 -- 172.21.15.39:0/3628993151 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4938078bc0 msgr2=0x7f493807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:44.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.176+0000 7f495ce00640 1 --2- 172.21.15.39:0/3628993151 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4938078bc0 0x7f493807b0d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4948001b10 tx=0x7f4948009040 comp rx=0 tx=0).stop 2024-09-21T07:35:44.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.176+0000 7f495ce00640 1 -- 172.21.15.39:0/3628993151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f495815a9b0 msgr2=0x7f495814ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:44.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.176+0000 7f495ce00640 1 --2- 172.21.15.39:0/3628993151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f495815a9b0 0x7f495814ad30 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f495000ea80 tx=0x7f495000ef40 comp rx=0 tx=0).stop 2024-09-21T07:35:44.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.176+0000 7f495ce00640 1 -- 172.21.15.39:0/3628993151 shutdown_connections 2024-09-21T07:35:44.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.176+0000 7f495ce00640 1 --2- 172.21.15.39:0/3628993151 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4938078bc0 0x7f493807b0d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:44.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.176+0000 7f495ce00640 1 --2- 172.21.15.39:0/3628993151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f495815a9b0 0x7f495814ad30 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:44.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.176+0000 7f495ce00640 1 --2- 172.21.15.39:0/3628993151 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49580cfae0 0x7f495814a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:44.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.176+0000 7f495ce00640 1 --2- 172.21.15.39:0/3628993151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49580cceb0 0x7f49580b0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:44.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.176+0000 7f495ce00640 1 -- 172.21.15.39:0/3628993151 >> 172.21.15.39:0/3628993151 conn(0x7f495806d7d0 msgr2=0x7f49580ae5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:44.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.176+0000 7f495ce00640 1 -- 172.21.15.39:0/3628993151 shutdown_connections 2024-09-21T07:35:44.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.176+0000 7f495ce00640 1 -- 172.21.15.39:0/3628993151 wait complete. 2024-09-21T07:35:44.199 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T07:35:44.199 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T07:35:44.199 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T07:35:44.199 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:35:44.200 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T07:35:44.200 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T07:35:44.200 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T07:35:44.200 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:35:44.200 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T07:35:44.200 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T07:35:44.200 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T07:35:44.200 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:35:44.201 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T07:35:44.201 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T07:35:44.201 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T07:35:44.201 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:35:44.224 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T07:35:44.224 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T07:35:44.224 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T07:35:44.224 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T07:35:44.233 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T07:35:44.233 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T07:35:44.233 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T07:35:44.234 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T07:35:44.268 INFO:tasks.ceph.mds.a:Started 2024-09-21T07:35:44.268 DEBUG:teuthology.parallel:result is None 2024-09-21T07:35:44.270 INFO:tasks.ceph.mds.c:Started 2024-09-21T07:35:44.270 DEBUG:teuthology.parallel:result is None 2024-09-21T07:35:44.279 INFO:tasks.ceph.mds.b:Started 2024-09-21T07:35:44.279 DEBUG:teuthology.parallel:result is None 2024-09-21T07:35:44.281 INFO:tasks.ceph.mds.d:Started 2024-09-21T07:35:44.281 DEBUG:teuthology.parallel:result is None 2024-09-21T07:35:44.282 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T07:35:44.469 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-21T07:35:44.470 INFO:tasks.ceph.mds.c.smithi039.stdout:starting mds.c at 2024-09-21T07:35:44.474 INFO:tasks.ceph.mds.b.smithi148.stdout:starting mds.b at 2024-09-21T07:35:44.475 INFO:tasks.ceph.mds.d.smithi148.stdout:starting mds.d at 2024-09-21T07:35:44.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.517+0000 7f4228e00640 1 -- 172.21.15.39:0/1602842467 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42240cd240 msgr2=0x7f42240cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:44.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.517+0000 7f4228e00640 1 --2- 172.21.15.39:0/1602842467 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42240cd240 0x7f42240cf010 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f420c00a970 tx=0x7f420c02b850 comp rx=0 tx=0).stop 2024-09-21T07:35:44.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.517+0000 7f4228e00640 1 -- 172.21.15.39:0/1602842467 shutdown_connections 2024-09-21T07:35:44.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.517+0000 7f4228e00640 1 --2- 172.21.15.39:0/1602842467 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42240cd240 0x7f42240cf010 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:44.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.517+0000 7f4228e00640 1 -- 172.21.15.39:0/1602842467 >> 172.21.15.39:0/1602842467 conn(0x7f42240b1cb0 msgr2=0x7f42240b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:44.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.517+0000 7f4228e00640 1 -- 172.21.15.39:0/1602842467 shutdown_connections 2024-09-21T07:35:44.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.517+0000 7f4228e00640 1 -- 172.21.15.39:0/1602842467 wait complete. 2024-09-21T07:35:44.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.519+0000 7f4228e00640 1 Processor -- start 2024-09-21T07:35:44.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.519+0000 7f4228e00640 1 -- start start 2024-09-21T07:35:44.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.521+0000 7f4228e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42240cf660 0x7f42241f8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:44.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.521+0000 7f4228e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42240cfb90 0x7f4224072e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:44.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.521+0000 7f4228e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42240d0810 0x7f4224073350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:44.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.521+0000 7f4228e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f42240710a0 con 0x7f42240cfb90 2024-09-21T07:35:44.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.521+0000 7f4228e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4224070f20 con 0x7f42240d0810 2024-09-21T07:35:44.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.521+0000 7f4228e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f42240b36c0 con 0x7f42240cf660 2024-09-21T07:35:44.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.521+0000 7f421be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42240cfb90 0x7f4224072e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:44.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.521+0000 7f421be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42240cfb90 0x7f4224072e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56970/0 (socket says 172.21.15.39:56970) 2024-09-21T07:35:44.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.521+0000 7f421be00640 1 -- 172.21.15.39:0/2468239742 learned_addr learned my addr 172.21.15.39:0/2468239742 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:44.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.521+0000 7f4222800640 1 --2- 172.21.15.39:0/2468239742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42240d0810 0x7f4224073350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:44.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.521+0000 7f4221e00640 1 --2- 172.21.15.39:0/2468239742 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42240cf660 0x7f42241f8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:44.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.522+0000 7f421be00640 1 -- 172.21.15.39:0/2468239742 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42240cf660 msgr2=0x7f42241f8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:44.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.522+0000 7f421be00640 1 --2- 172.21.15.39:0/2468239742 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42240cf660 0x7f42241f8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:44.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.522+0000 7f421be00640 1 -- 172.21.15.39:0/2468239742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42240d0810 msgr2=0x7f4224073350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:44.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.522+0000 7f421be00640 1 --2- 172.21.15.39:0/2468239742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42240d0810 0x7f4224073350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:44.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.522+0000 7f421be00640 1 -- 172.21.15.39:0/2468239742 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42240748a0 con 0x7f42240cfb90 2024-09-21T07:35:44.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.522+0000 7f4222800640 1 --2- 172.21.15.39:0/2468239742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42240d0810 0x7f4224073350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:35:44.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.522+0000 7f421be00640 1 --2- 172.21.15.39:0/2468239742 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42240cfb90 0x7f4224072e10 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f420c00aa90 tx=0x7f420c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:44.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.522+0000 7f421a000640 1 -- 172.21.15.39:0/2468239742 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c041070 con 0x7f42240cfb90 2024-09-21T07:35:44.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.522+0000 7f421a000640 1 -- 172.21.15.39:0/2468239742 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f420c005110 con 0x7f42240cfb90 2024-09-21T07:35:44.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.522+0000 7f421a000640 1 -- 172.21.15.39:0/2468239742 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c005560 con 0x7f42240cfb90 2024-09-21T07:35:44.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.522+0000 7f4228e00640 1 -- 172.21.15.39:0/2468239742 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4224074b80 con 0x7f42240cfb90 2024-09-21T07:35:44.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.522+0000 7f4228e00640 1 -- 172.21.15.39:0/2468239742 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4224073890 con 0x7f42240cfb90 2024-09-21T07:35:44.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.523+0000 7f4228e00640 1 -- 172.21.15.39:0/2468239742 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42240d0d60 con 0x7f42240cfb90 2024-09-21T07:35:44.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.525+0000 7f421a000640 1 -- 172.21.15.39:0/2468239742 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f420c03b040 con 0x7f42240cfb90 2024-09-21T07:35:44.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.526+0000 7f421a000640 1 --2- 172.21.15.39:0/2468239742 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4200078bc0 0x7f420007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:44.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.526+0000 7f421a000640 1 -- 172.21.15.39:0/2468239742 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6006+0+0 (secure 0 0 0) 0x7f420c0bbaf0 con 0x7f42240cfb90 2024-09-21T07:35:44.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.526+0000 7f4221e00640 1 --2- 172.21.15.39:0/2468239742 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4200078bc0 0x7f420007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:44.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.526+0000 7f4221e00640 1 --2- 172.21.15.39:0/2468239742 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4200078bc0 0x7f420007b0d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4214000f60 tx=0x7f4214009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:44.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.530+0000 7f421a000640 1 -- 172.21.15.39:0/2468239742 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f420c0837a0 con 0x7f42240cfb90 2024-09-21T07:35:44.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:44.740+0000 7f4228e00640 1 -- 172.21.15.39:0/2468239742 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f42240b3b90 con 0x7f42240cfb90 2024-09-21T07:35:45.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.408+0000 7f421a000640 1 -- 172.21.15.39:0/2468239742 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v72) ==== 97+0+0 (secure 0 0 0) 0x7f420c0881d0 con 0x7f42240cfb90 2024-09-21T07:35:45.410 INFO:teuthology.orchestra.run.smithi039.stderr: removed all blocklist entries 2024-09-21T07:35:45.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.412+0000 7f4228e00640 1 -- 172.21.15.39:0/2468239742 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4200078bc0 msgr2=0x7f420007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:45.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.412+0000 7f4228e00640 1 --2- 172.21.15.39:0/2468239742 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4200078bc0 0x7f420007b0d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4214000f60 tx=0x7f4214009040 comp rx=0 tx=0).stop 2024-09-21T07:35:45.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.412+0000 7f4228e00640 1 -- 172.21.15.39:0/2468239742 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42240cfb90 msgr2=0x7f4224072e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:45.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.412+0000 7f4228e00640 1 --2- 172.21.15.39:0/2468239742 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42240cfb90 0x7f4224072e10 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f420c00aa90 tx=0x7f420c003c10 comp rx=0 tx=0).stop 2024-09-21T07:35:45.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.412+0000 7f4228e00640 1 -- 172.21.15.39:0/2468239742 shutdown_connections 2024-09-21T07:35:45.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.412+0000 7f4228e00640 1 --2- 172.21.15.39:0/2468239742 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4200078bc0 0x7f420007b0d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:45.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.412+0000 7f4228e00640 1 --2- 172.21.15.39:0/2468239742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42240d0810 0x7f4224073350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:45.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.412+0000 7f4228e00640 1 --2- 172.21.15.39:0/2468239742 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42240cfb90 0x7f4224072e10 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:45.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.412+0000 7f4228e00640 1 --2- 172.21.15.39:0/2468239742 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42240cf660 0x7f42241f8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:45.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.412+0000 7f4228e00640 1 -- 172.21.15.39:0/2468239742 >> 172.21.15.39:0/2468239742 conn(0x7f42240b1cb0 msgr2=0x7f42240acf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:45.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.413+0000 7f4228e00640 1 -- 172.21.15.39:0/2468239742 shutdown_connections 2024-09-21T07:35:45.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.413+0000 7f4228e00640 1 -- 172.21.15.39:0/2468239742 wait complete. 2024-09-21T07:35:45.446 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T07:35:45.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.674+0000 7fa65f400640 1 -- 172.21.15.39:0/4014936110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6580af3c0 msgr2=0x7fa6580b40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:45.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.674+0000 7fa65f400640 1 --2- 172.21.15.39:0/4014936110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6580af3c0 0x7fa6580b40b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa658072dc0 tx=0x7fa64402a5f0 comp rx=0 tx=0).stop 2024-09-21T07:35:45.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.674+0000 7fa65f400640 1 -- 172.21.15.39:0/4014936110 shutdown_connections 2024-09-21T07:35:45.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.674+0000 7fa65f400640 1 --2- 172.21.15.39:0/4014936110 >> v2:172.21.15.148:3300/0 conn(0x7fa6580ccb50 0x7fa6580ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:45.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.674+0000 7fa65f400640 1 --2- 172.21.15.39:0/4014936110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6580af3c0 0x7fa6580b40b0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:45.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.674+0000 7fa65f400640 1 -- 172.21.15.39:0/4014936110 >> 172.21.15.39:0/4014936110 conn(0x7fa65806d7d0 msgr2=0x7fa6580b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:45.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.675+0000 7fa65f400640 1 -- 172.21.15.39:0/4014936110 shutdown_connections 2024-09-21T07:35:45.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.675+0000 7fa65f400640 1 -- 172.21.15.39:0/4014936110 wait complete. 2024-09-21T07:35:45.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.677+0000 7fa65f400640 1 Processor -- start 2024-09-21T07:35:45.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.677+0000 7fa65f400640 1 -- start start 2024-09-21T07:35:45.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.678+0000 7fa65f400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6580af3c0 0x7fa65815e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:45.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.678+0000 7fa65f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6580cc1e0 0x7fa65815e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:45.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.678+0000 7fa65f400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6580ccb50 0x7fa6581e6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:45.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.678+0000 7fa65f400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa6581533a0 con 0x7fa6580cc1e0 2024-09-21T07:35:45.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.678+0000 7fa65f400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa6580af8e0 con 0x7fa6580ccb50 2024-09-21T07:35:45.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.678+0000 7fa65f400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa658153520 con 0x7fa6580af3c0 2024-09-21T07:35:45.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.679+0000 7fa657e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6580cc1e0 0x7fa65815e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:45.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.679+0000 7fa657e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6580cc1e0 0x7fa65815e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57002/0 (socket says 172.21.15.39:57002) 2024-09-21T07:35:45.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.679+0000 7fa657e00640 1 -- 172.21.15.39:0/1502779236 learned_addr learned my addr 172.21.15.39:0/1502779236 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:45.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.679+0000 7fa65d400640 1 --2- 172.21.15.39:0/1502779236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6580ccb50 0x7fa6581e6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:45.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.679+0000 7fa657e00640 1 -- 172.21.15.39:0/1502779236 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6580af3c0 msgr2=0x7fa65815e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:45.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.679+0000 7fa657e00640 1 --2- 172.21.15.39:0/1502779236 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6580af3c0 0x7fa65815e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:45.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.679+0000 7fa657e00640 1 -- 172.21.15.39:0/1502779236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6580ccb50 msgr2=0x7fa6581e6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:45.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.679+0000 7fa657e00640 1 --2- 172.21.15.39:0/1502779236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6580ccb50 0x7fa6581e6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:45.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.679+0000 7fa657e00640 1 -- 172.21.15.39:0/1502779236 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa65815f110 con 0x7fa6580cc1e0 2024-09-21T07:35:45.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.679+0000 7fa65d400640 1 --2- 172.21.15.39:0/1502779236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6580ccb50 0x7fa6581e6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:35:45.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.679+0000 7fa657e00640 1 --2- 172.21.15.39:0/1502779236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6580cc1e0 0x7fa65815e900 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa644034040 tx=0x7fa644003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:45.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.680+0000 7fa654c00640 1 -- 172.21.15.39:0/1502779236 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64403e030 con 0x7fa6580cc1e0 2024-09-21T07:35:45.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.680+0000 7fa654c00640 1 -- 172.21.15.39:0/1502779236 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa644003d10 con 0x7fa6580cc1e0 2024-09-21T07:35:45.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.680+0000 7fa654c00640 1 -- 172.21.15.39:0/1502779236 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa644035040 con 0x7fa6580cc1e0 2024-09-21T07:35:45.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.680+0000 7fa65f400640 1 -- 172.21.15.39:0/1502779236 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa65815f3f0 con 0x7fa6580cc1e0 2024-09-21T07:35:45.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.680+0000 7fa65f400640 1 -- 172.21.15.39:0/1502779236 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa65813f7e0 con 0x7fa6580cc1e0 2024-09-21T07:35:45.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.681+0000 7fa65f400640 1 -- 172.21.15.39:0/1502779236 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa658153a40 con 0x7fa6580cc1e0 2024-09-21T07:35:45.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.682+0000 7fa654c00640 1 -- 172.21.15.39:0/1502779236 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa644041030 con 0x7fa6580cc1e0 2024-09-21T07:35:45.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.683+0000 7fa654c00640 1 --2- 172.21.15.39:0/1502779236 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa638078b90 0x7fa63807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:45.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.683+0000 7fa654c00640 1 -- 172.21.15.39:0/1502779236 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5920+0+0 (secure 0 0 0) 0x7fa6440bb4a0 con 0x7fa6580cc1e0 2024-09-21T07:35:45.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.683+0000 7fa65ca00640 1 --2- 172.21.15.39:0/1502779236 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa638078b90 0x7fa63807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:45.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.684+0000 7fa65ca00640 1 --2- 172.21.15.39:0/1502779236 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa638078b90 0x7fa63807b0a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa64c000f60 tx=0x7fa64c0079b0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:45.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.687+0000 7fa654c00640 1 -- 172.21.15.39:0/1502779236 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa6440838b0 con 0x7fa6580cc1e0 2024-09-21T07:35:45.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.958+0000 7fa65f400640 1 -- 172.21.15.39:0/1502779236 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fa65815eea0 con 0x7fa6580cc1e0 2024-09-21T07:35:45.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.958+0000 7fa654c00640 1 -- 172.21.15.39:0/1502779236 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) ==== 80+0+5304 (secure 0 0 0) 0x7fa644088e20 con 0x7fa6580cc1e0 2024-09-21T07:35:45.959 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:35:45.959 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:35:45.959 INFO:teuthology.orchestra.run.smithi039.stdout: "auth_dump": [ 2024-09-21T07:35:45.960 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:45.960 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.a", 2024-09-21T07:35:45.960 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5m+P6EERAABQqmlkrocSi1+hwc73Adrg==", 2024-09-21T07:35:45.960 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:35:45.960 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:35:45.960 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:35:45.960 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:35:45.960 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:35:45.960 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:45.960 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:35:45.960 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:45.960 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.b", 2024-09-21T07:35:45.960 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5m5/GeLRAAfE8fr5dbitQa89rNK9pZ2Q==", 2024-09-21T07:35:45.960 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:35:45.960 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:35:45.960 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:35:45.961 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:35:45.961 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:35:45.961 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:45.961 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:35:45.961 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:45.961 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.c", 2024-09-21T07:35:45.961 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5msbx6HxAALsF8ORvIoJpIAFZ5uxmOEw==", 2024-09-21T07:35:45.961 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:35:45.961 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:35:45.961 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:35:45.961 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:35:45.961 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:35:45.961 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:45.961 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:35:45.962 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:45.962 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.d", 2024-09-21T07:35:45.962 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mIrcBABAApKAB/7FkZ5glNTxlbkIsBw==", 2024-09-21T07:35:45.962 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:35:45.962 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:35:45.962 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:35:45.962 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:35:45.962 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:35:45.962 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:45.962 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:35:45.962 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:45.962 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.0", 2024-09-21T07:35:45.962 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB7du5msw81DhAAmVimqN4x9K3YpT0nbovppQ==", 2024-09-21T07:35:45.963 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:35:45.963 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:35:45.963 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:35:45.963 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:35:45.963 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:45.963 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:35:45.963 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:45.963 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.1", 2024-09-21T07:35:45.963 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB+du5mdnCIOhAAtVB4ThehQwb0zRriyKcaxA==", 2024-09-21T07:35:45.963 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:35:45.963 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:35:45.963 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:35:45.963 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:35:45.963 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:45.963 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:35:45.964 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:45.964 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.2", 2024-09-21T07:35:45.964 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCCdu5mTfWxDhAAadJCbPN5xBx/f9L0v2yTdg==", 2024-09-21T07:35:45.964 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:35:45.964 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:35:45.964 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:35:45.964 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:35:45.964 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:45.964 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:35:45.964 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:45.964 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.3", 2024-09-21T07:35:45.964 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCFdu5mL1E2HxAAyOhY97uUXEw5L1CcDeONVA==", 2024-09-21T07:35:45.964 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:35:45.964 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:35:45.964 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:35:45.964 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:35:45.965 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:45.965 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:35:45.965 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:45.965 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.4", 2024-09-21T07:35:45.965 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCLdu5m6PY/IhAAgvcTgm+6uWA7lIG4tcTIEg==", 2024-09-21T07:35:45.965 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:35:45.965 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:35:45.965 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:35:45.965 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:35:45.965 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:45.965 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:35:45.965 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:45.965 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.5", 2024-09-21T07:35:45.965 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCOdu5mE/wlMxAA3JBg4xH+N/PPC1Uv2ZVrYQ==", 2024-09-21T07:35:45.965 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:35:45.966 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:35:45.966 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:35:45.966 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:35:45.966 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:45.966 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:35:45.966 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:45.966 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.6", 2024-09-21T07:35:45.966 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCSdu5mC3TgBxAAwsArjeIdFcApY3/1OJffEg==", 2024-09-21T07:35:45.966 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:35:45.966 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:35:45.966 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:35:45.966 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:35:45.966 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:45.967 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:35:45.967 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:45.967 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.7", 2024-09-21T07:35:45.967 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCVdu5mSWHWFBAA9qPHWRdsvrB74inrp1lj4Q==", 2024-09-21T07:35:45.967 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:35:45.967 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:35:45.967 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:35:45.967 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:35:45.967 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:45.967 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:35:45.967 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:45.967 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.0", 2024-09-21T07:35:45.967 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mGfb1DBAA/aSMKR5or3xy53rB/iUjpA==", 2024-09-21T07:35:45.967 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:35:45.968 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:35:45.968 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow r", 2024-09-21T07:35:45.968 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T07:35:45.968 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:45.968 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:35:45.968 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:45.968 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.1", 2024-09-21T07:35:45.968 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mEkciIhAAid+bxJEr4P7vMnZ42PAy5w==", 2024-09-21T07:35:45.968 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:35:45.968 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:35:45.968 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow r", 2024-09-21T07:35:45.968 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow rw", 2024-09-21T07:35:45.968 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rwx" 2024-09-21T07:35:45.968 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:45.968 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:35:45.969 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:45.969 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.2", 2024-09-21T07:35:45.969 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mSu2oFxAAgjy2m/wWKh6xaoh+YDoU1Q==", 2024-09-21T07:35:45.969 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:35:45.969 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:35:45.969 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow r", 2024-09-21T07:35:45.969 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow rw", 2024-09-21T07:35:45.969 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rwx" 2024-09-21T07:35:45.969 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:45.969 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:35:45.969 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:45.969 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.3", 2024-09-21T07:35:45.969 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mCHy0LBAAZneF2jUWgfuyGehZFbOxDg==", 2024-09-21T07:35:45.969 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:35:45.969 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:35:45.969 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow r", 2024-09-21T07:35:45.970 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow rw", 2024-09-21T07:35:45.970 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rwx" 2024-09-21T07:35:45.970 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:45.970 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:35:45.970 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:45.970 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.admin", 2024-09-21T07:35:45.970 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBydu5m9LInERAAAlDpqb94f2Qb9VYJJwTBcA==", 2024-09-21T07:35:45.970 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:35:45.970 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow *", 2024-09-21T07:35:45.970 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:35:45.970 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:35:45.970 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:35:45.970 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:45.970 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:35:45.970 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:45.971 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mgr.x", 2024-09-21T07:35:45.971 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5mA2GRBhAAkyCKdgH6laHy8xNt080dXg==", 2024-09-21T07:35:45.971 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:35:45.971 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow *", 2024-09-21T07:35:45.971 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile mgr", 2024-09-21T07:35:45.971 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:35:45.971 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:45.971 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:35:45.971 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:45.971 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mgr.y", 2024-09-21T07:35:45.971 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBydu5mr+8aNxAA7TFYNms2WsXUfa4ITTJOHw==", 2024-09-21T07:35:45.971 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:35:45.971 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow *", 2024-09-21T07:35:45.971 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile mgr", 2024-09-21T07:35:45.971 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:35:45.971 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:45.972 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:45.972 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-21T07:35:45.972 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:35:45.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.961+0000 7fa65f400640 1 -- 172.21.15.39:0/1502779236 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa638078b90 msgr2=0x7fa63807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:45.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.961+0000 7fa65f400640 1 --2- 172.21.15.39:0/1502779236 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa638078b90 0x7fa63807b0a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa64c000f60 tx=0x7fa64c0079b0 comp rx=0 tx=0).stop 2024-09-21T07:35:45.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.961+0000 7fa65f400640 1 -- 172.21.15.39:0/1502779236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6580cc1e0 msgr2=0x7fa65815e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:45.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.961+0000 7fa65f400640 1 --2- 172.21.15.39:0/1502779236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6580cc1e0 0x7fa65815e900 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa644034040 tx=0x7fa644003820 comp rx=0 tx=0).stop 2024-09-21T07:35:45.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.961+0000 7fa65f400640 1 -- 172.21.15.39:0/1502779236 shutdown_connections 2024-09-21T07:35:45.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.961+0000 7fa65f400640 1 --2- 172.21.15.39:0/1502779236 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa638078b90 0x7fa63807b0a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:45.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.961+0000 7fa65f400640 1 --2- 172.21.15.39:0/1502779236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6580ccb50 0x7fa6581e6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:45.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.961+0000 7fa65f400640 1 --2- 172.21.15.39:0/1502779236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6580cc1e0 0x7fa65815e900 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:45.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.961+0000 7fa65f400640 1 --2- 172.21.15.39:0/1502779236 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6580af3c0 0x7fa65815e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:45.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.961+0000 7fa65f400640 1 -- 172.21.15.39:0/1502779236 >> 172.21.15.39:0/1502779236 conn(0x7fa65806d7d0 msgr2=0x7fa6580afc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:45.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.961+0000 7fa65f400640 1 -- 172.21.15.39:0/1502779236 shutdown_connections 2024-09-21T07:35:45.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:45.961+0000 7fa65f400640 1 -- 172.21.15.39:0/1502779236 wait complete. 2024-09-21T07:35:45.985 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T07:35:46.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.212+0000 7fb7f4800640 1 -- 172.21.15.39:0/1426392555 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb7ec15ef10 msgr2=0x7fb7ec0cc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:46.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.212+0000 7fb7f4800640 1 --2- 172.21.15.39:0/1426392555 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb7ec15ef10 0x7fb7ec0cc970 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc009830 tx=0x7fb7dc02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:35:46.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.212+0000 7fb7f4800640 1 -- 172.21.15.39:0/1426392555 shutdown_connections 2024-09-21T07:35:46.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.212+0000 7fb7f4800640 1 --2- 172.21.15.39:0/1426392555 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb7ec15ef10 0x7fb7ec0cc970 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:46.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.212+0000 7fb7f4800640 1 --2- 172.21.15.39:0/1426392555 >> v2:172.21.15.148:3300/0 conn(0x7fb7ec15a9b0 0x7fb7ec0cfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:46.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.212+0000 7fb7f4800640 1 -- 172.21.15.39:0/1426392555 >> 172.21.15.39:0/1426392555 conn(0x7fb7ec06d7d0 msgr2=0x7fb7ec0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:46.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.212+0000 7fb7f4800640 1 -- 172.21.15.39:0/1426392555 shutdown_connections 2024-09-21T07:35:46.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.212+0000 7fb7f4800640 1 -- 172.21.15.39:0/1426392555 wait complete. 2024-09-21T07:35:46.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.215+0000 7fb7f4800640 1 Processor -- start 2024-09-21T07:35:46.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.215+0000 7fb7f4800640 1 -- start start 2024-09-21T07:35:46.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.216+0000 7fb7f4800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb7ec0cceb0 0x7fb7ec145cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:46.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.216+0000 7fb7f4800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7ec15a9b0 0x7fb7ec159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:46.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.216+0000 7fb7f4800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7ec15ef10 0x7fb7ec159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:46.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.216+0000 7fb7f4800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec0cc760 con 0x7fb7ec15a9b0 2024-09-21T07:35:46.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.216+0000 7fb7f4800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec0cc5e0 con 0x7fb7ec15ef10 2024-09-21T07:35:46.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.216+0000 7fb7f4800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb7ec070f20 con 0x7fb7ec0cceb0 2024-09-21T07:35:46.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.216+0000 7fb7f1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7ec15a9b0 0x7fb7ec159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:46.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.217+0000 7fb7f1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7ec15a9b0 0x7fb7ec159a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57006/0 (socket says 172.21.15.39:57006) 2024-09-21T07:35:46.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.217+0000 7fb7f1400640 1 -- 172.21.15.39:0/1783604758 learned_addr learned my addr 172.21.15.39:0/1783604758 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:46.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.217+0000 7fb7f1e00640 1 --2- 172.21.15.39:0/1783604758 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb7ec0cceb0 0x7fb7ec145cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:46.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.217+0000 7fb7f2800640 1 --2- 172.21.15.39:0/1783604758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7ec15ef10 0x7fb7ec159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:46.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.217+0000 7fb7f1400640 1 -- 172.21.15.39:0/1783604758 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb7ec0cceb0 msgr2=0x7fb7ec145cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:46.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.217+0000 7fb7f1400640 1 --2- 172.21.15.39:0/1783604758 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb7ec0cceb0 0x7fb7ec145cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:46.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.217+0000 7fb7f1400640 1 -- 172.21.15.39:0/1783604758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7ec15ef10 msgr2=0x7fb7ec159f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:46.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.217+0000 7fb7f1400640 1 --2- 172.21.15.39:0/1783604758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7ec15ef10 0x7fb7ec159f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:46.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.217+0000 7fb7f1400640 1 -- 172.21.15.39:0/1783604758 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7ec15b410 con 0x7fb7ec15a9b0 2024-09-21T07:35:46.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.217+0000 7fb7f1400640 1 --2- 172.21.15.39:0/1783604758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7ec15a9b0 0x7fb7ec159a50 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc000c00 tx=0x7fb7dc033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:46.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.218+0000 7fb7e6000640 1 -- 172.21.15.39:0/1783604758 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc031b50 con 0x7fb7ec15a9b0 2024-09-21T07:35:46.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.218+0000 7fb7e6000640 1 -- 172.21.15.39:0/1783604758 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb7dc031de0 con 0x7fb7ec15a9b0 2024-09-21T07:35:46.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.218+0000 7fb7f4800640 1 -- 172.21.15.39:0/1783604758 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7ec0b08a0 con 0x7fb7ec15a9b0 2024-09-21T07:35:46.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.218+0000 7fb7e6000640 1 -- 172.21.15.39:0/1783604758 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc033330 con 0x7fb7ec15a9b0 2024-09-21T07:35:46.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.218+0000 7fb7f4800640 1 -- 172.21.15.39:0/1783604758 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7ec146460 con 0x7fb7ec15a9b0 2024-09-21T07:35:46.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.219+0000 7fb7db400640 1 -- 172.21.15.39:0/1783604758 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7c4005290 con 0x7fb7ec15a9b0 2024-09-21T07:35:46.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.221+0000 7fb7e6000640 1 -- 172.21.15.39:0/1783604758 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb7dc0334d0 con 0x7fb7ec15a9b0 2024-09-21T07:35:46.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.221+0000 7fb7e6000640 1 --2- 172.21.15.39:0/1783604758 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb7d0078b90 0x7fb7d007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:46.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.222+0000 7fb7e6000640 1 -- 172.21.15.39:0/1783604758 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5920+0+0 (secure 0 0 0) 0x7fb7dc0bc4d0 con 0x7fb7ec15a9b0 2024-09-21T07:35:46.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.222+0000 7fb7f1e00640 1 --2- 172.21.15.39:0/1783604758 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb7d0078b90 0x7fb7d007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:46.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.223+0000 7fb7f1e00640 1 --2- 172.21.15.39:0/1783604758 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb7d0078b90 0x7fb7d007b0a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0009ac0 tx=0x7fb7e0005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:46.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.225+0000 7fb7e6000640 1 -- 172.21.15.39:0/1783604758 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb7dc052080 con 0x7fb7ec15a9b0 2024-09-21T07:35:46.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.434+0000 7fb7db400640 1 -- 172.21.15.39:0/1783604758 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb7c4005850 con 0x7fb7ec15a9b0 2024-09-21T07:35:46.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.434+0000 7fb7e6000640 1 -- 172.21.15.39:0/1783604758 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v74) ==== 84+0+287 (secure 0 0 0) 0x7fb7dc03f030 con 0x7fb7ec15a9b0 2024-09-21T07:35:46.435 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:35:46.435 INFO:teuthology.orchestra.run.smithi039.stdout:[ 2024-09-21T07:35:46.435 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:46.436 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 1, 2024-09-21T07:35:46.436 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": ".mgr" 2024-09-21T07:35:46.436 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:35:46.436 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:46.436 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 6, 2024-09-21T07:35:46.436 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": "snap-test-pool" 2024-09-21T07:35:46.436 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:35:46.436 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:46.436 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 11, 2024-09-21T07:35:46.436 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": "test_data_pool" 2024-09-21T07:35:46.436 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:35:46.436 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:35:46.436 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 12, 2024-09-21T07:35:46.436 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": "test_metadata_pool" 2024-09-21T07:35:46.437 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:35:46.437 INFO:teuthology.orchestra.run.smithi039.stdout:] 2024-09-21T07:35:46.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.436+0000 7fb7db400640 1 -- 172.21.15.39:0/1783604758 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb7d0078b90 msgr2=0x7fb7d007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:46.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.436+0000 7fb7db400640 1 --2- 172.21.15.39:0/1783604758 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb7d0078b90 0x7fb7d007b0a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0009ac0 tx=0x7fb7e0005f90 comp rx=0 tx=0).stop 2024-09-21T07:35:46.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.436+0000 7fb7db400640 1 -- 172.21.15.39:0/1783604758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7ec15a9b0 msgr2=0x7fb7ec159a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:46.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.436+0000 7fb7db400640 1 --2- 172.21.15.39:0/1783604758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7ec15a9b0 0x7fb7ec159a50 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc000c00 tx=0x7fb7dc033ba0 comp rx=0 tx=0).stop 2024-09-21T07:35:46.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.436+0000 7fb7db400640 1 -- 172.21.15.39:0/1783604758 shutdown_connections 2024-09-21T07:35:46.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.436+0000 7fb7db400640 1 --2- 172.21.15.39:0/1783604758 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb7d0078b90 0x7fb7d007b0a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:46.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.436+0000 7fb7db400640 1 --2- 172.21.15.39:0/1783604758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7ec15ef10 0x7fb7ec159f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:46.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.436+0000 7fb7db400640 1 --2- 172.21.15.39:0/1783604758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7ec15a9b0 0x7fb7ec159a50 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:46.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.436+0000 7fb7db400640 1 --2- 172.21.15.39:0/1783604758 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb7ec0cceb0 0x7fb7ec145cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:46.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.437+0000 7fb7db400640 1 -- 172.21.15.39:0/1783604758 >> 172.21.15.39:0/1783604758 conn(0x7fb7ec06d7d0 msgr2=0x7fb7ec0ad6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:46.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.437+0000 7fb7db400640 1 -- 172.21.15.39:0/1783604758 shutdown_connections 2024-09-21T07:35:46.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.437+0000 7fb7db400640 1 -- 172.21.15.39:0/1783604758 wait complete. 2024-09-21T07:35:46.460 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T07:35:46.460 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T07:35:46.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.692+0000 7ffa38c00640 1 -- 172.21.15.39:0/2307079004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa18004800 msgr2=0x7ffa18004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:46.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.692+0000 7ffa38c00640 1 --2- 172.21.15.39:0/2307079004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa18004800 0x7ffa18004c10 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c006f80 tx=0x7ffa18005830 comp rx=0 tx=0).stop 2024-09-21T07:35:46.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.692+0000 7ffa38c00640 1 -- 172.21.15.39:0/2307079004 shutdown_connections 2024-09-21T07:35:46.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.692+0000 7ffa38c00640 1 --2- 172.21.15.39:0/2307079004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa18004800 0x7ffa18004c10 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:46.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.692+0000 7ffa38c00640 1 -- 172.21.15.39:0/2307079004 >> 172.21.15.39:0/2307079004 conn(0x7ffa340d0130 msgr2=0x7ffa340d0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:46.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.692+0000 7ffa38c00640 1 -- 172.21.15.39:0/2307079004 shutdown_connections 2024-09-21T07:35:46.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.692+0000 7ffa38c00640 1 -- 172.21.15.39:0/2307079004 wait complete. 2024-09-21T07:35:46.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.694+0000 7ffa38c00640 1 Processor -- start 2024-09-21T07:35:46.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.694+0000 7ffa38c00640 1 -- start start 2024-09-21T07:35:46.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.696+0000 7ffa38c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa18004800 0x7ffa340b5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:46.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.696+0000 7ffa38c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa3414ae70 0x7ffa3414d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:46.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.696+0000 7ffa38c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffa3414dbc0 0x7ffa3414b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:46.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.696+0000 7ffa38c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ffa340cadc0 con 0x7ffa18004800 2024-09-21T07:35:46.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.696+0000 7ffa38c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ffa34073b90 con 0x7ffa3414ae70 2024-09-21T07:35:46.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.696+0000 7ffa38c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7ffa340caf40 con 0x7ffa3414dbc0 2024-09-21T07:35:46.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.696+0000 7ffa31e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa18004800 0x7ffa340b5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:46.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.696+0000 7ffa31e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa18004800 0x7ffa340b5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57018/0 (socket says 172.21.15.39:57018) 2024-09-21T07:35:46.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.696+0000 7ffa31e00640 1 -- 172.21.15.39:0/1793487735 learned_addr learned my addr 172.21.15.39:0/1793487735 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:46.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.696+0000 7ffa2be00640 1 --2- 172.21.15.39:0/1793487735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa3414ae70 0x7ffa3414d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:46.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.696+0000 7ffa32800640 1 --2- 172.21.15.39:0/1793487735 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffa3414dbc0 0x7ffa3414b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:46.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.696+0000 7ffa31e00640 1 -- 172.21.15.39:0/1793487735 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffa3414dbc0 msgr2=0x7ffa3414b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:46.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.696+0000 7ffa31e00640 1 --2- 172.21.15.39:0/1793487735 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffa3414dbc0 0x7ffa3414b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:46.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.697+0000 7ffa31e00640 1 -- 172.21.15.39:0/1793487735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa3414ae70 msgr2=0x7ffa3414d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:46.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.697+0000 7ffa31e00640 1 --2- 172.21.15.39:0/1793487735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa3414ae70 0x7ffa3414d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:46.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.697+0000 7ffa31e00640 1 -- 172.21.15.39:0/1793487735 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa3414e3b0 con 0x7ffa18004800 2024-09-21T07:35:46.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.697+0000 7ffa31e00640 1 --2- 172.21.15.39:0/1793487735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa18004800 0x7ffa340b5b50 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7ffa24011780 tx=0x7ffa24011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:46.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.697+0000 7ffa2a000640 1 -- 172.21.15.39:0/1793487735 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2401f070 con 0x7ffa18004800 2024-09-21T07:35:46.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.697+0000 7ffa2a000640 1 -- 172.21.15.39:0/1793487735 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa24011e90 con 0x7ffa18004800 2024-09-21T07:35:46.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.697+0000 7ffa38c00640 1 -- 172.21.15.39:0/1793487735 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa340b5070 con 0x7ffa18004800 2024-09-21T07:35:46.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.697+0000 7ffa2a000640 1 -- 172.21.15.39:0/1793487735 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa240126b0 con 0x7ffa18004800 2024-09-21T07:35:46.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.697+0000 7ffa38c00640 1 -- 172.21.15.39:0/1793487735 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa3414c090 con 0x7ffa18004800 2024-09-21T07:35:46.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.698+0000 7ffa38c00640 1 -- 172.21.15.39:0/1793487735 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa340cb450 con 0x7ffa18004800 2024-09-21T07:35:46.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.700+0000 7ffa2a000640 1 -- 172.21.15.39:0/1793487735 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ffa24022070 con 0x7ffa18004800 2024-09-21T07:35:46.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.701+0000 7ffa2a000640 1 --2- 172.21.15.39:0/1793487735 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ffa10078b90 0x7ffa1007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:46.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.701+0000 7ffa2a000640 1 -- 172.21.15.39:0/1793487735 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5920+0+0 (secure 0 0 0) 0x7ffa2409c070 con 0x7ffa18004800 2024-09-21T07:35:46.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.701+0000 7ffa2be00640 1 --2- 172.21.15.39:0/1793487735 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ffa10078b90 0x7ffa1007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:46.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.702+0000 7ffa2be00640 1 --2- 172.21.15.39:0/1793487735 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ffa10078b90 0x7ffa1007b0a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c033000 tx=0x7ffa1c001930 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:46.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.704+0000 7ffa2a000640 1 -- 172.21.15.39:0/1793487735 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ffa24063e20 con 0x7ffa18004800 2024-09-21T07:35:46.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:46.902+0000 7ffa38c00640 1 -- 172.21.15.39:0/1793487735 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ffa340c9040 con 0x7ffa18004800 2024-09-21T07:35:47.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.428+0000 7ffa2a000640 1 -- 172.21.15.39:0/1793487735 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v75) ==== 138+0+0 (secure 0 0 0) 0x7ffa24068850 con 0x7ffa18004800 2024-09-21T07:35:47.431 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' created 2024-09-21T07:35:47.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.432+0000 7ffa38c00640 1 -- 172.21.15.39:0/1793487735 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ffa10078b90 msgr2=0x7ffa1007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:47.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.432+0000 7ffa38c00640 1 --2- 172.21.15.39:0/1793487735 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ffa10078b90 0x7ffa1007b0a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c033000 tx=0x7ffa1c001930 comp rx=0 tx=0).stop 2024-09-21T07:35:47.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.432+0000 7ffa38c00640 1 -- 172.21.15.39:0/1793487735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa18004800 msgr2=0x7ffa340b5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:47.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.432+0000 7ffa38c00640 1 --2- 172.21.15.39:0/1793487735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa18004800 0x7ffa340b5b50 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7ffa24011780 tx=0x7ffa24011c40 comp rx=0 tx=0).stop 2024-09-21T07:35:47.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.432+0000 7ffa38c00640 1 -- 172.21.15.39:0/1793487735 shutdown_connections 2024-09-21T07:35:47.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.432+0000 7ffa38c00640 1 --2- 172.21.15.39:0/1793487735 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ffa10078b90 0x7ffa1007b0a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:47.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.432+0000 7ffa38c00640 1 --2- 172.21.15.39:0/1793487735 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffa3414dbc0 0x7ffa3414b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:47.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.432+0000 7ffa38c00640 1 --2- 172.21.15.39:0/1793487735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa3414ae70 0x7ffa3414d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:47.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.432+0000 7ffa38c00640 1 --2- 172.21.15.39:0/1793487735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa18004800 0x7ffa340b5b50 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:47.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.432+0000 7ffa38c00640 1 -- 172.21.15.39:0/1793487735 >> 172.21.15.39:0/1793487735 conn(0x7ffa340d0130 msgr2=0x7ffa340cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:47.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.433+0000 7ffa38c00640 1 -- 172.21.15.39:0/1793487735 shutdown_connections 2024-09-21T07:35:47.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.433+0000 7ffa38c00640 1 -- 172.21.15.39:0/1793487735 wait complete. 2024-09-21T07:35:47.468 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T07:35:47.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.700+0000 7f0a2f200640 1 -- 172.21.15.39:0/2098106032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a300b3750 msgr2=0x7f0a300b4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:47.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.700+0000 7f0a2f200640 1 --2- 172.21.15.39:0/2098106032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a300b3750 0x7f0a300b4700 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0a30074310 tx=0x7f0a2002a5f0 comp rx=0 tx=0).stop 2024-09-21T07:35:47.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.700+0000 7f0a2f200640 1 -- 172.21.15.39:0/2098106032 shutdown_connections 2024-09-21T07:35:47.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.700+0000 7f0a2f200640 1 --2- 172.21.15.39:0/2098106032 >> v2:172.21.15.148:3300/0 conn(0x7f0a300d0fb0 0x7f0a3009aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:47.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.700+0000 7f0a2f200640 1 --2- 172.21.15.39:0/2098106032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a300b3750 0x7f0a300b4700 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:47.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.700+0000 7f0a2f200640 1 -- 172.21.15.39:0/2098106032 >> 172.21.15.39:0/2098106032 conn(0x7f0a300b3be0 msgr2=0x7f0a300b3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:47.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.700+0000 7f0a2f200640 1 -- 172.21.15.39:0/2098106032 shutdown_connections 2024-09-21T07:35:47.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.700+0000 7f0a2f200640 1 -- 172.21.15.39:0/2098106032 wait complete. 2024-09-21T07:35:47.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.703+0000 7f0a2f200640 1 Processor -- start 2024-09-21T07:35:47.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.703+0000 7f0a2f200640 1 -- start start 2024-09-21T07:35:47.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.704+0000 7f0a2f200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a3009af70 0x7f0a30161370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:47.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.704+0000 7f0a2f200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0a300b3750 0x7f0a30158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:47.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.704+0000 7f0a2f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a300d0fb0 0x7f0a30159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:47.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.704+0000 7f0a2f200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3006e930 con 0x7f0a300d0fb0 2024-09-21T07:35:47.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.704+0000 7f0a2f200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0a300b6620 con 0x7f0a3009af70 2024-09-21T07:35:47.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.704+0000 7f0a2f200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f0a3006eab0 con 0x7f0a300b3750 2024-09-21T07:35:47.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.704+0000 7f0a2e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a300d0fb0 0x7f0a30159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:47.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.705+0000 7f0a2e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a300d0fb0 0x7f0a30159400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57032/0 (socket says 172.21.15.39:57032) 2024-09-21T07:35:47.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.705+0000 7f0a2e800640 1 -- 172.21.15.39:0/3811935443 learned_addr learned my addr 172.21.15.39:0/3811935443 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:47.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.705+0000 7f0a27e00640 1 --2- 172.21.15.39:0/3811935443 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0a300b3750 0x7f0a30158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:47.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.705+0000 7f0a2de00640 1 --2- 172.21.15.39:0/3811935443 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a3009af70 0x7f0a30161370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:47.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.705+0000 7f0a2e800640 1 -- 172.21.15.39:0/3811935443 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0a300b3750 msgr2=0x7f0a30158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:47.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.705+0000 7f0a2e800640 1 --2- 172.21.15.39:0/3811935443 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0a300b3750 0x7f0a30158ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:47.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.705+0000 7f0a2e800640 1 -- 172.21.15.39:0/3811935443 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a3009af70 msgr2=0x7f0a30161370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:47.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.705+0000 7f0a2e800640 1 --2- 172.21.15.39:0/3811935443 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a3009af70 0x7f0a30161370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:47.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.705+0000 7f0a2e800640 1 -- 172.21.15.39:0/3811935443 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a30158100 con 0x7f0a300d0fb0 2024-09-21T07:35:47.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.705+0000 7f0a2de00640 1 --2- 172.21.15.39:0/3811935443 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a3009af70 0x7f0a30161370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:35:47.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.705+0000 7f0a2e800640 1 --2- 172.21.15.39:0/3811935443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a300d0fb0 0x7f0a30159400 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0a28002230 tx=0x7f0a2800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:47.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.705+0000 7f0a26000640 1 -- 172.21.15.39:0/3811935443 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a28003b70 con 0x7f0a300d0fb0 2024-09-21T07:35:47.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.706+0000 7f0a26000640 1 -- 172.21.15.39:0/3811935443 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a28003d10 con 0x7f0a300d0fb0 2024-09-21T07:35:47.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.706+0000 7f0a2f200640 1 -- 172.21.15.39:0/3811935443 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a301583e0 con 0x7f0a300d0fb0 2024-09-21T07:35:47.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.706+0000 7f0a26000640 1 -- 172.21.15.39:0/3811935443 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a28005330 con 0x7f0a300d0fb0 2024-09-21T07:35:47.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.706+0000 7f0a2f200640 1 -- 172.21.15.39:0/3811935443 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a301589a0 con 0x7f0a300d0fb0 2024-09-21T07:35:47.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.707+0000 7f0a2f200640 1 -- 172.21.15.39:0/3811935443 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a3006f000 con 0x7f0a300d0fb0 2024-09-21T07:35:47.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.708+0000 7f0a26000640 1 -- 172.21.15.39:0/3811935443 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0a280056e0 con 0x7f0a300d0fb0 2024-09-21T07:35:47.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.709+0000 7f0a26000640 1 --2- 172.21.15.39:0/3811935443 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0a0c078bc0 0x7f0a0c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:47.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.709+0000 7f0a26000640 1 -- 172.21.15.39:0/3811935443 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6290+0+0 (secure 0 0 0) 0x7f0a2809af20 con 0x7f0a300d0fb0 2024-09-21T07:35:47.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.710+0000 7f0a2de00640 1 --2- 172.21.15.39:0/3811935443 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0a0c078bc0 0x7f0a0c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:47.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.711+0000 7f0a2de00640 1 --2- 172.21.15.39:0/3811935443 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0a0c078bc0 0x7f0a0c07b0d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0a20034000 tx=0x7f0a20004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:47.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.715+0000 7f0a26000640 1 -- 172.21.15.39:0/3811935443 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0a3009b380 con 0x7f0a300d0fb0 2024-09-21T07:35:47.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:47.913+0000 7f0a2f200640 1 -- 172.21.15.39:0/3811935443 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f0a30069f50 con 0x7f0a300d0fb0 2024-09-21T07:35:48.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.437+0000 7f0a26000640 1 -- 172.21.15.39:0/3811935443 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v76) ==== 170+0+0 (secure 0 0 0) 0x7f0a28063070 con 0x7f0a300d0fb0 2024-09-21T07:35:48.439 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' created 2024-09-21T07:35:48.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.440+0000 7f0a2f200640 1 -- 172.21.15.39:0/3811935443 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0a0c078bc0 msgr2=0x7f0a0c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:48.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.440+0000 7f0a2f200640 1 --2- 172.21.15.39:0/3811935443 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0a0c078bc0 0x7f0a0c07b0d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0a20034000 tx=0x7f0a20004730 comp rx=0 tx=0).stop 2024-09-21T07:35:48.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.440+0000 7f0a2f200640 1 -- 172.21.15.39:0/3811935443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a300d0fb0 msgr2=0x7f0a30159400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:48.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.440+0000 7f0a2f200640 1 --2- 172.21.15.39:0/3811935443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a300d0fb0 0x7f0a30159400 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0a28002230 tx=0x7f0a2800cb90 comp rx=0 tx=0).stop 2024-09-21T07:35:48.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.440+0000 7f0a2f200640 1 -- 172.21.15.39:0/3811935443 shutdown_connections 2024-09-21T07:35:48.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.440+0000 7f0a2f200640 1 --2- 172.21.15.39:0/3811935443 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0a0c078bc0 0x7f0a0c07b0d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:48.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.440+0000 7f0a2f200640 1 --2- 172.21.15.39:0/3811935443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a300d0fb0 0x7f0a30159400 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:48.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.440+0000 7f0a2f200640 1 --2- 172.21.15.39:0/3811935443 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0a300b3750 0x7f0a30158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:48.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.440+0000 7f0a2f200640 1 --2- 172.21.15.39:0/3811935443 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a3009af70 0x7f0a30161370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:48.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.440+0000 7f0a2f200640 1 -- 172.21.15.39:0/3811935443 >> 172.21.15.39:0/3811935443 conn(0x7f0a300b3be0 msgr2=0x7f0a300b5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:48.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.440+0000 7f0a2f200640 1 -- 172.21.15.39:0/3811935443 shutdown_connections 2024-09-21T07:35:48.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.440+0000 7f0a2f200640 1 -- 172.21.15.39:0/3811935443 wait complete. 2024-09-21T07:35:48.482 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T07:35:48.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.711+0000 7f7d55c00640 1 -- 172.21.15.39:0/2610981386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d5015ca50 msgr2=0x7f7d500cc830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:48.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.711+0000 7f7d55c00640 1 --2- 172.21.15.39:0/2610981386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d5015ca50 0x7f7d500cc830 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f7d500b5220 tx=0x7f7d4002a4b0 comp rx=0 tx=0).stop 2024-09-21T07:35:48.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.712+0000 7f7d55c00640 1 -- 172.21.15.39:0/2610981386 shutdown_connections 2024-09-21T07:35:48.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.712+0000 7f7d55c00640 1 --2- 172.21.15.39:0/2610981386 >> v2:172.21.15.148:3300/0 conn(0x7f7d500cf540 0x7f7d5006ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:48.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.712+0000 7f7d55c00640 1 --2- 172.21.15.39:0/2610981386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d5015ca50 0x7f7d500cc830 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:48.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.712+0000 7f7d55c00640 1 -- 172.21.15.39:0/2610981386 >> 172.21.15.39:0/2610981386 conn(0x7f7d5006d410 msgr2=0x7f7d500b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:48.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.712+0000 7f7d55c00640 1 -- 172.21.15.39:0/2610981386 shutdown_connections 2024-09-21T07:35:48.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.712+0000 7f7d55c00640 1 -- 172.21.15.39:0/2610981386 wait complete. 2024-09-21T07:35:48.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.714+0000 7f7d55c00640 1 Processor -- start 2024-09-21T07:35:48.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.714+0000 7f7d55c00640 1 -- start start 2024-09-21T07:35:48.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.715+0000 7f7d55c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7d500cf540 0x7f7d501f9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:48.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.716+0000 7f7d55c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d500cf950 0x7f7d5013f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:48.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.716+0000 7f7d55c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d5015ca50 0x7f7d500b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:48.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.716+0000 7f7d55c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5015de10 con 0x7f7d500cf950 2024-09-21T07:35:48.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.716+0000 7f7d55c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5006f740 con 0x7f7d5015ca50 2024-09-21T07:35:48.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.716+0000 7f7d55c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f7d5015df90 con 0x7f7d500cf540 2024-09-21T07:35:48.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.716+0000 7f7d4f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d500cf950 0x7f7d5013f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:48.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.716+0000 7f7d4f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d500cf950 0x7f7d5013f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57048/0 (socket says 172.21.15.39:57048) 2024-09-21T07:35:48.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.716+0000 7f7d4f400640 1 -- 172.21.15.39:0/2463053057 learned_addr learned my addr 172.21.15.39:0/2463053057 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:48.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.716+0000 7f7d4fe00640 1 --2- 172.21.15.39:0/2463053057 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7d500cf540 0x7f7d501f9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:48.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.716+0000 7f7d55200640 1 --2- 172.21.15.39:0/2463053057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d5015ca50 0x7f7d500b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:48.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.716+0000 7f7d4f400640 1 -- 172.21.15.39:0/2463053057 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7d500cf540 msgr2=0x7f7d501f9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:48.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.716+0000 7f7d4f400640 1 --2- 172.21.15.39:0/2463053057 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7d500cf540 0x7f7d501f9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:48.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.716+0000 7f7d4f400640 1 -- 172.21.15.39:0/2463053057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d5015ca50 msgr2=0x7f7d500b0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:48.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.716+0000 7f7d4f400640 1 --2- 172.21.15.39:0/2463053057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d5015ca50 0x7f7d500b0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:48.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.716+0000 7f7d4f400640 1 -- 172.21.15.39:0/2463053057 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d50145e60 con 0x7f7d500cf950 2024-09-21T07:35:48.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.716+0000 7f7d4fe00640 1 --2- 172.21.15.39:0/2463053057 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7d500cf540 0x7f7d501f9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:35:48.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.716+0000 7f7d55200640 1 --2- 172.21.15.39:0/2463053057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d5015ca50 0x7f7d500b0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:35:48.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.717+0000 7f7d4f400640 1 --2- 172.21.15.39:0/2463053057 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d500cf950 0x7f7d5013f940 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c00d740 tx=0x7f7d3c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:48.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.717+0000 7f7d47e00640 1 -- 172.21.15.39:0/2463053057 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c017070 con 0x7f7d500cf950 2024-09-21T07:35:48.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.717+0000 7f7d47e00640 1 -- 172.21.15.39:0/2463053057 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d3c009d20 con 0x7f7d500cf950 2024-09-21T07:35:48.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.717+0000 7f7d47e00640 1 -- 172.21.15.39:0/2463053057 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c0106c0 con 0x7f7d500cf950 2024-09-21T07:35:48.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.717+0000 7f7d55c00640 1 -- 172.21.15.39:0/2463053057 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d501f9b40 con 0x7f7d500cf950 2024-09-21T07:35:48.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.717+0000 7f7d55c00640 1 -- 172.21.15.39:0/2463053057 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d501f5910 con 0x7f7d500cf950 2024-09-21T07:35:48.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.718+0000 7f7d55c00640 1 -- 172.21.15.39:0/2463053057 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d501565a0 con 0x7f7d500cf950 2024-09-21T07:35:48.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.719+0000 7f7d47e00640 1 -- 172.21.15.39:0/2463053057 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7d3c01a070 con 0x7f7d500cf950 2024-09-21T07:35:48.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.720+0000 7f7d47e00640 1 --2- 172.21.15.39:0/2463053057 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7d30078bc0 0x7f7d3007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:48.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.721+0000 7f7d47e00640 1 -- 172.21.15.39:0/2463053057 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6672+0+0 (secure 0 0 0) 0x7f7d3c09b330 con 0x7f7d500cf950 2024-09-21T07:35:48.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.721+0000 7f7d4fe00640 1 --2- 172.21.15.39:0/2463053057 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7d30078bc0 0x7f7d3007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:48.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.722+0000 7f7d4fe00640 1 --2- 172.21.15.39:0/2463053057 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7d30078bc0 0x7f7d3007b0d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7d40033000 tx=0x7f7d40004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:48.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.726+0000 7f7d47e00640 1 -- 172.21.15.39:0/2463053057 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7d501f9b40 con 0x7f7d500cf950 2024-09-21T07:35:48.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:48.963+0000 7f7d55c00640 1 -- 172.21.15.39:0/2463053057 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f7d5015e440 con 0x7f7d500cf950 2024-09-21T07:35:49.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.439+0000 7f7d47e00640 1 -- 172.21.15.39:0/2463053057 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v40) ==== 174+0+0 (secure 0 0 0) 0x7f7d50145e60 con 0x7f7d500cf950 2024-09-21T07:35:49.441 INFO:teuthology.orchestra.run.smithi039.stderr:new fs with metadata pool 13 and data pool 14 2024-09-21T07:35:49.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.442+0000 7f7d55c00640 1 -- 172.21.15.39:0/2463053057 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7d30078bc0 msgr2=0x7f7d3007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:49.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.442+0000 7f7d55c00640 1 --2- 172.21.15.39:0/2463053057 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7d30078bc0 0x7f7d3007b0d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7d40033000 tx=0x7f7d40004730 comp rx=0 tx=0).stop 2024-09-21T07:35:49.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.442+0000 7f7d55c00640 1 -- 172.21.15.39:0/2463053057 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d500cf950 msgr2=0x7f7d5013f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:49.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.442+0000 7f7d55c00640 1 --2- 172.21.15.39:0/2463053057 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d500cf950 0x7f7d5013f940 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c00d740 tx=0x7f7d3c00dc00 comp rx=0 tx=0).stop 2024-09-21T07:35:49.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.442+0000 7f7d55c00640 1 -- 172.21.15.39:0/2463053057 shutdown_connections 2024-09-21T07:35:49.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.442+0000 7f7d55c00640 1 --2- 172.21.15.39:0/2463053057 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7d30078bc0 0x7f7d3007b0d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:49.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.442+0000 7f7d55c00640 1 --2- 172.21.15.39:0/2463053057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d5015ca50 0x7f7d500b0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:49.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.442+0000 7f7d55c00640 1 --2- 172.21.15.39:0/2463053057 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d500cf950 0x7f7d5013f940 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:49.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.442+0000 7f7d55c00640 1 --2- 172.21.15.39:0/2463053057 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7d500cf540 0x7f7d501f9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:49.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.442+0000 7f7d55c00640 1 -- 172.21.15.39:0/2463053057 >> 172.21.15.39:0/2463053057 conn(0x7f7d5006d410 msgr2=0x7f7d500b3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:49.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.442+0000 7f7d55c00640 1 -- 172.21.15.39:0/2463053057 shutdown_connections 2024-09-21T07:35:49.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.442+0000 7f7d55c00640 1 -- 172.21.15.39:0/2463053057 wait complete. 2024-09-21T07:35:49.486 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:35:49.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.719+0000 7fad52000640 1 -- 172.21.15.39:0/4115093293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad30005250 msgr2=0x7fad3000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:49.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.719+0000 7fad52000640 1 --2- 172.21.15.39:0/4115093293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad30005250 0x7fad3000b9c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fad44009ed0 tx=0x7fad30005830 comp rx=0 tx=0).stop 2024-09-21T07:35:49.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.719+0000 7fad52000640 1 -- 172.21.15.39:0/4115093293 shutdown_connections 2024-09-21T07:35:49.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.719+0000 7fad52000640 1 --2- 172.21.15.39:0/4115093293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad30005250 0x7fad3000b9c0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:49.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.719+0000 7fad52000640 1 -- 172.21.15.39:0/4115093293 >> 172.21.15.39:0/4115093293 conn(0x7fad4c06d7d0 msgr2=0x7fad4c0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:49.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.719+0000 7fad52000640 1 -- 172.21.15.39:0/4115093293 shutdown_connections 2024-09-21T07:35:49.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.719+0000 7fad52000640 1 -- 172.21.15.39:0/4115093293 wait complete. 2024-09-21T07:35:49.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.721+0000 7fad52000640 1 Processor -- start 2024-09-21T07:35:49.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.722+0000 7fad52000640 1 -- start start 2024-09-21T07:35:49.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.723+0000 7fad52000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad30005250 0x7fad4c15e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:49.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.723+0000 7fad52000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad4c1f9be0 0x7fad4c0c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:49.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.723+0000 7fad52000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fad4c1527a0 0x7fad4c073e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:49.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.723+0000 7fad52000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c0cd0e0 con 0x7fad4c1f9be0 2024-09-21T07:35:49.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.723+0000 7fad52000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c0ccf60 con 0x7fad30005250 2024-09-21T07:35:49.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.723+0000 7fad52000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fad4c0cd260 con 0x7fad4c1527a0 2024-09-21T07:35:49.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.723+0000 7fad43e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad4c1f9be0 0x7fad4c0c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:49.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.723+0000 7fad43e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad4c1f9be0 0x7fad4c0c9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57074/0 (socket says 172.21.15.39:57074) 2024-09-21T07:35:49.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.723+0000 7fad43e00640 1 -- 172.21.15.39:0/3879761054 learned_addr learned my addr 172.21.15.39:0/3879761054 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:49.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.724+0000 7fad43e00640 1 -- 172.21.15.39:0/3879761054 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fad4c1527a0 msgr2=0x7fad4c073e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:49.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.724+0000 7fad43e00640 1 --2- 172.21.15.39:0/3879761054 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fad4c1527a0 0x7fad4c073e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:49.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.724+0000 7fad43e00640 1 -- 172.21.15.39:0/3879761054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad30005250 msgr2=0x7fad4c15e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:49.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.724+0000 7fad43e00640 1 --2- 172.21.15.39:0/3879761054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad30005250 0x7fad4c15e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:49.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.724+0000 7fad43e00640 1 -- 172.21.15.39:0/3879761054 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad4c073080 con 0x7fad4c1f9be0 2024-09-21T07:35:49.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.724+0000 7fad43e00640 1 --2- 172.21.15.39:0/3879761054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad4c1f9be0 0x7fad4c0c9370 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fad34004f00 tx=0x7fad34001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:49.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.724+0000 7fad43400640 1 -- 172.21.15.39:0/3879761054 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad34015020 con 0x7fad4c1f9be0 2024-09-21T07:35:49.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.724+0000 7fad43400640 1 -- 172.21.15.39:0/3879761054 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad34011960 con 0x7fad4c1f9be0 2024-09-21T07:35:49.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.724+0000 7fad43400640 1 -- 172.21.15.39:0/3879761054 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad34011d90 con 0x7fad4c1f9be0 2024-09-21T07:35:49.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.724+0000 7fad52000640 1 -- 172.21.15.39:0/3879761054 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad4c073360 con 0x7fad4c1f9be0 2024-09-21T07:35:49.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.725+0000 7fad52000640 1 -- 172.21.15.39:0/3879761054 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad4c0c98b0 con 0x7fad4c1f9be0 2024-09-21T07:35:49.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.726+0000 7fad52000640 1 -- 172.21.15.39:0/3879761054 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad4c15e800 con 0x7fad4c1f9be0 2024-09-21T07:35:49.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.727+0000 7fad43400640 1 -- 172.21.15.39:0/3879761054 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fad34018020 con 0x7fad4c1f9be0 2024-09-21T07:35:49.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.728+0000 7fad43400640 1 --2- 172.21.15.39:0/3879761054 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fad28078b90 0x7fad2807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:49.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.729+0000 7fad43400640 1 -- 172.21.15.39:0/3879761054 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6772+0+0 (secure 0 0 0) 0x7fad3409bdf0 con 0x7fad4c1f9be0 2024-09-21T07:35:49.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.729+0000 7fad4b400640 1 --2- 172.21.15.39:0/3879761054 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fad28078b90 0x7fad2807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:49.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.730+0000 7fad4b400640 1 --2- 172.21.15.39:0/3879761054 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fad28078b90 0x7fad2807b0a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fad3c0047a0 tx=0x7fad3c000ed0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:49.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.734+0000 7fad43400640 1 -- 172.21.15.39:0/3879761054 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fad3401f020 con 0x7fad4c1f9be0 2024-09-21T07:35:49.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.929+0000 7fad52000640 1 -- 172.21.15.39:0/3879761054 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad4c0b42c0 con 0x7fad4c1f9be0 2024-09-21T07:35:49.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.931+0000 7fad43400640 1 -- 172.21.15.39:0/3879761054 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22100 (secure 0 0 0) 0x7fad34063d60 con 0x7fad4c1f9be0 2024-09-21T07:35:49.933 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:35:49.933 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":77,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:35:48.965462+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:35:46.914721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-21T07:35:47.926559+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:35:49.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.933+0000 7fad52000640 1 -- 172.21.15.39:0/3879761054 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fad28078b90 msgr2=0x7fad2807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:49.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.933+0000 7fad52000640 1 --2- 172.21.15.39:0/3879761054 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fad28078b90 0x7fad2807b0a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fad3c0047a0 tx=0x7fad3c000ed0 comp rx=0 tx=0).stop 2024-09-21T07:35:49.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.933+0000 7fad52000640 1 -- 172.21.15.39:0/3879761054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad4c1f9be0 msgr2=0x7fad4c0c9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:49.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.933+0000 7fad52000640 1 --2- 172.21.15.39:0/3879761054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad4c1f9be0 0x7fad4c0c9370 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fad34004f00 tx=0x7fad34001630 comp rx=0 tx=0).stop 2024-09-21T07:35:49.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.933+0000 7fad52000640 1 -- 172.21.15.39:0/3879761054 shutdown_connections 2024-09-21T07:35:49.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.933+0000 7fad52000640 1 --2- 172.21.15.39:0/3879761054 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fad28078b90 0x7fad2807b0a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:49.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.933+0000 7fad52000640 1 --2- 172.21.15.39:0/3879761054 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fad4c1527a0 0x7fad4c073e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:49.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.933+0000 7fad52000640 1 --2- 172.21.15.39:0/3879761054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad4c1f9be0 0x7fad4c0c9370 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:49.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.933+0000 7fad52000640 1 --2- 172.21.15.39:0/3879761054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad30005250 0x7fad4c15e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:49.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.933+0000 7fad52000640 1 -- 172.21.15.39:0/3879761054 >> 172.21.15.39:0/3879761054 conn(0x7fad4c06d7d0 msgr2=0x7fad4c0b16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:49.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.933+0000 7fad52000640 1 -- 172.21.15.39:0/3879761054 shutdown_connections 2024-09-21T07:35:49.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:49.933+0000 7fad52000640 1 -- 172.21.15.39:0/3879761054 wait complete. 2024-09-21T07:35:49.957 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:35:50.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.182+0000 7f36d4800640 1 -- 172.21.15.39:0/2860075058 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36cc15d8a0 msgr2=0x7f36cc0b40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:50.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.182+0000 7f36d4800640 1 --2- 172.21.15.39:0/2860075058 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36cc15d8a0 0x7f36cc0b40b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f36bc009830 tx=0x7f36bc02a4a0 comp rx=0 tx=0).stop 2024-09-21T07:35:50.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.182+0000 7f36d4800640 1 -- 172.21.15.39:0/2860075058 shutdown_connections 2024-09-21T07:35:50.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.182+0000 7f36d4800640 1 --2- 172.21.15.39:0/2860075058 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36cc15d8a0 0x7f36cc0b40b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:50.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.182+0000 7f36d4800640 1 --2- 172.21.15.39:0/2860075058 >> v2:172.21.15.148:3300/0 conn(0x7f36cc0cc1e0 0x7f36cc0af3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:50.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.182+0000 7f36d4800640 1 -- 172.21.15.39:0/2860075058 >> 172.21.15.39:0/2860075058 conn(0x7f36cc06d7d0 msgr2=0x7f36cc0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:50.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.182+0000 7f36d4800640 1 -- 172.21.15.39:0/2860075058 shutdown_connections 2024-09-21T07:35:50.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.182+0000 7f36d4800640 1 -- 172.21.15.39:0/2860075058 wait complete. 2024-09-21T07:35:50.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.184+0000 7f36d4800640 1 Processor -- start 2024-09-21T07:35:50.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.184+0000 7f36d4800640 1 -- start start 2024-09-21T07:35:50.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.186+0000 7f36d4800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36cc0cc1e0 0x7f36cc1547e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:50.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.186+0000 7f36d4800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36cc0ccb50 0x7f36cc143a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:50.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.186+0000 7f36d4800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36cc15d8a0 0x7f36cc1e9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:50.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.186+0000 7f36d4800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f36cc153520 con 0x7f36cc0ccb50 2024-09-21T07:35:50.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.186+0000 7f36d4800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f36cc1533a0 con 0x7f36cc0cc1e0 2024-09-21T07:35:50.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.186+0000 7f36d4800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f36cc1536a0 con 0x7f36cc15d8a0 2024-09-21T07:35:50.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.186+0000 7f36d1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36cc0ccb50 0x7f36cc143a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:50.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.186+0000 7f36d1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36cc0ccb50 0x7f36cc143a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57084/0 (socket says 172.21.15.39:57084) 2024-09-21T07:35:50.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.186+0000 7f36d1400640 1 -- 172.21.15.39:0/808689048 learned_addr learned my addr 172.21.15.39:0/808689048 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:50.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.186+0000 7f36d1e00640 1 --2- 172.21.15.39:0/808689048 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36cc0cc1e0 0x7f36cc1547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:50.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.186+0000 7f36d2800640 1 --2- 172.21.15.39:0/808689048 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36cc15d8a0 0x7f36cc1e9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:50.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.187+0000 7f36d1400640 1 -- 172.21.15.39:0/808689048 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36cc15d8a0 msgr2=0x7f36cc1e9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:50.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.187+0000 7f36d1400640 1 --2- 172.21.15.39:0/808689048 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36cc15d8a0 0x7f36cc1e9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:50.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.187+0000 7f36d1400640 1 -- 172.21.15.39:0/808689048 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36cc0cc1e0 msgr2=0x7f36cc1547e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:50.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.187+0000 7f36d1400640 1 --2- 172.21.15.39:0/808689048 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36cc0cc1e0 0x7f36cc1547e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:50.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.187+0000 7f36d1400640 1 -- 172.21.15.39:0/808689048 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36cc154d20 con 0x7f36cc0ccb50 2024-09-21T07:35:50.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.187+0000 7f36d1400640 1 --2- 172.21.15.39:0/808689048 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36cc0ccb50 0x7f36cc143a90 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f36bc009950 tx=0x7f36bc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:50.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.187+0000 7f36c6000640 1 -- 172.21.15.39:0/808689048 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc005c90 con 0x7f36cc0ccb50 2024-09-21T07:35:50.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.187+0000 7f36c6000640 1 -- 172.21.15.39:0/808689048 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36bc005020 con 0x7f36cc0ccb50 2024-09-21T07:35:50.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.187+0000 7f36c6000640 1 -- 172.21.15.39:0/808689048 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc005450 con 0x7f36cc0ccb50 2024-09-21T07:35:50.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.187+0000 7f36d4800640 1 -- 172.21.15.39:0/808689048 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36cc1f5f10 con 0x7f36cc0ccb50 2024-09-21T07:35:50.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.188+0000 7f36d4800640 1 -- 172.21.15.39:0/808689048 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36cc1f64d0 con 0x7f36cc0ccb50 2024-09-21T07:35:50.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.189+0000 7f36d4800640 1 -- 172.21.15.39:0/808689048 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36cc153b50 con 0x7f36cc0ccb50 2024-09-21T07:35:50.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.190+0000 7f36c6000640 1 -- 172.21.15.39:0/808689048 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f36bc0317c0 con 0x7f36cc0ccb50 2024-09-21T07:35:50.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.191+0000 7f36c6000640 1 --2- 172.21.15.39:0/808689048 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f36b0078bc0 0x7f36b007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:50.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.192+0000 7f36c6000640 1 -- 172.21.15.39:0/808689048 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6772+0+0 (secure 0 0 0) 0x7f36bc0bc070 con 0x7f36cc0ccb50 2024-09-21T07:35:50.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.192+0000 7f36d1e00640 1 --2- 172.21.15.39:0/808689048 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f36b0078bc0 0x7f36b007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:50.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.193+0000 7f36d1e00640 1 --2- 172.21.15.39:0/808689048 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f36b0078bc0 0x7f36b007b0d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f36c0009b30 tx=0x7f36c0005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:50.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.196+0000 7f36c6000640 1 -- 172.21.15.39:0/808689048 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f36bc03f030 con 0x7f36cc0ccb50 2024-09-21T07:35:50.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.394+0000 7f36d4800640 1 -- 172.21.15.39:0/808689048 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f36cc0b1790 con 0x7f36cc0ccb50 2024-09-21T07:35:50.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.395+0000 7f36c6000640 1 -- 172.21.15.39:0/808689048 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22100 (secure 0 0 0) 0x7f36bc083ac0 con 0x7f36cc0ccb50 2024-09-21T07:35:50.398 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:35:50.398 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":77,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:35:48.965462+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:35:46.914721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-21T07:35:47.926559+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:35:50.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.398+0000 7f36d4800640 1 -- 172.21.15.39:0/808689048 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f36b0078bc0 msgr2=0x7f36b007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:50.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.398+0000 7f36d4800640 1 --2- 172.21.15.39:0/808689048 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f36b0078bc0 0x7f36b007b0d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f36c0009b30 tx=0x7f36c0005f90 comp rx=0 tx=0).stop 2024-09-21T07:35:50.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.398+0000 7f36d4800640 1 -- 172.21.15.39:0/808689048 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36cc0ccb50 msgr2=0x7f36cc143a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:50.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.398+0000 7f36d4800640 1 --2- 172.21.15.39:0/808689048 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36cc0ccb50 0x7f36cc143a90 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f36bc009950 tx=0x7f36bc003c10 comp rx=0 tx=0).stop 2024-09-21T07:35:50.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.398+0000 7f36d4800640 1 -- 172.21.15.39:0/808689048 shutdown_connections 2024-09-21T07:35:50.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.398+0000 7f36d4800640 1 --2- 172.21.15.39:0/808689048 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f36b0078bc0 0x7f36b007b0d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:50.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.398+0000 7f36d4800640 1 --2- 172.21.15.39:0/808689048 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36cc15d8a0 0x7f36cc1e9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:50.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.398+0000 7f36d4800640 1 --2- 172.21.15.39:0/808689048 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36cc0ccb50 0x7f36cc143a90 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:50.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.398+0000 7f36d4800640 1 --2- 172.21.15.39:0/808689048 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36cc0cc1e0 0x7f36cc1547e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:50.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.398+0000 7f36d4800640 1 -- 172.21.15.39:0/808689048 >> 172.21.15.39:0/808689048 conn(0x7f36cc06d7d0 msgr2=0x7f36cc0afc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:50.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.398+0000 7f36d4800640 1 -- 172.21.15.39:0/808689048 shutdown_connections 2024-09-21T07:35:50.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.398+0000 7f36d4800640 1 -- 172.21.15.39:0/808689048 wait complete. 2024-09-21T07:35:50.426 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T07:35:50.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.656+0000 7f6ce1a00640 1 -- 172.21.15.39:0/4235553427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cdc0ccc30 msgr2=0x7f6cdc0cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:50.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.656+0000 7f6ce1a00640 1 --2- 172.21.15.39:0/4235553427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cdc0ccc30 0x7f6cdc0cec70 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc005fd0 tx=0x7f6ccc02a540 comp rx=0 tx=0).stop 2024-09-21T07:35:50.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.656+0000 7f6ce1a00640 1 -- 172.21.15.39:0/4235553427 shutdown_connections 2024-09-21T07:35:50.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.656+0000 7f6ce1a00640 1 --2- 172.21.15.39:0/4235553427 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6cdc0cbe00 0x7f6cdc071860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:50.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.656+0000 7f6ce1a00640 1 --2- 172.21.15.39:0/4235553427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cdc0ccc30 0x7f6cdc0cec70 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:50.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.656+0000 7f6ce1a00640 1 -- 172.21.15.39:0/4235553427 >> 172.21.15.39:0/4235553427 conn(0x7f6cdc0cdb20 msgr2=0x7f6cdc0cdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:50.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.656+0000 7f6ce1a00640 1 -- 172.21.15.39:0/4235553427 shutdown_connections 2024-09-21T07:35:50.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.656+0000 7f6ce1a00640 1 -- 172.21.15.39:0/4235553427 wait complete. 2024-09-21T07:35:50.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.659+0000 7f6ce1a00640 1 Processor -- start 2024-09-21T07:35:50.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.659+0000 7f6ce1a00640 1 -- start start 2024-09-21T07:35:50.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.661+0000 7f6ce1a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6cdc0ccc30 0x7f6cdc0b6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:50.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.661+0000 7f6ce1a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6cdc0ce640 0x7f6cdc0b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:50.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.661+0000 7f6ce1a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cdc0b4710 0x7f6cdc1e90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:50.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.661+0000 7f6ce1a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6cdc071f20 con 0x7f6cdc0b4710 2024-09-21T07:35:50.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.661+0000 7f6ce1a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6cdc071da0 con 0x7f6cdc0ce640 2024-09-21T07:35:50.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.661+0000 7f6ce1a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f6cdc0720a0 con 0x7f6cdc0ccc30 2024-09-21T07:35:50.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.661+0000 7f6ce1000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cdc0b4710 0x7f6cdc1e90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:50.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.661+0000 7f6ce1000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cdc0b4710 0x7f6cdc1e90f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57100/0 (socket says 172.21.15.39:57100) 2024-09-21T07:35:50.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.661+0000 7f6ce1000640 1 -- 172.21.15.39:0/1226113536 learned_addr learned my addr 172.21.15.39:0/1226113536 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:50.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.661+0000 7f6cdbe00640 1 --2- 172.21.15.39:0/1226113536 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6cdc0ccc30 0x7f6cdc0b6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:50.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.661+0000 7f6cdb400640 1 --2- 172.21.15.39:0/1226113536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6cdc0ce640 0x7f6cdc0b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:50.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.662+0000 7f6ce1000640 1 -- 172.21.15.39:0/1226113536 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6cdc0ccc30 msgr2=0x7f6cdc0b6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:50.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.662+0000 7f6ce1000640 1 --2- 172.21.15.39:0/1226113536 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6cdc0ccc30 0x7f6cdc0b6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:50.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.662+0000 7f6ce1000640 1 -- 172.21.15.39:0/1226113536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6cdc0ce640 msgr2=0x7f6cdc0b41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:50.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.662+0000 7f6ce1000640 1 --2- 172.21.15.39:0/1226113536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6cdc0ce640 0x7f6cdc0b41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:50.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.662+0000 7f6ce1000640 1 -- 172.21.15.39:0/1226113536 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cdc1f5e00 con 0x7f6cdc0b4710 2024-09-21T07:35:50.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.662+0000 7f6cdb400640 1 --2- 172.21.15.39:0/1226113536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6cdc0ce640 0x7f6cdc0b41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:35:50.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.662+0000 7f6ce1000640 1 --2- 172.21.15.39:0/1226113536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cdc0b4710 0x7f6cdc1e90f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6cd400e990 tx=0x7f6cd400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:50.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.662+0000 7f6cd3e00640 1 -- 172.21.15.39:0/1226113536 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd4004d20 con 0x7f6cdc0b4710 2024-09-21T07:35:50.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.662+0000 7f6cd3e00640 1 -- 172.21.15.39:0/1226113536 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6cd400caa0 con 0x7f6cdc0b4710 2024-09-21T07:35:50.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.662+0000 7f6ce1a00640 1 -- 172.21.15.39:0/1226113536 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cdc1f60a0 con 0x7f6cdc0b4710 2024-09-21T07:35:50.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.662+0000 7f6cd3e00640 1 -- 172.21.15.39:0/1226113536 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd4005020 con 0x7f6cdc0b4710 2024-09-21T07:35:50.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.662+0000 7f6ce1a00640 1 -- 172.21.15.39:0/1226113536 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cdc1f6660 con 0x7f6cdc0b4710 2024-09-21T07:35:50.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.664+0000 7f6ce1a00640 1 -- 172.21.15.39:0/1226113536 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cdc071860 con 0x7f6cdc0b4710 2024-09-21T07:35:50.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.665+0000 7f6cd3e00640 1 -- 172.21.15.39:0/1226113536 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6cd40051c0 con 0x7f6cdc0b4710 2024-09-21T07:35:50.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.666+0000 7f6cd3e00640 1 --2- 172.21.15.39:0/1226113536 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6cbc078b90 0x7f6cbc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:50.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.666+0000 7f6cd3e00640 1 -- 172.21.15.39:0/1226113536 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6772+0+0 (secure 0 0 0) 0x7f6cd409b9a0 con 0x7f6cdc0b4710 2024-09-21T07:35:50.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.666+0000 7f6cdbe00640 1 --2- 172.21.15.39:0/1226113536 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6cbc078b90 0x7f6cbc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:50.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.667+0000 7f6cdbe00640 1 --2- 172.21.15.39:0/1226113536 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6cbc078b90 0x7f6cbc07b0a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc033000 tx=0x7f6ccc004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:50.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.670+0000 7f6cd3e00640 1 -- 172.21.15.39:0/1226113536 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6cdc1f5e00 con 0x7f6cdc0b4710 2024-09-21T07:35:50.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:50.905+0000 7f6ce1a00640 1 -- 172.21.15.39:0/1226113536 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f6cdc157ab0 con 0x7f6cdc0b4710 2024-09-21T07:35:51.420 INFO:tasks.ceph.mds.c.smithi039.stderr:2024-09-21T07:35:51.418+0000 7fcd10a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:35:51.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.471+0000 7f6cd3e00640 1 -- 172.21.15.39:0/1226113536 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v43) ==== 118+0+0 (secure 0 0 0) 0x7f6cd4016030 con 0x7f6cdc0b4710 2024-09-21T07:35:51.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.473+0000 7f6ce1a00640 1 -- 172.21.15.39:0/1226113536 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6cbc078b90 msgr2=0x7f6cbc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:51.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.473+0000 7f6ce1a00640 1 --2- 172.21.15.39:0/1226113536 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6cbc078b90 0x7f6cbc07b0a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc033000 tx=0x7f6ccc004730 comp rx=0 tx=0).stop 2024-09-21T07:35:51.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.474+0000 7f6ce1a00640 1 -- 172.21.15.39:0/1226113536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cdc0b4710 msgr2=0x7f6cdc1e90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:51.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.474+0000 7f6ce1a00640 1 --2- 172.21.15.39:0/1226113536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cdc0b4710 0x7f6cdc1e90f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6cd400e990 tx=0x7f6cd400ee50 comp rx=0 tx=0).stop 2024-09-21T07:35:51.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.474+0000 7f6ce1a00640 1 -- 172.21.15.39:0/1226113536 shutdown_connections 2024-09-21T07:35:51.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.474+0000 7f6ce1a00640 1 --2- 172.21.15.39:0/1226113536 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6cbc078b90 0x7f6cbc07b0a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:51.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.474+0000 7f6ce1a00640 1 --2- 172.21.15.39:0/1226113536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cdc0b4710 0x7f6cdc1e90f0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:51.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.474+0000 7f6ce1a00640 1 --2- 172.21.15.39:0/1226113536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6cdc0ce640 0x7f6cdc0b41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:51.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.474+0000 7f6ce1a00640 1 --2- 172.21.15.39:0/1226113536 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6cdc0ccc30 0x7f6cdc0b6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:51.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.474+0000 7f6ce1a00640 1 -- 172.21.15.39:0/1226113536 >> 172.21.15.39:0/1226113536 conn(0x7f6cdc0cdb20 msgr2=0x7f6cdc0cbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:51.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.474+0000 7f6ce1a00640 1 -- 172.21.15.39:0/1226113536 shutdown_connections 2024-09-21T07:35:51.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.474+0000 7f6ce1a00640 1 -- 172.21.15.39:0/1226113536 wait complete. 2024-09-21T07:35:51.499 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T07:35:51.499 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T07:35:51.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.729+0000 7f792d800640 1 -- 172.21.15.39:0/4124319554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f792815d4f0 msgr2=0x7f792806e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:51.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.729+0000 7f792d800640 1 --2- 172.21.15.39:0/4124319554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f792815d4f0 0x7f792806e2d0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f7918009830 tx=0x7f791802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:35:51.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.729+0000 7f792d800640 1 -- 172.21.15.39:0/4124319554 shutdown_connections 2024-09-21T07:35:51.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.729+0000 7f792d800640 1 --2- 172.21.15.39:0/4124319554 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79280d0130 0x7f792806fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:51.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.729+0000 7f792d800640 1 --2- 172.21.15.39:0/4124319554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f792815d4f0 0x7f792806e2d0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:51.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.729+0000 7f792d800640 1 -- 172.21.15.39:0/4124319554 >> 172.21.15.39:0/4124319554 conn(0x7f79280d0840 msgr2=0x7f79280d0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:51.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.729+0000 7f792d800640 1 -- 172.21.15.39:0/4124319554 shutdown_connections 2024-09-21T07:35:51.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.729+0000 7f792d800640 1 -- 172.21.15.39:0/4124319554 wait complete. 2024-09-21T07:35:51.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.731+0000 7f792d800640 1 Processor -- start 2024-09-21T07:35:51.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.732+0000 7f792d800640 1 -- start start 2024-09-21T07:35:51.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.733+0000 7f792d800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79280cf610 0x7f7928147110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:51.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.733+0000 7f792d800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79280d0130 0x7f7928159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:51.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.733+0000 7f792d800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f792815d4f0 0x7f792815a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:51.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.733+0000 7f792d800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f79280703a0 con 0x7f792815d4f0 2024-09-21T07:35:51.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.733+0000 7f792d800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7928070220 con 0x7f79280d0130 2024-09-21T07:35:51.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.733+0000 7f792d800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f7928070520 con 0x7f79280cf610 2024-09-21T07:35:51.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.733+0000 7f792ce00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f792815d4f0 0x7f792815a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:51.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.733+0000 7f792ce00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f792815d4f0 0x7f792815a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57122/0 (socket says 172.21.15.39:57122) 2024-09-21T07:35:51.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.733+0000 7f792ce00640 1 -- 172.21.15.39:0/1280679850 learned_addr learned my addr 172.21.15.39:0/1280679850 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:51.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.734+0000 7f7927e00640 1 --2- 172.21.15.39:0/1280679850 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79280cf610 0x7f7928147110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:51.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.734+0000 7f792ce00640 1 -- 172.21.15.39:0/1280679850 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79280cf610 msgr2=0x7f7928147110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:51.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.734+0000 7f792ce00640 1 --2- 172.21.15.39:0/1280679850 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79280cf610 0x7f7928147110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:51.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.734+0000 7f792ce00640 1 -- 172.21.15.39:0/1280679850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79280d0130 msgr2=0x7f7928159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:51.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.734+0000 7f791f400640 1 --2- 172.21.15.39:0/1280679850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79280d0130 0x7f7928159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:51.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.734+0000 7f792ce00640 1 --2- 172.21.15.39:0/1280679850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79280d0130 0x7f7928159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:51.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.734+0000 7f792ce00640 1 -- 172.21.15.39:0/1280679850 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7928157130 con 0x7f792815d4f0 2024-09-21T07:35:51.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.734+0000 7f791f400640 1 --2- 172.21.15.39:0/1280679850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79280d0130 0x7f7928159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:35:51.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.734+0000 7f792ce00640 1 --2- 172.21.15.39:0/1280679850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f792815d4f0 0x7f792815a4f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f792000da90 tx=0x7f792000df50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:51.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.734+0000 7f7924c00640 1 -- 172.21.15.39:0/1280679850 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7920017070 con 0x7f792815d4f0 2024-09-21T07:35:51.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.734+0000 7f7924c00640 1 -- 172.21.15.39:0/1280679850 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7920003970 con 0x7f792815d4f0 2024-09-21T07:35:51.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.734+0000 7f792d800640 1 -- 172.21.15.39:0/1280679850 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7928147650 con 0x7f792815d4f0 2024-09-21T07:35:51.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.735+0000 7f7924c00640 1 -- 172.21.15.39:0/1280679850 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7920003dc0 con 0x7f792815d4f0 2024-09-21T07:35:51.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.735+0000 7f792d800640 1 -- 172.21.15.39:0/1280679850 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79280b0a80 con 0x7f792815d4f0 2024-09-21T07:35:51.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.736+0000 7f792d800640 1 -- 172.21.15.39:0/1280679850 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7928070a60 con 0x7f792815d4f0 2024-09-21T07:35:51.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.737+0000 7f7924c00640 1 -- 172.21.15.39:0/1280679850 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7920003f60 con 0x7f792815d4f0 2024-09-21T07:35:51.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.738+0000 7f7924c00640 1 --2- 172.21.15.39:0/1280679850 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7904078bc0 0x7f790407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:51.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.738+0000 7f7924c00640 1 -- 172.21.15.39:0/1280679850 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6772+0+0 (secure 0 0 0) 0x7f792009b570 con 0x7f792815d4f0 2024-09-21T07:35:51.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.738+0000 7f7927e00640 1 --2- 172.21.15.39:0/1280679850 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7904078bc0 0x7f790407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:51.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.739+0000 7f7927e00640 1 --2- 172.21.15.39:0/1280679850 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7904078bc0 0x7f790407b0d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7918034000 tx=0x7f7918004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:51.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.742+0000 7f7924c00640 1 -- 172.21.15.39:0/1280679850 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7920063570 con 0x7f792815d4f0 2024-09-21T07:35:51.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:51.979+0000 7f792d800640 1 -- 172.21.15.39:0/1280679850 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f79280b04f0 con 0x7f792815d4f0 2024-09-21T07:35:52.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.478+0000 7f7924c00640 1 -- 172.21.15.39:0/1280679850 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v44) ==== 122+0+0 (secure 0 0 0) 0x7f7920062e70 con 0x7f792815d4f0 2024-09-21T07:35:52.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.481+0000 7f792d800640 1 -- 172.21.15.39:0/1280679850 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7904078bc0 msgr2=0x7f790407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:52.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.481+0000 7f792d800640 1 --2- 172.21.15.39:0/1280679850 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7904078bc0 0x7f790407b0d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7918034000 tx=0x7f7918004730 comp rx=0 tx=0).stop 2024-09-21T07:35:52.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.481+0000 7f792d800640 1 -- 172.21.15.39:0/1280679850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f792815d4f0 msgr2=0x7f792815a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:52.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.481+0000 7f792d800640 1 --2- 172.21.15.39:0/1280679850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f792815d4f0 0x7f792815a4f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f792000da90 tx=0x7f792000df50 comp rx=0 tx=0).stop 2024-09-21T07:35:52.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.481+0000 7f792d800640 1 -- 172.21.15.39:0/1280679850 shutdown_connections 2024-09-21T07:35:52.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.481+0000 7f792d800640 1 --2- 172.21.15.39:0/1280679850 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7904078bc0 0x7f790407b0d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:52.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.481+0000 7f792d800640 1 --2- 172.21.15.39:0/1280679850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f792815d4f0 0x7f792815a4f0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:52.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.481+0000 7f792d800640 1 --2- 172.21.15.39:0/1280679850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79280d0130 0x7f7928159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:52.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.481+0000 7f792d800640 1 --2- 172.21.15.39:0/1280679850 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f79280cf610 0x7f7928147110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:52.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.481+0000 7f792d800640 1 -- 172.21.15.39:0/1280679850 >> 172.21.15.39:0/1280679850 conn(0x7f79280d0840 msgr2=0x7f792815fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:52.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.482+0000 7f792d800640 1 -- 172.21.15.39:0/1280679850 shutdown_connections 2024-09-21T07:35:52.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.482+0000 7f792d800640 1 -- 172.21.15.39:0/1280679850 wait complete. 2024-09-21T07:35:52.509 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T07:35:52.510 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:35:52.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.737+0000 7ff2a8800640 1 -- 172.21.15.39:0/1307338708 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2a0073e50 msgr2=0x7ff2a0074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:52.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.737+0000 7ff2a8800640 1 --2- 172.21.15.39:0/1307338708 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2a0073e50 0x7ff2a0074260 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff29c005fd0 tx=0x7ff29c02aa90 comp rx=0 tx=0).stop 2024-09-21T07:35:52.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.737+0000 7ff2a8800640 1 -- 172.21.15.39:0/1307338708 shutdown_connections 2024-09-21T07:35:52.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.737+0000 7ff2a8800640 1 --2- 172.21.15.39:0/1307338708 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2a0073e50 0x7ff2a0074260 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:52.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.737+0000 7ff2a8800640 1 --2- 172.21.15.39:0/1307338708 >> v2:172.21.15.148:3300/0 conn(0x7ff2a00cfc00 0x7ff2a00ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:52.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.737+0000 7ff2a8800640 1 -- 172.21.15.39:0/1307338708 >> 172.21.15.39:0/1307338708 conn(0x7ff2a00d15f0 msgr2=0x7ff2a0073740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:52.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.737+0000 7ff2a8800640 1 -- 172.21.15.39:0/1307338708 shutdown_connections 2024-09-21T07:35:52.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.737+0000 7ff2a8800640 1 -- 172.21.15.39:0/1307338708 wait complete. 2024-09-21T07:35:52.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.739+0000 7ff2a8800640 1 Processor -- start 2024-09-21T07:35:52.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.739+0000 7ff2a8800640 1 -- start start 2024-09-21T07:35:52.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.741+0000 7ff2a8800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2a0073e50 0x7ff2a0152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:52.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.741+0000 7ff2a8800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2a00cfc00 0x7ff2a014fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:52.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.741+0000 7ff2a8800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2a0153970 0x7ff2a0150400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:52.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.741+0000 7ff2a8800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a015e2e0 con 0x7ff2a0073e50 2024-09-21T07:35:52.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.741+0000 7ff2a8800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a015e160 con 0x7ff2a00cfc00 2024-09-21T07:35:52.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.741+0000 7ff2a8800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7ff2a015e460 con 0x7ff2a0153970 2024-09-21T07:35:52.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.741+0000 7ff2a5e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2a0073e50 0x7ff2a0152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:52.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.741+0000 7ff2a5e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2a0073e50 0x7ff2a0152b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45498/0 (socket says 172.21.15.39:45498) 2024-09-21T07:35:52.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.741+0000 7ff2a5e00640 1 -- 172.21.15.39:0/482867850 learned_addr learned my addr 172.21.15.39:0/482867850 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:52.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.741+0000 7ff2a5400640 1 --2- 172.21.15.39:0/482867850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2a00cfc00 0x7ff2a014fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:52.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.741+0000 7ff2a6800640 1 --2- 172.21.15.39:0/482867850 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2a0153970 0x7ff2a0150400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:52.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.742+0000 7ff2a5e00640 1 -- 172.21.15.39:0/482867850 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2a0153970 msgr2=0x7ff2a0150400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:52.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.742+0000 7ff2a5e00640 1 --2- 172.21.15.39:0/482867850 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2a0153970 0x7ff2a0150400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:52.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.742+0000 7ff2a5e00640 1 -- 172.21.15.39:0/482867850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2a00cfc00 msgr2=0x7ff2a014fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:52.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.742+0000 7ff2a5e00640 1 --2- 172.21.15.39:0/482867850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2a00cfc00 0x7ff2a014fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:52.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.742+0000 7ff2a5e00640 1 -- 172.21.15.39:0/482867850 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a015a370 con 0x7ff2a0073e50 2024-09-21T07:35:52.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.742+0000 7ff2a5e00640 1 --2- 172.21.15.39:0/482867850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2a0073e50 0x7ff2a0152b40 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ff29400c730 tx=0x7ff29400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:52.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.742+0000 7ff29a000640 1 -- 172.21.15.39:0/482867850 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2940052d0 con 0x7ff2a0073e50 2024-09-21T07:35:52.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.742+0000 7ff29a000640 1 -- 172.21.15.39:0/482867850 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff294005490 con 0x7ff2a0073e50 2024-09-21T07:35:52.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.742+0000 7ff2a8800640 1 -- 172.21.15.39:0/482867850 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2a01f6bd0 con 0x7ff2a0073e50 2024-09-21T07:35:52.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.742+0000 7ff29a000640 1 -- 172.21.15.39:0/482867850 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29400a8a0 con 0x7ff2a0073e50 2024-09-21T07:35:52.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.742+0000 7ff2a8800640 1 -- 172.21.15.39:0/482867850 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2a01522e0 con 0x7ff2a0073e50 2024-09-21T07:35:52.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.743+0000 7ff2a8800640 1 -- 172.21.15.39:0/482867850 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2a0152540 con 0x7ff2a0073e50 2024-09-21T07:35:52.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.745+0000 7ff29a000640 1 -- 172.21.15.39:0/482867850 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff29400aa40 con 0x7ff2a0073e50 2024-09-21T07:35:52.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.746+0000 7ff29a000640 1 --2- 172.21.15.39:0/482867850 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff284078b90 0x7ff28407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:52.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.746+0000 7ff29a000640 1 -- 172.21.15.39:0/482867850 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6772+0+0 (secure 0 0 0) 0x7ff29409aa80 con 0x7ff2a0073e50 2024-09-21T07:35:52.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.746+0000 7ff2a5400640 1 --2- 172.21.15.39:0/482867850 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff284078b90 0x7ff28407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:52.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.747+0000 7ff2a5400640 1 --2- 172.21.15.39:0/482867850 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff284078b90 0x7ff28407b0a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0150e40 tx=0x7ff290005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:52.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.750+0000 7ff29a000640 1 -- 172.21.15.39:0/482867850 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff2a015cd90 con 0x7ff2a0073e50 2024-09-21T07:35:52.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.987+0000 7ff2a8800640 1 -- 172.21.15.39:0/482867850 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2a015e930 con 0x7ff2a0073e50 2024-09-21T07:35:52.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.988+0000 7ff29a000640 1 -- 172.21.15.39:0/482867850 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5225 (secure 0 0 0) 0x7ff2940624d0 con 0x7ff2a0073e50 2024-09-21T07:35:52.990 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:35:52.990 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":44,"btime":"2024-09-21T07:35:52:472978+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5426,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/707367131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":707367131},{"type":"v1","addr":"172.21.15.39:6835","nonce":707367131}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5429,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/784956164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":784956164},{"type":"v1","addr":"172.21.15.148:6836","nonce":784956164}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5432,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3742990359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3742990359},{"type":"v1","addr":"172.21.15.148:6837","nonce":3742990359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:35:48.965416+0000","modified":"2024-09-21T07:35:51.980637+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5842},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5842":{"gid":5842,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/1032917299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":1032917299},{"type":"v1","addr":"172.21.15.39:6834","nonce":1032917299}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5842,"qdb_cluster":[5842]},"id":5}]} 2024-09-21T07:35:52.990 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 44 2024-09-21T07:35:52.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.990+0000 7ff2a8800640 1 -- 172.21.15.39:0/482867850 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff284078b90 msgr2=0x7ff28407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:52.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.990+0000 7ff2a8800640 1 --2- 172.21.15.39:0/482867850 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff284078b90 0x7ff28407b0a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0150e40 tx=0x7ff290005f90 comp rx=0 tx=0).stop 2024-09-21T07:35:52.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.990+0000 7ff2a8800640 1 -- 172.21.15.39:0/482867850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2a0073e50 msgr2=0x7ff2a0152b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:52.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.990+0000 7ff2a8800640 1 --2- 172.21.15.39:0/482867850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2a0073e50 0x7ff2a0152b40 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ff29400c730 tx=0x7ff29400cbf0 comp rx=0 tx=0).stop 2024-09-21T07:35:52.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.991+0000 7ff2a8800640 1 -- 172.21.15.39:0/482867850 shutdown_connections 2024-09-21T07:35:52.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.991+0000 7ff2a8800640 1 --2- 172.21.15.39:0/482867850 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff284078b90 0x7ff28407b0a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:52.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.991+0000 7ff2a8800640 1 --2- 172.21.15.39:0/482867850 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2a0153970 0x7ff2a0150400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:52.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.991+0000 7ff2a8800640 1 --2- 172.21.15.39:0/482867850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2a00cfc00 0x7ff2a014fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:52.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.991+0000 7ff2a8800640 1 --2- 172.21.15.39:0/482867850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2a0073e50 0x7ff2a0152b40 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:52.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.991+0000 7ff2a8800640 1 -- 172.21.15.39:0/482867850 >> 172.21.15.39:0/482867850 conn(0x7ff2a00d15f0 msgr2=0x7ff2a015f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:52.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.991+0000 7ff2a8800640 1 -- 172.21.15.39:0/482867850 shutdown_connections 2024-09-21T07:35:52.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:52.991+0000 7ff2a8800640 1 -- 172.21.15.39:0/482867850 wait complete. 2024-09-21T07:35:53.015 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:35:53.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.241+0000 7f1a52200640 1 -- 172.21.15.39:0/3085108926 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a30003c30 msgr2=0x7f1a30004080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:53.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.241+0000 7f1a52200640 1 --2- 172.21.15.39:0/3085108926 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a30003c30 0x7f1a30004080 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c00c4a0 tx=0x7f1a30005830 comp rx=0 tx=0).stop 2024-09-21T07:35:53.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.241+0000 7f1a52200640 1 -- 172.21.15.39:0/3085108926 shutdown_connections 2024-09-21T07:35:53.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.241+0000 7f1a52200640 1 --2- 172.21.15.39:0/3085108926 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a30003c30 0x7f1a30004080 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:53.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.241+0000 7f1a52200640 1 -- 172.21.15.39:0/3085108926 >> 172.21.15.39:0/3085108926 conn(0x7f1a4c06d7d0 msgr2=0x7f1a4c0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:53.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.241+0000 7f1a52200640 1 -- 172.21.15.39:0/3085108926 shutdown_connections 2024-09-21T07:35:53.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.241+0000 7f1a52200640 1 -- 172.21.15.39:0/3085108926 wait complete. 2024-09-21T07:35:53.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.244+0000 7f1a52200640 1 Processor -- start 2024-09-21T07:35:53.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.244+0000 7f1a52200640 1 -- start start 2024-09-21T07:35:53.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.245+0000 7f1a52200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a30003c30 0x7f1a4c15a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:53.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.245+0000 7f1a52200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1a4c158510 0x7f1a4c14bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:53.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.245+0000 7f1a52200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a4c1e9310 0x7f1a4c1e9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:53.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.245+0000 7f1a52200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4c0cd0e0 con 0x7f1a4c1e9310 2024-09-21T07:35:53.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.245+0000 7f1a52200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4c0ccf60 con 0x7f1a30003c30 2024-09-21T07:35:53.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.245+0000 7f1a52200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f1a4c0cd260 con 0x7f1a4c158510 2024-09-21T07:35:53.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.246+0000 7f1a4be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a4c1e9310 0x7f1a4c1e9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:53.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.246+0000 7f1a4be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a4c1e9310 0x7f1a4c1e9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45512/0 (socket says 172.21.15.39:45512) 2024-09-21T07:35:53.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.246+0000 7f1a4be00640 1 -- 172.21.15.39:0/3780011013 learned_addr learned my addr 172.21.15.39:0/3780011013 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:53.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.246+0000 7f1a4b400640 1 --2- 172.21.15.39:0/3780011013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a30003c30 0x7f1a4c15a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:53.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.246+0000 7f1a4aa00640 1 --2- 172.21.15.39:0/3780011013 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1a4c158510 0x7f1a4c14bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:53.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.246+0000 7f1a4be00640 1 -- 172.21.15.39:0/3780011013 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1a4c158510 msgr2=0x7f1a4c14bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:53.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.246+0000 7f1a4be00640 1 --2- 172.21.15.39:0/3780011013 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1a4c158510 0x7f1a4c14bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:53.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.246+0000 7f1a4be00640 1 -- 172.21.15.39:0/3780011013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a30003c30 msgr2=0x7f1a4c15a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:53.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.246+0000 7f1a4be00640 1 --2- 172.21.15.39:0/3780011013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a30003c30 0x7f1a4c15a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:53.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.246+0000 7f1a4be00640 1 -- 172.21.15.39:0/3780011013 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a4c1e9c60 con 0x7f1a4c1e9310 2024-09-21T07:35:53.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.246+0000 7f1a4be00640 1 --2- 172.21.15.39:0/3780011013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a4c1e9310 0x7f1a4c1e9720 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c0cd710 tx=0x7f1a44010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:53.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.247+0000 7f1a43400640 1 -- 172.21.15.39:0/3780011013 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a44017070 con 0x7f1a4c1e9310 2024-09-21T07:35:53.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.247+0000 7f1a43400640 1 -- 172.21.15.39:0/3780011013 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a44015070 con 0x7f1a4c1e9310 2024-09-21T07:35:53.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.247+0000 7f1a43400640 1 -- 172.21.15.39:0/3780011013 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4401c6a0 con 0x7f1a4c1e9310 2024-09-21T07:35:53.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.247+0000 7f1a52200640 1 -- 172.21.15.39:0/3780011013 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a4c158bf0 con 0x7f1a4c1e9310 2024-09-21T07:35:53.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.247+0000 7f1a52200640 1 -- 172.21.15.39:0/3780011013 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a4c1591b0 con 0x7f1a4c1e9310 2024-09-21T07:35:53.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.248+0000 7f1a52200640 1 -- 172.21.15.39:0/3780011013 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a4c06a5e0 con 0x7f1a4c1e9310 2024-09-21T07:35:53.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.249+0000 7f1a43400640 1 -- 172.21.15.39:0/3780011013 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1a4401c840 con 0x7f1a4c1e9310 2024-09-21T07:35:53.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.250+0000 7f1a43400640 1 --2- 172.21.15.39:0/3780011013 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1a2c078b90 0x7f1a2c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:53.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.250+0000 7f1a43400640 1 -- 172.21.15.39:0/3780011013 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6772+0+0 (secure 0 0 0) 0x7f1a4409bbe0 con 0x7f1a4c1e9310 2024-09-21T07:35:53.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.251+0000 7f1a4b400640 1 --2- 172.21.15.39:0/3780011013 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1a2c078b90 0x7f1a2c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:53.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.252+0000 7f1a4b400640 1 --2- 172.21.15.39:0/3780011013 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1a2c078b90 0x7f1a2c07b0a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c033000 tx=0x7f1a4c0b4bb0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:53.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.255+0000 7f1a43400640 1 -- 172.21.15.39:0/3780011013 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1a4c06a5e0 con 0x7f1a4c1e9310 2024-09-21T07:35:53.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.452+0000 7f1a52200640 1 -- 172.21.15.39:0/3780011013 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1a4c158980 con 0x7f1a4c1e9310 2024-09-21T07:35:53.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.454+0000 7f1a43400640 1 -- 172.21.15.39:0/3780011013 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22087 (secure 0 0 0) 0x7f1a44023020 con 0x7f1a4c1e9310 2024-09-21T07:35:53.456 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:35:53.456 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":78,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:35:50.455024+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:35:46.914721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-21T07:35:47.926559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:35:53.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.456+0000 7f1a52200640 1 -- 172.21.15.39:0/3780011013 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1a2c078b90 msgr2=0x7f1a2c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:53.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.456+0000 7f1a52200640 1 --2- 172.21.15.39:0/3780011013 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1a2c078b90 0x7f1a2c07b0a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c033000 tx=0x7f1a4c0b4bb0 comp rx=0 tx=0).stop 2024-09-21T07:35:53.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.456+0000 7f1a52200640 1 -- 172.21.15.39:0/3780011013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a4c1e9310 msgr2=0x7f1a4c1e9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:53.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.456+0000 7f1a52200640 1 --2- 172.21.15.39:0/3780011013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a4c1e9310 0x7f1a4c1e9720 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c0cd710 tx=0x7f1a44010f40 comp rx=0 tx=0).stop 2024-09-21T07:35:53.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.456+0000 7f1a52200640 1 -- 172.21.15.39:0/3780011013 shutdown_connections 2024-09-21T07:35:53.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.456+0000 7f1a52200640 1 --2- 172.21.15.39:0/3780011013 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1a2c078b90 0x7f1a2c07b0a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:53.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.456+0000 7f1a52200640 1 --2- 172.21.15.39:0/3780011013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a4c1e9310 0x7f1a4c1e9720 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:53.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.456+0000 7f1a52200640 1 --2- 172.21.15.39:0/3780011013 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1a4c158510 0x7f1a4c14bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:53.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.456+0000 7f1a52200640 1 --2- 172.21.15.39:0/3780011013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a30003c30 0x7f1a4c15a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:53.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.456+0000 7f1a52200640 1 -- 172.21.15.39:0/3780011013 >> 172.21.15.39:0/3780011013 conn(0x7f1a4c06d7d0 msgr2=0x7f1a4c06ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:53.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.456+0000 7f1a52200640 1 -- 172.21.15.39:0/3780011013 shutdown_connections 2024-09-21T07:35:53.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.456+0000 7f1a52200640 1 -- 172.21.15.39:0/3780011013 wait complete. 2024-09-21T07:35:53.480 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T07:35:53.480 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:35:53.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.705+0000 7f2b32c00640 1 -- 172.21.15.39:0/3764054520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b2c0cd3e0 msgr2=0x7f2b2c070180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:53.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.705+0000 7f2b32c00640 1 --2- 172.21.15.39:0/3764054520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b2c0cd3e0 0x7f2b2c070180 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c0ce0e0 tx=0x7f2b1802a560 comp rx=0 tx=0).stop 2024-09-21T07:35:53.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.706+0000 7f2b32c00640 1 -- 172.21.15.39:0/3764054520 shutdown_connections 2024-09-21T07:35:53.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.706+0000 7f2b32c00640 1 --2- 172.21.15.39:0/3764054520 >> v2:172.21.15.148:3300/0 conn(0x7f2b2c0706c0 0x7f2b2c072ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:53.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.706+0000 7f2b32c00640 1 --2- 172.21.15.39:0/3764054520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b2c0cd3e0 0x7f2b2c070180 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:53.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.706+0000 7f2b32c00640 1 -- 172.21.15.39:0/3764054520 >> 172.21.15.39:0/3764054520 conn(0x7f2b2c06d7d0 msgr2=0x7f2b2c0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:53.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.706+0000 7f2b32c00640 1 -- 172.21.15.39:0/3764054520 shutdown_connections 2024-09-21T07:35:53.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.706+0000 7f2b32c00640 1 -- 172.21.15.39:0/3764054520 wait complete. 2024-09-21T07:35:53.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.708+0000 7f2b32c00640 1 Processor -- start 2024-09-21T07:35:53.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.708+0000 7f2b32c00640 1 -- start start 2024-09-21T07:35:53.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.709+0000 7f2b32c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b2c0706c0 0x7f2b2c1e7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:53.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.709+0000 7f2b32c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2b2c0af780 0x7f2b2c15ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:53.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.710+0000 7f2b32c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b2c0cd3e0 0x7f2b2c1f5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:53.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.710+0000 7f2b32c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2c073af0 con 0x7f2b2c0706c0 2024-09-21T07:35:53.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.710+0000 7f2b32c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2c073970 con 0x7f2b2c0cd3e0 2024-09-21T07:35:53.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.710+0000 7f2b32c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f2b2c073c70 con 0x7f2b2c0af780 2024-09-21T07:35:53.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.710+0000 7f2b2be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b2c0706c0 0x7f2b2c1e7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:53.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.710+0000 7f2b2be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b2c0706c0 0x7f2b2c1e7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45528/0 (socket says 172.21.15.39:45528) 2024-09-21T07:35:53.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.710+0000 7f2b2be00640 1 -- 172.21.15.39:0/2313817380 learned_addr learned my addr 172.21.15.39:0/2313817380 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:53.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.710+0000 7f2b30c00640 1 --2- 172.21.15.39:0/2313817380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b2c0cd3e0 0x7f2b2c1f5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:53.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.710+0000 7f2b2b400640 1 --2- 172.21.15.39:0/2313817380 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2b2c0af780 0x7f2b2c15ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:53.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.710+0000 7f2b2be00640 1 -- 172.21.15.39:0/2313817380 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2b2c0af780 msgr2=0x7f2b2c15ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:53.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.710+0000 7f2b2be00640 1 --2- 172.21.15.39:0/2313817380 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2b2c0af780 0x7f2b2c15ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:53.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.710+0000 7f2b2be00640 1 -- 172.21.15.39:0/2313817380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b2c0cd3e0 msgr2=0x7f2b2c1f5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:53.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.710+0000 7f2b2be00640 1 --2- 172.21.15.39:0/2313817380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b2c0cd3e0 0x7f2b2c1f5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:53.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.710+0000 7f2b2be00640 1 -- 172.21.15.39:0/2313817380 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b2c09b550 con 0x7f2b2c0706c0 2024-09-21T07:35:53.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.711+0000 7f2b2be00640 1 --2- 172.21.15.39:0/2313817380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b2c0706c0 0x7f2b2c1e7930 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c00c3d0 tx=0x7f2b1c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:53.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.711+0000 7f2b23e00640 1 -- 172.21.15.39:0/2313817380 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c017070 con 0x7f2b2c0706c0 2024-09-21T07:35:53.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.711+0000 7f2b23e00640 1 -- 172.21.15.39:0/2313817380 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b1c0053c0 con 0x7f2b2c0706c0 2024-09-21T07:35:53.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.711+0000 7f2b23e00640 1 -- 172.21.15.39:0/2313817380 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c003850 con 0x7f2b2c0706c0 2024-09-21T07:35:53.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.711+0000 7f2b32c00640 1 -- 172.21.15.39:0/2313817380 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b2c09b830 con 0x7f2b2c0706c0 2024-09-21T07:35:53.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.711+0000 7f2b32c00640 1 -- 172.21.15.39:0/2313817380 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b2c09bdf0 con 0x7f2b2c0706c0 2024-09-21T07:35:53.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.712+0000 7f2b32c00640 1 -- 172.21.15.39:0/2313817380 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b2c072ff0 con 0x7f2b2c0706c0 2024-09-21T07:35:53.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.713+0000 7f2b23e00640 1 -- 172.21.15.39:0/2313817380 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2b1c01a070 con 0x7f2b2c0706c0 2024-09-21T07:35:53.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.714+0000 7f2b23e00640 1 --2- 172.21.15.39:0/2313817380 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2b0c078b90 0x7f2b0c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:53.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.715+0000 7f2b23e00640 1 -- 172.21.15.39:0/2313817380 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6772+0+0 (secure 0 0 0) 0x7f2b1c09a490 con 0x7f2b2c0706c0 2024-09-21T07:35:53.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.715+0000 7f2b2b400640 1 --2- 172.21.15.39:0/2313817380 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2b0c078b90 0x7f2b0c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:53.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.716+0000 7f2b2b400640 1 --2- 172.21.15.39:0/2313817380 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2b0c078b90 0x7f2b0c07b0a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2b18033000 tx=0x7f2b18004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:53.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.719+0000 7f2b23e00640 1 -- 172.21.15.39:0/2313817380 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2b1c061e40 con 0x7f2b2c0706c0 2024-09-21T07:35:53.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.910+0000 7f2b32c00640 1 -- 172.21.15.39:0/2313817380 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2b2c0cf110 con 0x7f2b0c078b90 2024-09-21T07:35:53.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.920+0000 7f2b23e00640 1 -- 172.21.15.39:0/2313817380 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+494623 (secure 0 0 0) 0x7f2b2c0cf110 con 0x7f2b0c078b90 2024-09-21T07:35:53.929 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:35:53.932 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:35:53.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.926+0000 7f2b32c00640 1 -- 172.21.15.39:0/2313817380 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2b0c078b90 msgr2=0x7f2b0c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:53.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.926+0000 7f2b32c00640 1 --2- 172.21.15.39:0/2313817380 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2b0c078b90 0x7f2b0c07b0a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2b18033000 tx=0x7f2b18004730 comp rx=0 tx=0).stop 2024-09-21T07:35:53.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.926+0000 7f2b32c00640 1 -- 172.21.15.39:0/2313817380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b2c0706c0 msgr2=0x7f2b2c1e7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:53.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.926+0000 7f2b32c00640 1 --2- 172.21.15.39:0/2313817380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b2c0706c0 0x7f2b2c1e7930 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c00c3d0 tx=0x7f2b1c00c890 comp rx=0 tx=0).stop 2024-09-21T07:35:53.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.926+0000 7f2b32c00640 1 -- 172.21.15.39:0/2313817380 shutdown_connections 2024-09-21T07:35:53.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.926+0000 7f2b32c00640 1 --2- 172.21.15.39:0/2313817380 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2b0c078b90 0x7f2b0c07b0a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:53.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.926+0000 7f2b32c00640 1 --2- 172.21.15.39:0/2313817380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b2c0cd3e0 0x7f2b2c1f5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:53.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.926+0000 7f2b32c00640 1 --2- 172.21.15.39:0/2313817380 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2b2c0af780 0x7f2b2c15ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:53.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.926+0000 7f2b32c00640 1 --2- 172.21.15.39:0/2313817380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b2c0706c0 0x7f2b2c1e7930 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:53.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.926+0000 7f2b32c00640 1 -- 172.21.15.39:0/2313817380 >> 172.21.15.39:0/2313817380 conn(0x7f2b2c06d7d0 msgr2=0x7f2b2c06ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:53.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.926+0000 7f2b32c00640 1 -- 172.21.15.39:0/2313817380 shutdown_connections 2024-09-21T07:35:53.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:53.926+0000 7f2b32c00640 1 -- 172.21.15.39:0/2313817380 wait complete. 2024-09-21T07:35:53.949 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":159,"stamp":"2024-09-21T07:35:53.576056+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":356,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258136,"kb_used_data":8584,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716584,"statfs":{"total":773094113280,"available":772829782016,"internally_reserved":0,"allocated":8790016,"data_stored":4655484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001823"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608939+0000","last_change":"2024-09-21T07:35:49.477604+0000","last_active":"2024-09-21T07:35:50.608939+0000","last_peered":"2024-09-21T07:35:50.608939+0000","last_clean":"2024-09-21T07:35:50.608939+0000","last_became_active":"2024-09-21T07:35:49.477253+0000","last_became_peered":"2024-09-21T07:35:49.477253+0000","last_unstale":"2024-09-21T07:35:50.608939+0000","last_undegraded":"2024-09-21T07:35:50.608939+0000","last_fullsized":"2024-09-21T07:35:50.608939+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:20.996757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.467198+0000","last_change":"2024-09-21T07:35:49.485146+0000","last_active":"2024-09-21T07:35:50.467198+0000","last_peered":"2024-09-21T07:35:50.467198+0000","last_clean":"2024-09-21T07:35:50.467198+0000","last_became_active":"2024-09-21T07:35:49.484826+0000","last_became_peered":"2024-09-21T07:35:49.484826+0000","last_unstale":"2024-09-21T07:35:50.467198+0000","last_undegraded":"2024-09-21T07:35:50.467198+0000","last_fullsized":"2024-09-21T07:35:50.467198+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:57.840668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608714+0000","last_change":"2024-09-21T07:35:49.472086+0000","last_active":"2024-09-21T07:35:50.608714+0000","last_peered":"2024-09-21T07:35:50.608714+0000","last_clean":"2024-09-21T07:35:50.608714+0000","last_became_active":"2024-09-21T07:35:49.471746+0000","last_became_peered":"2024-09-21T07:35:49.471746+0000","last_unstale":"2024-09-21T07:35:50.608714+0000","last_undegraded":"2024-09-21T07:35:50.608714+0000","last_fullsized":"2024-09-21T07:35:50.608714+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:52:19.381733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466009+0000","last_change":"2024-09-21T07:35:49.477135+0000","last_active":"2024-09-21T07:35:50.466009+0000","last_peered":"2024-09-21T07:35:50.466009+0000","last_clean":"2024-09-21T07:35:50.466009+0000","last_became_active":"2024-09-21T07:35:49.476707+0000","last_became_peered":"2024-09-21T07:35:49.476707+0000","last_unstale":"2024-09-21T07:35:50.466009+0000","last_undegraded":"2024-09-21T07:35:50.466009+0000","last_fullsized":"2024-09-21T07:35:50.466009+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:29.846901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.607826+0000","last_change":"2024-09-21T07:35:49.475067+0000","last_active":"2024-09-21T07:35:50.607826+0000","last_peered":"2024-09-21T07:35:50.607826+0000","last_clean":"2024-09-21T07:35:50.607826+0000","last_became_active":"2024-09-21T07:35:49.474751+0000","last_became_peered":"2024-09-21T07:35:49.474751+0000","last_unstale":"2024-09-21T07:35:50.607826+0000","last_undegraded":"2024-09-21T07:35:50.607826+0000","last_fullsized":"2024-09-21T07:35:50.607826+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:14.946214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609949+0000","last_change":"2024-09-21T07:35:49.478940+0000","last_active":"2024-09-21T07:35:50.609949+0000","last_peered":"2024-09-21T07:35:50.609949+0000","last_clean":"2024-09-21T07:35:50.609949+0000","last_became_active":"2024-09-21T07:35:49.478560+0000","last_became_peered":"2024-09-21T07:35:49.478560+0000","last_unstale":"2024-09-21T07:35:50.609949+0000","last_undegraded":"2024-09-21T07:35:50.609949+0000","last_fullsized":"2024-09-21T07:35:50.609949+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:38.592795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608211+0000","last_change":"2024-09-21T07:35:49.494342+0000","last_active":"2024-09-21T07:35:50.608211+0000","last_peered":"2024-09-21T07:35:50.608211+0000","last_clean":"2024-09-21T07:35:50.608211+0000","last_became_active":"2024-09-21T07:35:49.494034+0000","last_became_peered":"2024-09-21T07:35:49.494034+0000","last_unstale":"2024-09-21T07:35:50.608211+0000","last_undegraded":"2024-09-21T07:35:50.608211+0000","last_fullsized":"2024-09-21T07:35:50.608211+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:30.939322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.465736+0000","last_change":"2024-09-21T07:35:49.488554+0000","last_active":"2024-09-21T07:35:50.465736+0000","last_peered":"2024-09-21T07:35:50.465736+0000","last_clean":"2024-09-21T07:35:50.465736+0000","last_became_active":"2024-09-21T07:35:49.486889+0000","last_became_peered":"2024-09-21T07:35:49.486889+0000","last_unstale":"2024-09-21T07:35:50.465736+0000","last_undegraded":"2024-09-21T07:35:50.465736+0000","last_fullsized":"2024-09-21T07:35:50.465736+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:26.004814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608401+0000","last_change":"2024-09-21T07:35:49.462927+0000","last_active":"2024-09-21T07:35:50.608401+0000","last_peered":"2024-09-21T07:35:50.608401+0000","last_clean":"2024-09-21T07:35:50.608401+0000","last_became_active":"2024-09-21T07:35:49.462406+0000","last_became_peered":"2024-09-21T07:35:49.462406+0000","last_unstale":"2024-09-21T07:35:50.608401+0000","last_undegraded":"2024-09-21T07:35:50.608401+0000","last_fullsized":"2024-09-21T07:35:50.608401+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:06:21.675282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609530+0000","last_change":"2024-09-21T07:35:49.476877+0000","last_active":"2024-09-21T07:35:50.609530+0000","last_peered":"2024-09-21T07:35:50.609530+0000","last_clean":"2024-09-21T07:35:50.609530+0000","last_became_active":"2024-09-21T07:35:49.476569+0000","last_became_peered":"2024-09-21T07:35:49.476569+0000","last_unstale":"2024-09-21T07:35:50.609530+0000","last_undegraded":"2024-09-21T07:35:50.609530+0000","last_fullsized":"2024-09-21T07:35:50.609530+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:42:18.942020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609064+0000","last_change":"2024-09-21T07:35:49.456055+0000","last_active":"2024-09-21T07:35:50.609064+0000","last_peered":"2024-09-21T07:35:50.609064+0000","last_clean":"2024-09-21T07:35:50.609064+0000","last_became_active":"2024-09-21T07:35:49.455683+0000","last_became_peered":"2024-09-21T07:35:49.455683+0000","last_unstale":"2024-09-21T07:35:50.609064+0000","last_undegraded":"2024-09-21T07:35:50.609064+0000","last_fullsized":"2024-09-21T07:35:50.609064+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:00.733107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609353+0000","last_change":"2024-09-21T07:35:49.472334+0000","last_active":"2024-09-21T07:35:50.609353+0000","last_peered":"2024-09-21T07:35:50.609353+0000","last_clean":"2024-09-21T07:35:50.609353+0000","last_became_active":"2024-09-21T07:35:49.471427+0000","last_became_peered":"2024-09-21T07:35:49.471427+0000","last_unstale":"2024-09-21T07:35:50.609353+0000","last_undegraded":"2024-09-21T07:35:50.609353+0000","last_fullsized":"2024-09-21T07:35:50.609353+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:23.802292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.465944+0000","last_change":"2024-09-21T07:35:49.492926+0000","last_active":"2024-09-21T07:35:50.465944+0000","last_peered":"2024-09-21T07:35:50.465944+0000","last_clean":"2024-09-21T07:35:50.465944+0000","last_became_active":"2024-09-21T07:35:49.492498+0000","last_became_peered":"2024-09-21T07:35:49.492498+0000","last_unstale":"2024-09-21T07:35:50.465944+0000","last_undegraded":"2024-09-21T07:35:50.465944+0000","last_fullsized":"2024-09-21T07:35:50.465944+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:26.372963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.607685+0000","last_change":"2024-09-21T07:35:49.479630+0000","last_active":"2024-09-21T07:35:50.607685+0000","last_peered":"2024-09-21T07:35:50.607685+0000","last_clean":"2024-09-21T07:35:50.607685+0000","last_became_active":"2024-09-21T07:35:49.479297+0000","last_became_peered":"2024-09-21T07:35:49.479297+0000","last_unstale":"2024-09-21T07:35:50.607685+0000","last_undegraded":"2024-09-21T07:35:50.607685+0000","last_fullsized":"2024-09-21T07:35:50.607685+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:20.600428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466973+0000","last_change":"2024-09-21T07:35:49.491503+0000","last_active":"2024-09-21T07:35:50.466973+0000","last_peered":"2024-09-21T07:35:50.466973+0000","last_clean":"2024-09-21T07:35:50.466973+0000","last_became_active":"2024-09-21T07:35:49.491081+0000","last_became_peered":"2024-09-21T07:35:49.491081+0000","last_unstale":"2024-09-21T07:35:50.466973+0000","last_undegraded":"2024-09-21T07:35:50.466973+0000","last_fullsized":"2024-09-21T07:35:50.466973+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:50:14.299542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.607883+0000","last_change":"2024-09-21T07:35:49.477963+0000","last_active":"2024-09-21T07:35:50.607883+0000","last_peered":"2024-09-21T07:35:50.607883+0000","last_clean":"2024-09-21T07:35:50.607883+0000","last_became_active":"2024-09-21T07:35:49.476006+0000","last_became_peered":"2024-09-21T07:35:49.476006+0000","last_unstale":"2024-09-21T07:35:50.607883+0000","last_undegraded":"2024-09-21T07:35:50.607883+0000","last_fullsized":"2024-09-21T07:35:50.607883+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:51:44.922902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.611405+0000","last_change":"2024-09-21T07:35:49.492438+0000","last_active":"2024-09-21T07:35:50.611405+0000","last_peered":"2024-09-21T07:35:50.611405+0000","last_clean":"2024-09-21T07:35:50.611405+0000","last_became_active":"2024-09-21T07:35:49.489827+0000","last_became_peered":"2024-09-21T07:35:49.489827+0000","last_unstale":"2024-09-21T07:35:50.611405+0000","last_undegraded":"2024-09-21T07:35:50.611405+0000","last_fullsized":"2024-09-21T07:35:50.611405+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:10.214526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609393+0000","last_change":"2024-09-21T07:35:49.482017+0000","last_active":"2024-09-21T07:35:50.609393+0000","last_peered":"2024-09-21T07:35:50.609393+0000","last_clean":"2024-09-21T07:35:50.609393+0000","last_became_active":"2024-09-21T07:35:49.481736+0000","last_became_peered":"2024-09-21T07:35:49.481736+0000","last_unstale":"2024-09-21T07:35:50.609393+0000","last_undegraded":"2024-09-21T07:35:50.609393+0000","last_fullsized":"2024-09-21T07:35:50.609393+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:25:30.270321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609142+0000","last_change":"2024-09-21T07:35:49.477461+0000","last_active":"2024-09-21T07:35:50.609142+0000","last_peered":"2024-09-21T07:35:50.609142+0000","last_clean":"2024-09-21T07:35:50.609142+0000","last_became_active":"2024-09-21T07:35:49.477152+0000","last_became_peered":"2024-09-21T07:35:49.477152+0000","last_unstale":"2024-09-21T07:35:50.609142+0000","last_undegraded":"2024-09-21T07:35:50.609142+0000","last_fullsized":"2024-09-21T07:35:50.609142+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:20:11.777136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466526+0000","last_change":"2024-09-21T07:35:49.491436+0000","last_active":"2024-09-21T07:35:50.466526+0000","last_peered":"2024-09-21T07:35:50.466526+0000","last_clean":"2024-09-21T07:35:50.466526+0000","last_became_active":"2024-09-21T07:35:49.491048+0000","last_became_peered":"2024-09-21T07:35:49.491048+0000","last_unstale":"2024-09-21T07:35:50.466526+0000","last_undegraded":"2024-09-21T07:35:50.466526+0000","last_fullsized":"2024-09-21T07:35:50.466526+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:47.032420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608134+0000","last_change":"2024-09-21T07:35:49.468432+0000","last_active":"2024-09-21T07:35:50.608134+0000","last_peered":"2024-09-21T07:35:50.608134+0000","last_clean":"2024-09-21T07:35:50.608134+0000","last_became_active":"2024-09-21T07:35:49.467983+0000","last_became_peered":"2024-09-21T07:35:49.467983+0000","last_unstale":"2024-09-21T07:35:50.608134+0000","last_undegraded":"2024-09-21T07:35:50.608134+0000","last_fullsized":"2024-09-21T07:35:50.608134+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:30:11.845940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.607688+0000","last_change":"2024-09-21T07:35:49.480387+0000","last_active":"2024-09-21T07:35:50.607688+0000","last_peered":"2024-09-21T07:35:50.607688+0000","last_clean":"2024-09-21T07:35:50.607688+0000","last_became_active":"2024-09-21T07:35:49.478378+0000","last_became_peered":"2024-09-21T07:35:49.478378+0000","last_unstale":"2024-09-21T07:35:50.607688+0000","last_undegraded":"2024-09-21T07:35:50.607688+0000","last_fullsized":"2024-09-21T07:35:50.607688+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:46.129473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.465810+0000","last_change":"2024-09-21T07:35:49.476827+0000","last_active":"2024-09-21T07:35:50.465810+0000","last_peered":"2024-09-21T07:35:50.465810+0000","last_clean":"2024-09-21T07:35:50.465810+0000","last_became_active":"2024-09-21T07:35:49.476486+0000","last_became_peered":"2024-09-21T07:35:49.476486+0000","last_unstale":"2024-09-21T07:35:50.465810+0000","last_undegraded":"2024-09-21T07:35:50.465810+0000","last_fullsized":"2024-09-21T07:35:50.465810+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:29.495534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.607784+0000","last_change":"2024-09-21T07:35:49.473330+0000","last_active":"2024-09-21T07:35:50.607784+0000","last_peered":"2024-09-21T07:35:50.607784+0000","last_clean":"2024-09-21T07:35:50.607784+0000","last_became_active":"2024-09-21T07:35:49.473022+0000","last_became_peered":"2024-09-21T07:35:49.473022+0000","last_unstale":"2024-09-21T07:35:50.607784+0000","last_undegraded":"2024-09-21T07:35:50.607784+0000","last_fullsized":"2024-09-21T07:35:50.607784+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:20:09.706907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.467549+0000","last_change":"2024-09-21T07:35:49.475917+0000","last_active":"2024-09-21T07:35:50.467549+0000","last_peered":"2024-09-21T07:35:50.467549+0000","last_clean":"2024-09-21T07:35:50.467549+0000","last_became_active":"2024-09-21T07:35:49.475559+0000","last_became_peered":"2024-09-21T07:35:49.475559+0000","last_unstale":"2024-09-21T07:35:50.467549+0000","last_undegraded":"2024-09-21T07:35:50.467549+0000","last_fullsized":"2024-09-21T07:35:50.467549+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:57:06.538812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608685+0000","last_change":"2024-09-21T07:35:49.478827+0000","last_active":"2024-09-21T07:35:50.608685+0000","last_peered":"2024-09-21T07:35:50.608685+0000","last_clean":"2024-09-21T07:35:50.608685+0000","last_became_active":"2024-09-21T07:35:49.477964+0000","last_became_peered":"2024-09-21T07:35:49.477964+0000","last_unstale":"2024-09-21T07:35:50.608685+0000","last_undegraded":"2024-09-21T07:35:50.608685+0000","last_fullsized":"2024-09-21T07:35:50.608685+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:19.502430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.610143+0000","last_change":"2024-09-21T07:35:49.474821+0000","last_active":"2024-09-21T07:35:50.610143+0000","last_peered":"2024-09-21T07:35:50.610143+0000","last_clean":"2024-09-21T07:35:50.610143+0000","last_became_active":"2024-09-21T07:35:49.474546+0000","last_became_peered":"2024-09-21T07:35:49.474546+0000","last_unstale":"2024-09-21T07:35:50.610143+0000","last_undegraded":"2024-09-21T07:35:50.610143+0000","last_fullsized":"2024-09-21T07:35:50.610143+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:56:26.806827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466552+0000","last_change":"2024-09-21T07:35:49.476604+0000","last_active":"2024-09-21T07:35:50.466552+0000","last_peered":"2024-09-21T07:35:50.466552+0000","last_clean":"2024-09-21T07:35:50.466552+0000","last_became_active":"2024-09-21T07:35:49.476270+0000","last_became_peered":"2024-09-21T07:35:49.476270+0000","last_unstale":"2024-09-21T07:35:50.466552+0000","last_undegraded":"2024-09-21T07:35:50.466552+0000","last_fullsized":"2024-09-21T07:35:50.466552+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:25.055048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609192+0000","last_change":"2024-09-21T07:35:49.482117+0000","last_active":"2024-09-21T07:35:50.609192+0000","last_peered":"2024-09-21T07:35:50.609192+0000","last_clean":"2024-09-21T07:35:50.609192+0000","last_became_active":"2024-09-21T07:35:49.479799+0000","last_became_peered":"2024-09-21T07:35:49.479799+0000","last_unstale":"2024-09-21T07:35:50.609192+0000","last_undegraded":"2024-09-21T07:35:50.609192+0000","last_fullsized":"2024-09-21T07:35:50.609192+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:50:24.393154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608764+0000","last_change":"2024-09-21T07:35:49.463449+0000","last_active":"2024-09-21T07:35:50.608764+0000","last_peered":"2024-09-21T07:35:50.608764+0000","last_clean":"2024-09-21T07:35:50.608764+0000","last_became_active":"2024-09-21T07:35:49.462439+0000","last_became_peered":"2024-09-21T07:35:49.462439+0000","last_unstale":"2024-09-21T07:35:50.608764+0000","last_undegraded":"2024-09-21T07:35:50.608764+0000","last_fullsized":"2024-09-21T07:35:50.608764+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:08:07.440136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.465782+0000","last_change":"2024-09-21T07:35:49.477637+0000","last_active":"2024-09-21T07:35:50.465782+0000","last_peered":"2024-09-21T07:35:50.465782+0000","last_clean":"2024-09-21T07:35:50.465782+0000","last_became_active":"2024-09-21T07:35:49.477370+0000","last_became_peered":"2024-09-21T07:35:49.477370+0000","last_unstale":"2024-09-21T07:35:50.465782+0000","last_undegraded":"2024-09-21T07:35:50.465782+0000","last_fullsized":"2024-09-21T07:35:50.465782+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:20.497297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608106+0000","last_change":"2024-09-21T07:35:49.475935+0000","last_active":"2024-09-21T07:35:50.608106+0000","last_peered":"2024-09-21T07:35:50.608106+0000","last_clean":"2024-09-21T07:35:50.608106+0000","last_became_active":"2024-09-21T07:35:49.475613+0000","last_became_peered":"2024-09-21T07:35:49.475613+0000","last_unstale":"2024-09-21T07:35:50.608106+0000","last_undegraded":"2024-09-21T07:35:50.608106+0000","last_fullsized":"2024-09-21T07:35:50.608106+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:24:26.502711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609801+0000","last_change":"2024-09-21T07:35:49.494330+0000","last_active":"2024-09-21T07:35:50.609801+0000","last_peered":"2024-09-21T07:35:50.609801+0000","last_clean":"2024-09-21T07:35:50.609801+0000","last_became_active":"2024-09-21T07:35:49.494019+0000","last_became_peered":"2024-09-21T07:35:49.494019+0000","last_unstale":"2024-09-21T07:35:50.609801+0000","last_undegraded":"2024-09-21T07:35:50.609801+0000","last_fullsized":"2024-09-21T07:35:50.609801+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:18:47.217069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466597+0000","last_change":"2024-09-21T07:35:48.469480+0000","last_active":"2024-09-21T07:35:50.466597+0000","last_peered":"2024-09-21T07:35:50.466597+0000","last_clean":"2024-09-21T07:35:50.466597+0000","last_became_active":"2024-09-21T07:35:48.468500+0000","last_became_peered":"2024-09-21T07:35:48.468500+0000","last_unstale":"2024-09-21T07:35:50.466597+0000","last_undegraded":"2024-09-21T07:35:50.466597+0000","last_fullsized":"2024-09-21T07:35:50.466597+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:25.184983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609270+0000","last_change":"2024-09-21T07:35:27.879605+0000","last_active":"2024-09-21T07:35:50.609270+0000","last_peered":"2024-09-21T07:35:50.609270+0000","last_clean":"2024-09-21T07:35:50.609270+0000","last_became_active":"2024-09-21T07:35:27.879274+0000","last_became_peered":"2024-09-21T07:35:27.879274+0000","last_unstale":"2024-09-21T07:35:50.609270+0000","last_undegraded":"2024-09-21T07:35:50.609270+0000","last_fullsized":"2024-09-21T07:35:50.609270+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:24.241636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608495+0000","last_change":"2024-09-21T07:35:29.491336+0000","last_active":"2024-09-21T07:35:50.608495+0000","last_peered":"2024-09-21T07:35:50.608495+0000","last_clean":"2024-09-21T07:35:50.608495+0000","last_became_active":"2024-09-21T07:35:26.866484+0000","last_became_peered":"2024-09-21T07:35:26.866484+0000","last_unstale":"2024-09-21T07:35:50.608495+0000","last_undegraded":"2024-09-21T07:35:50.608495+0000","last_fullsized":"2024-09-21T07:35:50.608495+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:10.449665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076010100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":219,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608202+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:35:50.608202+0000","last_peered":"2024-09-21T07:35:50.608202+0000","last_clean":"2024-09-21T07:35:50.608202+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:35:50.608202+0000","last_undegraded":"2024-09-21T07:35:50.608202+0000","last_fullsized":"2024-09-21T07:35:50.608202+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":96,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.467873+0000","last_change":"2024-09-21T07:34:24.539248+0000","last_active":"2024-09-21T07:35:50.467873+0000","last_peered":"2024-09-21T07:35:50.467873+0000","last_clean":"2024-09-21T07:35:50.467873+0000","last_became_active":"2024-09-21T07:34:23.273465+0000","last_became_peered":"2024-09-21T07:34:23.273465+0000","last_unstale":"2024-09-21T07:35:50.467873+0000","last_undegraded":"2024-09-21T07:35:50.467873+0000","last_fullsized":"2024-09-21T07:35:50.467873+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:55.517452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060627700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466091+0000","last_change":"2024-09-21T07:35:49.482924+0000","last_active":"2024-09-21T07:35:50.466091+0000","last_peered":"2024-09-21T07:35:50.466091+0000","last_clean":"2024-09-21T07:35:50.466091+0000","last_became_active":"2024-09-21T07:35:49.478417+0000","last_became_peered":"2024-09-21T07:35:49.478417+0000","last_unstale":"2024-09-21T07:35:50.466091+0000","last_undegraded":"2024-09-21T07:35:50.466091+0000","last_fullsized":"2024-09-21T07:35:50.466091+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:04.130257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.465460+0000","last_change":"2024-09-21T07:35:48.477741+0000","last_active":"2024-09-21T07:35:50.465460+0000","last_peered":"2024-09-21T07:35:50.465460+0000","last_clean":"2024-09-21T07:35:50.465460+0000","last_became_active":"2024-09-21T07:35:48.477356+0000","last_became_peered":"2024-09-21T07:35:48.477356+0000","last_unstale":"2024-09-21T07:35:50.465460+0000","last_undegraded":"2024-09-21T07:35:50.465460+0000","last_fullsized":"2024-09-21T07:35:50.465460+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:20:00.655949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.607759+0000","last_change":"2024-09-21T07:35:27.864492+0000","last_active":"2024-09-21T07:35:50.607759+0000","last_peered":"2024-09-21T07:35:50.607759+0000","last_clean":"2024-09-21T07:35:50.607759+0000","last_became_active":"2024-09-21T07:35:27.864176+0000","last_became_peered":"2024-09-21T07:35:27.864176+0000","last_unstale":"2024-09-21T07:35:50.607759+0000","last_undegraded":"2024-09-21T07:35:50.607759+0000","last_fullsized":"2024-09-21T07:35:50.607759+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:05.153467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609998+0000","last_change":"2024-09-21T07:35:29.493678+0000","last_active":"2024-09-21T07:35:50.609998+0000","last_peered":"2024-09-21T07:35:50.609998+0000","last_clean":"2024-09-21T07:35:50.609998+0000","last_became_active":"2024-09-21T07:35:26.870749+0000","last_became_peered":"2024-09-21T07:35:26.870749+0000","last_unstale":"2024-09-21T07:35:50.609998+0000","last_undegraded":"2024-09-21T07:35:50.609998+0000","last_fullsized":"2024-09-21T07:35:50.609998+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:55.367877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010159570000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":90,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608887+0000","last_change":"2024-09-21T07:34:24.597274+0000","last_active":"2024-09-21T07:35:50.608887+0000","last_peered":"2024-09-21T07:35:50.608887+0000","last_clean":"2024-09-21T07:35:50.608887+0000","last_became_active":"2024-09-21T07:34:23.279560+0000","last_became_peered":"2024-09-21T07:34:23.279560+0000","last_unstale":"2024-09-21T07:35:50.608887+0000","last_undegraded":"2024-09-21T07:35:50.608887+0000","last_fullsized":"2024-09-21T07:35:50.608887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:15.845265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059699799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.610255+0000","last_change":"2024-09-21T07:35:49.483638+0000","last_active":"2024-09-21T07:35:50.610255+0000","last_peered":"2024-09-21T07:35:50.610255+0000","last_clean":"2024-09-21T07:35:50.610255+0000","last_became_active":"2024-09-21T07:35:49.483358+0000","last_became_peered":"2024-09-21T07:35:49.483358+0000","last_unstale":"2024-09-21T07:35:50.610255+0000","last_undegraded":"2024-09-21T07:35:50.610255+0000","last_fullsized":"2024-09-21T07:35:50.610255+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:34:17.263329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.607267+0000","last_change":"2024-09-21T07:35:48.472242+0000","last_active":"2024-09-21T07:35:50.607267+0000","last_peered":"2024-09-21T07:35:50.607267+0000","last_clean":"2024-09-21T07:35:50.607267+0000","last_became_active":"2024-09-21T07:35:48.471808+0000","last_became_peered":"2024-09-21T07:35:48.471808+0000","last_unstale":"2024-09-21T07:35:50.607267+0000","last_undegraded":"2024-09-21T07:35:50.607267+0000","last_fullsized":"2024-09-21T07:35:50.607267+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:10:29.731170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":36,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608130+0000","last_change":"2024-09-21T07:35:27.874387+0000","last_active":"2024-09-21T07:35:50.608130+0000","last_peered":"2024-09-21T07:35:50.608130+0000","last_clean":"2024-09-21T07:35:50.608130+0000","last_became_active":"2024-09-21T07:35:27.873277+0000","last_became_peered":"2024-09-21T07:35:27.873277+0000","last_unstale":"2024-09-21T07:35:50.608130+0000","last_undegraded":"2024-09-21T07:35:50.608130+0000","last_fullsized":"2024-09-21T07:35:50.608130+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:04.581463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609198+0000","last_change":"2024-09-21T07:35:29.387094+0000","last_active":"2024-09-21T07:35:50.609198+0000","last_peered":"2024-09-21T07:35:50.609198+0000","last_clean":"2024-09-21T07:35:50.609198+0000","last_became_active":"2024-09-21T07:35:26.874757+0000","last_became_peered":"2024-09-21T07:35:26.874757+0000","last_unstale":"2024-09-21T07:35:50.609198+0000","last_undegraded":"2024-09-21T07:35:50.609198+0000","last_fullsized":"2024-09-21T07:35:50.609198+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:33.542750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061065700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":94,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609515+0000","last_change":"2024-09-21T07:34:24.259906+0000","last_active":"2024-09-21T07:35:50.609515+0000","last_peered":"2024-09-21T07:35:50.609515+0000","last_clean":"2024-09-21T07:35:50.609515+0000","last_became_active":"2024-09-21T07:34:23.635282+0000","last_became_peered":"2024-09-21T07:34:23.635282+0000","last_unstale":"2024-09-21T07:35:50.609515+0000","last_undegraded":"2024-09-21T07:35:50.609515+0000","last_fullsized":"2024-09-21T07:35:50.609515+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:00.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001527664,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608637+0000","last_change":"2024-09-21T07:35:49.478853+0000","last_active":"2024-09-21T07:35:50.608637+0000","last_peered":"2024-09-21T07:35:50.608637+0000","last_clean":"2024-09-21T07:35:50.608637+0000","last_became_active":"2024-09-21T07:35:49.478490+0000","last_became_peered":"2024-09-21T07:35:49.478490+0000","last_unstale":"2024-09-21T07:35:50.608637+0000","last_undegraded":"2024-09-21T07:35:50.608637+0000","last_fullsized":"2024-09-21T07:35:50.608637+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:44.711617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"77'2","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.465912+0000","last_change":"2024-09-21T07:35:48.477646+0000","last_active":"2024-09-21T07:35:50.465912+0000","last_peered":"2024-09-21T07:35:50.465912+0000","last_clean":"2024-09-21T07:35:50.465912+0000","last_became_active":"2024-09-21T07:35:48.477112+0000","last_became_peered":"2024-09-21T07:35:48.477112+0000","last_unstale":"2024-09-21T07:35:50.465912+0000","last_undegraded":"2024-09-21T07:35:50.465912+0000","last_fullsized":"2024-09-21T07:35:50.465912+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:05.943925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":36,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.607906+0000","last_change":"2024-09-21T07:35:27.873645+0000","last_active":"2024-09-21T07:35:50.607906+0000","last_peered":"2024-09-21T07:35:50.607906+0000","last_clean":"2024-09-21T07:35:50.607906+0000","last_became_active":"2024-09-21T07:35:27.873245+0000","last_became_peered":"2024-09-21T07:35:27.873245+0000","last_unstale":"2024-09-21T07:35:50.607906+0000","last_undegraded":"2024-09-21T07:35:50.607906+0000","last_fullsized":"2024-09-21T07:35:50.607906+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:55.090953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":43,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.610950+0000","last_change":"2024-09-21T07:35:28.861744+0000","last_active":"2024-09-21T07:35:50.610950+0000","last_peered":"2024-09-21T07:35:50.610950+0000","last_clean":"2024-09-21T07:35:50.610950+0000","last_became_active":"2024-09-21T07:35:26.876462+0000","last_became_peered":"2024-09-21T07:35:26.876462+0000","last_unstale":"2024-09-21T07:35:50.610950+0000","last_undegraded":"2024-09-21T07:35:50.610950+0000","last_fullsized":"2024-09-21T07:35:50.610950+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:48.632568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012020920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":92,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608550+0000","last_change":"2024-09-21T07:34:25.085454+0000","last_active":"2024-09-21T07:35:50.608550+0000","last_peered":"2024-09-21T07:35:50.608550+0000","last_clean":"2024-09-21T07:35:50.608550+0000","last_became_active":"2024-09-21T07:34:23.279205+0000","last_became_peered":"2024-09-21T07:34:23.279205+0000","last_unstale":"2024-09-21T07:35:50.608550+0000","last_undegraded":"2024-09-21T07:35:50.608550+0000","last_fullsized":"2024-09-21T07:35:50.608550+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:54.231666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095320599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466748+0000","last_change":"2024-09-21T07:35:49.485445+0000","last_active":"2024-09-21T07:35:50.466748+0000","last_peered":"2024-09-21T07:35:50.466748+0000","last_clean":"2024-09-21T07:35:50.466748+0000","last_became_active":"2024-09-21T07:35:49.484912+0000","last_became_peered":"2024-09-21T07:35:49.484912+0000","last_unstale":"2024-09-21T07:35:50.466748+0000","last_undegraded":"2024-09-21T07:35:50.466748+0000","last_fullsized":"2024-09-21T07:35:50.466748+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:26:24.284853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.607993+0000","last_change":"2024-09-21T07:35:48.464790+0000","last_active":"2024-09-21T07:35:50.607993+0000","last_peered":"2024-09-21T07:35:50.607993+0000","last_clean":"2024-09-21T07:35:50.607993+0000","last_became_active":"2024-09-21T07:35:48.464398+0000","last_became_peered":"2024-09-21T07:35:48.464398+0000","last_unstale":"2024-09-21T07:35:50.607993+0000","last_undegraded":"2024-09-21T07:35:50.607993+0000","last_fullsized":"2024-09-21T07:35:50.607993+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:28:32.537073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608673+0000","last_change":"2024-09-21T07:35:27.861373+0000","last_active":"2024-09-21T07:35:50.608673+0000","last_peered":"2024-09-21T07:35:50.608673+0000","last_clean":"2024-09-21T07:35:50.608673+0000","last_became_active":"2024-09-21T07:35:27.861016+0000","last_became_peered":"2024-09-21T07:35:27.861016+0000","last_unstale":"2024-09-21T07:35:50.608673+0000","last_undegraded":"2024-09-21T07:35:50.608673+0000","last_fullsized":"2024-09-21T07:35:50.608673+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:56.043763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":43,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466625+0000","last_change":"2024-09-21T07:35:29.053677+0000","last_active":"2024-09-21T07:35:50.466625+0000","last_peered":"2024-09-21T07:35:50.466625+0000","last_clean":"2024-09-21T07:35:50.466625+0000","last_became_active":"2024-09-21T07:35:26.875496+0000","last_became_peered":"2024-09-21T07:35:26.875496+0000","last_unstale":"2024-09-21T07:35:50.466625+0000","last_undegraded":"2024-09-21T07:35:50.466625+0000","last_fullsized":"2024-09-21T07:35:50.466625+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:04.850784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065859000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":94,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466113+0000","last_change":"2024-09-21T07:34:25.028521+0000","last_active":"2024-09-21T07:35:50.466113+0000","last_peered":"2024-09-21T07:35:50.466113+0000","last_clean":"2024-09-21T07:35:50.466113+0000","last_became_active":"2024-09-21T07:34:23.635467+0000","last_became_peered":"2024-09-21T07:34:23.635467+0000","last_unstale":"2024-09-21T07:35:50.466113+0000","last_undegraded":"2024-09-21T07:35:50.466113+0000","last_fullsized":"2024-09-21T07:35:50.466113+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:33.729123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000660558,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609485+0000","last_change":"2024-09-21T07:35:49.473102+0000","last_active":"2024-09-21T07:35:50.609485+0000","last_peered":"2024-09-21T07:35:50.609485+0000","last_clean":"2024-09-21T07:35:50.609485+0000","last_became_active":"2024-09-21T07:35:49.471147+0000","last_became_peered":"2024-09-21T07:35:49.471147+0000","last_unstale":"2024-09-21T07:35:50.609485+0000","last_undegraded":"2024-09-21T07:35:50.609485+0000","last_fullsized":"2024-09-21T07:35:50.609485+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:02.599930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"77'1","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609337+0000","last_change":"2024-09-21T07:35:48.472515+0000","last_active":"2024-09-21T07:35:50.609337+0000","last_peered":"2024-09-21T07:35:50.609337+0000","last_clean":"2024-09-21T07:35:50.609337+0000","last_became_active":"2024-09-21T07:35:48.472176+0000","last_became_peered":"2024-09-21T07:35:48.472176+0000","last_unstale":"2024-09-21T07:35:50.609337+0000","last_undegraded":"2024-09-21T07:35:50.609337+0000","last_fullsized":"2024-09-21T07:35:50.609337+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:19:41.363336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.610700+0000","last_change":"2024-09-21T07:35:27.871201+0000","last_active":"2024-09-21T07:35:50.610700+0000","last_peered":"2024-09-21T07:35:50.610700+0000","last_clean":"2024-09-21T07:35:50.610700+0000","last_became_active":"2024-09-21T07:35:27.870816+0000","last_became_peered":"2024-09-21T07:35:27.870816+0000","last_unstale":"2024-09-21T07:35:50.610700+0000","last_undegraded":"2024-09-21T07:35:50.610700+0000","last_fullsized":"2024-09-21T07:35:50.610700+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:04.787450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609553+0000","last_change":"2024-09-21T07:35:29.491874+0000","last_active":"2024-09-21T07:35:50.609553+0000","last_peered":"2024-09-21T07:35:50.609553+0000","last_clean":"2024-09-21T07:35:50.609553+0000","last_became_active":"2024-09-21T07:35:26.866381+0000","last_became_peered":"2024-09-21T07:35:26.866381+0000","last_unstale":"2024-09-21T07:35:50.609553+0000","last_undegraded":"2024-09-21T07:35:50.609553+0000","last_fullsized":"2024-09-21T07:35:50.609553+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:36.404647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073235399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":94,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609873+0000","last_change":"2024-09-21T07:34:24.260046+0000","last_active":"2024-09-21T07:35:50.609873+0000","last_peered":"2024-09-21T07:35:50.609873+0000","last_clean":"2024-09-21T07:35:50.609873+0000","last_became_active":"2024-09-21T07:34:23.273431+0000","last_became_peered":"2024-09-21T07:34:23.273431+0000","last_unstale":"2024-09-21T07:35:50.609873+0000","last_undegraded":"2024-09-21T07:35:50.609873+0000","last_fullsized":"2024-09-21T07:35:50.609873+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:18.876057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019803310000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.610552+0000","last_change":"2024-09-21T07:35:49.462274+0000","last_active":"2024-09-21T07:35:50.610552+0000","last_peered":"2024-09-21T07:35:50.610552+0000","last_clean":"2024-09-21T07:35:50.610552+0000","last_became_active":"2024-09-21T07:35:49.461939+0000","last_became_peered":"2024-09-21T07:35:49.461939+0000","last_unstale":"2024-09-21T07:35:50.610552+0000","last_undegraded":"2024-09-21T07:35:50.610552+0000","last_fullsized":"2024-09-21T07:35:50.610552+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:05.599486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"77'1","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.468176+0000","last_change":"2024-09-21T07:35:48.459406+0000","last_active":"2024-09-21T07:35:50.468176+0000","last_peered":"2024-09-21T07:35:50.468176+0000","last_clean":"2024-09-21T07:35:50.468176+0000","last_became_active":"2024-09-21T07:35:48.458952+0000","last_became_peered":"2024-09-21T07:35:48.458952+0000","last_unstale":"2024-09-21T07:35:50.468176+0000","last_undegraded":"2024-09-21T07:35:50.468176+0000","last_fullsized":"2024-09-21T07:35:50.468176+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:00:08.139371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.610813+0000","last_change":"2024-09-21T07:35:27.871736+0000","last_active":"2024-09-21T07:35:50.610813+0000","last_peered":"2024-09-21T07:35:50.610813+0000","last_clean":"2024-09-21T07:35:50.610813+0000","last_became_active":"2024-09-21T07:35:27.871232+0000","last_became_peered":"2024-09-21T07:35:27.871232+0000","last_unstale":"2024-09-21T07:35:50.610813+0000","last_undegraded":"2024-09-21T07:35:50.610813+0000","last_fullsized":"2024-09-21T07:35:50.610813+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:04.419301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608082+0000","last_change":"2024-09-21T07:35:29.491246+0000","last_active":"2024-09-21T07:35:50.608082+0000","last_peered":"2024-09-21T07:35:50.608082+0000","last_clean":"2024-09-21T07:35:50.608082+0000","last_became_active":"2024-09-21T07:35:26.866783+0000","last_became_peered":"2024-09-21T07:35:26.866783+0000","last_unstale":"2024-09-21T07:35:50.608082+0000","last_undegraded":"2024-09-21T07:35:50.608082+0000","last_fullsized":"2024-09-21T07:35:50.608082+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:28.881914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098474099999999992,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":94,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466717+0000","last_change":"2024-09-21T07:34:25.028640+0000","last_active":"2024-09-21T07:35:50.466717+0000","last_peered":"2024-09-21T07:35:50.466717+0000","last_clean":"2024-09-21T07:35:50.466717+0000","last_became_active":"2024-09-21T07:34:23.260950+0000","last_became_peered":"2024-09-21T07:34:23.260950+0000","last_unstale":"2024-09-21T07:35:50.466717+0000","last_undegraded":"2024-09-21T07:35:50.466717+0000","last_fullsized":"2024-09-21T07:35:50.466717+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:23:42.121398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055135099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.467998+0000","last_change":"2024-09-21T07:35:49.475001+0000","last_active":"2024-09-21T07:35:50.467998+0000","last_peered":"2024-09-21T07:35:50.467998+0000","last_clean":"2024-09-21T07:35:50.467998+0000","last_became_active":"2024-09-21T07:35:49.474511+0000","last_became_peered":"2024-09-21T07:35:49.474511+0000","last_unstale":"2024-09-21T07:35:50.467998+0000","last_undegraded":"2024-09-21T07:35:50.467998+0000","last_fullsized":"2024-09-21T07:35:50.467998+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:26.004814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"77'2","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609915+0000","last_change":"2024-09-21T07:35:48.463983+0000","last_active":"2024-09-21T07:35:50.609915+0000","last_peered":"2024-09-21T07:35:50.609915+0000","last_clean":"2024-09-21T07:35:50.609915+0000","last_became_active":"2024-09-21T07:35:48.463661+0000","last_became_peered":"2024-09-21T07:35:48.463661+0000","last_unstale":"2024-09-21T07:35:50.609915+0000","last_undegraded":"2024-09-21T07:35:50.609915+0000","last_fullsized":"2024-09-21T07:35:50.609915+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:48.919917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.468430+0000","last_change":"2024-09-21T07:35:27.872027+0000","last_active":"2024-09-21T07:35:50.468430+0000","last_peered":"2024-09-21T07:35:50.468430+0000","last_clean":"2024-09-21T07:35:50.468430+0000","last_became_active":"2024-09-21T07:35:27.870426+0000","last_became_peered":"2024-09-21T07:35:27.870426+0000","last_unstale":"2024-09-21T07:35:50.468430+0000","last_undegraded":"2024-09-21T07:35:50.468430+0000","last_fullsized":"2024-09-21T07:35:50.468430+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.607945+0000","last_change":"2024-09-21T07:35:29.491008+0000","last_active":"2024-09-21T07:35:50.607945+0000","last_peered":"2024-09-21T07:35:50.607945+0000","last_clean":"2024-09-21T07:35:50.607945+0000","last_became_active":"2024-09-21T07:35:26.866750+0000","last_became_peered":"2024-09-21T07:35:26.866750+0000","last_unstale":"2024-09-21T07:35:50.607945+0000","last_undegraded":"2024-09-21T07:35:50.607945+0000","last_fullsized":"2024-09-21T07:35:50.607945+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:25.490406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083529700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":94,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609716+0000","last_change":"2024-09-21T07:34:24.260665+0000","last_active":"2024-09-21T07:35:50.609716+0000","last_peered":"2024-09-21T07:35:50.609716+0000","last_clean":"2024-09-21T07:35:50.609716+0000","last_became_active":"2024-09-21T07:34:23.635909+0000","last_became_peered":"2024-09-21T07:34:23.635909+0000","last_unstale":"2024-09-21T07:35:50.609716+0000","last_undegraded":"2024-09-21T07:35:50.609716+0000","last_fullsized":"2024-09-21T07:35:50.609716+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:44.536885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466269+0000","last_change":"2024-09-21T07:35:49.480549+0000","last_active":"2024-09-21T07:35:50.466269+0000","last_peered":"2024-09-21T07:35:50.466269+0000","last_clean":"2024-09-21T07:35:50.466269+0000","last_became_active":"2024-09-21T07:35:49.480128+0000","last_became_peered":"2024-09-21T07:35:49.480128+0000","last_unstale":"2024-09-21T07:35:50.466269+0000","last_undegraded":"2024-09-21T07:35:50.466269+0000","last_fullsized":"2024-09-21T07:35:50.466269+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:38.532303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.465345+0000","last_change":"2024-09-21T07:35:48.466284+0000","last_active":"2024-09-21T07:35:50.465345+0000","last_peered":"2024-09-21T07:35:50.465345+0000","last_clean":"2024-09-21T07:35:50.465345+0000","last_became_active":"2024-09-21T07:35:48.465970+0000","last_became_peered":"2024-09-21T07:35:48.465970+0000","last_unstale":"2024-09-21T07:35:50.465345+0000","last_undegraded":"2024-09-21T07:35:50.465345+0000","last_fullsized":"2024-09-21T07:35:50.465345+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:15:41.920172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.607575+0000","last_change":"2024-09-21T07:35:27.874772+0000","last_active":"2024-09-21T07:35:50.607575+0000","last_peered":"2024-09-21T07:35:50.607575+0000","last_clean":"2024-09-21T07:35:50.607575+0000","last_became_active":"2024-09-21T07:35:27.874434+0000","last_became_peered":"2024-09-21T07:35:27.874434+0000","last_unstale":"2024-09-21T07:35:50.607575+0000","last_undegraded":"2024-09-21T07:35:50.607575+0000","last_fullsized":"2024-09-21T07:35:50.607575+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:31.998755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":44,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.467820+0000","last_change":"2024-09-21T07:35:28.859153+0000","last_active":"2024-09-21T07:35:50.467820+0000","last_peered":"2024-09-21T07:35:50.467820+0000","last_clean":"2024-09-21T07:35:50.467820+0000","last_became_active":"2024-09-21T07:35:26.874428+0000","last_became_peered":"2024-09-21T07:35:26.874428+0000","last_unstale":"2024-09-21T07:35:50.467820+0000","last_undegraded":"2024-09-21T07:35:50.467820+0000","last_fullsized":"2024-09-21T07:35:50.467820+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:04.487621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048557299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":96,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466227+0000","last_change":"2024-09-21T07:34:24.538845+0000","last_active":"2024-09-21T07:35:50.466227+0000","last_peered":"2024-09-21T07:35:50.466227+0000","last_clean":"2024-09-21T07:35:50.466227+0000","last_became_active":"2024-09-21T07:34:23.636372+0000","last_became_peered":"2024-09-21T07:34:23.636372+0000","last_unstale":"2024-09-21T07:35:50.466227+0000","last_undegraded":"2024-09-21T07:35:50.466227+0000","last_fullsized":"2024-09-21T07:35:50.466227+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:29.393813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841451,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.610313+0000","last_change":"2024-09-21T07:35:49.481457+0000","last_active":"2024-09-21T07:35:50.610313+0000","last_peered":"2024-09-21T07:35:50.610313+0000","last_clean":"2024-09-21T07:35:50.610313+0000","last_became_active":"2024-09-21T07:35:49.476056+0000","last_became_peered":"2024-09-21T07:35:49.476056+0000","last_unstale":"2024-09-21T07:35:50.610313+0000","last_undegraded":"2024-09-21T07:35:50.610313+0000","last_fullsized":"2024-09-21T07:35:50.610313+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:52:01.228329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.607796+0000","last_change":"2024-09-21T07:35:48.464324+0000","last_active":"2024-09-21T07:35:50.607796+0000","last_peered":"2024-09-21T07:35:50.607796+0000","last_clean":"2024-09-21T07:35:50.607796+0000","last_became_active":"2024-09-21T07:35:48.463877+0000","last_became_peered":"2024-09-21T07:35:48.463877+0000","last_unstale":"2024-09-21T07:35:50.607796+0000","last_undegraded":"2024-09-21T07:35:50.607796+0000","last_fullsized":"2024-09-21T07:35:50.607796+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:22.251973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.465848+0000","last_change":"2024-09-21T07:35:27.877801+0000","last_active":"2024-09-21T07:35:50.465848+0000","last_peered":"2024-09-21T07:35:50.465848+0000","last_clean":"2024-09-21T07:35:50.465848+0000","last_became_active":"2024-09-21T07:35:27.877464+0000","last_became_peered":"2024-09-21T07:35:27.877464+0000","last_unstale":"2024-09-21T07:35:50.465848+0000","last_undegraded":"2024-09-21T07:35:50.465848+0000","last_fullsized":"2024-09-21T07:35:50.465848+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:08.814929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.611053+0000","last_change":"2024-09-21T07:35:28.863732+0000","last_active":"2024-09-21T07:35:50.611053+0000","last_peered":"2024-09-21T07:35:50.611053+0000","last_clean":"2024-09-21T07:35:50.611053+0000","last_became_active":"2024-09-21T07:35:26.877392+0000","last_became_peered":"2024-09-21T07:35:26.877392+0000","last_unstale":"2024-09-21T07:35:50.611053+0000","last_undegraded":"2024-09-21T07:35:50.611053+0000","last_fullsized":"2024-09-21T07:35:50.611053+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071425700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":94,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609652+0000","last_change":"2024-09-21T07:34:24.259686+0000","last_active":"2024-09-21T07:35:50.609652+0000","last_peered":"2024-09-21T07:35:50.609652+0000","last_clean":"2024-09-21T07:35:50.609652+0000","last_became_active":"2024-09-21T07:34:23.274405+0000","last_became_peered":"2024-09-21T07:34:23.274405+0000","last_unstale":"2024-09-21T07:35:50.609652+0000","last_undegraded":"2024-09-21T07:35:50.609652+0000","last_fullsized":"2024-09-21T07:35:50.609652+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:26.304693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022009920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.607600+0000","last_change":"2024-09-21T07:35:49.487661+0000","last_active":"2024-09-21T07:35:50.607600+0000","last_peered":"2024-09-21T07:35:50.607600+0000","last_clean":"2024-09-21T07:35:50.607600+0000","last_became_active":"2024-09-21T07:35:49.487295+0000","last_became_peered":"2024-09-21T07:35:49.487295+0000","last_unstale":"2024-09-21T07:35:50.607600+0000","last_undegraded":"2024-09-21T07:35:50.607600+0000","last_fullsized":"2024-09-21T07:35:50.607600+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:04:12.059484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609915+0000","last_change":"2024-09-21T07:35:48.472374+0000","last_active":"2024-09-21T07:35:50.609915+0000","last_peered":"2024-09-21T07:35:50.609915+0000","last_clean":"2024-09-21T07:35:50.609915+0000","last_became_active":"2024-09-21T07:35:48.471831+0000","last_became_peered":"2024-09-21T07:35:48.471831+0000","last_unstale":"2024-09-21T07:35:50.609915+0000","last_undegraded":"2024-09-21T07:35:50.609915+0000","last_fullsized":"2024-09-21T07:35:50.609915+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:35:38.133415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608635+0000","last_change":"2024-09-21T07:35:27.867336+0000","last_active":"2024-09-21T07:35:50.608635+0000","last_peered":"2024-09-21T07:35:50.608635+0000","last_clean":"2024-09-21T07:35:50.608635+0000","last_became_active":"2024-09-21T07:35:27.867015+0000","last_became_peered":"2024-09-21T07:35:27.867015+0000","last_unstale":"2024-09-21T07:35:50.608635+0000","last_undegraded":"2024-09-21T07:35:50.608635+0000","last_fullsized":"2024-09-21T07:35:50.608635+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:59.214139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609402+0000","last_change":"2024-09-21T07:35:29.493137+0000","last_active":"2024-09-21T07:35:50.609402+0000","last_peered":"2024-09-21T07:35:50.609402+0000","last_clean":"2024-09-21T07:35:50.609402+0000","last_became_active":"2024-09-21T07:35:26.872230+0000","last_became_peered":"2024-09-21T07:35:26.872230+0000","last_unstale":"2024-09-21T07:35:50.609402+0000","last_undegraded":"2024-09-21T07:35:50.609402+0000","last_fullsized":"2024-09-21T07:35:50.609402+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:24.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001197449,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":94,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608083+0000","last_change":"2024-09-21T07:34:24.258754+0000","last_active":"2024-09-21T07:35:50.608083+0000","last_peered":"2024-09-21T07:35:50.608083+0000","last_clean":"2024-09-21T07:35:50.608083+0000","last_became_active":"2024-09-21T07:34:23.272657+0000","last_became_peered":"2024-09-21T07:34:23.272657+0000","last_unstale":"2024-09-21T07:35:50.608083+0000","last_undegraded":"2024-09-21T07:35:50.608083+0000","last_fullsized":"2024-09-21T07:35:50.608083+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:05.477099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067143200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608194+0000","last_change":"2024-09-21T07:35:49.488812+0000","last_active":"2024-09-21T07:35:50.608194+0000","last_peered":"2024-09-21T07:35:50.608194+0000","last_clean":"2024-09-21T07:35:50.608194+0000","last_became_active":"2024-09-21T07:35:49.482857+0000","last_became_peered":"2024-09-21T07:35:49.482857+0000","last_unstale":"2024-09-21T07:35:50.608194+0000","last_undegraded":"2024-09-21T07:35:50.608194+0000","last_fullsized":"2024-09-21T07:35:50.608194+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:28:04.816276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466099+0000","last_change":"2024-09-21T07:35:48.463728+0000","last_active":"2024-09-21T07:35:50.466099+0000","last_peered":"2024-09-21T07:35:50.466099+0000","last_clean":"2024-09-21T07:35:50.466099+0000","last_became_active":"2024-09-21T07:35:48.463347+0000","last_became_peered":"2024-09-21T07:35:48.463347+0000","last_unstale":"2024-09-21T07:35:50.466099+0000","last_undegraded":"2024-09-21T07:35:50.466099+0000","last_fullsized":"2024-09-21T07:35:50.466099+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:12.497861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.610391+0000","last_change":"2024-09-21T07:35:27.876021+0000","last_active":"2024-09-21T07:35:50.610391+0000","last_peered":"2024-09-21T07:35:50.610391+0000","last_clean":"2024-09-21T07:35:50.610391+0000","last_became_active":"2024-09-21T07:35:27.875724+0000","last_became_peered":"2024-09-21T07:35:27.875724+0000","last_unstale":"2024-09-21T07:35:50.610391+0000","last_undegraded":"2024-09-21T07:35:50.610391+0000","last_fullsized":"2024-09-21T07:35:50.610391+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:03.022812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608309+0000","last_change":"2024-09-21T07:35:29.492570+0000","last_active":"2024-09-21T07:35:50.608309+0000","last_peered":"2024-09-21T07:35:50.608309+0000","last_clean":"2024-09-21T07:35:50.608309+0000","last_became_active":"2024-09-21T07:35:26.874316+0000","last_became_peered":"2024-09-21T07:35:26.874316+0000","last_unstale":"2024-09-21T07:35:50.608309+0000","last_undegraded":"2024-09-21T07:35:50.608309+0000","last_fullsized":"2024-09-21T07:35:50.608309+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:16.512323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011990760000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":93,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609895+0000","last_change":"2024-09-21T07:34:25.085232+0000","last_active":"2024-09-21T07:35:50.609895+0000","last_peered":"2024-09-21T07:35:50.609895+0000","last_clean":"2024-09-21T07:35:50.609895+0000","last_became_active":"2024-09-21T07:34:23.277807+0000","last_became_peered":"2024-09-21T07:34:23.277807+0000","last_unstale":"2024-09-21T07:35:50.609895+0000","last_undegraded":"2024-09-21T07:35:50.609895+0000","last_fullsized":"2024-09-21T07:35:50.609895+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:36.362919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086619099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.610953+0000","last_change":"2024-09-21T07:35:49.474323+0000","last_active":"2024-09-21T07:35:50.610953+0000","last_peered":"2024-09-21T07:35:50.610953+0000","last_clean":"2024-09-21T07:35:50.610953+0000","last_became_active":"2024-09-21T07:35:49.474057+0000","last_became_peered":"2024-09-21T07:35:49.474057+0000","last_unstale":"2024-09-21T07:35:50.610953+0000","last_undegraded":"2024-09-21T07:35:50.610953+0000","last_fullsized":"2024-09-21T07:35:50.610953+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:59.379416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609417+0000","last_change":"2024-09-21T07:35:48.477694+0000","last_active":"2024-09-21T07:35:50.609417+0000","last_peered":"2024-09-21T07:35:50.609417+0000","last_clean":"2024-09-21T07:35:50.609417+0000","last_became_active":"2024-09-21T07:35:48.477368+0000","last_became_peered":"2024-09-21T07:35:48.477368+0000","last_unstale":"2024-09-21T07:35:50.609417+0000","last_undegraded":"2024-09-21T07:35:50.609417+0000","last_fullsized":"2024-09-21T07:35:50.609417+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:57.866409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466736+0000","last_change":"2024-09-21T07:35:27.872798+0000","last_active":"2024-09-21T07:35:50.466736+0000","last_peered":"2024-09-21T07:35:50.466736+0000","last_clean":"2024-09-21T07:35:50.466736+0000","last_became_active":"2024-09-21T07:35:27.872483+0000","last_became_peered":"2024-09-21T07:35:27.872483+0000","last_unstale":"2024-09-21T07:35:50.466736+0000","last_undegraded":"2024-09-21T07:35:50.466736+0000","last_fullsized":"2024-09-21T07:35:50.466736+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:11.933469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.465575+0000","last_change":"2024-09-21T07:35:29.206723+0000","last_active":"2024-09-21T07:35:50.465575+0000","last_peered":"2024-09-21T07:35:50.465575+0000","last_clean":"2024-09-21T07:35:50.465575+0000","last_became_active":"2024-09-21T07:35:26.872377+0000","last_became_peered":"2024-09-21T07:35:26.872377+0000","last_unstale":"2024-09-21T07:35:50.465575+0000","last_undegraded":"2024-09-21T07:35:50.465575+0000","last_fullsized":"2024-09-21T07:35:50.465575+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:29.086862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083077200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":90,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609755+0000","last_change":"2024-09-21T07:34:24.597672+0000","last_active":"2024-09-21T07:35:50.609755+0000","last_peered":"2024-09-21T07:35:50.609755+0000","last_clean":"2024-09-21T07:35:50.609755+0000","last_became_active":"2024-09-21T07:34:23.267770+0000","last_became_peered":"2024-09-21T07:34:23.267770+0000","last_unstale":"2024-09-21T07:35:50.609755+0000","last_undegraded":"2024-09-21T07:35:50.609755+0000","last_fullsized":"2024-09-21T07:35:50.609755+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:51.629476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086378500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.467668+0000","last_change":"2024-09-21T07:35:49.460209+0000","last_active":"2024-09-21T07:35:50.467668+0000","last_peered":"2024-09-21T07:35:50.467668+0000","last_clean":"2024-09-21T07:35:50.467668+0000","last_became_active":"2024-09-21T07:35:49.459838+0000","last_became_peered":"2024-09-21T07:35:49.459838+0000","last_unstale":"2024-09-21T07:35:50.467668+0000","last_undegraded":"2024-09-21T07:35:50.467668+0000","last_fullsized":"2024-09-21T07:35:50.467668+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:26:49.229370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466661+0000","last_change":"2024-09-21T07:35:48.466416+0000","last_active":"2024-09-21T07:35:50.466661+0000","last_peered":"2024-09-21T07:35:50.466661+0000","last_clean":"2024-09-21T07:35:50.466661+0000","last_became_active":"2024-09-21T07:35:48.466066+0000","last_became_peered":"2024-09-21T07:35:48.466066+0000","last_unstale":"2024-09-21T07:35:50.466661+0000","last_undegraded":"2024-09-21T07:35:50.466661+0000","last_fullsized":"2024-09-21T07:35:50.466661+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:41.865640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.467656+0000","last_change":"2024-09-21T07:35:27.870121+0000","last_active":"2024-09-21T07:35:50.467656+0000","last_peered":"2024-09-21T07:35:50.467656+0000","last_clean":"2024-09-21T07:35:50.467656+0000","last_became_active":"2024-09-21T07:35:27.869811+0000","last_became_peered":"2024-09-21T07:35:27.869811+0000","last_unstale":"2024-09-21T07:35:50.467656+0000","last_undegraded":"2024-09-21T07:35:50.467656+0000","last_fullsized":"2024-09-21T07:35:50.467656+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:31.778461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":43,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.607809+0000","last_change":"2024-09-21T07:35:28.860699+0000","last_active":"2024-09-21T07:35:50.607809+0000","last_peered":"2024-09-21T07:35:50.607809+0000","last_clean":"2024-09-21T07:35:50.607809+0000","last_became_active":"2024-09-21T07:35:26.876779+0000","last_became_peered":"2024-09-21T07:35:26.876779+0000","last_unstale":"2024-09-21T07:35:50.607809+0000","last_undegraded":"2024-09-21T07:35:50.607809+0000","last_fullsized":"2024-09-21T07:35:50.607809+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:28.286403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000913188,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":95,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.611310+0000","last_change":"2024-09-21T07:34:24.260181+0000","last_active":"2024-09-21T07:35:50.611310+0000","last_peered":"2024-09-21T07:35:50.611310+0000","last_clean":"2024-09-21T07:35:50.611310+0000","last_became_active":"2024-09-21T07:34:23.275580+0000","last_became_peered":"2024-09-21T07:34:23.275580+0000","last_unstale":"2024-09-21T07:35:50.611310+0000","last_undegraded":"2024-09-21T07:35:50.611310+0000","last_fullsized":"2024-09-21T07:35:50.611310+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:49.857622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014706980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.467022+0000","last_change":"2024-09-21T07:35:49.477557+0000","last_active":"2024-09-21T07:35:50.467022+0000","last_peered":"2024-09-21T07:35:50.467022+0000","last_clean":"2024-09-21T07:35:50.467022+0000","last_became_active":"2024-09-21T07:35:49.477265+0000","last_became_peered":"2024-09-21T07:35:49.477265+0000","last_unstale":"2024-09-21T07:35:50.467022+0000","last_undegraded":"2024-09-21T07:35:50.467022+0000","last_fullsized":"2024-09-21T07:35:50.467022+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:25:59.317284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"77'2","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609551+0000","last_change":"2024-09-21T07:35:48.466624+0000","last_active":"2024-09-21T07:35:50.609551+0000","last_peered":"2024-09-21T07:35:50.609551+0000","last_clean":"2024-09-21T07:35:50.609551+0000","last_became_active":"2024-09-21T07:35:48.466246+0000","last_became_peered":"2024-09-21T07:35:48.466246+0000","last_unstale":"2024-09-21T07:35:50.609551+0000","last_undegraded":"2024-09-21T07:35:50.609551+0000","last_fullsized":"2024-09-21T07:35:50.609551+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:36.646120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":36,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.467339+0000","last_change":"2024-09-21T07:35:27.875393+0000","last_active":"2024-09-21T07:35:50.467339+0000","last_peered":"2024-09-21T07:35:50.467339+0000","last_clean":"2024-09-21T07:35:50.467339+0000","last_became_active":"2024-09-21T07:35:27.874671+0000","last_became_peered":"2024-09-21T07:35:27.874671+0000","last_unstale":"2024-09-21T07:35:50.467339+0000","last_undegraded":"2024-09-21T07:35:50.467339+0000","last_fullsized":"2024-09-21T07:35:50.467339+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466315+0000","last_change":"2024-09-21T07:35:29.206296+0000","last_active":"2024-09-21T07:35:50.466315+0000","last_peered":"2024-09-21T07:35:50.466315+0000","last_clean":"2024-09-21T07:35:50.466315+0000","last_became_active":"2024-09-21T07:35:26.874575+0000","last_became_peered":"2024-09-21T07:35:26.874575+0000","last_unstale":"2024-09-21T07:35:50.466315+0000","last_undegraded":"2024-09-21T07:35:50.466315+0000","last_fullsized":"2024-09-21T07:35:50.466315+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:22.771154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061011599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":94,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608912+0000","last_change":"2024-09-21T07:34:24.259112+0000","last_active":"2024-09-21T07:35:50.608912+0000","last_peered":"2024-09-21T07:35:50.608912+0000","last_clean":"2024-09-21T07:35:50.608912+0000","last_became_active":"2024-09-21T07:34:23.635444+0000","last_became_peered":"2024-09-21T07:34:23.635444+0000","last_unstale":"2024-09-21T07:35:50.608912+0000","last_undegraded":"2024-09-21T07:35:50.608912+0000","last_fullsized":"2024-09-21T07:35:50.608912+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:46.255385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001952636,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.611134+0000","last_change":"2024-09-21T07:35:49.469045+0000","last_active":"2024-09-21T07:35:50.611134+0000","last_peered":"2024-09-21T07:35:50.611134+0000","last_clean":"2024-09-21T07:35:50.611134+0000","last_became_active":"2024-09-21T07:35:49.464330+0000","last_became_peered":"2024-09-21T07:35:49.464330+0000","last_unstale":"2024-09-21T07:35:50.611134+0000","last_undegraded":"2024-09-21T07:35:50.611134+0000","last_fullsized":"2024-09-21T07:35:50.611134+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:45.434454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"78'1","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:51.449813+0000","last_change":"2024-09-21T07:35:48.469515+0000","last_active":"2024-09-21T07:35:51.449813+0000","last_peered":"2024-09-21T07:35:51.449813+0000","last_clean":"2024-09-21T07:35:51.449813+0000","last_became_active":"2024-09-21T07:35:48.469155+0000","last_became_peered":"2024-09-21T07:35:48.469155+0000","last_unstale":"2024-09-21T07:35:51.449813+0000","last_undegraded":"2024-09-21T07:35:51.449813+0000","last_fullsized":"2024-09-21T07:35:51.449813+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:32.952980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":36,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.467323+0000","last_change":"2024-09-21T07:35:27.878356+0000","last_active":"2024-09-21T07:35:50.467323+0000","last_peered":"2024-09-21T07:35:50.467323+0000","last_clean":"2024-09-21T07:35:50.467323+0000","last_became_active":"2024-09-21T07:35:27.877982+0000","last_became_peered":"2024-09-21T07:35:27.877982+0000","last_unstale":"2024-09-21T07:35:50.467323+0000","last_undegraded":"2024-09-21T07:35:50.467323+0000","last_fullsized":"2024-09-21T07:35:50.467323+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":43,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466211+0000","last_change":"2024-09-21T07:35:29.053131+0000","last_active":"2024-09-21T07:35:50.466211+0000","last_peered":"2024-09-21T07:35:50.466211+0000","last_clean":"2024-09-21T07:35:50.466211+0000","last_became_active":"2024-09-21T07:35:26.870516+0000","last_became_peered":"2024-09-21T07:35:26.870516+0000","last_unstale":"2024-09-21T07:35:50.466211+0000","last_undegraded":"2024-09-21T07:35:50.466211+0000","last_fullsized":"2024-09-21T07:35:50.466211+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:10.924491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070705699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":96,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.465696+0000","last_change":"2024-09-21T07:34:24.539152+0000","last_active":"2024-09-21T07:35:50.465696+0000","last_peered":"2024-09-21T07:35:50.465696+0000","last_clean":"2024-09-21T07:35:50.465696+0000","last_became_active":"2024-09-21T07:34:23.259573+0000","last_became_peered":"2024-09-21T07:34:23.259573+0000","last_unstale":"2024-09-21T07:35:50.465696+0000","last_undegraded":"2024-09-21T07:35:50.465696+0000","last_fullsized":"2024-09-21T07:35:50.465696+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:26.986900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058108400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.610210+0000","last_change":"2024-09-21T07:35:49.478512+0000","last_active":"2024-09-21T07:35:50.610210+0000","last_peered":"2024-09-21T07:35:50.610210+0000","last_clean":"2024-09-21T07:35:50.610210+0000","last_became_active":"2024-09-21T07:35:49.478207+0000","last_became_peered":"2024-09-21T07:35:49.478207+0000","last_unstale":"2024-09-21T07:35:50.610210+0000","last_undegraded":"2024-09-21T07:35:50.610210+0000","last_fullsized":"2024-09-21T07:35:50.610210+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:05.236363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.607832+0000","last_change":"2024-09-21T07:35:48.453787+0000","last_active":"2024-09-21T07:35:50.607832+0000","last_peered":"2024-09-21T07:35:50.607832+0000","last_clean":"2024-09-21T07:35:50.607832+0000","last_became_active":"2024-09-21T07:35:48.453478+0000","last_became_peered":"2024-09-21T07:35:48.453478+0000","last_unstale":"2024-09-21T07:35:50.607832+0000","last_undegraded":"2024-09-21T07:35:50.607832+0000","last_fullsized":"2024-09-21T07:35:50.607832+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:18:29.322070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":36,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466090+0000","last_change":"2024-09-21T07:35:27.873726+0000","last_active":"2024-09-21T07:35:50.466090+0000","last_peered":"2024-09-21T07:35:50.466090+0000","last_clean":"2024-09-21T07:35:50.466090+0000","last_became_active":"2024-09-21T07:35:27.871421+0000","last_became_peered":"2024-09-21T07:35:27.871421+0000","last_unstale":"2024-09-21T07:35:50.466090+0000","last_undegraded":"2024-09-21T07:35:50.466090+0000","last_fullsized":"2024-09-21T07:35:50.466090+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:00.168915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":43,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.610796+0000","last_change":"2024-09-21T07:35:28.861032+0000","last_active":"2024-09-21T07:35:50.610796+0000","last_peered":"2024-09-21T07:35:50.610796+0000","last_clean":"2024-09-21T07:35:50.610796+0000","last_became_active":"2024-09-21T07:35:26.873149+0000","last_became_peered":"2024-09-21T07:35:26.873149+0000","last_unstale":"2024-09-21T07:35:50.610796+0000","last_undegraded":"2024-09-21T07:35:50.610796+0000","last_fullsized":"2024-09-21T07:35:50.610796+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:59.435981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011988369999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":94,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608742+0000","last_change":"2024-09-21T07:34:24.259902+0000","last_active":"2024-09-21T07:35:50.608742+0000","last_peered":"2024-09-21T07:35:50.608742+0000","last_clean":"2024-09-21T07:35:50.608742+0000","last_became_active":"2024-09-21T07:34:23.271902+0000","last_became_peered":"2024-09-21T07:34:23.271902+0000","last_unstale":"2024-09-21T07:35:50.608742+0000","last_undegraded":"2024-09-21T07:35:50.608742+0000","last_fullsized":"2024-09-21T07:35:50.608742+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:59.439525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001686358,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466202+0000","last_change":"2024-09-21T07:35:49.479584+0000","last_active":"2024-09-21T07:35:50.466202+0000","last_peered":"2024-09-21T07:35:50.466202+0000","last_clean":"2024-09-21T07:35:50.466202+0000","last_became_active":"2024-09-21T07:35:49.476299+0000","last_became_peered":"2024-09-21T07:35:49.476299+0000","last_unstale":"2024-09-21T07:35:50.466202+0000","last_undegraded":"2024-09-21T07:35:50.466202+0000","last_fullsized":"2024-09-21T07:35:50.466202+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:45.827149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.607657+0000","last_change":"2024-09-21T07:35:48.469152+0000","last_active":"2024-09-21T07:35:50.607657+0000","last_peered":"2024-09-21T07:35:50.607657+0000","last_clean":"2024-09-21T07:35:50.607657+0000","last_became_active":"2024-09-21T07:35:48.468821+0000","last_became_peered":"2024-09-21T07:35:48.468821+0000","last_unstale":"2024-09-21T07:35:50.607657+0000","last_undegraded":"2024-09-21T07:35:50.607657+0000","last_fullsized":"2024-09-21T07:35:50.607657+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:31.575643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608383+0000","last_change":"2024-09-21T07:35:27.872637+0000","last_active":"2024-09-21T07:35:50.608383+0000","last_peered":"2024-09-21T07:35:50.608383+0000","last_clean":"2024-09-21T07:35:50.608383+0000","last_became_active":"2024-09-21T07:35:27.872149+0000","last_became_peered":"2024-09-21T07:35:27.872149+0000","last_unstale":"2024-09-21T07:35:50.608383+0000","last_undegraded":"2024-09-21T07:35:50.608383+0000","last_fullsized":"2024-09-21T07:35:50.608383+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:27.697872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609520+0000","last_change":"2024-09-21T07:35:29.493362+0000","last_active":"2024-09-21T07:35:50.609520+0000","last_peered":"2024-09-21T07:35:50.609520+0000","last_clean":"2024-09-21T07:35:50.609520+0000","last_became_active":"2024-09-21T07:35:26.873018+0000","last_became_peered":"2024-09-21T07:35:26.873018+0000","last_unstale":"2024-09-21T07:35:50.609520+0000","last_undegraded":"2024-09-21T07:35:50.609520+0000","last_fullsized":"2024-09-21T07:35:50.609520+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:02.460557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012496510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":94,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609363+0000","last_change":"2024-09-21T07:34:24.259662+0000","last_active":"2024-09-21T07:35:50.609363+0000","last_peered":"2024-09-21T07:35:50.609363+0000","last_clean":"2024-09-21T07:35:50.609363+0000","last_became_active":"2024-09-21T07:34:23.278812+0000","last_became_peered":"2024-09-21T07:34:23.278812+0000","last_unstale":"2024-09-21T07:35:50.609363+0000","last_undegraded":"2024-09-21T07:35:50.609363+0000","last_fullsized":"2024-09-21T07:35:50.609363+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:49.829974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356463,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466023+0000","last_change":"2024-09-21T07:35:49.493721+0000","last_active":"2024-09-21T07:35:50.466023+0000","last_peered":"2024-09-21T07:35:50.466023+0000","last_clean":"2024-09-21T07:35:50.466023+0000","last_became_active":"2024-09-21T07:35:49.493430+0000","last_became_peered":"2024-09-21T07:35:49.493430+0000","last_unstale":"2024-09-21T07:35:50.466023+0000","last_undegraded":"2024-09-21T07:35:50.466023+0000","last_fullsized":"2024-09-21T07:35:50.466023+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:43:22.507011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"77'1","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608310+0000","last_change":"2024-09-21T07:35:48.476792+0000","last_active":"2024-09-21T07:35:50.608310+0000","last_peered":"2024-09-21T07:35:50.608310+0000","last_clean":"2024-09-21T07:35:50.608310+0000","last_became_active":"2024-09-21T07:35:48.476413+0000","last_became_peered":"2024-09-21T07:35:48.476413+0000","last_unstale":"2024-09-21T07:35:50.608310+0000","last_undegraded":"2024-09-21T07:35:50.608310+0000","last_fullsized":"2024-09-21T07:35:50.608310+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:57.273910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608457+0000","last_change":"2024-09-21T07:35:27.868448+0000","last_active":"2024-09-21T07:35:50.608457+0000","last_peered":"2024-09-21T07:35:50.608457+0000","last_clean":"2024-09-21T07:35:50.608457+0000","last_became_active":"2024-09-21T07:35:27.868105+0000","last_became_peered":"2024-09-21T07:35:27.868105+0000","last_unstale":"2024-09-21T07:35:50.608457+0000","last_undegraded":"2024-09-21T07:35:50.608457+0000","last_fullsized":"2024-09-21T07:35:50.608457+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:41.468702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608872+0000","last_change":"2024-09-21T07:35:29.492974+0000","last_active":"2024-09-21T07:35:50.608872+0000","last_peered":"2024-09-21T07:35:50.608872+0000","last_clean":"2024-09-21T07:35:50.608872+0000","last_became_active":"2024-09-21T07:35:26.872165+0000","last_became_peered":"2024-09-21T07:35:26.872165+0000","last_unstale":"2024-09-21T07:35:50.608872+0000","last_undegraded":"2024-09-21T07:35:50.608872+0000","last_fullsized":"2024-09-21T07:35:50.608872+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:59.912520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013795179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":93,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608446+0000","last_change":"2024-09-21T07:34:25.084861+0000","last_active":"2024-09-21T07:35:50.608446+0000","last_peered":"2024-09-21T07:35:50.608446+0000","last_clean":"2024-09-21T07:35:50.608446+0000","last_became_active":"2024-09-21T07:34:23.277708+0000","last_became_peered":"2024-09-21T07:34:23.277708+0000","last_unstale":"2024-09-21T07:35:50.608446+0000","last_undegraded":"2024-09-21T07:35:50.608446+0000","last_fullsized":"2024-09-21T07:35:50.608446+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:40.816488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000597566,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466237+0000","last_change":"2024-09-21T07:35:49.471119+0000","last_active":"2024-09-21T07:35:50.466237+0000","last_peered":"2024-09-21T07:35:50.466237+0000","last_clean":"2024-09-21T07:35:50.466237+0000","last_became_active":"2024-09-21T07:35:49.470681+0000","last_became_peered":"2024-09-21T07:35:49.470681+0000","last_unstale":"2024-09-21T07:35:50.466237+0000","last_undegraded":"2024-09-21T07:35:50.466237+0000","last_fullsized":"2024-09-21T07:35:50.466237+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:24.608325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"77'1","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.465930+0000","last_change":"2024-09-21T07:35:48.472971+0000","last_active":"2024-09-21T07:35:50.465930+0000","last_peered":"2024-09-21T07:35:50.465930+0000","last_clean":"2024-09-21T07:35:50.465930+0000","last_became_active":"2024-09-21T07:35:48.472657+0000","last_became_peered":"2024-09-21T07:35:48.472657+0000","last_unstale":"2024-09-21T07:35:50.465930+0000","last_undegraded":"2024-09-21T07:35:50.465930+0000","last_fullsized":"2024-09-21T07:35:50.465930+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:47:22.761346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608559+0000","last_change":"2024-09-21T07:35:27.874640+0000","last_active":"2024-09-21T07:35:50.608559+0000","last_peered":"2024-09-21T07:35:50.608559+0000","last_clean":"2024-09-21T07:35:50.608559+0000","last_became_active":"2024-09-21T07:35:27.874265+0000","last_became_peered":"2024-09-21T07:35:27.874265+0000","last_unstale":"2024-09-21T07:35:50.608559+0000","last_undegraded":"2024-09-21T07:35:50.608559+0000","last_fullsized":"2024-09-21T07:35:50.608559+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:45.754654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466907+0000","last_change":"2024-09-21T07:35:29.207392+0000","last_active":"2024-09-21T07:35:50.466907+0000","last_peered":"2024-09-21T07:35:50.466907+0000","last_clean":"2024-09-21T07:35:50.466907+0000","last_became_active":"2024-09-21T07:35:26.870917+0000","last_became_peered":"2024-09-21T07:35:26.870917+0000","last_unstale":"2024-09-21T07:35:50.466907+0000","last_undegraded":"2024-09-21T07:35:50.466907+0000","last_fullsized":"2024-09-21T07:35:50.466907+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:59.159937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059198499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":94,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608262+0000","last_change":"2024-09-21T07:34:24.258709+0000","last_active":"2024-09-21T07:35:50.608262+0000","last_peered":"2024-09-21T07:35:50.608262+0000","last_clean":"2024-09-21T07:35:50.608262+0000","last_became_active":"2024-09-21T07:34:23.279042+0000","last_became_peered":"2024-09-21T07:34:23.279042+0000","last_unstale":"2024-09-21T07:35:50.608262+0000","last_undegraded":"2024-09-21T07:35:50.608262+0000","last_fullsized":"2024-09-21T07:35:50.608262+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:01.762117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019282100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609852+0000","last_change":"2024-09-21T07:35:49.482465+0000","last_active":"2024-09-21T07:35:50.609852+0000","last_peered":"2024-09-21T07:35:50.609852+0000","last_clean":"2024-09-21T07:35:50.609852+0000","last_became_active":"2024-09-21T07:35:49.482124+0000","last_became_peered":"2024-09-21T07:35:49.482124+0000","last_unstale":"2024-09-21T07:35:50.609852+0000","last_undegraded":"2024-09-21T07:35:50.609852+0000","last_fullsized":"2024-09-21T07:35:50.609852+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:03:24.555132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"77'2","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609723+0000","last_change":"2024-09-21T07:35:48.468248+0000","last_active":"2024-09-21T07:35:50.609723+0000","last_peered":"2024-09-21T07:35:50.609723+0000","last_clean":"2024-09-21T07:35:50.609723+0000","last_became_active":"2024-09-21T07:35:48.467906+0000","last_became_peered":"2024-09-21T07:35:48.467906+0000","last_unstale":"2024-09-21T07:35:50.609723+0000","last_undegraded":"2024-09-21T07:35:50.609723+0000","last_fullsized":"2024-09-21T07:35:50.609723+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:23.050859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.610628+0000","last_change":"2024-09-21T07:35:27.870260+0000","last_active":"2024-09-21T07:35:50.610628+0000","last_peered":"2024-09-21T07:35:50.610628+0000","last_clean":"2024-09-21T07:35:50.610628+0000","last_became_active":"2024-09-21T07:35:27.869448+0000","last_became_peered":"2024-09-21T07:35:27.869448+0000","last_unstale":"2024-09-21T07:35:50.610628+0000","last_undegraded":"2024-09-21T07:35:50.610628+0000","last_fullsized":"2024-09-21T07:35:50.610628+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:16.946790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608401+0000","last_change":"2024-09-21T07:35:29.386367+0000","last_active":"2024-09-21T07:35:50.608401+0000","last_peered":"2024-09-21T07:35:50.608401+0000","last_clean":"2024-09-21T07:35:50.608401+0000","last_became_active":"2024-09-21T07:35:26.879238+0000","last_became_peered":"2024-09-21T07:35:26.879238+0000","last_unstale":"2024-09-21T07:35:50.608401+0000","last_undegraded":"2024-09-21T07:35:50.608401+0000","last_fullsized":"2024-09-21T07:35:50.608401+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:16.528260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092223100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":93,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.611141+0000","last_change":"2024-09-21T07:34:25.085635+0000","last_active":"2024-09-21T07:35:50.611141+0000","last_peered":"2024-09-21T07:35:50.611141+0000","last_clean":"2024-09-21T07:35:50.611141+0000","last_became_active":"2024-09-21T07:34:23.273566+0000","last_became_peered":"2024-09-21T07:34:23.273566+0000","last_unstale":"2024-09-21T07:35:50.611141+0000","last_undegraded":"2024-09-21T07:35:50.611141+0000","last_fullsized":"2024-09-21T07:35:50.611141+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:14.142272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079897300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.611446+0000","last_change":"2024-09-21T07:35:49.485489+0000","last_active":"2024-09-21T07:35:50.611446+0000","last_peered":"2024-09-21T07:35:50.611446+0000","last_clean":"2024-09-21T07:35:50.611446+0000","last_became_active":"2024-09-21T07:35:49.480270+0000","last_became_peered":"2024-09-21T07:35:49.480270+0000","last_unstale":"2024-09-21T07:35:50.611446+0000","last_undegraded":"2024-09-21T07:35:50.611446+0000","last_fullsized":"2024-09-21T07:35:50.611446+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:36.494776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"77'2","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466508+0000","last_change":"2024-09-21T07:35:48.462321+0000","last_active":"2024-09-21T07:35:50.466508+0000","last_peered":"2024-09-21T07:35:50.466508+0000","last_clean":"2024-09-21T07:35:50.466508+0000","last_became_active":"2024-09-21T07:35:48.461989+0000","last_became_peered":"2024-09-21T07:35:48.461989+0000","last_unstale":"2024-09-21T07:35:50.466508+0000","last_undegraded":"2024-09-21T07:35:50.466508+0000","last_fullsized":"2024-09-21T07:35:50.466508+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:25.184983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.610890+0000","last_change":"2024-09-21T07:35:27.878728+0000","last_active":"2024-09-21T07:35:50.610890+0000","last_peered":"2024-09-21T07:35:50.610890+0000","last_clean":"2024-09-21T07:35:50.610890+0000","last_became_active":"2024-09-21T07:35:27.878335+0000","last_became_peered":"2024-09-21T07:35:27.878335+0000","last_unstale":"2024-09-21T07:35:50.610890+0000","last_undegraded":"2024-09-21T07:35:50.610890+0000","last_fullsized":"2024-09-21T07:35:50.610890+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:25:37.731771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":44,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.465958+0000","last_change":"2024-09-21T07:35:28.858418+0000","last_active":"2024-09-21T07:35:50.465958+0000","last_peered":"2024-09-21T07:35:50.465958+0000","last_clean":"2024-09-21T07:35:50.465958+0000","last_became_active":"2024-09-21T07:35:26.876944+0000","last_became_peered":"2024-09-21T07:35:26.876944+0000","last_unstale":"2024-09-21T07:35:50.465958+0000","last_undegraded":"2024-09-21T07:35:50.465958+0000","last_fullsized":"2024-09-21T07:35:50.465958+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:04.514304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090862799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":90,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608502+0000","last_change":"2024-09-21T07:34:24.597926+0000","last_active":"2024-09-21T07:35:50.608502+0000","last_peered":"2024-09-21T07:35:50.608502+0000","last_clean":"2024-09-21T07:35:50.608502+0000","last_became_active":"2024-09-21T07:34:23.268926+0000","last_became_peered":"2024-09-21T07:34:23.268926+0000","last_unstale":"2024-09-21T07:35:50.608502+0000","last_undegraded":"2024-09-21T07:35:50.608502+0000","last_fullsized":"2024-09-21T07:35:50.608502+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:46.190752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101851,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.610487+0000","last_change":"2024-09-21T07:35:49.459448+0000","last_active":"2024-09-21T07:35:50.610487+0000","last_peered":"2024-09-21T07:35:50.610487+0000","last_clean":"2024-09-21T07:35:50.610487+0000","last_became_active":"2024-09-21T07:35:49.459127+0000","last_became_peered":"2024-09-21T07:35:49.459127+0000","last_unstale":"2024-09-21T07:35:50.610487+0000","last_undegraded":"2024-09-21T07:35:50.610487+0000","last_fullsized":"2024-09-21T07:35:50.610487+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:40.407157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"77'4","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.468039+0000","last_change":"2024-09-21T07:35:48.473637+0000","last_active":"2024-09-21T07:35:50.468039+0000","last_peered":"2024-09-21T07:35:50.468039+0000","last_clean":"2024-09-21T07:35:50.468039+0000","last_became_active":"2024-09-21T07:35:48.473220+0000","last_became_peered":"2024-09-21T07:35:48.473220+0000","last_unstale":"2024-09-21T07:35:50.468039+0000","last_undegraded":"2024-09-21T07:35:50.468039+0000","last_fullsized":"2024-09-21T07:35:50.468039+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:47:22.761346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609073+0000","last_change":"2024-09-21T07:35:27.877426+0000","last_active":"2024-09-21T07:35:50.609073+0000","last_peered":"2024-09-21T07:35:50.609073+0000","last_clean":"2024-09-21T07:35:50.609073+0000","last_became_active":"2024-09-21T07:35:27.877050+0000","last_became_peered":"2024-09-21T07:35:27.877050+0000","last_unstale":"2024-09-21T07:35:50.609073+0000","last_undegraded":"2024-09-21T07:35:50.609073+0000","last_fullsized":"2024-09-21T07:35:50.609073+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:36.268334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609781+0000","last_change":"2024-09-21T07:35:28.862273+0000","last_active":"2024-09-21T07:35:50.609781+0000","last_peered":"2024-09-21T07:35:50.609781+0000","last_clean":"2024-09-21T07:35:50.609781+0000","last_became_active":"2024-09-21T07:35:26.874011+0000","last_became_peered":"2024-09-21T07:35:26.874011+0000","last_unstale":"2024-09-21T07:35:50.609781+0000","last_undegraded":"2024-09-21T07:35:50.609781+0000","last_fullsized":"2024-09-21T07:35:50.609781+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:11.986950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071644799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":95,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609545+0000","last_change":"2024-09-21T07:34:24.259794+0000","last_active":"2024-09-21T07:35:50.609545+0000","last_peered":"2024-09-21T07:35:50.609545+0000","last_clean":"2024-09-21T07:35:50.609545+0000","last_became_active":"2024-09-21T07:34:23.265679+0000","last_became_peered":"2024-09-21T07:34:23.265679+0000","last_unstale":"2024-09-21T07:35:50.609545+0000","last_undegraded":"2024-09-21T07:35:50.609545+0000","last_fullsized":"2024-09-21T07:35:50.609545+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:02.575796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001298518,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.610676+0000","last_change":"2024-09-21T07:35:49.469429+0000","last_active":"2024-09-21T07:35:50.610676+0000","last_peered":"2024-09-21T07:35:50.610676+0000","last_clean":"2024-09-21T07:35:50.610676+0000","last_became_active":"2024-09-21T07:35:49.464946+0000","last_became_peered":"2024-09-21T07:35:49.464946+0000","last_unstale":"2024-09-21T07:35:50.610676+0000","last_undegraded":"2024-09-21T07:35:50.610676+0000","last_fullsized":"2024-09-21T07:35:50.610676+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:30.602494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"77'1","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608922+0000","last_change":"2024-09-21T07:35:48.470671+0000","last_active":"2024-09-21T07:35:50.608922+0000","last_peered":"2024-09-21T07:35:50.608922+0000","last_clean":"2024-09-21T07:35:50.608922+0000","last_became_active":"2024-09-21T07:35:48.470127+0000","last_became_peered":"2024-09-21T07:35:48.470127+0000","last_unstale":"2024-09-21T07:35:50.608922+0000","last_undegraded":"2024-09-21T07:35:50.608922+0000","last_fullsized":"2024-09-21T07:35:50.608922+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:09.916603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.468307+0000","last_change":"2024-09-21T07:35:27.877725+0000","last_active":"2024-09-21T07:35:50.468307+0000","last_peered":"2024-09-21T07:35:50.468307+0000","last_clean":"2024-09-21T07:35:50.468307+0000","last_became_active":"2024-09-21T07:35:27.877398+0000","last_became_peered":"2024-09-21T07:35:27.877398+0000","last_unstale":"2024-09-21T07:35:50.468307+0000","last_undegraded":"2024-09-21T07:35:50.468307+0000","last_fullsized":"2024-09-21T07:35:50.468307+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:05.859762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":43,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.607394+0000","last_change":"2024-09-21T07:35:28.860612+0000","last_active":"2024-09-21T07:35:50.607394+0000","last_peered":"2024-09-21T07:35:50.607394+0000","last_clean":"2024-09-21T07:35:50.607394+0000","last_became_active":"2024-09-21T07:35:26.876562+0000","last_became_peered":"2024-09-21T07:35:26.876562+0000","last_unstale":"2024-09-21T07:35:50.607394+0000","last_undegraded":"2024-09-21T07:35:50.607394+0000","last_fullsized":"2024-09-21T07:35:50.607394+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:06.824331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012252470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":94,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609575+0000","last_change":"2024-09-21T07:34:24.260330+0000","last_active":"2024-09-21T07:35:50.609575+0000","last_peered":"2024-09-21T07:35:50.609575+0000","last_clean":"2024-09-21T07:35:50.609575+0000","last_became_active":"2024-09-21T07:34:23.637210+0000","last_became_peered":"2024-09-21T07:34:23.637210+0000","last_unstale":"2024-09-21T07:35:50.609575+0000","last_undegraded":"2024-09-21T07:35:50.609575+0000","last_fullsized":"2024-09-21T07:35:50.609575+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.488451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001537093,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.607679+0000","last_change":"2024-09-21T07:35:49.478712+0000","last_active":"2024-09-21T07:35:50.607679+0000","last_peered":"2024-09-21T07:35:50.607679+0000","last_clean":"2024-09-21T07:35:50.607679+0000","last_became_active":"2024-09-21T07:35:49.478390+0000","last_became_peered":"2024-09-21T07:35:49.478390+0000","last_unstale":"2024-09-21T07:35:50.607679+0000","last_undegraded":"2024-09-21T07:35:50.607679+0000","last_fullsized":"2024-09-21T07:35:50.607679+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:32:30.142348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.465891+0000","last_change":"2024-09-21T07:35:48.472103+0000","last_active":"2024-09-21T07:35:50.465891+0000","last_peered":"2024-09-21T07:35:50.465891+0000","last_clean":"2024-09-21T07:35:50.465891+0000","last_became_active":"2024-09-21T07:35:48.471732+0000","last_became_peered":"2024-09-21T07:35:48.471732+0000","last_unstale":"2024-09-21T07:35:50.465891+0000","last_undegraded":"2024-09-21T07:35:50.465891+0000","last_fullsized":"2024-09-21T07:35:50.465891+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:41.865640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608261+0000","last_change":"2024-09-21T07:35:27.874757+0000","last_active":"2024-09-21T07:35:50.608261+0000","last_peered":"2024-09-21T07:35:50.608261+0000","last_clean":"2024-09-21T07:35:50.608261+0000","last_became_active":"2024-09-21T07:35:27.874360+0000","last_became_peered":"2024-09-21T07:35:27.874360+0000","last_unstale":"2024-09-21T07:35:50.608261+0000","last_undegraded":"2024-09-21T07:35:50.608261+0000","last_fullsized":"2024-09-21T07:35:50.608261+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:17.646912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609900+0000","last_change":"2024-09-21T07:35:29.493431+0000","last_active":"2024-09-21T07:35:50.609900+0000","last_peered":"2024-09-21T07:35:50.609900+0000","last_clean":"2024-09-21T07:35:50.609900+0000","last_became_active":"2024-09-21T07:35:26.873691+0000","last_became_peered":"2024-09-21T07:35:26.873691+0000","last_unstale":"2024-09-21T07:35:50.609900+0000","last_undegraded":"2024-09-21T07:35:50.609900+0000","last_fullsized":"2024-09-21T07:35:50.609900+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:51.705051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001162464,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":90,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608778+0000","last_change":"2024-09-21T07:34:24.597286+0000","last_active":"2024-09-21T07:35:50.608778+0000","last_peered":"2024-09-21T07:35:50.608778+0000","last_clean":"2024-09-21T07:35:50.608778+0000","last_became_active":"2024-09-21T07:34:23.267133+0000","last_became_peered":"2024-09-21T07:34:23.267133+0000","last_unstale":"2024-09-21T07:35:50.608778+0000","last_undegraded":"2024-09-21T07:35:50.608778+0000","last_fullsized":"2024-09-21T07:35:50.608778+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:18.257010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609169+0000","last_change":"2024-09-21T07:35:49.478854+0000","last_active":"2024-09-21T07:35:50.609169+0000","last_peered":"2024-09-21T07:35:50.609169+0000","last_clean":"2024-09-21T07:35:50.609169+0000","last_became_active":"2024-09-21T07:35:49.478563+0000","last_became_peered":"2024-09-21T07:35:49.478563+0000","last_unstale":"2024-09-21T07:35:50.609169+0000","last_undegraded":"2024-09-21T07:35:50.609169+0000","last_fullsized":"2024-09-21T07:35:50.609169+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:16:19.354123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"77'8","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.465811+0000","last_change":"2024-09-21T07:35:48.473957+0000","last_active":"2024-09-21T07:35:50.465811+0000","last_peered":"2024-09-21T07:35:50.465811+0000","last_clean":"2024-09-21T07:35:50.465811+0000","last_became_active":"2024-09-21T07:35:48.470611+0000","last_became_peered":"2024-09-21T07:35:48.470611+0000","last_unstale":"2024-09-21T07:35:50.465811+0000","last_undegraded":"2024-09-21T07:35:50.465811+0000","last_fullsized":"2024-09-21T07:35:50.465811+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:15:41.920172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466832+0000","last_change":"2024-09-21T07:35:27.864458+0000","last_active":"2024-09-21T07:35:50.466832+0000","last_peered":"2024-09-21T07:35:50.466832+0000","last_clean":"2024-09-21T07:35:50.466832+0000","last_became_active":"2024-09-21T07:35:27.864136+0000","last_became_peered":"2024-09-21T07:35:27.864136+0000","last_unstale":"2024-09-21T07:35:50.466832+0000","last_undegraded":"2024-09-21T07:35:50.466832+0000","last_fullsized":"2024-09-21T07:35:50.466832+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:20.162240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":44,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.467235+0000","last_change":"2024-09-21T07:35:28.858784+0000","last_active":"2024-09-21T07:35:50.467235+0000","last_peered":"2024-09-21T07:35:50.467235+0000","last_clean":"2024-09-21T07:35:50.467235+0000","last_became_active":"2024-09-21T07:35:26.876890+0000","last_became_peered":"2024-09-21T07:35:26.876890+0000","last_unstale":"2024-09-21T07:35:50.467235+0000","last_undegraded":"2024-09-21T07:35:50.467235+0000","last_fullsized":"2024-09-21T07:35:50.467235+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:40.738327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064543499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":94,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609260+0000","last_change":"2024-09-21T07:34:24.258712+0000","last_active":"2024-09-21T07:35:50.609260+0000","last_peered":"2024-09-21T07:35:50.609260+0000","last_clean":"2024-09-21T07:35:50.609260+0000","last_became_active":"2024-09-21T07:34:23.274692+0000","last_became_peered":"2024-09-21T07:34:23.274692+0000","last_unstale":"2024-09-21T07:35:50.609260+0000","last_undegraded":"2024-09-21T07:35:50.609260+0000","last_fullsized":"2024-09-21T07:35:50.609260+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:43.279454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018051479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466479+0000","last_change":"2024-09-21T07:35:49.489413+0000","last_active":"2024-09-21T07:35:50.466479+0000","last_peered":"2024-09-21T07:35:50.466479+0000","last_clean":"2024-09-21T07:35:50.466479+0000","last_became_active":"2024-09-21T07:35:49.487895+0000","last_became_peered":"2024-09-21T07:35:49.487895+0000","last_unstale":"2024-09-21T07:35:50.466479+0000","last_undegraded":"2024-09-21T07:35:50.466479+0000","last_fullsized":"2024-09-21T07:35:50.466479+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:39.106814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"77'2","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609243+0000","last_change":"2024-09-21T07:35:48.473340+0000","last_active":"2024-09-21T07:35:50.609243+0000","last_peered":"2024-09-21T07:35:50.609243+0000","last_clean":"2024-09-21T07:35:50.609243+0000","last_became_active":"2024-09-21T07:35:48.473063+0000","last_became_peered":"2024-09-21T07:35:48.473063+0000","last_unstale":"2024-09-21T07:35:50.609243+0000","last_undegraded":"2024-09-21T07:35:50.609243+0000","last_fullsized":"2024-09-21T07:35:50.609243+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:30.813114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608660+0000","last_change":"2024-09-21T07:35:27.878020+0000","last_active":"2024-09-21T07:35:50.608660+0000","last_peered":"2024-09-21T07:35:50.608660+0000","last_clean":"2024-09-21T07:35:50.608660+0000","last_became_active":"2024-09-21T07:35:27.877532+0000","last_became_peered":"2024-09-21T07:35:27.877532+0000","last_unstale":"2024-09-21T07:35:50.608660+0000","last_undegraded":"2024-09-21T07:35:50.608660+0000","last_fullsized":"2024-09-21T07:35:50.608660+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:19.756073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":43,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.610846+0000","last_change":"2024-09-21T07:35:28.861049+0000","last_active":"2024-09-21T07:35:50.610846+0000","last_peered":"2024-09-21T07:35:50.610846+0000","last_clean":"2024-09-21T07:35:50.610846+0000","last_became_active":"2024-09-21T07:35:26.876819+0000","last_became_peered":"2024-09-21T07:35:26.876819+0000","last_unstale":"2024-09-21T07:35:50.610846+0000","last_undegraded":"2024-09-21T07:35:50.610846+0000","last_fullsized":"2024-09-21T07:35:50.610846+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:19.633803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055780599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":93,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608368+0000","last_change":"2024-09-21T07:34:25.084849+0000","last_active":"2024-09-21T07:35:50.608368+0000","last_peered":"2024-09-21T07:35:50.608368+0000","last_clean":"2024-09-21T07:35:50.608368+0000","last_became_active":"2024-09-21T07:34:23.277713+0000","last_became_peered":"2024-09-21T07:34:23.277713+0000","last_unstale":"2024-09-21T07:35:50.608368+0000","last_undegraded":"2024-09-21T07:35:50.608368+0000","last_fullsized":"2024-09-21T07:35:50.608368+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:05.811810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059079499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.607623+0000","last_change":"2024-09-21T07:35:49.468874+0000","last_active":"2024-09-21T07:35:50.607623+0000","last_peered":"2024-09-21T07:35:50.607623+0000","last_clean":"2024-09-21T07:35:50.607623+0000","last_became_active":"2024-09-21T07:35:49.468514+0000","last_became_peered":"2024-09-21T07:35:49.468514+0000","last_unstale":"2024-09-21T07:35:50.607623+0000","last_undegraded":"2024-09-21T07:35:50.607623+0000","last_fullsized":"2024-09-21T07:35:50.607623+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:35.690753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.610141+0000","last_change":"2024-09-21T07:35:48.474361+0000","last_active":"2024-09-21T07:35:50.610141+0000","last_peered":"2024-09-21T07:35:50.610141+0000","last_clean":"2024-09-21T07:35:50.610141+0000","last_became_active":"2024-09-21T07:35:48.474032+0000","last_became_peered":"2024-09-21T07:35:48.474032+0000","last_unstale":"2024-09-21T07:35:50.610141+0000","last_undegraded":"2024-09-21T07:35:50.610141+0000","last_fullsized":"2024-09-21T07:35:50.610141+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:16.959573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":36,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466224+0000","last_change":"2024-09-21T07:35:27.884248+0000","last_active":"2024-09-21T07:35:50.466224+0000","last_peered":"2024-09-21T07:35:50.466224+0000","last_clean":"2024-09-21T07:35:50.466224+0000","last_became_active":"2024-09-21T07:35:27.883861+0000","last_became_peered":"2024-09-21T07:35:27.883861+0000","last_unstale":"2024-09-21T07:35:50.466224+0000","last_undegraded":"2024-09-21T07:35:50.466224+0000","last_fullsized":"2024-09-21T07:35:50.466224+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466794+0000","last_change":"2024-09-21T07:35:29.207135+0000","last_active":"2024-09-21T07:35:50.466794+0000","last_peered":"2024-09-21T07:35:50.466794+0000","last_clean":"2024-09-21T07:35:50.466794+0000","last_became_active":"2024-09-21T07:35:26.872787+0000","last_became_peered":"2024-09-21T07:35:26.872787+0000","last_unstale":"2024-09-21T07:35:50.466794+0000","last_undegraded":"2024-09-21T07:35:50.466794+0000","last_fullsized":"2024-09-21T07:35:50.466794+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:27.543217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011157509999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":96,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466956+0000","last_change":"2024-09-21T07:34:24.538610+0000","last_active":"2024-09-21T07:35:50.466956+0000","last_peered":"2024-09-21T07:35:50.466956+0000","last_clean":"2024-09-21T07:35:50.466956+0000","last_became_active":"2024-09-21T07:34:23.259700+0000","last_became_peered":"2024-09-21T07:34:23.259700+0000","last_unstale":"2024-09-21T07:35:50.466956+0000","last_undegraded":"2024-09-21T07:35:50.466956+0000","last_fullsized":"2024-09-21T07:35:50.466956+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:57.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070522399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609610+0000","last_change":"2024-09-21T07:35:49.488287+0000","last_active":"2024-09-21T07:35:50.609610+0000","last_peered":"2024-09-21T07:35:50.609610+0000","last_clean":"2024-09-21T07:35:50.609610+0000","last_became_active":"2024-09-21T07:35:49.487928+0000","last_became_peered":"2024-09-21T07:35:49.487928+0000","last_unstale":"2024-09-21T07:35:50.609610+0000","last_undegraded":"2024-09-21T07:35:50.609610+0000","last_fullsized":"2024-09-21T07:35:50.609610+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:00.876603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"77'3","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609289+0000","last_change":"2024-09-21T07:35:48.478146+0000","last_active":"2024-09-21T07:35:50.609289+0000","last_peered":"2024-09-21T07:35:50.609289+0000","last_clean":"2024-09-21T07:35:50.609289+0000","last_became_active":"2024-09-21T07:35:48.477767+0000","last_became_peered":"2024-09-21T07:35:48.477767+0000","last_unstale":"2024-09-21T07:35:50.609289+0000","last_undegraded":"2024-09-21T07:35:50.609289+0000","last_fullsized":"2024-09-21T07:35:50.609289+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:35:53.405184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466361+0000","last_change":"2024-09-21T07:35:27.868024+0000","last_active":"2024-09-21T07:35:50.466361+0000","last_peered":"2024-09-21T07:35:50.466361+0000","last_clean":"2024-09-21T07:35:50.466361+0000","last_became_active":"2024-09-21T07:35:27.867339+0000","last_became_peered":"2024-09-21T07:35:27.867339+0000","last_unstale":"2024-09-21T07:35:50.466361+0000","last_undegraded":"2024-09-21T07:35:50.466361+0000","last_fullsized":"2024-09-21T07:35:50.466361+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":44,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466059+0000","last_change":"2024-09-21T07:35:28.858505+0000","last_active":"2024-09-21T07:35:50.466059+0000","last_peered":"2024-09-21T07:35:50.466059+0000","last_clean":"2024-09-21T07:35:50.466059+0000","last_became_active":"2024-09-21T07:35:26.877320+0000","last_became_peered":"2024-09-21T07:35:26.877320+0000","last_unstale":"2024-09-21T07:35:50.466059+0000","last_undegraded":"2024-09-21T07:35:50.466059+0000","last_fullsized":"2024-09-21T07:35:50.466059+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:53.672782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244119999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":90,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609974+0000","last_change":"2024-09-21T07:34:24.597597+0000","last_active":"2024-09-21T07:35:50.609974+0000","last_peered":"2024-09-21T07:35:50.609974+0000","last_clean":"2024-09-21T07:35:50.609974+0000","last_became_active":"2024-09-21T07:34:23.635204+0000","last_became_peered":"2024-09-21T07:34:23.635204+0000","last_unstale":"2024-09-21T07:35:50.609974+0000","last_undegraded":"2024-09-21T07:35:50.609974+0000","last_fullsized":"2024-09-21T07:35:50.609974+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:48.644789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081139199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.467242+0000","last_change":"2024-09-21T07:35:49.476856+0000","last_active":"2024-09-21T07:35:50.467242+0000","last_peered":"2024-09-21T07:35:50.467242+0000","last_clean":"2024-09-21T07:35:50.467242+0000","last_became_active":"2024-09-21T07:35:49.476517+0000","last_became_peered":"2024-09-21T07:35:49.476517+0000","last_unstale":"2024-09-21T07:35:50.467242+0000","last_undegraded":"2024-09-21T07:35:50.467242+0000","last_fullsized":"2024-09-21T07:35:50.467242+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:17.962368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"77'6","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.610567+0000","last_change":"2024-09-21T07:35:48.475454+0000","last_active":"2024-09-21T07:35:50.610567+0000","last_peered":"2024-09-21T07:35:50.610567+0000","last_clean":"2024-09-21T07:35:50.610567+0000","last_became_active":"2024-09-21T07:35:48.474874+0000","last_became_peered":"2024-09-21T07:35:48.474874+0000","last_unstale":"2024-09-21T07:35:50.610567+0000","last_undegraded":"2024-09-21T07:35:50.610567+0000","last_fullsized":"2024-09-21T07:35:50.610567+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:20.723211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.465588+0000","last_change":"2024-09-21T07:35:27.877503+0000","last_active":"2024-09-21T07:35:50.465588+0000","last_peered":"2024-09-21T07:35:50.465588+0000","last_clean":"2024-09-21T07:35:50.465588+0000","last_became_active":"2024-09-21T07:35:27.875775+0000","last_became_peered":"2024-09-21T07:35:27.875775+0000","last_unstale":"2024-09-21T07:35:50.465588+0000","last_undegraded":"2024-09-21T07:35:50.465588+0000","last_fullsized":"2024-09-21T07:35:50.465588+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:49.291023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608699+0000","last_change":"2024-09-21T07:35:29.491412+0000","last_active":"2024-09-21T07:35:50.608699+0000","last_peered":"2024-09-21T07:35:50.608699+0000","last_clean":"2024-09-21T07:35:50.608699+0000","last_became_active":"2024-09-21T07:35:26.876404+0000","last_became_peered":"2024-09-21T07:35:26.876404+0000","last_unstale":"2024-09-21T07:35:50.608699+0000","last_undegraded":"2024-09-21T07:35:50.608699+0000","last_fullsized":"2024-09-21T07:35:50.608699+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063918000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":94,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608970+0000","last_change":"2024-09-21T07:34:24.259026+0000","last_active":"2024-09-21T07:35:50.608970+0000","last_peered":"2024-09-21T07:35:50.608970+0000","last_clean":"2024-09-21T07:35:50.608970+0000","last_became_active":"2024-09-21T07:34:23.635456+0000","last_became_peered":"2024-09-21T07:34:23.635456+0000","last_unstale":"2024-09-21T07:35:50.608970+0000","last_undegraded":"2024-09-21T07:35:50.608970+0000","last_fullsized":"2024-09-21T07:35:50.608970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:09.592752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018654419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608299+0000","last_change":"2024-09-21T07:35:49.482878+0000","last_active":"2024-09-21T07:35:50.608299+0000","last_peered":"2024-09-21T07:35:50.608299+0000","last_clean":"2024-09-21T07:35:50.608299+0000","last_became_active":"2024-09-21T07:35:49.482570+0000","last_became_peered":"2024-09-21T07:35:49.482570+0000","last_unstale":"2024-09-21T07:35:50.608299+0000","last_undegraded":"2024-09-21T07:35:50.608299+0000","last_fullsized":"2024-09-21T07:35:50.608299+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:11:10.283242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.467656+0000","last_change":"2024-09-21T07:35:48.472867+0000","last_active":"2024-09-21T07:35:50.467656+0000","last_peered":"2024-09-21T07:35:50.467656+0000","last_clean":"2024-09-21T07:35:50.467656+0000","last_became_active":"2024-09-21T07:35:48.472432+0000","last_became_peered":"2024-09-21T07:35:48.472432+0000","last_unstale":"2024-09-21T07:35:50.467656+0000","last_undegraded":"2024-09-21T07:35:50.467656+0000","last_fullsized":"2024-09-21T07:35:50.467656+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:40:59.551317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.468174+0000","last_change":"2024-09-21T07:35:27.871984+0000","last_active":"2024-09-21T07:35:50.468174+0000","last_peered":"2024-09-21T07:35:50.468174+0000","last_clean":"2024-09-21T07:35:50.468174+0000","last_became_active":"2024-09-21T07:35:27.870336+0000","last_became_peered":"2024-09-21T07:35:27.870336+0000","last_unstale":"2024-09-21T07:35:50.468174+0000","last_undegraded":"2024-09-21T07:35:50.468174+0000","last_fullsized":"2024-09-21T07:35:50.468174+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:57.120479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":43,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608195+0000","last_change":"2024-09-21T07:35:28.860351+0000","last_active":"2024-09-21T07:35:50.608195+0000","last_peered":"2024-09-21T07:35:50.608195+0000","last_clean":"2024-09-21T07:35:50.608195+0000","last_became_active":"2024-09-21T07:35:26.872046+0000","last_became_peered":"2024-09-21T07:35:26.872046+0000","last_unstale":"2024-09-21T07:35:50.608195+0000","last_undegraded":"2024-09-21T07:35:50.608195+0000","last_fullsized":"2024-09-21T07:35:50.608195+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:47.074327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098384399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":94,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608640+0000","last_change":"2024-09-21T07:34:24.260300+0000","last_active":"2024-09-21T07:35:50.608640+0000","last_peered":"2024-09-21T07:35:50.608640+0000","last_clean":"2024-09-21T07:35:50.608640+0000","last_became_active":"2024-09-21T07:34:23.270128+0000","last_became_peered":"2024-09-21T07:34:23.270128+0000","last_unstale":"2024-09-21T07:35:50.608640+0000","last_undegraded":"2024-09-21T07:35:50.608640+0000","last_fullsized":"2024-09-21T07:35:50.608640+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:25.260931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00176651,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609183+0000","last_change":"2024-09-21T07:35:49.470810+0000","last_active":"2024-09-21T07:35:50.609183+0000","last_peered":"2024-09-21T07:35:50.609183+0000","last_clean":"2024-09-21T07:35:50.609183+0000","last_became_active":"2024-09-21T07:35:49.470423+0000","last_became_peered":"2024-09-21T07:35:49.470423+0000","last_unstale":"2024-09-21T07:35:50.609183+0000","last_undegraded":"2024-09-21T07:35:50.609183+0000","last_fullsized":"2024-09-21T07:35:50.609183+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:32.113551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609724+0000","last_change":"2024-09-21T07:35:48.464501+0000","last_active":"2024-09-21T07:35:50.609724+0000","last_peered":"2024-09-21T07:35:50.609724+0000","last_clean":"2024-09-21T07:35:50.609724+0000","last_became_active":"2024-09-21T07:35:48.464249+0000","last_became_peered":"2024-09-21T07:35:48.464249+0000","last_unstale":"2024-09-21T07:35:50.609724+0000","last_undegraded":"2024-09-21T07:35:50.609724+0000","last_fullsized":"2024-09-21T07:35:50.609724+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:49.239670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.468310+0000","last_change":"2024-09-21T07:35:27.877861+0000","last_active":"2024-09-21T07:35:50.468310+0000","last_peered":"2024-09-21T07:35:50.468310+0000","last_clean":"2024-09-21T07:35:50.468310+0000","last_became_active":"2024-09-21T07:35:27.877434+0000","last_became_peered":"2024-09-21T07:35:27.877434+0000","last_unstale":"2024-09-21T07:35:50.468310+0000","last_undegraded":"2024-09-21T07:35:50.468310+0000","last_fullsized":"2024-09-21T07:35:50.468310+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608707+0000","last_change":"2024-09-21T07:35:28.862954+0000","last_active":"2024-09-21T07:35:50.608707+0000","last_peered":"2024-09-21T07:35:50.608707+0000","last_clean":"2024-09-21T07:35:50.608707+0000","last_became_active":"2024-09-21T07:35:26.870613+0000","last_became_peered":"2024-09-21T07:35:26.870613+0000","last_unstale":"2024-09-21T07:35:50.608707+0000","last_undegraded":"2024-09-21T07:35:50.608707+0000","last_fullsized":"2024-09-21T07:35:50.608707+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:06.901043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059842100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":94,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466911+0000","last_change":"2024-09-21T07:34:25.028904+0000","last_active":"2024-09-21T07:35:50.466911+0000","last_peered":"2024-09-21T07:35:50.466911+0000","last_clean":"2024-09-21T07:35:50.466911+0000","last_became_active":"2024-09-21T07:34:23.260942+0000","last_became_peered":"2024-09-21T07:34:23.260942+0000","last_unstale":"2024-09-21T07:35:50.466911+0000","last_undegraded":"2024-09-21T07:35:50.466911+0000","last_fullsized":"2024-09-21T07:35:50.466911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:22.709328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060553899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509477,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32704,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1544192,"data_stored":1024165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509479,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":962560,"data_stored":435957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509477,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32112,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339728,"statfs":{"total":96636764160,"available":96603881472,"internally_reserved":0,"allocated":942080,"data_stored":433741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509478,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32112,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339728,"statfs":{"total":96636764160,"available":96603881472,"internally_reserved":0,"allocated":942080,"data_stored":433741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509479,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":946176,"data_stored":433811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509479,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32720,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339120,"statfs":{"total":96636764160,"available":96603258880,"internally_reserved":0,"allocated":1560576,"data_stored":1026427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542182,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32124,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339716,"statfs":{"total":96636764160,"available":96603869184,"internally_reserved":0,"allocated":950272,"data_stored":433901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542181,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":942080,"data_stored":433741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:35:53.956 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:35:54.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.213+0000 7fcf70a00640 1 -- 172.21.15.39:0/1148606297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf6c0afec0 msgr2=0x7fcf6c153480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:54.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.213+0000 7fcf70a00640 1 --2- 172.21.15.39:0/1148606297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf6c0afec0 0x7fcf6c153480 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fcf64005fd0 tx=0x7fcf6402a8b0 comp rx=0 tx=0).stop 2024-09-21T07:35:54.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.213+0000 7fcf70a00640 1 -- 172.21.15.39:0/1148606297 shutdown_connections 2024-09-21T07:35:54.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.213+0000 7fcf70a00640 1 --2- 172.21.15.39:0/1148606297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf6c0afec0 0x7fcf6c153480 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:54.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.213+0000 7fcf70a00640 1 -- 172.21.15.39:0/1148606297 >> 172.21.15.39:0/1148606297 conn(0x7fcf6c0d0830 msgr2=0x7fcf6c0d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:54.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.213+0000 7fcf70a00640 1 -- 172.21.15.39:0/1148606297 shutdown_connections 2024-09-21T07:35:54.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.213+0000 7fcf70a00640 1 -- 172.21.15.39:0/1148606297 wait complete. 2024-09-21T07:35:54.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.216+0000 7fcf70a00640 1 Processor -- start 2024-09-21T07:35:54.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.216+0000 7fcf70a00640 1 -- start start 2024-09-21T07:35:54.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.217+0000 7fcf70a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf6c0afec0 0x7fcf6c0b1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:54.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.217+0000 7fcf70a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcf6c0b3b10 0x7fcf6c0b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:54.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.217+0000 7fcf70a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf6c1f5cf0 0x7fcf6c14cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:54.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.217+0000 7fcf70a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6c06e130 con 0x7fcf6c1f5cf0 2024-09-21T07:35:54.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.217+0000 7fcf70a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6c06dfb0 con 0x7fcf6c0afec0 2024-09-21T07:35:54.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.217+0000 7fcf70a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fcf6c06e2b0 con 0x7fcf6c0b3b10 2024-09-21T07:35:54.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.217+0000 7fcf6b200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf6c1f5cf0 0x7fcf6c14cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:54.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.218+0000 7fcf6b200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf6c1f5cf0 0x7fcf6c14cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45550/0 (socket says 172.21.15.39:45550) 2024-09-21T07:35:54.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.218+0000 7fcf6b200640 1 -- 172.21.15.39:0/1832046356 learned_addr learned my addr 172.21.15.39:0/1832046356 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:54.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.218+0000 7fcf63e00640 1 --2- 172.21.15.39:0/1832046356 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcf6c0b3b10 0x7fcf6c0b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:54.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.218+0000 7fcf6a800640 1 --2- 172.21.15.39:0/1832046356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf6c0afec0 0x7fcf6c0b1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:54.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.218+0000 7fcf6b200640 1 -- 172.21.15.39:0/1832046356 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcf6c0b3b10 msgr2=0x7fcf6c0b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:54.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.218+0000 7fcf6b200640 1 --2- 172.21.15.39:0/1832046356 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcf6c0b3b10 0x7fcf6c0b1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:54.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.218+0000 7fcf6b200640 1 -- 172.21.15.39:0/1832046356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf6c0afec0 msgr2=0x7fcf6c0b1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:54.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.218+0000 7fcf6b200640 1 --2- 172.21.15.39:0/1832046356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf6c0afec0 0x7fcf6c0b1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:54.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.218+0000 7fcf6b200640 1 -- 172.21.15.39:0/1832046356 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf6c14ecb0 con 0x7fcf6c1f5cf0 2024-09-21T07:35:54.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.218+0000 7fcf6b200640 1 --2- 172.21.15.39:0/1832046356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf6c1f5cf0 0x7fcf6c14cbe0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fcf64009cd0 tx=0x7fcf64003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:54.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.219+0000 7fcf62a00640 1 -- 172.21.15.39:0/1832046356 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf64031b90 con 0x7fcf6c1f5cf0 2024-09-21T07:35:54.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.219+0000 7fcf62a00640 1 -- 172.21.15.39:0/1832046356 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf64031e20 con 0x7fcf6c1f5cf0 2024-09-21T07:35:54.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.219+0000 7fcf62a00640 1 -- 172.21.15.39:0/1832046356 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf640332f0 con 0x7fcf6c1f5cf0 2024-09-21T07:35:54.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.219+0000 7fcf70a00640 1 -- 172.21.15.39:0/1832046356 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf6c1f5210 con 0x7fcf6c1f5cf0 2024-09-21T07:35:54.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.219+0000 7fcf70a00640 1 -- 172.21.15.39:0/1832046356 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf6c1f57d0 con 0x7fcf6c1f5cf0 2024-09-21T07:35:54.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.220+0000 7fcf70a00640 1 -- 172.21.15.39:0/1832046356 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf6c14ea20 con 0x7fcf6c1f5cf0 2024-09-21T07:35:54.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.221+0000 7fcf62a00640 1 -- 172.21.15.39:0/1832046356 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcf64031660 con 0x7fcf6c1f5cf0 2024-09-21T07:35:54.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.222+0000 7fcf62a00640 1 --2- 172.21.15.39:0/1832046356 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcf48078bc0 0x7fcf4807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:54.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.222+0000 7fcf62a00640 1 -- 172.21.15.39:0/1832046356 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6772+0+0 (secure 0 0 0) 0x7fcf640bc690 con 0x7fcf6c1f5cf0 2024-09-21T07:35:54.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.222+0000 7fcf6a800640 1 --2- 172.21.15.39:0/1832046356 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcf48078bc0 0x7fcf4807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:54.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.223+0000 7fcf6a800640 1 --2- 172.21.15.39:0/1832046356 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcf48078bc0 0x7fcf4807b0d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c001150 tx=0x7fcf5c0010e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:54.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.226+0000 7fcf62a00640 1 -- 172.21.15.39:0/1832046356 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcf6403f030 con 0x7fcf6c1f5cf0 2024-09-21T07:35:54.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.421+0000 7fcf70a00640 1 -- 172.21.15.39:0/1832046356 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcf6c153480 con 0x7fcf48078bc0 2024-09-21T07:35:54.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.431+0000 7fcf62a00640 1 -- 172.21.15.39:0/1832046356 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+494623 (secure 0 0 0) 0x7fcf6c153480 con 0x7fcf48078bc0 2024-09-21T07:35:54.438 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:35:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:35:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.437+0000 7fcf70a00640 1 -- 172.21.15.39:0/1832046356 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcf48078bc0 msgr2=0x7fcf4807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.437+0000 7fcf70a00640 1 --2- 172.21.15.39:0/1832046356 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcf48078bc0 0x7fcf4807b0d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c001150 tx=0x7fcf5c0010e0 comp rx=0 tx=0).stop 2024-09-21T07:35:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.437+0000 7fcf70a00640 1 -- 172.21.15.39:0/1832046356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf6c1f5cf0 msgr2=0x7fcf6c14cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.437+0000 7fcf70a00640 1 --2- 172.21.15.39:0/1832046356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf6c1f5cf0 0x7fcf6c14cbe0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fcf64009cd0 tx=0x7fcf64003ba0 comp rx=0 tx=0).stop 2024-09-21T07:35:54.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.437+0000 7fcf70a00640 1 -- 172.21.15.39:0/1832046356 shutdown_connections 2024-09-21T07:35:54.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.437+0000 7fcf70a00640 1 --2- 172.21.15.39:0/1832046356 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcf48078bc0 0x7fcf4807b0d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:54.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.437+0000 7fcf70a00640 1 --2- 172.21.15.39:0/1832046356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf6c1f5cf0 0x7fcf6c14cbe0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:54.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.437+0000 7fcf70a00640 1 --2- 172.21.15.39:0/1832046356 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcf6c0b3b10 0x7fcf6c0b1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:54.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.437+0000 7fcf70a00640 1 --2- 172.21.15.39:0/1832046356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf6c0afec0 0x7fcf6c0b1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:54.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.437+0000 7fcf70a00640 1 -- 172.21.15.39:0/1832046356 >> 172.21.15.39:0/1832046356 conn(0x7fcf6c0d0830 msgr2=0x7fcf6c0b4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:54.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.437+0000 7fcf70a00640 1 -- 172.21.15.39:0/1832046356 shutdown_connections 2024-09-21T07:35:54.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.437+0000 7fcf70a00640 1 -- 172.21.15.39:0/1832046356 wait complete. 2024-09-21T07:35:54.461 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":159,"stamp":"2024-09-21T07:35:53.576056+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":356,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258136,"kb_used_data":8584,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716584,"statfs":{"total":773094113280,"available":772829782016,"internally_reserved":0,"allocated":8790016,"data_stored":4655484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001823"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608939+0000","last_change":"2024-09-21T07:35:49.477604+0000","last_active":"2024-09-21T07:35:50.608939+0000","last_peered":"2024-09-21T07:35:50.608939+0000","last_clean":"2024-09-21T07:35:50.608939+0000","last_became_active":"2024-09-21T07:35:49.477253+0000","last_became_peered":"2024-09-21T07:35:49.477253+0000","last_unstale":"2024-09-21T07:35:50.608939+0000","last_undegraded":"2024-09-21T07:35:50.608939+0000","last_fullsized":"2024-09-21T07:35:50.608939+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:20.996757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.467198+0000","last_change":"2024-09-21T07:35:49.485146+0000","last_active":"2024-09-21T07:35:50.467198+0000","last_peered":"2024-09-21T07:35:50.467198+0000","last_clean":"2024-09-21T07:35:50.467198+0000","last_became_active":"2024-09-21T07:35:49.484826+0000","last_became_peered":"2024-09-21T07:35:49.484826+0000","last_unstale":"2024-09-21T07:35:50.467198+0000","last_undegraded":"2024-09-21T07:35:50.467198+0000","last_fullsized":"2024-09-21T07:35:50.467198+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:57.840668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608714+0000","last_change":"2024-09-21T07:35:49.472086+0000","last_active":"2024-09-21T07:35:50.608714+0000","last_peered":"2024-09-21T07:35:50.608714+0000","last_clean":"2024-09-21T07:35:50.608714+0000","last_became_active":"2024-09-21T07:35:49.471746+0000","last_became_peered":"2024-09-21T07:35:49.471746+0000","last_unstale":"2024-09-21T07:35:50.608714+0000","last_undegraded":"2024-09-21T07:35:50.608714+0000","last_fullsized":"2024-09-21T07:35:50.608714+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:52:19.381733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466009+0000","last_change":"2024-09-21T07:35:49.477135+0000","last_active":"2024-09-21T07:35:50.466009+0000","last_peered":"2024-09-21T07:35:50.466009+0000","last_clean":"2024-09-21T07:35:50.466009+0000","last_became_active":"2024-09-21T07:35:49.476707+0000","last_became_peered":"2024-09-21T07:35:49.476707+0000","last_unstale":"2024-09-21T07:35:50.466009+0000","last_undegraded":"2024-09-21T07:35:50.466009+0000","last_fullsized":"2024-09-21T07:35:50.466009+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:29.846901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.607826+0000","last_change":"2024-09-21T07:35:49.475067+0000","last_active":"2024-09-21T07:35:50.607826+0000","last_peered":"2024-09-21T07:35:50.607826+0000","last_clean":"2024-09-21T07:35:50.607826+0000","last_became_active":"2024-09-21T07:35:49.474751+0000","last_became_peered":"2024-09-21T07:35:49.474751+0000","last_unstale":"2024-09-21T07:35:50.607826+0000","last_undegraded":"2024-09-21T07:35:50.607826+0000","last_fullsized":"2024-09-21T07:35:50.607826+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:14.946214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609949+0000","last_change":"2024-09-21T07:35:49.478940+0000","last_active":"2024-09-21T07:35:50.609949+0000","last_peered":"2024-09-21T07:35:50.609949+0000","last_clean":"2024-09-21T07:35:50.609949+0000","last_became_active":"2024-09-21T07:35:49.478560+0000","last_became_peered":"2024-09-21T07:35:49.478560+0000","last_unstale":"2024-09-21T07:35:50.609949+0000","last_undegraded":"2024-09-21T07:35:50.609949+0000","last_fullsized":"2024-09-21T07:35:50.609949+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:38.592795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608211+0000","last_change":"2024-09-21T07:35:49.494342+0000","last_active":"2024-09-21T07:35:50.608211+0000","last_peered":"2024-09-21T07:35:50.608211+0000","last_clean":"2024-09-21T07:35:50.608211+0000","last_became_active":"2024-09-21T07:35:49.494034+0000","last_became_peered":"2024-09-21T07:35:49.494034+0000","last_unstale":"2024-09-21T07:35:50.608211+0000","last_undegraded":"2024-09-21T07:35:50.608211+0000","last_fullsized":"2024-09-21T07:35:50.608211+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:30.939322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.465736+0000","last_change":"2024-09-21T07:35:49.488554+0000","last_active":"2024-09-21T07:35:50.465736+0000","last_peered":"2024-09-21T07:35:50.465736+0000","last_clean":"2024-09-21T07:35:50.465736+0000","last_became_active":"2024-09-21T07:35:49.486889+0000","last_became_peered":"2024-09-21T07:35:49.486889+0000","last_unstale":"2024-09-21T07:35:50.465736+0000","last_undegraded":"2024-09-21T07:35:50.465736+0000","last_fullsized":"2024-09-21T07:35:50.465736+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:26.004814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608401+0000","last_change":"2024-09-21T07:35:49.462927+0000","last_active":"2024-09-21T07:35:50.608401+0000","last_peered":"2024-09-21T07:35:50.608401+0000","last_clean":"2024-09-21T07:35:50.608401+0000","last_became_active":"2024-09-21T07:35:49.462406+0000","last_became_peered":"2024-09-21T07:35:49.462406+0000","last_unstale":"2024-09-21T07:35:50.608401+0000","last_undegraded":"2024-09-21T07:35:50.608401+0000","last_fullsized":"2024-09-21T07:35:50.608401+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:06:21.675282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609530+0000","last_change":"2024-09-21T07:35:49.476877+0000","last_active":"2024-09-21T07:35:50.609530+0000","last_peered":"2024-09-21T07:35:50.609530+0000","last_clean":"2024-09-21T07:35:50.609530+0000","last_became_active":"2024-09-21T07:35:49.476569+0000","last_became_peered":"2024-09-21T07:35:49.476569+0000","last_unstale":"2024-09-21T07:35:50.609530+0000","last_undegraded":"2024-09-21T07:35:50.609530+0000","last_fullsized":"2024-09-21T07:35:50.609530+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:42:18.942020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609064+0000","last_change":"2024-09-21T07:35:49.456055+0000","last_active":"2024-09-21T07:35:50.609064+0000","last_peered":"2024-09-21T07:35:50.609064+0000","last_clean":"2024-09-21T07:35:50.609064+0000","last_became_active":"2024-09-21T07:35:49.455683+0000","last_became_peered":"2024-09-21T07:35:49.455683+0000","last_unstale":"2024-09-21T07:35:50.609064+0000","last_undegraded":"2024-09-21T07:35:50.609064+0000","last_fullsized":"2024-09-21T07:35:50.609064+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:00.733107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609353+0000","last_change":"2024-09-21T07:35:49.472334+0000","last_active":"2024-09-21T07:35:50.609353+0000","last_peered":"2024-09-21T07:35:50.609353+0000","last_clean":"2024-09-21T07:35:50.609353+0000","last_became_active":"2024-09-21T07:35:49.471427+0000","last_became_peered":"2024-09-21T07:35:49.471427+0000","last_unstale":"2024-09-21T07:35:50.609353+0000","last_undegraded":"2024-09-21T07:35:50.609353+0000","last_fullsized":"2024-09-21T07:35:50.609353+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:23.802292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.465944+0000","last_change":"2024-09-21T07:35:49.492926+0000","last_active":"2024-09-21T07:35:50.465944+0000","last_peered":"2024-09-21T07:35:50.465944+0000","last_clean":"2024-09-21T07:35:50.465944+0000","last_became_active":"2024-09-21T07:35:49.492498+0000","last_became_peered":"2024-09-21T07:35:49.492498+0000","last_unstale":"2024-09-21T07:35:50.465944+0000","last_undegraded":"2024-09-21T07:35:50.465944+0000","last_fullsized":"2024-09-21T07:35:50.465944+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:26.372963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.607685+0000","last_change":"2024-09-21T07:35:49.479630+0000","last_active":"2024-09-21T07:35:50.607685+0000","last_peered":"2024-09-21T07:35:50.607685+0000","last_clean":"2024-09-21T07:35:50.607685+0000","last_became_active":"2024-09-21T07:35:49.479297+0000","last_became_peered":"2024-09-21T07:35:49.479297+0000","last_unstale":"2024-09-21T07:35:50.607685+0000","last_undegraded":"2024-09-21T07:35:50.607685+0000","last_fullsized":"2024-09-21T07:35:50.607685+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:20.600428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466973+0000","last_change":"2024-09-21T07:35:49.491503+0000","last_active":"2024-09-21T07:35:50.466973+0000","last_peered":"2024-09-21T07:35:50.466973+0000","last_clean":"2024-09-21T07:35:50.466973+0000","last_became_active":"2024-09-21T07:35:49.491081+0000","last_became_peered":"2024-09-21T07:35:49.491081+0000","last_unstale":"2024-09-21T07:35:50.466973+0000","last_undegraded":"2024-09-21T07:35:50.466973+0000","last_fullsized":"2024-09-21T07:35:50.466973+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:50:14.299542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.607883+0000","last_change":"2024-09-21T07:35:49.477963+0000","last_active":"2024-09-21T07:35:50.607883+0000","last_peered":"2024-09-21T07:35:50.607883+0000","last_clean":"2024-09-21T07:35:50.607883+0000","last_became_active":"2024-09-21T07:35:49.476006+0000","last_became_peered":"2024-09-21T07:35:49.476006+0000","last_unstale":"2024-09-21T07:35:50.607883+0000","last_undegraded":"2024-09-21T07:35:50.607883+0000","last_fullsized":"2024-09-21T07:35:50.607883+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:51:44.922902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.611405+0000","last_change":"2024-09-21T07:35:49.492438+0000","last_active":"2024-09-21T07:35:50.611405+0000","last_peered":"2024-09-21T07:35:50.611405+0000","last_clean":"2024-09-21T07:35:50.611405+0000","last_became_active":"2024-09-21T07:35:49.489827+0000","last_became_peered":"2024-09-21T07:35:49.489827+0000","last_unstale":"2024-09-21T07:35:50.611405+0000","last_undegraded":"2024-09-21T07:35:50.611405+0000","last_fullsized":"2024-09-21T07:35:50.611405+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:10.214526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609393+0000","last_change":"2024-09-21T07:35:49.482017+0000","last_active":"2024-09-21T07:35:50.609393+0000","last_peered":"2024-09-21T07:35:50.609393+0000","last_clean":"2024-09-21T07:35:50.609393+0000","last_became_active":"2024-09-21T07:35:49.481736+0000","last_became_peered":"2024-09-21T07:35:49.481736+0000","last_unstale":"2024-09-21T07:35:50.609393+0000","last_undegraded":"2024-09-21T07:35:50.609393+0000","last_fullsized":"2024-09-21T07:35:50.609393+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:25:30.270321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609142+0000","last_change":"2024-09-21T07:35:49.477461+0000","last_active":"2024-09-21T07:35:50.609142+0000","last_peered":"2024-09-21T07:35:50.609142+0000","last_clean":"2024-09-21T07:35:50.609142+0000","last_became_active":"2024-09-21T07:35:49.477152+0000","last_became_peered":"2024-09-21T07:35:49.477152+0000","last_unstale":"2024-09-21T07:35:50.609142+0000","last_undegraded":"2024-09-21T07:35:50.609142+0000","last_fullsized":"2024-09-21T07:35:50.609142+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:20:11.777136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466526+0000","last_change":"2024-09-21T07:35:49.491436+0000","last_active":"2024-09-21T07:35:50.466526+0000","last_peered":"2024-09-21T07:35:50.466526+0000","last_clean":"2024-09-21T07:35:50.466526+0000","last_became_active":"2024-09-21T07:35:49.491048+0000","last_became_peered":"2024-09-21T07:35:49.491048+0000","last_unstale":"2024-09-21T07:35:50.466526+0000","last_undegraded":"2024-09-21T07:35:50.466526+0000","last_fullsized":"2024-09-21T07:35:50.466526+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:47.032420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608134+0000","last_change":"2024-09-21T07:35:49.468432+0000","last_active":"2024-09-21T07:35:50.608134+0000","last_peered":"2024-09-21T07:35:50.608134+0000","last_clean":"2024-09-21T07:35:50.608134+0000","last_became_active":"2024-09-21T07:35:49.467983+0000","last_became_peered":"2024-09-21T07:35:49.467983+0000","last_unstale":"2024-09-21T07:35:50.608134+0000","last_undegraded":"2024-09-21T07:35:50.608134+0000","last_fullsized":"2024-09-21T07:35:50.608134+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:30:11.845940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.607688+0000","last_change":"2024-09-21T07:35:49.480387+0000","last_active":"2024-09-21T07:35:50.607688+0000","last_peered":"2024-09-21T07:35:50.607688+0000","last_clean":"2024-09-21T07:35:50.607688+0000","last_became_active":"2024-09-21T07:35:49.478378+0000","last_became_peered":"2024-09-21T07:35:49.478378+0000","last_unstale":"2024-09-21T07:35:50.607688+0000","last_undegraded":"2024-09-21T07:35:50.607688+0000","last_fullsized":"2024-09-21T07:35:50.607688+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:46.129473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.465810+0000","last_change":"2024-09-21T07:35:49.476827+0000","last_active":"2024-09-21T07:35:50.465810+0000","last_peered":"2024-09-21T07:35:50.465810+0000","last_clean":"2024-09-21T07:35:50.465810+0000","last_became_active":"2024-09-21T07:35:49.476486+0000","last_became_peered":"2024-09-21T07:35:49.476486+0000","last_unstale":"2024-09-21T07:35:50.465810+0000","last_undegraded":"2024-09-21T07:35:50.465810+0000","last_fullsized":"2024-09-21T07:35:50.465810+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:29.495534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.607784+0000","last_change":"2024-09-21T07:35:49.473330+0000","last_active":"2024-09-21T07:35:50.607784+0000","last_peered":"2024-09-21T07:35:50.607784+0000","last_clean":"2024-09-21T07:35:50.607784+0000","last_became_active":"2024-09-21T07:35:49.473022+0000","last_became_peered":"2024-09-21T07:35:49.473022+0000","last_unstale":"2024-09-21T07:35:50.607784+0000","last_undegraded":"2024-09-21T07:35:50.607784+0000","last_fullsized":"2024-09-21T07:35:50.607784+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:20:09.706907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.467549+0000","last_change":"2024-09-21T07:35:49.475917+0000","last_active":"2024-09-21T07:35:50.467549+0000","last_peered":"2024-09-21T07:35:50.467549+0000","last_clean":"2024-09-21T07:35:50.467549+0000","last_became_active":"2024-09-21T07:35:49.475559+0000","last_became_peered":"2024-09-21T07:35:49.475559+0000","last_unstale":"2024-09-21T07:35:50.467549+0000","last_undegraded":"2024-09-21T07:35:50.467549+0000","last_fullsized":"2024-09-21T07:35:50.467549+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:57:06.538812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608685+0000","last_change":"2024-09-21T07:35:49.478827+0000","last_active":"2024-09-21T07:35:50.608685+0000","last_peered":"2024-09-21T07:35:50.608685+0000","last_clean":"2024-09-21T07:35:50.608685+0000","last_became_active":"2024-09-21T07:35:49.477964+0000","last_became_peered":"2024-09-21T07:35:49.477964+0000","last_unstale":"2024-09-21T07:35:50.608685+0000","last_undegraded":"2024-09-21T07:35:50.608685+0000","last_fullsized":"2024-09-21T07:35:50.608685+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:19.502430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.610143+0000","last_change":"2024-09-21T07:35:49.474821+0000","last_active":"2024-09-21T07:35:50.610143+0000","last_peered":"2024-09-21T07:35:50.610143+0000","last_clean":"2024-09-21T07:35:50.610143+0000","last_became_active":"2024-09-21T07:35:49.474546+0000","last_became_peered":"2024-09-21T07:35:49.474546+0000","last_unstale":"2024-09-21T07:35:50.610143+0000","last_undegraded":"2024-09-21T07:35:50.610143+0000","last_fullsized":"2024-09-21T07:35:50.610143+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:56:26.806827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466552+0000","last_change":"2024-09-21T07:35:49.476604+0000","last_active":"2024-09-21T07:35:50.466552+0000","last_peered":"2024-09-21T07:35:50.466552+0000","last_clean":"2024-09-21T07:35:50.466552+0000","last_became_active":"2024-09-21T07:35:49.476270+0000","last_became_peered":"2024-09-21T07:35:49.476270+0000","last_unstale":"2024-09-21T07:35:50.466552+0000","last_undegraded":"2024-09-21T07:35:50.466552+0000","last_fullsized":"2024-09-21T07:35:50.466552+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:25.055048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609192+0000","last_change":"2024-09-21T07:35:49.482117+0000","last_active":"2024-09-21T07:35:50.609192+0000","last_peered":"2024-09-21T07:35:50.609192+0000","last_clean":"2024-09-21T07:35:50.609192+0000","last_became_active":"2024-09-21T07:35:49.479799+0000","last_became_peered":"2024-09-21T07:35:49.479799+0000","last_unstale":"2024-09-21T07:35:50.609192+0000","last_undegraded":"2024-09-21T07:35:50.609192+0000","last_fullsized":"2024-09-21T07:35:50.609192+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:50:24.393154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608764+0000","last_change":"2024-09-21T07:35:49.463449+0000","last_active":"2024-09-21T07:35:50.608764+0000","last_peered":"2024-09-21T07:35:50.608764+0000","last_clean":"2024-09-21T07:35:50.608764+0000","last_became_active":"2024-09-21T07:35:49.462439+0000","last_became_peered":"2024-09-21T07:35:49.462439+0000","last_unstale":"2024-09-21T07:35:50.608764+0000","last_undegraded":"2024-09-21T07:35:50.608764+0000","last_fullsized":"2024-09-21T07:35:50.608764+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:08:07.440136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.465782+0000","last_change":"2024-09-21T07:35:49.477637+0000","last_active":"2024-09-21T07:35:50.465782+0000","last_peered":"2024-09-21T07:35:50.465782+0000","last_clean":"2024-09-21T07:35:50.465782+0000","last_became_active":"2024-09-21T07:35:49.477370+0000","last_became_peered":"2024-09-21T07:35:49.477370+0000","last_unstale":"2024-09-21T07:35:50.465782+0000","last_undegraded":"2024-09-21T07:35:50.465782+0000","last_fullsized":"2024-09-21T07:35:50.465782+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:20.497297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608106+0000","last_change":"2024-09-21T07:35:49.475935+0000","last_active":"2024-09-21T07:35:50.608106+0000","last_peered":"2024-09-21T07:35:50.608106+0000","last_clean":"2024-09-21T07:35:50.608106+0000","last_became_active":"2024-09-21T07:35:49.475613+0000","last_became_peered":"2024-09-21T07:35:49.475613+0000","last_unstale":"2024-09-21T07:35:50.608106+0000","last_undegraded":"2024-09-21T07:35:50.608106+0000","last_fullsized":"2024-09-21T07:35:50.608106+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:24:26.502711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609801+0000","last_change":"2024-09-21T07:35:49.494330+0000","last_active":"2024-09-21T07:35:50.609801+0000","last_peered":"2024-09-21T07:35:50.609801+0000","last_clean":"2024-09-21T07:35:50.609801+0000","last_became_active":"2024-09-21T07:35:49.494019+0000","last_became_peered":"2024-09-21T07:35:49.494019+0000","last_unstale":"2024-09-21T07:35:50.609801+0000","last_undegraded":"2024-09-21T07:35:50.609801+0000","last_fullsized":"2024-09-21T07:35:50.609801+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:18:47.217069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466597+0000","last_change":"2024-09-21T07:35:48.469480+0000","last_active":"2024-09-21T07:35:50.466597+0000","last_peered":"2024-09-21T07:35:50.466597+0000","last_clean":"2024-09-21T07:35:50.466597+0000","last_became_active":"2024-09-21T07:35:48.468500+0000","last_became_peered":"2024-09-21T07:35:48.468500+0000","last_unstale":"2024-09-21T07:35:50.466597+0000","last_undegraded":"2024-09-21T07:35:50.466597+0000","last_fullsized":"2024-09-21T07:35:50.466597+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:25.184983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609270+0000","last_change":"2024-09-21T07:35:27.879605+0000","last_active":"2024-09-21T07:35:50.609270+0000","last_peered":"2024-09-21T07:35:50.609270+0000","last_clean":"2024-09-21T07:35:50.609270+0000","last_became_active":"2024-09-21T07:35:27.879274+0000","last_became_peered":"2024-09-21T07:35:27.879274+0000","last_unstale":"2024-09-21T07:35:50.609270+0000","last_undegraded":"2024-09-21T07:35:50.609270+0000","last_fullsized":"2024-09-21T07:35:50.609270+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:24.241636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608495+0000","last_change":"2024-09-21T07:35:29.491336+0000","last_active":"2024-09-21T07:35:50.608495+0000","last_peered":"2024-09-21T07:35:50.608495+0000","last_clean":"2024-09-21T07:35:50.608495+0000","last_became_active":"2024-09-21T07:35:26.866484+0000","last_became_peered":"2024-09-21T07:35:26.866484+0000","last_unstale":"2024-09-21T07:35:50.608495+0000","last_undegraded":"2024-09-21T07:35:50.608495+0000","last_fullsized":"2024-09-21T07:35:50.608495+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:10.449665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076010100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":219,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608202+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:35:50.608202+0000","last_peered":"2024-09-21T07:35:50.608202+0000","last_clean":"2024-09-21T07:35:50.608202+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:35:50.608202+0000","last_undegraded":"2024-09-21T07:35:50.608202+0000","last_fullsized":"2024-09-21T07:35:50.608202+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":96,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.467873+0000","last_change":"2024-09-21T07:34:24.539248+0000","last_active":"2024-09-21T07:35:50.467873+0000","last_peered":"2024-09-21T07:35:50.467873+0000","last_clean":"2024-09-21T07:35:50.467873+0000","last_became_active":"2024-09-21T07:34:23.273465+0000","last_became_peered":"2024-09-21T07:34:23.273465+0000","last_unstale":"2024-09-21T07:35:50.467873+0000","last_undegraded":"2024-09-21T07:35:50.467873+0000","last_fullsized":"2024-09-21T07:35:50.467873+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:55.517452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060627700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466091+0000","last_change":"2024-09-21T07:35:49.482924+0000","last_active":"2024-09-21T07:35:50.466091+0000","last_peered":"2024-09-21T07:35:50.466091+0000","last_clean":"2024-09-21T07:35:50.466091+0000","last_became_active":"2024-09-21T07:35:49.478417+0000","last_became_peered":"2024-09-21T07:35:49.478417+0000","last_unstale":"2024-09-21T07:35:50.466091+0000","last_undegraded":"2024-09-21T07:35:50.466091+0000","last_fullsized":"2024-09-21T07:35:50.466091+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:04.130257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.465460+0000","last_change":"2024-09-21T07:35:48.477741+0000","last_active":"2024-09-21T07:35:50.465460+0000","last_peered":"2024-09-21T07:35:50.465460+0000","last_clean":"2024-09-21T07:35:50.465460+0000","last_became_active":"2024-09-21T07:35:48.477356+0000","last_became_peered":"2024-09-21T07:35:48.477356+0000","last_unstale":"2024-09-21T07:35:50.465460+0000","last_undegraded":"2024-09-21T07:35:50.465460+0000","last_fullsized":"2024-09-21T07:35:50.465460+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:20:00.655949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.607759+0000","last_change":"2024-09-21T07:35:27.864492+0000","last_active":"2024-09-21T07:35:50.607759+0000","last_peered":"2024-09-21T07:35:50.607759+0000","last_clean":"2024-09-21T07:35:50.607759+0000","last_became_active":"2024-09-21T07:35:27.864176+0000","last_became_peered":"2024-09-21T07:35:27.864176+0000","last_unstale":"2024-09-21T07:35:50.607759+0000","last_undegraded":"2024-09-21T07:35:50.607759+0000","last_fullsized":"2024-09-21T07:35:50.607759+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:05.153467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609998+0000","last_change":"2024-09-21T07:35:29.493678+0000","last_active":"2024-09-21T07:35:50.609998+0000","last_peered":"2024-09-21T07:35:50.609998+0000","last_clean":"2024-09-21T07:35:50.609998+0000","last_became_active":"2024-09-21T07:35:26.870749+0000","last_became_peered":"2024-09-21T07:35:26.870749+0000","last_unstale":"2024-09-21T07:35:50.609998+0000","last_undegraded":"2024-09-21T07:35:50.609998+0000","last_fullsized":"2024-09-21T07:35:50.609998+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:55.367877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010159570000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":90,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608887+0000","last_change":"2024-09-21T07:34:24.597274+0000","last_active":"2024-09-21T07:35:50.608887+0000","last_peered":"2024-09-21T07:35:50.608887+0000","last_clean":"2024-09-21T07:35:50.608887+0000","last_became_active":"2024-09-21T07:34:23.279560+0000","last_became_peered":"2024-09-21T07:34:23.279560+0000","last_unstale":"2024-09-21T07:35:50.608887+0000","last_undegraded":"2024-09-21T07:35:50.608887+0000","last_fullsized":"2024-09-21T07:35:50.608887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:15.845265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059699799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.610255+0000","last_change":"2024-09-21T07:35:49.483638+0000","last_active":"2024-09-21T07:35:50.610255+0000","last_peered":"2024-09-21T07:35:50.610255+0000","last_clean":"2024-09-21T07:35:50.610255+0000","last_became_active":"2024-09-21T07:35:49.483358+0000","last_became_peered":"2024-09-21T07:35:49.483358+0000","last_unstale":"2024-09-21T07:35:50.610255+0000","last_undegraded":"2024-09-21T07:35:50.610255+0000","last_fullsized":"2024-09-21T07:35:50.610255+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:34:17.263329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.607267+0000","last_change":"2024-09-21T07:35:48.472242+0000","last_active":"2024-09-21T07:35:50.607267+0000","last_peered":"2024-09-21T07:35:50.607267+0000","last_clean":"2024-09-21T07:35:50.607267+0000","last_became_active":"2024-09-21T07:35:48.471808+0000","last_became_peered":"2024-09-21T07:35:48.471808+0000","last_unstale":"2024-09-21T07:35:50.607267+0000","last_undegraded":"2024-09-21T07:35:50.607267+0000","last_fullsized":"2024-09-21T07:35:50.607267+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:10:29.731170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":36,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608130+0000","last_change":"2024-09-21T07:35:27.874387+0000","last_active":"2024-09-21T07:35:50.608130+0000","last_peered":"2024-09-21T07:35:50.608130+0000","last_clean":"2024-09-21T07:35:50.608130+0000","last_became_active":"2024-09-21T07:35:27.873277+0000","last_became_peered":"2024-09-21T07:35:27.873277+0000","last_unstale":"2024-09-21T07:35:50.608130+0000","last_undegraded":"2024-09-21T07:35:50.608130+0000","last_fullsized":"2024-09-21T07:35:50.608130+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:04.581463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609198+0000","last_change":"2024-09-21T07:35:29.387094+0000","last_active":"2024-09-21T07:35:50.609198+0000","last_peered":"2024-09-21T07:35:50.609198+0000","last_clean":"2024-09-21T07:35:50.609198+0000","last_became_active":"2024-09-21T07:35:26.874757+0000","last_became_peered":"2024-09-21T07:35:26.874757+0000","last_unstale":"2024-09-21T07:35:50.609198+0000","last_undegraded":"2024-09-21T07:35:50.609198+0000","last_fullsized":"2024-09-21T07:35:50.609198+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:33.542750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061065700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":94,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609515+0000","last_change":"2024-09-21T07:34:24.259906+0000","last_active":"2024-09-21T07:35:50.609515+0000","last_peered":"2024-09-21T07:35:50.609515+0000","last_clean":"2024-09-21T07:35:50.609515+0000","last_became_active":"2024-09-21T07:34:23.635282+0000","last_became_peered":"2024-09-21T07:34:23.635282+0000","last_unstale":"2024-09-21T07:35:50.609515+0000","last_undegraded":"2024-09-21T07:35:50.609515+0000","last_fullsized":"2024-09-21T07:35:50.609515+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:00.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001527664,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608637+0000","last_change":"2024-09-21T07:35:49.478853+0000","last_active":"2024-09-21T07:35:50.608637+0000","last_peered":"2024-09-21T07:35:50.608637+0000","last_clean":"2024-09-21T07:35:50.608637+0000","last_became_active":"2024-09-21T07:35:49.478490+0000","last_became_peered":"2024-09-21T07:35:49.478490+0000","last_unstale":"2024-09-21T07:35:50.608637+0000","last_undegraded":"2024-09-21T07:35:50.608637+0000","last_fullsized":"2024-09-21T07:35:50.608637+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:44.711617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"77'2","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.465912+0000","last_change":"2024-09-21T07:35:48.477646+0000","last_active":"2024-09-21T07:35:50.465912+0000","last_peered":"2024-09-21T07:35:50.465912+0000","last_clean":"2024-09-21T07:35:50.465912+0000","last_became_active":"2024-09-21T07:35:48.477112+0000","last_became_peered":"2024-09-21T07:35:48.477112+0000","last_unstale":"2024-09-21T07:35:50.465912+0000","last_undegraded":"2024-09-21T07:35:50.465912+0000","last_fullsized":"2024-09-21T07:35:50.465912+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:05.943925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":36,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.607906+0000","last_change":"2024-09-21T07:35:27.873645+0000","last_active":"2024-09-21T07:35:50.607906+0000","last_peered":"2024-09-21T07:35:50.607906+0000","last_clean":"2024-09-21T07:35:50.607906+0000","last_became_active":"2024-09-21T07:35:27.873245+0000","last_became_peered":"2024-09-21T07:35:27.873245+0000","last_unstale":"2024-09-21T07:35:50.607906+0000","last_undegraded":"2024-09-21T07:35:50.607906+0000","last_fullsized":"2024-09-21T07:35:50.607906+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:55.090953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":43,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.610950+0000","last_change":"2024-09-21T07:35:28.861744+0000","last_active":"2024-09-21T07:35:50.610950+0000","last_peered":"2024-09-21T07:35:50.610950+0000","last_clean":"2024-09-21T07:35:50.610950+0000","last_became_active":"2024-09-21T07:35:26.876462+0000","last_became_peered":"2024-09-21T07:35:26.876462+0000","last_unstale":"2024-09-21T07:35:50.610950+0000","last_undegraded":"2024-09-21T07:35:50.610950+0000","last_fullsized":"2024-09-21T07:35:50.610950+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:48.632568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012020920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":92,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608550+0000","last_change":"2024-09-21T07:34:25.085454+0000","last_active":"2024-09-21T07:35:50.608550+0000","last_peered":"2024-09-21T07:35:50.608550+0000","last_clean":"2024-09-21T07:35:50.608550+0000","last_became_active":"2024-09-21T07:34:23.279205+0000","last_became_peered":"2024-09-21T07:34:23.279205+0000","last_unstale":"2024-09-21T07:35:50.608550+0000","last_undegraded":"2024-09-21T07:35:50.608550+0000","last_fullsized":"2024-09-21T07:35:50.608550+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:54.231666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095320599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466748+0000","last_change":"2024-09-21T07:35:49.485445+0000","last_active":"2024-09-21T07:35:50.466748+0000","last_peered":"2024-09-21T07:35:50.466748+0000","last_clean":"2024-09-21T07:35:50.466748+0000","last_became_active":"2024-09-21T07:35:49.484912+0000","last_became_peered":"2024-09-21T07:35:49.484912+0000","last_unstale":"2024-09-21T07:35:50.466748+0000","last_undegraded":"2024-09-21T07:35:50.466748+0000","last_fullsized":"2024-09-21T07:35:50.466748+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:26:24.284853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.607993+0000","last_change":"2024-09-21T07:35:48.464790+0000","last_active":"2024-09-21T07:35:50.607993+0000","last_peered":"2024-09-21T07:35:50.607993+0000","last_clean":"2024-09-21T07:35:50.607993+0000","last_became_active":"2024-09-21T07:35:48.464398+0000","last_became_peered":"2024-09-21T07:35:48.464398+0000","last_unstale":"2024-09-21T07:35:50.607993+0000","last_undegraded":"2024-09-21T07:35:50.607993+0000","last_fullsized":"2024-09-21T07:35:50.607993+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:28:32.537073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608673+0000","last_change":"2024-09-21T07:35:27.861373+0000","last_active":"2024-09-21T07:35:50.608673+0000","last_peered":"2024-09-21T07:35:50.608673+0000","last_clean":"2024-09-21T07:35:50.608673+0000","last_became_active":"2024-09-21T07:35:27.861016+0000","last_became_peered":"2024-09-21T07:35:27.861016+0000","last_unstale":"2024-09-21T07:35:50.608673+0000","last_undegraded":"2024-09-21T07:35:50.608673+0000","last_fullsized":"2024-09-21T07:35:50.608673+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:56.043763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":43,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466625+0000","last_change":"2024-09-21T07:35:29.053677+0000","last_active":"2024-09-21T07:35:50.466625+0000","last_peered":"2024-09-21T07:35:50.466625+0000","last_clean":"2024-09-21T07:35:50.466625+0000","last_became_active":"2024-09-21T07:35:26.875496+0000","last_became_peered":"2024-09-21T07:35:26.875496+0000","last_unstale":"2024-09-21T07:35:50.466625+0000","last_undegraded":"2024-09-21T07:35:50.466625+0000","last_fullsized":"2024-09-21T07:35:50.466625+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:04.850784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065859000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":94,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466113+0000","last_change":"2024-09-21T07:34:25.028521+0000","last_active":"2024-09-21T07:35:50.466113+0000","last_peered":"2024-09-21T07:35:50.466113+0000","last_clean":"2024-09-21T07:35:50.466113+0000","last_became_active":"2024-09-21T07:34:23.635467+0000","last_became_peered":"2024-09-21T07:34:23.635467+0000","last_unstale":"2024-09-21T07:35:50.466113+0000","last_undegraded":"2024-09-21T07:35:50.466113+0000","last_fullsized":"2024-09-21T07:35:50.466113+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:33.729123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000660558,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609485+0000","last_change":"2024-09-21T07:35:49.473102+0000","last_active":"2024-09-21T07:35:50.609485+0000","last_peered":"2024-09-21T07:35:50.609485+0000","last_clean":"2024-09-21T07:35:50.609485+0000","last_became_active":"2024-09-21T07:35:49.471147+0000","last_became_peered":"2024-09-21T07:35:49.471147+0000","last_unstale":"2024-09-21T07:35:50.609485+0000","last_undegraded":"2024-09-21T07:35:50.609485+0000","last_fullsized":"2024-09-21T07:35:50.609485+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:02.599930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"77'1","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609337+0000","last_change":"2024-09-21T07:35:48.472515+0000","last_active":"2024-09-21T07:35:50.609337+0000","last_peered":"2024-09-21T07:35:50.609337+0000","last_clean":"2024-09-21T07:35:50.609337+0000","last_became_active":"2024-09-21T07:35:48.472176+0000","last_became_peered":"2024-09-21T07:35:48.472176+0000","last_unstale":"2024-09-21T07:35:50.609337+0000","last_undegraded":"2024-09-21T07:35:50.609337+0000","last_fullsized":"2024-09-21T07:35:50.609337+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:19:41.363336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.610700+0000","last_change":"2024-09-21T07:35:27.871201+0000","last_active":"2024-09-21T07:35:50.610700+0000","last_peered":"2024-09-21T07:35:50.610700+0000","last_clean":"2024-09-21T07:35:50.610700+0000","last_became_active":"2024-09-21T07:35:27.870816+0000","last_became_peered":"2024-09-21T07:35:27.870816+0000","last_unstale":"2024-09-21T07:35:50.610700+0000","last_undegraded":"2024-09-21T07:35:50.610700+0000","last_fullsized":"2024-09-21T07:35:50.610700+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:04.787450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609553+0000","last_change":"2024-09-21T07:35:29.491874+0000","last_active":"2024-09-21T07:35:50.609553+0000","last_peered":"2024-09-21T07:35:50.609553+0000","last_clean":"2024-09-21T07:35:50.609553+0000","last_became_active":"2024-09-21T07:35:26.866381+0000","last_became_peered":"2024-09-21T07:35:26.866381+0000","last_unstale":"2024-09-21T07:35:50.609553+0000","last_undegraded":"2024-09-21T07:35:50.609553+0000","last_fullsized":"2024-09-21T07:35:50.609553+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:36.404647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073235399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":94,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609873+0000","last_change":"2024-09-21T07:34:24.260046+0000","last_active":"2024-09-21T07:35:50.609873+0000","last_peered":"2024-09-21T07:35:50.609873+0000","last_clean":"2024-09-21T07:35:50.609873+0000","last_became_active":"2024-09-21T07:34:23.273431+0000","last_became_peered":"2024-09-21T07:34:23.273431+0000","last_unstale":"2024-09-21T07:35:50.609873+0000","last_undegraded":"2024-09-21T07:35:50.609873+0000","last_fullsized":"2024-09-21T07:35:50.609873+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:18.876057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019803310000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.610552+0000","last_change":"2024-09-21T07:35:49.462274+0000","last_active":"2024-09-21T07:35:50.610552+0000","last_peered":"2024-09-21T07:35:50.610552+0000","last_clean":"2024-09-21T07:35:50.610552+0000","last_became_active":"2024-09-21T07:35:49.461939+0000","last_became_peered":"2024-09-21T07:35:49.461939+0000","last_unstale":"2024-09-21T07:35:50.610552+0000","last_undegraded":"2024-09-21T07:35:50.610552+0000","last_fullsized":"2024-09-21T07:35:50.610552+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:05.599486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"77'1","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.468176+0000","last_change":"2024-09-21T07:35:48.459406+0000","last_active":"2024-09-21T07:35:50.468176+0000","last_peered":"2024-09-21T07:35:50.468176+0000","last_clean":"2024-09-21T07:35:50.468176+0000","last_became_active":"2024-09-21T07:35:48.458952+0000","last_became_peered":"2024-09-21T07:35:48.458952+0000","last_unstale":"2024-09-21T07:35:50.468176+0000","last_undegraded":"2024-09-21T07:35:50.468176+0000","last_fullsized":"2024-09-21T07:35:50.468176+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:00:08.139371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.610813+0000","last_change":"2024-09-21T07:35:27.871736+0000","last_active":"2024-09-21T07:35:50.610813+0000","last_peered":"2024-09-21T07:35:50.610813+0000","last_clean":"2024-09-21T07:35:50.610813+0000","last_became_active":"2024-09-21T07:35:27.871232+0000","last_became_peered":"2024-09-21T07:35:27.871232+0000","last_unstale":"2024-09-21T07:35:50.610813+0000","last_undegraded":"2024-09-21T07:35:50.610813+0000","last_fullsized":"2024-09-21T07:35:50.610813+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:04.419301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608082+0000","last_change":"2024-09-21T07:35:29.491246+0000","last_active":"2024-09-21T07:35:50.608082+0000","last_peered":"2024-09-21T07:35:50.608082+0000","last_clean":"2024-09-21T07:35:50.608082+0000","last_became_active":"2024-09-21T07:35:26.866783+0000","last_became_peered":"2024-09-21T07:35:26.866783+0000","last_unstale":"2024-09-21T07:35:50.608082+0000","last_undegraded":"2024-09-21T07:35:50.608082+0000","last_fullsized":"2024-09-21T07:35:50.608082+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:28.881914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098474099999999992,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":94,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466717+0000","last_change":"2024-09-21T07:34:25.028640+0000","last_active":"2024-09-21T07:35:50.466717+0000","last_peered":"2024-09-21T07:35:50.466717+0000","last_clean":"2024-09-21T07:35:50.466717+0000","last_became_active":"2024-09-21T07:34:23.260950+0000","last_became_peered":"2024-09-21T07:34:23.260950+0000","last_unstale":"2024-09-21T07:35:50.466717+0000","last_undegraded":"2024-09-21T07:35:50.466717+0000","last_fullsized":"2024-09-21T07:35:50.466717+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:23:42.121398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055135099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.467998+0000","last_change":"2024-09-21T07:35:49.475001+0000","last_active":"2024-09-21T07:35:50.467998+0000","last_peered":"2024-09-21T07:35:50.467998+0000","last_clean":"2024-09-21T07:35:50.467998+0000","last_became_active":"2024-09-21T07:35:49.474511+0000","last_became_peered":"2024-09-21T07:35:49.474511+0000","last_unstale":"2024-09-21T07:35:50.467998+0000","last_undegraded":"2024-09-21T07:35:50.467998+0000","last_fullsized":"2024-09-21T07:35:50.467998+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:26.004814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"77'2","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609915+0000","last_change":"2024-09-21T07:35:48.463983+0000","last_active":"2024-09-21T07:35:50.609915+0000","last_peered":"2024-09-21T07:35:50.609915+0000","last_clean":"2024-09-21T07:35:50.609915+0000","last_became_active":"2024-09-21T07:35:48.463661+0000","last_became_peered":"2024-09-21T07:35:48.463661+0000","last_unstale":"2024-09-21T07:35:50.609915+0000","last_undegraded":"2024-09-21T07:35:50.609915+0000","last_fullsized":"2024-09-21T07:35:50.609915+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:48.919917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.468430+0000","last_change":"2024-09-21T07:35:27.872027+0000","last_active":"2024-09-21T07:35:50.468430+0000","last_peered":"2024-09-21T07:35:50.468430+0000","last_clean":"2024-09-21T07:35:50.468430+0000","last_became_active":"2024-09-21T07:35:27.870426+0000","last_became_peered":"2024-09-21T07:35:27.870426+0000","last_unstale":"2024-09-21T07:35:50.468430+0000","last_undegraded":"2024-09-21T07:35:50.468430+0000","last_fullsized":"2024-09-21T07:35:50.468430+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.607945+0000","last_change":"2024-09-21T07:35:29.491008+0000","last_active":"2024-09-21T07:35:50.607945+0000","last_peered":"2024-09-21T07:35:50.607945+0000","last_clean":"2024-09-21T07:35:50.607945+0000","last_became_active":"2024-09-21T07:35:26.866750+0000","last_became_peered":"2024-09-21T07:35:26.866750+0000","last_unstale":"2024-09-21T07:35:50.607945+0000","last_undegraded":"2024-09-21T07:35:50.607945+0000","last_fullsized":"2024-09-21T07:35:50.607945+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:25.490406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083529700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":94,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609716+0000","last_change":"2024-09-21T07:34:24.260665+0000","last_active":"2024-09-21T07:35:50.609716+0000","last_peered":"2024-09-21T07:35:50.609716+0000","last_clean":"2024-09-21T07:35:50.609716+0000","last_became_active":"2024-09-21T07:34:23.635909+0000","last_became_peered":"2024-09-21T07:34:23.635909+0000","last_unstale":"2024-09-21T07:35:50.609716+0000","last_undegraded":"2024-09-21T07:35:50.609716+0000","last_fullsized":"2024-09-21T07:35:50.609716+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:44.536885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466269+0000","last_change":"2024-09-21T07:35:49.480549+0000","last_active":"2024-09-21T07:35:50.466269+0000","last_peered":"2024-09-21T07:35:50.466269+0000","last_clean":"2024-09-21T07:35:50.466269+0000","last_became_active":"2024-09-21T07:35:49.480128+0000","last_became_peered":"2024-09-21T07:35:49.480128+0000","last_unstale":"2024-09-21T07:35:50.466269+0000","last_undegraded":"2024-09-21T07:35:50.466269+0000","last_fullsized":"2024-09-21T07:35:50.466269+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:38.532303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.465345+0000","last_change":"2024-09-21T07:35:48.466284+0000","last_active":"2024-09-21T07:35:50.465345+0000","last_peered":"2024-09-21T07:35:50.465345+0000","last_clean":"2024-09-21T07:35:50.465345+0000","last_became_active":"2024-09-21T07:35:48.465970+0000","last_became_peered":"2024-09-21T07:35:48.465970+0000","last_unstale":"2024-09-21T07:35:50.465345+0000","last_undegraded":"2024-09-21T07:35:50.465345+0000","last_fullsized":"2024-09-21T07:35:50.465345+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:15:41.920172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.607575+0000","last_change":"2024-09-21T07:35:27.874772+0000","last_active":"2024-09-21T07:35:50.607575+0000","last_peered":"2024-09-21T07:35:50.607575+0000","last_clean":"2024-09-21T07:35:50.607575+0000","last_became_active":"2024-09-21T07:35:27.874434+0000","last_became_peered":"2024-09-21T07:35:27.874434+0000","last_unstale":"2024-09-21T07:35:50.607575+0000","last_undegraded":"2024-09-21T07:35:50.607575+0000","last_fullsized":"2024-09-21T07:35:50.607575+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:31.998755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":44,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.467820+0000","last_change":"2024-09-21T07:35:28.859153+0000","last_active":"2024-09-21T07:35:50.467820+0000","last_peered":"2024-09-21T07:35:50.467820+0000","last_clean":"2024-09-21T07:35:50.467820+0000","last_became_active":"2024-09-21T07:35:26.874428+0000","last_became_peered":"2024-09-21T07:35:26.874428+0000","last_unstale":"2024-09-21T07:35:50.467820+0000","last_undegraded":"2024-09-21T07:35:50.467820+0000","last_fullsized":"2024-09-21T07:35:50.467820+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:04.487621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048557299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":96,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466227+0000","last_change":"2024-09-21T07:34:24.538845+0000","last_active":"2024-09-21T07:35:50.466227+0000","last_peered":"2024-09-21T07:35:50.466227+0000","last_clean":"2024-09-21T07:35:50.466227+0000","last_became_active":"2024-09-21T07:34:23.636372+0000","last_became_peered":"2024-09-21T07:34:23.636372+0000","last_unstale":"2024-09-21T07:35:50.466227+0000","last_undegraded":"2024-09-21T07:35:50.466227+0000","last_fullsized":"2024-09-21T07:35:50.466227+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:29.393813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841451,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.610313+0000","last_change":"2024-09-21T07:35:49.481457+0000","last_active":"2024-09-21T07:35:50.610313+0000","last_peered":"2024-09-21T07:35:50.610313+0000","last_clean":"2024-09-21T07:35:50.610313+0000","last_became_active":"2024-09-21T07:35:49.476056+0000","last_became_peered":"2024-09-21T07:35:49.476056+0000","last_unstale":"2024-09-21T07:35:50.610313+0000","last_undegraded":"2024-09-21T07:35:50.610313+0000","last_fullsized":"2024-09-21T07:35:50.610313+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:52:01.228329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.607796+0000","last_change":"2024-09-21T07:35:48.464324+0000","last_active":"2024-09-21T07:35:50.607796+0000","last_peered":"2024-09-21T07:35:50.607796+0000","last_clean":"2024-09-21T07:35:50.607796+0000","last_became_active":"2024-09-21T07:35:48.463877+0000","last_became_peered":"2024-09-21T07:35:48.463877+0000","last_unstale":"2024-09-21T07:35:50.607796+0000","last_undegraded":"2024-09-21T07:35:50.607796+0000","last_fullsized":"2024-09-21T07:35:50.607796+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:22.251973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.465848+0000","last_change":"2024-09-21T07:35:27.877801+0000","last_active":"2024-09-21T07:35:50.465848+0000","last_peered":"2024-09-21T07:35:50.465848+0000","last_clean":"2024-09-21T07:35:50.465848+0000","last_became_active":"2024-09-21T07:35:27.877464+0000","last_became_peered":"2024-09-21T07:35:27.877464+0000","last_unstale":"2024-09-21T07:35:50.465848+0000","last_undegraded":"2024-09-21T07:35:50.465848+0000","last_fullsized":"2024-09-21T07:35:50.465848+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:08.814929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.611053+0000","last_change":"2024-09-21T07:35:28.863732+0000","last_active":"2024-09-21T07:35:50.611053+0000","last_peered":"2024-09-21T07:35:50.611053+0000","last_clean":"2024-09-21T07:35:50.611053+0000","last_became_active":"2024-09-21T07:35:26.877392+0000","last_became_peered":"2024-09-21T07:35:26.877392+0000","last_unstale":"2024-09-21T07:35:50.611053+0000","last_undegraded":"2024-09-21T07:35:50.611053+0000","last_fullsized":"2024-09-21T07:35:50.611053+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071425700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":94,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609652+0000","last_change":"2024-09-21T07:34:24.259686+0000","last_active":"2024-09-21T07:35:50.609652+0000","last_peered":"2024-09-21T07:35:50.609652+0000","last_clean":"2024-09-21T07:35:50.609652+0000","last_became_active":"2024-09-21T07:34:23.274405+0000","last_became_peered":"2024-09-21T07:34:23.274405+0000","last_unstale":"2024-09-21T07:35:50.609652+0000","last_undegraded":"2024-09-21T07:35:50.609652+0000","last_fullsized":"2024-09-21T07:35:50.609652+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:26.304693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022009920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.607600+0000","last_change":"2024-09-21T07:35:49.487661+0000","last_active":"2024-09-21T07:35:50.607600+0000","last_peered":"2024-09-21T07:35:50.607600+0000","last_clean":"2024-09-21T07:35:50.607600+0000","last_became_active":"2024-09-21T07:35:49.487295+0000","last_became_peered":"2024-09-21T07:35:49.487295+0000","last_unstale":"2024-09-21T07:35:50.607600+0000","last_undegraded":"2024-09-21T07:35:50.607600+0000","last_fullsized":"2024-09-21T07:35:50.607600+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:04:12.059484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609915+0000","last_change":"2024-09-21T07:35:48.472374+0000","last_active":"2024-09-21T07:35:50.609915+0000","last_peered":"2024-09-21T07:35:50.609915+0000","last_clean":"2024-09-21T07:35:50.609915+0000","last_became_active":"2024-09-21T07:35:48.471831+0000","last_became_peered":"2024-09-21T07:35:48.471831+0000","last_unstale":"2024-09-21T07:35:50.609915+0000","last_undegraded":"2024-09-21T07:35:50.609915+0000","last_fullsized":"2024-09-21T07:35:50.609915+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:35:38.133415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608635+0000","last_change":"2024-09-21T07:35:27.867336+0000","last_active":"2024-09-21T07:35:50.608635+0000","last_peered":"2024-09-21T07:35:50.608635+0000","last_clean":"2024-09-21T07:35:50.608635+0000","last_became_active":"2024-09-21T07:35:27.867015+0000","last_became_peered":"2024-09-21T07:35:27.867015+0000","last_unstale":"2024-09-21T07:35:50.608635+0000","last_undegraded":"2024-09-21T07:35:50.608635+0000","last_fullsized":"2024-09-21T07:35:50.608635+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:59.214139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609402+0000","last_change":"2024-09-21T07:35:29.493137+0000","last_active":"2024-09-21T07:35:50.609402+0000","last_peered":"2024-09-21T07:35:50.609402+0000","last_clean":"2024-09-21T07:35:50.609402+0000","last_became_active":"2024-09-21T07:35:26.872230+0000","last_became_peered":"2024-09-21T07:35:26.872230+0000","last_unstale":"2024-09-21T07:35:50.609402+0000","last_undegraded":"2024-09-21T07:35:50.609402+0000","last_fullsized":"2024-09-21T07:35:50.609402+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:24.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001197449,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":94,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608083+0000","last_change":"2024-09-21T07:34:24.258754+0000","last_active":"2024-09-21T07:35:50.608083+0000","last_peered":"2024-09-21T07:35:50.608083+0000","last_clean":"2024-09-21T07:35:50.608083+0000","last_became_active":"2024-09-21T07:34:23.272657+0000","last_became_peered":"2024-09-21T07:34:23.272657+0000","last_unstale":"2024-09-21T07:35:50.608083+0000","last_undegraded":"2024-09-21T07:35:50.608083+0000","last_fullsized":"2024-09-21T07:35:50.608083+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:05.477099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067143200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608194+0000","last_change":"2024-09-21T07:35:49.488812+0000","last_active":"2024-09-21T07:35:50.608194+0000","last_peered":"2024-09-21T07:35:50.608194+0000","last_clean":"2024-09-21T07:35:50.608194+0000","last_became_active":"2024-09-21T07:35:49.482857+0000","last_became_peered":"2024-09-21T07:35:49.482857+0000","last_unstale":"2024-09-21T07:35:50.608194+0000","last_undegraded":"2024-09-21T07:35:50.608194+0000","last_fullsized":"2024-09-21T07:35:50.608194+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:28:04.816276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466099+0000","last_change":"2024-09-21T07:35:48.463728+0000","last_active":"2024-09-21T07:35:50.466099+0000","last_peered":"2024-09-21T07:35:50.466099+0000","last_clean":"2024-09-21T07:35:50.466099+0000","last_became_active":"2024-09-21T07:35:48.463347+0000","last_became_peered":"2024-09-21T07:35:48.463347+0000","last_unstale":"2024-09-21T07:35:50.466099+0000","last_undegraded":"2024-09-21T07:35:50.466099+0000","last_fullsized":"2024-09-21T07:35:50.466099+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:12.497861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.610391+0000","last_change":"2024-09-21T07:35:27.876021+0000","last_active":"2024-09-21T07:35:50.610391+0000","last_peered":"2024-09-21T07:35:50.610391+0000","last_clean":"2024-09-21T07:35:50.610391+0000","last_became_active":"2024-09-21T07:35:27.875724+0000","last_became_peered":"2024-09-21T07:35:27.875724+0000","last_unstale":"2024-09-21T07:35:50.610391+0000","last_undegraded":"2024-09-21T07:35:50.610391+0000","last_fullsized":"2024-09-21T07:35:50.610391+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:03.022812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608309+0000","last_change":"2024-09-21T07:35:29.492570+0000","last_active":"2024-09-21T07:35:50.608309+0000","last_peered":"2024-09-21T07:35:50.608309+0000","last_clean":"2024-09-21T07:35:50.608309+0000","last_became_active":"2024-09-21T07:35:26.874316+0000","last_became_peered":"2024-09-21T07:35:26.874316+0000","last_unstale":"2024-09-21T07:35:50.608309+0000","last_undegraded":"2024-09-21T07:35:50.608309+0000","last_fullsized":"2024-09-21T07:35:50.608309+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:16.512323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011990760000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":93,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609895+0000","last_change":"2024-09-21T07:34:25.085232+0000","last_active":"2024-09-21T07:35:50.609895+0000","last_peered":"2024-09-21T07:35:50.609895+0000","last_clean":"2024-09-21T07:35:50.609895+0000","last_became_active":"2024-09-21T07:34:23.277807+0000","last_became_peered":"2024-09-21T07:34:23.277807+0000","last_unstale":"2024-09-21T07:35:50.609895+0000","last_undegraded":"2024-09-21T07:35:50.609895+0000","last_fullsized":"2024-09-21T07:35:50.609895+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:36.362919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086619099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.610953+0000","last_change":"2024-09-21T07:35:49.474323+0000","last_active":"2024-09-21T07:35:50.610953+0000","last_peered":"2024-09-21T07:35:50.610953+0000","last_clean":"2024-09-21T07:35:50.610953+0000","last_became_active":"2024-09-21T07:35:49.474057+0000","last_became_peered":"2024-09-21T07:35:49.474057+0000","last_unstale":"2024-09-21T07:35:50.610953+0000","last_undegraded":"2024-09-21T07:35:50.610953+0000","last_fullsized":"2024-09-21T07:35:50.610953+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:59.379416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609417+0000","last_change":"2024-09-21T07:35:48.477694+0000","last_active":"2024-09-21T07:35:50.609417+0000","last_peered":"2024-09-21T07:35:50.609417+0000","last_clean":"2024-09-21T07:35:50.609417+0000","last_became_active":"2024-09-21T07:35:48.477368+0000","last_became_peered":"2024-09-21T07:35:48.477368+0000","last_unstale":"2024-09-21T07:35:50.609417+0000","last_undegraded":"2024-09-21T07:35:50.609417+0000","last_fullsized":"2024-09-21T07:35:50.609417+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:57.866409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466736+0000","last_change":"2024-09-21T07:35:27.872798+0000","last_active":"2024-09-21T07:35:50.466736+0000","last_peered":"2024-09-21T07:35:50.466736+0000","last_clean":"2024-09-21T07:35:50.466736+0000","last_became_active":"2024-09-21T07:35:27.872483+0000","last_became_peered":"2024-09-21T07:35:27.872483+0000","last_unstale":"2024-09-21T07:35:50.466736+0000","last_undegraded":"2024-09-21T07:35:50.466736+0000","last_fullsized":"2024-09-21T07:35:50.466736+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:11.933469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.465575+0000","last_change":"2024-09-21T07:35:29.206723+0000","last_active":"2024-09-21T07:35:50.465575+0000","last_peered":"2024-09-21T07:35:50.465575+0000","last_clean":"2024-09-21T07:35:50.465575+0000","last_became_active":"2024-09-21T07:35:26.872377+0000","last_became_peered":"2024-09-21T07:35:26.872377+0000","last_unstale":"2024-09-21T07:35:50.465575+0000","last_undegraded":"2024-09-21T07:35:50.465575+0000","last_fullsized":"2024-09-21T07:35:50.465575+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:29.086862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083077200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":90,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609755+0000","last_change":"2024-09-21T07:34:24.597672+0000","last_active":"2024-09-21T07:35:50.609755+0000","last_peered":"2024-09-21T07:35:50.609755+0000","last_clean":"2024-09-21T07:35:50.609755+0000","last_became_active":"2024-09-21T07:34:23.267770+0000","last_became_peered":"2024-09-21T07:34:23.267770+0000","last_unstale":"2024-09-21T07:35:50.609755+0000","last_undegraded":"2024-09-21T07:35:50.609755+0000","last_fullsized":"2024-09-21T07:35:50.609755+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:51.629476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086378500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.467668+0000","last_change":"2024-09-21T07:35:49.460209+0000","last_active":"2024-09-21T07:35:50.467668+0000","last_peered":"2024-09-21T07:35:50.467668+0000","last_clean":"2024-09-21T07:35:50.467668+0000","last_became_active":"2024-09-21T07:35:49.459838+0000","last_became_peered":"2024-09-21T07:35:49.459838+0000","last_unstale":"2024-09-21T07:35:50.467668+0000","last_undegraded":"2024-09-21T07:35:50.467668+0000","last_fullsized":"2024-09-21T07:35:50.467668+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:26:49.229370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466661+0000","last_change":"2024-09-21T07:35:48.466416+0000","last_active":"2024-09-21T07:35:50.466661+0000","last_peered":"2024-09-21T07:35:50.466661+0000","last_clean":"2024-09-21T07:35:50.466661+0000","last_became_active":"2024-09-21T07:35:48.466066+0000","last_became_peered":"2024-09-21T07:35:48.466066+0000","last_unstale":"2024-09-21T07:35:50.466661+0000","last_undegraded":"2024-09-21T07:35:50.466661+0000","last_fullsized":"2024-09-21T07:35:50.466661+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:41.865640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.467656+0000","last_change":"2024-09-21T07:35:27.870121+0000","last_active":"2024-09-21T07:35:50.467656+0000","last_peered":"2024-09-21T07:35:50.467656+0000","last_clean":"2024-09-21T07:35:50.467656+0000","last_became_active":"2024-09-21T07:35:27.869811+0000","last_became_peered":"2024-09-21T07:35:27.869811+0000","last_unstale":"2024-09-21T07:35:50.467656+0000","last_undegraded":"2024-09-21T07:35:50.467656+0000","last_fullsized":"2024-09-21T07:35:50.467656+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:31.778461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":43,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.607809+0000","last_change":"2024-09-21T07:35:28.860699+0000","last_active":"2024-09-21T07:35:50.607809+0000","last_peered":"2024-09-21T07:35:50.607809+0000","last_clean":"2024-09-21T07:35:50.607809+0000","last_became_active":"2024-09-21T07:35:26.876779+0000","last_became_peered":"2024-09-21T07:35:26.876779+0000","last_unstale":"2024-09-21T07:35:50.607809+0000","last_undegraded":"2024-09-21T07:35:50.607809+0000","last_fullsized":"2024-09-21T07:35:50.607809+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:28.286403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000913188,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":95,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.611310+0000","last_change":"2024-09-21T07:34:24.260181+0000","last_active":"2024-09-21T07:35:50.611310+0000","last_peered":"2024-09-21T07:35:50.611310+0000","last_clean":"2024-09-21T07:35:50.611310+0000","last_became_active":"2024-09-21T07:34:23.275580+0000","last_became_peered":"2024-09-21T07:34:23.275580+0000","last_unstale":"2024-09-21T07:35:50.611310+0000","last_undegraded":"2024-09-21T07:35:50.611310+0000","last_fullsized":"2024-09-21T07:35:50.611310+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:49.857622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014706980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.467022+0000","last_change":"2024-09-21T07:35:49.477557+0000","last_active":"2024-09-21T07:35:50.467022+0000","last_peered":"2024-09-21T07:35:50.467022+0000","last_clean":"2024-09-21T07:35:50.467022+0000","last_became_active":"2024-09-21T07:35:49.477265+0000","last_became_peered":"2024-09-21T07:35:49.477265+0000","last_unstale":"2024-09-21T07:35:50.467022+0000","last_undegraded":"2024-09-21T07:35:50.467022+0000","last_fullsized":"2024-09-21T07:35:50.467022+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:25:59.317284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"77'2","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609551+0000","last_change":"2024-09-21T07:35:48.466624+0000","last_active":"2024-09-21T07:35:50.609551+0000","last_peered":"2024-09-21T07:35:50.609551+0000","last_clean":"2024-09-21T07:35:50.609551+0000","last_became_active":"2024-09-21T07:35:48.466246+0000","last_became_peered":"2024-09-21T07:35:48.466246+0000","last_unstale":"2024-09-21T07:35:50.609551+0000","last_undegraded":"2024-09-21T07:35:50.609551+0000","last_fullsized":"2024-09-21T07:35:50.609551+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:36.646120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":36,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.467339+0000","last_change":"2024-09-21T07:35:27.875393+0000","last_active":"2024-09-21T07:35:50.467339+0000","last_peered":"2024-09-21T07:35:50.467339+0000","last_clean":"2024-09-21T07:35:50.467339+0000","last_became_active":"2024-09-21T07:35:27.874671+0000","last_became_peered":"2024-09-21T07:35:27.874671+0000","last_unstale":"2024-09-21T07:35:50.467339+0000","last_undegraded":"2024-09-21T07:35:50.467339+0000","last_fullsized":"2024-09-21T07:35:50.467339+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466315+0000","last_change":"2024-09-21T07:35:29.206296+0000","last_active":"2024-09-21T07:35:50.466315+0000","last_peered":"2024-09-21T07:35:50.466315+0000","last_clean":"2024-09-21T07:35:50.466315+0000","last_became_active":"2024-09-21T07:35:26.874575+0000","last_became_peered":"2024-09-21T07:35:26.874575+0000","last_unstale":"2024-09-21T07:35:50.466315+0000","last_undegraded":"2024-09-21T07:35:50.466315+0000","last_fullsized":"2024-09-21T07:35:50.466315+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:22.771154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061011599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":94,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608912+0000","last_change":"2024-09-21T07:34:24.259112+0000","last_active":"2024-09-21T07:35:50.608912+0000","last_peered":"2024-09-21T07:35:50.608912+0000","last_clean":"2024-09-21T07:35:50.608912+0000","last_became_active":"2024-09-21T07:34:23.635444+0000","last_became_peered":"2024-09-21T07:34:23.635444+0000","last_unstale":"2024-09-21T07:35:50.608912+0000","last_undegraded":"2024-09-21T07:35:50.608912+0000","last_fullsized":"2024-09-21T07:35:50.608912+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:46.255385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001952636,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.611134+0000","last_change":"2024-09-21T07:35:49.469045+0000","last_active":"2024-09-21T07:35:50.611134+0000","last_peered":"2024-09-21T07:35:50.611134+0000","last_clean":"2024-09-21T07:35:50.611134+0000","last_became_active":"2024-09-21T07:35:49.464330+0000","last_became_peered":"2024-09-21T07:35:49.464330+0000","last_unstale":"2024-09-21T07:35:50.611134+0000","last_undegraded":"2024-09-21T07:35:50.611134+0000","last_fullsized":"2024-09-21T07:35:50.611134+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:45.434454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"78'1","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:51.449813+0000","last_change":"2024-09-21T07:35:48.469515+0000","last_active":"2024-09-21T07:35:51.449813+0000","last_peered":"2024-09-21T07:35:51.449813+0000","last_clean":"2024-09-21T07:35:51.449813+0000","last_became_active":"2024-09-21T07:35:48.469155+0000","last_became_peered":"2024-09-21T07:35:48.469155+0000","last_unstale":"2024-09-21T07:35:51.449813+0000","last_undegraded":"2024-09-21T07:35:51.449813+0000","last_fullsized":"2024-09-21T07:35:51.449813+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:32.952980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":36,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.467323+0000","last_change":"2024-09-21T07:35:27.878356+0000","last_active":"2024-09-21T07:35:50.467323+0000","last_peered":"2024-09-21T07:35:50.467323+0000","last_clean":"2024-09-21T07:35:50.467323+0000","last_became_active":"2024-09-21T07:35:27.877982+0000","last_became_peered":"2024-09-21T07:35:27.877982+0000","last_unstale":"2024-09-21T07:35:50.467323+0000","last_undegraded":"2024-09-21T07:35:50.467323+0000","last_fullsized":"2024-09-21T07:35:50.467323+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":43,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466211+0000","last_change":"2024-09-21T07:35:29.053131+0000","last_active":"2024-09-21T07:35:50.466211+0000","last_peered":"2024-09-21T07:35:50.466211+0000","last_clean":"2024-09-21T07:35:50.466211+0000","last_became_active":"2024-09-21T07:35:26.870516+0000","last_became_peered":"2024-09-21T07:35:26.870516+0000","last_unstale":"2024-09-21T07:35:50.466211+0000","last_undegraded":"2024-09-21T07:35:50.466211+0000","last_fullsized":"2024-09-21T07:35:50.466211+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:10.924491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070705699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":96,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.465696+0000","last_change":"2024-09-21T07:34:24.539152+0000","last_active":"2024-09-21T07:35:50.465696+0000","last_peered":"2024-09-21T07:35:50.465696+0000","last_clean":"2024-09-21T07:35:50.465696+0000","last_became_active":"2024-09-21T07:34:23.259573+0000","last_became_peered":"2024-09-21T07:34:23.259573+0000","last_unstale":"2024-09-21T07:35:50.465696+0000","last_undegraded":"2024-09-21T07:35:50.465696+0000","last_fullsized":"2024-09-21T07:35:50.465696+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:26.986900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058108400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.610210+0000","last_change":"2024-09-21T07:35:49.478512+0000","last_active":"2024-09-21T07:35:50.610210+0000","last_peered":"2024-09-21T07:35:50.610210+0000","last_clean":"2024-09-21T07:35:50.610210+0000","last_became_active":"2024-09-21T07:35:49.478207+0000","last_became_peered":"2024-09-21T07:35:49.478207+0000","last_unstale":"2024-09-21T07:35:50.610210+0000","last_undegraded":"2024-09-21T07:35:50.610210+0000","last_fullsized":"2024-09-21T07:35:50.610210+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:05.236363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.607832+0000","last_change":"2024-09-21T07:35:48.453787+0000","last_active":"2024-09-21T07:35:50.607832+0000","last_peered":"2024-09-21T07:35:50.607832+0000","last_clean":"2024-09-21T07:35:50.607832+0000","last_became_active":"2024-09-21T07:35:48.453478+0000","last_became_peered":"2024-09-21T07:35:48.453478+0000","last_unstale":"2024-09-21T07:35:50.607832+0000","last_undegraded":"2024-09-21T07:35:50.607832+0000","last_fullsized":"2024-09-21T07:35:50.607832+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:18:29.322070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":36,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466090+0000","last_change":"2024-09-21T07:35:27.873726+0000","last_active":"2024-09-21T07:35:50.466090+0000","last_peered":"2024-09-21T07:35:50.466090+0000","last_clean":"2024-09-21T07:35:50.466090+0000","last_became_active":"2024-09-21T07:35:27.871421+0000","last_became_peered":"2024-09-21T07:35:27.871421+0000","last_unstale":"2024-09-21T07:35:50.466090+0000","last_undegraded":"2024-09-21T07:35:50.466090+0000","last_fullsized":"2024-09-21T07:35:50.466090+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:00.168915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":43,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.610796+0000","last_change":"2024-09-21T07:35:28.861032+0000","last_active":"2024-09-21T07:35:50.610796+0000","last_peered":"2024-09-21T07:35:50.610796+0000","last_clean":"2024-09-21T07:35:50.610796+0000","last_became_active":"2024-09-21T07:35:26.873149+0000","last_became_peered":"2024-09-21T07:35:26.873149+0000","last_unstale":"2024-09-21T07:35:50.610796+0000","last_undegraded":"2024-09-21T07:35:50.610796+0000","last_fullsized":"2024-09-21T07:35:50.610796+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:59.435981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011988369999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":94,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608742+0000","last_change":"2024-09-21T07:34:24.259902+0000","last_active":"2024-09-21T07:35:50.608742+0000","last_peered":"2024-09-21T07:35:50.608742+0000","last_clean":"2024-09-21T07:35:50.608742+0000","last_became_active":"2024-09-21T07:34:23.271902+0000","last_became_peered":"2024-09-21T07:34:23.271902+0000","last_unstale":"2024-09-21T07:35:50.608742+0000","last_undegraded":"2024-09-21T07:35:50.608742+0000","last_fullsized":"2024-09-21T07:35:50.608742+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:59.439525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001686358,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466202+0000","last_change":"2024-09-21T07:35:49.479584+0000","last_active":"2024-09-21T07:35:50.466202+0000","last_peered":"2024-09-21T07:35:50.466202+0000","last_clean":"2024-09-21T07:35:50.466202+0000","last_became_active":"2024-09-21T07:35:49.476299+0000","last_became_peered":"2024-09-21T07:35:49.476299+0000","last_unstale":"2024-09-21T07:35:50.466202+0000","last_undegraded":"2024-09-21T07:35:50.466202+0000","last_fullsized":"2024-09-21T07:35:50.466202+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:45.827149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.607657+0000","last_change":"2024-09-21T07:35:48.469152+0000","last_active":"2024-09-21T07:35:50.607657+0000","last_peered":"2024-09-21T07:35:50.607657+0000","last_clean":"2024-09-21T07:35:50.607657+0000","last_became_active":"2024-09-21T07:35:48.468821+0000","last_became_peered":"2024-09-21T07:35:48.468821+0000","last_unstale":"2024-09-21T07:35:50.607657+0000","last_undegraded":"2024-09-21T07:35:50.607657+0000","last_fullsized":"2024-09-21T07:35:50.607657+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:31.575643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608383+0000","last_change":"2024-09-21T07:35:27.872637+0000","last_active":"2024-09-21T07:35:50.608383+0000","last_peered":"2024-09-21T07:35:50.608383+0000","last_clean":"2024-09-21T07:35:50.608383+0000","last_became_active":"2024-09-21T07:35:27.872149+0000","last_became_peered":"2024-09-21T07:35:27.872149+0000","last_unstale":"2024-09-21T07:35:50.608383+0000","last_undegraded":"2024-09-21T07:35:50.608383+0000","last_fullsized":"2024-09-21T07:35:50.608383+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:27.697872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609520+0000","last_change":"2024-09-21T07:35:29.493362+0000","last_active":"2024-09-21T07:35:50.609520+0000","last_peered":"2024-09-21T07:35:50.609520+0000","last_clean":"2024-09-21T07:35:50.609520+0000","last_became_active":"2024-09-21T07:35:26.873018+0000","last_became_peered":"2024-09-21T07:35:26.873018+0000","last_unstale":"2024-09-21T07:35:50.609520+0000","last_undegraded":"2024-09-21T07:35:50.609520+0000","last_fullsized":"2024-09-21T07:35:50.609520+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:02.460557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012496510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":94,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609363+0000","last_change":"2024-09-21T07:34:24.259662+0000","last_active":"2024-09-21T07:35:50.609363+0000","last_peered":"2024-09-21T07:35:50.609363+0000","last_clean":"2024-09-21T07:35:50.609363+0000","last_became_active":"2024-09-21T07:34:23.278812+0000","last_became_peered":"2024-09-21T07:34:23.278812+0000","last_unstale":"2024-09-21T07:35:50.609363+0000","last_undegraded":"2024-09-21T07:35:50.609363+0000","last_fullsized":"2024-09-21T07:35:50.609363+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:49.829974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356463,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466023+0000","last_change":"2024-09-21T07:35:49.493721+0000","last_active":"2024-09-21T07:35:50.466023+0000","last_peered":"2024-09-21T07:35:50.466023+0000","last_clean":"2024-09-21T07:35:50.466023+0000","last_became_active":"2024-09-21T07:35:49.493430+0000","last_became_peered":"2024-09-21T07:35:49.493430+0000","last_unstale":"2024-09-21T07:35:50.466023+0000","last_undegraded":"2024-09-21T07:35:50.466023+0000","last_fullsized":"2024-09-21T07:35:50.466023+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:43:22.507011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"77'1","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608310+0000","last_change":"2024-09-21T07:35:48.476792+0000","last_active":"2024-09-21T07:35:50.608310+0000","last_peered":"2024-09-21T07:35:50.608310+0000","last_clean":"2024-09-21T07:35:50.608310+0000","last_became_active":"2024-09-21T07:35:48.476413+0000","last_became_peered":"2024-09-21T07:35:48.476413+0000","last_unstale":"2024-09-21T07:35:50.608310+0000","last_undegraded":"2024-09-21T07:35:50.608310+0000","last_fullsized":"2024-09-21T07:35:50.608310+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:57.273910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608457+0000","last_change":"2024-09-21T07:35:27.868448+0000","last_active":"2024-09-21T07:35:50.608457+0000","last_peered":"2024-09-21T07:35:50.608457+0000","last_clean":"2024-09-21T07:35:50.608457+0000","last_became_active":"2024-09-21T07:35:27.868105+0000","last_became_peered":"2024-09-21T07:35:27.868105+0000","last_unstale":"2024-09-21T07:35:50.608457+0000","last_undegraded":"2024-09-21T07:35:50.608457+0000","last_fullsized":"2024-09-21T07:35:50.608457+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:41.468702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608872+0000","last_change":"2024-09-21T07:35:29.492974+0000","last_active":"2024-09-21T07:35:50.608872+0000","last_peered":"2024-09-21T07:35:50.608872+0000","last_clean":"2024-09-21T07:35:50.608872+0000","last_became_active":"2024-09-21T07:35:26.872165+0000","last_became_peered":"2024-09-21T07:35:26.872165+0000","last_unstale":"2024-09-21T07:35:50.608872+0000","last_undegraded":"2024-09-21T07:35:50.608872+0000","last_fullsized":"2024-09-21T07:35:50.608872+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:59.912520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013795179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":93,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608446+0000","last_change":"2024-09-21T07:34:25.084861+0000","last_active":"2024-09-21T07:35:50.608446+0000","last_peered":"2024-09-21T07:35:50.608446+0000","last_clean":"2024-09-21T07:35:50.608446+0000","last_became_active":"2024-09-21T07:34:23.277708+0000","last_became_peered":"2024-09-21T07:34:23.277708+0000","last_unstale":"2024-09-21T07:35:50.608446+0000","last_undegraded":"2024-09-21T07:35:50.608446+0000","last_fullsized":"2024-09-21T07:35:50.608446+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:40.816488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000597566,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466237+0000","last_change":"2024-09-21T07:35:49.471119+0000","last_active":"2024-09-21T07:35:50.466237+0000","last_peered":"2024-09-21T07:35:50.466237+0000","last_clean":"2024-09-21T07:35:50.466237+0000","last_became_active":"2024-09-21T07:35:49.470681+0000","last_became_peered":"2024-09-21T07:35:49.470681+0000","last_unstale":"2024-09-21T07:35:50.466237+0000","last_undegraded":"2024-09-21T07:35:50.466237+0000","last_fullsized":"2024-09-21T07:35:50.466237+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:24.608325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"77'1","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.465930+0000","last_change":"2024-09-21T07:35:48.472971+0000","last_active":"2024-09-21T07:35:50.465930+0000","last_peered":"2024-09-21T07:35:50.465930+0000","last_clean":"2024-09-21T07:35:50.465930+0000","last_became_active":"2024-09-21T07:35:48.472657+0000","last_became_peered":"2024-09-21T07:35:48.472657+0000","last_unstale":"2024-09-21T07:35:50.465930+0000","last_undegraded":"2024-09-21T07:35:50.465930+0000","last_fullsized":"2024-09-21T07:35:50.465930+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:47:22.761346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608559+0000","last_change":"2024-09-21T07:35:27.874640+0000","last_active":"2024-09-21T07:35:50.608559+0000","last_peered":"2024-09-21T07:35:50.608559+0000","last_clean":"2024-09-21T07:35:50.608559+0000","last_became_active":"2024-09-21T07:35:27.874265+0000","last_became_peered":"2024-09-21T07:35:27.874265+0000","last_unstale":"2024-09-21T07:35:50.608559+0000","last_undegraded":"2024-09-21T07:35:50.608559+0000","last_fullsized":"2024-09-21T07:35:50.608559+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:45.754654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466907+0000","last_change":"2024-09-21T07:35:29.207392+0000","last_active":"2024-09-21T07:35:50.466907+0000","last_peered":"2024-09-21T07:35:50.466907+0000","last_clean":"2024-09-21T07:35:50.466907+0000","last_became_active":"2024-09-21T07:35:26.870917+0000","last_became_peered":"2024-09-21T07:35:26.870917+0000","last_unstale":"2024-09-21T07:35:50.466907+0000","last_undegraded":"2024-09-21T07:35:50.466907+0000","last_fullsized":"2024-09-21T07:35:50.466907+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:59.159937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059198499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":94,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608262+0000","last_change":"2024-09-21T07:34:24.258709+0000","last_active":"2024-09-21T07:35:50.608262+0000","last_peered":"2024-09-21T07:35:50.608262+0000","last_clean":"2024-09-21T07:35:50.608262+0000","last_became_active":"2024-09-21T07:34:23.279042+0000","last_became_peered":"2024-09-21T07:34:23.279042+0000","last_unstale":"2024-09-21T07:35:50.608262+0000","last_undegraded":"2024-09-21T07:35:50.608262+0000","last_fullsized":"2024-09-21T07:35:50.608262+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:01.762117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019282100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609852+0000","last_change":"2024-09-21T07:35:49.482465+0000","last_active":"2024-09-21T07:35:50.609852+0000","last_peered":"2024-09-21T07:35:50.609852+0000","last_clean":"2024-09-21T07:35:50.609852+0000","last_became_active":"2024-09-21T07:35:49.482124+0000","last_became_peered":"2024-09-21T07:35:49.482124+0000","last_unstale":"2024-09-21T07:35:50.609852+0000","last_undegraded":"2024-09-21T07:35:50.609852+0000","last_fullsized":"2024-09-21T07:35:50.609852+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:03:24.555132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"77'2","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609723+0000","last_change":"2024-09-21T07:35:48.468248+0000","last_active":"2024-09-21T07:35:50.609723+0000","last_peered":"2024-09-21T07:35:50.609723+0000","last_clean":"2024-09-21T07:35:50.609723+0000","last_became_active":"2024-09-21T07:35:48.467906+0000","last_became_peered":"2024-09-21T07:35:48.467906+0000","last_unstale":"2024-09-21T07:35:50.609723+0000","last_undegraded":"2024-09-21T07:35:50.609723+0000","last_fullsized":"2024-09-21T07:35:50.609723+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:23.050859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.610628+0000","last_change":"2024-09-21T07:35:27.870260+0000","last_active":"2024-09-21T07:35:50.610628+0000","last_peered":"2024-09-21T07:35:50.610628+0000","last_clean":"2024-09-21T07:35:50.610628+0000","last_became_active":"2024-09-21T07:35:27.869448+0000","last_became_peered":"2024-09-21T07:35:27.869448+0000","last_unstale":"2024-09-21T07:35:50.610628+0000","last_undegraded":"2024-09-21T07:35:50.610628+0000","last_fullsized":"2024-09-21T07:35:50.610628+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:16.946790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608401+0000","last_change":"2024-09-21T07:35:29.386367+0000","last_active":"2024-09-21T07:35:50.608401+0000","last_peered":"2024-09-21T07:35:50.608401+0000","last_clean":"2024-09-21T07:35:50.608401+0000","last_became_active":"2024-09-21T07:35:26.879238+0000","last_became_peered":"2024-09-21T07:35:26.879238+0000","last_unstale":"2024-09-21T07:35:50.608401+0000","last_undegraded":"2024-09-21T07:35:50.608401+0000","last_fullsized":"2024-09-21T07:35:50.608401+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:16.528260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092223100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":93,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.611141+0000","last_change":"2024-09-21T07:34:25.085635+0000","last_active":"2024-09-21T07:35:50.611141+0000","last_peered":"2024-09-21T07:35:50.611141+0000","last_clean":"2024-09-21T07:35:50.611141+0000","last_became_active":"2024-09-21T07:34:23.273566+0000","last_became_peered":"2024-09-21T07:34:23.273566+0000","last_unstale":"2024-09-21T07:35:50.611141+0000","last_undegraded":"2024-09-21T07:35:50.611141+0000","last_fullsized":"2024-09-21T07:35:50.611141+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:14.142272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079897300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.611446+0000","last_change":"2024-09-21T07:35:49.485489+0000","last_active":"2024-09-21T07:35:50.611446+0000","last_peered":"2024-09-21T07:35:50.611446+0000","last_clean":"2024-09-21T07:35:50.611446+0000","last_became_active":"2024-09-21T07:35:49.480270+0000","last_became_peered":"2024-09-21T07:35:49.480270+0000","last_unstale":"2024-09-21T07:35:50.611446+0000","last_undegraded":"2024-09-21T07:35:50.611446+0000","last_fullsized":"2024-09-21T07:35:50.611446+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:36.494776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"77'2","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466508+0000","last_change":"2024-09-21T07:35:48.462321+0000","last_active":"2024-09-21T07:35:50.466508+0000","last_peered":"2024-09-21T07:35:50.466508+0000","last_clean":"2024-09-21T07:35:50.466508+0000","last_became_active":"2024-09-21T07:35:48.461989+0000","last_became_peered":"2024-09-21T07:35:48.461989+0000","last_unstale":"2024-09-21T07:35:50.466508+0000","last_undegraded":"2024-09-21T07:35:50.466508+0000","last_fullsized":"2024-09-21T07:35:50.466508+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:25.184983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.610890+0000","last_change":"2024-09-21T07:35:27.878728+0000","last_active":"2024-09-21T07:35:50.610890+0000","last_peered":"2024-09-21T07:35:50.610890+0000","last_clean":"2024-09-21T07:35:50.610890+0000","last_became_active":"2024-09-21T07:35:27.878335+0000","last_became_peered":"2024-09-21T07:35:27.878335+0000","last_unstale":"2024-09-21T07:35:50.610890+0000","last_undegraded":"2024-09-21T07:35:50.610890+0000","last_fullsized":"2024-09-21T07:35:50.610890+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:25:37.731771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":44,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.465958+0000","last_change":"2024-09-21T07:35:28.858418+0000","last_active":"2024-09-21T07:35:50.465958+0000","last_peered":"2024-09-21T07:35:50.465958+0000","last_clean":"2024-09-21T07:35:50.465958+0000","last_became_active":"2024-09-21T07:35:26.876944+0000","last_became_peered":"2024-09-21T07:35:26.876944+0000","last_unstale":"2024-09-21T07:35:50.465958+0000","last_undegraded":"2024-09-21T07:35:50.465958+0000","last_fullsized":"2024-09-21T07:35:50.465958+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:04.514304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090862799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":90,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608502+0000","last_change":"2024-09-21T07:34:24.597926+0000","last_active":"2024-09-21T07:35:50.608502+0000","last_peered":"2024-09-21T07:35:50.608502+0000","last_clean":"2024-09-21T07:35:50.608502+0000","last_became_active":"2024-09-21T07:34:23.268926+0000","last_became_peered":"2024-09-21T07:34:23.268926+0000","last_unstale":"2024-09-21T07:35:50.608502+0000","last_undegraded":"2024-09-21T07:35:50.608502+0000","last_fullsized":"2024-09-21T07:35:50.608502+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:46.190752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101851,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.610487+0000","last_change":"2024-09-21T07:35:49.459448+0000","last_active":"2024-09-21T07:35:50.610487+0000","last_peered":"2024-09-21T07:35:50.610487+0000","last_clean":"2024-09-21T07:35:50.610487+0000","last_became_active":"2024-09-21T07:35:49.459127+0000","last_became_peered":"2024-09-21T07:35:49.459127+0000","last_unstale":"2024-09-21T07:35:50.610487+0000","last_undegraded":"2024-09-21T07:35:50.610487+0000","last_fullsized":"2024-09-21T07:35:50.610487+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:40.407157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"77'4","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.468039+0000","last_change":"2024-09-21T07:35:48.473637+0000","last_active":"2024-09-21T07:35:50.468039+0000","last_peered":"2024-09-21T07:35:50.468039+0000","last_clean":"2024-09-21T07:35:50.468039+0000","last_became_active":"2024-09-21T07:35:48.473220+0000","last_became_peered":"2024-09-21T07:35:48.473220+0000","last_unstale":"2024-09-21T07:35:50.468039+0000","last_undegraded":"2024-09-21T07:35:50.468039+0000","last_fullsized":"2024-09-21T07:35:50.468039+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:47:22.761346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":34,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609073+0000","last_change":"2024-09-21T07:35:27.877426+0000","last_active":"2024-09-21T07:35:50.609073+0000","last_peered":"2024-09-21T07:35:50.609073+0000","last_clean":"2024-09-21T07:35:50.609073+0000","last_became_active":"2024-09-21T07:35:27.877050+0000","last_became_peered":"2024-09-21T07:35:27.877050+0000","last_unstale":"2024-09-21T07:35:50.609073+0000","last_undegraded":"2024-09-21T07:35:50.609073+0000","last_fullsized":"2024-09-21T07:35:50.609073+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:36.268334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609781+0000","last_change":"2024-09-21T07:35:28.862273+0000","last_active":"2024-09-21T07:35:50.609781+0000","last_peered":"2024-09-21T07:35:50.609781+0000","last_clean":"2024-09-21T07:35:50.609781+0000","last_became_active":"2024-09-21T07:35:26.874011+0000","last_became_peered":"2024-09-21T07:35:26.874011+0000","last_unstale":"2024-09-21T07:35:50.609781+0000","last_undegraded":"2024-09-21T07:35:50.609781+0000","last_fullsized":"2024-09-21T07:35:50.609781+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:11.986950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071644799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":95,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609545+0000","last_change":"2024-09-21T07:34:24.259794+0000","last_active":"2024-09-21T07:35:50.609545+0000","last_peered":"2024-09-21T07:35:50.609545+0000","last_clean":"2024-09-21T07:35:50.609545+0000","last_became_active":"2024-09-21T07:34:23.265679+0000","last_became_peered":"2024-09-21T07:34:23.265679+0000","last_unstale":"2024-09-21T07:35:50.609545+0000","last_undegraded":"2024-09-21T07:35:50.609545+0000","last_fullsized":"2024-09-21T07:35:50.609545+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:02.575796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001298518,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.610676+0000","last_change":"2024-09-21T07:35:49.469429+0000","last_active":"2024-09-21T07:35:50.610676+0000","last_peered":"2024-09-21T07:35:50.610676+0000","last_clean":"2024-09-21T07:35:50.610676+0000","last_became_active":"2024-09-21T07:35:49.464946+0000","last_became_peered":"2024-09-21T07:35:49.464946+0000","last_unstale":"2024-09-21T07:35:50.610676+0000","last_undegraded":"2024-09-21T07:35:50.610676+0000","last_fullsized":"2024-09-21T07:35:50.610676+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:30.602494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"77'1","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608922+0000","last_change":"2024-09-21T07:35:48.470671+0000","last_active":"2024-09-21T07:35:50.608922+0000","last_peered":"2024-09-21T07:35:50.608922+0000","last_clean":"2024-09-21T07:35:50.608922+0000","last_became_active":"2024-09-21T07:35:48.470127+0000","last_became_peered":"2024-09-21T07:35:48.470127+0000","last_unstale":"2024-09-21T07:35:50.608922+0000","last_undegraded":"2024-09-21T07:35:50.608922+0000","last_fullsized":"2024-09-21T07:35:50.608922+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:09.916603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.468307+0000","last_change":"2024-09-21T07:35:27.877725+0000","last_active":"2024-09-21T07:35:50.468307+0000","last_peered":"2024-09-21T07:35:50.468307+0000","last_clean":"2024-09-21T07:35:50.468307+0000","last_became_active":"2024-09-21T07:35:27.877398+0000","last_became_peered":"2024-09-21T07:35:27.877398+0000","last_unstale":"2024-09-21T07:35:50.468307+0000","last_undegraded":"2024-09-21T07:35:50.468307+0000","last_fullsized":"2024-09-21T07:35:50.468307+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:05.859762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":43,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.607394+0000","last_change":"2024-09-21T07:35:28.860612+0000","last_active":"2024-09-21T07:35:50.607394+0000","last_peered":"2024-09-21T07:35:50.607394+0000","last_clean":"2024-09-21T07:35:50.607394+0000","last_became_active":"2024-09-21T07:35:26.876562+0000","last_became_peered":"2024-09-21T07:35:26.876562+0000","last_unstale":"2024-09-21T07:35:50.607394+0000","last_undegraded":"2024-09-21T07:35:50.607394+0000","last_fullsized":"2024-09-21T07:35:50.607394+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:06.824331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012252470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":94,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609575+0000","last_change":"2024-09-21T07:34:24.260330+0000","last_active":"2024-09-21T07:35:50.609575+0000","last_peered":"2024-09-21T07:35:50.609575+0000","last_clean":"2024-09-21T07:35:50.609575+0000","last_became_active":"2024-09-21T07:34:23.637210+0000","last_became_peered":"2024-09-21T07:34:23.637210+0000","last_unstale":"2024-09-21T07:35:50.609575+0000","last_undegraded":"2024-09-21T07:35:50.609575+0000","last_fullsized":"2024-09-21T07:35:50.609575+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.488451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001537093,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.607679+0000","last_change":"2024-09-21T07:35:49.478712+0000","last_active":"2024-09-21T07:35:50.607679+0000","last_peered":"2024-09-21T07:35:50.607679+0000","last_clean":"2024-09-21T07:35:50.607679+0000","last_became_active":"2024-09-21T07:35:49.478390+0000","last_became_peered":"2024-09-21T07:35:49.478390+0000","last_unstale":"2024-09-21T07:35:50.607679+0000","last_undegraded":"2024-09-21T07:35:50.607679+0000","last_fullsized":"2024-09-21T07:35:50.607679+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:32:30.142348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.465891+0000","last_change":"2024-09-21T07:35:48.472103+0000","last_active":"2024-09-21T07:35:50.465891+0000","last_peered":"2024-09-21T07:35:50.465891+0000","last_clean":"2024-09-21T07:35:50.465891+0000","last_became_active":"2024-09-21T07:35:48.471732+0000","last_became_peered":"2024-09-21T07:35:48.471732+0000","last_unstale":"2024-09-21T07:35:50.465891+0000","last_undegraded":"2024-09-21T07:35:50.465891+0000","last_fullsized":"2024-09-21T07:35:50.465891+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:41.865640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":35,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608261+0000","last_change":"2024-09-21T07:35:27.874757+0000","last_active":"2024-09-21T07:35:50.608261+0000","last_peered":"2024-09-21T07:35:50.608261+0000","last_clean":"2024-09-21T07:35:50.608261+0000","last_became_active":"2024-09-21T07:35:27.874360+0000","last_became_peered":"2024-09-21T07:35:27.874360+0000","last_unstale":"2024-09-21T07:35:50.608261+0000","last_undegraded":"2024-09-21T07:35:50.608261+0000","last_fullsized":"2024-09-21T07:35:50.608261+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:17.646912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609900+0000","last_change":"2024-09-21T07:35:29.493431+0000","last_active":"2024-09-21T07:35:50.609900+0000","last_peered":"2024-09-21T07:35:50.609900+0000","last_clean":"2024-09-21T07:35:50.609900+0000","last_became_active":"2024-09-21T07:35:26.873691+0000","last_became_peered":"2024-09-21T07:35:26.873691+0000","last_unstale":"2024-09-21T07:35:50.609900+0000","last_undegraded":"2024-09-21T07:35:50.609900+0000","last_fullsized":"2024-09-21T07:35:50.609900+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:51.705051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001162464,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":90,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608778+0000","last_change":"2024-09-21T07:34:24.597286+0000","last_active":"2024-09-21T07:35:50.608778+0000","last_peered":"2024-09-21T07:35:50.608778+0000","last_clean":"2024-09-21T07:35:50.608778+0000","last_became_active":"2024-09-21T07:34:23.267133+0000","last_became_peered":"2024-09-21T07:34:23.267133+0000","last_unstale":"2024-09-21T07:35:50.608778+0000","last_undegraded":"2024-09-21T07:35:50.608778+0000","last_fullsized":"2024-09-21T07:35:50.608778+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:18.257010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609169+0000","last_change":"2024-09-21T07:35:49.478854+0000","last_active":"2024-09-21T07:35:50.609169+0000","last_peered":"2024-09-21T07:35:50.609169+0000","last_clean":"2024-09-21T07:35:50.609169+0000","last_became_active":"2024-09-21T07:35:49.478563+0000","last_became_peered":"2024-09-21T07:35:49.478563+0000","last_unstale":"2024-09-21T07:35:50.609169+0000","last_undegraded":"2024-09-21T07:35:50.609169+0000","last_fullsized":"2024-09-21T07:35:50.609169+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:16:19.354123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"77'8","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.465811+0000","last_change":"2024-09-21T07:35:48.473957+0000","last_active":"2024-09-21T07:35:50.465811+0000","last_peered":"2024-09-21T07:35:50.465811+0000","last_clean":"2024-09-21T07:35:50.465811+0000","last_became_active":"2024-09-21T07:35:48.470611+0000","last_became_peered":"2024-09-21T07:35:48.470611+0000","last_unstale":"2024-09-21T07:35:50.465811+0000","last_undegraded":"2024-09-21T07:35:50.465811+0000","last_fullsized":"2024-09-21T07:35:50.465811+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:15:41.920172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466832+0000","last_change":"2024-09-21T07:35:27.864458+0000","last_active":"2024-09-21T07:35:50.466832+0000","last_peered":"2024-09-21T07:35:50.466832+0000","last_clean":"2024-09-21T07:35:50.466832+0000","last_became_active":"2024-09-21T07:35:27.864136+0000","last_became_peered":"2024-09-21T07:35:27.864136+0000","last_unstale":"2024-09-21T07:35:50.466832+0000","last_undegraded":"2024-09-21T07:35:50.466832+0000","last_fullsized":"2024-09-21T07:35:50.466832+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:20.162240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":44,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.467235+0000","last_change":"2024-09-21T07:35:28.858784+0000","last_active":"2024-09-21T07:35:50.467235+0000","last_peered":"2024-09-21T07:35:50.467235+0000","last_clean":"2024-09-21T07:35:50.467235+0000","last_became_active":"2024-09-21T07:35:26.876890+0000","last_became_peered":"2024-09-21T07:35:26.876890+0000","last_unstale":"2024-09-21T07:35:50.467235+0000","last_undegraded":"2024-09-21T07:35:50.467235+0000","last_fullsized":"2024-09-21T07:35:50.467235+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:40.738327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064543499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":94,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609260+0000","last_change":"2024-09-21T07:34:24.258712+0000","last_active":"2024-09-21T07:35:50.609260+0000","last_peered":"2024-09-21T07:35:50.609260+0000","last_clean":"2024-09-21T07:35:50.609260+0000","last_became_active":"2024-09-21T07:34:23.274692+0000","last_became_peered":"2024-09-21T07:34:23.274692+0000","last_unstale":"2024-09-21T07:35:50.609260+0000","last_undegraded":"2024-09-21T07:35:50.609260+0000","last_fullsized":"2024-09-21T07:35:50.609260+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:43.279454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018051479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466479+0000","last_change":"2024-09-21T07:35:49.489413+0000","last_active":"2024-09-21T07:35:50.466479+0000","last_peered":"2024-09-21T07:35:50.466479+0000","last_clean":"2024-09-21T07:35:50.466479+0000","last_became_active":"2024-09-21T07:35:49.487895+0000","last_became_peered":"2024-09-21T07:35:49.487895+0000","last_unstale":"2024-09-21T07:35:50.466479+0000","last_undegraded":"2024-09-21T07:35:50.466479+0000","last_fullsized":"2024-09-21T07:35:50.466479+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:39.106814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"77'2","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609243+0000","last_change":"2024-09-21T07:35:48.473340+0000","last_active":"2024-09-21T07:35:50.609243+0000","last_peered":"2024-09-21T07:35:50.609243+0000","last_clean":"2024-09-21T07:35:50.609243+0000","last_became_active":"2024-09-21T07:35:48.473063+0000","last_became_peered":"2024-09-21T07:35:48.473063+0000","last_unstale":"2024-09-21T07:35:50.609243+0000","last_undegraded":"2024-09-21T07:35:50.609243+0000","last_fullsized":"2024-09-21T07:35:50.609243+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:30.813114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608660+0000","last_change":"2024-09-21T07:35:27.878020+0000","last_active":"2024-09-21T07:35:50.608660+0000","last_peered":"2024-09-21T07:35:50.608660+0000","last_clean":"2024-09-21T07:35:50.608660+0000","last_became_active":"2024-09-21T07:35:27.877532+0000","last_became_peered":"2024-09-21T07:35:27.877532+0000","last_unstale":"2024-09-21T07:35:50.608660+0000","last_undegraded":"2024-09-21T07:35:50.608660+0000","last_fullsized":"2024-09-21T07:35:50.608660+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:19.756073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":43,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.610846+0000","last_change":"2024-09-21T07:35:28.861049+0000","last_active":"2024-09-21T07:35:50.610846+0000","last_peered":"2024-09-21T07:35:50.610846+0000","last_clean":"2024-09-21T07:35:50.610846+0000","last_became_active":"2024-09-21T07:35:26.876819+0000","last_became_peered":"2024-09-21T07:35:26.876819+0000","last_unstale":"2024-09-21T07:35:50.610846+0000","last_undegraded":"2024-09-21T07:35:50.610846+0000","last_fullsized":"2024-09-21T07:35:50.610846+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:19.633803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055780599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":93,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608368+0000","last_change":"2024-09-21T07:34:25.084849+0000","last_active":"2024-09-21T07:35:50.608368+0000","last_peered":"2024-09-21T07:35:50.608368+0000","last_clean":"2024-09-21T07:35:50.608368+0000","last_became_active":"2024-09-21T07:34:23.277713+0000","last_became_peered":"2024-09-21T07:34:23.277713+0000","last_unstale":"2024-09-21T07:35:50.608368+0000","last_undegraded":"2024-09-21T07:35:50.608368+0000","last_fullsized":"2024-09-21T07:35:50.608368+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:05.811810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059079499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.607623+0000","last_change":"2024-09-21T07:35:49.468874+0000","last_active":"2024-09-21T07:35:50.607623+0000","last_peered":"2024-09-21T07:35:50.607623+0000","last_clean":"2024-09-21T07:35:50.607623+0000","last_became_active":"2024-09-21T07:35:49.468514+0000","last_became_peered":"2024-09-21T07:35:49.468514+0000","last_unstale":"2024-09-21T07:35:50.607623+0000","last_undegraded":"2024-09-21T07:35:50.607623+0000","last_fullsized":"2024-09-21T07:35:50.607623+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:35.690753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.610141+0000","last_change":"2024-09-21T07:35:48.474361+0000","last_active":"2024-09-21T07:35:50.610141+0000","last_peered":"2024-09-21T07:35:50.610141+0000","last_clean":"2024-09-21T07:35:50.610141+0000","last_became_active":"2024-09-21T07:35:48.474032+0000","last_became_peered":"2024-09-21T07:35:48.474032+0000","last_unstale":"2024-09-21T07:35:50.610141+0000","last_undegraded":"2024-09-21T07:35:50.610141+0000","last_fullsized":"2024-09-21T07:35:50.610141+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:16.959573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":36,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466224+0000","last_change":"2024-09-21T07:35:27.884248+0000","last_active":"2024-09-21T07:35:50.466224+0000","last_peered":"2024-09-21T07:35:50.466224+0000","last_clean":"2024-09-21T07:35:50.466224+0000","last_became_active":"2024-09-21T07:35:27.883861+0000","last_became_peered":"2024-09-21T07:35:27.883861+0000","last_unstale":"2024-09-21T07:35:50.466224+0000","last_undegraded":"2024-09-21T07:35:50.466224+0000","last_fullsized":"2024-09-21T07:35:50.466224+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":40,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466794+0000","last_change":"2024-09-21T07:35:29.207135+0000","last_active":"2024-09-21T07:35:50.466794+0000","last_peered":"2024-09-21T07:35:50.466794+0000","last_clean":"2024-09-21T07:35:50.466794+0000","last_became_active":"2024-09-21T07:35:26.872787+0000","last_became_peered":"2024-09-21T07:35:26.872787+0000","last_unstale":"2024-09-21T07:35:50.466794+0000","last_undegraded":"2024-09-21T07:35:50.466794+0000","last_fullsized":"2024-09-21T07:35:50.466794+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:27.543217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011157509999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":96,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466956+0000","last_change":"2024-09-21T07:34:24.538610+0000","last_active":"2024-09-21T07:35:50.466956+0000","last_peered":"2024-09-21T07:35:50.466956+0000","last_clean":"2024-09-21T07:35:50.466956+0000","last_became_active":"2024-09-21T07:34:23.259700+0000","last_became_peered":"2024-09-21T07:34:23.259700+0000","last_unstale":"2024-09-21T07:35:50.466956+0000","last_undegraded":"2024-09-21T07:35:50.466956+0000","last_fullsized":"2024-09-21T07:35:50.466956+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:57.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070522399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609610+0000","last_change":"2024-09-21T07:35:49.488287+0000","last_active":"2024-09-21T07:35:50.609610+0000","last_peered":"2024-09-21T07:35:50.609610+0000","last_clean":"2024-09-21T07:35:50.609610+0000","last_became_active":"2024-09-21T07:35:49.487928+0000","last_became_peered":"2024-09-21T07:35:49.487928+0000","last_unstale":"2024-09-21T07:35:50.609610+0000","last_undegraded":"2024-09-21T07:35:50.609610+0000","last_fullsized":"2024-09-21T07:35:50.609610+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:00.876603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"77'3","reported_seq":18,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609289+0000","last_change":"2024-09-21T07:35:48.478146+0000","last_active":"2024-09-21T07:35:50.609289+0000","last_peered":"2024-09-21T07:35:50.609289+0000","last_clean":"2024-09-21T07:35:50.609289+0000","last_became_active":"2024-09-21T07:35:48.477767+0000","last_became_peered":"2024-09-21T07:35:48.477767+0000","last_unstale":"2024-09-21T07:35:50.609289+0000","last_undegraded":"2024-09-21T07:35:50.609289+0000","last_fullsized":"2024-09-21T07:35:50.609289+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:35:53.405184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466361+0000","last_change":"2024-09-21T07:35:27.868024+0000","last_active":"2024-09-21T07:35:50.466361+0000","last_peered":"2024-09-21T07:35:50.466361+0000","last_clean":"2024-09-21T07:35:50.466361+0000","last_became_active":"2024-09-21T07:35:27.867339+0000","last_became_peered":"2024-09-21T07:35:27.867339+0000","last_unstale":"2024-09-21T07:35:50.466361+0000","last_undegraded":"2024-09-21T07:35:50.466361+0000","last_fullsized":"2024-09-21T07:35:50.466361+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":44,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466059+0000","last_change":"2024-09-21T07:35:28.858505+0000","last_active":"2024-09-21T07:35:50.466059+0000","last_peered":"2024-09-21T07:35:50.466059+0000","last_clean":"2024-09-21T07:35:50.466059+0000","last_became_active":"2024-09-21T07:35:26.877320+0000","last_became_peered":"2024-09-21T07:35:26.877320+0000","last_unstale":"2024-09-21T07:35:50.466059+0000","last_undegraded":"2024-09-21T07:35:50.466059+0000","last_fullsized":"2024-09-21T07:35:50.466059+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:53.672782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244119999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":90,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609974+0000","last_change":"2024-09-21T07:34:24.597597+0000","last_active":"2024-09-21T07:35:50.609974+0000","last_peered":"2024-09-21T07:35:50.609974+0000","last_clean":"2024-09-21T07:35:50.609974+0000","last_became_active":"2024-09-21T07:34:23.635204+0000","last_became_peered":"2024-09-21T07:34:23.635204+0000","last_unstale":"2024-09-21T07:35:50.609974+0000","last_undegraded":"2024-09-21T07:35:50.609974+0000","last_fullsized":"2024-09-21T07:35:50.609974+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:48.644789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081139199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.467242+0000","last_change":"2024-09-21T07:35:49.476856+0000","last_active":"2024-09-21T07:35:50.467242+0000","last_peered":"2024-09-21T07:35:50.467242+0000","last_clean":"2024-09-21T07:35:50.467242+0000","last_became_active":"2024-09-21T07:35:49.476517+0000","last_became_peered":"2024-09-21T07:35:49.476517+0000","last_unstale":"2024-09-21T07:35:50.467242+0000","last_undegraded":"2024-09-21T07:35:50.467242+0000","last_fullsized":"2024-09-21T07:35:50.467242+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:17.962368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"77'6","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.610567+0000","last_change":"2024-09-21T07:35:48.475454+0000","last_active":"2024-09-21T07:35:50.610567+0000","last_peered":"2024-09-21T07:35:50.610567+0000","last_clean":"2024-09-21T07:35:50.610567+0000","last_became_active":"2024-09-21T07:35:48.474874+0000","last_became_peered":"2024-09-21T07:35:48.474874+0000","last_unstale":"2024-09-21T07:35:50.610567+0000","last_undegraded":"2024-09-21T07:35:50.610567+0000","last_fullsized":"2024-09-21T07:35:50.610567+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:20.723211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.465588+0000","last_change":"2024-09-21T07:35:27.877503+0000","last_active":"2024-09-21T07:35:50.465588+0000","last_peered":"2024-09-21T07:35:50.465588+0000","last_clean":"2024-09-21T07:35:50.465588+0000","last_became_active":"2024-09-21T07:35:27.875775+0000","last_became_peered":"2024-09-21T07:35:27.875775+0000","last_unstale":"2024-09-21T07:35:50.465588+0000","last_undegraded":"2024-09-21T07:35:50.465588+0000","last_fullsized":"2024-09-21T07:35:50.465588+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:49.291023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608699+0000","last_change":"2024-09-21T07:35:29.491412+0000","last_active":"2024-09-21T07:35:50.608699+0000","last_peered":"2024-09-21T07:35:50.608699+0000","last_clean":"2024-09-21T07:35:50.608699+0000","last_became_active":"2024-09-21T07:35:26.876404+0000","last_became_peered":"2024-09-21T07:35:26.876404+0000","last_unstale":"2024-09-21T07:35:50.608699+0000","last_undegraded":"2024-09-21T07:35:50.608699+0000","last_fullsized":"2024-09-21T07:35:50.608699+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063918000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":94,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608970+0000","last_change":"2024-09-21T07:34:24.259026+0000","last_active":"2024-09-21T07:35:50.608970+0000","last_peered":"2024-09-21T07:35:50.608970+0000","last_clean":"2024-09-21T07:35:50.608970+0000","last_became_active":"2024-09-21T07:34:23.635456+0000","last_became_peered":"2024-09-21T07:34:23.635456+0000","last_unstale":"2024-09-21T07:35:50.608970+0000","last_undegraded":"2024-09-21T07:35:50.608970+0000","last_fullsized":"2024-09-21T07:35:50.608970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:09.592752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018654419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608299+0000","last_change":"2024-09-21T07:35:49.482878+0000","last_active":"2024-09-21T07:35:50.608299+0000","last_peered":"2024-09-21T07:35:50.608299+0000","last_clean":"2024-09-21T07:35:50.608299+0000","last_became_active":"2024-09-21T07:35:49.482570+0000","last_became_peered":"2024-09-21T07:35:49.482570+0000","last_unstale":"2024-09-21T07:35:50.608299+0000","last_undegraded":"2024-09-21T07:35:50.608299+0000","last_fullsized":"2024-09-21T07:35:50.608299+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:11:10.283242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.467656+0000","last_change":"2024-09-21T07:35:48.472867+0000","last_active":"2024-09-21T07:35:50.467656+0000","last_peered":"2024-09-21T07:35:50.467656+0000","last_clean":"2024-09-21T07:35:50.467656+0000","last_became_active":"2024-09-21T07:35:48.472432+0000","last_became_peered":"2024-09-21T07:35:48.472432+0000","last_unstale":"2024-09-21T07:35:50.467656+0000","last_undegraded":"2024-09-21T07:35:50.467656+0000","last_fullsized":"2024-09-21T07:35:50.467656+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:40:59.551317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.468174+0000","last_change":"2024-09-21T07:35:27.871984+0000","last_active":"2024-09-21T07:35:50.468174+0000","last_peered":"2024-09-21T07:35:50.468174+0000","last_clean":"2024-09-21T07:35:50.468174+0000","last_became_active":"2024-09-21T07:35:27.870336+0000","last_became_peered":"2024-09-21T07:35:27.870336+0000","last_unstale":"2024-09-21T07:35:50.468174+0000","last_undegraded":"2024-09-21T07:35:50.468174+0000","last_fullsized":"2024-09-21T07:35:50.468174+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:57.120479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":43,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608195+0000","last_change":"2024-09-21T07:35:28.860351+0000","last_active":"2024-09-21T07:35:50.608195+0000","last_peered":"2024-09-21T07:35:50.608195+0000","last_clean":"2024-09-21T07:35:50.608195+0000","last_became_active":"2024-09-21T07:35:26.872046+0000","last_became_peered":"2024-09-21T07:35:26.872046+0000","last_unstale":"2024-09-21T07:35:50.608195+0000","last_undegraded":"2024-09-21T07:35:50.608195+0000","last_fullsized":"2024-09-21T07:35:50.608195+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:47.074327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098384399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":94,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608640+0000","last_change":"2024-09-21T07:34:24.260300+0000","last_active":"2024-09-21T07:35:50.608640+0000","last_peered":"2024-09-21T07:35:50.608640+0000","last_clean":"2024-09-21T07:35:50.608640+0000","last_became_active":"2024-09-21T07:34:23.270128+0000","last_became_peered":"2024-09-21T07:34:23.270128+0000","last_unstale":"2024-09-21T07:35:50.608640+0000","last_undegraded":"2024-09-21T07:35:50.608640+0000","last_fullsized":"2024-09-21T07:35:50.608640+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:25.260931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00176651,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609183+0000","last_change":"2024-09-21T07:35:49.470810+0000","last_active":"2024-09-21T07:35:50.609183+0000","last_peered":"2024-09-21T07:35:50.609183+0000","last_clean":"2024-09-21T07:35:50.609183+0000","last_became_active":"2024-09-21T07:35:49.470423+0000","last_became_peered":"2024-09-21T07:35:49.470423+0000","last_unstale":"2024-09-21T07:35:50.609183+0000","last_undegraded":"2024-09-21T07:35:50.609183+0000","last_fullsized":"2024-09-21T07:35:50.609183+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:48.430484+0000","last_clean_scrub_stamp":"2024-09-21T07:35:48.430484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:32.113551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.609724+0000","last_change":"2024-09-21T07:35:48.464501+0000","last_active":"2024-09-21T07:35:50.609724+0000","last_peered":"2024-09-21T07:35:50.609724+0000","last_clean":"2024-09-21T07:35:50.609724+0000","last_became_active":"2024-09-21T07:35:48.464249+0000","last_became_peered":"2024-09-21T07:35:48.464249+0000","last_unstale":"2024-09-21T07:35:50.609724+0000","last_undegraded":"2024-09-21T07:35:50.609724+0000","last_fullsized":"2024-09-21T07:35:50.609724+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:47.427688+0000","last_clean_scrub_stamp":"2024-09-21T07:35:47.427688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:49.239670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":33,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.468310+0000","last_change":"2024-09-21T07:35:27.877861+0000","last_active":"2024-09-21T07:35:50.468310+0000","last_peered":"2024-09-21T07:35:50.468310+0000","last_clean":"2024-09-21T07:35:50.468310+0000","last_became_active":"2024-09-21T07:35:27.877434+0000","last_became_peered":"2024-09-21T07:35:27.877434+0000","last_unstale":"2024-09-21T07:35:50.468310+0000","last_undegraded":"2024-09-21T07:35:50.468310+0000","last_fullsized":"2024-09-21T07:35:50.468310+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.608707+0000","last_change":"2024-09-21T07:35:28.862954+0000","last_active":"2024-09-21T07:35:50.608707+0000","last_peered":"2024-09-21T07:35:50.608707+0000","last_clean":"2024-09-21T07:35:50.608707+0000","last_became_active":"2024-09-21T07:35:26.870613+0000","last_became_peered":"2024-09-21T07:35:26.870613+0000","last_unstale":"2024-09-21T07:35:50.608707+0000","last_undegraded":"2024-09-21T07:35:50.608707+0000","last_fullsized":"2024-09-21T07:35:50.608707+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:06.901043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059842100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":94,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-21T07:35:50.466911+0000","last_change":"2024-09-21T07:34:25.028904+0000","last_active":"2024-09-21T07:35:50.466911+0000","last_peered":"2024-09-21T07:35:50.466911+0000","last_clean":"2024-09-21T07:35:50.466911+0000","last_became_active":"2024-09-21T07:34:23.260942+0000","last_became_peered":"2024-09-21T07:34:23.260942+0000","last_unstale":"2024-09-21T07:35:50.466911+0000","last_undegraded":"2024-09-21T07:35:50.466911+0000","last_fullsized":"2024-09-21T07:35:50.466911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:22.709328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060553899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509477,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32704,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1544192,"data_stored":1024165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509479,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":962560,"data_stored":435957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509477,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32112,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339728,"statfs":{"total":96636764160,"available":96603881472,"internally_reserved":0,"allocated":942080,"data_stored":433741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509478,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32112,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339728,"statfs":{"total":96636764160,"available":96603881472,"internally_reserved":0,"allocated":942080,"data_stored":433741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509479,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":946176,"data_stored":433811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509479,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32720,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339120,"statfs":{"total":96636764160,"available":96603258880,"internally_reserved":0,"allocated":1560576,"data_stored":1026427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542182,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32124,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339716,"statfs":{"total":96636764160,"available":96603869184,"internally_reserved":0,"allocated":950272,"data_stored":433901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542181,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":942080,"data_stored":433741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:35:54.468 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T07:35:54.468 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T07:35:54.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.734+0000 7f475a400640 1 -- 172.21.15.39:0/2594118515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4738003fd0 msgr2=0x7f4738004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:54.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.734+0000 7f475a400640 1 --2- 172.21.15.39:0/2594118515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4738003fd0 0x7f4738004800 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f4754073580 tx=0x7f4738005830 comp rx=0 tx=0).stop 2024-09-21T07:35:54.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.734+0000 7f4753e00640 1 -- 172.21.15.39:0/2594118515 reap_dead start 2024-09-21T07:35:54.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.734+0000 7f475a400640 1 -- 172.21.15.39:0/2594118515 shutdown_connections 2024-09-21T07:35:54.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.734+0000 7f475a400640 1 -- 172.21.15.39:0/2594118515 >> 172.21.15.39:0/2594118515 conn(0x7f47540b1cb0 msgr2=0x7f47540b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:54.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.734+0000 7f475a400640 1 -- 172.21.15.39:0/2594118515 shutdown_connections 2024-09-21T07:35:54.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.734+0000 7f475a400640 1 -- 172.21.15.39:0/2594118515 wait complete. 2024-09-21T07:35:54.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.736+0000 7f475a400640 1 Processor -- start 2024-09-21T07:35:54.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.736+0000 7f475a400640 1 -- start start 2024-09-21T07:35:54.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.738+0000 7f475a400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4738003fd0 0x7f47540c97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:54.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.738+0000 7f475a400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47540cb8a0 0x7f47541f23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:54.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.738+0000 7f475a400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47540cbcb0 0x7f47541f28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:54.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.738+0000 7f475a400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f47540b35d0 con 0x7f47540cb8a0 2024-09-21T07:35:54.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.738+0000 7f475a400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f475415cc60 con 0x7f47540cbcb0 2024-09-21T07:35:54.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.738+0000 7f475a400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f47540b3750 con 0x7f4738003fd0 2024-09-21T07:35:54.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.738+0000 7f474be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47540cb8a0 0x7f47541f23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:54.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.738+0000 7f474be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47540cb8a0 0x7f47541f23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45560/0 (socket says 172.21.15.39:45560) 2024-09-21T07:35:54.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.738+0000 7f474be00640 1 -- 172.21.15.39:0/3876302922 learned_addr learned my addr 172.21.15.39:0/3876302922 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:54.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.738+0000 7f4753e00640 1 --2- 172.21.15.39:0/3876302922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47540cbcb0 0x7f47541f28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:54.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.738+0000 7f4753400640 1 --2- 172.21.15.39:0/3876302922 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4738003fd0 0x7f47540c97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:54.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.739+0000 7f474be00640 1 -- 172.21.15.39:0/3876302922 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4738003fd0 msgr2=0x7f47540c97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:54.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.739+0000 7f474be00640 1 --2- 172.21.15.39:0/3876302922 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4738003fd0 0x7f47540c97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:54.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.739+0000 7f474be00640 1 -- 172.21.15.39:0/3876302922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47540cbcb0 msgr2=0x7f47541f28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:54.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.739+0000 7f474be00640 1 --2- 172.21.15.39:0/3876302922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47540cbcb0 0x7f47541f28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:54.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.739+0000 7f474be00640 1 -- 172.21.15.39:0/3876302922 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47541f5500 con 0x7f47540cb8a0 2024-09-21T07:35:54.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.739+0000 7f474be00640 1 --2- 172.21.15.39:0/3876302922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47540cb8a0 0x7f47541f23a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f473c02b0a0 tx=0x7f473c006d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:54.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.739+0000 7f474b400640 1 -- 172.21.15.39:0/3876302922 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c032070 con 0x7f47540cb8a0 2024-09-21T07:35:54.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.739+0000 7f474b400640 1 -- 172.21.15.39:0/3876302922 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f473c0038f0 con 0x7f47540cb8a0 2024-09-21T07:35:54.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.739+0000 7f475a400640 1 -- 172.21.15.39:0/3876302922 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47541f57e0 con 0x7f47540cb8a0 2024-09-21T07:35:54.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.739+0000 7f474b400640 1 -- 172.21.15.39:0/3876302922 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c003d40 con 0x7f47540cb8a0 2024-09-21T07:35:54.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.740+0000 7f475a400640 1 -- 172.21.15.39:0/3876302922 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47541f5da0 con 0x7f47540cb8a0 2024-09-21T07:35:54.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.741+0000 7f475a400640 1 -- 172.21.15.39:0/3876302922 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47540c9fe0 con 0x7f47540cb8a0 2024-09-21T07:35:54.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.742+0000 7f474b400640 1 -- 172.21.15.39:0/3876302922 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f473c035020 con 0x7f47540cb8a0 2024-09-21T07:35:54.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.743+0000 7f474b400640 1 --2- 172.21.15.39:0/3876302922 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4730078b90 0x7f473007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:54.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.743+0000 7f474b400640 1 -- 172.21.15.39:0/3876302922 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6772+0+0 (secure 0 0 0) 0x7f473c0bba00 con 0x7f47540cb8a0 2024-09-21T07:35:54.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.743+0000 7f4753400640 1 --2- 172.21.15.39:0/3876302922 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4730078b90 0x7f473007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:54.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.744+0000 7f4753400640 1 --2- 172.21.15.39:0/3876302922 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4730078b90 0x7f473007b0a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4744009bb0 tx=0x7f4744005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:54.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:54.747+0000 7f474b400640 1 -- 172.21.15.39:0/3876302922 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f473c046060 con 0x7f47540cb8a0 2024-09-21T07:35:55.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.017+0000 7f475a400640 1 -- 172.21.15.39:0/3876302922 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f47540b3090 con 0x7f47540cb8a0 2024-09-21T07:35:55.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.021+0000 7f474b400640 1 -- 172.21.15.39:0/3876302922 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) ==== 190+0+147 (secure 0 0 0) 0x7f473c083aa0 con 0x7f47540cb8a0 2024-09-21T07:35:55.023 INFO:teuthology.orchestra.run.smithi039.stdout:[client.0] 2024-09-21T07:35:55.023 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQB0du5mGfb1DBAA/aSMKR5or3xy53rB/iUjpA== 2024-09-21T07:35:55.023 INFO:teuthology.orchestra.run.smithi039.stdout: caps mds = "allow" 2024-09-21T07:35:55.023 INFO:teuthology.orchestra.run.smithi039.stdout: caps mon = "allow r" 2024-09-21T07:35:55.024 INFO:teuthology.orchestra.run.smithi039.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T07:35:55.024 INFO:teuthology.orchestra.run.smithi039.stderr:updated caps for client.0 2024-09-21T07:35:55.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.023+0000 7f475a400640 1 -- 172.21.15.39:0/3876302922 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4730078b90 msgr2=0x7f473007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:55.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.023+0000 7f475a400640 1 --2- 172.21.15.39:0/3876302922 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4730078b90 0x7f473007b0a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4744009bb0 tx=0x7f4744005f90 comp rx=0 tx=0).stop 2024-09-21T07:35:55.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.023+0000 7f475a400640 1 -- 172.21.15.39:0/3876302922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47540cb8a0 msgr2=0x7f47541f23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:55.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.023+0000 7f475a400640 1 --2- 172.21.15.39:0/3876302922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47540cb8a0 0x7f47541f23a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f473c02b0a0 tx=0x7f473c006d80 comp rx=0 tx=0).stop 2024-09-21T07:35:55.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.024+0000 7f475a400640 1 -- 172.21.15.39:0/3876302922 shutdown_connections 2024-09-21T07:35:55.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.024+0000 7f475a400640 1 --2- 172.21.15.39:0/3876302922 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4730078b90 0x7f473007b0a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:55.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.024+0000 7f475a400640 1 --2- 172.21.15.39:0/3876302922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47540cbcb0 0x7f47541f28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:55.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.024+0000 7f475a400640 1 --2- 172.21.15.39:0/3876302922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47540cb8a0 0x7f47541f23a0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:55.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.024+0000 7f475a400640 1 --2- 172.21.15.39:0/3876302922 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4738003fd0 0x7f47540c97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:55.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.024+0000 7f475a400640 1 -- 172.21.15.39:0/3876302922 >> 172.21.15.39:0/3876302922 conn(0x7f47540b1cb0 msgr2=0x7f475415cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:55.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.024+0000 7f475a400640 1 -- 172.21.15.39:0/3876302922 shutdown_connections 2024-09-21T07:35:55.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.024+0000 7f475a400640 1 -- 172.21.15.39:0/3876302922 wait complete. 2024-09-21T07:35:55.047 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:35:55.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.271+0000 7fdf17400640 1 -- 172.21.15.39:0/2570418531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf10073170 msgr2=0x7fdf1015c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:55.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.271+0000 7fdf17400640 1 --2- 172.21.15.39:0/2570418531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf10073170 0x7fdf1015c780 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdf04009830 tx=0x7fdf0402a480 comp rx=0 tx=0).stop 2024-09-21T07:35:55.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.271+0000 7fdf17400640 1 -- 172.21.15.39:0/2570418531 shutdown_connections 2024-09-21T07:35:55.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.271+0000 7fdf17400640 1 --2- 172.21.15.39:0/2570418531 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf10070030 0x7fdf10074640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:55.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.271+0000 7fdf17400640 1 --2- 172.21.15.39:0/2570418531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf10073170 0x7fdf1015c780 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:55.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.271+0000 7fdf17400640 1 -- 172.21.15.39:0/2570418531 >> 172.21.15.39:0/2570418531 conn(0x7fdf100b1cb0 msgr2=0x7fdf100b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:55.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.271+0000 7fdf17400640 1 -- 172.21.15.39:0/2570418531 shutdown_connections 2024-09-21T07:35:55.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.271+0000 7fdf17400640 1 -- 172.21.15.39:0/2570418531 wait complete. 2024-09-21T07:35:55.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.273+0000 7fdf17400640 1 Processor -- start 2024-09-21T07:35:55.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.273+0000 7fdf17400640 1 -- start start 2024-09-21T07:35:55.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.275+0000 7fdf17400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf10070030 0x7fdf101f35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:55.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.275+0000 7fdf17400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf10070440 0x7fdf101e50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:55.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.275+0000 7fdf17400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf10073170 0x7fdf101e55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:55.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.275+0000 7fdf17400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1006f5a0 con 0x7fdf10070440 2024-09-21T07:35:55.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.275+0000 7fdf17400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdf10074f80 con 0x7fdf10070030 2024-09-21T07:35:55.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.275+0000 7fdf17400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fdf1006f720 con 0x7fdf10073170 2024-09-21T07:35:55.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.275+0000 7fdf0fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf10070440 0x7fdf101e50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:55.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.275+0000 7fdf0fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf10070440 0x7fdf101e50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45568/0 (socket says 172.21.15.39:45568) 2024-09-21T07:35:55.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.275+0000 7fdf0fe00640 1 -- 172.21.15.39:0/4066210132 learned_addr learned my addr 172.21.15.39:0/4066210132 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:55.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.275+0000 7fdf14a00640 1 --2- 172.21.15.39:0/4066210132 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf10070030 0x7fdf101f35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:55.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.275+0000 7fdf15400640 1 --2- 172.21.15.39:0/4066210132 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf10073170 0x7fdf101e55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:55.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.276+0000 7fdf0fe00640 1 -- 172.21.15.39:0/4066210132 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf10073170 msgr2=0x7fdf101e55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:55.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.276+0000 7fdf0fe00640 1 --2- 172.21.15.39:0/4066210132 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf10073170 0x7fdf101e55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:55.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.276+0000 7fdf0fe00640 1 -- 172.21.15.39:0/4066210132 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf10070030 msgr2=0x7fdf101f35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:55.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.276+0000 7fdf0fe00640 1 --2- 172.21.15.39:0/4066210132 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf10070030 0x7fdf101f35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:55.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.276+0000 7fdf0fe00640 1 -- 172.21.15.39:0/4066210132 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf1009dec0 con 0x7fdf10070440 2024-09-21T07:35:55.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.276+0000 7fdf15400640 1 --2- 172.21.15.39:0/4066210132 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf10073170 0x7fdf101e55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:35:55.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.276+0000 7fdf0fe00640 1 --2- 172.21.15.39:0/4066210132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf10070440 0x7fdf101e50b0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fdefc00e790 tx=0x7fdefc00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:55.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.276+0000 7fdf0cc00640 1 -- 172.21.15.39:0/4066210132 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdefc00cac0 con 0x7fdf10070440 2024-09-21T07:35:55.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.276+0000 7fdf0cc00640 1 -- 172.21.15.39:0/4066210132 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdefc00cc60 con 0x7fdf10070440 2024-09-21T07:35:55.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.276+0000 7fdf0cc00640 1 -- 172.21.15.39:0/4066210132 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdefc005280 con 0x7fdf10070440 2024-09-21T07:35:55.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.276+0000 7fdf17400640 1 -- 172.21.15.39:0/4066210132 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf101e7020 con 0x7fdf10070440 2024-09-21T07:35:55.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.276+0000 7fdf17400640 1 -- 172.21.15.39:0/4066210132 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf101e75e0 con 0x7fdf10070440 2024-09-21T07:35:55.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.277+0000 7fdf17400640 1 -- 172.21.15.39:0/4066210132 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf1006fc40 con 0x7fdf10070440 2024-09-21T07:35:55.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.279+0000 7fdf0cc00640 1 -- 172.21.15.39:0/4066210132 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdefc005420 con 0x7fdf10070440 2024-09-21T07:35:55.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.280+0000 7fdf0cc00640 1 --2- 172.21.15.39:0/4066210132 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdef0078bc0 0x7fdef007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:55.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.280+0000 7fdf0cc00640 1 -- 172.21.15.39:0/4066210132 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6772+0+0 (secure 0 0 0) 0x7fdefc09b970 con 0x7fdf10070440 2024-09-21T07:35:55.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.280+0000 7fdf14a00640 1 --2- 172.21.15.39:0/4066210132 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdef0078bc0 0x7fdef007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:55.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.281+0000 7fdf14a00640 1 --2- 172.21.15.39:0/4066210132 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdef0078bc0 0x7fdef007b0d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdf04034000 tx=0x7fdf04004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:55.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.284+0000 7fdf0cc00640 1 -- 172.21.15.39:0/4066210132 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdefc063a10 con 0x7fdf10070440 2024-09-21T07:35:55.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.522+0000 7fdf17400640 1 -- 172.21.15.39:0/4066210132 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf100b14d0 con 0x7fdf10070440 2024-09-21T07:35:55.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.522+0000 7fdf0cc00640 1 -- 172.21.15.39:0/4066210132 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5225 (secure 0 0 0) 0x7fdefc063310 con 0x7fdf10070440 2024-09-21T07:35:55.524 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:35:55.525 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":44,"btime":"2024-09-21T07:35:52:472978+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5426,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/707367131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":707367131},{"type":"v1","addr":"172.21.15.39:6835","nonce":707367131}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5429,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/784956164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":784956164},{"type":"v1","addr":"172.21.15.148:6836","nonce":784956164}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5432,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3742990359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3742990359},{"type":"v1","addr":"172.21.15.148:6837","nonce":3742990359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:35:48.965416+0000","modified":"2024-09-21T07:35:51.980637+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5842},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5842":{"gid":5842,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/1032917299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":1032917299},{"type":"v1","addr":"172.21.15.39:6834","nonce":1032917299}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5842,"qdb_cluster":[5842]},"id":5}]} 2024-09-21T07:35:55.525 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 44 2024-09-21T07:35:55.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.525+0000 7fdf17400640 1 -- 172.21.15.39:0/4066210132 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdef0078bc0 msgr2=0x7fdef007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:55.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.525+0000 7fdf17400640 1 --2- 172.21.15.39:0/4066210132 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdef0078bc0 0x7fdef007b0d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdf04034000 tx=0x7fdf04004730 comp rx=0 tx=0).stop 2024-09-21T07:35:55.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.525+0000 7fdf17400640 1 -- 172.21.15.39:0/4066210132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf10070440 msgr2=0x7fdf101e50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:55.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.525+0000 7fdf17400640 1 --2- 172.21.15.39:0/4066210132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf10070440 0x7fdf101e50b0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fdefc00e790 tx=0x7fdefc00ec50 comp rx=0 tx=0).stop 2024-09-21T07:35:55.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.525+0000 7fdf17400640 1 -- 172.21.15.39:0/4066210132 shutdown_connections 2024-09-21T07:35:55.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.525+0000 7fdf17400640 1 --2- 172.21.15.39:0/4066210132 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdef0078bc0 0x7fdef007b0d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:55.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.525+0000 7fdf17400640 1 --2- 172.21.15.39:0/4066210132 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdf10073170 0x7fdf101e55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:55.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.525+0000 7fdf17400640 1 --2- 172.21.15.39:0/4066210132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf10070440 0x7fdf101e50b0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:55.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.525+0000 7fdf17400640 1 --2- 172.21.15.39:0/4066210132 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdf10070030 0x7fdf101f35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:55.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.525+0000 7fdf17400640 1 -- 172.21.15.39:0/4066210132 >> 172.21.15.39:0/4066210132 conn(0x7fdf100b1cb0 msgr2=0x7fdf100ad950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:55.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.525+0000 7fdf17400640 1 -- 172.21.15.39:0/4066210132 shutdown_connections 2024-09-21T07:35:55.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:55.525+0000 7fdf17400640 1 -- 172.21.15.39:0/4066210132 wait complete. 2024-09-21T07:35:55.954 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 44, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T07:35:48.965416+0000', 'modified': '2024-09-21T07:35:51.980637+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5842}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5842': {'gid': 5842, 'name': 'c', 'rank': 0, 'incarnation': 40, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.39:6834/1032917299', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 1032917299}, {'type': 'v1', 'addr': '172.21.15.39:6834', 'nonce': 1032917299}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5842, 'qdb_cluster': [5842]} 2024-09-21T07:35:55.954 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T07:35:55.954 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T07:35:56.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.180+0000 7f9970e00640 1 -- 172.21.15.39:0/1932102434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f996c0cc1e0 msgr2=0x7f996c0af3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:56.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.180+0000 7f9970e00640 1 --2- 172.21.15.39:0/1932102434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f996c0cc1e0 0x7f996c0af3c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f995c009830 tx=0x7f995c02a5c0 comp rx=0 tx=0).stop 2024-09-21T07:35:56.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.180+0000 7f9970e00640 1 -- 172.21.15.39:0/1932102434 shutdown_connections 2024-09-21T07:35:56.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.180+0000 7f9970e00640 1 --2- 172.21.15.39:0/1932102434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f996c0cc1e0 0x7f996c0af3c0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:56.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.180+0000 7f9970e00640 1 -- 172.21.15.39:0/1932102434 >> 172.21.15.39:0/1932102434 conn(0x7f996c06d7d0 msgr2=0x7f996c0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:56.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.180+0000 7f9970e00640 1 -- 172.21.15.39:0/1932102434 shutdown_connections 2024-09-21T07:35:56.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.180+0000 7f9970e00640 1 -- 172.21.15.39:0/1932102434 wait complete. 2024-09-21T07:35:56.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.183+0000 7f9970e00640 1 Processor -- start 2024-09-21T07:35:56.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.183+0000 7f9970e00640 1 -- start start 2024-09-21T07:35:56.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.184+0000 7f9970e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f996c0cc1e0 0x7f996c0cb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:56.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.184+0000 7f9970e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f996c0ccb50 0x7f996c152390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:56.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.184+0000 7f9970e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f996c15d8a0 0x7f996c0cfe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:56.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.184+0000 7f9970e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f996c153520 con 0x7f996c0ccb50 2024-09-21T07:35:56.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.185+0000 7f9970e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f996c1533a0 con 0x7f996c0cc1e0 2024-09-21T07:35:56.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.185+0000 7f9970e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f996c1536a0 con 0x7f996c15d8a0 2024-09-21T07:35:56.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.185+0000 7f9961400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f996c0ccb50 0x7f996c152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:56.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.185+0000 7f9961400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f996c0ccb50 0x7f996c152390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45586/0 (socket says 172.21.15.39:45586) 2024-09-21T07:35:56.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.185+0000 7f9961400640 1 -- 172.21.15.39:0/1910319757 learned_addr learned my addr 172.21.15.39:0/1910319757 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:56.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.185+0000 7f9969e00640 1 --2- 172.21.15.39:0/1910319757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f996c0cc1e0 0x7f996c0cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:56.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.185+0000 7f996a800640 1 --2- 172.21.15.39:0/1910319757 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f996c15d8a0 0x7f996c0cfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:56.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.185+0000 7f9961400640 1 -- 172.21.15.39:0/1910319757 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f996c15d8a0 msgr2=0x7f996c0cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:56.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.185+0000 7f9961400640 1 --2- 172.21.15.39:0/1910319757 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f996c15d8a0 0x7f996c0cfe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:56.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.185+0000 7f9961400640 1 -- 172.21.15.39:0/1910319757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f996c0cc1e0 msgr2=0x7f996c0cb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:56.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.185+0000 7f9961400640 1 --2- 172.21.15.39:0/1910319757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f996c0cc1e0 0x7f996c0cb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:56.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.185+0000 7f9961400640 1 -- 172.21.15.39:0/1910319757 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f996c1528d0 con 0x7f996c0ccb50 2024-09-21T07:35:56.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.185+0000 7f9969e00640 1 --2- 172.21.15.39:0/1910319757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f996c0cc1e0 0x7f996c0cb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:35:56.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.185+0000 7f996a800640 1 --2- 172.21.15.39:0/1910319757 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f996c15d8a0 0x7f996c0cfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:35:56.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.186+0000 7f9961400640 1 --2- 172.21.15.39:0/1910319757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f996c0ccb50 0x7f996c152390 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f995400c3a0 tx=0x7f995400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:56.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.186+0000 7f9962a00640 1 -- 172.21.15.39:0/1910319757 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9954017070 con 0x7f996c0ccb50 2024-09-21T07:35:56.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.186+0000 7f9962a00640 1 -- 172.21.15.39:0/1910319757 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f99540053c0 con 0x7f996c0ccb50 2024-09-21T07:35:56.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.186+0000 7f9970e00640 1 -- 172.21.15.39:0/1910319757 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f996c1f6050 con 0x7f996c0ccb50 2024-09-21T07:35:56.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.186+0000 7f9962a00640 1 -- 172.21.15.39:0/1910319757 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9954003850 con 0x7f996c0ccb50 2024-09-21T07:35:56.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.186+0000 7f9970e00640 1 -- 172.21.15.39:0/1910319757 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f996c1f6610 con 0x7f996c0ccb50 2024-09-21T07:35:56.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.188+0000 7f9962a00640 1 -- 172.21.15.39:0/1910319757 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f995401a070 con 0x7f996c0ccb50 2024-09-21T07:35:56.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.189+0000 7f9962a00640 1 --2- 172.21.15.39:0/1910319757 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9948078b90 0x7f994807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:56.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.189+0000 7f9962a00640 1 -- 172.21.15.39:0/1910319757 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6772+0+0 (secure 0 0 0) 0x7f995409a5e0 con 0x7f996c0ccb50 2024-09-21T07:35:56.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.190+0000 7f9969e00640 1 --2- 172.21.15.39:0/1910319757 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9948078b90 0x7f994807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:56.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.190+0000 7f9977fdf740 1 Processor -- start 2024-09-21T07:35:56.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.190+0000 7f9977fdf740 1 -- start start 2024-09-21T07:35:56.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.191+0000 7f9969e00640 1 --2- 172.21.15.39:0/1910319757 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9948078b90 0x7f994807b0a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f995c034000 tx=0x7f995c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:56.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.192+0000 7f9977fdf740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55838d658270 0x55838da28970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:56.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.192+0000 7f9977fdf740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55838d62f3b0 0x55838da28eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:56.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.192+0000 7f9977fdf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55838d637590 0x55838d9f8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:56.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.192+0000 7f9977fdf740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55838d982430 con 0x55838d658270 2024-09-21T07:35:56.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.192+0000 7f9977fdf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55838d9725d0 con 0x55838d637590 2024-09-21T07:35:56.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.192+0000 7f9977fdf740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55838d961a00 con 0x55838d62f3b0 2024-09-21T07:35:56.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.192+0000 7f9969e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55838d658270 0x55838da28970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:56.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.192+0000 7f9969e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55838d658270 0x55838da28970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45602/0 (socket says 172.21.15.39:45602) 2024-09-21T07:35:56.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.192+0000 7f9969e00640 1 -- 172.21.15.39:0/1371737137 learned_addr learned my addr 172.21.15.39:0/1371737137 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:56.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.192+0000 7f996a800640 1 --2- 172.21.15.39:0/1371737137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55838d637590 0x55838d9f8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:56.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.192+0000 7f9961400640 1 --2- 172.21.15.39:0/1371737137 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55838d62f3b0 0x55838da28eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:56.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.192+0000 7f9969e00640 1 -- 172.21.15.39:0/1371737137 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55838d62f3b0 msgr2=0x55838da28eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:35:56.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.192+0000 7f9969e00640 1 --2- 172.21.15.39:0/1371737137 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55838d62f3b0 0x55838da28eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:56.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.192+0000 7f9969e00640 1 -- 172.21.15.39:0/1371737137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55838d637590 msgr2=0x55838d9f8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:35:56.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.192+0000 7f9969e00640 1 --2- 172.21.15.39:0/1371737137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55838d637590 0x55838d9f8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:56.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.192+0000 7f9969e00640 1 -- 172.21.15.39:0/1371737137 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55838d92a520 con 0x55838d658270 2024-09-21T07:35:56.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.192+0000 7f9961400640 1 --2- 172.21.15.39:0/1371737137 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55838d62f3b0 0x55838da28eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:35:56.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.192+0000 7f996a800640 1 --2- 172.21.15.39:0/1371737137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55838d637590 0x55838d9f8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:35:56.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.193+0000 7f9969e00640 1 --2- 172.21.15.39:0/1371737137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55838d658270 0x55838da28970 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f995c009920 tx=0x7f995c03e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22623722b29b3e0a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:56.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.193+0000 7f995aa00640 1 -- 172.21.15.39:0/1371737137 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c049420 con 0x55838d658270 2024-09-21T07:35:56.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.193+0000 7f995aa00640 1 -- 172.21.15.39:0/1371737137 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f995c0496b0 con 0x55838d658270 2024-09-21T07:35:56.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.193+0000 7f995aa00640 1 -- 172.21.15.39:0/1371737137 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c049a60 con 0x55838d658270 2024-09-21T07:35:56.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.194+0000 7f9977fdf740 1 -- 172.21.15.39:0/1371737137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55838d658270 msgr2=0x55838da28970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:56.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.194+0000 7f9977fdf740 1 --2- 172.21.15.39:0/1371737137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55838d658270 0x55838da28970 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f995c009920 tx=0x7f995c03e070 comp rx=0 tx=0).stop 2024-09-21T07:35:56.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.194+0000 7f9977fdf740 1 -- 172.21.15.39:0/1371737137 shutdown_connections 2024-09-21T07:35:56.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.194+0000 7f9977fdf740 1 --2- 172.21.15.39:0/1371737137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55838d637590 0x55838d9f8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:56.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.194+0000 7f9977fdf740 1 --2- 172.21.15.39:0/1371737137 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55838d62f3b0 0x55838da28eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:56.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.194+0000 7f9977fdf740 1 --2- 172.21.15.39:0/1371737137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55838d658270 0x55838da28970 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:56.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.194+0000 7f9977fdf740 1 -- 172.21.15.39:0/1371737137 >> 172.21.15.39:0/1371737137 conn(0x55838d64c3c0 msgr2=0x55838d9fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:56.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.194+0000 7f9977fdf740 1 -- 172.21.15.39:0/1371737137 shutdown_connections 2024-09-21T07:35:56.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.194+0000 7f9977fdf740 1 -- 172.21.15.39:0/1371737137 wait complete. 2024-09-21T07:35:56.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.195+0000 7f9977fdf740 1 -- start start 2024-09-21T07:35:56.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.195+0000 7f9977fdf740 1 Processor -- start 2024-09-21T07:35:56.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.197+0000 7f9977fdf740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55838d62f3b0 0x55838da3d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:56.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.197+0000 7f9977fdf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55838d637590 0x55838da170e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:56.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.197+0000 7f9977fdf740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55838d658270 0x55838d9f9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:56.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.197+0000 7f9977fdf740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55838d908330 con 0x55838d658270 2024-09-21T07:35:56.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.197+0000 7f9977fdf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55838d965810 con 0x55838d637590 2024-09-21T07:35:56.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.197+0000 7f9977fdf740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55838d6d2620 con 0x55838d62f3b0 2024-09-21T07:35:56.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.197+0000 7f9977fdf740 10 client.0 _ll_register_callbacks cb 0x55838d90d420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:35:56.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.197+0000 7f996a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55838d658270 0x55838d9f9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:56.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.197+0000 7f996a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55838d658270 0x55838d9f9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45614/0 (socket says 172.21.15.39:45614) 2024-09-21T07:35:56.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.197+0000 7f996a800640 1 -- 172.21.15.39:0/2320442367 learned_addr learned my addr 172.21.15.39:0/2320442367 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:56.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.197+0000 7f9961400640 1 --2- 172.21.15.39:0/2320442367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55838d637590 0x55838da170e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:56.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.197+0000 7f9969e00640 1 --2- 172.21.15.39:0/2320442367 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55838d62f3b0 0x55838da3d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:56.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.198+0000 7f996a800640 1 -- 172.21.15.39:0/2320442367 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55838d62f3b0 msgr2=0x55838da3d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:35:56.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.198+0000 7f996a800640 1 --2- 172.21.15.39:0/2320442367 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55838d62f3b0 0x55838da3d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:56.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.198+0000 7f996a800640 1 -- 172.21.15.39:0/2320442367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55838d637590 msgr2=0x55838da170e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:35:56.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.198+0000 7f996a800640 1 --2- 172.21.15.39:0/2320442367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55838d637590 0x55838da170e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:56.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.198+0000 7f996a800640 1 -- 172.21.15.39:0/2320442367 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55838d6c3850 con 0x55838d658270 2024-09-21T07:35:56.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.198+0000 7f996a800640 1 --2- 172.21.15.39:0/2320442367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55838d658270 0x55838d9f9c70 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f996400e840 tx=0x7f996400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ac919c0c650a8ec server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:56.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.198+0000 7f994f400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:35:56.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.198+0000 7f994f400640 1 -- 172.21.15.39:0/2320442367 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9964017070 con 0x55838d658270 2024-09-21T07:35:56.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.198+0000 7f994f400640 1 -- 172.21.15.39:0/2320442367 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 6772+0+0 (secure 0 0 0) 0x7f996400cb30 con 0x55838d658270 2024-09-21T07:35:56.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.198+0000 7f9977fdf740 1 -- 172.21.15.39:0/2320442367 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f995c0496b0 con 0x55838d658270 2024-09-21T07:35:56.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.198+0000 7f994f400640 1 -- 172.21.15.39:0/2320442367 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f9964022050 con 0x55838d658270 2024-09-21T07:35:56.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.198+0000 7f9977fdf740 10 client.6120 fetch_fsmap learned FSMap version 44 2024-09-21T07:35:56.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.198+0000 7f9977fdf740 1 -- 172.21.15.39:0/2320442367 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) -- 0x55838d73a6b0 con 0x55838d658270 2024-09-21T07:35:56.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.199+0000 7f994f400640 1 -- 172.21.15.39:0/2320442367 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f995c0496b0 con 0x55838d658270 2024-09-21T07:35:56.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.199+0000 7f994f400640 1 -- 172.21.15.39:0/2320442367 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 44) ==== 3502+0+0 (secure 0 0 0) 0x7f9964025020 con 0x55838d658270 2024-09-21T07:35:56.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.199+0000 7f9977fdf740 10 client.6120 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T07:35:56.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.199+0000 7f9977fdf740 10 client.6120 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:35:56.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.199+0000 7f9977fdf740 20 client.6120 populate_metadata read hostname 'smithi039' 2024-09-21T07:35:56.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.199+0000 7f9977fdf740 1 --2- 172.21.15.39:0/2320442367 >> [v2:172.21.15.39:6833/1032917299,v1:172.21.15.39:6834/1032917299] conn(0x55838d616e40 0x55838da3b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:56.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.199+0000 7f9977fdf740 4 client.6120 mds_command: new command op to 5842 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:35:56.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.199+0000 7f9977fdf740 1 -- 172.21.15.39:0/2320442367 --> [v2:172.21.15.39:6833/1032917299,v1:172.21.15.39:6834/1032917299] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55838d974520 con 0x55838d616e40 2024-09-21T07:35:56.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.199+0000 7f9969e00640 1 --2- 172.21.15.39:0/2320442367 >> [v2:172.21.15.39:6833/1032917299,v1:172.21.15.39:6834/1032917299] conn(0x55838d616e40 0x55838da3b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:56.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.200+0000 7f994f400640 10 client.6120 ms_handle_connect on v2:172.21.15.39:6833/1032917299 2024-09-21T07:35:56.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.200+0000 7f9969e00640 1 --2- 172.21.15.39:0/2320442367 >> [v2:172.21.15.39:6833/1032917299,v1:172.21.15.39:6834/1032917299] conn(0x55838d616e40 0x55838da3b9c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e87bf823a3d3a55e server_cookie=1f7916ec0fbd91c in_seq=0 out_seq=0 2024-09-21T07:35:56.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.201+0000 7f994f400640 1 -- 172.21.15.39:0/2320442367 <== mds.0 v2:172.21.15.39:6833/1032917299 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55838d974520 con 0x55838d616e40 2024-09-21T07:35:56.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.201+0000 7f994f400640 10 client.6120 handle_command_reply: tid=0 2024-09-21T07:35:56.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.201+0000 7f9977fdf740 1 client.6120 shutdown 2024-09-21T07:35:56.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.201+0000 7f9977fdf740 1 -- 172.21.15.39:0/2320442367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55838d658270 msgr2=0x55838d9f9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:56.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.201+0000 7f9977fdf740 1 --2- 172.21.15.39:0/2320442367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55838d658270 0x55838d9f9c70 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f996400e840 tx=0x7f996400ed00 comp rx=0 tx=0).stop 2024-09-21T07:35:56.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.201+0000 7f9977fdf740 1 -- 172.21.15.39:0/2320442367 shutdown_connections 2024-09-21T07:35:56.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.201+0000 7f9977fdf740 1 --2- 172.21.15.39:0/2320442367 >> [v2:172.21.15.39:6833/1032917299,v1:172.21.15.39:6834/1032917299] conn(0x55838d616e40 0x55838da3b9c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:56.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.201+0000 7f9977fdf740 1 --2- 172.21.15.39:0/2320442367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55838d658270 0x55838d9f9c70 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:56.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.201+0000 7f9977fdf740 1 --2- 172.21.15.39:0/2320442367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55838d637590 0x55838da170e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:56.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.201+0000 7f994f400640 0 client.6120 ms_handle_reset on v2:172.21.15.39:6833/1032917299 2024-09-21T07:35:56.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.201+0000 7f9977fdf740 1 --2- 172.21.15.39:0/2320442367 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55838d62f3b0 0x55838da3d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:56.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.201+0000 7f9977fdf740 1 -- 172.21.15.39:0/2320442367 >> 172.21.15.39:0/2320442367 conn(0x55838d64c3c0 msgr2=0x55838da458c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:56.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.201+0000 7f9977fdf740 1 -- 172.21.15.39:0/2320442367 shutdown_connections 2024-09-21T07:35:56.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.201+0000 7f9977fdf740 1 -- 172.21.15.39:0/2320442367 wait complete. 2024-09-21T07:35:56.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.201+0000 7f9977fdf740 20 client.6120 trim_cache size 0 max 16384 2024-09-21T07:35:56.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.220+0000 7f9977fdf740 1 Processor -- start 2024-09-21T07:35:56.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.221+0000 7f9977fdf740 1 -- start start 2024-09-21T07:35:56.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.222+0000 7f9977fdf740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55838d62f3b0 0x55838da67070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:56.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.222+0000 7f9977fdf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55838d637590 0x55838da16eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:56.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.222+0000 7f9977fdf740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55838d658270 0x55838da21a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:56.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.222+0000 7f9977fdf740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f996c203770 con 0x55838d658270 2024-09-21T07:35:56.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.222+0000 7f9977fdf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f996c1ebf20 con 0x55838d637590 2024-09-21T07:35:56.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.222+0000 7f9977fdf740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f996c2032c0 con 0x55838d62f3b0 2024-09-21T07:35:56.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.222+0000 7f996a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55838d658270 0x55838da21a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:56.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.222+0000 7f996a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55838d658270 0x55838da21a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45626/0 (socket says 172.21.15.39:45626) 2024-09-21T07:35:56.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.222+0000 7f996a800640 1 -- 172.21.15.39:0/3953152095 learned_addr learned my addr 172.21.15.39:0/3953152095 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:56.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.222+0000 7f996a800640 1 -- 172.21.15.39:0/3953152095 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55838d62f3b0 msgr2=0x55838da67070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:35:56.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.222+0000 7f996a800640 1 --2- 172.21.15.39:0/3953152095 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55838d62f3b0 0x55838da67070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:56.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.223+0000 7f996a800640 1 -- 172.21.15.39:0/3953152095 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55838d637590 msgr2=0x55838da16eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:35:56.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.223+0000 7f996a800640 1 --2- 172.21.15.39:0/3953152095 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55838d637590 0x55838da16eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:56.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.223+0000 7f996a800640 1 -- 172.21.15.39:0/3953152095 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55838d9f9fe0 con 0x55838d658270 2024-09-21T07:35:56.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.223+0000 7f996a800640 1 --2- 172.21.15.39:0/3953152095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55838d658270 0x55838da21a10 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f996400d000 tx=0x7f9964001b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2ffe3e62f9a5de0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:56.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.223+0000 7f995a000640 1 -- 172.21.15.39:0/3953152095 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996402a020 con 0x55838d658270 2024-09-21T07:35:56.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.223+0000 7f995a000640 1 -- 172.21.15.39:0/3953152095 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f996400cc20 con 0x55838d658270 2024-09-21T07:35:56.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.223+0000 7f995a000640 1 -- 172.21.15.39:0/3953152095 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9964017070 con 0x55838d658270 2024-09-21T07:35:56.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.224+0000 7f9977fdf740 1 -- 172.21.15.39:0/3953152095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55838d658270 msgr2=0x55838da21a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:56.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.224+0000 7f9977fdf740 1 --2- 172.21.15.39:0/3953152095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55838d658270 0x55838da21a10 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f996400d000 tx=0x7f9964001b80 comp rx=0 tx=0).stop 2024-09-21T07:35:56.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.224+0000 7f9977fdf740 1 -- 172.21.15.39:0/3953152095 shutdown_connections 2024-09-21T07:35:56.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.224+0000 7f9977fdf740 1 --2- 172.21.15.39:0/3953152095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55838d658270 0x55838da21a10 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:56.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.224+0000 7f9977fdf740 1 --2- 172.21.15.39:0/3953152095 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55838d637590 0x55838da16eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:56.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.224+0000 7f9977fdf740 1 --2- 172.21.15.39:0/3953152095 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55838d62f3b0 0x55838da67070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:56.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.224+0000 7f9977fdf740 1 -- 172.21.15.39:0/3953152095 >> 172.21.15.39:0/3953152095 conn(0x55838d64c3c0 msgr2=0x55838da28c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:56.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.224+0000 7f9977fdf740 1 -- 172.21.15.39:0/3953152095 shutdown_connections 2024-09-21T07:35:56.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.224+0000 7f9977fdf740 1 -- 172.21.15.39:0/3953152095 wait complete. 2024-09-21T07:35:56.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.225+0000 7f9977fdf740 1 -- start start 2024-09-21T07:35:56.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.225+0000 7f9977fdf740 1 Processor -- start 2024-09-21T07:35:56.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.227+0000 7f9977fdf740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55838d62f3b0 0x55838daa8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:56.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.227+0000 7f9977fdf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55838d637590 0x55838daa93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:56.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.227+0000 7f9977fdf740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55838d658270 0x55838da19a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:56.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.227+0000 7f9977fdf740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55838da1e3a0 con 0x55838d62f3b0 2024-09-21T07:35:56.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.227+0000 7f9977fdf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55838da56d30 con 0x55838d637590 2024-09-21T07:35:56.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.227+0000 7f9977fdf740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55838da20fe0 con 0x55838d658270 2024-09-21T07:35:56.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.227+0000 7f9977fdf740 10 client.0 _ll_register_callbacks cb 0x7f996400e3f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:35:56.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.227+0000 7f9969e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55838d62f3b0 0x55838daa8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:56.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.227+0000 7f9969e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55838d62f3b0 0x55838daa8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45638/0 (socket says 172.21.15.39:45638) 2024-09-21T07:35:56.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.227+0000 7f9969e00640 1 -- 172.21.15.39:0/347097002 learned_addr learned my addr 172.21.15.39:0/347097002 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:56.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.227+0000 7f996a800640 1 --2- 172.21.15.39:0/347097002 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55838d658270 0x55838da19a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:56.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.227+0000 7f9969e00640 1 -- 172.21.15.39:0/347097002 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55838d658270 msgr2=0x55838da19a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:35:56.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.227+0000 7f9969e00640 1 --2- 172.21.15.39:0/347097002 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55838d658270 0x55838da19a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:56.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.227+0000 7f9969e00640 1 -- 172.21.15.39:0/347097002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55838d637590 msgr2=0x55838daa93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:35:56.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.227+0000 7f9969e00640 1 --2- 172.21.15.39:0/347097002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55838d637590 0x55838daa93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:56.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.227+0000 7f9969e00640 1 -- 172.21.15.39:0/347097002 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55838da670f0 con 0x55838d62f3b0 2024-09-21T07:35:56.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.227+0000 7f996a800640 1 --2- 172.21.15.39:0/347097002 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55838d658270 0x55838da19a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:35:56.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.228+0000 7f9969e00640 1 --2- 172.21.15.39:0/347097002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55838d62f3b0 0x55838daa8ea0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f995c009830 tx=0x7f995c0097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=720a5b7bc3d0a311 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:56.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.228+0000 7f994f400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:35:56.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.228+0000 7f994f400640 1 -- 172.21.15.39:0/347097002 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c0498b0 con 0x55838d62f3b0 2024-09-21T07:35:56.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.228+0000 7f994f400640 1 -- 172.21.15.39:0/347097002 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 6772+0+0 (secure 0 0 0) 0x7f995c049a50 con 0x55838d62f3b0 2024-09-21T07:35:56.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.228+0000 7f9977fdf740 1 -- 172.21.15.39:0/347097002 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f996400cc20 con 0x55838d62f3b0 2024-09-21T07:35:56.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.228+0000 7f994f400640 1 -- 172.21.15.39:0/347097002 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f995c06b050 con 0x55838d62f3b0 2024-09-21T07:35:56.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.228+0000 7f9977fdf740 10 client.6126 fetch_fsmap learned FSMap version 44 2024-09-21T07:35:56.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.228+0000 7f9977fdf740 1 -- 172.21.15.39:0/347097002 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) -- 0x55838da45de0 con 0x55838d62f3b0 2024-09-21T07:35:56.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.229+0000 7f994f400640 1 -- 172.21.15.39:0/347097002 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f996400cc20 con 0x55838d62f3b0 2024-09-21T07:35:56.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.229+0000 7f994f400640 1 -- 172.21.15.39:0/347097002 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 44) ==== 3502+0+0 (secure 0 0 0) 0x7f995c06e020 con 0x55838d62f3b0 2024-09-21T07:35:56.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.229+0000 7f9977fdf740 10 client.6126 fetch_fsmap finished waiting for FSMap version 44 2024-09-21T07:35:56.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.229+0000 7f9977fdf740 10 client.6126 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:35:56.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.229+0000 7f9977fdf740 20 client.6126 populate_metadata read hostname 'smithi039' 2024-09-21T07:35:56.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.229+0000 7f9977fdf740 1 --2- 172.21.15.39:0/347097002 >> [v2:172.21.15.39:6833/1032917299,v1:172.21.15.39:6834/1032917299] conn(0x55838dac4280 0x55838da54d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:56.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.229+0000 7f9977fdf740 4 client.6126 mds_command: new command op to 5842 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:35:56.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.229+0000 7f9977fdf740 1 -- 172.21.15.39:0/347097002 --> [v2:172.21.15.39:6833/1032917299,v1:172.21.15.39:6834/1032917299] -- command(tid 0: {"prefix": "status"}) -- 0x55838d96ffd0 con 0x55838dac4280 2024-09-21T07:35:56.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.229+0000 7f9961400640 1 --2- 172.21.15.39:0/347097002 >> [v2:172.21.15.39:6833/1032917299,v1:172.21.15.39:6834/1032917299] conn(0x55838dac4280 0x55838da54d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:56.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.230+0000 7f9961400640 1 --2- 172.21.15.39:0/347097002 >> [v2:172.21.15.39:6833/1032917299,v1:172.21.15.39:6834/1032917299] conn(0x55838dac4280 0x55838da54d80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52c6ee57b97948f2 server_cookie=cbf97c3220aede2b in_seq=0 out_seq=0 2024-09-21T07:35:56.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.230+0000 7f994f400640 10 client.6126 ms_handle_connect on v2:172.21.15.39:6833/1032917299 2024-09-21T07:35:56.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.230+0000 7f994f400640 1 -- 172.21.15.39:0/347097002 <== mds.0 v2:172.21.15.39:6833/1032917299 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55838d96ffd0 con 0x55838dac4280 2024-09-21T07:35:56.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.230+0000 7f994f400640 10 client.6126 handle_command_reply: tid=0 2024-09-21T07:35:56.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.230+0000 7f9977fdf740 1 client.6126 shutdown 2024-09-21T07:35:56.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.230+0000 7f9977fdf740 1 -- 172.21.15.39:0/347097002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55838d62f3b0 msgr2=0x55838daa8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:56.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.230+0000 7f9977fdf740 1 --2- 172.21.15.39:0/347097002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55838d62f3b0 0x55838daa8ea0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f995c009830 tx=0x7f995c0097b0 comp rx=0 tx=0).stop 2024-09-21T07:35:56.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.230+0000 7f9977fdf740 1 -- 172.21.15.39:0/347097002 shutdown_connections 2024-09-21T07:35:56.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.230+0000 7f9977fdf740 1 --2- 172.21.15.39:0/347097002 >> [v2:172.21.15.39:6833/1032917299,v1:172.21.15.39:6834/1032917299] conn(0x55838dac4280 0x55838da54d80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:56.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.230+0000 7f9977fdf740 1 --2- 172.21.15.39:0/347097002 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55838d658270 0x55838da19a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:56.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.230+0000 7f9977fdf740 1 --2- 172.21.15.39:0/347097002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55838d637590 0x55838daa93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:56.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.230+0000 7f9977fdf740 1 --2- 172.21.15.39:0/347097002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55838d62f3b0 0x55838daa8ea0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:56.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.230+0000 7f994f400640 0 client.6126 ms_handle_reset on v2:172.21.15.39:6833/1032917299 2024-09-21T07:35:56.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.230+0000 7f9977fdf740 1 -- 172.21.15.39:0/347097002 >> 172.21.15.39:0/347097002 conn(0x55838d64c3c0 msgr2=0x55838da23460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:56.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.230+0000 7f9977fdf740 1 -- 172.21.15.39:0/347097002 shutdown_connections 2024-09-21T07:35:56.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.230+0000 7f9977fdf740 1 -- 172.21.15.39:0/347097002 wait complete. 2024-09-21T07:35:56.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.231+0000 7f9977fdf740 20 client.6126 trim_cache size 0 max 16384 2024-09-21T07:35:56.236 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:35:56.236 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:35:56.236 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-09-21T07:35:56.236 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 5842, 2024-09-21T07:35:56.236 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:35:56.236 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:35:56.236 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:35:56.236 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 6.7887784419999999, 2024-09-21T07:35:56.236 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 44, 2024-09-21T07:35:56.237 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 78, 2024-09-21T07:35:56.237 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 78, 2024-09-21T07:35:56.237 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 11.759380821000001 2024-09-21T07:35:56.237 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:35:56.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.231+0000 7f9970e00640 1 -- 172.21.15.39:0/1910319757 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9948078b90 msgr2=0x7f994807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:56.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.231+0000 7f9970e00640 1 --2- 172.21.15.39:0/1910319757 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9948078b90 0x7f994807b0a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f995c034000 tx=0x7f995c004730 comp rx=0 tx=0).stop 2024-09-21T07:35:56.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.231+0000 7f9970e00640 1 -- 172.21.15.39:0/1910319757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f996c0ccb50 msgr2=0x7f996c152390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:56.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.231+0000 7f9970e00640 1 --2- 172.21.15.39:0/1910319757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f996c0ccb50 0x7f996c152390 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f995400c3a0 tx=0x7f995400c860 comp rx=0 tx=0).stop 2024-09-21T07:35:56.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.231+0000 7f9970e00640 1 -- 172.21.15.39:0/1910319757 shutdown_connections 2024-09-21T07:35:56.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.231+0000 7f9970e00640 1 --2- 172.21.15.39:0/1910319757 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9948078b90 0x7f994807b0a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:56.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.231+0000 7f9970e00640 1 --2- 172.21.15.39:0/1910319757 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f996c15d8a0 0x7f996c0cfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:56.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.231+0000 7f9970e00640 1 --2- 172.21.15.39:0/1910319757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f996c0ccb50 0x7f996c152390 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:56.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.231+0000 7f9970e00640 1 --2- 172.21.15.39:0/1910319757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f996c0cc1e0 0x7f996c0cb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:56.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.231+0000 7f9970e00640 1 -- 172.21.15.39:0/1910319757 >> 172.21.15.39:0/1910319757 conn(0x7f996c06d7d0 msgr2=0x7f996c0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:56.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.231+0000 7f9970e00640 1 -- 172.21.15.39:0/1910319757 shutdown_connections 2024-09-21T07:35:56.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.231+0000 7f9970e00640 1 -- 172.21.15.39:0/1910319757 wait complete. 2024-09-21T07:35:56.254 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:35:56.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.478+0000 7faae1c00640 1 -- 172.21.15.39:0/1954859588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faadc15e950 msgr2=0x7faadc0d0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:56.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.478+0000 7faae1c00640 1 --2- 172.21.15.39:0/1954859588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faadc15e950 0x7faadc0d0980 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7faacc009830 tx=0x7faacc02a520 comp rx=0 tx=0).stop 2024-09-21T07:35:56.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.478+0000 7faae1c00640 1 -- 172.21.15.39:0/1954859588 shutdown_connections 2024-09-21T07:35:56.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.478+0000 7faae1c00640 1 --2- 172.21.15.39:0/1954859588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faadc15e950 0x7faadc0d0980 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:56.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.478+0000 7faae1c00640 1 -- 172.21.15.39:0/1954859588 >> 172.21.15.39:0/1954859588 conn(0x7faadc0cee20 msgr2=0x7faadc0cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:56.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.478+0000 7faae1c00640 1 -- 172.21.15.39:0/1954859588 shutdown_connections 2024-09-21T07:35:56.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.478+0000 7faae1c00640 1 -- 172.21.15.39:0/1954859588 wait complete. 2024-09-21T07:35:56.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.481+0000 7faae1c00640 1 Processor -- start 2024-09-21T07:35:56.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.481+0000 7faae1c00640 1 -- start start 2024-09-21T07:35:56.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.482+0000 7faae1c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faadc0b3e20 0x7faadc1f6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:56.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.482+0000 7faae1c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faadc0cd1f0 0x7faadc15c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:56.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.482+0000 7faae1c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faadc15e950 0x7faadc15ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:56.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.482+0000 7faae1c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7faadc0cde90 con 0x7faadc0b3e20 2024-09-21T07:35:56.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.482+0000 7faae1c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7faadc0cdd10 con 0x7faadc15e950 2024-09-21T07:35:56.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.482+0000 7faae1c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7faadc0ce010 con 0x7faadc0cd1f0 2024-09-21T07:35:56.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.482+0000 7faadaa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faadc0b3e20 0x7faadc1f6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:56.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.482+0000 7faadaa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faadc0b3e20 0x7faadc1f6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45652/0 (socket says 172.21.15.39:45652) 2024-09-21T07:35:56.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.482+0000 7faadaa00640 1 -- 172.21.15.39:0/1069620045 learned_addr learned my addr 172.21.15.39:0/1069620045 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:56.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.483+0000 7faadb400640 1 --2- 172.21.15.39:0/1069620045 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faadc15e950 0x7faadc15ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:56.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.483+0000 7faada000640 1 --2- 172.21.15.39:0/1069620045 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faadc0cd1f0 0x7faadc15c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:56.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.483+0000 7faadaa00640 1 -- 172.21.15.39:0/1069620045 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faadc0cd1f0 msgr2=0x7faadc15c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:56.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.483+0000 7faadaa00640 1 --2- 172.21.15.39:0/1069620045 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faadc0cd1f0 0x7faadc15c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:56.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.483+0000 7faadaa00640 1 -- 172.21.15.39:0/1069620045 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faadc15e950 msgr2=0x7faadc15ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:56.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.483+0000 7faadaa00640 1 --2- 172.21.15.39:0/1069620045 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faadc15e950 0x7faadc15ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:56.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.483+0000 7faadaa00640 1 -- 172.21.15.39:0/1069620045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faadc0c6420 con 0x7faadc0b3e20 2024-09-21T07:35:56.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.483+0000 7faadaa00640 1 --2- 172.21.15.39:0/1069620045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faadc0b3e20 0x7faadc1f6950 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7faacc009950 tx=0x7faacc033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:56.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.484+0000 7faad2a00640 1 -- 172.21.15.39:0/1069620045 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faacc031b50 con 0x7faadc0b3e20 2024-09-21T07:35:56.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.484+0000 7faad2a00640 1 -- 172.21.15.39:0/1069620045 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faacc031de0 con 0x7faadc0b3e20 2024-09-21T07:35:56.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.484+0000 7faae1c00640 1 -- 172.21.15.39:0/1069620045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faadc0c6700 con 0x7faadc0b3e20 2024-09-21T07:35:56.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.484+0000 7faad2a00640 1 -- 172.21.15.39:0/1069620045 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faacc005280 con 0x7faadc0b3e20 2024-09-21T07:35:56.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.484+0000 7faae1c00640 1 -- 172.21.15.39:0/1069620045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faadc0c6cc0 con 0x7faadc0b3e20 2024-09-21T07:35:56.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.485+0000 7faae1c00640 1 -- 172.21.15.39:0/1069620045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faadc0d0980 con 0x7faadc0b3e20 2024-09-21T07:35:56.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.486+0000 7faad2a00640 1 -- 172.21.15.39:0/1069620045 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7faacc005420 con 0x7faadc0b3e20 2024-09-21T07:35:56.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.487+0000 7faad2a00640 1 --2- 172.21.15.39:0/1069620045 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faabc078bc0 0x7faabc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:56.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.487+0000 7faad2a00640 1 -- 172.21.15.39:0/1069620045 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6772+0+0 (secure 0 0 0) 0x7faacc0bc300 con 0x7faadc0b3e20 2024-09-21T07:35:56.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.487+0000 7faada000640 1 --2- 172.21.15.39:0/1069620045 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faabc078bc0 0x7faabc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:56.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.488+0000 7faada000640 1 --2- 172.21.15.39:0/1069620045 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faabc078bc0 0x7faabc07b0d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7faadc0c7900 tx=0x7faac8009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:56.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.492+0000 7faad2a00640 1 -- 172.21.15.39:0/1069620045 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7faacc03f030 con 0x7faadc0b3e20 2024-09-21T07:35:56.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.727+0000 7faae1c00640 1 -- 172.21.15.39:0/1069620045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7faadc0b8320 con 0x7faadc0b3e20 2024-09-21T07:35:56.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.728+0000 7faad2a00640 1 -- 172.21.15.39:0/1069620045 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v44) ==== 53+0+73 (secure 0 0 0) 0x7faacc031770 con 0x7faadc0b3e20 2024-09-21T07:35:56.730 INFO:teuthology.orchestra.run.smithi039.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:35:56.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.730+0000 7faae1c00640 1 -- 172.21.15.39:0/1069620045 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faabc078bc0 msgr2=0x7faabc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:56.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.730+0000 7faae1c00640 1 --2- 172.21.15.39:0/1069620045 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faabc078bc0 0x7faabc07b0d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7faadc0c7900 tx=0x7faac8009040 comp rx=0 tx=0).stop 2024-09-21T07:35:56.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.730+0000 7faae1c00640 1 -- 172.21.15.39:0/1069620045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faadc0b3e20 msgr2=0x7faadc1f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:56.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.730+0000 7faae1c00640 1 --2- 172.21.15.39:0/1069620045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faadc0b3e20 0x7faadc1f6950 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7faacc009950 tx=0x7faacc033ba0 comp rx=0 tx=0).stop 2024-09-21T07:35:56.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.730+0000 7faae1c00640 1 -- 172.21.15.39:0/1069620045 shutdown_connections 2024-09-21T07:35:56.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.730+0000 7faae1c00640 1 --2- 172.21.15.39:0/1069620045 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faabc078bc0 0x7faabc07b0d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:56.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.730+0000 7faae1c00640 1 --2- 172.21.15.39:0/1069620045 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faadc15e950 0x7faadc15ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:56.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.730+0000 7faae1c00640 1 --2- 172.21.15.39:0/1069620045 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faadc0cd1f0 0x7faadc15c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:56.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.730+0000 7faae1c00640 1 --2- 172.21.15.39:0/1069620045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faadc0b3e20 0x7faadc1f6950 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:56.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.730+0000 7faae1c00640 1 -- 172.21.15.39:0/1069620045 >> 172.21.15.39:0/1069620045 conn(0x7faadc0cee20 msgr2=0x7faadc162410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:56.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.730+0000 7faae1c00640 1 -- 172.21.15.39:0/1069620045 shutdown_connections 2024-09-21T07:35:56.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:56.730+0000 7faae1c00640 1 -- 172.21.15.39:0/1069620045 wait complete. 2024-09-21T07:35:56.754 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:35:56.754 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:35:56.754 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi039.front.sepia.ceph.com 2024-09-21T07:35:56.755 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T07:35:56.755 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:35:56.755 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:35:56.755 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:35:56.755 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:35:56.755 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:35:56.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:35:56.755 DEBUG:teuthology.orchestra.run.smithi039:> ip addr 2024-09-21T07:35:56.791 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:35:56.791 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:35:56.791 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:35:56.791 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:35:56.791 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2024-09-21T07:35:56.791 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:35:56.791 INFO:teuthology.orchestra.run.smithi039.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:35:56.791 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:35:56.792 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f0 2024-09-21T07:35:56.792 INFO:teuthology.orchestra.run.smithi039.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:35:56.792 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c7 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:35:56.792 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f1 2024-09-21T07:35:56.792 INFO:teuthology.orchestra.run.smithi039.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:35:56.792 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:70 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:35:56.792 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f0 2024-09-21T07:35:56.792 INFO:teuthology.orchestra.run.smithi039.stdout: inet 172.21.15.39/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T07:35:56.792 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft 42562sec preferred_lft 42562sec 2024-09-21T07:35:56.792 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec4:7aff:febd:1370/64 scope link 2024-09-21T07:35:56.792 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:35:56.792 INFO:teuthology.orchestra.run.smithi039.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:35:56.792 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:71 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:35:56.792 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f1 2024-09-21T07:35:56.793 INFO:teuthology.orchestra.run.smithi039.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:35:56.793 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff 2024-09-21T07:35:56.793 INFO:teuthology.orchestra.run.smithi039.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:35:56.793 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:35:56.793 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::b079:ddff:fe0e:6333/64 scope link 2024-09-21T07:35:56.793 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:35:56.793 INFO:teuthology.orchestra.run.smithi039.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:35:56.793 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:35:56.793 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::2ca7:98ff:fe8a:d0ed/64 scope link 2024-09-21T07:35:56.793 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:35:56.793 INFO:teuthology.orchestra.run.smithi039.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:35:56.793 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether ee:10:53:01:0c:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:35:56.793 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec10:53ff:fe01:c44/64 scope link 2024-09-21T07:35:56.793 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:35:56.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:35:56.794 DEBUG:teuthology.orchestra.run.smithi039:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:35:56.885 INFO:teuthology.orchestra.run.smithi039.stdout:1 2024-09-21T07:35:56.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:35:56.889 DEBUG:teuthology.orchestra.run.smithi039:> route 2024-09-21T07:35:56.924 INFO:teuthology.orchestra.run.smithi039.stdout:Kernel IP routing table 2024-09-21T07:35:56.924 INFO:teuthology.orchestra.run.smithi039.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:35:56.924 INFO:teuthology.orchestra.run.smithi039.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T07:35:56.924 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T07:35:56.925 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:35:56.925 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:35:56.925 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-21T07:35:56.925 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T07:35:56.925 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T07:35:56.925 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T07:35:56.925 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-21T07:35:57.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:35:57.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:35:57.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:35:57.123 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-09-21T07:35:57.158 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:35:57.158 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T07:35:57.158 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:35:57.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:35:57.158 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-09-21T07:35:57.253 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:35:57.253 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T07:35:57.288 INFO:teuthology.orchestra.run.smithi039.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:35:57.288 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:35:57.288 DEBUG:teuthology.orchestra.run.smithi039:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:35:57.363 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-21T07:35:57.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:35:57.363 DEBUG:teuthology.orchestra.run.smithi039:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:35:57.562 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:35:57.562 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:35:57.596 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:35:57.597 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:35:57.597 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: resolved to: "172.21.15.39:3300,172.21.15.148:3300,172.21.15.148:3301" 2024-09-21T07:35:57.597 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: trying mount with new device syntax: 0@7ab9b04a-f88d-49aa-a493-e5c076637ffb.cephfs=/ 2024-09-21T07:35:57.597 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.39:3300/172.21.15.148:3300/172.21.15.148:3301" will pass to kernel 2024-09-21T07:35:57.598 INFO:teuthology.orchestra.run.smithi039.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T07:35:57.598 INFO:teuthology.orchestra.run.smithi039.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:35:57.599 INFO:teuthology.orchestra.run.smithi039.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:35:57.599 INFO:teuthology.orchestra.run.smithi039.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:35:57.599 INFO:teuthology.orchestra.run.smithi039.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:35:57.601 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:35:57.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:35:57.602 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:35:57.658 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:35:57.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:35:57.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:35:57.755 INFO:teuthology.orchestra.run.smithi039.stdout:client5962 2024-09-21T07:35:57.756 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:35:57.756 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:35:57.791 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get mds_max_snaps_per_dir 2024-09-21T07:35:58.079 INFO:teuthology.orchestra.run.smithi039.stdout:{"mds_max_snaps_per_dir":"100"} 2024-09-21T07:35:58.080 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_max_snaps_per_dir": "100" } 2024-09-21T07:35:58.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:35:58.080 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test-allow-snaps) 2024-09-21T07:35:58.124 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps false --yes-i-really-mean-it 2024-09-21T07:35:58.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:58.394+0000 7fbb40c00640 1 -- 172.21.15.39:0/557257954 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb3c15fcc0 msgr2=0x7fbb3c152500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:58.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:58.394+0000 7fbb40c00640 1 --2- 172.21.15.39:0/557257954 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb3c15fcc0 0x7fbb3c152500 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c005fd0 tx=0x7fbb2c02a5d0 comp rx=0 tx=0).stop 2024-09-21T07:35:58.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:58.394+0000 7fbb40c00640 1 -- 172.21.15.39:0/557257954 shutdown_connections 2024-09-21T07:35:58.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:58.394+0000 7fbb40c00640 1 --2- 172.21.15.39:0/557257954 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb3c15fcc0 0x7fbb3c152500 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:58.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:58.394+0000 7fbb40c00640 1 -- 172.21.15.39:0/557257954 >> 172.21.15.39:0/557257954 conn(0x7fbb3c0cd3f0 msgr2=0x7fbb3c0cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:58.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:58.394+0000 7fbb40c00640 1 -- 172.21.15.39:0/557257954 shutdown_connections 2024-09-21T07:35:58.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:58.394+0000 7fbb40c00640 1 -- 172.21.15.39:0/557257954 wait complete. 2024-09-21T07:35:58.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:58.397+0000 7fbb40c00640 1 Processor -- start 2024-09-21T07:35:58.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:58.397+0000 7fbb40c00640 1 -- start start 2024-09-21T07:35:58.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:58.398+0000 7fbb40c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbb3c15bae0 0x7fbb3c0c9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:58.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:58.398+0000 7fbb40c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb3c15fcc0 0x7fbb3c1f8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:58.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:58.398+0000 7fbb40c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb3c0ae2b0 0x7fbb3c1f8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:58.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:58.398+0000 7fbb40c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3c0c6b40 con 0x7fbb3c15fcc0 2024-09-21T07:35:58.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:58.398+0000 7fbb40c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3c0c69c0 con 0x7fbb3c0ae2b0 2024-09-21T07:35:58.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:58.398+0000 7fbb40c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fbb3c0c6cc0 con 0x7fbb3c15bae0 2024-09-21T07:35:58.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:58.399+0000 7fbb33e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb3c15fcc0 0x7fbb3c1f8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:58.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:58.399+0000 7fbb33e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb3c15fcc0 0x7fbb3c1f8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45674/0 (socket says 172.21.15.39:45674) 2024-09-21T07:35:58.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:58.399+0000 7fbb33e00640 1 -- 172.21.15.39:0/2790527219 learned_addr learned my addr 172.21.15.39:0/2790527219 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:58.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:58.399+0000 7fbb3b400640 1 --2- 172.21.15.39:0/2790527219 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbb3c15bae0 0x7fbb3c0c9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:58.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:58.399+0000 7fbb3be00640 1 --2- 172.21.15.39:0/2790527219 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb3c0ae2b0 0x7fbb3c1f8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:58.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:58.399+0000 7fbb33e00640 1 -- 172.21.15.39:0/2790527219 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbb3c15bae0 msgr2=0x7fbb3c0c9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:58.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:58.399+0000 7fbb33e00640 1 --2- 172.21.15.39:0/2790527219 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbb3c15bae0 0x7fbb3c0c9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:58.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:58.399+0000 7fbb33e00640 1 -- 172.21.15.39:0/2790527219 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb3c0ae2b0 msgr2=0x7fbb3c1f8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:58.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:58.399+0000 7fbb33e00640 1 --2- 172.21.15.39:0/2790527219 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb3c0ae2b0 0x7fbb3c1f8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:58.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:58.399+0000 7fbb33e00640 1 -- 172.21.15.39:0/2790527219 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb3c0ca3d0 con 0x7fbb3c15fcc0 2024-09-21T07:35:58.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:58.399+0000 7fbb33e00640 1 --2- 172.21.15.39:0/2790527219 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb3c15fcc0 0x7fbb3c1f8a80 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fbb2400c3c0 tx=0x7fbb2400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:58.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:58.400+0000 7fbb33400640 1 -- 172.21.15.39:0/2790527219 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb24017070 con 0x7fbb3c15fcc0 2024-09-21T07:35:58.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:58.400+0000 7fbb33400640 1 -- 172.21.15.39:0/2790527219 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb240035f0 con 0x7fbb3c15fcc0 2024-09-21T07:35:58.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:58.400+0000 7fbb33400640 1 -- 172.21.15.39:0/2790527219 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb24003a80 con 0x7fbb3c15fcc0 2024-09-21T07:35:58.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:58.400+0000 7fbb40c00640 1 -- 172.21.15.39:0/2790527219 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb3c0ccb40 con 0x7fbb3c15fcc0 2024-09-21T07:35:58.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:58.400+0000 7fbb40c00640 1 -- 172.21.15.39:0/2790527219 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb3c0cc200 con 0x7fbb3c15fcc0 2024-09-21T07:35:58.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:58.401+0000 7fbb40c00640 1 -- 172.21.15.39:0/2790527219 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb3c1f9690 con 0x7fbb3c15fcc0 2024-09-21T07:35:58.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:58.402+0000 7fbb33400640 1 -- 172.21.15.39:0/2790527219 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbb2401a070 con 0x7fbb3c15fcc0 2024-09-21T07:35:58.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:58.403+0000 7fbb33400640 1 --2- 172.21.15.39:0/2790527219 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbb18078b90 0x7fbb1807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:58.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:58.403+0000 7fbb33400640 1 -- 172.21.15.39:0/2790527219 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6772+0+0 (secure 0 0 0) 0x7fbb2409a3e0 con 0x7fbb3c15fcc0 2024-09-21T07:35:58.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:58.404+0000 7fbb3b400640 1 --2- 172.21.15.39:0/2790527219 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbb18078b90 0x7fbb1807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:58.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:58.404+0000 7fbb3b400640 1 --2- 172.21.15.39:0/2790527219 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbb18078b90 0x7fbb1807b0a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c033000 tx=0x7fbb2c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:58.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:58.408+0000 7fbb33400640 1 -- 172.21.15.39:0/2790527219 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbb24061d90 con 0x7fbb3c15fcc0 2024-09-21T07:35:58.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:58.643+0000 7fbb40c00640 1 -- 172.21.15.39:0/2790527219 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "false", "yes_i_really_mean_it": true} v 0) -- 0x7fbb3c0c71f0 con 0x7fbb3c15fcc0 2024-09-21T07:35:59.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.509+0000 7fbb33400640 1 -- 172.21.15.39:0/2790527219 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "false", "yes_i_really_mean_it": true}]=0 disabled new snapshots v45) ==== 169+0+0 (secure 0 0 0) 0x7fbb240667c0 con 0x7fbb3c15fcc0 2024-09-21T07:35:59.512 INFO:teuthology.orchestra.run.smithi039.stderr:disabled new snapshots 2024-09-21T07:35:59.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.512+0000 7fbb40c00640 1 -- 172.21.15.39:0/2790527219 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbb18078b90 msgr2=0x7fbb1807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:59.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.512+0000 7fbb40c00640 1 --2- 172.21.15.39:0/2790527219 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbb18078b90 0x7fbb1807b0a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c033000 tx=0x7fbb2c004730 comp rx=0 tx=0).stop 2024-09-21T07:35:59.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.512+0000 7fbb40c00640 1 -- 172.21.15.39:0/2790527219 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb3c15fcc0 msgr2=0x7fbb3c1f8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:59.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.512+0000 7fbb40c00640 1 --2- 172.21.15.39:0/2790527219 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb3c15fcc0 0x7fbb3c1f8a80 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fbb2400c3c0 tx=0x7fbb2400c880 comp rx=0 tx=0).stop 2024-09-21T07:35:59.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.512+0000 7fbb40c00640 1 -- 172.21.15.39:0/2790527219 shutdown_connections 2024-09-21T07:35:59.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.512+0000 7fbb40c00640 1 --2- 172.21.15.39:0/2790527219 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbb18078b90 0x7fbb1807b0a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:59.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.512+0000 7fbb40c00640 1 --2- 172.21.15.39:0/2790527219 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb3c0ae2b0 0x7fbb3c1f8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:59.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.512+0000 7fbb40c00640 1 --2- 172.21.15.39:0/2790527219 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb3c15fcc0 0x7fbb3c1f8a80 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:59.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.512+0000 7fbb40c00640 1 --2- 172.21.15.39:0/2790527219 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbb3c15bae0 0x7fbb3c0c9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:59.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.512+0000 7fbb40c00640 1 -- 172.21.15.39:0/2790527219 >> 172.21.15.39:0/2790527219 conn(0x7fbb3c0cd3f0 msgr2=0x7fbb3c068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:59.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.513+0000 7fbb40c00640 1 -- 172.21.15.39:0/2790527219 shutdown_connections 2024-09-21T07:35:59.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.513+0000 7fbb40c00640 1 -- 172.21.15.39:0/2790527219 wait complete. 2024-09-21T07:35:59.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:35:59.542 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test-allow-snaps/.snap/snap00) 2024-09-21T07:35:59.581 INFO:teuthology.orchestra.run.smithi039.stderr:mkdir: cannot create directory ‘test-allow-snaps/.snap/snap00’: Operation not permitted 2024-09-21T07:35:59.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:35:59.582 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2024-09-21T07:35:59.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.852+0000 7fa92ca00640 1 -- 172.21.15.39:0/3585504110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9280cc560 msgr2=0x7fa928073820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:59.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.852+0000 7fa92ca00640 1 --2- 172.21.15.39:0/3585504110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9280cc560 0x7fa928073820 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa928074b80 tx=0x7fa91802a4d0 comp rx=0 tx=0).stop 2024-09-21T07:35:59.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.852+0000 7fa92ca00640 1 -- 172.21.15.39:0/3585504110 shutdown_connections 2024-09-21T07:35:59.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.852+0000 7fa92ca00640 1 --2- 172.21.15.39:0/3585504110 >> v2:172.21.15.148:3300/0 conn(0x7fa9280cb4d0 0x7fa9280acf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:59.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.852+0000 7fa92ca00640 1 --2- 172.21.15.39:0/3585504110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9280cc560 0x7fa928073820 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:59.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.852+0000 7fa92ca00640 1 -- 172.21.15.39:0/3585504110 >> 172.21.15.39:0/3585504110 conn(0x7fa9280d0130 msgr2=0x7fa9280d0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:35:59.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.852+0000 7fa92ca00640 1 -- 172.21.15.39:0/3585504110 shutdown_connections 2024-09-21T07:35:59.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.852+0000 7fa92ca00640 1 -- 172.21.15.39:0/3585504110 wait complete. 2024-09-21T07:35:59.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.854+0000 7fa92ca00640 1 Processor -- start 2024-09-21T07:35:59.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.854+0000 7fa92ca00640 1 -- start start 2024-09-21T07:35:59.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.856+0000 7fa92ca00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa9280cc560 0x7fa92815b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:59.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.856+0000 7fa92ca00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa92815fb80 0x7fa92815bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:59.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.856+0000 7fa92ca00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9280b2f60 0x7fa9280b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:59.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.856+0000 7fa92ca00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa9280ad960 con 0x7fa9280b2f60 2024-09-21T07:35:59.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.856+0000 7fa92ca00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa9280ad7e0 con 0x7fa92815fb80 2024-09-21T07:35:59.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.856+0000 7fa92ca00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa9280adae0 con 0x7fa9280cc560 2024-09-21T07:35:59.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.856+0000 7fa926800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9280b2f60 0x7fa9280b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:59.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.856+0000 7fa926800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9280b2f60 0x7fa9280b5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45684/0 (socket says 172.21.15.39:45684) 2024-09-21T07:35:59.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.856+0000 7fa926800640 1 -- 172.21.15.39:0/4249716820 learned_addr learned my addr 172.21.15.39:0/4249716820 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:35:59.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.856+0000 7fa91fe00640 1 --2- 172.21.15.39:0/4249716820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa92815fb80 0x7fa92815bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:59.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.856+0000 7fa925e00640 1 --2- 172.21.15.39:0/4249716820 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa9280cc560 0x7fa92815b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:59.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.857+0000 7fa926800640 1 -- 172.21.15.39:0/4249716820 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa9280cc560 msgr2=0x7fa92815b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:59.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.857+0000 7fa926800640 1 --2- 172.21.15.39:0/4249716820 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa9280cc560 0x7fa92815b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:59.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.857+0000 7fa926800640 1 -- 172.21.15.39:0/4249716820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa92815fb80 msgr2=0x7fa92815bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:35:59.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.857+0000 7fa926800640 1 --2- 172.21.15.39:0/4249716820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa92815fb80 0x7fa92815bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:35:59.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.857+0000 7fa926800640 1 -- 172.21.15.39:0/4249716820 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9280ce510 con 0x7fa9280b2f60 2024-09-21T07:35:59.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.857+0000 7fa926800640 1 --2- 172.21.15.39:0/4249716820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9280b2f60 0x7fa9280b5130 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa92000e960 tx=0x7fa92000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:59.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.857+0000 7fa91e000640 1 -- 172.21.15.39:0/4249716820 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa920004d20 con 0x7fa9280b2f60 2024-09-21T07:35:59.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.857+0000 7fa91e000640 1 -- 172.21.15.39:0/4249716820 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa92000cb20 con 0x7fa9280b2f60 2024-09-21T07:35:59.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.857+0000 7fa92ca00640 1 -- 172.21.15.39:0/4249716820 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9280ce7f0 con 0x7fa9280b2f60 2024-09-21T07:35:59.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.857+0000 7fa91e000640 1 -- 172.21.15.39:0/4249716820 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa920005020 con 0x7fa9280b2f60 2024-09-21T07:35:59.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.857+0000 7fa92ca00640 1 -- 172.21.15.39:0/4249716820 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9280b4c10 con 0x7fa9280b2f60 2024-09-21T07:35:59.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.858+0000 7fa92ca00640 1 -- 172.21.15.39:0/4249716820 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9280ae000 con 0x7fa9280b2f60 2024-09-21T07:35:59.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.860+0000 7fa91e000640 1 -- 172.21.15.39:0/4249716820 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa9200051c0 con 0x7fa9280b2f60 2024-09-21T07:35:59.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.861+0000 7fa91e000640 1 --2- 172.21.15.39:0/4249716820 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa904078b90 0x7fa90407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:35:59.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.861+0000 7fa91e000640 1 -- 172.21.15.39:0/4249716820 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6772+0+0 (secure 0 0 0) 0x7fa92009ba40 con 0x7fa9280b2f60 2024-09-21T07:35:59.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.861+0000 7fa925e00640 1 --2- 172.21.15.39:0/4249716820 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa904078b90 0x7fa90407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:35:59.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.862+0000 7fa925e00640 1 --2- 172.21.15.39:0/4249716820 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa904078b90 0x7fa90407b0a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa918033000 tx=0x7fa918004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:35:59.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:35:59.866+0000 7fa91e000640 1 -- 172.21.15.39:0/4249716820 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa920016030 con 0x7fa9280b2f60 2024-09-21T07:36:00.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:00.105+0000 7fa92ca00640 1 -- 172.21.15.39:0/4249716820 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fa9280ae740 con 0x7fa9280b2f60 2024-09-21T07:36:00.254 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:36:00.258 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:36:00.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:00.516+0000 7fa91e000640 1 -- 172.21.15.39:0/4249716820 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v46) ==== 167+0+0 (secure 0 0 0) 0x7fa9200633f0 con 0x7fa9280b2f60 2024-09-21T07:36:00.519 INFO:teuthology.orchestra.run.smithi039.stderr:enabled new snapshots 2024-09-21T07:36:00.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:00.519+0000 7fa92ca00640 1 -- 172.21.15.39:0/4249716820 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa904078b90 msgr2=0x7fa90407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:00.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:00.519+0000 7fa92ca00640 1 --2- 172.21.15.39:0/4249716820 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa904078b90 0x7fa90407b0a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa918033000 tx=0x7fa918004730 comp rx=0 tx=0).stop 2024-09-21T07:36:00.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:00.519+0000 7fa92ca00640 1 -- 172.21.15.39:0/4249716820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9280b2f60 msgr2=0x7fa9280b5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:00.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:00.519+0000 7fa92ca00640 1 --2- 172.21.15.39:0/4249716820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9280b2f60 0x7fa9280b5130 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa92000e960 tx=0x7fa92000ee20 comp rx=0 tx=0).stop 2024-09-21T07:36:00.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:00.519+0000 7fa92ca00640 1 -- 172.21.15.39:0/4249716820 shutdown_connections 2024-09-21T07:36:00.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:00.519+0000 7fa92ca00640 1 --2- 172.21.15.39:0/4249716820 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa904078b90 0x7fa90407b0a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:00.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:00.519+0000 7fa92ca00640 1 --2- 172.21.15.39:0/4249716820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9280b2f60 0x7fa9280b5130 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:00.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:00.519+0000 7fa92ca00640 1 --2- 172.21.15.39:0/4249716820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa92815fb80 0x7fa92815bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:00.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:00.519+0000 7fa92ca00640 1 --2- 172.21.15.39:0/4249716820 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa9280cc560 0x7fa92815b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:00.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:00.519+0000 7fa92ca00640 1 -- 172.21.15.39:0/4249716820 >> 172.21.15.39:0/4249716820 conn(0x7fa9280d0130 msgr2=0x7fa9280cb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:00.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:00.519+0000 7fa92ca00640 1 -- 172.21.15.39:0/4249716820 shutdown_connections 2024-09-21T07:36:00.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:00.519+0000 7fa92ca00640 1 -- 172.21.15.39:0/4249716820 wait complete. 2024-09-21T07:36:00.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:36:00.543 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test-allow-snaps/.snap/snap00) 2024-09-21T07:36:00.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:36:00.606 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir test-allow-snaps/.snap/snap00) 2024-09-21T07:36:00.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:36:00.649 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir test-allow-snaps) 2024-09-21T07:36:00.727 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:36:00.727 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:36:00.861 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:36:00.950 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:36:00.950 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:36:00.950 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:36:00.951 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:36:00.986 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:36:00.986 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:36:01.061 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:36:01.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.327+0000 7f25d6800640 1 -- 172.21.15.39:0/92745203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25d00d0260 msgr2=0x7f25d0161340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:01.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.327+0000 7f25d6800640 1 --2- 172.21.15.39:0/92745203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25d00d0260 0x7f25d0161340 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f25c8005fd0 tx=0x7f25c802aaa0 comp rx=0 tx=0).stop 2024-09-21T07:36:01.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.327+0000 7f25d6800640 1 -- 172.21.15.39:0/92745203 shutdown_connections 2024-09-21T07:36:01.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.327+0000 7f25d6800640 1 --2- 172.21.15.39:0/92745203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25d00d0260 0x7f25d0161340 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:01.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.327+0000 7f25d6800640 1 --2- 172.21.15.39:0/92745203 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f25d00d2af0 0x7f25d00b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:01.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.327+0000 7f25d6800640 1 -- 172.21.15.39:0/92745203 >> 172.21.15.39:0/92745203 conn(0x7f25d009d690 msgr2=0x7f25d00d4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:01.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.327+0000 7f25d6800640 1 -- 172.21.15.39:0/92745203 shutdown_connections 2024-09-21T07:36:01.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.327+0000 7f25d6800640 1 -- 172.21.15.39:0/92745203 wait complete. 2024-09-21T07:36:01.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.329+0000 7f25d6800640 1 Processor -- start 2024-09-21T07:36:01.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.329+0000 7f25d6800640 1 -- start start 2024-09-21T07:36:01.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.331+0000 7f25d6800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25d00b3b60 0x7f25d00cd470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:01.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.331+0000 7f25d6800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f25d00d0260 0x7f25d0156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:01.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.331+0000 7f25d6800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25d00d2af0 0x7f25d015a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:01.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.331+0000 7f25d6800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f25d0161880 con 0x7f25d00b3b60 2024-09-21T07:36:01.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.331+0000 7f25d6800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f25d00d0880 con 0x7f25d00d2af0 2024-09-21T07:36:01.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.331+0000 7f25d6800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f25d0161a00 con 0x7f25d00d0260 2024-09-21T07:36:01.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.331+0000 7f25d5400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25d00b3b60 0x7f25d00cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:01.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.331+0000 7f25d5400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25d00b3b60 0x7f25d00cd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45690/0 (socket says 172.21.15.39:45690) 2024-09-21T07:36:01.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.331+0000 7f25d5400640 1 -- 172.21.15.39:0/3137960727 learned_addr learned my addr 172.21.15.39:0/3137960727 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:01.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.331+0000 7f25d5e00640 1 --2- 172.21.15.39:0/3137960727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25d00d2af0 0x7f25d015a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:01.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.331+0000 7f25d4a00640 1 --2- 172.21.15.39:0/3137960727 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f25d00d0260 0x7f25d0156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:01.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.331+0000 7f25d5400640 1 -- 172.21.15.39:0/3137960727 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f25d00d0260 msgr2=0x7f25d0156ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:01.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.331+0000 7f25d5400640 1 --2- 172.21.15.39:0/3137960727 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f25d00d0260 0x7f25d0156ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:01.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.331+0000 7f25d5400640 1 -- 172.21.15.39:0/3137960727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25d00d2af0 msgr2=0x7f25d015a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:01.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.331+0000 7f25d5400640 1 --2- 172.21.15.39:0/3137960727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25d00d2af0 0x7f25d015a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:01.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.331+0000 7f25d5400640 1 -- 172.21.15.39:0/3137960727 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25d00cbc60 con 0x7f25d00b3b60 2024-09-21T07:36:01.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.332+0000 7f25d5400640 1 --2- 172.21.15.39:0/3137960727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25d00b3b60 0x7f25d00cd470 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f25c400d770 tx=0x7f25c400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:01.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.332+0000 7f25cd600640 1 -- 172.21.15.39:0/3137960727 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c4017070 con 0x7f25d00b3b60 2024-09-21T07:36:01.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.332+0000 7f25cd600640 1 -- 172.21.15.39:0/3137960727 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25c4009d20 con 0x7f25d00b3b60 2024-09-21T07:36:01.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.332+0000 7f25d6800640 1 -- 172.21.15.39:0/3137960727 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25d00cbf00 con 0x7f25d00b3b60 2024-09-21T07:36:01.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.332+0000 7f25cd600640 1 -- 172.21.15.39:0/3137960727 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c40106c0 con 0x7f25d00b3b60 2024-09-21T07:36:01.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.332+0000 7f25d6800640 1 -- 172.21.15.39:0/3137960727 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25d00cc4c0 con 0x7f25d00b3b60 2024-09-21T07:36:01.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.333+0000 7f25d6800640 1 -- 172.21.15.39:0/3137960727 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25d00b5560 con 0x7f25d00b3b60 2024-09-21T07:36:01.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.334+0000 7f25cd600640 1 -- 172.21.15.39:0/3137960727 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f25c401a070 con 0x7f25d00b3b60 2024-09-21T07:36:01.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.335+0000 7f25cd600640 1 --2- 172.21.15.39:0/3137960727 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f25b0078b90 0x7f25b007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:01.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.335+0000 7f25cd600640 1 -- 172.21.15.39:0/3137960727 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6772+0+0 (secure 0 0 0) 0x7f25c409b1d0 con 0x7f25d00b3b60 2024-09-21T07:36:01.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.336+0000 7f25d4a00640 1 --2- 172.21.15.39:0/3137960727 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f25b0078b90 0x7f25b007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:01.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.336+0000 7f25d4a00640 1 --2- 172.21.15.39:0/3137960727 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f25b0078b90 0x7f25b007b0a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f25d015a4f0 tx=0x7f25bc009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:01.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.340+0000 7f25cd600640 1 -- 172.21.15.39:0/3137960727 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f25c4063300 con 0x7f25d00b3b60 2024-09-21T07:36:01.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.548+0000 7f25d6800640 1 -- 172.21.15.39:0/3137960727 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f25d0069180 con 0x7f25d00b3b60 2024-09-21T07:36:01.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.549+0000 7f25cd600640 1 -- 172.21.15.39:0/3137960727 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v78) ==== 80+0+0 (secure 0 0 0) 0x7f25c4062c00 con 0x7f25d00b3b60 2024-09-21T07:36:01.551 INFO:teuthology.orchestra.run.smithi039.stderr:listed 0 entries 2024-09-21T07:36:01.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.552+0000 7f25d6800640 1 -- 172.21.15.39:0/3137960727 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f25b0078b90 msgr2=0x7f25b007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:01.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.552+0000 7f25d6800640 1 --2- 172.21.15.39:0/3137960727 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f25b0078b90 0x7f25b007b0a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f25d015a4f0 tx=0x7f25bc009040 comp rx=0 tx=0).stop 2024-09-21T07:36:01.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.552+0000 7f25d6800640 1 -- 172.21.15.39:0/3137960727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25d00b3b60 msgr2=0x7f25d00cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:01.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.552+0000 7f25d6800640 1 --2- 172.21.15.39:0/3137960727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25d00b3b60 0x7f25d00cd470 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f25c400d770 tx=0x7f25c400dc30 comp rx=0 tx=0).stop 2024-09-21T07:36:01.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.552+0000 7f25d6800640 1 -- 172.21.15.39:0/3137960727 shutdown_connections 2024-09-21T07:36:01.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.552+0000 7f25d6800640 1 --2- 172.21.15.39:0/3137960727 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f25b0078b90 0x7f25b007b0a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:01.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.552+0000 7f25d6800640 1 --2- 172.21.15.39:0/3137960727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25d00d2af0 0x7f25d015a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:01.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.552+0000 7f25d6800640 1 --2- 172.21.15.39:0/3137960727 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f25d00d0260 0x7f25d0156ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:01.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.552+0000 7f25d6800640 1 --2- 172.21.15.39:0/3137960727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25d00b3b60 0x7f25d00cd470 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:01.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.552+0000 7f25d6800640 1 -- 172.21.15.39:0/3137960727 >> 172.21.15.39:0/3137960727 conn(0x7f25d009d690 msgr2=0x7f25d00d4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:01.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.552+0000 7f25d6800640 1 -- 172.21.15.39:0/3137960727 shutdown_connections 2024-09-21T07:36:01.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:01.552+0000 7f25d6800640 1 -- 172.21.15.39:0/3137960727 wait complete. 2024-09-21T07:36:01.579 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T07:36:01.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:36:01.579 DEBUG:teuthology.orchestra.run.smithi039:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T07:36:01.649 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T07:36:01.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:36:01.649 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:36:01.685 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:36:01.686 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:36:01.686 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:36:01.725 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:36:01.726 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:36:01.726 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:36:01.726 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:36:01.761 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:36:01.761 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:36:01.761 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:36:01.761 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:36:01.804 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:36:01.804 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:36:02.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.029+0000 7f858ca00640 1 -- 172.21.15.39:0/437137475 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f856c0051a0 msgr2=0x7f856c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:02.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.029+0000 7f858ca00640 1 --2- 172.21.15.39:0/437137475 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f856c0051a0 0x7f856c00b9c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f85880d1cc0 tx=0x7f856c005830 comp rx=0 tx=0).stop 2024-09-21T07:36:02.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.029+0000 7f8586800640 1 -- 172.21.15.39:0/437137475 reap_dead start 2024-09-21T07:36:02.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.029+0000 7f858ca00640 1 -- 172.21.15.39:0/437137475 shutdown_connections 2024-09-21T07:36:02.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.029+0000 7f858ca00640 1 -- 172.21.15.39:0/437137475 >> 172.21.15.39:0/437137475 conn(0x7f858806d4f0 msgr2=0x7f8588071130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:02.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.030+0000 7f858ca00640 1 -- 172.21.15.39:0/437137475 shutdown_connections 2024-09-21T07:36:02.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.030+0000 7f858ca00640 1 -- 172.21.15.39:0/437137475 wait complete. 2024-09-21T07:36:02.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.032+0000 7f858ca00640 1 Processor -- start 2024-09-21T07:36:02.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.032+0000 7f858ca00640 1 -- start start 2024-09-21T07:36:02.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.033+0000 7f858ca00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f856c0051a0 0x7f85880cd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:02.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.033+0000 7f858ca00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f858815a720 0x7f85880cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:02.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.033+0000 7f858ca00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85880b3910 0x7f85880b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:02.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.033+0000 7f858ca00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f85880d0350 con 0x7f85880b3910 2024-09-21T07:36:02.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.033+0000 7f858ca00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f85880d01d0 con 0x7f858815a720 2024-09-21T07:36:02.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.033+0000 7f858ca00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f85880d04d0 con 0x7f856c0051a0 2024-09-21T07:36:02.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.034+0000 7f8586800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85880b3910 0x7f85880b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:02.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.034+0000 7f8586800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85880b3910 0x7f85880b3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48342/0 (socket says 172.21.15.39:48342) 2024-09-21T07:36:02.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.034+0000 7f8586800640 1 -- 172.21.15.39:0/4000486463 learned_addr learned my addr 172.21.15.39:0/4000486463 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:02.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.034+0000 7f8585e00640 1 --2- 172.21.15.39:0/4000486463 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f856c0051a0 0x7f85880cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:02.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.034+0000 7f8585400640 1 --2- 172.21.15.39:0/4000486463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f858815a720 0x7f85880cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:02.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.034+0000 7f8586800640 1 -- 172.21.15.39:0/4000486463 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f856c0051a0 msgr2=0x7f85880cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:02.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.034+0000 7f8586800640 1 --2- 172.21.15.39:0/4000486463 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f856c0051a0 0x7f85880cd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:02.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.034+0000 7f8586800640 1 -- 172.21.15.39:0/4000486463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f858815a720 msgr2=0x7f85880cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:02.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.034+0000 7f8586800640 1 --2- 172.21.15.39:0/4000486463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f858815a720 0x7f85880cd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:02.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.034+0000 7f8586800640 1 -- 172.21.15.39:0/4000486463 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f858815ae50 con 0x7f85880b3910 2024-09-21T07:36:02.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.034+0000 7f8586800640 1 --2- 172.21.15.39:0/4000486463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85880b3910 0x7f85880b3d20 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f858002b640 tx=0x7f858000a450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:02.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.035+0000 7f857e000640 1 -- 172.21.15.39:0/4000486463 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8580004e30 con 0x7f85880b3910 2024-09-21T07:36:02.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.035+0000 7f857e000640 1 -- 172.21.15.39:0/4000486463 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f858002ba80 con 0x7f85880b3910 2024-09-21T07:36:02.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.035+0000 7f857e000640 1 -- 172.21.15.39:0/4000486463 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85800060e0 con 0x7f85880b3910 2024-09-21T07:36:02.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.035+0000 7f858ca00640 1 -- 172.21.15.39:0/4000486463 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f858815b130 con 0x7f85880b3910 2024-09-21T07:36:02.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.035+0000 7f858ca00640 1 -- 172.21.15.39:0/4000486463 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f858815b6f0 con 0x7f85880b3910 2024-09-21T07:36:02.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.036+0000 7f858ca00640 1 -- 172.21.15.39:0/4000486463 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f858815abc0 con 0x7f85880b3910 2024-09-21T07:36:02.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.037+0000 7f857e000640 1 -- 172.21.15.39:0/4000486463 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8580006280 con 0x7f85880b3910 2024-09-21T07:36:02.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.038+0000 7f857e000640 1 --2- 172.21.15.39:0/4000486463 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8568078b90 0x7f856807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:02.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.038+0000 7f857e000640 1 -- 172.21.15.39:0/4000486463 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6772+0+0 (secure 0 0 0) 0x7f85800bada0 con 0x7f85880b3910 2024-09-21T07:36:02.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.039+0000 7f8585e00640 1 --2- 172.21.15.39:0/4000486463 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8568078b90 0x7f856807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:02.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.040+0000 7f8585e00640 1 --2- 172.21.15.39:0/4000486463 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8568078b90 0x7f856807b0a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8578009c50 tx=0x7f8578005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:02.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.043+0000 7f857e000640 1 -- 172.21.15.39:0/4000486463 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f85880d0b20 con 0x7f85880b3910 2024-09-21T07:36:02.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.280+0000 7f858ca00640 1 -- 172.21.15.39:0/4000486463 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f858806f1a0 con 0x7f85880b3910 2024-09-21T07:36:02.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.281+0000 7f857e000640 1 -- 172.21.15.39:0/4000486463 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5225 (secure 0 0 0) 0x7f85800826c0 con 0x7f85880b3910 2024-09-21T07:36:02.283 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:36:02.283 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":46,"btime":"2024-09-21T07:36:00:511098+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5426,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/707367131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":707367131},{"type":"v1","addr":"172.21.15.39:6835","nonce":707367131}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5429,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/784956164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":784956164},{"type":"v1","addr":"172.21.15.148:6836","nonce":784956164}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5432,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3742990359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3742990359},{"type":"v1","addr":"172.21.15.148:6837","nonce":3742990359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:35:48.965416+0000","modified":"2024-09-21T07:36:00.107330+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5842},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5842":{"gid":5842,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/1032917299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":1032917299},{"type":"v1","addr":"172.21.15.39:6834","nonce":1032917299}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5842,"qdb_cluster":[5842]},"id":5}]} 2024-09-21T07:36:02.284 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 46 2024-09-21T07:36:02.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.284+0000 7f858ca00640 1 -- 172.21.15.39:0/4000486463 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8568078b90 msgr2=0x7f856807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:02.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.284+0000 7f858ca00640 1 --2- 172.21.15.39:0/4000486463 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8568078b90 0x7f856807b0a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8578009c50 tx=0x7f8578005f90 comp rx=0 tx=0).stop 2024-09-21T07:36:02.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.284+0000 7f858ca00640 1 -- 172.21.15.39:0/4000486463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85880b3910 msgr2=0x7f85880b3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:02.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.284+0000 7f858ca00640 1 --2- 172.21.15.39:0/4000486463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85880b3910 0x7f85880b3d20 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f858002b640 tx=0x7f858000a450 comp rx=0 tx=0).stop 2024-09-21T07:36:02.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.284+0000 7f858ca00640 1 -- 172.21.15.39:0/4000486463 shutdown_connections 2024-09-21T07:36:02.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.284+0000 7f858ca00640 1 --2- 172.21.15.39:0/4000486463 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8568078b90 0x7f856807b0a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:02.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.284+0000 7f858ca00640 1 --2- 172.21.15.39:0/4000486463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85880b3910 0x7f85880b3d20 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:02.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.284+0000 7f858ca00640 1 --2- 172.21.15.39:0/4000486463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f858815a720 0x7f85880cd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:02.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.284+0000 7f858ca00640 1 --2- 172.21.15.39:0/4000486463 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f856c0051a0 0x7f85880cd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:02.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.284+0000 7f858ca00640 1 -- 172.21.15.39:0/4000486463 >> 172.21.15.39:0/4000486463 conn(0x7f858806d4f0 msgr2=0x7f85880cc070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:02.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.284+0000 7f858ca00640 1 -- 172.21.15.39:0/4000486463 shutdown_connections 2024-09-21T07:36:02.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.284+0000 7f858ca00640 1 -- 172.21.15.39:0/4000486463 wait complete. 2024-09-21T07:36:02.308 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:36:02.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.534+0000 7f8f81000640 1 -- 172.21.15.39:0/2987454730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f7c0ccb50 msgr2=0x7f8f7c0ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:02.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.534+0000 7f8f81000640 1 --2- 172.21.15.39:0/2987454730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f7c0ccb50 0x7f8f7c0ccf60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8f74005fd0 tx=0x7f8f7402a8b0 comp rx=0 tx=0).stop 2024-09-21T07:36:02.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.534+0000 7f8f81000640 1 -- 172.21.15.39:0/2987454730 shutdown_connections 2024-09-21T07:36:02.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.534+0000 7f8f81000640 1 --2- 172.21.15.39:0/2987454730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f7c0ccb50 0x7f8f7c0ccf60 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:02.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.534+0000 7f8f81000640 1 --2- 172.21.15.39:0/2987454730 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f7c0af3c0 0x7f8f7c0b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:02.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.534+0000 7f8f81000640 1 -- 172.21.15.39:0/2987454730 >> 172.21.15.39:0/2987454730 conn(0x7f8f7c06d7d0 msgr2=0x7f8f7c0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:02.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.535+0000 7f8f81000640 1 -- 172.21.15.39:0/2987454730 shutdown_connections 2024-09-21T07:36:02.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.535+0000 7f8f81000640 1 -- 172.21.15.39:0/2987454730 wait complete. 2024-09-21T07:36:02.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.537+0000 7f8f81000640 1 Processor -- start 2024-09-21T07:36:02.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.537+0000 7f8f81000640 1 -- start start 2024-09-21T07:36:02.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.539+0000 7f8f81000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f7c0af3c0 0x7f8f7c15e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:02.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.539+0000 7f8f81000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f7c0cc1e0 0x7f8f7c15eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:02.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.539+0000 7f8f81000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f7c0ccb50 0x7f8f7c1f6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:02.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.539+0000 7f8f81000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c153520 con 0x7f8f7c0af3c0 2024-09-21T07:36:02.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.539+0000 7f8f81000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c1533a0 con 0x7f8f7c0cc1e0 2024-09-21T07:36:02.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.539+0000 7f8f81000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f8f7c1536a0 con 0x7f8f7c0ccb50 2024-09-21T07:36:02.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.539+0000 7f8f79e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f7c0af3c0 0x7f8f7c15e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:02.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.539+0000 7f8f79e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f7c0af3c0 0x7f8f7c15e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48350/0 (socket says 172.21.15.39:48350) 2024-09-21T07:36:02.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.539+0000 7f8f79e00640 1 -- 172.21.15.39:0/1107003653 learned_addr learned my addr 172.21.15.39:0/1107003653 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:02.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.539+0000 7f8f79400640 1 --2- 172.21.15.39:0/1107003653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f7c0cc1e0 0x7f8f7c15eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:02.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.539+0000 7f8f7a800640 1 --2- 172.21.15.39:0/1107003653 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f7c0ccb50 0x7f8f7c1f6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:02.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.539+0000 7f8f79e00640 1 -- 172.21.15.39:0/1107003653 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f7c0ccb50 msgr2=0x7f8f7c1f6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:02.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.539+0000 7f8f79e00640 1 --2- 172.21.15.39:0/1107003653 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f7c0ccb50 0x7f8f7c1f6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:02.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.539+0000 7f8f79e00640 1 -- 172.21.15.39:0/1107003653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f7c0cc1e0 msgr2=0x7f8f7c15eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:02.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.539+0000 7f8f79e00640 1 --2- 172.21.15.39:0/1107003653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f7c0cc1e0 0x7f8f7c15eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:02.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.539+0000 7f8f79e00640 1 -- 172.21.15.39:0/1107003653 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f7c1e7e00 con 0x7f8f7c0af3c0 2024-09-21T07:36:02.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.539+0000 7f8f7a800640 1 --2- 172.21.15.39:0/1107003653 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f7c0ccb50 0x7f8f7c1f6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:36:02.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.540+0000 7f8f79e00640 1 --2- 172.21.15.39:0/1107003653 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f7c0af3c0 0x7f8f7c15e860 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c00c3a0 tx=0x7f8f6c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:02.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.540+0000 7f8f72000640 1 -- 172.21.15.39:0/1107003653 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c011790 con 0x7f8f7c0af3c0 2024-09-21T07:36:02.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.540+0000 7f8f72000640 1 -- 172.21.15.39:0/1107003653 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f6c017070 con 0x7f8f7c0af3c0 2024-09-21T07:36:02.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.540+0000 7f8f72000640 1 -- 172.21.15.39:0/1107003653 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c011cc0 con 0x7f8f7c0af3c0 2024-09-21T07:36:02.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.540+0000 7f8f81000640 1 -- 172.21.15.39:0/1107003653 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f7c09d7e0 con 0x7f8f7c0af3c0 2024-09-21T07:36:02.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.540+0000 7f8f81000640 1 -- 172.21.15.39:0/1107003653 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f7c09cb80 con 0x7f8f7c0af3c0 2024-09-21T07:36:02.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.541+0000 7f8f81000640 1 -- 172.21.15.39:0/1107003653 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f7c09cde0 con 0x7f8f7c0af3c0 2024-09-21T07:36:02.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.542+0000 7f8f72000640 1 -- 172.21.15.39:0/1107003653 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8f6c01c460 con 0x7f8f7c0af3c0 2024-09-21T07:36:02.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.543+0000 7f8f72000640 1 --2- 172.21.15.39:0/1107003653 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8f5c078bc0 0x7f8f5c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:02.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.543+0000 7f8f72000640 1 -- 172.21.15.39:0/1107003653 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6772+0+0 (secure 0 0 0) 0x7f8f6c09aee0 con 0x7f8f7c0af3c0 2024-09-21T07:36:02.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.544+0000 7f8f79400640 1 --2- 172.21.15.39:0/1107003653 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8f5c078bc0 0x7f8f5c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:02.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.544+0000 7f8f79400640 1 --2- 172.21.15.39:0/1107003653 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8f5c078bc0 0x7f8f5c07b0d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c15f4b0 tx=0x7f8f68005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:02.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.548+0000 7f8f72000640 1 -- 172.21.15.39:0/1107003653 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8f7c0cd4a0 con 0x7f8f7c0af3c0 2024-09-21T07:36:02.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.786+0000 7f8f81000640 1 -- 172.21.15.39:0/1107003653 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f7c153bc0 con 0x7f8f7c0af3c0 2024-09-21T07:36:02.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.787+0000 7f8f72000640 1 -- 172.21.15.39:0/1107003653 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5225 (secure 0 0 0) 0x7f8f6c062ee0 con 0x7f8f7c0af3c0 2024-09-21T07:36:02.789 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:36:02.789 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":46,"btime":"2024-09-21T07:36:00:511098+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5426,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/707367131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":707367131},{"type":"v1","addr":"172.21.15.39:6835","nonce":707367131}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5429,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/784956164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":784956164},{"type":"v1","addr":"172.21.15.148:6836","nonce":784956164}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5432,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3742990359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3742990359},{"type":"v1","addr":"172.21.15.148:6837","nonce":3742990359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:35:48.965416+0000","modified":"2024-09-21T07:36:00.107330+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5842},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5842":{"gid":5842,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/1032917299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":1032917299},{"type":"v1","addr":"172.21.15.39:6834","nonce":1032917299}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5842,"qdb_cluster":[5842]},"id":5}]} 2024-09-21T07:36:02.789 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 46 2024-09-21T07:36:02.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.790+0000 7f8f81000640 1 -- 172.21.15.39:0/1107003653 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8f5c078bc0 msgr2=0x7f8f5c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:02.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.790+0000 7f8f81000640 1 --2- 172.21.15.39:0/1107003653 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8f5c078bc0 0x7f8f5c07b0d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c15f4b0 tx=0x7f8f68005f90 comp rx=0 tx=0).stop 2024-09-21T07:36:02.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.790+0000 7f8f81000640 1 -- 172.21.15.39:0/1107003653 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f7c0af3c0 msgr2=0x7f8f7c15e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:02.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.790+0000 7f8f81000640 1 --2- 172.21.15.39:0/1107003653 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f7c0af3c0 0x7f8f7c15e860 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c00c3a0 tx=0x7f8f6c00c860 comp rx=0 tx=0).stop 2024-09-21T07:36:02.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.790+0000 7f8f81000640 1 -- 172.21.15.39:0/1107003653 shutdown_connections 2024-09-21T07:36:02.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.790+0000 7f8f81000640 1 --2- 172.21.15.39:0/1107003653 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8f5c078bc0 0x7f8f5c07b0d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:02.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.790+0000 7f8f81000640 1 --2- 172.21.15.39:0/1107003653 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f7c0ccb50 0x7f8f7c1f6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:02.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.790+0000 7f8f81000640 1 --2- 172.21.15.39:0/1107003653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f7c0cc1e0 0x7f8f7c15eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:02.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.790+0000 7f8f81000640 1 --2- 172.21.15.39:0/1107003653 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f7c0af3c0 0x7f8f7c15e860 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:02.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.790+0000 7f8f81000640 1 -- 172.21.15.39:0/1107003653 >> 172.21.15.39:0/1107003653 conn(0x7f8f7c06d7d0 msgr2=0x7f8f7c15aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:02.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.790+0000 7f8f81000640 1 -- 172.21.15.39:0/1107003653 shutdown_connections 2024-09-21T07:36:02.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:02.790+0000 7f8f81000640 1 -- 172.21.15.39:0/1107003653 wait complete. 2024-09-21T07:36:02.815 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:36:03.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.038+0000 7f18edc00640 1 -- 172.21.15.39:0/3664280873 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18e80cf660 msgr2=0x7f18e80cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:03.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.038+0000 7f18edc00640 1 --2- 172.21.15.39:0/3664280873 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18e80cf660 0x7f18e80cf010 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f18d8005fd0 tx=0x7f18d802b250 comp rx=0 tx=0).stop 2024-09-21T07:36:03.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.038+0000 7f18edc00640 1 -- 172.21.15.39:0/3664280873 shutdown_connections 2024-09-21T07:36:03.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.038+0000 7f18edc00640 1 --2- 172.21.15.39:0/3664280873 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18e80cf660 0x7f18e80cf010 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:03.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.038+0000 7f18edc00640 1 -- 172.21.15.39:0/3664280873 >> 172.21.15.39:0/3664280873 conn(0x7f18e80b1cb0 msgr2=0x7f18e80b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:03.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.039+0000 7f18edc00640 1 -- 172.21.15.39:0/3664280873 shutdown_connections 2024-09-21T07:36:03.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.039+0000 7f18edc00640 1 -- 172.21.15.39:0/3664280873 wait complete. 2024-09-21T07:36:03.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.041+0000 7f18edc00640 1 Processor -- start 2024-09-21T07:36:03.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.041+0000 7f18edc00640 1 -- start start 2024-09-21T07:36:03.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.042+0000 7f18edc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18e80cf660 0x7f18e81f2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:03.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.042+0000 7f18edc00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f18e80cfb90 0x7f18e80c6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:03.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.042+0000 7f18edc00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18e81e94f0 0x7f18e815b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:03.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.042+0000 7f18edc00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f18e80710a0 con 0x7f18e80cf660 2024-09-21T07:36:03.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.042+0000 7f18edc00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f18e8070f20 con 0x7f18e81e94f0 2024-09-21T07:36:03.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.042+0000 7f18edc00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f18e80b36c0 con 0x7f18e80cfb90 2024-09-21T07:36:03.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.043+0000 7f18e6a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18e80cf660 0x7f18e81f2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:03.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.043+0000 7f18e6a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18e80cf660 0x7f18e81f2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48366/0 (socket says 172.21.15.39:48366) 2024-09-21T07:36:03.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.043+0000 7f18e6a00640 1 -- 172.21.15.39:0/455610094 learned_addr learned my addr 172.21.15.39:0/455610094 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:03.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.043+0000 7f18e6000640 1 --2- 172.21.15.39:0/455610094 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f18e80cfb90 0x7f18e80c6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:03.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.043+0000 7f18e7400640 1 --2- 172.21.15.39:0/455610094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18e81e94f0 0x7f18e815b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:03.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.043+0000 7f18e6a00640 1 -- 172.21.15.39:0/455610094 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f18e80cfb90 msgr2=0x7f18e80c6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:03.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.043+0000 7f18e6a00640 1 --2- 172.21.15.39:0/455610094 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f18e80cfb90 0x7f18e80c6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:03.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.043+0000 7f18e6a00640 1 -- 172.21.15.39:0/455610094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18e81e94f0 msgr2=0x7f18e815b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:03.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.043+0000 7f18e6a00640 1 --2- 172.21.15.39:0/455610094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18e81e94f0 0x7f18e815b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:03.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.043+0000 7f18e6a00640 1 -- 172.21.15.39:0/455610094 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18e80c6b40 con 0x7f18e80cf660 2024-09-21T07:36:03.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.043+0000 7f18e7400640 1 --2- 172.21.15.39:0/455610094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18e81e94f0 0x7f18e815b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:36:03.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.043+0000 7f18e6a00640 1 --2- 172.21.15.39:0/455610094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18e80cf660 0x7f18e81f2c60 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f18d802b840 tx=0x7f18d80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:03.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.044+0000 7f18dea00640 1 -- 172.21.15.39:0/455610094 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d8040070 con 0x7f18e80cf660 2024-09-21T07:36:03.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.044+0000 7f18dea00640 1 -- 172.21.15.39:0/455610094 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18d800a0d0 con 0x7f18e80cf660 2024-09-21T07:36:03.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.044+0000 7f18dea00640 1 -- 172.21.15.39:0/455610094 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d800a4a0 con 0x7f18e80cf660 2024-09-21T07:36:03.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.044+0000 7f18edc00640 1 -- 172.21.15.39:0/455610094 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18e81f3640 con 0x7f18e80cf660 2024-09-21T07:36:03.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.044+0000 7f18edc00640 1 -- 172.21.15.39:0/455610094 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18e80c5e00 con 0x7f18e80cf660 2024-09-21T07:36:03.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.045+0000 7f18edc00640 1 -- 172.21.15.39:0/455610094 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18e80b3b70 con 0x7f18e80cf660 2024-09-21T07:36:03.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.046+0000 7f18dea00640 1 -- 172.21.15.39:0/455610094 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f18d8043070 con 0x7f18e80cf660 2024-09-21T07:36:03.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.047+0000 7f18dea00640 1 --2- 172.21.15.39:0/455610094 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f18c8078bc0 0x7f18c807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:03.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.047+0000 7f18dea00640 1 -- 172.21.15.39:0/455610094 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6772+0+0 (secure 0 0 0) 0x7f18d80baec0 con 0x7f18e80cf660 2024-09-21T07:36:03.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.047+0000 7f18e6000640 1 --2- 172.21.15.39:0/455610094 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f18c8078bc0 0x7f18c807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:03.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.048+0000 7f18e6000640 1 --2- 172.21.15.39:0/455610094 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f18c8078bc0 0x7f18c807b0d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f18d4007fd0 tx=0x7f18d4009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:03.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.051+0000 7f18dea00640 1 -- 172.21.15.39:0/455610094 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f18d8082e30 con 0x7f18e80cf660 2024-09-21T07:36:03.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.248+0000 7f18edc00640 1 -- 172.21.15.39:0/455610094 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f18e806ddf0 con 0x7f18e80cf660 2024-09-21T07:36:03.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.250+0000 7f18dea00640 1 -- 172.21.15.39:0/455610094 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22087 (secure 0 0 0) 0x7f18d8082730 con 0x7f18e80cf660 2024-09-21T07:36:03.252 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:36:03.252 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":78,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:35:50.455024+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:35:46.914721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-21T07:35:47.926559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:36:03.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.252+0000 7f18edc00640 1 -- 172.21.15.39:0/455610094 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f18c8078bc0 msgr2=0x7f18c807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:03.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.252+0000 7f18edc00640 1 --2- 172.21.15.39:0/455610094 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f18c8078bc0 0x7f18c807b0d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f18d4007fd0 tx=0x7f18d4009040 comp rx=0 tx=0).stop 2024-09-21T07:36:03.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.252+0000 7f18edc00640 1 -- 172.21.15.39:0/455610094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18e80cf660 msgr2=0x7f18e81f2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:03.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.252+0000 7f18edc00640 1 --2- 172.21.15.39:0/455610094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18e80cf660 0x7f18e81f2c60 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f18d802b840 tx=0x7f18d80038a0 comp rx=0 tx=0).stop 2024-09-21T07:36:03.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.253+0000 7f18edc00640 1 -- 172.21.15.39:0/455610094 shutdown_connections 2024-09-21T07:36:03.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.253+0000 7f18edc00640 1 --2- 172.21.15.39:0/455610094 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f18c8078bc0 0x7f18c807b0d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:03.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.253+0000 7f18edc00640 1 --2- 172.21.15.39:0/455610094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18e81e94f0 0x7f18e815b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:03.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.253+0000 7f18edc00640 1 --2- 172.21.15.39:0/455610094 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f18e80cfb90 0x7f18e80c6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:03.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.253+0000 7f18edc00640 1 --2- 172.21.15.39:0/455610094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18e80cf660 0x7f18e81f2c60 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:03.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.253+0000 7f18edc00640 1 -- 172.21.15.39:0/455610094 >> 172.21.15.39:0/455610094 conn(0x7f18e80b1cb0 msgr2=0x7f18e80b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:03.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.253+0000 7f18edc00640 1 -- 172.21.15.39:0/455610094 shutdown_connections 2024-09-21T07:36:03.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.253+0000 7f18edc00640 1 -- 172.21.15.39:0/455610094 wait complete. 2024-09-21T07:36:03.277 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T07:36:03.277 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:36:03.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.501+0000 7ff2e6a00640 1 -- 172.21.15.39:0/4097955207 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2e0070440 msgr2=0x7ff2e00b3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:03.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.501+0000 7ff2e6a00640 1 --2- 172.21.15.39:0/4097955207 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2e0070440 0x7ff2e00b3090 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8005fd0 tx=0x7ff2d802a8b0 comp rx=0 tx=0).stop 2024-09-21T07:36:03.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.501+0000 7ff2e6a00640 1 -- 172.21.15.39:0/4097955207 shutdown_connections 2024-09-21T07:36:03.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.501+0000 7ff2e6a00640 1 --2- 172.21.15.39:0/4097955207 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2e0070440 0x7ff2e00b3090 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:03.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.501+0000 7ff2e6a00640 1 --2- 172.21.15.39:0/4097955207 >> v2:172.21.15.148:3300/0 conn(0x7ff2e0073170 0x7ff2e015c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:03.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.501+0000 7ff2e6a00640 1 -- 172.21.15.39:0/4097955207 >> 172.21.15.39:0/4097955207 conn(0x7ff2e00b1cb0 msgr2=0x7ff2e00b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:03.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.502+0000 7ff2e6a00640 1 -- 172.21.15.39:0/4097955207 shutdown_connections 2024-09-21T07:36:03.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.502+0000 7ff2e6a00640 1 -- 172.21.15.39:0/4097955207 wait complete. 2024-09-21T07:36:03.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.504+0000 7ff2e6a00640 1 Processor -- start 2024-09-21T07:36:03.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.504+0000 7ff2e6a00640 1 -- start start 2024-09-21T07:36:03.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.505+0000 7ff2e6a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2e0070030 0x7ff2e014f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:03.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.505+0000 7ff2e6a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2e0070440 0x7ff2e00d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:03.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.505+0000 7ff2e6a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2e0073170 0x7ff2e00d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:03.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.505+0000 7ff2e6a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e00b3b00 con 0x7ff2e0070440 2024-09-21T07:36:03.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.505+0000 7ff2e6a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e00b3980 con 0x7ff2e0073170 2024-09-21T07:36:03.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.505+0000 7ff2e6a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7ff2e00b3c80 con 0x7ff2e0070030 2024-09-21T07:36:03.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.506+0000 7ff2d7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2e0070440 0x7ff2e00d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:03.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.506+0000 7ff2d7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2e0070440 0x7ff2e00d01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48372/0 (socket says 172.21.15.39:48372) 2024-09-21T07:36:03.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.506+0000 7ff2d7e00640 1 -- 172.21.15.39:0/3084568057 learned_addr learned my addr 172.21.15.39:0/3084568057 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:03.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.506+0000 7ff2dfe00640 1 --2- 172.21.15.39:0/3084568057 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2e0070030 0x7ff2e014f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:03.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.506+0000 7ff2e4a00640 1 --2- 172.21.15.39:0/3084568057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2e0073170 0x7ff2e00d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:03.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.506+0000 7ff2d7e00640 1 -- 172.21.15.39:0/3084568057 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2e0070030 msgr2=0x7ff2e014f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:03.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.506+0000 7ff2d7e00640 1 --2- 172.21.15.39:0/3084568057 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2e0070030 0x7ff2e014f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:03.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.506+0000 7ff2d7e00640 1 -- 172.21.15.39:0/3084568057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2e0073170 msgr2=0x7ff2e00d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:03.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.506+0000 7ff2d7e00640 1 --2- 172.21.15.39:0/3084568057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2e0073170 0x7ff2e00d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:03.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.506+0000 7ff2d7e00640 1 -- 172.21.15.39:0/3084568057 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2e009d7d0 con 0x7ff2e0070440 2024-09-21T07:36:03.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.506+0000 7ff2e4a00640 1 --2- 172.21.15.39:0/3084568057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2e0073170 0x7ff2e00d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:36:03.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.506+0000 7ff2dfe00640 1 --2- 172.21.15.39:0/3084568057 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2e0070030 0x7ff2e014f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:36:03.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.506+0000 7ff2d7e00640 1 --2- 172.21.15.39:0/3084568057 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2e0070440 0x7ff2e00d01b0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff2c800c430 tx=0x7ff2c800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:03.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.507+0000 7ff2dcc00640 1 -- 172.21.15.39:0/3084568057 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8011790 con 0x7ff2e0070440 2024-09-21T07:36:03.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.507+0000 7ff2dcc00640 1 -- 172.21.15.39:0/3084568057 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2c8017070 con 0x7ff2e0070440 2024-09-21T07:36:03.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.507+0000 7ff2e6a00640 1 -- 172.21.15.39:0/3084568057 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2e014f9f0 con 0x7ff2e0070440 2024-09-21T07:36:03.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.507+0000 7ff2dcc00640 1 -- 172.21.15.39:0/3084568057 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8011cc0 con 0x7ff2e0070440 2024-09-21T07:36:03.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.507+0000 7ff2e6a00640 1 -- 172.21.15.39:0/3084568057 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2e009bcd0 con 0x7ff2e0070440 2024-09-21T07:36:03.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.508+0000 7ff2e6a00640 1 -- 172.21.15.39:0/3084568057 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2e00b4130 con 0x7ff2e0070440 2024-09-21T07:36:03.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.509+0000 7ff2dcc00640 1 -- 172.21.15.39:0/3084568057 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff2c801c460 con 0x7ff2e0070440 2024-09-21T07:36:03.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.510+0000 7ff2dcc00640 1 --2- 172.21.15.39:0/3084568057 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff2bc078b90 0x7ff2bc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:03.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.510+0000 7ff2dcc00640 1 -- 172.21.15.39:0/3084568057 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6772+0+0 (secure 0 0 0) 0x7ff2c809ad30 con 0x7ff2e0070440 2024-09-21T07:36:03.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.511+0000 7ff2dfe00640 1 --2- 172.21.15.39:0/3084568057 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff2bc078b90 0x7ff2bc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:03.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.512+0000 7ff2dfe00640 1 --2- 172.21.15.39:0/3084568057 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff2bc078b90 0x7ff2bc07b0a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0009ab0 tx=0x7ff2d0005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:03.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.515+0000 7ff2dcc00640 1 -- 172.21.15.39:0/3084568057 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff2c8021020 con 0x7ff2e0070440 2024-09-21T07:36:03.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.755+0000 7ff2e6a00640 1 -- 172.21.15.39:0/3084568057 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2e00c3860 con 0x7ff2e0070440 2024-09-21T07:36:03.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.756+0000 7ff2dcc00640 1 -- 172.21.15.39:0/3084568057 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5225 (secure 0 0 0) 0x7ff2e009d7d0 con 0x7ff2e0070440 2024-09-21T07:36:03.758 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:36:03.758 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":46,"btime":"2024-09-21T07:36:00:511098+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5426,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/707367131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":707367131},{"type":"v1","addr":"172.21.15.39:6835","nonce":707367131}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5429,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/784956164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":784956164},{"type":"v1","addr":"172.21.15.148:6836","nonce":784956164}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5432,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3742990359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3742990359},{"type":"v1","addr":"172.21.15.148:6837","nonce":3742990359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:35:48.965416+0000","modified":"2024-09-21T07:36:00.107330+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5842},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5842":{"gid":5842,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/1032917299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":1032917299},{"type":"v1","addr":"172.21.15.39:6834","nonce":1032917299}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5842,"qdb_cluster":[5842]},"id":5}]} 2024-09-21T07:36:03.758 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 46 2024-09-21T07:36:03.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.758+0000 7ff2e6a00640 1 -- 172.21.15.39:0/3084568057 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff2bc078b90 msgr2=0x7ff2bc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:03.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.758+0000 7ff2e6a00640 1 --2- 172.21.15.39:0/3084568057 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff2bc078b90 0x7ff2bc07b0a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0009ab0 tx=0x7ff2d0005f90 comp rx=0 tx=0).stop 2024-09-21T07:36:03.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.758+0000 7ff2e6a00640 1 -- 172.21.15.39:0/3084568057 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2e0070440 msgr2=0x7ff2e00d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:03.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.758+0000 7ff2e6a00640 1 --2- 172.21.15.39:0/3084568057 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2e0070440 0x7ff2e00d01b0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff2c800c430 tx=0x7ff2c800c8f0 comp rx=0 tx=0).stop 2024-09-21T07:36:03.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.758+0000 7ff2e6a00640 1 -- 172.21.15.39:0/3084568057 shutdown_connections 2024-09-21T07:36:03.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.758+0000 7ff2e6a00640 1 --2- 172.21.15.39:0/3084568057 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff2bc078b90 0x7ff2bc07b0a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:03.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.758+0000 7ff2e6a00640 1 --2- 172.21.15.39:0/3084568057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2e0073170 0x7ff2e00d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:03.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.758+0000 7ff2e6a00640 1 --2- 172.21.15.39:0/3084568057 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2e0070440 0x7ff2e00d01b0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:03.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.758+0000 7ff2e6a00640 1 --2- 172.21.15.39:0/3084568057 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2e0070030 0x7ff2e014f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:03.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.758+0000 7ff2e6a00640 1 -- 172.21.15.39:0/3084568057 >> 172.21.15.39:0/3084568057 conn(0x7ff2e00b1cb0 msgr2=0x7ff2e015f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:03.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.758+0000 7ff2e6a00640 1 -- 172.21.15.39:0/3084568057 shutdown_connections 2024-09-21T07:36:03.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:03.758+0000 7ff2e6a00640 1 -- 172.21.15.39:0/3084568057 wait complete. 2024-09-21T07:36:03.785 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:36:04.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.009+0000 7f8e7d600640 1 -- 172.21.15.39:0/1021097235 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e780af780 msgr2=0x7f8e780b39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:04.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.009+0000 7f8e7d600640 1 --2- 172.21.15.39:0/1021097235 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e780af780 0x7f8e780b39c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f8e68009830 tx=0x7f8e6802a510 comp rx=0 tx=0).stop 2024-09-21T07:36:04.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.010+0000 7f8e7d600640 1 -- 172.21.15.39:0/1021097235 shutdown_connections 2024-09-21T07:36:04.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.010+0000 7f8e7d600640 1 --2- 172.21.15.39:0/1021097235 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e78072ff0 0x7f8e78073400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:04.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.010+0000 7f8e7d600640 1 --2- 172.21.15.39:0/1021097235 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e780af780 0x7f8e780b39c0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:04.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.010+0000 7f8e7d600640 1 -- 172.21.15.39:0/1021097235 >> 172.21.15.39:0/1021097235 conn(0x7f8e7806d7d0 msgr2=0x7f8e780b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:04.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.010+0000 7f8e7d600640 1 -- 172.21.15.39:0/1021097235 shutdown_connections 2024-09-21T07:36:04.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.010+0000 7f8e7d600640 1 -- 172.21.15.39:0/1021097235 wait complete. 2024-09-21T07:36:04.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.012+0000 7f8e7d600640 1 Processor -- start 2024-09-21T07:36:04.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.012+0000 7f8e7d600640 1 -- start start 2024-09-21T07:36:04.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.013+0000 7f8e7d600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e78072ff0 0x7f8e7809cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:04.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.013+0000 7f8e7d600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e780af780 0x7f8e781f9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:04.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.013+0000 7f8e7d600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e780cd3e0 0x7f8e781506a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:04.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.013+0000 7f8e7d600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8e78073d10 con 0x7f8e780cd3e0 2024-09-21T07:36:04.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.013+0000 7f8e7d600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8e78073b90 con 0x7f8e780af780 2024-09-21T07:36:04.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.013+0000 7f8e7d600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f8e7815f930 con 0x7f8e78072ff0 2024-09-21T07:36:04.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.014+0000 7f8e77400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e780cd3e0 0x7f8e781506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:04.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.014+0000 7f8e77400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e780cd3e0 0x7f8e781506a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48402/0 (socket says 172.21.15.39:48402) 2024-09-21T07:36:04.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.014+0000 7f8e77400640 1 -- 172.21.15.39:0/1430563526 learned_addr learned my addr 172.21.15.39:0/1430563526 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:04.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.014+0000 7f8e76a00640 1 --2- 172.21.15.39:0/1430563526 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e78072ff0 0x7f8e7809cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:04.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.014+0000 7f8e6fe00640 1 --2- 172.21.15.39:0/1430563526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e780af780 0x7f8e781f9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:04.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.014+0000 7f8e77400640 1 -- 172.21.15.39:0/1430563526 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e78072ff0 msgr2=0x7f8e7809cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:04.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.014+0000 7f8e77400640 1 --2- 172.21.15.39:0/1430563526 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e78072ff0 0x7f8e7809cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:04.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.014+0000 7f8e77400640 1 -- 172.21.15.39:0/1430563526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e780af780 msgr2=0x7f8e781f9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:04.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.014+0000 7f8e77400640 1 --2- 172.21.15.39:0/1430563526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e780af780 0x7f8e781f9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:04.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.014+0000 7f8e77400640 1 -- 172.21.15.39:0/1430563526 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e7809d620 con 0x7f8e780cd3e0 2024-09-21T07:36:04.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.014+0000 7f8e6fe00640 1 --2- 172.21.15.39:0/1430563526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e780af780 0x7f8e781f9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:36:04.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.015+0000 7f8e77400640 1 --2- 172.21.15.39:0/1430563526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e780cd3e0 0x7f8e781506a0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f8e7000ea40 tx=0x7f8e7000ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:04.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.015+0000 7f8e6ea00640 1 -- 172.21.15.39:0/1430563526 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7000ce40 con 0x7f8e780cd3e0 2024-09-21T07:36:04.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.015+0000 7f8e6ea00640 1 -- 172.21.15.39:0/1430563526 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e70004c10 con 0x7f8e780cd3e0 2024-09-21T07:36:04.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.015+0000 7f8e6ea00640 1 -- 172.21.15.39:0/1430563526 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e70005020 con 0x7f8e780cd3e0 2024-09-21T07:36:04.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.015+0000 7f8e7d600640 1 -- 172.21.15.39:0/1430563526 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e7809d900 con 0x7f8e780cd3e0 2024-09-21T07:36:04.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.015+0000 7f8e7d600640 1 -- 172.21.15.39:0/1430563526 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e781f9410 con 0x7f8e780cd3e0 2024-09-21T07:36:04.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.016+0000 7f8e7d600640 1 -- 172.21.15.39:0/1430563526 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e781f8f50 con 0x7f8e780cd3e0 2024-09-21T07:36:04.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.018+0000 7f8e6ea00640 1 -- 172.21.15.39:0/1430563526 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8e700051c0 con 0x7f8e780cd3e0 2024-09-21T07:36:04.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.019+0000 7f8e6ea00640 1 --2- 172.21.15.39:0/1430563526 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8e54078b90 0x7f8e5407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:04.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.019+0000 7f8e6ea00640 1 -- 172.21.15.39:0/1430563526 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6772+0+0 (secure 0 0 0) 0x7f8e7009c070 con 0x7f8e780cd3e0 2024-09-21T07:36:04.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.019+0000 7f8e76a00640 1 --2- 172.21.15.39:0/1430563526 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8e54078b90 0x7f8e5407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:04.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.020+0000 7f8e76a00640 1 --2- 172.21.15.39:0/1430563526 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8e54078b90 0x7f8e5407b0a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8e68034000 tx=0x7f8e68004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:04.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.023+0000 7f8e6ea00640 1 -- 172.21.15.39:0/1430563526 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8e781f8f50 con 0x7f8e780cd3e0 2024-09-21T07:36:04.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.262+0000 7f8e7d600640 1 -- 172.21.15.39:0/1430563526 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e781e6370 con 0x7f8e780cd3e0 2024-09-21T07:36:04.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.263+0000 7f8e6ea00640 1 -- 172.21.15.39:0/1430563526 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5225 (secure 0 0 0) 0x7f8e70063fe0 con 0x7f8e780cd3e0 2024-09-21T07:36:04.265 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:36:04.265 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":46,"btime":"2024-09-21T07:36:00:511098+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5426,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/707367131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":707367131},{"type":"v1","addr":"172.21.15.39:6835","nonce":707367131}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5429,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/784956164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":784956164},{"type":"v1","addr":"172.21.15.148:6836","nonce":784956164}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5432,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3742990359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3742990359},{"type":"v1","addr":"172.21.15.148:6837","nonce":3742990359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:35:48.965416+0000","modified":"2024-09-21T07:36:00.107330+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5842},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5842":{"gid":5842,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/1032917299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":1032917299},{"type":"v1","addr":"172.21.15.39:6834","nonce":1032917299}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5842,"qdb_cluster":[5842]},"id":5}]} 2024-09-21T07:36:04.265 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 46 2024-09-21T07:36:04.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.266+0000 7f8e7d600640 1 -- 172.21.15.39:0/1430563526 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8e54078b90 msgr2=0x7f8e5407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:04.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.266+0000 7f8e7d600640 1 --2- 172.21.15.39:0/1430563526 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8e54078b90 0x7f8e5407b0a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8e68034000 tx=0x7f8e68004730 comp rx=0 tx=0).stop 2024-09-21T07:36:04.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.266+0000 7f8e7d600640 1 -- 172.21.15.39:0/1430563526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e780cd3e0 msgr2=0x7f8e781506a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:04.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.266+0000 7f8e7d600640 1 --2- 172.21.15.39:0/1430563526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e780cd3e0 0x7f8e781506a0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f8e7000ea40 tx=0x7f8e7000ef00 comp rx=0 tx=0).stop 2024-09-21T07:36:04.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.266+0000 7f8e7d600640 1 -- 172.21.15.39:0/1430563526 shutdown_connections 2024-09-21T07:36:04.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.266+0000 7f8e7d600640 1 --2- 172.21.15.39:0/1430563526 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8e54078b90 0x7f8e5407b0a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:04.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.266+0000 7f8e7d600640 1 --2- 172.21.15.39:0/1430563526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e780cd3e0 0x7f8e781506a0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:04.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.266+0000 7f8e7d600640 1 --2- 172.21.15.39:0/1430563526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e780af780 0x7f8e781f9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:04.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.266+0000 7f8e7d600640 1 --2- 172.21.15.39:0/1430563526 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e78072ff0 0x7f8e7809cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:04.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.266+0000 7f8e7d600640 1 -- 172.21.15.39:0/1430563526 >> 172.21.15.39:0/1430563526 conn(0x7f8e7806d7d0 msgr2=0x7f8e780adf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:04.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.266+0000 7f8e7d600640 1 -- 172.21.15.39:0/1430563526 shutdown_connections 2024-09-21T07:36:04.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.266+0000 7f8e7d600640 1 -- 172.21.15.39:0/1430563526 wait complete. 2024-09-21T07:36:04.290 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:36:04.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.515+0000 7efd90a00640 1 -- 172.21.15.39:0/1274099884 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd8c162470 msgr2=0x7efd8c162880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:04.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.515+0000 7efd90a00640 1 --2- 172.21.15.39:0/1274099884 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd8c162470 0x7efd8c162880 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7efd84005fd0 tx=0x7efd8402aa90 comp rx=0 tx=0).stop 2024-09-21T07:36:04.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.515+0000 7efd90a00640 1 -- 172.21.15.39:0/1274099884 shutdown_connections 2024-09-21T07:36:04.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.515+0000 7efd90a00640 1 --2- 172.21.15.39:0/1274099884 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd8c162470 0x7efd8c162880 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:04.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.515+0000 7efd90a00640 1 --2- 172.21.15.39:0/1274099884 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efd8c163110 0x7efd8c0b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:04.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.515+0000 7efd90a00640 1 -- 172.21.15.39:0/1274099884 >> 172.21.15.39:0/1274099884 conn(0x7efd8c09d6a0 msgr2=0x7efd8c0d43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:04.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.515+0000 7efd90a00640 1 -- 172.21.15.39:0/1274099884 shutdown_connections 2024-09-21T07:36:04.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.515+0000 7efd90a00640 1 -- 172.21.15.39:0/1274099884 wait complete. 2024-09-21T07:36:04.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.517+0000 7efd90a00640 1 Processor -- start 2024-09-21T07:36:04.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.517+0000 7efd90a00640 1 -- start start 2024-09-21T07:36:04.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.519+0000 7efd90a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efd8c0b5a20 0x7efd8c15fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:04.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.519+0000 7efd90a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd8c162470 0x7efd8c1ea570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:04.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.519+0000 7efd90a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd8c163110 0x7efd8c1f54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:04.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.519+0000 7efd90a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c0cfd00 con 0x7efd8c163110 2024-09-21T07:36:04.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.519+0000 7efd90a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c0cfb80 con 0x7efd8c162470 2024-09-21T07:36:04.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.519+0000 7efd90a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7efd8c0cfe80 con 0x7efd8c0b5a20 2024-09-21T07:36:04.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.519+0000 7efd8be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd8c163110 0x7efd8c1f54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:04.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.519+0000 7efd8b400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efd8c0b5a20 0x7efd8c15fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:04.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.519+0000 7efd8be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd8c163110 0x7efd8c1f54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48414/0 (socket says 172.21.15.39:48414) 2024-09-21T07:36:04.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.519+0000 7efd8be00640 1 -- 172.21.15.39:0/3918819497 learned_addr learned my addr 172.21.15.39:0/3918819497 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:04.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.519+0000 7efd83e00640 1 --2- 172.21.15.39:0/3918819497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd8c162470 0x7efd8c1ea570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:04.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.520+0000 7efd8be00640 1 -- 172.21.15.39:0/3918819497 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efd8c0b5a20 msgr2=0x7efd8c15fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:04.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.520+0000 7efd8be00640 1 --2- 172.21.15.39:0/3918819497 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efd8c0b5a20 0x7efd8c15fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:04.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.520+0000 7efd8be00640 1 -- 172.21.15.39:0/3918819497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd8c162470 msgr2=0x7efd8c1ea570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:04.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.520+0000 7efd8be00640 1 --2- 172.21.15.39:0/3918819497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd8c162470 0x7efd8c1ea570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:04.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.520+0000 7efd8be00640 1 -- 172.21.15.39:0/3918819497 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd8c1600f0 con 0x7efd8c163110 2024-09-21T07:36:04.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.520+0000 7efd8be00640 1 --2- 172.21.15.39:0/3918819497 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd8c163110 0x7efd8c1f54d0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7efd84009d90 tx=0x7efd84003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:04.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.520+0000 7efd83400640 1 -- 172.21.15.39:0/3918819497 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd84044070 con 0x7efd8c163110 2024-09-21T07:36:04.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.520+0000 7efd83400640 1 -- 172.21.15.39:0/3918819497 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd84004250 con 0x7efd8c163110 2024-09-21T07:36:04.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.520+0000 7efd83400640 1 -- 172.21.15.39:0/3918819497 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd84004660 con 0x7efd8c163110 2024-09-21T07:36:04.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.520+0000 7efd90a00640 1 -- 172.21.15.39:0/3918819497 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd8c160420 con 0x7efd8c163110 2024-09-21T07:36:04.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.521+0000 7efd90a00640 1 -- 172.21.15.39:0/3918819497 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd8c09f0e0 con 0x7efd8c163110 2024-09-21T07:36:04.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.522+0000 7efd90a00640 1 -- 172.21.15.39:0/3918819497 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd8c0d0330 con 0x7efd8c163110 2024-09-21T07:36:04.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.523+0000 7efd83400640 1 -- 172.21.15.39:0/3918819497 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7efd84047070 con 0x7efd8c163110 2024-09-21T07:36:04.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.524+0000 7efd83400640 1 --2- 172.21.15.39:0/3918819497 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efd68078b90 0x7efd6807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:04.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.525+0000 7efd83400640 1 -- 172.21.15.39:0/3918819497 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6772+0+0 (secure 0 0 0) 0x7efd840bbf60 con 0x7efd8c163110 2024-09-21T07:36:04.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.525+0000 7efd8b400640 1 --2- 172.21.15.39:0/3918819497 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efd68078b90 0x7efd6807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:04.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.526+0000 7efd8b400640 1 --2- 172.21.15.39:0/3918819497 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efd68078b90 0x7efd6807b0a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7efd7c00bb30 tx=0x7efd7c00b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:04.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.529+0000 7efd83400640 1 -- 172.21.15.39:0/3918819497 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7efd8c1600f0 con 0x7efd8c163110 2024-09-21T07:36:04.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.735+0000 7efd90a00640 1 -- 172.21.15.39:0/3918819497 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efd8c069ea0 con 0x7efd8c163110 2024-09-21T07:36:04.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.737+0000 7efd83400640 1 -- 172.21.15.39:0/3918819497 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22197 (secure 0 0 0) 0x7efd84032030 con 0x7efd8c163110 2024-09-21T07:36:04.739 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:36:04.740 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":79,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:36:04.533996+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:35:46.914721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-21T07:35:47.926559+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":79,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":14,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:36:04.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.739+0000 7efd90a00640 1 -- 172.21.15.39:0/3918819497 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efd68078b90 msgr2=0x7efd6807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:04.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.739+0000 7efd90a00640 1 --2- 172.21.15.39:0/3918819497 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efd68078b90 0x7efd6807b0a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7efd7c00bb30 tx=0x7efd7c00b2e0 comp rx=0 tx=0).stop 2024-09-21T07:36:04.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.739+0000 7efd90a00640 1 -- 172.21.15.39:0/3918819497 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd8c163110 msgr2=0x7efd8c1f54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:04.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.739+0000 7efd90a00640 1 --2- 172.21.15.39:0/3918819497 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd8c163110 0x7efd8c1f54d0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7efd84009d90 tx=0x7efd84003ce0 comp rx=0 tx=0).stop 2024-09-21T07:36:04.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.739+0000 7efd90a00640 1 -- 172.21.15.39:0/3918819497 shutdown_connections 2024-09-21T07:36:04.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.739+0000 7efd90a00640 1 --2- 172.21.15.39:0/3918819497 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efd68078b90 0x7efd6807b0a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:04.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.739+0000 7efd90a00640 1 --2- 172.21.15.39:0/3918819497 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd8c163110 0x7efd8c1f54d0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:04.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.739+0000 7efd90a00640 1 --2- 172.21.15.39:0/3918819497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd8c162470 0x7efd8c1ea570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:04.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.739+0000 7efd90a00640 1 --2- 172.21.15.39:0/3918819497 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efd8c0b5a20 0x7efd8c15fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:04.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.739+0000 7efd90a00640 1 -- 172.21.15.39:0/3918819497 >> 172.21.15.39:0/3918819497 conn(0x7efd8c09d6a0 msgr2=0x7efd8c0b41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:04.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.739+0000 7efd90a00640 1 -- 172.21.15.39:0/3918819497 shutdown_connections 2024-09-21T07:36:04.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.740+0000 7efd90a00640 1 -- 172.21.15.39:0/3918819497 wait complete. 2024-09-21T07:36:04.764 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T07:36:05.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.997+0000 7f1d2fe00640 1 -- 172.21.15.39:0/4074119271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d28070440 msgr2=0x7f1d28074b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:05.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.997+0000 7f1d2fe00640 1 --2- 172.21.15.39:0/4074119271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d28070440 0x7f1d28074b80 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f1d280710f0 tx=0x7f1d2002aac0 comp rx=0 tx=0).stop 2024-09-21T07:36:05.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.997+0000 7f1d2fe00640 1 -- 172.21.15.39:0/4074119271 shutdown_connections 2024-09-21T07:36:05.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.997+0000 7f1d2fe00640 1 --2- 172.21.15.39:0/4074119271 >> v2:172.21.15.148:3300/0 conn(0x7f1d28070030 0x7f1d28074640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:05.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.997+0000 7f1d2fe00640 1 --2- 172.21.15.39:0/4074119271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d28070440 0x7f1d28074b80 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:05.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.997+0000 7f1d2fe00640 1 -- 172.21.15.39:0/4074119271 >> 172.21.15.39:0/4074119271 conn(0x7f1d280b1cb0 msgr2=0x7f1d280b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:05.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.997+0000 7f1d2fe00640 1 -- 172.21.15.39:0/4074119271 shutdown_connections 2024-09-21T07:36:05.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:04.997+0000 7f1d2fe00640 1 -- 172.21.15.39:0/4074119271 wait complete. 2024-09-21T07:36:05.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.000+0000 7f1d2fe00640 1 Processor -- start 2024-09-21T07:36:05.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.000+0000 7f1d2fe00640 1 -- start start 2024-09-21T07:36:05.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.001+0000 7f1d2fe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d28070030 0x7f1d281f4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:05.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.001+0000 7f1d2fe00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d28070440 0x7f1d28159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:05.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.001+0000 7f1d2fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d28073170 0x7f1d281f6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:05.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.001+0000 7f1d2fe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1d280b3a40 con 0x7f1d28073170 2024-09-21T07:36:05.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.001+0000 7f1d2fe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1d280b38c0 con 0x7f1d28070030 2024-09-21T07:36:05.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.001+0000 7f1d2fe00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f1d280b3bc0 con 0x7f1d28070440 2024-09-21T07:36:05.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.002+0000 7f1d2de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d28073170 0x7f1d281f6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:05.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.002+0000 7f1d2de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d28073170 0x7f1d281f6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48440/0 (socket says 172.21.15.39:48440) 2024-09-21T07:36:05.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.002+0000 7f1d2de00640 1 -- 172.21.15.39:0/1817789901 learned_addr learned my addr 172.21.15.39:0/1817789901 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:05.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.002+0000 7f1d2ca00640 1 --2- 172.21.15.39:0/1817789901 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d28070440 0x7f1d28159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:05.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.002+0000 7f1d2d400640 1 --2- 172.21.15.39:0/1817789901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d28070030 0x7f1d281f4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:05.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.002+0000 7f1d2de00640 1 -- 172.21.15.39:0/1817789901 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d28070440 msgr2=0x7f1d28159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:05.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.002+0000 7f1d2de00640 1 --2- 172.21.15.39:0/1817789901 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d28070440 0x7f1d28159060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:05.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.002+0000 7f1d2de00640 1 -- 172.21.15.39:0/1817789901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d28070030 msgr2=0x7f1d281f4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:05.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.002+0000 7f1d2de00640 1 --2- 172.21.15.39:0/1817789901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d28070030 0x7f1d281f4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:05.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.002+0000 7f1d2de00640 1 -- 172.21.15.39:0/1817789901 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d280c3b90 con 0x7f1d28073170 2024-09-21T07:36:05.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.002+0000 7f1d2d400640 1 --2- 172.21.15.39:0/1817789901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d28070030 0x7f1d281f4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:36:05.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.003+0000 7f1d2de00640 1 --2- 172.21.15.39:0/1817789901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d28073170 0x7f1d281f6ce0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f1d20009d60 tx=0x7f1d20003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:05.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.003+0000 7f1d25600640 1 -- 172.21.15.39:0/1817789901 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d20044070 con 0x7f1d28073170 2024-09-21T07:36:05.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.003+0000 7f1d25600640 1 -- 172.21.15.39:0/1817789901 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d20004250 con 0x7f1d28073170 2024-09-21T07:36:05.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.003+0000 7f1d2fe00640 1 -- 172.21.15.39:0/1817789901 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d280c3e70 con 0x7f1d28073170 2024-09-21T07:36:05.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.003+0000 7f1d25600640 1 -- 172.21.15.39:0/1817789901 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d20004680 con 0x7f1d28073170 2024-09-21T07:36:05.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.003+0000 7f1d2fe00640 1 -- 172.21.15.39:0/1817789901 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d281595a0 con 0x7f1d28073170 2024-09-21T07:36:05.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.004+0000 7f1d2fe00640 1 -- 172.21.15.39:0/1817789901 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d280cf070 con 0x7f1d28073170 2024-09-21T07:36:05.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.005+0000 7f1d25600640 1 -- 172.21.15.39:0/1817789901 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1d20036400 con 0x7f1d28073170 2024-09-21T07:36:05.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.006+0000 7f1d25600640 1 --2- 172.21.15.39:0/1817789901 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1d08078b90 0x7f1d0807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:05.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.006+0000 7f1d25600640 1 -- 172.21.15.39:0/1817789901 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6828+0+0 (secure 0 0 0) 0x7f1d200bb920 con 0x7f1d28073170 2024-09-21T07:36:05.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.007+0000 7f1d2d400640 1 --2- 172.21.15.39:0/1817789901 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1d08078b90 0x7f1d0807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:05.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.008+0000 7f1d2d400640 1 --2- 172.21.15.39:0/1817789901 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1d08078b90 0x7f1d0807b0a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c007fd0 tx=0x7f1d1c009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:05.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.010+0000 7f1d25600640 1 -- 172.21.15.39:0/1817789901 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1d20032030 con 0x7f1d28073170 2024-09-21T07:36:05.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.248+0000 7f1d2fe00640 1 -- 172.21.15.39:0/1817789901 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1d280b48a0 con 0x7f1d28073170 2024-09-21T07:36:05.251 INFO:tasks.ceph.mon.a.smithi039.stderr:2024-09-21T07:36:05.249+0000 7fabab600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T07:36:05.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.251+0000 7f1d25600640 1 -- 172.21.15.39:0/1817789901 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v47) ==== 161+0+0 (secure 0 0 0) 0x7f1d20004820 con 0x7f1d28073170 2024-09-21T07:36:05.253 INFO:teuthology.orchestra.run.smithi039.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T07:36:05.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.253+0000 7f1d2fe00640 1 -- 172.21.15.39:0/1817789901 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1d08078b90 msgr2=0x7f1d0807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:05.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.253+0000 7f1d2fe00640 1 --2- 172.21.15.39:0/1817789901 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1d08078b90 0x7f1d0807b0a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c007fd0 tx=0x7f1d1c009040 comp rx=0 tx=0).stop 2024-09-21T07:36:05.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.253+0000 7f1d2fe00640 1 -- 172.21.15.39:0/1817789901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d28073170 msgr2=0x7f1d281f6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:05.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.253+0000 7f1d2fe00640 1 --2- 172.21.15.39:0/1817789901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d28073170 0x7f1d281f6ce0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f1d20009d60 tx=0x7f1d20003f10 comp rx=0 tx=0).stop 2024-09-21T07:36:05.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.254+0000 7f1d2fe00640 1 -- 172.21.15.39:0/1817789901 shutdown_connections 2024-09-21T07:36:05.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.254+0000 7f1d2fe00640 1 --2- 172.21.15.39:0/1817789901 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1d08078b90 0x7f1d0807b0a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:05.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.254+0000 7f1d2fe00640 1 --2- 172.21.15.39:0/1817789901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d28073170 0x7f1d281f6ce0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:05.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.254+0000 7f1d2fe00640 1 --2- 172.21.15.39:0/1817789901 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d28070440 0x7f1d28159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:05.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.254+0000 7f1d2fe00640 1 --2- 172.21.15.39:0/1817789901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d28070030 0x7f1d281f4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:05.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.254+0000 7f1d2fe00640 1 -- 172.21.15.39:0/1817789901 >> 172.21.15.39:0/1817789901 conn(0x7f1d280b1cb0 msgr2=0x7f1d2815ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:05.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.254+0000 7f1d2fe00640 1 -- 172.21.15.39:0/1817789901 shutdown_connections 2024-09-21T07:36:05.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.255+0000 7f1d2fe00640 1 -- 172.21.15.39:0/1817789901 wait complete. 2024-09-21T07:36:05.265 INFO:tasks.ceph.mds.c.smithi039.stderr: -3933> 2024-09-21T07:35:51.418+0000 7fcd10a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:36:05.285 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T07:36:05.388 INFO:tasks.ceph.mds.c.smithi039.stdout:starting mds.c at 2024-09-21T07:36:05.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.523+0000 7fa8c2800640 1 -- 172.21.15.39:0/3199426218 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8a0003a00 msgr2=0x7fa8a0003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:05.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.523+0000 7fa8c2800640 1 --2- 172.21.15.39:0/3199426218 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8a0003a00 0x7fa8a0003e30 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac009b50 tx=0x7fa8a0005830 comp rx=0 tx=0).stop 2024-09-21T07:36:05.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.523+0000 7fa8c2800640 1 -- 172.21.15.39:0/3199426218 shutdown_connections 2024-09-21T07:36:05.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.523+0000 7fa8c2800640 1 --2- 172.21.15.39:0/3199426218 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8a0003a00 0x7fa8a0003e30 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:05.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.523+0000 7fa8c2800640 1 -- 172.21.15.39:0/3199426218 >> 172.21.15.39:0/3199426218 conn(0x7fa8bc06d7d0 msgr2=0x7fa8bc0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:05.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.524+0000 7fa8c2800640 1 -- 172.21.15.39:0/3199426218 shutdown_connections 2024-09-21T07:36:05.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.524+0000 7fa8c2800640 1 -- 172.21.15.39:0/3199426218 wait complete. 2024-09-21T07:36:05.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.526+0000 7fa8c2800640 1 Processor -- start 2024-09-21T07:36:05.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.526+0000 7fa8c2800640 1 -- start start 2024-09-21T07:36:05.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.527+0000 7fa8c2800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8a0003a00 0x7fa8bc1f9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:05.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.527+0000 7fa8c2800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8bc1f6890 0x7fa8bc155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:05.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.527+0000 7fa8c2800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8bc073250 0x7fa8bc0b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:05.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.527+0000 7fa8c2800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa8bc0af750 con 0x7fa8a0003a00 2024-09-21T07:36:05.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.527+0000 7fa8c2800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa8bc0af5d0 con 0x7fa8bc1f6890 2024-09-21T07:36:05.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.527+0000 7fa8c2800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa8bc0af8d0 con 0x7fa8bc073250 2024-09-21T07:36:05.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.528+0000 7fa8bb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8a0003a00 0x7fa8bc1f9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:05.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.528+0000 7fa8bb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8a0003a00 0x7fa8bc1f9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48476/0 (socket says 172.21.15.39:48476) 2024-09-21T07:36:05.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.528+0000 7fa8bb400640 1 -- 172.21.15.39:0/2199690384 learned_addr learned my addr 172.21.15.39:0/2199690384 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:05.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.528+0000 7fa8b3e00640 1 --2- 172.21.15.39:0/2199690384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8bc1f6890 0x7fa8bc155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:05.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.528+0000 7fa8bbe00640 1 --2- 172.21.15.39:0/2199690384 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8bc073250 0x7fa8bc0b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:05.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.528+0000 7fa8bb400640 1 -- 172.21.15.39:0/2199690384 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8bc073250 msgr2=0x7fa8bc0b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:05.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.528+0000 7fa8bb400640 1 --2- 172.21.15.39:0/2199690384 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8bc073250 0x7fa8bc0b0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:05.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.528+0000 7fa8bb400640 1 -- 172.21.15.39:0/2199690384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8bc1f6890 msgr2=0x7fa8bc155eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:05.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.528+0000 7fa8bb400640 1 --2- 172.21.15.39:0/2199690384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8bc1f6890 0x7fa8bc155eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:05.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.528+0000 7fa8bb400640 1 -- 172.21.15.39:0/2199690384 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8bc1f99d0 con 0x7fa8a0003a00 2024-09-21T07:36:05.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.528+0000 7fa8bb400640 1 --2- 172.21.15.39:0/2199690384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8a0003a00 0x7fa8bc1f9490 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac009a70 tx=0x7fa8ac0336f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:05.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.529+0000 7fa8b3400640 1 -- 172.21.15.39:0/2199690384 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac038040 con 0x7fa8a0003a00 2024-09-21T07:36:05.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.529+0000 7fa8b3400640 1 -- 172.21.15.39:0/2199690384 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8ac0338e0 con 0x7fa8a0003a00 2024-09-21T07:36:05.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.529+0000 7fa8b3400640 1 -- 172.21.15.39:0/2199690384 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac033d30 con 0x7fa8a0003a00 2024-09-21T07:36:05.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.529+0000 7fa8c2800640 1 -- 172.21.15.39:0/2199690384 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8bc1f9ee0 con 0x7fa8a0003a00 2024-09-21T07:36:05.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.529+0000 7fa8c2800640 1 -- 172.21.15.39:0/2199690384 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8bc1f6090 con 0x7fa8a0003a00 2024-09-21T07:36:05.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.530+0000 7fa8c2800640 1 -- 172.21.15.39:0/2199690384 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8bc0b4350 con 0x7fa8a0003a00 2024-09-21T07:36:05.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.531+0000 7fa8b3400640 1 -- 172.21.15.39:0/2199690384 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa8ac043020 con 0x7fa8a0003a00 2024-09-21T07:36:05.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.533+0000 7fa8b3400640 1 --2- 172.21.15.39:0/2199690384 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa898078bc0 0x7fa89807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:05.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.533+0000 7fa8b3400640 1 -- 172.21.15.39:0/2199690384 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6886+0+0 (secure 0 0 0) 0x7fa8ac0bb110 con 0x7fa8a0003a00 2024-09-21T07:36:05.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.533+0000 7fa8b3e00640 1 --2- 172.21.15.39:0/2199690384 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa898078bc0 0x7fa89807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:05.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.534+0000 7fa8b3e00640 1 --2- 172.21.15.39:0/2199690384 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa898078bc0 0x7fa89807b0d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc070b50 tx=0x7fa8a4006c40 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:05.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.538+0000 7fa8b3400640 1 -- 172.21.15.39:0/2199690384 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa8ac04a020 con 0x7fa8a0003a00 2024-09-21T07:36:05.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:05.774+0000 7fa8c2800640 1 -- 172.21.15.39:0/2199690384 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa8bc0d0760 con 0x7fa8a0003a00 2024-09-21T07:36:06.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.252+0000 7fa8b3400640 1 -- 172.21.15.39:0/2199690384 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v48) ==== 104+0+0 (secure 0 0 0) 0x7fa8ac082ae0 con 0x7fa8a0003a00 2024-09-21T07:36:06.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.255+0000 7fa8c2800640 1 -- 172.21.15.39:0/2199690384 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa898078bc0 msgr2=0x7fa89807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:06.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.255+0000 7fa8c2800640 1 --2- 172.21.15.39:0/2199690384 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa898078bc0 0x7fa89807b0d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc070b50 tx=0x7fa8a4006c40 comp rx=0 tx=0).stop 2024-09-21T07:36:06.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.255+0000 7fa8c2800640 1 -- 172.21.15.39:0/2199690384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8a0003a00 msgr2=0x7fa8bc1f9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:06.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.255+0000 7fa8c2800640 1 --2- 172.21.15.39:0/2199690384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8a0003a00 0x7fa8bc1f9490 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac009a70 tx=0x7fa8ac0336f0 comp rx=0 tx=0).stop 2024-09-21T07:36:06.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.256+0000 7fa8c2800640 1 -- 172.21.15.39:0/2199690384 shutdown_connections 2024-09-21T07:36:06.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.256+0000 7fa8c2800640 1 --2- 172.21.15.39:0/2199690384 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa898078bc0 0x7fa89807b0d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:06.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.256+0000 7fa8c2800640 1 --2- 172.21.15.39:0/2199690384 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8bc073250 0x7fa8bc0b0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:06.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.256+0000 7fa8c2800640 1 --2- 172.21.15.39:0/2199690384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8bc1f6890 0x7fa8bc155eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:06.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.256+0000 7fa8c2800640 1 --2- 172.21.15.39:0/2199690384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8a0003a00 0x7fa8bc1f9490 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:06.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.256+0000 7fa8c2800640 1 -- 172.21.15.39:0/2199690384 >> 172.21.15.39:0/2199690384 conn(0x7fa8bc06d7d0 msgr2=0x7fa8bc0cb970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:06.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.256+0000 7fa8c2800640 1 -- 172.21.15.39:0/2199690384 shutdown_connections 2024-09-21T07:36:06.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.256+0000 7fa8c2800640 1 -- 172.21.15.39:0/2199690384 wait complete. 2024-09-21T07:36:06.287 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T07:36:06.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.519+0000 7fbaeca00640 1 -- 172.21.15.39:0/3191447219 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbae80cb4d0 msgr2=0x7fbae8073820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:06.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.519+0000 7fbaeca00640 1 --2- 172.21.15.39:0/3191447219 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbae80cb4d0 0x7fbae8073820 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fbad4009830 tx=0x7fbad402a480 comp rx=0 tx=0).stop 2024-09-21T07:36:06.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.520+0000 7fbaeca00640 1 -- 172.21.15.39:0/3191447219 shutdown_connections 2024-09-21T07:36:06.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.520+0000 7fbaeca00640 1 --2- 172.21.15.39:0/3191447219 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbae815fb80 0x7fbae80acf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:06.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.520+0000 7fbaeca00640 1 --2- 172.21.15.39:0/3191447219 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbae80cb4d0 0x7fbae8073820 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:06.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.520+0000 7fbaeca00640 1 -- 172.21.15.39:0/3191447219 >> 172.21.15.39:0/3191447219 conn(0x7fbae80d0130 msgr2=0x7fbae80d0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:06.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.520+0000 7fbaeca00640 1 -- 172.21.15.39:0/3191447219 shutdown_connections 2024-09-21T07:36:06.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.520+0000 7fbaeca00640 1 -- 172.21.15.39:0/3191447219 wait complete. 2024-09-21T07:36:06.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.522+0000 7fbaeca00640 1 Processor -- start 2024-09-21T07:36:06.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.522+0000 7fbaeca00640 1 -- start start 2024-09-21T07:36:06.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.523+0000 7fbaeca00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbae80cc560 0x7fbae81ef460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:06.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.523+0000 7fbaeca00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbae815fb80 0x7fbae80b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:06.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.523+0000 7fbaeca00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbae809d580 0x7fbae80b5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:06.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.523+0000 7fbaeca00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbae80ad9b0 con 0x7fbae815fb80 2024-09-21T07:36:06.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.523+0000 7fbaeca00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbae80ad830 con 0x7fbae809d580 2024-09-21T07:36:06.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.523+0000 7fbaeca00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fbae80adb30 con 0x7fbae80cc560 2024-09-21T07:36:06.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.524+0000 7fbae5400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbae815fb80 0x7fbae80b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:06.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.524+0000 7fbae5400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbae815fb80 0x7fbae80b51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48496/0 (socket says 172.21.15.39:48496) 2024-09-21T07:36:06.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.524+0000 7fbae5400640 1 -- 172.21.15.39:0/3768058343 learned_addr learned my addr 172.21.15.39:0/3768058343 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:06.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.524+0000 7fbae5e00640 1 --2- 172.21.15.39:0/3768058343 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbae80cc560 0x7fbae81ef460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:06.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.524+0000 7fbae6800640 1 --2- 172.21.15.39:0/3768058343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbae809d580 0x7fbae80b5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:06.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.524+0000 7fbae5400640 1 -- 172.21.15.39:0/3768058343 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbae80cc560 msgr2=0x7fbae81ef460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:06.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.524+0000 7fbae5400640 1 --2- 172.21.15.39:0/3768058343 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbae80cc560 0x7fbae81ef460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:06.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.524+0000 7fbae5400640 1 -- 172.21.15.39:0/3768058343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbae809d580 msgr2=0x7fbae80b5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:06.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.524+0000 7fbae5400640 1 --2- 172.21.15.39:0/3768058343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbae809d580 0x7fbae80b5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:06.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.524+0000 7fbae5400640 1 -- 172.21.15.39:0/3768058343 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae80c9b20 con 0x7fbae815fb80 2024-09-21T07:36:06.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.524+0000 7fbae5e00640 1 --2- 172.21.15.39:0/3768058343 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbae80cc560 0x7fbae81ef460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:36:06.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.524+0000 7fbae6800640 1 --2- 172.21.15.39:0/3768058343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbae809d580 0x7fbae80b5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:36:06.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.525+0000 7fbae5400640 1 --2- 172.21.15.39:0/3768058343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbae815fb80 0x7fbae80b51e0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fbad402aa70 tx=0x7fbad4033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:06.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.525+0000 7fbade000640 1 -- 172.21.15.39:0/3768058343 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad4031d60 con 0x7fbae815fb80 2024-09-21T07:36:06.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.525+0000 7fbade000640 1 -- 172.21.15.39:0/3768058343 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbad4033e10 con 0x7fbae815fb80 2024-09-21T07:36:06.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.525+0000 7fbaeca00640 1 -- 172.21.15.39:0/3768058343 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbae81ef9a0 con 0x7fbae815fb80 2024-09-21T07:36:06.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.525+0000 7fbade000640 1 -- 172.21.15.39:0/3768058343 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad4005260 con 0x7fbae815fb80 2024-09-21T07:36:06.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.525+0000 7fbaeca00640 1 -- 172.21.15.39:0/3768058343 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbae80c9dc0 con 0x7fbae815fb80 2024-09-21T07:36:06.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.526+0000 7fbaeca00640 1 -- 172.21.15.39:0/3768058343 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbae80ae050 con 0x7fbae815fb80 2024-09-21T07:36:06.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.527+0000 7fbade000640 1 -- 172.21.15.39:0/3768058343 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbad4005400 con 0x7fbae815fb80 2024-09-21T07:36:06.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.528+0000 7fbade000640 1 --2- 172.21.15.39:0/3768058343 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbac8078b90 0x7fbac807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:06.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.528+0000 7fbade000640 1 -- 172.21.15.39:0/3768058343 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6886+0+0 (secure 0 0 0) 0x7fbad40bc550 con 0x7fbae815fb80 2024-09-21T07:36:06.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.529+0000 7fbae5e00640 1 --2- 172.21.15.39:0/3768058343 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbac8078b90 0x7fbac807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:06.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.529+0000 7fbae5e00640 1 --2- 172.21.15.39:0/3768058343 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbac8078b90 0x7fbac807b0a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbad8000f60 tx=0x7fbad8009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:06.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.533+0000 7fbade000640 1 -- 172.21.15.39:0/3768058343 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbad403f030 con 0x7fbae815fb80 2024-09-21T07:36:06.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:06.731+0000 7fbaeca00640 1 -- 172.21.15.39:0/3768058343 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbae80af8f0 con 0x7fbae815fb80 2024-09-21T07:36:07.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.579+0000 7fbade000640 1 -- 172.21.15.39:0/3768058343 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v81) ==== 181+0+0 (secure 0 0 0) 0x7fbad4005d00 con 0x7fbae815fb80 2024-09-21T07:36:07.581 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' removed 2024-09-21T07:36:07.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.581+0000 7fbaeca00640 1 -- 172.21.15.39:0/3768058343 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbac8078b90 msgr2=0x7fbac807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:07.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.581+0000 7fbaeca00640 1 --2- 172.21.15.39:0/3768058343 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbac8078b90 0x7fbac807b0a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbad8000f60 tx=0x7fbad8009040 comp rx=0 tx=0).stop 2024-09-21T07:36:07.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.581+0000 7fbaeca00640 1 -- 172.21.15.39:0/3768058343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbae815fb80 msgr2=0x7fbae80b51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:07.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.581+0000 7fbaeca00640 1 --2- 172.21.15.39:0/3768058343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbae815fb80 0x7fbae80b51e0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fbad402aa70 tx=0x7fbad4033c20 comp rx=0 tx=0).stop 2024-09-21T07:36:07.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.582+0000 7fbaeca00640 1 -- 172.21.15.39:0/3768058343 shutdown_connections 2024-09-21T07:36:07.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.582+0000 7fbaeca00640 1 --2- 172.21.15.39:0/3768058343 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbac8078b90 0x7fbac807b0a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:07.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.582+0000 7fbaeca00640 1 --2- 172.21.15.39:0/3768058343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbae809d580 0x7fbae80b5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:07.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.582+0000 7fbaeca00640 1 --2- 172.21.15.39:0/3768058343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbae815fb80 0x7fbae80b51e0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:07.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.582+0000 7fbaeca00640 1 --2- 172.21.15.39:0/3768058343 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbae80cc560 0x7fbae81ef460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:07.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.582+0000 7fbaeca00640 1 -- 172.21.15.39:0/3768058343 >> 172.21.15.39:0/3768058343 conn(0x7fbae80d0130 msgr2=0x7fbae80d0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:07.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.583+0000 7fbaeca00640 1 -- 172.21.15.39:0/3768058343 shutdown_connections 2024-09-21T07:36:07.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.584+0000 7fbaeca00640 1 -- 172.21.15.39:0/3768058343 wait complete. 2024-09-21T07:36:07.610 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T07:36:07.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.839+0000 7f3020000640 1 -- 172.21.15.39:0/2483865166 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30180b3650 msgr2=0x7f301806f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:07.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.839+0000 7f3020000640 1 --2- 172.21.15.39:0/2483865166 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30180b3650 0x7f301806f380 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f30180742a0 tx=0x7f301002a9a0 comp rx=0 tx=0).stop 2024-09-21T07:36:07.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.839+0000 7f3020000640 1 -- 172.21.15.39:0/2483865166 shutdown_connections 2024-09-21T07:36:07.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.839+0000 7f3020000640 1 --2- 172.21.15.39:0/2483865166 >> v2:172.21.15.148:3300/0 conn(0x7f3018073e90 0x7f301806ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:07.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.839+0000 7f3020000640 1 --2- 172.21.15.39:0/2483865166 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30180b3650 0x7f301806f380 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:07.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.839+0000 7f3020000640 1 -- 172.21.15.39:0/2483865166 >> 172.21.15.39:0/2483865166 conn(0x7f30180b1c40 msgr2=0x7f30180b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:07.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.839+0000 7f3020000640 1 -- 172.21.15.39:0/2483865166 shutdown_connections 2024-09-21T07:36:07.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.839+0000 7f3020000640 1 -- 172.21.15.39:0/2483865166 wait complete. 2024-09-21T07:36:07.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.841+0000 7f3020000640 1 Processor -- start 2024-09-21T07:36:07.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.841+0000 7f3020000640 1 -- start start 2024-09-21T07:36:07.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.843+0000 7f3020000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3018073e90 0x7f30180c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:07.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.843+0000 7f3020000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f30180748f0 0x7f30181f34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:07.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.843+0000 7f3020000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30180b3650 0x7f30181f67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:07.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.843+0000 7f3020000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f301806fc50 con 0x7f3018073e90 2024-09-21T07:36:07.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.843+0000 7f3020000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f301806fad0 con 0x7f30180b3650 2024-09-21T07:36:07.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.843+0000 7f3020000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f301806fdd0 con 0x7f30180748f0 2024-09-21T07:36:07.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.843+0000 7f301d600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3018073e90 0x7f30180c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:07.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.843+0000 7f301d600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3018073e90 0x7f30180c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48502/0 (socket says 172.21.15.39:48502) 2024-09-21T07:36:07.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.843+0000 7f301d600640 1 -- 172.21.15.39:0/1573657011 learned_addr learned my addr 172.21.15.39:0/1573657011 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:07.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.843+0000 7f301e000640 1 --2- 172.21.15.39:0/1573657011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30180b3650 0x7f30181f67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:07.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.843+0000 7f301cc00640 1 --2- 172.21.15.39:0/1573657011 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f30180748f0 0x7f30181f34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:07.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.844+0000 7f301d600640 1 -- 172.21.15.39:0/1573657011 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f30180748f0 msgr2=0x7f30181f34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:07.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.844+0000 7f301d600640 1 --2- 172.21.15.39:0/1573657011 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f30180748f0 0x7f30181f34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:07.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.844+0000 7f301d600640 1 -- 172.21.15.39:0/1573657011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30180b3650 msgr2=0x7f30181f67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:07.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.844+0000 7f301d600640 1 --2- 172.21.15.39:0/1573657011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30180b3650 0x7f30181f67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:07.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.844+0000 7f301d600640 1 -- 172.21.15.39:0/1573657011 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30180c8460 con 0x7f3018073e90 2024-09-21T07:36:07.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.844+0000 7f301d600640 1 --2- 172.21.15.39:0/1573657011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3018073e90 0x7f30180c66f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f300c00c3a0 tx=0x7f300c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:07.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.844+0000 7f3015600640 1 -- 172.21.15.39:0/1573657011 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300c017070 con 0x7f3018073e90 2024-09-21T07:36:07.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.844+0000 7f3015600640 1 -- 172.21.15.39:0/1573657011 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f300c0053c0 con 0x7f3018073e90 2024-09-21T07:36:07.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.844+0000 7f3015600640 1 -- 172.21.15.39:0/1573657011 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300c003850 con 0x7f3018073e90 2024-09-21T07:36:07.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.844+0000 7f3020000640 1 -- 172.21.15.39:0/1573657011 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30180c8740 con 0x7f3018073e90 2024-09-21T07:36:07.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.845+0000 7f3020000640 1 -- 172.21.15.39:0/1573657011 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30180c8d00 con 0x7f3018073e90 2024-09-21T07:36:07.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.846+0000 7f3020000640 1 -- 172.21.15.39:0/1573657011 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30180b2f40 con 0x7f3018073e90 2024-09-21T07:36:07.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.847+0000 7f3015600640 1 -- 172.21.15.39:0/1573657011 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f300c01a070 con 0x7f3018073e90 2024-09-21T07:36:07.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.848+0000 7f3015600640 1 --2- 172.21.15.39:0/1573657011 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2ff8078bc0 0x7f2ff807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:07.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.848+0000 7f3015600640 1 -- 172.21.15.39:0/1573657011 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6448+0+0 (secure 0 0 0) 0x7f300c09a2a0 con 0x7f3018073e90 2024-09-21T07:36:07.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.848+0000 7f301cc00640 1 --2- 172.21.15.39:0/1573657011 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2ff8078bc0 0x7f2ff807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:07.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.849+0000 7f301cc00640 1 --2- 172.21.15.39:0/1573657011 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2ff8078bc0 0x7f2ff807b0d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f30180b21a0 tx=0x7f300400b390 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:07.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:07.853+0000 7f3015600640 1 -- 172.21.15.39:0/1573657011 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f300c061dd0 con 0x7f3018073e90 2024-09-21T07:36:08.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.051+0000 7f3020000640 1 -- 172.21.15.39:0/1573657011 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f30180cce20 con 0x7f3018073e90 2024-09-21T07:36:08.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.587+0000 7f3015600640 1 -- 172.21.15.39:0/1573657011 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v82) ==== 169+0+0 (secure 0 0 0) 0x7f300c066800 con 0x7f3018073e90 2024-09-21T07:36:08.590 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' removed 2024-09-21T07:36:08.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.590+0000 7f3020000640 1 -- 172.21.15.39:0/1573657011 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2ff8078bc0 msgr2=0x7f2ff807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:08.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.590+0000 7f3020000640 1 --2- 172.21.15.39:0/1573657011 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2ff8078bc0 0x7f2ff807b0d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f30180b21a0 tx=0x7f300400b390 comp rx=0 tx=0).stop 2024-09-21T07:36:08.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.591+0000 7f3020000640 1 -- 172.21.15.39:0/1573657011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3018073e90 msgr2=0x7f30180c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:08.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.591+0000 7f3020000640 1 --2- 172.21.15.39:0/1573657011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3018073e90 0x7f30180c66f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f300c00c3a0 tx=0x7f300c00c860 comp rx=0 tx=0).stop 2024-09-21T07:36:08.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.591+0000 7f3020000640 1 -- 172.21.15.39:0/1573657011 shutdown_connections 2024-09-21T07:36:08.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.591+0000 7f3020000640 1 --2- 172.21.15.39:0/1573657011 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2ff8078bc0 0x7f2ff807b0d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:08.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.591+0000 7f3020000640 1 --2- 172.21.15.39:0/1573657011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30180b3650 0x7f30181f67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:08.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.591+0000 7f3020000640 1 --2- 172.21.15.39:0/1573657011 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f30180748f0 0x7f30181f34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:08.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.591+0000 7f3020000640 1 --2- 172.21.15.39:0/1573657011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3018073e90 0x7f30180c66f0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:08.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.591+0000 7f3020000640 1 -- 172.21.15.39:0/1573657011 >> 172.21.15.39:0/1573657011 conn(0x7f30180b1c40 msgr2=0x7f30180ae940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:08.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.591+0000 7f3020000640 1 -- 172.21.15.39:0/1573657011 shutdown_connections 2024-09-21T07:36:08.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.591+0000 7f3020000640 1 -- 172.21.15.39:0/1573657011 wait complete. 2024-09-21T07:36:08.623 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T07:36:08.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.852+0000 7f6ada400640 1 -- 172.21.15.39:0/3370281372 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6ad40cfb90 msgr2=0x7f6ad40b2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:08.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.852+0000 7f6ada400640 1 --2- 172.21.15.39:0/3370281372 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6ad40cfb90 0x7f6ad40b2fb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6acc005fd0 tx=0x7f6acc02a820 comp rx=0 tx=0).stop 2024-09-21T07:36:08.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.852+0000 7f6ada400640 1 -- 172.21.15.39:0/3370281372 shutdown_connections 2024-09-21T07:36:08.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.852+0000 7f6ada400640 1 --2- 172.21.15.39:0/3370281372 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6ad40cfb90 0x7f6ad40b2fb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:08.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.852+0000 7f6ada400640 1 --2- 172.21.15.39:0/3370281372 >> v2:172.21.15.148:3300/0 conn(0x7f6ad40cd240 0x7f6ad40cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:08.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.852+0000 7f6ada400640 1 -- 172.21.15.39:0/3370281372 >> 172.21.15.39:0/3370281372 conn(0x7f6ad40b1cb0 msgr2=0x7f6ad40b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:08.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.852+0000 7f6ada400640 1 -- 172.21.15.39:0/3370281372 shutdown_connections 2024-09-21T07:36:08.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.852+0000 7f6ada400640 1 -- 172.21.15.39:0/3370281372 wait complete. 2024-09-21T07:36:08.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.854+0000 7f6ada400640 1 Processor -- start 2024-09-21T07:36:08.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.855+0000 7f6ada400640 1 -- start start 2024-09-21T07:36:08.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.856+0000 7f6ada400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ad40cf660 0x7f6ad40c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:08.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.856+0000 7f6ada400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6ad40cfb90 0x7f6ad4146d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:08.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.856+0000 7f6ada400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ad41578d0 0x7f6ad409cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:08.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.856+0000 7f6ada400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad40710a0 con 0x7f6ad40cf660 2024-09-21T07:36:08.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.856+0000 7f6ada400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad4070f20 con 0x7f6ad41578d0 2024-09-21T07:36:08.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.856+0000 7f6ada400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f6ad40b34f0 con 0x7f6ad40cfb90 2024-09-21T07:36:08.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.856+0000 7f6ad3400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ad40cf660 0x7f6ad40c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:08.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.856+0000 7f6ad3400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ad40cf660 0x7f6ad40c9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48504/0 (socket says 172.21.15.39:48504) 2024-09-21T07:36:08.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.856+0000 7f6ad3400640 1 -- 172.21.15.39:0/3578395163 learned_addr learned my addr 172.21.15.39:0/3578395163 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:08.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.856+0000 7f6ad3e00640 1 --2- 172.21.15.39:0/3578395163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ad41578d0 0x7f6ad409cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:08.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.856+0000 7f6ad2a00640 1 --2- 172.21.15.39:0/3578395163 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6ad40cfb90 0x7f6ad4146d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:08.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.857+0000 7f6ad3400640 1 -- 172.21.15.39:0/3578395163 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6ad40cfb90 msgr2=0x7f6ad4146d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:08.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.857+0000 7f6ad3400640 1 --2- 172.21.15.39:0/3578395163 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6ad40cfb90 0x7f6ad4146d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:08.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.857+0000 7f6ad3400640 1 -- 172.21.15.39:0/3578395163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ad41578d0 msgr2=0x7f6ad409cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:08.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.857+0000 7f6ad3400640 1 --2- 172.21.15.39:0/3578395163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ad41578d0 0x7f6ad409cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:08.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.857+0000 7f6ad3400640 1 -- 172.21.15.39:0/3578395163 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad4073680 con 0x7f6ad40cf660 2024-09-21T07:36:08.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.857+0000 7f6ad3400640 1 --2- 172.21.15.39:0/3578395163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ad40cf660 0x7f6ad40c9530 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f6ac400c400 tx=0x7f6ac400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:08.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.857+0000 7f6acb400640 1 -- 172.21.15.39:0/3578395163 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac4011790 con 0x7f6ad40cf660 2024-09-21T07:36:08.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.857+0000 7f6acb400640 1 -- 172.21.15.39:0/3578395163 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ac4017070 con 0x7f6ad40cf660 2024-09-21T07:36:08.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.857+0000 7f6ada400640 1 -- 172.21.15.39:0/3578395163 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ad4147280 con 0x7f6ad40cf660 2024-09-21T07:36:08.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.857+0000 7f6acb400640 1 -- 172.21.15.39:0/3578395163 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac4011cc0 con 0x7f6ad40cf660 2024-09-21T07:36:08.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.858+0000 7f6ada400640 1 -- 172.21.15.39:0/3578395163 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ad40730c0 con 0x7f6ad40cf660 2024-09-21T07:36:08.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.859+0000 7f6ada400640 1 -- 172.21.15.39:0/3578395163 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ad40b4b20 con 0x7f6ad40cf660 2024-09-21T07:36:08.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.860+0000 7f6acb400640 1 -- 172.21.15.39:0/3578395163 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6ac401c460 con 0x7f6ad40cf660 2024-09-21T07:36:08.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.861+0000 7f6acb400640 1 --2- 172.21.15.39:0/3578395163 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6ab4078b90 0x7f6ab407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:08.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.862+0000 7f6acb400640 1 -- 172.21.15.39:0/3578395163 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6034+0+0 (secure 0 0 0) 0x7f6ac409ab80 con 0x7f6ad40cf660 2024-09-21T07:36:08.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.862+0000 7f6ad2a00640 1 --2- 172.21.15.39:0/3578395163 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6ab4078b90 0x7f6ab407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:08.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.863+0000 7f6ad2a00640 1 --2- 172.21.15.39:0/3578395163 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6ab4078b90 0x7f6ab407b0a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6ad40c9280 tx=0x7f6ac0005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:08.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:08.866+0000 7f6acb400640 1 -- 172.21.15.39:0/3578395163 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6ad4073680 con 0x7f6ad40cf660 2024-09-21T07:36:09.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.063+0000 7f6ada400640 1 -- 172.21.15.39:0/3578395163 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f6ad406ddf0 con 0x7f6ad40cf660 2024-09-21T07:36:09.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.067+0000 7f6acb400640 1 -- 172.21.15.39:0/3578395163 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v40) ==== 71+0+0 (secure 0 0 0) 0x7f6ac40628f0 con 0x7f6ad40cf660 2024-09-21T07:36:09.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.070+0000 7f6ada400640 1 -- 172.21.15.39:0/3578395163 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6ab4078b90 msgr2=0x7f6ab407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:09.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.070+0000 7f6ada400640 1 --2- 172.21.15.39:0/3578395163 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6ab4078b90 0x7f6ab407b0a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6ad40c9280 tx=0x7f6ac0005f90 comp rx=0 tx=0).stop 2024-09-21T07:36:09.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.070+0000 7f6ada400640 1 -- 172.21.15.39:0/3578395163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ad40cf660 msgr2=0x7f6ad40c9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:09.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.070+0000 7f6ada400640 1 --2- 172.21.15.39:0/3578395163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ad40cf660 0x7f6ad40c9530 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f6ac400c400 tx=0x7f6ac400c8c0 comp rx=0 tx=0).stop 2024-09-21T07:36:09.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.070+0000 7f6ada400640 1 -- 172.21.15.39:0/3578395163 shutdown_connections 2024-09-21T07:36:09.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.070+0000 7f6ada400640 1 --2- 172.21.15.39:0/3578395163 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6ab4078b90 0x7f6ab407b0a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:09.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.070+0000 7f6ada400640 1 --2- 172.21.15.39:0/3578395163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ad41578d0 0x7f6ad409cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:09.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.070+0000 7f6ada400640 1 --2- 172.21.15.39:0/3578395163 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6ad40cfb90 0x7f6ad4146d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:09.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.070+0000 7f6ada400640 1 --2- 172.21.15.39:0/3578395163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ad40cf660 0x7f6ad40c9530 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:09.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.070+0000 7f6ada400640 1 -- 172.21.15.39:0/3578395163 >> 172.21.15.39:0/3578395163 conn(0x7f6ad40b1cb0 msgr2=0x7f6ad40b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:09.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.070+0000 7f6ada400640 1 -- 172.21.15.39:0/3578395163 shutdown_connections 2024-09-21T07:36:09.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.070+0000 7f6ada400640 1 -- 172.21.15.39:0/3578395163 wait complete. 2024-09-21T07:36:09.097 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_allow_new_snaps_config' 2024-09-21T07:36:09.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.323+0000 7f9808a00640 1 -- 172.21.15.39:0/2198822638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98040d08b0 msgr2=0x7f98041606c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:09.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.323+0000 7f9808a00640 1 --2- 172.21.15.39:0/2198822638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98040d08b0 0x7f98041606c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f97f4009830 tx=0x7f97f402a530 comp rx=0 tx=0).stop 2024-09-21T07:36:09.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.323+0000 7f9808a00640 1 -- 172.21.15.39:0/2198822638 shutdown_connections 2024-09-21T07:36:09.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.323+0000 7f9808a00640 1 --2- 172.21.15.39:0/2198822638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98040d08b0 0x7f98041606c0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:09.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.323+0000 7f9808a00640 1 -- 172.21.15.39:0/2198822638 >> 172.21.15.39:0/2198822638 conn(0x7f98040b3090 msgr2=0x7f98040b34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:09.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.323+0000 7f9808a00640 1 -- 172.21.15.39:0/2198822638 shutdown_connections 2024-09-21T07:36:09.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.323+0000 7f9808a00640 1 -- 172.21.15.39:0/2198822638 wait complete. 2024-09-21T07:36:09.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.325+0000 7f9808a00640 1 Processor -- start 2024-09-21T07:36:09.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.325+0000 7f9808a00640 1 -- start start 2024-09-21T07:36:09.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.327+0000 7f9808a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f980406ff60 0x7f9804137510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:09.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.327+0000 7f9808a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98040d08b0 0x7f98041f23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:09.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.327+0000 7f9808a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9804160c00 0x7f98041f28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:09.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.327+0000 7f9808a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f98040cdf80 con 0x7f9804160c00 2024-09-21T07:36:09.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.327+0000 7f9808a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f98040cde00 con 0x7f98040d08b0 2024-09-21T07:36:09.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.327+0000 7f9808a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f98040ce100 con 0x7f980406ff60 2024-09-21T07:36:09.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.327+0000 7f9803200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9804160c00 0x7f98041f28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:09.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.327+0000 7f9803200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9804160c00 0x7f98041f28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48528/0 (socket says 172.21.15.39:48528) 2024-09-21T07:36:09.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.327+0000 7f9803200640 1 -- 172.21.15.39:0/2796219085 learned_addr learned my addr 172.21.15.39:0/2796219085 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:09.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.327+0000 7f9802800640 1 --2- 172.21.15.39:0/2796219085 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f980406ff60 0x7f9804137510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:09.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.327+0000 7f97f9e00640 1 --2- 172.21.15.39:0/2796219085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98040d08b0 0x7f98041f23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:09.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.328+0000 7f9803200640 1 -- 172.21.15.39:0/2796219085 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f980406ff60 msgr2=0x7f9804137510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:09.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.328+0000 7f9803200640 1 --2- 172.21.15.39:0/2796219085 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f980406ff60 0x7f9804137510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:09.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.328+0000 7f9803200640 1 -- 172.21.15.39:0/2796219085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98040d08b0 msgr2=0x7f98041f23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:09.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.328+0000 7f9803200640 1 --2- 172.21.15.39:0/2796219085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98040d08b0 0x7f98041f23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:09.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.328+0000 7f9803200640 1 -- 172.21.15.39:0/2796219085 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9804132670 con 0x7f9804160c00 2024-09-21T07:36:09.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.328+0000 7f9802800640 1 --2- 172.21.15.39:0/2796219085 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f980406ff60 0x7f9804137510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:36:09.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.328+0000 7f9803200640 1 --2- 172.21.15.39:0/2796219085 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9804160c00 0x7f98041f28f0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f97fc00c6b0 tx=0x7f97fc00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:09.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.328+0000 7f97fb400640 1 -- 172.21.15.39:0/2796219085 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97fc003b70 con 0x7f9804160c00 2024-09-21T07:36:09.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.328+0000 7f97fb400640 1 -- 172.21.15.39:0/2796219085 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f97fc003d30 con 0x7f9804160c00 2024-09-21T07:36:09.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.328+0000 7f9808a00640 1 -- 172.21.15.39:0/2796219085 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9804132950 con 0x7f9804160c00 2024-09-21T07:36:09.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.328+0000 7f97fb400640 1 -- 172.21.15.39:0/2796219085 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97fc005330 con 0x7f9804160c00 2024-09-21T07:36:09.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.328+0000 7f9808a00640 1 -- 172.21.15.39:0/2796219085 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9804132f10 con 0x7f9804160c00 2024-09-21T07:36:09.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.329+0000 7f9808a00640 1 -- 172.21.15.39:0/2796219085 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9804156380 con 0x7f9804160c00 2024-09-21T07:36:09.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.331+0000 7f97fb400640 1 -- 172.21.15.39:0/2796219085 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f97fc0054d0 con 0x7f9804160c00 2024-09-21T07:36:09.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.332+0000 7f97fb400640 1 --2- 172.21.15.39:0/2796219085 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f97e0078b90 0x7f97e007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:09.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.332+0000 7f97fb400640 1 -- 172.21.15.39:0/2796219085 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6034+0+0 (secure 0 0 0) 0x7f97fc09ad00 con 0x7f9804160c00 2024-09-21T07:36:09.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.332+0000 7f9802800640 1 --2- 172.21.15.39:0/2796219085 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f97e0078b90 0x7f97e007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:09.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.333+0000 7f9802800640 1 --2- 172.21.15.39:0/2796219085 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f97e0078b90 0x7f97e007b0a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f97f4034000 tx=0x7f97f4004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:09.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.336+0000 7f97fb400640 1 -- 172.21.15.39:0/2796219085 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9804156380 con 0x7f9804160c00 2024-09-21T07:36:09.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:09.618+0000 7f9808a00640 1 -- 172.21.15.39:0/2796219085 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_allow_new_snaps_config"]} v 0) -- 0x7f98041606c0 con 0x7f9804160c00 2024-09-21T07:36:10.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.593+0000 7f97fb400640 1 -- 172.21.15.39:0/2796219085 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_allow_new_snaps_config"]}]=0 v186) ==== 148+0+0 (secure 0 0 0) 0x7f97fc062f60 con 0x7f9804160c00 2024-09-21T07:36:10.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.596+0000 7f9808a00640 1 -- 172.21.15.39:0/2796219085 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f97e0078b90 msgr2=0x7f97e007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:10.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.596+0000 7f9808a00640 1 --2- 172.21.15.39:0/2796219085 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f97e0078b90 0x7f97e007b0a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f97f4034000 tx=0x7f97f4004730 comp rx=0 tx=0).stop 2024-09-21T07:36:10.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.596+0000 7f9808a00640 1 -- 172.21.15.39:0/2796219085 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9804160c00 msgr2=0x7f98041f28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:10.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.596+0000 7f9808a00640 1 --2- 172.21.15.39:0/2796219085 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9804160c00 0x7f98041f28f0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f97fc00c6b0 tx=0x7f97fc00cb70 comp rx=0 tx=0).stop 2024-09-21T07:36:10.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.596+0000 7f9808a00640 1 -- 172.21.15.39:0/2796219085 shutdown_connections 2024-09-21T07:36:10.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.596+0000 7f9808a00640 1 --2- 172.21.15.39:0/2796219085 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f97e0078b90 0x7f97e007b0a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:10.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.596+0000 7f9808a00640 1 --2- 172.21.15.39:0/2796219085 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9804160c00 0x7f98041f28f0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:10.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.596+0000 7f9808a00640 1 --2- 172.21.15.39:0/2796219085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98040d08b0 0x7f98041f23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:10.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.596+0000 7f9808a00640 1 --2- 172.21.15.39:0/2796219085 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f980406ff60 0x7f9804137510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:10.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.596+0000 7f9808a00640 1 -- 172.21.15.39:0/2796219085 >> 172.21.15.39:0/2796219085 conn(0x7f98040b3090 msgr2=0x7f980406d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:10.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.597+0000 7f9808a00640 1 -- 172.21.15.39:0/2796219085 shutdown_connections 2024-09-21T07:36:10.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.597+0000 7f9808a00640 1 -- 172.21.15.39:0/2796219085 wait complete. 2024-09-21T07:36:10.628 INFO:tasks.cephfs_test_runner:Check whether 'allow_new_snaps' setting works ... ok 2024-09-21T07:36:10.628 INFO:tasks.cephfs_test_runner:Starting test: test_kill_mdstable (tasks.cephfs.test_snapshots.TestSnapshots) check snaptable transcation 2024-09-21T07:36:10.628 INFO:tasks.cephfs_test_runner:test_kill_mdstable (tasks.cephfs.test_snapshots.TestSnapshots) 2024-09-21T07:36:10.629 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:36:10.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.859+0000 7f42aa600640 1 -- 172.21.15.39:0/2579250824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4288004d40 msgr2=0x7f4288005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:10.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.859+0000 7f42aa600640 1 --2- 172.21.15.39:0/2579250824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4288004d40 0x7f4288005150 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f429c005fd0 tx=0x7f4288005830 comp rx=0 tx=0).stop 2024-09-21T07:36:10.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.859+0000 7f42a3e00640 1 -- 172.21.15.39:0/2579250824 reap_dead start 2024-09-21T07:36:10.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.859+0000 7f42aa600640 1 -- 172.21.15.39:0/2579250824 shutdown_connections 2024-09-21T07:36:10.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.859+0000 7f42aa600640 1 -- 172.21.15.39:0/2579250824 >> 172.21.15.39:0/2579250824 conn(0x7f42a40b1cb0 msgr2=0x7f42a40b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:10.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.859+0000 7f42aa600640 1 -- 172.21.15.39:0/2579250824 shutdown_connections 2024-09-21T07:36:10.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.859+0000 7f42aa600640 1 -- 172.21.15.39:0/2579250824 wait complete. 2024-09-21T07:36:10.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.862+0000 7f42aa600640 1 Processor -- start 2024-09-21T07:36:10.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.862+0000 7f42aa600640 1 -- start start 2024-09-21T07:36:10.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.863+0000 7f42aa600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4288004d40 0x7f42a41f9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:10.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.863+0000 7f42aa600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42a414f6b0 0x7f42a409aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:10.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.863+0000 7f42aa600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42a41583f0 0x7f42a4158800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:10.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.863+0000 7f42aa600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f42a40b35d0 con 0x7f42a414f6b0 2024-09-21T07:36:10.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.863+0000 7f42aa600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f42a415cc70 con 0x7f4288004d40 2024-09-21T07:36:10.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.863+0000 7f42aa600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f42a40b3750 con 0x7f42a41583f0 2024-09-21T07:36:10.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.864+0000 7f42a2a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42a414f6b0 0x7f42a409aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:10.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.864+0000 7f42a2a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42a414f6b0 0x7f42a409aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48542/0 (socket says 172.21.15.39:48542) 2024-09-21T07:36:10.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.864+0000 7f42a2a00640 1 -- 172.21.15.39:0/3172590286 learned_addr learned my addr 172.21.15.39:0/3172590286 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:10.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.864+0000 7f42a3400640 1 --2- 172.21.15.39:0/3172590286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4288004d40 0x7f42a41f9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:10.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.864+0000 7f42a3e00640 1 --2- 172.21.15.39:0/3172590286 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42a41583f0 0x7f42a4158800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:10.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.864+0000 7f42a2a00640 1 -- 172.21.15.39:0/3172590286 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42a41583f0 msgr2=0x7f42a4158800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:10.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.864+0000 7f42a2a00640 1 --2- 172.21.15.39:0/3172590286 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42a41583f0 0x7f42a4158800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:10.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.864+0000 7f42a2a00640 1 -- 172.21.15.39:0/3172590286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4288004d40 msgr2=0x7f42a41f9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:10.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.864+0000 7f42a2a00640 1 --2- 172.21.15.39:0/3172590286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4288004d40 0x7f42a41f9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:10.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.864+0000 7f42a2a00640 1 -- 172.21.15.39:0/3172590286 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42a4157350 con 0x7f42a414f6b0 2024-09-21T07:36:10.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.864+0000 7f42a2a00640 1 --2- 172.21.15.39:0/3172590286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42a414f6b0 0x7f42a409aa30 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f4290006630 tx=0x7f42900039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:10.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.865+0000 7f429b400640 1 -- 172.21.15.39:0/3172590286 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4290015070 con 0x7f42a414f6b0 2024-09-21T07:36:10.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.865+0000 7f429b400640 1 -- 172.21.15.39:0/3172590286 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4290011900 con 0x7f42a414f6b0 2024-09-21T07:36:10.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.865+0000 7f42aa600640 1 -- 172.21.15.39:0/3172590286 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42a4157630 con 0x7f42a414f6b0 2024-09-21T07:36:10.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.865+0000 7f429b400640 1 -- 172.21.15.39:0/3172590286 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4290011d50 con 0x7f42a414f6b0 2024-09-21T07:36:10.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.865+0000 7f42aa600640 1 -- 172.21.15.39:0/3172590286 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42a4157bf0 con 0x7f42a414f6b0 2024-09-21T07:36:10.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.866+0000 7f42aa600640 1 -- 172.21.15.39:0/3172590286 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42a40c3860 con 0x7f42a414f6b0 2024-09-21T07:36:10.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.867+0000 7f429b400640 1 -- 172.21.15.39:0/3172590286 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4290018070 con 0x7f42a414f6b0 2024-09-21T07:36:10.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.868+0000 7f429b400640 1 --2- 172.21.15.39:0/3172590286 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4284078b90 0x7f428407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:10.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.868+0000 7f429b400640 1 -- 172.21.15.39:0/3172590286 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6034+0+0 (secure 0 0 0) 0x7f429009bb10 con 0x7f42a414f6b0 2024-09-21T07:36:10.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.868+0000 7f42a3400640 1 --2- 172.21.15.39:0/3172590286 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4284078b90 0x7f428407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:10.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.869+0000 7f42a3400640 1 --2- 172.21.15.39:0/3172590286 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4284078b90 0x7f428407b0a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4294009c50 tx=0x7f4294005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:10.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:10.872+0000 7f429b400640 1 -- 172.21.15.39:0/3172590286 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f42900637a0 con 0x7f42a414f6b0 2024-09-21T07:36:11.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.070+0000 7f42aa600640 1 -- 172.21.15.39:0/3172590286 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f42a41570c0 con 0x7f42a414f6b0 2024-09-21T07:36:11.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.071+0000 7f429b400640 1 -- 172.21.15.39:0/3172590286 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18492 (secure 0 0 0) 0x7f42900681d0 con 0x7f42a414f6b0 2024-09-21T07:36:11.073 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:36:11.074 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":82,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:36:08.580550+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6834/1032917299":"2024-09-22T07:36:05.249854+0000","172.21.15.39:6833/1032917299":"2024-09-22T07:36:05.249854+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:36:11.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.074+0000 7f42aa600640 1 -- 172.21.15.39:0/3172590286 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4284078b90 msgr2=0x7f428407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:11.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.074+0000 7f42aa600640 1 --2- 172.21.15.39:0/3172590286 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4284078b90 0x7f428407b0a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4294009c50 tx=0x7f4294005f90 comp rx=0 tx=0).stop 2024-09-21T07:36:11.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.074+0000 7f42aa600640 1 -- 172.21.15.39:0/3172590286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42a414f6b0 msgr2=0x7f42a409aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:11.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.074+0000 7f42aa600640 1 --2- 172.21.15.39:0/3172590286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42a414f6b0 0x7f42a409aa30 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f4290006630 tx=0x7f42900039c0 comp rx=0 tx=0).stop 2024-09-21T07:36:11.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.074+0000 7f42aa600640 1 -- 172.21.15.39:0/3172590286 shutdown_connections 2024-09-21T07:36:11.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.074+0000 7f42aa600640 1 --2- 172.21.15.39:0/3172590286 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4284078b90 0x7f428407b0a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:11.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.074+0000 7f42aa600640 1 --2- 172.21.15.39:0/3172590286 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42a41583f0 0x7f42a4158800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:11.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.074+0000 7f42aa600640 1 --2- 172.21.15.39:0/3172590286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42a414f6b0 0x7f42a409aa30 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:11.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.074+0000 7f42aa600640 1 --2- 172.21.15.39:0/3172590286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4288004d40 0x7f42a41f9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:11.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.074+0000 7f42aa600640 1 -- 172.21.15.39:0/3172590286 >> 172.21.15.39:0/3172590286 conn(0x7f42a40b1cb0 msgr2=0x7f42a40aeb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:11.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.074+0000 7f42aa600640 1 -- 172.21.15.39:0/3172590286 shutdown_connections 2024-09-21T07:36:11.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.074+0000 7f42aa600640 1 -- 172.21.15.39:0/3172590286 wait complete. 2024-09-21T07:36:11.101 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T07:33:01.569433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'snap-test-pool', 'create_time': '2024-09-21T07:34:22.032185+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 1, 'snap_epoch': 37, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'test_data_pool', 'create_time': '2024-09-21T07:35:25.604532+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 1, 'snap_epoch': 66, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_metadata_pool', 'create_time': '2024-09-21T07:35:26.327607+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T07:36:11.101 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T07:36:11.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.331+0000 7f6ecfc00640 1 -- 172.21.15.39:0/3257667989 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6eb0004800 msgr2=0x7f6eb0004cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:11.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.331+0000 7f6ecfc00640 1 --2- 172.21.15.39:0/3257667989 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6eb0004800 0x7f6eb0004cb0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f6ec80728c0 tx=0x7f6eb0005830 comp rx=0 tx=0).stop 2024-09-21T07:36:11.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.331+0000 7f6ecdc00640 1 -- 172.21.15.39:0/3257667989 reap_dead start 2024-09-21T07:36:11.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.331+0000 7f6ecfc00640 1 -- 172.21.15.39:0/3257667989 shutdown_connections 2024-09-21T07:36:11.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.331+0000 7f6ecfc00640 1 -- 172.21.15.39:0/3257667989 >> 172.21.15.39:0/3257667989 conn(0x7f6ec80d0160 msgr2=0x7f6ec80d0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:11.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.331+0000 7f6ecfc00640 1 -- 172.21.15.39:0/3257667989 shutdown_connections 2024-09-21T07:36:11.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.331+0000 7f6ecfc00640 1 -- 172.21.15.39:0/3257667989 wait complete. 2024-09-21T07:36:11.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.333+0000 7f6ecfc00640 1 Processor -- start 2024-09-21T07:36:11.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.333+0000 7f6ecfc00640 1 -- start start 2024-09-21T07:36:11.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.335+0000 7f6ecfc00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6eb0004800 0x7f6ec809c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:11.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.335+0000 7f6ecfc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ec8156200 0x7f6ec809db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:11.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.335+0000 7f6ecfc00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6ec809bf80 0x7f6ec815cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:11.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.335+0000 7f6ecfc00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec80cb370 con 0x7f6ec8156200 2024-09-21T07:36:11.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.335+0000 7f6ecfc00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec80b5a20 con 0x7f6eb0004800 2024-09-21T07:36:11.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.335+0000 7f6ecfc00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f6ec80cb4f0 con 0x7f6ec809bf80 2024-09-21T07:36:11.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.335+0000 7f6ec7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ec8156200 0x7f6ec809db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:11.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.335+0000 7f6ec7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ec8156200 0x7f6ec809db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48568/0 (socket says 172.21.15.39:48568) 2024-09-21T07:36:11.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.335+0000 7f6ec7e00640 1 -- 172.21.15.39:0/3107077159 learned_addr learned my addr 172.21.15.39:0/3107077159 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:11.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.335+0000 7f6ecd200640 1 --2- 172.21.15.39:0/3107077159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6eb0004800 0x7f6ec809c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:11.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.335+0000 7f6ecdc00640 1 --2- 172.21.15.39:0/3107077159 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6ec809bf80 0x7f6ec815cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:11.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.336+0000 7f6ec7e00640 1 -- 172.21.15.39:0/3107077159 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6ec809bf80 msgr2=0x7f6ec815cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:11.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.336+0000 7f6ec7e00640 1 --2- 172.21.15.39:0/3107077159 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6ec809bf80 0x7f6ec815cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:11.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.336+0000 7f6ec7e00640 1 -- 172.21.15.39:0/3107077159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6eb0004800 msgr2=0x7f6ec809c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:11.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.336+0000 7f6ec7e00640 1 --2- 172.21.15.39:0/3107077159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6eb0004800 0x7f6ec809c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:11.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.336+0000 7f6ec7e00640 1 -- 172.21.15.39:0/3107077159 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ec815aa10 con 0x7f6ec8156200 2024-09-21T07:36:11.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.336+0000 7f6ec7e00640 1 --2- 172.21.15.39:0/3107077159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ec8156200 0x7f6ec809db10 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f6eb402b040 tx=0x7f6eb4006060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:11.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.336+0000 7f6ec4c00640 1 -- 172.21.15.39:0/3107077159 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4032030 con 0x7f6ec8156200 2024-09-21T07:36:11.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.336+0000 7f6ec4c00640 1 -- 172.21.15.39:0/3107077159 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6eb40037f0 con 0x7f6ec8156200 2024-09-21T07:36:11.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.336+0000 7f6ecfc00640 1 -- 172.21.15.39:0/3107077159 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ec815dce0 con 0x7f6ec8156200 2024-09-21T07:36:11.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.336+0000 7f6ec4c00640 1 -- 172.21.15.39:0/3107077159 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4003be0 con 0x7f6ec8156200 2024-09-21T07:36:11.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.336+0000 7f6ecfc00640 1 -- 172.21.15.39:0/3107077159 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ec815b2d0 con 0x7f6ec8156200 2024-09-21T07:36:11.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.337+0000 7f6ecfc00640 1 -- 172.21.15.39:0/3107077159 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ec80cba80 con 0x7f6ec8156200 2024-09-21T07:36:11.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.339+0000 7f6ec4c00640 1 -- 172.21.15.39:0/3107077159 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6eb4035020 con 0x7f6ec8156200 2024-09-21T07:36:11.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.340+0000 7f6ec4c00640 1 --2- 172.21.15.39:0/3107077159 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6ea8078b90 0x7f6ea807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:11.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.340+0000 7f6ec4c00640 1 -- 172.21.15.39:0/3107077159 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6034+0+0 (secure 0 0 0) 0x7f6eb40bb150 con 0x7f6ec8156200 2024-09-21T07:36:11.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.340+0000 7f6ecd200640 1 --2- 172.21.15.39:0/3107077159 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6ea8078b90 0x7f6ea807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:11.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.341+0000 7f6ecd200640 1 --2- 172.21.15.39:0/3107077159 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6ea8078b90 0x7f6ea807b0a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6ec80cdc70 tx=0x7f6ebc005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:11.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.345+0000 7f6ec4c00640 1 -- 172.21.15.39:0/3107077159 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6ec80cc440 con 0x7f6ec8156200 2024-09-21T07:36:11.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.544+0000 7f6ecfc00640 1 -- 172.21.15.39:0/3107077159 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6ec81f4690 con 0x7f6ec8156200 2024-09-21T07:36:11.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.544+0000 7f6ec4c00640 1 -- 172.21.15.39:0/3107077159 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f6eb4082e70 con 0x7f6ec8156200 2024-09-21T07:36:11.547 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-09-21T07:36:11.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.547+0000 7f6ecfc00640 1 -- 172.21.15.39:0/3107077159 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6ea8078b90 msgr2=0x7f6ea807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:11.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.547+0000 7f6ecfc00640 1 --2- 172.21.15.39:0/3107077159 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6ea8078b90 0x7f6ea807b0a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6ec80cdc70 tx=0x7f6ebc005f90 comp rx=0 tx=0).stop 2024-09-21T07:36:11.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.547+0000 7f6ecfc00640 1 -- 172.21.15.39:0/3107077159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ec8156200 msgr2=0x7f6ec809db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:11.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.547+0000 7f6ecfc00640 1 --2- 172.21.15.39:0/3107077159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ec8156200 0x7f6ec809db10 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f6eb402b040 tx=0x7f6eb4006060 comp rx=0 tx=0).stop 2024-09-21T07:36:11.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.547+0000 7f6ecfc00640 1 -- 172.21.15.39:0/3107077159 shutdown_connections 2024-09-21T07:36:11.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.547+0000 7f6ecfc00640 1 --2- 172.21.15.39:0/3107077159 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6ea8078b90 0x7f6ea807b0a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:11.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.547+0000 7f6ecfc00640 1 --2- 172.21.15.39:0/3107077159 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6ec809bf80 0x7f6ec815cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:11.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.547+0000 7f6ecfc00640 1 --2- 172.21.15.39:0/3107077159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ec8156200 0x7f6ec809db10 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:11.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.547+0000 7f6ecfc00640 1 --2- 172.21.15.39:0/3107077159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6eb0004800 0x7f6ec809c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:11.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.547+0000 7f6ecfc00640 1 -- 172.21.15.39:0/3107077159 >> 172.21.15.39:0/3107077159 conn(0x7f6ec80d0160 msgr2=0x7f6ec815e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:11.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.547+0000 7f6ecfc00640 1 -- 172.21.15.39:0/3107077159 shutdown_connections 2024-09-21T07:36:11.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.547+0000 7f6ecfc00640 1 -- 172.21.15.39:0/3107077159 wait complete. 2024-09-21T07:36:11.571 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2024-09-21T07:36:11.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.801+0000 7f3a9ec00640 1 -- 172.21.15.39:0/1032205152 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a980cd050 msgr2=0x7f3a9815f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:11.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.801+0000 7f3a9ec00640 1 --2- 172.21.15.39:0/1032205152 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a980cd050 0x7f3a9815f410 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f3a90005fd0 tx=0x7f3a9002aa90 comp rx=0 tx=0).stop 2024-09-21T07:36:11.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.802+0000 7f3a9ec00640 1 -- 172.21.15.39:0/1032205152 shutdown_connections 2024-09-21T07:36:11.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.802+0000 7f3a9ec00640 1 --2- 172.21.15.39:0/1032205152 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3a9806f2a0 0x7f3a980ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:11.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.802+0000 7f3a9ec00640 1 --2- 172.21.15.39:0/1032205152 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a980cd050 0x7f3a9815f410 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:11.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.802+0000 7f3a9ec00640 1 -- 172.21.15.39:0/1032205152 >> 172.21.15.39:0/1032205152 conn(0x7f3a9806d7d0 msgr2=0x7f3a980b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:11.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.802+0000 7f3a9ec00640 1 -- 172.21.15.39:0/1032205152 shutdown_connections 2024-09-21T07:36:11.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.802+0000 7f3a9ec00640 1 -- 172.21.15.39:0/1032205152 wait complete. 2024-09-21T07:36:11.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.804+0000 7f3a9ec00640 1 Processor -- start 2024-09-21T07:36:11.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.804+0000 7f3a9ec00640 1 -- start start 2024-09-21T07:36:11.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.805+0000 7f3a9ec00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a9806ee90 0x7f3a9814eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:11.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.805+0000 7f3a9ec00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3a9806f2a0 0x7f3a98074830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:11.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.805+0000 7f3a9ec00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a980cd050 0x7f3a980b0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:11.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.806+0000 7f3a9ec00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3a98162730 con 0x7f3a980cd050 2024-09-21T07:36:11.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.806+0000 7f3a9ec00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3a981625b0 con 0x7f3a9806ee90 2024-09-21T07:36:11.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.806+0000 7f3a9ec00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f3a9815f950 con 0x7f3a9806f2a0 2024-09-21T07:36:11.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.806+0000 7f3a9cc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a980cd050 0x7f3a980b0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:11.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.806+0000 7f3a9cc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a980cd050 0x7f3a980b0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48582/0 (socket says 172.21.15.39:48582) 2024-09-21T07:36:11.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.806+0000 7f3a9cc00640 1 -- 172.21.15.39:0/1487621901 learned_addr learned my addr 172.21.15.39:0/1487621901 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:11.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.806+0000 7f3a97400640 1 --2- 172.21.15.39:0/1487621901 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3a9806f2a0 0x7f3a98074830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:11.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.806+0000 7f3a97e00640 1 --2- 172.21.15.39:0/1487621901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a9806ee90 0x7f3a9814eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:11.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.806+0000 7f3a9cc00640 1 -- 172.21.15.39:0/1487621901 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3a9806f2a0 msgr2=0x7f3a98074830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:11.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.806+0000 7f3a9cc00640 1 --2- 172.21.15.39:0/1487621901 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3a9806f2a0 0x7f3a98074830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:11.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.806+0000 7f3a9cc00640 1 -- 172.21.15.39:0/1487621901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a9806ee90 msgr2=0x7f3a9814eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:11.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.806+0000 7f3a9cc00640 1 --2- 172.21.15.39:0/1487621901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a9806ee90 0x7f3a9814eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:11.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.806+0000 7f3a9cc00640 1 -- 172.21.15.39:0/1487621901 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a981e9730 con 0x7f3a980cd050 2024-09-21T07:36:11.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.806+0000 7f3a97e00640 1 --2- 172.21.15.39:0/1487621901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a9806ee90 0x7f3a9814eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:36:11.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.807+0000 7f3a9cc00640 1 --2- 172.21.15.39:0/1487621901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a980cd050 0x7f3a980b0210 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3a90009bf0 tx=0x7f3a90003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:11.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.807+0000 7f3a8fe00640 1 -- 172.21.15.39:0/1487621901 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a90044070 con 0x7f3a980cd050 2024-09-21T07:36:11.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.807+0000 7f3a8fe00640 1 -- 172.21.15.39:0/1487621901 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a90004250 con 0x7f3a980cd050 2024-09-21T07:36:11.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.807+0000 7f3a8fe00640 1 -- 172.21.15.39:0/1487621901 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a90004660 con 0x7f3a980cd050 2024-09-21T07:36:11.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.807+0000 7f3a9ec00640 1 -- 172.21.15.39:0/1487621901 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a981e9a10 con 0x7f3a980cd050 2024-09-21T07:36:11.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.807+0000 7f3a9ec00640 1 -- 172.21.15.39:0/1487621901 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a981e9fd0 con 0x7f3a980cd050 2024-09-21T07:36:11.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.808+0000 7f3a9ec00640 1 -- 172.21.15.39:0/1487621901 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a980750f0 con 0x7f3a980cd050 2024-09-21T07:36:11.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.809+0000 7f3a8fe00640 1 -- 172.21.15.39:0/1487621901 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3a90047070 con 0x7f3a980cd050 2024-09-21T07:36:11.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.810+0000 7f3a8fe00640 1 --2- 172.21.15.39:0/1487621901 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3a78078bc0 0x7f3a7807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:11.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.811+0000 7f3a8fe00640 1 -- 172.21.15.39:0/1487621901 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6034+0+0 (secure 0 0 0) 0x7f3a900bbe50 con 0x7f3a980cd050 2024-09-21T07:36:11.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.811+0000 7f3a97e00640 1 --2- 172.21.15.39:0/1487621901 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3a78078bc0 0x7f3a7807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:11.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.812+0000 7f3a97e00640 1 --2- 172.21.15.39:0/1487621901 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3a78078bc0 0x7f3a7807b0d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3a980cd630 tx=0x7f3a88009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:11.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:11.815+0000 7f3a8fe00640 1 -- 172.21.15.39:0/1487621901 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3a90032030 con 0x7f3a980cd050 2024-09-21T07:36:12.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.011+0000 7f3a9ec00640 1 -- 172.21.15.39:0/1487621901 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) -- 0x7f3a98070b70 con 0x7f3a980cd050 2024-09-21T07:36:12.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.011+0000 7f3a8fe00640 1 -- 172.21.15.39:0/1487621901 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v82) ==== 103+0+11 (secure 0 0 0) 0x7f3a900840b0 con 0x7f3a980cd050 2024-09-21T07:36:12.013 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-21T07:36:12.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.014+0000 7f3a9ec00640 1 -- 172.21.15.39:0/1487621901 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3a78078bc0 msgr2=0x7f3a7807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:12.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.014+0000 7f3a9ec00640 1 --2- 172.21.15.39:0/1487621901 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3a78078bc0 0x7f3a7807b0d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3a980cd630 tx=0x7f3a88009040 comp rx=0 tx=0).stop 2024-09-21T07:36:12.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.014+0000 7f3a9ec00640 1 -- 172.21.15.39:0/1487621901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a980cd050 msgr2=0x7f3a980b0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:12.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.014+0000 7f3a9ec00640 1 --2- 172.21.15.39:0/1487621901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a980cd050 0x7f3a980b0210 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3a90009bf0 tx=0x7f3a90003ce0 comp rx=0 tx=0).stop 2024-09-21T07:36:12.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.014+0000 7f3a9ec00640 1 -- 172.21.15.39:0/1487621901 shutdown_connections 2024-09-21T07:36:12.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.014+0000 7f3a9ec00640 1 --2- 172.21.15.39:0/1487621901 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3a78078bc0 0x7f3a7807b0d0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:12.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.014+0000 7f3a9ec00640 1 --2- 172.21.15.39:0/1487621901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a980cd050 0x7f3a980b0210 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:12.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.014+0000 7f3a9ec00640 1 --2- 172.21.15.39:0/1487621901 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3a9806f2a0 0x7f3a98074830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:12.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.014+0000 7f3a9ec00640 1 --2- 172.21.15.39:0/1487621901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a9806ee90 0x7f3a9814eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:12.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.014+0000 7f3a9ec00640 1 -- 172.21.15.39:0/1487621901 >> 172.21.15.39:0/1487621901 conn(0x7f3a9806d7d0 msgr2=0x7f3a9815cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:12.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.014+0000 7f3a9ec00640 1 -- 172.21.15.39:0/1487621901 shutdown_connections 2024-09-21T07:36:12.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.014+0000 7f3a9ec00640 1 -- 172.21.15.39:0/1487621901 wait complete. 2024-09-21T07:36:12.038 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2024-09-21T07:36:12.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.266+0000 7f12a2c00640 1 -- 172.21.15.39:0/2139300939 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f129c161f30 msgr2=0x7f129c0b5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:12.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.266+0000 7f12a2c00640 1 --2- 172.21.15.39:0/2139300939 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f129c161f30 0x7f129c0b5530 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f129c08d7d0 tx=0x7f129002a560 comp rx=0 tx=0).stop 2024-09-21T07:36:12.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.266+0000 7f12a2c00640 1 -- 172.21.15.39:0/2139300939 shutdown_connections 2024-09-21T07:36:12.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.266+0000 7f12a2c00640 1 --2- 172.21.15.39:0/2139300939 >> v2:172.21.15.148:3300/0 conn(0x7f129c0d2b70 0x7f129c0b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:12.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.266+0000 7f12a2c00640 1 --2- 172.21.15.39:0/2139300939 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f129c0d4de0 0x7f129c0d2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:12.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.266+0000 7f12a2c00640 1 --2- 172.21.15.39:0/2139300939 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f129c161f30 0x7f129c0b5530 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:12.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.266+0000 7f12a2c00640 1 -- 172.21.15.39:0/2139300939 >> 172.21.15.39:0/2139300939 conn(0x7f129c09d6a0 msgr2=0x7f129c0d4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:12.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.266+0000 7f12a2c00640 1 -- 172.21.15.39:0/2139300939 shutdown_connections 2024-09-21T07:36:12.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.267+0000 7f12a2c00640 1 -- 172.21.15.39:0/2139300939 wait complete. 2024-09-21T07:36:12.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.269+0000 7f12a2c00640 1 Processor -- start 2024-09-21T07:36:12.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.269+0000 7f12a2c00640 1 -- start start 2024-09-21T07:36:12.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.270+0000 7f12a2c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f129c0d2b70 0x7f129c15d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:12.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.270+0000 7f12a2c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f129c0d4de0 0x7f129c0d04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:12.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.270+0000 7f12a2c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f129c161f30 0x7f129c0d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:12.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.270+0000 7f12a2c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f129c0b43e0 con 0x7f129c161f30 2024-09-21T07:36:12.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.270+0000 7f12a2c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f129c0b4260 con 0x7f129c0d4de0 2024-09-21T07:36:12.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.270+0000 7f12a2c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f129c0b4560 con 0x7f129c0d2b70 2024-09-21T07:36:12.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.271+0000 7f12a2200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f129c161f30 0x7f129c0d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:12.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.271+0000 7f12a2200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f129c161f30 0x7f129c0d0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52574/0 (socket says 172.21.15.39:52574) 2024-09-21T07:36:12.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.271+0000 7f12a2200640 1 -- 172.21.15.39:0/1076771813 learned_addr learned my addr 172.21.15.39:0/1076771813 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:12.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.271+0000 7f12a0e00640 1 --2- 172.21.15.39:0/1076771813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f129c0d4de0 0x7f129c0d04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:12.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.271+0000 7f12a1800640 1 --2- 172.21.15.39:0/1076771813 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f129c0d2b70 0x7f129c15d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:12.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.271+0000 7f12a2200640 1 -- 172.21.15.39:0/1076771813 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f129c0d2b70 msgr2=0x7f129c15d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:12.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.271+0000 7f12a2200640 1 --2- 172.21.15.39:0/1076771813 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f129c0d2b70 0x7f129c15d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:12.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.271+0000 7f12a2200640 1 -- 172.21.15.39:0/1076771813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f129c0d4de0 msgr2=0x7f129c0d04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:12.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.271+0000 7f12a2200640 1 --2- 172.21.15.39:0/1076771813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f129c0d4de0 0x7f129c0d04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:12.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.271+0000 7f12a2200640 1 -- 172.21.15.39:0/1076771813 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f129c0c8a30 con 0x7f129c161f30 2024-09-21T07:36:12.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.271+0000 7f12a0e00640 1 --2- 172.21.15.39:0/1076771813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f129c0d4de0 0x7f129c0d04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:36:12.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.271+0000 7f12a2200640 1 --2- 172.21.15.39:0/1076771813 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f129c161f30 0x7f129c0d0a10 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f129400d9b0 tx=0x7f129400de70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:12.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.272+0000 7f1299600640 1 -- 172.21.15.39:0/1076771813 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1294016070 con 0x7f129c161f30 2024-09-21T07:36:12.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.272+0000 7f1299600640 1 -- 172.21.15.39:0/1076771813 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f129400bd00 con 0x7f129c161f30 2024-09-21T07:36:12.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.272+0000 7f1299600640 1 -- 172.21.15.39:0/1076771813 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12940106c0 con 0x7f129c161f30 2024-09-21T07:36:12.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.272+0000 7f12a2c00640 1 -- 172.21.15.39:0/1076771813 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f129c0c8d10 con 0x7f129c161f30 2024-09-21T07:36:12.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.272+0000 7f12a2c00640 1 -- 172.21.15.39:0/1076771813 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f129c0c82e0 con 0x7f129c161f30 2024-09-21T07:36:12.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.273+0000 7f12a2c00640 1 -- 172.21.15.39:0/1076771813 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f129c0c8f10 con 0x7f129c161f30 2024-09-21T07:36:12.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.274+0000 7f1299600640 1 -- 172.21.15.39:0/1076771813 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1294019070 con 0x7f129c161f30 2024-09-21T07:36:12.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.275+0000 7f1299600640 1 --2- 172.21.15.39:0/1076771813 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f127c078b90 0x7f127c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:12.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.275+0000 7f1299600640 1 -- 172.21.15.39:0/1076771813 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6034+0+0 (secure 0 0 0) 0x7f129409b5b0 con 0x7f129c161f30 2024-09-21T07:36:12.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.276+0000 7f12a1800640 1 --2- 172.21.15.39:0/1076771813 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f127c078b90 0x7f127c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:12.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.276+0000 7f12a1800640 1 --2- 172.21.15.39:0/1076771813 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f127c078b90 0x7f127c07b0a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1290033000 tx=0x7f1290004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:12.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.280+0000 7f1299600640 1 -- 172.21.15.39:0/1076771813 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f129c0c8f10 con 0x7f129c161f30 2024-09-21T07:36:12.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.477+0000 7f12a2c00640 1 -- 172.21.15.39:0/1076771813 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) -- 0x7f129c0d2630 con 0x7f129c161f30 2024-09-21T07:36:12.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.477+0000 7f1299600640 1 -- 172.21.15.39:0/1076771813 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v82) ==== 103+0+11 (secure 0 0 0) 0x7f12940638a0 con 0x7f129c161f30 2024-09-21T07:36:12.479 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-21T07:36:12.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.480+0000 7f12a2c00640 1 -- 172.21.15.39:0/1076771813 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f127c078b90 msgr2=0x7f127c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:12.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.480+0000 7f12a2c00640 1 --2- 172.21.15.39:0/1076771813 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f127c078b90 0x7f127c07b0a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1290033000 tx=0x7f1290004730 comp rx=0 tx=0).stop 2024-09-21T07:36:12.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.480+0000 7f12a2c00640 1 -- 172.21.15.39:0/1076771813 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f129c161f30 msgr2=0x7f129c0d0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:12.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.480+0000 7f12a2c00640 1 --2- 172.21.15.39:0/1076771813 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f129c161f30 0x7f129c0d0a10 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f129400d9b0 tx=0x7f129400de70 comp rx=0 tx=0).stop 2024-09-21T07:36:12.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.480+0000 7f12a2c00640 1 -- 172.21.15.39:0/1076771813 shutdown_connections 2024-09-21T07:36:12.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.480+0000 7f12a2c00640 1 --2- 172.21.15.39:0/1076771813 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f127c078b90 0x7f127c07b0a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:12.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.480+0000 7f12a2c00640 1 --2- 172.21.15.39:0/1076771813 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f129c161f30 0x7f129c0d0a10 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:12.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.480+0000 7f12a2c00640 1 --2- 172.21.15.39:0/1076771813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f129c0d4de0 0x7f129c0d04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:12.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.480+0000 7f12a2c00640 1 --2- 172.21.15.39:0/1076771813 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f129c0d2b70 0x7f129c15d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:12.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.480+0000 7f12a2c00640 1 -- 172.21.15.39:0/1076771813 >> 172.21.15.39:0/1076771813 conn(0x7f129c09d6a0 msgr2=0x7f129c0b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:12.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.480+0000 7f12a2c00640 1 -- 172.21.15.39:0/1076771813 shutdown_connections 2024-09-21T07:36:12.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.480+0000 7f12a2c00640 1 -- 172.21.15.39:0/1076771813 wait complete. 2024-09-21T07:36:12.503 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2024-09-21T07:36:12.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.731+0000 7fba31c00640 1 -- 172.21.15.39:0/622441221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba2c1535c0 msgr2=0x7fba2c15ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:12.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.731+0000 7fba31c00640 1 --2- 172.21.15.39:0/622441221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba2c1535c0 0x7fba2c15ca40 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fba1c009830 tx=0x7fba1c02a480 comp rx=0 tx=0).stop 2024-09-21T07:36:12.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.731+0000 7fba31c00640 1 -- 172.21.15.39:0/622441221 shutdown_connections 2024-09-21T07:36:12.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.731+0000 7fba31c00640 1 --2- 172.21.15.39:0/622441221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba2c1535c0 0x7fba2c15ca40 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:12.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.731+0000 7fba31c00640 1 -- 172.21.15.39:0/622441221 >> 172.21.15.39:0/622441221 conn(0x7fba2c0cd940 msgr2=0x7fba2c0d0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:12.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.731+0000 7fba31c00640 1 -- 172.21.15.39:0/622441221 shutdown_connections 2024-09-21T07:36:12.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.731+0000 7fba31c00640 1 -- 172.21.15.39:0/622441221 wait complete. 2024-09-21T07:36:12.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.734+0000 7fba31c00640 1 Processor -- start 2024-09-21T07:36:12.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.734+0000 7fba31c00640 1 -- start start 2024-09-21T07:36:12.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.735+0000 7fba31c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fba2c0b38c0 0x7fba2c141dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:12.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.735+0000 7fba31c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba2c0ce440 0x7fba2c142580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:12.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.735+0000 7fba31c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba2c1535c0 0x7fba2c152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:12.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.735+0000 7fba31c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fba2c0b5d80 con 0x7fba2c1535c0 2024-09-21T07:36:12.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.735+0000 7fba31c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fba2c0b5c00 con 0x7fba2c0ce440 2024-09-21T07:36:12.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.735+0000 7fba31c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fba2c0b5f00 con 0x7fba2c0b38c0 2024-09-21T07:36:12.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.735+0000 7fba31200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba2c1535c0 0x7fba2c152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:12.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.736+0000 7fba31200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba2c1535c0 0x7fba2c152220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52582/0 (socket says 172.21.15.39:52582) 2024-09-21T07:36:12.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.736+0000 7fba31200640 1 -- 172.21.15.39:0/3833519576 learned_addr learned my addr 172.21.15.39:0/3833519576 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:12.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.736+0000 7fba23e00640 1 --2- 172.21.15.39:0/3833519576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba2c0ce440 0x7fba2c142580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:12.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.736+0000 7fba2be00640 1 --2- 172.21.15.39:0/3833519576 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fba2c0b38c0 0x7fba2c141dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:12.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.736+0000 7fba31200640 1 -- 172.21.15.39:0/3833519576 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fba2c0b38c0 msgr2=0x7fba2c141dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:12.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.736+0000 7fba31200640 1 --2- 172.21.15.39:0/3833519576 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fba2c0b38c0 0x7fba2c141dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:12.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.736+0000 7fba31200640 1 -- 172.21.15.39:0/3833519576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba2c0ce440 msgr2=0x7fba2c142580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:12.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.736+0000 7fba31200640 1 --2- 172.21.15.39:0/3833519576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba2c0ce440 0x7fba2c142580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:12.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.736+0000 7fba31200640 1 -- 172.21.15.39:0/3833519576 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba2c153ea0 con 0x7fba2c1535c0 2024-09-21T07:36:12.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.736+0000 7fba31200640 1 --2- 172.21.15.39:0/3833519576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba2c1535c0 0x7fba2c152220 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fba2400b9e0 tx=0x7fba2400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:12.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.737+0000 7fba28c00640 1 -- 172.21.15.39:0/3833519576 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba24003b70 con 0x7fba2c1535c0 2024-09-21T07:36:12.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.737+0000 7fba28c00640 1 -- 172.21.15.39:0/3833519576 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba24003e00 con 0x7fba2c1535c0 2024-09-21T07:36:12.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.737+0000 7fba31c00640 1 -- 172.21.15.39:0/3833519576 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba2c154180 con 0x7fba2c1535c0 2024-09-21T07:36:12.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.737+0000 7fba28c00640 1 -- 172.21.15.39:0/3833519576 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2400cb80 con 0x7fba2c1535c0 2024-09-21T07:36:12.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.737+0000 7fba31c00640 1 -- 172.21.15.39:0/3833519576 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba2c151a20 con 0x7fba2c1535c0 2024-09-21T07:36:12.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.738+0000 7fba31c00640 1 -- 172.21.15.39:0/3833519576 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba2c156d60 con 0x7fba2c1535c0 2024-09-21T07:36:12.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.739+0000 7fba28c00640 1 -- 172.21.15.39:0/3833519576 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fba24004000 con 0x7fba2c1535c0 2024-09-21T07:36:12.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.740+0000 7fba28c00640 1 --2- 172.21.15.39:0/3833519576 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fba08078b90 0x7fba0807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:12.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.740+0000 7fba28c00640 1 -- 172.21.15.39:0/3833519576 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6034+0+0 (secure 0 0 0) 0x7fba2409adb0 con 0x7fba2c1535c0 2024-09-21T07:36:12.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.740+0000 7fba2be00640 1 --2- 172.21.15.39:0/3833519576 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fba08078b90 0x7fba0807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:12.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.741+0000 7fba2be00640 1 --2- 172.21.15.39:0/3833519576 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fba08078b90 0x7fba0807b0a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fba1c034000 tx=0x7fba1c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:12.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.744+0000 7fba28c00640 1 -- 172.21.15.39:0/3833519576 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fba2409f070 con 0x7fba2c1535c0 2024-09-21T07:36:12.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.939+0000 7fba31c00640 1 -- 172.21.15.39:0/3833519576 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) -- 0x7fba2c15ca40 con 0x7fba2c1535c0 2024-09-21T07:36:12.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.939+0000 7fba28c00640 1 -- 172.21.15.39:0/3833519576 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v82) ==== 107+0+11 (secure 0 0 0) 0x7fba24063000 con 0x7fba2c1535c0 2024-09-21T07:36:12.941 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-21T07:36:12.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.942+0000 7fba31c00640 1 -- 172.21.15.39:0/3833519576 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fba08078b90 msgr2=0x7fba0807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:12.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.942+0000 7fba31c00640 1 --2- 172.21.15.39:0/3833519576 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fba08078b90 0x7fba0807b0a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fba1c034000 tx=0x7fba1c004730 comp rx=0 tx=0).stop 2024-09-21T07:36:12.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.942+0000 7fba31c00640 1 -- 172.21.15.39:0/3833519576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba2c1535c0 msgr2=0x7fba2c152220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:12.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.942+0000 7fba31c00640 1 --2- 172.21.15.39:0/3833519576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba2c1535c0 0x7fba2c152220 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fba2400b9e0 tx=0x7fba2400bea0 comp rx=0 tx=0).stop 2024-09-21T07:36:12.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.942+0000 7fba31c00640 1 -- 172.21.15.39:0/3833519576 shutdown_connections 2024-09-21T07:36:12.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.942+0000 7fba31c00640 1 --2- 172.21.15.39:0/3833519576 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fba08078b90 0x7fba0807b0a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:12.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.942+0000 7fba31c00640 1 --2- 172.21.15.39:0/3833519576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba2c1535c0 0x7fba2c152220 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:12.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.942+0000 7fba31c00640 1 --2- 172.21.15.39:0/3833519576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba2c0ce440 0x7fba2c142580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:12.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.942+0000 7fba31c00640 1 --2- 172.21.15.39:0/3833519576 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fba2c0b38c0 0x7fba2c141dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:12.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.942+0000 7fba31c00640 1 -- 172.21.15.39:0/3833519576 >> 172.21.15.39:0/3833519576 conn(0x7fba2c0cd940 msgr2=0x7fba2c15db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:12.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.942+0000 7fba31c00640 1 -- 172.21.15.39:0/3833519576 shutdown_connections 2024-09-21T07:36:12.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:12.942+0000 7fba31c00640 1 -- 172.21.15.39:0/3833519576 wait complete. 2024-09-21T07:36:12.970 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_kill_mdstable' 2024-09-21T07:36:13.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.195+0000 7fbd5e200640 1 -- 172.21.15.39:0/1870450749 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd580cfcc0 msgr2=0x7fbd580d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:13.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.195+0000 7fbd5e200640 1 --2- 172.21.15.39:0/1870450749 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd580cfcc0 0x7fbd580d00d0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fbd44005fd0 tx=0x7fbd4402a530 comp rx=0 tx=0).stop 2024-09-21T07:36:13.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.195+0000 7fbd5e200640 1 -- 172.21.15.39:0/1870450749 shutdown_connections 2024-09-21T07:36:13.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.195+0000 7fbd5e200640 1 --2- 172.21.15.39:0/1870450749 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd580cfcc0 0x7fbd580d00d0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:13.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.195+0000 7fbd5e200640 1 --2- 172.21.15.39:0/1870450749 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbd580b6cd0 0x7fbd580b70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:13.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.195+0000 7fbd5e200640 1 -- 172.21.15.39:0/1870450749 >> 172.21.15.39:0/1870450749 conn(0x7fbd580cec00 msgr2=0x7fbd580cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:13.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.195+0000 7fbd5e200640 1 -- 172.21.15.39:0/1870450749 shutdown_connections 2024-09-21T07:36:13.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.195+0000 7fbd5e200640 1 -- 172.21.15.39:0/1870450749 wait complete. 2024-09-21T07:36:13.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.198+0000 7fbd5e200640 1 Processor -- start 2024-09-21T07:36:13.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.198+0000 7fbd5e200640 1 -- start start 2024-09-21T07:36:13.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.199+0000 7fbd5e200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbd580b5140 0x7fbd580c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:13.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.199+0000 7fbd5e200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd580b6cd0 0x7fbd5806fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:13.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.199+0000 7fbd5e200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd580cfcc0 0x7fbd58156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:13.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.199+0000 7fbd5e200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbd580b5c60 con 0x7fbd580cfcc0 2024-09-21T07:36:13.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.199+0000 7fbd5e200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbd580ce1a0 con 0x7fbd580b6cd0 2024-09-21T07:36:13.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.199+0000 7fbd5e200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fbd580b5de0 con 0x7fbd580b5140 2024-09-21T07:36:13.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.200+0000 7fbd5ce00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbd580b5140 0x7fbd580c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:13.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.200+0000 7fbd5d800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd580cfcc0 0x7fbd58156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:13.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.200+0000 7fbd5ce00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbd580b5140 0x7fbd580c8c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.39:50700/0 (socket says 172.21.15.39:50700) 2024-09-21T07:36:13.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.200+0000 7fbd57e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd580b6cd0 0x7fbd5806fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:13.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.200+0000 7fbd5ce00640 1 -- 172.21.15.39:0/3287544245 learned_addr learned my addr 172.21.15.39:0/3287544245 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:13.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.200+0000 7fbd5d800640 1 -- 172.21.15.39:0/3287544245 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbd580b5140 msgr2=0x7fbd580c8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:13.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.200+0000 7fbd5d800640 1 --2- 172.21.15.39:0/3287544245 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbd580b5140 0x7fbd580c8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:13.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.200+0000 7fbd5d800640 1 -- 172.21.15.39:0/3287544245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd580b6cd0 msgr2=0x7fbd5806fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:13.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.200+0000 7fbd5d800640 1 --2- 172.21.15.39:0/3287544245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd580b6cd0 0x7fbd5806fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:13.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.200+0000 7fbd5d800640 1 -- 172.21.15.39:0/3287544245 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd580c81b0 con 0x7fbd580cfcc0 2024-09-21T07:36:13.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.201+0000 7fbd5d800640 1 --2- 172.21.15.39:0/3287544245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd580cfcc0 0x7fbd58156280 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000be00 tx=0x7fbd5000bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:13.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.201+0000 7fbd54c00640 1 -- 172.21.15.39:0/3287544245 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd50003970 con 0x7fbd580cfcc0 2024-09-21T07:36:13.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.201+0000 7fbd54c00640 1 -- 172.21.15.39:0/3287544245 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd50003c00 con 0x7fbd580cfcc0 2024-09-21T07:36:13.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.201+0000 7fbd54c00640 1 -- 172.21.15.39:0/3287544245 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd50004020 con 0x7fbd580cfcc0 2024-09-21T07:36:13.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.201+0000 7fbd5e200640 1 -- 172.21.15.39:0/3287544245 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd580c5d50 con 0x7fbd580cfcc0 2024-09-21T07:36:13.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.201+0000 7fbd5e200640 1 -- 172.21.15.39:0/3287544245 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd580c8700 con 0x7fbd580cfcc0 2024-09-21T07:36:13.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.202+0000 7fbd5e200640 1 -- 172.21.15.39:0/3287544245 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd580b6320 con 0x7fbd580cfcc0 2024-09-21T07:36:13.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.203+0000 7fbd54c00640 1 -- 172.21.15.39:0/3287544245 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbd500041c0 con 0x7fbd580cfcc0 2024-09-21T07:36:13.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.204+0000 7fbd54c00640 1 --2- 172.21.15.39:0/3287544245 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbd38078b90 0x7fbd3807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:13.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.205+0000 7fbd54c00640 1 -- 172.21.15.39:0/3287544245 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6034+0+0 (secure 0 0 0) 0x7fbd5009af10 con 0x7fbd580cfcc0 2024-09-21T07:36:13.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.205+0000 7fbd5ce00640 1 --2- 172.21.15.39:0/3287544245 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbd38078b90 0x7fbd3807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:13.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.206+0000 7fbd5ce00640 1 --2- 172.21.15.39:0/3287544245 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbd38078b90 0x7fbd3807b0a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c005860 tx=0x7fbd4c00b3c0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:13.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.209+0000 7fbd54c00640 1 -- 172.21.15.39:0/3287544245 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbd50062ba0 con 0x7fbd580cfcc0 2024-09-21T07:36:13.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.479+0000 7fbd5e200640 1 -- 172.21.15.39:0/3287544245 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_kill_mdstable"]} v 0) -- 0x7fbd580c5ae0 con 0x7fbd580cfcc0 2024-09-21T07:36:13.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.612+0000 7fbd54c00640 1 -- 172.21.15.39:0/3287544245 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_kill_mdstable"]}]=0 v189) ==== 142+0+0 (secure 0 0 0) 0x7fbd500675d0 con 0x7fbd580cfcc0 2024-09-21T07:36:13.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.614+0000 7fbd5e200640 1 -- 172.21.15.39:0/3287544245 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbd38078b90 msgr2=0x7fbd3807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:13.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.614+0000 7fbd5e200640 1 --2- 172.21.15.39:0/3287544245 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbd38078b90 0x7fbd3807b0a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c005860 tx=0x7fbd4c00b3c0 comp rx=0 tx=0).stop 2024-09-21T07:36:13.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.614+0000 7fbd5e200640 1 -- 172.21.15.39:0/3287544245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd580cfcc0 msgr2=0x7fbd58156280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:13.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.614+0000 7fbd5e200640 1 --2- 172.21.15.39:0/3287544245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd580cfcc0 0x7fbd58156280 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000be00 tx=0x7fbd5000bef0 comp rx=0 tx=0).stop 2024-09-21T07:36:13.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.614+0000 7fbd5e200640 1 -- 172.21.15.39:0/3287544245 shutdown_connections 2024-09-21T07:36:13.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.614+0000 7fbd5e200640 1 --2- 172.21.15.39:0/3287544245 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbd38078b90 0x7fbd3807b0a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:13.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.614+0000 7fbd5e200640 1 --2- 172.21.15.39:0/3287544245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd580cfcc0 0x7fbd58156280 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:13.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.614+0000 7fbd5e200640 1 --2- 172.21.15.39:0/3287544245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd580b6cd0 0x7fbd5806fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:13.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.614+0000 7fbd5e200640 1 --2- 172.21.15.39:0/3287544245 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbd580b5140 0x7fbd580c8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:13.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.614+0000 7fbd5e200640 1 -- 172.21.15.39:0/3287544245 >> 172.21.15.39:0/3287544245 conn(0x7fbd580cec00 msgr2=0x7fbd580af980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:13.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.615+0000 7fbd5e200640 1 -- 172.21.15.39:0/3287544245 shutdown_connections 2024-09-21T07:36:13.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.615+0000 7fbd5e200640 1 -- 172.21.15.39:0/3287544245 wait complete. 2024-09-21T07:36:13.638 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T07:36:13.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.863+0000 7f95fc600640 1 -- 172.21.15.39:0/3666809161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95f4070030 msgr2=0x7f95f4074640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:13.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.863+0000 7f95fc600640 1 --2- 172.21.15.39:0/3666809161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95f4070030 0x7f95f4074640 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f95e0005fd0 tx=0x7f95e002a5d0 comp rx=0 tx=0).stop 2024-09-21T07:36:13.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.863+0000 7f95fc600640 1 -- 172.21.15.39:0/3666809161 shutdown_connections 2024-09-21T07:36:13.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.863+0000 7f95fc600640 1 --2- 172.21.15.39:0/3666809161 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95f4070440 0x7f95f4074b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:13.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.863+0000 7f95fc600640 1 --2- 172.21.15.39:0/3666809161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95f4070030 0x7f95f4074640 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:13.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.863+0000 7f95fc600640 1 -- 172.21.15.39:0/3666809161 >> 172.21.15.39:0/3666809161 conn(0x7f95f40b1cb0 msgr2=0x7f95f40b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:13.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.863+0000 7f95fc600640 1 -- 172.21.15.39:0/3666809161 shutdown_connections 2024-09-21T07:36:13.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.863+0000 7f95fc600640 1 -- 172.21.15.39:0/3666809161 wait complete. 2024-09-21T07:36:13.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.865+0000 7f95fc600640 1 Processor -- start 2024-09-21T07:36:13.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.865+0000 7f95fc600640 1 -- start start 2024-09-21T07:36:13.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.867+0000 7f95fc600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95f4070030 0x7f95f414b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:13.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.867+0000 7f95fc600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95f4070440 0x7f95f4148180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:13.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.867+0000 7f95fc600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95f4073170 0x7f95f40d0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:13.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.867+0000 7f95fc600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f95f40b3a40 con 0x7f95f4070440 2024-09-21T07:36:13.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.867+0000 7f95fc600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f95f40b38c0 con 0x7f95f4070030 2024-09-21T07:36:13.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.867+0000 7f95fc600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f95f40b3bc0 con 0x7f95f4073170 2024-09-21T07:36:13.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.867+0000 7f95f9200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95f4070440 0x7f95f4148180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:13.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.867+0000 7f95f9200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95f4070440 0x7f95f4148180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52600/0 (socket says 172.21.15.39:52600) 2024-09-21T07:36:13.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.867+0000 7f95fa600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95f4073170 0x7f95f40d0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:13.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.867+0000 7f95f9200640 1 -- 172.21.15.39:0/2158946119 learned_addr learned my addr 172.21.15.39:0/2158946119 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:13.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.867+0000 7f95f9c00640 1 --2- 172.21.15.39:0/2158946119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95f4070030 0x7f95f414b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:13.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.868+0000 7f95f9200640 1 -- 172.21.15.39:0/2158946119 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95f4073170 msgr2=0x7f95f40d0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:13.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.868+0000 7f95f9200640 1 --2- 172.21.15.39:0/2158946119 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95f4073170 0x7f95f40d0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:13.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.868+0000 7f95f9200640 1 -- 172.21.15.39:0/2158946119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95f4070030 msgr2=0x7f95f414b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:13.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.868+0000 7f95f9200640 1 --2- 172.21.15.39:0/2158946119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95f4070030 0x7f95f414b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:13.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.868+0000 7f95f9200640 1 -- 172.21.15.39:0/2158946119 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f414baa0 con 0x7f95f4070440 2024-09-21T07:36:13.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.868+0000 7f95fa600640 1 --2- 172.21.15.39:0/2158946119 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95f4073170 0x7f95f40d0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:36:13.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.868+0000 7f95f9c00640 1 --2- 172.21.15.39:0/2158946119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95f4070030 0x7f95f414b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:36:13.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.868+0000 7f95f9200640 1 --2- 172.21.15.39:0/2158946119 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95f4070440 0x7f95f4148180 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f95e0034040 tx=0x7f95e00035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:13.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.868+0000 7f95f1600640 1 -- 172.21.15.39:0/2158946119 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e003e030 con 0x7f95f4070440 2024-09-21T07:36:13.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.868+0000 7f95f1600640 1 -- 172.21.15.39:0/2158946119 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95e0003ae0 con 0x7f95f4070440 2024-09-21T07:36:13.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.869+0000 7f95f1600640 1 -- 172.21.15.39:0/2158946119 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e0035040 con 0x7f95f4070440 2024-09-21T07:36:13.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.869+0000 7f95fc600640 1 -- 172.21.15.39:0/2158946119 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95f40cf2c0 con 0x7f95f4070440 2024-09-21T07:36:13.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.869+0000 7f95fc600640 1 -- 172.21.15.39:0/2158946119 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95f40cf880 con 0x7f95f4070440 2024-09-21T07:36:13.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.870+0000 7f95fc600640 1 -- 172.21.15.39:0/2158946119 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95f40c8490 con 0x7f95f4070440 2024-09-21T07:36:13.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.871+0000 7f95f1600640 1 -- 172.21.15.39:0/2158946119 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f95e0041030 con 0x7f95f4070440 2024-09-21T07:36:13.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.872+0000 7f95f1600640 1 --2- 172.21.15.39:0/2158946119 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f95d4078b90 0x7f95d407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:13.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.872+0000 7f95f1600640 1 -- 172.21.15.39:0/2158946119 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6034+0+0 (secure 0 0 0) 0x7f95e00bb1e0 con 0x7f95f4070440 2024-09-21T07:36:13.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.872+0000 7f95f9c00640 1 --2- 172.21.15.39:0/2158946119 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f95d4078b90 0x7f95d407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:13.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.873+0000 7f95f9c00640 1 --2- 172.21.15.39:0/2158946119 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f95d4078b90 0x7f95d407b0a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f95e8001b90 tx=0x7f95e8009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:13.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:13.876+0000 7f95f1600640 1 -- 172.21.15.39:0/2158946119 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f95e0082f00 con 0x7f95f4070440 2024-09-21T07:36:14.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.074+0000 7f95fc600640 1 -- 172.21.15.39:0/2158946119 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f95f40b40e0 con 0x7f95f4070440 2024-09-21T07:36:14.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.074+0000 7f95f1600640 1 -- 172.21.15.39:0/2158946119 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v40) ==== 59+0+2294 (secure 0 0 0) 0x7f95e0087930 con 0x7f95f4070440 2024-09-21T07:36:14.076 INFO:teuthology.orchestra.run.smithi039.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T07:36:14.076 INFO:teuthology.orchestra.run.smithi039.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T07:36:14.077 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced client_mount_timeout 30 2024-09-21T07:36:14.077 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_client 20/20 2024-09-21T07:36:14.077 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_mgr 20/20 2024-09-21T07:36:14.077 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_ms 1/1 2024-09-21T07:36:14.077 INFO:teuthology.orchestra.run.smithi039.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T07:36:14.077 INFO:teuthology.orchestra.run.smithi039.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T07:36:14.077 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3421.881759 2024-09-21T07:36:14.077 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3520.252978 2024-09-21T07:36:14.077 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3511.183316 2024-09-21T07:36:14.077 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3829.631847 2024-09-21T07:36:14.077 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 5474.545872 2024-09-21T07:36:14.077 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4013.628559 2024-09-21T07:36:14.078 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4230.187288 2024-09-21T07:36:14.078 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4148.983107 2024-09-21T07:36:14.078 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_mds 20/20 2024-09-21T07:36:14.078 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T07:36:14.078 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_ms 2/2 2024-09-21T07:36:14.078 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_debug_frag true 2024-09-21T07:36:14.078 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_debug_scatterstat true 2024-09-21T07:36:14.078 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T07:36:14.078 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_verify_scatter true 2024-09-21T07:36:14.078 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T07:36:14.078 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T07:36:14.078 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T07:36:14.078 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced client_mount_timeout 600 2024-09-21T07:36:14.079 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced debug_client 20/20 2024-09-21T07:36:14.079 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced debug_ms 1/1 2024-09-21T07:36:14.079 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T07:36:14.079 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T07:36:14.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.077+0000 7f95fc600640 1 -- 172.21.15.39:0/2158946119 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f95d4078b90 msgr2=0x7f95d407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:14.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.077+0000 7f95fc600640 1 --2- 172.21.15.39:0/2158946119 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f95d4078b90 0x7f95d407b0a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f95e8001b90 tx=0x7f95e8009040 comp rx=0 tx=0).stop 2024-09-21T07:36:14.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.077+0000 7f95fc600640 1 -- 172.21.15.39:0/2158946119 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95f4070440 msgr2=0x7f95f4148180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:14.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.077+0000 7f95fc600640 1 --2- 172.21.15.39:0/2158946119 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95f4070440 0x7f95f4148180 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f95e0034040 tx=0x7f95e00035f0 comp rx=0 tx=0).stop 2024-09-21T07:36:14.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.077+0000 7f95fc600640 1 -- 172.21.15.39:0/2158946119 shutdown_connections 2024-09-21T07:36:14.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.077+0000 7f95fc600640 1 --2- 172.21.15.39:0/2158946119 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f95d4078b90 0x7f95d407b0a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:14.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.077+0000 7f95fc600640 1 --2- 172.21.15.39:0/2158946119 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95f4073170 0x7f95f40d0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:14.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.077+0000 7f95fc600640 1 --2- 172.21.15.39:0/2158946119 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95f4070440 0x7f95f4148180 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:14.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.077+0000 7f95fc600640 1 --2- 172.21.15.39:0/2158946119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95f4070030 0x7f95f414b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:14.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.077+0000 7f95fc600640 1 -- 172.21.15.39:0/2158946119 >> 172.21.15.39:0/2158946119 conn(0x7f95f40b1cb0 msgr2=0x7f95f40ae980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:14.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.078+0000 7f95fc600640 1 -- 172.21.15.39:0/2158946119 shutdown_connections 2024-09-21T07:36:14.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.078+0000 7f95fc600640 1 -- 172.21.15.39:0/2158946119 wait complete. 2024-09-21T07:36:14.102 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T07:36:14.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.330+0000 7ff512800640 1 -- 172.21.15.39:0/1522662270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff50c0cd3e0 msgr2=0x7ff50c070180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:14.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.330+0000 7ff512800640 1 --2- 172.21.15.39:0/1522662270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff50c0cd3e0 0x7ff50c070180 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff50c0ce0e0 tx=0x7ff4f402a4d0 comp rx=0 tx=0).stop 2024-09-21T07:36:14.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.330+0000 7ff512800640 1 -- 172.21.15.39:0/1522662270 shutdown_connections 2024-09-21T07:36:14.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.330+0000 7ff512800640 1 --2- 172.21.15.39:0/1522662270 >> v2:172.21.15.148:3300/0 conn(0x7ff50c0706c0 0x7ff50c072ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:14.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.330+0000 7ff512800640 1 --2- 172.21.15.39:0/1522662270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff50c0cd3e0 0x7ff50c070180 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:14.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.330+0000 7ff512800640 1 -- 172.21.15.39:0/1522662270 >> 172.21.15.39:0/1522662270 conn(0x7ff50c06d7d0 msgr2=0x7ff50c0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:14.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.330+0000 7ff512800640 1 -- 172.21.15.39:0/1522662270 shutdown_connections 2024-09-21T07:36:14.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.330+0000 7ff512800640 1 -- 172.21.15.39:0/1522662270 wait complete. 2024-09-21T07:36:14.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.332+0000 7ff512800640 1 Processor -- start 2024-09-21T07:36:14.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.332+0000 7ff512800640 1 -- start start 2024-09-21T07:36:14.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.334+0000 7ff512800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff50c0706c0 0x7ff50c15cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:14.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.334+0000 7ff512800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff50c0af780 0x7ff50c09cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:14.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.334+0000 7ff512800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff50c0cd3e0 0x7ff50c1497c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:14.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.334+0000 7ff512800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c073af0 con 0x7ff50c0706c0 2024-09-21T07:36:14.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.334+0000 7ff512800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c073970 con 0x7ff50c0af780 2024-09-21T07:36:14.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.334+0000 7ff512800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7ff50c073c70 con 0x7ff50c0cd3e0 2024-09-21T07:36:14.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.334+0000 7ff50b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff50c0706c0 0x7ff50c15cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:14.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.334+0000 7ff50b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff50c0706c0 0x7ff50c15cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52616/0 (socket says 172.21.15.39:52616) 2024-09-21T07:36:14.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.334+0000 7ff50b400640 1 -- 172.21.15.39:0/795152754 learned_addr learned my addr 172.21.15.39:0/795152754 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:14.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.334+0000 7ff503e00640 1 --2- 172.21.15.39:0/795152754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff50c0af780 0x7ff50c09cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:14.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.334+0000 7ff50be00640 1 --2- 172.21.15.39:0/795152754 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff50c0cd3e0 0x7ff50c1497c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:14.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.335+0000 7ff50b400640 1 -- 172.21.15.39:0/795152754 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff50c0cd3e0 msgr2=0x7ff50c1497c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:14.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.335+0000 7ff50b400640 1 --2- 172.21.15.39:0/795152754 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff50c0cd3e0 0x7ff50c1497c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:14.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.335+0000 7ff50b400640 1 -- 172.21.15.39:0/795152754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff50c0af780 msgr2=0x7ff50c09cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:14.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.335+0000 7ff50b400640 1 --2- 172.21.15.39:0/795152754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff50c0af780 0x7ff50c09cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:14.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.335+0000 7ff50b400640 1 -- 172.21.15.39:0/795152754 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff50c1e8890 con 0x7ff50c0706c0 2024-09-21T07:36:14.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.335+0000 7ff50be00640 1 --2- 172.21.15.39:0/795152754 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff50c0cd3e0 0x7ff50c1497c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:36:14.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.335+0000 7ff50b400640 1 --2- 172.21.15.39:0/795152754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff50c0706c0 0x7ff50c15cb50 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc00b3f0 tx=0x7ff4fc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:14.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.335+0000 7ff503400640 1 -- 172.21.15.39:0/795152754 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc015020 con 0x7ff50c0706c0 2024-09-21T07:36:14.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.335+0000 7ff503400640 1 -- 172.21.15.39:0/795152754 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4fc011920 con 0x7ff50c0706c0 2024-09-21T07:36:14.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.335+0000 7ff512800640 1 -- 172.21.15.39:0/795152754 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff50c1e6a50 con 0x7ff50c0706c0 2024-09-21T07:36:14.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.335+0000 7ff503400640 1 -- 172.21.15.39:0/795152754 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc011d70 con 0x7ff50c0706c0 2024-09-21T07:36:14.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.335+0000 7ff512800640 1 -- 172.21.15.39:0/795152754 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff50c1e7010 con 0x7ff50c0706c0 2024-09-21T07:36:14.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.336+0000 7ff512800640 1 -- 172.21.15.39:0/795152754 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff50c072ff0 con 0x7ff50c0706c0 2024-09-21T07:36:14.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.338+0000 7ff503400640 1 -- 172.21.15.39:0/795152754 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff4fc018020 con 0x7ff50c0706c0 2024-09-21T07:36:14.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.339+0000 7ff503400640 1 --2- 172.21.15.39:0/795152754 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff4e8078bc0 0x7ff4e807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:14.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.339+0000 7ff503400640 1 -- 172.21.15.39:0/795152754 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6034+0+0 (secure 0 0 0) 0x7ff4fc09acf0 con 0x7ff50c0706c0 2024-09-21T07:36:14.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.339+0000 7ff503e00640 1 --2- 172.21.15.39:0/795152754 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff4e8078bc0 0x7ff4e807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:14.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.340+0000 7ff503e00640 1 --2- 172.21.15.39:0/795152754 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff4e8078bc0 0x7ff4e807b0d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4033000 tx=0x7ff4f4004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:14.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.343+0000 7ff503400640 1 -- 172.21.15.39:0/795152754 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff4fc09f070 con 0x7ff50c0706c0 2024-09-21T07:36:14.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.540+0000 7ff512800640 1 -- 172.21.15.39:0/795152754 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7ff50c15e1f0 con 0x7ff50c0706c0 2024-09-21T07:36:14.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.545+0000 7ff503400640 1 -- 172.21.15.39:0/795152754 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v41)=0 v41) ==== 122+0+0 (secure 0 0 0) 0x7ff4fc01f020 con 0x7ff50c0706c0 2024-09-21T07:36:14.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.548+0000 7ff512800640 1 -- 172.21.15.39:0/795152754 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff4e8078bc0 msgr2=0x7ff4e807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:14.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.548+0000 7ff512800640 1 --2- 172.21.15.39:0/795152754 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff4e8078bc0 0x7ff4e807b0d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4033000 tx=0x7ff4f4004730 comp rx=0 tx=0).stop 2024-09-21T07:36:14.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.548+0000 7ff512800640 1 -- 172.21.15.39:0/795152754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff50c0706c0 msgr2=0x7ff50c15cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:14.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.548+0000 7ff512800640 1 --2- 172.21.15.39:0/795152754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff50c0706c0 0x7ff50c15cb50 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc00b3f0 tx=0x7ff4fc00b8b0 comp rx=0 tx=0).stop 2024-09-21T07:36:14.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.548+0000 7ff512800640 1 -- 172.21.15.39:0/795152754 shutdown_connections 2024-09-21T07:36:14.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.548+0000 7ff512800640 1 --2- 172.21.15.39:0/795152754 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff4e8078bc0 0x7ff4e807b0d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:14.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.548+0000 7ff512800640 1 --2- 172.21.15.39:0/795152754 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff50c0cd3e0 0x7ff50c1497c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:14.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.548+0000 7ff512800640 1 --2- 172.21.15.39:0/795152754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff50c0af780 0x7ff50c09cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:14.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.548+0000 7ff512800640 1 --2- 172.21.15.39:0/795152754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff50c0706c0 0x7ff50c15cb50 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:14.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.548+0000 7ff512800640 1 -- 172.21.15.39:0/795152754 >> 172.21.15.39:0/795152754 conn(0x7ff50c06d7d0 msgr2=0x7ff50c073e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:14.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.548+0000 7ff512800640 1 -- 172.21.15.39:0/795152754 shutdown_connections 2024-09-21T07:36:14.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:14.548+0000 7ff512800640 1 -- 172.21.15.39:0/795152754 wait complete. 2024-09-21T07:36:14.575 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:36:14.575 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:36:14.662 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:36:14.754 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:36:14.755 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:36:14.755 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:36:14.790 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:36:14.790 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:36:14.790 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:36:14.826 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:36:14.826 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:36:14.826 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:36:14.864 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:36:14.865 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:36:14.865 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:36:14.903 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:36:14.903 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T07:36:14.904 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:36:15.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.127+0000 7f76bce00640 1 -- 172.21.15.39:0/1390730228 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76b80ccb50 msgr2=0x7f76b80ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:15.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.127+0000 7f76bce00640 1 --2- 172.21.15.39:0/1390730228 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76b80ccb50 0x7f76b80ccf60 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f76b8071d60 tx=0x7f76b002aa30 comp rx=0 tx=0).stop 2024-09-21T07:36:15.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.127+0000 7f76bce00640 1 -- 172.21.15.39:0/1390730228 shutdown_connections 2024-09-21T07:36:15.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.127+0000 7f76bce00640 1 --2- 172.21.15.39:0/1390730228 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76b80ccb50 0x7f76b80ccf60 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:15.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.127+0000 7f76bce00640 1 --2- 172.21.15.39:0/1390730228 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f76b815d8a0 0x7f76b80b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:15.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.127+0000 7f76bce00640 1 --2- 172.21.15.39:0/1390730228 >> v2:172.21.15.148:3300/0 conn(0x7f76b80cc1e0 0x7f76b80af3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:15.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.128+0000 7f76bce00640 1 -- 172.21.15.39:0/1390730228 >> 172.21.15.39:0/1390730228 conn(0x7f76b806d7d0 msgr2=0x7f76b80b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:15.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.128+0000 7f76bce00640 1 -- 172.21.15.39:0/1390730228 shutdown_connections 2024-09-21T07:36:15.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.128+0000 7f76bce00640 1 -- 172.21.15.39:0/1390730228 wait complete. 2024-09-21T07:36:15.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.130+0000 7f76bce00640 1 Processor -- start 2024-09-21T07:36:15.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.130+0000 7f76bce00640 1 -- start start 2024-09-21T07:36:15.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.131+0000 7f76bce00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76b80cc1e0 0x7f76b81f63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:15.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.131+0000 7f76bce00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f76b80ccb50 0x7f76b80ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:15.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.131+0000 7f76bce00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76b815d8a0 0x7f76b81f8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:15.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.131+0000 7f76bce00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f76b80b4770 con 0x7f76b815d8a0 2024-09-21T07:36:15.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.131+0000 7f76bce00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f76b80b45f0 con 0x7f76b80cc1e0 2024-09-21T07:36:15.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.131+0000 7f76bce00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f76b81533a0 con 0x7f76b80ccb50 2024-09-21T07:36:15.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.132+0000 7f76b6800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76b815d8a0 0x7f76b81f8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:15.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.132+0000 7f76b6800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76b815d8a0 0x7f76b81f8f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52624/0 (socket says 172.21.15.39:52624) 2024-09-21T07:36:15.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.132+0000 7f76b6800640 1 -- 172.21.15.39:0/3179736277 learned_addr learned my addr 172.21.15.39:0/3179736277 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:15.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.132+0000 7f76b5e00640 1 --2- 172.21.15.39:0/3179736277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76b80cc1e0 0x7f76b81f63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:15.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.132+0000 7f76afe00640 1 --2- 172.21.15.39:0/3179736277 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f76b80ccb50 0x7f76b80ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:15.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.132+0000 7f76b6800640 1 -- 172.21.15.39:0/3179736277 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f76b80ccb50 msgr2=0x7f76b80ae3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:15.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.132+0000 7f76b6800640 1 --2- 172.21.15.39:0/3179736277 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f76b80ccb50 0x7f76b80ae3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:15.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.132+0000 7f76b6800640 1 -- 172.21.15.39:0/3179736277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76b80cc1e0 msgr2=0x7f76b81f63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:15.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.132+0000 7f76b6800640 1 --2- 172.21.15.39:0/3179736277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76b80cc1e0 0x7f76b81f63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:15.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.132+0000 7f76b6800640 1 -- 172.21.15.39:0/3179736277 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b80ae8f0 con 0x7f76b815d8a0 2024-09-21T07:36:15.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.133+0000 7f76b6800640 1 --2- 172.21.15.39:0/3179736277 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76b815d8a0 0x7f76b81f8f10 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f76b0009d60 tx=0x7f76b0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:15.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.133+0000 7f76ae000640 1 -- 172.21.15.39:0/3179736277 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b0033050 con 0x7f76b815d8a0 2024-09-21T07:36:15.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.133+0000 7f76ae000640 1 -- 172.21.15.39:0/3179736277 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76b0004450 con 0x7f76b815d8a0 2024-09-21T07:36:15.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.133+0000 7f76bce00640 1 -- 172.21.15.39:0/3179736277 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76b81f99b0 con 0x7f76b815d8a0 2024-09-21T07:36:15.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.133+0000 7f76ae000640 1 -- 172.21.15.39:0/3179736277 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b0038450 con 0x7f76b815d8a0 2024-09-21T07:36:15.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.133+0000 7f76bce00640 1 -- 172.21.15.39:0/3179736277 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76b81f5ba0 con 0x7f76b815d8a0 2024-09-21T07:36:15.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.134+0000 7f76bce00640 1 -- 172.21.15.39:0/3179736277 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76b80ccf60 con 0x7f76b815d8a0 2024-09-21T07:36:15.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.135+0000 7f76ae000640 1 -- 172.21.15.39:0/3179736277 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f76b0036070 con 0x7f76b815d8a0 2024-09-21T07:36:15.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.136+0000 7f76ae000640 1 --2- 172.21.15.39:0/3179736277 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7694078b90 0x7f769407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:15.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.136+0000 7f76ae000640 1 -- 172.21.15.39:0/3179736277 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6034+0+0 (secure 0 0 0) 0x7f76b00bc310 con 0x7f76b815d8a0 2024-09-21T07:36:15.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.137+0000 7f76b5e00640 1 --2- 172.21.15.39:0/3179736277 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7694078b90 0x7f769407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:15.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.137+0000 7f76b5e00640 1 --2- 172.21.15.39:0/3179736277 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7694078b90 0x7f769407b0a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f76a8005fd0 tx=0x7f76a800b260 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:15.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.141+0000 7f76ae000640 1 -- 172.21.15.39:0/3179736277 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f76b80ae8f0 con 0x7f76b815d8a0 2024-09-21T07:36:15.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.378+0000 7f76bce00640 1 -- 172.21.15.39:0/3179736277 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76b815b510 con 0x7f76b815d8a0 2024-09-21T07:36:15.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.378+0000 7f76ae000640 1 -- 172.21.15.39:0/3179736277 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+3911 (secure 0 0 0) 0x7f76b0083f10 con 0x7f76b815d8a0 2024-09-21T07:36:15.381 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:36:15.381 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":48,"btime":"2024-09-21T07:36:06:252155+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5426,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/707367131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":707367131},{"type":"v1","addr":"172.21.15.39:6835","nonce":707367131}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":5429,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/784956164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":784956164},{"type":"v1","addr":"172.21.15.148:6836","nonce":784956164}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5432,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3742990359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3742990359},{"type":"v1","addr":"172.21.15.148:6837","nonce":3742990359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6204,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2025602963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2025602963},{"type":"v1","addr":"172.21.15.39:6834","nonce":2025602963}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[]} 2024-09-21T07:36:15.381 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 48 2024-09-21T07:36:15.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.381+0000 7f76bce00640 1 -- 172.21.15.39:0/3179736277 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7694078b90 msgr2=0x7f769407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:15.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.381+0000 7f76bce00640 1 --2- 172.21.15.39:0/3179736277 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7694078b90 0x7f769407b0a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f76a8005fd0 tx=0x7f76a800b260 comp rx=0 tx=0).stop 2024-09-21T07:36:15.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.381+0000 7f76bce00640 1 -- 172.21.15.39:0/3179736277 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76b815d8a0 msgr2=0x7f76b81f8f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:15.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.381+0000 7f76bce00640 1 --2- 172.21.15.39:0/3179736277 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76b815d8a0 0x7f76b81f8f10 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f76b0009d60 tx=0x7f76b0003f10 comp rx=0 tx=0).stop 2024-09-21T07:36:15.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.381+0000 7f76bce00640 1 -- 172.21.15.39:0/3179736277 shutdown_connections 2024-09-21T07:36:15.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.381+0000 7f76bce00640 1 --2- 172.21.15.39:0/3179736277 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7694078b90 0x7f769407b0a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:15.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.381+0000 7f76bce00640 1 --2- 172.21.15.39:0/3179736277 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76b815d8a0 0x7f76b81f8f10 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:15.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.381+0000 7f76bce00640 1 --2- 172.21.15.39:0/3179736277 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f76b80ccb50 0x7f76b80ae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:15.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.381+0000 7f76bce00640 1 --2- 172.21.15.39:0/3179736277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76b80cc1e0 0x7f76b81f63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:15.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.381+0000 7f76bce00640 1 -- 172.21.15.39:0/3179736277 >> 172.21.15.39:0/3179736277 conn(0x7f76b806d7d0 msgr2=0x7f76b80cb6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:15.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.381+0000 7f76bce00640 1 -- 172.21.15.39:0/3179736277 shutdown_connections 2024-09-21T07:36:15.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.381+0000 7f76bce00640 1 -- 172.21.15.39:0/3179736277 wait complete. 2024-09-21T07:36:15.406 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T07:36:15.406 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T07:36:15.406 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T07:36:15.406 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:36:15.406 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T07:36:15.406 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T07:36:15.407 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T07:36:15.407 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:36:15.407 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T07:36:15.407 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T07:36:15.407 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T07:36:15.407 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:36:15.407 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T07:36:15.407 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T07:36:15.408 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T07:36:15.408 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:36:15.436 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T07:36:15.437 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T07:36:15.439 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T07:36:15.439 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T07:36:15.440 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T07:36:15.440 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T07:36:15.441 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T07:36:15.441 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T07:36:15.482 INFO:tasks.ceph.mds.a:Started 2024-09-21T07:36:15.482 DEBUG:teuthology.parallel:result is None 2024-09-21T07:36:15.484 INFO:tasks.ceph.mds.b:Started 2024-09-21T07:36:15.484 DEBUG:teuthology.parallel:result is None 2024-09-21T07:36:15.485 INFO:tasks.ceph.mds.c:Started 2024-09-21T07:36:15.485 DEBUG:teuthology.parallel:result is None 2024-09-21T07:36:15.487 INFO:tasks.ceph.mds.d:Started 2024-09-21T07:36:15.487 DEBUG:teuthology.parallel:result is None 2024-09-21T07:36:15.487 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T07:36:15.680 INFO:tasks.ceph.mds.b.smithi148.stdout:starting mds.b at 2024-09-21T07:36:15.683 INFO:tasks.ceph.mds.d.smithi148.stdout:starting mds.d at 2024-09-21T07:36:15.684 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-21T07:36:15.684 INFO:tasks.ceph.mds.c.smithi039.stdout:starting mds.c at 2024-09-21T07:36:15.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.723+0000 7f3dcc000640 1 -- 172.21.15.39:0/1618286460 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3dc40cfb90 msgr2=0x7f3dc40b2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:15.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.723+0000 7f3dcc000640 1 --2- 172.21.15.39:0/1618286460 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3dc40cfb90 0x7f3dc40b2fb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc005fd0 tx=0x7f3dbc02a820 comp rx=0 tx=0).stop 2024-09-21T07:36:15.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.723+0000 7f3dcc000640 1 -- 172.21.15.39:0/1618286460 shutdown_connections 2024-09-21T07:36:15.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.723+0000 7f3dcc000640 1 --2- 172.21.15.39:0/1618286460 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3dc40cfb90 0x7f3dc40b2fb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:15.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.723+0000 7f3dcc000640 1 --2- 172.21.15.39:0/1618286460 >> v2:172.21.15.148:3300/0 conn(0x7f3dc40cd240 0x7f3dc40cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:15.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.723+0000 7f3dcc000640 1 -- 172.21.15.39:0/1618286460 >> 172.21.15.39:0/1618286460 conn(0x7f3dc40b1cb0 msgr2=0x7f3dc40b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:15.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.723+0000 7f3dcc000640 1 -- 172.21.15.39:0/1618286460 shutdown_connections 2024-09-21T07:36:15.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.723+0000 7f3dcc000640 1 -- 172.21.15.39:0/1618286460 wait complete. 2024-09-21T07:36:15.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.725+0000 7f3dcc000640 1 Processor -- start 2024-09-21T07:36:15.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.725+0000 7f3dcc000640 1 -- start start 2024-09-21T07:36:15.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.727+0000 7f3dcc000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dc40cf660 0x7f3dc40adc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:15.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.727+0000 7f3dcc000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3dc40cfb90 0x7f3dc409cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:15.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.727+0000 7f3dcc000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dc40ae160 0x7f3dc4147120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:15.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.727+0000 7f3dcc000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc40710a0 con 0x7f3dc40ae160 2024-09-21T07:36:15.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.727+0000 7f3dcc000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc4070f20 con 0x7f3dc40cf660 2024-09-21T07:36:15.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.727+0000 7f3dcc000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f3dc40b34f0 con 0x7f3dc40cfb90 2024-09-21T07:36:15.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.727+0000 7f3dca000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dc40ae160 0x7f3dc4147120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:15.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.727+0000 7f3dca000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dc40ae160 0x7f3dc4147120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52656/0 (socket says 172.21.15.39:52656) 2024-09-21T07:36:15.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.727+0000 7f3dca000640 1 -- 172.21.15.39:0/1552464243 learned_addr learned my addr 172.21.15.39:0/1552464243 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:15.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.727+0000 7f3dc8c00640 1 --2- 172.21.15.39:0/1552464243 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3dc40cfb90 0x7f3dc409cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:15.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.727+0000 7f3dc9600640 1 --2- 172.21.15.39:0/1552464243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dc40cf660 0x7f3dc40adc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:15.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.727+0000 7f3dca000640 1 -- 172.21.15.39:0/1552464243 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3dc40cfb90 msgr2=0x7f3dc409cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:15.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.727+0000 7f3dca000640 1 --2- 172.21.15.39:0/1552464243 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3dc40cfb90 0x7f3dc409cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:15.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.727+0000 7f3dca000640 1 -- 172.21.15.39:0/1552464243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dc40cf660 msgr2=0x7f3dc40adc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:15.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.727+0000 7f3dca000640 1 --2- 172.21.15.39:0/1552464243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dc40cf660 0x7f3dc40adc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:15.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.727+0000 7f3dca000640 1 -- 172.21.15.39:0/1552464243 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc41f6060 con 0x7f3dc40ae160 2024-09-21T07:36:15.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.728+0000 7f3dca000640 1 --2- 172.21.15.39:0/1552464243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dc40ae160 0x7f3dc4147120 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc035040 tx=0x7f3dbc032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:15.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.728+0000 7f3dc1600640 1 -- 172.21.15.39:0/1552464243 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc037040 con 0x7f3dc40ae160 2024-09-21T07:36:15.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.728+0000 7f3dc1600640 1 -- 172.21.15.39:0/1552464243 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3dbc042070 con 0x7f3dc40ae160 2024-09-21T07:36:15.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.728+0000 7f3dc1600640 1 -- 172.21.15.39:0/1552464243 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc003e60 con 0x7f3dc40ae160 2024-09-21T07:36:15.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.728+0000 7f3dcc000640 1 -- 172.21.15.39:0/1552464243 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dc41f6340 con 0x7f3dc40ae160 2024-09-21T07:36:15.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.728+0000 7f3dcc000640 1 -- 172.21.15.39:0/1552464243 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dc41f6900 con 0x7f3dc40ae160 2024-09-21T07:36:15.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.729+0000 7f3dcc000640 1 -- 172.21.15.39:0/1552464243 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dc40b4b20 con 0x7f3dc40ae160 2024-09-21T07:36:15.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.731+0000 7f3dc1600640 1 -- 172.21.15.39:0/1552464243 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3dbc004260 con 0x7f3dc40ae160 2024-09-21T07:36:15.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.732+0000 7f3dc1600640 1 --2- 172.21.15.39:0/1552464243 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3da4078bc0 0x7f3da407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:15.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.732+0000 7f3dc1600640 1 -- 172.21.15.39:0/1552464243 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6034+0+0 (secure 0 0 0) 0x7f3dbc0bba00 con 0x7f3dc40ae160 2024-09-21T07:36:15.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.733+0000 7f3dc9600640 1 --2- 172.21.15.39:0/1552464243 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3da4078bc0 0x7f3da407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:15.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.733+0000 7f3dc9600640 1 --2- 172.21.15.39:0/1552464243 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3da4078bc0 0x7f3da407b0d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3db8007fd0 tx=0x7f3db8001370 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:15.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.737+0000 7f3dc1600640 1 -- 172.21.15.39:0/1552464243 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3dbc0836d0 con 0x7f3dc40ae160 2024-09-21T07:36:15.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:15.948+0000 7f3dcc000640 1 -- 172.21.15.39:0/1552464243 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f3dc409d760 con 0x7f3dc40ae160 2024-09-21T07:36:16.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.635+0000 7f3dc1600640 1 -- 172.21.15.39:0/1552464243 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v84) ==== 97+0+0 (secure 0 0 0) 0x7f3dbc088100 con 0x7f3dc40ae160 2024-09-21T07:36:16.638 INFO:teuthology.orchestra.run.smithi039.stderr: removed all blocklist entries 2024-09-21T07:36:16.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.639+0000 7f3dcc000640 1 -- 172.21.15.39:0/1552464243 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3da4078bc0 msgr2=0x7f3da407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:16.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.639+0000 7f3dcc000640 1 --2- 172.21.15.39:0/1552464243 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3da4078bc0 0x7f3da407b0d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3db8007fd0 tx=0x7f3db8001370 comp rx=0 tx=0).stop 2024-09-21T07:36:16.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.639+0000 7f3dcc000640 1 -- 172.21.15.39:0/1552464243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dc40ae160 msgr2=0x7f3dc4147120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:16.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.639+0000 7f3dcc000640 1 --2- 172.21.15.39:0/1552464243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dc40ae160 0x7f3dc4147120 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc035040 tx=0x7f3dbc032c60 comp rx=0 tx=0).stop 2024-09-21T07:36:16.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.639+0000 7f3dcc000640 1 -- 172.21.15.39:0/1552464243 shutdown_connections 2024-09-21T07:36:16.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.639+0000 7f3dcc000640 1 --2- 172.21.15.39:0/1552464243 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3da4078bc0 0x7f3da407b0d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:16.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.639+0000 7f3dcc000640 1 --2- 172.21.15.39:0/1552464243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dc40ae160 0x7f3dc4147120 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:16.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.639+0000 7f3dcc000640 1 --2- 172.21.15.39:0/1552464243 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3dc40cfb90 0x7f3dc409cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:16.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.639+0000 7f3dcc000640 1 --2- 172.21.15.39:0/1552464243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dc40cf660 0x7f3dc40adc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:16.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.639+0000 7f3dcc000640 1 -- 172.21.15.39:0/1552464243 >> 172.21.15.39:0/1552464243 conn(0x7f3dc40b1cb0 msgr2=0x7f3dc40b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:16.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.639+0000 7f3dcc000640 1 -- 172.21.15.39:0/1552464243 shutdown_connections 2024-09-21T07:36:16.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.639+0000 7f3dcc000640 1 -- 172.21.15.39:0/1552464243 wait complete. 2024-09-21T07:36:16.671 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T07:36:16.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.899+0000 7f7eeb000640 1 -- 172.21.15.39:0/137281583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ee40cf950 msgr2=0x7f7ee406f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:16.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.899+0000 7f7eeb000640 1 --2- 172.21.15.39:0/137281583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ee40cf950 0x7f7ee406f3b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7edc005fd0 tx=0x7f7edc02aa10 comp rx=0 tx=0).stop 2024-09-21T07:36:16.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.899+0000 7f7eeb000640 1 -- 172.21.15.39:0/137281583 shutdown_connections 2024-09-21T07:36:16.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.899+0000 7f7eeb000640 1 --2- 172.21.15.39:0/137281583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ee40cf950 0x7f7ee406f3b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:16.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.899+0000 7f7eeb000640 1 --2- 172.21.15.39:0/137281583 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7ee40cf540 0x7f7ee406ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:16.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.899+0000 7f7eeb000640 1 -- 172.21.15.39:0/137281583 >> 172.21.15.39:0/137281583 conn(0x7f7ee406d410 msgr2=0x7f7ee40b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:16.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.899+0000 7f7eeb000640 1 -- 172.21.15.39:0/137281583 shutdown_connections 2024-09-21T07:36:16.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.899+0000 7f7eeb000640 1 -- 172.21.15.39:0/137281583 wait complete. 2024-09-21T07:36:16.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.901+0000 7f7eeb000640 1 Processor -- start 2024-09-21T07:36:16.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.902+0000 7f7eeb000640 1 -- start start 2024-09-21T07:36:16.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.903+0000 7f7eeb000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7ee40cf540 0x7f7ee4153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:16.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.903+0000 7f7eeb000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ee40cf950 0x7f7ee40a9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:16.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.903+0000 7f7eeb000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ee415ca50 0x7f7ee40aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:16.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.903+0000 7f7eeb000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee415de20 con 0x7f7ee415ca50 2024-09-21T07:36:16.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.903+0000 7f7eeb000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee415dca0 con 0x7f7ee40cf950 2024-09-21T07:36:16.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.903+0000 7f7eeb000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f7ee415dfa0 con 0x7f7ee40cf540 2024-09-21T07:36:16.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.903+0000 7f7eea600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ee415ca50 0x7f7ee40aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:16.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.903+0000 7f7ee9c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7ee40cf540 0x7f7ee4153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:16.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.903+0000 7f7ee9200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ee40cf950 0x7f7ee40a9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:16.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.903+0000 7f7eea600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ee415ca50 0x7f7ee40aeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52658/0 (socket says 172.21.15.39:52658) 2024-09-21T07:36:16.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.903+0000 7f7eea600640 1 -- 172.21.15.39:0/2144378121 learned_addr learned my addr 172.21.15.39:0/2144378121 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:16.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.904+0000 7f7eea600640 1 -- 172.21.15.39:0/2144378121 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7ee40cf540 msgr2=0x7f7ee4153ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:16.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.904+0000 7f7eea600640 1 --2- 172.21.15.39:0/2144378121 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7ee40cf540 0x7f7ee4153ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:16.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.904+0000 7f7eea600640 1 -- 172.21.15.39:0/2144378121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ee40cf950 msgr2=0x7f7ee40a9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:16.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.904+0000 7f7eea600640 1 --2- 172.21.15.39:0/2144378121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ee40cf950 0x7f7ee40a9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:16.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.904+0000 7f7eea600640 1 -- 172.21.15.39:0/2144378121 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee40a99d0 con 0x7f7ee415ca50 2024-09-21T07:36:16.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.904+0000 7f7eea600640 1 --2- 172.21.15.39:0/2144378121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ee415ca50 0x7f7ee40aeb10 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f7edc006010 tx=0x7f7edc003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:16.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.904+0000 7f7ee1600640 1 -- 172.21.15.39:0/2144378121 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7edc046070 con 0x7f7ee415ca50 2024-09-21T07:36:16.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.905+0000 7f7eeb000640 1 -- 172.21.15.39:0/2144378121 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ee40a9ee0 con 0x7f7ee415ca50 2024-09-21T07:36:16.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.905+0000 7f7ee1600640 1 -- 172.21.15.39:0/2144378121 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7edc004540 con 0x7f7ee415ca50 2024-09-21T07:36:16.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.905+0000 7f7ee1600640 1 -- 172.21.15.39:0/2144378121 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7edc033070 con 0x7f7ee415ca50 2024-09-21T07:36:16.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.905+0000 7f7eeb000640 1 -- 172.21.15.39:0/2144378121 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ee4155cd0 con 0x7f7ee415ca50 2024-09-21T07:36:16.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.906+0000 7f7eeb000640 1 -- 172.21.15.39:0/2144378121 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ee40a9c50 con 0x7f7ee415ca50 2024-09-21T07:36:16.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.907+0000 7f7ee1600640 1 -- 172.21.15.39:0/2144378121 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7edc049070 con 0x7f7ee415ca50 2024-09-21T07:36:16.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.908+0000 7f7ee1600640 1 --2- 172.21.15.39:0/2144378121 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7ec4078b90 0x7f7ec407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:16.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.908+0000 7f7ee1600640 1 -- 172.21.15.39:0/2144378121 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5948+0+0 (secure 0 0 0) 0x7f7edc0bbf90 con 0x7f7ee415ca50 2024-09-21T07:36:16.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.908+0000 7f7ee9c00640 1 --2- 172.21.15.39:0/2144378121 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7ec4078b90 0x7f7ec407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:16.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.909+0000 7f7ee9c00640 1 --2- 172.21.15.39:0/2144378121 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7ec4078b90 0x7f7ec407b0a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4153ab0 tx=0x7f7ed8009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:16.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:16.912+0000 7f7ee1600640 1 -- 172.21.15.39:0/2144378121 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7edc03e070 con 0x7f7ee415ca50 2024-09-21T07:36:17.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.185+0000 7f7eeb000640 1 -- 172.21.15.39:0/2144378121 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f7ee40aa400 con 0x7f7ee415ca50 2024-09-21T07:36:17.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.185+0000 7f7ee1600640 1 -- 172.21.15.39:0/2144378121 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) ==== 80+0+5304 (secure 0 0 0) 0x7f7ee40aa400 con 0x7f7ee415ca50 2024-09-21T07:36:17.187 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:36:17.187 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:36:17.187 INFO:teuthology.orchestra.run.smithi039.stdout: "auth_dump": [ 2024-09-21T07:36:17.188 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:36:17.188 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.a", 2024-09-21T07:36:17.188 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5m+P6EERAABQqmlkrocSi1+hwc73Adrg==", 2024-09-21T07:36:17.188 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:36:17.188 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:36:17.188 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:36:17.188 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:36:17.188 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:36:17.188 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:36:17.188 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:36:17.188 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:36:17.188 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.b", 2024-09-21T07:36:17.188 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5m5/GeLRAAfE8fr5dbitQa89rNK9pZ2Q==", 2024-09-21T07:36:17.188 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:36:17.189 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:36:17.189 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:36:17.189 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:36:17.189 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:36:17.189 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:36:17.189 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:36:17.189 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:36:17.189 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.c", 2024-09-21T07:36:17.189 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5msbx6HxAALsF8ORvIoJpIAFZ5uxmOEw==", 2024-09-21T07:36:17.189 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:36:17.189 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:36:17.189 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:36:17.189 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:36:17.190 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:36:17.190 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:36:17.190 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:36:17.190 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:36:17.190 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.d", 2024-09-21T07:36:17.190 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mIrcBABAApKAB/7FkZ5glNTxlbkIsBw==", 2024-09-21T07:36:17.190 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:36:17.190 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:36:17.190 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:36:17.190 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:36:17.190 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:36:17.190 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:36:17.190 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:36:17.190 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:36:17.191 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.0", 2024-09-21T07:36:17.191 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB7du5msw81DhAAmVimqN4x9K3YpT0nbovppQ==", 2024-09-21T07:36:17.191 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:36:17.191 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:36:17.191 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:36:17.191 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:36:17.191 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:36:17.191 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:36:17.191 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:36:17.191 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.1", 2024-09-21T07:36:17.191 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB+du5mdnCIOhAAtVB4ThehQwb0zRriyKcaxA==", 2024-09-21T07:36:17.191 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:36:17.191 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:36:17.191 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:36:17.192 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:36:17.192 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:36:17.192 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:36:17.192 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:36:17.192 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.2", 2024-09-21T07:36:17.192 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCCdu5mTfWxDhAAadJCbPN5xBx/f9L0v2yTdg==", 2024-09-21T07:36:17.192 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:36:17.192 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:36:17.192 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:36:17.192 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:36:17.192 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:36:17.192 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:36:17.192 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:36:17.193 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.3", 2024-09-21T07:36:17.193 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCFdu5mL1E2HxAAyOhY97uUXEw5L1CcDeONVA==", 2024-09-21T07:36:17.193 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:36:17.193 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:36:17.193 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:36:17.193 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:36:17.193 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:36:17.193 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:36:17.193 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:36:17.193 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.4", 2024-09-21T07:36:17.193 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCLdu5m6PY/IhAAgvcTgm+6uWA7lIG4tcTIEg==", 2024-09-21T07:36:17.193 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:36:17.193 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:36:17.193 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:36:17.194 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:36:17.194 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:36:17.194 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:36:17.194 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:36:17.194 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.5", 2024-09-21T07:36:17.194 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCOdu5mE/wlMxAA3JBg4xH+N/PPC1Uv2ZVrYQ==", 2024-09-21T07:36:17.194 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:36:17.194 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:36:17.194 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:36:17.194 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:36:17.194 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:36:17.194 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:36:17.194 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:36:17.194 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.6", 2024-09-21T07:36:17.195 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCSdu5mC3TgBxAAwsArjeIdFcApY3/1OJffEg==", 2024-09-21T07:36:17.195 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:36:17.195 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:36:17.195 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:36:17.195 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:36:17.195 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:36:17.195 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:36:17.195 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:36:17.195 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.7", 2024-09-21T07:36:17.195 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCVdu5mSWHWFBAA9qPHWRdsvrB74inrp1lj4Q==", 2024-09-21T07:36:17.195 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:36:17.195 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:36:17.195 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:36:17.195 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:36:17.195 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:36:17.195 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:36:17.196 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:36:17.196 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.0", 2024-09-21T07:36:17.196 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mGfb1DBAA/aSMKR5or3xy53rB/iUjpA==", 2024-09-21T07:36:17.196 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:36:17.196 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:36:17.196 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow r", 2024-09-21T07:36:17.196 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T07:36:17.196 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:36:17.196 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:36:17.196 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:36:17.196 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.1", 2024-09-21T07:36:17.196 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mEkciIhAAid+bxJEr4P7vMnZ42PAy5w==", 2024-09-21T07:36:17.196 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:36:17.196 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:36:17.196 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow r", 2024-09-21T07:36:17.196 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow rw", 2024-09-21T07:36:17.197 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rwx" 2024-09-21T07:36:17.197 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:36:17.197 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:36:17.197 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:36:17.197 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.2", 2024-09-21T07:36:17.197 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mSu2oFxAAgjy2m/wWKh6xaoh+YDoU1Q==", 2024-09-21T07:36:17.197 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:36:17.197 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:36:17.197 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow r", 2024-09-21T07:36:17.197 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow rw", 2024-09-21T07:36:17.197 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rwx" 2024-09-21T07:36:17.197 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:36:17.197 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:36:17.198 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:36:17.198 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.3", 2024-09-21T07:36:17.198 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mCHy0LBAAZneF2jUWgfuyGehZFbOxDg==", 2024-09-21T07:36:17.198 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:36:17.198 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:36:17.198 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow r", 2024-09-21T07:36:17.198 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow rw", 2024-09-21T07:36:17.198 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rwx" 2024-09-21T07:36:17.198 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:36:17.198 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:36:17.198 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:36:17.198 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.admin", 2024-09-21T07:36:17.198 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBydu5m9LInERAAAlDpqb94f2Qb9VYJJwTBcA==", 2024-09-21T07:36:17.198 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:36:17.199 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow *", 2024-09-21T07:36:17.199 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:36:17.199 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:36:17.199 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:36:17.199 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:36:17.199 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:36:17.199 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:36:17.199 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mgr.x", 2024-09-21T07:36:17.199 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5mA2GRBhAAkyCKdgH6laHy8xNt080dXg==", 2024-09-21T07:36:17.199 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:36:17.199 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow *", 2024-09-21T07:36:17.199 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile mgr", 2024-09-21T07:36:17.199 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:36:17.199 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:36:17.200 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:36:17.200 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:36:17.200 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mgr.y", 2024-09-21T07:36:17.200 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBydu5mr+8aNxAA7TFYNms2WsXUfa4ITTJOHw==", 2024-09-21T07:36:17.200 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:36:17.200 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow *", 2024-09-21T07:36:17.200 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile mgr", 2024-09-21T07:36:17.200 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:36:17.200 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:36:17.200 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:36:17.200 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-21T07:36:17.200 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:36:17.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.188+0000 7f7eeb000640 1 -- 172.21.15.39:0/2144378121 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7ec4078b90 msgr2=0x7f7ec407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:17.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.188+0000 7f7eeb000640 1 --2- 172.21.15.39:0/2144378121 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7ec4078b90 0x7f7ec407b0a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4153ab0 tx=0x7f7ed8009040 comp rx=0 tx=0).stop 2024-09-21T07:36:17.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.188+0000 7f7eeb000640 1 -- 172.21.15.39:0/2144378121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ee415ca50 msgr2=0x7f7ee40aeb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:17.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.188+0000 7f7eeb000640 1 --2- 172.21.15.39:0/2144378121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ee415ca50 0x7f7ee40aeb10 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f7edc006010 tx=0x7f7edc003f10 comp rx=0 tx=0).stop 2024-09-21T07:36:17.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.188+0000 7f7eeb000640 1 -- 172.21.15.39:0/2144378121 shutdown_connections 2024-09-21T07:36:17.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.188+0000 7f7eeb000640 1 --2- 172.21.15.39:0/2144378121 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7ec4078b90 0x7f7ec407b0a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:17.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.188+0000 7f7eeb000640 1 --2- 172.21.15.39:0/2144378121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ee415ca50 0x7f7ee40aeb10 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:17.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.188+0000 7f7eeb000640 1 --2- 172.21.15.39:0/2144378121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ee40cf950 0x7f7ee40a9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:17.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.188+0000 7f7eeb000640 1 --2- 172.21.15.39:0/2144378121 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7ee40cf540 0x7f7ee4153ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:17.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.188+0000 7f7eeb000640 1 -- 172.21.15.39:0/2144378121 >> 172.21.15.39:0/2144378121 conn(0x7f7ee406d410 msgr2=0x7f7ee415b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:17.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.188+0000 7f7eeb000640 1 -- 172.21.15.39:0/2144378121 shutdown_connections 2024-09-21T07:36:17.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.188+0000 7f7eeb000640 1 -- 172.21.15.39:0/2144378121 wait complete. 2024-09-21T07:36:17.216 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T07:36:17.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.441+0000 7fa356c00640 1 -- 172.21.15.39:0/1534584985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa350070030 msgr2=0x7fa350074640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:17.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.441+0000 7fa356c00640 1 --2- 172.21.15.39:0/1534584985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa350070030 0x7fa350074640 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fa338005fd0 tx=0x7fa33802a540 comp rx=0 tx=0).stop 2024-09-21T07:36:17.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.441+0000 7fa356c00640 1 -- 172.21.15.39:0/1534584985 shutdown_connections 2024-09-21T07:36:17.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.441+0000 7fa356c00640 1 --2- 172.21.15.39:0/1534584985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa350070030 0x7fa350074640 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:17.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.441+0000 7fa356c00640 1 --2- 172.21.15.39:0/1534584985 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa350073170 0x7fa35015c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:17.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.441+0000 7fa356c00640 1 -- 172.21.15.39:0/1534584985 >> 172.21.15.39:0/1534584985 conn(0x7fa3500b1cb0 msgr2=0x7fa3500b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:17.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.441+0000 7fa356c00640 1 -- 172.21.15.39:0/1534584985 shutdown_connections 2024-09-21T07:36:17.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.441+0000 7fa356c00640 1 -- 172.21.15.39:0/1534584985 wait complete. 2024-09-21T07:36:17.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.443+0000 7fa356c00640 1 Processor -- start 2024-09-21T07:36:17.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.443+0000 7fa356c00640 1 -- start start 2024-09-21T07:36:17.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.445+0000 7fa356c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa350070030 0x7fa350151760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:17.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.445+0000 7fa356c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa350070440 0x7fa3500c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:17.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.445+0000 7fa356c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa350073170 0x7fa3501520b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:17.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.445+0000 7fa356c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa35006f5a0 con 0x7fa350070030 2024-09-21T07:36:17.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.445+0000 7fa356c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa350074f80 con 0x7fa350073170 2024-09-21T07:36:17.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.445+0000 7fa356c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa35006f720 con 0x7fa350070440 2024-09-21T07:36:17.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.445+0000 7fa34fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa350070030 0x7fa350151760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:17.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.445+0000 7fa34fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa350070030 0x7fa350151760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52676/0 (socket says 172.21.15.39:52676) 2024-09-21T07:36:17.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.445+0000 7fa34fe00640 1 -- 172.21.15.39:0/2729408580 learned_addr learned my addr 172.21.15.39:0/2729408580 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:17.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.445+0000 7fa354c00640 1 --2- 172.21.15.39:0/2729408580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa350073170 0x7fa3501520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:17.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.445+0000 7fa347400640 1 --2- 172.21.15.39:0/2729408580 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa350070440 0x7fa3500c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:17.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.446+0000 7fa34fe00640 1 -- 172.21.15.39:0/2729408580 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa350070440 msgr2=0x7fa3500c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:17.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.446+0000 7fa34fe00640 1 --2- 172.21.15.39:0/2729408580 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa350070440 0x7fa3500c92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:17.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.446+0000 7fa34fe00640 1 -- 172.21.15.39:0/2729408580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa350073170 msgr2=0x7fa3501520b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:17.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.446+0000 7fa34fe00640 1 --2- 172.21.15.39:0/2729408580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa350073170 0x7fa3501520b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:17.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.446+0000 7fa34fe00640 1 -- 172.21.15.39:0/2729408580 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35009c3a0 con 0x7fa350070030 2024-09-21T07:36:17.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.446+0000 7fa34fe00640 1 --2- 172.21.15.39:0/2729408580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa350070030 0x7fa350151760 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fa34000d740 tx=0x7fa34000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:17.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.446+0000 7fa34cc00640 1 -- 172.21.15.39:0/2729408580 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa340017070 con 0x7fa350070030 2024-09-21T07:36:17.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.446+0000 7fa34cc00640 1 -- 172.21.15.39:0/2729408580 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa340009d20 con 0x7fa350070030 2024-09-21T07:36:17.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.446+0000 7fa356c00640 1 -- 172.21.15.39:0/2729408580 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa35009bde0 con 0x7fa350070030 2024-09-21T07:36:17.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.446+0000 7fa34cc00640 1 -- 172.21.15.39:0/2729408580 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3400106c0 con 0x7fa350070030 2024-09-21T07:36:17.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.446+0000 7fa356c00640 1 -- 172.21.15.39:0/2729408580 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3500c8ad0 con 0x7fa350070030 2024-09-21T07:36:17.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.447+0000 7fa356c00640 1 -- 172.21.15.39:0/2729408580 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa35006fc40 con 0x7fa350070030 2024-09-21T07:36:17.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.449+0000 7fa34cc00640 1 -- 172.21.15.39:0/2729408580 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa34001a070 con 0x7fa350070030 2024-09-21T07:36:17.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.450+0000 7fa34cc00640 1 --2- 172.21.15.39:0/2729408580 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa32c078bc0 0x7fa32c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:17.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.450+0000 7fa34cc00640 1 -- 172.21.15.39:0/2729408580 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5948+0+0 (secure 0 0 0) 0x7fa34009afb0 con 0x7fa350070030 2024-09-21T07:36:17.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.450+0000 7fa347400640 1 --2- 172.21.15.39:0/2729408580 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa32c078bc0 0x7fa32c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:17.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.451+0000 7fa347400640 1 --2- 172.21.15.39:0/2729408580 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa32c078bc0 0x7fa32c07b0d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa338033000 tx=0x7fa338004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:17.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.454+0000 7fa34cc00640 1 -- 172.21.15.39:0/2729408580 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa340062b90 con 0x7fa350070030 2024-09-21T07:36:17.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.662+0000 7fa356c00640 1 -- 172.21.15.39:0/2729408580 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fa3500c3860 con 0x7fa350070030 2024-09-21T07:36:17.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.663+0000 7fa34cc00640 1 -- 172.21.15.39:0/2729408580 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v86) ==== 84+0+287 (secure 0 0 0) 0x7fa3400675c0 con 0x7fa350070030 2024-09-21T07:36:17.665 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:36:17.665 INFO:teuthology.orchestra.run.smithi039.stdout:[ 2024-09-21T07:36:17.665 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:36:17.665 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 1, 2024-09-21T07:36:17.665 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": ".mgr" 2024-09-21T07:36:17.665 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:36:17.665 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:36:17.665 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 6, 2024-09-21T07:36:17.665 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": "snap-test-pool" 2024-09-21T07:36:17.666 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:36:17.666 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:36:17.666 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 11, 2024-09-21T07:36:17.666 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": "test_data_pool" 2024-09-21T07:36:17.666 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:36:17.666 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:36:17.666 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 12, 2024-09-21T07:36:17.666 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": "test_metadata_pool" 2024-09-21T07:36:17.666 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:36:17.666 INFO:teuthology.orchestra.run.smithi039.stdout:] 2024-09-21T07:36:17.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.665+0000 7fa356c00640 1 -- 172.21.15.39:0/2729408580 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa32c078bc0 msgr2=0x7fa32c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:17.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.665+0000 7fa356c00640 1 --2- 172.21.15.39:0/2729408580 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa32c078bc0 0x7fa32c07b0d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa338033000 tx=0x7fa338004730 comp rx=0 tx=0).stop 2024-09-21T07:36:17.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.665+0000 7fa356c00640 1 -- 172.21.15.39:0/2729408580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa350070030 msgr2=0x7fa350151760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:17.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.665+0000 7fa356c00640 1 --2- 172.21.15.39:0/2729408580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa350070030 0x7fa350151760 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fa34000d740 tx=0x7fa34000dc00 comp rx=0 tx=0).stop 2024-09-21T07:36:17.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.666+0000 7fa356c00640 1 -- 172.21.15.39:0/2729408580 shutdown_connections 2024-09-21T07:36:17.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.666+0000 7fa356c00640 1 --2- 172.21.15.39:0/2729408580 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa32c078bc0 0x7fa32c07b0d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:17.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.666+0000 7fa356c00640 1 --2- 172.21.15.39:0/2729408580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa350073170 0x7fa3501520b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:17.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.666+0000 7fa356c00640 1 --2- 172.21.15.39:0/2729408580 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa350070440 0x7fa3500c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:17.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.666+0000 7fa356c00640 1 --2- 172.21.15.39:0/2729408580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa350070030 0x7fa350151760 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:17.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.666+0000 7fa356c00640 1 -- 172.21.15.39:0/2729408580 >> 172.21.15.39:0/2729408580 conn(0x7fa3500b1cb0 msgr2=0x7fa35015d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:17.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.666+0000 7fa356c00640 1 -- 172.21.15.39:0/2729408580 shutdown_connections 2024-09-21T07:36:17.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.666+0000 7fa356c00640 1 -- 172.21.15.39:0/2729408580 wait complete. 2024-09-21T07:36:17.690 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T07:36:17.690 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T07:36:17.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.920+0000 7f29afe00640 1 -- 172.21.15.39:0/3502043745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2990004800 msgr2=0x7f2990004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:17.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.920+0000 7f29afe00640 1 --2- 172.21.15.39:0/3502043745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2990004800 0x7f2990004c10 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f2994009b50 tx=0x7f2990005830 comp rx=0 tx=0).stop 2024-09-21T07:36:17.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.921+0000 7f29afe00640 1 -- 172.21.15.39:0/3502043745 shutdown_connections 2024-09-21T07:36:17.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.921+0000 7f29afe00640 1 --2- 172.21.15.39:0/3502043745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2990004800 0x7f2990004c10 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:17.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.921+0000 7f29afe00640 1 -- 172.21.15.39:0/3502043745 >> 172.21.15.39:0/3502043745 conn(0x7f29a80d0130 msgr2=0x7f29a80d0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:17.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.921+0000 7f29afe00640 1 -- 172.21.15.39:0/3502043745 shutdown_connections 2024-09-21T07:36:17.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.921+0000 7f29afe00640 1 -- 172.21.15.39:0/3502043745 wait complete. 2024-09-21T07:36:17.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.923+0000 7f29afe00640 1 Processor -- start 2024-09-21T07:36:17.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.923+0000 7f29afe00640 1 -- start start 2024-09-21T07:36:17.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.924+0000 7f29afe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2990004800 0x7f29a815b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:17.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.924+0000 7f29afe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29a80c45b0 0x7f29a80c49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:17.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.924+0000 7f29afe00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f29a80c4f00 0x7f29a81f6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:17.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.924+0000 7f29afe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f29a80cadc0 con 0x7f2990004800 2024-09-21T07:36:17.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.924+0000 7f29afe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f29a8073b90 con 0x7f29a80c45b0 2024-09-21T07:36:17.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.924+0000 7f29afe00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f29a80caf40 con 0x7f29a80c4f00 2024-09-21T07:36:17.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.925+0000 7f29ad400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2990004800 0x7f29a815b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:17.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.925+0000 7f29ad400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2990004800 0x7f29a815b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52706/0 (socket says 172.21.15.39:52706) 2024-09-21T07:36:17.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.925+0000 7f29ad400640 1 -- 172.21.15.39:0/4279051295 learned_addr learned my addr 172.21.15.39:0/4279051295 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:17.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.925+0000 7f29aca00640 1 --2- 172.21.15.39:0/4279051295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29a80c45b0 0x7f29a80c49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:17.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.925+0000 7f29ade00640 1 --2- 172.21.15.39:0/4279051295 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f29a80c4f00 0x7f29a81f6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:17.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.925+0000 7f29ad400640 1 -- 172.21.15.39:0/4279051295 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f29a80c4f00 msgr2=0x7f29a81f6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:17.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.925+0000 7f29ad400640 1 --2- 172.21.15.39:0/4279051295 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f29a80c4f00 0x7f29a81f6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:17.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.925+0000 7f29ad400640 1 -- 172.21.15.39:0/4279051295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29a80c45b0 msgr2=0x7f29a80c49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:17.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.925+0000 7f29ad400640 1 --2- 172.21.15.39:0/4279051295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29a80c45b0 0x7f29a80c49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:17.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.925+0000 7f29ad400640 1 -- 172.21.15.39:0/4279051295 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a815ec90 con 0x7f2990004800 2024-09-21T07:36:17.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.925+0000 7f29ade00640 1 --2- 172.21.15.39:0/4279051295 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f29a80c4f00 0x7f29a81f6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:36:17.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.925+0000 7f29ad400640 1 --2- 172.21.15.39:0/4279051295 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2990004800 0x7f29a815b910 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f299c00e570 tx=0x7f299c00ea30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:17.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.926+0000 7f29a5600640 1 -- 172.21.15.39:0/4279051295 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299c015020 con 0x7f2990004800 2024-09-21T07:36:17.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.926+0000 7f29a5600640 1 -- 172.21.15.39:0/4279051295 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f299c00c9e0 con 0x7f2990004800 2024-09-21T07:36:17.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.926+0000 7f29a5600640 1 -- 172.21.15.39:0/4279051295 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299c00ce30 con 0x7f2990004800 2024-09-21T07:36:17.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.926+0000 7f29afe00640 1 -- 172.21.15.39:0/4279051295 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29a81f2e20 con 0x7f2990004800 2024-09-21T07:36:17.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.926+0000 7f29afe00640 1 -- 172.21.15.39:0/4279051295 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29a81f45f0 con 0x7f2990004800 2024-09-21T07:36:17.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.927+0000 7f29afe00640 1 -- 172.21.15.39:0/4279051295 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29a80cb450 con 0x7f2990004800 2024-09-21T07:36:17.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.928+0000 7f29a5600640 1 -- 172.21.15.39:0/4279051295 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f299c018020 con 0x7f2990004800 2024-09-21T07:36:17.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.929+0000 7f29a5600640 1 --2- 172.21.15.39:0/4279051295 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2988078b90 0x7f298807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:17.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.929+0000 7f29a5600640 1 -- 172.21.15.39:0/4279051295 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5948+0+0 (secure 0 0 0) 0x7f299c09b260 con 0x7f2990004800 2024-09-21T07:36:17.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.930+0000 7f29aca00640 1 --2- 172.21.15.39:0/4279051295 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2988078b90 0x7f298807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:17.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.931+0000 7f29aca00640 1 --2- 172.21.15.39:0/4279051295 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2988078b90 0x7f298807b0a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2994035000 tx=0x7f29940046c0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:17.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:17.934+0000 7f29a5600640 1 -- 172.21.15.39:0/4279051295 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f299c01f020 con 0x7f2990004800 2024-09-21T07:36:18.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.129+0000 7f29afe00640 1 -- 172.21.15.39:0/4279051295 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f29a80adb80 con 0x7f2990004800 2024-09-21T07:36:18.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.658+0000 7f29a5600640 1 -- 172.21.15.39:0/4279051295 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v87) ==== 138+0+0 (secure 0 0 0) 0x7f299c0636d0 con 0x7f2990004800 2024-09-21T07:36:18.660 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' created 2024-09-21T07:36:18.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.662+0000 7f29afe00640 1 -- 172.21.15.39:0/4279051295 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2988078b90 msgr2=0x7f298807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:18.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.662+0000 7f29afe00640 1 --2- 172.21.15.39:0/4279051295 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2988078b90 0x7f298807b0a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2994035000 tx=0x7f29940046c0 comp rx=0 tx=0).stop 2024-09-21T07:36:18.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.662+0000 7f29afe00640 1 -- 172.21.15.39:0/4279051295 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2990004800 msgr2=0x7f29a815b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:18.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.662+0000 7f29afe00640 1 --2- 172.21.15.39:0/4279051295 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2990004800 0x7f29a815b910 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f299c00e570 tx=0x7f299c00ea30 comp rx=0 tx=0).stop 2024-09-21T07:36:18.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.662+0000 7f29afe00640 1 -- 172.21.15.39:0/4279051295 shutdown_connections 2024-09-21T07:36:18.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.662+0000 7f29afe00640 1 --2- 172.21.15.39:0/4279051295 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2988078b90 0x7f298807b0a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:18.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.662+0000 7f29afe00640 1 --2- 172.21.15.39:0/4279051295 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f29a80c4f00 0x7f29a81f6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:18.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.662+0000 7f29afe00640 1 --2- 172.21.15.39:0/4279051295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29a80c45b0 0x7f29a80c49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:18.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.662+0000 7f29afe00640 1 --2- 172.21.15.39:0/4279051295 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2990004800 0x7f29a815b910 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:18.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.662+0000 7f29afe00640 1 -- 172.21.15.39:0/4279051295 >> 172.21.15.39:0/4279051295 conn(0x7f29a80d0130 msgr2=0x7f29a815c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:18.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.662+0000 7f29afe00640 1 -- 172.21.15.39:0/4279051295 shutdown_connections 2024-09-21T07:36:18.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.662+0000 7f29afe00640 1 -- 172.21.15.39:0/4279051295 wait complete. 2024-09-21T07:36:18.691 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T07:36:18.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.924+0000 7fee93200640 1 -- 172.21.15.39:0/2728252389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee74005150 msgr2=0x7fee7400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:18.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.924+0000 7fee93200640 1 --2- 172.21.15.39:0/2728252389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee74005150 0x7fee7400b9c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fee8c0734a0 tx=0x7fee74005830 comp rx=0 tx=0).stop 2024-09-21T07:36:18.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.924+0000 7fee92800640 1 -- 172.21.15.39:0/2728252389 reap_dead start 2024-09-21T07:36:18.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.925+0000 7fee93200640 1 -- 172.21.15.39:0/2728252389 shutdown_connections 2024-09-21T07:36:18.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.925+0000 7fee93200640 1 -- 172.21.15.39:0/2728252389 >> 172.21.15.39:0/2728252389 conn(0x7fee8c06ee50 msgr2=0x7fee8c06f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:18.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.925+0000 7fee93200640 1 -- 172.21.15.39:0/2728252389 shutdown_connections 2024-09-21T07:36:18.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.925+0000 7fee93200640 1 -- 172.21.15.39:0/2728252389 wait complete. 2024-09-21T07:36:18.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.927+0000 7fee93200640 1 Processor -- start 2024-09-21T07:36:18.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.927+0000 7fee93200640 1 -- start start 2024-09-21T07:36:18.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.928+0000 7fee93200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee74005150 0x7fee8c1f6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:18.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.928+0000 7fee93200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee8c1f3da0 0x7fee8c154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:18.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.928+0000 7fee93200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee8c152f40 0x7fee8c15a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:18.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.928+0000 7fee93200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c0d0410 con 0x7fee8c152f40 2024-09-21T07:36:18.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.928+0000 7fee93200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c0d0290 con 0x7fee74005150 2024-09-21T07:36:18.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.928+0000 7fee93200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fee8c074470 con 0x7fee8c1f3da0 2024-09-21T07:36:18.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.929+0000 7fee92800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee8c152f40 0x7fee8c15a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:18.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.929+0000 7fee92800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee8c152f40 0x7fee8c15a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52730/0 (socket says 172.21.15.39:52730) 2024-09-21T07:36:18.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.929+0000 7fee92800640 1 -- 172.21.15.39:0/2331381905 learned_addr learned my addr 172.21.15.39:0/2331381905 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:18.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.929+0000 7fee91e00640 1 --2- 172.21.15.39:0/2331381905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee74005150 0x7fee8c1f6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:18.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.929+0000 7fee91400640 1 --2- 172.21.15.39:0/2331381905 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee8c1f3da0 0x7fee8c154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:18.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.929+0000 7fee92800640 1 -- 172.21.15.39:0/2331381905 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee8c1f3da0 msgr2=0x7fee8c154420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:18.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.929+0000 7fee92800640 1 --2- 172.21.15.39:0/2331381905 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee8c1f3da0 0x7fee8c154420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:18.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.929+0000 7fee92800640 1 -- 172.21.15.39:0/2331381905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee74005150 msgr2=0x7fee8c1f6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:18.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.929+0000 7fee92800640 1 --2- 172.21.15.39:0/2331381905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee74005150 0x7fee8c1f6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:18.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.929+0000 7fee92800640 1 -- 172.21.15.39:0/2331381905 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee8c1f6ea0 con 0x7fee8c152f40 2024-09-21T07:36:18.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.930+0000 7fee92800640 1 --2- 172.21.15.39:0/2331381905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee8c152f40 0x7fee8c15a4b0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fee88036040 tx=0x7fee88033c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:18.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.930+0000 7fee86000640 1 -- 172.21.15.39:0/2331381905 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee88038040 con 0x7fee8c152f40 2024-09-21T07:36:18.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.930+0000 7fee86000640 1 -- 172.21.15.39:0/2331381905 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee88003970 con 0x7fee8c152f40 2024-09-21T07:36:18.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.930+0000 7fee86000640 1 -- 172.21.15.39:0/2331381905 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee88009a90 con 0x7fee8c152f40 2024-09-21T07:36:18.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.930+0000 7fee93200640 1 -- 172.21.15.39:0/2331381905 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee8c1f73b0 con 0x7fee8c152f40 2024-09-21T07:36:18.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.930+0000 7fee93200640 1 -- 172.21.15.39:0/2331381905 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee8c1f35a0 con 0x7fee8c152f40 2024-09-21T07:36:18.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.931+0000 7fee93200640 1 -- 172.21.15.39:0/2331381905 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee8c15c360 con 0x7fee8c152f40 2024-09-21T07:36:18.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.932+0000 7fee86000640 1 -- 172.21.15.39:0/2331381905 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fee88045060 con 0x7fee8c152f40 2024-09-21T07:36:18.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.933+0000 7fee86000640 1 --2- 172.21.15.39:0/2331381905 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fee70078bc0 0x7fee7007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:18.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.933+0000 7fee86000640 1 -- 172.21.15.39:0/2331381905 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6318+0+0 (secure 0 0 0) 0x7fee880bb550 con 0x7fee8c152f40 2024-09-21T07:36:18.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.934+0000 7fee91e00640 1 --2- 172.21.15.39:0/2331381905 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fee70078bc0 0x7fee7007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:18.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.935+0000 7fee91e00640 1 --2- 172.21.15.39:0/2331381905 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fee70078bc0 0x7fee7007b0d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fee80000fb0 tx=0x7fee80006910 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:18.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:18.939+0000 7fee86000640 1 -- 172.21.15.39:0/2331381905 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fee88083170 con 0x7fee8c152f40 2024-09-21T07:36:19.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.136+0000 7fee93200640 1 -- 172.21.15.39:0/2331381905 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fee8c1f7140 con 0x7fee8c152f40 2024-09-21T07:36:19.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.666+0000 7fee86000640 1 -- 172.21.15.39:0/2331381905 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v88) ==== 170+0+0 (secure 0 0 0) 0x7fee88087ba0 con 0x7fee8c152f40 2024-09-21T07:36:19.668 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' created 2024-09-21T07:36:19.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.669+0000 7fee93200640 1 -- 172.21.15.39:0/2331381905 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fee70078bc0 msgr2=0x7fee7007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:19.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.669+0000 7fee93200640 1 --2- 172.21.15.39:0/2331381905 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fee70078bc0 0x7fee7007b0d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fee80000fb0 tx=0x7fee80006910 comp rx=0 tx=0).stop 2024-09-21T07:36:19.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.669+0000 7fee93200640 1 -- 172.21.15.39:0/2331381905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee8c152f40 msgr2=0x7fee8c15a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:19.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.669+0000 7fee93200640 1 --2- 172.21.15.39:0/2331381905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee8c152f40 0x7fee8c15a4b0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fee88036040 tx=0x7fee88033c30 comp rx=0 tx=0).stop 2024-09-21T07:36:19.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.669+0000 7fee93200640 1 -- 172.21.15.39:0/2331381905 shutdown_connections 2024-09-21T07:36:19.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.669+0000 7fee93200640 1 --2- 172.21.15.39:0/2331381905 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fee70078bc0 0x7fee7007b0d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:19.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.669+0000 7fee93200640 1 --2- 172.21.15.39:0/2331381905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee8c152f40 0x7fee8c15a4b0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:19.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.669+0000 7fee93200640 1 --2- 172.21.15.39:0/2331381905 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee8c1f3da0 0x7fee8c154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:19.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.669+0000 7fee93200640 1 --2- 172.21.15.39:0/2331381905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee74005150 0x7fee8c1f6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:19.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.669+0000 7fee93200640 1 -- 172.21.15.39:0/2331381905 >> 172.21.15.39:0/2331381905 conn(0x7fee8c06ee50 msgr2=0x7fee8c152240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:19.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.669+0000 7fee93200640 1 -- 172.21.15.39:0/2331381905 shutdown_connections 2024-09-21T07:36:19.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.669+0000 7fee93200640 1 -- 172.21.15.39:0/2331381905 wait complete. 2024-09-21T07:36:19.707 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T07:36:19.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.937+0000 7fba3c000640 1 -- 172.21.15.39:0/2784402338 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba1c004d40 msgr2=0x7fba1c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:19.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.937+0000 7fba3c000640 1 --2- 172.21.15.39:0/2784402338 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba1c004d40 0x7fba1c005190 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fba2c00c530 tx=0x7fba1c005830 comp rx=0 tx=0).stop 2024-09-21T07:36:19.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.937+0000 7fba3a000640 1 -- 172.21.15.39:0/2784402338 reap_dead start 2024-09-21T07:36:19.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.937+0000 7fba3c000640 1 -- 172.21.15.39:0/2784402338 shutdown_connections 2024-09-21T07:36:19.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.937+0000 7fba3c000640 1 -- 172.21.15.39:0/2784402338 >> 172.21.15.39:0/2784402338 conn(0x7fba340b1cb0 msgr2=0x7fba340b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:19.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.937+0000 7fba3c000640 1 -- 172.21.15.39:0/2784402338 shutdown_connections 2024-09-21T07:36:19.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.937+0000 7fba3c000640 1 -- 172.21.15.39:0/2784402338 wait complete. 2024-09-21T07:36:19.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.939+0000 7fba3c000640 1 Processor -- start 2024-09-21T07:36:19.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.939+0000 7fba3c000640 1 -- start start 2024-09-21T07:36:19.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.941+0000 7fba3c000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba1c004d40 0x7fba340cb140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:19.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.941+0000 7fba3c000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fba34151540 0x7fba3409cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:19.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.941+0000 7fba3c000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba34151950 0x7fba3409d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:19.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.941+0000 7fba3c000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fba340b37a0 con 0x7fba34151950 2024-09-21T07:36:19.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.941+0000 7fba3c000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fba34075070 con 0x7fba1c004d40 2024-09-21T07:36:19.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.941+0000 7fba3c000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fba340b3920 con 0x7fba34151540 2024-09-21T07:36:19.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.941+0000 7fba3a000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba34151950 0x7fba3409d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:19.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.941+0000 7fba3a000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba34151950 0x7fba3409d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52744/0 (socket says 172.21.15.39:52744) 2024-09-21T07:36:19.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.941+0000 7fba3a000640 1 -- 172.21.15.39:0/1288072367 learned_addr learned my addr 172.21.15.39:0/1288072367 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:19.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.941+0000 7fba39600640 1 --2- 172.21.15.39:0/1288072367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba1c004d40 0x7fba340cb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:19.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.941+0000 7fba38c00640 1 --2- 172.21.15.39:0/1288072367 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fba34151540 0x7fba3409cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:19.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.941+0000 7fba3a000640 1 -- 172.21.15.39:0/1288072367 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fba34151540 msgr2=0x7fba3409cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:19.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.941+0000 7fba3a000640 1 --2- 172.21.15.39:0/1288072367 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fba34151540 0x7fba3409cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:19.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.941+0000 7fba3a000640 1 -- 172.21.15.39:0/1288072367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba1c004d40 msgr2=0x7fba340cb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:19.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.941+0000 7fba3a000640 1 --2- 172.21.15.39:0/1288072367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba1c004d40 0x7fba340cb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:19.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.941+0000 7fba3a000640 1 -- 172.21.15.39:0/1288072367 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba340cc1f0 con 0x7fba34151950 2024-09-21T07:36:19.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.942+0000 7fba3a000640 1 --2- 172.21.15.39:0/1288072367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba34151950 0x7fba3409d140 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fba2c02b3a0 tx=0x7fba2c001aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:19.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.942+0000 7fba31600640 1 -- 172.21.15.39:0/1288072367 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c039070 con 0x7fba34151950 2024-09-21T07:36:19.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.942+0000 7fba31600640 1 -- 172.21.15.39:0/1288072367 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba2c02b9e0 con 0x7fba34151950 2024-09-21T07:36:19.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.942+0000 7fba3c000640 1 -- 172.21.15.39:0/1288072367 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba340ce7a0 con 0x7fba34151950 2024-09-21T07:36:19.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.942+0000 7fba31600640 1 -- 172.21.15.39:0/1288072367 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c001fa0 con 0x7fba34151950 2024-09-21T07:36:19.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.942+0000 7fba3c000640 1 -- 172.21.15.39:0/1288072367 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba340ce4c0 con 0x7fba34151950 2024-09-21T07:36:19.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.943+0000 7fba3c000640 1 -- 172.21.15.39:0/1288072367 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba340ce200 con 0x7fba34151950 2024-09-21T07:36:19.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.945+0000 7fba31600640 1 -- 172.21.15.39:0/1288072367 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fba2c034680 con 0x7fba34151950 2024-09-21T07:36:19.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.946+0000 7fba31600640 1 --2- 172.21.15.39:0/1288072367 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fba14078b90 0x7fba1407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:19.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.946+0000 7fba31600640 1 -- 172.21.15.39:0/1288072367 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6700+0+0 (secure 0 0 0) 0x7fba2c0bafb0 con 0x7fba34151950 2024-09-21T07:36:19.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.946+0000 7fba39600640 1 --2- 172.21.15.39:0/1288072367 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fba14078b90 0x7fba1407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:19.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.947+0000 7fba39600640 1 --2- 172.21.15.39:0/1288072367 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fba14078b90 0x7fba1407b0a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fba28006fd0 tx=0x7fba28006b10 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:19.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:19.951+0000 7fba31600640 1 -- 172.21.15.39:0/1288072367 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fba2c082a70 con 0x7fba34151950 2024-09-21T07:36:20.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.187+0000 7fba3c000640 1 -- 172.21.15.39:0/1288072367 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fba340b3eb0 con 0x7fba34151950 2024-09-21T07:36:20.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.668+0000 7fba31600640 1 -- 172.21.15.39:0/1288072367 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v51) ==== 174+0+0 (secure 0 0 0) 0x7fba2c0874a0 con 0x7fba34151950 2024-09-21T07:36:20.670 INFO:teuthology.orchestra.run.smithi039.stderr:new fs with metadata pool 15 and data pool 16 2024-09-21T07:36:20.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.670+0000 7fba3c000640 1 -- 172.21.15.39:0/1288072367 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fba14078b90 msgr2=0x7fba1407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:20.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.670+0000 7fba3c000640 1 --2- 172.21.15.39:0/1288072367 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fba14078b90 0x7fba1407b0a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fba28006fd0 tx=0x7fba28006b10 comp rx=0 tx=0).stop 2024-09-21T07:36:20.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.670+0000 7fba3c000640 1 -- 172.21.15.39:0/1288072367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba34151950 msgr2=0x7fba3409d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:20.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.670+0000 7fba3c000640 1 --2- 172.21.15.39:0/1288072367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba34151950 0x7fba3409d140 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fba2c02b3a0 tx=0x7fba2c001aa0 comp rx=0 tx=0).stop 2024-09-21T07:36:20.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.671+0000 7fba3c000640 1 -- 172.21.15.39:0/1288072367 shutdown_connections 2024-09-21T07:36:20.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.671+0000 7fba3c000640 1 --2- 172.21.15.39:0/1288072367 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fba14078b90 0x7fba1407b0a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:20.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.671+0000 7fba3c000640 1 --2- 172.21.15.39:0/1288072367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba34151950 0x7fba3409d140 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:20.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.671+0000 7fba3c000640 1 --2- 172.21.15.39:0/1288072367 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fba34151540 0x7fba3409cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:20.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.671+0000 7fba3c000640 1 --2- 172.21.15.39:0/1288072367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba1c004d40 0x7fba340cb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:20.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.671+0000 7fba3c000640 1 -- 172.21.15.39:0/1288072367 >> 172.21.15.39:0/1288072367 conn(0x7fba340b1cb0 msgr2=0x7fba340ad950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:20.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.671+0000 7fba3c000640 1 -- 172.21.15.39:0/1288072367 shutdown_connections 2024-09-21T07:36:20.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.671+0000 7fba3c000640 1 -- 172.21.15.39:0/1288072367 wait complete. 2024-09-21T07:36:20.723 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:36:20.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.960+0000 7f27a4000640 1 -- 172.21.15.39:0/2428186002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f279c0cc1e0 msgr2=0x7f279c0af3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:20.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.960+0000 7f27a4000640 1 --2- 172.21.15.39:0/2428186002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f279c0cc1e0 0x7f279c0af3c0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f2790005fd0 tx=0x7f279002a5d0 comp rx=0 tx=0).stop 2024-09-21T07:36:20.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.960+0000 7f27a4000640 1 -- 172.21.15.39:0/2428186002 shutdown_connections 2024-09-21T07:36:20.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.960+0000 7f27a4000640 1 --2- 172.21.15.39:0/2428186002 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f279c0ccb50 0x7f279c0ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:20.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.960+0000 7f27a4000640 1 --2- 172.21.15.39:0/2428186002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f279c0cc1e0 0x7f279c0af3c0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:20.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.960+0000 7f27a4000640 1 -- 172.21.15.39:0/2428186002 >> 172.21.15.39:0/2428186002 conn(0x7f279c06d7d0 msgr2=0x7f279c0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:20.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.960+0000 7f27a4000640 1 -- 172.21.15.39:0/2428186002 shutdown_connections 2024-09-21T07:36:20.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.960+0000 7f27a4000640 1 -- 172.21.15.39:0/2428186002 wait complete. 2024-09-21T07:36:20.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.962+0000 7f27a4000640 1 Processor -- start 2024-09-21T07:36:20.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.963+0000 7f27a4000640 1 -- start start 2024-09-21T07:36:20.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.964+0000 7f27a4000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f279c0cc1e0 0x7f279c15a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:20.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.964+0000 7f27a4000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f279c0ccb50 0x7f279c1f9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:20.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.964+0000 7f27a4000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f279c15d8a0 0x7f279c1f9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:20.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.964+0000 7f27a4000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f279c1533a0 con 0x7f279c15d8a0 2024-09-21T07:36:20.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.964+0000 7f27a4000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f279c153520 con 0x7f279c0ccb50 2024-09-21T07:36:20.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.964+0000 7f27a4000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f279c1536a0 con 0x7f279c0cc1e0 2024-09-21T07:36:20.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.964+0000 7f27a2000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f279c15d8a0 0x7f279c1f9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:20.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.964+0000 7f27a2000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f279c15d8a0 0x7f279c1f9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52768/0 (socket says 172.21.15.39:52768) 2024-09-21T07:36:20.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.964+0000 7f27a2000640 1 -- 172.21.15.39:0/217845777 learned_addr learned my addr 172.21.15.39:0/217845777 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:20.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.964+0000 7f27a1600640 1 --2- 172.21.15.39:0/217845777 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f279c0cc1e0 0x7f279c15a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:20.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.964+0000 7f27a0c00640 1 --2- 172.21.15.39:0/217845777 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f279c0ccb50 0x7f279c1f9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:20.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.965+0000 7f27a2000640 1 -- 172.21.15.39:0/217845777 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f279c0cc1e0 msgr2=0x7f279c15a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:20.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.965+0000 7f27a2000640 1 --2- 172.21.15.39:0/217845777 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f279c0cc1e0 0x7f279c15a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:20.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.965+0000 7f27a2000640 1 -- 172.21.15.39:0/217845777 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f279c0ccb50 msgr2=0x7f279c1f9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:20.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.965+0000 7f27a2000640 1 --2- 172.21.15.39:0/217845777 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f279c0ccb50 0x7f279c1f9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:20.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.965+0000 7f27a2000640 1 -- 172.21.15.39:0/217845777 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f279c0ad130 con 0x7f279c15d8a0 2024-09-21T07:36:20.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.965+0000 7f27a2000640 1 --2- 172.21.15.39:0/217845777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f279c15d8a0 0x7f279c1f9770 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f279400cad0 tx=0x7f279400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:20.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.965+0000 7f2799600640 1 -- 172.21.15.39:0/217845777 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279400ae20 con 0x7f279c15d8a0 2024-09-21T07:36:20.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.965+0000 7f2799600640 1 -- 172.21.15.39:0/217845777 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f279400f540 con 0x7f279c15d8a0 2024-09-21T07:36:20.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.965+0000 7f2799600640 1 -- 172.21.15.39:0/217845777 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279400f990 con 0x7f279c15d8a0 2024-09-21T07:36:20.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.966+0000 7f27a4000640 1 -- 172.21.15.39:0/217845777 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f279c09d640 con 0x7f279c15d8a0 2024-09-21T07:36:20.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.966+0000 7f27a4000640 1 -- 172.21.15.39:0/217845777 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f279c0cb570 con 0x7f279c15d8a0 2024-09-21T07:36:20.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.967+0000 7f27a4000640 1 -- 172.21.15.39:0/217845777 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f279c153bc0 con 0x7f279c15d8a0 2024-09-21T07:36:20.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.968+0000 7f2799600640 1 -- 172.21.15.39:0/217845777 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f279400fb30 con 0x7f279c15d8a0 2024-09-21T07:36:20.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.969+0000 7f2799600640 1 --2- 172.21.15.39:0/217845777 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f277c078b90 0x7f277c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:20.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.970+0000 7f2799600640 1 -- 172.21.15.39:0/217845777 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6800+0+0 (secure 0 0 0) 0x7f279409ba90 con 0x7f279c15d8a0 2024-09-21T07:36:20.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.970+0000 7f27a1600640 1 --2- 172.21.15.39:0/217845777 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f277c078b90 0x7f277c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:20.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.971+0000 7f27a1600640 1 --2- 172.21.15.39:0/217845777 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f277c078b90 0x7f277c07b0a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2790033000 tx=0x7f2790004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:20.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:20.974+0000 7f2799600640 1 -- 172.21.15.39:0/217845777 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2794014030 con 0x7f279c15d8a0 2024-09-21T07:36:21.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.172+0000 7f27a4000640 1 -- 172.21.15.39:0/217845777 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f279c0b1f00 con 0x7f279c15d8a0 2024-09-21T07:36:21.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.174+0000 7f2799600640 1 -- 172.21.15.39:0/217845777 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22132 (secure 0 0 0) 0x7f279400fe20 con 0x7f279c15d8a0 2024-09-21T07:36:21.179 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:36:21.179 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":89,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:36:20.189332+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:36:18.142953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-21T07:36:19.149503+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:36:21.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.176+0000 7f27a4000640 1 -- 172.21.15.39:0/217845777 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f277c078b90 msgr2=0x7f277c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:21.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.176+0000 7f27a4000640 1 --2- 172.21.15.39:0/217845777 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f277c078b90 0x7f277c07b0a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2790033000 tx=0x7f2790004730 comp rx=0 tx=0).stop 2024-09-21T07:36:21.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.176+0000 7f27a4000640 1 -- 172.21.15.39:0/217845777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f279c15d8a0 msgr2=0x7f279c1f9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:21.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.176+0000 7f27a4000640 1 --2- 172.21.15.39:0/217845777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f279c15d8a0 0x7f279c1f9770 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f279400cad0 tx=0x7f279400cf90 comp rx=0 tx=0).stop 2024-09-21T07:36:21.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.177+0000 7f27a4000640 1 -- 172.21.15.39:0/217845777 shutdown_connections 2024-09-21T07:36:21.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.177+0000 7f27a4000640 1 --2- 172.21.15.39:0/217845777 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f277c078b90 0x7f277c07b0a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:21.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.177+0000 7f27a4000640 1 --2- 172.21.15.39:0/217845777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f279c15d8a0 0x7f279c1f9770 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:21.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.177+0000 7f27a4000640 1 --2- 172.21.15.39:0/217845777 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f279c0ccb50 0x7f279c1f9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:21.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.177+0000 7f27a4000640 1 --2- 172.21.15.39:0/217845777 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f279c0cc1e0 0x7f279c15a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:21.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.177+0000 7f27a4000640 1 -- 172.21.15.39:0/217845777 >> 172.21.15.39:0/217845777 conn(0x7f279c06d7d0 msgr2=0x7f279c0ce890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:21.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.177+0000 7f27a4000640 1 -- 172.21.15.39:0/217845777 shutdown_connections 2024-09-21T07:36:21.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.177+0000 7f27a4000640 1 -- 172.21.15.39:0/217845777 wait complete. 2024-09-21T07:36:21.205 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:36:21.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.434+0000 7f98f2000640 1 -- 172.21.15.39:0/2326115191 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f98ec0af3c0 msgr2=0x7f98ec0b40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:21.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.434+0000 7f98f2000640 1 --2- 172.21.15.39:0/2326115191 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f98ec0af3c0 0x7f98ec0b40b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f98d4005fd0 tx=0x7f98d402a5d0 comp rx=0 tx=0).stop 2024-09-21T07:36:21.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.434+0000 7f98f2000640 1 -- 172.21.15.39:0/2326115191 shutdown_connections 2024-09-21T07:36:21.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.434+0000 7f98f2000640 1 --2- 172.21.15.39:0/2326115191 >> v2:172.21.15.148:3300/0 conn(0x7f98ec0ccb50 0x7f98ec0ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:21.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.434+0000 7f98f2000640 1 --2- 172.21.15.39:0/2326115191 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f98ec0af3c0 0x7f98ec0b40b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:21.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.434+0000 7f98f2000640 1 -- 172.21.15.39:0/2326115191 >> 172.21.15.39:0/2326115191 conn(0x7f98ec06d7d0 msgr2=0x7f98ec0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:21.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.434+0000 7f98f2000640 1 -- 172.21.15.39:0/2326115191 shutdown_connections 2024-09-21T07:36:21.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.434+0000 7f98f2000640 1 -- 172.21.15.39:0/2326115191 wait complete. 2024-09-21T07:36:21.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.436+0000 7f98f2000640 1 Processor -- start 2024-09-21T07:36:21.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.436+0000 7f98f2000640 1 -- start start 2024-09-21T07:36:21.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.438+0000 7f98f2000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f98ec0af3c0 0x7f98ec09bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:21.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.438+0000 7f98f2000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98ec0cc1e0 0x7f98ec09aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:21.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.438+0000 7f98f2000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98ec0ccb50 0x7f98ec154f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:21.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.438+0000 7f98f2000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec153520 con 0x7f98ec0cc1e0 2024-09-21T07:36:21.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.438+0000 7f98f2000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec1533a0 con 0x7f98ec0ccb50 2024-09-21T07:36:21.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.438+0000 7f98f2000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f98ec1536a0 con 0x7f98ec0af3c0 2024-09-21T07:36:21.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.438+0000 7f98e2a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98ec0cc1e0 0x7f98ec09aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:21.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.438+0000 7f98e2a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98ec0cc1e0 0x7f98ec09aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52776/0 (socket says 172.21.15.39:52776) 2024-09-21T07:36:21.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.438+0000 7f98e2a00640 1 -- 172.21.15.39:0/2868812595 learned_addr learned my addr 172.21.15.39:0/2868812595 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:21.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.438+0000 7f98eb400640 1 --2- 172.21.15.39:0/2868812595 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f98ec0af3c0 0x7f98ec09bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:21.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.438+0000 7f98ebe00640 1 --2- 172.21.15.39:0/2868812595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98ec0ccb50 0x7f98ec154f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:21.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.438+0000 7f98e2a00640 1 -- 172.21.15.39:0/2868812595 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f98ec0af3c0 msgr2=0x7f98ec09bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:21.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.438+0000 7f98e2a00640 1 --2- 172.21.15.39:0/2868812595 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f98ec0af3c0 0x7f98ec09bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:21.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.438+0000 7f98e2a00640 1 -- 172.21.15.39:0/2868812595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98ec0ccb50 msgr2=0x7f98ec154f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:21.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.438+0000 7f98e2a00640 1 --2- 172.21.15.39:0/2868812595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98ec0ccb50 0x7f98ec154f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:21.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.438+0000 7f98e2a00640 1 -- 172.21.15.39:0/2868812595 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98ec09af70 con 0x7f98ec0cc1e0 2024-09-21T07:36:21.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.438+0000 7f98eb400640 1 --2- 172.21.15.39:0/2868812595 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f98ec0af3c0 0x7f98ec09bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:36:21.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.438+0000 7f98ebe00640 1 --2- 172.21.15.39:0/2868812595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98ec0ccb50 0x7f98ec154f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:36:21.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.439+0000 7f98e2a00640 1 --2- 172.21.15.39:0/2868812595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98ec0cc1e0 0x7f98ec09aa30 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f98d4034040 tx=0x7f98d4005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:21.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.439+0000 7f98e3e00640 1 -- 172.21.15.39:0/2868812595 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d403e030 con 0x7f98ec0cc1e0 2024-09-21T07:36:21.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.439+0000 7f98e3e00640 1 -- 172.21.15.39:0/2868812595 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98d4003a70 con 0x7f98ec0cc1e0 2024-09-21T07:36:21.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.439+0000 7f98e3e00640 1 -- 172.21.15.39:0/2868812595 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d4035040 con 0x7f98ec0cc1e0 2024-09-21T07:36:21.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.439+0000 7f98f2000640 1 -- 172.21.15.39:0/2868812595 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98ec1f4400 con 0x7f98ec0cc1e0 2024-09-21T07:36:21.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.439+0000 7f98f2000640 1 -- 172.21.15.39:0/2868812595 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98ec154790 con 0x7f98ec0cc1e0 2024-09-21T07:36:21.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.440+0000 7f98f2000640 1 -- 172.21.15.39:0/2868812595 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98ec1f4170 con 0x7f98ec0cc1e0 2024-09-21T07:36:21.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.441+0000 7f98e3e00640 1 -- 172.21.15.39:0/2868812595 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f98d4041030 con 0x7f98ec0cc1e0 2024-09-21T07:36:21.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.442+0000 7f98e3e00640 1 --2- 172.21.15.39:0/2868812595 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f98c8078bc0 0x7f98c807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:21.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.443+0000 7f98e3e00640 1 -- 172.21.15.39:0/2868812595 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6800+0+0 (secure 0 0 0) 0x7f98d40bb5f0 con 0x7f98ec0cc1e0 2024-09-21T07:36:21.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.443+0000 7f98eb400640 1 --2- 172.21.15.39:0/2868812595 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f98c8078bc0 0x7f98c807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:21.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.444+0000 7f98eb400640 1 --2- 172.21.15.39:0/2868812595 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f98c8078bc0 0x7f98c807b0d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f98dc001680 tx=0x7f98dc009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:21.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.447+0000 7f98e3e00640 1 -- 172.21.15.39:0/2868812595 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f98d40835f0 con 0x7f98ec0cc1e0 2024-09-21T07:36:21.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.646+0000 7f98f2000640 1 -- 172.21.15.39:0/2868812595 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f98ec153bc0 con 0x7f98ec0cc1e0 2024-09-21T07:36:21.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.648+0000 7f98e3e00640 1 -- 172.21.15.39:0/2868812595 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22132 (secure 0 0 0) 0x7f98d4082ef0 con 0x7f98ec0cc1e0 2024-09-21T07:36:21.650 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:36:21.650 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":89,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:36:20.189332+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:36:18.142953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-21T07:36:19.149503+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:36:21.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.650+0000 7f98f2000640 1 -- 172.21.15.39:0/2868812595 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f98c8078bc0 msgr2=0x7f98c807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:21.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.650+0000 7f98f2000640 1 --2- 172.21.15.39:0/2868812595 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f98c8078bc0 0x7f98c807b0d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f98dc001680 tx=0x7f98dc009040 comp rx=0 tx=0).stop 2024-09-21T07:36:21.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.650+0000 7f98f2000640 1 -- 172.21.15.39:0/2868812595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98ec0cc1e0 msgr2=0x7f98ec09aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:21.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.650+0000 7f98f2000640 1 --2- 172.21.15.39:0/2868812595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98ec0cc1e0 0x7f98ec09aa30 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f98d4034040 tx=0x7f98d4005770 comp rx=0 tx=0).stop 2024-09-21T07:36:21.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.650+0000 7f98f2000640 1 -- 172.21.15.39:0/2868812595 shutdown_connections 2024-09-21T07:36:21.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.650+0000 7f98f2000640 1 --2- 172.21.15.39:0/2868812595 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f98c8078bc0 0x7f98c807b0d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:21.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.650+0000 7f98f2000640 1 --2- 172.21.15.39:0/2868812595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98ec0ccb50 0x7f98ec154f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:21.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.650+0000 7f98f2000640 1 --2- 172.21.15.39:0/2868812595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98ec0cc1e0 0x7f98ec09aa30 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:21.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.650+0000 7f98f2000640 1 --2- 172.21.15.39:0/2868812595 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f98ec0af3c0 0x7f98ec09bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:21.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.650+0000 7f98f2000640 1 -- 172.21.15.39:0/2868812595 >> 172.21.15.39:0/2868812595 conn(0x7f98ec06d7d0 msgr2=0x7f98ec1526d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:21.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.651+0000 7f98f2000640 1 -- 172.21.15.39:0/2868812595 shutdown_connections 2024-09-21T07:36:21.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.651+0000 7f98f2000640 1 -- 172.21.15.39:0/2868812595 wait complete. 2024-09-21T07:36:21.676 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T07:36:21.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.905+0000 7fbe7a400640 1 -- 172.21.15.39:0/3718758052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe740cf540 msgr2=0x7fbe7406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:21.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.905+0000 7fbe7a400640 1 --2- 172.21.15.39:0/3718758052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe740cf540 0x7fbe7406ee70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbe60005fd0 tx=0x7fbe6002a540 comp rx=0 tx=0).stop 2024-09-21T07:36:21.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.905+0000 7fbe7a400640 1 -- 172.21.15.39:0/3718758052 shutdown_connections 2024-09-21T07:36:21.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.905+0000 7fbe7a400640 1 --2- 172.21.15.39:0/3718758052 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe740cf950 0x7fbe7406f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:21.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.905+0000 7fbe7a400640 1 --2- 172.21.15.39:0/3718758052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe740cf540 0x7fbe7406ee70 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:21.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.905+0000 7fbe7a400640 1 -- 172.21.15.39:0/3718758052 >> 172.21.15.39:0/3718758052 conn(0x7fbe7406d410 msgr2=0x7fbe740b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:21.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.906+0000 7fbe7a400640 1 -- 172.21.15.39:0/3718758052 shutdown_connections 2024-09-21T07:36:21.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.906+0000 7fbe7a400640 1 -- 172.21.15.39:0/3718758052 wait complete. 2024-09-21T07:36:21.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.908+0000 7fbe7a400640 1 Processor -- start 2024-09-21T07:36:21.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.908+0000 7fbe7a400640 1 -- start start 2024-09-21T07:36:21.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.909+0000 7fbe7a400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe740cf540 0x7fbe74144740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:21.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.909+0000 7fbe7a400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe740cf950 0x7fbe741f62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:21.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.909+0000 7fbe7a400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe7415ca50 0x7fbe741591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:21.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.909+0000 7fbe7a400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7415ddd0 con 0x7fbe740cf540 2024-09-21T07:36:21.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.909+0000 7fbe7a400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7415dc50 con 0x7fbe740cf950 2024-09-21T07:36:21.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.909+0000 7fbe7a400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fbe7415df50 con 0x7fbe7415ca50 2024-09-21T07:36:21.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.910+0000 7fbe79000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe740cf540 0x7fbe74144740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:21.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.910+0000 7fbe79000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe740cf540 0x7fbe74144740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52792/0 (socket says 172.21.15.39:52792) 2024-09-21T07:36:21.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.910+0000 7fbe79000640 1 -- 172.21.15.39:0/513096706 learned_addr learned my addr 172.21.15.39:0/513096706 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:21.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.910+0000 7fbe79a00640 1 --2- 172.21.15.39:0/513096706 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe7415ca50 0x7fbe741591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:21.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.910+0000 7fbe73e00640 1 --2- 172.21.15.39:0/513096706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe740cf950 0x7fbe741f62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:21.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.910+0000 7fbe79000640 1 -- 172.21.15.39:0/513096706 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe7415ca50 msgr2=0x7fbe741591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:21.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.910+0000 7fbe79000640 1 --2- 172.21.15.39:0/513096706 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe7415ca50 0x7fbe741591e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:21.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.910+0000 7fbe79000640 1 -- 172.21.15.39:0/513096706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe740cf950 msgr2=0x7fbe741f62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:21.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.910+0000 7fbe79000640 1 --2- 172.21.15.39:0/513096706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe740cf950 0x7fbe741f62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:21.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.910+0000 7fbe79000640 1 -- 172.21.15.39:0/513096706 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe74157320 con 0x7fbe740cf540 2024-09-21T07:36:21.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.911+0000 7fbe79000640 1 --2- 172.21.15.39:0/513096706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe740cf540 0x7fbe74144740 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fbe6800c340 tx=0x7fbe6800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:21.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.911+0000 7fbe70c00640 1 -- 172.21.15.39:0/513096706 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe68017070 con 0x7fbe740cf540 2024-09-21T07:36:21.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.911+0000 7fbe70c00640 1 -- 172.21.15.39:0/513096706 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe680053c0 con 0x7fbe740cf540 2024-09-21T07:36:21.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.911+0000 7fbe70c00640 1 -- 172.21.15.39:0/513096706 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe68003850 con 0x7fbe740cf540 2024-09-21T07:36:21.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.911+0000 7fbe7a400640 1 -- 172.21.15.39:0/513096706 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe74157800 con 0x7fbe740cf540 2024-09-21T07:36:21.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.911+0000 7fbe7a400640 1 -- 172.21.15.39:0/513096706 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe741f5aa0 con 0x7fbe740cf540 2024-09-21T07:36:21.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.912+0000 7fbe7a400640 1 -- 172.21.15.39:0/513096706 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe7406fb50 con 0x7fbe740cf540 2024-09-21T07:36:21.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.913+0000 7fbe70c00640 1 -- 172.21.15.39:0/513096706 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbe6801a070 con 0x7fbe740cf540 2024-09-21T07:36:21.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.914+0000 7fbe70c00640 1 --2- 172.21.15.39:0/513096706 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbe54078bc0 0x7fbe5407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:21.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.914+0000 7fbe70c00640 1 -- 172.21.15.39:0/513096706 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6800+0+0 (secure 0 0 0) 0x7fbe6809a4e0 con 0x7fbe740cf540 2024-09-21T07:36:21.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.915+0000 7fbe73e00640 1 --2- 172.21.15.39:0/513096706 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbe54078bc0 0x7fbe5407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:21.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.916+0000 7fbe73e00640 1 --2- 172.21.15.39:0/513096706 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbe54078bc0 0x7fbe5407b0d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbe60033000 tx=0x7fbe60004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:21.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:21.919+0000 7fbe70c00640 1 -- 172.21.15.39:0/513096706 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbe74157320 con 0x7fbe740cf540 2024-09-21T07:36:22.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.153+0000 7fbe7a400640 1 -- 172.21.15.39:0/513096706 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fbe740b3b10 con 0x7fbe740cf540 2024-09-21T07:36:22.642 INFO:tasks.ceph.mds.c.smithi039.stderr:2024-09-21T07:36:22.640+0000 7f3c21c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:36:22.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.702+0000 7fbe70c00640 1 -- 172.21.15.39:0/513096706 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v53) ==== 118+0+0 (secure 0 0 0) 0x7fbe68062470 con 0x7fbe740cf540 2024-09-21T07:36:22.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.706+0000 7fbe7a400640 1 -- 172.21.15.39:0/513096706 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbe54078bc0 msgr2=0x7fbe5407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:22.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.706+0000 7fbe7a400640 1 --2- 172.21.15.39:0/513096706 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbe54078bc0 0x7fbe5407b0d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbe60033000 tx=0x7fbe60004730 comp rx=0 tx=0).stop 2024-09-21T07:36:22.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.706+0000 7fbe7a400640 1 -- 172.21.15.39:0/513096706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe740cf540 msgr2=0x7fbe74144740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:22.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.706+0000 7fbe7a400640 1 --2- 172.21.15.39:0/513096706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe740cf540 0x7fbe74144740 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fbe6800c340 tx=0x7fbe6800c800 comp rx=0 tx=0).stop 2024-09-21T07:36:22.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.706+0000 7fbe7a400640 1 -- 172.21.15.39:0/513096706 shutdown_connections 2024-09-21T07:36:22.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.706+0000 7fbe7a400640 1 --2- 172.21.15.39:0/513096706 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbe54078bc0 0x7fbe5407b0d0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:22.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.706+0000 7fbe7a400640 1 --2- 172.21.15.39:0/513096706 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe7415ca50 0x7fbe741591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:22.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.706+0000 7fbe7a400640 1 --2- 172.21.15.39:0/513096706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe740cf950 0x7fbe741f62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:22.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.706+0000 7fbe7a400640 1 --2- 172.21.15.39:0/513096706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe740cf540 0x7fbe74144740 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:22.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.706+0000 7fbe7a400640 1 -- 172.21.15.39:0/513096706 >> 172.21.15.39:0/513096706 conn(0x7fbe7406d410 msgr2=0x7fbe74145690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:22.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.706+0000 7fbe7a400640 1 -- 172.21.15.39:0/513096706 shutdown_connections 2024-09-21T07:36:22.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.706+0000 7fbe7a400640 1 -- 172.21.15.39:0/513096706 wait complete. 2024-09-21T07:36:22.733 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T07:36:22.733 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T07:36:22.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.963+0000 7f5ed5400640 1 -- 172.21.15.39:0/2756180915 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ed00cad30 msgr2=0x7f5ed006ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:22.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.963+0000 7f5ed5400640 1 --2- 172.21.15.39:0/2756180915 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ed00cad30 0x7f5ed006ffa0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f5ed00c71a0 tx=0x7f5eb802a560 comp rx=0 tx=0).stop 2024-09-21T07:36:22.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.963+0000 7f5ed5400640 1 -- 172.21.15.39:0/2756180915 shutdown_connections 2024-09-21T07:36:22.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.963+0000 7f5ed5400640 1 --2- 172.21.15.39:0/2756180915 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5ed00704e0 0x7f5ed00cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:22.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.963+0000 7f5ed5400640 1 --2- 172.21.15.39:0/2756180915 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ed00cad30 0x7f5ed006ffa0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:22.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.963+0000 7f5ed5400640 1 --2- 172.21.15.39:0/2756180915 >> v2:172.21.15.148:3300/0 conn(0x7f5ed00ca800 0x7f5ed00cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:22.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.963+0000 7f5ed5400640 1 -- 172.21.15.39:0/2756180915 >> 172.21.15.39:0/2756180915 conn(0x7f5ed00b1cb0 msgr2=0x7f5ed00b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:22.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.963+0000 7f5ed5400640 1 -- 172.21.15.39:0/2756180915 shutdown_connections 2024-09-21T07:36:22.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.963+0000 7f5ed5400640 1 -- 172.21.15.39:0/2756180915 wait complete. 2024-09-21T07:36:22.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.966+0000 7f5ed5400640 1 Processor -- start 2024-09-21T07:36:22.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.966+0000 7f5ed5400640 1 -- start start 2024-09-21T07:36:22.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.967+0000 7f5ed5400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ed00704e0 0x7f5ed015a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:22.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.967+0000 7f5ed5400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ed00ca800 0x7f5ed00c66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:22.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.967+0000 7f5ed5400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5ed00cad30 0x7f5ed015d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:22.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.967+0000 7f5ed5400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed00cd140 con 0x7f5ed00ca800 2024-09-21T07:36:22.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.967+0000 7f5ed5400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed00ccfc0 con 0x7f5ed00704e0 2024-09-21T07:36:22.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.967+0000 7f5ed5400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f5ed00cd2c0 con 0x7f5ed00cad30 2024-09-21T07:36:22.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.968+0000 7f5ec5e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ed00ca800 0x7f5ed00c66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:22.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.968+0000 7f5ec5e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ed00ca800 0x7f5ed00c66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52154/0 (socket says 172.21.15.39:52154) 2024-09-21T07:36:22.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.968+0000 7f5ec5e00640 1 -- 172.21.15.39:0/632204331 learned_addr learned my addr 172.21.15.39:0/632204331 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:22.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.968+0000 7f5ece800640 1 --2- 172.21.15.39:0/632204331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ed00704e0 0x7f5ed015a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:22.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.968+0000 7f5ecf200640 1 --2- 172.21.15.39:0/632204331 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5ed00cad30 0x7f5ed015d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:22.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.968+0000 7f5ec5e00640 1 -- 172.21.15.39:0/632204331 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5ed00cad30 msgr2=0x7f5ed015d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:22.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.968+0000 7f5ec5e00640 1 --2- 172.21.15.39:0/632204331 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5ed00cad30 0x7f5ed015d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:22.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.968+0000 7f5ec5e00640 1 -- 172.21.15.39:0/632204331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ed00704e0 msgr2=0x7f5ed015a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:22.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.968+0000 7f5ec5e00640 1 --2- 172.21.15.39:0/632204331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ed00704e0 0x7f5ed015a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:22.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.968+0000 7f5ec5e00640 1 -- 172.21.15.39:0/632204331 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed00c74c0 con 0x7f5ed00ca800 2024-09-21T07:36:22.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.968+0000 7f5ece800640 1 --2- 172.21.15.39:0/632204331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ed00704e0 0x7f5ed015a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:36:22.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.969+0000 7f5ec5e00640 1 --2- 172.21.15.39:0/632204331 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ed00ca800 0x7f5ed00c66e0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8034040 tx=0x7f5eb80039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:22.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.969+0000 7f5ec7400640 1 -- 172.21.15.39:0/632204331 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb803e030 con 0x7f5ed00ca800 2024-09-21T07:36:22.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.969+0000 7f5ec7400640 1 -- 172.21.15.39:0/632204331 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5eb80047a0 con 0x7f5ed00ca800 2024-09-21T07:36:22.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.969+0000 7f5ec7400640 1 -- 172.21.15.39:0/632204331 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8035040 con 0x7f5ed00ca800 2024-09-21T07:36:22.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.969+0000 7f5ed5400640 1 -- 172.21.15.39:0/632204331 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ed00ca140 con 0x7f5ed00ca800 2024-09-21T07:36:22.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.969+0000 7f5ed5400640 1 -- 172.21.15.39:0/632204331 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ed00c6c20 con 0x7f5ed00ca800 2024-09-21T07:36:22.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.970+0000 7f5ed5400640 1 -- 172.21.15.39:0/632204331 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ed00cba00 con 0x7f5ed00ca800 2024-09-21T07:36:22.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.972+0000 7f5ec7400640 1 -- 172.21.15.39:0/632204331 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5eb8041030 con 0x7f5ed00ca800 2024-09-21T07:36:22.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.972+0000 7f5ec7400640 1 --2- 172.21.15.39:0/632204331 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5eac078bc0 0x7f5eac07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:22.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.973+0000 7f5ec7400640 1 -- 172.21.15.39:0/632204331 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6800+0+0 (secure 0 0 0) 0x7f5eb80bb8c0 con 0x7f5ed00ca800 2024-09-21T07:36:22.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.973+0000 7f5ece800640 1 --2- 172.21.15.39:0/632204331 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5eac078bc0 0x7f5eac07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:22.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.974+0000 7f5ece800640 1 --2- 172.21.15.39:0/632204331 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5eac078bc0 0x7f5eac07b0d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0005cc0 tx=0x7f5ec0005bf0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:22.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:22.977+0000 7f5ec7400640 1 -- 172.21.15.39:0/632204331 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5eb8083390 con 0x7f5ed00ca800 2024-09-21T07:36:23.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.219+0000 7f5ed5400640 1 -- 172.21.15.39:0/632204331 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f5ed00c90a0 con 0x7f5ed00ca800 2024-09-21T07:36:23.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.709+0000 7f5ec7400640 1 -- 172.21.15.39:0/632204331 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v54) ==== 122+0+0 (secure 0 0 0) 0x7f5eb8087dc0 con 0x7f5ed00ca800 2024-09-21T07:36:23.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.712+0000 7f5ed5400640 1 -- 172.21.15.39:0/632204331 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5eac078bc0 msgr2=0x7f5eac07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:23.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.712+0000 7f5ed5400640 1 --2- 172.21.15.39:0/632204331 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5eac078bc0 0x7f5eac07b0d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0005cc0 tx=0x7f5ec0005bf0 comp rx=0 tx=0).stop 2024-09-21T07:36:23.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.712+0000 7f5ed5400640 1 -- 172.21.15.39:0/632204331 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ed00ca800 msgr2=0x7f5ed00c66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:23.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.712+0000 7f5ed5400640 1 --2- 172.21.15.39:0/632204331 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ed00ca800 0x7f5ed00c66e0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8034040 tx=0x7f5eb80039e0 comp rx=0 tx=0).stop 2024-09-21T07:36:23.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.712+0000 7f5ed5400640 1 -- 172.21.15.39:0/632204331 shutdown_connections 2024-09-21T07:36:23.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.712+0000 7f5ed5400640 1 --2- 172.21.15.39:0/632204331 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5eac078bc0 0x7f5eac07b0d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:23.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.712+0000 7f5ed5400640 1 --2- 172.21.15.39:0/632204331 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5ed00cad30 0x7f5ed015d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:23.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.712+0000 7f5ed5400640 1 --2- 172.21.15.39:0/632204331 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ed00ca800 0x7f5ed00c66e0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:23.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.712+0000 7f5ed5400640 1 --2- 172.21.15.39:0/632204331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ed00704e0 0x7f5ed015a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:23.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.712+0000 7f5ed5400640 1 -- 172.21.15.39:0/632204331 >> 172.21.15.39:0/632204331 conn(0x7f5ed00b1cb0 msgr2=0x7f5ed00b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:23.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.712+0000 7f5ed5400640 1 -- 172.21.15.39:0/632204331 shutdown_connections 2024-09-21T07:36:23.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.712+0000 7f5ed5400640 1 -- 172.21.15.39:0/632204331 wait complete. 2024-09-21T07:36:23.737 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T07:36:23.737 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:36:23.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.964+0000 7f83b3a00640 1 -- 172.21.15.39:0/170873223 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83ac15ef10 msgr2=0x7f83ac0cc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:23.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.964+0000 7f83b3a00640 1 --2- 172.21.15.39:0/170873223 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83ac15ef10 0x7f83ac0cc970 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8398005fd0 tx=0x7f839802a4b0 comp rx=0 tx=0).stop 2024-09-21T07:36:23.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.964+0000 7f83b3a00640 1 -- 172.21.15.39:0/170873223 shutdown_connections 2024-09-21T07:36:23.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.964+0000 7f83b3a00640 1 --2- 172.21.15.39:0/170873223 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83ac15ef10 0x7f83ac0cc970 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:23.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.964+0000 7f83b3a00640 1 --2- 172.21.15.39:0/170873223 >> v2:172.21.15.148:3300/0 conn(0x7f83ac15a9b0 0x7f83ac0cfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:23.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.964+0000 7f83b3a00640 1 -- 172.21.15.39:0/170873223 >> 172.21.15.39:0/170873223 conn(0x7f83ac06d7d0 msgr2=0x7f83ac0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:23.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.964+0000 7f83b3a00640 1 -- 172.21.15.39:0/170873223 shutdown_connections 2024-09-21T07:36:23.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.964+0000 7f83b3a00640 1 -- 172.21.15.39:0/170873223 wait complete. 2024-09-21T07:36:23.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.966+0000 7f83b3a00640 1 Processor -- start 2024-09-21T07:36:23.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.966+0000 7f83b3a00640 1 -- start start 2024-09-21T07:36:23.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.968+0000 7f83b3a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83ac0cceb0 0x7f83ac14bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:23.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.968+0000 7f83b3a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83ac15a9b0 0x7f83ac09dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:23.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.968+0000 7f83b3a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83ac15ef10 0x7f83ac0ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:23.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.968+0000 7f83b3a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f83ac0cc760 con 0x7f83ac15ef10 2024-09-21T07:36:23.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.968+0000 7f83b3a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f83ac0cc5e0 con 0x7f83ac15a9b0 2024-09-21T07:36:23.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.968+0000 7f83b3a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f83ac070f20 con 0x7f83ac0cceb0 2024-09-21T07:36:23.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.968+0000 7f83b1a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83ac15ef10 0x7f83ac0ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:23.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.968+0000 7f83b1a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83ac15ef10 0x7f83ac0ca9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52158/0 (socket says 172.21.15.39:52158) 2024-09-21T07:36:23.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.968+0000 7f83b1a00640 1 -- 172.21.15.39:0/1369560200 learned_addr learned my addr 172.21.15.39:0/1369560200 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:23.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.968+0000 7f83b1000640 1 --2- 172.21.15.39:0/1369560200 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83ac0cceb0 0x7f83ac14bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:23.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.968+0000 7f83abe00640 1 --2- 172.21.15.39:0/1369560200 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83ac15a9b0 0x7f83ac09dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:23.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.969+0000 7f83b1a00640 1 -- 172.21.15.39:0/1369560200 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83ac0cceb0 msgr2=0x7f83ac14bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:23.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.969+0000 7f83b1a00640 1 --2- 172.21.15.39:0/1369560200 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83ac0cceb0 0x7f83ac14bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:23.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.969+0000 7f83b1a00640 1 -- 172.21.15.39:0/1369560200 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83ac15a9b0 msgr2=0x7f83ac09dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:23.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.969+0000 7f83b1a00640 1 --2- 172.21.15.39:0/1369560200 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83ac15a9b0 0x7f83ac09dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:23.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.969+0000 7f83b1a00640 1 -- 172.21.15.39:0/1369560200 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83ac09aa30 con 0x7f83ac15ef10 2024-09-21T07:36:23.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.969+0000 7f83b1a00640 1 --2- 172.21.15.39:0/1369560200 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83ac15ef10 0x7f83ac0ca9f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f83a400c730 tx=0x7f83a400cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:23.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.969+0000 7f83a8c00640 1 -- 172.21.15.39:0/1369560200 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a4003b70 con 0x7f83ac15ef10 2024-09-21T07:36:23.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.969+0000 7f83a8c00640 1 -- 172.21.15.39:0/1369560200 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f83a4003d10 con 0x7f83ac15ef10 2024-09-21T07:36:23.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.969+0000 7f83b3a00640 1 -- 172.21.15.39:0/1369560200 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83ac09af30 con 0x7f83ac15ef10 2024-09-21T07:36:23.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.969+0000 7f83a8c00640 1 -- 172.21.15.39:0/1369560200 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a4005330 con 0x7f83ac15ef10 2024-09-21T07:36:23.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.970+0000 7f83b3a00640 1 -- 172.21.15.39:0/1369560200 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83ac09b4f0 con 0x7f83ac15ef10 2024-09-21T07:36:23.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.970+0000 7f83b3a00640 1 -- 172.21.15.39:0/1369560200 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83ac09aca0 con 0x7f83ac15ef10 2024-09-21T07:36:23.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.972+0000 7f83a8c00640 1 -- 172.21.15.39:0/1369560200 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f83a40054d0 con 0x7f83ac15ef10 2024-09-21T07:36:23.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.973+0000 7f83a8c00640 1 --2- 172.21.15.39:0/1369560200 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f838c078b90 0x7f838c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:23.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.973+0000 7f83a8c00640 1 -- 172.21.15.39:0/1369560200 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6800+0+0 (secure 0 0 0) 0x7f83a409ad40 con 0x7f83ac15ef10 2024-09-21T07:36:23.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.973+0000 7f83b1000640 1 --2- 172.21.15.39:0/1369560200 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f838c078b90 0x7f838c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:23.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.974+0000 7f83b1000640 1 --2- 172.21.15.39:0/1369560200 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f838c078b90 0x7f838c07b0a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f83a00095f0 tx=0x7f83a0005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:23.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:23.978+0000 7f83a8c00640 1 -- 172.21.15.39:0/1369560200 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f83ac0b4780 con 0x7f83ac15ef10 2024-09-21T07:36:24.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.217+0000 7f83b3a00640 1 -- 172.21.15.39:0/1369560200 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83ac0cfae0 con 0x7f83ac15ef10 2024-09-21T07:36:24.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.217+0000 7f83a8c00640 1 -- 172.21.15.39:0/1369560200 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5225 (secure 0 0 0) 0x7f83a4062790 con 0x7f83ac15ef10 2024-09-21T07:36:24.219 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:36:24.219 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":54,"btime":"2024-09-21T07:36:23:704206+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5621,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3026827614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3026827614},{"type":"v1","addr":"172.21.15.148:6837","nonce":3026827614}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6115,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:36:23.220980+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6321},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6321":{"gid":6321,"name":"c","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/829716143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":829716143},{"type":"v1","addr":"172.21.15.39:6834","nonce":829716143}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6321,"qdb_cluster":[6321]},"id":6}]} 2024-09-21T07:36:24.220 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 54 2024-09-21T07:36:24.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.220+0000 7f83b3a00640 1 -- 172.21.15.39:0/1369560200 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f838c078b90 msgr2=0x7f838c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:24.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.220+0000 7f83b3a00640 1 --2- 172.21.15.39:0/1369560200 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f838c078b90 0x7f838c07b0a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f83a00095f0 tx=0x7f83a0005f90 comp rx=0 tx=0).stop 2024-09-21T07:36:24.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.220+0000 7f83b3a00640 1 -- 172.21.15.39:0/1369560200 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83ac15ef10 msgr2=0x7f83ac0ca9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:24.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.220+0000 7f83b3a00640 1 --2- 172.21.15.39:0/1369560200 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83ac15ef10 0x7f83ac0ca9f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f83a400c730 tx=0x7f83a400cb30 comp rx=0 tx=0).stop 2024-09-21T07:36:24.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.220+0000 7f83b3a00640 1 -- 172.21.15.39:0/1369560200 shutdown_connections 2024-09-21T07:36:24.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.220+0000 7f83b3a00640 1 --2- 172.21.15.39:0/1369560200 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f838c078b90 0x7f838c07b0a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:24.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.220+0000 7f83b3a00640 1 --2- 172.21.15.39:0/1369560200 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83ac15ef10 0x7f83ac0ca9f0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:24.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.220+0000 7f83b3a00640 1 --2- 172.21.15.39:0/1369560200 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83ac15a9b0 0x7f83ac09dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:24.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.220+0000 7f83b3a00640 1 --2- 172.21.15.39:0/1369560200 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83ac0cceb0 0x7f83ac14bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:24.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.220+0000 7f83b3a00640 1 -- 172.21.15.39:0/1369560200 >> 172.21.15.39:0/1369560200 conn(0x7f83ac06d7d0 msgr2=0x7f83ac0ae440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:24.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.220+0000 7f83b3a00640 1 -- 172.21.15.39:0/1369560200 shutdown_connections 2024-09-21T07:36:24.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.220+0000 7f83b3a00640 1 -- 172.21.15.39:0/1369560200 wait complete. 2024-09-21T07:36:24.248 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:36:24.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.472+0000 7f06f8e00640 1 -- 172.21.15.39:0/1127727204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06f415fbd0 msgr2=0x7f06f4152360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:24.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.472+0000 7f06f8e00640 1 --2- 172.21.15.39:0/1127727204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06f415fbd0 0x7f06f4152360 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f06f4070db0 tx=0x7f06e402a560 comp rx=0 tx=0).stop 2024-09-21T07:36:24.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.472+0000 7f06f8e00640 1 -- 172.21.15.39:0/1127727204 shutdown_connections 2024-09-21T07:36:24.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.472+0000 7f06f8e00640 1 --2- 172.21.15.39:0/1127727204 >> v2:172.21.15.148:3300/0 conn(0x7f06f4073040 0x7f06f415f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:24.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.472+0000 7f06f8e00640 1 --2- 172.21.15.39:0/1127727204 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f06f4072c30 0x7f06f415ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:24.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.472+0000 7f06f8e00640 1 --2- 172.21.15.39:0/1127727204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06f415fbd0 0x7f06f4152360 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:24.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.472+0000 7f06f8e00640 1 -- 172.21.15.39:0/1127727204 >> 172.21.15.39:0/1127727204 conn(0x7f06f406d410 msgr2=0x7f06f40b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:24.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.472+0000 7f06f8e00640 1 -- 172.21.15.39:0/1127727204 shutdown_connections 2024-09-21T07:36:24.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.472+0000 7f06f8e00640 1 -- 172.21.15.39:0/1127727204 wait complete. 2024-09-21T07:36:24.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.474+0000 7f06f8e00640 1 Processor -- start 2024-09-21T07:36:24.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.474+0000 7f06f8e00640 1 -- start start 2024-09-21T07:36:24.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.476+0000 7f06f8e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06f4072c30 0x7f06f41349d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:24.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.476+0000 7f06f8e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f06f4073040 0x7f06f41483f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:24.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.476+0000 7f06f8e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06f40b4680 0x7f06f4136af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:24.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.476+0000 7f06f8e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f06f415c1e0 con 0x7f06f40b4680 2024-09-21T07:36:24.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.476+0000 7f06f8e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f06f415c060 con 0x7f06f4072c30 2024-09-21T07:36:24.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.476+0000 7f06f8e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f06f415c360 con 0x7f06f4073040 2024-09-21T07:36:24.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.476+0000 7f06f3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06f40b4680 0x7f06f4136af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:24.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.476+0000 7f06f3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06f40b4680 0x7f06f4136af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52188/0 (socket says 172.21.15.39:52188) 2024-09-21T07:36:24.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.476+0000 7f06f3e00640 1 -- 172.21.15.39:0/2179406896 learned_addr learned my addr 172.21.15.39:0/2179406896 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:24.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.476+0000 7f06ebe00640 1 --2- 172.21.15.39:0/2179406896 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f06f4073040 0x7f06f41483f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:24.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.476+0000 7f06f3400640 1 --2- 172.21.15.39:0/2179406896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06f4072c30 0x7f06f41349d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:24.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.476+0000 7f06f3e00640 1 -- 172.21.15.39:0/2179406896 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f06f4073040 msgr2=0x7f06f41483f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:24.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.477+0000 7f06f3e00640 1 --2- 172.21.15.39:0/2179406896 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f06f4073040 0x7f06f41483f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:24.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.477+0000 7f06f3e00640 1 -- 172.21.15.39:0/2179406896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06f4072c30 msgr2=0x7f06f41349d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:24.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.477+0000 7f06f3e00640 1 --2- 172.21.15.39:0/2179406896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06f4072c30 0x7f06f41349d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:24.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.477+0000 7f06f3e00640 1 -- 172.21.15.39:0/2179406896 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f4152b20 con 0x7f06f40b4680 2024-09-21T07:36:24.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.477+0000 7f06f3e00640 1 --2- 172.21.15.39:0/2179406896 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06f40b4680 0x7f06f4136af0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f06ec00ea70 tx=0x7f06ec00ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:24.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.477+0000 7f06eb400640 1 -- 172.21.15.39:0/2179406896 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ec00ce40 con 0x7f06f40b4680 2024-09-21T07:36:24.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.477+0000 7f06eb400640 1 -- 172.21.15.39:0/2179406896 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06ec004c10 con 0x7f06f40b4680 2024-09-21T07:36:24.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.477+0000 7f06eb400640 1 -- 172.21.15.39:0/2179406896 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ec005020 con 0x7f06f40b4680 2024-09-21T07:36:24.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.477+0000 7f06f8e00640 1 -- 172.21.15.39:0/2179406896 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06f4137630 con 0x7f06f40b4680 2024-09-21T07:36:24.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.477+0000 7f06f8e00640 1 -- 172.21.15.39:0/2179406896 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06f41530a0 con 0x7f06f40b4680 2024-09-21T07:36:24.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.478+0000 7f06f8e00640 1 -- 172.21.15.39:0/2179406896 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06f415fbd0 con 0x7f06f40b4680 2024-09-21T07:36:24.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.480+0000 7f06eb400640 1 -- 172.21.15.39:0/2179406896 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f06ec0051c0 con 0x7f06f40b4680 2024-09-21T07:36:24.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.480+0000 7f06eb400640 1 --2- 172.21.15.39:0/2179406896 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f06d0078bc0 0x7f06d007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:24.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.481+0000 7f06eb400640 1 -- 172.21.15.39:0/2179406896 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6800+0+0 (secure 0 0 0) 0x7f06ec09bcc0 con 0x7f06f40b4680 2024-09-21T07:36:24.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.481+0000 7f06f3400640 1 --2- 172.21.15.39:0/2179406896 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f06d0078bc0 0x7f06d007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:24.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.482+0000 7f06f3400640 1 --2- 172.21.15.39:0/2179406896 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f06d0078bc0 0x7f06d007b0d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f06e4033000 tx=0x7f06e4004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:24.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.485+0000 7f06eb400640 1 -- 172.21.15.39:0/2179406896 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f06ec063700 con 0x7f06f40b4680 2024-09-21T07:36:24.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.682+0000 7f06f8e00640 1 -- 172.21.15.39:0/2179406896 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f06f40750a0 con 0x7f06f40b4680 2024-09-21T07:36:24.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.683+0000 7f06eb400640 1 -- 172.21.15.39:0/2179406896 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22119 (secure 0 0 0) 0x7f06ec068130 con 0x7f06f40b4680 2024-09-21T07:36:24.686 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:36:24.686 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":90,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:36:21.669422+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:36:18.142953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-21T07:36:19.149503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:36:24.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.686+0000 7f06f8e00640 1 -- 172.21.15.39:0/2179406896 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f06d0078bc0 msgr2=0x7f06d007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:24.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.686+0000 7f06f8e00640 1 --2- 172.21.15.39:0/2179406896 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f06d0078bc0 0x7f06d007b0d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f06e4033000 tx=0x7f06e4004730 comp rx=0 tx=0).stop 2024-09-21T07:36:24.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.686+0000 7f06f8e00640 1 -- 172.21.15.39:0/2179406896 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06f40b4680 msgr2=0x7f06f4136af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:24.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.686+0000 7f06f8e00640 1 --2- 172.21.15.39:0/2179406896 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06f40b4680 0x7f06f4136af0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f06ec00ea70 tx=0x7f06ec00ef30 comp rx=0 tx=0).stop 2024-09-21T07:36:24.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.686+0000 7f06f8e00640 1 -- 172.21.15.39:0/2179406896 shutdown_connections 2024-09-21T07:36:24.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.686+0000 7f06f8e00640 1 --2- 172.21.15.39:0/2179406896 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f06d0078bc0 0x7f06d007b0d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:24.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.686+0000 7f06f8e00640 1 --2- 172.21.15.39:0/2179406896 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06f40b4680 0x7f06f4136af0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:24.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.686+0000 7f06f8e00640 1 --2- 172.21.15.39:0/2179406896 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f06f4073040 0x7f06f41483f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:24.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.686+0000 7f06f8e00640 1 --2- 172.21.15.39:0/2179406896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06f4072c30 0x7f06f41349d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:24.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.686+0000 7f06f8e00640 1 -- 172.21.15.39:0/2179406896 >> 172.21.15.39:0/2179406896 conn(0x7f06f406d410 msgr2=0x7f06f4150c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:24.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.686+0000 7f06f8e00640 1 -- 172.21.15.39:0/2179406896 shutdown_connections 2024-09-21T07:36:24.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.686+0000 7f06f8e00640 1 -- 172.21.15.39:0/2179406896 wait complete. 2024-09-21T07:36:24.710 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T07:36:24.710 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:36:24.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.935+0000 7fcde2000640 1 -- 172.21.15.39:0/1820731341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcdc0003a50 msgr2=0x7fcdc0003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:24.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.935+0000 7fcde2000640 1 --2- 172.21.15.39:0/1820731341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcdc0003a50 0x7fcdc0003ec0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc0016c0 tx=0x7fcdc0005830 comp rx=0 tx=0).stop 2024-09-21T07:36:24.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.935+0000 7fcde2000640 1 -- 172.21.15.39:0/1820731341 shutdown_connections 2024-09-21T07:36:24.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.935+0000 7fcde2000640 1 --2- 172.21.15.39:0/1820731341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcdc0003a50 0x7fcdc0003ec0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:24.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.935+0000 7fcde2000640 1 -- 172.21.15.39:0/1820731341 >> 172.21.15.39:0/1820731341 conn(0x7fcddc06d7d0 msgr2=0x7fcddc0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:24.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.935+0000 7fcde2000640 1 -- 172.21.15.39:0/1820731341 shutdown_connections 2024-09-21T07:36:24.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.935+0000 7fcde2000640 1 -- 172.21.15.39:0/1820731341 wait complete. 2024-09-21T07:36:24.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.938+0000 7fcde2000640 1 Processor -- start 2024-09-21T07:36:24.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.938+0000 7fcde2000640 1 -- start start 2024-09-21T07:36:24.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.939+0000 7fcde2000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdc0003a50 0x7fcddc0741a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:24.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.939+0000 7fcde2000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcddc157ce0 0x7fcddc151be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:24.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.939+0000 7fcde2000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcddc0adfe0 0x7fcddc0ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:24.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.939+0000 7fcde2000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc15dc30 con 0x7fcddc157ce0 2024-09-21T07:36:24.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.939+0000 7fcde2000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc15dab0 con 0x7fcdc0003a50 2024-09-21T07:36:24.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.939+0000 7fcde2000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fcddc15ddb0 con 0x7fcddc0adfe0 2024-09-21T07:36:24.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.939+0000 7fcdd3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcddc157ce0 0x7fcddc151be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:24.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.939+0000 7fcdd3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcddc157ce0 0x7fcddc151be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52212/0 (socket says 172.21.15.39:52212) 2024-09-21T07:36:24.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.939+0000 7fcdd3e00640 1 -- 172.21.15.39:0/3946472277 learned_addr learned my addr 172.21.15.39:0/3946472277 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:24.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.940+0000 7fcddb400640 1 --2- 172.21.15.39:0/3946472277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdc0003a50 0x7fcddc0741a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:24.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.940+0000 7fcddbe00640 1 --2- 172.21.15.39:0/3946472277 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcddc0adfe0 0x7fcddc0ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:24.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.940+0000 7fcdd3e00640 1 -- 172.21.15.39:0/3946472277 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcddc0adfe0 msgr2=0x7fcddc0ae3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:24.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.940+0000 7fcdd3e00640 1 --2- 172.21.15.39:0/3946472277 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcddc0adfe0 0x7fcddc0ae3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:24.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.940+0000 7fcdd3e00640 1 -- 172.21.15.39:0/3946472277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdc0003a50 msgr2=0x7fcddc0741a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:24.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.940+0000 7fcdd3e00640 1 --2- 172.21.15.39:0/3946472277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdc0003a50 0x7fcddc0741a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:24.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.940+0000 7fcdd3e00640 1 -- 172.21.15.39:0/3946472277 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcddc0ae930 con 0x7fcddc157ce0 2024-09-21T07:36:24.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.940+0000 7fcddbe00640 1 --2- 172.21.15.39:0/3946472277 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcddc0adfe0 0x7fcddc0ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:36:24.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.940+0000 7fcdd3e00640 1 --2- 172.21.15.39:0/3946472277 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcddc157ce0 0x7fcddc151be0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4011780 tx=0x7fcdc4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:24.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.940+0000 7fcdd3400640 1 -- 172.21.15.39:0/3946472277 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc401f070 con 0x7fcddc157ce0 2024-09-21T07:36:24.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.940+0000 7fcdd3400640 1 -- 172.21.15.39:0/3946472277 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcdc4011e90 con 0x7fcddc157ce0 2024-09-21T07:36:24.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.940+0000 7fcdd3400640 1 -- 172.21.15.39:0/3946472277 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc40126b0 con 0x7fcddc157ce0 2024-09-21T07:36:24.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.940+0000 7fcde2000640 1 -- 172.21.15.39:0/3946472277 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcddc072fd0 con 0x7fcddc157ce0 2024-09-21T07:36:24.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.941+0000 7fcde2000640 1 -- 172.21.15.39:0/3946472277 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcddc073590 con 0x7fcddc157ce0 2024-09-21T07:36:24.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.942+0000 7fcde2000640 1 -- 172.21.15.39:0/3946472277 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcddc0cd2e0 con 0x7fcddc157ce0 2024-09-21T07:36:24.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.943+0000 7fcdd3400640 1 -- 172.21.15.39:0/3946472277 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcdc4022070 con 0x7fcddc157ce0 2024-09-21T07:36:24.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.944+0000 7fcdd3400640 1 --2- 172.21.15.39:0/3946472277 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcdb8078bc0 0x7fcdb807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:24.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.944+0000 7fcdd3400640 1 -- 172.21.15.39:0/3946472277 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6800+0+0 (secure 0 0 0) 0x7fcdc409b980 con 0x7fcddc157ce0 2024-09-21T07:36:24.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.944+0000 7fcddb400640 1 --2- 172.21.15.39:0/3946472277 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcdb8078bc0 0x7fcdb807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:24.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.945+0000 7fcddb400640 1 --2- 172.21.15.39:0/3946472277 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcdb8078bc0 0x7fcdb807b0d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc032000 tx=0x7fcdcc001190 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:24.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:24.949+0000 7fcdd3400640 1 -- 172.21.15.39:0/3946472277 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcddc0ae930 con 0x7fcddc157ce0 2024-09-21T07:36:25.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.142+0000 7fcde2000640 1 -- 172.21.15.39:0/3946472277 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcddc073d90 con 0x7fcdb8078bc0 2024-09-21T07:36:25.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.151+0000 7fcdd3400640 1 -- 172.21.15.39:0/3946472277 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+494676 (secure 0 0 0) 0x7fcddc073d90 con 0x7fcdb8078bc0 2024-09-21T07:36:25.159 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:36:25.162 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:36:25.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.158+0000 7fcde2000640 1 -- 172.21.15.39:0/3946472277 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcdb8078bc0 msgr2=0x7fcdb807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:25.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.158+0000 7fcde2000640 1 --2- 172.21.15.39:0/3946472277 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcdb8078bc0 0x7fcdb807b0d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc032000 tx=0x7fcdcc001190 comp rx=0 tx=0).stop 2024-09-21T07:36:25.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.158+0000 7fcde2000640 1 -- 172.21.15.39:0/3946472277 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcddc157ce0 msgr2=0x7fcddc151be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:25.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.158+0000 7fcde2000640 1 --2- 172.21.15.39:0/3946472277 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcddc157ce0 0x7fcddc151be0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4011780 tx=0x7fcdc4011c40 comp rx=0 tx=0).stop 2024-09-21T07:36:25.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.158+0000 7fcde2000640 1 -- 172.21.15.39:0/3946472277 shutdown_connections 2024-09-21T07:36:25.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.158+0000 7fcde2000640 1 --2- 172.21.15.39:0/3946472277 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcdb8078bc0 0x7fcdb807b0d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:25.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.158+0000 7fcde2000640 1 --2- 172.21.15.39:0/3946472277 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcddc0adfe0 0x7fcddc0ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:25.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.158+0000 7fcde2000640 1 --2- 172.21.15.39:0/3946472277 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcddc157ce0 0x7fcddc151be0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:25.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.158+0000 7fcde2000640 1 --2- 172.21.15.39:0/3946472277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdc0003a50 0x7fcddc0741a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:25.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.158+0000 7fcde2000640 1 -- 172.21.15.39:0/3946472277 >> 172.21.15.39:0/3946472277 conn(0x7fcddc06d7d0 msgr2=0x7fcddc0b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:25.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.158+0000 7fcde2000640 1 -- 172.21.15.39:0/3946472277 shutdown_connections 2024-09-21T07:36:25.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.158+0000 7fcde2000640 1 -- 172.21.15.39:0/3946472277 wait complete. 2024-09-21T07:36:25.183 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":186,"stamp":"2024-09-21T07:36:23.586363+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":312,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259284,"kb_used_data":9740,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715436,"statfs":{"total":773094113280,"available":772828606464,"internally_reserved":0,"allocated":9973760,"data_stored":5279713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.924856"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676183+0000","last_change":"2024-09-21T07:36:20.711879+0000","last_active":"2024-09-21T07:36:21.676183+0000","last_peered":"2024-09-21T07:36:21.676183+0000","last_clean":"2024-09-21T07:36:21.676183+0000","last_became_active":"2024-09-21T07:36:20.711145+0000","last_became_peered":"2024-09-21T07:36:20.711145+0000","last_unstale":"2024-09-21T07:36:21.676183+0000","last_undegraded":"2024-09-21T07:36:21.676183+0000","last_fullsized":"2024-09-21T07:36:21.676183+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:30:23.922313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.708916+0000","last_change":"2024-09-21T07:36:20.705516+0000","last_active":"2024-09-21T07:36:20.708916+0000","last_peered":"2024-09-21T07:36:20.708916+0000","last_clean":"2024-09-21T07:36:20.708916+0000","last_became_active":"2024-09-21T07:36:20.705120+0000","last_became_peered":"2024-09-21T07:36:20.705120+0000","last_unstale":"2024-09-21T07:36:20.708916+0000","last_undegraded":"2024-09-21T07:36:20.708916+0000","last_fullsized":"2024-09-21T07:36:20.708916+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:28:02.168720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.675547+0000","last_change":"2024-09-21T07:36:20.701523+0000","last_active":"2024-09-21T07:36:21.675547+0000","last_peered":"2024-09-21T07:36:21.675547+0000","last_clean":"2024-09-21T07:36:21.675547+0000","last_became_active":"2024-09-21T07:36:20.701219+0000","last_became_peered":"2024-09-21T07:36:20.701219+0000","last_unstale":"2024-09-21T07:36:21.675547+0000","last_undegraded":"2024-09-21T07:36:21.675547+0000","last_fullsized":"2024-09-21T07:36:21.675547+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:11:46.879742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.710224+0000","last_change":"2024-09-21T07:36:20.701537+0000","last_active":"2024-09-21T07:36:20.710224+0000","last_peered":"2024-09-21T07:36:20.710224+0000","last_clean":"2024-09-21T07:36:20.710224+0000","last_became_active":"2024-09-21T07:36:20.699649+0000","last_became_peered":"2024-09-21T07:36:20.699649+0000","last_unstale":"2024-09-21T07:36:20.710224+0000","last_undegraded":"2024-09-21T07:36:20.710224+0000","last_fullsized":"2024-09-21T07:36:20.710224+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:55:47.577879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.703742+0000","last_change":"2024-09-21T07:36:20.693828+0000","last_active":"2024-09-21T07:36:20.703742+0000","last_peered":"2024-09-21T07:36:20.703742+0000","last_clean":"2024-09-21T07:36:20.703742+0000","last_became_active":"2024-09-21T07:36:20.693523+0000","last_became_peered":"2024-09-21T07:36:20.693523+0000","last_unstale":"2024-09-21T07:36:20.703742+0000","last_undegraded":"2024-09-21T07:36:20.703742+0000","last_fullsized":"2024-09-21T07:36:20.703742+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:18.446466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.707841+0000","last_change":"2024-09-21T07:36:20.702053+0000","last_active":"2024-09-21T07:36:20.707841+0000","last_peered":"2024-09-21T07:36:20.707841+0000","last_clean":"2024-09-21T07:36:20.707841+0000","last_became_active":"2024-09-21T07:36:20.701747+0000","last_became_peered":"2024-09-21T07:36:20.701747+0000","last_unstale":"2024-09-21T07:36:20.707841+0000","last_undegraded":"2024-09-21T07:36:20.707841+0000","last_fullsized":"2024-09-21T07:36:20.707841+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:41.655534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.712733+0000","last_change":"2024-09-21T07:36:20.706977+0000","last_active":"2024-09-21T07:36:20.712733+0000","last_peered":"2024-09-21T07:36:20.712733+0000","last_clean":"2024-09-21T07:36:20.712733+0000","last_became_active":"2024-09-21T07:36:20.706608+0000","last_became_peered":"2024-09-21T07:36:20.706608+0000","last_unstale":"2024-09-21T07:36:20.712733+0000","last_undegraded":"2024-09-21T07:36:20.712733+0000","last_fullsized":"2024-09-21T07:36:20.712733+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:03.342948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677647+0000","last_change":"2024-09-21T07:36:20.715964+0000","last_active":"2024-09-21T07:36:21.677647+0000","last_peered":"2024-09-21T07:36:21.677647+0000","last_clean":"2024-09-21T07:36:21.677647+0000","last_became_active":"2024-09-21T07:36:20.714432+0000","last_became_peered":"2024-09-21T07:36:20.714432+0000","last_unstale":"2024-09-21T07:36:21.677647+0000","last_undegraded":"2024-09-21T07:36:21.677647+0000","last_fullsized":"2024-09-21T07:36:21.677647+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:28:09.218894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.705224+0000","last_change":"2024-09-21T07:36:20.697401+0000","last_active":"2024-09-21T07:36:20.705224+0000","last_peered":"2024-09-21T07:36:20.705224+0000","last_clean":"2024-09-21T07:36:20.705224+0000","last_became_active":"2024-09-21T07:36:20.697040+0000","last_became_peered":"2024-09-21T07:36:20.697040+0000","last_unstale":"2024-09-21T07:36:20.705224+0000","last_undegraded":"2024-09-21T07:36:20.705224+0000","last_fullsized":"2024-09-21T07:36:20.705224+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:12:55.818494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.713779+0000","last_change":"2024-09-21T07:36:20.709724+0000","last_active":"2024-09-21T07:36:20.713779+0000","last_peered":"2024-09-21T07:36:20.713779+0000","last_clean":"2024-09-21T07:36:20.713779+0000","last_became_active":"2024-09-21T07:36:20.709430+0000","last_became_peered":"2024-09-21T07:36:20.709430+0000","last_unstale":"2024-09-21T07:36:20.713779+0000","last_undegraded":"2024-09-21T07:36:20.713779+0000","last_fullsized":"2024-09-21T07:36:20.713779+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:57.039667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677664+0000","last_change":"2024-09-21T07:36:20.688718+0000","last_active":"2024-09-21T07:36:21.677664+0000","last_peered":"2024-09-21T07:36:21.677664+0000","last_clean":"2024-09-21T07:36:21.677664+0000","last_became_active":"2024-09-21T07:36:20.688274+0000","last_became_peered":"2024-09-21T07:36:20.688274+0000","last_unstale":"2024-09-21T07:36:21.677664+0000","last_undegraded":"2024-09-21T07:36:21.677664+0000","last_fullsized":"2024-09-21T07:36:21.677664+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:01.831892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.686963+0000","last_change":"2024-09-21T07:36:20.683006+0000","last_active":"2024-09-21T07:36:20.686963+0000","last_peered":"2024-09-21T07:36:20.686963+0000","last_clean":"2024-09-21T07:36:20.686963+0000","last_became_active":"2024-09-21T07:36:20.682762+0000","last_became_peered":"2024-09-21T07:36:20.682762+0000","last_unstale":"2024-09-21T07:36:20.686963+0000","last_undegraded":"2024-09-21T07:36:20.686963+0000","last_fullsized":"2024-09-21T07:36:20.686963+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:21.976503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.690390+0000","last_change":"2024-09-21T07:36:20.687428+0000","last_active":"2024-09-21T07:36:20.690390+0000","last_peered":"2024-09-21T07:36:20.690390+0000","last_clean":"2024-09-21T07:36:20.690390+0000","last_became_active":"2024-09-21T07:36:20.687124+0000","last_became_peered":"2024-09-21T07:36:20.687124+0000","last_unstale":"2024-09-21T07:36:20.690390+0000","last_undegraded":"2024-09-21T07:36:20.690390+0000","last_fullsized":"2024-09-21T07:36:20.690390+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:12:00.106146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.713348+0000","last_change":"2024-09-21T07:36:20.711337+0000","last_active":"2024-09-21T07:36:20.713348+0000","last_peered":"2024-09-21T07:36:20.713348+0000","last_clean":"2024-09-21T07:36:20.713348+0000","last_became_active":"2024-09-21T07:36:20.711013+0000","last_became_peered":"2024-09-21T07:36:20.711013+0000","last_unstale":"2024-09-21T07:36:20.713348+0000","last_undegraded":"2024-09-21T07:36:20.713348+0000","last_fullsized":"2024-09-21T07:36:20.713348+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:28:19.161094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.706640+0000","last_change":"2024-09-21T07:36:20.699465+0000","last_active":"2024-09-21T07:36:20.706640+0000","last_peered":"2024-09-21T07:36:20.706640+0000","last_clean":"2024-09-21T07:36:20.706640+0000","last_became_active":"2024-09-21T07:36:20.699157+0000","last_became_peered":"2024-09-21T07:36:20.699157+0000","last_unstale":"2024-09-21T07:36:20.706640+0000","last_undegraded":"2024-09-21T07:36:20.706640+0000","last_fullsized":"2024-09-21T07:36:20.706640+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:27:29.520243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.708140+0000","last_change":"2024-09-21T07:36:20.700443+0000","last_active":"2024-09-21T07:36:20.708140+0000","last_peered":"2024-09-21T07:36:20.708140+0000","last_clean":"2024-09-21T07:36:20.708140+0000","last_became_active":"2024-09-21T07:36:20.700142+0000","last_became_peered":"2024-09-21T07:36:20.700142+0000","last_unstale":"2024-09-21T07:36:20.708140+0000","last_undegraded":"2024-09-21T07:36:20.708140+0000","last_fullsized":"2024-09-21T07:36:20.708140+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:25:28.479799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676534+0000","last_change":"2024-09-21T07:36:20.709423+0000","last_active":"2024-09-21T07:36:21.676534+0000","last_peered":"2024-09-21T07:36:21.676534+0000","last_clean":"2024-09-21T07:36:21.676534+0000","last_became_active":"2024-09-21T07:36:20.709025+0000","last_became_peered":"2024-09-21T07:36:20.709025+0000","last_unstale":"2024-09-21T07:36:21.676534+0000","last_undegraded":"2024-09-21T07:36:21.676534+0000","last_fullsized":"2024-09-21T07:36:21.676534+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:37:03.337376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.708889+0000","last_change":"2024-09-21T07:36:20.701300+0000","last_active":"2024-09-21T07:36:20.708889+0000","last_peered":"2024-09-21T07:36:20.708889+0000","last_clean":"2024-09-21T07:36:20.708889+0000","last_became_active":"2024-09-21T07:36:20.700989+0000","last_became_peered":"2024-09-21T07:36:20.700989+0000","last_unstale":"2024-09-21T07:36:20.708889+0000","last_undegraded":"2024-09-21T07:36:20.708889+0000","last_fullsized":"2024-09-21T07:36:20.708889+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:23:47.936336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676538+0000","last_change":"2024-09-21T07:36:20.707709+0000","last_active":"2024-09-21T07:36:21.676538+0000","last_peered":"2024-09-21T07:36:21.676538+0000","last_clean":"2024-09-21T07:36:21.676538+0000","last_became_active":"2024-09-21T07:36:20.707386+0000","last_became_peered":"2024-09-21T07:36:20.707386+0000","last_unstale":"2024-09-21T07:36:21.676538+0000","last_undegraded":"2024-09-21T07:36:21.676538+0000","last_fullsized":"2024-09-21T07:36:21.676538+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:26:56.362436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.705670+0000","last_change":"2024-09-21T07:36:20.701846+0000","last_active":"2024-09-21T07:36:20.705670+0000","last_peered":"2024-09-21T07:36:20.705670+0000","last_clean":"2024-09-21T07:36:20.705670+0000","last_became_active":"2024-09-21T07:36:20.701540+0000","last_became_peered":"2024-09-21T07:36:20.701540+0000","last_unstale":"2024-09-21T07:36:20.705670+0000","last_undegraded":"2024-09-21T07:36:20.705670+0000","last_fullsized":"2024-09-21T07:36:20.705670+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:02:55.717889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.678134+0000","last_change":"2024-09-21T07:36:20.702794+0000","last_active":"2024-09-21T07:36:21.678134+0000","last_peered":"2024-09-21T07:36:21.678134+0000","last_clean":"2024-09-21T07:36:21.678134+0000","last_became_active":"2024-09-21T07:36:20.700258+0000","last_became_peered":"2024-09-21T07:36:20.700258+0000","last_unstale":"2024-09-21T07:36:21.678134+0000","last_undegraded":"2024-09-21T07:36:21.678134+0000","last_fullsized":"2024-09-21T07:36:21.678134+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:03:21.669594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.703894+0000","last_change":"2024-09-21T07:36:20.694722+0000","last_active":"2024-09-21T07:36:20.703894+0000","last_peered":"2024-09-21T07:36:20.703894+0000","last_clean":"2024-09-21T07:36:20.703894+0000","last_became_active":"2024-09-21T07:36:20.694460+0000","last_became_peered":"2024-09-21T07:36:20.694460+0000","last_unstale":"2024-09-21T07:36:20.703894+0000","last_undegraded":"2024-09-21T07:36:20.703894+0000","last_fullsized":"2024-09-21T07:36:20.703894+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:03:55.784529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676378+0000","last_change":"2024-09-21T07:36:20.698013+0000","last_active":"2024-09-21T07:36:21.676378+0000","last_peered":"2024-09-21T07:36:21.676378+0000","last_clean":"2024-09-21T07:36:21.676378+0000","last_became_active":"2024-09-21T07:36:20.697601+0000","last_became_peered":"2024-09-21T07:36:20.697601+0000","last_unstale":"2024-09-21T07:36:21.676378+0000","last_undegraded":"2024-09-21T07:36:21.676378+0000","last_fullsized":"2024-09-21T07:36:21.676378+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:01.024833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.712771+0000","last_change":"2024-09-21T07:36:20.699558+0000","last_active":"2024-09-21T07:36:20.712771+0000","last_peered":"2024-09-21T07:36:20.712771+0000","last_clean":"2024-09-21T07:36:20.712771+0000","last_became_active":"2024-09-21T07:36:20.699117+0000","last_became_peered":"2024-09-21T07:36:20.699117+0000","last_unstale":"2024-09-21T07:36:20.712771+0000","last_undegraded":"2024-09-21T07:36:20.712771+0000","last_fullsized":"2024-09-21T07:36:20.712771+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:27.445419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.687259+0000","last_change":"2024-09-21T07:36:20.682933+0000","last_active":"2024-09-21T07:36:20.687259+0000","last_peered":"2024-09-21T07:36:20.687259+0000","last_clean":"2024-09-21T07:36:20.687259+0000","last_became_active":"2024-09-21T07:36:20.682570+0000","last_became_peered":"2024-09-21T07:36:20.682570+0000","last_unstale":"2024-09-21T07:36:20.687259+0000","last_undegraded":"2024-09-21T07:36:20.687259+0000","last_fullsized":"2024-09-21T07:36:20.687259+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:37.117540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.700756+0000","last_change":"2024-09-21T07:36:20.695933+0000","last_active":"2024-09-21T07:36:20.700756+0000","last_peered":"2024-09-21T07:36:20.700756+0000","last_clean":"2024-09-21T07:36:20.700756+0000","last_became_active":"2024-09-21T07:36:20.695613+0000","last_became_peered":"2024-09-21T07:36:20.695613+0000","last_unstale":"2024-09-21T07:36:20.700756+0000","last_undegraded":"2024-09-21T07:36:20.700756+0000","last_fullsized":"2024-09-21T07:36:20.700756+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:48.741933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677409+0000","last_change":"2024-09-21T07:36:20.704112+0000","last_active":"2024-09-21T07:36:21.677409+0000","last_peered":"2024-09-21T07:36:21.677409+0000","last_clean":"2024-09-21T07:36:21.677409+0000","last_became_active":"2024-09-21T07:36:20.701493+0000","last_became_peered":"2024-09-21T07:36:20.701493+0000","last_unstale":"2024-09-21T07:36:21.677409+0000","last_undegraded":"2024-09-21T07:36:21.677409+0000","last_fullsized":"2024-09-21T07:36:21.677409+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:36.828883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.702052+0000","last_change":"2024-09-21T07:36:20.692642+0000","last_active":"2024-09-21T07:36:20.702052+0000","last_peered":"2024-09-21T07:36:20.702052+0000","last_clean":"2024-09-21T07:36:20.702052+0000","last_became_active":"2024-09-21T07:36:20.692293+0000","last_became_peered":"2024-09-21T07:36:20.692293+0000","last_unstale":"2024-09-21T07:36:20.702052+0000","last_undegraded":"2024-09-21T07:36:20.702052+0000","last_fullsized":"2024-09-21T07:36:20.702052+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:31:57.778179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677985+0000","last_change":"2024-09-21T07:36:20.705662+0000","last_active":"2024-09-21T07:36:21.677985+0000","last_peered":"2024-09-21T07:36:21.677985+0000","last_clean":"2024-09-21T07:36:21.677985+0000","last_became_active":"2024-09-21T07:36:20.705334+0000","last_became_peered":"2024-09-21T07:36:20.705334+0000","last_unstale":"2024-09-21T07:36:21.677985+0000","last_undegraded":"2024-09-21T07:36:21.677985+0000","last_fullsized":"2024-09-21T07:36:21.677985+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:29.009355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.687550+0000","last_change":"2024-09-21T07:36:20.681869+0000","last_active":"2024-09-21T07:36:20.687550+0000","last_peered":"2024-09-21T07:36:20.687550+0000","last_clean":"2024-09-21T07:36:20.687550+0000","last_became_active":"2024-09-21T07:36:20.681506+0000","last_became_peered":"2024-09-21T07:36:20.681506+0000","last_unstale":"2024-09-21T07:36:20.687550+0000","last_undegraded":"2024-09-21T07:36:20.687550+0000","last_fullsized":"2024-09-21T07:36:20.687550+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:01:41.699172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.706561+0000","last_change":"2024-09-21T07:36:20.699832+0000","last_active":"2024-09-21T07:36:20.706561+0000","last_peered":"2024-09-21T07:36:20.706561+0000","last_clean":"2024-09-21T07:36:20.706561+0000","last_became_active":"2024-09-21T07:36:20.699557+0000","last_became_peered":"2024-09-21T07:36:20.699557+0000","last_unstale":"2024-09-21T07:36:20.706561+0000","last_undegraded":"2024-09-21T07:36:20.706561+0000","last_fullsized":"2024-09-21T07:36:20.706561+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:06.467649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676711+0000","last_change":"2024-09-21T07:36:20.698978+0000","last_active":"2024-09-21T07:36:21.676711+0000","last_peered":"2024-09-21T07:36:21.676711+0000","last_clean":"2024-09-21T07:36:21.676711+0000","last_became_active":"2024-09-21T07:36:20.698672+0000","last_became_peered":"2024-09-21T07:36:20.698672+0000","last_unstale":"2024-09-21T07:36:21.676711+0000","last_undegraded":"2024-09-21T07:36:21.676711+0000","last_fullsized":"2024-09-21T07:36:21.676711+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:22:34.562215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676595+0000","last_change":"2024-09-21T07:36:20.703012+0000","last_active":"2024-09-21T07:36:21.676595+0000","last_peered":"2024-09-21T07:36:21.676595+0000","last_clean":"2024-09-21T07:36:21.676595+0000","last_became_active":"2024-09-21T07:36:20.702414+0000","last_became_peered":"2024-09-21T07:36:20.702414+0000","last_unstale":"2024-09-21T07:36:21.676595+0000","last_undegraded":"2024-09-21T07:36:21.676595+0000","last_fullsized":"2024-09-21T07:36:21.676595+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:36.465760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"89'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.678689+0000","last_change":"2024-09-21T07:36:19.686760+0000","last_active":"2024-09-21T07:36:21.678689+0000","last_peered":"2024-09-21T07:36:21.678689+0000","last_clean":"2024-09-21T07:36:21.678689+0000","last_became_active":"2024-09-21T07:36:19.686364+0000","last_became_peered":"2024-09-21T07:36:19.686364+0000","last_unstale":"2024-09-21T07:36:21.678689+0000","last_undegraded":"2024-09-21T07:36:21.678689+0000","last_fullsized":"2024-09-21T07:36:21.678689+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:46:20.360745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.693477+0000","last_change":"2024-09-21T07:35:27.879605+0000","last_active":"2024-09-21T07:36:20.693477+0000","last_peered":"2024-09-21T07:36:20.693477+0000","last_clean":"2024-09-21T07:36:20.693477+0000","last_became_active":"2024-09-21T07:35:27.879274+0000","last_became_peered":"2024-09-21T07:35:27.879274+0000","last_unstale":"2024-09-21T07:36:20.693477+0000","last_undegraded":"2024-09-21T07:36:20.693477+0000","last_fullsized":"2024-09-21T07:36:20.693477+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:24.241636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":61,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.677868+0000","last_change":"2024-09-21T07:35:29.491336+0000","last_active":"2024-09-21T07:36:20.677868+0000","last_peered":"2024-09-21T07:36:20.677868+0000","last_clean":"2024-09-21T07:36:20.677868+0000","last_became_active":"2024-09-21T07:35:26.866484+0000","last_became_peered":"2024-09-21T07:35:26.866484+0000","last_unstale":"2024-09-21T07:36:20.677868+0000","last_undegraded":"2024-09-21T07:36:20.677868+0000","last_fullsized":"2024-09-21T07:36:20.677868+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:10.449665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076010100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":240,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.686234+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:36:20.686234+0000","last_peered":"2024-09-21T07:36:20.686234+0000","last_clean":"2024-09-21T07:36:20.686234+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:36:20.686234+0000","last_undegraded":"2024-09-21T07:36:20.686234+0000","last_fullsized":"2024-09-21T07:36:20.686234+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":118,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677885+0000","last_change":"2024-09-21T07:34:24.539248+0000","last_active":"2024-09-21T07:36:21.677885+0000","last_peered":"2024-09-21T07:36:21.677885+0000","last_clean":"2024-09-21T07:36:21.677885+0000","last_became_active":"2024-09-21T07:34:23.273465+0000","last_became_peered":"2024-09-21T07:34:23.273465+0000","last_unstale":"2024-09-21T07:36:21.677885+0000","last_undegraded":"2024-09-21T07:36:21.677885+0000","last_fullsized":"2024-09-21T07:36:21.677885+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:55.517452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060627700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.675425+0000","last_change":"2024-09-21T07:36:20.711925+0000","last_active":"2024-09-21T07:36:21.675425+0000","last_peered":"2024-09-21T07:36:21.675425+0000","last_clean":"2024-09-21T07:36:21.675425+0000","last_became_active":"2024-09-21T07:36:20.711623+0000","last_became_peered":"2024-09-21T07:36:20.711623+0000","last_unstale":"2024-09-21T07:36:21.675425+0000","last_undegraded":"2024-09-21T07:36:21.675425+0000","last_fullsized":"2024-09-21T07:36:21.675425+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:58.003195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"89'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677910+0000","last_change":"2024-09-21T07:36:19.702398+0000","last_active":"2024-09-21T07:36:21.677910+0000","last_peered":"2024-09-21T07:36:21.677910+0000","last_clean":"2024-09-21T07:36:21.677910+0000","last_became_active":"2024-09-21T07:36:19.701799+0000","last_became_peered":"2024-09-21T07:36:19.701799+0000","last_unstale":"2024-09-21T07:36:21.677910+0000","last_undegraded":"2024-09-21T07:36:21.677910+0000","last_fullsized":"2024-09-21T07:36:21.677910+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:57:15.588682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.677154+0000","last_change":"2024-09-21T07:35:27.864492+0000","last_active":"2024-09-21T07:36:20.677154+0000","last_peered":"2024-09-21T07:36:20.677154+0000","last_clean":"2024-09-21T07:36:20.677154+0000","last_became_active":"2024-09-21T07:35:27.864176+0000","last_became_peered":"2024-09-21T07:35:27.864176+0000","last_unstale":"2024-09-21T07:36:20.677154+0000","last_undegraded":"2024-09-21T07:36:20.677154+0000","last_fullsized":"2024-09-21T07:36:20.677154+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:05.153467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":62,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.694038+0000","last_change":"2024-09-21T07:35:29.493678+0000","last_active":"2024-09-21T07:36:20.694038+0000","last_peered":"2024-09-21T07:36:20.694038+0000","last_clean":"2024-09-21T07:36:20.694038+0000","last_became_active":"2024-09-21T07:35:26.870749+0000","last_became_peered":"2024-09-21T07:35:26.870749+0000","last_unstale":"2024-09-21T07:36:20.694038+0000","last_undegraded":"2024-09-21T07:36:20.694038+0000","last_fullsized":"2024-09-21T07:36:20.694038+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:55.367877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010159570000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":110,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.677930+0000","last_change":"2024-09-21T07:34:24.597274+0000","last_active":"2024-09-21T07:36:20.677930+0000","last_peered":"2024-09-21T07:36:20.677930+0000","last_clean":"2024-09-21T07:36:20.677930+0000","last_became_active":"2024-09-21T07:34:23.279560+0000","last_became_peered":"2024-09-21T07:34:23.279560+0000","last_unstale":"2024-09-21T07:36:20.677930+0000","last_undegraded":"2024-09-21T07:36:20.677930+0000","last_fullsized":"2024-09-21T07:36:20.677930+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:15.845265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059699799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.675794+0000","last_change":"2024-09-21T07:36:20.703914+0000","last_active":"2024-09-21T07:36:21.675794+0000","last_peered":"2024-09-21T07:36:21.675794+0000","last_clean":"2024-09-21T07:36:21.675794+0000","last_became_active":"2024-09-21T07:36:20.703281+0000","last_became_peered":"2024-09-21T07:36:20.703281+0000","last_unstale":"2024-09-21T07:36:21.675794+0000","last_undegraded":"2024-09-21T07:36:21.675794+0000","last_fullsized":"2024-09-21T07:36:21.675794+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:17.742247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677560+0000","last_change":"2024-09-21T07:36:19.695042+0000","last_active":"2024-09-21T07:36:21.677560+0000","last_peered":"2024-09-21T07:36:21.677560+0000","last_clean":"2024-09-21T07:36:21.677560+0000","last_became_active":"2024-09-21T07:36:19.694728+0000","last_became_peered":"2024-09-21T07:36:19.694728+0000","last_unstale":"2024-09-21T07:36:21.677560+0000","last_undegraded":"2024-09-21T07:36:21.677560+0000","last_fullsized":"2024-09-21T07:36:21.677560+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:48.692195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677525+0000","last_change":"2024-09-21T07:35:27.874387+0000","last_active":"2024-09-21T07:36:21.677525+0000","last_peered":"2024-09-21T07:36:21.677525+0000","last_clean":"2024-09-21T07:36:21.677525+0000","last_became_active":"2024-09-21T07:35:27.873277+0000","last_became_peered":"2024-09-21T07:35:27.873277+0000","last_unstale":"2024-09-21T07:36:21.677525+0000","last_undegraded":"2024-09-21T07:36:21.677525+0000","last_fullsized":"2024-09-21T07:36:21.677525+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:04.581463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":61,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.676602+0000","last_change":"2024-09-21T07:35:29.387094+0000","last_active":"2024-09-21T07:36:20.676602+0000","last_peered":"2024-09-21T07:36:20.676602+0000","last_clean":"2024-09-21T07:36:20.676602+0000","last_became_active":"2024-09-21T07:35:26.874757+0000","last_became_peered":"2024-09-21T07:35:26.874757+0000","last_unstale":"2024-09-21T07:36:20.676602+0000","last_undegraded":"2024-09-21T07:36:20.676602+0000","last_fullsized":"2024-09-21T07:36:20.676602+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:33.542750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061065700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":115,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.686941+0000","last_change":"2024-09-21T07:34:24.259906+0000","last_active":"2024-09-21T07:36:20.686941+0000","last_peered":"2024-09-21T07:36:20.686941+0000","last_clean":"2024-09-21T07:36:20.686941+0000","last_became_active":"2024-09-21T07:34:23.635282+0000","last_became_peered":"2024-09-21T07:34:23.635282+0000","last_unstale":"2024-09-21T07:36:20.686941+0000","last_undegraded":"2024-09-21T07:36:20.686941+0000","last_fullsized":"2024-09-21T07:36:20.686941+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:00.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001527664,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.709857+0000","last_change":"2024-09-21T07:36:20.696699+0000","last_active":"2024-09-21T07:36:20.709857+0000","last_peered":"2024-09-21T07:36:20.709857+0000","last_clean":"2024-09-21T07:36:20.709857+0000","last_became_active":"2024-09-21T07:36:20.695323+0000","last_became_peered":"2024-09-21T07:36:20.695323+0000","last_unstale":"2024-09-21T07:36:20.709857+0000","last_undegraded":"2024-09-21T07:36:20.709857+0000","last_fullsized":"2024-09-21T07:36:20.709857+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:41.486783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.686344+0000","last_change":"2024-09-21T07:36:19.705034+0000","last_active":"2024-09-21T07:36:20.686344+0000","last_peered":"2024-09-21T07:36:20.686344+0000","last_clean":"2024-09-21T07:36:20.686344+0000","last_became_active":"2024-09-21T07:36:19.704783+0000","last_became_peered":"2024-09-21T07:36:19.704783+0000","last_unstale":"2024-09-21T07:36:20.686344+0000","last_undegraded":"2024-09-21T07:36:20.686344+0000","last_fullsized":"2024-09-21T07:36:20.686344+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:26.947126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677865+0000","last_change":"2024-09-21T07:35:27.873645+0000","last_active":"2024-09-21T07:36:21.677865+0000","last_peered":"2024-09-21T07:36:21.677865+0000","last_clean":"2024-09-21T07:36:21.677865+0000","last_became_active":"2024-09-21T07:35:27.873245+0000","last_became_peered":"2024-09-21T07:35:27.873245+0000","last_unstale":"2024-09-21T07:36:21.677865+0000","last_undegraded":"2024-09-21T07:36:21.677865+0000","last_fullsized":"2024-09-21T07:36:21.677865+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:55.090953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":64,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.678344+0000","last_change":"2024-09-21T07:35:28.861744+0000","last_active":"2024-09-21T07:36:21.678344+0000","last_peered":"2024-09-21T07:36:21.678344+0000","last_clean":"2024-09-21T07:36:21.678344+0000","last_became_active":"2024-09-21T07:35:26.876462+0000","last_became_peered":"2024-09-21T07:35:26.876462+0000","last_unstale":"2024-09-21T07:36:21.678344+0000","last_undegraded":"2024-09-21T07:36:21.678344+0000","last_fullsized":"2024-09-21T07:36:21.678344+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:48.632568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012020920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":116,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676277+0000","last_change":"2024-09-21T07:34:25.085454+0000","last_active":"2024-09-21T07:36:21.676277+0000","last_peered":"2024-09-21T07:36:21.676277+0000","last_clean":"2024-09-21T07:36:21.676277+0000","last_became_active":"2024-09-21T07:34:23.279205+0000","last_became_peered":"2024-09-21T07:34:23.279205+0000","last_unstale":"2024-09-21T07:36:21.676277+0000","last_undegraded":"2024-09-21T07:36:21.676277+0000","last_fullsized":"2024-09-21T07:36:21.676277+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:54.231666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095320599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.678455+0000","last_change":"2024-09-21T07:36:20.711544+0000","last_active":"2024-09-21T07:36:21.678455+0000","last_peered":"2024-09-21T07:36:21.678455+0000","last_clean":"2024-09-21T07:36:21.678455+0000","last_became_active":"2024-09-21T07:36:20.711070+0000","last_became_peered":"2024-09-21T07:36:20.711070+0000","last_unstale":"2024-09-21T07:36:21.678455+0000","last_undegraded":"2024-09-21T07:36:21.678455+0000","last_fullsized":"2024-09-21T07:36:21.678455+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:38.176117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677154+0000","last_change":"2024-09-21T07:36:19.699864+0000","last_active":"2024-09-21T07:36:21.677154+0000","last_peered":"2024-09-21T07:36:21.677154+0000","last_clean":"2024-09-21T07:36:21.677154+0000","last_became_active":"2024-09-21T07:36:19.698495+0000","last_became_peered":"2024-09-21T07:36:19.698495+0000","last_unstale":"2024-09-21T07:36:21.677154+0000","last_undegraded":"2024-09-21T07:36:21.677154+0000","last_fullsized":"2024-09-21T07:36:21.677154+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:32.697304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.678204+0000","last_change":"2024-09-21T07:35:27.861373+0000","last_active":"2024-09-21T07:36:20.678204+0000","last_peered":"2024-09-21T07:36:20.678204+0000","last_clean":"2024-09-21T07:36:20.678204+0000","last_became_active":"2024-09-21T07:35:27.861016+0000","last_became_peered":"2024-09-21T07:35:27.861016+0000","last_unstale":"2024-09-21T07:36:20.678204+0000","last_undegraded":"2024-09-21T07:36:20.678204+0000","last_fullsized":"2024-09-21T07:36:20.678204+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:56.043763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":62,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.689102+0000","last_change":"2024-09-21T07:35:29.053677+0000","last_active":"2024-09-21T07:36:20.689102+0000","last_peered":"2024-09-21T07:36:20.689102+0000","last_clean":"2024-09-21T07:36:20.689102+0000","last_became_active":"2024-09-21T07:35:26.875496+0000","last_became_peered":"2024-09-21T07:35:26.875496+0000","last_unstale":"2024-09-21T07:36:20.689102+0000","last_undegraded":"2024-09-21T07:36:20.689102+0000","last_fullsized":"2024-09-21T07:36:20.689102+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:04.850784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065859000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":113,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.684332+0000","last_change":"2024-09-21T07:34:25.028521+0000","last_active":"2024-09-21T07:36:20.684332+0000","last_peered":"2024-09-21T07:36:20.684332+0000","last_clean":"2024-09-21T07:36:20.684332+0000","last_became_active":"2024-09-21T07:34:23.635467+0000","last_became_peered":"2024-09-21T07:34:23.635467+0000","last_unstale":"2024-09-21T07:36:20.684332+0000","last_undegraded":"2024-09-21T07:36:20.684332+0000","last_fullsized":"2024-09-21T07:36:20.684332+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:33.729123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000660558,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677266+0000","last_change":"2024-09-21T07:36:20.703298+0000","last_active":"2024-09-21T07:36:21.677266+0000","last_peered":"2024-09-21T07:36:21.677266+0000","last_clean":"2024-09-21T07:36:21.677266+0000","last_became_active":"2024-09-21T07:36:20.703006+0000","last_became_peered":"2024-09-21T07:36:20.703006+0000","last_unstale":"2024-09-21T07:36:21.677266+0000","last_undegraded":"2024-09-21T07:36:21.677266+0000","last_fullsized":"2024-09-21T07:36:21.677266+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:58.311509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"89'1","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.719016+0000","last_change":"2024-09-21T07:36:19.700560+0000","last_active":"2024-09-21T07:36:20.719016+0000","last_peered":"2024-09-21T07:36:20.719016+0000","last_clean":"2024-09-21T07:36:20.719016+0000","last_became_active":"2024-09-21T07:36:19.700098+0000","last_became_peered":"2024-09-21T07:36:19.700098+0000","last_unstale":"2024-09-21T07:36:20.719016+0000","last_undegraded":"2024-09-21T07:36:20.719016+0000","last_fullsized":"2024-09-21T07:36:20.719016+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:02.038946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.675512+0000","last_change":"2024-09-21T07:35:27.871201+0000","last_active":"2024-09-21T07:36:21.675512+0000","last_peered":"2024-09-21T07:36:21.675512+0000","last_clean":"2024-09-21T07:36:21.675512+0000","last_became_active":"2024-09-21T07:35:27.870816+0000","last_became_peered":"2024-09-21T07:35:27.870816+0000","last_unstale":"2024-09-21T07:36:21.675512+0000","last_undegraded":"2024-09-21T07:36:21.675512+0000","last_fullsized":"2024-09-21T07:36:21.675512+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:04.787450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":61,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.678519+0000","last_change":"2024-09-21T07:35:29.491874+0000","last_active":"2024-09-21T07:36:20.678519+0000","last_peered":"2024-09-21T07:36:20.678519+0000","last_clean":"2024-09-21T07:36:20.678519+0000","last_became_active":"2024-09-21T07:35:26.866381+0000","last_became_peered":"2024-09-21T07:35:26.866381+0000","last_unstale":"2024-09-21T07:36:20.678519+0000","last_undegraded":"2024-09-21T07:36:20.678519+0000","last_fullsized":"2024-09-21T07:36:20.678519+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:36.404647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073235399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":115,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.680854+0000","last_change":"2024-09-21T07:34:24.260046+0000","last_active":"2024-09-21T07:36:20.680854+0000","last_peered":"2024-09-21T07:36:20.680854+0000","last_clean":"2024-09-21T07:36:20.680854+0000","last_became_active":"2024-09-21T07:34:23.273431+0000","last_became_peered":"2024-09-21T07:34:23.273431+0000","last_unstale":"2024-09-21T07:36:20.680854+0000","last_undegraded":"2024-09-21T07:36:20.680854+0000","last_fullsized":"2024-09-21T07:36:20.680854+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:18.876057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019803310000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.708812+0000","last_change":"2024-09-21T07:36:20.703432+0000","last_active":"2024-09-21T07:36:20.708812+0000","last_peered":"2024-09-21T07:36:20.708812+0000","last_clean":"2024-09-21T07:36:20.708812+0000","last_became_active":"2024-09-21T07:36:20.703103+0000","last_became_peered":"2024-09-21T07:36:20.703103+0000","last_unstale":"2024-09-21T07:36:20.708812+0000","last_undegraded":"2024-09-21T07:36:20.708812+0000","last_fullsized":"2024-09-21T07:36:20.708812+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:13.429695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"89'1","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.718935+0000","last_change":"2024-09-21T07:36:19.698179+0000","last_active":"2024-09-21T07:36:20.718935+0000","last_peered":"2024-09-21T07:36:20.718935+0000","last_clean":"2024-09-21T07:36:20.718935+0000","last_became_active":"2024-09-21T07:36:19.696268+0000","last_became_peered":"2024-09-21T07:36:19.696268+0000","last_unstale":"2024-09-21T07:36:20.718935+0000","last_undegraded":"2024-09-21T07:36:20.718935+0000","last_fullsized":"2024-09-21T07:36:20.718935+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:50:26.838770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.675845+0000","last_change":"2024-09-21T07:35:27.871736+0000","last_active":"2024-09-21T07:36:21.675845+0000","last_peered":"2024-09-21T07:36:21.675845+0000","last_clean":"2024-09-21T07:36:21.675845+0000","last_became_active":"2024-09-21T07:35:27.871232+0000","last_became_peered":"2024-09-21T07:35:27.871232+0000","last_unstale":"2024-09-21T07:36:21.675845+0000","last_undegraded":"2024-09-21T07:36:21.675845+0000","last_fullsized":"2024-09-21T07:36:21.675845+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:04.419301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":61,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.677288+0000","last_change":"2024-09-21T07:35:29.491246+0000","last_active":"2024-09-21T07:36:20.677288+0000","last_peered":"2024-09-21T07:36:20.677288+0000","last_clean":"2024-09-21T07:36:20.677288+0000","last_became_active":"2024-09-21T07:35:26.866783+0000","last_became_peered":"2024-09-21T07:35:26.866783+0000","last_unstale":"2024-09-21T07:36:20.677288+0000","last_undegraded":"2024-09-21T07:36:20.677288+0000","last_fullsized":"2024-09-21T07:36:20.677288+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:28.881914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098474099999999992,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":113,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.690170+0000","last_change":"2024-09-21T07:34:25.028640+0000","last_active":"2024-09-21T07:36:20.690170+0000","last_peered":"2024-09-21T07:36:20.690170+0000","last_clean":"2024-09-21T07:36:20.690170+0000","last_became_active":"2024-09-21T07:34:23.260950+0000","last_became_peered":"2024-09-21T07:34:23.260950+0000","last_unstale":"2024-09-21T07:36:20.690170+0000","last_undegraded":"2024-09-21T07:36:20.690170+0000","last_fullsized":"2024-09-21T07:36:20.690170+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:23:42.121398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055135099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.712094+0000","last_change":"2024-09-21T07:36:20.700977+0000","last_active":"2024-09-21T07:36:20.712094+0000","last_peered":"2024-09-21T07:36:20.712094+0000","last_clean":"2024-09-21T07:36:20.712094+0000","last_became_active":"2024-09-21T07:36:20.700633+0000","last_became_peered":"2024-09-21T07:36:20.700633+0000","last_unstale":"2024-09-21T07:36:20.712094+0000","last_undegraded":"2024-09-21T07:36:20.712094+0000","last_fullsized":"2024-09-21T07:36:20.712094+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:28:03.325645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676855+0000","last_change":"2024-09-21T07:36:19.698791+0000","last_active":"2024-09-21T07:36:21.676855+0000","last_peered":"2024-09-21T07:36:21.676855+0000","last_clean":"2024-09-21T07:36:21.676855+0000","last_became_active":"2024-09-21T07:36:19.698440+0000","last_became_peered":"2024-09-21T07:36:19.698440+0000","last_unstale":"2024-09-21T07:36:21.676855+0000","last_undegraded":"2024-09-21T07:36:21.676855+0000","last_fullsized":"2024-09-21T07:36:21.676855+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:54.276692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.687175+0000","last_change":"2024-09-21T07:35:27.872027+0000","last_active":"2024-09-21T07:36:20.687175+0000","last_peered":"2024-09-21T07:36:20.687175+0000","last_clean":"2024-09-21T07:36:20.687175+0000","last_became_active":"2024-09-21T07:35:27.870426+0000","last_became_peered":"2024-09-21T07:35:27.870426+0000","last_unstale":"2024-09-21T07:36:20.687175+0000","last_undegraded":"2024-09-21T07:36:20.687175+0000","last_fullsized":"2024-09-21T07:36:20.687175+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":61,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.677389+0000","last_change":"2024-09-21T07:35:29.491008+0000","last_active":"2024-09-21T07:36:20.677389+0000","last_peered":"2024-09-21T07:36:20.677389+0000","last_clean":"2024-09-21T07:36:20.677389+0000","last_became_active":"2024-09-21T07:35:26.866750+0000","last_became_peered":"2024-09-21T07:35:26.866750+0000","last_unstale":"2024-09-21T07:36:20.677389+0000","last_undegraded":"2024-09-21T07:36:20.677389+0000","last_fullsized":"2024-09-21T07:36:20.677389+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:25.490406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083529700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":115,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.693817+0000","last_change":"2024-09-21T07:34:24.260665+0000","last_active":"2024-09-21T07:36:20.693817+0000","last_peered":"2024-09-21T07:36:20.693817+0000","last_clean":"2024-09-21T07:36:20.693817+0000","last_became_active":"2024-09-21T07:34:23.635909+0000","last_became_peered":"2024-09-21T07:34:23.635909+0000","last_unstale":"2024-09-21T07:36:20.693817+0000","last_undegraded":"2024-09-21T07:36:20.693817+0000","last_fullsized":"2024-09-21T07:36:20.693817+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:44.536885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.678179+0000","last_change":"2024-09-21T07:36:20.719208+0000","last_active":"2024-09-21T07:36:21.678179+0000","last_peered":"2024-09-21T07:36:21.678179+0000","last_clean":"2024-09-21T07:36:21.678179+0000","last_became_active":"2024-09-21T07:36:20.716467+0000","last_became_peered":"2024-09-21T07:36:20.716467+0000","last_unstale":"2024-09-21T07:36:21.678179+0000","last_undegraded":"2024-09-21T07:36:21.678179+0000","last_fullsized":"2024-09-21T07:36:21.678179+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:01.356772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.678055+0000","last_change":"2024-09-21T07:36:19.694266+0000","last_active":"2024-09-21T07:36:20.678055+0000","last_peered":"2024-09-21T07:36:20.678055+0000","last_clean":"2024-09-21T07:36:20.678055+0000","last_became_active":"2024-09-21T07:36:19.693886+0000","last_became_peered":"2024-09-21T07:36:19.693886+0000","last_unstale":"2024-09-21T07:36:20.678055+0000","last_undegraded":"2024-09-21T07:36:20.678055+0000","last_fullsized":"2024-09-21T07:36:20.678055+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:24.625061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.675272+0000","last_change":"2024-09-21T07:35:27.874772+0000","last_active":"2024-09-21T07:36:20.675272+0000","last_peered":"2024-09-21T07:36:20.675272+0000","last_clean":"2024-09-21T07:36:20.675272+0000","last_became_active":"2024-09-21T07:35:27.874434+0000","last_became_peered":"2024-09-21T07:35:27.874434+0000","last_unstale":"2024-09-21T07:36:20.675272+0000","last_undegraded":"2024-09-21T07:36:20.675272+0000","last_fullsized":"2024-09-21T07:36:20.675272+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:31.998755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":66,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677871+0000","last_change":"2024-09-21T07:35:28.859153+0000","last_active":"2024-09-21T07:36:21.677871+0000","last_peered":"2024-09-21T07:36:21.677871+0000","last_clean":"2024-09-21T07:36:21.677871+0000","last_became_active":"2024-09-21T07:35:26.874428+0000","last_became_peered":"2024-09-21T07:35:26.874428+0000","last_unstale":"2024-09-21T07:36:21.677871+0000","last_undegraded":"2024-09-21T07:36:21.677871+0000","last_fullsized":"2024-09-21T07:36:21.677871+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:04.487621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048557299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":118,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676620+0000","last_change":"2024-09-21T07:34:24.538845+0000","last_active":"2024-09-21T07:36:21.676620+0000","last_peered":"2024-09-21T07:36:21.676620+0000","last_clean":"2024-09-21T07:36:21.676620+0000","last_became_active":"2024-09-21T07:34:23.636372+0000","last_became_peered":"2024-09-21T07:34:23.636372+0000","last_unstale":"2024-09-21T07:36:21.676620+0000","last_undegraded":"2024-09-21T07:36:21.676620+0000","last_fullsized":"2024-09-21T07:36:21.676620+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:29.393813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841451,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.678270+0000","last_change":"2024-09-21T07:36:20.712419+0000","last_active":"2024-09-21T07:36:21.678270+0000","last_peered":"2024-09-21T07:36:21.678270+0000","last_clean":"2024-09-21T07:36:21.678270+0000","last_became_active":"2024-09-21T07:36:20.712122+0000","last_became_peered":"2024-09-21T07:36:20.712122+0000","last_unstale":"2024-09-21T07:36:21.678270+0000","last_undegraded":"2024-09-21T07:36:21.678270+0000","last_fullsized":"2024-09-21T07:36:21.678270+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:24:34.481862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677759+0000","last_change":"2024-09-21T07:36:19.698371+0000","last_active":"2024-09-21T07:36:21.677759+0000","last_peered":"2024-09-21T07:36:21.677759+0000","last_clean":"2024-09-21T07:36:21.677759+0000","last_became_active":"2024-09-21T07:36:19.698035+0000","last_became_peered":"2024-09-21T07:36:19.698035+0000","last_unstale":"2024-09-21T07:36:21.677759+0000","last_undegraded":"2024-09-21T07:36:21.677759+0000","last_fullsized":"2024-09-21T07:36:21.677759+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:37:21.904390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.686842+0000","last_change":"2024-09-21T07:35:27.877801+0000","last_active":"2024-09-21T07:36:20.686842+0000","last_peered":"2024-09-21T07:36:20.686842+0000","last_clean":"2024-09-21T07:36:20.686842+0000","last_became_active":"2024-09-21T07:35:27.877464+0000","last_became_peered":"2024-09-21T07:35:27.877464+0000","last_unstale":"2024-09-21T07:36:20.686842+0000","last_undegraded":"2024-09-21T07:36:20.686842+0000","last_fullsized":"2024-09-21T07:36:20.686842+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:08.814929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":65,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.675951+0000","last_change":"2024-09-21T07:35:28.863732+0000","last_active":"2024-09-21T07:36:21.675951+0000","last_peered":"2024-09-21T07:36:21.675951+0000","last_clean":"2024-09-21T07:36:21.675951+0000","last_became_active":"2024-09-21T07:35:26.877392+0000","last_became_peered":"2024-09-21T07:35:26.877392+0000","last_unstale":"2024-09-21T07:36:21.675951+0000","last_undegraded":"2024-09-21T07:36:21.675951+0000","last_fullsized":"2024-09-21T07:36:21.675951+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071425700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":115,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.678617+0000","last_change":"2024-09-21T07:34:24.259686+0000","last_active":"2024-09-21T07:36:20.678617+0000","last_peered":"2024-09-21T07:36:20.678617+0000","last_clean":"2024-09-21T07:36:20.678617+0000","last_became_active":"2024-09-21T07:34:23.274405+0000","last_became_peered":"2024-09-21T07:34:23.274405+0000","last_unstale":"2024-09-21T07:36:20.678617+0000","last_undegraded":"2024-09-21T07:36:20.678617+0000","last_fullsized":"2024-09-21T07:36:20.678617+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:26.304693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022009920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.707102+0000","last_change":"2024-09-21T07:36:20.704336+0000","last_active":"2024-09-21T07:36:20.707102+0000","last_peered":"2024-09-21T07:36:20.707102+0000","last_clean":"2024-09-21T07:36:20.707102+0000","last_became_active":"2024-09-21T07:36:20.704032+0000","last_became_peered":"2024-09-21T07:36:20.704032+0000","last_unstale":"2024-09-21T07:36:20.707102+0000","last_undegraded":"2024-09-21T07:36:20.707102+0000","last_fullsized":"2024-09-21T07:36:20.707102+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:33.475211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.686415+0000","last_change":"2024-09-21T07:36:19.703472+0000","last_active":"2024-09-21T07:36:20.686415+0000","last_peered":"2024-09-21T07:36:20.686415+0000","last_clean":"2024-09-21T07:36:20.686415+0000","last_became_active":"2024-09-21T07:36:19.703094+0000","last_became_peered":"2024-09-21T07:36:19.703094+0000","last_unstale":"2024-09-21T07:36:20.686415+0000","last_undegraded":"2024-09-21T07:36:20.686415+0000","last_fullsized":"2024-09-21T07:36:20.686415+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:59.859873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.677705+0000","last_change":"2024-09-21T07:35:27.867336+0000","last_active":"2024-09-21T07:36:20.677705+0000","last_peered":"2024-09-21T07:36:20.677705+0000","last_clean":"2024-09-21T07:36:20.677705+0000","last_became_active":"2024-09-21T07:35:27.867015+0000","last_became_peered":"2024-09-21T07:35:27.867015+0000","last_unstale":"2024-09-21T07:36:20.677705+0000","last_undegraded":"2024-09-21T07:36:20.677705+0000","last_fullsized":"2024-09-21T07:36:20.677705+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:59.214139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":62,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.693605+0000","last_change":"2024-09-21T07:35:29.493137+0000","last_active":"2024-09-21T07:36:20.693605+0000","last_peered":"2024-09-21T07:36:20.693605+0000","last_clean":"2024-09-21T07:36:20.693605+0000","last_became_active":"2024-09-21T07:35:26.872230+0000","last_became_peered":"2024-09-21T07:35:26.872230+0000","last_unstale":"2024-09-21T07:36:20.693605+0000","last_undegraded":"2024-09-21T07:36:20.693605+0000","last_fullsized":"2024-09-21T07:36:20.693605+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:24.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001197449,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":115,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.686120+0000","last_change":"2024-09-21T07:34:24.258754+0000","last_active":"2024-09-21T07:36:20.686120+0000","last_peered":"2024-09-21T07:36:20.686120+0000","last_clean":"2024-09-21T07:36:20.686120+0000","last_became_active":"2024-09-21T07:34:23.272657+0000","last_became_peered":"2024-09-21T07:34:23.272657+0000","last_unstale":"2024-09-21T07:36:20.686120+0000","last_undegraded":"2024-09-21T07:36:20.686120+0000","last_fullsized":"2024-09-21T07:36:20.686120+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:05.477099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067143200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677796+0000","last_change":"2024-09-21T07:36:20.698701+0000","last_active":"2024-09-21T07:36:21.677796+0000","last_peered":"2024-09-21T07:36:21.677796+0000","last_clean":"2024-09-21T07:36:21.677796+0000","last_became_active":"2024-09-21T07:36:20.698307+0000","last_became_peered":"2024-09-21T07:36:20.698307+0000","last_unstale":"2024-09-21T07:36:21.677796+0000","last_undegraded":"2024-09-21T07:36:21.677796+0000","last_fullsized":"2024-09-21T07:36:21.677796+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:30.608814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677284+0000","last_change":"2024-09-21T07:36:19.697623+0000","last_active":"2024-09-21T07:36:21.677284+0000","last_peered":"2024-09-21T07:36:21.677284+0000","last_clean":"2024-09-21T07:36:21.677284+0000","last_became_active":"2024-09-21T07:36:19.697300+0000","last_became_peered":"2024-09-21T07:36:19.697300+0000","last_unstale":"2024-09-21T07:36:21.677284+0000","last_undegraded":"2024-09-21T07:36:21.677284+0000","last_fullsized":"2024-09-21T07:36:21.677284+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:00.623479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.675995+0000","last_change":"2024-09-21T07:35:27.876021+0000","last_active":"2024-09-21T07:36:21.675995+0000","last_peered":"2024-09-21T07:36:21.675995+0000","last_clean":"2024-09-21T07:36:21.675995+0000","last_became_active":"2024-09-21T07:35:27.875724+0000","last_became_peered":"2024-09-21T07:35:27.875724+0000","last_unstale":"2024-09-21T07:36:21.675995+0000","last_undegraded":"2024-09-21T07:36:21.675995+0000","last_fullsized":"2024-09-21T07:36:21.675995+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:03.022812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":62,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.689003+0000","last_change":"2024-09-21T07:35:29.492570+0000","last_active":"2024-09-21T07:36:20.689003+0000","last_peered":"2024-09-21T07:36:20.689003+0000","last_clean":"2024-09-21T07:36:20.689003+0000","last_became_active":"2024-09-21T07:35:26.874316+0000","last_became_peered":"2024-09-21T07:35:26.874316+0000","last_unstale":"2024-09-21T07:36:20.689003+0000","last_undegraded":"2024-09-21T07:36:20.689003+0000","last_fullsized":"2024-09-21T07:36:20.689003+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:16.512323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011990760000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":116,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.675452+0000","last_change":"2024-09-21T07:34:25.085232+0000","last_active":"2024-09-21T07:36:21.675452+0000","last_peered":"2024-09-21T07:36:21.675452+0000","last_clean":"2024-09-21T07:36:21.675452+0000","last_became_active":"2024-09-21T07:34:23.277807+0000","last_became_peered":"2024-09-21T07:34:23.277807+0000","last_unstale":"2024-09-21T07:36:21.675452+0000","last_undegraded":"2024-09-21T07:36:21.675452+0000","last_fullsized":"2024-09-21T07:36:21.675452+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:36.362919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086619099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.675151+0000","last_change":"2024-09-21T07:36:20.708922+0000","last_active":"2024-09-21T07:36:21.675151+0000","last_peered":"2024-09-21T07:36:21.675151+0000","last_clean":"2024-09-21T07:36:21.675151+0000","last_became_active":"2024-09-21T07:36:20.707981+0000","last_became_peered":"2024-09-21T07:36:20.707981+0000","last_unstale":"2024-09-21T07:36:21.675151+0000","last_undegraded":"2024-09-21T07:36:21.675151+0000","last_fullsized":"2024-09-21T07:36:21.675151+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:55.533464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"89'2","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.722353+0000","last_change":"2024-09-21T07:36:19.700137+0000","last_active":"2024-09-21T07:36:20.722353+0000","last_peered":"2024-09-21T07:36:20.722353+0000","last_clean":"2024-09-21T07:36:20.722353+0000","last_became_active":"2024-09-21T07:36:19.699771+0000","last_became_peered":"2024-09-21T07:36:19.699771+0000","last_unstale":"2024-09-21T07:36:20.722353+0000","last_undegraded":"2024-09-21T07:36:20.722353+0000","last_fullsized":"2024-09-21T07:36:20.722353+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:50:54.616191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.688222+0000","last_change":"2024-09-21T07:35:27.872798+0000","last_active":"2024-09-21T07:36:20.688222+0000","last_peered":"2024-09-21T07:36:20.688222+0000","last_clean":"2024-09-21T07:36:20.688222+0000","last_became_active":"2024-09-21T07:35:27.872483+0000","last_became_peered":"2024-09-21T07:35:27.872483+0000","last_unstale":"2024-09-21T07:36:20.688222+0000","last_undegraded":"2024-09-21T07:36:20.688222+0000","last_fullsized":"2024-09-21T07:36:20.688222+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:11.933469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.686635+0000","last_change":"2024-09-21T07:35:29.206723+0000","last_active":"2024-09-21T07:36:20.686635+0000","last_peered":"2024-09-21T07:36:20.686635+0000","last_clean":"2024-09-21T07:36:20.686635+0000","last_became_active":"2024-09-21T07:35:26.872377+0000","last_became_peered":"2024-09-21T07:35:26.872377+0000","last_unstale":"2024-09-21T07:36:20.686635+0000","last_undegraded":"2024-09-21T07:36:20.686635+0000","last_fullsized":"2024-09-21T07:36:20.686635+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:29.086862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083077200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":110,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.676872+0000","last_change":"2024-09-21T07:34:24.597672+0000","last_active":"2024-09-21T07:36:20.676872+0000","last_peered":"2024-09-21T07:36:20.676872+0000","last_clean":"2024-09-21T07:36:20.676872+0000","last_became_active":"2024-09-21T07:34:23.267770+0000","last_became_peered":"2024-09-21T07:34:23.267770+0000","last_unstale":"2024-09-21T07:36:20.676872+0000","last_undegraded":"2024-09-21T07:36:20.676872+0000","last_fullsized":"2024-09-21T07:36:20.676872+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:51.629476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086378500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.686506+0000","last_change":"2024-09-21T07:36:20.681896+0000","last_active":"2024-09-21T07:36:20.686506+0000","last_peered":"2024-09-21T07:36:20.686506+0000","last_clean":"2024-09-21T07:36:20.686506+0000","last_became_active":"2024-09-21T07:36:20.681504+0000","last_became_peered":"2024-09-21T07:36:20.681504+0000","last_unstale":"2024-09-21T07:36:20.686506+0000","last_undegraded":"2024-09-21T07:36:20.686506+0000","last_fullsized":"2024-09-21T07:36:20.686506+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:00.189091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"90'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.678850+0000","last_change":"2024-09-21T07:36:19.699176+0000","last_active":"2024-09-21T07:36:21.678850+0000","last_peered":"2024-09-21T07:36:21.678850+0000","last_clean":"2024-09-21T07:36:21.678850+0000","last_became_active":"2024-09-21T07:36:19.698605+0000","last_became_peered":"2024-09-21T07:36:19.698605+0000","last_unstale":"2024-09-21T07:36:21.678850+0000","last_undegraded":"2024-09-21T07:36:21.678850+0000","last_fullsized":"2024-09-21T07:36:21.678850+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:36.907692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.687371+0000","last_change":"2024-09-21T07:35:27.870121+0000","last_active":"2024-09-21T07:36:20.687371+0000","last_peered":"2024-09-21T07:36:20.687371+0000","last_clean":"2024-09-21T07:36:20.687371+0000","last_became_active":"2024-09-21T07:35:27.869811+0000","last_became_peered":"2024-09-21T07:35:27.869811+0000","last_unstale":"2024-09-21T07:36:20.687371+0000","last_undegraded":"2024-09-21T07:36:20.687371+0000","last_fullsized":"2024-09-21T07:36:20.687371+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:31.778461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":64,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677831+0000","last_change":"2024-09-21T07:35:28.860699+0000","last_active":"2024-09-21T07:36:21.677831+0000","last_peered":"2024-09-21T07:36:21.677831+0000","last_clean":"2024-09-21T07:36:21.677831+0000","last_became_active":"2024-09-21T07:35:26.876779+0000","last_became_peered":"2024-09-21T07:35:26.876779+0000","last_unstale":"2024-09-21T07:36:21.677831+0000","last_undegraded":"2024-09-21T07:36:21.677831+0000","last_fullsized":"2024-09-21T07:36:21.677831+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:28.286403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000913188,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":116,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.678558+0000","last_change":"2024-09-21T07:34:24.260181+0000","last_active":"2024-09-21T07:36:21.678558+0000","last_peered":"2024-09-21T07:36:21.678558+0000","last_clean":"2024-09-21T07:36:21.678558+0000","last_became_active":"2024-09-21T07:34:23.275580+0000","last_became_peered":"2024-09-21T07:34:23.275580+0000","last_unstale":"2024-09-21T07:36:21.678558+0000","last_undegraded":"2024-09-21T07:36:21.678558+0000","last_fullsized":"2024-09-21T07:36:21.678558+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:49.857622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014706980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.710262+0000","last_change":"2024-09-21T07:36:20.702121+0000","last_active":"2024-09-21T07:36:20.710262+0000","last_peered":"2024-09-21T07:36:20.710262+0000","last_clean":"2024-09-21T07:36:20.710262+0000","last_became_active":"2024-09-21T07:36:20.696640+0000","last_became_peered":"2024-09-21T07:36:20.696640+0000","last_unstale":"2024-09-21T07:36:20.710262+0000","last_undegraded":"2024-09-21T07:36:20.710262+0000","last_fullsized":"2024-09-21T07:36:20.710262+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:48.234657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.687152+0000","last_change":"2024-09-21T07:36:19.700980+0000","last_active":"2024-09-21T07:36:20.687152+0000","last_peered":"2024-09-21T07:36:20.687152+0000","last_clean":"2024-09-21T07:36:20.687152+0000","last_became_active":"2024-09-21T07:36:19.700574+0000","last_became_peered":"2024-09-21T07:36:19.700574+0000","last_unstale":"2024-09-21T07:36:20.687152+0000","last_undegraded":"2024-09-21T07:36:20.687152+0000","last_fullsized":"2024-09-21T07:36:20.687152+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:53.477806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.692232+0000","last_change":"2024-09-21T07:35:27.875393+0000","last_active":"2024-09-21T07:36:20.692232+0000","last_peered":"2024-09-21T07:36:20.692232+0000","last_clean":"2024-09-21T07:36:20.692232+0000","last_became_active":"2024-09-21T07:35:27.874671+0000","last_became_peered":"2024-09-21T07:35:27.874671+0000","last_unstale":"2024-09-21T07:36:20.692232+0000","last_undegraded":"2024-09-21T07:36:20.692232+0000","last_fullsized":"2024-09-21T07:36:20.692232+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.686639+0000","last_change":"2024-09-21T07:35:29.206296+0000","last_active":"2024-09-21T07:36:20.686639+0000","last_peered":"2024-09-21T07:36:20.686639+0000","last_clean":"2024-09-21T07:36:20.686639+0000","last_became_active":"2024-09-21T07:35:26.874575+0000","last_became_peered":"2024-09-21T07:35:26.874575+0000","last_unstale":"2024-09-21T07:36:20.686639+0000","last_undegraded":"2024-09-21T07:36:20.686639+0000","last_fullsized":"2024-09-21T07:36:20.686639+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:22.771154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061011599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":115,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.681862+0000","last_change":"2024-09-21T07:34:24.259112+0000","last_active":"2024-09-21T07:36:20.681862+0000","last_peered":"2024-09-21T07:36:20.681862+0000","last_clean":"2024-09-21T07:36:20.681862+0000","last_became_active":"2024-09-21T07:34:23.635444+0000","last_became_peered":"2024-09-21T07:34:23.635444+0000","last_unstale":"2024-09-21T07:36:20.681862+0000","last_undegraded":"2024-09-21T07:36:20.681862+0000","last_fullsized":"2024-09-21T07:36:20.681862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:46.255385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001952636,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677302+0000","last_change":"2024-09-21T07:36:20.702519+0000","last_active":"2024-09-21T07:36:21.677302+0000","last_peered":"2024-09-21T07:36:21.677302+0000","last_clean":"2024-09-21T07:36:21.677302+0000","last_became_active":"2024-09-21T07:36:20.701778+0000","last_became_peered":"2024-09-21T07:36:20.701778+0000","last_unstale":"2024-09-21T07:36:21.677302+0000","last_undegraded":"2024-09-21T07:36:21.677302+0000","last_fullsized":"2024-09-21T07:36:21.677302+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:00:11.636554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.689082+0000","last_change":"2024-09-21T07:36:19.703789+0000","last_active":"2024-09-21T07:36:20.689082+0000","last_peered":"2024-09-21T07:36:20.689082+0000","last_clean":"2024-09-21T07:36:20.689082+0000","last_became_active":"2024-09-21T07:36:19.703481+0000","last_became_peered":"2024-09-21T07:36:19.703481+0000","last_unstale":"2024-09-21T07:36:20.689082+0000","last_undegraded":"2024-09-21T07:36:20.689082+0000","last_fullsized":"2024-09-21T07:36:20.689082+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:02.336588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.693566+0000","last_change":"2024-09-21T07:35:27.878356+0000","last_active":"2024-09-21T07:36:20.693566+0000","last_peered":"2024-09-21T07:36:20.693566+0000","last_clean":"2024-09-21T07:36:20.693566+0000","last_became_active":"2024-09-21T07:35:27.877982+0000","last_became_peered":"2024-09-21T07:35:27.877982+0000","last_unstale":"2024-09-21T07:36:20.693566+0000","last_undegraded":"2024-09-21T07:36:20.693566+0000","last_fullsized":"2024-09-21T07:36:20.693566+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":62,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.684130+0000","last_change":"2024-09-21T07:35:29.053131+0000","last_active":"2024-09-21T07:36:20.684130+0000","last_peered":"2024-09-21T07:36:20.684130+0000","last_clean":"2024-09-21T07:36:20.684130+0000","last_became_active":"2024-09-21T07:35:26.870516+0000","last_became_peered":"2024-09-21T07:35:26.870516+0000","last_unstale":"2024-09-21T07:36:20.684130+0000","last_undegraded":"2024-09-21T07:36:20.684130+0000","last_fullsized":"2024-09-21T07:36:20.684130+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:10.924491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070705699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":118,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677497+0000","last_change":"2024-09-21T07:34:24.539152+0000","last_active":"2024-09-21T07:36:21.677497+0000","last_peered":"2024-09-21T07:36:21.677497+0000","last_clean":"2024-09-21T07:36:21.677497+0000","last_became_active":"2024-09-21T07:34:23.259573+0000","last_became_peered":"2024-09-21T07:34:23.259573+0000","last_unstale":"2024-09-21T07:36:21.677497+0000","last_undegraded":"2024-09-21T07:36:21.677497+0000","last_fullsized":"2024-09-21T07:36:21.677497+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:26.986900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058108400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.710149+0000","last_change":"2024-09-21T07:36:20.696988+0000","last_active":"2024-09-21T07:36:20.710149+0000","last_peered":"2024-09-21T07:36:20.710149+0000","last_clean":"2024-09-21T07:36:20.710149+0000","last_became_active":"2024-09-21T07:36:20.696662+0000","last_became_peered":"2024-09-21T07:36:20.696662+0000","last_unstale":"2024-09-21T07:36:20.710149+0000","last_undegraded":"2024-09-21T07:36:20.710149+0000","last_fullsized":"2024-09-21T07:36:20.710149+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:20.897738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"89'1","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.714231+0000","last_change":"2024-09-21T07:36:19.700460+0000","last_active":"2024-09-21T07:36:20.714231+0000","last_peered":"2024-09-21T07:36:20.714231+0000","last_clean":"2024-09-21T07:36:20.714231+0000","last_became_active":"2024-09-21T07:36:19.697272+0000","last_became_peered":"2024-09-21T07:36:19.697272+0000","last_unstale":"2024-09-21T07:36:20.714231+0000","last_undegraded":"2024-09-21T07:36:20.714231+0000","last_fullsized":"2024-09-21T07:36:20.714231+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:30.992886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.684046+0000","last_change":"2024-09-21T07:35:27.873726+0000","last_active":"2024-09-21T07:36:20.684046+0000","last_peered":"2024-09-21T07:36:20.684046+0000","last_clean":"2024-09-21T07:36:20.684046+0000","last_became_active":"2024-09-21T07:35:27.871421+0000","last_became_peered":"2024-09-21T07:35:27.871421+0000","last_unstale":"2024-09-21T07:36:20.684046+0000","last_undegraded":"2024-09-21T07:36:20.684046+0000","last_fullsized":"2024-09-21T07:36:20.684046+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:00.168915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":64,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.678383+0000","last_change":"2024-09-21T07:35:28.861032+0000","last_active":"2024-09-21T07:36:21.678383+0000","last_peered":"2024-09-21T07:36:21.678383+0000","last_clean":"2024-09-21T07:36:21.678383+0000","last_became_active":"2024-09-21T07:35:26.873149+0000","last_became_peered":"2024-09-21T07:35:26.873149+0000","last_unstale":"2024-09-21T07:36:21.678383+0000","last_undegraded":"2024-09-21T07:36:21.678383+0000","last_fullsized":"2024-09-21T07:36:21.678383+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:59.435981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011988369999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":115,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.690817+0000","last_change":"2024-09-21T07:34:24.259902+0000","last_active":"2024-09-21T07:36:20.690817+0000","last_peered":"2024-09-21T07:36:20.690817+0000","last_clean":"2024-09-21T07:36:20.690817+0000","last_became_active":"2024-09-21T07:34:23.271902+0000","last_became_peered":"2024-09-21T07:34:23.271902+0000","last_unstale":"2024-09-21T07:36:20.690817+0000","last_undegraded":"2024-09-21T07:36:20.690817+0000","last_fullsized":"2024-09-21T07:36:20.690817+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:59.439525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001686358,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676365+0000","last_change":"2024-09-21T07:36:20.709720+0000","last_active":"2024-09-21T07:36:21.676365+0000","last_peered":"2024-09-21T07:36:21.676365+0000","last_clean":"2024-09-21T07:36:21.676365+0000","last_became_active":"2024-09-21T07:36:20.709378+0000","last_became_peered":"2024-09-21T07:36:20.709378+0000","last_unstale":"2024-09-21T07:36:21.676365+0000","last_undegraded":"2024-09-21T07:36:21.676365+0000","last_fullsized":"2024-09-21T07:36:21.676365+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:29.696718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"89'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676475+0000","last_change":"2024-09-21T07:36:19.695455+0000","last_active":"2024-09-21T07:36:21.676475+0000","last_peered":"2024-09-21T07:36:21.676475+0000","last_clean":"2024-09-21T07:36:21.676475+0000","last_became_active":"2024-09-21T07:36:19.695140+0000","last_became_peered":"2024-09-21T07:36:19.695140+0000","last_unstale":"2024-09-21T07:36:21.676475+0000","last_undegraded":"2024-09-21T07:36:21.676475+0000","last_fullsized":"2024-09-21T07:36:21.676475+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:20.147254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.675034+0000","last_change":"2024-09-21T07:35:27.872637+0000","last_active":"2024-09-21T07:36:21.675034+0000","last_peered":"2024-09-21T07:36:21.675034+0000","last_clean":"2024-09-21T07:36:21.675034+0000","last_became_active":"2024-09-21T07:35:27.872149+0000","last_became_peered":"2024-09-21T07:35:27.872149+0000","last_unstale":"2024-09-21T07:36:21.675034+0000","last_undegraded":"2024-09-21T07:36:21.675034+0000","last_fullsized":"2024-09-21T07:36:21.675034+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:27.697872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":62,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.693013+0000","last_change":"2024-09-21T07:35:29.493362+0000","last_active":"2024-09-21T07:36:20.693013+0000","last_peered":"2024-09-21T07:36:20.693013+0000","last_clean":"2024-09-21T07:36:20.693013+0000","last_became_active":"2024-09-21T07:35:26.873018+0000","last_became_peered":"2024-09-21T07:35:26.873018+0000","last_unstale":"2024-09-21T07:36:20.693013+0000","last_undegraded":"2024-09-21T07:36:20.693013+0000","last_fullsized":"2024-09-21T07:36:20.693013+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:02.460557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012496510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":115,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.678420+0000","last_change":"2024-09-21T07:34:24.259662+0000","last_active":"2024-09-21T07:36:20.678420+0000","last_peered":"2024-09-21T07:36:20.678420+0000","last_clean":"2024-09-21T07:36:20.678420+0000","last_became_active":"2024-09-21T07:34:23.278812+0000","last_became_peered":"2024-09-21T07:34:23.278812+0000","last_unstale":"2024-09-21T07:36:20.678420+0000","last_undegraded":"2024-09-21T07:36:20.678420+0000","last_fullsized":"2024-09-21T07:36:20.678420+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:49.829974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356463,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.705379+0000","last_change":"2024-09-21T07:36:20.694067+0000","last_active":"2024-09-21T07:36:20.705379+0000","last_peered":"2024-09-21T07:36:20.705379+0000","last_clean":"2024-09-21T07:36:20.705379+0000","last_became_active":"2024-09-21T07:36:20.693747+0000","last_became_peered":"2024-09-21T07:36:20.693747+0000","last_unstale":"2024-09-21T07:36:20.705379+0000","last_undegraded":"2024-09-21T07:36:20.705379+0000","last_fullsized":"2024-09-21T07:36:20.705379+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:24:21.152110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.677366+0000","last_change":"2024-09-21T07:36:19.697871+0000","last_active":"2024-09-21T07:36:20.677366+0000","last_peered":"2024-09-21T07:36:20.677366+0000","last_clean":"2024-09-21T07:36:20.677366+0000","last_became_active":"2024-09-21T07:36:19.697500+0000","last_became_peered":"2024-09-21T07:36:19.697500+0000","last_unstale":"2024-09-21T07:36:20.677366+0000","last_undegraded":"2024-09-21T07:36:20.677366+0000","last_fullsized":"2024-09-21T07:36:20.677366+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:51.365852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.677563+0000","last_change":"2024-09-21T07:35:27.868448+0000","last_active":"2024-09-21T07:36:20.677563+0000","last_peered":"2024-09-21T07:36:20.677563+0000","last_clean":"2024-09-21T07:36:20.677563+0000","last_became_active":"2024-09-21T07:35:27.868105+0000","last_became_peered":"2024-09-21T07:35:27.868105+0000","last_unstale":"2024-09-21T07:36:20.677563+0000","last_undegraded":"2024-09-21T07:36:20.677563+0000","last_fullsized":"2024-09-21T07:36:20.677563+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:41.468702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":62,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.693204+0000","last_change":"2024-09-21T07:35:29.492974+0000","last_active":"2024-09-21T07:36:20.693204+0000","last_peered":"2024-09-21T07:36:20.693204+0000","last_clean":"2024-09-21T07:36:20.693204+0000","last_became_active":"2024-09-21T07:35:26.872165+0000","last_became_peered":"2024-09-21T07:35:26.872165+0000","last_unstale":"2024-09-21T07:36:20.693204+0000","last_undegraded":"2024-09-21T07:36:20.693204+0000","last_fullsized":"2024-09-21T07:36:20.693204+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:59.912520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013795179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":116,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676110+0000","last_change":"2024-09-21T07:34:25.084861+0000","last_active":"2024-09-21T07:36:21.676110+0000","last_peered":"2024-09-21T07:36:21.676110+0000","last_clean":"2024-09-21T07:36:21.676110+0000","last_became_active":"2024-09-21T07:34:23.277708+0000","last_became_peered":"2024-09-21T07:34:23.277708+0000","last_unstale":"2024-09-21T07:36:21.676110+0000","last_undegraded":"2024-09-21T07:36:21.676110+0000","last_fullsized":"2024-09-21T07:36:21.676110+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:40.816488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000597566,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.714792+0000","last_change":"2024-09-21T07:36:20.700998+0000","last_active":"2024-09-21T07:36:20.714792+0000","last_peered":"2024-09-21T07:36:20.714792+0000","last_clean":"2024-09-21T07:36:20.714792+0000","last_became_active":"2024-09-21T07:36:20.700506+0000","last_became_peered":"2024-09-21T07:36:20.700506+0000","last_unstale":"2024-09-21T07:36:20.714792+0000","last_undegraded":"2024-09-21T07:36:20.714792+0000","last_fullsized":"2024-09-21T07:36:20.714792+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:04:20.035234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.677213+0000","last_change":"2024-09-21T07:36:19.704544+0000","last_active":"2024-09-21T07:36:20.677213+0000","last_peered":"2024-09-21T07:36:20.677213+0000","last_clean":"2024-09-21T07:36:20.677213+0000","last_became_active":"2024-09-21T07:36:19.703753+0000","last_became_peered":"2024-09-21T07:36:19.703753+0000","last_unstale":"2024-09-21T07:36:20.677213+0000","last_undegraded":"2024-09-21T07:36:20.677213+0000","last_fullsized":"2024-09-21T07:36:20.677213+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:22:06.078848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.678127+0000","last_change":"2024-09-21T07:35:27.874640+0000","last_active":"2024-09-21T07:36:20.678127+0000","last_peered":"2024-09-21T07:36:20.678127+0000","last_clean":"2024-09-21T07:36:20.678127+0000","last_became_active":"2024-09-21T07:35:27.874265+0000","last_became_peered":"2024-09-21T07:35:27.874265+0000","last_unstale":"2024-09-21T07:36:20.678127+0000","last_undegraded":"2024-09-21T07:36:20.678127+0000","last_fullsized":"2024-09-21T07:36:20.678127+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:45.754654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.688457+0000","last_change":"2024-09-21T07:35:29.207392+0000","last_active":"2024-09-21T07:36:20.688457+0000","last_peered":"2024-09-21T07:36:20.688457+0000","last_clean":"2024-09-21T07:36:20.688457+0000","last_became_active":"2024-09-21T07:35:26.870917+0000","last_became_peered":"2024-09-21T07:35:26.870917+0000","last_unstale":"2024-09-21T07:36:20.688457+0000","last_undegraded":"2024-09-21T07:36:20.688457+0000","last_fullsized":"2024-09-21T07:36:20.688457+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:59.159937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059198499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":115,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.677405+0000","last_change":"2024-09-21T07:34:24.258709+0000","last_active":"2024-09-21T07:36:20.677405+0000","last_peered":"2024-09-21T07:36:20.677405+0000","last_clean":"2024-09-21T07:36:20.677405+0000","last_became_active":"2024-09-21T07:34:23.279042+0000","last_became_peered":"2024-09-21T07:34:23.279042+0000","last_unstale":"2024-09-21T07:36:20.677405+0000","last_undegraded":"2024-09-21T07:36:20.677405+0000","last_fullsized":"2024-09-21T07:36:20.677405+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:01.762117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019282100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.675441+0000","last_change":"2024-09-21T07:36:20.710007+0000","last_active":"2024-09-21T07:36:21.675441+0000","last_peered":"2024-09-21T07:36:21.675441+0000","last_clean":"2024-09-21T07:36:21.675441+0000","last_became_active":"2024-09-21T07:36:20.708378+0000","last_became_peered":"2024-09-21T07:36:20.708378+0000","last_unstale":"2024-09-21T07:36:21.675441+0000","last_undegraded":"2024-09-21T07:36:21.675441+0000","last_fullsized":"2024-09-21T07:36:21.675441+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:52.256493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"89'4","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.724439+0000","last_change":"2024-09-21T07:36:19.695955+0000","last_active":"2024-09-21T07:36:20.724439+0000","last_peered":"2024-09-21T07:36:20.724439+0000","last_clean":"2024-09-21T07:36:20.724439+0000","last_became_active":"2024-09-21T07:36:19.695677+0000","last_became_peered":"2024-09-21T07:36:19.695677+0000","last_unstale":"2024-09-21T07:36:20.724439+0000","last_undegraded":"2024-09-21T07:36:20.724439+0000","last_fullsized":"2024-09-21T07:36:20.724439+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:08.660370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.675385+0000","last_change":"2024-09-21T07:35:27.870260+0000","last_active":"2024-09-21T07:36:21.675385+0000","last_peered":"2024-09-21T07:36:21.675385+0000","last_clean":"2024-09-21T07:36:21.675385+0000","last_became_active":"2024-09-21T07:35:27.869448+0000","last_became_peered":"2024-09-21T07:35:27.869448+0000","last_unstale":"2024-09-21T07:36:21.675385+0000","last_undegraded":"2024-09-21T07:36:21.675385+0000","last_fullsized":"2024-09-21T07:36:21.675385+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:16.946790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":61,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.678046+0000","last_change":"2024-09-21T07:35:29.386367+0000","last_active":"2024-09-21T07:36:20.678046+0000","last_peered":"2024-09-21T07:36:20.678046+0000","last_clean":"2024-09-21T07:36:20.678046+0000","last_became_active":"2024-09-21T07:35:26.879238+0000","last_became_peered":"2024-09-21T07:35:26.879238+0000","last_unstale":"2024-09-21T07:36:20.678046+0000","last_undegraded":"2024-09-21T07:36:20.678046+0000","last_fullsized":"2024-09-21T07:36:20.678046+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:16.528260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092223100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":116,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676120+0000","last_change":"2024-09-21T07:34:25.085635+0000","last_active":"2024-09-21T07:36:21.676120+0000","last_peered":"2024-09-21T07:36:21.676120+0000","last_clean":"2024-09-21T07:36:21.676120+0000","last_became_active":"2024-09-21T07:34:23.273566+0000","last_became_peered":"2024-09-21T07:34:23.273566+0000","last_unstale":"2024-09-21T07:36:21.676120+0000","last_undegraded":"2024-09-21T07:36:21.676120+0000","last_fullsized":"2024-09-21T07:36:21.676120+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:14.142272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079897300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677352+0000","last_change":"2024-09-21T07:36:20.716130+0000","last_active":"2024-09-21T07:36:21.677352+0000","last_peered":"2024-09-21T07:36:21.677352+0000","last_clean":"2024-09-21T07:36:21.677352+0000","last_became_active":"2024-09-21T07:36:20.714528+0000","last_became_peered":"2024-09-21T07:36:20.714528+0000","last_unstale":"2024-09-21T07:36:21.677352+0000","last_undegraded":"2024-09-21T07:36:21.677352+0000","last_fullsized":"2024-09-21T07:36:21.677352+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:54:18.393992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"89'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.678556+0000","last_change":"2024-09-21T07:36:19.687469+0000","last_active":"2024-09-21T07:36:21.678556+0000","last_peered":"2024-09-21T07:36:21.678556+0000","last_clean":"2024-09-21T07:36:21.678556+0000","last_became_active":"2024-09-21T07:36:19.686496+0000","last_became_peered":"2024-09-21T07:36:19.686496+0000","last_unstale":"2024-09-21T07:36:21.678556+0000","last_undegraded":"2024-09-21T07:36:21.678556+0000","last_fullsized":"2024-09-21T07:36:21.678556+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:41.099573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676502+0000","last_change":"2024-09-21T07:35:27.878728+0000","last_active":"2024-09-21T07:36:21.676502+0000","last_peered":"2024-09-21T07:36:21.676502+0000","last_clean":"2024-09-21T07:36:21.676502+0000","last_became_active":"2024-09-21T07:35:27.878335+0000","last_became_peered":"2024-09-21T07:35:27.878335+0000","last_unstale":"2024-09-21T07:36:21.676502+0000","last_undegraded":"2024-09-21T07:36:21.676502+0000","last_fullsized":"2024-09-21T07:36:21.676502+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:25:37.731771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":66,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676467+0000","last_change":"2024-09-21T07:35:28.858418+0000","last_active":"2024-09-21T07:36:21.676467+0000","last_peered":"2024-09-21T07:36:21.676467+0000","last_clean":"2024-09-21T07:36:21.676467+0000","last_became_active":"2024-09-21T07:35:26.876944+0000","last_became_peered":"2024-09-21T07:35:26.876944+0000","last_unstale":"2024-09-21T07:36:21.676467+0000","last_undegraded":"2024-09-21T07:36:21.676467+0000","last_fullsized":"2024-09-21T07:36:21.676467+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:04.514304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090862799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":110,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.677114+0000","last_change":"2024-09-21T07:34:24.597926+0000","last_active":"2024-09-21T07:36:20.677114+0000","last_peered":"2024-09-21T07:36:20.677114+0000","last_clean":"2024-09-21T07:36:20.677114+0000","last_became_active":"2024-09-21T07:34:23.268926+0000","last_became_peered":"2024-09-21T07:34:23.268926+0000","last_unstale":"2024-09-21T07:36:20.677114+0000","last_undegraded":"2024-09-21T07:36:20.677114+0000","last_fullsized":"2024-09-21T07:36:20.677114+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:46.190752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101851,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.691996+0000","last_change":"2024-09-21T07:36:20.689928+0000","last_active":"2024-09-21T07:36:20.691996+0000","last_peered":"2024-09-21T07:36:20.691996+0000","last_clean":"2024-09-21T07:36:20.691996+0000","last_became_active":"2024-09-21T07:36:20.689591+0000","last_became_peered":"2024-09-21T07:36:20.689591+0000","last_unstale":"2024-09-21T07:36:20.691996+0000","last_undegraded":"2024-09-21T07:36:20.691996+0000","last_fullsized":"2024-09-21T07:36:20.691996+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:18.360631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"89'2","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.716222+0000","last_change":"2024-09-21T07:36:19.696125+0000","last_active":"2024-09-21T07:36:20.716222+0000","last_peered":"2024-09-21T07:36:20.716222+0000","last_clean":"2024-09-21T07:36:20.716222+0000","last_became_active":"2024-09-21T07:36:19.695030+0000","last_became_peered":"2024-09-21T07:36:19.695030+0000","last_unstale":"2024-09-21T07:36:20.716222+0000","last_undegraded":"2024-09-21T07:36:20.716222+0000","last_fullsized":"2024-09-21T07:36:20.716222+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:37:22.026182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.676470+0000","last_change":"2024-09-21T07:35:27.877426+0000","last_active":"2024-09-21T07:36:20.676470+0000","last_peered":"2024-09-21T07:36:20.676470+0000","last_clean":"2024-09-21T07:36:20.676470+0000","last_became_active":"2024-09-21T07:35:27.877050+0000","last_became_peered":"2024-09-21T07:35:27.877050+0000","last_unstale":"2024-09-21T07:36:20.676470+0000","last_undegraded":"2024-09-21T07:36:20.676470+0000","last_fullsized":"2024-09-21T07:36:20.676470+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:36.268334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":65,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.675186+0000","last_change":"2024-09-21T07:35:28.862273+0000","last_active":"2024-09-21T07:36:21.675186+0000","last_peered":"2024-09-21T07:36:21.675186+0000","last_clean":"2024-09-21T07:36:21.675186+0000","last_became_active":"2024-09-21T07:35:26.874011+0000","last_became_peered":"2024-09-21T07:35:26.874011+0000","last_unstale":"2024-09-21T07:36:21.675186+0000","last_undegraded":"2024-09-21T07:36:21.675186+0000","last_fullsized":"2024-09-21T07:36:21.675186+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:11.986950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071644799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":116,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.678184+0000","last_change":"2024-09-21T07:34:24.259794+0000","last_active":"2024-09-21T07:36:21.678184+0000","last_peered":"2024-09-21T07:36:21.678184+0000","last_clean":"2024-09-21T07:36:21.678184+0000","last_became_active":"2024-09-21T07:34:23.265679+0000","last_became_peered":"2024-09-21T07:34:23.265679+0000","last_unstale":"2024-09-21T07:36:21.678184+0000","last_undegraded":"2024-09-21T07:36:21.678184+0000","last_fullsized":"2024-09-21T07:36:21.678184+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:02.575796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001298518,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677003+0000","last_change":"2024-09-21T07:36:20.680585+0000","last_active":"2024-09-21T07:36:21.677003+0000","last_peered":"2024-09-21T07:36:21.677003+0000","last_clean":"2024-09-21T07:36:21.677003+0000","last_became_active":"2024-09-21T07:36:20.679797+0000","last_became_peered":"2024-09-21T07:36:20.679797+0000","last_unstale":"2024-09-21T07:36:21.677003+0000","last_undegraded":"2024-09-21T07:36:21.677003+0000","last_fullsized":"2024-09-21T07:36:21.677003+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:34:48.492726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"89'2","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.717271+0000","last_change":"2024-09-21T07:36:19.704551+0000","last_active":"2024-09-21T07:36:20.717271+0000","last_peered":"2024-09-21T07:36:20.717271+0000","last_clean":"2024-09-21T07:36:20.717271+0000","last_became_active":"2024-09-21T07:36:19.704219+0000","last_became_peered":"2024-09-21T07:36:19.704219+0000","last_unstale":"2024-09-21T07:36:20.717271+0000","last_undegraded":"2024-09-21T07:36:20.717271+0000","last_fullsized":"2024-09-21T07:36:20.717271+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:28:01.162360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.686848+0000","last_change":"2024-09-21T07:35:27.877725+0000","last_active":"2024-09-21T07:36:20.686848+0000","last_peered":"2024-09-21T07:36:20.686848+0000","last_clean":"2024-09-21T07:36:20.686848+0000","last_became_active":"2024-09-21T07:35:27.877398+0000","last_became_peered":"2024-09-21T07:35:27.877398+0000","last_unstale":"2024-09-21T07:36:20.686848+0000","last_undegraded":"2024-09-21T07:36:20.686848+0000","last_fullsized":"2024-09-21T07:36:20.686848+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:05.859762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":64,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677434+0000","last_change":"2024-09-21T07:35:28.860612+0000","last_active":"2024-09-21T07:36:21.677434+0000","last_peered":"2024-09-21T07:36:21.677434+0000","last_clean":"2024-09-21T07:36:21.677434+0000","last_became_active":"2024-09-21T07:35:26.876562+0000","last_became_peered":"2024-09-21T07:35:26.876562+0000","last_unstale":"2024-09-21T07:36:21.677434+0000","last_undegraded":"2024-09-21T07:36:21.677434+0000","last_fullsized":"2024-09-21T07:36:21.677434+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:06.824331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012252470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":115,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.693709+0000","last_change":"2024-09-21T07:34:24.260330+0000","last_active":"2024-09-21T07:36:20.693709+0000","last_peered":"2024-09-21T07:36:20.693709+0000","last_clean":"2024-09-21T07:36:20.693709+0000","last_became_active":"2024-09-21T07:34:23.637210+0000","last_became_peered":"2024-09-21T07:34:23.637210+0000","last_unstale":"2024-09-21T07:36:20.693709+0000","last_undegraded":"2024-09-21T07:36:20.693709+0000","last_fullsized":"2024-09-21T07:36:20.693709+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.488451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001537093,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677282+0000","last_change":"2024-09-21T07:36:20.703443+0000","last_active":"2024-09-21T07:36:21.677282+0000","last_peered":"2024-09-21T07:36:21.677282+0000","last_clean":"2024-09-21T07:36:21.677282+0000","last_became_active":"2024-09-21T07:36:20.703018+0000","last_became_peered":"2024-09-21T07:36:20.703018+0000","last_unstale":"2024-09-21T07:36:21.677282+0000","last_undegraded":"2024-09-21T07:36:21.677282+0000","last_fullsized":"2024-09-21T07:36:21.677282+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:52:32.457726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"89'2","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.724922+0000","last_change":"2024-09-21T07:36:19.704574+0000","last_active":"2024-09-21T07:36:20.724922+0000","last_peered":"2024-09-21T07:36:20.724922+0000","last_clean":"2024-09-21T07:36:20.724922+0000","last_became_active":"2024-09-21T07:36:19.702309+0000","last_became_peered":"2024-09-21T07:36:19.702309+0000","last_unstale":"2024-09-21T07:36:20.724922+0000","last_undegraded":"2024-09-21T07:36:20.724922+0000","last_fullsized":"2024-09-21T07:36:20.724922+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:12.423335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.675518+0000","last_change":"2024-09-21T07:35:27.874757+0000","last_active":"2024-09-21T07:36:21.675518+0000","last_peered":"2024-09-21T07:36:21.675518+0000","last_clean":"2024-09-21T07:36:21.675518+0000","last_became_active":"2024-09-21T07:35:27.874360+0000","last_became_peered":"2024-09-21T07:35:27.874360+0000","last_unstale":"2024-09-21T07:36:21.675518+0000","last_undegraded":"2024-09-21T07:36:21.675518+0000","last_fullsized":"2024-09-21T07:36:21.675518+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:17.646912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":62,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.693892+0000","last_change":"2024-09-21T07:35:29.493431+0000","last_active":"2024-09-21T07:36:20.693892+0000","last_peered":"2024-09-21T07:36:20.693892+0000","last_clean":"2024-09-21T07:36:20.693892+0000","last_became_active":"2024-09-21T07:35:26.873691+0000","last_became_peered":"2024-09-21T07:35:26.873691+0000","last_unstale":"2024-09-21T07:36:20.693892+0000","last_undegraded":"2024-09-21T07:36:20.693892+0000","last_fullsized":"2024-09-21T07:36:20.693892+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:51.705051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001162464,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":110,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.677821+0000","last_change":"2024-09-21T07:34:24.597286+0000","last_active":"2024-09-21T07:36:20.677821+0000","last_peered":"2024-09-21T07:36:20.677821+0000","last_clean":"2024-09-21T07:36:20.677821+0000","last_became_active":"2024-09-21T07:34:23.267133+0000","last_became_peered":"2024-09-21T07:34:23.267133+0000","last_unstale":"2024-09-21T07:36:20.677821+0000","last_undegraded":"2024-09-21T07:36:20.677821+0000","last_fullsized":"2024-09-21T07:36:20.677821+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:18.257010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.706781+0000","last_change":"2024-09-21T07:36:20.697901+0000","last_active":"2024-09-21T07:36:20.706781+0000","last_peered":"2024-09-21T07:36:20.706781+0000","last_clean":"2024-09-21T07:36:20.706781+0000","last_became_active":"2024-09-21T07:36:20.697210+0000","last_became_peered":"2024-09-21T07:36:20.697210+0000","last_unstale":"2024-09-21T07:36:20.706781+0000","last_undegraded":"2024-09-21T07:36:20.706781+0000","last_fullsized":"2024-09-21T07:36:20.706781+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:25.961352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.677762+0000","last_change":"2024-09-21T07:36:19.692941+0000","last_active":"2024-09-21T07:36:20.677762+0000","last_peered":"2024-09-21T07:36:20.677762+0000","last_clean":"2024-09-21T07:36:20.677762+0000","last_became_active":"2024-09-21T07:36:19.692578+0000","last_became_peered":"2024-09-21T07:36:19.692578+0000","last_unstale":"2024-09-21T07:36:20.677762+0000","last_undegraded":"2024-09-21T07:36:20.677762+0000","last_fullsized":"2024-09-21T07:36:20.677762+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:23:37.659200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":59,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676377+0000","last_change":"2024-09-21T07:35:27.864458+0000","last_active":"2024-09-21T07:36:21.676377+0000","last_peered":"2024-09-21T07:36:21.676377+0000","last_clean":"2024-09-21T07:36:21.676377+0000","last_became_active":"2024-09-21T07:35:27.864136+0000","last_became_peered":"2024-09-21T07:35:27.864136+0000","last_unstale":"2024-09-21T07:36:21.676377+0000","last_undegraded":"2024-09-21T07:36:21.676377+0000","last_fullsized":"2024-09-21T07:36:21.676377+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:20.162240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":66,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677453+0000","last_change":"2024-09-21T07:35:28.858784+0000","last_active":"2024-09-21T07:36:21.677453+0000","last_peered":"2024-09-21T07:36:21.677453+0000","last_clean":"2024-09-21T07:36:21.677453+0000","last_became_active":"2024-09-21T07:35:26.876890+0000","last_became_peered":"2024-09-21T07:35:26.876890+0000","last_unstale":"2024-09-21T07:36:21.677453+0000","last_undegraded":"2024-09-21T07:36:21.677453+0000","last_fullsized":"2024-09-21T07:36:21.677453+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:40.738327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064543499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":115,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.678303+0000","last_change":"2024-09-21T07:34:24.258712+0000","last_active":"2024-09-21T07:36:20.678303+0000","last_peered":"2024-09-21T07:36:20.678303+0000","last_clean":"2024-09-21T07:36:20.678303+0000","last_became_active":"2024-09-21T07:34:23.274692+0000","last_became_peered":"2024-09-21T07:34:23.274692+0000","last_unstale":"2024-09-21T07:36:20.678303+0000","last_undegraded":"2024-09-21T07:36:20.678303+0000","last_fullsized":"2024-09-21T07:36:20.678303+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:43.279454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018051479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.678426+0000","last_change":"2024-09-21T07:36:20.702900+0000","last_active":"2024-09-21T07:36:21.678426+0000","last_peered":"2024-09-21T07:36:21.678426+0000","last_clean":"2024-09-21T07:36:21.678426+0000","last_became_active":"2024-09-21T07:36:20.698707+0000","last_became_peered":"2024-09-21T07:36:20.698707+0000","last_unstale":"2024-09-21T07:36:21.678426+0000","last_undegraded":"2024-09-21T07:36:21.678426+0000","last_fullsized":"2024-09-21T07:36:21.678426+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:08.851652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677533+0000","last_change":"2024-09-21T07:36:19.697061+0000","last_active":"2024-09-21T07:36:21.677533+0000","last_peered":"2024-09-21T07:36:21.677533+0000","last_clean":"2024-09-21T07:36:21.677533+0000","last_became_active":"2024-09-21T07:36:19.695586+0000","last_became_peered":"2024-09-21T07:36:19.695586+0000","last_unstale":"2024-09-21T07:36:21.677533+0000","last_undegraded":"2024-09-21T07:36:21.677533+0000","last_fullsized":"2024-09-21T07:36:21.677533+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:41:42.356201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.678140+0000","last_change":"2024-09-21T07:35:27.878020+0000","last_active":"2024-09-21T07:36:20.678140+0000","last_peered":"2024-09-21T07:36:20.678140+0000","last_clean":"2024-09-21T07:36:20.678140+0000","last_became_active":"2024-09-21T07:35:27.877532+0000","last_became_peered":"2024-09-21T07:35:27.877532+0000","last_unstale":"2024-09-21T07:36:20.678140+0000","last_undegraded":"2024-09-21T07:36:20.678140+0000","last_fullsized":"2024-09-21T07:36:20.678140+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:19.756073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":64,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.678324+0000","last_change":"2024-09-21T07:35:28.861049+0000","last_active":"2024-09-21T07:36:21.678324+0000","last_peered":"2024-09-21T07:36:21.678324+0000","last_clean":"2024-09-21T07:36:21.678324+0000","last_became_active":"2024-09-21T07:35:26.876819+0000","last_became_peered":"2024-09-21T07:35:26.876819+0000","last_unstale":"2024-09-21T07:36:21.678324+0000","last_undegraded":"2024-09-21T07:36:21.678324+0000","last_fullsized":"2024-09-21T07:36:21.678324+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:19.633803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055780599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":116,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.675723+0000","last_change":"2024-09-21T07:34:25.084849+0000","last_active":"2024-09-21T07:36:21.675723+0000","last_peered":"2024-09-21T07:36:21.675723+0000","last_clean":"2024-09-21T07:36:21.675723+0000","last_became_active":"2024-09-21T07:34:23.277713+0000","last_became_peered":"2024-09-21T07:34:23.277713+0000","last_unstale":"2024-09-21T07:36:21.675723+0000","last_undegraded":"2024-09-21T07:36:21.675723+0000","last_fullsized":"2024-09-21T07:36:21.675723+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:05.811810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059079499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676688+0000","last_change":"2024-09-21T07:36:20.699395+0000","last_active":"2024-09-21T07:36:21.676688+0000","last_peered":"2024-09-21T07:36:21.676688+0000","last_clean":"2024-09-21T07:36:21.676688+0000","last_became_active":"2024-09-21T07:36:20.698734+0000","last_became_peered":"2024-09-21T07:36:20.698734+0000","last_unstale":"2024-09-21T07:36:21.676688+0000","last_undegraded":"2024-09-21T07:36:21.676688+0000","last_fullsized":"2024-09-21T07:36:21.676688+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:33:19.465270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"89'8","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.718616+0000","last_change":"2024-09-21T07:36:19.698151+0000","last_active":"2024-09-21T07:36:20.718616+0000","last_peered":"2024-09-21T07:36:20.718616+0000","last_clean":"2024-09-21T07:36:20.718616+0000","last_became_active":"2024-09-21T07:36:19.697825+0000","last_became_peered":"2024-09-21T07:36:19.697825+0000","last_unstale":"2024-09-21T07:36:20.718616+0000","last_undegraded":"2024-09-21T07:36:20.718616+0000","last_fullsized":"2024-09-21T07:36:20.718616+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:39.036115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.687672+0000","last_change":"2024-09-21T07:35:27.884248+0000","last_active":"2024-09-21T07:36:20.687672+0000","last_peered":"2024-09-21T07:36:20.687672+0000","last_clean":"2024-09-21T07:36:20.687672+0000","last_became_active":"2024-09-21T07:35:27.883861+0000","last_became_peered":"2024-09-21T07:35:27.883861+0000","last_unstale":"2024-09-21T07:36:20.687672+0000","last_undegraded":"2024-09-21T07:36:20.687672+0000","last_fullsized":"2024-09-21T07:36:20.687672+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.688329+0000","last_change":"2024-09-21T07:35:29.207135+0000","last_active":"2024-09-21T07:36:20.688329+0000","last_peered":"2024-09-21T07:36:20.688329+0000","last_clean":"2024-09-21T07:36:20.688329+0000","last_became_active":"2024-09-21T07:35:26.872787+0000","last_became_peered":"2024-09-21T07:35:26.872787+0000","last_unstale":"2024-09-21T07:36:20.688329+0000","last_undegraded":"2024-09-21T07:36:20.688329+0000","last_fullsized":"2024-09-21T07:36:20.688329+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:27.543217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011157509999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":118,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676633+0000","last_change":"2024-09-21T07:34:24.538610+0000","last_active":"2024-09-21T07:36:21.676633+0000","last_peered":"2024-09-21T07:36:21.676633+0000","last_clean":"2024-09-21T07:36:21.676633+0000","last_became_active":"2024-09-21T07:34:23.259700+0000","last_became_peered":"2024-09-21T07:34:23.259700+0000","last_unstale":"2024-09-21T07:36:21.676633+0000","last_undegraded":"2024-09-21T07:36:21.676633+0000","last_fullsized":"2024-09-21T07:36:21.676633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:57.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070522399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676043+0000","last_change":"2024-09-21T07:36:20.713460+0000","last_active":"2024-09-21T07:36:21.676043+0000","last_peered":"2024-09-21T07:36:21.676043+0000","last_clean":"2024-09-21T07:36:21.676043+0000","last_became_active":"2024-09-21T07:36:20.708796+0000","last_became_peered":"2024-09-21T07:36:20.708796+0000","last_unstale":"2024-09-21T07:36:21.676043+0000","last_undegraded":"2024-09-21T07:36:21.676043+0000","last_fullsized":"2024-09-21T07:36:21.676043+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:19.626858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.677033+0000","last_change":"2024-09-21T07:36:19.697662+0000","last_active":"2024-09-21T07:36:20.677033+0000","last_peered":"2024-09-21T07:36:20.677033+0000","last_clean":"2024-09-21T07:36:20.677033+0000","last_became_active":"2024-09-21T07:36:19.697099+0000","last_became_peered":"2024-09-21T07:36:19.697099+0000","last_unstale":"2024-09-21T07:36:20.677033+0000","last_undegraded":"2024-09-21T07:36:20.677033+0000","last_fullsized":"2024-09-21T07:36:20.677033+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:51.059346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":59,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676832+0000","last_change":"2024-09-21T07:35:27.868024+0000","last_active":"2024-09-21T07:36:21.676832+0000","last_peered":"2024-09-21T07:36:21.676832+0000","last_clean":"2024-09-21T07:36:21.676832+0000","last_became_active":"2024-09-21T07:35:27.867339+0000","last_became_peered":"2024-09-21T07:35:27.867339+0000","last_unstale":"2024-09-21T07:36:21.676832+0000","last_undegraded":"2024-09-21T07:36:21.676832+0000","last_fullsized":"2024-09-21T07:36:21.676832+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":66,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677137+0000","last_change":"2024-09-21T07:35:28.858505+0000","last_active":"2024-09-21T07:36:21.677137+0000","last_peered":"2024-09-21T07:36:21.677137+0000","last_clean":"2024-09-21T07:36:21.677137+0000","last_became_active":"2024-09-21T07:35:26.877320+0000","last_became_peered":"2024-09-21T07:35:26.877320+0000","last_unstale":"2024-09-21T07:36:21.677137+0000","last_undegraded":"2024-09-21T07:36:21.677137+0000","last_fullsized":"2024-09-21T07:36:21.677137+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:53.672782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244119999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":110,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.677254+0000","last_change":"2024-09-21T07:34:24.597597+0000","last_active":"2024-09-21T07:36:20.677254+0000","last_peered":"2024-09-21T07:36:20.677254+0000","last_clean":"2024-09-21T07:36:20.677254+0000","last_became_active":"2024-09-21T07:34:23.635204+0000","last_became_peered":"2024-09-21T07:34:23.635204+0000","last_unstale":"2024-09-21T07:36:20.677254+0000","last_undegraded":"2024-09-21T07:36:20.677254+0000","last_fullsized":"2024-09-21T07:36:20.677254+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:48.644789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081139199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.713529+0000","last_change":"2024-09-21T07:36:20.707168+0000","last_active":"2024-09-21T07:36:20.713529+0000","last_peered":"2024-09-21T07:36:20.713529+0000","last_clean":"2024-09-21T07:36:20.713529+0000","last_became_active":"2024-09-21T07:36:20.704758+0000","last_became_peered":"2024-09-21T07:36:20.704758+0000","last_unstale":"2024-09-21T07:36:20.713529+0000","last_undegraded":"2024-09-21T07:36:20.713529+0000","last_fullsized":"2024-09-21T07:36:20.713529+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:06.231964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.675663+0000","last_change":"2024-09-21T07:36:19.701061+0000","last_active":"2024-09-21T07:36:21.675663+0000","last_peered":"2024-09-21T07:36:21.675663+0000","last_clean":"2024-09-21T07:36:21.675663+0000","last_became_active":"2024-09-21T07:36:19.700180+0000","last_became_peered":"2024-09-21T07:36:19.700180+0000","last_unstale":"2024-09-21T07:36:21.675663+0000","last_undegraded":"2024-09-21T07:36:21.675663+0000","last_fullsized":"2024-09-21T07:36:21.675663+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:22.207628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.687113+0000","last_change":"2024-09-21T07:35:27.877503+0000","last_active":"2024-09-21T07:36:20.687113+0000","last_peered":"2024-09-21T07:36:20.687113+0000","last_clean":"2024-09-21T07:36:20.687113+0000","last_became_active":"2024-09-21T07:35:27.875775+0000","last_became_peered":"2024-09-21T07:35:27.875775+0000","last_unstale":"2024-09-21T07:36:20.687113+0000","last_undegraded":"2024-09-21T07:36:20.687113+0000","last_fullsized":"2024-09-21T07:36:20.687113+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:49.291023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":62,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.690748+0000","last_change":"2024-09-21T07:35:29.491412+0000","last_active":"2024-09-21T07:36:20.690748+0000","last_peered":"2024-09-21T07:36:20.690748+0000","last_clean":"2024-09-21T07:36:20.690748+0000","last_became_active":"2024-09-21T07:35:26.876404+0000","last_became_peered":"2024-09-21T07:35:26.876404+0000","last_unstale":"2024-09-21T07:36:20.690748+0000","last_undegraded":"2024-09-21T07:36:20.690748+0000","last_fullsized":"2024-09-21T07:36:20.690748+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063918000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":115,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.681239+0000","last_change":"2024-09-21T07:34:24.259026+0000","last_active":"2024-09-21T07:36:20.681239+0000","last_peered":"2024-09-21T07:36:20.681239+0000","last_clean":"2024-09-21T07:36:20.681239+0000","last_became_active":"2024-09-21T07:34:23.635456+0000","last_became_peered":"2024-09-21T07:34:23.635456+0000","last_unstale":"2024-09-21T07:36:20.681239+0000","last_undegraded":"2024-09-21T07:36:20.681239+0000","last_fullsized":"2024-09-21T07:36:20.681239+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:09.592752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018654419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677794+0000","last_change":"2024-09-21T07:36:20.717757+0000","last_active":"2024-09-21T07:36:21.677794+0000","last_peered":"2024-09-21T07:36:21.677794+0000","last_clean":"2024-09-21T07:36:21.677794+0000","last_became_active":"2024-09-21T07:36:20.717460+0000","last_became_peered":"2024-09-21T07:36:20.717460+0000","last_unstale":"2024-09-21T07:36:21.677794+0000","last_undegraded":"2024-09-21T07:36:21.677794+0000","last_fullsized":"2024-09-21T07:36:21.677794+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:27.141668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"89'3","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.718832+0000","last_change":"2024-09-21T07:36:19.702916+0000","last_active":"2024-09-21T07:36:20.718832+0000","last_peered":"2024-09-21T07:36:20.718832+0000","last_clean":"2024-09-21T07:36:20.718832+0000","last_became_active":"2024-09-21T07:36:19.702453+0000","last_became_peered":"2024-09-21T07:36:19.702453+0000","last_unstale":"2024-09-21T07:36:20.718832+0000","last_undegraded":"2024-09-21T07:36:20.718832+0000","last_fullsized":"2024-09-21T07:36:20.718832+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:33.305022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.687422+0000","last_change":"2024-09-21T07:35:27.871984+0000","last_active":"2024-09-21T07:36:20.687422+0000","last_peered":"2024-09-21T07:36:20.687422+0000","last_clean":"2024-09-21T07:36:20.687422+0000","last_became_active":"2024-09-21T07:35:27.870336+0000","last_became_peered":"2024-09-21T07:35:27.870336+0000","last_unstale":"2024-09-21T07:36:20.687422+0000","last_undegraded":"2024-09-21T07:36:20.687422+0000","last_fullsized":"2024-09-21T07:36:20.687422+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:57.120479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":64,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677606+0000","last_change":"2024-09-21T07:35:28.860351+0000","last_active":"2024-09-21T07:36:21.677606+0000","last_peered":"2024-09-21T07:36:21.677606+0000","last_clean":"2024-09-21T07:36:21.677606+0000","last_became_active":"2024-09-21T07:35:26.872046+0000","last_became_peered":"2024-09-21T07:35:26.872046+0000","last_unstale":"2024-09-21T07:36:21.677606+0000","last_undegraded":"2024-09-21T07:36:21.677606+0000","last_fullsized":"2024-09-21T07:36:21.677606+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:47.074327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098384399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":115,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.689179+0000","last_change":"2024-09-21T07:34:24.260300+0000","last_active":"2024-09-21T07:36:20.689179+0000","last_peered":"2024-09-21T07:36:20.689179+0000","last_clean":"2024-09-21T07:36:20.689179+0000","last_became_active":"2024-09-21T07:34:23.270128+0000","last_became_peered":"2024-09-21T07:34:23.270128+0000","last_unstale":"2024-09-21T07:36:20.689179+0000","last_undegraded":"2024-09-21T07:36:20.689179+0000","last_fullsized":"2024-09-21T07:36:20.689179+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:25.260931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00176651,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.713679+0000","last_change":"2024-09-21T07:36:20.708209+0000","last_active":"2024-09-21T07:36:20.713679+0000","last_peered":"2024-09-21T07:36:20.713679+0000","last_clean":"2024-09-21T07:36:20.713679+0000","last_became_active":"2024-09-21T07:36:20.707900+0000","last_became_peered":"2024-09-21T07:36:20.707900+0000","last_unstale":"2024-09-21T07:36:20.713679+0000","last_undegraded":"2024-09-21T07:36:20.713679+0000","last_fullsized":"2024-09-21T07:36:20.713679+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:23.290559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"89'6","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.725086+0000","last_change":"2024-09-21T07:36:19.702326+0000","last_active":"2024-09-21T07:36:20.725086+0000","last_peered":"2024-09-21T07:36:20.725086+0000","last_clean":"2024-09-21T07:36:20.725086+0000","last_became_active":"2024-09-21T07:36:19.701937+0000","last_became_peered":"2024-09-21T07:36:19.701937+0000","last_unstale":"2024-09-21T07:36:20.725086+0000","last_undegraded":"2024-09-21T07:36:20.725086+0000","last_fullsized":"2024-09-21T07:36:20.725086+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:51:07.871953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.687047+0000","last_change":"2024-09-21T07:35:27.877861+0000","last_active":"2024-09-21T07:36:20.687047+0000","last_peered":"2024-09-21T07:36:20.687047+0000","last_clean":"2024-09-21T07:36:20.687047+0000","last_became_active":"2024-09-21T07:35:27.877434+0000","last_became_peered":"2024-09-21T07:35:27.877434+0000","last_unstale":"2024-09-21T07:36:20.687047+0000","last_undegraded":"2024-09-21T07:36:20.687047+0000","last_fullsized":"2024-09-21T07:36:20.687047+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":65,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676337+0000","last_change":"2024-09-21T07:35:28.862954+0000","last_active":"2024-09-21T07:36:21.676337+0000","last_peered":"2024-09-21T07:36:21.676337+0000","last_clean":"2024-09-21T07:36:21.676337+0000","last_became_active":"2024-09-21T07:35:26.870613+0000","last_became_peered":"2024-09-21T07:35:26.870613+0000","last_unstale":"2024-09-21T07:36:21.676337+0000","last_undegraded":"2024-09-21T07:36:21.676337+0000","last_fullsized":"2024-09-21T07:36:21.676337+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:06.901043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059842100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":113,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.692274+0000","last_change":"2024-09-21T07:34:25.028904+0000","last_active":"2024-09-21T07:36:20.692274+0000","last_peered":"2024-09-21T07:36:20.692274+0000","last_clean":"2024-09-21T07:36:20.692274+0000","last_became_active":"2024-09-21T07:34:23.260942+0000","last_became_peered":"2024-09-21T07:34:23.260942+0000","last_unstale":"2024-09-21T07:36:20.692274+0000","last_undegraded":"2024-09-21T07:36:20.692274+0000","last_fullsized":"2024-09-21T07:36:20.692274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:22.709328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060553899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509483,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32844,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338996,"statfs":{"total":96636764160,"available":96603131904,"internally_reserved":0,"allocated":1687552,"data_stored":1099526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509485,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32272,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339568,"statfs":{"total":96636764160,"available":96603717632,"internally_reserved":0,"allocated":1105920,"data_stored":511214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509483,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1097728,"data_stored":516389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509484,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32264,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339576,"statfs":{"total":96636764160,"available":96603725824,"internally_reserved":0,"allocated":1101824,"data_stored":516423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509485,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1089536,"data_stored":509032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509485,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1708032,"data_stored":1108459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542188,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32264,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339576,"statfs":{"total":96636764160,"available":96603725824,"internally_reserved":0,"allocated":1093632,"data_stored":509090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542187,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32260,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339580,"statfs":{"total":96636764160,"available":96603729920,"internally_reserved":0,"allocated":1089536,"data_stored":509580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:36:25.229 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:36:25.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.452+0000 7fccb0600640 1 -- 172.21.15.39:0/3163664273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcca80cd240 msgr2=0x7fcca80cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:25.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.452+0000 7fccb0600640 1 --2- 172.21.15.39:0/3163664273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcca80cd240 0x7fcca80cf010 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fcc94005fd0 tx=0x7fcc9402a540 comp rx=0 tx=0).stop 2024-09-21T07:36:25.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.452+0000 7fccb0600640 1 -- 172.21.15.39:0/3163664273 shutdown_connections 2024-09-21T07:36:25.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.452+0000 7fccb0600640 1 --2- 172.21.15.39:0/3163664273 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcca80cfb90 0x7fcca80b2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:25.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.452+0000 7fccb0600640 1 --2- 172.21.15.39:0/3163664273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcca80cd240 0x7fcca80cf010 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:25.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.452+0000 7fccb0600640 1 -- 172.21.15.39:0/3163664273 >> 172.21.15.39:0/3163664273 conn(0x7fcca80b1cb0 msgr2=0x7fcca80b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:25.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.452+0000 7fccb0600640 1 -- 172.21.15.39:0/3163664273 shutdown_connections 2024-09-21T07:36:25.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.452+0000 7fccb0600640 1 -- 172.21.15.39:0/3163664273 wait complete. 2024-09-21T07:36:25.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.455+0000 7fccb0600640 1 Processor -- start 2024-09-21T07:36:25.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.455+0000 7fccb0600640 1 -- start start 2024-09-21T07:36:25.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.456+0000 7fccb0600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcca80cf660 0x7fcca81f53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:25.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.456+0000 7fccb0600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcca80cfb90 0x7fcca80c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:25.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.456+0000 7fccb0600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcca81537f0 0x7fcca8153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:25.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.456+0000 7fccb0600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcca80710a0 con 0x7fcca80cf660 2024-09-21T07:36:25.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.456+0000 7fccb0600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcca8070f20 con 0x7fcca81537f0 2024-09-21T07:36:25.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.456+0000 7fccb0600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fcca80b34f0 con 0x7fcca80cfb90 2024-09-21T07:36:25.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.456+0000 7fccadc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcca80cf660 0x7fcca81f53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:25.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.456+0000 7fccadc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcca80cf660 0x7fcca81f53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52234/0 (socket says 172.21.15.39:52234) 2024-09-21T07:36:25.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.457+0000 7fccadc00640 1 -- 172.21.15.39:0/3357914618 learned_addr learned my addr 172.21.15.39:0/3357914618 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:25.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.457+0000 7fccae600640 1 --2- 172.21.15.39:0/3357914618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcca81537f0 0x7fcca8153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:25.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.457+0000 7fccad200640 1 --2- 172.21.15.39:0/3357914618 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcca80cfb90 0x7fcca80c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:25.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.457+0000 7fccadc00640 1 -- 172.21.15.39:0/3357914618 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcca80cfb90 msgr2=0x7fcca80c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:25.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.457+0000 7fccadc00640 1 --2- 172.21.15.39:0/3357914618 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcca80cfb90 0x7fcca80c9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:25.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.457+0000 7fccadc00640 1 -- 172.21.15.39:0/3357914618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcca81537f0 msgr2=0x7fcca8153c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:25.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.457+0000 7fccadc00640 1 --2- 172.21.15.39:0/3357914618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcca81537f0 0x7fcca8153c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:25.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.457+0000 7fccadc00640 1 -- 172.21.15.39:0/3357914618 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca81545e0 con 0x7fcca80cf660 2024-09-21T07:36:25.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.457+0000 7fccae600640 1 --2- 172.21.15.39:0/3357914618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcca81537f0 0x7fcca8153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:36:25.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.457+0000 7fccadc00640 1 --2- 172.21.15.39:0/3357914618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcca80cf660 0x7fcca81f53a0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c00c3a0 tx=0x7fcc9c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:25.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.458+0000 7fcca5600640 1 -- 172.21.15.39:0/3357914618 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c017070 con 0x7fcca80cf660 2024-09-21T07:36:25.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.458+0000 7fcca5600640 1 -- 172.21.15.39:0/3357914618 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc9c0053c0 con 0x7fcca80cf660 2024-09-21T07:36:25.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.458+0000 7fcca5600640 1 -- 172.21.15.39:0/3357914618 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c003850 con 0x7fcca80cf660 2024-09-21T07:36:25.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.458+0000 7fccb0600640 1 -- 172.21.15.39:0/3357914618 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcca8154140 con 0x7fcca80cf660 2024-09-21T07:36:25.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.458+0000 7fccb0600640 1 -- 172.21.15.39:0/3357914618 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcca80737c0 con 0x7fcca80cf660 2024-09-21T07:36:25.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.459+0000 7fccb0600640 1 -- 172.21.15.39:0/3357914618 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcca80b39a0 con 0x7fcca80cf660 2024-09-21T07:36:25.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.460+0000 7fcca5600640 1 -- 172.21.15.39:0/3357914618 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcc9c01a070 con 0x7fcca80cf660 2024-09-21T07:36:25.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.461+0000 7fcca5600640 1 --2- 172.21.15.39:0/3357914618 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcc88078bc0 0x7fcc8807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:25.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.462+0000 7fcca5600640 1 -- 172.21.15.39:0/3357914618 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6800+0+0 (secure 0 0 0) 0x7fcc9c09a5c0 con 0x7fcca80cf660 2024-09-21T07:36:25.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.462+0000 7fccad200640 1 --2- 172.21.15.39:0/3357914618 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcc88078bc0 0x7fcc8807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:25.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.463+0000 7fccad200640 1 --2- 172.21.15.39:0/3357914618 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcc88078bc0 0x7fcc8807b0d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fcc94033000 tx=0x7fcc94004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:25.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.466+0000 7fcca5600640 1 -- 172.21.15.39:0/3357914618 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcca8154140 con 0x7fcca80cf660 2024-09-21T07:36:25.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.660+0000 7fccb0600640 1 -- 172.21.15.39:0/3357914618 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcca80aff50 con 0x7fcc88078bc0 2024-09-21T07:36:25.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.668+0000 7fcca5600640 1 -- 172.21.15.39:0/3357914618 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+494676 (secure 0 0 0) 0x7fcca80aff50 con 0x7fcc88078bc0 2024-09-21T07:36:25.676 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:36:25.679 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:36:25.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.674+0000 7fccb0600640 1 -- 172.21.15.39:0/3357914618 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcc88078bc0 msgr2=0x7fcc8807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:25.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.674+0000 7fccb0600640 1 --2- 172.21.15.39:0/3357914618 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcc88078bc0 0x7fcc8807b0d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fcc94033000 tx=0x7fcc94004730 comp rx=0 tx=0).stop 2024-09-21T07:36:25.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.674+0000 7fccb0600640 1 -- 172.21.15.39:0/3357914618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcca80cf660 msgr2=0x7fcca81f53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:25.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.674+0000 7fccb0600640 1 --2- 172.21.15.39:0/3357914618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcca80cf660 0x7fcca81f53a0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c00c3a0 tx=0x7fcc9c00c860 comp rx=0 tx=0).stop 2024-09-21T07:36:25.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.674+0000 7fccb0600640 1 -- 172.21.15.39:0/3357914618 shutdown_connections 2024-09-21T07:36:25.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.674+0000 7fccb0600640 1 --2- 172.21.15.39:0/3357914618 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcc88078bc0 0x7fcc8807b0d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:25.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.674+0000 7fccb0600640 1 --2- 172.21.15.39:0/3357914618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcca81537f0 0x7fcca8153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:25.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.674+0000 7fccb0600640 1 --2- 172.21.15.39:0/3357914618 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcca80cfb90 0x7fcca80c9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:25.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.674+0000 7fccb0600640 1 --2- 172.21.15.39:0/3357914618 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcca80cf660 0x7fcca81f53a0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:25.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.674+0000 7fccb0600640 1 -- 172.21.15.39:0/3357914618 >> 172.21.15.39:0/3357914618 conn(0x7fcca80b1cb0 msgr2=0x7fcca815a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:25.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.674+0000 7fccb0600640 1 -- 172.21.15.39:0/3357914618 shutdown_connections 2024-09-21T07:36:25.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.674+0000 7fccb0600640 1 -- 172.21.15.39:0/3357914618 wait complete. 2024-09-21T07:36:25.698 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":187,"stamp":"2024-09-21T07:36:25.587153+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":312,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259284,"kb_used_data":9740,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715436,"statfs":{"total":773094113280,"available":772828606464,"internally_reserved":0,"allocated":9973760,"data_stored":5279713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002007"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676183+0000","last_change":"2024-09-21T07:36:20.711879+0000","last_active":"2024-09-21T07:36:21.676183+0000","last_peered":"2024-09-21T07:36:21.676183+0000","last_clean":"2024-09-21T07:36:21.676183+0000","last_became_active":"2024-09-21T07:36:20.711145+0000","last_became_peered":"2024-09-21T07:36:20.711145+0000","last_unstale":"2024-09-21T07:36:21.676183+0000","last_undegraded":"2024-09-21T07:36:21.676183+0000","last_fullsized":"2024-09-21T07:36:21.676183+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:30:23.922313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.708916+0000","last_change":"2024-09-21T07:36:20.705516+0000","last_active":"2024-09-21T07:36:20.708916+0000","last_peered":"2024-09-21T07:36:20.708916+0000","last_clean":"2024-09-21T07:36:20.708916+0000","last_became_active":"2024-09-21T07:36:20.705120+0000","last_became_peered":"2024-09-21T07:36:20.705120+0000","last_unstale":"2024-09-21T07:36:20.708916+0000","last_undegraded":"2024-09-21T07:36:20.708916+0000","last_fullsized":"2024-09-21T07:36:20.708916+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:28:02.168720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.675547+0000","last_change":"2024-09-21T07:36:20.701523+0000","last_active":"2024-09-21T07:36:21.675547+0000","last_peered":"2024-09-21T07:36:21.675547+0000","last_clean":"2024-09-21T07:36:21.675547+0000","last_became_active":"2024-09-21T07:36:20.701219+0000","last_became_peered":"2024-09-21T07:36:20.701219+0000","last_unstale":"2024-09-21T07:36:21.675547+0000","last_undegraded":"2024-09-21T07:36:21.675547+0000","last_fullsized":"2024-09-21T07:36:21.675547+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:11:46.879742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.710224+0000","last_change":"2024-09-21T07:36:20.701537+0000","last_active":"2024-09-21T07:36:20.710224+0000","last_peered":"2024-09-21T07:36:20.710224+0000","last_clean":"2024-09-21T07:36:20.710224+0000","last_became_active":"2024-09-21T07:36:20.699649+0000","last_became_peered":"2024-09-21T07:36:20.699649+0000","last_unstale":"2024-09-21T07:36:20.710224+0000","last_undegraded":"2024-09-21T07:36:20.710224+0000","last_fullsized":"2024-09-21T07:36:20.710224+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:55:47.577879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.703742+0000","last_change":"2024-09-21T07:36:20.693828+0000","last_active":"2024-09-21T07:36:20.703742+0000","last_peered":"2024-09-21T07:36:20.703742+0000","last_clean":"2024-09-21T07:36:20.703742+0000","last_became_active":"2024-09-21T07:36:20.693523+0000","last_became_peered":"2024-09-21T07:36:20.693523+0000","last_unstale":"2024-09-21T07:36:20.703742+0000","last_undegraded":"2024-09-21T07:36:20.703742+0000","last_fullsized":"2024-09-21T07:36:20.703742+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:18.446466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.707841+0000","last_change":"2024-09-21T07:36:20.702053+0000","last_active":"2024-09-21T07:36:20.707841+0000","last_peered":"2024-09-21T07:36:20.707841+0000","last_clean":"2024-09-21T07:36:20.707841+0000","last_became_active":"2024-09-21T07:36:20.701747+0000","last_became_peered":"2024-09-21T07:36:20.701747+0000","last_unstale":"2024-09-21T07:36:20.707841+0000","last_undegraded":"2024-09-21T07:36:20.707841+0000","last_fullsized":"2024-09-21T07:36:20.707841+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:41.655534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.712733+0000","last_change":"2024-09-21T07:36:20.706977+0000","last_active":"2024-09-21T07:36:20.712733+0000","last_peered":"2024-09-21T07:36:20.712733+0000","last_clean":"2024-09-21T07:36:20.712733+0000","last_became_active":"2024-09-21T07:36:20.706608+0000","last_became_peered":"2024-09-21T07:36:20.706608+0000","last_unstale":"2024-09-21T07:36:20.712733+0000","last_undegraded":"2024-09-21T07:36:20.712733+0000","last_fullsized":"2024-09-21T07:36:20.712733+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:03.342948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677647+0000","last_change":"2024-09-21T07:36:20.715964+0000","last_active":"2024-09-21T07:36:21.677647+0000","last_peered":"2024-09-21T07:36:21.677647+0000","last_clean":"2024-09-21T07:36:21.677647+0000","last_became_active":"2024-09-21T07:36:20.714432+0000","last_became_peered":"2024-09-21T07:36:20.714432+0000","last_unstale":"2024-09-21T07:36:21.677647+0000","last_undegraded":"2024-09-21T07:36:21.677647+0000","last_fullsized":"2024-09-21T07:36:21.677647+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:28:09.218894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.705224+0000","last_change":"2024-09-21T07:36:20.697401+0000","last_active":"2024-09-21T07:36:20.705224+0000","last_peered":"2024-09-21T07:36:20.705224+0000","last_clean":"2024-09-21T07:36:20.705224+0000","last_became_active":"2024-09-21T07:36:20.697040+0000","last_became_peered":"2024-09-21T07:36:20.697040+0000","last_unstale":"2024-09-21T07:36:20.705224+0000","last_undegraded":"2024-09-21T07:36:20.705224+0000","last_fullsized":"2024-09-21T07:36:20.705224+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:12:55.818494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.713779+0000","last_change":"2024-09-21T07:36:20.709724+0000","last_active":"2024-09-21T07:36:20.713779+0000","last_peered":"2024-09-21T07:36:20.713779+0000","last_clean":"2024-09-21T07:36:20.713779+0000","last_became_active":"2024-09-21T07:36:20.709430+0000","last_became_peered":"2024-09-21T07:36:20.709430+0000","last_unstale":"2024-09-21T07:36:20.713779+0000","last_undegraded":"2024-09-21T07:36:20.713779+0000","last_fullsized":"2024-09-21T07:36:20.713779+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:57.039667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677664+0000","last_change":"2024-09-21T07:36:20.688718+0000","last_active":"2024-09-21T07:36:21.677664+0000","last_peered":"2024-09-21T07:36:21.677664+0000","last_clean":"2024-09-21T07:36:21.677664+0000","last_became_active":"2024-09-21T07:36:20.688274+0000","last_became_peered":"2024-09-21T07:36:20.688274+0000","last_unstale":"2024-09-21T07:36:21.677664+0000","last_undegraded":"2024-09-21T07:36:21.677664+0000","last_fullsized":"2024-09-21T07:36:21.677664+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:01.831892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.686963+0000","last_change":"2024-09-21T07:36:20.683006+0000","last_active":"2024-09-21T07:36:20.686963+0000","last_peered":"2024-09-21T07:36:20.686963+0000","last_clean":"2024-09-21T07:36:20.686963+0000","last_became_active":"2024-09-21T07:36:20.682762+0000","last_became_peered":"2024-09-21T07:36:20.682762+0000","last_unstale":"2024-09-21T07:36:20.686963+0000","last_undegraded":"2024-09-21T07:36:20.686963+0000","last_fullsized":"2024-09-21T07:36:20.686963+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:21.976503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.690390+0000","last_change":"2024-09-21T07:36:20.687428+0000","last_active":"2024-09-21T07:36:20.690390+0000","last_peered":"2024-09-21T07:36:20.690390+0000","last_clean":"2024-09-21T07:36:20.690390+0000","last_became_active":"2024-09-21T07:36:20.687124+0000","last_became_peered":"2024-09-21T07:36:20.687124+0000","last_unstale":"2024-09-21T07:36:20.690390+0000","last_undegraded":"2024-09-21T07:36:20.690390+0000","last_fullsized":"2024-09-21T07:36:20.690390+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:12:00.106146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.713348+0000","last_change":"2024-09-21T07:36:20.711337+0000","last_active":"2024-09-21T07:36:20.713348+0000","last_peered":"2024-09-21T07:36:20.713348+0000","last_clean":"2024-09-21T07:36:20.713348+0000","last_became_active":"2024-09-21T07:36:20.711013+0000","last_became_peered":"2024-09-21T07:36:20.711013+0000","last_unstale":"2024-09-21T07:36:20.713348+0000","last_undegraded":"2024-09-21T07:36:20.713348+0000","last_fullsized":"2024-09-21T07:36:20.713348+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:28:19.161094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.706640+0000","last_change":"2024-09-21T07:36:20.699465+0000","last_active":"2024-09-21T07:36:20.706640+0000","last_peered":"2024-09-21T07:36:20.706640+0000","last_clean":"2024-09-21T07:36:20.706640+0000","last_became_active":"2024-09-21T07:36:20.699157+0000","last_became_peered":"2024-09-21T07:36:20.699157+0000","last_unstale":"2024-09-21T07:36:20.706640+0000","last_undegraded":"2024-09-21T07:36:20.706640+0000","last_fullsized":"2024-09-21T07:36:20.706640+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:27:29.520243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.708140+0000","last_change":"2024-09-21T07:36:20.700443+0000","last_active":"2024-09-21T07:36:20.708140+0000","last_peered":"2024-09-21T07:36:20.708140+0000","last_clean":"2024-09-21T07:36:20.708140+0000","last_became_active":"2024-09-21T07:36:20.700142+0000","last_became_peered":"2024-09-21T07:36:20.700142+0000","last_unstale":"2024-09-21T07:36:20.708140+0000","last_undegraded":"2024-09-21T07:36:20.708140+0000","last_fullsized":"2024-09-21T07:36:20.708140+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:25:28.479799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676534+0000","last_change":"2024-09-21T07:36:20.709423+0000","last_active":"2024-09-21T07:36:21.676534+0000","last_peered":"2024-09-21T07:36:21.676534+0000","last_clean":"2024-09-21T07:36:21.676534+0000","last_became_active":"2024-09-21T07:36:20.709025+0000","last_became_peered":"2024-09-21T07:36:20.709025+0000","last_unstale":"2024-09-21T07:36:21.676534+0000","last_undegraded":"2024-09-21T07:36:21.676534+0000","last_fullsized":"2024-09-21T07:36:21.676534+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:37:03.337376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.708889+0000","last_change":"2024-09-21T07:36:20.701300+0000","last_active":"2024-09-21T07:36:20.708889+0000","last_peered":"2024-09-21T07:36:20.708889+0000","last_clean":"2024-09-21T07:36:20.708889+0000","last_became_active":"2024-09-21T07:36:20.700989+0000","last_became_peered":"2024-09-21T07:36:20.700989+0000","last_unstale":"2024-09-21T07:36:20.708889+0000","last_undegraded":"2024-09-21T07:36:20.708889+0000","last_fullsized":"2024-09-21T07:36:20.708889+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:23:47.936336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676538+0000","last_change":"2024-09-21T07:36:20.707709+0000","last_active":"2024-09-21T07:36:21.676538+0000","last_peered":"2024-09-21T07:36:21.676538+0000","last_clean":"2024-09-21T07:36:21.676538+0000","last_became_active":"2024-09-21T07:36:20.707386+0000","last_became_peered":"2024-09-21T07:36:20.707386+0000","last_unstale":"2024-09-21T07:36:21.676538+0000","last_undegraded":"2024-09-21T07:36:21.676538+0000","last_fullsized":"2024-09-21T07:36:21.676538+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:26:56.362436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.705670+0000","last_change":"2024-09-21T07:36:20.701846+0000","last_active":"2024-09-21T07:36:20.705670+0000","last_peered":"2024-09-21T07:36:20.705670+0000","last_clean":"2024-09-21T07:36:20.705670+0000","last_became_active":"2024-09-21T07:36:20.701540+0000","last_became_peered":"2024-09-21T07:36:20.701540+0000","last_unstale":"2024-09-21T07:36:20.705670+0000","last_undegraded":"2024-09-21T07:36:20.705670+0000","last_fullsized":"2024-09-21T07:36:20.705670+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:02:55.717889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.678134+0000","last_change":"2024-09-21T07:36:20.702794+0000","last_active":"2024-09-21T07:36:21.678134+0000","last_peered":"2024-09-21T07:36:21.678134+0000","last_clean":"2024-09-21T07:36:21.678134+0000","last_became_active":"2024-09-21T07:36:20.700258+0000","last_became_peered":"2024-09-21T07:36:20.700258+0000","last_unstale":"2024-09-21T07:36:21.678134+0000","last_undegraded":"2024-09-21T07:36:21.678134+0000","last_fullsized":"2024-09-21T07:36:21.678134+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:03:21.669594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.703894+0000","last_change":"2024-09-21T07:36:20.694722+0000","last_active":"2024-09-21T07:36:20.703894+0000","last_peered":"2024-09-21T07:36:20.703894+0000","last_clean":"2024-09-21T07:36:20.703894+0000","last_became_active":"2024-09-21T07:36:20.694460+0000","last_became_peered":"2024-09-21T07:36:20.694460+0000","last_unstale":"2024-09-21T07:36:20.703894+0000","last_undegraded":"2024-09-21T07:36:20.703894+0000","last_fullsized":"2024-09-21T07:36:20.703894+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:03:55.784529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676378+0000","last_change":"2024-09-21T07:36:20.698013+0000","last_active":"2024-09-21T07:36:21.676378+0000","last_peered":"2024-09-21T07:36:21.676378+0000","last_clean":"2024-09-21T07:36:21.676378+0000","last_became_active":"2024-09-21T07:36:20.697601+0000","last_became_peered":"2024-09-21T07:36:20.697601+0000","last_unstale":"2024-09-21T07:36:21.676378+0000","last_undegraded":"2024-09-21T07:36:21.676378+0000","last_fullsized":"2024-09-21T07:36:21.676378+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:01.024833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.712771+0000","last_change":"2024-09-21T07:36:20.699558+0000","last_active":"2024-09-21T07:36:20.712771+0000","last_peered":"2024-09-21T07:36:20.712771+0000","last_clean":"2024-09-21T07:36:20.712771+0000","last_became_active":"2024-09-21T07:36:20.699117+0000","last_became_peered":"2024-09-21T07:36:20.699117+0000","last_unstale":"2024-09-21T07:36:20.712771+0000","last_undegraded":"2024-09-21T07:36:20.712771+0000","last_fullsized":"2024-09-21T07:36:20.712771+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:27.445419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.687259+0000","last_change":"2024-09-21T07:36:20.682933+0000","last_active":"2024-09-21T07:36:20.687259+0000","last_peered":"2024-09-21T07:36:20.687259+0000","last_clean":"2024-09-21T07:36:20.687259+0000","last_became_active":"2024-09-21T07:36:20.682570+0000","last_became_peered":"2024-09-21T07:36:20.682570+0000","last_unstale":"2024-09-21T07:36:20.687259+0000","last_undegraded":"2024-09-21T07:36:20.687259+0000","last_fullsized":"2024-09-21T07:36:20.687259+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:37.117540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.700756+0000","last_change":"2024-09-21T07:36:20.695933+0000","last_active":"2024-09-21T07:36:20.700756+0000","last_peered":"2024-09-21T07:36:20.700756+0000","last_clean":"2024-09-21T07:36:20.700756+0000","last_became_active":"2024-09-21T07:36:20.695613+0000","last_became_peered":"2024-09-21T07:36:20.695613+0000","last_unstale":"2024-09-21T07:36:20.700756+0000","last_undegraded":"2024-09-21T07:36:20.700756+0000","last_fullsized":"2024-09-21T07:36:20.700756+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:48.741933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677409+0000","last_change":"2024-09-21T07:36:20.704112+0000","last_active":"2024-09-21T07:36:21.677409+0000","last_peered":"2024-09-21T07:36:21.677409+0000","last_clean":"2024-09-21T07:36:21.677409+0000","last_became_active":"2024-09-21T07:36:20.701493+0000","last_became_peered":"2024-09-21T07:36:20.701493+0000","last_unstale":"2024-09-21T07:36:21.677409+0000","last_undegraded":"2024-09-21T07:36:21.677409+0000","last_fullsized":"2024-09-21T07:36:21.677409+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:36.828883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.702052+0000","last_change":"2024-09-21T07:36:20.692642+0000","last_active":"2024-09-21T07:36:20.702052+0000","last_peered":"2024-09-21T07:36:20.702052+0000","last_clean":"2024-09-21T07:36:20.702052+0000","last_became_active":"2024-09-21T07:36:20.692293+0000","last_became_peered":"2024-09-21T07:36:20.692293+0000","last_unstale":"2024-09-21T07:36:20.702052+0000","last_undegraded":"2024-09-21T07:36:20.702052+0000","last_fullsized":"2024-09-21T07:36:20.702052+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:31:57.778179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677985+0000","last_change":"2024-09-21T07:36:20.705662+0000","last_active":"2024-09-21T07:36:21.677985+0000","last_peered":"2024-09-21T07:36:21.677985+0000","last_clean":"2024-09-21T07:36:21.677985+0000","last_became_active":"2024-09-21T07:36:20.705334+0000","last_became_peered":"2024-09-21T07:36:20.705334+0000","last_unstale":"2024-09-21T07:36:21.677985+0000","last_undegraded":"2024-09-21T07:36:21.677985+0000","last_fullsized":"2024-09-21T07:36:21.677985+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:29.009355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.687550+0000","last_change":"2024-09-21T07:36:20.681869+0000","last_active":"2024-09-21T07:36:20.687550+0000","last_peered":"2024-09-21T07:36:20.687550+0000","last_clean":"2024-09-21T07:36:20.687550+0000","last_became_active":"2024-09-21T07:36:20.681506+0000","last_became_peered":"2024-09-21T07:36:20.681506+0000","last_unstale":"2024-09-21T07:36:20.687550+0000","last_undegraded":"2024-09-21T07:36:20.687550+0000","last_fullsized":"2024-09-21T07:36:20.687550+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:01:41.699172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.706561+0000","last_change":"2024-09-21T07:36:20.699832+0000","last_active":"2024-09-21T07:36:20.706561+0000","last_peered":"2024-09-21T07:36:20.706561+0000","last_clean":"2024-09-21T07:36:20.706561+0000","last_became_active":"2024-09-21T07:36:20.699557+0000","last_became_peered":"2024-09-21T07:36:20.699557+0000","last_unstale":"2024-09-21T07:36:20.706561+0000","last_undegraded":"2024-09-21T07:36:20.706561+0000","last_fullsized":"2024-09-21T07:36:20.706561+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:06.467649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676711+0000","last_change":"2024-09-21T07:36:20.698978+0000","last_active":"2024-09-21T07:36:21.676711+0000","last_peered":"2024-09-21T07:36:21.676711+0000","last_clean":"2024-09-21T07:36:21.676711+0000","last_became_active":"2024-09-21T07:36:20.698672+0000","last_became_peered":"2024-09-21T07:36:20.698672+0000","last_unstale":"2024-09-21T07:36:21.676711+0000","last_undegraded":"2024-09-21T07:36:21.676711+0000","last_fullsized":"2024-09-21T07:36:21.676711+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:22:34.562215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676595+0000","last_change":"2024-09-21T07:36:20.703012+0000","last_active":"2024-09-21T07:36:21.676595+0000","last_peered":"2024-09-21T07:36:21.676595+0000","last_clean":"2024-09-21T07:36:21.676595+0000","last_became_active":"2024-09-21T07:36:20.702414+0000","last_became_peered":"2024-09-21T07:36:20.702414+0000","last_unstale":"2024-09-21T07:36:21.676595+0000","last_undegraded":"2024-09-21T07:36:21.676595+0000","last_fullsized":"2024-09-21T07:36:21.676595+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:36.465760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"89'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.678689+0000","last_change":"2024-09-21T07:36:19.686760+0000","last_active":"2024-09-21T07:36:21.678689+0000","last_peered":"2024-09-21T07:36:21.678689+0000","last_clean":"2024-09-21T07:36:21.678689+0000","last_became_active":"2024-09-21T07:36:19.686364+0000","last_became_peered":"2024-09-21T07:36:19.686364+0000","last_unstale":"2024-09-21T07:36:21.678689+0000","last_undegraded":"2024-09-21T07:36:21.678689+0000","last_fullsized":"2024-09-21T07:36:21.678689+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:46:20.360745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.693477+0000","last_change":"2024-09-21T07:35:27.879605+0000","last_active":"2024-09-21T07:36:20.693477+0000","last_peered":"2024-09-21T07:36:20.693477+0000","last_clean":"2024-09-21T07:36:20.693477+0000","last_became_active":"2024-09-21T07:35:27.879274+0000","last_became_peered":"2024-09-21T07:35:27.879274+0000","last_unstale":"2024-09-21T07:36:20.693477+0000","last_undegraded":"2024-09-21T07:36:20.693477+0000","last_fullsized":"2024-09-21T07:36:20.693477+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:24.241636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":61,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.677868+0000","last_change":"2024-09-21T07:35:29.491336+0000","last_active":"2024-09-21T07:36:20.677868+0000","last_peered":"2024-09-21T07:36:20.677868+0000","last_clean":"2024-09-21T07:36:20.677868+0000","last_became_active":"2024-09-21T07:35:26.866484+0000","last_became_peered":"2024-09-21T07:35:26.866484+0000","last_unstale":"2024-09-21T07:36:20.677868+0000","last_undegraded":"2024-09-21T07:36:20.677868+0000","last_fullsized":"2024-09-21T07:36:20.677868+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:10.449665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076010100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":240,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.686234+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:36:20.686234+0000","last_peered":"2024-09-21T07:36:20.686234+0000","last_clean":"2024-09-21T07:36:20.686234+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:36:20.686234+0000","last_undegraded":"2024-09-21T07:36:20.686234+0000","last_fullsized":"2024-09-21T07:36:20.686234+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":118,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677885+0000","last_change":"2024-09-21T07:34:24.539248+0000","last_active":"2024-09-21T07:36:21.677885+0000","last_peered":"2024-09-21T07:36:21.677885+0000","last_clean":"2024-09-21T07:36:21.677885+0000","last_became_active":"2024-09-21T07:34:23.273465+0000","last_became_peered":"2024-09-21T07:34:23.273465+0000","last_unstale":"2024-09-21T07:36:21.677885+0000","last_undegraded":"2024-09-21T07:36:21.677885+0000","last_fullsized":"2024-09-21T07:36:21.677885+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:55.517452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060627700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.675425+0000","last_change":"2024-09-21T07:36:20.711925+0000","last_active":"2024-09-21T07:36:21.675425+0000","last_peered":"2024-09-21T07:36:21.675425+0000","last_clean":"2024-09-21T07:36:21.675425+0000","last_became_active":"2024-09-21T07:36:20.711623+0000","last_became_peered":"2024-09-21T07:36:20.711623+0000","last_unstale":"2024-09-21T07:36:21.675425+0000","last_undegraded":"2024-09-21T07:36:21.675425+0000","last_fullsized":"2024-09-21T07:36:21.675425+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:58.003195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"89'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677910+0000","last_change":"2024-09-21T07:36:19.702398+0000","last_active":"2024-09-21T07:36:21.677910+0000","last_peered":"2024-09-21T07:36:21.677910+0000","last_clean":"2024-09-21T07:36:21.677910+0000","last_became_active":"2024-09-21T07:36:19.701799+0000","last_became_peered":"2024-09-21T07:36:19.701799+0000","last_unstale":"2024-09-21T07:36:21.677910+0000","last_undegraded":"2024-09-21T07:36:21.677910+0000","last_fullsized":"2024-09-21T07:36:21.677910+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:57:15.588682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.677154+0000","last_change":"2024-09-21T07:35:27.864492+0000","last_active":"2024-09-21T07:36:20.677154+0000","last_peered":"2024-09-21T07:36:20.677154+0000","last_clean":"2024-09-21T07:36:20.677154+0000","last_became_active":"2024-09-21T07:35:27.864176+0000","last_became_peered":"2024-09-21T07:35:27.864176+0000","last_unstale":"2024-09-21T07:36:20.677154+0000","last_undegraded":"2024-09-21T07:36:20.677154+0000","last_fullsized":"2024-09-21T07:36:20.677154+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:05.153467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":62,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.694038+0000","last_change":"2024-09-21T07:35:29.493678+0000","last_active":"2024-09-21T07:36:20.694038+0000","last_peered":"2024-09-21T07:36:20.694038+0000","last_clean":"2024-09-21T07:36:20.694038+0000","last_became_active":"2024-09-21T07:35:26.870749+0000","last_became_peered":"2024-09-21T07:35:26.870749+0000","last_unstale":"2024-09-21T07:36:20.694038+0000","last_undegraded":"2024-09-21T07:36:20.694038+0000","last_fullsized":"2024-09-21T07:36:20.694038+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:55.367877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010159570000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":110,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.677930+0000","last_change":"2024-09-21T07:34:24.597274+0000","last_active":"2024-09-21T07:36:20.677930+0000","last_peered":"2024-09-21T07:36:20.677930+0000","last_clean":"2024-09-21T07:36:20.677930+0000","last_became_active":"2024-09-21T07:34:23.279560+0000","last_became_peered":"2024-09-21T07:34:23.279560+0000","last_unstale":"2024-09-21T07:36:20.677930+0000","last_undegraded":"2024-09-21T07:36:20.677930+0000","last_fullsized":"2024-09-21T07:36:20.677930+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:15.845265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059699799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.675794+0000","last_change":"2024-09-21T07:36:20.703914+0000","last_active":"2024-09-21T07:36:21.675794+0000","last_peered":"2024-09-21T07:36:21.675794+0000","last_clean":"2024-09-21T07:36:21.675794+0000","last_became_active":"2024-09-21T07:36:20.703281+0000","last_became_peered":"2024-09-21T07:36:20.703281+0000","last_unstale":"2024-09-21T07:36:21.675794+0000","last_undegraded":"2024-09-21T07:36:21.675794+0000","last_fullsized":"2024-09-21T07:36:21.675794+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:17.742247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677560+0000","last_change":"2024-09-21T07:36:19.695042+0000","last_active":"2024-09-21T07:36:21.677560+0000","last_peered":"2024-09-21T07:36:21.677560+0000","last_clean":"2024-09-21T07:36:21.677560+0000","last_became_active":"2024-09-21T07:36:19.694728+0000","last_became_peered":"2024-09-21T07:36:19.694728+0000","last_unstale":"2024-09-21T07:36:21.677560+0000","last_undegraded":"2024-09-21T07:36:21.677560+0000","last_fullsized":"2024-09-21T07:36:21.677560+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:48.692195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677525+0000","last_change":"2024-09-21T07:35:27.874387+0000","last_active":"2024-09-21T07:36:21.677525+0000","last_peered":"2024-09-21T07:36:21.677525+0000","last_clean":"2024-09-21T07:36:21.677525+0000","last_became_active":"2024-09-21T07:35:27.873277+0000","last_became_peered":"2024-09-21T07:35:27.873277+0000","last_unstale":"2024-09-21T07:36:21.677525+0000","last_undegraded":"2024-09-21T07:36:21.677525+0000","last_fullsized":"2024-09-21T07:36:21.677525+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:04.581463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":61,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.676602+0000","last_change":"2024-09-21T07:35:29.387094+0000","last_active":"2024-09-21T07:36:20.676602+0000","last_peered":"2024-09-21T07:36:20.676602+0000","last_clean":"2024-09-21T07:36:20.676602+0000","last_became_active":"2024-09-21T07:35:26.874757+0000","last_became_peered":"2024-09-21T07:35:26.874757+0000","last_unstale":"2024-09-21T07:36:20.676602+0000","last_undegraded":"2024-09-21T07:36:20.676602+0000","last_fullsized":"2024-09-21T07:36:20.676602+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:33.542750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061065700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":115,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.686941+0000","last_change":"2024-09-21T07:34:24.259906+0000","last_active":"2024-09-21T07:36:20.686941+0000","last_peered":"2024-09-21T07:36:20.686941+0000","last_clean":"2024-09-21T07:36:20.686941+0000","last_became_active":"2024-09-21T07:34:23.635282+0000","last_became_peered":"2024-09-21T07:34:23.635282+0000","last_unstale":"2024-09-21T07:36:20.686941+0000","last_undegraded":"2024-09-21T07:36:20.686941+0000","last_fullsized":"2024-09-21T07:36:20.686941+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:00.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001527664,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.709857+0000","last_change":"2024-09-21T07:36:20.696699+0000","last_active":"2024-09-21T07:36:20.709857+0000","last_peered":"2024-09-21T07:36:20.709857+0000","last_clean":"2024-09-21T07:36:20.709857+0000","last_became_active":"2024-09-21T07:36:20.695323+0000","last_became_peered":"2024-09-21T07:36:20.695323+0000","last_unstale":"2024-09-21T07:36:20.709857+0000","last_undegraded":"2024-09-21T07:36:20.709857+0000","last_fullsized":"2024-09-21T07:36:20.709857+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:41.486783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.686344+0000","last_change":"2024-09-21T07:36:19.705034+0000","last_active":"2024-09-21T07:36:20.686344+0000","last_peered":"2024-09-21T07:36:20.686344+0000","last_clean":"2024-09-21T07:36:20.686344+0000","last_became_active":"2024-09-21T07:36:19.704783+0000","last_became_peered":"2024-09-21T07:36:19.704783+0000","last_unstale":"2024-09-21T07:36:20.686344+0000","last_undegraded":"2024-09-21T07:36:20.686344+0000","last_fullsized":"2024-09-21T07:36:20.686344+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:26.947126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677865+0000","last_change":"2024-09-21T07:35:27.873645+0000","last_active":"2024-09-21T07:36:21.677865+0000","last_peered":"2024-09-21T07:36:21.677865+0000","last_clean":"2024-09-21T07:36:21.677865+0000","last_became_active":"2024-09-21T07:35:27.873245+0000","last_became_peered":"2024-09-21T07:35:27.873245+0000","last_unstale":"2024-09-21T07:36:21.677865+0000","last_undegraded":"2024-09-21T07:36:21.677865+0000","last_fullsized":"2024-09-21T07:36:21.677865+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:55.090953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":64,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.678344+0000","last_change":"2024-09-21T07:35:28.861744+0000","last_active":"2024-09-21T07:36:21.678344+0000","last_peered":"2024-09-21T07:36:21.678344+0000","last_clean":"2024-09-21T07:36:21.678344+0000","last_became_active":"2024-09-21T07:35:26.876462+0000","last_became_peered":"2024-09-21T07:35:26.876462+0000","last_unstale":"2024-09-21T07:36:21.678344+0000","last_undegraded":"2024-09-21T07:36:21.678344+0000","last_fullsized":"2024-09-21T07:36:21.678344+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:48.632568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012020920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":116,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676277+0000","last_change":"2024-09-21T07:34:25.085454+0000","last_active":"2024-09-21T07:36:21.676277+0000","last_peered":"2024-09-21T07:36:21.676277+0000","last_clean":"2024-09-21T07:36:21.676277+0000","last_became_active":"2024-09-21T07:34:23.279205+0000","last_became_peered":"2024-09-21T07:34:23.279205+0000","last_unstale":"2024-09-21T07:36:21.676277+0000","last_undegraded":"2024-09-21T07:36:21.676277+0000","last_fullsized":"2024-09-21T07:36:21.676277+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:54.231666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095320599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.678455+0000","last_change":"2024-09-21T07:36:20.711544+0000","last_active":"2024-09-21T07:36:21.678455+0000","last_peered":"2024-09-21T07:36:21.678455+0000","last_clean":"2024-09-21T07:36:21.678455+0000","last_became_active":"2024-09-21T07:36:20.711070+0000","last_became_peered":"2024-09-21T07:36:20.711070+0000","last_unstale":"2024-09-21T07:36:21.678455+0000","last_undegraded":"2024-09-21T07:36:21.678455+0000","last_fullsized":"2024-09-21T07:36:21.678455+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:38.176117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677154+0000","last_change":"2024-09-21T07:36:19.699864+0000","last_active":"2024-09-21T07:36:21.677154+0000","last_peered":"2024-09-21T07:36:21.677154+0000","last_clean":"2024-09-21T07:36:21.677154+0000","last_became_active":"2024-09-21T07:36:19.698495+0000","last_became_peered":"2024-09-21T07:36:19.698495+0000","last_unstale":"2024-09-21T07:36:21.677154+0000","last_undegraded":"2024-09-21T07:36:21.677154+0000","last_fullsized":"2024-09-21T07:36:21.677154+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:32.697304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.678204+0000","last_change":"2024-09-21T07:35:27.861373+0000","last_active":"2024-09-21T07:36:20.678204+0000","last_peered":"2024-09-21T07:36:20.678204+0000","last_clean":"2024-09-21T07:36:20.678204+0000","last_became_active":"2024-09-21T07:35:27.861016+0000","last_became_peered":"2024-09-21T07:35:27.861016+0000","last_unstale":"2024-09-21T07:36:20.678204+0000","last_undegraded":"2024-09-21T07:36:20.678204+0000","last_fullsized":"2024-09-21T07:36:20.678204+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:56.043763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":62,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.689102+0000","last_change":"2024-09-21T07:35:29.053677+0000","last_active":"2024-09-21T07:36:20.689102+0000","last_peered":"2024-09-21T07:36:20.689102+0000","last_clean":"2024-09-21T07:36:20.689102+0000","last_became_active":"2024-09-21T07:35:26.875496+0000","last_became_peered":"2024-09-21T07:35:26.875496+0000","last_unstale":"2024-09-21T07:36:20.689102+0000","last_undegraded":"2024-09-21T07:36:20.689102+0000","last_fullsized":"2024-09-21T07:36:20.689102+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:04.850784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065859000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":113,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.684332+0000","last_change":"2024-09-21T07:34:25.028521+0000","last_active":"2024-09-21T07:36:20.684332+0000","last_peered":"2024-09-21T07:36:20.684332+0000","last_clean":"2024-09-21T07:36:20.684332+0000","last_became_active":"2024-09-21T07:34:23.635467+0000","last_became_peered":"2024-09-21T07:34:23.635467+0000","last_unstale":"2024-09-21T07:36:20.684332+0000","last_undegraded":"2024-09-21T07:36:20.684332+0000","last_fullsized":"2024-09-21T07:36:20.684332+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:33.729123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000660558,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677266+0000","last_change":"2024-09-21T07:36:20.703298+0000","last_active":"2024-09-21T07:36:21.677266+0000","last_peered":"2024-09-21T07:36:21.677266+0000","last_clean":"2024-09-21T07:36:21.677266+0000","last_became_active":"2024-09-21T07:36:20.703006+0000","last_became_peered":"2024-09-21T07:36:20.703006+0000","last_unstale":"2024-09-21T07:36:21.677266+0000","last_undegraded":"2024-09-21T07:36:21.677266+0000","last_fullsized":"2024-09-21T07:36:21.677266+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:58.311509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"89'1","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.719016+0000","last_change":"2024-09-21T07:36:19.700560+0000","last_active":"2024-09-21T07:36:20.719016+0000","last_peered":"2024-09-21T07:36:20.719016+0000","last_clean":"2024-09-21T07:36:20.719016+0000","last_became_active":"2024-09-21T07:36:19.700098+0000","last_became_peered":"2024-09-21T07:36:19.700098+0000","last_unstale":"2024-09-21T07:36:20.719016+0000","last_undegraded":"2024-09-21T07:36:20.719016+0000","last_fullsized":"2024-09-21T07:36:20.719016+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:02.038946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.675512+0000","last_change":"2024-09-21T07:35:27.871201+0000","last_active":"2024-09-21T07:36:21.675512+0000","last_peered":"2024-09-21T07:36:21.675512+0000","last_clean":"2024-09-21T07:36:21.675512+0000","last_became_active":"2024-09-21T07:35:27.870816+0000","last_became_peered":"2024-09-21T07:35:27.870816+0000","last_unstale":"2024-09-21T07:36:21.675512+0000","last_undegraded":"2024-09-21T07:36:21.675512+0000","last_fullsized":"2024-09-21T07:36:21.675512+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:04.787450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":61,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.678519+0000","last_change":"2024-09-21T07:35:29.491874+0000","last_active":"2024-09-21T07:36:20.678519+0000","last_peered":"2024-09-21T07:36:20.678519+0000","last_clean":"2024-09-21T07:36:20.678519+0000","last_became_active":"2024-09-21T07:35:26.866381+0000","last_became_peered":"2024-09-21T07:35:26.866381+0000","last_unstale":"2024-09-21T07:36:20.678519+0000","last_undegraded":"2024-09-21T07:36:20.678519+0000","last_fullsized":"2024-09-21T07:36:20.678519+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:36.404647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073235399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":115,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.680854+0000","last_change":"2024-09-21T07:34:24.260046+0000","last_active":"2024-09-21T07:36:20.680854+0000","last_peered":"2024-09-21T07:36:20.680854+0000","last_clean":"2024-09-21T07:36:20.680854+0000","last_became_active":"2024-09-21T07:34:23.273431+0000","last_became_peered":"2024-09-21T07:34:23.273431+0000","last_unstale":"2024-09-21T07:36:20.680854+0000","last_undegraded":"2024-09-21T07:36:20.680854+0000","last_fullsized":"2024-09-21T07:36:20.680854+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:18.876057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019803310000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.708812+0000","last_change":"2024-09-21T07:36:20.703432+0000","last_active":"2024-09-21T07:36:20.708812+0000","last_peered":"2024-09-21T07:36:20.708812+0000","last_clean":"2024-09-21T07:36:20.708812+0000","last_became_active":"2024-09-21T07:36:20.703103+0000","last_became_peered":"2024-09-21T07:36:20.703103+0000","last_unstale":"2024-09-21T07:36:20.708812+0000","last_undegraded":"2024-09-21T07:36:20.708812+0000","last_fullsized":"2024-09-21T07:36:20.708812+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:13.429695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"89'1","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.718935+0000","last_change":"2024-09-21T07:36:19.698179+0000","last_active":"2024-09-21T07:36:20.718935+0000","last_peered":"2024-09-21T07:36:20.718935+0000","last_clean":"2024-09-21T07:36:20.718935+0000","last_became_active":"2024-09-21T07:36:19.696268+0000","last_became_peered":"2024-09-21T07:36:19.696268+0000","last_unstale":"2024-09-21T07:36:20.718935+0000","last_undegraded":"2024-09-21T07:36:20.718935+0000","last_fullsized":"2024-09-21T07:36:20.718935+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:50:26.838770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.675845+0000","last_change":"2024-09-21T07:35:27.871736+0000","last_active":"2024-09-21T07:36:21.675845+0000","last_peered":"2024-09-21T07:36:21.675845+0000","last_clean":"2024-09-21T07:36:21.675845+0000","last_became_active":"2024-09-21T07:35:27.871232+0000","last_became_peered":"2024-09-21T07:35:27.871232+0000","last_unstale":"2024-09-21T07:36:21.675845+0000","last_undegraded":"2024-09-21T07:36:21.675845+0000","last_fullsized":"2024-09-21T07:36:21.675845+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:04.419301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":61,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.677288+0000","last_change":"2024-09-21T07:35:29.491246+0000","last_active":"2024-09-21T07:36:20.677288+0000","last_peered":"2024-09-21T07:36:20.677288+0000","last_clean":"2024-09-21T07:36:20.677288+0000","last_became_active":"2024-09-21T07:35:26.866783+0000","last_became_peered":"2024-09-21T07:35:26.866783+0000","last_unstale":"2024-09-21T07:36:20.677288+0000","last_undegraded":"2024-09-21T07:36:20.677288+0000","last_fullsized":"2024-09-21T07:36:20.677288+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:28.881914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098474099999999992,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":113,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.690170+0000","last_change":"2024-09-21T07:34:25.028640+0000","last_active":"2024-09-21T07:36:20.690170+0000","last_peered":"2024-09-21T07:36:20.690170+0000","last_clean":"2024-09-21T07:36:20.690170+0000","last_became_active":"2024-09-21T07:34:23.260950+0000","last_became_peered":"2024-09-21T07:34:23.260950+0000","last_unstale":"2024-09-21T07:36:20.690170+0000","last_undegraded":"2024-09-21T07:36:20.690170+0000","last_fullsized":"2024-09-21T07:36:20.690170+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:23:42.121398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055135099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.712094+0000","last_change":"2024-09-21T07:36:20.700977+0000","last_active":"2024-09-21T07:36:20.712094+0000","last_peered":"2024-09-21T07:36:20.712094+0000","last_clean":"2024-09-21T07:36:20.712094+0000","last_became_active":"2024-09-21T07:36:20.700633+0000","last_became_peered":"2024-09-21T07:36:20.700633+0000","last_unstale":"2024-09-21T07:36:20.712094+0000","last_undegraded":"2024-09-21T07:36:20.712094+0000","last_fullsized":"2024-09-21T07:36:20.712094+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:28:03.325645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676855+0000","last_change":"2024-09-21T07:36:19.698791+0000","last_active":"2024-09-21T07:36:21.676855+0000","last_peered":"2024-09-21T07:36:21.676855+0000","last_clean":"2024-09-21T07:36:21.676855+0000","last_became_active":"2024-09-21T07:36:19.698440+0000","last_became_peered":"2024-09-21T07:36:19.698440+0000","last_unstale":"2024-09-21T07:36:21.676855+0000","last_undegraded":"2024-09-21T07:36:21.676855+0000","last_fullsized":"2024-09-21T07:36:21.676855+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:54.276692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.687175+0000","last_change":"2024-09-21T07:35:27.872027+0000","last_active":"2024-09-21T07:36:20.687175+0000","last_peered":"2024-09-21T07:36:20.687175+0000","last_clean":"2024-09-21T07:36:20.687175+0000","last_became_active":"2024-09-21T07:35:27.870426+0000","last_became_peered":"2024-09-21T07:35:27.870426+0000","last_unstale":"2024-09-21T07:36:20.687175+0000","last_undegraded":"2024-09-21T07:36:20.687175+0000","last_fullsized":"2024-09-21T07:36:20.687175+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":61,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.677389+0000","last_change":"2024-09-21T07:35:29.491008+0000","last_active":"2024-09-21T07:36:20.677389+0000","last_peered":"2024-09-21T07:36:20.677389+0000","last_clean":"2024-09-21T07:36:20.677389+0000","last_became_active":"2024-09-21T07:35:26.866750+0000","last_became_peered":"2024-09-21T07:35:26.866750+0000","last_unstale":"2024-09-21T07:36:20.677389+0000","last_undegraded":"2024-09-21T07:36:20.677389+0000","last_fullsized":"2024-09-21T07:36:20.677389+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:25.490406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083529700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":115,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.693817+0000","last_change":"2024-09-21T07:34:24.260665+0000","last_active":"2024-09-21T07:36:20.693817+0000","last_peered":"2024-09-21T07:36:20.693817+0000","last_clean":"2024-09-21T07:36:20.693817+0000","last_became_active":"2024-09-21T07:34:23.635909+0000","last_became_peered":"2024-09-21T07:34:23.635909+0000","last_unstale":"2024-09-21T07:36:20.693817+0000","last_undegraded":"2024-09-21T07:36:20.693817+0000","last_fullsized":"2024-09-21T07:36:20.693817+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:44.536885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.678179+0000","last_change":"2024-09-21T07:36:20.719208+0000","last_active":"2024-09-21T07:36:21.678179+0000","last_peered":"2024-09-21T07:36:21.678179+0000","last_clean":"2024-09-21T07:36:21.678179+0000","last_became_active":"2024-09-21T07:36:20.716467+0000","last_became_peered":"2024-09-21T07:36:20.716467+0000","last_unstale":"2024-09-21T07:36:21.678179+0000","last_undegraded":"2024-09-21T07:36:21.678179+0000","last_fullsized":"2024-09-21T07:36:21.678179+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:01.356772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.678055+0000","last_change":"2024-09-21T07:36:19.694266+0000","last_active":"2024-09-21T07:36:20.678055+0000","last_peered":"2024-09-21T07:36:20.678055+0000","last_clean":"2024-09-21T07:36:20.678055+0000","last_became_active":"2024-09-21T07:36:19.693886+0000","last_became_peered":"2024-09-21T07:36:19.693886+0000","last_unstale":"2024-09-21T07:36:20.678055+0000","last_undegraded":"2024-09-21T07:36:20.678055+0000","last_fullsized":"2024-09-21T07:36:20.678055+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:24.625061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.675272+0000","last_change":"2024-09-21T07:35:27.874772+0000","last_active":"2024-09-21T07:36:20.675272+0000","last_peered":"2024-09-21T07:36:20.675272+0000","last_clean":"2024-09-21T07:36:20.675272+0000","last_became_active":"2024-09-21T07:35:27.874434+0000","last_became_peered":"2024-09-21T07:35:27.874434+0000","last_unstale":"2024-09-21T07:36:20.675272+0000","last_undegraded":"2024-09-21T07:36:20.675272+0000","last_fullsized":"2024-09-21T07:36:20.675272+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:31.998755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":66,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677871+0000","last_change":"2024-09-21T07:35:28.859153+0000","last_active":"2024-09-21T07:36:21.677871+0000","last_peered":"2024-09-21T07:36:21.677871+0000","last_clean":"2024-09-21T07:36:21.677871+0000","last_became_active":"2024-09-21T07:35:26.874428+0000","last_became_peered":"2024-09-21T07:35:26.874428+0000","last_unstale":"2024-09-21T07:36:21.677871+0000","last_undegraded":"2024-09-21T07:36:21.677871+0000","last_fullsized":"2024-09-21T07:36:21.677871+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:04.487621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048557299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":118,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676620+0000","last_change":"2024-09-21T07:34:24.538845+0000","last_active":"2024-09-21T07:36:21.676620+0000","last_peered":"2024-09-21T07:36:21.676620+0000","last_clean":"2024-09-21T07:36:21.676620+0000","last_became_active":"2024-09-21T07:34:23.636372+0000","last_became_peered":"2024-09-21T07:34:23.636372+0000","last_unstale":"2024-09-21T07:36:21.676620+0000","last_undegraded":"2024-09-21T07:36:21.676620+0000","last_fullsized":"2024-09-21T07:36:21.676620+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:29.393813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841451,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.678270+0000","last_change":"2024-09-21T07:36:20.712419+0000","last_active":"2024-09-21T07:36:21.678270+0000","last_peered":"2024-09-21T07:36:21.678270+0000","last_clean":"2024-09-21T07:36:21.678270+0000","last_became_active":"2024-09-21T07:36:20.712122+0000","last_became_peered":"2024-09-21T07:36:20.712122+0000","last_unstale":"2024-09-21T07:36:21.678270+0000","last_undegraded":"2024-09-21T07:36:21.678270+0000","last_fullsized":"2024-09-21T07:36:21.678270+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:24:34.481862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677759+0000","last_change":"2024-09-21T07:36:19.698371+0000","last_active":"2024-09-21T07:36:21.677759+0000","last_peered":"2024-09-21T07:36:21.677759+0000","last_clean":"2024-09-21T07:36:21.677759+0000","last_became_active":"2024-09-21T07:36:19.698035+0000","last_became_peered":"2024-09-21T07:36:19.698035+0000","last_unstale":"2024-09-21T07:36:21.677759+0000","last_undegraded":"2024-09-21T07:36:21.677759+0000","last_fullsized":"2024-09-21T07:36:21.677759+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:37:21.904390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.686842+0000","last_change":"2024-09-21T07:35:27.877801+0000","last_active":"2024-09-21T07:36:20.686842+0000","last_peered":"2024-09-21T07:36:20.686842+0000","last_clean":"2024-09-21T07:36:20.686842+0000","last_became_active":"2024-09-21T07:35:27.877464+0000","last_became_peered":"2024-09-21T07:35:27.877464+0000","last_unstale":"2024-09-21T07:36:20.686842+0000","last_undegraded":"2024-09-21T07:36:20.686842+0000","last_fullsized":"2024-09-21T07:36:20.686842+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:08.814929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":65,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.675951+0000","last_change":"2024-09-21T07:35:28.863732+0000","last_active":"2024-09-21T07:36:21.675951+0000","last_peered":"2024-09-21T07:36:21.675951+0000","last_clean":"2024-09-21T07:36:21.675951+0000","last_became_active":"2024-09-21T07:35:26.877392+0000","last_became_peered":"2024-09-21T07:35:26.877392+0000","last_unstale":"2024-09-21T07:36:21.675951+0000","last_undegraded":"2024-09-21T07:36:21.675951+0000","last_fullsized":"2024-09-21T07:36:21.675951+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071425700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":115,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.678617+0000","last_change":"2024-09-21T07:34:24.259686+0000","last_active":"2024-09-21T07:36:20.678617+0000","last_peered":"2024-09-21T07:36:20.678617+0000","last_clean":"2024-09-21T07:36:20.678617+0000","last_became_active":"2024-09-21T07:34:23.274405+0000","last_became_peered":"2024-09-21T07:34:23.274405+0000","last_unstale":"2024-09-21T07:36:20.678617+0000","last_undegraded":"2024-09-21T07:36:20.678617+0000","last_fullsized":"2024-09-21T07:36:20.678617+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:26.304693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022009920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.707102+0000","last_change":"2024-09-21T07:36:20.704336+0000","last_active":"2024-09-21T07:36:20.707102+0000","last_peered":"2024-09-21T07:36:20.707102+0000","last_clean":"2024-09-21T07:36:20.707102+0000","last_became_active":"2024-09-21T07:36:20.704032+0000","last_became_peered":"2024-09-21T07:36:20.704032+0000","last_unstale":"2024-09-21T07:36:20.707102+0000","last_undegraded":"2024-09-21T07:36:20.707102+0000","last_fullsized":"2024-09-21T07:36:20.707102+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:33.475211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.686415+0000","last_change":"2024-09-21T07:36:19.703472+0000","last_active":"2024-09-21T07:36:20.686415+0000","last_peered":"2024-09-21T07:36:20.686415+0000","last_clean":"2024-09-21T07:36:20.686415+0000","last_became_active":"2024-09-21T07:36:19.703094+0000","last_became_peered":"2024-09-21T07:36:19.703094+0000","last_unstale":"2024-09-21T07:36:20.686415+0000","last_undegraded":"2024-09-21T07:36:20.686415+0000","last_fullsized":"2024-09-21T07:36:20.686415+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:59.859873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.677705+0000","last_change":"2024-09-21T07:35:27.867336+0000","last_active":"2024-09-21T07:36:20.677705+0000","last_peered":"2024-09-21T07:36:20.677705+0000","last_clean":"2024-09-21T07:36:20.677705+0000","last_became_active":"2024-09-21T07:35:27.867015+0000","last_became_peered":"2024-09-21T07:35:27.867015+0000","last_unstale":"2024-09-21T07:36:20.677705+0000","last_undegraded":"2024-09-21T07:36:20.677705+0000","last_fullsized":"2024-09-21T07:36:20.677705+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:59.214139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":62,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.693605+0000","last_change":"2024-09-21T07:35:29.493137+0000","last_active":"2024-09-21T07:36:20.693605+0000","last_peered":"2024-09-21T07:36:20.693605+0000","last_clean":"2024-09-21T07:36:20.693605+0000","last_became_active":"2024-09-21T07:35:26.872230+0000","last_became_peered":"2024-09-21T07:35:26.872230+0000","last_unstale":"2024-09-21T07:36:20.693605+0000","last_undegraded":"2024-09-21T07:36:20.693605+0000","last_fullsized":"2024-09-21T07:36:20.693605+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:24.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001197449,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":115,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.686120+0000","last_change":"2024-09-21T07:34:24.258754+0000","last_active":"2024-09-21T07:36:20.686120+0000","last_peered":"2024-09-21T07:36:20.686120+0000","last_clean":"2024-09-21T07:36:20.686120+0000","last_became_active":"2024-09-21T07:34:23.272657+0000","last_became_peered":"2024-09-21T07:34:23.272657+0000","last_unstale":"2024-09-21T07:36:20.686120+0000","last_undegraded":"2024-09-21T07:36:20.686120+0000","last_fullsized":"2024-09-21T07:36:20.686120+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:05.477099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067143200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677796+0000","last_change":"2024-09-21T07:36:20.698701+0000","last_active":"2024-09-21T07:36:21.677796+0000","last_peered":"2024-09-21T07:36:21.677796+0000","last_clean":"2024-09-21T07:36:21.677796+0000","last_became_active":"2024-09-21T07:36:20.698307+0000","last_became_peered":"2024-09-21T07:36:20.698307+0000","last_unstale":"2024-09-21T07:36:21.677796+0000","last_undegraded":"2024-09-21T07:36:21.677796+0000","last_fullsized":"2024-09-21T07:36:21.677796+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:30.608814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677284+0000","last_change":"2024-09-21T07:36:19.697623+0000","last_active":"2024-09-21T07:36:21.677284+0000","last_peered":"2024-09-21T07:36:21.677284+0000","last_clean":"2024-09-21T07:36:21.677284+0000","last_became_active":"2024-09-21T07:36:19.697300+0000","last_became_peered":"2024-09-21T07:36:19.697300+0000","last_unstale":"2024-09-21T07:36:21.677284+0000","last_undegraded":"2024-09-21T07:36:21.677284+0000","last_fullsized":"2024-09-21T07:36:21.677284+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:00.623479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.675995+0000","last_change":"2024-09-21T07:35:27.876021+0000","last_active":"2024-09-21T07:36:21.675995+0000","last_peered":"2024-09-21T07:36:21.675995+0000","last_clean":"2024-09-21T07:36:21.675995+0000","last_became_active":"2024-09-21T07:35:27.875724+0000","last_became_peered":"2024-09-21T07:35:27.875724+0000","last_unstale":"2024-09-21T07:36:21.675995+0000","last_undegraded":"2024-09-21T07:36:21.675995+0000","last_fullsized":"2024-09-21T07:36:21.675995+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:03.022812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":62,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.689003+0000","last_change":"2024-09-21T07:35:29.492570+0000","last_active":"2024-09-21T07:36:20.689003+0000","last_peered":"2024-09-21T07:36:20.689003+0000","last_clean":"2024-09-21T07:36:20.689003+0000","last_became_active":"2024-09-21T07:35:26.874316+0000","last_became_peered":"2024-09-21T07:35:26.874316+0000","last_unstale":"2024-09-21T07:36:20.689003+0000","last_undegraded":"2024-09-21T07:36:20.689003+0000","last_fullsized":"2024-09-21T07:36:20.689003+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:16.512323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011990760000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":116,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.675452+0000","last_change":"2024-09-21T07:34:25.085232+0000","last_active":"2024-09-21T07:36:21.675452+0000","last_peered":"2024-09-21T07:36:21.675452+0000","last_clean":"2024-09-21T07:36:21.675452+0000","last_became_active":"2024-09-21T07:34:23.277807+0000","last_became_peered":"2024-09-21T07:34:23.277807+0000","last_unstale":"2024-09-21T07:36:21.675452+0000","last_undegraded":"2024-09-21T07:36:21.675452+0000","last_fullsized":"2024-09-21T07:36:21.675452+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:36.362919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086619099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.675151+0000","last_change":"2024-09-21T07:36:20.708922+0000","last_active":"2024-09-21T07:36:21.675151+0000","last_peered":"2024-09-21T07:36:21.675151+0000","last_clean":"2024-09-21T07:36:21.675151+0000","last_became_active":"2024-09-21T07:36:20.707981+0000","last_became_peered":"2024-09-21T07:36:20.707981+0000","last_unstale":"2024-09-21T07:36:21.675151+0000","last_undegraded":"2024-09-21T07:36:21.675151+0000","last_fullsized":"2024-09-21T07:36:21.675151+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:55.533464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"89'2","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.722353+0000","last_change":"2024-09-21T07:36:19.700137+0000","last_active":"2024-09-21T07:36:20.722353+0000","last_peered":"2024-09-21T07:36:20.722353+0000","last_clean":"2024-09-21T07:36:20.722353+0000","last_became_active":"2024-09-21T07:36:19.699771+0000","last_became_peered":"2024-09-21T07:36:19.699771+0000","last_unstale":"2024-09-21T07:36:20.722353+0000","last_undegraded":"2024-09-21T07:36:20.722353+0000","last_fullsized":"2024-09-21T07:36:20.722353+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:50:54.616191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.688222+0000","last_change":"2024-09-21T07:35:27.872798+0000","last_active":"2024-09-21T07:36:20.688222+0000","last_peered":"2024-09-21T07:36:20.688222+0000","last_clean":"2024-09-21T07:36:20.688222+0000","last_became_active":"2024-09-21T07:35:27.872483+0000","last_became_peered":"2024-09-21T07:35:27.872483+0000","last_unstale":"2024-09-21T07:36:20.688222+0000","last_undegraded":"2024-09-21T07:36:20.688222+0000","last_fullsized":"2024-09-21T07:36:20.688222+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:11.933469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.686635+0000","last_change":"2024-09-21T07:35:29.206723+0000","last_active":"2024-09-21T07:36:20.686635+0000","last_peered":"2024-09-21T07:36:20.686635+0000","last_clean":"2024-09-21T07:36:20.686635+0000","last_became_active":"2024-09-21T07:35:26.872377+0000","last_became_peered":"2024-09-21T07:35:26.872377+0000","last_unstale":"2024-09-21T07:36:20.686635+0000","last_undegraded":"2024-09-21T07:36:20.686635+0000","last_fullsized":"2024-09-21T07:36:20.686635+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:29.086862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083077200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":110,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.676872+0000","last_change":"2024-09-21T07:34:24.597672+0000","last_active":"2024-09-21T07:36:20.676872+0000","last_peered":"2024-09-21T07:36:20.676872+0000","last_clean":"2024-09-21T07:36:20.676872+0000","last_became_active":"2024-09-21T07:34:23.267770+0000","last_became_peered":"2024-09-21T07:34:23.267770+0000","last_unstale":"2024-09-21T07:36:20.676872+0000","last_undegraded":"2024-09-21T07:36:20.676872+0000","last_fullsized":"2024-09-21T07:36:20.676872+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:51.629476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086378500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.686506+0000","last_change":"2024-09-21T07:36:20.681896+0000","last_active":"2024-09-21T07:36:20.686506+0000","last_peered":"2024-09-21T07:36:20.686506+0000","last_clean":"2024-09-21T07:36:20.686506+0000","last_became_active":"2024-09-21T07:36:20.681504+0000","last_became_peered":"2024-09-21T07:36:20.681504+0000","last_unstale":"2024-09-21T07:36:20.686506+0000","last_undegraded":"2024-09-21T07:36:20.686506+0000","last_fullsized":"2024-09-21T07:36:20.686506+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:00.189091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"90'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.678850+0000","last_change":"2024-09-21T07:36:19.699176+0000","last_active":"2024-09-21T07:36:21.678850+0000","last_peered":"2024-09-21T07:36:21.678850+0000","last_clean":"2024-09-21T07:36:21.678850+0000","last_became_active":"2024-09-21T07:36:19.698605+0000","last_became_peered":"2024-09-21T07:36:19.698605+0000","last_unstale":"2024-09-21T07:36:21.678850+0000","last_undegraded":"2024-09-21T07:36:21.678850+0000","last_fullsized":"2024-09-21T07:36:21.678850+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:36.907692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.687371+0000","last_change":"2024-09-21T07:35:27.870121+0000","last_active":"2024-09-21T07:36:20.687371+0000","last_peered":"2024-09-21T07:36:20.687371+0000","last_clean":"2024-09-21T07:36:20.687371+0000","last_became_active":"2024-09-21T07:35:27.869811+0000","last_became_peered":"2024-09-21T07:35:27.869811+0000","last_unstale":"2024-09-21T07:36:20.687371+0000","last_undegraded":"2024-09-21T07:36:20.687371+0000","last_fullsized":"2024-09-21T07:36:20.687371+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:31.778461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":64,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677831+0000","last_change":"2024-09-21T07:35:28.860699+0000","last_active":"2024-09-21T07:36:21.677831+0000","last_peered":"2024-09-21T07:36:21.677831+0000","last_clean":"2024-09-21T07:36:21.677831+0000","last_became_active":"2024-09-21T07:35:26.876779+0000","last_became_peered":"2024-09-21T07:35:26.876779+0000","last_unstale":"2024-09-21T07:36:21.677831+0000","last_undegraded":"2024-09-21T07:36:21.677831+0000","last_fullsized":"2024-09-21T07:36:21.677831+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:28.286403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000913188,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":116,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.678558+0000","last_change":"2024-09-21T07:34:24.260181+0000","last_active":"2024-09-21T07:36:21.678558+0000","last_peered":"2024-09-21T07:36:21.678558+0000","last_clean":"2024-09-21T07:36:21.678558+0000","last_became_active":"2024-09-21T07:34:23.275580+0000","last_became_peered":"2024-09-21T07:34:23.275580+0000","last_unstale":"2024-09-21T07:36:21.678558+0000","last_undegraded":"2024-09-21T07:36:21.678558+0000","last_fullsized":"2024-09-21T07:36:21.678558+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:49.857622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014706980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.710262+0000","last_change":"2024-09-21T07:36:20.702121+0000","last_active":"2024-09-21T07:36:20.710262+0000","last_peered":"2024-09-21T07:36:20.710262+0000","last_clean":"2024-09-21T07:36:20.710262+0000","last_became_active":"2024-09-21T07:36:20.696640+0000","last_became_peered":"2024-09-21T07:36:20.696640+0000","last_unstale":"2024-09-21T07:36:20.710262+0000","last_undegraded":"2024-09-21T07:36:20.710262+0000","last_fullsized":"2024-09-21T07:36:20.710262+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:48.234657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.687152+0000","last_change":"2024-09-21T07:36:19.700980+0000","last_active":"2024-09-21T07:36:20.687152+0000","last_peered":"2024-09-21T07:36:20.687152+0000","last_clean":"2024-09-21T07:36:20.687152+0000","last_became_active":"2024-09-21T07:36:19.700574+0000","last_became_peered":"2024-09-21T07:36:19.700574+0000","last_unstale":"2024-09-21T07:36:20.687152+0000","last_undegraded":"2024-09-21T07:36:20.687152+0000","last_fullsized":"2024-09-21T07:36:20.687152+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:53.477806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.692232+0000","last_change":"2024-09-21T07:35:27.875393+0000","last_active":"2024-09-21T07:36:20.692232+0000","last_peered":"2024-09-21T07:36:20.692232+0000","last_clean":"2024-09-21T07:36:20.692232+0000","last_became_active":"2024-09-21T07:35:27.874671+0000","last_became_peered":"2024-09-21T07:35:27.874671+0000","last_unstale":"2024-09-21T07:36:20.692232+0000","last_undegraded":"2024-09-21T07:36:20.692232+0000","last_fullsized":"2024-09-21T07:36:20.692232+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.686639+0000","last_change":"2024-09-21T07:35:29.206296+0000","last_active":"2024-09-21T07:36:20.686639+0000","last_peered":"2024-09-21T07:36:20.686639+0000","last_clean":"2024-09-21T07:36:20.686639+0000","last_became_active":"2024-09-21T07:35:26.874575+0000","last_became_peered":"2024-09-21T07:35:26.874575+0000","last_unstale":"2024-09-21T07:36:20.686639+0000","last_undegraded":"2024-09-21T07:36:20.686639+0000","last_fullsized":"2024-09-21T07:36:20.686639+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:22.771154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061011599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":115,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.681862+0000","last_change":"2024-09-21T07:34:24.259112+0000","last_active":"2024-09-21T07:36:20.681862+0000","last_peered":"2024-09-21T07:36:20.681862+0000","last_clean":"2024-09-21T07:36:20.681862+0000","last_became_active":"2024-09-21T07:34:23.635444+0000","last_became_peered":"2024-09-21T07:34:23.635444+0000","last_unstale":"2024-09-21T07:36:20.681862+0000","last_undegraded":"2024-09-21T07:36:20.681862+0000","last_fullsized":"2024-09-21T07:36:20.681862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:46.255385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001952636,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677302+0000","last_change":"2024-09-21T07:36:20.702519+0000","last_active":"2024-09-21T07:36:21.677302+0000","last_peered":"2024-09-21T07:36:21.677302+0000","last_clean":"2024-09-21T07:36:21.677302+0000","last_became_active":"2024-09-21T07:36:20.701778+0000","last_became_peered":"2024-09-21T07:36:20.701778+0000","last_unstale":"2024-09-21T07:36:21.677302+0000","last_undegraded":"2024-09-21T07:36:21.677302+0000","last_fullsized":"2024-09-21T07:36:21.677302+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:00:11.636554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.689082+0000","last_change":"2024-09-21T07:36:19.703789+0000","last_active":"2024-09-21T07:36:20.689082+0000","last_peered":"2024-09-21T07:36:20.689082+0000","last_clean":"2024-09-21T07:36:20.689082+0000","last_became_active":"2024-09-21T07:36:19.703481+0000","last_became_peered":"2024-09-21T07:36:19.703481+0000","last_unstale":"2024-09-21T07:36:20.689082+0000","last_undegraded":"2024-09-21T07:36:20.689082+0000","last_fullsized":"2024-09-21T07:36:20.689082+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:02.336588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.693566+0000","last_change":"2024-09-21T07:35:27.878356+0000","last_active":"2024-09-21T07:36:20.693566+0000","last_peered":"2024-09-21T07:36:20.693566+0000","last_clean":"2024-09-21T07:36:20.693566+0000","last_became_active":"2024-09-21T07:35:27.877982+0000","last_became_peered":"2024-09-21T07:35:27.877982+0000","last_unstale":"2024-09-21T07:36:20.693566+0000","last_undegraded":"2024-09-21T07:36:20.693566+0000","last_fullsized":"2024-09-21T07:36:20.693566+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":62,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.684130+0000","last_change":"2024-09-21T07:35:29.053131+0000","last_active":"2024-09-21T07:36:20.684130+0000","last_peered":"2024-09-21T07:36:20.684130+0000","last_clean":"2024-09-21T07:36:20.684130+0000","last_became_active":"2024-09-21T07:35:26.870516+0000","last_became_peered":"2024-09-21T07:35:26.870516+0000","last_unstale":"2024-09-21T07:36:20.684130+0000","last_undegraded":"2024-09-21T07:36:20.684130+0000","last_fullsized":"2024-09-21T07:36:20.684130+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:10.924491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070705699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":118,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677497+0000","last_change":"2024-09-21T07:34:24.539152+0000","last_active":"2024-09-21T07:36:21.677497+0000","last_peered":"2024-09-21T07:36:21.677497+0000","last_clean":"2024-09-21T07:36:21.677497+0000","last_became_active":"2024-09-21T07:34:23.259573+0000","last_became_peered":"2024-09-21T07:34:23.259573+0000","last_unstale":"2024-09-21T07:36:21.677497+0000","last_undegraded":"2024-09-21T07:36:21.677497+0000","last_fullsized":"2024-09-21T07:36:21.677497+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:26.986900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058108400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.710149+0000","last_change":"2024-09-21T07:36:20.696988+0000","last_active":"2024-09-21T07:36:20.710149+0000","last_peered":"2024-09-21T07:36:20.710149+0000","last_clean":"2024-09-21T07:36:20.710149+0000","last_became_active":"2024-09-21T07:36:20.696662+0000","last_became_peered":"2024-09-21T07:36:20.696662+0000","last_unstale":"2024-09-21T07:36:20.710149+0000","last_undegraded":"2024-09-21T07:36:20.710149+0000","last_fullsized":"2024-09-21T07:36:20.710149+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:20.897738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"89'1","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.714231+0000","last_change":"2024-09-21T07:36:19.700460+0000","last_active":"2024-09-21T07:36:20.714231+0000","last_peered":"2024-09-21T07:36:20.714231+0000","last_clean":"2024-09-21T07:36:20.714231+0000","last_became_active":"2024-09-21T07:36:19.697272+0000","last_became_peered":"2024-09-21T07:36:19.697272+0000","last_unstale":"2024-09-21T07:36:20.714231+0000","last_undegraded":"2024-09-21T07:36:20.714231+0000","last_fullsized":"2024-09-21T07:36:20.714231+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:30.992886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.684046+0000","last_change":"2024-09-21T07:35:27.873726+0000","last_active":"2024-09-21T07:36:20.684046+0000","last_peered":"2024-09-21T07:36:20.684046+0000","last_clean":"2024-09-21T07:36:20.684046+0000","last_became_active":"2024-09-21T07:35:27.871421+0000","last_became_peered":"2024-09-21T07:35:27.871421+0000","last_unstale":"2024-09-21T07:36:20.684046+0000","last_undegraded":"2024-09-21T07:36:20.684046+0000","last_fullsized":"2024-09-21T07:36:20.684046+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:00.168915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":64,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.678383+0000","last_change":"2024-09-21T07:35:28.861032+0000","last_active":"2024-09-21T07:36:21.678383+0000","last_peered":"2024-09-21T07:36:21.678383+0000","last_clean":"2024-09-21T07:36:21.678383+0000","last_became_active":"2024-09-21T07:35:26.873149+0000","last_became_peered":"2024-09-21T07:35:26.873149+0000","last_unstale":"2024-09-21T07:36:21.678383+0000","last_undegraded":"2024-09-21T07:36:21.678383+0000","last_fullsized":"2024-09-21T07:36:21.678383+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:59.435981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011988369999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":115,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.690817+0000","last_change":"2024-09-21T07:34:24.259902+0000","last_active":"2024-09-21T07:36:20.690817+0000","last_peered":"2024-09-21T07:36:20.690817+0000","last_clean":"2024-09-21T07:36:20.690817+0000","last_became_active":"2024-09-21T07:34:23.271902+0000","last_became_peered":"2024-09-21T07:34:23.271902+0000","last_unstale":"2024-09-21T07:36:20.690817+0000","last_undegraded":"2024-09-21T07:36:20.690817+0000","last_fullsized":"2024-09-21T07:36:20.690817+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:59.439525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001686358,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676365+0000","last_change":"2024-09-21T07:36:20.709720+0000","last_active":"2024-09-21T07:36:21.676365+0000","last_peered":"2024-09-21T07:36:21.676365+0000","last_clean":"2024-09-21T07:36:21.676365+0000","last_became_active":"2024-09-21T07:36:20.709378+0000","last_became_peered":"2024-09-21T07:36:20.709378+0000","last_unstale":"2024-09-21T07:36:21.676365+0000","last_undegraded":"2024-09-21T07:36:21.676365+0000","last_fullsized":"2024-09-21T07:36:21.676365+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:29.696718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"89'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676475+0000","last_change":"2024-09-21T07:36:19.695455+0000","last_active":"2024-09-21T07:36:21.676475+0000","last_peered":"2024-09-21T07:36:21.676475+0000","last_clean":"2024-09-21T07:36:21.676475+0000","last_became_active":"2024-09-21T07:36:19.695140+0000","last_became_peered":"2024-09-21T07:36:19.695140+0000","last_unstale":"2024-09-21T07:36:21.676475+0000","last_undegraded":"2024-09-21T07:36:21.676475+0000","last_fullsized":"2024-09-21T07:36:21.676475+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:20.147254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.675034+0000","last_change":"2024-09-21T07:35:27.872637+0000","last_active":"2024-09-21T07:36:21.675034+0000","last_peered":"2024-09-21T07:36:21.675034+0000","last_clean":"2024-09-21T07:36:21.675034+0000","last_became_active":"2024-09-21T07:35:27.872149+0000","last_became_peered":"2024-09-21T07:35:27.872149+0000","last_unstale":"2024-09-21T07:36:21.675034+0000","last_undegraded":"2024-09-21T07:36:21.675034+0000","last_fullsized":"2024-09-21T07:36:21.675034+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:27.697872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":62,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.693013+0000","last_change":"2024-09-21T07:35:29.493362+0000","last_active":"2024-09-21T07:36:20.693013+0000","last_peered":"2024-09-21T07:36:20.693013+0000","last_clean":"2024-09-21T07:36:20.693013+0000","last_became_active":"2024-09-21T07:35:26.873018+0000","last_became_peered":"2024-09-21T07:35:26.873018+0000","last_unstale":"2024-09-21T07:36:20.693013+0000","last_undegraded":"2024-09-21T07:36:20.693013+0000","last_fullsized":"2024-09-21T07:36:20.693013+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:02.460557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012496510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":115,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.678420+0000","last_change":"2024-09-21T07:34:24.259662+0000","last_active":"2024-09-21T07:36:20.678420+0000","last_peered":"2024-09-21T07:36:20.678420+0000","last_clean":"2024-09-21T07:36:20.678420+0000","last_became_active":"2024-09-21T07:34:23.278812+0000","last_became_peered":"2024-09-21T07:34:23.278812+0000","last_unstale":"2024-09-21T07:36:20.678420+0000","last_undegraded":"2024-09-21T07:36:20.678420+0000","last_fullsized":"2024-09-21T07:36:20.678420+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:49.829974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356463,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.705379+0000","last_change":"2024-09-21T07:36:20.694067+0000","last_active":"2024-09-21T07:36:20.705379+0000","last_peered":"2024-09-21T07:36:20.705379+0000","last_clean":"2024-09-21T07:36:20.705379+0000","last_became_active":"2024-09-21T07:36:20.693747+0000","last_became_peered":"2024-09-21T07:36:20.693747+0000","last_unstale":"2024-09-21T07:36:20.705379+0000","last_undegraded":"2024-09-21T07:36:20.705379+0000","last_fullsized":"2024-09-21T07:36:20.705379+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:24:21.152110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.677366+0000","last_change":"2024-09-21T07:36:19.697871+0000","last_active":"2024-09-21T07:36:20.677366+0000","last_peered":"2024-09-21T07:36:20.677366+0000","last_clean":"2024-09-21T07:36:20.677366+0000","last_became_active":"2024-09-21T07:36:19.697500+0000","last_became_peered":"2024-09-21T07:36:19.697500+0000","last_unstale":"2024-09-21T07:36:20.677366+0000","last_undegraded":"2024-09-21T07:36:20.677366+0000","last_fullsized":"2024-09-21T07:36:20.677366+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:51.365852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.677563+0000","last_change":"2024-09-21T07:35:27.868448+0000","last_active":"2024-09-21T07:36:20.677563+0000","last_peered":"2024-09-21T07:36:20.677563+0000","last_clean":"2024-09-21T07:36:20.677563+0000","last_became_active":"2024-09-21T07:35:27.868105+0000","last_became_peered":"2024-09-21T07:35:27.868105+0000","last_unstale":"2024-09-21T07:36:20.677563+0000","last_undegraded":"2024-09-21T07:36:20.677563+0000","last_fullsized":"2024-09-21T07:36:20.677563+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:41.468702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":62,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.693204+0000","last_change":"2024-09-21T07:35:29.492974+0000","last_active":"2024-09-21T07:36:20.693204+0000","last_peered":"2024-09-21T07:36:20.693204+0000","last_clean":"2024-09-21T07:36:20.693204+0000","last_became_active":"2024-09-21T07:35:26.872165+0000","last_became_peered":"2024-09-21T07:35:26.872165+0000","last_unstale":"2024-09-21T07:36:20.693204+0000","last_undegraded":"2024-09-21T07:36:20.693204+0000","last_fullsized":"2024-09-21T07:36:20.693204+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:59.912520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013795179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":116,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676110+0000","last_change":"2024-09-21T07:34:25.084861+0000","last_active":"2024-09-21T07:36:21.676110+0000","last_peered":"2024-09-21T07:36:21.676110+0000","last_clean":"2024-09-21T07:36:21.676110+0000","last_became_active":"2024-09-21T07:34:23.277708+0000","last_became_peered":"2024-09-21T07:34:23.277708+0000","last_unstale":"2024-09-21T07:36:21.676110+0000","last_undegraded":"2024-09-21T07:36:21.676110+0000","last_fullsized":"2024-09-21T07:36:21.676110+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:40.816488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000597566,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.714792+0000","last_change":"2024-09-21T07:36:20.700998+0000","last_active":"2024-09-21T07:36:20.714792+0000","last_peered":"2024-09-21T07:36:20.714792+0000","last_clean":"2024-09-21T07:36:20.714792+0000","last_became_active":"2024-09-21T07:36:20.700506+0000","last_became_peered":"2024-09-21T07:36:20.700506+0000","last_unstale":"2024-09-21T07:36:20.714792+0000","last_undegraded":"2024-09-21T07:36:20.714792+0000","last_fullsized":"2024-09-21T07:36:20.714792+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:04:20.035234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.677213+0000","last_change":"2024-09-21T07:36:19.704544+0000","last_active":"2024-09-21T07:36:20.677213+0000","last_peered":"2024-09-21T07:36:20.677213+0000","last_clean":"2024-09-21T07:36:20.677213+0000","last_became_active":"2024-09-21T07:36:19.703753+0000","last_became_peered":"2024-09-21T07:36:19.703753+0000","last_unstale":"2024-09-21T07:36:20.677213+0000","last_undegraded":"2024-09-21T07:36:20.677213+0000","last_fullsized":"2024-09-21T07:36:20.677213+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:22:06.078848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.678127+0000","last_change":"2024-09-21T07:35:27.874640+0000","last_active":"2024-09-21T07:36:20.678127+0000","last_peered":"2024-09-21T07:36:20.678127+0000","last_clean":"2024-09-21T07:36:20.678127+0000","last_became_active":"2024-09-21T07:35:27.874265+0000","last_became_peered":"2024-09-21T07:35:27.874265+0000","last_unstale":"2024-09-21T07:36:20.678127+0000","last_undegraded":"2024-09-21T07:36:20.678127+0000","last_fullsized":"2024-09-21T07:36:20.678127+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:45.754654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.688457+0000","last_change":"2024-09-21T07:35:29.207392+0000","last_active":"2024-09-21T07:36:20.688457+0000","last_peered":"2024-09-21T07:36:20.688457+0000","last_clean":"2024-09-21T07:36:20.688457+0000","last_became_active":"2024-09-21T07:35:26.870917+0000","last_became_peered":"2024-09-21T07:35:26.870917+0000","last_unstale":"2024-09-21T07:36:20.688457+0000","last_undegraded":"2024-09-21T07:36:20.688457+0000","last_fullsized":"2024-09-21T07:36:20.688457+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:59.159937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059198499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":115,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.677405+0000","last_change":"2024-09-21T07:34:24.258709+0000","last_active":"2024-09-21T07:36:20.677405+0000","last_peered":"2024-09-21T07:36:20.677405+0000","last_clean":"2024-09-21T07:36:20.677405+0000","last_became_active":"2024-09-21T07:34:23.279042+0000","last_became_peered":"2024-09-21T07:34:23.279042+0000","last_unstale":"2024-09-21T07:36:20.677405+0000","last_undegraded":"2024-09-21T07:36:20.677405+0000","last_fullsized":"2024-09-21T07:36:20.677405+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:01.762117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019282100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.675441+0000","last_change":"2024-09-21T07:36:20.710007+0000","last_active":"2024-09-21T07:36:21.675441+0000","last_peered":"2024-09-21T07:36:21.675441+0000","last_clean":"2024-09-21T07:36:21.675441+0000","last_became_active":"2024-09-21T07:36:20.708378+0000","last_became_peered":"2024-09-21T07:36:20.708378+0000","last_unstale":"2024-09-21T07:36:21.675441+0000","last_undegraded":"2024-09-21T07:36:21.675441+0000","last_fullsized":"2024-09-21T07:36:21.675441+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:52.256493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"89'4","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.724439+0000","last_change":"2024-09-21T07:36:19.695955+0000","last_active":"2024-09-21T07:36:20.724439+0000","last_peered":"2024-09-21T07:36:20.724439+0000","last_clean":"2024-09-21T07:36:20.724439+0000","last_became_active":"2024-09-21T07:36:19.695677+0000","last_became_peered":"2024-09-21T07:36:19.695677+0000","last_unstale":"2024-09-21T07:36:20.724439+0000","last_undegraded":"2024-09-21T07:36:20.724439+0000","last_fullsized":"2024-09-21T07:36:20.724439+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:08.660370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.675385+0000","last_change":"2024-09-21T07:35:27.870260+0000","last_active":"2024-09-21T07:36:21.675385+0000","last_peered":"2024-09-21T07:36:21.675385+0000","last_clean":"2024-09-21T07:36:21.675385+0000","last_became_active":"2024-09-21T07:35:27.869448+0000","last_became_peered":"2024-09-21T07:35:27.869448+0000","last_unstale":"2024-09-21T07:36:21.675385+0000","last_undegraded":"2024-09-21T07:36:21.675385+0000","last_fullsized":"2024-09-21T07:36:21.675385+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:16.946790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":61,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.678046+0000","last_change":"2024-09-21T07:35:29.386367+0000","last_active":"2024-09-21T07:36:20.678046+0000","last_peered":"2024-09-21T07:36:20.678046+0000","last_clean":"2024-09-21T07:36:20.678046+0000","last_became_active":"2024-09-21T07:35:26.879238+0000","last_became_peered":"2024-09-21T07:35:26.879238+0000","last_unstale":"2024-09-21T07:36:20.678046+0000","last_undegraded":"2024-09-21T07:36:20.678046+0000","last_fullsized":"2024-09-21T07:36:20.678046+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:16.528260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092223100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":116,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676120+0000","last_change":"2024-09-21T07:34:25.085635+0000","last_active":"2024-09-21T07:36:21.676120+0000","last_peered":"2024-09-21T07:36:21.676120+0000","last_clean":"2024-09-21T07:36:21.676120+0000","last_became_active":"2024-09-21T07:34:23.273566+0000","last_became_peered":"2024-09-21T07:34:23.273566+0000","last_unstale":"2024-09-21T07:36:21.676120+0000","last_undegraded":"2024-09-21T07:36:21.676120+0000","last_fullsized":"2024-09-21T07:36:21.676120+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:14.142272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079897300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677352+0000","last_change":"2024-09-21T07:36:20.716130+0000","last_active":"2024-09-21T07:36:21.677352+0000","last_peered":"2024-09-21T07:36:21.677352+0000","last_clean":"2024-09-21T07:36:21.677352+0000","last_became_active":"2024-09-21T07:36:20.714528+0000","last_became_peered":"2024-09-21T07:36:20.714528+0000","last_unstale":"2024-09-21T07:36:21.677352+0000","last_undegraded":"2024-09-21T07:36:21.677352+0000","last_fullsized":"2024-09-21T07:36:21.677352+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:54:18.393992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"89'1","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.678556+0000","last_change":"2024-09-21T07:36:19.687469+0000","last_active":"2024-09-21T07:36:21.678556+0000","last_peered":"2024-09-21T07:36:21.678556+0000","last_clean":"2024-09-21T07:36:21.678556+0000","last_became_active":"2024-09-21T07:36:19.686496+0000","last_became_peered":"2024-09-21T07:36:19.686496+0000","last_unstale":"2024-09-21T07:36:21.678556+0000","last_undegraded":"2024-09-21T07:36:21.678556+0000","last_fullsized":"2024-09-21T07:36:21.678556+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:41.099573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676502+0000","last_change":"2024-09-21T07:35:27.878728+0000","last_active":"2024-09-21T07:36:21.676502+0000","last_peered":"2024-09-21T07:36:21.676502+0000","last_clean":"2024-09-21T07:36:21.676502+0000","last_became_active":"2024-09-21T07:35:27.878335+0000","last_became_peered":"2024-09-21T07:35:27.878335+0000","last_unstale":"2024-09-21T07:36:21.676502+0000","last_undegraded":"2024-09-21T07:36:21.676502+0000","last_fullsized":"2024-09-21T07:36:21.676502+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:25:37.731771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":66,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676467+0000","last_change":"2024-09-21T07:35:28.858418+0000","last_active":"2024-09-21T07:36:21.676467+0000","last_peered":"2024-09-21T07:36:21.676467+0000","last_clean":"2024-09-21T07:36:21.676467+0000","last_became_active":"2024-09-21T07:35:26.876944+0000","last_became_peered":"2024-09-21T07:35:26.876944+0000","last_unstale":"2024-09-21T07:36:21.676467+0000","last_undegraded":"2024-09-21T07:36:21.676467+0000","last_fullsized":"2024-09-21T07:36:21.676467+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:04.514304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090862799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":110,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.677114+0000","last_change":"2024-09-21T07:34:24.597926+0000","last_active":"2024-09-21T07:36:20.677114+0000","last_peered":"2024-09-21T07:36:20.677114+0000","last_clean":"2024-09-21T07:36:20.677114+0000","last_became_active":"2024-09-21T07:34:23.268926+0000","last_became_peered":"2024-09-21T07:34:23.268926+0000","last_unstale":"2024-09-21T07:36:20.677114+0000","last_undegraded":"2024-09-21T07:36:20.677114+0000","last_fullsized":"2024-09-21T07:36:20.677114+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:46.190752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101851,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.691996+0000","last_change":"2024-09-21T07:36:20.689928+0000","last_active":"2024-09-21T07:36:20.691996+0000","last_peered":"2024-09-21T07:36:20.691996+0000","last_clean":"2024-09-21T07:36:20.691996+0000","last_became_active":"2024-09-21T07:36:20.689591+0000","last_became_peered":"2024-09-21T07:36:20.689591+0000","last_unstale":"2024-09-21T07:36:20.691996+0000","last_undegraded":"2024-09-21T07:36:20.691996+0000","last_fullsized":"2024-09-21T07:36:20.691996+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:18.360631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"89'2","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.716222+0000","last_change":"2024-09-21T07:36:19.696125+0000","last_active":"2024-09-21T07:36:20.716222+0000","last_peered":"2024-09-21T07:36:20.716222+0000","last_clean":"2024-09-21T07:36:20.716222+0000","last_became_active":"2024-09-21T07:36:19.695030+0000","last_became_peered":"2024-09-21T07:36:19.695030+0000","last_unstale":"2024-09-21T07:36:20.716222+0000","last_undegraded":"2024-09-21T07:36:20.716222+0000","last_fullsized":"2024-09-21T07:36:20.716222+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:37:22.026182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.676470+0000","last_change":"2024-09-21T07:35:27.877426+0000","last_active":"2024-09-21T07:36:20.676470+0000","last_peered":"2024-09-21T07:36:20.676470+0000","last_clean":"2024-09-21T07:36:20.676470+0000","last_became_active":"2024-09-21T07:35:27.877050+0000","last_became_peered":"2024-09-21T07:35:27.877050+0000","last_unstale":"2024-09-21T07:36:20.676470+0000","last_undegraded":"2024-09-21T07:36:20.676470+0000","last_fullsized":"2024-09-21T07:36:20.676470+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:36.268334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":65,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.675186+0000","last_change":"2024-09-21T07:35:28.862273+0000","last_active":"2024-09-21T07:36:21.675186+0000","last_peered":"2024-09-21T07:36:21.675186+0000","last_clean":"2024-09-21T07:36:21.675186+0000","last_became_active":"2024-09-21T07:35:26.874011+0000","last_became_peered":"2024-09-21T07:35:26.874011+0000","last_unstale":"2024-09-21T07:36:21.675186+0000","last_undegraded":"2024-09-21T07:36:21.675186+0000","last_fullsized":"2024-09-21T07:36:21.675186+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:11.986950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071644799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":116,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.678184+0000","last_change":"2024-09-21T07:34:24.259794+0000","last_active":"2024-09-21T07:36:21.678184+0000","last_peered":"2024-09-21T07:36:21.678184+0000","last_clean":"2024-09-21T07:36:21.678184+0000","last_became_active":"2024-09-21T07:34:23.265679+0000","last_became_peered":"2024-09-21T07:34:23.265679+0000","last_unstale":"2024-09-21T07:36:21.678184+0000","last_undegraded":"2024-09-21T07:36:21.678184+0000","last_fullsized":"2024-09-21T07:36:21.678184+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:02.575796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001298518,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677003+0000","last_change":"2024-09-21T07:36:20.680585+0000","last_active":"2024-09-21T07:36:21.677003+0000","last_peered":"2024-09-21T07:36:21.677003+0000","last_clean":"2024-09-21T07:36:21.677003+0000","last_became_active":"2024-09-21T07:36:20.679797+0000","last_became_peered":"2024-09-21T07:36:20.679797+0000","last_unstale":"2024-09-21T07:36:21.677003+0000","last_undegraded":"2024-09-21T07:36:21.677003+0000","last_fullsized":"2024-09-21T07:36:21.677003+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:34:48.492726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"89'2","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.717271+0000","last_change":"2024-09-21T07:36:19.704551+0000","last_active":"2024-09-21T07:36:20.717271+0000","last_peered":"2024-09-21T07:36:20.717271+0000","last_clean":"2024-09-21T07:36:20.717271+0000","last_became_active":"2024-09-21T07:36:19.704219+0000","last_became_peered":"2024-09-21T07:36:19.704219+0000","last_unstale":"2024-09-21T07:36:20.717271+0000","last_undegraded":"2024-09-21T07:36:20.717271+0000","last_fullsized":"2024-09-21T07:36:20.717271+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:28:01.162360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.686848+0000","last_change":"2024-09-21T07:35:27.877725+0000","last_active":"2024-09-21T07:36:20.686848+0000","last_peered":"2024-09-21T07:36:20.686848+0000","last_clean":"2024-09-21T07:36:20.686848+0000","last_became_active":"2024-09-21T07:35:27.877398+0000","last_became_peered":"2024-09-21T07:35:27.877398+0000","last_unstale":"2024-09-21T07:36:20.686848+0000","last_undegraded":"2024-09-21T07:36:20.686848+0000","last_fullsized":"2024-09-21T07:36:20.686848+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:05.859762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":64,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677434+0000","last_change":"2024-09-21T07:35:28.860612+0000","last_active":"2024-09-21T07:36:21.677434+0000","last_peered":"2024-09-21T07:36:21.677434+0000","last_clean":"2024-09-21T07:36:21.677434+0000","last_became_active":"2024-09-21T07:35:26.876562+0000","last_became_peered":"2024-09-21T07:35:26.876562+0000","last_unstale":"2024-09-21T07:36:21.677434+0000","last_undegraded":"2024-09-21T07:36:21.677434+0000","last_fullsized":"2024-09-21T07:36:21.677434+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:06.824331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012252470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":115,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.693709+0000","last_change":"2024-09-21T07:34:24.260330+0000","last_active":"2024-09-21T07:36:20.693709+0000","last_peered":"2024-09-21T07:36:20.693709+0000","last_clean":"2024-09-21T07:36:20.693709+0000","last_became_active":"2024-09-21T07:34:23.637210+0000","last_became_peered":"2024-09-21T07:34:23.637210+0000","last_unstale":"2024-09-21T07:36:20.693709+0000","last_undegraded":"2024-09-21T07:36:20.693709+0000","last_fullsized":"2024-09-21T07:36:20.693709+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.488451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001537093,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677282+0000","last_change":"2024-09-21T07:36:20.703443+0000","last_active":"2024-09-21T07:36:21.677282+0000","last_peered":"2024-09-21T07:36:21.677282+0000","last_clean":"2024-09-21T07:36:21.677282+0000","last_became_active":"2024-09-21T07:36:20.703018+0000","last_became_peered":"2024-09-21T07:36:20.703018+0000","last_unstale":"2024-09-21T07:36:21.677282+0000","last_undegraded":"2024-09-21T07:36:21.677282+0000","last_fullsized":"2024-09-21T07:36:21.677282+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:52:32.457726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"89'2","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.724922+0000","last_change":"2024-09-21T07:36:19.704574+0000","last_active":"2024-09-21T07:36:20.724922+0000","last_peered":"2024-09-21T07:36:20.724922+0000","last_clean":"2024-09-21T07:36:20.724922+0000","last_became_active":"2024-09-21T07:36:19.702309+0000","last_became_peered":"2024-09-21T07:36:19.702309+0000","last_unstale":"2024-09-21T07:36:20.724922+0000","last_undegraded":"2024-09-21T07:36:20.724922+0000","last_fullsized":"2024-09-21T07:36:20.724922+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:12.423335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.675518+0000","last_change":"2024-09-21T07:35:27.874757+0000","last_active":"2024-09-21T07:36:21.675518+0000","last_peered":"2024-09-21T07:36:21.675518+0000","last_clean":"2024-09-21T07:36:21.675518+0000","last_became_active":"2024-09-21T07:35:27.874360+0000","last_became_peered":"2024-09-21T07:35:27.874360+0000","last_unstale":"2024-09-21T07:36:21.675518+0000","last_undegraded":"2024-09-21T07:36:21.675518+0000","last_fullsized":"2024-09-21T07:36:21.675518+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:17.646912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":62,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.693892+0000","last_change":"2024-09-21T07:35:29.493431+0000","last_active":"2024-09-21T07:36:20.693892+0000","last_peered":"2024-09-21T07:36:20.693892+0000","last_clean":"2024-09-21T07:36:20.693892+0000","last_became_active":"2024-09-21T07:35:26.873691+0000","last_became_peered":"2024-09-21T07:35:26.873691+0000","last_unstale":"2024-09-21T07:36:20.693892+0000","last_undegraded":"2024-09-21T07:36:20.693892+0000","last_fullsized":"2024-09-21T07:36:20.693892+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:51.705051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001162464,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":110,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.677821+0000","last_change":"2024-09-21T07:34:24.597286+0000","last_active":"2024-09-21T07:36:20.677821+0000","last_peered":"2024-09-21T07:36:20.677821+0000","last_clean":"2024-09-21T07:36:20.677821+0000","last_became_active":"2024-09-21T07:34:23.267133+0000","last_became_peered":"2024-09-21T07:34:23.267133+0000","last_unstale":"2024-09-21T07:36:20.677821+0000","last_undegraded":"2024-09-21T07:36:20.677821+0000","last_fullsized":"2024-09-21T07:36:20.677821+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:18.257010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.706781+0000","last_change":"2024-09-21T07:36:20.697901+0000","last_active":"2024-09-21T07:36:20.706781+0000","last_peered":"2024-09-21T07:36:20.706781+0000","last_clean":"2024-09-21T07:36:20.706781+0000","last_became_active":"2024-09-21T07:36:20.697210+0000","last_became_peered":"2024-09-21T07:36:20.697210+0000","last_unstale":"2024-09-21T07:36:20.706781+0000","last_undegraded":"2024-09-21T07:36:20.706781+0000","last_fullsized":"2024-09-21T07:36:20.706781+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:25.961352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.677762+0000","last_change":"2024-09-21T07:36:19.692941+0000","last_active":"2024-09-21T07:36:20.677762+0000","last_peered":"2024-09-21T07:36:20.677762+0000","last_clean":"2024-09-21T07:36:20.677762+0000","last_became_active":"2024-09-21T07:36:19.692578+0000","last_became_peered":"2024-09-21T07:36:19.692578+0000","last_unstale":"2024-09-21T07:36:20.677762+0000","last_undegraded":"2024-09-21T07:36:20.677762+0000","last_fullsized":"2024-09-21T07:36:20.677762+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:23:37.659200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":59,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676377+0000","last_change":"2024-09-21T07:35:27.864458+0000","last_active":"2024-09-21T07:36:21.676377+0000","last_peered":"2024-09-21T07:36:21.676377+0000","last_clean":"2024-09-21T07:36:21.676377+0000","last_became_active":"2024-09-21T07:35:27.864136+0000","last_became_peered":"2024-09-21T07:35:27.864136+0000","last_unstale":"2024-09-21T07:36:21.676377+0000","last_undegraded":"2024-09-21T07:36:21.676377+0000","last_fullsized":"2024-09-21T07:36:21.676377+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:20.162240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":66,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677453+0000","last_change":"2024-09-21T07:35:28.858784+0000","last_active":"2024-09-21T07:36:21.677453+0000","last_peered":"2024-09-21T07:36:21.677453+0000","last_clean":"2024-09-21T07:36:21.677453+0000","last_became_active":"2024-09-21T07:35:26.876890+0000","last_became_peered":"2024-09-21T07:35:26.876890+0000","last_unstale":"2024-09-21T07:36:21.677453+0000","last_undegraded":"2024-09-21T07:36:21.677453+0000","last_fullsized":"2024-09-21T07:36:21.677453+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:40.738327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064543499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":115,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.678303+0000","last_change":"2024-09-21T07:34:24.258712+0000","last_active":"2024-09-21T07:36:20.678303+0000","last_peered":"2024-09-21T07:36:20.678303+0000","last_clean":"2024-09-21T07:36:20.678303+0000","last_became_active":"2024-09-21T07:34:23.274692+0000","last_became_peered":"2024-09-21T07:34:23.274692+0000","last_unstale":"2024-09-21T07:36:20.678303+0000","last_undegraded":"2024-09-21T07:36:20.678303+0000","last_fullsized":"2024-09-21T07:36:20.678303+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:43.279454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018051479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.678426+0000","last_change":"2024-09-21T07:36:20.702900+0000","last_active":"2024-09-21T07:36:21.678426+0000","last_peered":"2024-09-21T07:36:21.678426+0000","last_clean":"2024-09-21T07:36:21.678426+0000","last_became_active":"2024-09-21T07:36:20.698707+0000","last_became_peered":"2024-09-21T07:36:20.698707+0000","last_unstale":"2024-09-21T07:36:21.678426+0000","last_undegraded":"2024-09-21T07:36:21.678426+0000","last_fullsized":"2024-09-21T07:36:21.678426+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:08.851652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677533+0000","last_change":"2024-09-21T07:36:19.697061+0000","last_active":"2024-09-21T07:36:21.677533+0000","last_peered":"2024-09-21T07:36:21.677533+0000","last_clean":"2024-09-21T07:36:21.677533+0000","last_became_active":"2024-09-21T07:36:19.695586+0000","last_became_peered":"2024-09-21T07:36:19.695586+0000","last_unstale":"2024-09-21T07:36:21.677533+0000","last_undegraded":"2024-09-21T07:36:21.677533+0000","last_fullsized":"2024-09-21T07:36:21.677533+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:41:42.356201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.678140+0000","last_change":"2024-09-21T07:35:27.878020+0000","last_active":"2024-09-21T07:36:20.678140+0000","last_peered":"2024-09-21T07:36:20.678140+0000","last_clean":"2024-09-21T07:36:20.678140+0000","last_became_active":"2024-09-21T07:35:27.877532+0000","last_became_peered":"2024-09-21T07:35:27.877532+0000","last_unstale":"2024-09-21T07:36:20.678140+0000","last_undegraded":"2024-09-21T07:36:20.678140+0000","last_fullsized":"2024-09-21T07:36:20.678140+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:19.756073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":64,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.678324+0000","last_change":"2024-09-21T07:35:28.861049+0000","last_active":"2024-09-21T07:36:21.678324+0000","last_peered":"2024-09-21T07:36:21.678324+0000","last_clean":"2024-09-21T07:36:21.678324+0000","last_became_active":"2024-09-21T07:35:26.876819+0000","last_became_peered":"2024-09-21T07:35:26.876819+0000","last_unstale":"2024-09-21T07:36:21.678324+0000","last_undegraded":"2024-09-21T07:36:21.678324+0000","last_fullsized":"2024-09-21T07:36:21.678324+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:19.633803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055780599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":116,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.675723+0000","last_change":"2024-09-21T07:34:25.084849+0000","last_active":"2024-09-21T07:36:21.675723+0000","last_peered":"2024-09-21T07:36:21.675723+0000","last_clean":"2024-09-21T07:36:21.675723+0000","last_became_active":"2024-09-21T07:34:23.277713+0000","last_became_peered":"2024-09-21T07:34:23.277713+0000","last_unstale":"2024-09-21T07:36:21.675723+0000","last_undegraded":"2024-09-21T07:36:21.675723+0000","last_fullsized":"2024-09-21T07:36:21.675723+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:05.811810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059079499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676688+0000","last_change":"2024-09-21T07:36:20.699395+0000","last_active":"2024-09-21T07:36:21.676688+0000","last_peered":"2024-09-21T07:36:21.676688+0000","last_clean":"2024-09-21T07:36:21.676688+0000","last_became_active":"2024-09-21T07:36:20.698734+0000","last_became_peered":"2024-09-21T07:36:20.698734+0000","last_unstale":"2024-09-21T07:36:21.676688+0000","last_undegraded":"2024-09-21T07:36:21.676688+0000","last_fullsized":"2024-09-21T07:36:21.676688+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:33:19.465270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"89'8","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.718616+0000","last_change":"2024-09-21T07:36:19.698151+0000","last_active":"2024-09-21T07:36:20.718616+0000","last_peered":"2024-09-21T07:36:20.718616+0000","last_clean":"2024-09-21T07:36:20.718616+0000","last_became_active":"2024-09-21T07:36:19.697825+0000","last_became_peered":"2024-09-21T07:36:19.697825+0000","last_unstale":"2024-09-21T07:36:20.718616+0000","last_undegraded":"2024-09-21T07:36:20.718616+0000","last_fullsized":"2024-09-21T07:36:20.718616+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:39.036115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.687672+0000","last_change":"2024-09-21T07:35:27.884248+0000","last_active":"2024-09-21T07:36:20.687672+0000","last_peered":"2024-09-21T07:36:20.687672+0000","last_clean":"2024-09-21T07:36:20.687672+0000","last_became_active":"2024-09-21T07:35:27.883861+0000","last_became_peered":"2024-09-21T07:35:27.883861+0000","last_unstale":"2024-09-21T07:36:20.687672+0000","last_undegraded":"2024-09-21T07:36:20.687672+0000","last_fullsized":"2024-09-21T07:36:20.687672+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.688329+0000","last_change":"2024-09-21T07:35:29.207135+0000","last_active":"2024-09-21T07:36:20.688329+0000","last_peered":"2024-09-21T07:36:20.688329+0000","last_clean":"2024-09-21T07:36:20.688329+0000","last_became_active":"2024-09-21T07:35:26.872787+0000","last_became_peered":"2024-09-21T07:35:26.872787+0000","last_unstale":"2024-09-21T07:36:20.688329+0000","last_undegraded":"2024-09-21T07:36:20.688329+0000","last_fullsized":"2024-09-21T07:36:20.688329+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:27.543217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011157509999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":118,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676633+0000","last_change":"2024-09-21T07:34:24.538610+0000","last_active":"2024-09-21T07:36:21.676633+0000","last_peered":"2024-09-21T07:36:21.676633+0000","last_clean":"2024-09-21T07:36:21.676633+0000","last_became_active":"2024-09-21T07:34:23.259700+0000","last_became_peered":"2024-09-21T07:34:23.259700+0000","last_unstale":"2024-09-21T07:36:21.676633+0000","last_undegraded":"2024-09-21T07:36:21.676633+0000","last_fullsized":"2024-09-21T07:36:21.676633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:57.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070522399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676043+0000","last_change":"2024-09-21T07:36:20.713460+0000","last_active":"2024-09-21T07:36:21.676043+0000","last_peered":"2024-09-21T07:36:21.676043+0000","last_clean":"2024-09-21T07:36:21.676043+0000","last_became_active":"2024-09-21T07:36:20.708796+0000","last_became_peered":"2024-09-21T07:36:20.708796+0000","last_unstale":"2024-09-21T07:36:21.676043+0000","last_undegraded":"2024-09-21T07:36:21.676043+0000","last_fullsized":"2024-09-21T07:36:21.676043+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:19.626858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.677033+0000","last_change":"2024-09-21T07:36:19.697662+0000","last_active":"2024-09-21T07:36:20.677033+0000","last_peered":"2024-09-21T07:36:20.677033+0000","last_clean":"2024-09-21T07:36:20.677033+0000","last_became_active":"2024-09-21T07:36:19.697099+0000","last_became_peered":"2024-09-21T07:36:19.697099+0000","last_unstale":"2024-09-21T07:36:20.677033+0000","last_undegraded":"2024-09-21T07:36:20.677033+0000","last_fullsized":"2024-09-21T07:36:20.677033+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:51.059346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":59,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676832+0000","last_change":"2024-09-21T07:35:27.868024+0000","last_active":"2024-09-21T07:36:21.676832+0000","last_peered":"2024-09-21T07:36:21.676832+0000","last_clean":"2024-09-21T07:36:21.676832+0000","last_became_active":"2024-09-21T07:35:27.867339+0000","last_became_peered":"2024-09-21T07:35:27.867339+0000","last_unstale":"2024-09-21T07:36:21.676832+0000","last_undegraded":"2024-09-21T07:36:21.676832+0000","last_fullsized":"2024-09-21T07:36:21.676832+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":66,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677137+0000","last_change":"2024-09-21T07:35:28.858505+0000","last_active":"2024-09-21T07:36:21.677137+0000","last_peered":"2024-09-21T07:36:21.677137+0000","last_clean":"2024-09-21T07:36:21.677137+0000","last_became_active":"2024-09-21T07:35:26.877320+0000","last_became_peered":"2024-09-21T07:35:26.877320+0000","last_unstale":"2024-09-21T07:36:21.677137+0000","last_undegraded":"2024-09-21T07:36:21.677137+0000","last_fullsized":"2024-09-21T07:36:21.677137+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:53.672782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244119999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":110,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.677254+0000","last_change":"2024-09-21T07:34:24.597597+0000","last_active":"2024-09-21T07:36:20.677254+0000","last_peered":"2024-09-21T07:36:20.677254+0000","last_clean":"2024-09-21T07:36:20.677254+0000","last_became_active":"2024-09-21T07:34:23.635204+0000","last_became_peered":"2024-09-21T07:34:23.635204+0000","last_unstale":"2024-09-21T07:36:20.677254+0000","last_undegraded":"2024-09-21T07:36:20.677254+0000","last_fullsized":"2024-09-21T07:36:20.677254+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:48.644789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081139199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.713529+0000","last_change":"2024-09-21T07:36:20.707168+0000","last_active":"2024-09-21T07:36:20.713529+0000","last_peered":"2024-09-21T07:36:20.713529+0000","last_clean":"2024-09-21T07:36:20.713529+0000","last_became_active":"2024-09-21T07:36:20.704758+0000","last_became_peered":"2024-09-21T07:36:20.704758+0000","last_unstale":"2024-09-21T07:36:20.713529+0000","last_undegraded":"2024-09-21T07:36:20.713529+0000","last_fullsized":"2024-09-21T07:36:20.713529+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:06.231964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.675663+0000","last_change":"2024-09-21T07:36:19.701061+0000","last_active":"2024-09-21T07:36:21.675663+0000","last_peered":"2024-09-21T07:36:21.675663+0000","last_clean":"2024-09-21T07:36:21.675663+0000","last_became_active":"2024-09-21T07:36:19.700180+0000","last_became_peered":"2024-09-21T07:36:19.700180+0000","last_unstale":"2024-09-21T07:36:21.675663+0000","last_undegraded":"2024-09-21T07:36:21.675663+0000","last_fullsized":"2024-09-21T07:36:21.675663+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:22.207628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.687113+0000","last_change":"2024-09-21T07:35:27.877503+0000","last_active":"2024-09-21T07:36:20.687113+0000","last_peered":"2024-09-21T07:36:20.687113+0000","last_clean":"2024-09-21T07:36:20.687113+0000","last_became_active":"2024-09-21T07:35:27.875775+0000","last_became_peered":"2024-09-21T07:35:27.875775+0000","last_unstale":"2024-09-21T07:36:20.687113+0000","last_undegraded":"2024-09-21T07:36:20.687113+0000","last_fullsized":"2024-09-21T07:36:20.687113+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:49.291023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":62,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.690748+0000","last_change":"2024-09-21T07:35:29.491412+0000","last_active":"2024-09-21T07:36:20.690748+0000","last_peered":"2024-09-21T07:36:20.690748+0000","last_clean":"2024-09-21T07:36:20.690748+0000","last_became_active":"2024-09-21T07:35:26.876404+0000","last_became_peered":"2024-09-21T07:35:26.876404+0000","last_unstale":"2024-09-21T07:36:20.690748+0000","last_undegraded":"2024-09-21T07:36:20.690748+0000","last_fullsized":"2024-09-21T07:36:20.690748+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063918000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":115,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.681239+0000","last_change":"2024-09-21T07:34:24.259026+0000","last_active":"2024-09-21T07:36:20.681239+0000","last_peered":"2024-09-21T07:36:20.681239+0000","last_clean":"2024-09-21T07:36:20.681239+0000","last_became_active":"2024-09-21T07:34:23.635456+0000","last_became_peered":"2024-09-21T07:34:23.635456+0000","last_unstale":"2024-09-21T07:36:20.681239+0000","last_undegraded":"2024-09-21T07:36:20.681239+0000","last_fullsized":"2024-09-21T07:36:20.681239+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:09.592752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018654419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677794+0000","last_change":"2024-09-21T07:36:20.717757+0000","last_active":"2024-09-21T07:36:21.677794+0000","last_peered":"2024-09-21T07:36:21.677794+0000","last_clean":"2024-09-21T07:36:21.677794+0000","last_became_active":"2024-09-21T07:36:20.717460+0000","last_became_peered":"2024-09-21T07:36:20.717460+0000","last_unstale":"2024-09-21T07:36:21.677794+0000","last_undegraded":"2024-09-21T07:36:21.677794+0000","last_fullsized":"2024-09-21T07:36:21.677794+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:27.141668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"89'3","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.718832+0000","last_change":"2024-09-21T07:36:19.702916+0000","last_active":"2024-09-21T07:36:20.718832+0000","last_peered":"2024-09-21T07:36:20.718832+0000","last_clean":"2024-09-21T07:36:20.718832+0000","last_became_active":"2024-09-21T07:36:19.702453+0000","last_became_peered":"2024-09-21T07:36:19.702453+0000","last_unstale":"2024-09-21T07:36:20.718832+0000","last_undegraded":"2024-09-21T07:36:20.718832+0000","last_fullsized":"2024-09-21T07:36:20.718832+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:33.305022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.687422+0000","last_change":"2024-09-21T07:35:27.871984+0000","last_active":"2024-09-21T07:36:20.687422+0000","last_peered":"2024-09-21T07:36:20.687422+0000","last_clean":"2024-09-21T07:36:20.687422+0000","last_became_active":"2024-09-21T07:35:27.870336+0000","last_became_peered":"2024-09-21T07:35:27.870336+0000","last_unstale":"2024-09-21T07:36:20.687422+0000","last_undegraded":"2024-09-21T07:36:20.687422+0000","last_fullsized":"2024-09-21T07:36:20.687422+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:57.120479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":64,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.677606+0000","last_change":"2024-09-21T07:35:28.860351+0000","last_active":"2024-09-21T07:36:21.677606+0000","last_peered":"2024-09-21T07:36:21.677606+0000","last_clean":"2024-09-21T07:36:21.677606+0000","last_became_active":"2024-09-21T07:35:26.872046+0000","last_became_peered":"2024-09-21T07:35:26.872046+0000","last_unstale":"2024-09-21T07:36:21.677606+0000","last_undegraded":"2024-09-21T07:36:21.677606+0000","last_fullsized":"2024-09-21T07:36:21.677606+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:47.074327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098384399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":115,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.689179+0000","last_change":"2024-09-21T07:34:24.260300+0000","last_active":"2024-09-21T07:36:20.689179+0000","last_peered":"2024-09-21T07:36:20.689179+0000","last_clean":"2024-09-21T07:36:20.689179+0000","last_became_active":"2024-09-21T07:34:23.270128+0000","last_became_peered":"2024-09-21T07:34:23.270128+0000","last_unstale":"2024-09-21T07:36:20.689179+0000","last_undegraded":"2024-09-21T07:36:20.689179+0000","last_fullsized":"2024-09-21T07:36:20.689179+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:25.260931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00176651,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.713679+0000","last_change":"2024-09-21T07:36:20.708209+0000","last_active":"2024-09-21T07:36:20.713679+0000","last_peered":"2024-09-21T07:36:20.713679+0000","last_clean":"2024-09-21T07:36:20.713679+0000","last_became_active":"2024-09-21T07:36:20.707900+0000","last_became_peered":"2024-09-21T07:36:20.707900+0000","last_unstale":"2024-09-21T07:36:20.713679+0000","last_undegraded":"2024-09-21T07:36:20.713679+0000","last_fullsized":"2024-09-21T07:36:20.713679+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:19.659881+0000","last_clean_scrub_stamp":"2024-09-21T07:36:19.659881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:23.290559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"89'6","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.725086+0000","last_change":"2024-09-21T07:36:19.702326+0000","last_active":"2024-09-21T07:36:20.725086+0000","last_peered":"2024-09-21T07:36:20.725086+0000","last_clean":"2024-09-21T07:36:20.725086+0000","last_became_active":"2024-09-21T07:36:19.701937+0000","last_became_peered":"2024-09-21T07:36:19.701937+0000","last_unstale":"2024-09-21T07:36:20.725086+0000","last_undegraded":"2024-09-21T07:36:20.725086+0000","last_fullsized":"2024-09-21T07:36:20.725086+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:36:18.653521+0000","last_clean_scrub_stamp":"2024-09-21T07:36:18.653521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:51:07.871953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.687047+0000","last_change":"2024-09-21T07:35:27.877861+0000","last_active":"2024-09-21T07:36:20.687047+0000","last_peered":"2024-09-21T07:36:20.687047+0000","last_clean":"2024-09-21T07:36:20.687047+0000","last_became_active":"2024-09-21T07:35:27.877434+0000","last_became_peered":"2024-09-21T07:35:27.877434+0000","last_unstale":"2024-09-21T07:36:20.687047+0000","last_undegraded":"2024-09-21T07:36:20.687047+0000","last_fullsized":"2024-09-21T07:36:20.687047+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":65,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-21T07:36:21.676337+0000","last_change":"2024-09-21T07:35:28.862954+0000","last_active":"2024-09-21T07:36:21.676337+0000","last_peered":"2024-09-21T07:36:21.676337+0000","last_clean":"2024-09-21T07:36:21.676337+0000","last_became_active":"2024-09-21T07:35:26.870613+0000","last_became_peered":"2024-09-21T07:35:26.870613+0000","last_unstale":"2024-09-21T07:36:21.676337+0000","last_undegraded":"2024-09-21T07:36:21.676337+0000","last_fullsized":"2024-09-21T07:36:21.676337+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:06.901043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059842100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":113,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T07:36:20.692274+0000","last_change":"2024-09-21T07:34:25.028904+0000","last_active":"2024-09-21T07:36:20.692274+0000","last_peered":"2024-09-21T07:36:20.692274+0000","last_clean":"2024-09-21T07:36:20.692274+0000","last_became_active":"2024-09-21T07:34:23.260942+0000","last_became_peered":"2024-09-21T07:34:23.260942+0000","last_unstale":"2024-09-21T07:36:20.692274+0000","last_undegraded":"2024-09-21T07:36:20.692274+0000","last_fullsized":"2024-09-21T07:36:20.692274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:22.709328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060553899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509483,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32844,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338996,"statfs":{"total":96636764160,"available":96603131904,"internally_reserved":0,"allocated":1687552,"data_stored":1099526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509485,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32272,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339568,"statfs":{"total":96636764160,"available":96603717632,"internally_reserved":0,"allocated":1105920,"data_stored":511214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509483,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1097728,"data_stored":516389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509484,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32264,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339576,"statfs":{"total":96636764160,"available":96603725824,"internally_reserved":0,"allocated":1101824,"data_stored":516423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509485,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1089536,"data_stored":509032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509485,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1708032,"data_stored":1108459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542188,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32264,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339576,"statfs":{"total":96636764160,"available":96603725824,"internally_reserved":0,"allocated":1093632,"data_stored":509090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542187,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32260,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339580,"statfs":{"total":96636764160,"available":96603729920,"internally_reserved":0,"allocated":1089536,"data_stored":509580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:36:25.706 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T07:36:25.706 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T07:36:25.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.971+0000 7fbeebe00640 1 -- 172.21.15.39:0/3650511805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeec0c9bf0 msgr2=0x7fbeec0cf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:25.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.971+0000 7fbeebe00640 1 --2- 172.21.15.39:0/3650511805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeec0c9bf0 0x7fbeec0cf6e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fbeec1621b0 tx=0x7fbedc02a560 comp rx=0 tx=0).stop 2024-09-21T07:36:25.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.971+0000 7fbeebe00640 1 -- 172.21.15.39:0/3650511805 shutdown_connections 2024-09-21T07:36:25.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.971+0000 7fbeebe00640 1 --2- 172.21.15.39:0/3650511805 >> v2:172.21.15.148:3300/0 conn(0x7fbeec0cff20 0x7fbeec0d0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:25.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.971+0000 7fbeebe00640 1 --2- 172.21.15.39:0/3650511805 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbeec0cb790 0x7fbeec15ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:25.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.971+0000 7fbeebe00640 1 --2- 172.21.15.39:0/3650511805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeec0c9bf0 0x7fbeec0cf6e0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:25.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.971+0000 7fbeebe00640 1 -- 172.21.15.39:0/3650511805 >> 172.21.15.39:0/3650511805 conn(0x7fbeec06ee50 msgr2=0x7fbeec06f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:25.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.971+0000 7fbeebe00640 1 -- 172.21.15.39:0/3650511805 shutdown_connections 2024-09-21T07:36:25.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.971+0000 7fbeebe00640 1 -- 172.21.15.39:0/3650511805 wait complete. 2024-09-21T07:36:25.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.973+0000 7fbeebe00640 1 Processor -- start 2024-09-21T07:36:25.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.974+0000 7fbeebe00640 1 -- start start 2024-09-21T07:36:25.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.975+0000 7fbeebe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbeec0c9bf0 0x7fbeec146130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:25.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.975+0000 7fbeebe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeec0cb790 0x7fbeec146670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:25.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.975+0000 7fbeebe00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbeec0cff20 0x7fbeec0b5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:25.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.975+0000 7fbeebe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec0ce720 con 0x7fbeec0cb790 2024-09-21T07:36:25.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.975+0000 7fbeebe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec0ce5a0 con 0x7fbeec0c9bf0 2024-09-21T07:36:25.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.975+0000 7fbeebe00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fbeec0ce8a0 con 0x7fbeec0cff20 2024-09-21T07:36:25.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.975+0000 7fbee2000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeec0cb790 0x7fbeec146670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:25.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.975+0000 7fbee2000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeec0cb790 0x7fbeec146670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52240/0 (socket says 172.21.15.39:52240) 2024-09-21T07:36:25.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.975+0000 7fbee2000640 1 -- 172.21.15.39:0/557739036 learned_addr learned my addr 172.21.15.39:0/557739036 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:25.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.975+0000 7fbeeaa00640 1 --2- 172.21.15.39:0/557739036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbeec0c9bf0 0x7fbeec146130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:25.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.975+0000 7fbeeb400640 1 --2- 172.21.15.39:0/557739036 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbeec0cff20 0x7fbeec0b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:25.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.976+0000 7fbee2000640 1 -- 172.21.15.39:0/557739036 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbeec0cff20 msgr2=0x7fbeec0b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:25.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.976+0000 7fbee2000640 1 --2- 172.21.15.39:0/557739036 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbeec0cff20 0x7fbeec0b5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:25.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.976+0000 7fbee2000640 1 -- 172.21.15.39:0/557739036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbeec0c9bf0 msgr2=0x7fbeec146130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:25.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.976+0000 7fbee2000640 1 --2- 172.21.15.39:0/557739036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbeec0c9bf0 0x7fbeec146130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:25.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.976+0000 7fbee2000640 1 -- 172.21.15.39:0/557739036 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeec14cca0 con 0x7fbeec0cb790 2024-09-21T07:36:25.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.976+0000 7fbee2000640 1 --2- 172.21.15.39:0/557739036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeec0cb790 0x7fbeec146670 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fbed400b410 tx=0x7fbed400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:25.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.976+0000 7fbee3400640 1 -- 172.21.15.39:0/557739036 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed4015020 con 0x7fbeec0cb790 2024-09-21T07:36:25.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.976+0000 7fbee3400640 1 -- 172.21.15.39:0/557739036 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbed4011940 con 0x7fbeec0cb790 2024-09-21T07:36:25.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.976+0000 7fbee3400640 1 -- 172.21.15.39:0/557739036 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed4011d90 con 0x7fbeec0cb790 2024-09-21T07:36:25.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.976+0000 7fbeebe00640 1 -- 172.21.15.39:0/557739036 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeec159890 con 0x7fbeec0cb790 2024-09-21T07:36:25.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.977+0000 7fbeebe00640 1 -- 172.21.15.39:0/557739036 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeec15bce0 con 0x7fbeec0cb790 2024-09-21T07:36:25.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.977+0000 7fbeebe00640 1 -- 172.21.15.39:0/557739036 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeec0cedd0 con 0x7fbeec0cb790 2024-09-21T07:36:25.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.979+0000 7fbee3400640 1 -- 172.21.15.39:0/557739036 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbed4018020 con 0x7fbeec0cb790 2024-09-21T07:36:25.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.980+0000 7fbee3400640 1 --2- 172.21.15.39:0/557739036 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbec8078b90 0x7fbec807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:25.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.980+0000 7fbee3400640 1 -- 172.21.15.39:0/557739036 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6800+0+0 (secure 0 0 0) 0x7fbed409aff0 con 0x7fbeec0cb790 2024-09-21T07:36:25.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.981+0000 7fbeeaa00640 1 --2- 172.21.15.39:0/557739036 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbec8078b90 0x7fbec807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:25.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.982+0000 7fbeeaa00640 1 --2- 172.21.15.39:0/557739036 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbec8078b90 0x7fbec807b0a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbedc033000 tx=0x7fbedc004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:25.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:25.985+0000 7fbee3400640 1 -- 172.21.15.39:0/557739036 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbed401f020 con 0x7fbeec0cb790 2024-09-21T07:36:26.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.252+0000 7fbeebe00640 1 -- 172.21.15.39:0/557739036 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fbeec15c630 con 0x7fbeec0cb790 2024-09-21T07:36:26.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.256+0000 7fbee3400640 1 -- 172.21.15.39:0/557739036 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7fbed4062920 con 0x7fbeec0cb790 2024-09-21T07:36:26.259 INFO:teuthology.orchestra.run.smithi039.stdout:[client.0] 2024-09-21T07:36:26.259 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQB0du5mGfb1DBAA/aSMKR5or3xy53rB/iUjpA== 2024-09-21T07:36:26.259 INFO:teuthology.orchestra.run.smithi039.stdout: caps mds = "allow" 2024-09-21T07:36:26.259 INFO:teuthology.orchestra.run.smithi039.stdout: caps mon = "allow r" 2024-09-21T07:36:26.259 INFO:teuthology.orchestra.run.smithi039.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T07:36:26.259 INFO:teuthology.orchestra.run.smithi039.stderr:updated caps for client.0 2024-09-21T07:36:26.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.259+0000 7fbeebe00640 1 -- 172.21.15.39:0/557739036 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbec8078b90 msgr2=0x7fbec807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:26.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.259+0000 7fbeebe00640 1 --2- 172.21.15.39:0/557739036 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbec8078b90 0x7fbec807b0a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbedc033000 tx=0x7fbedc004730 comp rx=0 tx=0).stop 2024-09-21T07:36:26.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.259+0000 7fbeebe00640 1 -- 172.21.15.39:0/557739036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeec0cb790 msgr2=0x7fbeec146670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:26.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.259+0000 7fbeebe00640 1 --2- 172.21.15.39:0/557739036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeec0cb790 0x7fbeec146670 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fbed400b410 tx=0x7fbed400b8d0 comp rx=0 tx=0).stop 2024-09-21T07:36:26.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.259+0000 7fbeebe00640 1 -- 172.21.15.39:0/557739036 shutdown_connections 2024-09-21T07:36:26.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.259+0000 7fbeebe00640 1 --2- 172.21.15.39:0/557739036 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbec8078b90 0x7fbec807b0a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:26.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.259+0000 7fbeebe00640 1 --2- 172.21.15.39:0/557739036 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbeec0cff20 0x7fbeec0b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:26.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.259+0000 7fbeebe00640 1 --2- 172.21.15.39:0/557739036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeec0cb790 0x7fbeec146670 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:26.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.259+0000 7fbeebe00640 1 --2- 172.21.15.39:0/557739036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbeec0c9bf0 0x7fbeec146130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:26.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.259+0000 7fbeebe00640 1 -- 172.21.15.39:0/557739036 >> 172.21.15.39:0/557739036 conn(0x7fbeec06ee50 msgr2=0x7fbeec15fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:26.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.259+0000 7fbeebe00640 1 -- 172.21.15.39:0/557739036 shutdown_connections 2024-09-21T07:36:26.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.259+0000 7fbeebe00640 1 -- 172.21.15.39:0/557739036 wait complete. 2024-09-21T07:36:26.283 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:36:26.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.507+0000 7f2cae400640 1 -- 172.21.15.39:0/2259006047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ca815fbd0 msgr2=0x7f2ca8152360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:26.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.507+0000 7f2cae400640 1 --2- 172.21.15.39:0/2259006047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ca815fbd0 0x7f2ca8152360 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c009830 tx=0x7f2c9c02a480 comp rx=0 tx=0).stop 2024-09-21T07:36:26.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.507+0000 7f2cae400640 1 -- 172.21.15.39:0/2259006047 shutdown_connections 2024-09-21T07:36:26.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.507+0000 7f2cae400640 1 --2- 172.21.15.39:0/2259006047 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ca815f660 0x7f2ca80b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:26.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.507+0000 7f2cae400640 1 --2- 172.21.15.39:0/2259006047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ca815fbd0 0x7f2ca8152360 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:26.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.507+0000 7f2cae400640 1 -- 172.21.15.39:0/2259006047 >> 172.21.15.39:0/2259006047 conn(0x7f2ca806d410 msgr2=0x7f2ca80b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:26.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.507+0000 7f2cae400640 1 -- 172.21.15.39:0/2259006047 shutdown_connections 2024-09-21T07:36:26.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.507+0000 7f2cae400640 1 -- 172.21.15.39:0/2259006047 wait complete. 2024-09-21T07:36:26.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.509+0000 7f2cae400640 1 Processor -- start 2024-09-21T07:36:26.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.509+0000 7f2cae400640 1 -- start start 2024-09-21T07:36:26.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.511+0000 7f2cae400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ca8072c30 0x7f2ca8156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:26.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.511+0000 7f2cae400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ca81f3c00 0x7f2ca80c5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:26.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.511+0000 7f2cae400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ca81467f0 0x7f2ca8157490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:26.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.511+0000 7f2cae400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca80b3fe0 con 0x7f2ca81467f0 2024-09-21T07:36:26.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.511+0000 7f2cae400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca80b3e60 con 0x7f2ca81f3c00 2024-09-21T07:36:26.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.511+0000 7f2cae400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f2ca80b4160 con 0x7f2ca8072c30 2024-09-21T07:36:26.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.511+0000 7f2cada00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ca81467f0 0x7f2ca8157490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:26.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.511+0000 7f2cada00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ca81467f0 0x7f2ca8157490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52264/0 (socket says 172.21.15.39:52264) 2024-09-21T07:36:26.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.511+0000 7f2cada00640 1 -- 172.21.15.39:0/3114563058 learned_addr learned my addr 172.21.15.39:0/3114563058 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:26.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.511+0000 7f2cad000640 1 --2- 172.21.15.39:0/3114563058 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ca8072c30 0x7f2ca8156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:26.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.511+0000 7f2ca7e00640 1 --2- 172.21.15.39:0/3114563058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ca81f3c00 0x7f2ca80c5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:26.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.512+0000 7f2cada00640 1 -- 172.21.15.39:0/3114563058 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ca8072c30 msgr2=0x7f2ca8156ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:26.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.512+0000 7f2cada00640 1 --2- 172.21.15.39:0/3114563058 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ca8072c30 0x7f2ca8156ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:26.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.512+0000 7f2cada00640 1 -- 172.21.15.39:0/3114563058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ca81f3c00 msgr2=0x7f2ca80c5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:26.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.512+0000 7f2cada00640 1 --2- 172.21.15.39:0/3114563058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ca81f3c00 0x7f2ca80c5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:26.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.512+0000 7f2cada00640 1 -- 172.21.15.39:0/3114563058 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca80c6100 con 0x7f2ca81467f0 2024-09-21T07:36:26.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.512+0000 7f2cada00640 1 --2- 172.21.15.39:0/3114563058 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ca81467f0 0x7f2ca8157490 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f2ca000cb00 tx=0x7f2ca000a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:26.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.512+0000 7f2ca4c00640 1 -- 172.21.15.39:0/3114563058 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca000f450 con 0x7f2ca81467f0 2024-09-21T07:36:26.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.512+0000 7f2ca4c00640 1 -- 172.21.15.39:0/3114563058 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ca000f700 con 0x7f2ca81467f0 2024-09-21T07:36:26.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.512+0000 7f2ca4c00640 1 -- 172.21.15.39:0/3114563058 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca000fb50 con 0x7f2ca81467f0 2024-09-21T07:36:26.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.512+0000 7f2cae400640 1 -- 172.21.15.39:0/3114563058 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ca80c5e20 con 0x7f2ca81467f0 2024-09-21T07:36:26.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.512+0000 7f2cae400640 1 -- 172.21.15.39:0/3114563058 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ca81564a0 con 0x7f2ca81467f0 2024-09-21T07:36:26.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.513+0000 7f2cae400640 1 -- 172.21.15.39:0/3114563058 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ca80b4680 con 0x7f2ca81467f0 2024-09-21T07:36:26.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.515+0000 7f2ca4c00640 1 -- 172.21.15.39:0/3114563058 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2ca000fcf0 con 0x7f2ca81467f0 2024-09-21T07:36:26.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.516+0000 7f2ca4c00640 1 --2- 172.21.15.39:0/3114563058 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2c88078b90 0x7f2c8807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:26.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.516+0000 7f2ca4c00640 1 -- 172.21.15.39:0/3114563058 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6800+0+0 (secure 0 0 0) 0x7f2ca009bba0 con 0x7f2ca81467f0 2024-09-21T07:36:26.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.516+0000 7f2cad000640 1 --2- 172.21.15.39:0/3114563058 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2c88078b90 0x7f2c8807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:26.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.517+0000 7f2cad000640 1 --2- 172.21.15.39:0/3114563058 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2c88078b90 0x7f2c8807b0a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c034000 tx=0x7f2c9c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:26.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.521+0000 7f2ca4c00640 1 -- 172.21.15.39:0/3114563058 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2ca80c6100 con 0x7f2ca81467f0 2024-09-21T07:36:26.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.756+0000 7f2cae400640 1 -- 172.21.15.39:0/3114563058 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ca8155de0 con 0x7f2ca81467f0 2024-09-21T07:36:26.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.757+0000 7f2ca4c00640 1 -- 172.21.15.39:0/3114563058 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5225 (secure 0 0 0) 0x7f2ca0014030 con 0x7f2ca81467f0 2024-09-21T07:36:26.759 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:36:26.759 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":54,"btime":"2024-09-21T07:36:23:704206+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5621,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3026827614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3026827614},{"type":"v1","addr":"172.21.15.148:6837","nonce":3026827614}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6115,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:36:23.220980+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6321},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6321":{"gid":6321,"name":"c","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/829716143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":829716143},{"type":"v1","addr":"172.21.15.39:6834","nonce":829716143}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6321,"qdb_cluster":[6321]},"id":6}]} 2024-09-21T07:36:26.759 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 54 2024-09-21T07:36:26.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.759+0000 7f2cae400640 1 -- 172.21.15.39:0/3114563058 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2c88078b90 msgr2=0x7f2c8807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:26.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.759+0000 7f2cae400640 1 --2- 172.21.15.39:0/3114563058 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2c88078b90 0x7f2c8807b0a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c034000 tx=0x7f2c9c004730 comp rx=0 tx=0).stop 2024-09-21T07:36:26.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.759+0000 7f2cae400640 1 -- 172.21.15.39:0/3114563058 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ca81467f0 msgr2=0x7f2ca8157490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:26.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.759+0000 7f2cae400640 1 --2- 172.21.15.39:0/3114563058 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ca81467f0 0x7f2ca8157490 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f2ca000cb00 tx=0x7f2ca000a250 comp rx=0 tx=0).stop 2024-09-21T07:36:26.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.759+0000 7f2cae400640 1 -- 172.21.15.39:0/3114563058 shutdown_connections 2024-09-21T07:36:26.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.759+0000 7f2cae400640 1 --2- 172.21.15.39:0/3114563058 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2c88078b90 0x7f2c8807b0a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:26.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.759+0000 7f2cae400640 1 --2- 172.21.15.39:0/3114563058 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ca81467f0 0x7f2ca8157490 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:26.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.759+0000 7f2cae400640 1 --2- 172.21.15.39:0/3114563058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ca81f3c00 0x7f2ca80c5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:26.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.759+0000 7f2cae400640 1 --2- 172.21.15.39:0/3114563058 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ca8072c30 0x7f2ca8156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:26.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.759+0000 7f2cae400640 1 -- 172.21.15.39:0/3114563058 >> 172.21.15.39:0/3114563058 conn(0x7f2ca806d410 msgr2=0x7f2ca806ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:26.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.759+0000 7f2cae400640 1 -- 172.21.15.39:0/3114563058 shutdown_connections 2024-09-21T07:36:26.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:26.759+0000 7f2cae400640 1 -- 172.21.15.39:0/3114563058 wait complete. 2024-09-21T07:36:27.188 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 54, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T07:36:20.189287+0000', 'modified': '2024-09-21T07:36:23.220980+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6321}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6321': {'gid': 6321, 'name': 'c', 'rank': 0, 'incarnation': 51, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.39:6834/829716143', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 829716143}, {'type': 'v1', 'addr': '172.21.15.39:6834', 'nonce': 829716143}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6321, 'qdb_cluster': [6321]} 2024-09-21T07:36:27.188 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T07:36:27.188 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T07:36:27.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.414+0000 7f2157200640 1 -- 172.21.15.39:0/3328153855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21500cf540 msgr2=0x7f215006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:27.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.414+0000 7f2157200640 1 --2- 172.21.15.39:0/3328153855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21500cf540 0x7f215006ee70 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f2150070dc0 tx=0x7f214002a4b0 comp rx=0 tx=0).stop 2024-09-21T07:36:27.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.415+0000 7f2157200640 1 -- 172.21.15.39:0/3328153855 shutdown_connections 2024-09-21T07:36:27.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.415+0000 7f2157200640 1 --2- 172.21.15.39:0/3328153855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21500cf540 0x7f215006ee70 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:27.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.415+0000 7f2157200640 1 --2- 172.21.15.39:0/3328153855 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f215015ca50 0x7f21500cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:27.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.415+0000 7f2157200640 1 -- 172.21.15.39:0/3328153855 >> 172.21.15.39:0/3328153855 conn(0x7f215006d410 msgr2=0x7f21500b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:27.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.415+0000 7f2157200640 1 -- 172.21.15.39:0/3328153855 shutdown_connections 2024-09-21T07:36:27.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.415+0000 7f2157200640 1 -- 172.21.15.39:0/3328153855 wait complete. 2024-09-21T07:36:27.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.417+0000 7f2157200640 1 Processor -- start 2024-09-21T07:36:27.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.417+0000 7f2157200640 1 -- start start 2024-09-21T07:36:27.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.418+0000 7f2157200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21500cf540 0x7f21500c60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:27.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.418+0000 7f2157200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21500cf950 0x7f2150157060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:27.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.418+0000 7f2157200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f215015ca50 0x7f2150158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:27.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.418+0000 7f2157200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f215015de10 con 0x7f21500cf540 2024-09-21T07:36:27.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.418+0000 7f2157200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f215015df90 con 0x7f21500cf950 2024-09-21T07:36:27.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.418+0000 7f2157200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f215015e110 con 0x7f215015ca50 2024-09-21T07:36:27.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.419+0000 7f2155e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21500cf540 0x7f21500c60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:27.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.419+0000 7f2155e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21500cf540 0x7f21500c60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52292/0 (socket says 172.21.15.39:52292) 2024-09-21T07:36:27.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.419+0000 7f2155e00640 1 -- 172.21.15.39:0/881984971 learned_addr learned my addr 172.21.15.39:0/881984971 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:27.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.419+0000 7f2156800640 1 --2- 172.21.15.39:0/881984971 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f215015ca50 0x7f2150158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:27.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.419+0000 7f2155400640 1 --2- 172.21.15.39:0/881984971 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21500cf950 0x7f2150157060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:27.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.419+0000 7f2155e00640 1 -- 172.21.15.39:0/881984971 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f215015ca50 msgr2=0x7f2150158ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:27.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.419+0000 7f2155e00640 1 --2- 172.21.15.39:0/881984971 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f215015ca50 0x7f2150158ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:27.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.419+0000 7f2155e00640 1 -- 172.21.15.39:0/881984971 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21500cf950 msgr2=0x7f2150157060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:27.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.419+0000 7f2155e00640 1 --2- 172.21.15.39:0/881984971 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21500cf950 0x7f2150157060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:27.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.419+0000 7f2155e00640 1 -- 172.21.15.39:0/881984971 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2150157960 con 0x7f21500cf540 2024-09-21T07:36:27.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.419+0000 7f2155400640 1 --2- 172.21.15.39:0/881984971 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21500cf950 0x7f2150157060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:36:27.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.419+0000 7f2156800640 1 --2- 172.21.15.39:0/881984971 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f215015ca50 0x7f2150158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:36:27.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.420+0000 7f2155e00640 1 --2- 172.21.15.39:0/881984971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21500cf540 0x7f21500c60f0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f214400b410 tx=0x7f214400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:27.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.420+0000 7f214a000640 1 -- 172.21.15.39:0/881984971 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2144015020 con 0x7f21500cf540 2024-09-21T07:36:27.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.420+0000 7f214a000640 1 -- 172.21.15.39:0/881984971 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2144011940 con 0x7f21500cf540 2024-09-21T07:36:27.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.420+0000 7f2157200640 1 -- 172.21.15.39:0/881984971 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2150157c40 con 0x7f21500cf540 2024-09-21T07:36:27.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.420+0000 7f214a000640 1 -- 172.21.15.39:0/881984971 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2144011d90 con 0x7f21500cf540 2024-09-21T07:36:27.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.420+0000 7f2157200640 1 -- 172.21.15.39:0/881984971 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2150157f20 con 0x7f21500cf540 2024-09-21T07:36:27.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.423+0000 7f214a000640 1 -- 172.21.15.39:0/881984971 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2144018020 con 0x7f21500cf540 2024-09-21T07:36:27.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.424+0000 7f214a000640 1 --2- 172.21.15.39:0/881984971 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2134078b90 0x7f213407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:27.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.425+0000 7f214a000640 1 -- 172.21.15.39:0/881984971 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6800+0+0 (secure 0 0 0) 0x7f214409b130 con 0x7f21500cf540 2024-09-21T07:36:27.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.425+0000 7f215f9cb740 1 Processor -- start 2024-09-21T07:36:27.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.425+0000 7f215f9cb740 1 -- start start 2024-09-21T07:36:27.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.425+0000 7f2155400640 1 --2- 172.21.15.39:0/881984971 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2134078b90 0x7f213407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:27.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.426+0000 7f2155400640 1 --2- 172.21.15.39:0/881984971 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2134078b90 0x7f213407b0a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2140033000 tx=0x7f2140004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:27.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.426+0000 7f215f9cb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56291d786270 0x56291db56b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:27.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.426+0000 7f215f9cb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56291d75d3b0 0x56291db570a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:27.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.426+0000 7f215f9cb740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56291d765590 0x56291db27150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:27.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.426+0000 7f215f9cb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56291dab0430 con 0x56291d75d3b0 2024-09-21T07:36:27.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.426+0000 7f215f9cb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56291daa05d0 con 0x56291d786270 2024-09-21T07:36:27.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.426+0000 7f215f9cb740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x56291da8fa00 con 0x56291d765590 2024-09-21T07:36:27.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.426+0000 7f2155400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56291d75d3b0 0x56291db570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:27.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.426+0000 7f2155400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56291d75d3b0 0x56291db570a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52302/0 (socket says 172.21.15.39:52302) 2024-09-21T07:36:27.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.426+0000 7f2155400640 1 -- 172.21.15.39:0/2227092699 learned_addr learned my addr 172.21.15.39:0/2227092699 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:27.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.427+0000 7f2155e00640 1 --2- 172.21.15.39:0/2227092699 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56291d786270 0x56291db56b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:27.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.427+0000 7f2156800640 1 --2- 172.21.15.39:0/2227092699 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56291d765590 0x56291db27150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:27.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.427+0000 7f2155400640 1 -- 172.21.15.39:0/2227092699 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56291d765590 msgr2=0x56291db27150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:36:27.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.427+0000 7f2155400640 1 --2- 172.21.15.39:0/2227092699 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56291d765590 0x56291db27150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:27.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.427+0000 7f2155400640 1 -- 172.21.15.39:0/2227092699 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56291d786270 msgr2=0x56291db56b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:36:27.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.427+0000 7f2155400640 1 --2- 172.21.15.39:0/2227092699 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56291d786270 0x56291db56b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:27.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.427+0000 7f2155400640 1 -- 172.21.15.39:0/2227092699 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56291da58520 con 0x56291d75d3b0 2024-09-21T07:36:27.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.427+0000 7f2155400640 1 --2- 172.21.15.39:0/2227092699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56291d75d3b0 0x56291db570a0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f214002aaa0 tx=0x7f214003b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3873ba4f8a06838 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:27.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.428+0000 7f213e600640 1 -- 172.21.15.39:0/2227092699 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2140035050 con 0x56291d75d3b0 2024-09-21T07:36:27.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.428+0000 7f213e600640 1 -- 172.21.15.39:0/2227092699 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f214004c070 con 0x56291d75d3b0 2024-09-21T07:36:27.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.428+0000 7f213e600640 1 -- 172.21.15.39:0/2227092699 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2140003da0 con 0x56291d75d3b0 2024-09-21T07:36:27.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.428+0000 7f215f9cb740 1 -- 172.21.15.39:0/2227092699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56291d75d3b0 msgr2=0x56291db570a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:27.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.428+0000 7f215f9cb740 1 --2- 172.21.15.39:0/2227092699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56291d75d3b0 0x56291db570a0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f214002aaa0 tx=0x7f214003b000 comp rx=0 tx=0).stop 2024-09-21T07:36:27.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.428+0000 7f215f9cb740 1 -- 172.21.15.39:0/2227092699 shutdown_connections 2024-09-21T07:36:27.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.428+0000 7f215f9cb740 1 --2- 172.21.15.39:0/2227092699 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56291d765590 0x56291db27150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:27.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.428+0000 7f215f9cb740 1 --2- 172.21.15.39:0/2227092699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56291d75d3b0 0x56291db570a0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:27.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.428+0000 7f215f9cb740 1 --2- 172.21.15.39:0/2227092699 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56291d786270 0x56291db56b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:27.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.428+0000 7f215f9cb740 1 -- 172.21.15.39:0/2227092699 >> 172.21.15.39:0/2227092699 conn(0x56291d77a3c0 msgr2=0x56291db2cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:27.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.429+0000 7f215f9cb740 1 -- 172.21.15.39:0/2227092699 shutdown_connections 2024-09-21T07:36:27.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.429+0000 7f215f9cb740 1 -- 172.21.15.39:0/2227092699 wait complete. 2024-09-21T07:36:27.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.430+0000 7f215f9cb740 1 -- start start 2024-09-21T07:36:27.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.430+0000 7f215f9cb740 1 Processor -- start 2024-09-21T07:36:27.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.431+0000 7f215f9cb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56291d75d3b0 0x56291db507a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:27.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.431+0000 7f215f9cb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56291d765590 0x56291db80980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:27.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.431+0000 7f215f9cb740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56291d786270 0x56291db6c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:27.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.431+0000 7f215f9cb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56291da36330 con 0x56291d765590 2024-09-21T07:36:27.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.431+0000 7f215f9cb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56291da93810 con 0x56291d75d3b0 2024-09-21T07:36:27.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.431+0000 7f215f9cb740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x56291d800620 con 0x56291d786270 2024-09-21T07:36:27.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.432+0000 7f215f9cb740 10 client.0 _ll_register_callbacks cb 0x56291da3b420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:36:27.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.432+0000 7f2155400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56291d765590 0x56291db80980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:27.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.432+0000 7f2155400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56291d765590 0x56291db80980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52306/0 (socket says 172.21.15.39:52306) 2024-09-21T07:36:27.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.432+0000 7f2155400640 1 -- 172.21.15.39:0/2259198587 learned_addr learned my addr 172.21.15.39:0/2259198587 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:27.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.432+0000 7f2155e00640 1 --2- 172.21.15.39:0/2259198587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56291d75d3b0 0x56291db507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:27.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.432+0000 7f2156800640 1 --2- 172.21.15.39:0/2259198587 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56291d786270 0x56291db6c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:27.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.432+0000 7f2155400640 1 -- 172.21.15.39:0/2259198587 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56291d786270 msgr2=0x56291db6c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:36:27.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.432+0000 7f2155400640 1 --2- 172.21.15.39:0/2259198587 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56291d786270 0x56291db6c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:27.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.432+0000 7f2155400640 1 -- 172.21.15.39:0/2259198587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56291d75d3b0 msgr2=0x56291db507a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:36:27.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.432+0000 7f2155400640 1 --2- 172.21.15.39:0/2259198587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56291d75d3b0 0x56291db507a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:27.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.432+0000 7f2155400640 1 -- 172.21.15.39:0/2259198587 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56291d7f1850 con 0x56291d765590 2024-09-21T07:36:27.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.432+0000 7f2155400640 1 --2- 172.21.15.39:0/2259198587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56291d765590 0x56291db80980 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f2140049040 tx=0x7f2140003df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c240af70407c5e1e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:27.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.432+0000 7f2132a00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:36:27.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.433+0000 7f2132a00640 1 -- 172.21.15.39:0/2259198587 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2140035040 con 0x56291d765590 2024-09-21T07:36:27.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.433+0000 7f2132a00640 1 -- 172.21.15.39:0/2259198587 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 6800+0+0 (secure 0 0 0) 0x7f21400049b0 con 0x56291d765590 2024-09-21T07:36:27.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.433+0000 7f215f9cb740 1 -- 172.21.15.39:0/2259198587 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f214004c070 con 0x56291d765590 2024-09-21T07:36:27.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.433+0000 7f2132a00640 1 -- 172.21.15.39:0/2259198587 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f2140061050 con 0x56291d765590 2024-09-21T07:36:27.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.433+0000 7f215f9cb740 10 client.6426 fetch_fsmap learned FSMap version 54 2024-09-21T07:36:27.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.433+0000 7f215f9cb740 1 -- 172.21.15.39:0/2259198587 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x56291d8686b0 con 0x56291d765590 2024-09-21T07:36:27.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.433+0000 7f2132a00640 1 -- 172.21.15.39:0/2259198587 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f214004c070 con 0x56291d765590 2024-09-21T07:36:27.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.433+0000 7f2132a00640 1 -- 172.21.15.39:0/2259198587 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 54) ==== 3502+0+0 (secure 0 0 0) 0x7f2140064020 con 0x56291d765590 2024-09-21T07:36:27.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.433+0000 7f215f9cb740 10 client.6426 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T07:36:27.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.433+0000 7f215f9cb740 10 client.6426 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:36:27.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.433+0000 7f215f9cb740 20 client.6426 populate_metadata read hostname 'smithi039' 2024-09-21T07:36:27.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.433+0000 7f215f9cb740 1 --2- 172.21.15.39:0/2259198587 >> [v2:172.21.15.39:6832/829716143,v1:172.21.15.39:6834/829716143] conn(0x56291d744e40 0x56291db83fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:27.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.433+0000 7f215f9cb740 4 client.6426 mds_command: new command op to 6321 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:36:27.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.433+0000 7f215f9cb740 1 -- 172.21.15.39:0/2259198587 --> [v2:172.21.15.39:6832/829716143,v1:172.21.15.39:6834/829716143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56291daa2520 con 0x56291d744e40 2024-09-21T07:36:27.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.434+0000 7f2155e00640 1 --2- 172.21.15.39:0/2259198587 >> [v2:172.21.15.39:6832/829716143,v1:172.21.15.39:6834/829716143] conn(0x56291d744e40 0x56291db83fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:27.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.434+0000 7f2155e00640 1 --2- 172.21.15.39:0/2259198587 >> [v2:172.21.15.39:6832/829716143,v1:172.21.15.39:6834/829716143] conn(0x56291d744e40 0x56291db83fc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb5b710f734d70f2 server_cookie=510a636cbde2de3c in_seq=0 out_seq=0 2024-09-21T07:36:27.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.434+0000 7f2132a00640 10 client.6426 ms_handle_connect on v2:172.21.15.39:6832/829716143 2024-09-21T07:36:27.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.435+0000 7f2132a00640 1 -- 172.21.15.39:0/2259198587 <== mds.0 v2:172.21.15.39:6832/829716143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56291daa2520 con 0x56291d744e40 2024-09-21T07:36:27.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.435+0000 7f2132a00640 10 client.6426 handle_command_reply: tid=0 2024-09-21T07:36:27.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.435+0000 7f215f9cb740 1 client.6426 shutdown 2024-09-21T07:36:27.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.435+0000 7f215f9cb740 1 -- 172.21.15.39:0/2259198587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56291d765590 msgr2=0x56291db80980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:27.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.435+0000 7f215f9cb740 1 --2- 172.21.15.39:0/2259198587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56291d765590 0x56291db80980 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f2140049040 tx=0x7f2140003df0 comp rx=0 tx=0).stop 2024-09-21T07:36:27.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.435+0000 7f215f9cb740 1 -- 172.21.15.39:0/2259198587 shutdown_connections 2024-09-21T07:36:27.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.435+0000 7f215f9cb740 1 --2- 172.21.15.39:0/2259198587 >> [v2:172.21.15.39:6832/829716143,v1:172.21.15.39:6834/829716143] conn(0x56291d744e40 0x56291db83fc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:27.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.435+0000 7f215f9cb740 1 --2- 172.21.15.39:0/2259198587 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56291d786270 0x56291db6c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:27.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.435+0000 7f215f9cb740 1 --2- 172.21.15.39:0/2259198587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56291d765590 0x56291db80980 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:27.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.435+0000 7f215f9cb740 1 --2- 172.21.15.39:0/2259198587 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56291d75d3b0 0x56291db507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:27.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.435+0000 7f215f9cb740 1 -- 172.21.15.39:0/2259198587 >> 172.21.15.39:0/2259198587 conn(0x56291d77a3c0 msgr2=0x56291db285c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:27.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.436+0000 7f2132a00640 0 client.6426 ms_handle_reset on v2:172.21.15.39:6832/829716143 2024-09-21T07:36:27.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.436+0000 7f215f9cb740 1 -- 172.21.15.39:0/2259198587 shutdown_connections 2024-09-21T07:36:27.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.436+0000 7f215f9cb740 1 -- 172.21.15.39:0/2259198587 wait complete. 2024-09-21T07:36:27.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.436+0000 7f215f9cb740 20 client.6426 trim_cache size 0 max 16384 2024-09-21T07:36:27.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.455+0000 7f215f9cb740 1 Processor -- start 2024-09-21T07:36:27.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.455+0000 7f215f9cb740 1 -- start start 2024-09-21T07:36:27.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.457+0000 7f215f9cb740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56291d75d3b0 0x56291db51100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:27.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.457+0000 7f215f9cb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56291d765590 0x56291db892e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:27.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.457+0000 7f215f9cb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56291d786270 0x56291db89820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:27.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.457+0000 7f215f9cb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2150203700 con 0x56291d786270 2024-09-21T07:36:27.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.457+0000 7f215f9cb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2150202a90 con 0x56291d765590 2024-09-21T07:36:27.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.457+0000 7f215f9cb740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f21502030b0 con 0x56291d75d3b0 2024-09-21T07:36:27.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.457+0000 7f2156800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56291d786270 0x56291db89820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:27.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.457+0000 7f2156800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56291d786270 0x56291db89820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52316/0 (socket says 172.21.15.39:52316) 2024-09-21T07:36:27.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.457+0000 7f2156800640 1 -- 172.21.15.39:0/2074217612 learned_addr learned my addr 172.21.15.39:0/2074217612 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:27.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.457+0000 7f2155400640 1 --2- 172.21.15.39:0/2074217612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56291d765590 0x56291db892e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:27.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.457+0000 7f2155e00640 1 --2- 172.21.15.39:0/2074217612 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56291d75d3b0 0x56291db51100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:27.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.457+0000 7f2156800640 1 -- 172.21.15.39:0/2074217612 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56291d75d3b0 msgr2=0x56291db51100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:36:27.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.457+0000 7f2156800640 1 --2- 172.21.15.39:0/2074217612 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56291d75d3b0 0x56291db51100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:27.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.457+0000 7f2156800640 1 -- 172.21.15.39:0/2074217612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56291d765590 msgr2=0x56291db892e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:36:27.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.457+0000 7f2156800640 1 --2- 172.21.15.39:0/2074217612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56291d765590 0x56291db892e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:27.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.457+0000 7f2156800640 1 -- 172.21.15.39:0/2074217612 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56291db2d840 con 0x56291d786270 2024-09-21T07:36:27.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.458+0000 7f2156800640 1 --2- 172.21.15.39:0/2074217612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56291d786270 0x56291db89820 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f214c007f10 tx=0x7f214c011800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27524b247a3e11af server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:27.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.458+0000 7f213dc00640 1 -- 172.21.15.39:0/2074217612 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c00f850 con 0x56291d786270 2024-09-21T07:36:27.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.458+0000 7f213dc00640 1 -- 172.21.15.39:0/2074217612 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f214c00fa10 con 0x56291d786270 2024-09-21T07:36:27.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.458+0000 7f213dc00640 1 -- 172.21.15.39:0/2074217612 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c00fe60 con 0x56291d786270 2024-09-21T07:36:27.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.459+0000 7f215f9cb740 1 -- 172.21.15.39:0/2074217612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56291d786270 msgr2=0x56291db89820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:27.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.459+0000 7f215f9cb740 1 --2- 172.21.15.39:0/2074217612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56291d786270 0x56291db89820 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f214c007f10 tx=0x7f214c011800 comp rx=0 tx=0).stop 2024-09-21T07:36:27.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.459+0000 7f215f9cb740 1 -- 172.21.15.39:0/2074217612 shutdown_connections 2024-09-21T07:36:27.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.459+0000 7f215f9cb740 1 --2- 172.21.15.39:0/2074217612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56291d786270 0x56291db89820 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:27.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.459+0000 7f215f9cb740 1 --2- 172.21.15.39:0/2074217612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56291d765590 0x56291db892e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:27.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.459+0000 7f215f9cb740 1 --2- 172.21.15.39:0/2074217612 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56291d75d3b0 0x56291db51100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:27.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.459+0000 7f215f9cb740 1 -- 172.21.15.39:0/2074217612 >> 172.21.15.39:0/2074217612 conn(0x56291d77a3c0 msgr2=0x56291db226a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:27.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.459+0000 7f215f9cb740 1 -- 172.21.15.39:0/2074217612 shutdown_connections 2024-09-21T07:36:27.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.459+0000 7f215f9cb740 1 -- 172.21.15.39:0/2074217612 wait complete. 2024-09-21T07:36:27.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.460+0000 7f215f9cb740 1 -- start start 2024-09-21T07:36:27.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.460+0000 7f215f9cb740 1 Processor -- start 2024-09-21T07:36:27.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.461+0000 7f215f9cb740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56291d75d3b0 0x56291dbf1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:27.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.461+0000 7f215f9cb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56291d765590 0x56291dbe7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:27.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.461+0000 7f215f9cb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56291d786270 0x56291db51100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:27.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.461+0000 7f215f9cb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56291db50d30 con 0x56291d765590 2024-09-21T07:36:27.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.461+0000 7f215f9cb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56291db817c0 con 0x56291d786270 2024-09-21T07:36:27.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.461+0000 7f215f9cb740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x56291db6ca20 con 0x56291d75d3b0 2024-09-21T07:36:27.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.461+0000 7f215f9cb740 10 client.0 _ll_register_callbacks cb 0x7f2140004260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:36:27.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.462+0000 7f2155400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56291d765590 0x56291dbe7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:27.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.462+0000 7f2155400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56291d765590 0x56291dbe7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52318/0 (socket says 172.21.15.39:52318) 2024-09-21T07:36:27.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.462+0000 7f2155400640 1 -- 172.21.15.39:0/164211829 learned_addr learned my addr 172.21.15.39:0/164211829 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:27.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.462+0000 7f2155e00640 1 --2- 172.21.15.39:0/164211829 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56291d75d3b0 0x56291dbf1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:27.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.462+0000 7f2156800640 1 --2- 172.21.15.39:0/164211829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56291d786270 0x56291db51100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:27.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.462+0000 7f2155400640 1 -- 172.21.15.39:0/164211829 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56291d75d3b0 msgr2=0x56291dbf1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:36:27.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.462+0000 7f2155400640 1 --2- 172.21.15.39:0/164211829 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56291d75d3b0 0x56291dbf1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:27.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.462+0000 7f2155400640 1 -- 172.21.15.39:0/164211829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56291d786270 msgr2=0x56291db51100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:36:27.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.462+0000 7f2155400640 1 --2- 172.21.15.39:0/164211829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56291d786270 0x56291db51100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:27.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.462+0000 7f2155400640 1 -- 172.21.15.39:0/164211829 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56291db89390 con 0x56291d765590 2024-09-21T07:36:27.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.462+0000 7f2155400640 1 --2- 172.21.15.39:0/164211829 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56291d765590 0x56291dbe7fa0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f2140004c00 tx=0x7f2140047cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb3942d0cbc1c52f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:27.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.462+0000 7f2132a00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:36:27.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.463+0000 7f2132a00640 1 -- 172.21.15.39:0/164211829 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214005e070 con 0x56291d765590 2024-09-21T07:36:27.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.463+0000 7f2132a00640 1 -- 172.21.15.39:0/164211829 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 6800+0+0 (secure 0 0 0) 0x7f2140004fa0 con 0x56291d765590 2024-09-21T07:36:27.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.463+0000 7f215f9cb740 1 -- 172.21.15.39:0/164211829 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f214c00fa10 con 0x56291d765590 2024-09-21T07:36:27.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.463+0000 7f2132a00640 1 -- 172.21.15.39:0/164211829 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f2140035040 con 0x56291d765590 2024-09-21T07:36:27.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.463+0000 7f215f9cb740 10 client.6432 fetch_fsmap learned FSMap version 54 2024-09-21T07:36:27.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.463+0000 7f215f9cb740 1 -- 172.21.15.39:0/164211829 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x56291dbf1c40 con 0x56291d765590 2024-09-21T07:36:27.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.463+0000 7f2132a00640 1 -- 172.21.15.39:0/164211829 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f214c00fa10 con 0x56291d765590 2024-09-21T07:36:27.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.463+0000 7f2132a00640 1 -- 172.21.15.39:0/164211829 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 54) ==== 3502+0+0 (secure 0 0 0) 0x7f2140064070 con 0x56291d765590 2024-09-21T07:36:27.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.463+0000 7f215f9cb740 10 client.6432 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T07:36:27.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.463+0000 7f215f9cb740 10 client.6432 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:36:27.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.463+0000 7f215f9cb740 20 client.6432 populate_metadata read hostname 'smithi039' 2024-09-21T07:36:27.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.463+0000 7f215f9cb740 1 --2- 172.21.15.39:0/164211829 >> [v2:172.21.15.39:6832/829716143,v1:172.21.15.39:6834/829716143] conn(0x56291db722a0 0x56291db56670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:27.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.463+0000 7f215f9cb740 4 client.6432 mds_command: new command op to 6321 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:36:27.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.463+0000 7f215f9cb740 1 -- 172.21.15.39:0/164211829 --> [v2:172.21.15.39:6832/829716143,v1:172.21.15.39:6834/829716143] -- command(tid 0: {"prefix": "status"}) -- 0x56291db6d0a0 con 0x56291db722a0 2024-09-21T07:36:27.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.464+0000 7f2155e00640 1 --2- 172.21.15.39:0/164211829 >> [v2:172.21.15.39:6832/829716143,v1:172.21.15.39:6834/829716143] conn(0x56291db722a0 0x56291db56670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:27.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.464+0000 7f2155e00640 1 --2- 172.21.15.39:0/164211829 >> [v2:172.21.15.39:6832/829716143,v1:172.21.15.39:6834/829716143] conn(0x56291db722a0 0x56291db56670 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2743b10d7d2bc1fa server_cookie=83c6d656572a4992 in_seq=0 out_seq=0 2024-09-21T07:36:27.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.464+0000 7f2132a00640 10 client.6432 ms_handle_connect on v2:172.21.15.39:6832/829716143 2024-09-21T07:36:27.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.464+0000 7f2132a00640 1 -- 172.21.15.39:0/164211829 <== mds.0 v2:172.21.15.39:6832/829716143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x56291db6d0a0 con 0x56291db722a0 2024-09-21T07:36:27.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.464+0000 7f2132a00640 10 client.6432 handle_command_reply: tid=0 2024-09-21T07:36:27.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.464+0000 7f215f9cb740 1 client.6432 shutdown 2024-09-21T07:36:27.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.465+0000 7f215f9cb740 1 -- 172.21.15.39:0/164211829 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56291d765590 msgr2=0x56291dbe7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:27.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.465+0000 7f215f9cb740 1 --2- 172.21.15.39:0/164211829 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56291d765590 0x56291dbe7fa0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f2140004c00 tx=0x7f2140047cc0 comp rx=0 tx=0).stop 2024-09-21T07:36:27.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.465+0000 7f215f9cb740 1 -- 172.21.15.39:0/164211829 shutdown_connections 2024-09-21T07:36:27.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.465+0000 7f215f9cb740 1 --2- 172.21.15.39:0/164211829 >> [v2:172.21.15.39:6832/829716143,v1:172.21.15.39:6834/829716143] conn(0x56291db722a0 0x56291db56670 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:27.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.465+0000 7f215f9cb740 1 --2- 172.21.15.39:0/164211829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56291d786270 0x56291db51100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:27.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.465+0000 7f215f9cb740 1 --2- 172.21.15.39:0/164211829 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56291d765590 0x56291dbe7fa0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:27.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.465+0000 7f215f9cb740 1 --2- 172.21.15.39:0/164211829 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56291d75d3b0 0x56291dbf1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:27.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.465+0000 7f2132a00640 0 client.6432 ms_handle_reset on v2:172.21.15.39:6832/829716143 2024-09-21T07:36:27.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.465+0000 7f215f9cb740 1 -- 172.21.15.39:0/164211829 >> 172.21.15.39:0/164211829 conn(0x56291d77a3c0 msgr2=0x56291db94be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:27.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.465+0000 7f215f9cb740 1 -- 172.21.15.39:0/164211829 shutdown_connections 2024-09-21T07:36:27.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.465+0000 7f215f9cb740 1 -- 172.21.15.39:0/164211829 wait complete. 2024-09-21T07:36:27.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.465+0000 7f215f9cb740 20 client.6432 trim_cache size 0 max 16384 2024-09-21T07:36:27.470 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:36:27.470 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:36:27.470 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-09-21T07:36:27.470 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 6321, 2024-09-21T07:36:27.470 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:36:27.470 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:36:27.470 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:36:27.470 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 6.7954706549999999, 2024-09-21T07:36:27.470 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 54, 2024-09-21T07:36:27.470 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 90, 2024-09-21T07:36:27.471 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 90, 2024-09-21T07:36:27.471 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 11.777923369 2024-09-21T07:36:27.471 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:36:27.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.465+0000 7f2157200640 1 -- 172.21.15.39:0/881984971 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2134078b90 msgr2=0x7f213407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:27.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.465+0000 7f2157200640 1 --2- 172.21.15.39:0/881984971 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2134078b90 0x7f213407b0a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2140033000 tx=0x7f2140004730 comp rx=0 tx=0).stop 2024-09-21T07:36:27.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.465+0000 7f2157200640 1 -- 172.21.15.39:0/881984971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21500cf540 msgr2=0x7f21500c60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:27.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.465+0000 7f2157200640 1 --2- 172.21.15.39:0/881984971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21500cf540 0x7f21500c60f0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f214400b410 tx=0x7f214400b8d0 comp rx=0 tx=0).stop 2024-09-21T07:36:27.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.466+0000 7f2157200640 1 -- 172.21.15.39:0/881984971 shutdown_connections 2024-09-21T07:36:27.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.466+0000 7f2157200640 1 --2- 172.21.15.39:0/881984971 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2134078b90 0x7f213407b0a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:27.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.466+0000 7f2157200640 1 --2- 172.21.15.39:0/881984971 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f215015ca50 0x7f2150158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:27.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.466+0000 7f2157200640 1 --2- 172.21.15.39:0/881984971 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21500cf950 0x7f2150157060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:27.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.466+0000 7f2157200640 1 --2- 172.21.15.39:0/881984971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21500cf540 0x7f21500c60f0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:27.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.466+0000 7f2157200640 1 -- 172.21.15.39:0/881984971 >> 172.21.15.39:0/881984971 conn(0x7f215006d410 msgr2=0x7f215015a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:27.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.466+0000 7f2157200640 1 -- 172.21.15.39:0/881984971 shutdown_connections 2024-09-21T07:36:27.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.466+0000 7f2157200640 1 -- 172.21.15.39:0/881984971 wait complete. 2024-09-21T07:36:27.493 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:36:27.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.722+0000 7f3443200640 1 -- 172.21.15.39:0/875429082 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3420004d40 msgr2=0x7f3420005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:27.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.722+0000 7f3443200640 1 --2- 172.21.15.39:0/875429082 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3420004d40 0x7f3420005210 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f343c0afbd0 tx=0x7f3420005830 comp rx=0 tx=0).stop 2024-09-21T07:36:27.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.722+0000 7f3441200640 1 -- 172.21.15.39:0/875429082 reap_dead start 2024-09-21T07:36:27.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.722+0000 7f3443200640 1 -- 172.21.15.39:0/875429082 shutdown_connections 2024-09-21T07:36:27.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.722+0000 7f3443200640 1 -- 172.21.15.39:0/875429082 >> 172.21.15.39:0/875429082 conn(0x7f343c06d7d0 msgr2=0x7f343c0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:27.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.722+0000 7f3443200640 1 -- 172.21.15.39:0/875429082 shutdown_connections 2024-09-21T07:36:27.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.722+0000 7f3443200640 1 -- 172.21.15.39:0/875429082 wait complete. 2024-09-21T07:36:27.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.725+0000 7f3443200640 1 Processor -- start 2024-09-21T07:36:27.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.725+0000 7f3443200640 1 -- start start 2024-09-21T07:36:27.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.727+0000 7f3443200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3420004d40 0x7f343c1f9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:27.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.727+0000 7f3443200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f343c09ab80 0x7f343c142bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:27.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.727+0000 7f3443200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f343c072e30 0x7f343c073240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:27.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.727+0000 7f3443200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f343c0b44d0 con 0x7f3420004d40 2024-09-21T07:36:27.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.727+0000 7f3443200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f343c0b4350 con 0x7f343c072e30 2024-09-21T07:36:27.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.727+0000 7f3443200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f343c0b4650 con 0x7f343c09ab80 2024-09-21T07:36:27.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.727+0000 7f343be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3420004d40 0x7f343c1f9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:27.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.728+0000 7f343be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3420004d40 0x7f343c1f9750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52342/0 (socket says 172.21.15.39:52342) 2024-09-21T07:36:27.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.728+0000 7f343be00640 1 -- 172.21.15.39:0/3738002327 learned_addr learned my addr 172.21.15.39:0/3738002327 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:27.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.728+0000 7f343b400640 1 --2- 172.21.15.39:0/3738002327 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f343c09ab80 0x7f343c142bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:27.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.728+0000 7f3441200640 1 --2- 172.21.15.39:0/3738002327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f343c072e30 0x7f343c073240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:27.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.728+0000 7f343be00640 1 -- 172.21.15.39:0/3738002327 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f343c09ab80 msgr2=0x7f343c142bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:27.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.728+0000 7f343be00640 1 --2- 172.21.15.39:0/3738002327 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f343c09ab80 0x7f343c142bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:27.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.728+0000 7f343be00640 1 -- 172.21.15.39:0/3738002327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f343c072e30 msgr2=0x7f343c073240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:27.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.728+0000 7f343be00640 1 --2- 172.21.15.39:0/3738002327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f343c072e30 0x7f343c073240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:27.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.728+0000 7f343be00640 1 -- 172.21.15.39:0/3738002327 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f343c143130 con 0x7f3420004d40 2024-09-21T07:36:27.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.728+0000 7f343be00640 1 --2- 172.21.15.39:0/3738002327 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3420004d40 0x7f343c1f9750 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f342c003f80 tx=0x7f342c001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:27.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.731+0000 7f3433e00640 1 -- 172.21.15.39:0/3738002327 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c011640 con 0x7f3420004d40 2024-09-21T07:36:27.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.731+0000 7f3433e00640 1 -- 172.21.15.39:0/3738002327 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f342c016020 con 0x7f3420004d40 2024-09-21T07:36:27.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.731+0000 7f3433e00640 1 -- 172.21.15.39:0/3738002327 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c011a40 con 0x7f3420004d40 2024-09-21T07:36:27.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.731+0000 7f3443200640 1 -- 172.21.15.39:0/3738002327 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f343c1f8c90 con 0x7f3420004d40 2024-09-21T07:36:27.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.731+0000 7f3443200640 1 -- 172.21.15.39:0/3738002327 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f343c09bd20 con 0x7f3420004d40 2024-09-21T07:36:27.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.732+0000 7f3443200640 1 -- 172.21.15.39:0/3738002327 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f343c15ea20 con 0x7f3420004d40 2024-09-21T07:36:27.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.733+0000 7f3433e00640 1 -- 172.21.15.39:0/3738002327 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f342c011be0 con 0x7f3420004d40 2024-09-21T07:36:27.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.734+0000 7f3433e00640 1 --2- 172.21.15.39:0/3738002327 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f341c078bc0 0x7f341c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:27.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.734+0000 7f3433e00640 1 -- 172.21.15.39:0/3738002327 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6800+0+0 (secure 0 0 0) 0x7f342c09bc50 con 0x7f3420004d40 2024-09-21T07:36:27.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.734+0000 7f343b400640 1 --2- 172.21.15.39:0/3738002327 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f341c078bc0 0x7f341c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:27.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.735+0000 7f343b400640 1 --2- 172.21.15.39:0/3738002327 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f341c078bc0 0x7f341c07b0d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f343c15ac60 tx=0x7f3428005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:27.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.738+0000 7f3433e00640 1 -- 172.21.15.39:0/3738002327 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f342c020020 con 0x7f3420004d40 2024-09-21T07:36:27.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.973+0000 7f3443200640 1 -- 172.21.15.39:0/3738002327 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f343c09bef0 con 0x7f3420004d40 2024-09-21T07:36:27.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.974+0000 7f3433e00640 1 -- 172.21.15.39:0/3738002327 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v54) ==== 53+0+73 (secure 0 0 0) 0x7f342c063c60 con 0x7f3420004d40 2024-09-21T07:36:27.976 INFO:teuthology.orchestra.run.smithi039.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:36:27.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.977+0000 7f3443200640 1 -- 172.21.15.39:0/3738002327 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f341c078bc0 msgr2=0x7f341c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:27.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.977+0000 7f3443200640 1 --2- 172.21.15.39:0/3738002327 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f341c078bc0 0x7f341c07b0d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f343c15ac60 tx=0x7f3428005f90 comp rx=0 tx=0).stop 2024-09-21T07:36:27.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.977+0000 7f3443200640 1 -- 172.21.15.39:0/3738002327 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3420004d40 msgr2=0x7f343c1f9750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:27.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.977+0000 7f3443200640 1 --2- 172.21.15.39:0/3738002327 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3420004d40 0x7f343c1f9750 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f342c003f80 tx=0x7f342c001630 comp rx=0 tx=0).stop 2024-09-21T07:36:27.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.977+0000 7f3443200640 1 -- 172.21.15.39:0/3738002327 shutdown_connections 2024-09-21T07:36:27.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.977+0000 7f3443200640 1 --2- 172.21.15.39:0/3738002327 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f341c078bc0 0x7f341c07b0d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:27.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.977+0000 7f3443200640 1 --2- 172.21.15.39:0/3738002327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f343c072e30 0x7f343c073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:27.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.977+0000 7f3443200640 1 --2- 172.21.15.39:0/3738002327 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f343c09ab80 0x7f343c142bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:27.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.977+0000 7f3443200640 1 --2- 172.21.15.39:0/3738002327 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3420004d40 0x7f343c1f9750 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:27.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.977+0000 7f3443200640 1 -- 172.21.15.39:0/3738002327 >> 172.21.15.39:0/3738002327 conn(0x7f343c06d7d0 msgr2=0x7f343c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:27.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.977+0000 7f3443200640 1 -- 172.21.15.39:0/3738002327 shutdown_connections 2024-09-21T07:36:27.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:27.977+0000 7f3443200640 1 -- 172.21.15.39:0/3738002327 wait complete. 2024-09-21T07:36:28.000 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:36:28.001 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:36:28.001 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi039.front.sepia.ceph.com 2024-09-21T07:36:28.001 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T07:36:28.001 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:36:28.001 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:36:28.001 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:36:28.001 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:36:28.001 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:36:28.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:36:28.001 DEBUG:teuthology.orchestra.run.smithi039:> ip addr 2024-09-21T07:36:28.037 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:36:28.037 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:36:28.037 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:36:28.037 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:36:28.037 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2024-09-21T07:36:28.037 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:36:28.037 INFO:teuthology.orchestra.run.smithi039.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:36:28.037 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:36:28.038 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f0 2024-09-21T07:36:28.038 INFO:teuthology.orchestra.run.smithi039.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:36:28.038 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c7 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:36:28.038 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f1 2024-09-21T07:36:28.038 INFO:teuthology.orchestra.run.smithi039.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:36:28.038 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:70 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:36:28.038 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f0 2024-09-21T07:36:28.038 INFO:teuthology.orchestra.run.smithi039.stdout: inet 172.21.15.39/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T07:36:28.038 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft 42531sec preferred_lft 42531sec 2024-09-21T07:36:28.038 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec4:7aff:febd:1370/64 scope link 2024-09-21T07:36:28.038 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:36:28.038 INFO:teuthology.orchestra.run.smithi039.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:36:28.038 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:71 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:36:28.039 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f1 2024-09-21T07:36:28.039 INFO:teuthology.orchestra.run.smithi039.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:36:28.039 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff 2024-09-21T07:36:28.039 INFO:teuthology.orchestra.run.smithi039.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:36:28.039 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:36:28.039 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::b079:ddff:fe0e:6333/64 scope link 2024-09-21T07:36:28.039 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:36:28.039 INFO:teuthology.orchestra.run.smithi039.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:36:28.039 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:36:28.039 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::2ca7:98ff:fe8a:d0ed/64 scope link 2024-09-21T07:36:28.039 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:36:28.039 INFO:teuthology.orchestra.run.smithi039.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:36:28.039 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether ee:10:53:01:0c:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:36:28.039 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec10:53ff:fe01:c44/64 scope link 2024-09-21T07:36:28.040 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:36:28.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:36:28.040 DEBUG:teuthology.orchestra.run.smithi039:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:36:28.131 INFO:teuthology.orchestra.run.smithi039.stdout:1 2024-09-21T07:36:28.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:36:28.134 DEBUG:teuthology.orchestra.run.smithi039:> route 2024-09-21T07:36:28.170 INFO:teuthology.orchestra.run.smithi039.stdout:Kernel IP routing table 2024-09-21T07:36:28.170 INFO:teuthology.orchestra.run.smithi039.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:36:28.170 INFO:teuthology.orchestra.run.smithi039.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T07:36:28.170 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T07:36:28.170 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:36:28.170 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:36:28.170 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-21T07:36:28.170 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T07:36:28.171 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T07:36:28.171 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T07:36:28.171 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-21T07:36:28.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:36:28.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:36:28.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:36:28.367 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-09-21T07:36:28.402 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:36:28.402 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T07:36:28.403 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:36:28.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:36:28.403 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-09-21T07:36:28.500 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:36:28.500 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T07:36:28.534 INFO:teuthology.orchestra.run.smithi039.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:36:28.534 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:36:28.535 DEBUG:teuthology.orchestra.run.smithi039:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:36:28.609 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-21T07:36:28.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:36:28.609 DEBUG:teuthology.orchestra.run.smithi039:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:36:28.809 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:36:28.809 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:36:28.846 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:36:28.846 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:36:28.846 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: resolved to: "172.21.15.39:3300,172.21.15.148:3300,172.21.15.148:3301" 2024-09-21T07:36:28.846 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: trying mount with new device syntax: 0@7ab9b04a-f88d-49aa-a493-e5c076637ffb.cephfs=/ 2024-09-21T07:36:28.846 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.39:3300/172.21.15.148:3300/172.21.15.148:3301" will pass to kernel 2024-09-21T07:36:28.848 INFO:teuthology.orchestra.run.smithi039.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T07:36:28.848 INFO:teuthology.orchestra.run.smithi039.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:36:28.848 INFO:teuthology.orchestra.run.smithi039.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:36:28.848 INFO:teuthology.orchestra.run.smithi039.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:36:28.848 INFO:teuthology.orchestra.run.smithi039.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:36:28.851 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:36:28.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:36:28.851 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:36:28.908 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:36:28.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:36:28.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:36:29.004 INFO:teuthology.orchestra.run.smithi039.stdout:client6441 2024-09-21T07:36:29.004 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:36:29.004 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:36:29.040 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get mds_max_snaps_per_dir 2024-09-21T07:36:29.321 INFO:teuthology.orchestra.run.smithi039.stdout:{"mds_max_snaps_per_dir":"100"} 2024-09-21T07:36:29.321 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_max_snaps_per_dir": "100" } 2024-09-21T07:36:29.321 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2024-09-21T07:36:29.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:29.551+0000 7f853cc00640 1 -- 172.21.15.39:0/2606043644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85380b3480 msgr2=0x7f8538072e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:29.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:29.551+0000 7f853cc00640 1 --2- 172.21.15.39:0/2606043644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85380b3480 0x7f8538072e80 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f8538071db0 tx=0x7f852002a410 comp rx=0 tx=0).stop 2024-09-21T07:36:29.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:29.551+0000 7f853cc00640 1 -- 172.21.15.39:0/2606043644 shutdown_connections 2024-09-21T07:36:29.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:29.551+0000 7f853cc00640 1 --2- 172.21.15.39:0/2606043644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85380b3480 0x7f8538072e80 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:29.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:29.551+0000 7f853cc00640 1 --2- 172.21.15.39:0/2606043644 >> v2:172.21.15.148:3300/0 conn(0x7f85380748f0 0x7f85380b2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:29.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:29.551+0000 7f853cc00640 1 -- 172.21.15.39:0/2606043644 >> 172.21.15.39:0/2606043644 conn(0x7f85380b1c40 msgr2=0x7f85380b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:29.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:29.551+0000 7f853cc00640 1 -- 172.21.15.39:0/2606043644 shutdown_connections 2024-09-21T07:36:29.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:29.551+0000 7f853cc00640 1 -- 172.21.15.39:0/2606043644 wait complete. 2024-09-21T07:36:29.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:29.553+0000 7f853cc00640 1 Processor -- start 2024-09-21T07:36:29.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:29.553+0000 7f853cc00640 1 -- start start 2024-09-21T07:36:29.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:29.555+0000 7f853cc00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f85380733c0 0x7f85380adf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:29.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:29.555+0000 7f853cc00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85380748f0 0x7f85380c7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:29.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:29.555+0000 7f853cc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85380b3480 0x7f85380c5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:29.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:29.555+0000 7f853cc00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8538074060 con 0x7f85380b3480 2024-09-21T07:36:29.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:29.555+0000 7f853cc00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8538073ee0 con 0x7f85380748f0 2024-09-21T07:36:29.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:29.555+0000 7f853cc00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f853806fe50 con 0x7f85380733c0 2024-09-21T07:36:29.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:29.555+0000 7f8536800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85380b3480 0x7f85380c5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:29.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:29.555+0000 7f8536800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85380b3480 0x7f85380c5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52360/0 (socket says 172.21.15.39:52360) 2024-09-21T07:36:29.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:29.555+0000 7f8536800640 1 -- 172.21.15.39:0/4272850825 learned_addr learned my addr 172.21.15.39:0/4272850825 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:29.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:29.555+0000 7f852d400640 1 --2- 172.21.15.39:0/4272850825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85380748f0 0x7f85380c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:29.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:29.555+0000 7f8535e00640 1 --2- 172.21.15.39:0/4272850825 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f85380733c0 0x7f85380adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:29.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:29.555+0000 7f8536800640 1 -- 172.21.15.39:0/4272850825 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f85380733c0 msgr2=0x7f85380adf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:29.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:29.555+0000 7f8536800640 1 --2- 172.21.15.39:0/4272850825 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f85380733c0 0x7f85380adf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:29.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:29.555+0000 7f8536800640 1 -- 172.21.15.39:0/4272850825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85380748f0 msgr2=0x7f85380c7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:29.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:29.555+0000 7f8536800640 1 --2- 172.21.15.39:0/4272850825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85380748f0 0x7f85380c7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:29.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:29.555+0000 7f8536800640 1 -- 172.21.15.39:0/4272850825 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85381f2ef0 con 0x7f85380b3480 2024-09-21T07:36:29.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:29.556+0000 7f8536800640 1 --2- 172.21.15.39:0/4272850825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85380b3480 0x7f85380c5ce0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f853000c6c0 tx=0x7f853000cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:29.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:29.556+0000 7f852ea00640 1 -- 172.21.15.39:0/4272850825 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8530017070 con 0x7f85380b3480 2024-09-21T07:36:29.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:29.556+0000 7f852ea00640 1 -- 172.21.15.39:0/4272850825 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8530003c60 con 0x7f85380b3480 2024-09-21T07:36:29.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:29.556+0000 7f853cc00640 1 -- 172.21.15.39:0/4272850825 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85380c9c10 con 0x7f85380b3480 2024-09-21T07:36:29.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:29.556+0000 7f852ea00640 1 -- 172.21.15.39:0/4272850825 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85300052d0 con 0x7f85380b3480 2024-09-21T07:36:29.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:29.556+0000 7f853cc00640 1 -- 172.21.15.39:0/4272850825 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85381f3490 con 0x7f85380b3480 2024-09-21T07:36:29.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:29.557+0000 7f853cc00640 1 -- 172.21.15.39:0/4272850825 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85380b2f40 con 0x7f85380b3480 2024-09-21T07:36:29.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:29.559+0000 7f852ea00640 1 -- 172.21.15.39:0/4272850825 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f853001a070 con 0x7f85380b3480 2024-09-21T07:36:29.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:29.560+0000 7f852ea00640 1 --2- 172.21.15.39:0/4272850825 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8514078bc0 0x7f851407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:29.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:29.560+0000 7f852ea00640 1 -- 172.21.15.39:0/4272850825 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6800+0+0 (secure 0 0 0) 0x7f853009a760 con 0x7f85380b3480 2024-09-21T07:36:29.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:29.560+0000 7f8535e00640 1 --2- 172.21.15.39:0/4272850825 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8514078bc0 0x7f851407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:29.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:29.561+0000 7f8535e00640 1 --2- 172.21.15.39:0/4272850825 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8514078bc0 0x7f851407b0d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8528005fd0 tx=0x7f8528005850 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:29.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:29.564+0000 7f852ea00640 1 -- 172.21.15.39:0/4272850825 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8530062770 con 0x7f85380b3480 2024-09-21T07:36:29.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:29.800+0000 7f853cc00640 1 -- 172.21.15.39:0/4272850825 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f85380c9fe0 con 0x7f85380b3480 2024-09-21T07:36:30.358 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:36:30.363 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:36:30.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:30.741+0000 7f852ea00640 1 -- 172.21.15.39:0/4272850825 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v55) ==== 167+0+0 (secure 0 0 0) 0x7f8530062070 con 0x7f85380b3480 2024-09-21T07:36:30.744 INFO:teuthology.orchestra.run.smithi039.stderr:enabled new snapshots 2024-09-21T07:36:30.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:30.744+0000 7f853cc00640 1 -- 172.21.15.39:0/4272850825 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8514078bc0 msgr2=0x7f851407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:30.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:30.744+0000 7f853cc00640 1 --2- 172.21.15.39:0/4272850825 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8514078bc0 0x7f851407b0d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8528005fd0 tx=0x7f8528005850 comp rx=0 tx=0).stop 2024-09-21T07:36:30.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:30.744+0000 7f853cc00640 1 -- 172.21.15.39:0/4272850825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85380b3480 msgr2=0x7f85380c5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:30.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:30.744+0000 7f853cc00640 1 --2- 172.21.15.39:0/4272850825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85380b3480 0x7f85380c5ce0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f853000c6c0 tx=0x7f853000cb80 comp rx=0 tx=0).stop 2024-09-21T07:36:30.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:30.744+0000 7f853cc00640 1 -- 172.21.15.39:0/4272850825 shutdown_connections 2024-09-21T07:36:30.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:30.744+0000 7f853cc00640 1 --2- 172.21.15.39:0/4272850825 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8514078bc0 0x7f851407b0d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:30.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:30.744+0000 7f853cc00640 1 --2- 172.21.15.39:0/4272850825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85380b3480 0x7f85380c5ce0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:30.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:30.744+0000 7f853cc00640 1 --2- 172.21.15.39:0/4272850825 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85380748f0 0x7f85380c7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:30.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:30.744+0000 7f853cc00640 1 --2- 172.21.15.39:0/4272850825 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f85380733c0 0x7f85380adf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:30.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:30.744+0000 7f853cc00640 1 -- 172.21.15.39:0/4272850825 >> 172.21.15.39:0/4272850825 conn(0x7f85380b1c40 msgr2=0x7f85380741e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:30.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:30.744+0000 7f853cc00640 1 -- 172.21.15.39:0/4272850825 shutdown_connections 2024-09-21T07:36:30.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:30.744+0000 7f853cc00640 1 -- 172.21.15.39:0/4272850825 wait complete. 2024-09-21T07:36:30.771 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 2 2024-09-21T07:36:31.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:30.998+0000 7f3a86200640 1 -- 172.21.15.39:0/3752633409 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a800cb0b0 msgr2=0x7f3a8006ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:31.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:30.998+0000 7f3a86200640 1 --2- 172.21.15.39:0/3752633409 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a800cb0b0 0x7f3a8006ee90 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f3a800d11c0 tx=0x7f3a7802b9a0 comp rx=0 tx=0).stop 2024-09-21T07:36:31.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:30.998+0000 7f3a86200640 1 -- 172.21.15.39:0/3752633409 shutdown_connections 2024-09-21T07:36:31.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:30.998+0000 7f3a86200640 1 --2- 172.21.15.39:0/3752633409 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a800cb0b0 0x7f3a8006ee90 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:31.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:30.998+0000 7f3a86200640 1 --2- 172.21.15.39:0/3752633409 >> v2:172.21.15.148:3300/0 conn(0x7f3a800cfb00 0x7f3a8015bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:31.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:30.998+0000 7f3a86200640 1 -- 172.21.15.39:0/3752633409 >> 172.21.15.39:0/3752633409 conn(0x7f3a800b1cb0 msgr2=0x7f3a800b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:31.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:30.998+0000 7f3a86200640 1 -- 172.21.15.39:0/3752633409 shutdown_connections 2024-09-21T07:36:31.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:30.998+0000 7f3a86200640 1 -- 172.21.15.39:0/3752633409 wait complete. 2024-09-21T07:36:31.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.000+0000 7f3a86200640 1 Processor -- start 2024-09-21T07:36:31.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.000+0000 7f3a86200640 1 -- start start 2024-09-21T07:36:31.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.002+0000 7f3a86200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a800cf160 0x7f3a801f62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:31.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.002+0000 7f3a86200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3a800cfb00 0x7f3a8015eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:31.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.002+0000 7f3a86200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a8009d6c0 0x7f3a8014e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:31.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.002+0000 7f3a86200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3a800ccba0 con 0x7f3a800cf160 2024-09-21T07:36:31.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.002+0000 7f3a86200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3a800cf780 con 0x7f3a8009d6c0 2024-09-21T07:36:31.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.002+0000 7f3a86200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f3a800ccd20 con 0x7f3a800cfb00 2024-09-21T07:36:31.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.002+0000 7f3a7f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a800cf160 0x7f3a801f62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:31.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.002+0000 7f3a7f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a800cf160 0x7f3a801f62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52384/0 (socket says 172.21.15.39:52384) 2024-09-21T07:36:31.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.002+0000 7f3a7f400640 1 -- 172.21.15.39:0/2286790095 learned_addr learned my addr 172.21.15.39:0/2286790095 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:31.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.002+0000 7f3a7fe00640 1 --2- 172.21.15.39:0/2286790095 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a8009d6c0 0x7f3a8014e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:31.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.002+0000 7f3a76a00640 1 --2- 172.21.15.39:0/2286790095 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3a800cfb00 0x7f3a8015eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:31.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.003+0000 7f3a7f400640 1 -- 172.21.15.39:0/2286790095 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3a800cfb00 msgr2=0x7f3a8015eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:31.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.003+0000 7f3a7f400640 1 --2- 172.21.15.39:0/2286790095 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3a800cfb00 0x7f3a8015eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:31.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.003+0000 7f3a7f400640 1 -- 172.21.15.39:0/2286790095 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a8009d6c0 msgr2=0x7f3a8014e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:31.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.003+0000 7f3a7f400640 1 --2- 172.21.15.39:0/2286790095 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a8009d6c0 0x7f3a8014e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:31.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.003+0000 7f3a7f400640 1 -- 172.21.15.39:0/2286790095 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a801f6800 con 0x7f3a800cf160 2024-09-21T07:36:31.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.003+0000 7f3a7f400640 1 --2- 172.21.15.39:0/2286790095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a800cf160 0x7f3a801f62c0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f3a7000d780 tx=0x7f3a7000dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:31.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.003+0000 7f3a77e00640 1 -- 172.21.15.39:0/2286790095 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a70017070 con 0x7f3a800cf160 2024-09-21T07:36:31.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.003+0000 7f3a77e00640 1 -- 172.21.15.39:0/2286790095 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a7000de90 con 0x7f3a800cf160 2024-09-21T07:36:31.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.003+0000 7f3a77e00640 1 -- 172.21.15.39:0/2286790095 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a700099d0 con 0x7f3a800cf160 2024-09-21T07:36:31.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.003+0000 7f3a86200640 1 -- 172.21.15.39:0/2286790095 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a8015da20 con 0x7f3a800cf160 2024-09-21T07:36:31.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.003+0000 7f3a86200640 1 -- 172.21.15.39:0/2286790095 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a8015dfe0 con 0x7f3a800cf160 2024-09-21T07:36:31.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.004+0000 7f3a86200640 1 -- 172.21.15.39:0/2286790095 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a80050f60 con 0x7f3a800cf160 2024-09-21T07:36:31.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.006+0000 7f3a77e00640 1 -- 172.21.15.39:0/2286790095 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3a70009b70 con 0x7f3a800cf160 2024-09-21T07:36:31.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.007+0000 7f3a77e00640 1 --2- 172.21.15.39:0/2286790095 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3a5c078bc0 0x7f3a5c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:31.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.007+0000 7f3a77e00640 1 -- 172.21.15.39:0/2286790095 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6800+0+0 (secure 0 0 0) 0x7f3a7009b460 con 0x7f3a800cf160 2024-09-21T07:36:31.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.007+0000 7f3a76a00640 1 --2- 172.21.15.39:0/2286790095 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3a5c078bc0 0x7f3a5c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:31.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.008+0000 7f3a76a00640 1 --2- 172.21.15.39:0/2286790095 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3a5c078bc0 0x7f3a5c07b0d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3a801efeb0 tx=0x7f3a68009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:31.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.011+0000 7f3a77e00640 1 -- 172.21.15.39:0/2286790095 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3a80050f60 con 0x7f3a800cf160 2024-09-21T07:36:31.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.254+0000 7f3a86200640 1 -- 172.21.15.39:0/2286790095 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) -- 0x7f3a800706e0 con 0x7f3a800cf160 2024-09-21T07:36:31.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.362+0000 7f3a77e00640 1 -- 172.21.15.39:0/2286790095 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v56) ==== 105+0+0 (secure 0 0 0) 0x7f3a800706e0 con 0x7f3a800cf160 2024-09-21T07:36:31.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.366+0000 7f3a86200640 1 -- 172.21.15.39:0/2286790095 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3a5c078bc0 msgr2=0x7f3a5c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:31.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.366+0000 7f3a86200640 1 --2- 172.21.15.39:0/2286790095 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3a5c078bc0 0x7f3a5c07b0d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3a801efeb0 tx=0x7f3a68009040 comp rx=0 tx=0).stop 2024-09-21T07:36:31.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.366+0000 7f3a86200640 1 -- 172.21.15.39:0/2286790095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a800cf160 msgr2=0x7f3a801f62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:31.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.366+0000 7f3a86200640 1 --2- 172.21.15.39:0/2286790095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a800cf160 0x7f3a801f62c0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f3a7000d780 tx=0x7f3a7000dc40 comp rx=0 tx=0).stop 2024-09-21T07:36:31.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.366+0000 7f3a86200640 1 -- 172.21.15.39:0/2286790095 shutdown_connections 2024-09-21T07:36:31.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.366+0000 7f3a86200640 1 --2- 172.21.15.39:0/2286790095 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3a5c078bc0 0x7f3a5c07b0d0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:31.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.366+0000 7f3a86200640 1 --2- 172.21.15.39:0/2286790095 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a8009d6c0 0x7f3a8014e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:31.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.366+0000 7f3a86200640 1 --2- 172.21.15.39:0/2286790095 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3a800cfb00 0x7f3a8015eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:31.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.366+0000 7f3a86200640 1 --2- 172.21.15.39:0/2286790095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a800cf160 0x7f3a801f62c0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:31.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.366+0000 7f3a86200640 1 -- 172.21.15.39:0/2286790095 >> 172.21.15.39:0/2286790095 conn(0x7f3a800b1cb0 msgr2=0x7f3a800b3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:31.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.366+0000 7f3a86200640 1 -- 172.21.15.39:0/2286790095 shutdown_connections 2024-09-21T07:36:31.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.366+0000 7f3a86200640 1 -- 172.21.15.39:0/2286790095 wait complete. 2024-09-21T07:36:31.394 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:36:31.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.620+0000 7ffab1a00640 1 -- 172.21.15.39:0/3534688640 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffaac0b54d0 msgr2=0x7ffaac0b58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:31.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.620+0000 7ffab1a00640 1 --2- 172.21.15.39:0/3534688640 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffaac0b54d0 0x7ffaac0b58e0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7ffaac071bd0 tx=0x7ffaa402aa10 comp rx=0 tx=0).stop 2024-09-21T07:36:31.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.620+0000 7ffab1a00640 1 -- 172.21.15.39:0/3534688640 shutdown_connections 2024-09-21T07:36:31.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.620+0000 7ffab1a00640 1 --2- 172.21.15.39:0/3534688640 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffaac0b54d0 0x7ffaac0b58e0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:31.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.620+0000 7ffab1a00640 1 --2- 172.21.15.39:0/3534688640 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffaac0735d0 0x7ffaac153510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:31.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.620+0000 7ffab1a00640 1 --2- 172.21.15.39:0/3534688640 >> v2:172.21.15.148:3300/0 conn(0x7ffaac0af700 0x7ffaac0caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:31.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.620+0000 7ffab1a00640 1 -- 172.21.15.39:0/3534688640 >> 172.21.15.39:0/3534688640 conn(0x7ffaac06d870 msgr2=0x7ffaac06ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:31.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.620+0000 7ffab1a00640 1 -- 172.21.15.39:0/3534688640 shutdown_connections 2024-09-21T07:36:31.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.620+0000 7ffab1a00640 1 -- 172.21.15.39:0/3534688640 wait complete. 2024-09-21T07:36:31.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.622+0000 7ffab1a00640 1 Processor -- start 2024-09-21T07:36:31.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.622+0000 7ffab1a00640 1 -- start start 2024-09-21T07:36:31.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.624+0000 7ffab1a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffaac0735d0 0x7ffaac140c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:31.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.624+0000 7ffab1a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffaac0af700 0x7ffaac15e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:31.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.624+0000 7ffab1a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffaac0b54d0 0x7ffaac15e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:31.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.624+0000 7ffab1a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ffaac15d1d0 con 0x7ffaac0af700 2024-09-21T07:36:31.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.624+0000 7ffab1a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ffaac15d050 con 0x7ffaac0b54d0 2024-09-21T07:36:31.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.624+0000 7ffab1a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7ffaac15d350 con 0x7ffaac0735d0 2024-09-21T07:36:31.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.624+0000 7ffaa3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffaac0af700 0x7ffaac15e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:31.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.624+0000 7ffaa3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffaac0af700 0x7ffaac15e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52408/0 (socket says 172.21.15.39:52408) 2024-09-21T07:36:31.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.624+0000 7ffaa3e00640 1 -- 172.21.15.39:0/2903921844 learned_addr learned my addr 172.21.15.39:0/2903921844 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:31.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.624+0000 7ffaabe00640 1 --2- 172.21.15.39:0/2903921844 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffaac0735d0 0x7ffaac140c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:31.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.624+0000 7ffab1000640 1 --2- 172.21.15.39:0/2903921844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffaac0b54d0 0x7ffaac15e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:31.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.625+0000 7ffaa3e00640 1 -- 172.21.15.39:0/2903921844 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffaac0735d0 msgr2=0x7ffaac140c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:31.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.625+0000 7ffaa3e00640 1 --2- 172.21.15.39:0/2903921844 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffaac0735d0 0x7ffaac140c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:31.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.625+0000 7ffaa3e00640 1 -- 172.21.15.39:0/2903921844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffaac0b54d0 msgr2=0x7ffaac15e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:31.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.625+0000 7ffaa3e00640 1 --2- 172.21.15.39:0/2903921844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffaac0b54d0 0x7ffaac15e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:31.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.625+0000 7ffaa3e00640 1 -- 172.21.15.39:0/2903921844 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaac141720 con 0x7ffaac0af700 2024-09-21T07:36:31.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.625+0000 7ffaa3e00640 1 --2- 172.21.15.39:0/2903921844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffaac0af700 0x7ffaac15e490 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ffa9400d690 tx=0x7ffa9400db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:31.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.625+0000 7ffaa8c00640 1 -- 172.21.15.39:0/2903921844 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa94017070 con 0x7ffaac0af700 2024-09-21T07:36:31.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.625+0000 7ffaa8c00640 1 -- 172.21.15.39:0/2903921844 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa94009b10 con 0x7ffaac0af700 2024-09-21T07:36:31.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.625+0000 7ffab1a00640 1 -- 172.21.15.39:0/2903921844 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaac141a00 con 0x7ffaac0af700 2024-09-21T07:36:31.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.625+0000 7ffaa8c00640 1 -- 172.21.15.39:0/2903921844 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa94010460 con 0x7ffaac0af700 2024-09-21T07:36:31.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.625+0000 7ffab1a00640 1 -- 172.21.15.39:0/2903921844 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaac1403b0 con 0x7ffaac0af700 2024-09-21T07:36:31.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.626+0000 7ffab1a00640 1 -- 172.21.15.39:0/2903921844 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaac15d870 con 0x7ffaac0af700 2024-09-21T07:36:31.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.628+0000 7ffaa8c00640 1 -- 172.21.15.39:0/2903921844 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ffa9401a070 con 0x7ffaac0af700 2024-09-21T07:36:31.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.629+0000 7ffaa8c00640 1 --2- 172.21.15.39:0/2903921844 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ffa88078b90 0x7ffa8807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:31.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.629+0000 7ffaa8c00640 1 -- 172.21.15.39:0/2903921844 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6800+0+0 (secure 0 0 0) 0x7ffa9409b150 con 0x7ffaac0af700 2024-09-21T07:36:31.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.629+0000 7ffaabe00640 1 --2- 172.21.15.39:0/2903921844 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ffa88078b90 0x7ffa8807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:31.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.630+0000 7ffaabe00640 1 --2- 172.21.15.39:0/2903921844 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ffa88078b90 0x7ffa8807b0a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c00bb50 tx=0x7ffa9c00b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:31.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.634+0000 7ffaa8c00640 1 -- 172.21.15.39:0/2903921844 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ffa94062ba0 con 0x7ffaac0af700 2024-09-21T07:36:31.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.869+0000 7ffab1a00640 1 -- 172.21.15.39:0/2903921844 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffaac0d0900 con 0x7ffaac0af700 2024-09-21T07:36:31.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.870+0000 7ffaa8c00640 1 -- 172.21.15.39:0/2903921844 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5241 (secure 0 0 0) 0x7ffa940675d0 con 0x7ffaac0af700 2024-09-21T07:36:31.871 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:36:31.871 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":56,"btime":"2024-09-21T07:36:31:360639+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5621,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3026827614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3026827614},{"type":"v1","addr":"172.21.15.148:6837","nonce":3026827614}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:36:31.360603+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6321,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:creating","state_seq":1,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6321":{"gid":6321,"name":"c","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/829716143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":829716143},{"type":"v1","addr":"172.21.15.39:6834","nonce":829716143}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6321,"qdb_cluster":[6321]},"id":6}]} 2024-09-21T07:36:31.871 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 56 2024-09-21T07:36:31.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.872+0000 7ffab1a00640 1 -- 172.21.15.39:0/2903921844 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ffa88078b90 msgr2=0x7ffa8807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:31.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.872+0000 7ffab1a00640 1 --2- 172.21.15.39:0/2903921844 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ffa88078b90 0x7ffa8807b0a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c00bb50 tx=0x7ffa9c00b2e0 comp rx=0 tx=0).stop 2024-09-21T07:36:31.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.872+0000 7ffab1a00640 1 -- 172.21.15.39:0/2903921844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffaac0af700 msgr2=0x7ffaac15e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:31.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.872+0000 7ffab1a00640 1 --2- 172.21.15.39:0/2903921844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffaac0af700 0x7ffaac15e490 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ffa9400d690 tx=0x7ffa9400db50 comp rx=0 tx=0).stop 2024-09-21T07:36:31.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.872+0000 7ffab1a00640 1 -- 172.21.15.39:0/2903921844 shutdown_connections 2024-09-21T07:36:31.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.872+0000 7ffab1a00640 1 --2- 172.21.15.39:0/2903921844 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ffa88078b90 0x7ffa8807b0a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:31.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.872+0000 7ffab1a00640 1 --2- 172.21.15.39:0/2903921844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffaac0b54d0 0x7ffaac15e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:31.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.872+0000 7ffab1a00640 1 --2- 172.21.15.39:0/2903921844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffaac0af700 0x7ffaac15e490 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:31.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.872+0000 7ffab1a00640 1 --2- 172.21.15.39:0/2903921844 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffaac0735d0 0x7ffaac140c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:31.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.872+0000 7ffab1a00640 1 -- 172.21.15.39:0/2903921844 >> 172.21.15.39:0/2903921844 conn(0x7ffaac06d870 msgr2=0x7ffaac06ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:31.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.872+0000 7ffab1a00640 1 -- 172.21.15.39:0/2903921844 shutdown_connections 2024-09-21T07:36:31.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:31.872+0000 7ffab1a00640 1 -- 172.21.15.39:0/2903921844 wait complete. 2024-09-21T07:36:32.303 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 56, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:36:20.189287+0000', 'modified': '2024-09-21T07:36:31.360603+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 6321, 'mds_1': 6115}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6115': {'gid': 6115, 'name': 'a', 'rank': 1, 'incarnation': 56, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.39:6835/364657398', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 364657398}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 364657398}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6321': {'gid': 6321, 'name': 'c', 'rank': 0, 'incarnation': 51, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.39:6834/829716143', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 829716143}, {'type': 'v1', 'addr': '172.21.15.39:6834', 'nonce': 829716143}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6321, 'qdb_cluster': [6321]} 2024-09-21T07:36:32.303 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6115:up:creating 2024-09-21T07:36:32.642 INFO:tasks.ceph.mds.c.smithi039.stderr:2024-09-21T07:36:32.641+0000 7f3c21c00640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:36:33.304 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:36:33.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.541+0000 7fdab2a00640 1 -- 172.21.15.39:0/1862647957 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdaac0d01f0 msgr2=0x7fdaac0ca710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:33.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.541+0000 7fdab2a00640 1 --2- 172.21.15.39:0/1862647957 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdaac0d01f0 0x7fdaac0ca710 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fdaac0ca550 tx=0x7fdaa402aac0 comp rx=0 tx=0).stop 2024-09-21T07:36:33.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.541+0000 7fdab2a00640 1 -- 172.21.15.39:0/1862647957 shutdown_connections 2024-09-21T07:36:33.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.541+0000 7fdab2a00640 1 --2- 172.21.15.39:0/1862647957 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdaac0d01f0 0x7fdaac0ca710 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:33.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.541+0000 7fdab2a00640 1 --2- 172.21.15.39:0/1862647957 >> v2:172.21.15.148:3300/0 conn(0x7fdaac073160 0x7fdaac0cfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:33.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.541+0000 7fdab2a00640 1 -- 172.21.15.39:0/1862647957 >> 172.21.15.39:0/1862647957 conn(0x7fdaac06d870 msgr2=0x7fdaac06ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:33.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.541+0000 7fdab2a00640 1 -- 172.21.15.39:0/1862647957 shutdown_connections 2024-09-21T07:36:33.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.541+0000 7fdab2a00640 1 -- 172.21.15.39:0/1862647957 wait complete. 2024-09-21T07:36:33.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.543+0000 7fdab2a00640 1 Processor -- start 2024-09-21T07:36:33.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.543+0000 7fdab2a00640 1 -- start start 2024-09-21T07:36:33.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.544+0000 7fdab2a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdaac073160 0x7fdaac15d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:33.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.545+0000 7fdab2a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdaac0d01f0 0x7fdaac144ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:33.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.545+0000 7fdab2a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdaac15e210 0x7fdaac15b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:33.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.545+0000 7fdab2a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac074c10 con 0x7fdaac073160 2024-09-21T07:36:33.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.545+0000 7fdab2a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac070fa0 con 0x7fdaac15e210 2024-09-21T07:36:33.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.545+0000 7fdab2a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fdaac0cac50 con 0x7fdaac0d01f0 2024-09-21T07:36:33.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.545+0000 7fdab1600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdaac073160 0x7fdaac15d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:33.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.545+0000 7fdab1600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdaac073160 0x7fdaac15d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50718/0 (socket says 172.21.15.39:50718) 2024-09-21T07:36:33.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.545+0000 7fdab1600640 1 -- 172.21.15.39:0/2376042243 learned_addr learned my addr 172.21.15.39:0/2376042243 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:33.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.545+0000 7fdab2000640 1 --2- 172.21.15.39:0/2376042243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdaac15e210 0x7fdaac15b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:33.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.545+0000 7fdab0c00640 1 --2- 172.21.15.39:0/2376042243 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdaac0d01f0 0x7fdaac144ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:33.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.545+0000 7fdab1600640 1 -- 172.21.15.39:0/2376042243 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdaac0d01f0 msgr2=0x7fdaac144ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:33.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.545+0000 7fdab1600640 1 --2- 172.21.15.39:0/2376042243 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdaac0d01f0 0x7fdaac144ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:33.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.545+0000 7fdab1600640 1 -- 172.21.15.39:0/2376042243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdaac15e210 msgr2=0x7fdaac15b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:33.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.545+0000 7fdab1600640 1 --2- 172.21.15.39:0/2376042243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdaac15e210 0x7fdaac15b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:33.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.545+0000 7fdab1600640 1 -- 172.21.15.39:0/2376042243 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaac152d00 con 0x7fdaac073160 2024-09-21T07:36:33.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.546+0000 7fdab1600640 1 --2- 172.21.15.39:0/2376042243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdaac073160 0x7fdaac15d2a0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000c3a0 tx=0x7fdaa000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:33.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.546+0000 7fdaa9600640 1 -- 172.21.15.39:0/2376042243 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0017070 con 0x7fdaac073160 2024-09-21T07:36:33.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.546+0000 7fdaa9600640 1 -- 172.21.15.39:0/2376042243 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdaa00053c0 con 0x7fdaac073160 2024-09-21T07:36:33.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.546+0000 7fdaa9600640 1 -- 172.21.15.39:0/2376042243 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0003850 con 0x7fdaac073160 2024-09-21T07:36:33.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.546+0000 7fdab2a00640 1 -- 172.21.15.39:0/2376042243 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaac152fe0 con 0x7fdaac073160 2024-09-21T07:36:33.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.546+0000 7fdab2a00640 1 -- 172.21.15.39:0/2376042243 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaac1535a0 con 0x7fdaac073160 2024-09-21T07:36:33.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.547+0000 7fdab2a00640 1 -- 172.21.15.39:0/2376042243 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaac051040 con 0x7fdaac073160 2024-09-21T07:36:33.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.548+0000 7fdaa9600640 1 -- 172.21.15.39:0/2376042243 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdaa001a070 con 0x7fdaac073160 2024-09-21T07:36:33.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.549+0000 7fdaa9600640 1 --2- 172.21.15.39:0/2376042243 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fda8c078b90 0x7fda8c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:33.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.549+0000 7fdaa9600640 1 -- 172.21.15.39:0/2376042243 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6800+0+0 (secure 0 0 0) 0x7fdaa009a5d0 con 0x7fdaac073160 2024-09-21T07:36:33.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.550+0000 7fdab0c00640 1 --2- 172.21.15.39:0/2376042243 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fda8c078b90 0x7fda8c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:33.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.550+0000 7fdab0c00640 1 --2- 172.21.15.39:0/2376042243 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fda8c078b90 0x7fda8c07b0a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fda98005fd0 tx=0x7fda98005850 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:33.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.554+0000 7fdaa9600640 1 -- 172.21.15.39:0/2376042243 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdaac051040 con 0x7fdaac073160 2024-09-21T07:36:33.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.793+0000 7fdab2a00640 1 -- 172.21.15.39:0/2376042243 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdaac156500 con 0x7fdaac073160 2024-09-21T07:36:33.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.794+0000 7fdaa9600640 1 -- 172.21.15.39:0/2376042243 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5244 (secure 0 0 0) 0x7fdaac0d0600 con 0x7fdaac073160 2024-09-21T07:36:33.795 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:36:33.795 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":57,"btime":"2024-09-21T07:36:32:363503+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5621,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3026827614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3026827614},{"type":"v1","addr":"172.21.15.148:6837","nonce":3026827614}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:36:32.363495+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6321,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6321":{"gid":6321,"name":"c","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/829716143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":829716143},{"type":"v1","addr":"172.21.15.39:6834","nonce":829716143}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6321,"qdb_cluster":[6115,6321]},"id":6}]} 2024-09-21T07:36:33.796 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 57 2024-09-21T07:36:33.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.796+0000 7fdab2a00640 1 -- 172.21.15.39:0/2376042243 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fda8c078b90 msgr2=0x7fda8c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:33.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.796+0000 7fdab2a00640 1 --2- 172.21.15.39:0/2376042243 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fda8c078b90 0x7fda8c07b0a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fda98005fd0 tx=0x7fda98005850 comp rx=0 tx=0).stop 2024-09-21T07:36:33.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.796+0000 7fdab2a00640 1 -- 172.21.15.39:0/2376042243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdaac073160 msgr2=0x7fdaac15d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:33.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.796+0000 7fdab2a00640 1 --2- 172.21.15.39:0/2376042243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdaac073160 0x7fdaac15d2a0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000c3a0 tx=0x7fdaa000c860 comp rx=0 tx=0).stop 2024-09-21T07:36:33.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.796+0000 7fdab2a00640 1 -- 172.21.15.39:0/2376042243 shutdown_connections 2024-09-21T07:36:33.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.796+0000 7fdab2a00640 1 --2- 172.21.15.39:0/2376042243 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fda8c078b90 0x7fda8c07b0a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:33.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.796+0000 7fdab2a00640 1 --2- 172.21.15.39:0/2376042243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdaac15e210 0x7fdaac15b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:33.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.796+0000 7fdab2a00640 1 --2- 172.21.15.39:0/2376042243 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdaac0d01f0 0x7fdaac144ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:33.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.796+0000 7fdab2a00640 1 --2- 172.21.15.39:0/2376042243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdaac073160 0x7fdaac15d2a0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:33.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.796+0000 7fdab2a00640 1 -- 172.21.15.39:0/2376042243 >> 172.21.15.39:0/2376042243 conn(0x7fdaac06d870 msgr2=0x7fdaac1569e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:33.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.797+0000 7fdab2a00640 1 -- 172.21.15.39:0/2376042243 shutdown_connections 2024-09-21T07:36:33.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:33.797+0000 7fdab2a00640 1 -- 172.21.15.39:0/2376042243 wait complete. 2024-09-21T07:36:34.222 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 57, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:36:20.189287+0000', 'modified': '2024-09-21T07:36:32.363495+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 6321, 'mds_1': 6115}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6115': {'gid': 6115, 'name': 'a', 'rank': 1, 'incarnation': 56, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6835/364657398', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 364657398}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 364657398}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6321': {'gid': 6321, 'name': 'c', 'rank': 0, 'incarnation': 51, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.39:6834/829716143', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 829716143}, {'type': 'v1', 'addr': '172.21.15.39:6834', 'nonce': 829716143}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6321, 'qdb_cluster': [6115, 6321]} 2024-09-21T07:36:34.222 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2024-09-21T07:36:34.222 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T07:36:34.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.452+0000 7f1842200640 1 -- 172.21.15.39:0/326495834 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1820004d90 msgr2=0x7f1820005240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:34.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.452+0000 7f1842200640 1 --2- 172.21.15.39:0/326495834 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1820004d90 0x7f1820005240 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f183400c610 tx=0x7f1820005830 comp rx=0 tx=0).stop 2024-09-21T07:36:34.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.452+0000 7f183be00640 1 -- 172.21.15.39:0/326495834 reap_dead start 2024-09-21T07:36:34.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.452+0000 7f1842200640 1 -- 172.21.15.39:0/326495834 shutdown_connections 2024-09-21T07:36:34.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.452+0000 7f1842200640 1 -- 172.21.15.39:0/326495834 >> 172.21.15.39:0/326495834 conn(0x7f183c06d7d0 msgr2=0x7f183c0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:34.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.453+0000 7f1842200640 1 -- 172.21.15.39:0/326495834 shutdown_connections 2024-09-21T07:36:34.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.453+0000 7f1842200640 1 -- 172.21.15.39:0/326495834 wait complete. 2024-09-21T07:36:34.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.455+0000 7f1842200640 1 Processor -- start 2024-09-21T07:36:34.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.455+0000 7f1842200640 1 -- start start 2024-09-21T07:36:34.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.456+0000 7f1842200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1820004d90 0x7f183c09b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:34.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.456+0000 7f1842200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f183c149bf0 0x7f183c0b0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:34.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.457+0000 7f1842200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f183c0b1460 0x7f183c15f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:34.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.457+0000 7f1842200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f183c0b44d0 con 0x7f1820004d90 2024-09-21T07:36:34.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.457+0000 7f1842200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f183c0b4350 con 0x7f183c0b1460 2024-09-21T07:36:34.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.457+0000 7f1842200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f183c0b4650 con 0x7f183c149bf0 2024-09-21T07:36:34.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.457+0000 7f183b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1820004d90 0x7f183c09b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:34.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.457+0000 7f183b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1820004d90 0x7f183c09b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50742/0 (socket says 172.21.15.39:50742) 2024-09-21T07:36:34.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.457+0000 7f183b400640 1 -- 172.21.15.39:0/3394716608 learned_addr learned my addr 172.21.15.39:0/3394716608 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:34.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.457+0000 7f183be00640 1 --2- 172.21.15.39:0/3394716608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f183c0b1460 0x7f183c15f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:34.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.457+0000 7f183b400640 1 -- 172.21.15.39:0/3394716608 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f183c149bf0 msgr2=0x7f183c0b0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:34.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.457+0000 7f1833e00640 1 --2- 172.21.15.39:0/3394716608 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f183c149bf0 0x7f183c0b0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:34.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.457+0000 7f183b400640 1 --2- 172.21.15.39:0/3394716608 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f183c149bf0 0x7f183c0b0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.457+0000 7f183b400640 1 -- 172.21.15.39:0/3394716608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f183c0b1460 msgr2=0x7f183c15f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:34.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.457+0000 7f183b400640 1 --2- 172.21.15.39:0/3394716608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f183c0b1460 0x7f183c15f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.457+0000 7f183b400640 1 -- 172.21.15.39:0/3394716608 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f183c0b1870 con 0x7f1820004d90 2024-09-21T07:36:34.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.457+0000 7f183be00640 1 --2- 172.21.15.39:0/3394716608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f183c0b1460 0x7f183c15f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:36:34.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.457+0000 7f1833e00640 1 --2- 172.21.15.39:0/3394716608 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f183c149bf0 0x7f183c0b0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:36:34.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.457+0000 7f183b400640 1 --2- 172.21.15.39:0/3394716608 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1820004d90 0x7f183c09b4b0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f182c004ed0 tx=0x7f182c001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:34.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.458+0000 7f1833400640 1 -- 172.21.15.39:0/3394716608 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182c015020 con 0x7f1820004d90 2024-09-21T07:36:34.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.458+0000 7f1833400640 1 -- 172.21.15.39:0/3394716608 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f182c010980 con 0x7f1820004d90 2024-09-21T07:36:34.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.458+0000 7f1842200640 1 -- 172.21.15.39:0/3394716608 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f183c15e400 con 0x7f1820004d90 2024-09-21T07:36:34.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.458+0000 7f1833400640 1 -- 172.21.15.39:0/3394716608 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182c010dd0 con 0x7f1820004d90 2024-09-21T07:36:34.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.458+0000 7f1842200640 1 -- 172.21.15.39:0/3394716608 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f183c15e9c0 con 0x7f1820004d90 2024-09-21T07:36:34.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.460+0000 7f1833400640 1 -- 172.21.15.39:0/3394716608 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f182c018020 con 0x7f1820004d90 2024-09-21T07:36:34.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.461+0000 7f1833400640 1 --2- 172.21.15.39:0/3394716608 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1818078b90 0x7f181807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:34.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.461+0000 7f1833400640 1 -- 172.21.15.39:0/3394716608 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6800+0+0 (secure 0 0 0) 0x7f182c09bdf0 con 0x7f1820004d90 2024-09-21T07:36:34.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.462+0000 7f1849203740 1 Processor -- start 2024-09-21T07:36:34.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.462+0000 7f1833e00640 1 --2- 172.21.15.39:0/3394716608 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1818078b90 0x7f181807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:34.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.462+0000 7f1849203740 1 -- start start 2024-09-21T07:36:34.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.463+0000 7f1833e00640 1 --2- 172.21.15.39:0/3394716608 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1818078b90 0x7f181807b0a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f183c070760 tx=0x7f1824005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:34.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.463+0000 7f1849203740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e47dfb2270 0x55e47e382970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:34.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.463+0000 7f1849203740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e47df893b0 0x55e47e382eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:34.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.463+0000 7f1849203740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e47df91590 0x55e47e352f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:34.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.463+0000 7f1849203740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55e47e2dc430 con 0x55e47df893b0 2024-09-21T07:36:34.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.463+0000 7f1849203740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e47e2cc5d0 con 0x55e47df91590 2024-09-21T07:36:34.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.463+0000 7f1849203740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55e47e2bba00 con 0x55e47dfb2270 2024-09-21T07:36:34.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.464+0000 7f1833e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e47df893b0 0x55e47e382eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:34.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.464+0000 7f1833e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e47df893b0 0x55e47e382eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50756/0 (socket says 172.21.15.39:50756) 2024-09-21T07:36:34.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.464+0000 7f1833e00640 1 -- 172.21.15.39:0/3916361275 learned_addr learned my addr 172.21.15.39:0/3916361275 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:34.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.464+0000 7f183b400640 1 --2- 172.21.15.39:0/3916361275 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e47dfb2270 0x55e47e382970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:34.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.464+0000 7f183be00640 1 --2- 172.21.15.39:0/3916361275 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e47df91590 0x55e47e352f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:34.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.464+0000 7f1833e00640 1 -- 172.21.15.39:0/3916361275 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e47dfb2270 msgr2=0x55e47e382970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:36:34.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.464+0000 7f1833e00640 1 --2- 172.21.15.39:0/3916361275 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e47dfb2270 0x55e47e382970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.464+0000 7f1833e00640 1 -- 172.21.15.39:0/3916361275 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e47df91590 msgr2=0x55e47e352f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:36:34.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.464+0000 7f1833e00640 1 --2- 172.21.15.39:0/3916361275 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e47df91590 0x55e47e352f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.464+0000 7f1833e00640 1 -- 172.21.15.39:0/3916361275 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e47e284520 con 0x55e47df893b0 2024-09-21T07:36:34.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.464+0000 7f1833e00640 1 --2- 172.21.15.39:0/3916361275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e47df893b0 0x55e47e382eb0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f1824013fd0 tx=0x7f1824016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1407ceddcca112 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:34.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.465+0000 7f182b400640 1 -- 172.21.15.39:0/3916361275 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182401c660 con 0x55e47df893b0 2024-09-21T07:36:34.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.465+0000 7f182b400640 1 -- 172.21.15.39:0/3916361275 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f182401c800 con 0x55e47df893b0 2024-09-21T07:36:34.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.465+0000 7f182b400640 1 -- 172.21.15.39:0/3916361275 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182401cbb0 con 0x55e47df893b0 2024-09-21T07:36:34.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.465+0000 7f1849203740 1 -- 172.21.15.39:0/3916361275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e47df893b0 msgr2=0x55e47e382eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:34.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.465+0000 7f1849203740 1 --2- 172.21.15.39:0/3916361275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e47df893b0 0x55e47e382eb0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f1824013fd0 tx=0x7f1824016000 comp rx=0 tx=0).stop 2024-09-21T07:36:34.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.466+0000 7f1849203740 1 -- 172.21.15.39:0/3916361275 shutdown_connections 2024-09-21T07:36:34.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.466+0000 7f1849203740 1 --2- 172.21.15.39:0/3916361275 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e47df91590 0x55e47e352f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.466+0000 7f1849203740 1 --2- 172.21.15.39:0/3916361275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e47df893b0 0x55e47e382eb0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.466+0000 7f1849203740 1 --2- 172.21.15.39:0/3916361275 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e47dfb2270 0x55e47e382970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.466+0000 7f1849203740 1 -- 172.21.15.39:0/3916361275 >> 172.21.15.39:0/3916361275 conn(0x55e47dfa63c0 msgr2=0x55e47e358970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:34.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.466+0000 7f1849203740 1 -- 172.21.15.39:0/3916361275 shutdown_connections 2024-09-21T07:36:34.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.466+0000 7f1849203740 1 -- 172.21.15.39:0/3916361275 wait complete. 2024-09-21T07:36:34.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.467+0000 7f1849203740 1 -- start start 2024-09-21T07:36:34.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.467+0000 7f1849203740 1 Processor -- start 2024-09-21T07:36:34.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.468+0000 7f1849203740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e47df893b0 0x55e47e3971b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:34.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.468+0000 7f1849203740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e47df91590 0x55e47e353c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:34.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.469+0000 7f1849203740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e47dfb2270 0x55e47e3541b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:34.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.469+0000 7f1849203740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55e47e262330 con 0x55e47dfb2270 2024-09-21T07:36:34.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.469+0000 7f1849203740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e47e2bf810 con 0x55e47df91590 2024-09-21T07:36:34.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.469+0000 7f1849203740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55e47e02c620 con 0x55e47df893b0 2024-09-21T07:36:34.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.469+0000 7f1849203740 10 client.0 _ll_register_callbacks cb 0x55e47e267420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:36:34.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.469+0000 7f183be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e47dfb2270 0x55e47e3541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:34.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.469+0000 7f183be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e47dfb2270 0x55e47e3541b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50762/0 (socket says 172.21.15.39:50762) 2024-09-21T07:36:34.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.469+0000 7f183be00640 1 -- 172.21.15.39:0/1491312069 learned_addr learned my addr 172.21.15.39:0/1491312069 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:34.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.469+0000 7f1833e00640 1 --2- 172.21.15.39:0/1491312069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e47df91590 0x55e47e353c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:34.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.469+0000 7f183b400640 1 --2- 172.21.15.39:0/1491312069 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e47df893b0 0x55e47e3971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:34.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.469+0000 7f183be00640 1 -- 172.21.15.39:0/1491312069 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e47df893b0 msgr2=0x55e47e3971b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:36:34.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.469+0000 7f183be00640 1 --2- 172.21.15.39:0/1491312069 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e47df893b0 0x55e47e3971b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.469+0000 7f183be00640 1 -- 172.21.15.39:0/1491312069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e47df91590 msgr2=0x55e47e353c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:36:34.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.469+0000 7f183be00640 1 --2- 172.21.15.39:0/1491312069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e47df91590 0x55e47e353c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.469+0000 7f183be00640 1 -- 172.21.15.39:0/1491312069 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e47e01d850 con 0x55e47dfb2270 2024-09-21T07:36:34.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.469+0000 7f183b400640 1 --2- 172.21.15.39:0/1491312069 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e47df893b0 0x55e47e3971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:36:34.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.470+0000 7f183be00640 1 --2- 172.21.15.39:0/1491312069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e47dfb2270 0x55e47e3541b0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f183402b3d0 tx=0x7f1834001500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66a580905cdcc16 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:34.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.470+0000 7f181fe00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:36:34.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.470+0000 7f181fe00640 1 -- 172.21.15.39:0/1491312069 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1834003d80 con 0x55e47dfb2270 2024-09-21T07:36:34.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.470+0000 7f181fe00640 1 -- 172.21.15.39:0/1491312069 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 6800+0+0 (secure 0 0 0) 0x7f183402b720 con 0x55e47dfb2270 2024-09-21T07:36:34.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.470+0000 7f1849203740 1 -- 172.21.15.39:0/1491312069 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f182401c800 con 0x55e47dfb2270 2024-09-21T07:36:34.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.470+0000 7f181fe00640 1 -- 172.21.15.39:0/1491312069 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f1834046050 con 0x55e47dfb2270 2024-09-21T07:36:34.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.470+0000 7f1849203740 10 client.6483 fetch_fsmap learned FSMap version 57 2024-09-21T07:36:34.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.470+0000 7f1849203740 1 -- 172.21.15.39:0/1491312069 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) -- 0x55e47e0946b0 con 0x55e47dfb2270 2024-09-21T07:36:34.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.470+0000 7f181fe00640 1 -- 172.21.15.39:0/1491312069 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f182401c800 con 0x55e47dfb2270 2024-09-21T07:36:34.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.470+0000 7f181fe00640 1 -- 172.21.15.39:0/1491312069 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 57) ==== 3522+0+0 (secure 0 0 0) 0x7f1834049020 con 0x55e47dfb2270 2024-09-21T07:36:34.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.470+0000 7f1849203740 10 client.6483 fetch_fsmap finished waiting for FSMap version 57 2024-09-21T07:36:34.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.471+0000 7f1849203740 10 client.6483 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:36:34.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.471+0000 7f1849203740 20 client.6483 populate_metadata read hostname 'smithi039' 2024-09-21T07:36:34.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.471+0000 7f1849203740 1 --2- 172.21.15.39:0/1491312069 >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x55e47df70e40 0x55e47e395bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:34.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.471+0000 7f1849203740 4 client.6483 mds_command: new command op to 6115 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:36:34.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.471+0000 7f1849203740 1 -- 172.21.15.39:0/1491312069 --> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e47e2ce520 con 0x55e47df70e40 2024-09-21T07:36:34.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.471+0000 7f183b400640 1 --2- 172.21.15.39:0/1491312069 >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x55e47df70e40 0x55e47e395bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:34.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.471+0000 7f181fe00640 10 client.6483 ms_handle_connect on v2:172.21.15.39:6833/364657398 2024-09-21T07:36:34.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.471+0000 7f183b400640 1 --2- 172.21.15.39:0/1491312069 >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x55e47df70e40 0x55e47e395bd0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4110f7f67dcc1a18 server_cookie=da1f5e0f1a368f36 in_seq=0 out_seq=0 2024-09-21T07:36:34.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.472+0000 7f181fe00640 1 -- 172.21.15.39:0/1491312069 <== mds.1 v2:172.21.15.39:6833/364657398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e47e2ce520 con 0x55e47df70e40 2024-09-21T07:36:34.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.472+0000 7f181fe00640 10 client.6483 handle_command_reply: tid=0 2024-09-21T07:36:34.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.472+0000 7f1849203740 1 client.6483 shutdown 2024-09-21T07:36:34.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.472+0000 7f1849203740 1 -- 172.21.15.39:0/1491312069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e47dfb2270 msgr2=0x55e47e3541b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:34.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.472+0000 7f1849203740 1 --2- 172.21.15.39:0/1491312069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e47dfb2270 0x55e47e3541b0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f183402b3d0 tx=0x7f1834001500 comp rx=0 tx=0).stop 2024-09-21T07:36:34.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.472+0000 7f1849203740 1 -- 172.21.15.39:0/1491312069 shutdown_connections 2024-09-21T07:36:34.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.472+0000 7f1849203740 1 --2- 172.21.15.39:0/1491312069 >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x55e47df70e40 0x55e47e395bd0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.472+0000 7f1849203740 1 --2- 172.21.15.39:0/1491312069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e47dfb2270 0x55e47e3541b0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.472+0000 7f1849203740 1 --2- 172.21.15.39:0/1491312069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e47df91590 0x55e47e353c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.473+0000 7f1849203740 1 --2- 172.21.15.39:0/1491312069 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e47df893b0 0x55e47e3971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.473+0000 7f181fe00640 0 client.6483 ms_handle_reset on v2:172.21.15.39:6833/364657398 2024-09-21T07:36:34.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.473+0000 7f1849203740 1 -- 172.21.15.39:0/1491312069 >> 172.21.15.39:0/1491312069 conn(0x55e47dfa63c0 msgr2=0x55e47e39f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:34.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.473+0000 7f1849203740 1 -- 172.21.15.39:0/1491312069 shutdown_connections 2024-09-21T07:36:34.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.473+0000 7f1849203740 1 -- 172.21.15.39:0/1491312069 wait complete. 2024-09-21T07:36:34.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.473+0000 7f1849203740 20 client.6483 trim_cache size 0 max 16384 2024-09-21T07:36:34.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.492+0000 7f1849203740 1 Processor -- start 2024-09-21T07:36:34.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.492+0000 7f1849203740 1 -- start start 2024-09-21T07:36:34.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.494+0000 7f1849203740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e47df893b0 0x55e47e395bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:34.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.494+0000 7f1849203740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e47df91590 0x55e47e3b6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:34.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.494+0000 7f1849203740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e47dfb2270 0x55e47e3b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:34.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.494+0000 7f1849203740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f183c204de0 con 0x55e47dfb2270 2024-09-21T07:36:34.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.494+0000 7f1849203740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f183c203130 con 0x55e47df91590 2024-09-21T07:36:34.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.494+0000 7f1849203740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f183c203780 con 0x55e47df893b0 2024-09-21T07:36:34.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.494+0000 7f183be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e47dfb2270 0x55e47e3b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:34.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.494+0000 7f183be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e47dfb2270 0x55e47e3b72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50774/0 (socket says 172.21.15.39:50774) 2024-09-21T07:36:34.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.494+0000 7f183be00640 1 -- 172.21.15.39:0/1839222449 learned_addr learned my addr 172.21.15.39:0/1839222449 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:34.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.494+0000 7f183be00640 1 -- 172.21.15.39:0/1839222449 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e47df893b0 msgr2=0x55e47e395bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:36:34.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.494+0000 7f183be00640 1 --2- 172.21.15.39:0/1839222449 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e47df893b0 0x55e47e395bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.494+0000 7f183be00640 1 -- 172.21.15.39:0/1839222449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e47df91590 msgr2=0x55e47e3b6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:36:34.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.494+0000 7f183be00640 1 --2- 172.21.15.39:0/1839222449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e47df91590 0x55e47e3b6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.494+0000 7f183be00640 1 -- 172.21.15.39:0/1839222449 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e47e3804c0 con 0x55e47dfb2270 2024-09-21T07:36:34.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.494+0000 7f183be00640 1 --2- 172.21.15.39:0/1839222449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e47dfb2270 0x55e47e3b72b0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f183402b8b0 tx=0x7f183402b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b00204b5e9055ac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:34.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.495+0000 7f182aa00640 1 -- 172.21.15.39:0/1839222449 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1834043070 con 0x55e47dfb2270 2024-09-21T07:36:34.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.495+0000 7f182aa00640 1 -- 172.21.15.39:0/1839222449 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f183403a070 con 0x55e47dfb2270 2024-09-21T07:36:34.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.495+0000 7f182aa00640 1 -- 172.21.15.39:0/1839222449 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183402be10 con 0x55e47dfb2270 2024-09-21T07:36:34.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.495+0000 7f1849203740 1 -- 172.21.15.39:0/1839222449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e47dfb2270 msgr2=0x55e47e3b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:34.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.495+0000 7f1849203740 1 --2- 172.21.15.39:0/1839222449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e47dfb2270 0x55e47e3b72b0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f183402b8b0 tx=0x7f183402b8e0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.496+0000 7f1849203740 1 -- 172.21.15.39:0/1839222449 shutdown_connections 2024-09-21T07:36:34.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.496+0000 7f1849203740 1 --2- 172.21.15.39:0/1839222449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e47dfb2270 0x55e47e3b72b0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.496+0000 7f1849203740 1 --2- 172.21.15.39:0/1839222449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e47df91590 0x55e47e3b6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.496+0000 7f1849203740 1 --2- 172.21.15.39:0/1839222449 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e47df893b0 0x55e47e395bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.496+0000 7f1849203740 1 -- 172.21.15.39:0/1839222449 >> 172.21.15.39:0/1839222449 conn(0x55e47dfa63c0 msgr2=0x55e47e3a0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:34.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.496+0000 7f1849203740 1 -- 172.21.15.39:0/1839222449 shutdown_connections 2024-09-21T07:36:34.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.496+0000 7f1849203740 1 -- 172.21.15.39:0/1839222449 wait complete. 2024-09-21T07:36:34.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.497+0000 7f1849203740 1 -- start start 2024-09-21T07:36:34.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.497+0000 7f1849203740 1 Processor -- start 2024-09-21T07:36:34.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.498+0000 7f1849203740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e47df893b0 0x55e47e3b6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:34.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.498+0000 7f1849203740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e47df91590 0x55e47e3b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:34.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.498+0000 7f1849203740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e47dfb2270 0x55e47e41d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:34.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.498+0000 7f1849203740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55e47e37afe0 con 0x55e47dfb2270 2024-09-21T07:36:34.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.498+0000 7f1849203740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e47e37f9b0 con 0x55e47df91590 2024-09-21T07:36:34.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.498+0000 7f1849203740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55e47e37b630 con 0x55e47df893b0 2024-09-21T07:36:34.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.498+0000 7f1849203740 10 client.0 _ll_register_callbacks cb 0x7f1820003880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:36:34.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.499+0000 7f183be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e47dfb2270 0x55e47e41d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:34.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.499+0000 7f183b400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e47df893b0 0x55e47e3b6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:34.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.499+0000 7f183be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e47dfb2270 0x55e47e41d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50788/0 (socket says 172.21.15.39:50788) 2024-09-21T07:36:34.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.499+0000 7f183be00640 1 -- 172.21.15.39:0/1880649817 learned_addr learned my addr 172.21.15.39:0/1880649817 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:34.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.499+0000 7f1833e00640 1 --2- 172.21.15.39:0/1880649817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e47df91590 0x55e47e3b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:34.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.499+0000 7f183be00640 1 -- 172.21.15.39:0/1880649817 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e47df893b0 msgr2=0x55e47e3b6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:36:34.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.499+0000 7f183be00640 1 --2- 172.21.15.39:0/1880649817 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e47df893b0 0x55e47e3b6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.499+0000 7f183be00640 1 -- 172.21.15.39:0/1880649817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e47df91590 msgr2=0x55e47e3b72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:36:34.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.499+0000 7f183be00640 1 --2- 172.21.15.39:0/1880649817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e47df91590 0x55e47e3b72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.499+0000 7f183be00640 1 -- 172.21.15.39:0/1880649817 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e47e359650 con 0x55e47dfb2270 2024-09-21T07:36:34.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.499+0000 7f1833e00640 1 --2- 172.21.15.39:0/1880649817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e47df91590 0x55e47e3b72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:36:34.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.499+0000 7f183be00640 1 --2- 172.21.15.39:0/1880649817 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e47dfb2270 0x55e47e41d830 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f183400c610 tx=0x7f18340062f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9931d700c971dd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:34.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.499+0000 7f181fe00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:36:34.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.500+0000 7f181fe00640 1 -- 172.21.15.39:0/1880649817 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183405a060 con 0x55e47dfb2270 2024-09-21T07:36:34.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.500+0000 7f181fe00640 1 -- 172.21.15.39:0/1880649817 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 6800+0+0 (secure 0 0 0) 0x7f1834006670 con 0x55e47dfb2270 2024-09-21T07:36:34.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.500+0000 7f1849203740 1 -- 172.21.15.39:0/1880649817 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f183403a070 con 0x55e47dfb2270 2024-09-21T07:36:34.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.500+0000 7f181fe00640 1 -- 172.21.15.39:0/1880649817 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f1834060050 con 0x55e47dfb2270 2024-09-21T07:36:34.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.500+0000 7f1849203740 10 client.6489 fetch_fsmap learned FSMap version 57 2024-09-21T07:36:34.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.500+0000 7f1849203740 1 -- 172.21.15.39:0/1880649817 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) -- 0x55e47e3c10f0 con 0x55e47dfb2270 2024-09-21T07:36:34.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.500+0000 7f181fe00640 1 -- 172.21.15.39:0/1880649817 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f183403a070 con 0x55e47dfb2270 2024-09-21T07:36:34.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.500+0000 7f181fe00640 1 -- 172.21.15.39:0/1880649817 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 57) ==== 3522+0+0 (secure 0 0 0) 0x7f1834063020 con 0x55e47dfb2270 2024-09-21T07:36:34.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.500+0000 7f1849203740 10 client.6489 fetch_fsmap finished waiting for FSMap version 57 2024-09-21T07:36:34.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.500+0000 7f1849203740 10 client.6489 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:36:34.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.500+0000 7f1849203740 20 client.6489 populate_metadata read hostname 'smithi039' 2024-09-21T07:36:34.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.500+0000 7f1849203740 1 --2- 172.21.15.39:0/1880649817 >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x55e47e41e020 0x55e47e3bfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:34.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.500+0000 7f1849203740 4 client.6489 mds_command: new command op to 6115 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:36:34.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.500+0000 7f1849203740 1 -- 172.21.15.39:0/1880649817 --> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] -- command(tid 0: {"prefix": "status"}) -- 0x55e47e2c9fd0 con 0x55e47e41e020 2024-09-21T07:36:34.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.501+0000 7f183b400640 1 --2- 172.21.15.39:0/1880649817 >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x55e47e41e020 0x55e47e3bfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:34.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.501+0000 7f183b400640 1 --2- 172.21.15.39:0/1880649817 >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x55e47e41e020 0x55e47e3bfc20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f17d9e2e16450d24 server_cookie=5850509a2077bd59 in_seq=0 out_seq=0 2024-09-21T07:36:34.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.501+0000 7f181fe00640 10 client.6489 ms_handle_connect on v2:172.21.15.39:6833/364657398 2024-09-21T07:36:34.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.501+0000 7f181fe00640 1 -- 172.21.15.39:0/1880649817 <== mds.1 v2:172.21.15.39:6833/364657398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+323 (crc 0 0 0) 0x55e47e2c9fd0 con 0x55e47e41e020 2024-09-21T07:36:34.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.501+0000 7f181fe00640 10 client.6489 handle_command_reply: tid=0 2024-09-21T07:36:34.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.501+0000 7f1849203740 1 client.6489 shutdown 2024-09-21T07:36:34.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.502+0000 7f1849203740 1 -- 172.21.15.39:0/1880649817 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e47dfb2270 msgr2=0x55e47e41d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:34.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.502+0000 7f1849203740 1 --2- 172.21.15.39:0/1880649817 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e47dfb2270 0x55e47e41d830 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f183400c610 tx=0x7f18340062f0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.502+0000 7f1849203740 1 -- 172.21.15.39:0/1880649817 shutdown_connections 2024-09-21T07:36:34.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.502+0000 7f1849203740 1 --2- 172.21.15.39:0/1880649817 >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x55e47e41e020 0x55e47e3bfc20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.502+0000 7f1849203740 1 --2- 172.21.15.39:0/1880649817 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e47dfb2270 0x55e47e41d830 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.502+0000 7f1849203740 1 --2- 172.21.15.39:0/1880649817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e47df91590 0x55e47e3b72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.502+0000 7f1849203740 1 --2- 172.21.15.39:0/1880649817 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e47df893b0 0x55e47e3b6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.502+0000 7f181fe00640 0 client.6489 ms_handle_reset on v2:172.21.15.39:6833/364657398 2024-09-21T07:36:34.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.502+0000 7f1849203740 1 -- 172.21.15.39:0/1880649817 >> 172.21.15.39:0/1880649817 conn(0x55e47dfa63c0 msgr2=0x55e47e39de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:34.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.502+0000 7f1849203740 1 -- 172.21.15.39:0/1880649817 shutdown_connections 2024-09-21T07:36:34.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.502+0000 7f1849203740 1 -- 172.21.15.39:0/1880649817 wait complete. 2024-09-21T07:36:34.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.502+0000 7f1849203740 20 client.6489 trim_cache size 0 max 16384 2024-09-21T07:36:34.506 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:36:34.506 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:36:34.506 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 1, 2024-09-21T07:36:34.507 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 6115, 2024-09-21T07:36:34.507 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:36:34.507 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:36:34.507 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:36:34.507 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 3.13747577, 2024-09-21T07:36:34.507 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 57, 2024-09-21T07:36:34.507 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 90, 2024-09-21T07:36:34.507 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 90, 2024-09-21T07:36:34.507 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 18.812157814999999 2024-09-21T07:36:34.507 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:36:34.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.503+0000 7f1842200640 1 -- 172.21.15.39:0/3394716608 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1818078b90 msgr2=0x7f181807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:34.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.503+0000 7f1842200640 1 --2- 172.21.15.39:0/3394716608 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1818078b90 0x7f181807b0a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f183c070760 tx=0x7f1824005f90 comp rx=0 tx=0).stop 2024-09-21T07:36:34.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.503+0000 7f1842200640 1 -- 172.21.15.39:0/3394716608 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1820004d90 msgr2=0x7f183c09b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:34.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.503+0000 7f1842200640 1 --2- 172.21.15.39:0/3394716608 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1820004d90 0x7f183c09b4b0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f182c004ed0 tx=0x7f182c001800 comp rx=0 tx=0).stop 2024-09-21T07:36:34.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.503+0000 7f1842200640 1 -- 172.21.15.39:0/3394716608 shutdown_connections 2024-09-21T07:36:34.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.503+0000 7f1842200640 1 --2- 172.21.15.39:0/3394716608 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1818078b90 0x7f181807b0a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.503+0000 7f1842200640 1 --2- 172.21.15.39:0/3394716608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f183c0b1460 0x7f183c15f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.503+0000 7f1842200640 1 --2- 172.21.15.39:0/3394716608 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f183c149bf0 0x7f183c0b0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.503+0000 7f1842200640 1 --2- 172.21.15.39:0/3394716608 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1820004d90 0x7f183c09b4b0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.503+0000 7f1842200640 1 -- 172.21.15.39:0/3394716608 >> 172.21.15.39:0/3394716608 conn(0x7f183c06d7d0 msgr2=0x7f183c0d0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:34.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.503+0000 7f1842200640 1 -- 172.21.15.39:0/3394716608 shutdown_connections 2024-09-21T07:36:34.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.503+0000 7f1842200640 1 -- 172.21.15.39:0/3394716608 wait complete. 2024-09-21T07:36:34.528 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T07:36:34.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.757+0000 7ff71e400640 1 -- 172.21.15.39:0/4236586972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7180b9bd0 msgr2=0x7ff7180b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:34.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.758+0000 7ff71e400640 1 --2- 172.21.15.39:0/4236586972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7180b9bd0 0x7ff7180b71a0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ff70c005fd0 tx=0x7ff70c02a540 comp rx=0 tx=0).stop 2024-09-21T07:36:34.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.758+0000 7ff71e400640 1 -- 172.21.15.39:0/4236586972 shutdown_connections 2024-09-21T07:36:34.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.758+0000 7ff71e400640 1 --2- 172.21.15.39:0/4236586972 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff7180b8fd0 0x7ff7180b93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.758+0000 7ff71e400640 1 --2- 172.21.15.39:0/4236586972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7180b9bd0 0x7ff7180b71a0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.758+0000 7ff71e400640 1 -- 172.21.15.39:0/4236586972 >> 172.21.15.39:0/4236586972 conn(0x7ff71809d6c0 msgr2=0x7ff7180d4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:34.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.758+0000 7ff71e400640 1 -- 172.21.15.39:0/4236586972 shutdown_connections 2024-09-21T07:36:34.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.758+0000 7ff71e400640 1 -- 172.21.15.39:0/4236586972 wait complete. 2024-09-21T07:36:34.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.760+0000 7ff71e400640 1 Processor -- start 2024-09-21T07:36:34.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.760+0000 7ff71e400640 1 -- start start 2024-09-21T07:36:34.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.762+0000 7ff71e400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7180b8fd0 0x7ff7181525a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:34.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.762+0000 7ff71e400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff7180b9bd0 0x7ff7180d0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:34.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.762+0000 7ff71e400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff718162440 0x7ff7180d07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:34.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.762+0000 7ff71e400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff7180b5360 con 0x7ff7180b8fd0 2024-09-21T07:36:34.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.762+0000 7ff71e400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff7180b51e0 con 0x7ff718162440 2024-09-21T07:36:34.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.762+0000 7ff71e400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7ff7180b54e0 con 0x7ff7180b9bd0 2024-09-21T07:36:34.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.762+0000 7ff71d000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7180b8fd0 0x7ff7181525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:34.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.762+0000 7ff71d000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7180b8fd0 0x7ff7181525a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50816/0 (socket says 172.21.15.39:50816) 2024-09-21T07:36:34.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.762+0000 7ff71d000640 1 -- 172.21.15.39:0/3005309854 learned_addr learned my addr 172.21.15.39:0/3005309854 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:34.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.762+0000 7ff71da00640 1 --2- 172.21.15.39:0/3005309854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff718162440 0x7ff7180d07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:34.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.762+0000 7ff717e00640 1 --2- 172.21.15.39:0/3005309854 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff7180b9bd0 0x7ff7180d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:34.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.762+0000 7ff71d000640 1 -- 172.21.15.39:0/3005309854 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff7180b9bd0 msgr2=0x7ff7180d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:34.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.763+0000 7ff71d000640 1 --2- 172.21.15.39:0/3005309854 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff7180b9bd0 0x7ff7180d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.763+0000 7ff71d000640 1 -- 172.21.15.39:0/3005309854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff718162440 msgr2=0x7ff7180d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:34.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.763+0000 7ff71d000640 1 --2- 172.21.15.39:0/3005309854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff718162440 0x7ff7180d07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.763+0000 7ff71d000640 1 -- 172.21.15.39:0/3005309854 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff71815c0b0 con 0x7ff7180b8fd0 2024-09-21T07:36:34.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.763+0000 7ff717e00640 1 --2- 172.21.15.39:0/3005309854 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff7180b9bd0 0x7ff7180d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:36:34.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.763+0000 7ff71da00640 1 --2- 172.21.15.39:0/3005309854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff718162440 0x7ff7180d07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:36:34.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.763+0000 7ff71d000640 1 --2- 172.21.15.39:0/3005309854 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7180b8fd0 0x7ff7181525a0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff70c034040 tx=0x7ff70c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:34.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.763+0000 7ff714c00640 1 -- 172.21.15.39:0/3005309854 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70c03e030 con 0x7ff7180b8fd0 2024-09-21T07:36:34.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.763+0000 7ff714c00640 1 -- 172.21.15.39:0/3005309854 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff70c003d90 con 0x7ff7180b8fd0 2024-09-21T07:36:34.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.763+0000 7ff71e400640 1 -- 172.21.15.39:0/3005309854 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7181f9340 con 0x7ff7180b8fd0 2024-09-21T07:36:34.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.763+0000 7ff714c00640 1 -- 172.21.15.39:0/3005309854 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70c035040 con 0x7ff7180b8fd0 2024-09-21T07:36:34.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.763+0000 7ff71e400640 1 -- 172.21.15.39:0/3005309854 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7181f9900 con 0x7ff7180b8fd0 2024-09-21T07:36:34.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.766+0000 7ff714c00640 1 -- 172.21.15.39:0/3005309854 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff70c041030 con 0x7ff7180b8fd0 2024-09-21T07:36:34.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.766+0000 7ff714c00640 1 --2- 172.21.15.39:0/3005309854 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff6f8078b90 0x7ff6f807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:34.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.767+0000 7ff714c00640 1 -- 172.21.15.39:0/3005309854 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6800+0+0 (secure 0 0 0) 0x7ff70c0bb8c0 con 0x7ff7180b8fd0 2024-09-21T07:36:34.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.767+0000 7ff717e00640 1 --2- 172.21.15.39:0/3005309854 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff6f8078b90 0x7ff6f807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:34.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.768+0000 7ff726c4d740 1 Processor -- start 2024-09-21T07:36:34.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.768+0000 7ff726c4d740 1 -- start start 2024-09-21T07:36:34.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.768+0000 7ff717e00640 1 --2- 172.21.15.39:0/3005309854 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff6f8078b90 0x7ff6f807b0a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff704007fd0 tx=0x7ff704009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:34.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.769+0000 7ff726c4d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b19895f270 0x55b198d2fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:34.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.769+0000 7ff726c4d740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b1989363b0 0x55b198d300a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:34.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.769+0000 7ff726c4d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b19893e590 0x55b198d00150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:34.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.769+0000 7ff726c4d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b198c89430 con 0x55b19895f270 2024-09-21T07:36:34.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.769+0000 7ff726c4d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b198c795d0 con 0x55b19893e590 2024-09-21T07:36:34.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.769+0000 7ff726c4d740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55b198c68a00 con 0x55b1989363b0 2024-09-21T07:36:34.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.769+0000 7ff71d000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b19895f270 0x55b198d2fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:34.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.770+0000 7ff71d000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b19895f270 0x55b198d2fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50822/0 (socket says 172.21.15.39:50822) 2024-09-21T07:36:34.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.770+0000 7ff71d000640 1 -- 172.21.15.39:0/1023517037 learned_addr learned my addr 172.21.15.39:0/1023517037 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:34.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.770+0000 7ff71da00640 1 --2- 172.21.15.39:0/1023517037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b19893e590 0x55b198d00150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:34.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.770+0000 7ff717e00640 1 --2- 172.21.15.39:0/1023517037 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b1989363b0 0x55b198d300a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:34.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.770+0000 7ff71d000640 1 -- 172.21.15.39:0/1023517037 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b1989363b0 msgr2=0x55b198d300a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:36:34.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.770+0000 7ff71d000640 1 --2- 172.21.15.39:0/1023517037 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b1989363b0 0x55b198d300a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.770+0000 7ff71d000640 1 -- 172.21.15.39:0/1023517037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b19893e590 msgr2=0x55b198d00150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:36:34.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.770+0000 7ff71d000640 1 --2- 172.21.15.39:0/1023517037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b19893e590 0x55b198d00150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.770+0000 7ff71d000640 1 -- 172.21.15.39:0/1023517037 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b198c31520 con 0x55b19895f270 2024-09-21T07:36:34.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.770+0000 7ff717e00640 1 --2- 172.21.15.39:0/1023517037 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b1989363b0 0x55b198d300a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:36:34.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.770+0000 7ff71d000640 1 --2- 172.21.15.39:0/1023517037 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b19895f270 0x55b198d2fb60 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff70c0b9920 tx=0x7ff70c082b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=969d1b65ee718385 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:34.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.770+0000 7ff709600640 1 -- 172.21.15.39:0/1023517037 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70c048030 con 0x55b19895f270 2024-09-21T07:36:34.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.771+0000 7ff709600640 1 -- 172.21.15.39:0/1023517037 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff70c0838c0 con 0x55b19895f270 2024-09-21T07:36:34.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.771+0000 7ff709600640 1 -- 172.21.15.39:0/1023517037 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70c08c950 con 0x55b19895f270 2024-09-21T07:36:34.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.771+0000 7ff726c4d740 1 -- 172.21.15.39:0/1023517037 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b19895f270 msgr2=0x55b198d2fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:34.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.771+0000 7ff726c4d740 1 --2- 172.21.15.39:0/1023517037 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b19895f270 0x55b198d2fb60 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff70c0b9920 tx=0x7ff70c082b10 comp rx=0 tx=0).stop 2024-09-21T07:36:34.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.771+0000 7ff726c4d740 1 -- 172.21.15.39:0/1023517037 shutdown_connections 2024-09-21T07:36:34.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.771+0000 7ff726c4d740 1 --2- 172.21.15.39:0/1023517037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b19893e590 0x55b198d00150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.771+0000 7ff726c4d740 1 --2- 172.21.15.39:0/1023517037 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b1989363b0 0x55b198d300a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.771+0000 7ff726c4d740 1 --2- 172.21.15.39:0/1023517037 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b19895f270 0x55b198d2fb60 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.771+0000 7ff726c4d740 1 -- 172.21.15.39:0/1023517037 >> 172.21.15.39:0/1023517037 conn(0x55b1989533c0 msgr2=0x55b198d05b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:34.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.771+0000 7ff726c4d740 1 -- 172.21.15.39:0/1023517037 shutdown_connections 2024-09-21T07:36:34.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.772+0000 7ff726c4d740 1 -- 172.21.15.39:0/1023517037 wait complete. 2024-09-21T07:36:34.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.773+0000 7ff726c4d740 1 -- start start 2024-09-21T07:36:34.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.773+0000 7ff726c4d740 1 Processor -- start 2024-09-21T07:36:34.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.774+0000 7ff726c4d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1989363b0 0x55b198d44690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:34.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.774+0000 7ff726c4d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b19893e590 0x55b198d25f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:34.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.774+0000 7ff726c4d740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b19895f270 0x55b198d5ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:34.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.774+0000 7ff726c4d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b198c0f330 con 0x55b19893e590 2024-09-21T07:36:34.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.774+0000 7ff726c4d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b198c6c810 con 0x55b1989363b0 2024-09-21T07:36:34.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.774+0000 7ff726c4d740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55b1989d9620 con 0x55b19895f270 2024-09-21T07:36:34.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.774+0000 7ff726c4d740 10 client.0 _ll_register_callbacks cb 0x55b198c14420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:36:34.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.775+0000 7ff717e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b19893e590 0x55b198d25f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:34.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.775+0000 7ff717e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b19893e590 0x55b198d25f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50836/0 (socket says 172.21.15.39:50836) 2024-09-21T07:36:34.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.775+0000 7ff717e00640 1 -- 172.21.15.39:0/4253861921 learned_addr learned my addr 172.21.15.39:0/4253861921 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:34.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.775+0000 7ff71da00640 1 --2- 172.21.15.39:0/4253861921 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b19895f270 0x55b198d5ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:34.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.775+0000 7ff71d000640 1 --2- 172.21.15.39:0/4253861921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1989363b0 0x55b198d44690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:34.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.775+0000 7ff717e00640 1 -- 172.21.15.39:0/4253861921 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b19895f270 msgr2=0x55b198d5ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:36:34.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.775+0000 7ff717e00640 1 --2- 172.21.15.39:0/4253861921 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b19895f270 0x55b198d5ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.775+0000 7ff717e00640 1 -- 172.21.15.39:0/4253861921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1989363b0 msgr2=0x55b198d44690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:36:34.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.775+0000 7ff717e00640 1 --2- 172.21.15.39:0/4253861921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1989363b0 0x55b198d44690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.775+0000 7ff717e00640 1 -- 172.21.15.39:0/4253861921 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1989ca850 con 0x55b19893e590 2024-09-21T07:36:34.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.775+0000 7ff71d000640 1 --2- 172.21.15.39:0/4253861921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1989363b0 0x55b198d44690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:36:34.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.775+0000 7ff717e00640 1 --2- 172.21.15.39:0/4253861921 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b19893e590 0x55b198d25f90 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff704001ea0 tx=0x7ff704000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d977b14b1c6ca15 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:34.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.775+0000 7ff6fe000640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:36:34.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.776+0000 7ff6fe000640 1 -- 172.21.15.39:0/4253861921 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff704027060 con 0x55b19893e590 2024-09-21T07:36:34.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.776+0000 7ff6fe000640 1 -- 172.21.15.39:0/4253861921 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 6800+0+0 (secure 0 0 0) 0x7ff704010040 con 0x55b19893e590 2024-09-21T07:36:34.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.776+0000 7ff726c4d740 1 -- 172.21.15.39:0/4253861921 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff70c0838c0 con 0x55b19893e590 2024-09-21T07:36:34.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.776+0000 7ff6fe000640 1 -- 172.21.15.39:0/4253861921 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7ff704003b00 con 0x55b19893e590 2024-09-21T07:36:34.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.776+0000 7ff726c4d740 10 client.6501 fetch_fsmap learned FSMap version 57 2024-09-21T07:36:34.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.776+0000 7ff726c4d740 1 -- 172.21.15.39:0/4253861921 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) -- 0x55b198a416b0 con 0x55b19893e590 2024-09-21T07:36:34.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.776+0000 7ff6fe000640 1 -- 172.21.15.39:0/4253861921 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff70c0838c0 con 0x55b19893e590 2024-09-21T07:36:34.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.776+0000 7ff6fe000640 1 -- 172.21.15.39:0/4253861921 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 57) ==== 3522+0+0 (secure 0 0 0) 0x7ff704013420 con 0x55b19893e590 2024-09-21T07:36:34.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.776+0000 7ff726c4d740 10 client.6501 fetch_fsmap finished waiting for FSMap version 57 2024-09-21T07:36:34.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.776+0000 7ff726c4d740 10 client.6501 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:36:34.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.776+0000 7ff726c4d740 20 client.6501 populate_metadata read hostname 'smithi039' 2024-09-21T07:36:34.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.776+0000 7ff726c4d740 1 --2- 172.21.15.39:0/4253861921 >> [v2:172.21.15.39:6832/829716143,v1:172.21.15.39:6834/829716143] conn(0x55b19891de40 0x55b198d59b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:34.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.777+0000 7ff726c4d740 4 client.6501 mds_command: new command op to 6321 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:36:34.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.777+0000 7ff726c4d740 1 -- 172.21.15.39:0/4253861921 --> [v2:172.21.15.39:6832/829716143,v1:172.21.15.39:6834/829716143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b198c7b520 con 0x55b19891de40 2024-09-21T07:36:34.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.777+0000 7ff71d000640 1 --2- 172.21.15.39:0/4253861921 >> [v2:172.21.15.39:6832/829716143,v1:172.21.15.39:6834/829716143] conn(0x55b19891de40 0x55b198d59b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:34.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.777+0000 7ff71d000640 1 --2- 172.21.15.39:0/4253861921 >> [v2:172.21.15.39:6832/829716143,v1:172.21.15.39:6834/829716143] conn(0x55b19891de40 0x55b198d59b40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fcfccb087d22da3 server_cookie=69d2240f1f33ce20 in_seq=0 out_seq=0 2024-09-21T07:36:34.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.777+0000 7ff6fe000640 10 client.6501 ms_handle_connect on v2:172.21.15.39:6832/829716143 2024-09-21T07:36:34.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.778+0000 7ff6fe000640 1 -- 172.21.15.39:0/4253861921 <== mds.0 v2:172.21.15.39:6832/829716143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b198c7b520 con 0x55b19891de40 2024-09-21T07:36:34.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.778+0000 7ff6fe000640 10 client.6501 handle_command_reply: tid=0 2024-09-21T07:36:34.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.778+0000 7ff726c4d740 1 client.6501 shutdown 2024-09-21T07:36:34.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.778+0000 7ff726c4d740 1 -- 172.21.15.39:0/4253861921 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b19893e590 msgr2=0x55b198d25f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:34.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.778+0000 7ff726c4d740 1 --2- 172.21.15.39:0/4253861921 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b19893e590 0x55b198d25f90 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff704001ea0 tx=0x7ff704000bc0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.778+0000 7ff726c4d740 1 -- 172.21.15.39:0/4253861921 shutdown_connections 2024-09-21T07:36:34.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.778+0000 7ff726c4d740 1 --2- 172.21.15.39:0/4253861921 >> [v2:172.21.15.39:6832/829716143,v1:172.21.15.39:6834/829716143] conn(0x55b19891de40 0x55b198d59b40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.778+0000 7ff726c4d740 1 --2- 172.21.15.39:0/4253861921 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b19895f270 0x55b198d5ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.778+0000 7ff726c4d740 1 --2- 172.21.15.39:0/4253861921 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b19893e590 0x55b198d25f90 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.778+0000 7ff726c4d740 1 --2- 172.21.15.39:0/4253861921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1989363b0 0x55b198d44690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.778+0000 7ff6fe000640 0 client.6501 ms_handle_reset on v2:172.21.15.39:6832/829716143 2024-09-21T07:36:34.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.778+0000 7ff726c4d740 1 -- 172.21.15.39:0/4253861921 >> 172.21.15.39:0/4253861921 conn(0x55b1989533c0 msgr2=0x55b198d4def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:34.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.779+0000 7ff726c4d740 1 -- 172.21.15.39:0/4253861921 shutdown_connections 2024-09-21T07:36:34.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.779+0000 7ff726c4d740 1 -- 172.21.15.39:0/4253861921 wait complete. 2024-09-21T07:36:34.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.779+0000 7ff726c4d740 20 client.6501 trim_cache size 0 max 16384 2024-09-21T07:36:34.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.798+0000 7ff726c4d740 1 Processor -- start 2024-09-21T07:36:34.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.798+0000 7ff726c4d740 1 -- start start 2024-09-21T07:36:34.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.800+0000 7ff726c4d740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b1989363b0 0x55b198d293b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:34.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.800+0000 7ff726c4d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b19893e590 0x55b198d258f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:34.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.800+0000 7ff726c4d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b19895f270 0x55b198d25e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:34.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.800+0000 7ff726c4d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff7182048d0 con 0x55b19893e590 2024-09-21T07:36:34.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.800+0000 7ff726c4d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff718202c20 con 0x55b19895f270 2024-09-21T07:36:34.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.800+0000 7ff726c4d740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7ff718203270 con 0x55b1989363b0 2024-09-21T07:36:34.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.800+0000 7ff717e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b19893e590 0x55b198d258f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:34.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.800+0000 7ff717e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b19893e590 0x55b198d258f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50844/0 (socket says 172.21.15.39:50844) 2024-09-21T07:36:34.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.800+0000 7ff717e00640 1 -- 172.21.15.39:0/645754414 learned_addr learned my addr 172.21.15.39:0/645754414 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:34.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.800+0000 7ff71da00640 1 --2- 172.21.15.39:0/645754414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b19895f270 0x55b198d25e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:34.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.800+0000 7ff71d000640 1 --2- 172.21.15.39:0/645754414 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b1989363b0 0x55b198d293b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:34.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.800+0000 7ff717e00640 1 -- 172.21.15.39:0/645754414 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b1989363b0 msgr2=0x55b198d293b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:36:34.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.800+0000 7ff717e00640 1 --2- 172.21.15.39:0/645754414 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b1989363b0 0x55b198d293b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.800+0000 7ff717e00640 1 -- 172.21.15.39:0/645754414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b19895f270 msgr2=0x55b198d25e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:36:34.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.800+0000 7ff717e00640 1 --2- 172.21.15.39:0/645754414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b19895f270 0x55b198d25e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.800+0000 7ff717e00640 1 -- 172.21.15.39:0/645754414 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b198d04c20 con 0x55b19893e590 2024-09-21T07:36:34.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.800+0000 7ff71da00640 1 --2- 172.21.15.39:0/645754414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b19895f270 0x55b198d25e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:36:34.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.801+0000 7ff717e00640 1 --2- 172.21.15.39:0/645754414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b19893e590 0x55b198d258f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ff704005040 tx=0x7ff704001000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb2c14d7a97ed8f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:34.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.801+0000 7ff708c00640 1 -- 172.21.15.39:0/645754414 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7040134e0 con 0x55b19893e590 2024-09-21T07:36:34.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.801+0000 7ff708c00640 1 -- 172.21.15.39:0/645754414 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff704003ca0 con 0x55b19893e590 2024-09-21T07:36:34.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.801+0000 7ff708c00640 1 -- 172.21.15.39:0/645754414 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff704013920 con 0x55b19893e590 2024-09-21T07:36:34.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.801+0000 7ff726c4d740 1 -- 172.21.15.39:0/645754414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b19893e590 msgr2=0x55b198d258f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:34.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.801+0000 7ff726c4d740 1 --2- 172.21.15.39:0/645754414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b19893e590 0x55b198d258f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ff704005040 tx=0x7ff704001000 comp rx=0 tx=0).stop 2024-09-21T07:36:34.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.802+0000 7ff726c4d740 1 -- 172.21.15.39:0/645754414 shutdown_connections 2024-09-21T07:36:34.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.802+0000 7ff726c4d740 1 --2- 172.21.15.39:0/645754414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b19895f270 0x55b198d25e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.802+0000 7ff726c4d740 1 --2- 172.21.15.39:0/645754414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b19893e590 0x55b198d258f0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.802+0000 7ff726c4d740 1 --2- 172.21.15.39:0/645754414 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b1989363b0 0x55b198d293b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.802+0000 7ff726c4d740 1 -- 172.21.15.39:0/645754414 >> 172.21.15.39:0/645754414 conn(0x55b1989533c0 msgr2=0x55b198cfb6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:34.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.802+0000 7ff726c4d740 1 -- 172.21.15.39:0/645754414 shutdown_connections 2024-09-21T07:36:34.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.802+0000 7ff726c4d740 1 -- 172.21.15.39:0/645754414 wait complete. 2024-09-21T07:36:34.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.803+0000 7ff726c4d740 1 -- start start 2024-09-21T07:36:34.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.803+0000 7ff726c4d740 1 Processor -- start 2024-09-21T07:36:34.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.804+0000 7ff726c4d740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b1989363b0 0x55b198dc74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:34.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.804+0000 7ff726c4d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b19893e590 0x55b198d43820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:34.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.804+0000 7ff726c4d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b19895f270 0x55b198d43d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:34.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.804+0000 7ff726c4d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b198d2bf50 con 0x55b19895f270 2024-09-21T07:36:34.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.804+0000 7ff726c4d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b198d2a710 con 0x55b19893e590 2024-09-21T07:36:34.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.804+0000 7ff726c4d740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55b198d28930 con 0x55b1989363b0 2024-09-21T07:36:34.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.804+0000 7ff726c4d740 10 client.0 _ll_register_callbacks cb 0x7ff704001cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:36:34.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.805+0000 7ff71da00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b19895f270 0x55b198d43d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:34.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.805+0000 7ff71da00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b19895f270 0x55b198d43d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50854/0 (socket says 172.21.15.39:50854) 2024-09-21T07:36:34.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.805+0000 7ff71da00640 1 -- 172.21.15.39:0/1011164943 learned_addr learned my addr 172.21.15.39:0/1011164943 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:34.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.805+0000 7ff717e00640 1 --2- 172.21.15.39:0/1011164943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b19893e590 0x55b198d43820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:34.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.805+0000 7ff71d000640 1 --2- 172.21.15.39:0/1011164943 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b1989363b0 0x55b198dc74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:34.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.805+0000 7ff71da00640 1 -- 172.21.15.39:0/1011164943 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b1989363b0 msgr2=0x55b198dc74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:36:34.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.805+0000 7ff71da00640 1 --2- 172.21.15.39:0/1011164943 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b1989363b0 0x55b198dc74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.805+0000 7ff71da00640 1 -- 172.21.15.39:0/1011164943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b19893e590 msgr2=0x55b198d43820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:36:34.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.805+0000 7ff71da00640 1 --2- 172.21.15.39:0/1011164943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b19893e590 0x55b198d43820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.805+0000 7ff71da00640 1 -- 172.21.15.39:0/1011164943 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b198d267c0 con 0x55b19895f270 2024-09-21T07:36:34.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.805+0000 7ff717e00640 1 --2- 172.21.15.39:0/1011164943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b19893e590 0x55b198d43820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:36:34.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.805+0000 7ff71da00640 1 --2- 172.21.15.39:0/1011164943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b19895f270 0x55b198d43d60 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff7100108c0 tx=0x7ff710010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9081411b69b04438 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:34.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.805+0000 7ff6fe000640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:36:34.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.805+0000 7ff6fe000640 1 -- 172.21.15.39:0/1011164943 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71001e020 con 0x55b19895f270 2024-09-21T07:36:34.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.806+0000 7ff726c4d740 1 -- 172.21.15.39:0/1011164943 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff704003ca0 con 0x55b19895f270 2024-09-21T07:36:34.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.806+0000 7ff6fe000640 1 -- 172.21.15.39:0/1011164943 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 6800+0+0 (secure 0 0 0) 0x7ff710004d20 con 0x55b19895f270 2024-09-21T07:36:34.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.806+0000 7ff6fe000640 1 -- 172.21.15.39:0/1011164943 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7ff710025050 con 0x55b19895f270 2024-09-21T07:36:34.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.806+0000 7ff726c4d740 10 client.6507 fetch_fsmap learned FSMap version 57 2024-09-21T07:36:34.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.806+0000 7ff726c4d740 1 -- 172.21.15.39:0/1011164943 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) -- 0x55b198dc7a00 con 0x55b19895f270 2024-09-21T07:36:34.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.806+0000 7ff6fe000640 1 -- 172.21.15.39:0/1011164943 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff704003ca0 con 0x55b19895f270 2024-09-21T07:36:34.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.806+0000 7ff6fe000640 1 -- 172.21.15.39:0/1011164943 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 57) ==== 3522+0+0 (secure 0 0 0) 0x7ff710028020 con 0x55b19895f270 2024-09-21T07:36:34.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.806+0000 7ff726c4d740 10 client.6507 fetch_fsmap finished waiting for FSMap version 57 2024-09-21T07:36:34.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.806+0000 7ff726c4d740 10 client.6507 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:36:34.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.806+0000 7ff726c4d740 20 client.6507 populate_metadata read hostname 'smithi039' 2024-09-21T07:36:34.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.806+0000 7ff726c4d740 1 --2- 172.21.15.39:0/1011164943 >> [v2:172.21.15.39:6832/829716143,v1:172.21.15.39:6834/829716143] conn(0x55b198d42900 0x55b198dcdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:34.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.806+0000 7ff726c4d740 4 client.6507 mds_command: new command op to 6321 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:36:34.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.806+0000 7ff726c4d740 1 -- 172.21.15.39:0/1011164943 --> [v2:172.21.15.39:6832/829716143,v1:172.21.15.39:6834/829716143] -- command(tid 0: {"prefix": "status"}) -- 0x55b198d01b50 con 0x55b198d42900 2024-09-21T07:36:34.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.807+0000 7ff71d000640 1 --2- 172.21.15.39:0/1011164943 >> [v2:172.21.15.39:6832/829716143,v1:172.21.15.39:6834/829716143] conn(0x55b198d42900 0x55b198dcdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:34.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.807+0000 7ff71d000640 1 --2- 172.21.15.39:0/1011164943 >> [v2:172.21.15.39:6832/829716143,v1:172.21.15.39:6834/829716143] conn(0x55b198d42900 0x55b198dcdd40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1276776cf55ba119 server_cookie=bd449380cd51ff9 in_seq=0 out_seq=0 2024-09-21T07:36:34.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.807+0000 7ff6fe000640 10 client.6507 ms_handle_connect on v2:172.21.15.39:6832/829716143 2024-09-21T07:36:34.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.807+0000 7ff6fe000640 1 -- 172.21.15.39:0/1011164943 <== mds.0 v2:172.21.15.39:6832/829716143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+318 (crc 0 0 0) 0x55b198d01b50 con 0x55b198d42900 2024-09-21T07:36:34.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.807+0000 7ff6fe000640 10 client.6507 handle_command_reply: tid=0 2024-09-21T07:36:34.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.807+0000 7ff726c4d740 1 client.6507 shutdown 2024-09-21T07:36:34.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.808+0000 7ff726c4d740 1 -- 172.21.15.39:0/1011164943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b19895f270 msgr2=0x55b198d43d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:34.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.808+0000 7ff726c4d740 1 --2- 172.21.15.39:0/1011164943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b19895f270 0x55b198d43d60 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff7100108c0 tx=0x7ff710010d80 comp rx=0 tx=0).stop 2024-09-21T07:36:34.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.808+0000 7ff726c4d740 1 -- 172.21.15.39:0/1011164943 shutdown_connections 2024-09-21T07:36:34.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.808+0000 7ff726c4d740 1 --2- 172.21.15.39:0/1011164943 >> [v2:172.21.15.39:6832/829716143,v1:172.21.15.39:6834/829716143] conn(0x55b198d42900 0x55b198dcdd40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.808+0000 7ff726c4d740 1 --2- 172.21.15.39:0/1011164943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b19895f270 0x55b198d43d60 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.808+0000 7ff726c4d740 1 --2- 172.21.15.39:0/1011164943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b19893e590 0x55b198d43820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.808+0000 7ff726c4d740 1 --2- 172.21.15.39:0/1011164943 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b1989363b0 0x55b198dc74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.808+0000 7ff6fe000640 0 client.6507 ms_handle_reset on v2:172.21.15.39:6832/829716143 2024-09-21T07:36:34.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.808+0000 7ff726c4d740 1 -- 172.21.15.39:0/1011164943 >> 172.21.15.39:0/1011164943 conn(0x55b1989533c0 msgr2=0x55b198d258f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:34.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.808+0000 7ff726c4d740 1 -- 172.21.15.39:0/1011164943 shutdown_connections 2024-09-21T07:36:34.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.808+0000 7ff726c4d740 1 -- 172.21.15.39:0/1011164943 wait complete. 2024-09-21T07:36:34.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.808+0000 7ff726c4d740 20 client.6507 trim_cache size 0 max 16384 2024-09-21T07:36:34.812 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:36:34.812 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:36:34.813 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-09-21T07:36:34.813 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 6321, 2024-09-21T07:36:34.813 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:36:34.813 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:36:34.813 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:36:34.813 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 14.13744649, 2024-09-21T07:36:34.813 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 57, 2024-09-21T07:36:34.813 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 90, 2024-09-21T07:36:34.813 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 90, 2024-09-21T07:36:34.813 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 19.119899647 2024-09-21T07:36:34.813 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:36:34.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.808+0000 7ff71e400640 1 -- 172.21.15.39:0/3005309854 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff6f8078b90 msgr2=0x7ff6f807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:34.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.809+0000 7ff71e400640 1 --2- 172.21.15.39:0/3005309854 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff6f8078b90 0x7ff6f807b0a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff704007fd0 tx=0x7ff704009040 comp rx=0 tx=0).stop 2024-09-21T07:36:34.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.809+0000 7ff71e400640 1 -- 172.21.15.39:0/3005309854 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7180b8fd0 msgr2=0x7ff7181525a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:34.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.809+0000 7ff71e400640 1 --2- 172.21.15.39:0/3005309854 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7180b8fd0 0x7ff7181525a0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff70c034040 tx=0x7ff70c0038a0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.809+0000 7ff71e400640 1 -- 172.21.15.39:0/3005309854 shutdown_connections 2024-09-21T07:36:34.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.809+0000 7ff71e400640 1 --2- 172.21.15.39:0/3005309854 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff6f8078b90 0x7ff6f807b0a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.809+0000 7ff71e400640 1 --2- 172.21.15.39:0/3005309854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff718162440 0x7ff7180d07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.809+0000 7ff71e400640 1 --2- 172.21.15.39:0/3005309854 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff7180b9bd0 0x7ff7180d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.809+0000 7ff71e400640 1 --2- 172.21.15.39:0/3005309854 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7180b8fd0 0x7ff7181525a0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:34.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.809+0000 7ff71e400640 1 -- 172.21.15.39:0/3005309854 >> 172.21.15.39:0/3005309854 conn(0x7ff71809d6c0 msgr2=0x7ff7180d1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:34.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.809+0000 7ff71e400640 1 -- 172.21.15.39:0/3005309854 shutdown_connections 2024-09-21T07:36:34.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:34.809+0000 7ff71e400640 1 -- 172.21.15.39:0/3005309854 wait complete. 2024-09-21T07:36:34.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:36:34.835 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/dir) 2024-09-21T07:36:34.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:36:34.878 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1) 2024-09-21T07:36:34.962 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:36:35.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.228+0000 7f6951c00640 1 -- 172.21.15.39:0/2220138256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f694c0af3c0 msgr2=0x7f694c0b40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:35.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.228+0000 7f6951c00640 1 --2- 172.21.15.39:0/2220138256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f694c0af3c0 0x7f694c0b40b0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f694c072dc0 tx=0x7f693802a540 comp rx=0 tx=0).stop 2024-09-21T07:36:35.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.228+0000 7f6951c00640 1 -- 172.21.15.39:0/2220138256 shutdown_connections 2024-09-21T07:36:35.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.228+0000 7f6951c00640 1 --2- 172.21.15.39:0/2220138256 >> v2:172.21.15.148:3300/0 conn(0x7f694c0ccb50 0x7f694c0ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:35.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.228+0000 7f6951c00640 1 --2- 172.21.15.39:0/2220138256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f694c0af3c0 0x7f694c0b40b0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:35.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.228+0000 7f6951c00640 1 -- 172.21.15.39:0/2220138256 >> 172.21.15.39:0/2220138256 conn(0x7f694c06d7d0 msgr2=0x7f694c0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:35.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.228+0000 7f6951c00640 1 -- 172.21.15.39:0/2220138256 shutdown_connections 2024-09-21T07:36:35.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.228+0000 7f6951c00640 1 -- 172.21.15.39:0/2220138256 wait complete. 2024-09-21T07:36:35.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.230+0000 7f6951c00640 1 Processor -- start 2024-09-21T07:36:35.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.230+0000 7f6951c00640 1 -- start start 2024-09-21T07:36:35.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.232+0000 7f6951c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f694c0af3c0 0x7f694c09bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:35.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.232+0000 7f6951c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f694c0cc1e0 0x7f694c09d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:35.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.232+0000 7f6951c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f694c0ccb50 0x7f694c1f36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:35.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.232+0000 7f6951c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f694c1533a0 con 0x7f694c0af3c0 2024-09-21T07:36:35.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.232+0000 7f6951c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f694c0af8e0 con 0x7f694c0ccb50 2024-09-21T07:36:35.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.232+0000 7f6951c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f694c153520 con 0x7f694c0cc1e0 2024-09-21T07:36:35.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.232+0000 7f694aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f694c0af3c0 0x7f694c09bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:35.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.232+0000 7f694aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f694c0af3c0 0x7f694c09bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50882/0 (socket says 172.21.15.39:50882) 2024-09-21T07:36:35.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.232+0000 7f694aa00640 1 -- 172.21.15.39:0/4098084554 learned_addr learned my addr 172.21.15.39:0/4098084554 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:35.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.232+0000 7f694b400640 1 --2- 172.21.15.39:0/4098084554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f694c0ccb50 0x7f694c1f36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:35.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.232+0000 7f694a000640 1 --2- 172.21.15.39:0/4098084554 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f694c0cc1e0 0x7f694c09d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:35.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.232+0000 7f694aa00640 1 -- 172.21.15.39:0/4098084554 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f694c0cc1e0 msgr2=0x7f694c09d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:35.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.232+0000 7f694aa00640 1 --2- 172.21.15.39:0/4098084554 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f694c0cc1e0 0x7f694c09d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:35.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.232+0000 7f694aa00640 1 -- 172.21.15.39:0/4098084554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f694c0ccb50 msgr2=0x7f694c1f36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:35.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.232+0000 7f694aa00640 1 --2- 172.21.15.39:0/4098084554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f694c0ccb50 0x7f694c1f36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:35.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.232+0000 7f694aa00640 1 -- 172.21.15.39:0/4098084554 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f694c09aa30 con 0x7f694c0af3c0 2024-09-21T07:36:35.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.232+0000 7f694b400640 1 --2- 172.21.15.39:0/4098084554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f694c0ccb50 0x7f694c1f36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:36:35.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.233+0000 7f694aa00640 1 --2- 172.21.15.39:0/4098084554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f694c0af3c0 0x7f694c09bfd0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f693c00c3c0 tx=0x7f693c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:35.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.233+0000 7f6942a00640 1 -- 172.21.15.39:0/4098084554 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693c017070 con 0x7f694c0af3c0 2024-09-21T07:36:35.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.233+0000 7f6942a00640 1 -- 172.21.15.39:0/4098084554 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f693c0035f0 con 0x7f694c0af3c0 2024-09-21T07:36:35.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.233+0000 7f6942a00640 1 -- 172.21.15.39:0/4098084554 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693c003a40 con 0x7f694c0af3c0 2024-09-21T07:36:35.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.233+0000 7f6951c00640 1 -- 172.21.15.39:0/4098084554 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f694c1530b0 con 0x7f694c0af3c0 2024-09-21T07:36:35.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.233+0000 7f6951c00640 1 -- 172.21.15.39:0/4098084554 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f694c09b210 con 0x7f694c0af3c0 2024-09-21T07:36:35.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.234+0000 7f6951c00640 1 -- 172.21.15.39:0/4098084554 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f694c153a40 con 0x7f694c0af3c0 2024-09-21T07:36:35.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.235+0000 7f6942a00640 1 -- 172.21.15.39:0/4098084554 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f693c01a070 con 0x7f694c0af3c0 2024-09-21T07:36:35.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.236+0000 7f6942a00640 1 --2- 172.21.15.39:0/4098084554 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f692c078bc0 0x7f692c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:35.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.236+0000 7f6942a00640 1 -- 172.21.15.39:0/4098084554 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6800+0+0 (secure 0 0 0) 0x7f693c09a4a0 con 0x7f694c0af3c0 2024-09-21T07:36:35.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.237+0000 7f694a000640 1 --2- 172.21.15.39:0/4098084554 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f692c078bc0 0x7f692c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:35.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.238+0000 7f694a000640 1 --2- 172.21.15.39:0/4098084554 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f692c078bc0 0x7f692c07b0d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6938034000 tx=0x7f6938004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:35.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.241+0000 7f6942a00640 1 -- 172.21.15.39:0/4098084554 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f694c0cd4a0 con 0x7f694c0af3c0 2024-09-21T07:36:35.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.477+0000 7f6951c00640 1 -- 172.21.15.39:0/4098084554 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f694c070c30 con 0x7f694c0af3c0 2024-09-21T07:36:35.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.478+0000 7f6942a00640 1 -- 172.21.15.39:0/4098084554 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5244 (secure 0 0 0) 0x7f693c0624b0 con 0x7f694c0af3c0 2024-09-21T07:36:35.479 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:36:35.479 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":57,"btime":"2024-09-21T07:36:32:363503+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5621,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3026827614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3026827614},{"type":"v1","addr":"172.21.15.148:6837","nonce":3026827614}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:36:32.363495+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6321,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6321":{"gid":6321,"name":"c","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/829716143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":829716143},{"type":"v1","addr":"172.21.15.39:6834","nonce":829716143}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6321,"qdb_cluster":[6115,6321]},"id":6}]} 2024-09-21T07:36:35.479 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 57 2024-09-21T07:36:35.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.480+0000 7f6951c00640 1 -- 172.21.15.39:0/4098084554 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f692c078bc0 msgr2=0x7f692c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:35.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.480+0000 7f6951c00640 1 --2- 172.21.15.39:0/4098084554 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f692c078bc0 0x7f692c07b0d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6938034000 tx=0x7f6938004730 comp rx=0 tx=0).stop 2024-09-21T07:36:35.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.480+0000 7f6951c00640 1 -- 172.21.15.39:0/4098084554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f694c0af3c0 msgr2=0x7f694c09bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:35.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.480+0000 7f6951c00640 1 --2- 172.21.15.39:0/4098084554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f694c0af3c0 0x7f694c09bfd0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f693c00c3c0 tx=0x7f693c00c880 comp rx=0 tx=0).stop 2024-09-21T07:36:35.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.480+0000 7f6951c00640 1 -- 172.21.15.39:0/4098084554 shutdown_connections 2024-09-21T07:36:35.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.480+0000 7f6951c00640 1 --2- 172.21.15.39:0/4098084554 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f692c078bc0 0x7f692c07b0d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:35.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.480+0000 7f6951c00640 1 --2- 172.21.15.39:0/4098084554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f694c0ccb50 0x7f694c1f36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:35.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.480+0000 7f6951c00640 1 --2- 172.21.15.39:0/4098084554 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f694c0cc1e0 0x7f694c09d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:35.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.480+0000 7f6951c00640 1 --2- 172.21.15.39:0/4098084554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f694c0af3c0 0x7f694c09bfd0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:35.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.480+0000 7f6951c00640 1 -- 172.21.15.39:0/4098084554 >> 172.21.15.39:0/4098084554 conn(0x7f694c06d7d0 msgr2=0x7f694c06f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:35.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.480+0000 7f6951c00640 1 -- 172.21.15.39:0/4098084554 shutdown_connections 2024-09-21T07:36:35.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:35.481+0000 7f6951c00640 1 -- 172.21.15.39:0/4098084554 wait complete. 2024-09-21T07:36:35.504 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-21T07:36:35.752 INFO:teuthology.orchestra.run.smithi039.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T07:36:35.752 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T07:36:35.752 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1', 1)] 2024-09-21T07:36:37.753 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:36:37.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:37.991+0000 7f930e000640 1 -- 172.21.15.39:0/3152702896 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93080cc4c0 msgr2=0x7f93080ce730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:37.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:37.991+0000 7f930e000640 1 --2- 172.21.15.39:0/3152702896 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93080cc4c0 0x7f93080ce730 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f9300005fd0 tx=0x7f930002a940 comp rx=0 tx=0).stop 2024-09-21T07:36:37.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:37.991+0000 7f930e000640 1 -- 172.21.15.39:0/3152702896 shutdown_connections 2024-09-21T07:36:37.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:37.991+0000 7f930e000640 1 --2- 172.21.15.39:0/3152702896 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93080cc4c0 0x7f93080ce730 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:37.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:37.991+0000 7f930e000640 1 -- 172.21.15.39:0/3152702896 >> 172.21.15.39:0/3152702896 conn(0x7f93080b1cb0 msgr2=0x7f93080b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:37.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:37.991+0000 7f930e000640 1 -- 172.21.15.39:0/3152702896 shutdown_connections 2024-09-21T07:36:37.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:37.992+0000 7f930e000640 1 -- 172.21.15.39:0/3152702896 wait complete. 2024-09-21T07:36:37.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:37.994+0000 7f930e000640 1 Processor -- start 2024-09-21T07:36:37.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:37.994+0000 7f930e000640 1 -- start start 2024-09-21T07:36:37.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:37.995+0000 7f930e000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93080cc4c0 0x7f9308148980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:37.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:37.995+0000 7f930e000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93080cf6f0 0x7f9308161c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:37.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:37.995+0000 7f930e000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f930809aa30 0x7f93080d0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:37.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:37.995+0000 7f930e000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f93080cf340 con 0x7f93080cf6f0 2024-09-21T07:36:37.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:37.995+0000 7f930e000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f93080cf1c0 con 0x7f93080cc4c0 2024-09-21T07:36:37.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:37.995+0000 7f930e000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f93080701b0 con 0x7f930809aa30 2024-09-21T07:36:37.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:37.996+0000 7f92ffe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93080cf6f0 0x7f9308161c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:37.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:37.996+0000 7f92ffe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93080cf6f0 0x7f9308161c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50908/0 (socket says 172.21.15.39:50908) 2024-09-21T07:36:37.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:37.996+0000 7f92ffe00640 1 -- 172.21.15.39:0/3055495217 learned_addr learned my addr 172.21.15.39:0/3055495217 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:37.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:37.996+0000 7f9307400640 1 --2- 172.21.15.39:0/3055495217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93080cc4c0 0x7f9308148980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:37.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:37.996+0000 7f9307e00640 1 --2- 172.21.15.39:0/3055495217 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f930809aa30 0x7f93080d0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:37.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:37.996+0000 7f92ffe00640 1 -- 172.21.15.39:0/3055495217 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f930809aa30 msgr2=0x7f93080d0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:37.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:37.996+0000 7f92ffe00640 1 --2- 172.21.15.39:0/3055495217 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f930809aa30 0x7f93080d0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:37.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:37.996+0000 7f92ffe00640 1 -- 172.21.15.39:0/3055495217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93080cc4c0 msgr2=0x7f9308148980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:37.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:37.996+0000 7f92ffe00640 1 --2- 172.21.15.39:0/3055495217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93080cc4c0 0x7f9308148980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:37.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:37.996+0000 7f92ffe00640 1 -- 172.21.15.39:0/3055495217 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93081f8dc0 con 0x7f93080cf6f0 2024-09-21T07:36:37.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:37.996+0000 7f9307e00640 1 --2- 172.21.15.39:0/3055495217 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f930809aa30 0x7f93080d0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:36:37.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:37.997+0000 7f92ffe00640 1 --2- 172.21.15.39:0/3055495217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93080cf6f0 0x7f9308161c10 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f92f000c3d0 tx=0x7f92f000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:37.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:37.997+0000 7f92ff400640 1 -- 172.21.15.39:0/3055495217 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f0017070 con 0x7f93080cf6f0 2024-09-21T07:36:37.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:37.997+0000 7f92ff400640 1 -- 172.21.15.39:0/3055495217 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f92f00053c0 con 0x7f93080cf6f0 2024-09-21T07:36:37.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:37.997+0000 7f92ff400640 1 -- 172.21.15.39:0/3055495217 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f0003850 con 0x7f93080cf6f0 2024-09-21T07:36:37.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:37.997+0000 7f930e000640 1 -- 172.21.15.39:0/3055495217 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93081f90a0 con 0x7f93080cf6f0 2024-09-21T07:36:37.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:37.997+0000 7f930e000640 1 -- 172.21.15.39:0/3055495217 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93081f9660 con 0x7f93080cf6f0 2024-09-21T07:36:37.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:37.998+0000 7f930e000640 1 -- 172.21.15.39:0/3055495217 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f930809aeb0 con 0x7f93080cf6f0 2024-09-21T07:36:38.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:37.999+0000 7f92ff400640 1 -- 172.21.15.39:0/3055495217 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f92f001a070 con 0x7f93080cf6f0 2024-09-21T07:36:38.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:38.001+0000 7f92ff400640 1 --2- 172.21.15.39:0/3055495217 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f92e4078b90 0x7f92e407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:38.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:38.001+0000 7f92ff400640 1 -- 172.21.15.39:0/3055495217 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6800+0+0 (secure 0 0 0) 0x7f92f009a480 con 0x7f93080cf6f0 2024-09-21T07:36:38.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:38.001+0000 7f9307400640 1 --2- 172.21.15.39:0/3055495217 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f92e4078b90 0x7f92e407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:38.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:38.002+0000 7f9307400640 1 --2- 172.21.15.39:0/3055495217 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f92e4078b90 0x7f92e407b0a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f92f8007fd0 tx=0x7f92f8001370 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:38.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:38.006+0000 7f92ff400640 1 -- 172.21.15.39:0/3055495217 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f92f0061e40 con 0x7f93080cf6f0 2024-09-21T07:36:38.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:38.240+0000 7f930e000640 1 -- 172.21.15.39:0/3055495217 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93080ccb30 con 0x7f93080cf6f0 2024-09-21T07:36:38.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:38.241+0000 7f92ff400640 1 -- 172.21.15.39:0/3055495217 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5244 (secure 0 0 0) 0x7f92f0066870 con 0x7f93080cf6f0 2024-09-21T07:36:38.242 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:36:38.242 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":57,"btime":"2024-09-21T07:36:32:363503+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5621,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3026827614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3026827614},{"type":"v1","addr":"172.21.15.148:6837","nonce":3026827614}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:36:32.363495+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6321,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6321":{"gid":6321,"name":"c","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/829716143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":829716143},{"type":"v1","addr":"172.21.15.39:6834","nonce":829716143}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6321,"qdb_cluster":[6115,6321]},"id":6}]} 2024-09-21T07:36:38.242 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 57 2024-09-21T07:36:38.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:38.243+0000 7f930e000640 1 -- 172.21.15.39:0/3055495217 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f92e4078b90 msgr2=0x7f92e407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:38.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:38.243+0000 7f930e000640 1 --2- 172.21.15.39:0/3055495217 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f92e4078b90 0x7f92e407b0a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f92f8007fd0 tx=0x7f92f8001370 comp rx=0 tx=0).stop 2024-09-21T07:36:38.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:38.244+0000 7f930e000640 1 -- 172.21.15.39:0/3055495217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93080cf6f0 msgr2=0x7f9308161c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:38.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:38.244+0000 7f930e000640 1 --2- 172.21.15.39:0/3055495217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93080cf6f0 0x7f9308161c10 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f92f000c3d0 tx=0x7f92f000c890 comp rx=0 tx=0).stop 2024-09-21T07:36:38.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:38.244+0000 7f930e000640 1 -- 172.21.15.39:0/3055495217 shutdown_connections 2024-09-21T07:36:38.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:38.244+0000 7f930e000640 1 --2- 172.21.15.39:0/3055495217 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f92e4078b90 0x7f92e407b0a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:38.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:38.244+0000 7f930e000640 1 --2- 172.21.15.39:0/3055495217 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f930809aa30 0x7f93080d0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:38.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:38.244+0000 7f930e000640 1 --2- 172.21.15.39:0/3055495217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93080cf6f0 0x7f9308161c10 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:38.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:38.244+0000 7f930e000640 1 --2- 172.21.15.39:0/3055495217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93080cc4c0 0x7f9308148980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:38.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:38.244+0000 7f930e000640 1 -- 172.21.15.39:0/3055495217 >> 172.21.15.39:0/3055495217 conn(0x7f93080b1cb0 msgr2=0x7f93080ce020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:38.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:38.244+0000 7f930e000640 1 -- 172.21.15.39:0/3055495217 shutdown_connections 2024-09-21T07:36:38.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:38.244+0000 7f930e000640 1 -- 172.21.15.39:0/3055495217 wait complete. 2024-09-21T07:36:38.268 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-21T07:36:38.514 INFO:teuthology.orchestra.run.smithi039.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T07:36:38.514 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T07:36:38.514 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1', 1)] 2024-09-21T07:36:40.515 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:36:40.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:40.752+0000 7feb61600640 1 -- 172.21.15.39:0/3780636907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb5c0735d0 msgr2=0x7feb5c0caa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:40.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:40.752+0000 7feb61600640 1 --2- 172.21.15.39:0/3780636907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb5c0735d0 0x7feb5c0caa80 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7feb44009830 tx=0x7feb4402a5c0 comp rx=0 tx=0).stop 2024-09-21T07:36:40.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:40.752+0000 7feb61600640 1 -- 172.21.15.39:0/3780636907 shutdown_connections 2024-09-21T07:36:40.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:40.752+0000 7feb61600640 1 --2- 172.21.15.39:0/3780636907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb5c0735d0 0x7feb5c0caa80 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:40.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:40.752+0000 7feb61600640 1 -- 172.21.15.39:0/3780636907 >> 172.21.15.39:0/3780636907 conn(0x7feb5c06d870 msgr2=0x7feb5c06ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:40.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:40.752+0000 7feb61600640 1 -- 172.21.15.39:0/3780636907 shutdown_connections 2024-09-21T07:36:40.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:40.752+0000 7feb61600640 1 -- 172.21.15.39:0/3780636907 wait complete. 2024-09-21T07:36:40.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:40.754+0000 7feb61600640 1 Processor -- start 2024-09-21T07:36:40.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:40.755+0000 7feb61600640 1 -- start start 2024-09-21T07:36:40.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:40.756+0000 7feb61600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb5c0735d0 0x7feb5c1f6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:40.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:40.756+0000 7feb61600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb5c0af700 0x7feb5c13f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:40.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:40.756+0000 7feb61600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb5c0b54d0 0x7feb5c15b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:40.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:40.756+0000 7feb61600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7feb5c15d4b0 con 0x7feb5c0735d0 2024-09-21T07:36:40.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:40.756+0000 7feb61600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7feb5c15d330 con 0x7feb5c0b54d0 2024-09-21T07:36:40.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:40.756+0000 7feb61600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7feb5c15d630 con 0x7feb5c0af700 2024-09-21T07:36:40.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:40.756+0000 7feb5be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb5c0735d0 0x7feb5c1f6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:40.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:40.756+0000 7feb5be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb5c0735d0 0x7feb5c1f6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50936/0 (socket says 172.21.15.39:50936) 2024-09-21T07:36:40.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:40.756+0000 7feb5be00640 1 -- 172.21.15.39:0/1990570489 learned_addr learned my addr 172.21.15.39:0/1990570489 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:40.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:40.757+0000 7feb53e00640 1 --2- 172.21.15.39:0/1990570489 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb5c0af700 0x7feb5c13f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:40.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:40.757+0000 7feb60c00640 1 --2- 172.21.15.39:0/1990570489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb5c0b54d0 0x7feb5c15b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:40.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:40.757+0000 7feb5be00640 1 -- 172.21.15.39:0/1990570489 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb5c0af700 msgr2=0x7feb5c13f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:40.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:40.757+0000 7feb5be00640 1 --2- 172.21.15.39:0/1990570489 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb5c0af700 0x7feb5c13f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:40.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:40.757+0000 7feb5be00640 1 -- 172.21.15.39:0/1990570489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb5c0b54d0 msgr2=0x7feb5c15b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:40.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:40.757+0000 7feb5be00640 1 --2- 172.21.15.39:0/1990570489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb5c0b54d0 0x7feb5c15b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:40.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:40.757+0000 7feb5be00640 1 -- 172.21.15.39:0/1990570489 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb5c141d00 con 0x7feb5c0735d0 2024-09-21T07:36:40.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:40.757+0000 7feb53e00640 1 --2- 172.21.15.39:0/1990570489 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb5c0af700 0x7feb5c13f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:36:40.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:40.757+0000 7feb5be00640 1 --2- 172.21.15.39:0/1990570489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb5c0735d0 0x7feb5c1f6070 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7feb4c00c430 tx=0x7feb4c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:40.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:40.757+0000 7feb58c00640 1 -- 172.21.15.39:0/1990570489 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4c017070 con 0x7feb5c0735d0 2024-09-21T07:36:40.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:40.757+0000 7feb58c00640 1 -- 172.21.15.39:0/1990570489 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb4c0053c0 con 0x7feb5c0735d0 2024-09-21T07:36:40.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:40.757+0000 7feb58c00640 1 -- 172.21.15.39:0/1990570489 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4c003850 con 0x7feb5c0735d0 2024-09-21T07:36:40.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:40.758+0000 7feb61600640 1 -- 172.21.15.39:0/1990570489 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb5c141fe0 con 0x7feb5c0735d0 2024-09-21T07:36:40.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:40.758+0000 7feb61600640 1 -- 172.21.15.39:0/1990570489 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb5c1425a0 con 0x7feb5c0735d0 2024-09-21T07:36:40.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:40.759+0000 7feb61600640 1 -- 172.21.15.39:0/1990570489 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb5c0caa80 con 0x7feb5c0735d0 2024-09-21T07:36:40.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:40.760+0000 7feb58c00640 1 -- 172.21.15.39:0/1990570489 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7feb4c01a070 con 0x7feb5c0735d0 2024-09-21T07:36:40.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:40.761+0000 7feb58c00640 1 --2- 172.21.15.39:0/1990570489 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feb38078bc0 0x7feb3807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:40.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:40.761+0000 7feb58c00640 1 -- 172.21.15.39:0/1990570489 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6800+0+0 (secure 0 0 0) 0x7feb4c09a390 con 0x7feb5c0735d0 2024-09-21T07:36:40.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:40.762+0000 7feb53e00640 1 --2- 172.21.15.39:0/1990570489 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feb38078bc0 0x7feb3807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:40.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:40.762+0000 7feb53e00640 1 --2- 172.21.15.39:0/1990570489 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feb38078bc0 0x7feb3807b0d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7feb44034000 tx=0x7feb44004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:40.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:40.765+0000 7feb58c00640 1 -- 172.21.15.39:0/1990570489 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7feb4c061dd0 con 0x7feb5c0735d0 2024-09-21T07:36:41.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.006+0000 7feb61600640 1 -- 172.21.15.39:0/1990570489 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb5c1505b0 con 0x7feb5c0735d0 2024-09-21T07:36:41.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.006+0000 7feb58c00640 1 -- 172.21.15.39:0/1990570489 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+5245 (secure 0 0 0) 0x7feb4c066800 con 0x7feb5c0735d0 2024-09-21T07:36:41.008 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:36:41.008 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":58,"btime":"2024-09-21T07:36:40:771296+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5621,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3026827614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3026827614},{"type":"v1","addr":"172.21.15.148:6837","nonce":3026827614}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:36:40.693437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6321,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6321":{"gid":6321,"name":"c","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/829716143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":829716143},{"type":"v1","addr":"172.21.15.39:6834","nonce":829716143}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6321,"qdb_cluster":[6321,6115]},"id":6}]} 2024-09-21T07:36:41.009 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 58 2024-09-21T07:36:41.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.009+0000 7feb61600640 1 -- 172.21.15.39:0/1990570489 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feb38078bc0 msgr2=0x7feb3807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:41.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.009+0000 7feb61600640 1 --2- 172.21.15.39:0/1990570489 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feb38078bc0 0x7feb3807b0d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7feb44034000 tx=0x7feb44004730 comp rx=0 tx=0).stop 2024-09-21T07:36:41.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.009+0000 7feb61600640 1 -- 172.21.15.39:0/1990570489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb5c0735d0 msgr2=0x7feb5c1f6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:41.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.009+0000 7feb61600640 1 --2- 172.21.15.39:0/1990570489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb5c0735d0 0x7feb5c1f6070 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7feb4c00c430 tx=0x7feb4c00c8f0 comp rx=0 tx=0).stop 2024-09-21T07:36:41.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.009+0000 7feb61600640 1 -- 172.21.15.39:0/1990570489 shutdown_connections 2024-09-21T07:36:41.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.009+0000 7feb61600640 1 --2- 172.21.15.39:0/1990570489 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feb38078bc0 0x7feb3807b0d0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:41.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.009+0000 7feb61600640 1 --2- 172.21.15.39:0/1990570489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb5c0b54d0 0x7feb5c15b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:41.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.009+0000 7feb61600640 1 --2- 172.21.15.39:0/1990570489 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb5c0af700 0x7feb5c13f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:41.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.009+0000 7feb61600640 1 --2- 172.21.15.39:0/1990570489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb5c0735d0 0x7feb5c1f6070 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:41.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.009+0000 7feb61600640 1 -- 172.21.15.39:0/1990570489 >> 172.21.15.39:0/1990570489 conn(0x7feb5c06d870 msgr2=0x7feb5c14f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:41.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.009+0000 7feb61600640 1 -- 172.21.15.39:0/1990570489 shutdown_connections 2024-09-21T07:36:41.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.009+0000 7feb61600640 1 -- 172.21.15.39:0/1990570489 wait complete. 2024-09-21T07:36:41.037 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-21T07:36:41.280 INFO:teuthology.orchestra.run.smithi039.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"8","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1","dirfrag":"0x10000000000","snapid_first":2,"projected_version":"3","version":"3","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-09-21T07:36:41.281 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "8" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000000", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "3", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "3" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T07:36:41.281 INFO:tasks.cephfs.cephfs_test_case:[('/d1', 1)] =?= [('/d1', 1)] 2024-09-21T07:36:41.281 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2024-09-21T07:36:41.528 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_snap":1,"last_created":1,"last_destroyed":1,"pending_noop":[],"snaps":[],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2024-09-21T07:36:41.529 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 1, "last_destroyed": 1, "last_snap": 1, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [] } 2024-09-21T07:36:41.529 INFO:tasks.cephfs.test_snapshots:testing snapserver mds_kill_mdstable_at=1 2024-09-21T07:36:41.529 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:36:41.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.755+0000 7f6eeb200640 1 -- 172.21.15.39:0/1461299295 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6eec0d0dc0 msgr2=0x7f6eec0b1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:41.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.755+0000 7f6eeb200640 1 --2- 172.21.15.39:0/1461299295 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6eec0d0dc0 0x7f6eec0b1a20 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4005fd0 tx=0x7f6ee402aa90 comp rx=0 tx=0).stop 2024-09-21T07:36:41.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.755+0000 7f6eeb200640 1 -- 172.21.15.39:0/1461299295 shutdown_connections 2024-09-21T07:36:41.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.755+0000 7f6eeb200640 1 --2- 172.21.15.39:0/1461299295 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6eec0b7580 0x7f6eec0d0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:41.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.755+0000 7f6eeb200640 1 --2- 172.21.15.39:0/1461299295 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6eec0d0dc0 0x7f6eec0b1a20 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:41.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.755+0000 7f6eeb200640 1 -- 172.21.15.39:0/1461299295 >> 172.21.15.39:0/1461299295 conn(0x7f6eec0ceba0 msgr2=0x7f6eec0cefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:41.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.755+0000 7f6eeb200640 1 -- 172.21.15.39:0/1461299295 shutdown_connections 2024-09-21T07:36:41.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.755+0000 7f6eeb200640 1 -- 172.21.15.39:0/1461299295 wait complete. 2024-09-21T07:36:41.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.757+0000 7f6eeb200640 1 Processor -- start 2024-09-21T07:36:41.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.757+0000 7f6eeb200640 1 -- start start 2024-09-21T07:36:41.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.758+0000 7f6eeb200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6eec0b7580 0x7f6eec0cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:41.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.758+0000 7f6eeb200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6eec0ce730 0x7f6eec1618a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:41.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.758+0000 7f6eeb200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6eec0d0dc0 0x7f6eec161de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:41.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.758+0000 7f6eeb200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6eec0b22f0 con 0x7f6eec0d0dc0 2024-09-21T07:36:41.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.758+0000 7f6eeb200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6eec0b2170 con 0x7f6eec0ce730 2024-09-21T07:36:41.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.759+0000 7f6eeb200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f6eec0b2470 con 0x7f6eec0b7580 2024-09-21T07:36:41.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.759+0000 7f6eea800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6eec0d0dc0 0x7f6eec161de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:41.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.759+0000 7f6eea800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6eec0d0dc0 0x7f6eec161de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50948/0 (socket says 172.21.15.39:50948) 2024-09-21T07:36:41.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.759+0000 7f6eea800640 1 -- 172.21.15.39:0/2336930314 learned_addr learned my addr 172.21.15.39:0/2336930314 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:41.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.759+0000 7f6ee3e00640 1 --2- 172.21.15.39:0/2336930314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6eec0ce730 0x7f6eec1618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:41.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.759+0000 7f6ee9e00640 1 --2- 172.21.15.39:0/2336930314 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6eec0b7580 0x7f6eec0cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:41.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.759+0000 7f6eea800640 1 -- 172.21.15.39:0/2336930314 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6eec0b7580 msgr2=0x7f6eec0cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:41.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.759+0000 7f6eea800640 1 --2- 172.21.15.39:0/2336930314 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6eec0b7580 0x7f6eec0cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:41.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.759+0000 7f6eea800640 1 -- 172.21.15.39:0/2336930314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6eec0ce730 msgr2=0x7f6eec1618a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:41.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.759+0000 7f6eea800640 1 --2- 172.21.15.39:0/2336930314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6eec0ce730 0x7f6eec1618a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:41.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.759+0000 7f6eea800640 1 -- 172.21.15.39:0/2336930314 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eec0cbdf0 con 0x7f6eec0d0dc0 2024-09-21T07:36:41.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.759+0000 7f6ee3e00640 1 --2- 172.21.15.39:0/2336930314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6eec0ce730 0x7f6eec1618a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:36:41.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.759+0000 7f6ee9e00640 1 --2- 172.21.15.39:0/2336930314 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6eec0b7580 0x7f6eec0cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:36:41.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.760+0000 7f6eea800640 1 --2- 172.21.15.39:0/2336930314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6eec0d0dc0 0x7f6eec161de0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f6ee402b080 tx=0x7f6ee4003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:41.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.760+0000 7f6ee2000640 1 -- 172.21.15.39:0/2336930314 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee402e7f0 con 0x7f6eec0d0dc0 2024-09-21T07:36:41.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.760+0000 7f6ee2000640 1 -- 172.21.15.39:0/2336930314 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ee4004250 con 0x7f6eec0d0dc0 2024-09-21T07:36:41.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.760+0000 7f6ee2000640 1 -- 172.21.15.39:0/2336930314 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee4004660 con 0x7f6eec0d0dc0 2024-09-21T07:36:41.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.760+0000 7f6eeb200640 1 -- 172.21.15.39:0/2336930314 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eec15c700 con 0x7f6eec0d0dc0 2024-09-21T07:36:41.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.760+0000 7f6eeb200640 1 -- 172.21.15.39:0/2336930314 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eec15ccc0 con 0x7f6eec0d0dc0 2024-09-21T07:36:41.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.761+0000 7f6eeb200640 1 -- 172.21.15.39:0/2336930314 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6eec0d0880 con 0x7f6eec0d0dc0 2024-09-21T07:36:41.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.762+0000 7f6ee2000640 1 -- 172.21.15.39:0/2336930314 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6ee4049440 con 0x7f6eec0d0dc0 2024-09-21T07:36:41.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.764+0000 7f6ee2000640 1 --2- 172.21.15.39:0/2336930314 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6ec8078b90 0x7f6ec807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:41.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.764+0000 7f6ee2000640 1 -- 172.21.15.39:0/2336930314 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6800+0+0 (secure 0 0 0) 0x7f6ee40bc670 con 0x7f6eec0d0dc0 2024-09-21T07:36:41.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.764+0000 7f6ee9e00640 1 --2- 172.21.15.39:0/2336930314 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6ec8078b90 0x7f6ec807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:41.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.765+0000 7f6ee9e00640 1 --2- 172.21.15.39:0/2336930314 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6ec8078b90 0x7f6ec807b0a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6edc0016d0 tx=0x7f6edc009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:41.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:41.769+0000 7f6ee2000640 1 -- 172.21.15.39:0/2336930314 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6ee4032030 con 0x7f6eec0d0dc0 2024-09-21T07:36:42.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.005+0000 7f6eeb200640 1 -- 172.21.15.39:0/2336930314 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6eec166bd0 con 0x7f6eec0d0dc0 2024-09-21T07:36:42.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.006+0000 7f6ee2000640 1 -- 172.21.15.39:0/2336930314 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+5245 (secure 0 0 0) 0x7f6ee40845f0 con 0x7f6eec0d0dc0 2024-09-21T07:36:42.008 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:36:42.008 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":58,"btime":"2024-09-21T07:36:40:771296+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5621,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3026827614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3026827614},{"type":"v1","addr":"172.21.15.148:6837","nonce":3026827614}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:36:40.693437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6321,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6321":{"gid":6321,"name":"c","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/829716143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":829716143},{"type":"v1","addr":"172.21.15.39:6834","nonce":829716143}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6321,"qdb_cluster":[6321,6115]},"id":6}]} 2024-09-21T07:36:42.008 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 58 2024-09-21T07:36:42.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.008+0000 7f6eeb200640 1 -- 172.21.15.39:0/2336930314 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6ec8078b90 msgr2=0x7f6ec807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:42.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.008+0000 7f6eeb200640 1 --2- 172.21.15.39:0/2336930314 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6ec8078b90 0x7f6ec807b0a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6edc0016d0 tx=0x7f6edc009040 comp rx=0 tx=0).stop 2024-09-21T07:36:42.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.008+0000 7f6eeb200640 1 -- 172.21.15.39:0/2336930314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6eec0d0dc0 msgr2=0x7f6eec161de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:42.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.008+0000 7f6eeb200640 1 --2- 172.21.15.39:0/2336930314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6eec0d0dc0 0x7f6eec161de0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f6ee402b080 tx=0x7f6ee4003f10 comp rx=0 tx=0).stop 2024-09-21T07:36:42.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.008+0000 7f6eeb200640 1 -- 172.21.15.39:0/2336930314 shutdown_connections 2024-09-21T07:36:42.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.008+0000 7f6eeb200640 1 --2- 172.21.15.39:0/2336930314 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6ec8078b90 0x7f6ec807b0a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:42.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.008+0000 7f6eeb200640 1 --2- 172.21.15.39:0/2336930314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6eec0d0dc0 0x7f6eec161de0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:42.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.008+0000 7f6eeb200640 1 --2- 172.21.15.39:0/2336930314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6eec0ce730 0x7f6eec1618a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:42.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.008+0000 7f6eeb200640 1 --2- 172.21.15.39:0/2336930314 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6eec0b7580 0x7f6eec0cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:42.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.008+0000 7f6eeb200640 1 -- 172.21.15.39:0/2336930314 >> 172.21.15.39:0/2336930314 conn(0x7f6eec0ceba0 msgr2=0x7f6eec0724c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:42.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.008+0000 7f6eeb200640 1 -- 172.21.15.39:0/2336930314 shutdown_connections 2024-09-21T07:36:42.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.008+0000 7f6eeb200640 1 -- 172.21.15.39:0/2336930314 wait complete. 2024-09-21T07:36:42.034 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:0 true 2024-09-21T07:36:42.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.260+0000 7f805a400640 1 -- 172.21.15.39:0/1669052171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f805406f580 msgr2=0x7f805415d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:42.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.260+0000 7f805a400640 1 --2- 172.21.15.39:0/1669052171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f805406f580 0x7f805415d700 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f804c005fd0 tx=0x7f804c02a8b0 comp rx=0 tx=0).stop 2024-09-21T07:36:42.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.260+0000 7f805a400640 1 -- 172.21.15.39:0/1669052171 shutdown_connections 2024-09-21T07:36:42.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.260+0000 7f805a400640 1 --2- 172.21.15.39:0/1669052171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f805406f580 0x7f805415d700 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:42.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.260+0000 7f805a400640 1 -- 172.21.15.39:0/1669052171 >> 172.21.15.39:0/1669052171 conn(0x7f805406d410 msgr2=0x7f80540b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:42.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.260+0000 7f805a400640 1 -- 172.21.15.39:0/1669052171 shutdown_connections 2024-09-21T07:36:42.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.260+0000 7f805a400640 1 -- 172.21.15.39:0/1669052171 wait complete. 2024-09-21T07:36:42.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.263+0000 7f805a400640 1 Processor -- start 2024-09-21T07:36:42.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.263+0000 7f805a400640 1 -- start start 2024-09-21T07:36:42.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.264+0000 7f805a400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80540cf540 0x7f80541f5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:42.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.264+0000 7f805a400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f805415ca50 0x7f80541435f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:42.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.264+0000 7f805a400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f805413ca70 0x7f80540b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:42.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.264+0000 7f805a400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f805415e010 con 0x7f80540cf540 2024-09-21T07:36:42.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.264+0000 7f805a400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f805415de90 con 0x7f805415ca50 2024-09-21T07:36:42.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.264+0000 7f805a400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f805415e190 con 0x7f805413ca70 2024-09-21T07:36:42.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.264+0000 7f8059000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80540cf540 0x7f80541f5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:42.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.265+0000 7f8059000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80540cf540 0x7f80541f5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54882/0 (socket says 172.21.15.39:54882) 2024-09-21T07:36:42.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.265+0000 7f8059000640 1 -- 172.21.15.39:0/203355678 learned_addr learned my addr 172.21.15.39:0/203355678 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:42.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.265+0000 7f8059a00640 1 --2- 172.21.15.39:0/203355678 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f805413ca70 0x7f80540b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:42.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.265+0000 7f804be00640 1 --2- 172.21.15.39:0/203355678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f805415ca50 0x7f80541435f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:42.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.265+0000 7f8059000640 1 -- 172.21.15.39:0/203355678 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f805413ca70 msgr2=0x7f80540b2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:42.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.265+0000 7f8059000640 1 --2- 172.21.15.39:0/203355678 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f805413ca70 0x7f80540b2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:42.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.265+0000 7f8059000640 1 -- 172.21.15.39:0/203355678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f805415ca50 msgr2=0x7f80541435f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:42.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.265+0000 7f8059000640 1 --2- 172.21.15.39:0/203355678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f805415ca50 0x7f80541435f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:42.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.265+0000 7f8059000640 1 -- 172.21.15.39:0/203355678 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80541403b0 con 0x7f80540cf540 2024-09-21T07:36:42.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.265+0000 7f8059000640 1 --2- 172.21.15.39:0/203355678 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80540cf540 0x7f80541f5df0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f804400c390 tx=0x7f804400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:42.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.266+0000 7f8051600640 1 -- 172.21.15.39:0/203355678 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8044011790 con 0x7f80540cf540 2024-09-21T07:36:42.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.266+0000 7f8051600640 1 -- 172.21.15.39:0/203355678 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8044017070 con 0x7f80540cf540 2024-09-21T07:36:42.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.266+0000 7f805a400640 1 -- 172.21.15.39:0/203355678 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8054140690 con 0x7f80540cf540 2024-09-21T07:36:42.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.266+0000 7f8051600640 1 -- 172.21.15.39:0/203355678 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8044011ca0 con 0x7f80540cf540 2024-09-21T07:36:42.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.266+0000 7f805a400640 1 -- 172.21.15.39:0/203355678 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80541f58d0 con 0x7f80540cf540 2024-09-21T07:36:42.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.267+0000 7f805a400640 1 -- 172.21.15.39:0/203355678 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f805415d700 con 0x7f80540cf540 2024-09-21T07:36:42.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.268+0000 7f8051600640 1 -- 172.21.15.39:0/203355678 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f804401c460 con 0x7f80540cf540 2024-09-21T07:36:42.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.269+0000 7f8051600640 1 --2- 172.21.15.39:0/203355678 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8030078bc0 0x7f803007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:42.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.269+0000 7f8051600640 1 -- 172.21.15.39:0/203355678 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6800+0+0 (secure 0 0 0) 0x7f804409ae20 con 0x7f80540cf540 2024-09-21T07:36:42.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.270+0000 7f804be00640 1 --2- 172.21.15.39:0/203355678 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8030078bc0 0x7f803007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:42.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.270+0000 7f804be00640 1 --2- 172.21.15.39:0/203355678 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8030078bc0 0x7f803007b0d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8054145a30 tx=0x7f803c009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:42.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.274+0000 7f8051600640 1 -- 172.21.15.39:0/203355678 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8044062750 con 0x7f80540cf540 2024-09-21T07:36:42.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.547+0000 7f805a400640 1 -- 172.21.15.39:0/203355678 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"} v 0) -- 0x7f805413ce80 con 0x7f80540cf540 2024-09-21T07:36:42.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.792+0000 7f8051600640 1 -- 172.21.15.39:0/203355678 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"}]=0 freezing mds.6321 v59) ==== 112+0+0 (secure 0 0 0) 0x7f8044067180 con 0x7f80540cf540 2024-09-21T07:36:42.794 INFO:teuthology.orchestra.run.smithi039.stderr:freezing mds.6321 2024-09-21T07:36:42.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.795+0000 7f805a400640 1 -- 172.21.15.39:0/203355678 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8030078bc0 msgr2=0x7f803007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:42.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.795+0000 7f805a400640 1 --2- 172.21.15.39:0/203355678 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8030078bc0 0x7f803007b0d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8054145a30 tx=0x7f803c009040 comp rx=0 tx=0).stop 2024-09-21T07:36:42.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.795+0000 7f805a400640 1 -- 172.21.15.39:0/203355678 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80540cf540 msgr2=0x7f80541f5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:42.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.795+0000 7f805a400640 1 --2- 172.21.15.39:0/203355678 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80540cf540 0x7f80541f5df0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f804400c390 tx=0x7f804400c850 comp rx=0 tx=0).stop 2024-09-21T07:36:42.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.795+0000 7f805a400640 1 -- 172.21.15.39:0/203355678 shutdown_connections 2024-09-21T07:36:42.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.795+0000 7f805a400640 1 --2- 172.21.15.39:0/203355678 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8030078bc0 0x7f803007b0d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:42.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.795+0000 7f805a400640 1 --2- 172.21.15.39:0/203355678 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f805413ca70 0x7f80540b2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:42.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.795+0000 7f805a400640 1 --2- 172.21.15.39:0/203355678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f805415ca50 0x7f80541435f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:42.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.795+0000 7f805a400640 1 --2- 172.21.15.39:0/203355678 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80540cf540 0x7f80541f5df0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:42.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.795+0000 7f805a400640 1 -- 172.21.15.39:0/203355678 >> 172.21.15.39:0/203355678 conn(0x7f805406d410 msgr2=0x7f8054073ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:42.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.795+0000 7f805a400640 1 -- 172.21.15.39:0/203355678 shutdown_connections 2024-09-21T07:36:42.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:42.795+0000 7f805a400640 1 -- 172.21.15.39:0/203355678 wait complete. 2024-09-21T07:36:42.820 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json config set mds_kill_mdstable_at 1 2024-09-21T07:36:43.063 INFO:teuthology.orchestra.run.smithi039.stdout:{"success":"mds_kill_mdstable_at = '' (not observed, change may require restart) "} 2024-09-21T07:36:43.063 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_kill_mdstable_at = '' (not observed, change may require restart) " } 2024-09-21T07:36:43.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:36:43.064 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d1/dir/.snap/s11) 2024-09-21T07:36:43.068 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-09-21T07:36:43.101 INFO:tasks.ceph.mds.c.smithi039.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_prepare(ceph::cref_t&)' thread 7f3c24e00640 time 2024-09-21T07:36:43.100545+0000 2024-09-21T07:36:43.101 INFO:tasks.ceph.mds.c.smithi039.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: 61: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 1) 2024-09-21T07:36:43.104 INFO:tasks.ceph.mds.c.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:36:43.104 INFO:tasks.ceph.mds.c.smithi039.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f3c2be5654b] 2024-09-21T07:36:43.104 INFO:tasks.ceph.mds.c.smithi039.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f3c2be56758] 2024-09-21T07:36:43.104 INFO:tasks.ceph.mds.c.smithi039.stderr: 3: (MDSTableServer::handle_prepare(boost::intrusive_ptr const&)+0x312) [0x5580d00b42a0] 2024-09-21T07:36:43.104 INFO:tasks.ceph.mds.c.smithi039.stderr: 4: (MDSTableServer::handle_request(boost::intrusive_ptr const&)+0x93) [0x5580d00b8909] 2024-09-21T07:36:43.104 INFO:tasks.ceph.mds.c.smithi039.stderr: 5: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xca5) [0x5580cfd51de3] 2024-09-21T07:36:43.104 INFO:tasks.ceph.mds.c.smithi039.stderr: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5580cfd55523] 2024-09-21T07:36:43.105 INFO:tasks.ceph.mds.c.smithi039.stderr: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5580cfd55e4a] 2024-09-21T07:36:43.105 INFO:tasks.ceph.mds.c.smithi039.stderr: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x5580cfd38fa4] 2024-09-21T07:36:43.105 INFO:tasks.ceph.mds.c.smithi039.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f3c2bfa57df] 2024-09-21T07:36:43.105 INFO:tasks.ceph.mds.c.smithi039.stderr: 10: (DispatchQueue::entry()+0x60f) [0x7f3c2bfa60d1] 2024-09-21T07:36:43.105 INFO:tasks.ceph.mds.c.smithi039.stderr: 11: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f3c2c093f5b] 2024-09-21T07:36:43.105 INFO:tasks.ceph.mds.c.smithi039.stderr: 12: (Thread::entry_wrapper()+0x43) [0x7f3c2be2fd57] 2024-09-21T07:36:43.105 INFO:tasks.ceph.mds.c.smithi039.stderr: 13: (Thread::_entry_func(void*)+0xd) [0x7f3c2be2fd73] 2024-09-21T07:36:43.105 INFO:tasks.ceph.mds.c.smithi039.stderr: 14: /lib64/libc.so.6(+0x89c52) [0x7f3c2b489c52] 2024-09-21T07:36:43.105 INFO:tasks.ceph.mds.c.smithi039.stderr: 15: /lib64/libc.so.6(+0x10ec80) [0x7f3c2b50ec80] 2024-09-21T07:36:43.105 INFO:tasks.ceph.mds.c.smithi039.stderr:2024-09-21T07:36:43.100+0000 7f3c24e00640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_prepare(ceph::cref_t&)' thread 7f3c24e00640 time 2024-09-21T07:36:43.100545+0000 2024-09-21T07:36:43.105 INFO:tasks.ceph.mds.c.smithi039.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: 61: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 1) 2024-09-21T07:36:43.105 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:36:43.105 INFO:tasks.ceph.mds.c.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:36:43.106 INFO:tasks.ceph.mds.c.smithi039.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f3c2be5654b] 2024-09-21T07:36:43.106 INFO:tasks.ceph.mds.c.smithi039.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f3c2be56758] 2024-09-21T07:36:43.106 INFO:tasks.ceph.mds.c.smithi039.stderr: 3: (MDSTableServer::handle_prepare(boost::intrusive_ptr const&)+0x312) [0x5580d00b42a0] 2024-09-21T07:36:43.106 INFO:tasks.ceph.mds.c.smithi039.stderr: 4: (MDSTableServer::handle_request(boost::intrusive_ptr const&)+0x93) [0x5580d00b8909] 2024-09-21T07:36:43.106 INFO:tasks.ceph.mds.c.smithi039.stderr: 5: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xca5) [0x5580cfd51de3] 2024-09-21T07:36:43.106 INFO:tasks.ceph.mds.c.smithi039.stderr: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5580cfd55523] 2024-09-21T07:36:43.106 INFO:tasks.ceph.mds.c.smithi039.stderr: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5580cfd55e4a] 2024-09-21T07:36:43.106 INFO:tasks.ceph.mds.c.smithi039.stderr: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x5580cfd38fa4] 2024-09-21T07:36:43.106 INFO:tasks.ceph.mds.c.smithi039.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f3c2bfa57df] 2024-09-21T07:36:43.106 INFO:tasks.ceph.mds.c.smithi039.stderr: 10: (DispatchQueue::entry()+0x60f) [0x7f3c2bfa60d1] 2024-09-21T07:36:43.106 INFO:tasks.ceph.mds.c.smithi039.stderr: 11: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f3c2c093f5b] 2024-09-21T07:36:43.106 INFO:tasks.ceph.mds.c.smithi039.stderr: 12: (Thread::entry_wrapper()+0x43) [0x7f3c2be2fd57] 2024-09-21T07:36:43.106 INFO:tasks.ceph.mds.c.smithi039.stderr: 13: (Thread::_entry_func(void*)+0xd) [0x7f3c2be2fd73] 2024-09-21T07:36:43.107 INFO:tasks.ceph.mds.c.smithi039.stderr: 14: /lib64/libc.so.6(+0x89c52) [0x7f3c2b489c52] 2024-09-21T07:36:43.107 INFO:tasks.ceph.mds.c.smithi039.stderr: 15: /lib64/libc.so.6(+0x10ec80) [0x7f3c2b50ec80] 2024-09-21T07:36:43.107 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:36:43.107 INFO:tasks.ceph.mds.c.smithi039.stderr:*** Caught signal (Aborted) ** 2024-09-21T07:36:43.107 INFO:tasks.ceph.mds.c.smithi039.stderr: in thread 7f3c24e00640 thread_name:ms_dispatch 2024-09-21T07:36:43.107 INFO:tasks.ceph.mds.c.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:36:43.107 INFO:tasks.ceph.mds.c.smithi039.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f3c2b43e6f0] 2024-09-21T07:36:43.107 INFO:tasks.ceph.mds.c.smithi039.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f3c2b48b99c] 2024-09-21T07:36:43.107 INFO:tasks.ceph.mds.c.smithi039.stderr: 3: raise() 2024-09-21T07:36:43.107 INFO:tasks.ceph.mds.c.smithi039.stderr: 4: abort() 2024-09-21T07:36:43.107 INFO:tasks.ceph.mds.c.smithi039.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7f3c2be56686] 2024-09-21T07:36:43.107 INFO:tasks.ceph.mds.c.smithi039.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f3c2be56758] 2024-09-21T07:36:43.107 INFO:tasks.ceph.mds.c.smithi039.stderr: 7: (MDSTableServer::handle_prepare(boost::intrusive_ptr const&)+0x312) [0x5580d00b42a0] 2024-09-21T07:36:43.107 INFO:tasks.ceph.mds.c.smithi039.stderr: 8: (MDSTableServer::handle_request(boost::intrusive_ptr const&)+0x93) [0x5580d00b8909] 2024-09-21T07:36:43.108 INFO:tasks.ceph.mds.c.smithi039.stderr: 9: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xca5) [0x5580cfd51de3] 2024-09-21T07:36:43.108 INFO:tasks.ceph.mds.c.smithi039.stderr: 10: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5580cfd55523] 2024-09-21T07:36:43.108 INFO:tasks.ceph.mds.c.smithi039.stderr: 11: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5580cfd55e4a] 2024-09-21T07:36:43.108 INFO:tasks.ceph.mds.c.smithi039.stderr: 12: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x5580cfd38fa4] 2024-09-21T07:36:43.108 INFO:tasks.ceph.mds.c.smithi039.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f3c2bfa57df] 2024-09-21T07:36:43.108 INFO:tasks.ceph.mds.c.smithi039.stderr: 14: (DispatchQueue::entry()+0x60f) [0x7f3c2bfa60d1] 2024-09-21T07:36:43.108 INFO:tasks.ceph.mds.c.smithi039.stderr: 15: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f3c2c093f5b] 2024-09-21T07:36:43.108 INFO:tasks.ceph.mds.c.smithi039.stderr: 16: (Thread::entry_wrapper()+0x43) [0x7f3c2be2fd57] 2024-09-21T07:36:43.108 INFO:tasks.ceph.mds.c.smithi039.stderr: 17: (Thread::_entry_func(void*)+0xd) [0x7f3c2be2fd73] 2024-09-21T07:36:43.108 INFO:tasks.ceph.mds.c.smithi039.stderr: 18: /lib64/libc.so.6(+0x89c52) [0x7f3c2b489c52] 2024-09-21T07:36:43.108 INFO:tasks.ceph.mds.c.smithi039.stderr: 19: /lib64/libc.so.6(+0x10ec80) [0x7f3c2b50ec80] 2024-09-21T07:36:43.108 INFO:tasks.ceph.mds.c.smithi039.stderr:2024-09-21T07:36:43.101+0000 7f3c24e00640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:36:43.108 INFO:tasks.ceph.mds.c.smithi039.stderr: in thread 7f3c24e00640 thread_name:ms_dispatch 2024-09-21T07:36:43.108 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:36:43.109 INFO:tasks.ceph.mds.c.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:36:43.109 INFO:tasks.ceph.mds.c.smithi039.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f3c2b43e6f0] 2024-09-21T07:36:43.109 INFO:tasks.ceph.mds.c.smithi039.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f3c2b48b99c] 2024-09-21T07:36:43.109 INFO:tasks.ceph.mds.c.smithi039.stderr: 3: raise() 2024-09-21T07:36:43.109 INFO:tasks.ceph.mds.c.smithi039.stderr: 4: abort() 2024-09-21T07:36:43.109 INFO:tasks.ceph.mds.c.smithi039.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7f3c2be56686] 2024-09-21T07:36:43.109 INFO:tasks.ceph.mds.c.smithi039.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f3c2be56758] 2024-09-21T07:36:43.109 INFO:tasks.ceph.mds.c.smithi039.stderr: 7: (MDSTableServer::handle_prepare(boost::intrusive_ptr const&)+0x312) [0x5580d00b42a0] 2024-09-21T07:36:43.109 INFO:tasks.ceph.mds.c.smithi039.stderr: 8: (MDSTableServer::handle_request(boost::intrusive_ptr const&)+0x93) [0x5580d00b8909] 2024-09-21T07:36:43.109 INFO:tasks.ceph.mds.c.smithi039.stderr: 9: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xca5) [0x5580cfd51de3] 2024-09-21T07:36:43.109 INFO:tasks.ceph.mds.c.smithi039.stderr: 10: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5580cfd55523] 2024-09-21T07:36:43.109 INFO:tasks.ceph.mds.c.smithi039.stderr: 11: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5580cfd55e4a] 2024-09-21T07:36:43.109 INFO:tasks.ceph.mds.c.smithi039.stderr: 12: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x5580cfd38fa4] 2024-09-21T07:36:43.109 INFO:tasks.ceph.mds.c.smithi039.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f3c2bfa57df] 2024-09-21T07:36:43.110 INFO:tasks.ceph.mds.c.smithi039.stderr: 14: (DispatchQueue::entry()+0x60f) [0x7f3c2bfa60d1] 2024-09-21T07:36:43.110 INFO:tasks.ceph.mds.c.smithi039.stderr: 15: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f3c2c093f5b] 2024-09-21T07:36:43.110 INFO:tasks.ceph.mds.c.smithi039.stderr: 16: (Thread::entry_wrapper()+0x43) [0x7f3c2be2fd57] 2024-09-21T07:36:43.110 INFO:tasks.ceph.mds.c.smithi039.stderr: 17: (Thread::_entry_func(void*)+0xd) [0x7f3c2be2fd73] 2024-09-21T07:36:43.110 INFO:tasks.ceph.mds.c.smithi039.stderr: 18: /lib64/libc.so.6(+0x89c52) [0x7f3c2b489c52] 2024-09-21T07:36:43.110 INFO:tasks.ceph.mds.c.smithi039.stderr: 19: /lib64/libc.so.6(+0x10ec80) [0x7f3c2b50ec80] 2024-09-21T07:36:43.110 INFO:tasks.ceph.mds.c.smithi039.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:36:43.110 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:36:43.112 INFO:tasks.ceph.mds.c.smithi039.stderr: -3914> 2024-09-21T07:36:22.640+0000 7f3c21c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:36:43.117 INFO:tasks.ceph.mds.c.smithi039.stderr: -1975> 2024-09-21T07:36:32.641+0000 7f3c21c00640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:36:43.121 INFO:tasks.ceph.mds.c.smithi039.stderr: -1> 2024-09-21T07:36:43.100+0000 7f3c24e00640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_prepare(ceph::cref_t&)' thread 7f3c24e00640 time 2024-09-21T07:36:43.100545+0000 2024-09-21T07:36:43.121 INFO:tasks.ceph.mds.c.smithi039.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: 61: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 1) 2024-09-21T07:36:43.121 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:36:43.121 INFO:tasks.ceph.mds.c.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:36:43.121 INFO:tasks.ceph.mds.c.smithi039.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f3c2be5654b] 2024-09-21T07:36:43.121 INFO:tasks.ceph.mds.c.smithi039.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f3c2be56758] 2024-09-21T07:36:43.121 INFO:tasks.ceph.mds.c.smithi039.stderr: 3: (MDSTableServer::handle_prepare(boost::intrusive_ptr const&)+0x312) [0x5580d00b42a0] 2024-09-21T07:36:43.121 INFO:tasks.ceph.mds.c.smithi039.stderr: 4: (MDSTableServer::handle_request(boost::intrusive_ptr const&)+0x93) [0x5580d00b8909] 2024-09-21T07:36:43.121 INFO:tasks.ceph.mds.c.smithi039.stderr: 5: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xca5) [0x5580cfd51de3] 2024-09-21T07:36:43.121 INFO:tasks.ceph.mds.c.smithi039.stderr: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5580cfd55523] 2024-09-21T07:36:43.121 INFO:tasks.ceph.mds.c.smithi039.stderr: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5580cfd55e4a] 2024-09-21T07:36:43.121 INFO:tasks.ceph.mds.c.smithi039.stderr: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x5580cfd38fa4] 2024-09-21T07:36:43.121 INFO:tasks.ceph.mds.c.smithi039.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f3c2bfa57df] 2024-09-21T07:36:43.122 INFO:tasks.ceph.mds.c.smithi039.stderr: 10: (DispatchQueue::entry()+0x60f) [0x7f3c2bfa60d1] 2024-09-21T07:36:43.122 INFO:tasks.ceph.mds.c.smithi039.stderr: 11: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f3c2c093f5b] 2024-09-21T07:36:43.122 INFO:tasks.ceph.mds.c.smithi039.stderr: 12: (Thread::entry_wrapper()+0x43) [0x7f3c2be2fd57] 2024-09-21T07:36:43.122 INFO:tasks.ceph.mds.c.smithi039.stderr: 13: (Thread::_entry_func(void*)+0xd) [0x7f3c2be2fd73] 2024-09-21T07:36:43.122 INFO:tasks.ceph.mds.c.smithi039.stderr: 14: /lib64/libc.so.6(+0x89c52) [0x7f3c2b489c52] 2024-09-21T07:36:43.122 INFO:tasks.ceph.mds.c.smithi039.stderr: 15: /lib64/libc.so.6(+0x10ec80) [0x7f3c2b50ec80] 2024-09-21T07:36:43.122 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:36:43.122 INFO:tasks.ceph.mds.c.smithi039.stderr: 0> 2024-09-21T07:36:43.101+0000 7f3c24e00640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:36:43.122 INFO:tasks.ceph.mds.c.smithi039.stderr: in thread 7f3c24e00640 thread_name:ms_dispatch 2024-09-21T07:36:43.122 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:36:43.122 INFO:tasks.ceph.mds.c.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:36:43.122 INFO:tasks.ceph.mds.c.smithi039.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f3c2b43e6f0] 2024-09-21T07:36:43.122 INFO:tasks.ceph.mds.c.smithi039.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f3c2b48b99c] 2024-09-21T07:36:43.122 INFO:tasks.ceph.mds.c.smithi039.stderr: 3: raise() 2024-09-21T07:36:43.122 INFO:tasks.ceph.mds.c.smithi039.stderr: 4: abort() 2024-09-21T07:36:43.123 INFO:tasks.ceph.mds.c.smithi039.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7f3c2be56686] 2024-09-21T07:36:43.123 INFO:tasks.ceph.mds.c.smithi039.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f3c2be56758] 2024-09-21T07:36:43.123 INFO:tasks.ceph.mds.c.smithi039.stderr: 7: (MDSTableServer::handle_prepare(boost::intrusive_ptr const&)+0x312) [0x5580d00b42a0] 2024-09-21T07:36:43.123 INFO:tasks.ceph.mds.c.smithi039.stderr: 8: (MDSTableServer::handle_request(boost::intrusive_ptr const&)+0x93) [0x5580d00b8909] 2024-09-21T07:36:43.123 INFO:tasks.ceph.mds.c.smithi039.stderr: 9: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xca5) [0x5580cfd51de3] 2024-09-21T07:36:43.123 INFO:tasks.ceph.mds.c.smithi039.stderr: 10: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5580cfd55523] 2024-09-21T07:36:43.123 INFO:tasks.ceph.mds.c.smithi039.stderr: 11: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5580cfd55e4a] 2024-09-21T07:36:43.123 INFO:tasks.ceph.mds.c.smithi039.stderr: 12: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x5580cfd38fa4] 2024-09-21T07:36:43.123 INFO:tasks.ceph.mds.c.smithi039.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f3c2bfa57df] 2024-09-21T07:36:43.123 INFO:tasks.ceph.mds.c.smithi039.stderr: 14: (DispatchQueue::entry()+0x60f) [0x7f3c2bfa60d1] 2024-09-21T07:36:43.123 INFO:tasks.ceph.mds.c.smithi039.stderr: 15: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f3c2c093f5b] 2024-09-21T07:36:43.123 INFO:tasks.ceph.mds.c.smithi039.stderr: 16: (Thread::entry_wrapper()+0x43) [0x7f3c2be2fd57] 2024-09-21T07:36:43.123 INFO:tasks.ceph.mds.c.smithi039.stderr: 17: (Thread::_entry_func(void*)+0xd) [0x7f3c2be2fd73] 2024-09-21T07:36:43.123 INFO:tasks.ceph.mds.c.smithi039.stderr: 18: /lib64/libc.so.6(+0x89c52) [0x7f3c2b489c52] 2024-09-21T07:36:43.124 INFO:tasks.ceph.mds.c.smithi039.stderr: 19: /lib64/libc.so.6(+0x10ec80) [0x7f3c2b50ec80] 2024-09-21T07:36:43.124 INFO:tasks.ceph.mds.c.smithi039.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:36:43.124 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:36:43.124 INFO:tasks.ceph.mds.c.smithi039.stderr: -3914> 2024-09-21T07:36:22.640+0000 7f3c21c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:36:43.125 INFO:tasks.ceph.mds.c.smithi039.stderr: -1975> 2024-09-21T07:36:32.641+0000 7f3c21c00640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:36:43.127 INFO:tasks.ceph.mds.c.smithi039.stderr: -1> 2024-09-21T07:36:43.100+0000 7f3c24e00640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_prepare(ceph::cref_t&)' thread 7f3c24e00640 time 2024-09-21T07:36:43.100545+0000 2024-09-21T07:36:43.127 INFO:tasks.ceph.mds.c.smithi039.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: 61: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 1) 2024-09-21T07:36:43.128 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:36:43.128 INFO:tasks.ceph.mds.c.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:36:43.128 INFO:tasks.ceph.mds.c.smithi039.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f3c2be5654b] 2024-09-21T07:36:43.128 INFO:tasks.ceph.mds.c.smithi039.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f3c2be56758] 2024-09-21T07:36:43.128 INFO:tasks.ceph.mds.c.smithi039.stderr: 3: (MDSTableServer::handle_prepare(boost::intrusive_ptr const&)+0x312) [0x5580d00b42a0] 2024-09-21T07:36:43.128 INFO:tasks.ceph.mds.c.smithi039.stderr: 4: (MDSTableServer::handle_request(boost::intrusive_ptr const&)+0x93) [0x5580d00b8909] 2024-09-21T07:36:43.128 INFO:tasks.ceph.mds.c.smithi039.stderr: 5: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xca5) [0x5580cfd51de3] 2024-09-21T07:36:43.128 INFO:tasks.ceph.mds.c.smithi039.stderr: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5580cfd55523] 2024-09-21T07:36:43.128 INFO:tasks.ceph.mds.c.smithi039.stderr: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5580cfd55e4a] 2024-09-21T07:36:43.128 INFO:tasks.ceph.mds.c.smithi039.stderr: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x5580cfd38fa4] 2024-09-21T07:36:43.128 INFO:tasks.ceph.mds.c.smithi039.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f3c2bfa57df] 2024-09-21T07:36:43.128 INFO:tasks.ceph.mds.c.smithi039.stderr: 10: (DispatchQueue::entry()+0x60f) [0x7f3c2bfa60d1] 2024-09-21T07:36:43.128 INFO:tasks.ceph.mds.c.smithi039.stderr: 11: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f3c2c093f5b] 2024-09-21T07:36:43.128 INFO:tasks.ceph.mds.c.smithi039.stderr: 12: (Thread::entry_wrapper()+0x43) [0x7f3c2be2fd57] 2024-09-21T07:36:43.129 INFO:tasks.ceph.mds.c.smithi039.stderr: 13: (Thread::_entry_func(void*)+0xd) [0x7f3c2be2fd73] 2024-09-21T07:36:43.129 INFO:tasks.ceph.mds.c.smithi039.stderr: 14: /lib64/libc.so.6(+0x89c52) [0x7f3c2b489c52] 2024-09-21T07:36:43.129 INFO:tasks.ceph.mds.c.smithi039.stderr: 15: /lib64/libc.so.6(+0x10ec80) [0x7f3c2b50ec80] 2024-09-21T07:36:43.129 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:36:43.129 INFO:tasks.ceph.mds.c.smithi039.stderr: 0> 2024-09-21T07:36:43.101+0000 7f3c24e00640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:36:43.129 INFO:tasks.ceph.mds.c.smithi039.stderr: in thread 7f3c24e00640 thread_name:ms_dispatch 2024-09-21T07:36:43.129 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:36:43.129 INFO:tasks.ceph.mds.c.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:36:43.129 INFO:tasks.ceph.mds.c.smithi039.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f3c2b43e6f0] 2024-09-21T07:36:43.129 INFO:tasks.ceph.mds.c.smithi039.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f3c2b48b99c] 2024-09-21T07:36:43.129 INFO:tasks.ceph.mds.c.smithi039.stderr: 3: raise() 2024-09-21T07:36:43.129 INFO:tasks.ceph.mds.c.smithi039.stderr: 4: abort() 2024-09-21T07:36:43.129 INFO:tasks.ceph.mds.c.smithi039.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7f3c2be56686] 2024-09-21T07:36:43.129 INFO:tasks.ceph.mds.c.smithi039.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f3c2be56758] 2024-09-21T07:36:43.129 INFO:tasks.ceph.mds.c.smithi039.stderr: 7: (MDSTableServer::handle_prepare(boost::intrusive_ptr const&)+0x312) [0x5580d00b42a0] 2024-09-21T07:36:43.130 INFO:tasks.ceph.mds.c.smithi039.stderr: 8: (MDSTableServer::handle_request(boost::intrusive_ptr const&)+0x93) [0x5580d00b8909] 2024-09-21T07:36:43.130 INFO:tasks.ceph.mds.c.smithi039.stderr: 9: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xca5) [0x5580cfd51de3] 2024-09-21T07:36:43.130 INFO:tasks.ceph.mds.c.smithi039.stderr: 10: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5580cfd55523] 2024-09-21T07:36:43.130 INFO:tasks.ceph.mds.c.smithi039.stderr: 11: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5580cfd55e4a] 2024-09-21T07:36:43.130 INFO:tasks.ceph.mds.c.smithi039.stderr: 12: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x5580cfd38fa4] 2024-09-21T07:36:43.130 INFO:tasks.ceph.mds.c.smithi039.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f3c2bfa57df] 2024-09-21T07:36:43.130 INFO:tasks.ceph.mds.c.smithi039.stderr: 14: (DispatchQueue::entry()+0x60f) [0x7f3c2bfa60d1] 2024-09-21T07:36:43.130 INFO:tasks.ceph.mds.c.smithi039.stderr: 15: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f3c2c093f5b] 2024-09-21T07:36:43.130 INFO:tasks.ceph.mds.c.smithi039.stderr: 16: (Thread::entry_wrapper()+0x43) [0x7f3c2be2fd57] 2024-09-21T07:36:43.130 INFO:tasks.ceph.mds.c.smithi039.stderr: 17: (Thread::_entry_func(void*)+0xd) [0x7f3c2be2fd73] 2024-09-21T07:36:43.130 INFO:tasks.ceph.mds.c.smithi039.stderr: 18: /lib64/libc.so.6(+0x89c52) [0x7f3c2b489c52] 2024-09-21T07:36:43.130 INFO:tasks.ceph.mds.c.smithi039.stderr: 19: /lib64/libc.so.6(+0x10ec80) [0x7f3c2b50ec80] 2024-09-21T07:36:43.130 INFO:tasks.ceph.mds.c.smithi039.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:36:43.130 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:36:43.337 INFO:teuthology.orchestra.run.smithi039.stdout:{"mds_beacon_grace":"15.000000"} 2024-09-21T07:36:43.337 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-09-21T07:36:43.337 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:36:43.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.601+0000 7fc1b3000640 1 -- 172.21.15.39:0/50092552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1ac0cc1e0 msgr2=0x7fc1ac0af3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:43.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.601+0000 7fc1b3000640 1 --2- 172.21.15.39:0/50092552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1ac0cc1e0 0x7fc1ac0af3c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc19c009830 tx=0x7fc19c02a540 comp rx=0 tx=0).stop 2024-09-21T07:36:43.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.601+0000 7fc1b3000640 1 -- 172.21.15.39:0/50092552 shutdown_connections 2024-09-21T07:36:43.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.601+0000 7fc1b3000640 1 --2- 172.21.15.39:0/50092552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1ac0cc1e0 0x7fc1ac0af3c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:43.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.601+0000 7fc1b3000640 1 -- 172.21.15.39:0/50092552 >> 172.21.15.39:0/50092552 conn(0x7fc1ac06d7d0 msgr2=0x7fc1ac0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:43.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.601+0000 7fc1b3000640 1 -- 172.21.15.39:0/50092552 shutdown_connections 2024-09-21T07:36:43.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.601+0000 7fc1b3000640 1 -- 172.21.15.39:0/50092552 wait complete. 2024-09-21T07:36:43.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.603+0000 7fc1b3000640 1 Processor -- start 2024-09-21T07:36:43.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.604+0000 7fc1b3000640 1 -- start start 2024-09-21T07:36:43.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.605+0000 7fc1b3000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1ac0cc1e0 0x7fc1ac0ae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:43.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.605+0000 7fc1b3000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc1ac0ccb50 0x7fc1ac073e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:43.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.605+0000 7fc1b3000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1ac15d8a0 0x7fc1ac0743c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:43.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.605+0000 7fc1b3000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ac153520 con 0x7fc1ac15d8a0 2024-09-21T07:36:43.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.605+0000 7fc1b3000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ac1533a0 con 0x7fc1ac0cc1e0 2024-09-21T07:36:43.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.605+0000 7fc1b3000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc1ac1536a0 con 0x7fc1ac0ccb50 2024-09-21T07:36:43.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.605+0000 7fc1b1000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1ac15d8a0 0x7fc1ac0743c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:43.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.605+0000 7fc1b1000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1ac15d8a0 0x7fc1ac0743c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54896/0 (socket says 172.21.15.39:54896) 2024-09-21T07:36:43.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.605+0000 7fc1b1000640 1 -- 172.21.15.39:0/2920362001 learned_addr learned my addr 172.21.15.39:0/2920362001 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:43.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.606+0000 7fc1abe00640 1 --2- 172.21.15.39:0/2920362001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1ac0cc1e0 0x7fc1ac0ae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:43.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.606+0000 7fc1ab400640 1 --2- 172.21.15.39:0/2920362001 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc1ac0ccb50 0x7fc1ac073e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:43.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.606+0000 7fc1b1000640 1 -- 172.21.15.39:0/2920362001 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc1ac0ccb50 msgr2=0x7fc1ac073e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:43.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.606+0000 7fc1b1000640 1 --2- 172.21.15.39:0/2920362001 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc1ac0ccb50 0x7fc1ac073e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:43.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.606+0000 7fc1b1000640 1 -- 172.21.15.39:0/2920362001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1ac0cc1e0 msgr2=0x7fc1ac0ae9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:43.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.606+0000 7fc1b1000640 1 --2- 172.21.15.39:0/2920362001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1ac0cc1e0 0x7fc1ac0ae9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:43.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.606+0000 7fc1b1000640 1 -- 172.21.15.39:0/2920362001 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1ac1f9420 con 0x7fc1ac15d8a0 2024-09-21T07:36:43.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.606+0000 7fc1b1000640 1 --2- 172.21.15.39:0/2920362001 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1ac15d8a0 0x7fc1ac0743c0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4002230 tx=0x7fc1a400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:43.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.606+0000 7fc1a3e00640 1 -- 172.21.15.39:0/2920362001 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a4016020 con 0x7fc1ac15d8a0 2024-09-21T07:36:43.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.606+0000 7fc1a3e00640 1 -- 172.21.15.39:0/2920362001 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1a400adf0 con 0x7fc1ac15d8a0 2024-09-21T07:36:43.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.606+0000 7fc1b3000640 1 -- 172.21.15.39:0/2920362001 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1ac1f9700 con 0x7fc1ac15d8a0 2024-09-21T07:36:43.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.607+0000 7fc1a3e00640 1 -- 172.21.15.39:0/2920362001 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a4012660 con 0x7fc1ac15d8a0 2024-09-21T07:36:43.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.607+0000 7fc1b3000640 1 -- 172.21.15.39:0/2920362001 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1ac1f9cc0 con 0x7fc1ac15d8a0 2024-09-21T07:36:43.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.608+0000 7fc1b3000640 1 -- 172.21.15.39:0/2920362001 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1ac153b50 con 0x7fc1ac15d8a0 2024-09-21T07:36:43.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.609+0000 7fc1a3e00640 1 -- 172.21.15.39:0/2920362001 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc1a4019020 con 0x7fc1ac15d8a0 2024-09-21T07:36:43.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.611+0000 7fc1a3e00640 1 --2- 172.21.15.39:0/2920362001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc18c078bc0 0x7fc18c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:43.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.611+0000 7fc1a3e00640 1 -- 172.21.15.39:0/2920362001 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6800+0+0 (secure 0 0 0) 0x7fc1a409b740 con 0x7fc1ac15d8a0 2024-09-21T07:36:43.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.611+0000 7fc1abe00640 1 --2- 172.21.15.39:0/2920362001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc18c078bc0 0x7fc18c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:43.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.612+0000 7fc1abe00640 1 --2- 172.21.15.39:0/2920362001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc18c078bc0 0x7fc18c07b0d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc19c034000 tx=0x7fc19c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:43.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.615+0000 7fc1a3e00640 1 -- 172.21.15.39:0/2920362001 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc1a4020020 con 0x7fc1ac15d8a0 2024-09-21T07:36:43.819 INFO:tasks.ceph.mds.c.smithi039.stderr:daemon-helper: command crashed with signal 6 2024-09-21T07:36:43.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.863+0000 7fc1b3000640 1 -- 172.21.15.39:0/2920362001 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1ac09e0b0 con 0x7fc1ac15d8a0 2024-09-21T07:36:43.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.863+0000 7fc1a3e00640 1 -- 172.21.15.39:0/2920362001 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5245 (secure 0 0 0) 0x7fc1a406ae20 con 0x7fc1ac15d8a0 2024-09-21T07:36:43.865 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:36:43.865 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":59,"btime":"2024-09-21T07:36:42:791796+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5621,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3026827614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3026827614},{"type":"v1","addr":"172.21.15.148:6837","nonce":3026827614}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:36:42.549487+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6321,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6321":{"gid":6321,"name":"c","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/829716143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":829716143},{"type":"v1","addr":"172.21.15.39:6834","nonce":829716143}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6321,"qdb_cluster":[6115,6321]},"id":6}]} 2024-09-21T07:36:43.866 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 59 2024-09-21T07:36:43.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.866+0000 7fc1b3000640 1 -- 172.21.15.39:0/2920362001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc18c078bc0 msgr2=0x7fc18c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:43.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.866+0000 7fc1b3000640 1 --2- 172.21.15.39:0/2920362001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc18c078bc0 0x7fc18c07b0d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc19c034000 tx=0x7fc19c004730 comp rx=0 tx=0).stop 2024-09-21T07:36:43.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.866+0000 7fc1b3000640 1 -- 172.21.15.39:0/2920362001 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1ac15d8a0 msgr2=0x7fc1ac0743c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:43.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.866+0000 7fc1b3000640 1 --2- 172.21.15.39:0/2920362001 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1ac15d8a0 0x7fc1ac0743c0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4002230 tx=0x7fc1a400a430 comp rx=0 tx=0).stop 2024-09-21T07:36:43.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.866+0000 7fc1b3000640 1 -- 172.21.15.39:0/2920362001 shutdown_connections 2024-09-21T07:36:43.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.866+0000 7fc1b3000640 1 --2- 172.21.15.39:0/2920362001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc18c078bc0 0x7fc18c07b0d0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:43.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.866+0000 7fc1b3000640 1 --2- 172.21.15.39:0/2920362001 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1ac15d8a0 0x7fc1ac0743c0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:43.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.866+0000 7fc1b3000640 1 --2- 172.21.15.39:0/2920362001 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc1ac0ccb50 0x7fc1ac073e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:43.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.866+0000 7fc1b3000640 1 --2- 172.21.15.39:0/2920362001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1ac0cc1e0 0x7fc1ac0ae9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:43.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.866+0000 7fc1b3000640 1 -- 172.21.15.39:0/2920362001 >> 172.21.15.39:0/2920362001 conn(0x7fc1ac06d7d0 msgr2=0x7fc1ac15c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:43.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.866+0000 7fc1b3000640 1 -- 172.21.15.39:0/2920362001 shutdown_connections 2024-09-21T07:36:43.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:43.866+0000 7fc1b3000640 1 -- 172.21.15.39:0/2920362001 wait complete. 2024-09-21T07:36:43.893 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:36:48.893 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:36:49.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.128+0000 7f241c000640 1 -- 172.21.15.39:0/3671933731 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24140cfb90 msgr2=0x7f24140ccb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:49.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.128+0000 7f241c000640 1 --2- 172.21.15.39:0/3671933731 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24140cfb90 0x7f24140ccb30 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f241409e2c0 tx=0x7f240002a530 comp rx=0 tx=0).stop 2024-09-21T07:36:49.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.128+0000 7f241c000640 1 -- 172.21.15.39:0/3671933731 shutdown_connections 2024-09-21T07:36:49.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.128+0000 7f241c000640 1 --2- 172.21.15.39:0/3671933731 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24140cfb90 0x7f24140ccb30 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:49.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.128+0000 7f241c000640 1 --2- 172.21.15.39:0/3671933731 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f24140cf660 0x7f24140cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:49.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.128+0000 7f241c000640 1 -- 172.21.15.39:0/3671933731 >> 172.21.15.39:0/3671933731 conn(0x7f24140b1cb0 msgr2=0x7f24140b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:49.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.128+0000 7f241c000640 1 -- 172.21.15.39:0/3671933731 shutdown_connections 2024-09-21T07:36:49.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.128+0000 7f241c000640 1 -- 172.21.15.39:0/3671933731 wait complete. 2024-09-21T07:36:49.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.131+0000 7f241c000640 1 Processor -- start 2024-09-21T07:36:49.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.131+0000 7f241c000640 1 -- start start 2024-09-21T07:36:49.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.132+0000 7f241c000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f24140cd070 0x7f24140ae710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:49.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.132+0000 7f241c000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24140cf660 0x7f2414156f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:49.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.132+0000 7f241c000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24140cfb90 0x7f24141574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:49.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.132+0000 7f241c000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2414070f20 con 0x7f24140cf660 2024-09-21T07:36:49.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.132+0000 7f241c000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f24140af810 con 0x7f24140cfb90 2024-09-21T07:36:49.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.132+0000 7f241c000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f24140710a0 con 0x7f24140cd070 2024-09-21T07:36:49.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.132+0000 7f2418c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24140cf660 0x7f2414156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:49.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.132+0000 7f2418c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24140cf660 0x7f2414156f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54906/0 (socket says 172.21.15.39:54906) 2024-09-21T07:36:49.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.132+0000 7f2418c00640 1 -- 172.21.15.39:0/3488049246 learned_addr learned my addr 172.21.15.39:0/3488049246 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:49.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.132+0000 7f2419600640 1 --2- 172.21.15.39:0/3488049246 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f24140cd070 0x7f24140ae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:49.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.133+0000 7f241a000640 1 --2- 172.21.15.39:0/3488049246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24140cfb90 0x7f24141574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:49.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.133+0000 7f2418c00640 1 -- 172.21.15.39:0/3488049246 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f24140cd070 msgr2=0x7f24140ae710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:49.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.133+0000 7f2418c00640 1 --2- 172.21.15.39:0/3488049246 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f24140cd070 0x7f24140ae710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:49.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.133+0000 7f2418c00640 1 -- 172.21.15.39:0/3488049246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24140cfb90 msgr2=0x7f24141574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:49.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.133+0000 7f2418c00640 1 --2- 172.21.15.39:0/3488049246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24140cfb90 0x7f24141574b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:49.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.133+0000 7f2418c00640 1 -- 172.21.15.39:0/3488049246 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f241415be60 con 0x7f24140cf660 2024-09-21T07:36:49.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.133+0000 7f241a000640 1 --2- 172.21.15.39:0/3488049246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24140cfb90 0x7f24141574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:36:49.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.133+0000 7f2418c00640 1 --2- 172.21.15.39:0/3488049246 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24140cf660 0x7f2414156f70 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f2400000c00 tx=0x7f2400003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:49.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.134+0000 7f2411600640 1 -- 172.21.15.39:0/3488049246 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240003e030 con 0x7f24140cf660 2024-09-21T07:36:49.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.134+0000 7f2411600640 1 -- 172.21.15.39:0/3488049246 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2400003d10 con 0x7f24140cf660 2024-09-21T07:36:49.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.134+0000 7f2411600640 1 -- 172.21.15.39:0/3488049246 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2400035040 con 0x7f24140cf660 2024-09-21T07:36:49.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.134+0000 7f241c000640 1 -- 172.21.15.39:0/3488049246 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f241415c140 con 0x7f24140cf660 2024-09-21T07:36:49.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.134+0000 7f241c000640 1 -- 172.21.15.39:0/3488049246 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f241415c700 con 0x7f24140cf660 2024-09-21T07:36:49.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.135+0000 7f241c000640 1 -- 172.21.15.39:0/3488049246 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24140cf010 con 0x7f24140cf660 2024-09-21T07:36:49.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.136+0000 7f2411600640 1 -- 172.21.15.39:0/3488049246 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2400041030 con 0x7f24140cf660 2024-09-21T07:36:49.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.137+0000 7f2411600640 1 --2- 172.21.15.39:0/3488049246 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f23f4078b90 0x7f23f407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:49.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.137+0000 7f2411600640 1 -- 172.21.15.39:0/3488049246 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6800+0+0 (secure 0 0 0) 0x7f24000bb7e0 con 0x7f24140cf660 2024-09-21T07:36:49.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.138+0000 7f2419600640 1 --2- 172.21.15.39:0/3488049246 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f23f4078b90 0x7f23f407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:49.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.138+0000 7f2419600640 1 --2- 172.21.15.39:0/3488049246 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f23f4078b90 0x7f23f407b0a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f24140c3830 tx=0x7f2408005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:49.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.142+0000 7f2411600640 1 -- 172.21.15.39:0/3488049246 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f24000831a0 con 0x7f24140cf660 2024-09-21T07:36:49.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.375+0000 7f241c000640 1 -- 172.21.15.39:0/3488049246 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24140b0d70 con 0x7f24140cf660 2024-09-21T07:36:49.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.375+0000 7f2411600640 1 -- 172.21.15.39:0/3488049246 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5245 (secure 0 0 0) 0x7f2400087bd0 con 0x7f24140cf660 2024-09-21T07:36:49.377 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:36:49.377 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":59,"btime":"2024-09-21T07:36:42:791796+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5621,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3026827614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3026827614},{"type":"v1","addr":"172.21.15.148:6837","nonce":3026827614}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:36:42.549487+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6321,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6321":{"gid":6321,"name":"c","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/829716143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":829716143},{"type":"v1","addr":"172.21.15.39:6834","nonce":829716143}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6321,"qdb_cluster":[6115,6321]},"id":6}]} 2024-09-21T07:36:49.378 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 59 2024-09-21T07:36:49.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.378+0000 7f241c000640 1 -- 172.21.15.39:0/3488049246 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f23f4078b90 msgr2=0x7f23f407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:49.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.378+0000 7f241c000640 1 --2- 172.21.15.39:0/3488049246 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f23f4078b90 0x7f23f407b0a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f24140c3830 tx=0x7f2408005f90 comp rx=0 tx=0).stop 2024-09-21T07:36:49.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.378+0000 7f241c000640 1 -- 172.21.15.39:0/3488049246 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24140cf660 msgr2=0x7f2414156f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:49.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.378+0000 7f241c000640 1 --2- 172.21.15.39:0/3488049246 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24140cf660 0x7f2414156f70 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f2400000c00 tx=0x7f2400003820 comp rx=0 tx=0).stop 2024-09-21T07:36:49.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.378+0000 7f241c000640 1 -- 172.21.15.39:0/3488049246 shutdown_connections 2024-09-21T07:36:49.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.378+0000 7f241c000640 1 --2- 172.21.15.39:0/3488049246 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f23f4078b90 0x7f23f407b0a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:49.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.378+0000 7f241c000640 1 --2- 172.21.15.39:0/3488049246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24140cfb90 0x7f24141574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:49.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.378+0000 7f241c000640 1 --2- 172.21.15.39:0/3488049246 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24140cf660 0x7f2414156f70 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:49.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.378+0000 7f241c000640 1 --2- 172.21.15.39:0/3488049246 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f24140cd070 0x7f24140ae710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:49.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.378+0000 7f241c000640 1 -- 172.21.15.39:0/3488049246 >> 172.21.15.39:0/3488049246 conn(0x7f24140b1cb0 msgr2=0x7f24140b0660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:49.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.378+0000 7f241c000640 1 -- 172.21.15.39:0/3488049246 shutdown_connections 2024-09-21T07:36:49.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:49.378+0000 7f241c000640 1 -- 172.21.15.39:0/3488049246 wait complete. 2024-09-21T07:36:49.405 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:36:49.825 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-09-21T07:36:54.406 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:36:54.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.643+0000 7f7b53200640 1 -- 172.21.15.39:0/519536600 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b38004d90 msgr2=0x7f7b38005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:54.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.643+0000 7f7b53200640 1 --2- 172.21.15.39:0/519536600 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b38004d90 0x7f7b38005220 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f7b540b2610 tx=0x7f7b38005830 comp rx=0 tx=0).stop 2024-09-21T07:36:54.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.643+0000 7f7b52800640 1 -- 172.21.15.39:0/519536600 reap_dead start 2024-09-21T07:36:54.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.643+0000 7f7b53200640 1 -- 172.21.15.39:0/519536600 shutdown_connections 2024-09-21T07:36:54.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.643+0000 7f7b53200640 1 -- 172.21.15.39:0/519536600 >> 172.21.15.39:0/519536600 conn(0x7f7b5406d410 msgr2=0x7f7b540b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:54.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.643+0000 7f7b53200640 1 -- 172.21.15.39:0/519536600 shutdown_connections 2024-09-21T07:36:54.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.643+0000 7f7b53200640 1 -- 172.21.15.39:0/519536600 wait complete. 2024-09-21T07:36:54.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.646+0000 7f7b53200640 1 Processor -- start 2024-09-21T07:36:54.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.646+0000 7f7b53200640 1 -- start start 2024-09-21T07:36:54.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.647+0000 7f7b53200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b38004d90 0x7f7b54152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:54.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.647+0000 7f7b53200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b54142050 0x7f7b541e8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:54.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.647+0000 7f7b53200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b54142460 0x7f7b541e93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:54.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.647+0000 7f7b53200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7b540ccd00 con 0x7f7b54142460 2024-09-21T07:36:54.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.647+0000 7f7b53200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7b540ccb80 con 0x7f7b54142050 2024-09-21T07:36:54.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.647+0000 7f7b53200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f7b540cce80 con 0x7f7b38004d90 2024-09-21T07:36:54.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.647+0000 7f7b52800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b54142460 0x7f7b541e93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:54.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.648+0000 7f7b52800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b54142460 0x7f7b541e93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48340/0 (socket says 172.21.15.39:48340) 2024-09-21T07:36:54.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.648+0000 7f7b52800640 1 -- 172.21.15.39:0/1412550053 learned_addr learned my addr 172.21.15.39:0/1412550053 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:36:54.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.648+0000 7f7b4be00640 1 --2- 172.21.15.39:0/1412550053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b54142050 0x7f7b541e8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:54.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.648+0000 7f7b51e00640 1 --2- 172.21.15.39:0/1412550053 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b38004d90 0x7f7b54152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:54.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.648+0000 7f7b52800640 1 -- 172.21.15.39:0/1412550053 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b38004d90 msgr2=0x7f7b54152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:54.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.648+0000 7f7b52800640 1 --2- 172.21.15.39:0/1412550053 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b38004d90 0x7f7b54152f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:54.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.648+0000 7f7b52800640 1 -- 172.21.15.39:0/1412550053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b54142050 msgr2=0x7f7b541e8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:54.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.648+0000 7f7b52800640 1 --2- 172.21.15.39:0/1412550053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b54142050 0x7f7b541e8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:54.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.648+0000 7f7b52800640 1 -- 172.21.15.39:0/1412550053 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b5413cd00 con 0x7f7b54142460 2024-09-21T07:36:54.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.648+0000 7f7b52800640 1 --2- 172.21.15.39:0/1412550053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b54142460 0x7f7b541e93f0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c00c620 tx=0x7f7b4c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:54.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.649+0000 7f7b4a000640 1 -- 172.21.15.39:0/1412550053 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b44009720 con 0x7f7b54142460 2024-09-21T07:36:54.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.649+0000 7f7b4a000640 1 -- 172.21.15.39:0/1412550053 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b4c02b6d0 con 0x7f7b54142460 2024-09-21T07:36:54.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.649+0000 7f7b4a000640 1 -- 172.21.15.39:0/1412550053 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c02bb00 con 0x7f7b54142460 2024-09-21T07:36:54.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.649+0000 7f7b53200640 1 -- 172.21.15.39:0/1412550053 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b54142c30 con 0x7f7b54142460 2024-09-21T07:36:54.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.649+0000 7f7b53200640 1 -- 172.21.15.39:0/1412550053 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b541431f0 con 0x7f7b54142460 2024-09-21T07:36:54.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.650+0000 7f7b53200640 1 -- 172.21.15.39:0/1412550053 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b54142980 con 0x7f7b54142460 2024-09-21T07:36:54.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.651+0000 7f7b4a000640 1 -- 172.21.15.39:0/1412550053 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7b4c02bca0 con 0x7f7b54142460 2024-09-21T07:36:54.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.652+0000 7f7b4a000640 1 --2- 172.21.15.39:0/1412550053 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7b30078b90 0x7f7b3007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:36:54.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.652+0000 7f7b4a000640 1 -- 172.21.15.39:0/1412550053 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6829+0+0 (secure 0 0 0) 0x7f7b4c0bad30 con 0x7f7b54142460 2024-09-21T07:36:54.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.652+0000 7f7b51e00640 1 --2- 172.21.15.39:0/1412550053 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7b30078b90 0x7f7b3007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:36:54.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.653+0000 7f7b51e00640 1 --2- 172.21.15.39:0/1412550053 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7b30078b90 0x7f7b3007b0a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7b44009df0 tx=0x7f7b44005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:36:54.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.657+0000 7f7b4a000640 1 -- 172.21.15.39:0/1412550053 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7b5413cd00 con 0x7f7b54142460 2024-09-21T07:36:54.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.899+0000 7f7b53200640 1 -- 172.21.15.39:0/1412550053 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b5406fb50 con 0x7f7b54142460 2024-09-21T07:36:54.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.900+0000 7f7b4a000640 1 -- 172.21.15.39:0/1412550053 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5245 (secure 0 0 0) 0x7f7b4c082620 con 0x7f7b54142460 2024-09-21T07:36:54.902 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:36:54.902 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":59,"btime":"2024-09-21T07:36:42:791796+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5621,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3026827614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3026827614},{"type":"v1","addr":"172.21.15.148:6837","nonce":3026827614}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:36:42.549487+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6321,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6321":{"gid":6321,"name":"c","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/829716143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":829716143},{"type":"v1","addr":"172.21.15.39:6834","nonce":829716143}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6321,"qdb_cluster":[6115,6321]},"id":6}]} 2024-09-21T07:36:54.902 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 59 2024-09-21T07:36:54.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.902+0000 7f7b53200640 1 -- 172.21.15.39:0/1412550053 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7b30078b90 msgr2=0x7f7b3007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:54.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.902+0000 7f7b53200640 1 --2- 172.21.15.39:0/1412550053 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7b30078b90 0x7f7b3007b0a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7b44009df0 tx=0x7f7b44005f90 comp rx=0 tx=0).stop 2024-09-21T07:36:54.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.902+0000 7f7b53200640 1 -- 172.21.15.39:0/1412550053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b54142460 msgr2=0x7f7b541e93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:36:54.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.902+0000 7f7b53200640 1 --2- 172.21.15.39:0/1412550053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b54142460 0x7f7b541e93f0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c00c620 tx=0x7f7b4c00c2d0 comp rx=0 tx=0).stop 2024-09-21T07:36:54.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.902+0000 7f7b53200640 1 -- 172.21.15.39:0/1412550053 shutdown_connections 2024-09-21T07:36:54.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.902+0000 7f7b53200640 1 --2- 172.21.15.39:0/1412550053 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7b30078b90 0x7f7b3007b0a0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:54.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.902+0000 7f7b53200640 1 --2- 172.21.15.39:0/1412550053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b54142460 0x7f7b541e93f0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:54.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.902+0000 7f7b53200640 1 --2- 172.21.15.39:0/1412550053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b54142050 0x7f7b541e8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:54.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.902+0000 7f7b53200640 1 --2- 172.21.15.39:0/1412550053 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b38004d90 0x7f7b54152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:36:54.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.903+0000 7f7b53200640 1 -- 172.21.15.39:0/1412550053 >> 172.21.15.39:0/1412550053 conn(0x7f7b5406d410 msgr2=0x7f7b54073b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:36:54.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.903+0000 7f7b53200640 1 -- 172.21.15.39:0/1412550053 shutdown_connections 2024-09-21T07:36:54.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:36:54.903+0000 7f7b53200640 1 -- 172.21.15.39:0/1412550053 wait complete. 2024-09-21T07:36:54.927 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:36:56.436 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~7s 2024-09-21T07:36:59.928 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:37:00.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.166+0000 7f502d600640 1 -- 172.21.15.39:0/3540099389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5028073170 msgr2=0x7f502815c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:00.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.166+0000 7f502d600640 1 --2- 172.21.15.39:0/3540099389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5028073170 0x7f502815c780 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f50280710f0 tx=0x7f501802a560 comp rx=0 tx=0).stop 2024-09-21T07:37:00.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.166+0000 7f502d600640 1 -- 172.21.15.39:0/3540099389 shutdown_connections 2024-09-21T07:37:00.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.166+0000 7f502d600640 1 --2- 172.21.15.39:0/3540099389 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5028070440 0x7f5028074b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:00.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.166+0000 7f502d600640 1 --2- 172.21.15.39:0/3540099389 >> v2:172.21.15.148:3300/0 conn(0x7f5028070030 0x7f5028074640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:00.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.166+0000 7f502d600640 1 --2- 172.21.15.39:0/3540099389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5028073170 0x7f502815c780 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:00.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.166+0000 7f502d600640 1 -- 172.21.15.39:0/3540099389 >> 172.21.15.39:0/3540099389 conn(0x7f50280b1cb0 msgr2=0x7f50280b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:00.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.166+0000 7f502d600640 1 -- 172.21.15.39:0/3540099389 shutdown_connections 2024-09-21T07:37:00.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.166+0000 7f502d600640 1 -- 172.21.15.39:0/3540099389 wait complete. 2024-09-21T07:37:00.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.169+0000 7f502d600640 1 Processor -- start 2024-09-21T07:37:00.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.169+0000 7f502d600640 1 -- start start 2024-09-21T07:37:00.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.170+0000 7f502d600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5028070030 0x7f5028149450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:00.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.170+0000 7f502d600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5028070440 0x7f502815b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:00.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.170+0000 7f502d600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5028073170 0x7f502815b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:00.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.170+0000 7f502d600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f502806f630 con 0x7f5028070030 2024-09-21T07:37:00.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.170+0000 7f502d600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f502806f4b0 con 0x7f5028073170 2024-09-21T07:37:00.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.170+0000 7f502d600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f502806f7b0 con 0x7f5028070440 2024-09-21T07:37:00.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.170+0000 7f5026a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5028070030 0x7f5028149450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:00.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.170+0000 7f5026a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5028070030 0x7f5028149450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48362/0 (socket says 172.21.15.39:48362) 2024-09-21T07:37:00.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.170+0000 7f5026a00640 1 -- 172.21.15.39:0/2921697465 learned_addr learned my addr 172.21.15.39:0/2921697465 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:00.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.171+0000 7f5027400640 1 --2- 172.21.15.39:0/2921697465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5028073170 0x7f502815b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:00.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.171+0000 7f501e000640 1 --2- 172.21.15.39:0/2921697465 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5028070440 0x7f502815b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:00.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.171+0000 7f5026a00640 1 -- 172.21.15.39:0/2921697465 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5028070440 msgr2=0x7f502815b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:00.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.171+0000 7f5026a00640 1 --2- 172.21.15.39:0/2921697465 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5028070440 0x7f502815b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:00.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.171+0000 7f5026a00640 1 -- 172.21.15.39:0/2921697465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5028073170 msgr2=0x7f502815b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:00.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.171+0000 7f5026a00640 1 --2- 172.21.15.39:0/2921697465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5028073170 0x7f502815b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:00.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.171+0000 7f5026a00640 1 -- 172.21.15.39:0/2921697465 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f502809cbe0 con 0x7f5028070030 2024-09-21T07:37:00.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.171+0000 7f5026a00640 1 --2- 172.21.15.39:0/2921697465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5028070030 0x7f5028149450 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f5018000c00 tx=0x7f5018003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:00.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.171+0000 7f501f400640 1 -- 172.21.15.39:0/2921697465 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501803e030 con 0x7f5028070030 2024-09-21T07:37:00.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.172+0000 7f501f400640 1 -- 172.21.15.39:0/2921697465 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5018003e00 con 0x7f5028070030 2024-09-21T07:37:00.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.172+0000 7f501f400640 1 -- 172.21.15.39:0/2921697465 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5018035040 con 0x7f5028070030 2024-09-21T07:37:00.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.172+0000 7f502d600640 1 -- 172.21.15.39:0/2921697465 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50280d0db0 con 0x7f5028070030 2024-09-21T07:37:00.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.172+0000 7f502d600640 1 -- 172.21.15.39:0/2921697465 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f502809d140 con 0x7f5028070030 2024-09-21T07:37:00.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.173+0000 7f502d600640 1 -- 172.21.15.39:0/2921697465 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f502815c780 con 0x7f5028070030 2024-09-21T07:37:00.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.174+0000 7f501f400640 1 -- 172.21.15.39:0/2921697465 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5018041030 con 0x7f5028070030 2024-09-21T07:37:00.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.176+0000 7f501f400640 1 --2- 172.21.15.39:0/2921697465 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5004078b90 0x7f500407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:00.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.176+0000 7f501f400640 1 -- 172.21.15.39:0/2921697465 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6829+0+0 (secure 0 0 0) 0x7f50180bb9d0 con 0x7f5028070030 2024-09-21T07:37:00.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.176+0000 7f501e000640 1 --2- 172.21.15.39:0/2921697465 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5004078b90 0x7f500407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:00.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.177+0000 7f501e000640 1 --2- 172.21.15.39:0/2921697465 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5004078b90 0x7f500407b0a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5010005fd0 tx=0x7f5010005c50 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:00.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.180+0000 7f501f400640 1 -- 172.21.15.39:0/2921697465 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f502809cbe0 con 0x7f5028070030 2024-09-21T07:37:00.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.419+0000 7f502d600640 1 -- 172.21.15.39:0/2921697465 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50280c3860 con 0x7f5028070030 2024-09-21T07:37:00.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.420+0000 7f501f400640 1 -- 172.21.15.39:0/2921697465 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+5293 (secure 0 0 0) 0x7f50180b9e20 con 0x7f5028070030 2024-09-21T07:37:00.422 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:37:00.422 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":60,"btime":"2024-09-21T07:36:56:365886+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5621,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3026827614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3026827614},{"type":"v1","addr":"172.21.15.148:6837","nonce":3026827614}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:36:56.365839+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6321,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6321":{"gid":6321,"name":"c","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/829716143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":829716143},{"type":"v1","addr":"172.21.15.39:6834","nonce":829716143}]},"join_fscid":-1,"laggy_since":"2024-09-21T07:36:56.365839+0000","export_targets":[1],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6321,"qdb_cluster":[6321,6115]},"id":6}]} 2024-09-21T07:37:00.422 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 60 2024-09-21T07:37:00.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.422+0000 7f502d600640 1 -- 172.21.15.39:0/2921697465 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5004078b90 msgr2=0x7f500407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:00.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.422+0000 7f502d600640 1 --2- 172.21.15.39:0/2921697465 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5004078b90 0x7f500407b0a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5010005fd0 tx=0x7f5010005c50 comp rx=0 tx=0).stop 2024-09-21T07:37:00.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.422+0000 7f502d600640 1 -- 172.21.15.39:0/2921697465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5028070030 msgr2=0x7f5028149450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:00.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.422+0000 7f502d600640 1 --2- 172.21.15.39:0/2921697465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5028070030 0x7f5028149450 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f5018000c00 tx=0x7f5018003910 comp rx=0 tx=0).stop 2024-09-21T07:37:00.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.422+0000 7f502d600640 1 -- 172.21.15.39:0/2921697465 shutdown_connections 2024-09-21T07:37:00.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.422+0000 7f502d600640 1 --2- 172.21.15.39:0/2921697465 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5004078b90 0x7f500407b0a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:00.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.422+0000 7f502d600640 1 --2- 172.21.15.39:0/2921697465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5028073170 0x7f502815b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:00.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.422+0000 7f502d600640 1 --2- 172.21.15.39:0/2921697465 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5028070440 0x7f502815b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:00.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.422+0000 7f502d600640 1 --2- 172.21.15.39:0/2921697465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5028070030 0x7f5028149450 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:00.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.422+0000 7f502d600640 1 -- 172.21.15.39:0/2921697465 >> 172.21.15.39:0/2921697465 conn(0x7f50280b1cb0 msgr2=0x7f50280ad980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:00.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.422+0000 7f502d600640 1 -- 172.21.15.39:0/2921697465 shutdown_connections 2024-09-21T07:37:00.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.423+0000 7f502d600640 1 -- 172.21.15.39:0/2921697465 wait complete. 2024-09-21T07:37:00.451 DEBUG:tasks.ceph_test_case:wait_until_true: success in 15s and 0 retries 2024-09-21T07:37:00.451 DEBUG:teuthology.orchestra.run.smithi039:> sudo sysctl -n kernel.core_pattern 2024-09-21T07:37:00.466 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:37:00.500 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:37:00.503 INFO:teuthology.orchestra.run.smithi039.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T07:37:00.506 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2024-09-21T07:37:00.506 DEBUG:teuthology.orchestra.run.smithi039:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2024-09-21T07:37:00.608 INFO:teuthology.orchestra.run.smithi039.stdout:1726904203.56683.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i c', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2024-09-21T07:37:00.612 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1726904203.56683.core'] 2024-09-21T07:37:00.612 INFO:tasks.cephfs.cephfs_test_case:Found core file 1726904203.56683.core, deleting it 2024-09-21T07:37:00.612 DEBUG:teuthology.orchestra.run.smithi039:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1726904203.56683.core 2024-09-21T07:37:00.746 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 6:0 2024-09-21T07:37:00.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.971+0000 7f4c6ba00640 1 -- 172.21.15.39:0/2731156468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c64070030 msgr2=0x7f4c64074640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:00.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.971+0000 7f4c6ba00640 1 --2- 172.21.15.39:0/2731156468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c64070030 0x7f4c64074640 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f4c640b4b40 tx=0x7f4c5002a530 comp rx=0 tx=0).stop 2024-09-21T07:37:00.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.972+0000 7f4c6ba00640 1 -- 172.21.15.39:0/2731156468 shutdown_connections 2024-09-21T07:37:00.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.972+0000 7f4c6ba00640 1 --2- 172.21.15.39:0/2731156468 >> v2:172.21.15.148:3300/0 conn(0x7f4c64070440 0x7f4c64074b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:00.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.972+0000 7f4c6ba00640 1 --2- 172.21.15.39:0/2731156468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c64070030 0x7f4c64074640 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:00.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.972+0000 7f4c6ba00640 1 -- 172.21.15.39:0/2731156468 >> 172.21.15.39:0/2731156468 conn(0x7f4c640b1cb0 msgr2=0x7f4c640b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:00.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.972+0000 7f4c6ba00640 1 -- 172.21.15.39:0/2731156468 shutdown_connections 2024-09-21T07:37:00.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.972+0000 7f4c6ba00640 1 -- 172.21.15.39:0/2731156468 wait complete. 2024-09-21T07:37:00.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.974+0000 7f4c6ba00640 1 Processor -- start 2024-09-21T07:37:00.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.974+0000 7f4c6ba00640 1 -- start start 2024-09-21T07:37:00.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.975+0000 7f4c6ba00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c64070030 0x7f4c6414c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:00.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.975+0000 7f4c6ba00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4c64070440 0x7f4c64159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:00.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.975+0000 7f4c6ba00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c64073170 0x7f4c6415a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:00.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.975+0000 7f4c6ba00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4c640b39f0 con 0x7f4c64070030 2024-09-21T07:37:00.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.975+0000 7f4c6ba00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4c640b3870 con 0x7f4c64073170 2024-09-21T07:37:00.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.975+0000 7f4c6ba00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f4c640b3b70 con 0x7f4c64070440 2024-09-21T07:37:00.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.976+0000 7f4c69000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c64070030 0x7f4c6414c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:00.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.976+0000 7f4c69000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c64070030 0x7f4c6414c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48378/0 (socket says 172.21.15.39:48378) 2024-09-21T07:37:00.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.976+0000 7f4c69000640 1 -- 172.21.15.39:0/3605521331 learned_addr learned my addr 172.21.15.39:0/3605521331 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:00.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.976+0000 7f4c69a00640 1 --2- 172.21.15.39:0/3605521331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c64073170 0x7f4c6415a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:00.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.976+0000 7f4c63e00640 1 --2- 172.21.15.39:0/3605521331 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4c64070440 0x7f4c64159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:00.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.976+0000 7f4c69000640 1 -- 172.21.15.39:0/3605521331 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4c64070440 msgr2=0x7f4c64159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:00.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.976+0000 7f4c69000640 1 --2- 172.21.15.39:0/3605521331 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4c64070440 0x7f4c64159ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:00.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.976+0000 7f4c69000640 1 -- 172.21.15.39:0/3605521331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c64073170 msgr2=0x7f4c6415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:00.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.976+0000 7f4c69000640 1 --2- 172.21.15.39:0/3605521331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c64073170 0x7f4c6415a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:00.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.976+0000 7f4c69000640 1 -- 172.21.15.39:0/3605521331 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c6414cb50 con 0x7f4c64070030 2024-09-21T07:37:00.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.976+0000 7f4c69a00640 1 --2- 172.21.15.39:0/3605521331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c64073170 0x7f4c6415a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:37:00.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.977+0000 7f4c69000640 1 --2- 172.21.15.39:0/3605521331 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c64070030 0x7f4c6414c610 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f4c5800c460 tx=0x7f4c5800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:00.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.977+0000 7f4c60c00640 1 -- 172.21.15.39:0/3605521331 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c58011790 con 0x7f4c64070030 2024-09-21T07:37:00.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.977+0000 7f4c60c00640 1 -- 172.21.15.39:0/3605521331 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c58017070 con 0x7f4c64070030 2024-09-21T07:37:00.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.977+0000 7f4c60c00640 1 -- 172.21.15.39:0/3605521331 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c58011ce0 con 0x7f4c64070030 2024-09-21T07:37:00.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.977+0000 7f4c6ba00640 1 -- 172.21.15.39:0/3605521331 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c640ce930 con 0x7f4c64070030 2024-09-21T07:37:00.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.977+0000 7f4c6ba00640 1 -- 172.21.15.39:0/3605521331 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c640ceef0 con 0x7f4c64070030 2024-09-21T07:37:00.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.978+0000 7f4c6ba00640 1 -- 172.21.15.39:0/3605521331 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c640b4090 con 0x7f4c64070030 2024-09-21T07:37:00.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.979+0000 7f4c60c00640 1 -- 172.21.15.39:0/3605521331 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4c5801c460 con 0x7f4c64070030 2024-09-21T07:37:00.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.980+0000 7f4c60c00640 1 --2- 172.21.15.39:0/3605521331 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4c44078bc0 0x7f4c4407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:00.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.980+0000 7f4c60c00640 1 -- 172.21.15.39:0/3605521331 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6829+0+0 (secure 0 0 0) 0x7f4c5809afa0 con 0x7f4c64070030 2024-09-21T07:37:00.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.981+0000 7f4c63e00640 1 --2- 172.21.15.39:0/3605521331 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4c44078bc0 0x7f4c4407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:00.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.982+0000 7f4c63e00640 1 --2- 172.21.15.39:0/3605521331 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4c44078bc0 0x7f4c4407b0d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4c50034000 tx=0x7f4c50004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:00.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:00.985+0000 7f4c60c00640 1 -- 172.21.15.39:0/3605521331 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4c6414cb50 con 0x7f4c64070030 2024-09-21T07:37:01.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.257+0000 7f4c6ba00640 1 -- 172.21.15.39:0/3605521331 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "6:0"} v 0) -- 0x7f4c640c3860 con 0x7f4c64070030 2024-09-21T07:37:01.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.261+0000 7f4c60c00640 1 -- 172.21.15.39:0/3605521331 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "6:0"}]=0 failed mds gid 6321 v61) ==== 97+0+0 (secure 0 0 0) 0x7f4c58062f70 con 0x7f4c64070030 2024-09-21T07:37:01.263 INFO:teuthology.orchestra.run.smithi039.stderr:failed mds gid 6321 2024-09-21T07:37:01.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.263+0000 7f4c6ba00640 1 -- 172.21.15.39:0/3605521331 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4c44078bc0 msgr2=0x7f4c4407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:01.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.263+0000 7f4c6ba00640 1 --2- 172.21.15.39:0/3605521331 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4c44078bc0 0x7f4c4407b0d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4c50034000 tx=0x7f4c50004730 comp rx=0 tx=0).stop 2024-09-21T07:37:01.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.263+0000 7f4c6ba00640 1 -- 172.21.15.39:0/3605521331 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c64070030 msgr2=0x7f4c6414c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:01.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.263+0000 7f4c6ba00640 1 --2- 172.21.15.39:0/3605521331 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c64070030 0x7f4c6414c610 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f4c5800c460 tx=0x7f4c5800c920 comp rx=0 tx=0).stop 2024-09-21T07:37:01.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.264+0000 7f4c6ba00640 1 -- 172.21.15.39:0/3605521331 shutdown_connections 2024-09-21T07:37:01.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.264+0000 7f4c6ba00640 1 --2- 172.21.15.39:0/3605521331 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4c44078bc0 0x7f4c4407b0d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:01.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.264+0000 7f4c6ba00640 1 --2- 172.21.15.39:0/3605521331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c64073170 0x7f4c6415a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:01.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.264+0000 7f4c6ba00640 1 --2- 172.21.15.39:0/3605521331 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4c64070440 0x7f4c64159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:01.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.264+0000 7f4c6ba00640 1 --2- 172.21.15.39:0/3605521331 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c64070030 0x7f4c6414c610 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:01.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.264+0000 7f4c6ba00640 1 -- 172.21.15.39:0/3605521331 >> 172.21.15.39:0/3605521331 conn(0x7f4c640b1cb0 msgr2=0x7f4c640b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:01.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.264+0000 7f4c6ba00640 1 -- 172.21.15.39:0/3605521331 shutdown_connections 2024-09-21T07:37:01.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.264+0000 7f4c6ba00640 1 -- 172.21.15.39:0/3605521331 wait complete. 2024-09-21T07:37:01.297 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T07:37:01.297 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T07:37:01.297 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T07:37:01.297 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:37:01.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:37:01.297 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/daemon/state.py", line 146, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi039 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c' 2024-09-21T07:37:01.298 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T07:37:01.298 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T07:37:01.302 INFO:tasks.ceph.mds.c:Started 2024-09-21T07:37:01.302 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:37:01.500 INFO:tasks.ceph.mds.c.smithi039.stdout:starting mds.c at 2024-09-21T07:37:01.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.530+0000 7fd1e2800640 1 -- 172.21.15.39:0/122194911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1dc0cfcc0 msgr2=0x7fd1dc0d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:01.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.530+0000 7fd1e2800640 1 --2- 172.21.15.39:0/122194911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1dc0cfcc0 0x7fd1dc0d00d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8005fd0 tx=0x7fd1c802a5d0 comp rx=0 tx=0).stop 2024-09-21T07:37:01.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.530+0000 7fd1e2800640 1 -- 172.21.15.39:0/122194911 shutdown_connections 2024-09-21T07:37:01.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.530+0000 7fd1e2800640 1 --2- 172.21.15.39:0/122194911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1dc0cfcc0 0x7fd1dc0d00d0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:01.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.530+0000 7fd1e2800640 1 --2- 172.21.15.39:0/122194911 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1dc0b6cd0 0x7fd1dc0b70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:01.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.530+0000 7fd1e2800640 1 -- 172.21.15.39:0/122194911 >> 172.21.15.39:0/122194911 conn(0x7fd1dc0cec00 msgr2=0x7fd1dc0cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:01.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.530+0000 7fd1e2800640 1 -- 172.21.15.39:0/122194911 shutdown_connections 2024-09-21T07:37:01.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.530+0000 7fd1e2800640 1 -- 172.21.15.39:0/122194911 wait complete. 2024-09-21T07:37:01.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.532+0000 7fd1e2800640 1 Processor -- start 2024-09-21T07:37:01.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.533+0000 7fd1e2800640 1 -- start start 2024-09-21T07:37:01.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.534+0000 7fd1e2800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1dc0b5140 0x7fd1dc15cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:01.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.534+0000 7fd1e2800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1dc0b6cd0 0x7fd1dc1f8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:01.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.534+0000 7fd1e2800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1dc0cfcc0 0x7fd1dc1f93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:01.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.534+0000 7fd1e2800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd1dc0b5c60 con 0x7fd1dc0cfcc0 2024-09-21T07:37:01.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.534+0000 7fd1e2800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd1dc0ce1a0 con 0x7fd1dc0b6cd0 2024-09-21T07:37:01.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.534+0000 7fd1e2800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd1dc0b5de0 con 0x7fd1dc0b5140 2024-09-21T07:37:01.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.534+0000 7fd1e1e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1dc0cfcc0 0x7fd1dc1f93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:01.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.535+0000 7fd1e0a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1dc0b6cd0 0x7fd1dc1f8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:01.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.535+0000 7fd1e1e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1dc0cfcc0 0x7fd1dc1f93e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48400/0 (socket says 172.21.15.39:48400) 2024-09-21T07:37:01.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.535+0000 7fd1e1400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1dc0b5140 0x7fd1dc15cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:01.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.535+0000 7fd1e1e00640 1 -- 172.21.15.39:0/1345460035 learned_addr learned my addr 172.21.15.39:0/1345460035 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:01.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.535+0000 7fd1e1e00640 1 -- 172.21.15.39:0/1345460035 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1dc0b5140 msgr2=0x7fd1dc15cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:01.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.535+0000 7fd1e1e00640 1 --2- 172.21.15.39:0/1345460035 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1dc0b5140 0x7fd1dc15cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:01.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.535+0000 7fd1e1e00640 1 -- 172.21.15.39:0/1345460035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1dc0b6cd0 msgr2=0x7fd1dc1f8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:01.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.535+0000 7fd1e1e00640 1 --2- 172.21.15.39:0/1345460035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1dc0b6cd0 0x7fd1dc1f8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:01.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.535+0000 7fd1e1e00640 1 -- 172.21.15.39:0/1345460035 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1dc15d320 con 0x7fd1dc0cfcc0 2024-09-21T07:37:01.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.535+0000 7fd1e1e00640 1 --2- 172.21.15.39:0/1345460035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1dc0cfcc0 0x7fd1dc1f93e0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fd1d400c6b0 tx=0x7fd1d400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:01.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.536+0000 7fd1d9600640 1 -- 172.21.15.39:0/1345460035 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4003b70 con 0x7fd1dc0cfcc0 2024-09-21T07:37:01.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.536+0000 7fd1e2800640 1 -- 172.21.15.39:0/1345460035 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1dc0cc370 con 0x7fd1dc0cfcc0 2024-09-21T07:37:01.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.536+0000 7fd1d9600640 1 -- 172.21.15.39:0/1345460035 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1d4003d30 con 0x7fd1dc0cfcc0 2024-09-21T07:37:01.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.536+0000 7fd1d9600640 1 -- 172.21.15.39:0/1345460035 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4005330 con 0x7fd1dc0cfcc0 2024-09-21T07:37:01.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.536+0000 7fd1e2800640 1 -- 172.21.15.39:0/1345460035 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1dc0b4550 con 0x7fd1dc0cfcc0 2024-09-21T07:37:01.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.537+0000 7fd1e2800640 1 -- 172.21.15.39:0/1345460035 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1dc0b7fe0 con 0x7fd1dc0cfcc0 2024-09-21T07:37:01.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.538+0000 7fd1d9600640 1 -- 172.21.15.39:0/1345460035 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd1d40054d0 con 0x7fd1dc0cfcc0 2024-09-21T07:37:01.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.539+0000 7fd1d9600640 1 --2- 172.21.15.39:0/1345460035 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd1bc078bc0 0x7fd1bc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:01.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.539+0000 7fd1d9600640 1 -- 172.21.15.39:0/1345460035 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6915+0+0 (secure 0 0 0) 0x7fd1d409aed0 con 0x7fd1dc0cfcc0 2024-09-21T07:37:01.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.540+0000 7fd1e1400640 1 --2- 172.21.15.39:0/1345460035 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd1bc078bc0 0x7fd1bc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:01.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.541+0000 7fd1e1400640 1 --2- 172.21.15.39:0/1345460035 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd1bc078bc0 0x7fd1bc07b0d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0009c00 tx=0x7fd1d0005f30 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:01.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.544+0000 7fd1d9600640 1 -- 172.21.15.39:0/1345460035 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd1dc0d0610 con 0x7fd1dc0cfcc0 2024-09-21T07:37:01.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.781+0000 7fd1e2800640 1 -- 172.21.15.39:0/1345460035 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1dc0b6300 con 0x7fd1dc0cfcc0 2024-09-21T07:37:01.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.781+0000 7fd1d9600640 1 -- 172.21.15.39:0/1345460035 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+4418 (secure 0 0 0) 0x7fd1d4062810 con 0x7fd1dc0cfcc0 2024-09-21T07:37:01.783 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:37:01.784 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":62,"btime":"2024-09-21T07:37:01:267422+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:37:01.267401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5621,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5621":{"gid":5621,"name":"d","rank":0,"incarnation":62,"state":"up:replay","state_seq":1,"addr":"172.21.15.148:6837/3026827614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3026827614},{"type":"v1","addr":"172.21.15.148:6837","nonce":3026827614}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6115,"qdb_cluster":[6115]},"id":6}]} 2024-09-21T07:37:01.784 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 62 2024-09-21T07:37:01.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.784+0000 7fd1e2800640 1 -- 172.21.15.39:0/1345460035 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd1bc078bc0 msgr2=0x7fd1bc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:01.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.784+0000 7fd1e2800640 1 --2- 172.21.15.39:0/1345460035 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd1bc078bc0 0x7fd1bc07b0d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0009c00 tx=0x7fd1d0005f30 comp rx=0 tx=0).stop 2024-09-21T07:37:01.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.784+0000 7fd1e2800640 1 -- 172.21.15.39:0/1345460035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1dc0cfcc0 msgr2=0x7fd1dc1f93e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:01.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.784+0000 7fd1e2800640 1 --2- 172.21.15.39:0/1345460035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1dc0cfcc0 0x7fd1dc1f93e0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fd1d400c6b0 tx=0x7fd1d400cb70 comp rx=0 tx=0).stop 2024-09-21T07:37:01.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.784+0000 7fd1e2800640 1 -- 172.21.15.39:0/1345460035 shutdown_connections 2024-09-21T07:37:01.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.784+0000 7fd1e2800640 1 --2- 172.21.15.39:0/1345460035 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd1bc078bc0 0x7fd1bc07b0d0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:01.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.784+0000 7fd1e2800640 1 --2- 172.21.15.39:0/1345460035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1dc0cfcc0 0x7fd1dc1f93e0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:01.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.784+0000 7fd1e2800640 1 --2- 172.21.15.39:0/1345460035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1dc0b6cd0 0x7fd1dc1f8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:01.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.784+0000 7fd1e2800640 1 --2- 172.21.15.39:0/1345460035 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1dc0b5140 0x7fd1dc15cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:01.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.784+0000 7fd1e2800640 1 -- 172.21.15.39:0/1345460035 >> 172.21.15.39:0/1345460035 conn(0x7fd1dc0cec00 msgr2=0x7fd1dc0ce320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:01.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.784+0000 7fd1e2800640 1 -- 172.21.15.39:0/1345460035 shutdown_connections 2024-09-21T07:37:01.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:01.784+0000 7fd1e2800640 1 -- 172.21.15.39:0/1345460035 wait complete. 2024-09-21T07:37:01.808 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-09-21T07:37:03.147 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-09-21T07:37:06.810 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:37:07.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.045+0000 7f5169c00640 1 -- 172.21.15.39:0/4204823263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51640706b0 msgr2=0x7f51640cba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:07.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.045+0000 7f5169c00640 1 --2- 172.21.15.39:0/4204823263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51640706b0 0x7f51640cba00 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f514c005fd0 tx=0x7f514c02a5d0 comp rx=0 tx=0).stop 2024-09-21T07:37:07.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.045+0000 7f5169c00640 1 -- 172.21.15.39:0/4204823263 shutdown_connections 2024-09-21T07:37:07.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.045+0000 7f5169c00640 1 --2- 172.21.15.39:0/4204823263 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51640cad30 0x7f51640735b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:07.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.045+0000 7f5169c00640 1 --2- 172.21.15.39:0/4204823263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51640706b0 0x7f51640cba00 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:07.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.045+0000 7f5169c00640 1 -- 172.21.15.39:0/4204823263 >> 172.21.15.39:0/4204823263 conn(0x7f51640b1cb0 msgr2=0x7f51640b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:07.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.045+0000 7f5169c00640 1 -- 172.21.15.39:0/4204823263 shutdown_connections 2024-09-21T07:37:07.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.045+0000 7f5169c00640 1 -- 172.21.15.39:0/4204823263 wait complete. 2024-09-21T07:37:07.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.048+0000 7f5169c00640 1 Processor -- start 2024-09-21T07:37:07.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.048+0000 7f5169c00640 1 -- start start 2024-09-21T07:37:07.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.049+0000 7f5169c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51640ca800 0x7f51641598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:07.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.049+0000 7f5169c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51640cad30 0x7f51640af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:07.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.049+0000 7f5169c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51640c5d90 0x7f51641f5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:07.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.049+0000 7f5169c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5164073f00 con 0x7f51640c5d90 2024-09-21T07:37:07.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.049+0000 7f5169c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5164073d80 con 0x7f51640ca800 2024-09-21T07:37:07.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.049+0000 7f5169c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f5164074080 con 0x7f51640cad30 2024-09-21T07:37:07.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.049+0000 7f5163400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51640c5d90 0x7f51641f5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:07.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.050+0000 7f5163400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51640c5d90 0x7f51641f5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43300/0 (socket says 172.21.15.39:43300) 2024-09-21T07:37:07.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.050+0000 7f5163400640 1 -- 172.21.15.39:0/692779905 learned_addr learned my addr 172.21.15.39:0/692779905 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:07.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.050+0000 7f515be00640 1 --2- 172.21.15.39:0/692779905 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51640cad30 0x7f51640af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:07.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.050+0000 7f5162a00640 1 --2- 172.21.15.39:0/692779905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51640ca800 0x7f51641598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:07.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.050+0000 7f5163400640 1 -- 172.21.15.39:0/692779905 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51640cad30 msgr2=0x7f51640af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:07.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.050+0000 7f5163400640 1 --2- 172.21.15.39:0/692779905 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51640cad30 0x7f51640af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:07.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.050+0000 7f5163400640 1 -- 172.21.15.39:0/692779905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51640ca800 msgr2=0x7f51641598d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:07.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.050+0000 7f5163400640 1 --2- 172.21.15.39:0/692779905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51640ca800 0x7f51641598d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:07.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.050+0000 7f5163400640 1 -- 172.21.15.39:0/692779905 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51641f5e10 con 0x7f51640c5d90 2024-09-21T07:37:07.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.050+0000 7f5163400640 1 --2- 172.21.15.39:0/692779905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51640c5d90 0x7f51641f5500 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f515c00ea40 tx=0x7f515c00ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:07.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.051+0000 7f515aa00640 1 -- 172.21.15.39:0/692779905 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515c00cde0 con 0x7f51640c5d90 2024-09-21T07:37:07.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.051+0000 7f515aa00640 1 -- 172.21.15.39:0/692779905 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f515c004c10 con 0x7f51640c5d90 2024-09-21T07:37:07.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.051+0000 7f5169c00640 1 -- 172.21.15.39:0/692779905 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51641f60f0 con 0x7f51640c5d90 2024-09-21T07:37:07.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.051+0000 7f515aa00640 1 -- 172.21.15.39:0/692779905 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515c005020 con 0x7f51640c5d90 2024-09-21T07:37:07.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.051+0000 7f5169c00640 1 -- 172.21.15.39:0/692779905 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51640c3f90 con 0x7f51640c5d90 2024-09-21T07:37:07.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.052+0000 7f5169c00640 1 -- 172.21.15.39:0/692779905 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51640745e0 con 0x7f51640c5d90 2024-09-21T07:37:07.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.053+0000 7f515aa00640 1 -- 172.21.15.39:0/692779905 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f515c0051c0 con 0x7f51640c5d90 2024-09-21T07:37:07.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.054+0000 7f515aa00640 1 --2- 172.21.15.39:0/692779905 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5140078bc0 0x7f514007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:07.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.054+0000 7f515aa00640 1 -- 172.21.15.39:0/692779905 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6915+0+0 (secure 0 0 0) 0x7f515c09be70 con 0x7f51640c5d90 2024-09-21T07:37:07.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.054+0000 7f5162a00640 1 --2- 172.21.15.39:0/692779905 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5140078bc0 0x7f514007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:07.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.055+0000 7f5162a00640 1 --2- 172.21.15.39:0/692779905 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5140078bc0 0x7f514007b0d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f51640cb250 tx=0x7f5154009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:07.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.059+0000 7f515aa00640 1 -- 172.21.15.39:0/692779905 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f515c063790 con 0x7f51640c5d90 2024-09-21T07:37:07.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.294+0000 7f5169c00640 1 -- 172.21.15.39:0/692779905 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51640b3290 con 0x7f51640c5d90 2024-09-21T07:37:07.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.294+0000 7f515aa00640 1 -- 172.21.15.39:0/692779905 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5249 (secure 0 0 0) 0x7f515c0681c0 con 0x7f51640c5d90 2024-09-21T07:37:07.296 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:37:07.296 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":66,"btime":"2024-09-21T07:37:05:276431+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6585,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/1472430787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1472430787},{"type":"v1","addr":"172.21.15.39:6834","nonce":1472430787}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:37:05.276415+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5621,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5621":{"gid":5621,"name":"d","rank":0,"incarnation":62,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3026827614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3026827614},{"type":"v1","addr":"172.21.15.148:6837","nonce":3026827614}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5621,"qdb_cluster":[5621,6115]},"id":6}]} 2024-09-21T07:37:07.297 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 66 2024-09-21T07:37:07.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.297+0000 7f5169c00640 1 -- 172.21.15.39:0/692779905 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5140078bc0 msgr2=0x7f514007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:07.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.297+0000 7f5169c00640 1 --2- 172.21.15.39:0/692779905 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5140078bc0 0x7f514007b0d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f51640cb250 tx=0x7f5154009040 comp rx=0 tx=0).stop 2024-09-21T07:37:07.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.297+0000 7f5169c00640 1 -- 172.21.15.39:0/692779905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51640c5d90 msgr2=0x7f51641f5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:07.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.297+0000 7f5169c00640 1 --2- 172.21.15.39:0/692779905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51640c5d90 0x7f51641f5500 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f515c00ea40 tx=0x7f515c00ef00 comp rx=0 tx=0).stop 2024-09-21T07:37:07.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.297+0000 7f5169c00640 1 -- 172.21.15.39:0/692779905 shutdown_connections 2024-09-21T07:37:07.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.297+0000 7f5169c00640 1 --2- 172.21.15.39:0/692779905 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5140078bc0 0x7f514007b0d0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:07.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.297+0000 7f5169c00640 1 --2- 172.21.15.39:0/692779905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51640c5d90 0x7f51641f5500 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:07.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.297+0000 7f5169c00640 1 --2- 172.21.15.39:0/692779905 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51640cad30 0x7f51640af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:07.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.297+0000 7f5169c00640 1 --2- 172.21.15.39:0/692779905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51640ca800 0x7f51641598d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:07.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.297+0000 7f5169c00640 1 -- 172.21.15.39:0/692779905 >> 172.21.15.39:0/692779905 conn(0x7f51640b1cb0 msgr2=0x7f516415b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:07.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.297+0000 7f5169c00640 1 -- 172.21.15.39:0/692779905 shutdown_connections 2024-09-21T07:37:07.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.297+0000 7f5169c00640 1 -- 172.21.15.39:0/692779905 wait complete. 2024-09-21T07:37:07.322 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-21T07:37:07.322 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:37:07.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.547+0000 7fc0e1600640 1 -- 172.21.15.39:0/4129781580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0dc0ccb50 msgr2=0x7fc0dc0ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:07.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.547+0000 7fc0e1600640 1 --2- 172.21.15.39:0/4129781580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0dc0ccb50 0x7fc0dc0ccf60 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4005fd0 tx=0x7fc0d402a8b0 comp rx=0 tx=0).stop 2024-09-21T07:37:07.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.547+0000 7fc0e1600640 1 -- 172.21.15.39:0/4129781580 shutdown_connections 2024-09-21T07:37:07.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.547+0000 7fc0e1600640 1 --2- 172.21.15.39:0/4129781580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0dc0ccb50 0x7fc0dc0ccf60 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:07.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.547+0000 7fc0e1600640 1 -- 172.21.15.39:0/4129781580 >> 172.21.15.39:0/4129781580 conn(0x7fc0dc06d7d0 msgr2=0x7fc0dc0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:07.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.548+0000 7fc0e1600640 1 -- 172.21.15.39:0/4129781580 shutdown_connections 2024-09-21T07:37:07.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.548+0000 7fc0e1600640 1 -- 172.21.15.39:0/4129781580 wait complete. 2024-09-21T07:37:07.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.550+0000 7fc0e1600640 1 Processor -- start 2024-09-21T07:37:07.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.550+0000 7fc0e1600640 1 -- start start 2024-09-21T07:37:07.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.551+0000 7fc0e1600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0dc0af3c0 0x7fc0dc152950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:07.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.551+0000 7fc0e1600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc0dc0cc1e0 0x7fc0dc074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:07.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.551+0000 7fc0e1600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0dc0ccb50 0x7fc0dc0adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:07.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.551+0000 7fc0e1600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc0dc153520 con 0x7fc0dc0af3c0 2024-09-21T07:37:07.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.551+0000 7fc0e1600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc0dc1533a0 con 0x7fc0dc0ccb50 2024-09-21T07:37:07.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.551+0000 7fc0e1600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc0dc1536a0 con 0x7fc0dc0cc1e0 2024-09-21T07:37:07.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.552+0000 7fc0daa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0dc0af3c0 0x7fc0dc152950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:07.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.552+0000 7fc0daa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0dc0af3c0 0x7fc0dc152950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43324/0 (socket says 172.21.15.39:43324) 2024-09-21T07:37:07.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.552+0000 7fc0daa00640 1 -- 172.21.15.39:0/84513899 learned_addr learned my addr 172.21.15.39:0/84513899 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:07.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.552+0000 7fc0d3e00640 1 --2- 172.21.15.39:0/84513899 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc0dc0cc1e0 0x7fc0dc074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:07.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.552+0000 7fc0db400640 1 --2- 172.21.15.39:0/84513899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0dc0ccb50 0x7fc0dc0adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:07.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.552+0000 7fc0daa00640 1 -- 172.21.15.39:0/84513899 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc0dc0cc1e0 msgr2=0x7fc0dc074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:07.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.552+0000 7fc0daa00640 1 --2- 172.21.15.39:0/84513899 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc0dc0cc1e0 0x7fc0dc074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:07.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.552+0000 7fc0daa00640 1 -- 172.21.15.39:0/84513899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0dc0ccb50 msgr2=0x7fc0dc0adfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:07.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.552+0000 7fc0daa00640 1 --2- 172.21.15.39:0/84513899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0dc0ccb50 0x7fc0dc0adfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:07.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.552+0000 7fc0daa00640 1 -- 172.21.15.39:0/84513899 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0dc0aefd0 con 0x7fc0dc0af3c0 2024-09-21T07:37:07.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.552+0000 7fc0d3e00640 1 --2- 172.21.15.39:0/84513899 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc0dc0cc1e0 0x7fc0dc074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:37:07.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.552+0000 7fc0daa00640 1 --2- 172.21.15.39:0/84513899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0dc0af3c0 0x7fc0dc152950 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc00b420 tx=0x7fc0cc00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:07.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.553+0000 7fc0d2a00640 1 -- 172.21.15.39:0/84513899 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc011830 con 0x7fc0dc0af3c0 2024-09-21T07:37:07.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.553+0000 7fc0d2a00640 1 -- 172.21.15.39:0/84513899 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0cc016070 con 0x7fc0dc0af3c0 2024-09-21T07:37:07.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.553+0000 7fc0d2a00640 1 -- 172.21.15.39:0/84513899 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc011d60 con 0x7fc0dc0af3c0 2024-09-21T07:37:07.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.553+0000 7fc0e1600640 1 -- 172.21.15.39:0/84513899 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0dc15a440 con 0x7fc0dc0af3c0 2024-09-21T07:37:07.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.553+0000 7fc0e1600640 1 -- 172.21.15.39:0/84513899 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0dc09b200 con 0x7fc0dc0af3c0 2024-09-21T07:37:07.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.554+0000 7fc0e1600640 1 -- 172.21.15.39:0/84513899 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0dc153b90 con 0x7fc0dc0af3c0 2024-09-21T07:37:07.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.555+0000 7fc0d2a00640 1 -- 172.21.15.39:0/84513899 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc0cc01b460 con 0x7fc0dc0af3c0 2024-09-21T07:37:07.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.556+0000 7fc0d2a00640 1 --2- 172.21.15.39:0/84513899 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc0b8078b90 0x7fc0b807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:07.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.556+0000 7fc0d2a00640 1 -- 172.21.15.39:0/84513899 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6915+0+0 (secure 0 0 0) 0x7fc0cc09ab90 con 0x7fc0dc0af3c0 2024-09-21T07:37:07.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.557+0000 7fc0d3e00640 1 --2- 172.21.15.39:0/84513899 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc0b8078b90 0x7fc0b807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:07.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.558+0000 7fc0d3e00640 1 --2- 172.21.15.39:0/84513899 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc0b8078b90 0x7fc0b807b0a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc153320 tx=0x7fc0c4009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:07.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.561+0000 7fc0d2a00640 1 -- 172.21.15.39:0/84513899 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc0cc020020 con 0x7fc0dc0af3c0 2024-09-21T07:37:07.645 INFO:tasks.ceph.mds.d.smithi148.stderr:2024-09-21T07:37:07.644+0000 7fb413600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:37:07.645 INFO:tasks.ceph.mds.d.smithi148.stderr:2024-09-21T07:37:07.644+0000 7fb413600640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:37:07.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.798+0000 7fc0e1600640 1 -- 172.21.15.39:0/84513899 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0dc069450 con 0x7fc0dc0af3c0 2024-09-21T07:37:07.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.799+0000 7fc0d2a00640 1 -- 172.21.15.39:0/84513899 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5249 (secure 0 0 0) 0x7fc0dc0aefd0 con 0x7fc0dc0af3c0 2024-09-21T07:37:07.801 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:37:07.801 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":66,"btime":"2024-09-21T07:37:05:276431+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6585,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/1472430787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1472430787},{"type":"v1","addr":"172.21.15.39:6834","nonce":1472430787}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:37:05.276415+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5621,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5621":{"gid":5621,"name":"d","rank":0,"incarnation":62,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3026827614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3026827614},{"type":"v1","addr":"172.21.15.148:6837","nonce":3026827614}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5621,"qdb_cluster":[5621,6115]},"id":6}]} 2024-09-21T07:37:07.801 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 66 2024-09-21T07:37:07.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.802+0000 7fc0e1600640 1 -- 172.21.15.39:0/84513899 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc0b8078b90 msgr2=0x7fc0b807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:07.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.802+0000 7fc0e1600640 1 --2- 172.21.15.39:0/84513899 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc0b8078b90 0x7fc0b807b0a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc153320 tx=0x7fc0c4009040 comp rx=0 tx=0).stop 2024-09-21T07:37:07.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.802+0000 7fc0e1600640 1 -- 172.21.15.39:0/84513899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0dc0af3c0 msgr2=0x7fc0dc152950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:07.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.802+0000 7fc0e1600640 1 --2- 172.21.15.39:0/84513899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0dc0af3c0 0x7fc0dc152950 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc00b420 tx=0x7fc0cc00b8e0 comp rx=0 tx=0).stop 2024-09-21T07:37:07.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.802+0000 7fc0e1600640 1 -- 172.21.15.39:0/84513899 shutdown_connections 2024-09-21T07:37:07.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.802+0000 7fc0e1600640 1 --2- 172.21.15.39:0/84513899 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc0b8078b90 0x7fc0b807b0a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:07.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.802+0000 7fc0e1600640 1 --2- 172.21.15.39:0/84513899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0dc0ccb50 0x7fc0dc0adfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:07.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.802+0000 7fc0e1600640 1 --2- 172.21.15.39:0/84513899 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc0dc0cc1e0 0x7fc0dc074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:07.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.802+0000 7fc0e1600640 1 --2- 172.21.15.39:0/84513899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0dc0af3c0 0x7fc0dc152950 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:07.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.802+0000 7fc0e1600640 1 -- 172.21.15.39:0/84513899 >> 172.21.15.39:0/84513899 conn(0x7fc0dc06d7d0 msgr2=0x7fc0dc0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:07.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.802+0000 7fc0e1600640 1 -- 172.21.15.39:0/84513899 shutdown_connections 2024-09-21T07:37:07.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:07.802+0000 7fc0e1600640 1 -- 172.21.15.39:0/84513899 wait complete. 2024-09-21T07:37:08.232 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 66, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:36:20.189287+0000', 'modified': '2024-09-21T07:37:05.276415+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 93, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 5621, 'mds_1': 6115}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5621': {'gid': 5621, 'name': 'd', 'rank': 0, 'incarnation': 62, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.148:6837/3026827614', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 3026827614}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 3026827614}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6115': {'gid': 6115, 'name': 'a', 'rank': 1, 'incarnation': 56, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6835/364657398', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 364657398}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 364657398}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5621, 'qdb_cluster': [5621, 6115]} 2024-09-21T07:37:08.232 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2024-09-21T07:37:08.232 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-21T07:37:08.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.461+0000 7fd7a0000640 1 -- 172.21.15.39:0/263376910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7980ccb50 msgr2=0x7fd7980ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:08.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.461+0000 7fd7a0000640 1 --2- 172.21.15.39:0/263376910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7980ccb50 0x7fd7980ccf60 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fd798071d60 tx=0x7fd79002aa30 comp rx=0 tx=0).stop 2024-09-21T07:37:08.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.461+0000 7fd7a0000640 1 -- 172.21.15.39:0/263376910 shutdown_connections 2024-09-21T07:37:08.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.461+0000 7fd7a0000640 1 --2- 172.21.15.39:0/263376910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7980ccb50 0x7fd7980ccf60 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.461+0000 7fd7a0000640 1 --2- 172.21.15.39:0/263376910 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd79815d8a0 0x7fd7980b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.461+0000 7fd7a0000640 1 --2- 172.21.15.39:0/263376910 >> v2:172.21.15.148:3300/0 conn(0x7fd7980cc1e0 0x7fd7980af3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.461+0000 7fd7a0000640 1 -- 172.21.15.39:0/263376910 >> 172.21.15.39:0/263376910 conn(0x7fd79806d7d0 msgr2=0x7fd7980b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:08.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.461+0000 7fd7a0000640 1 -- 172.21.15.39:0/263376910 shutdown_connections 2024-09-21T07:37:08.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.461+0000 7fd7a0000640 1 -- 172.21.15.39:0/263376910 wait complete. 2024-09-21T07:37:08.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.463+0000 7fd7a0000640 1 Processor -- start 2024-09-21T07:37:08.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.463+0000 7fd7a0000640 1 -- start start 2024-09-21T07:37:08.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.465+0000 7fd7a0000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7980cc1e0 0x7fd7981537d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:08.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.465+0000 7fd7a0000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7980ccb50 0x7fd7981551b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:08.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.465+0000 7fd7a0000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd79815d8a0 0x7fd7981e9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:08.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.465+0000 7fd7a0000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd7980b4770 con 0x7fd7980cc1e0 2024-09-21T07:37:08.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.465+0000 7fd7a0000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd7980b45f0 con 0x7fd7980ccb50 2024-09-21T07:37:08.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.465+0000 7fd7a0000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd7981533a0 con 0x7fd79815d8a0 2024-09-21T07:37:08.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.465+0000 7fd79d600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7980cc1e0 0x7fd7981537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:08.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.465+0000 7fd79d600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7980cc1e0 0x7fd7981537d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43340/0 (socket says 172.21.15.39:43340) 2024-09-21T07:37:08.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.465+0000 7fd79d600640 1 -- 172.21.15.39:0/2922312422 learned_addr learned my addr 172.21.15.39:0/2922312422 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:08.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.465+0000 7fd79e000640 1 --2- 172.21.15.39:0/2922312422 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd79815d8a0 0x7fd7981e9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:08.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.465+0000 7fd79cc00640 1 --2- 172.21.15.39:0/2922312422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7980ccb50 0x7fd7981551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:08.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.465+0000 7fd79d600640 1 -- 172.21.15.39:0/2922312422 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd79815d8a0 msgr2=0x7fd7981e9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:08.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.465+0000 7fd79d600640 1 --2- 172.21.15.39:0/2922312422 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd79815d8a0 0x7fd7981e9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.465+0000 7fd79d600640 1 -- 172.21.15.39:0/2922312422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7980ccb50 msgr2=0x7fd7981551b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:08.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.465+0000 7fd79d600640 1 --2- 172.21.15.39:0/2922312422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7980ccb50 0x7fd7981551b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.465+0000 7fd79d600640 1 -- 172.21.15.39:0/2922312422 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7980747e0 con 0x7fd7980cc1e0 2024-09-21T07:37:08.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.466+0000 7fd79d600640 1 --2- 172.21.15.39:0/2922312422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7980cc1e0 0x7fd7981537d0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fd78c00d650 tx=0x7fd78c00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:08.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.466+0000 7fd795600640 1 -- 172.21.15.39:0/2922312422 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c017070 con 0x7fd7980cc1e0 2024-09-21T07:37:08.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.466+0000 7fd795600640 1 -- 172.21.15.39:0/2922312422 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd78c004310 con 0x7fd7980cc1e0 2024-09-21T07:37:08.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.466+0000 7fd7a0000640 1 -- 172.21.15.39:0/2922312422 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd798074ac0 con 0x7fd7980cc1e0 2024-09-21T07:37:08.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.466+0000 7fd795600640 1 -- 172.21.15.39:0/2922312422 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c005280 con 0x7fd7980cc1e0 2024-09-21T07:37:08.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.466+0000 7fd7a0000640 1 -- 172.21.15.39:0/2922312422 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd798072ed0 con 0x7fd7980cc1e0 2024-09-21T07:37:08.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.469+0000 7fd795600640 1 -- 172.21.15.39:0/2922312422 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd78c005420 con 0x7fd7980cc1e0 2024-09-21T07:37:08.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.470+0000 7fd795600640 1 --2- 172.21.15.39:0/2922312422 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd778078b90 0x7fd77807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:08.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.470+0000 7fd795600640 1 -- 172.21.15.39:0/2922312422 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6915+0+0 (secure 0 0 0) 0x7fd78c09b5f0 con 0x7fd7980cc1e0 2024-09-21T07:37:08.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.470+0000 7fd79cc00640 1 --2- 172.21.15.39:0/2922312422 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd778078b90 0x7fd77807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:08.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.470+0000 7fd7a71ee740 1 Processor -- start 2024-09-21T07:37:08.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.470+0000 7fd7a71ee740 1 -- start start 2024-09-21T07:37:08.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.471+0000 7fd79cc00640 1 --2- 172.21.15.39:0/2922312422 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd778078b90 0x7fd77807b0a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd79809d950 tx=0x7fd784005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:08.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.472+0000 7fd7a71ee740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e3b25c6270 0x55e3b2996970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:08.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.472+0000 7fd7a71ee740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3b259d3b0 0x55e3b2996eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:08.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.472+0000 7fd7a71ee740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b25a5590 0x55e3b2966f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:08.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.472+0000 7fd7a71ee740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55e3b28f0430 con 0x55e3b25a5590 2024-09-21T07:37:08.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.472+0000 7fd7a71ee740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e3b28e05d0 con 0x55e3b259d3b0 2024-09-21T07:37:08.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.472+0000 7fd7a71ee740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55e3b28cfa00 con 0x55e3b25c6270 2024-09-21T07:37:08.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.472+0000 7fd79d600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e3b25c6270 0x55e3b2996970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:08.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.472+0000 7fd79d600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e3b25c6270 0x55e3b2996970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.39:60954/0 (socket says 172.21.15.39:60954) 2024-09-21T07:37:08.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.472+0000 7fd79e000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b25a5590 0x55e3b2966f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:08.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.472+0000 7fd79d600640 1 -- 172.21.15.39:0/30017532 learned_addr learned my addr 172.21.15.39:0/30017532 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:08.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.472+0000 7fd79cc00640 1 --2- 172.21.15.39:0/30017532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3b259d3b0 0x55e3b2996eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:08.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.473+0000 7fd79d600640 1 -- 172.21.15.39:0/30017532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3b259d3b0 msgr2=0x55e3b2996eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:37:08.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.473+0000 7fd79d600640 1 --2- 172.21.15.39:0/30017532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3b259d3b0 0x55e3b2996eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.473+0000 7fd79d600640 1 -- 172.21.15.39:0/30017532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b25a5590 msgr2=0x55e3b2966f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:37:08.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.473+0000 7fd79d600640 1 --2- 172.21.15.39:0/30017532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b25a5590 0x55e3b2966f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.473+0000 7fd79d600640 1 -- 172.21.15.39:0/30017532 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3b2898520 con 0x55e3b25c6270 2024-09-21T07:37:08.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.473+0000 7fd79d600640 1 --2- 172.21.15.39:0/30017532 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e3b25c6270 0x55e3b2996970 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd78c05ee30 tx=0x7fd78c062690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fb67d8e4d0201f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:08.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.473+0000 7fd789c00640 1 -- 172.21.15.39:0/30017532 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c062e60 con 0x55e3b25c6270 2024-09-21T07:37:08.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.473+0000 7fd789c00640 1 -- 172.21.15.39:0/30017532 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd78c067940 con 0x55e3b25c6270 2024-09-21T07:37:08.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.473+0000 7fd789c00640 1 -- 172.21.15.39:0/30017532 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c06c7f0 con 0x55e3b25c6270 2024-09-21T07:37:08.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.474+0000 7fd7a71ee740 1 -- 172.21.15.39:0/30017532 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e3b25c6270 msgr2=0x55e3b2996970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:08.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.474+0000 7fd7a71ee740 1 --2- 172.21.15.39:0/30017532 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e3b25c6270 0x55e3b2996970 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd78c05ee30 tx=0x7fd78c062690 comp rx=0 tx=0).stop 2024-09-21T07:37:08.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.474+0000 7fd7a71ee740 1 -- 172.21.15.39:0/30017532 shutdown_connections 2024-09-21T07:37:08.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.474+0000 7fd7a71ee740 1 --2- 172.21.15.39:0/30017532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b25a5590 0x55e3b2966f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.474+0000 7fd7a71ee740 1 --2- 172.21.15.39:0/30017532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3b259d3b0 0x55e3b2996eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.474+0000 7fd7a71ee740 1 --2- 172.21.15.39:0/30017532 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e3b25c6270 0x55e3b2996970 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.474+0000 7fd7a71ee740 1 -- 172.21.15.39:0/30017532 >> 172.21.15.39:0/30017532 conn(0x55e3b25ba3c0 msgr2=0x55e3b296c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:08.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.474+0000 7fd7a71ee740 1 -- 172.21.15.39:0/30017532 shutdown_connections 2024-09-21T07:37:08.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.474+0000 7fd7a71ee740 1 -- 172.21.15.39:0/30017532 wait complete. 2024-09-21T07:37:08.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.475+0000 7fd7a71ee740 1 -- start start 2024-09-21T07:37:08.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.476+0000 7fd7a71ee740 1 Processor -- start 2024-09-21T07:37:08.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.477+0000 7fd7a71ee740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e3b259d3b0 0x55e3b29c1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:08.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.477+0000 7fd7a71ee740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b25a5590 0x55e3b298ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:08.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.477+0000 7fd7a71ee740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3b25c6270 0x55e3b29ab080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:08.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.477+0000 7fd7a71ee740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55e3b2876330 con 0x55e3b25a5590 2024-09-21T07:37:08.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.477+0000 7fd7a71ee740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e3b28d3810 con 0x55e3b25c6270 2024-09-21T07:37:08.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.477+0000 7fd7a71ee740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55e3b2640620 con 0x55e3b259d3b0 2024-09-21T07:37:08.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.477+0000 7fd7a71ee740 10 client.0 _ll_register_callbacks cb 0x55e3b287b420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:37:08.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.477+0000 7fd79cc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b25a5590 0x55e3b298ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:08.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.477+0000 7fd79cc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b25a5590 0x55e3b298ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43358/0 (socket says 172.21.15.39:43358) 2024-09-21T07:37:08.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.477+0000 7fd79cc00640 1 -- 172.21.15.39:0/1419628532 learned_addr learned my addr 172.21.15.39:0/1419628532 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:08.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.478+0000 7fd79d600640 1 --2- 172.21.15.39:0/1419628532 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e3b259d3b0 0x55e3b29c1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:08.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.478+0000 7fd79e000640 1 --2- 172.21.15.39:0/1419628532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3b25c6270 0x55e3b29ab080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:08.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.478+0000 7fd79cc00640 1 -- 172.21.15.39:0/1419628532 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e3b259d3b0 msgr2=0x55e3b29c1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:37:08.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.478+0000 7fd79cc00640 1 --2- 172.21.15.39:0/1419628532 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e3b259d3b0 0x55e3b29c1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.478+0000 7fd79cc00640 1 -- 172.21.15.39:0/1419628532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3b25c6270 msgr2=0x55e3b29ab080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:37:08.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.478+0000 7fd79cc00640 1 --2- 172.21.15.39:0/1419628532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3b25c6270 0x55e3b29ab080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.478+0000 7fd79cc00640 1 -- 172.21.15.39:0/1419628532 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3b2631850 con 0x55e3b25a5590 2024-09-21T07:37:08.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.478+0000 7fd79cc00640 1 --2- 172.21.15.39:0/1419628532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b25a5590 0x55e3b298ffc0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fd7840098a0 tx=0x7fd784004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2cd1d8ca591e71 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:08.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.478+0000 7fd77e000640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:37:08.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.478+0000 7fd77e000640 1 -- 172.21.15.39:0/1419628532 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd784020dc0 con 0x55e3b25a5590 2024-09-21T07:37:08.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.478+0000 7fd77e000640 1 -- 172.21.15.39:0/1419628532 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6915+0+0 (secure 0 0 0) 0x7fd784004020 con 0x55e3b25a5590 2024-09-21T07:37:08.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.479+0000 7fd7a71ee740 1 -- 172.21.15.39:0/1419628532 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd78c067940 con 0x55e3b25a5590 2024-09-21T07:37:08.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.479+0000 7fd77e000640 1 -- 172.21.15.39:0/1419628532 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fd784012e50 con 0x55e3b25a5590 2024-09-21T07:37:08.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.479+0000 7fd7a71ee740 10 client.6615 fetch_fsmap learned FSMap version 66 2024-09-21T07:37:08.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.479+0000 7fd7a71ee740 1 -- 172.21.15.39:0/1419628532 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x55e3b26a86b0 con 0x55e3b25a5590 2024-09-21T07:37:08.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.479+0000 7fd77e000640 1 -- 172.21.15.39:0/1419628532 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd78c067940 con 0x55e3b25a5590 2024-09-21T07:37:08.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.479+0000 7fd77e000640 1 -- 172.21.15.39:0/1419628532 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 66) ==== 3522+0+0 (secure 0 0 0) 0x7fd784030050 con 0x55e3b25a5590 2024-09-21T07:37:08.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.479+0000 7fd7a71ee740 10 client.6615 fetch_fsmap finished waiting for FSMap version 66 2024-09-21T07:37:08.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.479+0000 7fd7a71ee740 10 client.6615 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T07:37:08.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.479+0000 7fd7a71ee740 20 client.6615 populate_metadata read hostname 'smithi039' 2024-09-21T07:37:08.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.479+0000 7fd7a71ee740 1 --2- 172.21.15.39:0/1419628532 >> [v2:172.21.15.148:6836/3026827614,v1:172.21.15.148:6837/3026827614] conn(0x55e3b2584e40 0x55e3b298ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:08.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.479+0000 7fd7a71ee740 4 client.6615 mds_command: new command op to 5621 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:37:08.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.479+0000 7fd7a71ee740 1 -- 172.21.15.39:0/1419628532 --> [v2:172.21.15.148:6836/3026827614,v1:172.21.15.148:6837/3026827614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3b28e2520 con 0x55e3b2584e40 2024-09-21T07:37:08.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.480+0000 7fd79d600640 1 --2- 172.21.15.39:0/1419628532 >> [v2:172.21.15.148:6836/3026827614,v1:172.21.15.148:6837/3026827614] conn(0x55e3b2584e40 0x55e3b298ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:08.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.480+0000 7fd79d600640 1 --2- 172.21.15.39:0/1419628532 >> [v2:172.21.15.148:6836/3026827614,v1:172.21.15.148:6837/3026827614] conn(0x55e3b2584e40 0x55e3b298ec00 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96dc55935b7d9d6d server_cookie=7c6b18fbc4bff4b6 in_seq=0 out_seq=0 2024-09-21T07:37:08.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.480+0000 7fd77e000640 10 client.6615 ms_handle_connect on v2:172.21.15.148:6836/3026827614 2024-09-21T07:37:08.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.482+0000 7fd77e000640 1 -- 172.21.15.39:0/1419628532 <== mds.0 v2:172.21.15.148:6836/3026827614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e3b28e2520 con 0x55e3b2584e40 2024-09-21T07:37:08.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.482+0000 7fd77e000640 10 client.6615 handle_command_reply: tid=0 2024-09-21T07:37:08.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.482+0000 7fd7a71ee740 1 client.6615 shutdown 2024-09-21T07:37:08.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.482+0000 7fd7a71ee740 1 -- 172.21.15.39:0/1419628532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b25a5590 msgr2=0x55e3b298ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:08.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.482+0000 7fd7a71ee740 1 --2- 172.21.15.39:0/1419628532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b25a5590 0x55e3b298ffc0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fd7840098a0 tx=0x7fd784004730 comp rx=0 tx=0).stop 2024-09-21T07:37:08.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.482+0000 7fd7a71ee740 1 -- 172.21.15.39:0/1419628532 shutdown_connections 2024-09-21T07:37:08.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.482+0000 7fd7a71ee740 1 --2- 172.21.15.39:0/1419628532 >> [v2:172.21.15.148:6836/3026827614,v1:172.21.15.148:6837/3026827614] conn(0x55e3b2584e40 0x55e3b298ec00 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.482+0000 7fd7a71ee740 1 --2- 172.21.15.39:0/1419628532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3b25c6270 0x55e3b29ab080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.482+0000 7fd7a71ee740 1 --2- 172.21.15.39:0/1419628532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b25a5590 0x55e3b298ffc0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.482+0000 7fd7a71ee740 1 --2- 172.21.15.39:0/1419628532 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e3b259d3b0 0x55e3b29c1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.482+0000 7fd77e000640 0 client.6615 ms_handle_reset on v2:172.21.15.148:6836/3026827614 2024-09-21T07:37:08.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.482+0000 7fd7a71ee740 1 -- 172.21.15.39:0/1419628532 >> 172.21.15.39:0/1419628532 conn(0x55e3b25ba3c0 msgr2=0x55e3b29b4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:08.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.482+0000 7fd7a71ee740 1 -- 172.21.15.39:0/1419628532 shutdown_connections 2024-09-21T07:37:08.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.482+0000 7fd7a71ee740 1 -- 172.21.15.39:0/1419628532 wait complete. 2024-09-21T07:37:08.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.483+0000 7fd7a71ee740 20 client.6615 trim_cache size 0 max 16384 2024-09-21T07:37:08.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.502+0000 7fd7a71ee740 1 Processor -- start 2024-09-21T07:37:08.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.502+0000 7fd7a71ee740 1 -- start start 2024-09-21T07:37:08.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.504+0000 7fd7a71ee740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b259d3b0 0x55e3b29a9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:08.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.504+0000 7fd7a71ee740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e3b25a5590 0x55e3b29b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:08.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.504+0000 7fd7a71ee740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3b25c6270 0x55e3b29b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:08.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.504+0000 7fd7a71ee740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd798204a40 con 0x55e3b259d3b0 2024-09-21T07:37:08.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.504+0000 7fd7a71ee740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd798202eb0 con 0x55e3b25c6270 2024-09-21T07:37:08.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.504+0000 7fd7a71ee740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd798203440 con 0x55e3b25a5590 2024-09-21T07:37:08.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.504+0000 7fd79d600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b259d3b0 0x55e3b29a9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:08.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.504+0000 7fd79d600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b259d3b0 0x55e3b29a9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43374/0 (socket says 172.21.15.39:43374) 2024-09-21T07:37:08.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.504+0000 7fd79d600640 1 -- 172.21.15.39:0/3902967555 learned_addr learned my addr 172.21.15.39:0/3902967555 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:08.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.504+0000 7fd79d600640 1 -- 172.21.15.39:0/3902967555 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e3b25a5590 msgr2=0x55e3b29b40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:37:08.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.504+0000 7fd79d600640 1 --2- 172.21.15.39:0/3902967555 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e3b25a5590 0x55e3b29b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.504+0000 7fd79d600640 1 -- 172.21.15.39:0/3902967555 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3b25c6270 msgr2=0x55e3b29b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:37:08.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.504+0000 7fd79d600640 1 --2- 172.21.15.39:0/3902967555 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3b25c6270 0x55e3b29b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.504+0000 7fd79d600640 1 -- 172.21.15.39:0/3902967555 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3b298ca70 con 0x55e3b259d3b0 2024-09-21T07:37:08.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.504+0000 7fd79d600640 1 --2- 172.21.15.39:0/3902967555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b259d3b0 0x55e3b29a9dd0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fd78c05ee70 tx=0x7fd78c00b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e397bad96e7cc950 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:08.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.505+0000 7fd789200640 1 -- 172.21.15.39:0/3902967555 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c05f4b0 con 0x55e3b259d3b0 2024-09-21T07:37:08.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.505+0000 7fd789200640 1 -- 172.21.15.39:0/3902967555 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd78c067940 con 0x55e3b259d3b0 2024-09-21T07:37:08.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.505+0000 7fd789200640 1 -- 172.21.15.39:0/3902967555 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c06c7f0 con 0x55e3b259d3b0 2024-09-21T07:37:08.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.505+0000 7fd7a71ee740 1 -- 172.21.15.39:0/3902967555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b259d3b0 msgr2=0x55e3b29a9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:08.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.505+0000 7fd7a71ee740 1 --2- 172.21.15.39:0/3902967555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b259d3b0 0x55e3b29a9dd0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fd78c05ee70 tx=0x7fd78c00b560 comp rx=0 tx=0).stop 2024-09-21T07:37:08.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.505+0000 7fd7a71ee740 1 -- 172.21.15.39:0/3902967555 shutdown_connections 2024-09-21T07:37:08.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.505+0000 7fd7a71ee740 1 --2- 172.21.15.39:0/3902967555 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3b25c6270 0x55e3b29b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.506+0000 7fd7a71ee740 1 --2- 172.21.15.39:0/3902967555 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e3b25a5590 0x55e3b29b40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.506+0000 7fd7a71ee740 1 --2- 172.21.15.39:0/3902967555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b259d3b0 0x55e3b29a9dd0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.506+0000 7fd7a71ee740 1 -- 172.21.15.39:0/3902967555 >> 172.21.15.39:0/3902967555 conn(0x55e3b25ba3c0 msgr2=0x55e3b29aac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:08.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.506+0000 7fd7a71ee740 1 -- 172.21.15.39:0/3902967555 shutdown_connections 2024-09-21T07:37:08.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.506+0000 7fd7a71ee740 1 -- 172.21.15.39:0/3902967555 wait complete. 2024-09-21T07:37:08.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.507+0000 7fd7a71ee740 1 -- start start 2024-09-21T07:37:08.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.507+0000 7fd7a71ee740 1 Processor -- start 2024-09-21T07:37:08.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.508+0000 7fd7a71ee740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b259d3b0 0x55e3b2a2afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:08.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.508+0000 7fd7a71ee740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3b25a5590 0x55e3b29c2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:08.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.508+0000 7fd7a71ee740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e3b25c6270 0x55e3b29c29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:08.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.508+0000 7fd7a71ee740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55e3b2992660 con 0x55e3b259d3b0 2024-09-21T07:37:08.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.508+0000 7fd7a71ee740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e3b29c4490 con 0x55e3b25a5590 2024-09-21T07:37:08.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.508+0000 7fd7a71ee740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55e3b2992cb0 con 0x55e3b25c6270 2024-09-21T07:37:08.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.508+0000 7fd7a71ee740 10 client.0 _ll_register_callbacks cb 0x7fd7840125b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:37:08.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.509+0000 7fd79d600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b259d3b0 0x55e3b2a2afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:08.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.509+0000 7fd79d600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b259d3b0 0x55e3b2a2afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43376/0 (socket says 172.21.15.39:43376) 2024-09-21T07:37:08.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.509+0000 7fd79d600640 1 -- 172.21.15.39:0/1386673369 learned_addr learned my addr 172.21.15.39:0/1386673369 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:08.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.509+0000 7fd79cc00640 1 --2- 172.21.15.39:0/1386673369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3b25a5590 0x55e3b29c2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:08.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.509+0000 7fd79e000640 1 --2- 172.21.15.39:0/1386673369 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e3b25c6270 0x55e3b29c29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:08.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.509+0000 7fd79d600640 1 -- 172.21.15.39:0/1386673369 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e3b25c6270 msgr2=0x55e3b29c29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:37:08.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.509+0000 7fd79d600640 1 --2- 172.21.15.39:0/1386673369 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e3b25c6270 0x55e3b29c29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.509+0000 7fd79d600640 1 -- 172.21.15.39:0/1386673369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3b25a5590 msgr2=0x55e3b29c2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:37:08.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.509+0000 7fd79d600640 1 --2- 172.21.15.39:0/1386673369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3b25a5590 0x55e3b29c2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.509+0000 7fd79d600640 1 -- 172.21.15.39:0/1386673369 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3b2a34810 con 0x55e3b259d3b0 2024-09-21T07:37:08.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.509+0000 7fd79e000640 1 --2- 172.21.15.39:0/1386673369 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e3b25c6270 0x55e3b29c29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:37:08.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.509+0000 7fd79d600640 1 --2- 172.21.15.39:0/1386673369 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b259d3b0 0x55e3b2a2afc0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fd78c0995c0 tx=0x7fd78c085350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb7d7ce8095cb8c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:08.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.509+0000 7fd77e000640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:37:08.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.509+0000 7fd77e000640 1 -- 172.21.15.39:0/1386673369 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c062e60 con 0x55e3b259d3b0 2024-09-21T07:37:08.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.509+0000 7fd77e000640 1 -- 172.21.15.39:0/1386673369 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6915+0+0 (secure 0 0 0) 0x7fd78c04e1d0 con 0x55e3b259d3b0 2024-09-21T07:37:08.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.509+0000 7fd7a71ee740 1 -- 172.21.15.39:0/1386673369 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd78c067940 con 0x55e3b259d3b0 2024-09-21T07:37:08.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.510+0000 7fd77e000640 1 -- 172.21.15.39:0/1386673369 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fd78c04e3d0 con 0x55e3b259d3b0 2024-09-21T07:37:08.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.510+0000 7fd7a71ee740 10 client.6621 fetch_fsmap learned FSMap version 66 2024-09-21T07:37:08.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.510+0000 7fd7a71ee740 1 -- 172.21.15.39:0/1386673369 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x55e3b29aa960 con 0x55e3b259d3b0 2024-09-21T07:37:08.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.510+0000 7fd77e000640 1 -- 172.21.15.39:0/1386673369 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd78c067940 con 0x55e3b259d3b0 2024-09-21T07:37:08.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.510+0000 7fd77e000640 1 -- 172.21.15.39:0/1386673369 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 66) ==== 3522+0+0 (secure 0 0 0) 0x7fd78c06c360 con 0x55e3b259d3b0 2024-09-21T07:37:08.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.510+0000 7fd7a71ee740 10 client.6621 fetch_fsmap finished waiting for FSMap version 66 2024-09-21T07:37:08.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.510+0000 7fd7a71ee740 10 client.6621 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T07:37:08.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.510+0000 7fd7a71ee740 20 client.6621 populate_metadata read hostname 'smithi039' 2024-09-21T07:37:08.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.510+0000 7fd7a71ee740 1 --2- 172.21.15.39:0/1386673369 >> [v2:172.21.15.148:6836/3026827614,v1:172.21.15.148:6837/3026827614] conn(0x55e3b2a312a0 0x55e3b2a316b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:08.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.510+0000 7fd7a71ee740 4 client.6621 mds_command: new command op to 5621 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:37:08.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.510+0000 7fd7a71ee740 1 -- 172.21.15.39:0/1386673369 --> [v2:172.21.15.148:6836/3026827614,v1:172.21.15.148:6837/3026827614] -- command(tid 0: {"prefix": "status"}) -- 0x55e3b28ddfd0 con 0x55e3b2a312a0 2024-09-21T07:37:08.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.510+0000 7fd79cc00640 1 --2- 172.21.15.39:0/1386673369 >> [v2:172.21.15.148:6836/3026827614,v1:172.21.15.148:6837/3026827614] conn(0x55e3b2a312a0 0x55e3b2a316b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:08.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.511+0000 7fd79cc00640 1 --2- 172.21.15.39:0/1386673369 >> [v2:172.21.15.148:6836/3026827614,v1:172.21.15.148:6837/3026827614] conn(0x55e3b2a312a0 0x55e3b2a316b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fddd2ed990ed314 server_cookie=1783899a1b6fee49 in_seq=0 out_seq=0 2024-09-21T07:37:08.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.511+0000 7fd77e000640 10 client.6621 ms_handle_connect on v2:172.21.15.148:6836/3026827614 2024-09-21T07:37:08.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.511+0000 7fd77e000640 1 -- 172.21.15.39:0/1386673369 <== mds.0 v2:172.21.15.148:6836/3026827614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55e3b28ddfd0 con 0x55e3b2a312a0 2024-09-21T07:37:08.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.511+0000 7fd77e000640 10 client.6621 handle_command_reply: tid=0 2024-09-21T07:37:08.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.511+0000 7fd7a71ee740 1 client.6621 shutdown 2024-09-21T07:37:08.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.512+0000 7fd7a71ee740 1 -- 172.21.15.39:0/1386673369 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b259d3b0 msgr2=0x55e3b2a2afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:08.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.512+0000 7fd7a71ee740 1 --2- 172.21.15.39:0/1386673369 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b259d3b0 0x55e3b2a2afc0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fd78c0995c0 tx=0x7fd78c085350 comp rx=0 tx=0).stop 2024-09-21T07:37:08.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.512+0000 7fd7a71ee740 1 -- 172.21.15.39:0/1386673369 shutdown_connections 2024-09-21T07:37:08.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.512+0000 7fd7a71ee740 1 --2- 172.21.15.39:0/1386673369 >> [v2:172.21.15.148:6836/3026827614,v1:172.21.15.148:6837/3026827614] conn(0x55e3b2a312a0 0x55e3b2a316b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.512+0000 7fd7a71ee740 1 --2- 172.21.15.39:0/1386673369 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55e3b25c6270 0x55e3b29c29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.512+0000 7fd7a71ee740 1 --2- 172.21.15.39:0/1386673369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3b25a5590 0x55e3b29c2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.512+0000 7fd7a71ee740 1 --2- 172.21.15.39:0/1386673369 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e3b259d3b0 0x55e3b2a2afc0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.512+0000 7fd77e000640 0 client.6621 ms_handle_reset on v2:172.21.15.148:6836/3026827614 2024-09-21T07:37:08.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.512+0000 7fd7a71ee740 1 -- 172.21.15.39:0/1386673369 >> 172.21.15.39:0/1386673369 conn(0x55e3b25ba3c0 msgr2=0x55e3b29c4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:08.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.512+0000 7fd7a71ee740 1 -- 172.21.15.39:0/1386673369 shutdown_connections 2024-09-21T07:37:08.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.512+0000 7fd7a71ee740 1 -- 172.21.15.39:0/1386673369 wait complete. 2024-09-21T07:37:08.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.512+0000 7fd7a71ee740 20 client.6621 trim_cache size 0 max 16384 2024-09-21T07:37:08.517 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:37:08.517 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:37:08.517 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-09-21T07:37:08.517 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 5621, 2024-09-21T07:37:08.517 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:37:08.517 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:37:08.518 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:37:08.518 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 7.2402717909999996, 2024-09-21T07:37:08.518 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 66, 2024-09-21T07:37:08.518 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 93, 2024-09-21T07:37:08.518 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 93, 2024-09-21T07:37:08.518 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 52.825468225000002 2024-09-21T07:37:08.518 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:37:08.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.512+0000 7fd7a0000640 1 -- 172.21.15.39:0/2922312422 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd778078b90 msgr2=0x7fd77807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:08.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.512+0000 7fd7a0000640 1 --2- 172.21.15.39:0/2922312422 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd778078b90 0x7fd77807b0a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd79809d950 tx=0x7fd784005f90 comp rx=0 tx=0).stop 2024-09-21T07:37:08.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.513+0000 7fd7a0000640 1 -- 172.21.15.39:0/2922312422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7980cc1e0 msgr2=0x7fd7981537d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:08.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.513+0000 7fd7a0000640 1 --2- 172.21.15.39:0/2922312422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7980cc1e0 0x7fd7981537d0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fd78c00d650 tx=0x7fd78c00db10 comp rx=0 tx=0).stop 2024-09-21T07:37:08.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.513+0000 7fd7a0000640 1 -- 172.21.15.39:0/2922312422 shutdown_connections 2024-09-21T07:37:08.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.513+0000 7fd7a0000640 1 --2- 172.21.15.39:0/2922312422 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd778078b90 0x7fd77807b0a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.513+0000 7fd7a0000640 1 --2- 172.21.15.39:0/2922312422 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd79815d8a0 0x7fd7981e9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.513+0000 7fd7a0000640 1 --2- 172.21.15.39:0/2922312422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7980ccb50 0x7fd7981551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.513+0000 7fd7a0000640 1 --2- 172.21.15.39:0/2922312422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7980cc1e0 0x7fd7981537d0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.513+0000 7fd7a0000640 1 -- 172.21.15.39:0/2922312422 >> 172.21.15.39:0/2922312422 conn(0x7fd79806d7d0 msgr2=0x7fd7980afb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:08.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.513+0000 7fd7a0000640 1 -- 172.21.15.39:0/2922312422 shutdown_connections 2024-09-21T07:37:08.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.513+0000 7fd7a0000640 1 -- 172.21.15.39:0/2922312422 wait complete. 2024-09-21T07:37:08.540 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T07:37:08.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.768+0000 7f5c88600640 1 -- 172.21.15.39:0/2873242391 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c801600a0 msgr2=0x7f5c80071250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:08.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.768+0000 7f5c88600640 1 --2- 172.21.15.39:0/2873242391 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c801600a0 0x7f5c80071250 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f5c8009f280 tx=0x7f5c7402a680 comp rx=0 tx=0).stop 2024-09-21T07:37:08.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.769+0000 7f5c88600640 1 -- 172.21.15.39:0/2873242391 shutdown_connections 2024-09-21T07:37:08.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.769+0000 7f5c88600640 1 --2- 172.21.15.39:0/2873242391 >> v2:172.21.15.148:3300/0 conn(0x7f5c8006d4f0 0x7f5c800cfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.769+0000 7f5c88600640 1 --2- 172.21.15.39:0/2873242391 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c801600a0 0x7f5c80071250 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.769+0000 7f5c88600640 1 -- 172.21.15.39:0/2873242391 >> 172.21.15.39:0/2873242391 conn(0x7f5c800cdcb0 msgr2=0x7f5c800ce0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:08.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.769+0000 7f5c88600640 1 -- 172.21.15.39:0/2873242391 shutdown_connections 2024-09-21T07:37:08.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.769+0000 7f5c88600640 1 -- 172.21.15.39:0/2873242391 wait complete. 2024-09-21T07:37:08.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.771+0000 7f5c88600640 1 Processor -- start 2024-09-21T07:37:08.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.771+0000 7f5c88600640 1 -- start start 2024-09-21T07:37:08.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.773+0000 7f5c88600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c800ce7d0 0x7f5c800caf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:08.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.773+0000 7f5c88600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c801600a0 0x7f5c800ccd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:08.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.773+0000 7f5c88600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c801e6ce0 0x7f5c801e70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:08.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.773+0000 7f5c88600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5c800d0370 con 0x7f5c801e6ce0 2024-09-21T07:37:08.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.773+0000 7f5c88600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5c800d01f0 con 0x7f5c800ce7d0 2024-09-21T07:37:08.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.773+0000 7f5c88600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f5c800d04f0 con 0x7f5c801600a0 2024-09-21T07:37:08.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.773+0000 7f5c86600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c801e6ce0 0x7f5c801e70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:08.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.773+0000 7f5c86600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c801e6ce0 0x7f5c801e70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43392/0 (socket says 172.21.15.39:43392) 2024-09-21T07:37:08.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.773+0000 7f5c86600640 1 -- 172.21.15.39:0/1231713628 learned_addr learned my addr 172.21.15.39:0/1231713628 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:08.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.773+0000 7f5c85200640 1 --2- 172.21.15.39:0/1231713628 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c801600a0 0x7f5c800ccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:08.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.773+0000 7f5c85c00640 1 --2- 172.21.15.39:0/1231713628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c800ce7d0 0x7f5c800caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:08.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.773+0000 7f5c86600640 1 -- 172.21.15.39:0/1231713628 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c801600a0 msgr2=0x7f5c800ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:08.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.773+0000 7f5c86600640 1 --2- 172.21.15.39:0/1231713628 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c801600a0 0x7f5c800ccd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.773+0000 7f5c86600640 1 -- 172.21.15.39:0/1231713628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c800ce7d0 msgr2=0x7f5c800caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:08.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.773+0000 7f5c86600640 1 --2- 172.21.15.39:0/1231713628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c800ce7d0 0x7f5c800caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.773+0000 7f5c86600640 1 -- 172.21.15.39:0/1231713628 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c801e7630 con 0x7f5c801e6ce0 2024-09-21T07:37:08.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.773+0000 7f5c85200640 1 --2- 172.21.15.39:0/1231713628 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c801600a0 0x7f5c800ccd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:37:08.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.773+0000 7f5c85c00640 1 --2- 172.21.15.39:0/1231713628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c800ce7d0 0x7f5c800caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:37:08.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.774+0000 7f5c86600640 1 --2- 172.21.15.39:0/1231713628 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c801e6ce0 0x7f5c801e70f0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f5c7800e990 tx=0x7f5c7800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:08.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.774+0000 7f5c7d600640 1 -- 172.21.15.39:0/1231713628 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c78004d20 con 0x7f5c801e6ce0 2024-09-21T07:37:08.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.774+0000 7f5c7d600640 1 -- 172.21.15.39:0/1231713628 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c7800caa0 con 0x7f5c801e6ce0 2024-09-21T07:37:08.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.774+0000 7f5c88600640 1 -- 172.21.15.39:0/1231713628 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c800c9280 con 0x7f5c801e6ce0 2024-09-21T07:37:08.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.774+0000 7f5c7d600640 1 -- 172.21.15.39:0/1231713628 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c78005020 con 0x7f5c801e6ce0 2024-09-21T07:37:08.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.774+0000 7f5c88600640 1 -- 172.21.15.39:0/1231713628 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c800c9840 con 0x7f5c801e6ce0 2024-09-21T07:37:08.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.777+0000 7f5c7d600640 1 -- 172.21.15.39:0/1231713628 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5c780051c0 con 0x7f5c801e6ce0 2024-09-21T07:37:08.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.777+0000 7f5c7d600640 1 --2- 172.21.15.39:0/1231713628 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5c60078b90 0x7f5c6007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:08.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.778+0000 7f5c7d600640 1 -- 172.21.15.39:0/1231713628 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6915+0+0 (secure 0 0 0) 0x7f5c7809bb50 con 0x7f5c801e6ce0 2024-09-21T07:37:08.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.778+0000 7f5c8f5c2740 1 Processor -- start 2024-09-21T07:37:08.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.778+0000 7f5c85c00640 1 --2- 172.21.15.39:0/1231713628 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5c60078b90 0x7f5c6007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:08.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.778+0000 7f5c8f5c2740 1 -- start start 2024-09-21T07:37:08.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.779+0000 7f5c85c00640 1 --2- 172.21.15.39:0/1231713628 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5c60078b90 0x7f5c6007b0a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5c74033000 tx=0x7f5c74004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:08.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.780+0000 7f5c8f5c2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c2b1b270 0x5568c2eeb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:08.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.780+0000 7f5c8f5c2740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5568c2af23b0 0x5568c2eebeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:08.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.780+0000 7f5c8f5c2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5568c2afa590 0x5568c2ebbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:08.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.780+0000 7f5c8f5c2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5568c2e45430 con 0x5568c2b1b270 2024-09-21T07:37:08.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.780+0000 7f5c8f5c2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5568c2e355d0 con 0x5568c2afa590 2024-09-21T07:37:08.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.780+0000 7f5c8f5c2740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x5568c2e24a00 con 0x5568c2af23b0 2024-09-21T07:37:08.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.780+0000 7f5c85c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c2b1b270 0x5568c2eeb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:08.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.780+0000 7f5c85c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c2b1b270 0x5568c2eeb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43408/0 (socket says 172.21.15.39:43408) 2024-09-21T07:37:08.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.780+0000 7f5c85c00640 1 -- 172.21.15.39:0/1298634721 learned_addr learned my addr 172.21.15.39:0/1298634721 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:08.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.780+0000 7f5c85200640 1 --2- 172.21.15.39:0/1298634721 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5568c2af23b0 0x5568c2eebeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:08.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.780+0000 7f5c86600640 1 --2- 172.21.15.39:0/1298634721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5568c2afa590 0x5568c2ebbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:08.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.780+0000 7f5c85c00640 1 -- 172.21.15.39:0/1298634721 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5568c2af23b0 msgr2=0x5568c2eebeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:37:08.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.780+0000 7f5c85c00640 1 --2- 172.21.15.39:0/1298634721 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5568c2af23b0 0x5568c2eebeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.780+0000 7f5c85c00640 1 -- 172.21.15.39:0/1298634721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5568c2afa590 msgr2=0x5568c2ebbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:37:08.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.780+0000 7f5c85c00640 1 --2- 172.21.15.39:0/1298634721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5568c2afa590 0x5568c2ebbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.780+0000 7f5c85c00640 1 -- 172.21.15.39:0/1298634721 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568c2ded520 con 0x5568c2b1b270 2024-09-21T07:37:08.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.781+0000 7f5c86600640 1 --2- 172.21.15.39:0/1298634721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5568c2afa590 0x5568c2ebbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:37:08.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.781+0000 7f5c85c00640 1 --2- 172.21.15.39:0/1298634721 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c2b1b270 0x5568c2eeb970 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f5c74034040 tx=0x7f5c7403c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7f75d27a22fe3fe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:08.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.781+0000 7f5c72000640 1 -- 172.21.15.39:0/1298634721 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c74035040 con 0x5568c2b1b270 2024-09-21T07:37:08.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.781+0000 7f5c72000640 1 -- 172.21.15.39:0/1298634721 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c7404d070 con 0x5568c2b1b270 2024-09-21T07:37:08.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.781+0000 7f5c72000640 1 -- 172.21.15.39:0/1298634721 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c74003ba0 con 0x5568c2b1b270 2024-09-21T07:37:08.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.782+0000 7f5c8f5c2740 1 -- 172.21.15.39:0/1298634721 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c2b1b270 msgr2=0x5568c2eeb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:08.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.782+0000 7f5c8f5c2740 1 --2- 172.21.15.39:0/1298634721 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c2b1b270 0x5568c2eeb970 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f5c74034040 tx=0x7f5c7403c000 comp rx=0 tx=0).stop 2024-09-21T07:37:08.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.782+0000 7f5c8f5c2740 1 -- 172.21.15.39:0/1298634721 shutdown_connections 2024-09-21T07:37:08.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.782+0000 7f5c8f5c2740 1 --2- 172.21.15.39:0/1298634721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5568c2afa590 0x5568c2ebbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.782+0000 7f5c8f5c2740 1 --2- 172.21.15.39:0/1298634721 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5568c2af23b0 0x5568c2eebeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.782+0000 7f5c8f5c2740 1 --2- 172.21.15.39:0/1298634721 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c2b1b270 0x5568c2eeb970 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.782+0000 7f5c8f5c2740 1 -- 172.21.15.39:0/1298634721 >> 172.21.15.39:0/1298634721 conn(0x5568c2b0f3c0 msgr2=0x5568c2ec1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:08.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.782+0000 7f5c8f5c2740 1 -- 172.21.15.39:0/1298634721 shutdown_connections 2024-09-21T07:37:08.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.782+0000 7f5c8f5c2740 1 -- 172.21.15.39:0/1298634721 wait complete. 2024-09-21T07:37:08.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.783+0000 7f5c8f5c2740 1 -- start start 2024-09-21T07:37:08.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.783+0000 7f5c8f5c2740 1 Processor -- start 2024-09-21T07:37:08.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.785+0000 7f5c8f5c2740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5568c2af23b0 0x5568c2ee1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:08.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.785+0000 7f5c8f5c2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c2afa590 0x5568c2f15bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:08.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.785+0000 7f5c8f5c2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5568c2b1b270 0x5568c2ee7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:08.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.785+0000 7f5c8f5c2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5568c2dcb330 con 0x5568c2afa590 2024-09-21T07:37:08.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.785+0000 7f5c8f5c2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5568c2e28810 con 0x5568c2b1b270 2024-09-21T07:37:08.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.785+0000 7f5c8f5c2740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x5568c2b95620 con 0x5568c2af23b0 2024-09-21T07:37:08.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.785+0000 7f5c8f5c2740 10 client.0 _ll_register_callbacks cb 0x5568c2dd0420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:37:08.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.785+0000 7f5c85200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c2afa590 0x5568c2f15bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:08.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.785+0000 7f5c85200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c2afa590 0x5568c2f15bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43424/0 (socket says 172.21.15.39:43424) 2024-09-21T07:37:08.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.785+0000 7f5c85200640 1 -- 172.21.15.39:0/994722034 learned_addr learned my addr 172.21.15.39:0/994722034 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:08.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.785+0000 7f5c85c00640 1 --2- 172.21.15.39:0/994722034 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5568c2af23b0 0x5568c2ee1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:08.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.785+0000 7f5c86600640 1 --2- 172.21.15.39:0/994722034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5568c2b1b270 0x5568c2ee7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:08.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.785+0000 7f5c85200640 1 -- 172.21.15.39:0/994722034 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5568c2af23b0 msgr2=0x5568c2ee1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:37:08.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.785+0000 7f5c85200640 1 --2- 172.21.15.39:0/994722034 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5568c2af23b0 0x5568c2ee1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.785+0000 7f5c85200640 1 -- 172.21.15.39:0/994722034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5568c2b1b270 msgr2=0x5568c2ee7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:37:08.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.785+0000 7f5c85200640 1 --2- 172.21.15.39:0/994722034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5568c2b1b270 0x5568c2ee7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.785+0000 7f5c85200640 1 -- 172.21.15.39:0/994722034 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568c2b86850 con 0x5568c2afa590 2024-09-21T07:37:08.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.786+0000 7f5c85200640 1 --2- 172.21.15.39:0/994722034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c2afa590 0x5568c2f15bd0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c003ec0 tx=0x7f5c6c0014b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeaa6b51b2318101 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:08.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.786+0000 7f5c66a00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:37:08.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.786+0000 7f5c66a00640 1 -- 172.21.15.39:0/994722034 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c014400 con 0x5568c2afa590 2024-09-21T07:37:08.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.786+0000 7f5c66a00640 1 -- 172.21.15.39:0/994722034 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6915+0+0 (secure 0 0 0) 0x7f5c6c0145a0 con 0x5568c2afa590 2024-09-21T07:37:08.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.786+0000 7f5c8f5c2740 1 -- 172.21.15.39:0/994722034 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c7404d070 con 0x5568c2afa590 2024-09-21T07:37:08.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.786+0000 7f5c66a00640 1 -- 172.21.15.39:0/994722034 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f5c6c021050 con 0x5568c2afa590 2024-09-21T07:37:08.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.786+0000 7f5c8f5c2740 10 client.6633 fetch_fsmap learned FSMap version 66 2024-09-21T07:37:08.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.786+0000 7f5c8f5c2740 1 -- 172.21.15.39:0/994722034 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x5568c2bfd6b0 con 0x5568c2afa590 2024-09-21T07:37:08.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.786+0000 7f5c66a00640 1 -- 172.21.15.39:0/994722034 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c7404d070 con 0x5568c2afa590 2024-09-21T07:37:08.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.787+0000 7f5c66a00640 1 -- 172.21.15.39:0/994722034 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 66) ==== 3522+0+0 (secure 0 0 0) 0x7f5c6c024020 con 0x5568c2afa590 2024-09-21T07:37:08.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.787+0000 7f5c8f5c2740 10 client.6633 fetch_fsmap finished waiting for FSMap version 66 2024-09-21T07:37:08.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.787+0000 7f5c8f5c2740 10 client.6633 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:37:08.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.787+0000 7f5c8f5c2740 20 client.6633 populate_metadata read hostname 'smithi039' 2024-09-21T07:37:08.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.787+0000 7f5c8f5c2740 1 --2- 172.21.15.39:0/994722034 >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x5568c2ad9e40 0x5568c2f191c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:08.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.787+0000 7f5c8f5c2740 4 client.6633 mds_command: new command op to 6115 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:37:08.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.787+0000 7f5c8f5c2740 1 -- 172.21.15.39:0/994722034 --> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568c2e37520 con 0x5568c2ad9e40 2024-09-21T07:37:08.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.787+0000 7f5c85c00640 1 --2- 172.21.15.39:0/994722034 >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x5568c2ad9e40 0x5568c2f191c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:08.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.788+0000 7f5c85c00640 1 --2- 172.21.15.39:0/994722034 >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x5568c2ad9e40 0x5568c2f191c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6e6f4ce66087e7a7 server_cookie=7f4c112eb8c9cf68 in_seq=0 out_seq=0 2024-09-21T07:37:08.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.788+0000 7f5c66a00640 10 client.6633 ms_handle_connect on v2:172.21.15.39:6833/364657398 2024-09-21T07:37:08.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.788+0000 7f5c66a00640 1 -- 172.21.15.39:0/994722034 <== mds.1 v2:172.21.15.39:6833/364657398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5568c2e37520 con 0x5568c2ad9e40 2024-09-21T07:37:08.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.788+0000 7f5c66a00640 10 client.6633 handle_command_reply: tid=0 2024-09-21T07:37:08.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.788+0000 7f5c8f5c2740 1 client.6633 shutdown 2024-09-21T07:37:08.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.789+0000 7f5c8f5c2740 1 -- 172.21.15.39:0/994722034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c2afa590 msgr2=0x5568c2f15bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:08.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.789+0000 7f5c8f5c2740 1 --2- 172.21.15.39:0/994722034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c2afa590 0x5568c2f15bd0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c003ec0 tx=0x7f5c6c0014b0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.789+0000 7f5c8f5c2740 1 -- 172.21.15.39:0/994722034 shutdown_connections 2024-09-21T07:37:08.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.789+0000 7f5c8f5c2740 1 --2- 172.21.15.39:0/994722034 >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x5568c2ad9e40 0x5568c2f191c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.789+0000 7f5c8f5c2740 1 --2- 172.21.15.39:0/994722034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5568c2b1b270 0x5568c2ee7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.789+0000 7f5c8f5c2740 1 --2- 172.21.15.39:0/994722034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c2afa590 0x5568c2f15bd0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.789+0000 7f5c8f5c2740 1 --2- 172.21.15.39:0/994722034 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5568c2af23b0 0x5568c2ee1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.789+0000 7f5c8f5c2740 1 -- 172.21.15.39:0/994722034 >> 172.21.15.39:0/994722034 conn(0x5568c2b0f3c0 msgr2=0x5568c2efcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:08.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.789+0000 7f5c66a00640 0 client.6633 ms_handle_reset on v2:172.21.15.39:6833/364657398 2024-09-21T07:37:08.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.789+0000 7f5c8f5c2740 1 -- 172.21.15.39:0/994722034 shutdown_connections 2024-09-21T07:37:08.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.789+0000 7f5c8f5c2740 1 -- 172.21.15.39:0/994722034 wait complete. 2024-09-21T07:37:08.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.789+0000 7f5c8f5c2740 20 client.6633 trim_cache size 0 max 16384 2024-09-21T07:37:08.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.808+0000 7f5c8f5c2740 1 Processor -- start 2024-09-21T07:37:08.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.808+0000 7f5c8f5c2740 1 -- start start 2024-09-21T07:37:08.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.810+0000 7f5c8f5c2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5568c2af23b0 0x5568c2ec19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:08.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.810+0000 7f5c8f5c2740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5568c2afa590 0x5568c2f087b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:08.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.810+0000 7f5c8f5c2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c2b1b270 0x5568c2f08cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:08.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.810+0000 7f5c8f5c2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5c802048d0 con 0x5568c2b1b270 2024-09-21T07:37:08.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.810+0000 7f5c8f5c2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5c80202bd0 con 0x5568c2af23b0 2024-09-21T07:37:08.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.810+0000 7f5c8f5c2740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f5c802031f0 con 0x5568c2afa590 2024-09-21T07:37:08.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.810+0000 7f5c86600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c2b1b270 0x5568c2f08cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:08.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.810+0000 7f5c86600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c2b1b270 0x5568c2f08cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43434/0 (socket says 172.21.15.39:43434) 2024-09-21T07:37:08.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.810+0000 7f5c86600640 1 -- 172.21.15.39:0/942778768 learned_addr learned my addr 172.21.15.39:0/942778768 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:08.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.810+0000 7f5c85200640 1 --2- 172.21.15.39:0/942778768 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5568c2afa590 0x5568c2f087b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:08.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.810+0000 7f5c85c00640 1 --2- 172.21.15.39:0/942778768 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5568c2af23b0 0x5568c2ec19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:08.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.810+0000 7f5c86600640 1 -- 172.21.15.39:0/942778768 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5568c2afa590 msgr2=0x5568c2f087b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:37:08.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.810+0000 7f5c86600640 1 --2- 172.21.15.39:0/942778768 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5568c2afa590 0x5568c2f087b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.810+0000 7f5c86600640 1 -- 172.21.15.39:0/942778768 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5568c2af23b0 msgr2=0x5568c2ec19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:37:08.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.810+0000 7f5c86600640 1 --2- 172.21.15.39:0/942778768 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5568c2af23b0 0x5568c2ec19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.810+0000 7f5c86600640 1 -- 172.21.15.39:0/942778768 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568c2f003c0 con 0x5568c2b1b270 2024-09-21T07:37:08.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.810+0000 7f5c85200640 1 --2- 172.21.15.39:0/942778768 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5568c2afa590 0x5568c2f087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:37:08.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.810+0000 7f5c85c00640 1 --2- 172.21.15.39:0/942778768 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5568c2af23b0 0x5568c2ec19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:37:08.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.811+0000 7f5c86600640 1 --2- 172.21.15.39:0/942778768 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c2b1b270 0x5568c2f08cf0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f5c78060630 tx=0x7f5c78062c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d59f01dc1e08b7f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:08.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.811+0000 7f5c71600640 1 -- 172.21.15.39:0/942778768 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7801f040 con 0x5568c2b1b270 2024-09-21T07:37:08.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.811+0000 7f5c71600640 1 -- 172.21.15.39:0/942778768 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c78089030 con 0x5568c2b1b270 2024-09-21T07:37:08.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.811+0000 7f5c71600640 1 -- 172.21.15.39:0/942778768 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c78085d70 con 0x5568c2b1b270 2024-09-21T07:37:08.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.812+0000 7f5c8f5c2740 1 -- 172.21.15.39:0/942778768 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c2b1b270 msgr2=0x5568c2f08cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:08.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.812+0000 7f5c8f5c2740 1 --2- 172.21.15.39:0/942778768 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c2b1b270 0x5568c2f08cf0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f5c78060630 tx=0x7f5c78062c40 comp rx=0 tx=0).stop 2024-09-21T07:37:08.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.812+0000 7f5c8f5c2740 1 -- 172.21.15.39:0/942778768 shutdown_connections 2024-09-21T07:37:08.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.812+0000 7f5c8f5c2740 1 --2- 172.21.15.39:0/942778768 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c2b1b270 0x5568c2f08cf0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.812+0000 7f5c8f5c2740 1 --2- 172.21.15.39:0/942778768 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5568c2afa590 0x5568c2f087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.812+0000 7f5c8f5c2740 1 --2- 172.21.15.39:0/942778768 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5568c2af23b0 0x5568c2ec19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.812+0000 7f5c8f5c2740 1 -- 172.21.15.39:0/942778768 >> 172.21.15.39:0/942778768 conn(0x5568c2b0f3c0 msgr2=0x5568c2f159b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:08.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.812+0000 7f5c8f5c2740 1 -- 172.21.15.39:0/942778768 shutdown_connections 2024-09-21T07:37:08.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.812+0000 7f5c8f5c2740 1 -- 172.21.15.39:0/942778768 wait complete. 2024-09-21T07:37:08.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.813+0000 7f5c8f5c2740 1 -- start start 2024-09-21T07:37:08.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.813+0000 7f5c8f5c2740 1 Processor -- start 2024-09-21T07:37:08.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.814+0000 7f5c8f5c2740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5568c2af23b0 0x5568c2f04500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:08.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.814+0000 7f5c8f5c2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c2afa590 0x5568c2f089a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:08.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.814+0000 7f5c8f5c2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5568c2b1b270 0x5568c2f88e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:08.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.814+0000 7f5c8f5c2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5568c2ee5180 con 0x5568c2afa590 2024-09-21T07:37:08.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.814+0000 7f5c8f5c2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5568c2f16b20 con 0x5568c2b1b270 2024-09-21T07:37:08.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.815+0000 7f5c8f5c2740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x5568c2ee57f0 con 0x5568c2af23b0 2024-09-21T07:37:08.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.815+0000 7f5c8f5c2740 10 client.0 _ll_register_callbacks cb 0x7f5c6c005050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:37:08.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.815+0000 7f5c85200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c2afa590 0x5568c2f089a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:08.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.815+0000 7f5c85200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c2afa590 0x5568c2f089a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43444/0 (socket says 172.21.15.39:43444) 2024-09-21T07:37:08.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.815+0000 7f5c85200640 1 -- 172.21.15.39:0/2545715526 learned_addr learned my addr 172.21.15.39:0/2545715526 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:08.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.815+0000 7f5c85200640 1 -- 172.21.15.39:0/2545715526 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5568c2af23b0 msgr2=0x5568c2f04500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:37:08.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.815+0000 7f5c85200640 1 --2- 172.21.15.39:0/2545715526 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5568c2af23b0 0x5568c2f04500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.815+0000 7f5c85200640 1 -- 172.21.15.39:0/2545715526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5568c2b1b270 msgr2=0x5568c2f88e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:37:08.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.815+0000 7f5c85200640 1 --2- 172.21.15.39:0/2545715526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5568c2b1b270 0x5568c2f88e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.815+0000 7f5c85200640 1 -- 172.21.15.39:0/2545715526 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568c2f20010 con 0x5568c2afa590 2024-09-21T07:37:08.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.815+0000 7f5c85200640 1 --2- 172.21.15.39:0/2545715526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c2afa590 0x5568c2f089a0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c003a10 tx=0x7f5c6c014d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0d80158e7245570 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:08.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.815+0000 7f5c66a00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:37:08.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.816+0000 7f5c66a00640 1 -- 172.21.15.39:0/2545715526 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c0019b0 con 0x5568c2afa590 2024-09-21T07:37:08.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.816+0000 7f5c66a00640 1 -- 172.21.15.39:0/2545715526 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6915+0+0 (secure 0 0 0) 0x7f5c6c01c350 con 0x5568c2afa590 2024-09-21T07:37:08.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.816+0000 7f5c8f5c2740 1 -- 172.21.15.39:0/2545715526 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c78089030 con 0x5568c2afa590 2024-09-21T07:37:08.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.816+0000 7f5c66a00640 1 -- 172.21.15.39:0/2545715526 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f5c6c005100 con 0x5568c2afa590 2024-09-21T07:37:08.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.816+0000 7f5c8f5c2740 10 client.6639 fetch_fsmap learned FSMap version 66 2024-09-21T07:37:08.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.816+0000 7f5c8f5c2740 1 -- 172.21.15.39:0/2545715526 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x5568c2eb77b0 con 0x5568c2afa590 2024-09-21T07:37:08.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.816+0000 7f5c66a00640 1 -- 172.21.15.39:0/2545715526 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c78089030 con 0x5568c2afa590 2024-09-21T07:37:08.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.816+0000 7f5c66a00640 1 -- 172.21.15.39:0/2545715526 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 66) ==== 3522+0+0 (secure 0 0 0) 0x7f5c6c024050 con 0x5568c2afa590 2024-09-21T07:37:08.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.817+0000 7f5c8f5c2740 10 client.6639 fetch_fsmap finished waiting for FSMap version 66 2024-09-21T07:37:08.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.817+0000 7f5c8f5c2740 10 client.6639 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:37:08.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.817+0000 7f5c8f5c2740 20 client.6639 populate_metadata read hostname 'smithi039' 2024-09-21T07:37:08.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.817+0000 7f5c8f5c2740 1 --2- 172.21.15.39:0/2545715526 >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x5568c2f895a0 0x5568c2f8a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:08.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.817+0000 7f5c8f5c2740 4 client.6639 mds_command: new command op to 6115 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:37:08.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.817+0000 7f5c8f5c2740 1 -- 172.21.15.39:0/2545715526 --> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] -- command(tid 0: {"prefix": "status"}) -- 0x5568c2e32fd0 con 0x5568c2f895a0 2024-09-21T07:37:08.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.817+0000 7f5c85c00640 1 --2- 172.21.15.39:0/2545715526 >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x5568c2f895a0 0x5568c2f8a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:08.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.817+0000 7f5c66a00640 10 client.6639 ms_handle_connect on v2:172.21.15.39:6833/364657398 2024-09-21T07:37:08.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.817+0000 7f5c85c00640 1 --2- 172.21.15.39:0/2545715526 >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x5568c2f895a0 0x5568c2f8a4f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5205af6e9e6b9c2c server_cookie=bd9a0c7c22ee212a in_seq=0 out_seq=0 2024-09-21T07:37:08.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.818+0000 7f5c66a00640 1 -- 172.21.15.39:0/2545715526 <== mds.1 v2:172.21.15.39:6833/364657398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5568c2e32fd0 con 0x5568c2f895a0 2024-09-21T07:37:08.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.818+0000 7f5c66a00640 10 client.6639 handle_command_reply: tid=0 2024-09-21T07:37:08.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.818+0000 7f5c8f5c2740 1 client.6639 shutdown 2024-09-21T07:37:08.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.818+0000 7f5c8f5c2740 1 -- 172.21.15.39:0/2545715526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c2afa590 msgr2=0x5568c2f089a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:08.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.818+0000 7f5c8f5c2740 1 --2- 172.21.15.39:0/2545715526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c2afa590 0x5568c2f089a0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c003a10 tx=0x7f5c6c014d70 comp rx=0 tx=0).stop 2024-09-21T07:37:08.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.818+0000 7f5c8f5c2740 1 -- 172.21.15.39:0/2545715526 shutdown_connections 2024-09-21T07:37:08.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.818+0000 7f5c8f5c2740 1 --2- 172.21.15.39:0/2545715526 >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x5568c2f895a0 0x5568c2f8a4f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.818+0000 7f5c8f5c2740 1 --2- 172.21.15.39:0/2545715526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5568c2b1b270 0x5568c2f88e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.818+0000 7f5c8f5c2740 1 --2- 172.21.15.39:0/2545715526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5568c2afa590 0x5568c2f089a0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.818+0000 7f5c8f5c2740 1 --2- 172.21.15.39:0/2545715526 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5568c2af23b0 0x5568c2f04500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.818+0000 7f5c66a00640 0 client.6639 ms_handle_reset on v2:172.21.15.39:6833/364657398 2024-09-21T07:37:08.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.818+0000 7f5c8f5c2740 1 -- 172.21.15.39:0/2545715526 >> 172.21.15.39:0/2545715526 conn(0x5568c2b0f3c0 msgr2=0x5568c2f29850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:08.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.818+0000 7f5c8f5c2740 1 -- 172.21.15.39:0/2545715526 shutdown_connections 2024-09-21T07:37:08.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.818+0000 7f5c8f5c2740 1 -- 172.21.15.39:0/2545715526 wait complete. 2024-09-21T07:37:08.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.818+0000 7f5c8f5c2740 20 client.6639 trim_cache size 0 max 16384 2024-09-21T07:37:08.823 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:37:08.823 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:37:08.823 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 1, 2024-09-21T07:37:08.823 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 6115, 2024-09-21T07:37:08.823 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:37:08.823 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:37:08.823 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:37:08.823 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 37.454708402000001, 2024-09-21T07:37:08.823 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 66, 2024-09-21T07:37:08.824 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 93, 2024-09-21T07:37:08.824 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 93, 2024-09-21T07:37:08.824 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 53.129391366 2024-09-21T07:37:08.824 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:37:08.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.819+0000 7f5c88600640 1 -- 172.21.15.39:0/1231713628 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5c60078b90 msgr2=0x7f5c6007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:08.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.819+0000 7f5c88600640 1 --2- 172.21.15.39:0/1231713628 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5c60078b90 0x7f5c6007b0a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5c74033000 tx=0x7f5c74004730 comp rx=0 tx=0).stop 2024-09-21T07:37:08.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.819+0000 7f5c88600640 1 -- 172.21.15.39:0/1231713628 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c801e6ce0 msgr2=0x7f5c801e70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:08.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.819+0000 7f5c88600640 1 --2- 172.21.15.39:0/1231713628 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c801e6ce0 0x7f5c801e70f0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f5c7800e990 tx=0x7f5c7800ee50 comp rx=0 tx=0).stop 2024-09-21T07:37:08.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.819+0000 7f5c88600640 1 -- 172.21.15.39:0/1231713628 shutdown_connections 2024-09-21T07:37:08.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.819+0000 7f5c88600640 1 --2- 172.21.15.39:0/1231713628 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5c60078b90 0x7f5c6007b0a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.819+0000 7f5c88600640 1 --2- 172.21.15.39:0/1231713628 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c801e6ce0 0x7f5c801e70f0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.819+0000 7f5c88600640 1 --2- 172.21.15.39:0/1231713628 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c801600a0 0x7f5c800ccd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.819+0000 7f5c88600640 1 --2- 172.21.15.39:0/1231713628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c800ce7d0 0x7f5c800caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:08.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.819+0000 7f5c88600640 1 -- 172.21.15.39:0/1231713628 >> 172.21.15.39:0/1231713628 conn(0x7f5c800cdcb0 msgr2=0x7f5c8015c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:08.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.819+0000 7f5c88600640 1 -- 172.21.15.39:0/1231713628 shutdown_connections 2024-09-21T07:37:08.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:08.819+0000 7f5c88600640 1 -- 172.21.15.39:0/1231713628 wait complete. 2024-09-21T07:37:08.843 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:37:09.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.068+0000 7f56b3e00640 1 -- 172.21.15.39:0/1881632079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56b40b5470 msgr2=0x7f56b40b9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:09.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.068+0000 7f56b3e00640 1 --2- 172.21.15.39:0/1881632079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56b40b5470 0x7f56b40b9100 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f56b409ddd0 tx=0x7f56a402a560 comp rx=0 tx=0).stop 2024-09-21T07:37:09.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.068+0000 7f56b3e00640 1 -- 172.21.15.39:0/1881632079 shutdown_connections 2024-09-21T07:37:09.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.068+0000 7f56b3e00640 1 --2- 172.21.15.39:0/1881632079 >> v2:172.21.15.148:3300/0 conn(0x7f56b40b9640 0x7f56b40d2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:09.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.068+0000 7f56b3e00640 1 --2- 172.21.15.39:0/1881632079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56b40b5470 0x7f56b40b9100 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:09.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.068+0000 7f56b3e00640 1 -- 172.21.15.39:0/1881632079 >> 172.21.15.39:0/1881632079 conn(0x7f56b409d6a0 msgr2=0x7f56b40d4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:09.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.068+0000 7f56b3e00640 1 -- 172.21.15.39:0/1881632079 shutdown_connections 2024-09-21T07:37:09.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.068+0000 7f56b3e00640 1 -- 172.21.15.39:0/1881632079 wait complete. 2024-09-21T07:37:09.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.070+0000 7f56b3e00640 1 Processor -- start 2024-09-21T07:37:09.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.070+0000 7f56b3e00640 1 -- start start 2024-09-21T07:37:09.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.071+0000 7f56b3e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56b40b5470 0x7f56b414d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:09.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.071+0000 7f56b3e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56b40b9640 0x7f56b41e9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:09.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.071+0000 7f56b3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56b40d2650 0x7f56b40b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:09.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.071+0000 7f56b3e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f56b40b4160 con 0x7f56b40d2650 2024-09-21T07:37:09.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.071+0000 7f56b3e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f56b40b8680 con 0x7f56b40b5470 2024-09-21T07:37:09.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.071+0000 7f56b3e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f56b40b42e0 con 0x7f56b40b9640 2024-09-21T07:37:09.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.072+0000 7f56b3400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56b40d2650 0x7f56b40b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:09.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.072+0000 7f56b3400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56b40d2650 0x7f56b40b2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43466/0 (socket says 172.21.15.39:43466) 2024-09-21T07:37:09.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.072+0000 7f56b3400640 1 -- 172.21.15.39:0/3198488087 learned_addr learned my addr 172.21.15.39:0/3198488087 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:09.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.072+0000 7f56b2a00640 1 --2- 172.21.15.39:0/3198488087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56b40b5470 0x7f56b414d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:09.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.072+0000 7f56abe00640 1 --2- 172.21.15.39:0/3198488087 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56b40b9640 0x7f56b41e9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:09.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.072+0000 7f56b3400640 1 -- 172.21.15.39:0/3198488087 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56b40b9640 msgr2=0x7f56b41e9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:09.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.072+0000 7f56b3400640 1 --2- 172.21.15.39:0/3198488087 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56b40b9640 0x7f56b41e9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:09.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.072+0000 7f56b3400640 1 -- 172.21.15.39:0/3198488087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56b40b5470 msgr2=0x7f56b414d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:09.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.072+0000 7f56b3400640 1 --2- 172.21.15.39:0/3198488087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56b40b5470 0x7f56b414d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:09.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.072+0000 7f56b3400640 1 -- 172.21.15.39:0/3198488087 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b41f97d0 con 0x7f56b40d2650 2024-09-21T07:37:09.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.072+0000 7f56abe00640 1 --2- 172.21.15.39:0/3198488087 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56b40b9640 0x7f56b41e9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:37:09.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.072+0000 7f56b3400640 1 --2- 172.21.15.39:0/3198488087 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56b40d2650 0x7f56b40b2080 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f56ac00b9e0 tx=0x7f56ac00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:09.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.073+0000 7f56aaa00640 1 -- 172.21.15.39:0/3198488087 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac019070 con 0x7f56b40d2650 2024-09-21T07:37:09.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.073+0000 7f56aaa00640 1 -- 172.21.15.39:0/3198488087 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56ac00cc10 con 0x7f56b40d2650 2024-09-21T07:37:09.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.073+0000 7f56aaa00640 1 -- 172.21.15.39:0/3198488087 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac0146a0 con 0x7f56b40d2650 2024-09-21T07:37:09.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.073+0000 7f56b3e00640 1 -- 172.21.15.39:0/3198488087 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56b41f9ab0 con 0x7f56b40d2650 2024-09-21T07:37:09.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.073+0000 7f56b3e00640 1 -- 172.21.15.39:0/3198488087 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56b41fa070 con 0x7f56b40d2650 2024-09-21T07:37:09.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.074+0000 7f56b3e00640 1 -- 172.21.15.39:0/3198488087 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56b40b9100 con 0x7f56b40d2650 2024-09-21T07:37:09.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.075+0000 7f56aaa00640 1 -- 172.21.15.39:0/3198488087 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f56ac01c070 con 0x7f56b40d2650 2024-09-21T07:37:09.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.076+0000 7f56aaa00640 1 --2- 172.21.15.39:0/3198488087 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5690078b90 0x7f569007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:09.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.076+0000 7f56aaa00640 1 -- 172.21.15.39:0/3198488087 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6915+0+0 (secure 0 0 0) 0x7f56ac09af40 con 0x7f56b40d2650 2024-09-21T07:37:09.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.077+0000 7f56b2a00640 1 --2- 172.21.15.39:0/3198488087 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5690078b90 0x7f569007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:09.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.077+0000 7f56b2a00640 1 --2- 172.21.15.39:0/3198488087 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5690078b90 0x7f569007b0a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f56a4033000 tx=0x7f56a4004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:09.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.081+0000 7f56aaa00640 1 -- 172.21.15.39:0/3198488087 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f56ac062eb0 con 0x7f56b40d2650 2024-09-21T07:37:09.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.313+0000 7f56b3e00640 1 -- 172.21.15.39:0/3198488087 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56b40d4900 con 0x7f56b40d2650 2024-09-21T07:37:09.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.314+0000 7f56aaa00640 1 -- 172.21.15.39:0/3198488087 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5249 (secure 0 0 0) 0x7f56ac0627b0 con 0x7f56b40d2650 2024-09-21T07:37:09.316 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:37:09.316 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":66,"btime":"2024-09-21T07:37:05:276431+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6585,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/1472430787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1472430787},{"type":"v1","addr":"172.21.15.39:6834","nonce":1472430787}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:37:05.276415+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5621,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5621":{"gid":5621,"name":"d","rank":0,"incarnation":62,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3026827614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3026827614},{"type":"v1","addr":"172.21.15.148:6837","nonce":3026827614}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5621,"qdb_cluster":[5621,6115]},"id":6}]} 2024-09-21T07:37:09.316 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 66 2024-09-21T07:37:09.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.316+0000 7f56b3e00640 1 -- 172.21.15.39:0/3198488087 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5690078b90 msgr2=0x7f569007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:09.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.316+0000 7f56b3e00640 1 --2- 172.21.15.39:0/3198488087 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5690078b90 0x7f569007b0a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f56a4033000 tx=0x7f56a4004730 comp rx=0 tx=0).stop 2024-09-21T07:37:09.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.316+0000 7f56b3e00640 1 -- 172.21.15.39:0/3198488087 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56b40d2650 msgr2=0x7f56b40b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:09.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.316+0000 7f56b3e00640 1 --2- 172.21.15.39:0/3198488087 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56b40d2650 0x7f56b40b2080 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f56ac00b9e0 tx=0x7f56ac00bea0 comp rx=0 tx=0).stop 2024-09-21T07:37:09.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.316+0000 7f56b3e00640 1 -- 172.21.15.39:0/3198488087 shutdown_connections 2024-09-21T07:37:09.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.316+0000 7f56b3e00640 1 --2- 172.21.15.39:0/3198488087 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5690078b90 0x7f569007b0a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:09.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.316+0000 7f56b3e00640 1 --2- 172.21.15.39:0/3198488087 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56b40d2650 0x7f56b40b2080 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:09.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.316+0000 7f56b3e00640 1 --2- 172.21.15.39:0/3198488087 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56b40b9640 0x7f56b41e9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:09.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.316+0000 7f56b3e00640 1 --2- 172.21.15.39:0/3198488087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56b40b5470 0x7f56b414d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:09.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.316+0000 7f56b3e00640 1 -- 172.21.15.39:0/3198488087 >> 172.21.15.39:0/3198488087 conn(0x7f56b409d6a0 msgr2=0x7f56b40d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:09.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.316+0000 7f56b3e00640 1 -- 172.21.15.39:0/3198488087 shutdown_connections 2024-09-21T07:37:09.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.316+0000 7f56b3e00640 1 -- 172.21.15.39:0/3198488087 wait complete. 2024-09-21T07:37:09.340 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json dump snaps --server 2024-09-21T07:37:09.589 INFO:teuthology.orchestra.run.smithi148.stdout:{"last_snap":2,"last_created":2,"last_destroyed":1,"pending_noop":[],"snaps":[{"snapid":2,"ino":1099511627777,"stamp":"2024-09-21T07:36:43.100278+0000","name":"s11","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2024-09-21T07:37:09.590 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 2, "last_destroyed": 1, "last_snap": 2, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s11", "snapid": 2, "stamp": "2024-09-21T07:36:43.100278+0000" } ] } 2024-09-21T07:37:09.590 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T07:37:09.590 INFO:tasks.cephfs.test_snapshots:testing snapserver mds_kill_mdstable_at=2 2024-09-21T07:37:09.590 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:37:09.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.816+0000 7fdb85600640 1 -- 172.21.15.39:0/3104958321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb800706b0 msgr2=0x7fdb800cba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:09.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.816+0000 7fdb85600640 1 --2- 172.21.15.39:0/3104958321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb800706b0 0x7fdb800cba00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdb68009830 tx=0x7fdb6802a5c0 comp rx=0 tx=0).stop 2024-09-21T07:37:09.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.816+0000 7fdb85600640 1 -- 172.21.15.39:0/3104958321 shutdown_connections 2024-09-21T07:37:09.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.816+0000 7fdb85600640 1 --2- 172.21.15.39:0/3104958321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb800706b0 0x7fdb800cba00 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:09.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.816+0000 7fdb85600640 1 -- 172.21.15.39:0/3104958321 >> 172.21.15.39:0/3104958321 conn(0x7fdb800b1cb0 msgr2=0x7fdb800b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:09.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.816+0000 7fdb85600640 1 -- 172.21.15.39:0/3104958321 shutdown_connections 2024-09-21T07:37:09.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.816+0000 7fdb85600640 1 -- 172.21.15.39:0/3104958321 wait complete. 2024-09-21T07:37:09.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.819+0000 7fdb85600640 1 Processor -- start 2024-09-21T07:37:09.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.819+0000 7fdb85600640 1 -- start start 2024-09-21T07:37:09.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.820+0000 7fdb85600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb800ca800 0x7fdb801e8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:09.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.820+0000 7fdb85600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdb800cad30 0x7fdb800c6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:09.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.820+0000 7fdb85600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb800c6680 0x7fdb8015c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:09.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.820+0000 7fdb85600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdb80074130 con 0x7fdb800c6680 2024-09-21T07:37:09.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.820+0000 7fdb85600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdb80073fb0 con 0x7fdb800ca800 2024-09-21T07:37:09.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.820+0000 7fdb85600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fdb800742b0 con 0x7fdb800cad30 2024-09-21T07:37:09.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.821+0000 7fdb7f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb800c6680 0x7fdb8015c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:09.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.821+0000 7fdb7f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb800c6680 0x7fdb8015c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43478/0 (socket says 172.21.15.39:43478) 2024-09-21T07:37:09.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.821+0000 7fdb7f400640 1 -- 172.21.15.39:0/2640751757 learned_addr learned my addr 172.21.15.39:0/2640751757 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:09.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.821+0000 7fdb7ea00640 1 --2- 172.21.15.39:0/2640751757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb800ca800 0x7fdb801e8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:09.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.821+0000 7fdb77e00640 1 --2- 172.21.15.39:0/2640751757 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdb800cad30 0x7fdb800c6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:09.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.821+0000 7fdb7f400640 1 -- 172.21.15.39:0/2640751757 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdb800cad30 msgr2=0x7fdb800c6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:09.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.821+0000 7fdb7f400640 1 --2- 172.21.15.39:0/2640751757 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdb800cad30 0x7fdb800c6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:09.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.821+0000 7fdb7f400640 1 -- 172.21.15.39:0/2640751757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb800ca800 msgr2=0x7fdb801e8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:09.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.821+0000 7fdb7f400640 1 --2- 172.21.15.39:0/2640751757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb800ca800 0x7fdb801e8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:09.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.821+0000 7fdb7f400640 1 -- 172.21.15.39:0/2640751757 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb801eb8a0 con 0x7fdb800c6680 2024-09-21T07:37:09.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.821+0000 7fdb7f400640 1 --2- 172.21.15.39:0/2640751757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb800c6680 0x7fdb8015c8b0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fdb7800c730 tx=0x7fdb7800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:09.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.822+0000 7fdb76a00640 1 -- 172.21.15.39:0/2640751757 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb78003b70 con 0x7fdb800c6680 2024-09-21T07:37:09.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.822+0000 7fdb76a00640 1 -- 172.21.15.39:0/2640751757 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb78003d10 con 0x7fdb800c6680 2024-09-21T07:37:09.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.822+0000 7fdb76a00640 1 -- 172.21.15.39:0/2640751757 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7800aab0 con 0x7fdb800c6680 2024-09-21T07:37:09.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.822+0000 7fdb85600640 1 -- 172.21.15.39:0/2640751757 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb8014ae90 con 0x7fdb800c6680 2024-09-21T07:37:09.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.822+0000 7fdb85600640 1 -- 172.21.15.39:0/2640751757 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb8014b450 con 0x7fdb800c6680 2024-09-21T07:37:09.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.823+0000 7fdb85600640 1 -- 172.21.15.39:0/2640751757 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb800747d0 con 0x7fdb800c6680 2024-09-21T07:37:09.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.825+0000 7fdb76a00640 1 -- 172.21.15.39:0/2640751757 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdb7800ac50 con 0x7fdb800c6680 2024-09-21T07:37:09.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.826+0000 7fdb76a00640 1 --2- 172.21.15.39:0/2640751757 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdb5c078bc0 0x7fdb5c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:09.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.826+0000 7fdb76a00640 1 -- 172.21.15.39:0/2640751757 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6915+0+0 (secure 0 0 0) 0x7fdb7809ae90 con 0x7fdb800c6680 2024-09-21T07:37:09.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.827+0000 7fdb7ea00640 1 --2- 172.21.15.39:0/2640751757 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdb5c078bc0 0x7fdb5c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:09.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.827+0000 7fdb7ea00640 1 --2- 172.21.15.39:0/2640751757 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdb5c078bc0 0x7fdb5c07b0d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdb700046b0 tx=0x7fdb70007cc0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:09.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:09.831+0000 7fdb76a00640 1 -- 172.21.15.39:0/2640751757 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdb780627b0 con 0x7fdb800c6680 2024-09-21T07:37:10.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.065+0000 7fdb85600640 1 -- 172.21.15.39:0/2640751757 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb801eb630 con 0x7fdb800c6680 2024-09-21T07:37:10.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.065+0000 7fdb76a00640 1 -- 172.21.15.39:0/2640751757 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5249 (secure 0 0 0) 0x7fdb780671e0 con 0x7fdb800c6680 2024-09-21T07:37:10.067 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:37:10.068 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":66,"btime":"2024-09-21T07:37:05:276431+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6585,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/1472430787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1472430787},{"type":"v1","addr":"172.21.15.39:6834","nonce":1472430787}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:37:05.276415+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5621,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5621":{"gid":5621,"name":"d","rank":0,"incarnation":62,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3026827614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3026827614},{"type":"v1","addr":"172.21.15.148:6837","nonce":3026827614}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5621,"qdb_cluster":[5621,6115]},"id":6}]} 2024-09-21T07:37:10.068 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 66 2024-09-21T07:37:10.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.068+0000 7fdb85600640 1 -- 172.21.15.39:0/2640751757 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdb5c078bc0 msgr2=0x7fdb5c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:10.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.068+0000 7fdb85600640 1 --2- 172.21.15.39:0/2640751757 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdb5c078bc0 0x7fdb5c07b0d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdb700046b0 tx=0x7fdb70007cc0 comp rx=0 tx=0).stop 2024-09-21T07:37:10.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.068+0000 7fdb85600640 1 -- 172.21.15.39:0/2640751757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb800c6680 msgr2=0x7fdb8015c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:10.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.068+0000 7fdb85600640 1 --2- 172.21.15.39:0/2640751757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb800c6680 0x7fdb8015c8b0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fdb7800c730 tx=0x7fdb7800cbf0 comp rx=0 tx=0).stop 2024-09-21T07:37:10.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.068+0000 7fdb85600640 1 -- 172.21.15.39:0/2640751757 shutdown_connections 2024-09-21T07:37:10.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.068+0000 7fdb85600640 1 --2- 172.21.15.39:0/2640751757 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdb5c078bc0 0x7fdb5c07b0d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:10.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.068+0000 7fdb85600640 1 --2- 172.21.15.39:0/2640751757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb800c6680 0x7fdb8015c8b0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:10.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.068+0000 7fdb85600640 1 --2- 172.21.15.39:0/2640751757 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdb800cad30 0x7fdb800c6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:10.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.068+0000 7fdb85600640 1 --2- 172.21.15.39:0/2640751757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb800ca800 0x7fdb801e8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:10.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.068+0000 7fdb85600640 1 -- 172.21.15.39:0/2640751757 >> 172.21.15.39:0/2640751757 conn(0x7fdb800b1cb0 msgr2=0x7fdb800b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:10.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.068+0000 7fdb85600640 1 -- 172.21.15.39:0/2640751757 shutdown_connections 2024-09-21T07:37:10.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.068+0000 7fdb85600640 1 -- 172.21.15.39:0/2640751757 wait complete. 2024-09-21T07:37:10.091 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:0 true 2024-09-21T07:37:10.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.319+0000 7ff879e00640 1 -- 172.21.15.39:0/4189331541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff874073170 msgr2=0x7ff87415c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:10.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.319+0000 7ff879e00640 1 --2- 172.21.15.39:0/4189331541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff874073170 0x7ff87415c780 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7ff8740b4b40 tx=0x7ff86402a4d0 comp rx=0 tx=0).stop 2024-09-21T07:37:10.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.320+0000 7ff879e00640 1 -- 172.21.15.39:0/4189331541 shutdown_connections 2024-09-21T07:37:10.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.320+0000 7ff879e00640 1 --2- 172.21.15.39:0/4189331541 >> v2:172.21.15.148:3300/0 conn(0x7ff874070440 0x7ff8740b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:10.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.320+0000 7ff879e00640 1 --2- 172.21.15.39:0/4189331541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff874073170 0x7ff87415c780 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:10.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.320+0000 7ff879e00640 1 -- 172.21.15.39:0/4189331541 >> 172.21.15.39:0/4189331541 conn(0x7ff8740b1cb0 msgr2=0x7ff8740b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:10.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.320+0000 7ff879e00640 1 -- 172.21.15.39:0/4189331541 shutdown_connections 2024-09-21T07:37:10.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.320+0000 7ff879e00640 1 -- 172.21.15.39:0/4189331541 wait complete. 2024-09-21T07:37:10.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.322+0000 7ff879e00640 1 Processor -- start 2024-09-21T07:37:10.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.322+0000 7ff879e00640 1 -- start start 2024-09-21T07:37:10.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.323+0000 7ff879e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff874070030 0x7ff874154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:10.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.323+0000 7ff879e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff874070440 0x7ff87409c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:10.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.323+0000 7ff879e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff874073170 0x7ff87409ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:10.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.323+0000 7ff879e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff8740b3a40 con 0x7ff874070440 2024-09-21T07:37:10.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.323+0000 7ff879e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff8740b38c0 con 0x7ff874070030 2024-09-21T07:37:10.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.323+0000 7ff879e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7ff8740b3bc0 con 0x7ff874073170 2024-09-21T07:37:10.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.324+0000 7ff872000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff874070440 0x7ff87409c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:10.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.324+0000 7ff872000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff874070440 0x7ff87409c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43492/0 (socket says 172.21.15.39:43492) 2024-09-21T07:37:10.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.324+0000 7ff872000640 1 -- 172.21.15.39:0/3713933172 learned_addr learned my addr 172.21.15.39:0/3713933172 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:10.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.324+0000 7ff872a00640 1 --2- 172.21.15.39:0/3713933172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff874070030 0x7ff874154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:10.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.324+0000 7ff873400640 1 --2- 172.21.15.39:0/3713933172 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff874073170 0x7ff87409ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:10.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.324+0000 7ff872000640 1 -- 172.21.15.39:0/3713933172 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff874073170 msgr2=0x7ff87409ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:10.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.324+0000 7ff872000640 1 --2- 172.21.15.39:0/3713933172 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff874073170 0x7ff87409ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:10.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.324+0000 7ff872000640 1 -- 172.21.15.39:0/3713933172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff874070030 msgr2=0x7ff874154bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:10.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.324+0000 7ff872000640 1 --2- 172.21.15.39:0/3713933172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff874070030 0x7ff874154bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:10.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.324+0000 7ff872000640 1 -- 172.21.15.39:0/3713933172 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8740cb3e0 con 0x7ff874070440 2024-09-21T07:37:10.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.324+0000 7ff872a00640 1 --2- 172.21.15.39:0/3713933172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff874070030 0x7ff874154bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:37:10.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.324+0000 7ff872000640 1 --2- 172.21.15.39:0/3713933172 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff874070440 0x7ff87409c1e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7ff86000b3f0 tx=0x7ff86000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:10.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.325+0000 7ff86b400640 1 -- 172.21.15.39:0/3713933172 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff860015020 con 0x7ff874070440 2024-09-21T07:37:10.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.325+0000 7ff86b400640 1 -- 172.21.15.39:0/3713933172 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff860011920 con 0x7ff874070440 2024-09-21T07:37:10.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.325+0000 7ff86b400640 1 -- 172.21.15.39:0/3713933172 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff860011d70 con 0x7ff874070440 2024-09-21T07:37:10.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.325+0000 7ff879e00640 1 -- 172.21.15.39:0/3713933172 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff87409d840 con 0x7ff874070440 2024-09-21T07:37:10.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.325+0000 7ff879e00640 1 -- 172.21.15.39:0/3713933172 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8740cbbc0 con 0x7ff874070440 2024-09-21T07:37:10.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.326+0000 7ff879e00640 1 -- 172.21.15.39:0/3713933172 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8740b4100 con 0x7ff874070440 2024-09-21T07:37:10.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.327+0000 7ff86b400640 1 -- 172.21.15.39:0/3713933172 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff860018020 con 0x7ff874070440 2024-09-21T07:37:10.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.328+0000 7ff86b400640 1 --2- 172.21.15.39:0/3713933172 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff854078b90 0x7ff85407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:10.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.328+0000 7ff86b400640 1 -- 172.21.15.39:0/3713933172 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6915+0+0 (secure 0 0 0) 0x7ff86009af30 con 0x7ff874070440 2024-09-21T07:37:10.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.329+0000 7ff872a00640 1 --2- 172.21.15.39:0/3713933172 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff854078b90 0x7ff85407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:10.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.330+0000 7ff872a00640 1 --2- 172.21.15.39:0/3713933172 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff854078b90 0x7ff85407b0a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff864033000 tx=0x7ff864004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:10.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.333+0000 7ff86b400640 1 -- 172.21.15.39:0/3713933172 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff86001f030 con 0x7ff874070440 2024-09-21T07:37:10.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:10.605+0000 7ff879e00640 1 -- 172.21.15.39:0/3713933172 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"} v 0) -- 0x7ff8740c3860 con 0x7ff874070440 2024-09-21T07:37:11.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:11.424+0000 7ff86b400640 1 -- 172.21.15.39:0/3713933172 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"}]=0 freezing mds.5621 v67) ==== 112+0+0 (secure 0 0 0) 0x7ff860062ea0 con 0x7ff874070440 2024-09-21T07:37:11.427 INFO:teuthology.orchestra.run.smithi039.stderr:freezing mds.5621 2024-09-21T07:37:11.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:11.427+0000 7ff879e00640 1 -- 172.21.15.39:0/3713933172 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff854078b90 msgr2=0x7ff85407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:11.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:11.427+0000 7ff879e00640 1 --2- 172.21.15.39:0/3713933172 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff854078b90 0x7ff85407b0a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff864033000 tx=0x7ff864004730 comp rx=0 tx=0).stop 2024-09-21T07:37:11.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:11.427+0000 7ff879e00640 1 -- 172.21.15.39:0/3713933172 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff874070440 msgr2=0x7ff87409c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:11.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:11.427+0000 7ff879e00640 1 --2- 172.21.15.39:0/3713933172 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff874070440 0x7ff87409c1e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7ff86000b3f0 tx=0x7ff86000b8b0 comp rx=0 tx=0).stop 2024-09-21T07:37:11.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:11.427+0000 7ff879e00640 1 -- 172.21.15.39:0/3713933172 shutdown_connections 2024-09-21T07:37:11.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:11.427+0000 7ff879e00640 1 --2- 172.21.15.39:0/3713933172 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff854078b90 0x7ff85407b0a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:11.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:11.427+0000 7ff879e00640 1 --2- 172.21.15.39:0/3713933172 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff874073170 0x7ff87409ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:11.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:11.427+0000 7ff879e00640 1 --2- 172.21.15.39:0/3713933172 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff874070440 0x7ff87409c1e0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:11.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:11.427+0000 7ff879e00640 1 --2- 172.21.15.39:0/3713933172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff874070030 0x7ff874154bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:11.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:11.427+0000 7ff879e00640 1 -- 172.21.15.39:0/3713933172 >> 172.21.15.39:0/3713933172 conn(0x7ff8740b1cb0 msgr2=0x7ff8740adc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:11.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:11.428+0000 7ff879e00640 1 -- 172.21.15.39:0/3713933172 shutdown_connections 2024-09-21T07:37:11.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:11.428+0000 7ff879e00640 1 -- 172.21.15.39:0/3713933172 wait complete. 2024-09-21T07:37:11.458 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json config set mds_kill_mdstable_at 2 2024-09-21T07:37:11.703 INFO:teuthology.orchestra.run.smithi148.stdout:{"success":"mds_kill_mdstable_at = '' (not observed, change may require restart) "} 2024-09-21T07:37:11.703 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_kill_mdstable_at = '' (not observed, change may require restart) " } 2024-09-21T07:37:11.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:37:11.704 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d1/dir/.snap/s12) 2024-09-21T07:37:11.708 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-09-21T07:37:11.745 INFO:tasks.ceph.mds.d.smithi148.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_prepare_logged(ceph::cref_t&, version_t)' thread 7fb40e600640 time 2024-09-21T07:37:11.745042+0000 2024-09-21T07:37:11.746 INFO:tasks.ceph.mds.d.smithi148.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: 77: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 2) 2024-09-21T07:37:11.746 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:37:11.746 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fb41d85654b] 2024-09-21T07:37:11.746 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7fb41d856758] 2024-09-21T07:37:11.746 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x396) [0x55c6f9fe27e4] 2024-09-21T07:37:11.747 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: ceph-mds(+0x47ac3e) [0x55c6f9fe2c3e] 2024-09-21T07:37:11.747 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (MDSContext::complete(int)+0x7c) [0x55c6fa02f6d8] 2024-09-21T07:37:11.747 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: (MDSIOContextBase::complete(int)+0x501) [0x55c6fa02fd57] 2024-09-21T07:37:11.747 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSLogContextBase::complete(int)+0x46) [0x55c6fa02fdd4] 2024-09-21T07:37:11.747 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: (Finisher::finisher_thread_entry()+0x495) [0x7fb41d7de34b] 2024-09-21T07:37:11.747 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: ceph-mds(+0x1052e1) [0x55c6f9c6d2e1] 2024-09-21T07:37:11.747 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: (Thread::entry_wrapper()+0x43) [0x7fb41d82fd57] 2024-09-21T07:37:11.747 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (Thread::_entry_func(void*)+0xd) [0x7fb41d82fd73] 2024-09-21T07:37:11.747 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: /lib64/libc.so.6(+0x89c52) [0x7fb41ce89c52] 2024-09-21T07:37:11.747 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: /lib64/libc.so.6(+0x10ec80) [0x7fb41cf0ec80] 2024-09-21T07:37:11.747 INFO:tasks.ceph.mds.d.smithi148.stderr:2024-09-21T07:37:11.745+0000 7fb40e600640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_prepare_logged(ceph::cref_t&, version_t)' thread 7fb40e600640 time 2024-09-21T07:37:11.745042+0000 2024-09-21T07:37:11.747 INFO:tasks.ceph.mds.d.smithi148.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: 77: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 2) 2024-09-21T07:37:11.747 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:37:11.748 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:37:11.748 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fb41d85654b] 2024-09-21T07:37:11.748 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7fb41d856758] 2024-09-21T07:37:11.748 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x396) [0x55c6f9fe27e4] 2024-09-21T07:37:11.748 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: ceph-mds(+0x47ac3e) [0x55c6f9fe2c3e] 2024-09-21T07:37:11.748 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (MDSContext::complete(int)+0x7c) [0x55c6fa02f6d8] 2024-09-21T07:37:11.748 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: (MDSIOContextBase::complete(int)+0x501) [0x55c6fa02fd57] 2024-09-21T07:37:11.748 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSLogContextBase::complete(int)+0x46) [0x55c6fa02fdd4] 2024-09-21T07:37:11.748 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: (Finisher::finisher_thread_entry()+0x495) [0x7fb41d7de34b] 2024-09-21T07:37:11.748 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: ceph-mds(+0x1052e1) [0x55c6f9c6d2e1] 2024-09-21T07:37:11.748 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: (Thread::entry_wrapper()+0x43) [0x7fb41d82fd57] 2024-09-21T07:37:11.748 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (Thread::_entry_func(void*)+0xd) [0x7fb41d82fd73] 2024-09-21T07:37:11.748 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: /lib64/libc.so.6(+0x89c52) [0x7fb41ce89c52] 2024-09-21T07:37:11.748 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: /lib64/libc.so.6(+0x10ec80) [0x7fb41cf0ec80] 2024-09-21T07:37:11.748 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:37:11.749 INFO:tasks.ceph.mds.d.smithi148.stderr:*** Caught signal (Aborted) ** 2024-09-21T07:37:11.749 INFO:tasks.ceph.mds.d.smithi148.stderr: in thread 7fb40e600640 thread_name:MR_Finisher 2024-09-21T07:37:11.749 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:37:11.749 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fb41ce3e6f0] 2024-09-21T07:37:11.749 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7fb41ce8b99c] 2024-09-21T07:37:11.749 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: raise() 2024-09-21T07:37:11.749 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: abort() 2024-09-21T07:37:11.749 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7fb41d856686] 2024-09-21T07:37:11.750 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7fb41d856758] 2024-09-21T07:37:11.750 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x396) [0x55c6f9fe27e4] 2024-09-21T07:37:11.750 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: ceph-mds(+0x47ac3e) [0x55c6f9fe2c3e] 2024-09-21T07:37:11.750 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: (MDSContext::complete(int)+0x7c) [0x55c6fa02f6d8] 2024-09-21T07:37:11.750 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: (MDSIOContextBase::complete(int)+0x501) [0x55c6fa02fd57] 2024-09-21T07:37:11.750 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (MDSLogContextBase::complete(int)+0x46) [0x55c6fa02fdd4] 2024-09-21T07:37:11.750 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: (Finisher::finisher_thread_entry()+0x495) [0x7fb41d7de34b] 2024-09-21T07:37:11.750 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: ceph-mds(+0x1052e1) [0x55c6f9c6d2e1] 2024-09-21T07:37:11.750 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: (Thread::entry_wrapper()+0x43) [0x7fb41d82fd57] 2024-09-21T07:37:11.750 INFO:tasks.ceph.mds.d.smithi148.stderr: 15: (Thread::_entry_func(void*)+0xd) [0x7fb41d82fd73] 2024-09-21T07:37:11.750 INFO:tasks.ceph.mds.d.smithi148.stderr: 16: /lib64/libc.so.6(+0x89c52) [0x7fb41ce89c52] 2024-09-21T07:37:11.750 INFO:tasks.ceph.mds.d.smithi148.stderr: 17: /lib64/libc.so.6(+0x10ec80) [0x7fb41cf0ec80] 2024-09-21T07:37:11.750 INFO:tasks.ceph.mds.d.smithi148.stderr:2024-09-21T07:37:11.746+0000 7fb40e600640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:37:11.750 INFO:tasks.ceph.mds.d.smithi148.stderr: in thread 7fb40e600640 thread_name:MR_Finisher 2024-09-21T07:37:11.751 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:37:11.751 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:37:11.751 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fb41ce3e6f0] 2024-09-21T07:37:11.751 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7fb41ce8b99c] 2024-09-21T07:37:11.751 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: raise() 2024-09-21T07:37:11.751 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: abort() 2024-09-21T07:37:11.751 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7fb41d856686] 2024-09-21T07:37:11.751 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7fb41d856758] 2024-09-21T07:37:11.751 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x396) [0x55c6f9fe27e4] 2024-09-21T07:37:11.751 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: ceph-mds(+0x47ac3e) [0x55c6f9fe2c3e] 2024-09-21T07:37:11.751 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: (MDSContext::complete(int)+0x7c) [0x55c6fa02f6d8] 2024-09-21T07:37:11.751 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: (MDSIOContextBase::complete(int)+0x501) [0x55c6fa02fd57] 2024-09-21T07:37:11.751 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (MDSLogContextBase::complete(int)+0x46) [0x55c6fa02fdd4] 2024-09-21T07:37:11.751 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: (Finisher::finisher_thread_entry()+0x495) [0x7fb41d7de34b] 2024-09-21T07:37:11.752 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: ceph-mds(+0x1052e1) [0x55c6f9c6d2e1] 2024-09-21T07:37:11.752 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: (Thread::entry_wrapper()+0x43) [0x7fb41d82fd57] 2024-09-21T07:37:11.752 INFO:tasks.ceph.mds.d.smithi148.stderr: 15: (Thread::_entry_func(void*)+0xd) [0x7fb41d82fd73] 2024-09-21T07:37:11.752 INFO:tasks.ceph.mds.d.smithi148.stderr: 16: /lib64/libc.so.6(+0x89c52) [0x7fb41ce89c52] 2024-09-21T07:37:11.752 INFO:tasks.ceph.mds.d.smithi148.stderr: 17: /lib64/libc.so.6(+0x10ec80) [0x7fb41cf0ec80] 2024-09-21T07:37:11.752 INFO:tasks.ceph.mds.d.smithi148.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:37:11.752 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:37:11.755 INFO:tasks.ceph.mds.d.smithi148.stderr: -559> 2024-09-21T07:37:07.644+0000 7fb413600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:37:11.755 INFO:tasks.ceph.mds.d.smithi148.stderr: -557> 2024-09-21T07:37:07.644+0000 7fb413600640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:37:11.756 INFO:tasks.ceph.mds.d.smithi148.stderr: -1> 2024-09-21T07:37:11.745+0000 7fb40e600640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_prepare_logged(ceph::cref_t&, version_t)' thread 7fb40e600640 time 2024-09-21T07:37:11.745042+0000 2024-09-21T07:37:11.757 INFO:tasks.ceph.mds.d.smithi148.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: 77: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 2) 2024-09-21T07:37:11.757 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:37:11.757 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:37:11.757 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fb41d85654b] 2024-09-21T07:37:11.757 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7fb41d856758] 2024-09-21T07:37:11.757 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x396) [0x55c6f9fe27e4] 2024-09-21T07:37:11.757 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: ceph-mds(+0x47ac3e) [0x55c6f9fe2c3e] 2024-09-21T07:37:11.757 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (MDSContext::complete(int)+0x7c) [0x55c6fa02f6d8] 2024-09-21T07:37:11.757 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: (MDSIOContextBase::complete(int)+0x501) [0x55c6fa02fd57] 2024-09-21T07:37:11.757 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSLogContextBase::complete(int)+0x46) [0x55c6fa02fdd4] 2024-09-21T07:37:11.757 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: (Finisher::finisher_thread_entry()+0x495) [0x7fb41d7de34b] 2024-09-21T07:37:11.757 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: ceph-mds(+0x1052e1) [0x55c6f9c6d2e1] 2024-09-21T07:37:11.757 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: (Thread::entry_wrapper()+0x43) [0x7fb41d82fd57] 2024-09-21T07:37:11.758 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (Thread::_entry_func(void*)+0xd) [0x7fb41d82fd73] 2024-09-21T07:37:11.758 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: /lib64/libc.so.6(+0x89c52) [0x7fb41ce89c52] 2024-09-21T07:37:11.758 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: /lib64/libc.so.6(+0x10ec80) [0x7fb41cf0ec80] 2024-09-21T07:37:11.758 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:37:11.758 INFO:tasks.ceph.mds.d.smithi148.stderr: 0> 2024-09-21T07:37:11.746+0000 7fb40e600640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:37:11.758 INFO:tasks.ceph.mds.d.smithi148.stderr: in thread 7fb40e600640 thread_name:MR_Finisher 2024-09-21T07:37:11.758 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:37:11.758 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:37:11.758 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fb41ce3e6f0] 2024-09-21T07:37:11.758 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7fb41ce8b99c] 2024-09-21T07:37:11.758 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: raise() 2024-09-21T07:37:11.758 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: abort() 2024-09-21T07:37:11.758 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7fb41d856686] 2024-09-21T07:37:11.759 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7fb41d856758] 2024-09-21T07:37:11.759 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x396) [0x55c6f9fe27e4] 2024-09-21T07:37:11.759 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: ceph-mds(+0x47ac3e) [0x55c6f9fe2c3e] 2024-09-21T07:37:11.759 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: (MDSContext::complete(int)+0x7c) [0x55c6fa02f6d8] 2024-09-21T07:37:11.759 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: (MDSIOContextBase::complete(int)+0x501) [0x55c6fa02fd57] 2024-09-21T07:37:11.759 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (MDSLogContextBase::complete(int)+0x46) [0x55c6fa02fdd4] 2024-09-21T07:37:11.759 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: (Finisher::finisher_thread_entry()+0x495) [0x7fb41d7de34b] 2024-09-21T07:37:11.759 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: ceph-mds(+0x1052e1) [0x55c6f9c6d2e1] 2024-09-21T07:37:11.759 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: (Thread::entry_wrapper()+0x43) [0x7fb41d82fd57] 2024-09-21T07:37:11.759 INFO:tasks.ceph.mds.d.smithi148.stderr: 15: (Thread::_entry_func(void*)+0xd) [0x7fb41d82fd73] 2024-09-21T07:37:11.759 INFO:tasks.ceph.mds.d.smithi148.stderr: 16: /lib64/libc.so.6(+0x89c52) [0x7fb41ce89c52] 2024-09-21T07:37:11.759 INFO:tasks.ceph.mds.d.smithi148.stderr: 17: /lib64/libc.so.6(+0x10ec80) [0x7fb41cf0ec80] 2024-09-21T07:37:11.759 INFO:tasks.ceph.mds.d.smithi148.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:37:11.760 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:37:11.763 INFO:tasks.ceph.mds.d.smithi148.stderr: -559> 2024-09-21T07:37:07.644+0000 7fb413600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:37:11.763 INFO:tasks.ceph.mds.d.smithi148.stderr: -557> 2024-09-21T07:37:07.644+0000 7fb413600640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:37:11.764 INFO:tasks.ceph.mds.d.smithi148.stderr: -1> 2024-09-21T07:37:11.745+0000 7fb40e600640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_prepare_logged(ceph::cref_t&, version_t)' thread 7fb40e600640 time 2024-09-21T07:37:11.745042+0000 2024-09-21T07:37:11.764 INFO:tasks.ceph.mds.d.smithi148.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: 77: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 2) 2024-09-21T07:37:11.764 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:37:11.764 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:37:11.764 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fb41d85654b] 2024-09-21T07:37:11.764 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7fb41d856758] 2024-09-21T07:37:11.764 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x396) [0x55c6f9fe27e4] 2024-09-21T07:37:11.764 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: ceph-mds(+0x47ac3e) [0x55c6f9fe2c3e] 2024-09-21T07:37:11.764 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (MDSContext::complete(int)+0x7c) [0x55c6fa02f6d8] 2024-09-21T07:37:11.764 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: (MDSIOContextBase::complete(int)+0x501) [0x55c6fa02fd57] 2024-09-21T07:37:11.764 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSLogContextBase::complete(int)+0x46) [0x55c6fa02fdd4] 2024-09-21T07:37:11.765 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: (Finisher::finisher_thread_entry()+0x495) [0x7fb41d7de34b] 2024-09-21T07:37:11.765 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: ceph-mds(+0x1052e1) [0x55c6f9c6d2e1] 2024-09-21T07:37:11.765 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: (Thread::entry_wrapper()+0x43) [0x7fb41d82fd57] 2024-09-21T07:37:11.765 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (Thread::_entry_func(void*)+0xd) [0x7fb41d82fd73] 2024-09-21T07:37:11.765 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: /lib64/libc.so.6(+0x89c52) [0x7fb41ce89c52] 2024-09-21T07:37:11.765 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: /lib64/libc.so.6(+0x10ec80) [0x7fb41cf0ec80] 2024-09-21T07:37:11.765 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:37:11.765 INFO:tasks.ceph.mds.d.smithi148.stderr: 0> 2024-09-21T07:37:11.746+0000 7fb40e600640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:37:11.765 INFO:tasks.ceph.mds.d.smithi148.stderr: in thread 7fb40e600640 thread_name:MR_Finisher 2024-09-21T07:37:11.765 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:37:11.765 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:37:11.765 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fb41ce3e6f0] 2024-09-21T07:37:11.766 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7fb41ce8b99c] 2024-09-21T07:37:11.766 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: raise() 2024-09-21T07:37:11.766 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: abort() 2024-09-21T07:37:11.766 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7fb41d856686] 2024-09-21T07:37:11.766 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7fb41d856758] 2024-09-21T07:37:11.766 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x396) [0x55c6f9fe27e4] 2024-09-21T07:37:11.766 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: ceph-mds(+0x47ac3e) [0x55c6f9fe2c3e] 2024-09-21T07:37:11.766 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: (MDSContext::complete(int)+0x7c) [0x55c6fa02f6d8] 2024-09-21T07:37:11.766 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: (MDSIOContextBase::complete(int)+0x501) [0x55c6fa02fd57] 2024-09-21T07:37:11.766 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (MDSLogContextBase::complete(int)+0x46) [0x55c6fa02fdd4] 2024-09-21T07:37:11.766 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: (Finisher::finisher_thread_entry()+0x495) [0x7fb41d7de34b] 2024-09-21T07:37:11.766 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: ceph-mds(+0x1052e1) [0x55c6f9c6d2e1] 2024-09-21T07:37:11.766 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: (Thread::entry_wrapper()+0x43) [0x7fb41d82fd57] 2024-09-21T07:37:11.767 INFO:tasks.ceph.mds.d.smithi148.stderr: 15: (Thread::_entry_func(void*)+0xd) [0x7fb41d82fd73] 2024-09-21T07:37:11.767 INFO:tasks.ceph.mds.d.smithi148.stderr: 16: /lib64/libc.so.6(+0x89c52) [0x7fb41ce89c52] 2024-09-21T07:37:11.767 INFO:tasks.ceph.mds.d.smithi148.stderr: 17: /lib64/libc.so.6(+0x10ec80) [0x7fb41cf0ec80] 2024-09-21T07:37:11.767 INFO:tasks.ceph.mds.d.smithi148.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:37:11.767 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:37:11.986 INFO:teuthology.orchestra.run.smithi039.stdout:{"mds_beacon_grace":"15.000000"} 2024-09-21T07:37:11.986 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-09-21T07:37:11.986 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:37:12.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.214+0000 7fa825800640 1 -- 172.21.15.39:0/1644285659 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa804003c10 msgr2=0x7fa804004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:12.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.214+0000 7fa825800640 1 --2- 172.21.15.39:0/1644285659 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa804003c10 0x7fa804004060 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa810009b50 tx=0x7fa804005830 comp rx=0 tx=0).stop 2024-09-21T07:37:12.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.214+0000 7fa824e00640 1 -- 172.21.15.39:0/1644285659 reap_dead start 2024-09-21T07:37:12.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.214+0000 7fa825800640 1 -- 172.21.15.39:0/1644285659 shutdown_connections 2024-09-21T07:37:12.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.214+0000 7fa825800640 1 -- 172.21.15.39:0/1644285659 >> 172.21.15.39:0/1644285659 conn(0x7fa8200b3090 msgr2=0x7fa8200b34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:12.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.215+0000 7fa825800640 1 -- 172.21.15.39:0/1644285659 shutdown_connections 2024-09-21T07:37:12.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.215+0000 7fa825800640 1 -- 172.21.15.39:0/1644285659 wait complete. 2024-09-21T07:37:12.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.217+0000 7fa825800640 1 Processor -- start 2024-09-21T07:37:12.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.217+0000 7fa825800640 1 -- start start 2024-09-21T07:37:12.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.218+0000 7fa825800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa804003c10 0x7fa8201449c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:12.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.218+0000 7fa825800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa820071310 0x7fa8200c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:12.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.218+0000 7fa825800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8200c8ac0 0x7fa8201f6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:12.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.218+0000 7fa825800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa8200cda60 con 0x7fa820071310 2024-09-21T07:37:12.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.218+0000 7fa825800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa8200cd8e0 con 0x7fa804003c10 2024-09-21T07:37:12.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.218+0000 7fa825800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa8200cdbe0 con 0x7fa8200c8ac0 2024-09-21T07:37:12.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.219+0000 7fa817400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa820071310 0x7fa8200c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:12.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.219+0000 7fa817400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa820071310 0x7fa8200c8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42518/0 (socket says 172.21.15.39:42518) 2024-09-21T07:37:12.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.219+0000 7fa817400640 1 -- 172.21.15.39:0/3373467099 learned_addr learned my addr 172.21.15.39:0/3373467099 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:12.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.219+0000 7fa81fe00640 1 --2- 172.21.15.39:0/3373467099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa804003c10 0x7fa8201449c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:12.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.219+0000 7fa824e00640 1 --2- 172.21.15.39:0/3373467099 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8200c8ac0 0x7fa8201f6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:12.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.219+0000 7fa817400640 1 -- 172.21.15.39:0/3373467099 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8200c8ac0 msgr2=0x7fa8201f6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:12.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.219+0000 7fa817400640 1 --2- 172.21.15.39:0/3373467099 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8200c8ac0 0x7fa8201f6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:12.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.219+0000 7fa817400640 1 -- 172.21.15.39:0/3373467099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa804003c10 msgr2=0x7fa8201449c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:12.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.219+0000 7fa817400640 1 --2- 172.21.15.39:0/3373467099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa804003c10 0x7fa8201449c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:12.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.219+0000 7fa817400640 1 -- 172.21.15.39:0/3373467099 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8200c8ed0 con 0x7fa820071310 2024-09-21T07:37:12.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.219+0000 7fa817400640 1 --2- 172.21.15.39:0/3373467099 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa820071310 0x7fa8200c8580 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fa808006fd0 tx=0x7fa808001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:12.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.220+0000 7fa81cc00640 1 -- 172.21.15.39:0/3373467099 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa808011640 con 0x7fa820071310 2024-09-21T07:37:12.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.220+0000 7fa81cc00640 1 -- 172.21.15.39:0/3373467099 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa808016020 con 0x7fa820071310 2024-09-21T07:37:12.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.220+0000 7fa825800640 1 -- 172.21.15.39:0/3373467099 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8200c62d0 con 0x7fa820071310 2024-09-21T07:37:12.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.220+0000 7fa81cc00640 1 -- 172.21.15.39:0/3373467099 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa808011a80 con 0x7fa820071310 2024-09-21T07:37:12.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.220+0000 7fa825800640 1 -- 172.21.15.39:0/3373467099 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8200c65b0 con 0x7fa820071310 2024-09-21T07:37:12.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.221+0000 7fa825800640 1 -- 172.21.15.39:0/3373467099 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8200ce140 con 0x7fa820071310 2024-09-21T07:37:12.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.222+0000 7fa81cc00640 1 -- 172.21.15.39:0/3373467099 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa808011c20 con 0x7fa820071310 2024-09-21T07:37:12.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.224+0000 7fa81cc00640 1 --2- 172.21.15.39:0/3373467099 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa7fc078b90 0x7fa7fc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:12.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.224+0000 7fa81cc00640 1 -- 172.21.15.39:0/3373467099 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6915+0+0 (secure 0 0 0) 0x7fa80809bcf0 con 0x7fa820071310 2024-09-21T07:37:12.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.224+0000 7fa81fe00640 1 --2- 172.21.15.39:0/3373467099 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa7fc078b90 0x7fa7fc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:12.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.225+0000 7fa81fe00640 1 --2- 172.21.15.39:0/3373467099 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa7fc078b90 0x7fa7fc07b0a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa810035000 tx=0x7fa81002d040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:12.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.229+0000 7fa81cc00640 1 -- 172.21.15.39:0/3373467099 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa8080a0070 con 0x7fa820071310 2024-09-21T07:37:12.465 INFO:tasks.ceph.mds.d.smithi148.stderr:daemon-helper: command crashed with signal 6 2024-09-21T07:37:12.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.466+0000 7fa825800640 1 -- 172.21.15.39:0/3373467099 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8200ca520 con 0x7fa820071310 2024-09-21T07:37:12.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.467+0000 7fa81cc00640 1 -- 172.21.15.39:0/3373467099 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5249 (secure 0 0 0) 0x7fa808020020 con 0x7fa820071310 2024-09-21T07:37:12.469 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:37:12.469 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":67,"btime":"2024-09-21T07:37:11:423990+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6585,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/1472430787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1472430787},{"type":"v1","addr":"172.21.15.39:6834","nonce":1472430787}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:37:10.607422+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5621,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5621":{"gid":5621,"name":"d","rank":0,"incarnation":62,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3026827614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3026827614},{"type":"v1","addr":"172.21.15.148:6837","nonce":3026827614}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5621,"qdb_cluster":[6115,5621]},"id":6}]} 2024-09-21T07:37:12.469 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 67 2024-09-21T07:37:12.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.469+0000 7fa825800640 1 -- 172.21.15.39:0/3373467099 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa7fc078b90 msgr2=0x7fa7fc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:12.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.469+0000 7fa825800640 1 --2- 172.21.15.39:0/3373467099 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa7fc078b90 0x7fa7fc07b0a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa810035000 tx=0x7fa81002d040 comp rx=0 tx=0).stop 2024-09-21T07:37:12.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.469+0000 7fa825800640 1 -- 172.21.15.39:0/3373467099 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa820071310 msgr2=0x7fa8200c8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:12.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.469+0000 7fa825800640 1 --2- 172.21.15.39:0/3373467099 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa820071310 0x7fa8200c8580 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fa808006fd0 tx=0x7fa808001800 comp rx=0 tx=0).stop 2024-09-21T07:37:12.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.469+0000 7fa825800640 1 -- 172.21.15.39:0/3373467099 shutdown_connections 2024-09-21T07:37:12.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.469+0000 7fa825800640 1 --2- 172.21.15.39:0/3373467099 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa7fc078b90 0x7fa7fc07b0a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:12.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.469+0000 7fa825800640 1 --2- 172.21.15.39:0/3373467099 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8200c8ac0 0x7fa8201f6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:12.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.469+0000 7fa825800640 1 --2- 172.21.15.39:0/3373467099 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa820071310 0x7fa8200c8580 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:12.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.469+0000 7fa825800640 1 --2- 172.21.15.39:0/3373467099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa804003c10 0x7fa8201449c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:12.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.469+0000 7fa825800640 1 -- 172.21.15.39:0/3373467099 >> 172.21.15.39:0/3373467099 conn(0x7fa8200b3090 msgr2=0x7fa82015c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:12.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.469+0000 7fa825800640 1 -- 172.21.15.39:0/3373467099 shutdown_connections 2024-09-21T07:37:12.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:12.469+0000 7fa825800640 1 -- 172.21.15.39:0/3373467099 wait complete. 2024-09-21T07:37:12.492 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:37:16.470 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-09-21T07:37:17.493 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:37:17.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.731+0000 7ff34f200640 1 -- 172.21.15.39:0/2574666045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3500b3b60 msgr2=0x7ff3500cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:17.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.731+0000 7ff34f200640 1 --2- 172.21.15.39:0/2574666045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3500b3b60 0x7ff3500cfb50 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7ff35009dda0 tx=0x7ff33802a560 comp rx=0 tx=0).stop 2024-09-21T07:37:17.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.731+0000 7ff34f200640 1 -- 172.21.15.39:0/2574666045 shutdown_connections 2024-09-21T07:37:17.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.731+0000 7ff34f200640 1 --2- 172.21.15.39:0/2574666045 >> v2:172.21.15.148:3300/0 conn(0x7ff3500d0090 0x7ff3500d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:17.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.731+0000 7ff34f200640 1 --2- 172.21.15.39:0/2574666045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3500b3b60 0x7ff3500cfb50 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:17.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.731+0000 7ff34f200640 1 --2- 172.21.15.39:0/2574666045 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff3500d2af0 0x7ff3500b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:17.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.731+0000 7ff34f200640 1 -- 172.21.15.39:0/2574666045 >> 172.21.15.39:0/2574666045 conn(0x7ff35009d690 msgr2=0x7ff3500d4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:17.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.731+0000 7ff34f200640 1 -- 172.21.15.39:0/2574666045 shutdown_connections 2024-09-21T07:37:17.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.731+0000 7ff34f200640 1 -- 172.21.15.39:0/2574666045 wait complete. 2024-09-21T07:37:17.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.733+0000 7ff34f200640 1 Processor -- start 2024-09-21T07:37:17.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.733+0000 7ff34f200640 1 -- start start 2024-09-21T07:37:17.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.735+0000 7ff34f200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3500b3b60 0x7ff350151770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:17.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.735+0000 7ff34f200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff3500d0090 0x7ff3500cd170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:17.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.735+0000 7ff34f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3500d2af0 0x7ff3500cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:17.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.735+0000 7ff34f200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff3500d1050 con 0x7ff3500d2af0 2024-09-21T07:37:17.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.735+0000 7ff34f200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff3500d0ed0 con 0x7ff3500b3b60 2024-09-21T07:37:17.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.735+0000 7ff34f200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7ff3500d11d0 con 0x7ff3500d0090 2024-09-21T07:37:17.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.735+0000 7ff34e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3500d2af0 0x7ff3500cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:17.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.735+0000 7ff34e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3500d2af0 0x7ff3500cd6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42532/0 (socket says 172.21.15.39:42532) 2024-09-21T07:37:17.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.735+0000 7ff34e800640 1 -- 172.21.15.39:0/133266826 learned_addr learned my addr 172.21.15.39:0/133266826 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:17.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.735+0000 7ff345400640 1 --2- 172.21.15.39:0/133266826 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff3500d0090 0x7ff3500cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:17.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.735+0000 7ff34de00640 1 --2- 172.21.15.39:0/133266826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3500b3b60 0x7ff350151770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:17.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.736+0000 7ff34e800640 1 -- 172.21.15.39:0/133266826 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff3500d0090 msgr2=0x7ff3500cd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:17.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.736+0000 7ff34e800640 1 --2- 172.21.15.39:0/133266826 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff3500d0090 0x7ff3500cd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:17.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.736+0000 7ff34e800640 1 -- 172.21.15.39:0/133266826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3500b3b60 msgr2=0x7ff350151770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:17.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.736+0000 7ff34e800640 1 --2- 172.21.15.39:0/133266826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3500b3b60 0x7ff350151770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:17.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.736+0000 7ff34e800640 1 -- 172.21.15.39:0/133266826 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff350151cb0 con 0x7ff3500d2af0 2024-09-21T07:37:17.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.736+0000 7ff34e800640 1 --2- 172.21.15.39:0/133266826 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3500d2af0 0x7ff3500cd6b0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff34800dac0 tx=0x7ff34800df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:17.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.736+0000 7ff346a00640 1 -- 172.21.15.39:0/133266826 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff348017070 con 0x7ff3500d2af0 2024-09-21T07:37:17.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.736+0000 7ff346a00640 1 -- 172.21.15.39:0/133266826 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff348010460 con 0x7ff3500d2af0 2024-09-21T07:37:17.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.736+0000 7ff346a00640 1 -- 172.21.15.39:0/133266826 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3480108b0 con 0x7ff3500d2af0 2024-09-21T07:37:17.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.736+0000 7ff34f200640 1 -- 172.21.15.39:0/133266826 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff35015c450 con 0x7ff3500d2af0 2024-09-21T07:37:17.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.736+0000 7ff34f200640 1 -- 172.21.15.39:0/133266826 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3500ce220 con 0x7ff3500d2af0 2024-09-21T07:37:17.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.737+0000 7ff34f200640 1 -- 172.21.15.39:0/133266826 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3500d04a0 con 0x7ff3500d2af0 2024-09-21T07:37:17.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.739+0000 7ff346a00640 1 -- 172.21.15.39:0/133266826 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff348010a50 con 0x7ff3500d2af0 2024-09-21T07:37:17.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.740+0000 7ff346a00640 1 --2- 172.21.15.39:0/133266826 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff32c078b90 0x7ff32c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:17.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.740+0000 7ff346a00640 1 -- 172.21.15.39:0/133266826 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6915+0+0 (secure 0 0 0) 0x7ff34809b820 con 0x7ff3500d2af0 2024-09-21T07:37:17.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.740+0000 7ff34de00640 1 --2- 172.21.15.39:0/133266826 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff32c078b90 0x7ff32c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:17.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.741+0000 7ff34de00640 1 --2- 172.21.15.39:0/133266826 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff32c078b90 0x7ff32c07b0a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff340009ab0 tx=0x7ff340005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:17.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.744+0000 7ff346a00640 1 -- 172.21.15.39:0/133266826 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff350151cb0 con 0x7ff3500d2af0 2024-09-21T07:37:17.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.979+0000 7ff34f200640 1 -- 172.21.15.39:0/133266826 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3500d4240 con 0x7ff3500d2af0 2024-09-21T07:37:17.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.980+0000 7ff346a00640 1 -- 172.21.15.39:0/133266826 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5249 (secure 0 0 0) 0x7ff348063140 con 0x7ff3500d2af0 2024-09-21T07:37:17.982 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:37:17.982 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":67,"btime":"2024-09-21T07:37:11:423990+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6585,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/1472430787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1472430787},{"type":"v1","addr":"172.21.15.39:6834","nonce":1472430787}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:37:10.607422+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5621,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5621":{"gid":5621,"name":"d","rank":0,"incarnation":62,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3026827614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3026827614},{"type":"v1","addr":"172.21.15.148:6837","nonce":3026827614}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5621,"qdb_cluster":[6115,5621]},"id":6}]} 2024-09-21T07:37:17.982 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 67 2024-09-21T07:37:17.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.982+0000 7ff34f200640 1 -- 172.21.15.39:0/133266826 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff32c078b90 msgr2=0x7ff32c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:17.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.982+0000 7ff34f200640 1 --2- 172.21.15.39:0/133266826 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff32c078b90 0x7ff32c07b0a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff340009ab0 tx=0x7ff340005f90 comp rx=0 tx=0).stop 2024-09-21T07:37:17.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.982+0000 7ff34f200640 1 -- 172.21.15.39:0/133266826 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3500d2af0 msgr2=0x7ff3500cd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:17.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.982+0000 7ff34f200640 1 --2- 172.21.15.39:0/133266826 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3500d2af0 0x7ff3500cd6b0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff34800dac0 tx=0x7ff34800df80 comp rx=0 tx=0).stop 2024-09-21T07:37:17.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.982+0000 7ff34f200640 1 -- 172.21.15.39:0/133266826 shutdown_connections 2024-09-21T07:37:17.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.982+0000 7ff34f200640 1 --2- 172.21.15.39:0/133266826 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff32c078b90 0x7ff32c07b0a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:17.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.982+0000 7ff34f200640 1 --2- 172.21.15.39:0/133266826 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3500d2af0 0x7ff3500cd6b0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:17.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.982+0000 7ff34f200640 1 --2- 172.21.15.39:0/133266826 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff3500d0090 0x7ff3500cd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:17.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.982+0000 7ff34f200640 1 --2- 172.21.15.39:0/133266826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3500b3b60 0x7ff350151770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:17.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.982+0000 7ff34f200640 1 -- 172.21.15.39:0/133266826 >> 172.21.15.39:0/133266826 conn(0x7ff35009d690 msgr2=0x7ff350068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:17.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.983+0000 7ff34f200640 1 -- 172.21.15.39:0/133266826 shutdown_connections 2024-09-21T07:37:17.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:17.983+0000 7ff34f200640 1 -- 172.21.15.39:0/133266826 wait complete. 2024-09-21T07:37:18.011 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:37:23.012 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:37:23.082 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~7s 2024-09-21T07:37:23.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.250+0000 7fd5c9400640 1 -- 172.21.15.39:0/3648748212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5a8005350 msgr2=0x7fd5a800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:23.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.250+0000 7fd5c9400640 1 --2- 172.21.15.39:0/3648748212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5a8005350 0x7fd5a800b9c0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc001d90 tx=0x7fd5a8005830 comp rx=0 tx=0).stop 2024-09-21T07:37:23.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.250+0000 7fd5c9400640 1 -- 172.21.15.39:0/3648748212 shutdown_connections 2024-09-21T07:37:23.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.250+0000 7fd5c9400640 1 --2- 172.21.15.39:0/3648748212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5a8005350 0x7fd5a800b9c0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:23.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.250+0000 7fd5c9400640 1 -- 172.21.15.39:0/3648748212 >> 172.21.15.39:0/3648748212 conn(0x7fd5c406d410 msgr2=0x7fd5c40b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:23.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.250+0000 7fd5c9400640 1 -- 172.21.15.39:0/3648748212 shutdown_connections 2024-09-21T07:37:23.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.250+0000 7fd5c9400640 1 -- 172.21.15.39:0/3648748212 wait complete. 2024-09-21T07:37:23.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.252+0000 7fd5c9400640 1 Processor -- start 2024-09-21T07:37:23.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.252+0000 7fd5c9400640 1 -- start start 2024-09-21T07:37:23.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.254+0000 7fd5c9400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5a8005350 0x7fd5c41f63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:23.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.254+0000 7fd5c9400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5c41400f0 0x7fd5c41426f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.254+0000 7fd5c9400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5c4142c30 0x7fd5c4140ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.254+0000 7fd5c9400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c406ee70 con 0x7fd5c4142c30 2024-09-21T07:37:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.254+0000 7fd5c9400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c40cce20 con 0x7fd5c41400f0 2024-09-21T07:37:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.254+0000 7fd5c9400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd5c406eff0 con 0x7fd5a8005350 2024-09-21T07:37:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.254+0000 7fd5c8a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5c4142c30 0x7fd5c4140ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.254+0000 7fd5c8a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5c4142c30 0x7fd5c4140ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56860/0 (socket says 172.21.15.39:56860) 2024-09-21T07:37:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.254+0000 7fd5c8a00640 1 -- 172.21.15.39:0/2201499014 learned_addr learned my addr 172.21.15.39:0/2201499014 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.254+0000 7fd5bbe00640 1 --2- 172.21.15.39:0/2201499014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5c41400f0 0x7fd5c41426f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.254+0000 7fd5c3e00640 1 --2- 172.21.15.39:0/2201499014 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5a8005350 0x7fd5c41f63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.254+0000 7fd5c8a00640 1 -- 172.21.15.39:0/2201499014 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5a8005350 msgr2=0x7fd5c41f63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.254+0000 7fd5c8a00640 1 --2- 172.21.15.39:0/2201499014 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5a8005350 0x7fd5c41f63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:23.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.254+0000 7fd5c8a00640 1 -- 172.21.15.39:0/2201499014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5c41400f0 msgr2=0x7fd5c41426f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:23.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.254+0000 7fd5c8a00640 1 --2- 172.21.15.39:0/2201499014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5c41400f0 0x7fd5c41426f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:23.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.254+0000 7fd5c8a00640 1 -- 172.21.15.39:0/2201499014 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c4143040 con 0x7fd5c4142c30 2024-09-21T07:37:23.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.254+0000 7fd5bbe00640 1 --2- 172.21.15.39:0/2201499014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5c41400f0 0x7fd5c41426f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:37:23.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.255+0000 7fd5c8a00640 1 --2- 172.21.15.39:0/2201499014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5c4142c30 0x7fd5c4140ef0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc001eb0 tx=0x7fd5bc006ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:23.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.255+0000 7fd5c0c00640 1 -- 172.21.15.39:0/2201499014 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc003d80 con 0x7fd5c4142c30 2024-09-21T07:37:23.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.255+0000 7fd5c0c00640 1 -- 172.21.15.39:0/2201499014 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd5bc005b80 con 0x7fd5c4142c30 2024-09-21T07:37:23.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.255+0000 7fd5c0c00640 1 -- 172.21.15.39:0/2201499014 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc005f90 con 0x7fd5c4142c30 2024-09-21T07:37:23.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.255+0000 7fd5c9400640 1 -- 172.21.15.39:0/2201499014 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5c41f6910 con 0x7fd5c4142c30 2024-09-21T07:37:23.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.255+0000 7fd5c9400640 1 -- 172.21.15.39:0/2201499014 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5c413e800 con 0x7fd5c4142c30 2024-09-21T07:37:23.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.256+0000 7fd5c9400640 1 -- 172.21.15.39:0/2201499014 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5c406f4f0 con 0x7fd5c4142c30 2024-09-21T07:37:23.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.257+0000 7fd5c0c00640 1 -- 172.21.15.39:0/2201499014 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd5bc006130 con 0x7fd5c4142c30 2024-09-21T07:37:23.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.258+0000 7fd5c0c00640 1 --2- 172.21.15.39:0/2201499014 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd5a0078b90 0x7fd5a007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:23.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.258+0000 7fd5c0c00640 1 -- 172.21.15.39:0/2201499014 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6915+0+0 (secure 0 0 0) 0x7fd5bc0bb410 con 0x7fd5c4142c30 2024-09-21T07:37:23.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.259+0000 7fd5c3e00640 1 --2- 172.21.15.39:0/2201499014 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd5a0078b90 0x7fd5a007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:23.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.260+0000 7fd5c3e00640 1 --2- 172.21.15.39:0/2201499014 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd5a0078b90 0x7fd5a007b0a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4000fd0 tx=0x7fd5b40067a0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:23.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.263+0000 7fd5c0c00640 1 -- 172.21.15.39:0/2201499014 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd5bc043060 con 0x7fd5c4142c30 2024-09-21T07:37:23.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.501+0000 7fd5c9400640 1 -- 172.21.15.39:0/2201499014 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5c406fb50 con 0x7fd5c4142c30 2024-09-21T07:37:23.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.501+0000 7fd5c0c00640 1 -- 172.21.15.39:0/2201499014 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5249 (secure 0 0 0) 0x7fd5c4143040 con 0x7fd5c4142c30 2024-09-21T07:37:23.504 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:37:23.504 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":67,"btime":"2024-09-21T07:37:11:423990+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6585,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/1472430787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1472430787},{"type":"v1","addr":"172.21.15.39:6834","nonce":1472430787}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:37:10.607422+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5621,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5621":{"gid":5621,"name":"d","rank":0,"incarnation":62,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3026827614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3026827614},{"type":"v1","addr":"172.21.15.148:6837","nonce":3026827614}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5621,"qdb_cluster":[6115,5621]},"id":6}]} 2024-09-21T07:37:23.504 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 67 2024-09-21T07:37:23.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.505+0000 7fd5c9400640 1 -- 172.21.15.39:0/2201499014 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd5a0078b90 msgr2=0x7fd5a007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:23.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.505+0000 7fd5c9400640 1 --2- 172.21.15.39:0/2201499014 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd5a0078b90 0x7fd5a007b0a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4000fd0 tx=0x7fd5b40067a0 comp rx=0 tx=0).stop 2024-09-21T07:37:23.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.505+0000 7fd5c9400640 1 -- 172.21.15.39:0/2201499014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5c4142c30 msgr2=0x7fd5c4140ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:23.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.505+0000 7fd5c9400640 1 --2- 172.21.15.39:0/2201499014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5c4142c30 0x7fd5c4140ef0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc001eb0 tx=0x7fd5bc006ce0 comp rx=0 tx=0).stop 2024-09-21T07:37:23.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.505+0000 7fd5c9400640 1 -- 172.21.15.39:0/2201499014 shutdown_connections 2024-09-21T07:37:23.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.505+0000 7fd5c9400640 1 --2- 172.21.15.39:0/2201499014 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd5a0078b90 0x7fd5a007b0a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:23.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.505+0000 7fd5c9400640 1 --2- 172.21.15.39:0/2201499014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5c4142c30 0x7fd5c4140ef0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:23.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.505+0000 7fd5c9400640 1 --2- 172.21.15.39:0/2201499014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5c41400f0 0x7fd5c41426f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:23.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.505+0000 7fd5c9400640 1 --2- 172.21.15.39:0/2201499014 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5a8005350 0x7fd5c41f63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:23.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.505+0000 7fd5c9400640 1 -- 172.21.15.39:0/2201499014 >> 172.21.15.39:0/2201499014 conn(0x7fd5c406d410 msgr2=0x7fd5c4074790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:23.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.505+0000 7fd5c9400640 1 -- 172.21.15.39:0/2201499014 shutdown_connections 2024-09-21T07:37:23.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:23.505+0000 7fd5c9400640 1 -- 172.21.15.39:0/2201499014 wait complete. 2024-09-21T07:37:23.528 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:37:28.529 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:37:28.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.766+0000 7fdda7200640 1 -- 172.21.15.39:0/1595447875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdda00ca800 msgr2=0x7fdda00cba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:28.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.766+0000 7fdda7200640 1 --2- 172.21.15.39:0/1595447875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdda00ca800 0x7fdda00cba00 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fdda0075080 tx=0x7fdd9002a560 comp rx=0 tx=0).stop 2024-09-21T07:37:28.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.766+0000 7fdda7200640 1 -- 172.21.15.39:0/1595447875 shutdown_connections 2024-09-21T07:37:28.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.766+0000 7fdda7200640 1 --2- 172.21.15.39:0/1595447875 >> v2:172.21.15.148:3300/0 conn(0x7fdda00704e0 0x7fdda00cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:28.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.766+0000 7fdda7200640 1 --2- 172.21.15.39:0/1595447875 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdda00cad30 0x7fdda006ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:28.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.766+0000 7fdda7200640 1 --2- 172.21.15.39:0/1595447875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdda00ca800 0x7fdda00cba00 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:28.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.766+0000 7fdda7200640 1 -- 172.21.15.39:0/1595447875 >> 172.21.15.39:0/1595447875 conn(0x7fdda00b1cb0 msgr2=0x7fdda00b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:28.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.766+0000 7fdda7200640 1 -- 172.21.15.39:0/1595447875 shutdown_connections 2024-09-21T07:37:28.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.766+0000 7fdda7200640 1 -- 172.21.15.39:0/1595447875 wait complete. 2024-09-21T07:37:28.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.769+0000 7fdda7200640 1 Processor -- start 2024-09-21T07:37:28.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.769+0000 7fdda7200640 1 -- start start 2024-09-21T07:37:28.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.770+0000 7fdda7200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdda00704e0 0x7fdda009ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:28.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.770+0000 7fdda7200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdda00ca800 0x7fdda01f5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:28.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.770+0000 7fdda7200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdda00cad30 0x7fdda01f9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:28.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.770+0000 7fdda7200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdda00ccf80 con 0x7fdda00704e0 2024-09-21T07:37:28.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.770+0000 7fdda7200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdda00cec60 con 0x7fdda00cad30 2024-09-21T07:37:28.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.770+0000 7fdda7200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fdda00cd100 con 0x7fdda00ca800 2024-09-21T07:37:28.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.770+0000 7fdd9fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdda00704e0 0x7fdda009ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:28.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.771+0000 7fdd9fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdda00704e0 0x7fdda009ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56882/0 (socket says 172.21.15.39:56882) 2024-09-21T07:37:28.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.771+0000 7fdd9fe00640 1 -- 172.21.15.39:0/1839812569 learned_addr learned my addr 172.21.15.39:0/1839812569 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:28.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.771+0000 7fdd97e00640 1 --2- 172.21.15.39:0/1839812569 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdda00ca800 0x7fdda01f5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:28.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.771+0000 7fdda5200640 1 --2- 172.21.15.39:0/1839812569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdda00cad30 0x7fdda01f9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:28.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.771+0000 7fdd9fe00640 1 -- 172.21.15.39:0/1839812569 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdda00ca800 msgr2=0x7fdda01f5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:28.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.771+0000 7fdd9fe00640 1 --2- 172.21.15.39:0/1839812569 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdda00ca800 0x7fdda01f5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:28.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.771+0000 7fdd9fe00640 1 -- 172.21.15.39:0/1839812569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdda00cad30 msgr2=0x7fdda01f9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:28.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.771+0000 7fdd9fe00640 1 --2- 172.21.15.39:0/1839812569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdda00cad30 0x7fdda01f9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:28.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.771+0000 7fdd9fe00640 1 -- 172.21.15.39:0/1839812569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda01fa450 con 0x7fdda00704e0 2024-09-21T07:37:28.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.771+0000 7fdd97e00640 1 --2- 172.21.15.39:0/1839812569 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdda00ca800 0x7fdda01f5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:37:28.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.771+0000 7fdda5200640 1 --2- 172.21.15.39:0/1839812569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdda00cad30 0x7fdda01f9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:37:28.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.771+0000 7fdd9fe00640 1 --2- 172.21.15.39:0/1839812569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdda00704e0 0x7fdda009ad60 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fdd90034040 tx=0x7fdd90003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:28.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.772+0000 7fdd9cc00640 1 -- 172.21.15.39:0/1839812569 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9003e030 con 0x7fdda00704e0 2024-09-21T07:37:28.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.772+0000 7fdd9cc00640 1 -- 172.21.15.39:0/1839812569 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd90003e00 con 0x7fdda00704e0 2024-09-21T07:37:28.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.772+0000 7fdd9cc00640 1 -- 172.21.15.39:0/1839812569 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd90035040 con 0x7fdda00704e0 2024-09-21T07:37:28.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.772+0000 7fdda7200640 1 -- 172.21.15.39:0/1839812569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdda009da00 con 0x7fdda00704e0 2024-09-21T07:37:28.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.772+0000 7fdda7200640 1 -- 172.21.15.39:0/1839812569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdda015dc20 con 0x7fdda00704e0 2024-09-21T07:37:28.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.773+0000 7fdda7200640 1 -- 172.21.15.39:0/1839812569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdda0071790 con 0x7fdda00704e0 2024-09-21T07:37:28.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.774+0000 7fdd9cc00640 1 -- 172.21.15.39:0/1839812569 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdd90041030 con 0x7fdda00704e0 2024-09-21T07:37:28.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.775+0000 7fdd9cc00640 1 --2- 172.21.15.39:0/1839812569 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdd7c078b90 0x7fdd7c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:28.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.775+0000 7fdd9cc00640 1 -- 172.21.15.39:0/1839812569 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6915+0+0 (secure 0 0 0) 0x7fdd900bbb00 con 0x7fdda00704e0 2024-09-21T07:37:28.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.775+0000 7fdd97e00640 1 --2- 172.21.15.39:0/1839812569 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdd7c078b90 0x7fdd7c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:28.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.776+0000 7fdd97e00640 1 --2- 172.21.15.39:0/1839812569 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdd7c078b90 0x7fdd7c07b0a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fdd88005fd0 tx=0x7fdd88005cf0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:28.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:28.780+0000 7fdd9cc00640 1 -- 172.21.15.39:0/1839812569 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdda0071790 con 0x7fdda00704e0 2024-09-21T07:37:29.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.014+0000 7fdda7200640 1 -- 172.21.15.39:0/1839812569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdda00cd690 con 0x7fdda00704e0 2024-09-21T07:37:29.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.015+0000 7fdd9cc00640 1 -- 172.21.15.39:0/1839812569 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5297 (secure 0 0 0) 0x7fdd900bbe20 con 0x7fdda00704e0 2024-09-21T07:37:29.016 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:37:29.016 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":68,"btime":"2024-09-21T07:37:26:371834+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6585,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/1472430787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1472430787},{"type":"v1","addr":"172.21.15.39:6834","nonce":1472430787}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:37:26.371808+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5621,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5621":{"gid":5621,"name":"d","rank":0,"incarnation":62,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3026827614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3026827614},{"type":"v1","addr":"172.21.15.148:6837","nonce":3026827614}]},"join_fscid":-1,"laggy_since":"2024-09-21T07:37:26.371808+0000","export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5621,"qdb_cluster":[5621,6115]},"id":6}]} 2024-09-21T07:37:29.016 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 68 2024-09-21T07:37:29.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.018+0000 7fdda7200640 1 -- 172.21.15.39:0/1839812569 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdd7c078b90 msgr2=0x7fdd7c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:29.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.018+0000 7fdda7200640 1 --2- 172.21.15.39:0/1839812569 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdd7c078b90 0x7fdd7c07b0a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fdd88005fd0 tx=0x7fdd88005cf0 comp rx=0 tx=0).stop 2024-09-21T07:37:29.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.018+0000 7fdda7200640 1 -- 172.21.15.39:0/1839812569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdda00704e0 msgr2=0x7fdda009ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:29.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.018+0000 7fdda7200640 1 --2- 172.21.15.39:0/1839812569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdda00704e0 0x7fdda009ad60 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fdd90034040 tx=0x7fdd90003910 comp rx=0 tx=0).stop 2024-09-21T07:37:29.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.018+0000 7fdda7200640 1 -- 172.21.15.39:0/1839812569 shutdown_connections 2024-09-21T07:37:29.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.018+0000 7fdda7200640 1 --2- 172.21.15.39:0/1839812569 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdd7c078b90 0x7fdd7c07b0a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:29.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.018+0000 7fdda7200640 1 --2- 172.21.15.39:0/1839812569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdda00cad30 0x7fdda01f9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:29.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.018+0000 7fdda7200640 1 --2- 172.21.15.39:0/1839812569 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdda00ca800 0x7fdda01f5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:29.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.018+0000 7fdda7200640 1 --2- 172.21.15.39:0/1839812569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdda00704e0 0x7fdda009ad60 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:29.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.018+0000 7fdda7200640 1 -- 172.21.15.39:0/1839812569 >> 172.21.15.39:0/1839812569 conn(0x7fdda00b1cb0 msgr2=0x7fdda0074400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:29.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.018+0000 7fdda7200640 1 -- 172.21.15.39:0/1839812569 shutdown_connections 2024-09-21T07:37:29.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.018+0000 7fdda7200640 1 -- 172.21.15.39:0/1839812569 wait complete. 2024-09-21T07:37:29.045 DEBUG:tasks.ceph_test_case:wait_until_true: success in 15s and 0 retries 2024-09-21T07:37:29.045 DEBUG:teuthology.orchestra.run.smithi148:> sudo sysctl -n kernel.core_pattern 2024-09-21T07:37:29.104 INFO:teuthology.orchestra.run.smithi148.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T07:37:29.106 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2024-09-21T07:37:29.107 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2024-09-21T07:37:29.171 INFO:teuthology.orchestra.run.smithi148.stdout:1726904231.42961.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i d', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2024-09-21T07:37:29.175 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1726904231.42961.core'] 2024-09-21T07:37:29.175 INFO:tasks.cephfs.cephfs_test_case:Found core file 1726904231.42961.core, deleting it 2024-09-21T07:37:29.175 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1726904231.42961.core 2024-09-21T07:37:29.265 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 6:0 2024-09-21T07:37:29.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.491+0000 7fdfbea00640 1 -- 172.21.15.39:0/1516721929 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfb80af3c0 msgr2=0x7fdfb80b40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:29.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.491+0000 7fdfbea00640 1 --2- 172.21.15.39:0/1516721929 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfb80af3c0 0x7fdfb80b40b0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4009830 tx=0x7fdfa402a510 comp rx=0 tx=0).stop 2024-09-21T07:37:29.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.491+0000 7fdfbea00640 1 -- 172.21.15.39:0/1516721929 shutdown_connections 2024-09-21T07:37:29.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.491+0000 7fdfbea00640 1 --2- 172.21.15.39:0/1516721929 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdfb80ccb50 0x7fdfb80ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:29.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.491+0000 7fdfbea00640 1 --2- 172.21.15.39:0/1516721929 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfb80af3c0 0x7fdfb80b40b0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:29.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.491+0000 7fdfbea00640 1 -- 172.21.15.39:0/1516721929 >> 172.21.15.39:0/1516721929 conn(0x7fdfb806d7d0 msgr2=0x7fdfb80b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:29.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.491+0000 7fdfbea00640 1 -- 172.21.15.39:0/1516721929 shutdown_connections 2024-09-21T07:37:29.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.492+0000 7fdfbea00640 1 -- 172.21.15.39:0/1516721929 wait complete. 2024-09-21T07:37:29.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.494+0000 7fdfbea00640 1 Processor -- start 2024-09-21T07:37:29.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.494+0000 7fdfbea00640 1 -- start start 2024-09-21T07:37:29.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.495+0000 7fdfbea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfb80af3c0 0x7fdfb8159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:29.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.495+0000 7fdfbea00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfb80cc1e0 0x7fdfb80afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:29.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.495+0000 7fdfbea00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdfb80ccb50 0x7fdfb80b0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:29.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.495+0000 7fdfbea00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb8153520 con 0x7fdfb80af3c0 2024-09-21T07:37:29.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.495+0000 7fdfbea00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb81533a0 con 0x7fdfb80cc1e0 2024-09-21T07:37:29.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.495+0000 7fdfbea00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fdfb81536a0 con 0x7fdfb80ccb50 2024-09-21T07:37:29.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.496+0000 7fdfb7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfb80af3c0 0x7fdfb8159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:29.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.496+0000 7fdfb7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfb80af3c0 0x7fdfb8159c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56902/0 (socket says 172.21.15.39:56902) 2024-09-21T07:37:29.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.496+0000 7fdfb7e00640 1 -- 172.21.15.39:0/2996331222 learned_addr learned my addr 172.21.15.39:0/2996331222 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:29.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.496+0000 7fdfbca00640 1 --2- 172.21.15.39:0/2996331222 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdfb80ccb50 0x7fdfb80b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:29.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.496+0000 7fdfb7400640 1 --2- 172.21.15.39:0/2996331222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfb80cc1e0 0x7fdfb80afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:29.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.496+0000 7fdfb7e00640 1 -- 172.21.15.39:0/2996331222 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdfb80ccb50 msgr2=0x7fdfb80b0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:29.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.496+0000 7fdfb7e00640 1 --2- 172.21.15.39:0/2996331222 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdfb80ccb50 0x7fdfb80b0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:29.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.496+0000 7fdfb7e00640 1 -- 172.21.15.39:0/2996331222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfb80cc1e0 msgr2=0x7fdfb80afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:29.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.496+0000 7fdfb7e00640 1 --2- 172.21.15.39:0/2996331222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfb80cc1e0 0x7fdfb80afbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:29.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.496+0000 7fdfb7e00640 1 -- 172.21.15.39:0/2996331222 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb81f9720 con 0x7fdfb80af3c0 2024-09-21T07:37:29.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.496+0000 7fdfb7e00640 1 --2- 172.21.15.39:0/2996331222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfb80af3c0 0x7fdfb8159c10 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fdfa800c3c0 tx=0x7fdfa800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:29.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.497+0000 7fdfafe00640 1 -- 172.21.15.39:0/2996331222 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa8017070 con 0x7fdfb80af3c0 2024-09-21T07:37:29.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.497+0000 7fdfafe00640 1 -- 172.21.15.39:0/2996331222 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfa80035f0 con 0x7fdfb80af3c0 2024-09-21T07:37:29.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.497+0000 7fdfafe00640 1 -- 172.21.15.39:0/2996331222 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa8003a80 con 0x7fdfb80af3c0 2024-09-21T07:37:29.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.497+0000 7fdfbea00640 1 -- 172.21.15.39:0/2996331222 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfb81f9a00 con 0x7fdfb80af3c0 2024-09-21T07:37:29.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.497+0000 7fdfbea00640 1 -- 172.21.15.39:0/2996331222 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfb81f9fc0 con 0x7fdfb80af3c0 2024-09-21T07:37:29.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.498+0000 7fdfbea00640 1 -- 172.21.15.39:0/2996331222 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfb8153b50 con 0x7fdfb80af3c0 2024-09-21T07:37:29.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.499+0000 7fdfafe00640 1 -- 172.21.15.39:0/2996331222 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdfa801a070 con 0x7fdfb80af3c0 2024-09-21T07:37:29.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.500+0000 7fdfafe00640 1 --2- 172.21.15.39:0/2996331222 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdf98078bc0 0x7fdf9807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:29.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.500+0000 7fdfafe00640 1 -- 172.21.15.39:0/2996331222 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6915+0+0 (secure 0 0 0) 0x7fdfa809a360 con 0x7fdfb80af3c0 2024-09-21T07:37:29.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.500+0000 7fdfb7400640 1 --2- 172.21.15.39:0/2996331222 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdf98078bc0 0x7fdf9807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:29.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.501+0000 7fdfb7400640 1 --2- 172.21.15.39:0/2996331222 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdf98078bc0 0x7fdf9807b0d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4034000 tx=0x7fdfa4004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:29.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.504+0000 7fdfafe00640 1 -- 172.21.15.39:0/2996331222 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdfb80cd4a0 con 0x7fdfb80af3c0 2024-09-21T07:37:29.693 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~13s 2024-09-21T07:37:29.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.780+0000 7fdfbea00640 1 -- 172.21.15.39:0/2996331222 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "6:0"} v 0) -- 0x7fdfb80cd760 con 0x7fdfb80af3c0 2024-09-21T07:37:29.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.783+0000 7fdfafe00640 1 -- 172.21.15.39:0/2996331222 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "6:0"}]=0 failed mds gid 5621 v69) ==== 97+0+0 (secure 0 0 0) 0x7fdfa8061d10 con 0x7fdfb80af3c0 2024-09-21T07:37:29.784 INFO:teuthology.orchestra.run.smithi039.stderr:failed mds gid 5621 2024-09-21T07:37:29.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.786+0000 7fdfbea00640 1 -- 172.21.15.39:0/2996331222 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdf98078bc0 msgr2=0x7fdf9807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:29.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.786+0000 7fdfbea00640 1 --2- 172.21.15.39:0/2996331222 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdf98078bc0 0x7fdf9807b0d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4034000 tx=0x7fdfa4004730 comp rx=0 tx=0).stop 2024-09-21T07:37:29.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.786+0000 7fdfbea00640 1 -- 172.21.15.39:0/2996331222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfb80af3c0 msgr2=0x7fdfb8159c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:29.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.786+0000 7fdfbea00640 1 --2- 172.21.15.39:0/2996331222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfb80af3c0 0x7fdfb8159c10 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fdfa800c3c0 tx=0x7fdfa800c880 comp rx=0 tx=0).stop 2024-09-21T07:37:29.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.786+0000 7fdfbea00640 1 -- 172.21.15.39:0/2996331222 shutdown_connections 2024-09-21T07:37:29.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.786+0000 7fdfbea00640 1 --2- 172.21.15.39:0/2996331222 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdf98078bc0 0x7fdf9807b0d0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:29.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.786+0000 7fdfbea00640 1 --2- 172.21.15.39:0/2996331222 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdfb80ccb50 0x7fdfb80b0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:29.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.786+0000 7fdfbea00640 1 --2- 172.21.15.39:0/2996331222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfb80cc1e0 0x7fdfb80afbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:29.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.786+0000 7fdfbea00640 1 --2- 172.21.15.39:0/2996331222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfb80af3c0 0x7fdfb8159c10 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:29.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.786+0000 7fdfbea00640 1 -- 172.21.15.39:0/2996331222 >> 172.21.15.39:0/2996331222 conn(0x7fdfb806d7d0 msgr2=0x7fdfb80b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:29.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.786+0000 7fdfbea00640 1 -- 172.21.15.39:0/2996331222 shutdown_connections 2024-09-21T07:37:29.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:29.786+0000 7fdfbea00640 1 -- 172.21.15.39:0/2996331222 wait complete. 2024-09-21T07:37:29.820 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T07:37:29.820 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T07:37:29.820 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T07:37:29.820 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:37:29.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:37:29.820 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/daemon/state.py", line 146, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi148 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d' 2024-09-21T07:37:29.820 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T07:37:29.821 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T07:37:29.825 INFO:tasks.ceph.mds.d:Started 2024-09-21T07:37:29.825 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:37:30.013 INFO:tasks.ceph.mds.d.smithi148.stdout:starting mds.d at 2024-09-21T07:37:30.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.050+0000 7f833b800640 1 -- 172.21.15.39:0/3931170916 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83340cf660 msgr2=0x7f83340cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:30.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.050+0000 7f833b800640 1 --2- 172.21.15.39:0/3931170916 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83340cf660 0x7f83340cf010 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f83340c7180 tx=0x7f832802a4a0 comp rx=0 tx=0).stop 2024-09-21T07:37:30.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.050+0000 7f833b800640 1 -- 172.21.15.39:0/3931170916 shutdown_connections 2024-09-21T07:37:30.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.050+0000 7f833b800640 1 --2- 172.21.15.39:0/3931170916 >> v2:172.21.15.148:3300/0 conn(0x7f83340cfb90 0x7f83340ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:30.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.050+0000 7f833b800640 1 --2- 172.21.15.39:0/3931170916 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83340cf660 0x7f83340cf010 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:30.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.050+0000 7f833b800640 1 -- 172.21.15.39:0/3931170916 >> 172.21.15.39:0/3931170916 conn(0x7f83340b1cb0 msgr2=0x7f83340b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:30.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.051+0000 7f833b800640 1 -- 172.21.15.39:0/3931170916 shutdown_connections 2024-09-21T07:37:30.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.051+0000 7f833b800640 1 -- 172.21.15.39:0/3931170916 wait complete. 2024-09-21T07:37:30.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.053+0000 7f833b800640 1 Processor -- start 2024-09-21T07:37:30.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.053+0000 7f833b800640 1 -- start start 2024-09-21T07:37:30.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.055+0000 7f833b800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83340cd070 0x7f833409aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:30.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.055+0000 7f833b800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83340cf660 0x7f8334152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:30.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.055+0000 7f833b800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83340cfb90 0x7f8334144150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:30.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.055+0000 7f833b800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8334070f20 con 0x7f83340cd070 2024-09-21T07:37:30.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.055+0000 7f833b800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f83340af810 con 0x7f83340cfb90 2024-09-21T07:37:30.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.055+0000 7f833b800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f83340710a0 con 0x7f83340cf660 2024-09-21T07:37:30.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.055+0000 7f8338e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83340cd070 0x7f833409aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:30.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.055+0000 7f8338e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83340cd070 0x7f833409aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56916/0 (socket says 172.21.15.39:56916) 2024-09-21T07:37:30.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.055+0000 7f8338e00640 1 -- 172.21.15.39:0/244995747 learned_addr learned my addr 172.21.15.39:0/244995747 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:30.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.055+0000 7f8339800640 1 --2- 172.21.15.39:0/244995747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83340cfb90 0x7f8334144150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:30.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.055+0000 7f8333e00640 1 --2- 172.21.15.39:0/244995747 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83340cf660 0x7f8334152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:30.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.055+0000 7f8338e00640 1 -- 172.21.15.39:0/244995747 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83340cf660 msgr2=0x7f8334152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:30.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.055+0000 7f8338e00640 1 --2- 172.21.15.39:0/244995747 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83340cf660 0x7f8334152a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:30.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.055+0000 7f8338e00640 1 -- 172.21.15.39:0/244995747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83340cfb90 msgr2=0x7f8334144150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:30.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.056+0000 7f8338e00640 1 --2- 172.21.15.39:0/244995747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83340cfb90 0x7f8334144150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:30.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.056+0000 7f8338e00640 1 -- 172.21.15.39:0/244995747 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8334158220 con 0x7f83340cd070 2024-09-21T07:37:30.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.056+0000 7f8339800640 1 --2- 172.21.15.39:0/244995747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83340cfb90 0x7f8334144150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:37:30.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.056+0000 7f8338e00640 1 --2- 172.21.15.39:0/244995747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83340cd070 0x7f833409aa30 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f8328009780 tx=0x7f8328033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:30.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.056+0000 7f8330c00640 1 -- 172.21.15.39:0/244995747 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8328031d60 con 0x7f83340cd070 2024-09-21T07:37:30.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.056+0000 7f8330c00640 1 -- 172.21.15.39:0/244995747 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8328033e10 con 0x7f83340cd070 2024-09-21T07:37:30.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.056+0000 7f8330c00640 1 -- 172.21.15.39:0/244995747 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8328005260 con 0x7f83340cd070 2024-09-21T07:37:30.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.056+0000 7f833b800640 1 -- 172.21.15.39:0/244995747 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f833415c2b0 con 0x7f83340cd070 2024-09-21T07:37:30.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.056+0000 7f833b800640 1 -- 172.21.15.39:0/244995747 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f833415c870 con 0x7f83340cd070 2024-09-21T07:37:30.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.057+0000 7f833b800640 1 -- 172.21.15.39:0/244995747 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83340cb370 con 0x7f83340cd070 2024-09-21T07:37:30.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.059+0000 7f8330c00640 1 -- 172.21.15.39:0/244995747 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8328005400 con 0x7f83340cd070 2024-09-21T07:37:30.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.060+0000 7f8330c00640 1 --2- 172.21.15.39:0/244995747 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8314078b90 0x7f831407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:30.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.060+0000 7f8330c00640 1 -- 172.21.15.39:0/244995747 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 7001+0+0 (secure 0 0 0) 0x7f83280bc490 con 0x7f83340cd070 2024-09-21T07:37:30.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.060+0000 7f8333e00640 1 --2- 172.21.15.39:0/244995747 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8314078b90 0x7f831407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:30.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.061+0000 7f8333e00640 1 --2- 172.21.15.39:0/244995747 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8314078b90 0x7f831407b0a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f833415e3c0 tx=0x7f8320005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:30.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.064+0000 7f8330c00640 1 -- 172.21.15.39:0/244995747 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f83340cb370 con 0x7f83340cd070 2024-09-21T07:37:30.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.300+0000 7f833b800640 1 -- 172.21.15.39:0/244995747 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83340cf010 con 0x7f83340cd070 2024-09-21T07:37:30.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.301+0000 7f8330c00640 1 -- 172.21.15.39:0/244995747 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+4415 (secure 0 0 0) 0x7f832803f030 con 0x7f83340cd070 2024-09-21T07:37:30.302 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:37:30.302 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":70,"btime":"2024-09-21T07:37:29:791146+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:37:29.791119+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6585,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6585":{"gid":6585,"name":"c","rank":0,"incarnation":70,"state":"up:replay","state_seq":1,"addr":"172.21.15.39:6834/1472430787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1472430787},{"type":"v1","addr":"172.21.15.39:6834","nonce":1472430787}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6115,"qdb_cluster":[6115]},"id":6}]} 2024-09-21T07:37:30.302 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 70 2024-09-21T07:37:30.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.303+0000 7f833b800640 1 -- 172.21.15.39:0/244995747 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8314078b90 msgr2=0x7f831407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:30.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.303+0000 7f833b800640 1 --2- 172.21.15.39:0/244995747 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8314078b90 0x7f831407b0a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f833415e3c0 tx=0x7f8320005f90 comp rx=0 tx=0).stop 2024-09-21T07:37:30.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.303+0000 7f833b800640 1 -- 172.21.15.39:0/244995747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83340cd070 msgr2=0x7f833409aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:30.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.303+0000 7f833b800640 1 --2- 172.21.15.39:0/244995747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83340cd070 0x7f833409aa30 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f8328009780 tx=0x7f8328033c20 comp rx=0 tx=0).stop 2024-09-21T07:37:30.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.303+0000 7f833b800640 1 -- 172.21.15.39:0/244995747 shutdown_connections 2024-09-21T07:37:30.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.303+0000 7f833b800640 1 --2- 172.21.15.39:0/244995747 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8314078b90 0x7f831407b0a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:30.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.303+0000 7f833b800640 1 --2- 172.21.15.39:0/244995747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83340cfb90 0x7f8334144150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:30.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.303+0000 7f833b800640 1 --2- 172.21.15.39:0/244995747 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83340cf660 0x7f8334152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:30.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.303+0000 7f833b800640 1 --2- 172.21.15.39:0/244995747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83340cd070 0x7f833409aa30 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:30.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.303+0000 7f833b800640 1 -- 172.21.15.39:0/244995747 >> 172.21.15.39:0/244995747 conn(0x7f83340b1cb0 msgr2=0x7f83340b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:30.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.304+0000 7f833b800640 1 -- 172.21.15.39:0/244995747 shutdown_connections 2024-09-21T07:37:30.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:30.304+0000 7f833b800640 1 -- 172.21.15.39:0/244995747 wait complete. 2024-09-21T07:37:30.327 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-09-21T07:37:30.616 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:37:30.620 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:37:35.328 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:37:35.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.566+0000 7f4ebca00640 1 -- 172.21.15.39:0/3380649284 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4eb815c5d0 msgr2=0x7f4eb80cad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:35.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.566+0000 7f4ebca00640 1 --2- 172.21.15.39:0/3380649284 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4eb815c5d0 0x7f4eb80cad30 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0009830 tx=0x7f4ea002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:37:35.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.566+0000 7f4ebca00640 1 -- 172.21.15.39:0/3380649284 shutdown_connections 2024-09-21T07:37:35.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.566+0000 7f4ebca00640 1 --2- 172.21.15.39:0/3380649284 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4eb80b4b80 0x7f4eb80cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:35.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.566+0000 7f4ebca00640 1 --2- 172.21.15.39:0/3380649284 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4eb815c5d0 0x7f4eb80cad30 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:35.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.566+0000 7f4ebca00640 1 -- 172.21.15.39:0/3380649284 >> 172.21.15.39:0/3380649284 conn(0x7f4eb806d870 msgr2=0x7f4eb806ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:35.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.567+0000 7f4ebca00640 1 -- 172.21.15.39:0/3380649284 shutdown_connections 2024-09-21T07:37:35.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.567+0000 7f4ebca00640 1 -- 172.21.15.39:0/3380649284 wait complete. 2024-09-21T07:37:35.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.569+0000 7f4ebca00640 1 Processor -- start 2024-09-21T07:37:35.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.569+0000 7f4ebca00640 1 -- start start 2024-09-21T07:37:35.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.570+0000 7f4ebca00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4eb80b4b80 0x7f4eb81539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:35.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.570+0000 7f4ebca00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4eb80ca5a0 0x7f4eb80afb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:35.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.570+0000 7f4ebca00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4eb815c5d0 0x7f4eb80b00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:35.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.570+0000 7f4ebca00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb80d0560 con 0x7f4eb80ca5a0 2024-09-21T07:37:35.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.570+0000 7f4ebca00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb80d03e0 con 0x7f4eb80b4b80 2024-09-21T07:37:35.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.570+0000 7f4ebca00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f4eb80d06e0 con 0x7f4eb815c5d0 2024-09-21T07:37:35.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.571+0000 7f4eaea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4eb80ca5a0 0x7f4eb80afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:35.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.571+0000 7f4eaea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4eb80ca5a0 0x7f4eb80afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40752/0 (socket says 172.21.15.39:40752) 2024-09-21T07:37:35.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.571+0000 7f4eaea00640 1 -- 172.21.15.39:0/426087436 learned_addr learned my addr 172.21.15.39:0/426087436 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:35.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.571+0000 7f4eb7e00640 1 --2- 172.21.15.39:0/426087436 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4eb815c5d0 0x7f4eb80b00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:35.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.571+0000 7f4eaea00640 1 -- 172.21.15.39:0/426087436 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4eb815c5d0 msgr2=0x7f4eb80b00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:35.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.571+0000 7f4eb7400640 1 --2- 172.21.15.39:0/426087436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4eb80b4b80 0x7f4eb81539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:35.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.571+0000 7f4eaea00640 1 --2- 172.21.15.39:0/426087436 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4eb815c5d0 0x7f4eb80b00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:35.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.571+0000 7f4eaea00640 1 -- 172.21.15.39:0/426087436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4eb80b4b80 msgr2=0x7f4eb81539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:35.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.571+0000 7f4eaea00640 1 --2- 172.21.15.39:0/426087436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4eb80b4b80 0x7f4eb81539b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:35.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.571+0000 7f4eaea00640 1 -- 172.21.15.39:0/426087436 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb80b0c40 con 0x7f4eb80ca5a0 2024-09-21T07:37:35.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.571+0000 7f4eb7e00640 1 --2- 172.21.15.39:0/426087436 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4eb815c5d0 0x7f4eb80b00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:37:35.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.571+0000 7f4eb7400640 1 --2- 172.21.15.39:0/426087436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4eb80b4b80 0x7f4eb81539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:37:35.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.571+0000 7f4eaea00640 1 --2- 172.21.15.39:0/426087436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4eb80ca5a0 0x7f4eb80afb90 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0009950 tx=0x7f4ea00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:35.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.572+0000 7f4eafe00640 1 -- 172.21.15.39:0/426087436 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea00044e0 con 0x7f4eb80ca5a0 2024-09-21T07:37:35.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.572+0000 7f4eafe00640 1 -- 172.21.15.39:0/426087436 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ea0005310 con 0x7f4eb80ca5a0 2024-09-21T07:37:35.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.572+0000 7f4eafe00640 1 -- 172.21.15.39:0/426087436 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0003800 con 0x7f4eb80ca5a0 2024-09-21T07:37:35.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.572+0000 7f4ebca00640 1 -- 172.21.15.39:0/426087436 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eb80b1750 con 0x7f4eb80ca5a0 2024-09-21T07:37:35.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.572+0000 7f4ebca00640 1 -- 172.21.15.39:0/426087436 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eb80b1190 con 0x7f4eb80ca5a0 2024-09-21T07:37:35.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.573+0000 7f4ebca00640 1 -- 172.21.15.39:0/426087436 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eb80b2ab0 con 0x7f4eb80ca5a0 2024-09-21T07:37:35.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.574+0000 7f4eafe00640 1 -- 172.21.15.39:0/426087436 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4ea00039a0 con 0x7f4eb80ca5a0 2024-09-21T07:37:35.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.575+0000 7f4eafe00640 1 --2- 172.21.15.39:0/426087436 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4e94078b90 0x7f4e9407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:35.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.575+0000 7f4eafe00640 1 -- 172.21.15.39:0/426087436 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 7001+0+0 (secure 0 0 0) 0x7f4ea00bc6a0 con 0x7f4eb80ca5a0 2024-09-21T07:37:35.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.576+0000 7f4eb7400640 1 --2- 172.21.15.39:0/426087436 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4e94078b90 0x7f4e9407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:35.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.576+0000 7f4eb7400640 1 --2- 172.21.15.39:0/426087436 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4e94078b90 0x7f4e9407b0a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4ea80046b0 tx=0x7f4ea8009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:35.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.580+0000 7f4eafe00640 1 -- 172.21.15.39:0/426087436 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4ea003f030 con 0x7f4eb80ca5a0 2024-09-21T07:37:35.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.814+0000 7f4ebca00640 1 -- 172.21.15.39:0/426087436 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4eb80d0bf0 con 0x7f4eb80ca5a0 2024-09-21T07:37:35.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.814+0000 7f4eafe00640 1 -- 172.21.15.39:0/426087436 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5249 (secure 0 0 0) 0x7f4ea00845c0 con 0x7f4eb80ca5a0 2024-09-21T07:37:35.816 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:37:35.817 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":74,"btime":"2024-09-21T07:37:33:801850+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5870,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2381504293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2381504293},{"type":"v1","addr":"172.21.15.148:6837","nonce":2381504293}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:37:33.801842+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6585,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6585":{"gid":6585,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6834/1472430787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1472430787},{"type":"v1","addr":"172.21.15.39:6834","nonce":1472430787}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6585,"qdb_cluster":[6115,6585]},"id":6}]} 2024-09-21T07:37:35.817 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 74 2024-09-21T07:37:35.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.817+0000 7f4ebca00640 1 -- 172.21.15.39:0/426087436 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4e94078b90 msgr2=0x7f4e9407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:35.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.817+0000 7f4ebca00640 1 --2- 172.21.15.39:0/426087436 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4e94078b90 0x7f4e9407b0a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4ea80046b0 tx=0x7f4ea8009040 comp rx=0 tx=0).stop 2024-09-21T07:37:35.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.817+0000 7f4ebca00640 1 -- 172.21.15.39:0/426087436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4eb80ca5a0 msgr2=0x7f4eb80afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:35.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.817+0000 7f4ebca00640 1 --2- 172.21.15.39:0/426087436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4eb80ca5a0 0x7f4eb80afb90 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0009950 tx=0x7f4ea00042f0 comp rx=0 tx=0).stop 2024-09-21T07:37:35.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.817+0000 7f4ebca00640 1 -- 172.21.15.39:0/426087436 shutdown_connections 2024-09-21T07:37:35.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.817+0000 7f4ebca00640 1 --2- 172.21.15.39:0/426087436 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4e94078b90 0x7f4e9407b0a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:35.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.817+0000 7f4ebca00640 1 --2- 172.21.15.39:0/426087436 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4eb815c5d0 0x7f4eb80b00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:35.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.817+0000 7f4ebca00640 1 --2- 172.21.15.39:0/426087436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4eb80ca5a0 0x7f4eb80afb90 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:35.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.817+0000 7f4ebca00640 1 --2- 172.21.15.39:0/426087436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4eb80b4b80 0x7f4eb81539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:35.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.817+0000 7f4ebca00640 1 -- 172.21.15.39:0/426087436 >> 172.21.15.39:0/426087436 conn(0x7f4eb806d870 msgr2=0x7f4eb806ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:35.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.817+0000 7f4ebca00640 1 -- 172.21.15.39:0/426087436 shutdown_connections 2024-09-21T07:37:35.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:35.817+0000 7f4ebca00640 1 -- 172.21.15.39:0/426087436 wait complete. 2024-09-21T07:37:35.842 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-21T07:37:35.842 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:37:36.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.069+0000 7f279da00640 1 -- 172.21.15.39:0/2568932485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27980cf200 msgr2=0x7f279815ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:36.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.069+0000 7f279da00640 1 --2- 172.21.15.39:0/2568932485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27980cf200 0x7f279815ba00 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f2798070e50 tx=0x7f278002a560 comp rx=0 tx=0).stop 2024-09-21T07:37:36.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.069+0000 7f279da00640 1 -- 172.21.15.39:0/2568932485 shutdown_connections 2024-09-21T07:37:36.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.069+0000 7f279da00640 1 --2- 172.21.15.39:0/2568932485 >> v2:172.21.15.148:3300/0 conn(0x7f279815bf40 0x7f279815eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:36.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.069+0000 7f279da00640 1 --2- 172.21.15.39:0/2568932485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27980cf200 0x7f279815ba00 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:36.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.069+0000 7f279da00640 1 --2- 172.21.15.39:0/2568932485 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27980d0500 0x7f27980cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:36.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.069+0000 7f279da00640 1 -- 172.21.15.39:0/2568932485 >> 172.21.15.39:0/2568932485 conn(0x7f279806d870 msgr2=0x7f279806ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:36.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.069+0000 7f279da00640 1 -- 172.21.15.39:0/2568932485 shutdown_connections 2024-09-21T07:37:36.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.069+0000 7f279da00640 1 -- 172.21.15.39:0/2568932485 wait complete. 2024-09-21T07:37:36.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.071+0000 7f279da00640 1 Processor -- start 2024-09-21T07:37:36.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.071+0000 7f279da00640 1 -- start start 2024-09-21T07:37:36.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.073+0000 7f279da00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27980cf200 0x7f279815b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:36.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.073+0000 7f279da00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27980d0500 0x7f27980b5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:36.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.073+0000 7f279da00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f279815bf40 0x7f2798140be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:36.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.073+0000 7f279da00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f279815f530 con 0x7f279815bf40 2024-09-21T07:37:36.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.073+0000 7f279da00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f279815c560 con 0x7f27980cf200 2024-09-21T07:37:36.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.073+0000 7f279da00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f279815f6b0 con 0x7f27980d0500 2024-09-21T07:37:36.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.073+0000 7f279d000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f279815bf40 0x7f2798140be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:36.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.073+0000 7f2797e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27980cf200 0x7f279815b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:36.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.073+0000 7f279d000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f279815bf40 0x7f2798140be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40780/0 (socket says 172.21.15.39:40780) 2024-09-21T07:37:36.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.073+0000 7f279d000640 1 -- 172.21.15.39:0/864272376 learned_addr learned my addr 172.21.15.39:0/864272376 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:36.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.073+0000 7f278fe00640 1 --2- 172.21.15.39:0/864272376 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27980d0500 0x7f27980b5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:36.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.074+0000 7f279d000640 1 -- 172.21.15.39:0/864272376 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27980d0500 msgr2=0x7f27980b5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:36.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.074+0000 7f279d000640 1 --2- 172.21.15.39:0/864272376 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27980d0500 0x7f27980b5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:36.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.074+0000 7f279d000640 1 -- 172.21.15.39:0/864272376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27980cf200 msgr2=0x7f279815b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:36.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.074+0000 7f279d000640 1 --2- 172.21.15.39:0/864272376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27980cf200 0x7f279815b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:36.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.074+0000 7f279d000640 1 -- 172.21.15.39:0/864272376 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27981f61e0 con 0x7f279815bf40 2024-09-21T07:37:36.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.074+0000 7f279d000640 1 --2- 172.21.15.39:0/864272376 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f279815bf40 0x7f2798140be0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f279000e960 tx=0x7f279000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:36.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.074+0000 7f2794c00640 1 -- 172.21.15.39:0/864272376 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790004d20 con 0x7f279815bf40 2024-09-21T07:37:36.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.074+0000 7f2794c00640 1 -- 172.21.15.39:0/864272376 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f279000cac0 con 0x7f279815bf40 2024-09-21T07:37:36.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.074+0000 7f2794c00640 1 -- 172.21.15.39:0/864272376 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790005020 con 0x7f279815bf40 2024-09-21T07:37:36.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.074+0000 7f279da00640 1 -- 172.21.15.39:0/864272376 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27981f55d0 con 0x7f279815bf40 2024-09-21T07:37:36.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.074+0000 7f279da00640 1 -- 172.21.15.39:0/864272376 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27981f5b90 con 0x7f279815bf40 2024-09-21T07:37:36.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.075+0000 7f279da00640 1 -- 172.21.15.39:0/864272376 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27980cd7d0 con 0x7f279815bf40 2024-09-21T07:37:36.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.077+0000 7f2794c00640 1 -- 172.21.15.39:0/864272376 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f27900051c0 con 0x7f279815bf40 2024-09-21T07:37:36.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.078+0000 7f2794c00640 1 --2- 172.21.15.39:0/864272376 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2774078bc0 0x7f277407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:36.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.078+0000 7f2794c00640 1 -- 172.21.15.39:0/864272376 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 7001+0+0 (secure 0 0 0) 0x7f279009b8a0 con 0x7f279815bf40 2024-09-21T07:37:36.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.078+0000 7f2797e00640 1 --2- 172.21.15.39:0/864272376 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2774078bc0 0x7f277407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:36.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.079+0000 7f2797e00640 1 --2- 172.21.15.39:0/864272376 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2774078bc0 0x7f277407b0d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f278800bb50 tx=0x7f278800b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:36.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.082+0000 7f2794c00640 1 -- 172.21.15.39:0/864272376 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2790016030 con 0x7f279815bf40 2024-09-21T07:37:36.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.316+0000 7f279da00640 1 -- 172.21.15.39:0/864272376 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f279815fb60 con 0x7f279815bf40 2024-09-21T07:37:36.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.316+0000 7f2794c00640 1 -- 172.21.15.39:0/864272376 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5249 (secure 0 0 0) 0x7f2790063280 con 0x7f279815bf40 2024-09-21T07:37:36.318 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:37:36.318 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":74,"btime":"2024-09-21T07:37:33:801850+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5870,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2381504293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2381504293},{"type":"v1","addr":"172.21.15.148:6837","nonce":2381504293}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:37:33.801842+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6585,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6585":{"gid":6585,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6834/1472430787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1472430787},{"type":"v1","addr":"172.21.15.39:6834","nonce":1472430787}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6585,"qdb_cluster":[6115,6585]},"id":6}]} 2024-09-21T07:37:36.319 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 74 2024-09-21T07:37:36.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.318+0000 7f279da00640 1 -- 172.21.15.39:0/864272376 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2774078bc0 msgr2=0x7f277407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:36.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.318+0000 7f279da00640 1 --2- 172.21.15.39:0/864272376 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2774078bc0 0x7f277407b0d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f278800bb50 tx=0x7f278800b2e0 comp rx=0 tx=0).stop 2024-09-21T07:37:36.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.318+0000 7f279da00640 1 -- 172.21.15.39:0/864272376 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f279815bf40 msgr2=0x7f2798140be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:36.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.318+0000 7f279da00640 1 --2- 172.21.15.39:0/864272376 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f279815bf40 0x7f2798140be0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f279000e960 tx=0x7f279000ee20 comp rx=0 tx=0).stop 2024-09-21T07:37:36.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.319+0000 7f279da00640 1 -- 172.21.15.39:0/864272376 shutdown_connections 2024-09-21T07:37:36.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.319+0000 7f279da00640 1 --2- 172.21.15.39:0/864272376 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2774078bc0 0x7f277407b0d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:36.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.319+0000 7f279da00640 1 --2- 172.21.15.39:0/864272376 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f279815bf40 0x7f2798140be0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:36.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.319+0000 7f279da00640 1 --2- 172.21.15.39:0/864272376 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27980d0500 0x7f27980b5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:36.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.319+0000 7f279da00640 1 --2- 172.21.15.39:0/864272376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27980cf200 0x7f279815b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:36.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.319+0000 7f279da00640 1 -- 172.21.15.39:0/864272376 >> 172.21.15.39:0/864272376 conn(0x7f279806d870 msgr2=0x7f279806ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:36.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.319+0000 7f279da00640 1 -- 172.21.15.39:0/864272376 shutdown_connections 2024-09-21T07:37:36.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.319+0000 7f279da00640 1 -- 172.21.15.39:0/864272376 wait complete. 2024-09-21T07:37:36.404 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-09-21T07:37:36.745 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:36:20.189287+0000', 'modified': '2024-09-21T07:37:33.801842+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 94, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 6585, 'mds_1': 6115}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6115': {'gid': 6115, 'name': 'a', 'rank': 1, 'incarnation': 56, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6835/364657398', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 364657398}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 364657398}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6585': {'gid': 6585, 'name': 'c', 'rank': 0, 'incarnation': 70, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.39:6834/1472430787', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 1472430787}, {'type': 'v1', 'addr': '172.21.15.39:6834', 'nonce': 1472430787}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6585, 'qdb_cluster': [6115, 6585]} 2024-09-21T07:37:36.746 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2024-09-21T07:37:36.746 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T07:37:36.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.975+0000 7fbfbea00640 1 -- 172.21.15.39:0/2152213769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfb815d8a0 msgr2=0x7fbfb80b40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:36.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.975+0000 7fbfbea00640 1 --2- 172.21.15.39:0/2152213769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfb815d8a0 0x7fbfb80b40b0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0005fd0 tx=0x7fbfa002a5d0 comp rx=0 tx=0).stop 2024-09-21T07:37:36.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.975+0000 7fbfbea00640 1 -- 172.21.15.39:0/2152213769 shutdown_connections 2024-09-21T07:37:36.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.975+0000 7fbfbea00640 1 --2- 172.21.15.39:0/2152213769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfb815d8a0 0x7fbfb80b40b0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:36.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.975+0000 7fbfbea00640 1 --2- 172.21.15.39:0/2152213769 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbfb80cc1e0 0x7fbfb80af3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:36.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.975+0000 7fbfbea00640 1 -- 172.21.15.39:0/2152213769 >> 172.21.15.39:0/2152213769 conn(0x7fbfb806d7d0 msgr2=0x7fbfb80b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:36.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.975+0000 7fbfbea00640 1 -- 172.21.15.39:0/2152213769 shutdown_connections 2024-09-21T07:37:36.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.975+0000 7fbfbea00640 1 -- 172.21.15.39:0/2152213769 wait complete. 2024-09-21T07:37:36.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.977+0000 7fbfbea00640 1 Processor -- start 2024-09-21T07:37:36.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.977+0000 7fbfbea00640 1 -- start start 2024-09-21T07:37:36.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.979+0000 7fbfbea00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbfb80cc1e0 0x7fbfb81f7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:36.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.979+0000 7fbfbea00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfb80ccb50 0x7fbfb8154670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:36.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.979+0000 7fbfbea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfb815d8a0 0x7fbfb8148a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:36.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.979+0000 7fbfbea00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb8153520 con 0x7fbfb815d8a0 2024-09-21T07:37:36.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.979+0000 7fbfbea00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb81533a0 con 0x7fbfb80ccb50 2024-09-21T07:37:36.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.979+0000 7fbfbea00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fbfb81536a0 con 0x7fbfb80cc1e0 2024-09-21T07:37:36.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.979+0000 7fbfbca00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfb815d8a0 0x7fbfb8148a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:36.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.979+0000 7fbfbca00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfb815d8a0 0x7fbfb8148a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40790/0 (socket says 172.21.15.39:40790) 2024-09-21T07:37:36.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.979+0000 7fbfbca00640 1 -- 172.21.15.39:0/365336183 learned_addr learned my addr 172.21.15.39:0/365336183 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:36.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.979+0000 7fbfafe00640 1 --2- 172.21.15.39:0/365336183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfb80ccb50 0x7fbfb8154670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:36.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.979+0000 7fbfb7e00640 1 --2- 172.21.15.39:0/365336183 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbfb80cc1e0 0x7fbfb81f7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:36.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.980+0000 7fbfbca00640 1 -- 172.21.15.39:0/365336183 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbfb80cc1e0 msgr2=0x7fbfb81f7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:36.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.980+0000 7fbfbca00640 1 --2- 172.21.15.39:0/365336183 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbfb80cc1e0 0x7fbfb81f7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:36.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.980+0000 7fbfbca00640 1 -- 172.21.15.39:0/365336183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfb80ccb50 msgr2=0x7fbfb8154670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:36.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.980+0000 7fbfbca00640 1 --2- 172.21.15.39:0/365336183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfb80ccb50 0x7fbfb8154670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:36.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.980+0000 7fbfbca00640 1 -- 172.21.15.39:0/365336183 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfb8148f70 con 0x7fbfb815d8a0 2024-09-21T07:37:36.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.980+0000 7fbfb7e00640 1 --2- 172.21.15.39:0/365336183 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbfb80cc1e0 0x7fbfb81f7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:37:36.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.980+0000 7fbfbca00640 1 --2- 172.21.15.39:0/365336183 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfb815d8a0 0x7fbfb8148a30 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fbfb000c760 tx=0x7fbfb000cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:36.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.980+0000 7fbfb4c00640 1 -- 172.21.15.39:0/365336183 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0003b70 con 0x7fbfb815d8a0 2024-09-21T07:37:36.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.980+0000 7fbfb4c00640 1 -- 172.21.15.39:0/365336183 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbfb0003d10 con 0x7fbfb815d8a0 2024-09-21T07:37:36.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.980+0000 7fbfbea00640 1 -- 172.21.15.39:0/365336183 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfb8153fb0 con 0x7fbfb815d8a0 2024-09-21T07:37:36.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.980+0000 7fbfb4c00640 1 -- 172.21.15.39:0/365336183 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0005330 con 0x7fbfb815d8a0 2024-09-21T07:37:36.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.980+0000 7fbfbea00640 1 -- 172.21.15.39:0/365336183 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfb81f7170 con 0x7fbfb815d8a0 2024-09-21T07:37:36.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.983+0000 7fbfb4c00640 1 -- 172.21.15.39:0/365336183 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbfb00054d0 con 0x7fbfb815d8a0 2024-09-21T07:37:36.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.984+0000 7fbfb4c00640 1 --2- 172.21.15.39:0/365336183 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbf94078b90 0x7fbf9407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:36.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.984+0000 7fbfb4c00640 1 -- 172.21.15.39:0/365336183 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 7001+0+0 (secure 0 0 0) 0x7fbfb009afb0 con 0x7fbfb815d8a0 2024-09-21T07:37:36.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.984+0000 7fbfb7e00640 1 --2- 172.21.15.39:0/365336183 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbf94078b90 0x7fbf9407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:36.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.985+0000 7fbfc5a05740 1 Processor -- start 2024-09-21T07:37:36.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.985+0000 7fbfc5a05740 1 -- start start 2024-09-21T07:37:36.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.985+0000 7fbfb7e00640 1 --2- 172.21.15.39:0/365336183 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbf94078b90 0x7fbf9407b0a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8009b80 tx=0x7fbfa8005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:36.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.986+0000 7fbfc5a05740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f406352270 0x55f40673da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:36.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.986+0000 7fbfc5a05740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4063293b0 0x55f4066f2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:36.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.986+0000 7fbfc5a05740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f406331590 0x55f4066f34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:36.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.986+0000 7fbfc5a05740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55f40667c430 con 0x55f4063293b0 2024-09-21T07:37:36.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.986+0000 7fbfc5a05740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f40666c5d0 con 0x55f406331590 2024-09-21T07:37:36.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.986+0000 7fbfc5a05740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55f40665ba00 con 0x55f406352270 2024-09-21T07:37:36.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.986+0000 7fbfafe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4063293b0 0x55f4066f2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:36.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.986+0000 7fbfafe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4063293b0 0x55f4066f2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40806/0 (socket says 172.21.15.39:40806) 2024-09-21T07:37:36.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.986+0000 7fbfafe00640 1 -- 172.21.15.39:0/3405868512 learned_addr learned my addr 172.21.15.39:0/3405868512 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:36.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.986+0000 7fbfb7e00640 1 --2- 172.21.15.39:0/3405868512 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f406352270 0x55f40673da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:36.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.987+0000 7fbfbca00640 1 --2- 172.21.15.39:0/3405868512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f406331590 0x55f4066f34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:36.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.987+0000 7fbfafe00640 1 -- 172.21.15.39:0/3405868512 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f406352270 msgr2=0x55f40673da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:37:36.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.987+0000 7fbfafe00640 1 --2- 172.21.15.39:0/3405868512 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f406352270 0x55f40673da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:36.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.987+0000 7fbfafe00640 1 -- 172.21.15.39:0/3405868512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f406331590 msgr2=0x55f4066f34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:37:36.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.987+0000 7fbfafe00640 1 --2- 172.21.15.39:0/3405868512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f406331590 0x55f4066f34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:36.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.987+0000 7fbfafe00640 1 -- 172.21.15.39:0/3405868512 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f406624520 con 0x55f4063293b0 2024-09-21T07:37:36.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.987+0000 7fbfafe00640 1 --2- 172.21.15.39:0/3405868512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4063293b0 0x55f4066f2f60 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0009230 tx=0x7fbfa00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7be0d255dacc701 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:36.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.987+0000 7fbfa7e00640 1 -- 172.21.15.39:0/3405868512 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa0040070 con 0x55f4063293b0 2024-09-21T07:37:36.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.988+0000 7fbfa7e00640 1 -- 172.21.15.39:0/3405868512 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbfa0004320 con 0x55f4063293b0 2024-09-21T07:37:36.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.988+0000 7fbfa7e00640 1 -- 172.21.15.39:0/3405868512 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa0035040 con 0x55f4063293b0 2024-09-21T07:37:36.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.988+0000 7fbfc5a05740 1 -- 172.21.15.39:0/3405868512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4063293b0 msgr2=0x55f4066f2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:36.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.988+0000 7fbfc5a05740 1 --2- 172.21.15.39:0/3405868512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4063293b0 0x55f4066f2f60 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0009230 tx=0x7fbfa00046c0 comp rx=0 tx=0).stop 2024-09-21T07:37:36.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.988+0000 7fbfc5a05740 1 -- 172.21.15.39:0/3405868512 shutdown_connections 2024-09-21T07:37:36.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.988+0000 7fbfc5a05740 1 --2- 172.21.15.39:0/3405868512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f406331590 0x55f4066f34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:36.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.988+0000 7fbfc5a05740 1 --2- 172.21.15.39:0/3405868512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f4063293b0 0x55f4066f2f60 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:36.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.988+0000 7fbfc5a05740 1 --2- 172.21.15.39:0/3405868512 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f406352270 0x55f40673da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:36.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.988+0000 7fbfc5a05740 1 -- 172.21.15.39:0/3405868512 >> 172.21.15.39:0/3405868512 conn(0x55f4063463c0 msgr2=0x55f4066f8970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:36.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.989+0000 7fbfc5a05740 1 -- 172.21.15.39:0/3405868512 shutdown_connections 2024-09-21T07:37:36.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.989+0000 7fbfc5a05740 1 -- 172.21.15.39:0/3405868512 wait complete. 2024-09-21T07:37:36.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.990+0000 7fbfc5a05740 1 -- start start 2024-09-21T07:37:36.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.990+0000 7fbfc5a05740 1 Processor -- start 2024-09-21T07:37:36.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.991+0000 7fbfc5a05740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f4063293b0 0x55f406754ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:36.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.991+0000 7fbfc5a05740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f406331590 0x55f406740680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:36.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.991+0000 7fbfc5a05740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f406352270 0x55f406751d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:36.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.991+0000 7fbfc5a05740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55f406602330 con 0x55f406331590 2024-09-21T07:37:36.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.991+0000 7fbfc5a05740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f40665f810 con 0x55f406352270 2024-09-21T07:37:36.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.991+0000 7fbfc5a05740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55f4063cc620 con 0x55f4063293b0 2024-09-21T07:37:36.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.991+0000 7fbfc5a05740 10 client.0 _ll_register_callbacks cb 0x55f406607420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:37:36.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.992+0000 7fbfafe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f406331590 0x55f406740680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:36.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.992+0000 7fbfafe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f406331590 0x55f406740680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40808/0 (socket says 172.21.15.39:40808) 2024-09-21T07:37:36.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.992+0000 7fbfafe00640 1 -- 172.21.15.39:0/63562433 learned_addr learned my addr 172.21.15.39:0/63562433 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:36.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.992+0000 7fbfafe00640 1 -- 172.21.15.39:0/63562433 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f4063293b0 msgr2=0x55f406754ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:37:36.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.992+0000 7fbfafe00640 1 --2- 172.21.15.39:0/63562433 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f4063293b0 0x55f406754ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:36.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.992+0000 7fbfafe00640 1 -- 172.21.15.39:0/63562433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f406352270 msgr2=0x55f406751d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:37:36.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.992+0000 7fbfbca00640 1 --2- 172.21.15.39:0/63562433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f406352270 0x55f406751d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:36.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.992+0000 7fbfafe00640 1 --2- 172.21.15.39:0/63562433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f406352270 0x55f406751d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:36.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.992+0000 7fbfafe00640 1 -- 172.21.15.39:0/63562433 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4063bd850 con 0x55f406331590 2024-09-21T07:37:36.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.992+0000 7fbfafe00640 1 --2- 172.21.15.39:0/63562433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f406331590 0x55f406740680 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fbfa00099f0 tx=0x7fbfa00044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e132bd5f8f96553d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:36.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.992+0000 7fbfa4c00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:37:36.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.993+0000 7fbfa4c00640 1 -- 172.21.15.39:0/63562433 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa0035030 con 0x55f406331590 2024-09-21T07:37:36.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.993+0000 7fbfa4c00640 1 -- 172.21.15.39:0/63562433 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 7001+0+0 (secure 0 0 0) 0x7fbfa00037f0 con 0x55f406331590 2024-09-21T07:37:36.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.993+0000 7fbfc5a05740 1 -- 172.21.15.39:0/63562433 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfa0004320 con 0x55f406331590 2024-09-21T07:37:36.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.993+0000 7fbfa4c00640 1 -- 172.21.15.39:0/63562433 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7fbfa0052050 con 0x55f406331590 2024-09-21T07:37:36.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.993+0000 7fbfc5a05740 10 client.6732 fetch_fsmap learned FSMap version 74 2024-09-21T07:37:36.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.993+0000 7fbfc5a05740 1 -- 172.21.15.39:0/63562433 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55f4064346b0 con 0x55f406331590 2024-09-21T07:37:36.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.993+0000 7fbfa4c00640 1 -- 172.21.15.39:0/63562433 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbfa0004320 con 0x55f406331590 2024-09-21T07:37:36.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.993+0000 7fbfa4c00640 1 -- 172.21.15.39:0/63562433 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 74) ==== 3522+0+0 (secure 0 0 0) 0x7fbfa0055020 con 0x55f406331590 2024-09-21T07:37:36.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.993+0000 7fbfc5a05740 10 client.6732 fetch_fsmap finished waiting for FSMap version 74 2024-09-21T07:37:36.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.993+0000 7fbfc5a05740 10 client.6732 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:37:36.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.993+0000 7fbfc5a05740 20 client.6732 populate_metadata read hostname 'smithi039' 2024-09-21T07:37:36.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.993+0000 7fbfc5a05740 1 --2- 172.21.15.39:0/63562433 >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x55f406310e40 0x55f40673e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:36.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.993+0000 7fbfc5a05740 4 client.6732 mds_command: new command op to 6115 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:37:36.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.993+0000 7fbfc5a05740 1 -- 172.21.15.39:0/63562433 --> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f40666e520 con 0x55f406310e40 2024-09-21T07:37:36.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.994+0000 7fbfb7e00640 1 --2- 172.21.15.39:0/63562433 >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x55f406310e40 0x55f40673e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:36.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.994+0000 7fbfb7e00640 1 --2- 172.21.15.39:0/63562433 >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x55f406310e40 0x55f40673e820 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1891543ef81ef210 server_cookie=92c6302cd7f3561b in_seq=0 out_seq=0 2024-09-21T07:37:36.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.994+0000 7fbfa4c00640 10 client.6732 ms_handle_connect on v2:172.21.15.39:6833/364657398 2024-09-21T07:37:36.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.995+0000 7fbfa4c00640 1 -- 172.21.15.39:0/63562433 <== mds.1 v2:172.21.15.39:6833/364657398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f40666e520 con 0x55f406310e40 2024-09-21T07:37:36.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.995+0000 7fbfa4c00640 10 client.6732 handle_command_reply: tid=0 2024-09-21T07:37:36.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.995+0000 7fbfc5a05740 1 client.6732 shutdown 2024-09-21T07:37:36.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.995+0000 7fbfc5a05740 1 -- 172.21.15.39:0/63562433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f406331590 msgr2=0x55f406740680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:36.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.995+0000 7fbfc5a05740 1 --2- 172.21.15.39:0/63562433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f406331590 0x55f406740680 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fbfa00099f0 tx=0x7fbfa00044d0 comp rx=0 tx=0).stop 2024-09-21T07:37:36.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.995+0000 7fbfc5a05740 1 -- 172.21.15.39:0/63562433 shutdown_connections 2024-09-21T07:37:36.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.995+0000 7fbfc5a05740 1 --2- 172.21.15.39:0/63562433 >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x55f406310e40 0x55f40673e820 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.995+0000 7fbfc5a05740 1 --2- 172.21.15.39:0/63562433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f406352270 0x55f406751d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.995+0000 7fbfc5a05740 1 --2- 172.21.15.39:0/63562433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f406331590 0x55f406740680 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.995+0000 7fbfc5a05740 1 --2- 172.21.15.39:0/63562433 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f4063293b0 0x55f406754ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.995+0000 7fbfc5a05740 1 -- 172.21.15.39:0/63562433 >> 172.21.15.39:0/63562433 conn(0x55f4063463c0 msgr2=0x55f4066f8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:37.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.995+0000 7fbfa4c00640 0 client.6732 ms_handle_reset on v2:172.21.15.39:6833/364657398 2024-09-21T07:37:37.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.995+0000 7fbfc5a05740 1 -- 172.21.15.39:0/63562433 shutdown_connections 2024-09-21T07:37:37.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.995+0000 7fbfc5a05740 1 -- 172.21.15.39:0/63562433 wait complete. 2024-09-21T07:37:37.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:36.996+0000 7fbfc5a05740 20 client.6732 trim_cache size 0 max 16384 2024-09-21T07:37:37.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.015+0000 7fbfc5a05740 1 Processor -- start 2024-09-21T07:37:37.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.015+0000 7fbfc5a05740 1 -- start start 2024-09-21T07:37:37.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.016+0000 7fbfc5a05740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f4063293b0 0x55f406733da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:37.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.016+0000 7fbfc5a05740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f406331590 0x55f406721770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:37.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.016+0000 7fbfc5a05740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f406352270 0x55f406721cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:37.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.016+0000 7fbfc5a05740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb82006c0 con 0x55f406352270 2024-09-21T07:37:37.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.016+0000 7fbfc5a05740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb8074e10 con 0x55f406331590 2024-09-21T07:37:37.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.016+0000 7fbfc5a05740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fbfb8152740 con 0x55f4063293b0 2024-09-21T07:37:37.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.017+0000 7fbfbca00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f406352270 0x55f406721cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:37.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.017+0000 7fbfbca00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f406352270 0x55f406721cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40824/0 (socket says 172.21.15.39:40824) 2024-09-21T07:37:37.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.017+0000 7fbfbca00640 1 -- 172.21.15.39:0/3959092328 learned_addr learned my addr 172.21.15.39:0/3959092328 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:37.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.017+0000 7fbfafe00640 1 --2- 172.21.15.39:0/3959092328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f406331590 0x55f406721770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:37.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.017+0000 7fbfb7e00640 1 --2- 172.21.15.39:0/3959092328 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f4063293b0 0x55f406733da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:37.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.017+0000 7fbfbca00640 1 -- 172.21.15.39:0/3959092328 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f4063293b0 msgr2=0x55f406733da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:37:37.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.017+0000 7fbfbca00640 1 --2- 172.21.15.39:0/3959092328 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f4063293b0 0x55f406733da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.017+0000 7fbfbca00640 1 -- 172.21.15.39:0/3959092328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f406331590 msgr2=0x55f406721770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:37:37.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.017+0000 7fbfbca00640 1 --2- 172.21.15.39:0/3959092328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f406331590 0x55f406721770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.017+0000 7fbfbca00640 1 -- 172.21.15.39:0/3959092328 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4066f63d0 con 0x55f406352270 2024-09-21T07:37:37.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.017+0000 7fbfafe00640 1 --2- 172.21.15.39:0/3959092328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f406331590 0x55f406721770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:37:37.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.017+0000 7fbfb7e00640 1 --2- 172.21.15.39:0/3959092328 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f4063293b0 0x55f406733da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:37:37.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.017+0000 7fbfbca00640 1 --2- 172.21.15.39:0/3959092328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f406352270 0x55f406721cb0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0098f30 tx=0x7fbfb0000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=957e2b9946ca8bc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:37.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.018+0000 7fbfa7400640 1 -- 172.21.15.39:0/3959092328 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0088070 con 0x55f406352270 2024-09-21T07:37:37.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.018+0000 7fbfa7400640 1 -- 172.21.15.39:0/3959092328 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbfb0062740 con 0x55f406352270 2024-09-21T07:37:37.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.018+0000 7fbfa7400640 1 -- 172.21.15.39:0/3959092328 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0067220 con 0x55f406352270 2024-09-21T07:37:37.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.018+0000 7fbfc5a05740 1 -- 172.21.15.39:0/3959092328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f406352270 msgr2=0x55f406721cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:37.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.018+0000 7fbfc5a05740 1 --2- 172.21.15.39:0/3959092328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f406352270 0x55f406721cb0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0098f30 tx=0x7fbfb0000f40 comp rx=0 tx=0).stop 2024-09-21T07:37:37.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.018+0000 7fbfc5a05740 1 -- 172.21.15.39:0/3959092328 shutdown_connections 2024-09-21T07:37:37.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.018+0000 7fbfc5a05740 1 --2- 172.21.15.39:0/3959092328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f406352270 0x55f406721cb0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.018+0000 7fbfc5a05740 1 --2- 172.21.15.39:0/3959092328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f406331590 0x55f406721770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.018+0000 7fbfc5a05740 1 --2- 172.21.15.39:0/3959092328 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f4063293b0 0x55f406733da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.018+0000 7fbfc5a05740 1 -- 172.21.15.39:0/3959092328 >> 172.21.15.39:0/3959092328 conn(0x55f4063463c0 msgr2=0x55f40673de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:37.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.018+0000 7fbfc5a05740 1 -- 172.21.15.39:0/3959092328 shutdown_connections 2024-09-21T07:37:37.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.018+0000 7fbfc5a05740 1 -- 172.21.15.39:0/3959092328 wait complete. 2024-09-21T07:37:37.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.019+0000 7fbfc5a05740 1 -- start start 2024-09-21T07:37:37.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.020+0000 7fbfc5a05740 1 Processor -- start 2024-09-21T07:37:37.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.021+0000 7fbfc5a05740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4063293b0 0x55f4067b8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:37.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.021+0000 7fbfc5a05740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f406331590 0x55f4066f1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:37.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.021+0000 7fbfc5a05740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f406352270 0x55f4066f2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:37.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.021+0000 7fbfc5a05740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55f406722f70 con 0x55f406331590 2024-09-21T07:37:37.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.021+0000 7fbfc5a05740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f406602330 con 0x55f4063293b0 2024-09-21T07:37:37.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.021+0000 7fbfc5a05740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55f406738d50 con 0x55f406352270 2024-09-21T07:37:37.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.021+0000 7fbfc5a05740 10 client.0 _ll_register_callbacks cb 0x7fbfa00053a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:37:37.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.021+0000 7fbfafe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f406331590 0x55f4066f1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:37.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.021+0000 7fbfafe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f406331590 0x55f4066f1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40828/0 (socket says 172.21.15.39:40828) 2024-09-21T07:37:37.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.021+0000 7fbfafe00640 1 -- 172.21.15.39:0/3788685869 learned_addr learned my addr 172.21.15.39:0/3788685869 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:37.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.022+0000 7fbfb7e00640 1 --2- 172.21.15.39:0/3788685869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4063293b0 0x55f4067b8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:37.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.022+0000 7fbfbca00640 1 --2- 172.21.15.39:0/3788685869 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f406352270 0x55f4066f2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:37.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.022+0000 7fbfafe00640 1 -- 172.21.15.39:0/3788685869 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f406352270 msgr2=0x55f4066f2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:37:37.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.022+0000 7fbfafe00640 1 --2- 172.21.15.39:0/3788685869 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f406352270 0x55f4066f2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.022+0000 7fbfafe00640 1 -- 172.21.15.39:0/3788685869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4063293b0 msgr2=0x55f4067b8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:37:37.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.022+0000 7fbfafe00640 1 --2- 172.21.15.39:0/3788685869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4063293b0 0x55f4067b8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.022+0000 7fbfafe00640 1 -- 172.21.15.39:0/3788685869 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f406721e40 con 0x55f406331590 2024-09-21T07:37:37.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.022+0000 7fbfafe00640 1 --2- 172.21.15.39:0/3788685869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f406331590 0x55f4066f1f00 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0006010 tx=0x7fbfa0005550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b3a84d0f0e2070e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:37.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.022+0000 7fbfa4c00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:37:37.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.022+0000 7fbfa4c00640 1 -- 172.21.15.39:0/3788685869 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa005b060 con 0x55f406331590 2024-09-21T07:37:37.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.022+0000 7fbfa4c00640 1 -- 172.21.15.39:0/3788685869 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 7001+0+0 (secure 0 0 0) 0x7fbfa0004930 con 0x55f406331590 2024-09-21T07:37:37.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.022+0000 7fbfc5a05740 1 -- 172.21.15.39:0/3788685869 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfb0062740 con 0x55f406331590 2024-09-21T07:37:37.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.023+0000 7fbfa4c00640 1 -- 172.21.15.39:0/3788685869 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7fbfa0061050 con 0x55f406331590 2024-09-21T07:37:37.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.023+0000 7fbfc5a05740 10 client.6738 fetch_fsmap learned FSMap version 74 2024-09-21T07:37:37.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.023+0000 7fbfc5a05740 1 -- 172.21.15.39:0/3788685869 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55f4067b8d50 con 0x55f406331590 2024-09-21T07:37:37.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.023+0000 7fbfa4c00640 1 -- 172.21.15.39:0/3788685869 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbfb0062740 con 0x55f406331590 2024-09-21T07:37:37.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.023+0000 7fbfa4c00640 1 -- 172.21.15.39:0/3788685869 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 74) ==== 3522+0+0 (secure 0 0 0) 0x7fbfa0064020 con 0x55f406331590 2024-09-21T07:37:37.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.023+0000 7fbfc5a05740 10 client.6738 fetch_fsmap finished waiting for FSMap version 74 2024-09-21T07:37:37.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.023+0000 7fbfc5a05740 10 client.6738 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:37:37.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.023+0000 7fbfc5a05740 20 client.6738 populate_metadata read hostname 'smithi039' 2024-09-21T07:37:37.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.023+0000 7fbfc5a05740 1 --2- 172.21.15.39:0/3788685869 >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x55f4067c5ea0 0x55f406736790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:37.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.023+0000 7fbfc5a05740 4 client.6738 mds_command: new command op to 6115 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:37:37.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.023+0000 7fbfc5a05740 1 -- 172.21.15.39:0/3788685869 --> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] -- command(tid 0: {"prefix": "status"}) -- 0x55f406669fd0 con 0x55f4067c5ea0 2024-09-21T07:37:37.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.023+0000 7fbfb7e00640 1 --2- 172.21.15.39:0/3788685869 >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x55f4067c5ea0 0x55f406736790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:37.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.024+0000 7fbfb7e00640 1 --2- 172.21.15.39:0/3788685869 >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x55f4067c5ea0 0x55f406736790 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7284a2e0f9b1f5c0 server_cookie=de91efff84029325 in_seq=0 out_seq=0 2024-09-21T07:37:37.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.024+0000 7fbfa4c00640 10 client.6738 ms_handle_connect on v2:172.21.15.39:6833/364657398 2024-09-21T07:37:37.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.024+0000 7fbfa4c00640 1 -- 172.21.15.39:0/3788685869 <== mds.1 v2:172.21.15.39:6833/364657398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55f406669fd0 con 0x55f4067c5ea0 2024-09-21T07:37:37.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.024+0000 7fbfa4c00640 10 client.6738 handle_command_reply: tid=0 2024-09-21T07:37:37.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.024+0000 7fbfc5a05740 1 client.6738 shutdown 2024-09-21T07:37:37.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.024+0000 7fbfc5a05740 1 -- 172.21.15.39:0/3788685869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f406331590 msgr2=0x55f4066f1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:37.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.024+0000 7fbfc5a05740 1 --2- 172.21.15.39:0/3788685869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f406331590 0x55f4066f1f00 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0006010 tx=0x7fbfa0005550 comp rx=0 tx=0).stop 2024-09-21T07:37:37.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.025+0000 7fbfc5a05740 1 -- 172.21.15.39:0/3788685869 shutdown_connections 2024-09-21T07:37:37.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.025+0000 7fbfc5a05740 1 --2- 172.21.15.39:0/3788685869 >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x55f4067c5ea0 0x55f406736790 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.025+0000 7fbfc5a05740 1 --2- 172.21.15.39:0/3788685869 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f406352270 0x55f4066f2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.025+0000 7fbfc5a05740 1 --2- 172.21.15.39:0/3788685869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f406331590 0x55f4066f1f00 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.025+0000 7fbfa4c00640 0 client.6738 ms_handle_reset on v2:172.21.15.39:6833/364657398 2024-09-21T07:37:37.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.025+0000 7fbfc5a05740 1 --2- 172.21.15.39:0/3788685869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4063293b0 0x55f4067b8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.025+0000 7fbfc5a05740 1 -- 172.21.15.39:0/3788685869 >> 172.21.15.39:0/3788685869 conn(0x55f4063463c0 msgr2=0x55f406754640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:37.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.025+0000 7fbfc5a05740 1 -- 172.21.15.39:0/3788685869 shutdown_connections 2024-09-21T07:37:37.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.025+0000 7fbfc5a05740 1 -- 172.21.15.39:0/3788685869 wait complete. 2024-09-21T07:37:37.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.025+0000 7fbfc5a05740 20 client.6738 trim_cache size 0 max 16384 2024-09-21T07:37:37.041 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:37:37.041 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:37:37.041 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 1, 2024-09-21T07:37:37.041 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 6115, 2024-09-21T07:37:37.042 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:37:37.042 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:37:37.042 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:37:37.042 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 65.661381270999996, 2024-09-21T07:37:37.042 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 74, 2024-09-21T07:37:37.042 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 94, 2024-09-21T07:37:37.043 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 94, 2024-09-21T07:37:37.043 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 81.336064026000003 2024-09-21T07:37:37.043 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:37:37.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.025+0000 7fbfbea00640 1 -- 172.21.15.39:0/365336183 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbf94078b90 msgr2=0x7fbf9407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:37.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.025+0000 7fbfbea00640 1 --2- 172.21.15.39:0/365336183 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbf94078b90 0x7fbf9407b0a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8009b80 tx=0x7fbfa8005f90 comp rx=0 tx=0).stop 2024-09-21T07:37:37.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.026+0000 7fbfbea00640 1 -- 172.21.15.39:0/365336183 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfb815d8a0 msgr2=0x7fbfb8148a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:37.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.026+0000 7fbfbea00640 1 --2- 172.21.15.39:0/365336183 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfb815d8a0 0x7fbfb8148a30 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fbfb000c760 tx=0x7fbfb000cb60 comp rx=0 tx=0).stop 2024-09-21T07:37:37.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.026+0000 7fbfbea00640 1 -- 172.21.15.39:0/365336183 shutdown_connections 2024-09-21T07:37:37.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.026+0000 7fbfbea00640 1 --2- 172.21.15.39:0/365336183 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbf94078b90 0x7fbf9407b0a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.026+0000 7fbfbea00640 1 --2- 172.21.15.39:0/365336183 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfb815d8a0 0x7fbfb8148a30 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.026+0000 7fbfbea00640 1 --2- 172.21.15.39:0/365336183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfb80ccb50 0x7fbfb8154670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.026+0000 7fbfbea00640 1 --2- 172.21.15.39:0/365336183 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbfb80cc1e0 0x7fbfb81f7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.026+0000 7fbfbea00640 1 -- 172.21.15.39:0/365336183 >> 172.21.15.39:0/365336183 conn(0x7fbfb806d7d0 msgr2=0x7fbfb806f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:37.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.026+0000 7fbfbea00640 1 -- 172.21.15.39:0/365336183 shutdown_connections 2024-09-21T07:37:37.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.026+0000 7fbfbea00640 1 -- 172.21.15.39:0/365336183 wait complete. 2024-09-21T07:37:37.053 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T07:37:37.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.279+0000 7efe7b800640 1 -- 172.21.15.39:0/3256786324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe74070030 msgr2=0x7efe74074640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:37.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.279+0000 7efe7b800640 1 --2- 172.21.15.39:0/3256786324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe74070030 0x7efe74074640 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7efe60005fd0 tx=0x7efe6002a670 comp rx=0 tx=0).stop 2024-09-21T07:37:37.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.279+0000 7efe7b800640 1 -- 172.21.15.39:0/3256786324 shutdown_connections 2024-09-21T07:37:37.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.279+0000 7efe7b800640 1 --2- 172.21.15.39:0/3256786324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe74070030 0x7efe74074640 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.279+0000 7efe7b800640 1 -- 172.21.15.39:0/3256786324 >> 172.21.15.39:0/3256786324 conn(0x7efe740b1cb0 msgr2=0x7efe740b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:37.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.280+0000 7efe7b800640 1 -- 172.21.15.39:0/3256786324 shutdown_connections 2024-09-21T07:37:37.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.280+0000 7efe7b800640 1 -- 172.21.15.39:0/3256786324 wait complete. 2024-09-21T07:37:37.283 INFO:tasks.ceph.mds.c.smithi039.stderr:2024-09-21T07:37:37.280+0000 7ff517c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:37:37.283 INFO:tasks.ceph.mds.c.smithi039.stderr:2024-09-21T07:37:37.280+0000 7ff517c00640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:37:37.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.282+0000 7efe7b800640 1 Processor -- start 2024-09-21T07:37:37.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.282+0000 7efe7b800640 1 -- start start 2024-09-21T07:37:37.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.283+0000 7efe7b800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe74070030 0x7efe7409de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:37.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.283+0000 7efe7b800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe74070440 0x7efe7414b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:37.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.283+0000 7efe7b800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe74073170 0x7efe7415a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:37.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.283+0000 7efe7b800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efe740b3c40 con 0x7efe74070030 2024-09-21T07:37:37.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.284+0000 7efe7b800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efe740b3ac0 con 0x7efe74073170 2024-09-21T07:37:37.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.284+0000 7efe7b800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7efe740b3dc0 con 0x7efe74070440 2024-09-21T07:37:37.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.284+0000 7efe78e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe74070030 0x7efe7409de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:37.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.284+0000 7efe78e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe74070030 0x7efe7409de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40848/0 (socket says 172.21.15.39:40848) 2024-09-21T07:37:37.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.284+0000 7efe78e00640 1 -- 172.21.15.39:0/2909280569 learned_addr learned my addr 172.21.15.39:0/2909280569 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:37.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.284+0000 7efe73e00640 1 --2- 172.21.15.39:0/2909280569 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe74070440 0x7efe7414b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:37.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.284+0000 7efe79800640 1 --2- 172.21.15.39:0/2909280569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe74073170 0x7efe7415a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:37.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.284+0000 7efe78e00640 1 -- 172.21.15.39:0/2909280569 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe74070440 msgr2=0x7efe7414b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:37.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.284+0000 7efe78e00640 1 --2- 172.21.15.39:0/2909280569 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe74070440 0x7efe7414b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.284+0000 7efe78e00640 1 -- 172.21.15.39:0/2909280569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe74073170 msgr2=0x7efe7415a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:37.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.284+0000 7efe78e00640 1 --2- 172.21.15.39:0/2909280569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe74073170 0x7efe7415a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.284+0000 7efe78e00640 1 -- 172.21.15.39:0/2909280569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe7409ada0 con 0x7efe74070030 2024-09-21T07:37:37.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.285+0000 7efe78e00640 1 --2- 172.21.15.39:0/2909280569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe74070030 0x7efe7409de30 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7efe6800c400 tx=0x7efe6800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:37.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.285+0000 7efe70c00640 1 -- 172.21.15.39:0/2909280569 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe68017070 con 0x7efe74070030 2024-09-21T07:37:37.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.285+0000 7efe70c00640 1 -- 172.21.15.39:0/2909280569 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe680053c0 con 0x7efe74070030 2024-09-21T07:37:37.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.285+0000 7efe7b800640 1 -- 172.21.15.39:0/2909280569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe7409b080 con 0x7efe74070030 2024-09-21T07:37:37.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.285+0000 7efe70c00640 1 -- 172.21.15.39:0/2909280569 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe68003850 con 0x7efe74070030 2024-09-21T07:37:37.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.285+0000 7efe7b800640 1 -- 172.21.15.39:0/2909280569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe7409b640 con 0x7efe74070030 2024-09-21T07:37:37.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.287+0000 7efe70c00640 1 -- 172.21.15.39:0/2909280569 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7efe6801a070 con 0x7efe74070030 2024-09-21T07:37:37.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.288+0000 7efe70c00640 1 --2- 172.21.15.39:0/2909280569 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efe54078bc0 0x7efe5407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:37.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.289+0000 7efe70c00640 1 -- 172.21.15.39:0/2909280569 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 7001+0+0 (secure 0 0 0) 0x7efe6809a5f0 con 0x7efe74070030 2024-09-21T07:37:37.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.289+0000 7efe82847740 1 Processor -- start 2024-09-21T07:37:37.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.289+0000 7efe73e00640 1 --2- 172.21.15.39:0/2909280569 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efe54078bc0 0x7efe5407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:37.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.289+0000 7efe82847740 1 -- start start 2024-09-21T07:37:37.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.290+0000 7efe73e00640 1 --2- 172.21.15.39:0/2909280569 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efe54078bc0 0x7efe5407b0d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7efe60033000 tx=0x7efe60004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:37.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.290+0000 7efe82847740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645695f7270 0x5645699e2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:37.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.290+0000 7efe82847740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5645695ce3b0 0x564569997f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:37.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.290+0000 7efe82847740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5645695d6590 0x5645699984a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:37.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.290+0000 7efe82847740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564569921430 con 0x5645695f7270 2024-09-21T07:37:37.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.290+0000 7efe82847740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5645699115d0 con 0x5645695d6590 2024-09-21T07:37:37.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.290+0000 7efe82847740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x564569900a00 con 0x5645695ce3b0 2024-09-21T07:37:37.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.291+0000 7efe78e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645695f7270 0x5645699e2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:37.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.291+0000 7efe78e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645695f7270 0x5645699e2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40860/0 (socket says 172.21.15.39:40860) 2024-09-21T07:37:37.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.291+0000 7efe78e00640 1 -- 172.21.15.39:0/2565550136 learned_addr learned my addr 172.21.15.39:0/2565550136 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:37.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.291+0000 7efe73e00640 1 --2- 172.21.15.39:0/2565550136 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5645695ce3b0 0x564569997f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:37.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.291+0000 7efe79800640 1 --2- 172.21.15.39:0/2565550136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5645695d6590 0x5645699984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:37.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.291+0000 7efe78e00640 1 -- 172.21.15.39:0/2565550136 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5645695ce3b0 msgr2=0x564569997f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:37:37.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.291+0000 7efe78e00640 1 --2- 172.21.15.39:0/2565550136 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5645695ce3b0 0x564569997f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.291+0000 7efe78e00640 1 -- 172.21.15.39:0/2565550136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5645695d6590 msgr2=0x5645699984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:37:37.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.291+0000 7efe78e00640 1 --2- 172.21.15.39:0/2565550136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5645695d6590 0x5645699984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.291+0000 7efe78e00640 1 -- 172.21.15.39:0/2565550136 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645698c9520 con 0x5645695f7270 2024-09-21T07:37:37.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.291+0000 7efe78e00640 1 --2- 172.21.15.39:0/2565550136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645695f7270 0x5645699e2a20 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7efe68061dc0 tx=0x7efe680616a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6073efff358919e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:37.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.292+0000 7efe65600640 1 -- 172.21.15.39:0/2565550136 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe68066860 con 0x5645695f7270 2024-09-21T07:37:37.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.292+0000 7efe65600640 1 -- 172.21.15.39:0/2565550136 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe6806b7a0 con 0x5645695f7270 2024-09-21T07:37:37.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.292+0000 7efe65600640 1 -- 172.21.15.39:0/2565550136 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe68069be0 con 0x5645695f7270 2024-09-21T07:37:37.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.292+0000 7efe82847740 1 -- 172.21.15.39:0/2565550136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645695f7270 msgr2=0x5645699e2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:37.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.292+0000 7efe82847740 1 --2- 172.21.15.39:0/2565550136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645695f7270 0x5645699e2a20 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7efe68061dc0 tx=0x7efe680616a0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.292+0000 7efe82847740 1 -- 172.21.15.39:0/2565550136 shutdown_connections 2024-09-21T07:37:37.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.292+0000 7efe82847740 1 --2- 172.21.15.39:0/2565550136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5645695d6590 0x5645699984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.292+0000 7efe82847740 1 --2- 172.21.15.39:0/2565550136 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5645695ce3b0 0x564569997f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.292+0000 7efe82847740 1 --2- 172.21.15.39:0/2565550136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645695f7270 0x5645699e2a20 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.292+0000 7efe82847740 1 -- 172.21.15.39:0/2565550136 >> 172.21.15.39:0/2565550136 conn(0x5645695eb3c0 msgr2=0x56456999d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:37.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.293+0000 7efe82847740 1 -- 172.21.15.39:0/2565550136 shutdown_connections 2024-09-21T07:37:37.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.293+0000 7efe82847740 1 -- 172.21.15.39:0/2565550136 wait complete. 2024-09-21T07:37:37.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.294+0000 7efe82847740 1 -- start start 2024-09-21T07:37:37.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.294+0000 7efe82847740 1 Processor -- start 2024-09-21T07:37:37.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.295+0000 7efe82847740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5645695ce3b0 0x5645699dbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:37.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.295+0000 7efe82847740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5645695d6590 0x5645699c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:37.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.295+0000 7efe82847740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645695f7270 0x5645699b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:37.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.295+0000 7efe82847740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5645698a7330 con 0x5645695f7270 2024-09-21T07:37:37.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.295+0000 7efe82847740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x564569904810 con 0x5645695d6590 2024-09-21T07:37:37.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.295+0000 7efe82847740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x564569671620 con 0x5645695ce3b0 2024-09-21T07:37:37.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.295+0000 7efe82847740 10 client.0 _ll_register_callbacks cb 0x5645698ac420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:37:37.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.296+0000 7efe79800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645695f7270 0x5645699b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:37.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.296+0000 7efe79800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645695f7270 0x5645699b1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40872/0 (socket says 172.21.15.39:40872) 2024-09-21T07:37:37.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.296+0000 7efe79800640 1 -- 172.21.15.39:0/3406251349 learned_addr learned my addr 172.21.15.39:0/3406251349 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:37.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.296+0000 7efe78e00640 1 --2- 172.21.15.39:0/3406251349 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5645695ce3b0 0x5645699dbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:37.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.296+0000 7efe73e00640 1 --2- 172.21.15.39:0/3406251349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5645695d6590 0x5645699c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:37.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.296+0000 7efe79800640 1 -- 172.21.15.39:0/3406251349 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5645695ce3b0 msgr2=0x5645699dbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:37:37.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.296+0000 7efe79800640 1 --2- 172.21.15.39:0/3406251349 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5645695ce3b0 0x5645699dbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.296+0000 7efe79800640 1 -- 172.21.15.39:0/3406251349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5645695d6590 msgr2=0x5645699c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:37:37.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.296+0000 7efe79800640 1 --2- 172.21.15.39:0/3406251349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5645695d6590 0x5645699c7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.296+0000 7efe79800640 1 -- 172.21.15.39:0/3406251349 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564569662850 con 0x5645695f7270 2024-09-21T07:37:37.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.296+0000 7efe78e00640 1 --2- 172.21.15.39:0/3406251349 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5645695ce3b0 0x5645699dbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:37:37.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.296+0000 7efe73e00640 1 --2- 172.21.15.39:0/3406251349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5645695d6590 0x5645699c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:37:37.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.296+0000 7efe79800640 1 --2- 172.21.15.39:0/3406251349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645695f7270 0x5645699b1b10 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7efe6c0056f0 tx=0x7efe6c001c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=348c76f111786ddd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:37.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.296+0000 7efe5a000640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:37:37.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.297+0000 7efe5a000640 1 -- 172.21.15.39:0/3406251349 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c006b30 con 0x5645695f7270 2024-09-21T07:37:37.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.297+0000 7efe5a000640 1 -- 172.21.15.39:0/3406251349 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 7001+0+0 (secure 0 0 0) 0x7efe6c010040 con 0x5645695f7270 2024-09-21T07:37:37.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.297+0000 7efe82847740 1 -- 172.21.15.39:0/3406251349 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe6806b7a0 con 0x5645695f7270 2024-09-21T07:37:37.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.297+0000 7efe5a000640 1 -- 172.21.15.39:0/3406251349 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7efe6c006cd0 con 0x5645695f7270 2024-09-21T07:37:37.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.297+0000 7efe82847740 10 client.6753 fetch_fsmap learned FSMap version 74 2024-09-21T07:37:37.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.297+0000 7efe82847740 1 -- 172.21.15.39:0/3406251349 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x5645696d96b0 con 0x5645695f7270 2024-09-21T07:37:37.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.297+0000 7efe5a000640 1 -- 172.21.15.39:0/3406251349 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe6806b7a0 con 0x5645695f7270 2024-09-21T07:37:37.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.297+0000 7efe5a000640 1 -- 172.21.15.39:0/3406251349 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 74) ==== 3522+0+0 (secure 0 0 0) 0x7efe6c0145e0 con 0x5645695f7270 2024-09-21T07:37:37.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.297+0000 7efe82847740 10 client.6753 fetch_fsmap finished waiting for FSMap version 74 2024-09-21T07:37:37.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.297+0000 7efe82847740 10 client.6753 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:37:37.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.297+0000 7efe82847740 20 client.6753 populate_metadata read hostname 'smithi039' 2024-09-21T07:37:37.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.297+0000 7efe82847740 1 --2- 172.21.15.39:0/3406251349 >> [v2:172.21.15.39:6832/1472430787,v1:172.21.15.39:6834/1472430787] conn(0x5645695b5e40 0x5645699f5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:37.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.297+0000 7efe82847740 4 client.6753 mds_command: new command op to 6585 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:37:37.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.297+0000 7efe82847740 1 -- 172.21.15.39:0/3406251349 --> [v2:172.21.15.39:6832/1472430787,v1:172.21.15.39:6834/1472430787] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564569913520 con 0x5645695b5e40 2024-09-21T07:37:37.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.298+0000 7efe78e00640 1 --2- 172.21.15.39:0/3406251349 >> [v2:172.21.15.39:6832/1472430787,v1:172.21.15.39:6834/1472430787] conn(0x5645695b5e40 0x5645699f5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:37.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.298+0000 7efe78e00640 1 --2- 172.21.15.39:0/3406251349 >> [v2:172.21.15.39:6832/1472430787,v1:172.21.15.39:6834/1472430787] conn(0x5645695b5e40 0x5645699f5dc0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcb0603fcc4067d5 server_cookie=4502954a452552c6 in_seq=0 out_seq=0 2024-09-21T07:37:37.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.298+0000 7efe5a000640 10 client.6753 ms_handle_connect on v2:172.21.15.39:6832/1472430787 2024-09-21T07:37:37.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.299+0000 7efe5a000640 1 -- 172.21.15.39:0/3406251349 <== mds.0 v2:172.21.15.39:6832/1472430787 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564569913520 con 0x5645695b5e40 2024-09-21T07:37:37.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.299+0000 7efe5a000640 10 client.6753 handle_command_reply: tid=0 2024-09-21T07:37:37.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.299+0000 7efe82847740 1 client.6753 shutdown 2024-09-21T07:37:37.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.300+0000 7efe82847740 1 -- 172.21.15.39:0/3406251349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645695f7270 msgr2=0x5645699b1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:37.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.300+0000 7efe82847740 1 --2- 172.21.15.39:0/3406251349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645695f7270 0x5645699b1b10 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7efe6c0056f0 tx=0x7efe6c001c10 comp rx=0 tx=0).stop 2024-09-21T07:37:37.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.300+0000 7efe82847740 1 -- 172.21.15.39:0/3406251349 shutdown_connections 2024-09-21T07:37:37.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.300+0000 7efe82847740 1 --2- 172.21.15.39:0/3406251349 >> [v2:172.21.15.39:6832/1472430787,v1:172.21.15.39:6834/1472430787] conn(0x5645695b5e40 0x5645699f5dc0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.300+0000 7efe82847740 1 --2- 172.21.15.39:0/3406251349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645695f7270 0x5645699b1b10 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.300+0000 7efe82847740 1 --2- 172.21.15.39:0/3406251349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5645695d6590 0x5645699c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.300+0000 7efe82847740 1 --2- 172.21.15.39:0/3406251349 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5645695ce3b0 0x5645699dbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.300+0000 7efe5a000640 0 client.6753 ms_handle_reset on v2:172.21.15.39:6832/1472430787 2024-09-21T07:37:37.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.300+0000 7efe82847740 1 -- 172.21.15.39:0/3406251349 >> 172.21.15.39:0/3406251349 conn(0x5645695eb3c0 msgr2=0x5645699c44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:37.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.300+0000 7efe82847740 1 -- 172.21.15.39:0/3406251349 shutdown_connections 2024-09-21T07:37:37.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.300+0000 7efe82847740 1 -- 172.21.15.39:0/3406251349 wait complete. 2024-09-21T07:37:37.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.300+0000 7efe82847740 20 client.6753 trim_cache size 0 max 16384 2024-09-21T07:37:37.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.319+0000 7efe82847740 1 Processor -- start 2024-09-21T07:37:37.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.320+0000 7efe82847740 1 -- start start 2024-09-21T07:37:37.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.321+0000 7efe82847740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645695ce3b0 0x56456999e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:37.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.321+0000 7efe82847740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5645695d6590 0x5645699b1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:37.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.321+0000 7efe82847740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5645695f7270 0x5645699f5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:37.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.321+0000 7efe82847740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efe742006e0 con 0x5645695ce3b0 2024-09-21T07:37:37.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.321+0000 7efe82847740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efe740cc220 con 0x5645695f7270 2024-09-21T07:37:37.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.321+0000 7efe82847740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7efe741ff220 con 0x5645695d6590 2024-09-21T07:37:37.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.321+0000 7efe78e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645695ce3b0 0x56456999e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:37.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.321+0000 7efe78e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645695ce3b0 0x56456999e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40876/0 (socket says 172.21.15.39:40876) 2024-09-21T07:37:37.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.321+0000 7efe78e00640 1 -- 172.21.15.39:0/2037374474 learned_addr learned my addr 172.21.15.39:0/2037374474 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:37.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.321+0000 7efe78e00640 1 -- 172.21.15.39:0/2037374474 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5645695d6590 msgr2=0x5645699b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:37:37.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.321+0000 7efe78e00640 1 --2- 172.21.15.39:0/2037374474 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5645695d6590 0x5645699b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.322+0000 7efe78e00640 1 -- 172.21.15.39:0/2037374474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5645695f7270 msgr2=0x5645699f5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:37:37.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.322+0000 7efe78e00640 1 --2- 172.21.15.39:0/2037374474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5645695f7270 0x5645699f5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.322+0000 7efe78e00640 1 -- 172.21.15.39:0/2037374474 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645699c6d70 con 0x5645695ce3b0 2024-09-21T07:37:37.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.322+0000 7efe78e00640 1 --2- 172.21.15.39:0/2037374474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645695ce3b0 0x56456999e860 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7efe68001eb0 tx=0x7efe68000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ebb8283f5fefffc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:37.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.322+0000 7efe64c00640 1 -- 172.21.15.39:0/2037374474 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe68069be0 con 0x5645695ce3b0 2024-09-21T07:37:37.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.322+0000 7efe64c00640 1 -- 172.21.15.39:0/2037374474 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe68061090 con 0x5645695ce3b0 2024-09-21T07:37:37.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.322+0000 7efe64c00640 1 -- 172.21.15.39:0/2037374474 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6805d780 con 0x5645695ce3b0 2024-09-21T07:37:37.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.323+0000 7efe82847740 1 -- 172.21.15.39:0/2037374474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645695ce3b0 msgr2=0x56456999e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:37.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.323+0000 7efe82847740 1 --2- 172.21.15.39:0/2037374474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645695ce3b0 0x56456999e860 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7efe68001eb0 tx=0x7efe68000ed0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.323+0000 7efe82847740 1 -- 172.21.15.39:0/2037374474 shutdown_connections 2024-09-21T07:37:37.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.323+0000 7efe82847740 1 --2- 172.21.15.39:0/2037374474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5645695f7270 0x5645699f5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.323+0000 7efe82847740 1 --2- 172.21.15.39:0/2037374474 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5645695d6590 0x5645699b1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.323+0000 7efe82847740 1 --2- 172.21.15.39:0/2037374474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645695ce3b0 0x56456999e860 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.323+0000 7efe82847740 1 -- 172.21.15.39:0/2037374474 >> 172.21.15.39:0/2037374474 conn(0x5645695eb3c0 msgr2=0x5645699ef4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:37.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.323+0000 7efe82847740 1 -- 172.21.15.39:0/2037374474 shutdown_connections 2024-09-21T07:37:37.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.323+0000 7efe82847740 1 -- 172.21.15.39:0/2037374474 wait complete. 2024-09-21T07:37:37.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.324+0000 7efe82847740 1 -- start start 2024-09-21T07:37:37.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.324+0000 7efe82847740 1 Processor -- start 2024-09-21T07:37:37.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.325+0000 7efe82847740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5645695ce3b0 0x564569a5db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:37.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.326+0000 7efe82847740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5645695d6590 0x5645699df4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:37.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.326+0000 7efe82847740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645695f7270 0x5645699dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:37.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.326+0000 7efe82847740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5645699c6b80 con 0x5645695f7270 2024-09-21T07:37:37.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.326+0000 7efe82847740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5645699c6190 con 0x5645695ce3b0 2024-09-21T07:37:37.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.326+0000 7efe82847740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x5645699f9270 con 0x5645695d6590 2024-09-21T07:37:37.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.326+0000 7efe82847740 10 client.0 _ll_register_callbacks cb 0x7efe6c005380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:37:37.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.326+0000 7efe79800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645695f7270 0x5645699dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:37.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.326+0000 7efe78e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5645695ce3b0 0x564569a5db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:37.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.326+0000 7efe79800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645695f7270 0x5645699dbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40886/0 (socket says 172.21.15.39:40886) 2024-09-21T07:37:37.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.326+0000 7efe79800640 1 -- 172.21.15.39:0/2797426297 learned_addr learned my addr 172.21.15.39:0/2797426297 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:37.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.326+0000 7efe73e00640 1 --2- 172.21.15.39:0/2797426297 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5645695d6590 0x5645699df4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:37.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.326+0000 7efe79800640 1 -- 172.21.15.39:0/2797426297 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5645695d6590 msgr2=0x5645699df4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:37:37.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.326+0000 7efe79800640 1 --2- 172.21.15.39:0/2797426297 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5645695d6590 0x5645699df4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.326+0000 7efe79800640 1 -- 172.21.15.39:0/2797426297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5645695ce3b0 msgr2=0x564569a5db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:37:37.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.326+0000 7efe79800640 1 --2- 172.21.15.39:0/2797426297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5645695ce3b0 0x564569a5db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.326+0000 7efe79800640 1 -- 172.21.15.39:0/2797426297 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645699937e0 con 0x5645695f7270 2024-09-21T07:37:37.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.326+0000 7efe73e00640 1 --2- 172.21.15.39:0/2797426297 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5645695d6590 0x5645699df4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:37:37.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.327+0000 7efe79800640 1 --2- 172.21.15.39:0/2797426297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645695f7270 0x5645699dbf80 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7efe6c014e10 tx=0x7efe6c002180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38348b7a2290bddd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:37.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.327+0000 7efe5a000640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:37:37.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.327+0000 7efe5a000640 1 -- 172.21.15.39:0/2797426297 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c006b30 con 0x5645695f7270 2024-09-21T07:37:37.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.327+0000 7efe5a000640 1 -- 172.21.15.39:0/2797426297 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 7001+0+0 (secure 0 0 0) 0x7efe6c015680 con 0x5645695f7270 2024-09-21T07:37:37.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.327+0000 7efe82847740 1 -- 172.21.15.39:0/2797426297 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe68061090 con 0x5645695f7270 2024-09-21T07:37:37.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.327+0000 7efe5a000640 1 -- 172.21.15.39:0/2797426297 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7efe6c033050 con 0x5645695f7270 2024-09-21T07:37:37.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.327+0000 7efe82847740 10 client.6759 fetch_fsmap learned FSMap version 74 2024-09-21T07:37:37.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.327+0000 7efe82847740 1 -- 172.21.15.39:0/2797426297 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x564569a5e0c0 con 0x5645695f7270 2024-09-21T07:37:37.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.327+0000 7efe5a000640 1 -- 172.21.15.39:0/2797426297 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe68061090 con 0x5645695f7270 2024-09-21T07:37:37.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.327+0000 7efe5a000640 1 -- 172.21.15.39:0/2797426297 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 74) ==== 3522+0+0 (secure 0 0 0) 0x7efe6c036020 con 0x5645695f7270 2024-09-21T07:37:37.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.327+0000 7efe82847740 10 client.6759 fetch_fsmap finished waiting for FSMap version 74 2024-09-21T07:37:37.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.327+0000 7efe82847740 10 client.6759 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:37:37.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.328+0000 7efe82847740 20 client.6759 populate_metadata read hostname 'smithi039' 2024-09-21T07:37:37.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.328+0000 7efe82847740 1 --2- 172.21.15.39:0/2797426297 >> [v2:172.21.15.39:6832/1472430787,v1:172.21.15.39:6834/1472430787] conn(0x5645699be960 0x5645699bed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:37.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.328+0000 7efe82847740 4 client.6759 mds_command: new command op to 6585 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:37:37.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.328+0000 7efe82847740 1 -- 172.21.15.39:0/2797426297 --> [v2:172.21.15.39:6832/1472430787,v1:172.21.15.39:6834/1472430787] -- command(tid 0: {"prefix": "status"}) -- 0x56456990efd0 con 0x5645699be960 2024-09-21T07:37:37.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.328+0000 7efe78e00640 1 --2- 172.21.15.39:0/2797426297 >> [v2:172.21.15.39:6832/1472430787,v1:172.21.15.39:6834/1472430787] conn(0x5645699be960 0x5645699bed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:37.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.328+0000 7efe5a000640 10 client.6759 ms_handle_connect on v2:172.21.15.39:6832/1472430787 2024-09-21T07:37:37.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.328+0000 7efe78e00640 1 --2- 172.21.15.39:0/2797426297 >> [v2:172.21.15.39:6832/1472430787,v1:172.21.15.39:6834/1472430787] conn(0x5645699be960 0x5645699bed70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f21edbbac75fff8c server_cookie=a8f894420bbb5b97 in_seq=0 out_seq=0 2024-09-21T07:37:37.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.329+0000 7efe5a000640 1 -- 172.21.15.39:0/2797426297 <== mds.0 v2:172.21.15.39:6832/1472430787 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x56456990efd0 con 0x5645699be960 2024-09-21T07:37:37.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.329+0000 7efe5a000640 10 client.6759 handle_command_reply: tid=0 2024-09-21T07:37:37.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.329+0000 7efe82847740 1 client.6759 shutdown 2024-09-21T07:37:37.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.329+0000 7efe82847740 1 -- 172.21.15.39:0/2797426297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645695f7270 msgr2=0x5645699dbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:37.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.329+0000 7efe82847740 1 --2- 172.21.15.39:0/2797426297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645695f7270 0x5645699dbf80 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7efe6c014e10 tx=0x7efe6c002180 comp rx=0 tx=0).stop 2024-09-21T07:37:37.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.329+0000 7efe82847740 1 -- 172.21.15.39:0/2797426297 shutdown_connections 2024-09-21T07:37:37.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.329+0000 7efe82847740 1 --2- 172.21.15.39:0/2797426297 >> [v2:172.21.15.39:6832/1472430787,v1:172.21.15.39:6834/1472430787] conn(0x5645699be960 0x5645699bed70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.329+0000 7efe82847740 1 --2- 172.21.15.39:0/2797426297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5645695f7270 0x5645699dbf80 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.329+0000 7efe82847740 1 --2- 172.21.15.39:0/2797426297 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5645695d6590 0x5645699df4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.329+0000 7efe82847740 1 --2- 172.21.15.39:0/2797426297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5645695ce3b0 0x564569a5db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.329+0000 7efe5a000640 0 client.6759 ms_handle_reset on v2:172.21.15.39:6832/1472430787 2024-09-21T07:37:37.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.329+0000 7efe82847740 1 -- 172.21.15.39:0/2797426297 >> 172.21.15.39:0/2797426297 conn(0x5645695eb3c0 msgr2=0x5645699b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:37.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.329+0000 7efe82847740 1 -- 172.21.15.39:0/2797426297 shutdown_connections 2024-09-21T07:37:37.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.329+0000 7efe82847740 1 -- 172.21.15.39:0/2797426297 wait complete. 2024-09-21T07:37:37.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.329+0000 7efe82847740 20 client.6759 trim_cache size 0 max 16384 2024-09-21T07:37:37.339 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:37:37.339 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:37:37.339 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-09-21T07:37:37.340 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 6585, 2024-09-21T07:37:37.340 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:37:37.340 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:37:37.340 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:37:37.340 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 7.535124755, 2024-09-21T07:37:37.340 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 74, 2024-09-21T07:37:37.340 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 94, 2024-09-21T07:37:37.340 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 94, 2024-09-21T07:37:37.340 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 35.826895100000002 2024-09-21T07:37:37.340 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:37:37.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.330+0000 7efe7b800640 1 -- 172.21.15.39:0/2909280569 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efe54078bc0 msgr2=0x7efe5407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:37.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.330+0000 7efe7b800640 1 --2- 172.21.15.39:0/2909280569 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efe54078bc0 0x7efe5407b0d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7efe60033000 tx=0x7efe60004730 comp rx=0 tx=0).stop 2024-09-21T07:37:37.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.330+0000 7efe7b800640 1 -- 172.21.15.39:0/2909280569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe74070030 msgr2=0x7efe7409de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:37.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.330+0000 7efe7b800640 1 --2- 172.21.15.39:0/2909280569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe74070030 0x7efe7409de30 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7efe6800c400 tx=0x7efe6800c8c0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.330+0000 7efe7b800640 1 -- 172.21.15.39:0/2909280569 shutdown_connections 2024-09-21T07:37:37.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.330+0000 7efe7b800640 1 --2- 172.21.15.39:0/2909280569 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efe54078bc0 0x7efe5407b0d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.330+0000 7efe7b800640 1 --2- 172.21.15.39:0/2909280569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe74073170 0x7efe7415a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.330+0000 7efe7b800640 1 --2- 172.21.15.39:0/2909280569 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efe74070440 0x7efe7414b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.330+0000 7efe7b800640 1 --2- 172.21.15.39:0/2909280569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe74070030 0x7efe7409de30 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.330+0000 7efe7b800640 1 -- 172.21.15.39:0/2909280569 >> 172.21.15.39:0/2909280569 conn(0x7efe740b1cb0 msgr2=0x7efe740ad950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:37.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.330+0000 7efe7b800640 1 -- 172.21.15.39:0/2909280569 shutdown_connections 2024-09-21T07:37:37.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.330+0000 7efe7b800640 1 -- 172.21.15.39:0/2909280569 wait complete. 2024-09-21T07:37:37.358 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:37:37.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.585+0000 7f181b400640 1 -- 172.21.15.39:0/3170381261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f181415bf20 msgr2=0x7f18140ca9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:37.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.585+0000 7f181b400640 1 --2- 172.21.15.39:0/3170381261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f181415bf20 0x7f18140ca9a0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f1800009830 tx=0x7f180002a5c0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.585+0000 7f181b400640 1 -- 172.21.15.39:0/3170381261 shutdown_connections 2024-09-21T07:37:37.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.585+0000 7f181b400640 1 --2- 172.21.15.39:0/3170381261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f181415bf20 0x7f18140ca9a0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.585+0000 7f181b400640 1 -- 172.21.15.39:0/3170381261 >> 172.21.15.39:0/3170381261 conn(0x7f18140b1cb0 msgr2=0x7f18140b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:37.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.585+0000 7f181b400640 1 -- 172.21.15.39:0/3170381261 shutdown_connections 2024-09-21T07:37:37.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.585+0000 7f181b400640 1 -- 172.21.15.39:0/3170381261 wait complete. 2024-09-21T07:37:37.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.588+0000 7f181b400640 1 Processor -- start 2024-09-21T07:37:37.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.588+0000 7f181b400640 1 -- start start 2024-09-21T07:37:37.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.589+0000 7f181b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18140caee0 0x7f181415eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:37.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.589+0000 7f181b400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18140cfb00 0x7f1814157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:37.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.589+0000 7f181b400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f181415bf20 0x7f1814158290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:37.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.589+0000 7f181b400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1814070730 con 0x7f18140caee0 2024-09-21T07:37:37.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.589+0000 7f181b400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f18140705b0 con 0x7f18140cfb00 2024-09-21T07:37:37.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.589+0000 7f181b400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f18140708b0 con 0x7f181415bf20 2024-09-21T07:37:37.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.589+0000 7f1818a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18140caee0 0x7f181415eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:37.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.589+0000 7f1818a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18140caee0 0x7f181415eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40908/0 (socket says 172.21.15.39:40908) 2024-09-21T07:37:37.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.589+0000 7f1818a00640 1 -- 172.21.15.39:0/2382237064 learned_addr learned my addr 172.21.15.39:0/2382237064 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:37.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.590+0000 7f1813e00640 1 --2- 172.21.15.39:0/2382237064 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18140cfb00 0x7f1814157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:37.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.590+0000 7f1818a00640 1 -- 172.21.15.39:0/2382237064 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f181415bf20 msgr2=0x7f1814158290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:37.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.590+0000 7f1819400640 1 --2- 172.21.15.39:0/2382237064 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f181415bf20 0x7f1814158290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:37.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.590+0000 7f1818a00640 1 --2- 172.21.15.39:0/2382237064 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f181415bf20 0x7f1814158290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.590+0000 7f1818a00640 1 -- 172.21.15.39:0/2382237064 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18140cfb00 msgr2=0x7f1814157d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:37.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.590+0000 7f1818a00640 1 --2- 172.21.15.39:0/2382237064 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18140cfb00 0x7f1814157d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.590+0000 7f1818a00640 1 -- 172.21.15.39:0/2382237064 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18140c97f0 con 0x7f18140caee0 2024-09-21T07:37:37.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.590+0000 7f1813e00640 1 --2- 172.21.15.39:0/2382237064 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18140cfb00 0x7f1814157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:37:37.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.590+0000 7f1819400640 1 --2- 172.21.15.39:0/2382237064 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f181415bf20 0x7f1814158290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:37:37.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.590+0000 7f1818a00640 1 --2- 172.21.15.39:0/2382237064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18140caee0 0x7f181415eda0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f180800c430 tx=0x7f180800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:37.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.590+0000 7f1810c00640 1 -- 172.21.15.39:0/2382237064 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1808017070 con 0x7f18140caee0 2024-09-21T07:37:37.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.591+0000 7f1810c00640 1 -- 172.21.15.39:0/2382237064 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18080053c0 con 0x7f18140caee0 2024-09-21T07:37:37.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.591+0000 7f181b400640 1 -- 172.21.15.39:0/2382237064 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18140c9ad0 con 0x7f18140caee0 2024-09-21T07:37:37.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.591+0000 7f1810c00640 1 -- 172.21.15.39:0/2382237064 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1808003850 con 0x7f18140caee0 2024-09-21T07:37:37.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.591+0000 7f181b400640 1 -- 172.21.15.39:0/2382237064 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18140ca090 con 0x7f18140caee0 2024-09-21T07:37:37.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.592+0000 7f181b400640 1 -- 172.21.15.39:0/2382237064 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18140ca4d0 con 0x7f18140caee0 2024-09-21T07:37:37.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.593+0000 7f1810c00640 1 -- 172.21.15.39:0/2382237064 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f180801a070 con 0x7f18140caee0 2024-09-21T07:37:37.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.594+0000 7f1810c00640 1 --2- 172.21.15.39:0/2382237064 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f17f4078b90 0x7f17f407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:37.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.594+0000 7f1810c00640 1 -- 172.21.15.39:0/2382237064 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 7001+0+0 (secure 0 0 0) 0x7f180809a6e0 con 0x7f18140caee0 2024-09-21T07:37:37.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.595+0000 7f1813e00640 1 --2- 172.21.15.39:0/2382237064 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f17f4078b90 0x7f17f407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:37.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.596+0000 7f1813e00640 1 --2- 172.21.15.39:0/2382237064 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f17f4078b90 0x7f17f407b0a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1800034000 tx=0x7f1800004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:37.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.599+0000 7f1810c00640 1 -- 172.21.15.39:0/2382237064 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1808062580 con 0x7f18140caee0 2024-09-21T07:37:37.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.836+0000 7f181b400640 1 -- 172.21.15.39:0/2382237064 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18140ca9a0 con 0x7f18140caee0 2024-09-21T07:37:37.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.837+0000 7f1810c00640 1 -- 172.21.15.39:0/2382237064 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5249 (secure 0 0 0) 0x7f1808061e80 con 0x7f18140caee0 2024-09-21T07:37:37.839 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:37:37.839 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":74,"btime":"2024-09-21T07:37:33:801850+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5870,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2381504293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2381504293},{"type":"v1","addr":"172.21.15.148:6837","nonce":2381504293}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:37:33.801842+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6585,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6585":{"gid":6585,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6834/1472430787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1472430787},{"type":"v1","addr":"172.21.15.39:6834","nonce":1472430787}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6585,"qdb_cluster":[6115,6585]},"id":6}]} 2024-09-21T07:37:37.840 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 74 2024-09-21T07:37:37.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.840+0000 7f181b400640 1 -- 172.21.15.39:0/2382237064 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f17f4078b90 msgr2=0x7f17f407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:37.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.840+0000 7f181b400640 1 --2- 172.21.15.39:0/2382237064 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f17f4078b90 0x7f17f407b0a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1800034000 tx=0x7f1800004730 comp rx=0 tx=0).stop 2024-09-21T07:37:37.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.840+0000 7f181b400640 1 -- 172.21.15.39:0/2382237064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18140caee0 msgr2=0x7f181415eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:37.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.840+0000 7f181b400640 1 --2- 172.21.15.39:0/2382237064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18140caee0 0x7f181415eda0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f180800c430 tx=0x7f180800c8f0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.840+0000 7f181b400640 1 -- 172.21.15.39:0/2382237064 shutdown_connections 2024-09-21T07:37:37.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.840+0000 7f181b400640 1 --2- 172.21.15.39:0/2382237064 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f17f4078b90 0x7f17f407b0a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.840+0000 7f181b400640 1 --2- 172.21.15.39:0/2382237064 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f181415bf20 0x7f1814158290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.840+0000 7f181b400640 1 --2- 172.21.15.39:0/2382237064 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18140cfb00 0x7f1814157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.840+0000 7f181b400640 1 --2- 172.21.15.39:0/2382237064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18140caee0 0x7f181415eda0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:37.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.840+0000 7f181b400640 1 -- 172.21.15.39:0/2382237064 >> 172.21.15.39:0/2382237064 conn(0x7f18140b1cb0 msgr2=0x7f18140b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:37.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.840+0000 7f181b400640 1 -- 172.21.15.39:0/2382237064 shutdown_connections 2024-09-21T07:37:37.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:37.840+0000 7f181b400640 1 -- 172.21.15.39:0/2382237064 wait complete. 2024-09-21T07:37:37.868 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2024-09-21T07:37:38.112 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_snap":3,"last_created":3,"last_destroyed":1,"pending_noop":[],"snaps":[{"snapid":2,"ino":1099511627777,"stamp":"2024-09-21T07:36:43.100278+0000","name":"s11","metadata":{}},{"snapid":3,"ino":1099511627777,"stamp":"2024-09-21T07:37:11.742421+0000","name":"s12","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2024-09-21T07:37:38.113 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 3, "last_destroyed": 1, "last_snap": 3, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s11", "snapid": 2, "stamp": "2024-09-21T07:36:43.100278+0000" }, { "ino": 1099511627777, "metadata": {}, "name": "s12", "snapid": 3, "stamp": "2024-09-21T07:37:11.742421+0000" } ] } 2024-09-21T07:37:38.113 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T07:37:38.113 INFO:tasks.cephfs.test_snapshots:testing snapserver mds_kill_mdstable_at=5 2024-09-21T07:37:38.113 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:37:38.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.338+0000 7f2afe000640 1 -- 172.21.15.39:0/938278590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2af80735d0 msgr2=0x7f2af8153510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:38.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.338+0000 7f2afe000640 1 --2- 172.21.15.39:0/938278590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2af80735d0 0x7f2af8153510 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4009830 tx=0x7f2ae402a510 comp rx=0 tx=0).stop 2024-09-21T07:37:38.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.338+0000 7f2afe000640 1 -- 172.21.15.39:0/938278590 shutdown_connections 2024-09-21T07:37:38.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.338+0000 7f2afe000640 1 --2- 172.21.15.39:0/938278590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2af80735d0 0x7f2af8153510 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:38.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.338+0000 7f2afe000640 1 --2- 172.21.15.39:0/938278590 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2af80af700 0x7f2af80caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:38.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.338+0000 7f2afe000640 1 -- 172.21.15.39:0/938278590 >> 172.21.15.39:0/938278590 conn(0x7f2af806d870 msgr2=0x7f2af806ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:38.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.338+0000 7f2afe000640 1 -- 172.21.15.39:0/938278590 shutdown_connections 2024-09-21T07:37:38.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.338+0000 7f2afe000640 1 -- 172.21.15.39:0/938278590 wait complete. 2024-09-21T07:37:38.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.340+0000 7f2afe000640 1 Processor -- start 2024-09-21T07:37:38.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.340+0000 7f2afe000640 1 -- start start 2024-09-21T07:37:38.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.342+0000 7f2afe000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2af80735d0 0x7f2af814dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:38.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.342+0000 7f2afe000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2af80af700 0x7f2af80b1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:38.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.342+0000 7f2afe000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2af80b54d0 0x7f2af80b22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:38.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.342+0000 7f2afe000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2af815d2b0 con 0x7f2af80b54d0 2024-09-21T07:37:38.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.342+0000 7f2afe000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2af815d130 con 0x7f2af80af700 2024-09-21T07:37:38.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.342+0000 7f2afe000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f2af815d430 con 0x7f2af80735d0 2024-09-21T07:37:38.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.342+0000 7f2afd600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2af80b54d0 0x7f2af80b22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:38.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.342+0000 7f2afd600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2af80b54d0 0x7f2af80b22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40928/0 (socket says 172.21.15.39:40928) 2024-09-21T07:37:38.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.342+0000 7f2afd600640 1 -- 172.21.15.39:0/1262537260 learned_addr learned my addr 172.21.15.39:0/1262537260 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:38.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.342+0000 7f2afcc00640 1 --2- 172.21.15.39:0/1262537260 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2af80735d0 0x7f2af814dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:38.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.342+0000 7f2af7e00640 1 --2- 172.21.15.39:0/1262537260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2af80af700 0x7f2af80b1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:38.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.342+0000 7f2afd600640 1 -- 172.21.15.39:0/1262537260 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2af80735d0 msgr2=0x7f2af814dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:38.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.342+0000 7f2afd600640 1 --2- 172.21.15.39:0/1262537260 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2af80735d0 0x7f2af814dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:38.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.343+0000 7f2afd600640 1 -- 172.21.15.39:0/1262537260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2af80af700 msgr2=0x7f2af80b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:38.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.343+0000 7f2afd600640 1 --2- 172.21.15.39:0/1262537260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2af80af700 0x7f2af80b1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:38.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.343+0000 7f2afd600640 1 -- 172.21.15.39:0/1262537260 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af814e110 con 0x7f2af80b54d0 2024-09-21T07:37:38.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.343+0000 7f2af7e00640 1 --2- 172.21.15.39:0/1262537260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2af80af700 0x7f2af80b1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:37:38.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.343+0000 7f2afd600640 1 --2- 172.21.15.39:0/1262537260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2af80b54d0 0x7f2af80b22d0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f2af000c730 tx=0x7f2af000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:38.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.343+0000 7f2af4c00640 1 -- 172.21.15.39:0/1262537260 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af0003b70 con 0x7f2af80b54d0 2024-09-21T07:37:38.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.343+0000 7f2af4c00640 1 -- 172.21.15.39:0/1262537260 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2af0003d30 con 0x7f2af80b54d0 2024-09-21T07:37:38.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.343+0000 7f2af4c00640 1 -- 172.21.15.39:0/1262537260 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af00052d0 con 0x7f2af80b54d0 2024-09-21T07:37:38.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.343+0000 7f2afe000640 1 -- 172.21.15.39:0/1262537260 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2af8156310 con 0x7f2af80b54d0 2024-09-21T07:37:38.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.343+0000 7f2afe000640 1 -- 172.21.15.39:0/1262537260 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2af80afd80 con 0x7f2af80b54d0 2024-09-21T07:37:38.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.344+0000 7f2afe000640 1 -- 172.21.15.39:0/1262537260 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2af815d8e0 con 0x7f2af80b54d0 2024-09-21T07:37:38.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.346+0000 7f2af4c00640 1 -- 172.21.15.39:0/1262537260 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2af0005680 con 0x7f2af80b54d0 2024-09-21T07:37:38.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.347+0000 7f2af4c00640 1 --2- 172.21.15.39:0/1262537260 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2ad8078b90 0x7f2ad807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:38.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.347+0000 7f2af4c00640 1 -- 172.21.15.39:0/1262537260 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 7001+0+0 (secure 0 0 0) 0x7f2af009b0b0 con 0x7f2af80b54d0 2024-09-21T07:37:38.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.347+0000 7f2afcc00640 1 --2- 172.21.15.39:0/1262537260 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2ad8078b90 0x7f2ad807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:38.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.348+0000 7f2afcc00640 1 --2- 172.21.15.39:0/1262537260 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2ad8078b90 0x7f2ad807b0a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2aec005fd0 tx=0x7f2aec005850 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:38.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.352+0000 7f2af4c00640 1 -- 172.21.15.39:0/1262537260 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2af0063100 con 0x7f2af80b54d0 2024-09-21T07:37:38.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.587+0000 7f2afe000640 1 -- 172.21.15.39:0/1262537260 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2af80b49e0 con 0x7f2af80b54d0 2024-09-21T07:37:38.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.587+0000 7f2af4c00640 1 -- 172.21.15.39:0/1262537260 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5249 (secure 0 0 0) 0x7f2af0062a00 con 0x7f2af80b54d0 2024-09-21T07:37:38.589 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:37:38.589 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":74,"btime":"2024-09-21T07:37:33:801850+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5870,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2381504293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2381504293},{"type":"v1","addr":"172.21.15.148:6837","nonce":2381504293}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:37:33.801842+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6585,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6585":{"gid":6585,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6834/1472430787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1472430787},{"type":"v1","addr":"172.21.15.39:6834","nonce":1472430787}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6585,"qdb_cluster":[6115,6585]},"id":6}]} 2024-09-21T07:37:38.590 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 74 2024-09-21T07:37:38.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.589+0000 7f2afe000640 1 -- 172.21.15.39:0/1262537260 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2ad8078b90 msgr2=0x7f2ad807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:38.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.589+0000 7f2afe000640 1 --2- 172.21.15.39:0/1262537260 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2ad8078b90 0x7f2ad807b0a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2aec005fd0 tx=0x7f2aec005850 comp rx=0 tx=0).stop 2024-09-21T07:37:38.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.589+0000 7f2afe000640 1 -- 172.21.15.39:0/1262537260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2af80b54d0 msgr2=0x7f2af80b22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:38.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.589+0000 7f2afe000640 1 --2- 172.21.15.39:0/1262537260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2af80b54d0 0x7f2af80b22d0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f2af000c730 tx=0x7f2af000cbf0 comp rx=0 tx=0).stop 2024-09-21T07:37:38.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.590+0000 7f2afe000640 1 -- 172.21.15.39:0/1262537260 shutdown_connections 2024-09-21T07:37:38.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.590+0000 7f2afe000640 1 --2- 172.21.15.39:0/1262537260 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2ad8078b90 0x7f2ad807b0a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:38.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.590+0000 7f2afe000640 1 --2- 172.21.15.39:0/1262537260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2af80b54d0 0x7f2af80b22d0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:38.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.590+0000 7f2afe000640 1 --2- 172.21.15.39:0/1262537260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2af80af700 0x7f2af80b1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:38.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.590+0000 7f2afe000640 1 --2- 172.21.15.39:0/1262537260 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2af80735d0 0x7f2af814dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:38.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.590+0000 7f2afe000640 1 -- 172.21.15.39:0/1262537260 >> 172.21.15.39:0/1262537260 conn(0x7f2af806d870 msgr2=0x7f2af814f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:38.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.590+0000 7f2afe000640 1 -- 172.21.15.39:0/1262537260 shutdown_connections 2024-09-21T07:37:38.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.590+0000 7f2afe000640 1 -- 172.21.15.39:0/1262537260 wait complete. 2024-09-21T07:37:38.619 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:0 true 2024-09-21T07:37:38.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.848+0000 7f2252c00640 1 -- 172.21.15.39:0/557650121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f224c0caf80 msgr2=0x7f224c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:38.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.848+0000 7f2252c00640 1 --2- 172.21.15.39:0/557650121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f224c0caf80 0x7f224c0b4d90 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f224c0ce970 tx=0x7f224002a4a0 comp rx=0 tx=0).stop 2024-09-21T07:37:38.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.848+0000 7f2252c00640 1 -- 172.21.15.39:0/557650121 shutdown_connections 2024-09-21T07:37:38.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.848+0000 7f2252c00640 1 --2- 172.21.15.39:0/557650121 >> v2:172.21.15.148:3300/0 conn(0x7f224c0d1100 0x7f224c0b71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:38.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.848+0000 7f2252c00640 1 --2- 172.21.15.39:0/557650121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f224c0caf80 0x7f224c0b4d90 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:38.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.848+0000 7f2252c00640 1 -- 172.21.15.39:0/557650121 >> 172.21.15.39:0/557650121 conn(0x7f224c09d6b0 msgr2=0x7f224c0d4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:38.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.848+0000 7f2252c00640 1 -- 172.21.15.39:0/557650121 shutdown_connections 2024-09-21T07:37:38.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.848+0000 7f2252c00640 1 -- 172.21.15.39:0/557650121 wait complete. 2024-09-21T07:37:38.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.851+0000 7f2252c00640 1 Processor -- start 2024-09-21T07:37:38.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.851+0000 7f2252c00640 1 -- start start 2024-09-21T07:37:38.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.852+0000 7f2252c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f224c0caf80 0x7f224c1f5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:38.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.852+0000 7f2252c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f224c0cffa0 0x7f224c1eeee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:38.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.852+0000 7f2252c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f224c0d1100 0x7f224c0a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:38.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.853+0000 7f2252c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f224c0b7a90 con 0x7f224c0caf80 2024-09-21T07:37:38.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.853+0000 7f2252c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f224c0b7910 con 0x7f224c0d1100 2024-09-21T07:37:38.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.853+0000 7f2252c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f224c0b7c10 con 0x7f224c0cffa0 2024-09-21T07:37:38.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.853+0000 7f2251800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f224c0caf80 0x7f224c1f5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:38.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.853+0000 7f2251800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f224c0caf80 0x7f224c1f5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40946/0 (socket says 172.21.15.39:40946) 2024-09-21T07:37:38.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.853+0000 7f2251800640 1 -- 172.21.15.39:0/3249507537 learned_addr learned my addr 172.21.15.39:0/3249507537 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:38.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.853+0000 7f2250e00640 1 --2- 172.21.15.39:0/3249507537 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f224c0cffa0 0x7f224c1eeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:38.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.853+0000 7f2252200640 1 --2- 172.21.15.39:0/3249507537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f224c0d1100 0x7f224c0a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:38.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.853+0000 7f2251800640 1 -- 172.21.15.39:0/3249507537 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f224c0cffa0 msgr2=0x7f224c1eeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:38.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.853+0000 7f2251800640 1 --2- 172.21.15.39:0/3249507537 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f224c0cffa0 0x7f224c1eeee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:38.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.853+0000 7f2251800640 1 -- 172.21.15.39:0/3249507537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f224c0d1100 msgr2=0x7f224c0a14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:38.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.853+0000 7f2251800640 1 --2- 172.21.15.39:0/3249507537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f224c0d1100 0x7f224c0a14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:38.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.853+0000 7f2251800640 1 -- 172.21.15.39:0/3249507537 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f224c0b3490 con 0x7f224c0caf80 2024-09-21T07:37:38.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.853+0000 7f2252200640 1 --2- 172.21.15.39:0/3249507537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f224c0d1100 0x7f224c0a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:37:38.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.853+0000 7f2250e00640 1 --2- 172.21.15.39:0/3249507537 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f224c0cffa0 0x7f224c1eeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:37:38.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.854+0000 7f2251800640 1 --2- 172.21.15.39:0/3249507537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f224c0caf80 0x7f224c1f5bd0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f2240009780 tx=0x7f2240033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:38.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.854+0000 7f2249600640 1 -- 172.21.15.39:0/3249507537 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2240031d60 con 0x7f224c0caf80 2024-09-21T07:37:38.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.854+0000 7f2249600640 1 -- 172.21.15.39:0/3249507537 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2240033e10 con 0x7f224c0caf80 2024-09-21T07:37:38.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.854+0000 7f2249600640 1 -- 172.21.15.39:0/3249507537 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2240005260 con 0x7f224c0caf80 2024-09-21T07:37:38.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.854+0000 7f2252c00640 1 -- 172.21.15.39:0/3249507537 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f224c0b1120 con 0x7f224c0caf80 2024-09-21T07:37:38.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.854+0000 7f2252c00640 1 -- 172.21.15.39:0/3249507537 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f224c0b3f10 con 0x7f224c0caf80 2024-09-21T07:37:38.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.855+0000 7f2252c00640 1 -- 172.21.15.39:0/3249507537 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f224c0b4d90 con 0x7f224c0caf80 2024-09-21T07:37:38.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.856+0000 7f2249600640 1 -- 172.21.15.39:0/3249507537 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2240005400 con 0x7f224c0caf80 2024-09-21T07:37:38.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.857+0000 7f2249600640 1 --2- 172.21.15.39:0/3249507537 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f222c078b90 0x7f222c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:38.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.857+0000 7f2249600640 1 -- 172.21.15.39:0/3249507537 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 7001+0+0 (secure 0 0 0) 0x7f22400bc500 con 0x7f224c0caf80 2024-09-21T07:37:38.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.858+0000 7f2250e00640 1 --2- 172.21.15.39:0/3249507537 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f222c078b90 0x7f222c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:38.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.858+0000 7f2250e00640 1 --2- 172.21.15.39:0/3249507537 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f222c078b90 0x7f222c07b0a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f224c0a22e0 tx=0x7f2238009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:38.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:38.862+0000 7f2249600640 1 -- 172.21.15.39:0/3249507537 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f224003f030 con 0x7f224c0caf80 2024-09-21T07:37:39.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:39.133+0000 7f2252c00640 1 -- 172.21.15.39:0/3249507537 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"} v 0) -- 0x7f224c0d0c20 con 0x7f224c0caf80 2024-09-21T07:37:39.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:39.853+0000 7f2249600640 1 -- 172.21.15.39:0/3249507537 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"}]=0 freezing mds.6585 v75) ==== 112+0+0 (secure 0 0 0) 0x7f22400844a0 con 0x7f224c0caf80 2024-09-21T07:37:39.855 INFO:teuthology.orchestra.run.smithi039.stderr:freezing mds.6585 2024-09-21T07:37:39.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:39.856+0000 7f2252c00640 1 -- 172.21.15.39:0/3249507537 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f222c078b90 msgr2=0x7f222c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:39.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:39.856+0000 7f2252c00640 1 --2- 172.21.15.39:0/3249507537 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f222c078b90 0x7f222c07b0a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f224c0a22e0 tx=0x7f2238009040 comp rx=0 tx=0).stop 2024-09-21T07:37:39.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:39.856+0000 7f2252c00640 1 -- 172.21.15.39:0/3249507537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f224c0caf80 msgr2=0x7f224c1f5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:39.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:39.856+0000 7f2252c00640 1 --2- 172.21.15.39:0/3249507537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f224c0caf80 0x7f224c1f5bd0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f2240009780 tx=0x7f2240033c20 comp rx=0 tx=0).stop 2024-09-21T07:37:39.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:39.856+0000 7f2252c00640 1 -- 172.21.15.39:0/3249507537 shutdown_connections 2024-09-21T07:37:39.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:39.856+0000 7f2252c00640 1 --2- 172.21.15.39:0/3249507537 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f222c078b90 0x7f222c07b0a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:39.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:39.856+0000 7f2252c00640 1 --2- 172.21.15.39:0/3249507537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f224c0d1100 0x7f224c0a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:39.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:39.856+0000 7f2252c00640 1 --2- 172.21.15.39:0/3249507537 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f224c0cffa0 0x7f224c1eeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:39.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:39.856+0000 7f2252c00640 1 --2- 172.21.15.39:0/3249507537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f224c0caf80 0x7f224c1f5bd0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:39.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:39.856+0000 7f2252c00640 1 -- 172.21.15.39:0/3249507537 >> 172.21.15.39:0/3249507537 conn(0x7f224c09d6b0 msgr2=0x7f224c0b9490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:39.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:39.857+0000 7f2252c00640 1 -- 172.21.15.39:0/3249507537 shutdown_connections 2024-09-21T07:37:39.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:39.857+0000 7f2252c00640 1 -- 172.21.15.39:0/3249507537 wait complete. 2024-09-21T07:37:39.889 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json config set mds_kill_mdstable_at 5 2024-09-21T07:37:40.139 INFO:teuthology.orchestra.run.smithi039.stdout:{"success":"mds_kill_mdstable_at = '' (not observed, change may require restart) "} 2024-09-21T07:37:40.140 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_kill_mdstable_at = '' (not observed, change may require restart) " } 2024-09-21T07:37:40.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:37:40.140 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d1/dir/.snap/s15) 2024-09-21T07:37:40.144 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-09-21T07:37:40.185 INFO:tasks.ceph.mds.c.smithi039.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_commit(ceph::cref_t&)' thread 7ff51ae00640 time 2024-09-21T07:37:40.184370+0000 2024-09-21T07:37:40.185 INFO:tasks.ceph.mds.c.smithi039.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: 147: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 5) 2024-09-21T07:37:40.187 INFO:tasks.ceph.mds.c.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:37:40.188 INFO:tasks.ceph.mds.c.smithi039.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7ff521e5654b] 2024-09-21T07:37:40.188 INFO:tasks.ceph.mds.c.smithi039.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7ff521e56758] 2024-09-21T07:37:40.188 INFO:tasks.ceph.mds.c.smithi039.stderr: 3: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x3bd) [0x55f55fdf930b] 2024-09-21T07:37:40.188 INFO:tasks.ceph.mds.c.smithi039.stderr: 4: (MDSTableServer::handle_request(boost::intrusive_ptr const&)+0x9a) [0x55f55fdf9910] 2024-09-21T07:37:40.188 INFO:tasks.ceph.mds.c.smithi039.stderr: 5: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xca5) [0x55f55fa92de3] 2024-09-21T07:37:40.188 INFO:tasks.ceph.mds.c.smithi039.stderr: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55f55fa96523] 2024-09-21T07:37:40.188 INFO:tasks.ceph.mds.c.smithi039.stderr: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55f55fa96e4a] 2024-09-21T07:37:40.188 INFO:tasks.ceph.mds.c.smithi039.stderr: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55f55fa79fa4] 2024-09-21T07:37:40.188 INFO:tasks.ceph.mds.c.smithi039.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7ff521fa57df] 2024-09-21T07:37:40.188 INFO:tasks.ceph.mds.c.smithi039.stderr: 10: (DispatchQueue::entry()+0x60f) [0x7ff521fa60d1] 2024-09-21T07:37:40.188 INFO:tasks.ceph.mds.c.smithi039.stderr: 11: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7ff522093f5b] 2024-09-21T07:37:40.188 INFO:tasks.ceph.mds.c.smithi039.stderr: 12: (Thread::entry_wrapper()+0x43) [0x7ff521e2fd57] 2024-09-21T07:37:40.188 INFO:tasks.ceph.mds.c.smithi039.stderr: 13: (Thread::_entry_func(void*)+0xd) [0x7ff521e2fd73] 2024-09-21T07:37:40.189 INFO:tasks.ceph.mds.c.smithi039.stderr: 14: /lib64/libc.so.6(+0x89c52) [0x7ff521489c52] 2024-09-21T07:37:40.189 INFO:tasks.ceph.mds.c.smithi039.stderr: 15: /lib64/libc.so.6(+0x10ec80) [0x7ff52150ec80] 2024-09-21T07:37:40.189 INFO:tasks.ceph.mds.c.smithi039.stderr:2024-09-21T07:37:40.184+0000 7ff51ae00640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_commit(ceph::cref_t&)' thread 7ff51ae00640 time 2024-09-21T07:37:40.184370+0000 2024-09-21T07:37:40.189 INFO:tasks.ceph.mds.c.smithi039.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: 147: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 5) 2024-09-21T07:37:40.189 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:37:40.189 INFO:tasks.ceph.mds.c.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:37:40.189 INFO:tasks.ceph.mds.c.smithi039.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7ff521e5654b] 2024-09-21T07:37:40.189 INFO:tasks.ceph.mds.c.smithi039.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7ff521e56758] 2024-09-21T07:37:40.189 INFO:tasks.ceph.mds.c.smithi039.stderr: 3: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x3bd) [0x55f55fdf930b] 2024-09-21T07:37:40.189 INFO:tasks.ceph.mds.c.smithi039.stderr: 4: (MDSTableServer::handle_request(boost::intrusive_ptr const&)+0x9a) [0x55f55fdf9910] 2024-09-21T07:37:40.189 INFO:tasks.ceph.mds.c.smithi039.stderr: 5: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xca5) [0x55f55fa92de3] 2024-09-21T07:37:40.189 INFO:tasks.ceph.mds.c.smithi039.stderr: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55f55fa96523] 2024-09-21T07:37:40.189 INFO:tasks.ceph.mds.c.smithi039.stderr: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55f55fa96e4a] 2024-09-21T07:37:40.189 INFO:tasks.ceph.mds.c.smithi039.stderr: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55f55fa79fa4] 2024-09-21T07:37:40.190 INFO:tasks.ceph.mds.c.smithi039.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7ff521fa57df] 2024-09-21T07:37:40.190 INFO:tasks.ceph.mds.c.smithi039.stderr: 10: (DispatchQueue::entry()+0x60f) [0x7ff521fa60d1] 2024-09-21T07:37:40.190 INFO:tasks.ceph.mds.c.smithi039.stderr: 11: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7ff522093f5b] 2024-09-21T07:37:40.190 INFO:tasks.ceph.mds.c.smithi039.stderr: 12: (Thread::entry_wrapper()+0x43) [0x7ff521e2fd57] 2024-09-21T07:37:40.190 INFO:tasks.ceph.mds.c.smithi039.stderr: 13: (Thread::_entry_func(void*)+0xd) [0x7ff521e2fd73] 2024-09-21T07:37:40.190 INFO:tasks.ceph.mds.c.smithi039.stderr: 14: /lib64/libc.so.6(+0x89c52) [0x7ff521489c52] 2024-09-21T07:37:40.190 INFO:tasks.ceph.mds.c.smithi039.stderr: 15: /lib64/libc.so.6(+0x10ec80) [0x7ff52150ec80] 2024-09-21T07:37:40.190 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:37:40.190 INFO:tasks.ceph.mds.c.smithi039.stderr:*** Caught signal (Aborted) ** 2024-09-21T07:37:40.190 INFO:tasks.ceph.mds.c.smithi039.stderr: in thread 7ff51ae00640 thread_name:ms_dispatch 2024-09-21T07:37:40.190 INFO:tasks.ceph.mds.c.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:37:40.190 INFO:tasks.ceph.mds.c.smithi039.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7ff52143e6f0] 2024-09-21T07:37:40.190 INFO:tasks.ceph.mds.c.smithi039.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7ff52148b99c] 2024-09-21T07:37:40.190 INFO:tasks.ceph.mds.c.smithi039.stderr: 3: raise() 2024-09-21T07:37:40.191 INFO:tasks.ceph.mds.c.smithi039.stderr: 4: abort() 2024-09-21T07:37:40.191 INFO:tasks.ceph.mds.c.smithi039.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7ff521e56686] 2024-09-21T07:37:40.191 INFO:tasks.ceph.mds.c.smithi039.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7ff521e56758] 2024-09-21T07:37:40.191 INFO:tasks.ceph.mds.c.smithi039.stderr: 7: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x3bd) [0x55f55fdf930b] 2024-09-21T07:37:40.191 INFO:tasks.ceph.mds.c.smithi039.stderr: 8: (MDSTableServer::handle_request(boost::intrusive_ptr const&)+0x9a) [0x55f55fdf9910] 2024-09-21T07:37:40.191 INFO:tasks.ceph.mds.c.smithi039.stderr: 9: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xca5) [0x55f55fa92de3] 2024-09-21T07:37:40.191 INFO:tasks.ceph.mds.c.smithi039.stderr: 10: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55f55fa96523] 2024-09-21T07:37:40.191 INFO:tasks.ceph.mds.c.smithi039.stderr: 11: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55f55fa96e4a] 2024-09-21T07:37:40.191 INFO:tasks.ceph.mds.c.smithi039.stderr: 12: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55f55fa79fa4] 2024-09-21T07:37:40.191 INFO:tasks.ceph.mds.c.smithi039.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7ff521fa57df] 2024-09-21T07:37:40.191 INFO:tasks.ceph.mds.c.smithi039.stderr: 14: (DispatchQueue::entry()+0x60f) [0x7ff521fa60d1] 2024-09-21T07:37:40.191 INFO:tasks.ceph.mds.c.smithi039.stderr: 15: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7ff522093f5b] 2024-09-21T07:37:40.191 INFO:tasks.ceph.mds.c.smithi039.stderr: 16: (Thread::entry_wrapper()+0x43) [0x7ff521e2fd57] 2024-09-21T07:37:40.191 INFO:tasks.ceph.mds.c.smithi039.stderr: 17: (Thread::_entry_func(void*)+0xd) [0x7ff521e2fd73] 2024-09-21T07:37:40.192 INFO:tasks.ceph.mds.c.smithi039.stderr: 18: /lib64/libc.so.6(+0x89c52) [0x7ff521489c52] 2024-09-21T07:37:40.192 INFO:tasks.ceph.mds.c.smithi039.stderr: 19: /lib64/libc.so.6(+0x10ec80) [0x7ff52150ec80] 2024-09-21T07:37:40.192 INFO:tasks.ceph.mds.c.smithi039.stderr:2024-09-21T07:37:40.185+0000 7ff51ae00640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:37:40.192 INFO:tasks.ceph.mds.c.smithi039.stderr: in thread 7ff51ae00640 thread_name:ms_dispatch 2024-09-21T07:37:40.192 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:37:40.192 INFO:tasks.ceph.mds.c.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:37:40.192 INFO:tasks.ceph.mds.c.smithi039.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7ff52143e6f0] 2024-09-21T07:37:40.192 INFO:tasks.ceph.mds.c.smithi039.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7ff52148b99c] 2024-09-21T07:37:40.192 INFO:tasks.ceph.mds.c.smithi039.stderr: 3: raise() 2024-09-21T07:37:40.192 INFO:tasks.ceph.mds.c.smithi039.stderr: 4: abort() 2024-09-21T07:37:40.192 INFO:tasks.ceph.mds.c.smithi039.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7ff521e56686] 2024-09-21T07:37:40.192 INFO:tasks.ceph.mds.c.smithi039.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7ff521e56758] 2024-09-21T07:37:40.192 INFO:tasks.ceph.mds.c.smithi039.stderr: 7: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x3bd) [0x55f55fdf930b] 2024-09-21T07:37:40.192 INFO:tasks.ceph.mds.c.smithi039.stderr: 8: (MDSTableServer::handle_request(boost::intrusive_ptr const&)+0x9a) [0x55f55fdf9910] 2024-09-21T07:37:40.193 INFO:tasks.ceph.mds.c.smithi039.stderr: 9: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xca5) [0x55f55fa92de3] 2024-09-21T07:37:40.193 INFO:tasks.ceph.mds.c.smithi039.stderr: 10: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55f55fa96523] 2024-09-21T07:37:40.193 INFO:tasks.ceph.mds.c.smithi039.stderr: 11: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55f55fa96e4a] 2024-09-21T07:37:40.193 INFO:tasks.ceph.mds.c.smithi039.stderr: 12: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55f55fa79fa4] 2024-09-21T07:37:40.193 INFO:tasks.ceph.mds.c.smithi039.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7ff521fa57df] 2024-09-21T07:37:40.193 INFO:tasks.ceph.mds.c.smithi039.stderr: 14: (DispatchQueue::entry()+0x60f) [0x7ff521fa60d1] 2024-09-21T07:37:40.193 INFO:tasks.ceph.mds.c.smithi039.stderr: 15: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7ff522093f5b] 2024-09-21T07:37:40.193 INFO:tasks.ceph.mds.c.smithi039.stderr: 16: (Thread::entry_wrapper()+0x43) [0x7ff521e2fd57] 2024-09-21T07:37:40.193 INFO:tasks.ceph.mds.c.smithi039.stderr: 17: (Thread::_entry_func(void*)+0xd) [0x7ff521e2fd73] 2024-09-21T07:37:40.193 INFO:tasks.ceph.mds.c.smithi039.stderr: 18: /lib64/libc.so.6(+0x89c52) [0x7ff521489c52] 2024-09-21T07:37:40.193 INFO:tasks.ceph.mds.c.smithi039.stderr: 19: /lib64/libc.so.6(+0x10ec80) [0x7ff52150ec80] 2024-09-21T07:37:40.193 INFO:tasks.ceph.mds.c.smithi039.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:37:40.194 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:37:40.195 INFO:tasks.ceph.mds.c.smithi039.stderr: -348> 2024-09-21T07:37:37.280+0000 7ff517c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:37:40.196 INFO:tasks.ceph.mds.c.smithi039.stderr: -346> 2024-09-21T07:37:37.280+0000 7ff517c00640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:37:40.196 INFO:tasks.ceph.mds.c.smithi039.stderr: -1> 2024-09-21T07:37:40.184+0000 7ff51ae00640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_commit(ceph::cref_t&)' thread 7ff51ae00640 time 2024-09-21T07:37:40.184370+0000 2024-09-21T07:37:40.196 INFO:tasks.ceph.mds.c.smithi039.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: 147: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 5) 2024-09-21T07:37:40.196 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:37:40.197 INFO:tasks.ceph.mds.c.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:37:40.197 INFO:tasks.ceph.mds.c.smithi039.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7ff521e5654b] 2024-09-21T07:37:40.197 INFO:tasks.ceph.mds.c.smithi039.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7ff521e56758] 2024-09-21T07:37:40.197 INFO:tasks.ceph.mds.c.smithi039.stderr: 3: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x3bd) [0x55f55fdf930b] 2024-09-21T07:37:40.197 INFO:tasks.ceph.mds.c.smithi039.stderr: 4: (MDSTableServer::handle_request(boost::intrusive_ptr const&)+0x9a) [0x55f55fdf9910] 2024-09-21T07:37:40.197 INFO:tasks.ceph.mds.c.smithi039.stderr: 5: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xca5) [0x55f55fa92de3] 2024-09-21T07:37:40.197 INFO:tasks.ceph.mds.c.smithi039.stderr: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55f55fa96523] 2024-09-21T07:37:40.197 INFO:tasks.ceph.mds.c.smithi039.stderr: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55f55fa96e4a] 2024-09-21T07:37:40.197 INFO:tasks.ceph.mds.c.smithi039.stderr: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55f55fa79fa4] 2024-09-21T07:37:40.197 INFO:tasks.ceph.mds.c.smithi039.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7ff521fa57df] 2024-09-21T07:37:40.197 INFO:tasks.ceph.mds.c.smithi039.stderr: 10: (DispatchQueue::entry()+0x60f) [0x7ff521fa60d1] 2024-09-21T07:37:40.197 INFO:tasks.ceph.mds.c.smithi039.stderr: 11: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7ff522093f5b] 2024-09-21T07:37:40.198 INFO:tasks.ceph.mds.c.smithi039.stderr: 12: (Thread::entry_wrapper()+0x43) [0x7ff521e2fd57] 2024-09-21T07:37:40.198 INFO:tasks.ceph.mds.c.smithi039.stderr: 13: (Thread::_entry_func(void*)+0xd) [0x7ff521e2fd73] 2024-09-21T07:37:40.198 INFO:tasks.ceph.mds.c.smithi039.stderr: 14: /lib64/libc.so.6(+0x89c52) [0x7ff521489c52] 2024-09-21T07:37:40.198 INFO:tasks.ceph.mds.c.smithi039.stderr: 15: /lib64/libc.so.6(+0x10ec80) [0x7ff52150ec80] 2024-09-21T07:37:40.198 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:37:40.198 INFO:tasks.ceph.mds.c.smithi039.stderr: 0> 2024-09-21T07:37:40.185+0000 7ff51ae00640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:37:40.198 INFO:tasks.ceph.mds.c.smithi039.stderr: in thread 7ff51ae00640 thread_name:ms_dispatch 2024-09-21T07:37:40.198 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:37:40.198 INFO:tasks.ceph.mds.c.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:37:40.198 INFO:tasks.ceph.mds.c.smithi039.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7ff52143e6f0] 2024-09-21T07:37:40.198 INFO:tasks.ceph.mds.c.smithi039.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7ff52148b99c] 2024-09-21T07:37:40.198 INFO:tasks.ceph.mds.c.smithi039.stderr: 3: raise() 2024-09-21T07:37:40.198 INFO:tasks.ceph.mds.c.smithi039.stderr: 4: abort() 2024-09-21T07:37:40.199 INFO:tasks.ceph.mds.c.smithi039.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7ff521e56686] 2024-09-21T07:37:40.199 INFO:tasks.ceph.mds.c.smithi039.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7ff521e56758] 2024-09-21T07:37:40.199 INFO:tasks.ceph.mds.c.smithi039.stderr: 7: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x3bd) [0x55f55fdf930b] 2024-09-21T07:37:40.199 INFO:tasks.ceph.mds.c.smithi039.stderr: 8: (MDSTableServer::handle_request(boost::intrusive_ptr const&)+0x9a) [0x55f55fdf9910] 2024-09-21T07:37:40.199 INFO:tasks.ceph.mds.c.smithi039.stderr: 9: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xca5) [0x55f55fa92de3] 2024-09-21T07:37:40.199 INFO:tasks.ceph.mds.c.smithi039.stderr: 10: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55f55fa96523] 2024-09-21T07:37:40.199 INFO:tasks.ceph.mds.c.smithi039.stderr: 11: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55f55fa96e4a] 2024-09-21T07:37:40.199 INFO:tasks.ceph.mds.c.smithi039.stderr: 12: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55f55fa79fa4] 2024-09-21T07:37:40.199 INFO:tasks.ceph.mds.c.smithi039.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7ff521fa57df] 2024-09-21T07:37:40.199 INFO:tasks.ceph.mds.c.smithi039.stderr: 14: (DispatchQueue::entry()+0x60f) [0x7ff521fa60d1] 2024-09-21T07:37:40.199 INFO:tasks.ceph.mds.c.smithi039.stderr: 15: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7ff522093f5b] 2024-09-21T07:37:40.199 INFO:tasks.ceph.mds.c.smithi039.stderr: 16: (Thread::entry_wrapper()+0x43) [0x7ff521e2fd57] 2024-09-21T07:37:40.200 INFO:tasks.ceph.mds.c.smithi039.stderr: 17: (Thread::_entry_func(void*)+0xd) [0x7ff521e2fd73] 2024-09-21T07:37:40.200 INFO:tasks.ceph.mds.c.smithi039.stderr: 18: /lib64/libc.so.6(+0x89c52) [0x7ff521489c52] 2024-09-21T07:37:40.200 INFO:tasks.ceph.mds.c.smithi039.stderr: 19: /lib64/libc.so.6(+0x10ec80) [0x7ff52150ec80] 2024-09-21T07:37:40.200 INFO:tasks.ceph.mds.c.smithi039.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:37:40.200 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:37:40.203 INFO:tasks.ceph.mds.c.smithi039.stderr: -348> 2024-09-21T07:37:37.280+0000 7ff517c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:37:40.203 INFO:tasks.ceph.mds.c.smithi039.stderr: -346> 2024-09-21T07:37:37.280+0000 7ff517c00640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:37:40.203 INFO:tasks.ceph.mds.c.smithi039.stderr: -1> 2024-09-21T07:37:40.184+0000 7ff51ae00640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_commit(ceph::cref_t&)' thread 7ff51ae00640 time 2024-09-21T07:37:40.184370+0000 2024-09-21T07:37:40.203 INFO:tasks.ceph.mds.c.smithi039.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: 147: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 5) 2024-09-21T07:37:40.203 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:37:40.203 INFO:tasks.ceph.mds.c.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:37:40.204 INFO:tasks.ceph.mds.c.smithi039.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7ff521e5654b] 2024-09-21T07:37:40.204 INFO:tasks.ceph.mds.c.smithi039.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7ff521e56758] 2024-09-21T07:37:40.204 INFO:tasks.ceph.mds.c.smithi039.stderr: 3: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x3bd) [0x55f55fdf930b] 2024-09-21T07:37:40.204 INFO:tasks.ceph.mds.c.smithi039.stderr: 4: (MDSTableServer::handle_request(boost::intrusive_ptr const&)+0x9a) [0x55f55fdf9910] 2024-09-21T07:37:40.204 INFO:tasks.ceph.mds.c.smithi039.stderr: 5: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xca5) [0x55f55fa92de3] 2024-09-21T07:37:40.204 INFO:tasks.ceph.mds.c.smithi039.stderr: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55f55fa96523] 2024-09-21T07:37:40.204 INFO:tasks.ceph.mds.c.smithi039.stderr: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55f55fa96e4a] 2024-09-21T07:37:40.204 INFO:tasks.ceph.mds.c.smithi039.stderr: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55f55fa79fa4] 2024-09-21T07:37:40.204 INFO:tasks.ceph.mds.c.smithi039.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7ff521fa57df] 2024-09-21T07:37:40.204 INFO:tasks.ceph.mds.c.smithi039.stderr: 10: (DispatchQueue::entry()+0x60f) [0x7ff521fa60d1] 2024-09-21T07:37:40.204 INFO:tasks.ceph.mds.c.smithi039.stderr: 11: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7ff522093f5b] 2024-09-21T07:37:40.204 INFO:tasks.ceph.mds.c.smithi039.stderr: 12: (Thread::entry_wrapper()+0x43) [0x7ff521e2fd57] 2024-09-21T07:37:40.204 INFO:tasks.ceph.mds.c.smithi039.stderr: 13: (Thread::_entry_func(void*)+0xd) [0x7ff521e2fd73] 2024-09-21T07:37:40.204 INFO:tasks.ceph.mds.c.smithi039.stderr: 14: /lib64/libc.so.6(+0x89c52) [0x7ff521489c52] 2024-09-21T07:37:40.204 INFO:tasks.ceph.mds.c.smithi039.stderr: 15: /lib64/libc.so.6(+0x10ec80) [0x7ff52150ec80] 2024-09-21T07:37:40.205 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:37:40.205 INFO:tasks.ceph.mds.c.smithi039.stderr: 0> 2024-09-21T07:37:40.185+0000 7ff51ae00640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:37:40.205 INFO:tasks.ceph.mds.c.smithi039.stderr: in thread 7ff51ae00640 thread_name:ms_dispatch 2024-09-21T07:37:40.205 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:37:40.205 INFO:tasks.ceph.mds.c.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:37:40.205 INFO:tasks.ceph.mds.c.smithi039.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7ff52143e6f0] 2024-09-21T07:37:40.205 INFO:tasks.ceph.mds.c.smithi039.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7ff52148b99c] 2024-09-21T07:37:40.205 INFO:tasks.ceph.mds.c.smithi039.stderr: 3: raise() 2024-09-21T07:37:40.205 INFO:tasks.ceph.mds.c.smithi039.stderr: 4: abort() 2024-09-21T07:37:40.205 INFO:tasks.ceph.mds.c.smithi039.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7ff521e56686] 2024-09-21T07:37:40.205 INFO:tasks.ceph.mds.c.smithi039.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7ff521e56758] 2024-09-21T07:37:40.205 INFO:tasks.ceph.mds.c.smithi039.stderr: 7: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x3bd) [0x55f55fdf930b] 2024-09-21T07:37:40.205 INFO:tasks.ceph.mds.c.smithi039.stderr: 8: (MDSTableServer::handle_request(boost::intrusive_ptr const&)+0x9a) [0x55f55fdf9910] 2024-09-21T07:37:40.205 INFO:tasks.ceph.mds.c.smithi039.stderr: 9: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xca5) [0x55f55fa92de3] 2024-09-21T07:37:40.206 INFO:tasks.ceph.mds.c.smithi039.stderr: 10: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55f55fa96523] 2024-09-21T07:37:40.206 INFO:tasks.ceph.mds.c.smithi039.stderr: 11: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55f55fa96e4a] 2024-09-21T07:37:40.206 INFO:tasks.ceph.mds.c.smithi039.stderr: 12: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55f55fa79fa4] 2024-09-21T07:37:40.206 INFO:tasks.ceph.mds.c.smithi039.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7ff521fa57df] 2024-09-21T07:37:40.206 INFO:tasks.ceph.mds.c.smithi039.stderr: 14: (DispatchQueue::entry()+0x60f) [0x7ff521fa60d1] 2024-09-21T07:37:40.206 INFO:tasks.ceph.mds.c.smithi039.stderr: 15: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7ff522093f5b] 2024-09-21T07:37:40.206 INFO:tasks.ceph.mds.c.smithi039.stderr: 16: (Thread::entry_wrapper()+0x43) [0x7ff521e2fd57] 2024-09-21T07:37:40.206 INFO:tasks.ceph.mds.c.smithi039.stderr: 17: (Thread::_entry_func(void*)+0xd) [0x7ff521e2fd73] 2024-09-21T07:37:40.206 INFO:tasks.ceph.mds.c.smithi039.stderr: 18: /lib64/libc.so.6(+0x89c52) [0x7ff521489c52] 2024-09-21T07:37:40.206 INFO:tasks.ceph.mds.c.smithi039.stderr: 19: /lib64/libc.so.6(+0x10ec80) [0x7ff52150ec80] 2024-09-21T07:37:40.206 INFO:tasks.ceph.mds.c.smithi039.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:37:40.206 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:37:40.422 INFO:teuthology.orchestra.run.smithi039.stdout:{"mds_beacon_grace":"15.000000"} 2024-09-21T07:37:40.423 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-09-21T07:37:40.423 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:37:40.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.651+0000 7fdc26a00640 1 -- 172.21.15.39:0/1741622731 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc200cf320 msgr2=0x7fdc200d0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:40.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.651+0000 7fdc26a00640 1 --2- 172.21.15.39:0/1741622731 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc200cf320 0x7fdc200d0980 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fdc18005fd0 tx=0x7fdc1802a8b0 comp rx=0 tx=0).stop 2024-09-21T07:37:40.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.651+0000 7fdc26a00640 1 -- 172.21.15.39:0/1741622731 shutdown_connections 2024-09-21T07:37:40.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.651+0000 7fdc26a00640 1 --2- 172.21.15.39:0/1741622731 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc200cf320 0x7fdc200d0980 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:40.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.651+0000 7fdc26a00640 1 --2- 172.21.15.39:0/1741622731 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdc201600b0 0x7fdc200ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:40.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.651+0000 7fdc26a00640 1 -- 172.21.15.39:0/1741622731 >> 172.21.15.39:0/1741622731 conn(0x7fdc2006d4f0 msgr2=0x7fdc20071130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:40.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.652+0000 7fdc26a00640 1 -- 172.21.15.39:0/1741622731 shutdown_connections 2024-09-21T07:37:40.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.652+0000 7fdc26a00640 1 -- 172.21.15.39:0/1741622731 wait complete. 2024-09-21T07:37:40.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.654+0000 7fdc26a00640 1 Processor -- start 2024-09-21T07:37:40.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.654+0000 7fdc26a00640 1 -- start start 2024-09-21T07:37:40.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.655+0000 7fdc26a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdc200b7cc0 0x7fdc200c9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:40.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.655+0000 7fdc26a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc200cf320 0x7fdc200cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:40.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.655+0000 7fdc26a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc201600b0 0x7fdc200b3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:40.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.655+0000 7fdc26a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdc200d1410 con 0x7fdc200cf320 2024-09-21T07:37:40.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.655+0000 7fdc26a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdc200d1290 con 0x7fdc201600b0 2024-09-21T07:37:40.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.655+0000 7fdc26a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fdc200d1590 con 0x7fdc200b7cc0 2024-09-21T07:37:40.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.656+0000 7fdc1f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc200cf320 0x7fdc200cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:40.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.656+0000 7fdc1f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc200cf320 0x7fdc200cab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40970/0 (socket says 172.21.15.39:40970) 2024-09-21T07:37:40.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.656+0000 7fdc1f400640 1 -- 172.21.15.39:0/3928147811 learned_addr learned my addr 172.21.15.39:0/3928147811 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:40.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.656+0000 7fdc1fe00640 1 --2- 172.21.15.39:0/3928147811 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdc200b7cc0 0x7fdc200c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:40.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.656+0000 7fdc24a00640 1 --2- 172.21.15.39:0/3928147811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc201600b0 0x7fdc200b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:40.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.656+0000 7fdc1f400640 1 -- 172.21.15.39:0/3928147811 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdc200b7cc0 msgr2=0x7fdc200c9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:40.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.656+0000 7fdc1f400640 1 --2- 172.21.15.39:0/3928147811 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdc200b7cc0 0x7fdc200c9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:40.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.656+0000 7fdc1f400640 1 -- 172.21.15.39:0/3928147811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc201600b0 msgr2=0x7fdc200b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:40.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.656+0000 7fdc1f400640 1 --2- 172.21.15.39:0/3928147811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc201600b0 0x7fdc200b3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:40.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.656+0000 7fdc1f400640 1 -- 172.21.15.39:0/3928147811 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc201f4240 con 0x7fdc200cf320 2024-09-21T07:37:40.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.656+0000 7fdc24a00640 1 --2- 172.21.15.39:0/3928147811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc201600b0 0x7fdc200b3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:37:40.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.657+0000 7fdc1f400640 1 --2- 172.21.15.39:0/3928147811 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc200cf320 0x7fdc200cab40 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00c3c0 tx=0x7fdc0c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:40.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.657+0000 7fdc17e00640 1 -- 172.21.15.39:0/3928147811 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c011790 con 0x7fdc200cf320 2024-09-21T07:37:40.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.657+0000 7fdc17e00640 1 -- 172.21.15.39:0/3928147811 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc0c017070 con 0x7fdc200cf320 2024-09-21T07:37:40.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.657+0000 7fdc26a00640 1 -- 172.21.15.39:0/3928147811 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc201f4520 con 0x7fdc200cf320 2024-09-21T07:37:40.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.657+0000 7fdc17e00640 1 -- 172.21.15.39:0/3928147811 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c011cc0 con 0x7fdc200cf320 2024-09-21T07:37:40.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.657+0000 7fdc26a00640 1 -- 172.21.15.39:0/3928147811 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc200b2f20 con 0x7fdc200cf320 2024-09-21T07:37:40.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.658+0000 7fdc26a00640 1 -- 172.21.15.39:0/3928147811 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc201f4a40 con 0x7fdc200cf320 2024-09-21T07:37:40.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.659+0000 7fdc17e00640 1 -- 172.21.15.39:0/3928147811 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdc0c01c460 con 0x7fdc200cf320 2024-09-21T07:37:40.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.660+0000 7fdc17e00640 1 --2- 172.21.15.39:0/3928147811 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdc00078bc0 0x7fdc0007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:40.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.660+0000 7fdc17e00640 1 -- 172.21.15.39:0/3928147811 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 7001+0+0 (secure 0 0 0) 0x7fdc0c09af00 con 0x7fdc200cf320 2024-09-21T07:37:40.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.661+0000 7fdc1fe00640 1 --2- 172.21.15.39:0/3928147811 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdc00078bc0 0x7fdc0007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:40.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.662+0000 7fdc1fe00640 1 --2- 172.21.15.39:0/3928147811 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdc00078bc0 0x7fdc0007b0d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fdc10009b30 tx=0x7fdc10005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:40.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.665+0000 7fdc17e00640 1 -- 172.21.15.39:0/3928147811 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdc201f4a40 con 0x7fdc200cf320 2024-09-21T07:37:40.854 INFO:tasks.ceph.mds.c.smithi039.stderr:daemon-helper: command crashed with signal 6 2024-09-21T07:37:40.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.909+0000 7fdc26a00640 1 -- 172.21.15.39:0/3928147811 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc200d1b20 con 0x7fdc200cf320 2024-09-21T07:37:40.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.910+0000 7fdc17e00640 1 -- 172.21.15.39:0/3928147811 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5249 (secure 0 0 0) 0x7fdc0c062e20 con 0x7fdc200cf320 2024-09-21T07:37:40.912 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:37:40.912 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":75,"btime":"2024-09-21T07:37:39:842222+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5870,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2381504293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2381504293},{"type":"v1","addr":"172.21.15.148:6837","nonce":2381504293}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:37:39.135025+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6585,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6585":{"gid":6585,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6834/1472430787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1472430787},{"type":"v1","addr":"172.21.15.39:6834","nonce":1472430787}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6585,"qdb_cluster":[6585,6115]},"id":6}]} 2024-09-21T07:37:40.912 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 75 2024-09-21T07:37:40.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.912+0000 7fdc26a00640 1 -- 172.21.15.39:0/3928147811 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdc00078bc0 msgr2=0x7fdc0007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:40.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.912+0000 7fdc26a00640 1 --2- 172.21.15.39:0/3928147811 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdc00078bc0 0x7fdc0007b0d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fdc10009b30 tx=0x7fdc10005f90 comp rx=0 tx=0).stop 2024-09-21T07:37:40.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.912+0000 7fdc26a00640 1 -- 172.21.15.39:0/3928147811 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc200cf320 msgr2=0x7fdc200cab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:40.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.912+0000 7fdc26a00640 1 --2- 172.21.15.39:0/3928147811 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc200cf320 0x7fdc200cab40 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00c3c0 tx=0x7fdc0c00c880 comp rx=0 tx=0).stop 2024-09-21T07:37:40.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.912+0000 7fdc26a00640 1 -- 172.21.15.39:0/3928147811 shutdown_connections 2024-09-21T07:37:40.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.912+0000 7fdc26a00640 1 --2- 172.21.15.39:0/3928147811 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdc00078bc0 0x7fdc0007b0d0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:40.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.912+0000 7fdc26a00640 1 --2- 172.21.15.39:0/3928147811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc201600b0 0x7fdc200b3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:40.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.912+0000 7fdc26a00640 1 --2- 172.21.15.39:0/3928147811 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc200cf320 0x7fdc200cab40 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:40.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.912+0000 7fdc26a00640 1 --2- 172.21.15.39:0/3928147811 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdc200b7cc0 0x7fdc200c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:40.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.912+0000 7fdc26a00640 1 -- 172.21.15.39:0/3928147811 >> 172.21.15.39:0/3928147811 conn(0x7fdc2006d4f0 msgr2=0x7fdc2015a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:40.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.912+0000 7fdc26a00640 1 -- 172.21.15.39:0/3928147811 shutdown_connections 2024-09-21T07:37:40.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:40.912+0000 7fdc26a00640 1 -- 172.21.15.39:0/3928147811 wait complete. 2024-09-21T07:37:40.936 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:37:43.018 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-09-21T07:37:45.937 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:37:46.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.174+0000 7f60d3200640 1 -- 172.21.15.39:0/3774391648 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f60d415b8f0 msgr2=0x7f60d4073160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:46.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.174+0000 7f60d3200640 1 --2- 172.21.15.39:0/3774391648 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f60d415b8f0 0x7f60d4073160 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f60c40092e0 tx=0x7f60c402a440 comp rx=0 tx=0).stop 2024-09-21T07:37:46.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.174+0000 7f60d3200640 1 -- 172.21.15.39:0/3774391648 shutdown_connections 2024-09-21T07:37:46.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.174+0000 7f60d3200640 1 --2- 172.21.15.39:0/3774391648 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f60d415b8f0 0x7f60d4073160 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:46.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.174+0000 7f60d3200640 1 -- 172.21.15.39:0/3774391648 >> 172.21.15.39:0/3774391648 conn(0x7f60d406d410 msgr2=0x7f60d40b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:46.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.174+0000 7f60d3200640 1 -- 172.21.15.39:0/3774391648 shutdown_connections 2024-09-21T07:37:46.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.174+0000 7f60d3200640 1 -- 172.21.15.39:0/3774391648 wait complete. 2024-09-21T07:37:46.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.176+0000 7f60d3200640 1 Processor -- start 2024-09-21T07:37:46.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.176+0000 7f60d3200640 1 -- start start 2024-09-21T07:37:46.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.178+0000 7f60d3200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f60d4151640 0x7f60d41f7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:46.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.178+0000 7f60d3200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60d415b8f0 0x7f60d4151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:46.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.178+0000 7f60d3200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60d4152310 0x7f60d41487d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:46.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.178+0000 7f60d3200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f60d40b4560 con 0x7f60d415b8f0 2024-09-21T07:37:46.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.178+0000 7f60d3200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f60d40b43e0 con 0x7f60d4152310 2024-09-21T07:37:46.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.178+0000 7f60d3200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f60d40b46e0 con 0x7f60d4151640 2024-09-21T07:37:46.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.178+0000 7f60cbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60d415b8f0 0x7f60d4151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:46.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.178+0000 7f60cbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60d415b8f0 0x7f60d4151dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58694/0 (socket says 172.21.15.39:58694) 2024-09-21T07:37:46.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.178+0000 7f60cbe00640 1 -- 172.21.15.39:0/4172995829 learned_addr learned my addr 172.21.15.39:0/4172995829 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:46.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.178+0000 7f60d1e00640 1 --2- 172.21.15.39:0/4172995829 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f60d4151640 0x7f60d41f7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:46.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.178+0000 7f60d2800640 1 --2- 172.21.15.39:0/4172995829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60d4152310 0x7f60d41487d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:46.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.178+0000 7f60cbe00640 1 -- 172.21.15.39:0/4172995829 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f60d4151640 msgr2=0x7f60d41f7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:46.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.178+0000 7f60cbe00640 1 --2- 172.21.15.39:0/4172995829 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f60d4151640 0x7f60d41f7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:46.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.178+0000 7f60cbe00640 1 -- 172.21.15.39:0/4172995829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60d4152310 msgr2=0x7f60d41487d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:46.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.178+0000 7f60cbe00640 1 --2- 172.21.15.39:0/4172995829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60d4152310 0x7f60d41487d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:46.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.179+0000 7f60cbe00640 1 -- 172.21.15.39:0/4172995829 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d41f6890 con 0x7f60d415b8f0 2024-09-21T07:37:46.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.179+0000 7f60d1e00640 1 --2- 172.21.15.39:0/4172995829 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f60d4151640 0x7f60d41f7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:37:46.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.179+0000 7f60d2800640 1 --2- 172.21.15.39:0/4172995829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60d4152310 0x7f60d41487d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:37:46.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.179+0000 7f60cbe00640 1 --2- 172.21.15.39:0/4172995829 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60d415b8f0 0x7f60d4151dd0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f60bc00c3f0 tx=0x7f60bc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:46.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.179+0000 7f60ca000640 1 -- 172.21.15.39:0/4172995829 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc017070 con 0x7f60d415b8f0 2024-09-21T07:37:46.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.179+0000 7f60ca000640 1 -- 172.21.15.39:0/4172995829 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60bc0035f0 con 0x7f60d415b8f0 2024-09-21T07:37:46.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.179+0000 7f60ca000640 1 -- 172.21.15.39:0/4172995829 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc003a40 con 0x7f60d415b8f0 2024-09-21T07:37:46.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.179+0000 7f60d3200640 1 -- 172.21.15.39:0/4172995829 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60d41f33c0 con 0x7f60d415b8f0 2024-09-21T07:37:46.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.179+0000 7f60d3200640 1 -- 172.21.15.39:0/4172995829 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60d41f3980 con 0x7f60d415b8f0 2024-09-21T07:37:46.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.180+0000 7f60d3200640 1 -- 172.21.15.39:0/4172995829 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60d40d02c0 con 0x7f60d415b8f0 2024-09-21T07:37:46.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.181+0000 7f60ca000640 1 -- 172.21.15.39:0/4172995829 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f60bc01a070 con 0x7f60d415b8f0 2024-09-21T07:37:46.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.182+0000 7f60ca000640 1 --2- 172.21.15.39:0/4172995829 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f60b0078b90 0x7f60b007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:46.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.183+0000 7f60ca000640 1 -- 172.21.15.39:0/4172995829 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 7001+0+0 (secure 0 0 0) 0x7f60bc09a4b0 con 0x7f60d415b8f0 2024-09-21T07:37:46.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.183+0000 7f60d1e00640 1 --2- 172.21.15.39:0/4172995829 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f60b0078b90 0x7f60b007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:46.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.184+0000 7f60d1e00640 1 --2- 172.21.15.39:0/4172995829 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f60b0078b90 0x7f60b007b0a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f60c4034000 tx=0x7f60c4004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:46.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.187+0000 7f60ca000640 1 -- 172.21.15.39:0/4172995829 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f60bc062450 con 0x7f60d415b8f0 2024-09-21T07:37:46.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.422+0000 7f60d3200640 1 -- 172.21.15.39:0/4172995829 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60d41f7650 con 0x7f60d415b8f0 2024-09-21T07:37:46.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.423+0000 7f60ca000640 1 -- 172.21.15.39:0/4172995829 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5249 (secure 0 0 0) 0x7f60bc061d50 con 0x7f60d415b8f0 2024-09-21T07:37:46.425 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:37:46.425 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":75,"btime":"2024-09-21T07:37:39:842222+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5870,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2381504293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2381504293},{"type":"v1","addr":"172.21.15.148:6837","nonce":2381504293}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:37:39.135025+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6585,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6585":{"gid":6585,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6834/1472430787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1472430787},{"type":"v1","addr":"172.21.15.39:6834","nonce":1472430787}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6585,"qdb_cluster":[6585,6115]},"id":6}]} 2024-09-21T07:37:46.426 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 75 2024-09-21T07:37:46.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.425+0000 7f60d3200640 1 -- 172.21.15.39:0/4172995829 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f60b0078b90 msgr2=0x7f60b007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:46.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.425+0000 7f60d3200640 1 --2- 172.21.15.39:0/4172995829 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f60b0078b90 0x7f60b007b0a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f60c4034000 tx=0x7f60c4004730 comp rx=0 tx=0).stop 2024-09-21T07:37:46.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.425+0000 7f60d3200640 1 -- 172.21.15.39:0/4172995829 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60d415b8f0 msgr2=0x7f60d4151dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:46.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.425+0000 7f60d3200640 1 --2- 172.21.15.39:0/4172995829 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60d415b8f0 0x7f60d4151dd0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f60bc00c3f0 tx=0x7f60bc00c8b0 comp rx=0 tx=0).stop 2024-09-21T07:37:46.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.425+0000 7f60d3200640 1 -- 172.21.15.39:0/4172995829 shutdown_connections 2024-09-21T07:37:46.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.425+0000 7f60d3200640 1 --2- 172.21.15.39:0/4172995829 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f60b0078b90 0x7f60b007b0a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:46.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.425+0000 7f60d3200640 1 --2- 172.21.15.39:0/4172995829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60d4152310 0x7f60d41487d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:46.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.425+0000 7f60d3200640 1 --2- 172.21.15.39:0/4172995829 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60d415b8f0 0x7f60d4151dd0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:46.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.425+0000 7f60d3200640 1 --2- 172.21.15.39:0/4172995829 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f60d4151640 0x7f60d41f7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:46.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.425+0000 7f60d3200640 1 -- 172.21.15.39:0/4172995829 >> 172.21.15.39:0/4172995829 conn(0x7f60d406d410 msgr2=0x7f60d414f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:46.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.426+0000 7f60d3200640 1 -- 172.21.15.39:0/4172995829 shutdown_connections 2024-09-21T07:37:46.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:46.426+0000 7f60d3200640 1 -- 172.21.15.39:0/4172995829 wait complete. 2024-09-21T07:37:46.452 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:37:49.630 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~7s 2024-09-21T07:37:51.452 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:37:51.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.687+0000 7fea2da00640 1 -- 172.21.15.39:0/2626609733 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea280706b0 msgr2=0x7fea280735b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:51.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.687+0000 7fea2da00640 1 --2- 172.21.15.39:0/2626609733 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea280706b0 0x7fea280735b0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fea20005fd0 tx=0x7fea2002a8b0 comp rx=0 tx=0).stop 2024-09-21T07:37:51.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.687+0000 7fea2da00640 1 -- 172.21.15.39:0/2626609733 shutdown_connections 2024-09-21T07:37:51.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.687+0000 7fea2da00640 1 --2- 172.21.15.39:0/2626609733 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea280706b0 0x7fea280735b0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:51.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.687+0000 7fea2da00640 1 --2- 172.21.15.39:0/2626609733 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea280ca800 0x7fea280cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:51.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.687+0000 7fea2da00640 1 -- 172.21.15.39:0/2626609733 >> 172.21.15.39:0/2626609733 conn(0x7fea280b1cb0 msgr2=0x7fea280b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:51.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.687+0000 7fea2da00640 1 -- 172.21.15.39:0/2626609733 shutdown_connections 2024-09-21T07:37:51.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.687+0000 7fea2da00640 1 -- 172.21.15.39:0/2626609733 wait complete. 2024-09-21T07:37:51.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.689+0000 7fea2da00640 1 Processor -- start 2024-09-21T07:37:51.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.690+0000 7fea2da00640 1 -- start start 2024-09-21T07:37:51.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.691+0000 7fea2da00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea280ca800 0x7fea2815e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:51.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.691+0000 7fea2da00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea280cad30 0x7fea280c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:51.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.691+0000 7fea2da00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea281f3c70 0x7fea2815d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:51.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.691+0000 7fea2da00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fea28073f00 con 0x7fea281f3c70 2024-09-21T07:37:51.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.691+0000 7fea2da00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fea28073d80 con 0x7fea280ca800 2024-09-21T07:37:51.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.691+0000 7fea2da00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fea28074080 con 0x7fea280cad30 2024-09-21T07:37:51.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.691+0000 7fea27400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea281f3c70 0x7fea2815d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:51.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.691+0000 7fea27400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea281f3c70 0x7fea2815d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58712/0 (socket says 172.21.15.39:58712) 2024-09-21T07:37:51.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.691+0000 7fea27400640 1 -- 172.21.15.39:0/681309015 learned_addr learned my addr 172.21.15.39:0/681309015 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:51.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.691+0000 7fea26a00640 1 --2- 172.21.15.39:0/681309015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea280ca800 0x7fea2815e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:51.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.691+0000 7fea1fe00640 1 --2- 172.21.15.39:0/681309015 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea280cad30 0x7fea280c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:51.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.692+0000 7fea27400640 1 -- 172.21.15.39:0/681309015 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea280cad30 msgr2=0x7fea280c9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:51.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.692+0000 7fea27400640 1 --2- 172.21.15.39:0/681309015 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea280cad30 0x7fea280c9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:51.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.692+0000 7fea27400640 1 -- 172.21.15.39:0/681309015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea280ca800 msgr2=0x7fea2815e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:51.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.692+0000 7fea27400640 1 --2- 172.21.15.39:0/681309015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea280ca800 0x7fea2815e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:51.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.692+0000 7fea27400640 1 -- 172.21.15.39:0/681309015 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea280c7dd0 con 0x7fea281f3c70 2024-09-21T07:37:51.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.692+0000 7fea26a00640 1 --2- 172.21.15.39:0/681309015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea280ca800 0x7fea2815e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:37:51.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.692+0000 7fea27400640 1 --2- 172.21.15.39:0/681309015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea281f3c70 0x7fea2815d460 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fea20009cd0 tx=0x7fea200316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:51.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.692+0000 7fea1ea00640 1 -- 172.21.15.39:0/681309015 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea20031da0 con 0x7fea281f3c70 2024-09-21T07:37:51.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.692+0000 7fea1ea00640 1 -- 172.21.15.39:0/681309015 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea20003a60 con 0x7fea281f3c70 2024-09-21T07:37:51.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.692+0000 7fea2da00640 1 -- 172.21.15.39:0/681309015 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea281f3190 con 0x7fea281f3c70 2024-09-21T07:37:51.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.692+0000 7fea1ea00640 1 -- 172.21.15.39:0/681309015 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea20033070 con 0x7fea281f3c70 2024-09-21T07:37:51.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.692+0000 7fea2da00640 1 -- 172.21.15.39:0/681309015 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea281f1820 con 0x7fea281f3c70 2024-09-21T07:37:51.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.694+0000 7fea2da00640 1 -- 172.21.15.39:0/681309015 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea280c7b40 con 0x7fea281f3c70 2024-09-21T07:37:51.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.695+0000 7fea1ea00640 1 -- 172.21.15.39:0/681309015 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fea20033210 con 0x7fea281f3c70 2024-09-21T07:37:51.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.696+0000 7fea1ea00640 1 --2- 172.21.15.39:0/681309015 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fea04078b90 0x7fea0407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:51.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.696+0000 7fea1ea00640 1 -- 172.21.15.39:0/681309015 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 7001+0+0 (secure 0 0 0) 0x7fea200bc740 con 0x7fea281f3c70 2024-09-21T07:37:51.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.696+0000 7fea26a00640 1 --2- 172.21.15.39:0/681309015 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fea04078b90 0x7fea0407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:51.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.697+0000 7fea26a00640 1 --2- 172.21.15.39:0/681309015 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fea04078b90 0x7fea0407b0a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fea280cb250 tx=0x7fea18005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:51.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.701+0000 7fea1ea00640 1 -- 172.21.15.39:0/681309015 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fea2003f030 con 0x7fea281f3c70 2024-09-21T07:37:51.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.936+0000 7fea2da00640 1 -- 172.21.15.39:0/681309015 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea281f1a80 con 0x7fea281f3c70 2024-09-21T07:37:51.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.937+0000 7fea1ea00640 1 -- 172.21.15.39:0/681309015 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5249 (secure 0 0 0) 0x7fea20084660 con 0x7fea281f3c70 2024-09-21T07:37:51.939 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:37:51.939 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":75,"btime":"2024-09-21T07:37:39:842222+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5870,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2381504293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2381504293},{"type":"v1","addr":"172.21.15.148:6837","nonce":2381504293}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:37:39.135025+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6585,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6585":{"gid":6585,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6834/1472430787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1472430787},{"type":"v1","addr":"172.21.15.39:6834","nonce":1472430787}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6585,"qdb_cluster":[6585,6115]},"id":6}]} 2024-09-21T07:37:51.939 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 75 2024-09-21T07:37:51.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.939+0000 7fea2da00640 1 -- 172.21.15.39:0/681309015 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fea04078b90 msgr2=0x7fea0407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:51.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.939+0000 7fea2da00640 1 --2- 172.21.15.39:0/681309015 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fea04078b90 0x7fea0407b0a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fea280cb250 tx=0x7fea18005f90 comp rx=0 tx=0).stop 2024-09-21T07:37:51.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.939+0000 7fea2da00640 1 -- 172.21.15.39:0/681309015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea281f3c70 msgr2=0x7fea2815d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:51.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.939+0000 7fea2da00640 1 --2- 172.21.15.39:0/681309015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea281f3c70 0x7fea2815d460 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fea20009cd0 tx=0x7fea200316f0 comp rx=0 tx=0).stop 2024-09-21T07:37:51.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.939+0000 7fea2da00640 1 -- 172.21.15.39:0/681309015 shutdown_connections 2024-09-21T07:37:51.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.939+0000 7fea2da00640 1 --2- 172.21.15.39:0/681309015 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fea04078b90 0x7fea0407b0a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:51.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.939+0000 7fea2da00640 1 --2- 172.21.15.39:0/681309015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea281f3c70 0x7fea2815d460 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:51.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.939+0000 7fea2da00640 1 --2- 172.21.15.39:0/681309015 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea280cad30 0x7fea280c9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:51.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.939+0000 7fea2da00640 1 --2- 172.21.15.39:0/681309015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea280ca800 0x7fea2815e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:51.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.939+0000 7fea2da00640 1 -- 172.21.15.39:0/681309015 >> 172.21.15.39:0/681309015 conn(0x7fea280b1cb0 msgr2=0x7fea280d0010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:51.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.940+0000 7fea2da00640 1 -- 172.21.15.39:0/681309015 shutdown_connections 2024-09-21T07:37:51.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:51.940+0000 7fea2da00640 1 -- 172.21.15.39:0/681309015 wait complete. 2024-09-21T07:37:51.970 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:37:56.241 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~13s 2024-09-21T07:37:56.971 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:37:57.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.207+0000 7faddbe00640 1 -- 172.21.15.39:0/3640065237 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faddc0cf200 msgr2=0x7faddc15ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:57.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.207+0000 7faddbe00640 1 --2- 172.21.15.39:0/3640065237 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faddc0cf200 0x7faddc15ba00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fadc4005fd0 tx=0x7fadc402a5d0 comp rx=0 tx=0).stop 2024-09-21T07:37:57.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.207+0000 7faddbe00640 1 -- 172.21.15.39:0/3640065237 shutdown_connections 2024-09-21T07:37:57.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.207+0000 7faddbe00640 1 --2- 172.21.15.39:0/3640065237 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faddc0cf200 0x7faddc15ba00 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:57.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.207+0000 7faddbe00640 1 --2- 172.21.15.39:0/3640065237 >> v2:172.21.15.148:3300/0 conn(0x7faddc0d0500 0x7faddc0cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:57.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.207+0000 7faddbe00640 1 -- 172.21.15.39:0/3640065237 >> 172.21.15.39:0/3640065237 conn(0x7faddc06d870 msgr2=0x7faddc06ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:57.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.207+0000 7faddbe00640 1 -- 172.21.15.39:0/3640065237 shutdown_connections 2024-09-21T07:37:57.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.208+0000 7faddbe00640 1 -- 172.21.15.39:0/3640065237 wait complete. 2024-09-21T07:37:57.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.210+0000 7faddbe00640 1 Processor -- start 2024-09-21T07:37:57.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.210+0000 7faddbe00640 1 -- start start 2024-09-21T07:37:57.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.211+0000 7faddbe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faddc0cf200 0x7faddc1f5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:57.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.211+0000 7faddbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faddc0d0500 0x7faddc0ca840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:57.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.211+0000 7faddbe00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faddc15bf40 0x7faddc0ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:57.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.211+0000 7faddbe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7faddc0cea10 con 0x7faddc0d0500 2024-09-21T07:37:57.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.211+0000 7faddbe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7faddc15c560 con 0x7faddc0cf200 2024-09-21T07:37:57.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.211+0000 7faddbe00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7faddc15f700 con 0x7faddc15bf40 2024-09-21T07:37:57.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.212+0000 7fadd3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faddc0d0500 0x7faddc0ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:57.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.212+0000 7fadd3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faddc0d0500 0x7faddc0ca840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53678/0 (socket says 172.21.15.39:53678) 2024-09-21T07:37:57.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.212+0000 7fadd3e00640 1 -- 172.21.15.39:0/363960700 learned_addr learned my addr 172.21.15.39:0/363960700 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:57.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.212+0000 7faddaa00640 1 --2- 172.21.15.39:0/363960700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faddc0cf200 0x7faddc1f5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:57.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.212+0000 7faddb400640 1 --2- 172.21.15.39:0/363960700 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faddc15bf40 0x7faddc0ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:57.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.212+0000 7fadd3e00640 1 -- 172.21.15.39:0/363960700 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faddc15bf40 msgr2=0x7faddc0ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:57.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.212+0000 7fadd3e00640 1 --2- 172.21.15.39:0/363960700 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faddc15bf40 0x7faddc0ccc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:57.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.212+0000 7fadd3e00640 1 -- 172.21.15.39:0/363960700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faddc0cf200 msgr2=0x7faddc1f5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:57.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.212+0000 7fadd3e00640 1 --2- 172.21.15.39:0/363960700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faddc0cf200 0x7faddc1f5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:57.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.212+0000 7fadd3e00640 1 -- 172.21.15.39:0/363960700 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faddc0cad80 con 0x7faddc0d0500 2024-09-21T07:37:57.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.212+0000 7fadd3e00640 1 --2- 172.21.15.39:0/363960700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faddc0d0500 0x7faddc0ca840 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fadc4034040 tx=0x7fadc40035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:57.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.213+0000 7fadd2a00640 1 -- 172.21.15.39:0/363960700 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc403e030 con 0x7faddc0d0500 2024-09-21T07:37:57.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.213+0000 7fadd2a00640 1 -- 172.21.15.39:0/363960700 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fadc4003ae0 con 0x7faddc0d0500 2024-09-21T07:37:57.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.213+0000 7fadd2a00640 1 -- 172.21.15.39:0/363960700 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc4035040 con 0x7faddc0d0500 2024-09-21T07:37:57.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.213+0000 7faddbe00640 1 -- 172.21.15.39:0/363960700 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faddc153450 con 0x7faddc0d0500 2024-09-21T07:37:57.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.213+0000 7faddbe00640 1 -- 172.21.15.39:0/363960700 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faddc145e90 con 0x7faddc0d0500 2024-09-21T07:37:57.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.214+0000 7faddbe00640 1 -- 172.21.15.39:0/363960700 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faddc1531c0 con 0x7faddc0d0500 2024-09-21T07:37:57.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.215+0000 7fadd2a00640 1 -- 172.21.15.39:0/363960700 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fadc4041030 con 0x7faddc0d0500 2024-09-21T07:37:57.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.216+0000 7fadd2a00640 1 --2- 172.21.15.39:0/363960700 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fadb8078bc0 0x7fadb807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:57.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.216+0000 7fadd2a00640 1 -- 172.21.15.39:0/363960700 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 7001+0+0 (secure 0 0 0) 0x7fadc40bb780 con 0x7faddc0d0500 2024-09-21T07:37:57.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.217+0000 7faddaa00640 1 --2- 172.21.15.39:0/363960700 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fadb8078bc0 0x7fadb807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:57.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.217+0000 7faddaa00640 1 --2- 172.21.15.39:0/363960700 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fadb8078bc0 0x7fadb807b0d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fadcc009bb0 tx=0x7fadcc005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:57.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.221+0000 7fadd2a00640 1 -- 172.21.15.39:0/363960700 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fadc4082fd0 con 0x7faddc0d0500 2024-09-21T07:37:57.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.456+0000 7faddbe00640 1 -- 172.21.15.39:0/363960700 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faddc15fbb0 con 0x7faddc0d0500 2024-09-21T07:37:57.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.457+0000 7fadd2a00640 1 -- 172.21.15.39:0/363960700 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5297 (secure 0 0 0) 0x7fadc4087a00 con 0x7faddc0d0500 2024-09-21T07:37:57.459 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:37:57.459 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":76,"btime":"2024-09-21T07:37:56:377500+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5870,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2381504293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2381504293},{"type":"v1","addr":"172.21.15.148:6837","nonce":2381504293}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:37:56.377470+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6585,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6585":{"gid":6585,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6834/1472430787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1472430787},{"type":"v1","addr":"172.21.15.39:6834","nonce":1472430787}]},"join_fscid":-1,"laggy_since":"2024-09-21T07:37:56.377470+0000","export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6585,"qdb_cluster":[6115,6585]},"id":6}]} 2024-09-21T07:37:57.459 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 76 2024-09-21T07:37:57.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.459+0000 7faddbe00640 1 -- 172.21.15.39:0/363960700 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fadb8078bc0 msgr2=0x7fadb807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:57.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.459+0000 7faddbe00640 1 --2- 172.21.15.39:0/363960700 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fadb8078bc0 0x7fadb807b0d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fadcc009bb0 tx=0x7fadcc005f90 comp rx=0 tx=0).stop 2024-09-21T07:37:57.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.459+0000 7faddbe00640 1 -- 172.21.15.39:0/363960700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faddc0d0500 msgr2=0x7faddc0ca840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:57.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.459+0000 7faddbe00640 1 --2- 172.21.15.39:0/363960700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faddc0d0500 0x7faddc0ca840 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fadc4034040 tx=0x7fadc40035f0 comp rx=0 tx=0).stop 2024-09-21T07:37:57.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.459+0000 7faddbe00640 1 -- 172.21.15.39:0/363960700 shutdown_connections 2024-09-21T07:37:57.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.459+0000 7faddbe00640 1 --2- 172.21.15.39:0/363960700 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fadb8078bc0 0x7fadb807b0d0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:57.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.459+0000 7faddbe00640 1 --2- 172.21.15.39:0/363960700 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faddc15bf40 0x7faddc0ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:57.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.459+0000 7faddbe00640 1 --2- 172.21.15.39:0/363960700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faddc0d0500 0x7faddc0ca840 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:57.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.459+0000 7faddbe00640 1 --2- 172.21.15.39:0/363960700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faddc0cf200 0x7faddc1f5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:57.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.459+0000 7faddbe00640 1 -- 172.21.15.39:0/363960700 >> 172.21.15.39:0/363960700 conn(0x7faddc06d870 msgr2=0x7faddc06ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:57.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.459+0000 7faddbe00640 1 -- 172.21.15.39:0/363960700 shutdown_connections 2024-09-21T07:37:57.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.459+0000 7faddbe00640 1 -- 172.21.15.39:0/363960700 wait complete. 2024-09-21T07:37:57.484 DEBUG:tasks.ceph_test_case:wait_until_true: success in 15s and 0 retries 2024-09-21T07:37:57.484 DEBUG:teuthology.orchestra.run.smithi039:> sudo sysctl -n kernel.core_pattern 2024-09-21T07:37:57.536 INFO:teuthology.orchestra.run.smithi039.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T07:37:57.539 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2024-09-21T07:37:57.539 DEBUG:teuthology.orchestra.run.smithi039:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2024-09-21T07:37:57.601 INFO:teuthology.orchestra.run.smithi039.stdout:1726904260.58781.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i c', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2024-09-21T07:37:57.605 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1726904260.58781.core'] 2024-09-21T07:37:57.605 INFO:tasks.cephfs.cephfs_test_case:Found core file 1726904260.58781.core, deleting it 2024-09-21T07:37:57.605 DEBUG:teuthology.orchestra.run.smithi039:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1726904260.58781.core 2024-09-21T07:37:57.699 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 6:0 2024-09-21T07:37:57.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.925+0000 7f3c5d800640 1 -- 172.21.15.39:0/4028707821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c58072ff0 msgr2=0x7f3c58073400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:57.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.925+0000 7f3c5d800640 1 --2- 172.21.15.39:0/4028707821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c58072ff0 0x7f3c58073400 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f3c58072040 tx=0x7f3c5002a8b0 comp rx=0 tx=0).stop 2024-09-21T07:37:57.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.925+0000 7f3c5d800640 1 -- 172.21.15.39:0/4028707821 shutdown_connections 2024-09-21T07:37:57.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.925+0000 7f3c5d800640 1 --2- 172.21.15.39:0/4028707821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c58072ff0 0x7f3c58073400 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:57.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.925+0000 7f3c5d800640 1 --2- 172.21.15.39:0/4028707821 >> v2:172.21.15.148:3300/0 conn(0x7f3c580af780 0x7f3c580b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:57.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.925+0000 7f3c5d800640 1 -- 172.21.15.39:0/4028707821 >> 172.21.15.39:0/4028707821 conn(0x7f3c5806d7d0 msgr2=0x7f3c580b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:57.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.925+0000 7f3c5d800640 1 -- 172.21.15.39:0/4028707821 shutdown_connections 2024-09-21T07:37:57.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.925+0000 7f3c5d800640 1 -- 172.21.15.39:0/4028707821 wait complete. 2024-09-21T07:37:57.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.927+0000 7f3c5d800640 1 Processor -- start 2024-09-21T07:37:57.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.928+0000 7f3c5d800640 1 -- start start 2024-09-21T07:37:57.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.929+0000 7f3c5d800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c58072ff0 0x7f3c5806f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:57.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.929+0000 7f3c5d800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c580af780 0x7f3c580cb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:57.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.929+0000 7f3c5d800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3c580cd3e0 0x7f3c580cb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:57.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.929+0000 7f3c5d800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3c58073b00 con 0x7f3c58072ff0 2024-09-21T07:37:57.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.929+0000 7f3c5d800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3c58073980 con 0x7f3c580af780 2024-09-21T07:37:57.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.929+0000 7f3c5d800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f3c58073c80 con 0x7f3c580cd3e0 2024-09-21T07:37:57.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.929+0000 7f3c56a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c58072ff0 0x7f3c5806f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:57.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.929+0000 7f3c56a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c58072ff0 0x7f3c5806f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53708/0 (socket says 172.21.15.39:53708) 2024-09-21T07:37:57.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.929+0000 7f3c56a00640 1 -- 172.21.15.39:0/2252906597 learned_addr learned my addr 172.21.15.39:0/2252906597 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:57.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.930+0000 7f3c4fe00640 1 --2- 172.21.15.39:0/2252906597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c580af780 0x7f3c580cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:57.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.930+0000 7f3c57400640 1 --2- 172.21.15.39:0/2252906597 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3c580cd3e0 0x7f3c580cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:57.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.930+0000 7f3c56a00640 1 -- 172.21.15.39:0/2252906597 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3c580cd3e0 msgr2=0x7f3c580cb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:57.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.930+0000 7f3c56a00640 1 --2- 172.21.15.39:0/2252906597 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3c580cd3e0 0x7f3c580cb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:57.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.930+0000 7f3c56a00640 1 -- 172.21.15.39:0/2252906597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c580af780 msgr2=0x7f3c580cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:57.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.930+0000 7f3c56a00640 1 --2- 172.21.15.39:0/2252906597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c580af780 0x7f3c580cb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:57.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.930+0000 7f3c56a00640 1 -- 172.21.15.39:0/2252906597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c581f9c60 con 0x7f3c58072ff0 2024-09-21T07:37:57.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.930+0000 7f3c56a00640 1 --2- 172.21.15.39:0/2252906597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c58072ff0 0x7f3c5806f070 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f3c4800d740 tx=0x7f3c4800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:57.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.930+0000 7f3c4ea00640 1 -- 172.21.15.39:0/2252906597 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c48017070 con 0x7f3c58072ff0 2024-09-21T07:37:57.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.930+0000 7f3c4ea00640 1 -- 172.21.15.39:0/2252906597 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c48009cc0 con 0x7f3c58072ff0 2024-09-21T07:37:57.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.930+0000 7f3c5d800640 1 -- 172.21.15.39:0/2252906597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c5814c620 con 0x7f3c58072ff0 2024-09-21T07:37:57.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.930+0000 7f3c4ea00640 1 -- 172.21.15.39:0/2252906597 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c480106c0 con 0x7f3c58072ff0 2024-09-21T07:37:57.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.931+0000 7f3c5d800640 1 -- 172.21.15.39:0/2252906597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c5814cbe0 con 0x7f3c58072ff0 2024-09-21T07:37:57.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.932+0000 7f3c5d800640 1 -- 172.21.15.39:0/2252906597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c58073400 con 0x7f3c58072ff0 2024-09-21T07:37:57.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.933+0000 7f3c4ea00640 1 -- 172.21.15.39:0/2252906597 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3c4801a070 con 0x7f3c58072ff0 2024-09-21T07:37:57.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.934+0000 7f3c4ea00640 1 --2- 172.21.15.39:0/2252906597 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3c34078b90 0x7f3c3407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:57.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.934+0000 7f3c4ea00640 1 -- 172.21.15.39:0/2252906597 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 7001+0+0 (secure 0 0 0) 0x7f3c4809b3b0 con 0x7f3c58072ff0 2024-09-21T07:37:57.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.934+0000 7f3c4fe00640 1 --2- 172.21.15.39:0/2252906597 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3c34078b90 0x7f3c3407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:57.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.935+0000 7f3c4fe00640 1 --2- 172.21.15.39:0/2252906597 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3c34078b90 0x7f3c3407b0a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3c58156c00 tx=0x7f3c40009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:57.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:57.938+0000 7f3c4ea00640 1 -- 172.21.15.39:0/2252906597 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3c480632d0 con 0x7f3c58072ff0 2024-09-21T07:37:58.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.211+0000 7f3c5d800640 1 -- 172.21.15.39:0/2252906597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "6:0"} v 0) -- 0x7f3c58070fb0 con 0x7f3c58072ff0 2024-09-21T07:37:58.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.218+0000 7f3c4ea00640 1 -- 172.21.15.39:0/2252906597 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "6:0"}]=0 failed mds gid 6585 v77) ==== 97+0+0 (secure 0 0 0) 0x7f3c48062bd0 con 0x7f3c58072ff0 2024-09-21T07:37:58.219 INFO:teuthology.orchestra.run.smithi039.stderr:failed mds gid 6585 2024-09-21T07:37:58.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.220+0000 7f3c5d800640 1 -- 172.21.15.39:0/2252906597 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3c34078b90 msgr2=0x7f3c3407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:58.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.220+0000 7f3c5d800640 1 --2- 172.21.15.39:0/2252906597 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3c34078b90 0x7f3c3407b0a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3c58156c00 tx=0x7f3c40009040 comp rx=0 tx=0).stop 2024-09-21T07:37:58.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.220+0000 7f3c5d800640 1 -- 172.21.15.39:0/2252906597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c58072ff0 msgr2=0x7f3c5806f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:58.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.220+0000 7f3c5d800640 1 --2- 172.21.15.39:0/2252906597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c58072ff0 0x7f3c5806f070 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f3c4800d740 tx=0x7f3c4800dc00 comp rx=0 tx=0).stop 2024-09-21T07:37:58.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.220+0000 7f3c5d800640 1 -- 172.21.15.39:0/2252906597 shutdown_connections 2024-09-21T07:37:58.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.220+0000 7f3c5d800640 1 --2- 172.21.15.39:0/2252906597 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3c34078b90 0x7f3c3407b0a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:58.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.220+0000 7f3c5d800640 1 --2- 172.21.15.39:0/2252906597 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3c580cd3e0 0x7f3c580cb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:58.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.220+0000 7f3c5d800640 1 --2- 172.21.15.39:0/2252906597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c580af780 0x7f3c580cb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:58.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.220+0000 7f3c5d800640 1 --2- 172.21.15.39:0/2252906597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c58072ff0 0x7f3c5806f070 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:58.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.220+0000 7f3c5d800640 1 -- 172.21.15.39:0/2252906597 >> 172.21.15.39:0/2252906597 conn(0x7f3c5806d7d0 msgr2=0x7f3c580745e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:58.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.220+0000 7f3c5d800640 1 -- 172.21.15.39:0/2252906597 shutdown_connections 2024-09-21T07:37:58.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.220+0000 7f3c5d800640 1 -- 172.21.15.39:0/2252906597 wait complete. 2024-09-21T07:37:58.251 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T07:37:58.251 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T07:37:58.251 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T07:37:58.251 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:37:58.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:37:58.252 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/daemon/state.py", line 146, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi039 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c' 2024-09-21T07:37:58.252 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T07:37:58.252 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T07:37:58.256 INFO:tasks.ceph.mds.c:Started 2024-09-21T07:37:58.256 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:37:58.453 INFO:tasks.ceph.mds.c.smithi039.stdout:starting mds.c at 2024-09-21T07:37:58.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.484+0000 7f93b4200640 1 -- 172.21.15.39:0/2553692556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93ac15a9b0 msgr2=0x7f93ac0cfae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:58.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.484+0000 7f93b4200640 1 --2- 172.21.15.39:0/2553692556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93ac15a9b0 0x7f93ac0cfae0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f93a0005fd0 tx=0x7f93a002b880 comp rx=0 tx=0).stop 2024-09-21T07:37:58.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.484+0000 7f93b4200640 1 -- 172.21.15.39:0/2553692556 shutdown_connections 2024-09-21T07:37:58.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.484+0000 7f93b4200640 1 --2- 172.21.15.39:0/2553692556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93ac15a9b0 0x7f93ac0cfae0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:58.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.484+0000 7f93b4200640 1 -- 172.21.15.39:0/2553692556 >> 172.21.15.39:0/2553692556 conn(0x7f93ac06d7d0 msgr2=0x7f93ac0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:58.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.484+0000 7f93b4200640 1 -- 172.21.15.39:0/2553692556 shutdown_connections 2024-09-21T07:37:58.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.484+0000 7f93b4200640 1 -- 172.21.15.39:0/2553692556 wait complete. 2024-09-21T07:37:58.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.486+0000 7f93b4200640 1 Processor -- start 2024-09-21T07:37:58.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.486+0000 7f93b4200640 1 -- start start 2024-09-21T07:37:58.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.488+0000 7f93b4200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93ac15a9b0 0x7f93ac15e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:58.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.488+0000 7f93b4200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f93ac15ef10 0x7f93ac14fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:58.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.488+0000 7f93b4200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93ac0cb7c0 0x7f93ac0cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:58.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.488+0000 7f93b4200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f93ac070f20 con 0x7f93ac0cb7c0 2024-09-21T07:37:58.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.488+0000 7f93b4200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f93ac15f4b0 con 0x7f93ac15a9b0 2024-09-21T07:37:58.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.488+0000 7f93b4200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f93ac0710a0 con 0x7f93ac15ef10 2024-09-21T07:37:58.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.488+0000 7f93b2200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93ac0cb7c0 0x7f93ac0cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:58.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.488+0000 7f93b2200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93ac0cb7c0 0x7f93ac0cbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53740/0 (socket says 172.21.15.39:53740) 2024-09-21T07:37:58.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.488+0000 7f93b2200640 1 -- 172.21.15.39:0/540692132 learned_addr learned my addr 172.21.15.39:0/540692132 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:37:58.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.488+0000 7f93b1800640 1 --2- 172.21.15.39:0/540692132 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93ac15a9b0 0x7f93ac15e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:58.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.488+0000 7f93b0e00640 1 --2- 172.21.15.39:0/540692132 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f93ac15ef10 0x7f93ac14fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:58.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.489+0000 7f93b2200640 1 -- 172.21.15.39:0/540692132 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f93ac15ef10 msgr2=0x7f93ac14fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:58.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.489+0000 7f93b2200640 1 --2- 172.21.15.39:0/540692132 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f93ac15ef10 0x7f93ac14fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:58.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.489+0000 7f93b2200640 1 -- 172.21.15.39:0/540692132 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93ac15a9b0 msgr2=0x7f93ac15e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:58.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.489+0000 7f93b2200640 1 --2- 172.21.15.39:0/540692132 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93ac15a9b0 0x7f93ac15e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:58.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.489+0000 7f93b2200640 1 -- 172.21.15.39:0/540692132 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93ac0cc110 con 0x7f93ac0cb7c0 2024-09-21T07:37:58.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.489+0000 7f93b2200640 1 --2- 172.21.15.39:0/540692132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93ac0cb7c0 0x7f93ac0cbbd0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f93a400c700 tx=0x7f93a400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:58.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.489+0000 7f93a9600640 1 -- 172.21.15.39:0/540692132 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a4003b70 con 0x7f93ac0cb7c0 2024-09-21T07:37:58.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.489+0000 7f93a9600640 1 -- 172.21.15.39:0/540692132 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93a4003d10 con 0x7f93ac0cb7c0 2024-09-21T07:37:58.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.489+0000 7f93a9600640 1 -- 172.21.15.39:0/540692132 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a400aab0 con 0x7f93ac0cb7c0 2024-09-21T07:37:58.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.489+0000 7f93b4200640 1 -- 172.21.15.39:0/540692132 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93ac15d750 con 0x7f93ac0cb7c0 2024-09-21T07:37:58.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.489+0000 7f93b4200640 1 -- 172.21.15.39:0/540692132 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93ac15dd10 con 0x7f93ac0cb7c0 2024-09-21T07:37:58.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.490+0000 7f93b4200640 1 -- 172.21.15.39:0/540692132 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93ac0cfae0 con 0x7f93ac0cb7c0 2024-09-21T07:37:58.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.492+0000 7f93a9600640 1 -- 172.21.15.39:0/540692132 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f93a400ac50 con 0x7f93ac0cb7c0 2024-09-21T07:37:58.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.493+0000 7f93a9600640 1 --2- 172.21.15.39:0/540692132 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f938c078b90 0x7f938c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:37:58.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.493+0000 7f93a9600640 1 -- 172.21.15.39:0/540692132 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 7087+0+0 (secure 0 0 0) 0x7f93a409aed0 con 0x7f93ac0cb7c0 2024-09-21T07:37:58.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.493+0000 7f93b1800640 1 --2- 172.21.15.39:0/540692132 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f938c078b90 0x7f938c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:37:58.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.494+0000 7f93b1800640 1 --2- 172.21.15.39:0/540692132 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f938c078b90 0x7f938c07b0a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f93a0035000 tx=0x7f93a0004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:37:58.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.497+0000 7f93a9600640 1 -- 172.21.15.39:0/540692132 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f93ac0cc110 con 0x7f93ac0cb7c0 2024-09-21T07:37:58.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.731+0000 7f93b4200640 1 -- 172.21.15.39:0/540692132 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93ac0b0d20 con 0x7f93ac0cb7c0 2024-09-21T07:37:58.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.731+0000 7f93a9600640 1 -- 172.21.15.39:0/540692132 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) ==== 94+0+4418 (secure 0 0 0) 0x7f93ac0b4780 con 0x7f93ac0cb7c0 2024-09-21T07:37:58.733 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:37:58.733 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":78,"btime":"2024-09-21T07:37:58:226689+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:37:58.226670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5870,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5870":{"gid":5870,"name":"d","rank":0,"incarnation":78,"state":"up:replay","state_seq":1,"addr":"172.21.15.148:6837/2381504293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2381504293},{"type":"v1","addr":"172.21.15.148:6837","nonce":2381504293}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6115,"qdb_cluster":[6115]},"id":6}]} 2024-09-21T07:37:58.734 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 78 2024-09-21T07:37:58.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.733+0000 7f93b4200640 1 -- 172.21.15.39:0/540692132 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f938c078b90 msgr2=0x7f938c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:58.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.733+0000 7f93b4200640 1 --2- 172.21.15.39:0/540692132 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f938c078b90 0x7f938c07b0a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f93a0035000 tx=0x7f93a0004730 comp rx=0 tx=0).stop 2024-09-21T07:37:58.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.734+0000 7f93b4200640 1 -- 172.21.15.39:0/540692132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93ac0cb7c0 msgr2=0x7f93ac0cbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:37:58.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.734+0000 7f93b4200640 1 --2- 172.21.15.39:0/540692132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93ac0cb7c0 0x7f93ac0cbbd0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f93a400c700 tx=0x7f93a400cbc0 comp rx=0 tx=0).stop 2024-09-21T07:37:58.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.734+0000 7f93b4200640 1 -- 172.21.15.39:0/540692132 shutdown_connections 2024-09-21T07:37:58.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.734+0000 7f93b4200640 1 --2- 172.21.15.39:0/540692132 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f938c078b90 0x7f938c07b0a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:58.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.734+0000 7f93b4200640 1 --2- 172.21.15.39:0/540692132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93ac0cb7c0 0x7f93ac0cbbd0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:58.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.734+0000 7f93b4200640 1 --2- 172.21.15.39:0/540692132 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f93ac15ef10 0x7f93ac14fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:58.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.734+0000 7f93b4200640 1 --2- 172.21.15.39:0/540692132 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93ac15a9b0 0x7f93ac15e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:37:58.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.734+0000 7f93b4200640 1 -- 172.21.15.39:0/540692132 >> 172.21.15.39:0/540692132 conn(0x7f93ac06d7d0 msgr2=0x7f93ac1599e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:37:58.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.734+0000 7f93b4200640 1 -- 172.21.15.39:0/540692132 shutdown_connections 2024-09-21T07:37:58.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:37:58.734+0000 7f93b4200640 1 -- 172.21.15.39:0/540692132 wait complete. 2024-09-21T07:37:58.762 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-09-21T07:38:00.712 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:38:00.717 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:38:02.954 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-09-21T07:38:03.763 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:38:04.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:03.998+0000 7f61a6e00640 1 -- 172.21.15.39:0/1926117263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61a0073170 msgr2=0x7f61a015c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:04.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:03.998+0000 7f61a6e00640 1 --2- 172.21.15.39:0/1926117263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61a0073170 0x7f61a015c780 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f61a00710f0 tx=0x7f619002a560 comp rx=0 tx=0).stop 2024-09-21T07:38:04.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:03.998+0000 7f61a6e00640 1 -- 172.21.15.39:0/1926117263 shutdown_connections 2024-09-21T07:38:04.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:03.998+0000 7f61a6e00640 1 --2- 172.21.15.39:0/1926117263 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f61a0070440 0x7f61a0074b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:04.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:03.998+0000 7f61a6e00640 1 --2- 172.21.15.39:0/1926117263 >> v2:172.21.15.148:3300/0 conn(0x7f61a0070030 0x7f61a0074640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:04.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:03.998+0000 7f61a6e00640 1 --2- 172.21.15.39:0/1926117263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61a0073170 0x7f61a015c780 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:04.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:03.998+0000 7f61a6e00640 1 -- 172.21.15.39:0/1926117263 >> 172.21.15.39:0/1926117263 conn(0x7f61a00b1cb0 msgr2=0x7f61a00b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:04.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:03.998+0000 7f61a6e00640 1 -- 172.21.15.39:0/1926117263 shutdown_connections 2024-09-21T07:38:04.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:03.999+0000 7f61a6e00640 1 -- 172.21.15.39:0/1926117263 wait complete. 2024-09-21T07:38:04.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.001+0000 7f61a6e00640 1 Processor -- start 2024-09-21T07:38:04.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.001+0000 7f61a6e00640 1 -- start start 2024-09-21T07:38:04.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.002+0000 7f61a6e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61a0070030 0x7f61a00d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:04.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.002+0000 7f61a6e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61a0070440 0x7f61a0154570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:04.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.002+0000 7f61a6e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f61a0073170 0x7f61a00cac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:04.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.002+0000 7f61a6e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f61a006f640 con 0x7f61a0070440 2024-09-21T07:38:04.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.002+0000 7f61a6e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f61a006f4c0 con 0x7f61a0070030 2024-09-21T07:38:04.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.002+0000 7f61a6e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f61a006f7c0 con 0x7f61a0073170 2024-09-21T07:38:04.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.003+0000 7f6197e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61a0070440 0x7f61a0154570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:04.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.003+0000 7f6197e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61a0070440 0x7f61a0154570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34814/0 (socket says 172.21.15.39:34814) 2024-09-21T07:38:04.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.003+0000 7f6197e00640 1 -- 172.21.15.39:0/3255705559 learned_addr learned my addr 172.21.15.39:0/3255705559 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:04.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.003+0000 7f619fe00640 1 --2- 172.21.15.39:0/3255705559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61a0070030 0x7f61a00d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:04.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.003+0000 7f61a4e00640 1 --2- 172.21.15.39:0/3255705559 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f61a0073170 0x7f61a00cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:04.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.003+0000 7f6197e00640 1 -- 172.21.15.39:0/3255705559 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f61a0073170 msgr2=0x7f61a00cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:04.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.003+0000 7f6197e00640 1 --2- 172.21.15.39:0/3255705559 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f61a0073170 0x7f61a00cac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:04.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.003+0000 7f6197e00640 1 -- 172.21.15.39:0/3255705559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61a0070030 msgr2=0x7f61a00d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:04.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.003+0000 7f6197e00640 1 --2- 172.21.15.39:0/3255705559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61a0070030 0x7f61a00d08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:04.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.003+0000 7f6197e00640 1 -- 172.21.15.39:0/3255705559 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61a00cbfd0 con 0x7f61a0070440 2024-09-21T07:38:04.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.003+0000 7f619fe00640 1 --2- 172.21.15.39:0/3255705559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61a0070030 0x7f61a00d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:38:04.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.003+0000 7f61a4e00640 1 --2- 172.21.15.39:0/3255705559 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f61a0073170 0x7f61a00cac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:38:04.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.004+0000 7f6197e00640 1 --2- 172.21.15.39:0/3255705559 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61a0070440 0x7f61a0154570 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f618800d6d0 tx=0x7f618800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:04.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.004+0000 7f619cc00640 1 -- 172.21.15.39:0/3255705559 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6188017070 con 0x7f61a0070440 2024-09-21T07:38:04.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.004+0000 7f619cc00640 1 -- 172.21.15.39:0/3255705559 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6188009b10 con 0x7f61a0070440 2024-09-21T07:38:04.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.004+0000 7f619cc00640 1 -- 172.21.15.39:0/3255705559 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6188010460 con 0x7f61a0070440 2024-09-21T07:38:04.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.004+0000 7f61a6e00640 1 -- 172.21.15.39:0/3255705559 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61a009d910 con 0x7f61a0070440 2024-09-21T07:38:04.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.004+0000 7f61a6e00640 1 -- 172.21.15.39:0/3255705559 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61a00cb7d0 con 0x7f61a0070440 2024-09-21T07:38:04.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.005+0000 7f61a6e00640 1 -- 172.21.15.39:0/3255705559 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61a00d0e70 con 0x7f61a0070440 2024-09-21T07:38:04.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.007+0000 7f619cc00640 1 -- 172.21.15.39:0/3255705559 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f618801a070 con 0x7f61a0070440 2024-09-21T07:38:04.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.007+0000 7f619cc00640 1 --2- 172.21.15.39:0/3255705559 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f617c078b90 0x7f617c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:04.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.008+0000 7f619cc00640 1 -- 172.21.15.39:0/3255705559 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 7087+0+0 (secure 0 0 0) 0x7f618809b480 con 0x7f61a0070440 2024-09-21T07:38:04.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.008+0000 7f619fe00640 1 --2- 172.21.15.39:0/3255705559 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f617c078b90 0x7f617c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:04.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.009+0000 7f619fe00640 1 --2- 172.21.15.39:0/3255705559 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f617c078b90 0x7f617c07b0a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6190033000 tx=0x7f6190004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:04.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.012+0000 7f619cc00640 1 -- 172.21.15.39:0/3255705559 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f61880632c0 con 0x7f61a0070440 2024-09-21T07:38:04.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.245+0000 7f61a6e00640 1 -- 172.21.15.39:0/3255705559 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61a015c780 con 0x7f61a0070440 2024-09-21T07:38:04.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.246+0000 7f619cc00640 1 -- 172.21.15.39:0/3255705559 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5246 (secure 0 0 0) 0x7f6188062bc0 con 0x7f61a0070440 2024-09-21T07:38:04.248 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:04.248 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":82,"btime":"2024-09-21T07:38:02:257520+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6825,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/855150775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":855150775},{"type":"v1","addr":"172.21.15.39:6834","nonce":855150775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:38:02.257514+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5870,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5870":{"gid":5870,"name":"d","rank":0,"incarnation":78,"state":"up:active","state_seq":12,"addr":"172.21.15.148:6837/2381504293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2381504293},{"type":"v1","addr":"172.21.15.148:6837","nonce":2381504293}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5870,"qdb_cluster":[5870,6115]},"id":6}]} 2024-09-21T07:38:04.248 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 82 2024-09-21T07:38:04.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.248+0000 7f61a6e00640 1 -- 172.21.15.39:0/3255705559 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f617c078b90 msgr2=0x7f617c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:04.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.248+0000 7f61a6e00640 1 --2- 172.21.15.39:0/3255705559 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f617c078b90 0x7f617c07b0a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6190033000 tx=0x7f6190004730 comp rx=0 tx=0).stop 2024-09-21T07:38:04.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.248+0000 7f61a6e00640 1 -- 172.21.15.39:0/3255705559 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61a0070440 msgr2=0x7f61a0154570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:04.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.248+0000 7f61a6e00640 1 --2- 172.21.15.39:0/3255705559 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61a0070440 0x7f61a0154570 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f618800d6d0 tx=0x7f618800db90 comp rx=0 tx=0).stop 2024-09-21T07:38:04.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.248+0000 7f61a6e00640 1 -- 172.21.15.39:0/3255705559 shutdown_connections 2024-09-21T07:38:04.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.248+0000 7f61a6e00640 1 --2- 172.21.15.39:0/3255705559 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f617c078b90 0x7f617c07b0a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:04.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.248+0000 7f61a6e00640 1 --2- 172.21.15.39:0/3255705559 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f61a0073170 0x7f61a00cac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:04.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.248+0000 7f61a6e00640 1 --2- 172.21.15.39:0/3255705559 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61a0070440 0x7f61a0154570 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:04.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.248+0000 7f61a6e00640 1 --2- 172.21.15.39:0/3255705559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61a0070030 0x7f61a00d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:04.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.248+0000 7f61a6e00640 1 -- 172.21.15.39:0/3255705559 >> 172.21.15.39:0/3255705559 conn(0x7f61a00b1cb0 msgr2=0x7f61a00acf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:04.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.248+0000 7f61a6e00640 1 -- 172.21.15.39:0/3255705559 shutdown_connections 2024-09-21T07:38:04.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.248+0000 7f61a6e00640 1 -- 172.21.15.39:0/3255705559 wait complete. 2024-09-21T07:38:04.273 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-21T07:38:04.273 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:38:04.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.498+0000 7f172f200640 1 -- 172.21.15.39:0/1078806660 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1730160b60 msgr2=0x7f1730073e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:04.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.498+0000 7f172f200640 1 --2- 172.21.15.39:0/1078806660 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1730160b60 0x7f1730073e70 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f17300d0760 tx=0x7f172802a4d0 comp rx=0 tx=0).stop 2024-09-21T07:38:04.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.498+0000 7f172f200640 1 -- 172.21.15.39:0/1078806660 shutdown_connections 2024-09-21T07:38:04.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.498+0000 7f172f200640 1 --2- 172.21.15.39:0/1078806660 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1730160b60 0x7f1730073e70 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:04.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.498+0000 7f172f200640 1 --2- 172.21.15.39:0/1078806660 >> v2:172.21.15.148:3300/0 conn(0x7f1730160750 0x7f1730073930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:04.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.498+0000 7f172f200640 1 --2- 172.21.15.39:0/1078806660 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f173015d4f0 0x7f17300b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:04.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.498+0000 7f172f200640 1 -- 172.21.15.39:0/1078806660 >> 172.21.15.39:0/1078806660 conn(0x7f17300cd6c0 msgr2=0x7f17300cdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:04.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.499+0000 7f172f200640 1 -- 172.21.15.39:0/1078806660 shutdown_connections 2024-09-21T07:38:04.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.499+0000 7f172f200640 1 -- 172.21.15.39:0/1078806660 wait complete. 2024-09-21T07:38:04.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.501+0000 7f172f200640 1 Processor -- start 2024-09-21T07:38:04.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.501+0000 7f172f200640 1 -- start start 2024-09-21T07:38:04.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.502+0000 7f172f200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f173015d4f0 0x7f1730143f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:04.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.502+0000 7f172f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1730160750 0x7f1730140b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:04.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.502+0000 7f172f200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1730160b60 0x7f17301f6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:04.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.502+0000 7f172f200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f17300b5670 con 0x7f1730160750 2024-09-21T07:38:04.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.502+0000 7f172f200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f17300b54f0 con 0x7f1730160b60 2024-09-21T07:38:04.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.502+0000 7f172f200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f17300b57f0 con 0x7f173015d4f0 2024-09-21T07:38:04.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.503+0000 7f1725400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1730160750 0x7f1730140b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:04.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.503+0000 7f1725400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1730160750 0x7f1730140b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34832/0 (socket says 172.21.15.39:34832) 2024-09-21T07:38:04.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.503+0000 7f1725400640 1 -- 172.21.15.39:0/1296627269 learned_addr learned my addr 172.21.15.39:0/1296627269 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:04.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.503+0000 7f172de00640 1 --2- 172.21.15.39:0/1296627269 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f173015d4f0 0x7f1730143f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:04.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.503+0000 7f172e800640 1 --2- 172.21.15.39:0/1296627269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1730160b60 0x7f17301f6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:04.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.503+0000 7f1725400640 1 -- 172.21.15.39:0/1296627269 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f173015d4f0 msgr2=0x7f1730143f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:04.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.503+0000 7f1725400640 1 --2- 172.21.15.39:0/1296627269 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f173015d4f0 0x7f1730143f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:04.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.503+0000 7f1725400640 1 -- 172.21.15.39:0/1296627269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1730160b60 msgr2=0x7f17301f6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:04.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.503+0000 7f1725400640 1 --2- 172.21.15.39:0/1296627269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1730160b60 0x7f17301f6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:04.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.503+0000 7f1725400640 1 -- 172.21.15.39:0/1296627269 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17301414a0 con 0x7f1730160750 2024-09-21T07:38:04.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.503+0000 7f172e800640 1 --2- 172.21.15.39:0/1296627269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1730160b60 0x7f17301f6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:38:04.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.503+0000 7f1725400640 1 --2- 172.21.15.39:0/1296627269 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1730160750 0x7f1730140b20 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f171800d770 tx=0x7f171800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:04.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.504+0000 7f1726a00640 1 -- 172.21.15.39:0/1296627269 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1718017070 con 0x7f1730160750 2024-09-21T07:38:04.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.504+0000 7f1726a00640 1 -- 172.21.15.39:0/1296627269 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1718009d20 con 0x7f1730160750 2024-09-21T07:38:04.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.504+0000 7f1726a00640 1 -- 172.21.15.39:0/1296627269 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17180106c0 con 0x7f1730160750 2024-09-21T07:38:04.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.504+0000 7f172f200640 1 -- 172.21.15.39:0/1296627269 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f173015c150 con 0x7f1730160750 2024-09-21T07:38:04.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.504+0000 7f172f200640 1 -- 172.21.15.39:0/1296627269 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17301547c0 con 0x7f1730160750 2024-09-21T07:38:04.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.505+0000 7f172f200640 1 -- 172.21.15.39:0/1296627269 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17300b3850 con 0x7f1730160750 2024-09-21T07:38:04.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.506+0000 7f1726a00640 1 -- 172.21.15.39:0/1296627269 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f171801a070 con 0x7f1730160750 2024-09-21T07:38:04.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.507+0000 7f1726a00640 1 --2- 172.21.15.39:0/1296627269 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f170c078b90 0x7f170c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:04.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.507+0000 7f1726a00640 1 -- 172.21.15.39:0/1296627269 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 7087+0+0 (secure 0 0 0) 0x7f171809b360 con 0x7f1730160750 2024-09-21T07:38:04.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.508+0000 7f172de00640 1 --2- 172.21.15.39:0/1296627269 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f170c078b90 0x7f170c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:04.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.508+0000 7f172de00640 1 --2- 172.21.15.39:0/1296627269 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f170c078b90 0x7f170c07b0a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1720005fd0 tx=0x7f17200095e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:04.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.512+0000 7f1726a00640 1 -- 172.21.15.39:0/1296627269 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1718062b50 con 0x7f1730160750 2024-09-21T07:38:04.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.752+0000 7f172f200640 1 -- 172.21.15.39:0/1296627269 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1730069450 con 0x7f1730160750 2024-09-21T07:38:04.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.753+0000 7f1726a00640 1 -- 172.21.15.39:0/1296627269 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5246 (secure 0 0 0) 0x7f1718067580 con 0x7f1730160750 2024-09-21T07:38:04.755 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:04.755 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":82,"btime":"2024-09-21T07:38:02:257520+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6825,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/855150775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":855150775},{"type":"v1","addr":"172.21.15.39:6834","nonce":855150775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:38:02.257514+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5870,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5870":{"gid":5870,"name":"d","rank":0,"incarnation":78,"state":"up:active","state_seq":12,"addr":"172.21.15.148:6837/2381504293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2381504293},{"type":"v1","addr":"172.21.15.148:6837","nonce":2381504293}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5870,"qdb_cluster":[5870,6115]},"id":6}]} 2024-09-21T07:38:04.756 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 82 2024-09-21T07:38:04.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.755+0000 7f172f200640 1 -- 172.21.15.39:0/1296627269 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f170c078b90 msgr2=0x7f170c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:04.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.755+0000 7f172f200640 1 --2- 172.21.15.39:0/1296627269 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f170c078b90 0x7f170c07b0a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1720005fd0 tx=0x7f17200095e0 comp rx=0 tx=0).stop 2024-09-21T07:38:04.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.755+0000 7f172f200640 1 -- 172.21.15.39:0/1296627269 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1730160750 msgr2=0x7f1730140b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:04.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.755+0000 7f172f200640 1 --2- 172.21.15.39:0/1296627269 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1730160750 0x7f1730140b20 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f171800d770 tx=0x7f171800dc30 comp rx=0 tx=0).stop 2024-09-21T07:38:04.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.756+0000 7f172f200640 1 -- 172.21.15.39:0/1296627269 shutdown_connections 2024-09-21T07:38:04.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.756+0000 7f172f200640 1 --2- 172.21.15.39:0/1296627269 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f170c078b90 0x7f170c07b0a0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:04.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.756+0000 7f172f200640 1 --2- 172.21.15.39:0/1296627269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1730160b60 0x7f17301f6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:04.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.756+0000 7f172f200640 1 --2- 172.21.15.39:0/1296627269 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1730160750 0x7f1730140b20 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:04.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.756+0000 7f172f200640 1 --2- 172.21.15.39:0/1296627269 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f173015d4f0 0x7f1730143f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:04.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.756+0000 7f172f200640 1 -- 172.21.15.39:0/1296627269 >> 172.21.15.39:0/1296627269 conn(0x7f17300cd6c0 msgr2=0x7f17300b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:04.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.756+0000 7f172f200640 1 -- 172.21.15.39:0/1296627269 shutdown_connections 2024-09-21T07:38:04.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:04.756+0000 7f172f200640 1 -- 172.21.15.39:0/1296627269 wait complete. 2024-09-21T07:38:05.182 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 82, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:36:20.189287+0000', 'modified': '2024-09-21T07:38:02.257514+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 95, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 5870, 'mds_1': 6115}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5870': {'gid': 5870, 'name': 'd', 'rank': 0, 'incarnation': 78, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.148:6837/2381504293', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 2381504293}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 2381504293}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6115': {'gid': 6115, 'name': 'a', 'rank': 1, 'incarnation': 56, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6835/364657398', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 364657398}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 364657398}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5870, 'qdb_cluster': [5870, 6115]} 2024-09-21T07:38:05.182 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2024-09-21T07:38:05.182 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-21T07:38:05.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.410+0000 7f456f200640 1 -- 172.21.15.39:0/3239990726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45700b4690 msgr2=0x7f45700b4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:05.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.410+0000 7f456f200640 1 --2- 172.21.15.39:0/3239990726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45700b4690 0x7f45700b4aa0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f4570074940 tx=0x7f456802aa30 comp rx=0 tx=0).stop 2024-09-21T07:38:05.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.410+0000 7f456f200640 1 -- 172.21.15.39:0/3239990726 shutdown_connections 2024-09-21T07:38:05.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.410+0000 7f456f200640 1 --2- 172.21.15.39:0/3239990726 >> v2:172.21.15.148:3300/0 conn(0x7f4570074040 0x7f457015c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.410+0000 7f456f200640 1 --2- 172.21.15.39:0/3239990726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45700b4690 0x7f45700b4aa0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.410+0000 7f456f200640 1 -- 172.21.15.39:0/3239990726 >> 172.21.15.39:0/3239990726 conn(0x7f45700cd6c0 msgr2=0x7f45700cdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:05.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.410+0000 7f456f200640 1 -- 172.21.15.39:0/3239990726 shutdown_connections 2024-09-21T07:38:05.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.410+0000 7f456f200640 1 -- 172.21.15.39:0/3239990726 wait complete. 2024-09-21T07:38:05.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.412+0000 7f456f200640 1 Processor -- start 2024-09-21T07:38:05.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.412+0000 7f456f200640 1 -- start start 2024-09-21T07:38:05.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.414+0000 7f456f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45700b4690 0x7f457015dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:05.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.414+0000 7f456f200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45700cea10 0x7f4570152900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:05.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.414+0000 7f456f200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45700706f0 0x7f4570070b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:05.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.414+0000 7f456f200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f45700b5370 con 0x7f45700b4690 2024-09-21T07:38:05.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.414+0000 7f456f200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f45700b51f0 con 0x7f45700706f0 2024-09-21T07:38:05.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.414+0000 7f456f200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f45700b54f0 con 0x7f45700cea10 2024-09-21T07:38:05.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.414+0000 7f456de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45700b4690 0x7f457015dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:05.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.414+0000 7f456de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45700b4690 0x7f457015dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34846/0 (socket says 172.21.15.39:34846) 2024-09-21T07:38:05.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.414+0000 7f456de00640 1 -- 172.21.15.39:0/2349496989 learned_addr learned my addr 172.21.15.39:0/2349496989 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:05.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.414+0000 7f4567e00640 1 --2- 172.21.15.39:0/2349496989 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45700cea10 0x7f4570152900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:05.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.414+0000 7f456e800640 1 --2- 172.21.15.39:0/2349496989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45700706f0 0x7f4570070b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:05.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.414+0000 7f456de00640 1 -- 172.21.15.39:0/2349496989 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45700cea10 msgr2=0x7f4570152900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:05.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.414+0000 7f456de00640 1 --2- 172.21.15.39:0/2349496989 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45700cea10 0x7f4570152900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.414+0000 7f456de00640 1 -- 172.21.15.39:0/2349496989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45700706f0 msgr2=0x7f4570070b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:05.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.414+0000 7f456de00640 1 --2- 172.21.15.39:0/2349496989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45700706f0 0x7f4570070b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.414+0000 7f456de00640 1 -- 172.21.15.39:0/2349496989 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4570071040 con 0x7f45700b4690 2024-09-21T07:38:05.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.415+0000 7f456de00640 1 --2- 172.21.15.39:0/2349496989 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45700b4690 0x7f457015dd90 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f456000c370 tx=0x7f456000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:05.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.415+0000 7f4566000640 1 -- 172.21.15.39:0/2349496989 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45600052d0 con 0x7f45700b4690 2024-09-21T07:38:05.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.415+0000 7f4566000640 1 -- 172.21.15.39:0/2349496989 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4560007020 con 0x7f45700b4690 2024-09-21T07:38:05.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.415+0000 7f456f200640 1 -- 172.21.15.39:0/2349496989 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f457015e850 con 0x7f45700b4690 2024-09-21T07:38:05.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.415+0000 7f4566000640 1 -- 172.21.15.39:0/2349496989 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4560007470 con 0x7f45700b4690 2024-09-21T07:38:05.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.415+0000 7f456f200640 1 -- 172.21.15.39:0/2349496989 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45701f8c20 con 0x7f45700b4690 2024-09-21T07:38:05.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.418+0000 7f4566000640 1 -- 172.21.15.39:0/2349496989 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f45600067d0 con 0x7f45700b4690 2024-09-21T07:38:05.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.419+0000 7f4566000640 1 --2- 172.21.15.39:0/2349496989 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f454c078bc0 0x7f454c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:05.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.420+0000 7f4566000640 1 -- 172.21.15.39:0/2349496989 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 7087+0+0 (secure 0 0 0) 0x7f456009a760 con 0x7f45700b4690 2024-09-21T07:38:05.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.420+0000 7f4567e00640 1 --2- 172.21.15.39:0/2349496989 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f454c078bc0 0x7f454c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:05.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.420+0000 7f457bff3740 1 Processor -- start 2024-09-21T07:38:05.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.420+0000 7f457bff3740 1 -- start start 2024-09-21T07:38:05.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.421+0000 7f4567e00640 1 --2- 172.21.15.39:0/2349496989 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f454c078bc0 0x7f454c07b0d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f45701f88a0 tx=0x7f4558005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:05.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.422+0000 7f457bff3740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f1bff0a270 0x55f1c02dab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:05.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.422+0000 7f457bff3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1bfee13b0 0x55f1c02db0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:05.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.422+0000 7f457bff3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1bfee9590 0x55f1c02ab150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:05.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.422+0000 7f457bff3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55f1c0234430 con 0x55f1bfee9590 2024-09-21T07:38:05.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.422+0000 7f457bff3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f1c02245d0 con 0x55f1bfee13b0 2024-09-21T07:38:05.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.422+0000 7f457bff3740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55f1c0213a00 con 0x55f1bff0a270 2024-09-21T07:38:05.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.422+0000 7f456e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1bfee9590 0x55f1c02ab150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:05.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.422+0000 7f456e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1bfee9590 0x55f1c02ab150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34850/0 (socket says 172.21.15.39:34850) 2024-09-21T07:38:05.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.422+0000 7f456e800640 1 -- 172.21.15.39:0/4090337716 learned_addr learned my addr 172.21.15.39:0/4090337716 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:05.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.422+0000 7f456de00640 1 --2- 172.21.15.39:0/4090337716 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f1bff0a270 0x55f1c02dab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:05.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.422+0000 7f4567e00640 1 --2- 172.21.15.39:0/4090337716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1bfee13b0 0x55f1c02db0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:05.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.422+0000 7f456e800640 1 -- 172.21.15.39:0/4090337716 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f1bff0a270 msgr2=0x55f1c02dab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:38:05.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.422+0000 7f456e800640 1 --2- 172.21.15.39:0/4090337716 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f1bff0a270 0x55f1c02dab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.422+0000 7f456e800640 1 -- 172.21.15.39:0/4090337716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1bfee13b0 msgr2=0x55f1c02db0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:38:05.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.422+0000 7f456e800640 1 --2- 172.21.15.39:0/4090337716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1bfee13b0 0x55f1c02db0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.422+0000 7f456e800640 1 -- 172.21.15.39:0/4090337716 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1c01dc520 con 0x55f1bfee9590 2024-09-21T07:38:05.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.423+0000 7f456e800640 1 --2- 172.21.15.39:0/4090337716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1bfee9590 0x55f1c02ab150 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f4568009bf0 tx=0x7f4568004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=581d630e82306eac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:05.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.423+0000 7f455ea00640 1 -- 172.21.15.39:0/4090337716 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4568033070 con 0x55f1bfee9590 2024-09-21T07:38:05.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.423+0000 7f455ea00640 1 -- 172.21.15.39:0/4090337716 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f456802f980 con 0x55f1bfee9590 2024-09-21T07:38:05.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.423+0000 7f455ea00640 1 -- 172.21.15.39:0/4090337716 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456802fdb0 con 0x55f1bfee9590 2024-09-21T07:38:05.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.424+0000 7f457bff3740 1 -- 172.21.15.39:0/4090337716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1bfee9590 msgr2=0x55f1c02ab150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:05.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.424+0000 7f457bff3740 1 --2- 172.21.15.39:0/4090337716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1bfee9590 0x55f1c02ab150 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f4568009bf0 tx=0x7f4568004a40 comp rx=0 tx=0).stop 2024-09-21T07:38:05.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.424+0000 7f457bff3740 1 -- 172.21.15.39:0/4090337716 shutdown_connections 2024-09-21T07:38:05.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.424+0000 7f457bff3740 1 --2- 172.21.15.39:0/4090337716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1bfee9590 0x55f1c02ab150 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.424+0000 7f457bff3740 1 --2- 172.21.15.39:0/4090337716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1bfee13b0 0x55f1c02db0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.424+0000 7f457bff3740 1 --2- 172.21.15.39:0/4090337716 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f1bff0a270 0x55f1c02dab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.424+0000 7f457bff3740 1 -- 172.21.15.39:0/4090337716 >> 172.21.15.39:0/4090337716 conn(0x55f1bfefe3c0 msgr2=0x55f1c02b0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:05.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.424+0000 7f457bff3740 1 -- 172.21.15.39:0/4090337716 shutdown_connections 2024-09-21T07:38:05.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.424+0000 7f457bff3740 1 -- 172.21.15.39:0/4090337716 wait complete. 2024-09-21T07:38:05.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.425+0000 7f457bff3740 1 -- start start 2024-09-21T07:38:05.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.425+0000 7f457bff3740 1 Processor -- start 2024-09-21T07:38:05.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.427+0000 7f457bff3740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f1bfee13b0 0x55f1c02d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:05.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.427+0000 7f457bff3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1bfee9590 0x55f1c02d4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:05.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.427+0000 7f457bff3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1bff0a270 0x55f1c02d5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:05.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.427+0000 7f457bff3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55f1c01ba330 con 0x55f1bff0a270 2024-09-21T07:38:05.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.427+0000 7f457bff3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f1c0217810 con 0x55f1bfee9590 2024-09-21T07:38:05.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.427+0000 7f457bff3740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55f1bff84620 con 0x55f1bfee13b0 2024-09-21T07:38:05.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.427+0000 7f457bff3740 10 client.0 _ll_register_callbacks cb 0x55f1c01bf420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:38:05.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.427+0000 7f456e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1bff0a270 0x55f1c02d5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:05.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.427+0000 7f456e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1bff0a270 0x55f1c02d5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34862/0 (socket says 172.21.15.39:34862) 2024-09-21T07:38:05.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.427+0000 7f456e800640 1 -- 172.21.15.39:0/1492482158 learned_addr learned my addr 172.21.15.39:0/1492482158 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:05.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.427+0000 7f4567e00640 1 --2- 172.21.15.39:0/1492482158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1bfee9590 0x55f1c02d4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:05.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.427+0000 7f456de00640 1 --2- 172.21.15.39:0/1492482158 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f1bfee13b0 0x55f1c02d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:05.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.427+0000 7f456e800640 1 -- 172.21.15.39:0/1492482158 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f1bfee13b0 msgr2=0x55f1c02d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:38:05.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.427+0000 7f456e800640 1 --2- 172.21.15.39:0/1492482158 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f1bfee13b0 0x55f1c02d1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.427+0000 7f456e800640 1 -- 172.21.15.39:0/1492482158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1bfee9590 msgr2=0x55f1c02d4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:38:05.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.427+0000 7f456e800640 1 --2- 172.21.15.39:0/1492482158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1bfee9590 0x55f1c02d4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.428+0000 7f456e800640 1 -- 172.21.15.39:0/1492482158 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1bff75850 con 0x55f1bff0a270 2024-09-21T07:38:05.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.428+0000 7f456e800640 1 --2- 172.21.15.39:0/1492482158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1bff0a270 0x55f1c02d5080 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f4568009d60 tx=0x7f4568003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad148abf8afb4f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:05.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.428+0000 7f4553400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:38:05.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.428+0000 7f4553400640 1 -- 172.21.15.39:0/1492482158 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4568033050 con 0x55f1bff0a270 2024-09-21T07:38:05.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.428+0000 7f4553400640 1 -- 172.21.15.39:0/1492482158 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 7087+0+0 (secure 0 0 0) 0x7f456802f690 con 0x55f1bff0a270 2024-09-21T07:38:05.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.428+0000 7f457bff3740 1 -- 172.21.15.39:0/1492482158 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f456802f980 con 0x55f1bff0a270 2024-09-21T07:38:05.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.428+0000 7f4553400640 1 -- 172.21.15.39:0/1492482158 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f4568043670 con 0x55f1bff0a270 2024-09-21T07:38:05.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.428+0000 7f457bff3740 10 client.6858 fetch_fsmap learned FSMap version 82 2024-09-21T07:38:05.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.428+0000 7f457bff3740 1 -- 172.21.15.39:0/1492482158 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x55f1bffec6b0 con 0x55f1bff0a270 2024-09-21T07:38:05.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.429+0000 7f4553400640 1 -- 172.21.15.39:0/1492482158 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f456802f980 con 0x55f1bff0a270 2024-09-21T07:38:05.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.429+0000 7f4553400640 1 -- 172.21.15.39:0/1492482158 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 82) ==== 3522+0+0 (secure 0 0 0) 0x7f4568043810 con 0x55f1bff0a270 2024-09-21T07:38:05.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.429+0000 7f457bff3740 10 client.6858 fetch_fsmap finished waiting for FSMap version 82 2024-09-21T07:38:05.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.429+0000 7f457bff3740 10 client.6858 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T07:38:05.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.429+0000 7f457bff3740 20 client.6858 populate_metadata read hostname 'smithi039' 2024-09-21T07:38:05.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.429+0000 7f457bff3740 1 --2- 172.21.15.39:0/1492482158 >> [v2:172.21.15.148:6836/2381504293,v1:172.21.15.148:6837/2381504293] conn(0x55f1bfec8e40 0x55f1c02d6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:05.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.429+0000 7f457bff3740 4 client.6858 mds_command: new command op to 5870 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:38:05.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.429+0000 7f457bff3740 1 -- 172.21.15.39:0/1492482158 --> [v2:172.21.15.148:6836/2381504293,v1:172.21.15.148:6837/2381504293] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1c0226520 con 0x55f1bfec8e40 2024-09-21T07:38:05.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.429+0000 7f456de00640 1 --2- 172.21.15.39:0/1492482158 >> [v2:172.21.15.148:6836/2381504293,v1:172.21.15.148:6837/2381504293] conn(0x55f1bfec8e40 0x55f1c02d6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:05.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.430+0000 7f456de00640 1 --2- 172.21.15.39:0/1492482158 >> [v2:172.21.15.148:6836/2381504293,v1:172.21.15.148:6837/2381504293] conn(0x55f1bfec8e40 0x55f1c02d6c40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=321f62a98c8ff835 server_cookie=9d0a05da05e6aba4 in_seq=0 out_seq=0 2024-09-21T07:38:05.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.430+0000 7f4553400640 10 client.6858 ms_handle_connect on v2:172.21.15.148:6836/2381504293 2024-09-21T07:38:05.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.431+0000 7f4553400640 1 -- 172.21.15.39:0/1492482158 <== mds.0 v2:172.21.15.148:6836/2381504293 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f1c0226520 con 0x55f1bfec8e40 2024-09-21T07:38:05.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.431+0000 7f4553400640 10 client.6858 handle_command_reply: tid=0 2024-09-21T07:38:05.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.431+0000 7f457bff3740 1 client.6858 shutdown 2024-09-21T07:38:05.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.431+0000 7f457bff3740 1 -- 172.21.15.39:0/1492482158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1bff0a270 msgr2=0x55f1c02d5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:05.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.431+0000 7f457bff3740 1 --2- 172.21.15.39:0/1492482158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1bff0a270 0x55f1c02d5080 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f4568009d60 tx=0x7f4568003ba0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.432+0000 7f457bff3740 1 -- 172.21.15.39:0/1492482158 shutdown_connections 2024-09-21T07:38:05.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.432+0000 7f457bff3740 1 --2- 172.21.15.39:0/1492482158 >> [v2:172.21.15.148:6836/2381504293,v1:172.21.15.148:6837/2381504293] conn(0x55f1bfec8e40 0x55f1c02d6c40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.432+0000 7f457bff3740 1 --2- 172.21.15.39:0/1492482158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1bff0a270 0x55f1c02d5080 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.432+0000 7f457bff3740 1 --2- 172.21.15.39:0/1492482158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1bfee9590 0x55f1c02d4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.432+0000 7f457bff3740 1 --2- 172.21.15.39:0/1492482158 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f1bfee13b0 0x55f1c02d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.432+0000 7f4553400640 0 client.6858 ms_handle_reset on v2:172.21.15.148:6836/2381504293 2024-09-21T07:38:05.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.432+0000 7f457bff3740 1 -- 172.21.15.39:0/1492482158 >> 172.21.15.39:0/1492482158 conn(0x55f1bfefe3c0 msgr2=0x55f1c02f86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:05.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.432+0000 7f457bff3740 1 -- 172.21.15.39:0/1492482158 shutdown_connections 2024-09-21T07:38:05.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.432+0000 7f457bff3740 1 -- 172.21.15.39:0/1492482158 wait complete. 2024-09-21T07:38:05.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.432+0000 7f457bff3740 20 client.6858 trim_cache size 0 max 16384 2024-09-21T07:38:05.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.451+0000 7f457bff3740 1 Processor -- start 2024-09-21T07:38:05.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.451+0000 7f457bff3740 1 -- start start 2024-09-21T07:38:05.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.452+0000 7f457bff3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1bfee13b0 0x55f1c02f6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:05.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.452+0000 7f457bff3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1bfee9590 0x55f1c02f7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:05.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.452+0000 7f457bff3740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f1bff0a270 0x55f1c0318a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:05.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.452+0000 7f457bff3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4570204300 con 0x55f1bfee9590 2024-09-21T07:38:05.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.452+0000 7f457bff3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f45702049d0 con 0x55f1bfee13b0 2024-09-21T07:38:05.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.452+0000 7f457bff3740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f4570202d00 con 0x55f1bff0a270 2024-09-21T07:38:05.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.453+0000 7f4567e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1bfee9590 0x55f1c02f7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:05.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.453+0000 7f4567e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1bfee9590 0x55f1c02f7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34866/0 (socket says 172.21.15.39:34866) 2024-09-21T07:38:05.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.453+0000 7f4567e00640 1 -- 172.21.15.39:0/3434067996 learned_addr learned my addr 172.21.15.39:0/3434067996 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:05.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.453+0000 7f456de00640 1 --2- 172.21.15.39:0/3434067996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1bfee13b0 0x55f1c02f6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:05.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.453+0000 7f456e800640 1 --2- 172.21.15.39:0/3434067996 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f1bff0a270 0x55f1c0318a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:05.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.453+0000 7f4567e00640 1 -- 172.21.15.39:0/3434067996 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f1bff0a270 msgr2=0x55f1c0318a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:38:05.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.453+0000 7f4567e00640 1 --2- 172.21.15.39:0/3434067996 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f1bff0a270 0x55f1c0318a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.453+0000 7f4567e00640 1 -- 172.21.15.39:0/3434067996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1bfee13b0 msgr2=0x55f1c02f6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:38:05.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.453+0000 7f4567e00640 1 --2- 172.21.15.39:0/3434067996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1bfee13b0 0x55f1c02f6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.453+0000 7f4567e00640 1 -- 172.21.15.39:0/3434067996 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1c02f6450 con 0x55f1bfee9590 2024-09-21T07:38:05.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.453+0000 7f456de00640 1 --2- 172.21.15.39:0/3434067996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1bfee13b0 0x55f1c02f6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:38:05.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.454+0000 7f4567e00640 1 --2- 172.21.15.39:0/3434067996 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1bfee9590 0x55f1c02f7160 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f4558012010 tx=0x7f4558014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e34b0aad1a34520 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:05.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.454+0000 7f455e000640 1 -- 172.21.15.39:0/3434067996 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455802a420 con 0x55f1bfee9590 2024-09-21T07:38:05.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.454+0000 7f455e000640 1 -- 172.21.15.39:0/3434067996 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f455802a5c0 con 0x55f1bfee9590 2024-09-21T07:38:05.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.454+0000 7f455e000640 1 -- 172.21.15.39:0/3434067996 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455802a970 con 0x55f1bfee9590 2024-09-21T07:38:05.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.454+0000 7f457bff3740 1 -- 172.21.15.39:0/3434067996 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1bfee9590 msgr2=0x55f1c02f7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:05.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.454+0000 7f457bff3740 1 --2- 172.21.15.39:0/3434067996 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1bfee9590 0x55f1c02f7160 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f4558012010 tx=0x7f4558014000 comp rx=0 tx=0).stop 2024-09-21T07:38:05.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.455+0000 7f457bff3740 1 -- 172.21.15.39:0/3434067996 shutdown_connections 2024-09-21T07:38:05.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.455+0000 7f457bff3740 1 --2- 172.21.15.39:0/3434067996 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f1bff0a270 0x55f1c0318a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.455+0000 7f457bff3740 1 --2- 172.21.15.39:0/3434067996 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1bfee9590 0x55f1c02f7160 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.455+0000 7f457bff3740 1 --2- 172.21.15.39:0/3434067996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1bfee13b0 0x55f1c02f6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.455+0000 7f457bff3740 1 -- 172.21.15.39:0/3434067996 >> 172.21.15.39:0/3434067996 conn(0x55f1bfefe3c0 msgr2=0x55f1c02ae5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:05.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.455+0000 7f457bff3740 1 -- 172.21.15.39:0/3434067996 shutdown_connections 2024-09-21T07:38:05.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.455+0000 7f457bff3740 1 -- 172.21.15.39:0/3434067996 wait complete. 2024-09-21T07:38:05.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.456+0000 7f457bff3740 1 -- start start 2024-09-21T07:38:05.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.456+0000 7f457bff3740 1 Processor -- start 2024-09-21T07:38:05.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.457+0000 7f457bff3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1bfee13b0 0x55f1c02d2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:05.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.457+0000 7f457bff3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1bfee9590 0x55f1c0374d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:05.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.457+0000 7f457bff3740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f1bff0a270 0x55f1c02d6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:05.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.457+0000 7f457bff3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55f1c01ba330 con 0x55f1bfee13b0 2024-09-21T07:38:05.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.457+0000 7f457bff3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4570203350 con 0x55f1bfee9590 2024-09-21T07:38:05.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.457+0000 7f457bff3740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55f1c02f0b90 con 0x55f1bff0a270 2024-09-21T07:38:05.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.457+0000 7f457bff3740 10 client.0 _ll_register_callbacks cb 0x7f4568009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:38:05.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.458+0000 7f456de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1bfee13b0 0x55f1c02d2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:05.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.458+0000 7f456de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1bfee13b0 0x55f1c02d2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34882/0 (socket says 172.21.15.39:34882) 2024-09-21T07:38:05.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.458+0000 7f456de00640 1 -- 172.21.15.39:0/575514509 learned_addr learned my addr 172.21.15.39:0/575514509 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:05.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.458+0000 7f4567e00640 1 --2- 172.21.15.39:0/575514509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1bfee9590 0x55f1c0374d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:05.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.458+0000 7f456e800640 1 --2- 172.21.15.39:0/575514509 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f1bff0a270 0x55f1c02d6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:05.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.458+0000 7f456de00640 1 -- 172.21.15.39:0/575514509 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f1bff0a270 msgr2=0x55f1c02d6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:38:05.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.458+0000 7f456de00640 1 --2- 172.21.15.39:0/575514509 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f1bff0a270 0x55f1c02d6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.458+0000 7f456de00640 1 -- 172.21.15.39:0/575514509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1bfee9590 msgr2=0x55f1c0374d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:38:05.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.458+0000 7f456de00640 1 --2- 172.21.15.39:0/575514509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1bfee9590 0x55f1c0374d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.458+0000 7f456de00640 1 -- 172.21.15.39:0/575514509 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1c02f74a0 con 0x55f1bfee13b0 2024-09-21T07:38:05.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.458+0000 7f456de00640 1 --2- 172.21.15.39:0/575514509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1bfee13b0 0x55f1c02d2bd0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f456000c340 tx=0x7f456001d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81272a01dfd338f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:05.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.458+0000 7f4553400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:38:05.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.458+0000 7f4553400640 1 -- 172.21.15.39:0/575514509 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456006b8c0 con 0x55f1bfee13b0 2024-09-21T07:38:05.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.458+0000 7f4553400640 1 -- 172.21.15.39:0/575514509 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 7087+0+0 (secure 0 0 0) 0x7f456005e1a0 con 0x55f1bfee13b0 2024-09-21T07:38:05.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.458+0000 7f457bff3740 1 -- 172.21.15.39:0/575514509 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f455802a5c0 con 0x55f1bfee13b0 2024-09-21T07:38:05.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.459+0000 7f4553400640 1 -- 172.21.15.39:0/575514509 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f456007c0b0 con 0x55f1bfee13b0 2024-09-21T07:38:05.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.459+0000 7f457bff3740 10 client.6864 fetch_fsmap learned FSMap version 82 2024-09-21T07:38:05.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.459+0000 7f457bff3740 1 -- 172.21.15.39:0/575514509 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x55f1c03195d0 con 0x55f1bfee13b0 2024-09-21T07:38:05.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.459+0000 7f4553400640 1 -- 172.21.15.39:0/575514509 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f455802a5c0 con 0x55f1bfee13b0 2024-09-21T07:38:05.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.459+0000 7f4553400640 1 -- 172.21.15.39:0/575514509 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 82) ==== 3522+0+0 (secure 0 0 0) 0x7f4560088150 con 0x55f1bfee13b0 2024-09-21T07:38:05.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.459+0000 7f457bff3740 10 client.6864 fetch_fsmap finished waiting for FSMap version 82 2024-09-21T07:38:05.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.459+0000 7f457bff3740 10 client.6864 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T07:38:05.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.459+0000 7f457bff3740 20 client.6864 populate_metadata read hostname 'smithi039' 2024-09-21T07:38:05.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.459+0000 7f457bff3740 1 --2- 172.21.15.39:0/575514509 >> [v2:172.21.15.148:6836/2381504293,v1:172.21.15.148:6837/2381504293] conn(0x55f1c02f7990 0x55f1c0378a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:05.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.459+0000 7f457bff3740 4 client.6864 mds_command: new command op to 5870 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:38:05.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.459+0000 7f457bff3740 1 -- 172.21.15.39:0/575514509 --> [v2:172.21.15.148:6836/2381504293,v1:172.21.15.148:6837/2381504293] -- command(tid 0: {"prefix": "status"}) -- 0x55f1c0221fd0 con 0x55f1c02f7990 2024-09-21T07:38:05.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.459+0000 7f4567e00640 1 --2- 172.21.15.39:0/575514509 >> [v2:172.21.15.148:6836/2381504293,v1:172.21.15.148:6837/2381504293] conn(0x55f1c02f7990 0x55f1c0378a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:05.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.460+0000 7f4567e00640 1 --2- 172.21.15.39:0/575514509 >> [v2:172.21.15.148:6836/2381504293,v1:172.21.15.148:6837/2381504293] conn(0x55f1c02f7990 0x55f1c0378a50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf3b24010a87fe48 server_cookie=d16f2785eb5e660d in_seq=0 out_seq=0 2024-09-21T07:38:05.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.460+0000 7f4553400640 10 client.6864 ms_handle_connect on v2:172.21.15.148:6836/2381504293 2024-09-21T07:38:05.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.460+0000 7f4553400640 1 -- 172.21.15.39:0/575514509 <== mds.0 v2:172.21.15.148:6836/2381504293 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55f1c0221fd0 con 0x55f1c02f7990 2024-09-21T07:38:05.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.460+0000 7f4553400640 10 client.6864 handle_command_reply: tid=0 2024-09-21T07:38:05.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.460+0000 7f457bff3740 1 client.6864 shutdown 2024-09-21T07:38:05.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.461+0000 7f457bff3740 1 -- 172.21.15.39:0/575514509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1bfee13b0 msgr2=0x55f1c02d2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:05.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.461+0000 7f457bff3740 1 --2- 172.21.15.39:0/575514509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1bfee13b0 0x55f1c02d2bd0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f456000c340 tx=0x7f456001d040 comp rx=0 tx=0).stop 2024-09-21T07:38:05.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.461+0000 7f457bff3740 1 -- 172.21.15.39:0/575514509 shutdown_connections 2024-09-21T07:38:05.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.461+0000 7f457bff3740 1 --2- 172.21.15.39:0/575514509 >> [v2:172.21.15.148:6836/2381504293,v1:172.21.15.148:6837/2381504293] conn(0x55f1c02f7990 0x55f1c0378a50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.461+0000 7f457bff3740 1 --2- 172.21.15.39:0/575514509 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55f1bff0a270 0x55f1c02d6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.461+0000 7f457bff3740 1 --2- 172.21.15.39:0/575514509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f1bfee9590 0x55f1c0374d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.461+0000 7f457bff3740 1 --2- 172.21.15.39:0/575514509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1bfee13b0 0x55f1c02d2bd0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.461+0000 7f457bff3740 1 -- 172.21.15.39:0/575514509 >> 172.21.15.39:0/575514509 conn(0x55f1bfefe3c0 msgr2=0x55f1c02d47c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:05.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.461+0000 7f4553400640 0 client.6864 ms_handle_reset on v2:172.21.15.148:6836/2381504293 2024-09-21T07:38:05.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.461+0000 7f457bff3740 1 -- 172.21.15.39:0/575514509 shutdown_connections 2024-09-21T07:38:05.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.461+0000 7f457bff3740 1 -- 172.21.15.39:0/575514509 wait complete. 2024-09-21T07:38:05.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.461+0000 7f457bff3740 20 client.6864 trim_cache size 0 max 16384 2024-09-21T07:38:05.468 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:38:05.468 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:38:05.468 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-09-21T07:38:05.468 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 5870, 2024-09-21T07:38:05.468 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:38:05.468 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:38:05.468 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:38:05.469 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 7.2292094550000003, 2024-09-21T07:38:05.469 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 82, 2024-09-21T07:38:05.469 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 95, 2024-09-21T07:38:05.469 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 95, 2024-09-21T07:38:05.469 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 35.445849752000001 2024-09-21T07:38:05.469 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:38:05.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.462+0000 7f456f200640 1 -- 172.21.15.39:0/2349496989 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f454c078bc0 msgr2=0x7f454c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:05.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.462+0000 7f456f200640 1 --2- 172.21.15.39:0/2349496989 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f454c078bc0 0x7f454c07b0d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f45701f88a0 tx=0x7f4558005f90 comp rx=0 tx=0).stop 2024-09-21T07:38:05.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.462+0000 7f456f200640 1 -- 172.21.15.39:0/2349496989 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45700b4690 msgr2=0x7f457015dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:05.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.462+0000 7f456f200640 1 --2- 172.21.15.39:0/2349496989 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45700b4690 0x7f457015dd90 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f456000c370 tx=0x7f456000c830 comp rx=0 tx=0).stop 2024-09-21T07:38:05.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.462+0000 7f456f200640 1 -- 172.21.15.39:0/2349496989 shutdown_connections 2024-09-21T07:38:05.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.462+0000 7f456f200640 1 --2- 172.21.15.39:0/2349496989 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f454c078bc0 0x7f454c07b0d0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.462+0000 7f456f200640 1 --2- 172.21.15.39:0/2349496989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45700706f0 0x7f4570070b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.462+0000 7f456f200640 1 --2- 172.21.15.39:0/2349496989 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45700cea10 0x7f4570152900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.462+0000 7f456f200640 1 --2- 172.21.15.39:0/2349496989 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45700b4690 0x7f457015dd90 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.462+0000 7f456f200640 1 -- 172.21.15.39:0/2349496989 >> 172.21.15.39:0/2349496989 conn(0x7f45700cd6c0 msgr2=0x7f45700ae5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:05.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.462+0000 7f456f200640 1 -- 172.21.15.39:0/2349496989 shutdown_connections 2024-09-21T07:38:05.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.462+0000 7f456f200640 1 -- 172.21.15.39:0/2349496989 wait complete. 2024-09-21T07:38:05.488 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T07:38:05.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.715+0000 7ff38f200640 1 -- 172.21.15.39:0/2566551353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff36c003a00 msgr2=0x7ff36c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:05.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.715+0000 7ff38f200640 1 --2- 172.21.15.39:0/2566551353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff36c003a00 0x7ff36c003e70 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff37800c260 tx=0x7ff36c005830 comp rx=0 tx=0).stop 2024-09-21T07:38:05.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.715+0000 7ff38d200640 1 -- 172.21.15.39:0/2566551353 reap_dead start 2024-09-21T07:38:05.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.715+0000 7ff38f200640 1 -- 172.21.15.39:0/2566551353 shutdown_connections 2024-09-21T07:38:05.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.715+0000 7ff38f200640 1 -- 172.21.15.39:0/2566551353 >> 172.21.15.39:0/2566551353 conn(0x7ff3880b1cb0 msgr2=0x7ff3880b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:05.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.715+0000 7ff38f200640 1 -- 172.21.15.39:0/2566551353 shutdown_connections 2024-09-21T07:38:05.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.715+0000 7ff38f200640 1 -- 172.21.15.39:0/2566551353 wait complete. 2024-09-21T07:38:05.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.717+0000 7ff38f200640 1 Processor -- start 2024-09-21T07:38:05.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.718+0000 7ff38f200640 1 -- start start 2024-09-21T07:38:05.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.719+0000 7ff38f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff36c003a00 0x7ff38809bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:05.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.719+0000 7ff38f200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff388161a10 0x7ff3881f59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:05.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.719+0000 7ff38f200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3881f5f00 0x7ff38815f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:05.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.719+0000 7ff38f200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff3880701b0 con 0x7ff36c003a00 2024-09-21T07:38:05.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.719+0000 7ff38f200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff3880cf3d0 con 0x7ff3881f5f00 2024-09-21T07:38:05.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.719+0000 7ff38f200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7ff388070330 con 0x7ff388161a10 2024-09-21T07:38:05.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.719+0000 7ff387e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff36c003a00 0x7ff38809bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:05.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.719+0000 7ff387e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff36c003a00 0x7ff38809bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34898/0 (socket says 172.21.15.39:34898) 2024-09-21T07:38:05.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.719+0000 7ff387e00640 1 -- 172.21.15.39:0/1459394726 learned_addr learned my addr 172.21.15.39:0/1459394726 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:05.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.720+0000 7ff38d200640 1 --2- 172.21.15.39:0/1459394726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3881f5f00 0x7ff38815f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:05.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.720+0000 7ff37f400640 1 --2- 172.21.15.39:0/1459394726 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff388161a10 0x7ff3881f59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:05.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.720+0000 7ff387e00640 1 -- 172.21.15.39:0/1459394726 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff388161a10 msgr2=0x7ff3881f59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:05.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.720+0000 7ff387e00640 1 --2- 172.21.15.39:0/1459394726 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff388161a10 0x7ff3881f59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.720+0000 7ff387e00640 1 -- 172.21.15.39:0/1459394726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3881f5f00 msgr2=0x7ff38815f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:05.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.720+0000 7ff387e00640 1 --2- 172.21.15.39:0/1459394726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3881f5f00 0x7ff38815f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.720+0000 7ff387e00640 1 -- 172.21.15.39:0/1459394726 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff38809ab60 con 0x7ff36c003a00 2024-09-21T07:38:05.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.720+0000 7ff387e00640 1 --2- 172.21.15.39:0/1459394726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff36c003a00 0x7ff38809bbc0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff37800c380 tx=0x7ff378006c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:05.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.720+0000 7ff384c00640 1 -- 172.21.15.39:0/1459394726 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff378031040 con 0x7ff36c003a00 2024-09-21T07:38:05.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.720+0000 7ff384c00640 1 -- 172.21.15.39:0/1459394726 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3780035f0 con 0x7ff36c003a00 2024-09-21T07:38:05.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.720+0000 7ff38f200640 1 -- 172.21.15.39:0/1459394726 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff38809ae00 con 0x7ff36c003a00 2024-09-21T07:38:05.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.720+0000 7ff384c00640 1 -- 172.21.15.39:0/1459394726 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff378003a60 con 0x7ff36c003a00 2024-09-21T07:38:05.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.720+0000 7ff38f200640 1 -- 172.21.15.39:0/1459394726 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff38809b3c0 con 0x7ff36c003a00 2024-09-21T07:38:05.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.723+0000 7ff384c00640 1 -- 172.21.15.39:0/1459394726 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff378034070 con 0x7ff36c003a00 2024-09-21T07:38:05.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.724+0000 7ff384c00640 1 --2- 172.21.15.39:0/1459394726 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff364078bf0 0x7ff36407b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:05.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.724+0000 7ff384c00640 1 -- 172.21.15.39:0/1459394726 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 7087+0+0 (secure 0 0 0) 0x7ff3780bb8c0 con 0x7ff36c003a00 2024-09-21T07:38:05.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.724+0000 7ff37f400640 1 --2- 172.21.15.39:0/1459394726 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff364078bf0 0x7ff36407b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:05.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.724+0000 7ff3963bd740 1 Processor -- start 2024-09-21T07:38:05.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.725+0000 7ff3963bd740 1 -- start start 2024-09-21T07:38:05.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.725+0000 7ff37f400640 1 --2- 172.21.15.39:0/1459394726 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff364078bf0 0x7ff36407b100 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff370005eb0 tx=0x7ff370005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:05.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.726+0000 7ff3963bd740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5646a4c9e270 0x5646a506e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:05.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.726+0000 7ff3963bd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646a4c753b0 0x5646a506eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:05.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.726+0000 7ff3963bd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5646a4c7d590 0x5646a503ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:05.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.726+0000 7ff3963bd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5646a4fc8430 con 0x5646a4c7d590 2024-09-21T07:38:05.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.726+0000 7ff3963bd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5646a4fb85d0 con 0x5646a4c753b0 2024-09-21T07:38:05.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.726+0000 7ff3963bd740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x5646a4fa7a00 con 0x5646a4c9e270 2024-09-21T07:38:05.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.726+0000 7ff38d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5646a4c7d590 0x5646a503ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:05.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.726+0000 7ff38d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5646a4c7d590 0x5646a503ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34902/0 (socket says 172.21.15.39:34902) 2024-09-21T07:38:05.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.726+0000 7ff38d200640 1 -- 172.21.15.39:0/1496481299 learned_addr learned my addr 172.21.15.39:0/1496481299 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:05.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.726+0000 7ff387e00640 1 --2- 172.21.15.39:0/1496481299 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5646a4c9e270 0x5646a506e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:05.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.726+0000 7ff37f400640 1 --2- 172.21.15.39:0/1496481299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646a4c753b0 0x5646a506eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:05.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.727+0000 7ff38d200640 1 -- 172.21.15.39:0/1496481299 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5646a4c9e270 msgr2=0x5646a506e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:38:05.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.727+0000 7ff38d200640 1 --2- 172.21.15.39:0/1496481299 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5646a4c9e270 0x5646a506e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.727+0000 7ff38d200640 1 -- 172.21.15.39:0/1496481299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646a4c753b0 msgr2=0x5646a506eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:38:05.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.727+0000 7ff38d200640 1 --2- 172.21.15.39:0/1496481299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646a4c753b0 0x5646a506eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.727+0000 7ff38d200640 1 -- 172.21.15.39:0/1496481299 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646a4f70520 con 0x5646a4c7d590 2024-09-21T07:38:05.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.727+0000 7ff37f400640 1 --2- 172.21.15.39:0/1496481299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646a4c753b0 0x5646a506eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:38:05.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.727+0000 7ff38d200640 1 --2- 172.21.15.39:0/1496481299 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5646a4c7d590 0x5646a503ef60 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7ff3880ca720 tx=0x7ff380003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d6ba43b5629b498 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:05.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.727+0000 7ff37cc00640 1 -- 172.21.15.39:0/1496481299 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff380004770 con 0x5646a4c7d590 2024-09-21T07:38:05.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.727+0000 7ff37cc00640 1 -- 172.21.15.39:0/1496481299 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff380009e30 con 0x5646a4c7d590 2024-09-21T07:38:05.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.727+0000 7ff37cc00640 1 -- 172.21.15.39:0/1496481299 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff380011660 con 0x5646a4c7d590 2024-09-21T07:38:05.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.728+0000 7ff3963bd740 1 -- 172.21.15.39:0/1496481299 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5646a4c7d590 msgr2=0x5646a503ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:05.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.728+0000 7ff3963bd740 1 --2- 172.21.15.39:0/1496481299 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5646a4c7d590 0x5646a503ef60 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7ff3880ca720 tx=0x7ff380003d40 comp rx=0 tx=0).stop 2024-09-21T07:38:05.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.728+0000 7ff3963bd740 1 -- 172.21.15.39:0/1496481299 shutdown_connections 2024-09-21T07:38:05.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.728+0000 7ff3963bd740 1 --2- 172.21.15.39:0/1496481299 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5646a4c7d590 0x5646a503ef60 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.728+0000 7ff3963bd740 1 --2- 172.21.15.39:0/1496481299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646a4c753b0 0x5646a506eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.728+0000 7ff3963bd740 1 --2- 172.21.15.39:0/1496481299 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5646a4c9e270 0x5646a506e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.728+0000 7ff3963bd740 1 -- 172.21.15.39:0/1496481299 >> 172.21.15.39:0/1496481299 conn(0x5646a4c923c0 msgr2=0x5646a5044970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:05.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.728+0000 7ff3963bd740 1 -- 172.21.15.39:0/1496481299 shutdown_connections 2024-09-21T07:38:05.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.728+0000 7ff3963bd740 1 -- 172.21.15.39:0/1496481299 wait complete. 2024-09-21T07:38:05.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.729+0000 7ff3963bd740 1 -- start start 2024-09-21T07:38:05.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.729+0000 7ff3963bd740 1 Processor -- start 2024-09-21T07:38:05.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.731+0000 7ff3963bd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646a4c753b0 0x5646a5083140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:05.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.731+0000 7ff3963bd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5646a4c7d590 0x5646a5083680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:05.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.731+0000 7ff3963bd740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5646a4c9e270 0x5646a509c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:05.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.731+0000 7ff3963bd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5646a4f4e330 con 0x5646a4c7d590 2024-09-21T07:38:05.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.731+0000 7ff3963bd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5646a4fab810 con 0x5646a4c753b0 2024-09-21T07:38:05.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.731+0000 7ff3963bd740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x5646a4d18620 con 0x5646a4c9e270 2024-09-21T07:38:05.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.731+0000 7ff3963bd740 10 client.0 _ll_register_callbacks cb 0x5646a4f53420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:38:05.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.731+0000 7ff37f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5646a4c7d590 0x5646a5083680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:05.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.731+0000 7ff37f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5646a4c7d590 0x5646a5083680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34908/0 (socket says 172.21.15.39:34908) 2024-09-21T07:38:05.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.731+0000 7ff37f400640 1 -- 172.21.15.39:0/1194226235 learned_addr learned my addr 172.21.15.39:0/1194226235 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:05.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.731+0000 7ff38d200640 1 --2- 172.21.15.39:0/1194226235 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5646a4c9e270 0x5646a509c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:05.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.732+0000 7ff387e00640 1 --2- 172.21.15.39:0/1194226235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646a4c753b0 0x5646a5083140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:05.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.732+0000 7ff37f400640 1 -- 172.21.15.39:0/1194226235 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5646a4c9e270 msgr2=0x5646a509c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:38:05.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.732+0000 7ff37f400640 1 --2- 172.21.15.39:0/1194226235 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5646a4c9e270 0x5646a509c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.732+0000 7ff37f400640 1 -- 172.21.15.39:0/1194226235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646a4c753b0 msgr2=0x5646a5083140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:38:05.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.732+0000 7ff37f400640 1 --2- 172.21.15.39:0/1194226235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646a4c753b0 0x5646a5083140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.732+0000 7ff37f400640 1 -- 172.21.15.39:0/1194226235 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646a4d09850 con 0x5646a4c7d590 2024-09-21T07:38:05.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.732+0000 7ff387e00640 1 --2- 172.21.15.39:0/1194226235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646a4c753b0 0x5646a5083140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:38:05.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.732+0000 7ff37f400640 1 --2- 172.21.15.39:0/1194226235 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5646a4c7d590 0x5646a5083680 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7ff370011c10 tx=0x7ff3700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70677316edcdc531 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:05.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.732+0000 7ff375600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:38:05.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.732+0000 7ff375600640 1 -- 172.21.15.39:0/1194226235 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37002b060 con 0x5646a4c7d590 2024-09-21T07:38:05.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.732+0000 7ff375600640 1 -- 172.21.15.39:0/1194226235 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 7087+0+0 (secure 0 0 0) 0x7ff37001db50 con 0x5646a4c7d590 2024-09-21T07:38:05.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.732+0000 7ff3963bd740 1 -- 172.21.15.39:0/1194226235 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff380009e30 con 0x5646a4c7d590 2024-09-21T07:38:05.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.733+0000 7ff375600640 1 -- 172.21.15.39:0/1194226235 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7ff370031050 con 0x5646a4c7d590 2024-09-21T07:38:05.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.733+0000 7ff3963bd740 10 client.6876 fetch_fsmap learned FSMap version 82 2024-09-21T07:38:05.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.733+0000 7ff3963bd740 1 -- 172.21.15.39:0/1194226235 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x5646a4d806b0 con 0x5646a4c7d590 2024-09-21T07:38:05.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.733+0000 7ff375600640 1 -- 172.21.15.39:0/1194226235 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff380009e30 con 0x5646a4c7d590 2024-09-21T07:38:05.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.733+0000 7ff375600640 1 -- 172.21.15.39:0/1194226235 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 82) ==== 3522+0+0 (secure 0 0 0) 0x7ff370034020 con 0x5646a4c7d590 2024-09-21T07:38:05.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.733+0000 7ff3963bd740 10 client.6876 fetch_fsmap finished waiting for FSMap version 82 2024-09-21T07:38:05.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.733+0000 7ff3963bd740 10 client.6876 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:38:05.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.733+0000 7ff3963bd740 20 client.6876 populate_metadata read hostname 'smithi039' 2024-09-21T07:38:05.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.733+0000 7ff3963bd740 1 --2- 172.21.15.39:0/1194226235 >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x5646a4c5ce40 0x5646a5098b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:05.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.733+0000 7ff3963bd740 4 client.6876 mds_command: new command op to 6115 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:38:05.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.733+0000 7ff3963bd740 1 -- 172.21.15.39:0/1194226235 --> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646a4fba520 con 0x5646a4c5ce40 2024-09-21T07:38:05.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.733+0000 7ff387e00640 1 --2- 172.21.15.39:0/1194226235 >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x5646a4c5ce40 0x5646a5098b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:05.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.734+0000 7ff387e00640 1 --2- 172.21.15.39:0/1194226235 >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x5646a4c5ce40 0x5646a5098b20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=952591b0566bd30d server_cookie=a25bcf02e97ad1f3 in_seq=0 out_seq=0 2024-09-21T07:38:05.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.734+0000 7ff375600640 10 client.6876 ms_handle_connect on v2:172.21.15.39:6833/364657398 2024-09-21T07:38:05.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.734+0000 7ff375600640 1 -- 172.21.15.39:0/1194226235 <== mds.1 v2:172.21.15.39:6833/364657398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5646a4fba520 con 0x5646a4c5ce40 2024-09-21T07:38:05.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.734+0000 7ff375600640 10 client.6876 handle_command_reply: tid=0 2024-09-21T07:38:05.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.735+0000 7ff3963bd740 1 client.6876 shutdown 2024-09-21T07:38:05.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.735+0000 7ff3963bd740 1 -- 172.21.15.39:0/1194226235 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5646a4c7d590 msgr2=0x5646a5083680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:05.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.735+0000 7ff3963bd740 1 --2- 172.21.15.39:0/1194226235 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5646a4c7d590 0x5646a5083680 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7ff370011c10 tx=0x7ff3700046c0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.735+0000 7ff3963bd740 1 -- 172.21.15.39:0/1194226235 shutdown_connections 2024-09-21T07:38:05.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.735+0000 7ff3963bd740 1 --2- 172.21.15.39:0/1194226235 >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x5646a4c5ce40 0x5646a5098b20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.735+0000 7ff3963bd740 1 --2- 172.21.15.39:0/1194226235 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5646a4c9e270 0x5646a509c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.735+0000 7ff3963bd740 1 --2- 172.21.15.39:0/1194226235 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5646a4c7d590 0x5646a5083680 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.735+0000 7ff3963bd740 1 --2- 172.21.15.39:0/1194226235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646a4c753b0 0x5646a5083140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.735+0000 7ff375600640 0 client.6876 ms_handle_reset on v2:172.21.15.39:6833/364657398 2024-09-21T07:38:05.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.735+0000 7ff3963bd740 1 -- 172.21.15.39:0/1194226235 >> 172.21.15.39:0/1194226235 conn(0x5646a4c923c0 msgr2=0x5646a503dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:05.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.735+0000 7ff3963bd740 1 -- 172.21.15.39:0/1194226235 shutdown_connections 2024-09-21T07:38:05.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.735+0000 7ff3963bd740 1 -- 172.21.15.39:0/1194226235 wait complete. 2024-09-21T07:38:05.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.735+0000 7ff3963bd740 20 client.6876 trim_cache size 0 max 16384 2024-09-21T07:38:05.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.754+0000 7ff3963bd740 1 Processor -- start 2024-09-21T07:38:05.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.755+0000 7ff3963bd740 1 -- start start 2024-09-21T07:38:05.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.756+0000 7ff3963bd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5646a4c753b0 0x5646a5065a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:05.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.756+0000 7ff3963bd740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5646a4c7d590 0x5646a5064f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:05.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.756+0000 7ff3963bd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646a4c9e270 0x5646a506e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:05.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.756+0000 7ff3963bd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff388204bf0 con 0x5646a4c753b0 2024-09-21T07:38:05.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.756+0000 7ff3963bd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff388202f00 con 0x5646a4c9e270 2024-09-21T07:38:05.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.756+0000 7ff3963bd740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7ff388203550 con 0x5646a4c7d590 2024-09-21T07:38:05.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.756+0000 7ff387e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5646a4c753b0 0x5646a5065a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:05.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.756+0000 7ff387e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5646a4c753b0 0x5646a5065a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34916/0 (socket says 172.21.15.39:34916) 2024-09-21T07:38:05.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.756+0000 7ff387e00640 1 -- 172.21.15.39:0/580869783 learned_addr learned my addr 172.21.15.39:0/580869783 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:05.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.756+0000 7ff37f400640 1 --2- 172.21.15.39:0/580869783 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5646a4c7d590 0x5646a5064f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:05.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.756+0000 7ff38d200640 1 --2- 172.21.15.39:0/580869783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646a4c9e270 0x5646a506e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:05.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.756+0000 7ff387e00640 1 -- 172.21.15.39:0/580869783 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5646a4c7d590 msgr2=0x5646a5064f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:38:05.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.757+0000 7ff387e00640 1 --2- 172.21.15.39:0/580869783 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5646a4c7d590 0x5646a5064f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.757+0000 7ff387e00640 1 -- 172.21.15.39:0/580869783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646a4c9e270 msgr2=0x5646a506e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:38:05.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.757+0000 7ff387e00640 1 --2- 172.21.15.39:0/580869783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646a4c9e270 0x5646a506e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.757+0000 7ff387e00640 1 -- 172.21.15.39:0/580869783 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646a509abf0 con 0x5646a4c753b0 2024-09-21T07:38:05.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.757+0000 7ff387e00640 1 --2- 172.21.15.39:0/580869783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5646a4c753b0 0x5646a5065a10 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7ff3780b9870 tx=0x7ff37807f800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4d512992e4c489 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:05.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.757+0000 7ff377e00640 1 -- 172.21.15.39:0/580869783 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff378087b60 con 0x5646a4c753b0 2024-09-21T07:38:05.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.757+0000 7ff377e00640 1 -- 172.21.15.39:0/580869783 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff37808caa0 con 0x5646a4c753b0 2024-09-21T07:38:05.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.757+0000 7ff377e00640 1 -- 172.21.15.39:0/580869783 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff378003e10 con 0x5646a4c753b0 2024-09-21T07:38:05.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.758+0000 7ff3963bd740 1 -- 172.21.15.39:0/580869783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5646a4c753b0 msgr2=0x5646a5065a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:05.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.758+0000 7ff3963bd740 1 --2- 172.21.15.39:0/580869783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5646a4c753b0 0x5646a5065a10 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7ff3780b9870 tx=0x7ff37807f800 comp rx=0 tx=0).stop 2024-09-21T07:38:05.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.758+0000 7ff3963bd740 1 -- 172.21.15.39:0/580869783 shutdown_connections 2024-09-21T07:38:05.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.758+0000 7ff3963bd740 1 --2- 172.21.15.39:0/580869783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646a4c9e270 0x5646a506e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.758+0000 7ff3963bd740 1 --2- 172.21.15.39:0/580869783 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5646a4c7d590 0x5646a5064f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.758+0000 7ff3963bd740 1 --2- 172.21.15.39:0/580869783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5646a4c753b0 0x5646a5065a10 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.758+0000 7ff3963bd740 1 -- 172.21.15.39:0/580869783 >> 172.21.15.39:0/580869783 conn(0x5646a4c923c0 msgr2=0x5646a50854b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:05.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.758+0000 7ff3963bd740 1 -- 172.21.15.39:0/580869783 shutdown_connections 2024-09-21T07:38:05.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.758+0000 7ff3963bd740 1 -- 172.21.15.39:0/580869783 wait complete. 2024-09-21T07:38:05.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.759+0000 7ff3963bd740 1 -- start start 2024-09-21T07:38:05.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.759+0000 7ff3963bd740 1 Processor -- start 2024-09-21T07:38:05.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.761+0000 7ff3963bd740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5646a4c753b0 0x5646a509b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:05.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.761+0000 7ff3963bd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646a4c7d590 0x5646a509adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:05.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.761+0000 7ff3963bd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5646a4c9e270 0x5646a5066a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:05.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.761+0000 7ff3963bd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5646a5099ed0 con 0x5646a4c9e270 2024-09-21T07:38:05.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.761+0000 7ff3963bd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5646a5084970 con 0x5646a4c7d590 2024-09-21T07:38:05.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.761+0000 7ff3963bd740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x5646a509a540 con 0x5646a4c753b0 2024-09-21T07:38:05.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.761+0000 7ff3963bd740 10 client.0 _ll_register_callbacks cb 0x7ff370011810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:38:05.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.761+0000 7ff38d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5646a4c9e270 0x5646a5066a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:05.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.761+0000 7ff38d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5646a4c9e270 0x5646a5066a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34932/0 (socket says 172.21.15.39:34932) 2024-09-21T07:38:05.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.761+0000 7ff38d200640 1 -- 172.21.15.39:0/1581892588 learned_addr learned my addr 172.21.15.39:0/1581892588 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:05.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.761+0000 7ff387e00640 1 --2- 172.21.15.39:0/1581892588 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5646a4c753b0 0x5646a509b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:05.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.761+0000 7ff37f400640 1 --2- 172.21.15.39:0/1581892588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646a4c7d590 0x5646a509adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:05.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.761+0000 7ff38d200640 1 -- 172.21.15.39:0/1581892588 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5646a4c753b0 msgr2=0x5646a509b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:38:05.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.761+0000 7ff38d200640 1 --2- 172.21.15.39:0/1581892588 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5646a4c753b0 0x5646a509b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.761+0000 7ff38d200640 1 -- 172.21.15.39:0/1581892588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646a4c7d590 msgr2=0x5646a509adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:38:05.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.761+0000 7ff38d200640 1 --2- 172.21.15.39:0/1581892588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646a4c7d590 0x5646a509adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.761+0000 7ff38d200640 1 -- 172.21.15.39:0/1581892588 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646a503dd00 con 0x5646a4c9e270 2024-09-21T07:38:05.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.762+0000 7ff38d200640 1 --2- 172.21.15.39:0/1581892588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5646a4c9e270 0x5646a5066a80 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7ff380016000 tx=0x7ff3800048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62df5fc5e2956d40 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:05.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.762+0000 7ff375600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:38:05.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.762+0000 7ff375600640 1 -- 172.21.15.39:0/1581892588 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff380004d20 con 0x5646a4c9e270 2024-09-21T07:38:05.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.762+0000 7ff375600640 1 -- 172.21.15.39:0/1581892588 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 7087+0+0 (secure 0 0 0) 0x7ff380011710 con 0x5646a4c9e270 2024-09-21T07:38:05.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.762+0000 7ff3963bd740 1 -- 172.21.15.39:0/1581892588 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff37808caa0 con 0x5646a4c9e270 2024-09-21T07:38:05.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.762+0000 7ff375600640 1 -- 172.21.15.39:0/1581892588 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7ff38002f050 con 0x5646a4c9e270 2024-09-21T07:38:05.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.762+0000 7ff3963bd740 10 client.6882 fetch_fsmap learned FSMap version 82 2024-09-21T07:38:05.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.762+0000 7ff3963bd740 1 -- 172.21.15.39:0/1581892588 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x5646a5108e90 con 0x5646a4c9e270 2024-09-21T07:38:05.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.763+0000 7ff375600640 1 -- 172.21.15.39:0/1581892588 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff37808caa0 con 0x5646a4c9e270 2024-09-21T07:38:05.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.763+0000 7ff375600640 1 -- 172.21.15.39:0/1581892588 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 82) ==== 3522+0+0 (secure 0 0 0) 0x7ff380032020 con 0x5646a4c9e270 2024-09-21T07:38:05.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.763+0000 7ff3963bd740 10 client.6882 fetch_fsmap finished waiting for FSMap version 82 2024-09-21T07:38:05.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.763+0000 7ff3963bd740 10 client.6882 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:38:05.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.763+0000 7ff3963bd740 20 client.6882 populate_metadata read hostname 'smithi039' 2024-09-21T07:38:05.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.763+0000 7ff3963bd740 1 --2- 172.21.15.39:0/1581892588 >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x5646a5089ee0 0x5646a508a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:05.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.763+0000 7ff3963bd740 4 client.6882 mds_command: new command op to 6115 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:38:05.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.763+0000 7ff3963bd740 1 -- 172.21.15.39:0/1581892588 --> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] -- command(tid 0: {"prefix": "status"}) -- 0x5646a50846c0 con 0x5646a5089ee0 2024-09-21T07:38:05.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.763+0000 7ff387e00640 1 --2- 172.21.15.39:0/1581892588 >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x5646a5089ee0 0x5646a508a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:05.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.763+0000 7ff387e00640 1 --2- 172.21.15.39:0/1581892588 >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x5646a5089ee0 0x5646a508a2f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5c47b2945d1d01ae server_cookie=f73e30414aaf472e in_seq=0 out_seq=0 2024-09-21T07:38:05.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.763+0000 7ff375600640 10 client.6882 ms_handle_connect on v2:172.21.15.39:6833/364657398 2024-09-21T07:38:05.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.764+0000 7ff375600640 1 -- 172.21.15.39:0/1581892588 <== mds.1 v2:172.21.15.39:6833/364657398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5646a50846c0 con 0x5646a5089ee0 2024-09-21T07:38:05.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.764+0000 7ff375600640 10 client.6882 handle_command_reply: tid=0 2024-09-21T07:38:05.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.764+0000 7ff3963bd740 1 client.6882 shutdown 2024-09-21T07:38:05.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.764+0000 7ff3963bd740 1 -- 172.21.15.39:0/1581892588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5646a4c9e270 msgr2=0x5646a5066a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:05.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.764+0000 7ff3963bd740 1 --2- 172.21.15.39:0/1581892588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5646a4c9e270 0x5646a5066a80 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7ff380016000 tx=0x7ff3800048a0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.764+0000 7ff3963bd740 1 -- 172.21.15.39:0/1581892588 shutdown_connections 2024-09-21T07:38:05.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.764+0000 7ff3963bd740 1 --2- 172.21.15.39:0/1581892588 >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x5646a5089ee0 0x5646a508a2f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.764+0000 7ff3963bd740 1 --2- 172.21.15.39:0/1581892588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5646a4c9e270 0x5646a5066a80 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.764+0000 7ff3963bd740 1 --2- 172.21.15.39:0/1581892588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5646a4c7d590 0x5646a509adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.764+0000 7ff3963bd740 1 --2- 172.21.15.39:0/1581892588 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5646a4c753b0 0x5646a509b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.764+0000 7ff3963bd740 1 -- 172.21.15.39:0/1581892588 >> 172.21.15.39:0/1581892588 conn(0x5646a4c923c0 msgr2=0x5646a5065a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:05.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.764+0000 7ff375600640 0 client.6882 ms_handle_reset on v2:172.21.15.39:6833/364657398 2024-09-21T07:38:05.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.764+0000 7ff3963bd740 1 -- 172.21.15.39:0/1581892588 shutdown_connections 2024-09-21T07:38:05.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.764+0000 7ff3963bd740 1 -- 172.21.15.39:0/1581892588 wait complete. 2024-09-21T07:38:05.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.764+0000 7ff3963bd740 20 client.6882 trim_cache size 0 max 16384 2024-09-21T07:38:05.770 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:38:05.770 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:38:05.770 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 1, 2024-09-21T07:38:05.770 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 6115, 2024-09-21T07:38:05.770 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:38:05.770 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:38:05.770 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:38:05.770 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 94.400833477999996, 2024-09-21T07:38:05.770 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 82, 2024-09-21T07:38:05.770 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 95, 2024-09-21T07:38:05.770 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 95, 2024-09-21T07:38:05.770 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 110.07551634399999 2024-09-21T07:38:05.770 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:38:05.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.765+0000 7ff38f200640 1 -- 172.21.15.39:0/1459394726 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff364078bf0 msgr2=0x7ff36407b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:05.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.765+0000 7ff38f200640 1 --2- 172.21.15.39:0/1459394726 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff364078bf0 0x7ff36407b100 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff370005eb0 tx=0x7ff370005f90 comp rx=0 tx=0).stop 2024-09-21T07:38:05.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.765+0000 7ff38f200640 1 -- 172.21.15.39:0/1459394726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff36c003a00 msgr2=0x7ff38809bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:05.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.765+0000 7ff38f200640 1 --2- 172.21.15.39:0/1459394726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff36c003a00 0x7ff38809bbc0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff37800c380 tx=0x7ff378006c50 comp rx=0 tx=0).stop 2024-09-21T07:38:05.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.765+0000 7ff38f200640 1 -- 172.21.15.39:0/1459394726 shutdown_connections 2024-09-21T07:38:05.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.765+0000 7ff38f200640 1 --2- 172.21.15.39:0/1459394726 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff364078bf0 0x7ff36407b100 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.765+0000 7ff38f200640 1 --2- 172.21.15.39:0/1459394726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3881f5f00 0x7ff38815f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.765+0000 7ff38f200640 1 --2- 172.21.15.39:0/1459394726 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff388161a10 0x7ff3881f59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.765+0000 7ff38f200640 1 --2- 172.21.15.39:0/1459394726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff36c003a00 0x7ff38809bbc0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:05.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.765+0000 7ff38f200640 1 -- 172.21.15.39:0/1459394726 >> 172.21.15.39:0/1459394726 conn(0x7ff3880b1cb0 msgr2=0x7ff3880b0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:05.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.765+0000 7ff38f200640 1 -- 172.21.15.39:0/1459394726 shutdown_connections 2024-09-21T07:38:05.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:05.765+0000 7ff38f200640 1 -- 172.21.15.39:0/1459394726 wait complete. 2024-09-21T07:38:05.788 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:38:05.807 INFO:tasks.ceph.mds.d.smithi148.stderr:2024-09-21T07:38:05.806+0000 7f6063a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:38:05.807 INFO:tasks.ceph.mds.d.smithi148.stderr:2024-09-21T07:38:05.806+0000 7f6063a00640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:38:06.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.012+0000 7f526de00640 1 -- 172.21.15.39:0/1669940566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5268070770 msgr2=0x7f52681604f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:06.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.012+0000 7f526de00640 1 --2- 172.21.15.39:0/1669940566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5268070770 0x7f52681604f0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f5254005fd0 tx=0x7f525402a5d0 comp rx=0 tx=0).stop 2024-09-21T07:38:06.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.012+0000 7f526de00640 1 -- 172.21.15.39:0/1669940566 shutdown_connections 2024-09-21T07:38:06.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.012+0000 7f526de00640 1 --2- 172.21.15.39:0/1669940566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5268070770 0x7f52681604f0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:06.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.012+0000 7f526de00640 1 -- 172.21.15.39:0/1669940566 >> 172.21.15.39:0/1669940566 conn(0x7f52680b3090 msgr2=0x7f52680b34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:06.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.012+0000 7f526de00640 1 -- 172.21.15.39:0/1669940566 shutdown_connections 2024-09-21T07:38:06.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.012+0000 7f526de00640 1 -- 172.21.15.39:0/1669940566 wait complete. 2024-09-21T07:38:06.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.014+0000 7f526de00640 1 Processor -- start 2024-09-21T07:38:06.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.014+0000 7f526de00640 1 -- start start 2024-09-21T07:38:06.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.016+0000 7f526de00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52680cfde0 0x7f52681f3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:06.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.016+0000 7f526de00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52680d01f0 0x7f52681f3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:06.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.016+0000 7f526de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5268157090 0x7f52680b1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:06.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.016+0000 7f526de00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5268160a30 con 0x7f5268157090 2024-09-21T07:38:06.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.016+0000 7f526de00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f526815d8b0 con 0x7f52680cfde0 2024-09-21T07:38:06.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.016+0000 7f526de00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f5268160bb0 con 0x7f52680d01f0 2024-09-21T07:38:06.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.016+0000 7f526d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5268157090 0x7f52680b1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:06.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.016+0000 7f526d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5268157090 0x7f52680b1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34948/0 (socket says 172.21.15.39:34948) 2024-09-21T07:38:06.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.016+0000 7f526d400640 1 -- 172.21.15.39:0/4096916708 learned_addr learned my addr 172.21.15.39:0/4096916708 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:06.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.016+0000 7f5267e00640 1 --2- 172.21.15.39:0/4096916708 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52680d01f0 0x7f52681f3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:06.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.016+0000 7f526ca00640 1 --2- 172.21.15.39:0/4096916708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52680cfde0 0x7f52681f3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:06.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.016+0000 7f526d400640 1 -- 172.21.15.39:0/4096916708 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52680d01f0 msgr2=0x7f52681f3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:06.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.016+0000 7f526d400640 1 --2- 172.21.15.39:0/4096916708 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52680d01f0 0x7f52681f3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:06.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.016+0000 7f526d400640 1 -- 172.21.15.39:0/4096916708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52680cfde0 msgr2=0x7f52681f3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:06.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.016+0000 7f526d400640 1 --2- 172.21.15.39:0/4096916708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52680cfde0 0x7f52681f3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:06.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.016+0000 7f526d400640 1 -- 172.21.15.39:0/4096916708 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52681537a0 con 0x7f5268157090 2024-09-21T07:38:06.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.016+0000 7f5267e00640 1 --2- 172.21.15.39:0/4096916708 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52680d01f0 0x7f52681f3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:38:06.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.017+0000 7f526d400640 1 --2- 172.21.15.39:0/4096916708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5268157090 0x7f52680b1d80 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f526000ba10 tx=0x7f526000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:06.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.017+0000 7f5264c00640 1 -- 172.21.15.39:0/4096916708 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5260019070 con 0x7f5268157090 2024-09-21T07:38:06.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.017+0000 7f5264c00640 1 -- 172.21.15.39:0/4096916708 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f526000cc90 con 0x7f5268157090 2024-09-21T07:38:06.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.017+0000 7f5264c00640 1 -- 172.21.15.39:0/4096916708 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52600146a0 con 0x7f5268157090 2024-09-21T07:38:06.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.017+0000 7f526de00640 1 -- 172.21.15.39:0/4096916708 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5268157870 con 0x7f5268157090 2024-09-21T07:38:06.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.017+0000 7f526de00640 1 -- 172.21.15.39:0/4096916708 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5268157e30 con 0x7f5268157090 2024-09-21T07:38:06.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.018+0000 7f526de00640 1 -- 172.21.15.39:0/4096916708 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52681575e0 con 0x7f5268157090 2024-09-21T07:38:06.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.020+0000 7f5264c00640 1 -- 172.21.15.39:0/4096916708 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f526001c070 con 0x7f5268157090 2024-09-21T07:38:06.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.021+0000 7f5264c00640 1 --2- 172.21.15.39:0/4096916708 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5248078b90 0x7f524807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:06.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.021+0000 7f5264c00640 1 -- 172.21.15.39:0/4096916708 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 7087+0+0 (secure 0 0 0) 0x7f526009b000 con 0x7f5268157090 2024-09-21T07:38:06.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.021+0000 7f526ca00640 1 --2- 172.21.15.39:0/4096916708 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5248078b90 0x7f524807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:06.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.022+0000 7f526ca00640 1 --2- 172.21.15.39:0/4096916708 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5248078b90 0x7f524807b0a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f525c0015e0 tx=0x7f525c009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:06.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.025+0000 7f5264c00640 1 -- 172.21.15.39:0/4096916708 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5260062ee0 con 0x7f5268157090 2024-09-21T07:38:06.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.258+0000 7f526de00640 1 -- 172.21.15.39:0/4096916708 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52681604f0 con 0x7f5268157090 2024-09-21T07:38:06.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.259+0000 7f5264c00640 1 -- 172.21.15.39:0/4096916708 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5246 (secure 0 0 0) 0x7f52600627e0 con 0x7f5268157090 2024-09-21T07:38:06.261 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:06.261 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":82,"btime":"2024-09-21T07:38:02:257520+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6825,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/855150775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":855150775},{"type":"v1","addr":"172.21.15.39:6834","nonce":855150775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:38:02.257514+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5870,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5870":{"gid":5870,"name":"d","rank":0,"incarnation":78,"state":"up:active","state_seq":12,"addr":"172.21.15.148:6837/2381504293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2381504293},{"type":"v1","addr":"172.21.15.148:6837","nonce":2381504293}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5870,"qdb_cluster":[5870,6115]},"id":6}]} 2024-09-21T07:38:06.262 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 82 2024-09-21T07:38:06.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.261+0000 7f526de00640 1 -- 172.21.15.39:0/4096916708 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5248078b90 msgr2=0x7f524807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:06.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.261+0000 7f526de00640 1 --2- 172.21.15.39:0/4096916708 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5248078b90 0x7f524807b0a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f525c0015e0 tx=0x7f525c009040 comp rx=0 tx=0).stop 2024-09-21T07:38:06.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.261+0000 7f526de00640 1 -- 172.21.15.39:0/4096916708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5268157090 msgr2=0x7f52680b1d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:06.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.261+0000 7f526de00640 1 --2- 172.21.15.39:0/4096916708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5268157090 0x7f52680b1d80 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f526000ba10 tx=0x7f526000bed0 comp rx=0 tx=0).stop 2024-09-21T07:38:06.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.261+0000 7f526de00640 1 -- 172.21.15.39:0/4096916708 shutdown_connections 2024-09-21T07:38:06.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.261+0000 7f526de00640 1 --2- 172.21.15.39:0/4096916708 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5248078b90 0x7f524807b0a0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:06.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.261+0000 7f526de00640 1 --2- 172.21.15.39:0/4096916708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5268157090 0x7f52680b1d80 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:06.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.261+0000 7f526de00640 1 --2- 172.21.15.39:0/4096916708 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52680d01f0 0x7f52681f3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:06.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.262+0000 7f526de00640 1 --2- 172.21.15.39:0/4096916708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52680cfde0 0x7f52681f3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:06.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.262+0000 7f526de00640 1 -- 172.21.15.39:0/4096916708 >> 172.21.15.39:0/4096916708 conn(0x7f52680b3090 msgr2=0x7f52680cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:06.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.262+0000 7f526de00640 1 -- 172.21.15.39:0/4096916708 shutdown_connections 2024-09-21T07:38:06.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.262+0000 7f526de00640 1 -- 172.21.15.39:0/4096916708 wait complete. 2024-09-21T07:38:06.289 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json dump snaps --server 2024-09-21T07:38:06.537 INFO:teuthology.orchestra.run.smithi148.stdout:{"last_snap":4,"last_created":4,"last_destroyed":1,"pending_noop":[],"snaps":[{"snapid":2,"ino":1099511627777,"stamp":"2024-09-21T07:36:43.100278+0000","name":"s11","metadata":{}},{"snapid":3,"ino":1099511627777,"stamp":"2024-09-21T07:37:11.742421+0000","name":"s12","metadata":{}},{"snapid":4,"ino":1099511627777,"stamp":"2024-09-21T07:37:40.176395+0000","name":"s15","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2024-09-21T07:38:06.537 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 4, "last_destroyed": 1, "last_snap": 4, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s11", "snapid": 2, "stamp": "2024-09-21T07:36:43.100278+0000" }, { "ino": 1099511627777, "metadata": {}, "name": "s12", "snapid": 3, "stamp": "2024-09-21T07:37:11.742421+0000" }, { "ino": 1099511627777, "metadata": {}, "name": "s15", "snapid": 4, "stamp": "2024-09-21T07:37:40.176395+0000" } ] } 2024-09-21T07:38:06.537 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T07:38:06.537 INFO:tasks.cephfs.test_snapshots:testing snapserver mds_kill_mdstable_at=6 2024-09-21T07:38:06.537 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:38:06.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.764+0000 7f64f5000640 1 -- 172.21.15.39:0/1472330697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64f00706b0 msgr2=0x7f64f00cba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:06.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.764+0000 7f64f5000640 1 --2- 172.21.15.39:0/1472330697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64f00706b0 0x7f64f00cba00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009830 tx=0x7f64dc02a5c0 comp rx=0 tx=0).stop 2024-09-21T07:38:06.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.764+0000 7f64f5000640 1 -- 172.21.15.39:0/1472330697 shutdown_connections 2024-09-21T07:38:06.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.764+0000 7f64f5000640 1 --2- 172.21.15.39:0/1472330697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64f00706b0 0x7f64f00cba00 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:06.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.764+0000 7f64f5000640 1 -- 172.21.15.39:0/1472330697 >> 172.21.15.39:0/1472330697 conn(0x7f64f00b1cb0 msgr2=0x7f64f00b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:06.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.764+0000 7f64f5000640 1 -- 172.21.15.39:0/1472330697 shutdown_connections 2024-09-21T07:38:06.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.764+0000 7f64f5000640 1 -- 172.21.15.39:0/1472330697 wait complete. 2024-09-21T07:38:06.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.766+0000 7f64f5000640 1 Processor -- start 2024-09-21T07:38:06.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.766+0000 7f64f5000640 1 -- start start 2024-09-21T07:38:06.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.768+0000 7f64f5000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64f00ca800 0x7f64f0155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:06.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.768+0000 7f64f5000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64f00cad30 0x7f64f00aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:06.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.768+0000 7f64f5000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f64f01f54e0 0x7f64f00af0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:06.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.768+0000 7f64f5000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f64f0074130 con 0x7f64f00cad30 2024-09-21T07:38:06.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.768+0000 7f64f5000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f64f0073fb0 con 0x7f64f00ca800 2024-09-21T07:38:06.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.768+0000 7f64f5000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f64f00742b0 con 0x7f64f01f54e0 2024-09-21T07:38:06.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.768+0000 7f64ed400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64f00cad30 0x7f64f00aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:06.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.768+0000 7f64ed400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64f00cad30 0x7f64f00aebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34964/0 (socket says 172.21.15.39:34964) 2024-09-21T07:38:06.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.768+0000 7f64ed400640 1 -- 172.21.15.39:0/896589187 learned_addr learned my addr 172.21.15.39:0/896589187 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:06.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.768+0000 7f64ee800640 1 --2- 172.21.15.39:0/896589187 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f64f01f54e0 0x7f64f00af0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:06.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.768+0000 7f64ede00640 1 --2- 172.21.15.39:0/896589187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64f00ca800 0x7f64f0155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:06.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.769+0000 7f64ed400640 1 -- 172.21.15.39:0/896589187 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f64f01f54e0 msgr2=0x7f64f00af0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:06.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.769+0000 7f64ed400640 1 --2- 172.21.15.39:0/896589187 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f64f01f54e0 0x7f64f00af0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:06.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.769+0000 7f64ed400640 1 -- 172.21.15.39:0/896589187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64f00ca800 msgr2=0x7f64f0155580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:06.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.769+0000 7f64ed400640 1 --2- 172.21.15.39:0/896589187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64f00ca800 0x7f64f0155580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:06.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.769+0000 7f64ed400640 1 -- 172.21.15.39:0/896589187 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f009d3d0 con 0x7f64f00cad30 2024-09-21T07:38:06.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.769+0000 7f64ed400640 1 --2- 172.21.15.39:0/896589187 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64f00cad30 0x7f64f00aebb0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f64dc02abb0 tx=0x7f64dc033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:06.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.769+0000 7f64e6000640 1 -- 172.21.15.39:0/896589187 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc031d60 con 0x7f64f00cad30 2024-09-21T07:38:06.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.769+0000 7f64e6000640 1 -- 172.21.15.39:0/896589187 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64dc033e10 con 0x7f64f00cad30 2024-09-21T07:38:06.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.769+0000 7f64e6000640 1 -- 172.21.15.39:0/896589187 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc005260 con 0x7f64f00cad30 2024-09-21T07:38:06.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.769+0000 7f64f5000640 1 -- 172.21.15.39:0/896589187 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64f009d8d0 con 0x7f64f00cad30 2024-09-21T07:38:06.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.769+0000 7f64f5000640 1 -- 172.21.15.39:0/896589187 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64f015d780 con 0x7f64f00cad30 2024-09-21T07:38:06.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.770+0000 7f64f5000640 1 -- 172.21.15.39:0/896589187 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64f009d640 con 0x7f64f00cad30 2024-09-21T07:38:06.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.772+0000 7f64e6000640 1 -- 172.21.15.39:0/896589187 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f64dc005400 con 0x7f64f00cad30 2024-09-21T07:38:06.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.773+0000 7f64e6000640 1 --2- 172.21.15.39:0/896589187 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f64d0078b90 0x7f64d007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:06.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.773+0000 7f64e6000640 1 -- 172.21.15.39:0/896589187 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 7087+0+0 (secure 0 0 0) 0x7f64dc0bc3b0 con 0x7f64f00cad30 2024-09-21T07:38:06.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.773+0000 7f64ede00640 1 --2- 172.21.15.39:0/896589187 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f64d0078b90 0x7f64d007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:06.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.774+0000 7f64ede00640 1 --2- 172.21.15.39:0/896589187 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f64d0078b90 0x7f64d007b0a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f64e0000f60 tx=0x7f64e0009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:06.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:06.777+0000 7f64e6000640 1 -- 172.21.15.39:0/896589187 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f64dc03f030 con 0x7f64f00cad30 2024-09-21T07:38:07.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.014+0000 7f64f5000640 1 -- 172.21.15.39:0/896589187 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f64f00747d0 con 0x7f64f00cad30 2024-09-21T07:38:07.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.014+0000 7f64e6000640 1 -- 172.21.15.39:0/896589187 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5246 (secure 0 0 0) 0x7f64dc083c30 con 0x7f64f00cad30 2024-09-21T07:38:07.016 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:07.017 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":82,"btime":"2024-09-21T07:38:02:257520+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6825,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/855150775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":855150775},{"type":"v1","addr":"172.21.15.39:6834","nonce":855150775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:38:02.257514+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5870,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5870":{"gid":5870,"name":"d","rank":0,"incarnation":78,"state":"up:active","state_seq":12,"addr":"172.21.15.148:6837/2381504293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2381504293},{"type":"v1","addr":"172.21.15.148:6837","nonce":2381504293}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5870,"qdb_cluster":[5870,6115]},"id":6}]} 2024-09-21T07:38:07.017 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 82 2024-09-21T07:38:07.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.017+0000 7f64f5000640 1 -- 172.21.15.39:0/896589187 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f64d0078b90 msgr2=0x7f64d007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:07.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.017+0000 7f64f5000640 1 --2- 172.21.15.39:0/896589187 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f64d0078b90 0x7f64d007b0a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f64e0000f60 tx=0x7f64e0009040 comp rx=0 tx=0).stop 2024-09-21T07:38:07.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.017+0000 7f64f5000640 1 -- 172.21.15.39:0/896589187 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64f00cad30 msgr2=0x7f64f00aebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:07.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.017+0000 7f64f5000640 1 --2- 172.21.15.39:0/896589187 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64f00cad30 0x7f64f00aebb0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f64dc02abb0 tx=0x7f64dc033c20 comp rx=0 tx=0).stop 2024-09-21T07:38:07.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.017+0000 7f64f5000640 1 -- 172.21.15.39:0/896589187 shutdown_connections 2024-09-21T07:38:07.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.017+0000 7f64f5000640 1 --2- 172.21.15.39:0/896589187 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f64d0078b90 0x7f64d007b0a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:07.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.017+0000 7f64f5000640 1 --2- 172.21.15.39:0/896589187 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f64f01f54e0 0x7f64f00af0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:07.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.017+0000 7f64f5000640 1 --2- 172.21.15.39:0/896589187 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64f00cad30 0x7f64f00aebb0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:07.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.017+0000 7f64f5000640 1 --2- 172.21.15.39:0/896589187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64f00ca800 0x7f64f0155580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:07.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.017+0000 7f64f5000640 1 -- 172.21.15.39:0/896589187 >> 172.21.15.39:0/896589187 conn(0x7f64f00b1cb0 msgr2=0x7f64f00d00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:07.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.017+0000 7f64f5000640 1 -- 172.21.15.39:0/896589187 shutdown_connections 2024-09-21T07:38:07.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.017+0000 7f64f5000640 1 -- 172.21.15.39:0/896589187 wait complete. 2024-09-21T07:38:07.045 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:0 true 2024-09-21T07:38:07.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.272+0000 7fde8f200640 1 -- 172.21.15.39:0/18621912 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde900742d0 msgr2=0x7fde901693d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:07.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.272+0000 7fde8f200640 1 --2- 172.21.15.39:0/18621912 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde900742d0 0x7fde901693d0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fde88005fd0 tx=0x7fde8802a8b0 comp rx=0 tx=0).stop 2024-09-21T07:38:07.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.272+0000 7fde8f200640 1 -- 172.21.15.39:0/18621912 shutdown_connections 2024-09-21T07:38:07.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.272+0000 7fde8f200640 1 --2- 172.21.15.39:0/18621912 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde900742d0 0x7fde901693d0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:07.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.272+0000 7fde8f200640 1 -- 172.21.15.39:0/18621912 >> 172.21.15.39:0/18621912 conn(0x7fde9006d410 msgr2=0x7fde900b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:07.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.272+0000 7fde8f200640 1 -- 172.21.15.39:0/18621912 shutdown_connections 2024-09-21T07:38:07.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.272+0000 7fde8f200640 1 -- 172.21.15.39:0/18621912 wait complete. 2024-09-21T07:38:07.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.275+0000 7fde8f200640 1 Processor -- start 2024-09-21T07:38:07.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.275+0000 7fde8f200640 1 -- start start 2024-09-21T07:38:07.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.276+0000 7fde8f200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde9006ee70 0x7fde901f9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:07.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.276+0000 7fde8f200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde900742d0 0x7fde90148310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:07.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.276+0000 7fde8f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde90158e60 0x7fde90157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:07.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.276+0000 7fde8f200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fde90169ca0 con 0x7fde90158e60 2024-09-21T07:38:07.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.276+0000 7fde8f200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fde90169b20 con 0x7fde9006ee70 2024-09-21T07:38:07.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.276+0000 7fde8f200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fde90169e20 con 0x7fde900742d0 2024-09-21T07:38:07.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.276+0000 7fde8e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde90158e60 0x7fde90157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:07.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.276+0000 7fde8e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde90158e60 0x7fde90157cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34986/0 (socket says 172.21.15.39:34986) 2024-09-21T07:38:07.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.277+0000 7fde8e800640 1 -- 172.21.15.39:0/2230421504 learned_addr learned my addr 172.21.15.39:0/2230421504 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:07.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.277+0000 7fde8de00640 1 --2- 172.21.15.39:0/2230421504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde9006ee70 0x7fde901f9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:07.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.277+0000 7fde8d400640 1 --2- 172.21.15.39:0/2230421504 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde900742d0 0x7fde90148310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:07.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.277+0000 7fde8e800640 1 -- 172.21.15.39:0/2230421504 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde900742d0 msgr2=0x7fde90148310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:07.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.277+0000 7fde8e800640 1 --2- 172.21.15.39:0/2230421504 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde900742d0 0x7fde90148310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:07.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.277+0000 7fde8e800640 1 -- 172.21.15.39:0/2230421504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde9006ee70 msgr2=0x7fde901f9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:07.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.277+0000 7fde8e800640 1 --2- 172.21.15.39:0/2230421504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde9006ee70 0x7fde901f9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:07.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.277+0000 7fde8e800640 1 -- 172.21.15.39:0/2230421504 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde900adbb0 con 0x7fde90158e60 2024-09-21T07:38:07.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.277+0000 7fde8e800640 1 --2- 172.21.15.39:0/2230421504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde90158e60 0x7fde90157cf0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fde8802aea0 tx=0x7fde880039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:07.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.277+0000 7fde86000640 1 -- 172.21.15.39:0/2230421504 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde88031990 con 0x7fde90158e60 2024-09-21T07:38:07.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.278+0000 7fde86000640 1 -- 172.21.15.39:0/2230421504 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde88031c20 con 0x7fde90158e60 2024-09-21T07:38:07.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.278+0000 7fde86000640 1 -- 172.21.15.39:0/2230421504 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde880330d0 con 0x7fde90158e60 2024-09-21T07:38:07.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.278+0000 7fde8f200640 1 -- 172.21.15.39:0/2230421504 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde900ade90 con 0x7fde90158e60 2024-09-21T07:38:07.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.278+0000 7fde8f200640 1 -- 172.21.15.39:0/2230421504 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde900ad8a0 con 0x7fde90158e60 2024-09-21T07:38:07.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.279+0000 7fde8f200640 1 -- 172.21.15.39:0/2230421504 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde9006f280 con 0x7fde90158e60 2024-09-21T07:38:07.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.280+0000 7fde86000640 1 -- 172.21.15.39:0/2230421504 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fde88033290 con 0x7fde90158e60 2024-09-21T07:38:07.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.281+0000 7fde86000640 1 --2- 172.21.15.39:0/2230421504 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fde70078bc0 0x7fde7007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:07.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.281+0000 7fde86000640 1 -- 172.21.15.39:0/2230421504 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 7087+0+0 (secure 0 0 0) 0x7fde880bc710 con 0x7fde90158e60 2024-09-21T07:38:07.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.282+0000 7fde8de00640 1 --2- 172.21.15.39:0/2230421504 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fde70078bc0 0x7fde7007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:07.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.282+0000 7fde8de00640 1 --2- 172.21.15.39:0/2230421504 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fde70078bc0 0x7fde7007b0d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fde80006fd0 tx=0x7fde80008040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:07.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.286+0000 7fde86000640 1 -- 172.21.15.39:0/2230421504 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fde8804a030 con 0x7fde90158e60 2024-09-21T07:38:07.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:07.558+0000 7fde8f200640 1 -- 172.21.15.39:0/2230421504 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"} v 0) -- 0x7fde9014e300 con 0x7fde90158e60 2024-09-21T07:38:08.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:08.314+0000 7fde86000640 1 -- 172.21.15.39:0/2230421504 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"}]=0 freezing mds.5870 v83) ==== 112+0+0 (secure 0 0 0) 0x7fde8803f030 con 0x7fde90158e60 2024-09-21T07:38:08.317 INFO:teuthology.orchestra.run.smithi039.stderr:freezing mds.5870 2024-09-21T07:38:08.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:08.317+0000 7fde8f200640 1 -- 172.21.15.39:0/2230421504 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fde70078bc0 msgr2=0x7fde7007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:08.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:08.317+0000 7fde8f200640 1 --2- 172.21.15.39:0/2230421504 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fde70078bc0 0x7fde7007b0d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fde80006fd0 tx=0x7fde80008040 comp rx=0 tx=0).stop 2024-09-21T07:38:08.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:08.317+0000 7fde8f200640 1 -- 172.21.15.39:0/2230421504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde90158e60 msgr2=0x7fde90157cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:08.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:08.317+0000 7fde8f200640 1 --2- 172.21.15.39:0/2230421504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde90158e60 0x7fde90157cf0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fde8802aea0 tx=0x7fde880039f0 comp rx=0 tx=0).stop 2024-09-21T07:38:08.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:08.317+0000 7fde8f200640 1 -- 172.21.15.39:0/2230421504 shutdown_connections 2024-09-21T07:38:08.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:08.317+0000 7fde8f200640 1 --2- 172.21.15.39:0/2230421504 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fde70078bc0 0x7fde7007b0d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:08.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:08.317+0000 7fde8f200640 1 --2- 172.21.15.39:0/2230421504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde90158e60 0x7fde90157cf0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:08.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:08.317+0000 7fde8f200640 1 --2- 172.21.15.39:0/2230421504 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde900742d0 0x7fde90148310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:08.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:08.317+0000 7fde8f200640 1 --2- 172.21.15.39:0/2230421504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde9006ee70 0x7fde901f9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:08.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:08.317+0000 7fde8f200640 1 -- 172.21.15.39:0/2230421504 >> 172.21.15.39:0/2230421504 conn(0x7fde9006d410 msgr2=0x7fde9015db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:08.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:08.317+0000 7fde8f200640 1 -- 172.21.15.39:0/2230421504 shutdown_connections 2024-09-21T07:38:08.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:08.317+0000 7fde8f200640 1 -- 172.21.15.39:0/2230421504 wait complete. 2024-09-21T07:38:08.346 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json config set mds_kill_mdstable_at 6 2024-09-21T07:38:08.594 INFO:teuthology.orchestra.run.smithi148.stdout:{"success":"mds_kill_mdstable_at = '' (not observed, change may require restart) "} 2024-09-21T07:38:08.594 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_kill_mdstable_at = '' (not observed, change may require restart) " } 2024-09-21T07:38:08.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:38:08.595 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d1/dir/.snap/s16) 2024-09-21T07:38:08.599 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-09-21T07:38:08.877 INFO:teuthology.orchestra.run.smithi039.stdout:{"mds_beacon_grace":"15.000000"} 2024-09-21T07:38:08.877 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-09-21T07:38:08.877 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:38:09.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.103+0000 7f8c58e00640 1 -- 172.21.15.39:0/2574585507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c540706b0 msgr2=0x7f8c540735b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:09.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.103+0000 7f8c58e00640 1 --2- 172.21.15.39:0/2574585507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c540706b0 0x7f8c540735b0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c005fd0 tx=0x7f8c4c02aaa0 comp rx=0 tx=0).stop 2024-09-21T07:38:09.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.104+0000 7f8c58e00640 1 -- 172.21.15.39:0/2574585507 shutdown_connections 2024-09-21T07:38:09.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.104+0000 7f8c58e00640 1 --2- 172.21.15.39:0/2574585507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c540706b0 0x7f8c540735b0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:09.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.104+0000 7f8c58e00640 1 --2- 172.21.15.39:0/2574585507 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8c540ca800 0x7f8c540cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:09.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.104+0000 7f8c58e00640 1 -- 172.21.15.39:0/2574585507 >> 172.21.15.39:0/2574585507 conn(0x7f8c540b1cb0 msgr2=0x7f8c540b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:09.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.104+0000 7f8c58e00640 1 -- 172.21.15.39:0/2574585507 shutdown_connections 2024-09-21T07:38:09.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.104+0000 7f8c58e00640 1 -- 172.21.15.39:0/2574585507 wait complete. 2024-09-21T07:38:09.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.106+0000 7f8c58e00640 1 Processor -- start 2024-09-21T07:38:09.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.106+0000 7f8c58e00640 1 -- start start 2024-09-21T07:38:09.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.108+0000 7f8c58e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c540ca800 0x7f8c5415b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:09.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.108+0000 7f8c58e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8c540cad30 0x7f8c541f5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:09.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.108+0000 7f8c58e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c540ae740 0x7f8c540c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:09.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.108+0000 7f8c58e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8c54073f60 con 0x7f8c540ae740 2024-09-21T07:38:09.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.108+0000 7f8c58e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8c54073de0 con 0x7f8c540ca800 2024-09-21T07:38:09.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.108+0000 7f8c58e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f8c540740e0 con 0x7f8c540cad30 2024-09-21T07:38:09.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.108+0000 7f8c52800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c540ae740 0x7f8c540c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:09.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.108+0000 7f8c52800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c540ae740 0x7f8c540c81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35012/0 (socket says 172.21.15.39:35012) 2024-09-21T07:38:09.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.108+0000 7f8c52800640 1 -- 172.21.15.39:0/2724259127 learned_addr learned my addr 172.21.15.39:0/2724259127 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:09.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.108+0000 7f8c51400640 1 --2- 172.21.15.39:0/2724259127 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8c540cad30 0x7f8c541f5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:09.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.108+0000 7f8c51e00640 1 --2- 172.21.15.39:0/2724259127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c540ca800 0x7f8c5415b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:09.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.108+0000 7f8c52800640 1 -- 172.21.15.39:0/2724259127 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8c540cad30 msgr2=0x7f8c541f5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:09.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.108+0000 7f8c52800640 1 --2- 172.21.15.39:0/2724259127 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8c540cad30 0x7f8c541f5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:09.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.108+0000 7f8c52800640 1 -- 172.21.15.39:0/2724259127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c540ca800 msgr2=0x7f8c5415b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:09.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.108+0000 7f8c52800640 1 --2- 172.21.15.39:0/2724259127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c540ca800 0x7f8c5415b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:09.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.108+0000 7f8c52800640 1 -- 172.21.15.39:0/2724259127 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c5409e0a0 con 0x7f8c540ae740 2024-09-21T07:38:09.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.109+0000 7f8c52800640 1 --2- 172.21.15.39:0/2724259127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c540ae740 0x7f8c540c81d0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c006010 tx=0x7f8c4c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:09.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.109+0000 7f8c4a000640 1 -- 172.21.15.39:0/2724259127 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c044070 con 0x7f8c540ae740 2024-09-21T07:38:09.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.109+0000 7f8c4a000640 1 -- 172.21.15.39:0/2724259127 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c4c004040 con 0x7f8c540ae740 2024-09-21T07:38:09.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.109+0000 7f8c58e00640 1 -- 172.21.15.39:0/2724259127 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c5409d820 con 0x7f8c540ae740 2024-09-21T07:38:09.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.109+0000 7f8c4a000640 1 -- 172.21.15.39:0/2724259127 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c004430 con 0x7f8c540ae740 2024-09-21T07:38:09.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.109+0000 7f8c58e00640 1 -- 172.21.15.39:0/2724259127 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c541f6040 con 0x7f8c540ae740 2024-09-21T07:38:09.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.110+0000 7f8c58e00640 1 -- 172.21.15.39:0/2724259127 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c54074600 con 0x7f8c540ae740 2024-09-21T07:38:09.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.112+0000 7f8c4a000640 1 -- 172.21.15.39:0/2724259127 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8c4c047070 con 0x7f8c540ae740 2024-09-21T07:38:09.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.113+0000 7f8c4a000640 1 --2- 172.21.15.39:0/2724259127 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8c34078b90 0x7f8c3407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:09.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.113+0000 7f8c4a000640 1 -- 172.21.15.39:0/2724259127 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 7087+0+0 (secure 0 0 0) 0x7f8c4c0bc080 con 0x7f8c540ae740 2024-09-21T07:38:09.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.113+0000 7f8c51e00640 1 --2- 172.21.15.39:0/2724259127 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8c34078b90 0x7f8c3407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:09.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.114+0000 7f8c51e00640 1 --2- 172.21.15.39:0/2724259127 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8c34078b90 0x7f8c3407b0a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8c540cb250 tx=0x7f8c44005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:09.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.117+0000 7f8c4a000640 1 -- 172.21.15.39:0/2724259127 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8c5409e0a0 con 0x7f8c540ae740 2024-09-21T07:38:09.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.358+0000 7f8c58e00640 1 -- 172.21.15.39:0/2724259127 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c540b4d20 con 0x7f8c540ae740 2024-09-21T07:38:09.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.358+0000 7f8c4a000640 1 -- 172.21.15.39:0/2724259127 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5246 (secure 0 0 0) 0x7f8c4c032030 con 0x7f8c540ae740 2024-09-21T07:38:09.360 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:09.361 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":83,"btime":"2024-09-21T07:38:08:302732+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6825,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/855150775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":855150775},{"type":"v1","addr":"172.21.15.39:6834","nonce":855150775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:38:07.560346+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5870,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5870":{"gid":5870,"name":"d","rank":0,"incarnation":78,"state":"up:active","state_seq":12,"addr":"172.21.15.148:6837/2381504293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2381504293},{"type":"v1","addr":"172.21.15.148:6837","nonce":2381504293}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5870,"qdb_cluster":[6115,5870]},"id":6}]} 2024-09-21T07:38:09.361 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 83 2024-09-21T07:38:09.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.361+0000 7f8c58e00640 1 -- 172.21.15.39:0/2724259127 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8c34078b90 msgr2=0x7f8c3407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:09.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.361+0000 7f8c58e00640 1 --2- 172.21.15.39:0/2724259127 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8c34078b90 0x7f8c3407b0a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8c540cb250 tx=0x7f8c44005f90 comp rx=0 tx=0).stop 2024-09-21T07:38:09.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.361+0000 7f8c58e00640 1 -- 172.21.15.39:0/2724259127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c540ae740 msgr2=0x7f8c540c81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:09.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.361+0000 7f8c58e00640 1 --2- 172.21.15.39:0/2724259127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c540ae740 0x7f8c540c81d0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c006010 tx=0x7f8c4c003ce0 comp rx=0 tx=0).stop 2024-09-21T07:38:09.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.361+0000 7f8c58e00640 1 -- 172.21.15.39:0/2724259127 shutdown_connections 2024-09-21T07:38:09.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.361+0000 7f8c58e00640 1 --2- 172.21.15.39:0/2724259127 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8c34078b90 0x7f8c3407b0a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:09.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.361+0000 7f8c58e00640 1 --2- 172.21.15.39:0/2724259127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c540ae740 0x7f8c540c81d0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:09.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.361+0000 7f8c58e00640 1 --2- 172.21.15.39:0/2724259127 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8c540cad30 0x7f8c541f5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:09.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.361+0000 7f8c58e00640 1 --2- 172.21.15.39:0/2724259127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c540ca800 0x7f8c5415b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:09.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.361+0000 7f8c58e00640 1 -- 172.21.15.39:0/2724259127 >> 172.21.15.39:0/2724259127 conn(0x7f8c540b1cb0 msgr2=0x7f8c540b3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:09.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.361+0000 7f8c58e00640 1 -- 172.21.15.39:0/2724259127 shutdown_connections 2024-09-21T07:38:09.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:09.361+0000 7f8c58e00640 1 -- 172.21.15.39:0/2724259127 wait complete. 2024-09-21T07:38:09.390 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:38:10.022 INFO:tasks.ceph.mds.d.smithi148.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_commit_logged(ceph::cref_t&)' thread 7f605ea00640 time 2024-09-21T07:38:10.021805+0000 2024-09-21T07:38:10.022 INFO:tasks.ceph.mds.d.smithi148.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: 173: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 6) 2024-09-21T07:38:10.023 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:38:10.023 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f606dc5654b] 2024-09-21T07:38:10.023 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f606dc56758] 2024-09-21T07:38:10.023 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x1d9) [0x55a96554a76f] 2024-09-21T07:38:10.023 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: ceph-mds(+0x479942) [0x55a96554a942] 2024-09-21T07:38:10.023 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (MDSContext::complete(int)+0x7c) [0x55a9655986d8] 2024-09-21T07:38:10.023 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: (MDSIOContextBase::complete(int)+0x501) [0x55a965598d57] 2024-09-21T07:38:10.023 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSLogContextBase::complete(int)+0x46) [0x55a965598dd4] 2024-09-21T07:38:10.023 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: (Finisher::finisher_thread_entry()+0x495) [0x7f606dbde34b] 2024-09-21T07:38:10.023 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: ceph-mds(+0x1052e1) [0x55a9651d62e1] 2024-09-21T07:38:10.023 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: (Thread::entry_wrapper()+0x43) [0x7f606dc2fd57] 2024-09-21T07:38:10.024 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (Thread::_entry_func(void*)+0xd) [0x7f606dc2fd73] 2024-09-21T07:38:10.024 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: /lib64/libc.so.6(+0x89c52) [0x7f606d289c52] 2024-09-21T07:38:10.024 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: /lib64/libc.so.6(+0x10ec80) [0x7f606d30ec80] 2024-09-21T07:38:10.024 INFO:tasks.ceph.mds.d.smithi148.stderr:2024-09-21T07:38:10.021+0000 7f605ea00640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_commit_logged(ceph::cref_t&)' thread 7f605ea00640 time 2024-09-21T07:38:10.021805+0000 2024-09-21T07:38:10.024 INFO:tasks.ceph.mds.d.smithi148.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: 173: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 6) 2024-09-21T07:38:10.024 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:38:10.024 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:38:10.024 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f606dc5654b] 2024-09-21T07:38:10.024 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f606dc56758] 2024-09-21T07:38:10.024 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x1d9) [0x55a96554a76f] 2024-09-21T07:38:10.024 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: ceph-mds(+0x479942) [0x55a96554a942] 2024-09-21T07:38:10.024 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (MDSContext::complete(int)+0x7c) [0x55a9655986d8] 2024-09-21T07:38:10.024 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: (MDSIOContextBase::complete(int)+0x501) [0x55a965598d57] 2024-09-21T07:38:10.025 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSLogContextBase::complete(int)+0x46) [0x55a965598dd4] 2024-09-21T07:38:10.025 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: (Finisher::finisher_thread_entry()+0x495) [0x7f606dbde34b] 2024-09-21T07:38:10.025 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: ceph-mds(+0x1052e1) [0x55a9651d62e1] 2024-09-21T07:38:10.025 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: (Thread::entry_wrapper()+0x43) [0x7f606dc2fd57] 2024-09-21T07:38:10.025 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (Thread::_entry_func(void*)+0xd) [0x7f606dc2fd73] 2024-09-21T07:38:10.025 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: /lib64/libc.so.6(+0x89c52) [0x7f606d289c52] 2024-09-21T07:38:10.025 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: /lib64/libc.so.6(+0x10ec80) [0x7f606d30ec80] 2024-09-21T07:38:10.025 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:38:10.025 INFO:tasks.ceph.mds.d.smithi148.stderr:*** Caught signal (Aborted) ** 2024-09-21T07:38:10.025 INFO:tasks.ceph.mds.d.smithi148.stderr: in thread 7f605ea00640 thread_name:MR_Finisher 2024-09-21T07:38:10.026 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:38:10.026 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f606d23e6f0] 2024-09-21T07:38:10.026 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f606d28b99c] 2024-09-21T07:38:10.026 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: raise() 2024-09-21T07:38:10.026 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: abort() 2024-09-21T07:38:10.026 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7f606dc56686] 2024-09-21T07:38:10.026 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f606dc56758] 2024-09-21T07:38:10.026 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x1d9) [0x55a96554a76f] 2024-09-21T07:38:10.026 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: ceph-mds(+0x479942) [0x55a96554a942] 2024-09-21T07:38:10.026 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: (MDSContext::complete(int)+0x7c) [0x55a9655986d8] 2024-09-21T07:38:10.026 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: (MDSIOContextBase::complete(int)+0x501) [0x55a965598d57] 2024-09-21T07:38:10.027 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (MDSLogContextBase::complete(int)+0x46) [0x55a965598dd4] 2024-09-21T07:38:10.027 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: (Finisher::finisher_thread_entry()+0x495) [0x7f606dbde34b] 2024-09-21T07:38:10.027 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: ceph-mds(+0x1052e1) [0x55a9651d62e1] 2024-09-21T07:38:10.027 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: (Thread::entry_wrapper()+0x43) [0x7f606dc2fd57] 2024-09-21T07:38:10.027 INFO:tasks.ceph.mds.d.smithi148.stderr: 15: (Thread::_entry_func(void*)+0xd) [0x7f606dc2fd73] 2024-09-21T07:38:10.027 INFO:tasks.ceph.mds.d.smithi148.stderr: 16: /lib64/libc.so.6(+0x89c52) [0x7f606d289c52] 2024-09-21T07:38:10.027 INFO:tasks.ceph.mds.d.smithi148.stderr: 17: /lib64/libc.so.6(+0x10ec80) [0x7f606d30ec80] 2024-09-21T07:38:10.027 INFO:tasks.ceph.mds.d.smithi148.stderr:2024-09-21T07:38:10.022+0000 7f605ea00640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:38:10.027 INFO:tasks.ceph.mds.d.smithi148.stderr: in thread 7f605ea00640 thread_name:MR_Finisher 2024-09-21T07:38:10.027 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:38:10.027 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:38:10.027 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f606d23e6f0] 2024-09-21T07:38:10.027 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f606d28b99c] 2024-09-21T07:38:10.028 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: raise() 2024-09-21T07:38:10.028 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: abort() 2024-09-21T07:38:10.028 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7f606dc56686] 2024-09-21T07:38:10.028 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f606dc56758] 2024-09-21T07:38:10.028 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x1d9) [0x55a96554a76f] 2024-09-21T07:38:10.028 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: ceph-mds(+0x479942) [0x55a96554a942] 2024-09-21T07:38:10.028 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: (MDSContext::complete(int)+0x7c) [0x55a9655986d8] 2024-09-21T07:38:10.028 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: (MDSIOContextBase::complete(int)+0x501) [0x55a965598d57] 2024-09-21T07:38:10.028 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (MDSLogContextBase::complete(int)+0x46) [0x55a965598dd4] 2024-09-21T07:38:10.028 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: (Finisher::finisher_thread_entry()+0x495) [0x7f606dbde34b] 2024-09-21T07:38:10.028 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: ceph-mds(+0x1052e1) [0x55a9651d62e1] 2024-09-21T07:38:10.028 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: (Thread::entry_wrapper()+0x43) [0x7f606dc2fd57] 2024-09-21T07:38:10.028 INFO:tasks.ceph.mds.d.smithi148.stderr: 15: (Thread::_entry_func(void*)+0xd) [0x7f606dc2fd73] 2024-09-21T07:38:10.029 INFO:tasks.ceph.mds.d.smithi148.stderr: 16: /lib64/libc.so.6(+0x89c52) [0x7f606d289c52] 2024-09-21T07:38:10.029 INFO:tasks.ceph.mds.d.smithi148.stderr: 17: /lib64/libc.so.6(+0x10ec80) [0x7f606d30ec80] 2024-09-21T07:38:10.029 INFO:tasks.ceph.mds.d.smithi148.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:38:10.029 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:38:10.034 INFO:tasks.ceph.mds.d.smithi148.stderr: -274> 2024-09-21T07:38:05.806+0000 7f6063a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:38:10.034 INFO:tasks.ceph.mds.d.smithi148.stderr: -272> 2024-09-21T07:38:05.806+0000 7f6063a00640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:38:10.034 INFO:tasks.ceph.mds.d.smithi148.stderr: -4> 2024-09-21T07:38:10.021+0000 7f605ea00640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_commit_logged(ceph::cref_t&)' thread 7f605ea00640 time 2024-09-21T07:38:10.021805+0000 2024-09-21T07:38:10.034 INFO:tasks.ceph.mds.d.smithi148.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: 173: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 6) 2024-09-21T07:38:10.035 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:38:10.035 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:38:10.035 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f606dc5654b] 2024-09-21T07:38:10.035 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f606dc56758] 2024-09-21T07:38:10.035 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x1d9) [0x55a96554a76f] 2024-09-21T07:38:10.035 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: ceph-mds(+0x479942) [0x55a96554a942] 2024-09-21T07:38:10.035 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (MDSContext::complete(int)+0x7c) [0x55a9655986d8] 2024-09-21T07:38:10.035 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: (MDSIOContextBase::complete(int)+0x501) [0x55a965598d57] 2024-09-21T07:38:10.035 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSLogContextBase::complete(int)+0x46) [0x55a965598dd4] 2024-09-21T07:38:10.036 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: (Finisher::finisher_thread_entry()+0x495) [0x7f606dbde34b] 2024-09-21T07:38:10.036 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: ceph-mds(+0x1052e1) [0x55a9651d62e1] 2024-09-21T07:38:10.036 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: (Thread::entry_wrapper()+0x43) [0x7f606dc2fd57] 2024-09-21T07:38:10.036 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (Thread::_entry_func(void*)+0xd) [0x7f606dc2fd73] 2024-09-21T07:38:10.036 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: /lib64/libc.so.6(+0x89c52) [0x7f606d289c52] 2024-09-21T07:38:10.036 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: /lib64/libc.so.6(+0x10ec80) [0x7f606d30ec80] 2024-09-21T07:38:10.036 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:38:10.036 INFO:tasks.ceph.mds.d.smithi148.stderr: 0> 2024-09-21T07:38:10.022+0000 7f605ea00640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:38:10.037 INFO:tasks.ceph.mds.d.smithi148.stderr: in thread 7f605ea00640 thread_name:MR_Finisher 2024-09-21T07:38:10.037 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:38:10.037 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:38:10.037 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f606d23e6f0] 2024-09-21T07:38:10.037 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f606d28b99c] 2024-09-21T07:38:10.037 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: raise() 2024-09-21T07:38:10.037 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: abort() 2024-09-21T07:38:10.037 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7f606dc56686] 2024-09-21T07:38:10.037 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f606dc56758] 2024-09-21T07:38:10.037 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x1d9) [0x55a96554a76f] 2024-09-21T07:38:10.038 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: ceph-mds(+0x479942) [0x55a96554a942] 2024-09-21T07:38:10.038 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: (MDSContext::complete(int)+0x7c) [0x55a9655986d8] 2024-09-21T07:38:10.038 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: (MDSIOContextBase::complete(int)+0x501) [0x55a965598d57] 2024-09-21T07:38:10.038 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (MDSLogContextBase::complete(int)+0x46) [0x55a965598dd4] 2024-09-21T07:38:10.038 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: (Finisher::finisher_thread_entry()+0x495) [0x7f606dbde34b] 2024-09-21T07:38:10.038 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: ceph-mds(+0x1052e1) [0x55a9651d62e1] 2024-09-21T07:38:10.038 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: (Thread::entry_wrapper()+0x43) [0x7f606dc2fd57] 2024-09-21T07:38:10.038 INFO:tasks.ceph.mds.d.smithi148.stderr: 15: (Thread::_entry_func(void*)+0xd) [0x7f606dc2fd73] 2024-09-21T07:38:10.038 INFO:tasks.ceph.mds.d.smithi148.stderr: 16: /lib64/libc.so.6(+0x89c52) [0x7f606d289c52] 2024-09-21T07:38:10.039 INFO:tasks.ceph.mds.d.smithi148.stderr: 17: /lib64/libc.so.6(+0x10ec80) [0x7f606d30ec80] 2024-09-21T07:38:10.039 INFO:tasks.ceph.mds.d.smithi148.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:38:10.039 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:38:10.041 INFO:tasks.ceph.mds.d.smithi148.stderr: -274> 2024-09-21T07:38:05.806+0000 7f6063a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:38:10.041 INFO:tasks.ceph.mds.d.smithi148.stderr: -272> 2024-09-21T07:38:05.806+0000 7f6063a00640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:38:10.041 INFO:tasks.ceph.mds.d.smithi148.stderr: -4> 2024-09-21T07:38:10.021+0000 7f605ea00640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_commit_logged(ceph::cref_t&)' thread 7f605ea00640 time 2024-09-21T07:38:10.021805+0000 2024-09-21T07:38:10.041 INFO:tasks.ceph.mds.d.smithi148.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: 173: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 6) 2024-09-21T07:38:10.041 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:38:10.042 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:38:10.042 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f606dc5654b] 2024-09-21T07:38:10.042 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f606dc56758] 2024-09-21T07:38:10.042 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x1d9) [0x55a96554a76f] 2024-09-21T07:38:10.042 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: ceph-mds(+0x479942) [0x55a96554a942] 2024-09-21T07:38:10.042 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (MDSContext::complete(int)+0x7c) [0x55a9655986d8] 2024-09-21T07:38:10.042 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: (MDSIOContextBase::complete(int)+0x501) [0x55a965598d57] 2024-09-21T07:38:10.042 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSLogContextBase::complete(int)+0x46) [0x55a965598dd4] 2024-09-21T07:38:10.042 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: (Finisher::finisher_thread_entry()+0x495) [0x7f606dbde34b] 2024-09-21T07:38:10.042 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: ceph-mds(+0x1052e1) [0x55a9651d62e1] 2024-09-21T07:38:10.042 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: (Thread::entry_wrapper()+0x43) [0x7f606dc2fd57] 2024-09-21T07:38:10.042 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (Thread::_entry_func(void*)+0xd) [0x7f606dc2fd73] 2024-09-21T07:38:10.042 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: /lib64/libc.so.6(+0x89c52) [0x7f606d289c52] 2024-09-21T07:38:10.042 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: /lib64/libc.so.6(+0x10ec80) [0x7f606d30ec80] 2024-09-21T07:38:10.043 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:38:10.043 INFO:tasks.ceph.mds.d.smithi148.stderr: 0> 2024-09-21T07:38:10.022+0000 7f605ea00640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:38:10.043 INFO:tasks.ceph.mds.d.smithi148.stderr: in thread 7f605ea00640 thread_name:MR_Finisher 2024-09-21T07:38:10.043 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:38:10.043 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:38:10.043 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f606d23e6f0] 2024-09-21T07:38:10.043 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f606d28b99c] 2024-09-21T07:38:10.043 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: raise() 2024-09-21T07:38:10.043 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: abort() 2024-09-21T07:38:10.043 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7f606dc56686] 2024-09-21T07:38:10.043 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f606dc56758] 2024-09-21T07:38:10.043 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x1d9) [0x55a96554a76f] 2024-09-21T07:38:10.043 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: ceph-mds(+0x479942) [0x55a96554a942] 2024-09-21T07:38:10.043 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: (MDSContext::complete(int)+0x7c) [0x55a9655986d8] 2024-09-21T07:38:10.043 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: (MDSIOContextBase::complete(int)+0x501) [0x55a965598d57] 2024-09-21T07:38:10.044 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (MDSLogContextBase::complete(int)+0x46) [0x55a965598dd4] 2024-09-21T07:38:10.044 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: (Finisher::finisher_thread_entry()+0x495) [0x7f606dbde34b] 2024-09-21T07:38:10.044 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: ceph-mds(+0x1052e1) [0x55a9651d62e1] 2024-09-21T07:38:10.044 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: (Thread::entry_wrapper()+0x43) [0x7f606dc2fd57] 2024-09-21T07:38:10.044 INFO:tasks.ceph.mds.d.smithi148.stderr: 15: (Thread::_entry_func(void*)+0xd) [0x7f606dc2fd73] 2024-09-21T07:38:10.044 INFO:tasks.ceph.mds.d.smithi148.stderr: 16: /lib64/libc.so.6(+0x89c52) [0x7f606d289c52] 2024-09-21T07:38:10.044 INFO:tasks.ceph.mds.d.smithi148.stderr: 17: /lib64/libc.so.6(+0x10ec80) [0x7f606d30ec80] 2024-09-21T07:38:10.044 INFO:tasks.ceph.mds.d.smithi148.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:38:10.044 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:38:10.778 INFO:tasks.ceph.mds.d.smithi148.stderr:daemon-helper: command crashed with signal 6 2024-09-21T07:38:14.390 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:38:14.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.628+0000 7fd195a00640 1 -- 172.21.15.39:0/1536431357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd174003a50 msgr2=0x7fd174003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:14.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.628+0000 7fd195a00640 1 --2- 172.21.15.39:0/1536431357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd174003a50 0x7fd174003ec0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fd180009b50 tx=0x7fd174005830 comp rx=0 tx=0).stop 2024-09-21T07:38:14.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.628+0000 7fd195a00640 1 -- 172.21.15.39:0/1536431357 shutdown_connections 2024-09-21T07:38:14.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.628+0000 7fd195a00640 1 --2- 172.21.15.39:0/1536431357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd174003a50 0x7fd174003ec0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:14.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.628+0000 7fd195a00640 1 -- 172.21.15.39:0/1536431357 >> 172.21.15.39:0/1536431357 conn(0x7fd1900b1cb0 msgr2=0x7fd1900b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:14.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.628+0000 7fd195a00640 1 -- 172.21.15.39:0/1536431357 shutdown_connections 2024-09-21T07:38:14.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.628+0000 7fd195a00640 1 -- 172.21.15.39:0/1536431357 wait complete. 2024-09-21T07:38:14.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.630+0000 7fd195a00640 1 Processor -- start 2024-09-21T07:38:14.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.631+0000 7fd195a00640 1 -- start start 2024-09-21T07:38:14.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.632+0000 7fd195a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd174003a50 0x7fd19009c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:14.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.632+0000 7fd195a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1901f8e00 0x7fd19009abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:14.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.632+0000 7fd195a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd190147ca0 0x7fd1900cf750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:14.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.632+0000 7fd195a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd190074a20 con 0x7fd1901f8e00 2024-09-21T07:38:14.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.632+0000 7fd195a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd1900748a0 con 0x7fd174003a50 2024-09-21T07:38:14.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.632+0000 7fd195a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd190074ba0 con 0x7fd190147ca0 2024-09-21T07:38:14.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.632+0000 7fd187e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1901f8e00 0x7fd19009abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:14.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.632+0000 7fd187e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1901f8e00 0x7fd19009abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50914/0 (socket says 172.21.15.39:50914) 2024-09-21T07:38:14.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.632+0000 7fd187e00640 1 -- 172.21.15.39:0/3776372573 learned_addr learned my addr 172.21.15.39:0/3776372573 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:14.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.633+0000 7fd18ea00640 1 --2- 172.21.15.39:0/3776372573 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd174003a50 0x7fd19009c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:14.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.633+0000 7fd18f400640 1 --2- 172.21.15.39:0/3776372573 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd190147ca0 0x7fd1900cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:14.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.633+0000 7fd187e00640 1 -- 172.21.15.39:0/3776372573 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd190147ca0 msgr2=0x7fd1900cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:14.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.633+0000 7fd187e00640 1 --2- 172.21.15.39:0/3776372573 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd190147ca0 0x7fd1900cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:14.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.633+0000 7fd187e00640 1 -- 172.21.15.39:0/3776372573 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd174003a50 msgr2=0x7fd19009c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:14.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.633+0000 7fd187e00640 1 --2- 172.21.15.39:0/3776372573 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd174003a50 0x7fd19009c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:14.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.633+0000 7fd187e00640 1 -- 172.21.15.39:0/3776372573 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1901f9570 con 0x7fd1901f8e00 2024-09-21T07:38:14.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.633+0000 7fd187e00640 1 --2- 172.21.15.39:0/3776372573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1901f8e00 0x7fd19009abf0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fd178003e40 tx=0x7fd178001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:14.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.633+0000 7fd186a00640 1 -- 172.21.15.39:0/3776372573 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd178015070 con 0x7fd1901f8e00 2024-09-21T07:38:14.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.633+0000 7fd186a00640 1 -- 172.21.15.39:0/3776372573 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd178011840 con 0x7fd1901f8e00 2024-09-21T07:38:14.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.633+0000 7fd186a00640 1 -- 172.21.15.39:0/3776372573 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd178011c90 con 0x7fd1901f8e00 2024-09-21T07:38:14.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.633+0000 7fd195a00640 1 -- 172.21.15.39:0/3776372573 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1901f9850 con 0x7fd1901f8e00 2024-09-21T07:38:14.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.634+0000 7fd195a00640 1 -- 172.21.15.39:0/3776372573 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1900cf230 con 0x7fd1901f8e00 2024-09-21T07:38:14.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.635+0000 7fd195a00640 1 -- 172.21.15.39:0/3776372573 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1901f92e0 con 0x7fd1901f8e00 2024-09-21T07:38:14.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.636+0000 7fd186a00640 1 -- 172.21.15.39:0/3776372573 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd178018070 con 0x7fd1901f8e00 2024-09-21T07:38:14.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.637+0000 7fd186a00640 1 --2- 172.21.15.39:0/3776372573 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd16c078bc0 0x7fd16c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:14.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.637+0000 7fd186a00640 1 -- 172.21.15.39:0/3776372573 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 7087+0+0 (secure 0 0 0) 0x7fd17809bc90 con 0x7fd1901f8e00 2024-09-21T07:38:14.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.637+0000 7fd18ea00640 1 --2- 172.21.15.39:0/3776372573 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd16c078bc0 0x7fd16c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:14.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.638+0000 7fd18ea00640 1 --2- 172.21.15.39:0/3776372573 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd16c078bc0 0x7fd16c07b0d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd180035000 tx=0x7fd1800046c0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:14.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.641+0000 7fd186a00640 1 -- 172.21.15.39:0/3776372573 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd178063510 con 0x7fd1901f8e00 2024-09-21T07:38:14.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.880+0000 7fd195a00640 1 -- 172.21.15.39:0/3776372573 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1900b25b0 con 0x7fd1901f8e00 2024-09-21T07:38:14.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.881+0000 7fd186a00640 1 -- 172.21.15.39:0/3776372573 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5246 (secure 0 0 0) 0x7fd178067f40 con 0x7fd1901f8e00 2024-09-21T07:38:14.883 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:14.883 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":83,"btime":"2024-09-21T07:38:08:302732+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6825,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/855150775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":855150775},{"type":"v1","addr":"172.21.15.39:6834","nonce":855150775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:38:07.560346+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5870,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5870":{"gid":5870,"name":"d","rank":0,"incarnation":78,"state":"up:active","state_seq":12,"addr":"172.21.15.148:6837/2381504293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2381504293},{"type":"v1","addr":"172.21.15.148:6837","nonce":2381504293}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5870,"qdb_cluster":[6115,5870]},"id":6}]} 2024-09-21T07:38:14.883 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 83 2024-09-21T07:38:14.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.883+0000 7fd195a00640 1 -- 172.21.15.39:0/3776372573 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd16c078bc0 msgr2=0x7fd16c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:14.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.883+0000 7fd195a00640 1 --2- 172.21.15.39:0/3776372573 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd16c078bc0 0x7fd16c07b0d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd180035000 tx=0x7fd1800046c0 comp rx=0 tx=0).stop 2024-09-21T07:38:14.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.883+0000 7fd195a00640 1 -- 172.21.15.39:0/3776372573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1901f8e00 msgr2=0x7fd19009abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:14.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.883+0000 7fd195a00640 1 --2- 172.21.15.39:0/3776372573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1901f8e00 0x7fd19009abf0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fd178003e40 tx=0x7fd178001630 comp rx=0 tx=0).stop 2024-09-21T07:38:14.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.883+0000 7fd195a00640 1 -- 172.21.15.39:0/3776372573 shutdown_connections 2024-09-21T07:38:14.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.883+0000 7fd195a00640 1 --2- 172.21.15.39:0/3776372573 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd16c078bc0 0x7fd16c07b0d0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:14.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.883+0000 7fd195a00640 1 --2- 172.21.15.39:0/3776372573 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd190147ca0 0x7fd1900cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:14.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.883+0000 7fd195a00640 1 --2- 172.21.15.39:0/3776372573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1901f8e00 0x7fd19009abf0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:14.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.883+0000 7fd195a00640 1 --2- 172.21.15.39:0/3776372573 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd174003a50 0x7fd19009c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:14.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.883+0000 7fd195a00640 1 -- 172.21.15.39:0/3776372573 >> 172.21.15.39:0/3776372573 conn(0x7fd1900b1cb0 msgr2=0x7fd1900ae7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:14.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.883+0000 7fd195a00640 1 -- 172.21.15.39:0/3776372573 shutdown_connections 2024-09-21T07:38:14.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:14.883+0000 7fd195a00640 1 -- 172.21.15.39:0/3776372573 wait complete. 2024-09-21T07:38:14.907 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:38:16.275 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-09-21T07:38:19.908 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:38:20.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.156+0000 7f9e86a00640 1 -- 172.21.15.39:0/649581985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e680051a0 msgr2=0x7f9e6800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:20.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.156+0000 7f9e86a00640 1 --2- 172.21.15.39:0/649581985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e680051a0 0x7f9e6800b9c0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f9e78009ed0 tx=0x7f9e68005830 comp rx=0 tx=0).stop 2024-09-21T07:38:20.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.156+0000 7f9e86000640 1 -- 172.21.15.39:0/649581985 reap_dead start 2024-09-21T07:38:20.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.156+0000 7f9e86a00640 1 -- 172.21.15.39:0/649581985 shutdown_connections 2024-09-21T07:38:20.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.156+0000 7f9e86a00640 1 -- 172.21.15.39:0/649581985 >> 172.21.15.39:0/649581985 conn(0x7f9e8006d410 msgr2=0x7f9e800b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:20.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.156+0000 7f9e86a00640 1 -- 172.21.15.39:0/649581985 shutdown_connections 2024-09-21T07:38:20.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.156+0000 7f9e86a00640 1 -- 172.21.15.39:0/649581985 wait complete. 2024-09-21T07:38:20.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.158+0000 7f9e86a00640 1 Processor -- start 2024-09-21T07:38:20.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.158+0000 7f9e86a00640 1 -- start start 2024-09-21T07:38:20.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.160+0000 7f9e86a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9e680051a0 0x7f9e801f2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:20.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.160+0000 7f9e86a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e80158050 0x7f9e80158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:20.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.160+0000 7f9e86a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e80152c50 0x7f9e80153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:20.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.160+0000 7f9e86a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9e801642d0 con 0x7f9e80158050 2024-09-21T07:38:20.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.160+0000 7f9e86a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9e800b4790 con 0x7f9e80152c50 2024-09-21T07:38:20.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.160+0000 7f9e86a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f9e80164450 con 0x7f9e680051a0 2024-09-21T07:38:20.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.160+0000 7f9e84c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e80158050 0x7f9e80158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:20.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.160+0000 7f9e84c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e80158050 0x7f9e80158460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50934/0 (socket says 172.21.15.39:50934) 2024-09-21T07:38:20.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.160+0000 7f9e84c00640 1 -- 172.21.15.39:0/670893058 learned_addr learned my addr 172.21.15.39:0/670893058 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:20.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.160+0000 7f9e86000640 1 --2- 172.21.15.39:0/670893058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e80152c50 0x7f9e80153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:20.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.160+0000 7f9e85600640 1 --2- 172.21.15.39:0/670893058 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9e680051a0 0x7f9e801f2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:20.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.161+0000 7f9e84c00640 1 -- 172.21.15.39:0/670893058 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9e680051a0 msgr2=0x7f9e801f2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:20.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.161+0000 7f9e84c00640 1 --2- 172.21.15.39:0/670893058 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9e680051a0 0x7f9e801f2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:20.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.161+0000 7f9e84c00640 1 -- 172.21.15.39:0/670893058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e80152c50 msgr2=0x7f9e80153060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:20.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.161+0000 7f9e84c00640 1 --2- 172.21.15.39:0/670893058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e80152c50 0x7f9e80153060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:20.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.161+0000 7f9e84c00640 1 -- 172.21.15.39:0/670893058 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e801591d0 con 0x7f9e80158050 2024-09-21T07:38:20.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.161+0000 7f9e84c00640 1 --2- 172.21.15.39:0/670893058 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e80158050 0x7f9e80158460 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c00e570 tx=0x7f9e6c00e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:20.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.161+0000 7f9e7d600640 1 -- 172.21.15.39:0/670893058 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c015070 con 0x7f9e80158050 2024-09-21T07:38:20.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.161+0000 7f9e7d600640 1 -- 172.21.15.39:0/670893058 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e6c010980 con 0x7f9e80158050 2024-09-21T07:38:20.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.161+0000 7f9e7d600640 1 -- 172.21.15.39:0/670893058 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c010d50 con 0x7f9e80158050 2024-09-21T07:38:20.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.161+0000 7f9e86a00640 1 -- 172.21.15.39:0/670893058 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e801f31b0 con 0x7f9e80158050 2024-09-21T07:38:20.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.161+0000 7f9e86a00640 1 -- 172.21.15.39:0/670893058 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e80158c10 con 0x7f9e80158050 2024-09-21T07:38:20.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.162+0000 7f9e86a00640 1 -- 172.21.15.39:0/670893058 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e8015fbd0 con 0x7f9e80158050 2024-09-21T07:38:20.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.164+0000 7f9e7d600640 1 -- 172.21.15.39:0/670893058 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9e6c018070 con 0x7f9e80158050 2024-09-21T07:38:20.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.165+0000 7f9e7d600640 1 --2- 172.21.15.39:0/670893058 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9e60078b90 0x7f9e6007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:20.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.165+0000 7f9e7d600640 1 -- 172.21.15.39:0/670893058 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 7087+0+0 (secure 0 0 0) 0x7f9e6c09b640 con 0x7f9e80158050 2024-09-21T07:38:20.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.165+0000 7f9e85600640 1 --2- 172.21.15.39:0/670893058 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9e60078b90 0x7f9e6007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:20.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.166+0000 7f9e85600640 1 --2- 172.21.15.39:0/670893058 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9e60078b90 0x7f9e6007b0a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9e74000c00 tx=0x7f9e74006b10 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:20.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.169+0000 7f9e7d600640 1 -- 172.21.15.39:0/670893058 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9e6c062ec0 con 0x7f9e80158050 2024-09-21T07:38:20.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.404+0000 7f9e86a00640 1 -- 172.21.15.39:0/670893058 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e801589a0 con 0x7f9e80158050 2024-09-21T07:38:20.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.404+0000 7f9e7d600640 1 -- 172.21.15.39:0/670893058 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5246 (secure 0 0 0) 0x7f9e6c0678f0 con 0x7f9e80158050 2024-09-21T07:38:20.406 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:20.406 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":83,"btime":"2024-09-21T07:38:08:302732+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6825,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/855150775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":855150775},{"type":"v1","addr":"172.21.15.39:6834","nonce":855150775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:38:07.560346+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5870,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5870":{"gid":5870,"name":"d","rank":0,"incarnation":78,"state":"up:active","state_seq":12,"addr":"172.21.15.148:6837/2381504293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2381504293},{"type":"v1","addr":"172.21.15.148:6837","nonce":2381504293}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5870,"qdb_cluster":[6115,5870]},"id":6}]} 2024-09-21T07:38:20.406 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 83 2024-09-21T07:38:20.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.407+0000 7f9e86a00640 1 -- 172.21.15.39:0/670893058 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9e60078b90 msgr2=0x7f9e6007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:20.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.407+0000 7f9e86a00640 1 --2- 172.21.15.39:0/670893058 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9e60078b90 0x7f9e6007b0a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9e74000c00 tx=0x7f9e74006b10 comp rx=0 tx=0).stop 2024-09-21T07:38:20.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.407+0000 7f9e86a00640 1 -- 172.21.15.39:0/670893058 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e80158050 msgr2=0x7f9e80158460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:20.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.407+0000 7f9e86a00640 1 --2- 172.21.15.39:0/670893058 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e80158050 0x7f9e80158460 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c00e570 tx=0x7f9e6c00e970 comp rx=0 tx=0).stop 2024-09-21T07:38:20.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.407+0000 7f9e86a00640 1 -- 172.21.15.39:0/670893058 shutdown_connections 2024-09-21T07:38:20.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.407+0000 7f9e86a00640 1 --2- 172.21.15.39:0/670893058 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9e60078b90 0x7f9e6007b0a0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:20.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.407+0000 7f9e86a00640 1 --2- 172.21.15.39:0/670893058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e80152c50 0x7f9e80153060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:20.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.407+0000 7f9e86a00640 1 --2- 172.21.15.39:0/670893058 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e80158050 0x7f9e80158460 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:20.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.407+0000 7f9e86a00640 1 --2- 172.21.15.39:0/670893058 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9e680051a0 0x7f9e801f2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:20.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.407+0000 7f9e86a00640 1 -- 172.21.15.39:0/670893058 >> 172.21.15.39:0/670893058 conn(0x7f9e8006d410 msgr2=0x7f9e80150da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:20.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.407+0000 7f9e86a00640 1 -- 172.21.15.39:0/670893058 shutdown_connections 2024-09-21T07:38:20.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:20.407+0000 7f9e86a00640 1 -- 172.21.15.39:0/670893058 wait complete. 2024-09-21T07:38:20.431 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:38:22.888 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~7s 2024-09-21T07:38:25.432 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:38:25.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.669+0000 7f7ebfa00640 1 -- 172.21.15.39:0/705853497 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7eb815d8a0 msgr2=0x7f7eb80b40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:25.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.669+0000 7f7ebfa00640 1 --2- 172.21.15.39:0/705853497 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7eb815d8a0 0x7f7eb80b40b0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f7eb80cd710 tx=0x7f7ea402a560 comp rx=0 tx=0).stop 2024-09-21T07:38:25.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.669+0000 7f7ebfa00640 1 -- 172.21.15.39:0/705853497 shutdown_connections 2024-09-21T07:38:25.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.669+0000 7f7ebfa00640 1 --2- 172.21.15.39:0/705853497 >> v2:172.21.15.148:3300/0 conn(0x7f7eb80ccb50 0x7f7eb80ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:25.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.669+0000 7f7ebfa00640 1 --2- 172.21.15.39:0/705853497 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7eb815d8a0 0x7f7eb80b40b0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:25.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.669+0000 7f7ebfa00640 1 --2- 172.21.15.39:0/705853497 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7eb80cc1e0 0x7f7eb80af3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:25.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.669+0000 7f7ebfa00640 1 -- 172.21.15.39:0/705853497 >> 172.21.15.39:0/705853497 conn(0x7f7eb806d7d0 msgr2=0x7f7eb80b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:25.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.669+0000 7f7ebfa00640 1 -- 172.21.15.39:0/705853497 shutdown_connections 2024-09-21T07:38:25.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.669+0000 7f7ebfa00640 1 -- 172.21.15.39:0/705853497 wait complete. 2024-09-21T07:38:25.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.671+0000 7f7ebfa00640 1 Processor -- start 2024-09-21T07:38:25.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.672+0000 7f7ebfa00640 1 -- start start 2024-09-21T07:38:25.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.673+0000 7f7ebfa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7eb80cc1e0 0x7f7eb809c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:25.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.673+0000 7f7ebfa00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7eb80ccb50 0x7f7eb8147f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:25.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.673+0000 7f7ebfa00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7eb815d8a0 0x7f7eb809da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:25.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.673+0000 7f7ebfa00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb80b4770 con 0x7f7eb80cc1e0 2024-09-21T07:38:25.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.673+0000 7f7ebfa00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb80b45f0 con 0x7f7eb815d8a0 2024-09-21T07:38:25.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.673+0000 7f7ebfa00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f7eb81533a0 con 0x7f7eb80ccb50 2024-09-21T07:38:25.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.673+0000 7f7ebd000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7eb80cc1e0 0x7f7eb809c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:25.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.673+0000 7f7ebd000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7eb80cc1e0 0x7f7eb809c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41010/0 (socket says 172.21.15.39:41010) 2024-09-21T07:38:25.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.673+0000 7f7ebd000640 1 -- 172.21.15.39:0/1003138364 learned_addr learned my addr 172.21.15.39:0/1003138364 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:25.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.673+0000 7f7eb7e00640 1 --2- 172.21.15.39:0/1003138364 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7eb80ccb50 0x7f7eb8147f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:25.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.674+0000 7f7ebda00640 1 --2- 172.21.15.39:0/1003138364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7eb815d8a0 0x7f7eb809da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:25.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.674+0000 7f7ebd000640 1 -- 172.21.15.39:0/1003138364 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7eb80ccb50 msgr2=0x7f7eb8147f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:25.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.674+0000 7f7ebd000640 1 --2- 172.21.15.39:0/1003138364 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7eb80ccb50 0x7f7eb8147f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:25.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.674+0000 7f7ebd000640 1 -- 172.21.15.39:0/1003138364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7eb815d8a0 msgr2=0x7f7eb809da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:25.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.674+0000 7f7ebd000640 1 --2- 172.21.15.39:0/1003138364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7eb815d8a0 0x7f7eb809da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:25.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.674+0000 7f7ebd000640 1 -- 172.21.15.39:0/1003138364 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb809c980 con 0x7f7eb80cc1e0 2024-09-21T07:38:25.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.674+0000 7f7ebda00640 1 --2- 172.21.15.39:0/1003138364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7eb815d8a0 0x7f7eb809da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:38:25.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.674+0000 7f7ebd000640 1 --2- 172.21.15.39:0/1003138364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7eb80cc1e0 0x7f7eb809c130 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00d720 tx=0x7f7eac00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:25.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.674+0000 7f7eb4c00640 1 -- 172.21.15.39:0/1003138364 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac017070 con 0x7f7eb80cc1e0 2024-09-21T07:38:25.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.674+0000 7f7eb4c00640 1 -- 172.21.15.39:0/1003138364 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7eac009ab0 con 0x7f7eb80cc1e0 2024-09-21T07:38:25.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.674+0000 7f7eb4c00640 1 -- 172.21.15.39:0/1003138364 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac010460 con 0x7f7eb80cc1e0 2024-09-21T07:38:25.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.674+0000 7f7ebfa00640 1 -- 172.21.15.39:0/1003138364 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eb809cc60 con 0x7f7eb80cc1e0 2024-09-21T07:38:25.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.675+0000 7f7ebfa00640 1 -- 172.21.15.39:0/1003138364 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eb809d220 con 0x7f7eb80cc1e0 2024-09-21T07:38:25.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.675+0000 7f7ebfa00640 1 -- 172.21.15.39:0/1003138364 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eb80ccf60 con 0x7f7eb80cc1e0 2024-09-21T07:38:25.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.677+0000 7f7eb4c00640 1 -- 172.21.15.39:0/1003138364 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7eac01a070 con 0x7f7eb80cc1e0 2024-09-21T07:38:25.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.678+0000 7f7eb4c00640 1 --2- 172.21.15.39:0/1003138364 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7e98078b90 0x7f7e9807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:25.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.678+0000 7f7eb4c00640 1 -- 172.21.15.39:0/1003138364 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 7087+0+0 (secure 0 0 0) 0x7f7eac09b3b0 con 0x7f7eb80cc1e0 2024-09-21T07:38:25.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.678+0000 7f7eb7e00640 1 --2- 172.21.15.39:0/1003138364 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7e98078b90 0x7f7e9807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:25.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.679+0000 7f7eb7e00640 1 --2- 172.21.15.39:0/1003138364 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7e98078b90 0x7f7e9807b0a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4033000 tx=0x7f7ea4004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:25.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.682+0000 7f7eb4c00640 1 -- 172.21.15.39:0/1003138364 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7eac062ba0 con 0x7f7eb80cc1e0 2024-09-21T07:38:25.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.915+0000 7f7ebfa00640 1 -- 172.21.15.39:0/1003138364 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7eb815fed0 con 0x7f7eb80cc1e0 2024-09-21T07:38:25.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.916+0000 7f7eb4c00640 1 -- 172.21.15.39:0/1003138364 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5246 (secure 0 0 0) 0x7f7eac0675d0 con 0x7f7eb80cc1e0 2024-09-21T07:38:25.918 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:25.918 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":83,"btime":"2024-09-21T07:38:08:302732+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6825,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/855150775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":855150775},{"type":"v1","addr":"172.21.15.39:6834","nonce":855150775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:38:07.560346+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5870,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5870":{"gid":5870,"name":"d","rank":0,"incarnation":78,"state":"up:active","state_seq":12,"addr":"172.21.15.148:6837/2381504293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2381504293},{"type":"v1","addr":"172.21.15.148:6837","nonce":2381504293}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5870,"qdb_cluster":[6115,5870]},"id":6}]} 2024-09-21T07:38:25.919 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 83 2024-09-21T07:38:25.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.919+0000 7f7ebfa00640 1 -- 172.21.15.39:0/1003138364 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7e98078b90 msgr2=0x7f7e9807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:25.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.919+0000 7f7ebfa00640 1 --2- 172.21.15.39:0/1003138364 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7e98078b90 0x7f7e9807b0a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4033000 tx=0x7f7ea4004730 comp rx=0 tx=0).stop 2024-09-21T07:38:25.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.919+0000 7f7ebfa00640 1 -- 172.21.15.39:0/1003138364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7eb80cc1e0 msgr2=0x7f7eb809c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:25.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.919+0000 7f7ebfa00640 1 --2- 172.21.15.39:0/1003138364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7eb80cc1e0 0x7f7eb809c130 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00d720 tx=0x7f7eac00dbe0 comp rx=0 tx=0).stop 2024-09-21T07:38:25.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.919+0000 7f7ebfa00640 1 -- 172.21.15.39:0/1003138364 shutdown_connections 2024-09-21T07:38:25.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.919+0000 7f7ebfa00640 1 --2- 172.21.15.39:0/1003138364 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7e98078b90 0x7f7e9807b0a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:25.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.919+0000 7f7ebfa00640 1 --2- 172.21.15.39:0/1003138364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7eb815d8a0 0x7f7eb809da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:25.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.919+0000 7f7ebfa00640 1 --2- 172.21.15.39:0/1003138364 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7eb80ccb50 0x7f7eb8147f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:25.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.919+0000 7f7ebfa00640 1 --2- 172.21.15.39:0/1003138364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7eb80cc1e0 0x7f7eb809c130 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:25.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.919+0000 7f7ebfa00640 1 -- 172.21.15.39:0/1003138364 >> 172.21.15.39:0/1003138364 conn(0x7f7eb806d7d0 msgr2=0x7f7eb80b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:25.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.919+0000 7f7ebfa00640 1 -- 172.21.15.39:0/1003138364 shutdown_connections 2024-09-21T07:38:25.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:25.919+0000 7f7ebfa00640 1 -- 172.21.15.39:0/1003138364 wait complete. 2024-09-21T07:38:25.943 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:38:29.499 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~13s 2024-09-21T07:38:30.808 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:38:30.813 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:38:30.944 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:38:31.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.176+0000 7fa86bc00640 1 -- 172.21.15.39:0/2195913335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8640cad30 msgr2=0x7fa86406ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:31.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.176+0000 7fa86bc00640 1 --2- 172.21.15.39:0/2195913335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8640cad30 0x7fa86406ffa0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fa8640c71a0 tx=0x7fa84c02a560 comp rx=0 tx=0).stop 2024-09-21T07:38:31.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.176+0000 7fa86bc00640 1 -- 172.21.15.39:0/2195913335 shutdown_connections 2024-09-21T07:38:31.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.176+0000 7fa86bc00640 1 --2- 172.21.15.39:0/2195913335 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8640704e0 0x7fa8640cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:31.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.176+0000 7fa86bc00640 1 --2- 172.21.15.39:0/2195913335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8640cad30 0x7fa86406ffa0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:31.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.176+0000 7fa86bc00640 1 --2- 172.21.15.39:0/2195913335 >> v2:172.21.15.148:3300/0 conn(0x7fa8640ca800 0x7fa8640cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:31.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.176+0000 7fa86bc00640 1 -- 172.21.15.39:0/2195913335 >> 172.21.15.39:0/2195913335 conn(0x7fa8640b1cb0 msgr2=0x7fa8640b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:31.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.176+0000 7fa86bc00640 1 -- 172.21.15.39:0/2195913335 shutdown_connections 2024-09-21T07:38:31.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.176+0000 7fa86bc00640 1 -- 172.21.15.39:0/2195913335 wait complete. 2024-09-21T07:38:31.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.178+0000 7fa86bc00640 1 Processor -- start 2024-09-21T07:38:31.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.178+0000 7fa86bc00640 1 -- start start 2024-09-21T07:38:31.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.180+0000 7fa86bc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8640704e0 0x7fa8640c3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:31.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.180+0000 7fa86bc00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8640ca800 0x7fa8640c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:31.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.180+0000 7fa86bc00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8640cad30 0x7fa8641f94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:31.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.180+0000 7fa86bc00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa8640cd140 con 0x7fa8640704e0 2024-09-21T07:38:31.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.180+0000 7fa86bc00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa8640ccfc0 con 0x7fa8640cad30 2024-09-21T07:38:31.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.180+0000 7fa86bc00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa8640cd2c0 con 0x7fa8640ca800 2024-09-21T07:38:31.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.180+0000 7fa869200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8640704e0 0x7fa8640c3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:31.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.180+0000 7fa869200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8640704e0 0x7fa8640c3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41024/0 (socket says 172.21.15.39:41024) 2024-09-21T07:38:31.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.180+0000 7fa869200640 1 -- 172.21.15.39:0/1060501068 learned_addr learned my addr 172.21.15.39:0/1060501068 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:31.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.180+0000 7fa869c00640 1 --2- 172.21.15.39:0/1060501068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8640cad30 0x7fa8641f94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:31.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.180+0000 7fa85be00640 1 --2- 172.21.15.39:0/1060501068 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8640ca800 0x7fa8640c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:31.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.181+0000 7fa869200640 1 -- 172.21.15.39:0/1060501068 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8640ca800 msgr2=0x7fa8640c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:31.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.181+0000 7fa869200640 1 --2- 172.21.15.39:0/1060501068 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8640ca800 0x7fa8640c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:31.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.181+0000 7fa869200640 1 -- 172.21.15.39:0/1060501068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8640cad30 msgr2=0x7fa8641f94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:31.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.181+0000 7fa869200640 1 --2- 172.21.15.39:0/1060501068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8640cad30 0x7fa8641f94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:31.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.181+0000 7fa869200640 1 -- 172.21.15.39:0/1060501068 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8640afa60 con 0x7fa8640704e0 2024-09-21T07:38:31.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.181+0000 7fa869c00640 1 --2- 172.21.15.39:0/1060501068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8640cad30 0x7fa8641f94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:38:31.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.181+0000 7fa85be00640 1 --2- 172.21.15.39:0/1060501068 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8640ca800 0x7fa8640c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:38:31.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.181+0000 7fa869200640 1 --2- 172.21.15.39:0/1060501068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8640704e0 0x7fa8640c3c10 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fa85400d780 tx=0x7fa85400dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:31.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.181+0000 7fa861600640 1 -- 172.21.15.39:0/1060501068 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa854017070 con 0x7fa8640704e0 2024-09-21T07:38:31.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.181+0000 7fa861600640 1 -- 172.21.15.39:0/1060501068 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa85400de90 con 0x7fa8640704e0 2024-09-21T07:38:31.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.181+0000 7fa86bc00640 1 -- 172.21.15.39:0/1060501068 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8640c4690 con 0x7fa8640704e0 2024-09-21T07:38:31.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.181+0000 7fa861600640 1 -- 172.21.15.39:0/1060501068 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa854004020 con 0x7fa8640704e0 2024-09-21T07:38:31.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.182+0000 7fa86bc00640 1 -- 172.21.15.39:0/1060501068 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8640c4c50 con 0x7fa8640704e0 2024-09-21T07:38:31.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.183+0000 7fa86bc00640 1 -- 172.21.15.39:0/1060501068 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8640cba00 con 0x7fa8640704e0 2024-09-21T07:38:31.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.184+0000 7fa861600640 1 -- 172.21.15.39:0/1060501068 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa8540041c0 con 0x7fa8640704e0 2024-09-21T07:38:31.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.185+0000 7fa861600640 1 --2- 172.21.15.39:0/1060501068 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa840078b90 0x7fa84007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:31.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.186+0000 7fa861600640 1 -- 172.21.15.39:0/1060501068 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 7087+0+0 (secure 0 0 0) 0x7fa85409b370 con 0x7fa8640704e0 2024-09-21T07:38:31.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.186+0000 7fa85be00640 1 --2- 172.21.15.39:0/1060501068 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa840078b90 0x7fa84007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:31.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.187+0000 7fa85be00640 1 --2- 172.21.15.39:0/1060501068 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa840078b90 0x7fa84007b0a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa84c033000 tx=0x7fa84c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:31.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.190+0000 7fa861600640 1 -- 172.21.15.39:0/1060501068 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa8540631b0 con 0x7fa8640704e0 2024-09-21T07:38:31.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.432+0000 7fa86bc00640 1 -- 172.21.15.39:0/1060501068 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8640716e0 con 0x7fa8640704e0 2024-09-21T07:38:31.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.433+0000 7fa861600640 1 -- 172.21.15.39:0/1060501068 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+5294 (secure 0 0 0) 0x7fa854062ab0 con 0x7fa8640704e0 2024-09-21T07:38:31.435 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:31.435 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":84,"btime":"2024-09-21T07:38:26:383835+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6825,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/855150775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":855150775},{"type":"v1","addr":"172.21.15.39:6834","nonce":855150775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:38:26.383790+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5870,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5870":{"gid":5870,"name":"d","rank":0,"incarnation":78,"state":"up:active","state_seq":12,"addr":"172.21.15.148:6837/2381504293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2381504293},{"type":"v1","addr":"172.21.15.148:6837","nonce":2381504293}]},"join_fscid":-1,"laggy_since":"2024-09-21T07:38:26.383790+0000","export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5870,"qdb_cluster":[5870,6115]},"id":6}]} 2024-09-21T07:38:31.435 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 84 2024-09-21T07:38:31.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.435+0000 7fa86bc00640 1 -- 172.21.15.39:0/1060501068 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa840078b90 msgr2=0x7fa84007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:31.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.435+0000 7fa86bc00640 1 --2- 172.21.15.39:0/1060501068 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa840078b90 0x7fa84007b0a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa84c033000 tx=0x7fa84c004730 comp rx=0 tx=0).stop 2024-09-21T07:38:31.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.435+0000 7fa86bc00640 1 -- 172.21.15.39:0/1060501068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8640704e0 msgr2=0x7fa8640c3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:31.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.435+0000 7fa86bc00640 1 --2- 172.21.15.39:0/1060501068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8640704e0 0x7fa8640c3c10 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fa85400d780 tx=0x7fa85400dc40 comp rx=0 tx=0).stop 2024-09-21T07:38:31.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.435+0000 7fa86bc00640 1 -- 172.21.15.39:0/1060501068 shutdown_connections 2024-09-21T07:38:31.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.435+0000 7fa86bc00640 1 --2- 172.21.15.39:0/1060501068 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa840078b90 0x7fa84007b0a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:31.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.435+0000 7fa86bc00640 1 --2- 172.21.15.39:0/1060501068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8640cad30 0x7fa8641f94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:31.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.435+0000 7fa86bc00640 1 --2- 172.21.15.39:0/1060501068 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8640ca800 0x7fa8640c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:31.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.435+0000 7fa86bc00640 1 --2- 172.21.15.39:0/1060501068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8640704e0 0x7fa8640c3c10 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:31.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.435+0000 7fa86bc00640 1 -- 172.21.15.39:0/1060501068 >> 172.21.15.39:0/1060501068 conn(0x7fa8640b1cb0 msgr2=0x7fa8640af1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:31.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.436+0000 7fa86bc00640 1 -- 172.21.15.39:0/1060501068 shutdown_connections 2024-09-21T07:38:31.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.436+0000 7fa86bc00640 1 -- 172.21.15.39:0/1060501068 wait complete. 2024-09-21T07:38:31.463 DEBUG:tasks.ceph_test_case:wait_until_true: success in 20s and 0 retries 2024-09-21T07:38:31.464 DEBUG:teuthology.orchestra.run.smithi148:> sudo sysctl -n kernel.core_pattern 2024-09-21T07:38:31.523 INFO:teuthology.orchestra.run.smithi148.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T07:38:31.526 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2024-09-21T07:38:31.527 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2024-09-21T07:38:31.591 INFO:teuthology.orchestra.run.smithi148.stdout:1726904290.43171.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i d', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2024-09-21T07:38:31.595 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1726904290.43171.core'] 2024-09-21T07:38:31.596 INFO:tasks.cephfs.cephfs_test_case:Found core file 1726904290.43171.core, deleting it 2024-09-21T07:38:31.596 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1726904290.43171.core 2024-09-21T07:38:31.687 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 6:0 2024-09-21T07:38:31.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.916+0000 7fd082800640 1 -- 172.21.15.39:0/107193349 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd07c06ee90 msgr2=0x7fd07c0b47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:31.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.916+0000 7fd082800640 1 --2- 172.21.15.39:0/107193349 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd07c06ee90 0x7fd07c0b47f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd06c009860 tx=0x7fd06c02a5a0 comp rx=0 tx=0).stop 2024-09-21T07:38:31.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.916+0000 7fd082800640 1 -- 172.21.15.39:0/107193349 shutdown_connections 2024-09-21T07:38:31.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.916+0000 7fd082800640 1 --2- 172.21.15.39:0/107193349 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd07c06ee90 0x7fd07c0b47f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:31.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.916+0000 7fd082800640 1 -- 172.21.15.39:0/107193349 >> 172.21.15.39:0/107193349 conn(0x7fd07c06d7d0 msgr2=0x7fd07c0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:31.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.916+0000 7fd082800640 1 -- 172.21.15.39:0/107193349 shutdown_connections 2024-09-21T07:38:31.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.917+0000 7fd082800640 1 -- 172.21.15.39:0/107193349 wait complete. 2024-09-21T07:38:31.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.919+0000 7fd082800640 1 Processor -- start 2024-09-21T07:38:31.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.919+0000 7fd082800640 1 -- start start 2024-09-21T07:38:31.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.920+0000 7fd082800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd07c06ee90 0x7fd07c0cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:31.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.920+0000 7fd082800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd07c0b4d30 0x7fd07c0b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:31.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.920+0000 7fd082800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd07c0cd220 0x7fd07c157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:31.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.920+0000 7fd082800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd07c15fb20 con 0x7fd07c06ee90 2024-09-21T07:38:31.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.920+0000 7fd082800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd07c06f530 con 0x7fd07c0b4d30 2024-09-21T07:38:31.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.920+0000 7fd082800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd07c15fca0 con 0x7fd07c0cd220 2024-09-21T07:38:31.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.920+0000 7fd07b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd07c06ee90 0x7fd07c0cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:31.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.921+0000 7fd07b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd07c06ee90 0x7fd07c0cb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41030/0 (socket says 172.21.15.39:41030) 2024-09-21T07:38:31.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.921+0000 7fd07b400640 1 -- 172.21.15.39:0/3214995624 learned_addr learned my addr 172.21.15.39:0/3214995624 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:31.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.921+0000 7fd07be00640 1 --2- 172.21.15.39:0/3214995624 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd07c0cd220 0x7fd07c157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:31.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.921+0000 7fd072a00640 1 --2- 172.21.15.39:0/3214995624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd07c0b4d30 0x7fd07c0b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:31.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.921+0000 7fd07b400640 1 -- 172.21.15.39:0/3214995624 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd07c0cd220 msgr2=0x7fd07c157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:31.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.921+0000 7fd07b400640 1 --2- 172.21.15.39:0/3214995624 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd07c0cd220 0x7fd07c157ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:31.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.921+0000 7fd07b400640 1 -- 172.21.15.39:0/3214995624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd07c0b4d30 msgr2=0x7fd07c0b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:31.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.921+0000 7fd07b400640 1 --2- 172.21.15.39:0/3214995624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd07c0b4d30 0x7fd07c0b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:31.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.921+0000 7fd07b400640 1 -- 172.21.15.39:0/3214995624 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd07c0cbdc0 con 0x7fd07c06ee90 2024-09-21T07:38:31.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.921+0000 7fd072a00640 1 --2- 172.21.15.39:0/3214995624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd07c0b4d30 0x7fd07c0b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:38:31.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.921+0000 7fd07b400640 1 --2- 172.21.15.39:0/3214995624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd07c06ee90 0x7fd07c0cb4c0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fd06c009980 tx=0x7fd06c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:31.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.922+0000 7fd073e00640 1 -- 172.21.15.39:0/3214995624 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c005c90 con 0x7fd07c06ee90 2024-09-21T07:38:31.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.922+0000 7fd073e00640 1 -- 172.21.15.39:0/3214995624 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd06c031e50 con 0x7fd07c06ee90 2024-09-21T07:38:31.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.922+0000 7fd073e00640 1 -- 172.21.15.39:0/3214995624 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c005260 con 0x7fd07c06ee90 2024-09-21T07:38:31.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.922+0000 7fd082800640 1 -- 172.21.15.39:0/3214995624 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd07c0cc0a0 con 0x7fd07c06ee90 2024-09-21T07:38:31.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.922+0000 7fd082800640 1 -- 172.21.15.39:0/3214995624 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd07c1579d0 con 0x7fd07c06ee90 2024-09-21T07:38:31.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.923+0000 7fd082800640 1 -- 172.21.15.39:0/3214995624 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd07c161ef0 con 0x7fd07c06ee90 2024-09-21T07:38:31.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.924+0000 7fd073e00640 1 -- 172.21.15.39:0/3214995624 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd06c005400 con 0x7fd07c06ee90 2024-09-21T07:38:31.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.925+0000 7fd073e00640 1 --2- 172.21.15.39:0/3214995624 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd058078b90 0x7fd05807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:31.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.925+0000 7fd073e00640 1 -- 172.21.15.39:0/3214995624 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 7087+0+0 (secure 0 0 0) 0x7fd06c0bc470 con 0x7fd07c06ee90 2024-09-21T07:38:31.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.925+0000 7fd072a00640 1 --2- 172.21.15.39:0/3214995624 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd058078b90 0x7fd05807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:31.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.926+0000 7fd072a00640 1 --2- 172.21.15.39:0/3214995624 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd058078b90 0x7fd05807b0a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd07c0b1000 tx=0x7fd064009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:31.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:31.930+0000 7fd073e00640 1 -- 172.21.15.39:0/3214995624 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd06c03f030 con 0x7fd07c06ee90 2024-09-21T07:38:32.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.205+0000 7fd082800640 1 -- 172.21.15.39:0/3214995624 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "6:0"} v 0) -- 0x7fd07c0750f0 con 0x7fd07c06ee90 2024-09-21T07:38:32.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.208+0000 7fd073e00640 1 -- 172.21.15.39:0/3214995624 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "6:0"}]=0 failed mds gid 5870 v85) ==== 97+0+0 (secure 0 0 0) 0x7fd06c084230 con 0x7fd07c06ee90 2024-09-21T07:38:32.210 INFO:teuthology.orchestra.run.smithi039.stderr:failed mds gid 5870 2024-09-21T07:38:32.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.211+0000 7fd082800640 1 -- 172.21.15.39:0/3214995624 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd058078b90 msgr2=0x7fd05807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:32.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.211+0000 7fd082800640 1 --2- 172.21.15.39:0/3214995624 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd058078b90 0x7fd05807b0a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd07c0b1000 tx=0x7fd064009040 comp rx=0 tx=0).stop 2024-09-21T07:38:32.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.211+0000 7fd082800640 1 -- 172.21.15.39:0/3214995624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd07c06ee90 msgr2=0x7fd07c0cb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:32.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.211+0000 7fd082800640 1 --2- 172.21.15.39:0/3214995624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd07c06ee90 0x7fd07c0cb4c0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fd06c009980 tx=0x7fd06c033c20 comp rx=0 tx=0).stop 2024-09-21T07:38:32.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.212+0000 7fd082800640 1 -- 172.21.15.39:0/3214995624 shutdown_connections 2024-09-21T07:38:32.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.212+0000 7fd082800640 1 --2- 172.21.15.39:0/3214995624 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd058078b90 0x7fd05807b0a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:32.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.212+0000 7fd082800640 1 --2- 172.21.15.39:0/3214995624 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd07c0cd220 0x7fd07c157ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:32.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.212+0000 7fd082800640 1 --2- 172.21.15.39:0/3214995624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd07c0b4d30 0x7fd07c0b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:32.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.212+0000 7fd082800640 1 --2- 172.21.15.39:0/3214995624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd07c06ee90 0x7fd07c0cb4c0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:32.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.212+0000 7fd082800640 1 -- 172.21.15.39:0/3214995624 >> 172.21.15.39:0/3214995624 conn(0x7fd07c06d7d0 msgr2=0x7fd07c15d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:32.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.213+0000 7fd082800640 1 -- 172.21.15.39:0/3214995624 shutdown_connections 2024-09-21T07:38:32.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.213+0000 7fd082800640 1 -- 172.21.15.39:0/3214995624 wait complete. 2024-09-21T07:38:32.244 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T07:38:32.244 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T07:38:32.245 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T07:38:32.245 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:38:32.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:38:32.245 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/daemon/state.py", line 146, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi148 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d' 2024-09-21T07:38:32.245 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T07:38:32.245 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T07:38:32.249 INFO:tasks.ceph.mds.d:Started 2024-09-21T07:38:32.250 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:38:32.442 INFO:tasks.ceph.mds.d.smithi148.stdout:starting mds.d at 2024-09-21T07:38:32.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.475+0000 7f6b50000640 1 -- 172.21.15.39:0/1932042205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b48070440 msgr2=0x7f6b48074b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:32.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.475+0000 7f6b50000640 1 --2- 172.21.15.39:0/1932042205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b48070440 0x7f6b48074b80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6b40005fd0 tx=0x7f6b4002a8b0 comp rx=0 tx=0).stop 2024-09-21T07:38:32.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.476+0000 7f6b50000640 1 -- 172.21.15.39:0/1932042205 shutdown_connections 2024-09-21T07:38:32.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.476+0000 7f6b50000640 1 --2- 172.21.15.39:0/1932042205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b48070440 0x7f6b48074b80 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:32.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.476+0000 7f6b50000640 1 --2- 172.21.15.39:0/1932042205 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b48070030 0x7f6b48074640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:32.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.476+0000 7f6b50000640 1 -- 172.21.15.39:0/1932042205 >> 172.21.15.39:0/1932042205 conn(0x7f6b480b1cb0 msgr2=0x7f6b480b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:32.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.476+0000 7f6b50000640 1 -- 172.21.15.39:0/1932042205 shutdown_connections 2024-09-21T07:38:32.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.476+0000 7f6b50000640 1 -- 172.21.15.39:0/1932042205 wait complete. 2024-09-21T07:38:32.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.478+0000 7f6b50000640 1 Processor -- start 2024-09-21T07:38:32.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.478+0000 7f6b50000640 1 -- start start 2024-09-21T07:38:32.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.480+0000 7f6b50000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b48070030 0x7f6b481f6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:32.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.480+0000 7f6b50000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b48070440 0x7f6b481536f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:32.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.480+0000 7f6b50000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b48073170 0x7f6b480c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:32.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.480+0000 7f6b50000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6b480b3a90 con 0x7f6b48070030 2024-09-21T07:38:32.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.480+0000 7f6b50000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6b480b3910 con 0x7f6b48070440 2024-09-21T07:38:32.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.480+0000 7f6b50000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f6b480b3c10 con 0x7f6b48073170 2024-09-21T07:38:32.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.480+0000 7f6b4d600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b48070030 0x7f6b481f6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:32.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.480+0000 7f6b4d600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b48070030 0x7f6b481f6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49880/0 (socket says 172.21.15.39:49880) 2024-09-21T07:38:32.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.480+0000 7f6b4d600640 1 -- 172.21.15.39:0/2040808042 learned_addr learned my addr 172.21.15.39:0/2040808042 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:32.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.480+0000 7f6b4cc00640 1 --2- 172.21.15.39:0/2040808042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b48070440 0x7f6b481536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:32.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.480+0000 7f6b4e000640 1 --2- 172.21.15.39:0/2040808042 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b48073170 0x7f6b480c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:32.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.480+0000 7f6b4d600640 1 -- 172.21.15.39:0/2040808042 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b48073170 msgr2=0x7f6b480c8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:32.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.480+0000 7f6b4d600640 1 --2- 172.21.15.39:0/2040808042 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b48073170 0x7f6b480c8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:32.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.480+0000 7f6b4d600640 1 -- 172.21.15.39:0/2040808042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b48070440 msgr2=0x7f6b481536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:32.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.480+0000 7f6b4d600640 1 --2- 172.21.15.39:0/2040808042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b48070440 0x7f6b481536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:32.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.480+0000 7f6b4d600640 1 -- 172.21.15.39:0/2040808042 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b480c8230 con 0x7f6b48070030 2024-09-21T07:38:32.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.481+0000 7f6b4d600640 1 --2- 172.21.15.39:0/2040808042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b48070030 0x7f6b481f6390 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c00c460 tx=0x7f6b3c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:32.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.481+0000 7f6b45600640 1 -- 172.21.15.39:0/2040808042 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c016020 con 0x7f6b48070030 2024-09-21T07:38:32.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.481+0000 7f6b45600640 1 -- 172.21.15.39:0/2040808042 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b3c012940 con 0x7f6b48070030 2024-09-21T07:38:32.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.481+0000 7f6b50000640 1 -- 172.21.15.39:0/2040808042 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b480c8510 con 0x7f6b48070030 2024-09-21T07:38:32.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.481+0000 7f6b45600640 1 -- 172.21.15.39:0/2040808042 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c012d90 con 0x7f6b48070030 2024-09-21T07:38:32.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.481+0000 7f6b50000640 1 -- 172.21.15.39:0/2040808042 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b48153c30 con 0x7f6b48070030 2024-09-21T07:38:32.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.482+0000 7f6b50000640 1 -- 172.21.15.39:0/2040808042 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b480ad530 con 0x7f6b48070030 2024-09-21T07:38:32.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.484+0000 7f6b45600640 1 -- 172.21.15.39:0/2040808042 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6b3c019020 con 0x7f6b48070030 2024-09-21T07:38:32.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.485+0000 7f6b45600640 1 --2- 172.21.15.39:0/2040808042 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6b28078b90 0x7f6b2807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:32.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.485+0000 7f6b45600640 1 -- 172.21.15.39:0/2040808042 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 7173+0+0 (secure 0 0 0) 0x7f6b3c09b680 con 0x7f6b48070030 2024-09-21T07:38:32.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.485+0000 7f6b4cc00640 1 --2- 172.21.15.39:0/2040808042 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6b28078b90 0x7f6b2807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:32.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.486+0000 7f6b4cc00640 1 --2- 172.21.15.39:0/2040808042 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6b28078b90 0x7f6b2807b0a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6b4815ce40 tx=0x7f6b34005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:32.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.488+0000 7f6b45600640 1 -- 172.21.15.39:0/2040808042 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6b480ad530 con 0x7f6b48070030 2024-09-21T07:38:32.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.727+0000 7f6b50000640 1 -- 172.21.15.39:0/2040808042 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b480b4130 con 0x7f6b48070030 2024-09-21T07:38:32.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.728+0000 7f6b45600640 1 -- 172.21.15.39:0/2040808042 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+4412 (secure 0 0 0) 0x7f6b3c020020 con 0x7f6b48070030 2024-09-21T07:38:32.730 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:32.730 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":86,"btime":"2024-09-21T07:38:32:213476+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:38:32.213457+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6825,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6825":{"gid":6825,"name":"c","rank":0,"incarnation":86,"state":"up:replay","state_seq":1,"addr":"172.21.15.39:6834/855150775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":855150775},{"type":"v1","addr":"172.21.15.39:6834","nonce":855150775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6115,"qdb_cluster":[6115]},"id":6}]} 2024-09-21T07:38:32.730 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 86 2024-09-21T07:38:32.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.730+0000 7f6b50000640 1 -- 172.21.15.39:0/2040808042 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6b28078b90 msgr2=0x7f6b2807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:32.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.730+0000 7f6b50000640 1 --2- 172.21.15.39:0/2040808042 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6b28078b90 0x7f6b2807b0a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6b4815ce40 tx=0x7f6b34005f90 comp rx=0 tx=0).stop 2024-09-21T07:38:32.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.730+0000 7f6b50000640 1 -- 172.21.15.39:0/2040808042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b48070030 msgr2=0x7f6b481f6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:32.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.730+0000 7f6b50000640 1 --2- 172.21.15.39:0/2040808042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b48070030 0x7f6b481f6390 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c00c460 tx=0x7f6b3c00c920 comp rx=0 tx=0).stop 2024-09-21T07:38:32.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.731+0000 7f6b50000640 1 -- 172.21.15.39:0/2040808042 shutdown_connections 2024-09-21T07:38:32.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.731+0000 7f6b50000640 1 --2- 172.21.15.39:0/2040808042 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6b28078b90 0x7f6b2807b0a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:32.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.731+0000 7f6b50000640 1 --2- 172.21.15.39:0/2040808042 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b48073170 0x7f6b480c8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:32.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.731+0000 7f6b50000640 1 --2- 172.21.15.39:0/2040808042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b48070440 0x7f6b481536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:32.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.731+0000 7f6b50000640 1 --2- 172.21.15.39:0/2040808042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b48070030 0x7f6b481f6390 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:32.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.731+0000 7f6b50000640 1 -- 172.21.15.39:0/2040808042 >> 172.21.15.39:0/2040808042 conn(0x7f6b480b1cb0 msgr2=0x7f6b4815bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:32.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.731+0000 7f6b50000640 1 -- 172.21.15.39:0/2040808042 shutdown_connections 2024-09-21T07:38:32.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:32.731+0000 7f6b50000640 1 -- 172.21.15.39:0/2040808042 wait complete. 2024-09-21T07:38:32.754 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-09-21T07:38:36.211 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-09-21T07:38:37.755 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:38:37.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:37.993+0000 7f9cae800640 1 -- 172.21.15.39:0/777103488 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9ca80cf6f0 msgr2=0x7f9ca80cc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:37.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:37.993+0000 7f9cae800640 1 --2- 172.21.15.39:0/777103488 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9ca80cf6f0 0x7f9ca80cc4c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9c98005fd0 tx=0x7f9c9802a5d0 comp rx=0 tx=0).stop 2024-09-21T07:38:37.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:37.993+0000 7f9cae800640 1 -- 172.21.15.39:0/777103488 shutdown_connections 2024-09-21T07:38:37.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:37.993+0000 7f9cae800640 1 --2- 172.21.15.39:0/777103488 >> v2:172.21.15.148:3300/0 conn(0x7f9ca80ccb30 0x7f9ca80ccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:37.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:37.993+0000 7f9cae800640 1 --2- 172.21.15.39:0/777103488 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9ca80cf6f0 0x7f9ca80cc4c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:37.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:37.993+0000 7f9cae800640 1 -- 172.21.15.39:0/777103488 >> 172.21.15.39:0/777103488 conn(0x7f9ca80b1cb0 msgr2=0x7f9ca80b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:37.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:37.994+0000 7f9cae800640 1 -- 172.21.15.39:0/777103488 shutdown_connections 2024-09-21T07:38:37.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:37.994+0000 7f9cae800640 1 -- 172.21.15.39:0/777103488 wait complete. 2024-09-21T07:38:37.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:37.996+0000 7f9cae800640 1 Processor -- start 2024-09-21T07:38:37.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:37.996+0000 7f9cae800640 1 -- start start 2024-09-21T07:38:38.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:37.997+0000 7f9cae800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ca80ccb30 0x7f9ca814f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:38.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:37.997+0000 7f9cae800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ca80ce020 0x7f9ca80b4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:38.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:37.998+0000 7f9cae800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9ca80cf6f0 0x7f9ca80b4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:38.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:37.998+0000 7f9cae800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca80cf1c0 con 0x7f9ca80ce020 2024-09-21T07:38:38.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:37.998+0000 7f9cae800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca80af940 con 0x7f9ca80ccb30 2024-09-21T07:38:38.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:37.998+0000 7f9cae800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f9ca80cf340 con 0x7f9ca80cf6f0 2024-09-21T07:38:38.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:37.998+0000 7f9c9ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ca80ce020 0x7f9ca80b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:38.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:37.998+0000 7f9c9ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ca80ce020 0x7f9ca80b4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49892/0 (socket says 172.21.15.39:49892) 2024-09-21T07:38:38.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:37.998+0000 7f9c9ea00640 1 -- 172.21.15.39:0/1923054878 learned_addr learned my addr 172.21.15.39:0/1923054878 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:38.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:37.998+0000 7f9ca7400640 1 --2- 172.21.15.39:0/1923054878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ca80ccb30 0x7f9ca814f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:38.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:37.998+0000 7f9ca7e00640 1 --2- 172.21.15.39:0/1923054878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9ca80cf6f0 0x7f9ca80b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:38.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:37.998+0000 7f9c9ea00640 1 -- 172.21.15.39:0/1923054878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9ca80cf6f0 msgr2=0x7f9ca80b4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:38.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:37.998+0000 7f9c9ea00640 1 --2- 172.21.15.39:0/1923054878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9ca80cf6f0 0x7f9ca80b4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:38.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:37.998+0000 7f9c9ea00640 1 -- 172.21.15.39:0/1923054878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ca80ccb30 msgr2=0x7f9ca814f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:38.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:37.998+0000 7f9c9ea00640 1 --2- 172.21.15.39:0/1923054878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ca80ccb30 0x7f9ca814f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:38.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:37.998+0000 7f9c9ea00640 1 -- 172.21.15.39:0/1923054878 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca809b320 con 0x7f9ca80ce020 2024-09-21T07:38:38.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:37.999+0000 7f9c9ea00640 1 --2- 172.21.15.39:0/1923054878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ca80ce020 0x7f9ca80b4000 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f9c9000e9c0 tx=0x7f9c9000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:38.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:37.999+0000 7f9c9fe00640 1 -- 172.21.15.39:0/1923054878 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9000cd80 con 0x7f9ca80ce020 2024-09-21T07:38:38.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:37.999+0000 7f9c9fe00640 1 -- 172.21.15.39:0/1923054878 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c90004020 con 0x7f9ca80ce020 2024-09-21T07:38:38.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:37.999+0000 7f9c9fe00640 1 -- 172.21.15.39:0/1923054878 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c90004450 con 0x7f9ca80ce020 2024-09-21T07:38:38.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:37.999+0000 7f9cae800640 1 -- 172.21.15.39:0/1923054878 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ca809b600 con 0x7f9ca80ce020 2024-09-21T07:38:38.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:37.999+0000 7f9cae800640 1 -- 172.21.15.39:0/1923054878 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ca815fdc0 con 0x7f9ca80ce020 2024-09-21T07:38:38.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.000+0000 7f9cae800640 1 -- 172.21.15.39:0/1923054878 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ca80ccf40 con 0x7f9ca80ce020 2024-09-21T07:38:38.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.002+0000 7f9c9fe00640 1 -- 172.21.15.39:0/1923054878 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9c900035f0 con 0x7f9ca80ce020 2024-09-21T07:38:38.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.002+0000 7f9c9fe00640 1 --2- 172.21.15.39:0/1923054878 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9c84078b90 0x7f9c8407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:38.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.003+0000 7f9c9fe00640 1 -- 172.21.15.39:0/1923054878 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 7173+0+0 (secure 0 0 0) 0x7f9c9009bc00 con 0x7f9ca80ce020 2024-09-21T07:38:38.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.003+0000 7f9ca7400640 1 --2- 172.21.15.39:0/1923054878 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9c84078b90 0x7f9c8407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:38.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.004+0000 7f9ca7400640 1 --2- 172.21.15.39:0/1923054878 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9c84078b90 0x7f9c8407b0a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9c98033000 tx=0x7f9c98004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:38.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.007+0000 7f9c9fe00640 1 -- 172.21.15.39:0/1923054878 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9c90063390 con 0x7f9ca80ce020 2024-09-21T07:38:38.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.242+0000 7f9cae800640 1 -- 172.21.15.39:0/1923054878 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ca809b7d0 con 0x7f9ca80ce020 2024-09-21T07:38:38.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.243+0000 7f9c9fe00640 1 -- 172.21.15.39:0/1923054878 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5243 (secure 0 0 0) 0x7f9c90067dc0 con 0x7f9ca80ce020 2024-09-21T07:38:38.245 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:38.246 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":90,"btime":"2024-09-21T07:38:36:232466+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/992188984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":992188984},{"type":"v1","addr":"172.21.15.148:6837","nonce":992188984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:38:36.232454+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6825,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6825":{"gid":6825,"name":"c","rank":0,"incarnation":86,"state":"up:active","state_seq":13,"addr":"172.21.15.39:6834/855150775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":855150775},{"type":"v1","addr":"172.21.15.39:6834","nonce":855150775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6825,"qdb_cluster":[6115,6825]},"id":6}]} 2024-09-21T07:38:38.246 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 90 2024-09-21T07:38:38.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.246+0000 7f9cae800640 1 -- 172.21.15.39:0/1923054878 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9c84078b90 msgr2=0x7f9c8407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:38.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.246+0000 7f9cae800640 1 --2- 172.21.15.39:0/1923054878 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9c84078b90 0x7f9c8407b0a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9c98033000 tx=0x7f9c98004730 comp rx=0 tx=0).stop 2024-09-21T07:38:38.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.246+0000 7f9cae800640 1 -- 172.21.15.39:0/1923054878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ca80ce020 msgr2=0x7f9ca80b4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:38.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.246+0000 7f9cae800640 1 --2- 172.21.15.39:0/1923054878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ca80ce020 0x7f9ca80b4000 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f9c9000e9c0 tx=0x7f9c9000ee80 comp rx=0 tx=0).stop 2024-09-21T07:38:38.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.246+0000 7f9cae800640 1 -- 172.21.15.39:0/1923054878 shutdown_connections 2024-09-21T07:38:38.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.246+0000 7f9cae800640 1 --2- 172.21.15.39:0/1923054878 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9c84078b90 0x7f9c8407b0a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:38.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.246+0000 7f9cae800640 1 --2- 172.21.15.39:0/1923054878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9ca80cf6f0 0x7f9ca80b4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:38.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.246+0000 7f9cae800640 1 --2- 172.21.15.39:0/1923054878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ca80ce020 0x7f9ca80b4000 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:38.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.246+0000 7f9cae800640 1 --2- 172.21.15.39:0/1923054878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ca80ccb30 0x7f9ca814f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:38.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.246+0000 7f9cae800640 1 -- 172.21.15.39:0/1923054878 >> 172.21.15.39:0/1923054878 conn(0x7f9ca80b1cb0 msgr2=0x7f9ca815e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:38.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.246+0000 7f9cae800640 1 -- 172.21.15.39:0/1923054878 shutdown_connections 2024-09-21T07:38:38.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.246+0000 7f9cae800640 1 -- 172.21.15.39:0/1923054878 wait complete. 2024-09-21T07:38:38.273 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-21T07:38:38.273 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:38:38.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.499+0000 7f7fd1c00640 1 -- 172.21.15.39:0/3943343437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7fcc0ccb50 msgr2=0x7f7fcc0ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:38.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.499+0000 7f7fd1c00640 1 --2- 172.21.15.39:0/3943343437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7fcc0ccb50 0x7f7fcc0ccf60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4005fd0 tx=0x7f7fc402a8b0 comp rx=0 tx=0).stop 2024-09-21T07:38:38.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.499+0000 7f7fd1c00640 1 -- 172.21.15.39:0/3943343437 shutdown_connections 2024-09-21T07:38:38.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.499+0000 7f7fd1c00640 1 --2- 172.21.15.39:0/3943343437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7fcc0ccb50 0x7f7fcc0ccf60 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:38.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.499+0000 7f7fd1c00640 1 --2- 172.21.15.39:0/3943343437 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7fcc0cc1e0 0x7f7fcc0af3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:38.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.499+0000 7f7fd1c00640 1 -- 172.21.15.39:0/3943343437 >> 172.21.15.39:0/3943343437 conn(0x7f7fcc06d7d0 msgr2=0x7f7fcc0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:38.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.499+0000 7f7fd1c00640 1 -- 172.21.15.39:0/3943343437 shutdown_connections 2024-09-21T07:38:38.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.499+0000 7f7fd1c00640 1 -- 172.21.15.39:0/3943343437 wait complete. 2024-09-21T07:38:38.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.501+0000 7f7fd1c00640 1 Processor -- start 2024-09-21T07:38:38.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.501+0000 7f7fd1c00640 1 -- start start 2024-09-21T07:38:38.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.503+0000 7f7fd1c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fcc0cc1e0 0x7f7fcc1f9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:38.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.503+0000 7f7fd1c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7fcc0ccb50 0x7f7fcc1f5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:38.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.503+0000 7f7fd1c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7fcc15d8a0 0x7f7fcc1e9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:38.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.503+0000 7f7fd1c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc1533a0 con 0x7f7fcc0cc1e0 2024-09-21T07:38:38.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.503+0000 7f7fd1c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc15d400 con 0x7f7fcc15d8a0 2024-09-21T07:38:38.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.503+0000 7f7fd1c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f7fcc153520 con 0x7f7fcc0ccb50 2024-09-21T07:38:38.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.503+0000 7f7fcaa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fcc0cc1e0 0x7f7fcc1f9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:38.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.503+0000 7f7fcaa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fcc0cc1e0 0x7f7fcc1f9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49900/0 (socket says 172.21.15.39:49900) 2024-09-21T07:38:38.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.503+0000 7f7fcaa00640 1 -- 172.21.15.39:0/419214866 learned_addr learned my addr 172.21.15.39:0/419214866 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:38.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.504+0000 7f7fcb400640 1 --2- 172.21.15.39:0/419214866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7fcc15d8a0 0x7f7fcc1e9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:38.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.504+0000 7f7fcaa00640 1 -- 172.21.15.39:0/419214866 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7fcc0ccb50 msgr2=0x7f7fcc1f5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:38.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.504+0000 7f7fc3e00640 1 --2- 172.21.15.39:0/419214866 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7fcc0ccb50 0x7f7fcc1f5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:38.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.504+0000 7f7fcaa00640 1 --2- 172.21.15.39:0/419214866 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7fcc0ccb50 0x7f7fcc1f5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:38.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.504+0000 7f7fcaa00640 1 -- 172.21.15.39:0/419214866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7fcc15d8a0 msgr2=0x7f7fcc1e9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:38.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.504+0000 7f7fcaa00640 1 --2- 172.21.15.39:0/419214866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7fcc15d8a0 0x7f7fcc1e9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:38.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.504+0000 7f7fcaa00640 1 -- 172.21.15.39:0/419214866 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fcc153b90 con 0x7f7fcc0cc1e0 2024-09-21T07:38:38.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.504+0000 7f7fcb400640 1 --2- 172.21.15.39:0/419214866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7fcc15d8a0 0x7f7fcc1e9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:38:38.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.504+0000 7f7fc3e00640 1 --2- 172.21.15.39:0/419214866 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7fcc0ccb50 0x7f7fcc1f5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:38:38.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.504+0000 7f7fcaa00640 1 --2- 172.21.15.39:0/419214866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fcc0cc1e0 0x7f7fcc1f9c20 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc00e7c0 tx=0x7f7fbc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:38.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.504+0000 7f7fc2a00640 1 -- 172.21.15.39:0/419214866 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc00ccc0 con 0x7f7fcc0cc1e0 2024-09-21T07:38:38.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.504+0000 7f7fc2a00640 1 -- 172.21.15.39:0/419214866 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7fbc010430 con 0x7f7fcc0cc1e0 2024-09-21T07:38:38.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.504+0000 7f7fd1c00640 1 -- 172.21.15.39:0/419214866 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fcc156450 con 0x7f7fcc0cc1e0 2024-09-21T07:38:38.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.504+0000 7f7fc2a00640 1 -- 172.21.15.39:0/419214866 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc010860 con 0x7f7fcc0cc1e0 2024-09-21T07:38:38.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.504+0000 7f7fd1c00640 1 -- 172.21.15.39:0/419214866 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fcc158df0 con 0x7f7fcc0cc1e0 2024-09-21T07:38:38.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.505+0000 7f7fd1c00640 1 -- 172.21.15.39:0/419214866 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fcc0af3c0 con 0x7f7fcc0cc1e0 2024-09-21T07:38:38.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.507+0000 7f7fc2a00640 1 -- 172.21.15.39:0/419214866 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7fbc010a00 con 0x7f7fcc0cc1e0 2024-09-21T07:38:38.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.507+0000 7f7fc2a00640 1 --2- 172.21.15.39:0/419214866 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7fa8078bc0 0x7f7fa807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:38.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.508+0000 7f7fc2a00640 1 -- 172.21.15.39:0/419214866 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 7173+0+0 (secure 0 0 0) 0x7f7fbc09bcd0 con 0x7f7fcc0cc1e0 2024-09-21T07:38:38.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.508+0000 7f7fc3e00640 1 --2- 172.21.15.39:0/419214866 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7fa8078bc0 0x7f7fa807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:38.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.509+0000 7f7fc3e00640 1 --2- 172.21.15.39:0/419214866 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7fa8078bc0 0x7f7fa807b0d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7fb40013f0 tx=0x7f7fb4009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:38.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.512+0000 7f7fc2a00640 1 -- 172.21.15.39:0/419214866 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7fbc063b40 con 0x7f7fcc0cc1e0 2024-09-21T07:38:38.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.748+0000 7f7fd1c00640 1 -- 172.21.15.39:0/419214866 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7fcc1f5a50 con 0x7f7fcc0cc1e0 2024-09-21T07:38:38.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.749+0000 7f7fc2a00640 1 -- 172.21.15.39:0/419214866 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5243 (secure 0 0 0) 0x7f7fbc063440 con 0x7f7fcc0cc1e0 2024-09-21T07:38:38.751 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:38.751 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":90,"btime":"2024-09-21T07:38:36:232466+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/992188984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":992188984},{"type":"v1","addr":"172.21.15.148:6837","nonce":992188984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:38:36.232454+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6825,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6825":{"gid":6825,"name":"c","rank":0,"incarnation":86,"state":"up:active","state_seq":13,"addr":"172.21.15.39:6834/855150775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":855150775},{"type":"v1","addr":"172.21.15.39:6834","nonce":855150775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6825,"qdb_cluster":[6115,6825]},"id":6}]} 2024-09-21T07:38:38.752 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 90 2024-09-21T07:38:38.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.751+0000 7f7fd1c00640 1 -- 172.21.15.39:0/419214866 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7fa8078bc0 msgr2=0x7f7fa807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:38.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.751+0000 7f7fd1c00640 1 --2- 172.21.15.39:0/419214866 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7fa8078bc0 0x7f7fa807b0d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7fb40013f0 tx=0x7f7fb4009040 comp rx=0 tx=0).stop 2024-09-21T07:38:38.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.751+0000 7f7fd1c00640 1 -- 172.21.15.39:0/419214866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fcc0cc1e0 msgr2=0x7f7fcc1f9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:38.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.751+0000 7f7fd1c00640 1 --2- 172.21.15.39:0/419214866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fcc0cc1e0 0x7f7fcc1f9c20 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc00e7c0 tx=0x7f7fbc00ec80 comp rx=0 tx=0).stop 2024-09-21T07:38:38.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.751+0000 7f7fd1c00640 1 -- 172.21.15.39:0/419214866 shutdown_connections 2024-09-21T07:38:38.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.751+0000 7f7fd1c00640 1 --2- 172.21.15.39:0/419214866 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7fa8078bc0 0x7f7fa807b0d0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:38.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.751+0000 7f7fd1c00640 1 --2- 172.21.15.39:0/419214866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7fcc15d8a0 0x7f7fcc1e9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:38.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.751+0000 7f7fd1c00640 1 --2- 172.21.15.39:0/419214866 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7fcc0ccb50 0x7f7fcc1f5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:38.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.751+0000 7f7fd1c00640 1 --2- 172.21.15.39:0/419214866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7fcc0cc1e0 0x7f7fcc1f9c20 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:38.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.751+0000 7f7fd1c00640 1 -- 172.21.15.39:0/419214866 >> 172.21.15.39:0/419214866 conn(0x7f7fcc06d7d0 msgr2=0x7f7fcc0cfe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:38.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.751+0000 7f7fd1c00640 1 -- 172.21.15.39:0/419214866 shutdown_connections 2024-09-21T07:38:38.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:38.752+0000 7f7fd1c00640 1 -- 172.21.15.39:0/419214866 wait complete. 2024-09-21T07:38:39.180 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:36:20.189287+0000', 'modified': '2024-09-21T07:38:36.232454+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 96, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 6825, 'mds_1': 6115}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6115': {'gid': 6115, 'name': 'a', 'rank': 1, 'incarnation': 56, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6835/364657398', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 364657398}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 364657398}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6825': {'gid': 6825, 'name': 'c', 'rank': 0, 'incarnation': 86, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.39:6834/855150775', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 855150775}, {'type': 'v1', 'addr': '172.21.15.39:6834', 'nonce': 855150775}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6825, 'qdb_cluster': [6115, 6825]} 2024-09-21T07:38:39.180 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2024-09-21T07:38:39.180 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T07:38:39.247 INFO:tasks.ceph.mds.c.smithi039.stderr:2024-09-21T07:38:39.245+0000 7f6c1be00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:38:39.247 INFO:tasks.ceph.mds.c.smithi039.stderr:2024-09-21T07:38:39.245+0000 7f6c1be00640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:38:39.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.410+0000 7f0e37200640 1 -- 172.21.15.39:0/316634962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e1c005320 msgr2=0x7f0e1c00b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:39.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.410+0000 7f0e37200640 1 --2- 172.21.15.39:0/316634962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e1c005320 0x7f0e1c00b9e0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f0e30001f00 tx=0x7f0e1c005830 comp rx=0 tx=0).stop 2024-09-21T07:38:39.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.410+0000 7f0e37200640 1 -- 172.21.15.39:0/316634962 shutdown_connections 2024-09-21T07:38:39.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.410+0000 7f0e37200640 1 --2- 172.21.15.39:0/316634962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e1c005320 0x7f0e1c00b9e0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.410+0000 7f0e37200640 1 -- 172.21.15.39:0/316634962 >> 172.21.15.39:0/316634962 conn(0x7f0e3809d620 msgr2=0x7f0e380d4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:39.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.410+0000 7f0e37200640 1 -- 172.21.15.39:0/316634962 shutdown_connections 2024-09-21T07:38:39.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.410+0000 7f0e37200640 1 -- 172.21.15.39:0/316634962 wait complete. 2024-09-21T07:38:39.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.412+0000 7f0e37200640 1 Processor -- start 2024-09-21T07:38:39.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.412+0000 7f0e37200640 1 -- start start 2024-09-21T07:38:39.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.414+0000 7f0e37200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e1c005320 0x7f0e381ea840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:39.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.414+0000 7f0e37200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e381dd540 0x7f0e380b3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:39.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.414+0000 7f0e37200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e380b3690 0x7f0e380c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:39.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.414+0000 7f0e37200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0e380b4c10 con 0x7f0e380b3690 2024-09-21T07:38:39.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.414+0000 7f0e37200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0e380d15a0 con 0x7f0e381dd540 2024-09-21T07:38:39.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.414+0000 7f0e37200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f0e380b4d90 con 0x7f0e1c005320 2024-09-21T07:38:39.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.414+0000 7f0e36800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e380b3690 0x7f0e380c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:39.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.414+0000 7f0e36800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e380b3690 0x7f0e380c8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49916/0 (socket says 172.21.15.39:49916) 2024-09-21T07:38:39.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.414+0000 7f0e36800640 1 -- 172.21.15.39:0/4132121221 learned_addr learned my addr 172.21.15.39:0/4132121221 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:39.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.414+0000 7f0e35e00640 1 --2- 172.21.15.39:0/4132121221 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e1c005320 0x7f0e381ea840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:39.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.414+0000 7f0e2fe00640 1 --2- 172.21.15.39:0/4132121221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e381dd540 0x7f0e380b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:39.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.414+0000 7f0e36800640 1 -- 172.21.15.39:0/4132121221 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e1c005320 msgr2=0x7f0e381ea840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:39.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.414+0000 7f0e36800640 1 --2- 172.21.15.39:0/4132121221 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e1c005320 0x7f0e381ea840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.414+0000 7f0e36800640 1 -- 172.21.15.39:0/4132121221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e381dd540 msgr2=0x7f0e380b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:39.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.414+0000 7f0e36800640 1 --2- 172.21.15.39:0/4132121221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e381dd540 0x7f0e380b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.415+0000 7f0e36800640 1 -- 172.21.15.39:0/4132121221 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e381e92c0 con 0x7f0e380b3690 2024-09-21T07:38:39.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.415+0000 7f0e2fe00640 1 --2- 172.21.15.39:0/4132121221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e381dd540 0x7f0e380b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:38:39.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.415+0000 7f0e36800640 1 --2- 172.21.15.39:0/4132121221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e380b3690 0x7f0e380c8c40 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f0e3002ada0 tx=0x7f0e30001c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:39.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.415+0000 7f0e2e000640 1 -- 172.21.15.39:0/4132121221 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e30003b80 con 0x7f0e380b3690 2024-09-21T07:38:39.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.415+0000 7f0e2e000640 1 -- 172.21.15.39:0/4132121221 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e30003e30 con 0x7f0e380b3690 2024-09-21T07:38:39.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.415+0000 7f0e2e000640 1 -- 172.21.15.39:0/4132121221 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e300326b0 con 0x7f0e380b3690 2024-09-21T07:38:39.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.415+0000 7f0e37200640 1 -- 172.21.15.39:0/4132121221 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e380b3aa0 con 0x7f0e380b3690 2024-09-21T07:38:39.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.415+0000 7f0e37200640 1 -- 172.21.15.39:0/4132121221 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e380c7e80 con 0x7f0e380b3690 2024-09-21T07:38:39.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.417+0000 7f0e2e000640 1 -- 172.21.15.39:0/4132121221 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0e30032850 con 0x7f0e380b3690 2024-09-21T07:38:39.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.418+0000 7f0e2e000640 1 --2- 172.21.15.39:0/4132121221 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0e14078b90 0x7f0e1407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:39.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.419+0000 7f0e2e000640 1 -- 172.21.15.39:0/4132121221 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 7173+0+0 (secure 0 0 0) 0x7f0e300bae70 con 0x7f0e380b3690 2024-09-21T07:38:39.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.419+0000 7f0e35e00640 1 --2- 172.21.15.39:0/4132121221 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0e14078b90 0x7f0e1407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:39.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.419+0000 7f0e43c95740 1 Processor -- start 2024-09-21T07:38:39.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.419+0000 7f0e43c95740 1 -- start start 2024-09-21T07:38:39.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.420+0000 7f0e35e00640 1 --2- 172.21.15.39:0/4132121221 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0e14078b90 0x7f0e1407b0a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0e28001eb0 tx=0x7f0e280066c0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:39.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.421+0000 7f0e43c95740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564d2190c270 0x564d21cdcb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:39.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.421+0000 7f0e43c95740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d218e33b0 0x564d21cdd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:39.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.421+0000 7f0e43c95740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564d218eb590 0x564d21cad150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:39.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.421+0000 7f0e43c95740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564d21c36430 con 0x564d218e33b0 2024-09-21T07:38:39.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.421+0000 7f0e43c95740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x564d21c265d0 con 0x564d2190c270 2024-09-21T07:38:39.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.421+0000 7f0e43c95740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x564d21c15a00 con 0x564d218eb590 2024-09-21T07:38:39.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.421+0000 7f0e2fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d218e33b0 0x564d21cdd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:39.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.421+0000 7f0e2fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d218e33b0 0x564d21cdd0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49924/0 (socket says 172.21.15.39:49924) 2024-09-21T07:38:39.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.421+0000 7f0e2fe00640 1 -- 172.21.15.39:0/75976875 learned_addr learned my addr 172.21.15.39:0/75976875 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:39.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.421+0000 7f0e35e00640 1 --2- 172.21.15.39:0/75976875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564d2190c270 0x564d21cdcb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:39.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.421+0000 7f0e36800640 1 --2- 172.21.15.39:0/75976875 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564d218eb590 0x564d21cad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:39.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.422+0000 7f0e2fe00640 1 -- 172.21.15.39:0/75976875 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564d218eb590 msgr2=0x564d21cad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:38:39.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.422+0000 7f0e2fe00640 1 --2- 172.21.15.39:0/75976875 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564d218eb590 0x564d21cad150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.422+0000 7f0e2fe00640 1 -- 172.21.15.39:0/75976875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564d2190c270 msgr2=0x564d21cdcb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:38:39.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.422+0000 7f0e2fe00640 1 --2- 172.21.15.39:0/75976875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564d2190c270 0x564d21cdcb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.422+0000 7f0e2fe00640 1 -- 172.21.15.39:0/75976875 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d21bde520 con 0x564d218e33b0 2024-09-21T07:38:39.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.422+0000 7f0e35e00640 1 --2- 172.21.15.39:0/75976875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564d2190c270 0x564d21cdcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:38:39.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.422+0000 7f0e2fe00640 1 --2- 172.21.15.39:0/75976875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d218e33b0 0x564d21cdd0a0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f0e20009b90 tx=0x7f0e20013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eee06a2e6338441b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:39.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.422+0000 7f0e26600640 1 -- 172.21.15.39:0/75976875 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e20009d40 con 0x564d218e33b0 2024-09-21T07:38:39.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.422+0000 7f0e26600640 1 -- 172.21.15.39:0/75976875 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e20005020 con 0x564d218e33b0 2024-09-21T07:38:39.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.422+0000 7f0e26600640 1 -- 172.21.15.39:0/75976875 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e200053d0 con 0x564d218e33b0 2024-09-21T07:38:39.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.423+0000 7f0e43c95740 1 -- 172.21.15.39:0/75976875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d218e33b0 msgr2=0x564d21cdd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:39.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.423+0000 7f0e43c95740 1 --2- 172.21.15.39:0/75976875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d218e33b0 0x564d21cdd0a0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f0e20009b90 tx=0x7f0e20013830 comp rx=0 tx=0).stop 2024-09-21T07:38:39.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.423+0000 7f0e43c95740 1 -- 172.21.15.39:0/75976875 shutdown_connections 2024-09-21T07:38:39.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.423+0000 7f0e43c95740 1 --2- 172.21.15.39:0/75976875 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564d218eb590 0x564d21cad150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.423+0000 7f0e43c95740 1 --2- 172.21.15.39:0/75976875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d218e33b0 0x564d21cdd0a0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.423+0000 7f0e43c95740 1 --2- 172.21.15.39:0/75976875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564d2190c270 0x564d21cdcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.423+0000 7f0e43c95740 1 -- 172.21.15.39:0/75976875 >> 172.21.15.39:0/75976875 conn(0x564d219003c0 msgr2=0x564d21cb2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:39.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.423+0000 7f0e43c95740 1 -- 172.21.15.39:0/75976875 shutdown_connections 2024-09-21T07:38:39.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.423+0000 7f0e43c95740 1 -- 172.21.15.39:0/75976875 wait complete. 2024-09-21T07:38:39.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.424+0000 7f0e43c95740 1 -- start start 2024-09-21T07:38:39.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.424+0000 7f0e43c95740 1 Processor -- start 2024-09-21T07:38:39.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.426+0000 7f0e43c95740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d218e33b0 0x564d21cd34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:39.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.426+0000 7f0e43c95740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564d218eb590 0x564d21cd6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:39.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.426+0000 7f0e43c95740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564d2190c270 0x564d21cd72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:39.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.426+0000 7f0e43c95740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564d21bbc330 con 0x564d218e33b0 2024-09-21T07:38:39.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.426+0000 7f0e43c95740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x564d21c19810 con 0x564d218eb590 2024-09-21T07:38:39.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.426+0000 7f0e43c95740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x564d21986620 con 0x564d2190c270 2024-09-21T07:38:39.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.426+0000 7f0e43c95740 10 client.0 _ll_register_callbacks cb 0x564d21bc1420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:38:39.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.426+0000 7f0e35e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d218e33b0 0x564d21cd34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:39.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.426+0000 7f0e35e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d218e33b0 0x564d21cd34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49926/0 (socket says 172.21.15.39:49926) 2024-09-21T07:38:39.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.426+0000 7f0e35e00640 1 -- 172.21.15.39:0/1886578077 learned_addr learned my addr 172.21.15.39:0/1886578077 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:39.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.426+0000 7f0e36800640 1 --2- 172.21.15.39:0/1886578077 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564d2190c270 0x564d21cd72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:39.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.426+0000 7f0e2fe00640 1 --2- 172.21.15.39:0/1886578077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564d218eb590 0x564d21cd6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:39.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.426+0000 7f0e35e00640 1 -- 172.21.15.39:0/1886578077 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564d2190c270 msgr2=0x564d21cd72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:38:39.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.426+0000 7f0e35e00640 1 --2- 172.21.15.39:0/1886578077 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564d2190c270 0x564d21cd72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.426+0000 7f0e35e00640 1 -- 172.21.15.39:0/1886578077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564d218eb590 msgr2=0x564d21cd6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:38:39.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.426+0000 7f0e35e00640 1 --2- 172.21.15.39:0/1886578077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564d218eb590 0x564d21cd6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.426+0000 7f0e35e00640 1 -- 172.21.15.39:0/1886578077 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d21977850 con 0x564d218e33b0 2024-09-21T07:38:39.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.427+0000 7f0e35e00640 1 --2- 172.21.15.39:0/1886578077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d218e33b0 0x564d21cd34e0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f0e28003b70 tx=0x7f0e380d52a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2062eee77cd9e84a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:39.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.427+0000 7f0e1aa00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:38:39.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.427+0000 7f0e1aa00640 1 -- 172.21.15.39:0/1886578077 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e28012bc0 con 0x564d218e33b0 2024-09-21T07:38:39.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.427+0000 7f0e1aa00640 1 -- 172.21.15.39:0/1886578077 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 7173+0+0 (secure 0 0 0) 0x7f0e28012d60 con 0x564d218e33b0 2024-09-21T07:38:39.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.427+0000 7f0e43c95740 1 -- 172.21.15.39:0/1886578077 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e20005020 con 0x564d218e33b0 2024-09-21T07:38:39.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.427+0000 7f0e1aa00640 1 -- 172.21.15.39:0/1886578077 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f0e2801e410 con 0x564d218e33b0 2024-09-21T07:38:39.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.427+0000 7f0e43c95740 10 client.6978 fetch_fsmap learned FSMap version 90 2024-09-21T07:38:39.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.427+0000 7f0e43c95740 1 -- 172.21.15.39:0/1886578077 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x564d219ee6b0 con 0x564d218e33b0 2024-09-21T07:38:39.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.428+0000 7f0e1aa00640 1 -- 172.21.15.39:0/1886578077 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e20005020 con 0x564d218e33b0 2024-09-21T07:38:39.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.428+0000 7f0e1aa00640 1 -- 172.21.15.39:0/1886578077 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 90) ==== 3522+0+0 (secure 0 0 0) 0x7f0e28027060 con 0x564d218e33b0 2024-09-21T07:38:39.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.428+0000 7f0e43c95740 10 client.6978 fetch_fsmap finished waiting for FSMap version 90 2024-09-21T07:38:39.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.428+0000 7f0e43c95740 10 client.6978 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:38:39.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.428+0000 7f0e43c95740 20 client.6978 populate_metadata read hostname 'smithi039' 2024-09-21T07:38:39.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.428+0000 7f0e43c95740 1 --2- 172.21.15.39:0/1886578077 >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x564d218cae40 0x564d21d07000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:39.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.428+0000 7f0e43c95740 4 client.6978 mds_command: new command op to 6115 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:38:39.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.428+0000 7f0e43c95740 1 -- 172.21.15.39:0/1886578077 --> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d21c28520 con 0x564d218cae40 2024-09-21T07:38:39.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.428+0000 7f0e2fe00640 1 --2- 172.21.15.39:0/1886578077 >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x564d218cae40 0x564d21d07000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:39.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.429+0000 7f0e1aa00640 10 client.6978 ms_handle_connect on v2:172.21.15.39:6833/364657398 2024-09-21T07:38:39.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.429+0000 7f0e2fe00640 1 --2- 172.21.15.39:0/1886578077 >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x564d218cae40 0x564d21d07000 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a3400d21773ea4d4 server_cookie=39b90d2be13c113a in_seq=0 out_seq=0 2024-09-21T07:38:39.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.429+0000 7f0e1aa00640 1 -- 172.21.15.39:0/1886578077 <== mds.1 v2:172.21.15.39:6833/364657398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564d21c28520 con 0x564d218cae40 2024-09-21T07:38:39.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.429+0000 7f0e1aa00640 10 client.6978 handle_command_reply: tid=0 2024-09-21T07:38:39.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.430+0000 7f0e43c95740 1 client.6978 shutdown 2024-09-21T07:38:39.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.430+0000 7f0e43c95740 1 -- 172.21.15.39:0/1886578077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d218e33b0 msgr2=0x564d21cd34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:39.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.430+0000 7f0e43c95740 1 --2- 172.21.15.39:0/1886578077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d218e33b0 0x564d21cd34e0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f0e28003b70 tx=0x7f0e380d52a0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.430+0000 7f0e43c95740 1 -- 172.21.15.39:0/1886578077 shutdown_connections 2024-09-21T07:38:39.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.430+0000 7f0e43c95740 1 --2- 172.21.15.39:0/1886578077 >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x564d218cae40 0x564d21d07000 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.430+0000 7f0e43c95740 1 --2- 172.21.15.39:0/1886578077 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564d2190c270 0x564d21cd72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.430+0000 7f0e43c95740 1 --2- 172.21.15.39:0/1886578077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564d218eb590 0x564d21cd6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.430+0000 7f0e1aa00640 0 client.6978 ms_handle_reset on v2:172.21.15.39:6833/364657398 2024-09-21T07:38:39.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.430+0000 7f0e43c95740 1 --2- 172.21.15.39:0/1886578077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d218e33b0 0x564d21cd34e0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.430+0000 7f0e43c95740 1 -- 172.21.15.39:0/1886578077 >> 172.21.15.39:0/1886578077 conn(0x564d219003c0 msgr2=0x564d21cdcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:39.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.430+0000 7f0e43c95740 1 -- 172.21.15.39:0/1886578077 shutdown_connections 2024-09-21T07:38:39.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.430+0000 7f0e43c95740 1 -- 172.21.15.39:0/1886578077 wait complete. 2024-09-21T07:38:39.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.430+0000 7f0e43c95740 20 client.6978 trim_cache size 0 max 16384 2024-09-21T07:38:39.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.450+0000 7f0e43c95740 1 Processor -- start 2024-09-21T07:38:39.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.450+0000 7f0e43c95740 1 -- start start 2024-09-21T07:38:39.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.451+0000 7f0e43c95740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564d218e33b0 0x564d21cf3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:39.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.451+0000 7f0e43c95740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d218eb590 0x564d21cfb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:39.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.451+0000 7f0e43c95740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564d2190c270 0x564d21cf9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:39.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.451+0000 7f0e43c95740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0e38205040 con 0x564d218eb590 2024-09-21T07:38:39.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.451+0000 7f0e43c95740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0e382033e0 con 0x564d2190c270 2024-09-21T07:38:39.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.451+0000 7f0e43c95740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f0e38203a30 con 0x564d218e33b0 2024-09-21T07:38:39.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.451+0000 7f0e2fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d218eb590 0x564d21cfb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:39.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.451+0000 7f0e2fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d218eb590 0x564d21cfb0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49942/0 (socket says 172.21.15.39:49942) 2024-09-21T07:38:39.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.451+0000 7f0e2fe00640 1 -- 172.21.15.39:0/2971239861 learned_addr learned my addr 172.21.15.39:0/2971239861 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:39.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.452+0000 7f0e36800640 1 --2- 172.21.15.39:0/2971239861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564d2190c270 0x564d21cf9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:39.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.452+0000 7f0e35e00640 1 --2- 172.21.15.39:0/2971239861 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564d218e33b0 0x564d21cf3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:39.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.452+0000 7f0e2fe00640 1 -- 172.21.15.39:0/2971239861 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564d218e33b0 msgr2=0x564d21cf3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:38:39.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.452+0000 7f0e2fe00640 1 --2- 172.21.15.39:0/2971239861 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564d218e33b0 0x564d21cf3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.452+0000 7f0e2fe00640 1 -- 172.21.15.39:0/2971239861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564d2190c270 msgr2=0x564d21cf9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:38:39.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.452+0000 7f0e2fe00640 1 --2- 172.21.15.39:0/2971239861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564d2190c270 0x564d21cf9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.452+0000 7f0e2fe00640 1 -- 172.21.15.39:0/2971239861 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d21cd7490 con 0x564d218eb590 2024-09-21T07:38:39.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.452+0000 7f0e36800640 1 --2- 172.21.15.39:0/2971239861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564d2190c270 0x564d21cf9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:38:39.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.452+0000 7f0e35e00640 1 --2- 172.21.15.39:0/2971239861 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564d218e33b0 0x564d21cf3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:38:39.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.452+0000 7f0e2fe00640 1 --2- 172.21.15.39:0/2971239861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d218eb590 0x564d21cfb0e0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f0e20019000 tx=0x7f0e200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21dbf534580f230f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:39.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.452+0000 7f0e25c00640 1 -- 172.21.15.39:0/2971239861 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e20004030 con 0x564d218eb590 2024-09-21T07:38:39.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.452+0000 7f0e25c00640 1 -- 172.21.15.39:0/2971239861 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e200041d0 con 0x564d218eb590 2024-09-21T07:38:39.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.452+0000 7f0e25c00640 1 -- 172.21.15.39:0/2971239861 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e200035f0 con 0x564d218eb590 2024-09-21T07:38:39.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.453+0000 7f0e43c95740 1 -- 172.21.15.39:0/2971239861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d218eb590 msgr2=0x564d21cfb0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:39.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.453+0000 7f0e43c95740 1 --2- 172.21.15.39:0/2971239861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d218eb590 0x564d21cfb0e0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f0e20019000 tx=0x7f0e200046c0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.453+0000 7f0e43c95740 1 -- 172.21.15.39:0/2971239861 shutdown_connections 2024-09-21T07:38:39.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.453+0000 7f0e43c95740 1 --2- 172.21.15.39:0/2971239861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564d2190c270 0x564d21cf9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.453+0000 7f0e43c95740 1 --2- 172.21.15.39:0/2971239861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d218eb590 0x564d21cfb0e0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.453+0000 7f0e43c95740 1 --2- 172.21.15.39:0/2971239861 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564d218e33b0 0x564d21cf3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.453+0000 7f0e43c95740 1 -- 172.21.15.39:0/2971239861 >> 172.21.15.39:0/2971239861 conn(0x564d219003c0 msgr2=0x564d21cf33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:39.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.453+0000 7f0e43c95740 1 -- 172.21.15.39:0/2971239861 shutdown_connections 2024-09-21T07:38:39.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.453+0000 7f0e43c95740 1 -- 172.21.15.39:0/2971239861 wait complete. 2024-09-21T07:38:39.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.454+0000 7f0e43c95740 1 -- start start 2024-09-21T07:38:39.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.454+0000 7f0e43c95740 1 Processor -- start 2024-09-21T07:38:39.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.456+0000 7f0e43c95740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564d218e33b0 0x564d21d5d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:39.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.456+0000 7f0e43c95740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564d218eb590 0x564d21cf30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:39.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.456+0000 7f0e43c95740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d2190c270 0x564d21cf3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:39.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.456+0000 7f0e43c95740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564d21d0a940 con 0x564d2190c270 2024-09-21T07:38:39.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.456+0000 7f0e43c95740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x564d21cd3a20 con 0x564d218e33b0 2024-09-21T07:38:39.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.456+0000 7f0e43c95740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x564d21d0a350 con 0x564d218eb590 2024-09-21T07:38:39.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.456+0000 7f0e43c95740 10 client.0 _ll_register_callbacks cb 0x7f0e28001910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:38:39.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.456+0000 7f0e36800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d2190c270 0x564d21cf3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:39.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.456+0000 7f0e36800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d2190c270 0x564d21cf3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49944/0 (socket says 172.21.15.39:49944) 2024-09-21T07:38:39.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.456+0000 7f0e36800640 1 -- 172.21.15.39:0/3006360260 learned_addr learned my addr 172.21.15.39:0/3006360260 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:39.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.456+0000 7f0e35e00640 1 --2- 172.21.15.39:0/3006360260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564d218e33b0 0x564d21d5d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:39.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.456+0000 7f0e2fe00640 1 --2- 172.21.15.39:0/3006360260 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564d218eb590 0x564d21cf30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:39.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.456+0000 7f0e36800640 1 -- 172.21.15.39:0/3006360260 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564d218eb590 msgr2=0x564d21cf30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:38:39.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.456+0000 7f0e36800640 1 --2- 172.21.15.39:0/3006360260 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564d218eb590 0x564d21cf30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.456+0000 7f0e36800640 1 -- 172.21.15.39:0/3006360260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564d218e33b0 msgr2=0x564d21d5d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:38:39.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.456+0000 7f0e36800640 1 --2- 172.21.15.39:0/3006360260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564d218e33b0 0x564d21d5d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.456+0000 7f0e36800640 1 -- 172.21.15.39:0/3006360260 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d21d5d7f0 con 0x564d2190c270 2024-09-21T07:38:39.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.457+0000 7f0e36800640 1 --2- 172.21.15.39:0/3006360260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d2190c270 0x564d21cf3610 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f0e300b8cb0 tx=0x7f0e30081f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0eb4360877e9e0f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:39.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.457+0000 7f0e1aa00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:38:39.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.457+0000 7f0e1aa00640 1 -- 172.21.15.39:0/3006360260 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e30044040 con 0x564d2190c270 2024-09-21T07:38:39.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.457+0000 7f0e1aa00640 1 -- 172.21.15.39:0/3006360260 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 7173+0+0 (secure 0 0 0) 0x7f0e30032b40 con 0x564d2190c270 2024-09-21T07:38:39.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.457+0000 7f0e43c95740 1 -- 172.21.15.39:0/3006360260 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e200041d0 con 0x564d2190c270 2024-09-21T07:38:39.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.457+0000 7f0e1aa00640 1 -- 172.21.15.39:0/3006360260 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f0e300a3050 con 0x564d2190c270 2024-09-21T07:38:39.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.457+0000 7f0e43c95740 10 client.6984 fetch_fsmap learned FSMap version 90 2024-09-21T07:38:39.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.457+0000 7f0e43c95740 1 -- 172.21.15.39:0/3006360260 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x564d21cf28d0 con 0x564d2190c270 2024-09-21T07:38:39.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.457+0000 7f0e1aa00640 1 -- 172.21.15.39:0/3006360260 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e200041d0 con 0x564d2190c270 2024-09-21T07:38:39.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.457+0000 7f0e1aa00640 1 -- 172.21.15.39:0/3006360260 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 90) ==== 3522+0+0 (secure 0 0 0) 0x7f0e30047060 con 0x564d2190c270 2024-09-21T07:38:39.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.458+0000 7f0e43c95740 10 client.6984 fetch_fsmap finished waiting for FSMap version 90 2024-09-21T07:38:39.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.458+0000 7f0e43c95740 10 client.6984 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:38:39.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.458+0000 7f0e43c95740 20 client.6984 populate_metadata read hostname 'smithi039' 2024-09-21T07:38:39.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.458+0000 7f0e43c95740 1 --2- 172.21.15.39:0/3006360260 >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x564d21d7b340 0x564d21d775a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:39.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.458+0000 7f0e43c95740 4 client.6984 mds_command: new command op to 6115 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:38:39.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.458+0000 7f0e43c95740 1 -- 172.21.15.39:0/3006360260 --> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] -- command(tid 0: {"prefix": "status"}) -- 0x564d21cb21a0 con 0x564d21d7b340 2024-09-21T07:38:39.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.458+0000 7f0e35e00640 1 --2- 172.21.15.39:0/3006360260 >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x564d21d7b340 0x564d21d775a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:39.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.458+0000 7f0e35e00640 1 --2- 172.21.15.39:0/3006360260 >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x564d21d7b340 0x564d21d775a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4d43957e6b0a47eb server_cookie=b97f05849e8d72e9 in_seq=0 out_seq=0 2024-09-21T07:38:39.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.458+0000 7f0e1aa00640 10 client.6984 ms_handle_connect on v2:172.21.15.39:6833/364657398 2024-09-21T07:38:39.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.459+0000 7f0e1aa00640 1 -- 172.21.15.39:0/3006360260 <== mds.1 v2:172.21.15.39:6833/364657398 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x564d21cb21a0 con 0x564d21d7b340 2024-09-21T07:38:39.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.459+0000 7f0e1aa00640 10 client.6984 handle_command_reply: tid=0 2024-09-21T07:38:39.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.459+0000 7f0e43c95740 1 client.6984 shutdown 2024-09-21T07:38:39.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.459+0000 7f0e43c95740 1 -- 172.21.15.39:0/3006360260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d2190c270 msgr2=0x564d21cf3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:39.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.459+0000 7f0e43c95740 1 --2- 172.21.15.39:0/3006360260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d2190c270 0x564d21cf3610 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f0e300b8cb0 tx=0x7f0e30081f70 comp rx=0 tx=0).stop 2024-09-21T07:38:39.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.459+0000 7f0e43c95740 1 -- 172.21.15.39:0/3006360260 shutdown_connections 2024-09-21T07:38:39.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.459+0000 7f0e43c95740 1 --2- 172.21.15.39:0/3006360260 >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x564d21d7b340 0x564d21d775a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.459+0000 7f0e43c95740 1 --2- 172.21.15.39:0/3006360260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564d2190c270 0x564d21cf3610 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.459+0000 7f0e43c95740 1 --2- 172.21.15.39:0/3006360260 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x564d218eb590 0x564d21cf30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.459+0000 7f0e43c95740 1 --2- 172.21.15.39:0/3006360260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564d218e33b0 0x564d21d5d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.459+0000 7f0e1aa00640 0 client.6984 ms_handle_reset on v2:172.21.15.39:6833/364657398 2024-09-21T07:38:39.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.459+0000 7f0e43c95740 1 -- 172.21.15.39:0/3006360260 >> 172.21.15.39:0/3006360260 conn(0x564d219003c0 msgr2=0x564d21cfb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:39.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.459+0000 7f0e43c95740 1 -- 172.21.15.39:0/3006360260 shutdown_connections 2024-09-21T07:38:39.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.459+0000 7f0e43c95740 1 -- 172.21.15.39:0/3006360260 wait complete. 2024-09-21T07:38:39.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.459+0000 7f0e43c95740 20 client.6984 trim_cache size 0 max 16384 2024-09-21T07:38:39.465 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:38:39.465 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:38:39.465 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 1, 2024-09-21T07:38:39.465 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 6115, 2024-09-21T07:38:39.465 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:38:39.465 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:38:39.465 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:38:39.465 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 128.09586600899999, 2024-09-21T07:38:39.465 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 90, 2024-09-21T07:38:39.465 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 96, 2024-09-21T07:38:39.465 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 96, 2024-09-21T07:38:39.466 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 143.77054884899999 2024-09-21T07:38:39.466 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:38:39.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.460+0000 7f0e37200640 1 -- 172.21.15.39:0/4132121221 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0e14078b90 msgr2=0x7f0e1407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:39.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.460+0000 7f0e37200640 1 --2- 172.21.15.39:0/4132121221 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0e14078b90 0x7f0e1407b0a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0e28001eb0 tx=0x7f0e280066c0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.460+0000 7f0e37200640 1 -- 172.21.15.39:0/4132121221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e380b3690 msgr2=0x7f0e380c8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:39.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.460+0000 7f0e37200640 1 --2- 172.21.15.39:0/4132121221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e380b3690 0x7f0e380c8c40 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f0e3002ada0 tx=0x7f0e30001c70 comp rx=0 tx=0).stop 2024-09-21T07:38:39.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.460+0000 7f0e37200640 1 -- 172.21.15.39:0/4132121221 shutdown_connections 2024-09-21T07:38:39.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.460+0000 7f0e37200640 1 --2- 172.21.15.39:0/4132121221 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0e14078b90 0x7f0e1407b0a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.460+0000 7f0e37200640 1 --2- 172.21.15.39:0/4132121221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e380b3690 0x7f0e380c8c40 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.460+0000 7f0e37200640 1 --2- 172.21.15.39:0/4132121221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e381dd540 0x7f0e380b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.460+0000 7f0e37200640 1 --2- 172.21.15.39:0/4132121221 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e1c005320 0x7f0e381ea840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.460+0000 7f0e37200640 1 -- 172.21.15.39:0/4132121221 >> 172.21.15.39:0/4132121221 conn(0x7f0e3809d620 msgr2=0x7f0e380b83f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:39.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.460+0000 7f0e37200640 1 -- 172.21.15.39:0/4132121221 shutdown_connections 2024-09-21T07:38:39.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.460+0000 7f0e37200640 1 -- 172.21.15.39:0/4132121221 wait complete. 2024-09-21T07:38:39.488 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T07:38:39.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.739+0000 7fbd39a00640 1 -- 172.21.15.39:0/3172938554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd3415ca50 msgr2=0x7fbd340cc830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:39.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.739+0000 7fbd39a00640 1 --2- 172.21.15.39:0/3172938554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd3415ca50 0x7fbd340cc830 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fbd24005fd0 tx=0x7fbd2402a540 comp rx=0 tx=0).stop 2024-09-21T07:38:39.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.739+0000 7fbd39a00640 1 -- 172.21.15.39:0/3172938554 shutdown_connections 2024-09-21T07:38:39.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.740+0000 7fbd39a00640 1 --2- 172.21.15.39:0/3172938554 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbd340cf540 0x7fbd3406ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.740+0000 7fbd39a00640 1 --2- 172.21.15.39:0/3172938554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd3415ca50 0x7fbd340cc830 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.740+0000 7fbd39a00640 1 -- 172.21.15.39:0/3172938554 >> 172.21.15.39:0/3172938554 conn(0x7fbd3406d410 msgr2=0x7fbd340b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:39.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.740+0000 7fbd39a00640 1 -- 172.21.15.39:0/3172938554 shutdown_connections 2024-09-21T07:38:39.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.740+0000 7fbd39a00640 1 -- 172.21.15.39:0/3172938554 wait complete. 2024-09-21T07:38:39.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.742+0000 7fbd39a00640 1 Processor -- start 2024-09-21T07:38:39.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.742+0000 7fbd39a00640 1 -- start start 2024-09-21T07:38:39.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.743+0000 7fbd39a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd340cf540 0x7fbd34131b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:39.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.743+0000 7fbd39a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbd340cf950 0x7fbd34131410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:39.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.743+0000 7fbd39a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd3415ca50 0x7fbd341e7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:39.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.743+0000 7fbd39a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3415de10 con 0x7fbd340cf540 2024-09-21T07:38:39.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.743+0000 7fbd39a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3415df90 con 0x7fbd3415ca50 2024-09-21T07:38:39.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.743+0000 7fbd39a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fbd3415e110 con 0x7fbd340cf950 2024-09-21T07:38:39.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.744+0000 7fbd33e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd340cf540 0x7fbd34131b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:39.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.744+0000 7fbd33e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd340cf540 0x7fbd34131b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49970/0 (socket says 172.21.15.39:49970) 2024-09-21T07:38:39.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.744+0000 7fbd33e00640 1 -- 172.21.15.39:0/3115971569 learned_addr learned my addr 172.21.15.39:0/3115971569 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:39.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.744+0000 7fbd33400640 1 --2- 172.21.15.39:0/3115971569 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbd340cf950 0x7fbd34131410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:39.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.744+0000 7fbd39000640 1 --2- 172.21.15.39:0/3115971569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd3415ca50 0x7fbd341e7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:39.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.744+0000 7fbd33e00640 1 -- 172.21.15.39:0/3115971569 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbd340cf950 msgr2=0x7fbd34131410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:39.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.744+0000 7fbd33e00640 1 --2- 172.21.15.39:0/3115971569 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbd340cf950 0x7fbd34131410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.744+0000 7fbd33e00640 1 -- 172.21.15.39:0/3115971569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd3415ca50 msgr2=0x7fbd341e7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:39.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.744+0000 7fbd33e00640 1 --2- 172.21.15.39:0/3115971569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd3415ca50 0x7fbd341e7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.744+0000 7fbd33e00640 1 -- 172.21.15.39:0/3115971569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd341e7b40 con 0x7fbd340cf540 2024-09-21T07:38:39.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.744+0000 7fbd33e00640 1 --2- 172.21.15.39:0/3115971569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd340cf540 0x7fbd34131b30 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fbd24006010 tx=0x7fbd240038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:39.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.745+0000 7fbd2be00640 1 -- 172.21.15.39:0/3115971569 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2403e030 con 0x7fbd340cf540 2024-09-21T07:38:39.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.745+0000 7fbd2be00640 1 -- 172.21.15.39:0/3115971569 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd24003d90 con 0x7fbd340cf540 2024-09-21T07:38:39.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.745+0000 7fbd2be00640 1 -- 172.21.15.39:0/3115971569 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd24035040 con 0x7fbd340cf540 2024-09-21T07:38:39.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.745+0000 7fbd39a00640 1 -- 172.21.15.39:0/3115971569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd34154bd0 con 0x7fbd340cf540 2024-09-21T07:38:39.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.745+0000 7fbd39a00640 1 -- 172.21.15.39:0/3115971569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd34155190 con 0x7fbd340cf540 2024-09-21T07:38:39.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.747+0000 7fbd2be00640 1 -- 172.21.15.39:0/3115971569 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbd24041030 con 0x7fbd340cf540 2024-09-21T07:38:39.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.748+0000 7fbd2be00640 1 --2- 172.21.15.39:0/3115971569 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbd14078b90 0x7fbd1407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:39.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.748+0000 7fbd2be00640 1 -- 172.21.15.39:0/3115971569 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 7173+0+0 (secure 0 0 0) 0x7fbd240bbb50 con 0x7fbd340cf540 2024-09-21T07:38:39.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.749+0000 7fbd33400640 1 --2- 172.21.15.39:0/3115971569 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbd14078b90 0x7fbd1407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:39.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.749+0000 7fbd4206c740 1 Processor -- start 2024-09-21T07:38:39.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.749+0000 7fbd4206c740 1 -- start start 2024-09-21T07:38:39.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.749+0000 7fbd33400640 1 --2- 172.21.15.39:0/3115971569 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbd14078b90 0x7fbd1407b0a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fbd20005fd0 tx=0x7fbd20005850 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:39.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.750+0000 7fbd4206c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffcdc82270 0x55ffce052b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:39.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.750+0000 7fbd4206c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffcdc593b0 0x55ffce0530a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:39.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.750+0000 7fbd4206c740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffcdc61590 0x55ffce023150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:39.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.750+0000 7fbd4206c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ffcdfac430 con 0x55ffcdc82270 2024-09-21T07:38:39.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.750+0000 7fbd4206c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ffcdf9c5d0 con 0x55ffcdc593b0 2024-09-21T07:38:39.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.750+0000 7fbd4206c740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55ffcdf8ba00 con 0x55ffcdc61590 2024-09-21T07:38:39.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.751+0000 7fbd33e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffcdc82270 0x55ffce052b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:39.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.751+0000 7fbd33e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffcdc82270 0x55ffce052b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49976/0 (socket says 172.21.15.39:49976) 2024-09-21T07:38:39.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.751+0000 7fbd33e00640 1 -- 172.21.15.39:0/3586747459 learned_addr learned my addr 172.21.15.39:0/3586747459 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:39.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.751+0000 7fbd33400640 1 --2- 172.21.15.39:0/3586747459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffcdc593b0 0x55ffce0530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:39.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.751+0000 7fbd39000640 1 --2- 172.21.15.39:0/3586747459 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffcdc61590 0x55ffce023150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:39.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.751+0000 7fbd33e00640 1 -- 172.21.15.39:0/3586747459 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffcdc61590 msgr2=0x55ffce023150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:38:39.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.751+0000 7fbd33e00640 1 --2- 172.21.15.39:0/3586747459 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffcdc61590 0x55ffce023150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.751+0000 7fbd33e00640 1 -- 172.21.15.39:0/3586747459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffcdc593b0 msgr2=0x55ffce0530a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:38:39.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.751+0000 7fbd33e00640 1 --2- 172.21.15.39:0/3586747459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffcdc593b0 0x55ffce0530a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.751+0000 7fbd33e00640 1 -- 172.21.15.39:0/3586747459 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffcdf54520 con 0x55ffcdc82270 2024-09-21T07:38:39.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.751+0000 7fbd33e00640 1 --2- 172.21.15.39:0/3586747459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffcdc82270 0x55ffce052b60 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fbd24034040 tx=0x7fbd24082a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9623378b6bc4544 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:39.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.752+0000 7fbd28c00640 1 -- 172.21.15.39:0/3586747459 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd24048040 con 0x55ffcdc82270 2024-09-21T07:38:39.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.752+0000 7fbd28c00640 1 -- 172.21.15.39:0/3586747459 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd240832b0 con 0x55ffcdc82270 2024-09-21T07:38:39.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.752+0000 7fbd28c00640 1 -- 172.21.15.39:0/3586747459 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2408ca40 con 0x55ffcdc82270 2024-09-21T07:38:39.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.752+0000 7fbd4206c740 1 -- 172.21.15.39:0/3586747459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffcdc82270 msgr2=0x55ffce052b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:39.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.752+0000 7fbd4206c740 1 --2- 172.21.15.39:0/3586747459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffcdc82270 0x55ffce052b60 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fbd24034040 tx=0x7fbd24082a20 comp rx=0 tx=0).stop 2024-09-21T07:38:39.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.753+0000 7fbd4206c740 1 -- 172.21.15.39:0/3586747459 shutdown_connections 2024-09-21T07:38:39.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.753+0000 7fbd4206c740 1 --2- 172.21.15.39:0/3586747459 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffcdc61590 0x55ffce023150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.753+0000 7fbd4206c740 1 --2- 172.21.15.39:0/3586747459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffcdc593b0 0x55ffce0530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.753+0000 7fbd4206c740 1 --2- 172.21.15.39:0/3586747459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffcdc82270 0x55ffce052b60 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.753+0000 7fbd4206c740 1 -- 172.21.15.39:0/3586747459 >> 172.21.15.39:0/3586747459 conn(0x55ffcdc763c0 msgr2=0x55ffce028b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:39.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.753+0000 7fbd4206c740 1 -- 172.21.15.39:0/3586747459 shutdown_connections 2024-09-21T07:38:39.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.753+0000 7fbd4206c740 1 -- 172.21.15.39:0/3586747459 wait complete. 2024-09-21T07:38:39.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.754+0000 7fbd4206c740 1 -- start start 2024-09-21T07:38:39.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.754+0000 7fbd4206c740 1 Processor -- start 2024-09-21T07:38:39.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.755+0000 7fbd4206c740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffcdc593b0 0x55ffce04b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:39.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.755+0000 7fbd4206c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffcdc61590 0x55ffce048e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:39.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.755+0000 7fbd4206c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffcdc82270 0x55ffce0659e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:39.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.755+0000 7fbd4206c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ffcdf32330 con 0x55ffcdc61590 2024-09-21T07:38:39.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.755+0000 7fbd4206c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ffcdf8f810 con 0x55ffcdc82270 2024-09-21T07:38:39.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.755+0000 7fbd4206c740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55ffcdcfc620 con 0x55ffcdc593b0 2024-09-21T07:38:39.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.755+0000 7fbd4206c740 10 client.0 _ll_register_callbacks cb 0x55ffcdf37420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:38:39.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.756+0000 7fbd33400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffcdc61590 0x55ffce048e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:39.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.756+0000 7fbd33400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffcdc61590 0x55ffce048e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49986/0 (socket says 172.21.15.39:49986) 2024-09-21T07:38:39.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.756+0000 7fbd33400640 1 -- 172.21.15.39:0/264346140 learned_addr learned my addr 172.21.15.39:0/264346140 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:39.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.756+0000 7fbd33e00640 1 --2- 172.21.15.39:0/264346140 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffcdc593b0 0x55ffce04b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:39.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.756+0000 7fbd39000640 1 --2- 172.21.15.39:0/264346140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffcdc82270 0x55ffce0659e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:39.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.756+0000 7fbd33400640 1 -- 172.21.15.39:0/264346140 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffcdc593b0 msgr2=0x55ffce04b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:38:39.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.756+0000 7fbd33400640 1 --2- 172.21.15.39:0/264346140 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffcdc593b0 0x55ffce04b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.756+0000 7fbd33400640 1 -- 172.21.15.39:0/264346140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffcdc82270 msgr2=0x55ffce0659e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:38:39.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.756+0000 7fbd33400640 1 --2- 172.21.15.39:0/264346140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffcdc82270 0x55ffce0659e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.756+0000 7fbd33400640 1 -- 172.21.15.39:0/264346140 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffcdced850 con 0x55ffcdc61590 2024-09-21T07:38:39.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.757+0000 7fbd33400640 1 --2- 172.21.15.39:0/264346140 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffcdc61590 0x55ffce048e90 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fbd340ce710 tx=0x7fbd200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f71ded7dbf45af0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:39.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.757+0000 7fbd1d600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:38:39.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.757+0000 7fbd1d600640 1 -- 172.21.15.39:0/264346140 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2001f840 con 0x55ffcdc61590 2024-09-21T07:38:39.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.757+0000 7fbd4206c740 1 -- 172.21.15.39:0/264346140 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd240832b0 con 0x55ffcdc61590 2024-09-21T07:38:39.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.757+0000 7fbd1d600640 1 -- 172.21.15.39:0/264346140 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 7173+0+0 (secure 0 0 0) 0x7fbd2001f9e0 con 0x55ffcdc61590 2024-09-21T07:38:39.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.757+0000 7fbd1d600640 1 -- 172.21.15.39:0/264346140 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fbd2002f050 con 0x55ffcdc61590 2024-09-21T07:38:39.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.757+0000 7fbd4206c740 10 client.6996 fetch_fsmap learned FSMap version 90 2024-09-21T07:38:39.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.757+0000 7fbd4206c740 1 -- 172.21.15.39:0/264346140 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x55ffcdd646b0 con 0x55ffcdc61590 2024-09-21T07:38:39.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.757+0000 7fbd1d600640 1 -- 172.21.15.39:0/264346140 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd240832b0 con 0x55ffcdc61590 2024-09-21T07:38:39.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.757+0000 7fbd1d600640 1 -- 172.21.15.39:0/264346140 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 90) ==== 3522+0+0 (secure 0 0 0) 0x7fbd2001fbe0 con 0x55ffcdc61590 2024-09-21T07:38:39.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.757+0000 7fbd4206c740 10 client.6996 fetch_fsmap finished waiting for FSMap version 90 2024-09-21T07:38:39.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.757+0000 7fbd4206c740 10 client.6996 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:38:39.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.757+0000 7fbd4206c740 20 client.6996 populate_metadata read hostname 'smithi039' 2024-09-21T07:38:39.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.758+0000 7fbd4206c740 1 --2- 172.21.15.39:0/264346140 >> [v2:172.21.15.39:6832/855150775,v1:172.21.15.39:6834/855150775] conn(0x55ffcdc40e40 0x55ffce07fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:39.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.758+0000 7fbd4206c740 4 client.6996 mds_command: new command op to 6825 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:38:39.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.758+0000 7fbd4206c740 1 -- 172.21.15.39:0/264346140 --> [v2:172.21.15.39:6832/855150775,v1:172.21.15.39:6834/855150775] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffcdf9e520 con 0x55ffcdc40e40 2024-09-21T07:38:39.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.758+0000 7fbd33e00640 1 --2- 172.21.15.39:0/264346140 >> [v2:172.21.15.39:6832/855150775,v1:172.21.15.39:6834/855150775] conn(0x55ffcdc40e40 0x55ffce07fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:39.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.758+0000 7fbd1d600640 10 client.6996 ms_handle_connect on v2:172.21.15.39:6832/855150775 2024-09-21T07:38:39.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.758+0000 7fbd33e00640 1 --2- 172.21.15.39:0/264346140 >> [v2:172.21.15.39:6832/855150775,v1:172.21.15.39:6834/855150775] conn(0x55ffcdc40e40 0x55ffce07fea0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=579a0785a4611f31 server_cookie=6a6aa388d483d45b in_seq=0 out_seq=0 2024-09-21T07:38:39.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.759+0000 7fbd1d600640 1 -- 172.21.15.39:0/264346140 <== mds.0 v2:172.21.15.39:6832/855150775 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ffcdf9e520 con 0x55ffcdc40e40 2024-09-21T07:38:39.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.759+0000 7fbd1d600640 10 client.6996 handle_command_reply: tid=0 2024-09-21T07:38:39.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.759+0000 7fbd4206c740 1 client.6996 shutdown 2024-09-21T07:38:39.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.759+0000 7fbd4206c740 1 -- 172.21.15.39:0/264346140 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffcdc61590 msgr2=0x55ffce048e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:39.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.759+0000 7fbd4206c740 1 --2- 172.21.15.39:0/264346140 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffcdc61590 0x55ffce048e90 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fbd340ce710 tx=0x7fbd200046c0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.760+0000 7fbd4206c740 1 -- 172.21.15.39:0/264346140 shutdown_connections 2024-09-21T07:38:39.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.760+0000 7fbd4206c740 1 --2- 172.21.15.39:0/264346140 >> [v2:172.21.15.39:6832/855150775,v1:172.21.15.39:6834/855150775] conn(0x55ffcdc40e40 0x55ffce07fea0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.760+0000 7fbd4206c740 1 --2- 172.21.15.39:0/264346140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffcdc82270 0x55ffce0659e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.760+0000 7fbd4206c740 1 --2- 172.21.15.39:0/264346140 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffcdc61590 0x55ffce048e90 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.760+0000 7fbd4206c740 1 --2- 172.21.15.39:0/264346140 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffcdc593b0 0x55ffce04b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.760+0000 7fbd4206c740 1 -- 172.21.15.39:0/264346140 >> 172.21.15.39:0/264346140 conn(0x55ffcdc763c0 msgr2=0x55ffce063ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:39.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.760+0000 7fbd1d600640 0 client.6996 ms_handle_reset on v2:172.21.15.39:6832/855150775 2024-09-21T07:38:39.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.760+0000 7fbd4206c740 1 -- 172.21.15.39:0/264346140 shutdown_connections 2024-09-21T07:38:39.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.760+0000 7fbd4206c740 1 -- 172.21.15.39:0/264346140 wait complete. 2024-09-21T07:38:39.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.760+0000 7fbd4206c740 20 client.6996 trim_cache size 0 max 16384 2024-09-21T07:38:39.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.779+0000 7fbd4206c740 1 Processor -- start 2024-09-21T07:38:39.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.779+0000 7fbd4206c740 1 -- start start 2024-09-21T07:38:39.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.781+0000 7fbd4206c740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffcdc593b0 0x55ffce090a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:39.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.781+0000 7fbd4206c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffcdc61590 0x55ffce063dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:39.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.781+0000 7fbd4206c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffcdc82270 0x55ffce064300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:39.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.781+0000 7fbd4206c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbd34202900 con 0x55ffcdc82270 2024-09-21T07:38:39.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.781+0000 7fbd4206c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbd34200cb0 con 0x55ffcdc61590 2024-09-21T07:38:39.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.781+0000 7fbd4206c740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fbd34201300 con 0x55ffcdc593b0 2024-09-21T07:38:39.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.781+0000 7fbd39000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffcdc82270 0x55ffce064300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:39.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.781+0000 7fbd39000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffcdc82270 0x55ffce064300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49990/0 (socket says 172.21.15.39:49990) 2024-09-21T07:38:39.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.781+0000 7fbd39000640 1 -- 172.21.15.39:0/2567538506 learned_addr learned my addr 172.21.15.39:0/2567538506 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:39.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.781+0000 7fbd33e00640 1 --2- 172.21.15.39:0/2567538506 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffcdc593b0 0x55ffce090a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:39.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.781+0000 7fbd33400640 1 --2- 172.21.15.39:0/2567538506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffcdc61590 0x55ffce063dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:39.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.781+0000 7fbd39000640 1 -- 172.21.15.39:0/2567538506 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffcdc593b0 msgr2=0x55ffce090a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:38:39.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.781+0000 7fbd39000640 1 --2- 172.21.15.39:0/2567538506 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffcdc593b0 0x55ffce090a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.781+0000 7fbd39000640 1 -- 172.21.15.39:0/2567538506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffcdc61590 msgr2=0x55ffce063dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:38:39.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.781+0000 7fbd39000640 1 --2- 172.21.15.39:0/2567538506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffcdc61590 0x55ffce063dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.781+0000 7fbd39000640 1 -- 172.21.15.39:0/2567538506 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffce025410 con 0x55ffcdc82270 2024-09-21T07:38:39.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.781+0000 7fbd33e00640 1 --2- 172.21.15.39:0/2567538506 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffcdc593b0 0x55ffce090a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:38:39.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.781+0000 7fbd33400640 1 --2- 172.21.15.39:0/2567538506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffcdc61590 0x55ffce063dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:38:39.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.782+0000 7fbd39000640 1 --2- 172.21.15.39:0/2567538506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffcdc82270 0x55ffce064300 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c003fd0 tx=0x7fbd2c005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29613289b2f5a5fd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:39.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.782+0000 7fbd1fe00640 1 -- 172.21.15.39:0/2567538506 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c01a020 con 0x55ffcdc82270 2024-09-21T07:38:39.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.782+0000 7fbd1fe00640 1 -- 172.21.15.39:0/2567538506 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd2c006b50 con 0x55ffcdc82270 2024-09-21T07:38:39.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.782+0000 7fbd1fe00640 1 -- 172.21.15.39:0/2567538506 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c0138b0 con 0x55ffcdc82270 2024-09-21T07:38:39.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.783+0000 7fbd4206c740 1 -- 172.21.15.39:0/2567538506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffcdc82270 msgr2=0x55ffce064300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:39.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.783+0000 7fbd4206c740 1 --2- 172.21.15.39:0/2567538506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffcdc82270 0x55ffce064300 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c003fd0 tx=0x7fbd2c005620 comp rx=0 tx=0).stop 2024-09-21T07:38:39.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.783+0000 7fbd4206c740 1 -- 172.21.15.39:0/2567538506 shutdown_connections 2024-09-21T07:38:39.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.783+0000 7fbd4206c740 1 --2- 172.21.15.39:0/2567538506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffcdc82270 0x55ffce064300 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.783+0000 7fbd4206c740 1 --2- 172.21.15.39:0/2567538506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffcdc61590 0x55ffce063dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.783+0000 7fbd4206c740 1 --2- 172.21.15.39:0/2567538506 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffcdc593b0 0x55ffce090a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.783+0000 7fbd4206c740 1 -- 172.21.15.39:0/2567538506 >> 172.21.15.39:0/2567538506 conn(0x55ffcdc763c0 msgr2=0x55ffce0265c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:39.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.783+0000 7fbd4206c740 1 -- 172.21.15.39:0/2567538506 shutdown_connections 2024-09-21T07:38:39.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.783+0000 7fbd4206c740 1 -- 172.21.15.39:0/2567538506 wait complete. 2024-09-21T07:38:39.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.784+0000 7fbd4206c740 1 -- start start 2024-09-21T07:38:39.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.784+0000 7fbd4206c740 1 Processor -- start 2024-09-21T07:38:39.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.785+0000 7fbd4206c740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffcdc593b0 0x55ffce0e7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:39.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.785+0000 7fbd4206c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffcdc61590 0x55ffce0f0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:39.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.786+0000 7fbd4206c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffcdc82270 0x55ffce0f0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:39.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.786+0000 7fbd4206c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ffce023e40 con 0x55ffcdc82270 2024-09-21T07:38:39.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.786+0000 7fbd4206c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ffce04b010 con 0x55ffcdc61590 2024-09-21T07:38:39.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.786+0000 7fbd4206c740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55ffce066510 con 0x55ffcdc593b0 2024-09-21T07:38:39.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.786+0000 7fbd4206c740 10 client.0 _ll_register_callbacks cb 0x7fbd2000fc30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:38:39.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.786+0000 7fbd39000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffcdc82270 0x55ffce0f0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:39.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.786+0000 7fbd39000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffcdc82270 0x55ffce0f0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50004/0 (socket says 172.21.15.39:50004) 2024-09-21T07:38:39.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.786+0000 7fbd39000640 1 -- 172.21.15.39:0/2561274882 learned_addr learned my addr 172.21.15.39:0/2561274882 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:39.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.786+0000 7fbd33e00640 1 --2- 172.21.15.39:0/2561274882 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffcdc593b0 0x55ffce0e7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:39.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.786+0000 7fbd33400640 1 --2- 172.21.15.39:0/2561274882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffcdc61590 0x55ffce0f0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:39.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.786+0000 7fbd39000640 1 -- 172.21.15.39:0/2561274882 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffcdc593b0 msgr2=0x55ffce0e7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:38:39.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.786+0000 7fbd39000640 1 --2- 172.21.15.39:0/2561274882 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffcdc593b0 0x55ffce0e7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.786+0000 7fbd39000640 1 -- 172.21.15.39:0/2561274882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffcdc61590 msgr2=0x55ffce0f0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:38:39.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.786+0000 7fbd39000640 1 --2- 172.21.15.39:0/2561274882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffcdc61590 0x55ffce0f0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.786+0000 7fbd39000640 1 -- 172.21.15.39:0/2561274882 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffce0712c0 con 0x55ffcdc82270 2024-09-21T07:38:39.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.786+0000 7fbd33400640 1 --2- 172.21.15.39:0/2561274882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffcdc61590 0x55ffce0f0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:38:39.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.787+0000 7fbd39000640 1 --2- 172.21.15.39:0/2561274882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffcdc82270 0x55ffce0f0d50 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fbd34153ab0 tx=0x7fbd2c013f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3747e78946450640 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:39.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.787+0000 7fbd1d600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:38:39.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.787+0000 7fbd1d600640 1 -- 172.21.15.39:0/2561274882 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c029060 con 0x55ffcdc82270 2024-09-21T07:38:39.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.787+0000 7fbd1d600640 1 -- 172.21.15.39:0/2561274882 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 7173+0+0 (secure 0 0 0) 0x7fbd2c016860 con 0x55ffcdc82270 2024-09-21T07:38:39.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.787+0000 7fbd4206c740 1 -- 172.21.15.39:0/2561274882 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd2c006b50 con 0x55ffcdc82270 2024-09-21T07:38:39.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.787+0000 7fbd1d600640 1 -- 172.21.15.39:0/2561274882 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fbd2c016a60 con 0x55ffcdc82270 2024-09-21T07:38:39.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.787+0000 7fbd4206c740 10 client.7002 fetch_fsmap learned FSMap version 90 2024-09-21T07:38:39.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.787+0000 7fbd4206c740 1 -- 172.21.15.39:0/2561274882 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x55ffce065080 con 0x55ffcdc82270 2024-09-21T07:38:39.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.787+0000 7fbd1d600640 1 -- 172.21.15.39:0/2561274882 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd2c006b50 con 0x55ffcdc82270 2024-09-21T07:38:39.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.787+0000 7fbd1d600640 1 -- 172.21.15.39:0/2561274882 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 90) ==== 3522+0+0 (secure 0 0 0) 0x7fbd2c016c00 con 0x55ffcdc82270 2024-09-21T07:38:39.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.787+0000 7fbd4206c740 10 client.7002 fetch_fsmap finished waiting for FSMap version 90 2024-09-21T07:38:39.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.788+0000 7fbd4206c740 10 client.7002 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:38:39.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.788+0000 7fbd4206c740 20 client.7002 populate_metadata read hostname 'smithi039' 2024-09-21T07:38:39.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.788+0000 7fbd4206c740 1 --2- 172.21.15.39:0/2561274882 >> [v2:172.21.15.39:6832/855150775,v1:172.21.15.39:6834/855150775] conn(0x55ffce0edc00 0x55ffce064000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:39.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.788+0000 7fbd4206c740 4 client.7002 mds_command: new command op to 6825 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:38:39.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.788+0000 7fbd4206c740 1 -- 172.21.15.39:0/2561274882 --> [v2:172.21.15.39:6832/855150775,v1:172.21.15.39:6834/855150775] -- command(tid 0: {"prefix": "status"}) -- 0x55ffce0281a0 con 0x55ffce0edc00 2024-09-21T07:38:39.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.788+0000 7fbd33e00640 1 --2- 172.21.15.39:0/2561274882 >> [v2:172.21.15.39:6832/855150775,v1:172.21.15.39:6834/855150775] conn(0x55ffce0edc00 0x55ffce064000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:39.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.788+0000 7fbd1d600640 10 client.7002 ms_handle_connect on v2:172.21.15.39:6832/855150775 2024-09-21T07:38:39.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.788+0000 7fbd33e00640 1 --2- 172.21.15.39:0/2561274882 >> [v2:172.21.15.39:6832/855150775,v1:172.21.15.39:6834/855150775] conn(0x55ffce0edc00 0x55ffce064000 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e73d733149bf047 server_cookie=330bb3a3be4f9111 in_seq=0 out_seq=0 2024-09-21T07:38:39.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.789+0000 7fbd1d600640 1 -- 172.21.15.39:0/2561274882 <== mds.0 v2:172.21.15.39:6832/855150775 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55ffce0281a0 con 0x55ffce0edc00 2024-09-21T07:38:39.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.789+0000 7fbd1d600640 10 client.7002 handle_command_reply: tid=0 2024-09-21T07:38:39.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.789+0000 7fbd4206c740 1 client.7002 shutdown 2024-09-21T07:38:39.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.789+0000 7fbd4206c740 1 -- 172.21.15.39:0/2561274882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffcdc82270 msgr2=0x55ffce0f0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:39.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.789+0000 7fbd4206c740 1 --2- 172.21.15.39:0/2561274882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffcdc82270 0x55ffce0f0d50 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fbd34153ab0 tx=0x7fbd2c013f90 comp rx=0 tx=0).stop 2024-09-21T07:38:39.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.789+0000 7fbd4206c740 1 -- 172.21.15.39:0/2561274882 shutdown_connections 2024-09-21T07:38:39.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.789+0000 7fbd4206c740 1 --2- 172.21.15.39:0/2561274882 >> [v2:172.21.15.39:6832/855150775,v1:172.21.15.39:6834/855150775] conn(0x55ffce0edc00 0x55ffce064000 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.789+0000 7fbd4206c740 1 --2- 172.21.15.39:0/2561274882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ffcdc82270 0x55ffce0f0d50 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.789+0000 7fbd4206c740 1 --2- 172.21.15.39:0/2561274882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ffcdc61590 0x55ffce0f0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.789+0000 7fbd4206c740 1 --2- 172.21.15.39:0/2561274882 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ffcdc593b0 0x55ffce0e7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.789+0000 7fbd1d600640 0 client.7002 ms_handle_reset on v2:172.21.15.39:6832/855150775 2024-09-21T07:38:39.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.789+0000 7fbd4206c740 1 -- 172.21.15.39:0/2561274882 >> 172.21.15.39:0/2561274882 conn(0x55ffcdc763c0 msgr2=0x55ffce070960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:39.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.789+0000 7fbd4206c740 1 -- 172.21.15.39:0/2561274882 shutdown_connections 2024-09-21T07:38:39.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.789+0000 7fbd4206c740 1 -- 172.21.15.39:0/2561274882 wait complete. 2024-09-21T07:38:39.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.789+0000 7fbd4206c740 20 client.7002 trim_cache size 0 max 16384 2024-09-21T07:38:39.795 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:38:39.795 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:38:39.795 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-09-21T07:38:39.795 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 6825, 2024-09-21T07:38:39.795 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:38:39.795 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:38:39.795 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:38:39.795 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 7.5715548520000002, 2024-09-21T07:38:39.795 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 90, 2024-09-21T07:38:39.795 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 96, 2024-09-21T07:38:39.795 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 96, 2024-09-21T07:38:39.795 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 41.332303822999997 2024-09-21T07:38:39.796 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:38:39.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.790+0000 7fbd39a00640 1 -- 172.21.15.39:0/3115971569 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbd14078b90 msgr2=0x7fbd1407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:39.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.790+0000 7fbd39a00640 1 --2- 172.21.15.39:0/3115971569 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbd14078b90 0x7fbd1407b0a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fbd20005fd0 tx=0x7fbd20005850 comp rx=0 tx=0).stop 2024-09-21T07:38:39.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.790+0000 7fbd39a00640 1 -- 172.21.15.39:0/3115971569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd340cf540 msgr2=0x7fbd34131b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:39.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.790+0000 7fbd39a00640 1 --2- 172.21.15.39:0/3115971569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd340cf540 0x7fbd34131b30 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fbd24006010 tx=0x7fbd240038a0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.790+0000 7fbd39a00640 1 -- 172.21.15.39:0/3115971569 shutdown_connections 2024-09-21T07:38:39.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.790+0000 7fbd39a00640 1 --2- 172.21.15.39:0/3115971569 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbd14078b90 0x7fbd1407b0a0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.790+0000 7fbd39a00640 1 --2- 172.21.15.39:0/3115971569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd3415ca50 0x7fbd341e7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.790+0000 7fbd39a00640 1 --2- 172.21.15.39:0/3115971569 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbd340cf950 0x7fbd34131410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.790+0000 7fbd39a00640 1 --2- 172.21.15.39:0/3115971569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd340cf540 0x7fbd34131b30 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:39.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.790+0000 7fbd39a00640 1 -- 172.21.15.39:0/3115971569 >> 172.21.15.39:0/3115971569 conn(0x7fbd3406d410 msgr2=0x7fbd341455e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:39.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.790+0000 7fbd39a00640 1 -- 172.21.15.39:0/3115971569 shutdown_connections 2024-09-21T07:38:39.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:39.790+0000 7fbd39a00640 1 -- 172.21.15.39:0/3115971569 wait complete. 2024-09-21T07:38:39.814 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:38:40.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.041+0000 7f65c3600640 1 -- 172.21.15.39:0/1083391254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65bc0cc1e0 msgr2=0x7f65bc0af3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:40.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.041+0000 7f65c3600640 1 --2- 172.21.15.39:0/1083391254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65bc0cc1e0 0x7f65bc0af3c0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f65ac009830 tx=0x7f65ac02a510 comp rx=0 tx=0).stop 2024-09-21T07:38:40.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.041+0000 7f65c3600640 1 -- 172.21.15.39:0/1083391254 shutdown_connections 2024-09-21T07:38:40.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.041+0000 7f65c3600640 1 --2- 172.21.15.39:0/1083391254 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f65bc15d8a0 0x7f65bc0b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:40.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.041+0000 7f65c3600640 1 --2- 172.21.15.39:0/1083391254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65bc0cc1e0 0x7f65bc0af3c0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:40.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.041+0000 7f65c3600640 1 -- 172.21.15.39:0/1083391254 >> 172.21.15.39:0/1083391254 conn(0x7f65bc06d7d0 msgr2=0x7f65bc0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:40.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.041+0000 7f65c3600640 1 -- 172.21.15.39:0/1083391254 shutdown_connections 2024-09-21T07:38:40.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.041+0000 7f65c3600640 1 -- 172.21.15.39:0/1083391254 wait complete. 2024-09-21T07:38:40.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.043+0000 7f65c3600640 1 Processor -- start 2024-09-21T07:38:40.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.044+0000 7f65c3600640 1 -- start start 2024-09-21T07:38:40.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.045+0000 7f65c3600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65bc0cc1e0 0x7f65bc1508a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:40.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.045+0000 7f65c3600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f65bc0ccb50 0x7f65bc0c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:40.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.045+0000 7f65c3600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65bc15d8a0 0x7f65bc0c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:40.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.045+0000 7f65c3600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc153520 con 0x7f65bc0cc1e0 2024-09-21T07:38:40.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.045+0000 7f65c3600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc1533a0 con 0x7f65bc15d8a0 2024-09-21T07:38:40.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.045+0000 7f65c3600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f65bc1536a0 con 0x7f65bc0ccb50 2024-09-21T07:38:40.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.045+0000 7f65c0c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65bc0cc1e0 0x7f65bc1508a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:40.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.045+0000 7f65c0c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65bc0cc1e0 0x7f65bc1508a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50026/0 (socket says 172.21.15.39:50026) 2024-09-21T07:38:40.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.045+0000 7f65c0c00640 1 -- 172.21.15.39:0/3370789368 learned_addr learned my addr 172.21.15.39:0/3370789368 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:40.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.046+0000 7f65b3e00640 1 --2- 172.21.15.39:0/3370789368 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f65bc0ccb50 0x7f65bc0c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:40.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.046+0000 7f65c1600640 1 --2- 172.21.15.39:0/3370789368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65bc15d8a0 0x7f65bc0c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:40.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.046+0000 7f65c0c00640 1 -- 172.21.15.39:0/3370789368 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f65bc0ccb50 msgr2=0x7f65bc0c9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:40.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.046+0000 7f65c0c00640 1 --2- 172.21.15.39:0/3370789368 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f65bc0ccb50 0x7f65bc0c9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:40.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.046+0000 7f65c0c00640 1 -- 172.21.15.39:0/3370789368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65bc15d8a0 msgr2=0x7f65bc0c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:40.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.046+0000 7f65c0c00640 1 --2- 172.21.15.39:0/3370789368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65bc15d8a0 0x7f65bc0c98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:40.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.046+0000 7f65c0c00640 1 -- 172.21.15.39:0/3370789368 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65bc073b20 con 0x7f65bc0cc1e0 2024-09-21T07:38:40.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.046+0000 7f65c1600640 1 --2- 172.21.15.39:0/3370789368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65bc15d8a0 0x7f65bc0c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:38:40.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.046+0000 7f65c0c00640 1 --2- 172.21.15.39:0/3370789368 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65bc0cc1e0 0x7f65bc1508a0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f65ac009950 tx=0x7f65ac033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:40.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.046+0000 7f65b9600640 1 -- 172.21.15.39:0/3370789368 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ac005c90 con 0x7f65bc0cc1e0 2024-09-21T07:38:40.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.047+0000 7f65b9600640 1 -- 172.21.15.39:0/3370789368 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65ac031e50 con 0x7f65bc0cc1e0 2024-09-21T07:38:40.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.047+0000 7f65b9600640 1 -- 172.21.15.39:0/3370789368 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ac005260 con 0x7f65bc0cc1e0 2024-09-21T07:38:40.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.047+0000 7f65c3600640 1 -- 172.21.15.39:0/3370789368 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65bc073e00 con 0x7f65bc0cc1e0 2024-09-21T07:38:40.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.047+0000 7f65c3600640 1 -- 172.21.15.39:0/3370789368 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65bc0743c0 con 0x7f65bc0cc1e0 2024-09-21T07:38:40.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.048+0000 7f65c3600640 1 -- 172.21.15.39:0/3370789368 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65bc153b50 con 0x7f65bc0cc1e0 2024-09-21T07:38:40.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.049+0000 7f65b9600640 1 -- 172.21.15.39:0/3370789368 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f65ac005400 con 0x7f65bc0cc1e0 2024-09-21T07:38:40.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.050+0000 7f65b9600640 1 --2- 172.21.15.39:0/3370789368 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6598078b90 0x7f659807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:40.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.050+0000 7f65b9600640 1 -- 172.21.15.39:0/3370789368 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 7173+0+0 (secure 0 0 0) 0x7f65ac0bc490 con 0x7f65bc0cc1e0 2024-09-21T07:38:40.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.051+0000 7f65b3e00640 1 --2- 172.21.15.39:0/3370789368 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6598078b90 0x7f659807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:40.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.051+0000 7f65b3e00640 1 --2- 172.21.15.39:0/3370789368 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6598078b90 0x7f659807b0a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f65bc09d880 tx=0x7f65a4005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:40.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.055+0000 7f65b9600640 1 -- 172.21.15.39:0/3370789368 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f65ac03f030 con 0x7f65bc0cc1e0 2024-09-21T07:38:40.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.290+0000 7f65c3600640 1 -- 172.21.15.39:0/3370789368 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f65bc070040 con 0x7f65bc0cc1e0 2024-09-21T07:38:40.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.291+0000 7f65b9600640 1 -- 172.21.15.39:0/3370789368 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5243 (secure 0 0 0) 0x7f65ac084300 con 0x7f65bc0cc1e0 2024-09-21T07:38:40.293 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:40.293 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":90,"btime":"2024-09-21T07:38:36:232466+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/992188984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":992188984},{"type":"v1","addr":"172.21.15.148:6837","nonce":992188984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:38:36.232454+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6825,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6825":{"gid":6825,"name":"c","rank":0,"incarnation":86,"state":"up:active","state_seq":13,"addr":"172.21.15.39:6834/855150775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":855150775},{"type":"v1","addr":"172.21.15.39:6834","nonce":855150775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6825,"qdb_cluster":[6115,6825]},"id":6}]} 2024-09-21T07:38:40.294 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 90 2024-09-21T07:38:40.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.293+0000 7f65c3600640 1 -- 172.21.15.39:0/3370789368 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6598078b90 msgr2=0x7f659807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:40.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.293+0000 7f65c3600640 1 --2- 172.21.15.39:0/3370789368 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6598078b90 0x7f659807b0a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f65bc09d880 tx=0x7f65a4005f90 comp rx=0 tx=0).stop 2024-09-21T07:38:40.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.293+0000 7f65c3600640 1 -- 172.21.15.39:0/3370789368 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65bc0cc1e0 msgr2=0x7f65bc1508a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:40.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.293+0000 7f65c3600640 1 --2- 172.21.15.39:0/3370789368 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65bc0cc1e0 0x7f65bc1508a0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f65ac009950 tx=0x7f65ac033c20 comp rx=0 tx=0).stop 2024-09-21T07:38:40.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.293+0000 7f65c3600640 1 -- 172.21.15.39:0/3370789368 shutdown_connections 2024-09-21T07:38:40.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.293+0000 7f65c3600640 1 --2- 172.21.15.39:0/3370789368 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6598078b90 0x7f659807b0a0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:40.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.293+0000 7f65c3600640 1 --2- 172.21.15.39:0/3370789368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65bc15d8a0 0x7f65bc0c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:40.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.293+0000 7f65c3600640 1 --2- 172.21.15.39:0/3370789368 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f65bc0ccb50 0x7f65bc0c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:40.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.293+0000 7f65c3600640 1 --2- 172.21.15.39:0/3370789368 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65bc0cc1e0 0x7f65bc1508a0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:40.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.293+0000 7f65c3600640 1 -- 172.21.15.39:0/3370789368 >> 172.21.15.39:0/3370789368 conn(0x7f65bc06d7d0 msgr2=0x7f65bc0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:40.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.294+0000 7f65c3600640 1 -- 172.21.15.39:0/3370789368 shutdown_connections 2024-09-21T07:38:40.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.294+0000 7f65c3600640 1 -- 172.21.15.39:0/3370789368 wait complete. 2024-09-21T07:38:40.320 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2024-09-21T07:38:40.564 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_snap":5,"last_created":5,"last_destroyed":1,"pending_noop":[],"snaps":[{"snapid":2,"ino":1099511627777,"stamp":"2024-09-21T07:36:43.100278+0000","name":"s11","metadata":{}},{"snapid":3,"ino":1099511627777,"stamp":"2024-09-21T07:37:11.742421+0000","name":"s12","metadata":{}},{"snapid":4,"ino":1099511627777,"stamp":"2024-09-21T07:37:40.176395+0000","name":"s15","metadata":{}},{"snapid":5,"ino":1099511627777,"stamp":"2024-09-21T07:38:08.632985+0000","name":"s16","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2024-09-21T07:38:40.565 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 5, "last_destroyed": 1, "last_snap": 5, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s11", "snapid": 2, "stamp": "2024-09-21T07:36:43.100278+0000" }, { "ino": 1099511627777, "metadata": {}, "name": "s12", "snapid": 3, "stamp": "2024-09-21T07:37:11.742421+0000" }, { "ino": 1099511627777, "metadata": {}, "name": "s15", "snapid": 4, "stamp": "2024-09-21T07:37:40.176395+0000" }, { "ino": 1099511627777, "metadata": {}, "name": "s16", "snapid": 5, "stamp": "2024-09-21T07:38:08.632985+0000" } ] } 2024-09-21T07:38:40.565 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T07:38:40.566 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T07:38:40.570 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T07:38:40.664 INFO:teuthology.orchestra.run.smithi039.stdout:[global] 2024-09-21T07:38:40.664 INFO:teuthology.orchestra.run.smithi039.stdout: chdir = "" 2024-09-21T07:38:40.664 INFO:teuthology.orchestra.run.smithi039.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T07:38:40.665 INFO:teuthology.orchestra.run.smithi039.stdout: auth supported = cephx 2024-09-21T07:38:40.665 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:40.665 INFO:teuthology.orchestra.run.smithi039.stdout: filestore xattr use omap = true 2024-09-21T07:38:40.665 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:40.665 INFO:teuthology.orchestra.run.smithi039.stdout: mon clock drift allowed = 1.000 2024-09-21T07:38:40.665 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:40.665 INFO:teuthology.orchestra.run.smithi039.stdout: osd crush chooseleaf type = 0 2024-09-21T07:38:40.665 INFO:teuthology.orchestra.run.smithi039.stdout: auth debug = true 2024-09-21T07:38:40.665 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:40.665 INFO:teuthology.orchestra.run.smithi039.stdout: ms die on old message = true 2024-09-21T07:38:40.665 INFO:teuthology.orchestra.run.smithi039.stdout: ms die on bug = true 2024-09-21T07:38:40.665 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:40.665 INFO:teuthology.orchestra.run.smithi039.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T07:38:40.665 INFO:teuthology.orchestra.run.smithi039.stdout: mon pg warn max object skew = 0 2024-09-21T07:38:40.666 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:40.666 INFO:teuthology.orchestra.run.smithi039.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T07:38:40.666 INFO:teuthology.orchestra.run.smithi039.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T07:38:40.666 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:40.666 INFO:teuthology.orchestra.run.smithi039.stdout: osd pool default size = 2 2024-09-21T07:38:40.666 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:40.666 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd allow primary affinity = true 2024-09-21T07:38:40.666 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd allow pg remap = true 2024-09-21T07:38:40.666 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on legacy crush tunables = false 2024-09-21T07:38:40.666 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on crush straw calc version zero = false 2024-09-21T07:38:40.666 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on no sortbitwise = false 2024-09-21T07:38:40.666 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on osd down out interval zero = false 2024-09-21T07:38:40.666 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on too few osds = false 2024-09-21T07:38:40.666 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T07:38:40.666 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T07:38:40.667 INFO:teuthology.orchestra.run.smithi039.stdout: mon_allow_pool_size_one = true 2024-09-21T07:38:40.667 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:40.667 INFO:teuthology.orchestra.run.smithi039.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T07:38:40.667 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:40.667 INFO:teuthology.orchestra.run.smithi039.stdout: osd default data pool replay window = 5 2024-09-21T07:38:40.667 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:40.667 INFO:teuthology.orchestra.run.smithi039.stdout: mon allow pool delete = true 2024-09-21T07:38:40.667 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:40.667 INFO:teuthology.orchestra.run.smithi039.stdout: mon cluster log file level = debug 2024-09-21T07:38:40.667 INFO:teuthology.orchestra.run.smithi039.stdout: debug asserts on shutdown = true 2024-09-21T07:38:40.667 INFO:teuthology.orchestra.run.smithi039.stdout: mon health detail to clog = false 2024-09-21T07:38:40.667 INFO:teuthology.orchestra.run.smithi039.stdout: mon host = "172.21.15.39,172.21.15.148,[v2:172.21.15.148:3301,v1:172.21.15.148:6790]" 2024-09-21T07:38:40.667 INFO:teuthology.orchestra.run.smithi039.stdout: ms die on skipped message = False 2024-09-21T07:38:40.667 INFO:teuthology.orchestra.run.smithi039.stdout: fsid = 7ab9b04a-f88d-49aa-a493-e5c076637ffb 2024-09-21T07:38:40.667 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:40.668 INFO:teuthology.orchestra.run.smithi039.stdout:[osd] 2024-09-21T07:38:40.668 INFO:teuthology.orchestra.run.smithi039.stdout: osd journal size = 100 2024-09-21T07:38:40.668 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:40.668 INFO:teuthology.orchestra.run.smithi039.stdout: osd scrub load threshold = 5.0 2024-09-21T07:38:40.668 INFO:teuthology.orchestra.run.smithi039.stdout: osd scrub max interval = 600 2024-09-21T07:38:40.668 INFO:teuthology.orchestra.run.smithi039.stdout: osd mclock profile = high_recovery_ops 2024-09-21T07:38:40.668 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:40.668 INFO:teuthology.orchestra.run.smithi039.stdout: osd recover clone overlap = true 2024-09-21T07:38:40.668 INFO:teuthology.orchestra.run.smithi039.stdout: osd recovery max chunk = 1048576 2024-09-21T07:38:40.668 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:40.668 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug shutdown = true 2024-09-21T07:38:40.668 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug op order = true 2024-09-21T07:38:40.668 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug verify stray on activate = true 2024-09-21T07:38:40.668 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug trim objects = true 2024-09-21T07:38:40.669 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:40.669 INFO:teuthology.orchestra.run.smithi039.stdout: osd open classes on start = true 2024-09-21T07:38:40.669 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug pg log writeout = true 2024-09-21T07:38:40.669 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:40.669 INFO:teuthology.orchestra.run.smithi039.stdout: osd deep scrub update digest min age = 30 2024-09-21T07:38:40.669 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:40.669 INFO:teuthology.orchestra.run.smithi039.stdout: osd map max advance = 10 2024-09-21T07:38:40.669 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:40.669 INFO:teuthology.orchestra.run.smithi039.stdout: journal zero on create = true 2024-09-21T07:38:40.669 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:40.669 INFO:teuthology.orchestra.run.smithi039.stdout: filestore ondisk finisher threads = 3 2024-09-21T07:38:40.669 INFO:teuthology.orchestra.run.smithi039.stdout: filestore apply finisher threads = 3 2024-09-21T07:38:40.669 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:40.669 INFO:teuthology.orchestra.run.smithi039.stdout: bdev debug aio = true 2024-09-21T07:38:40.670 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug misdirected ops = true 2024-09-21T07:38:40.670 INFO:teuthology.orchestra.run.smithi039.stdout: bdev async discard = True 2024-09-21T07:38:40.670 INFO:teuthology.orchestra.run.smithi039.stdout: bdev enable discard = True 2024-09-21T07:38:40.670 INFO:teuthology.orchestra.run.smithi039.stdout: bluestore allocator = bitmap 2024-09-21T07:38:40.670 INFO:teuthology.orchestra.run.smithi039.stdout: bluestore block size = 96636764160 2024-09-21T07:38:40.670 INFO:teuthology.orchestra.run.smithi039.stdout: bluestore fsck on mount = True 2024-09-21T07:38:40.670 INFO:teuthology.orchestra.run.smithi039.stdout: debug bluefs = 1/20 2024-09-21T07:38:40.670 INFO:teuthology.orchestra.run.smithi039.stdout: debug bluestore = 1/20 2024-09-21T07:38:40.670 INFO:teuthology.orchestra.run.smithi039.stdout: debug ms = 1 2024-09-21T07:38:40.670 INFO:teuthology.orchestra.run.smithi039.stdout: debug osd = 20 2024-09-21T07:38:40.670 INFO:teuthology.orchestra.run.smithi039.stdout: debug rocksdb = 4/10 2024-09-21T07:38:40.670 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T07:38:40.670 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd full ratio = 0.9 2024-09-21T07:38:40.670 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd nearfull ratio = 0.8 2024-09-21T07:38:40.671 INFO:teuthology.orchestra.run.smithi039.stdout: osd failsafe full ratio = 0.95 2024-09-21T07:38:40.671 INFO:teuthology.orchestra.run.smithi039.stdout: osd objectstore = bluestore 2024-09-21T07:38:40.671 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:40.671 INFO:teuthology.orchestra.run.smithi039.stdout:[mgr] 2024-09-21T07:38:40.671 INFO:teuthology.orchestra.run.smithi039.stdout: debug ms = 1 2024-09-21T07:38:40.671 INFO:teuthology.orchestra.run.smithi039.stdout: debug mgr = 20 2024-09-21T07:38:40.671 INFO:teuthology.orchestra.run.smithi039.stdout: debug mon = 20 2024-09-21T07:38:40.671 INFO:teuthology.orchestra.run.smithi039.stdout: debug auth = 20 2024-09-21T07:38:40.671 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min pgs per osd = 4 2024-09-21T07:38:40.671 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min bytes per osd = 10 2024-09-21T07:38:40.671 INFO:teuthology.orchestra.run.smithi039.stdout: mgr/telemetry/nag = false 2024-09-21T07:38:40.671 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:40.671 INFO:teuthology.orchestra.run.smithi039.stdout:[mon] 2024-09-21T07:38:40.671 INFO:teuthology.orchestra.run.smithi039.stdout: debug ms = 1 2024-09-21T07:38:40.671 INFO:teuthology.orchestra.run.smithi039.stdout: debug mon = 20 2024-09-21T07:38:40.672 INFO:teuthology.orchestra.run.smithi039.stdout: debug paxos = 20 2024-09-21T07:38:40.672 INFO:teuthology.orchestra.run.smithi039.stdout: debug auth = 20 2024-09-21T07:38:40.672 INFO:teuthology.orchestra.run.smithi039.stdout: mon data avail warn = 5 2024-09-21T07:38:40.672 INFO:teuthology.orchestra.run.smithi039.stdout: mon mgr mkfs grace = 240 2024-09-21T07:38:40.672 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min pgs per osd = 4 2024-09-21T07:38:40.672 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd reporter subtree level = osd 2024-09-21T07:38:40.672 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd prime pg temp = true 2024-09-21T07:38:40.672 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min bytes per osd = 10 2024-09-21T07:38:40.672 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:40.672 INFO:teuthology.orchestra.run.smithi039.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T07:38:40.672 INFO:teuthology.orchestra.run.smithi039.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T07:38:40.672 INFO:teuthology.orchestra.run.smithi039.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T07:38:40.672 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:40.672 INFO:teuthology.orchestra.run.smithi039.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T07:38:40.672 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T07:38:40.673 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T07:38:40.673 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:40.673 INFO:teuthology.orchestra.run.smithi039.stdout: # 1m isn't quite enough 2024-09-21T07:38:40.673 INFO:teuthology.orchestra.run.smithi039.stdout: mon_down_mkfs_grace = 2m 2024-09-21T07:38:40.673 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:40.673 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_filestore_osds = false 2024-09-21T07:38:40.673 INFO:teuthology.orchestra.run.smithi039.stdout: mon down mkfs grace = 300 2024-09-21T07:38:40.673 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:40.673 INFO:teuthology.orchestra.run.smithi039.stdout:[client] 2024-09-21T07:38:40.673 INFO:teuthology.orchestra.run.smithi039.stdout: rgw cache enabled = true 2024-09-21T07:38:40.673 INFO:teuthology.orchestra.run.smithi039.stdout: rgw enable ops log = true 2024-09-21T07:38:40.673 INFO:teuthology.orchestra.run.smithi039.stdout: rgw enable usage log = true 2024-09-21T07:38:40.673 INFO:teuthology.orchestra.run.smithi039.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T07:38:40.673 INFO:teuthology.orchestra.run.smithi039.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T07:38:40.673 INFO:teuthology.orchestra.run.smithi039.stdout: pid file = "" 2024-09-21T07:38:40.674 INFO:teuthology.orchestra.run.smithi039.stdout:[mon.a] 2024-09-21T07:38:40.674 INFO:teuthology.orchestra.run.smithi039.stdout:[mon.b] 2024-09-21T07:38:40.674 INFO:teuthology.orchestra.run.smithi039.stdout:[mon.c] 2024-09-21T07:38:40.674 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.a] 2024-09-21T07:38:40.674 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.c] 2024-09-21T07:38:40.674 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.b] 2024-09-21T07:38:40.674 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.d] 2024-09-21T07:38:40.674 INFO:teuthology.orchestra.run.smithi039.stdout:[mds] 2024-09-21T07:38:40.674 INFO:teuthology.orchestra.run.smithi039.stdout: mds_reconnect_timeout = 5 2024-09-21T07:38:40.677 INFO:teuthology.orchestra.run.smithi148.stdout:[global] 2024-09-21T07:38:40.677 INFO:teuthology.orchestra.run.smithi148.stdout: chdir = "" 2024-09-21T07:38:40.677 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T07:38:40.677 INFO:teuthology.orchestra.run.smithi148.stdout: auth supported = cephx 2024-09-21T07:38:40.678 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:38:40.678 INFO:teuthology.orchestra.run.smithi148.stdout: filestore xattr use omap = true 2024-09-21T07:38:40.678 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:38:40.678 INFO:teuthology.orchestra.run.smithi148.stdout: mon clock drift allowed = 1.000 2024-09-21T07:38:40.678 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:38:40.678 INFO:teuthology.orchestra.run.smithi148.stdout: osd crush chooseleaf type = 0 2024-09-21T07:38:40.678 INFO:teuthology.orchestra.run.smithi148.stdout: auth debug = true 2024-09-21T07:38:40.678 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:38:40.678 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on old message = true 2024-09-21T07:38:40.678 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on bug = true 2024-09-21T07:38:40.678 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:38:40.678 INFO:teuthology.orchestra.run.smithi148.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T07:38:40.678 INFO:teuthology.orchestra.run.smithi148.stdout: mon pg warn max object skew = 0 2024-09-21T07:38:40.678 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:38:40.679 INFO:teuthology.orchestra.run.smithi148.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T07:38:40.679 INFO:teuthology.orchestra.run.smithi148.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T07:38:40.679 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:38:40.679 INFO:teuthology.orchestra.run.smithi148.stdout: osd pool default size = 2 2024-09-21T07:38:40.679 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:38:40.679 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd allow primary affinity = true 2024-09-21T07:38:40.679 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd allow pg remap = true 2024-09-21T07:38:40.679 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on legacy crush tunables = false 2024-09-21T07:38:40.679 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on crush straw calc version zero = false 2024-09-21T07:38:40.679 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on no sortbitwise = false 2024-09-21T07:38:40.679 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on osd down out interval zero = false 2024-09-21T07:38:40.679 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on too few osds = false 2024-09-21T07:38:40.679 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T07:38:40.679 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T07:38:40.680 INFO:teuthology.orchestra.run.smithi148.stdout: mon_allow_pool_size_one = true 2024-09-21T07:38:40.680 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:38:40.680 INFO:teuthology.orchestra.run.smithi148.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T07:38:40.680 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:38:40.680 INFO:teuthology.orchestra.run.smithi148.stdout: osd default data pool replay window = 5 2024-09-21T07:38:40.680 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:38:40.680 INFO:teuthology.orchestra.run.smithi148.stdout: mon allow pool delete = true 2024-09-21T07:38:40.680 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:38:40.680 INFO:teuthology.orchestra.run.smithi148.stdout: mon cluster log file level = debug 2024-09-21T07:38:40.680 INFO:teuthology.orchestra.run.smithi148.stdout: debug asserts on shutdown = true 2024-09-21T07:38:40.680 INFO:teuthology.orchestra.run.smithi148.stdout: mon health detail to clog = false 2024-09-21T07:38:40.680 INFO:teuthology.orchestra.run.smithi148.stdout: mon host = "172.21.15.39,172.21.15.148,[v2:172.21.15.148:3301,v1:172.21.15.148:6790]" 2024-09-21T07:38:40.680 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on skipped message = False 2024-09-21T07:38:40.680 INFO:teuthology.orchestra.run.smithi148.stdout: fsid = 7ab9b04a-f88d-49aa-a493-e5c076637ffb 2024-09-21T07:38:40.680 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:38:40.681 INFO:teuthology.orchestra.run.smithi148.stdout:[osd] 2024-09-21T07:38:40.681 INFO:teuthology.orchestra.run.smithi148.stdout: osd journal size = 100 2024-09-21T07:38:40.681 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:38:40.681 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub load threshold = 5.0 2024-09-21T07:38:40.681 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub max interval = 600 2024-09-21T07:38:40.681 INFO:teuthology.orchestra.run.smithi148.stdout: osd mclock profile = high_recovery_ops 2024-09-21T07:38:40.681 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:38:40.681 INFO:teuthology.orchestra.run.smithi148.stdout: osd recover clone overlap = true 2024-09-21T07:38:40.681 INFO:teuthology.orchestra.run.smithi148.stdout: osd recovery max chunk = 1048576 2024-09-21T07:38:40.681 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:38:40.681 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug shutdown = true 2024-09-21T07:38:40.681 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug op order = true 2024-09-21T07:38:40.681 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug verify stray on activate = true 2024-09-21T07:38:40.681 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug trim objects = true 2024-09-21T07:38:40.681 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:38:40.682 INFO:teuthology.orchestra.run.smithi148.stdout: osd open classes on start = true 2024-09-21T07:38:40.682 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug pg log writeout = true 2024-09-21T07:38:40.682 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:38:40.682 INFO:teuthology.orchestra.run.smithi148.stdout: osd deep scrub update digest min age = 30 2024-09-21T07:38:40.682 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:38:40.682 INFO:teuthology.orchestra.run.smithi148.stdout: osd map max advance = 10 2024-09-21T07:38:40.682 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:38:40.682 INFO:teuthology.orchestra.run.smithi148.stdout: journal zero on create = true 2024-09-21T07:38:40.682 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:38:40.682 INFO:teuthology.orchestra.run.smithi148.stdout: filestore ondisk finisher threads = 3 2024-09-21T07:38:40.682 INFO:teuthology.orchestra.run.smithi148.stdout: filestore apply finisher threads = 3 2024-09-21T07:38:40.682 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:38:40.682 INFO:teuthology.orchestra.run.smithi148.stdout: bdev debug aio = true 2024-09-21T07:38:40.682 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug misdirected ops = true 2024-09-21T07:38:40.682 INFO:teuthology.orchestra.run.smithi148.stdout: bdev async discard = True 2024-09-21T07:38:40.683 INFO:teuthology.orchestra.run.smithi148.stdout: bdev enable discard = True 2024-09-21T07:38:40.683 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore allocator = bitmap 2024-09-21T07:38:40.683 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore block size = 96636764160 2024-09-21T07:38:40.683 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore fsck on mount = True 2024-09-21T07:38:40.683 INFO:teuthology.orchestra.run.smithi148.stdout: debug bluefs = 1/20 2024-09-21T07:38:40.683 INFO:teuthology.orchestra.run.smithi148.stdout: debug bluestore = 1/20 2024-09-21T07:38:40.683 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-09-21T07:38:40.683 INFO:teuthology.orchestra.run.smithi148.stdout: debug osd = 20 2024-09-21T07:38:40.683 INFO:teuthology.orchestra.run.smithi148.stdout: debug rocksdb = 4/10 2024-09-21T07:38:40.683 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T07:38:40.683 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd full ratio = 0.9 2024-09-21T07:38:40.683 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd nearfull ratio = 0.8 2024-09-21T07:38:40.683 INFO:teuthology.orchestra.run.smithi148.stdout: osd failsafe full ratio = 0.95 2024-09-21T07:38:40.683 INFO:teuthology.orchestra.run.smithi148.stdout: osd objectstore = bluestore 2024-09-21T07:38:40.683 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:38:40.684 INFO:teuthology.orchestra.run.smithi148.stdout:[mgr] 2024-09-21T07:38:40.684 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-09-21T07:38:40.684 INFO:teuthology.orchestra.run.smithi148.stdout: debug mgr = 20 2024-09-21T07:38:40.684 INFO:teuthology.orchestra.run.smithi148.stdout: debug mon = 20 2024-09-21T07:38:40.684 INFO:teuthology.orchestra.run.smithi148.stdout: debug auth = 20 2024-09-21T07:38:40.684 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min pgs per osd = 4 2024-09-21T07:38:40.684 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min bytes per osd = 10 2024-09-21T07:38:40.684 INFO:teuthology.orchestra.run.smithi148.stdout: mgr/telemetry/nag = false 2024-09-21T07:38:40.684 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:38:40.684 INFO:teuthology.orchestra.run.smithi148.stdout:[mon] 2024-09-21T07:38:40.684 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-09-21T07:38:40.684 INFO:teuthology.orchestra.run.smithi148.stdout: debug mon = 20 2024-09-21T07:38:40.684 INFO:teuthology.orchestra.run.smithi148.stdout: debug paxos = 20 2024-09-21T07:38:40.684 INFO:teuthology.orchestra.run.smithi148.stdout: debug auth = 20 2024-09-21T07:38:40.685 INFO:teuthology.orchestra.run.smithi148.stdout: mon data avail warn = 5 2024-09-21T07:38:40.685 INFO:teuthology.orchestra.run.smithi148.stdout: mon mgr mkfs grace = 240 2024-09-21T07:38:40.685 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min pgs per osd = 4 2024-09-21T07:38:40.685 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd reporter subtree level = osd 2024-09-21T07:38:40.685 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd prime pg temp = true 2024-09-21T07:38:40.685 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min bytes per osd = 10 2024-09-21T07:38:40.685 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:38:40.685 INFO:teuthology.orchestra.run.smithi148.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T07:38:40.685 INFO:teuthology.orchestra.run.smithi148.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T07:38:40.685 INFO:teuthology.orchestra.run.smithi148.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T07:38:40.685 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:38:40.685 INFO:teuthology.orchestra.run.smithi148.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T07:38:40.685 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T07:38:40.685 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T07:38:40.685 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:38:40.686 INFO:teuthology.orchestra.run.smithi148.stdout: # 1m isn't quite enough 2024-09-21T07:38:40.686 INFO:teuthology.orchestra.run.smithi148.stdout: mon_down_mkfs_grace = 2m 2024-09-21T07:38:40.686 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:38:40.686 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_filestore_osds = false 2024-09-21T07:38:40.686 INFO:teuthology.orchestra.run.smithi148.stdout: mon down mkfs grace = 300 2024-09-21T07:38:40.686 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:38:40.686 INFO:teuthology.orchestra.run.smithi148.stdout:[client] 2024-09-21T07:38:40.686 INFO:teuthology.orchestra.run.smithi148.stdout: rgw cache enabled = true 2024-09-21T07:38:40.686 INFO:teuthology.orchestra.run.smithi148.stdout: rgw enable ops log = true 2024-09-21T07:38:40.686 INFO:teuthology.orchestra.run.smithi148.stdout: rgw enable usage log = true 2024-09-21T07:38:40.686 INFO:teuthology.orchestra.run.smithi148.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T07:38:40.686 INFO:teuthology.orchestra.run.smithi148.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T07:38:40.686 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = "" 2024-09-21T07:38:40.686 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.a] 2024-09-21T07:38:40.687 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.b] 2024-09-21T07:38:40.687 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.c] 2024-09-21T07:38:40.687 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.a] 2024-09-21T07:38:40.687 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.c] 2024-09-21T07:38:40.687 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.b] 2024-09-21T07:38:40.687 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.d] 2024-09-21T07:38:40.687 INFO:teuthology.orchestra.run.smithi148.stdout:[mds] 2024-09-21T07:38:40.687 INFO:teuthology.orchestra.run.smithi148.stdout: mds_reconnect_timeout = 5 2024-09-21T07:38:40.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:38:40.704 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir d1/dir/.snap/*) 2024-09-21T07:38:40.768 INFO:tasks.cephfs.test_snapshots:testing snapserver mds_kill_mdstable_at=2, also kill snapclient 2024-09-21T07:38:40.768 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:38:40.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.994+0000 7f4169600640 1 -- 172.21.15.39:0/16014312 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41640d0620 msgr2=0x7f41640d0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:40.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.994+0000 7f4169600640 1 --2- 172.21.15.39:0/16014312 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41640d0620 0x7f41640d0a30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f415c005fd0 tx=0x7f415c02a8b0 comp rx=0 tx=0).stop 2024-09-21T07:38:40.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.994+0000 7f4169600640 1 -- 172.21.15.39:0/16014312 shutdown_connections 2024-09-21T07:38:40.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.994+0000 7f4169600640 1 --2- 172.21.15.39:0/16014312 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41640d0620 0x7f41640d0a30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:40.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.994+0000 7f4169600640 1 --2- 172.21.15.39:0/16014312 >> v2:172.21.15.148:3300/0 conn(0x7f41640cfd10 0x7f41640cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:40.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.994+0000 7f4169600640 1 -- 172.21.15.39:0/16014312 >> 172.21.15.39:0/16014312 conn(0x7f416406cfc0 msgr2=0x7f416406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:40.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.994+0000 7f4169600640 1 -- 172.21.15.39:0/16014312 shutdown_connections 2024-09-21T07:38:40.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.994+0000 7f4169600640 1 -- 172.21.15.39:0/16014312 wait complete. 2024-09-21T07:38:40.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.996+0000 7f4169600640 1 Processor -- start 2024-09-21T07:38:40.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.996+0000 7f4169600640 1 -- start start 2024-09-21T07:38:41.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.998+0000 7f4169600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41640cfd10 0x7f41641f5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:41.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.998+0000 7f4169600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41640d0620 0x7f41640b4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:41.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.998+0000 7f4169600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f416415cd30 0x7f41641e8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:41.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.998+0000 7f4169600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4164073890 con 0x7f416415cd30 2024-09-21T07:38:41.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.998+0000 7f4169600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4164073710 con 0x7f41640d0620 2024-09-21T07:38:41.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.998+0000 7f4169600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f4164073a10 con 0x7f41640cfd10 2024-09-21T07:38:41.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.998+0000 7f4163400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f416415cd30 0x7f41641e8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:41.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.998+0000 7f4163400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f416415cd30 0x7f41641e8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50034/0 (socket says 172.21.15.39:50034) 2024-09-21T07:38:41.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.998+0000 7f4163400640 1 -- 172.21.15.39:0/2801511018 learned_addr learned my addr 172.21.15.39:0/2801511018 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:41.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.998+0000 7f4162a00640 1 --2- 172.21.15.39:0/2801511018 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41640cfd10 0x7f41641f5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:41.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.998+0000 7f415be00640 1 --2- 172.21.15.39:0/2801511018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41640d0620 0x7f41640b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:41.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.999+0000 7f4163400640 1 -- 172.21.15.39:0/2801511018 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41640cfd10 msgr2=0x7f41641f5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:41.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.999+0000 7f4163400640 1 --2- 172.21.15.39:0/2801511018 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41640cfd10 0x7f41641f5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:41.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.999+0000 7f4163400640 1 -- 172.21.15.39:0/2801511018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41640d0620 msgr2=0x7f41640b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:41.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.999+0000 7f4163400640 1 --2- 172.21.15.39:0/2801511018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41640d0620 0x7f41640b4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:41.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.999+0000 7f4163400640 1 -- 172.21.15.39:0/2801511018 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41641e7780 con 0x7f416415cd30 2024-09-21T07:38:41.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.999+0000 7f4163400640 1 --2- 172.21.15.39:0/2801511018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f416415cd30 0x7f41641e8820 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f415c02aea0 tx=0x7f415c031670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:41.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.999+0000 7f415aa00640 1 -- 172.21.15.39:0/2801511018 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415c031e60 con 0x7f416415cd30 2024-09-21T07:38:41.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.999+0000 7f415aa00640 1 -- 172.21.15.39:0/2801511018 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f415c003a80 con 0x7f416415cd30 2024-09-21T07:38:41.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.999+0000 7f415aa00640 1 -- 172.21.15.39:0/2801511018 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415c033070 con 0x7f416415cd30 2024-09-21T07:38:41.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.999+0000 7f4169600640 1 -- 172.21.15.39:0/2801511018 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41641e7a60 con 0x7f416415cd30 2024-09-21T07:38:41.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:40.999+0000 7f4169600640 1 -- 172.21.15.39:0/2801511018 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41641e8020 con 0x7f416415cd30 2024-09-21T07:38:41.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.001+0000 7f4169600640 1 -- 172.21.15.39:0/2801511018 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4164073f50 con 0x7f416415cd30 2024-09-21T07:38:41.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.002+0000 7f415aa00640 1 -- 172.21.15.39:0/2801511018 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f415c033210 con 0x7f416415cd30 2024-09-21T07:38:41.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.003+0000 7f415aa00640 1 --2- 172.21.15.39:0/2801511018 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4140078b90 0x7f414007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:41.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.003+0000 7f415aa00640 1 -- 172.21.15.39:0/2801511018 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 7173+0+0 (secure 0 0 0) 0x7f415c0bc590 con 0x7f416415cd30 2024-09-21T07:38:41.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.003+0000 7f4162a00640 1 --2- 172.21.15.39:0/2801511018 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4140078b90 0x7f414007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:41.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.004+0000 7f4162a00640 1 --2- 172.21.15.39:0/2801511018 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4140078b90 0x7f414007b0a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4154001710 tx=0x7f4154007c60 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:41.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.007+0000 7f415aa00640 1 -- 172.21.15.39:0/2801511018 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f415c03f030 con 0x7f416415cd30 2024-09-21T07:38:41.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.247+0000 7f4169600640 1 -- 172.21.15.39:0/2801511018 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41640cc810 con 0x7f416415cd30 2024-09-21T07:38:41.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.254+0000 7f415aa00640 1 -- 172.21.15.39:0/2801511018 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5243 (secure 0 0 0) 0x7f415c083e60 con 0x7f416415cd30 2024-09-21T07:38:41.257 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:41.257 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":90,"btime":"2024-09-21T07:38:36:232466+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/992188984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":992188984},{"type":"v1","addr":"172.21.15.148:6837","nonce":992188984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:38:36.232454+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6825,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6825":{"gid":6825,"name":"c","rank":0,"incarnation":86,"state":"up:active","state_seq":13,"addr":"172.21.15.39:6834/855150775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":855150775},{"type":"v1","addr":"172.21.15.39:6834","nonce":855150775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6825,"qdb_cluster":[6115,6825]},"id":6}]} 2024-09-21T07:38:41.257 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 90 2024-09-21T07:38:41.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.257+0000 7f4169600640 1 -- 172.21.15.39:0/2801511018 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4140078b90 msgr2=0x7f414007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:41.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.257+0000 7f4169600640 1 --2- 172.21.15.39:0/2801511018 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4140078b90 0x7f414007b0a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4154001710 tx=0x7f4154007c60 comp rx=0 tx=0).stop 2024-09-21T07:38:41.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.257+0000 7f4169600640 1 -- 172.21.15.39:0/2801511018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f416415cd30 msgr2=0x7f41641e8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:41.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.257+0000 7f4169600640 1 --2- 172.21.15.39:0/2801511018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f416415cd30 0x7f41641e8820 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f415c02aea0 tx=0x7f415c031670 comp rx=0 tx=0).stop 2024-09-21T07:38:41.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.257+0000 7f4169600640 1 -- 172.21.15.39:0/2801511018 shutdown_connections 2024-09-21T07:38:41.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.257+0000 7f4169600640 1 --2- 172.21.15.39:0/2801511018 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4140078b90 0x7f414007b0a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:41.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.257+0000 7f4169600640 1 --2- 172.21.15.39:0/2801511018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f416415cd30 0x7f41641e8820 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:41.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.257+0000 7f4169600640 1 --2- 172.21.15.39:0/2801511018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41640d0620 0x7f41640b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:41.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.257+0000 7f4169600640 1 --2- 172.21.15.39:0/2801511018 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41640cfd10 0x7f41641f5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:41.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.257+0000 7f4169600640 1 -- 172.21.15.39:0/2801511018 >> 172.21.15.39:0/2801511018 conn(0x7f416406cfc0 msgr2=0x7f41640b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:41.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.257+0000 7f4169600640 1 -- 172.21.15.39:0/2801511018 shutdown_connections 2024-09-21T07:38:41.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.257+0000 7f4169600640 1 -- 172.21.15.39:0/2801511018 wait complete. 2024-09-21T07:38:41.280 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2024-09-21T07:38:41.524 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_snap":9,"last_created":5,"last_destroyed":8,"pending_noop":[],"snaps":[{"snapid":5,"ino":1099511627777,"stamp":"2024-09-21T07:38:08.632985+0000","name":"s16","metadata":{}}],"need_to_purge":{"16":[2,3,4,6,7,8]},"pending_update":[],"pending_destroy":[{"version":16,"removed_snap":5,"seq":9}]} 2024-09-21T07:38:41.524 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 5, "last_destroyed": 8, "last_snap": 9, "need_to_purge": { "16": [ 2, 3, 4, 6, 7, 8 ] }, "pending_destroy": [ { "removed_snap": 5, "seq": 9, "version": 16 } ], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s16", "snapid": 5, "stamp": "2024-09-21T07:38:08.632985+0000" } ] } 2024-09-21T07:38:41.524 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:0 true 2024-09-21T07:38:41.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.750+0000 7fde51200640 1 -- 172.21.15.39:0/1198216602 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde4c0ce910 msgr2=0x7fde4c0caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:41.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.750+0000 7fde51200640 1 --2- 172.21.15.39:0/1198216602 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde4c0ce910 0x7fde4c0caad0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fde4c072f30 tx=0x7fde4402aac0 comp rx=0 tx=0).stop 2024-09-21T07:38:41.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.750+0000 7fde51200640 1 -- 172.21.15.39:0/1198216602 shutdown_connections 2024-09-21T07:38:41.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.750+0000 7fde51200640 1 --2- 172.21.15.39:0/1198216602 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde4c0ce910 0x7fde4c0caad0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:41.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.750+0000 7fde51200640 1 --2- 172.21.15.39:0/1198216602 >> v2:172.21.15.148:3300/0 conn(0x7fde4c0b3c90 0x7fde4c0cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:41.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.750+0000 7fde51200640 1 -- 172.21.15.39:0/1198216602 >> 172.21.15.39:0/1198216602 conn(0x7fde4c0b25f0 msgr2=0x7fde4c0d01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:41.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.750+0000 7fde51200640 1 -- 172.21.15.39:0/1198216602 shutdown_connections 2024-09-21T07:38:41.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.750+0000 7fde51200640 1 -- 172.21.15.39:0/1198216602 wait complete. 2024-09-21T07:38:41.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.752+0000 7fde51200640 1 Processor -- start 2024-09-21T07:38:41.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.752+0000 7fde51200640 1 -- start start 2024-09-21T07:38:41.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.754+0000 7fde51200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde4c0b3c90 0x7fde4c1f4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:41.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.754+0000 7fde51200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde4c0ce110 0x7fde4c0c6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:41.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.754+0000 7fde51200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde4c0ce910 0x7fde4c0ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:41.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.754+0000 7fde51200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fde4c0cb510 con 0x7fde4c0ce110 2024-09-21T07:38:41.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.754+0000 7fde51200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fde4c0cb390 con 0x7fde4c0ce910 2024-09-21T07:38:41.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.754+0000 7fde51200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fde4c0cb690 con 0x7fde4c0b3c90 2024-09-21T07:38:41.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.754+0000 7fde43e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde4c0ce110 0x7fde4c0c6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:41.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.754+0000 7fde43e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde4c0ce110 0x7fde4c0c6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50048/0 (socket says 172.21.15.39:50048) 2024-09-21T07:38:41.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.754+0000 7fde43e00640 1 -- 172.21.15.39:0/3845668001 learned_addr learned my addr 172.21.15.39:0/3845668001 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:41.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.754+0000 7fde4be00640 1 --2- 172.21.15.39:0/3845668001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde4c0ce910 0x7fde4c0ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:41.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.754+0000 7fde4b400640 1 --2- 172.21.15.39:0/3845668001 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde4c0b3c90 0x7fde4c1f4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:41.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.754+0000 7fde43e00640 1 -- 172.21.15.39:0/3845668001 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde4c0b3c90 msgr2=0x7fde4c1f4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:41.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.754+0000 7fde43e00640 1 --2- 172.21.15.39:0/3845668001 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde4c0b3c90 0x7fde4c1f4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:41.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.754+0000 7fde43e00640 1 -- 172.21.15.39:0/3845668001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde4c0ce910 msgr2=0x7fde4c0ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:41.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.754+0000 7fde43e00640 1 --2- 172.21.15.39:0/3845668001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde4c0ce910 0x7fde4c0ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:41.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.754+0000 7fde43e00640 1 -- 172.21.15.39:0/3845668001 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde4c1f5480 con 0x7fde4c0ce110 2024-09-21T07:38:41.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.754+0000 7fde4b400640 1 --2- 172.21.15.39:0/3845668001 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde4c0b3c90 0x7fde4c1f4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:38:41.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.755+0000 7fde43e00640 1 --2- 172.21.15.39:0/3845668001 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde4c0ce110 0x7fde4c0c6730 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fde3400d6d0 tx=0x7fde3400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:41.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.755+0000 7fde43400640 1 -- 172.21.15.39:0/3845668001 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde34017070 con 0x7fde4c0ce110 2024-09-21T07:38:41.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.755+0000 7fde43400640 1 -- 172.21.15.39:0/3845668001 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde34004310 con 0x7fde4c0ce110 2024-09-21T07:38:41.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.755+0000 7fde43400640 1 -- 172.21.15.39:0/3845668001 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde34009a80 con 0x7fde4c0ce110 2024-09-21T07:38:41.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.755+0000 7fde51200640 1 -- 172.21.15.39:0/3845668001 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde4c1f9140 con 0x7fde4c0ce110 2024-09-21T07:38:41.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.755+0000 7fde51200640 1 -- 172.21.15.39:0/3845668001 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde4c1f9700 con 0x7fde4c0ce110 2024-09-21T07:38:41.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.756+0000 7fde51200640 1 -- 172.21.15.39:0/3845668001 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde4c0cbb40 con 0x7fde4c0ce110 2024-09-21T07:38:41.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.757+0000 7fde43400640 1 -- 172.21.15.39:0/3845668001 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fde340035f0 con 0x7fde4c0ce110 2024-09-21T07:38:41.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.758+0000 7fde43400640 1 --2- 172.21.15.39:0/3845668001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fde28078b90 0x7fde2807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:41.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.758+0000 7fde43400640 1 -- 172.21.15.39:0/3845668001 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 7173+0+0 (secure 0 0 0) 0x7fde3409b5e0 con 0x7fde4c0ce110 2024-09-21T07:38:41.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.759+0000 7fde4b400640 1 --2- 172.21.15.39:0/3845668001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fde28078b90 0x7fde2807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:41.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.760+0000 7fde4b400640 1 --2- 172.21.15.39:0/3845668001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fde28078b90 0x7fde2807b0a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fde3c000f60 tx=0x7fde3c009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:41.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:41.764+0000 7fde43400640 1 -- 172.21.15.39:0/3845668001 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fde34063450 con 0x7fde4c0ce110 2024-09-21T07:38:42.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.034+0000 7fde51200640 1 -- 172.21.15.39:0/3845668001 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"} v 0) -- 0x7fde4c06ffa0 con 0x7fde4c0ce110 2024-09-21T07:38:42.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.257+0000 7fde43400640 1 -- 172.21.15.39:0/3845668001 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"}]=0 freezing mds.6825 v91) ==== 112+0+0 (secure 0 0 0) 0x7fde34062d50 con 0x7fde4c0ce110 2024-09-21T07:38:42.259 INFO:teuthology.orchestra.run.smithi039.stderr:freezing mds.6825 2024-09-21T07:38:42.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.260+0000 7fde51200640 1 -- 172.21.15.39:0/3845668001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fde28078b90 msgr2=0x7fde2807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:42.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.260+0000 7fde51200640 1 --2- 172.21.15.39:0/3845668001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fde28078b90 0x7fde2807b0a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fde3c000f60 tx=0x7fde3c009040 comp rx=0 tx=0).stop 2024-09-21T07:38:42.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.260+0000 7fde51200640 1 -- 172.21.15.39:0/3845668001 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde4c0ce110 msgr2=0x7fde4c0c6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:42.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.260+0000 7fde51200640 1 --2- 172.21.15.39:0/3845668001 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde4c0ce110 0x7fde4c0c6730 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fde3400d6d0 tx=0x7fde3400db90 comp rx=0 tx=0).stop 2024-09-21T07:38:42.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.260+0000 7fde51200640 1 -- 172.21.15.39:0/3845668001 shutdown_connections 2024-09-21T07:38:42.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.260+0000 7fde51200640 1 --2- 172.21.15.39:0/3845668001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fde28078b90 0x7fde2807b0a0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:42.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.260+0000 7fde51200640 1 --2- 172.21.15.39:0/3845668001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde4c0ce910 0x7fde4c0ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:42.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.260+0000 7fde51200640 1 --2- 172.21.15.39:0/3845668001 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde4c0ce110 0x7fde4c0c6730 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:42.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.260+0000 7fde51200640 1 --2- 172.21.15.39:0/3845668001 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde4c0b3c90 0x7fde4c1f4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:42.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.260+0000 7fde51200640 1 -- 172.21.15.39:0/3845668001 >> 172.21.15.39:0/3845668001 conn(0x7fde4c0b25f0 msgr2=0x7fde4c0b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:42.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.260+0000 7fde51200640 1 -- 172.21.15.39:0/3845668001 shutdown_connections 2024-09-21T07:38:42.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.260+0000 7fde51200640 1 -- 172.21.15.39:0/3845668001 wait complete. 2024-09-21T07:38:42.288 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:1 true 2024-09-21T07:38:42.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.517+0000 7f635e000640 1 -- 172.21.15.39:0/1763805310 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f633c004d90 msgr2=0x7f633c005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:42.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.517+0000 7f635e000640 1 --2- 172.21.15.39:0/1763805310 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f633c004d90 0x7f633c005220 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f6350001f30 tx=0x7f633c005830 comp rx=0 tx=0).stop 2024-09-21T07:38:42.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.517+0000 7f6357e00640 1 -- 172.21.15.39:0/1763805310 reap_dead start 2024-09-21T07:38:42.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.517+0000 7f635e000640 1 -- 172.21.15.39:0/1763805310 shutdown_connections 2024-09-21T07:38:42.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.517+0000 7f635e000640 1 -- 172.21.15.39:0/1763805310 >> 172.21.15.39:0/1763805310 conn(0x7f635806d440 msgr2=0x7f63580b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:42.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.517+0000 7f635e000640 1 -- 172.21.15.39:0/1763805310 shutdown_connections 2024-09-21T07:38:42.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.517+0000 7f635e000640 1 -- 172.21.15.39:0/1763805310 wait complete. 2024-09-21T07:38:42.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.520+0000 7f635e000640 1 Processor -- start 2024-09-21T07:38:42.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.520+0000 7f635e000640 1 -- start start 2024-09-21T07:38:42.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.521+0000 7f635e000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f633c004d90 0x7f63581f6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:42.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.521+0000 7f635e000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f635815a880 0x7f63580b1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:42.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.521+0000 7f635e000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f635815ac90 0x7f63580736d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:42.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.521+0000 7f635e000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f63580b47a0 con 0x7f633c004d90 2024-09-21T07:38:42.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.521+0000 7f635e000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f63580cc5b0 con 0x7f635815a880 2024-09-21T07:38:42.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.521+0000 7f635e000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f63580b4920 con 0x7f635815ac90 2024-09-21T07:38:42.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.521+0000 7f6357400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f633c004d90 0x7f63581f6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:42.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.522+0000 7f6357400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f633c004d90 0x7f63581f6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47366/0 (socket says 172.21.15.39:47366) 2024-09-21T07:38:42.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.522+0000 7f6357400640 1 -- 172.21.15.39:0/4285963425 learned_addr learned my addr 172.21.15.39:0/4285963425 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:42.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.522+0000 7f6357e00640 1 --2- 172.21.15.39:0/4285963425 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f635815ac90 0x7f63580736d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:42.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.522+0000 7f634ea00640 1 --2- 172.21.15.39:0/4285963425 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f635815a880 0x7f63580b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:42.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.522+0000 7f6357400640 1 -- 172.21.15.39:0/4285963425 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f635815ac90 msgr2=0x7f63580736d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:42.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.522+0000 7f6357400640 1 --2- 172.21.15.39:0/4285963425 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f635815ac90 0x7f63580736d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:42.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.522+0000 7f6357400640 1 -- 172.21.15.39:0/4285963425 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f635815a880 msgr2=0x7f63580b1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:42.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.522+0000 7f6357400640 1 --2- 172.21.15.39:0/4285963425 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f635815a880 0x7f63580b1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:42.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.522+0000 7f6357400640 1 -- 172.21.15.39:0/4285963425 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6358159b00 con 0x7f633c004d90 2024-09-21T07:38:42.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.522+0000 7f634ea00640 1 --2- 172.21.15.39:0/4285963425 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f635815a880 0x7f63580b1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:38:42.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.522+0000 7f6357400640 1 --2- 172.21.15.39:0/4285963425 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f633c004d90 0x7f63581f6140 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f6348011a60 tx=0x7f6348011f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:42.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.523+0000 7f634fe00640 1 -- 172.21.15.39:0/4285963425 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634801e070 con 0x7f633c004d90 2024-09-21T07:38:42.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.523+0000 7f634fe00640 1 -- 172.21.15.39:0/4285963425 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f634800fe70 con 0x7f633c004d90 2024-09-21T07:38:42.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.523+0000 7f634fe00640 1 -- 172.21.15.39:0/4285963425 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6348004290 con 0x7f633c004d90 2024-09-21T07:38:42.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.523+0000 7f635e000640 1 -- 172.21.15.39:0/4285963425 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63581f6680 con 0x7f633c004d90 2024-09-21T07:38:42.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.523+0000 7f635e000640 1 -- 172.21.15.39:0/4285963425 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f635815a080 con 0x7f633c004d90 2024-09-21T07:38:42.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.524+0000 7f635e000640 1 -- 172.21.15.39:0/4285963425 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63580b4e80 con 0x7f633c004d90 2024-09-21T07:38:42.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.525+0000 7f634fe00640 1 -- 172.21.15.39:0/4285963425 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6348021070 con 0x7f633c004d90 2024-09-21T07:38:42.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.526+0000 7f634fe00640 1 --2- 172.21.15.39:0/4285963425 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6334078b90 0x7f633407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:42.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.526+0000 7f634fe00640 1 -- 172.21.15.39:0/4285963425 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 7173+0+0 (secure 0 0 0) 0x7f634809beb0 con 0x7f633c004d90 2024-09-21T07:38:42.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.526+0000 7f634ea00640 1 --2- 172.21.15.39:0/4285963425 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6334078b90 0x7f633407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:42.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.527+0000 7f634ea00640 1 --2- 172.21.15.39:0/4285963425 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6334078b90 0x7f633407b0a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f63580733c0 tx=0x7f6340000ed0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:42.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.531+0000 7f634fe00640 1 -- 172.21.15.39:0/4285963425 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6348063680 con 0x7f633c004d90 2024-09-21T07:38:42.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:42.805+0000 7f635e000640 1 -- 172.21.15.39:0/4285963425 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:1", "val": "true"} v 0) -- 0x7f6358069010 con 0x7f633c004d90 2024-09-21T07:38:43.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:43.270+0000 7f634fe00640 1 -- 172.21.15.39:0/4285963425 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:1", "val": "true"}]=0 freezing mds.6115 v92) ==== 112+0+0 (secure 0 0 0) 0x7f63480680b0 con 0x7f633c004d90 2024-09-21T07:38:43.273 INFO:teuthology.orchestra.run.smithi039.stderr:freezing mds.6115 2024-09-21T07:38:43.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:43.273+0000 7f635e000640 1 -- 172.21.15.39:0/4285963425 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6334078b90 msgr2=0x7f633407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:43.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:43.273+0000 7f635e000640 1 --2- 172.21.15.39:0/4285963425 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6334078b90 0x7f633407b0a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f63580733c0 tx=0x7f6340000ed0 comp rx=0 tx=0).stop 2024-09-21T07:38:43.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:43.273+0000 7f635e000640 1 -- 172.21.15.39:0/4285963425 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f633c004d90 msgr2=0x7f63581f6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:43.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:43.273+0000 7f635e000640 1 --2- 172.21.15.39:0/4285963425 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f633c004d90 0x7f63581f6140 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f6348011a60 tx=0x7f6348011f20 comp rx=0 tx=0).stop 2024-09-21T07:38:43.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:43.273+0000 7f635e000640 1 -- 172.21.15.39:0/4285963425 shutdown_connections 2024-09-21T07:38:43.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:43.273+0000 7f635e000640 1 --2- 172.21.15.39:0/4285963425 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6334078b90 0x7f633407b0a0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:43.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:43.273+0000 7f635e000640 1 --2- 172.21.15.39:0/4285963425 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f635815ac90 0x7f63580736d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:43.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:43.273+0000 7f635e000640 1 --2- 172.21.15.39:0/4285963425 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f635815a880 0x7f63580b1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:43.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:43.273+0000 7f635e000640 1 --2- 172.21.15.39:0/4285963425 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f633c004d90 0x7f63581f6140 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:43.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:43.273+0000 7f635e000640 1 -- 172.21.15.39:0/4285963425 >> 172.21.15.39:0/4285963425 conn(0x7f635806d440 msgr2=0x7f63581e47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:43.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:43.273+0000 7f635e000640 1 -- 172.21.15.39:0/4285963425 shutdown_connections 2024-09-21T07:38:43.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:43.273+0000 7f635e000640 1 -- 172.21.15.39:0/4285963425 wait complete. 2024-09-21T07:38:43.297 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json config set mds_kill_mdstable_at 2 2024-09-21T07:38:43.539 INFO:teuthology.orchestra.run.smithi039.stdout:{"success":"mds_kill_mdstable_at = '' (not observed, change may require restart) "} 2024-09-21T07:38:43.540 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_kill_mdstable_at = '' (not observed, change may require restart) " } 2024-09-21T07:38:43.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:38:43.540 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d1/dir/.snap/s22) 2024-09-21T07:38:43.544 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-09-21T07:38:43.578 INFO:tasks.ceph.mds.c.smithi039.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_prepare_logged(ceph::cref_t&, version_t)' thread 7f6c16e00640 time 2024-09-21T07:38:43.578187+0000 2024-09-21T07:38:43.578 INFO:tasks.ceph.mds.c.smithi039.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: 77: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 2) 2024-09-21T07:38:43.582 INFO:tasks.ceph.mds.c.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:38:43.582 INFO:tasks.ceph.mds.c.smithi039.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f6c2605654b] 2024-09-21T07:38:43.583 INFO:tasks.ceph.mds.c.smithi039.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f6c26056758] 2024-09-21T07:38:43.583 INFO:tasks.ceph.mds.c.smithi039.stderr: 3: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x396) [0x5646f105a7e4] 2024-09-21T07:38:43.583 INFO:tasks.ceph.mds.c.smithi039.stderr: 4: ceph-mds(+0x47ac3e) [0x5646f105ac3e] 2024-09-21T07:38:43.583 INFO:tasks.ceph.mds.c.smithi039.stderr: 5: (MDSContext::complete(int)+0x7c) [0x5646f10a76d8] 2024-09-21T07:38:43.583 INFO:tasks.ceph.mds.c.smithi039.stderr: 6: (MDSIOContextBase::complete(int)+0x501) [0x5646f10a7d57] 2024-09-21T07:38:43.583 INFO:tasks.ceph.mds.c.smithi039.stderr: 7: (MDSLogContextBase::complete(int)+0x46) [0x5646f10a7dd4] 2024-09-21T07:38:43.583 INFO:tasks.ceph.mds.c.smithi039.stderr: 8: (Finisher::finisher_thread_entry()+0x495) [0x7f6c25fde34b] 2024-09-21T07:38:43.583 INFO:tasks.ceph.mds.c.smithi039.stderr: 9: ceph-mds(+0x1052e1) [0x5646f0ce52e1] 2024-09-21T07:38:43.583 INFO:tasks.ceph.mds.c.smithi039.stderr: 10: (Thread::entry_wrapper()+0x43) [0x7f6c2602fd57] 2024-09-21T07:38:43.583 INFO:tasks.ceph.mds.c.smithi039.stderr: 11: (Thread::_entry_func(void*)+0xd) [0x7f6c2602fd73] 2024-09-21T07:38:43.583 INFO:tasks.ceph.mds.c.smithi039.stderr: 12: /lib64/libc.so.6(+0x89c52) [0x7f6c25689c52] 2024-09-21T07:38:43.584 INFO:tasks.ceph.mds.c.smithi039.stderr: 13: /lib64/libc.so.6(+0x10ec80) [0x7f6c2570ec80] 2024-09-21T07:38:43.584 INFO:tasks.ceph.mds.c.smithi039.stderr:2024-09-21T07:38:43.577+0000 7f6c16e00640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_prepare_logged(ceph::cref_t&, version_t)' thread 7f6c16e00640 time 2024-09-21T07:38:43.578187+0000 2024-09-21T07:38:43.584 INFO:tasks.ceph.mds.c.smithi039.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: 77: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 2) 2024-09-21T07:38:43.584 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:38:43.584 INFO:tasks.ceph.mds.c.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:38:43.584 INFO:tasks.ceph.mds.c.smithi039.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f6c2605654b] 2024-09-21T07:38:43.584 INFO:tasks.ceph.mds.c.smithi039.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f6c26056758] 2024-09-21T07:38:43.584 INFO:tasks.ceph.mds.c.smithi039.stderr: 3: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x396) [0x5646f105a7e4] 2024-09-21T07:38:43.584 INFO:tasks.ceph.mds.c.smithi039.stderr: 4: ceph-mds(+0x47ac3e) [0x5646f105ac3e] 2024-09-21T07:38:43.584 INFO:tasks.ceph.mds.c.smithi039.stderr: 5: (MDSContext::complete(int)+0x7c) [0x5646f10a76d8] 2024-09-21T07:38:43.584 INFO:tasks.ceph.mds.c.smithi039.stderr: 6: (MDSIOContextBase::complete(int)+0x501) [0x5646f10a7d57] 2024-09-21T07:38:43.584 INFO:tasks.ceph.mds.c.smithi039.stderr: 7: (MDSLogContextBase::complete(int)+0x46) [0x5646f10a7dd4] 2024-09-21T07:38:43.585 INFO:tasks.ceph.mds.c.smithi039.stderr: 8: (Finisher::finisher_thread_entry()+0x495) [0x7f6c25fde34b] 2024-09-21T07:38:43.585 INFO:tasks.ceph.mds.c.smithi039.stderr: 9: ceph-mds(+0x1052e1) [0x5646f0ce52e1] 2024-09-21T07:38:43.585 INFO:tasks.ceph.mds.c.smithi039.stderr: 10: (Thread::entry_wrapper()+0x43) [0x7f6c2602fd57] 2024-09-21T07:38:43.585 INFO:tasks.ceph.mds.c.smithi039.stderr: 11: (Thread::_entry_func(void*)+0xd) [0x7f6c2602fd73] 2024-09-21T07:38:43.585 INFO:tasks.ceph.mds.c.smithi039.stderr: 12: /lib64/libc.so.6(+0x89c52) [0x7f6c25689c52] 2024-09-21T07:38:43.585 INFO:tasks.ceph.mds.c.smithi039.stderr: 13: /lib64/libc.so.6(+0x10ec80) [0x7f6c2570ec80] 2024-09-21T07:38:43.585 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:38:43.585 INFO:tasks.ceph.mds.c.smithi039.stderr:*** Caught signal (Aborted) ** 2024-09-21T07:38:43.585 INFO:tasks.ceph.mds.c.smithi039.stderr: in thread 7f6c16e00640 thread_name:MR_Finisher 2024-09-21T07:38:43.585 INFO:tasks.ceph.mds.c.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:38:43.585 INFO:tasks.ceph.mds.c.smithi039.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f6c2563e6f0] 2024-09-21T07:38:43.585 INFO:tasks.ceph.mds.c.smithi039.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f6c2568b99c] 2024-09-21T07:38:43.586 INFO:tasks.ceph.mds.c.smithi039.stderr: 3: raise() 2024-09-21T07:38:43.586 INFO:tasks.ceph.mds.c.smithi039.stderr: 4: abort() 2024-09-21T07:38:43.586 INFO:tasks.ceph.mds.c.smithi039.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7f6c26056686] 2024-09-21T07:38:43.586 INFO:tasks.ceph.mds.c.smithi039.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f6c26056758] 2024-09-21T07:38:43.586 INFO:tasks.ceph.mds.c.smithi039.stderr: 7: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x396) [0x5646f105a7e4] 2024-09-21T07:38:43.586 INFO:tasks.ceph.mds.c.smithi039.stderr: 8: ceph-mds(+0x47ac3e) [0x5646f105ac3e] 2024-09-21T07:38:43.586 INFO:tasks.ceph.mds.c.smithi039.stderr: 9: (MDSContext::complete(int)+0x7c) [0x5646f10a76d8] 2024-09-21T07:38:43.586 INFO:tasks.ceph.mds.c.smithi039.stderr: 10: (MDSIOContextBase::complete(int)+0x501) [0x5646f10a7d57] 2024-09-21T07:38:43.586 INFO:tasks.ceph.mds.c.smithi039.stderr: 11: (MDSLogContextBase::complete(int)+0x46) [0x5646f10a7dd4] 2024-09-21T07:38:43.586 INFO:tasks.ceph.mds.c.smithi039.stderr: 12: (Finisher::finisher_thread_entry()+0x495) [0x7f6c25fde34b] 2024-09-21T07:38:43.586 INFO:tasks.ceph.mds.c.smithi039.stderr: 13: ceph-mds(+0x1052e1) [0x5646f0ce52e1] 2024-09-21T07:38:43.586 INFO:tasks.ceph.mds.c.smithi039.stderr: 14: (Thread::entry_wrapper()+0x43) [0x7f6c2602fd57] 2024-09-21T07:38:43.587 INFO:tasks.ceph.mds.c.smithi039.stderr: 15: (Thread::_entry_func(void*)+0xd) [0x7f6c2602fd73] 2024-09-21T07:38:43.587 INFO:tasks.ceph.mds.c.smithi039.stderr: 16: /lib64/libc.so.6(+0x89c52) [0x7f6c25689c52] 2024-09-21T07:38:43.587 INFO:tasks.ceph.mds.c.smithi039.stderr: 17: /lib64/libc.so.6(+0x10ec80) [0x7f6c2570ec80] 2024-09-21T07:38:43.587 INFO:tasks.ceph.mds.c.smithi039.stderr:2024-09-21T07:38:43.579+0000 7f6c16e00640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:38:43.587 INFO:tasks.ceph.mds.c.smithi039.stderr: in thread 7f6c16e00640 thread_name:MR_Finisher 2024-09-21T07:38:43.587 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:38:43.587 INFO:tasks.ceph.mds.c.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:38:43.587 INFO:tasks.ceph.mds.c.smithi039.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f6c2563e6f0] 2024-09-21T07:38:43.587 INFO:tasks.ceph.mds.c.smithi039.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f6c2568b99c] 2024-09-21T07:38:43.587 INFO:tasks.ceph.mds.c.smithi039.stderr: 3: raise() 2024-09-21T07:38:43.587 INFO:tasks.ceph.mds.c.smithi039.stderr: 4: abort() 2024-09-21T07:38:43.587 INFO:tasks.ceph.mds.c.smithi039.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7f6c26056686] 2024-09-21T07:38:43.588 INFO:tasks.ceph.mds.c.smithi039.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f6c26056758] 2024-09-21T07:38:43.588 INFO:tasks.ceph.mds.c.smithi039.stderr: 7: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x396) [0x5646f105a7e4] 2024-09-21T07:38:43.588 INFO:tasks.ceph.mds.c.smithi039.stderr: 8: ceph-mds(+0x47ac3e) [0x5646f105ac3e] 2024-09-21T07:38:43.588 INFO:tasks.ceph.mds.c.smithi039.stderr: 9: (MDSContext::complete(int)+0x7c) [0x5646f10a76d8] 2024-09-21T07:38:43.588 INFO:tasks.ceph.mds.c.smithi039.stderr: 10: (MDSIOContextBase::complete(int)+0x501) [0x5646f10a7d57] 2024-09-21T07:38:43.588 INFO:tasks.ceph.mds.c.smithi039.stderr: 11: (MDSLogContextBase::complete(int)+0x46) [0x5646f10a7dd4] 2024-09-21T07:38:43.588 INFO:tasks.ceph.mds.c.smithi039.stderr: 12: (Finisher::finisher_thread_entry()+0x495) [0x7f6c25fde34b] 2024-09-21T07:38:43.588 INFO:tasks.ceph.mds.c.smithi039.stderr: 13: ceph-mds(+0x1052e1) [0x5646f0ce52e1] 2024-09-21T07:38:43.588 INFO:tasks.ceph.mds.c.smithi039.stderr: 14: (Thread::entry_wrapper()+0x43) [0x7f6c2602fd57] 2024-09-21T07:38:43.588 INFO:tasks.ceph.mds.c.smithi039.stderr: 15: (Thread::_entry_func(void*)+0xd) [0x7f6c2602fd73] 2024-09-21T07:38:43.588 INFO:tasks.ceph.mds.c.smithi039.stderr: 16: /lib64/libc.so.6(+0x89c52) [0x7f6c25689c52] 2024-09-21T07:38:43.588 INFO:tasks.ceph.mds.c.smithi039.stderr: 17: /lib64/libc.so.6(+0x10ec80) [0x7f6c2570ec80] 2024-09-21T07:38:43.588 INFO:tasks.ceph.mds.c.smithi039.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:38:43.589 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:38:43.591 INFO:tasks.ceph.mds.c.smithi039.stderr: -744> 2024-09-21T07:38:39.245+0000 7f6c1be00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:38:43.591 INFO:tasks.ceph.mds.c.smithi039.stderr: -742> 2024-09-21T07:38:39.245+0000 7f6c1be00640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:38:43.592 INFO:tasks.ceph.mds.c.smithi039.stderr: -1> 2024-09-21T07:38:43.577+0000 7f6c16e00640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_prepare_logged(ceph::cref_t&, version_t)' thread 7f6c16e00640 time 2024-09-21T07:38:43.578187+0000 2024-09-21T07:38:43.593 INFO:tasks.ceph.mds.c.smithi039.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: 77: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 2) 2024-09-21T07:38:43.593 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:38:43.593 INFO:tasks.ceph.mds.c.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:38:43.593 INFO:tasks.ceph.mds.c.smithi039.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f6c2605654b] 2024-09-21T07:38:43.593 INFO:tasks.ceph.mds.c.smithi039.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f6c26056758] 2024-09-21T07:38:43.593 INFO:tasks.ceph.mds.c.smithi039.stderr: 3: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x396) [0x5646f105a7e4] 2024-09-21T07:38:43.593 INFO:tasks.ceph.mds.c.smithi039.stderr: 4: ceph-mds(+0x47ac3e) [0x5646f105ac3e] 2024-09-21T07:38:43.593 INFO:tasks.ceph.mds.c.smithi039.stderr: 5: (MDSContext::complete(int)+0x7c) [0x5646f10a76d8] 2024-09-21T07:38:43.593 INFO:tasks.ceph.mds.c.smithi039.stderr: 6: (MDSIOContextBase::complete(int)+0x501) [0x5646f10a7d57] 2024-09-21T07:38:43.593 INFO:tasks.ceph.mds.c.smithi039.stderr: 7: (MDSLogContextBase::complete(int)+0x46) [0x5646f10a7dd4] 2024-09-21T07:38:43.593 INFO:tasks.ceph.mds.c.smithi039.stderr: 8: (Finisher::finisher_thread_entry()+0x495) [0x7f6c25fde34b] 2024-09-21T07:38:43.594 INFO:tasks.ceph.mds.c.smithi039.stderr: 9: ceph-mds(+0x1052e1) [0x5646f0ce52e1] 2024-09-21T07:38:43.594 INFO:tasks.ceph.mds.c.smithi039.stderr: 10: (Thread::entry_wrapper()+0x43) [0x7f6c2602fd57] 2024-09-21T07:38:43.594 INFO:tasks.ceph.mds.c.smithi039.stderr: 11: (Thread::_entry_func(void*)+0xd) [0x7f6c2602fd73] 2024-09-21T07:38:43.594 INFO:tasks.ceph.mds.c.smithi039.stderr: 12: /lib64/libc.so.6(+0x89c52) [0x7f6c25689c52] 2024-09-21T07:38:43.594 INFO:tasks.ceph.mds.c.smithi039.stderr: 13: /lib64/libc.so.6(+0x10ec80) [0x7f6c2570ec80] 2024-09-21T07:38:43.594 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:38:43.594 INFO:tasks.ceph.mds.c.smithi039.stderr: 0> 2024-09-21T07:38:43.579+0000 7f6c16e00640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:38:43.594 INFO:tasks.ceph.mds.c.smithi039.stderr: in thread 7f6c16e00640 thread_name:MR_Finisher 2024-09-21T07:38:43.594 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:38:43.594 INFO:tasks.ceph.mds.c.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:38:43.594 INFO:tasks.ceph.mds.c.smithi039.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f6c2563e6f0] 2024-09-21T07:38:43.594 INFO:tasks.ceph.mds.c.smithi039.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f6c2568b99c] 2024-09-21T07:38:43.595 INFO:tasks.ceph.mds.c.smithi039.stderr: 3: raise() 2024-09-21T07:38:43.595 INFO:tasks.ceph.mds.c.smithi039.stderr: 4: abort() 2024-09-21T07:38:43.595 INFO:tasks.ceph.mds.c.smithi039.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7f6c26056686] 2024-09-21T07:38:43.595 INFO:tasks.ceph.mds.c.smithi039.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f6c26056758] 2024-09-21T07:38:43.595 INFO:tasks.ceph.mds.c.smithi039.stderr: 7: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x396) [0x5646f105a7e4] 2024-09-21T07:38:43.595 INFO:tasks.ceph.mds.c.smithi039.stderr: 8: ceph-mds(+0x47ac3e) [0x5646f105ac3e] 2024-09-21T07:38:43.595 INFO:tasks.ceph.mds.c.smithi039.stderr: 9: (MDSContext::complete(int)+0x7c) [0x5646f10a76d8] 2024-09-21T07:38:43.595 INFO:tasks.ceph.mds.c.smithi039.stderr: 10: (MDSIOContextBase::complete(int)+0x501) [0x5646f10a7d57] 2024-09-21T07:38:43.595 INFO:tasks.ceph.mds.c.smithi039.stderr: 11: (MDSLogContextBase::complete(int)+0x46) [0x5646f10a7dd4] 2024-09-21T07:38:43.595 INFO:tasks.ceph.mds.c.smithi039.stderr: 12: (Finisher::finisher_thread_entry()+0x495) [0x7f6c25fde34b] 2024-09-21T07:38:43.595 INFO:tasks.ceph.mds.c.smithi039.stderr: 13: ceph-mds(+0x1052e1) [0x5646f0ce52e1] 2024-09-21T07:38:43.595 INFO:tasks.ceph.mds.c.smithi039.stderr: 14: (Thread::entry_wrapper()+0x43) [0x7f6c2602fd57] 2024-09-21T07:38:43.596 INFO:tasks.ceph.mds.c.smithi039.stderr: 15: (Thread::_entry_func(void*)+0xd) [0x7f6c2602fd73] 2024-09-21T07:38:43.596 INFO:tasks.ceph.mds.c.smithi039.stderr: 16: /lib64/libc.so.6(+0x89c52) [0x7f6c25689c52] 2024-09-21T07:38:43.596 INFO:tasks.ceph.mds.c.smithi039.stderr: 17: /lib64/libc.so.6(+0x10ec80) [0x7f6c2570ec80] 2024-09-21T07:38:43.596 INFO:tasks.ceph.mds.c.smithi039.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:38:43.596 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:38:43.599 INFO:tasks.ceph.mds.c.smithi039.stderr: -744> 2024-09-21T07:38:39.245+0000 7f6c1be00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:38:43.599 INFO:tasks.ceph.mds.c.smithi039.stderr: -742> 2024-09-21T07:38:39.245+0000 7f6c1be00640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:38:43.601 INFO:tasks.ceph.mds.c.smithi039.stderr: -1> 2024-09-21T07:38:43.577+0000 7f6c16e00640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_prepare_logged(ceph::cref_t&, version_t)' thread 7f6c16e00640 time 2024-09-21T07:38:43.578187+0000 2024-09-21T07:38:43.601 INFO:tasks.ceph.mds.c.smithi039.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: 77: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 2) 2024-09-21T07:38:43.601 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:38:43.601 INFO:tasks.ceph.mds.c.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:38:43.601 INFO:tasks.ceph.mds.c.smithi039.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f6c2605654b] 2024-09-21T07:38:43.601 INFO:tasks.ceph.mds.c.smithi039.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f6c26056758] 2024-09-21T07:38:43.601 INFO:tasks.ceph.mds.c.smithi039.stderr: 3: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x396) [0x5646f105a7e4] 2024-09-21T07:38:43.601 INFO:tasks.ceph.mds.c.smithi039.stderr: 4: ceph-mds(+0x47ac3e) [0x5646f105ac3e] 2024-09-21T07:38:43.601 INFO:tasks.ceph.mds.c.smithi039.stderr: 5: (MDSContext::complete(int)+0x7c) [0x5646f10a76d8] 2024-09-21T07:38:43.601 INFO:tasks.ceph.mds.c.smithi039.stderr: 6: (MDSIOContextBase::complete(int)+0x501) [0x5646f10a7d57] 2024-09-21T07:38:43.601 INFO:tasks.ceph.mds.c.smithi039.stderr: 7: (MDSLogContextBase::complete(int)+0x46) [0x5646f10a7dd4] 2024-09-21T07:38:43.601 INFO:tasks.ceph.mds.c.smithi039.stderr: 8: (Finisher::finisher_thread_entry()+0x495) [0x7f6c25fde34b] 2024-09-21T07:38:43.602 INFO:tasks.ceph.mds.c.smithi039.stderr: 9: ceph-mds(+0x1052e1) [0x5646f0ce52e1] 2024-09-21T07:38:43.602 INFO:tasks.ceph.mds.c.smithi039.stderr: 10: (Thread::entry_wrapper()+0x43) [0x7f6c2602fd57] 2024-09-21T07:38:43.602 INFO:tasks.ceph.mds.c.smithi039.stderr: 11: (Thread::_entry_func(void*)+0xd) [0x7f6c2602fd73] 2024-09-21T07:38:43.602 INFO:tasks.ceph.mds.c.smithi039.stderr: 12: /lib64/libc.so.6(+0x89c52) [0x7f6c25689c52] 2024-09-21T07:38:43.602 INFO:tasks.ceph.mds.c.smithi039.stderr: 13: /lib64/libc.so.6(+0x10ec80) [0x7f6c2570ec80] 2024-09-21T07:38:43.602 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:38:43.602 INFO:tasks.ceph.mds.c.smithi039.stderr: 0> 2024-09-21T07:38:43.579+0000 7f6c16e00640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:38:43.602 INFO:tasks.ceph.mds.c.smithi039.stderr: in thread 7f6c16e00640 thread_name:MR_Finisher 2024-09-21T07:38:43.602 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:38:43.602 INFO:tasks.ceph.mds.c.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:38:43.602 INFO:tasks.ceph.mds.c.smithi039.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f6c2563e6f0] 2024-09-21T07:38:43.602 INFO:tasks.ceph.mds.c.smithi039.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f6c2568b99c] 2024-09-21T07:38:43.602 INFO:tasks.ceph.mds.c.smithi039.stderr: 3: raise() 2024-09-21T07:38:43.603 INFO:tasks.ceph.mds.c.smithi039.stderr: 4: abort() 2024-09-21T07:38:43.603 INFO:tasks.ceph.mds.c.smithi039.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7f6c26056686] 2024-09-21T07:38:43.603 INFO:tasks.ceph.mds.c.smithi039.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f6c26056758] 2024-09-21T07:38:43.603 INFO:tasks.ceph.mds.c.smithi039.stderr: 7: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x396) [0x5646f105a7e4] 2024-09-21T07:38:43.603 INFO:tasks.ceph.mds.c.smithi039.stderr: 8: ceph-mds(+0x47ac3e) [0x5646f105ac3e] 2024-09-21T07:38:43.603 INFO:tasks.ceph.mds.c.smithi039.stderr: 9: (MDSContext::complete(int)+0x7c) [0x5646f10a76d8] 2024-09-21T07:38:43.603 INFO:tasks.ceph.mds.c.smithi039.stderr: 10: (MDSIOContextBase::complete(int)+0x501) [0x5646f10a7d57] 2024-09-21T07:38:43.603 INFO:tasks.ceph.mds.c.smithi039.stderr: 11: (MDSLogContextBase::complete(int)+0x46) [0x5646f10a7dd4] 2024-09-21T07:38:43.603 INFO:tasks.ceph.mds.c.smithi039.stderr: 12: (Finisher::finisher_thread_entry()+0x495) [0x7f6c25fde34b] 2024-09-21T07:38:43.603 INFO:tasks.ceph.mds.c.smithi039.stderr: 13: ceph-mds(+0x1052e1) [0x5646f0ce52e1] 2024-09-21T07:38:43.603 INFO:tasks.ceph.mds.c.smithi039.stderr: 14: (Thread::entry_wrapper()+0x43) [0x7f6c2602fd57] 2024-09-21T07:38:43.603 INFO:tasks.ceph.mds.c.smithi039.stderr: 15: (Thread::_entry_func(void*)+0xd) [0x7f6c2602fd73] 2024-09-21T07:38:43.603 INFO:tasks.ceph.mds.c.smithi039.stderr: 16: /lib64/libc.so.6(+0x89c52) [0x7f6c25689c52] 2024-09-21T07:38:43.604 INFO:tasks.ceph.mds.c.smithi039.stderr: 17: /lib64/libc.so.6(+0x10ec80) [0x7f6c2570ec80] 2024-09-21T07:38:43.604 INFO:tasks.ceph.mds.c.smithi039.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:38:43.604 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:38:43.818 INFO:teuthology.orchestra.run.smithi039.stdout:{"mds_beacon_grace":"15.000000"} 2024-09-21T07:38:43.818 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-09-21T07:38:43.819 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:38:44.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.089+0000 7f5198c00640 1 -- 172.21.15.39:0/1864918248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5178003c10 msgr2=0x7f5178004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:44.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.089+0000 7f5198c00640 1 --2- 172.21.15.39:0/1864918248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5178003c10 0x7f5178004060 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f51840017d0 tx=0x7f5178005830 comp rx=0 tx=0).stop 2024-09-21T07:38:44.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.089+0000 7f5192800640 1 -- 172.21.15.39:0/1864918248 reap_dead start 2024-09-21T07:38:44.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.089+0000 7f5198c00640 1 -- 172.21.15.39:0/1864918248 shutdown_connections 2024-09-21T07:38:44.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.089+0000 7f5198c00640 1 -- 172.21.15.39:0/1864918248 >> 172.21.15.39:0/1864918248 conn(0x7f519406cfc0 msgr2=0x7f519406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:44.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.089+0000 7f5198c00640 1 -- 172.21.15.39:0/1864918248 shutdown_connections 2024-09-21T07:38:44.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.089+0000 7f5198c00640 1 -- 172.21.15.39:0/1864918248 wait complete. 2024-09-21T07:38:44.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.092+0000 7f5198c00640 1 Processor -- start 2024-09-21T07:38:44.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.092+0000 7f5198c00640 1 -- start start 2024-09-21T07:38:44.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.093+0000 7f5198c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5178003c10 0x7f519415b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:44.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.093+0000 7f5198c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51941f9840 0x7f51940b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:44.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.093+0000 7f5198c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51940a9480 0x7f51940a9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:44.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.093+0000 7f5198c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f51940d0f00 con 0x7f51940a9480 2024-09-21T07:38:44.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.093+0000 7f5198c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f51940d0d80 con 0x7f5178003c10 2024-09-21T07:38:44.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.093+0000 7f5198c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f5194073710 con 0x7f51941f9840 2024-09-21T07:38:44.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.094+0000 7f5192800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51940a9480 0x7f51940a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:44.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.094+0000 7f5192800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51940a9480 0x7f51940a9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47390/0 (socket says 172.21.15.39:47390) 2024-09-21T07:38:44.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.094+0000 7f5192800640 1 -- 172.21.15.39:0/4014527995 learned_addr learned my addr 172.21.15.39:0/4014527995 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:44.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.094+0000 7f5191e00640 1 --2- 172.21.15.39:0/4014527995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5178003c10 0x7f519415b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:44.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.094+0000 7f518be00640 1 --2- 172.21.15.39:0/4014527995 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51941f9840 0x7f51940b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:44.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.094+0000 7f5192800640 1 -- 172.21.15.39:0/4014527995 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51941f9840 msgr2=0x7f51940b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:44.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.094+0000 7f5192800640 1 --2- 172.21.15.39:0/4014527995 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51941f9840 0x7f51940b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:44.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.094+0000 7f5192800640 1 -- 172.21.15.39:0/4014527995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5178003c10 msgr2=0x7f519415b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:44.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.094+0000 7f5192800640 1 --2- 172.21.15.39:0/4014527995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5178003c10 0x7f519415b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:44.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.094+0000 7f5192800640 1 -- 172.21.15.39:0/4014527995 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5194155ff0 con 0x7f51940a9480 2024-09-21T07:38:44.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.095+0000 7f5192800640 1 --2- 172.21.15.39:0/4014527995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51940a9480 0x7f51940a9890 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f5194157330 tx=0x7f518c00f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:44.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.095+0000 7f518a000640 1 -- 172.21.15.39:0/4014527995 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c004b20 con 0x7f51940a9480 2024-09-21T07:38:44.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.095+0000 7f518a000640 1 -- 172.21.15.39:0/4014527995 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f518c004cc0 con 0x7f51940a9480 2024-09-21T07:38:44.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.095+0000 7f518a000640 1 -- 172.21.15.39:0/4014527995 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c003bd0 con 0x7f51940a9480 2024-09-21T07:38:44.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.095+0000 7f5198c00640 1 -- 172.21.15.39:0/4014527995 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f519415c2e0 con 0x7f51940a9480 2024-09-21T07:38:44.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.095+0000 7f5198c00640 1 -- 172.21.15.39:0/4014527995 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51941f9040 con 0x7f51940a9480 2024-09-21T07:38:44.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.096+0000 7f5198c00640 1 -- 172.21.15.39:0/4014527995 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f519415fb60 con 0x7f51940a9480 2024-09-21T07:38:44.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.097+0000 7f518a000640 1 -- 172.21.15.39:0/4014527995 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f518c004030 con 0x7f51940a9480 2024-09-21T07:38:44.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.098+0000 7f518a000640 1 --2- 172.21.15.39:0/4014527995 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5170078bc0 0x7f517007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:44.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.098+0000 7f518a000640 1 -- 172.21.15.39:0/4014527995 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 7173+0+0 (secure 0 0 0) 0x7f518c09b750 con 0x7f51940a9480 2024-09-21T07:38:44.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.099+0000 7f5191e00640 1 --2- 172.21.15.39:0/4014527995 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5170078bc0 0x7f517007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:44.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.099+0000 7f5191e00640 1 --2- 172.21.15.39:0/4014527995 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5170078bc0 0x7f517007b0d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5184032000 tx=0x7f5184001360 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:44.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.103+0000 7f518a000640 1 -- 172.21.15.39:0/4014527995 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f519415fb60 con 0x7f51940a9480 2024-09-21T07:38:44.216 INFO:tasks.ceph.mds.c.smithi039.stderr:daemon-helper: command crashed with signal 6 2024-09-21T07:38:44.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.351+0000 7f5198c00640 1 -- 172.21.15.39:0/4014527995 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f519415c070 con 0x7f51940a9480 2024-09-21T07:38:44.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.351+0000 7f518a000640 1 -- 172.21.15.39:0/4014527995 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5243 (secure 0 0 0) 0x7f518c062f70 con 0x7f51940a9480 2024-09-21T07:38:44.353 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:44.353 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":92,"btime":"2024-09-21T07:38:43:259438+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/992188984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":992188984},{"type":"v1","addr":"172.21.15.148:6837","nonce":992188984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:38:42.806952+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6825,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6825":{"gid":6825,"name":"c","rank":0,"incarnation":86,"state":"up:active","state_seq":13,"addr":"172.21.15.39:6834/855150775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":855150775},{"type":"v1","addr":"172.21.15.39:6834","nonce":855150775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6825,"qdb_cluster":[6115,6825]},"id":6}]} 2024-09-21T07:38:44.354 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 92 2024-09-21T07:38:44.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.354+0000 7f5198c00640 1 -- 172.21.15.39:0/4014527995 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5170078bc0 msgr2=0x7f517007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:44.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.354+0000 7f5198c00640 1 --2- 172.21.15.39:0/4014527995 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5170078bc0 0x7f517007b0d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5184032000 tx=0x7f5184001360 comp rx=0 tx=0).stop 2024-09-21T07:38:44.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.354+0000 7f5198c00640 1 -- 172.21.15.39:0/4014527995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51940a9480 msgr2=0x7f51940a9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:44.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.354+0000 7f5198c00640 1 --2- 172.21.15.39:0/4014527995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51940a9480 0x7f51940a9890 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f5194157330 tx=0x7f518c00f3c0 comp rx=0 tx=0).stop 2024-09-21T07:38:44.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.354+0000 7f5198c00640 1 -- 172.21.15.39:0/4014527995 shutdown_connections 2024-09-21T07:38:44.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.354+0000 7f5198c00640 1 --2- 172.21.15.39:0/4014527995 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5170078bc0 0x7f517007b0d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:44.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.354+0000 7f5198c00640 1 --2- 172.21.15.39:0/4014527995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51940a9480 0x7f51940a9890 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:44.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.354+0000 7f5198c00640 1 --2- 172.21.15.39:0/4014527995 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51941f9840 0x7f51940b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:44.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.354+0000 7f5198c00640 1 --2- 172.21.15.39:0/4014527995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5178003c10 0x7f519415b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:44.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.354+0000 7f5198c00640 1 -- 172.21.15.39:0/4014527995 >> 172.21.15.39:0/4014527995 conn(0x7f519406cfc0 msgr2=0x7f519406f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:44.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.354+0000 7f5198c00640 1 -- 172.21.15.39:0/4014527995 shutdown_connections 2024-09-21T07:38:44.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:44.354+0000 7f5198c00640 1 -- 172.21.15.39:0/4014527995 wait complete. 2024-09-21T07:38:44.378 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:38:49.379 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:38:49.534 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-09-21T07:38:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.617+0000 7f1792400640 1 -- 172.21.15.39:0/2122108122 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f178c06fc10 msgr2=0x7f178c0d0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.617+0000 7f1792400640 1 --2- 172.21.15.39:0/2122108122 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f178c06fc10 0x7f178c0d0620 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1784005fd0 tx=0x7f178402a8a0 comp rx=0 tx=0).stop 2024-09-21T07:38:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.617+0000 7f1792400640 1 -- 172.21.15.39:0/2122108122 shutdown_connections 2024-09-21T07:38:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.617+0000 7f1792400640 1 --2- 172.21.15.39:0/2122108122 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f178c06fc10 0x7f178c0d0620 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.617+0000 7f1792400640 1 --2- 172.21.15.39:0/2122108122 >> v2:172.21.15.148:3300/0 conn(0x7f178c15cd30 0x7f178c0cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:49.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.617+0000 7f1792400640 1 -- 172.21.15.39:0/2122108122 >> 172.21.15.39:0/2122108122 conn(0x7f178c06cfc0 msgr2=0x7f178c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:49.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.617+0000 7f1792400640 1 -- 172.21.15.39:0/2122108122 shutdown_connections 2024-09-21T07:38:49.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.617+0000 7f1792400640 1 -- 172.21.15.39:0/2122108122 wait complete. 2024-09-21T07:38:49.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.619+0000 7f1792400640 1 Processor -- start 2024-09-21T07:38:49.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.619+0000 7f1792400640 1 -- start start 2024-09-21T07:38:49.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.621+0000 7f1792400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f178c06f800 0x7f178c0a9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:49.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.621+0000 7f1792400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f178c06fc10 0x7f178c152060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:49.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.621+0000 7f1792400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f178c15cd30 0x7f178c1525a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:49.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.621+0000 7f1792400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f178c073890 con 0x7f178c06fc10 2024-09-21T07:38:49.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.621+0000 7f1792400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f178c073710 con 0x7f178c06f800 2024-09-21T07:38:49.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.621+0000 7f1792400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f178c073a10 con 0x7f178c15cd30 2024-09-21T07:38:49.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.621+0000 7f1783e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f178c06fc10 0x7f178c152060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:49.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.621+0000 7f1783e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f178c06fc10 0x7f178c152060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47392/0 (socket says 172.21.15.39:47392) 2024-09-21T07:38:49.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.621+0000 7f1783e00640 1 -- 172.21.15.39:0/3603547075 learned_addr learned my addr 172.21.15.39:0/3603547075 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:49.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.621+0000 7f178be00640 1 --2- 172.21.15.39:0/3603547075 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f178c15cd30 0x7f178c1525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:49.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.621+0000 7f178b400640 1 --2- 172.21.15.39:0/3603547075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f178c06f800 0x7f178c0a9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:49.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.622+0000 7f1783e00640 1 -- 172.21.15.39:0/3603547075 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f178c15cd30 msgr2=0x7f178c1525a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:49.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.622+0000 7f1783e00640 1 --2- 172.21.15.39:0/3603547075 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f178c15cd30 0x7f178c1525a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:49.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.622+0000 7f1783e00640 1 -- 172.21.15.39:0/3603547075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f178c06f800 msgr2=0x7f178c0a9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:49.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.622+0000 7f1783e00640 1 --2- 172.21.15.39:0/3603547075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f178c06f800 0x7f178c0a9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:49.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.622+0000 7f1783e00640 1 -- 172.21.15.39:0/3603547075 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f178c1f6770 con 0x7f178c06fc10 2024-09-21T07:38:49.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.622+0000 7f1783e00640 1 --2- 172.21.15.39:0/3603547075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f178c06fc10 0x7f178c152060 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f177400c370 tx=0x7f177400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:49.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.622+0000 7f1783400640 1 -- 172.21.15.39:0/3603547075 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1774011790 con 0x7f178c06fc10 2024-09-21T07:38:49.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.622+0000 7f1783400640 1 -- 172.21.15.39:0/3603547075 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1774017070 con 0x7f178c06fc10 2024-09-21T07:38:49.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.622+0000 7f1792400640 1 -- 172.21.15.39:0/3603547075 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f178c1f6c70 con 0x7f178c06fc10 2024-09-21T07:38:49.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.622+0000 7f1783400640 1 -- 172.21.15.39:0/3603547075 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1774011cc0 con 0x7f178c06fc10 2024-09-21T07:38:49.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.622+0000 7f1792400640 1 -- 172.21.15.39:0/3603547075 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f178c1f7230 con 0x7f178c06fc10 2024-09-21T07:38:49.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.623+0000 7f1792400640 1 -- 172.21.15.39:0/3603547075 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f178c073ec0 con 0x7f178c06fc10 2024-09-21T07:38:49.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.625+0000 7f1783400640 1 -- 172.21.15.39:0/3603547075 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f177401c460 con 0x7f178c06fc10 2024-09-21T07:38:49.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.626+0000 7f1783400640 1 --2- 172.21.15.39:0/3603547075 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1768078b90 0x7f176807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:49.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.626+0000 7f1783400640 1 -- 172.21.15.39:0/3603547075 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 7261+0+0 (secure 0 0 0) 0x7f177409afe0 con 0x7f178c06fc10 2024-09-21T07:38:49.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.626+0000 7f178b400640 1 --2- 172.21.15.39:0/3603547075 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1768078b90 0x7f176807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:49.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.627+0000 7f178b400640 1 --2- 172.21.15.39:0/3603547075 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1768078b90 0x7f176807b0a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f177c0095f0 tx=0x7f177c005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:49.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.630+0000 7f1783400640 1 -- 172.21.15.39:0/3603547075 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1774062730 con 0x7f178c06fc10 2024-09-21T07:38:49.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.865+0000 7f1792400640 1 -- 172.21.15.39:0/3603547075 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f178c1f6a00 con 0x7f178c06fc10 2024-09-21T07:38:49.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.865+0000 7f1783400640 1 -- 172.21.15.39:0/3603547075 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5243 (secure 0 0 0) 0x7f1774067160 con 0x7f178c06fc10 2024-09-21T07:38:49.867 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:49.867 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":92,"btime":"2024-09-21T07:38:43:259438+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/992188984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":992188984},{"type":"v1","addr":"172.21.15.148:6837","nonce":992188984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:38:42.806952+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6825,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6825":{"gid":6825,"name":"c","rank":0,"incarnation":86,"state":"up:active","state_seq":13,"addr":"172.21.15.39:6834/855150775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":855150775},{"type":"v1","addr":"172.21.15.39:6834","nonce":855150775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6825,"qdb_cluster":[6115,6825]},"id":6}]} 2024-09-21T07:38:49.868 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 92 2024-09-21T07:38:49.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.868+0000 7f1792400640 1 -- 172.21.15.39:0/3603547075 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1768078b90 msgr2=0x7f176807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:49.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.868+0000 7f1792400640 1 --2- 172.21.15.39:0/3603547075 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1768078b90 0x7f176807b0a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f177c0095f0 tx=0x7f177c005f90 comp rx=0 tx=0).stop 2024-09-21T07:38:49.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.868+0000 7f1792400640 1 -- 172.21.15.39:0/3603547075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f178c06fc10 msgr2=0x7f178c152060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:49.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.868+0000 7f1792400640 1 --2- 172.21.15.39:0/3603547075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f178c06fc10 0x7f178c152060 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f177400c370 tx=0x7f177400c830 comp rx=0 tx=0).stop 2024-09-21T07:38:49.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.868+0000 7f1792400640 1 -- 172.21.15.39:0/3603547075 shutdown_connections 2024-09-21T07:38:49.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.868+0000 7f1792400640 1 --2- 172.21.15.39:0/3603547075 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1768078b90 0x7f176807b0a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:49.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.868+0000 7f1792400640 1 --2- 172.21.15.39:0/3603547075 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f178c15cd30 0x7f178c1525a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:49.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.868+0000 7f1792400640 1 --2- 172.21.15.39:0/3603547075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f178c06fc10 0x7f178c152060 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:49.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.868+0000 7f1792400640 1 --2- 172.21.15.39:0/3603547075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f178c06f800 0x7f178c0a9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:49.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.868+0000 7f1792400640 1 -- 172.21.15.39:0/3603547075 >> 172.21.15.39:0/3603547075 conn(0x7f178c06cfc0 msgr2=0x7f178c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:49.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.868+0000 7f1792400640 1 -- 172.21.15.39:0/3603547075 shutdown_connections 2024-09-21T07:38:49.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:49.868+0000 7f1792400640 1 -- 172.21.15.39:0/3603547075 wait complete. 2024-09-21T07:38:49.892 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:38:54.892 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:38:55.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.126+0000 7fcb08a00640 1 -- 172.21.15.39:0/1183474578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb040d0f20 msgr2=0x7fcb0415c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:55.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.126+0000 7fcb08a00640 1 --2- 172.21.15.39:0/1183474578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb040d0f20 0x7fcb0415c830 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4005fd0 tx=0x7fcaf402a5d0 comp rx=0 tx=0).stop 2024-09-21T07:38:55.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.126+0000 7fcb08a00640 1 -- 172.21.15.39:0/1183474578 shutdown_connections 2024-09-21T07:38:55.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.126+0000 7fcb08a00640 1 --2- 172.21.15.39:0/1183474578 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcb040cbd70 0x7fcb04072fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:55.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.126+0000 7fcb08a00640 1 --2- 172.21.15.39:0/1183474578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb040d0f20 0x7fcb0415c830 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:55.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.126+0000 7fcb08a00640 1 -- 172.21.15.39:0/1183474578 >> 172.21.15.39:0/1183474578 conn(0x7fcb040b2510 msgr2=0x7fcb040d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:55.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.126+0000 7fcb08a00640 1 -- 172.21.15.39:0/1183474578 shutdown_connections 2024-09-21T07:38:55.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.126+0000 7fcb08a00640 1 -- 172.21.15.39:0/1183474578 wait complete. 2024-09-21T07:38:55.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.128+0000 7fcb08a00640 1 Processor -- start 2024-09-21T07:38:55.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.128+0000 7fcb08a00640 1 -- start start 2024-09-21T07:38:55.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.130+0000 7fcb08a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb040cb960 0x7fcb04159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:55.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.130+0000 7fcb08a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcb040cbd70 0x7fcb041f91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:55.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.130+0000 7fcb08a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb040d0f20 0x7fcb04156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:55.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.130+0000 7fcb08a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcb040b4570 con 0x7fcb040cb960 2024-09-21T07:38:55.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.130+0000 7fcb08a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcb040b43f0 con 0x7fcb040d0f20 2024-09-21T07:38:55.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.130+0000 7fcb08a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fcb040b46f0 con 0x7fcb040cbd70 2024-09-21T07:38:55.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.130+0000 7fcb03400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb040cb960 0x7fcb04159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:55.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.130+0000 7fcb03400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb040cb960 0x7fcb04159fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43670/0 (socket says 172.21.15.39:43670) 2024-09-21T07:38:55.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.130+0000 7fcb03400640 1 -- 172.21.15.39:0/2760996211 learned_addr learned my addr 172.21.15.39:0/2760996211 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:38:55.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.130+0000 7fcafbe00640 1 --2- 172.21.15.39:0/2760996211 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcb040cbd70 0x7fcb041f91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:55.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.130+0000 7fcb03e00640 1 --2- 172.21.15.39:0/2760996211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb040d0f20 0x7fcb04156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:55.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.130+0000 7fcb03400640 1 -- 172.21.15.39:0/2760996211 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcb040cbd70 msgr2=0x7fcb041f91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:55.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.130+0000 7fcb03400640 1 --2- 172.21.15.39:0/2760996211 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcb040cbd70 0x7fcb041f91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:55.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.131+0000 7fcb03400640 1 -- 172.21.15.39:0/2760996211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb040d0f20 msgr2=0x7fcb04156bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:55.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.131+0000 7fcb03400640 1 --2- 172.21.15.39:0/2760996211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb040d0f20 0x7fcb04156bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:55.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.131+0000 7fcb03400640 1 -- 172.21.15.39:0/2760996211 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb0415a830 con 0x7fcb040cb960 2024-09-21T07:38:55.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.131+0000 7fcb03e00640 1 --2- 172.21.15.39:0/2760996211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb040d0f20 0x7fcb04156bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:38:55.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.131+0000 7fcb03400640 1 --2- 172.21.15.39:0/2760996211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb040cb960 0x7fcb04159fe0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4034040 tx=0x7fcaf40035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:55.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.131+0000 7fcafb400640 1 -- 172.21.15.39:0/2760996211 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf403e030 con 0x7fcb040cb960 2024-09-21T07:38:55.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.131+0000 7fcafb400640 1 -- 172.21.15.39:0/2760996211 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcaf4003ae0 con 0x7fcb040cb960 2024-09-21T07:38:55.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.131+0000 7fcafb400640 1 -- 172.21.15.39:0/2760996211 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4035040 con 0x7fcb040cb960 2024-09-21T07:38:55.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.131+0000 7fcb08a00640 1 -- 172.21.15.39:0/2760996211 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb0415ab10 con 0x7fcb040cb960 2024-09-21T07:38:55.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.131+0000 7fcb08a00640 1 -- 172.21.15.39:0/2760996211 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb0415b0d0 con 0x7fcb040cb960 2024-09-21T07:38:55.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.132+0000 7fcb08a00640 1 -- 172.21.15.39:0/2760996211 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb040b4c80 con 0x7fcb040cb960 2024-09-21T07:38:55.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.134+0000 7fcafb400640 1 -- 172.21.15.39:0/2760996211 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcaf4041030 con 0x7fcb040cb960 2024-09-21T07:38:55.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.135+0000 7fcafb400640 1 --2- 172.21.15.39:0/2760996211 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcae0078b90 0x7fcae007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:38:55.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.135+0000 7fcafb400640 1 -- 172.21.15.39:0/2760996211 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 7217+0+0 (secure 0 0 0) 0x7fcaf40bb8c0 con 0x7fcb040cb960 2024-09-21T07:38:55.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.136+0000 7fcafbe00640 1 --2- 172.21.15.39:0/2760996211 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcae0078b90 0x7fcae007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:38:55.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.136+0000 7fcafbe00640 1 --2- 172.21.15.39:0/2760996211 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcae0078b90 0x7fcae007b0a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fcb041fe090 tx=0x7fcaec009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:38:55.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.140+0000 7fcafb400640 1 -- 172.21.15.39:0/2760996211 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcaf40836f0 con 0x7fcb040cb960 2024-09-21T07:38:55.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.376+0000 7fcb08a00640 1 -- 172.21.15.39:0/2760996211 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb040b1b00 con 0x7fcb040cb960 2024-09-21T07:38:55.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.377+0000 7fcafb400640 1 -- 172.21.15.39:0/2760996211 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5243 (secure 0 0 0) 0x7fcaf4082ff0 con 0x7fcb040cb960 2024-09-21T07:38:55.379 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:38:55.379 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":92,"btime":"2024-09-21T07:38:43:259438+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/992188984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":992188984},{"type":"v1","addr":"172.21.15.148:6837","nonce":992188984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:38:42.806952+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6825,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6825":{"gid":6825,"name":"c","rank":0,"incarnation":86,"state":"up:active","state_seq":13,"addr":"172.21.15.39:6834/855150775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":855150775},{"type":"v1","addr":"172.21.15.39:6834","nonce":855150775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6825,"qdb_cluster":[6115,6825]},"id":6}]} 2024-09-21T07:38:55.379 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 92 2024-09-21T07:38:55.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.379+0000 7fcb08a00640 1 -- 172.21.15.39:0/2760996211 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcae0078b90 msgr2=0x7fcae007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:55.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.379+0000 7fcb08a00640 1 --2- 172.21.15.39:0/2760996211 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcae0078b90 0x7fcae007b0a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fcb041fe090 tx=0x7fcaec009040 comp rx=0 tx=0).stop 2024-09-21T07:38:55.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.379+0000 7fcb08a00640 1 -- 172.21.15.39:0/2760996211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb040cb960 msgr2=0x7fcb04159fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:38:55.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.379+0000 7fcb08a00640 1 --2- 172.21.15.39:0/2760996211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb040cb960 0x7fcb04159fe0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4034040 tx=0x7fcaf40035f0 comp rx=0 tx=0).stop 2024-09-21T07:38:55.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.379+0000 7fcb08a00640 1 -- 172.21.15.39:0/2760996211 shutdown_connections 2024-09-21T07:38:55.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.379+0000 7fcb08a00640 1 --2- 172.21.15.39:0/2760996211 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcae0078b90 0x7fcae007b0a0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:55.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.379+0000 7fcb08a00640 1 --2- 172.21.15.39:0/2760996211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb040d0f20 0x7fcb04156bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:55.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.379+0000 7fcb08a00640 1 --2- 172.21.15.39:0/2760996211 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcb040cbd70 0x7fcb041f91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:55.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.379+0000 7fcb08a00640 1 --2- 172.21.15.39:0/2760996211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb040cb960 0x7fcb04159fe0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:38:55.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.379+0000 7fcb08a00640 1 -- 172.21.15.39:0/2760996211 >> 172.21.15.39:0/2760996211 conn(0x7fcb040b2510 msgr2=0x7fcb0415eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:38:55.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.379+0000 7fcb08a00640 1 -- 172.21.15.39:0/2760996211 shutdown_connections 2024-09-21T07:38:55.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:38:55.379+0000 7fcb08a00640 1 -- 172.21.15.39:0/2760996211 wait complete. 2024-09-21T07:38:55.404 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:38:56.145 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~7s 2024-09-21T07:39:00.406 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:39:00.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.642+0000 7fbe28400640 1 -- 172.21.15.39:0/2066868847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe2015f360 msgr2=0x7fbe2015bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:00.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.642+0000 7fbe28400640 1 --2- 172.21.15.39:0/2066868847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe2015f360 0x7fbe2015bd70 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fbe14009830 tx=0x7fbe1402a510 comp rx=0 tx=0).stop 2024-09-21T07:39:00.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.642+0000 7fbe28400640 1 -- 172.21.15.39:0/2066868847 shutdown_connections 2024-09-21T07:39:00.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.642+0000 7fbe28400640 1 --2- 172.21.15.39:0/2066868847 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe20155e80 0x7fbe200b4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:00.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.642+0000 7fbe28400640 1 --2- 172.21.15.39:0/2066868847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe2015f360 0x7fbe2015bd70 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:00.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.642+0000 7fbe28400640 1 -- 172.21.15.39:0/2066868847 >> 172.21.15.39:0/2066868847 conn(0x7fbe2006cfc0 msgr2=0x7fbe2006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:00.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.642+0000 7fbe28400640 1 -- 172.21.15.39:0/2066868847 shutdown_connections 2024-09-21T07:39:00.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.642+0000 7fbe28400640 1 -- 172.21.15.39:0/2066868847 wait complete. 2024-09-21T07:39:00.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.644+0000 7fbe28400640 1 Processor -- start 2024-09-21T07:39:00.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.644+0000 7fbe28400640 1 -- start start 2024-09-21T07:39:00.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.646+0000 7fbe28400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe20155a70 0x7fbe2015d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:00.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.646+0000 7fbe28400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe20155e80 0x7fbe2014a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:00.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.646+0000 7fbe28400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe2015f360 0x7fbe2014d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:00.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.646+0000 7fbe28400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbe200d0620 con 0x7fbe2015f360 2024-09-21T07:39:00.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.646+0000 7fbe28400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbe200b4ef0 con 0x7fbe20155a70 2024-09-21T07:39:00.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.646+0000 7fbe28400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fbe200d07a0 con 0x7fbe20155e80 2024-09-21T07:39:00.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.646+0000 7fbe26400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe2015f360 0x7fbe2014d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:00.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.646+0000 7fbe26400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe2015f360 0x7fbe2014d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43682/0 (socket says 172.21.15.39:43682) 2024-09-21T07:39:00.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.646+0000 7fbe26400640 1 -- 172.21.15.39:0/1141728141 learned_addr learned my addr 172.21.15.39:0/1141728141 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:39:00.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.646+0000 7fbe25000640 1 --2- 172.21.15.39:0/1141728141 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe20155e80 0x7fbe2014a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:00.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.646+0000 7fbe25a00640 1 --2- 172.21.15.39:0/1141728141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe20155a70 0x7fbe2015d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:00.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.647+0000 7fbe26400640 1 -- 172.21.15.39:0/1141728141 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe20155e80 msgr2=0x7fbe2014a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:00.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.647+0000 7fbe26400640 1 --2- 172.21.15.39:0/1141728141 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe20155e80 0x7fbe2014a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:00.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.647+0000 7fbe26400640 1 -- 172.21.15.39:0/1141728141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe20155a70 msgr2=0x7fbe2015d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:00.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.647+0000 7fbe26400640 1 --2- 172.21.15.39:0/1141728141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe20155a70 0x7fbe2015d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:00.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.647+0000 7fbe26400640 1 -- 172.21.15.39:0/1141728141 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe2014b1f0 con 0x7fbe2015f360 2024-09-21T07:39:00.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.647+0000 7fbe26400640 1 --2- 172.21.15.39:0/1141728141 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe2015f360 0x7fbe2014d980 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fbe1800eab0 tx=0x7fbe1800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:00.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.647+0000 7fbe1d600640 1 -- 172.21.15.39:0/1141728141 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1800cea0 con 0x7fbe2015f360 2024-09-21T07:39:00.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.647+0000 7fbe1d600640 1 -- 172.21.15.39:0/1141728141 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe18004b20 con 0x7fbe2015f360 2024-09-21T07:39:00.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.647+0000 7fbe1d600640 1 -- 172.21.15.39:0/1141728141 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe18005020 con 0x7fbe2015f360 2024-09-21T07:39:00.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.647+0000 7fbe28400640 1 -- 172.21.15.39:0/1141728141 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe200aa650 con 0x7fbe2015f360 2024-09-21T07:39:00.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.648+0000 7fbe28400640 1 -- 172.21.15.39:0/1141728141 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe200aac10 con 0x7fbe2015f360 2024-09-21T07:39:00.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.649+0000 7fbe28400640 1 -- 172.21.15.39:0/1141728141 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe2014af60 con 0x7fbe2015f360 2024-09-21T07:39:00.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.650+0000 7fbe1d600640 1 -- 172.21.15.39:0/1141728141 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbe180051c0 con 0x7fbe2015f360 2024-09-21T07:39:00.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.651+0000 7fbe1d600640 1 --2- 172.21.15.39:0/1141728141 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbe00078b90 0x7fbe0007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:00.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.651+0000 7fbe1d600640 1 -- 172.21.15.39:0/1141728141 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 7217+0+0 (secure 0 0 0) 0x7fbe1809bf70 con 0x7fbe2015f360 2024-09-21T07:39:00.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.651+0000 7fbe25a00640 1 --2- 172.21.15.39:0/1141728141 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbe00078b90 0x7fbe0007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:00.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.652+0000 7fbe25a00640 1 --2- 172.21.15.39:0/1141728141 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbe00078b90 0x7fbe0007b0a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbe14034000 tx=0x7fbe14004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:00.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.656+0000 7fbe1d600640 1 -- 172.21.15.39:0/1141728141 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbe180636d0 con 0x7fbe2015f360 2024-09-21T07:39:00.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.894+0000 7fbe28400640 1 -- 172.21.15.39:0/1141728141 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe200708d0 con 0x7fbe2015f360 2024-09-21T07:39:00.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.895+0000 7fbe1d600640 1 -- 172.21.15.39:0/1141728141 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5243 (secure 0 0 0) 0x7fbe18068100 con 0x7fbe2015f360 2024-09-21T07:39:00.897 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:39:00.897 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":92,"btime":"2024-09-21T07:38:43:259438+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/992188984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":992188984},{"type":"v1","addr":"172.21.15.148:6837","nonce":992188984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:38:42.806952+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6825,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6825":{"gid":6825,"name":"c","rank":0,"incarnation":86,"state":"up:active","state_seq":13,"addr":"172.21.15.39:6834/855150775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":855150775},{"type":"v1","addr":"172.21.15.39:6834","nonce":855150775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6825,"qdb_cluster":[6115,6825]},"id":6}]} 2024-09-21T07:39:00.897 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 92 2024-09-21T07:39:00.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.898+0000 7fbe28400640 1 -- 172.21.15.39:0/1141728141 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbe00078b90 msgr2=0x7fbe0007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:00.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.898+0000 7fbe28400640 1 --2- 172.21.15.39:0/1141728141 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbe00078b90 0x7fbe0007b0a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbe14034000 tx=0x7fbe14004730 comp rx=0 tx=0).stop 2024-09-21T07:39:00.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.898+0000 7fbe28400640 1 -- 172.21.15.39:0/1141728141 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe2015f360 msgr2=0x7fbe2014d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:00.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.898+0000 7fbe28400640 1 --2- 172.21.15.39:0/1141728141 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe2015f360 0x7fbe2014d980 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fbe1800eab0 tx=0x7fbe1800ef70 comp rx=0 tx=0).stop 2024-09-21T07:39:00.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.898+0000 7fbe28400640 1 -- 172.21.15.39:0/1141728141 shutdown_connections 2024-09-21T07:39:00.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.898+0000 7fbe28400640 1 --2- 172.21.15.39:0/1141728141 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbe00078b90 0x7fbe0007b0a0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:00.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.898+0000 7fbe28400640 1 --2- 172.21.15.39:0/1141728141 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe2015f360 0x7fbe2014d980 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:00.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.898+0000 7fbe28400640 1 --2- 172.21.15.39:0/1141728141 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe20155e80 0x7fbe2014a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:00.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.898+0000 7fbe28400640 1 --2- 172.21.15.39:0/1141728141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe20155a70 0x7fbe2015d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:00.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.898+0000 7fbe28400640 1 -- 172.21.15.39:0/1141728141 >> 172.21.15.39:0/1141728141 conn(0x7fbe2006cfc0 msgr2=0x7fbe200cf620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:00.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.898+0000 7fbe28400640 1 -- 172.21.15.39:0/1141728141 shutdown_connections 2024-09-21T07:39:00.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:00.898+0000 7fbe28400640 1 -- 172.21.15.39:0/1141728141 wait complete. 2024-09-21T07:39:00.908 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:39:00.912 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:39:00.922 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:39:02.758 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~13s 2024-09-21T07:39:05.923 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:39:06.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.157+0000 7f1818e00640 1 -- 172.21.15.39:0/3462479506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18140d0f20 msgr2=0x7f181415c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:06.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.157+0000 7f1818e00640 1 --2- 172.21.15.39:0/3462479506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18140d0f20 0x7f181415c830 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f18140cdda0 tx=0x7f180402a530 comp rx=0 tx=0).stop 2024-09-21T07:39:06.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.157+0000 7f1818e00640 1 -- 172.21.15.39:0/3462479506 shutdown_connections 2024-09-21T07:39:06.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.157+0000 7f1818e00640 1 --2- 172.21.15.39:0/3462479506 >> v2:172.21.15.148:3300/0 conn(0x7f18140cbd70 0x7f1814072fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:06.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.157+0000 7f1818e00640 1 --2- 172.21.15.39:0/3462479506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18140d0f20 0x7f181415c830 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:06.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.157+0000 7f1818e00640 1 -- 172.21.15.39:0/3462479506 >> 172.21.15.39:0/3462479506 conn(0x7f18140b2510 msgr2=0x7f18140d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:06.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.157+0000 7f1818e00640 1 -- 172.21.15.39:0/3462479506 shutdown_connections 2024-09-21T07:39:06.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.157+0000 7f1818e00640 1 -- 172.21.15.39:0/3462479506 wait complete. 2024-09-21T07:39:06.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.159+0000 7f1818e00640 1 Processor -- start 2024-09-21T07:39:06.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.159+0000 7f1818e00640 1 -- start start 2024-09-21T07:39:06.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.161+0000 7f1818e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18140cb960 0x7f18141593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:06.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.161+0000 7f1818e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18140cbd70 0x7f181409b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:06.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.161+0000 7f1818e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f18140d0f20 0x7f181409b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:06.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.161+0000 7f1818e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f18140b43f0 con 0x7f18140cb960 2024-09-21T07:39:06.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.161+0000 7f1818e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1814073830 con 0x7f18140cbd70 2024-09-21T07:39:06.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.161+0000 7f1818e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f18140b4570 con 0x7f18140d0f20 2024-09-21T07:39:06.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.161+0000 7f1813400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18140cb960 0x7f18141593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:06.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.161+0000 7f1813400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18140cb960 0x7f18141593d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35806/0 (socket says 172.21.15.39:35806) 2024-09-21T07:39:06.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.161+0000 7f1813400640 1 -- 172.21.15.39:0/3034761487 learned_addr learned my addr 172.21.15.39:0/3034761487 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:39:06.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.161+0000 7f1813400640 1 -- 172.21.15.39:0/3034761487 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f18140d0f20 msgr2=0x7f181409b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:06.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.161+0000 7f1813400640 1 --2- 172.21.15.39:0/3034761487 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f18140d0f20 0x7f181409b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:06.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.161+0000 7f1813400640 1 -- 172.21.15.39:0/3034761487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18140cbd70 msgr2=0x7f181409b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:06.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.161+0000 7f180aa00640 1 --2- 172.21.15.39:0/3034761487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18140cbd70 0x7f181409b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:06.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.161+0000 7f1813400640 1 --2- 172.21.15.39:0/3034761487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18140cbd70 0x7f181409b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:06.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.161+0000 7f1813400640 1 -- 172.21.15.39:0/3034761487 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1814159910 con 0x7f18140cb960 2024-09-21T07:39:06.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.161+0000 7f180aa00640 1 --2- 172.21.15.39:0/3034761487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18140cbd70 0x7f181409b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:39:06.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.162+0000 7f1813400640 1 --2- 172.21.15.39:0/3034761487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18140cb960 0x7f18141593d0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f180402ab20 tx=0x7f1804033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:06.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.162+0000 7f180be00640 1 -- 172.21.15.39:0/3034761487 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1804005c90 con 0x7f18140cb960 2024-09-21T07:39:06.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.162+0000 7f180be00640 1 -- 172.21.15.39:0/3034761487 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1804031e50 con 0x7f18140cb960 2024-09-21T07:39:06.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.162+0000 7f1818e00640 1 -- 172.21.15.39:0/3034761487 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18140ad490 con 0x7f18140cb960 2024-09-21T07:39:06.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.162+0000 7f180be00640 1 -- 172.21.15.39:0/3034761487 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1804005260 con 0x7f18140cb960 2024-09-21T07:39:06.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.162+0000 7f1818e00640 1 -- 172.21.15.39:0/3034761487 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18140ada50 con 0x7f18140cb960 2024-09-21T07:39:06.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.163+0000 7f1818e00640 1 -- 172.21.15.39:0/3034761487 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f181415c290 con 0x7f18140cb960 2024-09-21T07:39:06.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.165+0000 7f180be00640 1 -- 172.21.15.39:0/3034761487 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1804005400 con 0x7f18140cb960 2024-09-21T07:39:06.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.166+0000 7f180be00640 1 --2- 172.21.15.39:0/3034761487 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f17f0078bc0 0x7f17f007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:06.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.166+0000 7f180be00640 1 -- 172.21.15.39:0/3034761487 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 7217+0+0 (secure 0 0 0) 0x7f18040bc490 con 0x7f18140cb960 2024-09-21T07:39:06.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.166+0000 7f180aa00640 1 --2- 172.21.15.39:0/3034761487 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f17f0078bc0 0x7f17f007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:06.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.167+0000 7f180aa00640 1 --2- 172.21.15.39:0/3034761487 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f17f0078bc0 0x7f17f007b0d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f181409e1b0 tx=0x7f17fc009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:06.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.170+0000 7f180be00640 1 -- 172.21.15.39:0/3034761487 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f181415c290 con 0x7f18140cb960 2024-09-21T07:39:06.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.406+0000 7f1818e00640 1 -- 172.21.15.39:0/3034761487 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18140b4b00 con 0x7f18140cb960 2024-09-21T07:39:06.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.407+0000 7f180be00640 1 -- 172.21.15.39:0/3034761487 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) ==== 94+0+5291 (secure 0 0 0) 0x7f1814159910 con 0x7f18140cb960 2024-09-21T07:39:06.409 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:39:06.409 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":93,"btime":"2024-09-21T07:39:01:391717+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/992188984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":992188984},{"type":"v1","addr":"172.21.15.148:6837","nonce":992188984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:39:01.391685+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6825,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6825":{"gid":6825,"name":"c","rank":0,"incarnation":86,"state":"up:active","state_seq":13,"addr":"172.21.15.39:6834/855150775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":855150775},{"type":"v1","addr":"172.21.15.39:6834","nonce":855150775}]},"join_fscid":-1,"laggy_since":"2024-09-21T07:39:01.391685+0000","export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6825,"qdb_cluster":[6825,6115]},"id":6}]} 2024-09-21T07:39:06.409 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 93 2024-09-21T07:39:06.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.409+0000 7f1818e00640 1 -- 172.21.15.39:0/3034761487 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f17f0078bc0 msgr2=0x7f17f007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:06.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.409+0000 7f1818e00640 1 --2- 172.21.15.39:0/3034761487 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f17f0078bc0 0x7f17f007b0d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f181409e1b0 tx=0x7f17fc009040 comp rx=0 tx=0).stop 2024-09-21T07:39:06.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.409+0000 7f1818e00640 1 -- 172.21.15.39:0/3034761487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18140cb960 msgr2=0x7f18141593d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:06.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.409+0000 7f1818e00640 1 --2- 172.21.15.39:0/3034761487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18140cb960 0x7f18141593d0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f180402ab20 tx=0x7f1804033c20 comp rx=0 tx=0).stop 2024-09-21T07:39:06.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.409+0000 7f1818e00640 1 -- 172.21.15.39:0/3034761487 shutdown_connections 2024-09-21T07:39:06.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.409+0000 7f1818e00640 1 --2- 172.21.15.39:0/3034761487 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f17f0078bc0 0x7f17f007b0d0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:06.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.409+0000 7f1818e00640 1 --2- 172.21.15.39:0/3034761487 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f18140d0f20 0x7f181409b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:06.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.409+0000 7f1818e00640 1 --2- 172.21.15.39:0/3034761487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18140cbd70 0x7f181409b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:06.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.409+0000 7f1818e00640 1 --2- 172.21.15.39:0/3034761487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18140cb960 0x7f18141593d0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:06.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.409+0000 7f1818e00640 1 -- 172.21.15.39:0/3034761487 >> 172.21.15.39:0/3034761487 conn(0x7f18140b2510 msgr2=0x7f181415e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:06.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.409+0000 7f1818e00640 1 -- 172.21.15.39:0/3034761487 shutdown_connections 2024-09-21T07:39:06.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.409+0000 7f1818e00640 1 -- 172.21.15.39:0/3034761487 wait complete. 2024-09-21T07:39:06.434 DEBUG:tasks.ceph_test_case:wait_until_true: success in 20s and 0 retries 2024-09-21T07:39:06.435 DEBUG:teuthology.orchestra.run.smithi039:> sudo sysctl -n kernel.core_pattern 2024-09-21T07:39:06.488 INFO:teuthology.orchestra.run.smithi039.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T07:39:06.491 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2024-09-21T07:39:06.491 DEBUG:teuthology.orchestra.run.smithi039:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2024-09-21T07:39:06.552 INFO:teuthology.orchestra.run.smithi039.stdout:1726904323.60214.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i c', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2024-09-21T07:39:06.556 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1726904323.60214.core'] 2024-09-21T07:39:06.556 INFO:tasks.cephfs.cephfs_test_case:Found core file 1726904323.60214.core, deleting it 2024-09-21T07:39:06.556 DEBUG:teuthology.orchestra.run.smithi039:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1726904323.60214.core 2024-09-21T07:39:06.648 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:39:06.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.873+0000 7f19d2200640 1 -- 172.21.15.39:0/1411949143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19cc0d0620 msgr2=0x7f19cc0d0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:06.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.873+0000 7f19d2200640 1 --2- 172.21.15.39:0/1411949143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19cc0d0620 0x7f19cc0d0a30 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f19c4005fd0 tx=0x7f19c402aa90 comp rx=0 tx=0).stop 2024-09-21T07:39:06.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.873+0000 7f19d2200640 1 -- 172.21.15.39:0/1411949143 shutdown_connections 2024-09-21T07:39:06.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.873+0000 7f19d2200640 1 --2- 172.21.15.39:0/1411949143 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f19cc0cfd10 0x7f19cc0cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:06.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.873+0000 7f19d2200640 1 --2- 172.21.15.39:0/1411949143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19cc0d0620 0x7f19cc0d0a30 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:06.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.873+0000 7f19d2200640 1 -- 172.21.15.39:0/1411949143 >> 172.21.15.39:0/1411949143 conn(0x7f19cc06cfc0 msgr2=0x7f19cc06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:06.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.873+0000 7f19d2200640 1 -- 172.21.15.39:0/1411949143 shutdown_connections 2024-09-21T07:39:06.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.873+0000 7f19d2200640 1 -- 172.21.15.39:0/1411949143 wait complete. 2024-09-21T07:39:06.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.875+0000 7f19d2200640 1 Processor -- start 2024-09-21T07:39:06.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.875+0000 7f19d2200640 1 -- start start 2024-09-21T07:39:06.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.877+0000 7f19d2200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f19cc0cfd10 0x7f19cc152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:06.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.877+0000 7f19d2200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19cc0d0620 0x7f19cc1591a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:06.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.877+0000 7f19d2200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19cc15cd30 0x7f19cc1596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:06.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.877+0000 7f19d2200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f19cc073890 con 0x7f19cc15cd30 2024-09-21T07:39:06.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.877+0000 7f19d2200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f19cc073710 con 0x7f19cc0d0620 2024-09-21T07:39:06.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.877+0000 7f19d2200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f19cc073a10 con 0x7f19cc0cfd10 2024-09-21T07:39:06.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.877+0000 7f19cbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19cc15cd30 0x7f19cc1596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:06.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.877+0000 7f19cbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19cc15cd30 0x7f19cc1596e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35826/0 (socket says 172.21.15.39:35826) 2024-09-21T07:39:06.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.877+0000 7f19cbe00640 1 -- 172.21.15.39:0/1831159483 learned_addr learned my addr 172.21.15.39:0/1831159483 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:39:06.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.877+0000 7f19cb400640 1 --2- 172.21.15.39:0/1831159483 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f19cc0cfd10 0x7f19cc152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:06.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.877+0000 7f19c3e00640 1 --2- 172.21.15.39:0/1831159483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19cc0d0620 0x7f19cc1591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:06.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.878+0000 7f19cbe00640 1 -- 172.21.15.39:0/1831159483 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f19cc0cfd10 msgr2=0x7f19cc152440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:06.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.878+0000 7f19cbe00640 1 --2- 172.21.15.39:0/1831159483 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f19cc0cfd10 0x7f19cc152440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:06.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.878+0000 7f19cbe00640 1 -- 172.21.15.39:0/1831159483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19cc0d0620 msgr2=0x7f19cc1591a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:06.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.878+0000 7f19cbe00640 1 --2- 172.21.15.39:0/1831159483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19cc0d0620 0x7f19cc1591a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:06.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.878+0000 7f19cbe00640 1 -- 172.21.15.39:0/1831159483 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19cc1f6520 con 0x7f19cc15cd30 2024-09-21T07:39:06.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.878+0000 7f19cbe00640 1 --2- 172.21.15.39:0/1831159483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19cc15cd30 0x7f19cc1596e0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f19c4009d90 tx=0x7f19c4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:06.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.878+0000 7f19c3400640 1 -- 172.21.15.39:0/1831159483 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c4044070 con 0x7f19cc15cd30 2024-09-21T07:39:06.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.878+0000 7f19c3400640 1 -- 172.21.15.39:0/1831159483 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19c4003e00 con 0x7f19cc15cd30 2024-09-21T07:39:06.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.878+0000 7f19c3400640 1 -- 172.21.15.39:0/1831159483 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c40044b0 con 0x7f19cc15cd30 2024-09-21T07:39:06.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.878+0000 7f19d2200640 1 -- 172.21.15.39:0/1831159483 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19cc0b4490 con 0x7f19cc15cd30 2024-09-21T07:39:06.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.878+0000 7f19d2200640 1 -- 172.21.15.39:0/1831159483 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19cc0b4a50 con 0x7f19cc15cd30 2024-09-21T07:39:06.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.880+0000 7f19d2200640 1 -- 172.21.15.39:0/1831159483 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19cc073f30 con 0x7f19cc15cd30 2024-09-21T07:39:06.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.881+0000 7f19c3400640 1 -- 172.21.15.39:0/1831159483 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f19c4047070 con 0x7f19cc15cd30 2024-09-21T07:39:06.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.882+0000 7f19c3400640 1 --2- 172.21.15.39:0/1831159483 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f19a8078b90 0x7f19a807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:06.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.882+0000 7f19c3400640 1 -- 172.21.15.39:0/1831159483 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 7217+0+0 (secure 0 0 0) 0x7f19c40bc0a0 con 0x7f19cc15cd30 2024-09-21T07:39:06.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.882+0000 7f19cb400640 1 --2- 172.21.15.39:0/1831159483 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f19a8078b90 0x7f19a807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:06.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.883+0000 7f19cb400640 1 --2- 172.21.15.39:0/1831159483 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f19a8078b90 0x7f19a807b0a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f19bc001720 tx=0x7f19bc009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:06.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:06.887+0000 7f19c3400640 1 -- 172.21.15.39:0/1831159483 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f19c4032030 con 0x7f19cc15cd30 2024-09-21T07:39:07.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.121+0000 7f19d2200640 1 -- 172.21.15.39:0/1831159483 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19cc152980 con 0x7f19cc15cd30 2024-09-21T07:39:07.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.121+0000 7f19c3400640 1 -- 172.21.15.39:0/1831159483 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) ==== 94+0+5291 (secure 0 0 0) 0x7f19c40098b0 con 0x7f19cc15cd30 2024-09-21T07:39:07.123 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:39:07.123 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":93,"btime":"2024-09-21T07:39:01:391717+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6619,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/992188984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":992188984},{"type":"v1","addr":"172.21.15.148:6837","nonce":992188984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:39:01.391685+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6825,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6825":{"gid":6825,"name":"c","rank":0,"incarnation":86,"state":"up:active","state_seq":13,"addr":"172.21.15.39:6834/855150775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":855150775},{"type":"v1","addr":"172.21.15.39:6834","nonce":855150775}]},"join_fscid":-1,"laggy_since":"2024-09-21T07:39:01.391685+0000","export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6825,"qdb_cluster":[6825,6115]},"id":6}]} 2024-09-21T07:39:07.124 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 93 2024-09-21T07:39:07.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.124+0000 7f19d2200640 1 -- 172.21.15.39:0/1831159483 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f19a8078b90 msgr2=0x7f19a807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:07.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.124+0000 7f19d2200640 1 --2- 172.21.15.39:0/1831159483 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f19a8078b90 0x7f19a807b0a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f19bc001720 tx=0x7f19bc009040 comp rx=0 tx=0).stop 2024-09-21T07:39:07.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.124+0000 7f19d2200640 1 -- 172.21.15.39:0/1831159483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19cc15cd30 msgr2=0x7f19cc1596e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:07.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.124+0000 7f19d2200640 1 --2- 172.21.15.39:0/1831159483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19cc15cd30 0x7f19cc1596e0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f19c4009d90 tx=0x7f19c4003c10 comp rx=0 tx=0).stop 2024-09-21T07:39:07.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.124+0000 7f19d2200640 1 -- 172.21.15.39:0/1831159483 shutdown_connections 2024-09-21T07:39:07.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.124+0000 7f19d2200640 1 --2- 172.21.15.39:0/1831159483 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f19a8078b90 0x7f19a807b0a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:07.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.124+0000 7f19d2200640 1 --2- 172.21.15.39:0/1831159483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19cc15cd30 0x7f19cc1596e0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:07.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.124+0000 7f19d2200640 1 --2- 172.21.15.39:0/1831159483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19cc0d0620 0x7f19cc1591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:07.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.124+0000 7f19d2200640 1 --2- 172.21.15.39:0/1831159483 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f19cc0cfd10 0x7f19cc152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:07.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.124+0000 7f19d2200640 1 -- 172.21.15.39:0/1831159483 >> 172.21.15.39:0/1831159483 conn(0x7f19cc06cfc0 msgr2=0x7f19cc070f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:07.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.124+0000 7f19d2200640 1 -- 172.21.15.39:0/1831159483 shutdown_connections 2024-09-21T07:39:07.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.124+0000 7f19d2200640 1 -- 172.21.15.39:0/1831159483 wait complete. 2024-09-21T07:39:07.151 INFO:tasks.ceph.mds.a:Sent signal 9 2024-09-21T07:39:07.152 INFO:tasks.cephfs.mount:Suspending the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:39:07.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:39:07.152 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 down 2024-09-21T07:39:07.217 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 6:0 2024-09-21T07:39:07.353 INFO:tasks.ceph.mds.a.smithi039.stderr:daemon-helper: command crashed with signal 9 2024-09-21T07:39:07.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.447+0000 7fbcffe00640 1 -- 172.21.15.39:0/1076507789 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbd000b4db0 msgr2=0x7fbd000d4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:07.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.447+0000 7fbcffe00640 1 --2- 172.21.15.39:0/1076507789 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbd000b4db0 0x7fbd000d4c90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbcf00098e0 tx=0x7fbcf002a5d0 comp rx=0 tx=0).stop 2024-09-21T07:39:07.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.447+0000 7fbcffe00640 1 -- 172.21.15.39:0/1076507789 shutdown_connections 2024-09-21T07:39:07.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.447+0000 7fbcffe00640 1 --2- 172.21.15.39:0/1076507789 >> v2:172.21.15.148:3300/0 conn(0x7fbd000d2a50 0x7fbd000d2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:07.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.447+0000 7fbcffe00640 1 --2- 172.21.15.39:0/1076507789 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbd000b4db0 0x7fbd000d4c90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:07.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.447+0000 7fbcffe00640 1 -- 172.21.15.39:0/1076507789 >> 172.21.15.39:0/1076507789 conn(0x7fbd0008bfb0 msgr2=0x7fbd000d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:07.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.447+0000 7fbcffe00640 1 -- 172.21.15.39:0/1076507789 shutdown_connections 2024-09-21T07:39:07.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.447+0000 7fbcffe00640 1 -- 172.21.15.39:0/1076507789 wait complete. 2024-09-21T07:39:07.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.449+0000 7fbcffe00640 1 Processor -- start 2024-09-21T07:39:07.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.449+0000 7fbcffe00640 1 -- start start 2024-09-21T07:39:07.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.452+0000 7fbcffe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd000b4db0 0x7fbd000cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:07.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.452+0000 7fbcffe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd000b6000 0x7fbd0015ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:07.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.452+0000 7fbcffe00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbd000d2a50 0x7fbd000cfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:07.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.452+0000 7fbcffe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbd000cee90 con 0x7fbd000b4db0 2024-09-21T07:39:07.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.452+0000 7fbcffe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbd000b6d70 con 0x7fbd000b6000 2024-09-21T07:39:07.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.452+0000 7fbcffe00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fbd000cf010 con 0x7fbd000d2a50 2024-09-21T07:39:07.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.452+0000 7fbcfea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd000b4db0 0x7fbd000cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:07.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.452+0000 7fbcfea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd000b4db0 0x7fbd000cbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35834/0 (socket says 172.21.15.39:35834) 2024-09-21T07:39:07.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.452+0000 7fbcfea00640 1 -- 172.21.15.39:0/3094844574 learned_addr learned my addr 172.21.15.39:0/3094844574 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:39:07.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.452+0000 7fbcfe000640 1 --2- 172.21.15.39:0/3094844574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd000b6000 0x7fbd0015ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:07.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.452+0000 7fbcff400640 1 --2- 172.21.15.39:0/3094844574 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbd000d2a50 0x7fbd000cfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:07.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.452+0000 7fbcfea00640 1 -- 172.21.15.39:0/3094844574 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbd000d2a50 msgr2=0x7fbd000cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:07.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.452+0000 7fbcfea00640 1 --2- 172.21.15.39:0/3094844574 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbd000d2a50 0x7fbd000cfbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:07.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.452+0000 7fbcfea00640 1 -- 172.21.15.39:0/3094844574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd000b6000 msgr2=0x7fbd0015ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:07.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.452+0000 7fbcfea00640 1 --2- 172.21.15.39:0/3094844574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd000b6000 0x7fbd0015ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:07.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.452+0000 7fbcfea00640 1 -- 172.21.15.39:0/3094844574 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd000cc230 con 0x7fbd000b4db0 2024-09-21T07:39:07.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.452+0000 7fbcff400640 1 --2- 172.21.15.39:0/3094844574 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbd000d2a50 0x7fbd000cfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:39:07.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.452+0000 7fbcfea00640 1 --2- 172.21.15.39:0/3094844574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd000b4db0 0x7fbd000cbcf0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0009a00 tx=0x7fbcf00316b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:07.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.452+0000 7fbcf6a00640 1 -- 172.21.15.39:0/3094844574 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf003f070 con 0x7fbd000b4db0 2024-09-21T07:39:07.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.452+0000 7fbcf6a00640 1 -- 172.21.15.39:0/3094844574 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbcf0031950 con 0x7fbd000b4db0 2024-09-21T07:39:07.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.452+0000 7fbcffe00640 1 -- 172.21.15.39:0/3094844574 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd000d0d00 con 0x7fbd000b4db0 2024-09-21T07:39:07.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.452+0000 7fbcf6a00640 1 -- 172.21.15.39:0/3094844574 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0031da0 con 0x7fbd000b4db0 2024-09-21T07:39:07.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.452+0000 7fbcffe00640 1 -- 172.21.15.39:0/3094844574 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd000d14e0 con 0x7fbd000b4db0 2024-09-21T07:39:07.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.453+0000 7fbcffe00640 1 -- 172.21.15.39:0/3094844574 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd000d4c90 con 0x7fbd000b4db0 2024-09-21T07:39:07.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.455+0000 7fbcf6a00640 1 -- 172.21.15.39:0/3094844574 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbcf0042070 con 0x7fbd000b4db0 2024-09-21T07:39:07.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.455+0000 7fbcf6a00640 1 --2- 172.21.15.39:0/3094844574 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbce0078bc0 0x7fbce007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:07.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.456+0000 7fbcf6a00640 1 -- 172.21.15.39:0/3094844574 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 7217+0+0 (secure 0 0 0) 0x7fbcf00bbea0 con 0x7fbd000b4db0 2024-09-21T07:39:07.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.456+0000 7fbcfe000640 1 --2- 172.21.15.39:0/3094844574 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbce0078bc0 0x7fbce007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:07.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.457+0000 7fbcfe000640 1 --2- 172.21.15.39:0/3094844574 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbce0078bc0 0x7fbce007b0d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fbcec009590 tx=0x7fbcec005f30 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:07.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.461+0000 7fbcf6a00640 1 -- 172.21.15.39:0/3094844574 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbcf0083730 con 0x7fbd000b4db0 2024-09-21T07:39:07.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.737+0000 7fbcffe00640 1 -- 172.21.15.39:0/3094844574 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "6:0"} v 0) -- 0x7fbd0006e920 con 0x7fbd000b4db0 2024-09-21T07:39:07.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.742+0000 7fbcf6a00640 1 -- 172.21.15.39:0/3094844574 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "6:0"}]=0 failed mds gid 6825 v94) ==== 97+0+0 (secure 0 0 0) 0x7fbcf0088160 con 0x7fbd000b4db0 2024-09-21T07:39:07.743 INFO:teuthology.orchestra.run.smithi039.stderr:failed mds gid 6825 2024-09-21T07:39:07.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.744+0000 7fbcffe00640 1 -- 172.21.15.39:0/3094844574 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbce0078bc0 msgr2=0x7fbce007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:07.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.744+0000 7fbcffe00640 1 --2- 172.21.15.39:0/3094844574 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbce0078bc0 0x7fbce007b0d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fbcec009590 tx=0x7fbcec005f30 comp rx=0 tx=0).stop 2024-09-21T07:39:07.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.744+0000 7fbcffe00640 1 -- 172.21.15.39:0/3094844574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd000b4db0 msgr2=0x7fbd000cbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:07.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.744+0000 7fbcffe00640 1 --2- 172.21.15.39:0/3094844574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd000b4db0 0x7fbd000cbcf0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0009a00 tx=0x7fbcf00316b0 comp rx=0 tx=0).stop 2024-09-21T07:39:07.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.745+0000 7fbcffe00640 1 -- 172.21.15.39:0/3094844574 shutdown_connections 2024-09-21T07:39:07.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.745+0000 7fbcffe00640 1 --2- 172.21.15.39:0/3094844574 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbce0078bc0 0x7fbce007b0d0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:07.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.745+0000 7fbcffe00640 1 --2- 172.21.15.39:0/3094844574 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbd000d2a50 0x7fbd000cfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:07.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.745+0000 7fbcffe00640 1 --2- 172.21.15.39:0/3094844574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd000b6000 0x7fbd0015ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:07.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.745+0000 7fbcffe00640 1 --2- 172.21.15.39:0/3094844574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd000b4db0 0x7fbd000cbcf0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:07.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.745+0000 7fbcffe00640 1 -- 172.21.15.39:0/3094844574 >> 172.21.15.39:0/3094844574 conn(0x7fbd0008bfb0 msgr2=0x7fbd000d4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:07.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.746+0000 7fbcffe00640 1 -- 172.21.15.39:0/3094844574 shutdown_connections 2024-09-21T07:39:07.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:07.747+0000 7fbcffe00640 1 -- 172.21.15.39:0/3094844574 wait complete. 2024-09-21T07:39:07.778 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T07:39:07.778 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T07:39:07.779 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T07:39:07.779 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:39:07.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:39:07.779 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/daemon/state.py", line 146, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi039 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c' 2024-09-21T07:39:07.779 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T07:39:07.780 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T07:39:07.784 INFO:tasks.ceph.mds.c:Started 2024-09-21T07:39:07.784 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:39:07.982 INFO:tasks.ceph.mds.c.smithi039.stdout:starting mds.c at 2024-09-21T07:39:08.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.011+0000 7fb8f3200640 1 -- 172.21.15.39:0/2260084295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8ec15c520 msgr2=0x7fb8ec0b3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:08.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.011+0000 7fb8f3200640 1 --2- 172.21.15.39:0/2260084295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8ec15c520 0x7fb8ec0b3ae0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0009830 tx=0x7fb8e002a5a0 comp rx=0 tx=0).stop 2024-09-21T07:39:08.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.011+0000 7fb8f3200640 1 -- 172.21.15.39:0/2260084295 shutdown_connections 2024-09-21T07:39:08.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.011+0000 7fb8f3200640 1 --2- 172.21.15.39:0/2260084295 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8ec0b4020 0x7fb8ec0b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:08.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.011+0000 7fb8f3200640 1 --2- 172.21.15.39:0/2260084295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8ec15c520 0x7fb8ec0b3ae0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:08.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.011+0000 7fb8f3200640 1 -- 172.21.15.39:0/2260084295 >> 172.21.15.39:0/2260084295 conn(0x7fb8ec074050 msgr2=0x7fb8ec074490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:08.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.011+0000 7fb8f3200640 1 -- 172.21.15.39:0/2260084295 shutdown_connections 2024-09-21T07:39:08.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.011+0000 7fb8f3200640 1 -- 172.21.15.39:0/2260084295 wait complete. 2024-09-21T07:39:08.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.013+0000 7fb8f3200640 1 Processor -- start 2024-09-21T07:39:08.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.014+0000 7fb8f3200640 1 -- start start 2024-09-21T07:39:08.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.015+0000 7fb8f3200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8ec0b4020 0x7fb8ec13ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:08.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.015+0000 7fb8f3200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8ec15c520 0x7fb8ec141440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:08.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.015+0000 7fb8f3200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8ec160150 0x7fb8ec141980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:08.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.015+0000 7fb8f3200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec0cf360 con 0x7fb8ec160150 2024-09-21T07:39:08.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.015+0000 7fb8f3200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ec0cf1e0 con 0x7fb8ec0b4020 2024-09-21T07:39:08.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.015+0000 7fb8f3200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb8ec0cf4e0 con 0x7fb8ec15c520 2024-09-21T07:39:08.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.015+0000 7fb8f2800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8ec160150 0x7fb8ec141980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:08.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.015+0000 7fb8f2800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8ec160150 0x7fb8ec141980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35866/0 (socket says 172.21.15.39:35866) 2024-09-21T07:39:08.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.015+0000 7fb8f2800640 1 -- 172.21.15.39:0/4228774706 learned_addr learned my addr 172.21.15.39:0/4228774706 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:39:08.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.016+0000 7fb8f1e00640 1 --2- 172.21.15.39:0/4228774706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8ec0b4020 0x7fb8ec13ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:08.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.016+0000 7fb8f1400640 1 --2- 172.21.15.39:0/4228774706 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8ec15c520 0x7fb8ec141440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:08.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.016+0000 7fb8f2800640 1 -- 172.21.15.39:0/4228774706 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8ec15c520 msgr2=0x7fb8ec141440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:08.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.016+0000 7fb8f2800640 1 --2- 172.21.15.39:0/4228774706 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8ec15c520 0x7fb8ec141440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:08.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.016+0000 7fb8f2800640 1 -- 172.21.15.39:0/4228774706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8ec0b4020 msgr2=0x7fb8ec13ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:08.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.016+0000 7fb8f2800640 1 --2- 172.21.15.39:0/4228774706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8ec0b4020 0x7fb8ec13ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:08.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.016+0000 7fb8f2800640 1 -- 172.21.15.39:0/4228774706 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8ec152c60 con 0x7fb8ec160150 2024-09-21T07:39:08.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.016+0000 7fb8f2800640 1 --2- 172.21.15.39:0/4228774706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8ec160150 0x7fb8ec141980 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fb8e800cb40 tx=0x7fb8e800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:08.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.017+0000 7fb8e6000640 1 -- 172.21.15.39:0/4228774706 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8016020 con 0x7fb8ec160150 2024-09-21T07:39:08.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.017+0000 7fb8e6000640 1 -- 172.21.15.39:0/4228774706 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8e800adb0 con 0x7fb8ec160150 2024-09-21T07:39:08.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.017+0000 7fb8e6000640 1 -- 172.21.15.39:0/4228774706 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8012680 con 0x7fb8ec160150 2024-09-21T07:39:08.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.017+0000 7fb8f3200640 1 -- 172.21.15.39:0/4228774706 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8ec154920 con 0x7fb8ec160150 2024-09-21T07:39:08.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.017+0000 7fb8f3200640 1 -- 172.21.15.39:0/4228774706 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8ec154ee0 con 0x7fb8ec160150 2024-09-21T07:39:08.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.018+0000 7fb8f3200640 1 -- 172.21.15.39:0/4228774706 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8ec154690 con 0x7fb8ec160150 2024-09-21T07:39:08.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.019+0000 7fb8e6000640 1 -- 172.21.15.39:0/4228774706 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb8e8019020 con 0x7fb8ec160150 2024-09-21T07:39:08.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.020+0000 7fb8e6000640 1 --2- 172.21.15.39:0/4228774706 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb8d0078b90 0x7fb8d007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:08.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.020+0000 7fb8e6000640 1 -- 172.21.15.39:0/4228774706 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 7259+0+0 (secure 0 0 0) 0x7fb8e809b870 con 0x7fb8ec160150 2024-09-21T07:39:08.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.021+0000 7fb8f1e00640 1 --2- 172.21.15.39:0/4228774706 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb8d0078b90 0x7fb8d007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:08.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.021+0000 7fb8f1e00640 1 --2- 172.21.15.39:0/4228774706 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb8d0078b90 0x7fb8d007b0a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0034000 tx=0x7fb8e0004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:08.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.025+0000 7fb8e6000640 1 -- 172.21.15.39:0/4228774706 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb8e8020020 con 0x7fb8ec160150 2024-09-21T07:39:08.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.262+0000 7fb8f3200640 1 -- 172.21.15.39:0/4228774706 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8ec160f10 con 0x7fb8ec160150 2024-09-21T07:39:08.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.263+0000 7fb8e6000640 1 -- 172.21.15.39:0/4228774706 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) ==== 94+0+4415 (secure 0 0 0) 0x7fb8e8062fb0 con 0x7fb8ec160150 2024-09-21T07:39:08.264 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:39:08.264 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":95,"btime":"2024-09-21T07:39:07:746906+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:39:07.746883+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6619,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6619":{"gid":6619,"name":"d","rank":0,"incarnation":95,"state":"up:replay","state_seq":1,"addr":"172.21.15.148:6837/992188984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":992188984},{"type":"v1","addr":"172.21.15.148:6837","nonce":992188984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6115,"qdb_cluster":[6115]},"id":6}]} 2024-09-21T07:39:08.265 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 95 2024-09-21T07:39:08.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.265+0000 7fb8f3200640 1 -- 172.21.15.39:0/4228774706 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb8d0078b90 msgr2=0x7fb8d007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:08.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.265+0000 7fb8f3200640 1 --2- 172.21.15.39:0/4228774706 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb8d0078b90 0x7fb8d007b0a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0034000 tx=0x7fb8e0004730 comp rx=0 tx=0).stop 2024-09-21T07:39:08.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.265+0000 7fb8f3200640 1 -- 172.21.15.39:0/4228774706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8ec160150 msgr2=0x7fb8ec141980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:08.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.265+0000 7fb8f3200640 1 --2- 172.21.15.39:0/4228774706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8ec160150 0x7fb8ec141980 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fb8e800cb40 tx=0x7fb8e800a060 comp rx=0 tx=0).stop 2024-09-21T07:39:08.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.266+0000 7fb8f3200640 1 -- 172.21.15.39:0/4228774706 shutdown_connections 2024-09-21T07:39:08.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.266+0000 7fb8f3200640 1 --2- 172.21.15.39:0/4228774706 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb8d0078b90 0x7fb8d007b0a0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:08.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.266+0000 7fb8f3200640 1 --2- 172.21.15.39:0/4228774706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8ec160150 0x7fb8ec141980 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:08.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.266+0000 7fb8f3200640 1 --2- 172.21.15.39:0/4228774706 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8ec15c520 0x7fb8ec141440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:08.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.266+0000 7fb8f3200640 1 --2- 172.21.15.39:0/4228774706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8ec0b4020 0x7fb8ec13ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:08.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.266+0000 7fb8f3200640 1 -- 172.21.15.39:0/4228774706 >> 172.21.15.39:0/4228774706 conn(0x7fb8ec074050 msgr2=0x7fb8ec0ccce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:08.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.266+0000 7fb8f3200640 1 -- 172.21.15.39:0/4228774706 shutdown_connections 2024-09-21T07:39:08.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:08.266+0000 7fb8f3200640 1 -- 172.21.15.39:0/4228774706 wait complete. 2024-09-21T07:39:08.288 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-09-21T07:39:08.757 INFO:tasks.ceph.mds.d.smithi148.stderr:2024-09-21T07:39:08.755+0000 7f76a0400640 -1 --2- [v2:172.21.15.148:6836/992188984,v1:172.21.15.148:6837/992188984] >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x5642e3ba3680 0x5642e3d4a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] is using msgr V1 protocol 2024-09-21T07:39:08.958 INFO:tasks.ceph.mds.d.smithi148.stderr:2024-09-21T07:39:08.956+0000 7f76a0400640 -1 --2- [v2:172.21.15.148:6836/992188984,v1:172.21.15.148:6837/992188984] >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x5642e3ba3680 0x5642e3d4a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] is using msgr V1 protocol 2024-09-21T07:39:09.361 INFO:tasks.ceph.mds.d.smithi148.stderr:2024-09-21T07:39:09.358+0000 7f76a0400640 -1 --2- [v2:172.21.15.148:6836/992188984,v1:172.21.15.148:6837/992188984] >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x5642e3ba3680 0x5642e3d4a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] is using msgr V1 protocol 2024-09-21T07:39:09.370 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-09-21T07:39:09.370 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-09-21T07:39:10.162 INFO:tasks.ceph.mds.d.smithi148.stderr:2024-09-21T07:39:10.160+0000 7f76a0400640 -1 --2- [v2:172.21.15.148:6836/992188984,v1:172.21.15.148:6837/992188984] >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x5642e3ba3680 0x5642e3d4a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] is using msgr V1 protocol 2024-09-21T07:39:11.765 INFO:tasks.ceph.mds.d.smithi148.stderr:2024-09-21T07:39:11.763+0000 7f76a0400640 -1 --2- [v2:172.21.15.148:6836/992188984,v1:172.21.15.148:6837/992188984] >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x5642e3ba3680 0x5642e3d4a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] is using msgr V1 protocol 2024-09-21T07:39:13.289 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:39:13.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.526+0000 7f28a5000640 1 -- 172.21.15.39:0/512400872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28a00af490 msgr2=0x7f28a00cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:13.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.526+0000 7f28a5000640 1 --2- 172.21.15.39:0/512400872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28a00af490 0x7f28a00cf9d0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f28a0072a50 tx=0x7f289802a8f0 comp rx=0 tx=0).stop 2024-09-21T07:39:13.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.526+0000 7f28a5000640 1 -- 172.21.15.39:0/512400872 shutdown_connections 2024-09-21T07:39:13.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.526+0000 7f28a5000640 1 --2- 172.21.15.39:0/512400872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28a00af490 0x7f28a00cf9d0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:13.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.526+0000 7f28a5000640 1 --2- 172.21.15.39:0/512400872 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28a015abe0 0x7f28a00cf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:13.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.526+0000 7f28a5000640 1 --2- 172.21.15.39:0/512400872 >> v2:172.21.15.148:3300/0 conn(0x7f28a00b43f0 0x7f28a00ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:13.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.526+0000 7f28a5000640 1 -- 172.21.15.39:0/512400872 >> 172.21.15.39:0/512400872 conn(0x7f28a00b2510 msgr2=0x7f28a00d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:13.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.526+0000 7f28a5000640 1 -- 172.21.15.39:0/512400872 shutdown_connections 2024-09-21T07:39:13.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.526+0000 7f28a5000640 1 -- 172.21.15.39:0/512400872 wait complete. 2024-09-21T07:39:13.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.528+0000 7f28a5000640 1 Processor -- start 2024-09-21T07:39:13.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.528+0000 7f28a5000640 1 -- start start 2024-09-21T07:39:13.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.530+0000 7f28a5000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28a00af490 0x7f28a01f6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:13.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.530+0000 7f28a5000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28a00b43f0 0x7f28a00caed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:13.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.530+0000 7f28a5000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28a015abe0 0x7f28a00cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:13.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.530+0000 7f28a5000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f28a0070e80 con 0x7f28a00b43f0 2024-09-21T07:39:13.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.530+0000 7f28a5000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f28a00712a0 con 0x7f28a015abe0 2024-09-21T07:39:13.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.530+0000 7f28a5000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f28a006f6c0 con 0x7f28a00af490 2024-09-21T07:39:13.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.530+0000 7f289ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28a00b43f0 0x7f28a00caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:13.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.530+0000 7f289ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28a00b43f0 0x7f28a00caed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33918/0 (socket says 172.21.15.39:33918) 2024-09-21T07:39:13.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.530+0000 7f289ea00640 1 -- 172.21.15.39:0/3640745334 learned_addr learned my addr 172.21.15.39:0/3640745334 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:39:13.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.530+0000 7f289f400640 1 --2- 172.21.15.39:0/3640745334 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28a00af490 0x7f28a01f6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:13.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.530+0000 7f289fe00640 1 --2- 172.21.15.39:0/3640745334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28a015abe0 0x7f28a00cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:13.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.530+0000 7f289ea00640 1 -- 172.21.15.39:0/3640745334 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28a00af490 msgr2=0x7f28a01f6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:13.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.530+0000 7f289ea00640 1 --2- 172.21.15.39:0/3640745334 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28a00af490 0x7f28a01f6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:13.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.530+0000 7f289ea00640 1 -- 172.21.15.39:0/3640745334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28a015abe0 msgr2=0x7f28a00cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:13.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.530+0000 7f289ea00640 1 --2- 172.21.15.39:0/3640745334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28a015abe0 0x7f28a00cb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:13.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.530+0000 7f289ea00640 1 -- 172.21.15.39:0/3640745334 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28a01f79e0 con 0x7f28a00b43f0 2024-09-21T07:39:13.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.530+0000 7f289f400640 1 --2- 172.21.15.39:0/3640745334 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28a00af490 0x7f28a01f6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:39:13.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.531+0000 7f289ea00640 1 --2- 172.21.15.39:0/3640745334 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28a00b43f0 0x7f28a00caed0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f288c0095d0 tx=0x7f288c009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:13.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.531+0000 7f2897400640 1 -- 172.21.15.39:0/3640745334 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288c018030 con 0x7f28a00b43f0 2024-09-21T07:39:13.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.531+0000 7f2897400640 1 -- 172.21.15.39:0/3640745334 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f288c005130 con 0x7f28a00b43f0 2024-09-21T07:39:13.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.531+0000 7f2897400640 1 -- 172.21.15.39:0/3640745334 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288c005580 con 0x7f28a00b43f0 2024-09-21T07:39:13.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.531+0000 7f28a5000640 1 -- 172.21.15.39:0/3640745334 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28a01f7420 con 0x7f28a00b43f0 2024-09-21T07:39:13.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.531+0000 7f28a5000640 1 -- 172.21.15.39:0/3640745334 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28a01f4740 con 0x7f28a00b43f0 2024-09-21T07:39:13.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.532+0000 7f28a5000640 1 -- 172.21.15.39:0/3640745334 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28a00ca890 con 0x7f28a00b43f0 2024-09-21T07:39:13.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.533+0000 7f2897400640 1 -- 172.21.15.39:0/3640745334 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f288c01b070 con 0x7f28a00b43f0 2024-09-21T07:39:13.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.534+0000 7f2897400640 1 --2- 172.21.15.39:0/3640745334 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2880078bc0 0x7f288007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:13.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.535+0000 7f2897400640 1 -- 172.21.15.39:0/3640745334 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 7259+0+0 (secure 0 0 0) 0x7f288c09b440 con 0x7f28a00b43f0 2024-09-21T07:39:13.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.535+0000 7f289f400640 1 --2- 172.21.15.39:0/3640745334 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2880078bc0 0x7f288007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:13.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.536+0000 7f289f400640 1 --2- 172.21.15.39:0/3640745334 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2880078bc0 0x7f288007b0d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2890009b00 tx=0x7f2890005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:13.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.539+0000 7f2897400640 1 -- 172.21.15.39:0/3640745334 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f288c011070 con 0x7f28a00b43f0 2024-09-21T07:39:13.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.774+0000 7f28a5000640 1 -- 172.21.15.39:0/3640745334 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28a0070ad0 con 0x7f28a00b43f0 2024-09-21T07:39:13.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.774+0000 7f2897400640 1 -- 172.21.15.39:0/3640745334 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+5242 (secure 0 0 0) 0x7f288c063320 con 0x7f28a00b43f0 2024-09-21T07:39:13.775 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:39:13.775 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":96,"btime":"2024-09-21T07:39:08:748965+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7080,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/1456343023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1456343023},{"type":"v1","addr":"172.21.15.39:6833","nonce":1456343023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:39:07.766926+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6619,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6619":{"gid":6619,"name":"d","rank":0,"incarnation":95,"state":"up:resolve","state_seq":10,"addr":"172.21.15.148:6837/992188984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":992188984},{"type":"v1","addr":"172.21.15.148:6837","nonce":992188984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6115,"qdb_cluster":[6115]},"id":6}]} 2024-09-21T07:39:13.776 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 96 2024-09-21T07:39:13.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.776+0000 7f28a5000640 1 -- 172.21.15.39:0/3640745334 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2880078bc0 msgr2=0x7f288007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:13.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.776+0000 7f28a5000640 1 --2- 172.21.15.39:0/3640745334 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2880078bc0 0x7f288007b0d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2890009b00 tx=0x7f2890005f90 comp rx=0 tx=0).stop 2024-09-21T07:39:13.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.777+0000 7f28a5000640 1 -- 172.21.15.39:0/3640745334 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28a00b43f0 msgr2=0x7f28a00caed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:13.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.777+0000 7f28a5000640 1 --2- 172.21.15.39:0/3640745334 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28a00b43f0 0x7f28a00caed0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f288c0095d0 tx=0x7f288c009a90 comp rx=0 tx=0).stop 2024-09-21T07:39:13.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.777+0000 7f28a5000640 1 -- 172.21.15.39:0/3640745334 shutdown_connections 2024-09-21T07:39:13.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.777+0000 7f28a5000640 1 --2- 172.21.15.39:0/3640745334 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2880078bc0 0x7f288007b0d0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:13.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.777+0000 7f28a5000640 1 --2- 172.21.15.39:0/3640745334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28a015abe0 0x7f28a00cb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:13.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.777+0000 7f28a5000640 1 --2- 172.21.15.39:0/3640745334 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28a00b43f0 0x7f28a00caed0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:13.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.777+0000 7f28a5000640 1 --2- 172.21.15.39:0/3640745334 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28a00af490 0x7f28a01f6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:13.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.777+0000 7f28a5000640 1 -- 172.21.15.39:0/3640745334 >> 172.21.15.39:0/3640745334 conn(0x7f28a00b2510 msgr2=0x7f28a015a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:13.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.777+0000 7f28a5000640 1 -- 172.21.15.39:0/3640745334 shutdown_connections 2024-09-21T07:39:13.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:13.777+0000 7f28a5000640 1 -- 172.21.15.39:0/3640745334 wait complete. 2024-09-21T07:39:13.800 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-21T07:39:13.800 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:39:14.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.027+0000 7f007be00640 1 -- 172.21.15.39:0/3979207400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f007c15abe0 msgr2=0x7f007c0ca890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:14.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.027+0000 7f007be00640 1 --2- 172.21.15.39:0/3979207400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f007c15abe0 0x7f007c0ca890 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f007c0cd880 tx=0x7f006402a4d0 comp rx=0 tx=0).stop 2024-09-21T07:39:14.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.027+0000 7f007be00640 1 -- 172.21.15.39:0/3979207400 shutdown_connections 2024-09-21T07:39:14.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.027+0000 7f007be00640 1 --2- 172.21.15.39:0/3979207400 >> v2:172.21.15.148:3300/0 conn(0x7f007c0af490 0x7f007c06f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:14.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.027+0000 7f007be00640 1 --2- 172.21.15.39:0/3979207400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f007c15abe0 0x7f007c0ca890 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:14.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.027+0000 7f007be00640 1 -- 172.21.15.39:0/3979207400 >> 172.21.15.39:0/3979207400 conn(0x7f007c0b2510 msgr2=0x7f007c0d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:14.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.027+0000 7f007be00640 1 -- 172.21.15.39:0/3979207400 shutdown_connections 2024-09-21T07:39:14.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.027+0000 7f007be00640 1 -- 172.21.15.39:0/3979207400 wait complete. 2024-09-21T07:39:14.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.029+0000 7f007be00640 1 Processor -- start 2024-09-21T07:39:14.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.029+0000 7f007be00640 1 -- start start 2024-09-21T07:39:14.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.031+0000 7f007be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f007c0af490 0x7f007c15cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:14.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.031+0000 7f007be00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f007c0b43f0 0x7f007c159920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:14.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.031+0000 7f007be00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f007c15abe0 0x7f007c0c9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:14.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.031+0000 7f007be00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f007c0cfba0 con 0x7f007c0af490 2024-09-21T07:39:14.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.031+0000 7f007be00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f007c0712d0 con 0x7f007c15abe0 2024-09-21T07:39:14.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.031+0000 7f007be00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f007c0cfd20 con 0x7f007c0b43f0 2024-09-21T07:39:14.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.031+0000 7f007aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f007c0af490 0x7f007c15cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:14.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.031+0000 7f007aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f007c0af490 0x7f007c15cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33926/0 (socket says 172.21.15.39:33926) 2024-09-21T07:39:14.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.031+0000 7f007aa00640 1 -- 172.21.15.39:0/3249181169 learned_addr learned my addr 172.21.15.39:0/3249181169 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:39:14.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.031+0000 7f007b400640 1 --2- 172.21.15.39:0/3249181169 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f007c15abe0 0x7f007c0c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:14.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.031+0000 7f0073e00640 1 --2- 172.21.15.39:0/3249181169 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f007c0b43f0 0x7f007c159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:14.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.032+0000 7f007aa00640 1 -- 172.21.15.39:0/3249181169 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f007c0b43f0 msgr2=0x7f007c159920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:14.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.032+0000 7f007aa00640 1 --2- 172.21.15.39:0/3249181169 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f007c0b43f0 0x7f007c159920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:14.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.032+0000 7f007aa00640 1 -- 172.21.15.39:0/3249181169 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f007c15abe0 msgr2=0x7f007c0c9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:14.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.032+0000 7f007aa00640 1 --2- 172.21.15.39:0/3249181169 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f007c15abe0 0x7f007c0c9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:14.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.032+0000 7f007aa00640 1 -- 172.21.15.39:0/3249181169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f007c1f5520 con 0x7f007c0af490 2024-09-21T07:39:14.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.032+0000 7f0073e00640 1 --2- 172.21.15.39:0/3249181169 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f007c0b43f0 0x7f007c159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:39:14.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.032+0000 7f007aa00640 1 --2- 172.21.15.39:0/3249181169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f007c0af490 0x7f007c15cf40 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f006c00c3d0 tx=0x7f006c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:14.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.032+0000 7f0072a00640 1 -- 172.21.15.39:0/3249181169 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006c017070 con 0x7f007c0af490 2024-09-21T07:39:14.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.032+0000 7f0072a00640 1 -- 172.21.15.39:0/3249181169 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f006c0053c0 con 0x7f007c0af490 2024-09-21T07:39:14.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.032+0000 7f0072a00640 1 -- 172.21.15.39:0/3249181169 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006c003850 con 0x7f007c0af490 2024-09-21T07:39:14.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.032+0000 7f007be00640 1 -- 172.21.15.39:0/3249181169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f007c159e60 con 0x7f007c0af490 2024-09-21T07:39:14.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.032+0000 7f007be00640 1 -- 172.21.15.39:0/3249181169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f007c1f5a70 con 0x7f007c0af490 2024-09-21T07:39:14.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.034+0000 7f007be00640 1 -- 172.21.15.39:0/3249181169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f007c06f6c0 con 0x7f007c0af490 2024-09-21T07:39:14.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.035+0000 7f0072a00640 1 -- 172.21.15.39:0/3249181169 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f006c01a070 con 0x7f007c0af490 2024-09-21T07:39:14.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.036+0000 7f0072a00640 1 --2- 172.21.15.39:0/3249181169 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0058078b90 0x7f005807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:14.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.036+0000 7f0072a00640 1 -- 172.21.15.39:0/3249181169 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 7259+0+0 (secure 0 0 0) 0x7f006c09a7c0 con 0x7f007c0af490 2024-09-21T07:39:14.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.036+0000 7f0073e00640 1 --2- 172.21.15.39:0/3249181169 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0058078b90 0x7f005807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:14.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.037+0000 7f0073e00640 1 --2- 172.21.15.39:0/3249181169 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0058078b90 0x7f005807b0a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0064033000 tx=0x7f0064004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:14.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.041+0000 7f0072a00640 1 -- 172.21.15.39:0/3249181169 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f006c061f00 con 0x7f007c0af490 2024-09-21T07:39:14.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.282+0000 7f007be00640 1 -- 172.21.15.39:0/3249181169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f007c0ad7e0 con 0x7f007c0af490 2024-09-21T07:39:14.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.282+0000 7f0072a00640 1 -- 172.21.15.39:0/3249181169 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+5242 (secure 0 0 0) 0x7f006c066930 con 0x7f007c0af490 2024-09-21T07:39:14.283 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:39:14.283 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":96,"btime":"2024-09-21T07:39:08:748965+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7080,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/1456343023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1456343023},{"type":"v1","addr":"172.21.15.39:6833","nonce":1456343023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:39:07.766926+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6619,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6619":{"gid":6619,"name":"d","rank":0,"incarnation":95,"state":"up:resolve","state_seq":10,"addr":"172.21.15.148:6837/992188984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":992188984},{"type":"v1","addr":"172.21.15.148:6837","nonce":992188984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6115,"qdb_cluster":[6115]},"id":6}]} 2024-09-21T07:39:14.284 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 96 2024-09-21T07:39:14.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.285+0000 7f007be00640 1 -- 172.21.15.39:0/3249181169 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0058078b90 msgr2=0x7f005807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:14.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.285+0000 7f007be00640 1 --2- 172.21.15.39:0/3249181169 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0058078b90 0x7f005807b0a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0064033000 tx=0x7f0064004730 comp rx=0 tx=0).stop 2024-09-21T07:39:14.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.285+0000 7f007be00640 1 -- 172.21.15.39:0/3249181169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f007c0af490 msgr2=0x7f007c15cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:14.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.285+0000 7f007be00640 1 --2- 172.21.15.39:0/3249181169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f007c0af490 0x7f007c15cf40 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f006c00c3d0 tx=0x7f006c00c890 comp rx=0 tx=0).stop 2024-09-21T07:39:14.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.285+0000 7f007be00640 1 -- 172.21.15.39:0/3249181169 shutdown_connections 2024-09-21T07:39:14.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.285+0000 7f007be00640 1 --2- 172.21.15.39:0/3249181169 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0058078b90 0x7f005807b0a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:14.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.285+0000 7f007be00640 1 --2- 172.21.15.39:0/3249181169 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f007c15abe0 0x7f007c0c9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:14.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.285+0000 7f007be00640 1 --2- 172.21.15.39:0/3249181169 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f007c0b43f0 0x7f007c159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:14.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.285+0000 7f007be00640 1 --2- 172.21.15.39:0/3249181169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f007c0af490 0x7f007c15cf40 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:14.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.285+0000 7f007be00640 1 -- 172.21.15.39:0/3249181169 >> 172.21.15.39:0/3249181169 conn(0x7f007c0b2510 msgr2=0x7f007c0b0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:14.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.285+0000 7f007be00640 1 -- 172.21.15.39:0/3249181169 shutdown_connections 2024-09-21T07:39:14.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.285+0000 7f007be00640 1 -- 172.21.15.39:0/3249181169 wait complete. 2024-09-21T07:39:14.312 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:resolve 2024-09-21T07:39:14.312 DEBUG:tasks.cephfs.filesystem:reached state 'up:resolve' in 0.5124306678771973s 2024-09-21T07:39:14.312 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:39:14.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.541+0000 7f84eec00640 1 -- 172.21.15.39:0/1038799630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d0004800 msgr2=0x7f84d0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:14.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.541+0000 7f84eec00640 1 --2- 172.21.15.39:0/1038799630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d0004800 0x7f84d0004c10 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f84d4009b50 tx=0x7f84d0005830 comp rx=0 tx=0).stop 2024-09-21T07:39:14.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.541+0000 7f84ee200640 1 -- 172.21.15.39:0/1038799630 reap_dead start 2024-09-21T07:39:14.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.541+0000 7f84eec00640 1 -- 172.21.15.39:0/1038799630 shutdown_connections 2024-09-21T07:39:14.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.541+0000 7f84eec00640 1 -- 172.21.15.39:0/1038799630 >> 172.21.15.39:0/1038799630 conn(0x7f84e80b2510 msgr2=0x7f84e80d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:14.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.541+0000 7f84eec00640 1 -- 172.21.15.39:0/1038799630 shutdown_connections 2024-09-21T07:39:14.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.541+0000 7f84eec00640 1 -- 172.21.15.39:0/1038799630 wait complete. 2024-09-21T07:39:14.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.543+0000 7f84eec00640 1 Processor -- start 2024-09-21T07:39:14.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.543+0000 7f84eec00640 1 -- start start 2024-09-21T07:39:14.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.545+0000 7f84eec00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84d0004800 0x7f84e814bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:14.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.545+0000 7f84eec00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84e814c180 0x7f84e80b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:14.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.545+0000 7f84eec00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84e80b14f0 0x7f84e809b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:14.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.545+0000 7f84eec00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f84e806fc30 con 0x7f84e814c180 2024-09-21T07:39:14.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.545+0000 7f84eec00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f84e806fab0 con 0x7f84e80b14f0 2024-09-21T07:39:14.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.545+0000 7f84eec00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f84e80b43f0 con 0x7f84d0004800 2024-09-21T07:39:14.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.545+0000 7f84ece00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84e814c180 0x7f84e80b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:14.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.545+0000 7f84ece00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84e814c180 0x7f84e80b0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33940/0 (socket says 172.21.15.39:33940) 2024-09-21T07:39:14.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.545+0000 7f84ece00640 1 -- 172.21.15.39:0/642875259 learned_addr learned my addr 172.21.15.39:0/642875259 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:39:14.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.545+0000 7f84ee200640 1 --2- 172.21.15.39:0/642875259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84e80b14f0 0x7f84e809b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:14.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.545+0000 7f84ed800640 1 --2- 172.21.15.39:0/642875259 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84d0004800 0x7f84e814bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:14.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.545+0000 7f84ece00640 1 -- 172.21.15.39:0/642875259 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84d0004800 msgr2=0x7f84e814bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:14.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.545+0000 7f84ece00640 1 --2- 172.21.15.39:0/642875259 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84d0004800 0x7f84e814bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:14.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.545+0000 7f84ece00640 1 -- 172.21.15.39:0/642875259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84e80b14f0 msgr2=0x7f84e809b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:14.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.545+0000 7f84ece00640 1 --2- 172.21.15.39:0/642875259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84e80b14f0 0x7f84e809b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:14.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.545+0000 7f84ece00640 1 -- 172.21.15.39:0/642875259 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e809aa50 con 0x7f84e814c180 2024-09-21T07:39:14.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.546+0000 7f84ed800640 1 --2- 172.21.15.39:0/642875259 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84d0004800 0x7f84e814bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:39:14.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.546+0000 7f84ece00640 1 --2- 172.21.15.39:0/642875259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84e814c180 0x7f84e80b0fb0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f84d4009a70 tx=0x7f84d4033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:14.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.546+0000 7f84e5600640 1 -- 172.21.15.39:0/642875259 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d4043070 con 0x7f84e814c180 2024-09-21T07:39:14.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.546+0000 7f84e5600640 1 -- 172.21.15.39:0/642875259 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f84d4004340 con 0x7f84e814c180 2024-09-21T07:39:14.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.546+0000 7f84eec00640 1 -- 172.21.15.39:0/642875259 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84e815b530 con 0x7f84e814c180 2024-09-21T07:39:14.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.546+0000 7f84e5600640 1 -- 172.21.15.39:0/642875259 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d4038040 con 0x7f84e814c180 2024-09-21T07:39:14.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.546+0000 7f84eec00640 1 -- 172.21.15.39:0/642875259 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84e809afd0 con 0x7f84e814c180 2024-09-21T07:39:14.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.547+0000 7f84eec00640 1 -- 172.21.15.39:0/642875259 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84e80b4950 con 0x7f84e814c180 2024-09-21T07:39:14.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.549+0000 7f84e5600640 1 -- 172.21.15.39:0/642875259 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f84d4046070 con 0x7f84e814c180 2024-09-21T07:39:14.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.550+0000 7f84e5600640 1 --2- 172.21.15.39:0/642875259 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f84c8078b90 0x7f84c807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:14.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.550+0000 7f84e5600640 1 -- 172.21.15.39:0/642875259 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 7259+0+0 (secure 0 0 0) 0x7f84d40bb950 con 0x7f84e814c180 2024-09-21T07:39:14.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.550+0000 7f84ed800640 1 --2- 172.21.15.39:0/642875259 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f84c8078b90 0x7f84c807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:14.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.551+0000 7f84ed800640 1 --2- 172.21.15.39:0/642875259 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f84c8078b90 0x7f84c807b0a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f84dc001cc0 tx=0x7f84dc001c10 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:14.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.554+0000 7f84e5600640 1 -- 172.21.15.39:0/642875259 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f84d4083090 con 0x7f84e814c180 2024-09-21T07:39:14.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.788+0000 7f84eec00640 1 -- 172.21.15.39:0/642875259 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84e81e5620 con 0x7f84e814c180 2024-09-21T07:39:14.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.788+0000 7f84e5600640 1 -- 172.21.15.39:0/642875259 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+5242 (secure 0 0 0) 0x7f84d4087ac0 con 0x7f84e814c180 2024-09-21T07:39:14.790 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:39:14.790 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":96,"btime":"2024-09-21T07:39:08:748965+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7080,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/1456343023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1456343023},{"type":"v1","addr":"172.21.15.39:6833","nonce":1456343023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:39:07.766926+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6619,"mds_1":6115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6115":{"gid":6115,"name":"a","rank":1,"incarnation":56,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/364657398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":364657398},{"type":"v1","addr":"172.21.15.39:6835","nonce":364657398}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6619":{"gid":6619,"name":"d","rank":0,"incarnation":95,"state":"up:resolve","state_seq":10,"addr":"172.21.15.148:6837/992188984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":992188984},{"type":"v1","addr":"172.21.15.148:6837","nonce":992188984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6115,"qdb_cluster":[6115]},"id":6}]} 2024-09-21T07:39:14.790 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 96 2024-09-21T07:39:14.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.791+0000 7f84eec00640 1 -- 172.21.15.39:0/642875259 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f84c8078b90 msgr2=0x7f84c807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:14.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.791+0000 7f84eec00640 1 --2- 172.21.15.39:0/642875259 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f84c8078b90 0x7f84c807b0a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f84dc001cc0 tx=0x7f84dc001c10 comp rx=0 tx=0).stop 2024-09-21T07:39:14.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.791+0000 7f84eec00640 1 -- 172.21.15.39:0/642875259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84e814c180 msgr2=0x7f84e80b0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:14.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.791+0000 7f84eec00640 1 --2- 172.21.15.39:0/642875259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84e814c180 0x7f84e80b0fb0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f84d4009a70 tx=0x7f84d4033670 comp rx=0 tx=0).stop 2024-09-21T07:39:14.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.791+0000 7f84eec00640 1 -- 172.21.15.39:0/642875259 shutdown_connections 2024-09-21T07:39:14.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.791+0000 7f84eec00640 1 --2- 172.21.15.39:0/642875259 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f84c8078b90 0x7f84c807b0a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:14.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.791+0000 7f84eec00640 1 --2- 172.21.15.39:0/642875259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84e80b14f0 0x7f84e809b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:14.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.791+0000 7f84eec00640 1 --2- 172.21.15.39:0/642875259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84e814c180 0x7f84e80b0fb0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:14.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.791+0000 7f84eec00640 1 --2- 172.21.15.39:0/642875259 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84d0004800 0x7f84e814bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:14.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.791+0000 7f84eec00640 1 -- 172.21.15.39:0/642875259 >> 172.21.15.39:0/642875259 conn(0x7f84e80b2510 msgr2=0x7f84e815d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:14.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.791+0000 7f84eec00640 1 -- 172.21.15.39:0/642875259 shutdown_connections 2024-09-21T07:39:14.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:14.791+0000 7f84eec00640 1 -- 172.21.15.39:0/642875259 wait complete. 2024-09-21T07:39:14.817 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json dump snaps --server 2024-09-21T07:39:14.967 INFO:tasks.ceph.mds.d.smithi148.stderr:2024-09-21T07:39:14.965+0000 7f76a0400640 -1 --2- [v2:172.21.15.148:6836/992188984,v1:172.21.15.148:6837/992188984] >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x5642e3ba3680 0x5642e3d4a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] is using msgr V1 protocol 2024-09-21T07:39:15.062 INFO:teuthology.orchestra.run.smithi148.stdout:{"last_snap":10,"last_created":5,"last_destroyed":9,"pending_noop":[],"snaps":[],"need_to_purge":{"16":[2,3,4,5,6,7,8,9]},"pending_update":[{"version":18,"snapinfo":{"snapid":10,"ino":1099511627777,"stamp":"2024-09-21T07:38:43.575316+0000","name":"s22","metadata":{}}}],"pending_destroy":[]} 2024-09-21T07:39:15.063 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 5, "last_destroyed": 9, "last_snap": 10, "need_to_purge": { "16": [ 2, 3, 4, 5, 6, 7, 8, 9 ] }, "pending_destroy": [], "pending_noop": [], "pending_update": [ { "snapInfinityo": { "ino": 1099511627777, "metadata": {}, "name": "s22", "snapid": 10, "stamp": "2024-09-21T07:38:43.575316+0000" }, "version": 18 } ], "snaps": [] } 2024-09-21T07:39:15.063 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 6:1 2024-09-21T07:39:15.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.291+0000 7f0953200640 1 -- 172.21.15.39:0/1369204298 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f095406fde0 msgr2=0x7f0954072fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:15.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.291+0000 7f0953200640 1 --2- 172.21.15.39:0/1369204298 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f095406fde0 0x7f0954072fa0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f094c005fd0 tx=0x7f094c02a7a0 comp rx=0 tx=0).stop 2024-09-21T07:39:15.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.291+0000 7f0953200640 1 -- 172.21.15.39:0/1369204298 shutdown_connections 2024-09-21T07:39:15.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.291+0000 7f0953200640 1 --2- 172.21.15.39:0/1369204298 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f095406fde0 0x7f0954072fa0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:15.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.291+0000 7f0953200640 1 --2- 172.21.15.39:0/1369204298 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f095415c830 0x7f095406f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:15.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.291+0000 7f0953200640 1 -- 172.21.15.39:0/1369204298 >> 172.21.15.39:0/1369204298 conn(0x7f09540b2510 msgr2=0x7f09540d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:15.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.291+0000 7f0953200640 1 -- 172.21.15.39:0/1369204298 shutdown_connections 2024-09-21T07:39:15.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.291+0000 7f0953200640 1 -- 172.21.15.39:0/1369204298 wait complete. 2024-09-21T07:39:15.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.293+0000 7f0953200640 1 Processor -- start 2024-09-21T07:39:15.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.294+0000 7f0953200640 1 -- start start 2024-09-21T07:39:15.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.295+0000 7f0953200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f095406fde0 0x7f0954158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:15.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.295+0000 7f0953200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09540d0f20 0x7f09541f6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:15.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.295+0000 7f0953200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f095415c830 0x7f09540ae2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:15.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.295+0000 7f0953200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f09540b4570 con 0x7f095415c830 2024-09-21T07:39:15.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.295+0000 7f0953200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f09540b43f0 con 0x7f09540d0f20 2024-09-21T07:39:15.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.295+0000 7f0953200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f09540b46f0 con 0x7f095406fde0 2024-09-21T07:39:15.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.295+0000 7f0952800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f095415c830 0x7f09540ae2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:15.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.295+0000 7f0952800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f095415c830 0x7f09540ae2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33950/0 (socket says 172.21.15.39:33950) 2024-09-21T07:39:15.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.295+0000 7f0952800640 1 -- 172.21.15.39:0/2020065702 learned_addr learned my addr 172.21.15.39:0/2020065702 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:39:15.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.295+0000 7f094be00640 1 --2- 172.21.15.39:0/2020065702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09540d0f20 0x7f09541f6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:15.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.295+0000 7f0951e00640 1 --2- 172.21.15.39:0/2020065702 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f095406fde0 0x7f0954158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:15.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.296+0000 7f0952800640 1 -- 172.21.15.39:0/2020065702 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f095406fde0 msgr2=0x7f0954158c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:15.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.296+0000 7f0952800640 1 --2- 172.21.15.39:0/2020065702 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f095406fde0 0x7f0954158c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:15.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.296+0000 7f0952800640 1 -- 172.21.15.39:0/2020065702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09540d0f20 msgr2=0x7f09541f6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:15.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.296+0000 7f0952800640 1 --2- 172.21.15.39:0/2020065702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09540d0f20 0x7f09541f6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:15.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.296+0000 7f0952800640 1 -- 172.21.15.39:0/2020065702 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09540adb60 con 0x7f095415c830 2024-09-21T07:39:15.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.296+0000 7f0952800640 1 --2- 172.21.15.39:0/2020065702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f095415c830 0x7f09540ae2f0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f094c034040 tx=0x7f094c032e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:15.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.297+0000 7f094a000640 1 -- 172.21.15.39:0/2020065702 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c041020 con 0x7f095415c830 2024-09-21T07:39:15.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.297+0000 7f094a000640 1 -- 172.21.15.39:0/2020065702 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f094c004040 con 0x7f095415c830 2024-09-21T07:39:15.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.297+0000 7f0953200640 1 -- 172.21.15.39:0/2020065702 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09541f6fe0 con 0x7f095415c830 2024-09-21T07:39:15.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.297+0000 7f094a000640 1 -- 172.21.15.39:0/2020065702 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c036040 con 0x7f095415c830 2024-09-21T07:39:15.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.297+0000 7f0953200640 1 -- 172.21.15.39:0/2020065702 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f095415bc60 con 0x7f095415c830 2024-09-21T07:39:15.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.298+0000 7f0953200640 1 -- 172.21.15.39:0/2020065702 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0954159290 con 0x7f095415c830 2024-09-21T07:39:15.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.299+0000 7f094a000640 1 -- 172.21.15.39:0/2020065702 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f094c044020 con 0x7f095415c830 2024-09-21T07:39:15.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.300+0000 7f094a000640 1 --2- 172.21.15.39:0/2020065702 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0930078b90 0x7f093007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:15.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.300+0000 7f094a000640 1 -- 172.21.15.39:0/2020065702 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 7259+0+0 (secure 0 0 0) 0x7f094c0bc3f0 con 0x7f095415c830 2024-09-21T07:39:15.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.301+0000 7f0951e00640 1 --2- 172.21.15.39:0/2020065702 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0930078b90 0x7f093007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:15.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.301+0000 7f0951e00640 1 --2- 172.21.15.39:0/2020065702 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0930078b90 0x7f093007b0a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f09440018b0 tx=0x7f0944009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:15.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.304+0000 7f094a000640 1 -- 172.21.15.39:0/2020065702 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f094c04b030 con 0x7f095415c830 2024-09-21T07:39:15.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.579+0000 7f0953200640 1 -- 172.21.15.39:0/2020065702 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "6:1"} v 0) -- 0x7f09540b4c10 con 0x7f095415c830 2024-09-21T07:39:15.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.581+0000 7f094a000640 1 -- 172.21.15.39:0/2020065702 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "6:1"}]=0 failed mds gid 6115 v97) ==== 97+0+0 (secure 0 0 0) 0x7f094c083b30 con 0x7f095415c830 2024-09-21T07:39:15.582 INFO:teuthology.orchestra.run.smithi039.stderr:failed mds gid 6115 2024-09-21T07:39:15.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.584+0000 7f0953200640 1 -- 172.21.15.39:0/2020065702 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0930078b90 msgr2=0x7f093007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:15.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.584+0000 7f0953200640 1 --2- 172.21.15.39:0/2020065702 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0930078b90 0x7f093007b0a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f09440018b0 tx=0x7f0944009040 comp rx=0 tx=0).stop 2024-09-21T07:39:15.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.584+0000 7f0953200640 1 -- 172.21.15.39:0/2020065702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f095415c830 msgr2=0x7f09540ae2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:15.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.584+0000 7f0953200640 1 --2- 172.21.15.39:0/2020065702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f095415c830 0x7f09540ae2f0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f094c034040 tx=0x7f094c032e90 comp rx=0 tx=0).stop 2024-09-21T07:39:15.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.584+0000 7f0953200640 1 -- 172.21.15.39:0/2020065702 shutdown_connections 2024-09-21T07:39:15.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.584+0000 7f0953200640 1 --2- 172.21.15.39:0/2020065702 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0930078b90 0x7f093007b0a0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:15.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.584+0000 7f0953200640 1 --2- 172.21.15.39:0/2020065702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f095415c830 0x7f09540ae2f0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:15.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.584+0000 7f0953200640 1 --2- 172.21.15.39:0/2020065702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09540d0f20 0x7f09541f6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:15.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.584+0000 7f0953200640 1 --2- 172.21.15.39:0/2020065702 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f095406fde0 0x7f0954158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:15.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.584+0000 7f0953200640 1 -- 172.21.15.39:0/2020065702 >> 172.21.15.39:0/2020065702 conn(0x7f09540b2510 msgr2=0x7f09540b05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:15.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.584+0000 7f0953200640 1 -- 172.21.15.39:0/2020065702 shutdown_connections 2024-09-21T07:39:15.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.584+0000 7f0953200640 1 -- 172.21.15.39:0/2020065702 wait complete. 2024-09-21T07:39:15.616 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T07:39:15.617 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T07:39:15.617 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T07:39:15.617 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:39:15.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:39:15.617 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/daemon/state.py", line 146, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi039 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a' 2024-09-21T07:39:15.617 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T07:39:15.617 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T07:39:15.621 INFO:tasks.ceph.mds.a:Started 2024-09-21T07:39:15.622 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:39:15.817 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-21T07:39:15.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.850+0000 7f6597c00640 1 -- 172.21.15.39:0/1965104987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65900d0620 msgr2=0x7f65900d0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:15.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.850+0000 7f6597c00640 1 --2- 172.21.15.39:0/1965104987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65900d0620 0x7f65900d0a30 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f659006de00 tx=0x7f658802a820 comp rx=0 tx=0).stop 2024-09-21T07:39:15.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.850+0000 7f6597c00640 1 -- 172.21.15.39:0/1965104987 shutdown_connections 2024-09-21T07:39:15.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.850+0000 7f6597c00640 1 --2- 172.21.15.39:0/1965104987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65900d0620 0x7f65900d0a30 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:15.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.850+0000 7f6597c00640 1 --2- 172.21.15.39:0/1965104987 >> v2:172.21.15.148:3300/0 conn(0x7f65900cfd10 0x7f65900cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:15.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.850+0000 7f6597c00640 1 -- 172.21.15.39:0/1965104987 >> 172.21.15.39:0/1965104987 conn(0x7f659006cfc0 msgr2=0x7f659006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:15.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.850+0000 7f6597c00640 1 -- 172.21.15.39:0/1965104987 shutdown_connections 2024-09-21T07:39:15.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.850+0000 7f6597c00640 1 -- 172.21.15.39:0/1965104987 wait complete. 2024-09-21T07:39:15.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.852+0000 7f6597c00640 1 Processor -- start 2024-09-21T07:39:15.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.852+0000 7f6597c00640 1 -- start start 2024-09-21T07:39:15.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.854+0000 7f6597c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65900cfd10 0x7f65901551a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:15.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.854+0000 7f6597c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f65900d0620 0x7f6590150760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:15.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.854+0000 7f6597c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f659015cd30 0x7f6590156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:15.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.854+0000 7f6597c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6590073890 con 0x7f65900cfd10 2024-09-21T07:39:15.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.854+0000 7f6597c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6590073710 con 0x7f659015cd30 2024-09-21T07:39:15.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.854+0000 7f6597c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f6590073a10 con 0x7f65900d0620 2024-09-21T07:39:15.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.854+0000 7f6595200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65900cfd10 0x7f65901551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:15.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.854+0000 7f6595200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65900cfd10 0x7f65901551a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33970/0 (socket says 172.21.15.39:33970) 2024-09-21T07:39:15.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.854+0000 7f6595200640 1 -- 172.21.15.39:0/200996446 learned_addr learned my addr 172.21.15.39:0/200996446 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:39:15.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.854+0000 7f6587e00640 1 --2- 172.21.15.39:0/200996446 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f65900d0620 0x7f6590150760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:15.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.855+0000 7f6595c00640 1 --2- 172.21.15.39:0/200996446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f659015cd30 0x7f6590156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:15.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.855+0000 7f6595200640 1 -- 172.21.15.39:0/200996446 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f65900d0620 msgr2=0x7f6590150760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:15.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.855+0000 7f6595200640 1 --2- 172.21.15.39:0/200996446 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f65900d0620 0x7f6590150760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:15.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.855+0000 7f6595200640 1 -- 172.21.15.39:0/200996446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f659015cd30 msgr2=0x7f6590156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:15.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.855+0000 7f6595200640 1 --2- 172.21.15.39:0/200996446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f659015cd30 0x7f6590156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:15.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.855+0000 7f6595200640 1 -- 172.21.15.39:0/200996446 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6590154100 con 0x7f65900cfd10 2024-09-21T07:39:15.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.855+0000 7f6587e00640 1 --2- 172.21.15.39:0/200996446 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f65900d0620 0x7f6590150760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:39:15.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.855+0000 7f6595200640 1 --2- 172.21.15.39:0/200996446 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65900cfd10 0x7f65901551a0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f658000c3f0 tx=0x7f658000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:15.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.855+0000 7f658d600640 1 -- 172.21.15.39:0/200996446 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6580017070 con 0x7f65900cfd10 2024-09-21T07:39:15.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.855+0000 7f658d600640 1 -- 172.21.15.39:0/200996446 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65800035f0 con 0x7f65900cfd10 2024-09-21T07:39:15.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.855+0000 7f6597c00640 1 -- 172.21.15.39:0/200996446 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65901543e0 con 0x7f65900cfd10 2024-09-21T07:39:15.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.855+0000 7f658d600640 1 -- 172.21.15.39:0/200996446 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6580003a40 con 0x7f65900cfd10 2024-09-21T07:39:15.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.855+0000 7f6597c00640 1 -- 172.21.15.39:0/200996446 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65901549a0 con 0x7f65900cfd10 2024-09-21T07:39:15.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.857+0000 7f6597c00640 1 -- 172.21.15.39:0/200996446 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6590073f50 con 0x7f65900cfd10 2024-09-21T07:39:15.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.858+0000 7f658d600640 1 -- 172.21.15.39:0/200996446 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f658001a070 con 0x7f65900cfd10 2024-09-21T07:39:15.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.859+0000 7f658d600640 1 --2- 172.21.15.39:0/200996446 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f656c078b90 0x7f656c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:15.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.859+0000 7f658d600640 1 -- 172.21.15.39:0/200996446 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 7345+0+0 (secure 0 0 0) 0x7f658009a780 con 0x7f65900cfd10 2024-09-21T07:39:15.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.859+0000 7f6587e00640 1 --2- 172.21.15.39:0/200996446 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f656c078b90 0x7f656c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:15.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.860+0000 7f6587e00640 1 --2- 172.21.15.39:0/200996446 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f656c078b90 0x7f656c07b0a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f659006f1b0 tx=0x7f6578005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:15.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:15.863+0000 7f658d600640 1 -- 172.21.15.39:0/200996446 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f65800624b0 con 0x7f65900cfd10 2024-09-21T07:39:15.980 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~7s 2024-09-21T07:39:16.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:16.098+0000 7f6597c00640 1 -- 172.21.15.39:0/200996446 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6590153630 con 0x7f65900cfd10 2024-09-21T07:39:16.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:16.099+0000 7f658d600640 1 -- 172.21.15.39:0/200996446 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+4414 (secure 0 0 0) 0x7f6580061db0 con 0x7f65900cfd10 2024-09-21T07:39:16.100 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:39:16.100 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":98,"btime":"2024-09-21T07:39:15:586329+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:39:15.586310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6619,"mds_1":7080},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6619":{"gid":6619,"name":"d","rank":0,"incarnation":95,"state":"up:resolve","state_seq":10,"addr":"172.21.15.148:6837/992188984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":992188984},{"type":"v1","addr":"172.21.15.148:6837","nonce":992188984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7080":{"gid":7080,"name":"c","rank":1,"incarnation":98,"state":"up:replay","state_seq":1,"addr":"172.21.15.39:6833/1456343023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1456343023},{"type":"v1","addr":"172.21.15.39:6833","nonce":1456343023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T07:39:16.100 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 98 2024-09-21T07:39:16.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:16.101+0000 7f6597c00640 1 -- 172.21.15.39:0/200996446 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f656c078b90 msgr2=0x7f656c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:16.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:16.101+0000 7f6597c00640 1 --2- 172.21.15.39:0/200996446 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f656c078b90 0x7f656c07b0a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f659006f1b0 tx=0x7f6578005f90 comp rx=0 tx=0).stop 2024-09-21T07:39:16.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:16.101+0000 7f6597c00640 1 -- 172.21.15.39:0/200996446 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65900cfd10 msgr2=0x7f65901551a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:16.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:16.101+0000 7f6597c00640 1 --2- 172.21.15.39:0/200996446 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65900cfd10 0x7f65901551a0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f658000c3f0 tx=0x7f658000c8b0 comp rx=0 tx=0).stop 2024-09-21T07:39:16.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:16.101+0000 7f6597c00640 1 -- 172.21.15.39:0/200996446 shutdown_connections 2024-09-21T07:39:16.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:16.101+0000 7f6597c00640 1 --2- 172.21.15.39:0/200996446 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f656c078b90 0x7f656c07b0a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:16.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:16.101+0000 7f6597c00640 1 --2- 172.21.15.39:0/200996446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f659015cd30 0x7f6590156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:16.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:16.101+0000 7f6597c00640 1 --2- 172.21.15.39:0/200996446 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f65900d0620 0x7f6590150760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:16.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:16.101+0000 7f6597c00640 1 --2- 172.21.15.39:0/200996446 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65900cfd10 0x7f65901551a0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:16.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:16.101+0000 7f6597c00640 1 -- 172.21.15.39:0/200996446 >> 172.21.15.39:0/200996446 conn(0x7f659006cfc0 msgr2=0x7f659015aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:16.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:16.101+0000 7f6597c00640 1 -- 172.21.15.39:0/200996446 shutdown_connections 2024-09-21T07:39:16.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:16.101+0000 7f6597c00640 1 -- 172.21.15.39:0/200996446 wait complete. 2024-09-21T07:39:16.127 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-09-21T07:39:21.128 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:39:21.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.364+0000 7fc63ac00640 1 -- 172.21.15.39:0/2294613727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6340b43f0 msgr2=0x7fc6340ca890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:21.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.364+0000 7fc63ac00640 1 --2- 172.21.15.39:0/2294613727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6340b43f0 0x7fc6340ca890 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fc62800a940 tx=0x7fc62802b7d0 comp rx=0 tx=0).stop 2024-09-21T07:39:21.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.364+0000 7fc63ac00640 1 -- 172.21.15.39:0/2294613727 shutdown_connections 2024-09-21T07:39:21.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.364+0000 7fc63ac00640 1 --2- 172.21.15.39:0/2294613727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6340b43f0 0x7fc6340ca890 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:21.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.364+0000 7fc63ac00640 1 -- 172.21.15.39:0/2294613727 >> 172.21.15.39:0/2294613727 conn(0x7fc6340b2510 msgr2=0x7fc6340d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:21.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.364+0000 7fc63ac00640 1 -- 172.21.15.39:0/2294613727 shutdown_connections 2024-09-21T07:39:21.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.364+0000 7fc63ac00640 1 -- 172.21.15.39:0/2294613727 wait complete. 2024-09-21T07:39:21.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.366+0000 7fc63ac00640 1 Processor -- start 2024-09-21T07:39:21.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.367+0000 7fc63ac00640 1 -- start start 2024-09-21T07:39:21.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.368+0000 7fc63ac00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6340af490 0x7fc63415c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:21.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.368+0000 7fc63ac00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6340b43f0 0x7fc6341f6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:21.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.368+0000 7fc63ac00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc63415abe0 0x7fc6341f6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:21.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.368+0000 7fc63ac00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc6340cfd20 con 0x7fc6340b43f0 2024-09-21T07:39:21.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.368+0000 7fc63ac00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc6340cfba0 con 0x7fc6340af490 2024-09-21T07:39:21.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.368+0000 7fc63ac00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc6340cfea0 con 0x7fc63415abe0 2024-09-21T07:39:21.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.368+0000 7fc638e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6340b43f0 0x7fc6341f6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:21.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.368+0000 7fc638e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6340b43f0 0x7fc6341f6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33986/0 (socket says 172.21.15.39:33986) 2024-09-21T07:39:21.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.368+0000 7fc638e00640 1 -- 172.21.15.39:0/2558345527 learned_addr learned my addr 172.21.15.39:0/2558345527 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:39:21.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.369+0000 7fc639800640 1 --2- 172.21.15.39:0/2558345527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6340af490 0x7fc63415c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:21.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.369+0000 7fc63a200640 1 --2- 172.21.15.39:0/2558345527 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc63415abe0 0x7fc6341f6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:21.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.369+0000 7fc638e00640 1 -- 172.21.15.39:0/2558345527 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc63415abe0 msgr2=0x7fc6341f6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:21.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.369+0000 7fc638e00640 1 --2- 172.21.15.39:0/2558345527 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc63415abe0 0x7fc6341f6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:21.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.369+0000 7fc638e00640 1 -- 172.21.15.39:0/2558345527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6340af490 msgr2=0x7fc63415c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:21.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.369+0000 7fc638e00640 1 --2- 172.21.15.39:0/2558345527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6340af490 0x7fc63415c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:21.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.369+0000 7fc638e00640 1 -- 172.21.15.39:0/2558345527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6341f7b00 con 0x7fc6340b43f0 2024-09-21T07:39:21.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.369+0000 7fc639800640 1 --2- 172.21.15.39:0/2558345527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6340af490 0x7fc63415c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:39:21.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.369+0000 7fc638e00640 1 --2- 172.21.15.39:0/2558345527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6340b43f0 0x7fc6341f6eb0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fc62000c370 tx=0x7fc62000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:21.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.369+0000 7fc631600640 1 -- 172.21.15.39:0/2558345527 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc620017070 con 0x7fc6340b43f0 2024-09-21T07:39:21.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.369+0000 7fc631600640 1 -- 172.21.15.39:0/2558345527 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc6200053c0 con 0x7fc6340b43f0 2024-09-21T07:39:21.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.370+0000 7fc631600640 1 -- 172.21.15.39:0/2558345527 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc620003850 con 0x7fc6340b43f0 2024-09-21T07:39:21.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.370+0000 7fc63ac00640 1 -- 172.21.15.39:0/2558345527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc63415d0e0 con 0x7fc6340b43f0 2024-09-21T07:39:21.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.370+0000 7fc63ac00640 1 -- 172.21.15.39:0/2558345527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc63415d6a0 con 0x7fc6340b43f0 2024-09-21T07:39:21.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.371+0000 7fc63ac00640 1 -- 172.21.15.39:0/2558345527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc63415ce50 con 0x7fc6340b43f0 2024-09-21T07:39:21.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.372+0000 7fc631600640 1 -- 172.21.15.39:0/2558345527 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc62001a070 con 0x7fc6340b43f0 2024-09-21T07:39:21.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.373+0000 7fc631600640 1 --2- 172.21.15.39:0/2558345527 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc614078b90 0x7fc61407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:21.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.373+0000 7fc631600640 1 -- 172.21.15.39:0/2558345527 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 7345+0+0 (secure 0 0 0) 0x7fc62009a780 con 0x7fc6340b43f0 2024-09-21T07:39:21.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.374+0000 7fc639800640 1 --2- 172.21.15.39:0/2558345527 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc614078b90 0x7fc61407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:21.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.374+0000 7fc639800640 1 --2- 172.21.15.39:0/2558345527 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc614078b90 0x7fc61407b0a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc628036000 tx=0x7fc628004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:21.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.378+0000 7fc631600640 1 -- 172.21.15.39:0/2558345527 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc6200624b0 con 0x7fc6340b43f0 2024-09-21T07:39:21.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.613+0000 7fc63ac00640 1 -- 172.21.15.39:0/2558345527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc6340b4180 con 0x7fc6340b43f0 2024-09-21T07:39:21.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.614+0000 7fc631600640 1 -- 172.21.15.39:0/2558345527 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5246 (secure 0 0 0) 0x7fc6340b4180 con 0x7fc6340b43f0 2024-09-21T07:39:21.616 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:39:21.616 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":100,"btime":"2024-09-21T07:39:17:600648+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7113,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2821446711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2821446711},{"type":"v1","addr":"172.21.15.39:6835","nonce":2821446711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:39:16.606828+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6619,"mds_1":7080},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6619":{"gid":6619,"name":"d","rank":0,"incarnation":95,"state":"up:reconnect","state_seq":13,"addr":"172.21.15.148:6837/992188984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":992188984},{"type":"v1","addr":"172.21.15.148:6837","nonce":992188984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7080":{"gid":7080,"name":"c","rank":1,"incarnation":98,"state":"up:reconnect","state_seq":4,"addr":"172.21.15.39:6833/1456343023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1456343023},{"type":"v1","addr":"172.21.15.39:6833","nonce":1456343023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T07:39:21.616 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 100 2024-09-21T07:39:21.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.616+0000 7fc63ac00640 1 -- 172.21.15.39:0/2558345527 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc614078b90 msgr2=0x7fc61407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:21.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.616+0000 7fc63ac00640 1 --2- 172.21.15.39:0/2558345527 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc614078b90 0x7fc61407b0a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc628036000 tx=0x7fc628004730 comp rx=0 tx=0).stop 2024-09-21T07:39:21.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.616+0000 7fc63ac00640 1 -- 172.21.15.39:0/2558345527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6340b43f0 msgr2=0x7fc6341f6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:21.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.616+0000 7fc63ac00640 1 --2- 172.21.15.39:0/2558345527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6340b43f0 0x7fc6341f6eb0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fc62000c370 tx=0x7fc62000c830 comp rx=0 tx=0).stop 2024-09-21T07:39:21.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.616+0000 7fc63ac00640 1 -- 172.21.15.39:0/2558345527 shutdown_connections 2024-09-21T07:39:21.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.616+0000 7fc63ac00640 1 --2- 172.21.15.39:0/2558345527 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc614078b90 0x7fc61407b0a0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:21.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.616+0000 7fc63ac00640 1 --2- 172.21.15.39:0/2558345527 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc63415abe0 0x7fc6341f6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:21.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.616+0000 7fc63ac00640 1 --2- 172.21.15.39:0/2558345527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6340b43f0 0x7fc6341f6eb0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:21.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.616+0000 7fc63ac00640 1 --2- 172.21.15.39:0/2558345527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6340af490 0x7fc63415c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:21.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.616+0000 7fc63ac00640 1 -- 172.21.15.39:0/2558345527 >> 172.21.15.39:0/2558345527 conn(0x7fc6340b2510 msgr2=0x7fc6340ae230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:21.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.616+0000 7fc63ac00640 1 -- 172.21.15.39:0/2558345527 shutdown_connections 2024-09-21T07:39:21.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.616+0000 7fc63ac00640 1 -- 172.21.15.39:0/2558345527 wait complete. 2024-09-21T07:39:21.642 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-21T07:39:21.642 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:39:21.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.868+0000 7fca68c00640 1 -- 172.21.15.39:0/2766833224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca64155e80 msgr2=0x7fca640b4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:21.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.868+0000 7fca68c00640 1 --2- 172.21.15.39:0/2766833224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca64155e80 0x7fca640b4690 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fca5c009830 tx=0x7fca5c02a510 comp rx=0 tx=0).stop 2024-09-21T07:39:21.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.868+0000 7fca68c00640 1 -- 172.21.15.39:0/2766833224 shutdown_connections 2024-09-21T07:39:21.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.868+0000 7fca68c00640 1 --2- 172.21.15.39:0/2766833224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca64155e80 0x7fca640b4690 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:21.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.868+0000 7fca68c00640 1 --2- 172.21.15.39:0/2766833224 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fca6415f360 0x7fca6415bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:21.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.868+0000 7fca68c00640 1 -- 172.21.15.39:0/2766833224 >> 172.21.15.39:0/2766833224 conn(0x7fca6406cfc0 msgr2=0x7fca6406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:21.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.868+0000 7fca68c00640 1 -- 172.21.15.39:0/2766833224 shutdown_connections 2024-09-21T07:39:21.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.868+0000 7fca68c00640 1 -- 172.21.15.39:0/2766833224 wait complete. 2024-09-21T07:39:21.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.870+0000 7fca68c00640 1 Processor -- start 2024-09-21T07:39:21.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.870+0000 7fca68c00640 1 -- start start 2024-09-21T07:39:21.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.872+0000 7fca68c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca64155a70 0x7fca641f3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:21.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.872+0000 7fca68c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca64155e80 0x7fca6413bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:21.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.872+0000 7fca68c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fca6415f360 0x7fca64151d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:21.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.872+0000 7fca68c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fca640d0620 con 0x7fca64155e80 2024-09-21T07:39:21.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.872+0000 7fca68c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fca640b4ef0 con 0x7fca64155a70 2024-09-21T07:39:21.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.872+0000 7fca68c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fca640d07a0 con 0x7fca6415f360 2024-09-21T07:39:21.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.872+0000 7fca59400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca64155e80 0x7fca6413bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:21.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.872+0000 7fca59400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca64155e80 0x7fca6413bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33994/0 (socket says 172.21.15.39:33994) 2024-09-21T07:39:21.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.872+0000 7fca59400640 1 -- 172.21.15.39:0/2273068693 learned_addr learned my addr 172.21.15.39:0/2273068693 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:39:21.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.872+0000 7fca62800640 1 --2- 172.21.15.39:0/2273068693 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fca6415f360 0x7fca64151d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:21.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.872+0000 7fca61e00640 1 --2- 172.21.15.39:0/2273068693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca64155a70 0x7fca641f3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:21.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.872+0000 7fca59400640 1 -- 172.21.15.39:0/2273068693 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fca6415f360 msgr2=0x7fca64151d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:21.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.872+0000 7fca59400640 1 --2- 172.21.15.39:0/2273068693 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fca6415f360 0x7fca64151d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:21.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.872+0000 7fca59400640 1 -- 172.21.15.39:0/2273068693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca64155a70 msgr2=0x7fca641f3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:21.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.872+0000 7fca59400640 1 --2- 172.21.15.39:0/2273068693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca64155a70 0x7fca641f3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:21.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.873+0000 7fca59400640 1 -- 172.21.15.39:0/2273068693 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca64150c60 con 0x7fca64155e80 2024-09-21T07:39:21.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.873+0000 7fca59400640 1 --2- 172.21.15.39:0/2273068693 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca64155e80 0x7fca6413bc00 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fca4c00cb70 tx=0x7fca4c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:21.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.873+0000 7fca5aa00640 1 -- 172.21.15.39:0/2273068693 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4c016020 con 0x7fca64155e80 2024-09-21T07:39:21.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.873+0000 7fca5aa00640 1 -- 172.21.15.39:0/2273068693 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca4c00ae50 con 0x7fca64155e80 2024-09-21T07:39:21.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.873+0000 7fca68c00640 1 -- 172.21.15.39:0/2273068693 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca64150f40 con 0x7fca64155e80 2024-09-21T07:39:21.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.873+0000 7fca5aa00640 1 -- 172.21.15.39:0/2273068693 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4c012680 con 0x7fca64155e80 2024-09-21T07:39:21.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.873+0000 7fca68c00640 1 -- 172.21.15.39:0/2273068693 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca64151500 con 0x7fca64155e80 2024-09-21T07:39:21.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.874+0000 7fca68c00640 1 -- 172.21.15.39:0/2273068693 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca6414f780 con 0x7fca64155e80 2024-09-21T07:39:21.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.875+0000 7fca5aa00640 1 -- 172.21.15.39:0/2273068693 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fca4c019020 con 0x7fca64155e80 2024-09-21T07:39:21.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.876+0000 7fca5aa00640 1 --2- 172.21.15.39:0/2273068693 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fca40078b90 0x7fca4007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:21.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.876+0000 7fca5aa00640 1 -- 172.21.15.39:0/2273068693 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 7345+0+0 (secure 0 0 0) 0x7fca4c09b940 con 0x7fca64155e80 2024-09-21T07:39:21.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.877+0000 7fca61e00640 1 --2- 172.21.15.39:0/2273068693 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fca40078b90 0x7fca4007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:21.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.878+0000 7fca61e00640 1 --2- 172.21.15.39:0/2273068693 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fca40078b90 0x7fca4007b0a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fca54009b30 tx=0x7fca54005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:21.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:21.881+0000 7fca5aa00640 1 -- 172.21.15.39:0/2273068693 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fca4c020020 con 0x7fca64155e80 2024-09-21T07:39:22.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:22.115+0000 7fca68c00640 1 -- 172.21.15.39:0/2273068693 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca640708d0 con 0x7fca64155e80 2024-09-21T07:39:22.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:22.115+0000 7fca5aa00640 1 -- 172.21.15.39:0/2273068693 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5246 (secure 0 0 0) 0x7fca4c063020 con 0x7fca64155e80 2024-09-21T07:39:22.117 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:39:22.118 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":100,"btime":"2024-09-21T07:39:17:600648+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7113,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2821446711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2821446711},{"type":"v1","addr":"172.21.15.39:6835","nonce":2821446711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:39:16.606828+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6619,"mds_1":7080},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6619":{"gid":6619,"name":"d","rank":0,"incarnation":95,"state":"up:reconnect","state_seq":13,"addr":"172.21.15.148:6837/992188984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":992188984},{"type":"v1","addr":"172.21.15.148:6837","nonce":992188984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7080":{"gid":7080,"name":"c","rank":1,"incarnation":98,"state":"up:reconnect","state_seq":4,"addr":"172.21.15.39:6833/1456343023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1456343023},{"type":"v1","addr":"172.21.15.39:6833","nonce":1456343023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T07:39:22.118 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 100 2024-09-21T07:39:22.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:22.118+0000 7fca68c00640 1 -- 172.21.15.39:0/2273068693 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fca40078b90 msgr2=0x7fca4007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:22.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:22.118+0000 7fca68c00640 1 --2- 172.21.15.39:0/2273068693 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fca40078b90 0x7fca4007b0a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fca54009b30 tx=0x7fca54005f90 comp rx=0 tx=0).stop 2024-09-21T07:39:22.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:22.118+0000 7fca68c00640 1 -- 172.21.15.39:0/2273068693 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca64155e80 msgr2=0x7fca6413bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:22.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:22.118+0000 7fca68c00640 1 --2- 172.21.15.39:0/2273068693 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca64155e80 0x7fca6413bc00 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fca4c00cb70 tx=0x7fca4c00a430 comp rx=0 tx=0).stop 2024-09-21T07:39:22.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:22.118+0000 7fca68c00640 1 -- 172.21.15.39:0/2273068693 shutdown_connections 2024-09-21T07:39:22.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:22.118+0000 7fca68c00640 1 --2- 172.21.15.39:0/2273068693 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fca40078b90 0x7fca4007b0a0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:22.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:22.118+0000 7fca68c00640 1 --2- 172.21.15.39:0/2273068693 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fca6415f360 0x7fca64151d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:22.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:22.118+0000 7fca68c00640 1 --2- 172.21.15.39:0/2273068693 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca64155e80 0x7fca6413bc00 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:22.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:22.118+0000 7fca68c00640 1 --2- 172.21.15.39:0/2273068693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca64155a70 0x7fca641f3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:22.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:22.118+0000 7fca68c00640 1 -- 172.21.15.39:0/2273068693 >> 172.21.15.39:0/2273068693 conn(0x7fca6406cfc0 msgr2=0x7fca640cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:22.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:22.118+0000 7fca68c00640 1 -- 172.21.15.39:0/2273068693 shutdown_connections 2024-09-21T07:39:22.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:22.118+0000 7fca68c00640 1 -- 172.21.15.39:0/2273068693 wait complete. 2024-09-21T07:39:22.141 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:reconnect 2024-09-21T07:39:22.693 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-09-21T07:39:23.142 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:39:23.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.379+0000 7fdfe8e00640 1 -- 172.21.15.39:0/1499545168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfe406f800 msgr2=0x7fdfe40cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:23.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.379+0000 7fdfe8e00640 1 --2- 172.21.15.39:0/1499545168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfe406f800 0x7fdfe40cd8e0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fdfe415a870 tx=0x7fdfcc02a560 comp rx=0 tx=0).stop 2024-09-21T07:39:23.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.379+0000 7fdfe8e00640 1 -- 172.21.15.39:0/1499545168 shutdown_connections 2024-09-21T07:39:23.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.379+0000 7fdfe8e00640 1 --2- 172.21.15.39:0/1499545168 >> v2:172.21.15.148:3300/0 conn(0x7fdfe406fc10 0x7fdfe40d0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:23.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.379+0000 7fdfe8e00640 1 --2- 172.21.15.39:0/1499545168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfe406f800 0x7fdfe40cd8e0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:23.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.379+0000 7fdfe8e00640 1 --2- 172.21.15.39:0/1499545168 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdfe415cd30 0x7fdfe40cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:23.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.379+0000 7fdfe8e00640 1 -- 172.21.15.39:0/1499545168 >> 172.21.15.39:0/1499545168 conn(0x7fdfe406cfc0 msgr2=0x7fdfe406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:23.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.379+0000 7fdfe8e00640 1 -- 172.21.15.39:0/1499545168 shutdown_connections 2024-09-21T07:39:23.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.379+0000 7fdfe8e00640 1 -- 172.21.15.39:0/1499545168 wait complete. 2024-09-21T07:39:23.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.381+0000 7fdfe8e00640 1 Processor -- start 2024-09-21T07:39:23.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.381+0000 7fdfe8e00640 1 -- start start 2024-09-21T07:39:23.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.383+0000 7fdfe8e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfe406f800 0x7fdfe40abf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:23.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.383+0000 7fdfe8e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfe406fc10 0x7fdfe4150640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:23.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.383+0000 7fdfe8e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdfe415cd30 0x7fdfe40ad420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:23.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.383+0000 7fdfe8e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe4073710 con 0x7fdfe406f800 2024-09-21T07:39:23.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.383+0000 7fdfe8e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe40d0f10 con 0x7fdfe406fc10 2024-09-21T07:39:23.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.383+0000 7fdfe8e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fdfe4073890 con 0x7fdfe415cd30 2024-09-21T07:39:23.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.383+0000 7fdfe1e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfe406f800 0x7fdfe40abf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:23.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.383+0000 7fdfe1e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfe406f800 0x7fdfe40abf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50864/0 (socket says 172.21.15.39:50864) 2024-09-21T07:39:23.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.383+0000 7fdfe1e00640 1 -- 172.21.15.39:0/599176497 learned_addr learned my addr 172.21.15.39:0/599176497 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:39:23.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.383+0000 7fdfdbe00640 1 --2- 172.21.15.39:0/599176497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfe406fc10 0x7fdfe4150640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:23.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.383+0000 7fdfe2800640 1 --2- 172.21.15.39:0/599176497 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdfe415cd30 0x7fdfe40ad420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:23.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.383+0000 7fdfe1e00640 1 -- 172.21.15.39:0/599176497 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdfe415cd30 msgr2=0x7fdfe40ad420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:23.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.383+0000 7fdfe1e00640 1 --2- 172.21.15.39:0/599176497 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdfe415cd30 0x7fdfe40ad420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:23.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.383+0000 7fdfe1e00640 1 -- 172.21.15.39:0/599176497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfe406fc10 msgr2=0x7fdfe4150640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:23.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.383+0000 7fdfe1e00640 1 --2- 172.21.15.39:0/599176497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfe406fc10 0x7fdfe4150640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:23.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.383+0000 7fdfe1e00640 1 -- 172.21.15.39:0/599176497 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfe4154b90 con 0x7fdfe406f800 2024-09-21T07:39:23.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.384+0000 7fdfe1e00640 1 --2- 172.21.15.39:0/599176497 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfe406f800 0x7fdfe40abf80 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fdfd400b410 tx=0x7fdfd400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:23.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.384+0000 7fdfda000640 1 -- 172.21.15.39:0/599176497 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd4015020 con 0x7fdfe406f800 2024-09-21T07:39:23.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.384+0000 7fdfda000640 1 -- 172.21.15.39:0/599176497 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfd4011940 con 0x7fdfe406f800 2024-09-21T07:39:23.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.384+0000 7fdfda000640 1 -- 172.21.15.39:0/599176497 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd4011d90 con 0x7fdfe406f800 2024-09-21T07:39:23.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.384+0000 7fdfe8e00640 1 -- 172.21.15.39:0/599176497 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfe40ab4a0 con 0x7fdfe406f800 2024-09-21T07:39:23.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.384+0000 7fdfe8e00640 1 -- 172.21.15.39:0/599176497 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfe40aba60 con 0x7fdfe406f800 2024-09-21T07:39:23.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.385+0000 7fdfe8e00640 1 -- 172.21.15.39:0/599176497 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfe40d0620 con 0x7fdfe406f800 2024-09-21T07:39:23.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.386+0000 7fdfda000640 1 -- 172.21.15.39:0/599176497 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdfd4018020 con 0x7fdfe406f800 2024-09-21T07:39:23.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.387+0000 7fdfda000640 1 --2- 172.21.15.39:0/599176497 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdfc0078b90 0x7fdfc007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:23.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.387+0000 7fdfda000640 1 -- 172.21.15.39:0/599176497 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 7345+0+0 (secure 0 0 0) 0x7fdfd409b390 con 0x7fdfe406f800 2024-09-21T07:39:23.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.388+0000 7fdfdbe00640 1 --2- 172.21.15.39:0/599176497 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdfc0078b90 0x7fdfc007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:23.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.389+0000 7fdfdbe00640 1 --2- 172.21.15.39:0/599176497 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdfc0078b90 0x7fdfc007b0a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc033000 tx=0x7fdfcc004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:23.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.392+0000 7fdfda000640 1 -- 172.21.15.39:0/599176497 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdfd401f020 con 0x7fdfe406f800 2024-09-21T07:39:23.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.629+0000 7fdfe8e00640 1 -- 172.21.15.39:0/599176497 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdfe4153400 con 0x7fdfe406f800 2024-09-21T07:39:23.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.630+0000 7fdfda000640 1 -- 172.21.15.39:0/599176497 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5246 (secure 0 0 0) 0x7fdfd4063150 con 0x7fdfe406f800 2024-09-21T07:39:23.632 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:39:23.632 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":100,"btime":"2024-09-21T07:39:17:600648+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7113,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2821446711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2821446711},{"type":"v1","addr":"172.21.15.39:6835","nonce":2821446711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:39:16.606828+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6619,"mds_1":7080},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6619":{"gid":6619,"name":"d","rank":0,"incarnation":95,"state":"up:reconnect","state_seq":13,"addr":"172.21.15.148:6837/992188984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":992188984},{"type":"v1","addr":"172.21.15.148:6837","nonce":992188984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7080":{"gid":7080,"name":"c","rank":1,"incarnation":98,"state":"up:reconnect","state_seq":4,"addr":"172.21.15.39:6833/1456343023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1456343023},{"type":"v1","addr":"172.21.15.39:6833","nonce":1456343023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T07:39:23.632 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 100 2024-09-21T07:39:23.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.632+0000 7fdfe8e00640 1 -- 172.21.15.39:0/599176497 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdfc0078b90 msgr2=0x7fdfc007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:23.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.632+0000 7fdfe8e00640 1 --2- 172.21.15.39:0/599176497 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdfc0078b90 0x7fdfc007b0a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc033000 tx=0x7fdfcc004730 comp rx=0 tx=0).stop 2024-09-21T07:39:23.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.632+0000 7fdfe8e00640 1 -- 172.21.15.39:0/599176497 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfe406f800 msgr2=0x7fdfe40abf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:23.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.632+0000 7fdfe8e00640 1 --2- 172.21.15.39:0/599176497 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfe406f800 0x7fdfe40abf80 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fdfd400b410 tx=0x7fdfd400b8d0 comp rx=0 tx=0).stop 2024-09-21T07:39:23.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.632+0000 7fdfe8e00640 1 -- 172.21.15.39:0/599176497 shutdown_connections 2024-09-21T07:39:23.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.632+0000 7fdfe8e00640 1 --2- 172.21.15.39:0/599176497 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdfc0078b90 0x7fdfc007b0a0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:23.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.632+0000 7fdfe8e00640 1 --2- 172.21.15.39:0/599176497 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdfe415cd30 0x7fdfe40ad420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:23.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.632+0000 7fdfe8e00640 1 --2- 172.21.15.39:0/599176497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfe406fc10 0x7fdfe4150640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:23.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.632+0000 7fdfe8e00640 1 --2- 172.21.15.39:0/599176497 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfe406f800 0x7fdfe40abf80 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:23.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.632+0000 7fdfe8e00640 1 -- 172.21.15.39:0/599176497 >> 172.21.15.39:0/599176497 conn(0x7fdfe406cfc0 msgr2=0x7fdfe4070f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:23.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.632+0000 7fdfe8e00640 1 -- 172.21.15.39:0/599176497 shutdown_connections 2024-09-21T07:39:23.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:23.632+0000 7fdfe8e00640 1 -- 172.21.15.39:0/599176497 wait complete. 2024-09-21T07:39:23.656 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:reconnect 2024-09-21T07:39:24.657 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:39:24.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:24.897+0000 7f1f1ae00640 1 -- 172.21.15.39:0/3321873160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f140d2a50 msgr2=0x7f1f140d2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:24.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:24.897+0000 7f1f1ae00640 1 --2- 172.21.15.39:0/3321873160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f140d2a50 0x7f1f140d2e60 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f1f1408df30 tx=0x7f1f0002a5f0 comp rx=0 tx=0).stop 2024-09-21T07:39:24.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:24.897+0000 7f1f1ae00640 1 -- 172.21.15.39:0/3321873160 shutdown_connections 2024-09-21T07:39:24.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:24.897+0000 7f1f1ae00640 1 --2- 172.21.15.39:0/3321873160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f140d2a50 0x7f1f140d2e60 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:24.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:24.897+0000 7f1f1ae00640 1 --2- 172.21.15.39:0/3321873160 >> v2:172.21.15.148:3300/0 conn(0x7f1f140b4db0 0x7f1f140d4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:24.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:24.897+0000 7f1f1ae00640 1 -- 172.21.15.39:0/3321873160 >> 172.21.15.39:0/3321873160 conn(0x7f1f1408bfb0 msgr2=0x7f1f140d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:24.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:24.897+0000 7f1f1ae00640 1 -- 172.21.15.39:0/3321873160 shutdown_connections 2024-09-21T07:39:24.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:24.897+0000 7f1f1ae00640 1 -- 172.21.15.39:0/3321873160 wait complete. 2024-09-21T07:39:24.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:24.899+0000 7f1f1ae00640 1 Processor -- start 2024-09-21T07:39:24.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:24.899+0000 7f1f1ae00640 1 -- start start 2024-09-21T07:39:24.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:24.901+0000 7f1f1ae00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f140b4db0 0x7f1f141621f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:24.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:24.901+0000 7f1f1ae00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f140b6000 0x7f1f14158a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:24.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:24.901+0000 7f1f1ae00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f140d2a50 0x7f1f140cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:24.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:24.901+0000 7f1f1ae00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1f140cee90 con 0x7f1f140b6000 2024-09-21T07:39:24.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:24.901+0000 7f1f1ae00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1f140b6d30 con 0x7f1f140b4db0 2024-09-21T07:39:24.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:24.901+0000 7f1f1ae00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f1f140cf010 con 0x7f1f140d2a50 2024-09-21T07:39:24.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:24.901+0000 7f1f19000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f140b6000 0x7f1f14158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:24.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:24.901+0000 7f1f19000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f140b6000 0x7f1f14158a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50892/0 (socket says 172.21.15.39:50892) 2024-09-21T07:39:24.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:24.901+0000 7f1f19000640 1 -- 172.21.15.39:0/2284054820 learned_addr learned my addr 172.21.15.39:0/2284054820 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:39:24.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:24.901+0000 7f1f19a00640 1 --2- 172.21.15.39:0/2284054820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f140b4db0 0x7f1f141621f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:24.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:24.901+0000 7f1f1a400640 1 --2- 172.21.15.39:0/2284054820 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f140d2a50 0x7f1f140cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:24.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:24.902+0000 7f1f19000640 1 -- 172.21.15.39:0/2284054820 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f140d2a50 msgr2=0x7f1f140cbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:24.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:24.902+0000 7f1f19000640 1 --2- 172.21.15.39:0/2284054820 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f140d2a50 0x7f1f140cbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:24.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:24.902+0000 7f1f19000640 1 -- 172.21.15.39:0/2284054820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f140b4db0 msgr2=0x7f1f141621f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:24.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:24.902+0000 7f1f19000640 1 --2- 172.21.15.39:0/2284054820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f140b4db0 0x7f1f141621f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:24.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:24.902+0000 7f1f19000640 1 -- 172.21.15.39:0/2284054820 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f141f5ed0 con 0x7f1f140b6000 2024-09-21T07:39:24.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:24.902+0000 7f1f19000640 1 --2- 172.21.15.39:0/2284054820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f140b6000 0x7f1f14158a40 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f1f000099d0 tx=0x7f1f000052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:24.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:24.902+0000 7f1f11600640 1 -- 172.21.15.39:0/2284054820 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f00005cf0 con 0x7f1f140b6000 2024-09-21T07:39:24.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:24.902+0000 7f1f11600640 1 -- 172.21.15.39:0/2284054820 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f00031b50 con 0x7f1f140b6000 2024-09-21T07:39:24.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:24.902+0000 7f1f1ae00640 1 -- 172.21.15.39:0/2284054820 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f141f61b0 con 0x7f1f140b6000 2024-09-21T07:39:24.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:24.902+0000 7f1f11600640 1 -- 172.21.15.39:0/2284054820 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f000330d0 con 0x7f1f140b6000 2024-09-21T07:39:24.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:24.903+0000 7f1f1ae00640 1 -- 172.21.15.39:0/2284054820 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f140cb660 con 0x7f1f140b6000 2024-09-21T07:39:24.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:24.904+0000 7f1f1ae00640 1 -- 172.21.15.39:0/2284054820 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f141f5c20 con 0x7f1f140b6000 2024-09-21T07:39:24.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:24.905+0000 7f1f11600640 1 -- 172.21.15.39:0/2284054820 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1f00033270 con 0x7f1f140b6000 2024-09-21T07:39:24.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:24.906+0000 7f1f11600640 1 --2- 172.21.15.39:0/2284054820 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1ef4078bc0 0x7f1ef407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:24.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:24.906+0000 7f1f11600640 1 -- 172.21.15.39:0/2284054820 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 7388+0+0 (secure 0 0 0) 0x7f1f000bc6c0 con 0x7f1f140b6000 2024-09-21T07:39:24.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:24.906+0000 7f1f19a00640 1 --2- 172.21.15.39:0/2284054820 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1ef4078bc0 0x7f1ef407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:24.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:24.907+0000 7f1f19a00640 1 --2- 172.21.15.39:0/2284054820 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1ef4078bc0 0x7f1ef407b0d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1f08009c00 tx=0x7f1f08005f30 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:24.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:24.910+0000 7f1f11600640 1 -- 172.21.15.39:0/2284054820 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1f0003f030 con 0x7f1f140b6000 2024-09-21T07:39:25.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:25.144+0000 7f1f1ae00640 1 -- 172.21.15.39:0/2284054820 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f140d4c90 con 0x7f1f140b6000 2024-09-21T07:39:25.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:25.145+0000 7f1f11600640 1 -- 172.21.15.39:0/2284054820 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) ==== 95+0+5240 (secure 0 0 0) 0x7f1f00031770 con 0x7f1f140b6000 2024-09-21T07:39:25.147 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:39:25.147 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":101,"btime":"2024-09-21T07:39:24:736119+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7113,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2821446711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2821446711},{"type":"v1","addr":"172.21.15.39:6835","nonce":2821446711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:39:23.750381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6619,"mds_1":7080},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6619":{"gid":6619,"name":"d","rank":0,"incarnation":95,"state":"up:rejoin","state_seq":15,"addr":"172.21.15.148:6837/992188984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":992188984},{"type":"v1","addr":"172.21.15.148:6837","nonce":992188984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7080":{"gid":7080,"name":"c","rank":1,"incarnation":98,"state":"up:rejoin","state_seq":6,"addr":"172.21.15.39:6833/1456343023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1456343023},{"type":"v1","addr":"172.21.15.39:6833","nonce":1456343023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T07:39:25.148 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 101 2024-09-21T07:39:25.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:25.148+0000 7f1f1ae00640 1 -- 172.21.15.39:0/2284054820 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1ef4078bc0 msgr2=0x7f1ef407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:25.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:25.148+0000 7f1f1ae00640 1 --2- 172.21.15.39:0/2284054820 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1ef4078bc0 0x7f1ef407b0d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1f08009c00 tx=0x7f1f08005f30 comp rx=0 tx=0).stop 2024-09-21T07:39:25.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:25.148+0000 7f1f1ae00640 1 -- 172.21.15.39:0/2284054820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f140b6000 msgr2=0x7f1f14158a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:25.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:25.148+0000 7f1f1ae00640 1 --2- 172.21.15.39:0/2284054820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f140b6000 0x7f1f14158a40 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f1f000099d0 tx=0x7f1f000052f0 comp rx=0 tx=0).stop 2024-09-21T07:39:25.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:25.148+0000 7f1f1ae00640 1 -- 172.21.15.39:0/2284054820 shutdown_connections 2024-09-21T07:39:25.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:25.148+0000 7f1f1ae00640 1 --2- 172.21.15.39:0/2284054820 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1ef4078bc0 0x7f1ef407b0d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:25.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:25.148+0000 7f1f1ae00640 1 --2- 172.21.15.39:0/2284054820 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f140d2a50 0x7f1f140cbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:25.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:25.148+0000 7f1f1ae00640 1 --2- 172.21.15.39:0/2284054820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f140b6000 0x7f1f14158a40 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:25.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:25.148+0000 7f1f1ae00640 1 --2- 172.21.15.39:0/2284054820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f140b4db0 0x7f1f141621f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:25.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:25.148+0000 7f1f1ae00640 1 -- 172.21.15.39:0/2284054820 >> 172.21.15.39:0/2284054820 conn(0x7f1f1408bfb0 msgr2=0x7f1f140d1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:25.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:25.148+0000 7f1f1ae00640 1 -- 172.21.15.39:0/2284054820 shutdown_connections 2024-09-21T07:39:25.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:25.148+0000 7f1f1ae00640 1 -- 172.21.15.39:0/2284054820 wait complete. 2024-09-21T07:39:25.176 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:rejoin 2024-09-21T07:39:26.178 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:39:26.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.411+0000 7f3dd3200640 1 -- 172.21.15.39:0/3596175797 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3db8005150 msgr2=0x7f3db800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:26.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.411+0000 7f3dd3200640 1 --2- 172.21.15.39:0/3596175797 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3db8005150 0x7f3db800b9c0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc00c610 tx=0x7f3db8005830 comp rx=0 tx=0).stop 2024-09-21T07:39:26.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.411+0000 7f3dd2800640 1 -- 172.21.15.39:0/3596175797 reap_dead start 2024-09-21T07:39:26.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.411+0000 7f3dd3200640 1 -- 172.21.15.39:0/3596175797 shutdown_connections 2024-09-21T07:39:26.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.411+0000 7f3dd3200640 1 -- 172.21.15.39:0/3596175797 >> 172.21.15.39:0/3596175797 conn(0x7f3dd4074050 msgr2=0x7f3dd4074490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:26.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.411+0000 7f3dd3200640 1 -- 172.21.15.39:0/3596175797 shutdown_connections 2024-09-21T07:39:26.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.411+0000 7f3dd3200640 1 -- 172.21.15.39:0/3596175797 wait complete. 2024-09-21T07:39:26.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.414+0000 7f3dd3200640 1 Processor -- start 2024-09-21T07:39:26.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.414+0000 7f3dd3200640 1 -- start start 2024-09-21T07:39:26.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.415+0000 7f3dd3200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd4158ee0 0x7f3dd4153a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:26.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.415+0000 7f3dd3200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3dd415b6b0 0x7f3dd4142e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:26.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.415+0000 7f3dd3200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dd415bac0 0x7f3dd41433b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:26.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.415+0000 7f3dd3200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd40cf1e0 con 0x7f3dd4158ee0 2024-09-21T07:39:26.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.415+0000 7f3dd3200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd4071e90 con 0x7f3dd415bac0 2024-09-21T07:39:26.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.415+0000 7f3dd3200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f3dd40cf360 con 0x7f3dd415b6b0 2024-09-21T07:39:26.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.415+0000 7f3dd1e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd4158ee0 0x7f3dd4153a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:26.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.415+0000 7f3dd1e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd4158ee0 0x7f3dd4153a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50912/0 (socket says 172.21.15.39:50912) 2024-09-21T07:39:26.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.415+0000 7f3dd1e00640 1 -- 172.21.15.39:0/3114939778 learned_addr learned my addr 172.21.15.39:0/3114939778 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:39:26.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.416+0000 7f3dd2800640 1 --2- 172.21.15.39:0/3114939778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dd415bac0 0x7f3dd41433b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:26.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.416+0000 7f3dd1400640 1 --2- 172.21.15.39:0/3114939778 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3dd415b6b0 0x7f3dd4142e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:26.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.416+0000 7f3dd1e00640 1 -- 172.21.15.39:0/3114939778 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3dd415b6b0 msgr2=0x7f3dd4142e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:26.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.416+0000 7f3dd1e00640 1 --2- 172.21.15.39:0/3114939778 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3dd415b6b0 0x7f3dd4142e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:26.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.416+0000 7f3dd1e00640 1 -- 172.21.15.39:0/3114939778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dd415bac0 msgr2=0x7f3dd41433b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:26.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.416+0000 7f3dd1e00640 1 --2- 172.21.15.39:0/3114939778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dd415bac0 0x7f3dd41433b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:26.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.416+0000 7f3dd1e00640 1 -- 172.21.15.39:0/3114939778 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd4155450 con 0x7f3dd4158ee0 2024-09-21T07:39:26.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.416+0000 7f3dd1400640 1 --2- 172.21.15.39:0/3114939778 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3dd415b6b0 0x7f3dd4142e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:39:26.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.416+0000 7f3dd1e00640 1 --2- 172.21.15.39:0/3114939778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd4158ee0 0x7f3dd4153a40 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4011a80 tx=0x7f3dc4011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:26.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.416+0000 7f3dca000640 1 -- 172.21.15.39:0/3114939778 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc401f070 con 0x7f3dd4158ee0 2024-09-21T07:39:26.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.416+0000 7f3dca000640 1 -- 172.21.15.39:0/3114939778 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3dc4012450 con 0x7f3dd4158ee0 2024-09-21T07:39:26.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.416+0000 7f3dd3200640 1 -- 172.21.15.39:0/3114939778 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dd41543b0 con 0x7f3dd4158ee0 2024-09-21T07:39:26.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.416+0000 7f3dca000640 1 -- 172.21.15.39:0/3114939778 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4012880 con 0x7f3dd4158ee0 2024-09-21T07:39:26.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.417+0000 7f3dd3200640 1 -- 172.21.15.39:0/3114939778 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dd4154970 con 0x7f3dd4158ee0 2024-09-21T07:39:26.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.418+0000 7f3dd3200640 1 -- 172.21.15.39:0/3114939778 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dd40d03e0 con 0x7f3dd4158ee0 2024-09-21T07:39:26.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.419+0000 7f3dca000640 1 -- 172.21.15.39:0/3114939778 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3dc4022070 con 0x7f3dd4158ee0 2024-09-21T07:39:26.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.420+0000 7f3dca000640 1 --2- 172.21.15.39:0/3114939778 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3db4078b90 0x7f3db407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:26.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.420+0000 7f3dca000640 1 -- 172.21.15.39:0/3114939778 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 7388+0+0 (secure 0 0 0) 0x7f3dc409c0a0 con 0x7f3dd4158ee0 2024-09-21T07:39:26.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.420+0000 7f3dd1400640 1 --2- 172.21.15.39:0/3114939778 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3db4078b90 0x7f3db407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:26.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.421+0000 7f3dd1400640 1 --2- 172.21.15.39:0/3114939778 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3db4078b90 0x7f3db407b0a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4073580 tx=0x7f3dc0005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:26.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.424+0000 7f3dca000640 1 -- 172.21.15.39:0/3114939778 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3dd40d03e0 con 0x7f3dd4158ee0 2024-09-21T07:39:26.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.662+0000 7f3dd3200640 1 -- 172.21.15.39:0/3114939778 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3dd40694e0 con 0x7f3dd4158ee0 2024-09-21T07:39:26.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.662+0000 7f3dca000640 1 -- 172.21.15.39:0/3114939778 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+5252 (secure 0 0 0) 0x7f3dd40694e0 con 0x7f3dd4158ee0 2024-09-21T07:39:26.664 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:39:26.665 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":102,"btime":"2024-09-21T07:39:25:746836+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7113,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2821446711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2821446711},{"type":"v1","addr":"172.21.15.39:6835","nonce":2821446711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:39:25.746826+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6619,"mds_1":7080},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6619":{"gid":6619,"name":"d","rank":0,"incarnation":95,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/992188984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":992188984},{"type":"v1","addr":"172.21.15.148:6837","nonce":992188984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7080":{"gid":7080,"name":"c","rank":1,"incarnation":98,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6833/1456343023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1456343023},{"type":"v1","addr":"172.21.15.39:6833","nonce":1456343023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6619,"qdb_cluster":[6619,7080]},"id":6}]} 2024-09-21T07:39:26.665 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 102 2024-09-21T07:39:26.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.664+0000 7f3dd3200640 1 -- 172.21.15.39:0/3114939778 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3db4078b90 msgr2=0x7f3db407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:26.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.664+0000 7f3dd3200640 1 --2- 172.21.15.39:0/3114939778 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3db4078b90 0x7f3db407b0a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4073580 tx=0x7f3dc0005f90 comp rx=0 tx=0).stop 2024-09-21T07:39:26.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.664+0000 7f3dd3200640 1 -- 172.21.15.39:0/3114939778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd4158ee0 msgr2=0x7f3dd4153a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:26.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.664+0000 7f3dd3200640 1 --2- 172.21.15.39:0/3114939778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd4158ee0 0x7f3dd4153a40 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4011a80 tx=0x7f3dc4011f40 comp rx=0 tx=0).stop 2024-09-21T07:39:26.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.665+0000 7f3dd3200640 1 -- 172.21.15.39:0/3114939778 shutdown_connections 2024-09-21T07:39:26.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.665+0000 7f3dd3200640 1 --2- 172.21.15.39:0/3114939778 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3db4078b90 0x7f3db407b0a0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:26.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.665+0000 7f3dd3200640 1 --2- 172.21.15.39:0/3114939778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dd415bac0 0x7f3dd41433b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:26.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.665+0000 7f3dd3200640 1 --2- 172.21.15.39:0/3114939778 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3dd415b6b0 0x7f3dd4142e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:26.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.665+0000 7f3dd3200640 1 --2- 172.21.15.39:0/3114939778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd4158ee0 0x7f3dd4153a40 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:26.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.665+0000 7f3dd3200640 1 -- 172.21.15.39:0/3114939778 >> 172.21.15.39:0/3114939778 conn(0x7f3dd4074050 msgr2=0x7f3dd4158140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:26.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.665+0000 7f3dd3200640 1 -- 172.21.15.39:0/3114939778 shutdown_connections 2024-09-21T07:39:26.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.665+0000 7f3dd3200640 1 -- 172.21.15.39:0/3114939778 wait complete. 2024-09-21T07:39:26.688 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:active 2024-09-21T07:39:26.689 DEBUG:tasks.cephfs.filesystem:reached state 'up:active' in 5.046415567398071s 2024-09-21T07:39:26.689 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:39:26.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.916+0000 7f677ce00640 1 -- 172.21.15.39:0/1647571683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f677815c830 msgr2=0x7f677806f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:26.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.916+0000 7f677ce00640 1 --2- 172.21.15.39:0/1647571683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f677815c830 0x7f677806f8a0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f6764005fd0 tx=0x7f676402a5d0 comp rx=0 tx=0).stop 2024-09-21T07:39:26.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.916+0000 7f677ce00640 1 -- 172.21.15.39:0/1647571683 shutdown_connections 2024-09-21T07:39:26.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.916+0000 7f677ce00640 1 --2- 172.21.15.39:0/1647571683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f677815c830 0x7f677806f8a0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:26.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.916+0000 7f677ce00640 1 -- 172.21.15.39:0/1647571683 >> 172.21.15.39:0/1647571683 conn(0x7f67780b2510 msgr2=0x7f67780d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:26.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.916+0000 7f677ce00640 1 -- 172.21.15.39:0/1647571683 shutdown_connections 2024-09-21T07:39:26.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.916+0000 7f677ce00640 1 -- 172.21.15.39:0/1647571683 wait complete. 2024-09-21T07:39:26.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.918+0000 7f677ce00640 1 Processor -- start 2024-09-21T07:39:26.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.918+0000 7f677ce00640 1 -- start start 2024-09-21T07:39:26.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.920+0000 7f677ce00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f677806fde0 0x7f6778150370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:26.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.920+0000 7f677ce00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67780d0f20 0x7f677809b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:26.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.920+0000 7f677ce00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f677815c830 0x7f67781f70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:26.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.920+0000 7f677ce00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f67780b47c0 con 0x7f67780d0f20 2024-09-21T07:39:26.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.920+0000 7f677ce00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f67780b4640 con 0x7f677806fde0 2024-09-21T07:39:26.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.920+0000 7f677ce00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f67780b4940 con 0x7f677815c830 2024-09-21T07:39:26.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.920+0000 7f6776a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67780d0f20 0x7f677809b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:26.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.920+0000 7f6777400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f677806fde0 0x7f6778150370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:26.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.920+0000 7f6776a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67780d0f20 0x7f677809b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50934/0 (socket says 172.21.15.39:50934) 2024-09-21T07:39:26.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.921+0000 7f6776a00640 1 -- 172.21.15.39:0/1849880006 learned_addr learned my addr 172.21.15.39:0/1849880006 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:39:26.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.921+0000 7f6777e00640 1 --2- 172.21.15.39:0/1849880006 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f677815c830 0x7f67781f70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:26.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.921+0000 7f6776a00640 1 -- 172.21.15.39:0/1849880006 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f677815c830 msgr2=0x7f67781f70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:26.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.921+0000 7f6776a00640 1 --2- 172.21.15.39:0/1849880006 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f677815c830 0x7f67781f70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:26.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.921+0000 7f6776a00640 1 -- 172.21.15.39:0/1849880006 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f677806fde0 msgr2=0x7f6778150370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:26.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.921+0000 7f6776a00640 1 --2- 172.21.15.39:0/1849880006 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f677806fde0 0x7f6778150370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:26.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.921+0000 7f6776a00640 1 -- 172.21.15.39:0/1849880006 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f677809a7d0 con 0x7f67780d0f20 2024-09-21T07:39:26.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.921+0000 7f6777e00640 1 --2- 172.21.15.39:0/1849880006 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f677815c830 0x7f67781f70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:39:26.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.921+0000 7f6776a00640 1 --2- 172.21.15.39:0/1849880006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67780d0f20 0x7f677809b870 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f6764034040 tx=0x7f67640038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:26.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.922+0000 7f676f400640 1 -- 172.21.15.39:0/1849880006 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676403e030 con 0x7f67780d0f20 2024-09-21T07:39:26.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.922+0000 7f676f400640 1 -- 172.21.15.39:0/1849880006 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6764003ca0 con 0x7f67780d0f20 2024-09-21T07:39:26.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.922+0000 7f677ce00640 1 -- 172.21.15.39:0/1849880006 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f677809aab0 con 0x7f67780d0f20 2024-09-21T07:39:26.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.922+0000 7f676f400640 1 -- 172.21.15.39:0/1849880006 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6764035040 con 0x7f67780d0f20 2024-09-21T07:39:26.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.922+0000 7f677ce00640 1 -- 172.21.15.39:0/1849880006 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f677809b070 con 0x7f67780d0f20 2024-09-21T07:39:26.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.923+0000 7f677ce00640 1 -- 172.21.15.39:0/1849880006 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f677806f8a0 con 0x7f67780d0f20 2024-09-21T07:39:26.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.924+0000 7f676f400640 1 -- 172.21.15.39:0/1849880006 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6764041030 con 0x7f67780d0f20 2024-09-21T07:39:26.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.925+0000 7f676f400640 1 --2- 172.21.15.39:0/1849880006 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6758078b90 0x7f675807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:26.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.925+0000 7f676f400640 1 -- 172.21.15.39:0/1849880006 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 7388+0+0 (secure 0 0 0) 0x7f67640bba70 con 0x7f67780d0f20 2024-09-21T07:39:26.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.925+0000 7f6777400640 1 --2- 172.21.15.39:0/1849880006 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6758078b90 0x7f675807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:26.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.926+0000 7f6777400640 1 --2- 172.21.15.39:0/1849880006 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6758078b90 0x7f675807b0a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6768006fd0 tx=0x7f6768009060 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:26.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:26.929+0000 7f676f400640 1 -- 172.21.15.39:0/1849880006 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6764083130 con 0x7f67780d0f20 2024-09-21T07:39:27.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.169+0000 7f677ce00640 1 -- 172.21.15.39:0/1849880006 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6778159b00 con 0x7f67780d0f20 2024-09-21T07:39:27.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.169+0000 7f676f400640 1 -- 172.21.15.39:0/1849880006 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+5252 (secure 0 0 0) 0x7f6764087b60 con 0x7f67780d0f20 2024-09-21T07:39:27.171 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:39:27.171 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":102,"btime":"2024-09-21T07:39:25:746836+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7113,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2821446711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2821446711},{"type":"v1","addr":"172.21.15.39:6835","nonce":2821446711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:39:25.746826+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6619,"mds_1":7080},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6619":{"gid":6619,"name":"d","rank":0,"incarnation":95,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/992188984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":992188984},{"type":"v1","addr":"172.21.15.148:6837","nonce":992188984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7080":{"gid":7080,"name":"c","rank":1,"incarnation":98,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6833/1456343023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1456343023},{"type":"v1","addr":"172.21.15.39:6833","nonce":1456343023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6619,"qdb_cluster":[6619,7080]},"id":6}]} 2024-09-21T07:39:27.172 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 102 2024-09-21T07:39:27.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.172+0000 7f677ce00640 1 -- 172.21.15.39:0/1849880006 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6758078b90 msgr2=0x7f675807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:27.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.172+0000 7f677ce00640 1 --2- 172.21.15.39:0/1849880006 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6758078b90 0x7f675807b0a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6768006fd0 tx=0x7f6768009060 comp rx=0 tx=0).stop 2024-09-21T07:39:27.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.172+0000 7f677ce00640 1 -- 172.21.15.39:0/1849880006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67780d0f20 msgr2=0x7f677809b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:27.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.172+0000 7f677ce00640 1 --2- 172.21.15.39:0/1849880006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67780d0f20 0x7f677809b870 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f6764034040 tx=0x7f67640038a0 comp rx=0 tx=0).stop 2024-09-21T07:39:27.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.172+0000 7f677ce00640 1 -- 172.21.15.39:0/1849880006 shutdown_connections 2024-09-21T07:39:27.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.172+0000 7f677ce00640 1 --2- 172.21.15.39:0/1849880006 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6758078b90 0x7f675807b0a0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:27.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.172+0000 7f677ce00640 1 --2- 172.21.15.39:0/1849880006 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f677815c830 0x7f67781f70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:27.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.172+0000 7f677ce00640 1 --2- 172.21.15.39:0/1849880006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67780d0f20 0x7f677809b870 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:27.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.172+0000 7f677ce00640 1 --2- 172.21.15.39:0/1849880006 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f677806fde0 0x7f6778150370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:27.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.172+0000 7f677ce00640 1 -- 172.21.15.39:0/1849880006 >> 172.21.15.39:0/1849880006 conn(0x7f67780b2510 msgr2=0x7f677815e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:27.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.172+0000 7f677ce00640 1 -- 172.21.15.39:0/1849880006 shutdown_connections 2024-09-21T07:39:27.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.172+0000 7f677ce00640 1 -- 172.21.15.39:0/1849880006 wait complete. 2024-09-21T07:39:27.198 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json dump snaps --server 2024-09-21T07:39:27.452 INFO:teuthology.orchestra.run.smithi148.stdout:{"last_snap":10,"last_created":5,"last_destroyed":9,"pending_noop":[],"snaps":[],"need_to_purge":{"16":[2,3,4,5,6,7,8,9]},"pending_update":[],"pending_destroy":[]} 2024-09-21T07:39:27.453 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 5, "last_destroyed": 9, "last_snap": 10, "need_to_purge": { "16": [ 2, 3, 4, 5, 6, 7, 8, 9 ] }, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [] } 2024-09-21T07:39:27.453 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T07:39:27.453 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:39:27.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.681+0000 7fefc7e00640 1 -- 172.21.15.39:0/703851693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefc80cc390 msgr2=0x7fefc80af600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:27.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.681+0000 7fefc7e00640 1 --2- 172.21.15.39:0/703851693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefc80cc390 0x7fefc80af600 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fefb8009830 tx=0x7fefb802a530 comp rx=0 tx=0).stop 2024-09-21T07:39:27.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.681+0000 7fefc7e00640 1 -- 172.21.15.39:0/703851693 shutdown_connections 2024-09-21T07:39:27.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.681+0000 7fefc7e00640 1 --2- 172.21.15.39:0/703851693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefc80cc390 0x7fefc80af600 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:27.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.681+0000 7fefc7e00640 1 -- 172.21.15.39:0/703851693 >> 172.21.15.39:0/703851693 conn(0x7fefc80b2510 msgr2=0x7fefc80d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:27.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.681+0000 7fefc7e00640 1 -- 172.21.15.39:0/703851693 shutdown_connections 2024-09-21T07:39:27.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.681+0000 7fefc7e00640 1 -- 172.21.15.39:0/703851693 wait complete. 2024-09-21T07:39:27.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.683+0000 7fefc7e00640 1 Processor -- start 2024-09-21T07:39:27.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.683+0000 7fefc7e00640 1 -- start start 2024-09-21T07:39:27.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.685+0000 7fefc7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefc80ca890 0x7fefc80aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:27.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.685+0000 7fefc7e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefc80cc390 0x7fefc81f83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:27.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.685+0000 7fefc7e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefc80ce310 0x7fefc8156e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:27.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.685+0000 7fefc7e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fefc80cb7c0 con 0x7fefc80ca890 2024-09-21T07:39:27.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.685+0000 7fefc7e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fefc80cb640 con 0x7fefc80ce310 2024-09-21T07:39:27.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.685+0000 7fefc7e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fefc80cb940 con 0x7fefc80cc390 2024-09-21T07:39:27.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.685+0000 7fefc6a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefc80ca890 0x7fefc80aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:27.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.685+0000 7fefc6a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefc80ca890 0x7fefc80aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50946/0 (socket says 172.21.15.39:50946) 2024-09-21T07:39:27.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.685+0000 7fefc6a00640 1 -- 172.21.15.39:0/2002968277 learned_addr learned my addr 172.21.15.39:0/2002968277 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:39:27.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.685+0000 7fefc7400640 1 --2- 172.21.15.39:0/2002968277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefc80ce310 0x7fefc8156e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:27.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.685+0000 7fefbfe00640 1 --2- 172.21.15.39:0/2002968277 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefc80cc390 0x7fefc81f83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:27.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.685+0000 7fefc6a00640 1 -- 172.21.15.39:0/2002968277 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefc80cc390 msgr2=0x7fefc81f83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:27.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.685+0000 7fefc6a00640 1 --2- 172.21.15.39:0/2002968277 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefc80cc390 0x7fefc81f83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:27.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.685+0000 7fefc6a00640 1 -- 172.21.15.39:0/2002968277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefc80ce310 msgr2=0x7fefc8156e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:27.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.685+0000 7fefc6a00640 1 --2- 172.21.15.39:0/2002968277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefc80ce310 0x7fefc8156e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:27.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.685+0000 7fefc6a00640 1 -- 172.21.15.39:0/2002968277 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc815c210 con 0x7fefc80ca890 2024-09-21T07:39:27.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.686+0000 7fefc6a00640 1 --2- 172.21.15.39:0/2002968277 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefc80ca890 0x7fefc80aed60 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fefb8009780 tx=0x7fefb8033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:27.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.686+0000 7fefbf400640 1 -- 172.21.15.39:0/2002968277 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb8031b50 con 0x7fefc80ca890 2024-09-21T07:39:27.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.686+0000 7fefbf400640 1 -- 172.21.15.39:0/2002968277 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fefb8031de0 con 0x7fefc80ca890 2024-09-21T07:39:27.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.686+0000 7fefbf400640 1 -- 172.21.15.39:0/2002968277 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb8005280 con 0x7fefc80ca890 2024-09-21T07:39:27.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.686+0000 7fefc7e00640 1 -- 172.21.15.39:0/2002968277 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc81e8ab0 con 0x7fefc80ca890 2024-09-21T07:39:27.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.686+0000 7fefc7e00640 1 -- 172.21.15.39:0/2002968277 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc80c3600 con 0x7fefc80ca890 2024-09-21T07:39:27.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.687+0000 7fefc7e00640 1 -- 172.21.15.39:0/2002968277 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefc80cbe80 con 0x7fefc80ca890 2024-09-21T07:39:27.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.688+0000 7fefbf400640 1 -- 172.21.15.39:0/2002968277 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fefb8005420 con 0x7fefc80ca890 2024-09-21T07:39:27.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.690+0000 7fefbf400640 1 --2- 172.21.15.39:0/2002968277 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fefa4078b90 0x7fefa407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:27.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.690+0000 7fefbf400640 1 -- 172.21.15.39:0/2002968277 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 7388+0+0 (secure 0 0 0) 0x7fefb80bc4c0 con 0x7fefc80ca890 2024-09-21T07:39:27.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.690+0000 7fefbfe00640 1 --2- 172.21.15.39:0/2002968277 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fefa4078b90 0x7fefa407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:27.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.691+0000 7fefbfe00640 1 --2- 172.21.15.39:0/2002968277 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fefa4078b90 0x7fefa407b0a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fefb0007fd0 tx=0x7fefb00013b0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:27.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.695+0000 7fefbf400640 1 -- 172.21.15.39:0/2002968277 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fefb803f030 con 0x7fefc80ca890 2024-09-21T07:39:27.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.938+0000 7fefc7e00640 1 -- 172.21.15.39:0/2002968277 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fefc80b3a60 con 0x7fefc80ca890 2024-09-21T07:39:27.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.939+0000 7fefbf400640 1 -- 172.21.15.39:0/2002968277 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+5252 (secure 0 0 0) 0x7fefb8031770 con 0x7fefc80ca890 2024-09-21T07:39:27.942 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:39:27.942 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":102,"btime":"2024-09-21T07:39:25:746836+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7113,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2821446711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2821446711},{"type":"v1","addr":"172.21.15.39:6835","nonce":2821446711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:39:25.746826+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6619,"mds_1":7080},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6619":{"gid":6619,"name":"d","rank":0,"incarnation":95,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/992188984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":992188984},{"type":"v1","addr":"172.21.15.148:6837","nonce":992188984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7080":{"gid":7080,"name":"c","rank":1,"incarnation":98,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6833/1456343023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1456343023},{"type":"v1","addr":"172.21.15.39:6833","nonce":1456343023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6619,"qdb_cluster":[6619,7080]},"id":6}]} 2024-09-21T07:39:27.942 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 102 2024-09-21T07:39:27.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.942+0000 7fefc7e00640 1 -- 172.21.15.39:0/2002968277 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fefa4078b90 msgr2=0x7fefa407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:27.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.942+0000 7fefc7e00640 1 --2- 172.21.15.39:0/2002968277 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fefa4078b90 0x7fefa407b0a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fefb0007fd0 tx=0x7fefb00013b0 comp rx=0 tx=0).stop 2024-09-21T07:39:27.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.942+0000 7fefc7e00640 1 -- 172.21.15.39:0/2002968277 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefc80ca890 msgr2=0x7fefc80aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:27.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.942+0000 7fefc7e00640 1 --2- 172.21.15.39:0/2002968277 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefc80ca890 0x7fefc80aed60 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fefb8009780 tx=0x7fefb8033ba0 comp rx=0 tx=0).stop 2024-09-21T07:39:27.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.942+0000 7fefc7e00640 1 -- 172.21.15.39:0/2002968277 shutdown_connections 2024-09-21T07:39:27.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.942+0000 7fefc7e00640 1 --2- 172.21.15.39:0/2002968277 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fefa4078b90 0x7fefa407b0a0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:27.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.942+0000 7fefc7e00640 1 --2- 172.21.15.39:0/2002968277 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefc80ce310 0x7fefc8156e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:27.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.942+0000 7fefc7e00640 1 --2- 172.21.15.39:0/2002968277 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefc80cc390 0x7fefc81f83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:27.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.942+0000 7fefc7e00640 1 --2- 172.21.15.39:0/2002968277 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefc80ca890 0x7fefc80aed60 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:27.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.942+0000 7fefc7e00640 1 -- 172.21.15.39:0/2002968277 >> 172.21.15.39:0/2002968277 conn(0x7fefc80b2510 msgr2=0x7fefc81566c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:27.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.942+0000 7fefc7e00640 1 -- 172.21.15.39:0/2002968277 shutdown_connections 2024-09-21T07:39:27.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:27.942+0000 7fefc7e00640 1 -- 172.21.15.39:0/2002968277 wait complete. 2024-09-21T07:39:27.967 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json dump snaps --server 2024-09-21T07:39:28.204 INFO:teuthology.orchestra.run.smithi148.stdout:{"last_snap":10,"last_created":5,"last_destroyed":9,"pending_noop":[],"snaps":[],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2024-09-21T07:39:28.205 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 5, "last_destroyed": 9, "last_snap": 10, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [] } 2024-09-21T07:39:28.205 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:39:28.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:39:28.205 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-09-21T07:39:28.233 INFO:tasks.ceph.mds.d.smithi148.stderr:2024-09-21T07:39:28.232+0000 7f7699600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:39:28.233 INFO:tasks.ceph.mds.d.smithi148.stderr:2024-09-21T07:39:28.232+0000 7f7699600640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:39:31.029 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:39:31.034 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:39:35.608 INFO:teuthology.orchestra.run.smithi039.stderr:mkdir: cannot create directory ‘d1/dir/.snap/s22’: Permission denied 2024-09-21T07:39:35.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:39:35.609 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:39:35.609 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:39:35.647 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:39:35.647 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:39:35.724 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:39:35.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:35.991+0000 7f2dfb600640 1 -- 172.21.15.39:0/3585606292 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2df415cd30 msgr2=0x7f2df40cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:35.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:35.991+0000 7f2dfb600640 1 --2- 172.21.15.39:0/3585606292 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2df415cd30 0x7f2df40cfd10 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f2de8005fd0 tx=0x7f2de802a5d0 comp rx=0 tx=0).stop 2024-09-21T07:39:35.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:35.991+0000 7f2dfb600640 1 -- 172.21.15.39:0/3585606292 shutdown_connections 2024-09-21T07:39:35.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:35.991+0000 7f2dfb600640 1 --2- 172.21.15.39:0/3585606292 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2df406f800 0x7f2df40cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:35.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:35.991+0000 7f2dfb600640 1 --2- 172.21.15.39:0/3585606292 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2df415cd30 0x7f2df40cfd10 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:35.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:35.991+0000 7f2dfb600640 1 -- 172.21.15.39:0/3585606292 >> 172.21.15.39:0/3585606292 conn(0x7f2df406cfc0 msgr2=0x7f2df406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:35.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:35.991+0000 7f2dfb600640 1 -- 172.21.15.39:0/3585606292 shutdown_connections 2024-09-21T07:39:35.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:35.991+0000 7f2dfb600640 1 -- 172.21.15.39:0/3585606292 wait complete. 2024-09-21T07:39:35.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:35.994+0000 7f2dfb600640 1 Processor -- start 2024-09-21T07:39:35.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:35.994+0000 7f2dfb600640 1 -- start start 2024-09-21T07:39:35.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:35.995+0000 7f2dfb600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2df406f800 0x7f2df4147cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:35.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:35.995+0000 7f2dfb600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2df406fc10 0x7f2df40b4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:35.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:35.995+0000 7f2dfb600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2df415cd30 0x7f2df41581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:35.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:35.995+0000 7f2dfb600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2df4073890 con 0x7f2df406f800 2024-09-21T07:39:35.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:35.995+0000 7f2dfb600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2df4073710 con 0x7f2df415cd30 2024-09-21T07:39:35.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:35.995+0000 7f2dfb600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f2df4073a10 con 0x7f2df406fc10 2024-09-21T07:39:35.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:35.995+0000 7f2df8c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2df406f800 0x7f2df4147cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:35.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:35.995+0000 7f2df8c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2df406f800 0x7f2df4147cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41534/0 (socket says 172.21.15.39:41534) 2024-09-21T07:39:35.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:35.995+0000 7f2df8c00640 1 -- 172.21.15.39:0/1666645188 learned_addr learned my addr 172.21.15.39:0/1666645188 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:39:35.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:35.996+0000 7f2df9600640 1 --2- 172.21.15.39:0/1666645188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2df415cd30 0x7f2df41581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:35.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:35.996+0000 7f2df3e00640 1 --2- 172.21.15.39:0/1666645188 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2df406fc10 0x7f2df40b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:35.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:35.996+0000 7f2df8c00640 1 -- 172.21.15.39:0/1666645188 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2df406fc10 msgr2=0x7f2df40b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:35.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:35.996+0000 7f2df8c00640 1 --2- 172.21.15.39:0/1666645188 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2df406fc10 0x7f2df40b4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:35.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:35.996+0000 7f2df8c00640 1 -- 172.21.15.39:0/1666645188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2df415cd30 msgr2=0x7f2df41581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:35.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:35.996+0000 7f2df8c00640 1 --2- 172.21.15.39:0/1666645188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2df415cd30 0x7f2df41581d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:35.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:35.996+0000 7f2df8c00640 1 -- 172.21.15.39:0/1666645188 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df41f59e0 con 0x7f2df406f800 2024-09-21T07:39:35.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:35.996+0000 7f2df9600640 1 --2- 172.21.15.39:0/1666645188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2df415cd30 0x7f2df41581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:39:35.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:35.996+0000 7f2df8c00640 1 --2- 172.21.15.39:0/1666645188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2df406f800 0x7f2df4147cc0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f2de8006010 tx=0x7f2de80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:36.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:35.996+0000 7f2df0c00640 1 -- 172.21.15.39:0/1666645188 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de803e030 con 0x7f2df406f800 2024-09-21T07:39:36.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:35.997+0000 7f2df0c00640 1 -- 172.21.15.39:0/1666645188 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2de8003ca0 con 0x7f2df406f800 2024-09-21T07:39:36.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:35.997+0000 7f2df0c00640 1 -- 172.21.15.39:0/1666645188 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de8035040 con 0x7f2df406f800 2024-09-21T07:39:36.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:35.997+0000 7f2dfb600640 1 -- 172.21.15.39:0/1666645188 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2df41e7660 con 0x7f2df406f800 2024-09-21T07:39:36.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:35.997+0000 7f2dfb600640 1 -- 172.21.15.39:0/1666645188 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2df41e7900 con 0x7f2df406f800 2024-09-21T07:39:36.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:35.998+0000 7f2dfb600640 1 -- 172.21.15.39:0/1666645188 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2df4073ec0 con 0x7f2df406f800 2024-09-21T07:39:36.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:35.999+0000 7f2df0c00640 1 -- 172.21.15.39:0/1666645188 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2de8041030 con 0x7f2df406f800 2024-09-21T07:39:36.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.000+0000 7f2df0c00640 1 --2- 172.21.15.39:0/1666645188 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2dd4078bc0 0x7f2dd407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:36.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.000+0000 7f2df0c00640 1 -- 172.21.15.39:0/1666645188 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 7476+0+0 (secure 0 0 0) 0x7f2de80bbbb0 con 0x7f2df406f800 2024-09-21T07:39:36.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.001+0000 7f2df3e00640 1 --2- 172.21.15.39:0/1666645188 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2dd4078bc0 0x7f2dd407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:36.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.002+0000 7f2df3e00640 1 --2- 172.21.15.39:0/1666645188 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2dd4078bc0 0x7f2dd407b0d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2de00058c0 tx=0x7f2de0005850 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:36.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.005+0000 7f2df0c00640 1 -- 172.21.15.39:0/1666645188 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2de8083860 con 0x7f2df406f800 2024-09-21T07:39:36.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.213+0000 7f2dfb600640 1 -- 172.21.15.39:0/1666645188 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2df4071450 con 0x7f2df406f800 2024-09-21T07:39:36.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.213+0000 7f2df0c00640 1 -- 172.21.15.39:0/1666645188 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 13 entries v102) ==== 81+0+785 (secure 0 0 0) 0x7f2de8083160 con 0x7f2df406f800 2024-09-21T07:39:36.216 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/364657398 2024-09-22T07:39:15.579613+0000 2024-09-21T07:39:36.216 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/855150775 2024-09-22T07:39:07.739045+0000 2024-09-21T07:39:36.216 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6833/364657398 2024-09-22T07:39:15.579613+0000 2024-09-21T07:39:36.216 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/855150775 2024-09-22T07:39:07.739045+0000 2024-09-21T07:39:36.216 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/2381504293 2024-09-22T07:38:32.207414+0000 2024-09-21T07:39:36.216 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/2381504293 2024-09-22T07:38:32.207414+0000 2024-09-21T07:39:36.216 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.1:0/90671776 2024-09-21T08:39:22.988045+0000 2024-09-21T07:39:36.216 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/1472430787 2024-09-22T07:37:58.212892+0000 2024-09-21T07:39:36.216 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/3026827614 2024-09-22T07:37:29.781472+0000 2024-09-21T07:39:36.216 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/3026827614 2024-09-22T07:37:29.781472+0000 2024-09-21T07:39:36.216 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/829716143 2024-09-22T07:37:01.259219+0000 2024-09-21T07:39:36.216 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/1472430787 2024-09-22T07:37:58.212892+0000 2024-09-21T07:39:36.216 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/829716143 2024-09-22T07:37:01.259219+0000 2024-09-21T07:39:36.217 INFO:teuthology.orchestra.run.smithi039.stderr:listed 13 entries 2024-09-21T07:39:36.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.216+0000 7f2dfb600640 1 -- 172.21.15.39:0/1666645188 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2dd4078bc0 msgr2=0x7f2dd407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:36.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.216+0000 7f2dfb600640 1 --2- 172.21.15.39:0/1666645188 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2dd4078bc0 0x7f2dd407b0d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2de00058c0 tx=0x7f2de0005850 comp rx=0 tx=0).stop 2024-09-21T07:39:36.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.216+0000 7f2dfb600640 1 -- 172.21.15.39:0/1666645188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2df406f800 msgr2=0x7f2df4147cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:36.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.216+0000 7f2dfb600640 1 --2- 172.21.15.39:0/1666645188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2df406f800 0x7f2df4147cc0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f2de8006010 tx=0x7f2de80038a0 comp rx=0 tx=0).stop 2024-09-21T07:39:36.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.216+0000 7f2dfb600640 1 -- 172.21.15.39:0/1666645188 shutdown_connections 2024-09-21T07:39:36.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.216+0000 7f2dfb600640 1 --2- 172.21.15.39:0/1666645188 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2dd4078bc0 0x7f2dd407b0d0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:36.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.216+0000 7f2dfb600640 1 --2- 172.21.15.39:0/1666645188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2df415cd30 0x7f2df41581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:36.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.216+0000 7f2dfb600640 1 --2- 172.21.15.39:0/1666645188 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2df406fc10 0x7f2df40b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:36.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.216+0000 7f2dfb600640 1 --2- 172.21.15.39:0/1666645188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2df406f800 0x7f2df4147cc0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:36.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.216+0000 7f2dfb600640 1 -- 172.21.15.39:0/1666645188 >> 172.21.15.39:0/1666645188 conn(0x7f2df406cfc0 msgr2=0x7f2df415c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:36.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.216+0000 7f2dfb600640 1 -- 172.21.15.39:0/1666645188 shutdown_connections 2024-09-21T07:39:36.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.216+0000 7f2dfb600640 1 -- 172.21.15.39:0/1666645188 wait complete. 2024-09-21T07:39:36.243 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T07:39:36.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:39:36.243 DEBUG:teuthology.orchestra.run.smithi039:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T07:39:36.583 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T07:39:36.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:39:36.584 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:39:36.621 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:39:36.621 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:39:36.697 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:39:36.697 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:39:36.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.966+0000 7f6443200640 1 -- 172.21.15.39:0/1022508515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64440d0f20 msgr2=0x7f644415c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:36.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.966+0000 7f6443200640 1 --2- 172.21.15.39:0/1022508515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64440d0f20 0x7f644415c830 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f6434005fd0 tx=0x7f643402a530 comp rx=0 tx=0).stop 2024-09-21T07:39:36.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.966+0000 7f6443200640 1 -- 172.21.15.39:0/1022508515 shutdown_connections 2024-09-21T07:39:36.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.966+0000 7f6443200640 1 --2- 172.21.15.39:0/1022508515 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f64440cb960 0x7f64440cbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:36.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.966+0000 7f6443200640 1 --2- 172.21.15.39:0/1022508515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64440d0f20 0x7f644415c830 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:36.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.966+0000 7f6443200640 1 -- 172.21.15.39:0/1022508515 >> 172.21.15.39:0/1022508515 conn(0x7f64440b2510 msgr2=0x7f64440d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:36.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.966+0000 7f6443200640 1 -- 172.21.15.39:0/1022508515 shutdown_connections 2024-09-21T07:39:36.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.966+0000 7f6443200640 1 -- 172.21.15.39:0/1022508515 wait complete. 2024-09-21T07:39:36.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.968+0000 7f6443200640 1 Processor -- start 2024-09-21T07:39:36.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.968+0000 7f6443200640 1 -- start start 2024-09-21T07:39:36.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.970+0000 7f6443200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f644406f8a0 0x7f64441f2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:36.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.970+0000 7f6443200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64440cb960 0x7f644415a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:36.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.970+0000 7f6443200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64440d0f20 0x7f644415cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:36.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.970+0000 7f6443200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f64440b4570 con 0x7f64440cb960 2024-09-21T07:39:36.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.970+0000 7f6443200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f64440b43f0 con 0x7f64440d0f20 2024-09-21T07:39:36.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.970+0000 7f6443200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f64440b46f0 con 0x7f644406f8a0 2024-09-21T07:39:36.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.970+0000 7f643be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64440cb960 0x7f644415a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:36.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.970+0000 7f643be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64440cb960 0x7f644415a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41560/0 (socket says 172.21.15.39:41560) 2024-09-21T07:39:36.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.970+0000 7f643be00640 1 -- 172.21.15.39:0/3033383628 learned_addr learned my addr 172.21.15.39:0/3033383628 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:39:36.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.970+0000 7f6441e00640 1 --2- 172.21.15.39:0/3033383628 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f644406f8a0 0x7f64441f2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:36.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.970+0000 7f6442800640 1 --2- 172.21.15.39:0/3033383628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64440d0f20 0x7f644415cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:36.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.970+0000 7f643be00640 1 -- 172.21.15.39:0/3033383628 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f644406f8a0 msgr2=0x7f64441f2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:36.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.970+0000 7f643be00640 1 --2- 172.21.15.39:0/3033383628 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f644406f8a0 0x7f64441f2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:36.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.971+0000 7f643be00640 1 -- 172.21.15.39:0/3033383628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64440d0f20 msgr2=0x7f644415cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:36.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.971+0000 7f643be00640 1 --2- 172.21.15.39:0/3033383628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64440d0f20 0x7f644415cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:36.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.971+0000 7f643be00640 1 -- 172.21.15.39:0/3033383628 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64441f5910 con 0x7f64440cb960 2024-09-21T07:39:36.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.971+0000 7f643be00640 1 --2- 172.21.15.39:0/3033383628 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64440cb960 0x7f644415a040 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f642c00e7f0 tx=0x7f642c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:36.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.971+0000 7f643a000640 1 -- 172.21.15.39:0/3033383628 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c00ccc0 con 0x7f64440cb960 2024-09-21T07:39:36.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.971+0000 7f643a000640 1 -- 172.21.15.39:0/3033383628 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f642c00ce80 con 0x7f64440cb960 2024-09-21T07:39:36.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.971+0000 7f643a000640 1 -- 172.21.15.39:0/3033383628 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c005280 con 0x7f64440cb960 2024-09-21T07:39:36.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.971+0000 7f6443200640 1 -- 172.21.15.39:0/3033383628 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64441f3160 con 0x7f64440cb960 2024-09-21T07:39:36.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.971+0000 7f6443200640 1 -- 172.21.15.39:0/3033383628 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64441f5ed0 con 0x7f64440cb960 2024-09-21T07:39:36.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.972+0000 7f6443200640 1 -- 172.21.15.39:0/3033383628 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64440b4c50 con 0x7f64440cb960 2024-09-21T07:39:36.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.974+0000 7f643a000640 1 -- 172.21.15.39:0/3033383628 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f642c005420 con 0x7f64440cb960 2024-09-21T07:39:36.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.975+0000 7f643a000640 1 --2- 172.21.15.39:0/3033383628 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6420078bc0 0x7f642007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:36.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.975+0000 7f643a000640 1 -- 172.21.15.39:0/3033383628 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 7432+0+0 (secure 0 0 0) 0x7f642c09bd20 con 0x7f64440cb960 2024-09-21T07:39:36.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.976+0000 7f6441e00640 1 --2- 172.21.15.39:0/3033383628 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6420078bc0 0x7f642007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:36.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.976+0000 7f6441e00640 1 --2- 172.21.15.39:0/3033383628 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6420078bc0 0x7f642007b0d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6434033000 tx=0x7f6434004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:36.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:36.980+0000 7f643a000640 1 -- 172.21.15.39:0/3033383628 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f642c0633b0 con 0x7f64440cb960 2024-09-21T07:39:37.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:37.218+0000 7f6443200640 1 -- 172.21.15.39:0/3033383628 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f64440702b0 con 0x7f64440cb960 2024-09-21T07:39:37.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:37.219+0000 7f643a000640 1 -- 172.21.15.39:0/3033383628 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v102) ==== 53+0+73 (secure 0 0 0) 0x7f642c067de0 con 0x7f64440cb960 2024-09-21T07:39:37.221 INFO:teuthology.orchestra.run.smithi039.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:39:37.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:37.221+0000 7f6443200640 1 -- 172.21.15.39:0/3033383628 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6420078bc0 msgr2=0x7f642007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:37.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:37.221+0000 7f6443200640 1 --2- 172.21.15.39:0/3033383628 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6420078bc0 0x7f642007b0d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6434033000 tx=0x7f6434004730 comp rx=0 tx=0).stop 2024-09-21T07:39:37.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:37.221+0000 7f6443200640 1 -- 172.21.15.39:0/3033383628 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64440cb960 msgr2=0x7f644415a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:37.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:37.221+0000 7f6443200640 1 --2- 172.21.15.39:0/3033383628 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64440cb960 0x7f644415a040 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f642c00e7f0 tx=0x7f642c00ecb0 comp rx=0 tx=0).stop 2024-09-21T07:39:37.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:37.221+0000 7f6443200640 1 -- 172.21.15.39:0/3033383628 shutdown_connections 2024-09-21T07:39:37.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:37.221+0000 7f6443200640 1 --2- 172.21.15.39:0/3033383628 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6420078bc0 0x7f642007b0d0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:37.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:37.221+0000 7f6443200640 1 --2- 172.21.15.39:0/3033383628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f64440d0f20 0x7f644415cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:37.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:37.221+0000 7f6443200640 1 --2- 172.21.15.39:0/3033383628 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64440cb960 0x7f644415a040 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:37.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:37.221+0000 7f6443200640 1 --2- 172.21.15.39:0/3033383628 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f644406f8a0 0x7f64441f2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:37.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:37.221+0000 7f6443200640 1 -- 172.21.15.39:0/3033383628 >> 172.21.15.39:0/3033383628 conn(0x7f64440b2510 msgr2=0x7f64440b15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:37.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:37.222+0000 7f6443200640 1 -- 172.21.15.39:0/3033383628 shutdown_connections 2024-09-21T07:39:37.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:37.222+0000 7f6443200640 1 -- 172.21.15.39:0/3033383628 wait complete. 2024-09-21T07:39:37.249 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:39:37.249 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:39:37.249 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi039.front.sepia.ceph.com 2024-09-21T07:39:37.249 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T07:39:37.249 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:39:37.249 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:39:37.249 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:39:37.249 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:39:37.250 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:39:37.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:39:37.250 DEBUG:teuthology.orchestra.run.smithi039:> ip addr 2024-09-21T07:39:37.286 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:39:37.286 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:39:37.286 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:39:37.286 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:39:37.287 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2024-09-21T07:39:37.287 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:39:37.287 INFO:teuthology.orchestra.run.smithi039.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:39:37.287 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:39:37.287 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f0 2024-09-21T07:39:37.287 INFO:teuthology.orchestra.run.smithi039.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:39:37.287 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c7 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:39:37.287 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f1 2024-09-21T07:39:37.287 INFO:teuthology.orchestra.run.smithi039.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:39:37.287 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:70 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:39:37.287 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f0 2024-09-21T07:39:37.287 INFO:teuthology.orchestra.run.smithi039.stdout: inet 172.21.15.39/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T07:39:37.288 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft 42342sec preferred_lft 42342sec 2024-09-21T07:39:37.288 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec4:7aff:febd:1370/64 scope link 2024-09-21T07:39:37.288 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:39:37.288 INFO:teuthology.orchestra.run.smithi039.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:39:37.288 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:71 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:39:37.288 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f1 2024-09-21T07:39:37.288 INFO:teuthology.orchestra.run.smithi039.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:39:37.288 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff 2024-09-21T07:39:37.288 INFO:teuthology.orchestra.run.smithi039.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:39:37.288 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:39:37.288 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::b079:ddff:fe0e:6333/64 scope link 2024-09-21T07:39:37.288 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:39:37.288 INFO:teuthology.orchestra.run.smithi039.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:39:37.288 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:39:37.289 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::2ca7:98ff:fe8a:d0ed/64 scope link 2024-09-21T07:39:37.289 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:39:37.289 INFO:teuthology.orchestra.run.smithi039.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:39:37.289 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether ee:10:53:01:0c:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:39:37.289 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec10:53ff:fe01:c44/64 scope link 2024-09-21T07:39:37.289 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:39:37.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:39:37.289 DEBUG:teuthology.orchestra.run.smithi039:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:39:37.380 INFO:teuthology.orchestra.run.smithi039.stdout:1 2024-09-21T07:39:37.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:39:37.384 DEBUG:teuthology.orchestra.run.smithi039:> route 2024-09-21T07:39:37.419 INFO:teuthology.orchestra.run.smithi039.stdout:Kernel IP routing table 2024-09-21T07:39:37.420 INFO:teuthology.orchestra.run.smithi039.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:39:37.420 INFO:teuthology.orchestra.run.smithi039.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T07:39:37.420 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T07:39:37.420 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:39:37.420 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:39:37.420 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-21T07:39:37.420 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T07:39:37.420 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T07:39:37.420 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T07:39:37.421 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-21T07:39:37.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:39:37.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:39:37.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:39:37.616 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-09-21T07:39:37.651 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:39:37.651 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T07:39:37.652 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:39:37.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:39:37.652 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-09-21T07:39:37.748 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:39:37.748 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T07:39:37.783 INFO:teuthology.orchestra.run.smithi039.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:39:37.783 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:39:37.783 DEBUG:teuthology.orchestra.run.smithi039:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:39:37.857 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-21T07:39:37.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:39:37.857 DEBUG:teuthology.orchestra.run.smithi039:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:39:38.057 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:39:38.057 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:39:38.094 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:39:38.094 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:39:38.094 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: resolved to: "172.21.15.39:3300,172.21.15.148:3300,172.21.15.148:3301" 2024-09-21T07:39:38.094 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: trying mount with new device syntax: 0@7ab9b04a-f88d-49aa-a493-e5c076637ffb.cephfs=/ 2024-09-21T07:39:38.094 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.39:3300/172.21.15.148:3300/172.21.15.148:3301" will pass to kernel 2024-09-21T07:39:38.095 INFO:teuthology.orchestra.run.smithi039.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T07:39:38.096 INFO:teuthology.orchestra.run.smithi039.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:39:38.096 INFO:teuthology.orchestra.run.smithi039.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:39:38.096 INFO:teuthology.orchestra.run.smithi039.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:39:38.096 INFO:teuthology.orchestra.run.smithi039.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:39:38.099 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:39:38.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:39:38.099 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:39:38.154 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:39:38.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:39:38.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:39:38.244 INFO:teuthology.orchestra.run.smithi039.stdout:client6826 2024-09-21T07:39:38.245 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:39:38.245 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:39:38.280 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 flush journal 2024-09-21T07:39:38.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.547+0000 7f54a3200640 1 -- 172.21.15.39:0/102766096 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f54a4156d60 msgr2=0x7f54a40cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:38.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.547+0000 7f54a3200640 1 --2- 172.21.15.39:0/102766096 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f54a4156d60 0x7f54a40cc430 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5494009830 tx=0x7f549402a510 comp rx=0 tx=0).stop 2024-09-21T07:39:38.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.547+0000 7f54a3200640 1 -- 172.21.15.39:0/102766096 shutdown_connections 2024-09-21T07:39:38.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.547+0000 7f54a3200640 1 --2- 172.21.15.39:0/102766096 >> v2:172.21.15.148:3300/0 conn(0x7f54a40af690 0x7f54a4160230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:38.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.547+0000 7f54a3200640 1 --2- 172.21.15.39:0/102766096 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f54a4156d60 0x7f54a40cc430 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:38.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.547+0000 7f54a3200640 1 -- 172.21.15.39:0/102766096 >> 172.21.15.39:0/102766096 conn(0x7f54a40b25f0 msgr2=0x7f54a40d01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:38.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.547+0000 7f54a3200640 1 -- 172.21.15.39:0/102766096 shutdown_connections 2024-09-21T07:39:38.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.547+0000 7f54a3200640 1 -- 172.21.15.39:0/102766096 wait complete. 2024-09-21T07:39:38.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.550+0000 7f54a3200640 1 Processor -- start 2024-09-21T07:39:38.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.550+0000 7f54a3200640 1 -- start start 2024-09-21T07:39:38.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.551+0000 7f54a3200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f54a40af690 0x7f54a4070110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:38.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.551+0000 7f54a3200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54a4156d60 0x7f54a4070650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:38.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.551+0000 7f54a3200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54a4160770 0x7f54a41f6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:38.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.551+0000 7f54a3200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f54a415d300 con 0x7f54a4156d60 2024-09-21T07:39:38.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.551+0000 7f54a3200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f54a40ae760 con 0x7f54a4160770 2024-09-21T07:39:38.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.551+0000 7f54a3200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f54a415d480 con 0x7f54a40af690 2024-09-21T07:39:38.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.551+0000 7f549be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54a4156d60 0x7f54a4070650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:38.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.551+0000 7f549be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54a4156d60 0x7f54a4070650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41568/0 (socket says 172.21.15.39:41568) 2024-09-21T07:39:38.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.551+0000 7f549be00640 1 -- 172.21.15.39:0/3318235440 learned_addr learned my addr 172.21.15.39:0/3318235440 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:39:38.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.552+0000 7f549be00640 1 -- 172.21.15.39:0/3318235440 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f54a40af690 msgr2=0x7f54a4070110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:38.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.552+0000 7f549be00640 1 --2- 172.21.15.39:0/3318235440 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f54a40af690 0x7f54a4070110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:38.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.552+0000 7f549be00640 1 -- 172.21.15.39:0/3318235440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54a4160770 msgr2=0x7f54a41f6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:38.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.552+0000 7f549be00640 1 --2- 172.21.15.39:0/3318235440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54a4160770 0x7f54a41f6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:38.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.552+0000 7f549be00640 1 -- 172.21.15.39:0/3318235440 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a40c7e50 con 0x7f54a4156d60 2024-09-21T07:39:38.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.552+0000 7f549be00640 1 --2- 172.21.15.39:0/3318235440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54a4156d60 0x7f54a4070650 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f548c00e9f0 tx=0x7f548c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:38.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.552+0000 7f549a000640 1 -- 172.21.15.39:0/3318235440 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548c00cd80 con 0x7f54a4156d60 2024-09-21T07:39:38.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.552+0000 7f549a000640 1 -- 172.21.15.39:0/3318235440 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f548c004020 con 0x7f54a4156d60 2024-09-21T07:39:38.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.552+0000 7f549a000640 1 -- 172.21.15.39:0/3318235440 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548c004470 con 0x7f54a4156d60 2024-09-21T07:39:38.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.552+0000 7f54a3200640 1 -- 172.21.15.39:0/3318235440 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54a41f3800 con 0x7f54a4156d60 2024-09-21T07:39:38.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.553+0000 7f54a3200640 1 -- 172.21.15.39:0/3318235440 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54a41f3dc0 con 0x7f54a4156d60 2024-09-21T07:39:38.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.556+0000 7f549a000640 1 -- 172.21.15.39:0/3318235440 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f548c0035f0 con 0x7f54a4156d60 2024-09-21T07:39:38.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.557+0000 7f549a000640 1 --2- 172.21.15.39:0/3318235440 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5480078b90 0x7f548007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:38.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.557+0000 7f54afe56740 1 Processor -- start 2024-09-21T07:39:38.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.557+0000 7f549a000640 1 -- 172.21.15.39:0/3318235440 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 7432+0+0 (secure 0 0 0) 0x7f548c09bd90 con 0x7f54a4156d60 2024-09-21T07:39:38.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.557+0000 7f54afe56740 1 -- start start 2024-09-21T07:39:38.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.558+0000 7f54a1e00640 1 --2- 172.21.15.39:0/3318235440 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5480078b90 0x7f548007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:38.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.558+0000 7f54a1e00640 1 --2- 172.21.15.39:0/3318235440 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5480078b90 0x7f548007b0a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5494034000 tx=0x7f54940046c0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:38.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.559+0000 7f54afe56740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557473873390 0x557473c61730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:38.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.559+0000 7f54afe56740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55747384a4d0 0x557473c44400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:38.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.559+0000 7f54afe56740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5574738526b0 0x557473c44940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:38.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.559+0000 7f54afe56740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557473b7a1c0 con 0x557473873390 2024-09-21T07:39:38.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.559+0000 7f54afe56740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557473b8d870 con 0x55747384a4d0 2024-09-21T07:39:38.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.559+0000 7f54afe56740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x557473b7cca0 con 0x5574738526b0 2024-09-21T07:39:38.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.559+0000 7f54a1e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557473873390 0x557473c61730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:38.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.559+0000 7f54a1e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557473873390 0x557473c61730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41572/0 (socket says 172.21.15.39:41572) 2024-09-21T07:39:38.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.559+0000 7f54a1e00640 1 -- 172.21.15.39:0/2153687353 learned_addr learned my addr 172.21.15.39:0/2153687353 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:39:38.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.560+0000 7f549be00640 1 --2- 172.21.15.39:0/2153687353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55747384a4d0 0x557473c44400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:38.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.560+0000 7f54a2800640 1 --2- 172.21.15.39:0/2153687353 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5574738526b0 0x557473c44940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:38.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.560+0000 7f54a1e00640 1 -- 172.21.15.39:0/2153687353 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5574738526b0 msgr2=0x557473c44940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:39:38.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.560+0000 7f54a1e00640 1 --2- 172.21.15.39:0/2153687353 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5574738526b0 0x557473c44940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:38.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.560+0000 7f54a1e00640 1 -- 172.21.15.39:0/2153687353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55747384a4d0 msgr2=0x557473c44400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:39:38.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.560+0000 7f54a1e00640 1 --2- 172.21.15.39:0/2153687353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55747384a4d0 0x557473c44400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:38.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.560+0000 7f54a1e00640 1 -- 172.21.15.39:0/2153687353 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574739557d0 con 0x557473873390 2024-09-21T07:39:38.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.560+0000 7f54a1e00640 1 --2- 172.21.15.39:0/2153687353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557473873390 0x557473c61730 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f54940042b0 tx=0x7f549403c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c93aec605200d2a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:38.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.560+0000 7f5492a00640 1 -- 172.21.15.39:0/2153687353 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549404c070 con 0x557473873390 2024-09-21T07:39:38.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.560+0000 7f5492a00640 1 -- 172.21.15.39:0/2153687353 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54940038e0 con 0x557473873390 2024-09-21T07:39:38.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.560+0000 7f5492a00640 1 -- 172.21.15.39:0/2153687353 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5494003cd0 con 0x557473873390 2024-09-21T07:39:38.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.561+0000 7f54afe56740 1 -- 172.21.15.39:0/2153687353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557473873390 msgr2=0x557473c61730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:38.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.561+0000 7f54afe56740 1 --2- 172.21.15.39:0/2153687353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557473873390 0x557473c61730 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f54940042b0 tx=0x7f549403c000 comp rx=0 tx=0).stop 2024-09-21T07:39:38.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.561+0000 7f54afe56740 1 -- 172.21.15.39:0/2153687353 shutdown_connections 2024-09-21T07:39:38.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.561+0000 7f54afe56740 1 --2- 172.21.15.39:0/2153687353 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5574738526b0 0x557473c44940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:38.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.561+0000 7f54afe56740 1 --2- 172.21.15.39:0/2153687353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55747384a4d0 0x557473c44400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:38.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.561+0000 7f54afe56740 1 --2- 172.21.15.39:0/2153687353 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557473873390 0x557473c61730 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:38.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.561+0000 7f54afe56740 1 -- 172.21.15.39:0/2153687353 >> 172.21.15.39:0/2153687353 conn(0x5574738674e0 msgr2=0x557473c1a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:38.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.561+0000 7f54afe56740 1 -- 172.21.15.39:0/2153687353 shutdown_connections 2024-09-21T07:39:38.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.561+0000 7f54afe56740 1 -- 172.21.15.39:0/2153687353 wait complete. 2024-09-21T07:39:38.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.562+0000 7f54afe56740 1 -- start start 2024-09-21T07:39:38.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.563+0000 7f54afe56740 1 Processor -- start 2024-09-21T07:39:38.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.564+0000 7f54afe56740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55747384a4d0 0x557473c5afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:38.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.564+0000 7f54afe56740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5574738526b0 0x557473c71c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:38.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.564+0000 7f54afe56740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557473873390 0x557473c72160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:38.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.564+0000 7f54afe56740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557473b23410 con 0x55747384a4d0 2024-09-21T07:39:38.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.564+0000 7f54afe56740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557473b80ab0 con 0x5574738526b0 2024-09-21T07:39:38.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.564+0000 7f54afe56740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x5574738ed740 con 0x557473873390 2024-09-21T07:39:38.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.565+0000 7f54afe56740 10 client.0 _ll_register_callbacks cb 0x557473b00360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:39:38.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.565+0000 7f54a1e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55747384a4d0 0x557473c5afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:38.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.565+0000 7f54a1e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55747384a4d0 0x557473c5afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41582/0 (socket says 172.21.15.39:41582) 2024-09-21T07:39:38.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.565+0000 7f54a1e00640 1 -- 172.21.15.39:0/1535193195 learned_addr learned my addr 172.21.15.39:0/1535193195 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:39:38.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.565+0000 7f54a2800640 1 --2- 172.21.15.39:0/1535193195 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557473873390 0x557473c72160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:38.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.565+0000 7f549be00640 1 --2- 172.21.15.39:0/1535193195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5574738526b0 0x557473c71c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:38.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.565+0000 7f54a1e00640 1 -- 172.21.15.39:0/1535193195 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557473873390 msgr2=0x557473c72160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:39:38.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.565+0000 7f54a1e00640 1 --2- 172.21.15.39:0/1535193195 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557473873390 0x557473c72160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:38.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.565+0000 7f54a1e00640 1 -- 172.21.15.39:0/1535193195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5574738526b0 msgr2=0x557473c71c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:39:38.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.565+0000 7f54a1e00640 1 --2- 172.21.15.39:0/1535193195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5574738526b0 0x557473c71c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:38.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.565+0000 7f54a1e00640 1 -- 172.21.15.39:0/1535193195 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574738de970 con 0x55747384a4d0 2024-09-21T07:39:38.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.565+0000 7f549be00640 1 --2- 172.21.15.39:0/1535193195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5574738526b0 0x557473c71c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:39:38.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.565+0000 7f54a1e00640 1 --2- 172.21.15.39:0/1535193195 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55747384a4d0 0x557473c5afc0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f549402ab00 tx=0x7f54940095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c45b51657e890b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:38.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.565+0000 7f5487400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:39:38.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.566+0000 7f5487400640 1 -- 172.21.15.39:0/1535193195 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549405a060 con 0x55747384a4d0 2024-09-21T07:39:38.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.566+0000 7f5487400640 1 -- 172.21.15.39:0/1535193195 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 7432+0+0 (secure 0 0 0) 0x7f5494048920 con 0x55747384a4d0 2024-09-21T07:39:38.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.566+0000 7f54afe56740 1 -- 172.21.15.39:0/1535193195 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54940038e0 con 0x55747384a4d0 2024-09-21T07:39:38.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.566+0000 7f5487400640 1 -- 172.21.15.39:0/1535193195 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7f5494060050 con 0x55747384a4d0 2024-09-21T07:39:38.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.566+0000 7f54afe56740 10 client.7200 fetch_fsmap learned FSMap version 102 2024-09-21T07:39:38.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.566+0000 7f54afe56740 1 -- 172.21.15.39:0/1535193195 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=2+}) -- 0x5574738bc6a0 con 0x55747384a4d0 2024-09-21T07:39:38.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.566+0000 7f5487400640 1 -- 172.21.15.39:0/1535193195 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54940038e0 con 0x55747384a4d0 2024-09-21T07:39:38.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.566+0000 7f5487400640 1 -- 172.21.15.39:0/1535193195 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 102) ==== 3522+0+0 (secure 0 0 0) 0x7f5494063020 con 0x55747384a4d0 2024-09-21T07:39:38.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.566+0000 7f54afe56740 10 client.7200 fetch_fsmap finished waiting for FSMap version 102 2024-09-21T07:39:38.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.566+0000 7f54afe56740 10 client.7200 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-09-21T07:39:38.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.566+0000 7f54afe56740 20 client.7200 populate_metadata read hostname 'smithi039' 2024-09-21T07:39:38.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.566+0000 7f54afe56740 1 --2- 172.21.15.39:0/1535193195 >> [v2:172.21.15.148:6836/992188984,v1:172.21.15.148:6837/992188984] conn(0x557473831f60 0x557473c59ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:38.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.567+0000 7f54afe56740 4 client.7200 mds_command: new command op to 6619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:39:38.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.567+0000 7f54afe56740 1 -- 172.21.15.39:0/1535193195 --> [v2:172.21.15.148:6836/992188984,v1:172.21.15.148:6837/992188984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557473b8f7c0 con 0x557473831f60 2024-09-21T07:39:38.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.567+0000 7f549be00640 1 --2- 172.21.15.39:0/1535193195 >> [v2:172.21.15.148:6836/992188984,v1:172.21.15.148:6837/992188984] conn(0x557473831f60 0x557473c59ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:38.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.568+0000 7f549be00640 1 --2- 172.21.15.39:0/1535193195 >> [v2:172.21.15.148:6836/992188984,v1:172.21.15.148:6837/992188984] conn(0x557473831f60 0x557473c59ca0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=640f5d20c91ccc5 server_cookie=56d33744ae57a89b in_seq=0 out_seq=0 2024-09-21T07:39:38.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.568+0000 7f5487400640 10 client.7200 ms_handle_connect on v2:172.21.15.148:6836/992188984 2024-09-21T07:39:38.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.568+0000 7f5487400640 1 -- 172.21.15.39:0/1535193195 <== mds.0 v2:172.21.15.148:6836/992188984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557473b8f7c0 con 0x557473831f60 2024-09-21T07:39:38.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.568+0000 7f5487400640 10 client.7200 handle_command_reply: tid=0 2024-09-21T07:39:38.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.569+0000 7f54afe56740 1 client.7200 shutdown 2024-09-21T07:39:38.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.569+0000 7f54afe56740 1 -- 172.21.15.39:0/1535193195 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55747384a4d0 msgr2=0x557473c5afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:38.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.569+0000 7f54afe56740 1 --2- 172.21.15.39:0/1535193195 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55747384a4d0 0x557473c5afc0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f549402ab00 tx=0x7f54940095b0 comp rx=0 tx=0).stop 2024-09-21T07:39:38.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.569+0000 7f54afe56740 1 -- 172.21.15.39:0/1535193195 shutdown_connections 2024-09-21T07:39:38.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.569+0000 7f54afe56740 1 --2- 172.21.15.39:0/1535193195 >> [v2:172.21.15.148:6836/992188984,v1:172.21.15.148:6837/992188984] conn(0x557473831f60 0x557473c59ca0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:38.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.569+0000 7f54afe56740 1 --2- 172.21.15.39:0/1535193195 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557473873390 0x557473c72160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:38.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.569+0000 7f54afe56740 1 --2- 172.21.15.39:0/1535193195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5574738526b0 0x557473c71c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:38.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.569+0000 7f54afe56740 1 --2- 172.21.15.39:0/1535193195 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55747384a4d0 0x557473c5afc0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:38.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.569+0000 7f5487400640 0 client.7200 ms_handle_reset on v2:172.21.15.148:6836/992188984 2024-09-21T07:39:38.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.569+0000 7f54afe56740 1 -- 172.21.15.39:0/1535193195 >> 172.21.15.39:0/1535193195 conn(0x5574738674e0 msgr2=0x557473c13830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:38.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.569+0000 7f54afe56740 1 -- 172.21.15.39:0/1535193195 shutdown_connections 2024-09-21T07:39:38.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.569+0000 7f54afe56740 1 -- 172.21.15.39:0/1535193195 wait complete. 2024-09-21T07:39:38.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.569+0000 7f54afe56740 20 client.7200 trim_cache size 0 max 16384 2024-09-21T07:39:38.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.585+0000 7f54afe56740 1 Processor -- start 2024-09-21T07:39:38.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.585+0000 7f54afe56740 1 -- start start 2024-09-21T07:39:38.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.586+0000 7f54afe56740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55747384a4d0 0x557473c13830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:38.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.586+0000 7f54afe56740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5574738526b0 0x557473c13d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:38.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.586+0000 7f54afe56740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557473873390 0x557473c59ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:38.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.586+0000 7f54afe56740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f54a4202470 con 0x5574738526b0 2024-09-21T07:39:38.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.586+0000 7f54afe56740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f54a4202b40 con 0x55747384a4d0 2024-09-21T07:39:38.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.586+0000 7f54afe56740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f54a4200e40 con 0x557473873390 2024-09-21T07:39:38.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.587+0000 7f549be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5574738526b0 0x557473c13d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:38.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.587+0000 7f549be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5574738526b0 0x557473c13d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41586/0 (socket says 172.21.15.39:41586) 2024-09-21T07:39:38.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.587+0000 7f549be00640 1 -- 172.21.15.39:0/490456986 learned_addr learned my addr 172.21.15.39:0/490456986 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:39:38.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.587+0000 7f54a1e00640 1 --2- 172.21.15.39:0/490456986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55747384a4d0 0x557473c13830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:38.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.587+0000 7f54a2800640 1 --2- 172.21.15.39:0/490456986 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557473873390 0x557473c59ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:38.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.587+0000 7f549be00640 1 -- 172.21.15.39:0/490456986 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557473873390 msgr2=0x557473c59ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:39:38.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.587+0000 7f549be00640 1 --2- 172.21.15.39:0/490456986 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557473873390 0x557473c59ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:38.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.587+0000 7f549be00640 1 -- 172.21.15.39:0/490456986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55747384a4d0 msgr2=0x557473c13830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:39:38.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.587+0000 7f549be00640 1 --2- 172.21.15.39:0/490456986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55747384a4d0 0x557473c13830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:38.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.587+0000 7f549be00640 1 -- 172.21.15.39:0/490456986 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557473c19e60 con 0x5574738526b0 2024-09-21T07:39:38.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.587+0000 7f54a1e00640 1 --2- 172.21.15.39:0/490456986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55747384a4d0 0x557473c13830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:39:38.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.587+0000 7f549be00640 1 --2- 172.21.15.39:0/490456986 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5574738526b0 0x557473c13d70 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f548c060680 tx=0x7f548c087040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e62d8f21d98977c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:38.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.588+0000 7f5492000640 1 -- 172.21.15.39:0/490456986 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548c063410 con 0x5574738526b0 2024-09-21T07:39:38.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.588+0000 7f5492000640 1 -- 172.21.15.39:0/490456986 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f548c06ce30 con 0x5574738526b0 2024-09-21T07:39:38.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.588+0000 7f5492000640 1 -- 172.21.15.39:0/490456986 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548c01fdd0 con 0x5574738526b0 2024-09-21T07:39:38.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.588+0000 7f54afe56740 1 -- 172.21.15.39:0/490456986 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5574738526b0 msgr2=0x557473c13d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:38.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.588+0000 7f54afe56740 1 --2- 172.21.15.39:0/490456986 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5574738526b0 0x557473c13d70 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f548c060680 tx=0x7f548c087040 comp rx=0 tx=0).stop 2024-09-21T07:39:38.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.589+0000 7f54afe56740 1 -- 172.21.15.39:0/490456986 shutdown_connections 2024-09-21T07:39:38.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.589+0000 7f54afe56740 1 --2- 172.21.15.39:0/490456986 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557473873390 0x557473c59ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:38.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.589+0000 7f54afe56740 1 --2- 172.21.15.39:0/490456986 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5574738526b0 0x557473c13d70 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:38.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.589+0000 7f54afe56740 1 --2- 172.21.15.39:0/490456986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55747384a4d0 0x557473c13830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:38.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.589+0000 7f54afe56740 1 -- 172.21.15.39:0/490456986 >> 172.21.15.39:0/490456986 conn(0x5574738674e0 msgr2=0x557473c5efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:38.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.589+0000 7f54afe56740 1 -- 172.21.15.39:0/490456986 shutdown_connections 2024-09-21T07:39:38.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.589+0000 7f54afe56740 1 -- 172.21.15.39:0/490456986 wait complete. 2024-09-21T07:39:38.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.590+0000 7f54afe56740 1 -- start start 2024-09-21T07:39:38.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.590+0000 7f54afe56740 1 Processor -- start 2024-09-21T07:39:38.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.591+0000 7f54afe56740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55747384a4d0 0x557473ce6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:38.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.591+0000 7f54afe56740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5574738526b0 0x557473cdcd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:38.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.591+0000 7f54afe56740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557473873390 0x557473c576a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:38.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.591+0000 7f54afe56740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557473c1a8a0 con 0x557473873390 2024-09-21T07:39:38.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.591+0000 7f54afe56740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f54a4201460 con 0x5574738526b0 2024-09-21T07:39:38.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.591+0000 7f54afe56740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x557473c172c0 con 0x55747384a4d0 2024-09-21T07:39:38.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.591+0000 7f54afe56740 10 client.0 _ll_register_callbacks cb 0x7f54880036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:39:38.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.591+0000 7f54a2800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557473873390 0x557473c576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:38.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.592+0000 7f54a2800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557473873390 0x557473c576a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41602/0 (socket says 172.21.15.39:41602) 2024-09-21T07:39:38.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.592+0000 7f54a2800640 1 -- 172.21.15.39:0/2800566791 learned_addr learned my addr 172.21.15.39:0/2800566791 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:39:38.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.592+0000 7f54a1e00640 1 --2- 172.21.15.39:0/2800566791 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55747384a4d0 0x557473ce6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:38.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.592+0000 7f549be00640 1 --2- 172.21.15.39:0/2800566791 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5574738526b0 0x557473cdcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:38.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.592+0000 7f54a2800640 1 -- 172.21.15.39:0/2800566791 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55747384a4d0 msgr2=0x557473ce6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:39:38.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.592+0000 7f54a2800640 1 --2- 172.21.15.39:0/2800566791 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55747384a4d0 0x557473ce6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:38.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.592+0000 7f54a2800640 1 -- 172.21.15.39:0/2800566791 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5574738526b0 msgr2=0x557473cdcd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:39:38.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.592+0000 7f54a2800640 1 --2- 172.21.15.39:0/2800566791 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5574738526b0 0x557473cdcd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:38.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.592+0000 7f54a2800640 1 -- 172.21.15.39:0/2800566791 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557473c57450 con 0x557473873390 2024-09-21T07:39:38.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.592+0000 7f549be00640 1 --2- 172.21.15.39:0/2800566791 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5574738526b0 0x557473cdcd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:39:38.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.592+0000 7f54a2800640 1 --2- 172.21.15.39:0/2800566791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557473873390 0x557473c576a0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f54a40cd6f0 tx=0x7f549c012a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=381613d38287b277 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:38.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.592+0000 7f5487400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:39:38.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.593+0000 7f5487400640 1 -- 172.21.15.39:0/2800566791 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c01b070 con 0x557473873390 2024-09-21T07:39:38.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.593+0000 7f5487400640 1 -- 172.21.15.39:0/2800566791 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 7432+0+0 (secure 0 0 0) 0x7f549c016850 con 0x557473873390 2024-09-21T07:39:38.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.593+0000 7f54afe56740 1 -- 172.21.15.39:0/2800566791 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f548c06ce30 con 0x557473873390 2024-09-21T07:39:38.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.593+0000 7f5487400640 1 -- 172.21.15.39:0/2800566791 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7f549c016a50 con 0x557473873390 2024-09-21T07:39:38.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.593+0000 7f54afe56740 10 client.7206 fetch_fsmap learned FSMap version 102 2024-09-21T07:39:38.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.593+0000 7f54afe56740 1 -- 172.21.15.39:0/2800566791 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=2+}) -- 0x557473c15a00 con 0x557473873390 2024-09-21T07:39:38.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.593+0000 7f5487400640 1 -- 172.21.15.39:0/2800566791 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f548c06ce30 con 0x557473873390 2024-09-21T07:39:38.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.593+0000 7f5487400640 1 -- 172.21.15.39:0/2800566791 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 102) ==== 3522+0+0 (secure 0 0 0) 0x7f549c016ce0 con 0x557473873390 2024-09-21T07:39:38.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.593+0000 7f54afe56740 10 client.7206 fetch_fsmap finished waiting for FSMap version 102 2024-09-21T07:39:38.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.593+0000 7f54afe56740 10 client.7206 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-09-21T07:39:38.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.593+0000 7f54afe56740 20 client.7206 populate_metadata read hostname 'smithi039' 2024-09-21T07:39:38.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.593+0000 7f54afe56740 1 --2- 172.21.15.39:0/2800566791 >> [v2:172.21.15.148:6836/992188984,v1:172.21.15.148:6837/992188984] conn(0x557473c57be0 0x557473c59ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:38.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.593+0000 7f54afe56740 4 client.7206 mds_command: new command op to 6619 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:39:38.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.593+0000 7f54afe56740 1 -- 172.21.15.39:0/2800566791 --> [v2:172.21.15.148:6836/992188984,v1:172.21.15.148:6837/992188984] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557473b8b270 con 0x557473c57be0 2024-09-21T07:39:38.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.594+0000 7f54a1e00640 1 --2- 172.21.15.39:0/2800566791 >> [v2:172.21.15.148:6836/992188984,v1:172.21.15.148:6837/992188984] conn(0x557473c57be0 0x557473c59ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:38.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.594+0000 7f54a1e00640 1 --2- 172.21.15.39:0/2800566791 >> [v2:172.21.15.148:6836/992188984,v1:172.21.15.148:6837/992188984] conn(0x557473c57be0 0x557473c59ca0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6635066cadd5f2c server_cookie=27898fdbd26193cd in_seq=0 out_seq=0 2024-09-21T07:39:38.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.594+0000 7f5487400640 10 client.7206 ms_handle_connect on v2:172.21.15.148:6836/992188984 2024-09-21T07:39:38.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.613+0000 7f5487400640 1 -- 172.21.15.39:0/2800566791 <== mds.0 v2:172.21.15.148:6836/992188984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557473b8b270 con 0x557473c57be0 2024-09-21T07:39:38.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.613+0000 7f5487400640 10 client.7206 handle_command_reply: tid=0 2024-09-21T07:39:38.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.613+0000 7f54afe56740 1 client.7206 shutdown 2024-09-21T07:39:38.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.613+0000 7f54afe56740 1 -- 172.21.15.39:0/2800566791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557473873390 msgr2=0x557473c576a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:38.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.613+0000 7f54afe56740 1 --2- 172.21.15.39:0/2800566791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557473873390 0x557473c576a0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f54a40cd6f0 tx=0x7f549c012a60 comp rx=0 tx=0).stop 2024-09-21T07:39:38.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.614+0000 7f54afe56740 1 -- 172.21.15.39:0/2800566791 shutdown_connections 2024-09-21T07:39:38.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.614+0000 7f54afe56740 1 --2- 172.21.15.39:0/2800566791 >> [v2:172.21.15.148:6836/992188984,v1:172.21.15.148:6837/992188984] conn(0x557473c57be0 0x557473c59ca0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:38.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.614+0000 7f54afe56740 1 --2- 172.21.15.39:0/2800566791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557473873390 0x557473c576a0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:38.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.614+0000 7f54afe56740 1 --2- 172.21.15.39:0/2800566791 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5574738526b0 0x557473cdcd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:38.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.614+0000 7f54afe56740 1 --2- 172.21.15.39:0/2800566791 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55747384a4d0 0x557473ce6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:38.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.614+0000 7f5487400640 0 client.7206 ms_handle_reset on v2:172.21.15.148:6836/992188984 2024-09-21T07:39:38.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.614+0000 7f54afe56740 1 -- 172.21.15.39:0/2800566791 >> 172.21.15.39:0/2800566791 conn(0x5574738674e0 msgr2=0x557473c14570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:38.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.614+0000 7f54afe56740 1 -- 172.21.15.39:0/2800566791 shutdown_connections 2024-09-21T07:39:38.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.614+0000 7f54afe56740 1 -- 172.21.15.39:0/2800566791 wait complete. 2024-09-21T07:39:38.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.614+0000 7f54afe56740 20 client.7206 trim_cache size 0 max 16384 2024-09-21T07:39:38.617 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:39:38.617 INFO:teuthology.orchestra.run.smithi039.stdout: "message": "", 2024-09-21T07:39:38.617 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0 2024-09-21T07:39:38.618 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:39:38.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.615+0000 7f54a3200640 1 -- 172.21.15.39:0/3318235440 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5480078b90 msgr2=0x7f548007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:38.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.615+0000 7f54a3200640 1 --2- 172.21.15.39:0/3318235440 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5480078b90 0x7f548007b0a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5494034000 tx=0x7f54940046c0 comp rx=0 tx=0).stop 2024-09-21T07:39:38.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.615+0000 7f54a3200640 1 -- 172.21.15.39:0/3318235440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54a4156d60 msgr2=0x7f54a4070650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:38.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.615+0000 7f54a3200640 1 --2- 172.21.15.39:0/3318235440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54a4156d60 0x7f54a4070650 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f548c00e9f0 tx=0x7f548c00eeb0 comp rx=0 tx=0).stop 2024-09-21T07:39:38.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.615+0000 7f54a3200640 1 -- 172.21.15.39:0/3318235440 shutdown_connections 2024-09-21T07:39:38.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.615+0000 7f54a3200640 1 --2- 172.21.15.39:0/3318235440 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5480078b90 0x7f548007b0a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:38.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.615+0000 7f54a3200640 1 --2- 172.21.15.39:0/3318235440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54a4160770 0x7f54a41f6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:38.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.615+0000 7f54a3200640 1 --2- 172.21.15.39:0/3318235440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54a4156d60 0x7f54a4070650 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:38.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.615+0000 7f54a3200640 1 --2- 172.21.15.39:0/3318235440 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f54a40af690 0x7f54a4070110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:38.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.615+0000 7f54a3200640 1 -- 172.21.15.39:0/3318235440 >> 172.21.15.39:0/3318235440 conn(0x7f54a40b25f0 msgr2=0x7f54a40d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:38.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.615+0000 7f54a3200640 1 -- 172.21.15.39:0/3318235440 shutdown_connections 2024-09-21T07:39:38.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.615+0000 7f54a3200640 1 -- 172.21.15.39:0/3318235440 wait complete. 2024-09-21T07:39:38.640 INFO:tasks.cephfs.test_snapshots:testing snapserver mds_kill_mdstable_at=5, also kill snapclient 2024-09-21T07:39:38.640 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:39:38.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.866+0000 7fe1fea00640 1 -- 172.21.15.39:0/535702977 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1f80cbd70 msgr2=0x7fe1f8072fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:38.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.866+0000 7fe1fea00640 1 --2- 172.21.15.39:0/535702977 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1f80cbd70 0x7fe1f8072fa0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0005fd0 tx=0x7fe1f002aa00 comp rx=0 tx=0).stop 2024-09-21T07:39:38.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.867+0000 7fe1fea00640 1 -- 172.21.15.39:0/535702977 shutdown_connections 2024-09-21T07:39:38.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.867+0000 7fe1fea00640 1 --2- 172.21.15.39:0/535702977 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1f80cbd70 0x7fe1f8072fa0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:38.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.867+0000 7fe1fea00640 1 --2- 172.21.15.39:0/535702977 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1f80d0f20 0x7fe1f815c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:38.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.867+0000 7fe1fea00640 1 -- 172.21.15.39:0/535702977 >> 172.21.15.39:0/535702977 conn(0x7fe1f80b2510 msgr2=0x7fe1f80d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:38.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.867+0000 7fe1fea00640 1 -- 172.21.15.39:0/535702977 shutdown_connections 2024-09-21T07:39:38.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.867+0000 7fe1fea00640 1 -- 172.21.15.39:0/535702977 wait complete. 2024-09-21T07:39:38.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.869+0000 7fe1fea00640 1 Processor -- start 2024-09-21T07:39:38.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.869+0000 7fe1fea00640 1 -- start start 2024-09-21T07:39:38.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.871+0000 7fe1fea00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1f80cb960 0x7fe1f81f5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:38.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.871+0000 7fe1fea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1f80cbd70 0x7fe1f80c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:38.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.871+0000 7fe1fea00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1f80d0f20 0x7fe1f80c9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:38.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.871+0000 7fe1fea00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f80b4570 con 0x7fe1f80cbd70 2024-09-21T07:39:38.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.871+0000 7fe1fea00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f80b43f0 con 0x7fe1f80d0f20 2024-09-21T07:39:38.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.871+0000 7fe1fea00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe1f80b46f0 con 0x7fe1f80cb960 2024-09-21T07:39:38.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.871+0000 7fe1fcc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1f80cbd70 0x7fe1f80c91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:38.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.871+0000 7fe1fd600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1f80cb960 0x7fe1f81f5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:38.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.871+0000 7fe1fcc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1f80cbd70 0x7fe1f80c91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41616/0 (socket says 172.21.15.39:41616) 2024-09-21T07:39:38.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.871+0000 7fe1fcc00640 1 -- 172.21.15.39:0/7068536 learned_addr learned my addr 172.21.15.39:0/7068536 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:39:38.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.871+0000 7fe1fe000640 1 --2- 172.21.15.39:0/7068536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1f80d0f20 0x7fe1f80c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:38.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.871+0000 7fe1fcc00640 1 -- 172.21.15.39:0/7068536 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1f80cb960 msgr2=0x7fe1f81f5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:38.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.871+0000 7fe1fcc00640 1 --2- 172.21.15.39:0/7068536 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1f80cb960 0x7fe1f81f5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:38.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.871+0000 7fe1fcc00640 1 -- 172.21.15.39:0/7068536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1f80d0f20 msgr2=0x7fe1f80c9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:38.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.871+0000 7fe1fcc00640 1 --2- 172.21.15.39:0/7068536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1f80d0f20 0x7fe1f80c9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:38.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.871+0000 7fe1fcc00640 1 -- 172.21.15.39:0/7068536 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f81f9730 con 0x7fe1f80cbd70 2024-09-21T07:39:38.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.872+0000 7fe1fcc00640 1 --2- 172.21.15.39:0/7068536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1f80cbd70 0x7fe1f80c91d0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fe1e400b3f0 tx=0x7fe1e400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:38.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.872+0000 7fe1f5600640 1 -- 172.21.15.39:0/7068536 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4015020 con 0x7fe1f80cbd70 2024-09-21T07:39:38.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.872+0000 7fe1f5600640 1 -- 172.21.15.39:0/7068536 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1e4011920 con 0x7fe1f80cbd70 2024-09-21T07:39:38.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.872+0000 7fe1fea00640 1 -- 172.21.15.39:0/7068536 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1f81f9a10 con 0x7fe1f80cbd70 2024-09-21T07:39:38.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.872+0000 7fe1f5600640 1 -- 172.21.15.39:0/7068536 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4011d70 con 0x7fe1f80cbd70 2024-09-21T07:39:38.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.872+0000 7fe1fea00640 1 -- 172.21.15.39:0/7068536 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1f81f9fd0 con 0x7fe1f80cbd70 2024-09-21T07:39:38.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.873+0000 7fe1fea00640 1 -- 172.21.15.39:0/7068536 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1f8074b50 con 0x7fe1f80cbd70 2024-09-21T07:39:38.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.874+0000 7fe1f5600640 1 -- 172.21.15.39:0/7068536 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe1e4018020 con 0x7fe1f80cbd70 2024-09-21T07:39:38.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.875+0000 7fe1f5600640 1 --2- 172.21.15.39:0/7068536 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe1d8078b90 0x7fe1d807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:38.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.875+0000 7fe1f5600640 1 -- 172.21.15.39:0/7068536 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 7432+0+0 (secure 0 0 0) 0x7fe1e409afa0 con 0x7fe1f80cbd70 2024-09-21T07:39:38.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.876+0000 7fe1fd600640 1 --2- 172.21.15.39:0/7068536 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe1d8078b90 0x7fe1d807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:38.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.877+0000 7fe1fd600640 1 --2- 172.21.15.39:0/7068536 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe1d8078b90 0x7fe1d807b0a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec00bb50 tx=0x7fe1ec00b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:38.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:38.880+0000 7fe1f5600640 1 -- 172.21.15.39:0/7068536 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe1f8074b50 con 0x7fe1f80cbd70 2024-09-21T07:39:39.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.114+0000 7fe1fea00640 1 -- 172.21.15.39:0/7068536 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1f80b4c80 con 0x7fe1f80cbd70 2024-09-21T07:39:39.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.115+0000 7fe1f5600640 1 -- 172.21.15.39:0/7068536 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+5252 (secure 0 0 0) 0x7fe1e401f030 con 0x7fe1f80cbd70 2024-09-21T07:39:39.117 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:39:39.117 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":102,"btime":"2024-09-21T07:39:25:746836+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7113,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2821446711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2821446711},{"type":"v1","addr":"172.21.15.39:6835","nonce":2821446711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:39:25.746826+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6619,"mds_1":7080},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6619":{"gid":6619,"name":"d","rank":0,"incarnation":95,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/992188984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":992188984},{"type":"v1","addr":"172.21.15.148:6837","nonce":992188984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7080":{"gid":7080,"name":"c","rank":1,"incarnation":98,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6833/1456343023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1456343023},{"type":"v1","addr":"172.21.15.39:6833","nonce":1456343023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6619,"qdb_cluster":[6619,7080]},"id":6}]} 2024-09-21T07:39:39.118 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 102 2024-09-21T07:39:39.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.118+0000 7fe1fea00640 1 -- 172.21.15.39:0/7068536 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe1d8078b90 msgr2=0x7fe1d807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:39.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.118+0000 7fe1fea00640 1 --2- 172.21.15.39:0/7068536 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe1d8078b90 0x7fe1d807b0a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec00bb50 tx=0x7fe1ec00b2e0 comp rx=0 tx=0).stop 2024-09-21T07:39:39.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.118+0000 7fe1fea00640 1 -- 172.21.15.39:0/7068536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1f80cbd70 msgr2=0x7fe1f80c91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:39.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.118+0000 7fe1fea00640 1 --2- 172.21.15.39:0/7068536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1f80cbd70 0x7fe1f80c91d0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fe1e400b3f0 tx=0x7fe1e400b8b0 comp rx=0 tx=0).stop 2024-09-21T07:39:39.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.118+0000 7fe1fea00640 1 -- 172.21.15.39:0/7068536 shutdown_connections 2024-09-21T07:39:39.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.118+0000 7fe1fea00640 1 --2- 172.21.15.39:0/7068536 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe1d8078b90 0x7fe1d807b0a0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:39.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.118+0000 7fe1fea00640 1 --2- 172.21.15.39:0/7068536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1f80d0f20 0x7fe1f80c9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:39.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.118+0000 7fe1fea00640 1 --2- 172.21.15.39:0/7068536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1f80cbd70 0x7fe1f80c91d0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:39.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.118+0000 7fe1fea00640 1 --2- 172.21.15.39:0/7068536 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1f80cb960 0x7fe1f81f5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:39.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.118+0000 7fe1fea00640 1 -- 172.21.15.39:0/7068536 >> 172.21.15.39:0/7068536 conn(0x7fe1f80b2510 msgr2=0x7fe1f80b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:39.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.118+0000 7fe1fea00640 1 -- 172.21.15.39:0/7068536 shutdown_connections 2024-09-21T07:39:39.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.118+0000 7fe1fea00640 1 -- 172.21.15.39:0/7068536 wait complete. 2024-09-21T07:39:39.142 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json dump snaps --server 2024-09-21T07:39:39.392 INFO:teuthology.orchestra.run.smithi148.stdout:{"last_snap":10,"last_created":5,"last_destroyed":9,"pending_noop":[],"snaps":[],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2024-09-21T07:39:39.393 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 5, "last_destroyed": 9, "last_snap": 10, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [] } 2024-09-21T07:39:39.393 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:0 true 2024-09-21T07:39:39.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.620+0000 7f1463000640 1 -- 172.21.15.39:0/3549842903 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f145c0cce70 msgr2=0x7f145c0cd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:39.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.620+0000 7f1463000640 1 --2- 172.21.15.39:0/3549842903 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f145c0cce70 0x7f145c0cd280 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f1454005fd0 tx=0x7f145402aa10 comp rx=0 tx=0).stop 2024-09-21T07:39:39.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.620+0000 7f1463000640 1 -- 172.21.15.39:0/3549842903 shutdown_connections 2024-09-21T07:39:39.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.620+0000 7f1463000640 1 --2- 172.21.15.39:0/3549842903 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f145c0cce70 0x7f145c0cd280 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:39.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.620+0000 7f1463000640 1 --2- 172.21.15.39:0/3549842903 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f145c15cf60 0x7f145c0b32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:39.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.620+0000 7f1463000640 1 -- 172.21.15.39:0/3549842903 >> 172.21.15.39:0/3549842903 conn(0x7f145c06d440 msgr2=0x7f145c0b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:39.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.620+0000 7f1463000640 1 -- 172.21.15.39:0/3549842903 shutdown_connections 2024-09-21T07:39:39.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.620+0000 7f1463000640 1 -- 172.21.15.39:0/3549842903 wait complete. 2024-09-21T07:39:39.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.622+0000 7f1463000640 1 Processor -- start 2024-09-21T07:39:39.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.623+0000 7f1463000640 1 -- start start 2024-09-21T07:39:39.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.624+0000 7f1463000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f145c0cce70 0x7f145c0c9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:39.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.624+0000 7f1463000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f145c15cf60 0x7f145c0ceba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:39.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.624+0000 7f1463000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f145c15d950 0x7f145c0ad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:39.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.624+0000 7f1463000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f145c06fa20 con 0x7f145c0cce70 2024-09-21T07:39:39.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.624+0000 7f1463000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f145c06f8a0 con 0x7f145c15d950 2024-09-21T07:39:39.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.624+0000 7f1463000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f145c06fba0 con 0x7f145c15cf60 2024-09-21T07:39:39.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.624+0000 7f145be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f145c0cce70 0x7f145c0c9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:39.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.624+0000 7f145be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f145c0cce70 0x7f145c0c9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41646/0 (socket says 172.21.15.39:41646) 2024-09-21T07:39:39.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.624+0000 7f145be00640 1 -- 172.21.15.39:0/4047131738 learned_addr learned my addr 172.21.15.39:0/4047131738 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:39:39.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.625+0000 7f1453400640 1 --2- 172.21.15.39:0/4047131738 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f145c15cf60 0x7f145c0ceba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:39.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.625+0000 7f1461000640 1 --2- 172.21.15.39:0/4047131738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f145c15d950 0x7f145c0ad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:39.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.625+0000 7f145be00640 1 -- 172.21.15.39:0/4047131738 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f145c15cf60 msgr2=0x7f145c0ceba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:39.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.625+0000 7f145be00640 1 --2- 172.21.15.39:0/4047131738 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f145c15cf60 0x7f145c0ceba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:39.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.625+0000 7f145be00640 1 -- 172.21.15.39:0/4047131738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f145c15d950 msgr2=0x7f145c0ad9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:39.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.625+0000 7f145be00640 1 --2- 172.21.15.39:0/4047131738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f145c15d950 0x7f145c0ad9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:39.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.625+0000 7f145be00640 1 -- 172.21.15.39:0/4047131738 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f145c151c10 con 0x7f145c0cce70 2024-09-21T07:39:39.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.625+0000 7f1453400640 1 --2- 172.21.15.39:0/4047131738 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f145c15cf60 0x7f145c0ceba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:39:39.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.625+0000 7f1461000640 1 --2- 172.21.15.39:0/4047131738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f145c15d950 0x7f145c0ad9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:39:39.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.625+0000 7f145be00640 1 --2- 172.21.15.39:0/4047131738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f145c0cce70 0x7f145c0c9970 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f144c00e7c0 tx=0x7f144c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:39.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.625+0000 7f1458c00640 1 -- 172.21.15.39:0/4047131738 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144c00ccc0 con 0x7f145c0cce70 2024-09-21T07:39:39.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.625+0000 7f1458c00640 1 -- 172.21.15.39:0/4047131738 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f144c010430 con 0x7f145c0cce70 2024-09-21T07:39:39.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.625+0000 7f1463000640 1 -- 172.21.15.39:0/4047131738 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f145c0c9eb0 con 0x7f145c0cce70 2024-09-21T07:39:39.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.625+0000 7f1458c00640 1 -- 172.21.15.39:0/4047131738 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144c010860 con 0x7f145c0cce70 2024-09-21T07:39:39.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.626+0000 7f1463000640 1 -- 172.21.15.39:0/4047131738 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f145c0cfa00 con 0x7f145c0cce70 2024-09-21T07:39:39.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.627+0000 7f1463000640 1 -- 172.21.15.39:0/4047131738 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f145c0700c0 con 0x7f145c0cce70 2024-09-21T07:39:39.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.628+0000 7f1458c00640 1 -- 172.21.15.39:0/4047131738 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f144c010a00 con 0x7f145c0cce70 2024-09-21T07:39:39.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.629+0000 7f1458c00640 1 --2- 172.21.15.39:0/4047131738 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1438078bc0 0x7f143807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:39.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.630+0000 7f1458c00640 1 -- 172.21.15.39:0/4047131738 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 7432+0+0 (secure 0 0 0) 0x7f144c09be70 con 0x7f145c0cce70 2024-09-21T07:39:39.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.630+0000 7f1453400640 1 --2- 172.21.15.39:0/4047131738 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1438078bc0 0x7f143807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:39.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.631+0000 7f1453400640 1 --2- 172.21.15.39:0/4047131738 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1438078bc0 0x7f143807b0d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f145c0b2690 tx=0x7f1444009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:39.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.634+0000 7f1458c00640 1 -- 172.21.15.39:0/4047131738 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f144c063be0 con 0x7f145c0cce70 2024-09-21T07:39:39.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:39.912+0000 7f1463000640 1 -- 172.21.15.39:0/4047131738 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"} v 0) -- 0x7f145c0753a0 con 0x7f145c0cce70 2024-09-21T07:39:40.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:40.815+0000 7f1458c00640 1 -- 172.21.15.39:0/4047131738 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"}]=0 freezing mds.6619 v103) ==== 112+0+0 (secure 0 0 0) 0x7f144c0634e0 con 0x7f145c0cce70 2024-09-21T07:39:40.817 INFO:teuthology.orchestra.run.smithi039.stderr:freezing mds.6619 2024-09-21T07:39:40.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:40.817+0000 7f1463000640 1 -- 172.21.15.39:0/4047131738 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1438078bc0 msgr2=0x7f143807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:40.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:40.817+0000 7f1463000640 1 --2- 172.21.15.39:0/4047131738 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1438078bc0 0x7f143807b0d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f145c0b2690 tx=0x7f1444009040 comp rx=0 tx=0).stop 2024-09-21T07:39:40.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:40.817+0000 7f1463000640 1 -- 172.21.15.39:0/4047131738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f145c0cce70 msgr2=0x7f145c0c9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:40.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:40.817+0000 7f1463000640 1 --2- 172.21.15.39:0/4047131738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f145c0cce70 0x7f145c0c9970 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f144c00e7c0 tx=0x7f144c00ec80 comp rx=0 tx=0).stop 2024-09-21T07:39:40.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:40.817+0000 7f1463000640 1 -- 172.21.15.39:0/4047131738 shutdown_connections 2024-09-21T07:39:40.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:40.817+0000 7f1463000640 1 --2- 172.21.15.39:0/4047131738 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1438078bc0 0x7f143807b0d0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:40.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:40.817+0000 7f1463000640 1 --2- 172.21.15.39:0/4047131738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f145c15d950 0x7f145c0ad9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:40.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:40.817+0000 7f1463000640 1 --2- 172.21.15.39:0/4047131738 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f145c15cf60 0x7f145c0ceba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:40.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:40.817+0000 7f1463000640 1 --2- 172.21.15.39:0/4047131738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f145c0cce70 0x7f145c0c9970 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:40.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:40.817+0000 7f1463000640 1 -- 172.21.15.39:0/4047131738 >> 172.21.15.39:0/4047131738 conn(0x7f145c06d440 msgr2=0x7f145c0d03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:40.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:40.818+0000 7f1463000640 1 -- 172.21.15.39:0/4047131738 shutdown_connections 2024-09-21T07:39:40.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:40.818+0000 7f1463000640 1 -- 172.21.15.39:0/4047131738 wait complete. 2024-09-21T07:39:40.846 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:1 true 2024-09-21T07:39:41.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.073+0000 7efcc5a00640 1 -- 172.21.15.39:0/2553087547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efcc015d950 msgr2=0x7efcc00cbb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:41.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.073+0000 7efcc5a00640 1 --2- 172.21.15.39:0/2553087547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efcc015d950 0x7efcc00cbb10 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7efcc0071f60 tx=0x7efcac02a4d0 comp rx=0 tx=0).stop 2024-09-21T07:39:41.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.073+0000 7efcc5a00640 1 -- 172.21.15.39:0/2553087547 shutdown_connections 2024-09-21T07:39:41.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.073+0000 7efcc5a00640 1 --2- 172.21.15.39:0/2553087547 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efcc00cce70 0x7efcc00cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:41.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.073+0000 7efcc5a00640 1 --2- 172.21.15.39:0/2553087547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efcc015d950 0x7efcc00cbb10 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:41.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.073+0000 7efcc5a00640 1 --2- 172.21.15.39:0/2553087547 >> v2:172.21.15.148:3300/0 conn(0x7efcc015cf60 0x7efcc00b32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:41.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.073+0000 7efcc5a00640 1 -- 172.21.15.39:0/2553087547 >> 172.21.15.39:0/2553087547 conn(0x7efcc006d440 msgr2=0x7efcc00b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:41.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.073+0000 7efcc5a00640 1 -- 172.21.15.39:0/2553087547 shutdown_connections 2024-09-21T07:39:41.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.074+0000 7efcc5a00640 1 -- 172.21.15.39:0/2553087547 wait complete. 2024-09-21T07:39:41.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.076+0000 7efcc5a00640 1 Processor -- start 2024-09-21T07:39:41.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.076+0000 7efcc5a00640 1 -- start start 2024-09-21T07:39:41.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.077+0000 7efcc5a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efcc00cce70 0x7efcc015aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:41.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.077+0000 7efcc5a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efcc015cf60 0x7efcc01e9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:41.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.078+0000 7efcc5a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efcc015d950 0x7efcc01e9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:41.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.078+0000 7efcc5a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efcc006f7d0 con 0x7efcc015cf60 2024-09-21T07:39:41.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.078+0000 7efcc5a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efcc006f650 con 0x7efcc015d950 2024-09-21T07:39:41.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.078+0000 7efcc5a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7efcc006f950 con 0x7efcc00cce70 2024-09-21T07:39:41.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.078+0000 7efcbe000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efcc015cf60 0x7efcc01e9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:41.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.078+0000 7efcbe000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efcc015cf60 0x7efcc01e9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41654/0 (socket says 172.21.15.39:41654) 2024-09-21T07:39:41.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.078+0000 7efcbe000640 1 -- 172.21.15.39:0/3169252450 learned_addr learned my addr 172.21.15.39:0/3169252450 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:39:41.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.078+0000 7efcbea00640 1 --2- 172.21.15.39:0/3169252450 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efcc00cce70 0x7efcc015aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:41.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.078+0000 7efcbf400640 1 --2- 172.21.15.39:0/3169252450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efcc015d950 0x7efcc01e9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:41.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.078+0000 7efcbe000640 1 -- 172.21.15.39:0/3169252450 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efcc00cce70 msgr2=0x7efcc015aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:41.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.078+0000 7efcbe000640 1 --2- 172.21.15.39:0/3169252450 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efcc00cce70 0x7efcc015aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:41.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.078+0000 7efcbe000640 1 -- 172.21.15.39:0/3169252450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efcc015d950 msgr2=0x7efcc01e9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:41.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.078+0000 7efcbe000640 1 --2- 172.21.15.39:0/3169252450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efcc015d950 0x7efcc01e9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:41.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.078+0000 7efcbe000640 1 -- 172.21.15.39:0/3169252450 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc015b290 con 0x7efcc015cf60 2024-09-21T07:39:41.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.079+0000 7efcbe000640 1 --2- 172.21.15.39:0/3169252450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efcc015cf60 0x7efcc01e9020 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7efcac034040 tx=0x7efcac0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:41.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.079+0000 7efcb6a00640 1 -- 172.21.15.39:0/3169252450 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcac03e030 con 0x7efcc015cf60 2024-09-21T07:39:41.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.079+0000 7efcb6a00640 1 -- 172.21.15.39:0/3169252450 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efcac0047a0 con 0x7efcc015cf60 2024-09-21T07:39:41.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.079+0000 7efcc5a00640 1 -- 172.21.15.39:0/3169252450 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcc01e4320 con 0x7efcc015cf60 2024-09-21T07:39:41.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.079+0000 7efcb6a00640 1 -- 172.21.15.39:0/3169252450 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcac035040 con 0x7efcc015cf60 2024-09-21T07:39:41.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.079+0000 7efcc5a00640 1 -- 172.21.15.39:0/3169252450 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcc01e48e0 con 0x7efcc015cf60 2024-09-21T07:39:41.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.081+0000 7efcc5a00640 1 -- 172.21.15.39:0/3169252450 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcc00b32b0 con 0x7efcc015cf60 2024-09-21T07:39:41.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.082+0000 7efcb6a00640 1 -- 172.21.15.39:0/3169252450 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7efcac041030 con 0x7efcc015cf60 2024-09-21T07:39:41.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.083+0000 7efcb6a00640 1 --2- 172.21.15.39:0/3169252450 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efca0078b90 0x7efca007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:41.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.083+0000 7efcb6a00640 1 -- 172.21.15.39:0/3169252450 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 7432+0+0 (secure 0 0 0) 0x7efcac0bbb40 con 0x7efcc015cf60 2024-09-21T07:39:41.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.083+0000 7efcbea00640 1 --2- 172.21.15.39:0/3169252450 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efca0078b90 0x7efca007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:41.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.084+0000 7efcbea00640 1 --2- 172.21.15.39:0/3169252450 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efca0078b90 0x7efca007b0a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7efcb0005fd0 tx=0x7efcb000a250 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:41.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.087+0000 7efcb6a00640 1 -- 172.21.15.39:0/3169252450 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7efcac083270 con 0x7efcc015cf60 2024-09-21T07:39:41.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.362+0000 7efcc5a00640 1 -- 172.21.15.39:0/3169252450 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:1", "val": "true"} v 0) -- 0x7efcc01e4060 con 0x7efcc015cf60 2024-09-21T07:39:41.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.830+0000 7efcb6a00640 1 -- 172.21.15.39:0/3169252450 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:1", "val": "true"}]=0 freezing mds.7080 v104) ==== 112+0+0 (secure 0 0 0) 0x7efcac087ca0 con 0x7efcc015cf60 2024-09-21T07:39:41.832 INFO:teuthology.orchestra.run.smithi039.stderr:freezing mds.7080 2024-09-21T07:39:41.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.833+0000 7efcc5a00640 1 -- 172.21.15.39:0/3169252450 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efca0078b90 msgr2=0x7efca007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:41.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.833+0000 7efcc5a00640 1 --2- 172.21.15.39:0/3169252450 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efca0078b90 0x7efca007b0a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7efcb0005fd0 tx=0x7efcb000a250 comp rx=0 tx=0).stop 2024-09-21T07:39:41.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.833+0000 7efcc5a00640 1 -- 172.21.15.39:0/3169252450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efcc015cf60 msgr2=0x7efcc01e9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:41.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.833+0000 7efcc5a00640 1 --2- 172.21.15.39:0/3169252450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efcc015cf60 0x7efcc01e9020 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7efcac034040 tx=0x7efcac0039e0 comp rx=0 tx=0).stop 2024-09-21T07:39:41.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.833+0000 7efcc5a00640 1 -- 172.21.15.39:0/3169252450 shutdown_connections 2024-09-21T07:39:41.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.833+0000 7efcc5a00640 1 --2- 172.21.15.39:0/3169252450 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efca0078b90 0x7efca007b0a0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:41.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.833+0000 7efcc5a00640 1 --2- 172.21.15.39:0/3169252450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efcc015d950 0x7efcc01e9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:41.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.833+0000 7efcc5a00640 1 --2- 172.21.15.39:0/3169252450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efcc015cf60 0x7efcc01e9020 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:41.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.833+0000 7efcc5a00640 1 --2- 172.21.15.39:0/3169252450 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efcc00cce70 0x7efcc015aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:41.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.833+0000 7efcc5a00640 1 -- 172.21.15.39:0/3169252450 >> 172.21.15.39:0/3169252450 conn(0x7efcc006d440 msgr2=0x7efcc00ad220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:41.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.833+0000 7efcc5a00640 1 -- 172.21.15.39:0/3169252450 shutdown_connections 2024-09-21T07:39:41.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:41.833+0000 7efcc5a00640 1 -- 172.21.15.39:0/3169252450 wait complete. 2024-09-21T07:39:41.862 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json config set mds_kill_mdstable_at 5 2024-09-21T07:39:42.108 INFO:teuthology.orchestra.run.smithi148.stdout:{"success":"mds_kill_mdstable_at = '' (not observed, change may require restart) "} 2024-09-21T07:39:42.109 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_kill_mdstable_at = '' (not observed, change may require restart) " } 2024-09-21T07:39:42.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:39:42.109 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d1/dir/.snap/s25) 2024-09-21T07:39:42.113 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-09-21T07:39:42.156 INFO:tasks.ceph.mds.d.smithi148.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_commit(ceph::cref_t&)' thread 7f769c800640 time 2024-09-21T07:39:42.156289+0000 2024-09-21T07:39:42.156 INFO:tasks.ceph.mds.d.smithi148.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: 147: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 5) 2024-09-21T07:39:42.157 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:39:42.157 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f76a385654b] 2024-09-21T07:39:42.157 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f76a3856758] 2024-09-21T07:39:42.157 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x3bd) [0x5642c922730b] 2024-09-21T07:39:42.158 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: (MDSTableServer::handle_request(boost::intrusive_ptr const&)+0x9a) [0x5642c9227910] 2024-09-21T07:39:42.158 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xca5) [0x5642c8ec0de3] 2024-09-21T07:39:42.158 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5642c8ec4523] 2024-09-21T07:39:42.158 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5642c8ec4e4a] 2024-09-21T07:39:42.158 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x5642c8ea7fa4] 2024-09-21T07:39:42.158 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f76a39a57df] 2024-09-21T07:39:42.158 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: (DispatchQueue::entry()+0x60f) [0x7f76a39a60d1] 2024-09-21T07:39:42.158 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f76a3a93f5b] 2024-09-21T07:39:42.158 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: (Thread::entry_wrapper()+0x43) [0x7f76a382fd57] 2024-09-21T07:39:42.158 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: (Thread::_entry_func(void*)+0xd) [0x7f76a382fd73] 2024-09-21T07:39:42.158 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: /lib64/libc.so.6(+0x89c52) [0x7f76a2e89c52] 2024-09-21T07:39:42.158 INFO:tasks.ceph.mds.d.smithi148.stderr: 15: /lib64/libc.so.6(+0x10ec80) [0x7f76a2f0ec80] 2024-09-21T07:39:42.158 INFO:tasks.ceph.mds.d.smithi148.stderr:*** Caught signal (Aborted) ** 2024-09-21T07:39:42.159 INFO:tasks.ceph.mds.d.smithi148.stderr: in thread 7f769c800640 thread_name:ms_dispatch 2024-09-21T07:39:42.159 INFO:tasks.ceph.mds.d.smithi148.stderr:2024-09-21T07:39:42.155+0000 7f769c800640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_commit(ceph::cref_t&)' thread 7f769c800640 time 2024-09-21T07:39:42.156289+0000 2024-09-21T07:39:42.159 INFO:tasks.ceph.mds.d.smithi148.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: 147: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 5) 2024-09-21T07:39:42.159 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:39:42.159 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:39:42.159 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f76a385654b] 2024-09-21T07:39:42.159 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f76a3856758] 2024-09-21T07:39:42.159 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x3bd) [0x5642c922730b] 2024-09-21T07:39:42.159 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: (MDSTableServer::handle_request(boost::intrusive_ptr const&)+0x9a) [0x5642c9227910] 2024-09-21T07:39:42.159 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xca5) [0x5642c8ec0de3] 2024-09-21T07:39:42.159 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5642c8ec4523] 2024-09-21T07:39:42.159 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5642c8ec4e4a] 2024-09-21T07:39:42.159 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x5642c8ea7fa4] 2024-09-21T07:39:42.159 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f76a39a57df] 2024-09-21T07:39:42.159 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: (DispatchQueue::entry()+0x60f) [0x7f76a39a60d1] 2024-09-21T07:39:42.160 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f76a3a93f5b] 2024-09-21T07:39:42.160 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: (Thread::entry_wrapper()+0x43) [0x7f76a382fd57] 2024-09-21T07:39:42.160 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: (Thread::_entry_func(void*)+0xd) [0x7f76a382fd73] 2024-09-21T07:39:42.160 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: /lib64/libc.so.6(+0x89c52) [0x7f76a2e89c52] 2024-09-21T07:39:42.160 INFO:tasks.ceph.mds.d.smithi148.stderr: 15: /lib64/libc.so.6(+0x10ec80) [0x7f76a2f0ec80] 2024-09-21T07:39:42.160 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:39:42.161 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:39:42.161 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f76a2e3e6f0] 2024-09-21T07:39:42.161 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f76a2e8b99c] 2024-09-21T07:39:42.161 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: raise() 2024-09-21T07:39:42.161 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: abort() 2024-09-21T07:39:42.161 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7f76a3856686] 2024-09-21T07:39:42.161 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f76a3856758] 2024-09-21T07:39:42.161 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x3bd) [0x5642c922730b] 2024-09-21T07:39:42.162 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: (MDSTableServer::handle_request(boost::intrusive_ptr const&)+0x9a) [0x5642c9227910] 2024-09-21T07:39:42.162 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xca5) [0x5642c8ec0de3] 2024-09-21T07:39:42.162 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5642c8ec4523] 2024-09-21T07:39:42.162 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5642c8ec4e4a] 2024-09-21T07:39:42.162 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x5642c8ea7fa4] 2024-09-21T07:39:42.162 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f76a39a57df] 2024-09-21T07:39:42.162 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: (DispatchQueue::entry()+0x60f) [0x7f76a39a60d1] 2024-09-21T07:39:42.162 INFO:tasks.ceph.mds.d.smithi148.stderr: 15: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f76a3a93f5b] 2024-09-21T07:39:42.162 INFO:tasks.ceph.mds.d.smithi148.stderr: 16: (Thread::entry_wrapper()+0x43) [0x7f76a382fd57] 2024-09-21T07:39:42.162 INFO:tasks.ceph.mds.d.smithi148.stderr: 17: (Thread::_entry_func(void*)+0xd) [0x7f76a382fd73] 2024-09-21T07:39:42.162 INFO:tasks.ceph.mds.d.smithi148.stderr: 18: /lib64/libc.so.6(+0x89c52) [0x7f76a2e89c52] 2024-09-21T07:39:42.162 INFO:tasks.ceph.mds.d.smithi148.stderr: 19: /lib64/libc.so.6(+0x10ec80) [0x7f76a2f0ec80] 2024-09-21T07:39:42.162 INFO:tasks.ceph.mds.d.smithi148.stderr:2024-09-21T07:39:42.156+0000 7f769c800640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:39:42.162 INFO:tasks.ceph.mds.d.smithi148.stderr: in thread 7f769c800640 thread_name:ms_dispatch 2024-09-21T07:39:42.163 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:39:42.163 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:39:42.163 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f76a2e3e6f0] 2024-09-21T07:39:42.163 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f76a2e8b99c] 2024-09-21T07:39:42.163 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: raise() 2024-09-21T07:39:42.163 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: abort() 2024-09-21T07:39:42.163 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7f76a3856686] 2024-09-21T07:39:42.163 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f76a3856758] 2024-09-21T07:39:42.163 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x3bd) [0x5642c922730b] 2024-09-21T07:39:42.163 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: (MDSTableServer::handle_request(boost::intrusive_ptr const&)+0x9a) [0x5642c9227910] 2024-09-21T07:39:42.163 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xca5) [0x5642c8ec0de3] 2024-09-21T07:39:42.163 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5642c8ec4523] 2024-09-21T07:39:42.163 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5642c8ec4e4a] 2024-09-21T07:39:42.164 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x5642c8ea7fa4] 2024-09-21T07:39:42.164 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f76a39a57df] 2024-09-21T07:39:42.164 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: (DispatchQueue::entry()+0x60f) [0x7f76a39a60d1] 2024-09-21T07:39:42.164 INFO:tasks.ceph.mds.d.smithi148.stderr: 15: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f76a3a93f5b] 2024-09-21T07:39:42.164 INFO:tasks.ceph.mds.d.smithi148.stderr: 16: (Thread::entry_wrapper()+0x43) [0x7f76a382fd57] 2024-09-21T07:39:42.164 INFO:tasks.ceph.mds.d.smithi148.stderr: 17: (Thread::_entry_func(void*)+0xd) [0x7f76a382fd73] 2024-09-21T07:39:42.164 INFO:tasks.ceph.mds.d.smithi148.stderr: 18: /lib64/libc.so.6(+0x89c52) [0x7f76a2e89c52] 2024-09-21T07:39:42.164 INFO:tasks.ceph.mds.d.smithi148.stderr: 19: /lib64/libc.so.6(+0x10ec80) [0x7f76a2f0ec80] 2024-09-21T07:39:42.164 INFO:tasks.ceph.mds.d.smithi148.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:39:42.164 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:39:42.166 INFO:tasks.ceph.mds.d.smithi148.stderr: -3308> 2024-09-21T07:39:08.755+0000 7f76a0400640 -1 --2- [v2:172.21.15.148:6836/992188984,v1:172.21.15.148:6837/992188984] >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x5642e3ba3680 0x5642e3d4a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] is using msgr V1 protocol 2024-09-21T07:39:42.166 INFO:tasks.ceph.mds.d.smithi148.stderr: -3303> 2024-09-21T07:39:08.956+0000 7f76a0400640 -1 --2- [v2:172.21.15.148:6836/992188984,v1:172.21.15.148:6837/992188984] >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x5642e3ba3680 0x5642e3d4a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] is using msgr V1 protocol 2024-09-21T07:39:42.166 INFO:tasks.ceph.mds.d.smithi148.stderr: -3301> 2024-09-21T07:39:09.358+0000 7f76a0400640 -1 --2- [v2:172.21.15.148:6836/992188984,v1:172.21.15.148:6837/992188984] >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x5642e3ba3680 0x5642e3d4a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] is using msgr V1 protocol 2024-09-21T07:39:42.166 INFO:tasks.ceph.mds.d.smithi148.stderr: -3290> 2024-09-21T07:39:10.160+0000 7f76a0400640 -1 --2- [v2:172.21.15.148:6836/992188984,v1:172.21.15.148:6837/992188984] >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x5642e3ba3680 0x5642e3d4a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] is using msgr V1 protocol 2024-09-21T07:39:42.166 INFO:tasks.ceph.mds.d.smithi148.stderr: -3274> 2024-09-21T07:39:11.763+0000 7f76a0400640 -1 --2- [v2:172.21.15.148:6836/992188984,v1:172.21.15.148:6837/992188984] >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x5642e3ba3680 0x5642e3d4a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] is using msgr V1 protocol 2024-09-21T07:39:42.166 INFO:tasks.ceph.mds.d.smithi148.stderr: -3242> 2024-09-21T07:39:14.965+0000 7f76a0400640 -1 --2- [v2:172.21.15.148:6836/992188984,v1:172.21.15.148:6837/992188984] >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x5642e3ba3680 0x5642e3d4a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] is using msgr V1 protocol 2024-09-21T07:39:42.169 INFO:tasks.ceph.mds.d.smithi148.stderr: -1511> 2024-09-21T07:39:28.232+0000 7f7699600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:39:42.169 INFO:tasks.ceph.mds.d.smithi148.stderr: -1509> 2024-09-21T07:39:28.232+0000 7f7699600640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:39:42.173 INFO:tasks.ceph.mds.d.smithi148.stderr: -1> 2024-09-21T07:39:42.155+0000 7f769c800640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_commit(ceph::cref_t&)' thread 7f769c800640 time 2024-09-21T07:39:42.156289+0000 2024-09-21T07:39:42.173 INFO:tasks.ceph.mds.d.smithi148.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: 147: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 5) 2024-09-21T07:39:42.173 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:39:42.173 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:39:42.173 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f76a385654b] 2024-09-21T07:39:42.173 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f76a3856758] 2024-09-21T07:39:42.173 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x3bd) [0x5642c922730b] 2024-09-21T07:39:42.173 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: (MDSTableServer::handle_request(boost::intrusive_ptr const&)+0x9a) [0x5642c9227910] 2024-09-21T07:39:42.173 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xca5) [0x5642c8ec0de3] 2024-09-21T07:39:42.173 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5642c8ec4523] 2024-09-21T07:39:42.173 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5642c8ec4e4a] 2024-09-21T07:39:42.173 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x5642c8ea7fa4] 2024-09-21T07:39:42.173 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f76a39a57df] 2024-09-21T07:39:42.174 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: (DispatchQueue::entry()+0x60f) [0x7f76a39a60d1] 2024-09-21T07:39:42.174 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f76a3a93f5b] 2024-09-21T07:39:42.174 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: (Thread::entry_wrapper()+0x43) [0x7f76a382fd57] 2024-09-21T07:39:42.174 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: (Thread::_entry_func(void*)+0xd) [0x7f76a382fd73] 2024-09-21T07:39:42.174 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: /lib64/libc.so.6(+0x89c52) [0x7f76a2e89c52] 2024-09-21T07:39:42.174 INFO:tasks.ceph.mds.d.smithi148.stderr: 15: /lib64/libc.so.6(+0x10ec80) [0x7f76a2f0ec80] 2024-09-21T07:39:42.174 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:39:42.174 INFO:tasks.ceph.mds.d.smithi148.stderr: 0> 2024-09-21T07:39:42.156+0000 7f769c800640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:39:42.174 INFO:tasks.ceph.mds.d.smithi148.stderr: in thread 7f769c800640 thread_name:ms_dispatch 2024-09-21T07:39:42.174 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:39:42.174 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:39:42.174 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f76a2e3e6f0] 2024-09-21T07:39:42.174 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f76a2e8b99c] 2024-09-21T07:39:42.174 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: raise() 2024-09-21T07:39:42.174 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: abort() 2024-09-21T07:39:42.175 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7f76a3856686] 2024-09-21T07:39:42.175 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f76a3856758] 2024-09-21T07:39:42.175 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x3bd) [0x5642c922730b] 2024-09-21T07:39:42.175 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: (MDSTableServer::handle_request(boost::intrusive_ptr const&)+0x9a) [0x5642c9227910] 2024-09-21T07:39:42.175 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xca5) [0x5642c8ec0de3] 2024-09-21T07:39:42.175 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5642c8ec4523] 2024-09-21T07:39:42.175 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5642c8ec4e4a] 2024-09-21T07:39:42.175 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x5642c8ea7fa4] 2024-09-21T07:39:42.175 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f76a39a57df] 2024-09-21T07:39:42.175 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: (DispatchQueue::entry()+0x60f) [0x7f76a39a60d1] 2024-09-21T07:39:42.175 INFO:tasks.ceph.mds.d.smithi148.stderr: 15: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f76a3a93f5b] 2024-09-21T07:39:42.175 INFO:tasks.ceph.mds.d.smithi148.stderr: 16: (Thread::entry_wrapper()+0x43) [0x7f76a382fd57] 2024-09-21T07:39:42.175 INFO:tasks.ceph.mds.d.smithi148.stderr: 17: (Thread::_entry_func(void*)+0xd) [0x7f76a382fd73] 2024-09-21T07:39:42.175 INFO:tasks.ceph.mds.d.smithi148.stderr: 18: /lib64/libc.so.6(+0x89c52) [0x7f76a2e89c52] 2024-09-21T07:39:42.176 INFO:tasks.ceph.mds.d.smithi148.stderr: 19: /lib64/libc.so.6(+0x10ec80) [0x7f76a2f0ec80] 2024-09-21T07:39:42.176 INFO:tasks.ceph.mds.d.smithi148.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:39:42.176 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:39:42.176 INFO:tasks.ceph.mds.d.smithi148.stderr: -3308> 2024-09-21T07:39:08.755+0000 7f76a0400640 -1 --2- [v2:172.21.15.148:6836/992188984,v1:172.21.15.148:6837/992188984] >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x5642e3ba3680 0x5642e3d4a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] is using msgr V1 protocol 2024-09-21T07:39:42.177 INFO:tasks.ceph.mds.d.smithi148.stderr: -3303> 2024-09-21T07:39:08.956+0000 7f76a0400640 -1 --2- [v2:172.21.15.148:6836/992188984,v1:172.21.15.148:6837/992188984] >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x5642e3ba3680 0x5642e3d4a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] is using msgr V1 protocol 2024-09-21T07:39:42.177 INFO:tasks.ceph.mds.d.smithi148.stderr: -3301> 2024-09-21T07:39:09.358+0000 7f76a0400640 -1 --2- [v2:172.21.15.148:6836/992188984,v1:172.21.15.148:6837/992188984] >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x5642e3ba3680 0x5642e3d4a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] is using msgr V1 protocol 2024-09-21T07:39:42.177 INFO:tasks.ceph.mds.d.smithi148.stderr: -3290> 2024-09-21T07:39:10.160+0000 7f76a0400640 -1 --2- [v2:172.21.15.148:6836/992188984,v1:172.21.15.148:6837/992188984] >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x5642e3ba3680 0x5642e3d4a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] is using msgr V1 protocol 2024-09-21T07:39:42.177 INFO:tasks.ceph.mds.d.smithi148.stderr: -3274> 2024-09-21T07:39:11.763+0000 7f76a0400640 -1 --2- [v2:172.21.15.148:6836/992188984,v1:172.21.15.148:6837/992188984] >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x5642e3ba3680 0x5642e3d4a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] is using msgr V1 protocol 2024-09-21T07:39:42.177 INFO:tasks.ceph.mds.d.smithi148.stderr: -3242> 2024-09-21T07:39:14.965+0000 7f76a0400640 -1 --2- [v2:172.21.15.148:6836/992188984,v1:172.21.15.148:6837/992188984] >> [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] conn(0x5642e3ba3680 0x5642e3d4a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.39:6833/364657398,v1:172.21.15.39:6835/364657398] is using msgr V1 protocol 2024-09-21T07:39:42.178 INFO:tasks.ceph.mds.d.smithi148.stderr: -1511> 2024-09-21T07:39:28.232+0000 7f7699600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:39:42.178 INFO:tasks.ceph.mds.d.smithi148.stderr: -1509> 2024-09-21T07:39:28.232+0000 7f7699600640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:39:42.179 INFO:tasks.ceph.mds.d.smithi148.stderr: -1> 2024-09-21T07:39:42.155+0000 7f769c800640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_commit(ceph::cref_t&)' thread 7f769c800640 time 2024-09-21T07:39:42.156289+0000 2024-09-21T07:39:42.179 INFO:tasks.ceph.mds.d.smithi148.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: 147: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 5) 2024-09-21T07:39:42.179 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:39:42.180 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:39:42.180 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f76a385654b] 2024-09-21T07:39:42.180 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f76a3856758] 2024-09-21T07:39:42.180 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x3bd) [0x5642c922730b] 2024-09-21T07:39:42.180 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: (MDSTableServer::handle_request(boost::intrusive_ptr const&)+0x9a) [0x5642c9227910] 2024-09-21T07:39:42.180 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xca5) [0x5642c8ec0de3] 2024-09-21T07:39:42.180 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5642c8ec4523] 2024-09-21T07:39:42.180 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5642c8ec4e4a] 2024-09-21T07:39:42.180 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x5642c8ea7fa4] 2024-09-21T07:39:42.180 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f76a39a57df] 2024-09-21T07:39:42.180 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: (DispatchQueue::entry()+0x60f) [0x7f76a39a60d1] 2024-09-21T07:39:42.180 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f76a3a93f5b] 2024-09-21T07:39:42.180 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: (Thread::entry_wrapper()+0x43) [0x7f76a382fd57] 2024-09-21T07:39:42.180 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: (Thread::_entry_func(void*)+0xd) [0x7f76a382fd73] 2024-09-21T07:39:42.180 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: /lib64/libc.so.6(+0x89c52) [0x7f76a2e89c52] 2024-09-21T07:39:42.181 INFO:tasks.ceph.mds.d.smithi148.stderr: 15: /lib64/libc.so.6(+0x10ec80) [0x7f76a2f0ec80] 2024-09-21T07:39:42.181 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:39:42.181 INFO:tasks.ceph.mds.d.smithi148.stderr: 0> 2024-09-21T07:39:42.156+0000 7f769c800640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:39:42.181 INFO:tasks.ceph.mds.d.smithi148.stderr: in thread 7f769c800640 thread_name:ms_dispatch 2024-09-21T07:39:42.181 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:39:42.181 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:39:42.181 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f76a2e3e6f0] 2024-09-21T07:39:42.181 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f76a2e8b99c] 2024-09-21T07:39:42.181 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: raise() 2024-09-21T07:39:42.181 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: abort() 2024-09-21T07:39:42.181 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7f76a3856686] 2024-09-21T07:39:42.181 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f76a3856758] 2024-09-21T07:39:42.181 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x3bd) [0x5642c922730b] 2024-09-21T07:39:42.181 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: (MDSTableServer::handle_request(boost::intrusive_ptr const&)+0x9a) [0x5642c9227910] 2024-09-21T07:39:42.181 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xca5) [0x5642c8ec0de3] 2024-09-21T07:39:42.181 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5642c8ec4523] 2024-09-21T07:39:42.182 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5642c8ec4e4a] 2024-09-21T07:39:42.182 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x5642c8ea7fa4] 2024-09-21T07:39:42.182 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f76a39a57df] 2024-09-21T07:39:42.182 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: (DispatchQueue::entry()+0x60f) [0x7f76a39a60d1] 2024-09-21T07:39:42.182 INFO:tasks.ceph.mds.d.smithi148.stderr: 15: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f76a3a93f5b] 2024-09-21T07:39:42.182 INFO:tasks.ceph.mds.d.smithi148.stderr: 16: (Thread::entry_wrapper()+0x43) [0x7f76a382fd57] 2024-09-21T07:39:42.182 INFO:tasks.ceph.mds.d.smithi148.stderr: 17: (Thread::_entry_func(void*)+0xd) [0x7f76a382fd73] 2024-09-21T07:39:42.182 INFO:tasks.ceph.mds.d.smithi148.stderr: 18: /lib64/libc.so.6(+0x89c52) [0x7f76a2e89c52] 2024-09-21T07:39:42.182 INFO:tasks.ceph.mds.d.smithi148.stderr: 19: /lib64/libc.so.6(+0x10ec80) [0x7f76a2f0ec80] 2024-09-21T07:39:42.182 INFO:tasks.ceph.mds.d.smithi148.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:39:42.182 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:39:42.390 INFO:teuthology.orchestra.run.smithi039.stdout:{"mds_beacon_grace":"15.000000"} 2024-09-21T07:39:42.391 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-09-21T07:39:42.391 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:39:42.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.617+0000 7feb26e00640 1 -- 172.21.15.39:0/3411831255 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb200b2ca0 msgr2=0x7feb200b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:42.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.617+0000 7feb26e00640 1 --2- 172.21.15.39:0/3411831255 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb200b2ca0 0x7feb200b4dc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7feb18005fd0 tx=0x7feb1802a8b0 comp rx=0 tx=0).stop 2024-09-21T07:39:42.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.617+0000 7feb26e00640 1 -- 172.21.15.39:0/3411831255 shutdown_connections 2024-09-21T07:39:42.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.617+0000 7feb26e00640 1 --2- 172.21.15.39:0/3411831255 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb200b2ca0 0x7feb200b4dc0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:42.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.617+0000 7feb26e00640 1 --2- 172.21.15.39:0/3411831255 >> v2:172.21.15.148:3300/0 conn(0x7feb200d4d10 0x7feb200b4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:42.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.617+0000 7feb26e00640 1 -- 172.21.15.39:0/3411831255 >> 172.21.15.39:0/3411831255 conn(0x7feb2008bfb0 msgr2=0x7feb2009d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:42.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.617+0000 7feb26e00640 1 -- 172.21.15.39:0/3411831255 shutdown_connections 2024-09-21T07:39:42.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.617+0000 7feb26e00640 1 -- 172.21.15.39:0/3411831255 wait complete. 2024-09-21T07:39:42.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.619+0000 7feb26e00640 1 Processor -- start 2024-09-21T07:39:42.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.619+0000 7feb26e00640 1 -- start start 2024-09-21T07:39:42.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.621+0000 7feb26e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb200b2ca0 0x7feb200ce580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:42.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.621+0000 7feb26e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb200d19f0 0x7feb20150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:42.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.621+0000 7feb26e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb20151ca0 0x7feb201514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:42.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.621+0000 7feb26e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7feb200b5480 con 0x7feb20151ca0 2024-09-21T07:39:42.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.621+0000 7feb26e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7feb200b5300 con 0x7feb200b2ca0 2024-09-21T07:39:42.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.621+0000 7feb26e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7feb200b5600 con 0x7feb200d19f0 2024-09-21T07:39:42.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.621+0000 7feb26400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb20151ca0 0x7feb201514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:42.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.621+0000 7feb26400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb20151ca0 0x7feb201514f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48510/0 (socket says 172.21.15.39:48510) 2024-09-21T07:39:42.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.621+0000 7feb26400640 1 -- 172.21.15.39:0/2953390750 learned_addr learned my addr 172.21.15.39:0/2953390750 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:39:42.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.621+0000 7feb25000640 1 --2- 172.21.15.39:0/2953390750 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb200d19f0 0x7feb20150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:42.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.621+0000 7feb25a00640 1 --2- 172.21.15.39:0/2953390750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb200b2ca0 0x7feb200ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:42.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.622+0000 7feb26400640 1 -- 172.21.15.39:0/2953390750 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb200d19f0 msgr2=0x7feb20150fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:42.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.622+0000 7feb26400640 1 --2- 172.21.15.39:0/2953390750 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb200d19f0 0x7feb20150fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:42.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.622+0000 7feb26400640 1 -- 172.21.15.39:0/2953390750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb200b2ca0 msgr2=0x7feb200ce580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:42.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.622+0000 7feb26400640 1 --2- 172.21.15.39:0/2953390750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb200b2ca0 0x7feb200ce580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:42.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.622+0000 7feb26400640 1 -- 172.21.15.39:0/2953390750 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb200a1d10 con 0x7feb20151ca0 2024-09-21T07:39:42.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.622+0000 7feb26400640 1 --2- 172.21.15.39:0/2953390750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb20151ca0 0x7feb201514f0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7feb18009cd0 tx=0x7feb180316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:42.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.622+0000 7feb1d600640 1 -- 172.21.15.39:0/2953390750 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb18031da0 con 0x7feb20151ca0 2024-09-21T07:39:42.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.622+0000 7feb1d600640 1 -- 172.21.15.39:0/2953390750 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb18003a60 con 0x7feb20151ca0 2024-09-21T07:39:42.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.622+0000 7feb26e00640 1 -- 172.21.15.39:0/2953390750 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb2014ef20 con 0x7feb20151ca0 2024-09-21T07:39:42.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.622+0000 7feb1d600640 1 -- 172.21.15.39:0/2953390750 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb180330d0 con 0x7feb20151ca0 2024-09-21T07:39:42.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.623+0000 7feb26e00640 1 -- 172.21.15.39:0/2953390750 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb2014f4e0 con 0x7feb20151ca0 2024-09-21T07:39:42.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.624+0000 7feb26e00640 1 -- 172.21.15.39:0/2953390750 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb20069a50 con 0x7feb20151ca0 2024-09-21T07:39:42.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.625+0000 7feb1d600640 1 -- 172.21.15.39:0/2953390750 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7feb18033270 con 0x7feb20151ca0 2024-09-21T07:39:42.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.626+0000 7feb1d600640 1 --2- 172.21.15.39:0/2953390750 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feb00078bc0 0x7feb0007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:42.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.626+0000 7feb1d600640 1 -- 172.21.15.39:0/2953390750 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 7432+0+0 (secure 0 0 0) 0x7feb180bc750 con 0x7feb20151ca0 2024-09-21T07:39:42.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.626+0000 7feb25a00640 1 --2- 172.21.15.39:0/2953390750 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feb00078bc0 0x7feb0007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:42.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.627+0000 7feb25a00640 1 --2- 172.21.15.39:0/2953390750 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feb00078bc0 0x7feb0007b0d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7feb14001890 tx=0x7feb140013a0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:42.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.630+0000 7feb1d600640 1 -- 172.21.15.39:0/2953390750 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7feb1803f030 con 0x7feb20151ca0 2024-09-21T07:39:42.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.865+0000 7feb26e00640 1 -- 172.21.15.39:0/2953390750 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb200a1aa0 con 0x7feb20151ca0 2024-09-21T07:39:42.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.866+0000 7feb1d600640 1 -- 172.21.15.39:0/2953390750 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) ==== 95+0+5252 (secure 0 0 0) 0x7feb18083e80 con 0x7feb20151ca0 2024-09-21T07:39:42.868 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:39:42.868 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":104,"btime":"2024-09-21T07:39:41:816459+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7113,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2821446711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2821446711},{"type":"v1","addr":"172.21.15.39:6835","nonce":2821446711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:39:41.364182+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6619,"mds_1":7080},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6619":{"gid":6619,"name":"d","rank":0,"incarnation":95,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/992188984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":992188984},{"type":"v1","addr":"172.21.15.148:6837","nonce":992188984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7080":{"gid":7080,"name":"c","rank":1,"incarnation":98,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6833/1456343023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1456343023},{"type":"v1","addr":"172.21.15.39:6833","nonce":1456343023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6619,"qdb_cluster":[6619,7080]},"id":6}]} 2024-09-21T07:39:42.868 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 104 2024-09-21T07:39:42.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.868+0000 7feb26e00640 1 -- 172.21.15.39:0/2953390750 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feb00078bc0 msgr2=0x7feb0007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:42.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.868+0000 7feb26e00640 1 --2- 172.21.15.39:0/2953390750 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feb00078bc0 0x7feb0007b0d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7feb14001890 tx=0x7feb140013a0 comp rx=0 tx=0).stop 2024-09-21T07:39:42.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.868+0000 7feb26e00640 1 -- 172.21.15.39:0/2953390750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb20151ca0 msgr2=0x7feb201514f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:42.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.868+0000 7feb26e00640 1 --2- 172.21.15.39:0/2953390750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb20151ca0 0x7feb201514f0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7feb18009cd0 tx=0x7feb180316f0 comp rx=0 tx=0).stop 2024-09-21T07:39:42.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.868+0000 7feb26e00640 1 -- 172.21.15.39:0/2953390750 shutdown_connections 2024-09-21T07:39:42.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.868+0000 7feb26e00640 1 --2- 172.21.15.39:0/2953390750 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feb00078bc0 0x7feb0007b0d0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:42.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.868+0000 7feb26e00640 1 --2- 172.21.15.39:0/2953390750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb20151ca0 0x7feb201514f0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:42.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.868+0000 7feb26e00640 1 --2- 172.21.15.39:0/2953390750 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb200d19f0 0x7feb20150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:42.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.868+0000 7feb26e00640 1 --2- 172.21.15.39:0/2953390750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb200b2ca0 0x7feb200ce580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:42.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.868+0000 7feb26e00640 1 -- 172.21.15.39:0/2953390750 >> 172.21.15.39:0/2953390750 conn(0x7feb2008bfb0 msgr2=0x7feb200cfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:42.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.868+0000 7feb26e00640 1 -- 172.21.15.39:0/2953390750 shutdown_connections 2024-09-21T07:39:42.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:42.868+0000 7feb26e00640 1 -- 172.21.15.39:0/2953390750 wait complete. 2024-09-21T07:39:42.896 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:39:43.047 INFO:tasks.ceph.mds.d.smithi148.stderr:daemon-helper: command crashed with signal 6 2024-09-21T07:39:47.897 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:39:48.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.134+0000 7f496e000640 1 -- 172.21.15.39:0/659862589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49680d4d10 msgr2=0x7f49680b20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:48.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.134+0000 7f496e000640 1 --2- 172.21.15.39:0/659862589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49680d4d10 0x7f49680b20a0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f4960005fd0 tx=0x7f496002aa10 comp rx=0 tx=0).stop 2024-09-21T07:39:48.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.134+0000 7f496e000640 1 -- 172.21.15.39:0/659862589 shutdown_connections 2024-09-21T07:39:48.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.134+0000 7f496e000640 1 --2- 172.21.15.39:0/659862589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49680d4d10 0x7f49680b20a0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:48.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.134+0000 7f496e000640 1 --2- 172.21.15.39:0/659862589 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49680b3e20 0x7f49680d1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:48.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.134+0000 7f496e000640 1 -- 172.21.15.39:0/659862589 >> 172.21.15.39:0/659862589 conn(0x7f496808bfb0 msgr2=0x7f496809d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:48.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.134+0000 7f496e000640 1 -- 172.21.15.39:0/659862589 shutdown_connections 2024-09-21T07:39:48.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.134+0000 7f496e000640 1 -- 172.21.15.39:0/659862589 wait complete. 2024-09-21T07:39:48.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.136+0000 7f496e000640 1 Processor -- start 2024-09-21T07:39:48.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.136+0000 7f496e000640 1 -- start start 2024-09-21T07:39:48.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.138+0000 7f496e000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49680b3e20 0x7f4968157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:48.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.138+0000 7f496e000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49680b5970 0x7f49680cd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:48.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.138+0000 7f496e000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4968158190 0x7f49680cd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:48.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.138+0000 7f496e000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f49680b27f0 con 0x7f49680b5970 2024-09-21T07:39:48.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.138+0000 7f496e000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f49680b5ef0 con 0x7f4968158190 2024-09-21T07:39:48.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.138+0000 7f496e000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f49680b2970 con 0x7f49680b3e20 2024-09-21T07:39:48.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.138+0000 7f4967e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49680b5970 0x7f49680cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:48.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.138+0000 7f4967e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49680b5970 0x7f49680cd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48528/0 (socket says 172.21.15.39:48528) 2024-09-21T07:39:48.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.138+0000 7f4967e00640 1 -- 172.21.15.39:0/736424973 learned_addr learned my addr 172.21.15.39:0/736424973 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:39:48.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.138+0000 7f496cc00640 1 --2- 172.21.15.39:0/736424973 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49680b3e20 0x7f4968157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:48.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.138+0000 7f496d600640 1 --2- 172.21.15.39:0/736424973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4968158190 0x7f49680cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:48.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.138+0000 7f4967e00640 1 -- 172.21.15.39:0/736424973 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49680b3e20 msgr2=0x7f4968157990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:48.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.138+0000 7f4967e00640 1 --2- 172.21.15.39:0/736424973 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49680b3e20 0x7f4968157990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:48.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.138+0000 7f4967e00640 1 -- 172.21.15.39:0/736424973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4968158190 msgr2=0x7f49680cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:48.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.138+0000 7f4967e00640 1 --2- 172.21.15.39:0/736424973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4968158190 0x7f49680cd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:48.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.138+0000 7f4967e00640 1 -- 172.21.15.39:0/736424973 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4968157ed0 con 0x7f49680b5970 2024-09-21T07:39:48.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.139+0000 7f4967e00640 1 --2- 172.21.15.39:0/736424973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49680b5970 0x7f49680cd340 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f495400c3d0 tx=0x7f495400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:48.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.139+0000 7f4964c00640 1 -- 172.21.15.39:0/736424973 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4954017070 con 0x7f49680b5970 2024-09-21T07:39:48.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.139+0000 7f4964c00640 1 -- 172.21.15.39:0/736424973 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49540053c0 con 0x7f49680b5970 2024-09-21T07:39:48.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.139+0000 7f4964c00640 1 -- 172.21.15.39:0/736424973 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4954003850 con 0x7f49680b5970 2024-09-21T07:39:48.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.139+0000 7f496e000640 1 -- 172.21.15.39:0/736424973 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49680ce6d0 con 0x7f49680b5970 2024-09-21T07:39:48.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.139+0000 7f496e000640 1 -- 172.21.15.39:0/736424973 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49680ccb40 con 0x7f49680b5970 2024-09-21T07:39:48.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.140+0000 7f496e000640 1 -- 172.21.15.39:0/736424973 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49680ce440 con 0x7f49680b5970 2024-09-21T07:39:48.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.141+0000 7f4964c00640 1 -- 172.21.15.39:0/736424973 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f495401a070 con 0x7f49680b5970 2024-09-21T07:39:48.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.142+0000 7f4964c00640 1 --2- 172.21.15.39:0/736424973 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4948078b90 0x7f494807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:48.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.142+0000 7f4964c00640 1 -- 172.21.15.39:0/736424973 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 7432+0+0 (secure 0 0 0) 0x7f495409a860 con 0x7f49680b5970 2024-09-21T07:39:48.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.143+0000 7f496cc00640 1 --2- 172.21.15.39:0/736424973 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4948078b90 0x7f494807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:48.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.144+0000 7f496cc00640 1 --2- 172.21.15.39:0/736424973 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4948078b90 0x7f494807b0a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f495c005d70 tx=0x7f495c005d00 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:48.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.147+0000 7f4964c00640 1 -- 172.21.15.39:0/736424973 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4954062540 con 0x7f49680b5970 2024-09-21T07:39:48.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.382+0000 7f496e000640 1 -- 172.21.15.39:0/736424973 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4968069a50 con 0x7f49680b5970 2024-09-21T07:39:48.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.382+0000 7f4964c00640 1 -- 172.21.15.39:0/736424973 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) ==== 95+0+5252 (secure 0 0 0) 0x7f4954061e40 con 0x7f49680b5970 2024-09-21T07:39:48.384 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:39:48.384 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":104,"btime":"2024-09-21T07:39:41:816459+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7113,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2821446711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2821446711},{"type":"v1","addr":"172.21.15.39:6835","nonce":2821446711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:39:41.364182+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6619,"mds_1":7080},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6619":{"gid":6619,"name":"d","rank":0,"incarnation":95,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/992188984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":992188984},{"type":"v1","addr":"172.21.15.148:6837","nonce":992188984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7080":{"gid":7080,"name":"c","rank":1,"incarnation":98,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6833/1456343023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1456343023},{"type":"v1","addr":"172.21.15.39:6833","nonce":1456343023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6619,"qdb_cluster":[6619,7080]},"id":6}]} 2024-09-21T07:39:48.385 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 104 2024-09-21T07:39:48.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.385+0000 7f496e000640 1 -- 172.21.15.39:0/736424973 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4948078b90 msgr2=0x7f494807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:48.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.385+0000 7f496e000640 1 --2- 172.21.15.39:0/736424973 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4948078b90 0x7f494807b0a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f495c005d70 tx=0x7f495c005d00 comp rx=0 tx=0).stop 2024-09-21T07:39:48.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.385+0000 7f496e000640 1 -- 172.21.15.39:0/736424973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49680b5970 msgr2=0x7f49680cd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:48.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.385+0000 7f496e000640 1 --2- 172.21.15.39:0/736424973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49680b5970 0x7f49680cd340 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f495400c3d0 tx=0x7f495400c890 comp rx=0 tx=0).stop 2024-09-21T07:39:48.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.385+0000 7f496e000640 1 -- 172.21.15.39:0/736424973 shutdown_connections 2024-09-21T07:39:48.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.385+0000 7f496e000640 1 --2- 172.21.15.39:0/736424973 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4948078b90 0x7f494807b0a0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:48.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.385+0000 7f496e000640 1 --2- 172.21.15.39:0/736424973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4968158190 0x7f49680cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:48.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.385+0000 7f496e000640 1 --2- 172.21.15.39:0/736424973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49680b5970 0x7f49680cd340 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:48.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.385+0000 7f496e000640 1 --2- 172.21.15.39:0/736424973 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49680b3e20 0x7f4968157990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:48.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.385+0000 7f496e000640 1 -- 172.21.15.39:0/736424973 >> 172.21.15.39:0/736424973 conn(0x7f496808bfb0 msgr2=0x7f49680b60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:48.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.385+0000 7f496e000640 1 -- 172.21.15.39:0/736424973 shutdown_connections 2024-09-21T07:39:48.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:48.385+0000 7f496e000640 1 -- 172.21.15.39:0/736424973 wait complete. 2024-09-21T07:39:48.411 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:39:49.439 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-09-21T07:39:53.412 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:39:53.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.648+0000 7f6345c00640 1 -- 172.21.15.39:0/225677022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f634015c830 msgr2=0x7f634006f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:53.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.648+0000 7f6345c00640 1 --2- 172.21.15.39:0/225677022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f634015c830 0x7f634006f8a0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f6328005fd0 tx=0x7f632802a5d0 comp rx=0 tx=0).stop 2024-09-21T07:39:53.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.649+0000 7f6345c00640 1 -- 172.21.15.39:0/225677022 shutdown_connections 2024-09-21T07:39:53.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.649+0000 7f6345c00640 1 --2- 172.21.15.39:0/225677022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f634015c830 0x7f634006f8a0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:53.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.649+0000 7f6345c00640 1 -- 172.21.15.39:0/225677022 >> 172.21.15.39:0/225677022 conn(0x7f63400b2510 msgr2=0x7f63400d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:53.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.649+0000 7f6345c00640 1 -- 172.21.15.39:0/225677022 shutdown_connections 2024-09-21T07:39:53.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.649+0000 7f6345c00640 1 -- 172.21.15.39:0/225677022 wait complete. 2024-09-21T07:39:53.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.651+0000 7f6345c00640 1 Processor -- start 2024-09-21T07:39:53.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.651+0000 7f6345c00640 1 -- start start 2024-09-21T07:39:53.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.653+0000 7f6345c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f634006fde0 0x7f6340150370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:53.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.653+0000 7f6345c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f63400d0f20 0x7f634009b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:53.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.653+0000 7f6345c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f634015c830 0x7f63401f70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:53.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.653+0000 7f6345c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f63400b47c0 con 0x7f634006fde0 2024-09-21T07:39:53.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.653+0000 7f6345c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f63400b4640 con 0x7f634015c830 2024-09-21T07:39:53.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.653+0000 7f6345c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f63400b4940 con 0x7f63400d0f20 2024-09-21T07:39:53.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.653+0000 7f633fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f634006fde0 0x7f6340150370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:53.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.653+0000 7f633fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f634006fde0 0x7f6340150370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57062/0 (socket says 172.21.15.39:57062) 2024-09-21T07:39:53.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.653+0000 7f633fe00640 1 -- 172.21.15.39:0/2356769704 learned_addr learned my addr 172.21.15.39:0/2356769704 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:39:53.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.653+0000 7f6337400640 1 --2- 172.21.15.39:0/2356769704 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f63400d0f20 0x7f634009b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:53.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.653+0000 7f6345200640 1 --2- 172.21.15.39:0/2356769704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f634015c830 0x7f63401f70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:53.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.653+0000 7f633fe00640 1 -- 172.21.15.39:0/2356769704 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f63400d0f20 msgr2=0x7f634009b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:53.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.653+0000 7f633fe00640 1 --2- 172.21.15.39:0/2356769704 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f63400d0f20 0x7f634009b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:53.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.653+0000 7f633fe00640 1 -- 172.21.15.39:0/2356769704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f634015c830 msgr2=0x7f63401f70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:53.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.653+0000 7f633fe00640 1 --2- 172.21.15.39:0/2356769704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f634015c830 0x7f63401f70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:53.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.653+0000 7f633fe00640 1 -- 172.21.15.39:0/2356769704 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f634009a7d0 con 0x7f634006fde0 2024-09-21T07:39:53.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.654+0000 7f633fe00640 1 --2- 172.21.15.39:0/2356769704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f634006fde0 0x7f6340150370 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f633000b3f0 tx=0x7f633000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:53.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.654+0000 7f633cc00640 1 -- 172.21.15.39:0/2356769704 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6330015020 con 0x7f634006fde0 2024-09-21T07:39:53.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.654+0000 7f633cc00640 1 -- 172.21.15.39:0/2356769704 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6330011920 con 0x7f634006fde0 2024-09-21T07:39:53.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.654+0000 7f633cc00640 1 -- 172.21.15.39:0/2356769704 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6330011d70 con 0x7f634006fde0 2024-09-21T07:39:53.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.654+0000 7f6345c00640 1 -- 172.21.15.39:0/2356769704 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f634009aab0 con 0x7f634006fde0 2024-09-21T07:39:53.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.654+0000 7f6345c00640 1 -- 172.21.15.39:0/2356769704 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f634009b070 con 0x7f634006fde0 2024-09-21T07:39:53.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.655+0000 7f6345c00640 1 -- 172.21.15.39:0/2356769704 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f634006f8a0 con 0x7f634006fde0 2024-09-21T07:39:53.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.656+0000 7f633cc00640 1 -- 172.21.15.39:0/2356769704 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6330018020 con 0x7f634006fde0 2024-09-21T07:39:53.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.657+0000 7f633cc00640 1 --2- 172.21.15.39:0/2356769704 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f631c078b90 0x7f631c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:53.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.658+0000 7f633cc00640 1 -- 172.21.15.39:0/2356769704 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 7432+0+0 (secure 0 0 0) 0x7f633009b200 con 0x7f634006fde0 2024-09-21T07:39:53.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.658+0000 7f6337400640 1 --2- 172.21.15.39:0/2356769704 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f631c078b90 0x7f631c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:53.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.659+0000 7f6337400640 1 --2- 172.21.15.39:0/2356769704 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f631c078b90 0x7f631c07b0a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6328033000 tx=0x7f6328004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:53.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.663+0000 7f633cc00640 1 -- 172.21.15.39:0/2356769704 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f633001f030 con 0x7f634006fde0 2024-09-21T07:39:53.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.899+0000 7f6345c00640 1 -- 172.21.15.39:0/2356769704 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6340159b00 con 0x7f634006fde0 2024-09-21T07:39:53.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.900+0000 7f633cc00640 1 -- 172.21.15.39:0/2356769704 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) ==== 95+0+5252 (secure 0 0 0) 0x7f6330062ee0 con 0x7f634006fde0 2024-09-21T07:39:53.902 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:39:53.902 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":104,"btime":"2024-09-21T07:39:41:816459+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7113,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2821446711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2821446711},{"type":"v1","addr":"172.21.15.39:6835","nonce":2821446711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:39:41.364182+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6619,"mds_1":7080},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6619":{"gid":6619,"name":"d","rank":0,"incarnation":95,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/992188984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":992188984},{"type":"v1","addr":"172.21.15.148:6837","nonce":992188984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7080":{"gid":7080,"name":"c","rank":1,"incarnation":98,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6833/1456343023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1456343023},{"type":"v1","addr":"172.21.15.39:6833","nonce":1456343023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6619,"qdb_cluster":[6619,7080]},"id":6}]} 2024-09-21T07:39:53.902 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 104 2024-09-21T07:39:53.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.902+0000 7f6345c00640 1 -- 172.21.15.39:0/2356769704 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f631c078b90 msgr2=0x7f631c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:53.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.902+0000 7f6345c00640 1 --2- 172.21.15.39:0/2356769704 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f631c078b90 0x7f631c07b0a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6328033000 tx=0x7f6328004730 comp rx=0 tx=0).stop 2024-09-21T07:39:53.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.902+0000 7f6345c00640 1 -- 172.21.15.39:0/2356769704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f634006fde0 msgr2=0x7f6340150370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:53.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.902+0000 7f6345c00640 1 --2- 172.21.15.39:0/2356769704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f634006fde0 0x7f6340150370 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f633000b3f0 tx=0x7f633000b8b0 comp rx=0 tx=0).stop 2024-09-21T07:39:53.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.902+0000 7f6345c00640 1 -- 172.21.15.39:0/2356769704 shutdown_connections 2024-09-21T07:39:53.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.902+0000 7f6345c00640 1 --2- 172.21.15.39:0/2356769704 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f631c078b90 0x7f631c07b0a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:53.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.902+0000 7f6345c00640 1 --2- 172.21.15.39:0/2356769704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f634015c830 0x7f63401f70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:53.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.902+0000 7f6345c00640 1 --2- 172.21.15.39:0/2356769704 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f63400d0f20 0x7f634009b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:53.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.902+0000 7f6345c00640 1 --2- 172.21.15.39:0/2356769704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f634006fde0 0x7f6340150370 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:53.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.902+0000 7f6345c00640 1 -- 172.21.15.39:0/2356769704 >> 172.21.15.39:0/2356769704 conn(0x7f63400b2510 msgr2=0x7f634015e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:53.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.902+0000 7f6345c00640 1 -- 172.21.15.39:0/2356769704 shutdown_connections 2024-09-21T07:39:53.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:53.902+0000 7f6345c00640 1 -- 172.21.15.39:0/2356769704 wait complete. 2024-09-21T07:39:53.926 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:39:56.051 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~7s 2024-09-21T07:39:58.927 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:39:59.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.162+0000 7f9efaa00640 1 -- 172.21.15.39:0/3585807844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ef40b3370 msgr2=0x7f9ef415f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:59.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.162+0000 7f9efaa00640 1 --2- 172.21.15.39:0/3585807844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ef40b3370 0x7f9ef415f310 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4005fd0 tx=0x7f9ee402a540 comp rx=0 tx=0).stop 2024-09-21T07:39:59.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.162+0000 7f9efaa00640 1 -- 172.21.15.39:0/3585807844 shutdown_connections 2024-09-21T07:39:59.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.162+0000 7f9efaa00640 1 --2- 172.21.15.39:0/3585807844 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9ef4152780 0x7f9ef40b4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:59.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.162+0000 7f9efaa00640 1 --2- 172.21.15.39:0/3585807844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ef40b3370 0x7f9ef415f310 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:59.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.162+0000 7f9efaa00640 1 -- 172.21.15.39:0/3585807844 >> 172.21.15.39:0/3585807844 conn(0x7f9ef406cfc0 msgr2=0x7f9ef406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:59.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.162+0000 7f9efaa00640 1 -- 172.21.15.39:0/3585807844 shutdown_connections 2024-09-21T07:39:59.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.162+0000 7f9efaa00640 1 -- 172.21.15.39:0/3585807844 wait complete. 2024-09-21T07:39:59.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.165+0000 7f9efaa00640 1 Processor -- start 2024-09-21T07:39:59.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.165+0000 7f9efaa00640 1 -- start start 2024-09-21T07:39:59.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.166+0000 7f9efaa00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9ef40b3370 0x7f9ef41f99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:59.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.166+0000 7f9efaa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ef40b4bd0 0x7f9ef414baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:59.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.166+0000 7f9efaa00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ef4152780 0x7f9ef414dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:59.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.166+0000 7f9efaa00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef40d0880 con 0x7f9ef40b4bd0 2024-09-21T07:39:59.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.166+0000 7f9efaa00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef40d0700 con 0x7f9ef4152780 2024-09-21T07:39:59.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.166+0000 7f9efaa00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f9ef40d0a00 con 0x7f9ef40b3370 2024-09-21T07:39:59.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.166+0000 7f9eeb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ef40b4bd0 0x7f9ef414baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:59.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.166+0000 7f9eeb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ef40b4bd0 0x7f9ef414baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57076/0 (socket says 172.21.15.39:57076) 2024-09-21T07:39:59.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.166+0000 7f9eeb400640 1 -- 172.21.15.39:0/12584661 learned_addr learned my addr 172.21.15.39:0/12584661 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:39:59.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.167+0000 7f9ef3e00640 1 --2- 172.21.15.39:0/12584661 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9ef40b3370 0x7f9ef41f99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:59.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.167+0000 7f9ef8a00640 1 --2- 172.21.15.39:0/12584661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ef4152780 0x7f9ef414dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:59.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.167+0000 7f9eeb400640 1 -- 172.21.15.39:0/12584661 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9ef40b3370 msgr2=0x7f9ef41f99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:59.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.167+0000 7f9eeb400640 1 --2- 172.21.15.39:0/12584661 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9ef40b3370 0x7f9ef41f99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:59.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.167+0000 7f9eeb400640 1 -- 172.21.15.39:0/12584661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ef4152780 msgr2=0x7f9ef414dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:59.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.167+0000 7f9eeb400640 1 --2- 172.21.15.39:0/12584661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ef4152780 0x7f9ef414dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:59.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.167+0000 7f9eeb400640 1 -- 172.21.15.39:0/12584661 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef414fb70 con 0x7f9ef40b4bd0 2024-09-21T07:39:59.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.167+0000 7f9eeb400640 1 --2- 172.21.15.39:0/12584661 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ef40b4bd0 0x7f9ef414baf0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f9edc00e7c0 tx=0x7f9edc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:59.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.167+0000 7f9ef0c00640 1 -- 172.21.15.39:0/12584661 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc00cc60 con 0x7f9ef40b4bd0 2024-09-21T07:39:59.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.167+0000 7f9ef0c00640 1 -- 172.21.15.39:0/12584661 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9edc010430 con 0x7f9ef40b4bd0 2024-09-21T07:39:59.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.168+0000 7f9efaa00640 1 -- 172.21.15.39:0/12584661 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ef414d100 con 0x7f9ef40b4bd0 2024-09-21T07:39:59.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.168+0000 7f9ef0c00640 1 -- 172.21.15.39:0/12584661 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc010860 con 0x7f9ef40b4bd0 2024-09-21T07:39:59.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.168+0000 7f9efaa00640 1 -- 172.21.15.39:0/12584661 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ef414d6c0 con 0x7f9ef40b4bd0 2024-09-21T07:39:59.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.169+0000 7f9efaa00640 1 -- 172.21.15.39:0/12584661 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ef40d0eb0 con 0x7f9ef40b4bd0 2024-09-21T07:39:59.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.170+0000 7f9ef0c00640 1 -- 172.21.15.39:0/12584661 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9edc010a00 con 0x7f9ef40b4bd0 2024-09-21T07:39:59.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.171+0000 7f9ef0c00640 1 --2- 172.21.15.39:0/12584661 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9ed0078bc0 0x7f9ed007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:59.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.171+0000 7f9ef0c00640 1 -- 172.21.15.39:0/12584661 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 7432+0+0 (secure 0 0 0) 0x7f9edc09bdb0 con 0x7f9ef40b4bd0 2024-09-21T07:39:59.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.171+0000 7f9ef3e00640 1 --2- 172.21.15.39:0/12584661 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9ed0078bc0 0x7f9ed007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:59.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.173+0000 7f9ef3e00640 1 --2- 172.21.15.39:0/12584661 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9ed0078bc0 0x7f9ed007b0d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4033000 tx=0x7f9ee4004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:59.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.176+0000 7f9ef0c00640 1 -- 172.21.15.39:0/12584661 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9edc0a0070 con 0x7f9ef40b4bd0 2024-09-21T07:39:59.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.427+0000 7f9efaa00640 1 -- 172.21.15.39:0/12584661 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ef40708b0 con 0x7f9ef40b4bd0 2024-09-21T07:39:59.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.428+0000 7f9ef0c00640 1 -- 172.21.15.39:0/12584661 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+5300 (secure 0 0 0) 0x7f9edc063440 con 0x7f9ef40b4bd0 2024-09-21T07:39:59.430 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:39:59.430 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":105,"btime":"2024-09-21T07:39:56:405074+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7113,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2821446711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2821446711},{"type":"v1","addr":"172.21.15.39:6835","nonce":2821446711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:39:56.405046+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6619,"mds_1":7080},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6619":{"gid":6619,"name":"d","rank":0,"incarnation":95,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/992188984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":992188984},{"type":"v1","addr":"172.21.15.148:6837","nonce":992188984}]},"join_fscid":-1,"laggy_since":"2024-09-21T07:39:56.405046+0000","export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7080":{"gid":7080,"name":"c","rank":1,"incarnation":98,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6833/1456343023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1456343023},{"type":"v1","addr":"172.21.15.39:6833","nonce":1456343023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6619,"qdb_cluster":[7080,6619]},"id":6}]} 2024-09-21T07:39:59.430 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 105 2024-09-21T07:39:59.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.430+0000 7f9efaa00640 1 -- 172.21.15.39:0/12584661 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9ed0078bc0 msgr2=0x7f9ed007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:59.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.430+0000 7f9efaa00640 1 --2- 172.21.15.39:0/12584661 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9ed0078bc0 0x7f9ed007b0d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4033000 tx=0x7f9ee4004730 comp rx=0 tx=0).stop 2024-09-21T07:39:59.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.430+0000 7f9efaa00640 1 -- 172.21.15.39:0/12584661 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ef40b4bd0 msgr2=0x7f9ef414baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:59.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.430+0000 7f9efaa00640 1 --2- 172.21.15.39:0/12584661 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ef40b4bd0 0x7f9ef414baf0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f9edc00e7c0 tx=0x7f9edc00ec80 comp rx=0 tx=0).stop 2024-09-21T07:39:59.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.430+0000 7f9efaa00640 1 -- 172.21.15.39:0/12584661 shutdown_connections 2024-09-21T07:39:59.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.430+0000 7f9efaa00640 1 --2- 172.21.15.39:0/12584661 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9ed0078bc0 0x7f9ed007b0d0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:59.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.430+0000 7f9efaa00640 1 --2- 172.21.15.39:0/12584661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ef4152780 0x7f9ef414dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:59.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.430+0000 7f9efaa00640 1 --2- 172.21.15.39:0/12584661 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ef40b4bd0 0x7f9ef414baf0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:59.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.430+0000 7f9efaa00640 1 --2- 172.21.15.39:0/12584661 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9ef40b3370 0x7f9ef41f99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:59.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.430+0000 7f9efaa00640 1 -- 172.21.15.39:0/12584661 >> 172.21.15.39:0/12584661 conn(0x7f9ef406cfc0 msgr2=0x7f9ef40b52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:59.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.431+0000 7f9efaa00640 1 -- 172.21.15.39:0/12584661 shutdown_connections 2024-09-21T07:39:59.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.431+0000 7f9efaa00640 1 -- 172.21.15.39:0/12584661 wait complete. 2024-09-21T07:39:59.458 DEBUG:tasks.ceph_test_case:wait_until_true: success in 15s and 0 retries 2024-09-21T07:39:59.458 DEBUG:teuthology.orchestra.run.smithi148:> sudo sysctl -n kernel.core_pattern 2024-09-21T07:39:59.517 INFO:teuthology.orchestra.run.smithi148.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T07:39:59.520 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2024-09-21T07:39:59.520 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2024-09-21T07:39:59.585 INFO:teuthology.orchestra.run.smithi148.stdout:1726904382.43380.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i d', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2024-09-21T07:39:59.589 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1726904382.43380.core'] 2024-09-21T07:39:59.589 INFO:tasks.cephfs.cephfs_test_case:Found core file 1726904382.43380.core, deleting it 2024-09-21T07:39:59.589 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1726904382.43380.core 2024-09-21T07:39:59.681 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:39:59.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.913+0000 7f03d9c00640 1 -- 172.21.15.39:0/313501324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03d415fc30 msgr2=0x7f03d4160320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:59.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.913+0000 7f03d9c00640 1 --2- 172.21.15.39:0/313501324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03d415fc30 0x7f03d4160320 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f03d40cd340 tx=0x7f03c402a5d0 comp rx=0 tx=0).stop 2024-09-21T07:39:59.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.913+0000 7f03d9c00640 1 -- 172.21.15.39:0/313501324 shutdown_connections 2024-09-21T07:39:59.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.913+0000 7f03d9c00640 1 --2- 172.21.15.39:0/313501324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03d415fc30 0x7f03d4160320 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:59.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.913+0000 7f03d9c00640 1 -- 172.21.15.39:0/313501324 >> 172.21.15.39:0/313501324 conn(0x7f03d40b2510 msgr2=0x7f03d40d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:39:59.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.913+0000 7f03d9c00640 1 -- 172.21.15.39:0/313501324 shutdown_connections 2024-09-21T07:39:59.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.913+0000 7f03d9c00640 1 -- 172.21.15.39:0/313501324 wait complete. 2024-09-21T07:39:59.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.915+0000 7f03d9c00640 1 Processor -- start 2024-09-21T07:39:59.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.915+0000 7f03d9c00640 1 -- start start 2024-09-21T07:39:59.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.917+0000 7f03d9c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03d40cc0c0 0x7f03d408bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:59.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.917+0000 7f03d9c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03d415fc30 0x7f03d40ae2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:59.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.917+0000 7f03d9c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03d4160860 0x7f03d41f7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:59.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.917+0000 7f03d9c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f03d406faa0 con 0x7f03d4160860 2024-09-21T07:39:59.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.917+0000 7f03d9c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f03d406f920 con 0x7f03d415fc30 2024-09-21T07:39:59.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.917+0000 7f03d9c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f03d406fc20 con 0x7f03d40cc0c0 2024-09-21T07:39:59.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.917+0000 7f03d9200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03d4160860 0x7f03d41f7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:59.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.917+0000 7f03d9200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03d4160860 0x7f03d41f7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57088/0 (socket says 172.21.15.39:57088) 2024-09-21T07:39:59.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.917+0000 7f03d9200640 1 -- 172.21.15.39:0/2114981515 learned_addr learned my addr 172.21.15.39:0/2114981515 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:39:59.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.917+0000 7f03cbe00640 1 --2- 172.21.15.39:0/2114981515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03d415fc30 0x7f03d40ae2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:59.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.917+0000 7f03d3e00640 1 --2- 172.21.15.39:0/2114981515 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03d40cc0c0 0x7f03d408bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:59.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.917+0000 7f03d9200640 1 -- 172.21.15.39:0/2114981515 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03d40cc0c0 msgr2=0x7f03d408bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:59.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.917+0000 7f03d9200640 1 --2- 172.21.15.39:0/2114981515 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03d40cc0c0 0x7f03d408bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:59.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.917+0000 7f03d9200640 1 -- 172.21.15.39:0/2114981515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03d415fc30 msgr2=0x7f03d40ae2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:39:59.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.917+0000 7f03d9200640 1 --2- 172.21.15.39:0/2114981515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03d415fc30 0x7f03d40ae2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:39:59.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.917+0000 7f03d9200640 1 -- 172.21.15.39:0/2114981515 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d408c8d0 con 0x7f03d4160860 2024-09-21T07:39:59.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.918+0000 7f03d9200640 1 --2- 172.21.15.39:0/2114981515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03d4160860 0x7f03d41f7770 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f03cc00c730 tx=0x7f03cc00cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:59.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.918+0000 7f03d0c00640 1 -- 172.21.15.39:0/2114981515 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03cc003b70 con 0x7f03d4160860 2024-09-21T07:39:59.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.918+0000 7f03d0c00640 1 -- 172.21.15.39:0/2114981515 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f03cc003d10 con 0x7f03d4160860 2024-09-21T07:39:59.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.918+0000 7f03d9c00640 1 -- 172.21.15.39:0/2114981515 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03d409a430 con 0x7f03d4160860 2024-09-21T07:39:59.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.918+0000 7f03d0c00640 1 -- 172.21.15.39:0/2114981515 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03cc005330 con 0x7f03d4160860 2024-09-21T07:39:59.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.918+0000 7f03d9c00640 1 -- 172.21.15.39:0/2114981515 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03d409ac70 con 0x7f03d4160860 2024-09-21T07:39:59.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.919+0000 7f03d9c00640 1 -- 172.21.15.39:0/2114981515 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03d4160320 con 0x7f03d4160860 2024-09-21T07:39:59.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.920+0000 7f03d0c00640 1 -- 172.21.15.39:0/2114981515 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f03cc0054d0 con 0x7f03d4160860 2024-09-21T07:39:59.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.922+0000 7f03d0c00640 1 --2- 172.21.15.39:0/2114981515 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f03b0078b90 0x7f03b007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:39:59.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.922+0000 7f03d0c00640 1 -- 172.21.15.39:0/2114981515 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 7432+0+0 (secure 0 0 0) 0x7f03cc09b2c0 con 0x7f03d4160860 2024-09-21T07:39:59.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.922+0000 7f03d3e00640 1 --2- 172.21.15.39:0/2114981515 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f03b0078b90 0x7f03b007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:39:59.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.923+0000 7f03d3e00640 1 --2- 172.21.15.39:0/2114981515 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f03b0078b90 0x7f03b007b0a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f03c4033000 tx=0x7f03c4004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:39:59.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:39:59.927+0000 7f03d0c00640 1 -- 172.21.15.39:0/2114981515 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f03d409a430 con 0x7f03d4160860 2024-09-21T07:40:00.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.163+0000 7f03d9c00640 1 -- 172.21.15.39:0/2114981515 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03d40d1080 con 0x7f03d4160860 2024-09-21T07:40:00.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.164+0000 7f03d0c00640 1 -- 172.21.15.39:0/2114981515 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+5300 (secure 0 0 0) 0x7f03d4160c70 con 0x7f03d4160860 2024-09-21T07:40:00.165 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:40:00.165 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":105,"btime":"2024-09-21T07:39:56:405074+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7113,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2821446711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2821446711},{"type":"v1","addr":"172.21.15.39:6835","nonce":2821446711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:39:56.405046+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6619,"mds_1":7080},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6619":{"gid":6619,"name":"d","rank":0,"incarnation":95,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/992188984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":992188984},{"type":"v1","addr":"172.21.15.148:6837","nonce":992188984}]},"join_fscid":-1,"laggy_since":"2024-09-21T07:39:56.405046+0000","export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7080":{"gid":7080,"name":"c","rank":1,"incarnation":98,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6833/1456343023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1456343023},{"type":"v1","addr":"172.21.15.39:6833","nonce":1456343023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6619,"qdb_cluster":[7080,6619]},"id":6}]} 2024-09-21T07:40:00.165 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 105 2024-09-21T07:40:00.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.166+0000 7f03d9c00640 1 -- 172.21.15.39:0/2114981515 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f03b0078b90 msgr2=0x7f03b007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:00.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.166+0000 7f03d9c00640 1 --2- 172.21.15.39:0/2114981515 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f03b0078b90 0x7f03b007b0a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f03c4033000 tx=0x7f03c4004730 comp rx=0 tx=0).stop 2024-09-21T07:40:00.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.166+0000 7f03d9c00640 1 -- 172.21.15.39:0/2114981515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03d4160860 msgr2=0x7f03d41f7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:00.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.166+0000 7f03d9c00640 1 --2- 172.21.15.39:0/2114981515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03d4160860 0x7f03d41f7770 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f03cc00c730 tx=0x7f03cc00cb30 comp rx=0 tx=0).stop 2024-09-21T07:40:00.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.166+0000 7f03d9c00640 1 -- 172.21.15.39:0/2114981515 shutdown_connections 2024-09-21T07:40:00.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.166+0000 7f03d9c00640 1 --2- 172.21.15.39:0/2114981515 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f03b0078b90 0x7f03b007b0a0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:00.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.166+0000 7f03d9c00640 1 --2- 172.21.15.39:0/2114981515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03d4160860 0x7f03d41f7770 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:00.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.166+0000 7f03d9c00640 1 --2- 172.21.15.39:0/2114981515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03d415fc30 0x7f03d40ae2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:00.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.166+0000 7f03d9c00640 1 --2- 172.21.15.39:0/2114981515 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03d40cc0c0 0x7f03d408bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:00.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.166+0000 7f03d9c00640 1 -- 172.21.15.39:0/2114981515 >> 172.21.15.39:0/2114981515 conn(0x7f03d40b2510 msgr2=0x7f03d40748f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:00.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.166+0000 7f03d9c00640 1 -- 172.21.15.39:0/2114981515 shutdown_connections 2024-09-21T07:40:00.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.166+0000 7f03d9c00640 1 -- 172.21.15.39:0/2114981515 wait complete. 2024-09-21T07:40:00.190 INFO:tasks.ceph.mds.c:Sent signal 9 2024-09-21T07:40:00.190 INFO:tasks.cephfs.mount:Suspending the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:40:00.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:40:00.190 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 down 2024-09-21T07:40:00.255 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 6:0 2024-09-21T07:40:00.391 INFO:tasks.ceph.mds.c.smithi039.stderr:daemon-helper: command crashed with signal 9 2024-09-21T07:40:00.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.487+0000 7fe77d000640 1 -- 172.21.15.39:0/143751073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe75c004800 msgr2=0x7fe75c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:00.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.487+0000 7fe77d000640 1 --2- 172.21.15.39:0/143751073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe75c004800 0x7fe75c004c10 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fe764006f80 tx=0x7fe75c005830 comp rx=0 tx=0).stop 2024-09-21T07:40:00.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.487+0000 7fe777e00640 1 -- 172.21.15.39:0/143751073 reap_dead start 2024-09-21T07:40:00.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.488+0000 7fe77d000640 1 -- 172.21.15.39:0/143751073 shutdown_connections 2024-09-21T07:40:00.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.488+0000 7fe77d000640 1 -- 172.21.15.39:0/143751073 >> 172.21.15.39:0/143751073 conn(0x7fe7780b2510 msgr2=0x7fe7780d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:00.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.488+0000 7fe77d000640 1 -- 172.21.15.39:0/143751073 shutdown_connections 2024-09-21T07:40:00.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.488+0000 7fe77d000640 1 -- 172.21.15.39:0/143751073 wait complete. 2024-09-21T07:40:00.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.490+0000 7fe77d000640 1 Processor -- start 2024-09-21T07:40:00.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.490+0000 7fe77d000640 1 -- start start 2024-09-21T07:40:00.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.491+0000 7fe77d000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe75c004800 0x7fe77815b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:00.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.491+0000 7fe77d000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe77809c300 0x7fe7780cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:00.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.492+0000 7fe77d000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7780ce4b0 0x7fe7781f8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:00.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.492+0000 7fe77d000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe77815cc40 con 0x7fe75c004800 2024-09-21T07:40:00.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.492+0000 7fe77d000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe77815cac0 con 0x7fe77809c300 2024-09-21T07:40:00.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.492+0000 7fe77d000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe7780b43f0 con 0x7fe7780ce4b0 2024-09-21T07:40:00.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.492+0000 7fe777400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe75c004800 0x7fe77815b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:00.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.492+0000 7fe777400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe75c004800 0x7fe77815b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57114/0 (socket says 172.21.15.39:57114) 2024-09-21T07:40:00.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.492+0000 7fe777400640 1 -- 172.21.15.39:0/844397429 learned_addr learned my addr 172.21.15.39:0/844397429 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:40:00.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.492+0000 7fe776a00640 1 --2- 172.21.15.39:0/844397429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe77809c300 0x7fe7780cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:00.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.492+0000 7fe777e00640 1 --2- 172.21.15.39:0/844397429 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7780ce4b0 0x7fe7781f8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:00.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.492+0000 7fe777400640 1 -- 172.21.15.39:0/844397429 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7780ce4b0 msgr2=0x7fe7781f8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:00.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.492+0000 7fe777400640 1 --2- 172.21.15.39:0/844397429 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7780ce4b0 0x7fe7781f8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:00.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.492+0000 7fe777400640 1 -- 172.21.15.39:0/844397429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe77809c300 msgr2=0x7fe7780cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:00.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.492+0000 7fe777400640 1 --2- 172.21.15.39:0/844397429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe77809c300 0x7fe7780cdf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:00.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.492+0000 7fe777400640 1 -- 172.21.15.39:0/844397429 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe77815d8a0 con 0x7fe75c004800 2024-09-21T07:40:00.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.493+0000 7fe777400640 1 --2- 172.21.15.39:0/844397429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe75c004800 0x7fe77815b510 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fe7680119e0 tx=0x7fe768011ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:00.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.493+0000 7fe76f400640 1 -- 172.21.15.39:0/844397429 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76800fd20 con 0x7fe75c004800 2024-09-21T07:40:00.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.493+0000 7fe76f400640 1 -- 172.21.15.39:0/844397429 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe768004030 con 0x7fe75c004800 2024-09-21T07:40:00.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.493+0000 7fe77d000640 1 -- 172.21.15.39:0/844397429 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe77809b540 con 0x7fe75c004800 2024-09-21T07:40:00.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.493+0000 7fe76f400640 1 -- 172.21.15.39:0/844397429 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe768004480 con 0x7fe75c004800 2024-09-21T07:40:00.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.493+0000 7fe77d000640 1 -- 172.21.15.39:0/844397429 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe77809bb00 con 0x7fe75c004800 2024-09-21T07:40:00.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.494+0000 7fe77d000640 1 -- 172.21.15.39:0/844397429 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe77815dfe0 con 0x7fe75c004800 2024-09-21T07:40:00.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.495+0000 7fe76f400640 1 -- 172.21.15.39:0/844397429 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe7680035f0 con 0x7fe75c004800 2024-09-21T07:40:00.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.496+0000 7fe76f400640 1 --2- 172.21.15.39:0/844397429 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe758078b90 0x7fe75807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:00.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.496+0000 7fe76f400640 1 -- 172.21.15.39:0/844397429 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 7432+0+0 (secure 0 0 0) 0x7fe76809c1b0 con 0x7fe75c004800 2024-09-21T07:40:00.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.497+0000 7fe776a00640 1 --2- 172.21.15.39:0/844397429 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe758078b90 0x7fe75807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:00.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.498+0000 7fe776a00640 1 --2- 172.21.15.39:0/844397429 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe758078b90 0x7fe75807b0a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe764033000 tx=0x7fe76402d000 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:00.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.501+0000 7fe76f400640 1 -- 172.21.15.39:0/844397429 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe77815dfe0 con 0x7fe75c004800 2024-09-21T07:40:00.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.777+0000 7fe77d000640 1 -- 172.21.15.39:0/844397429 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "6:0"} v 0) -- 0x7fe7780b4950 con 0x7fe75c004800 2024-09-21T07:40:00.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.780+0000 7fe76f400640 1 -- 172.21.15.39:0/844397429 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "6:0"}]=0 failed mds gid 6619 v106) ==== 97+0+0 (secure 0 0 0) 0x7fe768063e90 con 0x7fe75c004800 2024-09-21T07:40:00.781 INFO:teuthology.orchestra.run.smithi039.stderr:failed mds gid 6619 2024-09-21T07:40:00.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.782+0000 7fe77d000640 1 -- 172.21.15.39:0/844397429 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe758078b90 msgr2=0x7fe75807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:00.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.783+0000 7fe77d000640 1 --2- 172.21.15.39:0/844397429 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe758078b90 0x7fe75807b0a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe764033000 tx=0x7fe76402d000 comp rx=0 tx=0).stop 2024-09-21T07:40:00.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.783+0000 7fe77d000640 1 -- 172.21.15.39:0/844397429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe75c004800 msgr2=0x7fe77815b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:00.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.783+0000 7fe77d000640 1 --2- 172.21.15.39:0/844397429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe75c004800 0x7fe77815b510 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fe7680119e0 tx=0x7fe768011ea0 comp rx=0 tx=0).stop 2024-09-21T07:40:00.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.783+0000 7fe77d000640 1 -- 172.21.15.39:0/844397429 shutdown_connections 2024-09-21T07:40:00.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.783+0000 7fe77d000640 1 --2- 172.21.15.39:0/844397429 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe758078b90 0x7fe75807b0a0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:00.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.783+0000 7fe77d000640 1 --2- 172.21.15.39:0/844397429 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7780ce4b0 0x7fe7781f8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:00.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.783+0000 7fe77d000640 1 --2- 172.21.15.39:0/844397429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe77809c300 0x7fe7780cdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:00.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.783+0000 7fe77d000640 1 --2- 172.21.15.39:0/844397429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe75c004800 0x7fe77815b510 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:00.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.783+0000 7fe77d000640 1 -- 172.21.15.39:0/844397429 >> 172.21.15.39:0/844397429 conn(0x7fe7780b2510 msgr2=0x7fe7781e8520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:00.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.783+0000 7fe77d000640 1 -- 172.21.15.39:0/844397429 shutdown_connections 2024-09-21T07:40:00.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:00.783+0000 7fe77d000640 1 -- 172.21.15.39:0/844397429 wait complete. 2024-09-21T07:40:00.816 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T07:40:00.816 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T07:40:00.817 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T07:40:00.817 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:40:00.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:40:00.817 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/daemon/state.py", line 146, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi148 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d' 2024-09-21T07:40:00.817 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T07:40:00.817 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T07:40:00.822 INFO:tasks.ceph.mds.d:Started 2024-09-21T07:40:00.822 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:40:01.019 INFO:tasks.ceph.mds.d.smithi148.stdout:starting mds.d at 2024-09-21T07:40:01.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.048+0000 7f7606400640 1 -- 172.21.15.39:0/469802274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76000d0d20 msgr2=0x7f76000d1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:01.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.048+0000 7f7606400640 1 --2- 172.21.15.39:0/469802274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76000d0d20 0x7f76000d1130 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f75f8005fd0 tx=0x7f75f802aaa0 comp rx=0 tx=0).stop 2024-09-21T07:40:01.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.048+0000 7f7606400640 1 -- 172.21.15.39:0/469802274 shutdown_connections 2024-09-21T07:40:01.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.048+0000 7f7606400640 1 --2- 172.21.15.39:0/469802274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76000d0d20 0x7f76000d1130 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:01.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.048+0000 7f7606400640 1 --2- 172.21.15.39:0/469802274 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f76000b96b0 0x7f7600161fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:01.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.048+0000 7f7606400640 1 -- 172.21.15.39:0/469802274 >> 172.21.15.39:0/469802274 conn(0x7f760008bfb0 msgr2=0x7f760009d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:01.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.048+0000 7f7606400640 1 -- 172.21.15.39:0/469802274 shutdown_connections 2024-09-21T07:40:01.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.048+0000 7f7606400640 1 -- 172.21.15.39:0/469802274 wait complete. 2024-09-21T07:40:01.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.050+0000 7f7606400640 1 Processor -- start 2024-09-21T07:40:01.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.050+0000 7f7606400640 1 -- start start 2024-09-21T07:40:01.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.052+0000 7f7606400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f760006df50 0x7f76000ca560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:01.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.052+0000 7f7606400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f76000b96b0 0x7f76001f5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:01.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.052+0000 7f7606400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76000d0d20 0x7f76000ce270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:01.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.052+0000 7f7606400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f76000d26a0 con 0x7f76000d0d20 2024-09-21T07:40:01.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.052+0000 7f7606400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f76000d2520 con 0x7f760006df50 2024-09-21T07:40:01.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.052+0000 7f7606400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f76000d2820 con 0x7f76000b96b0 2024-09-21T07:40:01.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.052+0000 7f7605a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76000d0d20 0x7f76000ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:01.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.052+0000 7f7605000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f760006df50 0x7f76000ca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:01.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.052+0000 7f7605a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76000d0d20 0x7f76000ce270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57136/0 (socket says 172.21.15.39:57136) 2024-09-21T07:40:01.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.052+0000 7f7605a00640 1 -- 172.21.15.39:0/408437758 learned_addr learned my addr 172.21.15.39:0/408437758 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:40:01.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.052+0000 7f75ffe00640 1 --2- 172.21.15.39:0/408437758 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f76000b96b0 0x7f76001f5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:01.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.053+0000 7f7605a00640 1 -- 172.21.15.39:0/408437758 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f76000b96b0 msgr2=0x7f76001f5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:01.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.053+0000 7f7605a00640 1 --2- 172.21.15.39:0/408437758 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f76000b96b0 0x7f76001f5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:01.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.053+0000 7f7605a00640 1 -- 172.21.15.39:0/408437758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f760006df50 msgr2=0x7f76000ca560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:01.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.053+0000 7f7605a00640 1 --2- 172.21.15.39:0/408437758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f760006df50 0x7f76000ca560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:01.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.053+0000 7f7605a00640 1 -- 172.21.15.39:0/408437758 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76001f63a0 con 0x7f76000d0d20 2024-09-21T07:40:01.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.053+0000 7f7605a00640 1 --2- 172.21.15.39:0/408437758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76000d0d20 0x7f76000ce270 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f75f8006010 tx=0x7f75f8003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:01.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.053+0000 7f75fcc00640 1 -- 172.21.15.39:0/408437758 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f8044070 con 0x7f76000d0d20 2024-09-21T07:40:01.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.053+0000 7f75fcc00640 1 -- 172.21.15.39:0/408437758 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75f8004250 con 0x7f76000d0d20 2024-09-21T07:40:01.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.053+0000 7f7606400640 1 -- 172.21.15.39:0/408437758 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76000c94c0 con 0x7f76000d0d20 2024-09-21T07:40:01.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.053+0000 7f75fcc00640 1 -- 172.21.15.39:0/408437758 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f8004680 con 0x7f76000d0d20 2024-09-21T07:40:01.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.053+0000 7f7606400640 1 -- 172.21.15.39:0/408437758 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76000c9a80 con 0x7f76000d0d20 2024-09-21T07:40:01.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.055+0000 7f7606400640 1 -- 172.21.15.39:0/408437758 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76000c9230 con 0x7f76000d0d20 2024-09-21T07:40:01.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.056+0000 7f75fcc00640 1 -- 172.21.15.39:0/408437758 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f75f8047070 con 0x7f76000d0d20 2024-09-21T07:40:01.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.057+0000 7f75fcc00640 1 --2- 172.21.15.39:0/408437758 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f75e0078b90 0x7f75e007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:01.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.057+0000 7f75fcc00640 1 -- 172.21.15.39:0/408437758 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 7474+0+0 (secure 0 0 0) 0x7f75f80bc5b0 con 0x7f76000d0d20 2024-09-21T07:40:01.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.057+0000 7f7605000640 1 --2- 172.21.15.39:0/408437758 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f75e0078b90 0x7f75e007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:01.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.058+0000 7f7605000640 1 --2- 172.21.15.39:0/408437758 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f75e0078b90 0x7f75e007b0a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f75f400ab20 tx=0x7f75f400a250 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:01.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.061+0000 7f75fcc00640 1 -- 172.21.15.39:0/408437758 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f76001f63a0 con 0x7f76000d0d20 2024-09-21T07:40:01.120 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:40:01.124 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:40:01.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.299+0000 7f7606400640 1 -- 172.21.15.39:0/408437758 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76000d2cd0 con 0x7f76000d0d20 2024-09-21T07:40:01.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.300+0000 7f75fcc00640 1 -- 172.21.15.39:0/408437758 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) ==== 95+0+4422 (secure 0 0 0) 0x7f75f8032030 con 0x7f76000d0d20 2024-09-21T07:40:01.301 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:40:01.302 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":107,"btime":"2024-09-21T07:40:00:785252+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:40:00.785235+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7113,"mds_1":7080},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7080":{"gid":7080,"name":"c","rank":1,"incarnation":98,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6833/1456343023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1456343023},{"type":"v1","addr":"172.21.15.39:6833","nonce":1456343023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7113":{"gid":7113,"name":"a","rank":0,"incarnation":107,"state":"up:replay","state_seq":1,"addr":"172.21.15.39:6835/2821446711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2821446711},{"type":"v1","addr":"172.21.15.39:6835","nonce":2821446711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7080,"qdb_cluster":[7080]},"id":6}]} 2024-09-21T07:40:01.302 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 107 2024-09-21T07:40:01.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.303+0000 7f7606400640 1 -- 172.21.15.39:0/408437758 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f75e0078b90 msgr2=0x7f75e007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:01.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.303+0000 7f7606400640 1 --2- 172.21.15.39:0/408437758 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f75e0078b90 0x7f75e007b0a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f75f400ab20 tx=0x7f75f400a250 comp rx=0 tx=0).stop 2024-09-21T07:40:01.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.303+0000 7f7606400640 1 -- 172.21.15.39:0/408437758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76000d0d20 msgr2=0x7f76000ce270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:01.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.303+0000 7f7606400640 1 --2- 172.21.15.39:0/408437758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76000d0d20 0x7f76000ce270 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f75f8006010 tx=0x7f75f8003f10 comp rx=0 tx=0).stop 2024-09-21T07:40:01.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.303+0000 7f7606400640 1 -- 172.21.15.39:0/408437758 shutdown_connections 2024-09-21T07:40:01.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.303+0000 7f7606400640 1 --2- 172.21.15.39:0/408437758 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f75e0078b90 0x7f75e007b0a0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:01.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.303+0000 7f7606400640 1 --2- 172.21.15.39:0/408437758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76000d0d20 0x7f76000ce270 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:01.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.303+0000 7f7606400640 1 --2- 172.21.15.39:0/408437758 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f76000b96b0 0x7f76001f5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:01.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.303+0000 7f7606400640 1 --2- 172.21.15.39:0/408437758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f760006df50 0x7f76000ca560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:01.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.303+0000 7f7606400640 1 -- 172.21.15.39:0/408437758 >> 172.21.15.39:0/408437758 conn(0x7f760008bfb0 msgr2=0x7f76000cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:01.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.303+0000 7f7606400640 1 -- 172.21.15.39:0/408437758 shutdown_connections 2024-09-21T07:40:01.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:01.303+0000 7f7606400640 1 -- 172.21.15.39:0/408437758 wait complete. 2024-09-21T07:40:01.328 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-09-21T07:40:02.662 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-09-21T07:40:02.662 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-09-21T07:40:06.328 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:40:06.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.571+0000 7fb6fe200640 1 -- 172.21.15.39:0/3204278644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6f80d0620 msgr2=0x7fb6f80d0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:06.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.571+0000 7fb6fe200640 1 --2- 172.21.15.39:0/3204278644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6f80d0620 0x7fb6f80d0a30 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0005fd0 tx=0x7fb6f002aaa0 comp rx=0 tx=0).stop 2024-09-21T07:40:06.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.571+0000 7fb6fe200640 1 -- 172.21.15.39:0/3204278644 shutdown_connections 2024-09-21T07:40:06.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.571+0000 7fb6fe200640 1 --2- 172.21.15.39:0/3204278644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6f80d0620 0x7fb6f80d0a30 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:06.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.571+0000 7fb6fe200640 1 -- 172.21.15.39:0/3204278644 >> 172.21.15.39:0/3204278644 conn(0x7fb6f806cfc0 msgr2=0x7fb6f806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:06.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.571+0000 7fb6fe200640 1 -- 172.21.15.39:0/3204278644 shutdown_connections 2024-09-21T07:40:06.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.571+0000 7fb6fe200640 1 -- 172.21.15.39:0/3204278644 wait complete. 2024-09-21T07:40:06.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.573+0000 7fb6fe200640 1 Processor -- start 2024-09-21T07:40:06.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.574+0000 7fb6fe200640 1 -- start start 2024-09-21T07:40:06.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.575+0000 7fb6fe200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6f80cfd10 0x7fb6f8146e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:06.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.575+0000 7fb6fe200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6f80d0620 0x7fb6f81e5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:06.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.575+0000 7fb6fe200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb6f815cd30 0x7fb6f81e5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:06.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.575+0000 7fb6fe200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f8073aa0 con 0x7fb6f80cfd10 2024-09-21T07:40:06.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.575+0000 7fb6fe200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f8073920 con 0x7fb6f80d0620 2024-09-21T07:40:06.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.575+0000 7fb6fe200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb6f8073c20 con 0x7fb6f815cd30 2024-09-21T07:40:06.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.575+0000 7fb6f7400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6f80cfd10 0x7fb6f8146e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:06.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.575+0000 7fb6f7400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6f80cfd10 0x7fb6f8146e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48852/0 (socket says 172.21.15.39:48852) 2024-09-21T07:40:06.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.575+0000 7fb6f7400640 1 -- 172.21.15.39:0/2485586913 learned_addr learned my addr 172.21.15.39:0/2485586913 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:40:06.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.576+0000 7fb6f7e00640 1 --2- 172.21.15.39:0/2485586913 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb6f815cd30 0x7fb6f81e5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:06.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.576+0000 7fb6eea00640 1 --2- 172.21.15.39:0/2485586913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6f80d0620 0x7fb6f81e5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:06.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.576+0000 7fb6f7400640 1 -- 172.21.15.39:0/2485586913 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb6f815cd30 msgr2=0x7fb6f81e5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:06.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.576+0000 7fb6f7400640 1 --2- 172.21.15.39:0/2485586913 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb6f815cd30 0x7fb6f81e5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:06.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.576+0000 7fb6f7400640 1 -- 172.21.15.39:0/2485586913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6f80d0620 msgr2=0x7fb6f81e5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:06.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.576+0000 7fb6f7400640 1 --2- 172.21.15.39:0/2485586913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6f80d0620 0x7fb6f81e5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:06.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.576+0000 7fb6f7400640 1 -- 172.21.15.39:0/2485586913 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f81e79a0 con 0x7fb6f80cfd10 2024-09-21T07:40:06.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.576+0000 7fb6f7e00640 1 --2- 172.21.15.39:0/2485586913 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb6f815cd30 0x7fb6f81e5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:40:06.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.576+0000 7fb6eea00640 1 --2- 172.21.15.39:0/2485586913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6f80d0620 0x7fb6f81e5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:40:06.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.576+0000 7fb6f7400640 1 --2- 172.21.15.39:0/2485586913 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6f80cfd10 0x7fb6f8146e40 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800c370 tx=0x7fb6e800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:06.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.576+0000 7fb6efe00640 1 -- 172.21.15.39:0/2485586913 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8017070 con 0x7fb6f80cfd10 2024-09-21T07:40:06.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.576+0000 7fb6efe00640 1 -- 172.21.15.39:0/2485586913 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb6e80035f0 con 0x7fb6f80cfd10 2024-09-21T07:40:06.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.576+0000 7fb6efe00640 1 -- 172.21.15.39:0/2485586913 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8003a40 con 0x7fb6f80cfd10 2024-09-21T07:40:06.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.576+0000 7fb6fe200640 1 -- 172.21.15.39:0/2485586913 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6f81e7c80 con 0x7fb6f80cfd10 2024-09-21T07:40:06.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.576+0000 7fb6fe200640 1 -- 172.21.15.39:0/2485586913 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6f81e8240 con 0x7fb6f80cfd10 2024-09-21T07:40:06.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.577+0000 7fb6fe200640 1 -- 172.21.15.39:0/2485586913 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6f8074ae0 con 0x7fb6f80cfd10 2024-09-21T07:40:06.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.579+0000 7fb6efe00640 1 -- 172.21.15.39:0/2485586913 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb6e801a070 con 0x7fb6f80cfd10 2024-09-21T07:40:06.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.580+0000 7fb6efe00640 1 --2- 172.21.15.39:0/2485586913 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb6d4078bc0 0x7fb6d407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:06.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.580+0000 7fb6efe00640 1 -- 172.21.15.39:0/2485586913 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 7474+0+0 (secure 0 0 0) 0x7fb6e809a760 con 0x7fb6f80cfd10 2024-09-21T07:40:06.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.580+0000 7fb6eea00640 1 --2- 172.21.15.39:0/2485586913 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb6d4078bc0 0x7fb6d407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:06.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.581+0000 7fb6eea00640 1 --2- 172.21.15.39:0/2485586913 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb6d4078bc0 0x7fb6d407b0d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0000f60 tx=0x7fb6e0009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:06.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.584+0000 7fb6efe00640 1 -- 172.21.15.39:0/2485586913 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb6f8074ae0 con 0x7fb6f80cfd10 2024-09-21T07:40:06.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.822+0000 7fb6fe200640 1 -- 172.21.15.39:0/2485586913 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6f80d0a30 con 0x7fb6f80cfd10 2024-09-21T07:40:06.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.824+0000 7fb6efe00640 1 -- 172.21.15.39:0/2485586913 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) ==== 95+0+5253 (secure 0 0 0) 0x7fb6e8062410 con 0x7fb6f80cfd10 2024-09-21T07:40:06.825 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:40:06.826 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":108,"btime":"2024-09-21T07:40:01:788092+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6269,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2336112936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2336112936},{"type":"v1","addr":"172.21.15.148:6837","nonce":2336112936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:40:00.809155+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7113,"mds_1":7080},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7080":{"gid":7080,"name":"c","rank":1,"incarnation":98,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6833/1456343023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1456343023},{"type":"v1","addr":"172.21.15.39:6833","nonce":1456343023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7113":{"gid":7113,"name":"a","rank":0,"incarnation":107,"state":"up:resolve","state_seq":13,"addr":"172.21.15.39:6835/2821446711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2821446711},{"type":"v1","addr":"172.21.15.39:6835","nonce":2821446711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7080,"qdb_cluster":[7080]},"id":6}]} 2024-09-21T07:40:06.826 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 108 2024-09-21T07:40:06.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.827+0000 7fb6fe200640 1 -- 172.21.15.39:0/2485586913 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb6d4078bc0 msgr2=0x7fb6d407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:06.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.827+0000 7fb6fe200640 1 --2- 172.21.15.39:0/2485586913 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb6d4078bc0 0x7fb6d407b0d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0000f60 tx=0x7fb6e0009040 comp rx=0 tx=0).stop 2024-09-21T07:40:06.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.827+0000 7fb6fe200640 1 -- 172.21.15.39:0/2485586913 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6f80cfd10 msgr2=0x7fb6f8146e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:06.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.827+0000 7fb6fe200640 1 --2- 172.21.15.39:0/2485586913 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6f80cfd10 0x7fb6f8146e40 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800c370 tx=0x7fb6e800c830 comp rx=0 tx=0).stop 2024-09-21T07:40:06.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.827+0000 7fb6fe200640 1 -- 172.21.15.39:0/2485586913 shutdown_connections 2024-09-21T07:40:06.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.827+0000 7fb6fe200640 1 --2- 172.21.15.39:0/2485586913 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb6d4078bc0 0x7fb6d407b0d0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:06.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.827+0000 7fb6fe200640 1 --2- 172.21.15.39:0/2485586913 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb6f815cd30 0x7fb6f81e5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:06.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.827+0000 7fb6fe200640 1 --2- 172.21.15.39:0/2485586913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6f80d0620 0x7fb6f81e5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:06.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.827+0000 7fb6fe200640 1 --2- 172.21.15.39:0/2485586913 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6f80cfd10 0x7fb6f8146e40 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:06.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.827+0000 7fb6fe200640 1 -- 172.21.15.39:0/2485586913 >> 172.21.15.39:0/2485586913 conn(0x7fb6f806cfc0 msgr2=0x7fb6f80aeb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:06.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.827+0000 7fb6fe200640 1 -- 172.21.15.39:0/2485586913 shutdown_connections 2024-09-21T07:40:06.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:06.827+0000 7fb6fe200640 1 -- 172.21.15.39:0/2485586913 wait complete. 2024-09-21T07:40:06.854 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-21T07:40:06.854 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:40:07.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.082+0000 7ff76ce00640 1 -- 172.21.15.39:0/2410051558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff74c004000 msgr2=0x7ff74c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:07.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.082+0000 7ff76ce00640 1 --2- 172.21.15.39:0/2410051558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff74c004000 0x7ff74c004800 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7ff750009b50 tx=0x7ff74c005830 comp rx=0 tx=0).stop 2024-09-21T07:40:07.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.082+0000 7ff766800640 1 -- 172.21.15.39:0/2410051558 reap_dead start 2024-09-21T07:40:07.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.082+0000 7ff76ce00640 1 -- 172.21.15.39:0/2410051558 shutdown_connections 2024-09-21T07:40:07.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.082+0000 7ff76ce00640 1 -- 172.21.15.39:0/2410051558 >> 172.21.15.39:0/2410051558 conn(0x7ff76806cfc0 msgr2=0x7ff76806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:07.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.082+0000 7ff76ce00640 1 -- 172.21.15.39:0/2410051558 shutdown_connections 2024-09-21T07:40:07.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.082+0000 7ff76ce00640 1 -- 172.21.15.39:0/2410051558 wait complete. 2024-09-21T07:40:07.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.084+0000 7ff76ce00640 1 Processor -- start 2024-09-21T07:40:07.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.085+0000 7ff76ce00640 1 -- start start 2024-09-21T07:40:07.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.086+0000 7ff76ce00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff74c004000 0x7ff768144960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:07.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.086+0000 7ff76ce00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff7680cbb10 0x7ff768147110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:07.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.086+0000 7ff76ce00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff768147650 0x7ff768146120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:07.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.086+0000 7ff76ce00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff7680cdee0 con 0x7ff768147650 2024-09-21T07:40:07.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.086+0000 7ff76ce00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff7680cdd60 con 0x7ff74c004000 2024-09-21T07:40:07.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.086+0000 7ff76ce00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7ff7680ce060 con 0x7ff7680cbb10 2024-09-21T07:40:07.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.086+0000 7ff766800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff768147650 0x7ff768146120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:07.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.086+0000 7ff766800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff768147650 0x7ff768146120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48868/0 (socket says 172.21.15.39:48868) 2024-09-21T07:40:07.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.086+0000 7ff766800640 1 -- 172.21.15.39:0/3698859870 learned_addr learned my addr 172.21.15.39:0/3698859870 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:40:07.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.087+0000 7ff765e00640 1 --2- 172.21.15.39:0/3698859870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff74c004000 0x7ff768144960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:07.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.087+0000 7ff75fe00640 1 --2- 172.21.15.39:0/3698859870 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff7680cbb10 0x7ff768147110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:07.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.087+0000 7ff766800640 1 -- 172.21.15.39:0/3698859870 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff7680cbb10 msgr2=0x7ff768147110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:07.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.087+0000 7ff766800640 1 --2- 172.21.15.39:0/3698859870 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff7680cbb10 0x7ff768147110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:07.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.087+0000 7ff766800640 1 -- 172.21.15.39:0/3698859870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff74c004000 msgr2=0x7ff768144960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:07.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.087+0000 7ff766800640 1 --2- 172.21.15.39:0/3698859870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff74c004000 0x7ff768144960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:07.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.087+0000 7ff766800640 1 -- 172.21.15.39:0/3698859870 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff768147a60 con 0x7ff768147650 2024-09-21T07:40:07.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.087+0000 7ff766800640 1 --2- 172.21.15.39:0/3698859870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff768147650 0x7ff768146120 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7ff7680745f0 tx=0x7ff7600019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:07.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.087+0000 7ff75e000640 1 -- 172.21.15.39:0/3698859870 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff760011870 con 0x7ff768147650 2024-09-21T07:40:07.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.087+0000 7ff75e000640 1 -- 172.21.15.39:0/3698859870 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff760017070 con 0x7ff768147650 2024-09-21T07:40:07.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.087+0000 7ff75e000640 1 -- 172.21.15.39:0/3698859870 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff760011da0 con 0x7ff768147650 2024-09-21T07:40:07.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.088+0000 7ff76ce00640 1 -- 172.21.15.39:0/3698859870 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff768145080 con 0x7ff768147650 2024-09-21T07:40:07.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.088+0000 7ff76ce00640 1 -- 172.21.15.39:0/3698859870 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff768145640 con 0x7ff768147650 2024-09-21T07:40:07.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.089+0000 7ff76ce00640 1 -- 172.21.15.39:0/3698859870 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7680ced60 con 0x7ff768147650 2024-09-21T07:40:07.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.090+0000 7ff75e000640 1 -- 172.21.15.39:0/3698859870 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff76001c460 con 0x7ff768147650 2024-09-21T07:40:07.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.091+0000 7ff75e000640 1 --2- 172.21.15.39:0/3698859870 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff744078b90 0x7ff74407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:07.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.091+0000 7ff75e000640 1 -- 172.21.15.39:0/3698859870 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 7474+0+0 (secure 0 0 0) 0x7ff76009bbf0 con 0x7ff768147650 2024-09-21T07:40:07.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.091+0000 7ff765e00640 1 --2- 172.21.15.39:0/3698859870 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff744078b90 0x7ff74407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:07.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.092+0000 7ff765e00640 1 --2- 172.21.15.39:0/3698859870 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff744078b90 0x7ff74407b0a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff758000fb0 tx=0x7ff758006040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:07.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.095+0000 7ff75e000640 1 -- 172.21.15.39:0/3698859870 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff7680ced60 con 0x7ff768147650 2024-09-21T07:40:07.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.332+0000 7ff76ce00640 1 -- 172.21.15.39:0/3698859870 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7680cc2f0 con 0x7ff768147650 2024-09-21T07:40:07.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.333+0000 7ff75e000640 1 -- 172.21.15.39:0/3698859870 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) ==== 95+0+5253 (secure 0 0 0) 0x7ff768147a60 con 0x7ff768147650 2024-09-21T07:40:07.333 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:40:07.334 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":108,"btime":"2024-09-21T07:40:01:788092+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6269,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2336112936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2336112936},{"type":"v1","addr":"172.21.15.148:6837","nonce":2336112936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:40:00.809155+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7113,"mds_1":7080},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7080":{"gid":7080,"name":"c","rank":1,"incarnation":98,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6833/1456343023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1456343023},{"type":"v1","addr":"172.21.15.39:6833","nonce":1456343023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7113":{"gid":7113,"name":"a","rank":0,"incarnation":107,"state":"up:resolve","state_seq":13,"addr":"172.21.15.39:6835/2821446711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2821446711},{"type":"v1","addr":"172.21.15.39:6835","nonce":2821446711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7080,"qdb_cluster":[7080]},"id":6}]} 2024-09-21T07:40:07.334 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 108 2024-09-21T07:40:07.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.335+0000 7ff76ce00640 1 -- 172.21.15.39:0/3698859870 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff744078b90 msgr2=0x7ff74407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:07.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.335+0000 7ff76ce00640 1 --2- 172.21.15.39:0/3698859870 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff744078b90 0x7ff74407b0a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff758000fb0 tx=0x7ff758006040 comp rx=0 tx=0).stop 2024-09-21T07:40:07.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.335+0000 7ff76ce00640 1 -- 172.21.15.39:0/3698859870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff768147650 msgr2=0x7ff768146120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:07.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.335+0000 7ff76ce00640 1 --2- 172.21.15.39:0/3698859870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff768147650 0x7ff768146120 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7ff7680745f0 tx=0x7ff7600019b0 comp rx=0 tx=0).stop 2024-09-21T07:40:07.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.335+0000 7ff76ce00640 1 -- 172.21.15.39:0/3698859870 shutdown_connections 2024-09-21T07:40:07.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.335+0000 7ff76ce00640 1 --2- 172.21.15.39:0/3698859870 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff744078b90 0x7ff74407b0a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:07.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.335+0000 7ff76ce00640 1 --2- 172.21.15.39:0/3698859870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff768147650 0x7ff768146120 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:07.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.335+0000 7ff76ce00640 1 --2- 172.21.15.39:0/3698859870 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff7680cbb10 0x7ff768147110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:07.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.335+0000 7ff76ce00640 1 --2- 172.21.15.39:0/3698859870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff74c004000 0x7ff768144960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:07.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.335+0000 7ff76ce00640 1 -- 172.21.15.39:0/3698859870 >> 172.21.15.39:0/3698859870 conn(0x7ff76806cfc0 msgr2=0x7ff768070f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:07.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.335+0000 7ff76ce00640 1 -- 172.21.15.39:0/3698859870 shutdown_connections 2024-09-21T07:40:07.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.335+0000 7ff76ce00640 1 -- 172.21.15.39:0/3698859870 wait complete. 2024-09-21T07:40:07.360 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:resolve 2024-09-21T07:40:07.360 DEBUG:tasks.cephfs.filesystem:reached state 'up:resolve' in 0.506171464920044s 2024-09-21T07:40:07.361 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:40:07.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.587+0000 7f6290a00640 1 -- 172.21.15.39:0/236025917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6270003c10 msgr2=0x7f6270004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:07.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.587+0000 7f6290a00640 1 --2- 172.21.15.39:0/236025917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6270003c10 0x7f6270004060 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f627c001830 tx=0x7f6270005830 comp rx=0 tx=0).stop 2024-09-21T07:40:07.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.587+0000 7f628a800640 1 -- 172.21.15.39:0/236025917 reap_dead start 2024-09-21T07:40:07.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.587+0000 7f6290a00640 1 -- 172.21.15.39:0/236025917 shutdown_connections 2024-09-21T07:40:07.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.587+0000 7f6290a00640 1 -- 172.21.15.39:0/236025917 >> 172.21.15.39:0/236025917 conn(0x7f628c06cfc0 msgr2=0x7f628c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:07.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.587+0000 7f6290a00640 1 -- 172.21.15.39:0/236025917 shutdown_connections 2024-09-21T07:40:07.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.587+0000 7f6290a00640 1 -- 172.21.15.39:0/236025917 wait complete. 2024-09-21T07:40:07.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.590+0000 7f6290a00640 1 Processor -- start 2024-09-21T07:40:07.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.590+0000 7f6290a00640 1 -- start start 2024-09-21T07:40:07.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.591+0000 7f6290a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6270003c10 0x7f628c144c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:07.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.591+0000 7f6290a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f628c147740 0x7f628c147b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:07.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.591+0000 7f6290a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f628c0b4620 0x7f628c0b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:07.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.591+0000 7f6290a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f628c0cdf40 con 0x7f628c147740 2024-09-21T07:40:07.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.591+0000 7f6290a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f628c0cddc0 con 0x7f628c0b4620 2024-09-21T07:40:07.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.591+0000 7f6290a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f628c073710 con 0x7f6270003c10 2024-09-21T07:40:07.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.591+0000 7f6283e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f628c147740 0x7f628c147b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:07.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.592+0000 7f6283e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f628c147740 0x7f628c147b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48882/0 (socket says 172.21.15.39:48882) 2024-09-21T07:40:07.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.592+0000 7f6283e00640 1 -- 172.21.15.39:0/923614634 learned_addr learned my addr 172.21.15.39:0/923614634 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:40:07.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.592+0000 7f628a800640 1 --2- 172.21.15.39:0/923614634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f628c0b4620 0x7f628c0b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:07.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.592+0000 7f6289e00640 1 --2- 172.21.15.39:0/923614634 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6270003c10 0x7f628c144c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:07.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.592+0000 7f6283e00640 1 -- 172.21.15.39:0/923614634 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6270003c10 msgr2=0x7f628c144c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:07.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.592+0000 7f6283e00640 1 --2- 172.21.15.39:0/923614634 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6270003c10 0x7f628c144c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:07.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.592+0000 7f6283e00640 1 -- 172.21.15.39:0/923614634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f628c0b4620 msgr2=0x7f628c0b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:07.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.592+0000 7f6283e00640 1 --2- 172.21.15.39:0/923614634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f628c0b4620 0x7f628c0b4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:07.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.592+0000 7f6283e00640 1 -- 172.21.15.39:0/923614634 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f628c0b4f70 con 0x7f628c147740 2024-09-21T07:40:07.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.592+0000 7f6289e00640 1 --2- 172.21.15.39:0/923614634 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6270003c10 0x7f628c144c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:40:07.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.592+0000 7f6283e00640 1 --2- 172.21.15.39:0/923614634 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f628c147740 0x7f628c147b50 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f6274011a80 tx=0x7f6274011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:07.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.593+0000 7f6282000640 1 -- 172.21.15.39:0/923614634 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627400fe10 con 0x7f628c147740 2024-09-21T07:40:07.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.593+0000 7f6282000640 1 -- 172.21.15.39:0/923614634 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6274016070 con 0x7f628c147740 2024-09-21T07:40:07.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.593+0000 7f6290a00640 1 -- 172.21.15.39:0/923614634 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f628c1f9d60 con 0x7f628c147740 2024-09-21T07:40:07.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.593+0000 7f6282000640 1 -- 172.21.15.39:0/923614634 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6274012690 con 0x7f628c147740 2024-09-21T07:40:07.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.593+0000 7f6290a00640 1 -- 172.21.15.39:0/923614634 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f628c1f9280 con 0x7f628c147740 2024-09-21T07:40:07.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.594+0000 7f6290a00640 1 -- 172.21.15.39:0/923614634 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f628c073ca0 con 0x7f628c147740 2024-09-21T07:40:07.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.595+0000 7f6282000640 1 -- 172.21.15.39:0/923614634 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6274018030 con 0x7f628c147740 2024-09-21T07:40:07.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.596+0000 7f6282000640 1 --2- 172.21.15.39:0/923614634 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6268078b90 0x7f626807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:07.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.596+0000 7f6282000640 1 -- 172.21.15.39:0/923614634 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 7474+0+0 (secure 0 0 0) 0x7f627409bdb0 con 0x7f628c147740 2024-09-21T07:40:07.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.597+0000 7f6289e00640 1 --2- 172.21.15.39:0/923614634 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6268078b90 0x7f626807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:07.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.597+0000 7f6289e00640 1 --2- 172.21.15.39:0/923614634 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6268078b90 0x7f626807b0a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f627c032000 tx=0x7f627c001360 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:07.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.601+0000 7f6282000640 1 -- 172.21.15.39:0/923614634 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f628c0b4f70 con 0x7f628c147740 2024-09-21T07:40:07.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.848+0000 7f6290a00640 1 -- 172.21.15.39:0/923614634 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f628c06d8c0 con 0x7f628c147740 2024-09-21T07:40:07.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.848+0000 7f6282000640 1 -- 172.21.15.39:0/923614634 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) ==== 95+0+5253 (secure 0 0 0) 0x7f6274063b00 con 0x7f628c147740 2024-09-21T07:40:07.849 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:40:07.849 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":108,"btime":"2024-09-21T07:40:01:788092+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6269,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2336112936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2336112936},{"type":"v1","addr":"172.21.15.148:6837","nonce":2336112936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:40:00.809155+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7113,"mds_1":7080},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7080":{"gid":7080,"name":"c","rank":1,"incarnation":98,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6833/1456343023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1456343023},{"type":"v1","addr":"172.21.15.39:6833","nonce":1456343023}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7113":{"gid":7113,"name":"a","rank":0,"incarnation":107,"state":"up:resolve","state_seq":13,"addr":"172.21.15.39:6835/2821446711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2821446711},{"type":"v1","addr":"172.21.15.39:6835","nonce":2821446711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7080,"qdb_cluster":[7080]},"id":6}]} 2024-09-21T07:40:07.850 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 108 2024-09-21T07:40:07.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.851+0000 7f6290a00640 1 -- 172.21.15.39:0/923614634 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6268078b90 msgr2=0x7f626807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:07.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.851+0000 7f6290a00640 1 --2- 172.21.15.39:0/923614634 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6268078b90 0x7f626807b0a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f627c032000 tx=0x7f627c001360 comp rx=0 tx=0).stop 2024-09-21T07:40:07.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.851+0000 7f6290a00640 1 -- 172.21.15.39:0/923614634 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f628c147740 msgr2=0x7f628c147b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:07.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.851+0000 7f6290a00640 1 --2- 172.21.15.39:0/923614634 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f628c147740 0x7f628c147b50 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f6274011a80 tx=0x7f6274011f40 comp rx=0 tx=0).stop 2024-09-21T07:40:07.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.851+0000 7f6290a00640 1 -- 172.21.15.39:0/923614634 shutdown_connections 2024-09-21T07:40:07.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.851+0000 7f6290a00640 1 --2- 172.21.15.39:0/923614634 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6268078b90 0x7f626807b0a0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:07.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.851+0000 7f6290a00640 1 --2- 172.21.15.39:0/923614634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f628c0b4620 0x7f628c0b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:07.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.851+0000 7f6290a00640 1 --2- 172.21.15.39:0/923614634 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f628c147740 0x7f628c147b50 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:07.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.851+0000 7f6290a00640 1 --2- 172.21.15.39:0/923614634 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6270003c10 0x7f628c144c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:07.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.851+0000 7f6290a00640 1 -- 172.21.15.39:0/923614634 >> 172.21.15.39:0/923614634 conn(0x7f628c06cfc0 msgr2=0x7f628c15de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:07.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.851+0000 7f6290a00640 1 -- 172.21.15.39:0/923614634 shutdown_connections 2024-09-21T07:40:07.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:07.851+0000 7f6290a00640 1 -- 172.21.15.39:0/923614634 wait complete. 2024-09-21T07:40:07.878 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump snaps --server 2024-09-21T07:40:08.124 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_snap":11,"last_created":5,"last_destroyed":9,"pending_noop":[],"snaps":[],"need_to_purge":{},"pending_update":[{"version":21,"snapinfo":{"snapid":11,"ino":1099511627777,"stamp":"2024-09-21T07:39:42.151005+0000","name":"s25","metadata":{}}}],"pending_destroy":[]} 2024-09-21T07:40:08.125 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 5, "last_destroyed": 9, "last_snap": 11, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [ { "snapInfinityo": { "ino": 1099511627777, "metadata": {}, "name": "s25", "snapid": 11, "stamp": "2024-09-21T07:39:42.151005+0000" }, "version": 21 } ], "snaps": [] } 2024-09-21T07:40:08.125 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 6:1 2024-09-21T07:40:08.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.351+0000 7fc210c00640 1 -- 172.21.15.39:0/3477657193 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1f0004800 msgr2=0x7fc1f0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:08.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.351+0000 7fc210c00640 1 --2- 172.21.15.39:0/3477657193 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1f0004800 0x7fc1f0004c10 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fc20c073100 tx=0x7fc1f0005830 comp rx=0 tx=0).stop 2024-09-21T07:40:08.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.351+0000 7fc210c00640 1 -- 172.21.15.39:0/3477657193 shutdown_connections 2024-09-21T07:40:08.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.351+0000 7fc210c00640 1 --2- 172.21.15.39:0/3477657193 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1f0004800 0x7fc1f0004c10 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:08.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.351+0000 7fc210c00640 1 -- 172.21.15.39:0/3477657193 >> 172.21.15.39:0/3477657193 conn(0x7fc20c06cfc0 msgr2=0x7fc20c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:08.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.351+0000 7fc210c00640 1 -- 172.21.15.39:0/3477657193 shutdown_connections 2024-09-21T07:40:08.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.351+0000 7fc210c00640 1 -- 172.21.15.39:0/3477657193 wait complete. 2024-09-21T07:40:08.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.354+0000 7fc210c00640 1 Processor -- start 2024-09-21T07:40:08.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.354+0000 7fc210c00640 1 -- start start 2024-09-21T07:40:08.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.355+0000 7fc210c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc1f0004800 0x7fc20c13f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:08.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.355+0000 7fc210c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc20c150f10 0x7fc20c142880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:08.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.355+0000 7fc210c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc20c1f9670 0x7fc20c1e92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:08.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.355+0000 7fc210c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c06f800 con 0x7fc20c150f10 2024-09-21T07:40:08.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.355+0000 7fc210c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c0d0e90 con 0x7fc20c1f9670 2024-09-21T07:40:08.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.355+0000 7fc210c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc20c06f980 con 0x7fc1f0004800 2024-09-21T07:40:08.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.355+0000 7fc201400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc20c150f10 0x7fc20c142880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:08.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.355+0000 7fc201400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc20c150f10 0x7fc20c142880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48898/0 (socket says 172.21.15.39:48898) 2024-09-21T07:40:08.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.355+0000 7fc201400640 1 -- 172.21.15.39:0/2720878788 learned_addr learned my addr 172.21.15.39:0/2720878788 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:40:08.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.356+0000 7fc209e00640 1 --2- 172.21.15.39:0/2720878788 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc1f0004800 0x7fc20c13f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:08.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.356+0000 7fc20a800640 1 --2- 172.21.15.39:0/2720878788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc20c1f9670 0x7fc20c1e92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:08.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.356+0000 7fc201400640 1 -- 172.21.15.39:0/2720878788 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc1f0004800 msgr2=0x7fc20c13f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:08.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.356+0000 7fc201400640 1 --2- 172.21.15.39:0/2720878788 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc1f0004800 0x7fc20c13f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:08.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.356+0000 7fc201400640 1 -- 172.21.15.39:0/2720878788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc20c1f9670 msgr2=0x7fc20c1e92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:08.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.356+0000 7fc201400640 1 --2- 172.21.15.39:0/2720878788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc20c1f9670 0x7fc20c1e92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:08.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.356+0000 7fc201400640 1 -- 172.21.15.39:0/2720878788 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc20c151a00 con 0x7fc20c150f10 2024-09-21T07:40:08.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.356+0000 7fc201400640 1 --2- 172.21.15.39:0/2720878788 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc20c150f10 0x7fc20c142880 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4036040 tx=0x7fc1f40336f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:08.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.357+0000 7fc202a00640 1 -- 172.21.15.39:0/2720878788 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4038040 con 0x7fc20c150f10 2024-09-21T07:40:08.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.357+0000 7fc210c00640 1 -- 172.21.15.39:0/2720878788 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc20c151ce0 con 0x7fc20c150f10 2024-09-21T07:40:08.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.357+0000 7fc202a00640 1 -- 172.21.15.39:0/2720878788 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1f40339d0 con 0x7fc20c150f10 2024-09-21T07:40:08.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.357+0000 7fc202a00640 1 -- 172.21.15.39:0/2720878788 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4033e20 con 0x7fc20c150f10 2024-09-21T07:40:08.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.357+0000 7fc210c00640 1 -- 172.21.15.39:0/2720878788 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc20c142360 con 0x7fc20c150f10 2024-09-21T07:40:08.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.358+0000 7fc210c00640 1 -- 172.21.15.39:0/2720878788 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc20c151f20 con 0x7fc20c150f10 2024-09-21T07:40:08.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.359+0000 7fc202a00640 1 -- 172.21.15.39:0/2720878788 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc1f4043020 con 0x7fc20c150f10 2024-09-21T07:40:08.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.361+0000 7fc202a00640 1 --2- 172.21.15.39:0/2720878788 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc1e8078b90 0x7fc1e807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:08.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.361+0000 7fc202a00640 1 -- 172.21.15.39:0/2720878788 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 7474+0+0 (secure 0 0 0) 0x7fc1f40bb590 con 0x7fc20c150f10 2024-09-21T07:40:08.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.361+0000 7fc209e00640 1 --2- 172.21.15.39:0/2720878788 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc1e8078b90 0x7fc1e807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:08.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.362+0000 7fc209e00640 1 --2- 172.21.15.39:0/2720878788 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc1e8078b90 0x7fc1e807b0a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc006fd0 tx=0x7fc1fc006d50 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:08.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.365+0000 7fc202a00640 1 -- 172.21.15.39:0/2720878788 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc1f404a020 con 0x7fc20c150f10 2024-09-21T07:40:08.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.636+0000 7fc210c00640 1 -- 172.21.15.39:0/2720878788 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "6:1"} v 0) -- 0x7fc20c06d550 con 0x7fc20c150f10 2024-09-21T07:40:08.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.642+0000 7fc202a00640 1 -- 172.21.15.39:0/2720878788 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "6:1"}]=0 failed mds gid 7080 v109) ==== 97+0+0 (secure 0 0 0) 0x7fc1f4082c90 con 0x7fc20c150f10 2024-09-21T07:40:08.644 INFO:teuthology.orchestra.run.smithi039.stderr:failed mds gid 7080 2024-09-21T07:40:08.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.645+0000 7fc210c00640 1 -- 172.21.15.39:0/2720878788 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc1e8078b90 msgr2=0x7fc1e807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:08.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.645+0000 7fc210c00640 1 --2- 172.21.15.39:0/2720878788 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc1e8078b90 0x7fc1e807b0a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc006fd0 tx=0x7fc1fc006d50 comp rx=0 tx=0).stop 2024-09-21T07:40:08.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.645+0000 7fc210c00640 1 -- 172.21.15.39:0/2720878788 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc20c150f10 msgr2=0x7fc20c142880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:08.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.645+0000 7fc210c00640 1 --2- 172.21.15.39:0/2720878788 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc20c150f10 0x7fc20c142880 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4036040 tx=0x7fc1f40336f0 comp rx=0 tx=0).stop 2024-09-21T07:40:08.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.645+0000 7fc210c00640 1 -- 172.21.15.39:0/2720878788 shutdown_connections 2024-09-21T07:40:08.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.645+0000 7fc210c00640 1 --2- 172.21.15.39:0/2720878788 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc1e8078b90 0x7fc1e807b0a0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:08.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.645+0000 7fc210c00640 1 --2- 172.21.15.39:0/2720878788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc20c1f9670 0x7fc20c1e92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:08.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.645+0000 7fc210c00640 1 --2- 172.21.15.39:0/2720878788 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc20c150f10 0x7fc20c142880 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:08.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.645+0000 7fc210c00640 1 --2- 172.21.15.39:0/2720878788 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc1f0004800 0x7fc20c13f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:08.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.645+0000 7fc210c00640 1 -- 172.21.15.39:0/2720878788 >> 172.21.15.39:0/2720878788 conn(0x7fc20c06cfc0 msgr2=0x7fc20c0b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:08.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.645+0000 7fc210c00640 1 -- 172.21.15.39:0/2720878788 shutdown_connections 2024-09-21T07:40:08.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.645+0000 7fc210c00640 1 -- 172.21.15.39:0/2720878788 wait complete. 2024-09-21T07:40:08.673 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T07:40:08.673 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T07:40:08.673 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T07:40:08.673 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:40:08.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:40:08.673 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/daemon/state.py", line 146, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi039 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c' 2024-09-21T07:40:08.674 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T07:40:08.674 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T07:40:08.678 INFO:tasks.ceph.mds.c:Started 2024-09-21T07:40:08.678 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:40:08.874 INFO:tasks.ceph.mds.c.smithi039.stdout:starting mds.c at 2024-09-21T07:40:08.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.904+0000 7fc239800640 1 -- 172.21.15.39:0/533003392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc23406fc10 msgr2=0x7fc2340d0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:08.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.904+0000 7fc239800640 1 --2- 172.21.15.39:0/533003392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc23406fc10 0x7fc2340d0620 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fc22c005fd0 tx=0x7fc22c02a8b0 comp rx=0 tx=0).stop 2024-09-21T07:40:08.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.904+0000 7fc239800640 1 -- 172.21.15.39:0/533003392 shutdown_connections 2024-09-21T07:40:08.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.905+0000 7fc239800640 1 --2- 172.21.15.39:0/533003392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc23406fc10 0x7fc2340d0620 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:08.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.905+0000 7fc239800640 1 --2- 172.21.15.39:0/533003392 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc23415cd30 0x7fc2340cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:08.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.905+0000 7fc239800640 1 -- 172.21.15.39:0/533003392 >> 172.21.15.39:0/533003392 conn(0x7fc23406cfc0 msgr2=0x7fc23406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:08.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.905+0000 7fc239800640 1 -- 172.21.15.39:0/533003392 shutdown_connections 2024-09-21T07:40:08.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.905+0000 7fc239800640 1 -- 172.21.15.39:0/533003392 wait complete. 2024-09-21T07:40:08.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.907+0000 7fc239800640 1 Processor -- start 2024-09-21T07:40:08.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.907+0000 7fc239800640 1 -- start start 2024-09-21T07:40:08.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.908+0000 7fc239800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc23406f800 0x7fc2341f9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:08.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.908+0000 7fc239800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc23406fc10 0x7fc2340b1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:08.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.908+0000 7fc239800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc23415cd30 0x7fc234145bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:08.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.908+0000 7fc239800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc234073890 con 0x7fc23406f800 2024-09-21T07:40:08.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.908+0000 7fc239800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc234073710 con 0x7fc23406fc10 2024-09-21T07:40:08.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.908+0000 7fc239800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc234073a10 con 0x7fc23415cd30 2024-09-21T07:40:08.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.909+0000 7fc232a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc23406f800 0x7fc2341f9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:08.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.909+0000 7fc232a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc23406f800 0x7fc2341f9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48918/0 (socket says 172.21.15.39:48918) 2024-09-21T07:40:08.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.909+0000 7fc232a00640 1 -- 172.21.15.39:0/2072706092 learned_addr learned my addr 172.21.15.39:0/2072706092 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:40:08.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.909+0000 7fc233400640 1 --2- 172.21.15.39:0/2072706092 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc23415cd30 0x7fc234145bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:08.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.909+0000 7fc22be00640 1 --2- 172.21.15.39:0/2072706092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc23406fc10 0x7fc2340b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:08.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.909+0000 7fc232a00640 1 -- 172.21.15.39:0/2072706092 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc23415cd30 msgr2=0x7fc234145bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:08.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.909+0000 7fc232a00640 1 --2- 172.21.15.39:0/2072706092 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc23415cd30 0x7fc234145bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:08.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.909+0000 7fc232a00640 1 -- 172.21.15.39:0/2072706092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc23406fc10 msgr2=0x7fc2340b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:08.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.909+0000 7fc232a00640 1 --2- 172.21.15.39:0/2072706092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc23406fc10 0x7fc2340b1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:08.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.909+0000 7fc232a00640 1 -- 172.21.15.39:0/2072706092 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2341f6150 con 0x7fc23406f800 2024-09-21T07:40:08.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.909+0000 7fc22be00640 1 --2- 172.21.15.39:0/2072706092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc23406fc10 0x7fc2340b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:40:08.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.909+0000 7fc232a00640 1 --2- 172.21.15.39:0/2072706092 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc23406f800 0x7fc2341f9280 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fc22400e790 tx=0x7fc22400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:08.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.910+0000 7fc22aa00640 1 -- 172.21.15.39:0/2072706092 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22400eea0 con 0x7fc23406f800 2024-09-21T07:40:08.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.910+0000 7fc22aa00640 1 -- 172.21.15.39:0/2072706092 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc22400cdb0 con 0x7fc23406f800 2024-09-21T07:40:08.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.910+0000 7fc22aa00640 1 -- 172.21.15.39:0/2072706092 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc224010690 con 0x7fc23406f800 2024-09-21T07:40:08.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.910+0000 7fc239800640 1 -- 172.21.15.39:0/2072706092 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2341f5390 con 0x7fc23406f800 2024-09-21T07:40:08.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.910+0000 7fc239800640 1 -- 172.21.15.39:0/2072706092 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2341f5950 con 0x7fc23406f800 2024-09-21T07:40:08.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.911+0000 7fc239800640 1 -- 172.21.15.39:0/2072706092 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc234073e70 con 0x7fc23406f800 2024-09-21T07:40:08.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.912+0000 7fc22aa00640 1 -- 172.21.15.39:0/2072706092 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc224010830 con 0x7fc23406f800 2024-09-21T07:40:08.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.914+0000 7fc22aa00640 1 --2- 172.21.15.39:0/2072706092 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc210078b90 0x7fc21007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:08.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.914+0000 7fc22aa00640 1 -- 172.21.15.39:0/2072706092 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 7560+0+0 (secure 0 0 0) 0x7fc22409be50 con 0x7fc23406f800 2024-09-21T07:40:08.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.914+0000 7fc22be00640 1 --2- 172.21.15.39:0/2072706092 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc210078b90 0x7fc21007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:08.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.915+0000 7fc22be00640 1 --2- 172.21.15.39:0/2072706092 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc210078b90 0x7fc21007b0a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc2340b4d10 tx=0x7fc21c009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:08.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:08.918+0000 7fc22aa00640 1 -- 172.21.15.39:0/2072706092 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc224063bd0 con 0x7fc23406f800 2024-09-21T07:40:09.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:09.155+0000 7fc239800640 1 -- 172.21.15.39:0/2072706092 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2341f5d30 con 0x7fc23406f800 2024-09-21T07:40:09.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:09.156+0000 7fc22aa00640 1 -- 172.21.15.39:0/2072706092 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) ==== 95+0+4421 (secure 0 0 0) 0x7fc2240634d0 con 0x7fc23406f800 2024-09-21T07:40:09.158 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:40:09.158 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":110,"btime":"2024-09-21T07:40:08:651134+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:40:08.651117+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7113,"mds_1":6269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6269":{"gid":6269,"name":"d","rank":1,"incarnation":110,"state":"up:replay","state_seq":1,"addr":"172.21.15.148:6837/2336112936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2336112936},{"type":"v1","addr":"172.21.15.148:6837","nonce":2336112936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7113":{"gid":7113,"name":"a","rank":0,"incarnation":107,"state":"up:resolve","state_seq":13,"addr":"172.21.15.39:6835/2821446711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2821446711},{"type":"v1","addr":"172.21.15.39:6835","nonce":2821446711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T07:40:09.159 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 110 2024-09-21T07:40:09.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:09.158+0000 7fc239800640 1 -- 172.21.15.39:0/2072706092 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc210078b90 msgr2=0x7fc21007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:09.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:09.158+0000 7fc239800640 1 --2- 172.21.15.39:0/2072706092 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc210078b90 0x7fc21007b0a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc2340b4d10 tx=0x7fc21c009040 comp rx=0 tx=0).stop 2024-09-21T07:40:09.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:09.158+0000 7fc239800640 1 -- 172.21.15.39:0/2072706092 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc23406f800 msgr2=0x7fc2341f9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:09.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:09.158+0000 7fc239800640 1 --2- 172.21.15.39:0/2072706092 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc23406f800 0x7fc2341f9280 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fc22400e790 tx=0x7fc22400ec50 comp rx=0 tx=0).stop 2024-09-21T07:40:09.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:09.158+0000 7fc239800640 1 -- 172.21.15.39:0/2072706092 shutdown_connections 2024-09-21T07:40:09.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:09.158+0000 7fc239800640 1 --2- 172.21.15.39:0/2072706092 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc210078b90 0x7fc21007b0a0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:09.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:09.158+0000 7fc239800640 1 --2- 172.21.15.39:0/2072706092 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc23415cd30 0x7fc234145bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:09.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:09.158+0000 7fc239800640 1 --2- 172.21.15.39:0/2072706092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc23406fc10 0x7fc2340b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:09.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:09.158+0000 7fc239800640 1 --2- 172.21.15.39:0/2072706092 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc23406f800 0x7fc2341f9280 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:09.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:09.158+0000 7fc239800640 1 -- 172.21.15.39:0/2072706092 >> 172.21.15.39:0/2072706092 conn(0x7fc23406cfc0 msgr2=0x7fc23415e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:09.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:09.159+0000 7fc239800640 1 -- 172.21.15.39:0/2072706092 shutdown_connections 2024-09-21T07:40:09.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:09.159+0000 7fc239800640 1 -- 172.21.15.39:0/2072706092 wait complete. 2024-09-21T07:40:09.185 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-09-21T07:40:09.373 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-09-21T07:40:14.186 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:40:14.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.422+0000 7fddf7200640 1 -- 172.21.15.39:0/3330502510 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddf80cc390 msgr2=0x7fddf80af600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:14.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.422+0000 7fddf7200640 1 --2- 172.21.15.39:0/3330502510 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddf80cc390 0x7fddf80af600 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fddf80d1340 tx=0x7fdde802a4d0 comp rx=0 tx=0).stop 2024-09-21T07:40:14.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.422+0000 7fddf7200640 1 -- 172.21.15.39:0/3330502510 shutdown_connections 2024-09-21T07:40:14.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.422+0000 7fddf7200640 1 --2- 172.21.15.39:0/3330502510 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fddf806f000 0x7fddf806f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:14.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.422+0000 7fddf7200640 1 --2- 172.21.15.39:0/3330502510 >> v2:172.21.15.148:3300/0 conn(0x7fddf80ce310 0x7fddf80ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:14.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.422+0000 7fddf7200640 1 --2- 172.21.15.39:0/3330502510 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddf80cc390 0x7fddf80af600 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:14.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.422+0000 7fddf7200640 1 -- 172.21.15.39:0/3330502510 >> 172.21.15.39:0/3330502510 conn(0x7fddf80b2510 msgr2=0x7fddf80d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:14.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.423+0000 7fddf7200640 1 -- 172.21.15.39:0/3330502510 shutdown_connections 2024-09-21T07:40:14.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.423+0000 7fddf7200640 1 -- 172.21.15.39:0/3330502510 wait complete. 2024-09-21T07:40:14.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.425+0000 7fddf7200640 1 Processor -- start 2024-09-21T07:40:14.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.425+0000 7fddf7200640 1 -- start start 2024-09-21T07:40:14.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.426+0000 7fddf7200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddf806f000 0x7fddf80997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:14.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.426+0000 7fddf7200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fddf80cc390 0x7fddf815a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:14.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.426+0000 7fddf7200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fddf80ce310 0x7fddf815a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:14.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.426+0000 7fddf7200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fddf806fe80 con 0x7fddf806f000 2024-09-21T07:40:14.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.426+0000 7fddf7200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fddf806fd00 con 0x7fddf80cc390 2024-09-21T07:40:14.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.426+0000 7fddf7200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fddf8070000 con 0x7fddf80ce310 2024-09-21T07:40:14.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.427+0000 7fddf5e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddf806f000 0x7fddf80997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:14.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.427+0000 7fddf5e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddf806f000 0x7fddf80997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35468/0 (socket says 172.21.15.39:35468) 2024-09-21T07:40:14.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.427+0000 7fddf5e00640 1 -- 172.21.15.39:0/1205688304 learned_addr learned my addr 172.21.15.39:0/1205688304 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:40:14.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.427+0000 7fddf5e00640 1 -- 172.21.15.39:0/1205688304 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fddf80ce310 msgr2=0x7fddf815a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:14.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.427+0000 7fddf5e00640 1 --2- 172.21.15.39:0/1205688304 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fddf80ce310 0x7fddf815a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:14.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.427+0000 7fddf5e00640 1 -- 172.21.15.39:0/1205688304 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fddf80cc390 msgr2=0x7fddf815a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:14.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.427+0000 7fddf5e00640 1 --2- 172.21.15.39:0/1205688304 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fddf80cc390 0x7fddf815a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:14.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.427+0000 7fddf5e00640 1 -- 172.21.15.39:0/1205688304 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf81f81b0 con 0x7fddf806f000 2024-09-21T07:40:14.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.427+0000 7fddf5e00640 1 --2- 172.21.15.39:0/1205688304 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddf806f000 0x7fddf80997b0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fdde8034040 tx=0x7fdde8003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:14.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.428+0000 7fddee000640 1 -- 172.21.15.39:0/1205688304 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde803e030 con 0x7fddf806f000 2024-09-21T07:40:14.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.428+0000 7fddee000640 1 -- 172.21.15.39:0/1205688304 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdde8003d10 con 0x7fddf806f000 2024-09-21T07:40:14.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.428+0000 7fddee000640 1 -- 172.21.15.39:0/1205688304 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde8035040 con 0x7fddf806f000 2024-09-21T07:40:14.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.428+0000 7fddf7200640 1 -- 172.21.15.39:0/1205688304 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddf81f7410 con 0x7fddf806f000 2024-09-21T07:40:14.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.428+0000 7fddf7200640 1 -- 172.21.15.39:0/1205688304 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddf81f79b0 con 0x7fddf806f000 2024-09-21T07:40:14.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.429+0000 7fddf7200640 1 -- 172.21.15.39:0/1205688304 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddf80af600 con 0x7fddf806f000 2024-09-21T07:40:14.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.430+0000 7fddee000640 1 -- 172.21.15.39:0/1205688304 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdde8041030 con 0x7fddf806f000 2024-09-21T07:40:14.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.431+0000 7fddee000640 1 --2- 172.21.15.39:0/1205688304 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fddd8078b90 0x7fddd807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:14.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.431+0000 7fddee000640 1 -- 172.21.15.39:0/1205688304 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 7560+0+0 (secure 0 0 0) 0x7fdde80bbd00 con 0x7fddf806f000 2024-09-21T07:40:14.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.432+0000 7fddf5400640 1 --2- 172.21.15.39:0/1205688304 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fddd8078b90 0x7fddd807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:14.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.432+0000 7fddf5400640 1 --2- 172.21.15.39:0/1205688304 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fddd8078b90 0x7fddd807b0a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fddf80c6dc0 tx=0x7fdde4005ce0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:14.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.436+0000 7fddee000640 1 -- 172.21.15.39:0/1205688304 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fddf80cec60 con 0x7fddf806f000 2024-09-21T07:40:14.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.670+0000 7fddf7200640 1 -- 172.21.15.39:0/1205688304 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fddf80b0e20 con 0x7fddf806f000 2024-09-21T07:40:14.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.671+0000 7fddee000640 1 -- 172.21.15.39:0/1205688304 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5252 (secure 0 0 0) 0x7fdde8083960 con 0x7fddf806f000 2024-09-21T07:40:14.673 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:40:14.673 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":112,"btime":"2024-09-21T07:40:10:657658+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7311,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:40:09.661903+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7113,"mds_1":6269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6269":{"gid":6269,"name":"d","rank":1,"incarnation":110,"state":"up:reconnect","state_seq":4,"addr":"172.21.15.148:6837/2336112936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2336112936},{"type":"v1","addr":"172.21.15.148:6837","nonce":2336112936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7113":{"gid":7113,"name":"a","rank":0,"incarnation":107,"state":"up:reconnect","state_seq":16,"addr":"172.21.15.39:6835/2821446711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2821446711},{"type":"v1","addr":"172.21.15.39:6835","nonce":2821446711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T07:40:14.674 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 112 2024-09-21T07:40:14.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.674+0000 7fddf7200640 1 -- 172.21.15.39:0/1205688304 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fddd8078b90 msgr2=0x7fddd807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:14.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.674+0000 7fddf7200640 1 --2- 172.21.15.39:0/1205688304 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fddd8078b90 0x7fddd807b0a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fddf80c6dc0 tx=0x7fdde4005ce0 comp rx=0 tx=0).stop 2024-09-21T07:40:14.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.674+0000 7fddf7200640 1 -- 172.21.15.39:0/1205688304 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddf806f000 msgr2=0x7fddf80997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:14.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.674+0000 7fddf7200640 1 --2- 172.21.15.39:0/1205688304 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddf806f000 0x7fddf80997b0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fdde8034040 tx=0x7fdde8003820 comp rx=0 tx=0).stop 2024-09-21T07:40:14.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.674+0000 7fddf7200640 1 -- 172.21.15.39:0/1205688304 shutdown_connections 2024-09-21T07:40:14.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.674+0000 7fddf7200640 1 --2- 172.21.15.39:0/1205688304 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fddd8078b90 0x7fddd807b0a0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:14.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.674+0000 7fddf7200640 1 --2- 172.21.15.39:0/1205688304 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fddf80ce310 0x7fddf815a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:14.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.674+0000 7fddf7200640 1 --2- 172.21.15.39:0/1205688304 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fddf80cc390 0x7fddf815a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:14.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.674+0000 7fddf7200640 1 --2- 172.21.15.39:0/1205688304 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddf806f000 0x7fddf80997b0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:14.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.674+0000 7fddf7200640 1 -- 172.21.15.39:0/1205688304 >> 172.21.15.39:0/1205688304 conn(0x7fddf80b2510 msgr2=0x7fddf80afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:14.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.674+0000 7fddf7200640 1 -- 172.21.15.39:0/1205688304 shutdown_connections 2024-09-21T07:40:14.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.674+0000 7fddf7200640 1 -- 172.21.15.39:0/1205688304 wait complete. 2024-09-21T07:40:14.702 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-21T07:40:14.703 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:40:14.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.929+0000 7fd6aaa00640 1 -- 172.21.15.39:0/3023591959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd688004d90 msgr2=0x7fd688005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:14.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.929+0000 7fd6aaa00640 1 --2- 172.21.15.39:0/3023591959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd688004d90 0x7fd688005220 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fd69c005fd0 tx=0x7fd688005830 comp rx=0 tx=0).stop 2024-09-21T07:40:14.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.929+0000 7fd6a8a00640 1 -- 172.21.15.39:0/3023591959 reap_dead start 2024-09-21T07:40:14.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.929+0000 7fd6aaa00640 1 -- 172.21.15.39:0/3023591959 shutdown_connections 2024-09-21T07:40:14.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.929+0000 7fd6aaa00640 1 -- 172.21.15.39:0/3023591959 >> 172.21.15.39:0/3023591959 conn(0x7fd6a406cfc0 msgr2=0x7fd6a406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:14.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.929+0000 7fd6aaa00640 1 -- 172.21.15.39:0/3023591959 shutdown_connections 2024-09-21T07:40:14.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.929+0000 7fd6aaa00640 1 -- 172.21.15.39:0/3023591959 wait complete. 2024-09-21T07:40:14.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.932+0000 7fd6aaa00640 1 Processor -- start 2024-09-21T07:40:14.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.932+0000 7fd6aaa00640 1 -- start start 2024-09-21T07:40:14.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.933+0000 7fd6aaa00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd688004d90 0x7fd6a4159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:14.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.933+0000 7fd6aaa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6a4158950 0x7fd6a40ae160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:14.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.933+0000 7fd6aaa00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd6a415c740 0x7fd6a40ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:14.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.933+0000 7fd6aaa00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a40d0ed0 con 0x7fd6a4158950 2024-09-21T07:40:14.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.933+0000 7fd6aaa00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a40d0d50 con 0x7fd688004d90 2024-09-21T07:40:14.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.933+0000 7fd6aaa00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd6a4073710 con 0x7fd6a415c740 2024-09-21T07:40:14.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.934+0000 7fd69be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6a4158950 0x7fd6a40ae160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:14.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.934+0000 7fd69be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6a4158950 0x7fd6a40ae160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35496/0 (socket says 172.21.15.39:35496) 2024-09-21T07:40:14.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.934+0000 7fd69be00640 1 -- 172.21.15.39:0/69636162 learned_addr learned my addr 172.21.15.39:0/69636162 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:40:14.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.934+0000 7fd6a3e00640 1 --2- 172.21.15.39:0/69636162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd688004d90 0x7fd6a4159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:14.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.934+0000 7fd6a8a00640 1 --2- 172.21.15.39:0/69636162 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd6a415c740 0x7fd6a40ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:14.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.934+0000 7fd69be00640 1 -- 172.21.15.39:0/69636162 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd6a415c740 msgr2=0x7fd6a40ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:14.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.934+0000 7fd69be00640 1 --2- 172.21.15.39:0/69636162 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd6a415c740 0x7fd6a40ae6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:14.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.934+0000 7fd69be00640 1 -- 172.21.15.39:0/69636162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd688004d90 msgr2=0x7fd6a4159ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:14.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.934+0000 7fd69be00640 1 --2- 172.21.15.39:0/69636162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd688004d90 0x7fd6a4159ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:14.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.934+0000 7fd69be00640 1 -- 172.21.15.39:0/69636162 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a415b700 con 0x7fd6a4158950 2024-09-21T07:40:14.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.934+0000 7fd6a8a00640 1 --2- 172.21.15.39:0/69636162 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd6a415c740 0x7fd6a40ae6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:40:14.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.934+0000 7fd6a3e00640 1 --2- 172.21.15.39:0/69636162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd688004d90 0x7fd6a4159ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:40:14.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.934+0000 7fd69be00640 1 --2- 172.21.15.39:0/69636162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6a4158950 0x7fd6a40ae160 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fd68c00e5c0 tx=0x7fd68c00ea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:14.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.935+0000 7fd6a0c00640 1 -- 172.21.15.39:0/69636162 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c015070 con 0x7fd6a4158950 2024-09-21T07:40:14.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.935+0000 7fd6a0c00640 1 -- 172.21.15.39:0/69636162 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd68c00ca60 con 0x7fd6a4158950 2024-09-21T07:40:14.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.935+0000 7fd6aaa00640 1 -- 172.21.15.39:0/69636162 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6a415b980 con 0x7fd6a4158950 2024-09-21T07:40:14.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.935+0000 7fd6a0c00640 1 -- 172.21.15.39:0/69636162 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c011460 con 0x7fd6a4158950 2024-09-21T07:40:14.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.935+0000 7fd6aaa00640 1 -- 172.21.15.39:0/69636162 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6a415bf40 con 0x7fd6a4158950 2024-09-21T07:40:14.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.936+0000 7fd6aaa00640 1 -- 172.21.15.39:0/69636162 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6a4073c70 con 0x7fd6a4158950 2024-09-21T07:40:14.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.937+0000 7fd6a0c00640 1 -- 172.21.15.39:0/69636162 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd68c018070 con 0x7fd6a4158950 2024-09-21T07:40:14.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.938+0000 7fd6a0c00640 1 --2- 172.21.15.39:0/69636162 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd680078b90 0x7fd68007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:14.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.938+0000 7fd6a0c00640 1 -- 172.21.15.39:0/69636162 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 7560+0+0 (secure 0 0 0) 0x7fd68c09ba00 con 0x7fd6a4158950 2024-09-21T07:40:14.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.939+0000 7fd6a3e00640 1 --2- 172.21.15.39:0/69636162 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd680078b90 0x7fd68007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:14.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.939+0000 7fd6a3e00640 1 --2- 172.21.15.39:0/69636162 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd680078b90 0x7fd68007b0a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd694000c00 tx=0x7fd694000ed0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:14.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:14.942+0000 7fd6a0c00640 1 -- 172.21.15.39:0/69636162 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd6a415b700 con 0x7fd6a4158950 2024-09-21T07:40:15.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:15.174+0000 7fd6aaa00640 1 -- 172.21.15.39:0/69636162 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6a4158dc0 con 0x7fd6a4158950 2024-09-21T07:40:15.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:15.175+0000 7fd6a0c00640 1 -- 172.21.15.39:0/69636162 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5252 (secure 0 0 0) 0x7fd68c063660 con 0x7fd6a4158950 2024-09-21T07:40:15.177 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:40:15.177 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":112,"btime":"2024-09-21T07:40:10:657658+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7311,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:40:09.661903+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7113,"mds_1":6269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6269":{"gid":6269,"name":"d","rank":1,"incarnation":110,"state":"up:reconnect","state_seq":4,"addr":"172.21.15.148:6837/2336112936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2336112936},{"type":"v1","addr":"172.21.15.148:6837","nonce":2336112936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7113":{"gid":7113,"name":"a","rank":0,"incarnation":107,"state":"up:reconnect","state_seq":16,"addr":"172.21.15.39:6835/2821446711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2821446711},{"type":"v1","addr":"172.21.15.39:6835","nonce":2821446711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T07:40:15.177 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 112 2024-09-21T07:40:15.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:15.177+0000 7fd6aaa00640 1 -- 172.21.15.39:0/69636162 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd680078b90 msgr2=0x7fd68007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:15.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:15.177+0000 7fd6aaa00640 1 --2- 172.21.15.39:0/69636162 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd680078b90 0x7fd68007b0a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd694000c00 tx=0x7fd694000ed0 comp rx=0 tx=0).stop 2024-09-21T07:40:15.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:15.177+0000 7fd6aaa00640 1 -- 172.21.15.39:0/69636162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6a4158950 msgr2=0x7fd6a40ae160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:15.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:15.177+0000 7fd6aaa00640 1 --2- 172.21.15.39:0/69636162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6a4158950 0x7fd6a40ae160 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fd68c00e5c0 tx=0x7fd68c00ea80 comp rx=0 tx=0).stop 2024-09-21T07:40:15.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:15.177+0000 7fd6aaa00640 1 -- 172.21.15.39:0/69636162 shutdown_connections 2024-09-21T07:40:15.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:15.177+0000 7fd6aaa00640 1 --2- 172.21.15.39:0/69636162 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd680078b90 0x7fd68007b0a0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:15.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:15.177+0000 7fd6aaa00640 1 --2- 172.21.15.39:0/69636162 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd6a415c740 0x7fd6a40ae6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:15.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:15.177+0000 7fd6aaa00640 1 --2- 172.21.15.39:0/69636162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6a4158950 0x7fd6a40ae160 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:15.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:15.177+0000 7fd6aaa00640 1 --2- 172.21.15.39:0/69636162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd688004d90 0x7fd6a4159ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:15.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:15.177+0000 7fd6aaa00640 1 -- 172.21.15.39:0/69636162 >> 172.21.15.39:0/69636162 conn(0x7fd6a406cfc0 msgr2=0x7fd6a40afe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:15.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:15.178+0000 7fd6aaa00640 1 -- 172.21.15.39:0/69636162 shutdown_connections 2024-09-21T07:40:15.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:15.178+0000 7fd6aaa00640 1 -- 172.21.15.39:0/69636162 wait complete. 2024-09-21T07:40:15.206 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:reconnect 2024-09-21T07:40:16.207 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:40:16.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.441+0000 7f84a1800640 1 -- 172.21.15.39:0/236206292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f849c15f790 msgr2=0x7f849c0d06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:16.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.441+0000 7f84a1800640 1 --2- 172.21.15.39:0/236206292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f849c15f790 0x7f849c0d06e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8488009830 tx=0x7f848802a540 comp rx=0 tx=0).stop 2024-09-21T07:40:16.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.442+0000 7f84a1800640 1 -- 172.21.15.39:0/236206292 shutdown_connections 2024-09-21T07:40:16.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.442+0000 7f84a1800640 1 --2- 172.21.15.39:0/236206292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f849c15f790 0x7f849c0d06e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:16.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.442+0000 7f84a1800640 1 -- 172.21.15.39:0/236206292 >> 172.21.15.39:0/236206292 conn(0x7f849c06cfc0 msgr2=0x7f849c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:16.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.442+0000 7f84a1800640 1 -- 172.21.15.39:0/236206292 shutdown_connections 2024-09-21T07:40:16.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.442+0000 7f84a1800640 1 -- 172.21.15.39:0/236206292 wait complete. 2024-09-21T07:40:16.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.444+0000 7f84a1800640 1 Processor -- start 2024-09-21T07:40:16.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.444+0000 7f84a1800640 1 -- start start 2024-09-21T07:40:16.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.445+0000 7f84a1800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f849c15f790 0x7f849c1f5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:16.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.445+0000 7f84a1800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f849c15fba0 0x7f849c158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:16.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.445+0000 7f84a1800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f849c156070 0x7f849c1e81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:16.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.445+0000 7f84a1800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f849c0ae670 con 0x7f849c15fba0 2024-09-21T07:40:16.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.445+0000 7f84a1800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f849c0ae4f0 con 0x7f849c15f790 2024-09-21T07:40:16.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.445+0000 7f84a1800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f849c0ae7f0 con 0x7f849c156070 2024-09-21T07:40:16.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.446+0000 7f849a000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f849c15fba0 0x7f849c158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:16.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.446+0000 7f849a000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f849c15fba0 0x7f849c158d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35512/0 (socket says 172.21.15.39:35512) 2024-09-21T07:40:16.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.446+0000 7f849a000640 1 -- 172.21.15.39:0/155304195 learned_addr learned my addr 172.21.15.39:0/155304195 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:40:16.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.446+0000 7f849aa00640 1 --2- 172.21.15.39:0/155304195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f849c15f790 0x7f849c1f5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:16.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.446+0000 7f849b400640 1 --2- 172.21.15.39:0/155304195 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f849c156070 0x7f849c1e81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:16.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.446+0000 7f849a000640 1 -- 172.21.15.39:0/155304195 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f849c156070 msgr2=0x7f849c1e81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:16.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.446+0000 7f849a000640 1 --2- 172.21.15.39:0/155304195 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f849c156070 0x7f849c1e81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:16.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.446+0000 7f849a000640 1 -- 172.21.15.39:0/155304195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f849c15f790 msgr2=0x7f849c1f5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:16.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.446+0000 7f849a000640 1 --2- 172.21.15.39:0/155304195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f849c15f790 0x7f849c1f5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:16.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.446+0000 7f849a000640 1 -- 172.21.15.39:0/155304195 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f849c157c70 con 0x7f849c15fba0 2024-09-21T07:40:16.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.447+0000 7f849a000640 1 --2- 172.21.15.39:0/155304195 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f849c15fba0 0x7f849c158d10 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f8488009950 tx=0x7f8488003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:16.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.447+0000 7f8492a00640 1 -- 172.21.15.39:0/155304195 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8488005c90 con 0x7f849c15fba0 2024-09-21T07:40:16.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.447+0000 7f8492a00640 1 -- 172.21.15.39:0/155304195 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8488005020 con 0x7f849c15fba0 2024-09-21T07:40:16.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.447+0000 7f84a1800640 1 -- 172.21.15.39:0/155304195 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f849c157f50 con 0x7f849c15fba0 2024-09-21T07:40:16.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.447+0000 7f8492a00640 1 -- 172.21.15.39:0/155304195 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8488005450 con 0x7f849c15fba0 2024-09-21T07:40:16.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.447+0000 7f84a1800640 1 -- 172.21.15.39:0/155304195 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f849c158510 con 0x7f849c15fba0 2024-09-21T07:40:16.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.448+0000 7f84a1800640 1 -- 172.21.15.39:0/155304195 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f849c0b47c0 con 0x7f849c15fba0 2024-09-21T07:40:16.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.449+0000 7f8492a00640 1 -- 172.21.15.39:0/155304195 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f84880317c0 con 0x7f849c15fba0 2024-09-21T07:40:16.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.450+0000 7f8492a00640 1 --2- 172.21.15.39:0/155304195 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f847c078bc0 0x7f847c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:16.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.450+0000 7f8492a00640 1 -- 172.21.15.39:0/155304195 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7603+0+0 (secure 0 0 0) 0x7f84880bc570 con 0x7f849c15fba0 2024-09-21T07:40:16.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.451+0000 7f849aa00640 1 --2- 172.21.15.39:0/155304195 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f847c078bc0 0x7f847c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:16.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.452+0000 7f849aa00640 1 --2- 172.21.15.39:0/155304195 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f847c078bc0 0x7f847c07b0d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f848c000f60 tx=0x7f848c009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:16.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.455+0000 7f8492a00640 1 -- 172.21.15.39:0/155304195 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f848803f030 con 0x7f849c15fba0 2024-09-21T07:40:16.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.690+0000 7f84a1800640 1 -- 172.21.15.39:0/155304195 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f849c15df30 con 0x7f849c15fba0 2024-09-21T07:40:16.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.691+0000 7f8492a00640 1 -- 172.21.15.39:0/155304195 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5252 (secure 0 0 0) 0x7f8488083b60 con 0x7f849c15fba0 2024-09-21T07:40:16.693 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:40:16.693 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":112,"btime":"2024-09-21T07:40:10:657658+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7311,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:40:09.661903+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7113,"mds_1":6269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6269":{"gid":6269,"name":"d","rank":1,"incarnation":110,"state":"up:reconnect","state_seq":4,"addr":"172.21.15.148:6837/2336112936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2336112936},{"type":"v1","addr":"172.21.15.148:6837","nonce":2336112936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7113":{"gid":7113,"name":"a","rank":0,"incarnation":107,"state":"up:reconnect","state_seq":16,"addr":"172.21.15.39:6835/2821446711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2821446711},{"type":"v1","addr":"172.21.15.39:6835","nonce":2821446711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T07:40:16.693 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 112 2024-09-21T07:40:16.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.693+0000 7f84a1800640 1 -- 172.21.15.39:0/155304195 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f847c078bc0 msgr2=0x7f847c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:16.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.693+0000 7f84a1800640 1 --2- 172.21.15.39:0/155304195 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f847c078bc0 0x7f847c07b0d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f848c000f60 tx=0x7f848c009040 comp rx=0 tx=0).stop 2024-09-21T07:40:16.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.693+0000 7f84a1800640 1 -- 172.21.15.39:0/155304195 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f849c15fba0 msgr2=0x7f849c158d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:16.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.693+0000 7f84a1800640 1 --2- 172.21.15.39:0/155304195 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f849c15fba0 0x7f849c158d10 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f8488009950 tx=0x7f8488003c10 comp rx=0 tx=0).stop 2024-09-21T07:40:16.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.693+0000 7f84a1800640 1 -- 172.21.15.39:0/155304195 shutdown_connections 2024-09-21T07:40:16.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.693+0000 7f84a1800640 1 --2- 172.21.15.39:0/155304195 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f847c078bc0 0x7f847c07b0d0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:16.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.693+0000 7f84a1800640 1 --2- 172.21.15.39:0/155304195 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f849c156070 0x7f849c1e81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:16.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.693+0000 7f84a1800640 1 --2- 172.21.15.39:0/155304195 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f849c15fba0 0x7f849c158d10 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:16.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.693+0000 7f84a1800640 1 --2- 172.21.15.39:0/155304195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f849c15f790 0x7f849c1f5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:16.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.693+0000 7f84a1800640 1 -- 172.21.15.39:0/155304195 >> 172.21.15.39:0/155304195 conn(0x7f849c06cfc0 msgr2=0x7f849c153260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:16.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.693+0000 7f84a1800640 1 -- 172.21.15.39:0/155304195 shutdown_connections 2024-09-21T07:40:16.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:16.693+0000 7f84a1800640 1 -- 172.21.15.39:0/155304195 wait complete. 2024-09-21T07:40:16.717 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:reconnect 2024-09-21T07:40:17.718 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:40:17.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:17.956+0000 7efdd6a00640 1 -- 172.21.15.39:0/562605256 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efdd00d0dd0 msgr2=0x7efdd00cdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:17.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:17.956+0000 7efdd6a00640 1 --2- 172.21.15.39:0/562605256 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efdd00d0dd0 0x7efdd00cdb90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efdc8005fd0 tx=0x7efdc802a820 comp rx=0 tx=0).stop 2024-09-21T07:40:17.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:17.956+0000 7efdd6a00640 1 -- 172.21.15.39:0/562605256 shutdown_connections 2024-09-21T07:40:17.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:17.956+0000 7efdd6a00640 1 --2- 172.21.15.39:0/562605256 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efdd00d0dd0 0x7efdd00cdb90 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:17.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:17.956+0000 7efdd6a00640 1 --2- 172.21.15.39:0/562605256 >> v2:172.21.15.148:3300/0 conn(0x7efdd015a460 0x7efdd006fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:17.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:17.956+0000 7efdd6a00640 1 -- 172.21.15.39:0/562605256 >> 172.21.15.39:0/562605256 conn(0x7efdd006cfc0 msgr2=0x7efdd006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:17.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:17.956+0000 7efdd6a00640 1 -- 172.21.15.39:0/562605256 shutdown_connections 2024-09-21T07:40:17.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:17.956+0000 7efdd6a00640 1 -- 172.21.15.39:0/562605256 wait complete. 2024-09-21T07:40:17.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:17.959+0000 7efdd6a00640 1 Processor -- start 2024-09-21T07:40:17.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:17.959+0000 7efdd6a00640 1 -- start start 2024-09-21T07:40:17.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:17.960+0000 7efdd6a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdd00cd780 0x7efdd01f6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:17.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:17.960+0000 7efdd6a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efdd00d0dd0 0x7efdd00b09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:17.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:17.960+0000 7efdd6a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efdd015a460 0x7efdd015bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:17.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:17.960+0000 7efdd6a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efdd015d560 con 0x7efdd00cd780 2024-09-21T07:40:17.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:17.960+0000 7efdd6a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efdd015d3e0 con 0x7efdd00d0dd0 2024-09-21T07:40:17.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:17.960+0000 7efdd6a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7efdd015d6e0 con 0x7efdd015a460 2024-09-21T07:40:17.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:17.960+0000 7efdcfe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdd00cd780 0x7efdd01f6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:17.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:17.960+0000 7efdcfe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdd00cd780 0x7efdd01f6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35522/0 (socket says 172.21.15.39:35522) 2024-09-21T07:40:17.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:17.960+0000 7efdcfe00640 1 -- 172.21.15.39:0/4033675565 learned_addr learned my addr 172.21.15.39:0/4033675565 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:40:17.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:17.961+0000 7efdd4a00640 1 --2- 172.21.15.39:0/4033675565 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efdd015a460 0x7efdd015bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:17.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:17.961+0000 7efdc7e00640 1 --2- 172.21.15.39:0/4033675565 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efdd00d0dd0 0x7efdd00b09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:17.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:17.961+0000 7efdcfe00640 1 -- 172.21.15.39:0/4033675565 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efdd015a460 msgr2=0x7efdd015bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:17.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:17.961+0000 7efdcfe00640 1 --2- 172.21.15.39:0/4033675565 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efdd015a460 0x7efdd015bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:17.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:17.961+0000 7efdcfe00640 1 -- 172.21.15.39:0/4033675565 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efdd00d0dd0 msgr2=0x7efdd00b09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:17.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:17.961+0000 7efdcfe00640 1 --2- 172.21.15.39:0/4033675565 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efdd00d0dd0 0x7efdd00b09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:17.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:17.961+0000 7efdcfe00640 1 -- 172.21.15.39:0/4033675565 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd00a95b0 con 0x7efdd00cd780 2024-09-21T07:40:17.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:17.961+0000 7efdcfe00640 1 --2- 172.21.15.39:0/4033675565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdd00cd780 0x7efdd01f6590 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7efdc000e940 tx=0x7efdc000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:17.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:17.962+0000 7efdccc00640 1 -- 172.21.15.39:0/4033675565 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc000cd80 con 0x7efdd00cd780 2024-09-21T07:40:17.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:17.962+0000 7efdd6a00640 1 -- 172.21.15.39:0/4033675565 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdd00a9890 con 0x7efdd00cd780 2024-09-21T07:40:17.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:17.962+0000 7efdccc00640 1 -- 172.21.15.39:0/4033675565 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efdc0010430 con 0x7efdd00cd780 2024-09-21T07:40:17.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:17.962+0000 7efdccc00640 1 -- 172.21.15.39:0/4033675565 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc0010880 con 0x7efdd00cd780 2024-09-21T07:40:17.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:17.962+0000 7efdd6a00640 1 -- 172.21.15.39:0/4033675565 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdd00a9e50 con 0x7efdd00cd780 2024-09-21T07:40:17.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:17.963+0000 7efdd6a00640 1 -- 172.21.15.39:0/4033675565 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdd015db90 con 0x7efdd00cd780 2024-09-21T07:40:17.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:17.964+0000 7efdccc00640 1 -- 172.21.15.39:0/4033675565 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7efdc0010a20 con 0x7efdd00cd780 2024-09-21T07:40:17.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:17.965+0000 7efdccc00640 1 --2- 172.21.15.39:0/4033675565 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efdac078b90 0x7efdac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:17.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:17.965+0000 7efdccc00640 1 -- 172.21.15.39:0/4033675565 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7603+0+0 (secure 0 0 0) 0x7efdc009bf70 con 0x7efdd00cd780 2024-09-21T07:40:17.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:17.966+0000 7efdc7e00640 1 --2- 172.21.15.39:0/4033675565 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efdac078b90 0x7efdac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:17.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:17.966+0000 7efdc7e00640 1 --2- 172.21.15.39:0/4033675565 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efdac078b90 0x7efdac07b0a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7efdb8007fd0 tx=0x7efdb8001390 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:17.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:17.970+0000 7efdccc00640 1 -- 172.21.15.39:0/4033675565 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7efdc00634d0 con 0x7efdd00cd780 2024-09-21T07:40:18.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.205+0000 7efdd6a00640 1 -- 172.21.15.39:0/4033675565 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efdd006d3d0 con 0x7efdd00cd780 2024-09-21T07:40:18.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.206+0000 7efdccc00640 1 -- 172.21.15.39:0/4033675565 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+5258 (secure 0 0 0) 0x7efdc0067f00 con 0x7efdd00cd780 2024-09-21T07:40:18.208 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:40:18.208 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":114,"btime":"2024-09-21T07:40:17:883769+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7311,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:40:17.883762+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7113,"mds_1":6269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6269":{"gid":6269,"name":"d","rank":1,"incarnation":110,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6837/2336112936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2336112936},{"type":"v1","addr":"172.21.15.148:6837","nonce":2336112936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7113":{"gid":7113,"name":"a","rank":0,"incarnation":107,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/2821446711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2821446711},{"type":"v1","addr":"172.21.15.39:6835","nonce":2821446711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7113,"qdb_cluster":[6269,7113]},"id":6}]} 2024-09-21T07:40:18.208 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 114 2024-09-21T07:40:18.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.208+0000 7efdd6a00640 1 -- 172.21.15.39:0/4033675565 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efdac078b90 msgr2=0x7efdac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:18.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.208+0000 7efdd6a00640 1 --2- 172.21.15.39:0/4033675565 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efdac078b90 0x7efdac07b0a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7efdb8007fd0 tx=0x7efdb8001390 comp rx=0 tx=0).stop 2024-09-21T07:40:18.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.208+0000 7efdd6a00640 1 -- 172.21.15.39:0/4033675565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdd00cd780 msgr2=0x7efdd01f6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:18.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.208+0000 7efdd6a00640 1 --2- 172.21.15.39:0/4033675565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdd00cd780 0x7efdd01f6590 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7efdc000e940 tx=0x7efdc000ee00 comp rx=0 tx=0).stop 2024-09-21T07:40:18.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.208+0000 7efdd6a00640 1 -- 172.21.15.39:0/4033675565 shutdown_connections 2024-09-21T07:40:18.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.208+0000 7efdd6a00640 1 --2- 172.21.15.39:0/4033675565 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efdac078b90 0x7efdac07b0a0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:18.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.208+0000 7efdd6a00640 1 --2- 172.21.15.39:0/4033675565 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efdd015a460 0x7efdd015bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:18.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.208+0000 7efdd6a00640 1 --2- 172.21.15.39:0/4033675565 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efdd00d0dd0 0x7efdd00b09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:18.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.208+0000 7efdd6a00640 1 --2- 172.21.15.39:0/4033675565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdd00cd780 0x7efdd01f6590 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:18.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.208+0000 7efdd6a00640 1 -- 172.21.15.39:0/4033675565 >> 172.21.15.39:0/4033675565 conn(0x7efdd006cfc0 msgr2=0x7efdd015ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:18.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.208+0000 7efdd6a00640 1 -- 172.21.15.39:0/4033675565 shutdown_connections 2024-09-21T07:40:18.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.209+0000 7efdd6a00640 1 -- 172.21.15.39:0/4033675565 wait complete. 2024-09-21T07:40:18.235 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:active 2024-09-21T07:40:18.235 DEBUG:tasks.cephfs.filesystem:reached state 'up:active' in 3.532198667526245s 2024-09-21T07:40:18.235 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:40:18.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.461+0000 7f5a09c00640 1 -- 172.21.15.39:0/502615887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59e8005250 msgr2=0x7f59e800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:18.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.461+0000 7f5a09c00640 1 --2- 172.21.15.39:0/502615887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59e8005250 0x7f59e800b9c0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f59fc001dc0 tx=0x7f59e8005830 comp rx=0 tx=0).stop 2024-09-21T07:40:18.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.461+0000 7f5a09c00640 1 -- 172.21.15.39:0/502615887 shutdown_connections 2024-09-21T07:40:18.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.461+0000 7f5a09c00640 1 --2- 172.21.15.39:0/502615887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59e8005250 0x7f59e800b9c0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:18.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.461+0000 7f5a09c00640 1 -- 172.21.15.39:0/502615887 >> 172.21.15.39:0/502615887 conn(0x7f5a040b2510 msgr2=0x7f5a040d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:18.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.461+0000 7f5a09c00640 1 -- 172.21.15.39:0/502615887 shutdown_connections 2024-09-21T07:40:18.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.461+0000 7f5a09c00640 1 -- 172.21.15.39:0/502615887 wait complete. 2024-09-21T07:40:18.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.463+0000 7f5a09c00640 1 Processor -- start 2024-09-21T07:40:18.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.463+0000 7f5a09c00640 1 -- start start 2024-09-21T07:40:18.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.465+0000 7f5a09c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59e8005250 0x7f5a0415b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:18.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.465+0000 7f5a09c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a040c97f0 0x7f5a040ae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:18.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.465+0000 7f5a09c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a0415fb10 0x7f5a040aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:18.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.465+0000 7f5a09c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0406fc30 con 0x7f5a040c97f0 2024-09-21T07:40:18.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.465+0000 7f5a09c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0406fab0 con 0x7f5a0415fb10 2024-09-21T07:40:18.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.465+0000 7f5a09c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f5a0406fdb0 con 0x7f59e8005250 2024-09-21T07:40:18.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.465+0000 7f5a03400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a040c97f0 0x7f5a040ae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:18.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.465+0000 7f5a03400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a040c97f0 0x7f5a040ae7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35538/0 (socket says 172.21.15.39:35538) 2024-09-21T07:40:18.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.465+0000 7f5a03400640 1 -- 172.21.15.39:0/985589348 learned_addr learned my addr 172.21.15.39:0/985589348 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:40:18.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.465+0000 7f5a03e00640 1 --2- 172.21.15.39:0/985589348 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59e8005250 0x7f5a0415b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:18.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.465+0000 7f5a09200640 1 --2- 172.21.15.39:0/985589348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a0415fb10 0x7f5a040aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:18.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.465+0000 7f5a03400640 1 -- 172.21.15.39:0/985589348 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59e8005250 msgr2=0x7f5a0415b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:18.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.465+0000 7f5a03400640 1 --2- 172.21.15.39:0/985589348 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59e8005250 0x7f5a0415b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:18.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.465+0000 7f5a03400640 1 -- 172.21.15.39:0/985589348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a0415fb10 msgr2=0x7f5a040aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:18.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.465+0000 7f5a03400640 1 --2- 172.21.15.39:0/985589348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a0415fb10 0x7f5a040aecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:18.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.465+0000 7f5a03400640 1 -- 172.21.15.39:0/985589348 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a041f53a0 con 0x7f5a040c97f0 2024-09-21T07:40:18.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.466+0000 7f5a09200640 1 --2- 172.21.15.39:0/985589348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a0415fb10 0x7f5a040aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:40:18.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.466+0000 7f5a03400640 1 --2- 172.21.15.39:0/985589348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a040c97f0 0x7f5a040ae7b0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f59f00066b0 tx=0x7f59f00039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:18.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.466+0000 7f59fbe00640 1 -- 172.21.15.39:0/985589348 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f0015070 con 0x7f5a040c97f0 2024-09-21T07:40:18.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.466+0000 7f59fbe00640 1 -- 172.21.15.39:0/985589348 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f59f0011900 con 0x7f5a040c97f0 2024-09-21T07:40:18.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.466+0000 7f5a09c00640 1 -- 172.21.15.39:0/985589348 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a040cf300 con 0x7f5a040c97f0 2024-09-21T07:40:18.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.466+0000 7f59fbe00640 1 -- 172.21.15.39:0/985589348 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f0011d70 con 0x7f5a040c97f0 2024-09-21T07:40:18.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.466+0000 7f5a09c00640 1 -- 172.21.15.39:0/985589348 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a040cf880 con 0x7f5a040c97f0 2024-09-21T07:40:18.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.467+0000 7f5a09c00640 1 -- 172.21.15.39:0/985589348 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a040c9d40 con 0x7f5a040c97f0 2024-09-21T07:40:18.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.468+0000 7f59fbe00640 1 -- 172.21.15.39:0/985589348 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f59f0018070 con 0x7f5a040c97f0 2024-09-21T07:40:18.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.469+0000 7f59fbe00640 1 --2- 172.21.15.39:0/985589348 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f59e4078b90 0x7f59e407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:18.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.470+0000 7f59fbe00640 1 -- 172.21.15.39:0/985589348 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7603+0+0 (secure 0 0 0) 0x7f59f009c3d0 con 0x7f5a040c97f0 2024-09-21T07:40:18.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.470+0000 7f5a03e00640 1 --2- 172.21.15.39:0/985589348 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f59e4078b90 0x7f59e407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:18.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.471+0000 7f5a03e00640 1 --2- 172.21.15.39:0/985589348 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f59e4078b90 0x7f59e407b0a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f59f4009bb0 tx=0x7f59f4005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:18.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.474+0000 7f59fbe00640 1 -- 172.21.15.39:0/985589348 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f59f0064010 con 0x7f5a040c97f0 2024-09-21T07:40:18.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.710+0000 7f5a09c00640 1 -- 172.21.15.39:0/985589348 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a04072fa0 con 0x7f5a040c97f0 2024-09-21T07:40:18.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.711+0000 7f59fbe00640 1 -- 172.21.15.39:0/985589348 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+5258 (secure 0 0 0) 0x7f59f0063910 con 0x7f5a040c97f0 2024-09-21T07:40:18.713 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:40:18.713 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":114,"btime":"2024-09-21T07:40:17:883769+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7311,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:40:17.883762+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7113,"mds_1":6269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6269":{"gid":6269,"name":"d","rank":1,"incarnation":110,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6837/2336112936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2336112936},{"type":"v1","addr":"172.21.15.148:6837","nonce":2336112936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7113":{"gid":7113,"name":"a","rank":0,"incarnation":107,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/2821446711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2821446711},{"type":"v1","addr":"172.21.15.39:6835","nonce":2821446711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7113,"qdb_cluster":[6269,7113]},"id":6}]} 2024-09-21T07:40:18.713 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 114 2024-09-21T07:40:18.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.714+0000 7f5a09c00640 1 -- 172.21.15.39:0/985589348 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f59e4078b90 msgr2=0x7f59e407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:18.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.714+0000 7f5a09c00640 1 --2- 172.21.15.39:0/985589348 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f59e4078b90 0x7f59e407b0a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f59f4009bb0 tx=0x7f59f4005f90 comp rx=0 tx=0).stop 2024-09-21T07:40:18.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.714+0000 7f5a09c00640 1 -- 172.21.15.39:0/985589348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a040c97f0 msgr2=0x7f5a040ae7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:18.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.714+0000 7f5a09c00640 1 --2- 172.21.15.39:0/985589348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a040c97f0 0x7f5a040ae7b0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f59f00066b0 tx=0x7f59f00039c0 comp rx=0 tx=0).stop 2024-09-21T07:40:18.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.714+0000 7f5a09c00640 1 -- 172.21.15.39:0/985589348 shutdown_connections 2024-09-21T07:40:18.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.714+0000 7f5a09c00640 1 --2- 172.21.15.39:0/985589348 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f59e4078b90 0x7f59e407b0a0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:18.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.714+0000 7f5a09c00640 1 --2- 172.21.15.39:0/985589348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a0415fb10 0x7f5a040aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:18.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.714+0000 7f5a09c00640 1 --2- 172.21.15.39:0/985589348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a040c97f0 0x7f5a040ae7b0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:18.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.714+0000 7f5a09c00640 1 --2- 172.21.15.39:0/985589348 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59e8005250 0x7f5a0415b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:18.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.714+0000 7f5a09c00640 1 -- 172.21.15.39:0/985589348 >> 172.21.15.39:0/985589348 conn(0x7f5a040b2510 msgr2=0x7f5a04073640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:18.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.714+0000 7f5a09c00640 1 -- 172.21.15.39:0/985589348 shutdown_connections 2024-09-21T07:40:18.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:18.714+0000 7f5a09c00640 1 -- 172.21.15.39:0/985589348 wait complete. 2024-09-21T07:40:18.738 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump snaps --server 2024-09-21T07:40:18.981 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_snap":11,"last_created":11,"last_destroyed":9,"pending_noop":[],"snaps":[{"snapid":11,"ino":1099511627777,"stamp":"2024-09-21T07:39:42.151005+0000","name":"s25","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2024-09-21T07:40:18.981 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 11, "last_destroyed": 9, "last_snap": 11, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s25", "snapid": 11, "stamp": "2024-09-21T07:39:42.151005+0000" } ] } 2024-09-21T07:40:18.981 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T07:40:18.982 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:40:19.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.207+0000 7f7714000640 1 -- 172.21.15.39:0/2902369644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76f4004d40 msgr2=0x7f76f4005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:19.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.207+0000 7f7714000640 1 --2- 172.21.15.39:0/2902369644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76f4004d40 0x7f76f4005150 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f7704006fd0 tx=0x7f76f4005830 comp rx=0 tx=0).stop 2024-09-21T07:40:19.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.207+0000 7f7712000640 1 -- 172.21.15.39:0/2902369644 reap_dead start 2024-09-21T07:40:19.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.207+0000 7f7714000640 1 -- 172.21.15.39:0/2902369644 shutdown_connections 2024-09-21T07:40:19.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.207+0000 7f7714000640 1 -- 172.21.15.39:0/2902369644 >> 172.21.15.39:0/2902369644 conn(0x7f770c06cfc0 msgr2=0x7f770c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:19.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.207+0000 7f7714000640 1 -- 172.21.15.39:0/2902369644 shutdown_connections 2024-09-21T07:40:19.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.207+0000 7f7714000640 1 -- 172.21.15.39:0/2902369644 wait complete. 2024-09-21T07:40:19.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.209+0000 7f7714000640 1 Processor -- start 2024-09-21T07:40:19.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.210+0000 7f7714000640 1 -- start start 2024-09-21T07:40:19.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.211+0000 7f7714000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f770c0af220 0x7f770c15be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:19.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.211+0000 7f7714000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f770c140220 0x7f770c13e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:19.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.211+0000 7f7714000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f770c13eef0 0x7f770c1f90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:19.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.211+0000 7f7714000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f770c06ffe0 con 0x7f770c0af220 2024-09-21T07:40:19.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.211+0000 7f7714000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f770c06fe60 con 0x7f770c13eef0 2024-09-21T07:40:19.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.211+0000 7f7714000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f770c070160 con 0x7f770c140220 2024-09-21T07:40:19.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.211+0000 7f7711600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f770c0af220 0x7f770c15be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:19.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.211+0000 7f7711600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f770c0af220 0x7f770c15be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35562/0 (socket says 172.21.15.39:35562) 2024-09-21T07:40:19.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.211+0000 7f7711600640 1 -- 172.21.15.39:0/1443206383 learned_addr learned my addr 172.21.15.39:0/1443206383 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:40:19.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.212+0000 7f7710c00640 1 --2- 172.21.15.39:0/1443206383 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f770c140220 0x7f770c13e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:19.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.212+0000 7f7712000640 1 --2- 172.21.15.39:0/1443206383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f770c13eef0 0x7f770c1f90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:19.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.212+0000 7f7711600640 1 -- 172.21.15.39:0/1443206383 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f770c140220 msgr2=0x7f770c13e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:19.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.212+0000 7f7711600640 1 --2- 172.21.15.39:0/1443206383 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f770c140220 0x7f770c13e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:19.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.212+0000 7f7711600640 1 -- 172.21.15.39:0/1443206383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f770c13eef0 msgr2=0x7f770c1f90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:19.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.212+0000 7f7711600640 1 --2- 172.21.15.39:0/1443206383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f770c13eef0 0x7f770c1f90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:19.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.212+0000 7f7711600640 1 -- 172.21.15.39:0/1443206383 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f770c13f300 con 0x7f770c0af220 2024-09-21T07:40:19.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.212+0000 7f7712000640 1 --2- 172.21.15.39:0/1443206383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f770c13eef0 0x7f770c1f90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:40:19.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.212+0000 7f7711600640 1 --2- 172.21.15.39:0/1443206383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f770c0af220 0x7f770c15be60 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f7700011af0 tx=0x7f770000f360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:19.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.212+0000 7f7709600640 1 -- 172.21.15.39:0/1443206383 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770001f070 con 0x7f770c0af220 2024-09-21T07:40:19.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.212+0000 7f7709600640 1 -- 172.21.15.39:0/1443206383 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7700012540 con 0x7f770c0af220 2024-09-21T07:40:19.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.212+0000 7f7709600640 1 -- 172.21.15.39:0/1443206383 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7700012930 con 0x7f770c0af220 2024-09-21T07:40:19.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.212+0000 7f7714000640 1 -- 172.21.15.39:0/1443206383 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f770c0ad930 con 0x7f770c0af220 2024-09-21T07:40:19.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.213+0000 7f7714000640 1 -- 172.21.15.39:0/1443206383 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f770c0adeb0 con 0x7f770c0af220 2024-09-21T07:40:19.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.214+0000 7f7714000640 1 -- 172.21.15.39:0/1443206383 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f770c160340 con 0x7f770c0af220 2024-09-21T07:40:19.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.215+0000 7f7709600640 1 -- 172.21.15.39:0/1443206383 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7700022070 con 0x7f770c0af220 2024-09-21T07:40:19.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.216+0000 7f7709600640 1 --2- 172.21.15.39:0/1443206383 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f76ec078bf0 0x7f76ec07b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:19.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.216+0000 7f7709600640 1 -- 172.21.15.39:0/1443206383 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7603+0+0 (secure 0 0 0) 0x7f770009c220 con 0x7f770c0af220 2024-09-21T07:40:19.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.216+0000 7f7710c00640 1 --2- 172.21.15.39:0/1443206383 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f76ec078bf0 0x7f76ec07b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:19.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.217+0000 7f7710c00640 1 --2- 172.21.15.39:0/1443206383 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f76ec078bf0 0x7f76ec07b100 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f770c155360 tx=0x7f76f8005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:19.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.220+0000 7f7709600640 1 -- 172.21.15.39:0/1443206383 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f770c13f300 con 0x7f770c0af220 2024-09-21T07:40:19.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.455+0000 7f7714000640 1 -- 172.21.15.39:0/1443206383 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f770c071470 con 0x7f770c0af220 2024-09-21T07:40:19.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.455+0000 7f7709600640 1 -- 172.21.15.39:0/1443206383 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+5258 (secure 0 0 0) 0x7f7700063890 con 0x7f770c0af220 2024-09-21T07:40:19.457 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:40:19.457 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":114,"btime":"2024-09-21T07:40:17:883769+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7311,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:40:17.883762+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7113,"mds_1":6269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6269":{"gid":6269,"name":"d","rank":1,"incarnation":110,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6837/2336112936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2336112936},{"type":"v1","addr":"172.21.15.148:6837","nonce":2336112936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7113":{"gid":7113,"name":"a","rank":0,"incarnation":107,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/2821446711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2821446711},{"type":"v1","addr":"172.21.15.39:6835","nonce":2821446711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7113,"qdb_cluster":[6269,7113]},"id":6}]} 2024-09-21T07:40:19.458 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 114 2024-09-21T07:40:19.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.457+0000 7f7714000640 1 -- 172.21.15.39:0/1443206383 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f76ec078bf0 msgr2=0x7f76ec07b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:19.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.457+0000 7f7714000640 1 --2- 172.21.15.39:0/1443206383 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f76ec078bf0 0x7f76ec07b100 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f770c155360 tx=0x7f76f8005f90 comp rx=0 tx=0).stop 2024-09-21T07:40:19.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.457+0000 7f7714000640 1 -- 172.21.15.39:0/1443206383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f770c0af220 msgr2=0x7f770c15be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:19.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.457+0000 7f7714000640 1 --2- 172.21.15.39:0/1443206383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f770c0af220 0x7f770c15be60 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f7700011af0 tx=0x7f770000f360 comp rx=0 tx=0).stop 2024-09-21T07:40:19.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.458+0000 7f7714000640 1 -- 172.21.15.39:0/1443206383 shutdown_connections 2024-09-21T07:40:19.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.458+0000 7f7714000640 1 --2- 172.21.15.39:0/1443206383 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f76ec078bf0 0x7f76ec07b100 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:19.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.458+0000 7f7714000640 1 --2- 172.21.15.39:0/1443206383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f770c13eef0 0x7f770c1f90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:19.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.458+0000 7f7714000640 1 --2- 172.21.15.39:0/1443206383 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f770c140220 0x7f770c13e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:19.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.458+0000 7f7714000640 1 --2- 172.21.15.39:0/1443206383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f770c0af220 0x7f770c15be60 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:19.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.458+0000 7f7714000640 1 -- 172.21.15.39:0/1443206383 >> 172.21.15.39:0/1443206383 conn(0x7f770c06cfc0 msgr2=0x7f770c15d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:19.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.458+0000 7f7714000640 1 -- 172.21.15.39:0/1443206383 shutdown_connections 2024-09-21T07:40:19.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:19.458+0000 7f7714000640 1 -- 172.21.15.39:0/1443206383 wait complete. 2024-09-21T07:40:19.481 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump snaps --server 2024-09-21T07:40:19.729 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_snap":11,"last_created":11,"last_destroyed":9,"pending_noop":[],"snaps":[{"snapid":11,"ino":1099511627777,"stamp":"2024-09-21T07:39:42.151005+0000","name":"s25","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2024-09-21T07:40:19.729 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 11, "last_destroyed": 9, "last_snap": 11, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s25", "snapid": 11, "stamp": "2024-09-21T07:39:42.151005+0000" } ] } 2024-09-21T07:40:19.729 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:40:19.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:40:19.729 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-09-21T07:40:19.786 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:40:19.786 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:40:19.821 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:40:19.821 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:40:19.898 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:40:20.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.164+0000 7feb11a00640 1 -- 172.21.15.39:0/3170401209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feaf0003a50 msgr2=0x7feaf0003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:20.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.164+0000 7feb11a00640 1 --2- 172.21.15.39:0/3170401209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feaf0003a50 0x7feaf0003ec0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7feafc0016c0 tx=0x7feaf0005830 comp rx=0 tx=0).stop 2024-09-21T07:40:20.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.164+0000 7feb11a00640 1 -- 172.21.15.39:0/3170401209 shutdown_connections 2024-09-21T07:40:20.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.164+0000 7feb11a00640 1 --2- 172.21.15.39:0/3170401209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feaf0003a50 0x7feaf0003ec0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:20.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.164+0000 7feb11a00640 1 -- 172.21.15.39:0/3170401209 >> 172.21.15.39:0/3170401209 conn(0x7feb0c06cfc0 msgr2=0x7feb0c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:20.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.164+0000 7feb11a00640 1 -- 172.21.15.39:0/3170401209 shutdown_connections 2024-09-21T07:40:20.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.164+0000 7feb11a00640 1 -- 172.21.15.39:0/3170401209 wait complete. 2024-09-21T07:40:20.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.167+0000 7feb11a00640 1 Processor -- start 2024-09-21T07:40:20.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.167+0000 7feb11a00640 1 -- start start 2024-09-21T07:40:20.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.168+0000 7feb11a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feaf0003a50 0x7feb0c150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:20.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.168+0000 7feb11a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb0c0c52e0 0x7feb0c0b1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:20.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.168+0000 7feb11a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb0c156f70 0x7feb0c0b21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:20.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.168+0000 7feb11a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c06f800 con 0x7feb0c156f70 2024-09-21T07:40:20.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.168+0000 7feb11a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c0d0e60 con 0x7feaf0003a50 2024-09-21T07:40:20.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.168+0000 7feb11a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7feb0c06f980 con 0x7feb0c0c52e0 2024-09-21T07:40:20.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.168+0000 7feb0b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb0c156f70 0x7feb0c0b21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:20.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.168+0000 7feb0b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb0c156f70 0x7feb0c0b21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35588/0 (socket says 172.21.15.39:35588) 2024-09-21T07:40:20.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.169+0000 7feb0b400640 1 -- 172.21.15.39:0/935214889 learned_addr learned my addr 172.21.15.39:0/935214889 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:40:20.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.169+0000 7feb0aa00640 1 --2- 172.21.15.39:0/935214889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feaf0003a50 0x7feb0c150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:20.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.169+0000 7feb03e00640 1 --2- 172.21.15.39:0/935214889 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb0c0c52e0 0x7feb0c0b1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:20.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.169+0000 7feb0b400640 1 -- 172.21.15.39:0/935214889 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb0c0c52e0 msgr2=0x7feb0c0b1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:20.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.169+0000 7feb0b400640 1 --2- 172.21.15.39:0/935214889 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb0c0c52e0 0x7feb0c0b1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:20.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.169+0000 7feb0b400640 1 -- 172.21.15.39:0/935214889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feaf0003a50 msgr2=0x7feb0c150ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:20.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.169+0000 7feb0b400640 1 --2- 172.21.15.39:0/935214889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feaf0003a50 0x7feb0c150ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:20.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.169+0000 7feb0b400640 1 -- 172.21.15.39:0/935214889 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb0c151420 con 0x7feb0c156f70 2024-09-21T07:40:20.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.169+0000 7feb03e00640 1 --2- 172.21.15.39:0/935214889 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb0c0c52e0 0x7feb0c0b1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:40:20.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.169+0000 7feb0b400640 1 --2- 172.21.15.39:0/935214889 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb0c156f70 0x7feb0c0b21c0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7feb04006fd0 tx=0x7feb04006720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:20.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.169+0000 7feb02a00640 1 -- 172.21.15.39:0/935214889 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0400fc90 con 0x7feb0c156f70 2024-09-21T07:40:20.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.170+0000 7feb02a00640 1 -- 172.21.15.39:0/935214889 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb04018070 con 0x7feb0c156f70 2024-09-21T07:40:20.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.170+0000 7feb02a00640 1 -- 172.21.15.39:0/935214889 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb04011690 con 0x7feb0c156f70 2024-09-21T07:40:20.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.170+0000 7feb11a00640 1 -- 172.21.15.39:0/935214889 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb0c0c5a30 con 0x7feb0c156f70 2024-09-21T07:40:20.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.170+0000 7feb11a00640 1 -- 172.21.15.39:0/935214889 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb0c0c5ff0 con 0x7feb0c156f70 2024-09-21T07:40:20.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.171+0000 7feb11a00640 1 -- 172.21.15.39:0/935214889 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb0c0c5770 con 0x7feb0c156f70 2024-09-21T07:40:20.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.172+0000 7feb02a00640 1 -- 172.21.15.39:0/935214889 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7feb04011830 con 0x7feb0c156f70 2024-09-21T07:40:20.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.173+0000 7feb02a00640 1 --2- 172.21.15.39:0/935214889 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feae8078bc0 0x7feae807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:20.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.173+0000 7feb02a00640 1 -- 172.21.15.39:0/935214889 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7603+0+0 (secure 0 0 0) 0x7feb0409bc40 con 0x7feb0c156f70 2024-09-21T07:40:20.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.173+0000 7feb0aa00640 1 --2- 172.21.15.39:0/935214889 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feae8078bc0 0x7feae807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:20.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.174+0000 7feb0aa00640 1 --2- 172.21.15.39:0/935214889 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feae8078bc0 0x7feae807b0d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7feafc032000 tx=0x7feafc001190 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:20.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.178+0000 7feb02a00640 1 -- 172.21.15.39:0/935214889 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7feb040639a0 con 0x7feb0c156f70 2024-09-21T07:40:20.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.388+0000 7feb11a00640 1 -- 172.21.15.39:0/935214889 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7feb0c071230 con 0x7feb0c156f70 2024-09-21T07:40:20.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.389+0000 7feb02a00640 1 -- 172.21.15.39:0/935214889 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 18 entries v106) ==== 81+0+1087 (secure 0 0 0) 0x7feb04099e20 con 0x7feb0c156f70 2024-09-21T07:40:20.391 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/1456343023 2024-09-22T07:40:08.637993+0000 2024-09-21T07:40:20.391 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6833/1456343023 2024-09-22T07:40:08.637993+0000 2024-09-21T07:40:20.391 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/992188984 2024-09-22T07:40:00.778049+0000 2024-09-21T07:40:20.391 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/3356393880 2024-09-21T08:40:15.824973+0000 2024-09-21T07:40:20.391 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/992188984 2024-09-22T07:40:00.778049+0000 2024-09-21T07:40:20.391 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/1472430787 2024-09-22T07:37:58.212892+0000 2024-09-21T07:40:20.391 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/829716143 2024-09-22T07:37:01.259219+0000 2024-09-21T07:40:20.391 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/3026827614 2024-09-22T07:37:29.781472+0000 2024-09-21T07:40:20.391 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/3026827614 2024-09-22T07:37:29.781472+0000 2024-09-21T07:40:20.391 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/1472430787 2024-09-22T07:37:58.212892+0000 2024-09-21T07:40:20.392 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.1:0/90671776 2024-09-21T08:39:22.988045+0000 2024-09-21T07:40:20.392 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/2381504293 2024-09-22T07:38:32.207414+0000 2024-09-21T07:40:20.392 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/2381504293 2024-09-22T07:38:32.207414+0000 2024-09-21T07:40:20.392 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/855150775 2024-09-22T07:39:07.739045+0000 2024-09-21T07:40:20.392 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/829716143 2024-09-22T07:37:01.259219+0000 2024-09-21T07:40:20.392 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6833/364657398 2024-09-22T07:39:15.579613+0000 2024-09-21T07:40:20.392 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/855150775 2024-09-22T07:39:07.739045+0000 2024-09-21T07:40:20.392 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/364657398 2024-09-22T07:39:15.579613+0000 2024-09-21T07:40:20.392 INFO:teuthology.orchestra.run.smithi039.stderr:listed 18 entries 2024-09-21T07:40:20.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.391+0000 7feb11a00640 1 -- 172.21.15.39:0/935214889 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feae8078bc0 msgr2=0x7feae807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:20.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.391+0000 7feb11a00640 1 --2- 172.21.15.39:0/935214889 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feae8078bc0 0x7feae807b0d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7feafc032000 tx=0x7feafc001190 comp rx=0 tx=0).stop 2024-09-21T07:40:20.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.391+0000 7feb11a00640 1 -- 172.21.15.39:0/935214889 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb0c156f70 msgr2=0x7feb0c0b21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:20.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.391+0000 7feb11a00640 1 --2- 172.21.15.39:0/935214889 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb0c156f70 0x7feb0c0b21c0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7feb04006fd0 tx=0x7feb04006720 comp rx=0 tx=0).stop 2024-09-21T07:40:20.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.391+0000 7feb11a00640 1 -- 172.21.15.39:0/935214889 shutdown_connections 2024-09-21T07:40:20.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.391+0000 7feb11a00640 1 --2- 172.21.15.39:0/935214889 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feae8078bc0 0x7feae807b0d0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:20.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.391+0000 7feb11a00640 1 --2- 172.21.15.39:0/935214889 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb0c156f70 0x7feb0c0b21c0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:20.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.391+0000 7feb11a00640 1 --2- 172.21.15.39:0/935214889 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb0c0c52e0 0x7feb0c0b1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:20.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.391+0000 7feb11a00640 1 --2- 172.21.15.39:0/935214889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feaf0003a50 0x7feb0c150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:20.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.391+0000 7feb11a00640 1 -- 172.21.15.39:0/935214889 >> 172.21.15.39:0/935214889 conn(0x7feb0c06cfc0 msgr2=0x7feb0c15de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:20.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.391+0000 7feb11a00640 1 -- 172.21.15.39:0/935214889 shutdown_connections 2024-09-21T07:40:20.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:20.391+0000 7feb11a00640 1 -- 172.21.15.39:0/935214889 wait complete. 2024-09-21T07:40:20.419 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T07:40:20.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:40:20.420 DEBUG:teuthology.orchestra.run.smithi039:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T07:40:21.611 INFO:tasks.ceph.mds.a.smithi039.stderr:2024-09-21T07:40:21.609+0000 7fbe0ce00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:40:21.611 INFO:tasks.ceph.mds.a.smithi039.stderr:2024-09-21T07:40:21.609+0000 7fbe0ce00640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:40:28.177 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T07:40:28.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:40:28.178 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:40:28.217 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:40:28.217 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:40:28.295 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:40:28.295 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:40:28.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.562+0000 7f647be00640 1 -- 172.21.15.39:0/3883317767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f647c160a30 msgr2=0x7f647c0cb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:28.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.562+0000 7f647be00640 1 --2- 172.21.15.39:0/3883317767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f647c160a30 0x7f647c0cb9b0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f6464005fd0 tx=0x7f646402a540 comp rx=0 tx=0).stop 2024-09-21T07:40:28.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.563+0000 7f647be00640 1 -- 172.21.15.39:0/3883317767 shutdown_connections 2024-09-21T07:40:28.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.563+0000 7f647be00640 1 --2- 172.21.15.39:0/3883317767 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f647c0cbef0 0x7f647c06f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:28.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.563+0000 7f647be00640 1 --2- 172.21.15.39:0/3883317767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f647c160a30 0x7f647c0cb9b0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:28.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.563+0000 7f647be00640 1 -- 172.21.15.39:0/3883317767 >> 172.21.15.39:0/3883317767 conn(0x7f647c0b2510 msgr2=0x7f647c0d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:28.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.563+0000 7f647be00640 1 -- 172.21.15.39:0/3883317767 shutdown_connections 2024-09-21T07:40:28.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.563+0000 7f647be00640 1 -- 172.21.15.39:0/3883317767 wait complete. 2024-09-21T07:40:28.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.565+0000 7f647be00640 1 Processor -- start 2024-09-21T07:40:28.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.565+0000 7f647be00640 1 -- start start 2024-09-21T07:40:28.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.566+0000 7f647be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f647c0cbef0 0x7f647c0b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:28.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.566+0000 7f647be00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f647c15fc30 0x7f647c0cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:28.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.566+0000 7f647be00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f647c0cf130 0x7f647c0cf540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:28.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.566+0000 7f647be00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f647c06f8d0 con 0x7f647c0cbef0 2024-09-21T07:40:28.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.566+0000 7f647be00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f647c06f750 con 0x7f647c15fc30 2024-09-21T07:40:28.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.566+0000 7f647be00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f647c06fa50 con 0x7f647c0cf130 2024-09-21T07:40:28.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.567+0000 7f647aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f647c0cbef0 0x7f647c0b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:28.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.567+0000 7f647aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f647c0cbef0 0x7f647c0b09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53278/0 (socket says 172.21.15.39:53278) 2024-09-21T07:40:28.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.567+0000 7f647aa00640 1 -- 172.21.15.39:0/2690616632 learned_addr learned my addr 172.21.15.39:0/2690616632 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:40:28.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.567+0000 7f6472000640 1 --2- 172.21.15.39:0/2690616632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f647c15fc30 0x7f647c0cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:28.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.567+0000 7f647b400640 1 --2- 172.21.15.39:0/2690616632 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f647c0cf130 0x7f647c0cf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:28.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.567+0000 7f647aa00640 1 -- 172.21.15.39:0/2690616632 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f647c0cf130 msgr2=0x7f647c0cf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:28.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.567+0000 7f647aa00640 1 --2- 172.21.15.39:0/2690616632 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f647c0cf130 0x7f647c0cf540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:28.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.567+0000 7f647aa00640 1 -- 172.21.15.39:0/2690616632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f647c15fc30 msgr2=0x7f647c0cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:28.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.567+0000 7f647aa00640 1 --2- 172.21.15.39:0/2690616632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f647c15fc30 0x7f647c0cebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:28.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.567+0000 7f647aa00640 1 -- 172.21.15.39:0/2690616632 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f647c0b0ee0 con 0x7f647c0cbef0 2024-09-21T07:40:28.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.567+0000 7f647b400640 1 --2- 172.21.15.39:0/2690616632 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f647c0cf130 0x7f647c0cf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:40:28.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.567+0000 7f6472000640 1 --2- 172.21.15.39:0/2690616632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f647c15fc30 0x7f647c0cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:40:28.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.567+0000 7f647aa00640 1 --2- 172.21.15.39:0/2690616632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f647c0cbef0 0x7f647c0b09a0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f646c00c3a0 tx=0x7f646c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:28.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.568+0000 7f6473400640 1 -- 172.21.15.39:0/2690616632 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646c0052d0 con 0x7f647c0cbef0 2024-09-21T07:40:28.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.568+0000 7f6473400640 1 -- 172.21.15.39:0/2690616632 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f646c007020 con 0x7f647c0cbef0 2024-09-21T07:40:28.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.568+0000 7f6473400640 1 -- 172.21.15.39:0/2690616632 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646c007470 con 0x7f647c0cbef0 2024-09-21T07:40:28.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.568+0000 7f647be00640 1 -- 172.21.15.39:0/2690616632 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f647c0c6090 con 0x7f647c0cbef0 2024-09-21T07:40:28.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.568+0000 7f647be00640 1 -- 172.21.15.39:0/2690616632 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f647c0ce3f0 con 0x7f647c0cbef0 2024-09-21T07:40:28.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.569+0000 7f647be00640 1 -- 172.21.15.39:0/2690616632 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f647c06ff70 con 0x7f647c0cbef0 2024-09-21T07:40:28.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.570+0000 7f6473400640 1 -- 172.21.15.39:0/2690616632 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f646c0067d0 con 0x7f647c0cbef0 2024-09-21T07:40:28.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.571+0000 7f6473400640 1 --2- 172.21.15.39:0/2690616632 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6458078bc0 0x7f645807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:28.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.571+0000 7f6473400640 1 -- 172.21.15.39:0/2690616632 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7603+0+0 (secure 0 0 0) 0x7f646c09aa20 con 0x7f647c0cbef0 2024-09-21T07:40:28.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.572+0000 7f6472000640 1 --2- 172.21.15.39:0/2690616632 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6458078bc0 0x7f645807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:28.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.573+0000 7f6472000640 1 --2- 172.21.15.39:0/2690616632 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6458078bc0 0x7f645807b0d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6464033000 tx=0x7f6464004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:28.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.576+0000 7f6473400640 1 -- 172.21.15.39:0/2690616632 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f647c0b0ee0 con 0x7f647c0cbef0 2024-09-21T07:40:28.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.814+0000 7f647be00640 1 -- 172.21.15.39:0/2690616632 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f647c0b4400 con 0x7f647c0cbef0 2024-09-21T07:40:28.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.814+0000 7f6473400640 1 -- 172.21.15.39:0/2690616632 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v114) ==== 53+0+73 (secure 0 0 0) 0x7f646c062650 con 0x7f647c0cbef0 2024-09-21T07:40:28.816 INFO:teuthology.orchestra.run.smithi039.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:40:28.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.816+0000 7f647be00640 1 -- 172.21.15.39:0/2690616632 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6458078bc0 msgr2=0x7f645807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:28.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.816+0000 7f647be00640 1 --2- 172.21.15.39:0/2690616632 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6458078bc0 0x7f645807b0d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6464033000 tx=0x7f6464004730 comp rx=0 tx=0).stop 2024-09-21T07:40:28.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.816+0000 7f647be00640 1 -- 172.21.15.39:0/2690616632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f647c0cbef0 msgr2=0x7f647c0b09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:28.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.816+0000 7f647be00640 1 --2- 172.21.15.39:0/2690616632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f647c0cbef0 0x7f647c0b09a0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f646c00c3a0 tx=0x7f646c00c860 comp rx=0 tx=0).stop 2024-09-21T07:40:28.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.817+0000 7f647be00640 1 -- 172.21.15.39:0/2690616632 shutdown_connections 2024-09-21T07:40:28.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.817+0000 7f647be00640 1 --2- 172.21.15.39:0/2690616632 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6458078bc0 0x7f645807b0d0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:28.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.817+0000 7f647be00640 1 --2- 172.21.15.39:0/2690616632 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f647c0cf130 0x7f647c0cf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:28.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.817+0000 7f647be00640 1 --2- 172.21.15.39:0/2690616632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f647c15fc30 0x7f647c0cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:28.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.817+0000 7f647be00640 1 --2- 172.21.15.39:0/2690616632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f647c0cbef0 0x7f647c0b09a0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:28.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.817+0000 7f647be00640 1 -- 172.21.15.39:0/2690616632 >> 172.21.15.39:0/2690616632 conn(0x7f647c0b2510 msgr2=0x7f647c0b11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:28.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.817+0000 7f647be00640 1 -- 172.21.15.39:0/2690616632 shutdown_connections 2024-09-21T07:40:28.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:28.817+0000 7f647be00640 1 -- 172.21.15.39:0/2690616632 wait complete. 2024-09-21T07:40:28.842 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:40:28.842 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:40:28.842 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi039.front.sepia.ceph.com 2024-09-21T07:40:28.842 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T07:40:28.842 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:40:28.842 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:40:28.842 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:40:28.842 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:40:28.842 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:40:28.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:40:28.843 DEBUG:teuthology.orchestra.run.smithi039:> ip addr 2024-09-21T07:40:28.878 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:40:28.878 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:40:28.878 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:40:28.878 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:40:28.878 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2024-09-21T07:40:28.878 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:40:28.878 INFO:teuthology.orchestra.run.smithi039.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:40:28.878 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:40:28.878 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f0 2024-09-21T07:40:28.879 INFO:teuthology.orchestra.run.smithi039.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:40:28.879 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c7 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:40:28.879 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f1 2024-09-21T07:40:28.879 INFO:teuthology.orchestra.run.smithi039.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:40:28.879 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:70 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:40:28.879 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f0 2024-09-21T07:40:28.879 INFO:teuthology.orchestra.run.smithi039.stdout: inet 172.21.15.39/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T07:40:28.879 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft 42290sec preferred_lft 42290sec 2024-09-21T07:40:28.879 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec4:7aff:febd:1370/64 scope link 2024-09-21T07:40:28.879 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:40:28.879 INFO:teuthology.orchestra.run.smithi039.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:40:28.879 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:71 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:40:28.879 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f1 2024-09-21T07:40:28.879 INFO:teuthology.orchestra.run.smithi039.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:40:28.880 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff 2024-09-21T07:40:28.880 INFO:teuthology.orchestra.run.smithi039.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:40:28.880 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:40:28.880 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::b079:ddff:fe0e:6333/64 scope link 2024-09-21T07:40:28.880 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:40:28.880 INFO:teuthology.orchestra.run.smithi039.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:40:28.880 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:40:28.880 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::2ca7:98ff:fe8a:d0ed/64 scope link 2024-09-21T07:40:28.880 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:40:28.880 INFO:teuthology.orchestra.run.smithi039.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:40:28.880 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether ee:10:53:01:0c:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:40:28.880 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec10:53ff:fe01:c44/64 scope link 2024-09-21T07:40:28.881 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:40:28.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:40:28.882 DEBUG:teuthology.orchestra.run.smithi039:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:40:28.975 INFO:teuthology.orchestra.run.smithi039.stdout:1 2024-09-21T07:40:28.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:40:28.979 DEBUG:teuthology.orchestra.run.smithi039:> route 2024-09-21T07:40:29.015 INFO:teuthology.orchestra.run.smithi039.stdout:Kernel IP routing table 2024-09-21T07:40:29.015 INFO:teuthology.orchestra.run.smithi039.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:40:29.015 INFO:teuthology.orchestra.run.smithi039.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T07:40:29.015 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T07:40:29.015 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:40:29.015 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:40:29.016 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-21T07:40:29.016 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T07:40:29.016 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T07:40:29.016 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T07:40:29.016 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-21T07:40:29.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:40:29.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:40:29.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:40:29.214 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-09-21T07:40:29.250 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:40:29.250 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T07:40:29.250 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:40:29.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:40:29.251 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-09-21T07:40:29.347 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:40:29.347 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T07:40:29.382 INFO:teuthology.orchestra.run.smithi039.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:40:29.382 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:40:29.382 DEBUG:teuthology.orchestra.run.smithi039:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:40:29.456 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-21T07:40:29.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:40:29.456 DEBUG:teuthology.orchestra.run.smithi039:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:40:29.656 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:40:29.656 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:40:29.693 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:40:29.694 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:40:29.694 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: resolved to: "172.21.15.39:3300,172.21.15.148:3300,172.21.15.148:3301" 2024-09-21T07:40:29.694 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: trying mount with new device syntax: 0@7ab9b04a-f88d-49aa-a493-e5c076637ffb.cephfs=/ 2024-09-21T07:40:29.694 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.39:3300/172.21.15.148:3300/172.21.15.148:3301" will pass to kernel 2024-09-21T07:40:29.696 INFO:teuthology.orchestra.run.smithi039.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T07:40:29.696 INFO:teuthology.orchestra.run.smithi039.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:40:29.696 INFO:teuthology.orchestra.run.smithi039.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:40:29.696 INFO:teuthology.orchestra.run.smithi039.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:40:29.696 INFO:teuthology.orchestra.run.smithi039.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:40:29.699 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:40:29.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:40:29.699 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:40:29.754 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:40:29.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:40:29.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:40:29.844 INFO:teuthology.orchestra.run.smithi039.stdout:client7368 2024-09-21T07:40:29.845 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:40:29.845 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:40:29.880 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 flush journal 2024-09-21T07:40:30.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.149+0000 7f4bc4400640 1 -- 172.21.15.39:0/2731327430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bbc0b32b0 msgr2=0x7f4bbc0cbb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:30.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.149+0000 7f4bc4400640 1 --2- 172.21.15.39:0/2731327430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bbc0b32b0 0x7f4bbc0cbb10 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8009830 tx=0x7f4ba802a480 comp rx=0 tx=0).stop 2024-09-21T07:40:30.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.149+0000 7f4bc4400640 1 -- 172.21.15.39:0/2731327430 shutdown_connections 2024-09-21T07:40:30.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.149+0000 7f4bc4400640 1 --2- 172.21.15.39:0/2731327430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bbc0b32b0 0x7f4bbc0cbb10 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:30.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.149+0000 7f4bc4400640 1 -- 172.21.15.39:0/2731327430 >> 172.21.15.39:0/2731327430 conn(0x7f4bbc06d440 msgr2=0x7f4bbc0b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:30.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.149+0000 7f4bc4400640 1 -- 172.21.15.39:0/2731327430 shutdown_connections 2024-09-21T07:40:30.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.150+0000 7f4bc4400640 1 -- 172.21.15.39:0/2731327430 wait complete. 2024-09-21T07:40:30.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.152+0000 7f4bc4400640 1 Processor -- start 2024-09-21T07:40:30.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.152+0000 7f4bc4400640 1 -- start start 2024-09-21T07:40:30.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.153+0000 7f4bc4400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bbc0b32b0 0x7f4bbc152010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:30.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.153+0000 7f4bc4400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4bbc0cce70 0x7f4bbc1f90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:30.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.153+0000 7f4bc4400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bbc15cf60 0x7f4bbc1f9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:30.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.153+0000 7f4bc4400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc06fad0 con 0x7f4bbc15cf60 2024-09-21T07:40:30.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.153+0000 7f4bc4400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc06f950 con 0x7f4bbc0b32b0 2024-09-21T07:40:30.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.153+0000 7f4bc4400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f4bbc06fc50 con 0x7f4bbc0cce70 2024-09-21T07:40:30.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.154+0000 7f4bc2400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bbc15cf60 0x7f4bbc1f9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:30.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.154+0000 7f4bc2400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bbc15cf60 0x7f4bbc1f9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53290/0 (socket says 172.21.15.39:53290) 2024-09-21T07:40:30.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.154+0000 7f4bc2400640 1 -- 172.21.15.39:0/1595281380 learned_addr learned my addr 172.21.15.39:0/1595281380 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:40:30.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.154+0000 7f4bc1a00640 1 --2- 172.21.15.39:0/1595281380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bbc0b32b0 0x7f4bbc152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:30.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.154+0000 7f4bc1000640 1 --2- 172.21.15.39:0/1595281380 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4bbc0cce70 0x7f4bbc1f90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:30.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.154+0000 7f4bc2400640 1 -- 172.21.15.39:0/1595281380 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4bbc0cce70 msgr2=0x7f4bbc1f90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:30.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.154+0000 7f4bc2400640 1 --2- 172.21.15.39:0/1595281380 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4bbc0cce70 0x7f4bbc1f90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:30.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.154+0000 7f4bc2400640 1 -- 172.21.15.39:0/1595281380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bbc0b32b0 msgr2=0x7f4bbc152010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:30.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.154+0000 7f4bc2400640 1 --2- 172.21.15.39:0/1595281380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bbc0b32b0 0x7f4bbc152010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:30.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.154+0000 7f4bc2400640 1 -- 172.21.15.39:0/1595281380 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bbc09b590 con 0x7f4bbc15cf60 2024-09-21T07:40:30.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.154+0000 7f4bc1a00640 1 --2- 172.21.15.39:0/1595281380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bbc0b32b0 0x7f4bbc152010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:40:30.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.155+0000 7f4bc2400640 1 --2- 172.21.15.39:0/1595281380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bbc15cf60 0x7f4bbc1f9630 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f4bb400b9f0 tx=0x7f4bb400beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:30.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.155+0000 7f4bb9600640 1 -- 172.21.15.39:0/1595281380 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4019070 con 0x7f4bbc15cf60 2024-09-21T07:40:30.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.155+0000 7f4bb9600640 1 -- 172.21.15.39:0/1595281380 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4bb400cc60 con 0x7f4bbc15cf60 2024-09-21T07:40:30.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.155+0000 7f4bc4400640 1 -- 172.21.15.39:0/1595281380 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bbc09b870 con 0x7f4bbc15cf60 2024-09-21T07:40:30.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.155+0000 7f4bb9600640 1 -- 172.21.15.39:0/1595281380 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb40146a0 con 0x7f4bbc15cf60 2024-09-21T07:40:30.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.155+0000 7f4bc4400640 1 -- 172.21.15.39:0/1595281380 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bbc160620 con 0x7f4bbc15cf60 2024-09-21T07:40:30.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.157+0000 7f4bb9600640 1 -- 172.21.15.39:0/1595281380 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4bb401c070 con 0x7f4bbc15cf60 2024-09-21T07:40:30.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.158+0000 7f4bb9600640 1 --2- 172.21.15.39:0/1595281380 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4b9c078bc0 0x7f4b9c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:30.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.159+0000 7f4bb9600640 1 -- 172.21.15.39:0/1595281380 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7603+0+0 (secure 0 0 0) 0x7f4bb409b300 con 0x7f4bbc15cf60 2024-09-21T07:40:30.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.159+0000 7f4bc1a00640 1 --2- 172.21.15.39:0/1595281380 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4b9c078bc0 0x7f4b9c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:30.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.159+0000 7f4bcb490740 1 Processor -- start 2024-09-21T07:40:30.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.159+0000 7f4bcb490740 1 -- start start 2024-09-21T07:40:30.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.160+0000 7f4bc1a00640 1 --2- 172.21.15.39:0/1595281380 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4b9c078bc0 0x7f4b9c07b0d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0007a50 tx=0x7f4bb0009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:30.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.160+0000 7f4bcb490740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560dc1a8d390 0x560dc1e7b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:30.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.160+0000 7f4bcb490740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560dc1a644d0 0x560dc1e5e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:30.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.160+0000 7f4bcb490740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc1a6c6b0 0x560dc1e5e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:30.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.160+0000 7f4bcb490740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560dc1d941c0 con 0x560dc1a6c6b0 2024-09-21T07:40:30.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.160+0000 7f4bcb490740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560dc1da7870 con 0x560dc1a8d390 2024-09-21T07:40:30.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.160+0000 7f4bcb490740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x560dc1d96ca0 con 0x560dc1a644d0 2024-09-21T07:40:30.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.161+0000 7f4bc2400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc1a6c6b0 0x560dc1e5e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:30.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.161+0000 7f4bc1a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560dc1a8d390 0x560dc1e7b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:30.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.161+0000 7f4bc2400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc1a6c6b0 0x560dc1e5e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53300/0 (socket says 172.21.15.39:53300) 2024-09-21T07:40:30.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.161+0000 7f4bc1000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560dc1a644d0 0x560dc1e5e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:30.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.161+0000 7f4bc2400640 1 -- 172.21.15.39:0/870742227 learned_addr learned my addr 172.21.15.39:0/870742227 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:40:30.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.161+0000 7f4bc2400640 1 -- 172.21.15.39:0/870742227 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560dc1a644d0 msgr2=0x560dc1e5e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:40:30.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.161+0000 7f4bc2400640 1 --2- 172.21.15.39:0/870742227 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560dc1a644d0 0x560dc1e5e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:30.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.161+0000 7f4bc2400640 1 -- 172.21.15.39:0/870742227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560dc1a8d390 msgr2=0x560dc1e7b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:40:30.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.161+0000 7f4bc2400640 1 --2- 172.21.15.39:0/870742227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560dc1a8d390 0x560dc1e7b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:30.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.161+0000 7f4bc2400640 1 -- 172.21.15.39:0/870742227 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dc1b6f7d0 con 0x560dc1a6c6b0 2024-09-21T07:40:30.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.162+0000 7f4bc2400640 1 --2- 172.21.15.39:0/870742227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc1a6c6b0 0x560dc1e5e760 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4061fa0 tx=0x7f4bb4062090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b6280ba92941f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:30.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.162+0000 7f4bae000640 1 -- 172.21.15.39:0/870742227 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4067310 con 0x560dc1a6c6b0 2024-09-21T07:40:30.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.162+0000 7f4bae000640 1 -- 172.21.15.39:0/870742227 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4bb406c250 con 0x560dc1a6c6b0 2024-09-21T07:40:30.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.162+0000 7f4bae000640 1 -- 172.21.15.39:0/870742227 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb406bdc0 con 0x560dc1a6c6b0 2024-09-21T07:40:30.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.163+0000 7f4bcb490740 1 -- 172.21.15.39:0/870742227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc1a6c6b0 msgr2=0x560dc1e5e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:30.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.163+0000 7f4bcb490740 1 --2- 172.21.15.39:0/870742227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc1a6c6b0 0x560dc1e5e760 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4061fa0 tx=0x7f4bb4062090 comp rx=0 tx=0).stop 2024-09-21T07:40:30.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.163+0000 7f4bcb490740 1 -- 172.21.15.39:0/870742227 shutdown_connections 2024-09-21T07:40:30.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.163+0000 7f4bcb490740 1 --2- 172.21.15.39:0/870742227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc1a6c6b0 0x560dc1e5e760 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:30.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.163+0000 7f4bcb490740 1 --2- 172.21.15.39:0/870742227 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560dc1a644d0 0x560dc1e5e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:30.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.163+0000 7f4bcb490740 1 --2- 172.21.15.39:0/870742227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560dc1a8d390 0x560dc1e7b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:30.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.163+0000 7f4bcb490740 1 -- 172.21.15.39:0/870742227 >> 172.21.15.39:0/870742227 conn(0x560dc1a814e0 msgr2=0x560dc1e34440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:30.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.163+0000 7f4bcb490740 1 -- 172.21.15.39:0/870742227 shutdown_connections 2024-09-21T07:40:30.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.163+0000 7f4bcb490740 1 -- 172.21.15.39:0/870742227 wait complete. 2024-09-21T07:40:30.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.164+0000 7f4bcb490740 1 -- start start 2024-09-21T07:40:30.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.164+0000 7f4bcb490740 1 Processor -- start 2024-09-21T07:40:30.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.165+0000 7f4bcb490740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560dc1a644d0 0x560dc1e774d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:30.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.165+0000 7f4bcb490740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc1a6c6b0 0x560dc1e4cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:30.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.166+0000 7f4bcb490740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560dc1a8d390 0x560dc1e788f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:30.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.166+0000 7f4bcb490740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560dc1d3d410 con 0x560dc1a6c6b0 2024-09-21T07:40:30.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.166+0000 7f4bcb490740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560dc1d9aab0 con 0x560dc1a8d390 2024-09-21T07:40:30.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.166+0000 7f4bcb490740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x560dc1b07740 con 0x560dc1a644d0 2024-09-21T07:40:30.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.166+0000 7f4bcb490740 10 client.0 _ll_register_callbacks cb 0x560dc1d1a360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:40:30.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.166+0000 7f4bc1000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc1a6c6b0 0x560dc1e4cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:30.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.166+0000 7f4bc1000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc1a6c6b0 0x560dc1e4cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53310/0 (socket says 172.21.15.39:53310) 2024-09-21T07:40:30.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.166+0000 7f4bc1000640 1 -- 172.21.15.39:0/2631849540 learned_addr learned my addr 172.21.15.39:0/2631849540 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:40:30.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.166+0000 7f4bc1a00640 1 --2- 172.21.15.39:0/2631849540 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560dc1a644d0 0x560dc1e774d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:30.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.166+0000 7f4bc2400640 1 --2- 172.21.15.39:0/2631849540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560dc1a8d390 0x560dc1e788f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:30.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.166+0000 7f4bc1000640 1 -- 172.21.15.39:0/2631849540 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560dc1a644d0 msgr2=0x560dc1e774d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:40:30.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.166+0000 7f4bc1000640 1 --2- 172.21.15.39:0/2631849540 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560dc1a644d0 0x560dc1e774d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:30.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.166+0000 7f4bc1000640 1 -- 172.21.15.39:0/2631849540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560dc1a8d390 msgr2=0x560dc1e788f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:40:30.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.166+0000 7f4bc1000640 1 --2- 172.21.15.39:0/2631849540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560dc1a8d390 0x560dc1e788f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:30.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.166+0000 7f4bc1000640 1 -- 172.21.15.39:0/2631849540 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dc1af8970 con 0x560dc1a6c6b0 2024-09-21T07:40:30.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.166+0000 7f4bc1a00640 1 --2- 172.21.15.39:0/2631849540 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560dc1a644d0 0x560dc1e774d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:40:30.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.166+0000 7f4bc2400640 1 --2- 172.21.15.39:0/2631849540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560dc1a8d390 0x560dc1e788f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:40:30.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.167+0000 7f4bc1000640 1 --2- 172.21.15.39:0/2631849540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc1a6c6b0 0x560dc1e4cf90 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f4ba802aa70 tx=0x7f4ba8003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72cb52af3a1a871 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:30.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.167+0000 7f4ba2a00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:40:30.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.167+0000 7f4ba2a00640 1 -- 172.21.15.39:0/2631849540 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba8003ba0 con 0x560dc1a6c6b0 2024-09-21T07:40:30.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.167+0000 7f4ba2a00640 1 -- 172.21.15.39:0/2631849540 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 7603+0+0 (secure 0 0 0) 0x7f4ba8003d40 con 0x560dc1a6c6b0 2024-09-21T07:40:30.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.167+0000 7f4bcb490740 1 -- 172.21.15.39:0/2631849540 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bb406c250 con 0x560dc1a6c6b0 2024-09-21T07:40:30.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.167+0000 7f4ba2a00640 1 -- 172.21.15.39:0/2631849540 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) ==== 24+0+0 (secure 0 0 0) 0x7f4ba804e050 con 0x560dc1a6c6b0 2024-09-21T07:40:30.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.167+0000 7f4bcb490740 10 client.7383 fetch_fsmap learned FSMap version 114 2024-09-21T07:40:30.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.167+0000 7f4bcb490740 1 -- 172.21.15.39:0/2631849540 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=2+}) -- 0x560dc1ad66a0 con 0x560dc1a6c6b0 2024-09-21T07:40:30.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.167+0000 7f4ba2a00640 1 -- 172.21.15.39:0/2631849540 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4bb406c250 con 0x560dc1a6c6b0 2024-09-21T07:40:30.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.168+0000 7f4ba2a00640 1 -- 172.21.15.39:0/2631849540 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 114) ==== 3522+0+0 (secure 0 0 0) 0x7f4ba8051020 con 0x560dc1a6c6b0 2024-09-21T07:40:30.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.168+0000 7f4bcb490740 10 client.7383 fetch_fsmap finished waiting for FSMap version 114 2024-09-21T07:40:30.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.168+0000 7f4bcb490740 10 client.7383 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-09-21T07:40:30.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.168+0000 7f4bcb490740 20 client.7383 populate_metadata read hostname 'smithi039' 2024-09-21T07:40:30.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.168+0000 7f4bcb490740 1 --2- 172.21.15.39:0/2631849540 >> [v2:172.21.15.39:6834/2821446711,v1:172.21.15.39:6835/2821446711] conn(0x560dc1a4bf60 0x560dc1e743b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:30.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.168+0000 7f4bcb490740 4 client.7383 mds_command: new command op to 7113 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:40:30.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.168+0000 7f4bcb490740 1 -- 172.21.15.39:0/2631849540 --> [v2:172.21.15.39:6834/2821446711,v1:172.21.15.39:6835/2821446711] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560dc1da97c0 con 0x560dc1a4bf60 2024-09-21T07:40:30.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.168+0000 7f4bc1a00640 1 --2- 172.21.15.39:0/2631849540 >> [v2:172.21.15.39:6834/2821446711,v1:172.21.15.39:6835/2821446711] conn(0x560dc1a4bf60 0x560dc1e743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:30.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.169+0000 7f4bc1a00640 1 --2- 172.21.15.39:0/2631849540 >> [v2:172.21.15.39:6834/2821446711,v1:172.21.15.39:6835/2821446711] conn(0x560dc1a4bf60 0x560dc1e743b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e013f0ddfa08b082 server_cookie=5fa1126d90ff067d in_seq=0 out_seq=0 2024-09-21T07:40:30.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.169+0000 7f4ba2a00640 10 client.7383 ms_handle_connect on v2:172.21.15.39:6834/2821446711 2024-09-21T07:40:30.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.169+0000 7f4ba2a00640 1 -- 172.21.15.39:0/2631849540 <== mds.0 v2:172.21.15.39:6834/2821446711 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560dc1da97c0 con 0x560dc1a4bf60 2024-09-21T07:40:30.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.169+0000 7f4ba2a00640 10 client.7383 handle_command_reply: tid=0 2024-09-21T07:40:30.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.169+0000 7f4bcb490740 1 client.7383 shutdown 2024-09-21T07:40:30.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.170+0000 7f4bcb490740 1 -- 172.21.15.39:0/2631849540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc1a6c6b0 msgr2=0x560dc1e4cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:30.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.170+0000 7f4bcb490740 1 --2- 172.21.15.39:0/2631849540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc1a6c6b0 0x560dc1e4cf90 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f4ba802aa70 tx=0x7f4ba8003820 comp rx=0 tx=0).stop 2024-09-21T07:40:30.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.170+0000 7f4bcb490740 1 -- 172.21.15.39:0/2631849540 shutdown_connections 2024-09-21T07:40:30.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.170+0000 7f4bcb490740 1 --2- 172.21.15.39:0/2631849540 >> [v2:172.21.15.39:6834/2821446711,v1:172.21.15.39:6835/2821446711] conn(0x560dc1a4bf60 0x560dc1e743b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:30.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.170+0000 7f4bcb490740 1 --2- 172.21.15.39:0/2631849540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560dc1a8d390 0x560dc1e788f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:30.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.170+0000 7f4bcb490740 1 --2- 172.21.15.39:0/2631849540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc1a6c6b0 0x560dc1e4cf90 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:30.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.170+0000 7f4bcb490740 1 --2- 172.21.15.39:0/2631849540 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560dc1a644d0 0x560dc1e774d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:30.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.170+0000 7f4ba2a00640 0 client.7383 ms_handle_reset on v2:172.21.15.39:6834/2821446711 2024-09-21T07:40:30.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.170+0000 7f4bcb490740 1 -- 172.21.15.39:0/2631849540 >> 172.21.15.39:0/2631849540 conn(0x560dc1a814e0 msgr2=0x560dc1e2eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:30.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.170+0000 7f4bcb490740 1 -- 172.21.15.39:0/2631849540 shutdown_connections 2024-09-21T07:40:30.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.170+0000 7f4bcb490740 1 -- 172.21.15.39:0/2631849540 wait complete. 2024-09-21T07:40:30.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.170+0000 7f4bcb490740 20 client.7383 trim_cache size 0 max 16384 2024-09-21T07:40:30.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.186+0000 7f4bcb490740 1 Processor -- start 2024-09-21T07:40:30.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.186+0000 7f4bcb490740 1 -- start start 2024-09-21T07:40:30.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.187+0000 7f4bcb490740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560dc1a644d0 0x560dc1e9c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:30.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.187+0000 7f4bcb490740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc1a6c6b0 0x560dc1e4c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:30.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.187+0000 7f4bcb490740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560dc1a8d390 0x560dc1e5e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:30.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.187+0000 7f4bcb490740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc2049e0 con 0x560dc1a6c6b0 2024-09-21T07:40:30.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.187+0000 7f4bcb490740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc2050b0 con 0x560dc1a644d0 2024-09-21T07:40:30.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.187+0000 7f4bcb490740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f4bbc203470 con 0x560dc1a8d390 2024-09-21T07:40:30.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.188+0000 7f4bc1000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc1a6c6b0 0x560dc1e4c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:30.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.188+0000 7f4bc1000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc1a6c6b0 0x560dc1e4c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53326/0 (socket says 172.21.15.39:53326) 2024-09-21T07:40:30.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.188+0000 7f4bc1000640 1 -- 172.21.15.39:0/658139658 learned_addr learned my addr 172.21.15.39:0/658139658 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:40:30.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.188+0000 7f4bc1a00640 1 --2- 172.21.15.39:0/658139658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560dc1a644d0 0x560dc1e9c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:30.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.188+0000 7f4bc2400640 1 --2- 172.21.15.39:0/658139658 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560dc1a8d390 0x560dc1e5e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:30.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.188+0000 7f4bc1000640 1 -- 172.21.15.39:0/658139658 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560dc1a8d390 msgr2=0x560dc1e5e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:40:30.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.188+0000 7f4bc1000640 1 --2- 172.21.15.39:0/658139658 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560dc1a8d390 0x560dc1e5e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:30.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.188+0000 7f4bc1000640 1 -- 172.21.15.39:0/658139658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560dc1a644d0 msgr2=0x560dc1e9c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:40:30.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.188+0000 7f4bc1000640 1 --2- 172.21.15.39:0/658139658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560dc1a644d0 0x560dc1e9c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:30.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.188+0000 7f4bc1000640 1 -- 172.21.15.39:0/658139658 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dc1e4c970 con 0x560dc1a6c6b0 2024-09-21T07:40:30.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.188+0000 7f4bc1a00640 1 --2- 172.21.15.39:0/658139658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560dc1a644d0 0x560dc1e9c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:40:30.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.188+0000 7f4bc1000640 1 --2- 172.21.15.39:0/658139658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc1a6c6b0 0x560dc1e4c430 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8001eb0 tx=0x7f4ba8003890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fcbbef6139ed007 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:30.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.189+0000 7f4bad600640 1 -- 172.21.15.39:0/658139658 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba8003ba0 con 0x560dc1a6c6b0 2024-09-21T07:40:30.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.189+0000 7f4bad600640 1 -- 172.21.15.39:0/658139658 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ba8005260 con 0x560dc1a6c6b0 2024-09-21T07:40:30.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.189+0000 7f4bad600640 1 -- 172.21.15.39:0/658139658 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba8005650 con 0x560dc1a6c6b0 2024-09-21T07:40:30.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.189+0000 7f4bcb490740 1 -- 172.21.15.39:0/658139658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc1a6c6b0 msgr2=0x560dc1e4c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:30.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.189+0000 7f4bcb490740 1 --2- 172.21.15.39:0/658139658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc1a6c6b0 0x560dc1e4c430 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8001eb0 tx=0x7f4ba8003890 comp rx=0 tx=0).stop 2024-09-21T07:40:30.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.189+0000 7f4bcb490740 1 -- 172.21.15.39:0/658139658 shutdown_connections 2024-09-21T07:40:30.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.189+0000 7f4bcb490740 1 --2- 172.21.15.39:0/658139658 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560dc1a8d390 0x560dc1e5e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:30.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.189+0000 7f4bcb490740 1 --2- 172.21.15.39:0/658139658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc1a6c6b0 0x560dc1e4c430 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:30.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.189+0000 7f4bcb490740 1 --2- 172.21.15.39:0/658139658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560dc1a644d0 0x560dc1e9c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:30.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.189+0000 7f4bcb490740 1 -- 172.21.15.39:0/658139658 >> 172.21.15.39:0/658139658 conn(0x560dc1a814e0 msgr2=0x560dc1e74390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:30.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.190+0000 7f4bcb490740 1 -- 172.21.15.39:0/658139658 shutdown_connections 2024-09-21T07:40:30.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.190+0000 7f4bcb490740 1 -- 172.21.15.39:0/658139658 wait complete. 2024-09-21T07:40:30.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.190+0000 7f4bcb490740 1 -- start start 2024-09-21T07:40:30.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.191+0000 7f4bcb490740 1 Processor -- start 2024-09-21T07:40:30.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.192+0000 7f4bcb490740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560dc1a644d0 0x560dc1ef8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:30.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.192+0000 7f4bcb490740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560dc1a6c6b0 0x560dc1e9af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:30.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.192+0000 7f4bcb490740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc1a8d390 0x560dc1e9b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:30.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.192+0000 7f4bcb490740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560dc1b07740 con 0x560dc1a8d390 2024-09-21T07:40:30.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.192+0000 7f4bcb490740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc203ac0 con 0x560dc1a6c6b0 2024-09-21T07:40:30.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.192+0000 7f4bcb490740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x560dc1e5e320 con 0x560dc1a644d0 2024-09-21T07:40:30.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.192+0000 7f4bcb490740 10 client.0 _ll_register_callbacks cb 0x7f4ba80096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:40:30.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.192+0000 7f4bc2400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc1a8d390 0x560dc1e9b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:30.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.192+0000 7f4bc2400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc1a8d390 0x560dc1e9b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53342/0 (socket says 172.21.15.39:53342) 2024-09-21T07:40:30.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.192+0000 7f4bc2400640 1 -- 172.21.15.39:0/1142399201 learned_addr learned my addr 172.21.15.39:0/1142399201 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:40:30.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.193+0000 7f4bc1a00640 1 --2- 172.21.15.39:0/1142399201 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560dc1a644d0 0x560dc1ef8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:30.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.193+0000 7f4bc1000640 1 --2- 172.21.15.39:0/1142399201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560dc1a6c6b0 0x560dc1e9af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:30.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.193+0000 7f4bc2400640 1 -- 172.21.15.39:0/1142399201 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560dc1a644d0 msgr2=0x560dc1ef8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:40:30.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.193+0000 7f4bc2400640 1 --2- 172.21.15.39:0/1142399201 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560dc1a644d0 0x560dc1ef8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:30.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.193+0000 7f4bc2400640 1 -- 172.21.15.39:0/1142399201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560dc1a6c6b0 msgr2=0x560dc1e9af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:40:30.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.193+0000 7f4bc2400640 1 --2- 172.21.15.39:0/1142399201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560dc1a6c6b0 0x560dc1e9af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:30.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.193+0000 7f4bc2400640 1 -- 172.21.15.39:0/1142399201 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dc1e9bc30 con 0x560dc1a8d390 2024-09-21T07:40:30.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.193+0000 7f4bc2400640 1 --2- 172.21.15.39:0/1142399201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc1a8d390 0x560dc1e9b4b0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f4bb40627c0 tx=0x7f4bb4098ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c630ad1eb2ac0219 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:30.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.193+0000 7f4ba2a00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:40:30.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.193+0000 7f4ba2a00640 1 -- 172.21.15.39:0/1142399201 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb409b0c0 con 0x560dc1a8d390 2024-09-21T07:40:30.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.193+0000 7f4ba2a00640 1 -- 172.21.15.39:0/1142399201 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 7603+0+0 (secure 0 0 0) 0x7f4bb4061a90 con 0x560dc1a8d390 2024-09-21T07:40:30.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.193+0000 7f4bcb490740 1 -- 172.21.15.39:0/1142399201 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ba8005260 con 0x560dc1a8d390 2024-09-21T07:40:30.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.194+0000 7f4ba2a00640 1 -- 172.21.15.39:0/1142399201 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) ==== 24+0+0 (secure 0 0 0) 0x7f4bb4067310 con 0x560dc1a8d390 2024-09-21T07:40:30.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.194+0000 7f4bcb490740 10 client.7389 fetch_fsmap learned FSMap version 114 2024-09-21T07:40:30.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.194+0000 7f4bcb490740 1 -- 172.21.15.39:0/1142399201 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=2+}) -- 0x560dc1e9cc80 con 0x560dc1a8d390 2024-09-21T07:40:30.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.194+0000 7f4ba2a00640 1 -- 172.21.15.39:0/1142399201 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ba8005260 con 0x560dc1a8d390 2024-09-21T07:40:30.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.194+0000 7f4ba2a00640 1 -- 172.21.15.39:0/1142399201 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 114) ==== 3522+0+0 (secure 0 0 0) 0x7f4bb405e180 con 0x560dc1a8d390 2024-09-21T07:40:30.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.194+0000 7f4bcb490740 10 client.7389 fetch_fsmap finished waiting for FSMap version 114 2024-09-21T07:40:30.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.194+0000 7f4bcb490740 10 client.7389 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-09-21T07:40:30.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.194+0000 7f4bcb490740 20 client.7389 populate_metadata read hostname 'smithi039' 2024-09-21T07:40:30.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.194+0000 7f4bcb490740 1 --2- 172.21.15.39:0/1142399201 >> [v2:172.21.15.39:6834/2821446711,v1:172.21.15.39:6835/2821446711] conn(0x560dc1e2d830 0x560dc1e2dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:30.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.194+0000 7f4bcb490740 4 client.7389 mds_command: new command op to 7113 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:40:30.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.194+0000 7f4bcb490740 1 -- 172.21.15.39:0/1142399201 --> [v2:172.21.15.39:6834/2821446711,v1:172.21.15.39:6835/2821446711] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560dc1da5270 con 0x560dc1e2d830 2024-09-21T07:40:30.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.194+0000 7f4bc1a00640 1 --2- 172.21.15.39:0/1142399201 >> [v2:172.21.15.39:6834/2821446711,v1:172.21.15.39:6835/2821446711] conn(0x560dc1e2d830 0x560dc1e2dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:30.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.195+0000 7f4ba2a00640 10 client.7389 ms_handle_connect on v2:172.21.15.39:6834/2821446711 2024-09-21T07:40:30.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.195+0000 7f4bc1a00640 1 --2- 172.21.15.39:0/1142399201 >> [v2:172.21.15.39:6834/2821446711,v1:172.21.15.39:6835/2821446711] conn(0x560dc1e2d830 0x560dc1e2dc40 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f517dbf20e76377e server_cookie=6e5c236b0cc3b760 in_seq=0 out_seq=0 2024-09-21T07:40:30.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.205+0000 7f4ba2a00640 1 -- 172.21.15.39:0/1142399201 <== mds.0 v2:172.21.15.39:6834/2821446711 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560dc1da5270 con 0x560dc1e2d830 2024-09-21T07:40:30.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.205+0000 7f4ba2a00640 10 client.7389 handle_command_reply: tid=0 2024-09-21T07:40:30.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.205+0000 7f4bcb490740 1 client.7389 shutdown 2024-09-21T07:40:30.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.206+0000 7f4bcb490740 1 -- 172.21.15.39:0/1142399201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc1a8d390 msgr2=0x560dc1e9b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:30.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.206+0000 7f4bcb490740 1 --2- 172.21.15.39:0/1142399201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc1a8d390 0x560dc1e9b4b0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f4bb40627c0 tx=0x7f4bb4098ef0 comp rx=0 tx=0).stop 2024-09-21T07:40:30.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.206+0000 7f4bcb490740 1 -- 172.21.15.39:0/1142399201 shutdown_connections 2024-09-21T07:40:30.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.206+0000 7f4bcb490740 1 --2- 172.21.15.39:0/1142399201 >> [v2:172.21.15.39:6834/2821446711,v1:172.21.15.39:6835/2821446711] conn(0x560dc1e2d830 0x560dc1e2dc40 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:30.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.206+0000 7f4bcb490740 1 --2- 172.21.15.39:0/1142399201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc1a8d390 0x560dc1e9b4b0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:30.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.206+0000 7f4bcb490740 1 --2- 172.21.15.39:0/1142399201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560dc1a6c6b0 0x560dc1e9af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:30.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.206+0000 7f4ba2a00640 0 client.7389 ms_handle_reset on v2:172.21.15.39:6834/2821446711 2024-09-21T07:40:30.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.206+0000 7f4bcb490740 1 --2- 172.21.15.39:0/1142399201 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x560dc1a644d0 0x560dc1ef8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:30.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.206+0000 7f4bcb490740 1 -- 172.21.15.39:0/1142399201 >> 172.21.15.39:0/1142399201 conn(0x560dc1a814e0 msgr2=0x560dc1e76e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:30.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.206+0000 7f4bcb490740 1 -- 172.21.15.39:0/1142399201 shutdown_connections 2024-09-21T07:40:30.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.206+0000 7f4bcb490740 1 -- 172.21.15.39:0/1142399201 wait complete. 2024-09-21T07:40:30.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.206+0000 7f4bcb490740 20 client.7389 trim_cache size 0 max 16384 2024-09-21T07:40:30.209 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:40:30.209 INFO:teuthology.orchestra.run.smithi039.stdout: "message": "", 2024-09-21T07:40:30.209 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0 2024-09-21T07:40:30.209 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:40:30.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.207+0000 7f4bc4400640 1 -- 172.21.15.39:0/1595281380 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4b9c078bc0 msgr2=0x7f4b9c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:30.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.207+0000 7f4bc4400640 1 --2- 172.21.15.39:0/1595281380 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4b9c078bc0 0x7f4b9c07b0d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0007a50 tx=0x7f4bb0009040 comp rx=0 tx=0).stop 2024-09-21T07:40:30.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.207+0000 7f4bc4400640 1 -- 172.21.15.39:0/1595281380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bbc15cf60 msgr2=0x7f4bbc1f9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:30.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.207+0000 7f4bc4400640 1 --2- 172.21.15.39:0/1595281380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bbc15cf60 0x7f4bbc1f9630 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f4bb400b9f0 tx=0x7f4bb400beb0 comp rx=0 tx=0).stop 2024-09-21T07:40:30.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.207+0000 7f4bc4400640 1 -- 172.21.15.39:0/1595281380 shutdown_connections 2024-09-21T07:40:30.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.207+0000 7f4bc4400640 1 --2- 172.21.15.39:0/1595281380 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4b9c078bc0 0x7f4b9c07b0d0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:30.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.207+0000 7f4bc4400640 1 --2- 172.21.15.39:0/1595281380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bbc15cf60 0x7f4bbc1f9630 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:30.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.207+0000 7f4bc4400640 1 --2- 172.21.15.39:0/1595281380 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4bbc0cce70 0x7f4bbc1f90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:30.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.207+0000 7f4bc4400640 1 --2- 172.21.15.39:0/1595281380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4bbc0b32b0 0x7f4bbc152010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:30.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.207+0000 7f4bc4400640 1 -- 172.21.15.39:0/1595281380 >> 172.21.15.39:0/1595281380 conn(0x7f4bbc06d440 msgr2=0x7f4bbc0d03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:30.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.207+0000 7f4bc4400640 1 -- 172.21.15.39:0/1595281380 shutdown_connections 2024-09-21T07:40:30.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.207+0000 7f4bc4400640 1 -- 172.21.15.39:0/1595281380 wait complete. 2024-09-21T07:40:30.234 INFO:tasks.cephfs.test_snapshots:testing snapserver mds_kill_mdstable_at=6, also kill snapclient 2024-09-21T07:40:30.235 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:40:30.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.458+0000 7f1d43000640 1 -- 172.21.15.39:0/2664327681 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d3c0cc390 msgr2=0x7f1d3c0af600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:30.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.458+0000 7f1d43000640 1 --2- 172.21.15.39:0/2664327681 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d3c0cc390 0x7f1d3c0af600 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1d30009830 tx=0x7f1d3002a480 comp rx=0 tx=0).stop 2024-09-21T07:40:30.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.459+0000 7f1d43000640 1 -- 172.21.15.39:0/2664327681 shutdown_connections 2024-09-21T07:40:30.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.459+0000 7f1d43000640 1 --2- 172.21.15.39:0/2664327681 >> v2:172.21.15.148:3300/0 conn(0x7f1d3c0ce310 0x7f1d3c0ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:30.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.459+0000 7f1d43000640 1 --2- 172.21.15.39:0/2664327681 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d3c0cc390 0x7f1d3c0af600 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:30.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.459+0000 7f1d43000640 1 -- 172.21.15.39:0/2664327681 >> 172.21.15.39:0/2664327681 conn(0x7f1d3c0b2510 msgr2=0x7f1d3c0d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:30.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.459+0000 7f1d43000640 1 -- 172.21.15.39:0/2664327681 shutdown_connections 2024-09-21T07:40:30.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.459+0000 7f1d43000640 1 -- 172.21.15.39:0/2664327681 wait complete. 2024-09-21T07:40:30.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.461+0000 7f1d43000640 1 Processor -- start 2024-09-21T07:40:30.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.461+0000 7f1d43000640 1 -- start start 2024-09-21T07:40:30.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.463+0000 7f1d43000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d3c06f000 0x7f1d3c1f9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:30.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.463+0000 7f1d43000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d3c0cc390 0x7f1d3c15afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:30.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.463+0000 7f1d43000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d3c0ce310 0x7f1d3c0ae420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:30.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.463+0000 7f1d43000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3c070070 con 0x7f1d3c0ce310 2024-09-21T07:40:30.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.463+0000 7f1d43000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3c06fef0 con 0x7f1d3c06f000 2024-09-21T07:40:30.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.463+0000 7f1d43000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f1d3c0cb8a0 con 0x7f1d3c0cc390 2024-09-21T07:40:30.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.463+0000 7f1d42600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d3c0ce310 0x7f1d3c0ae420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:30.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.463+0000 7f1d42600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d3c0ce310 0x7f1d3c0ae420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53354/0 (socket says 172.21.15.39:53354) 2024-09-21T07:40:30.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.463+0000 7f1d42600640 1 -- 172.21.15.39:0/551207943 learned_addr learned my addr 172.21.15.39:0/551207943 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:40:30.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.463+0000 7f1d41200640 1 --2- 172.21.15.39:0/551207943 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d3c0cc390 0x7f1d3c15afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:30.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.463+0000 7f1d41c00640 1 --2- 172.21.15.39:0/551207943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d3c06f000 0x7f1d3c1f9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:30.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.464+0000 7f1d42600640 1 -- 172.21.15.39:0/551207943 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d3c0cc390 msgr2=0x7f1d3c15afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:30.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.464+0000 7f1d42600640 1 --2- 172.21.15.39:0/551207943 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d3c0cc390 0x7f1d3c15afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:30.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.464+0000 7f1d42600640 1 -- 172.21.15.39:0/551207943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d3c06f000 msgr2=0x7f1d3c1f9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:30.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.464+0000 7f1d42600640 1 --2- 172.21.15.39:0/551207943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d3c06f000 0x7f1d3c1f9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:30.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.464+0000 7f1d42600640 1 -- 172.21.15.39:0/551207943 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d3c15cfa0 con 0x7f1d3c0ce310 2024-09-21T07:40:30.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.464+0000 7f1d42600640 1 --2- 172.21.15.39:0/551207943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d3c0ce310 0x7f1d3c0ae420 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f1d3400cb40 tx=0x7f1d3400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:30.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.464+0000 7f1d39600640 1 -- 172.21.15.39:0/551207943 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d34016020 con 0x7f1d3c0ce310 2024-09-21T07:40:30.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.464+0000 7f1d39600640 1 -- 172.21.15.39:0/551207943 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d3400ad30 con 0x7f1d3c0ce310 2024-09-21T07:40:30.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.464+0000 7f1d43000640 1 -- 172.21.15.39:0/551207943 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d3c1e5270 con 0x7f1d3c0ce310 2024-09-21T07:40:30.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.464+0000 7f1d39600640 1 -- 172.21.15.39:0/551207943 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d34012680 con 0x7f1d3c0ce310 2024-09-21T07:40:30.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.464+0000 7f1d43000640 1 -- 172.21.15.39:0/551207943 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d3c1e57d0 con 0x7f1d3c0ce310 2024-09-21T07:40:30.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.465+0000 7f1d43000640 1 -- 172.21.15.39:0/551207943 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d3c0cbde0 con 0x7f1d3c0ce310 2024-09-21T07:40:30.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.466+0000 7f1d39600640 1 -- 172.21.15.39:0/551207943 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1d34019020 con 0x7f1d3c0ce310 2024-09-21T07:40:30.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.467+0000 7f1d39600640 1 --2- 172.21.15.39:0/551207943 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1d1c078b90 0x7f1d1c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:30.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.468+0000 7f1d39600640 1 -- 172.21.15.39:0/551207943 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7603+0+0 (secure 0 0 0) 0x7f1d3409b6b0 con 0x7f1d3c0ce310 2024-09-21T07:40:30.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.468+0000 7f1d41c00640 1 --2- 172.21.15.39:0/551207943 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1d1c078b90 0x7f1d1c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:30.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.469+0000 7f1d41c00640 1 --2- 172.21.15.39:0/551207943 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1d1c078b90 0x7f1d1c07b0a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1d30034000 tx=0x7f1d30004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:30.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.472+0000 7f1d39600640 1 -- 172.21.15.39:0/551207943 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1d34020030 con 0x7f1d3c0ce310 2024-09-21T07:40:30.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.705+0000 7f1d43000640 1 -- 172.21.15.39:0/551207943 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d3c1fa0a0 con 0x7f1d3c0ce310 2024-09-21T07:40:30.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.705+0000 7f1d39600640 1 -- 172.21.15.39:0/551207943 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+5258 (secure 0 0 0) 0x7f1d3c0cec60 con 0x7f1d3c0ce310 2024-09-21T07:40:30.707 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:40:30.707 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":114,"btime":"2024-09-21T07:40:17:883769+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7311,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:40:17.883762+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7113,"mds_1":6269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6269":{"gid":6269,"name":"d","rank":1,"incarnation":110,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6837/2336112936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2336112936},{"type":"v1","addr":"172.21.15.148:6837","nonce":2336112936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7113":{"gid":7113,"name":"a","rank":0,"incarnation":107,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/2821446711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2821446711},{"type":"v1","addr":"172.21.15.39:6835","nonce":2821446711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7113,"qdb_cluster":[6269,7113]},"id":6}]} 2024-09-21T07:40:30.708 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 114 2024-09-21T07:40:30.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.708+0000 7f1d43000640 1 -- 172.21.15.39:0/551207943 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1d1c078b90 msgr2=0x7f1d1c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:30.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.708+0000 7f1d43000640 1 --2- 172.21.15.39:0/551207943 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1d1c078b90 0x7f1d1c07b0a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1d30034000 tx=0x7f1d30004730 comp rx=0 tx=0).stop 2024-09-21T07:40:30.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.708+0000 7f1d43000640 1 -- 172.21.15.39:0/551207943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d3c0ce310 msgr2=0x7f1d3c0ae420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:30.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.708+0000 7f1d43000640 1 --2- 172.21.15.39:0/551207943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d3c0ce310 0x7f1d3c0ae420 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f1d3400cb40 tx=0x7f1d3400a060 comp rx=0 tx=0).stop 2024-09-21T07:40:30.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.708+0000 7f1d43000640 1 -- 172.21.15.39:0/551207943 shutdown_connections 2024-09-21T07:40:30.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.708+0000 7f1d43000640 1 --2- 172.21.15.39:0/551207943 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1d1c078b90 0x7f1d1c07b0a0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:30.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.708+0000 7f1d43000640 1 --2- 172.21.15.39:0/551207943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d3c0ce310 0x7f1d3c0ae420 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:30.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.708+0000 7f1d43000640 1 --2- 172.21.15.39:0/551207943 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d3c0cc390 0x7f1d3c15afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:30.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.708+0000 7f1d43000640 1 --2- 172.21.15.39:0/551207943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d3c06f000 0x7f1d3c1f9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:30.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.708+0000 7f1d43000640 1 -- 172.21.15.39:0/551207943 >> 172.21.15.39:0/551207943 conn(0x7f1d3c0b2510 msgr2=0x7f1d3c074270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:30.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.708+0000 7f1d43000640 1 -- 172.21.15.39:0/551207943 shutdown_connections 2024-09-21T07:40:30.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:30.708+0000 7f1d43000640 1 -- 172.21.15.39:0/551207943 wait complete. 2024-09-21T07:40:30.735 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump snaps --server 2024-09-21T07:40:30.983 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_snap":11,"last_created":11,"last_destroyed":9,"pending_noop":[],"snaps":[{"snapid":11,"ino":1099511627777,"stamp":"2024-09-21T07:39:42.151005+0000","name":"s25","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2024-09-21T07:40:30.984 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 11, "last_destroyed": 9, "last_snap": 11, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s25", "snapid": 11, "stamp": "2024-09-21T07:39:42.151005+0000" } ] } 2024-09-21T07:40:30.984 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:0 true 2024-09-21T07:40:31.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.214+0000 7ff4e3200640 1 -- 172.21.15.39:0/3465293038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4c8005270 msgr2=0x7ff4c800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:31.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.214+0000 7ff4e3200640 1 --2- 172.21.15.39:0/3465293038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4c8005270 0x7ff4c800b9c0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7ff4e40d0ea0 tx=0x7ff4c8005830 comp rx=0 tx=0).stop 2024-09-21T07:40:31.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.214+0000 7ff4e3200640 1 -- 172.21.15.39:0/3465293038 shutdown_connections 2024-09-21T07:40:31.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.214+0000 7ff4e3200640 1 --2- 172.21.15.39:0/3465293038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4c8005270 0x7ff4c800b9c0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:31.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.214+0000 7ff4e3200640 1 -- 172.21.15.39:0/3465293038 >> 172.21.15.39:0/3465293038 conn(0x7ff4e40b25f0 msgr2=0x7ff4e40d01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:31.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.214+0000 7ff4e3200640 1 -- 172.21.15.39:0/3465293038 shutdown_connections 2024-09-21T07:40:31.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.214+0000 7ff4e3200640 1 -- 172.21.15.39:0/3465293038 wait complete. 2024-09-21T07:40:31.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.217+0000 7ff4e3200640 1 Processor -- start 2024-09-21T07:40:31.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.217+0000 7ff4e3200640 1 -- start start 2024-09-21T07:40:31.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.218+0000 7ff4e3200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4c8005270 0x7ff4e40c5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:31.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.218+0000 7ff4e3200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4e415cd40 0x7ff4e40c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:31.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.218+0000 7ff4e3200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4e40cb200 0x7ff4e40cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:31.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.218+0000 7ff4e3200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e40b3580 con 0x7ff4e415cd40 2024-09-21T07:40:31.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.218+0000 7ff4e3200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e40cca50 con 0x7ff4e40cb200 2024-09-21T07:40:31.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.218+0000 7ff4e3200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7ff4e40b3700 con 0x7ff4c8005270 2024-09-21T07:40:31.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.219+0000 7ff4dbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4e415cd40 0x7ff4e40c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:31.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.219+0000 7ff4dbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4e415cd40 0x7ff4e40c9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53370/0 (socket says 172.21.15.39:53370) 2024-09-21T07:40:31.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.219+0000 7ff4dbe00640 1 -- 172.21.15.39:0/1980285508 learned_addr learned my addr 172.21.15.39:0/1980285508 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:40:31.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.219+0000 7ff4e1e00640 1 --2- 172.21.15.39:0/1980285508 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4c8005270 0x7ff4e40c5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:31.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.219+0000 7ff4e2800640 1 --2- 172.21.15.39:0/1980285508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4e40cb200 0x7ff4e40cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:31.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.219+0000 7ff4dbe00640 1 -- 172.21.15.39:0/1980285508 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4c8005270 msgr2=0x7ff4e40c5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:31.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.219+0000 7ff4dbe00640 1 --2- 172.21.15.39:0/1980285508 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4c8005270 0x7ff4e40c5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:31.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.219+0000 7ff4dbe00640 1 -- 172.21.15.39:0/1980285508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4e40cb200 msgr2=0x7ff4e40cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:31.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.219+0000 7ff4dbe00640 1 --2- 172.21.15.39:0/1980285508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4e40cb200 0x7ff4e40cb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:31.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.219+0000 7ff4dbe00640 1 -- 172.21.15.39:0/1980285508 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e40cbb50 con 0x7ff4e415cd40 2024-09-21T07:40:31.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.219+0000 7ff4dbe00640 1 --2- 172.21.15.39:0/1980285508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4e415cd40 0x7ff4e40c9b00 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc004f00 tx=0x7ff4cc001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:31.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.220+0000 7ff4da000640 1 -- 172.21.15.39:0/1980285508 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc015020 con 0x7ff4e415cd40 2024-09-21T07:40:31.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.220+0000 7ff4da000640 1 -- 172.21.15.39:0/1980285508 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4cc011960 con 0x7ff4e415cd40 2024-09-21T07:40:31.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.220+0000 7ff4e3200640 1 -- 172.21.15.39:0/1980285508 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4e414ad70 con 0x7ff4e415cd40 2024-09-21T07:40:31.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.220+0000 7ff4da000640 1 -- 172.21.15.39:0/1980285508 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc011d90 con 0x7ff4e415cd40 2024-09-21T07:40:31.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.220+0000 7ff4e3200640 1 -- 172.21.15.39:0/1980285508 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4e414b330 con 0x7ff4e415cd40 2024-09-21T07:40:31.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.221+0000 7ff4e3200640 1 -- 172.21.15.39:0/1980285508 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4e414aae0 con 0x7ff4e415cd40 2024-09-21T07:40:31.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.222+0000 7ff4da000640 1 -- 172.21.15.39:0/1980285508 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff4cc018020 con 0x7ff4e415cd40 2024-09-21T07:40:31.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.223+0000 7ff4da000640 1 --2- 172.21.15.39:0/1980285508 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff4c0078b90 0x7ff4c007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:31.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.223+0000 7ff4da000640 1 -- 172.21.15.39:0/1980285508 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7603+0+0 (secure 0 0 0) 0x7ff4cc09beb0 con 0x7ff4e415cd40 2024-09-21T07:40:31.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.223+0000 7ff4e1e00640 1 --2- 172.21.15.39:0/1980285508 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff4c0078b90 0x7ff4c007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:31.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.224+0000 7ff4e1e00640 1 --2- 172.21.15.39:0/1980285508 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff4c0078b90 0x7ff4c007b0a0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4001eb0 tx=0x7ff4d4006040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:31.229 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:40:31.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.228+0000 7ff4da000640 1 -- 172.21.15.39:0/1980285508 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff4cc01f020 con 0x7ff4e415cd40 2024-09-21T07:40:31.233 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:40:31.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.507+0000 7ff4e3200640 1 -- 172.21.15.39:0/1980285508 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"} v 0) -- 0x7ff4e40b1820 con 0x7ff4e415cd40 2024-09-21T07:40:31.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.927+0000 7ff4da000640 1 -- 172.21.15.39:0/1980285508 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"}]=0 freezing mds.7113 v115) ==== 112+0+0 (secure 0 0 0) 0x7ff4e40b1820 con 0x7ff4e415cd40 2024-09-21T07:40:31.929 INFO:teuthology.orchestra.run.smithi039.stderr:freezing mds.7113 2024-09-21T07:40:31.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.930+0000 7ff4e3200640 1 -- 172.21.15.39:0/1980285508 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff4c0078b90 msgr2=0x7ff4c007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:31.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.930+0000 7ff4e3200640 1 --2- 172.21.15.39:0/1980285508 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff4c0078b90 0x7ff4c007b0a0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4001eb0 tx=0x7ff4d4006040 comp rx=0 tx=0).stop 2024-09-21T07:40:31.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.930+0000 7ff4e3200640 1 -- 172.21.15.39:0/1980285508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4e415cd40 msgr2=0x7ff4e40c9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:31.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.930+0000 7ff4e3200640 1 --2- 172.21.15.39:0/1980285508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4e415cd40 0x7ff4e40c9b00 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc004f00 tx=0x7ff4cc001630 comp rx=0 tx=0).stop 2024-09-21T07:40:31.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.930+0000 7ff4e3200640 1 -- 172.21.15.39:0/1980285508 shutdown_connections 2024-09-21T07:40:31.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.930+0000 7ff4e3200640 1 --2- 172.21.15.39:0/1980285508 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff4c0078b90 0x7ff4c007b0a0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:31.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.930+0000 7ff4e3200640 1 --2- 172.21.15.39:0/1980285508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4e40cb200 0x7ff4e40cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:31.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.930+0000 7ff4e3200640 1 --2- 172.21.15.39:0/1980285508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4e415cd40 0x7ff4e40c9b00 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:31.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.930+0000 7ff4e3200640 1 --2- 172.21.15.39:0/1980285508 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4c8005270 0x7ff4e40c5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:31.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.930+0000 7ff4e3200640 1 -- 172.21.15.39:0/1980285508 >> 172.21.15.39:0/1980285508 conn(0x7ff4e40b25f0 msgr2=0x7ff4e40ccc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:31.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.930+0000 7ff4e3200640 1 -- 172.21.15.39:0/1980285508 shutdown_connections 2024-09-21T07:40:31.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:31.930+0000 7ff4e3200640 1 -- 172.21.15.39:0/1980285508 wait complete. 2024-09-21T07:40:31.957 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:1 true 2024-09-21T07:40:32.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.182+0000 7fa165600640 1 -- 172.21.15.39:0/2690879088 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa16015cf60 msgr2=0x7fa1600b32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:32.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.182+0000 7fa165600640 1 --2- 172.21.15.39:0/2690879088 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa16015cf60 0x7fa1600b32b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa150009830 tx=0x7fa15002a480 comp rx=0 tx=0).stop 2024-09-21T07:40:32.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.182+0000 7fa165600640 1 -- 172.21.15.39:0/2690879088 shutdown_connections 2024-09-21T07:40:32.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.182+0000 7fa165600640 1 --2- 172.21.15.39:0/2690879088 >> v2:172.21.15.148:3300/0 conn(0x7fa1600cce70 0x7fa1600cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:32.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.182+0000 7fa165600640 1 --2- 172.21.15.39:0/2690879088 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa16015cf60 0x7fa1600b32b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:32.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.182+0000 7fa165600640 1 -- 172.21.15.39:0/2690879088 >> 172.21.15.39:0/2690879088 conn(0x7fa16006d440 msgr2=0x7fa1600b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:32.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.183+0000 7fa165600640 1 -- 172.21.15.39:0/2690879088 shutdown_connections 2024-09-21T07:40:32.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.183+0000 7fa165600640 1 -- 172.21.15.39:0/2690879088 wait complete. 2024-09-21T07:40:32.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.185+0000 7fa165600640 1 Processor -- start 2024-09-21T07:40:32.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.185+0000 7fa165600640 1 -- start start 2024-09-21T07:40:32.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.186+0000 7fa165600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1600cce70 0x7fa1600cfba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:32.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.186+0000 7fa165600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa16015cf60 0x7fa160160330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:32.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.186+0000 7fa165600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa16015d950 0x7fa160160870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:32.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.186+0000 7fa165600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa16006fa20 con 0x7fa1600cce70 2024-09-21T07:40:32.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.186+0000 7fa165600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa16006f8a0 con 0x7fa16015d950 2024-09-21T07:40:32.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.186+0000 7fa165600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa16006fba0 con 0x7fa16015cf60 2024-09-21T07:40:32.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.187+0000 7fa15ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1600cce70 0x7fa1600cfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:32.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.187+0000 7fa15ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1600cce70 0x7fa1600cfba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35640/0 (socket says 172.21.15.39:35640) 2024-09-21T07:40:32.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.187+0000 7fa15ea00640 1 -- 172.21.15.39:0/3709421464 learned_addr learned my addr 172.21.15.39:0/3709421464 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:40:32.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.187+0000 7fa15f400640 1 --2- 172.21.15.39:0/3709421464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa16015d950 0x7fa160160870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:32.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.187+0000 7fa156000640 1 --2- 172.21.15.39:0/3709421464 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa16015cf60 0x7fa160160330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:32.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.187+0000 7fa15ea00640 1 -- 172.21.15.39:0/3709421464 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa16015cf60 msgr2=0x7fa160160330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:32.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.187+0000 7fa15ea00640 1 --2- 172.21.15.39:0/3709421464 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa16015cf60 0x7fa160160330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:32.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.187+0000 7fa15ea00640 1 -- 172.21.15.39:0/3709421464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa16015d950 msgr2=0x7fa160160870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:32.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.187+0000 7fa15ea00640 1 --2- 172.21.15.39:0/3709421464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa16015d950 0x7fa160160870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:32.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.187+0000 7fa15ea00640 1 -- 172.21.15.39:0/3709421464 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1601f5a40 con 0x7fa1600cce70 2024-09-21T07:40:32.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.187+0000 7fa15ea00640 1 --2- 172.21.15.39:0/3709421464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1600cce70 0x7fa1600cfba0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fa150009950 tx=0x7fa150033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:32.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.188+0000 7fa157400640 1 -- 172.21.15.39:0/3709421464 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa150005c90 con 0x7fa1600cce70 2024-09-21T07:40:32.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.188+0000 7fa165600640 1 -- 172.21.15.39:0/3709421464 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1601f5d20 con 0x7fa1600cce70 2024-09-21T07:40:32.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.188+0000 7fa157400640 1 -- 172.21.15.39:0/3709421464 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa150031e50 con 0x7fa1600cce70 2024-09-21T07:40:32.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.188+0000 7fa157400640 1 -- 172.21.15.39:0/3709421464 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa150005260 con 0x7fa1600cce70 2024-09-21T07:40:32.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.188+0000 7fa165600640 1 -- 172.21.15.39:0/3709421464 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1601f62e0 con 0x7fa1600cce70 2024-09-21T07:40:32.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.189+0000 7fa165600640 1 -- 172.21.15.39:0/3709421464 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1600700c0 con 0x7fa1600cce70 2024-09-21T07:40:32.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.190+0000 7fa157400640 1 -- 172.21.15.39:0/3709421464 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa150005400 con 0x7fa1600cce70 2024-09-21T07:40:32.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.191+0000 7fa157400640 1 --2- 172.21.15.39:0/3709421464 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa13c078bc0 0x7fa13c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:32.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.191+0000 7fa157400640 1 -- 172.21.15.39:0/3709421464 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7603+0+0 (secure 0 0 0) 0x7fa1500bc450 con 0x7fa1600cce70 2024-09-21T07:40:32.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.191+0000 7fa156000640 1 --2- 172.21.15.39:0/3709421464 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa13c078bc0 0x7fa13c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:32.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.192+0000 7fa156000640 1 --2- 172.21.15.39:0/3709421464 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa13c078bc0 0x7fa13c07b0d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa148007d60 tx=0x7fa148007cf0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:32.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.196+0000 7fa157400640 1 -- 172.21.15.39:0/3709421464 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa15003f030 con 0x7fa1600cce70 2024-09-21T07:40:32.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.467+0000 7fa165600640 1 -- 172.21.15.39:0/3709421464 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:1", "val": "true"} v 0) -- 0x7fa1600753a0 con 0x7fa1600cce70 2024-09-21T07:40:32.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.936+0000 7fa157400640 1 -- 172.21.15.39:0/3709421464 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:1", "val": "true"}]=0 freezing mds.6269 v116) ==== 112+0+0 (secure 0 0 0) 0x7fa150083a40 con 0x7fa1600cce70 2024-09-21T07:40:32.938 INFO:teuthology.orchestra.run.smithi039.stderr:freezing mds.6269 2024-09-21T07:40:32.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.939+0000 7fa165600640 1 -- 172.21.15.39:0/3709421464 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa13c078bc0 msgr2=0x7fa13c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:32.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.939+0000 7fa165600640 1 --2- 172.21.15.39:0/3709421464 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa13c078bc0 0x7fa13c07b0d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa148007d60 tx=0x7fa148007cf0 comp rx=0 tx=0).stop 2024-09-21T07:40:32.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.939+0000 7fa165600640 1 -- 172.21.15.39:0/3709421464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1600cce70 msgr2=0x7fa1600cfba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:32.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.939+0000 7fa165600640 1 --2- 172.21.15.39:0/3709421464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1600cce70 0x7fa1600cfba0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fa150009950 tx=0x7fa150033c20 comp rx=0 tx=0).stop 2024-09-21T07:40:32.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.939+0000 7fa165600640 1 -- 172.21.15.39:0/3709421464 shutdown_connections 2024-09-21T07:40:32.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.939+0000 7fa165600640 1 --2- 172.21.15.39:0/3709421464 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa13c078bc0 0x7fa13c07b0d0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:32.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.939+0000 7fa165600640 1 --2- 172.21.15.39:0/3709421464 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa16015d950 0x7fa160160870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:32.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.939+0000 7fa165600640 1 --2- 172.21.15.39:0/3709421464 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa16015cf60 0x7fa160160330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:32.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.939+0000 7fa165600640 1 --2- 172.21.15.39:0/3709421464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1600cce70 0x7fa1600cfba0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:32.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.939+0000 7fa165600640 1 -- 172.21.15.39:0/3709421464 >> 172.21.15.39:0/3709421464 conn(0x7fa16006d440 msgr2=0x7fa1600adf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:32.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.939+0000 7fa165600640 1 -- 172.21.15.39:0/3709421464 shutdown_connections 2024-09-21T07:40:32.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:32.939+0000 7fa165600640 1 -- 172.21.15.39:0/3709421464 wait complete. 2024-09-21T07:40:32.965 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_kill_mdstable_at 6 2024-09-21T07:40:33.208 INFO:teuthology.orchestra.run.smithi039.stdout:{"success":"mds_kill_mdstable_at = '' (not observed, change may require restart) "} 2024-09-21T07:40:33.209 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_kill_mdstable_at = '' (not observed, change may require restart) " } 2024-09-21T07:40:33.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:40:33.209 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d1/dir/.snap/s26) 2024-09-21T07:40:33.213 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-09-21T07:40:33.488 INFO:teuthology.orchestra.run.smithi039.stdout:{"mds_beacon_grace":"15.000000"} 2024-09-21T07:40:33.489 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-09-21T07:40:33.489 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:40:33.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.717+0000 7fde07200640 1 -- 172.21.15.39:0/1192567768 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde000d0620 msgr2=0x7fde000d0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:33.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.717+0000 7fde07200640 1 --2- 172.21.15.39:0/1192567768 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde000d0620 0x7fde000d0a30 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fddf8005fd0 tx=0x7fddf802aaa0 comp rx=0 tx=0).stop 2024-09-21T07:40:33.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.717+0000 7fde07200640 1 -- 172.21.15.39:0/1192567768 shutdown_connections 2024-09-21T07:40:33.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.717+0000 7fde07200640 1 --2- 172.21.15.39:0/1192567768 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde000cfd10 0x7fde000cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:33.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.717+0000 7fde07200640 1 --2- 172.21.15.39:0/1192567768 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde000d0620 0x7fde000d0a30 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:33.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.717+0000 7fde07200640 1 -- 172.21.15.39:0/1192567768 >> 172.21.15.39:0/1192567768 conn(0x7fde0006cfc0 msgr2=0x7fde0006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:33.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.717+0000 7fde07200640 1 -- 172.21.15.39:0/1192567768 shutdown_connections 2024-09-21T07:40:33.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.718+0000 7fde07200640 1 -- 172.21.15.39:0/1192567768 wait complete. 2024-09-21T07:40:33.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.720+0000 7fde07200640 1 Processor -- start 2024-09-21T07:40:33.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.720+0000 7fde07200640 1 -- start start 2024-09-21T07:40:33.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.721+0000 7fde07200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde000cfd10 0x7fde00147e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:33.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.721+0000 7fde07200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde000d0620 0x7fde001483b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:33.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.721+0000 7fde07200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde0015cd30 0x7fde00146c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:33.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.721+0000 7fde07200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fde00073890 con 0x7fde000d0620 2024-09-21T07:40:33.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.721+0000 7fde07200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fde00073710 con 0x7fde0015cd30 2024-09-21T07:40:33.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.721+0000 7fde07200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fde00073a10 con 0x7fde000cfd10 2024-09-21T07:40:33.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.722+0000 7fddf7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde000d0620 0x7fde001483b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:33.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.722+0000 7fddf7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde000d0620 0x7fde001483b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35658/0 (socket says 172.21.15.39:35658) 2024-09-21T07:40:33.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.722+0000 7fddf7e00640 1 -- 172.21.15.39:0/3867671797 learned_addr learned my addr 172.21.15.39:0/3867671797 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:40:33.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.722+0000 7fddffe00640 1 --2- 172.21.15.39:0/3867671797 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde000cfd10 0x7fde00147e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:33.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.722+0000 7fde05200640 1 --2- 172.21.15.39:0/3867671797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde0015cd30 0x7fde00146c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:33.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.722+0000 7fddf7e00640 1 -- 172.21.15.39:0/3867671797 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde000cfd10 msgr2=0x7fde00147e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:33.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.722+0000 7fddf7e00640 1 --2- 172.21.15.39:0/3867671797 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde000cfd10 0x7fde00147e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:33.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.722+0000 7fddf7e00640 1 -- 172.21.15.39:0/3867671797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde0015cd30 msgr2=0x7fde00146c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:33.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.722+0000 7fddf7e00640 1 --2- 172.21.15.39:0/3867671797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde0015cd30 0x7fde00146c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:33.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.722+0000 7fddf7e00640 1 -- 172.21.15.39:0/3867671797 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde00145ea0 con 0x7fde000d0620 2024-09-21T07:40:33.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.723+0000 7fddf7e00640 1 --2- 172.21.15.39:0/3867671797 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde000d0620 0x7fde001483b0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fdde800d6d0 tx=0x7fdde800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:33.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.723+0000 7fddfcc00640 1 -- 172.21.15.39:0/3867671797 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde8017070 con 0x7fde000d0620 2024-09-21T07:40:33.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.723+0000 7fddfcc00640 1 -- 172.21.15.39:0/3867671797 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdde8005330 con 0x7fde000d0620 2024-09-21T07:40:33.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.723+0000 7fde07200640 1 -- 172.21.15.39:0/3867671797 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde00146180 con 0x7fde000d0620 2024-09-21T07:40:33.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.723+0000 7fddfcc00640 1 -- 172.21.15.39:0/3867671797 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde8009820 con 0x7fde000d0620 2024-09-21T07:40:33.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.723+0000 7fde07200640 1 -- 172.21.15.39:0/3867671797 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde001530b0 con 0x7fde000d0620 2024-09-21T07:40:33.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.724+0000 7fde07200640 1 -- 172.21.15.39:0/3867671797 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde00073f30 con 0x7fde000d0620 2024-09-21T07:40:33.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.725+0000 7fddfcc00640 1 -- 172.21.15.39:0/3867671797 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdde8009bd0 con 0x7fde000d0620 2024-09-21T07:40:33.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.726+0000 7fddfcc00640 1 --2- 172.21.15.39:0/3867671797 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdddc078b90 0x7fdddc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:33.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.727+0000 7fddfcc00640 1 -- 172.21.15.39:0/3867671797 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7603+0+0 (secure 0 0 0) 0x7fdde809b710 con 0x7fde000d0620 2024-09-21T07:40:33.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.727+0000 7fddffe00640 1 --2- 172.21.15.39:0/3867671797 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdddc078b90 0x7fdddc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:33.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.728+0000 7fddffe00640 1 --2- 172.21.15.39:0/3867671797 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdddc078b90 0x7fdddc07b0a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fddf0001720 tx=0x7fddf0009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:33.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.731+0000 7fddfcc00640 1 -- 172.21.15.39:0/3867671797 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdde8062da0 con 0x7fde000d0620 2024-09-21T07:40:33.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.967+0000 7fde07200640 1 -- 172.21.15.39:0/3867671797 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde000ced60 con 0x7fde000d0620 2024-09-21T07:40:33.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.967+0000 7fddfcc00640 1 -- 172.21.15.39:0/3867671797 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) ==== 95+0+5258 (secure 0 0 0) 0x7fdde80677d0 con 0x7fde000d0620 2024-09-21T07:40:33.970 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:40:33.970 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":116,"btime":"2024-09-21T07:40:32:928729+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7311,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:40:32.468779+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7113,"mds_1":6269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6269":{"gid":6269,"name":"d","rank":1,"incarnation":110,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6837/2336112936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2336112936},{"type":"v1","addr":"172.21.15.148:6837","nonce":2336112936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7113":{"gid":7113,"name":"a","rank":0,"incarnation":107,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/2821446711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2821446711},{"type":"v1","addr":"172.21.15.39:6835","nonce":2821446711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7113,"qdb_cluster":[6269,7113]},"id":6}]} 2024-09-21T07:40:33.970 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 116 2024-09-21T07:40:33.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.970+0000 7fde07200640 1 -- 172.21.15.39:0/3867671797 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdddc078b90 msgr2=0x7fdddc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:33.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.970+0000 7fde07200640 1 --2- 172.21.15.39:0/3867671797 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdddc078b90 0x7fdddc07b0a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fddf0001720 tx=0x7fddf0009040 comp rx=0 tx=0).stop 2024-09-21T07:40:33.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.970+0000 7fde07200640 1 -- 172.21.15.39:0/3867671797 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde000d0620 msgr2=0x7fde001483b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:33.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.970+0000 7fde07200640 1 --2- 172.21.15.39:0/3867671797 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde000d0620 0x7fde001483b0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fdde800d6d0 tx=0x7fdde800db90 comp rx=0 tx=0).stop 2024-09-21T07:40:33.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.970+0000 7fde07200640 1 -- 172.21.15.39:0/3867671797 shutdown_connections 2024-09-21T07:40:33.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.970+0000 7fde07200640 1 --2- 172.21.15.39:0/3867671797 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdddc078b90 0x7fdddc07b0a0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:33.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.970+0000 7fde07200640 1 --2- 172.21.15.39:0/3867671797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde0015cd30 0x7fde00146c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:33.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.970+0000 7fde07200640 1 --2- 172.21.15.39:0/3867671797 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde000d0620 0x7fde001483b0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:33.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.970+0000 7fde07200640 1 --2- 172.21.15.39:0/3867671797 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde000cfd10 0x7fde00147e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:33.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.970+0000 7fde07200640 1 -- 172.21.15.39:0/3867671797 >> 172.21.15.39:0/3867671797 conn(0x7fde0006cfc0 msgr2=0x7fde0006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:33.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.970+0000 7fde07200640 1 -- 172.21.15.39:0/3867671797 shutdown_connections 2024-09-21T07:40:33.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:33.970+0000 7fde07200640 1 -- 172.21.15.39:0/3867671797 wait complete. 2024-09-21T07:40:33.994 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:40:35.828 INFO:tasks.ceph.mds.a.smithi039.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_commit_logged(ceph::cref_t&)' thread 7fbe07e00640 time 2024-09-21T07:40:35.828133+0000 2024-09-21T07:40:35.829 INFO:tasks.ceph.mds.a.smithi039.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: 173: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 6) 2024-09-21T07:40:35.829 INFO:tasks.ceph.mds.a.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:40:35.829 INFO:tasks.ceph.mds.a.smithi039.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fbe1705654b] 2024-09-21T07:40:35.829 INFO:tasks.ceph.mds.a.smithi039.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7fbe17056758] 2024-09-21T07:40:35.829 INFO:tasks.ceph.mds.a.smithi039.stderr: 3: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x1d9) [0x5572eb02676f] 2024-09-21T07:40:35.829 INFO:tasks.ceph.mds.a.smithi039.stderr: 4: ceph-mds(+0x479942) [0x5572eb026942] 2024-09-21T07:40:35.829 INFO:tasks.ceph.mds.a.smithi039.stderr: 5: (MDSContext::complete(int)+0x7c) [0x5572eb0746d8] 2024-09-21T07:40:35.830 INFO:tasks.ceph.mds.a.smithi039.stderr: 6: (MDSIOContextBase::complete(int)+0x501) [0x5572eb074d57] 2024-09-21T07:40:35.830 INFO:tasks.ceph.mds.a.smithi039.stderr: 7: (MDSLogContextBase::complete(int)+0x46) [0x5572eb074dd4] 2024-09-21T07:40:35.830 INFO:tasks.ceph.mds.a.smithi039.stderr: 8: (Finisher::finisher_thread_entry()+0x495) [0x7fbe16fde34b] 2024-09-21T07:40:35.830 INFO:tasks.ceph.mds.a.smithi039.stderr: 9: ceph-mds(+0x1052e1) [0x5572eacb22e1] 2024-09-21T07:40:35.830 INFO:tasks.ceph.mds.a.smithi039.stderr: 10: (Thread::entry_wrapper()+0x43) [0x7fbe1702fd57] 2024-09-21T07:40:35.830 INFO:tasks.ceph.mds.a.smithi039.stderr: 11: (Thread::_entry_func(void*)+0xd) [0x7fbe1702fd73] 2024-09-21T07:40:35.830 INFO:tasks.ceph.mds.a.smithi039.stderr: 12: /lib64/libc.so.6(+0x89c52) [0x7fbe16689c52] 2024-09-21T07:40:35.830 INFO:tasks.ceph.mds.a.smithi039.stderr: 13: /lib64/libc.so.6(+0x10ec80) [0x7fbe1670ec80] 2024-09-21T07:40:35.830 INFO:tasks.ceph.mds.a.smithi039.stderr:2024-09-21T07:40:35.827+0000 7fbe07e00640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_commit_logged(ceph::cref_t&)' thread 7fbe07e00640 time 2024-09-21T07:40:35.828133+0000 2024-09-21T07:40:35.830 INFO:tasks.ceph.mds.a.smithi039.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: 173: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 6) 2024-09-21T07:40:35.830 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:40:35.830 INFO:tasks.ceph.mds.a.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:40:35.830 INFO:tasks.ceph.mds.a.smithi039.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fbe1705654b] 2024-09-21T07:40:35.831 INFO:tasks.ceph.mds.a.smithi039.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7fbe17056758] 2024-09-21T07:40:35.831 INFO:tasks.ceph.mds.a.smithi039.stderr: 3: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x1d9) [0x5572eb02676f] 2024-09-21T07:40:35.831 INFO:tasks.ceph.mds.a.smithi039.stderr: 4: ceph-mds(+0x479942) [0x5572eb026942] 2024-09-21T07:40:35.831 INFO:tasks.ceph.mds.a.smithi039.stderr: 5: (MDSContext::complete(int)+0x7c) [0x5572eb0746d8] 2024-09-21T07:40:35.831 INFO:tasks.ceph.mds.a.smithi039.stderr: 6: (MDSIOContextBase::complete(int)+0x501) [0x5572eb074d57] 2024-09-21T07:40:35.831 INFO:tasks.ceph.mds.a.smithi039.stderr: 7: (MDSLogContextBase::complete(int)+0x46) [0x5572eb074dd4] 2024-09-21T07:40:35.831 INFO:tasks.ceph.mds.a.smithi039.stderr: 8: (Finisher::finisher_thread_entry()+0x495) [0x7fbe16fde34b] 2024-09-21T07:40:35.831 INFO:tasks.ceph.mds.a.smithi039.stderr: 9: ceph-mds(+0x1052e1) [0x5572eacb22e1] 2024-09-21T07:40:35.831 INFO:tasks.ceph.mds.a.smithi039.stderr: 10: (Thread::entry_wrapper()+0x43) [0x7fbe1702fd57] 2024-09-21T07:40:35.831 INFO:tasks.ceph.mds.a.smithi039.stderr: 11: (Thread::_entry_func(void*)+0xd) [0x7fbe1702fd73] 2024-09-21T07:40:35.831 INFO:tasks.ceph.mds.a.smithi039.stderr: 12: /lib64/libc.so.6(+0x89c52) [0x7fbe16689c52] 2024-09-21T07:40:35.831 INFO:tasks.ceph.mds.a.smithi039.stderr: 13: /lib64/libc.so.6(+0x10ec80) [0x7fbe1670ec80] 2024-09-21T07:40:35.831 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:40:35.831 INFO:tasks.ceph.mds.a.smithi039.stderr:*** Caught signal (Aborted) ** 2024-09-21T07:40:35.832 INFO:tasks.ceph.mds.a.smithi039.stderr: in thread 7fbe07e00640 thread_name:MR_Finisher 2024-09-21T07:40:35.832 INFO:tasks.ceph.mds.a.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:40:35.832 INFO:tasks.ceph.mds.a.smithi039.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fbe1663e6f0] 2024-09-21T07:40:35.832 INFO:tasks.ceph.mds.a.smithi039.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7fbe1668b99c] 2024-09-21T07:40:35.832 INFO:tasks.ceph.mds.a.smithi039.stderr: 3: raise() 2024-09-21T07:40:35.832 INFO:tasks.ceph.mds.a.smithi039.stderr: 4: abort() 2024-09-21T07:40:35.832 INFO:tasks.ceph.mds.a.smithi039.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7fbe17056686] 2024-09-21T07:40:35.833 INFO:tasks.ceph.mds.a.smithi039.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7fbe17056758] 2024-09-21T07:40:35.833 INFO:tasks.ceph.mds.a.smithi039.stderr: 7: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x1d9) [0x5572eb02676f] 2024-09-21T07:40:35.833 INFO:tasks.ceph.mds.a.smithi039.stderr: 8: ceph-mds(+0x479942) [0x5572eb026942] 2024-09-21T07:40:35.833 INFO:tasks.ceph.mds.a.smithi039.stderr: 9: (MDSContext::complete(int)+0x7c) [0x5572eb0746d8] 2024-09-21T07:40:35.833 INFO:tasks.ceph.mds.a.smithi039.stderr: 10: (MDSIOContextBase::complete(int)+0x501) [0x5572eb074d57] 2024-09-21T07:40:35.833 INFO:tasks.ceph.mds.a.smithi039.stderr: 11: (MDSLogContextBase::complete(int)+0x46) [0x5572eb074dd4] 2024-09-21T07:40:35.833 INFO:tasks.ceph.mds.a.smithi039.stderr: 12: (Finisher::finisher_thread_entry()+0x495) [0x7fbe16fde34b] 2024-09-21T07:40:35.833 INFO:tasks.ceph.mds.a.smithi039.stderr: 13: ceph-mds(+0x1052e1) [0x5572eacb22e1] 2024-09-21T07:40:35.833 INFO:tasks.ceph.mds.a.smithi039.stderr: 14: (Thread::entry_wrapper()+0x43) [0x7fbe1702fd57] 2024-09-21T07:40:35.833 INFO:tasks.ceph.mds.a.smithi039.stderr: 15: (Thread::_entry_func(void*)+0xd) [0x7fbe1702fd73] 2024-09-21T07:40:35.833 INFO:tasks.ceph.mds.a.smithi039.stderr: 16: /lib64/libc.so.6(+0x89c52) [0x7fbe16689c52] 2024-09-21T07:40:35.833 INFO:tasks.ceph.mds.a.smithi039.stderr: 17: /lib64/libc.so.6(+0x10ec80) [0x7fbe1670ec80] 2024-09-21T07:40:35.833 INFO:tasks.ceph.mds.a.smithi039.stderr:2024-09-21T07:40:35.829+0000 7fbe07e00640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:40:35.833 INFO:tasks.ceph.mds.a.smithi039.stderr: in thread 7fbe07e00640 thread_name:MR_Finisher 2024-09-21T07:40:35.834 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:40:35.834 INFO:tasks.ceph.mds.a.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:40:35.834 INFO:tasks.ceph.mds.a.smithi039.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fbe1663e6f0] 2024-09-21T07:40:35.834 INFO:tasks.ceph.mds.a.smithi039.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7fbe1668b99c] 2024-09-21T07:40:35.834 INFO:tasks.ceph.mds.a.smithi039.stderr: 3: raise() 2024-09-21T07:40:35.834 INFO:tasks.ceph.mds.a.smithi039.stderr: 4: abort() 2024-09-21T07:40:35.834 INFO:tasks.ceph.mds.a.smithi039.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7fbe17056686] 2024-09-21T07:40:35.834 INFO:tasks.ceph.mds.a.smithi039.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7fbe17056758] 2024-09-21T07:40:35.834 INFO:tasks.ceph.mds.a.smithi039.stderr: 7: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x1d9) [0x5572eb02676f] 2024-09-21T07:40:35.834 INFO:tasks.ceph.mds.a.smithi039.stderr: 8: ceph-mds(+0x479942) [0x5572eb026942] 2024-09-21T07:40:35.834 INFO:tasks.ceph.mds.a.smithi039.stderr: 9: (MDSContext::complete(int)+0x7c) [0x5572eb0746d8] 2024-09-21T07:40:35.834 INFO:tasks.ceph.mds.a.smithi039.stderr: 10: (MDSIOContextBase::complete(int)+0x501) [0x5572eb074d57] 2024-09-21T07:40:35.834 INFO:tasks.ceph.mds.a.smithi039.stderr: 11: (MDSLogContextBase::complete(int)+0x46) [0x5572eb074dd4] 2024-09-21T07:40:35.834 INFO:tasks.ceph.mds.a.smithi039.stderr: 12: (Finisher::finisher_thread_entry()+0x495) [0x7fbe16fde34b] 2024-09-21T07:40:35.835 INFO:tasks.ceph.mds.a.smithi039.stderr: 13: ceph-mds(+0x1052e1) [0x5572eacb22e1] 2024-09-21T07:40:35.835 INFO:tasks.ceph.mds.a.smithi039.stderr: 14: (Thread::entry_wrapper()+0x43) [0x7fbe1702fd57] 2024-09-21T07:40:35.835 INFO:tasks.ceph.mds.a.smithi039.stderr: 15: (Thread::_entry_func(void*)+0xd) [0x7fbe1702fd73] 2024-09-21T07:40:35.835 INFO:tasks.ceph.mds.a.smithi039.stderr: 16: /lib64/libc.so.6(+0x89c52) [0x7fbe16689c52] 2024-09-21T07:40:35.835 INFO:tasks.ceph.mds.a.smithi039.stderr: 17: /lib64/libc.so.6(+0x10ec80) [0x7fbe1670ec80] 2024-09-21T07:40:35.835 INFO:tasks.ceph.mds.a.smithi039.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:40:35.835 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:40:35.841 INFO:tasks.ceph.mds.a.smithi039.stderr: -1245> 2024-09-21T07:40:21.609+0000 7fbe0ce00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:40:35.841 INFO:tasks.ceph.mds.a.smithi039.stderr: -1243> 2024-09-21T07:40:21.609+0000 7fbe0ce00640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:40:35.844 INFO:tasks.ceph.mds.a.smithi039.stderr: -1> 2024-09-21T07:40:35.827+0000 7fbe07e00640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_commit_logged(ceph::cref_t&)' thread 7fbe07e00640 time 2024-09-21T07:40:35.828133+0000 2024-09-21T07:40:35.844 INFO:tasks.ceph.mds.a.smithi039.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: 173: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 6) 2024-09-21T07:40:35.844 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:40:35.844 INFO:tasks.ceph.mds.a.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:40:35.845 INFO:tasks.ceph.mds.a.smithi039.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fbe1705654b] 2024-09-21T07:40:35.845 INFO:tasks.ceph.mds.a.smithi039.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7fbe17056758] 2024-09-21T07:40:35.845 INFO:tasks.ceph.mds.a.smithi039.stderr: 3: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x1d9) [0x5572eb02676f] 2024-09-21T07:40:35.845 INFO:tasks.ceph.mds.a.smithi039.stderr: 4: ceph-mds(+0x479942) [0x5572eb026942] 2024-09-21T07:40:35.845 INFO:tasks.ceph.mds.a.smithi039.stderr: 5: (MDSContext::complete(int)+0x7c) [0x5572eb0746d8] 2024-09-21T07:40:35.845 INFO:tasks.ceph.mds.a.smithi039.stderr: 6: (MDSIOContextBase::complete(int)+0x501) [0x5572eb074d57] 2024-09-21T07:40:35.845 INFO:tasks.ceph.mds.a.smithi039.stderr: 7: (MDSLogContextBase::complete(int)+0x46) [0x5572eb074dd4] 2024-09-21T07:40:35.845 INFO:tasks.ceph.mds.a.smithi039.stderr: 8: (Finisher::finisher_thread_entry()+0x495) [0x7fbe16fde34b] 2024-09-21T07:40:35.845 INFO:tasks.ceph.mds.a.smithi039.stderr: 9: ceph-mds(+0x1052e1) [0x5572eacb22e1] 2024-09-21T07:40:35.845 INFO:tasks.ceph.mds.a.smithi039.stderr: 10: (Thread::entry_wrapper()+0x43) [0x7fbe1702fd57] 2024-09-21T07:40:35.845 INFO:tasks.ceph.mds.a.smithi039.stderr: 11: (Thread::_entry_func(void*)+0xd) [0x7fbe1702fd73] 2024-09-21T07:40:35.845 INFO:tasks.ceph.mds.a.smithi039.stderr: 12: /lib64/libc.so.6(+0x89c52) [0x7fbe16689c52] 2024-09-21T07:40:35.845 INFO:tasks.ceph.mds.a.smithi039.stderr: 13: /lib64/libc.so.6(+0x10ec80) [0x7fbe1670ec80] 2024-09-21T07:40:35.845 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:40:35.845 INFO:tasks.ceph.mds.a.smithi039.stderr: 0> 2024-09-21T07:40:35.829+0000 7fbe07e00640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:40:35.846 INFO:tasks.ceph.mds.a.smithi039.stderr: in thread 7fbe07e00640 thread_name:MR_Finisher 2024-09-21T07:40:35.846 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:40:35.846 INFO:tasks.ceph.mds.a.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:40:35.846 INFO:tasks.ceph.mds.a.smithi039.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fbe1663e6f0] 2024-09-21T07:40:35.846 INFO:tasks.ceph.mds.a.smithi039.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7fbe1668b99c] 2024-09-21T07:40:35.846 INFO:tasks.ceph.mds.a.smithi039.stderr: 3: raise() 2024-09-21T07:40:35.846 INFO:tasks.ceph.mds.a.smithi039.stderr: 4: abort() 2024-09-21T07:40:35.846 INFO:tasks.ceph.mds.a.smithi039.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7fbe17056686] 2024-09-21T07:40:35.846 INFO:tasks.ceph.mds.a.smithi039.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7fbe17056758] 2024-09-21T07:40:35.846 INFO:tasks.ceph.mds.a.smithi039.stderr: 7: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x1d9) [0x5572eb02676f] 2024-09-21T07:40:35.846 INFO:tasks.ceph.mds.a.smithi039.stderr: 8: ceph-mds(+0x479942) [0x5572eb026942] 2024-09-21T07:40:35.846 INFO:tasks.ceph.mds.a.smithi039.stderr: 9: (MDSContext::complete(int)+0x7c) [0x5572eb0746d8] 2024-09-21T07:40:35.846 INFO:tasks.ceph.mds.a.smithi039.stderr: 10: (MDSIOContextBase::complete(int)+0x501) [0x5572eb074d57] 2024-09-21T07:40:35.846 INFO:tasks.ceph.mds.a.smithi039.stderr: 11: (MDSLogContextBase::complete(int)+0x46) [0x5572eb074dd4] 2024-09-21T07:40:35.846 INFO:tasks.ceph.mds.a.smithi039.stderr: 12: (Finisher::finisher_thread_entry()+0x495) [0x7fbe16fde34b] 2024-09-21T07:40:35.847 INFO:tasks.ceph.mds.a.smithi039.stderr: 13: ceph-mds(+0x1052e1) [0x5572eacb22e1] 2024-09-21T07:40:35.847 INFO:tasks.ceph.mds.a.smithi039.stderr: 14: (Thread::entry_wrapper()+0x43) [0x7fbe1702fd57] 2024-09-21T07:40:35.847 INFO:tasks.ceph.mds.a.smithi039.stderr: 15: (Thread::_entry_func(void*)+0xd) [0x7fbe1702fd73] 2024-09-21T07:40:35.847 INFO:tasks.ceph.mds.a.smithi039.stderr: 16: /lib64/libc.so.6(+0x89c52) [0x7fbe16689c52] 2024-09-21T07:40:35.847 INFO:tasks.ceph.mds.a.smithi039.stderr: 17: /lib64/libc.so.6(+0x10ec80) [0x7fbe1670ec80] 2024-09-21T07:40:35.847 INFO:tasks.ceph.mds.a.smithi039.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:40:35.847 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:40:35.851 INFO:tasks.ceph.mds.a.smithi039.stderr: -1248> 2024-09-21T07:40:21.609+0000 7fbe0ce00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:40:35.851 INFO:tasks.ceph.mds.a.smithi039.stderr: -1246> 2024-09-21T07:40:21.609+0000 7fbe0ce00640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:40:35.852 INFO:tasks.ceph.mds.a.smithi039.stderr: -4> 2024-09-21T07:40:35.827+0000 7fbe07e00640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_commit_logged(ceph::cref_t&)' thread 7fbe07e00640 time 2024-09-21T07:40:35.828133+0000 2024-09-21T07:40:35.852 INFO:tasks.ceph.mds.a.smithi039.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: 173: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 6) 2024-09-21T07:40:35.852 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:40:35.852 INFO:tasks.ceph.mds.a.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:40:35.852 INFO:tasks.ceph.mds.a.smithi039.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fbe1705654b] 2024-09-21T07:40:35.852 INFO:tasks.ceph.mds.a.smithi039.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7fbe17056758] 2024-09-21T07:40:35.852 INFO:tasks.ceph.mds.a.smithi039.stderr: 3: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x1d9) [0x5572eb02676f] 2024-09-21T07:40:35.852 INFO:tasks.ceph.mds.a.smithi039.stderr: 4: ceph-mds(+0x479942) [0x5572eb026942] 2024-09-21T07:40:35.853 INFO:tasks.ceph.mds.a.smithi039.stderr: 5: (MDSContext::complete(int)+0x7c) [0x5572eb0746d8] 2024-09-21T07:40:35.853 INFO:tasks.ceph.mds.a.smithi039.stderr: 6: (MDSIOContextBase::complete(int)+0x501) [0x5572eb074d57] 2024-09-21T07:40:35.853 INFO:tasks.ceph.mds.a.smithi039.stderr: 7: (MDSLogContextBase::complete(int)+0x46) [0x5572eb074dd4] 2024-09-21T07:40:35.853 INFO:tasks.ceph.mds.a.smithi039.stderr: 8: (Finisher::finisher_thread_entry()+0x495) [0x7fbe16fde34b] 2024-09-21T07:40:35.853 INFO:tasks.ceph.mds.a.smithi039.stderr: 9: ceph-mds(+0x1052e1) [0x5572eacb22e1] 2024-09-21T07:40:35.853 INFO:tasks.ceph.mds.a.smithi039.stderr: 10: (Thread::entry_wrapper()+0x43) [0x7fbe1702fd57] 2024-09-21T07:40:35.853 INFO:tasks.ceph.mds.a.smithi039.stderr: 11: (Thread::_entry_func(void*)+0xd) [0x7fbe1702fd73] 2024-09-21T07:40:35.853 INFO:tasks.ceph.mds.a.smithi039.stderr: 12: /lib64/libc.so.6(+0x89c52) [0x7fbe16689c52] 2024-09-21T07:40:35.853 INFO:tasks.ceph.mds.a.smithi039.stderr: 13: /lib64/libc.so.6(+0x10ec80) [0x7fbe1670ec80] 2024-09-21T07:40:35.853 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:40:35.854 INFO:tasks.ceph.mds.a.smithi039.stderr: -3> 2024-09-21T07:40:35.829+0000 7fbe07e00640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:40:35.854 INFO:tasks.ceph.mds.a.smithi039.stderr: in thread 7fbe07e00640 thread_name:MR_Finisher 2024-09-21T07:40:35.854 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:40:35.854 INFO:tasks.ceph.mds.a.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:40:35.854 INFO:tasks.ceph.mds.a.smithi039.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fbe1663e6f0] 2024-09-21T07:40:35.854 INFO:tasks.ceph.mds.a.smithi039.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7fbe1668b99c] 2024-09-21T07:40:35.854 INFO:tasks.ceph.mds.a.smithi039.stderr: 3: raise() 2024-09-21T07:40:35.854 INFO:tasks.ceph.mds.a.smithi039.stderr: 4: abort() 2024-09-21T07:40:35.854 INFO:tasks.ceph.mds.a.smithi039.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7fbe17056686] 2024-09-21T07:40:35.854 INFO:tasks.ceph.mds.a.smithi039.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7fbe17056758] 2024-09-21T07:40:35.854 INFO:tasks.ceph.mds.a.smithi039.stderr: 7: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x1d9) [0x5572eb02676f] 2024-09-21T07:40:35.854 INFO:tasks.ceph.mds.a.smithi039.stderr: 8: ceph-mds(+0x479942) [0x5572eb026942] 2024-09-21T07:40:35.854 INFO:tasks.ceph.mds.a.smithi039.stderr: 9: (MDSContext::complete(int)+0x7c) [0x5572eb0746d8] 2024-09-21T07:40:35.854 INFO:tasks.ceph.mds.a.smithi039.stderr: 10: (MDSIOContextBase::complete(int)+0x501) [0x5572eb074d57] 2024-09-21T07:40:35.855 INFO:tasks.ceph.mds.a.smithi039.stderr: 11: (MDSLogContextBase::complete(int)+0x46) [0x5572eb074dd4] 2024-09-21T07:40:35.855 INFO:tasks.ceph.mds.a.smithi039.stderr: 12: (Finisher::finisher_thread_entry()+0x495) [0x7fbe16fde34b] 2024-09-21T07:40:35.855 INFO:tasks.ceph.mds.a.smithi039.stderr: 13: ceph-mds(+0x1052e1) [0x5572eacb22e1] 2024-09-21T07:40:35.855 INFO:tasks.ceph.mds.a.smithi039.stderr: 14: (Thread::entry_wrapper()+0x43) [0x7fbe1702fd57] 2024-09-21T07:40:35.855 INFO:tasks.ceph.mds.a.smithi039.stderr: 15: (Thread::_entry_func(void*)+0xd) [0x7fbe1702fd73] 2024-09-21T07:40:35.855 INFO:tasks.ceph.mds.a.smithi039.stderr: 16: /lib64/libc.so.6(+0x89c52) [0x7fbe16689c52] 2024-09-21T07:40:35.855 INFO:tasks.ceph.mds.a.smithi039.stderr: 17: /lib64/libc.so.6(+0x10ec80) [0x7fbe1670ec80] 2024-09-21T07:40:35.855 INFO:tasks.ceph.mds.a.smithi039.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:40:35.855 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:40:36.632 INFO:tasks.ceph.mds.a.smithi039.stderr:daemon-helper: command crashed with signal 6 2024-09-21T07:40:38.994 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:40:39.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.231+0000 7fbcfe400640 1 -- 172.21.15.39:0/2855480314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcdc003ff0 msgr2=0x7fbcdc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:39.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.231+0000 7fbcfe400640 1 --2- 172.21.15.39:0/2855480314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcdc003ff0 0x7fbcdc004800 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fbce0009b50 tx=0x7fbcdc005830 comp rx=0 tx=0).stop 2024-09-21T07:40:39.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.231+0000 7fbcfda00640 1 -- 172.21.15.39:0/2855480314 reap_dead start 2024-09-21T07:40:39.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.231+0000 7fbcfe400640 1 -- 172.21.15.39:0/2855480314 shutdown_connections 2024-09-21T07:40:39.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.231+0000 7fbcfe400640 1 -- 172.21.15.39:0/2855480314 >> 172.21.15.39:0/2855480314 conn(0x7fbcf80b2510 msgr2=0x7fbcf80d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:39.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.231+0000 7fbcfe400640 1 -- 172.21.15.39:0/2855480314 shutdown_connections 2024-09-21T07:40:39.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.231+0000 7fbcfe400640 1 -- 172.21.15.39:0/2855480314 wait complete. 2024-09-21T07:40:39.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.234+0000 7fbcfe400640 1 Processor -- start 2024-09-21T07:40:39.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.234+0000 7fbcfe400640 1 -- start start 2024-09-21T07:40:39.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.235+0000 7fbcfe400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbcdc003ff0 0x7fbcf815b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:39.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.235+0000 7fbcfe400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcf8158450 0x7fbcf815bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:39.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.235+0000 7fbcfe400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcf80b30d0 0x7fbcf80b34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:39.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.235+0000 7fbcfe400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf806f8a0 con 0x7fbcf80b30d0 2024-09-21T07:40:39.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.235+0000 7fbcfe400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf80737e0 con 0x7fbcf8158450 2024-09-21T07:40:39.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.235+0000 7fbcfe400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fbcf806fa20 con 0x7fbcdc003ff0 2024-09-21T07:40:39.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.235+0000 7fbcfda00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcf80b30d0 0x7fbcf80b34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:39.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.235+0000 7fbcfda00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcf80b30d0 0x7fbcf80b34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35682/0 (socket says 172.21.15.39:35682) 2024-09-21T07:40:39.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.236+0000 7fbcfda00640 1 -- 172.21.15.39:0/1385648475 learned_addr learned my addr 172.21.15.39:0/1385648475 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:40:39.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.236+0000 7fbcfd000640 1 --2- 172.21.15.39:0/1385648475 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbcdc003ff0 0x7fbcf815b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:39.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.236+0000 7fbcefe00640 1 --2- 172.21.15.39:0/1385648475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcf8158450 0x7fbcf815bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:39.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.236+0000 7fbcfda00640 1 -- 172.21.15.39:0/1385648475 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbcdc003ff0 msgr2=0x7fbcf815b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:39.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.236+0000 7fbcfda00640 1 --2- 172.21.15.39:0/1385648475 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbcdc003ff0 0x7fbcf815b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:39.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.236+0000 7fbcfda00640 1 -- 172.21.15.39:0/1385648475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcf8158450 msgr2=0x7fbcf815bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:39.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.236+0000 7fbcfda00640 1 --2- 172.21.15.39:0/1385648475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcf8158450 0x7fbcf815bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:39.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.236+0000 7fbcfda00640 1 -- 172.21.15.39:0/1385648475 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf80b3a20 con 0x7fbcf80b30d0 2024-09-21T07:40:39.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.236+0000 7fbcfda00640 1 --2- 172.21.15.39:0/1385648475 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcf80b30d0 0x7fbcf80b34e0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fbcf80cddc0 tx=0x7fbcf0010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:39.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.236+0000 7fbcf5600640 1 -- 172.21.15.39:0/1385648475 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf000edf0 con 0x7fbcf80b30d0 2024-09-21T07:40:39.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.236+0000 7fbcf5600640 1 -- 172.21.15.39:0/1385648475 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbcf0015070 con 0x7fbcf80b30d0 2024-09-21T07:40:39.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.237+0000 7fbcfe400640 1 -- 172.21.15.39:0/1385648475 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcf81f5800 con 0x7fbcf80b30d0 2024-09-21T07:40:39.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.237+0000 7fbcf5600640 1 -- 172.21.15.39:0/1385648475 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0003d00 con 0x7fbcf80b30d0 2024-09-21T07:40:39.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.237+0000 7fbcfe400640 1 -- 172.21.15.39:0/1385648475 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcf81f5dc0 con 0x7fbcf80b30d0 2024-09-21T07:40:39.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.238+0000 7fbcfe400640 1 -- 172.21.15.39:0/1385648475 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcf8072fa0 con 0x7fbcf80b30d0 2024-09-21T07:40:39.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.239+0000 7fbcf5600640 1 -- 172.21.15.39:0/1385648475 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbcf0004030 con 0x7fbcf80b30d0 2024-09-21T07:40:39.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.240+0000 7fbcf5600640 1 --2- 172.21.15.39:0/1385648475 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbcd4078bc0 0x7fbcd407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:39.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.240+0000 7fbcf5600640 1 -- 172.21.15.39:0/1385648475 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7603+0+0 (secure 0 0 0) 0x7fbcf009be30 con 0x7fbcf80b30d0 2024-09-21T07:40:39.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.241+0000 7fbcfd000640 1 --2- 172.21.15.39:0/1385648475 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbcd4078bc0 0x7fbcd407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:39.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.241+0000 7fbcfd000640 1 --2- 172.21.15.39:0/1385648475 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbcd4078bc0 0x7fbcd407b0d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fbce8001550 tx=0x7fbce8006c10 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:39.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.245+0000 7fbcf5600640 1 -- 172.21.15.39:0/1385648475 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbcf80b3a20 con 0x7fbcf80b30d0 2024-09-21T07:40:39.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.482+0000 7fbcfe400640 1 -- 172.21.15.39:0/1385648475 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbcf81f5560 con 0x7fbcf80b30d0 2024-09-21T07:40:39.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.482+0000 7fbcf5600640 1 -- 172.21.15.39:0/1385648475 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) ==== 95+0+5258 (secure 0 0 0) 0x7fbcf0063c20 con 0x7fbcf80b30d0 2024-09-21T07:40:39.484 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:40:39.484 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":116,"btime":"2024-09-21T07:40:32:928729+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7311,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:40:32.468779+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7113,"mds_1":6269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6269":{"gid":6269,"name":"d","rank":1,"incarnation":110,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6837/2336112936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2336112936},{"type":"v1","addr":"172.21.15.148:6837","nonce":2336112936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7113":{"gid":7113,"name":"a","rank":0,"incarnation":107,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/2821446711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2821446711},{"type":"v1","addr":"172.21.15.39:6835","nonce":2821446711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7113,"qdb_cluster":[6269,7113]},"id":6}]} 2024-09-21T07:40:39.485 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 116 2024-09-21T07:40:39.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.485+0000 7fbcfe400640 1 -- 172.21.15.39:0/1385648475 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbcd4078bc0 msgr2=0x7fbcd407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:39.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.485+0000 7fbcfe400640 1 --2- 172.21.15.39:0/1385648475 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbcd4078bc0 0x7fbcd407b0d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fbce8001550 tx=0x7fbce8006c10 comp rx=0 tx=0).stop 2024-09-21T07:40:39.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.485+0000 7fbcfe400640 1 -- 172.21.15.39:0/1385648475 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcf80b30d0 msgr2=0x7fbcf80b34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:39.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.485+0000 7fbcfe400640 1 --2- 172.21.15.39:0/1385648475 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcf80b30d0 0x7fbcf80b34e0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fbcf80cddc0 tx=0x7fbcf0010f40 comp rx=0 tx=0).stop 2024-09-21T07:40:39.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.485+0000 7fbcfe400640 1 -- 172.21.15.39:0/1385648475 shutdown_connections 2024-09-21T07:40:39.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.485+0000 7fbcfe400640 1 --2- 172.21.15.39:0/1385648475 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbcd4078bc0 0x7fbcd407b0d0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:39.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.485+0000 7fbcfe400640 1 --2- 172.21.15.39:0/1385648475 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcf80b30d0 0x7fbcf80b34e0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:39.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.485+0000 7fbcfe400640 1 --2- 172.21.15.39:0/1385648475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcf8158450 0x7fbcf815bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:39.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.485+0000 7fbcfe400640 1 --2- 172.21.15.39:0/1385648475 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbcdc003ff0 0x7fbcf815b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:39.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.485+0000 7fbcfe400640 1 -- 172.21.15.39:0/1385648475 >> 172.21.15.39:0/1385648475 conn(0x7fbcf80b2510 msgr2=0x7fbcf80b4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:39.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.485+0000 7fbcfe400640 1 -- 172.21.15.39:0/1385648475 shutdown_connections 2024-09-21T07:40:39.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:39.485+0000 7fbcfe400640 1 -- 172.21.15.39:0/1385648475 wait complete. 2024-09-21T07:40:39.509 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:40:42.836 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-09-21T07:40:44.510 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:40:44.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.749+0000 7fe6a2e00640 1 -- 172.21.15.39:0/3319200400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe69c06f800 msgr2=0x7fe69c0cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:44.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.749+0000 7fe6a2e00640 1 --2- 172.21.15.39:0/3319200400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe69c06f800 0x7fe69c0cd8e0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fe688005fd0 tx=0x7fe68802a5d0 comp rx=0 tx=0).stop 2024-09-21T07:40:44.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.749+0000 7fe6a2e00640 1 -- 172.21.15.39:0/3319200400 shutdown_connections 2024-09-21T07:40:44.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.749+0000 7fe6a2e00640 1 --2- 172.21.15.39:0/3319200400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe69c06f800 0x7fe69c0cd8e0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:44.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.749+0000 7fe6a2e00640 1 --2- 172.21.15.39:0/3319200400 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe69c15cd30 0x7fe69c0cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:44.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.749+0000 7fe6a2e00640 1 -- 172.21.15.39:0/3319200400 >> 172.21.15.39:0/3319200400 conn(0x7fe69c06cfc0 msgr2=0x7fe69c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:44.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.749+0000 7fe6a2e00640 1 -- 172.21.15.39:0/3319200400 shutdown_connections 2024-09-21T07:40:44.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.749+0000 7fe6a2e00640 1 -- 172.21.15.39:0/3319200400 wait complete. 2024-09-21T07:40:44.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.751+0000 7fe6a2e00640 1 Processor -- start 2024-09-21T07:40:44.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.751+0000 7fe6a2e00640 1 -- start start 2024-09-21T07:40:44.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.753+0000 7fe6a2e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe69c06f800 0x7fe69c150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:44.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.753+0000 7fe6a2e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe69c06fc10 0x7fe69c1585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:44.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.753+0000 7fe6a2e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe69c15cd30 0x7fe69c1f8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:44.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.753+0000 7fe6a2e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe69c073890 con 0x7fe69c15cd30 2024-09-21T07:40:44.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.753+0000 7fe6a2e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe69c073710 con 0x7fe69c06fc10 2024-09-21T07:40:44.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.753+0000 7fe6a2e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe69c073a10 con 0x7fe69c06f800 2024-09-21T07:40:44.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.753+0000 7fe6a0e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe69c15cd30 0x7fe69c1f8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:44.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.753+0000 7fe6a0e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe69c15cd30 0x7fe69c1f8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48706/0 (socket says 172.21.15.39:48706) 2024-09-21T07:40:44.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.753+0000 7fe6a0e00640 1 -- 172.21.15.39:0/987482920 learned_addr learned my addr 172.21.15.39:0/987482920 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:40:44.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.754+0000 7fe69be00640 1 --2- 172.21.15.39:0/987482920 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe69c06f800 0x7fe69c150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:44.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.754+0000 7fe69b400640 1 --2- 172.21.15.39:0/987482920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe69c06fc10 0x7fe69c1585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:44.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.754+0000 7fe6a0e00640 1 -- 172.21.15.39:0/987482920 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe69c06f800 msgr2=0x7fe69c150ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:44.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.754+0000 7fe6a0e00640 1 --2- 172.21.15.39:0/987482920 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe69c06f800 0x7fe69c150ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:44.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.754+0000 7fe6a0e00640 1 -- 172.21.15.39:0/987482920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe69c06fc10 msgr2=0x7fe69c1585a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:44.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.754+0000 7fe6a0e00640 1 --2- 172.21.15.39:0/987482920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe69c06fc10 0x7fe69c1585a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:44.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.754+0000 7fe6a0e00640 1 -- 172.21.15.39:0/987482920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe69c1592d0 con 0x7fe69c15cd30 2024-09-21T07:40:44.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.754+0000 7fe6a0e00640 1 --2- 172.21.15.39:0/987482920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe69c15cd30 0x7fe69c1f8d10 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fe69400c760 tx=0x7fe69400cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:44.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.754+0000 7fe693e00640 1 -- 172.21.15.39:0/987482920 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe694003b70 con 0x7fe69c15cd30 2024-09-21T07:40:44.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.754+0000 7fe693e00640 1 -- 172.21.15.39:0/987482920 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe694003d10 con 0x7fe69c15cd30 2024-09-21T07:40:44.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.754+0000 7fe693e00640 1 -- 172.21.15.39:0/987482920 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6940052d0 con 0x7fe69c15cd30 2024-09-21T07:40:44.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.754+0000 7fe6a2e00640 1 -- 172.21.15.39:0/987482920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe69c1595b0 con 0x7fe69c15cd30 2024-09-21T07:40:44.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.755+0000 7fe6a2e00640 1 -- 172.21.15.39:0/987482920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe69c159b70 con 0x7fe69c15cd30 2024-09-21T07:40:44.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.756+0000 7fe6a2e00640 1 -- 172.21.15.39:0/987482920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe69c073ec0 con 0x7fe69c15cd30 2024-09-21T07:40:44.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.757+0000 7fe693e00640 1 -- 172.21.15.39:0/987482920 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe694005470 con 0x7fe69c15cd30 2024-09-21T07:40:44.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.758+0000 7fe693e00640 1 --2- 172.21.15.39:0/987482920 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe67c078b90 0x7fe67c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:44.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.758+0000 7fe693e00640 1 -- 172.21.15.39:0/987482920 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7603+0+0 (secure 0 0 0) 0x7fe69409b2a0 con 0x7fe69c15cd30 2024-09-21T07:40:44.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.758+0000 7fe69be00640 1 --2- 172.21.15.39:0/987482920 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe67c078b90 0x7fe67c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:44.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.759+0000 7fe69be00640 1 --2- 172.21.15.39:0/987482920 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe67c078b90 0x7fe67c07b0a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe68c005fd0 tx=0x7fe68c005850 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:44.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.762+0000 7fe693e00640 1 -- 172.21.15.39:0/987482920 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe694062f60 con 0x7fe69c15cd30 2024-09-21T07:40:44.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.995+0000 7fe6a2e00640 1 -- 172.21.15.39:0/987482920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe69c159060 con 0x7fe69c15cd30 2024-09-21T07:40:44.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.996+0000 7fe693e00640 1 -- 172.21.15.39:0/987482920 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) ==== 95+0+5258 (secure 0 0 0) 0x7fe694062860 con 0x7fe69c15cd30 2024-09-21T07:40:44.998 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:40:44.998 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":116,"btime":"2024-09-21T07:40:32:928729+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7311,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:40:32.468779+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7113,"mds_1":6269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6269":{"gid":6269,"name":"d","rank":1,"incarnation":110,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6837/2336112936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2336112936},{"type":"v1","addr":"172.21.15.148:6837","nonce":2336112936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7113":{"gid":7113,"name":"a","rank":0,"incarnation":107,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/2821446711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2821446711},{"type":"v1","addr":"172.21.15.39:6835","nonce":2821446711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7113,"qdb_cluster":[6269,7113]},"id":6}]} 2024-09-21T07:40:44.998 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 116 2024-09-21T07:40:45.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.998+0000 7fe6a2e00640 1 -- 172.21.15.39:0/987482920 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe67c078b90 msgr2=0x7fe67c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:45.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.998+0000 7fe6a2e00640 1 --2- 172.21.15.39:0/987482920 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe67c078b90 0x7fe67c07b0a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe68c005fd0 tx=0x7fe68c005850 comp rx=0 tx=0).stop 2024-09-21T07:40:45.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.998+0000 7fe6a2e00640 1 -- 172.21.15.39:0/987482920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe69c15cd30 msgr2=0x7fe69c1f8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:45.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.998+0000 7fe6a2e00640 1 --2- 172.21.15.39:0/987482920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe69c15cd30 0x7fe69c1f8d10 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fe69400c760 tx=0x7fe69400cb60 comp rx=0 tx=0).stop 2024-09-21T07:40:45.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.998+0000 7fe6a2e00640 1 -- 172.21.15.39:0/987482920 shutdown_connections 2024-09-21T07:40:45.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.998+0000 7fe6a2e00640 1 --2- 172.21.15.39:0/987482920 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe67c078b90 0x7fe67c07b0a0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:45.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.998+0000 7fe6a2e00640 1 --2- 172.21.15.39:0/987482920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe69c15cd30 0x7fe69c1f8d10 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:45.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.998+0000 7fe6a2e00640 1 --2- 172.21.15.39:0/987482920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe69c06fc10 0x7fe69c1585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:45.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.998+0000 7fe6a2e00640 1 --2- 172.21.15.39:0/987482920 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe69c06f800 0x7fe69c150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:45.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.998+0000 7fe6a2e00640 1 -- 172.21.15.39:0/987482920 >> 172.21.15.39:0/987482920 conn(0x7fe69c06cfc0 msgr2=0x7fe69c070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:45.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.998+0000 7fe6a2e00640 1 -- 172.21.15.39:0/987482920 shutdown_connections 2024-09-21T07:40:45.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:44.998+0000 7fe6a2e00640 1 -- 172.21.15.39:0/987482920 wait complete. 2024-09-21T07:40:45.026 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:40:49.448 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~7s 2024-09-21T07:40:50.027 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:40:50.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.263+0000 7f306ca00640 1 -- 172.21.15.39:0/2583462395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f304c003a00 msgr2=0x7f304c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:50.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.263+0000 7f306ca00640 1 --2- 172.21.15.39:0/2583462395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f304c003a00 0x7f304c003e70 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f3068071120 tx=0x7f304c005830 comp rx=0 tx=0).stop 2024-09-21T07:40:50.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.263+0000 7f3067200640 1 -- 172.21.15.39:0/2583462395 reap_dead start 2024-09-21T07:40:50.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.264+0000 7f306ca00640 1 -- 172.21.15.39:0/2583462395 shutdown_connections 2024-09-21T07:40:50.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.264+0000 7f306ca00640 1 -- 172.21.15.39:0/2583462395 >> 172.21.15.39:0/2583462395 conn(0x7f30680b2510 msgr2=0x7f30680d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:50.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.264+0000 7f306ca00640 1 -- 172.21.15.39:0/2583462395 shutdown_connections 2024-09-21T07:40:50.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.264+0000 7f306ca00640 1 -- 172.21.15.39:0/2583462395 wait complete. 2024-09-21T07:40:50.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.266+0000 7f306ca00640 1 Processor -- start 2024-09-21T07:40:50.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.266+0000 7f306ca00640 1 -- start start 2024-09-21T07:40:50.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.267+0000 7f306ca00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f304c003a00 0x7f30681f53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:50.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.267+0000 7f306ca00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f30680b0a00 0x7f30681f66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:50.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.267+0000 7f306ca00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30680c5d80 0x7f3068159680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:50.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.268+0000 7f306ca00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f306806fb20 con 0x7f30680c5d80 2024-09-21T07:40:50.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.268+0000 7f306ca00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f30680af950 con 0x7f304c003a00 2024-09-21T07:40:50.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.268+0000 7f306ca00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f306806fca0 con 0x7f30680b0a00 2024-09-21T07:40:50.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.268+0000 7f3067200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30680c5d80 0x7f3068159680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:50.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.268+0000 7f3067200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30680c5d80 0x7f3068159680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48718/0 (socket says 172.21.15.39:48718) 2024-09-21T07:40:50.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.268+0000 7f3067200640 1 -- 172.21.15.39:0/3858327993 learned_addr learned my addr 172.21.15.39:0/3858327993 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:40:50.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.268+0000 7f305de00640 1 --2- 172.21.15.39:0/3858327993 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f30680b0a00 0x7f30681f66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:50.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.268+0000 7f3066800640 1 --2- 172.21.15.39:0/3858327993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f304c003a00 0x7f30681f53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:50.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.268+0000 7f3067200640 1 -- 172.21.15.39:0/3858327993 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f30680b0a00 msgr2=0x7f30681f66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:50.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.268+0000 7f3067200640 1 --2- 172.21.15.39:0/3858327993 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f30680b0a00 0x7f30681f66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:50.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.268+0000 7f3067200640 1 -- 172.21.15.39:0/3858327993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f304c003a00 msgr2=0x7f30681f53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:50.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.268+0000 7f3067200640 1 --2- 172.21.15.39:0/3858327993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f304c003a00 0x7f30681f53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:50.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.268+0000 7f3067200640 1 -- 172.21.15.39:0/3858327993 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30681f5920 con 0x7f30680c5d80 2024-09-21T07:40:50.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.269+0000 7f3067200640 1 --2- 172.21.15.39:0/3858327993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30680c5d80 0x7f3068159680 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f30680b3420 tx=0x7f3060006660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:50.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.269+0000 7f305f400640 1 -- 172.21.15.39:0/3858327993 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306000fbb0 con 0x7f30680c5d80 2024-09-21T07:40:50.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.269+0000 7f305f400640 1 -- 172.21.15.39:0/3858327993 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f306000fd70 con 0x7f30680c5d80 2024-09-21T07:40:50.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.269+0000 7f306ca00640 1 -- 172.21.15.39:0/3858327993 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f306815a350 con 0x7f30680c5d80 2024-09-21T07:40:50.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.269+0000 7f305f400640 1 -- 172.21.15.39:0/3858327993 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3060011690 con 0x7f30680c5d80 2024-09-21T07:40:50.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.269+0000 7f306ca00640 1 -- 172.21.15.39:0/3858327993 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3068157bd0 con 0x7f30680c5d80 2024-09-21T07:40:50.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.270+0000 7f306ca00640 1 -- 172.21.15.39:0/3858327993 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30680c6950 con 0x7f30680c5d80 2024-09-21T07:40:50.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.271+0000 7f305f400640 1 -- 172.21.15.39:0/3858327993 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3060018020 con 0x7f30680c5d80 2024-09-21T07:40:50.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.273+0000 7f305f400640 1 --2- 172.21.15.39:0/3858327993 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3044078b90 0x7f304407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:50.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.273+0000 7f305f400640 1 -- 172.21.15.39:0/3858327993 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7603+0+0 (secure 0 0 0) 0x7f306009bdd0 con 0x7f30680c5d80 2024-09-21T07:40:50.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.273+0000 7f3066800640 1 --2- 172.21.15.39:0/3858327993 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3044078b90 0x7f304407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:50.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.274+0000 7f3066800640 1 --2- 172.21.15.39:0/3858327993 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3044078b90 0x7f304407b0a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3058035000 tx=0x7f305802d040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:50.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.278+0000 7f305f400640 1 -- 172.21.15.39:0/3858327993 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3060015030 con 0x7f30680c5d80 2024-09-21T07:40:50.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.514+0000 7f306ca00640 1 -- 172.21.15.39:0/3858327993 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30680ce720 con 0x7f30680c5d80 2024-09-21T07:40:50.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.515+0000 7f305f400640 1 -- 172.21.15.39:0/3858327993 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) ==== 95+0+5258 (secure 0 0 0) 0x7f306009d090 con 0x7f30680c5d80 2024-09-21T07:40:50.517 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:40:50.517 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":116,"btime":"2024-09-21T07:40:32:928729+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7311,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:40:32.468779+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7113,"mds_1":6269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6269":{"gid":6269,"name":"d","rank":1,"incarnation":110,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6837/2336112936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2336112936},{"type":"v1","addr":"172.21.15.148:6837","nonce":2336112936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7113":{"gid":7113,"name":"a","rank":0,"incarnation":107,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/2821446711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2821446711},{"type":"v1","addr":"172.21.15.39:6835","nonce":2821446711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7113,"qdb_cluster":[6269,7113]},"id":6}]} 2024-09-21T07:40:50.517 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 116 2024-09-21T07:40:50.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.517+0000 7f306ca00640 1 -- 172.21.15.39:0/3858327993 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3044078b90 msgr2=0x7f304407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:50.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.517+0000 7f306ca00640 1 --2- 172.21.15.39:0/3858327993 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3044078b90 0x7f304407b0a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3058035000 tx=0x7f305802d040 comp rx=0 tx=0).stop 2024-09-21T07:40:50.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.517+0000 7f306ca00640 1 -- 172.21.15.39:0/3858327993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30680c5d80 msgr2=0x7f3068159680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:50.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.517+0000 7f306ca00640 1 --2- 172.21.15.39:0/3858327993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30680c5d80 0x7f3068159680 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f30680b3420 tx=0x7f3060006660 comp rx=0 tx=0).stop 2024-09-21T07:40:50.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.517+0000 7f306ca00640 1 -- 172.21.15.39:0/3858327993 shutdown_connections 2024-09-21T07:40:50.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.517+0000 7f306ca00640 1 --2- 172.21.15.39:0/3858327993 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3044078b90 0x7f304407b0a0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:50.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.517+0000 7f306ca00640 1 --2- 172.21.15.39:0/3858327993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30680c5d80 0x7f3068159680 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:50.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.517+0000 7f306ca00640 1 --2- 172.21.15.39:0/3858327993 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f30680b0a00 0x7f30681f66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:50.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.517+0000 7f306ca00640 1 --2- 172.21.15.39:0/3858327993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f304c003a00 0x7f30681f53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:50.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.517+0000 7f306ca00640 1 -- 172.21.15.39:0/3858327993 >> 172.21.15.39:0/3858327993 conn(0x7f30680b2510 msgr2=0x7f30680cb6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:50.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.517+0000 7f306ca00640 1 -- 172.21.15.39:0/3858327993 shutdown_connections 2024-09-21T07:40:50.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:50.517+0000 7f306ca00640 1 -- 172.21.15.39:0/3858327993 wait complete. 2024-09-21T07:40:50.545 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:40:55.546 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:40:55.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:55.785+0000 7f681de00640 1 -- 172.21.15.39:0/3815898392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67fc004800 msgr2=0x7f67fc004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:55.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:55.785+0000 7f681de00640 1 --2- 172.21.15.39:0/3815898392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67fc004800 0x7f67fc004c10 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f6804001830 tx=0x7f67fc005830 comp rx=0 tx=0).stop 2024-09-21T07:40:55.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:55.785+0000 7f6817400640 1 -- 172.21.15.39:0/3815898392 reap_dead start 2024-09-21T07:40:55.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:55.785+0000 7f681de00640 1 -- 172.21.15.39:0/3815898392 shutdown_connections 2024-09-21T07:40:55.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:55.785+0000 7f681de00640 1 -- 172.21.15.39:0/3815898392 >> 172.21.15.39:0/3815898392 conn(0x7f681806cfc0 msgr2=0x7f681806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:55.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:55.785+0000 7f681de00640 1 -- 172.21.15.39:0/3815898392 shutdown_connections 2024-09-21T07:40:55.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:55.785+0000 7f681de00640 1 -- 172.21.15.39:0/3815898392 wait complete. 2024-09-21T07:40:55.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:55.787+0000 7f681de00640 1 Processor -- start 2024-09-21T07:40:55.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:55.787+0000 7f681de00640 1 -- start start 2024-09-21T07:40:55.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:55.789+0000 7f681de00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67fc004800 0x7f6818145b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:55.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:55.789+0000 7f681de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68181f8ef0 0x7f68181f9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:55.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:55.789+0000 7f681de00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68181f9840 0x7f68180aff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:55.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:55.789+0000 7f681de00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f68180b4e30 con 0x7f68181f8ef0 2024-09-21T07:40:55.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:55.789+0000 7f681de00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f68180b4cb0 con 0x7f67fc004800 2024-09-21T07:40:55.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:55.789+0000 7f681de00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f68180b4240 con 0x7f68181f9840 2024-09-21T07:40:55.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:55.789+0000 7f6816000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68181f8ef0 0x7f68181f9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:55.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:55.789+0000 7f6816000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68181f8ef0 0x7f68181f9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37174/0 (socket says 172.21.15.39:37174) 2024-09-21T07:40:55.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:55.789+0000 7f6816000640 1 -- 172.21.15.39:0/3616047297 learned_addr learned my addr 172.21.15.39:0/3616047297 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:40:55.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:55.789+0000 7f6816a00640 1 --2- 172.21.15.39:0/3616047297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67fc004800 0x7f6818145b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:55.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:55.789+0000 7f6817400640 1 --2- 172.21.15.39:0/3616047297 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68181f9840 0x7f68180aff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:55.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:55.789+0000 7f6816000640 1 -- 172.21.15.39:0/3616047297 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68181f9840 msgr2=0x7f68180aff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:55.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:55.789+0000 7f6816000640 1 --2- 172.21.15.39:0/3616047297 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68181f9840 0x7f68180aff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:55.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:55.790+0000 7f6816000640 1 -- 172.21.15.39:0/3616047297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67fc004800 msgr2=0x7f6818145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:55.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:55.790+0000 7f6816000640 1 --2- 172.21.15.39:0/3616047297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67fc004800 0x7f6818145b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:55.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:55.790+0000 7f6816000640 1 -- 172.21.15.39:0/3616047297 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f681815a7a0 con 0x7f68181f8ef0 2024-09-21T07:40:55.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:55.790+0000 7f6816000640 1 --2- 172.21.15.39:0/3616047297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68181f8ef0 0x7f68181f9300 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f680402aa50 tx=0x7f6804006150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:55.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:55.790+0000 7f680f400640 1 -- 172.21.15.39:0/3616047297 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6804043070 con 0x7f68181f8ef0 2024-09-21T07:40:55.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:55.790+0000 7f680f400640 1 -- 172.21.15.39:0/3616047297 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6804003af0 con 0x7f68181f8ef0 2024-09-21T07:40:55.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:55.790+0000 7f681de00640 1 -- 172.21.15.39:0/3616047297 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f681815ac40 con 0x7f68181f8ef0 2024-09-21T07:40:55.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:55.790+0000 7f680f400640 1 -- 172.21.15.39:0/3616047297 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6804033480 con 0x7f68181f8ef0 2024-09-21T07:40:55.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:55.790+0000 7f681de00640 1 -- 172.21.15.39:0/3616047297 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f681815b200 con 0x7f68181f8ef0 2024-09-21T07:40:55.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:55.791+0000 7f681de00640 1 -- 172.21.15.39:0/3616047297 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68180cc670 con 0x7f68181f8ef0 2024-09-21T07:40:55.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:55.793+0000 7f680f400640 1 -- 172.21.15.39:0/3616047297 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6804046070 con 0x7f68181f8ef0 2024-09-21T07:40:55.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:55.794+0000 7f680f400640 1 --2- 172.21.15.39:0/3616047297 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f67f8078bc0 0x7f67f807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:55.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:55.794+0000 7f680f400640 1 -- 172.21.15.39:0/3616047297 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7603+0+0 (secure 0 0 0) 0x7f68040bb530 con 0x7f68181f8ef0 2024-09-21T07:40:55.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:55.794+0000 7f6816a00640 1 --2- 172.21.15.39:0/3616047297 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f67f8078bc0 0x7f67f807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:55.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:55.795+0000 7f6816a00640 1 --2- 172.21.15.39:0/3616047297 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f67f8078bc0 0x7f67f807b0d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f6808000fd0 tx=0x7f6808001ad0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:55.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:55.798+0000 7f680f400640 1 -- 172.21.15.39:0/3616047297 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f68180cc670 con 0x7f68181f8ef0 2024-09-21T07:40:56.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.036+0000 7f681de00640 1 -- 172.21.15.39:0/3616047297 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68180cd6b0 con 0x7f68181f8ef0 2024-09-21T07:40:56.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.036+0000 7f680f400640 1 -- 172.21.15.39:0/3616047297 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) ==== 95+0+5306 (secure 0 0 0) 0x7f68180cd6b0 con 0x7f68181f8ef0 2024-09-21T07:40:56.038 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:40:56.038 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":117,"btime":"2024-09-21T07:40:51:419289+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7311,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111}],"filesystems":[{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:40:51.419263+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7113,"mds_1":6269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6269":{"gid":6269,"name":"d","rank":1,"incarnation":110,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6837/2336112936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2336112936},{"type":"v1","addr":"172.21.15.148:6837","nonce":2336112936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7113":{"gid":7113,"name":"a","rank":0,"incarnation":107,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/2821446711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2821446711},{"type":"v1","addr":"172.21.15.39:6835","nonce":2821446711}]},"join_fscid":-1,"laggy_since":"2024-09-21T07:40:51.419263+0000","export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7113,"qdb_cluster":[7113,6269]},"id":6}]} 2024-09-21T07:40:56.038 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 117 2024-09-21T07:40:56.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.039+0000 7f681de00640 1 -- 172.21.15.39:0/3616047297 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f67f8078bc0 msgr2=0x7f67f807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:56.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.039+0000 7f681de00640 1 --2- 172.21.15.39:0/3616047297 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f67f8078bc0 0x7f67f807b0d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f6808000fd0 tx=0x7f6808001ad0 comp rx=0 tx=0).stop 2024-09-21T07:40:56.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.039+0000 7f681de00640 1 -- 172.21.15.39:0/3616047297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68181f8ef0 msgr2=0x7f68181f9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:56.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.039+0000 7f681de00640 1 --2- 172.21.15.39:0/3616047297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68181f8ef0 0x7f68181f9300 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f680402aa50 tx=0x7f6804006150 comp rx=0 tx=0).stop 2024-09-21T07:40:56.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.039+0000 7f681de00640 1 -- 172.21.15.39:0/3616047297 shutdown_connections 2024-09-21T07:40:56.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.039+0000 7f681de00640 1 --2- 172.21.15.39:0/3616047297 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f67f8078bc0 0x7f67f807b0d0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:56.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.039+0000 7f681de00640 1 --2- 172.21.15.39:0/3616047297 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68181f9840 0x7f68180aff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:56.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.039+0000 7f681de00640 1 --2- 172.21.15.39:0/3616047297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68181f8ef0 0x7f68181f9300 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:56.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.039+0000 7f681de00640 1 --2- 172.21.15.39:0/3616047297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67fc004800 0x7f6818145b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:56.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.039+0000 7f681de00640 1 -- 172.21.15.39:0/3616047297 >> 172.21.15.39:0/3616047297 conn(0x7f681806cfc0 msgr2=0x7f681806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:56.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.039+0000 7f681de00640 1 -- 172.21.15.39:0/3616047297 shutdown_connections 2024-09-21T07:40:56.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.039+0000 7f681de00640 1 -- 172.21.15.39:0/3616047297 wait complete. 2024-09-21T07:40:56.061 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~13s 2024-09-21T07:40:56.063 DEBUG:tasks.ceph_test_case:wait_until_true: success in 20s and 0 retries 2024-09-21T07:40:56.064 DEBUG:teuthology.orchestra.run.smithi039:> sudo sysctl -n kernel.core_pattern 2024-09-21T07:40:56.117 INFO:teuthology.orchestra.run.smithi039.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T07:40:56.120 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2024-09-21T07:40:56.120 DEBUG:teuthology.orchestra.run.smithi039:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2024-09-21T07:40:56.182 INFO:teuthology.orchestra.run.smithi039.stdout:1726904435.62196.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i a', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2024-09-21T07:40:56.186 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1726904435.62196.core'] 2024-09-21T07:40:56.186 INFO:tasks.cephfs.cephfs_test_case:Found core file 1726904435.62196.core, deleting it 2024-09-21T07:40:56.187 DEBUG:teuthology.orchestra.run.smithi039:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1726904435.62196.core 2024-09-21T07:40:56.283 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:40:56.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.511+0000 7fb6dd000640 1 -- 172.21.15.39:0/3932419617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6d815cd30 msgr2=0x7fb6d80cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:56.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.511+0000 7fb6dd000640 1 --2- 172.21.15.39:0/3932419617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6d815cd30 0x7fb6d80cfd10 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8005fd0 tx=0x7fb6c802a5d0 comp rx=0 tx=0).stop 2024-09-21T07:40:56.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.511+0000 7fb6dd000640 1 -- 172.21.15.39:0/3932419617 shutdown_connections 2024-09-21T07:40:56.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.511+0000 7fb6dd000640 1 --2- 172.21.15.39:0/3932419617 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb6d806fc10 0x7fb6d80d0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:56.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.511+0000 7fb6dd000640 1 --2- 172.21.15.39:0/3932419617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6d815cd30 0x7fb6d80cfd10 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:56.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.511+0000 7fb6dd000640 1 -- 172.21.15.39:0/3932419617 >> 172.21.15.39:0/3932419617 conn(0x7fb6d806cfc0 msgr2=0x7fb6d806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:56.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.511+0000 7fb6dd000640 1 -- 172.21.15.39:0/3932419617 shutdown_connections 2024-09-21T07:40:56.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.511+0000 7fb6dd000640 1 -- 172.21.15.39:0/3932419617 wait complete. 2024-09-21T07:40:56.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.513+0000 7fb6dd000640 1 Processor -- start 2024-09-21T07:40:56.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.513+0000 7fb6dd000640 1 -- start start 2024-09-21T07:40:56.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.515+0000 7fb6dd000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6d806f800 0x7fb6d81e4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:56.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.515+0000 7fb6dd000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6d806fc10 0x7fb6d81458e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:56.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.515+0000 7fb6dd000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb6d815cd30 0x7fb6d81e67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:56.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.515+0000 7fb6dd000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d8073890 con 0x7fb6d806f800 2024-09-21T07:40:56.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.515+0000 7fb6dd000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d8073710 con 0x7fb6d806fc10 2024-09-21T07:40:56.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.515+0000 7fb6dd000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb6d8073a10 con 0x7fb6d815cd30 2024-09-21T07:40:56.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.515+0000 7fb6d5e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6d806f800 0x7fb6d81e4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:56.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.515+0000 7fb6d5e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6d806f800 0x7fb6d81e4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37178/0 (socket says 172.21.15.39:37178) 2024-09-21T07:40:56.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.515+0000 7fb6d5e00640 1 -- 172.21.15.39:0/3526362073 learned_addr learned my addr 172.21.15.39:0/3526362073 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:40:56.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.515+0000 7fb6d5400640 1 --2- 172.21.15.39:0/3526362073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6d806fc10 0x7fb6d81458e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:56.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.515+0000 7fb6d6800640 1 --2- 172.21.15.39:0/3526362073 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb6d815cd30 0x7fb6d81e67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:56.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.515+0000 7fb6d5e00640 1 -- 172.21.15.39:0/3526362073 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb6d815cd30 msgr2=0x7fb6d81e67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:56.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.515+0000 7fb6d5e00640 1 --2- 172.21.15.39:0/3526362073 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb6d815cd30 0x7fb6d81e67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:56.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.515+0000 7fb6d5e00640 1 -- 172.21.15.39:0/3526362073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6d806fc10 msgr2=0x7fb6d81458e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:56.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.515+0000 7fb6d5e00640 1 --2- 172.21.15.39:0/3526362073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6d806fc10 0x7fb6d81458e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:56.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.515+0000 7fb6d5e00640 1 -- 172.21.15.39:0/3526362073 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d8145e20 con 0x7fb6d806f800 2024-09-21T07:40:56.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.516+0000 7fb6d5e00640 1 --2- 172.21.15.39:0/3526362073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6d806f800 0x7fb6d81e4ec0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8000c00 tx=0x7fb6c80035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:56.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.516+0000 7fb6ce000640 1 -- 172.21.15.39:0/3526362073 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c803e030 con 0x7fb6d806f800 2024-09-21T07:40:56.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.516+0000 7fb6dd000640 1 -- 172.21.15.39:0/3526362073 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6d81e5710 con 0x7fb6d806f800 2024-09-21T07:40:56.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.516+0000 7fb6ce000640 1 -- 172.21.15.39:0/3526362073 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb6c8003ae0 con 0x7fb6d806f800 2024-09-21T07:40:56.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.516+0000 7fb6ce000640 1 -- 172.21.15.39:0/3526362073 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8035040 con 0x7fb6d806f800 2024-09-21T07:40:56.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.516+0000 7fb6dd000640 1 -- 172.21.15.39:0/3526362073 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6d81e5cd0 con 0x7fb6d806f800 2024-09-21T07:40:56.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.517+0000 7fb6dd000640 1 -- 172.21.15.39:0/3526362073 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6d8073ec0 con 0x7fb6d806f800 2024-09-21T07:40:56.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.519+0000 7fb6ce000640 1 -- 172.21.15.39:0/3526362073 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb6c8041030 con 0x7fb6d806f800 2024-09-21T07:40:56.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.519+0000 7fb6ce000640 1 --2- 172.21.15.39:0/3526362073 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb6b8078b90 0x7fb6b807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:56.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.520+0000 7fb6ce000640 1 -- 172.21.15.39:0/3526362073 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7603+0+0 (secure 0 0 0) 0x7fb6c80bba30 con 0x7fb6d806f800 2024-09-21T07:40:56.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.520+0000 7fb6d5400640 1 --2- 172.21.15.39:0/3526362073 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb6b8078b90 0x7fb6b807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:56.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.521+0000 7fb6d5400640 1 --2- 172.21.15.39:0/3526362073 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb6b8078b90 0x7fb6b807b0a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb6d815a7f0 tx=0x7fb6c4009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:56.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.524+0000 7fb6ce000640 1 -- 172.21.15.39:0/3526362073 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb6c8082f90 con 0x7fb6d806f800 2024-09-21T07:40:56.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.761+0000 7fb6dd000640 1 -- 172.21.15.39:0/3526362073 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6d80d0790 con 0x7fb6d806f800 2024-09-21T07:40:56.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.761+0000 7fb6ce000640 1 -- 172.21.15.39:0/3526362073 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) ==== 95+0+5306 (secure 0 0 0) 0x7fb6c80879c0 con 0x7fb6d806f800 2024-09-21T07:40:56.763 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:40:56.763 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":117,"btime":"2024-09-21T07:40:51:419289+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7311,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111}],"filesystems":[{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:40:51.419263+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7113,"mds_1":6269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6269":{"gid":6269,"name":"d","rank":1,"incarnation":110,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6837/2336112936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2336112936},{"type":"v1","addr":"172.21.15.148:6837","nonce":2336112936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7113":{"gid":7113,"name":"a","rank":0,"incarnation":107,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/2821446711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2821446711},{"type":"v1","addr":"172.21.15.39:6835","nonce":2821446711}]},"join_fscid":-1,"laggy_since":"2024-09-21T07:40:51.419263+0000","export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7113,"qdb_cluster":[7113,6269]},"id":6}]} 2024-09-21T07:40:56.763 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 117 2024-09-21T07:40:56.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.764+0000 7fb6dd000640 1 -- 172.21.15.39:0/3526362073 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb6b8078b90 msgr2=0x7fb6b807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:56.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.764+0000 7fb6dd000640 1 --2- 172.21.15.39:0/3526362073 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb6b8078b90 0x7fb6b807b0a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb6d815a7f0 tx=0x7fb6c4009040 comp rx=0 tx=0).stop 2024-09-21T07:40:56.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.764+0000 7fb6dd000640 1 -- 172.21.15.39:0/3526362073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6d806f800 msgr2=0x7fb6d81e4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:56.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.764+0000 7fb6dd000640 1 --2- 172.21.15.39:0/3526362073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6d806f800 0x7fb6d81e4ec0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8000c00 tx=0x7fb6c80035f0 comp rx=0 tx=0).stop 2024-09-21T07:40:56.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.764+0000 7fb6dd000640 1 -- 172.21.15.39:0/3526362073 shutdown_connections 2024-09-21T07:40:56.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.764+0000 7fb6dd000640 1 --2- 172.21.15.39:0/3526362073 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb6b8078b90 0x7fb6b807b0a0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:56.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.764+0000 7fb6dd000640 1 --2- 172.21.15.39:0/3526362073 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb6d815cd30 0x7fb6d81e67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:56.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.764+0000 7fb6dd000640 1 --2- 172.21.15.39:0/3526362073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6d806fc10 0x7fb6d81458e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:56.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.764+0000 7fb6dd000640 1 --2- 172.21.15.39:0/3526362073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6d806f800 0x7fb6d81e4ec0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:56.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.764+0000 7fb6dd000640 1 -- 172.21.15.39:0/3526362073 >> 172.21.15.39:0/3526362073 conn(0x7fb6d806cfc0 msgr2=0x7fb6d80b02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:56.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.764+0000 7fb6dd000640 1 -- 172.21.15.39:0/3526362073 shutdown_connections 2024-09-21T07:40:56.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:56.764+0000 7fb6dd000640 1 -- 172.21.15.39:0/3526362073 wait complete. 2024-09-21T07:40:56.791 INFO:tasks.ceph.mds.d:Sent signal 9 2024-09-21T07:40:56.791 INFO:tasks.cephfs.mount:Suspending the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:40:56.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:40:56.791 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 down 2024-09-21T07:40:56.854 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 6:0 2024-09-21T07:40:56.993 INFO:tasks.ceph.mds.d.smithi148.stderr:daemon-helper: command crashed with signal 9 2024-09-21T07:40:57.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.080+0000 7fe0eec00640 1 -- 172.21.15.39:0/873188981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0e8160440 msgr2=0x7fe0e80cda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:57.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.080+0000 7fe0eec00640 1 --2- 172.21.15.39:0/873188981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0e8160440 0x7fe0e80cda40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8009830 tx=0x7fe0d802a520 comp rx=0 tx=0).stop 2024-09-21T07:40:57.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.080+0000 7fe0eec00640 1 -- 172.21.15.39:0/873188981 shutdown_connections 2024-09-21T07:40:57.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.080+0000 7fe0eec00640 1 --2- 172.21.15.39:0/873188981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0e8160440 0x7fe0e80cda40 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:57.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.080+0000 7fe0eec00640 1 -- 172.21.15.39:0/873188981 >> 172.21.15.39:0/873188981 conn(0x7fe0e806cfc0 msgr2=0x7fe0e806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:57.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.080+0000 7fe0eec00640 1 -- 172.21.15.39:0/873188981 shutdown_connections 2024-09-21T07:40:57.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.080+0000 7fe0eec00640 1 -- 172.21.15.39:0/873188981 wait complete. 2024-09-21T07:40:57.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.083+0000 7fe0eec00640 1 Processor -- start 2024-09-21T07:40:57.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.083+0000 7fe0eec00640 1 -- start start 2024-09-21T07:40:57.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.084+0000 7fe0eec00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0e80d09a0 0x7fe0e81f6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:57.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.084+0000 7fe0eec00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0e815db90 0x7fe0e81f2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:57.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.084+0000 7fe0eec00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe0e8160440 0x7fe0e8073150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:57.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.084+0000 7fe0eec00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e8070af0 con 0x7fe0e80d09a0 2024-09-21T07:40:57.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.084+0000 7fe0eec00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e80d1000 con 0x7fe0e815db90 2024-09-21T07:40:57.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.084+0000 7fe0eec00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe0e8070c70 con 0x7fe0e8160440 2024-09-21T07:40:57.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.085+0000 7fe0e7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0e80d09a0 0x7fe0e81f6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:57.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.085+0000 7fe0e7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0e80d09a0 0x7fe0e81f6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37184/0 (socket says 172.21.15.39:37184) 2024-09-21T07:40:57.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.085+0000 7fe0e7e00640 1 -- 172.21.15.39:0/3752047436 learned_addr learned my addr 172.21.15.39:0/3752047436 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:40:57.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.085+0000 7fe0df400640 1 --2- 172.21.15.39:0/3752047436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0e815db90 0x7fe0e81f2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:57.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.085+0000 7fe0ecc00640 1 --2- 172.21.15.39:0/3752047436 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe0e8160440 0x7fe0e8073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:57.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.085+0000 7fe0e7e00640 1 -- 172.21.15.39:0/3752047436 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe0e8160440 msgr2=0x7fe0e8073150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:57.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.085+0000 7fe0e7e00640 1 --2- 172.21.15.39:0/3752047436 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe0e8160440 0x7fe0e8073150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:57.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.085+0000 7fe0e7e00640 1 -- 172.21.15.39:0/3752047436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0e815db90 msgr2=0x7fe0e81f2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:57.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.085+0000 7fe0e7e00640 1 --2- 172.21.15.39:0/3752047436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0e815db90 0x7fe0e81f2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:57.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.085+0000 7fe0e7e00640 1 -- 172.21.15.39:0/3752047436 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0e81f7290 con 0x7fe0e80d09a0 2024-09-21T07:40:57.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.085+0000 7fe0df400640 1 --2- 172.21.15.39:0/3752047436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0e815db90 0x7fe0e81f2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:40:57.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.086+0000 7fe0e7e00640 1 --2- 172.21.15.39:0/3752047436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0e80d09a0 0x7fe0e81f6920 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fe0d802ab10 tx=0x7fe0d8033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:57.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.086+0000 7fe0e4c00640 1 -- 172.21.15.39:0/3752047436 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8031b50 con 0x7fe0e80d09a0 2024-09-21T07:40:57.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.086+0000 7fe0e4c00640 1 -- 172.21.15.39:0/3752047436 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe0d8031de0 con 0x7fe0e80d09a0 2024-09-21T07:40:57.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.086+0000 7fe0e4c00640 1 -- 172.21.15.39:0/3752047436 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8005280 con 0x7fe0e80d09a0 2024-09-21T07:40:57.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.086+0000 7fe0eec00640 1 -- 172.21.15.39:0/3752047436 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0e81f7570 con 0x7fe0e80d09a0 2024-09-21T07:40:57.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.086+0000 7fe0eec00640 1 -- 172.21.15.39:0/3752047436 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0e81f3660 con 0x7fe0e80d09a0 2024-09-21T07:40:57.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.087+0000 7fe0eec00640 1 -- 172.21.15.39:0/3752047436 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0e81f38c0 con 0x7fe0e80d09a0 2024-09-21T07:40:57.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.088+0000 7fe0e4c00640 1 -- 172.21.15.39:0/3752047436 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe0d8005420 con 0x7fe0e80d09a0 2024-09-21T07:40:57.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.089+0000 7fe0e4c00640 1 --2- 172.21.15.39:0/3752047436 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe0c4078b90 0x7fe0c407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:57.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.089+0000 7fe0e4c00640 1 -- 172.21.15.39:0/3752047436 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 7603+0+0 (secure 0 0 0) 0x7fe0d80bc5d0 con 0x7fe0e80d09a0 2024-09-21T07:40:57.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.090+0000 7fe0df400640 1 --2- 172.21.15.39:0/3752047436 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe0c4078b90 0x7fe0c407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:57.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.091+0000 7fe0df400640 1 --2- 172.21.15.39:0/3752047436 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe0c4078b90 0x7fe0c407b0a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8074110 tx=0x7fe0d0009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:57.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.094+0000 7fe0e4c00640 1 -- 172.21.15.39:0/3752047436 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe0d803f030 con 0x7fe0e80d09a0 2024-09-21T07:40:57.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.371+0000 7fe0eec00640 1 -- 172.21.15.39:0/3752047436 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "6:0"} v 0) -- 0x7fe0e80cda40 con 0x7fe0e80d09a0 2024-09-21T07:40:57.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.374+0000 7fe0e4c00640 1 -- 172.21.15.39:0/3752047436 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "6:0"}]=0 failed mds gid 7113 v118) ==== 97+0+0 (secure 0 0 0) 0x7fe0d8084290 con 0x7fe0e80d09a0 2024-09-21T07:40:57.375 INFO:teuthology.orchestra.run.smithi039.stderr:failed mds gid 7113 2024-09-21T07:40:57.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.377+0000 7fe0eec00640 1 -- 172.21.15.39:0/3752047436 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe0c4078b90 msgr2=0x7fe0c407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:57.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.377+0000 7fe0eec00640 1 --2- 172.21.15.39:0/3752047436 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe0c4078b90 0x7fe0c407b0a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8074110 tx=0x7fe0d0009040 comp rx=0 tx=0).stop 2024-09-21T07:40:57.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.377+0000 7fe0eec00640 1 -- 172.21.15.39:0/3752047436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0e80d09a0 msgr2=0x7fe0e81f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:57.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.377+0000 7fe0eec00640 1 --2- 172.21.15.39:0/3752047436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0e80d09a0 0x7fe0e81f6920 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fe0d802ab10 tx=0x7fe0d8033ba0 comp rx=0 tx=0).stop 2024-09-21T07:40:57.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.377+0000 7fe0eec00640 1 -- 172.21.15.39:0/3752047436 shutdown_connections 2024-09-21T07:40:57.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.377+0000 7fe0eec00640 1 --2- 172.21.15.39:0/3752047436 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe0c4078b90 0x7fe0c407b0a0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:57.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.377+0000 7fe0eec00640 1 --2- 172.21.15.39:0/3752047436 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe0e8160440 0x7fe0e8073150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:57.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.377+0000 7fe0eec00640 1 --2- 172.21.15.39:0/3752047436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0e815db90 0x7fe0e81f2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:57.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.377+0000 7fe0eec00640 1 --2- 172.21.15.39:0/3752047436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0e80d09a0 0x7fe0e81f6920 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:57.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.377+0000 7fe0eec00640 1 -- 172.21.15.39:0/3752047436 >> 172.21.15.39:0/3752047436 conn(0x7fe0e806cfc0 msgr2=0x7fe0e80cd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:57.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.377+0000 7fe0eec00640 1 -- 172.21.15.39:0/3752047436 shutdown_connections 2024-09-21T07:40:57.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.377+0000 7fe0eec00640 1 -- 172.21.15.39:0/3752047436 wait complete. 2024-09-21T07:40:57.408 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T07:40:57.408 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T07:40:57.408 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T07:40:57.408 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:40:57.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:40:57.409 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/daemon/state.py", line 146, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi039 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a' 2024-09-21T07:40:57.409 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T07:40:57.410 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T07:40:57.414 INFO:tasks.ceph.mds.a:Started 2024-09-21T07:40:57.414 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:40:57.614 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-21T07:40:57.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.645+0000 7f8199400640 1 -- 172.21.15.39:0/1643903901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81940cfb20 msgr2=0x7f81940d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:57.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.645+0000 7f8199400640 1 --2- 172.21.15.39:0/1643903901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81940cfb20 0x7f81940d1fc0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8184009820 tx=0x7f818402a540 comp rx=0 tx=0).stop 2024-09-21T07:40:57.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.645+0000 7f8199400640 1 -- 172.21.15.39:0/1643903901 shutdown_connections 2024-09-21T07:40:57.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.645+0000 7f8199400640 1 --2- 172.21.15.39:0/1643903901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81940cfb20 0x7f81940d1fc0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:57.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.645+0000 7f8199400640 1 -- 172.21.15.39:0/1643903901 >> 172.21.15.39:0/1643903901 conn(0x7f81940cf000 msgr2=0x7f81940cf410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:57.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.645+0000 7f8199400640 1 -- 172.21.15.39:0/1643903901 shutdown_connections 2024-09-21T07:40:57.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.645+0000 7f8199400640 1 -- 172.21.15.39:0/1643903901 wait complete. 2024-09-21T07:40:57.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.647+0000 7f8199400640 1 Processor -- start 2024-09-21T07:40:57.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.647+0000 7f8199400640 1 -- start start 2024-09-21T07:40:57.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.649+0000 7f8199400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f81940b53d0 0x7f819415bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:57.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.649+0000 7f8199400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81940b7560 0x7f81941516c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:57.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.649+0000 7f8199400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81940cfb20 0x7f81941f5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:57.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.649+0000 7f8199400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f819416a080 con 0x7f81940cfb20 2024-09-21T07:40:57.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.649+0000 7f8199400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8194169f00 con 0x7f81940b7560 2024-09-21T07:40:57.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.649+0000 7f8199400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f819416a200 con 0x7f81940b53d0 2024-09-21T07:40:57.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.649+0000 7f8193200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81940cfb20 0x7f81941f5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:57.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.649+0000 7f8193200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81940cfb20 0x7f81941f5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37200/0 (socket says 172.21.15.39:37200) 2024-09-21T07:40:57.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.649+0000 7f8193200640 1 -- 172.21.15.39:0/2612894441 learned_addr learned my addr 172.21.15.39:0/2612894441 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:40:57.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.649+0000 7f8191e00640 1 --2- 172.21.15.39:0/2612894441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81940b7560 0x7f81941516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:57.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.649+0000 7f8192800640 1 --2- 172.21.15.39:0/2612894441 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f81940b53d0 0x7f819415bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:57.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.650+0000 7f8193200640 1 -- 172.21.15.39:0/2612894441 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f81940b53d0 msgr2=0x7f819415bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:57.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.650+0000 7f8193200640 1 --2- 172.21.15.39:0/2612894441 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f81940b53d0 0x7f819415bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:57.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.650+0000 7f8193200640 1 -- 172.21.15.39:0/2612894441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81940b7560 msgr2=0x7f81941516c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:57.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.650+0000 7f8193200640 1 --2- 172.21.15.39:0/2612894441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81940b7560 0x7f81941516c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:57.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.650+0000 7f8193200640 1 -- 172.21.15.39:0/2612894441 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81941f60e0 con 0x7f81940cfb20 2024-09-21T07:40:57.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.650+0000 7f8193200640 1 --2- 172.21.15.39:0/2612894441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81940cfb20 0x7f81941f5ba0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f818c00cbc0 tx=0x7f818c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:57.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.650+0000 7f818aa00640 1 -- 172.21.15.39:0/2612894441 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818c016020 con 0x7f81940cfb20 2024-09-21T07:40:57.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.650+0000 7f818aa00640 1 -- 172.21.15.39:0/2612894441 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f818c00ae10 con 0x7f81940cfb20 2024-09-21T07:40:57.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.650+0000 7f818aa00640 1 -- 172.21.15.39:0/2612894441 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818c012680 con 0x7f81940cfb20 2024-09-21T07:40:57.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.650+0000 7f8199400640 1 -- 172.21.15.39:0/2612894441 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81941f65d0 con 0x7f81940cfb20 2024-09-21T07:40:57.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.650+0000 7f8199400640 1 -- 172.21.15.39:0/2612894441 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81941589e0 con 0x7f81940cfb20 2024-09-21T07:40:57.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.651+0000 7f8199400640 1 -- 172.21.15.39:0/2612894441 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81941f6340 con 0x7f81940cfb20 2024-09-21T07:40:57.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.653+0000 7f818aa00640 1 -- 172.21.15.39:0/2612894441 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f818c019020 con 0x7f81940cfb20 2024-09-21T07:40:57.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.654+0000 7f818aa00640 1 --2- 172.21.15.39:0/2612894441 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8174078b90 0x7f817407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:40:57.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.654+0000 7f818aa00640 1 -- 172.21.15.39:0/2612894441 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7689+0+0 (secure 0 0 0) 0x7f818c09b850 con 0x7f81940cfb20 2024-09-21T07:40:57.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.654+0000 7f8192800640 1 --2- 172.21.15.39:0/2612894441 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8174078b90 0x7f817407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:40:57.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.655+0000 7f8192800640 1 --2- 172.21.15.39:0/2612894441 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8174078b90 0x7f817407b0a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8184034000 tx=0x7f8184004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:40:57.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.658+0000 7f818aa00640 1 -- 172.21.15.39:0/2612894441 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f818c020030 con 0x7f81940cfb20 2024-09-21T07:40:57.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.894+0000 7f8199400640 1 -- 172.21.15.39:0/2612894441 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81940d1fc0 con 0x7f81940cfb20 2024-09-21T07:40:57.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.895+0000 7f818aa00640 1 -- 172.21.15.39:0/2612894441 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) ==== 95+0+4426 (secure 0 0 0) 0x7f818c063550 con 0x7f81940cfb20 2024-09-21T07:40:57.896 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:40:57.896 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":119,"btime":"2024-09-21T07:40:57:383355+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:40:57.383335+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":107,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":6269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6269":{"gid":6269,"name":"d","rank":1,"incarnation":110,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6837/2336112936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2336112936},{"type":"v1","addr":"172.21.15.148:6837","nonce":2336112936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:replay","state_seq":1,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6269,"qdb_cluster":[6269]},"id":6}]} 2024-09-21T07:40:57.896 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 119 2024-09-21T07:40:57.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.897+0000 7f8199400640 1 -- 172.21.15.39:0/2612894441 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8174078b90 msgr2=0x7f817407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:57.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.897+0000 7f8199400640 1 --2- 172.21.15.39:0/2612894441 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8174078b90 0x7f817407b0a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8184034000 tx=0x7f8184004730 comp rx=0 tx=0).stop 2024-09-21T07:40:57.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.897+0000 7f8199400640 1 -- 172.21.15.39:0/2612894441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81940cfb20 msgr2=0x7f81941f5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:40:57.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.897+0000 7f8199400640 1 --2- 172.21.15.39:0/2612894441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81940cfb20 0x7f81941f5ba0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f818c00cbc0 tx=0x7f818c00a430 comp rx=0 tx=0).stop 2024-09-21T07:40:57.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.897+0000 7f8199400640 1 -- 172.21.15.39:0/2612894441 shutdown_connections 2024-09-21T07:40:57.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.897+0000 7f8199400640 1 --2- 172.21.15.39:0/2612894441 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8174078b90 0x7f817407b0a0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:57.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.897+0000 7f8199400640 1 --2- 172.21.15.39:0/2612894441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81940cfb20 0x7f81941f5ba0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:57.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.897+0000 7f8199400640 1 --2- 172.21.15.39:0/2612894441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81940b7560 0x7f81941516c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:57.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.897+0000 7f8199400640 1 --2- 172.21.15.39:0/2612894441 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f81940b53d0 0x7f819415bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:40:57.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.897+0000 7f8199400640 1 -- 172.21.15.39:0/2612894441 >> 172.21.15.39:0/2612894441 conn(0x7f81940cf000 msgr2=0x7f819406e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:40:57.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.898+0000 7f8199400640 1 -- 172.21.15.39:0/2612894441 shutdown_connections 2024-09-21T07:40:57.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:40:57.898+0000 7f8199400640 1 -- 172.21.15.39:0/2612894441 wait complete. 2024-09-21T07:40:57.923 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-09-21T07:41:01.348 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:41:01.352 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:41:02.674 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-09-21T07:41:02.674 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-09-21T07:41:02.924 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:41:03.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.159+0000 7fd4d1800640 1 -- 172.21.15.39:0/3719147846 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4cc06f800 msgr2=0x7fd4cc0cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:03.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.159+0000 7fd4d1800640 1 --2- 172.21.15.39:0/3719147846 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4cc06f800 0x7fd4cc0cd8e0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc0d1240 tx=0x7fd4b802a560 comp rx=0 tx=0).stop 2024-09-21T07:41:03.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.160+0000 7fd4d1800640 1 -- 172.21.15.39:0/3719147846 shutdown_connections 2024-09-21T07:41:03.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.160+0000 7fd4d1800640 1 --2- 172.21.15.39:0/3719147846 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4cc06f800 0x7fd4cc0cd8e0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:03.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.160+0000 7fd4d1800640 1 --2- 172.21.15.39:0/3719147846 >> v2:172.21.15.148:3300/0 conn(0x7fd4cc15cd30 0x7fd4cc0cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:03.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.160+0000 7fd4d1800640 1 -- 172.21.15.39:0/3719147846 >> 172.21.15.39:0/3719147846 conn(0x7fd4cc06cfc0 msgr2=0x7fd4cc06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:03.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.160+0000 7fd4d1800640 1 -- 172.21.15.39:0/3719147846 shutdown_connections 2024-09-21T07:41:03.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.160+0000 7fd4d1800640 1 -- 172.21.15.39:0/3719147846 wait complete. 2024-09-21T07:41:03.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.162+0000 7fd4d1800640 1 Processor -- start 2024-09-21T07:41:03.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.162+0000 7fd4d1800640 1 -- start start 2024-09-21T07:41:03.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.163+0000 7fd4d1800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd4cc06f800 0x7fd4cc157320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:03.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.163+0000 7fd4d1800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4cc06fc10 0x7fd4cc149d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:03.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.163+0000 7fd4d1800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4cc15cd30 0x7fd4cc0b44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:03.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.163+0000 7fd4d1800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc073890 con 0x7fd4cc06fc10 2024-09-21T07:41:03.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.163+0000 7fd4d1800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc073710 con 0x7fd4cc15cd30 2024-09-21T07:41:03.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.163+0000 7fd4d1800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd4cc073a10 con 0x7fd4cc06f800 2024-09-21T07:41:03.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.164+0000 7fd4ca000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4cc06fc10 0x7fd4cc149d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:03.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.164+0000 7fd4ca000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4cc06fc10 0x7fd4cc149d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56034/0 (socket says 172.21.15.39:56034) 2024-09-21T07:41:03.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.164+0000 7fd4ca000640 1 -- 172.21.15.39:0/4256524017 learned_addr learned my addr 172.21.15.39:0/4256524017 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:41:03.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.164+0000 7fd4caa00640 1 --2- 172.21.15.39:0/4256524017 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd4cc06f800 0x7fd4cc157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:03.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.164+0000 7fd4cb400640 1 --2- 172.21.15.39:0/4256524017 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4cc15cd30 0x7fd4cc0b44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:03.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.164+0000 7fd4ca000640 1 -- 172.21.15.39:0/4256524017 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd4cc06f800 msgr2=0x7fd4cc157320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:03.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.164+0000 7fd4ca000640 1 --2- 172.21.15.39:0/4256524017 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd4cc06f800 0x7fd4cc157320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:03.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.164+0000 7fd4ca000640 1 -- 172.21.15.39:0/4256524017 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4cc15cd30 msgr2=0x7fd4cc0b44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:03.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.164+0000 7fd4ca000640 1 --2- 172.21.15.39:0/4256524017 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4cc15cd30 0x7fd4cc0b44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:03.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.164+0000 7fd4ca000640 1 -- 172.21.15.39:0/4256524017 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4cc15a1d0 con 0x7fd4cc06fc10 2024-09-21T07:41:03.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.164+0000 7fd4ca000640 1 --2- 172.21.15.39:0/4256524017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4cc06fc10 0x7fd4cc149d50 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8034040 tx=0x7fd4b80039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:03.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.165+0000 7fd4c2a00640 1 -- 172.21.15.39:0/4256524017 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b803e030 con 0x7fd4cc06fc10 2024-09-21T07:41:03.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.165+0000 7fd4c2a00640 1 -- 172.21.15.39:0/4256524017 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4b80047a0 con 0x7fd4cc06fc10 2024-09-21T07:41:03.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.165+0000 7fd4c2a00640 1 -- 172.21.15.39:0/4256524017 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8035040 con 0x7fd4cc06fc10 2024-09-21T07:41:03.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.165+0000 7fd4d1800640 1 -- 172.21.15.39:0/4256524017 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4cc1f5b30 con 0x7fd4cc06fc10 2024-09-21T07:41:03.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.165+0000 7fd4d1800640 1 -- 172.21.15.39:0/4256524017 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4cc1f60f0 con 0x7fd4cc06fc10 2024-09-21T07:41:03.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.166+0000 7fd4d1800640 1 -- 172.21.15.39:0/4256524017 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4cc073ec0 con 0x7fd4cc06fc10 2024-09-21T07:41:03.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.167+0000 7fd4c2a00640 1 -- 172.21.15.39:0/4256524017 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd4b8041030 con 0x7fd4cc06fc10 2024-09-21T07:41:03.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.169+0000 7fd4c2a00640 1 --2- 172.21.15.39:0/4256524017 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd4ac078b90 0x7fd4ac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:03.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.169+0000 7fd4c2a00640 1 -- 172.21.15.39:0/4256524017 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7689+0+0 (secure 0 0 0) 0x7fd4b80bbcd0 con 0x7fd4cc06fc10 2024-09-21T07:41:03.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.170+0000 7fd4caa00640 1 --2- 172.21.15.39:0/4256524017 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd4ac078b90 0x7fd4ac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:03.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.170+0000 7fd4caa00640 1 --2- 172.21.15.39:0/4256524017 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd4ac078b90 0x7fd4ac07b0a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc00bb50 tx=0x7fd4bc00b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:03.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.174+0000 7fd4c2a00640 1 -- 172.21.15.39:0/4256524017 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd4b8083260 con 0x7fd4cc06fc10 2024-09-21T07:41:03.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.422+0000 7fd4d1800640 1 -- 172.21.15.39:0/4256524017 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4cc15c930 con 0x7fd4cc06fc10 2024-09-21T07:41:03.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.423+0000 7fd4c2a00640 1 -- 172.21.15.39:0/4256524017 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) ==== 95+0+5254 (secure 0 0 0) 0x7fd4b8087c90 con 0x7fd4cc06fc10 2024-09-21T07:41:03.425 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:41:03.425 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":120,"btime":"2024-09-21T07:40:58:384903+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7452,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3695489158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3695489158},{"type":"v1","addr":"172.21.15.39:6835","nonce":3695489158}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":120}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:40:57.400480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":107,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":6269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6269":{"gid":6269,"name":"d","rank":1,"incarnation":110,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6837/2336112936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2336112936},{"type":"v1","addr":"172.21.15.148:6837","nonce":2336112936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:resolve","state_seq":14,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6269,"qdb_cluster":[6269]},"id":6}]} 2024-09-21T07:41:03.425 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 120 2024-09-21T07:41:03.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.425+0000 7fd4d1800640 1 -- 172.21.15.39:0/4256524017 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd4ac078b90 msgr2=0x7fd4ac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:03.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.425+0000 7fd4d1800640 1 --2- 172.21.15.39:0/4256524017 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd4ac078b90 0x7fd4ac07b0a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc00bb50 tx=0x7fd4bc00b2e0 comp rx=0 tx=0).stop 2024-09-21T07:41:03.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.425+0000 7fd4d1800640 1 -- 172.21.15.39:0/4256524017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4cc06fc10 msgr2=0x7fd4cc149d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:03.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.425+0000 7fd4d1800640 1 --2- 172.21.15.39:0/4256524017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4cc06fc10 0x7fd4cc149d50 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8034040 tx=0x7fd4b80039e0 comp rx=0 tx=0).stop 2024-09-21T07:41:03.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.425+0000 7fd4d1800640 1 -- 172.21.15.39:0/4256524017 shutdown_connections 2024-09-21T07:41:03.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.425+0000 7fd4d1800640 1 --2- 172.21.15.39:0/4256524017 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd4ac078b90 0x7fd4ac07b0a0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:03.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.425+0000 7fd4d1800640 1 --2- 172.21.15.39:0/4256524017 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4cc15cd30 0x7fd4cc0b44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:03.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.425+0000 7fd4d1800640 1 --2- 172.21.15.39:0/4256524017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4cc06fc10 0x7fd4cc149d50 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:03.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.425+0000 7fd4d1800640 1 --2- 172.21.15.39:0/4256524017 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd4cc06f800 0x7fd4cc157320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:03.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.425+0000 7fd4d1800640 1 -- 172.21.15.39:0/4256524017 >> 172.21.15.39:0/4256524017 conn(0x7fd4cc06cfc0 msgr2=0x7fd4cc070f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:03.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.426+0000 7fd4d1800640 1 -- 172.21.15.39:0/4256524017 shutdown_connections 2024-09-21T07:41:03.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.426+0000 7fd4d1800640 1 -- 172.21.15.39:0/4256524017 wait complete. 2024-09-21T07:41:03.453 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-21T07:41:03.453 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:41:03.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.679+0000 7f16cfe00640 1 -- 172.21.15.39:0/3427763842 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f16d00d4d10 msgr2=0x7f16d00b4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:03.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.679+0000 7f16cfe00640 1 --2- 172.21.15.39:0/3427763842 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f16d00d4d10 0x7f16d00b4880 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f16b8005fd0 tx=0x7f16b802a5d0 comp rx=0 tx=0).stop 2024-09-21T07:41:03.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.679+0000 7f16cfe00640 1 -- 172.21.15.39:0/3427763842 shutdown_connections 2024-09-21T07:41:03.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.679+0000 7f16cfe00640 1 --2- 172.21.15.39:0/3427763842 >> v2:172.21.15.148:3300/0 conn(0x7f16d00b2ca0 0x7f16d00b4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:03.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.679+0000 7f16cfe00640 1 --2- 172.21.15.39:0/3427763842 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f16d00d4d10 0x7f16d00b4880 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:03.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.679+0000 7f16cfe00640 1 -- 172.21.15.39:0/3427763842 >> 172.21.15.39:0/3427763842 conn(0x7f16d008bfb0 msgr2=0x7f16d009d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:03.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.679+0000 7f16cfe00640 1 -- 172.21.15.39:0/3427763842 shutdown_connections 2024-09-21T07:41:03.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.679+0000 7f16cfe00640 1 -- 172.21.15.39:0/3427763842 wait complete. 2024-09-21T07:41:03.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.681+0000 7f16cfe00640 1 Processor -- start 2024-09-21T07:41:03.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.682+0000 7f16cfe00640 1 -- start start 2024-09-21T07:41:03.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.683+0000 7f16cfe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16d00b2ca0 0x7f16d01eabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:03.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.683+0000 7f16cfe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16d00d19f0 0x7f16d00cd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:03.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.683+0000 7f16cfe00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f16d00a1520 0x7f16d00a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:03.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.683+0000 7f16cfe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f16d00b5480 con 0x7f16d00d19f0 2024-09-21T07:41:03.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.683+0000 7f16cfe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f16d00b5300 con 0x7f16d00b2ca0 2024-09-21T07:41:03.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.683+0000 7f16cfe00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f16d00b5600 con 0x7f16d00a1520 2024-09-21T07:41:03.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.683+0000 7f16c7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16d00d19f0 0x7f16d00cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:03.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.683+0000 7f16c7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16d00d19f0 0x7f16d00cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56050/0 (socket says 172.21.15.39:56050) 2024-09-21T07:41:03.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.683+0000 7f16c7e00640 1 -- 172.21.15.39:0/592686021 learned_addr learned my addr 172.21.15.39:0/592686021 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:41:03.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.684+0000 7f16cea00640 1 --2- 172.21.15.39:0/592686021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16d00b2ca0 0x7f16d01eabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:03.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.684+0000 7f16cf400640 1 --2- 172.21.15.39:0/592686021 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f16d00a1520 0x7f16d00a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:03.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.684+0000 7f16c7e00640 1 -- 172.21.15.39:0/592686021 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f16d00a1520 msgr2=0x7f16d00a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:03.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.684+0000 7f16c7e00640 1 --2- 172.21.15.39:0/592686021 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f16d00a1520 0x7f16d00a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:03.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.684+0000 7f16c7e00640 1 -- 172.21.15.39:0/592686021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16d00b2ca0 msgr2=0x7f16d01eabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:03.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.684+0000 7f16c7e00640 1 --2- 172.21.15.39:0/592686021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16d00b2ca0 0x7f16d01eabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:03.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.684+0000 7f16c7e00640 1 -- 172.21.15.39:0/592686021 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d00a1e70 con 0x7f16d00d19f0 2024-09-21T07:41:03.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.684+0000 7f16cf400640 1 --2- 172.21.15.39:0/592686021 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f16d00a1520 0x7f16d00a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:41:03.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.684+0000 7f16cea00640 1 --2- 172.21.15.39:0/592686021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16d00b2ca0 0x7f16d01eabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:41:03.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.684+0000 7f16c7e00640 1 --2- 172.21.15.39:0/592686021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16d00d19f0 0x7f16d00cd700 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f16b8006010 tx=0x7f16b80035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:03.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.684+0000 7f16c6a00640 1 -- 172.21.15.39:0/592686021 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b803e030 con 0x7f16d00d19f0 2024-09-21T07:41:03.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.685+0000 7f16c6a00640 1 -- 172.21.15.39:0/592686021 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f16b8003ae0 con 0x7f16d00d19f0 2024-09-21T07:41:03.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.685+0000 7f16c6a00640 1 -- 172.21.15.39:0/592686021 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b8035040 con 0x7f16d00d19f0 2024-09-21T07:41:03.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.685+0000 7f16cfe00640 1 -- 172.21.15.39:0/592686021 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16d01f6450 con 0x7f16d00d19f0 2024-09-21T07:41:03.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.685+0000 7f16cfe00640 1 -- 172.21.15.39:0/592686021 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16d01f5e90 con 0x7f16d00d19f0 2024-09-21T07:41:03.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.686+0000 7f16cfe00640 1 -- 172.21.15.39:0/592686021 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16d01f5920 con 0x7f16d00d19f0 2024-09-21T07:41:03.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.687+0000 7f16c6a00640 1 -- 172.21.15.39:0/592686021 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f16b8041030 con 0x7f16d00d19f0 2024-09-21T07:41:03.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.688+0000 7f16c6a00640 1 --2- 172.21.15.39:0/592686021 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f16ac078b90 0x7f16ac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:03.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.688+0000 7f16c6a00640 1 -- 172.21.15.39:0/592686021 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7689+0+0 (secure 0 0 0) 0x7f16b80bbb10 con 0x7f16d00d19f0 2024-09-21T07:41:03.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.688+0000 7f16cea00640 1 --2- 172.21.15.39:0/592686021 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f16ac078b90 0x7f16ac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:03.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.689+0000 7f16cea00640 1 --2- 172.21.15.39:0/592686021 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f16ac078b90 0x7f16ac07b0a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f16c0001750 tx=0x7f16c0009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:03.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.692+0000 7f16c6a00640 1 -- 172.21.15.39:0/592686021 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f16d00a1e70 con 0x7f16d00d19f0 2024-09-21T07:41:03.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.929+0000 7f16cfe00640 1 -- 172.21.15.39:0/592686021 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16d0069a50 con 0x7f16d00d19f0 2024-09-21T07:41:03.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.929+0000 7f16c6a00640 1 -- 172.21.15.39:0/592686021 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) ==== 95+0+5254 (secure 0 0 0) 0x7f16b8083780 con 0x7f16d00d19f0 2024-09-21T07:41:03.932 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:41:03.932 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":120,"btime":"2024-09-21T07:40:58:384903+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7452,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3695489158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3695489158},{"type":"v1","addr":"172.21.15.39:6835","nonce":3695489158}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":120}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:40:57.400480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":107,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":6269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6269":{"gid":6269,"name":"d","rank":1,"incarnation":110,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6837/2336112936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2336112936},{"type":"v1","addr":"172.21.15.148:6837","nonce":2336112936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:resolve","state_seq":14,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6269,"qdb_cluster":[6269]},"id":6}]} 2024-09-21T07:41:03.932 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 120 2024-09-21T07:41:03.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.932+0000 7f16cfe00640 1 -- 172.21.15.39:0/592686021 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f16ac078b90 msgr2=0x7f16ac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:03.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.932+0000 7f16cfe00640 1 --2- 172.21.15.39:0/592686021 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f16ac078b90 0x7f16ac07b0a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f16c0001750 tx=0x7f16c0009040 comp rx=0 tx=0).stop 2024-09-21T07:41:03.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.932+0000 7f16cfe00640 1 -- 172.21.15.39:0/592686021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16d00d19f0 msgr2=0x7f16d00cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:03.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.932+0000 7f16cfe00640 1 --2- 172.21.15.39:0/592686021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16d00d19f0 0x7f16d00cd700 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f16b8006010 tx=0x7f16b80035f0 comp rx=0 tx=0).stop 2024-09-21T07:41:03.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.932+0000 7f16cfe00640 1 -- 172.21.15.39:0/592686021 shutdown_connections 2024-09-21T07:41:03.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.932+0000 7f16cfe00640 1 --2- 172.21.15.39:0/592686021 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f16ac078b90 0x7f16ac07b0a0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:03.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.932+0000 7f16cfe00640 1 --2- 172.21.15.39:0/592686021 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f16d00a1520 0x7f16d00a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:03.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.932+0000 7f16cfe00640 1 --2- 172.21.15.39:0/592686021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16d00d19f0 0x7f16d00cd700 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:03.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.932+0000 7f16cfe00640 1 --2- 172.21.15.39:0/592686021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16d00b2ca0 0x7f16d01eabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:03.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.932+0000 7f16cfe00640 1 -- 172.21.15.39:0/592686021 >> 172.21.15.39:0/592686021 conn(0x7f16d008bfb0 msgr2=0x7f16d00b94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:03.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.932+0000 7f16cfe00640 1 -- 172.21.15.39:0/592686021 shutdown_connections 2024-09-21T07:41:03.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:03.932+0000 7f16cfe00640 1 -- 172.21.15.39:0/592686021 wait complete. 2024-09-21T07:41:03.958 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:resolve 2024-09-21T07:41:03.958 DEBUG:tasks.cephfs.filesystem:reached state 'up:resolve' in 0.5052449703216553s 2024-09-21T07:41:03.959 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:41:04.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.185+0000 7f1667600640 1 -- 172.21.15.39:0/2813537972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1648003a50 msgr2=0x7f1648003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:04.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.185+0000 7f1667600640 1 --2- 172.21.15.39:0/2813537972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1648003a50 0x7f1648003ec0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f1654006fa0 tx=0x7f1648005830 comp rx=0 tx=0).stop 2024-09-21T07:41:04.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.185+0000 7f1667600640 1 -- 172.21.15.39:0/2813537972 shutdown_connections 2024-09-21T07:41:04.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.185+0000 7f1667600640 1 --2- 172.21.15.39:0/2813537972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1648003a50 0x7f1648003ec0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:04.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.185+0000 7f1667600640 1 -- 172.21.15.39:0/2813537972 >> 172.21.15.39:0/2813537972 conn(0x7f166006cfc0 msgr2=0x7f166006d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:04.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.185+0000 7f1667600640 1 -- 172.21.15.39:0/2813537972 shutdown_connections 2024-09-21T07:41:04.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.185+0000 7f1667600640 1 -- 172.21.15.39:0/2813537972 wait complete. 2024-09-21T07:41:04.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.187+0000 7f1667600640 1 Processor -- start 2024-09-21T07:41:04.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.188+0000 7f1667600640 1 -- start start 2024-09-21T07:41:04.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.189+0000 7f1667600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1648003a50 0x7f16601f44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:04.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.189+0000 7f1667600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f166014d7a0 0x7f16601f4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:04.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.189+0000 7f1667600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16601f4f50 0x7f16600b15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:04.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.189+0000 7f1667600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f16600d0b70 con 0x7f166014d7a0 2024-09-21T07:41:04.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.189+0000 7f1667600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1660156140 con 0x7f16601f4f50 2024-09-21T07:41:04.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.189+0000 7f1667600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f16600d0cf0 con 0x7f1648003a50 2024-09-21T07:41:04.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.189+0000 7f165fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f166014d7a0 0x7f16601f4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:04.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.189+0000 7f165fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f166014d7a0 0x7f16601f4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56074/0 (socket says 172.21.15.39:56074) 2024-09-21T07:41:04.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.189+0000 7f165fe00640 1 -- 172.21.15.39:0/3284279524 learned_addr learned my addr 172.21.15.39:0/3284279524 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:41:04.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.189+0000 7f1664c00640 1 --2- 172.21.15.39:0/3284279524 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1648003a50 0x7f16601f44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:04.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.189+0000 7f1665600640 1 --2- 172.21.15.39:0/3284279524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16601f4f50 0x7f16600b15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:04.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.190+0000 7f165fe00640 1 -- 172.21.15.39:0/3284279524 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1648003a50 msgr2=0x7f16601f44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:04.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.190+0000 7f165fe00640 1 --2- 172.21.15.39:0/3284279524 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1648003a50 0x7f16601f44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:04.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.190+0000 7f165fe00640 1 -- 172.21.15.39:0/3284279524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16601f4f50 msgr2=0x7f16600b15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:04.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.190+0000 7f165fe00640 1 --2- 172.21.15.39:0/3284279524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16601f4f50 0x7f16600b15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:04.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.190+0000 7f165fe00640 1 -- 172.21.15.39:0/3284279524 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1660150a30 con 0x7f166014d7a0 2024-09-21T07:41:04.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.190+0000 7f1665600640 1 --2- 172.21.15.39:0/3284279524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16601f4f50 0x7f16600b15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:41:04.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.190+0000 7f165fe00640 1 --2- 172.21.15.39:0/3284279524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f166014d7a0 0x7f16601f4a10 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f164c011b10 tx=0x7f164c009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:04.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.190+0000 7f165cc00640 1 -- 172.21.15.39:0/3284279524 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164c00fce0 con 0x7f166014d7a0 2024-09-21T07:41:04.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.190+0000 7f165cc00640 1 -- 172.21.15.39:0/3284279524 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f164c00fe80 con 0x7f166014d7a0 2024-09-21T07:41:04.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.190+0000 7f165cc00640 1 -- 172.21.15.39:0/3284279524 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164c004280 con 0x7f166014d7a0 2024-09-21T07:41:04.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.190+0000 7f1667600640 1 -- 172.21.15.39:0/3284279524 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1660150190 con 0x7f166014d7a0 2024-09-21T07:41:04.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.190+0000 7f1667600640 1 -- 172.21.15.39:0/3284279524 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1660150750 con 0x7f166014d7a0 2024-09-21T07:41:04.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.192+0000 7f1667600640 1 -- 172.21.15.39:0/3284279524 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16600d1250 con 0x7f166014d7a0 2024-09-21T07:41:04.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.193+0000 7f165cc00640 1 -- 172.21.15.39:0/3284279524 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f164c0035f0 con 0x7f166014d7a0 2024-09-21T07:41:04.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.194+0000 7f165cc00640 1 --2- 172.21.15.39:0/3284279524 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1640078b90 0x7f164007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:04.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.195+0000 7f165cc00640 1 -- 172.21.15.39:0/3284279524 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7689+0+0 (secure 0 0 0) 0x7f164c09bf20 con 0x7f166014d7a0 2024-09-21T07:41:04.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.195+0000 7f1664c00640 1 --2- 172.21.15.39:0/3284279524 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1640078b90 0x7f164007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:04.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.196+0000 7f1664c00640 1 --2- 172.21.15.39:0/3284279524 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1640078b90 0x7f164007b0a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1654033000 tx=0x7f166015c9e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:04.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.199+0000 7f165cc00640 1 -- 172.21.15.39:0/3284279524 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f164c063540 con 0x7f166014d7a0 2024-09-21T07:41:04.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.438+0000 7f1667600640 1 -- 172.21.15.39:0/3284279524 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f166015da70 con 0x7f166014d7a0 2024-09-21T07:41:04.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.439+0000 7f165cc00640 1 -- 172.21.15.39:0/3284279524 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) ==== 95+0+5254 (secure 0 0 0) 0x7f164c067f70 con 0x7f166014d7a0 2024-09-21T07:41:04.441 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:41:04.441 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":120,"btime":"2024-09-21T07:40:58:384903+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7452,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3695489158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3695489158},{"type":"v1","addr":"172.21.15.39:6835","nonce":3695489158}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":120}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:40:57.400480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":107,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":6269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6269":{"gid":6269,"name":"d","rank":1,"incarnation":110,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6837/2336112936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2336112936},{"type":"v1","addr":"172.21.15.148:6837","nonce":2336112936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:resolve","state_seq":14,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6269,"qdb_cluster":[6269]},"id":6}]} 2024-09-21T07:41:04.441 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 120 2024-09-21T07:41:04.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.441+0000 7f1667600640 1 -- 172.21.15.39:0/3284279524 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1640078b90 msgr2=0x7f164007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:04.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.441+0000 7f1667600640 1 --2- 172.21.15.39:0/3284279524 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1640078b90 0x7f164007b0a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1654033000 tx=0x7f166015c9e0 comp rx=0 tx=0).stop 2024-09-21T07:41:04.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.441+0000 7f1667600640 1 -- 172.21.15.39:0/3284279524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f166014d7a0 msgr2=0x7f16601f4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:04.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.441+0000 7f1667600640 1 --2- 172.21.15.39:0/3284279524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f166014d7a0 0x7f16601f4a10 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f164c011b10 tx=0x7f164c009f10 comp rx=0 tx=0).stop 2024-09-21T07:41:04.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.441+0000 7f1667600640 1 -- 172.21.15.39:0/3284279524 shutdown_connections 2024-09-21T07:41:04.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.441+0000 7f1667600640 1 --2- 172.21.15.39:0/3284279524 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1640078b90 0x7f164007b0a0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:04.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.441+0000 7f1667600640 1 --2- 172.21.15.39:0/3284279524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16601f4f50 0x7f16600b15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:04.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.441+0000 7f1667600640 1 --2- 172.21.15.39:0/3284279524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f166014d7a0 0x7f16601f4a10 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:04.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.441+0000 7f1667600640 1 --2- 172.21.15.39:0/3284279524 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1648003a50 0x7f16601f44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:04.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.441+0000 7f1667600640 1 -- 172.21.15.39:0/3284279524 >> 172.21.15.39:0/3284279524 conn(0x7f166006cfc0 msgr2=0x7f166015e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:04.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.441+0000 7f1667600640 1 -- 172.21.15.39:0/3284279524 shutdown_connections 2024-09-21T07:41:04.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.441+0000 7f1667600640 1 -- 172.21.15.39:0/3284279524 wait complete. 2024-09-21T07:41:04.466 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2024-09-21T07:41:04.713 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_snap":12,"last_created":12,"last_destroyed":9,"pending_noop":[],"snaps":[{"snapid":11,"ino":1099511627777,"stamp":"2024-09-21T07:39:42.151005+0000","name":"s25","metadata":{}},{"snapid":12,"ino":1099511627777,"stamp":"2024-09-21T07:40:33.249709+0000","name":"s26","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2024-09-21T07:41:04.713 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 12, "last_destroyed": 9, "last_snap": 12, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s25", "snapid": 11, "stamp": "2024-09-21T07:39:42.151005+0000" }, { "ino": 1099511627777, "metadata": {}, "name": "s26", "snapid": 12, "stamp": "2024-09-21T07:40:33.249709+0000" } ] } 2024-09-21T07:41:04.713 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:41:04.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.938+0000 7f276f400640 1 -- 172.21.15.39:0/1442858668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27680cfd10 msgr2=0x7f27680cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:04.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.938+0000 7f276f400640 1 --2- 172.21.15.39:0/1442858668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27680cfd10 0x7f27680cd8e0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f2754005fd0 tx=0x7f275402a540 comp rx=0 tx=0).stop 2024-09-21T07:41:04.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.938+0000 7f276f400640 1 -- 172.21.15.39:0/1442858668 shutdown_connections 2024-09-21T07:41:04.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.938+0000 7f276f400640 1 --2- 172.21.15.39:0/1442858668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27680cfd10 0x7f27680cd8e0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:04.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.938+0000 7f276f400640 1 -- 172.21.15.39:0/1442858668 >> 172.21.15.39:0/1442858668 conn(0x7f276806cfc0 msgr2=0x7f276806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:04.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.938+0000 7f276f400640 1 -- 172.21.15.39:0/1442858668 shutdown_connections 2024-09-21T07:41:04.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.938+0000 7f276f400640 1 -- 172.21.15.39:0/1442858668 wait complete. 2024-09-21T07:41:04.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.940+0000 7f276f400640 1 Processor -- start 2024-09-21T07:41:04.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.940+0000 7f276f400640 1 -- start start 2024-09-21T07:41:04.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.942+0000 7f276f400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27680cfd10 0x7f27681454d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:04.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.942+0000 7f276f400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27680d0620 0x7f2768146710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:04.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.942+0000 7f276f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f276815cd30 0x7f2768146c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:04.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.942+0000 7f276f400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2768073aa0 con 0x7f276815cd30 2024-09-21T07:41:04.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.942+0000 7f276f400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2768073920 con 0x7f27680cfd10 2024-09-21T07:41:04.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.942+0000 7f276f400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f2768073c20 con 0x7f27680d0620 2024-09-21T07:41:04.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.942+0000 7f276d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f276815cd30 0x7f2768146c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:04.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.942+0000 7f276d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f276815cd30 0x7f2768146c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56090/0 (socket says 172.21.15.39:56090) 2024-09-21T07:41:04.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.942+0000 7f276d400640 1 -- 172.21.15.39:0/1571221685 learned_addr learned my addr 172.21.15.39:0/1571221685 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:41:04.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.942+0000 7f276ca00640 1 --2- 172.21.15.39:0/1571221685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27680cfd10 0x7f27681454d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:04.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.942+0000 7f2767e00640 1 --2- 172.21.15.39:0/1571221685 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27680d0620 0x7f2768146710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:04.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.943+0000 7f276d400640 1 -- 172.21.15.39:0/1571221685 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27680d0620 msgr2=0x7f2768146710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:04.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.943+0000 7f276d400640 1 --2- 172.21.15.39:0/1571221685 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27680d0620 0x7f2768146710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:04.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.943+0000 7f276d400640 1 -- 172.21.15.39:0/1571221685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27680cfd10 msgr2=0x7f27681454d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:04.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.943+0000 7f276d400640 1 --2- 172.21.15.39:0/1571221685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27680cfd10 0x7f27681454d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:04.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.943+0000 7f276d400640 1 -- 172.21.15.39:0/1571221685 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2768155420 con 0x7f276815cd30 2024-09-21T07:41:04.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.943+0000 7f276d400640 1 --2- 172.21.15.39:0/1571221685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f276815cd30 0x7f2768146c50 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f276000b9e0 tx=0x7f276000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:04.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.943+0000 7f2764c00640 1 -- 172.21.15.39:0/1571221685 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2760003b70 con 0x7f276815cd30 2024-09-21T07:41:04.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.943+0000 7f2764c00640 1 -- 172.21.15.39:0/1571221685 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2760003e00 con 0x7f276815cd30 2024-09-21T07:41:04.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.943+0000 7f2764c00640 1 -- 172.21.15.39:0/1571221685 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276000cb80 con 0x7f276815cd30 2024-09-21T07:41:04.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.943+0000 7f276f400640 1 -- 172.21.15.39:0/1571221685 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2768155700 con 0x7f276815cd30 2024-09-21T07:41:04.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.944+0000 7f276f400640 1 -- 172.21.15.39:0/1571221685 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2768145a10 con 0x7f276815cd30 2024-09-21T07:41:04.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.945+0000 7f276f400640 1 -- 172.21.15.39:0/1571221685 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2768155940 con 0x7f276815cd30 2024-09-21T07:41:04.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.946+0000 7f2764c00640 1 -- 172.21.15.39:0/1571221685 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2760004000 con 0x7f276815cd30 2024-09-21T07:41:04.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.947+0000 7f2764c00640 1 --2- 172.21.15.39:0/1571221685 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2748078b90 0x7f274807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:04.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.947+0000 7f2764c00640 1 -- 172.21.15.39:0/1571221685 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7689+0+0 (secure 0 0 0) 0x7f276009b400 con 0x7f276815cd30 2024-09-21T07:41:04.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.947+0000 7f276ca00640 1 --2- 172.21.15.39:0/1571221685 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2748078b90 0x7f274807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:04.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.948+0000 7f276ca00640 1 --2- 172.21.15.39:0/1571221685 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2748078b90 0x7f274807b0a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f275c001720 tx=0x7f275c009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:04.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:04.952+0000 7f2764c00640 1 -- 172.21.15.39:0/1571221685 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2768155940 con 0x7f276815cd30 2024-09-21T07:41:05.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.192+0000 7f276f400640 1 -- 172.21.15.39:0/1571221685 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27681551b0 con 0x7f276815cd30 2024-09-21T07:41:05.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.193+0000 7f2764c00640 1 -- 172.21.15.39:0/1571221685 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) ==== 95+0+5254 (secure 0 0 0) 0x7f2760062990 con 0x7f276815cd30 2024-09-21T07:41:05.195 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:41:05.195 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":120,"btime":"2024-09-21T07:40:58:384903+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7452,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3695489158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3695489158},{"type":"v1","addr":"172.21.15.39:6835","nonce":3695489158}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":120}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:40:57.400480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":107,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":6269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6269":{"gid":6269,"name":"d","rank":1,"incarnation":110,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6837/2336112936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2336112936},{"type":"v1","addr":"172.21.15.148:6837","nonce":2336112936}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:resolve","state_seq":14,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6269,"qdb_cluster":[6269]},"id":6}]} 2024-09-21T07:41:05.195 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 120 2024-09-21T07:41:05.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.196+0000 7f276f400640 1 -- 172.21.15.39:0/1571221685 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2748078b90 msgr2=0x7f274807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:05.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.196+0000 7f276f400640 1 --2- 172.21.15.39:0/1571221685 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2748078b90 0x7f274807b0a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f275c001720 tx=0x7f275c009040 comp rx=0 tx=0).stop 2024-09-21T07:41:05.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.196+0000 7f276f400640 1 -- 172.21.15.39:0/1571221685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f276815cd30 msgr2=0x7f2768146c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:05.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.196+0000 7f276f400640 1 --2- 172.21.15.39:0/1571221685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f276815cd30 0x7f2768146c50 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f276000b9e0 tx=0x7f276000bea0 comp rx=0 tx=0).stop 2024-09-21T07:41:05.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.196+0000 7f276f400640 1 -- 172.21.15.39:0/1571221685 shutdown_connections 2024-09-21T07:41:05.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.196+0000 7f276f400640 1 --2- 172.21.15.39:0/1571221685 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2748078b90 0x7f274807b0a0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:05.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.196+0000 7f276f400640 1 --2- 172.21.15.39:0/1571221685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f276815cd30 0x7f2768146c50 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:05.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.196+0000 7f276f400640 1 --2- 172.21.15.39:0/1571221685 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27680d0620 0x7f2768146710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:05.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.196+0000 7f276f400640 1 --2- 172.21.15.39:0/1571221685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27680cfd10 0x7f27681454d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:05.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.196+0000 7f276f400640 1 -- 172.21.15.39:0/1571221685 >> 172.21.15.39:0/1571221685 conn(0x7f276806cfc0 msgr2=0x7f276815b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:05.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.196+0000 7f276f400640 1 -- 172.21.15.39:0/1571221685 shutdown_connections 2024-09-21T07:41:05.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.196+0000 7f276f400640 1 -- 172.21.15.39:0/1571221685 wait complete. 2024-09-21T07:41:05.220 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2024-09-21T07:41:05.465 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_snap":12,"last_created":12,"last_destroyed":9,"pending_noop":[],"snaps":[{"snapid":11,"ino":1099511627777,"stamp":"2024-09-21T07:39:42.151005+0000","name":"s25","metadata":{}},{"snapid":12,"ino":1099511627777,"stamp":"2024-09-21T07:40:33.249709+0000","name":"s26","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2024-09-21T07:41:05.465 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 12, "last_destroyed": 9, "last_snap": 12, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s25", "snapid": 11, "stamp": "2024-09-21T07:39:42.151005+0000" }, { "ino": 1099511627777, "metadata": {}, "name": "s26", "snapid": 12, "stamp": "2024-09-21T07:40:33.249709+0000" } ] } 2024-09-21T07:41:05.465 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 6:1 2024-09-21T07:41:05.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.691+0000 7fd6b6e00640 1 -- 172.21.15.39:0/1526137788 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd6b015cd30 msgr2=0x7fd6b00cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:05.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.691+0000 7fd6b6e00640 1 --2- 172.21.15.39:0/1526137788 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd6b015cd30 0x7fd6b00cfd10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0009830 tx=0x7fd6a002a510 comp rx=0 tx=0).stop 2024-09-21T07:41:05.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.691+0000 7fd6b6e00640 1 -- 172.21.15.39:0/1526137788 shutdown_connections 2024-09-21T07:41:05.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.691+0000 7fd6b6e00640 1 --2- 172.21.15.39:0/1526137788 >> v2:172.21.15.148:3300/0 conn(0x7fd6b006f800 0x7fd6b00cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:05.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.691+0000 7fd6b6e00640 1 --2- 172.21.15.39:0/1526137788 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd6b015cd30 0x7fd6b00cfd10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:05.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.691+0000 7fd6b6e00640 1 -- 172.21.15.39:0/1526137788 >> 172.21.15.39:0/1526137788 conn(0x7fd6b006cfc0 msgr2=0x7fd6b006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:05.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.691+0000 7fd6b6e00640 1 -- 172.21.15.39:0/1526137788 shutdown_connections 2024-09-21T07:41:05.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.691+0000 7fd6b6e00640 1 -- 172.21.15.39:0/1526137788 wait complete. 2024-09-21T07:41:05.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.694+0000 7fd6b6e00640 1 Processor -- start 2024-09-21T07:41:05.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.694+0000 7fd6b6e00640 1 -- start start 2024-09-21T07:41:05.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.695+0000 7fd6b6e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6b006f800 0x7fd6b0151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:05.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.695+0000 7fd6b6e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd6b006fc10 0x7fd6b01f3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:05.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.695+0000 7fd6b6e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6b015cd30 0x7fd6b01f38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:05.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.695+0000 7fd6b6e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b0073890 con 0x7fd6b006f800 2024-09-21T07:41:05.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.695+0000 7fd6b6e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b0073710 con 0x7fd6b015cd30 2024-09-21T07:41:05.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.695+0000 7fd6b6e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd6b0073a10 con 0x7fd6b006fc10 2024-09-21T07:41:05.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.696+0000 7fd6afe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6b006f800 0x7fd6b0151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:05.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.696+0000 7fd6afe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6b006f800 0x7fd6b0151da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56092/0 (socket says 172.21.15.39:56092) 2024-09-21T07:41:05.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.696+0000 7fd6afe00640 1 -- 172.21.15.39:0/3611370597 learned_addr learned my addr 172.21.15.39:0/3611370597 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:41:05.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.696+0000 7fd6b4e00640 1 --2- 172.21.15.39:0/3611370597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6b015cd30 0x7fd6b01f38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:05.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.696+0000 7fd6af400640 1 --2- 172.21.15.39:0/3611370597 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd6b006fc10 0x7fd6b01f3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:05.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.696+0000 7fd6afe00640 1 -- 172.21.15.39:0/3611370597 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd6b006fc10 msgr2=0x7fd6b01f3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:05.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.696+0000 7fd6afe00640 1 --2- 172.21.15.39:0/3611370597 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd6b006fc10 0x7fd6b01f3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:05.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.696+0000 7fd6afe00640 1 -- 172.21.15.39:0/3611370597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6b015cd30 msgr2=0x7fd6b01f38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:05.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.696+0000 7fd6afe00640 1 --2- 172.21.15.39:0/3611370597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6b015cd30 0x7fd6b01f38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:05.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.696+0000 7fd6afe00640 1 -- 172.21.15.39:0/3611370597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b0153f60 con 0x7fd6b006f800 2024-09-21T07:41:05.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.696+0000 7fd6af400640 1 --2- 172.21.15.39:0/3611370597 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd6b006fc10 0x7fd6b01f3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:41:05.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.696+0000 7fd6afe00640 1 --2- 172.21.15.39:0/3611370597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6b006f800 0x7fd6b0151da0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0000c00 tx=0x7fd6a0033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:05.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.697+0000 7fd6a7e00640 1 -- 172.21.15.39:0/3611370597 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0031d60 con 0x7fd6b006f800 2024-09-21T07:41:05.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.697+0000 7fd6a7e00640 1 -- 172.21.15.39:0/3611370597 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6a0033e10 con 0x7fd6b006f800 2024-09-21T07:41:05.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.697+0000 7fd6a7e00640 1 -- 172.21.15.39:0/3611370597 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0033310 con 0x7fd6b006f800 2024-09-21T07:41:05.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.697+0000 7fd6b6e00640 1 -- 172.21.15.39:0/3611370597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6b0154240 con 0x7fd6b006f800 2024-09-21T07:41:05.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.697+0000 7fd6b6e00640 1 -- 172.21.15.39:0/3611370597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6b0154800 con 0x7fd6b006f800 2024-09-21T07:41:05.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.698+0000 7fd6b6e00640 1 -- 172.21.15.39:0/3611370597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6b0073ec0 con 0x7fd6b006f800 2024-09-21T07:41:05.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.699+0000 7fd6a7e00640 1 -- 172.21.15.39:0/3611370597 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd6a00334b0 con 0x7fd6b006f800 2024-09-21T07:41:05.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.700+0000 7fd6a7e00640 1 --2- 172.21.15.39:0/3611370597 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd690078b90 0x7fd69007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:05.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.700+0000 7fd6a7e00640 1 -- 172.21.15.39:0/3611370597 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 7689+0+0 (secure 0 0 0) 0x7fd6a00bc540 con 0x7fd6b006f800 2024-09-21T07:41:05.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.700+0000 7fd6af400640 1 --2- 172.21.15.39:0/3611370597 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd690078b90 0x7fd69007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:05.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.701+0000 7fd6af400640 1 --2- 172.21.15.39:0/3611370597 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd690078b90 0x7fd69007b0a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd6b01558a0 tx=0x7fd69c005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:05.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.705+0000 7fd6a7e00640 1 -- 172.21.15.39:0/3611370597 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd6a004a030 con 0x7fd6b006f800 2024-09-21T07:41:05.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.977+0000 7fd6b6e00640 1 -- 172.21.15.39:0/3611370597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "6:1"} v 0) -- 0x7fd6b0154d20 con 0x7fd6b006f800 2024-09-21T07:41:05.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.980+0000 7fd6a7e00640 1 -- 172.21.15.39:0/3611370597 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "6:1"}]=0 failed mds gid 6269 v121) ==== 97+0+0 (secure 0 0 0) 0x7fd6a003f030 con 0x7fd6b006f800 2024-09-21T07:41:05.982 INFO:teuthology.orchestra.run.smithi039.stderr:failed mds gid 6269 2024-09-21T07:41:05.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.983+0000 7fd6b6e00640 1 -- 172.21.15.39:0/3611370597 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd690078b90 msgr2=0x7fd69007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:05.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.983+0000 7fd6b6e00640 1 --2- 172.21.15.39:0/3611370597 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd690078b90 0x7fd69007b0a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd6b01558a0 tx=0x7fd69c005f90 comp rx=0 tx=0).stop 2024-09-21T07:41:05.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.983+0000 7fd6b6e00640 1 -- 172.21.15.39:0/3611370597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6b006f800 msgr2=0x7fd6b0151da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:05.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.983+0000 7fd6b6e00640 1 --2- 172.21.15.39:0/3611370597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6b006f800 0x7fd6b0151da0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0000c00 tx=0x7fd6a0033c20 comp rx=0 tx=0).stop 2024-09-21T07:41:05.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.983+0000 7fd6b6e00640 1 -- 172.21.15.39:0/3611370597 shutdown_connections 2024-09-21T07:41:05.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.983+0000 7fd6b6e00640 1 --2- 172.21.15.39:0/3611370597 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd690078b90 0x7fd69007b0a0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:05.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.983+0000 7fd6b6e00640 1 --2- 172.21.15.39:0/3611370597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6b015cd30 0x7fd6b01f38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:05.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.983+0000 7fd6b6e00640 1 --2- 172.21.15.39:0/3611370597 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd6b006fc10 0x7fd6b01f3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:05.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.983+0000 7fd6b6e00640 1 --2- 172.21.15.39:0/3611370597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6b006f800 0x7fd6b0151da0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:05.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.983+0000 7fd6b6e00640 1 -- 172.21.15.39:0/3611370597 >> 172.21.15.39:0/3611370597 conn(0x7fd6b006cfc0 msgr2=0x7fd6b00b2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:05.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.984+0000 7fd6b6e00640 1 -- 172.21.15.39:0/3611370597 shutdown_connections 2024-09-21T07:41:05.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:05.984+0000 7fd6b6e00640 1 -- 172.21.15.39:0/3611370597 wait complete. 2024-09-21T07:41:06.017 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T07:41:06.017 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T07:41:06.017 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T07:41:06.017 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:41:06.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:41:06.017 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/daemon/state.py", line 146, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi148 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d' 2024-09-21T07:41:06.018 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T07:41:06.018 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T07:41:06.022 INFO:tasks.ceph.mds.d:Started 2024-09-21T07:41:06.022 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:41:06.217 INFO:tasks.ceph.mds.d.smithi148.stdout:starting mds.d at 2024-09-21T07:41:06.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.248+0000 7f0e61600640 1 -- 172.21.15.39:0/2015433006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e5c0d0ba0 msgr2=0x7f0e5c0cb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:06.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.248+0000 7f0e61600640 1 --2- 172.21.15.39:0/2015433006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e5c0d0ba0 0x7f0e5c0cb840 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c0ced10 tx=0x7f0e5402aa30 comp rx=0 tx=0).stop 2024-09-21T07:41:06.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.248+0000 7f0e61600640 1 -- 172.21.15.39:0/2015433006 shutdown_connections 2024-09-21T07:41:06.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.248+0000 7f0e61600640 1 --2- 172.21.15.39:0/2015433006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e5c0d0ba0 0x7f0e5c0cb840 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:06.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.248+0000 7f0e61600640 1 --2- 172.21.15.39:0/2015433006 >> v2:172.21.15.148:3300/0 conn(0x7f0e5c0b1b70 0x7f0e5c0b6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:06.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.248+0000 7f0e61600640 1 -- 172.21.15.39:0/2015433006 >> 172.21.15.39:0/2015433006 conn(0x7f0e5c0b2d40 msgr2=0x7f0e5c0b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:06.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.248+0000 7f0e61600640 1 -- 172.21.15.39:0/2015433006 shutdown_connections 2024-09-21T07:41:06.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.248+0000 7f0e61600640 1 -- 172.21.15.39:0/2015433006 wait complete. 2024-09-21T07:41:06.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.250+0000 7f0e61600640 1 Processor -- start 2024-09-21T07:41:06.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.250+0000 7f0e61600640 1 -- start start 2024-09-21T07:41:06.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.252+0000 7f0e61600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e5c0b1b70 0x7f0e5c09cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:06.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.252+0000 7f0e61600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e5c0b6940 0x7f0e5c1608c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:06.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.252+0000 7f0e61600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e5c0d0ba0 0x7f0e5c15f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:06.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.252+0000 7f0e61600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5c0cc110 con 0x7f0e5c0b1b70 2024-09-21T07:41:06.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.252+0000 7f0e61600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5c0cbf90 con 0x7f0e5c0d0ba0 2024-09-21T07:41:06.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.252+0000 7f0e61600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f0e5c0cc290 con 0x7f0e5c0b6940 2024-09-21T07:41:06.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.252+0000 7f0e5be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e5c0b1b70 0x7f0e5c09cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:06.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.252+0000 7f0e5be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e5c0b1b70 0x7f0e5c09cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56118/0 (socket says 172.21.15.39:56118) 2024-09-21T07:41:06.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.252+0000 7f0e5be00640 1 -- 172.21.15.39:0/3279324579 learned_addr learned my addr 172.21.15.39:0/3279324579 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:41:06.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.252+0000 7f0e60c00640 1 --2- 172.21.15.39:0/3279324579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e5c0d0ba0 0x7f0e5c15f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:06.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.252+0000 7f0e53e00640 1 --2- 172.21.15.39:0/3279324579 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e5c0b6940 0x7f0e5c1608c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:06.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.252+0000 7f0e5be00640 1 -- 172.21.15.39:0/3279324579 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e5c0b6940 msgr2=0x7f0e5c1608c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:06.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.252+0000 7f0e5be00640 1 --2- 172.21.15.39:0/3279324579 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e5c0b6940 0x7f0e5c1608c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:06.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.252+0000 7f0e5be00640 1 -- 172.21.15.39:0/3279324579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e5c0d0ba0 msgr2=0x7f0e5c15f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:06.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.252+0000 7f0e5be00640 1 --2- 172.21.15.39:0/3279324579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e5c0d0ba0 0x7f0e5c15f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:06.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.252+0000 7f0e5be00640 1 -- 172.21.15.39:0/3279324579 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e5c09bbc0 con 0x7f0e5c0b1b70 2024-09-21T07:41:06.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.252+0000 7f0e53e00640 1 --2- 172.21.15.39:0/3279324579 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e5c0b6940 0x7f0e5c1608c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:41:06.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.253+0000 7f0e5be00640 1 --2- 172.21.15.39:0/3279324579 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e5c0b1b70 0x7f0e5c09cc60 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c00e790 tx=0x7f0e4c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:06.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.253+0000 7f0e58c00640 1 -- 172.21.15.39:0/3279324579 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c00eea0 con 0x7f0e5c0b1b70 2024-09-21T07:41:06.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.253+0000 7f0e58c00640 1 -- 172.21.15.39:0/3279324579 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e4c00cdb0 con 0x7f0e5c0b1b70 2024-09-21T07:41:06.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.253+0000 7f0e58c00640 1 -- 172.21.15.39:0/3279324579 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c010690 con 0x7f0e5c0b1b70 2024-09-21T07:41:06.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.253+0000 7f0e61600640 1 -- 172.21.15.39:0/3279324579 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e5c09bea0 con 0x7f0e5c0b1b70 2024-09-21T07:41:06.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.253+0000 7f0e61600640 1 -- 172.21.15.39:0/3279324579 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e5c09c460 con 0x7f0e5c0b1b70 2024-09-21T07:41:06.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.254+0000 7f0e61600640 1 -- 172.21.15.39:0/3279324579 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e5c0cc820 con 0x7f0e5c0b1b70 2024-09-21T07:41:06.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.255+0000 7f0e58c00640 1 -- 172.21.15.39:0/3279324579 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0e4c010830 con 0x7f0e5c0b1b70 2024-09-21T07:41:06.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.256+0000 7f0e58c00640 1 --2- 172.21.15.39:0/3279324579 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0e38078b90 0x7f0e3807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:06.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.257+0000 7f0e58c00640 1 -- 172.21.15.39:0/3279324579 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7775+0+0 (secure 0 0 0) 0x7f0e4c09bf00 con 0x7f0e5c0b1b70 2024-09-21T07:41:06.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.257+0000 7f0e53e00640 1 --2- 172.21.15.39:0/3279324579 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0e38078b90 0x7f0e3807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:06.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.258+0000 7f0e53e00640 1 --2- 172.21.15.39:0/3279324579 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0e38078b90 0x7f0e3807b0a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0e44007fd0 tx=0x7f0e44009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:06.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.261+0000 7f0e58c00640 1 -- 172.21.15.39:0/3279324579 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0e4c0634d0 con 0x7f0e5c0b1b70 2024-09-21T07:41:06.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.495+0000 7f0e61600640 1 -- 172.21.15.39:0/3279324579 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e5c160e00 con 0x7f0e5c0b1b70 2024-09-21T07:41:06.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.496+0000 7f0e58c00640 1 -- 172.21.15.39:0/3279324579 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) ==== 95+0+4418 (secure 0 0 0) 0x7f0e4c067f00 con 0x7f0e5c0b1b70 2024-09-21T07:41:06.498 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:41:06.498 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":122,"btime":"2024-09-21T07:41:05:987782+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":122,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:41:05.987762+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:resolve","state_seq":14,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7452":{"gid":7452,"name":"a","rank":1,"incarnation":122,"state":"up:replay","state_seq":1,"addr":"172.21.15.39:6835/3695489158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3695489158},{"type":"v1","addr":"172.21.15.39:6835","nonce":3695489158}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T07:41:06.498 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 122 2024-09-21T07:41:06.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.498+0000 7f0e61600640 1 -- 172.21.15.39:0/3279324579 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0e38078b90 msgr2=0x7f0e3807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:06.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.498+0000 7f0e61600640 1 --2- 172.21.15.39:0/3279324579 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0e38078b90 0x7f0e3807b0a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0e44007fd0 tx=0x7f0e44009040 comp rx=0 tx=0).stop 2024-09-21T07:41:06.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.498+0000 7f0e61600640 1 -- 172.21.15.39:0/3279324579 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e5c0b1b70 msgr2=0x7f0e5c09cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:06.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.498+0000 7f0e61600640 1 --2- 172.21.15.39:0/3279324579 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e5c0b1b70 0x7f0e5c09cc60 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c00e790 tx=0x7f0e4c00ec50 comp rx=0 tx=0).stop 2024-09-21T07:41:06.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.498+0000 7f0e61600640 1 -- 172.21.15.39:0/3279324579 shutdown_connections 2024-09-21T07:41:06.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.498+0000 7f0e61600640 1 --2- 172.21.15.39:0/3279324579 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0e38078b90 0x7f0e3807b0a0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:06.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.498+0000 7f0e61600640 1 --2- 172.21.15.39:0/3279324579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e5c0d0ba0 0x7f0e5c15f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:06.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.498+0000 7f0e61600640 1 --2- 172.21.15.39:0/3279324579 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e5c0b6940 0x7f0e5c1608c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:06.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.498+0000 7f0e61600640 1 --2- 172.21.15.39:0/3279324579 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e5c0b1b70 0x7f0e5c09cc60 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:06.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.499+0000 7f0e61600640 1 -- 172.21.15.39:0/3279324579 >> 172.21.15.39:0/3279324579 conn(0x7f0e5c0b2d40 msgr2=0x7f0e5c0b5880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:06.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.499+0000 7f0e61600640 1 -- 172.21.15.39:0/3279324579 shutdown_connections 2024-09-21T07:41:06.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:06.499+0000 7f0e61600640 1 -- 172.21.15.39:0/3279324579 wait complete. 2024-09-21T07:41:06.525 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-09-21T07:41:09.386 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-09-21T07:41:11.527 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:41:11.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:11.763+0000 7ffa7f000640 1 -- 172.21.15.39:0/2145975767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa7815abe0 msgr2=0x7ffa780cf490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:11.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:11.763+0000 7ffa7f000640 1 --2- 172.21.15.39:0/2145975767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa7815abe0 0x7ffa780cf490 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7ffa78072a50 tx=0x7ffa6402a4d0 comp rx=0 tx=0).stop 2024-09-21T07:41:11.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:11.763+0000 7ffa7f000640 1 -- 172.21.15.39:0/2145975767 shutdown_connections 2024-09-21T07:41:11.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:11.763+0000 7ffa7f000640 1 --2- 172.21.15.39:0/2145975767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa7815abe0 0x7ffa780cf490 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:11.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:11.763+0000 7ffa7f000640 1 --2- 172.21.15.39:0/2145975767 >> v2:172.21.15.148:3300/0 conn(0x7ffa780b43f0 0x7ffa780ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:11.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:11.763+0000 7ffa7f000640 1 -- 172.21.15.39:0/2145975767 >> 172.21.15.39:0/2145975767 conn(0x7ffa780b2510 msgr2=0x7ffa780d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:11.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:11.763+0000 7ffa7f000640 1 -- 172.21.15.39:0/2145975767 shutdown_connections 2024-09-21T07:41:11.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:11.763+0000 7ffa7f000640 1 -- 172.21.15.39:0/2145975767 wait complete. 2024-09-21T07:41:11.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:11.765+0000 7ffa7f000640 1 Processor -- start 2024-09-21T07:41:11.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:11.765+0000 7ffa7f000640 1 -- start start 2024-09-21T07:41:11.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:11.767+0000 7ffa7f000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa780af490 0x7ffa781f63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:11.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:11.767+0000 7ffa7f000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa780b43f0 0x7ffa781550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:11.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:11.767+0000 7ffa7f000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffa7815abe0 0x7ffa780746a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:11.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:11.767+0000 7ffa7f000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ffa780cf9d0 con 0x7ffa780af490 2024-09-21T07:41:11.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:11.767+0000 7ffa7f000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ffa780712d0 con 0x7ffa780b43f0 2024-09-21T07:41:11.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:11.767+0000 7ffa7f000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7ffa780cfb50 con 0x7ffa7815abe0 2024-09-21T07:41:11.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:11.767+0000 7ffa7dc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa780af490 0x7ffa781f63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:11.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:11.767+0000 7ffa7dc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa780af490 0x7ffa781f63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56128/0 (socket says 172.21.15.39:56128) 2024-09-21T07:41:11.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:11.767+0000 7ffa7dc00640 1 -- 172.21.15.39:0/478821548 learned_addr learned my addr 172.21.15.39:0/478821548 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:41:11.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:11.767+0000 7ffa7d200640 1 --2- 172.21.15.39:0/478821548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa780b43f0 0x7ffa781550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:11.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:11.767+0000 7ffa7e600640 1 --2- 172.21.15.39:0/478821548 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffa7815abe0 0x7ffa780746a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:11.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:11.768+0000 7ffa7dc00640 1 -- 172.21.15.39:0/478821548 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffa7815abe0 msgr2=0x7ffa780746a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:11.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:11.768+0000 7ffa7dc00640 1 --2- 172.21.15.39:0/478821548 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffa7815abe0 0x7ffa780746a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:11.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:11.768+0000 7ffa7dc00640 1 -- 172.21.15.39:0/478821548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa780b43f0 msgr2=0x7ffa781550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:11.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:11.768+0000 7ffa7dc00640 1 --2- 172.21.15.39:0/478821548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa780b43f0 0x7ffa781550d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:11.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:11.768+0000 7ffa7dc00640 1 -- 172.21.15.39:0/478821548 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa78073080 con 0x7ffa780af490 2024-09-21T07:41:11.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:11.768+0000 7ffa7dc00640 1 --2- 172.21.15.39:0/478821548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa780af490 0x7ffa781f63f0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c00e7c0 tx=0x7ffa6c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:11.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:11.768+0000 7ffa75600640 1 -- 172.21.15.39:0/478821548 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c00ccc0 con 0x7ffa780af490 2024-09-21T07:41:11.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:11.768+0000 7ffa75600640 1 -- 172.21.15.39:0/478821548 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa6c010430 con 0x7ffa780af490 2024-09-21T07:41:11.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:11.768+0000 7ffa75600640 1 -- 172.21.15.39:0/478821548 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c010860 con 0x7ffa780af490 2024-09-21T07:41:11.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:11.768+0000 7ffa7f000640 1 -- 172.21.15.39:0/478821548 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa78073360 con 0x7ffa780af490 2024-09-21T07:41:11.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:11.768+0000 7ffa7f000640 1 -- 172.21.15.39:0/478821548 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa7815cf30 con 0x7ffa780af490 2024-09-21T07:41:11.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:11.769+0000 7ffa7f000640 1 -- 172.21.15.39:0/478821548 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa780ca890 con 0x7ffa780af490 2024-09-21T07:41:11.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:11.770+0000 7ffa75600640 1 -- 172.21.15.39:0/478821548 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ffa6c010a00 con 0x7ffa780af490 2024-09-21T07:41:11.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:11.771+0000 7ffa75600640 1 --2- 172.21.15.39:0/478821548 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ffa58078bc0 0x7ffa5807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:11.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:11.772+0000 7ffa75600640 1 -- 172.21.15.39:0/478821548 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7775+0+0 (secure 0 0 0) 0x7ffa6c09bec0 con 0x7ffa780af490 2024-09-21T07:41:11.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:11.772+0000 7ffa7d200640 1 --2- 172.21.15.39:0/478821548 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ffa58078bc0 0x7ffa5807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:11.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:11.773+0000 7ffa7d200640 1 --2- 172.21.15.39:0/478821548 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ffa58078bc0 0x7ffa5807b0d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ffa64033000 tx=0x7ffa64004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:11.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:11.776+0000 7ffa75600640 1 -- 172.21.15.39:0/478821548 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ffa6c063490 con 0x7ffa780af490 2024-09-21T07:41:12.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.015+0000 7ffa7f000640 1 -- 172.21.15.39:0/478821548 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa78070940 con 0x7ffa780af490 2024-09-21T07:41:12.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.016+0000 7ffa75600640 1 -- 172.21.15.39:0/478821548 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5246 (secure 0 0 0) 0x7ffa6c06c020 con 0x7ffa780af490 2024-09-21T07:41:12.018 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:41:12.018 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":124,"btime":"2024-09-21T07:41:07:991457+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6425,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/73220122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":73220122},{"type":"v1","addr":"172.21.15.148:6837","nonce":73220122}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:41:06.994864+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:reconnect","state_seq":17,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7452":{"gid":7452,"name":"a","rank":1,"incarnation":122,"state":"up:reconnect","state_seq":5,"addr":"172.21.15.39:6835/3695489158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3695489158},{"type":"v1","addr":"172.21.15.39:6835","nonce":3695489158}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T07:41:12.019 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 124 2024-09-21T07:41:12.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.019+0000 7ffa7f000640 1 -- 172.21.15.39:0/478821548 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ffa58078bc0 msgr2=0x7ffa5807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:12.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.019+0000 7ffa7f000640 1 --2- 172.21.15.39:0/478821548 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ffa58078bc0 0x7ffa5807b0d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ffa64033000 tx=0x7ffa64004730 comp rx=0 tx=0).stop 2024-09-21T07:41:12.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.019+0000 7ffa7f000640 1 -- 172.21.15.39:0/478821548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa780af490 msgr2=0x7ffa781f63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:12.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.019+0000 7ffa7f000640 1 --2- 172.21.15.39:0/478821548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa780af490 0x7ffa781f63f0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c00e7c0 tx=0x7ffa6c00ec80 comp rx=0 tx=0).stop 2024-09-21T07:41:12.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.019+0000 7ffa7f000640 1 -- 172.21.15.39:0/478821548 shutdown_connections 2024-09-21T07:41:12.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.019+0000 7ffa7f000640 1 --2- 172.21.15.39:0/478821548 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ffa58078bc0 0x7ffa5807b0d0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:12.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.019+0000 7ffa7f000640 1 --2- 172.21.15.39:0/478821548 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffa7815abe0 0x7ffa780746a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:12.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.019+0000 7ffa7f000640 1 --2- 172.21.15.39:0/478821548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa780b43f0 0x7ffa781550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:12.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.019+0000 7ffa7f000640 1 --2- 172.21.15.39:0/478821548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa780af490 0x7ffa781f63f0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:12.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.019+0000 7ffa7f000640 1 -- 172.21.15.39:0/478821548 >> 172.21.15.39:0/478821548 conn(0x7ffa780b2510 msgr2=0x7ffa780ad560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:12.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.019+0000 7ffa7f000640 1 -- 172.21.15.39:0/478821548 shutdown_connections 2024-09-21T07:41:12.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.019+0000 7ffa7f000640 1 -- 172.21.15.39:0/478821548 wait complete. 2024-09-21T07:41:12.045 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-21T07:41:12.046 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:41:12.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.270+0000 7fda97e00640 1 -- 172.21.15.39:0/3881773860 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda980cbd70 msgr2=0x7fda98072fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:12.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.270+0000 7fda97e00640 1 --2- 172.21.15.39:0/3881773860 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda980cbd70 0x7fda98072fa0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fda90005fd0 tx=0x7fda9002aa90 comp rx=0 tx=0).stop 2024-09-21T07:41:12.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.270+0000 7fda97e00640 1 -- 172.21.15.39:0/3881773860 shutdown_connections 2024-09-21T07:41:12.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.270+0000 7fda97e00640 1 --2- 172.21.15.39:0/3881773860 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda980cbd70 0x7fda98072fa0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:12.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.270+0000 7fda97e00640 1 --2- 172.21.15.39:0/3881773860 >> v2:172.21.15.148:3300/0 conn(0x7fda980d0f20 0x7fda9815c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:12.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.270+0000 7fda97e00640 1 -- 172.21.15.39:0/3881773860 >> 172.21.15.39:0/3881773860 conn(0x7fda980b2510 msgr2=0x7fda980d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:12.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.270+0000 7fda97e00640 1 -- 172.21.15.39:0/3881773860 shutdown_connections 2024-09-21T07:41:12.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.270+0000 7fda97e00640 1 -- 172.21.15.39:0/3881773860 wait complete. 2024-09-21T07:41:12.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.272+0000 7fda97e00640 1 Processor -- start 2024-09-21T07:41:12.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.273+0000 7fda97e00640 1 -- start start 2024-09-21T07:41:12.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.274+0000 7fda97e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda980cb960 0x7fda98158af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:12.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.274+0000 7fda97e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda980cbd70 0x7fda98159030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:12.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.274+0000 7fda97e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda980d0f20 0x7fda981f3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:12.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.274+0000 7fda97e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fda980b4570 con 0x7fda980cbd70 2024-09-21T07:41:12.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.274+0000 7fda97e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fda980b43f0 con 0x7fda980cb960 2024-09-21T07:41:12.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.274+0000 7fda97e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fda980b46f0 con 0x7fda980d0f20 2024-09-21T07:41:12.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.274+0000 7fda8fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda980cbd70 0x7fda98159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:12.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.274+0000 7fda8fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda980cbd70 0x7fda98159030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54856/0 (socket says 172.21.15.39:54856) 2024-09-21T07:41:12.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.274+0000 7fda8fe00640 1 -- 172.21.15.39:0/324563207 learned_addr learned my addr 172.21.15.39:0/324563207 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:41:12.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.275+0000 7fda96a00640 1 --2- 172.21.15.39:0/324563207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda980cb960 0x7fda98158af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:12.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.275+0000 7fda97400640 1 --2- 172.21.15.39:0/324563207 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda980d0f20 0x7fda981f3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:12.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.275+0000 7fda8fe00640 1 -- 172.21.15.39:0/324563207 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda980d0f20 msgr2=0x7fda981f3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:12.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.275+0000 7fda8fe00640 1 --2- 172.21.15.39:0/324563207 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda980d0f20 0x7fda981f3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:12.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.275+0000 7fda8fe00640 1 -- 172.21.15.39:0/324563207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda980cb960 msgr2=0x7fda98158af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:12.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.275+0000 7fda8fe00640 1 --2- 172.21.15.39:0/324563207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda980cb960 0x7fda98158af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:12.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.275+0000 7fda8fe00640 1 -- 172.21.15.39:0/324563207 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda981f6c30 con 0x7fda980cbd70 2024-09-21T07:41:12.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.275+0000 7fda8fe00640 1 --2- 172.21.15.39:0/324563207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda980cbd70 0x7fda98159030 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fda8000c3a0 tx=0x7fda8000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:12.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.275+0000 7fda8ea00640 1 -- 172.21.15.39:0/324563207 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda80017070 con 0x7fda980cbd70 2024-09-21T07:41:12.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.275+0000 7fda8ea00640 1 -- 172.21.15.39:0/324563207 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda800053c0 con 0x7fda980cbd70 2024-09-21T07:41:12.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.276+0000 7fda8ea00640 1 -- 172.21.15.39:0/324563207 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda80003850 con 0x7fda980cbd70 2024-09-21T07:41:12.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.276+0000 7fda97e00640 1 -- 172.21.15.39:0/324563207 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda981f6f10 con 0x7fda980cbd70 2024-09-21T07:41:12.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.276+0000 7fda97e00640 1 -- 172.21.15.39:0/324563207 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda981f74d0 con 0x7fda980cbd70 2024-09-21T07:41:12.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.277+0000 7fda97e00640 1 -- 172.21.15.39:0/324563207 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda980b4c10 con 0x7fda980cbd70 2024-09-21T07:41:12.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.278+0000 7fda8ea00640 1 -- 172.21.15.39:0/324563207 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fda8001a070 con 0x7fda980cbd70 2024-09-21T07:41:12.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.279+0000 7fda8ea00640 1 --2- 172.21.15.39:0/324563207 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fda74078b90 0x7fda7407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:12.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.279+0000 7fda8ea00640 1 -- 172.21.15.39:0/324563207 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7775+0+0 (secure 0 0 0) 0x7fda8009aa40 con 0x7fda980cbd70 2024-09-21T07:41:12.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.280+0000 7fda96a00640 1 --2- 172.21.15.39:0/324563207 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fda74078b90 0x7fda7407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:12.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.280+0000 7fda96a00640 1 --2- 172.21.15.39:0/324563207 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fda74078b90 0x7fda7407b0a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fda88009ac0 tx=0x7fda88005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:12.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.283+0000 7fda8ea00640 1 -- 172.21.15.39:0/324563207 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fda80062550 con 0x7fda980cbd70 2024-09-21T07:41:12.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.528+0000 7fda97e00640 1 -- 172.21.15.39:0/324563207 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda9806df80 con 0x7fda980cbd70 2024-09-21T07:41:12.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.529+0000 7fda8ea00640 1 -- 172.21.15.39:0/324563207 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5246 (secure 0 0 0) 0x7fda80061e50 con 0x7fda980cbd70 2024-09-21T07:41:12.531 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:41:12.531 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":124,"btime":"2024-09-21T07:41:07:991457+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6425,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/73220122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":73220122},{"type":"v1","addr":"172.21.15.148:6837","nonce":73220122}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:41:06.994864+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:reconnect","state_seq":17,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7452":{"gid":7452,"name":"a","rank":1,"incarnation":122,"state":"up:reconnect","state_seq":5,"addr":"172.21.15.39:6835/3695489158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3695489158},{"type":"v1","addr":"172.21.15.39:6835","nonce":3695489158}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T07:41:12.531 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 124 2024-09-21T07:41:12.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.531+0000 7fda97e00640 1 -- 172.21.15.39:0/324563207 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fda74078b90 msgr2=0x7fda7407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:12.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.531+0000 7fda97e00640 1 --2- 172.21.15.39:0/324563207 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fda74078b90 0x7fda7407b0a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fda88009ac0 tx=0x7fda88005f90 comp rx=0 tx=0).stop 2024-09-21T07:41:12.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.531+0000 7fda97e00640 1 -- 172.21.15.39:0/324563207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda980cbd70 msgr2=0x7fda98159030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:12.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.531+0000 7fda97e00640 1 --2- 172.21.15.39:0/324563207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda980cbd70 0x7fda98159030 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fda8000c3a0 tx=0x7fda8000c860 comp rx=0 tx=0).stop 2024-09-21T07:41:12.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.531+0000 7fda97e00640 1 -- 172.21.15.39:0/324563207 shutdown_connections 2024-09-21T07:41:12.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.531+0000 7fda97e00640 1 --2- 172.21.15.39:0/324563207 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fda74078b90 0x7fda7407b0a0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:12.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.531+0000 7fda97e00640 1 --2- 172.21.15.39:0/324563207 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda980d0f20 0x7fda981f3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:12.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.531+0000 7fda97e00640 1 --2- 172.21.15.39:0/324563207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda980cbd70 0x7fda98159030 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:12.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.531+0000 7fda97e00640 1 --2- 172.21.15.39:0/324563207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda980cb960 0x7fda98158af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:12.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.531+0000 7fda97e00640 1 -- 172.21.15.39:0/324563207 >> 172.21.15.39:0/324563207 conn(0x7fda980b2510 msgr2=0x7fda98074ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:12.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.531+0000 7fda97e00640 1 -- 172.21.15.39:0/324563207 shutdown_connections 2024-09-21T07:41:12.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:12.531+0000 7fda97e00640 1 -- 172.21.15.39:0/324563207 wait complete. 2024-09-21T07:41:12.559 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:reconnect 2024-09-21T07:41:13.560 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:41:13.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:13.795+0000 7fa31d600640 1 -- 172.21.15.39:0/1407381182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3180cfd10 msgr2=0x7fa3180cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:13.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:13.795+0000 7fa31d600640 1 --2- 172.21.15.39:0/1407381182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3180cfd10 0x7fa3180cd8e0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fa300005fd0 tx=0x7fa30002a5f0 comp rx=0 tx=0).stop 2024-09-21T07:41:13.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:13.795+0000 7fa31d600640 1 -- 172.21.15.39:0/1407381182 shutdown_connections 2024-09-21T07:41:13.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:13.795+0000 7fa31d600640 1 --2- 172.21.15.39:0/1407381182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3180cfd10 0x7fa3180cd8e0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:13.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:13.795+0000 7fa31d600640 1 -- 172.21.15.39:0/1407381182 >> 172.21.15.39:0/1407381182 conn(0x7fa31806cfc0 msgr2=0x7fa31806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:13.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:13.796+0000 7fa31d600640 1 -- 172.21.15.39:0/1407381182 shutdown_connections 2024-09-21T07:41:13.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:13.796+0000 7fa31d600640 1 -- 172.21.15.39:0/1407381182 wait complete. 2024-09-21T07:41:13.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:13.798+0000 7fa31d600640 1 Processor -- start 2024-09-21T07:41:13.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:13.798+0000 7fa31d600640 1 -- start start 2024-09-21T07:41:13.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:13.799+0000 7fa31d600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3180cfd10 0x7fa3180b4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:13.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:13.799+0000 7fa31d600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3180d0620 0x7fa31814cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:13.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:13.799+0000 7fa31d600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa31815cd30 0x7fa3181e93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:13.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:13.799+0000 7fa31d600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa318073aa0 con 0x7fa3180d0620 2024-09-21T07:41:13.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:13.799+0000 7fa31d600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa318073920 con 0x7fa3180cfd10 2024-09-21T07:41:13.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:13.799+0000 7fa31d600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa318073c20 con 0x7fa31815cd30 2024-09-21T07:41:13.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:13.800+0000 7fa30fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3180d0620 0x7fa31814cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:13.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:13.800+0000 7fa30fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3180d0620 0x7fa31814cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54864/0 (socket says 172.21.15.39:54864) 2024-09-21T07:41:13.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:13.800+0000 7fa30fe00640 1 -- 172.21.15.39:0/2549276642 learned_addr learned my addr 172.21.15.39:0/2549276642 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:41:13.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:13.800+0000 7fa316a00640 1 --2- 172.21.15.39:0/2549276642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3180cfd10 0x7fa3180b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:13.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:13.800+0000 7fa317400640 1 --2- 172.21.15.39:0/2549276642 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa31815cd30 0x7fa3181e93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:13.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:13.800+0000 7fa30fe00640 1 -- 172.21.15.39:0/2549276642 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa31815cd30 msgr2=0x7fa3181e93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:13.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:13.800+0000 7fa30fe00640 1 --2- 172.21.15.39:0/2549276642 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa31815cd30 0x7fa3181e93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:13.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:13.800+0000 7fa30fe00640 1 -- 172.21.15.39:0/2549276642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3180cfd10 msgr2=0x7fa3180b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:13.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:13.800+0000 7fa30fe00640 1 --2- 172.21.15.39:0/2549276642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3180cfd10 0x7fa3180b4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:13.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:13.800+0000 7fa30fe00640 1 -- 172.21.15.39:0/2549276642 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa31814eb00 con 0x7fa3180d0620 2024-09-21T07:41:13.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:13.800+0000 7fa316a00640 1 --2- 172.21.15.39:0/2549276642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3180cfd10 0x7fa3180b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:41:13.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:13.800+0000 7fa30fe00640 1 --2- 172.21.15.39:0/2549276642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3180d0620 0x7fa31814cb70 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fa300006010 tx=0x7fa3000039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:13.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:13.801+0000 7fa30ea00640 1 -- 172.21.15.39:0/2549276642 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30003e030 con 0x7fa3180d0620 2024-09-21T07:41:13.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:13.801+0000 7fa30ea00640 1 -- 172.21.15.39:0/2549276642 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3000047a0 con 0x7fa3180d0620 2024-09-21T07:41:13.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:13.801+0000 7fa30ea00640 1 -- 172.21.15.39:0/2549276642 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa300035040 con 0x7fa3180d0620 2024-09-21T07:41:13.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:13.801+0000 7fa31d600640 1 -- 172.21.15.39:0/2549276642 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa31814ede0 con 0x7fa3180d0620 2024-09-21T07:41:13.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:13.801+0000 7fa31d600640 1 -- 172.21.15.39:0/2549276642 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa31814f3a0 con 0x7fa3180d0620 2024-09-21T07:41:13.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:13.802+0000 7fa31d600640 1 -- 172.21.15.39:0/2549276642 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa318152810 con 0x7fa3180d0620 2024-09-21T07:41:13.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:13.803+0000 7fa30ea00640 1 -- 172.21.15.39:0/2549276642 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa300041030 con 0x7fa3180d0620 2024-09-21T07:41:13.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:13.804+0000 7fa30ea00640 1 --2- 172.21.15.39:0/2549276642 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa2f4078b90 0x7fa2f407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:13.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:13.804+0000 7fa30ea00640 1 -- 172.21.15.39:0/2549276642 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 7775+0+0 (secure 0 0 0) 0x7fa3000bbde0 con 0x7fa3180d0620 2024-09-21T07:41:13.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:13.805+0000 7fa316a00640 1 --2- 172.21.15.39:0/2549276642 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa2f4078b90 0x7fa2f407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:13.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:13.806+0000 7fa316a00640 1 --2- 172.21.15.39:0/2549276642 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa2f4078b90 0x7fa2f407b0a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa308000f60 tx=0x7fa308009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:13.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:13.809+0000 7fa30ea00640 1 -- 172.21.15.39:0/2549276642 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa318152810 con 0x7fa3180d0620 2024-09-21T07:41:14.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:14.052+0000 7fa31d600640 1 -- 172.21.15.39:0/2549276642 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3180ce760 con 0x7fa3180d0620 2024-09-21T07:41:14.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:14.053+0000 7fa30ea00640 1 -- 172.21.15.39:0/2549276642 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5246 (secure 0 0 0) 0x7fa3180ce760 con 0x7fa3180d0620 2024-09-21T07:41:14.055 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:41:14.055 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":124,"btime":"2024-09-21T07:41:07:991457+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6425,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/73220122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":73220122},{"type":"v1","addr":"172.21.15.148:6837","nonce":73220122}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:41:06.994864+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:reconnect","state_seq":17,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7452":{"gid":7452,"name":"a","rank":1,"incarnation":122,"state":"up:reconnect","state_seq":5,"addr":"172.21.15.39:6835/3695489158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3695489158},{"type":"v1","addr":"172.21.15.39:6835","nonce":3695489158}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T07:41:14.055 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 124 2024-09-21T07:41:14.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:14.055+0000 7fa31d600640 1 -- 172.21.15.39:0/2549276642 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa2f4078b90 msgr2=0x7fa2f407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:14.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:14.055+0000 7fa31d600640 1 --2- 172.21.15.39:0/2549276642 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa2f4078b90 0x7fa2f407b0a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa308000f60 tx=0x7fa308009040 comp rx=0 tx=0).stop 2024-09-21T07:41:14.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:14.055+0000 7fa31d600640 1 -- 172.21.15.39:0/2549276642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3180d0620 msgr2=0x7fa31814cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:14.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:14.055+0000 7fa31d600640 1 --2- 172.21.15.39:0/2549276642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3180d0620 0x7fa31814cb70 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fa300006010 tx=0x7fa3000039e0 comp rx=0 tx=0).stop 2024-09-21T07:41:14.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:14.055+0000 7fa31d600640 1 -- 172.21.15.39:0/2549276642 shutdown_connections 2024-09-21T07:41:14.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:14.056+0000 7fa31d600640 1 --2- 172.21.15.39:0/2549276642 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa2f4078b90 0x7fa2f407b0a0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:14.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:14.056+0000 7fa31d600640 1 --2- 172.21.15.39:0/2549276642 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa31815cd30 0x7fa3181e93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:14.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:14.056+0000 7fa31d600640 1 --2- 172.21.15.39:0/2549276642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3180d0620 0x7fa31814cb70 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:14.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:14.056+0000 7fa31d600640 1 --2- 172.21.15.39:0/2549276642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3180cfd10 0x7fa3180b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:14.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:14.056+0000 7fa31d600640 1 -- 172.21.15.39:0/2549276642 >> 172.21.15.39:0/2549276642 conn(0x7fa31806cfc0 msgr2=0x7fa318152100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:14.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:14.056+0000 7fa31d600640 1 -- 172.21.15.39:0/2549276642 shutdown_connections 2024-09-21T07:41:14.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:14.056+0000 7fa31d600640 1 -- 172.21.15.39:0/2549276642 wait complete. 2024-09-21T07:41:14.081 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:reconnect 2024-09-21T07:41:15.082 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:41:15.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.317+0000 7effd9800640 1 -- 172.21.15.39:0/2759642830 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effd40d19f0 msgr2=0x7effd40d4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:15.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.317+0000 7effd9800640 1 --2- 172.21.15.39:0/2759642830 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effd40d19f0 0x7effd40d4600 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7effc4009820 tx=0x7effc402a510 comp rx=0 tx=0).stop 2024-09-21T07:41:15.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.317+0000 7effd9800640 1 -- 172.21.15.39:0/2759642830 shutdown_connections 2024-09-21T07:41:15.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.317+0000 7effd9800640 1 --2- 172.21.15.39:0/2759642830 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effd40d19f0 0x7effd40d4600 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:15.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.317+0000 7effd9800640 1 -- 172.21.15.39:0/2759642830 >> 172.21.15.39:0/2759642830 conn(0x7effd408bfb0 msgr2=0x7effd409d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:15.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.317+0000 7effd9800640 1 -- 172.21.15.39:0/2759642830 shutdown_connections 2024-09-21T07:41:15.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.317+0000 7effd9800640 1 -- 172.21.15.39:0/2759642830 wait complete. 2024-09-21T07:41:15.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.319+0000 7effd9800640 1 Processor -- start 2024-09-21T07:41:15.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.319+0000 7effd9800640 1 -- start start 2024-09-21T07:41:15.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.321+0000 7effd9800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effd40b5890 0x7effd4157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:15.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.321+0000 7effd9800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7effd40d19f0 0x7effd40a1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:15.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.321+0000 7effd9800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effd40d4b40 0x7effd40a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:15.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.321+0000 7effd9800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7effd40b5e20 con 0x7effd40b5890 2024-09-21T07:41:15.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.321+0000 7effd9800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7effd40b5ca0 con 0x7effd40d4b40 2024-09-21T07:41:15.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.321+0000 7effd9800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7effd40b2ab0 con 0x7effd40d19f0 2024-09-21T07:41:15.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.321+0000 7effd3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effd40b5890 0x7effd4157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:15.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.321+0000 7effd3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effd40b5890 0x7effd4157cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54878/0 (socket says 172.21.15.39:54878) 2024-09-21T07:41:15.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.321+0000 7effd3e00640 1 -- 172.21.15.39:0/3689089295 learned_addr learned my addr 172.21.15.39:0/3689089295 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:41:15.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.321+0000 7effd8e00640 1 --2- 172.21.15.39:0/3689089295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effd40d4b40 0x7effd40a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:15.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.321+0000 7effcb400640 1 --2- 172.21.15.39:0/3689089295 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7effd40d19f0 0x7effd40a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:15.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.321+0000 7effd3e00640 1 -- 172.21.15.39:0/3689089295 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7effd40d19f0 msgr2=0x7effd40a1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:15.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.321+0000 7effd3e00640 1 --2- 172.21.15.39:0/3689089295 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7effd40d19f0 0x7effd40a1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:15.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.321+0000 7effd3e00640 1 -- 172.21.15.39:0/3689089295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effd40d4b40 msgr2=0x7effd40a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:15.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.321+0000 7effd3e00640 1 --2- 172.21.15.39:0/3689089295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effd40d4b40 0x7effd40a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:15.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.321+0000 7effd3e00640 1 -- 172.21.15.39:0/3689089295 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd40c8710 con 0x7effd40b5890 2024-09-21T07:41:15.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.321+0000 7effcb400640 1 --2- 172.21.15.39:0/3689089295 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7effd40d19f0 0x7effd40a1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:41:15.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.321+0000 7effd8e00640 1 --2- 172.21.15.39:0/3689089295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effd40d4b40 0x7effd40a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:41:15.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.322+0000 7effd3e00640 1 --2- 172.21.15.39:0/3689089295 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effd40b5890 0x7effd4157cc0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7effc4000c00 tx=0x7effc4005360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:15.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.322+0000 7effd0c00640 1 -- 172.21.15.39:0/3689089295 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc403f070 con 0x7effd40b5890 2024-09-21T07:41:15.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.322+0000 7effd0c00640 1 -- 172.21.15.39:0/3689089295 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7effc4004340 con 0x7effd40b5890 2024-09-21T07:41:15.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.322+0000 7effd0c00640 1 -- 172.21.15.39:0/3689089295 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc4031bb0 con 0x7effd40b5890 2024-09-21T07:41:15.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.322+0000 7effd9800640 1 -- 172.21.15.39:0/3689089295 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effd40c89f0 con 0x7effd40b5890 2024-09-21T07:41:15.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.322+0000 7effd9800640 1 -- 172.21.15.39:0/3689089295 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effd40c8fb0 con 0x7effd40b5890 2024-09-21T07:41:15.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.323+0000 7effd9800640 1 -- 172.21.15.39:0/3689089295 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effd4069a50 con 0x7effd40b5890 2024-09-21T07:41:15.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.325+0000 7effd0c00640 1 -- 172.21.15.39:0/3689089295 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7effc4042070 con 0x7effd40b5890 2024-09-21T07:41:15.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.325+0000 7effd0c00640 1 --2- 172.21.15.39:0/3689089295 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7effb0078bc0 0x7effb007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:15.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.326+0000 7effd0c00640 1 -- 172.21.15.39:0/3689089295 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7818+0+0 (secure 0 0 0) 0x7effc40bc5e0 con 0x7effd40b5890 2024-09-21T07:41:15.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.326+0000 7effcb400640 1 --2- 172.21.15.39:0/3689089295 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7effb0078bc0 0x7effb007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:15.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.327+0000 7effcb400640 1 --2- 172.21.15.39:0/3689089295 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7effb0078bc0 0x7effb007b0d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7effd4162030 tx=0x7effbc009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:15.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.330+0000 7effd0c00640 1 -- 172.21.15.39:0/3689089295 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7effc40841d0 con 0x7effd40b5890 2024-09-21T07:41:15.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.564+0000 7effd9800640 1 -- 172.21.15.39:0/3689089295 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7effd40c91b0 con 0x7effd40b5890 2024-09-21T07:41:15.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.564+0000 7effd0c00640 1 -- 172.21.15.39:0/3689089295 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) ==== 95+0+5240 (secure 0 0 0) 0x7effd40c91b0 con 0x7effd40b5890 2024-09-21T07:41:15.566 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:41:15.566 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":125,"btime":"2024-09-21T07:41:15:014294+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6425,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/73220122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":73220122},{"type":"v1","addr":"172.21.15.148:6837","nonce":73220122}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:41:14.026580+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:rejoin","state_seq":19,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7452":{"gid":7452,"name":"a","rank":1,"incarnation":122,"state":"up:rejoin","state_seq":7,"addr":"172.21.15.39:6835/3695489158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3695489158},{"type":"v1","addr":"172.21.15.39:6835","nonce":3695489158}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T07:41:15.567 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 125 2024-09-21T07:41:15.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.566+0000 7effd9800640 1 -- 172.21.15.39:0/3689089295 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7effb0078bc0 msgr2=0x7effb007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:15.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.566+0000 7effd9800640 1 --2- 172.21.15.39:0/3689089295 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7effb0078bc0 0x7effb007b0d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7effd4162030 tx=0x7effbc009040 comp rx=0 tx=0).stop 2024-09-21T07:41:15.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.566+0000 7effd9800640 1 -- 172.21.15.39:0/3689089295 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effd40b5890 msgr2=0x7effd4157cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:15.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.566+0000 7effd9800640 1 --2- 172.21.15.39:0/3689089295 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effd40b5890 0x7effd4157cc0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7effc4000c00 tx=0x7effc4005360 comp rx=0 tx=0).stop 2024-09-21T07:41:15.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.567+0000 7effd9800640 1 -- 172.21.15.39:0/3689089295 shutdown_connections 2024-09-21T07:41:15.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.567+0000 7effd9800640 1 --2- 172.21.15.39:0/3689089295 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7effb0078bc0 0x7effb007b0d0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:15.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.567+0000 7effd9800640 1 --2- 172.21.15.39:0/3689089295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7effd40d4b40 0x7effd40a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:15.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.567+0000 7effd9800640 1 --2- 172.21.15.39:0/3689089295 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7effd40d19f0 0x7effd40a1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:15.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.567+0000 7effd9800640 1 --2- 172.21.15.39:0/3689089295 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effd40b5890 0x7effd4157cc0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:15.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.567+0000 7effd9800640 1 -- 172.21.15.39:0/3689089295 >> 172.21.15.39:0/3689089295 conn(0x7effd408bfb0 msgr2=0x7effd40b94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:15.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.567+0000 7effd9800640 1 -- 172.21.15.39:0/3689089295 shutdown_connections 2024-09-21T07:41:15.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:15.567+0000 7effd9800640 1 -- 172.21.15.39:0/3689089295 wait complete. 2024-09-21T07:41:15.592 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:rejoin 2024-09-21T07:41:16.593 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:41:16.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.832+0000 7fa557e00640 1 -- 172.21.15.39:0/3685167048 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa55806f000 msgr2=0x7fa55806f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:16.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.832+0000 7fa557e00640 1 --2- 172.21.15.39:0/3685167048 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa55806f000 0x7fa55806f410 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fa5580710f0 tx=0x7fa55002a980 comp rx=0 tx=0).stop 2024-09-21T07:41:16.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.832+0000 7fa557e00640 1 -- 172.21.15.39:0/3685167048 shutdown_connections 2024-09-21T07:41:16.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.832+0000 7fa557e00640 1 --2- 172.21.15.39:0/3685167048 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa55806f000 0x7fa55806f410 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:16.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.832+0000 7fa557e00640 1 --2- 172.21.15.39:0/3685167048 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa5580ce310 0x7fa5580ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:16.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.832+0000 7fa557e00640 1 --2- 172.21.15.39:0/3685167048 >> v2:172.21.15.148:3300/0 conn(0x7fa5580cc390 0x7fa5580af600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:16.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.832+0000 7fa557e00640 1 -- 172.21.15.39:0/3685167048 >> 172.21.15.39:0/3685167048 conn(0x7fa5580b2510 msgr2=0x7fa5580d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:16.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.832+0000 7fa557e00640 1 -- 172.21.15.39:0/3685167048 shutdown_connections 2024-09-21T07:41:16.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.832+0000 7fa557e00640 1 -- 172.21.15.39:0/3685167048 wait complete. 2024-09-21T07:41:16.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.834+0000 7fa557e00640 1 Processor -- start 2024-09-21T07:41:16.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.835+0000 7fa557e00640 1 -- start start 2024-09-21T07:41:16.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.836+0000 7fa557e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa55806f000 0x7fa55815d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:16.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.836+0000 7fa557e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5580cc390 0x7fa55815c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:16.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.836+0000 7fa557e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa5580ce310 0x7fa55814dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:16.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.836+0000 7fa557e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa55806fe80 con 0x7fa5580cc390 2024-09-21T07:41:16.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.836+0000 7fa557e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa55806fd00 con 0x7fa55806f000 2024-09-21T07:41:16.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.836+0000 7fa557e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa558070000 con 0x7fa5580ce310 2024-09-21T07:41:16.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.836+0000 7fa556000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5580cc390 0x7fa55815c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:16.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.836+0000 7fa556000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5580cc390 0x7fa55815c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54898/0 (socket says 172.21.15.39:54898) 2024-09-21T07:41:16.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.836+0000 7fa556000640 1 -- 172.21.15.39:0/1244027854 learned_addr learned my addr 172.21.15.39:0/1244027854 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:41:16.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.837+0000 7fa556a00640 1 --2- 172.21.15.39:0/1244027854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa55806f000 0x7fa55815d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:16.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.837+0000 7fa557400640 1 --2- 172.21.15.39:0/1244027854 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa5580ce310 0x7fa55814dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:16.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.837+0000 7fa556000640 1 -- 172.21.15.39:0/1244027854 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa5580ce310 msgr2=0x7fa55814dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:16.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.837+0000 7fa556000640 1 --2- 172.21.15.39:0/1244027854 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa5580ce310 0x7fa55814dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:16.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.837+0000 7fa556000640 1 -- 172.21.15.39:0/1244027854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa55806f000 msgr2=0x7fa55815d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:16.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.837+0000 7fa556000640 1 --2- 172.21.15.39:0/1244027854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa55806f000 0x7fa55815d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:16.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.837+0000 7fa556000640 1 -- 172.21.15.39:0/1244027854 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5580cb7e0 con 0x7fa5580cc390 2024-09-21T07:41:16.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.837+0000 7fa556a00640 1 --2- 172.21.15.39:0/1244027854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa55806f000 0x7fa55815d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:41:16.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.837+0000 7fa556000640 1 --2- 172.21.15.39:0/1244027854 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5580cc390 0x7fa55815c130 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fa54400d6b0 tx=0x7fa54400db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:16.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.837+0000 7fa54ea00640 1 -- 172.21.15.39:0/1244027854 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa544017070 con 0x7fa5580cc390 2024-09-21T07:41:16.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.837+0000 7fa54ea00640 1 -- 172.21.15.39:0/1244027854 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa544005310 con 0x7fa5580cc390 2024-09-21T07:41:16.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.837+0000 7fa54ea00640 1 -- 172.21.15.39:0/1244027854 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5440097c0 con 0x7fa5580cc390 2024-09-21T07:41:16.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.837+0000 7fa557e00640 1 -- 172.21.15.39:0/1244027854 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa55815cb00 con 0x7fa5580cc390 2024-09-21T07:41:16.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.838+0000 7fa557e00640 1 -- 172.21.15.39:0/1244027854 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa55814eae0 con 0x7fa5580cc390 2024-09-21T07:41:16.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.838+0000 7fa557e00640 1 -- 172.21.15.39:0/1244027854 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5580b30d0 con 0x7fa5580cc390 2024-09-21T07:41:16.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.840+0000 7fa54ea00640 1 -- 172.21.15.39:0/1244027854 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa544009960 con 0x7fa5580cc390 2024-09-21T07:41:16.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.841+0000 7fa54ea00640 1 --2- 172.21.15.39:0/1244027854 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa538078b90 0x7fa53807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:16.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.841+0000 7fa54ea00640 1 -- 172.21.15.39:0/1244027854 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7818+0+0 (secure 0 0 0) 0x7fa54409b720 con 0x7fa5580cc390 2024-09-21T07:41:16.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.841+0000 7fa556a00640 1 --2- 172.21.15.39:0/1244027854 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa538078b90 0x7fa53807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:16.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.842+0000 7fa556a00640 1 --2- 172.21.15.39:0/1244027854 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa538078b90 0x7fa53807b0a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa54800bb40 tx=0x7fa548005e70 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:16.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:16.845+0000 7fa54ea00640 1 -- 172.21.15.39:0/1244027854 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa5580b30d0 con 0x7fa5580cc390 2024-09-21T07:41:17.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.084+0000 7fa557e00640 1 -- 172.21.15.39:0/1244027854 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5580ae6f0 con 0x7fa5580cc390 2024-09-21T07:41:17.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.085+0000 7fa54ea00640 1 -- 172.21.15.39:0/1244027854 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 126 v126) ==== 95+0+5252 (secure 0 0 0) 0x7fa544063280 con 0x7fa5580cc390 2024-09-21T07:41:17.087 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:41:17.087 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":126,"btime":"2024-09-21T07:41:16:016923+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6425,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/73220122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":73220122},{"type":"v1","addr":"172.21.15.148:6837","nonce":73220122}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":126,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:41:16.016912+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7452":{"gid":7452,"name":"a","rank":1,"incarnation":122,"state":"up:active","state_seq":8,"addr":"172.21.15.39:6835/3695489158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3695489158},{"type":"v1","addr":"172.21.15.39:6835","nonce":3695489158}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311,7452]},"id":6}]} 2024-09-21T07:41:17.087 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 126 2024-09-21T07:41:17.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.087+0000 7fa557e00640 1 -- 172.21.15.39:0/1244027854 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa538078b90 msgr2=0x7fa53807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:17.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.087+0000 7fa557e00640 1 --2- 172.21.15.39:0/1244027854 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa538078b90 0x7fa53807b0a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa54800bb40 tx=0x7fa548005e70 comp rx=0 tx=0).stop 2024-09-21T07:41:17.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.087+0000 7fa557e00640 1 -- 172.21.15.39:0/1244027854 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5580cc390 msgr2=0x7fa55815c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:17.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.087+0000 7fa557e00640 1 --2- 172.21.15.39:0/1244027854 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5580cc390 0x7fa55815c130 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fa54400d6b0 tx=0x7fa54400db70 comp rx=0 tx=0).stop 2024-09-21T07:41:17.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.088+0000 7fa557e00640 1 -- 172.21.15.39:0/1244027854 shutdown_connections 2024-09-21T07:41:17.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.088+0000 7fa557e00640 1 --2- 172.21.15.39:0/1244027854 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa538078b90 0x7fa53807b0a0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:17.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.088+0000 7fa557e00640 1 --2- 172.21.15.39:0/1244027854 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa5580ce310 0x7fa55814dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:17.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.088+0000 7fa557e00640 1 --2- 172.21.15.39:0/1244027854 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5580cc390 0x7fa55815c130 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:17.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.088+0000 7fa557e00640 1 --2- 172.21.15.39:0/1244027854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa55806f000 0x7fa55815d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:17.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.088+0000 7fa557e00640 1 -- 172.21.15.39:0/1244027854 >> 172.21.15.39:0/1244027854 conn(0x7fa5580b2510 msgr2=0x7fa5580b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:17.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.088+0000 7fa557e00640 1 -- 172.21.15.39:0/1244027854 shutdown_connections 2024-09-21T07:41:17.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.088+0000 7fa557e00640 1 -- 172.21.15.39:0/1244027854 wait complete. 2024-09-21T07:41:17.113 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:active 2024-09-21T07:41:17.113 DEBUG:tasks.cephfs.filesystem:reached state 'up:active' in 5.067479372024536s 2024-09-21T07:41:17.113 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:41:17.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:41:17.113 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-09-21T07:41:17.169 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:41:17.170 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:41:17.205 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:41:17.205 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:41:17.280 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:41:17.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.548+0000 7fc6db000640 1 -- 172.21.15.39:0/4136887370 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6d40cc390 msgr2=0x7fc6d40af600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:17.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.548+0000 7fc6db000640 1 --2- 172.21.15.39:0/4136887370 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6d40cc390 0x7fc6d40af600 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8005fd0 tx=0x7fc6c802a5f0 comp rx=0 tx=0).stop 2024-09-21T07:41:17.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.548+0000 7fc6db000640 1 -- 172.21.15.39:0/4136887370 shutdown_connections 2024-09-21T07:41:17.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.548+0000 7fc6db000640 1 --2- 172.21.15.39:0/4136887370 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6d40cc390 0x7fc6d40af600 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:17.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.548+0000 7fc6db000640 1 -- 172.21.15.39:0/4136887370 >> 172.21.15.39:0/4136887370 conn(0x7fc6d40b2510 msgr2=0x7fc6d40d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:17.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.548+0000 7fc6db000640 1 -- 172.21.15.39:0/4136887370 shutdown_connections 2024-09-21T07:41:17.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.548+0000 7fc6db000640 1 -- 172.21.15.39:0/4136887370 wait complete. 2024-09-21T07:41:17.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.550+0000 7fc6db000640 1 Processor -- start 2024-09-21T07:41:17.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.550+0000 7fc6db000640 1 -- start start 2024-09-21T07:41:17.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.552+0000 7fc6db000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc6d40ca890 0x7fc6d41f5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:17.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.552+0000 7fc6db000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6d40cc390 0x7fc6d415d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:17.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.552+0000 7fc6db000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6d40ce310 0x7fc6d40c6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:17.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.552+0000 7fc6db000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d40cb7c0 con 0x7fc6d40cc390 2024-09-21T07:41:17.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.552+0000 7fc6db000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d40cb640 con 0x7fc6d40ce310 2024-09-21T07:41:17.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.552+0000 7fc6db000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc6d40cb940 con 0x7fc6d40ca890 2024-09-21T07:41:17.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.552+0000 7fc6d9200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6d40cc390 0x7fc6d415d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:17.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.552+0000 7fc6d9200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6d40cc390 0x7fc6d415d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54908/0 (socket says 172.21.15.39:54908) 2024-09-21T07:41:17.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.552+0000 7fc6d9200640 1 -- 172.21.15.39:0/2015489915 learned_addr learned my addr 172.21.15.39:0/2015489915 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:41:17.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.552+0000 7fc6da600640 1 --2- 172.21.15.39:0/2015489915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6d40ce310 0x7fc6d40c6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:17.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.552+0000 7fc6d9c00640 1 --2- 172.21.15.39:0/2015489915 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc6d40ca890 0x7fc6d41f5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:17.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.552+0000 7fc6d9200640 1 -- 172.21.15.39:0/2015489915 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc6d40ca890 msgr2=0x7fc6d41f5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:17.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.552+0000 7fc6d9200640 1 --2- 172.21.15.39:0/2015489915 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc6d40ca890 0x7fc6d41f5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:17.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.552+0000 7fc6d9200640 1 -- 172.21.15.39:0/2015489915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6d40ce310 msgr2=0x7fc6d40c6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:17.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.552+0000 7fc6d9200640 1 --2- 172.21.15.39:0/2015489915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6d40ce310 0x7fc6d40c6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:17.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.552+0000 7fc6d9200640 1 -- 172.21.15.39:0/2015489915 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6d40c93d0 con 0x7fc6d40cc390 2024-09-21T07:41:17.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.552+0000 7fc6da600640 1 --2- 172.21.15.39:0/2015489915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6d40ce310 0x7fc6d40c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:41:17.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.553+0000 7fc6d9200640 1 --2- 172.21.15.39:0/2015489915 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6d40cc390 0x7fc6d415d1b0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fc6c000c370 tx=0x7fc6c000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:17.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.553+0000 7fc6d1600640 1 -- 172.21.15.39:0/2015489915 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c0017070 con 0x7fc6d40cc390 2024-09-21T07:41:17.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.553+0000 7fc6d1600640 1 -- 172.21.15.39:0/2015489915 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc6c00053c0 con 0x7fc6d40cc390 2024-09-21T07:41:17.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.553+0000 7fc6d1600640 1 -- 172.21.15.39:0/2015489915 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c0003850 con 0x7fc6d40cc390 2024-09-21T07:41:17.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.553+0000 7fc6db000640 1 -- 172.21.15.39:0/2015489915 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6d40c96b0 con 0x7fc6d40cc390 2024-09-21T07:41:17.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.553+0000 7fc6db000640 1 -- 172.21.15.39:0/2015489915 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6d40c9c70 con 0x7fc6d40cc390 2024-09-21T07:41:17.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.554+0000 7fc6db000640 1 -- 172.21.15.39:0/2015489915 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6d40cbe60 con 0x7fc6d40cc390 2024-09-21T07:41:17.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.556+0000 7fc6d1600640 1 -- 172.21.15.39:0/2015489915 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc6c001a070 con 0x7fc6d40cc390 2024-09-21T07:41:17.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.557+0000 7fc6d1600640 1 --2- 172.21.15.39:0/2015489915 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc6b4078b90 0x7fc6b407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:17.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.557+0000 7fc6d1600640 1 -- 172.21.15.39:0/2015489915 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7818+0+0 (secure 0 0 0) 0x7fc6c009a970 con 0x7fc6d40cc390 2024-09-21T07:41:17.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.557+0000 7fc6d9c00640 1 --2- 172.21.15.39:0/2015489915 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc6b4078b90 0x7fc6b407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:17.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.558+0000 7fc6d9c00640 1 --2- 172.21.15.39:0/2015489915 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc6b4078b90 0x7fc6b407b0a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8033000 tx=0x7fc6c8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:17.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.562+0000 7fc6d1600640 1 -- 172.21.15.39:0/2015489915 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc6c00624d0 con 0x7fc6d40cc390 2024-09-21T07:41:17.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.771+0000 7fc6db000640 1 -- 172.21.15.39:0/2015489915 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc6d40753d0 con 0x7fc6d40cc390 2024-09-21T07:41:17.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.772+0000 7fc6d1600640 1 -- 172.21.15.39:0/2015489915 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 23 entries v109) ==== 81+0+1389 (secure 0 0 0) 0x7fc6c0061dd0 con 0x7fc6d40cc390 2024-09-21T07:41:17.774 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/2336112936 2024-09-22T07:41:05.979253+0000 2024-09-21T07:41:17.774 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/2821446711 2024-09-22T07:40:57.372409+0000 2024-09-21T07:41:17.774 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/1456343023 2024-09-22T07:40:08.637993+0000 2024-09-21T07:41:17.775 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6833/1456343023 2024-09-22T07:40:08.637993+0000 2024-09-21T07:41:17.775 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/992188984 2024-09-22T07:40:00.778049+0000 2024-09-21T07:41:17.775 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/3356393880 2024-09-21T08:40:15.824973+0000 2024-09-21T07:41:17.775 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/992188984 2024-09-22T07:40:00.778049+0000 2024-09-21T07:41:17.775 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/1472430787 2024-09-22T07:37:58.212892+0000 2024-09-21T07:41:17.775 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/829716143 2024-09-22T07:37:01.259219+0000 2024-09-21T07:41:17.775 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/3026827614 2024-09-22T07:37:29.781472+0000 2024-09-21T07:41:17.775 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/3026827614 2024-09-22T07:37:29.781472+0000 2024-09-21T07:41:17.775 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/1472430787 2024-09-22T07:37:58.212892+0000 2024-09-21T07:41:17.775 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.1:0/90671776 2024-09-21T08:39:22.988045+0000 2024-09-21T07:41:17.775 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/2336112936 2024-09-22T07:41:05.979253+0000 2024-09-21T07:41:17.776 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/2821446711 2024-09-22T07:40:57.372409+0000 2024-09-21T07:41:17.776 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/2381504293 2024-09-22T07:38:32.207414+0000 2024-09-21T07:41:17.776 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/2381504293 2024-09-22T07:38:32.207414+0000 2024-09-21T07:41:17.776 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/855150775 2024-09-22T07:39:07.739045+0000 2024-09-21T07:41:17.776 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/829716143 2024-09-22T07:37:01.259219+0000 2024-09-21T07:41:17.776 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6833/364657398 2024-09-22T07:39:15.579613+0000 2024-09-21T07:41:17.776 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/855150775 2024-09-22T07:39:07.739045+0000 2024-09-21T07:41:17.776 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.1:0/177249410 2024-09-21T08:41:13.882767+0000 2024-09-21T07:41:17.776 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/364657398 2024-09-22T07:39:15.579613+0000 2024-09-21T07:41:17.777 INFO:teuthology.orchestra.run.smithi039.stderr:listed 23 entries 2024-09-21T07:41:17.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.774+0000 7fc6db000640 1 -- 172.21.15.39:0/2015489915 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc6b4078b90 msgr2=0x7fc6b407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:17.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.774+0000 7fc6db000640 1 --2- 172.21.15.39:0/2015489915 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc6b4078b90 0x7fc6b407b0a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8033000 tx=0x7fc6c8004730 comp rx=0 tx=0).stop 2024-09-21T07:41:17.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.774+0000 7fc6db000640 1 -- 172.21.15.39:0/2015489915 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6d40cc390 msgr2=0x7fc6d415d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:17.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.774+0000 7fc6db000640 1 --2- 172.21.15.39:0/2015489915 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6d40cc390 0x7fc6d415d1b0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fc6c000c370 tx=0x7fc6c000c830 comp rx=0 tx=0).stop 2024-09-21T07:41:17.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.774+0000 7fc6db000640 1 -- 172.21.15.39:0/2015489915 shutdown_connections 2024-09-21T07:41:17.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.774+0000 7fc6db000640 1 --2- 172.21.15.39:0/2015489915 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc6b4078b90 0x7fc6b407b0a0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:17.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.774+0000 7fc6db000640 1 --2- 172.21.15.39:0/2015489915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6d40ce310 0x7fc6d40c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:17.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.775+0000 7fc6db000640 1 --2- 172.21.15.39:0/2015489915 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6d40cc390 0x7fc6d415d1b0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:17.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.775+0000 7fc6db000640 1 --2- 172.21.15.39:0/2015489915 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc6d40ca890 0x7fc6d41f5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:17.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.775+0000 7fc6db000640 1 -- 172.21.15.39:0/2015489915 >> 172.21.15.39:0/2015489915 conn(0x7fc6d40b2510 msgr2=0x7fc6d40b0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:17.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.775+0000 7fc6db000640 1 -- 172.21.15.39:0/2015489915 shutdown_connections 2024-09-21T07:41:17.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:17.775+0000 7fc6db000640 1 -- 172.21.15.39:0/2015489915 wait complete. 2024-09-21T07:41:17.799 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T07:41:17.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:41:17.799 DEBUG:teuthology.orchestra.run.smithi039:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T07:41:19.669 INFO:tasks.ceph.mds.c.smithi039.stderr:2024-09-21T07:41:19.667+0000 7f9202600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:41:19.669 INFO:tasks.ceph.mds.c.smithi039.stderr:2024-09-21T07:41:19.667+0000 7f9202600640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:41:22.311 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T07:41:22.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:41:22.312 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:41:22.351 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:41:22.351 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:41:22.428 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:41:22.428 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:41:22.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.696+0000 7f1403200640 1 -- 172.21.15.39:0/2392130894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14040cc500 msgr2=0x7f14040af590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:22.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.696+0000 7f1403200640 1 --2- 172.21.15.39:0/2392130894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14040cc500 0x7f14040af590 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f13f4009830 tx=0x7f13f402a510 comp rx=0 tx=0).stop 2024-09-21T07:41:22.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.696+0000 7f1403200640 1 -- 172.21.15.39:0/2392130894 shutdown_connections 2024-09-21T07:41:22.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.696+0000 7f1403200640 1 --2- 172.21.15.39:0/2392130894 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f14040d0e70 0x7f14040cca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:22.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.696+0000 7f1403200640 1 --2- 172.21.15.39:0/2392130894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14040cc500 0x7f14040af590 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:22.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.696+0000 7f1403200640 1 -- 172.21.15.39:0/2392130894 >> 172.21.15.39:0/2392130894 conn(0x7f14040b2510 msgr2=0x7f14040d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:22.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.696+0000 7f1403200640 1 -- 172.21.15.39:0/2392130894 shutdown_connections 2024-09-21T07:41:22.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.696+0000 7f1403200640 1 -- 172.21.15.39:0/2392130894 wait complete. 2024-09-21T07:41:22.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.698+0000 7f1403200640 1 Processor -- start 2024-09-21T07:41:22.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.699+0000 7f1403200640 1 -- start start 2024-09-21T07:41:22.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.700+0000 7f1403200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14040cc500 0x7f140415c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:22.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.700+0000 7f1403200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14040ccf90 0x7f1404161940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:22.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.700+0000 7f1403200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f14040d0e70 0x7f14040b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:22.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.700+0000 7f1403200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f140406faf0 con 0x7f14040cc500 2024-09-21T07:41:22.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.700+0000 7f1403200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f140406fc70 con 0x7f14040ccf90 2024-09-21T07:41:22.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.700+0000 7f1403200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f140406fdf0 con 0x7f14040d0e70 2024-09-21T07:41:22.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.700+0000 7f1401e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14040cc500 0x7f140415c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:22.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.700+0000 7f1401e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14040cc500 0x7f140415c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39532/0 (socket says 172.21.15.39:39532) 2024-09-21T07:41:22.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.700+0000 7f1401e00640 1 -- 172.21.15.39:0/2872280116 learned_addr learned my addr 172.21.15.39:0/2872280116 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:41:22.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.700+0000 7f13fbe00640 1 --2- 172.21.15.39:0/2872280116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14040ccf90 0x7f1404161940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:22.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.700+0000 7f1402800640 1 --2- 172.21.15.39:0/2872280116 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f14040d0e70 0x7f14040b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:22.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.701+0000 7f1401e00640 1 -- 172.21.15.39:0/2872280116 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f14040d0e70 msgr2=0x7f14040b0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:22.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.701+0000 7f1401e00640 1 --2- 172.21.15.39:0/2872280116 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f14040d0e70 0x7f14040b0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:22.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.701+0000 7f1401e00640 1 -- 172.21.15.39:0/2872280116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14040ccf90 msgr2=0x7f1404161940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:22.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.701+0000 7f1401e00640 1 --2- 172.21.15.39:0/2872280116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14040ccf90 0x7f1404161940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:22.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.701+0000 7f1401e00640 1 -- 172.21.15.39:0/2872280116 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1404161e80 con 0x7f14040cc500 2024-09-21T07:41:22.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.701+0000 7f1401e00640 1 --2- 172.21.15.39:0/2872280116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14040cc500 0x7f140415c090 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f13f4009950 tx=0x7f13f4033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:22.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.701+0000 7f13fa000640 1 -- 172.21.15.39:0/2872280116 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f4031d60 con 0x7f14040cc500 2024-09-21T07:41:22.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.701+0000 7f13fa000640 1 -- 172.21.15.39:0/2872280116 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f13f4033e10 con 0x7f14040cc500 2024-09-21T07:41:22.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.701+0000 7f13fa000640 1 -- 172.21.15.39:0/2872280116 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f4005260 con 0x7f14040cc500 2024-09-21T07:41:22.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.702+0000 7f1403200640 1 -- 172.21.15.39:0/2872280116 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1404160270 con 0x7f14040cc500 2024-09-21T07:41:22.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.702+0000 7f1403200640 1 -- 172.21.15.39:0/2872280116 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1404160830 con 0x7f14040cc500 2024-09-21T07:41:22.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.703+0000 7f1403200640 1 -- 172.21.15.39:0/2872280116 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14040702a0 con 0x7f14040cc500 2024-09-21T07:41:22.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.704+0000 7f13fa000640 1 -- 172.21.15.39:0/2872280116 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f13f4005400 con 0x7f14040cc500 2024-09-21T07:41:22.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.705+0000 7f13fa000640 1 --2- 172.21.15.39:0/2872280116 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f13e0078b90 0x7f13e007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:22.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.705+0000 7f13fa000640 1 -- 172.21.15.39:0/2872280116 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7818+0+0 (secure 0 0 0) 0x7f13f40bc820 con 0x7f14040cc500 2024-09-21T07:41:22.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.705+0000 7f13fbe00640 1 --2- 172.21.15.39:0/2872280116 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f13e0078b90 0x7f13e007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:22.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.706+0000 7f13fbe00640 1 --2- 172.21.15.39:0/2872280116 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f13e0078b90 0x7f13e007b0a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1404161440 tx=0x7f13ec00b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:22.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.709+0000 7f13fa000640 1 -- 172.21.15.39:0/2872280116 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f13f403f030 con 0x7f14040cc500 2024-09-21T07:41:22.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.951+0000 7f1403200640 1 -- 172.21.15.39:0/2872280116 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1404050ff0 con 0x7f14040cc500 2024-09-21T07:41:22.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.951+0000 7f13fa000640 1 -- 172.21.15.39:0/2872280116 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v126) ==== 53+0+73 (secure 0 0 0) 0x7f13f4005d00 con 0x7f14040cc500 2024-09-21T07:41:22.953 INFO:teuthology.orchestra.run.smithi039.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:41:22.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.954+0000 7f1403200640 1 -- 172.21.15.39:0/2872280116 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f13e0078b90 msgr2=0x7f13e007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:22.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.954+0000 7f1403200640 1 --2- 172.21.15.39:0/2872280116 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f13e0078b90 0x7f13e007b0a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1404161440 tx=0x7f13ec00b2e0 comp rx=0 tx=0).stop 2024-09-21T07:41:22.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.954+0000 7f1403200640 1 -- 172.21.15.39:0/2872280116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14040cc500 msgr2=0x7f140415c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:22.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.954+0000 7f1403200640 1 --2- 172.21.15.39:0/2872280116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14040cc500 0x7f140415c090 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f13f4009950 tx=0x7f13f4033c20 comp rx=0 tx=0).stop 2024-09-21T07:41:22.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.954+0000 7f1403200640 1 -- 172.21.15.39:0/2872280116 shutdown_connections 2024-09-21T07:41:22.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.954+0000 7f1403200640 1 --2- 172.21.15.39:0/2872280116 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f13e0078b90 0x7f13e007b0a0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:22.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.954+0000 7f1403200640 1 --2- 172.21.15.39:0/2872280116 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f14040d0e70 0x7f14040b0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:22.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.954+0000 7f1403200640 1 --2- 172.21.15.39:0/2872280116 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14040ccf90 0x7f1404161940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:22.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.954+0000 7f1403200640 1 --2- 172.21.15.39:0/2872280116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14040cc500 0x7f140415c090 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:22.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.954+0000 7f1403200640 1 -- 172.21.15.39:0/2872280116 >> 172.21.15.39:0/2872280116 conn(0x7f14040b2510 msgr2=0x7f14040ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:22.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.954+0000 7f1403200640 1 -- 172.21.15.39:0/2872280116 shutdown_connections 2024-09-21T07:41:22.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:22.954+0000 7f1403200640 1 -- 172.21.15.39:0/2872280116 wait complete. 2024-09-21T07:41:22.982 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:41:22.982 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:41:22.982 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi039.front.sepia.ceph.com 2024-09-21T07:41:22.982 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T07:41:22.982 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:41:22.982 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:41:22.982 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:41:22.982 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:41:22.982 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:41:22.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:41:22.983 DEBUG:teuthology.orchestra.run.smithi039:> ip addr 2024-09-21T07:41:23.019 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:41:23.020 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:41:23.020 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:41:23.020 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:41:23.020 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2024-09-21T07:41:23.020 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:41:23.021 INFO:teuthology.orchestra.run.smithi039.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:41:23.021 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:41:23.021 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f0 2024-09-21T07:41:23.021 INFO:teuthology.orchestra.run.smithi039.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:41:23.021 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c7 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:41:23.021 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f1 2024-09-21T07:41:23.022 INFO:teuthology.orchestra.run.smithi039.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:41:23.022 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:70 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:41:23.022 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f0 2024-09-21T07:41:23.022 INFO:teuthology.orchestra.run.smithi039.stdout: inet 172.21.15.39/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T07:41:23.022 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft 42236sec preferred_lft 42236sec 2024-09-21T07:41:23.022 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec4:7aff:febd:1370/64 scope link 2024-09-21T07:41:23.023 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:41:23.023 INFO:teuthology.orchestra.run.smithi039.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:41:23.023 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:71 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:41:23.023 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f1 2024-09-21T07:41:23.023 INFO:teuthology.orchestra.run.smithi039.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:41:23.023 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff 2024-09-21T07:41:23.024 INFO:teuthology.orchestra.run.smithi039.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:41:23.024 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:41:23.024 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::b079:ddff:fe0e:6333/64 scope link 2024-09-21T07:41:23.024 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:41:23.024 INFO:teuthology.orchestra.run.smithi039.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:41:23.024 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:41:23.024 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::2ca7:98ff:fe8a:d0ed/64 scope link 2024-09-21T07:41:23.025 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:41:23.025 INFO:teuthology.orchestra.run.smithi039.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:41:23.025 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether ee:10:53:01:0c:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:41:23.025 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec10:53ff:fe01:c44/64 scope link 2024-09-21T07:41:23.025 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:41:23.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:41:23.026 DEBUG:teuthology.orchestra.run.smithi039:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:41:23.114 INFO:teuthology.orchestra.run.smithi039.stdout:1 2024-09-21T07:41:23.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:41:23.117 DEBUG:teuthology.orchestra.run.smithi039:> route 2024-09-21T07:41:23.153 INFO:teuthology.orchestra.run.smithi039.stdout:Kernel IP routing table 2024-09-21T07:41:23.153 INFO:teuthology.orchestra.run.smithi039.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:41:23.153 INFO:teuthology.orchestra.run.smithi039.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T07:41:23.153 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T07:41:23.153 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:41:23.154 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:41:23.154 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-21T07:41:23.154 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T07:41:23.154 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T07:41:23.154 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T07:41:23.154 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-21T07:41:23.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:41:23.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:41:23.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:41:23.351 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-09-21T07:41:23.386 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:41:23.387 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T07:41:23.387 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:41:23.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:41:23.387 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-09-21T07:41:23.483 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:41:23.483 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T07:41:23.518 INFO:teuthology.orchestra.run.smithi039.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:41:23.518 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:41:23.518 DEBUG:teuthology.orchestra.run.smithi039:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:41:23.593 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-21T07:41:23.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:41:23.594 DEBUG:teuthology.orchestra.run.smithi039:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:41:23.793 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:41:23.793 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:41:23.830 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:41:23.830 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:41:23.830 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: resolved to: "172.21.15.39:3300,172.21.15.148:3300,172.21.15.148:3301" 2024-09-21T07:41:23.830 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: trying mount with new device syntax: 0@7ab9b04a-f88d-49aa-a493-e5c076637ffb.cephfs=/ 2024-09-21T07:41:23.830 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.39:3300/172.21.15.148:3300/172.21.15.148:3301" will pass to kernel 2024-09-21T07:41:23.832 INFO:teuthology.orchestra.run.smithi039.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T07:41:23.832 INFO:teuthology.orchestra.run.smithi039.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:41:23.832 INFO:teuthology.orchestra.run.smithi039.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:41:23.832 INFO:teuthology.orchestra.run.smithi039.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:41:23.832 INFO:teuthology.orchestra.run.smithi039.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:41:23.835 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:41:23.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:41:23.835 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:41:23.892 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:41:23.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:41:23.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:41:23.986 INFO:teuthology.orchestra.run.smithi039.stdout:client6458 2024-09-21T07:41:23.986 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:41:23.986 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:41:24.022 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 flush journal 2024-09-21T07:41:24.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.291+0000 7f5b2c000640 1 -- 172.21.15.39:0/309497312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b240cce70 msgr2=0x7f5b240cd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:24.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.291+0000 7f5b2c000640 1 --2- 172.21.15.39:0/309497312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b240cce70 0x7f5b240cd280 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c005fd0 tx=0x7f5b1c02a820 comp rx=0 tx=0).stop 2024-09-21T07:41:24.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.291+0000 7f5b2c000640 1 -- 172.21.15.39:0/309497312 shutdown_connections 2024-09-21T07:41:24.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.291+0000 7f5b2c000640 1 --2- 172.21.15.39:0/309497312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b240cce70 0x7f5b240cd280 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.291+0000 7f5b2c000640 1 --2- 172.21.15.39:0/309497312 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b2415cf60 0x7f5b240b32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.291+0000 7f5b2c000640 1 -- 172.21.15.39:0/309497312 >> 172.21.15.39:0/309497312 conn(0x7f5b2406d440 msgr2=0x7f5b240b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:24.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.291+0000 7f5b2c000640 1 -- 172.21.15.39:0/309497312 shutdown_connections 2024-09-21T07:41:24.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.291+0000 7f5b2c000640 1 -- 172.21.15.39:0/309497312 wait complete. 2024-09-21T07:41:24.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.293+0000 7f5b2c000640 1 Processor -- start 2024-09-21T07:41:24.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.294+0000 7f5b2c000640 1 -- start start 2024-09-21T07:41:24.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.295+0000 7f5b2c000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b240cce70 0x7f5b240c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:24.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.295+0000 7f5b2c000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b2415cf60 0x7f5b2409daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:24.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.295+0000 7f5b2c000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b2415d950 0x7f5b241f5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:24.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.295+0000 7f5b2c000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5b2406fa20 con 0x7f5b2415cf60 2024-09-21T07:41:24.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.295+0000 7f5b2c000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5b2406f8a0 con 0x7f5b240cce70 2024-09-21T07:41:24.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.295+0000 7f5b2c000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f5b2406fba0 con 0x7f5b2415d950 2024-09-21T07:41:24.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.295+0000 7f5b28c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b2415cf60 0x7f5b2409daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:24.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.295+0000 7f5b28c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b2415cf60 0x7f5b2409daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39550/0 (socket says 172.21.15.39:39550) 2024-09-21T07:41:24.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.295+0000 7f5b28c00640 1 -- 172.21.15.39:0/1376635975 learned_addr learned my addr 172.21.15.39:0/1376635975 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:41:24.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.295+0000 7f5b29600640 1 --2- 172.21.15.39:0/1376635975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b240cce70 0x7f5b240c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:24.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.296+0000 7f5b2a000640 1 --2- 172.21.15.39:0/1376635975 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b2415d950 0x7f5b241f5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:24.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.296+0000 7f5b28c00640 1 -- 172.21.15.39:0/1376635975 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b2415d950 msgr2=0x7f5b241f5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:24.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.296+0000 7f5b28c00640 1 --2- 172.21.15.39:0/1376635975 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b2415d950 0x7f5b241f5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.296+0000 7f5b28c00640 1 -- 172.21.15.39:0/1376635975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b240cce70 msgr2=0x7f5b240c7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:24.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.296+0000 7f5b28c00640 1 --2- 172.21.15.39:0/1376635975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b240cce70 0x7f5b240c7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.296+0000 7f5b28c00640 1 -- 172.21.15.39:0/1376635975 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b240c8160 con 0x7f5b2415cf60 2024-09-21T07:41:24.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.296+0000 7f5b28c00640 1 --2- 172.21.15.39:0/1376635975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b2415cf60 0x7f5b2409daf0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f5b1000c730 tx=0x7f5b1000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:24.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.296+0000 7f5b21600640 1 -- 172.21.15.39:0/1376635975 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b10016020 con 0x7f5b2415cf60 2024-09-21T07:41:24.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.296+0000 7f5b21600640 1 -- 172.21.15.39:0/1376635975 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b1000a950 con 0x7f5b2415cf60 2024-09-21T07:41:24.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.297+0000 7f5b21600640 1 -- 172.21.15.39:0/1376635975 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1000ada0 con 0x7f5b2415cf60 2024-09-21T07:41:24.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.297+0000 7f5b2c000640 1 -- 172.21.15.39:0/1376635975 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b240c3e60 con 0x7f5b2415cf60 2024-09-21T07:41:24.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.297+0000 7f5b2c000640 1 -- 172.21.15.39:0/1376635975 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b240c4420 con 0x7f5b2415cf60 2024-09-21T07:41:24.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.299+0000 7f5b21600640 1 -- 172.21.15.39:0/1376635975 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5b10019020 con 0x7f5b2415cf60 2024-09-21T07:41:24.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.300+0000 7f5b21600640 1 --2- 172.21.15.39:0/1376635975 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5b04078b90 0x7f5b0407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:24.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.300+0000 7f5b21600640 1 -- 172.21.15.39:0/1376635975 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7818+0+0 (secure 0 0 0) 0x7f5b1009aff0 con 0x7f5b2415cf60 2024-09-21T07:41:24.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.300+0000 7f5b29600640 1 --2- 172.21.15.39:0/1376635975 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5b04078b90 0x7f5b0407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:24.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.301+0000 7f5b32ffb740 1 Processor -- start 2024-09-21T07:41:24.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.301+0000 7f5b32ffb740 1 -- start start 2024-09-21T07:41:24.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.301+0000 7f5b29600640 1 --2- 172.21.15.39:0/1376635975 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5b04078b90 0x7f5b0407b0a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f5b18005fd0 tx=0x7f5b1800b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:24.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.302+0000 7f5b32ffb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562dc69f390 0x5562dca8d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:24.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.302+0000 7f5b32ffb740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5562dc6764d0 0x5562dca70220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:24.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.302+0000 7f5b32ffb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562dc67e6b0 0x5562dca70760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:24.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.302+0000 7f5b32ffb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5562dc9a61c0 con 0x5562dc67e6b0 2024-09-21T07:41:24.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.302+0000 7f5b32ffb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5562dc9b9870 con 0x5562dc69f390 2024-09-21T07:41:24.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.302+0000 7f5b32ffb740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x5562dc9a8ca0 con 0x5562dc6764d0 2024-09-21T07:41:24.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.302+0000 7f5b2a000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562dc67e6b0 0x5562dca70760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:24.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.302+0000 7f5b2a000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562dc67e6b0 0x5562dca70760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39560/0 (socket says 172.21.15.39:39560) 2024-09-21T07:41:24.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.302+0000 7f5b2a000640 1 -- 172.21.15.39:0/491309842 learned_addr learned my addr 172.21.15.39:0/491309842 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:41:24.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.303+0000 7f5b29600640 1 --2- 172.21.15.39:0/491309842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562dc69f390 0x5562dca8d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:24.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.303+0000 7f5b28c00640 1 --2- 172.21.15.39:0/491309842 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5562dc6764d0 0x5562dca70220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:24.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.303+0000 7f5b2a000640 1 -- 172.21.15.39:0/491309842 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5562dc6764d0 msgr2=0x5562dca70220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:41:24.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.303+0000 7f5b2a000640 1 --2- 172.21.15.39:0/491309842 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5562dc6764d0 0x5562dca70220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.303+0000 7f5b2a000640 1 -- 172.21.15.39:0/491309842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562dc69f390 msgr2=0x5562dca8d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:41:24.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.303+0000 7f5b2a000640 1 --2- 172.21.15.39:0/491309842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562dc69f390 0x5562dca8d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.303+0000 7f5b2a000640 1 -- 172.21.15.39:0/491309842 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562dc7817d0 con 0x5562dc67e6b0 2024-09-21T07:41:24.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.303+0000 7f5b2a000640 1 --2- 172.21.15.39:0/491309842 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562dc67e6b0 0x5562dca70760 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c02ae10 tx=0x7f5b1c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0fdf06f3b73a389 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:24.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.303+0000 7f5b15c00640 1 -- 172.21.15.39:0/491309842 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c042070 con 0x5562dc67e6b0 2024-09-21T07:41:24.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.303+0000 7f5b15c00640 1 -- 172.21.15.39:0/491309842 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b1c003be0 con 0x5562dc67e6b0 2024-09-21T07:41:24.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.304+0000 7f5b15c00640 1 -- 172.21.15.39:0/491309842 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c037040 con 0x5562dc67e6b0 2024-09-21T07:41:24.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.304+0000 7f5b32ffb740 1 -- 172.21.15.39:0/491309842 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562dc67e6b0 msgr2=0x5562dca70760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:24.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.304+0000 7f5b32ffb740 1 --2- 172.21.15.39:0/491309842 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562dc67e6b0 0x5562dca70760 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c02ae10 tx=0x7f5b1c004a40 comp rx=0 tx=0).stop 2024-09-21T07:41:24.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.304+0000 7f5b32ffb740 1 -- 172.21.15.39:0/491309842 shutdown_connections 2024-09-21T07:41:24.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.304+0000 7f5b32ffb740 1 --2- 172.21.15.39:0/491309842 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562dc67e6b0 0x5562dca70760 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.304+0000 7f5b32ffb740 1 --2- 172.21.15.39:0/491309842 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5562dc6764d0 0x5562dca70220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.304+0000 7f5b32ffb740 1 --2- 172.21.15.39:0/491309842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562dc69f390 0x5562dca8d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.304+0000 7f5b32ffb740 1 -- 172.21.15.39:0/491309842 >> 172.21.15.39:0/491309842 conn(0x5562dc6934e0 msgr2=0x5562dca46440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:24.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.304+0000 7f5b32ffb740 1 -- 172.21.15.39:0/491309842 shutdown_connections 2024-09-21T07:41:24.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.304+0000 7f5b32ffb740 1 -- 172.21.15.39:0/491309842 wait complete. 2024-09-21T07:41:24.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.305+0000 7f5b32ffb740 1 -- start start 2024-09-21T07:41:24.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.306+0000 7f5b32ffb740 1 Processor -- start 2024-09-21T07:41:24.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.307+0000 7f5b32ffb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562dc6764d0 0x5562dca86ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:24.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.307+0000 7f5b32ffb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562dc67e6b0 0x5562dca45a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:24.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.307+0000 7f5b32ffb740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5562dc69f390 0x5562dca45fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:24.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.307+0000 7f5b32ffb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5562dc94f410 con 0x5562dc6764d0 2024-09-21T07:41:24.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.307+0000 7f5b32ffb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5562dc9acab0 con 0x5562dc67e6b0 2024-09-21T07:41:24.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.307+0000 7f5b32ffb740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x5562dc719740 con 0x5562dc69f390 2024-09-21T07:41:24.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.307+0000 7f5b32ffb740 10 client.0 _ll_register_callbacks cb 0x5562dc92c360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:41:24.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.308+0000 7f5b29600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562dc6764d0 0x5562dca86ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:24.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.308+0000 7f5b29600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562dc6764d0 0x5562dca86ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39562/0 (socket says 172.21.15.39:39562) 2024-09-21T07:41:24.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.308+0000 7f5b29600640 1 -- 172.21.15.39:0/419156389 learned_addr learned my addr 172.21.15.39:0/419156389 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:41:24.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.308+0000 7f5b2a000640 1 --2- 172.21.15.39:0/419156389 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5562dc69f390 0x5562dca45fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:24.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.308+0000 7f5b28c00640 1 --2- 172.21.15.39:0/419156389 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562dc67e6b0 0x5562dca45a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:24.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.308+0000 7f5b29600640 1 -- 172.21.15.39:0/419156389 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5562dc69f390 msgr2=0x5562dca45fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:41:24.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.308+0000 7f5b29600640 1 --2- 172.21.15.39:0/419156389 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5562dc69f390 0x5562dca45fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.308+0000 7f5b29600640 1 -- 172.21.15.39:0/419156389 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562dc67e6b0 msgr2=0x5562dca45a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:41:24.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.308+0000 7f5b29600640 1 --2- 172.21.15.39:0/419156389 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562dc67e6b0 0x5562dca45a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.308+0000 7f5b29600640 1 -- 172.21.15.39:0/419156389 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562dc70a970 con 0x5562dc6764d0 2024-09-21T07:41:24.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.308+0000 7f5b29600640 1 --2- 172.21.15.39:0/419156389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562dc6764d0 0x5562dca86ce0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f5b180116f0 tx=0x7f5b18004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c818c47383b341a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:24.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.308+0000 7f5b0a000640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:41:24.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.309+0000 7f5b0a000640 1 -- 172.21.15.39:0/419156389 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1801e700 con 0x5562dc6764d0 2024-09-21T07:41:24.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.309+0000 7f5b32ffb740 1 -- 172.21.15.39:0/419156389 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b1c003be0 con 0x5562dc6764d0 2024-09-21T07:41:24.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.309+0000 7f5b0a000640 1 -- 172.21.15.39:0/419156389 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 7818+0+0 (secure 0 0 0) 0x7f5b1801e8a0 con 0x5562dc6764d0 2024-09-21T07:41:24.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.309+0000 7f5b0a000640 1 -- 172.21.15.39:0/419156389 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=126) ==== 24+0+0 (secure 0 0 0) 0x7f5b1801eaa0 con 0x5562dc6764d0 2024-09-21T07:41:24.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.309+0000 7f5b32ffb740 10 client.7569 fetch_fsmap learned FSMap version 126 2024-09-21T07:41:24.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.309+0000 7f5b32ffb740 1 -- 172.21.15.39:0/419156389 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=126,monmap=2+}) -- 0x5562dc6e86a0 con 0x5562dc6764d0 2024-09-21T07:41:24.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.309+0000 7f5b0a000640 1 -- 172.21.15.39:0/419156389 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b1c003be0 con 0x5562dc6764d0 2024-09-21T07:41:24.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.310+0000 7f5b0a000640 1 -- 172.21.15.39:0/419156389 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 126) ==== 3522+0+0 (secure 0 0 0) 0x7f5b1801ec40 con 0x5562dc6764d0 2024-09-21T07:41:24.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.310+0000 7f5b32ffb740 10 client.7569 fetch_fsmap finished waiting for FSMap version 126 2024-09-21T07:41:24.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.310+0000 7f5b32ffb740 10 client.7569 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-21T07:41:24.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.310+0000 7f5b32ffb740 20 client.7569 populate_metadata read hostname 'smithi039' 2024-09-21T07:41:24.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.310+0000 7f5b32ffb740 1 --2- 172.21.15.39:0/419156389 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x5562dc65df60 0x5562dca855f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:24.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.310+0000 7f5b32ffb740 4 client.7569 mds_command: new command op to 7311 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:41:24.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.310+0000 7f5b32ffb740 1 -- 172.21.15.39:0/419156389 --> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562dc9bb7c0 con 0x5562dc65df60 2024-09-21T07:41:24.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.310+0000 7f5b28c00640 1 --2- 172.21.15.39:0/419156389 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x5562dc65df60 0x5562dca855f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:24.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.310+0000 7f5b28c00640 1 --2- 172.21.15.39:0/419156389 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x5562dc65df60 0x5562dca855f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39eb5e162f2f74b7 server_cookie=415a6a41a8838511 in_seq=0 out_seq=0 2024-09-21T07:41:24.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.310+0000 7f5b0a000640 10 client.7569 ms_handle_connect on v2:172.21.15.39:6832/1790581246 2024-09-21T07:41:24.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.311+0000 7f5b0a000640 1 -- 172.21.15.39:0/419156389 <== mds.0 v2:172.21.15.39:6832/1790581246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5562dc9bb7c0 con 0x5562dc65df60 2024-09-21T07:41:24.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.311+0000 7f5b0a000640 10 client.7569 handle_command_reply: tid=0 2024-09-21T07:41:24.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.311+0000 7f5b32ffb740 1 client.7569 shutdown 2024-09-21T07:41:24.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.311+0000 7f5b32ffb740 1 -- 172.21.15.39:0/419156389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562dc6764d0 msgr2=0x5562dca86ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:24.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.311+0000 7f5b32ffb740 1 --2- 172.21.15.39:0/419156389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562dc6764d0 0x5562dca86ce0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f5b180116f0 tx=0x7f5b18004730 comp rx=0 tx=0).stop 2024-09-21T07:41:24.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.311+0000 7f5b32ffb740 1 -- 172.21.15.39:0/419156389 shutdown_connections 2024-09-21T07:41:24.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.311+0000 7f5b32ffb740 1 --2- 172.21.15.39:0/419156389 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5562dc69f390 0x5562dca45fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.311+0000 7f5b32ffb740 1 --2- 172.21.15.39:0/419156389 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x5562dc65df60 0x5562dca855f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.311+0000 7f5b32ffb740 1 --2- 172.21.15.39:0/419156389 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562dc67e6b0 0x5562dca45a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.311+0000 7f5b32ffb740 1 --2- 172.21.15.39:0/419156389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562dc6764d0 0x5562dca86ce0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.312+0000 7f5b32ffb740 1 -- 172.21.15.39:0/419156389 >> 172.21.15.39:0/419156389 conn(0x5562dc6934e0 msgr2=0x5562dca6d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:24.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.312+0000 7f5b0a000640 0 client.7569 ms_handle_reset on v2:172.21.15.39:6832/1790581246 2024-09-21T07:41:24.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.312+0000 7f5b32ffb740 1 -- 172.21.15.39:0/419156389 shutdown_connections 2024-09-21T07:41:24.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.312+0000 7f5b32ffb740 1 -- 172.21.15.39:0/419156389 wait complete. 2024-09-21T07:41:24.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.312+0000 7f5b32ffb740 20 client.7569 trim_cache size 0 max 16384 2024-09-21T07:41:24.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.327+0000 7f5b32ffb740 1 Processor -- start 2024-09-21T07:41:24.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.327+0000 7f5b32ffb740 1 -- start start 2024-09-21T07:41:24.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.329+0000 7f5b32ffb740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5562dc6764d0 0x5562dca47310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:24.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.329+0000 7f5b32ffb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562dc67e6b0 0x5562dca5fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:24.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.329+0000 7f5b32ffb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562dc69f390 0x5562dca600a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:24.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.329+0000 7f5b32ffb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5b24200800 con 0x5562dc67e6b0 2024-09-21T07:41:24.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.329+0000 7f5b32ffb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5b24200ed0 con 0x5562dc69f390 2024-09-21T07:41:24.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.329+0000 7f5b32ffb740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f5b240cfe30 con 0x5562dc6764d0 2024-09-21T07:41:24.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.329+0000 7f5b28c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562dc67e6b0 0x5562dca5fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:24.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.329+0000 7f5b28c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562dc67e6b0 0x5562dca5fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39570/0 (socket says 172.21.15.39:39570) 2024-09-21T07:41:24.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.329+0000 7f5b28c00640 1 -- 172.21.15.39:0/1250748317 learned_addr learned my addr 172.21.15.39:0/1250748317 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:41:24.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.329+0000 7f5b29600640 1 --2- 172.21.15.39:0/1250748317 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5562dc6764d0 0x5562dca47310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:24.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.329+0000 7f5b2a000640 1 --2- 172.21.15.39:0/1250748317 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562dc69f390 0x5562dca600a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:24.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.330+0000 7f5b28c00640 1 -- 172.21.15.39:0/1250748317 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5562dc6764d0 msgr2=0x5562dca47310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:41:24.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.330+0000 7f5b28c00640 1 --2- 172.21.15.39:0/1250748317 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5562dc6764d0 0x5562dca47310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.330+0000 7f5b28c00640 1 -- 172.21.15.39:0/1250748317 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562dc69f390 msgr2=0x5562dca600a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:41:24.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.330+0000 7f5b28c00640 1 --2- 172.21.15.39:0/1250748317 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562dc69f390 0x5562dca600a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.330+0000 7f5b28c00640 1 -- 172.21.15.39:0/1250748317 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562dca6d6c0 con 0x5562dc67e6b0 2024-09-21T07:41:24.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.330+0000 7f5b28c00640 1 --2- 172.21.15.39:0/1250748317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562dc67e6b0 0x5562dca5fb60 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f5b1005f810 tx=0x7f5b10098bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d1bf8a0fa8241 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:24.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.330+0000 7f5b15200640 1 -- 172.21.15.39:0/1250748317 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1006bf10 con 0x5562dc67e6b0 2024-09-21T07:41:24.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.330+0000 7f5b15200640 1 -- 172.21.15.39:0/1250748317 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b10098df0 con 0x5562dc67e6b0 2024-09-21T07:41:24.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.330+0000 7f5b15200640 1 -- 172.21.15.39:0/1250748317 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b10061800 con 0x5562dc67e6b0 2024-09-21T07:41:24.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.331+0000 7f5b32ffb740 1 -- 172.21.15.39:0/1250748317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562dc67e6b0 msgr2=0x5562dca5fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:24.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.331+0000 7f5b32ffb740 1 --2- 172.21.15.39:0/1250748317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562dc67e6b0 0x5562dca5fb60 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f5b1005f810 tx=0x7f5b10098bb0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.331+0000 7f5b32ffb740 1 -- 172.21.15.39:0/1250748317 shutdown_connections 2024-09-21T07:41:24.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.331+0000 7f5b32ffb740 1 --2- 172.21.15.39:0/1250748317 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562dc69f390 0x5562dca600a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.331+0000 7f5b32ffb740 1 --2- 172.21.15.39:0/1250748317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562dc67e6b0 0x5562dca5fb60 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.331+0000 7f5b32ffb740 1 --2- 172.21.15.39:0/1250748317 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5562dc6764d0 0x5562dca47310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.331+0000 7f5b32ffb740 1 -- 172.21.15.39:0/1250748317 >> 172.21.15.39:0/1250748317 conn(0x5562dc6934e0 msgr2=0x5562dcaa1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:24.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.331+0000 7f5b32ffb740 1 -- 172.21.15.39:0/1250748317 shutdown_connections 2024-09-21T07:41:24.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.331+0000 7f5b32ffb740 1 -- 172.21.15.39:0/1250748317 wait complete. 2024-09-21T07:41:24.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.332+0000 7f5b32ffb740 1 -- start start 2024-09-21T07:41:24.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.332+0000 7f5b32ffb740 1 Processor -- start 2024-09-21T07:41:24.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.334+0000 7f5b32ffb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562dc6764d0 0x5562dca863f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:24.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.334+0000 7f5b32ffb740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5562dc69f390 0x5562dcb0df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:24.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.334+0000 7f5b32ffb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562dca82dd0 0x5562dcab2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:24.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.334+0000 7f5b32ffb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5b241ff8e0 con 0x5562dca82dd0 2024-09-21T07:41:24.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.334+0000 7f5b32ffb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5562dca40d00 con 0x5562dc6764d0 2024-09-21T07:41:24.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.334+0000 7f5b32ffb740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x5562dca86e00 con 0x5562dc69f390 2024-09-21T07:41:24.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.334+0000 7f5b32ffb740 10 client.0 _ll_register_callbacks cb 0x7f5b1800b880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:41:24.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.334+0000 7f5b2a000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562dca82dd0 0x5562dcab2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:24.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.334+0000 7f5b2a000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562dca82dd0 0x5562dcab2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39584/0 (socket says 172.21.15.39:39584) 2024-09-21T07:41:24.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.334+0000 7f5b2a000640 1 -- 172.21.15.39:0/887008652 learned_addr learned my addr 172.21.15.39:0/887008652 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:41:24.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.334+0000 7f5b28c00640 1 --2- 172.21.15.39:0/887008652 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5562dc69f390 0x5562dcb0df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:24.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.334+0000 7f5b29600640 1 --2- 172.21.15.39:0/887008652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562dc6764d0 0x5562dca863f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:24.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.334+0000 7f5b2a000640 1 -- 172.21.15.39:0/887008652 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5562dc69f390 msgr2=0x5562dcb0df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:41:24.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.334+0000 7f5b2a000640 1 --2- 172.21.15.39:0/887008652 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5562dc69f390 0x5562dcb0df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.334+0000 7f5b2a000640 1 -- 172.21.15.39:0/887008652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562dc6764d0 msgr2=0x5562dca863f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:41:24.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.334+0000 7f5b2a000640 1 --2- 172.21.15.39:0/887008652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562dc6764d0 0x5562dca863f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.334+0000 7f5b2a000640 1 -- 172.21.15.39:0/887008652 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562dca86930 con 0x5562dca82dd0 2024-09-21T07:41:24.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.335+0000 7f5b2a000640 1 --2- 172.21.15.39:0/887008652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562dca82dd0 0x5562dcab2220 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c005fd0 tx=0x7f5b1c0098f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c586ec9ea418dc6b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:24.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.335+0000 7f5b0a000640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:41:24.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.335+0000 7f5b0a000640 1 -- 172.21.15.39:0/887008652 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c04f060 con 0x5562dca82dd0 2024-09-21T07:41:24.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.335+0000 7f5b0a000640 1 -- 172.21.15.39:0/887008652 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 7818+0+0 (secure 0 0 0) 0x7f5b1c004350 con 0x5562dca82dd0 2024-09-21T07:41:24.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.335+0000 7f5b32ffb740 1 -- 172.21.15.39:0/887008652 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b10098df0 con 0x5562dca82dd0 2024-09-21T07:41:24.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.335+0000 7f5b0a000640 1 -- 172.21.15.39:0/887008652 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=126) ==== 24+0+0 (secure 0 0 0) 0x7f5b1c037040 con 0x5562dca82dd0 2024-09-21T07:41:24.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.335+0000 7f5b32ffb740 10 client.7575 fetch_fsmap learned FSMap version 126 2024-09-21T07:41:24.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.335+0000 7f5b32ffb740 1 -- 172.21.15.39:0/887008652 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=126,monmap=2+}) -- 0x5562dca86b40 con 0x5562dca82dd0 2024-09-21T07:41:24.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.335+0000 7f5b0a000640 1 -- 172.21.15.39:0/887008652 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b10098df0 con 0x5562dca82dd0 2024-09-21T07:41:24.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.335+0000 7f5b0a000640 1 -- 172.21.15.39:0/887008652 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 126) ==== 3522+0+0 (secure 0 0 0) 0x7f5b1c004640 con 0x5562dca82dd0 2024-09-21T07:41:24.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.335+0000 7f5b32ffb740 10 client.7575 fetch_fsmap finished waiting for FSMap version 126 2024-09-21T07:41:24.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.335+0000 7f5b32ffb740 10 client.7575 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-21T07:41:24.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.335+0000 7f5b32ffb740 20 client.7575 populate_metadata read hostname 'smithi039' 2024-09-21T07:41:24.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.336+0000 7f5b32ffb740 1 --2- 172.21.15.39:0/887008652 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x5562dca831e0 0x5562dcab2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:24.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.336+0000 7f5b32ffb740 4 client.7575 mds_command: new command op to 7311 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:41:24.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.336+0000 7f5b32ffb740 1 -- 172.21.15.39:0/887008652 --> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5562dc9b7270 con 0x5562dca831e0 2024-09-21T07:41:24.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.336+0000 7f5b29600640 1 --2- 172.21.15.39:0/887008652 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x5562dca831e0 0x5562dcab2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:24.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.336+0000 7f5b29600640 1 --2- 172.21.15.39:0/887008652 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x5562dca831e0 0x5562dcab2760 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba54b517b52a6d42 server_cookie=5cab846b8e58f0e0 in_seq=0 out_seq=0 2024-09-21T07:41:24.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.336+0000 7f5b0a000640 10 client.7575 ms_handle_connect on v2:172.21.15.39:6832/1790581246 2024-09-21T07:41:24.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.348+0000 7f5b0a000640 1 -- 172.21.15.39:0/887008652 <== mds.0 v2:172.21.15.39:6832/1790581246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5562dc9b7270 con 0x5562dca831e0 2024-09-21T07:41:24.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.348+0000 7f5b0a000640 10 client.7575 handle_command_reply: tid=0 2024-09-21T07:41:24.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.348+0000 7f5b32ffb740 1 client.7575 shutdown 2024-09-21T07:41:24.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.349+0000 7f5b32ffb740 1 -- 172.21.15.39:0/887008652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562dca82dd0 msgr2=0x5562dcab2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:24.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.349+0000 7f5b32ffb740 1 --2- 172.21.15.39:0/887008652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562dca82dd0 0x5562dcab2220 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c005fd0 tx=0x7f5b1c0098f0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.349+0000 7f5b32ffb740 1 -- 172.21.15.39:0/887008652 shutdown_connections 2024-09-21T07:41:24.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.349+0000 7f5b32ffb740 1 --2- 172.21.15.39:0/887008652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562dca82dd0 0x5562dcab2220 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.349+0000 7f5b32ffb740 1 --2- 172.21.15.39:0/887008652 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5562dc69f390 0x5562dcb0df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.349+0000 7f5b32ffb740 1 --2- 172.21.15.39:0/887008652 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x5562dca831e0 0x5562dcab2760 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.349+0000 7f5b32ffb740 1 --2- 172.21.15.39:0/887008652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562dc6764d0 0x5562dca863f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.349+0000 7f5b32ffb740 1 -- 172.21.15.39:0/887008652 >> 172.21.15.39:0/887008652 conn(0x5562dc6934e0 msgr2=0x5562dca843d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:24.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.349+0000 7f5b0a000640 0 client.7575 ms_handle_reset on v2:172.21.15.39:6832/1790581246 2024-09-21T07:41:24.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.349+0000 7f5b32ffb740 1 -- 172.21.15.39:0/887008652 shutdown_connections 2024-09-21T07:41:24.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.349+0000 7f5b32ffb740 1 -- 172.21.15.39:0/887008652 wait complete. 2024-09-21T07:41:24.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.349+0000 7f5b32ffb740 20 client.7575 trim_cache size 0 max 16384 2024-09-21T07:41:24.352 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:41:24.352 INFO:teuthology.orchestra.run.smithi039.stdout: "message": "", 2024-09-21T07:41:24.352 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0 2024-09-21T07:41:24.353 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:41:24.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.350+0000 7f5b2c000640 1 -- 172.21.15.39:0/1376635975 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5b04078b90 msgr2=0x7f5b0407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:24.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.350+0000 7f5b2c000640 1 --2- 172.21.15.39:0/1376635975 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5b04078b90 0x7f5b0407b0a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f5b18005fd0 tx=0x7f5b1800b2e0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.350+0000 7f5b2c000640 1 -- 172.21.15.39:0/1376635975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b2415cf60 msgr2=0x7f5b2409daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:24.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.350+0000 7f5b2c000640 1 --2- 172.21.15.39:0/1376635975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b2415cf60 0x7f5b2409daf0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f5b1000c730 tx=0x7f5b1000cbf0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.350+0000 7f5b2c000640 1 -- 172.21.15.39:0/1376635975 shutdown_connections 2024-09-21T07:41:24.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.350+0000 7f5b2c000640 1 --2- 172.21.15.39:0/1376635975 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5b04078b90 0x7f5b0407b0a0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.350+0000 7f5b2c000640 1 --2- 172.21.15.39:0/1376635975 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b2415d950 0x7f5b241f5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.350+0000 7f5b2c000640 1 --2- 172.21.15.39:0/1376635975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b2415cf60 0x7f5b2409daf0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.350+0000 7f5b2c000640 1 --2- 172.21.15.39:0/1376635975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b240cce70 0x7f5b240c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.350+0000 7f5b2c000640 1 -- 172.21.15.39:0/1376635975 >> 172.21.15.39:0/1376635975 conn(0x7f5b2406d440 msgr2=0x7f5b240d03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:24.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.350+0000 7f5b2c000640 1 -- 172.21.15.39:0/1376635975 shutdown_connections 2024-09-21T07:41:24.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.350+0000 7f5b2c000640 1 -- 172.21.15.39:0/1376635975 wait complete. 2024-09-21T07:41:24.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:41:24.376 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir d1/dir/.snap/*) 2024-09-21T07:41:24.427 INFO:tasks.cephfs.test_snapshots:testing snapclient mds_kill_mdstable_at=3 2024-09-21T07:41:24.427 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:41:24.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.651+0000 7fb82b200640 1 -- 172.21.15.39:0/2286684530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb82406df50 msgr2=0x7fb82406f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:24.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.651+0000 7fb82b200640 1 --2- 172.21.15.39:0/2286684530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb82406df50 0x7fb82406f7d0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fb82408d230 tx=0x7fb81402a5f0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.651+0000 7fb82b200640 1 -- 172.21.15.39:0/2286684530 shutdown_connections 2024-09-21T07:41:24.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.651+0000 7fb82b200640 1 --2- 172.21.15.39:0/2286684530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb82406df50 0x7fb82406f7d0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.651+0000 7fb82b200640 1 --2- 172.21.15.39:0/2286684530 >> v2:172.21.15.148:3300/0 conn(0x7fb8240b96b0 0x7fb824161fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.651+0000 7fb82b200640 1 -- 172.21.15.39:0/2286684530 >> 172.21.15.39:0/2286684530 conn(0x7fb82408bfb0 msgr2=0x7fb82409d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:24.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.651+0000 7fb82b200640 1 -- 172.21.15.39:0/2286684530 shutdown_connections 2024-09-21T07:41:24.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.651+0000 7fb82b200640 1 -- 172.21.15.39:0/2286684530 wait complete. 2024-09-21T07:41:24.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.653+0000 7fb82b200640 1 Processor -- start 2024-09-21T07:41:24.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.653+0000 7fb82b200640 1 -- start start 2024-09-21T07:41:24.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.655+0000 7fb82b200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb82406df50 0x7fb8241529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:24.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.655+0000 7fb82b200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8240b96b0 0x7fb82415cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:24.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.655+0000 7fb82b200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8240d0d20 0x7fb82415d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:24.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.655+0000 7fb82b200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb8240d26a0 con 0x7fb8240d0d20 2024-09-21T07:41:24.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.655+0000 7fb82b200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb8240d2520 con 0x7fb82406df50 2024-09-21T07:41:24.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.655+0000 7fb82b200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb8240d2820 con 0x7fb8240b96b0 2024-09-21T07:41:24.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.655+0000 7fb82a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8240d0d20 0x7fb82415d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:24.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.655+0000 7fb82a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8240d0d20 0x7fb82415d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39600/0 (socket says 172.21.15.39:39600) 2024-09-21T07:41:24.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.655+0000 7fb82a800640 1 -- 172.21.15.39:0/2060216296 learned_addr learned my addr 172.21.15.39:0/2060216296 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:41:24.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.655+0000 7fb829e00640 1 --2- 172.21.15.39:0/2060216296 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb82406df50 0x7fb8241529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:24.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.655+0000 7fb829400640 1 --2- 172.21.15.39:0/2060216296 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8240b96b0 0x7fb82415cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:24.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.655+0000 7fb82a800640 1 -- 172.21.15.39:0/2060216296 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8240b96b0 msgr2=0x7fb82415cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:24.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.655+0000 7fb82a800640 1 --2- 172.21.15.39:0/2060216296 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8240b96b0 0x7fb82415cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.655+0000 7fb82a800640 1 -- 172.21.15.39:0/2060216296 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb82406df50 msgr2=0x7fb8241529c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:24.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.655+0000 7fb82a800640 1 --2- 172.21.15.39:0/2060216296 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb82406df50 0x7fb8241529c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.655+0000 7fb82a800640 1 -- 172.21.15.39:0/2060216296 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8240b4120 con 0x7fb8240d0d20 2024-09-21T07:41:24.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.656+0000 7fb82a800640 1 --2- 172.21.15.39:0/2060216296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8240d0d20 0x7fb82415d070 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fb820002230 tx=0x7fb82000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:24.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.656+0000 7fb81e000640 1 -- 172.21.15.39:0/2060216296 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb820003b70 con 0x7fb8240d0d20 2024-09-21T07:41:24.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.656+0000 7fb81e000640 1 -- 172.21.15.39:0/2060216296 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb820003d10 con 0x7fb8240d0d20 2024-09-21T07:41:24.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.656+0000 7fb81e000640 1 -- 172.21.15.39:0/2060216296 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb820005330 con 0x7fb8240d0d20 2024-09-21T07:41:24.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.656+0000 7fb82b200640 1 -- 172.21.15.39:0/2060216296 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8240c99f0 con 0x7fb8240d0d20 2024-09-21T07:41:24.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.656+0000 7fb82b200640 1 -- 172.21.15.39:0/2060216296 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb82415b210 con 0x7fb8240d0d20 2024-09-21T07:41:24.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.657+0000 7fb82b200640 1 -- 172.21.15.39:0/2060216296 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8241f9b80 con 0x7fb8240d0d20 2024-09-21T07:41:24.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.659+0000 7fb81e000640 1 -- 172.21.15.39:0/2060216296 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb8200056e0 con 0x7fb8240d0d20 2024-09-21T07:41:24.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.660+0000 7fb81e000640 1 --2- 172.21.15.39:0/2060216296 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb808078b90 0x7fb80807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:24.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.660+0000 7fb81e000640 1 -- 172.21.15.39:0/2060216296 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7818+0+0 (secure 0 0 0) 0x7fb82009b4d0 con 0x7fb8240d0d20 2024-09-21T07:41:24.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.660+0000 7fb829e00640 1 --2- 172.21.15.39:0/2060216296 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb808078b90 0x7fb80807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:24.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.661+0000 7fb829e00640 1 --2- 172.21.15.39:0/2060216296 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb808078b90 0x7fb80807b0a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb818009c00 tx=0x7fb818005f30 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:24.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.664+0000 7fb81e000640 1 -- 172.21.15.39:0/2060216296 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb8200629e0 con 0x7fb8240d0d20 2024-09-21T07:41:24.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.902+0000 7fb82b200640 1 -- 172.21.15.39:0/2060216296 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8240d2cb0 con 0x7fb8240d0d20 2024-09-21T07:41:24.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.902+0000 7fb81e000640 1 -- 172.21.15.39:0/2060216296 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 126 v126) ==== 95+0+5252 (secure 0 0 0) 0x7fb820067410 con 0x7fb8240d0d20 2024-09-21T07:41:24.904 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:41:24.905 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":126,"btime":"2024-09-21T07:41:16:016923+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6425,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/73220122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":73220122},{"type":"v1","addr":"172.21.15.148:6837","nonce":73220122}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":126,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:41:16.016912+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7452":{"gid":7452,"name":"a","rank":1,"incarnation":122,"state":"up:active","state_seq":8,"addr":"172.21.15.39:6835/3695489158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3695489158},{"type":"v1","addr":"172.21.15.39:6835","nonce":3695489158}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311,7452]},"id":6}]} 2024-09-21T07:41:24.905 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 126 2024-09-21T07:41:24.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.905+0000 7fb82b200640 1 -- 172.21.15.39:0/2060216296 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb808078b90 msgr2=0x7fb80807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:24.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.905+0000 7fb82b200640 1 --2- 172.21.15.39:0/2060216296 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb808078b90 0x7fb80807b0a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb818009c00 tx=0x7fb818005f30 comp rx=0 tx=0).stop 2024-09-21T07:41:24.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.905+0000 7fb82b200640 1 -- 172.21.15.39:0/2060216296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8240d0d20 msgr2=0x7fb82415d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:24.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.905+0000 7fb82b200640 1 --2- 172.21.15.39:0/2060216296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8240d0d20 0x7fb82415d070 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fb820002230 tx=0x7fb82000cb90 comp rx=0 tx=0).stop 2024-09-21T07:41:24.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.905+0000 7fb82b200640 1 -- 172.21.15.39:0/2060216296 shutdown_connections 2024-09-21T07:41:24.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.905+0000 7fb82b200640 1 --2- 172.21.15.39:0/2060216296 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb808078b90 0x7fb80807b0a0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.905+0000 7fb82b200640 1 --2- 172.21.15.39:0/2060216296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8240d0d20 0x7fb82415d070 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.905+0000 7fb82b200640 1 --2- 172.21.15.39:0/2060216296 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8240b96b0 0x7fb82415cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.905+0000 7fb82b200640 1 --2- 172.21.15.39:0/2060216296 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb82406df50 0x7fb8241529c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:24.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.905+0000 7fb82b200640 1 -- 172.21.15.39:0/2060216296 >> 172.21.15.39:0/2060216296 conn(0x7fb82408bfb0 msgr2=0x7fb8240b1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:24.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.905+0000 7fb82b200640 1 -- 172.21.15.39:0/2060216296 shutdown_connections 2024-09-21T07:41:24.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:24.905+0000 7fb82b200640 1 -- 172.21.15.39:0/2060216296 wait complete. 2024-09-21T07:41:24.932 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2024-09-21T07:41:25.178 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_snap":14,"last_created":12,"last_destroyed":13,"pending_noop":[],"snaps":[{"snapid":12,"ino":1099511627777,"stamp":"2024-09-21T07:40:33.249709+0000","name":"s26","metadata":{}}],"need_to_purge":{"16":[11,13]},"pending_update":[],"pending_destroy":[{"version":27,"removed_snap":12,"seq":14}]} 2024-09-21T07:41:25.178 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 12, "last_destroyed": 13, "last_snap": 14, "need_to_purge": { "16": [ 11, 13 ] }, "pending_destroy": [ { "removed_snap": 12, "seq": 14, "version": 27 } ], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s26", "snapid": 12, "stamp": "2024-09-21T07:40:33.249709+0000" } ] } 2024-09-21T07:41:25.178 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:41:25.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.404+0000 7f5b0be00640 1 -- 172.21.15.39:0/4081448831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b0415c0c0 msgr2=0x7f5b0415c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:25.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.404+0000 7f5b0be00640 1 --2- 172.21.15.39:0/4081448831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b0415c0c0 0x7f5b0415c4d0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f5b04073190 tx=0x7f5afc02a830 comp rx=0 tx=0).stop 2024-09-21T07:41:25.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.404+0000 7f5b0be00640 1 -- 172.21.15.39:0/4081448831 shutdown_connections 2024-09-21T07:41:25.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.404+0000 7f5b0be00640 1 --2- 172.21.15.39:0/4081448831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b0415c0c0 0x7f5b0415c4d0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:25.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.404+0000 7f5b0be00640 1 --2- 172.21.15.39:0/4081448831 >> v2:172.21.15.148:3300/0 conn(0x7f5b040cceb0 0x7f5b040b67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:25.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.404+0000 7f5b0be00640 1 -- 172.21.15.39:0/4081448831 >> 172.21.15.39:0/4081448831 conn(0x7f5b040b3010 msgr2=0x7f5b040b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:25.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.404+0000 7f5b0be00640 1 -- 172.21.15.39:0/4081448831 shutdown_connections 2024-09-21T07:41:25.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.404+0000 7f5b0be00640 1 -- 172.21.15.39:0/4081448831 wait complete. 2024-09-21T07:41:25.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.406+0000 7f5b0be00640 1 Processor -- start 2024-09-21T07:41:25.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.406+0000 7f5b0be00640 1 -- start start 2024-09-21T07:41:25.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.408+0000 7f5b0be00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b0406cde0 0x7f5b0415e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:25.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.408+0000 7f5b0be00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b040cceb0 0x7f5b0415e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:25.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.408+0000 7f5b0be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b0415c0c0 0x7f5b041f7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:25.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.408+0000 7f5b0be00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0415cda0 con 0x7f5b0415c0c0 2024-09-21T07:41:25.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.408+0000 7f5b0be00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0415cc20 con 0x7f5b0406cde0 2024-09-21T07:41:25.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.408+0000 7f5b0be00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f5b040b11f0 con 0x7f5b040cceb0 2024-09-21T07:41:25.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.408+0000 7f5b09e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b0415c0c0 0x7f5b041f7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:25.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.408+0000 7f5b09e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b0415c0c0 0x7f5b041f7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39622/0 (socket says 172.21.15.39:39622) 2024-09-21T07:41:25.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.408+0000 7f5b09e00640 1 -- 172.21.15.39:0/4183030644 learned_addr learned my addr 172.21.15.39:0/4183030644 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:41:25.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.408+0000 7f5b09400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b0406cde0 0x7f5b0415e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:25.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.408+0000 7f5b08a00640 1 --2- 172.21.15.39:0/4183030644 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b040cceb0 0x7f5b0415e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:25.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.409+0000 7f5b09e00640 1 -- 172.21.15.39:0/4183030644 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b040cceb0 msgr2=0x7f5b0415e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.409+0000 7f5b09e00640 1 --2- 172.21.15.39:0/4183030644 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b040cceb0 0x7f5b0415e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.409+0000 7f5b09e00640 1 -- 172.21.15.39:0/4183030644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b0406cde0 msgr2=0x7f5b0415e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.409+0000 7f5b09e00640 1 --2- 172.21.15.39:0/4183030644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b0406cde0 0x7f5b0415e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.409+0000 7f5b09e00640 1 -- 172.21.15.39:0/4183030644 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b0409edb0 con 0x7f5b0415c0c0 2024-09-21T07:41:25.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.409+0000 7f5b09e00640 1 --2- 172.21.15.39:0/4183030644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b0415c0c0 0x7f5b041f7620 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f5afc035040 tx=0x7f5afc033a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:25.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.409+0000 7f5b01600640 1 -- 172.21.15.39:0/4183030644 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc042020 con 0x7f5b0415c0c0 2024-09-21T07:41:25.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.409+0000 7f5b01600640 1 -- 172.21.15.39:0/4183030644 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5afc004040 con 0x7f5b0415c0c0 2024-09-21T07:41:25.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.409+0000 7f5b01600640 1 -- 172.21.15.39:0/4183030644 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc037040 con 0x7f5b0415c0c0 2024-09-21T07:41:25.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.409+0000 7f5b0be00640 1 -- 172.21.15.39:0/4183030644 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b0409f020 con 0x7f5b0415c0c0 2024-09-21T07:41:25.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.410+0000 7f5b0be00640 1 -- 172.21.15.39:0/4183030644 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b041496d0 con 0x7f5b0415c0c0 2024-09-21T07:41:25.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.411+0000 7f5b0be00640 1 -- 172.21.15.39:0/4183030644 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b040b67d0 con 0x7f5b0415c0c0 2024-09-21T07:41:25.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.412+0000 7f5b01600640 1 -- 172.21.15.39:0/4183030644 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5afc045020 con 0x7f5b0415c0c0 2024-09-21T07:41:25.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.413+0000 7f5b01600640 1 --2- 172.21.15.39:0/4183030644 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5ae4078bc0 0x7f5ae407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:25.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.414+0000 7f5b01600640 1 -- 172.21.15.39:0/4183030644 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7818+0+0 (secure 0 0 0) 0x7f5afc0bc570 con 0x7f5b0415c0c0 2024-09-21T07:41:25.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.414+0000 7f5b09400640 1 --2- 172.21.15.39:0/4183030644 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5ae4078bc0 0x7f5ae407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:25.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.415+0000 7f5b09400640 1 --2- 172.21.15.39:0/4183030644 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5ae4078bc0 0x7f5ae407b0d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f5af800a490 tx=0x7f5af800a420 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:25.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.418+0000 7f5b01600640 1 -- 172.21.15.39:0/4183030644 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5afc04c030 con 0x7f5b0415c0c0 2024-09-21T07:41:25.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.654+0000 7f5b0be00640 1 -- 172.21.15.39:0/4183030644 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b041602f0 con 0x7f5b0415c0c0 2024-09-21T07:41:25.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.655+0000 7f5b01600640 1 -- 172.21.15.39:0/4183030644 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 126 v126) ==== 95+0+5252 (secure 0 0 0) 0x7f5afc083b10 con 0x7f5b0415c0c0 2024-09-21T07:41:25.657 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:41:25.657 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":126,"btime":"2024-09-21T07:41:16:016923+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6425,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/73220122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":73220122},{"type":"v1","addr":"172.21.15.148:6837","nonce":73220122}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":126,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:41:16.016912+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7452":{"gid":7452,"name":"a","rank":1,"incarnation":122,"state":"up:active","state_seq":8,"addr":"172.21.15.39:6835/3695489158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3695489158},{"type":"v1","addr":"172.21.15.39:6835","nonce":3695489158}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311,7452]},"id":6}]} 2024-09-21T07:41:25.657 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 126 2024-09-21T07:41:25.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.657+0000 7f5b0be00640 1 -- 172.21.15.39:0/4183030644 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5ae4078bc0 msgr2=0x7f5ae407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:25.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.657+0000 7f5b0be00640 1 --2- 172.21.15.39:0/4183030644 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5ae4078bc0 0x7f5ae407b0d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f5af800a490 tx=0x7f5af800a420 comp rx=0 tx=0).stop 2024-09-21T07:41:25.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.657+0000 7f5b0be00640 1 -- 172.21.15.39:0/4183030644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b0415c0c0 msgr2=0x7f5b041f7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:25.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.657+0000 7f5b0be00640 1 --2- 172.21.15.39:0/4183030644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b0415c0c0 0x7f5b041f7620 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f5afc035040 tx=0x7f5afc033a50 comp rx=0 tx=0).stop 2024-09-21T07:41:25.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.657+0000 7f5b0be00640 1 -- 172.21.15.39:0/4183030644 shutdown_connections 2024-09-21T07:41:25.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.657+0000 7f5b0be00640 1 --2- 172.21.15.39:0/4183030644 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5ae4078bc0 0x7f5ae407b0d0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:25.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.657+0000 7f5b0be00640 1 --2- 172.21.15.39:0/4183030644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b0415c0c0 0x7f5b041f7620 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:25.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.657+0000 7f5b0be00640 1 --2- 172.21.15.39:0/4183030644 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5b040cceb0 0x7f5b0415e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:25.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.657+0000 7f5b0be00640 1 --2- 172.21.15.39:0/4183030644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b0406cde0 0x7f5b0415e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:25.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.657+0000 7f5b0be00640 1 -- 172.21.15.39:0/4183030644 >> 172.21.15.39:0/4183030644 conn(0x7f5b040b3010 msgr2=0x7f5b0415acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:25.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.658+0000 7f5b0be00640 1 -- 172.21.15.39:0/4183030644 shutdown_connections 2024-09-21T07:41:25.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.658+0000 7f5b0be00640 1 -- 172.21.15.39:0/4183030644 wait complete. 2024-09-21T07:41:25.686 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:1 true 2024-09-21T07:41:25.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.913+0000 7fa4dfe00640 1 -- 172.21.15.39:0/2733101980 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4e0156d60 msgr2=0x7fa4e00cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:25.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.913+0000 7fa4dfe00640 1 --2- 172.21.15.39:0/2733101980 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4e0156d60 0x7fa4e00cc430 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0005fd0 tx=0x7fa4d002a5d0 comp rx=0 tx=0).stop 2024-09-21T07:41:25.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.913+0000 7fa4dfe00640 1 -- 172.21.15.39:0/2733101980 shutdown_connections 2024-09-21T07:41:25.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.913+0000 7fa4dfe00640 1 --2- 172.21.15.39:0/2733101980 >> v2:172.21.15.148:3300/0 conn(0x7fa4e00af690 0x7fa4e0160230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:25.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.913+0000 7fa4dfe00640 1 --2- 172.21.15.39:0/2733101980 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4e0156d60 0x7fa4e00cc430 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:25.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.913+0000 7fa4dfe00640 1 -- 172.21.15.39:0/2733101980 >> 172.21.15.39:0/2733101980 conn(0x7fa4e00b25f0 msgr2=0x7fa4e00d01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:25.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.913+0000 7fa4dfe00640 1 -- 172.21.15.39:0/2733101980 shutdown_connections 2024-09-21T07:41:25.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.913+0000 7fa4dfe00640 1 -- 172.21.15.39:0/2733101980 wait complete. 2024-09-21T07:41:25.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.916+0000 7fa4dfe00640 1 Processor -- start 2024-09-21T07:41:25.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.916+0000 7fa4dfe00640 1 -- start start 2024-09-21T07:41:25.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.917+0000 7fa4dfe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4e00af690 0x7fa4e00c7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:25.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.917+0000 7fa4dfe00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4e0156d60 0x7fa4e01f9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:25.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.917+0000 7fa4dfe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4e0160770 0x7fa4e01f9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:25.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.917+0000 7fa4dfe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e015d300 con 0x7fa4e0160770 2024-09-21T07:41:25.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.917+0000 7fa4dfe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e00ae760 con 0x7fa4e00af690 2024-09-21T07:41:25.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.917+0000 7fa4dfe00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa4e015d480 con 0x7fa4e0156d60 2024-09-21T07:41:25.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.917+0000 7fa4df400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4e0160770 0x7fa4e01f9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:25.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.918+0000 7fa4d7e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4e0156d60 0x7fa4e01f9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:25.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.918+0000 7fa4dea00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4e00af690 0x7fa4e00c7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:25.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.918+0000 7fa4d7e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4e0156d60 0x7fa4e01f9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.39:43764/0 (socket says 172.21.15.39:43764) 2024-09-21T07:41:25.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.918+0000 7fa4d7e00640 1 -- 172.21.15.39:0/3508408554 learned_addr learned my addr 172.21.15.39:0/3508408554 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:41:25.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.918+0000 7fa4df400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4e0160770 0x7fa4e01f9990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39624/0 (socket says 172.21.15.39:39624) 2024-09-21T07:41:25.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.918+0000 7fa4dea00640 1 -- 172.21.15.39:0/3508408554 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4e0156d60 msgr2=0x7fa4e01f9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:25.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.918+0000 7fa4dea00640 1 --2- 172.21.15.39:0/3508408554 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4e0156d60 0x7fa4e01f9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:25.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.918+0000 7fa4dea00640 1 -- 172.21.15.39:0/3508408554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4e0160770 msgr2=0x7fa4e01f9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:25.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.918+0000 7fa4dea00640 1 --2- 172.21.15.39:0/3508408554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4e0160770 0x7fa4e01f9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:25.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.918+0000 7fa4dea00640 1 -- 172.21.15.39:0/3508408554 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4e00c81d0 con 0x7fa4e00af690 2024-09-21T07:41:25.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.918+0000 7fa4df400640 1 --2- 172.21.15.39:0/3508408554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4e0160770 0x7fa4e01f9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:41:25.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.919+0000 7fa4dea00640 1 --2- 172.21.15.39:0/3508408554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4e00af690 0x7fa4e00c7590 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0031000 tx=0x7fa4d0005770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:25.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.919+0000 7fa4d7400640 1 -- 172.21.15.39:0/3508408554 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d003e030 con 0x7fa4e00af690 2024-09-21T07:41:25.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.919+0000 7fa4d7400640 1 -- 172.21.15.39:0/3508408554 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4d00039f0 con 0x7fa4e00af690 2024-09-21T07:41:25.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.919+0000 7fa4dfe00640 1 -- 172.21.15.39:0/3508408554 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4e00c7ef0 con 0x7fa4e00af690 2024-09-21T07:41:25.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.919+0000 7fa4d7400640 1 -- 172.21.15.39:0/3508408554 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d0035040 con 0x7fa4e00af690 2024-09-21T07:41:25.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.919+0000 7fa4dfe00640 1 -- 172.21.15.39:0/3508408554 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4e01f5d10 con 0x7fa4e00af690 2024-09-21T07:41:25.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.920+0000 7fa4dfe00640 1 -- 172.21.15.39:0/3508408554 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4e015d930 con 0x7fa4e00af690 2024-09-21T07:41:25.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.922+0000 7fa4d7400640 1 -- 172.21.15.39:0/3508408554 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa4d0041030 con 0x7fa4e00af690 2024-09-21T07:41:25.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.923+0000 7fa4d7400640 1 --2- 172.21.15.39:0/3508408554 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa4bc078b90 0x7fa4bc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:25.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.923+0000 7fa4d7400640 1 -- 172.21.15.39:0/3508408554 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7818+0+0 (secure 0 0 0) 0x7fa4d00bbe30 con 0x7fa4e00af690 2024-09-21T07:41:25.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.923+0000 7fa4d7e00640 1 --2- 172.21.15.39:0/3508408554 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa4bc078b90 0x7fa4bc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:25.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.924+0000 7fa4d7e00640 1 --2- 172.21.15.39:0/3508408554 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa4bc078b90 0x7fa4bc07b0a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00c3200 tx=0x7fa4c8005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:25.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:25.928+0000 7fa4d7400640 1 -- 172.21.15.39:0/3508408554 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa4d0083340 con 0x7fa4e00af690 2024-09-21T07:41:26.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:26.201+0000 7fa4dfe00640 1 -- 172.21.15.39:0/3508408554 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:1", "val": "true"} v 0) -- 0x7fa4e0074e80 con 0x7fa4e00af690 2024-09-21T07:41:27.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.096+0000 7fa4d7400640 1 -- 172.21.15.39:0/3508408554 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:1", "val": "true"}]=0 freezing mds.7452 v127) ==== 112+0+0 (secure 0 0 0) 0x7fa4d0087d70 con 0x7fa4e00af690 2024-09-21T07:41:27.098 INFO:teuthology.orchestra.run.smithi039.stderr:freezing mds.7452 2024-09-21T07:41:27.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.099+0000 7fa4dfe00640 1 -- 172.21.15.39:0/3508408554 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa4bc078b90 msgr2=0x7fa4bc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:27.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.099+0000 7fa4dfe00640 1 --2- 172.21.15.39:0/3508408554 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa4bc078b90 0x7fa4bc07b0a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00c3200 tx=0x7fa4c8005f90 comp rx=0 tx=0).stop 2024-09-21T07:41:27.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.099+0000 7fa4dfe00640 1 -- 172.21.15.39:0/3508408554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4e00af690 msgr2=0x7fa4e00c7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:27.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.099+0000 7fa4dfe00640 1 --2- 172.21.15.39:0/3508408554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4e00af690 0x7fa4e00c7590 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0031000 tx=0x7fa4d0005770 comp rx=0 tx=0).stop 2024-09-21T07:41:27.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.099+0000 7fa4dfe00640 1 -- 172.21.15.39:0/3508408554 shutdown_connections 2024-09-21T07:41:27.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.099+0000 7fa4dfe00640 1 --2- 172.21.15.39:0/3508408554 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa4bc078b90 0x7fa4bc07b0a0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:27.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.099+0000 7fa4dfe00640 1 --2- 172.21.15.39:0/3508408554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4e0160770 0x7fa4e01f9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:27.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.099+0000 7fa4dfe00640 1 --2- 172.21.15.39:0/3508408554 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4e0156d60 0x7fa4e01f9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:27.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.099+0000 7fa4dfe00640 1 --2- 172.21.15.39:0/3508408554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4e00af690 0x7fa4e00c7590 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:27.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.099+0000 7fa4dfe00640 1 -- 172.21.15.39:0/3508408554 >> 172.21.15.39:0/3508408554 conn(0x7fa4e00b25f0 msgr2=0x7fa4e00d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:27.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.099+0000 7fa4dfe00640 1 -- 172.21.15.39:0/3508408554 shutdown_connections 2024-09-21T07:41:27.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.099+0000 7fa4dfe00640 1 -- 172.21.15.39:0/3508408554 wait complete. 2024-09-21T07:41:27.126 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_kill_mdstable_at 3 2024-09-21T07:41:27.375 INFO:teuthology.orchestra.run.smithi039.stdout:{"success":"mds_kill_mdstable_at = '' (not observed, change may require restart) "} 2024-09-21T07:41:27.375 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_kill_mdstable_at = '' (not observed, change may require restart) " } 2024-09-21T07:41:27.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:41:27.375 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d1/dir/.snap/s33) 2024-09-21T07:41:27.380 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-09-21T07:41:27.415 INFO:tasks.ceph.mds.a.smithi039.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::handle_request(ceph::cref_t&)' thread 7ff542a00640 time 2024-09-21T07:41:27.414519+0000 2024-09-21T07:41:27.415 INFO:tasks.ceph.mds.a.smithi039.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: 76: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 3) 2024-09-21T07:41:27.418 INFO:tasks.ceph.mds.a.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:41:27.418 INFO:tasks.ceph.mds.a.smithi039.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7ff549a5654b] 2024-09-21T07:41:27.418 INFO:tasks.ceph.mds.a.smithi039.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7ff549a56758] 2024-09-21T07:41:27.418 INFO:tasks.ceph.mds.a.smithi039.stderr: 3: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x506) [0x5572d33c670a] 2024-09-21T07:41:27.418 INFO:tasks.ceph.mds.a.smithi039.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xc96) [0x5572d305edd4] 2024-09-21T07:41:27.418 INFO:tasks.ceph.mds.a.smithi039.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5572d3062523] 2024-09-21T07:41:27.418 INFO:tasks.ceph.mds.a.smithi039.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5572d3062e4a] 2024-09-21T07:41:27.418 INFO:tasks.ceph.mds.a.smithi039.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x5572d3045fa4] 2024-09-21T07:41:27.418 INFO:tasks.ceph.mds.a.smithi039.stderr: 8: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7ff549ba57df] 2024-09-21T07:41:27.418 INFO:tasks.ceph.mds.a.smithi039.stderr: 9: (DispatchQueue::entry()+0x60f) [0x7ff549ba60d1] 2024-09-21T07:41:27.419 INFO:tasks.ceph.mds.a.smithi039.stderr: 10: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7ff549c93f5b] 2024-09-21T07:41:27.419 INFO:tasks.ceph.mds.a.smithi039.stderr: 11: (Thread::entry_wrapper()+0x43) [0x7ff549a2fd57] 2024-09-21T07:41:27.419 INFO:tasks.ceph.mds.a.smithi039.stderr: 12: (Thread::_entry_func(void*)+0xd) [0x7ff549a2fd73] 2024-09-21T07:41:27.419 INFO:tasks.ceph.mds.a.smithi039.stderr: 13: /lib64/libc.so.6(+0x89c52) [0x7ff549089c52] 2024-09-21T07:41:27.419 INFO:tasks.ceph.mds.a.smithi039.stderr: 14: /lib64/libc.so.6(+0x10ec80) [0x7ff54910ec80] 2024-09-21T07:41:27.419 INFO:tasks.ceph.mds.a.smithi039.stderr:2024-09-21T07:41:27.414+0000 7ff542a00640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::handle_request(ceph::cref_t&)' thread 7ff542a00640 time 2024-09-21T07:41:27.414519+0000 2024-09-21T07:41:27.419 INFO:tasks.ceph.mds.a.smithi039.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: 76: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 3) 2024-09-21T07:41:27.419 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:41:27.419 INFO:tasks.ceph.mds.a.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:41:27.419 INFO:tasks.ceph.mds.a.smithi039.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7ff549a5654b] 2024-09-21T07:41:27.419 INFO:tasks.ceph.mds.a.smithi039.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7ff549a56758] 2024-09-21T07:41:27.419 INFO:tasks.ceph.mds.a.smithi039.stderr: 3: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x506) [0x5572d33c670a] 2024-09-21T07:41:27.419 INFO:tasks.ceph.mds.a.smithi039.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xc96) [0x5572d305edd4] 2024-09-21T07:41:27.419 INFO:tasks.ceph.mds.a.smithi039.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5572d3062523] 2024-09-21T07:41:27.420 INFO:tasks.ceph.mds.a.smithi039.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5572d3062e4a] 2024-09-21T07:41:27.420 INFO:tasks.ceph.mds.a.smithi039.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x5572d3045fa4] 2024-09-21T07:41:27.420 INFO:tasks.ceph.mds.a.smithi039.stderr: 8: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7ff549ba57df] 2024-09-21T07:41:27.420 INFO:tasks.ceph.mds.a.smithi039.stderr: 9: (DispatchQueue::entry()+0x60f) [0x7ff549ba60d1] 2024-09-21T07:41:27.420 INFO:tasks.ceph.mds.a.smithi039.stderr: 10: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7ff549c93f5b] 2024-09-21T07:41:27.420 INFO:tasks.ceph.mds.a.smithi039.stderr: 11: (Thread::entry_wrapper()+0x43) [0x7ff549a2fd57] 2024-09-21T07:41:27.420 INFO:tasks.ceph.mds.a.smithi039.stderr: 12: (Thread::_entry_func(void*)+0xd) [0x7ff549a2fd73] 2024-09-21T07:41:27.420 INFO:tasks.ceph.mds.a.smithi039.stderr: 13: /lib64/libc.so.6(+0x89c52) [0x7ff549089c52] 2024-09-21T07:41:27.420 INFO:tasks.ceph.mds.a.smithi039.stderr: 14: /lib64/libc.so.6(+0x10ec80) [0x7ff54910ec80] 2024-09-21T07:41:27.420 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:41:27.420 INFO:tasks.ceph.mds.a.smithi039.stderr:*** Caught signal (Aborted) ** 2024-09-21T07:41:27.420 INFO:tasks.ceph.mds.a.smithi039.stderr: in thread 7ff542a00640 thread_name:ms_dispatch 2024-09-21T07:41:27.420 INFO:tasks.ceph.mds.a.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:41:27.420 INFO:tasks.ceph.mds.a.smithi039.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7ff54903e6f0] 2024-09-21T07:41:27.421 INFO:tasks.ceph.mds.a.smithi039.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7ff54908b99c] 2024-09-21T07:41:27.421 INFO:tasks.ceph.mds.a.smithi039.stderr: 3: raise() 2024-09-21T07:41:27.421 INFO:tasks.ceph.mds.a.smithi039.stderr: 4: abort() 2024-09-21T07:41:27.421 INFO:tasks.ceph.mds.a.smithi039.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7ff549a56686] 2024-09-21T07:41:27.421 INFO:tasks.ceph.mds.a.smithi039.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7ff549a56758] 2024-09-21T07:41:27.421 INFO:tasks.ceph.mds.a.smithi039.stderr: 7: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x506) [0x5572d33c670a] 2024-09-21T07:41:27.421 INFO:tasks.ceph.mds.a.smithi039.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xc96) [0x5572d305edd4] 2024-09-21T07:41:27.421 INFO:tasks.ceph.mds.a.smithi039.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5572d3062523] 2024-09-21T07:41:27.421 INFO:tasks.ceph.mds.a.smithi039.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5572d3062e4a] 2024-09-21T07:41:27.421 INFO:tasks.ceph.mds.a.smithi039.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x5572d3045fa4] 2024-09-21T07:41:27.421 INFO:tasks.ceph.mds.a.smithi039.stderr: 12: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7ff549ba57df] 2024-09-21T07:41:27.421 INFO:tasks.ceph.mds.a.smithi039.stderr: 13: (DispatchQueue::entry()+0x60f) [0x7ff549ba60d1] 2024-09-21T07:41:27.421 INFO:tasks.ceph.mds.a.smithi039.stderr: 14: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7ff549c93f5b] 2024-09-21T07:41:27.421 INFO:tasks.ceph.mds.a.smithi039.stderr: 15: (Thread::entry_wrapper()+0x43) [0x7ff549a2fd57] 2024-09-21T07:41:27.422 INFO:tasks.ceph.mds.a.smithi039.stderr: 16: (Thread::_entry_func(void*)+0xd) [0x7ff549a2fd73] 2024-09-21T07:41:27.422 INFO:tasks.ceph.mds.a.smithi039.stderr: 17: /lib64/libc.so.6(+0x89c52) [0x7ff549089c52] 2024-09-21T07:41:27.422 INFO:tasks.ceph.mds.a.smithi039.stderr: 18: /lib64/libc.so.6(+0x10ec80) [0x7ff54910ec80] 2024-09-21T07:41:27.422 INFO:tasks.ceph.mds.a.smithi039.stderr:2024-09-21T07:41:27.415+0000 7ff542a00640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:41:27.422 INFO:tasks.ceph.mds.a.smithi039.stderr: in thread 7ff542a00640 thread_name:ms_dispatch 2024-09-21T07:41:27.422 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:41:27.422 INFO:tasks.ceph.mds.a.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:41:27.422 INFO:tasks.ceph.mds.a.smithi039.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7ff54903e6f0] 2024-09-21T07:41:27.422 INFO:tasks.ceph.mds.a.smithi039.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7ff54908b99c] 2024-09-21T07:41:27.422 INFO:tasks.ceph.mds.a.smithi039.stderr: 3: raise() 2024-09-21T07:41:27.422 INFO:tasks.ceph.mds.a.smithi039.stderr: 4: abort() 2024-09-21T07:41:27.422 INFO:tasks.ceph.mds.a.smithi039.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7ff549a56686] 2024-09-21T07:41:27.422 INFO:tasks.ceph.mds.a.smithi039.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7ff549a56758] 2024-09-21T07:41:27.422 INFO:tasks.ceph.mds.a.smithi039.stderr: 7: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x506) [0x5572d33c670a] 2024-09-21T07:41:27.422 INFO:tasks.ceph.mds.a.smithi039.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xc96) [0x5572d305edd4] 2024-09-21T07:41:27.423 INFO:tasks.ceph.mds.a.smithi039.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5572d3062523] 2024-09-21T07:41:27.423 INFO:tasks.ceph.mds.a.smithi039.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5572d3062e4a] 2024-09-21T07:41:27.423 INFO:tasks.ceph.mds.a.smithi039.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x5572d3045fa4] 2024-09-21T07:41:27.423 INFO:tasks.ceph.mds.a.smithi039.stderr: 12: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7ff549ba57df] 2024-09-21T07:41:27.423 INFO:tasks.ceph.mds.a.smithi039.stderr: 13: (DispatchQueue::entry()+0x60f) [0x7ff549ba60d1] 2024-09-21T07:41:27.423 INFO:tasks.ceph.mds.a.smithi039.stderr: 14: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7ff549c93f5b] 2024-09-21T07:41:27.423 INFO:tasks.ceph.mds.a.smithi039.stderr: 15: (Thread::entry_wrapper()+0x43) [0x7ff549a2fd57] 2024-09-21T07:41:27.423 INFO:tasks.ceph.mds.a.smithi039.stderr: 16: (Thread::_entry_func(void*)+0xd) [0x7ff549a2fd73] 2024-09-21T07:41:27.423 INFO:tasks.ceph.mds.a.smithi039.stderr: 17: /lib64/libc.so.6(+0x89c52) [0x7ff549089c52] 2024-09-21T07:41:27.423 INFO:tasks.ceph.mds.a.smithi039.stderr: 18: /lib64/libc.so.6(+0x10ec80) [0x7ff54910ec80] 2024-09-21T07:41:27.423 INFO:tasks.ceph.mds.a.smithi039.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:41:27.423 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:41:27.428 INFO:tasks.ceph.mds.a.smithi039.stderr: -1> 2024-09-21T07:41:27.414+0000 7ff542a00640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::handle_request(ceph::cref_t&)' thread 7ff542a00640 time 2024-09-21T07:41:27.414519+0000 2024-09-21T07:41:27.428 INFO:tasks.ceph.mds.a.smithi039.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: 76: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 3) 2024-09-21T07:41:27.428 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:41:27.428 INFO:tasks.ceph.mds.a.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:41:27.428 INFO:tasks.ceph.mds.a.smithi039.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7ff549a5654b] 2024-09-21T07:41:27.429 INFO:tasks.ceph.mds.a.smithi039.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7ff549a56758] 2024-09-21T07:41:27.429 INFO:tasks.ceph.mds.a.smithi039.stderr: 3: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x506) [0x5572d33c670a] 2024-09-21T07:41:27.429 INFO:tasks.ceph.mds.a.smithi039.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xc96) [0x5572d305edd4] 2024-09-21T07:41:27.429 INFO:tasks.ceph.mds.a.smithi039.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5572d3062523] 2024-09-21T07:41:27.429 INFO:tasks.ceph.mds.a.smithi039.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5572d3062e4a] 2024-09-21T07:41:27.429 INFO:tasks.ceph.mds.a.smithi039.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x5572d3045fa4] 2024-09-21T07:41:27.429 INFO:tasks.ceph.mds.a.smithi039.stderr: 8: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7ff549ba57df] 2024-09-21T07:41:27.429 INFO:tasks.ceph.mds.a.smithi039.stderr: 9: (DispatchQueue::entry()+0x60f) [0x7ff549ba60d1] 2024-09-21T07:41:27.429 INFO:tasks.ceph.mds.a.smithi039.stderr: 10: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7ff549c93f5b] 2024-09-21T07:41:27.429 INFO:tasks.ceph.mds.a.smithi039.stderr: 11: (Thread::entry_wrapper()+0x43) [0x7ff549a2fd57] 2024-09-21T07:41:27.429 INFO:tasks.ceph.mds.a.smithi039.stderr: 12: (Thread::_entry_func(void*)+0xd) [0x7ff549a2fd73] 2024-09-21T07:41:27.429 INFO:tasks.ceph.mds.a.smithi039.stderr: 13: /lib64/libc.so.6(+0x89c52) [0x7ff549089c52] 2024-09-21T07:41:27.429 INFO:tasks.ceph.mds.a.smithi039.stderr: 14: /lib64/libc.so.6(+0x10ec80) [0x7ff54910ec80] 2024-09-21T07:41:27.429 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:41:27.429 INFO:tasks.ceph.mds.a.smithi039.stderr: 0> 2024-09-21T07:41:27.415+0000 7ff542a00640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:41:27.430 INFO:tasks.ceph.mds.a.smithi039.stderr: in thread 7ff542a00640 thread_name:ms_dispatch 2024-09-21T07:41:27.430 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:41:27.430 INFO:tasks.ceph.mds.a.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:41:27.430 INFO:tasks.ceph.mds.a.smithi039.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7ff54903e6f0] 2024-09-21T07:41:27.430 INFO:tasks.ceph.mds.a.smithi039.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7ff54908b99c] 2024-09-21T07:41:27.430 INFO:tasks.ceph.mds.a.smithi039.stderr: 3: raise() 2024-09-21T07:41:27.430 INFO:tasks.ceph.mds.a.smithi039.stderr: 4: abort() 2024-09-21T07:41:27.430 INFO:tasks.ceph.mds.a.smithi039.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7ff549a56686] 2024-09-21T07:41:27.430 INFO:tasks.ceph.mds.a.smithi039.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7ff549a56758] 2024-09-21T07:41:27.430 INFO:tasks.ceph.mds.a.smithi039.stderr: 7: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x506) [0x5572d33c670a] 2024-09-21T07:41:27.430 INFO:tasks.ceph.mds.a.smithi039.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xc96) [0x5572d305edd4] 2024-09-21T07:41:27.430 INFO:tasks.ceph.mds.a.smithi039.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5572d3062523] 2024-09-21T07:41:27.430 INFO:tasks.ceph.mds.a.smithi039.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5572d3062e4a] 2024-09-21T07:41:27.430 INFO:tasks.ceph.mds.a.smithi039.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x5572d3045fa4] 2024-09-21T07:41:27.431 INFO:tasks.ceph.mds.a.smithi039.stderr: 12: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7ff549ba57df] 2024-09-21T07:41:27.431 INFO:tasks.ceph.mds.a.smithi039.stderr: 13: (DispatchQueue::entry()+0x60f) [0x7ff549ba60d1] 2024-09-21T07:41:27.431 INFO:tasks.ceph.mds.a.smithi039.stderr: 14: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7ff549c93f5b] 2024-09-21T07:41:27.431 INFO:tasks.ceph.mds.a.smithi039.stderr: 15: (Thread::entry_wrapper()+0x43) [0x7ff549a2fd57] 2024-09-21T07:41:27.431 INFO:tasks.ceph.mds.a.smithi039.stderr: 16: (Thread::_entry_func(void*)+0xd) [0x7ff549a2fd73] 2024-09-21T07:41:27.431 INFO:tasks.ceph.mds.a.smithi039.stderr: 17: /lib64/libc.so.6(+0x89c52) [0x7ff549089c52] 2024-09-21T07:41:27.431 INFO:tasks.ceph.mds.a.smithi039.stderr: 18: /lib64/libc.so.6(+0x10ec80) [0x7ff54910ec80] 2024-09-21T07:41:27.431 INFO:tasks.ceph.mds.a.smithi039.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:41:27.431 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:41:27.436 INFO:tasks.ceph.mds.a.smithi039.stderr: -1> 2024-09-21T07:41:27.414+0000 7ff542a00640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::handle_request(ceph::cref_t&)' thread 7ff542a00640 time 2024-09-21T07:41:27.414519+0000 2024-09-21T07:41:27.436 INFO:tasks.ceph.mds.a.smithi039.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: 76: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 3) 2024-09-21T07:41:27.436 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:41:27.436 INFO:tasks.ceph.mds.a.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:41:27.436 INFO:tasks.ceph.mds.a.smithi039.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7ff549a5654b] 2024-09-21T07:41:27.436 INFO:tasks.ceph.mds.a.smithi039.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7ff549a56758] 2024-09-21T07:41:27.436 INFO:tasks.ceph.mds.a.smithi039.stderr: 3: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x506) [0x5572d33c670a] 2024-09-21T07:41:27.436 INFO:tasks.ceph.mds.a.smithi039.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xc96) [0x5572d305edd4] 2024-09-21T07:41:27.436 INFO:tasks.ceph.mds.a.smithi039.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5572d3062523] 2024-09-21T07:41:27.436 INFO:tasks.ceph.mds.a.smithi039.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5572d3062e4a] 2024-09-21T07:41:27.436 INFO:tasks.ceph.mds.a.smithi039.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x5572d3045fa4] 2024-09-21T07:41:27.436 INFO:tasks.ceph.mds.a.smithi039.stderr: 8: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7ff549ba57df] 2024-09-21T07:41:27.437 INFO:tasks.ceph.mds.a.smithi039.stderr: 9: (DispatchQueue::entry()+0x60f) [0x7ff549ba60d1] 2024-09-21T07:41:27.437 INFO:tasks.ceph.mds.a.smithi039.stderr: 10: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7ff549c93f5b] 2024-09-21T07:41:27.437 INFO:tasks.ceph.mds.a.smithi039.stderr: 11: (Thread::entry_wrapper()+0x43) [0x7ff549a2fd57] 2024-09-21T07:41:27.437 INFO:tasks.ceph.mds.a.smithi039.stderr: 12: (Thread::_entry_func(void*)+0xd) [0x7ff549a2fd73] 2024-09-21T07:41:27.437 INFO:tasks.ceph.mds.a.smithi039.stderr: 13: /lib64/libc.so.6(+0x89c52) [0x7ff549089c52] 2024-09-21T07:41:27.437 INFO:tasks.ceph.mds.a.smithi039.stderr: 14: /lib64/libc.so.6(+0x10ec80) [0x7ff54910ec80] 2024-09-21T07:41:27.437 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:41:27.437 INFO:tasks.ceph.mds.a.smithi039.stderr: 0> 2024-09-21T07:41:27.415+0000 7ff542a00640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:41:27.437 INFO:tasks.ceph.mds.a.smithi039.stderr: in thread 7ff542a00640 thread_name:ms_dispatch 2024-09-21T07:41:27.437 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:41:27.437 INFO:tasks.ceph.mds.a.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:41:27.437 INFO:tasks.ceph.mds.a.smithi039.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7ff54903e6f0] 2024-09-21T07:41:27.437 INFO:tasks.ceph.mds.a.smithi039.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7ff54908b99c] 2024-09-21T07:41:27.437 INFO:tasks.ceph.mds.a.smithi039.stderr: 3: raise() 2024-09-21T07:41:27.437 INFO:tasks.ceph.mds.a.smithi039.stderr: 4: abort() 2024-09-21T07:41:27.438 INFO:tasks.ceph.mds.a.smithi039.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7ff549a56686] 2024-09-21T07:41:27.438 INFO:tasks.ceph.mds.a.smithi039.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7ff549a56758] 2024-09-21T07:41:27.438 INFO:tasks.ceph.mds.a.smithi039.stderr: 7: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x506) [0x5572d33c670a] 2024-09-21T07:41:27.438 INFO:tasks.ceph.mds.a.smithi039.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xc96) [0x5572d305edd4] 2024-09-21T07:41:27.438 INFO:tasks.ceph.mds.a.smithi039.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5572d3062523] 2024-09-21T07:41:27.438 INFO:tasks.ceph.mds.a.smithi039.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5572d3062e4a] 2024-09-21T07:41:27.438 INFO:tasks.ceph.mds.a.smithi039.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x5572d3045fa4] 2024-09-21T07:41:27.438 INFO:tasks.ceph.mds.a.smithi039.stderr: 12: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7ff549ba57df] 2024-09-21T07:41:27.438 INFO:tasks.ceph.mds.a.smithi039.stderr: 13: (DispatchQueue::entry()+0x60f) [0x7ff549ba60d1] 2024-09-21T07:41:27.438 INFO:tasks.ceph.mds.a.smithi039.stderr: 14: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7ff549c93f5b] 2024-09-21T07:41:27.438 INFO:tasks.ceph.mds.a.smithi039.stderr: 15: (Thread::entry_wrapper()+0x43) [0x7ff549a2fd57] 2024-09-21T07:41:27.438 INFO:tasks.ceph.mds.a.smithi039.stderr: 16: (Thread::_entry_func(void*)+0xd) [0x7ff549a2fd73] 2024-09-21T07:41:27.438 INFO:tasks.ceph.mds.a.smithi039.stderr: 17: /lib64/libc.so.6(+0x89c52) [0x7ff549089c52] 2024-09-21T07:41:27.439 INFO:tasks.ceph.mds.a.smithi039.stderr: 18: /lib64/libc.so.6(+0x10ec80) [0x7ff54910ec80] 2024-09-21T07:41:27.439 INFO:tasks.ceph.mds.a.smithi039.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:41:27.439 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:41:27.651 INFO:teuthology.orchestra.run.smithi039.stdout:{"mds_beacon_grace":"15.000000"} 2024-09-21T07:41:27.651 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-09-21T07:41:27.652 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:41:27.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.923+0000 7f3eef200640 1 -- 172.21.15.39:0/2211306213 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ef00b43f0 msgr2=0x7f3ef00ca890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:27.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.923+0000 7f3eef200640 1 --2- 172.21.15.39:0/2211306213 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ef00b43f0 0x7f3ef00ca890 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f3ef015e4f0 tx=0x7f3ee002a440 comp rx=0 tx=0).stop 2024-09-21T07:41:27.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.924+0000 7f3eef200640 1 -- 172.21.15.39:0/2211306213 shutdown_connections 2024-09-21T07:41:27.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.924+0000 7f3eef200640 1 --2- 172.21.15.39:0/2211306213 >> v2:172.21.15.148:3300/0 conn(0x7f3ef00af490 0x7f3ef00cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:27.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.924+0000 7f3eef200640 1 --2- 172.21.15.39:0/2211306213 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ef015abe0 0x7f3ef00cf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:27.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.924+0000 7f3eef200640 1 --2- 172.21.15.39:0/2211306213 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ef00b43f0 0x7f3ef00ca890 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:27.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.924+0000 7f3eef200640 1 -- 172.21.15.39:0/2211306213 >> 172.21.15.39:0/2211306213 conn(0x7f3ef00b2510 msgr2=0x7f3ef00d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:27.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.924+0000 7f3eef200640 1 -- 172.21.15.39:0/2211306213 shutdown_connections 2024-09-21T07:41:27.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.924+0000 7f3eef200640 1 -- 172.21.15.39:0/2211306213 wait complete. 2024-09-21T07:41:27.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.926+0000 7f3eef200640 1 Processor -- start 2024-09-21T07:41:27.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.926+0000 7f3eef200640 1 -- start start 2024-09-21T07:41:27.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.927+0000 7f3eef200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ef00af490 0x7f3ef01f73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:27.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.927+0000 7f3eef200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ef00b43f0 0x7f3ef009ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:27.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.928+0000 7f3eef200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ef015abe0 0x7f3ef00b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:27.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.928+0000 7f3eef200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef0070e80 con 0x7f3ef00b43f0 2024-09-21T07:41:27.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.928+0000 7f3eef200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef00712a0 con 0x7f3ef00af490 2024-09-21T07:41:27.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.928+0000 7f3eef200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f3ef006f6c0 con 0x7f3ef015abe0 2024-09-21T07:41:27.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.928+0000 7f3eed400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ef00b43f0 0x7f3ef009ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:27.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.928+0000 7f3eed400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ef00b43f0 0x7f3ef009ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39648/0 (socket says 172.21.15.39:39648) 2024-09-21T07:41:27.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.928+0000 7f3eed400640 1 -- 172.21.15.39:0/502694206 learned_addr learned my addr 172.21.15.39:0/502694206 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:41:27.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.928+0000 7f3eede00640 1 --2- 172.21.15.39:0/502694206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ef00af490 0x7f3ef01f73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:27.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.928+0000 7f3eee800640 1 --2- 172.21.15.39:0/502694206 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ef015abe0 0x7f3ef00b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:27.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.928+0000 7f3eed400640 1 -- 172.21.15.39:0/502694206 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ef015abe0 msgr2=0x7f3ef00b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:27.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.928+0000 7f3eed400640 1 --2- 172.21.15.39:0/502694206 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ef015abe0 0x7f3ef00b0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:27.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.928+0000 7f3eed400640 1 -- 172.21.15.39:0/502694206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ef00af490 msgr2=0x7f3ef01f73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:27.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.928+0000 7f3eed400640 1 --2- 172.21.15.39:0/502694206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ef00af490 0x7f3ef01f73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:27.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.928+0000 7f3eed400640 1 -- 172.21.15.39:0/502694206 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef01e0d70 con 0x7f3ef00b43f0 2024-09-21T07:41:27.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.929+0000 7f3eed400640 1 --2- 172.21.15.39:0/502694206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ef00b43f0 0x7f3ef009ab20 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f3edc00d740 tx=0x7f3edc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:27.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.929+0000 7f3ee6000640 1 -- 172.21.15.39:0/502694206 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3edc017070 con 0x7f3ef00b43f0 2024-09-21T07:41:27.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.929+0000 7f3ee6000640 1 -- 172.21.15.39:0/502694206 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3edc009d20 con 0x7f3ef00b43f0 2024-09-21T07:41:27.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.929+0000 7f3eef200640 1 -- 172.21.15.39:0/502694206 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ef01dffb0 con 0x7f3ef00b43f0 2024-09-21T07:41:27.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.929+0000 7f3ee6000640 1 -- 172.21.15.39:0/502694206 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3edc0106c0 con 0x7f3ef00b43f0 2024-09-21T07:41:27.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.929+0000 7f3eef200640 1 -- 172.21.15.39:0/502694206 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ef01e0570 con 0x7f3ef00b43f0 2024-09-21T07:41:27.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.930+0000 7f3eef200640 1 -- 172.21.15.39:0/502694206 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ef00ca890 con 0x7f3ef00b43f0 2024-09-21T07:41:27.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.931+0000 7f3ee6000640 1 -- 172.21.15.39:0/502694206 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3edc01a070 con 0x7f3ef00b43f0 2024-09-21T07:41:27.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.933+0000 7f3ee6000640 1 --2- 172.21.15.39:0/502694206 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3ed0078b90 0x7f3ed007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:27.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.933+0000 7f3ee6000640 1 -- 172.21.15.39:0/502694206 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7818+0+0 (secure 0 0 0) 0x7f3edc09b760 con 0x7f3ef00b43f0 2024-09-21T07:41:27.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.933+0000 7f3eede00640 1 --2- 172.21.15.39:0/502694206 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3ed0078b90 0x7f3ed007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:27.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.934+0000 7f3eede00640 1 --2- 172.21.15.39:0/502694206 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3ed0078b90 0x7f3ed007b0a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0033000 tx=0x7f3ee0004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:27.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:27.937+0000 7f3ee6000640 1 -- 172.21.15.39:0/502694206 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3edc063350 con 0x7f3ef00b43f0 2024-09-21T07:41:28.154 INFO:tasks.ceph.mds.a.smithi039.stderr:daemon-helper: command crashed with signal 6 2024-09-21T07:41:28.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:28.178+0000 7f3eef200640 1 -- 172.21.15.39:0/502694206 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ef01e6f30 con 0x7f3ef00b43f0 2024-09-21T07:41:28.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:28.179+0000 7f3ee6000640 1 -- 172.21.15.39:0/502694206 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 127 v127) ==== 95+0+5252 (secure 0 0 0) 0x7f3edc062c50 con 0x7f3ef00b43f0 2024-09-21T07:41:28.181 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:41:28.181 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":127,"btime":"2024-09-21T07:41:27:077658+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6425,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/73220122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":73220122},{"type":"v1","addr":"172.21.15.148:6837","nonce":73220122}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":127,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:41:26.203217+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7452":{"gid":7452,"name":"a","rank":1,"incarnation":122,"state":"up:active","state_seq":8,"addr":"172.21.15.39:6835/3695489158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3695489158},{"type":"v1","addr":"172.21.15.39:6835","nonce":3695489158}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7452,7311]},"id":6}]} 2024-09-21T07:41:28.181 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 127 2024-09-21T07:41:28.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:28.181+0000 7f3eef200640 1 -- 172.21.15.39:0/502694206 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3ed0078b90 msgr2=0x7f3ed007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:28.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:28.181+0000 7f3eef200640 1 --2- 172.21.15.39:0/502694206 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3ed0078b90 0x7f3ed007b0a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0033000 tx=0x7f3ee0004730 comp rx=0 tx=0).stop 2024-09-21T07:41:28.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:28.181+0000 7f3eef200640 1 -- 172.21.15.39:0/502694206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ef00b43f0 msgr2=0x7f3ef009ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:28.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:28.181+0000 7f3eef200640 1 --2- 172.21.15.39:0/502694206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ef00b43f0 0x7f3ef009ab20 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f3edc00d740 tx=0x7f3edc00dc00 comp rx=0 tx=0).stop 2024-09-21T07:41:28.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:28.181+0000 7f3eef200640 1 -- 172.21.15.39:0/502694206 shutdown_connections 2024-09-21T07:41:28.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:28.181+0000 7f3eef200640 1 --2- 172.21.15.39:0/502694206 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3ed0078b90 0x7f3ed007b0a0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:28.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:28.181+0000 7f3eef200640 1 --2- 172.21.15.39:0/502694206 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ef015abe0 0x7f3ef00b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:28.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:28.181+0000 7f3eef200640 1 --2- 172.21.15.39:0/502694206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ef00b43f0 0x7f3ef009ab20 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:28.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:28.181+0000 7f3eef200640 1 --2- 172.21.15.39:0/502694206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ef00af490 0x7f3ef01f73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:28.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:28.181+0000 7f3eef200640 1 -- 172.21.15.39:0/502694206 >> 172.21.15.39:0/502694206 conn(0x7f3ef00b2510 msgr2=0x7f3ef00ae5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:28.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:28.182+0000 7f3eef200640 1 -- 172.21.15.39:0/502694206 shutdown_connections 2024-09-21T07:41:28.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:28.182+0000 7f3eef200640 1 -- 172.21.15.39:0/502694206 wait complete. 2024-09-21T07:41:28.208 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:41:29.425 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-09-21T07:41:31.454 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:41:31.459 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:41:33.209 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:41:33.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.444+0000 7f4a73200640 1 -- 172.21.15.39:0/534533761 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a740d19f0 msgr2=0x7f4a740d4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:33.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.444+0000 7f4a73200640 1 --2- 172.21.15.39:0/534533761 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a740d19f0 0x7f4a740d4600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4a64009850 tx=0x7f4a6402a510 comp rx=0 tx=0).stop 2024-09-21T07:41:33.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.444+0000 7f4a73200640 1 -- 172.21.15.39:0/534533761 shutdown_connections 2024-09-21T07:41:33.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.444+0000 7f4a73200640 1 --2- 172.21.15.39:0/534533761 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a740d19f0 0x7f4a740d4600 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:33.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.444+0000 7f4a73200640 1 -- 172.21.15.39:0/534533761 >> 172.21.15.39:0/534533761 conn(0x7f4a7408bfb0 msgr2=0x7f4a7409d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:33.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.444+0000 7f4a73200640 1 -- 172.21.15.39:0/534533761 shutdown_connections 2024-09-21T07:41:33.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.445+0000 7f4a73200640 1 -- 172.21.15.39:0/534533761 wait complete. 2024-09-21T07:41:33.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.447+0000 7f4a73200640 1 Processor -- start 2024-09-21T07:41:33.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.447+0000 7f4a73200640 1 -- start start 2024-09-21T07:41:33.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.448+0000 7f4a73200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a740b5890 0x7f4a74157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:33.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.448+0000 7f4a73200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a740d19f0 0x7f4a740c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:33.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.449+0000 7f4a73200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a740d4b40 0x7f4a740c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:33.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.449+0000 7f4a73200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4a740b5e20 con 0x7f4a740d19f0 2024-09-21T07:41:33.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.449+0000 7f4a73200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4a740b5ca0 con 0x7f4a740b5890 2024-09-21T07:41:33.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.449+0000 7f4a73200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f4a740b2ab0 con 0x7f4a740d4b40 2024-09-21T07:41:33.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.449+0000 7f4a6be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a740d19f0 0x7f4a740c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:33.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.449+0000 7f4a6be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a740d19f0 0x7f4a740c8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45346/0 (socket says 172.21.15.39:45346) 2024-09-21T07:41:33.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.449+0000 7f4a6be00640 1 -- 172.21.15.39:0/1989646341 learned_addr learned my addr 172.21.15.39:0/1989646341 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:41:33.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.449+0000 7f4a71e00640 1 --2- 172.21.15.39:0/1989646341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a740b5890 0x7f4a74157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:33.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.449+0000 7f4a72800640 1 --2- 172.21.15.39:0/1989646341 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a740d4b40 0x7f4a740c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:33.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.449+0000 7f4a6be00640 1 -- 172.21.15.39:0/1989646341 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a740d4b40 msgr2=0x7f4a740c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:33.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.449+0000 7f4a6be00640 1 --2- 172.21.15.39:0/1989646341 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a740d4b40 0x7f4a740c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:33.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.449+0000 7f4a6be00640 1 -- 172.21.15.39:0/1989646341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a740b5890 msgr2=0x7f4a74157d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:33.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.449+0000 7f4a6be00640 1 --2- 172.21.15.39:0/1989646341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a740b5890 0x7f4a74157d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:33.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.449+0000 7f4a6be00640 1 -- 172.21.15.39:0/1989646341 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a740a2290 con 0x7f4a740d19f0 2024-09-21T07:41:33.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.449+0000 7f4a72800640 1 --2- 172.21.15.39:0/1989646341 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a740d4b40 0x7f4a740c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:41:33.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.449+0000 7f4a71e00640 1 --2- 172.21.15.39:0/1989646341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a740b5890 0x7f4a74157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:41:33.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.450+0000 7f4a6be00640 1 --2- 172.21.15.39:0/1989646341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a740d19f0 0x7f4a740c8560 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c00c760 tx=0x7f4a5c00cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:33.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.450+0000 7f4a6a000640 1 -- 172.21.15.39:0/1989646341 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c00a840 con 0x7f4a740d19f0 2024-09-21T07:41:33.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.450+0000 7f4a6a000640 1 -- 172.21.15.39:0/1989646341 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a5c017070 con 0x7f4a740d19f0 2024-09-21T07:41:33.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.450+0000 7f4a6a000640 1 -- 172.21.15.39:0/1989646341 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c00ad70 con 0x7f4a740d19f0 2024-09-21T07:41:33.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.450+0000 7f4a73200640 1 -- 172.21.15.39:0/1989646341 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a740a04e0 con 0x7f4a740d19f0 2024-09-21T07:41:33.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.450+0000 7f4a73200640 1 -- 172.21.15.39:0/1989646341 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a740a0a60 con 0x7f4a740d19f0 2024-09-21T07:41:33.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.451+0000 7f4a73200640 1 -- 172.21.15.39:0/1989646341 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a74069a50 con 0x7f4a740d19f0 2024-09-21T07:41:33.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.452+0000 7f4a6a000640 1 -- 172.21.15.39:0/1989646341 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4a5c01c460 con 0x7f4a740d19f0 2024-09-21T07:41:33.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.453+0000 7f4a6a000640 1 --2- 172.21.15.39:0/1989646341 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4a50078b90 0x7f4a5007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:33.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.453+0000 7f4a6a000640 1 -- 172.21.15.39:0/1989646341 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7874+0+0 (secure 0 0 0) 0x7f4a5c09b6d0 con 0x7f4a740d19f0 2024-09-21T07:41:33.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.454+0000 7f4a71e00640 1 --2- 172.21.15.39:0/1989646341 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4a50078b90 0x7f4a5007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:33.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.454+0000 7f4a71e00640 1 --2- 172.21.15.39:0/1989646341 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4a50078b90 0x7f4a5007b0a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4a64034000 tx=0x7f4a64004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:33.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.458+0000 7f4a6a000640 1 -- 172.21.15.39:0/1989646341 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4a740d4f50 con 0x7f4a740d19f0 2024-09-21T07:41:33.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.692+0000 7f4a73200640 1 -- 172.21.15.39:0/1989646341 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a740a0ca0 con 0x7f4a740d19f0 2024-09-21T07:41:33.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.693+0000 7f4a6a000640 1 -- 172.21.15.39:0/1989646341 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 127 v127) ==== 95+0+5252 (secure 0 0 0) 0x7f4a740a0ca0 con 0x7f4a740d19f0 2024-09-21T07:41:33.695 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:41:33.695 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":127,"btime":"2024-09-21T07:41:27:077658+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6425,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/73220122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":73220122},{"type":"v1","addr":"172.21.15.148:6837","nonce":73220122}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":127,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:41:26.203217+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7452":{"gid":7452,"name":"a","rank":1,"incarnation":122,"state":"up:active","state_seq":8,"addr":"172.21.15.39:6835/3695489158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3695489158},{"type":"v1","addr":"172.21.15.39:6835","nonce":3695489158}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7452,7311]},"id":6}]} 2024-09-21T07:41:33.695 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 127 2024-09-21T07:41:33.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.695+0000 7f4a73200640 1 -- 172.21.15.39:0/1989646341 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4a50078b90 msgr2=0x7f4a5007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:33.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.695+0000 7f4a73200640 1 --2- 172.21.15.39:0/1989646341 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4a50078b90 0x7f4a5007b0a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4a64034000 tx=0x7f4a64004730 comp rx=0 tx=0).stop 2024-09-21T07:41:33.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.695+0000 7f4a73200640 1 -- 172.21.15.39:0/1989646341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a740d19f0 msgr2=0x7f4a740c8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:33.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.695+0000 7f4a73200640 1 --2- 172.21.15.39:0/1989646341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a740d19f0 0x7f4a740c8560 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c00c760 tx=0x7f4a5c00cb60 comp rx=0 tx=0).stop 2024-09-21T07:41:33.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.695+0000 7f4a73200640 1 -- 172.21.15.39:0/1989646341 shutdown_connections 2024-09-21T07:41:33.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.695+0000 7f4a73200640 1 --2- 172.21.15.39:0/1989646341 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4a50078b90 0x7f4a5007b0a0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:33.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.695+0000 7f4a73200640 1 --2- 172.21.15.39:0/1989646341 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a740d4b40 0x7f4a740c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:33.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.695+0000 7f4a73200640 1 --2- 172.21.15.39:0/1989646341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a740d19f0 0x7f4a740c8560 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:33.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.695+0000 7f4a73200640 1 --2- 172.21.15.39:0/1989646341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a740b5890 0x7f4a74157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:33.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.695+0000 7f4a73200640 1 -- 172.21.15.39:0/1989646341 >> 172.21.15.39:0/1989646341 conn(0x7f4a7408bfb0 msgr2=0x7f4a74068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:33.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.695+0000 7f4a73200640 1 -- 172.21.15.39:0/1989646341 shutdown_connections 2024-09-21T07:41:33.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:33.695+0000 7f4a73200640 1 -- 172.21.15.39:0/1989646341 wait complete. 2024-09-21T07:41:33.720 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:41:36.038 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~7s 2024-09-21T07:41:38.721 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:41:38.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:38.957+0000 7fb9a2600640 1 -- 172.21.15.39:0/1695381547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb99c0b4db0 msgr2=0x7fb99c0d4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:38.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:38.957+0000 7fb9a2600640 1 --2- 172.21.15.39:0/1695381547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb99c0b4db0 0x7fb99c0d4c90 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fb99c08d830 tx=0x7fb99002a540 comp rx=0 tx=0).stop 2024-09-21T07:41:38.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:38.957+0000 7fb9a2600640 1 -- 172.21.15.39:0/1695381547 shutdown_connections 2024-09-21T07:41:38.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:38.957+0000 7fb9a2600640 1 --2- 172.21.15.39:0/1695381547 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb99c0b6000 0x7fb99c0b6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:38.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:38.957+0000 7fb9a2600640 1 --2- 172.21.15.39:0/1695381547 >> v2:172.21.15.148:3300/0 conn(0x7fb99c0d2a50 0x7fb99c0d2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:38.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:38.957+0000 7fb9a2600640 1 --2- 172.21.15.39:0/1695381547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb99c0b4db0 0x7fb99c0d4c90 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:38.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:38.957+0000 7fb9a2600640 1 -- 172.21.15.39:0/1695381547 >> 172.21.15.39:0/1695381547 conn(0x7fb99c08bfb0 msgr2=0x7fb99c0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:38.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:38.957+0000 7fb9a2600640 1 -- 172.21.15.39:0/1695381547 shutdown_connections 2024-09-21T07:41:38.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:38.957+0000 7fb9a2600640 1 -- 172.21.15.39:0/1695381547 wait complete. 2024-09-21T07:41:38.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:38.960+0000 7fb9a2600640 1 Processor -- start 2024-09-21T07:41:38.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:38.960+0000 7fb9a2600640 1 -- start start 2024-09-21T07:41:38.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:38.961+0000 7fb9a2600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb99c0b4db0 0x7fb99c0cd350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:38.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:38.961+0000 7fb9a2600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb99c0b6000 0x7fb99c1f8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:38.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:38.961+0000 7fb9a2600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb99c0d2a50 0x7fb99c1f9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:38.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:38.961+0000 7fb9a2600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb99c0b6ce0 con 0x7fb99c0b6000 2024-09-21T07:41:38.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:38.961+0000 7fb9a2600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb99c0b6b60 con 0x7fb99c0b4db0 2024-09-21T07:41:38.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:38.961+0000 7fb9a2600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb99c0b6e60 con 0x7fb99c0d2a50 2024-09-21T07:41:38.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:38.961+0000 7fb99be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb99c0b6000 0x7fb99c1f8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:38.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:38.962+0000 7fb99be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb99c0b6000 0x7fb99c1f8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45362/0 (socket says 172.21.15.39:45362) 2024-09-21T07:41:38.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:38.962+0000 7fb99be00640 1 -- 172.21.15.39:0/3239254992 learned_addr learned my addr 172.21.15.39:0/3239254992 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:41:38.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:38.962+0000 7fb9a1c00640 1 --2- 172.21.15.39:0/3239254992 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb99c0d2a50 0x7fb99c1f9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:38.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:38.962+0000 7fb9a1200640 1 --2- 172.21.15.39:0/3239254992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb99c0b4db0 0x7fb99c0cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:38.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:38.962+0000 7fb99be00640 1 -- 172.21.15.39:0/3239254992 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb99c0d2a50 msgr2=0x7fb99c1f9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:38.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:38.962+0000 7fb99be00640 1 --2- 172.21.15.39:0/3239254992 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb99c0d2a50 0x7fb99c1f9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:38.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:38.962+0000 7fb99be00640 1 -- 172.21.15.39:0/3239254992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb99c0b4db0 msgr2=0x7fb99c0cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:38.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:38.962+0000 7fb99be00640 1 --2- 172.21.15.39:0/3239254992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb99c0b4db0 0x7fb99c0cd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:38.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:38.962+0000 7fb99be00640 1 -- 172.21.15.39:0/3239254992 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb99c0cdaf0 con 0x7fb99c0b6000 2024-09-21T07:41:38.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:38.962+0000 7fb99be00640 1 --2- 172.21.15.39:0/3239254992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb99c0b6000 0x7fb99c1f8bd0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fb988005b60 tx=0x7fb98800daf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:38.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:38.963+0000 7fb998c00640 1 -- 172.21.15.39:0/3239254992 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb988004220 con 0x7fb99c0b6000 2024-09-21T07:41:38.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:38.963+0000 7fb998c00640 1 -- 172.21.15.39:0/3239254992 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9880044b0 con 0x7fb99c0b6000 2024-09-21T07:41:38.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:38.963+0000 7fb998c00640 1 -- 172.21.15.39:0/3239254992 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb988005270 con 0x7fb99c0b6000 2024-09-21T07:41:38.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:38.963+0000 7fb9a2600640 1 -- 172.21.15.39:0/3239254992 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb99c15f670 con 0x7fb99c0b6000 2024-09-21T07:41:38.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:38.963+0000 7fb9a2600640 1 -- 172.21.15.39:0/3239254992 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb99c0cddd0 con 0x7fb99c0b6000 2024-09-21T07:41:38.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:38.964+0000 7fb9a2600640 1 -- 172.21.15.39:0/3239254992 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb99c0b6410 con 0x7fb99c0b6000 2024-09-21T07:41:38.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:38.965+0000 7fb998c00640 1 -- 172.21.15.39:0/3239254992 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb988005410 con 0x7fb99c0b6000 2024-09-21T07:41:38.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:38.966+0000 7fb998c00640 1 --2- 172.21.15.39:0/3239254992 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb97c078b90 0x7fb97c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:38.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:38.966+0000 7fb998c00640 1 -- 172.21.15.39:0/3239254992 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7846+0+0 (secure 0 0 0) 0x7fb98809b9f0 con 0x7fb99c0b6000 2024-09-21T07:41:38.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:38.966+0000 7fb9a1200640 1 --2- 172.21.15.39:0/3239254992 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb97c078b90 0x7fb97c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:38.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:38.967+0000 7fb9a1200640 1 --2- 172.21.15.39:0/3239254992 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb97c078b90 0x7fb97c07b0a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb990033000 tx=0x7fb990004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:38.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:38.971+0000 7fb998c00640 1 -- 172.21.15.39:0/3239254992 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb988020020 con 0x7fb99c0b6000 2024-09-21T07:41:39.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:39.204+0000 7fb9a2600640 1 -- 172.21.15.39:0/3239254992 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb99c15f340 con 0x7fb99c0b6000 2024-09-21T07:41:39.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:39.205+0000 7fb998c00640 1 -- 172.21.15.39:0/3239254992 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 127 v127) ==== 95+0+5252 (secure 0 0 0) 0x7fb988017030 con 0x7fb99c0b6000 2024-09-21T07:41:39.208 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:41:39.208 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":127,"btime":"2024-09-21T07:41:27:077658+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6425,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/73220122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":73220122},{"type":"v1","addr":"172.21.15.148:6837","nonce":73220122}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":127,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:41:26.203217+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7452":{"gid":7452,"name":"a","rank":1,"incarnation":122,"state":"up:active","state_seq":8,"addr":"172.21.15.39:6835/3695489158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3695489158},{"type":"v1","addr":"172.21.15.39:6835","nonce":3695489158}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7452,7311]},"id":6}]} 2024-09-21T07:41:39.208 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 127 2024-09-21T07:41:39.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:39.208+0000 7fb9a2600640 1 -- 172.21.15.39:0/3239254992 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb97c078b90 msgr2=0x7fb97c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:39.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:39.208+0000 7fb9a2600640 1 --2- 172.21.15.39:0/3239254992 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb97c078b90 0x7fb97c07b0a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb990033000 tx=0x7fb990004730 comp rx=0 tx=0).stop 2024-09-21T07:41:39.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:39.208+0000 7fb9a2600640 1 -- 172.21.15.39:0/3239254992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb99c0b6000 msgr2=0x7fb99c1f8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:39.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:39.208+0000 7fb9a2600640 1 --2- 172.21.15.39:0/3239254992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb99c0b6000 0x7fb99c1f8bd0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fb988005b60 tx=0x7fb98800daf0 comp rx=0 tx=0).stop 2024-09-21T07:41:39.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:39.208+0000 7fb9a2600640 1 -- 172.21.15.39:0/3239254992 shutdown_connections 2024-09-21T07:41:39.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:39.208+0000 7fb9a2600640 1 --2- 172.21.15.39:0/3239254992 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb97c078b90 0x7fb97c07b0a0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:39.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:39.208+0000 7fb9a2600640 1 --2- 172.21.15.39:0/3239254992 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb99c0d2a50 0x7fb99c1f9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:39.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:39.208+0000 7fb9a2600640 1 --2- 172.21.15.39:0/3239254992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb99c0b6000 0x7fb99c1f8bd0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:39.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:39.208+0000 7fb9a2600640 1 --2- 172.21.15.39:0/3239254992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb99c0b4db0 0x7fb99c0cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:39.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:39.208+0000 7fb9a2600640 1 -- 172.21.15.39:0/3239254992 >> 172.21.15.39:0/3239254992 conn(0x7fb99c08bfb0 msgr2=0x7fb99c06f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:39.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:39.208+0000 7fb9a2600640 1 -- 172.21.15.39:0/3239254992 shutdown_connections 2024-09-21T07:41:39.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:39.208+0000 7fb9a2600640 1 -- 172.21.15.39:0/3239254992 wait complete. 2024-09-21T07:41:39.235 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:41:42.651 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~13s 2024-09-21T07:41:44.236 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:41:44.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.472+0000 7f7bd9200640 1 -- 172.21.15.39:0/2409548429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bd40d0620 msgr2=0x7f7bd40d0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:44.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.472+0000 7f7bd9200640 1 --2- 172.21.15.39:0/2409548429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bd40d0620 0x7f7bd40d0a30 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc005fd0 tx=0x7f7bcc02a9f0 comp rx=0 tx=0).stop 2024-09-21T07:41:44.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.472+0000 7f7bd9200640 1 -- 172.21.15.39:0/2409548429 shutdown_connections 2024-09-21T07:41:44.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.472+0000 7f7bd9200640 1 --2- 172.21.15.39:0/2409548429 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7bd40cfd10 0x7f7bd40cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:44.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.472+0000 7f7bd9200640 1 --2- 172.21.15.39:0/2409548429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bd40d0620 0x7f7bd40d0a30 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:44.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.472+0000 7f7bd9200640 1 -- 172.21.15.39:0/2409548429 >> 172.21.15.39:0/2409548429 conn(0x7f7bd406cfc0 msgr2=0x7f7bd406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:44.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.472+0000 7f7bd9200640 1 -- 172.21.15.39:0/2409548429 shutdown_connections 2024-09-21T07:41:44.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.472+0000 7f7bd9200640 1 -- 172.21.15.39:0/2409548429 wait complete. 2024-09-21T07:41:44.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.475+0000 7f7bd9200640 1 Processor -- start 2024-09-21T07:41:44.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.475+0000 7f7bd9200640 1 -- start start 2024-09-21T07:41:44.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.476+0000 7f7bd9200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bd40cfd10 0x7f7bd4159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:44.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.476+0000 7f7bd9200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7bd40d0620 0x7f7bd41f2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:44.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.476+0000 7f7bd9200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bd415cd30 0x7f7bd41e7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:44.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.476+0000 7f7bd9200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd4073890 con 0x7f7bd415cd30 2024-09-21T07:41:44.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.476+0000 7f7bd9200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd4073710 con 0x7f7bd40cfd10 2024-09-21T07:41:44.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.476+0000 7f7bd9200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f7bd4073a10 con 0x7f7bd40d0620 2024-09-21T07:41:44.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.477+0000 7f7bd2800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bd415cd30 0x7f7bd41e7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:44.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.477+0000 7f7bd2800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bd415cd30 0x7f7bd41e7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45956/0 (socket says 172.21.15.39:45956) 2024-09-21T07:41:44.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.477+0000 7f7bd2800640 1 -- 172.21.15.39:0/4161079885 learned_addr learned my addr 172.21.15.39:0/4161079885 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:41:44.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.477+0000 7f7bd1e00640 1 --2- 172.21.15.39:0/4161079885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bd40cfd10 0x7f7bd4159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:44.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.477+0000 7f7bd1400640 1 --2- 172.21.15.39:0/4161079885 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7bd40d0620 0x7f7bd41f2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:44.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.477+0000 7f7bd2800640 1 -- 172.21.15.39:0/4161079885 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7bd40d0620 msgr2=0x7f7bd41f2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:44.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.477+0000 7f7bd2800640 1 --2- 172.21.15.39:0/4161079885 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7bd40d0620 0x7f7bd41f2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:44.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.477+0000 7f7bd2800640 1 -- 172.21.15.39:0/4161079885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bd40cfd10 msgr2=0x7f7bd4159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:44.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.477+0000 7f7bd2800640 1 --2- 172.21.15.39:0/4161079885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bd40cfd10 0x7f7bd4159810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:44.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.477+0000 7f7bd2800640 1 -- 172.21.15.39:0/4161079885 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bd4154820 con 0x7f7bd415cd30 2024-09-21T07:41:44.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.477+0000 7f7bd2800640 1 --2- 172.21.15.39:0/4161079885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bd415cd30 0x7f7bd41e7a60 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc006010 tx=0x7f7bcc003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:44.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.477+0000 7f7bca000640 1 -- 172.21.15.39:0/4161079885 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc046070 con 0x7f7bd415cd30 2024-09-21T07:41:44.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.478+0000 7f7bca000640 1 -- 172.21.15.39:0/4161079885 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7bcc004640 con 0x7f7bd415cd30 2024-09-21T07:41:44.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.478+0000 7f7bca000640 1 -- 172.21.15.39:0/4161079885 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc033070 con 0x7f7bd415cd30 2024-09-21T07:41:44.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.478+0000 7f7bd9200640 1 -- 172.21.15.39:0/4161079885 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bd4158a50 con 0x7f7bd415cd30 2024-09-21T07:41:44.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.478+0000 7f7bd9200640 1 -- 172.21.15.39:0/4161079885 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bd4159010 con 0x7f7bd415cd30 2024-09-21T07:41:44.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.479+0000 7f7bd9200640 1 -- 172.21.15.39:0/4161079885 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bd4154590 con 0x7f7bd415cd30 2024-09-21T07:41:44.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.480+0000 7f7bca000640 1 -- 172.21.15.39:0/4161079885 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7bcc049070 con 0x7f7bd415cd30 2024-09-21T07:41:44.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.481+0000 7f7bca000640 1 --2- 172.21.15.39:0/4161079885 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7bb4078b90 0x7f7bb407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:44.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.481+0000 7f7bca000640 1 -- 172.21.15.39:0/4161079885 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7846+0+0 (secure 0 0 0) 0x7f7bcc0bc860 con 0x7f7bd415cd30 2024-09-21T07:41:44.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.482+0000 7f7bd1e00640 1 --2- 172.21.15.39:0/4161079885 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7bb4078b90 0x7f7bb407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:44.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.483+0000 7f7bd1e00640 1 --2- 172.21.15.39:0/4161079885 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7bb4078b90 0x7f7bb407b0a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4006fd0 tx=0x7f7bc4008040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:44.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.486+0000 7f7bca000640 1 -- 172.21.15.39:0/4161079885 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7bcc03e070 con 0x7f7bd415cd30 2024-09-21T07:41:44.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.721+0000 7f7bd9200640 1 -- 172.21.15.39:0/4161079885 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7bd4073fb0 con 0x7f7bd415cd30 2024-09-21T07:41:44.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.721+0000 7f7bca000640 1 -- 172.21.15.39:0/4161079885 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 127 v127) ==== 95+0+5252 (secure 0 0 0) 0x7f7bcc084430 con 0x7f7bd415cd30 2024-09-21T07:41:44.722 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:41:44.723 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":127,"btime":"2024-09-21T07:41:27:077658+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6425,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/73220122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":73220122},{"type":"v1","addr":"172.21.15.148:6837","nonce":73220122}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":127,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:41:26.203217+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7452":{"gid":7452,"name":"a","rank":1,"incarnation":122,"state":"up:active","state_seq":8,"addr":"172.21.15.39:6835/3695489158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3695489158},{"type":"v1","addr":"172.21.15.39:6835","nonce":3695489158}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7452,7311]},"id":6}]} 2024-09-21T07:41:44.723 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 127 2024-09-21T07:41:44.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.724+0000 7f7bd9200640 1 -- 172.21.15.39:0/4161079885 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7bb4078b90 msgr2=0x7f7bb407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:44.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.724+0000 7f7bd9200640 1 --2- 172.21.15.39:0/4161079885 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7bb4078b90 0x7f7bb407b0a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4006fd0 tx=0x7f7bc4008040 comp rx=0 tx=0).stop 2024-09-21T07:41:44.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.724+0000 7f7bd9200640 1 -- 172.21.15.39:0/4161079885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bd415cd30 msgr2=0x7f7bd41e7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:44.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.724+0000 7f7bd9200640 1 --2- 172.21.15.39:0/4161079885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bd415cd30 0x7f7bd41e7a60 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc006010 tx=0x7f7bcc003f10 comp rx=0 tx=0).stop 2024-09-21T07:41:44.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.724+0000 7f7bd9200640 1 -- 172.21.15.39:0/4161079885 shutdown_connections 2024-09-21T07:41:44.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.724+0000 7f7bd9200640 1 --2- 172.21.15.39:0/4161079885 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7bb4078b90 0x7f7bb407b0a0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:44.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.724+0000 7f7bd9200640 1 --2- 172.21.15.39:0/4161079885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7bd415cd30 0x7f7bd41e7a60 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:44.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.724+0000 7f7bd9200640 1 --2- 172.21.15.39:0/4161079885 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7bd40d0620 0x7f7bd41f2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:44.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.724+0000 7f7bd9200640 1 --2- 172.21.15.39:0/4161079885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7bd40cfd10 0x7f7bd4159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:44.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.724+0000 7f7bd9200640 1 -- 172.21.15.39:0/4161079885 >> 172.21.15.39:0/4161079885 conn(0x7f7bd406cfc0 msgr2=0x7f7bd40afe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:44.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.724+0000 7f7bd9200640 1 -- 172.21.15.39:0/4161079885 shutdown_connections 2024-09-21T07:41:44.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:44.724+0000 7f7bd9200640 1 -- 172.21.15.39:0/4161079885 wait complete. 2024-09-21T07:41:44.750 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:41:49.261 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~20s 2024-09-21T07:41:49.751 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:41:49.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:49.987+0000 7f9c6ee00640 1 -- 172.21.15.39:0/4145932835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c680cc390 msgr2=0x7f9c680af600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:49.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:49.987+0000 7f9c6ee00640 1 --2- 172.21.15.39:0/4145932835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c680cc390 0x7f9c680af600 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c009830 tx=0x7f9c5c02a420 comp rx=0 tx=0).stop 2024-09-21T07:41:49.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:49.987+0000 7f9c6ee00640 1 -- 172.21.15.39:0/4145932835 shutdown_connections 2024-09-21T07:41:49.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:49.987+0000 7f9c6ee00640 1 --2- 172.21.15.39:0/4145932835 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9c680ce310 0x7f9c680ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:49.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:49.987+0000 7f9c6ee00640 1 --2- 172.21.15.39:0/4145932835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c680cc390 0x7f9c680af600 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:49.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:49.987+0000 7f9c6ee00640 1 -- 172.21.15.39:0/4145932835 >> 172.21.15.39:0/4145932835 conn(0x7f9c680b2510 msgr2=0x7f9c680d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:49.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:49.987+0000 7f9c6ee00640 1 -- 172.21.15.39:0/4145932835 shutdown_connections 2024-09-21T07:41:49.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:49.987+0000 7f9c6ee00640 1 -- 172.21.15.39:0/4145932835 wait complete. 2024-09-21T07:41:49.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:49.989+0000 7f9c6ee00640 1 Processor -- start 2024-09-21T07:41:49.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:49.990+0000 7f9c6ee00640 1 -- start start 2024-09-21T07:41:49.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:49.991+0000 7f9c6ee00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c6806f000 0x7f9c680c4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:49.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:49.991+0000 7f9c6ee00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9c680cc390 0x7f9c681f8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:49.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:49.991+0000 7f9c6ee00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c680ce310 0x7f9c680c2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:49.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:49.991+0000 7f9c6ee00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9c68070020 con 0x7f9c680ce310 2024-09-21T07:41:49.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:49.991+0000 7f9c6ee00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9c6806fea0 con 0x7f9c6806f000 2024-09-21T07:41:49.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:49.991+0000 7f9c6ee00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f9c680701a0 con 0x7f9c680cc390 2024-09-21T07:41:49.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:49.991+0000 7f9c6e400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c680ce310 0x7f9c680c2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:49.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:49.992+0000 7f9c6e400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c680ce310 0x7f9c680c2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45964/0 (socket says 172.21.15.39:45964) 2024-09-21T07:41:49.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:49.992+0000 7f9c6e400640 1 -- 172.21.15.39:0/1072576552 learned_addr learned my addr 172.21.15.39:0/1072576552 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:41:49.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:49.992+0000 7f9c6da00640 1 --2- 172.21.15.39:0/1072576552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c6806f000 0x7f9c680c4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:49.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:49.992+0000 7f9c6d000640 1 --2- 172.21.15.39:0/1072576552 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9c680cc390 0x7f9c681f8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:49.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:49.992+0000 7f9c6e400640 1 -- 172.21.15.39:0/1072576552 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9c680cc390 msgr2=0x7f9c681f8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:49.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:49.992+0000 7f9c6e400640 1 --2- 172.21.15.39:0/1072576552 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9c680cc390 0x7f9c681f8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:49.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:49.992+0000 7f9c6e400640 1 -- 172.21.15.39:0/1072576552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c6806f000 msgr2=0x7f9c680c4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:49.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:49.992+0000 7f9c6e400640 1 --2- 172.21.15.39:0/1072576552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c6806f000 0x7f9c680c4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:49.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:49.992+0000 7f9c6e400640 1 -- 172.21.15.39:0/1072576552 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c6809a4f0 con 0x7f9c680ce310 2024-09-21T07:41:49.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:49.992+0000 7f9c6e400640 1 --2- 172.21.15.39:0/1072576552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c680ce310 0x7f9c680c2cb0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f9c6000cbb0 tx=0x7f9c6000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:49.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:49.993+0000 7f9c65600640 1 -- 172.21.15.39:0/1072576552 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c60016020 con 0x7f9c680ce310 2024-09-21T07:41:49.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:49.993+0000 7f9c65600640 1 -- 172.21.15.39:0/1072576552 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c6000ae50 con 0x7f9c680ce310 2024-09-21T07:41:49.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:49.993+0000 7f9c65600640 1 -- 172.21.15.39:0/1072576552 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c60012680 con 0x7f9c680ce310 2024-09-21T07:41:49.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:49.993+0000 7f9c6ee00640 1 -- 172.21.15.39:0/1072576552 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c6809bed0 con 0x7f9c680ce310 2024-09-21T07:41:49.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:49.993+0000 7f9c6ee00640 1 -- 172.21.15.39:0/1072576552 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c6809da30 con 0x7f9c680ce310 2024-09-21T07:41:49.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:49.994+0000 7f9c6ee00640 1 -- 172.21.15.39:0/1072576552 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c6809dc90 con 0x7f9c680ce310 2024-09-21T07:41:49.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:49.995+0000 7f9c65600640 1 -- 172.21.15.39:0/1072576552 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9c60019020 con 0x7f9c680ce310 2024-09-21T07:41:49.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:49.996+0000 7f9c65600640 1 --2- 172.21.15.39:0/1072576552 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9c48078b90 0x7f9c4807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:49.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:49.996+0000 7f9c65600640 1 -- 172.21.15.39:0/1072576552 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7846+0+0 (secure 0 0 0) 0x7f9c6009bc00 con 0x7f9c680ce310 2024-09-21T07:41:49.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:49.997+0000 7f9c6da00640 1 --2- 172.21.15.39:0/1072576552 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9c48078b90 0x7f9c4807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:49.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:49.997+0000 7f9c6da00640 1 --2- 172.21.15.39:0/1072576552 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9c48078b90 0x7f9c4807b0a0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c034000 tx=0x7f9c5c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:50.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.001+0000 7f9c65600640 1 -- 172.21.15.39:0/1072576552 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9c60020020 con 0x7f9c680ce310 2024-09-21T07:41:50.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.235+0000 7f9c6ee00640 1 -- 172.21.15.39:0/1072576552 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c680cbc90 con 0x7f9c680ce310 2024-09-21T07:41:50.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.235+0000 7f9c65600640 1 -- 172.21.15.39:0/1072576552 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 128 v128) ==== 95+0+5300 (secure 0 0 0) 0x7f9c6006ae20 con 0x7f9c680ce310 2024-09-21T07:41:50.236 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:41:50.237 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":128,"btime":"2024-09-21T07:41:46:435551+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6425,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/73220122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":73220122},{"type":"v1","addr":"172.21.15.148:6837","nonce":73220122}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:41:46.435521+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7452":{"gid":7452,"name":"a","rank":1,"incarnation":122,"state":"up:active","state_seq":8,"addr":"172.21.15.39:6835/3695489158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3695489158},{"type":"v1","addr":"172.21.15.39:6835","nonce":3695489158}]},"join_fscid":-1,"laggy_since":"2024-09-21T07:41:46.435521+0000","export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311,7452]},"id":6}]} 2024-09-21T07:41:50.237 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 128 2024-09-21T07:41:50.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.238+0000 7f9c6ee00640 1 -- 172.21.15.39:0/1072576552 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9c48078b90 msgr2=0x7f9c4807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:50.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.238+0000 7f9c6ee00640 1 --2- 172.21.15.39:0/1072576552 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9c48078b90 0x7f9c4807b0a0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c034000 tx=0x7f9c5c004730 comp rx=0 tx=0).stop 2024-09-21T07:41:50.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.238+0000 7f9c6ee00640 1 -- 172.21.15.39:0/1072576552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c680ce310 msgr2=0x7f9c680c2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:50.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.238+0000 7f9c6ee00640 1 --2- 172.21.15.39:0/1072576552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c680ce310 0x7f9c680c2cb0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f9c6000cbb0 tx=0x7f9c6000a430 comp rx=0 tx=0).stop 2024-09-21T07:41:50.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.238+0000 7f9c6ee00640 1 -- 172.21.15.39:0/1072576552 shutdown_connections 2024-09-21T07:41:50.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.238+0000 7f9c6ee00640 1 --2- 172.21.15.39:0/1072576552 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9c48078b90 0x7f9c4807b0a0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:50.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.238+0000 7f9c6ee00640 1 --2- 172.21.15.39:0/1072576552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c680ce310 0x7f9c680c2cb0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:50.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.238+0000 7f9c6ee00640 1 --2- 172.21.15.39:0/1072576552 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9c680cc390 0x7f9c681f8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:50.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.238+0000 7f9c6ee00640 1 --2- 172.21.15.39:0/1072576552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c6806f000 0x7f9c680c4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:50.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.238+0000 7f9c6ee00640 1 -- 172.21.15.39:0/1072576552 >> 172.21.15.39:0/1072576552 conn(0x7f9c680b2510 msgr2=0x7f9c6815b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:50.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.238+0000 7f9c6ee00640 1 -- 172.21.15.39:0/1072576552 shutdown_connections 2024-09-21T07:41:50.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.238+0000 7f9c6ee00640 1 -- 172.21.15.39:0/1072576552 wait complete. 2024-09-21T07:41:50.264 DEBUG:tasks.ceph_test_case:wait_until_true: success in 20s and 0 retries 2024-09-21T07:41:50.264 DEBUG:teuthology.orchestra.run.smithi039:> sudo sysctl -n kernel.core_pattern 2024-09-21T07:41:50.316 INFO:teuthology.orchestra.run.smithi039.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T07:41:50.319 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2024-09-21T07:41:50.319 DEBUG:teuthology.orchestra.run.smithi039:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2024-09-21T07:41:50.381 INFO:teuthology.orchestra.run.smithi039.stdout:1726904487.65239.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i a', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2024-09-21T07:41:50.384 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1726904487.65239.core'] 2024-09-21T07:41:50.385 INFO:tasks.cephfs.cephfs_test_case:Found core file 1726904487.65239.core, deleting it 2024-09-21T07:41:50.385 DEBUG:teuthology.orchestra.run.smithi039:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1726904487.65239.core 2024-09-21T07:41:50.478 INFO:tasks.cephfs.mount:Suspending the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:41:50.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:41:50.478 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 down 2024-09-21T07:41:50.541 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:41:50.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.766+0000 7f07ae400640 1 -- 172.21.15.39:0/555755194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07a80cbef0 msgr2=0x7f07a80b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:50.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.766+0000 7f07ae400640 1 --2- 172.21.15.39:0/555755194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07a80cbef0 0x7f07a80b30d0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f07a8071260 tx=0x7f07a002a980 comp rx=0 tx=0).stop 2024-09-21T07:41:50.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.766+0000 7f07ae400640 1 -- 172.21.15.39:0/555755194 shutdown_connections 2024-09-21T07:41:50.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.766+0000 7f07ae400640 1 --2- 172.21.15.39:0/555755194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07a80cbef0 0x7f07a80b30d0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:50.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.766+0000 7f07ae400640 1 --2- 172.21.15.39:0/555755194 >> v2:172.21.15.148:3300/0 conn(0x7f07a8160860 0x7f07a80cb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:50.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.766+0000 7f07ae400640 1 --2- 172.21.15.39:0/555755194 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f07a815fc30 0x7f07a8160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:50.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.766+0000 7f07ae400640 1 -- 172.21.15.39:0/555755194 >> 172.21.15.39:0/555755194 conn(0x7f07a80b2510 msgr2=0x7f07a80d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:50.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.766+0000 7f07ae400640 1 -- 172.21.15.39:0/555755194 shutdown_connections 2024-09-21T07:41:50.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.766+0000 7f07ae400640 1 -- 172.21.15.39:0/555755194 wait complete. 2024-09-21T07:41:50.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.768+0000 7f07ae400640 1 Processor -- start 2024-09-21T07:41:50.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.768+0000 7f07ae400640 1 -- start start 2024-09-21T07:41:50.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.770+0000 7f07ae400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07a80cbef0 0x7f07a81f3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:50.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.770+0000 7f07ae400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f07a815fc30 0x7f07a80c9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:50.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.770+0000 7f07ae400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07a8160860 0x7f07a80c72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:50.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.770+0000 7f07ae400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f07a80b3790 con 0x7f07a8160860 2024-09-21T07:41:50.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.770+0000 7f07ae400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f07a80b3610 con 0x7f07a80cbef0 2024-09-21T07:41:50.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.770+0000 7f07ae400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f07a80b3910 con 0x7f07a815fc30 2024-09-21T07:41:50.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.770+0000 7f07ada00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07a8160860 0x7f07a80c72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:50.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.770+0000 7f07ada00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07a8160860 0x7f07a80c72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45982/0 (socket says 172.21.15.39:45982) 2024-09-21T07:41:50.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.770+0000 7f07ada00640 1 -- 172.21.15.39:0/3534557494 learned_addr learned my addr 172.21.15.39:0/3534557494 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:41:50.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.770+0000 7f07ad000640 1 --2- 172.21.15.39:0/3534557494 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07a80cbef0 0x7f07a81f3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:50.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.770+0000 7f079fe00640 1 --2- 172.21.15.39:0/3534557494 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f07a815fc30 0x7f07a80c9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:50.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.770+0000 7f07ada00640 1 -- 172.21.15.39:0/3534557494 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f07a815fc30 msgr2=0x7f07a80c9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:50.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.770+0000 7f07ada00640 1 --2- 172.21.15.39:0/3534557494 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f07a815fc30 0x7f07a80c9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:50.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.771+0000 7f07ada00640 1 -- 172.21.15.39:0/3534557494 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07a80cbef0 msgr2=0x7f07a81f3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:50.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.771+0000 7f07ada00640 1 --2- 172.21.15.39:0/3534557494 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07a80cbef0 0x7f07a81f3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:50.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.771+0000 7f07ada00640 1 -- 172.21.15.39:0/3534557494 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07a81f3c90 con 0x7f07a8160860 2024-09-21T07:41:50.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.771+0000 7f07ada00640 1 --2- 172.21.15.39:0/3534557494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07a8160860 0x7f07a80c72d0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f07a0009d60 tx=0x7f07a0003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:50.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.771+0000 7f07a5600640 1 -- 172.21.15.39:0/3534557494 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a0034040 con 0x7f07a8160860 2024-09-21T07:41:50.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.771+0000 7f07a5600640 1 -- 172.21.15.39:0/3534557494 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07a0004640 con 0x7f07a8160860 2024-09-21T07:41:50.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.771+0000 7f07ae400640 1 -- 172.21.15.39:0/3534557494 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07a80c9c70 con 0x7f07a8160860 2024-09-21T07:41:50.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.771+0000 7f07a5600640 1 -- 172.21.15.39:0/3534557494 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a0045480 con 0x7f07a8160860 2024-09-21T07:41:50.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.771+0000 7f07ae400640 1 -- 172.21.15.39:0/3534557494 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07a81f4480 con 0x7f07a8160860 2024-09-21T07:41:50.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.773+0000 7f07ae400640 1 -- 172.21.15.39:0/3534557494 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07a8076960 con 0x7f07a8160860 2024-09-21T07:41:50.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.774+0000 7f07a5600640 1 -- 172.21.15.39:0/3534557494 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f07a0037020 con 0x7f07a8160860 2024-09-21T07:41:50.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.775+0000 7f07a5600640 1 --2- 172.21.15.39:0/3534557494 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0784078b90 0x7f078407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:50.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.775+0000 7f07a5600640 1 -- 172.21.15.39:0/3534557494 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7846+0+0 (secure 0 0 0) 0x7f07a00bce60 con 0x7f07a8160860 2024-09-21T07:41:50.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.775+0000 7f07ad000640 1 --2- 172.21.15.39:0/3534557494 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0784078b90 0x7f078407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:50.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.776+0000 7f07ad000640 1 --2- 172.21.15.39:0/3534557494 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0784078b90 0x7f078407b0a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f079800a4d0 tx=0x7f079800a460 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:50.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:50.779+0000 7f07a5600640 1 -- 172.21.15.39:0/3534557494 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f07a8076960 con 0x7f07a8160860 2024-09-21T07:41:51.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.019+0000 7f07ae400640 1 -- 172.21.15.39:0/3534557494 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07a81f3f30 con 0x7f07a8160860 2024-09-21T07:41:51.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.019+0000 7f07a5600640 1 -- 172.21.15.39:0/3534557494 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 128 v128) ==== 95+0+5300 (secure 0 0 0) 0x7f07a0084480 con 0x7f07a8160860 2024-09-21T07:41:51.021 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:41:51.021 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":128,"btime":"2024-09-21T07:41:46:435551+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6425,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/73220122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":73220122},{"type":"v1","addr":"172.21.15.148:6837","nonce":73220122}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:41:46.435521+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7452":{"gid":7452,"name":"a","rank":1,"incarnation":122,"state":"up:active","state_seq":8,"addr":"172.21.15.39:6835/3695489158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3695489158},{"type":"v1","addr":"172.21.15.39:6835","nonce":3695489158}]},"join_fscid":-1,"laggy_since":"2024-09-21T07:41:46.435521+0000","export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311,7452]},"id":6}]} 2024-09-21T07:41:51.021 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 128 2024-09-21T07:41:51.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.022+0000 7f07ae400640 1 -- 172.21.15.39:0/3534557494 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0784078b90 msgr2=0x7f078407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:51.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.022+0000 7f07ae400640 1 --2- 172.21.15.39:0/3534557494 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0784078b90 0x7f078407b0a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f079800a4d0 tx=0x7f079800a460 comp rx=0 tx=0).stop 2024-09-21T07:41:51.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.022+0000 7f07ae400640 1 -- 172.21.15.39:0/3534557494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07a8160860 msgr2=0x7f07a80c72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:51.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.022+0000 7f07ae400640 1 --2- 172.21.15.39:0/3534557494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07a8160860 0x7f07a80c72d0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f07a0009d60 tx=0x7f07a0003ef0 comp rx=0 tx=0).stop 2024-09-21T07:41:51.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.022+0000 7f07ae400640 1 -- 172.21.15.39:0/3534557494 shutdown_connections 2024-09-21T07:41:51.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.022+0000 7f07ae400640 1 --2- 172.21.15.39:0/3534557494 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0784078b90 0x7f078407b0a0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:51.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.022+0000 7f07ae400640 1 --2- 172.21.15.39:0/3534557494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07a8160860 0x7f07a80c72d0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:51.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.022+0000 7f07ae400640 1 --2- 172.21.15.39:0/3534557494 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f07a815fc30 0x7f07a80c9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:51.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.022+0000 7f07ae400640 1 --2- 172.21.15.39:0/3534557494 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07a80cbef0 0x7f07a81f3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:51.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.022+0000 7f07ae400640 1 -- 172.21.15.39:0/3534557494 >> 172.21.15.39:0/3534557494 conn(0x7f07a80b2510 msgr2=0x7f07a806f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:51.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.022+0000 7f07ae400640 1 -- 172.21.15.39:0/3534557494 shutdown_connections 2024-09-21T07:41:51.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.022+0000 7f07ae400640 1 -- 172.21.15.39:0/3534557494 wait complete. 2024-09-21T07:41:51.049 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2024-09-21T07:41:51.294 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_snap":15,"last_created":12,"last_destroyed":14,"pending_noop":[],"snaps":[],"need_to_purge":{},"pending_update":[{"version":29,"snapinfo":{"snapid":15,"ino":1099511627777,"stamp":"2024-09-21T07:41:27.411060+0000","name":"s33","metadata":{}}}],"pending_destroy":[]} 2024-09-21T07:41:51.294 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 12, "last_destroyed": 14, "last_snap": 15, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [ { "snapInfinityo": { "ino": 1099511627777, "metadata": {}, "name": "s33", "snapid": 15, "stamp": "2024-09-21T07:41:27.411060+0000" }, "version": 29 } ], "snaps": [] } 2024-09-21T07:41:51.294 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 6:1 2024-09-21T07:41:51.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.521+0000 7fb092000640 1 -- 172.21.15.39:0/712303607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb08c15f790 msgr2=0x7fb08c0d0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:51.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.521+0000 7fb092000640 1 --2- 172.21.15.39:0/712303607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb08c15f790 0x7fb08c0d0620 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fb08c0af890 tx=0x7fb07402a560 comp rx=0 tx=0).stop 2024-09-21T07:41:51.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.521+0000 7fb092000640 1 -- 172.21.15.39:0/712303607 shutdown_connections 2024-09-21T07:41:51.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.521+0000 7fb092000640 1 --2- 172.21.15.39:0/712303607 >> v2:172.21.15.148:3300/0 conn(0x7fb08c15fba0 0x7fb08c0d0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:51.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.521+0000 7fb092000640 1 --2- 172.21.15.39:0/712303607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb08c15f790 0x7fb08c0d0620 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:51.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.521+0000 7fb092000640 1 --2- 172.21.15.39:0/712303607 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb08c0b47c0 0x7fb08c155800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:51.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.521+0000 7fb092000640 1 -- 172.21.15.39:0/712303607 >> 172.21.15.39:0/712303607 conn(0x7fb08c06cfc0 msgr2=0x7fb08c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:51.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.521+0000 7fb092000640 1 -- 172.21.15.39:0/712303607 shutdown_connections 2024-09-21T07:41:51.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.521+0000 7fb092000640 1 -- 172.21.15.39:0/712303607 wait complete. 2024-09-21T07:41:51.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.523+0000 7fb092000640 1 Processor -- start 2024-09-21T07:41:51.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.524+0000 7fb092000640 1 -- start start 2024-09-21T07:41:51.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.525+0000 7fb092000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb08c15f790 0x7fb08c14b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:51.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.525+0000 7fb092000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb08c15fba0 0x7fb08c1581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:51.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.525+0000 7fb092000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb08c14bc00 0x7fb08c1586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:51.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.525+0000 7fb092000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c162b00 con 0x7fb08c14bc00 2024-09-21T07:41:51.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.525+0000 7fb092000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c162980 con 0x7fb08c15f790 2024-09-21T07:41:51.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.525+0000 7fb092000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb08c162c80 con 0x7fb08c15fba0 2024-09-21T07:41:51.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.525+0000 7fb08be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb08c14bc00 0x7fb08c1586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:51.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.525+0000 7fb08be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb08c14bc00 0x7fb08c1586f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46008/0 (socket says 172.21.15.39:46008) 2024-09-21T07:41:51.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.525+0000 7fb08be00640 1 -- 172.21.15.39:0/1792244771 learned_addr learned my addr 172.21.15.39:0/1792244771 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:41:51.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.525+0000 7fb08b400640 1 --2- 172.21.15.39:0/1792244771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb08c15f790 0x7fb08c14b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:51.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.526+0000 7fb082a00640 1 --2- 172.21.15.39:0/1792244771 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb08c15fba0 0x7fb08c1581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:51.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.526+0000 7fb08be00640 1 -- 172.21.15.39:0/1792244771 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb08c15fba0 msgr2=0x7fb08c1581b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:51.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.526+0000 7fb08be00640 1 --2- 172.21.15.39:0/1792244771 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb08c15fba0 0x7fb08c1581b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:51.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.526+0000 7fb08be00640 1 -- 172.21.15.39:0/1792244771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb08c15f790 msgr2=0x7fb08c14b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:51.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.526+0000 7fb08be00640 1 --2- 172.21.15.39:0/1792244771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb08c15f790 0x7fb08c14b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:51.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.526+0000 7fb08be00640 1 -- 172.21.15.39:0/1792244771 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb08c15dc20 con 0x7fb08c14bc00 2024-09-21T07:41:51.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.526+0000 7fb08b400640 1 --2- 172.21.15.39:0/1792244771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb08c15f790 0x7fb08c14b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:41:51.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.526+0000 7fb08be00640 1 --2- 172.21.15.39:0/1792244771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb08c14bc00 0x7fb08c1586f0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fb08400dac0 tx=0x7fb08400df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:51.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.526+0000 7fb083e00640 1 -- 172.21.15.39:0/1792244771 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb084017070 con 0x7fb08c14bc00 2024-09-21T07:41:51.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.526+0000 7fb083e00640 1 -- 172.21.15.39:0/1792244771 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb084010460 con 0x7fb08c14bc00 2024-09-21T07:41:51.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.526+0000 7fb083e00640 1 -- 172.21.15.39:0/1792244771 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0840108b0 con 0x7fb08c14bc00 2024-09-21T07:41:51.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.527+0000 7fb092000640 1 -- 172.21.15.39:0/1792244771 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb08c15e4e0 con 0x7fb08c14bc00 2024-09-21T07:41:51.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.527+0000 7fb092000640 1 -- 172.21.15.39:0/1792244771 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb08c15e170 con 0x7fb08c14bc00 2024-09-21T07:41:51.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.528+0000 7fb092000640 1 -- 172.21.15.39:0/1792244771 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb08c0b52a0 con 0x7fb08c14bc00 2024-09-21T07:41:51.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.529+0000 7fb083e00640 1 -- 172.21.15.39:0/1792244771 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb084010a50 con 0x7fb08c14bc00 2024-09-21T07:41:51.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.530+0000 7fb083e00640 1 --2- 172.21.15.39:0/1792244771 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb068078b90 0x7fb06807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:51.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.530+0000 7fb083e00640 1 -- 172.21.15.39:0/1792244771 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7846+0+0 (secure 0 0 0) 0x7fb08409bb10 con 0x7fb08c14bc00 2024-09-21T07:41:51.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.530+0000 7fb08b400640 1 --2- 172.21.15.39:0/1792244771 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb068078b90 0x7fb06807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:51.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.531+0000 7fb08b400640 1 --2- 172.21.15.39:0/1792244771 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb068078b90 0x7fb06807b0a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb07c00bbc0 tx=0x7fb07c005ef0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:51.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.535+0000 7fb083e00640 1 -- 172.21.15.39:0/1792244771 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb084063120 con 0x7fb08c14bc00 2024-09-21T07:41:51.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.806+0000 7fb092000640 1 -- 172.21.15.39:0/1792244771 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "6:1"} v 0) -- 0x7fb08c0b0a50 con 0x7fb08c14bc00 2024-09-21T07:41:51.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.812+0000 7fb083e00640 1 -- 172.21.15.39:0/1792244771 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "6:1"}]=0 failed mds gid 7452 v129) ==== 97+0+0 (secure 0 0 0) 0x7fb084067b50 con 0x7fb08c14bc00 2024-09-21T07:41:51.814 INFO:teuthology.orchestra.run.smithi039.stderr:failed mds gid 7452 2024-09-21T07:41:51.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.815+0000 7fb092000640 1 -- 172.21.15.39:0/1792244771 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb068078b90 msgr2=0x7fb06807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:51.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.815+0000 7fb092000640 1 --2- 172.21.15.39:0/1792244771 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb068078b90 0x7fb06807b0a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb07c00bbc0 tx=0x7fb07c005ef0 comp rx=0 tx=0).stop 2024-09-21T07:41:51.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.815+0000 7fb092000640 1 -- 172.21.15.39:0/1792244771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb08c14bc00 msgr2=0x7fb08c1586f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:51.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.815+0000 7fb092000640 1 --2- 172.21.15.39:0/1792244771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb08c14bc00 0x7fb08c1586f0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fb08400dac0 tx=0x7fb08400df80 comp rx=0 tx=0).stop 2024-09-21T07:41:51.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.815+0000 7fb092000640 1 -- 172.21.15.39:0/1792244771 shutdown_connections 2024-09-21T07:41:51.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.815+0000 7fb092000640 1 --2- 172.21.15.39:0/1792244771 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb068078b90 0x7fb06807b0a0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:51.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.815+0000 7fb092000640 1 --2- 172.21.15.39:0/1792244771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb08c14bc00 0x7fb08c1586f0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:51.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.815+0000 7fb092000640 1 --2- 172.21.15.39:0/1792244771 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb08c15fba0 0x7fb08c1581b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:51.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.815+0000 7fb092000640 1 --2- 172.21.15.39:0/1792244771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb08c15f790 0x7fb08c14b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:51.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.815+0000 7fb092000640 1 -- 172.21.15.39:0/1792244771 >> 172.21.15.39:0/1792244771 conn(0x7fb08c06cfc0 msgr2=0x7fb08c1608b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:51.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.815+0000 7fb092000640 1 -- 172.21.15.39:0/1792244771 shutdown_connections 2024-09-21T07:41:51.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:51.815+0000 7fb092000640 1 -- 172.21.15.39:0/1792244771 wait complete. 2024-09-21T07:41:51.846 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T07:41:51.846 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T07:41:51.846 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T07:41:51.846 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:41:51.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:41:51.846 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/daemon/state.py", line 146, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi039 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a' 2024-09-21T07:41:51.846 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T07:41:51.847 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T07:41:51.851 INFO:tasks.ceph.mds.a:Started 2024-09-21T07:41:51.851 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:41:52.050 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-21T07:41:52.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.080+0000 7f7df8800640 1 -- 172.21.15.39:0/4167406041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7df00b3780 msgr2=0x7f7df015b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:52.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.080+0000 7f7df8800640 1 --2- 172.21.15.39:0/4167406041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7df00b3780 0x7f7df015b5d0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f7dec005fd0 tx=0x7f7dec02b9a0 comp rx=0 tx=0).stop 2024-09-21T07:41:52.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.080+0000 7f7df8800640 1 -- 172.21.15.39:0/4167406041 shutdown_connections 2024-09-21T07:41:52.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.080+0000 7f7df8800640 1 --2- 172.21.15.39:0/4167406041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7df00b3780 0x7f7df015b5d0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:52.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.080+0000 7f7df8800640 1 -- 172.21.15.39:0/4167406041 >> 172.21.15.39:0/4167406041 conn(0x7f7df006cfc0 msgr2=0x7f7df006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:52.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.080+0000 7f7df8800640 1 -- 172.21.15.39:0/4167406041 shutdown_connections 2024-09-21T07:41:52.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.080+0000 7f7df8800640 1 -- 172.21.15.39:0/4167406041 wait complete. 2024-09-21T07:41:52.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.082+0000 7f7df8800640 1 Processor -- start 2024-09-21T07:41:52.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.083+0000 7f7df8800640 1 -- start start 2024-09-21T07:41:52.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.084+0000 7f7df8800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7df00b3370 0x7f7df00aae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:52.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.084+0000 7f7df8800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7df00b3780 0x7f7df01efa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:52.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.084+0000 7f7df8800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7df015e590 0x7f7df01599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:52.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.084+0000 7f7df8800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7df015bb10 con 0x7f7df00b3370 2024-09-21T07:41:52.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.084+0000 7f7df8800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7df015bc90 con 0x7f7df015e590 2024-09-21T07:41:52.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.084+0000 7f7df8800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f7df015be10 con 0x7f7df00b3780 2024-09-21T07:41:52.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.084+0000 7f7df5e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7df00b3370 0x7f7df00aae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:52.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.084+0000 7f7df5e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7df00b3370 0x7f7df00aae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38822/0 (socket says 172.21.15.39:38822) 2024-09-21T07:41:52.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.084+0000 7f7df5e00640 1 -- 172.21.15.39:0/2705977547 learned_addr learned my addr 172.21.15.39:0/2705977547 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:41:52.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.084+0000 7f7df6800640 1 --2- 172.21.15.39:0/2705977547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7df015e590 0x7f7df01599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:52.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.084+0000 7f7df5400640 1 --2- 172.21.15.39:0/2705977547 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7df00b3780 0x7f7df01efa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:52.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.085+0000 7f7df5e00640 1 -- 172.21.15.39:0/2705977547 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7df00b3780 msgr2=0x7f7df01efa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:52.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.085+0000 7f7df5e00640 1 --2- 172.21.15.39:0/2705977547 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7df00b3780 0x7f7df01efa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:52.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.085+0000 7f7df5e00640 1 -- 172.21.15.39:0/2705977547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7df015e590 msgr2=0x7f7df01599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:52.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.085+0000 7f7df5e00640 1 --2- 172.21.15.39:0/2705977547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7df015e590 0x7f7df01599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:52.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.085+0000 7f7df5e00640 1 -- 172.21.15.39:0/2705977547 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df00ab690 con 0x7f7df00b3370 2024-09-21T07:41:52.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.085+0000 7f7df5400640 1 --2- 172.21.15.39:0/2705977547 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7df00b3780 0x7f7df01efa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:41:52.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.085+0000 7f7df5e00640 1 --2- 172.21.15.39:0/2705977547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7df00b3370 0x7f7df00aae60 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f7de400c340 tx=0x7f7de400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:52.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.085+0000 7f7dea000640 1 -- 172.21.15.39:0/2705977547 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de4017070 con 0x7f7df00b3370 2024-09-21T07:41:52.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.085+0000 7f7dea000640 1 -- 172.21.15.39:0/2705977547 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7de40053c0 con 0x7f7df00b3370 2024-09-21T07:41:52.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.085+0000 7f7dea000640 1 -- 172.21.15.39:0/2705977547 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de4003870 con 0x7f7df00b3370 2024-09-21T07:41:52.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.085+0000 7f7df8800640 1 -- 172.21.15.39:0/2705977547 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df00ab970 con 0x7f7df00b3370 2024-09-21T07:41:52.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.086+0000 7f7df8800640 1 -- 172.21.15.39:0/2705977547 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df00abf30 con 0x7f7df00b3370 2024-09-21T07:41:52.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.087+0000 7f7df8800640 1 -- 172.21.15.39:0/2705977547 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7df00affe0 con 0x7f7df00b3370 2024-09-21T07:41:52.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.088+0000 7f7dea000640 1 -- 172.21.15.39:0/2705977547 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7de401a070 con 0x7f7df00b3370 2024-09-21T07:41:52.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.089+0000 7f7dea000640 1 --2- 172.21.15.39:0/2705977547 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7dd4078b90 0x7f7dd407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:52.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.089+0000 7f7dea000640 1 -- 172.21.15.39:0/2705977547 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7904+0+0 (secure 0 0 0) 0x7f7de409a7f0 con 0x7f7df00b3370 2024-09-21T07:41:52.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.089+0000 7f7df5400640 1 --2- 172.21.15.39:0/2705977547 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7dd4078b90 0x7f7dd407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:52.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.090+0000 7f7df5400640 1 --2- 172.21.15.39:0/2705977547 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7dd4078b90 0x7f7dd407b0a0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7de0007fd0 tx=0x7f7de0009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:52.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.093+0000 7f7dea000640 1 -- 172.21.15.39:0/2705977547 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7df00affe0 con 0x7f7df00b3370 2024-09-21T07:41:52.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.332+0000 7f7df8800640 1 -- 172.21.15.39:0/2705977547 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7df015b5d0 con 0x7f7df00b3370 2024-09-21T07:41:52.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.333+0000 7f7dea000640 1 -- 172.21.15.39:0/2705977547 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 130 v130) ==== 95+0+4421 (secure 0 0 0) 0x7f7de4061df0 con 0x7f7df00b3370 2024-09-21T07:41:52.334 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:41:52.334 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":130,"btime":"2024-09-21T07:41:51:820948+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:41:51.820926+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":6425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6425":{"gid":6425,"name":"d","rank":1,"incarnation":130,"state":"up:replay","state_seq":1,"addr":"172.21.15.148:6837/73220122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":73220122},{"type":"v1","addr":"172.21.15.148:6837","nonce":73220122}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311]},"id":6}]} 2024-09-21T07:41:52.334 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 130 2024-09-21T07:41:52.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.335+0000 7f7df8800640 1 -- 172.21.15.39:0/2705977547 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7dd4078b90 msgr2=0x7f7dd407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:52.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.335+0000 7f7df8800640 1 --2- 172.21.15.39:0/2705977547 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7dd4078b90 0x7f7dd407b0a0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7de0007fd0 tx=0x7f7de0009040 comp rx=0 tx=0).stop 2024-09-21T07:41:52.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.335+0000 7f7df8800640 1 -- 172.21.15.39:0/2705977547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7df00b3370 msgr2=0x7f7df00aae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:52.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.335+0000 7f7df8800640 1 --2- 172.21.15.39:0/2705977547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7df00b3370 0x7f7df00aae60 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f7de400c340 tx=0x7f7de400c800 comp rx=0 tx=0).stop 2024-09-21T07:41:52.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.335+0000 7f7df8800640 1 -- 172.21.15.39:0/2705977547 shutdown_connections 2024-09-21T07:41:52.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.335+0000 7f7df8800640 1 --2- 172.21.15.39:0/2705977547 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7dd4078b90 0x7f7dd407b0a0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:52.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.335+0000 7f7df8800640 1 --2- 172.21.15.39:0/2705977547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7df015e590 0x7f7df01599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:52.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.335+0000 7f7df8800640 1 --2- 172.21.15.39:0/2705977547 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7df00b3780 0x7f7df01efa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:52.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.335+0000 7f7df8800640 1 --2- 172.21.15.39:0/2705977547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7df00b3370 0x7f7df00aae60 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:52.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.335+0000 7f7df8800640 1 -- 172.21.15.39:0/2705977547 >> 172.21.15.39:0/2705977547 conn(0x7f7df006cfc0 msgr2=0x7f7df00b0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:52.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.335+0000 7f7df8800640 1 -- 172.21.15.39:0/2705977547 shutdown_connections 2024-09-21T07:41:52.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:52.335+0000 7f7df8800640 1 -- 172.21.15.39:0/2705977547 wait complete. 2024-09-21T07:41:52.361 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-09-21T07:41:55.976 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-09-21T07:41:57.362 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:41:57.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.600+0000 7f9450800640 1 -- 172.21.15.39:0/3464035245 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f944815f360 msgr2=0x7f944815bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:57.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.600+0000 7f9450800640 1 --2- 172.21.15.39:0/3464035245 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f944815f360 0x7f944815bd70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f943c009860 tx=0x7f943c02a5a0 comp rx=0 tx=0).stop 2024-09-21T07:41:57.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.600+0000 7f9450800640 1 -- 172.21.15.39:0/3464035245 shutdown_connections 2024-09-21T07:41:57.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.600+0000 7f9450800640 1 --2- 172.21.15.39:0/3464035245 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f944815f360 0x7f944815bd70 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:57.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.600+0000 7f9450800640 1 -- 172.21.15.39:0/3464035245 >> 172.21.15.39:0/3464035245 conn(0x7f944806cfc0 msgr2=0x7f944806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:57.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.600+0000 7f9450800640 1 -- 172.21.15.39:0/3464035245 shutdown_connections 2024-09-21T07:41:57.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.600+0000 7f9450800640 1 -- 172.21.15.39:0/3464035245 wait complete. 2024-09-21T07:41:57.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.603+0000 7f9450800640 1 Processor -- start 2024-09-21T07:41:57.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.603+0000 7f9450800640 1 -- start start 2024-09-21T07:41:57.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.604+0000 7f9450800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9448155a70 0x7f944814fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:57.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.604+0000 7f9450800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9448155e80 0x7f9448156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:57.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.604+0000 7f9450800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f944815f360 0x7f94480b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:57.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.605+0000 7f9450800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f94480d07a0 con 0x7f9448155e80 2024-09-21T07:41:57.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.605+0000 7f9450800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f94480d0620 con 0x7f944815f360 2024-09-21T07:41:57.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.605+0000 7f9450800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f94480d0920 con 0x7f9448155a70 2024-09-21T07:41:57.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.605+0000 7f944d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9448155e80 0x7f9448156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:57.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.605+0000 7f944d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9448155e80 0x7f9448156380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38832/0 (socket says 172.21.15.39:38832) 2024-09-21T07:41:57.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.605+0000 7f944d400640 1 -- 172.21.15.39:0/796189390 learned_addr learned my addr 172.21.15.39:0/796189390 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:41:57.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.605+0000 7f944de00640 1 --2- 172.21.15.39:0/796189390 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9448155a70 0x7f944814fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:57.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.605+0000 7f944e800640 1 --2- 172.21.15.39:0/796189390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f944815f360 0x7f94480b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:57.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.605+0000 7f944d400640 1 -- 172.21.15.39:0/796189390 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9448155a70 msgr2=0x7f944814fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:57.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.605+0000 7f944d400640 1 --2- 172.21.15.39:0/796189390 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9448155a70 0x7f944814fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:57.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.605+0000 7f944d400640 1 -- 172.21.15.39:0/796189390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f944815f360 msgr2=0x7f94480b27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:57.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.605+0000 7f944d400640 1 --2- 172.21.15.39:0/796189390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f944815f360 0x7f94480b27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:57.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.605+0000 7f944d400640 1 -- 172.21.15.39:0/796189390 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94481f9740 con 0x7f9448155e80 2024-09-21T07:41:57.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.606+0000 7f944d400640 1 --2- 172.21.15.39:0/796189390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9448155e80 0x7f9448156380 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f943800c460 tx=0x7f943800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:57.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.606+0000 7f9442000640 1 -- 172.21.15.39:0/796189390 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94380052d0 con 0x7f9448155e80 2024-09-21T07:41:57.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.606+0000 7f9442000640 1 -- 172.21.15.39:0/796189390 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9438005490 con 0x7f9448155e80 2024-09-21T07:41:57.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.606+0000 7f9450800640 1 -- 172.21.15.39:0/796189390 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94481f9a20 con 0x7f9448155e80 2024-09-21T07:41:57.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.606+0000 7f9442000640 1 -- 172.21.15.39:0/796189390 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9438012870 con 0x7f9448155e80 2024-09-21T07:41:57.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.606+0000 7f9450800640 1 -- 172.21.15.39:0/796189390 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94481f9fe0 con 0x7f9448155e80 2024-09-21T07:41:57.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.607+0000 7f9450800640 1 -- 172.21.15.39:0/796189390 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94480d0e40 con 0x7f9448155e80 2024-09-21T07:41:57.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.608+0000 7f9442000640 1 -- 172.21.15.39:0/796189390 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9438012a10 con 0x7f9448155e80 2024-09-21T07:41:57.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.609+0000 7f9442000640 1 --2- 172.21.15.39:0/796189390 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f942c078b90 0x7f942c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:57.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.610+0000 7f9442000640 1 -- 172.21.15.39:0/796189390 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7904+0+0 (secure 0 0 0) 0x7f943809b0c0 con 0x7f9448155e80 2024-09-21T07:41:57.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.610+0000 7f944de00640 1 --2- 172.21.15.39:0/796189390 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f942c078b90 0x7f942c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:57.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.611+0000 7f944de00640 1 --2- 172.21.15.39:0/796189390 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f942c078b90 0x7f942c07b0a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f943c034000 tx=0x7f943c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:57.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.614+0000 7f9442000640 1 -- 172.21.15.39:0/796189390 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9438062520 con 0x7f9448155e80 2024-09-21T07:41:57.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.854+0000 7f9450800640 1 -- 172.21.15.39:0/796189390 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9448153950 con 0x7f9448155e80 2024-09-21T07:41:57.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.855+0000 7f9442000640 1 -- 172.21.15.39:0/796189390 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 132 v132) ==== 95+0+5251 (secure 0 0 0) 0x7f9438066f50 con 0x7f9448155e80 2024-09-21T07:41:57.857 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:41:57.857 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":132,"btime":"2024-09-21T07:41:53:824647+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7644,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1326711417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1326711417},{"type":"v1","addr":"172.21.15.39:6835","nonce":1326711417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":131}],"filesystems":[{"mdsmap":{"epoch":132,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:41:52.829432+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":6425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6425":{"gid":6425,"name":"d","rank":1,"incarnation":130,"state":"up:reconnect","state_seq":14,"addr":"172.21.15.148:6837/73220122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":73220122},{"type":"v1","addr":"172.21.15.148:6837","nonce":73220122}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311]},"id":6}]} 2024-09-21T07:41:57.857 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 132 2024-09-21T07:41:57.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.857+0000 7f9450800640 1 -- 172.21.15.39:0/796189390 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f942c078b90 msgr2=0x7f942c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:57.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.857+0000 7f9450800640 1 --2- 172.21.15.39:0/796189390 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f942c078b90 0x7f942c07b0a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f943c034000 tx=0x7f943c004730 comp rx=0 tx=0).stop 2024-09-21T07:41:57.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.857+0000 7f9450800640 1 -- 172.21.15.39:0/796189390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9448155e80 msgr2=0x7f9448156380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:57.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.857+0000 7f9450800640 1 --2- 172.21.15.39:0/796189390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9448155e80 0x7f9448156380 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f943800c460 tx=0x7f943800c920 comp rx=0 tx=0).stop 2024-09-21T07:41:57.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.858+0000 7f9450800640 1 -- 172.21.15.39:0/796189390 shutdown_connections 2024-09-21T07:41:57.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.858+0000 7f9450800640 1 --2- 172.21.15.39:0/796189390 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f942c078b90 0x7f942c07b0a0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:57.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.858+0000 7f9450800640 1 --2- 172.21.15.39:0/796189390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f944815f360 0x7f94480b27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:57.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.858+0000 7f9450800640 1 --2- 172.21.15.39:0/796189390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9448155e80 0x7f9448156380 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:57.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.858+0000 7f9450800640 1 --2- 172.21.15.39:0/796189390 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9448155a70 0x7f944814fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:57.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.858+0000 7f9450800640 1 -- 172.21.15.39:0/796189390 >> 172.21.15.39:0/796189390 conn(0x7f944806cfc0 msgr2=0x7f944815b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:57.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.858+0000 7f9450800640 1 -- 172.21.15.39:0/796189390 shutdown_connections 2024-09-21T07:41:57.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:57.858+0000 7f9450800640 1 -- 172.21.15.39:0/796189390 wait complete. 2024-09-21T07:41:57.882 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-21T07:41:57.883 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:41:58.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.108+0000 7fee92e00640 1 -- 172.21.15.39:0/4225142082 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee8c0d2e60 msgr2=0x7fee8c0b6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:58.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.108+0000 7fee92e00640 1 --2- 172.21.15.39:0/4225142082 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee8c0d2e60 0x7fee8c0b6710 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fee8c08df30 tx=0x7fee8402aac0 comp rx=0 tx=0).stop 2024-09-21T07:41:58.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.108+0000 7fee92e00640 1 -- 172.21.15.39:0/4225142082 shutdown_connections 2024-09-21T07:41:58.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.108+0000 7fee92e00640 1 --2- 172.21.15.39:0/4225142082 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee8c0d2e60 0x7fee8c0b6710 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:58.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.108+0000 7fee92e00640 1 --2- 172.21.15.39:0/4225142082 >> v2:172.21.15.148:3300/0 conn(0x7fee8c0b4db0 0x7fee8c0d4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:58.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.108+0000 7fee92e00640 1 -- 172.21.15.39:0/4225142082 >> 172.21.15.39:0/4225142082 conn(0x7fee8c08bfb0 msgr2=0x7fee8c0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:58.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.108+0000 7fee92e00640 1 -- 172.21.15.39:0/4225142082 shutdown_connections 2024-09-21T07:41:58.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.108+0000 7fee92e00640 1 -- 172.21.15.39:0/4225142082 wait complete. 2024-09-21T07:41:58.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.110+0000 7fee92e00640 1 Processor -- start 2024-09-21T07:41:58.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.110+0000 7fee92e00640 1 -- start start 2024-09-21T07:41:58.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.112+0000 7fee92e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee8c0b4db0 0x7fee8c0cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:58.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.112+0000 7fee92e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee8c0d2a50 0x7fee8c1579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:58.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.112+0000 7fee92e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee8c0d2e60 0x7fee8c0b2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:58.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.112+0000 7fee92e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c0cee90 con 0x7fee8c0d2a50 2024-09-21T07:41:58.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.112+0000 7fee92e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c0b9a00 con 0x7fee8c0b4db0 2024-09-21T07:41:58.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.112+0000 7fee92e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fee8c0cf010 con 0x7fee8c0d2e60 2024-09-21T07:41:58.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.112+0000 7fee91000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee8c0d2a50 0x7fee8c1579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:58.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.112+0000 7fee91000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee8c0d2a50 0x7fee8c1579c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38854/0 (socket says 172.21.15.39:38854) 2024-09-21T07:41:58.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.112+0000 7fee91000640 1 -- 172.21.15.39:0/3984277241 learned_addr learned my addr 172.21.15.39:0/3984277241 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:41:58.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.112+0000 7fee92400640 1 --2- 172.21.15.39:0/3984277241 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee8c0d2e60 0x7fee8c0b2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:58.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.112+0000 7fee91a00640 1 --2- 172.21.15.39:0/3984277241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee8c0b4db0 0x7fee8c0cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:58.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.113+0000 7fee91000640 1 -- 172.21.15.39:0/3984277241 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee8c0d2e60 msgr2=0x7fee8c0b2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:58.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.113+0000 7fee91000640 1 --2- 172.21.15.39:0/3984277241 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee8c0d2e60 0x7fee8c0b2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:58.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.113+0000 7fee91000640 1 -- 172.21.15.39:0/3984277241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee8c0b4db0 msgr2=0x7fee8c0cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:58.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.113+0000 7fee91000640 1 --2- 172.21.15.39:0/3984277241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee8c0b4db0 0x7fee8c0cc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:58.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.113+0000 7fee91000640 1 -- 172.21.15.39:0/3984277241 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee8c1f5240 con 0x7fee8c0d2a50 2024-09-21T07:41:58.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.113+0000 7fee91000640 1 --2- 172.21.15.39:0/3984277241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee8c0d2a50 0x7fee8c1579c0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fee7800c3c0 tx=0x7fee7800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:58.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.113+0000 7fee89600640 1 -- 172.21.15.39:0/3984277241 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee78017070 con 0x7fee8c0d2a50 2024-09-21T07:41:58.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.113+0000 7fee89600640 1 -- 172.21.15.39:0/3984277241 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee780035f0 con 0x7fee8c0d2a50 2024-09-21T07:41:58.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.113+0000 7fee92e00640 1 -- 172.21.15.39:0/3984277241 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee8c1f56a0 con 0x7fee8c0d2a50 2024-09-21T07:41:58.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.113+0000 7fee89600640 1 -- 172.21.15.39:0/3984277241 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee78003a80 con 0x7fee8c0d2a50 2024-09-21T07:41:58.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.113+0000 7fee92e00640 1 -- 172.21.15.39:0/3984277241 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee8c1f5c60 con 0x7fee8c0d2a50 2024-09-21T07:41:58.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.115+0000 7fee92e00640 1 -- 172.21.15.39:0/3984277241 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee8c1f5410 con 0x7fee8c0d2a50 2024-09-21T07:41:58.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.115+0000 7fee89600640 1 -- 172.21.15.39:0/3984277241 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fee7801a070 con 0x7fee8c0d2a50 2024-09-21T07:41:58.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.116+0000 7fee89600640 1 --2- 172.21.15.39:0/3984277241 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fee6c078bc0 0x7fee6c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:41:58.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.117+0000 7fee89600640 1 -- 172.21.15.39:0/3984277241 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7904+0+0 (secure 0 0 0) 0x7fee7809a6a0 con 0x7fee8c0d2a50 2024-09-21T07:41:58.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.117+0000 7fee91a00640 1 --2- 172.21.15.39:0/3984277241 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fee6c078bc0 0x7fee6c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:41:58.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.118+0000 7fee91a00640 1 --2- 172.21.15.39:0/3984277241 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fee6c078bc0 0x7fee6c07b0d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fee8000bb30 tx=0x7fee8000b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:41:58.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.121+0000 7fee89600640 1 -- 172.21.15.39:0/3984277241 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fee78061ca0 con 0x7fee8c0d2a50 2024-09-21T07:41:58.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.355+0000 7fee92e00640 1 -- 172.21.15.39:0/3984277241 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee8c0d4c90 con 0x7fee8c0d2a50 2024-09-21T07:41:58.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.356+0000 7fee89600640 1 -- 172.21.15.39:0/3984277241 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 132 v132) ==== 95+0+5251 (secure 0 0 0) 0x7fee780666d0 con 0x7fee8c0d2a50 2024-09-21T07:41:58.358 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:41:58.358 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":132,"btime":"2024-09-21T07:41:53:824647+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7644,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1326711417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1326711417},{"type":"v1","addr":"172.21.15.39:6835","nonce":1326711417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":131}],"filesystems":[{"mdsmap":{"epoch":132,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:41:52.829432+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":6425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6425":{"gid":6425,"name":"d","rank":1,"incarnation":130,"state":"up:reconnect","state_seq":14,"addr":"172.21.15.148:6837/73220122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":73220122},{"type":"v1","addr":"172.21.15.148:6837","nonce":73220122}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311]},"id":6}]} 2024-09-21T07:41:58.358 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 132 2024-09-21T07:41:58.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.358+0000 7fee92e00640 1 -- 172.21.15.39:0/3984277241 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fee6c078bc0 msgr2=0x7fee6c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:58.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.358+0000 7fee92e00640 1 --2- 172.21.15.39:0/3984277241 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fee6c078bc0 0x7fee6c07b0d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fee8000bb30 tx=0x7fee8000b2e0 comp rx=0 tx=0).stop 2024-09-21T07:41:58.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.358+0000 7fee92e00640 1 -- 172.21.15.39:0/3984277241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee8c0d2a50 msgr2=0x7fee8c1579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:41:58.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.358+0000 7fee92e00640 1 --2- 172.21.15.39:0/3984277241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee8c0d2a50 0x7fee8c1579c0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fee7800c3c0 tx=0x7fee7800c880 comp rx=0 tx=0).stop 2024-09-21T07:41:58.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.358+0000 7fee92e00640 1 -- 172.21.15.39:0/3984277241 shutdown_connections 2024-09-21T07:41:58.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.358+0000 7fee92e00640 1 --2- 172.21.15.39:0/3984277241 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fee6c078bc0 0x7fee6c07b0d0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:58.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.358+0000 7fee92e00640 1 --2- 172.21.15.39:0/3984277241 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee8c0d2e60 0x7fee8c0b2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:58.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.358+0000 7fee92e00640 1 --2- 172.21.15.39:0/3984277241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee8c0d2a50 0x7fee8c1579c0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:58.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.358+0000 7fee92e00640 1 --2- 172.21.15.39:0/3984277241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee8c0b4db0 0x7fee8c0cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:41:58.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.358+0000 7fee92e00640 1 -- 172.21.15.39:0/3984277241 >> 172.21.15.39:0/3984277241 conn(0x7fee8c08bfb0 msgr2=0x7fee8c0d4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:41:58.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.358+0000 7fee92e00640 1 -- 172.21.15.39:0/3984277241 shutdown_connections 2024-09-21T07:41:58.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:41:58.358+0000 7fee92e00640 1 -- 172.21.15.39:0/3984277241 wait complete. 2024-09-21T07:41:58.787 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 132, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:36:20.189287+0000', 'modified': '2024-09-21T07:41:52.829432+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 112, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 7311, 'mds_1': 6425}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6425': {'gid': 6425, 'name': 'd', 'rank': 1, 'incarnation': 130, 'state': 'up:reconnect', 'state_seq': 14, 'addr': '172.21.15.148:6837/73220122', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 73220122}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 73220122}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_7311': {'gid': 7311, 'name': 'c', 'rank': 0, 'incarnation': 119, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.39:6833/1790581246', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 1790581246}, {'type': 'v1', 'addr': '172.21.15.39:6833', 'nonce': 1790581246}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7311, 'qdb_cluster': [7311]} 2024-09-21T07:41:58.788 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6425:up:reconnect 2024-09-21T07:41:59.788 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:42:00.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.027+0000 7ff3b3200640 1 -- 172.21.15.39:0/2002150879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3ac15fc30 msgr2=0x7ff3ac160320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:00.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.027+0000 7ff3b3200640 1 --2- 172.21.15.39:0/2002150879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3ac15fc30 0x7ff3ac160320 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0009830 tx=0x7ff3a002a480 comp rx=0 tx=0).stop 2024-09-21T07:42:00.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.027+0000 7ff3b3200640 1 -- 172.21.15.39:0/2002150879 shutdown_connections 2024-09-21T07:42:00.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.027+0000 7ff3b3200640 1 --2- 172.21.15.39:0/2002150879 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff3ac0cc0c0 0x7ff3ac06f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:00.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.027+0000 7ff3b3200640 1 --2- 172.21.15.39:0/2002150879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3ac15fc30 0x7ff3ac160320 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:00.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.027+0000 7ff3b3200640 1 -- 172.21.15.39:0/2002150879 >> 172.21.15.39:0/2002150879 conn(0x7ff3ac0b2510 msgr2=0x7ff3ac0d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:00.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.027+0000 7ff3b3200640 1 -- 172.21.15.39:0/2002150879 shutdown_connections 2024-09-21T07:42:00.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.027+0000 7ff3b3200640 1 -- 172.21.15.39:0/2002150879 wait complete. 2024-09-21T07:42:00.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.029+0000 7ff3b3200640 1 Processor -- start 2024-09-21T07:42:00.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.030+0000 7ff3b3200640 1 -- start start 2024-09-21T07:42:00.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.031+0000 7ff3b3200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3ac0cc0c0 0x7ff3ac0c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:00.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.031+0000 7ff3b3200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3ac15fc30 0x7ff3ac153a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:00.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.031+0000 7ff3b3200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff3ac160860 0x7ff3ac15d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:00.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.031+0000 7ff3b3200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac06f750 con 0x7ff3ac0cc0c0 2024-09-21T07:42:00.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.031+0000 7ff3b3200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac0cd770 con 0x7ff3ac15fc30 2024-09-21T07:42:00.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.031+0000 7ff3b3200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7ff3ac06f8d0 con 0x7ff3ac160860 2024-09-21T07:42:00.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.031+0000 7ff3b1e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3ac0cc0c0 0x7ff3ac0c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:00.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.031+0000 7ff3b1e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3ac0cc0c0 0x7ff3ac0c9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38882/0 (socket says 172.21.15.39:38882) 2024-09-21T07:42:00.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.031+0000 7ff3b1e00640 1 -- 172.21.15.39:0/1070849896 learned_addr learned my addr 172.21.15.39:0/1070849896 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:00.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.031+0000 7ff3b2800640 1 --2- 172.21.15.39:0/1070849896 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff3ac160860 0x7ff3ac15d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:00.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.031+0000 7ff3b1400640 1 --2- 172.21.15.39:0/1070849896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3ac15fc30 0x7ff3ac153a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:00.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.032+0000 7ff3b1e00640 1 -- 172.21.15.39:0/1070849896 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff3ac160860 msgr2=0x7ff3ac15d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:00.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.032+0000 7ff3b1e00640 1 --2- 172.21.15.39:0/1070849896 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff3ac160860 0x7ff3ac15d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:00.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.032+0000 7ff3b1e00640 1 -- 172.21.15.39:0/1070849896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3ac15fc30 msgr2=0x7ff3ac153a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:00.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.032+0000 7ff3b1e00640 1 --2- 172.21.15.39:0/1070849896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3ac15fc30 0x7ff3ac153a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:00.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.032+0000 7ff3b1e00640 1 -- 172.21.15.39:0/1070849896 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3ac153f90 con 0x7ff3ac0cc0c0 2024-09-21T07:42:00.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.032+0000 7ff3b1e00640 1 --2- 172.21.15.39:0/1070849896 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3ac0cc0c0 0x7ff3ac0c9c90 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7ff3a002aa70 tx=0x7ff3a0033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:00.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.032+0000 7ff3a6000640 1 -- 172.21.15.39:0/1070849896 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0005c90 con 0x7ff3ac0cc0c0 2024-09-21T07:42:00.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.032+0000 7ff3a6000640 1 -- 172.21.15.39:0/1070849896 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3a0031e50 con 0x7ff3ac0cc0c0 2024-09-21T07:42:00.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.032+0000 7ff3b3200640 1 -- 172.21.15.39:0/1070849896 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3ac1512e0 con 0x7ff3ac0cc0c0 2024-09-21T07:42:00.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.032+0000 7ff3a6000640 1 -- 172.21.15.39:0/1070849896 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0005260 con 0x7ff3ac0cc0c0 2024-09-21T07:42:00.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.032+0000 7ff3b3200640 1 -- 172.21.15.39:0/1070849896 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3ac1518a0 con 0x7ff3ac0cc0c0 2024-09-21T07:42:00.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.033+0000 7ff3b3200640 1 -- 172.21.15.39:0/1070849896 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3ac06fdf0 con 0x7ff3ac0cc0c0 2024-09-21T07:42:00.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.035+0000 7ff3a6000640 1 -- 172.21.15.39:0/1070849896 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff3a0005400 con 0x7ff3ac0cc0c0 2024-09-21T07:42:00.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.036+0000 7ff3a6000640 1 --2- 172.21.15.39:0/1070849896 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff390078bc0 0x7ff39007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:00.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.036+0000 7ff3a6000640 1 -- 172.21.15.39:0/1070849896 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7904+0+0 (secure 0 0 0) 0x7ff3a00bc760 con 0x7ff3ac0cc0c0 2024-09-21T07:42:00.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.037+0000 7ff3b1400640 1 --2- 172.21.15.39:0/1070849896 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff390078bc0 0x7ff39007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:00.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.037+0000 7ff3b1400640 1 --2- 172.21.15.39:0/1070849896 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff390078bc0 0x7ff39007b0d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff39c0013f0 tx=0x7ff39c009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:00.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.041+0000 7ff3a6000640 1 -- 172.21.15.39:0/1070849896 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff3a003f030 con 0x7ff3ac0cc0c0 2024-09-21T07:42:00.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.278+0000 7ff3b3200640 1 -- 172.21.15.39:0/1070849896 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3ac0b4400 con 0x7ff3ac0cc0c0 2024-09-21T07:42:00.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.278+0000 7ff3a6000640 1 -- 172.21.15.39:0/1070849896 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 132 v132) ==== 95+0+5251 (secure 0 0 0) 0x7ff3a0084290 con 0x7ff3ac0cc0c0 2024-09-21T07:42:00.281 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:42:00.281 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":132,"btime":"2024-09-21T07:41:53:824647+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7644,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1326711417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1326711417},{"type":"v1","addr":"172.21.15.39:6835","nonce":1326711417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":131}],"filesystems":[{"mdsmap":{"epoch":132,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:41:52.829432+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":6425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6425":{"gid":6425,"name":"d","rank":1,"incarnation":130,"state":"up:reconnect","state_seq":14,"addr":"172.21.15.148:6837/73220122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":73220122},{"type":"v1","addr":"172.21.15.148:6837","nonce":73220122}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311]},"id":6}]} 2024-09-21T07:42:00.282 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 132 2024-09-21T07:42:00.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.281+0000 7ff3b3200640 1 -- 172.21.15.39:0/1070849896 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff390078bc0 msgr2=0x7ff39007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:00.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.281+0000 7ff3b3200640 1 --2- 172.21.15.39:0/1070849896 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff390078bc0 0x7ff39007b0d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff39c0013f0 tx=0x7ff39c009040 comp rx=0 tx=0).stop 2024-09-21T07:42:00.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.281+0000 7ff3b3200640 1 -- 172.21.15.39:0/1070849896 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3ac0cc0c0 msgr2=0x7ff3ac0c9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:00.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.281+0000 7ff3b3200640 1 --2- 172.21.15.39:0/1070849896 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3ac0cc0c0 0x7ff3ac0c9c90 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7ff3a002aa70 tx=0x7ff3a0033c20 comp rx=0 tx=0).stop 2024-09-21T07:42:00.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.281+0000 7ff3b3200640 1 -- 172.21.15.39:0/1070849896 shutdown_connections 2024-09-21T07:42:00.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.281+0000 7ff3b3200640 1 --2- 172.21.15.39:0/1070849896 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff390078bc0 0x7ff39007b0d0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:00.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.281+0000 7ff3b3200640 1 --2- 172.21.15.39:0/1070849896 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff3ac160860 0x7ff3ac15d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:00.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.281+0000 7ff3b3200640 1 --2- 172.21.15.39:0/1070849896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff3ac15fc30 0x7ff3ac153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:00.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.281+0000 7ff3b3200640 1 --2- 172.21.15.39:0/1070849896 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3ac0cc0c0 0x7ff3ac0c9c90 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:00.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.281+0000 7ff3b3200640 1 -- 172.21.15.39:0/1070849896 >> 172.21.15.39:0/1070849896 conn(0x7ff3ac0b2510 msgr2=0x7ff3ac0b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:00.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.281+0000 7ff3b3200640 1 -- 172.21.15.39:0/1070849896 shutdown_connections 2024-09-21T07:42:00.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:00.281+0000 7ff3b3200640 1 -- 172.21.15.39:0/1070849896 wait complete. 2024-09-21T07:42:00.709 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 132, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:36:20.189287+0000', 'modified': '2024-09-21T07:41:52.829432+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 112, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 7311, 'mds_1': 6425}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6425': {'gid': 6425, 'name': 'd', 'rank': 1, 'incarnation': 130, 'state': 'up:reconnect', 'state_seq': 14, 'addr': '172.21.15.148:6837/73220122', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 73220122}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 73220122}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_7311': {'gid': 7311, 'name': 'c', 'rank': 0, 'incarnation': 119, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.39:6833/1790581246', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 1790581246}, {'type': 'v1', 'addr': '172.21.15.39:6833', 'nonce': 1790581246}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7311, 'qdb_cluster': [7311]} 2024-09-21T07:42:00.709 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6425:up:reconnect 2024-09-21T07:42:01.550 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:42:01.555 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:42:01.710 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:42:01.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:01.945+0000 7f771b200640 1 -- 172.21.15.39:0/878719766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76f8005250 msgr2=0x7f76f800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:01.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:01.945+0000 7f771b200640 1 --2- 172.21.15.39:0/878719766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76f8005250 0x7f76f800b9c0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f770c009ed0 tx=0x7f76f8005830 comp rx=0 tx=0).stop 2024-09-21T07:42:01.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:01.945+0000 7f771b200640 1 -- 172.21.15.39:0/878719766 shutdown_connections 2024-09-21T07:42:01.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:01.945+0000 7f771b200640 1 --2- 172.21.15.39:0/878719766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76f8005250 0x7f76f800b9c0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:01.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:01.945+0000 7f771b200640 1 -- 172.21.15.39:0/878719766 >> 172.21.15.39:0/878719766 conn(0x7f771406cfc0 msgr2=0x7f771406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:01.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:01.945+0000 7f771b200640 1 -- 172.21.15.39:0/878719766 shutdown_connections 2024-09-21T07:42:01.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:01.945+0000 7f771b200640 1 -- 172.21.15.39:0/878719766 wait complete. 2024-09-21T07:42:01.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:01.947+0000 7f771b200640 1 Processor -- start 2024-09-21T07:42:01.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:01.947+0000 7f771b200640 1 -- start start 2024-09-21T07:42:01.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:01.949+0000 7f771b200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f76f8005250 0x7f7714152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:01.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:01.949+0000 7f771b200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7714156c50 0x7f7714155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:01.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:01.949+0000 7f771b200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7714156110 0x7f7714156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:01.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:01.949+0000 7f771b200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f77140d0860 con 0x7f7714156c50 2024-09-21T07:42:01.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:01.949+0000 7f771b200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f77140d06e0 con 0x7f7714156110 2024-09-21T07:42:01.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:01.949+0000 7f771b200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f77140d09e0 con 0x7f76f8005250 2024-09-21T07:42:01.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:01.949+0000 7f770b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7714156c50 0x7f7714155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:01.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:01.949+0000 7f770b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7714156c50 0x7f7714155bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38908/0 (socket says 172.21.15.39:38908) 2024-09-21T07:42:01.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:01.949+0000 7f770b400640 1 -- 172.21.15.39:0/2998667920 learned_addr learned my addr 172.21.15.39:0/2998667920 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:01.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:01.949+0000 7f7713e00640 1 --2- 172.21.15.39:0/2998667920 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f76f8005250 0x7f7714152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:01.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:01.949+0000 7f7719200640 1 --2- 172.21.15.39:0/2998667920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7714156110 0x7f7714156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:01.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:01.950+0000 7f770b400640 1 -- 172.21.15.39:0/2998667920 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f76f8005250 msgr2=0x7f7714152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:01.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:01.950+0000 7f770b400640 1 --2- 172.21.15.39:0/2998667920 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f76f8005250 0x7f7714152a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:01.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:01.950+0000 7f770b400640 1 -- 172.21.15.39:0/2998667920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7714156110 msgr2=0x7f7714156520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:01.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:01.950+0000 7f770b400640 1 --2- 172.21.15.39:0/2998667920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7714156110 0x7f7714156520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:01.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:01.950+0000 7f770b400640 1 -- 172.21.15.39:0/2998667920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77141f51d0 con 0x7f7714156c50 2024-09-21T07:42:01.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:01.950+0000 7f770b400640 1 --2- 172.21.15.39:0/2998667920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7714156c50 0x7f7714155bd0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f76fc000fb0 tx=0x7f76fc00e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:01.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:01.950+0000 7f7710c00640 1 -- 172.21.15.39:0/2998667920 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76fc010890 con 0x7f7714156c50 2024-09-21T07:42:01.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:01.950+0000 7f7710c00640 1 -- 172.21.15.39:0/2998667920 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76fc016070 con 0x7f7714156c50 2024-09-21T07:42:01.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:01.950+0000 7f771b200640 1 -- 172.21.15.39:0/2998667920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77141577b0 con 0x7f7714156c50 2024-09-21T07:42:01.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:01.950+0000 7f7710c00640 1 -- 172.21.15.39:0/2998667920 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76fc010dc0 con 0x7f7714156c50 2024-09-21T07:42:01.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:01.950+0000 7f771b200640 1 -- 172.21.15.39:0/2998667920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7714157d50 con 0x7f7714156c50 2024-09-21T07:42:01.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:01.952+0000 7f771b200640 1 -- 172.21.15.39:0/2998667920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7714155850 con 0x7f7714156c50 2024-09-21T07:42:01.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:01.953+0000 7f7710c00640 1 -- 172.21.15.39:0/2998667920 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f76fc01b460 con 0x7f7714156c50 2024-09-21T07:42:01.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:01.954+0000 7f7710c00640 1 --2- 172.21.15.39:0/2998667920 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f76f0078b90 0x7f76f007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:01.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:01.954+0000 7f7710c00640 1 -- 172.21.15.39:0/2998667920 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7947+0+0 (secure 0 0 0) 0x7f76fc09b9b0 con 0x7f7714156c50 2024-09-21T07:42:01.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:01.954+0000 7f7713e00640 1 --2- 172.21.15.39:0/2998667920 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f76f0078b90 0x7f76f007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:01.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:01.955+0000 7f7713e00640 1 --2- 172.21.15.39:0/2998667920 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f76f0078b90 0x7f76f007b0a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7704000c00 tx=0x7f7704006b70 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:01.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:01.959+0000 7f7710c00640 1 -- 172.21.15.39:0/2998667920 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f76fc063670 con 0x7f7714156c50 2024-09-21T07:42:02.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:02.195+0000 7f771b200640 1 -- 172.21.15.39:0/2998667920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7714161050 con 0x7f7714156c50 2024-09-21T07:42:02.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:02.195+0000 7f7710c00640 1 -- 172.21.15.39:0/2998667920 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 132 v132) ==== 95+0+5251 (secure 0 0 0) 0x7f76fc062f70 con 0x7f7714156c50 2024-09-21T07:42:02.197 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:42:02.198 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":132,"btime":"2024-09-21T07:41:53:824647+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7644,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1326711417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1326711417},{"type":"v1","addr":"172.21.15.39:6835","nonce":1326711417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":131}],"filesystems":[{"mdsmap":{"epoch":132,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:41:52.829432+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":6425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6425":{"gid":6425,"name":"d","rank":1,"incarnation":130,"state":"up:reconnect","state_seq":14,"addr":"172.21.15.148:6837/73220122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":73220122},{"type":"v1","addr":"172.21.15.148:6837","nonce":73220122}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311]},"id":6}]} 2024-09-21T07:42:02.198 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 132 2024-09-21T07:42:02.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:02.198+0000 7f771b200640 1 -- 172.21.15.39:0/2998667920 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f76f0078b90 msgr2=0x7f76f007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:02.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:02.198+0000 7f771b200640 1 --2- 172.21.15.39:0/2998667920 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f76f0078b90 0x7f76f007b0a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7704000c00 tx=0x7f7704006b70 comp rx=0 tx=0).stop 2024-09-21T07:42:02.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:02.198+0000 7f771b200640 1 -- 172.21.15.39:0/2998667920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7714156c50 msgr2=0x7f7714155bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:02.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:02.198+0000 7f771b200640 1 --2- 172.21.15.39:0/2998667920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7714156c50 0x7f7714155bd0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f76fc000fb0 tx=0x7f76fc00e940 comp rx=0 tx=0).stop 2024-09-21T07:42:02.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:02.198+0000 7f771b200640 1 -- 172.21.15.39:0/2998667920 shutdown_connections 2024-09-21T07:42:02.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:02.198+0000 7f771b200640 1 --2- 172.21.15.39:0/2998667920 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f76f0078b90 0x7f76f007b0a0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:02.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:02.198+0000 7f771b200640 1 --2- 172.21.15.39:0/2998667920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7714156110 0x7f7714156520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:02.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:02.198+0000 7f771b200640 1 --2- 172.21.15.39:0/2998667920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7714156c50 0x7f7714155bd0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:02.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:02.198+0000 7f771b200640 1 --2- 172.21.15.39:0/2998667920 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f76f8005250 0x7f7714152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:02.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:02.198+0000 7f771b200640 1 -- 172.21.15.39:0/2998667920 >> 172.21.15.39:0/2998667920 conn(0x7f771406cfc0 msgr2=0x7f77141618e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:02.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:02.198+0000 7f771b200640 1 -- 172.21.15.39:0/2998667920 shutdown_connections 2024-09-21T07:42:02.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:02.198+0000 7f771b200640 1 -- 172.21.15.39:0/2998667920 wait complete. 2024-09-21T07:42:02.625 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 132, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:36:20.189287+0000', 'modified': '2024-09-21T07:41:52.829432+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 112, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 7311, 'mds_1': 6425}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6425': {'gid': 6425, 'name': 'd', 'rank': 1, 'incarnation': 130, 'state': 'up:reconnect', 'state_seq': 14, 'addr': '172.21.15.148:6837/73220122', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 73220122}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 73220122}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_7311': {'gid': 7311, 'name': 'c', 'rank': 0, 'incarnation': 119, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.39:6833/1790581246', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 1790581246}, {'type': 'v1', 'addr': '172.21.15.39:6833', 'nonce': 1790581246}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7311, 'qdb_cluster': [7311]} 2024-09-21T07:42:02.625 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6425:up:reconnect 2024-09-21T07:42:03.626 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:42:03.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:03.864+0000 7f1920a00640 1 -- 172.21.15.39:0/609902958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f191c0b5230 msgr2=0x7f191c15f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:03.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:03.864+0000 7f1920a00640 1 --2- 172.21.15.39:0/609902958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f191c0b5230 0x7f191c15f060 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f191c06e290 tx=0x7f191402a980 comp rx=0 tx=0).stop 2024-09-21T07:42:03.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:03.864+0000 7f1920a00640 1 -- 172.21.15.39:0/609902958 shutdown_connections 2024-09-21T07:42:03.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:03.864+0000 7f1920a00640 1 --2- 172.21.15.39:0/609902958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f191c0b5230 0x7f191c15f060 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:03.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:03.864+0000 7f1920a00640 1 --2- 172.21.15.39:0/609902958 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f191c1524a0 0x7f191c0b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:03.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:03.864+0000 7f1920a00640 1 --2- 172.21.15.39:0/609902958 >> v2:172.21.15.148:3300/0 conn(0x7f191c0d04a0 0x7f191c0cdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:03.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:03.864+0000 7f1920a00640 1 -- 172.21.15.39:0/609902958 >> 172.21.15.39:0/609902958 conn(0x7f191c074050 msgr2=0x7f191c074490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:03.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:03.864+0000 7f1920a00640 1 -- 172.21.15.39:0/609902958 shutdown_connections 2024-09-21T07:42:03.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:03.864+0000 7f1920a00640 1 -- 172.21.15.39:0/609902958 wait complete. 2024-09-21T07:42:03.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:03.866+0000 7f1920a00640 1 Processor -- start 2024-09-21T07:42:03.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:03.866+0000 7f1920a00640 1 -- start start 2024-09-21T07:42:03.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:03.868+0000 7f1920a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f191c0b5230 0x7f191c1f6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:03.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:03.868+0000 7f1920a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f191c0d04a0 0x7f191c14ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:03.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:03.868+0000 7f1920a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f191c1524a0 0x7f191c14d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:03.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:03.868+0000 7f1920a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f191c15f930 con 0x7f191c1524a0 2024-09-21T07:42:03.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:03.868+0000 7f1920a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f191c15f7b0 con 0x7f191c0b5230 2024-09-21T07:42:03.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:03.868+0000 7f1920a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f191c15fab0 con 0x7f191c0d04a0 2024-09-21T07:42:03.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:03.868+0000 7f191be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f191c1524a0 0x7f191c14d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:03.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:03.868+0000 7f191be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f191c1524a0 0x7f191c14d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51730/0 (socket says 172.21.15.39:51730) 2024-09-21T07:42:03.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:03.868+0000 7f191be00640 1 -- 172.21.15.39:0/1188979851 learned_addr learned my addr 172.21.15.39:0/1188979851 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:03.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:03.868+0000 7f1912a00640 1 --2- 172.21.15.39:0/1188979851 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f191c0d04a0 0x7f191c14ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:03.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:03.868+0000 7f191b400640 1 --2- 172.21.15.39:0/1188979851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f191c0b5230 0x7f191c1f6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:03.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:03.868+0000 7f191be00640 1 -- 172.21.15.39:0/1188979851 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f191c0d04a0 msgr2=0x7f191c14ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:03.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:03.869+0000 7f191be00640 1 --2- 172.21.15.39:0/1188979851 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f191c0d04a0 0x7f191c14ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:03.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:03.869+0000 7f191be00640 1 -- 172.21.15.39:0/1188979851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f191c0b5230 msgr2=0x7f191c1f6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:03.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:03.869+0000 7f191be00640 1 --2- 172.21.15.39:0/1188979851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f191c0b5230 0x7f191c1f6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:03.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:03.869+0000 7f191be00640 1 -- 172.21.15.39:0/1188979851 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f191c14ef70 con 0x7f191c1524a0 2024-09-21T07:42:03.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:03.869+0000 7f191be00640 1 --2- 172.21.15.39:0/1188979851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f191c1524a0 0x7f191c14d1d0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f1914009d60 tx=0x7f1914003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:03.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:03.869+0000 7f1913e00640 1 -- 172.21.15.39:0/1188979851 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1914034040 con 0x7f191c1524a0 2024-09-21T07:42:03.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:03.869+0000 7f1913e00640 1 -- 172.21.15.39:0/1188979851 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1914004540 con 0x7f191c1524a0 2024-09-21T07:42:03.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:03.869+0000 7f1913e00640 1 -- 172.21.15.39:0/1188979851 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1914045480 con 0x7f191c1524a0 2024-09-21T07:42:03.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:03.869+0000 7f1920a00640 1 -- 172.21.15.39:0/1188979851 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f191c15cb30 con 0x7f191c1524a0 2024-09-21T07:42:03.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:03.869+0000 7f1920a00640 1 -- 172.21.15.39:0/1188979851 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f191c150c20 con 0x7f191c1524a0 2024-09-21T07:42:03.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:03.871+0000 7f1920a00640 1 -- 172.21.15.39:0/1188979851 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f191c0cdea0 con 0x7f191c1524a0 2024-09-21T07:42:03.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:03.872+0000 7f1913e00640 1 -- 172.21.15.39:0/1188979851 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1914037020 con 0x7f191c1524a0 2024-09-21T07:42:03.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:03.873+0000 7f1913e00640 1 --2- 172.21.15.39:0/1188979851 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f18f8078b90 0x7f18f807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:03.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:03.873+0000 7f1913e00640 1 -- 172.21.15.39:0/1188979851 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7947+0+0 (secure 0 0 0) 0x7f19140bce50 con 0x7f191c1524a0 2024-09-21T07:42:03.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:03.874+0000 7f191b400640 1 --2- 172.21.15.39:0/1188979851 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f18f8078b90 0x7f18f807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:03.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:03.875+0000 7f191b400640 1 --2- 172.21.15.39:0/1188979851 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f18f8078b90 0x7f18f807b0a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f190c00a4d0 tx=0x7f190c00a460 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:03.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:03.878+0000 7f1913e00640 1 -- 172.21.15.39:0/1188979851 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1914084380 con 0x7f191c1524a0 2024-09-21T07:42:04.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.117+0000 7f1920a00640 1 -- 172.21.15.39:0/1188979851 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f191c15e990 con 0x7f191c1524a0 2024-09-21T07:42:04.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.117+0000 7f1913e00640 1 -- 172.21.15.39:0/1188979851 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 134 v134) ==== 95+0+5253 (secure 0 0 0) 0x7f1914088db0 con 0x7f191c1524a0 2024-09-21T07:42:04.120 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:42:04.120 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":134,"btime":"2024-09-21T07:42:03:879182+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7644,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1326711417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1326711417},{"type":"v1","addr":"172.21.15.39:6835","nonce":1326711417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":131}],"filesystems":[{"mdsmap":{"epoch":134,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:42:03.879172+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":6425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6425":{"gid":6425,"name":"d","rank":1,"incarnation":130,"state":"up:active","state_seq":18,"addr":"172.21.15.148:6837/73220122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":73220122},{"type":"v1","addr":"172.21.15.148:6837","nonce":73220122}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[6425,7311]},"id":6}]} 2024-09-21T07:42:04.121 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 134 2024-09-21T07:42:04.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.120+0000 7f1920a00640 1 -- 172.21.15.39:0/1188979851 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f18f8078b90 msgr2=0x7f18f807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:04.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.120+0000 7f1920a00640 1 --2- 172.21.15.39:0/1188979851 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f18f8078b90 0x7f18f807b0a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f190c00a4d0 tx=0x7f190c00a460 comp rx=0 tx=0).stop 2024-09-21T07:42:04.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.120+0000 7f1920a00640 1 -- 172.21.15.39:0/1188979851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f191c1524a0 msgr2=0x7f191c14d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:04.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.120+0000 7f1920a00640 1 --2- 172.21.15.39:0/1188979851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f191c1524a0 0x7f191c14d1d0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f1914009d60 tx=0x7f1914003f10 comp rx=0 tx=0).stop 2024-09-21T07:42:04.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.120+0000 7f1920a00640 1 -- 172.21.15.39:0/1188979851 shutdown_connections 2024-09-21T07:42:04.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.120+0000 7f1920a00640 1 --2- 172.21.15.39:0/1188979851 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f18f8078b90 0x7f18f807b0a0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:04.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.120+0000 7f1920a00640 1 --2- 172.21.15.39:0/1188979851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f191c1524a0 0x7f191c14d1d0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:04.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.120+0000 7f1920a00640 1 --2- 172.21.15.39:0/1188979851 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f191c0d04a0 0x7f191c14ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:04.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.120+0000 7f1920a00640 1 --2- 172.21.15.39:0/1188979851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f191c0b5230 0x7f191c1f6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:04.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.120+0000 7f1920a00640 1 -- 172.21.15.39:0/1188979851 >> 172.21.15.39:0/1188979851 conn(0x7f191c074050 msgr2=0x7f191c0aff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:04.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.120+0000 7f1920a00640 1 -- 172.21.15.39:0/1188979851 shutdown_connections 2024-09-21T07:42:04.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.120+0000 7f1920a00640 1 -- 172.21.15.39:0/1188979851 wait complete. 2024-09-21T07:42:04.549 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 134, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:36:20.189287+0000', 'modified': '2024-09-21T07:42:03.879172+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 112, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 7311, 'mds_1': 6425}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6425': {'gid': 6425, 'name': 'd', 'rank': 1, 'incarnation': 130, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.148:6837/73220122', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 73220122}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 73220122}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_7311': {'gid': 7311, 'name': 'c', 'rank': 0, 'incarnation': 119, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.39:6833/1790581246', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 1790581246}, {'type': 'v1', 'addr': '172.21.15.39:6833', 'nonce': 1790581246}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7311, 'qdb_cluster': [6425, 7311]} 2024-09-21T07:42:04.550 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2024-09-21T07:42:04.550 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-21T07:42:04.671 INFO:tasks.ceph.mds.c.smithi039.stderr:2024-09-21T07:42:04.670+0000 7f9202600640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:42:04.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.782+0000 7fe558a00640 1 -- 172.21.15.39:0/3973258449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe538004d40 msgr2=0x7fe5380051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:04.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.782+0000 7fe558a00640 1 --2- 172.21.15.39:0/3973258449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe538004d40 0x7fe5380051f0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fe54c00c780 tx=0x7fe538005830 comp rx=0 tx=0).stop 2024-09-21T07:42:04.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.782+0000 7fe553200640 1 -- 172.21.15.39:0/3973258449 reap_dead start 2024-09-21T07:42:04.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.783+0000 7fe558a00640 1 -- 172.21.15.39:0/3973258449 shutdown_connections 2024-09-21T07:42:04.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.783+0000 7fe558a00640 1 -- 172.21.15.39:0/3973258449 >> 172.21.15.39:0/3973258449 conn(0x7fe554074050 msgr2=0x7fe554074490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:04.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.783+0000 7fe558a00640 1 -- 172.21.15.39:0/3973258449 shutdown_connections 2024-09-21T07:42:04.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.783+0000 7fe558a00640 1 -- 172.21.15.39:0/3973258449 wait complete. 2024-09-21T07:42:04.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.785+0000 7fe558a00640 1 Processor -- start 2024-09-21T07:42:04.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.785+0000 7fe558a00640 1 -- start start 2024-09-21T07:42:04.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.786+0000 7fe558a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe538004d40 0x7fe554151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:04.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.786+0000 7fe558a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5541e69f0 0x7fe55414fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:04.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.786+0000 7fe558a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe554153d70 0x7fe5541f8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:04.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.786+0000 7fe558a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe55415d290 con 0x7fe5541e69f0 2024-09-21T07:42:04.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.786+0000 7fe558a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe5540ce320 con 0x7fe538004d40 2024-09-21T07:42:04.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.787+0000 7fe558a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe55415d410 con 0x7fe554153d70 2024-09-21T07:42:04.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.787+0000 7fe551e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5541e69f0 0x7fe55414fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:04.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.787+0000 7fe551e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5541e69f0 0x7fe55414fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51752/0 (socket says 172.21.15.39:51752) 2024-09-21T07:42:04.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.787+0000 7fe551e00640 1 -- 172.21.15.39:0/2548425651 learned_addr learned my addr 172.21.15.39:0/2548425651 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:04.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.787+0000 7fe552800640 1 --2- 172.21.15.39:0/2548425651 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe538004d40 0x7fe554151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:04.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.787+0000 7fe553200640 1 --2- 172.21.15.39:0/2548425651 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe554153d70 0x7fe5541f8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:04.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.787+0000 7fe551e00640 1 -- 172.21.15.39:0/2548425651 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe554153d70 msgr2=0x7fe5541f8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:04.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.787+0000 7fe551e00640 1 --2- 172.21.15.39:0/2548425651 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe554153d70 0x7fe5541f8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:04.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.787+0000 7fe551e00640 1 -- 172.21.15.39:0/2548425651 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe538004d40 msgr2=0x7fe554151340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:04.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.787+0000 7fe551e00640 1 --2- 172.21.15.39:0/2548425651 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe538004d40 0x7fe554151340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:04.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.787+0000 7fe551e00640 1 -- 172.21.15.39:0/2548425651 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5540717e0 con 0x7fe5541e69f0 2024-09-21T07:42:04.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.787+0000 7fe552800640 1 --2- 172.21.15.39:0/2548425651 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe538004d40 0x7fe554151340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:42:04.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.787+0000 7fe553200640 1 --2- 172.21.15.39:0/2548425651 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe554153d70 0x7fe5541f8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:42:04.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.787+0000 7fe551e00640 1 --2- 172.21.15.39:0/2548425651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5541e69f0 0x7fe55414fef0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fe540003f20 tx=0x7fe5400016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:04.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.788+0000 7fe54aa00640 1 -- 172.21.15.39:0/2548425651 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe540011640 con 0x7fe5541e69f0 2024-09-21T07:42:04.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.788+0000 7fe54aa00640 1 -- 172.21.15.39:0/2548425651 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe540016020 con 0x7fe5541e69f0 2024-09-21T07:42:04.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.788+0000 7fe558a00640 1 -- 172.21.15.39:0/2548425651 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe554071a80 con 0x7fe5541e69f0 2024-09-21T07:42:04.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.788+0000 7fe54aa00640 1 -- 172.21.15.39:0/2548425651 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe540011a80 con 0x7fe5541e69f0 2024-09-21T07:42:04.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.788+0000 7fe558a00640 1 -- 172.21.15.39:0/2548425651 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe554153570 con 0x7fe5541e69f0 2024-09-21T07:42:04.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.790+0000 7fe54aa00640 1 -- 172.21.15.39:0/2548425651 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe540011c20 con 0x7fe5541e69f0 2024-09-21T07:42:04.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.791+0000 7fe54aa00640 1 --2- 172.21.15.39:0/2548425651 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe534078b90 0x7fe53407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:04.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.792+0000 7fe54aa00640 1 -- 172.21.15.39:0/2548425651 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7947+0+0 (secure 0 0 0) 0x7fe54009c210 con 0x7fe5541e69f0 2024-09-21T07:42:04.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.792+0000 7fe552800640 1 --2- 172.21.15.39:0/2548425651 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe534078b90 0x7fe53407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:04.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.792+0000 7fe5603dd740 1 Processor -- start 2024-09-21T07:42:04.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.792+0000 7fe5603dd740 1 -- start start 2024-09-21T07:42:04.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.793+0000 7fe552800640 1 --2- 172.21.15.39:0/2548425651 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe534078b90 0x7fe53407b0a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe544009cf0 tx=0x7fe544005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:04.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.794+0000 7fe5603dd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7c1a95270 0x55c7c1e65b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:04.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.794+0000 7fe5603dd740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c7c1a6c3b0 0x55c7c1e660a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:04.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.794+0000 7fe5603dd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c7c1a74590 0x55c7c1e36150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:04.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.794+0000 7fe5603dd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c7c1dbf430 con 0x55c7c1a95270 2024-09-21T07:42:04.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.794+0000 7fe5603dd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c7c1daf5d0 con 0x55c7c1a74590 2024-09-21T07:42:04.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.794+0000 7fe5603dd740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55c7c1d9ea00 con 0x55c7c1a6c3b0 2024-09-21T07:42:04.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.794+0000 7fe552800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7c1a95270 0x55c7c1e65b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:04.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.794+0000 7fe552800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7c1a95270 0x55c7c1e65b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51764/0 (socket says 172.21.15.39:51764) 2024-09-21T07:42:04.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.794+0000 7fe552800640 1 -- 172.21.15.39:0/724098222 learned_addr learned my addr 172.21.15.39:0/724098222 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:04.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.794+0000 7fe551e00640 1 --2- 172.21.15.39:0/724098222 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c7c1a6c3b0 0x55c7c1e660a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:04.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.794+0000 7fe553200640 1 --2- 172.21.15.39:0/724098222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c7c1a74590 0x55c7c1e36150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:04.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.794+0000 7fe552800640 1 -- 172.21.15.39:0/724098222 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c7c1a6c3b0 msgr2=0x55c7c1e660a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:42:04.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.794+0000 7fe552800640 1 --2- 172.21.15.39:0/724098222 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c7c1a6c3b0 0x55c7c1e660a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:04.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.794+0000 7fe552800640 1 -- 172.21.15.39:0/724098222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c7c1a74590 msgr2=0x55c7c1e36150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:42:04.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.794+0000 7fe552800640 1 --2- 172.21.15.39:0/724098222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c7c1a74590 0x55c7c1e36150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:04.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.794+0000 7fe552800640 1 -- 172.21.15.39:0/724098222 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7c1d67520 con 0x55c7c1a95270 2024-09-21T07:42:04.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.794+0000 7fe553200640 1 --2- 172.21.15.39:0/724098222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c7c1a74590 0x55c7c1e36150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:42:04.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.795+0000 7fe552800640 1 --2- 172.21.15.39:0/724098222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7c1a95270 0x55c7c1e65b60 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fe5440114d0 tx=0x7fe544016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa0a944290ef9e55 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:04.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.795+0000 7fe53f000640 1 -- 172.21.15.39:0/724098222 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe544011e00 con 0x55c7c1a95270 2024-09-21T07:42:04.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.795+0000 7fe53f000640 1 -- 172.21.15.39:0/724098222 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe544028430 con 0x55c7c1a95270 2024-09-21T07:42:04.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.795+0000 7fe53f000640 1 -- 172.21.15.39:0/724098222 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe544028820 con 0x55c7c1a95270 2024-09-21T07:42:04.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.796+0000 7fe5603dd740 1 -- 172.21.15.39:0/724098222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7c1a95270 msgr2=0x55c7c1e65b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:04.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.796+0000 7fe5603dd740 1 --2- 172.21.15.39:0/724098222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7c1a95270 0x55c7c1e65b60 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fe5440114d0 tx=0x7fe544016000 comp rx=0 tx=0).stop 2024-09-21T07:42:04.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.796+0000 7fe5603dd740 1 -- 172.21.15.39:0/724098222 shutdown_connections 2024-09-21T07:42:04.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.796+0000 7fe5603dd740 1 --2- 172.21.15.39:0/724098222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c7c1a74590 0x55c7c1e36150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:04.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.796+0000 7fe5603dd740 1 --2- 172.21.15.39:0/724098222 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c7c1a6c3b0 0x55c7c1e660a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:04.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.796+0000 7fe5603dd740 1 --2- 172.21.15.39:0/724098222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7c1a95270 0x55c7c1e65b60 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:04.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.796+0000 7fe5603dd740 1 -- 172.21.15.39:0/724098222 >> 172.21.15.39:0/724098222 conn(0x55c7c1a893c0 msgr2=0x55c7c1e3bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:04.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.796+0000 7fe5603dd740 1 -- 172.21.15.39:0/724098222 shutdown_connections 2024-09-21T07:42:04.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.796+0000 7fe5603dd740 1 -- 172.21.15.39:0/724098222 wait complete. 2024-09-21T07:42:04.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.797+0000 7fe5603dd740 1 -- start start 2024-09-21T07:42:04.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.797+0000 7fe5603dd740 1 Processor -- start 2024-09-21T07:42:04.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.799+0000 7fe5603dd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c7c1a6c3b0 0x55c7c1e5be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:04.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.799+0000 7fe5603dd740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c7c1a74590 0x55c7c1e8f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:04.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.799+0000 7fe5603dd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7c1a95270 0x55c7c1e83ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:04.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.799+0000 7fe5603dd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c7c1d45330 con 0x55c7c1a95270 2024-09-21T07:42:04.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.799+0000 7fe5603dd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c7c1da2810 con 0x55c7c1a6c3b0 2024-09-21T07:42:04.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.799+0000 7fe5603dd740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55c7c1b0f620 con 0x55c7c1a74590 2024-09-21T07:42:04.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.799+0000 7fe5603dd740 10 client.0 _ll_register_callbacks cb 0x55c7c1d4a420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:42:04.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.799+0000 7fe553200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7c1a95270 0x55c7c1e83ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:04.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.799+0000 7fe552800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c7c1a6c3b0 0x55c7c1e5be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:04.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.799+0000 7fe551e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c7c1a74590 0x55c7c1e8f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:04.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.799+0000 7fe553200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7c1a95270 0x55c7c1e83ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51774/0 (socket says 172.21.15.39:51774) 2024-09-21T07:42:04.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.799+0000 7fe552800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c7c1a6c3b0 0x55c7c1e5be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.39:43088/0 (socket says 172.21.15.39:43088) 2024-09-21T07:42:04.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.799+0000 7fe553200640 1 -- 172.21.15.39:0/1148286990 learned_addr learned my addr 172.21.15.39:0/1148286990 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:04.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.800+0000 7fe553200640 1 -- 172.21.15.39:0/1148286990 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c7c1a74590 msgr2=0x55c7c1e8f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:42:04.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.800+0000 7fe553200640 1 --2- 172.21.15.39:0/1148286990 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c7c1a74590 0x55c7c1e8f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:04.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.800+0000 7fe553200640 1 -- 172.21.15.39:0/1148286990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c7c1a6c3b0 msgr2=0x55c7c1e5be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:42:04.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.800+0000 7fe553200640 1 --2- 172.21.15.39:0/1148286990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c7c1a6c3b0 0x55c7c1e5be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:04.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.800+0000 7fe553200640 1 -- 172.21.15.39:0/1148286990 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7c1b00850 con 0x55c7c1a95270 2024-09-21T07:42:04.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.800+0000 7fe552800640 1 --2- 172.21.15.39:0/1148286990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c7c1a6c3b0 0x55c7c1e5be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:42:04.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.800+0000 7fe553200640 1 --2- 172.21.15.39:0/1148286990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7c1a95270 0x55c7c1e83ef0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fe54c02b640 tx=0x7fe54c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508c67d721bdf98 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:04.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.800+0000 7fe533400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:42:04.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.800+0000 7fe533400640 1 -- 172.21.15.39:0/1148286990 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe544009720 con 0x55c7c1a95270 2024-09-21T07:42:04.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.800+0000 7fe5603dd740 1 -- 172.21.15.39:0/1148286990 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe544028430 con 0x55c7c1a95270 2024-09-21T07:42:04.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.800+0000 7fe533400640 1 -- 172.21.15.39:0/1148286990 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 7947+0+0 (secure 0 0 0) 0x7fe54c001a40 con 0x55c7c1a95270 2024-09-21T07:42:04.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.801+0000 7fe533400640 1 -- 172.21.15.39:0/1148286990 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=134) ==== 24+0+0 (secure 0 0 0) 0x7fe54c048050 con 0x55c7c1a95270 2024-09-21T07:42:04.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.801+0000 7fe5603dd740 10 client.7701 fetch_fsmap learned FSMap version 134 2024-09-21T07:42:04.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.801+0000 7fe5603dd740 1 -- 172.21.15.39:0/1148286990 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=134,monmap=2+}) -- 0x55c7c1b776b0 con 0x55c7c1a95270 2024-09-21T07:42:04.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.801+0000 7fe533400640 1 -- 172.21.15.39:0/1148286990 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe544028430 con 0x55c7c1a95270 2024-09-21T07:42:04.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.801+0000 7fe533400640 1 -- 172.21.15.39:0/1148286990 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 134) ==== 3522+0+0 (secure 0 0 0) 0x7fe54c004b40 con 0x55c7c1a95270 2024-09-21T07:42:04.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.801+0000 7fe5603dd740 10 client.7701 fetch_fsmap finished waiting for FSMap version 134 2024-09-21T07:42:04.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.801+0000 7fe5603dd740 10 client.7701 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T07:42:04.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.801+0000 7fe5603dd740 20 client.7701 populate_metadata read hostname 'smithi039' 2024-09-21T07:42:04.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.801+0000 7fe5603dd740 1 --2- 172.21.15.39:0/1148286990 >> [v2:172.21.15.148:6836/73220122,v1:172.21.15.148:6837/73220122] conn(0x55c7c1a53e40 0x55c7c1e779c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:04.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.801+0000 7fe5603dd740 4 client.7701 mds_command: new command op to 6425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:42:04.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.801+0000 7fe5603dd740 1 -- 172.21.15.39:0/1148286990 --> [v2:172.21.15.148:6836/73220122,v1:172.21.15.148:6837/73220122] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7c1db1520 con 0x55c7c1a53e40 2024-09-21T07:42:04.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.802+0000 7fe552800640 1 --2- 172.21.15.39:0/1148286990 >> [v2:172.21.15.148:6836/73220122,v1:172.21.15.148:6837/73220122] conn(0x55c7c1a53e40 0x55c7c1e779c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:04.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.802+0000 7fe552800640 1 --2- 172.21.15.39:0/1148286990 >> [v2:172.21.15.148:6836/73220122,v1:172.21.15.148:6837/73220122] conn(0x55c7c1a53e40 0x55c7c1e779c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=af6a4ba1197000a9 server_cookie=3be18e7ebb1eee6f in_seq=0 out_seq=0 2024-09-21T07:42:04.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.802+0000 7fe533400640 10 client.7701 ms_handle_connect on v2:172.21.15.148:6836/73220122 2024-09-21T07:42:04.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.803+0000 7fe533400640 1 -- 172.21.15.39:0/1148286990 <== mds.1 v2:172.21.15.148:6836/73220122 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c7c1db1520 con 0x55c7c1a53e40 2024-09-21T07:42:04.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.803+0000 7fe533400640 10 client.7701 handle_command_reply: tid=0 2024-09-21T07:42:04.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.803+0000 7fe5603dd740 1 client.7701 shutdown 2024-09-21T07:42:04.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.803+0000 7fe5603dd740 1 -- 172.21.15.39:0/1148286990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7c1a95270 msgr2=0x55c7c1e83ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:04.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.803+0000 7fe5603dd740 1 --2- 172.21.15.39:0/1148286990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7c1a95270 0x55c7c1e83ef0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fe54c02b640 tx=0x7fe54c00c3a0 comp rx=0 tx=0).stop 2024-09-21T07:42:04.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.803+0000 7fe5603dd740 1 -- 172.21.15.39:0/1148286990 shutdown_connections 2024-09-21T07:42:04.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.804+0000 7fe5603dd740 1 --2- 172.21.15.39:0/1148286990 >> [v2:172.21.15.148:6836/73220122,v1:172.21.15.148:6837/73220122] conn(0x55c7c1a53e40 0x55c7c1e779c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:04.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.804+0000 7fe5603dd740 1 --2- 172.21.15.39:0/1148286990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7c1a95270 0x55c7c1e83ef0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:04.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.804+0000 7fe5603dd740 1 --2- 172.21.15.39:0/1148286990 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c7c1a74590 0x55c7c1e8f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:04.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.804+0000 7fe5603dd740 1 --2- 172.21.15.39:0/1148286990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c7c1a6c3b0 0x55c7c1e5be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:04.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.804+0000 7fe533400640 0 client.7701 ms_handle_reset on v2:172.21.15.148:6836/73220122 2024-09-21T07:42:04.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.804+0000 7fe5603dd740 1 -- 172.21.15.39:0/1148286990 >> 172.21.15.39:0/1148286990 conn(0x55c7c1a893c0 msgr2=0x55c7c1e5d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:04.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.804+0000 7fe5603dd740 1 -- 172.21.15.39:0/1148286990 shutdown_connections 2024-09-21T07:42:04.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.804+0000 7fe5603dd740 1 -- 172.21.15.39:0/1148286990 wait complete. 2024-09-21T07:42:04.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.804+0000 7fe5603dd740 20 client.7701 trim_cache size 0 max 16384 2024-09-21T07:42:04.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.823+0000 7fe5603dd740 1 Processor -- start 2024-09-21T07:42:04.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.823+0000 7fe5603dd740 1 -- start start 2024-09-21T07:42:04.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.824+0000 7fe5603dd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c7c1a6c3b0 0x55c7c1e3b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:04.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.824+0000 7fe5603dd740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c7c1a74590 0x55c7c1e3bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:04.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.824+0000 7fe5603dd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7c1a95270 0x55c7c1e5faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:04.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.824+0000 7fe5603dd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe554205650 con 0x55c7c1a95270 2024-09-21T07:42:04.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.824+0000 7fe5603dd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe554203900 con 0x55c7c1a6c3b0 2024-09-21T07:42:04.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.824+0000 7fe5603dd740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe554203f50 con 0x55c7c1a74590 2024-09-21T07:42:04.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.825+0000 7fe553200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7c1a95270 0x55c7c1e5faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:04.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.825+0000 7fe553200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7c1a95270 0x55c7c1e5faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51786/0 (socket says 172.21.15.39:51786) 2024-09-21T07:42:04.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.825+0000 7fe553200640 1 -- 172.21.15.39:0/1159865056 learned_addr learned my addr 172.21.15.39:0/1159865056 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:04.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.825+0000 7fe553200640 1 -- 172.21.15.39:0/1159865056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c7c1a74590 msgr2=0x55c7c1e3bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:42:04.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.825+0000 7fe553200640 1 --2- 172.21.15.39:0/1159865056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c7c1a74590 0x55c7c1e3bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:04.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.825+0000 7fe553200640 1 -- 172.21.15.39:0/1159865056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c7c1a6c3b0 msgr2=0x55c7c1e3b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:42:04.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.825+0000 7fe552800640 1 --2- 172.21.15.39:0/1159865056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c7c1a6c3b0 0x55c7c1e3b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:04.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.825+0000 7fe553200640 1 --2- 172.21.15.39:0/1159865056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c7c1a6c3b0 0x55c7c1e3b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:04.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.825+0000 7fe553200640 1 -- 172.21.15.39:0/1159865056 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7c1e8f8e0 con 0x55c7c1a95270 2024-09-21T07:42:04.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.825+0000 7fe552800640 1 --2- 172.21.15.39:0/1159865056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c7c1a6c3b0 0x55c7c1e3b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:42:04.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.825+0000 7fe553200640 1 --2- 172.21.15.39:0/1159865056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7c1a95270 0x55c7c1e5faa0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fe54c001650 tx=0x7fe54c001680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f08edd04a5c3af server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:04.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.826+0000 7fe53e600640 1 -- 172.21.15.39:0/1159865056 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c001a00 con 0x55c7c1a95270 2024-09-21T07:42:04.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.826+0000 7fe53e600640 1 -- 172.21.15.39:0/1159865056 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe54c02f070 con 0x55c7c1a95270 2024-09-21T07:42:04.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.826+0000 7fe53e600640 1 -- 172.21.15.39:0/1159865056 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c02bbf0 con 0x55c7c1a95270 2024-09-21T07:42:04.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.826+0000 7fe5603dd740 1 -- 172.21.15.39:0/1159865056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7c1a95270 msgr2=0x55c7c1e5faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:04.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.826+0000 7fe5603dd740 1 --2- 172.21.15.39:0/1159865056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7c1a95270 0x55c7c1e5faa0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fe54c001650 tx=0x7fe54c001680 comp rx=0 tx=0).stop 2024-09-21T07:42:04.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.826+0000 7fe5603dd740 1 -- 172.21.15.39:0/1159865056 shutdown_connections 2024-09-21T07:42:04.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.826+0000 7fe5603dd740 1 --2- 172.21.15.39:0/1159865056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7c1a95270 0x55c7c1e5faa0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:04.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.826+0000 7fe5603dd740 1 --2- 172.21.15.39:0/1159865056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c7c1a74590 0x55c7c1e3bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:04.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.826+0000 7fe5603dd740 1 --2- 172.21.15.39:0/1159865056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c7c1a6c3b0 0x55c7c1e3b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:04.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.826+0000 7fe5603dd740 1 -- 172.21.15.39:0/1159865056 >> 172.21.15.39:0/1159865056 conn(0x55c7c1a893c0 msgr2=0x55c7c1e78f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:04.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.826+0000 7fe5603dd740 1 -- 172.21.15.39:0/1159865056 shutdown_connections 2024-09-21T07:42:04.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.827+0000 7fe5603dd740 1 -- 172.21.15.39:0/1159865056 wait complete. 2024-09-21T07:42:04.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.828+0000 7fe5603dd740 1 -- start start 2024-09-21T07:42:04.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.828+0000 7fe5603dd740 1 Processor -- start 2024-09-21T07:42:04.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.829+0000 7fe5603dd740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c7c1a6c3b0 0x55c7c1e949c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:04.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.829+0000 7fe5603dd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7c1a74590 0x55c7c1e78380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:04.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.829+0000 7fe5603dd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c7c1a95270 0x55c7c1efeff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:04.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.829+0000 7fe5603dd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c7c1e61b00 con 0x55c7c1a74590 2024-09-21T07:42:04.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.829+0000 7fe5603dd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c7c1e61230 con 0x55c7c1a95270 2024-09-21T07:42:04.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.829+0000 7fe5603dd740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55c7c1e62150 con 0x55c7c1a6c3b0 2024-09-21T07:42:04.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.829+0000 7fe5603dd740 10 client.0 _ll_register_callbacks cb 0x7fe544005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:42:04.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.830+0000 7fe551e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7c1a74590 0x55c7c1e78380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:04.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.830+0000 7fe551e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7c1a74590 0x55c7c1e78380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51802/0 (socket says 172.21.15.39:51802) 2024-09-21T07:42:04.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.830+0000 7fe551e00640 1 -- 172.21.15.39:0/4278205675 learned_addr learned my addr 172.21.15.39:0/4278205675 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:04.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.830+0000 7fe552800640 1 --2- 172.21.15.39:0/4278205675 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c7c1a6c3b0 0x55c7c1e949c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:04.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.830+0000 7fe553200640 1 --2- 172.21.15.39:0/4278205675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c7c1a95270 0x55c7c1efeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:04.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.830+0000 7fe551e00640 1 -- 172.21.15.39:0/4278205675 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c7c1a6c3b0 msgr2=0x55c7c1e949c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:42:04.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.830+0000 7fe551e00640 1 --2- 172.21.15.39:0/4278205675 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c7c1a6c3b0 0x55c7c1e949c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:04.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.830+0000 7fe551e00640 1 -- 172.21.15.39:0/4278205675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c7c1a95270 msgr2=0x55c7c1efeff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:42:04.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.830+0000 7fe551e00640 1 --2- 172.21.15.39:0/4278205675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c7c1a95270 0x55c7c1efeff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:04.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.830+0000 7fe551e00640 1 -- 172.21.15.39:0/4278205675 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7c1ea5010 con 0x55c7c1a74590 2024-09-21T07:42:04.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.830+0000 7fe552800640 1 --2- 172.21.15.39:0/4278205675 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c7c1a6c3b0 0x55c7c1e949c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:42:04.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.830+0000 7fe551e00640 1 --2- 172.21.15.39:0/4278205675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7c1a74590 0x55c7c1e78380 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fe540060980 tx=0x7fe540062e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1c6b872b3dca28 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:04.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.830+0000 7fe533400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:42:04.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.831+0000 7fe533400640 1 -- 172.21.15.39:0/4278205675 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe540015030 con 0x55c7c1a74590 2024-09-21T07:42:04.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.831+0000 7fe533400640 1 -- 172.21.15.39:0/4278205675 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 7947+0+0 (secure 0 0 0) 0x7fe54006c020 con 0x55c7c1a74590 2024-09-21T07:42:04.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.831+0000 7fe5603dd740 1 -- 172.21.15.39:0/4278205675 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe54c02f070 con 0x55c7c1a74590 2024-09-21T07:42:04.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.831+0000 7fe533400640 1 -- 172.21.15.39:0/4278205675 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=134) ==== 24+0+0 (secure 0 0 0) 0x7fe540063220 con 0x55c7c1a74590 2024-09-21T07:42:04.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.831+0000 7fe5603dd740 10 client.7707 fetch_fsmap learned FSMap version 134 2024-09-21T07:42:04.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.831+0000 7fe5603dd740 1 -- 172.21.15.39:0/4278205675 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=134,monmap=2+}) -- 0x55c7c1e5b960 con 0x55c7c1a74590 2024-09-21T07:42:04.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.831+0000 7fe533400640 1 -- 172.21.15.39:0/4278205675 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe54c02f070 con 0x55c7c1a74590 2024-09-21T07:42:04.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.831+0000 7fe533400640 1 -- 172.21.15.39:0/4278205675 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 134) ==== 3522+0+0 (secure 0 0 0) 0x7fe54007dbd0 con 0x55c7c1a74590 2024-09-21T07:42:04.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.831+0000 7fe5603dd740 10 client.7707 fetch_fsmap finished waiting for FSMap version 134 2024-09-21T07:42:04.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.831+0000 7fe5603dd740 10 client.7707 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T07:42:04.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.831+0000 7fe5603dd740 20 client.7707 populate_metadata read hostname 'smithi039' 2024-09-21T07:42:04.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.831+0000 7fe5603dd740 1 --2- 172.21.15.39:0/4278205675 >> [v2:172.21.15.148:6836/73220122,v1:172.21.15.148:6837/73220122] conn(0x55c7c1e9aab0 0x55c7c1f02d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:04.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.831+0000 7fe5603dd740 4 client.7707 mds_command: new command op to 6425 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:42:04.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.831+0000 7fe5603dd740 1 -- 172.21.15.39:0/4278205675 --> [v2:172.21.15.148:6836/73220122,v1:172.21.15.148:6837/73220122] -- command(tid 0: {"prefix": "status"}) -- 0x55c7c1e8f520 con 0x55c7c1e9aab0 2024-09-21T07:42:04.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.832+0000 7fe552800640 1 --2- 172.21.15.39:0/4278205675 >> [v2:172.21.15.148:6836/73220122,v1:172.21.15.148:6837/73220122] conn(0x55c7c1e9aab0 0x55c7c1f02d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:04.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.832+0000 7fe552800640 1 --2- 172.21.15.39:0/4278205675 >> [v2:172.21.15.148:6836/73220122,v1:172.21.15.148:6837/73220122] conn(0x55c7c1e9aab0 0x55c7c1f02d90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=936b6a6d94852ee9 server_cookie=98aee2b1a091059d in_seq=0 out_seq=0 2024-09-21T07:42:04.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.832+0000 7fe533400640 10 client.7707 ms_handle_connect on v2:172.21.15.148:6836/73220122 2024-09-21T07:42:04.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.832+0000 7fe533400640 1 -- 172.21.15.39:0/4278205675 <== mds.1 v2:172.21.15.148:6836/73220122 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55c7c1e8f520 con 0x55c7c1e9aab0 2024-09-21T07:42:04.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.832+0000 7fe533400640 10 client.7707 handle_command_reply: tid=0 2024-09-21T07:42:04.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.832+0000 7fe5603dd740 1 client.7707 shutdown 2024-09-21T07:42:04.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.833+0000 7fe5603dd740 1 -- 172.21.15.39:0/4278205675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7c1a74590 msgr2=0x55c7c1e78380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:04.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.833+0000 7fe5603dd740 1 --2- 172.21.15.39:0/4278205675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7c1a74590 0x55c7c1e78380 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fe540060980 tx=0x7fe540062e80 comp rx=0 tx=0).stop 2024-09-21T07:42:04.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.833+0000 7fe5603dd740 1 -- 172.21.15.39:0/4278205675 shutdown_connections 2024-09-21T07:42:04.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.833+0000 7fe5603dd740 1 --2- 172.21.15.39:0/4278205675 >> [v2:172.21.15.148:6836/73220122,v1:172.21.15.148:6837/73220122] conn(0x55c7c1e9aab0 0x55c7c1f02d90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:04.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.833+0000 7fe5603dd740 1 --2- 172.21.15.39:0/4278205675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c7c1a95270 0x55c7c1efeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:04.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.833+0000 7fe5603dd740 1 --2- 172.21.15.39:0/4278205675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7c1a74590 0x55c7c1e78380 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:04.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.833+0000 7fe533400640 0 client.7707 ms_handle_reset on v2:172.21.15.148:6836/73220122 2024-09-21T07:42:04.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.833+0000 7fe5603dd740 1 --2- 172.21.15.39:0/4278205675 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c7c1a6c3b0 0x55c7c1e949c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:04.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.833+0000 7fe5603dd740 1 -- 172.21.15.39:0/4278205675 >> 172.21.15.39:0/4278205675 conn(0x55c7c1a893c0 msgr2=0x55c7c1e37090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:04.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.833+0000 7fe5603dd740 1 -- 172.21.15.39:0/4278205675 shutdown_connections 2024-09-21T07:42:04.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.833+0000 7fe5603dd740 1 -- 172.21.15.39:0/4278205675 wait complete. 2024-09-21T07:42:04.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.833+0000 7fe5603dd740 20 client.7707 trim_cache size 0 max 16384 2024-09-21T07:42:04.838 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:42:04.838 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:42:04.838 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 1, 2024-09-21T07:42:04.838 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 6425, 2024-09-21T07:42:04.838 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:42:04.838 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:42:04.839 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:42:04.839 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 13.008722991999999, 2024-09-21T07:42:04.839 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 134, 2024-09-21T07:42:04.839 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 113, 2024-09-21T07:42:04.839 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 113, 2024-09-21T07:42:04.839 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 58.613557155999999 2024-09-21T07:42:04.839 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:42:04.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.834+0000 7fe558a00640 1 -- 172.21.15.39:0/2548425651 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe534078b90 msgr2=0x7fe53407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:04.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.834+0000 7fe558a00640 1 --2- 172.21.15.39:0/2548425651 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe534078b90 0x7fe53407b0a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe544009cf0 tx=0x7fe544005f90 comp rx=0 tx=0).stop 2024-09-21T07:42:04.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.834+0000 7fe558a00640 1 -- 172.21.15.39:0/2548425651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5541e69f0 msgr2=0x7fe55414fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:04.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.834+0000 7fe558a00640 1 --2- 172.21.15.39:0/2548425651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5541e69f0 0x7fe55414fef0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fe540003f20 tx=0x7fe5400016f0 comp rx=0 tx=0).stop 2024-09-21T07:42:04.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.834+0000 7fe558a00640 1 -- 172.21.15.39:0/2548425651 shutdown_connections 2024-09-21T07:42:04.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.834+0000 7fe558a00640 1 --2- 172.21.15.39:0/2548425651 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe534078b90 0x7fe53407b0a0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:04.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.834+0000 7fe558a00640 1 --2- 172.21.15.39:0/2548425651 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe554153d70 0x7fe5541f8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:04.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.834+0000 7fe558a00640 1 --2- 172.21.15.39:0/2548425651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5541e69f0 0x7fe55414fef0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:04.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.834+0000 7fe558a00640 1 --2- 172.21.15.39:0/2548425651 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe538004d40 0x7fe554151340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:04.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.834+0000 7fe558a00640 1 -- 172.21.15.39:0/2548425651 >> 172.21.15.39:0/2548425651 conn(0x7fe554074050 msgr2=0x7fe5541e7120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:04.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.834+0000 7fe558a00640 1 -- 172.21.15.39:0/2548425651 shutdown_connections 2024-09-21T07:42:04.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:04.834+0000 7fe558a00640 1 -- 172.21.15.39:0/2548425651 wait complete. 2024-09-21T07:42:04.857 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T07:42:05.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.083+0000 7fca45400640 1 -- 172.21.15.39:0/570363914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca400b47c0 msgr2=0x7fca40155800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:05.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.083+0000 7fca45400640 1 --2- 172.21.15.39:0/570363914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca400b47c0 0x7fca40155800 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fca30005fd0 tx=0x7fca3002a5d0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.083+0000 7fca45400640 1 -- 172.21.15.39:0/570363914 shutdown_connections 2024-09-21T07:42:05.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.083+0000 7fca45400640 1 --2- 172.21.15.39:0/570363914 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fca400d0db0 0x7fca400adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.083+0000 7fca45400640 1 --2- 172.21.15.39:0/570363914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca400b47c0 0x7fca40155800 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.083+0000 7fca45400640 1 -- 172.21.15.39:0/570363914 >> 172.21.15.39:0/570363914 conn(0x7fca4006cfc0 msgr2=0x7fca4006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:05.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.083+0000 7fca45400640 1 -- 172.21.15.39:0/570363914 shutdown_connections 2024-09-21T07:42:05.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.083+0000 7fca45400640 1 -- 172.21.15.39:0/570363914 wait complete. 2024-09-21T07:42:05.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.085+0000 7fca45400640 1 Processor -- start 2024-09-21T07:42:05.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.086+0000 7fca45400640 1 -- start start 2024-09-21T07:42:05.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.087+0000 7fca45400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fca4015f790 0x7fca40142680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:05.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.087+0000 7fca45400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca40144f10 0x7fca40142ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:05.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.087+0000 7fca45400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca40158a80 0x7fca40158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:05.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.087+0000 7fca45400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fca400ae6b0 con 0x7fca40158a80 2024-09-21T07:42:05.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.087+0000 7fca45400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fca400ae530 con 0x7fca40144f10 2024-09-21T07:42:05.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.087+0000 7fca45400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fca400ae830 con 0x7fca4015f790 2024-09-21T07:42:05.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.087+0000 7fca3f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca40158a80 0x7fca40158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:05.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.087+0000 7fca3f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca40158a80 0x7fca40158e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51812/0 (socket says 172.21.15.39:51812) 2024-09-21T07:42:05.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.087+0000 7fca3f200640 1 -- 172.21.15.39:0/2637890771 learned_addr learned my addr 172.21.15.39:0/2637890771 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:05.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.087+0000 7fca3e800640 1 --2- 172.21.15.39:0/2637890771 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fca4015f790 0x7fca40142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:05.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.087+0000 7fca35e00640 1 --2- 172.21.15.39:0/2637890771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca40144f10 0x7fca40142ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:05.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.088+0000 7fca3f200640 1 -- 172.21.15.39:0/2637890771 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fca4015f790 msgr2=0x7fca40142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:05.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.088+0000 7fca3f200640 1 --2- 172.21.15.39:0/2637890771 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fca4015f790 0x7fca40142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.088+0000 7fca3f200640 1 -- 172.21.15.39:0/2637890771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca40144f10 msgr2=0x7fca40142ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:05.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.088+0000 7fca3f200640 1 --2- 172.21.15.39:0/2637890771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca40144f10 0x7fca40142ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.088+0000 7fca3f200640 1 -- 172.21.15.39:0/2637890771 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca40142bc0 con 0x7fca40158a80 2024-09-21T07:42:05.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.088+0000 7fca3f200640 1 --2- 172.21.15.39:0/2637890771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca40158a80 0x7fca40158e90 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fca3800e9c0 tx=0x7fca3800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:05.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.088+0000 7fca37400640 1 -- 172.21.15.39:0/2637890771 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca38004d20 con 0x7fca40158a80 2024-09-21T07:42:05.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.088+0000 7fca37400640 1 -- 172.21.15.39:0/2637890771 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca3800cb20 con 0x7fca40158a80 2024-09-21T07:42:05.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.088+0000 7fca37400640 1 -- 172.21.15.39:0/2637890771 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca38005020 con 0x7fca40158a80 2024-09-21T07:42:05.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.089+0000 7fca45400640 1 -- 172.21.15.39:0/2637890771 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca40145a20 con 0x7fca40158a80 2024-09-21T07:42:05.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.089+0000 7fca45400640 1 -- 172.21.15.39:0/2637890771 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca40158280 con 0x7fca40158a80 2024-09-21T07:42:05.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.091+0000 7fca37400640 1 -- 172.21.15.39:0/2637890771 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fca380051c0 con 0x7fca40158a80 2024-09-21T07:42:05.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.092+0000 7fca37400640 1 --2- 172.21.15.39:0/2637890771 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fca1c078b90 0x7fca1c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:05.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.092+0000 7fca37400640 1 -- 172.21.15.39:0/2637890771 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7947+0+0 (secure 0 0 0) 0x7fca3809bf60 con 0x7fca40158a80 2024-09-21T07:42:05.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.092+0000 7fca3e800640 1 --2- 172.21.15.39:0/2637890771 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fca1c078b90 0x7fca1c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:05.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.092+0000 7fca4c4b7740 1 Processor -- start 2024-09-21T07:42:05.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.093+0000 7fca4c4b7740 1 -- start start 2024-09-21T07:42:05.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.093+0000 7fca3e800640 1 --2- 172.21.15.39:0/2637890771 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fca1c078b90 0x7fca1c07b0a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fca30033000 tx=0x7fca30004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:05.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.094+0000 7fca4c4b7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf8a750270 0x55cf8ab20970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:05.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.094+0000 7fca4c4b7740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cf8a7273b0 0x55cf8ab20eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:05.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.094+0000 7fca4c4b7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf8a72f590 0x55cf8aaf0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:05.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.094+0000 7fca4c4b7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55cf8aa7a430 con 0x55cf8a72f590 2024-09-21T07:42:05.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.094+0000 7fca4c4b7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55cf8aa6a5d0 con 0x55cf8a750270 2024-09-21T07:42:05.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.094+0000 7fca4c4b7740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55cf8aa59a00 con 0x55cf8a7273b0 2024-09-21T07:42:05.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.094+0000 7fca3f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf8a72f590 0x55cf8aaf0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:05.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.095+0000 7fca3f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf8a72f590 0x55cf8aaf0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51824/0 (socket says 172.21.15.39:51824) 2024-09-21T07:42:05.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.095+0000 7fca3f200640 1 -- 172.21.15.39:0/286995822 learned_addr learned my addr 172.21.15.39:0/286995822 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:05.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.095+0000 7fca3e800640 1 --2- 172.21.15.39:0/286995822 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf8a750270 0x55cf8ab20970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:05.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.095+0000 7fca35e00640 1 --2- 172.21.15.39:0/286995822 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cf8a7273b0 0x55cf8ab20eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:05.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.095+0000 7fca3f200640 1 -- 172.21.15.39:0/286995822 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cf8a7273b0 msgr2=0x55cf8ab20eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:42:05.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.095+0000 7fca3f200640 1 --2- 172.21.15.39:0/286995822 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cf8a7273b0 0x55cf8ab20eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.095+0000 7fca3f200640 1 -- 172.21.15.39:0/286995822 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf8a750270 msgr2=0x55cf8ab20970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:42:05.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.095+0000 7fca3f200640 1 --2- 172.21.15.39:0/286995822 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf8a750270 0x55cf8ab20970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.095+0000 7fca3f200640 1 -- 172.21.15.39:0/286995822 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf8aa22520 con 0x55cf8a72f590 2024-09-21T07:42:05.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.095+0000 7fca35e00640 1 --2- 172.21.15.39:0/286995822 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cf8a7273b0 0x55cf8ab20eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:42:05.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.095+0000 7fca3e800640 1 --2- 172.21.15.39:0/286995822 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf8a750270 0x55cf8ab20970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:42:05.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.095+0000 7fca3f200640 1 --2- 172.21.15.39:0/286995822 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf8a72f590 0x55cf8aaf0f60 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fca38063380 tx=0x7fca38062b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47561c952f85d1d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:05.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.095+0000 7fca2f000640 1 -- 172.21.15.39:0/286995822 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3801f030 con 0x55cf8a72f590 2024-09-21T07:42:05.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.095+0000 7fca2f000640 1 -- 172.21.15.39:0/286995822 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca38067e20 con 0x55cf8a72f590 2024-09-21T07:42:05.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.095+0000 7fca2f000640 1 -- 172.21.15.39:0/286995822 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca38085d40 con 0x55cf8a72f590 2024-09-21T07:42:05.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.096+0000 7fca4c4b7740 1 -- 172.21.15.39:0/286995822 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf8a72f590 msgr2=0x55cf8aaf0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:05.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.096+0000 7fca4c4b7740 1 --2- 172.21.15.39:0/286995822 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf8a72f590 0x55cf8aaf0f60 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fca38063380 tx=0x7fca38062b60 comp rx=0 tx=0).stop 2024-09-21T07:42:05.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.096+0000 7fca4c4b7740 1 -- 172.21.15.39:0/286995822 shutdown_connections 2024-09-21T07:42:05.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.096+0000 7fca4c4b7740 1 --2- 172.21.15.39:0/286995822 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf8a72f590 0x55cf8aaf0f60 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.096+0000 7fca4c4b7740 1 --2- 172.21.15.39:0/286995822 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cf8a7273b0 0x55cf8ab20eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.096+0000 7fca4c4b7740 1 --2- 172.21.15.39:0/286995822 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf8a750270 0x55cf8ab20970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.096+0000 7fca4c4b7740 1 -- 172.21.15.39:0/286995822 >> 172.21.15.39:0/286995822 conn(0x55cf8a7443c0 msgr2=0x55cf8aaf6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:05.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.096+0000 7fca4c4b7740 1 -- 172.21.15.39:0/286995822 shutdown_connections 2024-09-21T07:42:05.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.096+0000 7fca4c4b7740 1 -- 172.21.15.39:0/286995822 wait complete. 2024-09-21T07:42:05.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.097+0000 7fca4c4b7740 1 -- start start 2024-09-21T07:42:05.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.098+0000 7fca4c4b7740 1 Processor -- start 2024-09-21T07:42:05.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.099+0000 7fca4c4b7740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cf8a7273b0 0x55cf8ab35d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:05.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.099+0000 7fca4c4b7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf8a72f590 0x55cf8ab1d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:05.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.099+0000 7fca4c4b7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf8a750270 0x55cf8ab1eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:05.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.099+0000 7fca4c4b7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55cf8aa00330 con 0x55cf8a750270 2024-09-21T07:42:05.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.099+0000 7fca4c4b7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55cf8aa5d810 con 0x55cf8a72f590 2024-09-21T07:42:05.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.099+0000 7fca4c4b7740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55cf8a7ca620 con 0x55cf8a7273b0 2024-09-21T07:42:05.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.099+0000 7fca4c4b7740 10 client.0 _ll_register_callbacks cb 0x55cf8aa05420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:42:05.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.099+0000 7fca3f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf8a750270 0x55cf8ab1eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:05.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.100+0000 7fca3f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf8a750270 0x55cf8ab1eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51836/0 (socket says 172.21.15.39:51836) 2024-09-21T07:42:05.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.100+0000 7fca3f200640 1 -- 172.21.15.39:0/1509696954 learned_addr learned my addr 172.21.15.39:0/1509696954 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:05.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.100+0000 7fca3e800640 1 --2- 172.21.15.39:0/1509696954 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cf8a7273b0 0x55cf8ab35d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:05.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.100+0000 7fca35e00640 1 --2- 172.21.15.39:0/1509696954 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf8a72f590 0x55cf8ab1d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:05.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.100+0000 7fca3f200640 1 -- 172.21.15.39:0/1509696954 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cf8a7273b0 msgr2=0x55cf8ab35d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:42:05.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.100+0000 7fca3f200640 1 --2- 172.21.15.39:0/1509696954 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cf8a7273b0 0x55cf8ab35d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.100+0000 7fca3f200640 1 -- 172.21.15.39:0/1509696954 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf8a72f590 msgr2=0x55cf8ab1d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:42:05.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.100+0000 7fca3f200640 1 --2- 172.21.15.39:0/1509696954 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf8a72f590 0x55cf8ab1d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.100+0000 7fca3f200640 1 -- 172.21.15.39:0/1509696954 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf8a7bb850 con 0x55cf8a750270 2024-09-21T07:42:05.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.100+0000 7fca3f200640 1 --2- 172.21.15.39:0/1509696954 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf8a750270 0x55cf8ab1eca0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fca38060660 tx=0x7fca38062900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec7942fe685b6b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:05.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.100+0000 7fca23400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:42:05.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.100+0000 7fca23400640 1 -- 172.21.15.39:0/1509696954 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3805ef50 con 0x55cf8a750270 2024-09-21T07:42:05.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.100+0000 7fca23400640 1 -- 172.21.15.39:0/1509696954 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 7947+0+0 (secure 0 0 0) 0x7fca380620c0 con 0x55cf8a750270 2024-09-21T07:42:05.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.100+0000 7fca4c4b7740 1 -- 172.21.15.39:0/1509696954 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca38067e20 con 0x55cf8a750270 2024-09-21T07:42:05.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.101+0000 7fca23400640 1 -- 172.21.15.39:0/1509696954 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=134) ==== 24+0+0 (secure 0 0 0) 0x7fca380622e0 con 0x55cf8a750270 2024-09-21T07:42:05.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.101+0000 7fca4c4b7740 10 client.7722 fetch_fsmap learned FSMap version 134 2024-09-21T07:42:05.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.101+0000 7fca4c4b7740 1 -- 172.21.15.39:0/1509696954 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=134,monmap=2+}) -- 0x55cf8a8326b0 con 0x55cf8a750270 2024-09-21T07:42:05.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.101+0000 7fca23400640 1 -- 172.21.15.39:0/1509696954 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca38067e20 con 0x55cf8a750270 2024-09-21T07:42:05.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.101+0000 7fca23400640 1 -- 172.21.15.39:0/1509696954 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 134) ==== 3522+0+0 (secure 0 0 0) 0x7fca3800f040 con 0x55cf8a750270 2024-09-21T07:42:05.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.101+0000 7fca4c4b7740 10 client.7722 fetch_fsmap finished waiting for FSMap version 134 2024-09-21T07:42:05.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.101+0000 7fca4c4b7740 10 client.7722 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:42:05.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.101+0000 7fca4c4b7740 20 client.7722 populate_metadata read hostname 'smithi039' 2024-09-21T07:42:05.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.101+0000 7fca4c4b7740 1 --2- 172.21.15.39:0/1509696954 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x55cf8a70ee40 0x55cf8ab4e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:05.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.101+0000 7fca4c4b7740 4 client.7722 mds_command: new command op to 7311 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:42:05.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.101+0000 7fca4c4b7740 1 -- 172.21.15.39:0/1509696954 --> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf8aa6c520 con 0x55cf8a70ee40 2024-09-21T07:42:05.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.101+0000 7fca3e800640 1 --2- 172.21.15.39:0/1509696954 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x55cf8a70ee40 0x55cf8ab4e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:05.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.102+0000 7fca3e800640 1 --2- 172.21.15.39:0/1509696954 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x55cf8a70ee40 0x55cf8ab4e050 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bd74def791fdd01 server_cookie=832bbf394d81c391 in_seq=0 out_seq=0 2024-09-21T07:42:05.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.102+0000 7fca23400640 10 client.7722 ms_handle_connect on v2:172.21.15.39:6832/1790581246 2024-09-21T07:42:05.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.103+0000 7fca23400640 1 -- 172.21.15.39:0/1509696954 <== mds.0 v2:172.21.15.39:6832/1790581246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cf8aa6c520 con 0x55cf8a70ee40 2024-09-21T07:42:05.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.103+0000 7fca23400640 10 client.7722 handle_command_reply: tid=0 2024-09-21T07:42:05.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.103+0000 7fca4c4b7740 1 client.7722 shutdown 2024-09-21T07:42:05.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.103+0000 7fca4c4b7740 1 -- 172.21.15.39:0/1509696954 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf8a750270 msgr2=0x55cf8ab1eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:05.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.103+0000 7fca4c4b7740 1 --2- 172.21.15.39:0/1509696954 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf8a750270 0x55cf8ab1eca0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fca38060660 tx=0x7fca38062900 comp rx=0 tx=0).stop 2024-09-21T07:42:05.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.103+0000 7fca4c4b7740 1 -- 172.21.15.39:0/1509696954 shutdown_connections 2024-09-21T07:42:05.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.103+0000 7fca4c4b7740 1 --2- 172.21.15.39:0/1509696954 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf8a750270 0x55cf8ab1eca0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.103+0000 7fca4c4b7740 1 --2- 172.21.15.39:0/1509696954 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x55cf8a70ee40 0x55cf8ab4e050 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.103+0000 7fca4c4b7740 1 --2- 172.21.15.39:0/1509696954 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf8a72f590 0x55cf8ab1d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.103+0000 7fca4c4b7740 1 --2- 172.21.15.39:0/1509696954 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cf8a7273b0 0x55cf8ab35d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.103+0000 7fca23400640 0 client.7722 ms_handle_reset on v2:172.21.15.39:6832/1790581246 2024-09-21T07:42:05.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.103+0000 7fca4c4b7740 1 -- 172.21.15.39:0/1509696954 >> 172.21.15.39:0/1509696954 conn(0x55cf8a7443c0 msgr2=0x55cf8ab1fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:05.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.103+0000 7fca4c4b7740 1 -- 172.21.15.39:0/1509696954 shutdown_connections 2024-09-21T07:42:05.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.103+0000 7fca4c4b7740 1 -- 172.21.15.39:0/1509696954 wait complete. 2024-09-21T07:42:05.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.103+0000 7fca4c4b7740 20 client.7722 trim_cache size 0 max 16384 2024-09-21T07:42:05.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.123+0000 7fca4c4b7740 1 Processor -- start 2024-09-21T07:42:05.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.123+0000 7fca4c4b7740 1 -- start start 2024-09-21T07:42:05.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.124+0000 7fca4c4b7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf8a7273b0 0x55cf8ab1d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:05.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.124+0000 7fca4c4b7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf8a72f590 0x55cf8aaf67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:05.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.124+0000 7fca4c4b7740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cf8a750270 0x55cf8aaf6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:05.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.124+0000 7fca4c4b7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fca402053e0 con 0x55cf8a7273b0 2024-09-21T07:42:05.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.124+0000 7fca4c4b7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fca40203720 con 0x55cf8a72f590 2024-09-21T07:42:05.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.124+0000 7fca4c4b7740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fca40203d70 con 0x55cf8a750270 2024-09-21T07:42:05.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.124+0000 7fca3e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf8a7273b0 0x55cf8ab1d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:05.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.124+0000 7fca3e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf8a7273b0 0x55cf8ab1d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51852/0 (socket says 172.21.15.39:51852) 2024-09-21T07:42:05.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.124+0000 7fca3e800640 1 -- 172.21.15.39:0/3791502108 learned_addr learned my addr 172.21.15.39:0/3791502108 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:05.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.125+0000 7fca3f200640 1 --2- 172.21.15.39:0/3791502108 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cf8a750270 0x55cf8aaf6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:05.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.125+0000 7fca35e00640 1 --2- 172.21.15.39:0/3791502108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf8a72f590 0x55cf8aaf67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:05.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.125+0000 7fca3e800640 1 -- 172.21.15.39:0/3791502108 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cf8a750270 msgr2=0x55cf8aaf6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:42:05.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.125+0000 7fca3e800640 1 --2- 172.21.15.39:0/3791502108 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cf8a750270 0x55cf8aaf6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.125+0000 7fca3e800640 1 -- 172.21.15.39:0/3791502108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf8a72f590 msgr2=0x55cf8aaf67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:42:05.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.125+0000 7fca3e800640 1 --2- 172.21.15.39:0/3791502108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf8a72f590 0x55cf8aaf67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.125+0000 7fca3e800640 1 -- 172.21.15.39:0/3791502108 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf8ab3e830 con 0x55cf8a7273b0 2024-09-21T07:42:05.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.125+0000 7fca3e800640 1 --2- 172.21.15.39:0/3791502108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf8a7273b0 0x55cf8ab1d440 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fca30009850 tx=0x7fca3003d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae91952ef592506 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:05.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.125+0000 7fca2e600640 1 -- 172.21.15.39:0/3791502108 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca30035040 con 0x55cf8a7273b0 2024-09-21T07:42:05.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.125+0000 7fca2e600640 1 -- 172.21.15.39:0/3791502108 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca30056020 con 0x55cf8a7273b0 2024-09-21T07:42:05.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.125+0000 7fca2e600640 1 -- 172.21.15.39:0/3791502108 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca30003c70 con 0x55cf8a7273b0 2024-09-21T07:42:05.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.126+0000 7fca4c4b7740 1 -- 172.21.15.39:0/3791502108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf8a7273b0 msgr2=0x55cf8ab1d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:05.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.126+0000 7fca4c4b7740 1 --2- 172.21.15.39:0/3791502108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf8a7273b0 0x55cf8ab1d440 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fca30009850 tx=0x7fca3003d000 comp rx=0 tx=0).stop 2024-09-21T07:42:05.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.126+0000 7fca4c4b7740 1 -- 172.21.15.39:0/3791502108 shutdown_connections 2024-09-21T07:42:05.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.126+0000 7fca4c4b7740 1 --2- 172.21.15.39:0/3791502108 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cf8a750270 0x55cf8aaf6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.126+0000 7fca4c4b7740 1 --2- 172.21.15.39:0/3791502108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf8a72f590 0x55cf8aaf67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.126+0000 7fca4c4b7740 1 --2- 172.21.15.39:0/3791502108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf8a7273b0 0x55cf8ab1d440 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.126+0000 7fca4c4b7740 1 -- 172.21.15.39:0/3791502108 >> 172.21.15.39:0/3791502108 conn(0x55cf8a7443c0 msgr2=0x55cf8ab364d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:05.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.126+0000 7fca4c4b7740 1 -- 172.21.15.39:0/3791502108 shutdown_connections 2024-09-21T07:42:05.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.126+0000 7fca4c4b7740 1 -- 172.21.15.39:0/3791502108 wait complete. 2024-09-21T07:42:05.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.127+0000 7fca4c4b7740 1 -- start start 2024-09-21T07:42:05.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.127+0000 7fca4c4b7740 1 Processor -- start 2024-09-21T07:42:05.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.129+0000 7fca4c4b7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf8a7273b0 0x55cf8ab18bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:05.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.129+0000 7fca4c4b7740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cf8a72f590 0x55cf8aaf67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:05.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.129+0000 7fca4c4b7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf8a750270 0x55cf8aaf6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:05.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.129+0000 7fca4c4b7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55cf8ab1bec0 con 0x55cf8a750270 2024-09-21T07:42:05.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.129+0000 7fca4c4b7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55cf8aa00330 con 0x55cf8a7273b0 2024-09-21T07:42:05.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.129+0000 7fca4c4b7740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55cf8ab1c480 con 0x55cf8a72f590 2024-09-21T07:42:05.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.129+0000 7fca4c4b7740 10 client.0 _ll_register_callbacks cb 0x7fca38004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:42:05.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.129+0000 7fca3f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf8a750270 0x55cf8aaf6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:05.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.129+0000 7fca3f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf8a750270 0x55cf8aaf6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51862/0 (socket says 172.21.15.39:51862) 2024-09-21T07:42:05.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.129+0000 7fca3f200640 1 -- 172.21.15.39:0/1255006278 learned_addr learned my addr 172.21.15.39:0/1255006278 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:05.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.129+0000 7fca35e00640 1 --2- 172.21.15.39:0/1255006278 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cf8a72f590 0x55cf8aaf67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:05.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.129+0000 7fca3e800640 1 --2- 172.21.15.39:0/1255006278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf8a7273b0 0x55cf8ab18bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:05.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.130+0000 7fca3f200640 1 -- 172.21.15.39:0/1255006278 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cf8a72f590 msgr2=0x55cf8aaf67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:42:05.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.130+0000 7fca3f200640 1 --2- 172.21.15.39:0/1255006278 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cf8a72f590 0x55cf8aaf67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.130+0000 7fca3f200640 1 -- 172.21.15.39:0/1255006278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf8a7273b0 msgr2=0x55cf8ab18bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:42:05.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.130+0000 7fca3f200640 1 --2- 172.21.15.39:0/1255006278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf8a7273b0 0x55cf8ab18bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.130+0000 7fca3f200640 1 -- 172.21.15.39:0/1255006278 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf8ab1fde0 con 0x55cf8a750270 2024-09-21T07:42:05.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.130+0000 7fca3f200640 1 --2- 172.21.15.39:0/1255006278 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf8a750270 0x55cf8aaf6d30 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fca38004ff0 tx=0x7fca3805fb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18bad08fbeba54a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:05.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.130+0000 7fca23400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:42:05.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.130+0000 7fca23400640 1 -- 172.21.15.39:0/1255006278 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3801f040 con 0x55cf8a750270 2024-09-21T07:42:05.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.130+0000 7fca23400640 1 -- 172.21.15.39:0/1255006278 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 7947+0+0 (secure 0 0 0) 0x7fca3805ef50 con 0x55cf8a750270 2024-09-21T07:42:05.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.130+0000 7fca4c4b7740 1 -- 172.21.15.39:0/1255006278 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca30056020 con 0x55cf8a750270 2024-09-21T07:42:05.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.130+0000 7fca23400640 1 -- 172.21.15.39:0/1255006278 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=134) ==== 24+0+0 (secure 0 0 0) 0x7fca38062970 con 0x55cf8a750270 2024-09-21T07:42:05.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.130+0000 7fca4c4b7740 10 client.7728 fetch_fsmap learned FSMap version 134 2024-09-21T07:42:05.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.130+0000 7fca4c4b7740 1 -- 172.21.15.39:0/1255006278 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=134,monmap=2+}) -- 0x55cf8ab1fde0 con 0x55cf8a750270 2024-09-21T07:42:05.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.130+0000 7fca23400640 1 -- 172.21.15.39:0/1255006278 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca30056020 con 0x55cf8a750270 2024-09-21T07:42:05.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.131+0000 7fca23400640 1 -- 172.21.15.39:0/1255006278 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 134) ==== 3522+0+0 (secure 0 0 0) 0x7fca380054b0 con 0x55cf8a750270 2024-09-21T07:42:05.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.131+0000 7fca4c4b7740 10 client.7728 fetch_fsmap finished waiting for FSMap version 134 2024-09-21T07:42:05.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.131+0000 7fca4c4b7740 10 client.7728 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:42:05.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.131+0000 7fca4c4b7740 20 client.7728 populate_metadata read hostname 'smithi039' 2024-09-21T07:42:05.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.131+0000 7fca4c4b7740 1 --2- 172.21.15.39:0/1255006278 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x55cf8ab16700 0x55cf8ab171d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:05.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.131+0000 7fca4c4b7740 4 client.7728 mds_command: new command op to 7311 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:42:05.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.131+0000 7fca4c4b7740 1 -- 172.21.15.39:0/1255006278 --> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] -- command(tid 0: {"prefix": "status"}) -- 0x55cf8ab379b0 con 0x55cf8ab16700 2024-09-21T07:42:05.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.131+0000 7fca3e800640 1 --2- 172.21.15.39:0/1255006278 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x55cf8ab16700 0x55cf8ab171d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:05.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.132+0000 7fca23400640 10 client.7728 ms_handle_connect on v2:172.21.15.39:6832/1790581246 2024-09-21T07:42:05.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.132+0000 7fca3e800640 1 --2- 172.21.15.39:0/1255006278 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x55cf8ab16700 0x55cf8ab171d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f42ade2284399e1 server_cookie=9bee495e826efc4e in_seq=0 out_seq=0 2024-09-21T07:42:05.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.132+0000 7fca23400640 1 -- 172.21.15.39:0/1255006278 <== mds.0 v2:172.21.15.39:6832/1790581246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+329 (crc 0 0 0) 0x55cf8ab379b0 con 0x55cf8ab16700 2024-09-21T07:42:05.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.132+0000 7fca23400640 10 client.7728 handle_command_reply: tid=0 2024-09-21T07:42:05.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.132+0000 7fca4c4b7740 1 client.7728 shutdown 2024-09-21T07:42:05.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.132+0000 7fca4c4b7740 1 -- 172.21.15.39:0/1255006278 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf8a750270 msgr2=0x55cf8aaf6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:05.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.132+0000 7fca4c4b7740 1 --2- 172.21.15.39:0/1255006278 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf8a750270 0x55cf8aaf6d30 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fca38004ff0 tx=0x7fca3805fb70 comp rx=0 tx=0).stop 2024-09-21T07:42:05.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.132+0000 7fca4c4b7740 1 -- 172.21.15.39:0/1255006278 shutdown_connections 2024-09-21T07:42:05.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.132+0000 7fca4c4b7740 1 --2- 172.21.15.39:0/1255006278 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cf8a750270 0x55cf8aaf6d30 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.132+0000 7fca4c4b7740 1 --2- 172.21.15.39:0/1255006278 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cf8a72f590 0x55cf8aaf67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.132+0000 7fca4c4b7740 1 --2- 172.21.15.39:0/1255006278 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x55cf8ab16700 0x55cf8ab171d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.132+0000 7fca4c4b7740 1 --2- 172.21.15.39:0/1255006278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cf8a7273b0 0x55cf8ab18bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.132+0000 7fca4c4b7740 1 -- 172.21.15.39:0/1255006278 >> 172.21.15.39:0/1255006278 conn(0x55cf8a7443c0 msgr2=0x55cf8ab364d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:05.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.132+0000 7fca23400640 0 client.7728 ms_handle_reset on v2:172.21.15.39:6832/1790581246 2024-09-21T07:42:05.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.132+0000 7fca4c4b7740 1 -- 172.21.15.39:0/1255006278 shutdown_connections 2024-09-21T07:42:05.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.132+0000 7fca4c4b7740 1 -- 172.21.15.39:0/1255006278 wait complete. 2024-09-21T07:42:05.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.132+0000 7fca4c4b7740 20 client.7728 trim_cache size 0 max 16384 2024-09-21T07:42:05.140 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:42:05.140 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:42:05.140 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-09-21T07:42:05.140 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 7311, 2024-09-21T07:42:05.140 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:42:05.140 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:42:05.141 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:42:05.141 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 67.747749100999997, 2024-09-21T07:42:05.141 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 134, 2024-09-21T07:42:05.141 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 113, 2024-09-21T07:42:05.141 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 113, 2024-09-21T07:42:05.141 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 116.255197149 2024-09-21T07:42:05.141 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:42:05.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.133+0000 7fca45400640 1 -- 172.21.15.39:0/2637890771 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fca1c078b90 msgr2=0x7fca1c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:05.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.133+0000 7fca45400640 1 --2- 172.21.15.39:0/2637890771 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fca1c078b90 0x7fca1c07b0a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fca30033000 tx=0x7fca30004730 comp rx=0 tx=0).stop 2024-09-21T07:42:05.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.133+0000 7fca45400640 1 -- 172.21.15.39:0/2637890771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca40158a80 msgr2=0x7fca40158e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:05.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.133+0000 7fca45400640 1 --2- 172.21.15.39:0/2637890771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca40158a80 0x7fca40158e90 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fca3800e9c0 tx=0x7fca3800ee80 comp rx=0 tx=0).stop 2024-09-21T07:42:05.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.133+0000 7fca45400640 1 -- 172.21.15.39:0/2637890771 shutdown_connections 2024-09-21T07:42:05.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.133+0000 7fca45400640 1 --2- 172.21.15.39:0/2637890771 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fca1c078b90 0x7fca1c07b0a0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.133+0000 7fca45400640 1 --2- 172.21.15.39:0/2637890771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca40158a80 0x7fca40158e90 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.133+0000 7fca45400640 1 --2- 172.21.15.39:0/2637890771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca40144f10 0x7fca40142ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.133+0000 7fca45400640 1 --2- 172.21.15.39:0/2637890771 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fca4015f790 0x7fca40142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.133+0000 7fca45400640 1 -- 172.21.15.39:0/2637890771 >> 172.21.15.39:0/2637890771 conn(0x7fca4006cfc0 msgr2=0x7fca4015e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:05.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.133+0000 7fca45400640 1 -- 172.21.15.39:0/2637890771 shutdown_connections 2024-09-21T07:42:05.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.133+0000 7fca45400640 1 -- 172.21.15.39:0/2637890771 wait complete. 2024-09-21T07:42:05.157 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:42:05.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.381+0000 7f3497200640 1 -- 172.21.15.39:0/1033569216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f349015c520 msgr2=0x7f34900b3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:05.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.381+0000 7f3497200640 1 --2- 172.21.15.39:0/1033569216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f349015c520 0x7f34900b3ae0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f349006e300 tx=0x7f348402a5e0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.381+0000 7f3497200640 1 -- 172.21.15.39:0/1033569216 shutdown_connections 2024-09-21T07:42:05.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.381+0000 7f3497200640 1 --2- 172.21.15.39:0/1033569216 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3490160150 0x7f3490160560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.382+0000 7f3497200640 1 --2- 172.21.15.39:0/1033569216 >> v2:172.21.15.148:3300/0 conn(0x7f34900b4020 0x7f34900b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.382+0000 7f3497200640 1 --2- 172.21.15.39:0/1033569216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f349015c520 0x7f34900b3ae0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.382+0000 7f3497200640 1 -- 172.21.15.39:0/1033569216 >> 172.21.15.39:0/1033569216 conn(0x7f3490074050 msgr2=0x7f3490074490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:05.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.382+0000 7f3497200640 1 -- 172.21.15.39:0/1033569216 shutdown_connections 2024-09-21T07:42:05.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.382+0000 7f3497200640 1 -- 172.21.15.39:0/1033569216 wait complete. 2024-09-21T07:42:05.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.384+0000 7f3497200640 1 Processor -- start 2024-09-21T07:42:05.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.384+0000 7f3497200640 1 -- start start 2024-09-21T07:42:05.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.386+0000 7f3497200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34900b4020 0x7f34900acd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:05.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.386+0000 7f3497200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f349015c520 0x7f3490147610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:05.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.386+0000 7f3497200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3490160150 0x7f34901e7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:05.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.386+0000 7f3497200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3490160aa0 con 0x7f34900b4020 2024-09-21T07:42:05.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.386+0000 7f3497200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3490071e90 con 0x7f3490160150 2024-09-21T07:42:05.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.386+0000 7f3497200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f3490160c20 con 0x7f349015c520 2024-09-21T07:42:05.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.386+0000 7f3495e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34900b4020 0x7f34900acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:05.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.386+0000 7f3495e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34900b4020 0x7f34900acd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51876/0 (socket says 172.21.15.39:51876) 2024-09-21T07:42:05.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.386+0000 7f3495e00640 1 -- 172.21.15.39:0/1907123074 learned_addr learned my addr 172.21.15.39:0/1907123074 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:05.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.386+0000 7f3495400640 1 --2- 172.21.15.39:0/1907123074 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f349015c520 0x7f3490147610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:05.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.386+0000 7f3496800640 1 --2- 172.21.15.39:0/1907123074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3490160150 0x7f34901e7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:05.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.386+0000 7f3495e00640 1 -- 172.21.15.39:0/1907123074 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f349015c520 msgr2=0x7f3490147610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:05.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.386+0000 7f3495e00640 1 --2- 172.21.15.39:0/1907123074 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f349015c520 0x7f3490147610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.386+0000 7f3495e00640 1 -- 172.21.15.39:0/1907123074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3490160150 msgr2=0x7f34901e7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:05.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.386+0000 7f3495e00640 1 --2- 172.21.15.39:0/1907123074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3490160150 0x7f34901e7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.386+0000 7f3495e00640 1 -- 172.21.15.39:0/1907123074 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34901e6190 con 0x7f34900b4020 2024-09-21T07:42:05.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.386+0000 7f3495400640 1 --2- 172.21.15.39:0/1907123074 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f349015c520 0x7f3490147610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:42:05.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.387+0000 7f3495e00640 1 --2- 172.21.15.39:0/1907123074 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34900b4020 0x7f34900acd30 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f348402ab00 tx=0x7f34840047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:05.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.387+0000 7f348a000640 1 -- 172.21.15.39:0/1907123074 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348403e070 con 0x7f34900b4020 2024-09-21T07:42:05.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.387+0000 7f348a000640 1 -- 172.21.15.39:0/1907123074 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34840035f0 con 0x7f34900b4020 2024-09-21T07:42:05.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.387+0000 7f3497200640 1 -- 172.21.15.39:0/1907123074 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34901e6450 con 0x7f34900b4020 2024-09-21T07:42:05.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.387+0000 7f348a000640 1 -- 172.21.15.39:0/1907123074 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3484035040 con 0x7f34900b4020 2024-09-21T07:42:05.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.387+0000 7f3497200640 1 -- 172.21.15.39:0/1907123074 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34901e6a10 con 0x7f34900b4020 2024-09-21T07:42:05.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.388+0000 7f3497200640 1 -- 172.21.15.39:0/1907123074 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34900b3ae0 con 0x7f34900b4020 2024-09-21T07:42:05.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.389+0000 7f348a000640 1 -- 172.21.15.39:0/1907123074 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3484041070 con 0x7f34900b4020 2024-09-21T07:42:05.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.390+0000 7f348a000640 1 --2- 172.21.15.39:0/1907123074 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3474078b90 0x7f347407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:05.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.391+0000 7f348a000640 1 -- 172.21.15.39:0/1907123074 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7947+0+0 (secure 0 0 0) 0x7f34840bbef0 con 0x7f34900b4020 2024-09-21T07:42:05.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.391+0000 7f3495400640 1 --2- 172.21.15.39:0/1907123074 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3474078b90 0x7f347407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:05.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.392+0000 7f3495400640 1 --2- 172.21.15.39:0/1907123074 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3474078b90 0x7f347407b0a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3480005fd0 tx=0x7f3480005ea0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:05.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.395+0000 7f348a000640 1 -- 172.21.15.39:0/1907123074 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3484083af0 con 0x7f34900b4020 2024-09-21T07:42:05.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.628+0000 7f3497200640 1 -- 172.21.15.39:0/1907123074 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34900ccf30 con 0x7f34900b4020 2024-09-21T07:42:05.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.629+0000 7f348a000640 1 -- 172.21.15.39:0/1907123074 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 134 v134) ==== 95+0+5253 (secure 0 0 0) 0x7f34840833f0 con 0x7f34900b4020 2024-09-21T07:42:05.631 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:42:05.631 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":134,"btime":"2024-09-21T07:42:03:879182+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7644,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1326711417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1326711417},{"type":"v1","addr":"172.21.15.39:6835","nonce":1326711417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":131}],"filesystems":[{"mdsmap":{"epoch":134,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:42:03.879172+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":6425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6425":{"gid":6425,"name":"d","rank":1,"incarnation":130,"state":"up:active","state_seq":18,"addr":"172.21.15.148:6837/73220122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":73220122},{"type":"v1","addr":"172.21.15.148:6837","nonce":73220122}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[6425,7311]},"id":6}]} 2024-09-21T07:42:05.631 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 134 2024-09-21T07:42:05.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.631+0000 7f3497200640 1 -- 172.21.15.39:0/1907123074 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3474078b90 msgr2=0x7f347407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:05.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.631+0000 7f3497200640 1 --2- 172.21.15.39:0/1907123074 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3474078b90 0x7f347407b0a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3480005fd0 tx=0x7f3480005ea0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.631+0000 7f3497200640 1 -- 172.21.15.39:0/1907123074 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34900b4020 msgr2=0x7f34900acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:05.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.631+0000 7f3497200640 1 --2- 172.21.15.39:0/1907123074 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34900b4020 0x7f34900acd30 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f348402ab00 tx=0x7f34840047a0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.632+0000 7f3497200640 1 -- 172.21.15.39:0/1907123074 shutdown_connections 2024-09-21T07:42:05.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.632+0000 7f3497200640 1 --2- 172.21.15.39:0/1907123074 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3474078b90 0x7f347407b0a0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.632+0000 7f3497200640 1 --2- 172.21.15.39:0/1907123074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3490160150 0x7f34901e7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.632+0000 7f3497200640 1 --2- 172.21.15.39:0/1907123074 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f349015c520 0x7f3490147610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.632+0000 7f3497200640 1 --2- 172.21.15.39:0/1907123074 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34900b4020 0x7f34900acd30 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:05.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.632+0000 7f3497200640 1 -- 172.21.15.39:0/1907123074 >> 172.21.15.39:0/1907123074 conn(0x7f3490074050 msgr2=0x7f34900ce140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:05.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.632+0000 7f3497200640 1 -- 172.21.15.39:0/1907123074 shutdown_connections 2024-09-21T07:42:05.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:05.632+0000 7f3497200640 1 -- 172.21.15.39:0/1907123074 wait complete. 2024-09-21T07:42:05.659 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2024-09-21T07:42:05.902 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_snap":15,"last_created":12,"last_destroyed":14,"pending_noop":[],"snaps":[],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2024-09-21T07:42:05.902 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 12, "last_destroyed": 14, "last_snap": 15, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [] } 2024-09-21T07:42:05.902 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T07:42:05.902 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:42:06.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.127+0000 7f1bafe00640 1 -- 172.21.15.39:0/3118347792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1bb00c4880 msgr2=0x7f1bb0160bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:06.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.127+0000 7f1bafe00640 1 --2- 172.21.15.39:0/3118347792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1bb00c4880 0x7f1bb0160bf0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f1b98005fd0 tx=0x7f1b9802a5d0 comp rx=0 tx=0).stop 2024-09-21T07:42:06.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.127+0000 7f1bafe00640 1 -- 172.21.15.39:0/3118347792 shutdown_connections 2024-09-21T07:42:06.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.127+0000 7f1bafe00640 1 --2- 172.21.15.39:0/3118347792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1bb00c4880 0x7f1bb0160bf0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:06.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.127+0000 7f1bafe00640 1 --2- 172.21.15.39:0/3118347792 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1bb00cd770 0x7f1bb015c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:06.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.127+0000 7f1bafe00640 1 -- 172.21.15.39:0/3118347792 >> 172.21.15.39:0/3118347792 conn(0x7f1bb00b2d40 msgr2=0x7f1bb00b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:06.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.127+0000 7f1bafe00640 1 -- 172.21.15.39:0/3118347792 shutdown_connections 2024-09-21T07:42:06.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.128+0000 7f1bafe00640 1 -- 172.21.15.39:0/3118347792 wait complete. 2024-09-21T07:42:06.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.130+0000 7f1bafe00640 1 Processor -- start 2024-09-21T07:42:06.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.130+0000 7f1bafe00640 1 -- start start 2024-09-21T07:42:06.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.131+0000 7f1bafe00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1bb00c4880 0x7f1bb00afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:06.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.131+0000 7f1bafe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bb00cd770 0x7f1bb0151700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:06.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.131+0000 7f1bafe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1bb0161130 0x7f1bb01f8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:06.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.131+0000 7f1bafe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb00b4040 con 0x7f1bb0161130 2024-09-21T07:42:06.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.131+0000 7f1bafe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb006d700 con 0x7f1bb00cd770 2024-09-21T07:42:06.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.131+0000 7f1bafe00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f1bb00b41c0 con 0x7f1bb00c4880 2024-09-21T07:42:06.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.132+0000 7f1baf400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1bb0161130 0x7f1bb01f8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:06.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.132+0000 7f1baf400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1bb0161130 0x7f1bb01f8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51884/0 (socket says 172.21.15.39:51884) 2024-09-21T07:42:06.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.132+0000 7f1baf400640 1 -- 172.21.15.39:0/2293317487 learned_addr learned my addr 172.21.15.39:0/2293317487 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:06.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.132+0000 7f1baea00640 1 --2- 172.21.15.39:0/2293317487 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1bb00c4880 0x7f1bb00afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:06.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.132+0000 7f1ba7e00640 1 --2- 172.21.15.39:0/2293317487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bb00cd770 0x7f1bb0151700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:06.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.132+0000 7f1baf400640 1 -- 172.21.15.39:0/2293317487 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1bb00c4880 msgr2=0x7f1bb00afe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:06.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.132+0000 7f1baf400640 1 --2- 172.21.15.39:0/2293317487 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1bb00c4880 0x7f1bb00afe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:06.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.132+0000 7f1baf400640 1 -- 172.21.15.39:0/2293317487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bb00cd770 msgr2=0x7f1bb0151700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:06.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.132+0000 7f1baf400640 1 --2- 172.21.15.39:0/2293317487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bb00cd770 0x7f1bb0151700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:06.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.132+0000 7f1baf400640 1 -- 172.21.15.39:0/2293317487 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bb00cee60 con 0x7f1bb0161130 2024-09-21T07:42:06.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.132+0000 7f1baf400640 1 --2- 172.21.15.39:0/2293317487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1bb0161130 0x7f1bb01f8c80 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f1ba800c730 tx=0x7f1ba800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:06.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.133+0000 7f1ba7400640 1 -- 172.21.15.39:0/2293317487 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba8003b70 con 0x7f1bb0161130 2024-09-21T07:42:06.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.133+0000 7f1ba7400640 1 -- 172.21.15.39:0/2293317487 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ba8003d30 con 0x7f1bb0161130 2024-09-21T07:42:06.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.133+0000 7f1ba7400640 1 -- 172.21.15.39:0/2293317487 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba8005330 con 0x7f1bb0161130 2024-09-21T07:42:06.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.133+0000 7f1bafe00640 1 -- 172.21.15.39:0/2293317487 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bb00cf140 con 0x7f1bb0161130 2024-09-21T07:42:06.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.133+0000 7f1bafe00640 1 -- 172.21.15.39:0/2293317487 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bb00cf700 con 0x7f1bb0161130 2024-09-21T07:42:06.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.134+0000 7f1bafe00640 1 -- 172.21.15.39:0/2293317487 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bb00b0580 con 0x7f1bb0161130 2024-09-21T07:42:06.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.135+0000 7f1ba7400640 1 -- 172.21.15.39:0/2293317487 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1ba80056e0 con 0x7f1bb0161130 2024-09-21T07:42:06.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.136+0000 7f1ba7400640 1 --2- 172.21.15.39:0/2293317487 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1b8c078b90 0x7f1b8c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:06.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.136+0000 7f1ba7400640 1 -- 172.21.15.39:0/2293317487 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7947+0+0 (secure 0 0 0) 0x7f1ba809b640 con 0x7f1bb0161130 2024-09-21T07:42:06.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.136+0000 7f1baea00640 1 --2- 172.21.15.39:0/2293317487 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1b8c078b90 0x7f1b8c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:06.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.137+0000 7f1baea00640 1 --2- 172.21.15.39:0/2293317487 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1b8c078b90 0x7f1b8c07b0a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0009ac0 tx=0x7f1ba0005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:06.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.140+0000 7f1ba7400640 1 -- 172.21.15.39:0/2293317487 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1ba8063120 con 0x7f1bb0161130 2024-09-21T07:42:06.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.374+0000 7f1bafe00640 1 -- 172.21.15.39:0/2293317487 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1bb00b4680 con 0x7f1bb0161130 2024-09-21T07:42:06.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.374+0000 7f1ba7400640 1 -- 172.21.15.39:0/2293317487 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 134 v134) ==== 95+0+5253 (secure 0 0 0) 0x7f1ba8062a20 con 0x7f1bb0161130 2024-09-21T07:42:06.376 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:42:06.377 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":134,"btime":"2024-09-21T07:42:03:879182+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7644,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1326711417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1326711417},{"type":"v1","addr":"172.21.15.39:6835","nonce":1326711417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":131}],"filesystems":[{"mdsmap":{"epoch":134,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:42:03.879172+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":6425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6425":{"gid":6425,"name":"d","rank":1,"incarnation":130,"state":"up:active","state_seq":18,"addr":"172.21.15.148:6837/73220122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":73220122},{"type":"v1","addr":"172.21.15.148:6837","nonce":73220122}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[6425,7311]},"id":6}]} 2024-09-21T07:42:06.377 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 134 2024-09-21T07:42:06.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.377+0000 7f1bafe00640 1 -- 172.21.15.39:0/2293317487 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1b8c078b90 msgr2=0x7f1b8c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:06.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.377+0000 7f1bafe00640 1 --2- 172.21.15.39:0/2293317487 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1b8c078b90 0x7f1b8c07b0a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0009ac0 tx=0x7f1ba0005f90 comp rx=0 tx=0).stop 2024-09-21T07:42:06.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.377+0000 7f1bafe00640 1 -- 172.21.15.39:0/2293317487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1bb0161130 msgr2=0x7f1bb01f8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:06.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.377+0000 7f1bafe00640 1 --2- 172.21.15.39:0/2293317487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1bb0161130 0x7f1bb01f8c80 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f1ba800c730 tx=0x7f1ba800cbf0 comp rx=0 tx=0).stop 2024-09-21T07:42:06.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.377+0000 7f1bafe00640 1 -- 172.21.15.39:0/2293317487 shutdown_connections 2024-09-21T07:42:06.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.377+0000 7f1bafe00640 1 --2- 172.21.15.39:0/2293317487 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1b8c078b90 0x7f1b8c07b0a0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:06.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.377+0000 7f1bafe00640 1 --2- 172.21.15.39:0/2293317487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1bb0161130 0x7f1bb01f8c80 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:06.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.377+0000 7f1bafe00640 1 --2- 172.21.15.39:0/2293317487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bb00cd770 0x7f1bb0151700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:06.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.377+0000 7f1bafe00640 1 --2- 172.21.15.39:0/2293317487 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1bb00c4880 0x7f1bb00afe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:06.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.377+0000 7f1bafe00640 1 -- 172.21.15.39:0/2293317487 >> 172.21.15.39:0/2293317487 conn(0x7f1bb00b2d40 msgr2=0x7f1bb00b5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:06.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.377+0000 7f1bafe00640 1 -- 172.21.15.39:0/2293317487 shutdown_connections 2024-09-21T07:42:06.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:06.377+0000 7f1bafe00640 1 -- 172.21.15.39:0/2293317487 wait complete. 2024-09-21T07:42:06.404 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2024-09-21T07:42:06.646 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_snap":15,"last_created":12,"last_destroyed":14,"pending_noop":[],"snaps":[],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2024-09-21T07:42:06.646 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 12, "last_destroyed": 14, "last_snap": 15, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [] } 2024-09-21T07:42:06.647 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:42:06.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:42:06.647 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-09-21T07:42:16.090 INFO:teuthology.orchestra.run.smithi039.stderr:mkdir: cannot create directory ‘d1/dir/.snap/s33’: Permission denied 2024-09-21T07:42:16.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:42:16.091 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:42:16.091 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:42:16.129 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:42:16.129 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:42:16.208 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:42:16.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.477+0000 7f277ec00640 1 -- 172.21.15.39:0/2124663383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27780d0620 msgr2=0x7f27780d0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:16.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.477+0000 7f277ec00640 1 --2- 172.21.15.39:0/2124663383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27780d0620 0x7f27780d0a30 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f2770005fd0 tx=0x7f277002a8b0 comp rx=0 tx=0).stop 2024-09-21T07:42:16.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.477+0000 7f277ec00640 1 -- 172.21.15.39:0/2124663383 shutdown_connections 2024-09-21T07:42:16.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.477+0000 7f277ec00640 1 --2- 172.21.15.39:0/2124663383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27780d0620 0x7f27780d0a30 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:16.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.477+0000 7f277ec00640 1 -- 172.21.15.39:0/2124663383 >> 172.21.15.39:0/2124663383 conn(0x7f277806cfc0 msgr2=0x7f277806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:16.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.477+0000 7f277ec00640 1 -- 172.21.15.39:0/2124663383 shutdown_connections 2024-09-21T07:42:16.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.477+0000 7f277ec00640 1 -- 172.21.15.39:0/2124663383 wait complete. 2024-09-21T07:42:16.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.479+0000 7f277ec00640 1 Processor -- start 2024-09-21T07:42:16.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.480+0000 7f277ec00640 1 -- start start 2024-09-21T07:42:16.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.481+0000 7f277ec00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27780cfd10 0x7f2778151590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:16.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.481+0000 7f277ec00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27780d0620 0x7f27780ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:16.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.481+0000 7f277ec00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f277815cd30 0x7f27780aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:16.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.481+0000 7f277ec00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2778073aa0 con 0x7f277815cd30 2024-09-21T07:42:16.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.481+0000 7f277ec00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2778073920 con 0x7f27780d0620 2024-09-21T07:42:16.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.481+0000 7f277ec00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f2778073c20 con 0x7f27780cfd10 2024-09-21T07:42:16.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.481+0000 7f277cc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f277815cd30 0x7f27780aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:16.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.482+0000 7f277cc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f277815cd30 0x7f27780aedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53622/0 (socket says 172.21.15.39:53622) 2024-09-21T07:42:16.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.482+0000 7f277cc00640 1 -- 172.21.15.39:0/3294395298 learned_addr learned my addr 172.21.15.39:0/3294395298 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:16.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.482+0000 7f2777e00640 1 --2- 172.21.15.39:0/3294395298 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27780cfd10 0x7f2778151590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:16.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.482+0000 7f276f400640 1 --2- 172.21.15.39:0/3294395298 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27780d0620 0x7f27780ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:16.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.482+0000 7f277cc00640 1 -- 172.21.15.39:0/3294395298 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27780cfd10 msgr2=0x7f2778151590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:16.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.482+0000 7f277cc00640 1 --2- 172.21.15.39:0/3294395298 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27780cfd10 0x7f2778151590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:16.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.482+0000 7f277cc00640 1 -- 172.21.15.39:0/3294395298 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27780d0620 msgr2=0x7f27780ae890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:16.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.482+0000 7f277cc00640 1 --2- 172.21.15.39:0/3294395298 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27780d0620 0x7f27780ae890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:16.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.482+0000 7f277cc00640 1 -- 172.21.15.39:0/3294395298 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27781e8750 con 0x7f277815cd30 2024-09-21T07:42:16.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.482+0000 7f277cc00640 1 --2- 172.21.15.39:0/3294395298 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f277815cd30 0x7f27780aedd0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f2770009cd0 tx=0x7f2770003a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:16.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.483+0000 7f2774c00640 1 -- 172.21.15.39:0/3294395298 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2770031b90 con 0x7f277815cd30 2024-09-21T07:42:16.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.483+0000 7f2774c00640 1 -- 172.21.15.39:0/3294395298 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2770031e20 con 0x7f277815cd30 2024-09-21T07:42:16.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.483+0000 7f2774c00640 1 -- 172.21.15.39:0/3294395298 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27700332f0 con 0x7f277815cd30 2024-09-21T07:42:16.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.483+0000 7f277ec00640 1 -- 172.21.15.39:0/3294395298 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27781f5640 con 0x7f277815cd30 2024-09-21T07:42:16.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.483+0000 7f277ec00640 1 -- 172.21.15.39:0/3294395298 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27781f5c00 con 0x7f277815cd30 2024-09-21T07:42:16.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.484+0000 7f277ec00640 1 -- 172.21.15.39:0/3294395298 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27780d0a30 con 0x7f277815cd30 2024-09-21T07:42:16.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.485+0000 7f2774c00640 1 -- 172.21.15.39:0/3294395298 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2770033490 con 0x7f277815cd30 2024-09-21T07:42:16.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.486+0000 7f2774c00640 1 --2- 172.21.15.39:0/3294395298 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2754078b90 0x7f275407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:16.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.486+0000 7f2774c00640 1 -- 172.21.15.39:0/3294395298 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7947+0+0 (secure 0 0 0) 0x7f27700bcb30 con 0x7f277815cd30 2024-09-21T07:42:16.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.487+0000 7f2777e00640 1 --2- 172.21.15.39:0/3294395298 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2754078b90 0x7f275407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:16.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.487+0000 7f2777e00640 1 --2- 172.21.15.39:0/3294395298 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2754078b90 0x7f275407b0a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2768000f60 tx=0x7f2768009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:16.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.491+0000 7f2774c00640 1 -- 172.21.15.39:0/3294395298 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f277003f030 con 0x7f277815cd30 2024-09-21T07:42:16.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.700+0000 7f277ec00640 1 -- 172.21.15.39:0/3294395298 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2778151ad0 con 0x7f277815cd30 2024-09-21T07:42:16.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.700+0000 7f2774c00640 1 -- 172.21.15.39:0/3294395298 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 26 entries v113) ==== 81+0+1568 (secure 0 0 0) 0x7f2770084050 con 0x7f277815cd30 2024-09-21T07:42:16.702 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/2336112936 2024-09-22T07:41:05.979253+0000 2024-09-21T07:42:16.702 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/2821446711 2024-09-22T07:40:57.372409+0000 2024-09-21T07:42:16.702 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/1456343023 2024-09-22T07:40:08.637993+0000 2024-09-21T07:42:16.703 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6833/1456343023 2024-09-22T07:40:08.637993+0000 2024-09-21T07:42:16.703 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/992188984 2024-09-22T07:40:00.778049+0000 2024-09-21T07:42:16.703 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/3356393880 2024-09-21T08:40:15.824973+0000 2024-09-21T07:42:16.703 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/992188984 2024-09-22T07:40:00.778049+0000 2024-09-21T07:42:16.703 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/1472430787 2024-09-22T07:37:58.212892+0000 2024-09-21T07:42:16.703 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/3695489158 2024-09-22T07:41:51.807018+0000 2024-09-21T07:42:16.703 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/829716143 2024-09-22T07:37:01.259219+0000 2024-09-21T07:42:16.703 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/3026827614 2024-09-22T07:37:29.781472+0000 2024-09-21T07:42:16.703 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/3026827614 2024-09-22T07:37:29.781472+0000 2024-09-21T07:42:16.703 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/1472430787 2024-09-22T07:37:58.212892+0000 2024-09-21T07:42:16.704 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.1:0/90671776 2024-09-21T08:39:22.988045+0000 2024-09-21T07:42:16.704 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/3695489158 2024-09-22T07:41:51.807018+0000 2024-09-21T07:42:16.704 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/2336112936 2024-09-22T07:41:05.979253+0000 2024-09-21T07:42:16.704 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/2821446711 2024-09-22T07:40:57.372409+0000 2024-09-21T07:42:16.704 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/2381504293 2024-09-22T07:38:32.207414+0000 2024-09-21T07:42:16.704 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/742594435 2024-09-21T08:42:01.225749+0000 2024-09-21T07:42:16.704 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/2381504293 2024-09-22T07:38:32.207414+0000 2024-09-21T07:42:16.704 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/855150775 2024-09-22T07:39:07.739045+0000 2024-09-21T07:42:16.704 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/829716143 2024-09-22T07:37:01.259219+0000 2024-09-21T07:42:16.704 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6833/364657398 2024-09-22T07:39:15.579613+0000 2024-09-21T07:42:16.704 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/855150775 2024-09-22T07:39:07.739045+0000 2024-09-21T07:42:16.705 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.1:0/177249410 2024-09-21T08:41:13.882767+0000 2024-09-21T07:42:16.705 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/364657398 2024-09-22T07:39:15.579613+0000 2024-09-21T07:42:16.706 INFO:teuthology.orchestra.run.smithi039.stderr:listed 26 entries 2024-09-21T07:42:16.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.703+0000 7f277ec00640 1 -- 172.21.15.39:0/3294395298 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2754078b90 msgr2=0x7f275407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:16.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.703+0000 7f277ec00640 1 --2- 172.21.15.39:0/3294395298 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2754078b90 0x7f275407b0a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2768000f60 tx=0x7f2768009040 comp rx=0 tx=0).stop 2024-09-21T07:42:16.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.703+0000 7f277ec00640 1 -- 172.21.15.39:0/3294395298 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f277815cd30 msgr2=0x7f27780aedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:16.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.703+0000 7f277ec00640 1 --2- 172.21.15.39:0/3294395298 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f277815cd30 0x7f27780aedd0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f2770009cd0 tx=0x7f2770003a10 comp rx=0 tx=0).stop 2024-09-21T07:42:16.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.703+0000 7f277ec00640 1 -- 172.21.15.39:0/3294395298 shutdown_connections 2024-09-21T07:42:16.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.703+0000 7f277ec00640 1 --2- 172.21.15.39:0/3294395298 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2754078b90 0x7f275407b0a0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:16.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.703+0000 7f277ec00640 1 --2- 172.21.15.39:0/3294395298 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f277815cd30 0x7f27780aedd0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:16.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.703+0000 7f277ec00640 1 --2- 172.21.15.39:0/3294395298 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27780d0620 0x7f27780ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:16.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.703+0000 7f277ec00640 1 --2- 172.21.15.39:0/3294395298 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27780cfd10 0x7f2778151590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:16.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.703+0000 7f277ec00640 1 -- 172.21.15.39:0/3294395298 >> 172.21.15.39:0/3294395298 conn(0x7f277806cfc0 msgr2=0x7f277815b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:16.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.703+0000 7f277ec00640 1 -- 172.21.15.39:0/3294395298 shutdown_connections 2024-09-21T07:42:16.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:16.703+0000 7f277ec00640 1 -- 172.21.15.39:0/3294395298 wait complete. 2024-09-21T07:42:16.730 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T07:42:16.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:42:16.730 DEBUG:teuthology.orchestra.run.smithi039:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T07:42:16.789 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T07:42:16.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:42:16.789 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:42:16.825 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:42:16.825 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:42:16.901 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:42:16.901 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:42:17.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.165+0000 7fc3e6e00640 1 -- 172.21.15.39:0/1499606139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3e00d0f20 msgr2=0x7fc3e015c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:17.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.165+0000 7fc3e6e00640 1 --2- 172.21.15.39:0/1499606139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3e00d0f20 0x7fc3e015c830 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4005fd0 tx=0x7fc3d402a5d0 comp rx=0 tx=0).stop 2024-09-21T07:42:17.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.165+0000 7fc3e6e00640 1 -- 172.21.15.39:0/1499606139 shutdown_connections 2024-09-21T07:42:17.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.165+0000 7fc3e6e00640 1 --2- 172.21.15.39:0/1499606139 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc3e00cb960 0x7fc3e00cbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:17.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.165+0000 7fc3e6e00640 1 --2- 172.21.15.39:0/1499606139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3e00d0f20 0x7fc3e015c830 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:17.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.165+0000 7fc3e6e00640 1 -- 172.21.15.39:0/1499606139 >> 172.21.15.39:0/1499606139 conn(0x7fc3e00b2510 msgr2=0x7fc3e00d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:17.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.165+0000 7fc3e6e00640 1 -- 172.21.15.39:0/1499606139 shutdown_connections 2024-09-21T07:42:17.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.165+0000 7fc3e6e00640 1 -- 172.21.15.39:0/1499606139 wait complete. 2024-09-21T07:42:17.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.167+0000 7fc3e6e00640 1 Processor -- start 2024-09-21T07:42:17.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.167+0000 7fc3e6e00640 1 -- start start 2024-09-21T07:42:17.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.169+0000 7fc3e6e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3e006f8a0 0x7fc3e009da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:17.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.169+0000 7fc3e6e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3e00cb960 0x7fc3e0158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:17.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.169+0000 7fc3e6e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc3e00d0f20 0x7fc3e01f7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:17.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.169+0000 7fc3e6e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e00b4570 con 0x7fc3e00cb960 2024-09-21T07:42:17.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.169+0000 7fc3e6e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e00b43f0 con 0x7fc3e006f8a0 2024-09-21T07:42:17.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.169+0000 7fc3e6e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc3e00b46f0 con 0x7fc3e00d0f20 2024-09-21T07:42:17.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.169+0000 7fc3e5000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3e00cb960 0x7fc3e0158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:17.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.169+0000 7fc3e5000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3e00cb960 0x7fc3e0158c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53646/0 (socket says 172.21.15.39:53646) 2024-09-21T07:42:17.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.169+0000 7fc3e5000640 1 -- 172.21.15.39:0/4027809 learned_addr learned my addr 172.21.15.39:0/4027809 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:17.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.169+0000 7fc3e5a00640 1 --2- 172.21.15.39:0/4027809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3e006f8a0 0x7fc3e009da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:17.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.169+0000 7fc3e6400640 1 --2- 172.21.15.39:0/4027809 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc3e00d0f20 0x7fc3e01f7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:17.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.169+0000 7fc3e5000640 1 -- 172.21.15.39:0/4027809 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc3e00d0f20 msgr2=0x7fc3e01f7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:17.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.169+0000 7fc3e5000640 1 --2- 172.21.15.39:0/4027809 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc3e00d0f20 0x7fc3e01f7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:17.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.169+0000 7fc3e5000640 1 -- 172.21.15.39:0/4027809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3e006f8a0 msgr2=0x7fc3e009da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:17.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.169+0000 7fc3e5000640 1 --2- 172.21.15.39:0/4027809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3e006f8a0 0x7fc3e009da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:17.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.169+0000 7fc3e5000640 1 -- 172.21.15.39:0/4027809 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e00ce400 con 0x7fc3e00cb960 2024-09-21T07:42:17.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.170+0000 7fc3e5000640 1 --2- 172.21.15.39:0/4027809 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3e00cb960 0x7fc3e0158c40 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc00d6d0 tx=0x7fc3cc00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:17.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.170+0000 7fc3dd600640 1 -- 172.21.15.39:0/4027809 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc017070 con 0x7fc3e00cb960 2024-09-21T07:42:17.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.170+0000 7fc3dd600640 1 -- 172.21.15.39:0/4027809 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc3cc005330 con 0x7fc3e00cb960 2024-09-21T07:42:17.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.170+0000 7fc3e6e00640 1 -- 172.21.15.39:0/4027809 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3e00ce6e0 con 0x7fc3e00cb960 2024-09-21T07:42:17.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.170+0000 7fc3dd600640 1 -- 172.21.15.39:0/4027809 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc009820 con 0x7fc3e00cb960 2024-09-21T07:42:17.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.170+0000 7fc3e6e00640 1 -- 172.21.15.39:0/4027809 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3e00ceca0 con 0x7fc3e00cb960 2024-09-21T07:42:17.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.171+0000 7fc3e6e00640 1 -- 172.21.15.39:0/4027809 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3e00b4c50 con 0x7fc3e00cb960 2024-09-21T07:42:17.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.172+0000 7fc3dd600640 1 -- 172.21.15.39:0/4027809 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc3cc009bd0 con 0x7fc3e00cb960 2024-09-21T07:42:17.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.173+0000 7fc3dd600640 1 --2- 172.21.15.39:0/4027809 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc3c0078bc0 0x7fc3c007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:17.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.174+0000 7fc3dd600640 1 -- 172.21.15.39:0/4027809 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7947+0+0 (secure 0 0 0) 0x7fc3cc09b8f0 con 0x7fc3e00cb960 2024-09-21T07:42:17.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.174+0000 7fc3e5a00640 1 --2- 172.21.15.39:0/4027809 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc3c0078bc0 0x7fc3c007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:17.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.175+0000 7fc3e5a00640 1 --2- 172.21.15.39:0/4027809 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc3c0078bc0 0x7fc3c007b0d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4033000 tx=0x7fc3d4004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:17.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.178+0000 7fc3dd600640 1 -- 172.21.15.39:0/4027809 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc3cc062d80 con 0x7fc3e00cb960 2024-09-21T07:42:17.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.410+0000 7fc3e6e00640 1 -- 172.21.15.39:0/4027809 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc3e00ce190 con 0x7fc3e00cb960 2024-09-21T07:42:17.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.411+0000 7fc3dd600640 1 -- 172.21.15.39:0/4027809 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v134) ==== 53+0+73 (secure 0 0 0) 0x7fc3cc0677b0 con 0x7fc3e00cb960 2024-09-21T07:42:17.413 INFO:teuthology.orchestra.run.smithi039.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:42:17.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.413+0000 7fc3e6e00640 1 -- 172.21.15.39:0/4027809 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc3c0078bc0 msgr2=0x7fc3c007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:17.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.413+0000 7fc3e6e00640 1 --2- 172.21.15.39:0/4027809 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc3c0078bc0 0x7fc3c007b0d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4033000 tx=0x7fc3d4004730 comp rx=0 tx=0).stop 2024-09-21T07:42:17.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.413+0000 7fc3e6e00640 1 -- 172.21.15.39:0/4027809 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3e00cb960 msgr2=0x7fc3e0158c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:17.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.413+0000 7fc3e6e00640 1 --2- 172.21.15.39:0/4027809 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3e00cb960 0x7fc3e0158c40 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc00d6d0 tx=0x7fc3cc00db90 comp rx=0 tx=0).stop 2024-09-21T07:42:17.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.413+0000 7fc3e6e00640 1 -- 172.21.15.39:0/4027809 shutdown_connections 2024-09-21T07:42:17.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.413+0000 7fc3e6e00640 1 --2- 172.21.15.39:0/4027809 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc3c0078bc0 0x7fc3c007b0d0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:17.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.413+0000 7fc3e6e00640 1 --2- 172.21.15.39:0/4027809 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc3e00d0f20 0x7fc3e01f7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:17.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.413+0000 7fc3e6e00640 1 --2- 172.21.15.39:0/4027809 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3e00cb960 0x7fc3e0158c40 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:17.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.413+0000 7fc3e6e00640 1 --2- 172.21.15.39:0/4027809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3e006f8a0 0x7fc3e009da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:17.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.413+0000 7fc3e6e00640 1 -- 172.21.15.39:0/4027809 >> 172.21.15.39:0/4027809 conn(0x7fc3e00b2510 msgr2=0x7fc3e015ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:17.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.413+0000 7fc3e6e00640 1 -- 172.21.15.39:0/4027809 shutdown_connections 2024-09-21T07:42:17.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17.413+0000 7fc3e6e00640 1 -- 172.21.15.39:0/4027809 wait complete. 2024-09-21T07:42:17.437 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:42:17.437 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:42:17.437 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi039.front.sepia.ceph.com 2024-09-21T07:42:17.437 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T07:42:17.437 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:42:17.437 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:42:17.437 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:42:17.437 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:42:17.437 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:42:17.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:42:17.438 DEBUG:teuthology.orchestra.run.smithi039:> ip addr 2024-09-21T07:42:17.473 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:42:17.473 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:42:17.473 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:42:17.474 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:42:17.474 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2024-09-21T07:42:17.474 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:42:17.474 INFO:teuthology.orchestra.run.smithi039.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:42:17.474 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:42:17.474 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f0 2024-09-21T07:42:17.474 INFO:teuthology.orchestra.run.smithi039.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:42:17.474 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c7 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:42:17.474 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f1 2024-09-21T07:42:17.474 INFO:teuthology.orchestra.run.smithi039.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:42:17.474 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:70 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:42:17.474 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f0 2024-09-21T07:42:17.474 INFO:teuthology.orchestra.run.smithi039.stdout: inet 172.21.15.39/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T07:42:17.474 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft 42182sec preferred_lft 42182sec 2024-09-21T07:42:17.474 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec4:7aff:febd:1370/64 scope link 2024-09-21T07:42:17.475 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:42:17.475 INFO:teuthology.orchestra.run.smithi039.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:42:17.475 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:71 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:42:17.475 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f1 2024-09-21T07:42:17.475 INFO:teuthology.orchestra.run.smithi039.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:42:17.475 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff 2024-09-21T07:42:17.475 INFO:teuthology.orchestra.run.smithi039.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:42:17.475 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:42:17.475 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::b079:ddff:fe0e:6333/64 scope link 2024-09-21T07:42:17.475 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:42:17.475 INFO:teuthology.orchestra.run.smithi039.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:42:17.475 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:42:17.475 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::2ca7:98ff:fe8a:d0ed/64 scope link 2024-09-21T07:42:17.475 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:42:17.475 INFO:teuthology.orchestra.run.smithi039.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:42:17.476 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether ee:10:53:01:0c:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:42:17.476 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec10:53ff:fe01:c44/64 scope link 2024-09-21T07:42:17.476 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:42:17.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:42:17.476 DEBUG:teuthology.orchestra.run.smithi039:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:42:17.567 INFO:teuthology.orchestra.run.smithi039.stdout:1 2024-09-21T07:42:17.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:42:17.570 DEBUG:teuthology.orchestra.run.smithi039:> route 2024-09-21T07:42:17.606 INFO:teuthology.orchestra.run.smithi039.stdout:Kernel IP routing table 2024-09-21T07:42:17.606 INFO:teuthology.orchestra.run.smithi039.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:42:17.606 INFO:teuthology.orchestra.run.smithi039.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T07:42:17.606 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T07:42:17.606 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:42:17.607 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:42:17.607 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-21T07:42:17.607 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T07:42:17.607 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T07:42:17.607 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T07:42:17.607 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-21T07:42:17.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:42:17.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:42:17.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:42:17.803 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-09-21T07:42:17.838 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:42:17.838 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T07:42:17.839 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:42:17.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:42:17.839 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-09-21T07:42:17.935 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:42:17.935 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T07:42:17.970 INFO:teuthology.orchestra.run.smithi039.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:42:17.970 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:42:17.971 DEBUG:teuthology.orchestra.run.smithi039:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:42:18.047 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-21T07:42:18.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:42:18.047 DEBUG:teuthology.orchestra.run.smithi039:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:42:18.247 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:42:18.247 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:42:18.282 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:42:18.282 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:42:18.282 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: resolved to: "172.21.15.39:3300,172.21.15.148:3300,172.21.15.148:3301" 2024-09-21T07:42:18.282 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: trying mount with new device syntax: 0@7ab9b04a-f88d-49aa-a493-e5c076637ffb.cephfs=/ 2024-09-21T07:42:18.283 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.39:3300/172.21.15.148:3300/172.21.15.148:3301" will pass to kernel 2024-09-21T07:42:18.284 INFO:teuthology.orchestra.run.smithi039.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T07:42:18.284 INFO:teuthology.orchestra.run.smithi039.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:42:18.284 INFO:teuthology.orchestra.run.smithi039.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:42:18.284 INFO:teuthology.orchestra.run.smithi039.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:42:18.284 INFO:teuthology.orchestra.run.smithi039.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:42:18.287 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:42:18.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:42:18.287 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:42:18.342 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:42:18.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:42:18.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:42:18.435 INFO:teuthology.orchestra.run.smithi039.stdout:client6602 2024-09-21T07:42:18.436 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:42:18.436 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:42:18.471 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 flush journal 2024-09-21T07:42:18.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.738+0000 7f4985000640 1 -- 172.21.15.39:0/2659241679 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f498015cf60 msgr2=0x7f49800b32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:18.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.738+0000 7f4985000640 1 --2- 172.21.15.39:0/2659241679 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f498015cf60 0x7f49800b32b0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f49800cb650 tx=0x7f497002a4d0 comp rx=0 tx=0).stop 2024-09-21T07:42:18.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.738+0000 7f4985000640 1 -- 172.21.15.39:0/2659241679 shutdown_connections 2024-09-21T07:42:18.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.739+0000 7f4985000640 1 --2- 172.21.15.39:0/2659241679 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49800cce70 0x7f49800cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:18.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.739+0000 7f4985000640 1 --2- 172.21.15.39:0/2659241679 >> v2:172.21.15.148:3300/0 conn(0x7f498015d950 0x7f49800cbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:18.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.739+0000 7f4985000640 1 --2- 172.21.15.39:0/2659241679 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f498015cf60 0x7f49800b32b0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:18.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.739+0000 7f4985000640 1 -- 172.21.15.39:0/2659241679 >> 172.21.15.39:0/2659241679 conn(0x7f498006d440 msgr2=0x7f49800b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:18.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.739+0000 7f4985000640 1 -- 172.21.15.39:0/2659241679 shutdown_connections 2024-09-21T07:42:18.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.739+0000 7f4985000640 1 -- 172.21.15.39:0/2659241679 wait complete. 2024-09-21T07:42:18.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.741+0000 7f4985000640 1 Processor -- start 2024-09-21T07:42:18.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.741+0000 7f4985000640 1 -- start start 2024-09-21T07:42:18.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.742+0000 7f4985000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49800cce70 0x7f498014fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:18.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.742+0000 7f4985000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f498015cf60 0x7f498009ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:18.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.742+0000 7f4985000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f498015d950 0x7f49800c9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:18.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.742+0000 7f4985000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f498006f7d0 con 0x7f498015cf60 2024-09-21T07:42:18.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.742+0000 7f4985000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f498006f650 con 0x7f49800cce70 2024-09-21T07:42:18.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.742+0000 7f4985000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f498006f950 con 0x7f498015d950 2024-09-21T07:42:18.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.743+0000 7f4977e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f498015cf60 0x7f498009ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:18.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.743+0000 7f4977e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f498015cf60 0x7f498009ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53658/0 (socket says 172.21.15.39:53658) 2024-09-21T07:42:18.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.743+0000 7f4977e00640 1 -- 172.21.15.39:0/689720839 learned_addr learned my addr 172.21.15.39:0/689720839 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:18.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.743+0000 7f497de00640 1 --2- 172.21.15.39:0/689720839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49800cce70 0x7f498014fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:18.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.743+0000 7f497e800640 1 --2- 172.21.15.39:0/689720839 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f498015d950 0x7f49800c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:18.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.743+0000 7f4977e00640 1 -- 172.21.15.39:0/689720839 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f498015d950 msgr2=0x7f49800c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:18.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.743+0000 7f4977e00640 1 --2- 172.21.15.39:0/689720839 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f498015d950 0x7f49800c9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:18.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.743+0000 7f4977e00640 1 -- 172.21.15.39:0/689720839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49800cce70 msgr2=0x7f498014fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:18.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.743+0000 7f4977e00640 1 --2- 172.21.15.39:0/689720839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49800cce70 0x7f498014fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:18.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.743+0000 7f4977e00640 1 -- 172.21.15.39:0/689720839 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f498009e310 con 0x7f498015cf60 2024-09-21T07:42:18.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.744+0000 7f4977e00640 1 --2- 172.21.15.39:0/689720839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f498015cf60 0x7f498009ddd0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f496800c940 tx=0x7f496800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:18.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.744+0000 7f4976000640 1 -- 172.21.15.39:0/689720839 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4968004230 con 0x7f498015cf60 2024-09-21T07:42:18.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.744+0000 7f4976000640 1 -- 172.21.15.39:0/689720839 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49680079b0 con 0x7f498015cf60 2024-09-21T07:42:18.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.744+0000 7f4985000640 1 -- 172.21.15.39:0/689720839 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4980153e70 con 0x7f498015cf60 2024-09-21T07:42:18.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.744+0000 7f4976000640 1 -- 172.21.15.39:0/689720839 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49680044c0 con 0x7f498015cf60 2024-09-21T07:42:18.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.744+0000 7f4985000640 1 -- 172.21.15.39:0/689720839 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4980154430 con 0x7f498015cf60 2024-09-21T07:42:18.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.748+0000 7f4976000640 1 -- 172.21.15.39:0/689720839 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f49680035f0 con 0x7f498015cf60 2024-09-21T07:42:18.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.748+0000 7f4976000640 1 --2- 172.21.15.39:0/689720839 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f495c078b90 0x7f495c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:18.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.749+0000 7f4976000640 1 -- 172.21.15.39:0/689720839 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7947+0+0 (secure 0 0 0) 0x7f496809b930 con 0x7f498015cf60 2024-09-21T07:42:18.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.749+0000 7f498c1c1740 1 Processor -- start 2024-09-21T07:42:18.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.749+0000 7f498c1c1740 1 -- start start 2024-09-21T07:42:18.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.749+0000 7f497de00640 1 --2- 172.21.15.39:0/689720839 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f495c078b90 0x7f495c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:18.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.750+0000 7f497de00640 1 --2- 172.21.15.39:0/689720839 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f495c078b90 0x7f495c07b0a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4970033000 tx=0x7f4970004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:18.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.750+0000 7f498c1c1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b31d657390 0x55b31da45550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:18.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.750+0000 7f498c1c1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31d62e4d0 0x55b31da28220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:18.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.750+0000 7f498c1c1740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b31d6366b0 0x55b31da28760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:18.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.750+0000 7f498c1c1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b31d95e1c0 con 0x55b31d657390 2024-09-21T07:42:18.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.750+0000 7f498c1c1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b31d971870 con 0x55b31d62e4d0 2024-09-21T07:42:18.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.750+0000 7f498c1c1740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55b31d960ca0 con 0x55b31d6366b0 2024-09-21T07:42:18.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.751+0000 7f497de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b31d657390 0x55b31da45550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:18.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.751+0000 7f497de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b31d657390 0x55b31da45550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53670/0 (socket says 172.21.15.39:53670) 2024-09-21T07:42:18.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.751+0000 7f497de00640 1 -- 172.21.15.39:0/1621827900 learned_addr learned my addr 172.21.15.39:0/1621827900 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:18.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.751+0000 7f4977e00640 1 --2- 172.21.15.39:0/1621827900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31d62e4d0 0x55b31da28220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:18.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.751+0000 7f497e800640 1 --2- 172.21.15.39:0/1621827900 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b31d6366b0 0x55b31da28760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:18.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.751+0000 7f497de00640 1 -- 172.21.15.39:0/1621827900 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b31d6366b0 msgr2=0x55b31da28760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:42:18.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.751+0000 7f497de00640 1 --2- 172.21.15.39:0/1621827900 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b31d6366b0 0x55b31da28760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:18.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.751+0000 7f497de00640 1 -- 172.21.15.39:0/1621827900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31d62e4d0 msgr2=0x55b31da28220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:42:18.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.751+0000 7f497de00640 1 --2- 172.21.15.39:0/1621827900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31d62e4d0 0x55b31da28220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:18.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.751+0000 7f497de00640 1 -- 172.21.15.39:0/1621827900 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b31d7397d0 con 0x55b31d657390 2024-09-21T07:42:18.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.751+0000 7f4977e00640 1 --2- 172.21.15.39:0/1621827900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31d62e4d0 0x55b31da28220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:42:18.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.751+0000 7f497de00640 1 --2- 172.21.15.39:0/1621827900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b31d657390 0x55b31da45550 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f497002aac0 tx=0x7f497003d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=182d8802c9c33452 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:18.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.752+0000 7f496ea00640 1 -- 172.21.15.39:0/1621827900 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4970035040 con 0x55b31d657390 2024-09-21T07:42:18.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.752+0000 7f496ea00640 1 -- 172.21.15.39:0/1621827900 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f497004e070 con 0x55b31d657390 2024-09-21T07:42:18.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.752+0000 7f496ea00640 1 -- 172.21.15.39:0/1621827900 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49700047a0 con 0x55b31d657390 2024-09-21T07:42:18.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.752+0000 7f498c1c1740 1 -- 172.21.15.39:0/1621827900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b31d657390 msgr2=0x55b31da45550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:18.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.752+0000 7f498c1c1740 1 --2- 172.21.15.39:0/1621827900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b31d657390 0x55b31da45550 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f497002aac0 tx=0x7f497003d000 comp rx=0 tx=0).stop 2024-09-21T07:42:18.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.753+0000 7f498c1c1740 1 -- 172.21.15.39:0/1621827900 shutdown_connections 2024-09-21T07:42:18.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.753+0000 7f498c1c1740 1 --2- 172.21.15.39:0/1621827900 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b31d6366b0 0x55b31da28760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:18.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.753+0000 7f498c1c1740 1 --2- 172.21.15.39:0/1621827900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31d62e4d0 0x55b31da28220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:18.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.753+0000 7f498c1c1740 1 --2- 172.21.15.39:0/1621827900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b31d657390 0x55b31da45550 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:18.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.753+0000 7f498c1c1740 1 -- 172.21.15.39:0/1621827900 >> 172.21.15.39:0/1621827900 conn(0x55b31d64b4e0 msgr2=0x55b31d9fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:18.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.753+0000 7f498c1c1740 1 -- 172.21.15.39:0/1621827900 shutdown_connections 2024-09-21T07:42:18.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.753+0000 7f498c1c1740 1 -- 172.21.15.39:0/1621827900 wait complete. 2024-09-21T07:42:18.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.754+0000 7f498c1c1740 1 -- start start 2024-09-21T07:42:18.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.754+0000 7f498c1c1740 1 Processor -- start 2024-09-21T07:42:18.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.755+0000 7f498c1c1740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b31d62e4d0 0x55b31da164f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:18.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.756+0000 7f498c1c1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b31d6366b0 0x55b31d9fda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:18.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.756+0000 7f498c1c1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31d657390 0x55b31d9fdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:18.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.756+0000 7f498c1c1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b31d907410 con 0x55b31d6366b0 2024-09-21T07:42:18.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.756+0000 7f498c1c1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b31d964ab0 con 0x55b31d657390 2024-09-21T07:42:18.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.756+0000 7f498c1c1740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55b31d6d1740 con 0x55b31d62e4d0 2024-09-21T07:42:18.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.756+0000 7f498c1c1740 10 client.0 _ll_register_callbacks cb 0x55b31d8e4360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:42:18.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.756+0000 7f4977e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b31d6366b0 0x55b31d9fda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:18.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.756+0000 7f4977e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b31d6366b0 0x55b31d9fda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53680/0 (socket says 172.21.15.39:53680) 2024-09-21T07:42:18.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.756+0000 7f4977e00640 1 -- 172.21.15.39:0/2938489738 learned_addr learned my addr 172.21.15.39:0/2938489738 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:18.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.756+0000 7f497de00640 1 --2- 172.21.15.39:0/2938489738 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b31d62e4d0 0x55b31da164f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:18.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.756+0000 7f497e800640 1 --2- 172.21.15.39:0/2938489738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31d657390 0x55b31d9fdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:18.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.756+0000 7f4977e00640 1 -- 172.21.15.39:0/2938489738 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b31d62e4d0 msgr2=0x55b31da164f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:42:18.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.756+0000 7f4977e00640 1 --2- 172.21.15.39:0/2938489738 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b31d62e4d0 0x55b31da164f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:18.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.756+0000 7f4977e00640 1 -- 172.21.15.39:0/2938489738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31d657390 msgr2=0x55b31d9fdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:42:18.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.756+0000 7f4977e00640 1 --2- 172.21.15.39:0/2938489738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31d657390 0x55b31d9fdf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:18.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.756+0000 7f4977e00640 1 -- 172.21.15.39:0/2938489738 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b31d6c2970 con 0x55b31d6366b0 2024-09-21T07:42:18.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.756+0000 7f497e800640 1 --2- 172.21.15.39:0/2938489738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31d657390 0x55b31d9fdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:42:18.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.756+0000 7f497de00640 1 --2- 172.21.15.39:0/2938489738 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b31d62e4d0 0x55b31da164f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:42:18.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.757+0000 7f4977e00640 1 --2- 172.21.15.39:0/2938489738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b31d6366b0 0x55b31d9fda10 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f4968062bd0 tx=0x7f49680634a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=424e729b4f53bc6a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:18.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.757+0000 7f4963400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:42:18.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.757+0000 7f4963400640 1 -- 172.21.15.39:0/2938489738 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4968089020 con 0x55b31d6366b0 2024-09-21T07:42:18.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.757+0000 7f498c1c1740 1 -- 172.21.15.39:0/2938489738 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f497004e070 con 0x55b31d6366b0 2024-09-21T07:42:18.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.757+0000 7f4963400640 1 -- 172.21.15.39:0/2938489738 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 7947+0+0 (secure 0 0 0) 0x7f4968062340 con 0x55b31d6366b0 2024-09-21T07:42:18.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.757+0000 7f4963400640 1 -- 172.21.15.39:0/2938489738 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=134) ==== 24+0+0 (secure 0 0 0) 0x7f496801d050 con 0x55b31d6366b0 2024-09-21T07:42:18.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.757+0000 7f498c1c1740 10 client.7767 fetch_fsmap learned FSMap version 134 2024-09-21T07:42:18.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.757+0000 7f498c1c1740 1 -- 172.21.15.39:0/2938489738 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=134,monmap=2+}) -- 0x55b31d6a06a0 con 0x55b31d6366b0 2024-09-21T07:42:18.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.757+0000 7f4963400640 1 -- 172.21.15.39:0/2938489738 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f497004e070 con 0x55b31d6366b0 2024-09-21T07:42:18.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.757+0000 7f4963400640 1 -- 172.21.15.39:0/2938489738 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 134) ==== 3522+0+0 (secure 0 0 0) 0x7f496804e1b0 con 0x55b31d6366b0 2024-09-21T07:42:18.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.757+0000 7f498c1c1740 10 client.7767 fetch_fsmap finished waiting for FSMap version 134 2024-09-21T07:42:18.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.757+0000 7f498c1c1740 10 client.7767 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-21T07:42:18.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.757+0000 7f498c1c1740 20 client.7767 populate_metadata read hostname 'smithi039' 2024-09-21T07:42:18.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.758+0000 7f498c1c1740 1 --2- 172.21.15.39:0/2938489738 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x55b31d615f60 0x55b31da52320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:18.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.758+0000 7f498c1c1740 4 client.7767 mds_command: new command op to 7311 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:42:18.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.758+0000 7f498c1c1740 1 -- 172.21.15.39:0/2938489738 --> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b31d9737c0 con 0x55b31d615f60 2024-09-21T07:42:18.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.758+0000 7f497de00640 1 --2- 172.21.15.39:0/2938489738 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x55b31d615f60 0x55b31da52320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:18.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.758+0000 7f497de00640 1 --2- 172.21.15.39:0/2938489738 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x55b31d615f60 0x55b31da52320 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32bffcb769d1aae6 server_cookie=b4e0fa23690b813 in_seq=0 out_seq=0 2024-09-21T07:42:18.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.758+0000 7f4963400640 10 client.7767 ms_handle_connect on v2:172.21.15.39:6832/1790581246 2024-09-21T07:42:18.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.759+0000 7f4963400640 1 -- 172.21.15.39:0/2938489738 <== mds.0 v2:172.21.15.39:6832/1790581246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b31d9737c0 con 0x55b31d615f60 2024-09-21T07:42:18.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.759+0000 7f4963400640 10 client.7767 handle_command_reply: tid=0 2024-09-21T07:42:18.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.759+0000 7f498c1c1740 1 client.7767 shutdown 2024-09-21T07:42:18.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.759+0000 7f498c1c1740 1 -- 172.21.15.39:0/2938489738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b31d6366b0 msgr2=0x55b31d9fda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:18.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.759+0000 7f498c1c1740 1 --2- 172.21.15.39:0/2938489738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b31d6366b0 0x55b31d9fda10 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f4968062bd0 tx=0x7f49680634a0 comp rx=0 tx=0).stop 2024-09-21T07:42:18.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.759+0000 7f498c1c1740 1 -- 172.21.15.39:0/2938489738 shutdown_connections 2024-09-21T07:42:18.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.759+0000 7f498c1c1740 1 --2- 172.21.15.39:0/2938489738 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x55b31d615f60 0x55b31da52320 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:18.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.759+0000 7f498c1c1740 1 --2- 172.21.15.39:0/2938489738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31d657390 0x55b31d9fdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:18.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.759+0000 7f498c1c1740 1 --2- 172.21.15.39:0/2938489738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b31d6366b0 0x55b31d9fda10 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:18.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.759+0000 7f498c1c1740 1 --2- 172.21.15.39:0/2938489738 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b31d62e4d0 0x55b31da164f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:18.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.759+0000 7f4963400640 0 client.7767 ms_handle_reset on v2:172.21.15.39:6832/1790581246 2024-09-21T07:42:18.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.759+0000 7f498c1c1740 1 -- 172.21.15.39:0/2938489738 >> 172.21.15.39:0/2938489738 conn(0x55b31d64b4e0 msgr2=0x55b31d9f7830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:18.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.760+0000 7f498c1c1740 1 -- 172.21.15.39:0/2938489738 shutdown_connections 2024-09-21T07:42:18.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.760+0000 7f498c1c1740 1 -- 172.21.15.39:0/2938489738 wait complete. 2024-09-21T07:42:18.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.760+0000 7f498c1c1740 20 client.7767 trim_cache size 0 max 16384 2024-09-21T07:42:18.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.775+0000 7f498c1c1740 1 Processor -- start 2024-09-21T07:42:18.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.776+0000 7f498c1c1740 1 -- start start 2024-09-21T07:42:18.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.777+0000 7f498c1c1740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b31d64b4e0 0x55b31d9f7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:18.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.777+0000 7f498c1c1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b31d62e4d0 0x55b31da65d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:18.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.777+0000 7f498c1c1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31d6366b0 0x55b31da662d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:18.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.777+0000 7f498c1c1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4980204cc0 con 0x55b31d62e4d0 2024-09-21T07:42:18.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.777+0000 7f498c1c1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4980205390 con 0x55b31d6366b0 2024-09-21T07:42:18.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.777+0000 7f498c1c1740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f4980203730 con 0x55b31d64b4e0 2024-09-21T07:42:18.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.777+0000 7f4977e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b31d62e4d0 0x55b31da65d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:18.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.777+0000 7f4977e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b31d62e4d0 0x55b31da65d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53692/0 (socket says 172.21.15.39:53692) 2024-09-21T07:42:18.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.777+0000 7f4977e00640 1 -- 172.21.15.39:0/340598677 learned_addr learned my addr 172.21.15.39:0/340598677 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:18.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.777+0000 7f497e800640 1 --2- 172.21.15.39:0/340598677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31d6366b0 0x55b31da662d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:18.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.777+0000 7f497de00640 1 --2- 172.21.15.39:0/340598677 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b31d64b4e0 0x55b31d9f7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:18.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.778+0000 7f4977e00640 1 -- 172.21.15.39:0/340598677 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b31d64b4e0 msgr2=0x55b31d9f7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:42:18.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.778+0000 7f4977e00640 1 --2- 172.21.15.39:0/340598677 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b31d64b4e0 0x55b31d9f7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:18.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.778+0000 7f4977e00640 1 -- 172.21.15.39:0/340598677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31d6366b0 msgr2=0x55b31da662d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:42:18.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.778+0000 7f4977e00640 1 --2- 172.21.15.39:0/340598677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31d6366b0 0x55b31da662d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:18.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.778+0000 7f4977e00640 1 -- 172.21.15.39:0/340598677 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b31da3cd10 con 0x55b31d62e4d0 2024-09-21T07:42:18.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.778+0000 7f4977e00640 1 --2- 172.21.15.39:0/340598677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b31d62e4d0 0x55b31da65d90 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f4968007590 tx=0x7f4968062f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=515f2bcb38ee6db server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:18.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.778+0000 7f496e000640 1 -- 172.21.15.39:0/340598677 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4968089020 con 0x55b31d62e4d0 2024-09-21T07:42:18.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.778+0000 7f496e000640 1 -- 172.21.15.39:0/340598677 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f496806c7b0 con 0x55b31d62e4d0 2024-09-21T07:42:18.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.778+0000 7f496e000640 1 -- 172.21.15.39:0/340598677 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f496804e1b0 con 0x55b31d62e4d0 2024-09-21T07:42:18.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.779+0000 7f498c1c1740 1 -- 172.21.15.39:0/340598677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b31d62e4d0 msgr2=0x55b31da65d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:18.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.779+0000 7f498c1c1740 1 --2- 172.21.15.39:0/340598677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b31d62e4d0 0x55b31da65d90 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f4968007590 tx=0x7f4968062f90 comp rx=0 tx=0).stop 2024-09-21T07:42:18.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.779+0000 7f498c1c1740 1 -- 172.21.15.39:0/340598677 shutdown_connections 2024-09-21T07:42:18.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.779+0000 7f498c1c1740 1 --2- 172.21.15.39:0/340598677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31d6366b0 0x55b31da662d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:18.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.779+0000 7f498c1c1740 1 --2- 172.21.15.39:0/340598677 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b31d62e4d0 0x55b31da65d90 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:18.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.779+0000 7f498c1c1740 1 --2- 172.21.15.39:0/340598677 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b31d64b4e0 0x55b31d9f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:18.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.779+0000 7f498c1c1740 1 -- 172.21.15.39:0/340598677 >> 172.21.15.39:0/340598677 conn(0x55b31d9fe990 msgr2=0x55b31d9f9720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:18.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.779+0000 7f498c1c1740 1 -- 172.21.15.39:0/340598677 shutdown_connections 2024-09-21T07:42:18.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.779+0000 7f498c1c1740 1 -- 172.21.15.39:0/340598677 wait complete. 2024-09-21T07:42:18.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.780+0000 7f498c1c1740 1 -- start start 2024-09-21T07:42:18.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.780+0000 7f498c1c1740 1 Processor -- start 2024-09-21T07:42:18.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.781+0000 7f498c1c1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b31d62e4d0 0x55b31dac1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:18.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.781+0000 7f498c1c1740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b31d6366b0 0x55b31da65ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:18.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.782+0000 7f498c1c1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31d64b4e0 0x55b31da660e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:18.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.782+0000 7f498c1c1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4980203d80 con 0x55b31d62e4d0 2024-09-21T07:42:18.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.782+0000 7f498c1c1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b31da57f90 con 0x55b31d64b4e0 2024-09-21T07:42:18.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.782+0000 7f498c1c1740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55b31da58550 con 0x55b31d6366b0 2024-09-21T07:42:18.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.782+0000 7f498c1c1740 10 client.0 _ll_register_callbacks cb 0x7f49780094f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:42:18.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.782+0000 7f497de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b31d62e4d0 0x55b31dac1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:18.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.782+0000 7f497de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b31d62e4d0 0x55b31dac1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53700/0 (socket says 172.21.15.39:53700) 2024-09-21T07:42:18.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.782+0000 7f497de00640 1 -- 172.21.15.39:0/313522459 learned_addr learned my addr 172.21.15.39:0/313522459 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:18.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.782+0000 7f4977e00640 1 --2- 172.21.15.39:0/313522459 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b31d6366b0 0x55b31da65ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:18.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.782+0000 7f497e800640 1 --2- 172.21.15.39:0/313522459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31d64b4e0 0x55b31da660e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:18.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.782+0000 7f497de00640 1 -- 172.21.15.39:0/313522459 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b31d6366b0 msgr2=0x55b31da65ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:42:18.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.782+0000 7f497de00640 1 --2- 172.21.15.39:0/313522459 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b31d6366b0 0x55b31da65ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:18.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.782+0000 7f497de00640 1 -- 172.21.15.39:0/313522459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31d64b4e0 msgr2=0x55b31da660e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:42:18.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.782+0000 7f497de00640 1 --2- 172.21.15.39:0/313522459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31d64b4e0 0x55b31da660e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:18.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.782+0000 7f497de00640 1 -- 172.21.15.39:0/313522459 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b31da51200 con 0x55b31d62e4d0 2024-09-21T07:42:18.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.782+0000 7f4977e00640 1 --2- 172.21.15.39:0/313522459 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b31d6366b0 0x55b31da65ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:42:18.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.782+0000 7f497e800640 1 --2- 172.21.15.39:0/313522459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31d64b4e0 0x55b31da660e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:42:18.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.782+0000 7f497de00640 1 --2- 172.21.15.39:0/313522459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b31d62e4d0 0x55b31dac1ca0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f49700099c0 tx=0x7f49700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48dd88c8ed09b671 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:18.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.782+0000 7f4963400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:42:18.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.783+0000 7f4963400640 1 -- 172.21.15.39:0/313522459 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4970063070 con 0x55b31d62e4d0 2024-09-21T07:42:18.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.783+0000 7f4963400640 1 -- 172.21.15.39:0/313522459 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 7947+0+0 (secure 0 0 0) 0x7f49700047a0 con 0x55b31d62e4d0 2024-09-21T07:42:18.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.783+0000 7f498c1c1740 1 -- 172.21.15.39:0/313522459 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f496806c7b0 con 0x55b31d62e4d0 2024-09-21T07:42:18.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.783+0000 7f4963400640 1 -- 172.21.15.39:0/313522459 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=134) ==== 24+0+0 (secure 0 0 0) 0x7f4970035040 con 0x55b31d62e4d0 2024-09-21T07:42:18.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.783+0000 7f498c1c1740 10 client.7773 fetch_fsmap learned FSMap version 134 2024-09-21T07:42:18.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.783+0000 7f498c1c1740 1 -- 172.21.15.39:0/313522459 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=134,monmap=2+}) -- 0x55b31da52750 con 0x55b31d62e4d0 2024-09-21T07:42:18.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.783+0000 7f4963400640 1 -- 172.21.15.39:0/313522459 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f496806c7b0 con 0x55b31d62e4d0 2024-09-21T07:42:18.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.783+0000 7f4963400640 1 -- 172.21.15.39:0/313522459 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 134) ==== 3522+0+0 (secure 0 0 0) 0x7f49700049a0 con 0x55b31d62e4d0 2024-09-21T07:42:18.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.783+0000 7f498c1c1740 10 client.7773 fetch_fsmap finished waiting for FSMap version 134 2024-09-21T07:42:18.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.783+0000 7f498c1c1740 10 client.7773 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-21T07:42:18.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.783+0000 7f498c1c1740 20 client.7773 populate_metadata read hostname 'smithi039' 2024-09-21T07:42:18.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.783+0000 7f498c1c1740 1 --2- 172.21.15.39:0/313522459 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x55b31d5eb3a0 0x55b31dac5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:18.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.783+0000 7f498c1c1740 4 client.7773 mds_command: new command op to 7311 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:42:18.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.783+0000 7f498c1c1740 1 -- 172.21.15.39:0/313522459 --> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b31da3cac0 con 0x55b31d5eb3a0 2024-09-21T07:42:18.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.784+0000 7f4977e00640 1 --2- 172.21.15.39:0/313522459 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x55b31d5eb3a0 0x55b31dac5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:18.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.784+0000 7f4977e00640 1 --2- 172.21.15.39:0/313522459 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x55b31d5eb3a0 0x55b31dac5ba0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=728d019c44974a7c server_cookie=6d60b35399b94913 in_seq=0 out_seq=0 2024-09-21T07:42:18.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.784+0000 7f4963400640 10 client.7773 ms_handle_connect on v2:172.21.15.39:6832/1790581246 2024-09-21T07:42:18.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.796+0000 7f4963400640 1 -- 172.21.15.39:0/313522459 <== mds.0 v2:172.21.15.39:6832/1790581246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b31da3cac0 con 0x55b31d5eb3a0 2024-09-21T07:42:18.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.796+0000 7f4963400640 10 client.7773 handle_command_reply: tid=0 2024-09-21T07:42:18.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.796+0000 7f498c1c1740 1 client.7773 shutdown 2024-09-21T07:42:18.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.796+0000 7f498c1c1740 1 -- 172.21.15.39:0/313522459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b31d62e4d0 msgr2=0x55b31dac1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:18.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.796+0000 7f498c1c1740 1 --2- 172.21.15.39:0/313522459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b31d62e4d0 0x55b31dac1ca0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f49700099c0 tx=0x7f49700046c0 comp rx=0 tx=0).stop 2024-09-21T07:42:18.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.796+0000 7f498c1c1740 1 -- 172.21.15.39:0/313522459 shutdown_connections 2024-09-21T07:42:18.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.796+0000 7f498c1c1740 1 --2- 172.21.15.39:0/313522459 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x55b31d5eb3a0 0x55b31dac5ba0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:18.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.796+0000 7f498c1c1740 1 --2- 172.21.15.39:0/313522459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b31d64b4e0 0x55b31da660e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:18.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.796+0000 7f498c1c1740 1 --2- 172.21.15.39:0/313522459 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b31d6366b0 0x55b31da65ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:18.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.796+0000 7f498c1c1740 1 --2- 172.21.15.39:0/313522459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b31d62e4d0 0x55b31dac1ca0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:18.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.796+0000 7f4963400640 0 client.7773 ms_handle_reset on v2:172.21.15.39:6832/1790581246 2024-09-21T07:42:18.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.796+0000 7f498c1c1740 1 -- 172.21.15.39:0/313522459 >> 172.21.15.39:0/313522459 conn(0x55b31d9fe990 msgr2=0x55b31d9f9680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:18.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.796+0000 7f498c1c1740 1 -- 172.21.15.39:0/313522459 shutdown_connections 2024-09-21T07:42:18.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.796+0000 7f498c1c1740 1 -- 172.21.15.39:0/313522459 wait complete. 2024-09-21T07:42:18.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.796+0000 7f498c1c1740 20 client.7773 trim_cache size 0 max 16384 2024-09-21T07:42:18.799 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:42:18.799 INFO:teuthology.orchestra.run.smithi039.stdout: "message": "", 2024-09-21T07:42:18.800 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0 2024-09-21T07:42:18.800 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:42:18.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.797+0000 7f4985000640 1 -- 172.21.15.39:0/689720839 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f495c078b90 msgr2=0x7f495c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:18.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.797+0000 7f4985000640 1 --2- 172.21.15.39:0/689720839 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f495c078b90 0x7f495c07b0a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4970033000 tx=0x7f4970004730 comp rx=0 tx=0).stop 2024-09-21T07:42:18.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.797+0000 7f4985000640 1 -- 172.21.15.39:0/689720839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f498015cf60 msgr2=0x7f498009ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:18.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.797+0000 7f4985000640 1 --2- 172.21.15.39:0/689720839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f498015cf60 0x7f498009ddd0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f496800c940 tx=0x7f496800ce00 comp rx=0 tx=0).stop 2024-09-21T07:42:18.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.797+0000 7f4985000640 1 -- 172.21.15.39:0/689720839 shutdown_connections 2024-09-21T07:42:18.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.797+0000 7f4985000640 1 --2- 172.21.15.39:0/689720839 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f495c078b90 0x7f495c07b0a0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:18.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.797+0000 7f4985000640 1 --2- 172.21.15.39:0/689720839 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f498015d950 0x7f49800c9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:18.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.797+0000 7f4985000640 1 --2- 172.21.15.39:0/689720839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f498015cf60 0x7f498009ddd0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:18.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.797+0000 7f4985000640 1 --2- 172.21.15.39:0/689720839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49800cce70 0x7f498014fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:18.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.797+0000 7f4985000640 1 -- 172.21.15.39:0/689720839 >> 172.21.15.39:0/689720839 conn(0x7f498006d440 msgr2=0x7f49800b3910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:18.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.797+0000 7f4985000640 1 -- 172.21.15.39:0/689720839 shutdown_connections 2024-09-21T07:42:18.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:18.797+0000 7f4985000640 1 -- 172.21.15.39:0/689720839 wait complete. 2024-09-21T07:42:18.822 INFO:tasks.cephfs.test_snapshots:testing snapclient mds_kill_mdstable_at=4 2024-09-21T07:42:18.823 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:42:19.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.049+0000 7fda79400640 1 -- 172.21.15.39:0/3083429915 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda7415c830 msgr2=0x7fda7406f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:19.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.049+0000 7fda79400640 1 --2- 172.21.15.39:0/3083429915 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda7415c830 0x7fda7406f8a0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fda5c005fd0 tx=0x7fda5c02a530 comp rx=0 tx=0).stop 2024-09-21T07:42:19.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.049+0000 7fda79400640 1 -- 172.21.15.39:0/3083429915 shutdown_connections 2024-09-21T07:42:19.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.049+0000 7fda79400640 1 --2- 172.21.15.39:0/3083429915 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda7406fde0 0x7fda74072fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:19.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.049+0000 7fda79400640 1 --2- 172.21.15.39:0/3083429915 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda7415c830 0x7fda7406f8a0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:19.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.049+0000 7fda79400640 1 -- 172.21.15.39:0/3083429915 >> 172.21.15.39:0/3083429915 conn(0x7fda740b2510 msgr2=0x7fda740d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:19.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.049+0000 7fda79400640 1 -- 172.21.15.39:0/3083429915 shutdown_connections 2024-09-21T07:42:19.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.049+0000 7fda79400640 1 -- 172.21.15.39:0/3083429915 wait complete. 2024-09-21T07:42:19.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.051+0000 7fda79400640 1 Processor -- start 2024-09-21T07:42:19.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.051+0000 7fda79400640 1 -- start start 2024-09-21T07:42:19.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.053+0000 7fda79400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda7406fde0 0x7fda741f99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:19.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.053+0000 7fda79400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda740d0f20 0x7fda741f8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:19.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.053+0000 7fda79400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda7415c830 0x7fda741f9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:19.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.053+0000 7fda79400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fda740b4570 con 0x7fda740d0f20 2024-09-21T07:42:19.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.053+0000 7fda79400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fda740b43f0 con 0x7fda7406fde0 2024-09-21T07:42:19.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.053+0000 7fda79400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fda740b46f0 con 0x7fda7415c830 2024-09-21T07:42:19.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.053+0000 7fda6be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda740d0f20 0x7fda741f8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:19.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.053+0000 7fda6be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda740d0f20 0x7fda741f8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53710/0 (socket says 172.21.15.39:53710) 2024-09-21T07:42:19.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.053+0000 7fda6be00640 1 -- 172.21.15.39:0/1885852008 learned_addr learned my addr 172.21.15.39:0/1885852008 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:19.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.053+0000 7fda73e00640 1 --2- 172.21.15.39:0/1885852008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda7406fde0 0x7fda741f99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:19.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.053+0000 7fda78a00640 1 --2- 172.21.15.39:0/1885852008 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda7415c830 0x7fda741f9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:19.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.053+0000 7fda6be00640 1 -- 172.21.15.39:0/1885852008 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda7415c830 msgr2=0x7fda741f9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:19.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.053+0000 7fda6be00640 1 --2- 172.21.15.39:0/1885852008 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda7415c830 0x7fda741f9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:19.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.054+0000 7fda6be00640 1 -- 172.21.15.39:0/1885852008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda7406fde0 msgr2=0x7fda741f99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:19.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.054+0000 7fda6be00640 1 --2- 172.21.15.39:0/1885852008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda7406fde0 0x7fda741f99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:19.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.054+0000 7fda6be00640 1 -- 172.21.15.39:0/1885852008 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda7409e170 con 0x7fda740d0f20 2024-09-21T07:42:19.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.054+0000 7fda73e00640 1 --2- 172.21.15.39:0/1885852008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda7406fde0 0x7fda741f99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:42:19.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.054+0000 7fda6be00640 1 --2- 172.21.15.39:0/1885852008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda740d0f20 0x7fda741f8e10 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fda5c006010 tx=0x7fda5c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:19.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.054+0000 7fda70c00640 1 -- 172.21.15.39:0/1885852008 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c03e030 con 0x7fda740d0f20 2024-09-21T07:42:19.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.054+0000 7fda70c00640 1 -- 172.21.15.39:0/1885852008 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda5c003d90 con 0x7fda740d0f20 2024-09-21T07:42:19.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.054+0000 7fda79400640 1 -- 172.21.15.39:0/1885852008 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda7409a4f0 con 0x7fda740d0f20 2024-09-21T07:42:19.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.054+0000 7fda70c00640 1 -- 172.21.15.39:0/1885852008 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c035040 con 0x7fda740d0f20 2024-09-21T07:42:19.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.054+0000 7fda79400640 1 -- 172.21.15.39:0/1885852008 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda740c3d70 con 0x7fda740d0f20 2024-09-21T07:42:19.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.055+0000 7fda79400640 1 -- 172.21.15.39:0/1885852008 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda7409dee0 con 0x7fda740d0f20 2024-09-21T07:42:19.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.057+0000 7fda70c00640 1 -- 172.21.15.39:0/1885852008 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fda5c041030 con 0x7fda740d0f20 2024-09-21T07:42:19.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.058+0000 7fda70c00640 1 --2- 172.21.15.39:0/1885852008 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fda50078bc0 0x7fda5007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:19.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.058+0000 7fda70c00640 1 -- 172.21.15.39:0/1885852008 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7947+0+0 (secure 0 0 0) 0x7fda5c0bbd80 con 0x7fda740d0f20 2024-09-21T07:42:19.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.058+0000 7fda73e00640 1 --2- 172.21.15.39:0/1885852008 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fda50078bc0 0x7fda5007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:19.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.059+0000 7fda73e00640 1 --2- 172.21.15.39:0/1885852008 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fda50078bc0 0x7fda5007b0d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fda64006fd0 tx=0x7fda64008040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:19.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.062+0000 7fda70c00640 1 -- 172.21.15.39:0/1885852008 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fda5c083990 con 0x7fda740d0f20 2024-09-21T07:42:19.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.299+0000 7fda79400640 1 -- 172.21.15.39:0/1885852008 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda740b4c10 con 0x7fda740d0f20 2024-09-21T07:42:19.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.300+0000 7fda70c00640 1 -- 172.21.15.39:0/1885852008 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 134 v134) ==== 95+0+5253 (secure 0 0 0) 0x7fda5c083290 con 0x7fda740d0f20 2024-09-21T07:42:19.302 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:42:19.302 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":134,"btime":"2024-09-21T07:42:03:879182+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7644,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1326711417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1326711417},{"type":"v1","addr":"172.21.15.39:6835","nonce":1326711417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":131}],"filesystems":[{"mdsmap":{"epoch":134,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:42:03.879172+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":6425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6425":{"gid":6425,"name":"d","rank":1,"incarnation":130,"state":"up:active","state_seq":18,"addr":"172.21.15.148:6837/73220122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":73220122},{"type":"v1","addr":"172.21.15.148:6837","nonce":73220122}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[6425,7311]},"id":6}]} 2024-09-21T07:42:19.303 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 134 2024-09-21T07:42:19.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.303+0000 7fda79400640 1 -- 172.21.15.39:0/1885852008 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fda50078bc0 msgr2=0x7fda5007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:19.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.303+0000 7fda79400640 1 --2- 172.21.15.39:0/1885852008 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fda50078bc0 0x7fda5007b0d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fda64006fd0 tx=0x7fda64008040 comp rx=0 tx=0).stop 2024-09-21T07:42:19.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.303+0000 7fda79400640 1 -- 172.21.15.39:0/1885852008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda740d0f20 msgr2=0x7fda741f8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:19.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.303+0000 7fda79400640 1 --2- 172.21.15.39:0/1885852008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda740d0f20 0x7fda741f8e10 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fda5c006010 tx=0x7fda5c0038a0 comp rx=0 tx=0).stop 2024-09-21T07:42:19.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.303+0000 7fda79400640 1 -- 172.21.15.39:0/1885852008 shutdown_connections 2024-09-21T07:42:19.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.303+0000 7fda79400640 1 --2- 172.21.15.39:0/1885852008 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fda50078bc0 0x7fda5007b0d0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:19.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.303+0000 7fda79400640 1 --2- 172.21.15.39:0/1885852008 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda7415c830 0x7fda741f9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:19.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.303+0000 7fda79400640 1 --2- 172.21.15.39:0/1885852008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda740d0f20 0x7fda741f8e10 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:19.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.303+0000 7fda79400640 1 --2- 172.21.15.39:0/1885852008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda7406fde0 0x7fda741f99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:19.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.303+0000 7fda79400640 1 -- 172.21.15.39:0/1885852008 >> 172.21.15.39:0/1885852008 conn(0x7fda740b2510 msgr2=0x7fda7415d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:19.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.303+0000 7fda79400640 1 -- 172.21.15.39:0/1885852008 shutdown_connections 2024-09-21T07:42:19.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.303+0000 7fda79400640 1 -- 172.21.15.39:0/1885852008 wait complete. 2024-09-21T07:42:19.327 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2024-09-21T07:42:19.571 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_snap":15,"last_created":12,"last_destroyed":14,"pending_noop":[],"snaps":[],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2024-09-21T07:42:19.571 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 12, "last_destroyed": 14, "last_snap": 15, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [] } 2024-09-21T07:42:19.571 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:42:19.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.798+0000 7fa80cc00640 1 -- 172.21.15.39:0/3934599507 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8080b2ca0 msgr2=0x7fa8081697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:19.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.798+0000 7fa80cc00640 1 --2- 172.21.15.39:0/3934599507 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8080b2ca0 0x7fa8081697d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa800009be0 tx=0x7fa80002a920 comp rx=0 tx=0).stop 2024-09-21T07:42:19.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.798+0000 7fa80cc00640 1 -- 172.21.15.39:0/3934599507 shutdown_connections 2024-09-21T07:42:19.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.798+0000 7fa80cc00640 1 --2- 172.21.15.39:0/3934599507 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8080b2ca0 0x7fa8081697d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:19.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.798+0000 7fa80cc00640 1 -- 172.21.15.39:0/3934599507 >> 172.21.15.39:0/3934599507 conn(0x7fa80808bfb0 msgr2=0x7fa80809d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:19.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.798+0000 7fa80cc00640 1 -- 172.21.15.39:0/3934599507 shutdown_connections 2024-09-21T07:42:19.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.799+0000 7fa80cc00640 1 -- 172.21.15.39:0/3934599507 wait complete. 2024-09-21T07:42:19.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.801+0000 7fa80cc00640 1 Processor -- start 2024-09-21T07:42:19.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.801+0000 7fa80cc00640 1 -- start start 2024-09-21T07:42:19.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.802+0000 7fa80cc00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8080b2ca0 0x7fa80814b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:19.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.802+0000 7fa80cc00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8080d19f0 0x7fa8081f8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:19.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.802+0000 7fa80cc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8081f9200 0x7fa8080b8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:19.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.802+0000 7fa80cc00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa808169e90 con 0x7fa8081f9200 2024-09-21T07:42:19.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.802+0000 7fa80cc00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa808169d10 con 0x7fa8080b2ca0 2024-09-21T07:42:19.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.802+0000 7fa80cc00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa80816a010 con 0x7fa8080d19f0 2024-09-21T07:42:19.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.802+0000 7fa807e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8081f9200 0x7fa8080b8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:19.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.803+0000 7fa807e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8081f9200 0x7fa8080b8620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53720/0 (socket says 172.21.15.39:53720) 2024-09-21T07:42:19.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.803+0000 7fa807e00640 1 -- 172.21.15.39:0/1945441173 learned_addr learned my addr 172.21.15.39:0/1945441173 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:19.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.803+0000 7fa807400640 1 --2- 172.21.15.39:0/1945441173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8080b2ca0 0x7fa80814b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:19.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.803+0000 7fa7ffe00640 1 --2- 172.21.15.39:0/1945441173 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8080d19f0 0x7fa8081f8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:19.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.803+0000 7fa807e00640 1 -- 172.21.15.39:0/1945441173 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8080d19f0 msgr2=0x7fa8081f8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:19.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.803+0000 7fa807e00640 1 --2- 172.21.15.39:0/1945441173 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8080d19f0 0x7fa8081f8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:19.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.803+0000 7fa807e00640 1 -- 172.21.15.39:0/1945441173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8080b2ca0 msgr2=0x7fa80814b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:19.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.803+0000 7fa807e00640 1 --2- 172.21.15.39:0/1945441173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8080b2ca0 0x7fa80814b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:19.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.803+0000 7fa807e00640 1 -- 172.21.15.39:0/1945441173 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8081f9a30 con 0x7fa8081f9200 2024-09-21T07:42:19.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.803+0000 7fa807e00640 1 --2- 172.21.15.39:0/1945441173 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8081f9200 0x7fa8080b8620 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fa800009be0 tx=0x7fa800003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:19.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.804+0000 7fa7ff400640 1 -- 172.21.15.39:0/1945441173 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa800030c50 con 0x7fa8081f9200 2024-09-21T07:42:19.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.804+0000 7fa7ff400640 1 -- 172.21.15.39:0/1945441173 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa80003e070 con 0x7fa8081f9200 2024-09-21T07:42:19.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.804+0000 7fa80cc00640 1 -- 172.21.15.39:0/1945441173 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa80814dc60 con 0x7fa8081f9200 2024-09-21T07:42:19.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.804+0000 7fa7ff400640 1 -- 172.21.15.39:0/1945441173 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa800032330 con 0x7fa8081f9200 2024-09-21T07:42:19.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.804+0000 7fa80cc00640 1 -- 172.21.15.39:0/1945441173 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa80814e220 con 0x7fa8081f9200 2024-09-21T07:42:19.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.805+0000 7fa80cc00640 1 -- 172.21.15.39:0/1945441173 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa808069a50 con 0x7fa8081f9200 2024-09-21T07:42:19.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.806+0000 7fa7ff400640 1 -- 172.21.15.39:0/1945441173 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa8000324d0 con 0x7fa8081f9200 2024-09-21T07:42:19.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.807+0000 7fa7ff400640 1 --2- 172.21.15.39:0/1945441173 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa7e4078b90 0x7fa7e407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:19.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.807+0000 7fa7ff400640 1 -- 172.21.15.39:0/1945441173 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7947+0+0 (secure 0 0 0) 0x7fa8000bcda0 con 0x7fa8081f9200 2024-09-21T07:42:19.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.808+0000 7fa807400640 1 --2- 172.21.15.39:0/1945441173 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa7e4078b90 0x7fa7e407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:19.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.808+0000 7fa807400640 1 --2- 172.21.15.39:0/1945441173 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa7e4078b90 0x7fa7e407b0a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8001760 tx=0x7fa7f8009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:19.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:19.812+0000 7fa7ff400640 1 -- 172.21.15.39:0/1945441173 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa80002d070 con 0x7fa8081f9200 2024-09-21T07:42:20.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.047+0000 7fa80cc00640 1 -- 172.21.15.39:0/1945441173 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8081f97c0 con 0x7fa8081f9200 2024-09-21T07:42:20.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.048+0000 7fa7ff400640 1 -- 172.21.15.39:0/1945441173 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 134 v134) ==== 95+0+5253 (secure 0 0 0) 0x7fa8000841b0 con 0x7fa8081f9200 2024-09-21T07:42:20.050 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:42:20.050 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":134,"btime":"2024-09-21T07:42:03:879182+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7644,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1326711417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1326711417},{"type":"v1","addr":"172.21.15.39:6835","nonce":1326711417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":131}],"filesystems":[{"mdsmap":{"epoch":134,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:42:03.879172+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":6425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6425":{"gid":6425,"name":"d","rank":1,"incarnation":130,"state":"up:active","state_seq":18,"addr":"172.21.15.148:6837/73220122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":73220122},{"type":"v1","addr":"172.21.15.148:6837","nonce":73220122}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[6425,7311]},"id":6}]} 2024-09-21T07:42:20.050 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 134 2024-09-21T07:42:20.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.050+0000 7fa80cc00640 1 -- 172.21.15.39:0/1945441173 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa7e4078b90 msgr2=0x7fa7e407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:20.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.050+0000 7fa80cc00640 1 --2- 172.21.15.39:0/1945441173 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa7e4078b90 0x7fa7e407b0a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8001760 tx=0x7fa7f8009040 comp rx=0 tx=0).stop 2024-09-21T07:42:20.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.050+0000 7fa80cc00640 1 -- 172.21.15.39:0/1945441173 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8081f9200 msgr2=0x7fa8080b8620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:20.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.050+0000 7fa80cc00640 1 --2- 172.21.15.39:0/1945441173 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8081f9200 0x7fa8080b8620 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fa800009be0 tx=0x7fa800003ba0 comp rx=0 tx=0).stop 2024-09-21T07:42:20.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.050+0000 7fa80cc00640 1 -- 172.21.15.39:0/1945441173 shutdown_connections 2024-09-21T07:42:20.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.050+0000 7fa80cc00640 1 --2- 172.21.15.39:0/1945441173 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa7e4078b90 0x7fa7e407b0a0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:20.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.050+0000 7fa80cc00640 1 --2- 172.21.15.39:0/1945441173 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8081f9200 0x7fa8080b8620 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:20.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.050+0000 7fa80cc00640 1 --2- 172.21.15.39:0/1945441173 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8080d19f0 0x7fa8081f8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:20.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.050+0000 7fa80cc00640 1 --2- 172.21.15.39:0/1945441173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8080b2ca0 0x7fa80814b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:20.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.050+0000 7fa80cc00640 1 -- 172.21.15.39:0/1945441173 >> 172.21.15.39:0/1945441173 conn(0x7fa80808bfb0 msgr2=0x7fa80806ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:20.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.051+0000 7fa80cc00640 1 -- 172.21.15.39:0/1945441173 shutdown_connections 2024-09-21T07:42:20.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.051+0000 7fa80cc00640 1 -- 172.21.15.39:0/1945441173 wait complete. 2024-09-21T07:42:20.074 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:1 true 2024-09-21T07:42:20.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.300+0000 7f3b47200640 1 -- 172.21.15.39:0/3103086307 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b24004800 msgr2=0x7f3b24004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:20.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.300+0000 7f3b47200640 1 --2- 172.21.15.39:0/3103086307 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b24004800 0x7f3b24004c10 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f3b28009b50 tx=0x7f3b24005830 comp rx=0 tx=0).stop 2024-09-21T07:42:20.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.300+0000 7f3b47200640 1 -- 172.21.15.39:0/3103086307 shutdown_connections 2024-09-21T07:42:20.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.300+0000 7f3b47200640 1 --2- 172.21.15.39:0/3103086307 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b24004800 0x7f3b24004c10 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:20.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.300+0000 7f3b47200640 1 -- 172.21.15.39:0/3103086307 >> 172.21.15.39:0/3103086307 conn(0x7f3b4006d440 msgr2=0x7f3b400b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:20.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.300+0000 7f3b47200640 1 -- 172.21.15.39:0/3103086307 shutdown_connections 2024-09-21T07:42:20.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.300+0000 7f3b47200640 1 -- 172.21.15.39:0/3103086307 wait complete. 2024-09-21T07:42:20.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.302+0000 7f3b47200640 1 Processor -- start 2024-09-21T07:42:20.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.302+0000 7f3b47200640 1 -- start start 2024-09-21T07:42:20.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.304+0000 7f3b47200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b24004800 0x7f3b40155160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:20.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.304+0000 7f3b47200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b40159280 0x7f3b40160040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:20.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.304+0000 7f3b47200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b40153080 0x7f3b40160580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:20.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.304+0000 7f3b47200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3b400cd280 con 0x7f3b24004800 2024-09-21T07:42:20.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.304+0000 7f3b47200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4015de30 con 0x7f3b40159280 2024-09-21T07:42:20.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.304+0000 7f3b47200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f3b400cd400 con 0x7f3b40153080 2024-09-21T07:42:20.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.304+0000 7f3b3fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b24004800 0x7f3b40155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:20.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.304+0000 7f3b3fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b24004800 0x7f3b40155160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53738/0 (socket says 172.21.15.39:53738) 2024-09-21T07:42:20.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.304+0000 7f3b3fe00640 1 -- 172.21.15.39:0/1144425884 learned_addr learned my addr 172.21.15.39:0/1144425884 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:20.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.304+0000 7f3b37400640 1 --2- 172.21.15.39:0/1144425884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b40159280 0x7f3b40160040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:20.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.304+0000 7f3b45200640 1 --2- 172.21.15.39:0/1144425884 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b40153080 0x7f3b40160580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:20.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.305+0000 7f3b3fe00640 1 -- 172.21.15.39:0/1144425884 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b40153080 msgr2=0x7f3b40160580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:20.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.305+0000 7f3b3fe00640 1 --2- 172.21.15.39:0/1144425884 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b40153080 0x7f3b40160580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:20.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.305+0000 7f3b3fe00640 1 -- 172.21.15.39:0/1144425884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b40159280 msgr2=0x7f3b40160040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:20.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.305+0000 7f3b3fe00640 1 --2- 172.21.15.39:0/1144425884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b40159280 0x7f3b40160040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:20.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.305+0000 7f3b3fe00640 1 -- 172.21.15.39:0/1144425884 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b401556a0 con 0x7f3b24004800 2024-09-21T07:42:20.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.305+0000 7f3b37400640 1 --2- 172.21.15.39:0/1144425884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b40159280 0x7f3b40160040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:42:20.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.305+0000 7f3b3fe00640 1 --2- 172.21.15.39:0/1144425884 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b24004800 0x7f3b40155160 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f3b3000c730 tx=0x7f3b3000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:20.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.305+0000 7f3b3cc00640 1 -- 172.21.15.39:0/1144425884 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b30017070 con 0x7f3b24004800 2024-09-21T07:42:20.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.305+0000 7f3b3cc00640 1 -- 172.21.15.39:0/1144425884 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b30005380 con 0x7f3b24004800 2024-09-21T07:42:20.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.305+0000 7f3b3cc00640 1 -- 172.21.15.39:0/1144425884 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b30003850 con 0x7f3b24004800 2024-09-21T07:42:20.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.305+0000 7f3b47200640 1 -- 172.21.15.39:0/1144425884 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b4015f280 con 0x7f3b24004800 2024-09-21T07:42:20.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.305+0000 7f3b47200640 1 -- 172.21.15.39:0/1144425884 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b4015f840 con 0x7f3b24004800 2024-09-21T07:42:20.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.306+0000 7f3b47200640 1 -- 172.21.15.39:0/1144425884 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b400cbb10 con 0x7f3b24004800 2024-09-21T07:42:20.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.308+0000 7f3b3cc00640 1 -- 172.21.15.39:0/1144425884 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3b3001a070 con 0x7f3b24004800 2024-09-21T07:42:20.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.309+0000 7f3b3cc00640 1 --2- 172.21.15.39:0/1144425884 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3b1c078bc0 0x7f3b1c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:20.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.309+0000 7f3b3cc00640 1 -- 172.21.15.39:0/1144425884 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7947+0+0 (secure 0 0 0) 0x7f3b3009abc0 con 0x7f3b24004800 2024-09-21T07:42:20.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.309+0000 7f3b37400640 1 --2- 172.21.15.39:0/1144425884 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3b1c078bc0 0x7f3b1c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:20.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.310+0000 7f3b37400640 1 --2- 172.21.15.39:0/1144425884 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3b1c078bc0 0x7f3b1c07b0d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3b28035000 tx=0x7f3b280046c0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:20.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.313+0000 7f3b3cc00640 1 -- 172.21.15.39:0/1144425884 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3b300626a0 con 0x7f3b24004800 2024-09-21T07:42:20.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.584+0000 7f3b47200640 1 -- 172.21.15.39:0/1144425884 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:1", "val": "true"} v 0) -- 0x7f3b40099e70 con 0x7f3b24004800 2024-09-21T07:42:20.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.942+0000 7f3b3cc00640 1 -- 172.21.15.39:0/1144425884 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:1", "val": "true"}]=0 freezing mds.6425 v135) ==== 112+0+0 (secure 0 0 0) 0x7f3b30061fa0 con 0x7f3b24004800 2024-09-21T07:42:20.945 INFO:teuthology.orchestra.run.smithi039.stderr:freezing mds.6425 2024-09-21T07:42:20.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.946+0000 7f3b47200640 1 -- 172.21.15.39:0/1144425884 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3b1c078bc0 msgr2=0x7f3b1c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:20.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.946+0000 7f3b47200640 1 --2- 172.21.15.39:0/1144425884 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3b1c078bc0 0x7f3b1c07b0d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3b28035000 tx=0x7f3b280046c0 comp rx=0 tx=0).stop 2024-09-21T07:42:20.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.946+0000 7f3b47200640 1 -- 172.21.15.39:0/1144425884 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b24004800 msgr2=0x7f3b40155160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:20.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.946+0000 7f3b47200640 1 --2- 172.21.15.39:0/1144425884 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b24004800 0x7f3b40155160 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f3b3000c730 tx=0x7f3b3000cbf0 comp rx=0 tx=0).stop 2024-09-21T07:42:20.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.946+0000 7f3b47200640 1 -- 172.21.15.39:0/1144425884 shutdown_connections 2024-09-21T07:42:20.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.946+0000 7f3b47200640 1 --2- 172.21.15.39:0/1144425884 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3b1c078bc0 0x7f3b1c07b0d0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:20.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.946+0000 7f3b47200640 1 --2- 172.21.15.39:0/1144425884 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b40153080 0x7f3b40160580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:20.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.946+0000 7f3b47200640 1 --2- 172.21.15.39:0/1144425884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b40159280 0x7f3b40160040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:20.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.946+0000 7f3b47200640 1 --2- 172.21.15.39:0/1144425884 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b24004800 0x7f3b40155160 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:20.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.946+0000 7f3b47200640 1 -- 172.21.15.39:0/1144425884 >> 172.21.15.39:0/1144425884 conn(0x7f3b4006d440 msgr2=0x7f3b4015a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:20.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.946+0000 7f3b47200640 1 -- 172.21.15.39:0/1144425884 shutdown_connections 2024-09-21T07:42:20.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:20.946+0000 7f3b47200640 1 -- 172.21.15.39:0/1144425884 wait complete. 2024-09-21T07:42:20.972 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json config set mds_kill_mdstable_at 4 2024-09-21T07:42:21.215 INFO:teuthology.orchestra.run.smithi148.stdout:{"success":"mds_kill_mdstable_at = '' (not observed, change may require restart) "} 2024-09-21T07:42:21.215 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_kill_mdstable_at = '' (not observed, change may require restart) " } 2024-09-21T07:42:21.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:42:21.215 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d1/dir/.snap/s34) 2024-09-21T07:42:21.220 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-09-21T07:42:21.260 INFO:tasks.ceph.mds.d.smithi148.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::commit(version_t, LogSegment*)' thread 7f2e32e00640 time 2024-09-21T07:42:21.260292+0000 2024-09-21T07:42:21.261 INFO:tasks.ceph.mds.d.smithi148.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: 195: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 4) 2024-09-21T07:42:21.261 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:42:21.261 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f2e41e5654b] 2024-09-21T07:42:21.261 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f2e41e56758] 2024-09-21T07:42:21.261 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: (MDSTableClient::commit(unsigned long, LogSegment*)+0x333) [0x55a9810d23d3] 2024-09-21T07:42:21.262 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: (Server::_mksnap_finish(boost::intrusive_ptr const&, CInode*, SnapInfo&)+0x9f) [0x55a980e85ac3] 2024-09-21T07:42:21.262 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: ceph-mds(+0x221794) [0x55a980e7a794] 2024-09-21T07:42:21.262 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: (MDSContext::complete(int)+0x7c) [0x55a9811206d8] 2024-09-21T07:42:21.262 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSIOContextBase::complete(int)+0x501) [0x55a981120d57] 2024-09-21T07:42:21.262 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: (MDSLogContextBase::complete(int)+0x46) [0x55a981120dd4] 2024-09-21T07:42:21.262 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: (Finisher::finisher_thread_entry()+0x495) [0x7f2e41dde34b] 2024-09-21T07:42:21.262 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: ceph-mds(+0x1052e1) [0x55a980d5e2e1] 2024-09-21T07:42:21.262 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (Thread::entry_wrapper()+0x43) [0x7f2e41e2fd57] 2024-09-21T07:42:21.262 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: (Thread::_entry_func(void*)+0xd) [0x7f2e41e2fd73] 2024-09-21T07:42:21.262 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: /lib64/libc.so.6(+0x89c52) [0x7f2e41489c52] 2024-09-21T07:42:21.262 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: /lib64/libc.so.6(+0x10ec80) [0x7f2e4150ec80] 2024-09-21T07:42:21.262 INFO:tasks.ceph.mds.d.smithi148.stderr:2024-09-21T07:42:21.259+0000 7f2e32e00640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::commit(version_t, LogSegment*)' thread 7f2e32e00640 time 2024-09-21T07:42:21.260292+0000 2024-09-21T07:42:21.262 INFO:tasks.ceph.mds.d.smithi148.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: 195: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 4) 2024-09-21T07:42:21.262 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:42:21.263 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:42:21.263 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f2e41e5654b] 2024-09-21T07:42:21.263 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f2e41e56758] 2024-09-21T07:42:21.263 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: (MDSTableClient::commit(unsigned long, LogSegment*)+0x333) [0x55a9810d23d3] 2024-09-21T07:42:21.263 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: (Server::_mksnap_finish(boost::intrusive_ptr const&, CInode*, SnapInfo&)+0x9f) [0x55a980e85ac3] 2024-09-21T07:42:21.263 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: ceph-mds(+0x221794) [0x55a980e7a794] 2024-09-21T07:42:21.263 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: (MDSContext::complete(int)+0x7c) [0x55a9811206d8] 2024-09-21T07:42:21.263 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSIOContextBase::complete(int)+0x501) [0x55a981120d57] 2024-09-21T07:42:21.263 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: (MDSLogContextBase::complete(int)+0x46) [0x55a981120dd4] 2024-09-21T07:42:21.263 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: (Finisher::finisher_thread_entry()+0x495) [0x7f2e41dde34b] 2024-09-21T07:42:21.263 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: ceph-mds(+0x1052e1) [0x55a980d5e2e1] 2024-09-21T07:42:21.263 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (Thread::entry_wrapper()+0x43) [0x7f2e41e2fd57] 2024-09-21T07:42:21.263 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: (Thread::_entry_func(void*)+0xd) [0x7f2e41e2fd73] 2024-09-21T07:42:21.263 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: /lib64/libc.so.6(+0x89c52) [0x7f2e41489c52] 2024-09-21T07:42:21.264 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: /lib64/libc.so.6(+0x10ec80) [0x7f2e4150ec80] 2024-09-21T07:42:21.264 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:42:21.264 INFO:tasks.ceph.mds.d.smithi148.stderr:*** Caught signal (Aborted) ** 2024-09-21T07:42:21.264 INFO:tasks.ceph.mds.d.smithi148.stderr: in thread 7f2e32e00640 thread_name:MR_Finisher 2024-09-21T07:42:21.264 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:42:21.264 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f2e4143e6f0] 2024-09-21T07:42:21.264 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f2e4148b99c] 2024-09-21T07:42:21.264 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: raise() 2024-09-21T07:42:21.264 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: abort() 2024-09-21T07:42:21.265 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7f2e41e56686] 2024-09-21T07:42:21.265 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f2e41e56758] 2024-09-21T07:42:21.265 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSTableClient::commit(unsigned long, LogSegment*)+0x333) [0x55a9810d23d3] 2024-09-21T07:42:21.265 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: (Server::_mksnap_finish(boost::intrusive_ptr const&, CInode*, SnapInfo&)+0x9f) [0x55a980e85ac3] 2024-09-21T07:42:21.265 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: ceph-mds(+0x221794) [0x55a980e7a794] 2024-09-21T07:42:21.265 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: (MDSContext::complete(int)+0x7c) [0x55a9811206d8] 2024-09-21T07:42:21.265 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (MDSIOContextBase::complete(int)+0x501) [0x55a981120d57] 2024-09-21T07:42:21.265 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: (MDSLogContextBase::complete(int)+0x46) [0x55a981120dd4] 2024-09-21T07:42:21.265 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: (Finisher::finisher_thread_entry()+0x495) [0x7f2e41dde34b] 2024-09-21T07:42:21.265 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: ceph-mds(+0x1052e1) [0x55a980d5e2e1] 2024-09-21T07:42:21.265 INFO:tasks.ceph.mds.d.smithi148.stderr: 15: (Thread::entry_wrapper()+0x43) [0x7f2e41e2fd57] 2024-09-21T07:42:21.265 INFO:tasks.ceph.mds.d.smithi148.stderr: 16: (Thread::_entry_func(void*)+0xd) [0x7f2e41e2fd73] 2024-09-21T07:42:21.265 INFO:tasks.ceph.mds.d.smithi148.stderr: 17: /lib64/libc.so.6(+0x89c52) [0x7f2e41489c52] 2024-09-21T07:42:21.265 INFO:tasks.ceph.mds.d.smithi148.stderr: 18: /lib64/libc.so.6(+0x10ec80) [0x7f2e4150ec80] 2024-09-21T07:42:21.265 INFO:tasks.ceph.mds.d.smithi148.stderr:2024-09-21T07:42:21.260+0000 7f2e32e00640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:42:21.266 INFO:tasks.ceph.mds.d.smithi148.stderr: in thread 7f2e32e00640 thread_name:MR_Finisher 2024-09-21T07:42:21.266 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:42:21.266 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:42:21.266 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f2e4143e6f0] 2024-09-21T07:42:21.266 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f2e4148b99c] 2024-09-21T07:42:21.266 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: raise() 2024-09-21T07:42:21.266 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: abort() 2024-09-21T07:42:21.267 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7f2e41e56686] 2024-09-21T07:42:21.267 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f2e41e56758] 2024-09-21T07:42:21.267 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSTableClient::commit(unsigned long, LogSegment*)+0x333) [0x55a9810d23d3] 2024-09-21T07:42:21.267 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: (Server::_mksnap_finish(boost::intrusive_ptr const&, CInode*, SnapInfo&)+0x9f) [0x55a980e85ac3] 2024-09-21T07:42:21.267 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: ceph-mds(+0x221794) [0x55a980e7a794] 2024-09-21T07:42:21.267 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: (MDSContext::complete(int)+0x7c) [0x55a9811206d8] 2024-09-21T07:42:21.267 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (MDSIOContextBase::complete(int)+0x501) [0x55a981120d57] 2024-09-21T07:42:21.268 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: (MDSLogContextBase::complete(int)+0x46) [0x55a981120dd4] 2024-09-21T07:42:21.268 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: (Finisher::finisher_thread_entry()+0x495) [0x7f2e41dde34b] 2024-09-21T07:42:21.268 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: ceph-mds(+0x1052e1) [0x55a980d5e2e1] 2024-09-21T07:42:21.268 INFO:tasks.ceph.mds.d.smithi148.stderr: 15: (Thread::entry_wrapper()+0x43) [0x7f2e41e2fd57] 2024-09-21T07:42:21.268 INFO:tasks.ceph.mds.d.smithi148.stderr: 16: (Thread::_entry_func(void*)+0xd) [0x7f2e41e2fd73] 2024-09-21T07:42:21.268 INFO:tasks.ceph.mds.d.smithi148.stderr: 17: /lib64/libc.so.6(+0x89c52) [0x7f2e41489c52] 2024-09-21T07:42:21.269 INFO:tasks.ceph.mds.d.smithi148.stderr: 18: /lib64/libc.so.6(+0x10ec80) [0x7f2e4150ec80] 2024-09-21T07:42:21.269 INFO:tasks.ceph.mds.d.smithi148.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:42:21.269 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:42:21.274 INFO:tasks.ceph.mds.d.smithi148.stderr: -1> 2024-09-21T07:42:21.259+0000 7f2e32e00640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::commit(version_t, LogSegment*)' thread 7f2e32e00640 time 2024-09-21T07:42:21.260292+0000 2024-09-21T07:42:21.275 INFO:tasks.ceph.mds.d.smithi148.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: 195: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 4) 2024-09-21T07:42:21.275 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:42:21.275 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:42:21.275 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f2e41e5654b] 2024-09-21T07:42:21.275 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f2e41e56758] 2024-09-21T07:42:21.275 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: (MDSTableClient::commit(unsigned long, LogSegment*)+0x333) [0x55a9810d23d3] 2024-09-21T07:42:21.275 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: (Server::_mksnap_finish(boost::intrusive_ptr const&, CInode*, SnapInfo&)+0x9f) [0x55a980e85ac3] 2024-09-21T07:42:21.276 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: ceph-mds(+0x221794) [0x55a980e7a794] 2024-09-21T07:42:21.276 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: (MDSContext::complete(int)+0x7c) [0x55a9811206d8] 2024-09-21T07:42:21.276 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSIOContextBase::complete(int)+0x501) [0x55a981120d57] 2024-09-21T07:42:21.276 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: (MDSLogContextBase::complete(int)+0x46) [0x55a981120dd4] 2024-09-21T07:42:21.276 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: (Finisher::finisher_thread_entry()+0x495) [0x7f2e41dde34b] 2024-09-21T07:42:21.276 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: ceph-mds(+0x1052e1) [0x55a980d5e2e1] 2024-09-21T07:42:21.277 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (Thread::entry_wrapper()+0x43) [0x7f2e41e2fd57] 2024-09-21T07:42:21.277 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: (Thread::_entry_func(void*)+0xd) [0x7f2e41e2fd73] 2024-09-21T07:42:21.277 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: /lib64/libc.so.6(+0x89c52) [0x7f2e41489c52] 2024-09-21T07:42:21.277 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: /lib64/libc.so.6(+0x10ec80) [0x7f2e4150ec80] 2024-09-21T07:42:21.277 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:42:21.277 INFO:tasks.ceph.mds.d.smithi148.stderr: 0> 2024-09-21T07:42:21.260+0000 7f2e32e00640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:42:21.277 INFO:tasks.ceph.mds.d.smithi148.stderr: in thread 7f2e32e00640 thread_name:MR_Finisher 2024-09-21T07:42:21.278 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:42:21.278 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:42:21.278 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f2e4143e6f0] 2024-09-21T07:42:21.278 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f2e4148b99c] 2024-09-21T07:42:21.278 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: raise() 2024-09-21T07:42:21.278 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: abort() 2024-09-21T07:42:21.279 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7f2e41e56686] 2024-09-21T07:42:21.279 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f2e41e56758] 2024-09-21T07:42:21.279 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSTableClient::commit(unsigned long, LogSegment*)+0x333) [0x55a9810d23d3] 2024-09-21T07:42:21.279 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: (Server::_mksnap_finish(boost::intrusive_ptr const&, CInode*, SnapInfo&)+0x9f) [0x55a980e85ac3] 2024-09-21T07:42:21.279 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: ceph-mds(+0x221794) [0x55a980e7a794] 2024-09-21T07:42:21.279 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: (MDSContext::complete(int)+0x7c) [0x55a9811206d8] 2024-09-21T07:42:21.279 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (MDSIOContextBase::complete(int)+0x501) [0x55a981120d57] 2024-09-21T07:42:21.280 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: (MDSLogContextBase::complete(int)+0x46) [0x55a981120dd4] 2024-09-21T07:42:21.280 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: (Finisher::finisher_thread_entry()+0x495) [0x7f2e41dde34b] 2024-09-21T07:42:21.280 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: ceph-mds(+0x1052e1) [0x55a980d5e2e1] 2024-09-21T07:42:21.280 INFO:tasks.ceph.mds.d.smithi148.stderr: 15: (Thread::entry_wrapper()+0x43) [0x7f2e41e2fd57] 2024-09-21T07:42:21.280 INFO:tasks.ceph.mds.d.smithi148.stderr: 16: (Thread::_entry_func(void*)+0xd) [0x7f2e41e2fd73] 2024-09-21T07:42:21.280 INFO:tasks.ceph.mds.d.smithi148.stderr: 17: /lib64/libc.so.6(+0x89c52) [0x7f2e41489c52] 2024-09-21T07:42:21.280 INFO:tasks.ceph.mds.d.smithi148.stderr: 18: /lib64/libc.so.6(+0x10ec80) [0x7f2e4150ec80] 2024-09-21T07:42:21.281 INFO:tasks.ceph.mds.d.smithi148.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:42:21.281 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:42:21.281 INFO:tasks.ceph.mds.d.smithi148.stderr: -1> 2024-09-21T07:42:21.259+0000 7f2e32e00640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::commit(version_t, LogSegment*)' thread 7f2e32e00640 time 2024-09-21T07:42:21.260292+0000 2024-09-21T07:42:21.281 INFO:tasks.ceph.mds.d.smithi148.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: 195: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 4) 2024-09-21T07:42:21.282 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:42:21.282 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:42:21.282 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f2e41e5654b] 2024-09-21T07:42:21.282 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f2e41e56758] 2024-09-21T07:42:21.282 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: (MDSTableClient::commit(unsigned long, LogSegment*)+0x333) [0x55a9810d23d3] 2024-09-21T07:42:21.282 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: (Server::_mksnap_finish(boost::intrusive_ptr const&, CInode*, SnapInfo&)+0x9f) [0x55a980e85ac3] 2024-09-21T07:42:21.283 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: ceph-mds(+0x221794) [0x55a980e7a794] 2024-09-21T07:42:21.283 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: (MDSContext::complete(int)+0x7c) [0x55a9811206d8] 2024-09-21T07:42:21.283 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSIOContextBase::complete(int)+0x501) [0x55a981120d57] 2024-09-21T07:42:21.283 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: (MDSLogContextBase::complete(int)+0x46) [0x55a981120dd4] 2024-09-21T07:42:21.283 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: (Finisher::finisher_thread_entry()+0x495) [0x7f2e41dde34b] 2024-09-21T07:42:21.283 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: ceph-mds(+0x1052e1) [0x55a980d5e2e1] 2024-09-21T07:42:21.284 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (Thread::entry_wrapper()+0x43) [0x7f2e41e2fd57] 2024-09-21T07:42:21.284 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: (Thread::_entry_func(void*)+0xd) [0x7f2e41e2fd73] 2024-09-21T07:42:21.284 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: /lib64/libc.so.6(+0x89c52) [0x7f2e41489c52] 2024-09-21T07:42:21.284 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: /lib64/libc.so.6(+0x10ec80) [0x7f2e4150ec80] 2024-09-21T07:42:21.284 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:42:21.284 INFO:tasks.ceph.mds.d.smithi148.stderr: 0> 2024-09-21T07:42:21.260+0000 7f2e32e00640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:42:21.284 INFO:tasks.ceph.mds.d.smithi148.stderr: in thread 7f2e32e00640 thread_name:MR_Finisher 2024-09-21T07:42:21.285 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:42:21.285 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:42:21.285 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f2e4143e6f0] 2024-09-21T07:42:21.285 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f2e4148b99c] 2024-09-21T07:42:21.285 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: raise() 2024-09-21T07:42:21.285 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: abort() 2024-09-21T07:42:21.286 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7f2e41e56686] 2024-09-21T07:42:21.286 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f2e41e56758] 2024-09-21T07:42:21.286 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSTableClient::commit(unsigned long, LogSegment*)+0x333) [0x55a9810d23d3] 2024-09-21T07:42:21.286 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: (Server::_mksnap_finish(boost::intrusive_ptr const&, CInode*, SnapInfo&)+0x9f) [0x55a980e85ac3] 2024-09-21T07:42:21.286 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: ceph-mds(+0x221794) [0x55a980e7a794] 2024-09-21T07:42:21.286 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: (MDSContext::complete(int)+0x7c) [0x55a9811206d8] 2024-09-21T07:42:21.287 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (MDSIOContextBase::complete(int)+0x501) [0x55a981120d57] 2024-09-21T07:42:21.287 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: (MDSLogContextBase::complete(int)+0x46) [0x55a981120dd4] 2024-09-21T07:42:21.287 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: (Finisher::finisher_thread_entry()+0x495) [0x7f2e41dde34b] 2024-09-21T07:42:21.287 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: ceph-mds(+0x1052e1) [0x55a980d5e2e1] 2024-09-21T07:42:21.287 INFO:tasks.ceph.mds.d.smithi148.stderr: 15: (Thread::entry_wrapper()+0x43) [0x7f2e41e2fd57] 2024-09-21T07:42:21.287 INFO:tasks.ceph.mds.d.smithi148.stderr: 16: (Thread::_entry_func(void*)+0xd) [0x7f2e41e2fd73] 2024-09-21T07:42:21.287 INFO:tasks.ceph.mds.d.smithi148.stderr: 17: /lib64/libc.so.6(+0x89c52) [0x7f2e41489c52] 2024-09-21T07:42:21.288 INFO:tasks.ceph.mds.d.smithi148.stderr: 18: /lib64/libc.so.6(+0x10ec80) [0x7f2e4150ec80] 2024-09-21T07:42:21.288 INFO:tasks.ceph.mds.d.smithi148.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:42:21.288 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:42:21.456 INFO:teuthology.orchestra.run.smithi039.stdout:{"mds_beacon_grace":"15.000000"} 2024-09-21T07:42:21.456 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-09-21T07:42:21.456 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:42:21.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.685+0000 7fa8c6400640 1 -- 172.21.15.39:0/2115897879 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8c00c4880 msgr2=0x7fa8c0160bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:21.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.685+0000 7fa8c6400640 1 --2- 172.21.15.39:0/2115897879 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8c00c4880 0x7fa8c0160bf0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0160620 tx=0x7fa8ac02a510 comp rx=0 tx=0).stop 2024-09-21T07:42:21.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.685+0000 7fa8c6400640 1 -- 172.21.15.39:0/2115897879 shutdown_connections 2024-09-21T07:42:21.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.685+0000 7fa8c6400640 1 --2- 172.21.15.39:0/2115897879 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8c00c4880 0x7fa8c0160bf0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:21.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.685+0000 7fa8c6400640 1 --2- 172.21.15.39:0/2115897879 >> v2:172.21.15.148:3300/0 conn(0x7fa8c00cd770 0x7fa8c015c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:21.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.686+0000 7fa8c6400640 1 -- 172.21.15.39:0/2115897879 >> 172.21.15.39:0/2115897879 conn(0x7fa8c00b2d40 msgr2=0x7fa8c00b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:21.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.686+0000 7fa8c6400640 1 -- 172.21.15.39:0/2115897879 shutdown_connections 2024-09-21T07:42:21.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.686+0000 7fa8c6400640 1 -- 172.21.15.39:0/2115897879 wait complete. 2024-09-21T07:42:21.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.688+0000 7fa8c6400640 1 Processor -- start 2024-09-21T07:42:21.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.688+0000 7fa8c6400640 1 -- start start 2024-09-21T07:42:21.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.689+0000 7fa8c6400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8c00c4880 0x7fa8c014ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:21.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.689+0000 7fa8c6400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8c00cd770 0x7fa8c00d0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:21.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.689+0000 7fa8c6400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8c0161130 0x7fa8c01f9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:21.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.689+0000 7fa8c6400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c00b4040 con 0x7fa8c00cd770 2024-09-21T07:42:21.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.689+0000 7fa8c6400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c006d740 con 0x7fa8c0161130 2024-09-21T07:42:21.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.689+0000 7fa8c6400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa8c00b41c0 con 0x7fa8c00c4880 2024-09-21T07:42:21.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.690+0000 7fa8bfe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8c00cd770 0x7fa8c00d0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:21.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.690+0000 7fa8bfe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8c00cd770 0x7fa8c00d0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53750/0 (socket says 172.21.15.39:53750) 2024-09-21T07:42:21.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.690+0000 7fa8bfe00640 1 -- 172.21.15.39:0/2324375715 learned_addr learned my addr 172.21.15.39:0/2324375715 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:21.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.690+0000 7fa8c5000640 1 --2- 172.21.15.39:0/2324375715 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8c00c4880 0x7fa8c014ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:21.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.690+0000 7fa8c5a00640 1 --2- 172.21.15.39:0/2324375715 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8c0161130 0x7fa8c01f9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:21.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.690+0000 7fa8bfe00640 1 -- 172.21.15.39:0/2324375715 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8c00c4880 msgr2=0x7fa8c014ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:21.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.690+0000 7fa8bfe00640 1 --2- 172.21.15.39:0/2324375715 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8c00c4880 0x7fa8c014ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:21.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.690+0000 7fa8bfe00640 1 -- 172.21.15.39:0/2324375715 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8c0161130 msgr2=0x7fa8c01f9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:21.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.690+0000 7fa8bfe00640 1 --2- 172.21.15.39:0/2324375715 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8c0161130 0x7fa8c01f9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:21.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.690+0000 7fa8bfe00640 1 -- 172.21.15.39:0/2324375715 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c015ec10 con 0x7fa8c00cd770 2024-09-21T07:42:21.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.690+0000 7fa8bfe00640 1 --2- 172.21.15.39:0/2324375715 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8c00cd770 0x7fa8c00d0710 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac02ab00 tx=0x7fa8ac033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:21.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.691+0000 7fa8bcc00640 1 -- 172.21.15.39:0/2324375715 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac005c90 con 0x7fa8c00cd770 2024-09-21T07:42:21.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.691+0000 7fa8bcc00640 1 -- 172.21.15.39:0/2324375715 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8ac031e50 con 0x7fa8c00cd770 2024-09-21T07:42:21.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.691+0000 7fa8c6400640 1 -- 172.21.15.39:0/2324375715 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8c00cfc30 con 0x7fa8c00cd770 2024-09-21T07:42:21.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.691+0000 7fa8bcc00640 1 -- 172.21.15.39:0/2324375715 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac005260 con 0x7fa8c00cd770 2024-09-21T07:42:21.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.691+0000 7fa8c6400640 1 -- 172.21.15.39:0/2324375715 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8c015eef0 con 0x7fa8c00cd770 2024-09-21T07:42:21.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.692+0000 7fa8c6400640 1 -- 172.21.15.39:0/2324375715 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8c0160bf0 con 0x7fa8c00cd770 2024-09-21T07:42:21.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.693+0000 7fa8bcc00640 1 -- 172.21.15.39:0/2324375715 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa8ac005400 con 0x7fa8c00cd770 2024-09-21T07:42:21.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.695+0000 7fa8bcc00640 1 --2- 172.21.15.39:0/2324375715 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa8a0078b90 0x7fa8a007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:21.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.695+0000 7fa8bcc00640 1 -- 172.21.15.39:0/2324375715 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7947+0+0 (secure 0 0 0) 0x7fa8ac0bc770 con 0x7fa8c00cd770 2024-09-21T07:42:21.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.695+0000 7fa8c5000640 1 --2- 172.21.15.39:0/2324375715 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa8a0078b90 0x7fa8a007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:21.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.696+0000 7fa8c5000640 1 --2- 172.21.15.39:0/2324375715 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa8a0078b90 0x7fa8a007b0a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4009530 tx=0x7fa8b4009260 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:21.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.700+0000 7fa8bcc00640 1 -- 172.21.15.39:0/2324375715 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa8ac03f030 con 0x7fa8c00cd770 2024-09-21T07:42:21.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.939+0000 7fa8c6400640 1 -- 172.21.15.39:0/2324375715 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8c015e9a0 con 0x7fa8c00cd770 2024-09-21T07:42:21.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.940+0000 7fa8bcc00640 1 -- 172.21.15.39:0/2324375715 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 135 v135) ==== 95+0+5253 (secure 0 0 0) 0x7fa8ac031770 con 0x7fa8c00cd770 2024-09-21T07:42:21.942 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:42:21.942 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":135,"btime":"2024-09-21T07:42:20:941595+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7644,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1326711417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1326711417},{"type":"v1","addr":"172.21.15.39:6835","nonce":1326711417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":131}],"filesystems":[{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:42:20.586480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":6425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6425":{"gid":6425,"name":"d","rank":1,"incarnation":130,"state":"up:active","state_seq":18,"addr":"172.21.15.148:6837/73220122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":73220122},{"type":"v1","addr":"172.21.15.148:6837","nonce":73220122}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311,6425]},"id":6}]} 2024-09-21T07:42:21.942 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 135 2024-09-21T07:42:21.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.942+0000 7fa8c6400640 1 -- 172.21.15.39:0/2324375715 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa8a0078b90 msgr2=0x7fa8a007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:21.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.942+0000 7fa8c6400640 1 --2- 172.21.15.39:0/2324375715 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa8a0078b90 0x7fa8a007b0a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4009530 tx=0x7fa8b4009260 comp rx=0 tx=0).stop 2024-09-21T07:42:21.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.942+0000 7fa8c6400640 1 -- 172.21.15.39:0/2324375715 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8c00cd770 msgr2=0x7fa8c00d0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:21.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.942+0000 7fa8c6400640 1 --2- 172.21.15.39:0/2324375715 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8c00cd770 0x7fa8c00d0710 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac02ab00 tx=0x7fa8ac033c20 comp rx=0 tx=0).stop 2024-09-21T07:42:21.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.942+0000 7fa8c6400640 1 -- 172.21.15.39:0/2324375715 shutdown_connections 2024-09-21T07:42:21.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.942+0000 7fa8c6400640 1 --2- 172.21.15.39:0/2324375715 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa8a0078b90 0x7fa8a007b0a0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:21.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.942+0000 7fa8c6400640 1 --2- 172.21.15.39:0/2324375715 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8c0161130 0x7fa8c01f9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:21.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.942+0000 7fa8c6400640 1 --2- 172.21.15.39:0/2324375715 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8c00cd770 0x7fa8c00d0710 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:21.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.942+0000 7fa8c6400640 1 --2- 172.21.15.39:0/2324375715 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8c00c4880 0x7fa8c014ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:21.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.942+0000 7fa8c6400640 1 -- 172.21.15.39:0/2324375715 >> 172.21.15.39:0/2324375715 conn(0x7fa8c00b2d40 msgr2=0x7fa8c015b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:21.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.942+0000 7fa8c6400640 1 -- 172.21.15.39:0/2324375715 shutdown_connections 2024-09-21T07:42:21.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:21.942+0000 7fa8c6400640 1 -- 172.21.15.39:0/2324375715 wait complete. 2024-09-21T07:42:21.968 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:42:22.035 INFO:tasks.ceph.mds.d.smithi148.stderr:daemon-helper: command crashed with signal 6 2024-09-21T07:42:22.724 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-09-21T07:42:26.969 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:42:27.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.206+0000 7feea3a00640 1 -- 172.21.15.39:0/2112084203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee9c15f360 msgr2=0x7fee9c15bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:27.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.206+0000 7feea3a00640 1 --2- 172.21.15.39:0/2112084203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee9c15f360 0x7fee9c15bd70 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fee90009830 tx=0x7fee9002a510 comp rx=0 tx=0).stop 2024-09-21T07:42:27.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.207+0000 7feea3a00640 1 -- 172.21.15.39:0/2112084203 shutdown_connections 2024-09-21T07:42:27.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.207+0000 7feea3a00640 1 --2- 172.21.15.39:0/2112084203 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee9c155e80 0x7fee9c0b4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:27.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.207+0000 7feea3a00640 1 --2- 172.21.15.39:0/2112084203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee9c15f360 0x7fee9c15bd70 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:27.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.207+0000 7feea3a00640 1 -- 172.21.15.39:0/2112084203 >> 172.21.15.39:0/2112084203 conn(0x7fee9c06cfc0 msgr2=0x7fee9c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:27.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.207+0000 7feea3a00640 1 -- 172.21.15.39:0/2112084203 shutdown_connections 2024-09-21T07:42:27.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.207+0000 7feea3a00640 1 -- 172.21.15.39:0/2112084203 wait complete. 2024-09-21T07:42:27.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.209+0000 7feea3a00640 1 Processor -- start 2024-09-21T07:42:27.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.209+0000 7feea3a00640 1 -- start start 2024-09-21T07:42:27.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.211+0000 7feea3a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee9c155a70 0x7fee9c0aec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:27.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.211+0000 7feea3a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee9c155e80 0x7fee9c1537d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:27.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.211+0000 7feea3a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee9c15f360 0x7fee9c153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:27.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.211+0000 7feea3a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fee9c0d0620 con 0x7fee9c15f360 2024-09-21T07:42:27.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.211+0000 7feea3a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fee9c0b4ef0 con 0x7fee9c155a70 2024-09-21T07:42:27.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.211+0000 7feea3a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fee9c0d07a0 con 0x7fee9c155e80 2024-09-21T07:42:27.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.211+0000 7feea1a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee9c15f360 0x7fee9c153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:27.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.211+0000 7feea1a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee9c15f360 0x7fee9c153d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38290/0 (socket says 172.21.15.39:38290) 2024-09-21T07:42:27.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.211+0000 7feea1a00640 1 -- 172.21.15.39:0/3340307506 learned_addr learned my addr 172.21.15.39:0/3340307506 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:27.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.211+0000 7fee9be00640 1 --2- 172.21.15.39:0/3340307506 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee9c155e80 0x7fee9c1537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:27.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.211+0000 7feea1000640 1 --2- 172.21.15.39:0/3340307506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee9c155a70 0x7fee9c0aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:27.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.211+0000 7feea1a00640 1 -- 172.21.15.39:0/3340307506 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee9c155e80 msgr2=0x7fee9c1537d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:27.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.211+0000 7feea1a00640 1 --2- 172.21.15.39:0/3340307506 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee9c155e80 0x7fee9c1537d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:27.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.211+0000 7feea1a00640 1 -- 172.21.15.39:0/3340307506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee9c155a70 msgr2=0x7fee9c0aec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:27.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.211+0000 7feea1a00640 1 --2- 172.21.15.39:0/3340307506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee9c155a70 0x7fee9c0aec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:27.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.211+0000 7feea1a00640 1 -- 172.21.15.39:0/3340307506 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee9c1f5870 con 0x7fee9c15f360 2024-09-21T07:42:27.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.212+0000 7feea1a00640 1 --2- 172.21.15.39:0/3340307506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee9c15f360 0x7fee9c153d10 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fee9400cad0 tx=0x7fee9400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:27.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.212+0000 7fee98c00640 1 -- 172.21.15.39:0/3340307506 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9400ae20 con 0x7fee9c15f360 2024-09-21T07:42:27.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.212+0000 7fee98c00640 1 -- 172.21.15.39:0/3340307506 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee9400f450 con 0x7fee9c15f360 2024-09-21T07:42:27.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.212+0000 7fee98c00640 1 -- 172.21.15.39:0/3340307506 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9400f8c0 con 0x7fee9c15f360 2024-09-21T07:42:27.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.212+0000 7feea3a00640 1 -- 172.21.15.39:0/3340307506 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee9c1f9330 con 0x7fee9c15f360 2024-09-21T07:42:27.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.212+0000 7feea3a00640 1 -- 172.21.15.39:0/3340307506 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee9c1f98f0 con 0x7fee9c15f360 2024-09-21T07:42:27.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.213+0000 7feea3a00640 1 -- 172.21.15.39:0/3340307506 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee9c0d0cc0 con 0x7fee9c15f360 2024-09-21T07:42:27.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.215+0000 7fee98c00640 1 -- 172.21.15.39:0/3340307506 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fee9400fa60 con 0x7fee9c15f360 2024-09-21T07:42:27.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.216+0000 7fee98c00640 1 --2- 172.21.15.39:0/3340307506 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fee7c078b90 0x7fee7c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:27.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.216+0000 7fee98c00640 1 -- 172.21.15.39:0/3340307506 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7947+0+0 (secure 0 0 0) 0x7fee9409c0a0 con 0x7fee9c15f360 2024-09-21T07:42:27.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.216+0000 7feea1000640 1 --2- 172.21.15.39:0/3340307506 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fee7c078b90 0x7fee7c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:27.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.217+0000 7feea1000640 1 --2- 172.21.15.39:0/3340307506 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fee7c078b90 0x7fee7c07b0a0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fee90034000 tx=0x7fee90004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:27.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.220+0000 7fee98c00640 1 -- 172.21.15.39:0/3340307506 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fee94014030 con 0x7fee9c15f360 2024-09-21T07:42:27.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.456+0000 7feea3a00640 1 -- 172.21.15.39:0/3340307506 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee9c0b5420 con 0x7fee9c15f360 2024-09-21T07:42:27.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.457+0000 7fee98c00640 1 -- 172.21.15.39:0/3340307506 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 135 v135) ==== 95+0+5253 (secure 0 0 0) 0x7fee94063530 con 0x7fee9c15f360 2024-09-21T07:42:27.459 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:42:27.459 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":135,"btime":"2024-09-21T07:42:20:941595+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7644,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1326711417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1326711417},{"type":"v1","addr":"172.21.15.39:6835","nonce":1326711417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":131}],"filesystems":[{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:42:20.586480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":6425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6425":{"gid":6425,"name":"d","rank":1,"incarnation":130,"state":"up:active","state_seq":18,"addr":"172.21.15.148:6837/73220122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":73220122},{"type":"v1","addr":"172.21.15.148:6837","nonce":73220122}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311,6425]},"id":6}]} 2024-09-21T07:42:27.459 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 135 2024-09-21T07:42:27.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.459+0000 7feea3a00640 1 -- 172.21.15.39:0/3340307506 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fee7c078b90 msgr2=0x7fee7c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:27.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.459+0000 7feea3a00640 1 --2- 172.21.15.39:0/3340307506 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fee7c078b90 0x7fee7c07b0a0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fee90034000 tx=0x7fee90004730 comp rx=0 tx=0).stop 2024-09-21T07:42:27.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.459+0000 7feea3a00640 1 -- 172.21.15.39:0/3340307506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee9c15f360 msgr2=0x7fee9c153d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:27.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.459+0000 7feea3a00640 1 --2- 172.21.15.39:0/3340307506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee9c15f360 0x7fee9c153d10 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fee9400cad0 tx=0x7fee9400cf90 comp rx=0 tx=0).stop 2024-09-21T07:42:27.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.459+0000 7feea3a00640 1 -- 172.21.15.39:0/3340307506 shutdown_connections 2024-09-21T07:42:27.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.460+0000 7feea3a00640 1 --2- 172.21.15.39:0/3340307506 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fee7c078b90 0x7fee7c07b0a0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:27.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.460+0000 7feea3a00640 1 --2- 172.21.15.39:0/3340307506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee9c15f360 0x7fee9c153d10 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:27.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.460+0000 7feea3a00640 1 --2- 172.21.15.39:0/3340307506 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee9c155e80 0x7fee9c1537d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:27.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.460+0000 7feea3a00640 1 --2- 172.21.15.39:0/3340307506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee9c155a70 0x7fee9c0aec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:27.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.460+0000 7feea3a00640 1 -- 172.21.15.39:0/3340307506 >> 172.21.15.39:0/3340307506 conn(0x7fee9c06cfc0 msgr2=0x7fee9c068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:27.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.460+0000 7feea3a00640 1 -- 172.21.15.39:0/3340307506 shutdown_connections 2024-09-21T07:42:27.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:27.460+0000 7feea3a00640 1 -- 172.21.15.39:0/3340307506 wait complete. 2024-09-21T07:42:27.487 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:42:29.336 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~7s 2024-09-21T07:42:31.672 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:42:31.677 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:42:32.488 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:42:32.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.725+0000 7fd9ba200640 1 -- 172.21.15.39:0/3839952525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9b4155a70 msgr2=0x7fd9b40b39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:32.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.725+0000 7fd9ba200640 1 --2- 172.21.15.39:0/3839952525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9b4155a70 0x7fd9b40b39f0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fd9b41598b0 tx=0x7fd99c02a560 comp rx=0 tx=0).stop 2024-09-21T07:42:32.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.725+0000 7fd9ba200640 1 -- 172.21.15.39:0/3839952525 shutdown_connections 2024-09-21T07:42:32.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.725+0000 7fd9ba200640 1 --2- 172.21.15.39:0/3839952525 >> v2:172.21.15.148:3300/0 conn(0x7fd9b40b3f30 0x7fd9b40b4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:32.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.725+0000 7fd9ba200640 1 --2- 172.21.15.39:0/3839952525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9b4155a70 0x7fd9b40b39f0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:32.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.725+0000 7fd9ba200640 1 --2- 172.21.15.39:0/3839952525 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd9b415f360 0x7fd9b415bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:32.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.725+0000 7fd9ba200640 1 -- 172.21.15.39:0/3839952525 >> 172.21.15.39:0/3839952525 conn(0x7fd9b406cfc0 msgr2=0x7fd9b406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:32.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.725+0000 7fd9ba200640 1 -- 172.21.15.39:0/3839952525 shutdown_connections 2024-09-21T07:42:32.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.725+0000 7fd9ba200640 1 -- 172.21.15.39:0/3839952525 wait complete. 2024-09-21T07:42:32.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.727+0000 7fd9ba200640 1 Processor -- start 2024-09-21T07:42:32.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.727+0000 7fd9ba200640 1 -- start start 2024-09-21T07:42:32.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.729+0000 7fd9ba200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd9b40b3f30 0x7fd9b414de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:32.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.729+0000 7fd9ba200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9b4155a70 0x7fd9b40cf380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:32.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.729+0000 7fd9ba200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9b415f360 0x7fd9b40cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:32.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.729+0000 7fd9ba200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b40b4f60 con 0x7fd9b415f360 2024-09-21T07:42:32.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.729+0000 7fd9ba200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b40b4de0 con 0x7fd9b4155a70 2024-09-21T07:42:32.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.729+0000 7fd9ba200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd9b40d0620 con 0x7fd9b40b3f30 2024-09-21T07:42:32.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.729+0000 7fd9b3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9b415f360 0x7fd9b40cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:32.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.729+0000 7fd9b3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9b415f360 0x7fd9b40cf8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45086/0 (socket says 172.21.15.39:45086) 2024-09-21T07:42:32.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.729+0000 7fd9b3e00640 1 -- 172.21.15.39:0/1667674018 learned_addr learned my addr 172.21.15.39:0/1667674018 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:32.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.729+0000 7fd9b3400640 1 --2- 172.21.15.39:0/1667674018 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd9b40b3f30 0x7fd9b414de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:32.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.729+0000 7fd9abe00640 1 --2- 172.21.15.39:0/1667674018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9b4155a70 0x7fd9b40cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:32.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.729+0000 7fd9b3e00640 1 -- 172.21.15.39:0/1667674018 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd9b40b3f30 msgr2=0x7fd9b414de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:32.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.729+0000 7fd9b3e00640 1 --2- 172.21.15.39:0/1667674018 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd9b40b3f30 0x7fd9b414de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:32.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.729+0000 7fd9b3e00640 1 -- 172.21.15.39:0/1667674018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9b4155a70 msgr2=0x7fd9b40cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:32.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.729+0000 7fd9b3e00640 1 --2- 172.21.15.39:0/1667674018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9b4155a70 0x7fd9b40cf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:32.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.730+0000 7fd9b3e00640 1 -- 172.21.15.39:0/1667674018 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9b4153150 con 0x7fd9b415f360 2024-09-21T07:42:32.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.730+0000 7fd9b3e00640 1 --2- 172.21.15.39:0/1667674018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9b415f360 0x7fd9b40cf8c0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac00caa0 tx=0x7fd9ac00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:32.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.730+0000 7fd9ab400640 1 -- 172.21.15.39:0/1667674018 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac00adf0 con 0x7fd9b415f360 2024-09-21T07:42:32.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.730+0000 7fd9ab400640 1 -- 172.21.15.39:0/1667674018 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9ac00f540 con 0x7fd9b415f360 2024-09-21T07:42:32.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.730+0000 7fd9ba200640 1 -- 172.21.15.39:0/1667674018 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9b4153430 con 0x7fd9b415f360 2024-09-21T07:42:32.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.730+0000 7fd9ab400640 1 -- 172.21.15.39:0/1667674018 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac00f990 con 0x7fd9b415f360 2024-09-21T07:42:32.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.730+0000 7fd9ba200640 1 -- 172.21.15.39:0/1667674018 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9b41539f0 con 0x7fd9b415f360 2024-09-21T07:42:32.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.731+0000 7fd9ba200640 1 -- 172.21.15.39:0/1667674018 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9b40b39f0 con 0x7fd9b415f360 2024-09-21T07:42:32.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.732+0000 7fd9ab400640 1 -- 172.21.15.39:0/1667674018 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd9ac00fb30 con 0x7fd9b415f360 2024-09-21T07:42:32.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.733+0000 7fd9ab400640 1 --2- 172.21.15.39:0/1667674018 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd990078bc0 0x7fd99007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:32.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.734+0000 7fd9ab400640 1 -- 172.21.15.39:0/1667674018 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7947+0+0 (secure 0 0 0) 0x7fd9ac09c080 con 0x7fd9b415f360 2024-09-21T07:42:32.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.734+0000 7fd9b3400640 1 --2- 172.21.15.39:0/1667674018 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd990078bc0 0x7fd99007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:32.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.735+0000 7fd9b3400640 1 --2- 172.21.15.39:0/1667674018 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd990078bc0 0x7fd99007b0d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd9a40045b0 tx=0x7fd9a400b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:32.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.738+0000 7fd9ab400640 1 -- 172.21.15.39:0/1667674018 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd9ac014030 con 0x7fd9b415f360 2024-09-21T07:42:32.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.971+0000 7fd9ba200640 1 -- 172.21.15.39:0/1667674018 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd9b40708d0 con 0x7fd9b415f360 2024-09-21T07:42:32.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.972+0000 7fd9ab400640 1 -- 172.21.15.39:0/1667674018 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 135 v135) ==== 95+0+5253 (secure 0 0 0) 0x7fd9ac00fe20 con 0x7fd9b415f360 2024-09-21T07:42:32.974 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:42:32.974 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":135,"btime":"2024-09-21T07:42:20:941595+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7644,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1326711417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1326711417},{"type":"v1","addr":"172.21.15.39:6835","nonce":1326711417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":131}],"filesystems":[{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:42:20.586480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":6425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6425":{"gid":6425,"name":"d","rank":1,"incarnation":130,"state":"up:active","state_seq":18,"addr":"172.21.15.148:6837/73220122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":73220122},{"type":"v1","addr":"172.21.15.148:6837","nonce":73220122}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311,6425]},"id":6}]} 2024-09-21T07:42:32.974 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 135 2024-09-21T07:42:32.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.974+0000 7fd9ba200640 1 -- 172.21.15.39:0/1667674018 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd990078bc0 msgr2=0x7fd99007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:32.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.974+0000 7fd9ba200640 1 --2- 172.21.15.39:0/1667674018 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd990078bc0 0x7fd99007b0d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd9a40045b0 tx=0x7fd9a400b2e0 comp rx=0 tx=0).stop 2024-09-21T07:42:32.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.974+0000 7fd9ba200640 1 -- 172.21.15.39:0/1667674018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9b415f360 msgr2=0x7fd9b40cf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:32.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.974+0000 7fd9ba200640 1 --2- 172.21.15.39:0/1667674018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9b415f360 0x7fd9b40cf8c0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac00caa0 tx=0x7fd9ac00cf60 comp rx=0 tx=0).stop 2024-09-21T07:42:32.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.974+0000 7fd9ba200640 1 -- 172.21.15.39:0/1667674018 shutdown_connections 2024-09-21T07:42:32.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.974+0000 7fd9ba200640 1 --2- 172.21.15.39:0/1667674018 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd990078bc0 0x7fd99007b0d0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:32.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.974+0000 7fd9ba200640 1 --2- 172.21.15.39:0/1667674018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9b415f360 0x7fd9b40cf8c0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:32.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.974+0000 7fd9ba200640 1 --2- 172.21.15.39:0/1667674018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9b4155a70 0x7fd9b40cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:32.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.974+0000 7fd9ba200640 1 --2- 172.21.15.39:0/1667674018 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd9b40b3f30 0x7fd9b414de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:32.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.974+0000 7fd9ba200640 1 -- 172.21.15.39:0/1667674018 >> 172.21.15.39:0/1667674018 conn(0x7fd9b406cfc0 msgr2=0x7fd9b4068f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:32.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.974+0000 7fd9ba200640 1 -- 172.21.15.39:0/1667674018 shutdown_connections 2024-09-21T07:42:32.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:32.975+0000 7fd9ba200640 1 -- 172.21.15.39:0/1667674018 wait complete. 2024-09-21T07:42:33.002 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:42:35.948 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~13s 2024-09-21T07:42:38.003 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:42:38.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.239+0000 7f5c02800640 1 -- 172.21.15.39:0/244353393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bfc15fe20 msgr2=0x7f5bfc09af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:38.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.239+0000 7f5c02800640 1 --2- 172.21.15.39:0/244353393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bfc15fe20 0x7f5bfc09af60 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc0cfde0 tx=0x7f5be802a440 comp rx=0 tx=0).stop 2024-09-21T07:42:38.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.240+0000 7f5c02800640 1 -- 172.21.15.39:0/244353393 shutdown_connections 2024-09-21T07:42:38.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.240+0000 7f5c02800640 1 --2- 172.21.15.39:0/244353393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bfc15fe20 0x7f5bfc09af60 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:38.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.240+0000 7f5c02800640 1 --2- 172.21.15.39:0/244353393 >> v2:172.21.15.148:3300/0 conn(0x7f5bfc09ba60 0x7f5bfc09aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:38.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.240+0000 7f5c02800640 1 -- 172.21.15.39:0/244353393 >> 172.21.15.39:0/244353393 conn(0x7f5bfc0b3e40 msgr2=0x7f5bfc0b4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:38.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.240+0000 7f5c02800640 1 -- 172.21.15.39:0/244353393 shutdown_connections 2024-09-21T07:42:38.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.240+0000 7f5c02800640 1 -- 172.21.15.39:0/244353393 wait complete. 2024-09-21T07:42:38.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.242+0000 7f5c02800640 1 Processor -- start 2024-09-21T07:42:38.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.242+0000 7f5c02800640 1 -- start start 2024-09-21T07:42:38.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.244+0000 7f5c02800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bfc09ba60 0x7f5bfc1f68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:38.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.244+0000 7f5c02800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5bfc15fe20 0x7f5bfc15b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:38.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.244+0000 7f5c02800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bfc160360 0x7f5bfc06e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:38.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.244+0000 7f5c02800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5bfc0cdfb0 con 0x7f5bfc09ba60 2024-09-21T07:42:38.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.244+0000 7f5c02800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5bfc0cec00 con 0x7f5bfc160360 2024-09-21T07:42:38.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.244+0000 7f5c02800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f5bfc0ce130 con 0x7f5bfc15fe20 2024-09-21T07:42:38.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.244+0000 7f5c01400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bfc09ba60 0x7f5bfc1f68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:38.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.244+0000 7f5c01400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bfc09ba60 0x7f5bfc1f68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45098/0 (socket says 172.21.15.39:45098) 2024-09-21T07:42:38.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.244+0000 7f5c01400640 1 -- 172.21.15.39:0/3058758436 learned_addr learned my addr 172.21.15.39:0/3058758436 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:38.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.244+0000 7f5c01e00640 1 --2- 172.21.15.39:0/3058758436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bfc160360 0x7f5bfc06e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:38.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.244+0000 7f5c00a00640 1 --2- 172.21.15.39:0/3058758436 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5bfc15fe20 0x7f5bfc15b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:38.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.244+0000 7f5c01400640 1 -- 172.21.15.39:0/3058758436 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5bfc15fe20 msgr2=0x7f5bfc15b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:38.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.244+0000 7f5c01400640 1 --2- 172.21.15.39:0/3058758436 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5bfc15fe20 0x7f5bfc15b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:38.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.244+0000 7f5c01400640 1 -- 172.21.15.39:0/3058758436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bfc160360 msgr2=0x7f5bfc06e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:38.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.244+0000 7f5c01400640 1 --2- 172.21.15.39:0/3058758436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bfc160360 0x7f5bfc06e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:38.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.244+0000 7f5c01400640 1 -- 172.21.15.39:0/3058758436 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bfc0b3550 con 0x7f5bfc09ba60 2024-09-21T07:42:38.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.245+0000 7f5c01400640 1 --2- 172.21.15.39:0/3058758436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bfc09ba60 0x7f5bfc1f68e0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f5bf000b420 tx=0x7f5bf000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:38.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.245+0000 7f5bf9600640 1 -- 172.21.15.39:0/3058758436 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf0005970 con 0x7f5bfc09ba60 2024-09-21T07:42:38.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.245+0000 7f5bf9600640 1 -- 172.21.15.39:0/3058758436 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5bf0005c00 con 0x7f5bfc09ba60 2024-09-21T07:42:38.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.245+0000 7f5c02800640 1 -- 172.21.15.39:0/3058758436 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bfc1f95b0 con 0x7f5bfc09ba60 2024-09-21T07:42:38.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.245+0000 7f5bf9600640 1 -- 172.21.15.39:0/3058758436 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf0006020 con 0x7f5bfc09ba60 2024-09-21T07:42:38.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.245+0000 7f5c02800640 1 -- 172.21.15.39:0/3058758436 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bfc15b9e0 con 0x7f5bfc09ba60 2024-09-21T07:42:38.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.246+0000 7f5c02800640 1 -- 172.21.15.39:0/3058758436 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bfc161010 con 0x7f5bfc09ba60 2024-09-21T07:42:38.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.247+0000 7f5bf9600640 1 -- 172.21.15.39:0/3058758436 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5bf0018020 con 0x7f5bfc09ba60 2024-09-21T07:42:38.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.249+0000 7f5bf9600640 1 --2- 172.21.15.39:0/3058758436 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5bdc078b90 0x7f5bdc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:38.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.249+0000 7f5bf9600640 1 -- 172.21.15.39:0/3058758436 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7947+0+0 (secure 0 0 0) 0x7f5bf009ad10 con 0x7f5bfc09ba60 2024-09-21T07:42:38.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.249+0000 7f5c00a00640 1 --2- 172.21.15.39:0/3058758436 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5bdc078b90 0x7f5bdc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:38.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.250+0000 7f5c00a00640 1 --2- 172.21.15.39:0/3058758436 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5bdc078b90 0x7f5bdc07b0a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5be8033000 tx=0x7f5be8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:38.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.254+0000 7f5bf9600640 1 -- 172.21.15.39:0/3058758436 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5bf00a0020 con 0x7f5bfc09ba60 2024-09-21T07:42:38.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.487+0000 7f5c02800640 1 -- 172.21.15.39:0/3058758436 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5bfc0cc240 con 0x7f5bfc09ba60 2024-09-21T07:42:38.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.488+0000 7f5bf9600640 1 -- 172.21.15.39:0/3058758436 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 136 v136) ==== 95+0+5301 (secure 0 0 0) 0x7f5bf00627f0 con 0x7f5bfc09ba60 2024-09-21T07:42:38.489 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:42:38.489 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":136,"btime":"2024-09-21T07:42:36:446339+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7644,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1326711417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1326711417},{"type":"v1","addr":"172.21.15.39:6835","nonce":1326711417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":131}],"filesystems":[{"mdsmap":{"epoch":136,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:42:36.446312+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":6425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6425":{"gid":6425,"name":"d","rank":1,"incarnation":130,"state":"up:active","state_seq":18,"addr":"172.21.15.148:6837/73220122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":73220122},{"type":"v1","addr":"172.21.15.148:6837","nonce":73220122}]},"join_fscid":-1,"laggy_since":"2024-09-21T07:42:36.446312+0000","export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[6425,7311]},"id":6}]} 2024-09-21T07:42:38.489 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 136 2024-09-21T07:42:38.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.491+0000 7f5c02800640 1 -- 172.21.15.39:0/3058758436 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5bdc078b90 msgr2=0x7f5bdc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:38.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.491+0000 7f5c02800640 1 --2- 172.21.15.39:0/3058758436 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5bdc078b90 0x7f5bdc07b0a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5be8033000 tx=0x7f5be8004730 comp rx=0 tx=0).stop 2024-09-21T07:42:38.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.491+0000 7f5c02800640 1 -- 172.21.15.39:0/3058758436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bfc09ba60 msgr2=0x7f5bfc1f68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:38.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.491+0000 7f5c02800640 1 --2- 172.21.15.39:0/3058758436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bfc09ba60 0x7f5bfc1f68e0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f5bf000b420 tx=0x7f5bf000b8e0 comp rx=0 tx=0).stop 2024-09-21T07:42:38.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.491+0000 7f5c02800640 1 -- 172.21.15.39:0/3058758436 shutdown_connections 2024-09-21T07:42:38.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.491+0000 7f5c02800640 1 --2- 172.21.15.39:0/3058758436 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5bdc078b90 0x7f5bdc07b0a0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:38.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.491+0000 7f5c02800640 1 --2- 172.21.15.39:0/3058758436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bfc160360 0x7f5bfc06e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:38.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.491+0000 7f5c02800640 1 --2- 172.21.15.39:0/3058758436 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5bfc15fe20 0x7f5bfc15b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:38.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.491+0000 7f5c02800640 1 --2- 172.21.15.39:0/3058758436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bfc09ba60 0x7f5bfc1f68e0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:38.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.491+0000 7f5c02800640 1 -- 172.21.15.39:0/3058758436 >> 172.21.15.39:0/3058758436 conn(0x7f5bfc0b3e40 msgr2=0x7f5bfc0cedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:38.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.491+0000 7f5c02800640 1 -- 172.21.15.39:0/3058758436 shutdown_connections 2024-09-21T07:42:38.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:38.491+0000 7f5c02800640 1 -- 172.21.15.39:0/3058758436 wait complete. 2024-09-21T07:42:38.518 DEBUG:tasks.ceph_test_case:wait_until_true: success in 15s and 0 retries 2024-09-21T07:42:38.519 DEBUG:teuthology.orchestra.run.smithi148:> sudo sysctl -n kernel.core_pattern 2024-09-21T07:42:38.578 INFO:teuthology.orchestra.run.smithi148.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T07:42:38.580 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2024-09-21T07:42:38.581 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2024-09-21T07:42:38.646 INFO:teuthology.orchestra.run.smithi148.stdout:1726904541.43785.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i d', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2024-09-21T07:42:38.649 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1726904541.43785.core'] 2024-09-21T07:42:38.649 INFO:tasks.cephfs.cephfs_test_case:Found core file 1726904541.43785.core, deleting it 2024-09-21T07:42:38.650 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1726904541.43785.core 2024-09-21T07:42:38.739 INFO:tasks.cephfs.mount:Suspending the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:42:38.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:42:38.740 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 down 2024-09-21T07:42:38.804 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:42:39.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.030+0000 7fbcff200640 1 -- 172.21.15.39:0/3459522454 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcf8155a70 msgr2=0x7fbcf80b39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:39.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.030+0000 7fbcff200640 1 --2- 172.21.15.39:0/3459522454 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcf8155a70 0x7fbcf80b39f0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fbcf81598b0 tx=0x7fbce402a560 comp rx=0 tx=0).stop 2024-09-21T07:42:39.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.030+0000 7fbcff200640 1 -- 172.21.15.39:0/3459522454 shutdown_connections 2024-09-21T07:42:39.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.030+0000 7fbcff200640 1 --2- 172.21.15.39:0/3459522454 >> v2:172.21.15.148:3300/0 conn(0x7fbcf80b3f30 0x7fbcf80b4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:39.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.030+0000 7fbcff200640 1 --2- 172.21.15.39:0/3459522454 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcf8155a70 0x7fbcf80b39f0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:39.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.030+0000 7fbcff200640 1 --2- 172.21.15.39:0/3459522454 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbcf815f360 0x7fbcf815bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:39.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.030+0000 7fbcff200640 1 -- 172.21.15.39:0/3459522454 >> 172.21.15.39:0/3459522454 conn(0x7fbcf806cfc0 msgr2=0x7fbcf806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:39.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.031+0000 7fbcff200640 1 -- 172.21.15.39:0/3459522454 shutdown_connections 2024-09-21T07:42:39.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.031+0000 7fbcff200640 1 -- 172.21.15.39:0/3459522454 wait complete. 2024-09-21T07:42:39.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.033+0000 7fbcff200640 1 Processor -- start 2024-09-21T07:42:39.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.033+0000 7fbcff200640 1 -- start start 2024-09-21T07:42:39.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.034+0000 7fbcff200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbcf80b3f30 0x7fbcf81466d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:39.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.034+0000 7fbcff200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcf8155a70 0x7fbcf813f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:39.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.034+0000 7fbcff200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcf815f360 0x7fbcf80b1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:39.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.034+0000 7fbcff200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf80b4f60 con 0x7fbcf815f360 2024-09-21T07:42:39.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.034+0000 7fbcff200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf80b4de0 con 0x7fbcf8155a70 2024-09-21T07:42:39.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.034+0000 7fbcff200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fbcf80d0620 con 0x7fbcf80b3f30 2024-09-21T07:42:39.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.035+0000 7fbcfd200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcf815f360 0x7fbcf80b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:39.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.035+0000 7fbcfd200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcf815f360 0x7fbcf80b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45124/0 (socket says 172.21.15.39:45124) 2024-09-21T07:42:39.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.035+0000 7fbcfd200640 1 -- 172.21.15.39:0/2542542199 learned_addr learned my addr 172.21.15.39:0/2542542199 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:39.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.035+0000 7fbcf7400640 1 --2- 172.21.15.39:0/2542542199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcf8155a70 0x7fbcf813f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:39.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.035+0000 7fbcf7e00640 1 --2- 172.21.15.39:0/2542542199 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbcf80b3f30 0x7fbcf81466d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:39.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.035+0000 7fbcfd200640 1 -- 172.21.15.39:0/2542542199 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbcf80b3f30 msgr2=0x7fbcf81466d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:39.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.035+0000 7fbcfd200640 1 --2- 172.21.15.39:0/2542542199 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbcf80b3f30 0x7fbcf81466d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:39.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.035+0000 7fbcfd200640 1 -- 172.21.15.39:0/2542542199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcf8155a70 msgr2=0x7fbcf813f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:39.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.035+0000 7fbcfd200640 1 --2- 172.21.15.39:0/2542542199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcf8155a70 0x7fbcf813f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:39.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.035+0000 7fbcfd200640 1 -- 172.21.15.39:0/2542542199 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf8144150 con 0x7fbcf815f360 2024-09-21T07:42:39.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.035+0000 7fbcfd200640 1 --2- 172.21.15.39:0/2542542199 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcf815f360 0x7fbcf80b1160 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fbcf000ea40 tx=0x7fbcf000ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:39.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.036+0000 7fbcefe00640 1 -- 172.21.15.39:0/2542542199 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf000ce40 con 0x7fbcf815f360 2024-09-21T07:42:39.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.036+0000 7fbcefe00640 1 -- 172.21.15.39:0/2542542199 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbcf0004c10 con 0x7fbcf815f360 2024-09-21T07:42:39.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.036+0000 7fbcefe00640 1 -- 172.21.15.39:0/2542542199 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0005020 con 0x7fbcf815f360 2024-09-21T07:42:39.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.036+0000 7fbcff200640 1 -- 172.21.15.39:0/2542542199 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcf8144430 con 0x7fbcf815f360 2024-09-21T07:42:39.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.036+0000 7fbcff200640 1 -- 172.21.15.39:0/2542542199 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcf80b0360 con 0x7fbcf815f360 2024-09-21T07:42:39.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.037+0000 7fbcff200640 1 -- 172.21.15.39:0/2542542199 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcf80b39f0 con 0x7fbcf815f360 2024-09-21T07:42:39.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.038+0000 7fbcefe00640 1 -- 172.21.15.39:0/2542542199 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbcf00051c0 con 0x7fbcf815f360 2024-09-21T07:42:39.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.040+0000 7fbcefe00640 1 --2- 172.21.15.39:0/2542542199 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbcd8078b90 0x7fbcd807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:39.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.040+0000 7fbcefe00640 1 -- 172.21.15.39:0/2542542199 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7947+0+0 (secure 0 0 0) 0x7fbcf009c200 con 0x7fbcf815f360 2024-09-21T07:42:39.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.040+0000 7fbcf7e00640 1 --2- 172.21.15.39:0/2542542199 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbcd8078b90 0x7fbcd807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:39.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.041+0000 7fbcf7e00640 1 --2- 172.21.15.39:0/2542542199 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbcd8078b90 0x7fbcd807b0a0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fbce8009ab0 tx=0x7fbce8005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:39.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.044+0000 7fbcefe00640 1 -- 172.21.15.39:0/2542542199 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbcf00637c0 con 0x7fbcf815f360 2024-09-21T07:42:39.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.278+0000 7fbcff200640 1 -- 172.21.15.39:0/2542542199 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbcf80708d0 con 0x7fbcf815f360 2024-09-21T07:42:39.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.279+0000 7fbcefe00640 1 -- 172.21.15.39:0/2542542199 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 136 v136) ==== 95+0+5301 (secure 0 0 0) 0x7fbcf00681f0 con 0x7fbcf815f360 2024-09-21T07:42:39.280 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:42:39.281 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":136,"btime":"2024-09-21T07:42:36:446339+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7644,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1326711417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1326711417},{"type":"v1","addr":"172.21.15.39:6835","nonce":1326711417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":131}],"filesystems":[{"mdsmap":{"epoch":136,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:42:36.446312+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":6425},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6425":{"gid":6425,"name":"d","rank":1,"incarnation":130,"state":"up:active","state_seq":18,"addr":"172.21.15.148:6837/73220122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":73220122},{"type":"v1","addr":"172.21.15.148:6837","nonce":73220122}]},"join_fscid":-1,"laggy_since":"2024-09-21T07:42:36.446312+0000","export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[6425,7311]},"id":6}]} 2024-09-21T07:42:39.281 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 136 2024-09-21T07:42:39.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.281+0000 7fbcff200640 1 -- 172.21.15.39:0/2542542199 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbcd8078b90 msgr2=0x7fbcd807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:39.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.281+0000 7fbcff200640 1 --2- 172.21.15.39:0/2542542199 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbcd8078b90 0x7fbcd807b0a0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fbce8009ab0 tx=0x7fbce8005f90 comp rx=0 tx=0).stop 2024-09-21T07:42:39.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.281+0000 7fbcff200640 1 -- 172.21.15.39:0/2542542199 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcf815f360 msgr2=0x7fbcf80b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:39.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.281+0000 7fbcff200640 1 --2- 172.21.15.39:0/2542542199 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcf815f360 0x7fbcf80b1160 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fbcf000ea40 tx=0x7fbcf000ef00 comp rx=0 tx=0).stop 2024-09-21T07:42:39.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.281+0000 7fbcff200640 1 -- 172.21.15.39:0/2542542199 shutdown_connections 2024-09-21T07:42:39.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.281+0000 7fbcff200640 1 --2- 172.21.15.39:0/2542542199 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbcd8078b90 0x7fbcd807b0a0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:39.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.281+0000 7fbcff200640 1 --2- 172.21.15.39:0/2542542199 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcf815f360 0x7fbcf80b1160 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:39.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.281+0000 7fbcff200640 1 --2- 172.21.15.39:0/2542542199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcf8155a70 0x7fbcf813f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:39.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.281+0000 7fbcff200640 1 --2- 172.21.15.39:0/2542542199 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbcf80b3f30 0x7fbcf81466d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:39.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.281+0000 7fbcff200640 1 -- 172.21.15.39:0/2542542199 >> 172.21.15.39:0/2542542199 conn(0x7fbcf806cfc0 msgr2=0x7fbcf806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:39.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.281+0000 7fbcff200640 1 -- 172.21.15.39:0/2542542199 shutdown_connections 2024-09-21T07:42:39.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.282+0000 7fbcff200640 1 -- 172.21.15.39:0/2542542199 wait complete. 2024-09-21T07:42:39.308 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2024-09-21T07:42:39.555 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_snap":16,"last_created":12,"last_destroyed":14,"pending_noop":[],"snaps":[],"need_to_purge":{},"pending_update":[{"version":33,"snapinfo":{"snapid":16,"ino":1099511627777,"stamp":"2024-09-21T07:42:21.254582+0000","name":"s34","metadata":{}}}],"pending_destroy":[]} 2024-09-21T07:42:39.555 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 12, "last_destroyed": 14, "last_snap": 16, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [ { "snapInfinityo": { "ino": 1099511627777, "metadata": {}, "name": "s34", "snapid": 16, "stamp": "2024-09-21T07:42:21.254582+0000" }, "version": 33 } ], "snaps": [] } 2024-09-21T07:42:39.556 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 6:1 2024-09-21T07:42:39.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.782+0000 7fdfd8c00640 1 -- 172.21.15.39:0/3286914161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfb8003a00 msgr2=0x7fdfb8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:39.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.782+0000 7fdfd8c00640 1 --2- 172.21.15.39:0/3286914161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfb8003a00 0x7fdfb8003e70 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fdfc400c290 tx=0x7fdfb8005830 comp rx=0 tx=0).stop 2024-09-21T07:42:39.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.782+0000 7fdfd2800640 1 -- 172.21.15.39:0/3286914161 reap_dead start 2024-09-21T07:42:39.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.782+0000 7fdfd8c00640 1 -- 172.21.15.39:0/3286914161 shutdown_connections 2024-09-21T07:42:39.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.782+0000 7fdfd8c00640 1 -- 172.21.15.39:0/3286914161 >> 172.21.15.39:0/3286914161 conn(0x7fdfd406cfc0 msgr2=0x7fdfd406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:39.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.783+0000 7fdfd8c00640 1 -- 172.21.15.39:0/3286914161 shutdown_connections 2024-09-21T07:42:39.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.783+0000 7fdfd8c00640 1 -- 172.21.15.39:0/3286914161 wait complete. 2024-09-21T07:42:39.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.785+0000 7fdfd8c00640 1 Processor -- start 2024-09-21T07:42:39.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.785+0000 7fdfd8c00640 1 -- start start 2024-09-21T07:42:39.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.786+0000 7fdfd8c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfb8003a00 0x7fdfd4152f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:39.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.786+0000 7fdfd8c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfd40ad380 0x7fdfd41f36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:39.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.786+0000 7fdfd8c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdfd40ac7a0 0x7fdfd41e85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:39.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.786+0000 7fdfd8c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd40cdcf0 con 0x7fdfd40ad380 2024-09-21T07:42:39.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.786+0000 7fdfd8c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd40cdb70 con 0x7fdfb8003a00 2024-09-21T07:42:39.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.786+0000 7fdfd8c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd40cde70 con 0x7fdfd40ac7a0 2024-09-21T07:42:39.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.787+0000 7fdfd1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfd40ad380 0x7fdfd41f36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:39.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.787+0000 7fdfd1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfd40ad380 0x7fdfd41f36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45146/0 (socket says 172.21.15.39:45146) 2024-09-21T07:42:39.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.787+0000 7fdfd1400640 1 -- 172.21.15.39:0/249516953 learned_addr learned my addr 172.21.15.39:0/249516953 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:39.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.787+0000 7fdfd1e00640 1 --2- 172.21.15.39:0/249516953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfb8003a00 0x7fdfd4152f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:39.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.787+0000 7fdfd2800640 1 --2- 172.21.15.39:0/249516953 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdfd40ac7a0 0x7fdfd41e85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:39.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.787+0000 7fdfd1400640 1 -- 172.21.15.39:0/249516953 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdfd40ac7a0 msgr2=0x7fdfd41e85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:39.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.787+0000 7fdfd1400640 1 --2- 172.21.15.39:0/249516953 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdfd40ac7a0 0x7fdfd41e85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:39.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.787+0000 7fdfd1400640 1 -- 172.21.15.39:0/249516953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfb8003a00 msgr2=0x7fdfd4152f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:39.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.787+0000 7fdfd1400640 1 --2- 172.21.15.39:0/249516953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfb8003a00 0x7fdfd4152f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:39.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.787+0000 7fdfd1400640 1 -- 172.21.15.39:0/249516953 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd40ab700 con 0x7fdfd40ad380 2024-09-21T07:42:39.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.787+0000 7fdfd2800640 1 --2- 172.21.15.39:0/249516953 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdfd40ac7a0 0x7fdfd41e85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:42:39.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.787+0000 7fdfd1400640 1 --2- 172.21.15.39:0/249516953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfd40ad380 0x7fdfd41f36e0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fdfc00119b0 tx=0x7fdfc0011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:39.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.788+0000 7fdfca000640 1 -- 172.21.15.39:0/249516953 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc000fcf0 con 0x7fdfd40ad380 2024-09-21T07:42:39.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.788+0000 7fdfca000640 1 -- 172.21.15.39:0/249516953 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfc0004030 con 0x7fdfd40ad380 2024-09-21T07:42:39.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.788+0000 7fdfca000640 1 -- 172.21.15.39:0/249516953 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0004480 con 0x7fdfd40ad380 2024-09-21T07:42:39.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.788+0000 7fdfd8c00640 1 -- 172.21.15.39:0/249516953 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfd40ab9e0 con 0x7fdfd40ad380 2024-09-21T07:42:39.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.788+0000 7fdfd8c00640 1 -- 172.21.15.39:0/249516953 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfd40abfa0 con 0x7fdfd40ad380 2024-09-21T07:42:39.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.789+0000 7fdfd8c00640 1 -- 172.21.15.39:0/249516953 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfd40d0a30 con 0x7fdfd40ad380 2024-09-21T07:42:39.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.790+0000 7fdfca000640 1 -- 172.21.15.39:0/249516953 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdfc00035f0 con 0x7fdfd40ad380 2024-09-21T07:42:39.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.791+0000 7fdfca000640 1 --2- 172.21.15.39:0/249516953 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdfb4078b90 0x7fdfb407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:39.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.791+0000 7fdfca000640 1 -- 172.21.15.39:0/249516953 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7947+0+0 (secure 0 0 0) 0x7fdfc009c3e0 con 0x7fdfd40ad380 2024-09-21T07:42:39.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.792+0000 7fdfd1e00640 1 --2- 172.21.15.39:0/249516953 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdfb4078b90 0x7fdfb407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:39.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.793+0000 7fdfd1e00640 1 --2- 172.21.15.39:0/249516953 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdfb4078b90 0x7fdfb407b0a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4033000 tx=0x7fdfc402d000 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:39.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:39.796+0000 7fdfca000640 1 -- 172.21.15.39:0/249516953 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdfc0063f60 con 0x7fdfd40ad380 2024-09-21T07:42:40.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.074+0000 7fdfd8c00640 1 -- 172.21.15.39:0/249516953 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "6:1"} v 0) -- 0x7fdfd415f920 con 0x7fdfd40ad380 2024-09-21T07:42:40.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.080+0000 7fdfca000640 1 -- 172.21.15.39:0/249516953 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "6:1"}]=0 failed mds gid 6425 v137) ==== 97+0+0 (secure 0 0 0) 0x7fdfc0063860 con 0x7fdfd40ad380 2024-09-21T07:42:40.081 INFO:teuthology.orchestra.run.smithi039.stderr:failed mds gid 6425 2024-09-21T07:42:40.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.084+0000 7fdfd8c00640 1 -- 172.21.15.39:0/249516953 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdfb4078b90 msgr2=0x7fdfb407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:40.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.084+0000 7fdfd8c00640 1 --2- 172.21.15.39:0/249516953 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdfb4078b90 0x7fdfb407b0a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4033000 tx=0x7fdfc402d000 comp rx=0 tx=0).stop 2024-09-21T07:42:40.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.084+0000 7fdfd8c00640 1 -- 172.21.15.39:0/249516953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfd40ad380 msgr2=0x7fdfd41f36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:40.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.084+0000 7fdfd8c00640 1 --2- 172.21.15.39:0/249516953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfd40ad380 0x7fdfd41f36e0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fdfc00119b0 tx=0x7fdfc0011e70 comp rx=0 tx=0).stop 2024-09-21T07:42:40.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.085+0000 7fdfd8c00640 1 -- 172.21.15.39:0/249516953 shutdown_connections 2024-09-21T07:42:40.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.085+0000 7fdfd8c00640 1 --2- 172.21.15.39:0/249516953 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdfb4078b90 0x7fdfb407b0a0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:40.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.085+0000 7fdfd8c00640 1 --2- 172.21.15.39:0/249516953 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdfd40ac7a0 0x7fdfd41e85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:40.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.085+0000 7fdfd8c00640 1 --2- 172.21.15.39:0/249516953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfd40ad380 0x7fdfd41f36e0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:40.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.085+0000 7fdfd8c00640 1 --2- 172.21.15.39:0/249516953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdfb8003a00 0x7fdfd4152f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:40.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.085+0000 7fdfd8c00640 1 -- 172.21.15.39:0/249516953 >> 172.21.15.39:0/249516953 conn(0x7fdfd406cfc0 msgr2=0x7fdfd40b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:40.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.085+0000 7fdfd8c00640 1 -- 172.21.15.39:0/249516953 shutdown_connections 2024-09-21T07:42:40.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.085+0000 7fdfd8c00640 1 -- 172.21.15.39:0/249516953 wait complete. 2024-09-21T07:42:40.114 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T07:42:40.114 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T07:42:40.114 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T07:42:40.114 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:42:40.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:42:40.115 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/daemon/state.py", line 146, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi148 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d' 2024-09-21T07:42:40.115 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T07:42:40.115 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T07:42:40.119 INFO:tasks.ceph.mds.d:Started 2024-09-21T07:42:40.119 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:42:40.309 INFO:tasks.ceph.mds.d.smithi148.stdout:starting mds.d at 2024-09-21T07:42:40.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.345+0000 7fd5a0600640 1 -- 172.21.15.39:0/1613636201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd598155bc0 msgr2=0x7fd5980d06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:40.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.345+0000 7fd5a0600640 1 --2- 172.21.15.39:0/1613636201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd598155bc0 0x7fd5980d06c0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fd584005fd0 tx=0x7fd58402a5d0 comp rx=0 tx=0).stop 2024-09-21T07:42:40.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.345+0000 7fd5a0600640 1 -- 172.21.15.39:0/1613636201 shutdown_connections 2024-09-21T07:42:40.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.345+0000 7fd5a0600640 1 --2- 172.21.15.39:0/1613636201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd598155bc0 0x7fd5980d06c0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:40.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.345+0000 7fd5a0600640 1 --2- 172.21.15.39:0/1613636201 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5981557b0 0x7fd5980b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:40.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.345+0000 7fd5a0600640 1 -- 172.21.15.39:0/1613636201 >> 172.21.15.39:0/1613636201 conn(0x7fd59806cfc0 msgr2=0x7fd59806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:40.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.345+0000 7fd5a0600640 1 -- 172.21.15.39:0/1613636201 shutdown_connections 2024-09-21T07:42:40.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.345+0000 7fd5a0600640 1 -- 172.21.15.39:0/1613636201 wait complete. 2024-09-21T07:42:40.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.347+0000 7fd5a0600640 1 Processor -- start 2024-09-21T07:42:40.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.348+0000 7fd5a0600640 1 -- start start 2024-09-21T07:42:40.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.349+0000 7fd5a0600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5980b4840 0x7fd598145800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:40.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.349+0000 7fd5a0600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5981557b0 0x7fd598144fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:40.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.349+0000 7fd5a0600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd598155bc0 0x7fd59815d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:40.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.349+0000 7fd5a0600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd5980d0fc0 con 0x7fd5981557b0 2024-09-21T07:42:40.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.349+0000 7fd5a0600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd5980d0e40 con 0x7fd598155bc0 2024-09-21T07:42:40.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.349+0000 7fd5a0600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd5980add50 con 0x7fd5980b4840 2024-09-21T07:42:40.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.349+0000 7fd59d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5981557b0 0x7fd598144fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:40.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.349+0000 7fd59d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5981557b0 0x7fd598144fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45154/0 (socket says 172.21.15.39:45154) 2024-09-21T07:42:40.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.349+0000 7fd59d200640 1 -- 172.21.15.39:0/2685462356 learned_addr learned my addr 172.21.15.39:0/2685462356 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:40.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.349+0000 7fd59dc00640 1 --2- 172.21.15.39:0/2685462356 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5980b4840 0x7fd598145800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:40.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.349+0000 7fd59e600640 1 --2- 172.21.15.39:0/2685462356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd598155bc0 0x7fd59815d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:40.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.350+0000 7fd59d200640 1 -- 172.21.15.39:0/2685462356 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5980b4840 msgr2=0x7fd598145800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:40.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.350+0000 7fd59d200640 1 --2- 172.21.15.39:0/2685462356 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5980b4840 0x7fd598145800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:40.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.350+0000 7fd59d200640 1 -- 172.21.15.39:0/2685462356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd598155bc0 msgr2=0x7fd59815d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:40.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.350+0000 7fd59d200640 1 --2- 172.21.15.39:0/2685462356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd598155bc0 0x7fd59815d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:40.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.350+0000 7fd59d200640 1 -- 172.21.15.39:0/2685462356 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd598145510 con 0x7fd5981557b0 2024-09-21T07:42:40.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.350+0000 7fd59d200640 1 --2- 172.21.15.39:0/2685462356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5981557b0 0x7fd598144fd0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fd584000c00 tx=0x7fd5840035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:40.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.351+0000 7fd595600640 1 -- 172.21.15.39:0/2685462356 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58403e030 con 0x7fd5981557b0 2024-09-21T07:42:40.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.351+0000 7fd595600640 1 -- 172.21.15.39:0/2685462356 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd584003ae0 con 0x7fd5981557b0 2024-09-21T07:42:40.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.351+0000 7fd595600640 1 -- 172.21.15.39:0/2685462356 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd584035040 con 0x7fd5981557b0 2024-09-21T07:42:40.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.351+0000 7fd5a0600640 1 -- 172.21.15.39:0/2685462356 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd59815c540 con 0x7fd5981557b0 2024-09-21T07:42:40.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.351+0000 7fd5a0600640 1 -- 172.21.15.39:0/2685462356 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd59815cb00 con 0x7fd5981557b0 2024-09-21T07:42:40.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.352+0000 7fd5a0600640 1 -- 172.21.15.39:0/2685462356 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5980b4300 con 0x7fd5981557b0 2024-09-21T07:42:40.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.353+0000 7fd595600640 1 -- 172.21.15.39:0/2685462356 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd584041030 con 0x7fd5981557b0 2024-09-21T07:42:40.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.354+0000 7fd595600640 1 --2- 172.21.15.39:0/2685462356 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd578078b90 0x7fd57807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:40.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.354+0000 7fd595600640 1 -- 172.21.15.39:0/2685462356 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8033+0+0 (secure 0 0 0) 0x7fd5840bbb10 con 0x7fd5981557b0 2024-09-21T07:42:40.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.354+0000 7fd59dc00640 1 --2- 172.21.15.39:0/2685462356 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd578078b90 0x7fd57807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:40.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.355+0000 7fd59dc00640 1 --2- 172.21.15.39:0/2685462356 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd578078b90 0x7fd57807b0a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd58c009b30 tx=0x7fd58c005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:40.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.358+0000 7fd595600640 1 -- 172.21.15.39:0/2685462356 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd584082fd0 con 0x7fd5981557b0 2024-09-21T07:42:40.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.598+0000 7fd5a0600640 1 -- 172.21.15.39:0/2685462356 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd59815c2d0 con 0x7fd5981557b0 2024-09-21T07:42:40.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.598+0000 7fd595600640 1 -- 172.21.15.39:0/2685462356 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 138 v138) ==== 95+0+4424 (secure 0 0 0) 0x7fd584087a00 con 0x7fd5981557b0 2024-09-21T07:42:40.599 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:42:40.600 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":138,"btime":"2024-09-21T07:42:40:083934+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":138,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:42:40.083915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7644},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7644":{"gid":7644,"name":"a","rank":1,"incarnation":138,"state":"up:replay","state_seq":1,"addr":"172.21.15.39:6835/1326711417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1326711417},{"type":"v1","addr":"172.21.15.39:6835","nonce":1326711417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311]},"id":6}]} 2024-09-21T07:42:40.600 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 138 2024-09-21T07:42:40.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.601+0000 7fd5a0600640 1 -- 172.21.15.39:0/2685462356 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd578078b90 msgr2=0x7fd57807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:40.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.601+0000 7fd5a0600640 1 --2- 172.21.15.39:0/2685462356 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd578078b90 0x7fd57807b0a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd58c009b30 tx=0x7fd58c005f90 comp rx=0 tx=0).stop 2024-09-21T07:42:40.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.601+0000 7fd5a0600640 1 -- 172.21.15.39:0/2685462356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5981557b0 msgr2=0x7fd598144fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:40.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.601+0000 7fd5a0600640 1 --2- 172.21.15.39:0/2685462356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5981557b0 0x7fd598144fd0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fd584000c00 tx=0x7fd5840035f0 comp rx=0 tx=0).stop 2024-09-21T07:42:40.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.601+0000 7fd5a0600640 1 -- 172.21.15.39:0/2685462356 shutdown_connections 2024-09-21T07:42:40.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.601+0000 7fd5a0600640 1 --2- 172.21.15.39:0/2685462356 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd578078b90 0x7fd57807b0a0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:40.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.601+0000 7fd5a0600640 1 --2- 172.21.15.39:0/2685462356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd598155bc0 0x7fd59815d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:40.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.601+0000 7fd5a0600640 1 --2- 172.21.15.39:0/2685462356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5981557b0 0x7fd598144fd0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:40.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.601+0000 7fd5a0600640 1 --2- 172.21.15.39:0/2685462356 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5980b4840 0x7fd598145800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:40.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.601+0000 7fd5a0600640 1 -- 172.21.15.39:0/2685462356 >> 172.21.15.39:0/2685462356 conn(0x7fd59806cfc0 msgr2=0x7fd5980b2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:40.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.601+0000 7fd5a0600640 1 -- 172.21.15.39:0/2685462356 shutdown_connections 2024-09-21T07:42:40.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:40.601+0000 7fd5a0600640 1 -- 172.21.15.39:0/2685462356 wait complete. 2024-09-21T07:42:40.627 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-09-21T07:42:42.662 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-09-21T07:42:45.628 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:42:45.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:45.862+0000 7f3ddd000640 1 -- 172.21.15.39:0/2366059662 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd80cbd70 msgr2=0x7f3dd8072fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:45.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:45.862+0000 7f3ddd000640 1 --2- 172.21.15.39:0/2366059662 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd80cbd70 0x7f3dd8072fa0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8071ed0 tx=0x7f3dd002aac0 comp rx=0 tx=0).stop 2024-09-21T07:42:45.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:45.862+0000 7f3ddd000640 1 -- 172.21.15.39:0/2366059662 shutdown_connections 2024-09-21T07:42:45.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:45.862+0000 7f3ddd000640 1 --2- 172.21.15.39:0/2366059662 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd80cbd70 0x7f3dd8072fa0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:45.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:45.862+0000 7f3ddd000640 1 --2- 172.21.15.39:0/2366059662 >> v2:172.21.15.148:3300/0 conn(0x7f3dd80d0f20 0x7f3dd815c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:45.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:45.862+0000 7f3ddd000640 1 -- 172.21.15.39:0/2366059662 >> 172.21.15.39:0/2366059662 conn(0x7f3dd80b2510 msgr2=0x7f3dd80d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:45.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:45.863+0000 7f3ddd000640 1 -- 172.21.15.39:0/2366059662 shutdown_connections 2024-09-21T07:42:45.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:45.863+0000 7f3ddd000640 1 -- 172.21.15.39:0/2366059662 wait complete. 2024-09-21T07:42:45.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:45.865+0000 7f3ddd000640 1 Processor -- start 2024-09-21T07:42:45.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:45.865+0000 7f3ddd000640 1 -- start start 2024-09-21T07:42:45.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:45.866+0000 7f3ddd000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dd80cb960 0x7f3dd809d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:45.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:45.866+0000 7f3ddd000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3dd80cbd70 0x7f3dd81f2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:45.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:45.867+0000 7f3ddd000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd80d0f20 0x7f3dd815f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:45.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:45.867+0000 7f3ddd000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd80b43f0 con 0x7f3dd80d0f20 2024-09-21T07:42:45.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:45.867+0000 7f3ddd000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd8073800 con 0x7f3dd80cb960 2024-09-21T07:42:45.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:45.867+0000 7f3ddd000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f3dd80b4570 con 0x7f3dd80cbd70 2024-09-21T07:42:45.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:45.867+0000 7f3dd7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd80d0f20 0x7f3dd815f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:45.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:45.867+0000 7f3dd7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd80d0f20 0x7f3dd815f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48726/0 (socket says 172.21.15.39:48726) 2024-09-21T07:42:45.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:45.867+0000 7f3dd7e00640 1 -- 172.21.15.39:0/1513008495 learned_addr learned my addr 172.21.15.39:0/1513008495 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:45.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:45.867+0000 7f3dd7400640 1 --2- 172.21.15.39:0/1513008495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dd80cb960 0x7f3dd809d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:45.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:45.867+0000 7f3dd6a00640 1 --2- 172.21.15.39:0/1513008495 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3dd80cbd70 0x7f3dd81f2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:45.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:45.867+0000 7f3dd7e00640 1 -- 172.21.15.39:0/1513008495 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3dd80cbd70 msgr2=0x7f3dd81f2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:45.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:45.867+0000 7f3dd7e00640 1 --2- 172.21.15.39:0/1513008495 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3dd80cbd70 0x7f3dd81f2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:45.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:45.867+0000 7f3dd7e00640 1 -- 172.21.15.39:0/1513008495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dd80cb960 msgr2=0x7f3dd809d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:45.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:45.867+0000 7f3dd7e00640 1 --2- 172.21.15.39:0/1513008495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dd80cb960 0x7f3dd809d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:45.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:45.867+0000 7f3dd7e00640 1 -- 172.21.15.39:0/1513008495 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd809e260 con 0x7f3dd80d0f20 2024-09-21T07:42:45.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:45.867+0000 7f3dd6a00640 1 --2- 172.21.15.39:0/1513008495 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3dd80cbd70 0x7f3dd81f2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:42:45.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:45.868+0000 7f3dd7e00640 1 --2- 172.21.15.39:0/1513008495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd80d0f20 0x7f3dd815f320 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f3dd002b0b0 tx=0x7f3dd0003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:45.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:45.868+0000 7f3dcf400640 1 -- 172.21.15.39:0/1513008495 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd0004250 con 0x7f3dd80d0f20 2024-09-21T07:42:45.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:45.868+0000 7f3dcf400640 1 -- 172.21.15.39:0/1513008495 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3dd00043f0 con 0x7f3dd80d0f20 2024-09-21T07:42:45.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:45.868+0000 7f3dcf400640 1 -- 172.21.15.39:0/1513008495 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd0004840 con 0x7f3dd80d0f20 2024-09-21T07:42:45.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:45.868+0000 7f3ddd000640 1 -- 172.21.15.39:0/1513008495 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dd80ce010 con 0x7f3dd80d0f20 2024-09-21T07:42:45.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:45.868+0000 7f3ddd000640 1 -- 172.21.15.39:0/1513008495 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dd80ce5d0 con 0x7f3dd80d0f20 2024-09-21T07:42:45.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:45.869+0000 7f3ddd000640 1 -- 172.21.15.39:0/1513008495 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dd80b4ab0 con 0x7f3dd80d0f20 2024-09-21T07:42:45.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:45.870+0000 7f3dcf400640 1 -- 172.21.15.39:0/1513008495 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3dd0049430 con 0x7f3dd80d0f20 2024-09-21T07:42:45.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:45.871+0000 7f3dcf400640 1 --2- 172.21.15.39:0/1513008495 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3db8078b90 0x7f3db807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:45.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:45.871+0000 7f3dcf400640 1 -- 172.21.15.39:0/1513008495 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8033+0+0 (secure 0 0 0) 0x7f3dd00bc510 con 0x7f3dd80d0f20 2024-09-21T07:42:45.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:45.872+0000 7f3dd7400640 1 --2- 172.21.15.39:0/1513008495 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3db8078b90 0x7f3db807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:45.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:45.873+0000 7f3dd7400640 1 --2- 172.21.15.39:0/1513008495 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3db8078b90 0x7f3db807b0a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8004630 tx=0x7f3dc800b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:45.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:45.876+0000 7f3dcf400640 1 -- 172.21.15.39:0/1513008495 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3dd0032030 con 0x7f3dd80d0f20 2024-09-21T07:42:46.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.112+0000 7f3ddd000640 1 -- 172.21.15.39:0/1513008495 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3dd8074510 con 0x7f3dd80d0f20 2024-09-21T07:42:46.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.112+0000 7f3dcf400640 1 -- 172.21.15.39:0/1513008495 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 140 v140) ==== 95+0+5254 (secure 0 0 0) 0x7f3dd0083f90 con 0x7f3dd80d0f20 2024-09-21T07:42:46.114 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:42:46.114 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":140,"btime":"2024-09-21T07:42:42:091818+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7366,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/759792812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":759792812},{"type":"v1","addr":"172.21.15.148:6837","nonce":759792812}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":139}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:42:41.094897+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7644},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7644":{"gid":7644,"name":"a","rank":1,"incarnation":138,"state":"up:reconnect","state_seq":15,"addr":"172.21.15.39:6835/1326711417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1326711417},{"type":"v1","addr":"172.21.15.39:6835","nonce":1326711417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311]},"id":6}]} 2024-09-21T07:42:46.115 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 140 2024-09-21T07:42:46.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.115+0000 7f3ddd000640 1 -- 172.21.15.39:0/1513008495 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3db8078b90 msgr2=0x7f3db807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:46.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.115+0000 7f3ddd000640 1 --2- 172.21.15.39:0/1513008495 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3db8078b90 0x7f3db807b0a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8004630 tx=0x7f3dc800b2e0 comp rx=0 tx=0).stop 2024-09-21T07:42:46.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.115+0000 7f3ddd000640 1 -- 172.21.15.39:0/1513008495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd80d0f20 msgr2=0x7f3dd815f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:46.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.115+0000 7f3ddd000640 1 --2- 172.21.15.39:0/1513008495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd80d0f20 0x7f3dd815f320 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f3dd002b0b0 tx=0x7f3dd0003ce0 comp rx=0 tx=0).stop 2024-09-21T07:42:46.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.115+0000 7f3ddd000640 1 -- 172.21.15.39:0/1513008495 shutdown_connections 2024-09-21T07:42:46.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.115+0000 7f3ddd000640 1 --2- 172.21.15.39:0/1513008495 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3db8078b90 0x7f3db807b0a0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:46.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.115+0000 7f3ddd000640 1 --2- 172.21.15.39:0/1513008495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dd80d0f20 0x7f3dd815f320 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:46.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.115+0000 7f3ddd000640 1 --2- 172.21.15.39:0/1513008495 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3dd80cbd70 0x7f3dd81f2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:46.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.115+0000 7f3ddd000640 1 --2- 172.21.15.39:0/1513008495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dd80cb960 0x7f3dd809d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:46.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.115+0000 7f3ddd000640 1 -- 172.21.15.39:0/1513008495 >> 172.21.15.39:0/1513008495 conn(0x7f3dd80b2510 msgr2=0x7f3dd80ccc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:46.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.115+0000 7f3ddd000640 1 -- 172.21.15.39:0/1513008495 shutdown_connections 2024-09-21T07:42:46.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.115+0000 7f3ddd000640 1 -- 172.21.15.39:0/1513008495 wait complete. 2024-09-21T07:42:46.142 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-21T07:42:46.143 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:42:46.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.366+0000 7f330a800640 1 -- 172.21.15.39:0/673571510 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3304155e80 msgr2=0x7f33040b4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:46.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.366+0000 7f330a800640 1 --2- 172.21.15.39:0/673571510 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3304155e80 0x7f33040b4690 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f3304072c00 tx=0x7f32fc02a980 comp rx=0 tx=0).stop 2024-09-21T07:42:46.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.367+0000 7f330a800640 1 -- 172.21.15.39:0/673571510 shutdown_connections 2024-09-21T07:42:46.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.367+0000 7f330a800640 1 --2- 172.21.15.39:0/673571510 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3304155e80 0x7f33040b4690 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:46.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.367+0000 7f330a800640 1 --2- 172.21.15.39:0/673571510 >> v2:172.21.15.148:3300/0 conn(0x7f330415f360 0x7f330415bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:46.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.367+0000 7f330a800640 1 -- 172.21.15.39:0/673571510 >> 172.21.15.39:0/673571510 conn(0x7f330406cfc0 msgr2=0x7f330406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:46.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.367+0000 7f330a800640 1 -- 172.21.15.39:0/673571510 shutdown_connections 2024-09-21T07:42:46.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.367+0000 7f330a800640 1 -- 172.21.15.39:0/673571510 wait complete. 2024-09-21T07:42:46.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.369+0000 7f330a800640 1 Processor -- start 2024-09-21T07:42:46.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.369+0000 7f330a800640 1 -- start start 2024-09-21T07:42:46.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.370+0000 7f330a800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3304155a70 0x7f33040b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:46.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.370+0000 7f330a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3304155e80 0x7f33041547c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:46.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.370+0000 7f330a800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f330415f360 0x7f3304146050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:46.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.371+0000 7f330a800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f33040b4fa0 con 0x7f3304155e80 2024-09-21T07:42:46.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.371+0000 7f330a800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f33040b4e20 con 0x7f3304155a70 2024-09-21T07:42:46.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.371+0000 7f330a800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f33040d0620 con 0x7f330415f360 2024-09-21T07:42:46.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.371+0000 7f3302a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3304155e80 0x7f33041547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:46.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.371+0000 7f3302a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3304155e80 0x7f33041547c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48752/0 (socket says 172.21.15.39:48752) 2024-09-21T07:42:46.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.371+0000 7f3302a00640 1 -- 172.21.15.39:0/2842697994 learned_addr learned my addr 172.21.15.39:0/2842697994 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:46.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.371+0000 7f3303400640 1 --2- 172.21.15.39:0/2842697994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3304155a70 0x7f33040b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:46.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.371+0000 7f3303e00640 1 --2- 172.21.15.39:0/2842697994 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f330415f360 0x7f3304146050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:46.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.371+0000 7f3302a00640 1 -- 172.21.15.39:0/2842697994 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f330415f360 msgr2=0x7f3304146050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:46.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.371+0000 7f3302a00640 1 --2- 172.21.15.39:0/2842697994 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f330415f360 0x7f3304146050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:46.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.371+0000 7f3302a00640 1 -- 172.21.15.39:0/2842697994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3304155a70 msgr2=0x7f33040b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:46.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.371+0000 7f3302a00640 1 --2- 172.21.15.39:0/2842697994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3304155a70 0x7f33040b1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:46.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.371+0000 7f3302a00640 1 -- 172.21.15.39:0/2842697994 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33040b0a90 con 0x7f3304155e80 2024-09-21T07:42:46.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.371+0000 7f3303400640 1 --2- 172.21.15.39:0/2842697994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3304155a70 0x7f33040b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:42:46.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.371+0000 7f3303e00640 1 --2- 172.21.15.39:0/2842697994 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f330415f360 0x7f3304146050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:42:46.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.372+0000 7f3302a00640 1 --2- 172.21.15.39:0/2842697994 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3304155e80 0x7f33041547c0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f32f000b3f0 tx=0x7f32f000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:46.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.372+0000 7f32fb400640 1 -- 172.21.15.39:0/2842697994 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f0015020 con 0x7f3304155e80 2024-09-21T07:42:46.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.372+0000 7f32fb400640 1 -- 172.21.15.39:0/2842697994 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32f0011920 con 0x7f3304155e80 2024-09-21T07:42:46.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.372+0000 7f330a800640 1 -- 172.21.15.39:0/2842697994 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33040b0d70 con 0x7f3304155e80 2024-09-21T07:42:46.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.372+0000 7f32fb400640 1 -- 172.21.15.39:0/2842697994 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f0011d70 con 0x7f3304155e80 2024-09-21T07:42:46.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.372+0000 7f330a800640 1 -- 172.21.15.39:0/2842697994 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33041e63b0 con 0x7f3304155e80 2024-09-21T07:42:46.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.373+0000 7f330a800640 1 -- 172.21.15.39:0/2842697994 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33040b4690 con 0x7f3304155e80 2024-09-21T07:42:46.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.374+0000 7f32fb400640 1 -- 172.21.15.39:0/2842697994 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f32f0018020 con 0x7f3304155e80 2024-09-21T07:42:46.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.375+0000 7f32fb400640 1 --2- 172.21.15.39:0/2842697994 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f32e4078b90 0x7f32e407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:46.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.375+0000 7f32fb400640 1 -- 172.21.15.39:0/2842697994 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8033+0+0 (secure 0 0 0) 0x7f32f009b300 con 0x7f3304155e80 2024-09-21T07:42:46.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.376+0000 7f3303400640 1 --2- 172.21.15.39:0/2842697994 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f32e4078b90 0x7f32e407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:46.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.377+0000 7f3303400640 1 --2- 172.21.15.39:0/2842697994 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f32e4078b90 0x7f32e407b0a0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f32f4009aa0 tx=0x7f32f4005db0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:46.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.380+0000 7f32fb400640 1 -- 172.21.15.39:0/2842697994 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f32f001f030 con 0x7f3304155e80 2024-09-21T07:42:46.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.619+0000 7f330a800640 1 -- 172.21.15.39:0/2842697994 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33040708d0 con 0x7f3304155e80 2024-09-21T07:42:46.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.619+0000 7f32fb400640 1 -- 172.21.15.39:0/2842697994 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 140 v140) ==== 95+0+5254 (secure 0 0 0) 0x7f32f0062ea0 con 0x7f3304155e80 2024-09-21T07:42:46.621 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:42:46.622 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":140,"btime":"2024-09-21T07:42:42:091818+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7366,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/759792812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":759792812},{"type":"v1","addr":"172.21.15.148:6837","nonce":759792812}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":139}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:42:41.094897+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7644},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7644":{"gid":7644,"name":"a","rank":1,"incarnation":138,"state":"up:reconnect","state_seq":15,"addr":"172.21.15.39:6835/1326711417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1326711417},{"type":"v1","addr":"172.21.15.39:6835","nonce":1326711417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311]},"id":6}]} 2024-09-21T07:42:46.622 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 140 2024-09-21T07:42:46.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.622+0000 7f330a800640 1 -- 172.21.15.39:0/2842697994 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f32e4078b90 msgr2=0x7f32e407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:46.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.622+0000 7f330a800640 1 --2- 172.21.15.39:0/2842697994 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f32e4078b90 0x7f32e407b0a0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f32f4009aa0 tx=0x7f32f4005db0 comp rx=0 tx=0).stop 2024-09-21T07:42:46.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.622+0000 7f330a800640 1 -- 172.21.15.39:0/2842697994 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3304155e80 msgr2=0x7f33041547c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:46.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.622+0000 7f330a800640 1 --2- 172.21.15.39:0/2842697994 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3304155e80 0x7f33041547c0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f32f000b3f0 tx=0x7f32f000b8b0 comp rx=0 tx=0).stop 2024-09-21T07:42:46.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.622+0000 7f330a800640 1 -- 172.21.15.39:0/2842697994 shutdown_connections 2024-09-21T07:42:46.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.622+0000 7f330a800640 1 --2- 172.21.15.39:0/2842697994 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f32e4078b90 0x7f32e407b0a0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:46.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.622+0000 7f330a800640 1 --2- 172.21.15.39:0/2842697994 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f330415f360 0x7f3304146050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:46.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.622+0000 7f330a800640 1 --2- 172.21.15.39:0/2842697994 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3304155e80 0x7f33041547c0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:46.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.622+0000 7f330a800640 1 --2- 172.21.15.39:0/2842697994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3304155a70 0x7f33040b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:46.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.622+0000 7f330a800640 1 -- 172.21.15.39:0/2842697994 >> 172.21.15.39:0/2842697994 conn(0x7f330406cfc0 msgr2=0x7f33040ce450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:46.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.622+0000 7f330a800640 1 -- 172.21.15.39:0/2842697994 shutdown_connections 2024-09-21T07:42:46.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:46.622+0000 7f330a800640 1 -- 172.21.15.39:0/2842697994 wait complete. 2024-09-21T07:42:47.049 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 140, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:36:20.189287+0000', 'modified': '2024-09-21T07:42:41.094897+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 114, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 7311, 'mds_1': 7644}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7311': {'gid': 7311, 'name': 'c', 'rank': 0, 'incarnation': 119, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.39:6833/1790581246', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 1790581246}, {'type': 'v1', 'addr': '172.21.15.39:6833', 'nonce': 1790581246}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_7644': {'gid': 7644, 'name': 'a', 'rank': 1, 'incarnation': 138, 'state': 'up:reconnect', 'state_seq': 15, 'addr': '172.21.15.39:6835/1326711417', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6834', 'nonce': 1326711417}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 1326711417}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7311, 'qdb_cluster': [7311]} 2024-09-21T07:42:47.049 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7644:up:reconnect 2024-09-21T07:42:48.050 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:42:48.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.287+0000 7f07d2800640 1 -- 172.21.15.39:0/1656564532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07cc0d0f20 msgr2=0x7f07cc15c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:48.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.287+0000 7f07d2800640 1 --2- 172.21.15.39:0/1656564532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07cc0d0f20 0x7f07cc15c830 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f07c0009830 tx=0x7f07c002a510 comp rx=0 tx=0).stop 2024-09-21T07:42:48.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.287+0000 7f07d2800640 1 -- 172.21.15.39:0/1656564532 shutdown_connections 2024-09-21T07:42:48.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.287+0000 7f07d2800640 1 --2- 172.21.15.39:0/1656564532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07cc0d0f20 0x7f07cc15c830 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:48.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.287+0000 7f07d2800640 1 -- 172.21.15.39:0/1656564532 >> 172.21.15.39:0/1656564532 conn(0x7f07cc0b2510 msgr2=0x7f07cc0d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:48.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.287+0000 7f07d2800640 1 -- 172.21.15.39:0/1656564532 shutdown_connections 2024-09-21T07:42:48.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.287+0000 7f07d2800640 1 -- 172.21.15.39:0/1656564532 wait complete. 2024-09-21T07:42:48.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.289+0000 7f07d2800640 1 Processor -- start 2024-09-21T07:42:48.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.289+0000 7f07d2800640 1 -- start start 2024-09-21T07:42:48.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.291+0000 7f07d2800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f07cc0cb960 0x7f07cc1f56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:48.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.291+0000 7f07d2800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07cc0cbd70 0x7f07cc158310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:48.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.291+0000 7f07d2800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07cc0d0f20 0x7f07cc15b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:48.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.291+0000 7f07d2800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc0b4780 con 0x7f07cc0d0f20 2024-09-21T07:42:48.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.291+0000 7f07d2800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc0b4600 con 0x7f07cc0cbd70 2024-09-21T07:42:48.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.291+0000 7f07d2800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f07cc0b4900 con 0x7f07cc0cb960 2024-09-21T07:42:48.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.291+0000 7f07d1e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07cc0d0f20 0x7f07cc15b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:48.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.291+0000 7f07d1e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07cc0d0f20 0x7f07cc15b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48776/0 (socket says 172.21.15.39:48776) 2024-09-21T07:42:48.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.291+0000 7f07d1e00640 1 -- 172.21.15.39:0/773630827 learned_addr learned my addr 172.21.15.39:0/773630827 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:48.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.291+0000 7f07d0a00640 1 --2- 172.21.15.39:0/773630827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07cc0cbd70 0x7f07cc158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:48.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.291+0000 7f07d1400640 1 --2- 172.21.15.39:0/773630827 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f07cc0cb960 0x7f07cc1f56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:48.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.292+0000 7f07d1e00640 1 -- 172.21.15.39:0/773630827 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f07cc0cb960 msgr2=0x7f07cc1f56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:48.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.292+0000 7f07d1e00640 1 --2- 172.21.15.39:0/773630827 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f07cc0cb960 0x7f07cc1f56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:48.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.292+0000 7f07d1e00640 1 -- 172.21.15.39:0/773630827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07cc0cbd70 msgr2=0x7f07cc158310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:48.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.292+0000 7f07d1e00640 1 --2- 172.21.15.39:0/773630827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07cc0cbd70 0x7f07cc158310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:48.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.292+0000 7f07d1e00640 1 -- 172.21.15.39:0/773630827 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07cc1f5fa0 con 0x7f07cc0d0f20 2024-09-21T07:42:48.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.292+0000 7f07d1e00640 1 --2- 172.21.15.39:0/773630827 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07cc0d0f20 0x7f07cc15b210 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f07c400b970 tx=0x7f07c400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:48.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.292+0000 7f07c9600640 1 -- 172.21.15.39:0/773630827 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c4019070 con 0x7f07cc0d0f20 2024-09-21T07:42:48.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.292+0000 7f07c9600640 1 -- 172.21.15.39:0/773630827 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07c400cc30 con 0x7f07cc0d0f20 2024-09-21T07:42:48.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.292+0000 7f07c9600640 1 -- 172.21.15.39:0/773630827 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c40146a0 con 0x7f07cc0d0f20 2024-09-21T07:42:48.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.292+0000 7f07d2800640 1 -- 172.21.15.39:0/773630827 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07cc1f6280 con 0x7f07cc0d0f20 2024-09-21T07:42:48.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.293+0000 7f07d2800640 1 -- 172.21.15.39:0/773630827 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07cc159150 con 0x7f07cc0d0f20 2024-09-21T07:42:48.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.294+0000 7f07d2800640 1 -- 172.21.15.39:0/773630827 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07cc1f5d10 con 0x7f07cc0d0f20 2024-09-21T07:42:48.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.295+0000 7f07c9600640 1 -- 172.21.15.39:0/773630827 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f07c401c070 con 0x7f07cc0d0f20 2024-09-21T07:42:48.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.296+0000 7f07c9600640 1 --2- 172.21.15.39:0/773630827 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f07ac078bc0 0x7f07ac07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:48.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.296+0000 7f07c9600640 1 -- 172.21.15.39:0/773630827 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8033+0+0 (secure 0 0 0) 0x7f07c409b250 con 0x7f07cc0d0f20 2024-09-21T07:42:48.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.297+0000 7f07d1400640 1 --2- 172.21.15.39:0/773630827 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f07ac078bc0 0x7f07ac07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:48.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.297+0000 7f07d1400640 1 --2- 172.21.15.39:0/773630827 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f07ac078bc0 0x7f07ac07b0d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f07c0034000 tx=0x7f07c0004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:48.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.301+0000 7f07c9600640 1 -- 172.21.15.39:0/773630827 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f07c4062710 con 0x7f07cc0d0f20 2024-09-21T07:42:48.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.537+0000 7f07d2800640 1 -- 172.21.15.39:0/773630827 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07cc09a2a0 con 0x7f07cc0d0f20 2024-09-21T07:42:48.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.537+0000 7f07c9600640 1 -- 172.21.15.39:0/773630827 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 140 v140) ==== 95+0+5254 (secure 0 0 0) 0x7f07c4067140 con 0x7f07cc0d0f20 2024-09-21T07:42:48.539 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:42:48.540 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":140,"btime":"2024-09-21T07:42:42:091818+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7366,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/759792812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":759792812},{"type":"v1","addr":"172.21.15.148:6837","nonce":759792812}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":139}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:42:41.094897+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7644},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7644":{"gid":7644,"name":"a","rank":1,"incarnation":138,"state":"up:reconnect","state_seq":15,"addr":"172.21.15.39:6835/1326711417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1326711417},{"type":"v1","addr":"172.21.15.39:6835","nonce":1326711417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311]},"id":6}]} 2024-09-21T07:42:48.540 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 140 2024-09-21T07:42:48.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.539+0000 7f07d2800640 1 -- 172.21.15.39:0/773630827 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f07ac078bc0 msgr2=0x7f07ac07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:48.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.539+0000 7f07d2800640 1 --2- 172.21.15.39:0/773630827 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f07ac078bc0 0x7f07ac07b0d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f07c0034000 tx=0x7f07c0004730 comp rx=0 tx=0).stop 2024-09-21T07:42:48.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.540+0000 7f07d2800640 1 -- 172.21.15.39:0/773630827 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07cc0d0f20 msgr2=0x7f07cc15b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:48.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.540+0000 7f07d2800640 1 --2- 172.21.15.39:0/773630827 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07cc0d0f20 0x7f07cc15b210 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f07c400b970 tx=0x7f07c400be30 comp rx=0 tx=0).stop 2024-09-21T07:42:48.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.540+0000 7f07d2800640 1 -- 172.21.15.39:0/773630827 shutdown_connections 2024-09-21T07:42:48.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.540+0000 7f07d2800640 1 --2- 172.21.15.39:0/773630827 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f07ac078bc0 0x7f07ac07b0d0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:48.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.540+0000 7f07d2800640 1 --2- 172.21.15.39:0/773630827 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07cc0d0f20 0x7f07cc15b210 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:48.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.540+0000 7f07d2800640 1 --2- 172.21.15.39:0/773630827 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07cc0cbd70 0x7f07cc158310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:48.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.540+0000 7f07d2800640 1 --2- 172.21.15.39:0/773630827 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f07cc0cb960 0x7f07cc1f56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:48.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.540+0000 7f07d2800640 1 -- 172.21.15.39:0/773630827 >> 172.21.15.39:0/773630827 conn(0x7f07cc0b2510 msgr2=0x7f07cc06d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:48.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.540+0000 7f07d2800640 1 -- 172.21.15.39:0/773630827 shutdown_connections 2024-09-21T07:42:48.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:48.540+0000 7f07d2800640 1 -- 172.21.15.39:0/773630827 wait complete. 2024-09-21T07:42:48.970 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 140, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:36:20.189287+0000', 'modified': '2024-09-21T07:42:41.094897+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 114, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 7311, 'mds_1': 7644}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7311': {'gid': 7311, 'name': 'c', 'rank': 0, 'incarnation': 119, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.39:6833/1790581246', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 1790581246}, {'type': 'v1', 'addr': '172.21.15.39:6833', 'nonce': 1790581246}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_7644': {'gid': 7644, 'name': 'a', 'rank': 1, 'incarnation': 138, 'state': 'up:reconnect', 'state_seq': 15, 'addr': '172.21.15.39:6835/1326711417', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6834', 'nonce': 1326711417}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 1326711417}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7311, 'qdb_cluster': [7311]} 2024-09-21T07:42:48.970 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7644:up:reconnect 2024-09-21T07:42:49.971 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:42:50.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.208+0000 7fdc25400640 1 -- 172.21.15.39:0/4011695379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc200d0cf0 msgr2=0x7fdc200b6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:50.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.208+0000 7fdc25400640 1 --2- 172.21.15.39:0/4011695379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc200d0cf0 0x7fdc200b6880 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c005fd0 tx=0x7fdc0c02a540 comp rx=0 tx=0).stop 2024-09-21T07:42:50.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.209+0000 7fdc25400640 1 -- 172.21.15.39:0/4011695379 shutdown_connections 2024-09-21T07:42:50.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.209+0000 7fdc25400640 1 --2- 172.21.15.39:0/4011695379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc200d0cf0 0x7fdc200b6880 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:50.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.209+0000 7fdc25400640 1 -- 172.21.15.39:0/4011695379 >> 172.21.15.39:0/4011695379 conn(0x7fdc200b3010 msgr2=0x7fdc200b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:50.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.209+0000 7fdc25400640 1 -- 172.21.15.39:0/4011695379 shutdown_connections 2024-09-21T07:42:50.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.209+0000 7fdc25400640 1 -- 172.21.15.39:0/4011695379 wait complete. 2024-09-21T07:42:50.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.211+0000 7fdc25400640 1 Processor -- start 2024-09-21T07:42:50.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.211+0000 7fdc25400640 1 -- start start 2024-09-21T07:42:50.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.212+0000 7fdc25400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdc200b0e80 0x7fdc2014bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:50.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.212+0000 7fdc25400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc200d0cf0 0x7fdc2014ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:50.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.212+0000 7fdc25400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc2015ecd0 0x7fdc2014efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:50.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.212+0000 7fdc25400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdc200b1b20 con 0x7fdc200d0cf0 2024-09-21T07:42:50.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.212+0000 7fdc25400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdc200b19a0 con 0x7fdc2015ecd0 2024-09-21T07:42:50.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.212+0000 7fdc25400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fdc200b1ca0 con 0x7fdc200b0e80 2024-09-21T07:42:50.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.213+0000 7fdc1de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc200d0cf0 0x7fdc2014ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:50.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.213+0000 7fdc1de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc200d0cf0 0x7fdc2014ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48798/0 (socket says 172.21.15.39:48798) 2024-09-21T07:42:50.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.213+0000 7fdc1de00640 1 -- 172.21.15.39:0/2283937130 learned_addr learned my addr 172.21.15.39:0/2283937130 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:50.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.213+0000 7fdc1f200640 1 --2- 172.21.15.39:0/2283937130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc2015ecd0 0x7fdc2014efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:50.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.213+0000 7fdc1e800640 1 --2- 172.21.15.39:0/2283937130 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdc200b0e80 0x7fdc2014bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:50.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.213+0000 7fdc1de00640 1 -- 172.21.15.39:0/2283937130 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdc200b0e80 msgr2=0x7fdc2014bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:50.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.213+0000 7fdc1de00640 1 --2- 172.21.15.39:0/2283937130 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdc200b0e80 0x7fdc2014bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:50.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.213+0000 7fdc1de00640 1 -- 172.21.15.39:0/2283937130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc2015ecd0 msgr2=0x7fdc2014efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:50.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.213+0000 7fdc1de00640 1 --2- 172.21.15.39:0/2283937130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc2015ecd0 0x7fdc2014efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:50.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.213+0000 7fdc1de00640 1 -- 172.21.15.39:0/2283937130 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc2014c8b0 con 0x7fdc200d0cf0 2024-09-21T07:42:50.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.213+0000 7fdc1e800640 1 --2- 172.21.15.39:0/2283937130 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdc200b0e80 0x7fdc2014bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:42:50.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.213+0000 7fdc1de00640 1 --2- 172.21.15.39:0/2283937130 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc200d0cf0 0x7fdc2014ea80 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c006010 tx=0x7fdc0c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:50.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.214+0000 7fdc16a00640 1 -- 172.21.15.39:0/2283937130 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c03e030 con 0x7fdc200d0cf0 2024-09-21T07:42:50.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.214+0000 7fdc16a00640 1 -- 172.21.15.39:0/2283937130 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc0c0047a0 con 0x7fdc200d0cf0 2024-09-21T07:42:50.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.214+0000 7fdc16a00640 1 -- 172.21.15.39:0/2283937130 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c035040 con 0x7fdc200d0cf0 2024-09-21T07:42:50.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.214+0000 7fdc25400640 1 -- 172.21.15.39:0/2283937130 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc200c9e60 con 0x7fdc200d0cf0 2024-09-21T07:42:50.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.214+0000 7fdc25400640 1 -- 172.21.15.39:0/2283937130 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc200ca100 con 0x7fdc200d0cf0 2024-09-21T07:42:50.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.215+0000 7fdc25400640 1 -- 172.21.15.39:0/2283937130 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc200b6880 con 0x7fdc200d0cf0 2024-09-21T07:42:50.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.216+0000 7fdc16a00640 1 -- 172.21.15.39:0/2283937130 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdc0c041030 con 0x7fdc200d0cf0 2024-09-21T07:42:50.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.217+0000 7fdc16a00640 1 --2- 172.21.15.39:0/2283937130 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdc00078bc0 0x7fdc0007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:50.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.217+0000 7fdc16a00640 1 -- 172.21.15.39:0/2283937130 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8033+0+0 (secure 0 0 0) 0x7fdc0c0bbff0 con 0x7fdc200d0cf0 2024-09-21T07:42:50.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.218+0000 7fdc1e800640 1 --2- 172.21.15.39:0/2283937130 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdc00078bc0 0x7fdc0007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:50.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.218+0000 7fdc1e800640 1 --2- 172.21.15.39:0/2283937130 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdc00078bc0 0x7fdc0007b0d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fdc10006fd0 tx=0x7fdc10008040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:50.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.222+0000 7fdc16a00640 1 -- 172.21.15.39:0/2283937130 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdc0c083a50 con 0x7fdc200d0cf0 2024-09-21T07:42:50.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.464+0000 7fdc25400640 1 -- 172.21.15.39:0/2283937130 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc2014f870 con 0x7fdc200d0cf0 2024-09-21T07:42:50.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.465+0000 7fdc16a00640 1 -- 172.21.15.39:0/2283937130 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 140 v140) ==== 95+0+5254 (secure 0 0 0) 0x7fdc0c08c020 con 0x7fdc200d0cf0 2024-09-21T07:42:50.467 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:42:50.467 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":140,"btime":"2024-09-21T07:42:42:091818+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7366,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/759792812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":759792812},{"type":"v1","addr":"172.21.15.148:6837","nonce":759792812}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":139}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:42:41.094897+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7644},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7644":{"gid":7644,"name":"a","rank":1,"incarnation":138,"state":"up:reconnect","state_seq":15,"addr":"172.21.15.39:6835/1326711417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1326711417},{"type":"v1","addr":"172.21.15.39:6835","nonce":1326711417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311]},"id":6}]} 2024-09-21T07:42:50.467 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 140 2024-09-21T07:42:50.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.467+0000 7fdc25400640 1 -- 172.21.15.39:0/2283937130 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdc00078bc0 msgr2=0x7fdc0007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:50.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.467+0000 7fdc25400640 1 --2- 172.21.15.39:0/2283937130 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdc00078bc0 0x7fdc0007b0d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fdc10006fd0 tx=0x7fdc10008040 comp rx=0 tx=0).stop 2024-09-21T07:42:50.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.467+0000 7fdc25400640 1 -- 172.21.15.39:0/2283937130 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc200d0cf0 msgr2=0x7fdc2014ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:50.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.467+0000 7fdc25400640 1 --2- 172.21.15.39:0/2283937130 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc200d0cf0 0x7fdc2014ea80 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c006010 tx=0x7fdc0c0039e0 comp rx=0 tx=0).stop 2024-09-21T07:42:50.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.467+0000 7fdc25400640 1 -- 172.21.15.39:0/2283937130 shutdown_connections 2024-09-21T07:42:50.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.467+0000 7fdc25400640 1 --2- 172.21.15.39:0/2283937130 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdc00078bc0 0x7fdc0007b0d0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:50.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.467+0000 7fdc25400640 1 --2- 172.21.15.39:0/2283937130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc2015ecd0 0x7fdc2014efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:50.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.467+0000 7fdc25400640 1 --2- 172.21.15.39:0/2283937130 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc200d0cf0 0x7fdc2014ea80 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:50.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.467+0000 7fdc25400640 1 --2- 172.21.15.39:0/2283937130 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdc200b0e80 0x7fdc2014bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:50.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.467+0000 7fdc25400640 1 -- 172.21.15.39:0/2283937130 >> 172.21.15.39:0/2283937130 conn(0x7fdc200b3010 msgr2=0x7fdc2015d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:50.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.467+0000 7fdc25400640 1 -- 172.21.15.39:0/2283937130 shutdown_connections 2024-09-21T07:42:50.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:50.467+0000 7fdc25400640 1 -- 172.21.15.39:0/2283937130 wait complete. 2024-09-21T07:42:50.894 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 140, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:36:20.189287+0000', 'modified': '2024-09-21T07:42:41.094897+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 114, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 7311, 'mds_1': 7644}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7311': {'gid': 7311, 'name': 'c', 'rank': 0, 'incarnation': 119, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.39:6833/1790581246', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 1790581246}, {'type': 'v1', 'addr': '172.21.15.39:6833', 'nonce': 1790581246}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_7644': {'gid': 7644, 'name': 'a', 'rank': 1, 'incarnation': 138, 'state': 'up:reconnect', 'state_seq': 15, 'addr': '172.21.15.39:6835/1326711417', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6834', 'nonce': 1326711417}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 1326711417}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7311, 'qdb_cluster': [7311]} 2024-09-21T07:42:50.894 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7644:up:reconnect 2024-09-21T07:42:51.895 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:42:52.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.130+0000 7f9997200640 1 -- 172.21.15.39:0/1511872997 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99900b30d0 msgr2=0x7f99900b4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:52.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.130+0000 7f9997200640 1 --2- 172.21.15.39:0/1511872997 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99900b30d0 0x7f99900b4510 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f99900b2e30 tx=0x7f998002a540 comp rx=0 tx=0).stop 2024-09-21T07:42:52.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.131+0000 7f9997200640 1 -- 172.21.15.39:0/1511872997 shutdown_connections 2024-09-21T07:42:52.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.131+0000 7f9997200640 1 --2- 172.21.15.39:0/1511872997 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99900b30d0 0x7f99900b4510 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:52.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.131+0000 7f9997200640 1 --2- 172.21.15.39:0/1511872997 >> v2:172.21.15.148:3300/0 conn(0x7f99900cc560 0x7f99900ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:52.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.131+0000 7f9997200640 1 -- 172.21.15.39:0/1511872997 >> 172.21.15.39:0/1511872997 conn(0x7f99900b2510 msgr2=0x7f99900d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:52.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.131+0000 7f9997200640 1 -- 172.21.15.39:0/1511872997 shutdown_connections 2024-09-21T07:42:52.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.131+0000 7f9997200640 1 -- 172.21.15.39:0/1511872997 wait complete. 2024-09-21T07:42:52.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.133+0000 7f9997200640 1 Processor -- start 2024-09-21T07:42:52.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.133+0000 7f9997200640 1 -- start start 2024-09-21T07:42:52.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.135+0000 7f9997200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99900b30d0 0x7f99901f2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:52.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.136+0000 7f9997200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99900b4a50 0x7f99900aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:52.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.136+0000 7f9997200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99900cc560 0x7f99900af330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:52.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.136+0000 7f9997200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f999006fe50 con 0x7f99900cc560 2024-09-21T07:42:52.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.136+0000 7f9997200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f999006fcd0 con 0x7f99900b30d0 2024-09-21T07:42:52.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.136+0000 7f9997200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f999006ffd0 con 0x7f99900b4a50 2024-09-21T07:42:52.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.136+0000 7f9996800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99900cc560 0x7f99900af330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:52.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.136+0000 7f9996800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99900cc560 0x7f99900af330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42714/0 (socket says 172.21.15.39:42714) 2024-09-21T07:42:52.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.136+0000 7f9996800640 1 -- 172.21.15.39:0/1011476567 learned_addr learned my addr 172.21.15.39:0/1011476567 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:52.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.136+0000 7f9995e00640 1 --2- 172.21.15.39:0/1011476567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99900b30d0 0x7f99901f2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:52.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.136+0000 7f9996800640 1 -- 172.21.15.39:0/1011476567 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99900b4a50 msgr2=0x7f99900aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:52.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.136+0000 7f9996800640 1 --2- 172.21.15.39:0/1011476567 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99900b4a50 0x7f99900aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:52.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.136+0000 7f9996800640 1 -- 172.21.15.39:0/1011476567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99900b30d0 msgr2=0x7f99901f2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:52.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.136+0000 7f9996800640 1 --2- 172.21.15.39:0/1011476567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99900b30d0 0x7f99901f2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:52.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.136+0000 7f9996800640 1 -- 172.21.15.39:0/1011476567 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99900c3d30 con 0x7f99900cc560 2024-09-21T07:42:52.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.137+0000 7f9996800640 1 --2- 172.21.15.39:0/1011476567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99900cc560 0x7f99900af330 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f998c00cb10 tx=0x7f998c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:52.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.137+0000 7f998a000640 1 -- 172.21.15.39:0/1011476567 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c016020 con 0x7f99900cc560 2024-09-21T07:42:52.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.137+0000 7f998a000640 1 -- 172.21.15.39:0/1011476567 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f998c00ad00 con 0x7f99900cc560 2024-09-21T07:42:52.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.137+0000 7f9997200640 1 -- 172.21.15.39:0/1011476567 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99901f3040 con 0x7f99900cc560 2024-09-21T07:42:52.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.137+0000 7f998a000640 1 -- 172.21.15.39:0/1011476567 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c012680 con 0x7f99900cc560 2024-09-21T07:42:52.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.137+0000 7f9997200640 1 -- 172.21.15.39:0/1011476567 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99900ae310 con 0x7f99900cc560 2024-09-21T07:42:52.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.138+0000 7f9997200640 1 -- 172.21.15.39:0/1011476567 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99900b4510 con 0x7f99900cc560 2024-09-21T07:42:52.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.140+0000 7f998a000640 1 -- 172.21.15.39:0/1011476567 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f998c019020 con 0x7f99900cc560 2024-09-21T07:42:52.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.140+0000 7f998a000640 1 --2- 172.21.15.39:0/1011476567 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9974078b90 0x7f997407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:52.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.141+0000 7f998a000640 1 -- 172.21.15.39:0/1011476567 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8076+0+0 (secure 0 0 0) 0x7f998c09b9c0 con 0x7f99900cc560 2024-09-21T07:42:52.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.142+0000 7f9995e00640 1 --2- 172.21.15.39:0/1011476567 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9974078b90 0x7f997407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:52.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.143+0000 7f9995e00640 1 --2- 172.21.15.39:0/1011476567 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9974078b90 0x7f997407b0a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f99840058c0 tx=0x7f9984005850 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:52.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.147+0000 7f998a000640 1 -- 172.21.15.39:0/1011476567 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f998c09de20 con 0x7f99900cc560 2024-09-21T07:42:52.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.382+0000 7f9997200640 1 -- 172.21.15.39:0/1011476567 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99900cb8f0 con 0x7f99900cc560 2024-09-21T07:42:52.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.383+0000 7f998a000640 1 -- 172.21.15.39:0/1011476567 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 140 v140) ==== 95+0+5254 (secure 0 0 0) 0x7f998c020030 con 0x7f99900cc560 2024-09-21T07:42:52.385 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:42:52.385 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":140,"btime":"2024-09-21T07:42:42:091818+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7366,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/759792812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":759792812},{"type":"v1","addr":"172.21.15.148:6837","nonce":759792812}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":139}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:42:41.094897+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7644},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7644":{"gid":7644,"name":"a","rank":1,"incarnation":138,"state":"up:reconnect","state_seq":15,"addr":"172.21.15.39:6835/1326711417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1326711417},{"type":"v1","addr":"172.21.15.39:6835","nonce":1326711417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311]},"id":6}]} 2024-09-21T07:42:52.385 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 140 2024-09-21T07:42:52.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.385+0000 7f9997200640 1 -- 172.21.15.39:0/1011476567 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9974078b90 msgr2=0x7f997407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:52.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.385+0000 7f9997200640 1 --2- 172.21.15.39:0/1011476567 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9974078b90 0x7f997407b0a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f99840058c0 tx=0x7f9984005850 comp rx=0 tx=0).stop 2024-09-21T07:42:52.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.385+0000 7f9997200640 1 -- 172.21.15.39:0/1011476567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99900cc560 msgr2=0x7f99900af330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:52.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.385+0000 7f9997200640 1 --2- 172.21.15.39:0/1011476567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99900cc560 0x7f99900af330 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f998c00cb10 tx=0x7f998c00a060 comp rx=0 tx=0).stop 2024-09-21T07:42:52.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.385+0000 7f9997200640 1 -- 172.21.15.39:0/1011476567 shutdown_connections 2024-09-21T07:42:52.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.385+0000 7f9997200640 1 --2- 172.21.15.39:0/1011476567 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9974078b90 0x7f997407b0a0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:52.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.385+0000 7f9997200640 1 --2- 172.21.15.39:0/1011476567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99900cc560 0x7f99900af330 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:52.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.385+0000 7f9997200640 1 --2- 172.21.15.39:0/1011476567 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99900b4a50 0x7f99900aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:52.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.385+0000 7f9997200640 1 --2- 172.21.15.39:0/1011476567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99900b30d0 0x7f99901f2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:52.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.385+0000 7f9997200640 1 -- 172.21.15.39:0/1011476567 >> 172.21.15.39:0/1011476567 conn(0x7f99900b2510 msgr2=0x7f99900ca560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:52.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.385+0000 7f9997200640 1 -- 172.21.15.39:0/1011476567 shutdown_connections 2024-09-21T07:42:52.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:52.386+0000 7f9997200640 1 -- 172.21.15.39:0/1011476567 wait complete. 2024-09-21T07:42:52.815 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 140, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:36:20.189287+0000', 'modified': '2024-09-21T07:42:41.094897+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 114, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 7311, 'mds_1': 7644}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7311': {'gid': 7311, 'name': 'c', 'rank': 0, 'incarnation': 119, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.39:6833/1790581246', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 1790581246}, {'type': 'v1', 'addr': '172.21.15.39:6833', 'nonce': 1790581246}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_7644': {'gid': 7644, 'name': 'a', 'rank': 1, 'incarnation': 138, 'state': 'up:reconnect', 'state_seq': 15, 'addr': '172.21.15.39:6835/1326711417', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6834', 'nonce': 1326711417}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 1326711417}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7311, 'qdb_cluster': [7311]} 2024-09-21T07:42:52.816 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7644:up:reconnect 2024-09-21T07:42:53.817 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:42:54.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.053+0000 7f25a3e00640 1 -- 172.21.15.39:0/3848519171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2588003a00 msgr2=0x7f2588003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:54.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.053+0000 7f25a3e00640 1 --2- 172.21.15.39:0/3848519171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2588003a00 0x7f2588003e70 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f2594009b40 tx=0x7f2588005830 comp rx=0 tx=0).stop 2024-09-21T07:42:54.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.053+0000 7f25a3400640 1 -- 172.21.15.39:0/3848519171 reap_dead start 2024-09-21T07:42:54.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.053+0000 7f25a3e00640 1 -- 172.21.15.39:0/3848519171 shutdown_connections 2024-09-21T07:42:54.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.053+0000 7f25a3e00640 1 -- 172.21.15.39:0/3848519171 >> 172.21.15.39:0/3848519171 conn(0x7f25a408bfb0 msgr2=0x7f25a409d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:54.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.053+0000 7f25a3e00640 1 -- 172.21.15.39:0/3848519171 shutdown_connections 2024-09-21T07:42:54.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.053+0000 7f25a3e00640 1 -- 172.21.15.39:0/3848519171 wait complete. 2024-09-21T07:42:54.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.055+0000 7f25a3e00640 1 Processor -- start 2024-09-21T07:42:54.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.055+0000 7f25a3e00640 1 -- start start 2024-09-21T07:42:54.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.057+0000 7f25a3e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2588003a00 0x7f25a4160680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:54.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.057+0000 7f25a3e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25a414dca0 0x7f25a4160bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:54.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.057+0000 7f25a3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25a4155bc0 0x7f25a40cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:54.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.057+0000 7f25a3e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f25a4162370 con 0x7f25a4155bc0 2024-09-21T07:42:54.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.057+0000 7f25a3e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f25a41621f0 con 0x7f25a414dca0 2024-09-21T07:42:54.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.057+0000 7f25a3e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f25a40b4390 con 0x7f2588003a00 2024-09-21T07:42:54.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.057+0000 7f25a3400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25a4155bc0 0x7f25a40cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:54.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.057+0000 7f25a3400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25a4155bc0 0x7f25a40cd5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42734/0 (socket says 172.21.15.39:42734) 2024-09-21T07:42:54.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.057+0000 7f25a3400640 1 -- 172.21.15.39:0/4006256666 learned_addr learned my addr 172.21.15.39:0/4006256666 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:54.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.057+0000 7f259a000640 1 --2- 172.21.15.39:0/4006256666 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25a414dca0 0x7f25a4160bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:54.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.057+0000 7f25a2a00640 1 --2- 172.21.15.39:0/4006256666 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2588003a00 0x7f25a4160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:54.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.057+0000 7f25a3400640 1 -- 172.21.15.39:0/4006256666 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2588003a00 msgr2=0x7f25a4160680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:54.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.057+0000 7f25a3400640 1 --2- 172.21.15.39:0/4006256666 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2588003a00 0x7f25a4160680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:54.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.058+0000 7f25a3400640 1 -- 172.21.15.39:0/4006256666 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25a414dca0 msgr2=0x7f25a4160bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:54.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.058+0000 7f25a3400640 1 --2- 172.21.15.39:0/4006256666 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25a414dca0 0x7f25a4160bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:54.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.058+0000 7f25a3400640 1 -- 172.21.15.39:0/4006256666 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25a40b6b20 con 0x7f25a4155bc0 2024-09-21T07:42:54.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.058+0000 7f259a000640 1 --2- 172.21.15.39:0/4006256666 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25a414dca0 0x7f25a4160bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:42:54.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.058+0000 7f25a3400640 1 --2- 172.21.15.39:0/4006256666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25a4155bc0 0x7f25a40cd5d0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f25a40ceb60 tx=0x7f259c010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:54.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.058+0000 7f259b400640 1 -- 172.21.15.39:0/4006256666 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f259c00ee50 con 0x7f25a4155bc0 2024-09-21T07:42:54.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.058+0000 7f259b400640 1 -- 172.21.15.39:0/4006256666 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f259c015070 con 0x7f25a4155bc0 2024-09-21T07:42:54.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.058+0000 7f259b400640 1 -- 172.21.15.39:0/4006256666 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f259c004d80 con 0x7f25a4155bc0 2024-09-21T07:42:54.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.058+0000 7f25a3e00640 1 -- 172.21.15.39:0/4006256666 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25a40ccdd0 con 0x7f25a4155bc0 2024-09-21T07:42:54.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.058+0000 7f25a3e00640 1 -- 172.21.15.39:0/4006256666 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25a40b40d0 con 0x7f25a4155bc0 2024-09-21T07:42:54.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.059+0000 7f25a3e00640 1 -- 172.21.15.39:0/4006256666 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25a408c3c0 con 0x7f25a4155bc0 2024-09-21T07:42:54.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.061+0000 7f259b400640 1 -- 172.21.15.39:0/4006256666 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f259c003970 con 0x7f25a4155bc0 2024-09-21T07:42:54.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.062+0000 7f259b400640 1 --2- 172.21.15.39:0/4006256666 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2580078b90 0x7f258007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:54.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.062+0000 7f259b400640 1 -- 172.21.15.39:0/4006256666 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8076+0+0 (secure 0 0 0) 0x7f259c09bc00 con 0x7f25a4155bc0 2024-09-21T07:42:54.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.062+0000 7f25a2a00640 1 --2- 172.21.15.39:0/4006256666 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2580078b90 0x7f258007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:54.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.063+0000 7f25a2a00640 1 --2- 172.21.15.39:0/4006256666 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2580078b90 0x7f258007b0a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2594009b40 tx=0x7f259402d040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:54.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.066+0000 7f259b400640 1 -- 172.21.15.39:0/4006256666 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f25a408c3c0 con 0x7f25a4155bc0 2024-09-21T07:42:54.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.307+0000 7f25a3e00640 1 -- 172.21.15.39:0/4006256666 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f25a40b9c20 con 0x7f25a4155bc0 2024-09-21T07:42:54.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.308+0000 7f259b400640 1 -- 172.21.15.39:0/4006256666 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 142 v142) ==== 95+0+5256 (secure 0 0 0) 0x7f25a40b9c20 con 0x7f25a4155bc0 2024-09-21T07:42:54.310 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:42:54.310 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":142,"btime":"2024-09-21T07:42:54:142083+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7366,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/759792812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":759792812},{"type":"v1","addr":"172.21.15.148:6837","nonce":759792812}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":139}],"filesystems":[{"mdsmap":{"epoch":142,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:42:54.142073+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7644},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7644":{"gid":7644,"name":"a","rank":1,"incarnation":138,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/1326711417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1326711417},{"type":"v1","addr":"172.21.15.39:6835","nonce":1326711417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311,7644]},"id":6}]} 2024-09-21T07:42:54.311 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 142 2024-09-21T07:42:54.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.310+0000 7f25a3e00640 1 -- 172.21.15.39:0/4006256666 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2580078b90 msgr2=0x7f258007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:54.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.310+0000 7f25a3e00640 1 --2- 172.21.15.39:0/4006256666 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2580078b90 0x7f258007b0a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2594009b40 tx=0x7f259402d040 comp rx=0 tx=0).stop 2024-09-21T07:42:54.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.310+0000 7f25a3e00640 1 -- 172.21.15.39:0/4006256666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25a4155bc0 msgr2=0x7f25a40cd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:54.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.310+0000 7f25a3e00640 1 --2- 172.21.15.39:0/4006256666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25a4155bc0 0x7f25a40cd5d0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f25a40ceb60 tx=0x7f259c010f40 comp rx=0 tx=0).stop 2024-09-21T07:42:54.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.310+0000 7f25a3e00640 1 -- 172.21.15.39:0/4006256666 shutdown_connections 2024-09-21T07:42:54.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.310+0000 7f25a3e00640 1 --2- 172.21.15.39:0/4006256666 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2580078b90 0x7f258007b0a0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:54.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.310+0000 7f25a3e00640 1 --2- 172.21.15.39:0/4006256666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25a4155bc0 0x7f25a40cd5d0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:54.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.310+0000 7f25a3e00640 1 --2- 172.21.15.39:0/4006256666 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25a414dca0 0x7f25a4160bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:54.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.310+0000 7f25a3e00640 1 --2- 172.21.15.39:0/4006256666 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2588003a00 0x7f25a4160680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:54.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.310+0000 7f25a3e00640 1 -- 172.21.15.39:0/4006256666 >> 172.21.15.39:0/4006256666 conn(0x7f25a408bfb0 msgr2=0x7f25a40d4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:54.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.310+0000 7f25a3e00640 1 -- 172.21.15.39:0/4006256666 shutdown_connections 2024-09-21T07:42:54.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.310+0000 7f25a3e00640 1 -- 172.21.15.39:0/4006256666 wait complete. 2024-09-21T07:42:54.676 INFO:tasks.ceph.mds.c.smithi039.stderr:2024-09-21T07:42:54.674+0000 7f9202600640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:42:54.740 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 142, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:36:20.189287+0000', 'modified': '2024-09-21T07:42:54.142073+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 114, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 7311, 'mds_1': 7644}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7311': {'gid': 7311, 'name': 'c', 'rank': 0, 'incarnation': 119, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.39:6833/1790581246', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 1790581246}, {'type': 'v1', 'addr': '172.21.15.39:6833', 'nonce': 1790581246}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_7644': {'gid': 7644, 'name': 'a', 'rank': 1, 'incarnation': 138, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.39:6835/1326711417', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6834', 'nonce': 1326711417}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 1326711417}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7311, 'qdb_cluster': [7311, 7644]} 2024-09-21T07:42:54.740 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2024-09-21T07:42:54.740 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T07:42:54.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.968+0000 7fc1a9800640 1 -- 172.21.15.39:0/3582373389 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc1a40cb960 msgr2=0x7fc1a40cbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:54.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.968+0000 7fc1a9800640 1 --2- 172.21.15.39:0/3582373389 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc1a40cb960 0x7fc1a40cbd70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009830 tx=0x7fc18c02a4b0 comp rx=0 tx=0).stop 2024-09-21T07:42:54.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.968+0000 7fc1a9800640 1 -- 172.21.15.39:0/3582373389 shutdown_connections 2024-09-21T07:42:54.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.969+0000 7fc1a9800640 1 --2- 172.21.15.39:0/3582373389 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc1a40cb960 0x7fc1a40cbd70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:54.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.969+0000 7fc1a9800640 1 --2- 172.21.15.39:0/3582373389 >> v2:172.21.15.148:3300/0 conn(0x7fc1a40d0f20 0x7fc1a415c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:54.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.969+0000 7fc1a9800640 1 -- 172.21.15.39:0/3582373389 >> 172.21.15.39:0/3582373389 conn(0x7fc1a40b2510 msgr2=0x7fc1a40d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:54.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.969+0000 7fc1a9800640 1 -- 172.21.15.39:0/3582373389 shutdown_connections 2024-09-21T07:42:54.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.969+0000 7fc1a9800640 1 -- 172.21.15.39:0/3582373389 wait complete. 2024-09-21T07:42:54.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.971+0000 7fc1a9800640 1 Processor -- start 2024-09-21T07:42:54.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.971+0000 7fc1a9800640 1 -- start start 2024-09-21T07:42:54.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.973+0000 7fc1a9800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1a406f8a0 0x7fc1a409c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:54.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.973+0000 7fc1a9800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc1a40cb960 0x7fc1a409d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:54.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.973+0000 7fc1a9800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1a40d0f20 0x7fc1a41f5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:54.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.973+0000 7fc1a9800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a40b4570 con 0x7fc1a406f8a0 2024-09-21T07:42:54.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.973+0000 7fc1a9800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a40b43f0 con 0x7fc1a40d0f20 2024-09-21T07:42:54.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.973+0000 7fc1a9800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc1a40b46f0 con 0x7fc1a40cb960 2024-09-21T07:42:54.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.973+0000 7fc1a3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1a406f8a0 0x7fc1a409c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:54.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.973+0000 7fc1a3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1a406f8a0 0x7fc1a409c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42744/0 (socket says 172.21.15.39:42744) 2024-09-21T07:42:54.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.973+0000 7fc1a3e00640 1 -- 172.21.15.39:0/1136063629 learned_addr learned my addr 172.21.15.39:0/1136063629 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:54.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.973+0000 7fc19b400640 1 --2- 172.21.15.39:0/1136063629 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc1a40cb960 0x7fc1a409d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:54.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.973+0000 7fc1a8e00640 1 --2- 172.21.15.39:0/1136063629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1a40d0f20 0x7fc1a41f5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:54.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.973+0000 7fc1a3e00640 1 -- 172.21.15.39:0/1136063629 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc1a40cb960 msgr2=0x7fc1a409d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:54.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.973+0000 7fc1a3e00640 1 --2- 172.21.15.39:0/1136063629 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc1a40cb960 0x7fc1a409d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:54.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.973+0000 7fc1a3e00640 1 -- 172.21.15.39:0/1136063629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1a40d0f20 msgr2=0x7fc1a41f5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:54.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.973+0000 7fc1a3e00640 1 --2- 172.21.15.39:0/1136063629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1a40d0f20 0x7fc1a41f5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:54.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.973+0000 7fc1a3e00640 1 -- 172.21.15.39:0/1136063629 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a41f6180 con 0x7fc1a406f8a0 2024-09-21T07:42:54.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.973+0000 7fc1a8e00640 1 --2- 172.21.15.39:0/1136063629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1a40d0f20 0x7fc1a41f5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:42:54.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.974+0000 7fc1a3e00640 1 --2- 172.21.15.39:0/1136063629 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1a406f8a0 0x7fc1a409c150 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fc19400ea20 tx=0x7fc19400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:54.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.974+0000 7fc1a0c00640 1 -- 172.21.15.39:0/1136063629 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19400ce40 con 0x7fc1a406f8a0 2024-09-21T07:42:54.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.974+0000 7fc1a0c00640 1 -- 172.21.15.39:0/1136063629 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc194004020 con 0x7fc1a406f8a0 2024-09-21T07:42:54.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.974+0000 7fc1a9800640 1 -- 172.21.15.39:0/1136063629 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1a409b390 con 0x7fc1a406f8a0 2024-09-21T07:42:54.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.974+0000 7fc1a0c00640 1 -- 172.21.15.39:0/1136063629 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194004470 con 0x7fc1a406f8a0 2024-09-21T07:42:54.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.974+0000 7fc1a9800640 1 -- 172.21.15.39:0/1136063629 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1a409b950 con 0x7fc1a406f8a0 2024-09-21T07:42:54.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.976+0000 7fc1a0c00640 1 -- 172.21.15.39:0/1136063629 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc1940035f0 con 0x7fc1a406f8a0 2024-09-21T07:42:54.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.978+0000 7fc1a0c00640 1 --2- 172.21.15.39:0/1136063629 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc180078bc0 0x7fc18007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:54.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.978+0000 7fc1a0c00640 1 -- 172.21.15.39:0/1136063629 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8076+0+0 (secure 0 0 0) 0x7fc19409c180 con 0x7fc1a406f8a0 2024-09-21T07:42:54.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.979+0000 7fc19b400640 1 --2- 172.21.15.39:0/1136063629 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc180078bc0 0x7fc18007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:54.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.979+0000 7fc1b1e2a740 1 Processor -- start 2024-09-21T07:42:54.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.979+0000 7fc1b1e2a740 1 -- start start 2024-09-21T07:42:54.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.979+0000 7fc19b400640 1 --2- 172.21.15.39:0/1136063629 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc180078bc0 0x7fc18007b0d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc18c034000 tx=0x7fc18c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:54.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.980+0000 7fc1b1e2a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d5d7cd270 0x561d5db9db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:54.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.980+0000 7fc1b1e2a740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561d5d7a43b0 0x561d5db9e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:54.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.980+0000 7fc1b1e2a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d5d7ac590 0x561d5db6e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:54.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.980+0000 7fc1b1e2a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561d5daf7430 con 0x561d5d7ac590 2024-09-21T07:42:54.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.980+0000 7fc1b1e2a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561d5dae75d0 con 0x561d5d7cd270 2024-09-21T07:42:54.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.980+0000 7fc1b1e2a740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x561d5dad6a00 con 0x561d5d7a43b0 2024-09-21T07:42:54.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.980+0000 7fc1a8e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d5d7ac590 0x561d5db6e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:54.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.981+0000 7fc1a8e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d5d7ac590 0x561d5db6e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42748/0 (socket says 172.21.15.39:42748) 2024-09-21T07:42:54.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.981+0000 7fc1a8e00640 1 -- 172.21.15.39:0/1667666909 learned_addr learned my addr 172.21.15.39:0/1667666909 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:54.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.981+0000 7fc1a3e00640 1 --2- 172.21.15.39:0/1667666909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d5d7cd270 0x561d5db9db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:54.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.981+0000 7fc19b400640 1 --2- 172.21.15.39:0/1667666909 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561d5d7a43b0 0x561d5db9e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:54.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.981+0000 7fc1a8e00640 1 -- 172.21.15.39:0/1667666909 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561d5d7a43b0 msgr2=0x561d5db9e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:42:54.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.981+0000 7fc1a8e00640 1 --2- 172.21.15.39:0/1667666909 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561d5d7a43b0 0x561d5db9e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:54.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.981+0000 7fc1a8e00640 1 -- 172.21.15.39:0/1667666909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d5d7cd270 msgr2=0x561d5db9db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:42:54.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.981+0000 7fc1a8e00640 1 --2- 172.21.15.39:0/1667666909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d5d7cd270 0x561d5db9db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:54.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.981+0000 7fc1a8e00640 1 -- 172.21.15.39:0/1667666909 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d5da9f520 con 0x561d5d7ac590 2024-09-21T07:42:54.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.981+0000 7fc1a8e00640 1 --2- 172.21.15.39:0/1667666909 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d5d7ac590 0x561d5db6e150 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fc19c001bc0 tx=0x7fc19c00d5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67854744a00059bb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:54.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.981+0000 7fc198c00640 1 -- 172.21.15.39:0/1667666909 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c016020 con 0x561d5d7ac590 2024-09-21T07:42:54.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.981+0000 7fc198c00640 1 -- 172.21.15.39:0/1667666909 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc19c0128f0 con 0x561d5d7ac590 2024-09-21T07:42:54.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.981+0000 7fc198c00640 1 -- 172.21.15.39:0/1667666909 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c012d40 con 0x561d5d7ac590 2024-09-21T07:42:54.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.982+0000 7fc1b1e2a740 1 -- 172.21.15.39:0/1667666909 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d5d7ac590 msgr2=0x561d5db6e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:54.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.982+0000 7fc1b1e2a740 1 --2- 172.21.15.39:0/1667666909 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d5d7ac590 0x561d5db6e150 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fc19c001bc0 tx=0x7fc19c00d5e0 comp rx=0 tx=0).stop 2024-09-21T07:42:54.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.982+0000 7fc1b1e2a740 1 -- 172.21.15.39:0/1667666909 shutdown_connections 2024-09-21T07:42:54.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.982+0000 7fc1b1e2a740 1 --2- 172.21.15.39:0/1667666909 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d5d7ac590 0x561d5db6e150 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:54.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.982+0000 7fc1b1e2a740 1 --2- 172.21.15.39:0/1667666909 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561d5d7a43b0 0x561d5db9e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:54.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.982+0000 7fc1b1e2a740 1 --2- 172.21.15.39:0/1667666909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d5d7cd270 0x561d5db9db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:54.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.982+0000 7fc1b1e2a740 1 -- 172.21.15.39:0/1667666909 >> 172.21.15.39:0/1667666909 conn(0x561d5d7c13c0 msgr2=0x561d5db73b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:54.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.982+0000 7fc1b1e2a740 1 -- 172.21.15.39:0/1667666909 shutdown_connections 2024-09-21T07:42:54.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.982+0000 7fc1b1e2a740 1 -- 172.21.15.39:0/1667666909 wait complete. 2024-09-21T07:42:54.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.983+0000 7fc1b1e2a740 1 -- start start 2024-09-21T07:42:54.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.984+0000 7fc1b1e2a740 1 Processor -- start 2024-09-21T07:42:54.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.985+0000 7fc1b1e2a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d5d7a43b0 0x561d5dbc85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:54.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.985+0000 7fc1b1e2a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d5d7ac590 0x561d5dbcb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:54.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.985+0000 7fc1b1e2a740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561d5d7cd270 0x561d5dbcb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:54.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.985+0000 7fc1b1e2a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561d5da7d330 con 0x561d5d7ac590 2024-09-21T07:42:54.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.985+0000 7fc1b1e2a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561d5dada810 con 0x561d5d7a43b0 2024-09-21T07:42:54.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.985+0000 7fc1b1e2a740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x561d5d847620 con 0x561d5d7cd270 2024-09-21T07:42:54.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.985+0000 7fc1b1e2a740 10 client.0 _ll_register_callbacks cb 0x561d5da82420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:42:54.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.985+0000 7fc19b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d5d7ac590 0x561d5dbcb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:54.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.986+0000 7fc19b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d5d7ac590 0x561d5dbcb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42760/0 (socket says 172.21.15.39:42760) 2024-09-21T07:42:54.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.986+0000 7fc19b400640 1 -- 172.21.15.39:0/3339998991 learned_addr learned my addr 172.21.15.39:0/3339998991 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:54.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.986+0000 7fc1a8e00640 1 --2- 172.21.15.39:0/3339998991 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561d5d7cd270 0x561d5dbcb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:54.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.986+0000 7fc1a3e00640 1 --2- 172.21.15.39:0/3339998991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d5d7a43b0 0x561d5dbc85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:54.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.986+0000 7fc19b400640 1 -- 172.21.15.39:0/3339998991 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561d5d7cd270 msgr2=0x561d5dbcb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:42:54.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.986+0000 7fc19b400640 1 --2- 172.21.15.39:0/3339998991 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561d5d7cd270 0x561d5dbcb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:54.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.986+0000 7fc19b400640 1 -- 172.21.15.39:0/3339998991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d5d7a43b0 msgr2=0x561d5dbc85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:42:54.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.986+0000 7fc19b400640 1 --2- 172.21.15.39:0/3339998991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d5d7a43b0 0x561d5dbc85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:54.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.986+0000 7fc19b400640 1 -- 172.21.15.39:0/3339998991 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d5d838850 con 0x561d5d7ac590 2024-09-21T07:42:54.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.986+0000 7fc19b400640 1 --2- 172.21.15.39:0/3339998991 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d5d7ac590 0x561d5dbcb300 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fc18c02aaa0 tx=0x7fc18c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b5fab84de6e07c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:54.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.986+0000 7fc191600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:42:54.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.986+0000 7fc191600640 1 -- 172.21.15.39:0/3339998991 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c04a840 con 0x561d5d7ac590 2024-09-21T07:42:54.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.987+0000 7fc1b1e2a740 1 -- 172.21.15.39:0/3339998991 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc19c0128f0 con 0x561d5d7ac590 2024-09-21T07:42:54.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.987+0000 7fc191600640 1 -- 172.21.15.39:0/3339998991 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8076+0+0 (secure 0 0 0) 0x7fc18c04a9e0 con 0x561d5d7ac590 2024-09-21T07:42:54.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.987+0000 7fc191600640 1 -- 172.21.15.39:0/3339998991 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=142) ==== 24+0+0 (secure 0 0 0) 0x7fc18c04abe0 con 0x561d5d7ac590 2024-09-21T07:42:54.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.987+0000 7fc1b1e2a740 10 client.7893 fetch_fsmap learned FSMap version 142 2024-09-21T07:42:54.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.987+0000 7fc1b1e2a740 1 -- 172.21.15.39:0/3339998991 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=142,monmap=2+}) -- 0x561d5d8af6b0 con 0x561d5d7ac590 2024-09-21T07:42:54.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.987+0000 7fc191600640 1 -- 172.21.15.39:0/3339998991 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc19c0128f0 con 0x561d5d7ac590 2024-09-21T07:42:54.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.987+0000 7fc191600640 1 -- 172.21.15.39:0/3339998991 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 142) ==== 3522+0+0 (secure 0 0 0) 0x7fc18c053470 con 0x561d5d7ac590 2024-09-21T07:42:54.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.987+0000 7fc1b1e2a740 10 client.7893 fetch_fsmap finished waiting for FSMap version 142 2024-09-21T07:42:54.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.987+0000 7fc1b1e2a740 10 client.7893 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:42:54.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.987+0000 7fc1b1e2a740 20 client.7893 populate_metadata read hostname 'smithi039' 2024-09-21T07:42:54.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.987+0000 7fc1b1e2a740 1 --2- 172.21.15.39:0/3339998991 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x561d5d78be40 0x561d5dbd5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:54.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.987+0000 7fc1b1e2a740 4 client.7893 mds_command: new command op to 7311 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:42:54.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.987+0000 7fc1b1e2a740 1 -- 172.21.15.39:0/3339998991 --> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d5dae9520 con 0x561d5d78be40 2024-09-21T07:42:54.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.988+0000 7fc1a3e00640 1 --2- 172.21.15.39:0/3339998991 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x561d5d78be40 0x561d5dbd5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:54.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.988+0000 7fc1a3e00640 1 --2- 172.21.15.39:0/3339998991 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x561d5d78be40 0x561d5dbd5750 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25acf0ea63460b95 server_cookie=7168631032df8f32 in_seq=0 out_seq=0 2024-09-21T07:42:54.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.988+0000 7fc191600640 10 client.7893 ms_handle_connect on v2:172.21.15.39:6832/1790581246 2024-09-21T07:42:54.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.989+0000 7fc191600640 1 -- 172.21.15.39:0/3339998991 <== mds.0 v2:172.21.15.39:6832/1790581246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561d5dae9520 con 0x561d5d78be40 2024-09-21T07:42:54.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.989+0000 7fc191600640 10 client.7893 handle_command_reply: tid=0 2024-09-21T07:42:54.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.989+0000 7fc1b1e2a740 1 client.7893 shutdown 2024-09-21T07:42:54.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.989+0000 7fc1b1e2a740 1 -- 172.21.15.39:0/3339998991 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d5d7ac590 msgr2=0x561d5dbcb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:54.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.989+0000 7fc1b1e2a740 1 --2- 172.21.15.39:0/3339998991 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d5d7ac590 0x561d5dbcb300 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fc18c02aaa0 tx=0x7fc18c005f90 comp rx=0 tx=0).stop 2024-09-21T07:42:54.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.989+0000 7fc1b1e2a740 1 -- 172.21.15.39:0/3339998991 shutdown_connections 2024-09-21T07:42:54.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.989+0000 7fc1b1e2a740 1 --2- 172.21.15.39:0/3339998991 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561d5d7cd270 0x561d5dbcb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:54.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.989+0000 7fc1b1e2a740 1 --2- 172.21.15.39:0/3339998991 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d5d7ac590 0x561d5dbcb300 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:54.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.989+0000 7fc1b1e2a740 1 --2- 172.21.15.39:0/3339998991 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x561d5d78be40 0x561d5dbd5750 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:54.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.989+0000 7fc1b1e2a740 1 --2- 172.21.15.39:0/3339998991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d5d7a43b0 0x561d5dbc85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:54.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.989+0000 7fc191600640 0 client.7893 ms_handle_reset on v2:172.21.15.39:6832/1790581246 2024-09-21T07:42:54.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.989+0000 7fc1b1e2a740 1 -- 172.21.15.39:0/3339998991 >> 172.21.15.39:0/3339998991 conn(0x561d5d7c13c0 msgr2=0x561d5dbbbef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:54.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.989+0000 7fc1b1e2a740 1 -- 172.21.15.39:0/3339998991 shutdown_connections 2024-09-21T07:42:54.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.989+0000 7fc1b1e2a740 1 -- 172.21.15.39:0/3339998991 wait complete. 2024-09-21T07:42:54.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:54.989+0000 7fc1b1e2a740 20 client.7893 trim_cache size 0 max 16384 2024-09-21T07:42:55.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.009+0000 7fc1b1e2a740 1 Processor -- start 2024-09-21T07:42:55.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.009+0000 7fc1b1e2a740 1 -- start start 2024-09-21T07:42:55.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.010+0000 7fc1b1e2a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d5d7a43b0 0x561d5dc324b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:55.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.010+0000 7fc1b1e2a740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561d5d7ac590 0x561d5db696c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:55.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.010+0000 7fc1b1e2a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d5d7cd270 0x561d5db9dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:55.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.010+0000 7fc1b1e2a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a4205250 con 0x561d5d7cd270 2024-09-21T07:42:55.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.010+0000 7fc1b1e2a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a42035c0 con 0x561d5d7a43b0 2024-09-21T07:42:55.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.010+0000 7fc1b1e2a740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc1a4203c10 con 0x561d5d7ac590 2024-09-21T07:42:55.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.010+0000 7fc1a8e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d5d7cd270 0x561d5db9dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:55.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.011+0000 7fc1a8e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d5d7cd270 0x561d5db9dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42776/0 (socket says 172.21.15.39:42776) 2024-09-21T07:42:55.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.011+0000 7fc1a8e00640 1 -- 172.21.15.39:0/2009532144 learned_addr learned my addr 172.21.15.39:0/2009532144 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:55.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.011+0000 7fc1a3e00640 1 --2- 172.21.15.39:0/2009532144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d5d7a43b0 0x561d5dc324b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:55.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.011+0000 7fc19b400640 1 --2- 172.21.15.39:0/2009532144 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561d5d7ac590 0x561d5db696c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:55.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.011+0000 7fc1a8e00640 1 -- 172.21.15.39:0/2009532144 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561d5d7ac590 msgr2=0x561d5db696c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:42:55.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.011+0000 7fc1a8e00640 1 --2- 172.21.15.39:0/2009532144 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561d5d7ac590 0x561d5db696c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.011+0000 7fc1a8e00640 1 -- 172.21.15.39:0/2009532144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d5d7a43b0 msgr2=0x561d5dc324b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:42:55.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.011+0000 7fc1a8e00640 1 --2- 172.21.15.39:0/2009532144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d5d7a43b0 0x561d5dc324b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.011+0000 7fc1a8e00640 1 -- 172.21.15.39:0/2009532144 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d5db74840 con 0x561d5d7cd270 2024-09-21T07:42:55.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.011+0000 7fc1a8e00640 1 --2- 172.21.15.39:0/2009532144 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d5d7cd270 0x561d5db9dad0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fc19c001b90 tx=0x7fc19c0049c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3841c5c16b35eb70 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:55.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.012+0000 7fc193e00640 1 -- 172.21.15.39:0/2009532144 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c016040 con 0x561d5d7cd270 2024-09-21T07:42:55.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.012+0000 7fc193e00640 1 -- 172.21.15.39:0/2009532144 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc19c001710 con 0x561d5d7cd270 2024-09-21T07:42:55.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.012+0000 7fc193e00640 1 -- 172.21.15.39:0/2009532144 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c003bb0 con 0x561d5d7cd270 2024-09-21T07:42:55.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.012+0000 7fc1b1e2a740 1 -- 172.21.15.39:0/2009532144 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d5d7cd270 msgr2=0x561d5db9dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:55.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.012+0000 7fc1b1e2a740 1 --2- 172.21.15.39:0/2009532144 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d5d7cd270 0x561d5db9dad0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fc19c001b90 tx=0x7fc19c0049c0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.012+0000 7fc1b1e2a740 1 -- 172.21.15.39:0/2009532144 shutdown_connections 2024-09-21T07:42:55.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.012+0000 7fc1b1e2a740 1 --2- 172.21.15.39:0/2009532144 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d5d7cd270 0x561d5db9dad0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.012+0000 7fc1b1e2a740 1 --2- 172.21.15.39:0/2009532144 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561d5d7ac590 0x561d5db696c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.012+0000 7fc1b1e2a740 1 --2- 172.21.15.39:0/2009532144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d5d7a43b0 0x561d5dc324b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.012+0000 7fc1b1e2a740 1 -- 172.21.15.39:0/2009532144 >> 172.21.15.39:0/2009532144 conn(0x561d5d7c13c0 msgr2=0x561d5dbbc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:55.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.013+0000 7fc1b1e2a740 1 -- 172.21.15.39:0/2009532144 shutdown_connections 2024-09-21T07:42:55.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.013+0000 7fc1b1e2a740 1 -- 172.21.15.39:0/2009532144 wait complete. 2024-09-21T07:42:55.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.013+0000 7fc1b1e2a740 1 -- start start 2024-09-21T07:42:55.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.014+0000 7fc1b1e2a740 1 Processor -- start 2024-09-21T07:42:55.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.015+0000 7fc1b1e2a740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561d5d7a43b0 0x561d5db82e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:55.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.015+0000 7fc1b1e2a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d5d7ac590 0x561d5dc11ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:55.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.015+0000 7fc1b1e2a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d5d7cd270 0x561d5db93be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:55.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.015+0000 7fc1b1e2a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561d5db973e0 con 0x561d5d7ac590 2024-09-21T07:42:55.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.015+0000 7fc1b1e2a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561d5db969a0 con 0x561d5d7cd270 2024-09-21T07:42:55.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.015+0000 7fc1b1e2a740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x561d5db850e0 con 0x561d5d7a43b0 2024-09-21T07:42:55.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.015+0000 7fc1b1e2a740 10 client.0 _ll_register_callbacks cb 0x7fc18c0096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:42:55.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.016+0000 7fc19b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d5d7ac590 0x561d5dc11ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:55.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.016+0000 7fc19b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d5d7ac590 0x561d5dc11ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42790/0 (socket says 172.21.15.39:42790) 2024-09-21T07:42:55.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.016+0000 7fc19b400640 1 -- 172.21.15.39:0/3997712492 learned_addr learned my addr 172.21.15.39:0/3997712492 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:55.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.016+0000 7fc1a8e00640 1 --2- 172.21.15.39:0/3997712492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d5d7cd270 0x561d5db93be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:55.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.016+0000 7fc1a3e00640 1 --2- 172.21.15.39:0/3997712492 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561d5d7a43b0 0x561d5db82e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:55.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.016+0000 7fc19b400640 1 -- 172.21.15.39:0/3997712492 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561d5d7a43b0 msgr2=0x561d5db82e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:42:55.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.016+0000 7fc19b400640 1 --2- 172.21.15.39:0/3997712492 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561d5d7a43b0 0x561d5db82e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.016+0000 7fc19b400640 1 -- 172.21.15.39:0/3997712492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d5d7cd270 msgr2=0x561d5db93be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:42:55.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.016+0000 7fc19b400640 1 --2- 172.21.15.39:0/3997712492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d5d7cd270 0x561d5db93be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.016+0000 7fc19b400640 1 -- 172.21.15.39:0/3997712492 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d5dc3c0e0 con 0x561d5d7ac590 2024-09-21T07:42:55.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.016+0000 7fc19b400640 1 --2- 172.21.15.39:0/3997712492 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d5d7ac590 0x561d5dc11ef0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x561d5dbc5760 tx=0x7fc18c04a870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63be8dedd7b73db5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:55.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.016+0000 7fc191600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:42:55.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.017+0000 7fc191600640 1 -- 172.21.15.39:0/3997712492 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c053dd0 con 0x561d5d7ac590 2024-09-21T07:42:55.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.017+0000 7fc191600640 1 -- 172.21.15.39:0/3997712492 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8076+0+0 (secure 0 0 0) 0x7fc18c03c420 con 0x561d5d7ac590 2024-09-21T07:42:55.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.017+0000 7fc1b1e2a740 1 -- 172.21.15.39:0/3997712492 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc19c001710 con 0x561d5d7ac590 2024-09-21T07:42:55.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.017+0000 7fc191600640 1 -- 172.21.15.39:0/3997712492 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=142) ==== 24+0+0 (secure 0 0 0) 0x7fc18c04aa10 con 0x561d5d7ac590 2024-09-21T07:42:55.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.017+0000 7fc1b1e2a740 10 client.7899 fetch_fsmap learned FSMap version 142 2024-09-21T07:42:55.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.017+0000 7fc1b1e2a740 1 -- 172.21.15.39:0/3997712492 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=142,monmap=2+}) -- 0x561d5dc2f4a0 con 0x561d5d7ac590 2024-09-21T07:42:55.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.017+0000 7fc191600640 1 -- 172.21.15.39:0/3997712492 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc19c001710 con 0x561d5d7ac590 2024-09-21T07:42:55.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.017+0000 7fc191600640 1 -- 172.21.15.39:0/3997712492 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 142) ==== 3522+0+0 (secure 0 0 0) 0x7fc18c03c620 con 0x561d5d7ac590 2024-09-21T07:42:55.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.017+0000 7fc1b1e2a740 10 client.7899 fetch_fsmap finished waiting for FSMap version 142 2024-09-21T07:42:55.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.017+0000 7fc1b1e2a740 10 client.7899 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:42:55.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.017+0000 7fc1b1e2a740 20 client.7899 populate_metadata read hostname 'smithi039' 2024-09-21T07:42:55.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.017+0000 7fc1b1e2a740 1 --2- 172.21.15.39:0/3997712492 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x561d5dc2ee90 0x561d5dbcad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:55.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.018+0000 7fc1b1e2a740 4 client.7899 mds_command: new command op to 7311 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:42:55.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.018+0000 7fc1b1e2a740 1 -- 172.21.15.39:0/3997712492 --> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] -- command(tid 0: {"prefix": "status"}) -- 0x561d5dae4fd0 con 0x561d5dc2ee90 2024-09-21T07:42:55.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.018+0000 7fc1a3e00640 1 --2- 172.21.15.39:0/3997712492 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x561d5dc2ee90 0x561d5dbcad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:55.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.018+0000 7fc1a3e00640 1 --2- 172.21.15.39:0/3997712492 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x561d5dc2ee90 0x561d5dbcad20 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b631e31db1a6c8c0 server_cookie=dbf6ef5871deeb33 in_seq=0 out_seq=0 2024-09-21T07:42:55.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.018+0000 7fc191600640 10 client.7899 ms_handle_connect on v2:172.21.15.39:6832/1790581246 2024-09-21T07:42:55.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.019+0000 7fc191600640 1 -- 172.21.15.39:0/3997712492 <== mds.0 v2:172.21.15.39:6832/1790581246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+329 (crc 0 0 0) 0x561d5dae4fd0 con 0x561d5dc2ee90 2024-09-21T07:42:55.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.019+0000 7fc191600640 10 client.7899 handle_command_reply: tid=0 2024-09-21T07:42:55.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.019+0000 7fc1b1e2a740 1 client.7899 shutdown 2024-09-21T07:42:55.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.019+0000 7fc1b1e2a740 1 -- 172.21.15.39:0/3997712492 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d5d7ac590 msgr2=0x561d5dc11ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:55.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.019+0000 7fc1b1e2a740 1 --2- 172.21.15.39:0/3997712492 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d5d7ac590 0x561d5dc11ef0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x561d5dbc5760 tx=0x7fc18c04a870 comp rx=0 tx=0).stop 2024-09-21T07:42:55.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.019+0000 7fc1b1e2a740 1 -- 172.21.15.39:0/3997712492 shutdown_connections 2024-09-21T07:42:55.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.019+0000 7fc1b1e2a740 1 --2- 172.21.15.39:0/3997712492 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x561d5dc2ee90 0x561d5dbcad20 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.019+0000 7fc1b1e2a740 1 --2- 172.21.15.39:0/3997712492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d5d7cd270 0x561d5db93be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.019+0000 7fc1b1e2a740 1 --2- 172.21.15.39:0/3997712492 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561d5d7ac590 0x561d5dc11ef0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.019+0000 7fc1b1e2a740 1 --2- 172.21.15.39:0/3997712492 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561d5d7a43b0 0x561d5db82e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.019+0000 7fc1b1e2a740 1 -- 172.21.15.39:0/3997712492 >> 172.21.15.39:0/3997712492 conn(0x561d5d7c13c0 msgr2=0x561d5dbbc370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:55.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.019+0000 7fc191600640 0 client.7899 ms_handle_reset on v2:172.21.15.39:6832/1790581246 2024-09-21T07:42:55.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.019+0000 7fc1b1e2a740 1 -- 172.21.15.39:0/3997712492 shutdown_connections 2024-09-21T07:42:55.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.019+0000 7fc1b1e2a740 1 -- 172.21.15.39:0/3997712492 wait complete. 2024-09-21T07:42:55.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.019+0000 7fc1b1e2a740 20 client.7899 trim_cache size 0 max 16384 2024-09-21T07:42:55.024 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:42:55.024 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:42:55.025 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-09-21T07:42:55.025 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 7311, 2024-09-21T07:42:55.025 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:42:55.025 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:42:55.025 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:42:55.025 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 117.634479591, 2024-09-21T07:42:55.025 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 142, 2024-09-21T07:42:55.025 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 115, 2024-09-21T07:42:55.025 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 115, 2024-09-21T07:42:55.025 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 166.14192577899999 2024-09-21T07:42:55.025 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:42:55.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.020+0000 7fc1a9800640 1 -- 172.21.15.39:0/1136063629 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc180078bc0 msgr2=0x7fc18007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:55.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.020+0000 7fc1a9800640 1 --2- 172.21.15.39:0/1136063629 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc180078bc0 0x7fc18007b0d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc18c034000 tx=0x7fc18c004730 comp rx=0 tx=0).stop 2024-09-21T07:42:55.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.020+0000 7fc1a9800640 1 -- 172.21.15.39:0/1136063629 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1a406f8a0 msgr2=0x7fc1a409c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:55.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.020+0000 7fc1a9800640 1 --2- 172.21.15.39:0/1136063629 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1a406f8a0 0x7fc1a409c150 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fc19400ea20 tx=0x7fc19400eee0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.020+0000 7fc1a9800640 1 -- 172.21.15.39:0/1136063629 shutdown_connections 2024-09-21T07:42:55.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.020+0000 7fc1a9800640 1 --2- 172.21.15.39:0/1136063629 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc180078bc0 0x7fc18007b0d0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.020+0000 7fc1a9800640 1 --2- 172.21.15.39:0/1136063629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1a40d0f20 0x7fc1a41f5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.020+0000 7fc1a9800640 1 --2- 172.21.15.39:0/1136063629 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc1a40cb960 0x7fc1a409d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.020+0000 7fc1a9800640 1 --2- 172.21.15.39:0/1136063629 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1a406f8a0 0x7fc1a409c150 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.020+0000 7fc1a9800640 1 -- 172.21.15.39:0/1136063629 >> 172.21.15.39:0/1136063629 conn(0x7fc1a40b2510 msgr2=0x7fc1a4073990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:55.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.020+0000 7fc1a9800640 1 -- 172.21.15.39:0/1136063629 shutdown_connections 2024-09-21T07:42:55.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.020+0000 7fc1a9800640 1 -- 172.21.15.39:0/1136063629 wait complete. 2024-09-21T07:42:55.044 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T07:42:55.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.270+0000 7f8392800640 1 -- 172.21.15.39:0/4015424152 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f838c15cd30 msgr2=0x7f838c0cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:55.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.270+0000 7f8392800640 1 --2- 172.21.15.39:0/4015424152 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f838c15cd30 0x7f838c0cfd10 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f837c009830 tx=0x7f837c02a510 comp rx=0 tx=0).stop 2024-09-21T07:42:55.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.271+0000 7f8392800640 1 -- 172.21.15.39:0/4015424152 shutdown_connections 2024-09-21T07:42:55.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.271+0000 7f8392800640 1 --2- 172.21.15.39:0/4015424152 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f838c06f800 0x7f838c0cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.271+0000 7f8392800640 1 --2- 172.21.15.39:0/4015424152 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f838c15cd30 0x7f838c0cfd10 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.271+0000 7f8392800640 1 -- 172.21.15.39:0/4015424152 >> 172.21.15.39:0/4015424152 conn(0x7f838c06cfc0 msgr2=0x7f838c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:55.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.271+0000 7f8392800640 1 -- 172.21.15.39:0/4015424152 shutdown_connections 2024-09-21T07:42:55.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.271+0000 7f8392800640 1 -- 172.21.15.39:0/4015424152 wait complete. 2024-09-21T07:42:55.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.273+0000 7f8392800640 1 Processor -- start 2024-09-21T07:42:55.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.273+0000 7f8392800640 1 -- start start 2024-09-21T07:42:55.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.274+0000 7f8392800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f838c06f800 0x7f838c0ae240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:55.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.274+0000 7f8392800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f838c06fc10 0x7f838c1525a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:55.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.274+0000 7f8392800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f838c15cd30 0x7f838c159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:55.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.275+0000 7f8392800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f838c073890 con 0x7f838c06fc10 2024-09-21T07:42:55.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.275+0000 7f8392800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f838c073710 con 0x7f838c06f800 2024-09-21T07:42:55.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.275+0000 7f8392800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f838c073a10 con 0x7f838c15cd30 2024-09-21T07:42:55.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.275+0000 7f838aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f838c06fc10 0x7f838c1525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:55.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.275+0000 7f838aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f838c06fc10 0x7f838c1525a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42806/0 (socket says 172.21.15.39:42806) 2024-09-21T07:42:55.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.275+0000 7f838aa00640 1 -- 172.21.15.39:0/1835802400 learned_addr learned my addr 172.21.15.39:0/1835802400 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:55.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.275+0000 7f838be00640 1 --2- 172.21.15.39:0/1835802400 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f838c15cd30 0x7f838c159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:55.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.275+0000 7f838b400640 1 --2- 172.21.15.39:0/1835802400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f838c06f800 0x7f838c0ae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:55.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.275+0000 7f838aa00640 1 -- 172.21.15.39:0/1835802400 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f838c15cd30 msgr2=0x7f838c159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:55.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.275+0000 7f838aa00640 1 --2- 172.21.15.39:0/1835802400 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f838c15cd30 0x7f838c159170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.275+0000 7f838aa00640 1 -- 172.21.15.39:0/1835802400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f838c06f800 msgr2=0x7f838c0ae240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:55.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.275+0000 7f838aa00640 1 --2- 172.21.15.39:0/1835802400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f838c06f800 0x7f838c0ae240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.275+0000 7f838aa00640 1 -- 172.21.15.39:0/1835802400 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f838c1f9a00 con 0x7f838c06fc10 2024-09-21T07:42:55.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.275+0000 7f838b400640 1 --2- 172.21.15.39:0/1835802400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f838c06f800 0x7f838c0ae240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:42:55.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.275+0000 7f838aa00640 1 --2- 172.21.15.39:0/1835802400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f838c06fc10 0x7f838c1525a0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f837800e790 tx=0x7f837800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:55.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.276+0000 7f8383400640 1 -- 172.21.15.39:0/1835802400 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f837800eea0 con 0x7f838c06fc10 2024-09-21T07:42:55.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.276+0000 7f8383400640 1 -- 172.21.15.39:0/1835802400 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f837800cdb0 con 0x7f838c06fc10 2024-09-21T07:42:55.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.276+0000 7f8383400640 1 -- 172.21.15.39:0/1835802400 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8378010690 con 0x7f838c06fc10 2024-09-21T07:42:55.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.276+0000 7f8392800640 1 -- 172.21.15.39:0/1835802400 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f838c0ae780 con 0x7f838c06fc10 2024-09-21T07:42:55.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.276+0000 7f8392800640 1 -- 172.21.15.39:0/1835802400 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f838c1f8f20 con 0x7f838c06fc10 2024-09-21T07:42:55.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.280+0000 7f8383400640 1 -- 172.21.15.39:0/1835802400 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8378010830 con 0x7f838c06fc10 2024-09-21T07:42:55.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.280+0000 7f8383400640 1 --2- 172.21.15.39:0/1835802400 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f836c078bc0 0x7f836c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:55.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.281+0000 7f8383400640 1 -- 172.21.15.39:0/1835802400 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8076+0+0 (secure 0 0 0) 0x7f837809c0c0 con 0x7f838c06fc10 2024-09-21T07:42:55.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.281+0000 7f8399818740 1 Processor -- start 2024-09-21T07:42:55.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.281+0000 7f8399818740 1 -- start start 2024-09-21T07:42:55.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.281+0000 7f838b400640 1 --2- 172.21.15.39:0/1835802400 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f836c078bc0 0x7f836c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:55.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.282+0000 7f838b400640 1 --2- 172.21.15.39:0/1835802400 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f836c078bc0 0x7f836c07b0d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f837c034000 tx=0x7f837c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:55.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.282+0000 7f8399818740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239afef270 0x56239b3bf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:55.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.282+0000 7f8399818740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56239afc63b0 0x56239b3bfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:55.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.282+0000 7f8399818740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239afce590 0x56239b38ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:55.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.282+0000 7f8399818740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56239b319430 con 0x56239afef270 2024-09-21T07:42:55.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.282+0000 7f8399818740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56239b3095d0 con 0x56239afce590 2024-09-21T07:42:55.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.282+0000 7f8399818740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x56239b2f8a00 con 0x56239afc63b0 2024-09-21T07:42:55.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.282+0000 7f838b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239afef270 0x56239b3bf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:55.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.282+0000 7f838b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239afef270 0x56239b3bf970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42814/0 (socket says 172.21.15.39:42814) 2024-09-21T07:42:55.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.282+0000 7f838b400640 1 -- 172.21.15.39:0/668342472 learned_addr learned my addr 172.21.15.39:0/668342472 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:55.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.283+0000 7f838be00640 1 --2- 172.21.15.39:0/668342472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239afce590 0x56239b38ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:55.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.283+0000 7f838aa00640 1 --2- 172.21.15.39:0/668342472 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56239afc63b0 0x56239b3bfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:55.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.283+0000 7f838b400640 1 -- 172.21.15.39:0/668342472 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56239afc63b0 msgr2=0x56239b3bfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:42:55.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.283+0000 7f838b400640 1 --2- 172.21.15.39:0/668342472 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56239afc63b0 0x56239b3bfeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.283+0000 7f838b400640 1 -- 172.21.15.39:0/668342472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239afce590 msgr2=0x56239b38ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:42:55.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.283+0000 7f838b400640 1 --2- 172.21.15.39:0/668342472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239afce590 0x56239b38ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.283+0000 7f838b400640 1 -- 172.21.15.39:0/668342472 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56239b2c1520 con 0x56239afef270 2024-09-21T07:42:55.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.283+0000 7f838aa00640 1 --2- 172.21.15.39:0/668342472 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56239afc63b0 0x56239b3bfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:42:55.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.283+0000 7f838b400640 1 --2- 172.21.15.39:0/668342472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239afef270 0x56239b3bf970 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f837c0095f0 tx=0x7f837c03c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce3714dfcb9e0cc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:55.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.284+0000 7f8377e00640 1 -- 172.21.15.39:0/668342472 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f837c04c070 con 0x56239afef270 2024-09-21T07:42:55.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.284+0000 7f8377e00640 1 -- 172.21.15.39:0/668342472 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f837c003cc0 con 0x56239afef270 2024-09-21T07:42:55.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.284+0000 7f8377e00640 1 -- 172.21.15.39:0/668342472 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f837c048450 con 0x56239afef270 2024-09-21T07:42:55.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.284+0000 7f8399818740 1 -- 172.21.15.39:0/668342472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239afef270 msgr2=0x56239b3bf970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:55.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.284+0000 7f8399818740 1 --2- 172.21.15.39:0/668342472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239afef270 0x56239b3bf970 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f837c0095f0 tx=0x7f837c03c000 comp rx=0 tx=0).stop 2024-09-21T07:42:55.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.285+0000 7f8399818740 1 -- 172.21.15.39:0/668342472 shutdown_connections 2024-09-21T07:42:55.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.285+0000 7f8399818740 1 --2- 172.21.15.39:0/668342472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239afce590 0x56239b38ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.285+0000 7f8399818740 1 --2- 172.21.15.39:0/668342472 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56239afc63b0 0x56239b3bfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.285+0000 7f8399818740 1 --2- 172.21.15.39:0/668342472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239afef270 0x56239b3bf970 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.285+0000 7f8399818740 1 -- 172.21.15.39:0/668342472 >> 172.21.15.39:0/668342472 conn(0x56239afe33c0 msgr2=0x56239b395970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:55.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.285+0000 7f8399818740 1 -- 172.21.15.39:0/668342472 shutdown_connections 2024-09-21T07:42:55.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.285+0000 7f8399818740 1 -- 172.21.15.39:0/668342472 wait complete. 2024-09-21T07:42:55.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.286+0000 7f8399818740 1 -- start start 2024-09-21T07:42:55.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.286+0000 7f8399818740 1 Processor -- start 2024-09-21T07:42:55.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.287+0000 7f8399818740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56239afc63b0 0x56239b3baab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:55.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.288+0000 7f8399818740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239afce590 0x56239b3edbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:55.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.288+0000 7f8399818740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239afef270 0x56239b3b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:55.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.288+0000 7f8399818740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56239b29f330 con 0x56239afce590 2024-09-21T07:42:55.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.288+0000 7f8399818740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56239b2fc810 con 0x56239afef270 2024-09-21T07:42:55.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.288+0000 7f8399818740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x56239b069620 con 0x56239afc63b0 2024-09-21T07:42:55.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.288+0000 7f8399818740 10 client.0 _ll_register_callbacks cb 0x56239b2a4420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:42:55.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.288+0000 7f838aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239afce590 0x56239b3edbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:55.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.288+0000 7f838b400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56239afc63b0 0x56239b3baab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:55.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.288+0000 7f838aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239afce590 0x56239b3edbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42822/0 (socket says 172.21.15.39:42822) 2024-09-21T07:42:55.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.288+0000 7f838b400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56239afc63b0 0x56239b3baab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.39:55510/0 (socket says 172.21.15.39:55510) 2024-09-21T07:42:55.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.288+0000 7f838aa00640 1 -- 172.21.15.39:0/293302581 learned_addr learned my addr 172.21.15.39:0/293302581 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:55.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.288+0000 7f838aa00640 1 -- 172.21.15.39:0/293302581 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56239afc63b0 msgr2=0x56239b3baab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:42:55.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.288+0000 7f838aa00640 1 --2- 172.21.15.39:0/293302581 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56239afc63b0 0x56239b3baab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.288+0000 7f838aa00640 1 -- 172.21.15.39:0/293302581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239afef270 msgr2=0x56239b3b60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:42:55.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.288+0000 7f838aa00640 1 --2- 172.21.15.39:0/293302581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239afef270 0x56239b3b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.288+0000 7f838aa00640 1 -- 172.21.15.39:0/293302581 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56239b05a850 con 0x56239afce590 2024-09-21T07:42:55.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.289+0000 7f838aa00640 1 --2- 172.21.15.39:0/293302581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239afce590 0x56239b3edbd0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f8378009fd0 tx=0x7f8378063ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d448cfd89f7e00b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:55.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.289+0000 7f8374c00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:42:55.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.289+0000 7f8374c00640 1 -- 172.21.15.39:0/293302581 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83780634c0 con 0x56239afce590 2024-09-21T07:42:55.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.289+0000 7f8374c00640 1 -- 172.21.15.39:0/293302581 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8076+0+0 (secure 0 0 0) 0x7f837805ee10 con 0x56239afce590 2024-09-21T07:42:55.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.289+0000 7f8399818740 1 -- 172.21.15.39:0/293302581 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f837c003cc0 con 0x56239afce590 2024-09-21T07:42:55.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.289+0000 7f8374c00640 1 -- 172.21.15.39:0/293302581 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=142) ==== 24+0+0 (secure 0 0 0) 0x7f8378067fa0 con 0x56239afce590 2024-09-21T07:42:55.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.289+0000 7f8399818740 10 client.7914 fetch_fsmap learned FSMap version 142 2024-09-21T07:42:55.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.289+0000 7f8399818740 1 -- 172.21.15.39:0/293302581 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=142,monmap=2+}) -- 0x56239b0d16b0 con 0x56239afce590 2024-09-21T07:42:55.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.289+0000 7f8374c00640 1 -- 172.21.15.39:0/293302581 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f837c003cc0 con 0x56239afce590 2024-09-21T07:42:55.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.290+0000 7f8374c00640 1 -- 172.21.15.39:0/293302581 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 142) ==== 3522+0+0 (secure 0 0 0) 0x7f837805f010 con 0x56239afce590 2024-09-21T07:42:55.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.290+0000 7f8399818740 10 client.7914 fetch_fsmap finished waiting for FSMap version 142 2024-09-21T07:42:55.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.290+0000 7f8399818740 10 client.7914 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:42:55.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.290+0000 7f8399818740 20 client.7914 populate_metadata read hostname 'smithi039' 2024-09-21T07:42:55.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.290+0000 7f8399818740 1 --2- 172.21.15.39:0/293302581 >> [v2:172.21.15.39:6834/1326711417,v1:172.21.15.39:6835/1326711417] conn(0x56239afade40 0x56239b3b8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:55.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.290+0000 7f8399818740 4 client.7914 mds_command: new command op to 7644 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:42:55.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.290+0000 7f8399818740 1 -- 172.21.15.39:0/293302581 --> [v2:172.21.15.39:6834/1326711417,v1:172.21.15.39:6835/1326711417] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56239b30b520 con 0x56239afade40 2024-09-21T07:42:55.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.290+0000 7f838b400640 1 --2- 172.21.15.39:0/293302581 >> [v2:172.21.15.39:6834/1326711417,v1:172.21.15.39:6835/1326711417] conn(0x56239afade40 0x56239b3b8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:55.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.291+0000 7f838b400640 1 --2- 172.21.15.39:0/293302581 >> [v2:172.21.15.39:6834/1326711417,v1:172.21.15.39:6835/1326711417] conn(0x56239afade40 0x56239b3b8cd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b539c8dff48177d5 server_cookie=1f6ee4b355165848 in_seq=0 out_seq=0 2024-09-21T07:42:55.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.291+0000 7f8374c00640 10 client.7914 ms_handle_connect on v2:172.21.15.39:6834/1326711417 2024-09-21T07:42:55.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.291+0000 7f8374c00640 1 -- 172.21.15.39:0/293302581 <== mds.1 v2:172.21.15.39:6834/1326711417 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56239b30b520 con 0x56239afade40 2024-09-21T07:42:55.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.291+0000 7f8374c00640 10 client.7914 handle_command_reply: tid=0 2024-09-21T07:42:55.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.291+0000 7f8399818740 1 client.7914 shutdown 2024-09-21T07:42:55.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.292+0000 7f8399818740 1 -- 172.21.15.39:0/293302581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239afce590 msgr2=0x56239b3edbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:55.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.292+0000 7f8399818740 1 --2- 172.21.15.39:0/293302581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239afce590 0x56239b3edbd0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f8378009fd0 tx=0x7f8378063ae0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.292+0000 7f8399818740 1 -- 172.21.15.39:0/293302581 shutdown_connections 2024-09-21T07:42:55.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.292+0000 7f8399818740 1 --2- 172.21.15.39:0/293302581 >> [v2:172.21.15.39:6834/1326711417,v1:172.21.15.39:6835/1326711417] conn(0x56239afade40 0x56239b3b8cd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.292+0000 7f8399818740 1 --2- 172.21.15.39:0/293302581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239afef270 0x56239b3b60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.292+0000 7f8399818740 1 --2- 172.21.15.39:0/293302581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239afce590 0x56239b3edbd0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.292+0000 7f8399818740 1 --2- 172.21.15.39:0/293302581 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56239afc63b0 0x56239b3baab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.292+0000 7f8399818740 1 -- 172.21.15.39:0/293302581 >> 172.21.15.39:0/293302581 conn(0x56239afe33c0 msgr2=0x56239b3b6e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:55.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.292+0000 7f8374c00640 0 client.7914 ms_handle_reset on v2:172.21.15.39:6834/1326711417 2024-09-21T07:42:55.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.293+0000 7f8399818740 1 -- 172.21.15.39:0/293302581 shutdown_connections 2024-09-21T07:42:55.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.293+0000 7f8399818740 1 -- 172.21.15.39:0/293302581 wait complete. 2024-09-21T07:42:55.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.293+0000 7f8399818740 20 client.7914 trim_cache size 0 max 16384 2024-09-21T07:42:55.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.312+0000 7f8399818740 1 Processor -- start 2024-09-21T07:42:55.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.312+0000 7f8399818740 1 -- start start 2024-09-21T07:42:55.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.314+0000 7f8399818740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239afc63b0 0x56239b396a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:55.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.314+0000 7f8399818740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56239afce590 0x56239b3910c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:55.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.314+0000 7f8399818740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239afef270 0x56239b450f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:55.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.314+0000 7f8399818740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f838c205270 con 0x56239afc63b0 2024-09-21T07:42:55.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.314+0000 7f8399818740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f838c2035a0 con 0x56239afef270 2024-09-21T07:42:55.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.314+0000 7f8399818740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f838c203bf0 con 0x56239afce590 2024-09-21T07:42:55.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.314+0000 7f838b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239afc63b0 0x56239b396a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:55.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.314+0000 7f838b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239afc63b0 0x56239b396a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42838/0 (socket says 172.21.15.39:42838) 2024-09-21T07:42:55.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.314+0000 7f838b400640 1 -- 172.21.15.39:0/1371523333 learned_addr learned my addr 172.21.15.39:0/1371523333 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:55.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.314+0000 7f838aa00640 1 --2- 172.21.15.39:0/1371523333 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56239afce590 0x56239b3910c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:55.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.314+0000 7f838be00640 1 --2- 172.21.15.39:0/1371523333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239afef270 0x56239b450f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:55.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.314+0000 7f838b400640 1 -- 172.21.15.39:0/1371523333 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56239afce590 msgr2=0x56239b3910c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:42:55.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.314+0000 7f838b400640 1 --2- 172.21.15.39:0/1371523333 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56239afce590 0x56239b3910c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.314+0000 7f838b400640 1 -- 172.21.15.39:0/1371523333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239afef270 msgr2=0x56239b450f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:42:55.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.314+0000 7f838b400640 1 --2- 172.21.15.39:0/1371523333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239afef270 0x56239b450f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.315+0000 7f838b400640 1 -- 172.21.15.39:0/1371523333 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56239b3baa70 con 0x56239afc63b0 2024-09-21T07:42:55.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.315+0000 7f838b400640 1 --2- 172.21.15.39:0/1371523333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239afc63b0 0x56239b396a30 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f837c0095f0 tx=0x7f837c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f8ed772979d2a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:55.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.315+0000 7f8377400640 1 -- 172.21.15.39:0/1371523333 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f837c04c070 con 0x56239afc63b0 2024-09-21T07:42:55.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.315+0000 7f8377400640 1 -- 172.21.15.39:0/1371523333 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f837c003ec0 con 0x56239afc63b0 2024-09-21T07:42:55.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.315+0000 7f8377400640 1 -- 172.21.15.39:0/1371523333 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f837c03f040 con 0x56239afc63b0 2024-09-21T07:42:55.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.316+0000 7f8399818740 1 -- 172.21.15.39:0/1371523333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239afc63b0 msgr2=0x56239b396a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:55.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.316+0000 7f8399818740 1 --2- 172.21.15.39:0/1371523333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239afc63b0 0x56239b396a30 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f837c0095f0 tx=0x7f837c0046c0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.316+0000 7f8399818740 1 -- 172.21.15.39:0/1371523333 shutdown_connections 2024-09-21T07:42:55.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.316+0000 7f8399818740 1 --2- 172.21.15.39:0/1371523333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239afef270 0x56239b450f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.316+0000 7f8399818740 1 --2- 172.21.15.39:0/1371523333 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56239afce590 0x56239b3910c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.316+0000 7f8399818740 1 --2- 172.21.15.39:0/1371523333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239afc63b0 0x56239b396a30 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.316+0000 7f8399818740 1 -- 172.21.15.39:0/1371523333 >> 172.21.15.39:0/1371523333 conn(0x56239afe33c0 msgr2=0x56239b395940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:55.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.316+0000 7f8399818740 1 -- 172.21.15.39:0/1371523333 shutdown_connections 2024-09-21T07:42:55.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.316+0000 7f8399818740 1 -- 172.21.15.39:0/1371523333 wait complete. 2024-09-21T07:42:55.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.317+0000 7f8399818740 1 -- start start 2024-09-21T07:42:55.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.317+0000 7f8399818740 1 Processor -- start 2024-09-21T07:42:55.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.319+0000 7f8399818740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239afc63b0 0x56239b4406b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:55.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.319+0000 7f8399818740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56239afce590 0x56239b3ecf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:55.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.319+0000 7f8399818740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239afef270 0x56239b3ed460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:55.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.319+0000 7f8399818740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56239b3bb100 con 0x56239afef270 2024-09-21T07:42:55.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.319+0000 7f8399818740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56239b3d5d30 con 0x56239afc63b0 2024-09-21T07:42:55.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.319+0000 7f8399818740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x56239b3d5150 con 0x56239afce590 2024-09-21T07:42:55.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.319+0000 7f8399818740 10 client.0 _ll_register_callbacks cb 0x7f8378062cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:42:55.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.319+0000 7f838be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239afef270 0x56239b3ed460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:55.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.319+0000 7f838b400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239afc63b0 0x56239b4406b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:55.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.319+0000 7f838be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239afef270 0x56239b3ed460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42848/0 (socket says 172.21.15.39:42848) 2024-09-21T07:42:55.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.319+0000 7f838be00640 1 -- 172.21.15.39:0/898206953 learned_addr learned my addr 172.21.15.39:0/898206953 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:55.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.319+0000 7f838aa00640 1 --2- 172.21.15.39:0/898206953 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56239afce590 0x56239b3ecf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:55.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.319+0000 7f838be00640 1 -- 172.21.15.39:0/898206953 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56239afce590 msgr2=0x56239b3ecf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:42:55.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.319+0000 7f838be00640 1 --2- 172.21.15.39:0/898206953 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56239afce590 0x56239b3ecf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.319+0000 7f838be00640 1 -- 172.21.15.39:0/898206953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239afc63b0 msgr2=0x56239b4406b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:42:55.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.319+0000 7f838be00640 1 --2- 172.21.15.39:0/898206953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239afc63b0 0x56239b4406b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.320+0000 7f838be00640 1 -- 172.21.15.39:0/898206953 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56239b3bf770 con 0x56239afef270 2024-09-21T07:42:55.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.320+0000 7f838be00640 1 --2- 172.21.15.39:0/898206953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239afef270 0x56239b3ed460 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f838c159000 tx=0x7f8384001c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30871954a9044500 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:55.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.320+0000 7f8374c00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:42:55.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.320+0000 7f8374c00640 1 -- 172.21.15.39:0/898206953 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838401a020 con 0x56239afef270 2024-09-21T07:42:55.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.320+0000 7f8374c00640 1 -- 172.21.15.39:0/898206953 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8076+0+0 (secure 0 0 0) 0x7f8384016660 con 0x56239afef270 2024-09-21T07:42:55.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.320+0000 7f8399818740 1 -- 172.21.15.39:0/898206953 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f837c003ec0 con 0x56239afef270 2024-09-21T07:42:55.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.320+0000 7f8374c00640 1 -- 172.21.15.39:0/898206953 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=142) ==== 24+0+0 (secure 0 0 0) 0x7f8384016860 con 0x56239afef270 2024-09-21T07:42:55.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.320+0000 7f8399818740 10 client.7920 fetch_fsmap learned FSMap version 142 2024-09-21T07:42:55.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.320+0000 7f8399818740 1 -- 172.21.15.39:0/898206953 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=142,monmap=2+}) -- 0x56239b3f1700 con 0x56239afef270 2024-09-21T07:42:55.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.321+0000 7f8374c00640 1 -- 172.21.15.39:0/898206953 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f837c003ec0 con 0x56239afef270 2024-09-21T07:42:55.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.321+0000 7f8374c00640 1 -- 172.21.15.39:0/898206953 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 142) ==== 3522+0+0 (secure 0 0 0) 0x7f8384016a00 con 0x56239afef270 2024-09-21T07:42:55.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.321+0000 7f8399818740 10 client.7920 fetch_fsmap finished waiting for FSMap version 142 2024-09-21T07:42:55.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.321+0000 7f8399818740 10 client.7920 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:42:55.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.321+0000 7f8399818740 20 client.7920 populate_metadata read hostname 'smithi039' 2024-09-21T07:42:55.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.321+0000 7f8399818740 1 --2- 172.21.15.39:0/898206953 >> [v2:172.21.15.39:6834/1326711417,v1:172.21.15.39:6835/1326711417] conn(0x56239b3fcc40 0x56239b3b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:55.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.321+0000 7f8399818740 4 client.7920 mds_command: new command op to 7644 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:42:55.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.321+0000 7f8399818740 1 -- 172.21.15.39:0/898206953 --> [v2:172.21.15.39:6834/1326711417,v1:172.21.15.39:6835/1326711417] -- command(tid 0: {"prefix": "status"}) -- 0x56239b3d6ea0 con 0x56239b3fcc40 2024-09-21T07:42:55.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.321+0000 7f838b400640 1 --2- 172.21.15.39:0/898206953 >> [v2:172.21.15.39:6834/1326711417,v1:172.21.15.39:6835/1326711417] conn(0x56239b3fcc40 0x56239b3b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:55.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.322+0000 7f838b400640 1 --2- 172.21.15.39:0/898206953 >> [v2:172.21.15.39:6834/1326711417,v1:172.21.15.39:6835/1326711417] conn(0x56239b3fcc40 0x56239b3b8ae0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d7c27f2c654ff62b server_cookie=fb151310dcdd0999 in_seq=0 out_seq=0 2024-09-21T07:42:55.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.322+0000 7f8374c00640 10 client.7920 ms_handle_connect on v2:172.21.15.39:6834/1326711417 2024-09-21T07:42:55.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.322+0000 7f8374c00640 1 -- 172.21.15.39:0/898206953 <== mds.1 v2:172.21.15.39:6834/1326711417 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x56239b3d6ea0 con 0x56239b3fcc40 2024-09-21T07:42:55.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.322+0000 7f8374c00640 10 client.7920 handle_command_reply: tid=0 2024-09-21T07:42:55.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.322+0000 7f8399818740 1 client.7920 shutdown 2024-09-21T07:42:55.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.322+0000 7f8399818740 1 -- 172.21.15.39:0/898206953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239afef270 msgr2=0x56239b3ed460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:55.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.322+0000 7f8399818740 1 --2- 172.21.15.39:0/898206953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239afef270 0x56239b3ed460 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f838c159000 tx=0x7f8384001c50 comp rx=0 tx=0).stop 2024-09-21T07:42:55.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.322+0000 7f8399818740 1 -- 172.21.15.39:0/898206953 shutdown_connections 2024-09-21T07:42:55.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.322+0000 7f8399818740 1 --2- 172.21.15.39:0/898206953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56239afef270 0x56239b3ed460 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.322+0000 7f8399818740 1 --2- 172.21.15.39:0/898206953 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56239afce590 0x56239b3ecf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.322+0000 7f8399818740 1 --2- 172.21.15.39:0/898206953 >> [v2:172.21.15.39:6834/1326711417,v1:172.21.15.39:6835/1326711417] conn(0x56239b3fcc40 0x56239b3b8ae0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.322+0000 7f8399818740 1 --2- 172.21.15.39:0/898206953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56239afc63b0 0x56239b4406b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.322+0000 7f8399818740 1 -- 172.21.15.39:0/898206953 >> 172.21.15.39:0/898206953 conn(0x56239afe33c0 msgr2=0x56239b395940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:55.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.322+0000 7f8374c00640 0 client.7920 ms_handle_reset on v2:172.21.15.39:6834/1326711417 2024-09-21T07:42:55.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.322+0000 7f8399818740 1 -- 172.21.15.39:0/898206953 shutdown_connections 2024-09-21T07:42:55.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.322+0000 7f8399818740 1 -- 172.21.15.39:0/898206953 wait complete. 2024-09-21T07:42:55.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.323+0000 7f8399818740 20 client.7920 trim_cache size 0 max 16384 2024-09-21T07:42:55.329 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:42:55.329 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:42:55.329 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 1, 2024-09-21T07:42:55.329 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 7644, 2024-09-21T07:42:55.329 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:42:55.329 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:42:55.329 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:42:55.329 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 15.233062904000001, 2024-09-21T07:42:55.329 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 142, 2024-09-21T07:42:55.329 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 115, 2024-09-21T07:42:55.329 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 115, 2024-09-21T07:42:55.330 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 63.269688887999997 2024-09-21T07:42:55.330 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:42:55.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.323+0000 7f8392800640 1 -- 172.21.15.39:0/1835802400 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f836c078bc0 msgr2=0x7f836c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:55.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.323+0000 7f8392800640 1 --2- 172.21.15.39:0/1835802400 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f836c078bc0 0x7f836c07b0d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f837c034000 tx=0x7f837c004730 comp rx=0 tx=0).stop 2024-09-21T07:42:55.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.323+0000 7f8392800640 1 -- 172.21.15.39:0/1835802400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f838c06fc10 msgr2=0x7f838c1525a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:55.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.323+0000 7f8392800640 1 --2- 172.21.15.39:0/1835802400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f838c06fc10 0x7f838c1525a0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f837800e790 tx=0x7f837800ec50 comp rx=0 tx=0).stop 2024-09-21T07:42:55.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.323+0000 7f8392800640 1 -- 172.21.15.39:0/1835802400 shutdown_connections 2024-09-21T07:42:55.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.323+0000 7f8392800640 1 --2- 172.21.15.39:0/1835802400 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f836c078bc0 0x7f836c07b0d0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.323+0000 7f8392800640 1 --2- 172.21.15.39:0/1835802400 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f838c15cd30 0x7f838c159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.323+0000 7f8392800640 1 --2- 172.21.15.39:0/1835802400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f838c06fc10 0x7f838c1525a0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.323+0000 7f8392800640 1 --2- 172.21.15.39:0/1835802400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f838c06f800 0x7f838c0ae240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.323+0000 7f8392800640 1 -- 172.21.15.39:0/1835802400 >> 172.21.15.39:0/1835802400 conn(0x7f838c06cfc0 msgr2=0x7f838c0b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:55.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.323+0000 7f8392800640 1 -- 172.21.15.39:0/1835802400 shutdown_connections 2024-09-21T07:42:55.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.323+0000 7f8392800640 1 -- 172.21.15.39:0/1835802400 wait complete. 2024-09-21T07:42:55.350 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:42:55.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.575+0000 7f27dfa00640 1 -- 172.21.15.39:0/562249329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27d806fc10 msgr2=0x7f27d80d0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:55.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.575+0000 7f27dfa00640 1 --2- 172.21.15.39:0/562249329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27d806fc10 0x7f27d80d0620 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f27d0005fd0 tx=0x7f27d002a8b0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.575+0000 7f27dfa00640 1 -- 172.21.15.39:0/562249329 shutdown_connections 2024-09-21T07:42:55.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.575+0000 7f27dfa00640 1 --2- 172.21.15.39:0/562249329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27d806fc10 0x7f27d80d0620 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.575+0000 7f27dfa00640 1 --2- 172.21.15.39:0/562249329 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27d815cd30 0x7f27d80cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.575+0000 7f27dfa00640 1 -- 172.21.15.39:0/562249329 >> 172.21.15.39:0/562249329 conn(0x7f27d806cfc0 msgr2=0x7f27d806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:55.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.575+0000 7f27dfa00640 1 -- 172.21.15.39:0/562249329 shutdown_connections 2024-09-21T07:42:55.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.575+0000 7f27dfa00640 1 -- 172.21.15.39:0/562249329 wait complete. 2024-09-21T07:42:55.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.578+0000 7f27dfa00640 1 Processor -- start 2024-09-21T07:42:55.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.578+0000 7f27dfa00640 1 -- start start 2024-09-21T07:42:55.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.579+0000 7f27dfa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27d806f800 0x7f27d81446a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:55.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.579+0000 7f27dfa00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27d806fc10 0x7f27d80c52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:55.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.579+0000 7f27dfa00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27d815cd30 0x7f27d80ade50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:55.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.579+0000 7f27dfa00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f27d8073890 con 0x7f27d806f800 2024-09-21T07:42:55.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.579+0000 7f27dfa00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f27d8073710 con 0x7f27d806fc10 2024-09-21T07:42:55.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.579+0000 7f27dfa00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f27d8073a10 con 0x7f27d815cd30 2024-09-21T07:42:55.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.579+0000 7f27dd000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27d806f800 0x7f27d81446a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:55.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.580+0000 7f27dd000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27d806f800 0x7f27d81446a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42850/0 (socket says 172.21.15.39:42850) 2024-09-21T07:42:55.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.580+0000 7f27dd000640 1 -- 172.21.15.39:0/1845449322 learned_addr learned my addr 172.21.15.39:0/1845449322 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:55.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.580+0000 7f27d7e00640 1 --2- 172.21.15.39:0/1845449322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27d806fc10 0x7f27d80c52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:55.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.580+0000 7f27dda00640 1 --2- 172.21.15.39:0/1845449322 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27d815cd30 0x7f27d80ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:55.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.580+0000 7f27dd000640 1 -- 172.21.15.39:0/1845449322 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27d815cd30 msgr2=0x7f27d80ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:55.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.580+0000 7f27dd000640 1 --2- 172.21.15.39:0/1845449322 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27d815cd30 0x7f27d80ade50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.580+0000 7f27dd000640 1 -- 172.21.15.39:0/1845449322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27d806fc10 msgr2=0x7f27d80c52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:55.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.580+0000 7f27dd000640 1 --2- 172.21.15.39:0/1845449322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27d806fc10 0x7f27d80c52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.580+0000 7f27dd000640 1 -- 172.21.15.39:0/1845449322 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d80c5c50 con 0x7f27d806f800 2024-09-21T07:42:55.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.580+0000 7f27dda00640 1 --2- 172.21.15.39:0/1845449322 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27d815cd30 0x7f27d80ade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:42:55.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.580+0000 7f27dd000640 1 --2- 172.21.15.39:0/1845449322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27d806f800 0x7f27d81446a0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f27cc00e790 tx=0x7f27cc00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:55.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.581+0000 7f27d4c00640 1 -- 172.21.15.39:0/1845449322 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27cc00eea0 con 0x7f27d806f800 2024-09-21T07:42:55.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.581+0000 7f27d4c00640 1 -- 172.21.15.39:0/1845449322 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f27cc00cdb0 con 0x7f27d806f800 2024-09-21T07:42:55.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.581+0000 7f27dfa00640 1 -- 172.21.15.39:0/1845449322 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27d80c5f30 con 0x7f27d806f800 2024-09-21T07:42:55.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.581+0000 7f27d4c00640 1 -- 172.21.15.39:0/1845449322 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27cc010690 con 0x7f27d806f800 2024-09-21T07:42:55.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.581+0000 7f27dfa00640 1 -- 172.21.15.39:0/1845449322 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27d80cafe0 con 0x7f27d806f800 2024-09-21T07:42:55.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.582+0000 7f27dfa00640 1 -- 172.21.15.39:0/1845449322 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27d8073e70 con 0x7f27d806f800 2024-09-21T07:42:55.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.583+0000 7f27d4c00640 1 -- 172.21.15.39:0/1845449322 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f27cc010830 con 0x7f27d806f800 2024-09-21T07:42:55.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.584+0000 7f27d4c00640 1 --2- 172.21.15.39:0/1845449322 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f27b8078b90 0x7f27b807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:55.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.584+0000 7f27d4c00640 1 -- 172.21.15.39:0/1845449322 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8076+0+0 (secure 0 0 0) 0x7f27cc09c0a0 con 0x7f27d806f800 2024-09-21T07:42:55.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.585+0000 7f27d7e00640 1 --2- 172.21.15.39:0/1845449322 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f27b8078b90 0x7f27b807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:55.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.586+0000 7f27d7e00640 1 --2- 172.21.15.39:0/1845449322 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f27b8078b90 0x7f27b807b0a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f27c4007fd0 tx=0x7f27c4001390 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:55.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.589+0000 7f27d4c00640 1 -- 172.21.15.39:0/1845449322 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f27cc0634d0 con 0x7f27d806f800 2024-09-21T07:42:55.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.823+0000 7f27dfa00640 1 -- 172.21.15.39:0/1845449322 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27d80b3a60 con 0x7f27d806f800 2024-09-21T07:42:55.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.824+0000 7f27d4c00640 1 -- 172.21.15.39:0/1845449322 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 142 v142) ==== 95+0+5256 (secure 0 0 0) 0x7f27cc067f00 con 0x7f27d806f800 2024-09-21T07:42:55.826 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:42:55.826 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":142,"btime":"2024-09-21T07:42:54:142083+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7366,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/759792812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":759792812},{"type":"v1","addr":"172.21.15.148:6837","nonce":759792812}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":139}],"filesystems":[{"mdsmap":{"epoch":142,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:42:54.142073+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7644},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7644":{"gid":7644,"name":"a","rank":1,"incarnation":138,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/1326711417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1326711417},{"type":"v1","addr":"172.21.15.39:6835","nonce":1326711417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311,7644]},"id":6}]} 2024-09-21T07:42:55.826 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 142 2024-09-21T07:42:55.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.826+0000 7f27dfa00640 1 -- 172.21.15.39:0/1845449322 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f27b8078b90 msgr2=0x7f27b807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:55.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.826+0000 7f27dfa00640 1 --2- 172.21.15.39:0/1845449322 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f27b8078b90 0x7f27b807b0a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f27c4007fd0 tx=0x7f27c4001390 comp rx=0 tx=0).stop 2024-09-21T07:42:55.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.826+0000 7f27dfa00640 1 -- 172.21.15.39:0/1845449322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27d806f800 msgr2=0x7f27d81446a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:55.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.826+0000 7f27dfa00640 1 --2- 172.21.15.39:0/1845449322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27d806f800 0x7f27d81446a0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f27cc00e790 tx=0x7f27cc00ec50 comp rx=0 tx=0).stop 2024-09-21T07:42:55.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.826+0000 7f27dfa00640 1 -- 172.21.15.39:0/1845449322 shutdown_connections 2024-09-21T07:42:55.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.826+0000 7f27dfa00640 1 --2- 172.21.15.39:0/1845449322 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f27b8078b90 0x7f27b807b0a0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.826+0000 7f27dfa00640 1 --2- 172.21.15.39:0/1845449322 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27d815cd30 0x7f27d80ade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.826+0000 7f27dfa00640 1 --2- 172.21.15.39:0/1845449322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27d806fc10 0x7f27d80c52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.826+0000 7f27dfa00640 1 --2- 172.21.15.39:0/1845449322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27d806f800 0x7f27d81446a0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:55.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.827+0000 7f27dfa00640 1 -- 172.21.15.39:0/1845449322 >> 172.21.15.39:0/1845449322 conn(0x7f27d806cfc0 msgr2=0x7f27d80b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:55.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.827+0000 7f27dfa00640 1 -- 172.21.15.39:0/1845449322 shutdown_connections 2024-09-21T07:42:55.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:55.827+0000 7f27dfa00640 1 -- 172.21.15.39:0/1845449322 wait complete. 2024-09-21T07:42:55.850 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2024-09-21T07:42:56.096 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_snap":16,"last_created":16,"last_destroyed":14,"pending_noop":[],"snaps":[{"snapid":16,"ino":1099511627777,"stamp":"2024-09-21T07:42:21.254582+0000","name":"s34","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2024-09-21T07:42:56.096 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 16, "last_destroyed": 14, "last_snap": 16, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s34", "snapid": 16, "stamp": "2024-09-21T07:42:21.254582+0000" } ] } 2024-09-21T07:42:56.097 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T07:42:56.097 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:42:56.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.323+0000 7f1e60600640 1 -- 172.21.15.39:0/203521053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e5815cd30 msgr2=0x7f1e580cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:56.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.323+0000 7f1e60600640 1 --2- 172.21.15.39:0/203521053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e5815cd30 0x7f1e580cfd10 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c005fd0 tx=0x7f1e4c02a5d0 comp rx=0 tx=0).stop 2024-09-21T07:42:56.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.323+0000 7f1e60600640 1 -- 172.21.15.39:0/203521053 shutdown_connections 2024-09-21T07:42:56.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.323+0000 7f1e60600640 1 --2- 172.21.15.39:0/203521053 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1e5806fc10 0x7f1e580d0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:56.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.323+0000 7f1e60600640 1 --2- 172.21.15.39:0/203521053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e5815cd30 0x7f1e580cfd10 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:56.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.323+0000 7f1e60600640 1 -- 172.21.15.39:0/203521053 >> 172.21.15.39:0/203521053 conn(0x7f1e5806cfc0 msgr2=0x7f1e5806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:56.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.323+0000 7f1e60600640 1 -- 172.21.15.39:0/203521053 shutdown_connections 2024-09-21T07:42:56.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.323+0000 7f1e60600640 1 -- 172.21.15.39:0/203521053 wait complete. 2024-09-21T07:42:56.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.326+0000 7f1e60600640 1 Processor -- start 2024-09-21T07:42:56.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.326+0000 7f1e60600640 1 -- start start 2024-09-21T07:42:56.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.327+0000 7f1e60600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1e5806f800 0x7f1e58152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:56.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.327+0000 7f1e60600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e5806fc10 0x7f1e581f9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:56.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.327+0000 7f1e60600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e5815cd30 0x7f1e581f9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:56.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.327+0000 7f1e60600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1e58073890 con 0x7f1e5806fc10 2024-09-21T07:42:56.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.327+0000 7f1e60600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1e58073710 con 0x7f1e5815cd30 2024-09-21T07:42:56.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.327+0000 7f1e60600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f1e58073a10 con 0x7f1e5806f800 2024-09-21T07:42:56.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.327+0000 7f1e5d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e5806fc10 0x7f1e581f9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:56.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.327+0000 7f1e5d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e5806fc10 0x7f1e581f9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42866/0 (socket says 172.21.15.39:42866) 2024-09-21T07:42:56.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.327+0000 7f1e5d200640 1 -- 172.21.15.39:0/2297661727 learned_addr learned my addr 172.21.15.39:0/2297661727 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:42:56.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.328+0000 7f1e5dc00640 1 --2- 172.21.15.39:0/2297661727 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1e5806f800 0x7f1e58152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:56.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.328+0000 7f1e5e600640 1 --2- 172.21.15.39:0/2297661727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e5815cd30 0x7f1e581f9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:56.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.328+0000 7f1e5d200640 1 -- 172.21.15.39:0/2297661727 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1e5806f800 msgr2=0x7f1e58152c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:56.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.328+0000 7f1e5d200640 1 --2- 172.21.15.39:0/2297661727 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1e5806f800 0x7f1e58152c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:56.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.328+0000 7f1e5d200640 1 -- 172.21.15.39:0/2297661727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e5815cd30 msgr2=0x7f1e581f9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:56.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.328+0000 7f1e5d200640 1 --2- 172.21.15.39:0/2297661727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e5815cd30 0x7f1e581f9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:56.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.328+0000 7f1e5d200640 1 -- 172.21.15.39:0/2297661727 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e580ae3d0 con 0x7f1e5806fc10 2024-09-21T07:42:56.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.328+0000 7f1e5e600640 1 --2- 172.21.15.39:0/2297661727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e5815cd30 0x7f1e581f9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:42:56.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.328+0000 7f1e5d200640 1 --2- 172.21.15.39:0/2297661727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e5806fc10 0x7f1e581f9020 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f1e4400c340 tx=0x7f1e4400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:56.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.328+0000 7f1e55600640 1 -- 172.21.15.39:0/2297661727 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e44017070 con 0x7f1e5806fc10 2024-09-21T07:42:56.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.328+0000 7f1e55600640 1 -- 172.21.15.39:0/2297661727 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e440053c0 con 0x7f1e5806fc10 2024-09-21T07:42:56.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.329+0000 7f1e55600640 1 -- 172.21.15.39:0/2297661727 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e44003850 con 0x7f1e5806fc10 2024-09-21T07:42:56.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.329+0000 7f1e60600640 1 -- 172.21.15.39:0/2297661727 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e580ae6b0 con 0x7f1e5806fc10 2024-09-21T07:42:56.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.329+0000 7f1e60600640 1 -- 172.21.15.39:0/2297661727 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e580aec70 con 0x7f1e5806fc10 2024-09-21T07:42:56.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.330+0000 7f1e60600640 1 -- 172.21.15.39:0/2297661727 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e58073ec0 con 0x7f1e5806fc10 2024-09-21T07:42:56.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.331+0000 7f1e55600640 1 -- 172.21.15.39:0/2297661727 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1e4401a070 con 0x7f1e5806fc10 2024-09-21T07:42:56.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.332+0000 7f1e55600640 1 --2- 172.21.15.39:0/2297661727 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1e38078b90 0x7f1e3807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:42:56.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.332+0000 7f1e55600640 1 -- 172.21.15.39:0/2297661727 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8076+0+0 (secure 0 0 0) 0x7f1e4409aad0 con 0x7f1e5806fc10 2024-09-21T07:42:56.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.332+0000 7f1e5dc00640 1 --2- 172.21.15.39:0/2297661727 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1e38078b90 0x7f1e3807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:42:56.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.333+0000 7f1e5dc00640 1 --2- 172.21.15.39:0/2297661727 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1e38078b90 0x7f1e3807b0a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c033000 tx=0x7f1e4c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:42:56.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.336+0000 7f1e55600640 1 -- 172.21.15.39:0/2297661727 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1e440624b0 con 0x7f1e5806fc10 2024-09-21T07:42:56.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.573+0000 7f1e60600640 1 -- 172.21.15.39:0/2297661727 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e580afe50 con 0x7f1e5806fc10 2024-09-21T07:42:56.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.574+0000 7f1e55600640 1 -- 172.21.15.39:0/2297661727 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 142 v142) ==== 95+0+5256 (secure 0 0 0) 0x7f1e44061db0 con 0x7f1e5806fc10 2024-09-21T07:42:56.576 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:42:56.576 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":142,"btime":"2024-09-21T07:42:54:142083+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7366,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/759792812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":759792812},{"type":"v1","addr":"172.21.15.148:6837","nonce":759792812}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":139}],"filesystems":[{"mdsmap":{"epoch":142,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:42:54.142073+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7644},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7644":{"gid":7644,"name":"a","rank":1,"incarnation":138,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/1326711417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1326711417},{"type":"v1","addr":"172.21.15.39:6835","nonce":1326711417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311,7644]},"id":6}]} 2024-09-21T07:42:56.576 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 142 2024-09-21T07:42:56.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.577+0000 7f1e60600640 1 -- 172.21.15.39:0/2297661727 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1e38078b90 msgr2=0x7f1e3807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:56.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.577+0000 7f1e60600640 1 --2- 172.21.15.39:0/2297661727 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1e38078b90 0x7f1e3807b0a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c033000 tx=0x7f1e4c004730 comp rx=0 tx=0).stop 2024-09-21T07:42:56.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.577+0000 7f1e60600640 1 -- 172.21.15.39:0/2297661727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e5806fc10 msgr2=0x7f1e581f9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:42:56.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.577+0000 7f1e60600640 1 --2- 172.21.15.39:0/2297661727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e5806fc10 0x7f1e581f9020 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f1e4400c340 tx=0x7f1e4400c800 comp rx=0 tx=0).stop 2024-09-21T07:42:56.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.577+0000 7f1e60600640 1 -- 172.21.15.39:0/2297661727 shutdown_connections 2024-09-21T07:42:56.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.577+0000 7f1e60600640 1 --2- 172.21.15.39:0/2297661727 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1e38078b90 0x7f1e3807b0a0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:56.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.577+0000 7f1e60600640 1 --2- 172.21.15.39:0/2297661727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e5815cd30 0x7f1e581f9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:56.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.577+0000 7f1e60600640 1 --2- 172.21.15.39:0/2297661727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e5806fc10 0x7f1e581f9020 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:56.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.577+0000 7f1e60600640 1 --2- 172.21.15.39:0/2297661727 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1e5806f800 0x7f1e58152c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:42:56.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.577+0000 7f1e60600640 1 -- 172.21.15.39:0/2297661727 >> 172.21.15.39:0/2297661727 conn(0x7f1e5806cfc0 msgr2=0x7f1e5815bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:42:56.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.577+0000 7f1e60600640 1 -- 172.21.15.39:0/2297661727 shutdown_connections 2024-09-21T07:42:56.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:42:56.577+0000 7f1e60600640 1 -- 172.21.15.39:0/2297661727 wait complete. 2024-09-21T07:42:56.604 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2024-09-21T07:42:56.848 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_snap":16,"last_created":16,"last_destroyed":14,"pending_noop":[],"snaps":[{"snapid":16,"ino":1099511627777,"stamp":"2024-09-21T07:42:21.254582+0000","name":"s34","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2024-09-21T07:42:56.848 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 16, "last_destroyed": 14, "last_snap": 16, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s34", "snapid": 16, "stamp": "2024-09-21T07:42:21.254582+0000" } ] } 2024-09-21T07:42:56.849 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:42:56.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:42:56.849 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-09-21T07:43:01.408 INFO:teuthology.orchestra.run.smithi039.stderr:mkdir: cannot create directory ‘d1/dir/.snap/s34’: Permission denied 2024-09-21T07:43:01.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:43:01.409 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:43:01.409 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:43:01.448 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:43:01.448 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:43:01.527 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:43:01.769 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:43:01.774 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:43:01.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:01.798+0000 7fa7ce600640 1 -- 172.21.15.39:0/1742842278 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7c80d04a0 msgr2=0x7fa7c80cdea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:01.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:01.798+0000 7fa7ce600640 1 --2- 172.21.15.39:0/1742842278 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7c80d04a0 0x7fa7c80cdea0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc005fd0 tx=0x7fa7bc02aaa0 comp rx=0 tx=0).stop 2024-09-21T07:43:01.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:01.798+0000 7fa7ce600640 1 -- 172.21.15.39:0/1742842278 shutdown_connections 2024-09-21T07:43:01.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:01.799+0000 7fa7ce600640 1 --2- 172.21.15.39:0/1742842278 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7c80d04a0 0x7fa7c80cdea0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:01.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:01.799+0000 7fa7ce600640 1 -- 172.21.15.39:0/1742842278 >> 172.21.15.39:0/1742842278 conn(0x7fa7c8074050 msgr2=0x7fa7c8074490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:01.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:01.799+0000 7fa7ce600640 1 -- 172.21.15.39:0/1742842278 shutdown_connections 2024-09-21T07:43:01.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:01.799+0000 7fa7ce600640 1 -- 172.21.15.39:0/1742842278 wait complete. 2024-09-21T07:43:01.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:01.801+0000 7fa7ce600640 1 Processor -- start 2024-09-21T07:43:01.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:01.801+0000 7fa7ce600640 1 -- start start 2024-09-21T07:43:01.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:01.803+0000 7fa7ce600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7c80d04a0 0x7fa7c81f6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:01.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:01.803+0000 7fa7ce600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7c81524a0 0x7fa7c806cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:01.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:01.803+0000 7fa7ce600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7c806d320 0x7fa7c81f6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:01.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:01.803+0000 7fa7ce600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c815d830 con 0x7fa7c80d04a0 2024-09-21T07:43:01.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:01.803+0000 7fa7ce600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c815d6b0 con 0x7fa7c806d320 2024-09-21T07:43:01.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:01.803+0000 7fa7ce600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa7c815d9b0 con 0x7fa7c81524a0 2024-09-21T07:43:01.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:01.803+0000 7fa7cd200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7c80d04a0 0x7fa7c81f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:01.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:01.803+0000 7fa7cd200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7c80d04a0 0x7fa7c81f6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42882/0 (socket says 172.21.15.39:42882) 2024-09-21T07:43:01.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:01.803+0000 7fa7cd200640 1 -- 172.21.15.39:0/3680955373 learned_addr learned my addr 172.21.15.39:0/3680955373 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:01.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:01.803+0000 7fa7cdc00640 1 --2- 172.21.15.39:0/3680955373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7c806d320 0x7fa7c81f6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:01.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:01.803+0000 7fa7c7e00640 1 --2- 172.21.15.39:0/3680955373 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7c81524a0 0x7fa7c806cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:01.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:01.803+0000 7fa7cd200640 1 -- 172.21.15.39:0/3680955373 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7c81524a0 msgr2=0x7fa7c806cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:01.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:01.803+0000 7fa7cd200640 1 --2- 172.21.15.39:0/3680955373 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7c81524a0 0x7fa7c806cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:01.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:01.803+0000 7fa7cd200640 1 -- 172.21.15.39:0/3680955373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7c806d320 msgr2=0x7fa7c81f6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:01.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:01.803+0000 7fa7cd200640 1 --2- 172.21.15.39:0/3680955373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7c806d320 0x7fa7c81f6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:01.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:01.803+0000 7fa7cd200640 1 -- 172.21.15.39:0/3680955373 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c806d730 con 0x7fa7c80d04a0 2024-09-21T07:43:01.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:01.804+0000 7fa7cd200640 1 --2- 172.21.15.39:0/3680955373 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7c80d04a0 0x7fa7c81f6240 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc000f80 tx=0x7fa7bc003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:01.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:01.804+0000 7fa7c4c00640 1 -- 172.21.15.39:0/3680955373 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc044070 con 0x7fa7c80d04a0 2024-09-21T07:43:01.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:01.804+0000 7fa7c4c00640 1 -- 172.21.15.39:0/3680955373 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa7bc004250 con 0x7fa7c80d04a0 2024-09-21T07:43:01.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:01.804+0000 7fa7c4c00640 1 -- 172.21.15.39:0/3680955373 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc004680 con 0x7fa7c80d04a0 2024-09-21T07:43:01.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:01.804+0000 7fa7ce600640 1 -- 172.21.15.39:0/3680955373 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7c81f3860 con 0x7fa7c80d04a0 2024-09-21T07:43:01.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:01.804+0000 7fa7ce600640 1 -- 172.21.15.39:0/3680955373 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7c81f3e20 con 0x7fa7c80d04a0 2024-09-21T07:43:01.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:01.805+0000 7fa7ce600640 1 -- 172.21.15.39:0/3680955373 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7c80cdea0 con 0x7fa7c80d04a0 2024-09-21T07:43:01.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:01.807+0000 7fa7c4c00640 1 -- 172.21.15.39:0/3680955373 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa7bc02c040 con 0x7fa7c80d04a0 2024-09-21T07:43:01.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:01.808+0000 7fa7c4c00640 1 --2- 172.21.15.39:0/3680955373 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa7a8078b90 0x7fa7a807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:01.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:01.808+0000 7fa7c4c00640 1 -- 172.21.15.39:0/3680955373 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8076+0+0 (secure 0 0 0) 0x7fa7bc0bc3e0 con 0x7fa7c80d04a0 2024-09-21T07:43:01.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:01.808+0000 7fa7c7e00640 1 --2- 172.21.15.39:0/3680955373 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa7a8078b90 0x7fa7a807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:01.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:01.809+0000 7fa7c7e00640 1 --2- 172.21.15.39:0/3680955373 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa7a8078b90 0x7fa7a807b0a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa7c81f1100 tx=0x7fa7b4009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:01.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:01.812+0000 7fa7c4c00640 1 -- 172.21.15.39:0/3680955373 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa7c806d730 con 0x7fa7c80d04a0 2024-09-21T07:43:02.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.020+0000 7fa7ce600640 1 -- 172.21.15.39:0/3680955373 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa7c8071b60 con 0x7fa7c80d04a0 2024-09-21T07:43:02.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.020+0000 7fa7c4c00640 1 -- 172.21.15.39:0/3680955373 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 29 entries v115) ==== 81+0+1746 (secure 0 0 0) 0x7fa7bc032030 con 0x7fa7c80d04a0 2024-09-21T07:43:02.023 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1669604593 2024-09-21T08:42:52.059256+0000 2024-09-21T07:43:02.023 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/73220122 2024-09-22T07:42:40.074859+0000 2024-09-21T07:43:02.023 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/2336112936 2024-09-22T07:41:05.979253+0000 2024-09-21T07:43:02.023 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/2821446711 2024-09-22T07:40:57.372409+0000 2024-09-21T07:43:02.023 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/1456343023 2024-09-22T07:40:08.637993+0000 2024-09-21T07:43:02.023 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6833/1456343023 2024-09-22T07:40:08.637993+0000 2024-09-21T07:43:02.023 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/992188984 2024-09-22T07:40:00.778049+0000 2024-09-21T07:43:02.023 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/3356393880 2024-09-21T08:40:15.824973+0000 2024-09-21T07:43:02.023 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/992188984 2024-09-22T07:40:00.778049+0000 2024-09-21T07:43:02.023 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/1472430787 2024-09-22T07:37:58.212892+0000 2024-09-21T07:43:02.023 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/3695489158 2024-09-22T07:41:51.807018+0000 2024-09-21T07:43:02.023 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/829716143 2024-09-22T07:37:01.259219+0000 2024-09-21T07:43:02.023 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/3026827614 2024-09-22T07:37:29.781472+0000 2024-09-21T07:43:02.023 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/3026827614 2024-09-22T07:37:29.781472+0000 2024-09-21T07:43:02.024 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/1472430787 2024-09-22T07:37:58.212892+0000 2024-09-21T07:43:02.024 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.1:0/90671776 2024-09-21T08:39:22.988045+0000 2024-09-21T07:43:02.024 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/3695489158 2024-09-22T07:41:51.807018+0000 2024-09-21T07:43:02.024 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/2336112936 2024-09-22T07:41:05.979253+0000 2024-09-21T07:43:02.024 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/2821446711 2024-09-22T07:40:57.372409+0000 2024-09-21T07:43:02.024 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/2381504293 2024-09-22T07:38:32.207414+0000 2024-09-21T07:43:02.024 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/742594435 2024-09-21T08:42:01.225749+0000 2024-09-21T07:43:02.024 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/2381504293 2024-09-22T07:38:32.207414+0000 2024-09-21T07:43:02.024 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/855150775 2024-09-22T07:39:07.739045+0000 2024-09-21T07:43:02.024 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/829716143 2024-09-22T07:37:01.259219+0000 2024-09-21T07:43:02.024 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6833/364657398 2024-09-22T07:39:15.579613+0000 2024-09-21T07:43:02.024 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/855150775 2024-09-22T07:39:07.739045+0000 2024-09-21T07:43:02.024 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/73220122 2024-09-22T07:42:40.074859+0000 2024-09-21T07:43:02.024 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.1:0/177249410 2024-09-21T08:41:13.882767+0000 2024-09-21T07:43:02.024 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/364657398 2024-09-22T07:39:15.579613+0000 2024-09-21T07:43:02.025 INFO:teuthology.orchestra.run.smithi039.stderr:listed 29 entries 2024-09-21T07:43:02.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.023+0000 7fa7ce600640 1 -- 172.21.15.39:0/3680955373 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa7a8078b90 msgr2=0x7fa7a807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:02.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.023+0000 7fa7ce600640 1 --2- 172.21.15.39:0/3680955373 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa7a8078b90 0x7fa7a807b0a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa7c81f1100 tx=0x7fa7b4009040 comp rx=0 tx=0).stop 2024-09-21T07:43:02.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.023+0000 7fa7ce600640 1 -- 172.21.15.39:0/3680955373 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7c80d04a0 msgr2=0x7fa7c81f6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:02.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.023+0000 7fa7ce600640 1 --2- 172.21.15.39:0/3680955373 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7c80d04a0 0x7fa7c81f6240 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc000f80 tx=0x7fa7bc003f10 comp rx=0 tx=0).stop 2024-09-21T07:43:02.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.023+0000 7fa7ce600640 1 -- 172.21.15.39:0/3680955373 shutdown_connections 2024-09-21T07:43:02.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.023+0000 7fa7ce600640 1 --2- 172.21.15.39:0/3680955373 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa7a8078b90 0x7fa7a807b0a0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:02.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.023+0000 7fa7ce600640 1 --2- 172.21.15.39:0/3680955373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7c806d320 0x7fa7c81f6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:02.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.023+0000 7fa7ce600640 1 --2- 172.21.15.39:0/3680955373 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa7c81524a0 0x7fa7c806cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:02.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.023+0000 7fa7ce600640 1 --2- 172.21.15.39:0/3680955373 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7c80d04a0 0x7fa7c81f6240 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:02.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.023+0000 7fa7ce600640 1 -- 172.21.15.39:0/3680955373 >> 172.21.15.39:0/3680955373 conn(0x7fa7c8074050 msgr2=0x7fa7c815de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:02.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.023+0000 7fa7ce600640 1 -- 172.21.15.39:0/3680955373 shutdown_connections 2024-09-21T07:43:02.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.023+0000 7fa7ce600640 1 -- 172.21.15.39:0/3680955373 wait complete. 2024-09-21T07:43:02.050 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T07:43:02.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:43:02.050 DEBUG:teuthology.orchestra.run.smithi039:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T07:43:02.126 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T07:43:02.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:43:02.126 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:43:02.162 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:43:02.257 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:43:02.293 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:43:02.293 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:43:02.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.559+0000 7f1a42000640 1 -- 172.21.15.39:0/4018842743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a3c0ca890 msgr2=0x7f1a3c0b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:02.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.559+0000 7f1a42000640 1 --2- 172.21.15.39:0/4018842743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a3c0ca890 0x7f1a3c0b30d0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f1a34005fd0 tx=0x7f1a3402ab40 comp rx=0 tx=0).stop 2024-09-21T07:43:02.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.559+0000 7f1a42000640 1 -- 172.21.15.39:0/4018842743 shutdown_connections 2024-09-21T07:43:02.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.559+0000 7f1a42000640 1 --2- 172.21.15.39:0/4018842743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a3c0ca890 0x7f1a3c0b30d0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:02.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.559+0000 7f1a42000640 1 -- 172.21.15.39:0/4018842743 >> 172.21.15.39:0/4018842743 conn(0x7f1a3c0b2510 msgr2=0x7f1a3c0d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:02.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.559+0000 7f1a42000640 1 -- 172.21.15.39:0/4018842743 shutdown_connections 2024-09-21T07:43:02.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.559+0000 7f1a42000640 1 -- 172.21.15.39:0/4018842743 wait complete. 2024-09-21T07:43:02.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.561+0000 7f1a42000640 1 Processor -- start 2024-09-21T07:43:02.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.561+0000 7f1a42000640 1 -- start start 2024-09-21T07:43:02.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.563+0000 7f1a42000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a3c0ca890 0x7f1a3c0c4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:02.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.563+0000 7f1a42000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1a3c0cc560 0x7f1a3c155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:02.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.563+0000 7f1a42000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a3c1562a0 0x7f1a3c166850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:02.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.563+0000 7f1a42000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c070060 con 0x7f1a3c1562a0 2024-09-21T07:43:02.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.563+0000 7f1a42000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c06fee0 con 0x7f1a3c0ca890 2024-09-21T07:43:02.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.563+0000 7f1a42000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f1a3c0701e0 con 0x7f1a3c0cc560 2024-09-21T07:43:02.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.563+0000 7f1a41600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a3c1562a0 0x7f1a3c166850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:02.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.563+0000 7f1a41600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a3c1562a0 0x7f1a3c166850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36684/0 (socket says 172.21.15.39:36684) 2024-09-21T07:43:02.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.563+0000 7f1a41600640 1 -- 172.21.15.39:0/1979152634 learned_addr learned my addr 172.21.15.39:0/1979152634 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:02.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.563+0000 7f1a40c00640 1 --2- 172.21.15.39:0/1979152634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a3c0ca890 0x7f1a3c0c4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:02.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.563+0000 7f1a33e00640 1 --2- 172.21.15.39:0/1979152634 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1a3c0cc560 0x7f1a3c155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:02.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.563+0000 7f1a41600640 1 -- 172.21.15.39:0/1979152634 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1a3c0cc560 msgr2=0x7f1a3c155d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:02.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.563+0000 7f1a41600640 1 --2- 172.21.15.39:0/1979152634 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1a3c0cc560 0x7f1a3c155d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:02.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.563+0000 7f1a41600640 1 -- 172.21.15.39:0/1979152634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a3c0ca890 msgr2=0x7f1a3c0c4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:02.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.563+0000 7f1a41600640 1 --2- 172.21.15.39:0/1979152634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a3c0ca890 0x7f1a3c0c4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:02.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.563+0000 7f1a41600640 1 -- 172.21.15.39:0/1979152634 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a3c0c6270 con 0x7f1a3c1562a0 2024-09-21T07:43:02.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.564+0000 7f1a41600640 1 --2- 172.21.15.39:0/1979152634 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a3c1562a0 0x7f1a3c166850 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f1a34009d90 tx=0x7f1a3402c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:02.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.564+0000 7f1a39600640 1 -- 172.21.15.39:0/1979152634 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a34044070 con 0x7f1a3c1562a0 2024-09-21T07:43:02.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.564+0000 7f1a39600640 1 -- 172.21.15.39:0/1979152634 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a34004250 con 0x7f1a3c1562a0 2024-09-21T07:43:02.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.564+0000 7f1a39600640 1 -- 172.21.15.39:0/1979152634 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a34004680 con 0x7f1a3c1562a0 2024-09-21T07:43:02.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.564+0000 7f1a42000640 1 -- 172.21.15.39:0/1979152634 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a3c0c6550 con 0x7f1a3c1562a0 2024-09-21T07:43:02.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.564+0000 7f1a42000640 1 -- 172.21.15.39:0/1979152634 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a3c0c6b10 con 0x7f1a3c1562a0 2024-09-21T07:43:02.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.565+0000 7f1a42000640 1 -- 172.21.15.39:0/1979152634 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a3c0b30d0 con 0x7f1a3c1562a0 2024-09-21T07:43:02.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.566+0000 7f1a39600640 1 -- 172.21.15.39:0/1979152634 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1a34036400 con 0x7f1a3c1562a0 2024-09-21T07:43:02.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.567+0000 7f1a39600640 1 --2- 172.21.15.39:0/1979152634 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1a18078bc0 0x7f1a1807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:02.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.567+0000 7f1a39600640 1 -- 172.21.15.39:0/1979152634 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8076+0+0 (secure 0 0 0) 0x7f1a340bbfe0 con 0x7f1a3c1562a0 2024-09-21T07:43:02.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.568+0000 7f1a40c00640 1 --2- 172.21.15.39:0/1979152634 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1a18078bc0 0x7f1a1807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:02.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.569+0000 7f1a40c00640 1 --2- 172.21.15.39:0/1979152634 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1a18078bc0 0x7f1a1807b0d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c000f60 tx=0x7f1a2c009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:02.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.572+0000 7f1a39600640 1 -- 172.21.15.39:0/1979152634 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1a34032030 con 0x7f1a3c1562a0 2024-09-21T07:43:02.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.808+0000 7f1a42000640 1 -- 172.21.15.39:0/1979152634 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1a3c15b1f0 con 0x7f1a3c1562a0 2024-09-21T07:43:02.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.808+0000 7f1a39600640 1 -- 172.21.15.39:0/1979152634 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v142) ==== 53+0+73 (secure 0 0 0) 0x7f1a34004820 con 0x7f1a3c1562a0 2024-09-21T07:43:02.811 INFO:teuthology.orchestra.run.smithi039.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:43:02.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.811+0000 7f1a42000640 1 -- 172.21.15.39:0/1979152634 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1a18078bc0 msgr2=0x7f1a1807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:02.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.811+0000 7f1a42000640 1 --2- 172.21.15.39:0/1979152634 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1a18078bc0 0x7f1a1807b0d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c000f60 tx=0x7f1a2c009040 comp rx=0 tx=0).stop 2024-09-21T07:43:02.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.811+0000 7f1a42000640 1 -- 172.21.15.39:0/1979152634 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a3c1562a0 msgr2=0x7f1a3c166850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:02.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.811+0000 7f1a42000640 1 --2- 172.21.15.39:0/1979152634 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a3c1562a0 0x7f1a3c166850 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f1a34009d90 tx=0x7f1a3402c8b0 comp rx=0 tx=0).stop 2024-09-21T07:43:02.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.811+0000 7f1a42000640 1 -- 172.21.15.39:0/1979152634 shutdown_connections 2024-09-21T07:43:02.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.811+0000 7f1a42000640 1 --2- 172.21.15.39:0/1979152634 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1a18078bc0 0x7f1a1807b0d0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:02.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.811+0000 7f1a42000640 1 --2- 172.21.15.39:0/1979152634 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a3c1562a0 0x7f1a3c166850 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:02.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.811+0000 7f1a42000640 1 --2- 172.21.15.39:0/1979152634 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1a3c0cc560 0x7f1a3c155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:02.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.811+0000 7f1a42000640 1 --2- 172.21.15.39:0/1979152634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a3c0ca890 0x7f1a3c0c4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:02.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.811+0000 7f1a42000640 1 -- 172.21.15.39:0/1979152634 >> 172.21.15.39:0/1979152634 conn(0x7f1a3c0b2510 msgr2=0x7f1a3c0af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:02.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.811+0000 7f1a42000640 1 -- 172.21.15.39:0/1979152634 shutdown_connections 2024-09-21T07:43:02.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:02.811+0000 7f1a42000640 1 -- 172.21.15.39:0/1979152634 wait complete. 2024-09-21T07:43:02.838 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:43:02.838 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:43:02.838 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi039.front.sepia.ceph.com 2024-09-21T07:43:02.838 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T07:43:02.838 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:43:02.839 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:43:02.839 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:43:02.839 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:43:02.839 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:43:02.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:43:02.839 DEBUG:teuthology.orchestra.run.smithi039:> ip addr 2024-09-21T07:43:02.875 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:43:02.875 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:43:02.875 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:43:02.875 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:43:02.875 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2024-09-21T07:43:02.876 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:43:02.876 INFO:teuthology.orchestra.run.smithi039.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:43:02.876 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:43:02.876 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f0 2024-09-21T07:43:02.876 INFO:teuthology.orchestra.run.smithi039.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:43:02.876 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c7 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:43:02.876 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f1 2024-09-21T07:43:02.876 INFO:teuthology.orchestra.run.smithi039.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:43:02.876 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:70 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:43:02.876 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f0 2024-09-21T07:43:02.876 INFO:teuthology.orchestra.run.smithi039.stdout: inet 172.21.15.39/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T07:43:02.876 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft 42136sec preferred_lft 42136sec 2024-09-21T07:43:02.876 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec4:7aff:febd:1370/64 scope link 2024-09-21T07:43:02.876 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:43:02.877 INFO:teuthology.orchestra.run.smithi039.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:43:02.877 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:71 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:43:02.877 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f1 2024-09-21T07:43:02.877 INFO:teuthology.orchestra.run.smithi039.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:43:02.877 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff 2024-09-21T07:43:02.877 INFO:teuthology.orchestra.run.smithi039.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:43:02.877 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:43:02.877 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::b079:ddff:fe0e:6333/64 scope link 2024-09-21T07:43:02.877 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:43:02.877 INFO:teuthology.orchestra.run.smithi039.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:43:02.877 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:43:02.877 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::2ca7:98ff:fe8a:d0ed/64 scope link 2024-09-21T07:43:02.877 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:43:02.877 INFO:teuthology.orchestra.run.smithi039.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:43:02.877 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether ee:10:53:01:0c:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:43:02.878 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec10:53ff:fe01:c44/64 scope link 2024-09-21T07:43:02.878 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:43:02.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:43:02.878 DEBUG:teuthology.orchestra.run.smithi039:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:43:02.969 INFO:teuthology.orchestra.run.smithi039.stdout:1 2024-09-21T07:43:02.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:43:02.972 DEBUG:teuthology.orchestra.run.smithi039:> route 2024-09-21T07:43:03.008 INFO:teuthology.orchestra.run.smithi039.stdout:Kernel IP routing table 2024-09-21T07:43:03.008 INFO:teuthology.orchestra.run.smithi039.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:43:03.008 INFO:teuthology.orchestra.run.smithi039.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T07:43:03.008 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T07:43:03.008 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:43:03.009 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:43:03.009 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-21T07:43:03.009 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T07:43:03.009 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T07:43:03.009 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T07:43:03.009 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-21T07:43:03.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:43:03.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:43:03.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:43:03.207 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-09-21T07:43:03.242 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:43:03.242 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T07:43:03.243 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:43:03.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:43:03.243 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-09-21T07:43:03.339 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:43:03.341 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T07:43:03.376 INFO:teuthology.orchestra.run.smithi039.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:43:03.376 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:43:03.377 DEBUG:teuthology.orchestra.run.smithi039:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:43:03.451 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-21T07:43:03.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:43:03.451 DEBUG:teuthology.orchestra.run.smithi039:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:43:03.650 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:43:03.650 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:43:03.686 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:43:03.686 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:43:03.687 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: resolved to: "172.21.15.39:3300,172.21.15.148:3300,172.21.15.148:3301" 2024-09-21T07:43:03.687 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: trying mount with new device syntax: 0@7ab9b04a-f88d-49aa-a493-e5c076637ffb.cephfs=/ 2024-09-21T07:43:03.687 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.39:3300/172.21.15.148:3300/172.21.15.148:3301" will pass to kernel 2024-09-21T07:43:03.688 INFO:teuthology.orchestra.run.smithi039.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T07:43:03.688 INFO:teuthology.orchestra.run.smithi039.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:43:03.688 INFO:teuthology.orchestra.run.smithi039.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:43:03.688 INFO:teuthology.orchestra.run.smithi039.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:43:03.688 INFO:teuthology.orchestra.run.smithi039.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:43:03.691 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:43:03.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:43:03.691 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:43:03.746 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:43:03.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:43:03.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:43:03.838 INFO:teuthology.orchestra.run.smithi039.stdout:client6737 2024-09-21T07:43:03.839 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:43:03.839 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:43:03.874 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 flush journal 2024-09-21T07:43:04.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.143+0000 7f32b0e00640 1 -- 172.21.15.39:0/3473609685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3290004070 msgr2=0x7f3290004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:04.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.143+0000 7f32b0e00640 1 --2- 172.21.15.39:0/3473609685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3290004070 0x7f3290004800 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f3294009b50 tx=0x7f3290005830 comp rx=0 tx=0).stop 2024-09-21T07:43:04.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.143+0000 7f32aa800640 1 -- 172.21.15.39:0/3473609685 reap_dead start 2024-09-21T07:43:04.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.143+0000 7f32b0e00640 1 -- 172.21.15.39:0/3473609685 shutdown_connections 2024-09-21T07:43:04.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.143+0000 7f32b0e00640 1 -- 172.21.15.39:0/3473609685 >> 172.21.15.39:0/3473609685 conn(0x7f32ac06d440 msgr2=0x7f32ac0b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:04.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.143+0000 7f32b0e00640 1 -- 172.21.15.39:0/3473609685 shutdown_connections 2024-09-21T07:43:04.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.143+0000 7f32b0e00640 1 -- 172.21.15.39:0/3473609685 wait complete. 2024-09-21T07:43:04.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.145+0000 7f32b0e00640 1 Processor -- start 2024-09-21T07:43:04.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.145+0000 7f32b0e00640 1 -- start start 2024-09-21T07:43:04.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.147+0000 7f32b0e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3290004070 0x7f32ac099950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:04.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.147+0000 7f32b0e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32ac15f0e0 0x7f32ac1f43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:04.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.147+0000 7f32b0e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32ac157bd0 0x7f32ac157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:04.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.147+0000 7f32b0e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac06f650 con 0x7f32ac15f0e0 2024-09-21T07:43:04.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.147+0000 7f32b0e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac0cbff0 con 0x7f32ac157bd0 2024-09-21T07:43:04.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.147+0000 7f32b0e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f32ac06f7d0 con 0x7f3290004070 2024-09-21T07:43:04.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.147+0000 7f32a1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32ac15f0e0 0x7f32ac1f43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:04.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.147+0000 7f32a1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32ac15f0e0 0x7f32ac1f43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36704/0 (socket says 172.21.15.39:36704) 2024-09-21T07:43:04.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.147+0000 7f32a1400640 1 -- 172.21.15.39:0/4182003023 learned_addr learned my addr 172.21.15.39:0/4182003023 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:04.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.147+0000 7f32a9e00640 1 --2- 172.21.15.39:0/4182003023 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3290004070 0x7f32ac099950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:04.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.147+0000 7f32aa800640 1 --2- 172.21.15.39:0/4182003023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32ac157bd0 0x7f32ac157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:04.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.147+0000 7f32a1400640 1 -- 172.21.15.39:0/4182003023 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3290004070 msgr2=0x7f32ac099950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:04.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.147+0000 7f32a1400640 1 --2- 172.21.15.39:0/4182003023 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3290004070 0x7f32ac099950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:04.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.147+0000 7f32a1400640 1 -- 172.21.15.39:0/4182003023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32ac157bd0 msgr2=0x7f32ac157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:04.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.147+0000 7f32a1400640 1 --2- 172.21.15.39:0/4182003023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32ac157bd0 0x7f32ac157fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:04.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.147+0000 7f32a1400640 1 -- 172.21.15.39:0/4182003023 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32ac158520 con 0x7f32ac15f0e0 2024-09-21T07:43:04.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.147+0000 7f32aa800640 1 --2- 172.21.15.39:0/4182003023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32ac157bd0 0x7f32ac157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:43:04.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.148+0000 7f32a1400640 1 --2- 172.21.15.39:0/4182003023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32ac15f0e0 0x7f32ac1f43d0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f3294036040 tx=0x7f3294033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:04.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.148+0000 7f32a2a00640 1 -- 172.21.15.39:0/4182003023 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3294038040 con 0x7f32ac15f0e0 2024-09-21T07:43:04.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.148+0000 7f32a2a00640 1 -- 172.21.15.39:0/4182003023 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3294041070 con 0x7f32ac15f0e0 2024-09-21T07:43:04.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.148+0000 7f32b0e00640 1 -- 172.21.15.39:0/4182003023 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32ac099e90 con 0x7f32ac15f0e0 2024-09-21T07:43:04.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.148+0000 7f32a2a00640 1 -- 172.21.15.39:0/4182003023 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3294033d90 con 0x7f32ac15f0e0 2024-09-21T07:43:04.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.148+0000 7f32b0e00640 1 -- 172.21.15.39:0/4182003023 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32ac156e10 con 0x7f32ac15f0e0 2024-09-21T07:43:04.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.151+0000 7f32a2a00640 1 -- 172.21.15.39:0/4182003023 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3294046460 con 0x7f32ac15f0e0 2024-09-21T07:43:04.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.151+0000 7f32a2a00640 1 --2- 172.21.15.39:0/4182003023 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3288078b90 0x7f328807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:04.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.152+0000 7f32a2a00640 1 -- 172.21.15.39:0/4182003023 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8076+0+0 (secure 0 0 0) 0x7f32940bc200 con 0x7f32ac15f0e0 2024-09-21T07:43:04.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.152+0000 7f32a9e00640 1 --2- 172.21.15.39:0/4182003023 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3288078b90 0x7f328807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:04.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.152+0000 7f32b7dc3740 1 Processor -- start 2024-09-21T07:43:04.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.152+0000 7f32b7dc3740 1 -- start start 2024-09-21T07:43:04.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.153+0000 7f32a9e00640 1 --2- 172.21.15.39:0/4182003023 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3288078b90 0x7f328807b0a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f329c006bb0 tx=0x7f329c006720 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:04.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.154+0000 7f32b7dc3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5591e4bdc390 0x5591e4f9d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:04.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.154+0000 7f32b7dc3740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5591e4bb34d0 0x5591e4fc90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:04.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.154+0000 7f32b7dc3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5591e4bbb6b0 0x5591e4faa150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:04.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.154+0000 7f32b7dc3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5591e4ee31c0 con 0x5591e4bbb6b0 2024-09-21T07:43:04.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.154+0000 7f32b7dc3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5591e4ef6870 con 0x5591e4bdc390 2024-09-21T07:43:04.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.154+0000 7f32b7dc3740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x5591e4ee5ca0 con 0x5591e4bb34d0 2024-09-21T07:43:04.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.154+0000 7f32aa800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5591e4bbb6b0 0x5591e4faa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:04.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.154+0000 7f32aa800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5591e4bbb6b0 0x5591e4faa150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36710/0 (socket says 172.21.15.39:36710) 2024-09-21T07:43:04.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.154+0000 7f32aa800640 1 -- 172.21.15.39:0/2461865539 learned_addr learned my addr 172.21.15.39:0/2461865539 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:04.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.154+0000 7f32a1400640 1 --2- 172.21.15.39:0/2461865539 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5591e4bb34d0 0x5591e4fc90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:04.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.154+0000 7f32a9e00640 1 --2- 172.21.15.39:0/2461865539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5591e4bdc390 0x5591e4f9d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:04.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.155+0000 7f32aa800640 1 -- 172.21.15.39:0/2461865539 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5591e4bb34d0 msgr2=0x5591e4fc90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:43:04.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.155+0000 7f32aa800640 1 --2- 172.21.15.39:0/2461865539 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5591e4bb34d0 0x5591e4fc90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:04.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.155+0000 7f32aa800640 1 -- 172.21.15.39:0/2461865539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5591e4bdc390 msgr2=0x5591e4f9d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:43:04.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.155+0000 7f32aa800640 1 --2- 172.21.15.39:0/2461865539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5591e4bdc390 0x5591e4f9d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:04.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.155+0000 7f32aa800640 1 -- 172.21.15.39:0/2461865539 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591e4cbe7d0 con 0x5591e4bbb6b0 2024-09-21T07:43:04.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.155+0000 7f32a9e00640 1 --2- 172.21.15.39:0/2461865539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5591e4bdc390 0x5591e4f9d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:43:04.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.155+0000 7f32aa800640 1 --2- 172.21.15.39:0/2461865539 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5591e4bbb6b0 0x5591e4faa150 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f32ac161530 tx=0x7f32a4001c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68eb6d28cceafb12 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:04.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.155+0000 7f329aa00640 1 -- 172.21.15.39:0/2461865539 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a40064e0 con 0x5591e4bbb6b0 2024-09-21T07:43:04.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.155+0000 7f329aa00640 1 -- 172.21.15.39:0/2461865539 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32a4016030 con 0x5591e4bbb6b0 2024-09-21T07:43:04.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.155+0000 7f329aa00640 1 -- 172.21.15.39:0/2461865539 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a4006880 con 0x5591e4bbb6b0 2024-09-21T07:43:04.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.156+0000 7f32b7dc3740 1 -- 172.21.15.39:0/2461865539 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5591e4bbb6b0 msgr2=0x5591e4faa150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:04.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.156+0000 7f32b7dc3740 1 --2- 172.21.15.39:0/2461865539 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5591e4bbb6b0 0x5591e4faa150 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f32ac161530 tx=0x7f32a4001c70 comp rx=0 tx=0).stop 2024-09-21T07:43:04.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.156+0000 7f32b7dc3740 1 -- 172.21.15.39:0/2461865539 shutdown_connections 2024-09-21T07:43:04.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.156+0000 7f32b7dc3740 1 --2- 172.21.15.39:0/2461865539 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5591e4bbb6b0 0x5591e4faa150 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:04.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.156+0000 7f32b7dc3740 1 --2- 172.21.15.39:0/2461865539 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5591e4bb34d0 0x5591e4fc90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:04.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.156+0000 7f32b7dc3740 1 --2- 172.21.15.39:0/2461865539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5591e4bdc390 0x5591e4f9d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:04.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.156+0000 7f32b7dc3740 1 -- 172.21.15.39:0/2461865539 >> 172.21.15.39:0/2461865539 conn(0x5591e4bd04e0 msgr2=0x5591e4f83440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:04.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.156+0000 7f32b7dc3740 1 -- 172.21.15.39:0/2461865539 shutdown_connections 2024-09-21T07:43:04.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.156+0000 7f32b7dc3740 1 -- 172.21.15.39:0/2461865539 wait complete. 2024-09-21T07:43:04.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.157+0000 7f32b7dc3740 1 -- start start 2024-09-21T07:43:04.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.157+0000 7f32b7dc3740 1 Processor -- start 2024-09-21T07:43:04.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.159+0000 7f32b7dc3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5591e4bb34d0 0x5591e4f98940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:04.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.159+0000 7f32b7dc3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5591e4bbb6b0 0x5591e4fde980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:04.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.159+0000 7f32b7dc3740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5591e4bdc390 0x5591e4fdeec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:04.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.159+0000 7f32b7dc3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5591e4e8c410 con 0x5591e4bb34d0 2024-09-21T07:43:04.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.159+0000 7f32b7dc3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5591e4ee9ab0 con 0x5591e4bbb6b0 2024-09-21T07:43:04.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.159+0000 7f32b7dc3740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x5591e4c56740 con 0x5591e4bdc390 2024-09-21T07:43:04.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.159+0000 7f32b7dc3740 10 client.0 _ll_register_callbacks cb 0x5591e4e69360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:43:04.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.159+0000 7f32a9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5591e4bb34d0 0x5591e4f98940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:04.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.159+0000 7f32a9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5591e4bb34d0 0x5591e4f98940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36718/0 (socket says 172.21.15.39:36718) 2024-09-21T07:43:04.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.159+0000 7f32a9e00640 1 -- 172.21.15.39:0/613608016 learned_addr learned my addr 172.21.15.39:0/613608016 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:04.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.159+0000 7f32aa800640 1 --2- 172.21.15.39:0/613608016 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5591e4bdc390 0x5591e4fdeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:04.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.159+0000 7f32a1400640 1 --2- 172.21.15.39:0/613608016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5591e4bbb6b0 0x5591e4fde980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:04.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.160+0000 7f32a9e00640 1 -- 172.21.15.39:0/613608016 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5591e4bdc390 msgr2=0x5591e4fdeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:43:04.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.160+0000 7f32a9e00640 1 --2- 172.21.15.39:0/613608016 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5591e4bdc390 0x5591e4fdeec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:04.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.160+0000 7f32a9e00640 1 -- 172.21.15.39:0/613608016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5591e4bbb6b0 msgr2=0x5591e4fde980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:43:04.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.160+0000 7f32a9e00640 1 --2- 172.21.15.39:0/613608016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5591e4bbb6b0 0x5591e4fde980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:04.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.160+0000 7f32a9e00640 1 -- 172.21.15.39:0/613608016 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591e4c47970 con 0x5591e4bb34d0 2024-09-21T07:43:04.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.160+0000 7f32a9e00640 1 --2- 172.21.15.39:0/613608016 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5591e4bb34d0 0x5591e4f98940 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f329c005d90 tx=0x7f32ac0ccaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db8ed1270c52dbf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:04.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.160+0000 7f328f400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:43:04.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.160+0000 7f328f400640 1 -- 172.21.15.39:0/613608016 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c0054e0 con 0x5591e4bb34d0 2024-09-21T07:43:04.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.160+0000 7f32b7dc3740 1 -- 172.21.15.39:0/613608016 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32a4016030 con 0x5591e4bb34d0 2024-09-21T07:43:04.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.160+0000 7f328f400640 1 -- 172.21.15.39:0/613608016 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8076+0+0 (secure 0 0 0) 0x7f329c0129c0 con 0x5591e4bb34d0 2024-09-21T07:43:04.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.160+0000 7f328f400640 1 -- 172.21.15.39:0/613608016 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=142) ==== 24+0+0 (secure 0 0 0) 0x7f329c012bc0 con 0x5591e4bb34d0 2024-09-21T07:43:04.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.160+0000 7f32b7dc3740 10 client.7965 fetch_fsmap learned FSMap version 142 2024-09-21T07:43:04.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.160+0000 7f32b7dc3740 1 -- 172.21.15.39:0/613608016 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=142,monmap=2+}) -- 0x5591e4c256a0 con 0x5591e4bb34d0 2024-09-21T07:43:04.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.161+0000 7f328f400640 1 -- 172.21.15.39:0/613608016 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32a4016030 con 0x5591e4bb34d0 2024-09-21T07:43:04.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.161+0000 7f328f400640 1 -- 172.21.15.39:0/613608016 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 142) ==== 3522+0+0 (secure 0 0 0) 0x7f329c027060 con 0x5591e4bb34d0 2024-09-21T07:43:04.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.161+0000 7f32b7dc3740 10 client.7965 fetch_fsmap finished waiting for FSMap version 142 2024-09-21T07:43:04.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.161+0000 7f32b7dc3740 10 client.7965 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-21T07:43:04.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.161+0000 7f32b7dc3740 20 client.7965 populate_metadata read hostname 'smithi039' 2024-09-21T07:43:04.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.161+0000 7f32b7dc3740 1 --2- 172.21.15.39:0/613608016 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x5591e4b9af60 0x5591e4faf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:04.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.161+0000 7f32b7dc3740 4 client.7965 mds_command: new command op to 7311 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:43:04.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.161+0000 7f32b7dc3740 1 -- 172.21.15.39:0/613608016 --> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591e4ef87c0 con 0x5591e4b9af60 2024-09-21T07:43:04.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.161+0000 7f32a1400640 1 --2- 172.21.15.39:0/613608016 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x5591e4b9af60 0x5591e4faf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:04.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.162+0000 7f328f400640 10 client.7965 ms_handle_connect on v2:172.21.15.39:6832/1790581246 2024-09-21T07:43:04.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.162+0000 7f32a1400640 1 --2- 172.21.15.39:0/613608016 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x5591e4b9af60 0x5591e4faf310 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a94b842b947931eb server_cookie=cf174247c3a765ba in_seq=0 out_seq=0 2024-09-21T07:43:04.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.162+0000 7f328f400640 1 -- 172.21.15.39:0/613608016 <== mds.0 v2:172.21.15.39:6832/1790581246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5591e4ef87c0 con 0x5591e4b9af60 2024-09-21T07:43:04.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.162+0000 7f328f400640 10 client.7965 handle_command_reply: tid=0 2024-09-21T07:43:04.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.162+0000 7f32b7dc3740 1 client.7965 shutdown 2024-09-21T07:43:04.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.163+0000 7f32b7dc3740 1 -- 172.21.15.39:0/613608016 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5591e4bb34d0 msgr2=0x5591e4f98940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:04.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.163+0000 7f32b7dc3740 1 --2- 172.21.15.39:0/613608016 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5591e4bb34d0 0x5591e4f98940 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f329c005d90 tx=0x7f32ac0ccaf0 comp rx=0 tx=0).stop 2024-09-21T07:43:04.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.163+0000 7f32b7dc3740 1 -- 172.21.15.39:0/613608016 shutdown_connections 2024-09-21T07:43:04.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.163+0000 7f32b7dc3740 1 --2- 172.21.15.39:0/613608016 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5591e4bdc390 0x5591e4fdeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:04.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.163+0000 7f32b7dc3740 1 --2- 172.21.15.39:0/613608016 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x5591e4b9af60 0x5591e4faf310 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:04.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.163+0000 7f32b7dc3740 1 --2- 172.21.15.39:0/613608016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5591e4bbb6b0 0x5591e4fde980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:04.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.163+0000 7f32b7dc3740 1 --2- 172.21.15.39:0/613608016 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5591e4bb34d0 0x5591e4f98940 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:04.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.163+0000 7f328f400640 0 client.7965 ms_handle_reset on v2:172.21.15.39:6832/1790581246 2024-09-21T07:43:04.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.163+0000 7f32b7dc3740 1 -- 172.21.15.39:0/613608016 >> 172.21.15.39:0/613608016 conn(0x5591e4bd04e0 msgr2=0x5591e4f7fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:04.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.163+0000 7f32b7dc3740 1 -- 172.21.15.39:0/613608016 shutdown_connections 2024-09-21T07:43:04.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.163+0000 7f32b7dc3740 1 -- 172.21.15.39:0/613608016 wait complete. 2024-09-21T07:43:04.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.163+0000 7f32b7dc3740 20 client.7965 trim_cache size 0 max 16384 2024-09-21T07:43:04.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.179+0000 7f32b7dc3740 1 Processor -- start 2024-09-21T07:43:04.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.179+0000 7f32b7dc3740 1 -- start start 2024-09-21T07:43:04.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.181+0000 7f32b7dc3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5591e4bb34d0 0x5591e4f7d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:04.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.181+0000 7f32b7dc3740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5591e4bbb6b0 0x5591e4fded20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:04.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.181+0000 7f32b7dc3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5591e4bdc390 0x5591e4faa150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:04.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.181+0000 7f32b7dc3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac201010 con 0x5591e4bdc390 2024-09-21T07:43:04.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.181+0000 7f32b7dc3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac2016e0 con 0x5591e4bb34d0 2024-09-21T07:43:04.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.181+0000 7f32b7dc3740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f32ac1ffc10 con 0x5591e4bbb6b0 2024-09-21T07:43:04.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.181+0000 7f32aa800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5591e4bdc390 0x5591e4faa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:04.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.181+0000 7f32aa800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5591e4bdc390 0x5591e4faa150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36728/0 (socket says 172.21.15.39:36728) 2024-09-21T07:43:04.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.181+0000 7f32aa800640 1 -- 172.21.15.39:0/3931370836 learned_addr learned my addr 172.21.15.39:0/3931370836 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:04.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.181+0000 7f32a9e00640 1 --2- 172.21.15.39:0/3931370836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5591e4bb34d0 0x5591e4f7d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:04.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.181+0000 7f32a1400640 1 --2- 172.21.15.39:0/3931370836 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5591e4bbb6b0 0x5591e4fded20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:04.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.181+0000 7f32aa800640 1 -- 172.21.15.39:0/3931370836 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5591e4bbb6b0 msgr2=0x5591e4fded20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:43:04.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.181+0000 7f32aa800640 1 --2- 172.21.15.39:0/3931370836 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5591e4bbb6b0 0x5591e4fded20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:04.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.181+0000 7f32aa800640 1 -- 172.21.15.39:0/3931370836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5591e4bb34d0 msgr2=0x5591e4f7d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:43:04.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.181+0000 7f32aa800640 1 --2- 172.21.15.39:0/3931370836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5591e4bb34d0 0x5591e4f7d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:04.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.181+0000 7f32aa800640 1 -- 172.21.15.39:0/3931370836 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591e4fbfbf0 con 0x5591e4bdc390 2024-09-21T07:43:04.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.182+0000 7f32aa800640 1 --2- 172.21.15.39:0/3931370836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5591e4bdc390 0x5591e4faa150 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f32a4002170 tx=0x7f32a4001510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cdf05d2d8750409 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:04.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.182+0000 7f329a000640 1 -- 172.21.15.39:0/3931370836 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a40064e0 con 0x5591e4bdc390 2024-09-21T07:43:04.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.182+0000 7f329a000640 1 -- 172.21.15.39:0/3931370836 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32a4006770 con 0x5591e4bdc390 2024-09-21T07:43:04.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.182+0000 7f329a000640 1 -- 172.21.15.39:0/3931370836 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a40266e0 con 0x5591e4bdc390 2024-09-21T07:43:04.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.183+0000 7f32b7dc3740 1 -- 172.21.15.39:0/3931370836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5591e4bdc390 msgr2=0x5591e4faa150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:04.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.183+0000 7f32b7dc3740 1 --2- 172.21.15.39:0/3931370836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5591e4bdc390 0x5591e4faa150 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f32a4002170 tx=0x7f32a4001510 comp rx=0 tx=0).stop 2024-09-21T07:43:04.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.183+0000 7f32b7dc3740 1 -- 172.21.15.39:0/3931370836 shutdown_connections 2024-09-21T07:43:04.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.183+0000 7f32b7dc3740 1 --2- 172.21.15.39:0/3931370836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5591e4bdc390 0x5591e4faa150 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:04.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.183+0000 7f32b7dc3740 1 --2- 172.21.15.39:0/3931370836 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5591e4bbb6b0 0x5591e4fded20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:04.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.183+0000 7f32b7dc3740 1 --2- 172.21.15.39:0/3931370836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5591e4bb34d0 0x5591e4f7d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:04.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.183+0000 7f32b7dc3740 1 -- 172.21.15.39:0/3931370836 >> 172.21.15.39:0/3931370836 conn(0x5591e4bd04e0 msgr2=0x5591e4fbf440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:04.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.183+0000 7f32b7dc3740 1 -- 172.21.15.39:0/3931370836 shutdown_connections 2024-09-21T07:43:04.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.183+0000 7f32b7dc3740 1 -- 172.21.15.39:0/3931370836 wait complete. 2024-09-21T07:43:04.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.184+0000 7f32b7dc3740 1 -- start start 2024-09-21T07:43:04.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.184+0000 7f32b7dc3740 1 Processor -- start 2024-09-21T07:43:04.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.186+0000 7f32b7dc3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5591e4bb34d0 0x5591e4fde610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:04.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.186+0000 7f32b7dc3740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5591e4bbb6b0 0x5591e4fef610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:04.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.186+0000 7f32b7dc3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5591e4bdc390 0x5591e4faa150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:04.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.186+0000 7f32b7dc3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac2000b0 con 0x5591e4bb34d0 2024-09-21T07:43:04.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.186+0000 7f32b7dc3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5591e4fc1340 con 0x5591e4bdc390 2024-09-21T07:43:04.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.186+0000 7f32b7dc3740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x5591e4fdb1b0 con 0x5591e4bbb6b0 2024-09-21T07:43:04.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.186+0000 7f32b7dc3740 10 client.0 _ll_register_callbacks cb 0x7f329c0056d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:43:04.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.186+0000 7f32a9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5591e4bb34d0 0x5591e4fde610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:04.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.186+0000 7f32a9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5591e4bb34d0 0x5591e4fde610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36744/0 (socket says 172.21.15.39:36744) 2024-09-21T07:43:04.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.186+0000 7f32a9e00640 1 -- 172.21.15.39:0/4159936031 learned_addr learned my addr 172.21.15.39:0/4159936031 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:04.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.186+0000 7f32a1400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5591e4bbb6b0 0x5591e4fef610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:04.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.186+0000 7f32aa800640 1 --2- 172.21.15.39:0/4159936031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5591e4bdc390 0x5591e4faa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:04.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.186+0000 7f32a9e00640 1 -- 172.21.15.39:0/4159936031 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5591e4bbb6b0 msgr2=0x5591e4fef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:43:04.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.187+0000 7f32a9e00640 1 --2- 172.21.15.39:0/4159936031 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5591e4bbb6b0 0x5591e4fef610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:04.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.187+0000 7f32a9e00640 1 -- 172.21.15.39:0/4159936031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5591e4bdc390 msgr2=0x5591e4faa150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:43:04.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.187+0000 7f32a9e00640 1 --2- 172.21.15.39:0/4159936031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5591e4bdc390 0x5591e4faa150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:04.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.187+0000 7f32a9e00640 1 -- 172.21.15.39:0/4159936031 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591e4fcad50 con 0x5591e4bb34d0 2024-09-21T07:43:04.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.187+0000 7f32a9e00640 1 --2- 172.21.15.39:0/4159936031 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5591e4bb34d0 0x5591e4fde610 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f329c006670 tx=0x7f329c005510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b0e93e2a7fc924 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:04.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.187+0000 7f328f400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:43:04.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.187+0000 7f328f400640 1 -- 172.21.15.39:0/4159936031 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c03a060 con 0x5591e4bb34d0 2024-09-21T07:43:04.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.187+0000 7f328f400640 1 -- 172.21.15.39:0/4159936031 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8076+0+0 (secure 0 0 0) 0x7f329c003b90 con 0x5591e4bb34d0 2024-09-21T07:43:04.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.187+0000 7f32b7dc3740 1 -- 172.21.15.39:0/4159936031 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32a4006770 con 0x5591e4bb34d0 2024-09-21T07:43:04.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.187+0000 7f328f400640 1 -- 172.21.15.39:0/4159936031 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=142) ==== 24+0+0 (secure 0 0 0) 0x7f329c003d90 con 0x5591e4bb34d0 2024-09-21T07:43:04.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.187+0000 7f32b7dc3740 10 client.7971 fetch_fsmap learned FSMap version 142 2024-09-21T07:43:04.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.187+0000 7f32b7dc3740 1 -- 172.21.15.39:0/4159936031 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=142,monmap=2+}) -- 0x5591e504fa70 con 0x5591e4bb34d0 2024-09-21T07:43:04.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.188+0000 7f328f400640 1 -- 172.21.15.39:0/4159936031 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32a4006770 con 0x5591e4bb34d0 2024-09-21T07:43:04.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.188+0000 7f328f400640 1 -- 172.21.15.39:0/4159936031 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 142) ==== 3522+0+0 (secure 0 0 0) 0x7f329c029480 con 0x5591e4bb34d0 2024-09-21T07:43:04.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.188+0000 7f32b7dc3740 10 client.7971 fetch_fsmap finished waiting for FSMap version 142 2024-09-21T07:43:04.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.188+0000 7f32b7dc3740 10 client.7971 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-21T07:43:04.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.188+0000 7f32b7dc3740 20 client.7971 populate_metadata read hostname 'smithi039' 2024-09-21T07:43:04.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.188+0000 7f32b7dc3740 1 --2- 172.21.15.39:0/4159936031 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x5591e4faa690 0x5591e504ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:04.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.188+0000 7f32b7dc3740 4 client.7971 mds_command: new command op to 7311 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:43:04.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.188+0000 7f32b7dc3740 1 -- 172.21.15.39:0/4159936031 --> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5591e4ef4270 con 0x5591e4faa690 2024-09-21T07:43:04.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.188+0000 7f32a1400640 1 --2- 172.21.15.39:0/4159936031 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x5591e4faa690 0x5591e504ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:04.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.189+0000 7f32a1400640 1 --2- 172.21.15.39:0/4159936031 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x5591e4faa690 0x5591e504ec40 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2186c3d7272c46d6 server_cookie=4cc88be4518fb2e9 in_seq=0 out_seq=0 2024-09-21T07:43:04.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.189+0000 7f328f400640 10 client.7971 ms_handle_connect on v2:172.21.15.39:6832/1790581246 2024-09-21T07:43:04.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.199+0000 7f328f400640 1 -- 172.21.15.39:0/4159936031 <== mds.0 v2:172.21.15.39:6832/1790581246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5591e4ef4270 con 0x5591e4faa690 2024-09-21T07:43:04.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.199+0000 7f328f400640 10 client.7971 handle_command_reply: tid=0 2024-09-21T07:43:04.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.199+0000 7f32b7dc3740 1 client.7971 shutdown 2024-09-21T07:43:04.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.199+0000 7f32b7dc3740 1 -- 172.21.15.39:0/4159936031 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5591e4bb34d0 msgr2=0x5591e4fde610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:04.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.199+0000 7f32b7dc3740 1 --2- 172.21.15.39:0/4159936031 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5591e4bb34d0 0x5591e4fde610 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f329c006670 tx=0x7f329c005510 comp rx=0 tx=0).stop 2024-09-21T07:43:04.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.199+0000 7f32b7dc3740 1 -- 172.21.15.39:0/4159936031 shutdown_connections 2024-09-21T07:43:04.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.199+0000 7f32b7dc3740 1 --2- 172.21.15.39:0/4159936031 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x5591e4faa690 0x5591e504ec40 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:04.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.199+0000 7f32b7dc3740 1 --2- 172.21.15.39:0/4159936031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5591e4bdc390 0x5591e4faa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:04.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.199+0000 7f32b7dc3740 1 --2- 172.21.15.39:0/4159936031 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5591e4bbb6b0 0x5591e4fef610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:04.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.199+0000 7f328f400640 0 client.7971 ms_handle_reset on v2:172.21.15.39:6832/1790581246 2024-09-21T07:43:04.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.199+0000 7f32b7dc3740 1 --2- 172.21.15.39:0/4159936031 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5591e4bb34d0 0x5591e4fde610 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:04.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.199+0000 7f32b7dc3740 1 -- 172.21.15.39:0/4159936031 >> 172.21.15.39:0/4159936031 conn(0x5591e4bd04e0 msgr2=0x5591e4fbfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:04.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.200+0000 7f32b7dc3740 1 -- 172.21.15.39:0/4159936031 shutdown_connections 2024-09-21T07:43:04.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.200+0000 7f32b7dc3740 1 -- 172.21.15.39:0/4159936031 wait complete. 2024-09-21T07:43:04.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.200+0000 7f32b7dc3740 20 client.7971 trim_cache size 0 max 16384 2024-09-21T07:43:04.203 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:43:04.203 INFO:teuthology.orchestra.run.smithi039.stdout: "message": "", 2024-09-21T07:43:04.203 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0 2024-09-21T07:43:04.203 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:43:04.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.200+0000 7f32b0e00640 1 -- 172.21.15.39:0/4182003023 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3288078b90 msgr2=0x7f328807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:04.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.200+0000 7f32b0e00640 1 --2- 172.21.15.39:0/4182003023 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3288078b90 0x7f328807b0a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f329c006bb0 tx=0x7f329c006720 comp rx=0 tx=0).stop 2024-09-21T07:43:04.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.200+0000 7f32b0e00640 1 -- 172.21.15.39:0/4182003023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32ac15f0e0 msgr2=0x7f32ac1f43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:04.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.201+0000 7f32b0e00640 1 --2- 172.21.15.39:0/4182003023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32ac15f0e0 0x7f32ac1f43d0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f3294036040 tx=0x7f3294033670 comp rx=0 tx=0).stop 2024-09-21T07:43:04.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.201+0000 7f32b0e00640 1 -- 172.21.15.39:0/4182003023 shutdown_connections 2024-09-21T07:43:04.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.201+0000 7f32b0e00640 1 --2- 172.21.15.39:0/4182003023 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3288078b90 0x7f328807b0a0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:04.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.201+0000 7f32b0e00640 1 --2- 172.21.15.39:0/4182003023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32ac157bd0 0x7f32ac157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:04.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.201+0000 7f32b0e00640 1 --2- 172.21.15.39:0/4182003023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32ac15f0e0 0x7f32ac1f43d0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:04.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.201+0000 7f32b0e00640 1 --2- 172.21.15.39:0/4182003023 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3290004070 0x7f32ac099950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:04.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.201+0000 7f32b0e00640 1 -- 172.21.15.39:0/4182003023 >> 172.21.15.39:0/4182003023 conn(0x7f32ac06d440 msgr2=0x7f32ac0b1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:04.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.201+0000 7f32b0e00640 1 -- 172.21.15.39:0/4182003023 shutdown_connections 2024-09-21T07:43:04.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.201+0000 7f32b0e00640 1 -- 172.21.15.39:0/4182003023 wait complete. 2024-09-21T07:43:04.224 INFO:tasks.cephfs.test_snapshots:testing snapclient mds_kill_mdstable_at=7 2024-09-21T07:43:04.224 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:43:04.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.448+0000 7f2b56800640 1 -- 172.21.15.39:0/330996179 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b500cb960 msgr2=0x7f2b500cbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:04.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.448+0000 7f2b56800640 1 --2- 172.21.15.39:0/330996179 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b500cb960 0x7f2b500cbd70 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c009830 tx=0x7f2b3c02a510 comp rx=0 tx=0).stop 2024-09-21T07:43:04.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.448+0000 7f2b56800640 1 -- 172.21.15.39:0/330996179 shutdown_connections 2024-09-21T07:43:04.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.448+0000 7f2b56800640 1 --2- 172.21.15.39:0/330996179 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b500cb960 0x7f2b500cbd70 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:04.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.448+0000 7f2b56800640 1 --2- 172.21.15.39:0/330996179 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2b500d0f20 0x7f2b5015c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:04.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.448+0000 7f2b56800640 1 -- 172.21.15.39:0/330996179 >> 172.21.15.39:0/330996179 conn(0x7f2b500b2510 msgr2=0x7f2b500d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:04.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.448+0000 7f2b56800640 1 -- 172.21.15.39:0/330996179 shutdown_connections 2024-09-21T07:43:04.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.448+0000 7f2b56800640 1 -- 172.21.15.39:0/330996179 wait complete. 2024-09-21T07:43:04.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.450+0000 7f2b56800640 1 Processor -- start 2024-09-21T07:43:04.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.450+0000 7f2b56800640 1 -- start start 2024-09-21T07:43:04.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.452+0000 7f2b56800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2b5006f8a0 0x7f2b5009c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:04.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.452+0000 7f2b56800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b500cb960 0x7f2b50159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:04.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.452+0000 7f2b56800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b500d0f20 0x7f2b5015f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:04.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.452+0000 7f2b56800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2b500b4570 con 0x7f2b500cb960 2024-09-21T07:43:04.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.452+0000 7f2b56800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2b500b43f0 con 0x7f2b500d0f20 2024-09-21T07:43:04.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.452+0000 7f2b56800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f2b500b46f0 con 0x7f2b5006f8a0 2024-09-21T07:43:04.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.452+0000 7f2b54a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b500cb960 0x7f2b50159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:04.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.452+0000 7f2b54a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b500cb960 0x7f2b50159010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36770/0 (socket says 172.21.15.39:36770) 2024-09-21T07:43:04.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.452+0000 7f2b54a00640 1 -- 172.21.15.39:0/1456363267 learned_addr learned my addr 172.21.15.39:0/1456363267 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:04.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.452+0000 7f2b55400640 1 --2- 172.21.15.39:0/1456363267 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2b5006f8a0 0x7f2b5009c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:04.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.452+0000 7f2b55e00640 1 --2- 172.21.15.39:0/1456363267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b500d0f20 0x7f2b5015f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:04.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.453+0000 7f2b54a00640 1 -- 172.21.15.39:0/1456363267 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2b5006f8a0 msgr2=0x7f2b5009c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:04.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.453+0000 7f2b54a00640 1 --2- 172.21.15.39:0/1456363267 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2b5006f8a0 0x7f2b5009c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:04.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.453+0000 7f2b54a00640 1 -- 172.21.15.39:0/1456363267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b500d0f20 msgr2=0x7f2b5015f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:04.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.453+0000 7f2b54a00640 1 --2- 172.21.15.39:0/1456363267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b500d0f20 0x7f2b5015f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:04.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.453+0000 7f2b54a00640 1 -- 172.21.15.39:0/1456363267 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b5009cb50 con 0x7f2b500cb960 2024-09-21T07:43:04.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.453+0000 7f2b54a00640 1 --2- 172.21.15.39:0/1456363267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b500cb960 0x7f2b50159010 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c02ab00 tx=0x7f2b3c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:04.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.453+0000 7f2b4d600640 1 -- 172.21.15.39:0/1456363267 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c031d60 con 0x7f2b500cb960 2024-09-21T07:43:04.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.453+0000 7f2b4d600640 1 -- 172.21.15.39:0/1456363267 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b3c033e10 con 0x7f2b500cb960 2024-09-21T07:43:04.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.453+0000 7f2b4d600640 1 -- 172.21.15.39:0/1456363267 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c005260 con 0x7f2b500cb960 2024-09-21T07:43:04.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.453+0000 7f2b56800640 1 -- 172.21.15.39:0/1456363267 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b501f3450 con 0x7f2b500cb960 2024-09-21T07:43:04.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.454+0000 7f2b56800640 1 -- 172.21.15.39:0/1456363267 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b501f3a10 con 0x7f2b500cb960 2024-09-21T07:43:04.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.455+0000 7f2b56800640 1 -- 172.21.15.39:0/1456363267 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b500b4c50 con 0x7f2b500cb960 2024-09-21T07:43:04.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.456+0000 7f2b4d600640 1 -- 172.21.15.39:0/1456363267 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2b3c005400 con 0x7f2b500cb960 2024-09-21T07:43:04.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.457+0000 7f2b4d600640 1 --2- 172.21.15.39:0/1456363267 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2b30078b90 0x7f2b3007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:04.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.457+0000 7f2b4d600640 1 -- 172.21.15.39:0/1456363267 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8076+0+0 (secure 0 0 0) 0x7f2b3c0bc980 con 0x7f2b500cb960 2024-09-21T07:43:04.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.457+0000 7f2b55400640 1 --2- 172.21.15.39:0/1456363267 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2b30078b90 0x7f2b3007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:04.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.458+0000 7f2b55400640 1 --2- 172.21.15.39:0/1456363267 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2b30078b90 0x7f2b3007b0a0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2b4400a490 tx=0x7f2b4400a420 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:04.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.461+0000 7f2b4d600640 1 -- 172.21.15.39:0/1456363267 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2b3c03f030 con 0x7f2b500cb960 2024-09-21T07:43:04.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.696+0000 7f2b56800640 1 -- 172.21.15.39:0/1456363267 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b5009c8c0 con 0x7f2b500cb960 2024-09-21T07:43:04.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.697+0000 7f2b4d600640 1 -- 172.21.15.39:0/1456363267 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 142 v142) ==== 95+0+5256 (secure 0 0 0) 0x7f2b3c083e20 con 0x7f2b500cb960 2024-09-21T07:43:04.699 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:43:04.699 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":142,"btime":"2024-09-21T07:42:54:142083+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7366,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/759792812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":759792812},{"type":"v1","addr":"172.21.15.148:6837","nonce":759792812}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":139}],"filesystems":[{"mdsmap":{"epoch":142,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:42:54.142073+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7644},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7644":{"gid":7644,"name":"a","rank":1,"incarnation":138,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/1326711417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1326711417},{"type":"v1","addr":"172.21.15.39:6835","nonce":1326711417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311,7644]},"id":6}]} 2024-09-21T07:43:04.699 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 142 2024-09-21T07:43:04.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.699+0000 7f2b56800640 1 -- 172.21.15.39:0/1456363267 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2b30078b90 msgr2=0x7f2b3007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:04.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.699+0000 7f2b56800640 1 --2- 172.21.15.39:0/1456363267 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2b30078b90 0x7f2b3007b0a0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2b4400a490 tx=0x7f2b4400a420 comp rx=0 tx=0).stop 2024-09-21T07:43:04.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.699+0000 7f2b56800640 1 -- 172.21.15.39:0/1456363267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b500cb960 msgr2=0x7f2b50159010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:04.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.699+0000 7f2b56800640 1 --2- 172.21.15.39:0/1456363267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b500cb960 0x7f2b50159010 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c02ab00 tx=0x7f2b3c033c20 comp rx=0 tx=0).stop 2024-09-21T07:43:04.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.700+0000 7f2b56800640 1 -- 172.21.15.39:0/1456363267 shutdown_connections 2024-09-21T07:43:04.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.700+0000 7f2b56800640 1 --2- 172.21.15.39:0/1456363267 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2b30078b90 0x7f2b3007b0a0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:04.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.700+0000 7f2b56800640 1 --2- 172.21.15.39:0/1456363267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b500d0f20 0x7f2b5015f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:04.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.700+0000 7f2b56800640 1 --2- 172.21.15.39:0/1456363267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b500cb960 0x7f2b50159010 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:04.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.700+0000 7f2b56800640 1 --2- 172.21.15.39:0/1456363267 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2b5006f8a0 0x7f2b5009c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:04.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.700+0000 7f2b56800640 1 -- 172.21.15.39:0/1456363267 >> 172.21.15.39:0/1456363267 conn(0x7f2b500b2510 msgr2=0x7f2b5015cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:04.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.700+0000 7f2b56800640 1 -- 172.21.15.39:0/1456363267 shutdown_connections 2024-09-21T07:43:04.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:04.700+0000 7f2b56800640 1 -- 172.21.15.39:0/1456363267 wait complete. 2024-09-21T07:43:04.727 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2024-09-21T07:43:04.971 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_snap":16,"last_created":16,"last_destroyed":14,"pending_noop":[],"snaps":[{"snapid":16,"ino":1099511627777,"stamp":"2024-09-21T07:42:21.254582+0000","name":"s34","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2024-09-21T07:43:04.971 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 16, "last_destroyed": 14, "last_snap": 16, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s34", "snapid": 16, "stamp": "2024-09-21T07:42:21.254582+0000" } ] } 2024-09-21T07:43:04.972 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:43:05.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.199+0000 7fab10c00640 1 -- 172.21.15.39:0/3275326615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab0c0b5970 msgr2=0x7fab0c0b20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:05.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.199+0000 7fab10c00640 1 --2- 172.21.15.39:0/3275326615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab0c0b5970 0x7fab0c0b20a0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fab0c0d27d0 tx=0x7fab0402aa30 comp rx=0 tx=0).stop 2024-09-21T07:43:05.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.199+0000 7fab10c00640 1 -- 172.21.15.39:0/3275326615 shutdown_connections 2024-09-21T07:43:05.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.199+0000 7fab10c00640 1 --2- 172.21.15.39:0/3275326615 >> v2:172.21.15.148:3300/0 conn(0x7fab0c0d4d10 0x7fab0c0d1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:05.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.199+0000 7fab10c00640 1 --2- 172.21.15.39:0/3275326615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab0c0b5970 0x7fab0c0b20a0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:05.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.199+0000 7fab10c00640 1 -- 172.21.15.39:0/3275326615 >> 172.21.15.39:0/3275326615 conn(0x7fab0c08bfb0 msgr2=0x7fab0c09d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:05.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.199+0000 7fab10c00640 1 -- 172.21.15.39:0/3275326615 shutdown_connections 2024-09-21T07:43:05.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.199+0000 7fab10c00640 1 -- 172.21.15.39:0/3275326615 wait complete. 2024-09-21T07:43:05.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.201+0000 7fab10c00640 1 Processor -- start 2024-09-21T07:43:05.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.201+0000 7fab10c00640 1 -- start start 2024-09-21T07:43:05.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.203+0000 7fab10c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab0c0b3e20 0x7fab0c1f5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:05.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.203+0000 7fab10c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab0c0b5970 0x7fab0c15f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:05.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.203+0000 7fab10c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab0c15ab00 0x7fab0c15af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:05.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.203+0000 7fab10c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c0b27f0 con 0x7fab0c0b3e20 2024-09-21T07:43:05.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.203+0000 7fab10c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c0b5ef0 con 0x7fab0c15ab00 2024-09-21T07:43:05.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.203+0000 7fab10c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fab0c0b2970 con 0x7fab0c0b5970 2024-09-21T07:43:05.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.203+0000 7fab0b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab0c0b3e20 0x7fab0c1f5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:05.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.203+0000 7fab0b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab0c0b3e20 0x7fab0c1f5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36790/0 (socket says 172.21.15.39:36790) 2024-09-21T07:43:05.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.203+0000 7fab0b400640 1 -- 172.21.15.39:0/4148520295 learned_addr learned my addr 172.21.15.39:0/4148520295 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:05.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.203+0000 7fab03e00640 1 --2- 172.21.15.39:0/4148520295 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab0c0b5970 0x7fab0c15f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:05.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.203+0000 7fab0be00640 1 --2- 172.21.15.39:0/4148520295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab0c15ab00 0x7fab0c15af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:05.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.203+0000 7fab0b400640 1 -- 172.21.15.39:0/4148520295 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab0c0b5970 msgr2=0x7fab0c15f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:05.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.203+0000 7fab0b400640 1 --2- 172.21.15.39:0/4148520295 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab0c0b5970 0x7fab0c15f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:05.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.203+0000 7fab0b400640 1 -- 172.21.15.39:0/4148520295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab0c15ab00 msgr2=0x7fab0c15af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:05.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.203+0000 7fab0b400640 1 --2- 172.21.15.39:0/4148520295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab0c15ab00 0x7fab0c15af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:05.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.203+0000 7fab0b400640 1 -- 172.21.15.39:0/4148520295 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab0c15b450 con 0x7fab0c0b3e20 2024-09-21T07:43:05.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.204+0000 7fab0b400640 1 --2- 172.21.15.39:0/4148520295 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab0c0b3e20 0x7fab0c1f5d90 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7faafc00c3a0 tx=0x7faafc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:05.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.204+0000 7fab03400640 1 -- 172.21.15.39:0/4148520295 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faafc017070 con 0x7fab0c0b3e20 2024-09-21T07:43:05.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.204+0000 7fab03400640 1 -- 172.21.15.39:0/4148520295 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faafc0053c0 con 0x7fab0c0b3e20 2024-09-21T07:43:05.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.204+0000 7fab10c00640 1 -- 172.21.15.39:0/4148520295 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab0c15d1d0 con 0x7fab0c0b3e20 2024-09-21T07:43:05.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.204+0000 7fab03400640 1 -- 172.21.15.39:0/4148520295 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faafc003850 con 0x7fab0c0b3e20 2024-09-21T07:43:05.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.204+0000 7fab10c00640 1 -- 172.21.15.39:0/4148520295 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab0c15d790 con 0x7fab0c0b3e20 2024-09-21T07:43:05.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.205+0000 7fab10c00640 1 -- 172.21.15.39:0/4148520295 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab0c069a50 con 0x7fab0c0b3e20 2024-09-21T07:43:05.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.207+0000 7fab03400640 1 -- 172.21.15.39:0/4148520295 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7faafc01a070 con 0x7fab0c0b3e20 2024-09-21T07:43:05.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.207+0000 7fab03400640 1 --2- 172.21.15.39:0/4148520295 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faae8078bc0 0x7faae807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:05.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.208+0000 7fab03400640 1 -- 172.21.15.39:0/4148520295 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8076+0+0 (secure 0 0 0) 0x7faafc09aa20 con 0x7fab0c0b3e20 2024-09-21T07:43:05.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.208+0000 7fab03e00640 1 --2- 172.21.15.39:0/4148520295 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faae8078bc0 0x7faae807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:05.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.209+0000 7fab03e00640 1 --2- 172.21.15.39:0/4148520295 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faae8078bc0 0x7faae807b0d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7faaf400bb00 tx=0x7faaf4005e30 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:05.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.212+0000 7fab03400640 1 -- 172.21.15.39:0/4148520295 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fab0c15b450 con 0x7fab0c0b3e20 2024-09-21T07:43:05.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.447+0000 7fab10c00640 1 -- 172.21.15.39:0/4148520295 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab0c0b2e20 con 0x7fab0c0b3e20 2024-09-21T07:43:05.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.447+0000 7fab03400640 1 -- 172.21.15.39:0/4148520295 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 142 v142) ==== 95+0+5256 (secure 0 0 0) 0x7faafc062520 con 0x7fab0c0b3e20 2024-09-21T07:43:05.449 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:43:05.450 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":142,"btime":"2024-09-21T07:42:54:142083+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7366,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/759792812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":759792812},{"type":"v1","addr":"172.21.15.148:6837","nonce":759792812}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":139}],"filesystems":[{"mdsmap":{"epoch":142,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:42:54.142073+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7644},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7644":{"gid":7644,"name":"a","rank":1,"incarnation":138,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/1326711417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1326711417},{"type":"v1","addr":"172.21.15.39:6835","nonce":1326711417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311,7644]},"id":6}]} 2024-09-21T07:43:05.450 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 142 2024-09-21T07:43:05.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.450+0000 7fab10c00640 1 -- 172.21.15.39:0/4148520295 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faae8078bc0 msgr2=0x7faae807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:05.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.450+0000 7fab10c00640 1 --2- 172.21.15.39:0/4148520295 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faae8078bc0 0x7faae807b0d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7faaf400bb00 tx=0x7faaf4005e30 comp rx=0 tx=0).stop 2024-09-21T07:43:05.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.450+0000 7fab10c00640 1 -- 172.21.15.39:0/4148520295 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab0c0b3e20 msgr2=0x7fab0c1f5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:05.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.450+0000 7fab10c00640 1 --2- 172.21.15.39:0/4148520295 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab0c0b3e20 0x7fab0c1f5d90 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7faafc00c3a0 tx=0x7faafc00c860 comp rx=0 tx=0).stop 2024-09-21T07:43:05.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.450+0000 7fab10c00640 1 -- 172.21.15.39:0/4148520295 shutdown_connections 2024-09-21T07:43:05.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.450+0000 7fab10c00640 1 --2- 172.21.15.39:0/4148520295 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faae8078bc0 0x7faae807b0d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:05.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.450+0000 7fab10c00640 1 --2- 172.21.15.39:0/4148520295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab0c15ab00 0x7fab0c15af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:05.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.450+0000 7fab10c00640 1 --2- 172.21.15.39:0/4148520295 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab0c0b5970 0x7fab0c15f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:05.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.450+0000 7fab10c00640 1 --2- 172.21.15.39:0/4148520295 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab0c0b3e20 0x7fab0c1f5d90 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:05.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.450+0000 7fab10c00640 1 -- 172.21.15.39:0/4148520295 >> 172.21.15.39:0/4148520295 conn(0x7fab0c08bfb0 msgr2=0x7fab0c161b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:05.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.450+0000 7fab10c00640 1 -- 172.21.15.39:0/4148520295 shutdown_connections 2024-09-21T07:43:05.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.450+0000 7fab10c00640 1 -- 172.21.15.39:0/4148520295 wait complete. 2024-09-21T07:43:05.473 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:1 true 2024-09-21T07:43:05.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.698+0000 7f540fe00640 1 -- 172.21.15.39:0/1124134374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f540815cf60 msgr2=0x7f54080b32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:05.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.698+0000 7f540fe00640 1 --2- 172.21.15.39:0/1124134374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f540815cf60 0x7f54080b32b0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f53fc005fd0 tx=0x7f53fc02a540 comp rx=0 tx=0).stop 2024-09-21T07:43:05.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.698+0000 7f540fe00640 1 -- 172.21.15.39:0/1124134374 shutdown_connections 2024-09-21T07:43:05.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.699+0000 7f540fe00640 1 --2- 172.21.15.39:0/1124134374 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f54080cce70 0x7f54080cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:05.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.699+0000 7f540fe00640 1 --2- 172.21.15.39:0/1124134374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f540815cf60 0x7f54080b32b0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:05.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.699+0000 7f540fe00640 1 -- 172.21.15.39:0/1124134374 >> 172.21.15.39:0/1124134374 conn(0x7f540806d440 msgr2=0x7f54080b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:05.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.699+0000 7f540fe00640 1 -- 172.21.15.39:0/1124134374 shutdown_connections 2024-09-21T07:43:05.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.699+0000 7f540fe00640 1 -- 172.21.15.39:0/1124134374 wait complete. 2024-09-21T07:43:05.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.701+0000 7f540fe00640 1 Processor -- start 2024-09-21T07:43:05.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.701+0000 7f540fe00640 1 -- start start 2024-09-21T07:43:05.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.702+0000 7f540fe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54080cce70 0x7f540815fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:05.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.702+0000 7f540fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f540815cf60 0x7f5408157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:05.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.703+0000 7f540fe00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f540815d950 0x7f5408157570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:05.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.703+0000 7f540fe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f540806f9e0 con 0x7f540815cf60 2024-09-21T07:43:05.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.703+0000 7f540fe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f540806f860 con 0x7f54080cce70 2024-09-21T07:43:05.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.703+0000 7f540fe00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f540806fb60 con 0x7f540815d950 2024-09-21T07:43:05.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.703+0000 7f540ca00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f540815cf60 0x7f5408157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:05.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.703+0000 7f540ca00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f540815cf60 0x7f5408157030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36808/0 (socket says 172.21.15.39:36808) 2024-09-21T07:43:05.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.703+0000 7f540ca00640 1 -- 172.21.15.39:0/217275230 learned_addr learned my addr 172.21.15.39:0/217275230 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:05.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.703+0000 7f540d400640 1 --2- 172.21.15.39:0/217275230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54080cce70 0x7f540815fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:05.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.703+0000 7f540de00640 1 --2- 172.21.15.39:0/217275230 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f540815d950 0x7f5408157570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:05.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.703+0000 7f540ca00640 1 -- 172.21.15.39:0/217275230 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f540815d950 msgr2=0x7f5408157570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:05.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.703+0000 7f540ca00640 1 --2- 172.21.15.39:0/217275230 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f540815d950 0x7f5408157570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:05.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.703+0000 7f540ca00640 1 -- 172.21.15.39:0/217275230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54080cce70 msgr2=0x7f540815fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:05.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.703+0000 7f540ca00640 1 --2- 172.21.15.39:0/217275230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54080cce70 0x7f540815fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:05.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.703+0000 7f540ca00640 1 -- 172.21.15.39:0/217275230 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5408159330 con 0x7f540815cf60 2024-09-21T07:43:05.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.703+0000 7f540d400640 1 --2- 172.21.15.39:0/217275230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54080cce70 0x7f540815fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:43:05.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.704+0000 7f540ca00640 1 --2- 172.21.15.39:0/217275230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f540815cf60 0x7f5408157030 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f53f400c3a0 tx=0x7f53f400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:05.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.704+0000 7f5405600640 1 -- 172.21.15.39:0/217275230 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f4017070 con 0x7f540815cf60 2024-09-21T07:43:05.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.704+0000 7f5405600640 1 -- 172.21.15.39:0/217275230 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53f40053c0 con 0x7f540815cf60 2024-09-21T07:43:05.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.704+0000 7f540fe00640 1 -- 172.21.15.39:0/217275230 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5408159610 con 0x7f540815cf60 2024-09-21T07:43:05.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.704+0000 7f5405600640 1 -- 172.21.15.39:0/217275230 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f4003850 con 0x7f540815cf60 2024-09-21T07:43:05.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.704+0000 7f540fe00640 1 -- 172.21.15.39:0/217275230 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5408159bd0 con 0x7f540815cf60 2024-09-21T07:43:05.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.705+0000 7f540fe00640 1 -- 172.21.15.39:0/217275230 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f540806fff0 con 0x7f540815cf60 2024-09-21T07:43:05.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.706+0000 7f5405600640 1 -- 172.21.15.39:0/217275230 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f53f401a070 con 0x7f540815cf60 2024-09-21T07:43:05.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.707+0000 7f5405600640 1 --2- 172.21.15.39:0/217275230 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f53e8078bc0 0x7f53e807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:05.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.707+0000 7f5405600640 1 -- 172.21.15.39:0/217275230 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8076+0+0 (secure 0 0 0) 0x7f53f409aa50 con 0x7f540815cf60 2024-09-21T07:43:05.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.708+0000 7f540d400640 1 --2- 172.21.15.39:0/217275230 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f53e8078bc0 0x7f53e807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:05.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.708+0000 7f540d400640 1 --2- 172.21.15.39:0/217275230 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f53e8078bc0 0x7f53e807b0d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f53fc033000 tx=0x7f53fc004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:05.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.712+0000 7f5405600640 1 -- 172.21.15.39:0/217275230 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f54080cd7c0 con 0x7f540815cf60 2024-09-21T07:43:05.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:05.986+0000 7f540fe00640 1 -- 172.21.15.39:0/217275230 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:1", "val": "true"} v 0) -- 0x7f54080753a0 con 0x7f540815cf60 2024-09-21T07:43:06.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.191+0000 7f5405600640 1 -- 172.21.15.39:0/217275230 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:1", "val": "true"}]=0 freezing mds.7644 v143) ==== 112+0+0 (secure 0 0 0) 0x7f53f40624b0 con 0x7f540815cf60 2024-09-21T07:43:06.193 INFO:teuthology.orchestra.run.smithi039.stderr:freezing mds.7644 2024-09-21T07:43:06.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.193+0000 7f540fe00640 1 -- 172.21.15.39:0/217275230 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f53e8078bc0 msgr2=0x7f53e807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:06.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.193+0000 7f540fe00640 1 --2- 172.21.15.39:0/217275230 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f53e8078bc0 0x7f53e807b0d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f53fc033000 tx=0x7f53fc004730 comp rx=0 tx=0).stop 2024-09-21T07:43:06.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.193+0000 7f540fe00640 1 -- 172.21.15.39:0/217275230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f540815cf60 msgr2=0x7f5408157030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:06.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.193+0000 7f540fe00640 1 --2- 172.21.15.39:0/217275230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f540815cf60 0x7f5408157030 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f53f400c3a0 tx=0x7f53f400c860 comp rx=0 tx=0).stop 2024-09-21T07:43:06.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.194+0000 7f540fe00640 1 -- 172.21.15.39:0/217275230 shutdown_connections 2024-09-21T07:43:06.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.194+0000 7f540fe00640 1 --2- 172.21.15.39:0/217275230 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f53e8078bc0 0x7f53e807b0d0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:06.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.194+0000 7f540fe00640 1 --2- 172.21.15.39:0/217275230 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f540815d950 0x7f5408157570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:06.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.194+0000 7f540fe00640 1 --2- 172.21.15.39:0/217275230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f540815cf60 0x7f5408157030 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:06.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.194+0000 7f540fe00640 1 --2- 172.21.15.39:0/217275230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54080cce70 0x7f540815fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:06.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.194+0000 7f540fe00640 1 -- 172.21.15.39:0/217275230 >> 172.21.15.39:0/217275230 conn(0x7f540806d440 msgr2=0x7f54080b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:06.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.194+0000 7f540fe00640 1 -- 172.21.15.39:0/217275230 shutdown_connections 2024-09-21T07:43:06.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.194+0000 7f540fe00640 1 -- 172.21.15.39:0/217275230 wait complete. 2024-09-21T07:43:06.219 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_kill_mdstable_at 7 2024-09-21T07:43:06.460 INFO:teuthology.orchestra.run.smithi039.stdout:{"success":"mds_kill_mdstable_at = '' (not observed, change may require restart) "} 2024-09-21T07:43:06.460 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_kill_mdstable_at = '' (not observed, change may require restart) " } 2024-09-21T07:43:06.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:43:06.461 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d1/dir/.snap/s37) 2024-09-21T07:43:06.465 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-09-21T07:43:06.738 INFO:teuthology.orchestra.run.smithi039.stdout:{"mds_beacon_grace":"15.000000"} 2024-09-21T07:43:06.739 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-09-21T07:43:06.739 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:43:06.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.962+0000 7f755f800640 1 -- 172.21.15.39:0/2338925582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75580b3370 msgr2=0x7f75580b3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:06.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.962+0000 7f755f800640 1 --2- 172.21.15.39:0/2338925582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75580b3370 0x7f75580b3780 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f755806fe60 tx=0x7f754402a4d0 comp rx=0 tx=0).stop 2024-09-21T07:43:06.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.963+0000 7f755f800640 1 -- 172.21.15.39:0/2338925582 shutdown_connections 2024-09-21T07:43:06.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.963+0000 7f755f800640 1 --2- 172.21.15.39:0/2338925582 >> v2:172.21.15.148:3300/0 conn(0x7f7558151590 0x7f75581519a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:06.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.963+0000 7f755f800640 1 --2- 172.21.15.39:0/2338925582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75580b3370 0x7f75580b3780 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:06.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.963+0000 7f755f800640 1 --2- 172.21.15.39:0/2338925582 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f755815e590 0x7f7558070a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:06.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.963+0000 7f755f800640 1 -- 172.21.15.39:0/2338925582 >> 172.21.15.39:0/2338925582 conn(0x7f755806cfc0 msgr2=0x7f755806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:06.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.963+0000 7f755f800640 1 -- 172.21.15.39:0/2338925582 shutdown_connections 2024-09-21T07:43:06.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.963+0000 7f755f800640 1 -- 172.21.15.39:0/2338925582 wait complete. 2024-09-21T07:43:06.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.965+0000 7f755f800640 1 Processor -- start 2024-09-21T07:43:06.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.965+0000 7f755f800640 1 -- start start 2024-09-21T07:43:06.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.966+0000 7f755f800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f75580b3370 0x7f755814a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:06.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.966+0000 7f755f800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7558151590 0x7f75580b2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:06.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.966+0000 7f755f800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f755815e590 0x7f7558157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:06.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.966+0000 7f755f800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f755815b750 con 0x7f7558151590 2024-09-21T07:43:06.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.966+0000 7f755f800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f755815b5d0 con 0x7f755815e590 2024-09-21T07:43:06.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.966+0000 7f755f800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f755815b8d0 con 0x7f75580b3370 2024-09-21T07:43:06.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.967+0000 7f7557e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7558151590 0x7f75580b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:06.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.967+0000 7f7557e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7558151590 0x7f75580b2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36830/0 (socket says 172.21.15.39:36830) 2024-09-21T07:43:06.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.967+0000 7f7557e00640 1 -- 172.21.15.39:0/3100287098 learned_addr learned my addr 172.21.15.39:0/3100287098 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:06.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.967+0000 7f755ce00640 1 --2- 172.21.15.39:0/3100287098 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f75580b3370 0x7f755814a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:06.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.967+0000 7f755d800640 1 --2- 172.21.15.39:0/3100287098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f755815e590 0x7f7558157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:06.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.967+0000 7f7557e00640 1 -- 172.21.15.39:0/3100287098 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f75580b3370 msgr2=0x7f755814a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:06.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.967+0000 7f7557e00640 1 --2- 172.21.15.39:0/3100287098 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f75580b3370 0x7f755814a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:06.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.967+0000 7f7557e00640 1 -- 172.21.15.39:0/3100287098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f755815e590 msgr2=0x7f7558157590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:06.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.967+0000 7f7557e00640 1 --2- 172.21.15.39:0/3100287098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f755815e590 0x7f7558157590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:06.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.967+0000 7f7557e00640 1 -- 172.21.15.39:0/3100287098 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7558156330 con 0x7f7558151590 2024-09-21T07:43:06.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.967+0000 7f7557e00640 1 --2- 172.21.15.39:0/3100287098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7558151590 0x7f75580b2890 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f7544034040 tx=0x7f75440038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:06.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.968+0000 7f7554c00640 1 -- 172.21.15.39:0/3100287098 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754403e030 con 0x7f7558151590 2024-09-21T07:43:06.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.968+0000 7f7554c00640 1 -- 172.21.15.39:0/3100287098 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7544003ca0 con 0x7f7558151590 2024-09-21T07:43:06.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.968+0000 7f7554c00640 1 -- 172.21.15.39:0/3100287098 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7544035040 con 0x7f7558151590 2024-09-21T07:43:06.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.968+0000 7f755f800640 1 -- 172.21.15.39:0/3100287098 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7558156610 con 0x7f7558151590 2024-09-21T07:43:06.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.968+0000 7f755f800640 1 -- 172.21.15.39:0/3100287098 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7558156bd0 con 0x7f7558151590 2024-09-21T07:43:06.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.969+0000 7f755f800640 1 -- 172.21.15.39:0/3100287098 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75580b16a0 con 0x7f7558151590 2024-09-21T07:43:06.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.970+0000 7f7554c00640 1 -- 172.21.15.39:0/3100287098 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7544041030 con 0x7f7558151590 2024-09-21T07:43:06.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.971+0000 7f7554c00640 1 --2- 172.21.15.39:0/3100287098 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7538078b90 0x7f753807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:06.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.971+0000 7f7554c00640 1 -- 172.21.15.39:0/3100287098 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8076+0+0 (secure 0 0 0) 0x7f75440bbd80 con 0x7f7558151590 2024-09-21T07:43:06.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.972+0000 7f755ce00640 1 --2- 172.21.15.39:0/3100287098 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7538078b90 0x7f753807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:06.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.972+0000 7f755ce00640 1 --2- 172.21.15.39:0/3100287098 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7538078b90 0x7f753807b0a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f754c00a6c0 tx=0x7f754c00c040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:06.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:06.976+0000 7f7554c00640 1 -- 172.21.15.39:0/3100287098 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7544083870 con 0x7f7558151590 2024-09-21T07:43:07.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:07.213+0000 7f755f800640 1 -- 172.21.15.39:0/3100287098 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f755815bdf0 con 0x7f7558151590 2024-09-21T07:43:07.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:07.214+0000 7f7554c00640 1 -- 172.21.15.39:0/3100287098 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 143 v143) ==== 95+0+5256 (secure 0 0 0) 0x7f7544083170 con 0x7f7558151590 2024-09-21T07:43:07.216 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:43:07.216 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":143,"btime":"2024-09-21T07:43:06:181997+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7366,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/759792812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":759792812},{"type":"v1","addr":"172.21.15.148:6837","nonce":759792812}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":139}],"filesystems":[{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:43:05.988389+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7644},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7644":{"gid":7644,"name":"a","rank":1,"incarnation":138,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/1326711417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1326711417},{"type":"v1","addr":"172.21.15.39:6835","nonce":1326711417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7644,7311]},"id":6}]} 2024-09-21T07:43:07.216 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 143 2024-09-21T07:43:07.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:07.216+0000 7f755f800640 1 -- 172.21.15.39:0/3100287098 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7538078b90 msgr2=0x7f753807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:07.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:07.216+0000 7f755f800640 1 --2- 172.21.15.39:0/3100287098 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7538078b90 0x7f753807b0a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f754c00a6c0 tx=0x7f754c00c040 comp rx=0 tx=0).stop 2024-09-21T07:43:07.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:07.216+0000 7f755f800640 1 -- 172.21.15.39:0/3100287098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7558151590 msgr2=0x7f75580b2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:07.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:07.216+0000 7f755f800640 1 --2- 172.21.15.39:0/3100287098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7558151590 0x7f75580b2890 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f7544034040 tx=0x7f75440038a0 comp rx=0 tx=0).stop 2024-09-21T07:43:07.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:07.216+0000 7f755f800640 1 -- 172.21.15.39:0/3100287098 shutdown_connections 2024-09-21T07:43:07.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:07.216+0000 7f755f800640 1 --2- 172.21.15.39:0/3100287098 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7538078b90 0x7f753807b0a0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:07.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:07.216+0000 7f755f800640 1 --2- 172.21.15.39:0/3100287098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f755815e590 0x7f7558157590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:07.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:07.216+0000 7f755f800640 1 --2- 172.21.15.39:0/3100287098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7558151590 0x7f75580b2890 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:07.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:07.216+0000 7f755f800640 1 --2- 172.21.15.39:0/3100287098 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f75580b3370 0x7f755814a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:07.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:07.216+0000 7f755f800640 1 -- 172.21.15.39:0/3100287098 >> 172.21.15.39:0/3100287098 conn(0x7f755806cfc0 msgr2=0x7f7558070670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:07.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:07.216+0000 7f755f800640 1 -- 172.21.15.39:0/3100287098 shutdown_connections 2024-09-21T07:43:07.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:07.216+0000 7f755f800640 1 -- 172.21.15.39:0/3100287098 wait complete. 2024-09-21T07:43:07.241 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:43:08.888 INFO:tasks.ceph.mds.a.smithi039.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::handle_request(ceph::cref_t&)' thread 7f77b6800640 time 2024-09-21T07:43:08.887850+0000 2024-09-21T07:43:08.889 INFO:tasks.ceph.mds.a.smithi039.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: 113: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 7) 2024-09-21T07:43:08.889 INFO:tasks.ceph.mds.a.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:43:08.889 INFO:tasks.ceph.mds.a.smithi039.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f77bd85654b] 2024-09-21T07:43:08.890 INFO:tasks.ceph.mds.a.smithi039.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f77bd856758] 2024-09-21T07:43:08.890 INFO:tasks.ceph.mds.a.smithi039.stderr: 3: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0xbb4) [0x5582a16e3db8] 2024-09-21T07:43:08.890 INFO:tasks.ceph.mds.a.smithi039.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xc96) [0x5582a137bdd4] 2024-09-21T07:43:08.890 INFO:tasks.ceph.mds.a.smithi039.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5582a137f523] 2024-09-21T07:43:08.890 INFO:tasks.ceph.mds.a.smithi039.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5582a137fe4a] 2024-09-21T07:43:08.890 INFO:tasks.ceph.mds.a.smithi039.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x5582a1362fa4] 2024-09-21T07:43:08.891 INFO:tasks.ceph.mds.a.smithi039.stderr: 8: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f77bd9a57df] 2024-09-21T07:43:08.891 INFO:tasks.ceph.mds.a.smithi039.stderr: 9: (DispatchQueue::entry()+0x60f) [0x7f77bd9a60d1] 2024-09-21T07:43:08.891 INFO:tasks.ceph.mds.a.smithi039.stderr: 10: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f77bda93f5b] 2024-09-21T07:43:08.891 INFO:tasks.ceph.mds.a.smithi039.stderr: 11: (Thread::entry_wrapper()+0x43) [0x7f77bd82fd57] 2024-09-21T07:43:08.891 INFO:tasks.ceph.mds.a.smithi039.stderr: 12: (Thread::_entry_func(void*)+0xd) [0x7f77bd82fd73] 2024-09-21T07:43:08.891 INFO:tasks.ceph.mds.a.smithi039.stderr: 13: /lib64/libc.so.6(+0x89c52) [0x7f77bce89c52] 2024-09-21T07:43:08.891 INFO:tasks.ceph.mds.a.smithi039.stderr: 14: /lib64/libc.so.6(+0x10ec80) [0x7f77bcf0ec80] 2024-09-21T07:43:08.891 INFO:tasks.ceph.mds.a.smithi039.stderr:*** Caught signal (Aborted) ** 2024-09-21T07:43:08.891 INFO:tasks.ceph.mds.a.smithi039.stderr: in thread 7f77b6800640 thread_name:ms_dispatch 2024-09-21T07:43:08.892 INFO:tasks.ceph.mds.a.smithi039.stderr:2024-09-21T07:43:08.887+0000 7f77b6800640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::handle_request(ceph::cref_t&)' thread 7f77b6800640 time 2024-09-21T07:43:08.887850+0000 2024-09-21T07:43:08.892 INFO:tasks.ceph.mds.a.smithi039.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: 113: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 7) 2024-09-21T07:43:08.892 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:43:08.892 INFO:tasks.ceph.mds.a.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:43:08.892 INFO:tasks.ceph.mds.a.smithi039.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f77bd85654b] 2024-09-21T07:43:08.892 INFO:tasks.ceph.mds.a.smithi039.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f77bd856758] 2024-09-21T07:43:08.892 INFO:tasks.ceph.mds.a.smithi039.stderr: 3: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0xbb4) [0x5582a16e3db8] 2024-09-21T07:43:08.892 INFO:tasks.ceph.mds.a.smithi039.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xc96) [0x5582a137bdd4] 2024-09-21T07:43:08.892 INFO:tasks.ceph.mds.a.smithi039.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5582a137f523] 2024-09-21T07:43:08.893 INFO:tasks.ceph.mds.a.smithi039.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5582a137fe4a] 2024-09-21T07:43:08.893 INFO:tasks.ceph.mds.a.smithi039.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x5582a1362fa4] 2024-09-21T07:43:08.893 INFO:tasks.ceph.mds.a.smithi039.stderr: 8: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f77bd9a57df] 2024-09-21T07:43:08.893 INFO:tasks.ceph.mds.a.smithi039.stderr: 9: (DispatchQueue::entry()+0x60f) [0x7f77bd9a60d1] 2024-09-21T07:43:08.893 INFO:tasks.ceph.mds.a.smithi039.stderr: 10: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f77bda93f5b] 2024-09-21T07:43:08.893 INFO:tasks.ceph.mds.a.smithi039.stderr: 11: (Thread::entry_wrapper()+0x43) [0x7f77bd82fd57] 2024-09-21T07:43:08.893 INFO:tasks.ceph.mds.a.smithi039.stderr: 12: (Thread::_entry_func(void*)+0xd) [0x7f77bd82fd73] 2024-09-21T07:43:08.893 INFO:tasks.ceph.mds.a.smithi039.stderr: 13: /lib64/libc.so.6(+0x89c52) [0x7f77bce89c52] 2024-09-21T07:43:08.894 INFO:tasks.ceph.mds.a.smithi039.stderr: 14: /lib64/libc.so.6(+0x10ec80) [0x7f77bcf0ec80] 2024-09-21T07:43:08.894 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:43:08.894 INFO:tasks.ceph.mds.a.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:43:08.894 INFO:tasks.ceph.mds.a.smithi039.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f77bce3e6f0] 2024-09-21T07:43:08.895 INFO:tasks.ceph.mds.a.smithi039.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f77bce8b99c] 2024-09-21T07:43:08.895 INFO:tasks.ceph.mds.a.smithi039.stderr: 3: raise() 2024-09-21T07:43:08.895 INFO:tasks.ceph.mds.a.smithi039.stderr: 4: abort() 2024-09-21T07:43:08.895 INFO:tasks.ceph.mds.a.smithi039.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7f77bd856686] 2024-09-21T07:43:08.895 INFO:tasks.ceph.mds.a.smithi039.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f77bd856758] 2024-09-21T07:43:08.895 INFO:tasks.ceph.mds.a.smithi039.stderr: 7: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0xbb4) [0x5582a16e3db8] 2024-09-21T07:43:08.895 INFO:tasks.ceph.mds.a.smithi039.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xc96) [0x5582a137bdd4] 2024-09-21T07:43:08.895 INFO:tasks.ceph.mds.a.smithi039.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5582a137f523] 2024-09-21T07:43:08.896 INFO:tasks.ceph.mds.a.smithi039.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5582a137fe4a] 2024-09-21T07:43:08.896 INFO:tasks.ceph.mds.a.smithi039.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x5582a1362fa4] 2024-09-21T07:43:08.896 INFO:tasks.ceph.mds.a.smithi039.stderr: 12: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f77bd9a57df] 2024-09-21T07:43:08.896 INFO:tasks.ceph.mds.a.smithi039.stderr: 13: (DispatchQueue::entry()+0x60f) [0x7f77bd9a60d1] 2024-09-21T07:43:08.896 INFO:tasks.ceph.mds.a.smithi039.stderr: 14: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f77bda93f5b] 2024-09-21T07:43:08.896 INFO:tasks.ceph.mds.a.smithi039.stderr: 15: (Thread::entry_wrapper()+0x43) [0x7f77bd82fd57] 2024-09-21T07:43:08.896 INFO:tasks.ceph.mds.a.smithi039.stderr: 16: (Thread::_entry_func(void*)+0xd) [0x7f77bd82fd73] 2024-09-21T07:43:08.896 INFO:tasks.ceph.mds.a.smithi039.stderr: 17: /lib64/libc.so.6(+0x89c52) [0x7f77bce89c52] 2024-09-21T07:43:08.897 INFO:tasks.ceph.mds.a.smithi039.stderr: 18: /lib64/libc.so.6(+0x10ec80) [0x7f77bcf0ec80] 2024-09-21T07:43:08.897 INFO:tasks.ceph.mds.a.smithi039.stderr:2024-09-21T07:43:08.888+0000 7f77b6800640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:43:08.897 INFO:tasks.ceph.mds.a.smithi039.stderr: in thread 7f77b6800640 thread_name:ms_dispatch 2024-09-21T07:43:08.897 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:43:08.897 INFO:tasks.ceph.mds.a.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:43:08.897 INFO:tasks.ceph.mds.a.smithi039.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f77bce3e6f0] 2024-09-21T07:43:08.897 INFO:tasks.ceph.mds.a.smithi039.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f77bce8b99c] 2024-09-21T07:43:08.897 INFO:tasks.ceph.mds.a.smithi039.stderr: 3: raise() 2024-09-21T07:43:08.897 INFO:tasks.ceph.mds.a.smithi039.stderr: 4: abort() 2024-09-21T07:43:08.898 INFO:tasks.ceph.mds.a.smithi039.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7f77bd856686] 2024-09-21T07:43:08.898 INFO:tasks.ceph.mds.a.smithi039.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f77bd856758] 2024-09-21T07:43:08.898 INFO:tasks.ceph.mds.a.smithi039.stderr: 7: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0xbb4) [0x5582a16e3db8] 2024-09-21T07:43:08.898 INFO:tasks.ceph.mds.a.smithi039.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xc96) [0x5582a137bdd4] 2024-09-21T07:43:08.898 INFO:tasks.ceph.mds.a.smithi039.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5582a137f523] 2024-09-21T07:43:08.898 INFO:tasks.ceph.mds.a.smithi039.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5582a137fe4a] 2024-09-21T07:43:08.898 INFO:tasks.ceph.mds.a.smithi039.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x5582a1362fa4] 2024-09-21T07:43:08.898 INFO:tasks.ceph.mds.a.smithi039.stderr: 12: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f77bd9a57df] 2024-09-21T07:43:08.899 INFO:tasks.ceph.mds.a.smithi039.stderr: 13: (DispatchQueue::entry()+0x60f) [0x7f77bd9a60d1] 2024-09-21T07:43:08.899 INFO:tasks.ceph.mds.a.smithi039.stderr: 14: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f77bda93f5b] 2024-09-21T07:43:08.899 INFO:tasks.ceph.mds.a.smithi039.stderr: 15: (Thread::entry_wrapper()+0x43) [0x7f77bd82fd57] 2024-09-21T07:43:08.899 INFO:tasks.ceph.mds.a.smithi039.stderr: 16: (Thread::_entry_func(void*)+0xd) [0x7f77bd82fd73] 2024-09-21T07:43:08.899 INFO:tasks.ceph.mds.a.smithi039.stderr: 17: /lib64/libc.so.6(+0x89c52) [0x7f77bce89c52] 2024-09-21T07:43:08.899 INFO:tasks.ceph.mds.a.smithi039.stderr: 18: /lib64/libc.so.6(+0x10ec80) [0x7f77bcf0ec80] 2024-09-21T07:43:08.899 INFO:tasks.ceph.mds.a.smithi039.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:43:08.899 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:43:08.903 INFO:tasks.ceph.mds.a.smithi039.stderr: -1> 2024-09-21T07:43:08.887+0000 7f77b6800640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::handle_request(ceph::cref_t&)' thread 7f77b6800640 time 2024-09-21T07:43:08.887850+0000 2024-09-21T07:43:08.904 INFO:tasks.ceph.mds.a.smithi039.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: 113: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 7) 2024-09-21T07:43:08.904 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:43:08.904 INFO:tasks.ceph.mds.a.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:43:08.904 INFO:tasks.ceph.mds.a.smithi039.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f77bd85654b] 2024-09-21T07:43:08.904 INFO:tasks.ceph.mds.a.smithi039.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f77bd856758] 2024-09-21T07:43:08.904 INFO:tasks.ceph.mds.a.smithi039.stderr: 3: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0xbb4) [0x5582a16e3db8] 2024-09-21T07:43:08.904 INFO:tasks.ceph.mds.a.smithi039.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xc96) [0x5582a137bdd4] 2024-09-21T07:43:08.904 INFO:tasks.ceph.mds.a.smithi039.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5582a137f523] 2024-09-21T07:43:08.904 INFO:tasks.ceph.mds.a.smithi039.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5582a137fe4a] 2024-09-21T07:43:08.904 INFO:tasks.ceph.mds.a.smithi039.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x5582a1362fa4] 2024-09-21T07:43:08.904 INFO:tasks.ceph.mds.a.smithi039.stderr: 8: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f77bd9a57df] 2024-09-21T07:43:08.904 INFO:tasks.ceph.mds.a.smithi039.stderr: 9: (DispatchQueue::entry()+0x60f) [0x7f77bd9a60d1] 2024-09-21T07:43:08.905 INFO:tasks.ceph.mds.a.smithi039.stderr: 10: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f77bda93f5b] 2024-09-21T07:43:08.905 INFO:tasks.ceph.mds.a.smithi039.stderr: 11: (Thread::entry_wrapper()+0x43) [0x7f77bd82fd57] 2024-09-21T07:43:08.905 INFO:tasks.ceph.mds.a.smithi039.stderr: 12: (Thread::_entry_func(void*)+0xd) [0x7f77bd82fd73] 2024-09-21T07:43:08.905 INFO:tasks.ceph.mds.a.smithi039.stderr: 13: /lib64/libc.so.6(+0x89c52) [0x7f77bce89c52] 2024-09-21T07:43:08.905 INFO:tasks.ceph.mds.a.smithi039.stderr: 14: /lib64/libc.so.6(+0x10ec80) [0x7f77bcf0ec80] 2024-09-21T07:43:08.905 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:43:08.905 INFO:tasks.ceph.mds.a.smithi039.stderr: 0> 2024-09-21T07:43:08.888+0000 7f77b6800640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:43:08.905 INFO:tasks.ceph.mds.a.smithi039.stderr: in thread 7f77b6800640 thread_name:ms_dispatch 2024-09-21T07:43:08.905 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:43:08.905 INFO:tasks.ceph.mds.a.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:43:08.905 INFO:tasks.ceph.mds.a.smithi039.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f77bce3e6f0] 2024-09-21T07:43:08.905 INFO:tasks.ceph.mds.a.smithi039.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f77bce8b99c] 2024-09-21T07:43:08.905 INFO:tasks.ceph.mds.a.smithi039.stderr: 3: raise() 2024-09-21T07:43:08.906 INFO:tasks.ceph.mds.a.smithi039.stderr: 4: abort() 2024-09-21T07:43:08.906 INFO:tasks.ceph.mds.a.smithi039.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7f77bd856686] 2024-09-21T07:43:08.906 INFO:tasks.ceph.mds.a.smithi039.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f77bd856758] 2024-09-21T07:43:08.906 INFO:tasks.ceph.mds.a.smithi039.stderr: 7: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0xbb4) [0x5582a16e3db8] 2024-09-21T07:43:08.906 INFO:tasks.ceph.mds.a.smithi039.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xc96) [0x5582a137bdd4] 2024-09-21T07:43:08.906 INFO:tasks.ceph.mds.a.smithi039.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5582a137f523] 2024-09-21T07:43:08.906 INFO:tasks.ceph.mds.a.smithi039.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5582a137fe4a] 2024-09-21T07:43:08.906 INFO:tasks.ceph.mds.a.smithi039.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x5582a1362fa4] 2024-09-21T07:43:08.906 INFO:tasks.ceph.mds.a.smithi039.stderr: 12: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f77bd9a57df] 2024-09-21T07:43:08.906 INFO:tasks.ceph.mds.a.smithi039.stderr: 13: (DispatchQueue::entry()+0x60f) [0x7f77bd9a60d1] 2024-09-21T07:43:08.906 INFO:tasks.ceph.mds.a.smithi039.stderr: 14: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f77bda93f5b] 2024-09-21T07:43:08.906 INFO:tasks.ceph.mds.a.smithi039.stderr: 15: (Thread::entry_wrapper()+0x43) [0x7f77bd82fd57] 2024-09-21T07:43:08.906 INFO:tasks.ceph.mds.a.smithi039.stderr: 16: (Thread::_entry_func(void*)+0xd) [0x7f77bd82fd73] 2024-09-21T07:43:08.906 INFO:tasks.ceph.mds.a.smithi039.stderr: 17: /lib64/libc.so.6(+0x89c52) [0x7f77bce89c52] 2024-09-21T07:43:08.907 INFO:tasks.ceph.mds.a.smithi039.stderr: 18: /lib64/libc.so.6(+0x10ec80) [0x7f77bcf0ec80] 2024-09-21T07:43:08.907 INFO:tasks.ceph.mds.a.smithi039.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:43:08.907 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:43:08.912 INFO:tasks.ceph.mds.a.smithi039.stderr: -1> 2024-09-21T07:43:08.887+0000 7f77b6800640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::handle_request(ceph::cref_t&)' thread 7f77b6800640 time 2024-09-21T07:43:08.887850+0000 2024-09-21T07:43:08.912 INFO:tasks.ceph.mds.a.smithi039.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: 113: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 7) 2024-09-21T07:43:08.912 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:43:08.912 INFO:tasks.ceph.mds.a.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:43:08.912 INFO:tasks.ceph.mds.a.smithi039.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f77bd85654b] 2024-09-21T07:43:08.912 INFO:tasks.ceph.mds.a.smithi039.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f77bd856758] 2024-09-21T07:43:08.912 INFO:tasks.ceph.mds.a.smithi039.stderr: 3: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0xbb4) [0x5582a16e3db8] 2024-09-21T07:43:08.912 INFO:tasks.ceph.mds.a.smithi039.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xc96) [0x5582a137bdd4] 2024-09-21T07:43:08.912 INFO:tasks.ceph.mds.a.smithi039.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5582a137f523] 2024-09-21T07:43:08.912 INFO:tasks.ceph.mds.a.smithi039.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5582a137fe4a] 2024-09-21T07:43:08.912 INFO:tasks.ceph.mds.a.smithi039.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x5582a1362fa4] 2024-09-21T07:43:08.912 INFO:tasks.ceph.mds.a.smithi039.stderr: 8: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f77bd9a57df] 2024-09-21T07:43:08.913 INFO:tasks.ceph.mds.a.smithi039.stderr: 9: (DispatchQueue::entry()+0x60f) [0x7f77bd9a60d1] 2024-09-21T07:43:08.913 INFO:tasks.ceph.mds.a.smithi039.stderr: 10: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f77bda93f5b] 2024-09-21T07:43:08.913 INFO:tasks.ceph.mds.a.smithi039.stderr: 11: (Thread::entry_wrapper()+0x43) [0x7f77bd82fd57] 2024-09-21T07:43:08.913 INFO:tasks.ceph.mds.a.smithi039.stderr: 12: (Thread::_entry_func(void*)+0xd) [0x7f77bd82fd73] 2024-09-21T07:43:08.913 INFO:tasks.ceph.mds.a.smithi039.stderr: 13: /lib64/libc.so.6(+0x89c52) [0x7f77bce89c52] 2024-09-21T07:43:08.913 INFO:tasks.ceph.mds.a.smithi039.stderr: 14: /lib64/libc.so.6(+0x10ec80) [0x7f77bcf0ec80] 2024-09-21T07:43:08.913 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:43:08.913 INFO:tasks.ceph.mds.a.smithi039.stderr: 0> 2024-09-21T07:43:08.888+0000 7f77b6800640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:43:08.913 INFO:tasks.ceph.mds.a.smithi039.stderr: in thread 7f77b6800640 thread_name:ms_dispatch 2024-09-21T07:43:08.913 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:43:08.913 INFO:tasks.ceph.mds.a.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:43:08.913 INFO:tasks.ceph.mds.a.smithi039.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f77bce3e6f0] 2024-09-21T07:43:08.913 INFO:tasks.ceph.mds.a.smithi039.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f77bce8b99c] 2024-09-21T07:43:08.914 INFO:tasks.ceph.mds.a.smithi039.stderr: 3: raise() 2024-09-21T07:43:08.914 INFO:tasks.ceph.mds.a.smithi039.stderr: 4: abort() 2024-09-21T07:43:08.914 INFO:tasks.ceph.mds.a.smithi039.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7f77bd856686] 2024-09-21T07:43:08.914 INFO:tasks.ceph.mds.a.smithi039.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f77bd856758] 2024-09-21T07:43:08.914 INFO:tasks.ceph.mds.a.smithi039.stderr: 7: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0xbb4) [0x5582a16e3db8] 2024-09-21T07:43:08.914 INFO:tasks.ceph.mds.a.smithi039.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xc96) [0x5582a137bdd4] 2024-09-21T07:43:08.914 INFO:tasks.ceph.mds.a.smithi039.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5582a137f523] 2024-09-21T07:43:08.914 INFO:tasks.ceph.mds.a.smithi039.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5582a137fe4a] 2024-09-21T07:43:08.914 INFO:tasks.ceph.mds.a.smithi039.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x5582a1362fa4] 2024-09-21T07:43:08.914 INFO:tasks.ceph.mds.a.smithi039.stderr: 12: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f77bd9a57df] 2024-09-21T07:43:08.914 INFO:tasks.ceph.mds.a.smithi039.stderr: 13: (DispatchQueue::entry()+0x60f) [0x7f77bd9a60d1] 2024-09-21T07:43:08.914 INFO:tasks.ceph.mds.a.smithi039.stderr: 14: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f77bda93f5b] 2024-09-21T07:43:08.914 INFO:tasks.ceph.mds.a.smithi039.stderr: 15: (Thread::entry_wrapper()+0x43) [0x7f77bd82fd57] 2024-09-21T07:43:08.915 INFO:tasks.ceph.mds.a.smithi039.stderr: 16: (Thread::_entry_func(void*)+0xd) [0x7f77bd82fd73] 2024-09-21T07:43:08.915 INFO:tasks.ceph.mds.a.smithi039.stderr: 17: /lib64/libc.so.6(+0x89c52) [0x7f77bce89c52] 2024-09-21T07:43:08.915 INFO:tasks.ceph.mds.a.smithi039.stderr: 18: /lib64/libc.so.6(+0x10ec80) [0x7f77bcf0ec80] 2024-09-21T07:43:08.915 INFO:tasks.ceph.mds.a.smithi039.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:43:08.915 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:43:09.658 INFO:tasks.ceph.mds.a.smithi039.stderr:daemon-helper: command crashed with signal 6 2024-09-21T07:43:12.241 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:43:12.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.476+0000 7fd1db000640 1 -- 172.21.15.39:0/692463964 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1d415cd30 msgr2=0x7fd1d40cfb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:12.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.476+0000 7fd1db000640 1 --2- 172.21.15.39:0/692463964 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1d415cd30 0x7fd1d40cfb30 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4153d90 tx=0x7fd1c402a4a0 comp rx=0 tx=0).stop 2024-09-21T07:43:12.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.476+0000 7fd1db000640 1 -- 172.21.15.39:0/692463964 shutdown_connections 2024-09-21T07:43:12.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.476+0000 7fd1db000640 1 --2- 172.21.15.39:0/692463964 >> v2:172.21.15.148:3300/0 conn(0x7fd1d406f800 0x7fd1d40cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:12.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.476+0000 7fd1db000640 1 --2- 172.21.15.39:0/692463964 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1d415cd30 0x7fd1d40cfb30 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:12.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.476+0000 7fd1db000640 1 -- 172.21.15.39:0/692463964 >> 172.21.15.39:0/692463964 conn(0x7fd1d406cfc0 msgr2=0x7fd1d406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:12.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.476+0000 7fd1db000640 1 -- 172.21.15.39:0/692463964 shutdown_connections 2024-09-21T07:43:12.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.476+0000 7fd1db000640 1 -- 172.21.15.39:0/692463964 wait complete. 2024-09-21T07:43:12.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.479+0000 7fd1db000640 1 Processor -- start 2024-09-21T07:43:12.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.479+0000 7fd1db000640 1 -- start start 2024-09-21T07:43:12.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.480+0000 7fd1db000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1d406f800 0x7fd1d415b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:12.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.480+0000 7fd1db000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1d406fc10 0x7fd1d4150d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:12.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.480+0000 7fd1db000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1d415cd30 0x7fd1d40b1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:12.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.480+0000 7fd1db000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d4073890 con 0x7fd1d406f800 2024-09-21T07:43:12.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.480+0000 7fd1db000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d4073710 con 0x7fd1d415cd30 2024-09-21T07:43:12.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.480+0000 7fd1db000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd1d4073a10 con 0x7fd1d406fc10 2024-09-21T07:43:12.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.480+0000 7fd1d3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1d406f800 0x7fd1d415b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:12.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.480+0000 7fd1d3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1d406f800 0x7fd1d415b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54112/0 (socket says 172.21.15.39:54112) 2024-09-21T07:43:12.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.480+0000 7fd1d3e00640 1 -- 172.21.15.39:0/3541240214 learned_addr learned my addr 172.21.15.39:0/3541240214 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:12.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.481+0000 7fd1d9000640 1 --2- 172.21.15.39:0/3541240214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1d415cd30 0x7fd1d40b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:12.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.481+0000 7fd1cb400640 1 --2- 172.21.15.39:0/3541240214 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1d406fc10 0x7fd1d4150d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:12.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.481+0000 7fd1d3e00640 1 -- 172.21.15.39:0/3541240214 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1d406fc10 msgr2=0x7fd1d4150d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:12.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.481+0000 7fd1d3e00640 1 --2- 172.21.15.39:0/3541240214 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1d406fc10 0x7fd1d4150d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:12.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.481+0000 7fd1d3e00640 1 -- 172.21.15.39:0/3541240214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1d415cd30 msgr2=0x7fd1d40b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:12.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.481+0000 7fd1d3e00640 1 --2- 172.21.15.39:0/3541240214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1d415cd30 0x7fd1d40b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:12.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.481+0000 7fd1d3e00640 1 -- 172.21.15.39:0/3541240214 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d40aff90 con 0x7fd1d406f800 2024-09-21T07:43:12.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.481+0000 7fd1cb400640 1 --2- 172.21.15.39:0/3541240214 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1d406fc10 0x7fd1d4150d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:43:12.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.481+0000 7fd1d9000640 1 --2- 172.21.15.39:0/3541240214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1d415cd30 0x7fd1d40b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:43:12.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.481+0000 7fd1d3e00640 1 --2- 172.21.15.39:0/3541240214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1d406f800 0x7fd1d415b8f0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fd1c402aa90 tx=0x7fd1c4033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:12.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.481+0000 7fd1d0c00640 1 -- 172.21.15.39:0/3541240214 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4031d60 con 0x7fd1d406f800 2024-09-21T07:43:12.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.481+0000 7fd1d0c00640 1 -- 172.21.15.39:0/3541240214 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1c4033e10 con 0x7fd1d406f800 2024-09-21T07:43:12.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.481+0000 7fd1d0c00640 1 -- 172.21.15.39:0/3541240214 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4005260 con 0x7fd1d406f800 2024-09-21T07:43:12.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.481+0000 7fd1db000640 1 -- 172.21.15.39:0/3541240214 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1d41f8a00 con 0x7fd1d406f800 2024-09-21T07:43:12.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.482+0000 7fd1db000640 1 -- 172.21.15.39:0/3541240214 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1d41f8fc0 con 0x7fd1d406f800 2024-09-21T07:43:12.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.483+0000 7fd1db000640 1 -- 172.21.15.39:0/3541240214 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1d40b0160 con 0x7fd1d406f800 2024-09-21T07:43:12.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.484+0000 7fd1d0c00640 1 -- 172.21.15.39:0/3541240214 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd1c4005400 con 0x7fd1d406f800 2024-09-21T07:43:12.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.485+0000 7fd1d0c00640 1 --2- 172.21.15.39:0/3541240214 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd1b0078b90 0x7fd1b007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:12.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.485+0000 7fd1d0c00640 1 -- 172.21.15.39:0/3541240214 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8076+0+0 (secure 0 0 0) 0x7fd1c40bc9d0 con 0x7fd1d406f800 2024-09-21T07:43:12.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.486+0000 7fd1cb400640 1 --2- 172.21.15.39:0/3541240214 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd1b0078b90 0x7fd1b007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:12.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.486+0000 7fd1cb400640 1 --2- 172.21.15.39:0/3541240214 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd1b0078b90 0x7fd1b007b0a0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc005fd0 tx=0x7fd1bc00f360 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:12.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.490+0000 7fd1d0c00640 1 -- 172.21.15.39:0/3541240214 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd1c403f030 con 0x7fd1d406f800 2024-09-21T07:43:12.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.724+0000 7fd1db000640 1 -- 172.21.15.39:0/3541240214 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1d4073ec0 con 0x7fd1d406f800 2024-09-21T07:43:12.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.725+0000 7fd1d0c00640 1 -- 172.21.15.39:0/3541240214 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 143 v143) ==== 95+0+5256 (secure 0 0 0) 0x7fd1c4005d00 con 0x7fd1d406f800 2024-09-21T07:43:12.727 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:43:12.727 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":143,"btime":"2024-09-21T07:43:06:181997+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7366,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/759792812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":759792812},{"type":"v1","addr":"172.21.15.148:6837","nonce":759792812}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":139}],"filesystems":[{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:43:05.988389+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7644},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7644":{"gid":7644,"name":"a","rank":1,"incarnation":138,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/1326711417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1326711417},{"type":"v1","addr":"172.21.15.39:6835","nonce":1326711417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7644,7311]},"id":6}]} 2024-09-21T07:43:12.727 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 143 2024-09-21T07:43:12.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.727+0000 7fd1db000640 1 -- 172.21.15.39:0/3541240214 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd1b0078b90 msgr2=0x7fd1b007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:12.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.727+0000 7fd1db000640 1 --2- 172.21.15.39:0/3541240214 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd1b0078b90 0x7fd1b007b0a0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc005fd0 tx=0x7fd1bc00f360 comp rx=0 tx=0).stop 2024-09-21T07:43:12.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.727+0000 7fd1db000640 1 -- 172.21.15.39:0/3541240214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1d406f800 msgr2=0x7fd1d415b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:12.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.727+0000 7fd1db000640 1 --2- 172.21.15.39:0/3541240214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1d406f800 0x7fd1d415b8f0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fd1c402aa90 tx=0x7fd1c4033c20 comp rx=0 tx=0).stop 2024-09-21T07:43:12.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.727+0000 7fd1db000640 1 -- 172.21.15.39:0/3541240214 shutdown_connections 2024-09-21T07:43:12.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.727+0000 7fd1db000640 1 --2- 172.21.15.39:0/3541240214 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd1b0078b90 0x7fd1b007b0a0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:12.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.727+0000 7fd1db000640 1 --2- 172.21.15.39:0/3541240214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1d415cd30 0x7fd1d40b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:12.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.727+0000 7fd1db000640 1 --2- 172.21.15.39:0/3541240214 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1d406fc10 0x7fd1d4150d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:12.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.727+0000 7fd1db000640 1 --2- 172.21.15.39:0/3541240214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1d406f800 0x7fd1d415b8f0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:12.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.727+0000 7fd1db000640 1 -- 172.21.15.39:0/3541240214 >> 172.21.15.39:0/3541240214 conn(0x7fd1d406cfc0 msgr2=0x7fd1d406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:12.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.727+0000 7fd1db000640 1 -- 172.21.15.39:0/3541240214 shutdown_connections 2024-09-21T07:43:12.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:12.727+0000 7fd1db000640 1 -- 172.21.15.39:0/3541240214 wait complete. 2024-09-21T07:43:12.753 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:43:16.187 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-09-21T07:43:17.754 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:43:17.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:17.989+0000 7f2110a00640 1 -- 172.21.15.39:0/3918423906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20f0005350 msgr2=0x7f20f000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:17.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:17.989+0000 7f2110a00640 1 --2- 172.21.15.39:0/3918423906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20f0005350 0x7f20f000b9c0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f210c0ce1e0 tx=0x7f20f0005830 comp rx=0 tx=0).stop 2024-09-21T07:43:17.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:17.989+0000 7f2110a00640 1 -- 172.21.15.39:0/3918423906 shutdown_connections 2024-09-21T07:43:17.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:17.989+0000 7f2110a00640 1 --2- 172.21.15.39:0/3918423906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20f0005350 0x7f20f000b9c0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:17.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:17.989+0000 7f2110a00640 1 -- 172.21.15.39:0/3918423906 >> 172.21.15.39:0/3918423906 conn(0x7f210c06cfc0 msgr2=0x7f210c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:17.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:17.989+0000 7f2110a00640 1 -- 172.21.15.39:0/3918423906 shutdown_connections 2024-09-21T07:43:17.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:17.989+0000 7f2110a00640 1 -- 172.21.15.39:0/3918423906 wait complete. 2024-09-21T07:43:17.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:17.992+0000 7f2110a00640 1 Processor -- start 2024-09-21T07:43:17.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:17.992+0000 7f2110a00640 1 -- start start 2024-09-21T07:43:17.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:17.993+0000 7f2110a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20f0005350 0x7f210c0aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:17.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:17.993+0000 7f2110a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f210c135c00 0x7f210c15ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:17.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:17.993+0000 7f2110a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f210c15b080 0x7f210c0c52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:17.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:17.993+0000 7f2110a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f210c0d0a50 con 0x7f210c135c00 2024-09-21T07:43:17.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:17.993+0000 7f2110a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f210c0d08d0 con 0x7f210c15b080 2024-09-21T07:43:17.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:17.993+0000 7f2110a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f210c0cd780 con 0x7f20f0005350 2024-09-21T07:43:17.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:17.994+0000 7f2103e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f210c135c00 0x7f210c15ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:17.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:17.994+0000 7f2103e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f210c135c00 0x7f210c15ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54126/0 (socket says 172.21.15.39:54126) 2024-09-21T07:43:17.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:17.994+0000 7f2103e00640 1 -- 172.21.15.39:0/1981239908 learned_addr learned my addr 172.21.15.39:0/1981239908 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:17.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:17.994+0000 7f2109e00640 1 --2- 172.21.15.39:0/1981239908 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20f0005350 0x7f210c0aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:17.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:17.994+0000 7f210a800640 1 --2- 172.21.15.39:0/1981239908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f210c15b080 0x7f210c0c52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:17.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:17.994+0000 7f2103e00640 1 -- 172.21.15.39:0/1981239908 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20f0005350 msgr2=0x7f210c0aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:17.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:17.994+0000 7f2103e00640 1 --2- 172.21.15.39:0/1981239908 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20f0005350 0x7f210c0aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:17.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:17.994+0000 7f2103e00640 1 -- 172.21.15.39:0/1981239908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f210c15b080 msgr2=0x7f210c0c52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:17.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:17.994+0000 7f2103e00640 1 --2- 172.21.15.39:0/1981239908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f210c15b080 0x7f210c0c52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:17.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:17.994+0000 7f2103e00640 1 -- 172.21.15.39:0/1981239908 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f210c0af2d0 con 0x7f210c135c00 2024-09-21T07:43:17.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:17.994+0000 7f210a800640 1 --2- 172.21.15.39:0/1981239908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f210c15b080 0x7f210c0c52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:43:17.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:17.994+0000 7f2103e00640 1 --2- 172.21.15.39:0/1981239908 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f210c135c00 0x7f210c15ab40 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f20f4005f60 tx=0x7f20f4011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:17.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:17.995+0000 7f2102000640 1 -- 172.21.15.39:0/1981239908 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f401f070 con 0x7f210c135c00 2024-09-21T07:43:17.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:17.995+0000 7f2102000640 1 -- 172.21.15.39:0/1981239908 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20f400fde0 con 0x7f210c135c00 2024-09-21T07:43:17.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:17.995+0000 7f2102000640 1 -- 172.21.15.39:0/1981239908 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f40126b0 con 0x7f210c135c00 2024-09-21T07:43:17.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:17.995+0000 7f2110a00640 1 -- 172.21.15.39:0/1981239908 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f210c1f8c60 con 0x7f210c135c00 2024-09-21T07:43:17.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:17.995+0000 7f2110a00640 1 -- 172.21.15.39:0/1981239908 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f210c1384e0 con 0x7f210c135c00 2024-09-21T07:43:17.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:17.996+0000 7f2110a00640 1 -- 172.21.15.39:0/1981239908 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f210c137f70 con 0x7f210c135c00 2024-09-21T07:43:17.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:17.997+0000 7f2102000640 1 -- 172.21.15.39:0/1981239908 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f20f4022070 con 0x7f210c135c00 2024-09-21T07:43:18.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:17.998+0000 7f2102000640 1 --2- 172.21.15.39:0/1981239908 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f20e8078b90 0x7f20e807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:18.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:17.998+0000 7f2102000640 1 -- 172.21.15.39:0/1981239908 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8076+0+0 (secure 0 0 0) 0x7f20f409c0a0 con 0x7f210c135c00 2024-09-21T07:43:18.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:17.999+0000 7f2109e00640 1 --2- 172.21.15.39:0/1981239908 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f20e8078b90 0x7f20e807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:18.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:17.999+0000 7f2109e00640 1 --2- 172.21.15.39:0/1981239908 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f20e8078b90 0x7f20e807b0a0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f210c0cec40 tx=0x7f20fc006b70 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:18.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:18.003+0000 7f2102000640 1 -- 172.21.15.39:0/1981239908 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f20f40634b0 con 0x7f210c135c00 2024-09-21T07:43:18.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:18.236+0000 7f2110a00640 1 -- 172.21.15.39:0/1981239908 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f210c0cdce0 con 0x7f210c135c00 2024-09-21T07:43:18.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:18.237+0000 7f2102000640 1 -- 172.21.15.39:0/1981239908 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 143 v143) ==== 95+0+5256 (secure 0 0 0) 0x7f20f4067ee0 con 0x7f210c135c00 2024-09-21T07:43:18.239 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:43:18.239 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":143,"btime":"2024-09-21T07:43:06:181997+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7366,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/759792812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":759792812},{"type":"v1","addr":"172.21.15.148:6837","nonce":759792812}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":139}],"filesystems":[{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:43:05.988389+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7644},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7644":{"gid":7644,"name":"a","rank":1,"incarnation":138,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/1326711417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1326711417},{"type":"v1","addr":"172.21.15.39:6835","nonce":1326711417}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7644,7311]},"id":6}]} 2024-09-21T07:43:18.239 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 143 2024-09-21T07:43:18.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:18.239+0000 7f2110a00640 1 -- 172.21.15.39:0/1981239908 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f20e8078b90 msgr2=0x7f20e807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:18.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:18.239+0000 7f2110a00640 1 --2- 172.21.15.39:0/1981239908 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f20e8078b90 0x7f20e807b0a0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f210c0cec40 tx=0x7f20fc006b70 comp rx=0 tx=0).stop 2024-09-21T07:43:18.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:18.239+0000 7f2110a00640 1 -- 172.21.15.39:0/1981239908 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f210c135c00 msgr2=0x7f210c15ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:18.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:18.239+0000 7f2110a00640 1 --2- 172.21.15.39:0/1981239908 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f210c135c00 0x7f210c15ab40 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f20f4005f60 tx=0x7f20f4011cb0 comp rx=0 tx=0).stop 2024-09-21T07:43:18.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:18.239+0000 7f2110a00640 1 -- 172.21.15.39:0/1981239908 shutdown_connections 2024-09-21T07:43:18.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:18.239+0000 7f2110a00640 1 --2- 172.21.15.39:0/1981239908 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f20e8078b90 0x7f20e807b0a0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:18.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:18.239+0000 7f2110a00640 1 --2- 172.21.15.39:0/1981239908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f210c15b080 0x7f210c0c52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:18.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:18.239+0000 7f2110a00640 1 --2- 172.21.15.39:0/1981239908 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f210c135c00 0x7f210c15ab40 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:18.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:18.239+0000 7f2110a00640 1 --2- 172.21.15.39:0/1981239908 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20f0005350 0x7f210c0aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:18.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:18.239+0000 7f2110a00640 1 -- 172.21.15.39:0/1981239908 >> 172.21.15.39:0/1981239908 conn(0x7f210c06cfc0 msgr2=0x7f210c0b0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:18.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:18.239+0000 7f2110a00640 1 -- 172.21.15.39:0/1981239908 shutdown_connections 2024-09-21T07:43:18.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:18.239+0000 7f2110a00640 1 -- 172.21.15.39:0/1981239908 wait complete. 2024-09-21T07:43:18.267 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:43:22.800 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~7s 2024-09-21T07:43:23.268 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:43:23.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.504+0000 7f6eade00640 1 -- 172.21.15.39:0/3672157265 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ea80d19f0 msgr2=0x7f6ea80d4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:23.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.504+0000 7f6eade00640 1 --2- 172.21.15.39:0/3672157265 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ea80d19f0 0x7f6ea80d4600 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f6e98009820 tx=0x7f6e9802a5a0 comp rx=0 tx=0).stop 2024-09-21T07:43:23.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.505+0000 7f6eade00640 1 -- 172.21.15.39:0/3672157265 shutdown_connections 2024-09-21T07:43:23.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.505+0000 7f6eade00640 1 --2- 172.21.15.39:0/3672157265 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ea80d19f0 0x7f6ea80d4600 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:23.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.505+0000 7f6eade00640 1 -- 172.21.15.39:0/3672157265 >> 172.21.15.39:0/3672157265 conn(0x7f6ea808bfb0 msgr2=0x7f6ea809d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:23.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.505+0000 7f6eade00640 1 -- 172.21.15.39:0/3672157265 shutdown_connections 2024-09-21T07:43:23.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.505+0000 7f6eade00640 1 -- 172.21.15.39:0/3672157265 wait complete. 2024-09-21T07:43:23.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.507+0000 7f6eade00640 1 Processor -- start 2024-09-21T07:43:23.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.507+0000 7f6eade00640 1 -- start start 2024-09-21T07:43:23.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.508+0000 7f6eade00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6ea80b5890 0x7f6ea815ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:23.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.508+0000 7f6eade00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ea80d19f0 0x7f6ea809e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:23.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.508+0000 7f6eade00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ea80d4b40 0x7f6ea806f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:23.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.508+0000 7f6eade00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea80b5e20 con 0x7f6ea80d19f0 2024-09-21T07:43:23.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.509+0000 7f6eade00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea80b5ca0 con 0x7f6ea80d4b40 2024-09-21T07:43:23.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.509+0000 7f6eade00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f6ea80b2ab0 con 0x7f6ea80b5890 2024-09-21T07:43:23.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.509+0000 7f6e9fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ea80d19f0 0x7f6ea809e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:23.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.509+0000 7f6e9fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ea80d19f0 0x7f6ea809e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55464/0 (socket says 172.21.15.39:55464) 2024-09-21T07:43:23.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.509+0000 7f6e9fe00640 1 -- 172.21.15.39:0/3342853352 learned_addr learned my addr 172.21.15.39:0/3342853352 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:23.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.509+0000 7f6ead400640 1 --2- 172.21.15.39:0/3342853352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ea80d4b40 0x7f6ea806f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:23.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.509+0000 7f6eaca00640 1 --2- 172.21.15.39:0/3342853352 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6ea80b5890 0x7f6ea815ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:23.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.509+0000 7f6e9fe00640 1 -- 172.21.15.39:0/3342853352 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6ea80b5890 msgr2=0x7f6ea815ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:23.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.509+0000 7f6e9fe00640 1 --2- 172.21.15.39:0/3342853352 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6ea80b5890 0x7f6ea815ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:23.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.509+0000 7f6e9fe00640 1 -- 172.21.15.39:0/3342853352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ea80d4b40 msgr2=0x7f6ea806f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:23.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.509+0000 7f6e9fe00640 1 --2- 172.21.15.39:0/3342853352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ea80d4b40 0x7f6ea806f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:23.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.509+0000 7f6e9fe00640 1 -- 172.21.15.39:0/3342853352 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea81e74d0 con 0x7f6ea80d19f0 2024-09-21T07:43:23.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.510+0000 7f6e9fe00640 1 --2- 172.21.15.39:0/3342853352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ea80d19f0 0x7f6ea809e5f0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f6e9000c760 tx=0x7f6e9000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:23.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.510+0000 7f6ea5600640 1 -- 172.21.15.39:0/3342853352 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e90016020 con 0x7f6ea80d19f0 2024-09-21T07:43:23.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.510+0000 7f6ea5600640 1 -- 172.21.15.39:0/3342853352 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e9000a950 con 0x7f6ea80d19f0 2024-09-21T07:43:23.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.510+0000 7f6eade00640 1 -- 172.21.15.39:0/3342853352 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ea81e7740 con 0x7f6ea80d19f0 2024-09-21T07:43:23.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.510+0000 7f6ea5600640 1 -- 172.21.15.39:0/3342853352 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9000ada0 con 0x7f6ea80d19f0 2024-09-21T07:43:23.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.510+0000 7f6eade00640 1 -- 172.21.15.39:0/3342853352 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ea81e7d00 con 0x7f6ea80d19f0 2024-09-21T07:43:23.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.511+0000 7f6eade00640 1 -- 172.21.15.39:0/3342853352 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ea8069a50 con 0x7f6ea80d19f0 2024-09-21T07:43:23.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.513+0000 7f6ea5600640 1 -- 172.21.15.39:0/3342853352 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6e90019020 con 0x7f6ea80d19f0 2024-09-21T07:43:23.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.514+0000 7f6ea5600640 1 --2- 172.21.15.39:0/3342853352 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6e84078b90 0x7f6e8407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:23.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.514+0000 7f6ea5600640 1 -- 172.21.15.39:0/3342853352 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8076+0+0 (secure 0 0 0) 0x7f6e9009b540 con 0x7f6ea80d19f0 2024-09-21T07:43:23.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.514+0000 7f6eaca00640 1 --2- 172.21.15.39:0/3342853352 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6e84078b90 0x7f6e8407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:23.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.515+0000 7f6eaca00640 1 --2- 172.21.15.39:0/3342853352 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6e84078b90 0x7f6e8407b0a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6e98034000 tx=0x7f6e98004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:23.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.518+0000 7f6ea5600640 1 -- 172.21.15.39:0/3342853352 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6e90020020 con 0x7f6ea80d19f0 2024-09-21T07:43:23.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.755+0000 7f6eade00640 1 -- 172.21.15.39:0/3342853352 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ea80d4600 con 0x7f6ea80d19f0 2024-09-21T07:43:23.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.755+0000 7f6ea5600640 1 -- 172.21.15.39:0/3342853352 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 144 v144) ==== 95+0+5304 (secure 0 0 0) 0x7f6ea80d4f50 con 0x7f6ea80d19f0 2024-09-21T07:43:23.758 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:43:23.758 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":144,"btime":"2024-09-21T07:43:21:456271+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7366,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/759792812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":759792812},{"type":"v1","addr":"172.21.15.148:6837","nonce":759792812}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":139}],"filesystems":[{"mdsmap":{"epoch":144,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:43:21.456230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7644},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7644":{"gid":7644,"name":"a","rank":1,"incarnation":138,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/1326711417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1326711417},{"type":"v1","addr":"172.21.15.39:6835","nonce":1326711417}]},"join_fscid":-1,"laggy_since":"2024-09-21T07:43:21.456230+0000","export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311,7644]},"id":6}]} 2024-09-21T07:43:23.759 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 144 2024-09-21T07:43:23.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.758+0000 7f6eade00640 1 -- 172.21.15.39:0/3342853352 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6e84078b90 msgr2=0x7f6e8407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:23.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.758+0000 7f6eade00640 1 --2- 172.21.15.39:0/3342853352 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6e84078b90 0x7f6e8407b0a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6e98034000 tx=0x7f6e98004730 comp rx=0 tx=0).stop 2024-09-21T07:43:23.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.758+0000 7f6eade00640 1 -- 172.21.15.39:0/3342853352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ea80d19f0 msgr2=0x7f6ea809e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:23.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.758+0000 7f6eade00640 1 --2- 172.21.15.39:0/3342853352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ea80d19f0 0x7f6ea809e5f0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f6e9000c760 tx=0x7f6e9000cc20 comp rx=0 tx=0).stop 2024-09-21T07:43:23.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.758+0000 7f6eade00640 1 -- 172.21.15.39:0/3342853352 shutdown_connections 2024-09-21T07:43:23.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.758+0000 7f6eade00640 1 --2- 172.21.15.39:0/3342853352 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6e84078b90 0x7f6e8407b0a0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:23.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.758+0000 7f6eade00640 1 --2- 172.21.15.39:0/3342853352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ea80d4b40 0x7f6ea806f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:23.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.758+0000 7f6eade00640 1 --2- 172.21.15.39:0/3342853352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ea80d19f0 0x7f6ea809e5f0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:23.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.758+0000 7f6eade00640 1 --2- 172.21.15.39:0/3342853352 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6ea80b5890 0x7f6ea815ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:23.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.758+0000 7f6eade00640 1 -- 172.21.15.39:0/3342853352 >> 172.21.15.39:0/3342853352 conn(0x7f6ea808bfb0 msgr2=0x7f6ea81eb1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:23.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.758+0000 7f6eade00640 1 -- 172.21.15.39:0/3342853352 shutdown_connections 2024-09-21T07:43:23.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:23.758+0000 7f6eade00640 1 -- 172.21.15.39:0/3342853352 wait complete. 2024-09-21T07:43:23.785 DEBUG:tasks.ceph_test_case:wait_until_true: success in 15s and 0 retries 2024-09-21T07:43:23.785 DEBUG:teuthology.orchestra.run.smithi039:> sudo sysctl -n kernel.core_pattern 2024-09-21T07:43:23.838 INFO:teuthology.orchestra.run.smithi039.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T07:43:23.840 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2024-09-21T07:43:23.840 DEBUG:teuthology.orchestra.run.smithi039:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2024-09-21T07:43:23.902 INFO:teuthology.orchestra.run.smithi039.stdout:1726904588.66964.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i a', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2024-09-21T07:43:23.905 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1726904588.66964.core'] 2024-09-21T07:43:23.906 INFO:tasks.cephfs.cephfs_test_case:Found core file 1726904588.66964.core, deleting it 2024-09-21T07:43:23.906 DEBUG:teuthology.orchestra.run.smithi039:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1726904588.66964.core 2024-09-21T07:43:23.998 INFO:tasks.cephfs.mount:Suspending the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:43:23.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:43:23.999 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 down 2024-09-21T07:43:24.062 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:43:24.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.287+0000 7fafdda00640 1 -- 172.21.15.39:0/682922570 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafd80cbd70 msgr2=0x7fafd8072fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:24.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.287+0000 7fafdda00640 1 --2- 172.21.15.39:0/682922570 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafd80cbd70 0x7fafd8072fa0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fafd0005fd0 tx=0x7fafd002aa90 comp rx=0 tx=0).stop 2024-09-21T07:43:24.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.287+0000 7fafdda00640 1 -- 172.21.15.39:0/682922570 shutdown_connections 2024-09-21T07:43:24.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.287+0000 7fafdda00640 1 --2- 172.21.15.39:0/682922570 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafd80cbd70 0x7fafd8072fa0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:24.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.287+0000 7fafdda00640 1 --2- 172.21.15.39:0/682922570 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fafd80d0f20 0x7fafd815c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:24.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.287+0000 7fafdda00640 1 -- 172.21.15.39:0/682922570 >> 172.21.15.39:0/682922570 conn(0x7fafd80b2510 msgr2=0x7fafd80d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:24.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.287+0000 7fafdda00640 1 -- 172.21.15.39:0/682922570 shutdown_connections 2024-09-21T07:43:24.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.287+0000 7fafdda00640 1 -- 172.21.15.39:0/682922570 wait complete. 2024-09-21T07:43:24.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.289+0000 7fafdda00640 1 Processor -- start 2024-09-21T07:43:24.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.289+0000 7fafdda00640 1 -- start start 2024-09-21T07:43:24.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.291+0000 7fafdda00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fafd80cb960 0x7fafd8159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:24.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.291+0000 7fafdda00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafd80cbd70 0x7fafd81595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:24.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.291+0000 7fafdda00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafd80d0f20 0x7fafd81f6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:24.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.291+0000 7fafdda00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fafd80b4570 con 0x7fafd80d0f20 2024-09-21T07:43:24.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.291+0000 7fafdda00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fafd80b43f0 con 0x7fafd80cbd70 2024-09-21T07:43:24.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.291+0000 7fafdda00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fafd80b46f0 con 0x7fafd80cb960 2024-09-21T07:43:24.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.291+0000 7fafdd000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafd80d0f20 0x7fafd81f6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:24.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.291+0000 7fafdd000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafd80d0f20 0x7fafd81f6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55490/0 (socket says 172.21.15.39:55490) 2024-09-21T07:43:24.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.291+0000 7fafdd000640 1 -- 172.21.15.39:0/2396100635 learned_addr learned my addr 172.21.15.39:0/2396100635 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:24.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.291+0000 7fafd7e00640 1 --2- 172.21.15.39:0/2396100635 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fafd80cb960 0x7fafd8159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:24.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.291+0000 7fafcf400640 1 --2- 172.21.15.39:0/2396100635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafd80cbd70 0x7fafd81595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:24.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.292+0000 7fafdd000640 1 -- 172.21.15.39:0/2396100635 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fafd80cb960 msgr2=0x7fafd8159090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:24.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.292+0000 7fafdd000640 1 --2- 172.21.15.39:0/2396100635 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fafd80cb960 0x7fafd8159090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:24.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.292+0000 7fafdd000640 1 -- 172.21.15.39:0/2396100635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafd80cbd70 msgr2=0x7fafd81595d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:24.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.292+0000 7fafdd000640 1 --2- 172.21.15.39:0/2396100635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafd80cbd70 0x7fafd81595d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:24.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.292+0000 7fafdd000640 1 -- 172.21.15.39:0/2396100635 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd8099b30 con 0x7fafd80d0f20 2024-09-21T07:43:24.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.292+0000 7fafdd000640 1 --2- 172.21.15.39:0/2396100635 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafd80d0f20 0x7fafd81f6dd0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fafd002b080 tx=0x7fafd0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:24.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.292+0000 7fafd4c00640 1 -- 172.21.15.39:0/2396100635 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd002e7f0 con 0x7fafd80d0f20 2024-09-21T07:43:24.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.292+0000 7fafd4c00640 1 -- 172.21.15.39:0/2396100635 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fafd0004250 con 0x7fafd80d0f20 2024-09-21T07:43:24.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.292+0000 7fafdda00640 1 -- 172.21.15.39:0/2396100635 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafd81f45d0 con 0x7fafd80d0f20 2024-09-21T07:43:24.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.292+0000 7fafd4c00640 1 -- 172.21.15.39:0/2396100635 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd0004660 con 0x7fafd80d0f20 2024-09-21T07:43:24.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.292+0000 7fafdda00640 1 -- 172.21.15.39:0/2396100635 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafd8159b10 con 0x7fafd80d0f20 2024-09-21T07:43:24.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.293+0000 7fafdda00640 1 -- 172.21.15.39:0/2396100635 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafd81f47a0 con 0x7fafd80d0f20 2024-09-21T07:43:24.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.295+0000 7fafd4c00640 1 -- 172.21.15.39:0/2396100635 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fafd0049440 con 0x7fafd80d0f20 2024-09-21T07:43:24.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.296+0000 7fafd4c00640 1 --2- 172.21.15.39:0/2396100635 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fafb4078bc0 0x7fafb407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:24.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.296+0000 7fafd4c00640 1 -- 172.21.15.39:0/2396100635 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8076+0+0 (secure 0 0 0) 0x7fafd00bc970 con 0x7fafd80d0f20 2024-09-21T07:43:24.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.296+0000 7fafd7e00640 1 --2- 172.21.15.39:0/2396100635 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fafb4078bc0 0x7fafb407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:24.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.297+0000 7fafd7e00640 1 --2- 172.21.15.39:0/2396100635 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fafb4078bc0 0x7fafb407b0d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fafc800bb50 tx=0x7fafc800b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:24.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.300+0000 7fafd4c00640 1 -- 172.21.15.39:0/2396100635 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fafd81f47a0 con 0x7fafd80d0f20 2024-09-21T07:43:24.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.536+0000 7fafdda00640 1 -- 172.21.15.39:0/2396100635 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fafd80b4c10 con 0x7fafd80d0f20 2024-09-21T07:43:24.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.537+0000 7fafd4c00640 1 -- 172.21.15.39:0/2396100635 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 144 v144) ==== 95+0+5304 (secure 0 0 0) 0x7fafd0032030 con 0x7fafd80d0f20 2024-09-21T07:43:24.539 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:43:24.539 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":144,"btime":"2024-09-21T07:43:21:456271+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7366,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/759792812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":759792812},{"type":"v1","addr":"172.21.15.148:6837","nonce":759792812}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":139}],"filesystems":[{"mdsmap":{"epoch":144,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:43:21.456230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7644},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7644":{"gid":7644,"name":"a","rank":1,"incarnation":138,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/1326711417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1326711417},{"type":"v1","addr":"172.21.15.39:6835","nonce":1326711417}]},"join_fscid":-1,"laggy_since":"2024-09-21T07:43:21.456230+0000","export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311,7644]},"id":6}]} 2024-09-21T07:43:24.539 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 144 2024-09-21T07:43:24.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.539+0000 7fafdda00640 1 -- 172.21.15.39:0/2396100635 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fafb4078bc0 msgr2=0x7fafb407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:24.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.539+0000 7fafdda00640 1 --2- 172.21.15.39:0/2396100635 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fafb4078bc0 0x7fafb407b0d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fafc800bb50 tx=0x7fafc800b2e0 comp rx=0 tx=0).stop 2024-09-21T07:43:24.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.539+0000 7fafdda00640 1 -- 172.21.15.39:0/2396100635 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafd80d0f20 msgr2=0x7fafd81f6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:24.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.539+0000 7fafdda00640 1 --2- 172.21.15.39:0/2396100635 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafd80d0f20 0x7fafd81f6dd0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fafd002b080 tx=0x7fafd0003f10 comp rx=0 tx=0).stop 2024-09-21T07:43:24.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.539+0000 7fafdda00640 1 -- 172.21.15.39:0/2396100635 shutdown_connections 2024-09-21T07:43:24.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.539+0000 7fafdda00640 1 --2- 172.21.15.39:0/2396100635 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fafb4078bc0 0x7fafb407b0d0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:24.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.539+0000 7fafdda00640 1 --2- 172.21.15.39:0/2396100635 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafd80d0f20 0x7fafd81f6dd0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:24.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.539+0000 7fafdda00640 1 --2- 172.21.15.39:0/2396100635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafd80cbd70 0x7fafd81595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:24.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.539+0000 7fafdda00640 1 --2- 172.21.15.39:0/2396100635 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fafd80cb960 0x7fafd8159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:24.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.539+0000 7fafdda00640 1 -- 172.21.15.39:0/2396100635 >> 172.21.15.39:0/2396100635 conn(0x7fafd80b2510 msgr2=0x7fafd80b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:24.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.539+0000 7fafdda00640 1 -- 172.21.15.39:0/2396100635 shutdown_connections 2024-09-21T07:43:24.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:24.539+0000 7fafdda00640 1 -- 172.21.15.39:0/2396100635 wait complete. 2024-09-21T07:43:24.564 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2024-09-21T07:43:24.813 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_snap":17,"last_created":17,"last_destroyed":14,"pending_noop":[],"snaps":[{"snapid":16,"ino":1099511627777,"stamp":"2024-09-21T07:42:21.254582+0000","name":"s34","metadata":{}},{"snapid":17,"ino":1099511627777,"stamp":"2024-09-21T07:43:06.501843+0000","name":"s37","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2024-09-21T07:43:24.813 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 17, "last_destroyed": 14, "last_snap": 17, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s34", "snapid": 16, "stamp": "2024-09-21T07:42:21.254582+0000" }, { "ino": 1099511627777, "metadata": {}, "name": "s37", "snapid": 17, "stamp": "2024-09-21T07:43:06.501843+0000" } ] } 2024-09-21T07:43:24.813 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:43:25.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.038+0000 7fb9c5200640 1 -- 172.21.15.39:0/2332923914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9c006e330 msgr2=0x7fb9c00cd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:25.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.038+0000 7fb9c5200640 1 --2- 172.21.15.39:0/2332923914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9c006e330 0x7fb9c00cd940 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8005fd0 tx=0x7fb9b802a820 comp rx=0 tx=0).stop 2024-09-21T07:43:25.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.038+0000 7fb9c5200640 1 -- 172.21.15.39:0/2332923914 shutdown_connections 2024-09-21T07:43:25.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.038+0000 7fb9c5200640 1 --2- 172.21.15.39:0/2332923914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9c006e330 0x7fb9c00cd940 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:25.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.038+0000 7fb9c5200640 1 --2- 172.21.15.39:0/2332923914 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb9c0071540 0x7fb9c006ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:25.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.038+0000 7fb9c5200640 1 -- 172.21.15.39:0/2332923914 >> 172.21.15.39:0/2332923914 conn(0x7fb9c0073980 msgr2=0x7fb9c0073d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:25.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.038+0000 7fb9c5200640 1 -- 172.21.15.39:0/2332923914 shutdown_connections 2024-09-21T07:43:25.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.038+0000 7fb9c5200640 1 -- 172.21.15.39:0/2332923914 wait complete. 2024-09-21T07:43:25.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.040+0000 7fb9c5200640 1 Processor -- start 2024-09-21T07:43:25.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.041+0000 7fb9c5200640 1 -- start start 2024-09-21T07:43:25.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.042+0000 7fb9c5200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9c006e330 0x7fb9c00ae410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:25.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.042+0000 7fb9c5200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9c0071540 0x7fb9c01e4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:25.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.042+0000 7fb9c5200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb9c0075dc0 0x7fb9c0146a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:25.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.042+0000 7fb9c5200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c00ce210 con 0x7fb9c006e330 2024-09-21T07:43:25.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.042+0000 7fb9c5200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c00ce090 con 0x7fb9c0071540 2024-09-21T07:43:25.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.042+0000 7fb9c5200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb9c00ce390 con 0x7fb9c0075dc0 2024-09-21T07:43:25.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.042+0000 7fb9bde00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9c006e330 0x7fb9c00ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:25.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.042+0000 7fb9bde00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9c006e330 0x7fb9c00ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55500/0 (socket says 172.21.15.39:55500) 2024-09-21T07:43:25.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.042+0000 7fb9bde00640 1 -- 172.21.15.39:0/822077034 learned_addr learned my addr 172.21.15.39:0/822077034 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:25.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.043+0000 7fb9b7e00640 1 --2- 172.21.15.39:0/822077034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9c0071540 0x7fb9c01e4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:25.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.043+0000 7fb9be800640 1 --2- 172.21.15.39:0/822077034 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb9c0075dc0 0x7fb9c0146a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:25.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.043+0000 7fb9bde00640 1 -- 172.21.15.39:0/822077034 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb9c0075dc0 msgr2=0x7fb9c0146a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:25.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.043+0000 7fb9bde00640 1 --2- 172.21.15.39:0/822077034 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb9c0075dc0 0x7fb9c0146a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:25.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.043+0000 7fb9bde00640 1 -- 172.21.15.39:0/822077034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9c0071540 msgr2=0x7fb9c01e4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:25.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.043+0000 7fb9bde00640 1 --2- 172.21.15.39:0/822077034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9c0071540 0x7fb9c01e4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:25.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.043+0000 7fb9bde00640 1 -- 172.21.15.39:0/822077034 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c0149690 con 0x7fb9c006e330 2024-09-21T07:43:25.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.043+0000 7fb9be800640 1 --2- 172.21.15.39:0/822077034 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb9c0075dc0 0x7fb9c0146a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:43:25.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.043+0000 7fb9bde00640 1 --2- 172.21.15.39:0/822077034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9c006e330 0x7fb9c00ae410 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fb9b000c3c0 tx=0x7fb9b000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:25.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.043+0000 7fb9b6000640 1 -- 172.21.15.39:0/822077034 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0017070 con 0x7fb9c006e330 2024-09-21T07:43:25.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.044+0000 7fb9b6000640 1 -- 172.21.15.39:0/822077034 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9b00035f0 con 0x7fb9c006e330 2024-09-21T07:43:25.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.044+0000 7fb9c5200640 1 -- 172.21.15.39:0/822077034 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9c0149970 con 0x7fb9c006e330 2024-09-21T07:43:25.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.044+0000 7fb9b6000640 1 -- 172.21.15.39:0/822077034 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0003a80 con 0x7fb9c006e330 2024-09-21T07:43:25.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.044+0000 7fb9c5200640 1 -- 172.21.15.39:0/822077034 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9c00adc10 con 0x7fb9c006e330 2024-09-21T07:43:25.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.045+0000 7fb9c5200640 1 -- 172.21.15.39:0/822077034 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9c00ce8b0 con 0x7fb9c006e330 2024-09-21T07:43:25.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.046+0000 7fb9b6000640 1 -- 172.21.15.39:0/822077034 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb9b001a070 con 0x7fb9c006e330 2024-09-21T07:43:25.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.047+0000 7fb9b6000640 1 --2- 172.21.15.39:0/822077034 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb99c078b90 0x7fb99c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:25.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.047+0000 7fb9b6000640 1 -- 172.21.15.39:0/822077034 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8076+0+0 (secure 0 0 0) 0x7fb9b009a9a0 con 0x7fb9c006e330 2024-09-21T07:43:25.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.047+0000 7fb9b7e00640 1 --2- 172.21.15.39:0/822077034 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb99c078b90 0x7fb99c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:25.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.048+0000 7fb9b7e00640 1 --2- 172.21.15.39:0/822077034 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb99c078b90 0x7fb99c07b0a0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb9c01f9e30 tx=0x7fb9a8005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:25.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.052+0000 7fb9b6000640 1 -- 172.21.15.39:0/822077034 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb9c006e740 con 0x7fb9c006e330 2024-09-21T07:43:25.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.287+0000 7fb9c5200640 1 -- 172.21.15.39:0/822077034 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb9c015e2a0 con 0x7fb9c006e330 2024-09-21T07:43:25.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.288+0000 7fb9b6000640 1 -- 172.21.15.39:0/822077034 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 144 v144) ==== 95+0+5304 (secure 0 0 0) 0x7fb9b0062490 con 0x7fb9c006e330 2024-09-21T07:43:25.290 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:43:25.290 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":144,"btime":"2024-09-21T07:43:21:456271+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7366,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/759792812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":759792812},{"type":"v1","addr":"172.21.15.148:6837","nonce":759792812}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":139}],"filesystems":[{"mdsmap":{"epoch":144,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:43:21.456230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7644},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7644":{"gid":7644,"name":"a","rank":1,"incarnation":138,"state":"up:active","state_seq":19,"addr":"172.21.15.39:6835/1326711417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1326711417},{"type":"v1","addr":"172.21.15.39:6835","nonce":1326711417}]},"join_fscid":-1,"laggy_since":"2024-09-21T07:43:21.456230+0000","export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311,7644]},"id":6}]} 2024-09-21T07:43:25.290 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 144 2024-09-21T07:43:25.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.290+0000 7fb9c5200640 1 -- 172.21.15.39:0/822077034 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb99c078b90 msgr2=0x7fb99c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:25.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.290+0000 7fb9c5200640 1 --2- 172.21.15.39:0/822077034 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb99c078b90 0x7fb99c07b0a0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb9c01f9e30 tx=0x7fb9a8005f90 comp rx=0 tx=0).stop 2024-09-21T07:43:25.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.290+0000 7fb9c5200640 1 -- 172.21.15.39:0/822077034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9c006e330 msgr2=0x7fb9c00ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:25.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.290+0000 7fb9c5200640 1 --2- 172.21.15.39:0/822077034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9c006e330 0x7fb9c00ae410 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fb9b000c3c0 tx=0x7fb9b000c880 comp rx=0 tx=0).stop 2024-09-21T07:43:25.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.290+0000 7fb9c5200640 1 -- 172.21.15.39:0/822077034 shutdown_connections 2024-09-21T07:43:25.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.290+0000 7fb9c5200640 1 --2- 172.21.15.39:0/822077034 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb99c078b90 0x7fb99c07b0a0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:25.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.290+0000 7fb9c5200640 1 --2- 172.21.15.39:0/822077034 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb9c0075dc0 0x7fb9c0146a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:25.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.290+0000 7fb9c5200640 1 --2- 172.21.15.39:0/822077034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9c0071540 0x7fb9c01e4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:25.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.290+0000 7fb9c5200640 1 --2- 172.21.15.39:0/822077034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9c006e330 0x7fb9c00ae410 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:25.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.290+0000 7fb9c5200640 1 -- 172.21.15.39:0/822077034 >> 172.21.15.39:0/822077034 conn(0x7fb9c0073980 msgr2=0x7fb9c015f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:25.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.291+0000 7fb9c5200640 1 -- 172.21.15.39:0/822077034 shutdown_connections 2024-09-21T07:43:25.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.291+0000 7fb9c5200640 1 -- 172.21.15.39:0/822077034 wait complete. 2024-09-21T07:43:25.318 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2024-09-21T07:43:25.561 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_snap":17,"last_created":17,"last_destroyed":14,"pending_noop":[],"snaps":[{"snapid":16,"ino":1099511627777,"stamp":"2024-09-21T07:42:21.254582+0000","name":"s34","metadata":{}},{"snapid":17,"ino":1099511627777,"stamp":"2024-09-21T07:43:06.501843+0000","name":"s37","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2024-09-21T07:43:25.562 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 17, "last_destroyed": 14, "last_snap": 17, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s34", "snapid": 16, "stamp": "2024-09-21T07:42:21.254582+0000" }, { "ino": 1099511627777, "metadata": {}, "name": "s37", "snapid": 17, "stamp": "2024-09-21T07:43:06.501843+0000" } ] } 2024-09-21T07:43:25.562 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 6:1 2024-09-21T07:43:25.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.788+0000 7fa38b200640 1 -- 172.21.15.39:0/2462807881 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa38c15c830 msgr2=0x7fa38c06f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:25.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.788+0000 7fa38b200640 1 --2- 172.21.15.39:0/2462807881 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa38c15c830 0x7fa38c06f8a0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fa374009830 tx=0x7fa37402a510 comp rx=0 tx=0).stop 2024-09-21T07:43:25.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.788+0000 7fa38b200640 1 -- 172.21.15.39:0/2462807881 shutdown_connections 2024-09-21T07:43:25.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.788+0000 7fa38b200640 1 --2- 172.21.15.39:0/2462807881 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa38c15c830 0x7fa38c06f8a0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:25.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.788+0000 7fa38b200640 1 -- 172.21.15.39:0/2462807881 >> 172.21.15.39:0/2462807881 conn(0x7fa38c0b2510 msgr2=0x7fa38c0d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:25.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.788+0000 7fa38b200640 1 -- 172.21.15.39:0/2462807881 shutdown_connections 2024-09-21T07:43:25.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.788+0000 7fa38b200640 1 -- 172.21.15.39:0/2462807881 wait complete. 2024-09-21T07:43:25.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.790+0000 7fa38b200640 1 Processor -- start 2024-09-21T07:43:25.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.790+0000 7fa38b200640 1 -- start start 2024-09-21T07:43:25.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.791+0000 7fa38b200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa38c06fde0 0x7fa38c09d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:25.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.792+0000 7fa38b200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa38c0d0f20 0x7fa38c1e7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:25.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.792+0000 7fa38b200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa38c15c830 0x7fa38c1e82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:25.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.792+0000 7fa38b200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa38c0b4640 con 0x7fa38c0d0f20 2024-09-21T07:43:25.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.792+0000 7fa38b200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa38c073850 con 0x7fa38c15c830 2024-09-21T07:43:25.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.792+0000 7fa38b200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa38c0b47c0 con 0x7fa38c06fde0 2024-09-21T07:43:25.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.792+0000 7fa381400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa38c0d0f20 0x7fa38c1e7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:25.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.792+0000 7fa381400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa38c0d0f20 0x7fa38c1e7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55524/0 (socket says 172.21.15.39:55524) 2024-09-21T07:43:25.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.792+0000 7fa381400640 1 -- 172.21.15.39:0/2492489968 learned_addr learned my addr 172.21.15.39:0/2492489968 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:25.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.792+0000 7fa389e00640 1 --2- 172.21.15.39:0/2492489968 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa38c06fde0 0x7fa38c09d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:25.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.792+0000 7fa38a800640 1 --2- 172.21.15.39:0/2492489968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa38c15c830 0x7fa38c1e82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:25.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.792+0000 7fa381400640 1 -- 172.21.15.39:0/2492489968 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa38c06fde0 msgr2=0x7fa38c09d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:25.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.792+0000 7fa381400640 1 --2- 172.21.15.39:0/2492489968 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa38c06fde0 0x7fa38c09d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:25.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.792+0000 7fa381400640 1 -- 172.21.15.39:0/2492489968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa38c15c830 msgr2=0x7fa38c1e82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:25.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.792+0000 7fa381400640 1 --2- 172.21.15.39:0/2492489968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa38c15c830 0x7fa38c1e82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:25.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.792+0000 7fa381400640 1 -- 172.21.15.39:0/2492489968 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa38c1f5ae0 con 0x7fa38c0d0f20 2024-09-21T07:43:25.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.793+0000 7fa381400640 1 --2- 172.21.15.39:0/2492489968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa38c0d0f20 0x7fa38c1e7db0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fa374000c00 tx=0x7fa374033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:25.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.793+0000 7fa382a00640 1 -- 172.21.15.39:0/2492489968 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa374031d60 con 0x7fa38c0d0f20 2024-09-21T07:43:25.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.793+0000 7fa382a00640 1 -- 172.21.15.39:0/2492489968 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3740330d0 con 0x7fa38c0d0f20 2024-09-21T07:43:25.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.793+0000 7fa38b200640 1 -- 172.21.15.39:0/2492489968 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa38c1f5dc0 con 0x7fa38c0d0f20 2024-09-21T07:43:25.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.793+0000 7fa382a00640 1 -- 172.21.15.39:0/2492489968 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa374033520 con 0x7fa38c0d0f20 2024-09-21T07:43:25.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.793+0000 7fa38b200640 1 -- 172.21.15.39:0/2492489968 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa38c1f5780 con 0x7fa38c0d0f20 2024-09-21T07:43:25.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.794+0000 7fa38b200640 1 -- 172.21.15.39:0/2492489968 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa38c06f8a0 con 0x7fa38c0d0f20 2024-09-21T07:43:25.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.796+0000 7fa382a00640 1 -- 172.21.15.39:0/2492489968 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa3740035f0 con 0x7fa38c0d0f20 2024-09-21T07:43:25.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.797+0000 7fa382a00640 1 --2- 172.21.15.39:0/2492489968 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa368078b90 0x7fa36807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:25.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.797+0000 7fa382a00640 1 -- 172.21.15.39:0/2492489968 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8076+0+0 (secure 0 0 0) 0x7fa3740bcc80 con 0x7fa38c0d0f20 2024-09-21T07:43:25.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.797+0000 7fa389e00640 1 --2- 172.21.15.39:0/2492489968 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa368078b90 0x7fa36807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:25.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.798+0000 7fa389e00640 1 --2- 172.21.15.39:0/2492489968 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa368078b90 0x7fa36807b0a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa37c006fd0 tx=0x7fa37c008040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:25.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:25.801+0000 7fa382a00640 1 -- 172.21.15.39:0/2492489968 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa37403f030 con 0x7fa38c0d0f20 2024-09-21T07:43:26.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.074+0000 7fa38b200640 1 -- 172.21.15.39:0/2492489968 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "6:1"} v 0) -- 0x7fa38c1e8890 con 0x7fa38c0d0f20 2024-09-21T07:43:26.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.081+0000 7fa382a00640 1 -- 172.21.15.39:0/2492489968 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "6:1"}]=0 failed mds gid 7644 v145) ==== 97+0+0 (secure 0 0 0) 0x7fa374084780 con 0x7fa38c0d0f20 2024-09-21T07:43:26.083 INFO:teuthology.orchestra.run.smithi039.stderr:failed mds gid 7644 2024-09-21T07:43:26.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.086+0000 7fa38b200640 1 -- 172.21.15.39:0/2492489968 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa368078b90 msgr2=0x7fa36807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:26.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.086+0000 7fa38b200640 1 --2- 172.21.15.39:0/2492489968 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa368078b90 0x7fa36807b0a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa37c006fd0 tx=0x7fa37c008040 comp rx=0 tx=0).stop 2024-09-21T07:43:26.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.086+0000 7fa38b200640 1 -- 172.21.15.39:0/2492489968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa38c0d0f20 msgr2=0x7fa38c1e7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:26.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.086+0000 7fa38b200640 1 --2- 172.21.15.39:0/2492489968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa38c0d0f20 0x7fa38c1e7db0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fa374000c00 tx=0x7fa374033d60 comp rx=0 tx=0).stop 2024-09-21T07:43:26.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.086+0000 7fa38b200640 1 -- 172.21.15.39:0/2492489968 shutdown_connections 2024-09-21T07:43:26.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.086+0000 7fa38b200640 1 --2- 172.21.15.39:0/2492489968 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa368078b90 0x7fa36807b0a0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:26.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.086+0000 7fa38b200640 1 --2- 172.21.15.39:0/2492489968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa38c15c830 0x7fa38c1e82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:26.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.086+0000 7fa38b200640 1 --2- 172.21.15.39:0/2492489968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa38c0d0f20 0x7fa38c1e7db0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:26.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.086+0000 7fa38b200640 1 --2- 172.21.15.39:0/2492489968 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa38c06fde0 0x7fa38c09d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:26.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.086+0000 7fa38b200640 1 -- 172.21.15.39:0/2492489968 >> 172.21.15.39:0/2492489968 conn(0x7fa38c0b2510 msgr2=0x7fa38c1e6ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:26.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.087+0000 7fa38b200640 1 -- 172.21.15.39:0/2492489968 shutdown_connections 2024-09-21T07:43:26.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.087+0000 7fa38b200640 1 -- 172.21.15.39:0/2492489968 wait complete. 2024-09-21T07:43:26.117 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T07:43:26.117 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T07:43:26.117 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T07:43:26.117 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:43:26.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:43:26.117 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/daemon/state.py", line 146, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi039 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a' 2024-09-21T07:43:26.118 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T07:43:26.118 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T07:43:26.122 INFO:tasks.ceph.mds.a:Started 2024-09-21T07:43:26.122 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:43:26.316 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-21T07:43:26.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.349+0000 7f9a11400640 1 -- 172.21.15.39:0/4131232465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a0c06e500 msgr2=0x7f9a0c167540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:26.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.349+0000 7f9a11400640 1 --2- 172.21.15.39:0/4131232465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a0c06e500 0x7f9a0c167540 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c15dea0 tx=0x7f9a0402a480 comp rx=0 tx=0).stop 2024-09-21T07:43:26.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.349+0000 7f9a11400640 1 -- 172.21.15.39:0/4131232465 shutdown_connections 2024-09-21T07:43:26.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.349+0000 7f9a11400640 1 --2- 172.21.15.39:0/4131232465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a0c06e500 0x7f9a0c167540 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:26.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.349+0000 7f9a11400640 1 -- 172.21.15.39:0/4131232465 >> 172.21.15.39:0/4131232465 conn(0x7f9a0c073980 msgr2=0x7f9a0c073d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:26.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.349+0000 7f9a11400640 1 -- 172.21.15.39:0/4131232465 shutdown_connections 2024-09-21T07:43:26.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.349+0000 7f9a11400640 1 -- 172.21.15.39:0/4131232465 wait complete. 2024-09-21T07:43:26.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.352+0000 7f9a11400640 1 Processor -- start 2024-09-21T07:43:26.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.352+0000 7f9a11400640 1 -- start start 2024-09-21T07:43:26.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.353+0000 7f9a11400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a0c071540 0x7f9a0c14ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:26.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.353+0000 7f9a11400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a0c075dc0 0x7f9a0c1593c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:26.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.353+0000 7f9a11400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a0c0b5070 0x7f9a0c0b5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:26.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.353+0000 7f9a11400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0c167e10 con 0x7f9a0c071540 2024-09-21T07:43:26.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.353+0000 7f9a11400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0c167c90 con 0x7f9a0c075dc0 2024-09-21T07:43:26.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.353+0000 7f9a11400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f9a0c167f90 con 0x7f9a0c0b5070 2024-09-21T07:43:26.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.353+0000 7f9a0a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a0c071540 0x7f9a0c14ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:26.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.353+0000 7f9a0a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a0c071540 0x7f9a0c14ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55556/0 (socket says 172.21.15.39:55556) 2024-09-21T07:43:26.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.353+0000 7f9a0a800640 1 -- 172.21.15.39:0/1528240568 learned_addr learned my addr 172.21.15.39:0/1528240568 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:26.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.354+0000 7f9a01e00640 1 --2- 172.21.15.39:0/1528240568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a0c075dc0 0x7f9a0c1593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:26.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.354+0000 7f9a0b200640 1 --2- 172.21.15.39:0/1528240568 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a0c0b5070 0x7f9a0c0b5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:26.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.354+0000 7f9a0a800640 1 -- 172.21.15.39:0/1528240568 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a0c0b5070 msgr2=0x7f9a0c0b5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:26.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.354+0000 7f9a0a800640 1 --2- 172.21.15.39:0/1528240568 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a0c0b5070 0x7f9a0c0b5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:26.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.354+0000 7f9a0a800640 1 -- 172.21.15.39:0/1528240568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a0c075dc0 msgr2=0x7f9a0c1593c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:26.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.354+0000 7f9a0a800640 1 --2- 172.21.15.39:0/1528240568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a0c075dc0 0x7f9a0c1593c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:26.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.354+0000 7f9a0a800640 1 -- 172.21.15.39:0/1528240568 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a0c0b59c0 con 0x7f9a0c071540 2024-09-21T07:43:26.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.354+0000 7f9a0b200640 1 --2- 172.21.15.39:0/1528240568 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a0c0b5070 0x7f9a0c0b5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:43:26.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.354+0000 7f9a0a800640 1 --2- 172.21.15.39:0/1528240568 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a0c071540 0x7f9a0c14ddb0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f99fc00ba50 tx=0x7f99fc00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:26.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.354+0000 7f9a03400640 1 -- 172.21.15.39:0/1528240568 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc019070 con 0x7f9a0c071540 2024-09-21T07:43:26.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.355+0000 7f9a03400640 1 -- 172.21.15.39:0/1528240568 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f99fc00cd10 con 0x7f9a0c071540 2024-09-21T07:43:26.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.355+0000 7f9a11400640 1 -- 172.21.15.39:0/1528240568 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a0c0b42b0 con 0x7f9a0c071540 2024-09-21T07:43:26.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.355+0000 7f9a03400640 1 -- 172.21.15.39:0/1528240568 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc0146a0 con 0x7f9a0c071540 2024-09-21T07:43:26.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.355+0000 7f9a11400640 1 -- 172.21.15.39:0/1528240568 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a0c0b4870 con 0x7f9a0c071540 2024-09-21T07:43:26.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.356+0000 7f9a11400640 1 -- 172.21.15.39:0/1528240568 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a0c15cdf0 con 0x7f9a0c071540 2024-09-21T07:43:26.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.357+0000 7f9a03400640 1 -- 172.21.15.39:0/1528240568 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f99fc01c070 con 0x7f9a0c071540 2024-09-21T07:43:26.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.358+0000 7f9a03400640 1 --2- 172.21.15.39:0/1528240568 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f99e8078b90 0x7f99e807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:26.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.359+0000 7f9a03400640 1 -- 172.21.15.39:0/1528240568 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8162+0+0 (secure 0 0 0) 0x7f99fc09b360 con 0x7f9a0c071540 2024-09-21T07:43:26.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.359+0000 7f9a01e00640 1 --2- 172.21.15.39:0/1528240568 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f99e8078b90 0x7f99e807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:26.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.360+0000 7f9a01e00640 1 --2- 172.21.15.39:0/1528240568 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f99e8078b90 0x7f99e807b0a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0b3ff0 tx=0x7f99f4009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:26.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.363+0000 7f9a03400640 1 -- 172.21.15.39:0/1528240568 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f99fc0627b0 con 0x7f9a0c071540 2024-09-21T07:43:26.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.599+0000 7f9a11400640 1 -- 172.21.15.39:0/1528240568 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a0c06ce80 con 0x7f9a0c071540 2024-09-21T07:43:26.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.600+0000 7f9a03400640 1 -- 172.21.15.39:0/1528240568 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) ==== 95+0+4424 (secure 0 0 0) 0x7f99fc0671e0 con 0x7f9a0c071540 2024-09-21T07:43:26.602 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:43:26.602 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":146,"btime":"2024-09-21T07:43:26:086257+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:43:26.086228+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7366},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7366":{"gid":7366,"name":"d","rank":1,"incarnation":146,"state":"up:replay","state_seq":1,"addr":"172.21.15.148:6837/759792812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":759792812},{"type":"v1","addr":"172.21.15.148:6837","nonce":759792812}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311]},"id":6}]} 2024-09-21T07:43:26.603 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 146 2024-09-21T07:43:26.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.603+0000 7f9a11400640 1 -- 172.21.15.39:0/1528240568 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f99e8078b90 msgr2=0x7f99e807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:26.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.603+0000 7f9a11400640 1 --2- 172.21.15.39:0/1528240568 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f99e8078b90 0x7f99e807b0a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0b3ff0 tx=0x7f99f4009040 comp rx=0 tx=0).stop 2024-09-21T07:43:26.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.603+0000 7f9a11400640 1 -- 172.21.15.39:0/1528240568 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a0c071540 msgr2=0x7f9a0c14ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:26.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.603+0000 7f9a11400640 1 --2- 172.21.15.39:0/1528240568 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a0c071540 0x7f9a0c14ddb0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f99fc00ba50 tx=0x7f99fc00bf10 comp rx=0 tx=0).stop 2024-09-21T07:43:26.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.603+0000 7f9a11400640 1 -- 172.21.15.39:0/1528240568 shutdown_connections 2024-09-21T07:43:26.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.603+0000 7f9a11400640 1 --2- 172.21.15.39:0/1528240568 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f99e8078b90 0x7f99e807b0a0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:26.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.603+0000 7f9a11400640 1 --2- 172.21.15.39:0/1528240568 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9a0c0b5070 0x7f9a0c0b5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:26.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.603+0000 7f9a11400640 1 --2- 172.21.15.39:0/1528240568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a0c075dc0 0x7f9a0c1593c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:26.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.603+0000 7f9a11400640 1 --2- 172.21.15.39:0/1528240568 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a0c071540 0x7f9a0c14ddb0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:26.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.603+0000 7f9a11400640 1 -- 172.21.15.39:0/1528240568 >> 172.21.15.39:0/1528240568 conn(0x7f9a0c073980 msgr2=0x7f9a0c06faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:26.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.603+0000 7f9a11400640 1 -- 172.21.15.39:0/1528240568 shutdown_connections 2024-09-21T07:43:26.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:26.603+0000 7f9a11400640 1 -- 172.21.15.39:0/1528240568 wait complete. 2024-09-21T07:43:26.630 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-09-21T07:43:29.512 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-09-21T07:43:31.631 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:43:31.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:31.869+0000 7f2468a00640 1 -- 172.21.15.39:0/3442480882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f246415cd30 msgr2=0x7f24640cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:31.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:31.869+0000 7f2468a00640 1 --2- 172.21.15.39:0/3442480882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f246415cd30 0x7f24640cfd10 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f2454005fd0 tx=0x7f245402a5d0 comp rx=0 tx=0).stop 2024-09-21T07:43:31.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:31.870+0000 7f2468a00640 1 -- 172.21.15.39:0/3442480882 shutdown_connections 2024-09-21T07:43:31.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:31.870+0000 7f2468a00640 1 --2- 172.21.15.39:0/3442480882 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f246406f800 0x7f24640cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:31.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:31.870+0000 7f2468a00640 1 --2- 172.21.15.39:0/3442480882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f246415cd30 0x7f24640cfd10 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:31.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:31.870+0000 7f2468a00640 1 -- 172.21.15.39:0/3442480882 >> 172.21.15.39:0/3442480882 conn(0x7f246406cfc0 msgr2=0x7f246406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:31.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:31.870+0000 7f2468a00640 1 -- 172.21.15.39:0/3442480882 shutdown_connections 2024-09-21T07:43:31.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:31.870+0000 7f2468a00640 1 -- 172.21.15.39:0/3442480882 wait complete. 2024-09-21T07:43:31.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:31.872+0000 7f2468a00640 1 Processor -- start 2024-09-21T07:43:31.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:31.872+0000 7f2468a00640 1 -- start start 2024-09-21T07:43:31.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:31.873+0000 7f2468a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f246406f800 0x7f24641f3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:31.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:31.873+0000 7f2468a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f246406fc10 0x7f2464158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:31.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:31.873+0000 7f2468a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f246415cd30 0x7f24641517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:31.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:31.873+0000 7f2468a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2464073890 con 0x7f246406fc10 2024-09-21T07:43:31.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:31.873+0000 7f2468a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2464073710 con 0x7f246406f800 2024-09-21T07:43:31.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:31.873+0000 7f2468a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f2464073a10 con 0x7f246415cd30 2024-09-21T07:43:31.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:31.874+0000 7f2461400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f246406fc10 0x7f2464158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:31.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:31.874+0000 7f2461400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f246406fc10 0x7f2464158eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55580/0 (socket says 172.21.15.39:55580) 2024-09-21T07:43:31.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:31.874+0000 7f2461400640 1 -- 172.21.15.39:0/2330470302 learned_addr learned my addr 172.21.15.39:0/2330470302 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:31.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:31.874+0000 7f2461e00640 1 --2- 172.21.15.39:0/2330470302 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f246406f800 0x7f24641f3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:31.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:31.874+0000 7f2462800640 1 --2- 172.21.15.39:0/2330470302 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f246415cd30 0x7f24641517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:31.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:31.874+0000 7f2461400640 1 -- 172.21.15.39:0/2330470302 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f246415cd30 msgr2=0x7f24641517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:31.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:31.874+0000 7f2461400640 1 --2- 172.21.15.39:0/2330470302 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f246415cd30 0x7f24641517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:31.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:31.874+0000 7f2461400640 1 -- 172.21.15.39:0/2330470302 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f246406f800 msgr2=0x7f24641f3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:31.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:31.874+0000 7f2461400640 1 --2- 172.21.15.39:0/2330470302 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f246406f800 0x7f24641f3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:31.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:31.874+0000 7f2461400640 1 -- 172.21.15.39:0/2330470302 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24641f6d90 con 0x7f246406fc10 2024-09-21T07:43:31.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:31.874+0000 7f2462800640 1 --2- 172.21.15.39:0/2330470302 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f246415cd30 0x7f24641517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:43:31.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:31.874+0000 7f2461400640 1 --2- 172.21.15.39:0/2330470302 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f246406fc10 0x7f2464158eb0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f245000d770 tx=0x7f245000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:31.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:31.875+0000 7f245a000640 1 -- 172.21.15.39:0/2330470302 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2450017070 con 0x7f246406fc10 2024-09-21T07:43:31.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:31.875+0000 7f245a000640 1 -- 172.21.15.39:0/2330470302 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2450009d20 con 0x7f246406fc10 2024-09-21T07:43:31.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:31.875+0000 7f245a000640 1 -- 172.21.15.39:0/2330470302 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24500106c0 con 0x7f246406fc10 2024-09-21T07:43:31.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:31.875+0000 7f2468a00640 1 -- 172.21.15.39:0/2330470302 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24641f7070 con 0x7f246406fc10 2024-09-21T07:43:31.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:31.875+0000 7f2468a00640 1 -- 172.21.15.39:0/2330470302 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24641f7630 con 0x7f246406fc10 2024-09-21T07:43:31.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:31.876+0000 7f2468a00640 1 -- 172.21.15.39:0/2330470302 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2464073ec0 con 0x7f246406fc10 2024-09-21T07:43:31.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:31.877+0000 7f245a000640 1 -- 172.21.15.39:0/2330470302 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f245001a070 con 0x7f246406fc10 2024-09-21T07:43:31.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:31.879+0000 7f245a000640 1 --2- 172.21.15.39:0/2330470302 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2444078b90 0x7f244407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:31.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:31.879+0000 7f245a000640 1 -- 172.21.15.39:0/2330470302 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8162+0+0 (secure 0 0 0) 0x7f245009b7b0 con 0x7f246406fc10 2024-09-21T07:43:31.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:31.879+0000 7f2461e00640 1 --2- 172.21.15.39:0/2330470302 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2444078b90 0x7f244407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:31.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:31.880+0000 7f2461e00640 1 --2- 172.21.15.39:0/2330470302 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2444078b90 0x7f244407b0a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f2454033000 tx=0x7f2454004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:31.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:31.884+0000 7f245a000640 1 -- 172.21.15.39:0/2330470302 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2450062bf0 con 0x7f246406fc10 2024-09-21T07:43:31.893 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:43:31.897 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:43:32.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.118+0000 7f2468a00640 1 -- 172.21.15.39:0/2330470302 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24641f6b00 con 0x7f246406fc10 2024-09-21T07:43:32.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.119+0000 7f245a000640 1 -- 172.21.15.39:0/2330470302 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 148 v148) ==== 95+0+5254 (secure 0 0 0) 0x7f2450067620 con 0x7f246406fc10 2024-09-21T07:43:32.120 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:43:32.120 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":148,"btime":"2024-09-21T07:43:28:096825+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":8052,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1551311805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1551311805},{"type":"v1","addr":"172.21.15.39:6835","nonce":1551311805}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":147}],"filesystems":[{"mdsmap":{"epoch":148,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:43:27.101219+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7366},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7366":{"gid":7366,"name":"d","rank":1,"incarnation":146,"state":"up:reconnect","state_seq":14,"addr":"172.21.15.148:6837/759792812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":759792812},{"type":"v1","addr":"172.21.15.148:6837","nonce":759792812}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311]},"id":6}]} 2024-09-21T07:43:32.120 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 148 2024-09-21T07:43:32.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.121+0000 7f2468a00640 1 -- 172.21.15.39:0/2330470302 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2444078b90 msgr2=0x7f244407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:32.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.121+0000 7f2468a00640 1 --2- 172.21.15.39:0/2330470302 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2444078b90 0x7f244407b0a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f2454033000 tx=0x7f2454004730 comp rx=0 tx=0).stop 2024-09-21T07:43:32.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.121+0000 7f2468a00640 1 -- 172.21.15.39:0/2330470302 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f246406fc10 msgr2=0x7f2464158eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:32.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.121+0000 7f2468a00640 1 --2- 172.21.15.39:0/2330470302 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f246406fc10 0x7f2464158eb0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f245000d770 tx=0x7f245000dc30 comp rx=0 tx=0).stop 2024-09-21T07:43:32.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.121+0000 7f2468a00640 1 -- 172.21.15.39:0/2330470302 shutdown_connections 2024-09-21T07:43:32.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.121+0000 7f2468a00640 1 --2- 172.21.15.39:0/2330470302 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2444078b90 0x7f244407b0a0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:32.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.121+0000 7f2468a00640 1 --2- 172.21.15.39:0/2330470302 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f246415cd30 0x7f24641517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:32.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.121+0000 7f2468a00640 1 --2- 172.21.15.39:0/2330470302 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f246406fc10 0x7f2464158eb0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:32.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.121+0000 7f2468a00640 1 --2- 172.21.15.39:0/2330470302 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f246406f800 0x7f24641f3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:32.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.121+0000 7f2468a00640 1 -- 172.21.15.39:0/2330470302 >> 172.21.15.39:0/2330470302 conn(0x7f246406cfc0 msgr2=0x7f24640cc490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:32.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.122+0000 7f2468a00640 1 -- 172.21.15.39:0/2330470302 shutdown_connections 2024-09-21T07:43:32.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.122+0000 7f2468a00640 1 -- 172.21.15.39:0/2330470302 wait complete. 2024-09-21T07:43:32.148 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-21T07:43:32.148 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:43:32.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.374+0000 7f49a7200640 1 -- 172.21.15.39:0/3074686367 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49a80ce310 msgr2=0x7f49a80ce720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:32.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.374+0000 7f49a7200640 1 --2- 172.21.15.39:0/3074686367 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49a80ce310 0x7f49a80ce720 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4994009830 tx=0x7f499402a480 comp rx=0 tx=0).stop 2024-09-21T07:43:32.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.374+0000 7f49a7200640 1 -- 172.21.15.39:0/3074686367 shutdown_connections 2024-09-21T07:43:32.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.374+0000 7f49a7200640 1 --2- 172.21.15.39:0/3074686367 >> v2:172.21.15.148:3300/0 conn(0x7f49a80ca890 0x7f49a80caca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:32.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.374+0000 7f49a7200640 1 --2- 172.21.15.39:0/3074686367 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49a80ce310 0x7f49a80ce720 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:32.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.374+0000 7f49a7200640 1 -- 172.21.15.39:0/3074686367 >> 172.21.15.39:0/3074686367 conn(0x7f49a80b2510 msgr2=0x7f49a80d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:32.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.374+0000 7f49a7200640 1 -- 172.21.15.39:0/3074686367 shutdown_connections 2024-09-21T07:43:32.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.374+0000 7f49a7200640 1 -- 172.21.15.39:0/3074686367 wait complete. 2024-09-21T07:43:32.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.376+0000 7f49a7200640 1 Processor -- start 2024-09-21T07:43:32.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.376+0000 7f49a7200640 1 -- start start 2024-09-21T07:43:32.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.378+0000 7f49a7200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49a80ca890 0x7f49a8099970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:32.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.378+0000 7f49a7200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49a80cc390 0x7f49a80ad0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:32.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.378+0000 7f49a7200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49a80ce310 0x7f49a80ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:32.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.378+0000 7f49a7200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f49a80cb5f0 con 0x7f49a80ce310 2024-09-21T07:43:32.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.378+0000 7f49a7200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f49a80cb470 con 0x7f49a80cc390 2024-09-21T07:43:32.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.378+0000 7f49a7200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f49a80cb770 con 0x7f49a80ca890 2024-09-21T07:43:32.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.378+0000 7f49a6800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49a80ce310 0x7f49a80ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:32.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.378+0000 7f49a6800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49a80ce310 0x7f49a80ad5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50292/0 (socket says 172.21.15.39:50292) 2024-09-21T07:43:32.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.378+0000 7f49a6800640 1 -- 172.21.15.39:0/3854753449 learned_addr learned my addr 172.21.15.39:0/3854753449 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:32.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.378+0000 7f49a5e00640 1 --2- 172.21.15.39:0/3854753449 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49a80ca890 0x7f49a8099970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:32.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.378+0000 7f49a5400640 1 --2- 172.21.15.39:0/3854753449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49a80cc390 0x7f49a80ad0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:32.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.379+0000 7f49a6800640 1 -- 172.21.15.39:0/3854753449 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49a80ca890 msgr2=0x7f49a8099970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:32.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.379+0000 7f49a6800640 1 --2- 172.21.15.39:0/3854753449 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49a80ca890 0x7f49a8099970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:32.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.379+0000 7f49a6800640 1 -- 172.21.15.39:0/3854753449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49a80cc390 msgr2=0x7f49a80ad0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:32.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.379+0000 7f49a6800640 1 --2- 172.21.15.39:0/3854753449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49a80cc390 0x7f49a80ad0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:32.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.379+0000 7f49a6800640 1 -- 172.21.15.39:0/3854753449 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a81e9620 con 0x7f49a80ce310 2024-09-21T07:43:32.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.379+0000 7f49a6800640 1 --2- 172.21.15.39:0/3854753449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49a80ce310 0x7f49a80ad5f0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f49a000ec50 tx=0x7f49a000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:32.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.379+0000 7f499e000640 1 -- 172.21.15.39:0/3854753449 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a0010630 con 0x7f49a80ce310 2024-09-21T07:43:32.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.379+0000 7f499e000640 1 -- 172.21.15.39:0/3854753449 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49a00107d0 con 0x7f49a80ce310 2024-09-21T07:43:32.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.379+0000 7f499e000640 1 -- 172.21.15.39:0/3854753449 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a0010c20 con 0x7f49a80ce310 2024-09-21T07:43:32.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.379+0000 7f49a7200640 1 -- 172.21.15.39:0/3854753449 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49a8099eb0 con 0x7f49a80ce310 2024-09-21T07:43:32.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.379+0000 7f49a7200640 1 -- 172.21.15.39:0/3854753449 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49a81e4f80 con 0x7f49a80ce310 2024-09-21T07:43:32.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.380+0000 7f49a7200640 1 -- 172.21.15.39:0/3854753449 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49a80ca6b0 con 0x7f49a80ce310 2024-09-21T07:43:32.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.382+0000 7f499e000640 1 -- 172.21.15.39:0/3854753449 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f49a0021440 con 0x7f49a80ce310 2024-09-21T07:43:32.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.383+0000 7f499e000640 1 --2- 172.21.15.39:0/3854753449 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4988078bc0 0x7f498807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:32.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.383+0000 7f499e000640 1 -- 172.21.15.39:0/3854753449 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8162+0+0 (secure 0 0 0) 0x7f49a009c4e0 con 0x7f49a80ce310 2024-09-21T07:43:32.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.383+0000 7f49a5e00640 1 --2- 172.21.15.39:0/3854753449 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4988078bc0 0x7f498807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:32.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.384+0000 7f49a5e00640 1 --2- 172.21.15.39:0/3854753449 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4988078bc0 0x7f498807b0d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f4998007d60 tx=0x7f4998007cf0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:32.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.387+0000 7f499e000640 1 -- 172.21.15.39:0/3854753449 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f49a80ca6b0 con 0x7f49a80ce310 2024-09-21T07:43:32.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.626+0000 7f49a7200640 1 -- 172.21.15.39:0/3854753449 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49a8070d10 con 0x7f49a80ce310 2024-09-21T07:43:32.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.627+0000 7f499e000640 1 -- 172.21.15.39:0/3854753449 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 148 v148) ==== 95+0+5254 (secure 0 0 0) 0x7f49a8070d10 con 0x7f49a80ce310 2024-09-21T07:43:32.628 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:43:32.628 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":148,"btime":"2024-09-21T07:43:28:096825+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":8052,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1551311805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1551311805},{"type":"v1","addr":"172.21.15.39:6835","nonce":1551311805}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":147}],"filesystems":[{"mdsmap":{"epoch":148,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:43:27.101219+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7366},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7366":{"gid":7366,"name":"d","rank":1,"incarnation":146,"state":"up:reconnect","state_seq":14,"addr":"172.21.15.148:6837/759792812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":759792812},{"type":"v1","addr":"172.21.15.148:6837","nonce":759792812}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311]},"id":6}]} 2024-09-21T07:43:32.628 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 148 2024-09-21T07:43:32.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.629+0000 7f49a7200640 1 -- 172.21.15.39:0/3854753449 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4988078bc0 msgr2=0x7f498807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:32.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.629+0000 7f49a7200640 1 --2- 172.21.15.39:0/3854753449 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4988078bc0 0x7f498807b0d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f4998007d60 tx=0x7f4998007cf0 comp rx=0 tx=0).stop 2024-09-21T07:43:32.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.629+0000 7f49a7200640 1 -- 172.21.15.39:0/3854753449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49a80ce310 msgr2=0x7f49a80ad5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:32.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.629+0000 7f49a7200640 1 --2- 172.21.15.39:0/3854753449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49a80ce310 0x7f49a80ad5f0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f49a000ec50 tx=0x7f49a000c6a0 comp rx=0 tx=0).stop 2024-09-21T07:43:32.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.629+0000 7f49a7200640 1 -- 172.21.15.39:0/3854753449 shutdown_connections 2024-09-21T07:43:32.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.629+0000 7f49a7200640 1 --2- 172.21.15.39:0/3854753449 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4988078bc0 0x7f498807b0d0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:32.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.629+0000 7f49a7200640 1 --2- 172.21.15.39:0/3854753449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49a80ce310 0x7f49a80ad5f0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:32.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.629+0000 7f49a7200640 1 --2- 172.21.15.39:0/3854753449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49a80cc390 0x7f49a80ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:32.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.629+0000 7f49a7200640 1 --2- 172.21.15.39:0/3854753449 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49a80ca890 0x7f49a8099970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:32.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.629+0000 7f49a7200640 1 -- 172.21.15.39:0/3854753449 >> 172.21.15.39:0/3854753449 conn(0x7f49a80b2510 msgr2=0x7f49a806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:32.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.629+0000 7f49a7200640 1 -- 172.21.15.39:0/3854753449 shutdown_connections 2024-09-21T07:43:32.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:32.630+0000 7f49a7200640 1 -- 172.21.15.39:0/3854753449 wait complete. 2024-09-21T07:43:33.057 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 148, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:36:20.189287+0000', 'modified': '2024-09-21T07:43:27.101219+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 116, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 7311, 'mds_1': 7366}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7311': {'gid': 7311, 'name': 'c', 'rank': 0, 'incarnation': 119, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.39:6833/1790581246', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 1790581246}, {'type': 'v1', 'addr': '172.21.15.39:6833', 'nonce': 1790581246}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_7366': {'gid': 7366, 'name': 'd', 'rank': 1, 'incarnation': 146, 'state': 'up:reconnect', 'state_seq': 14, 'addr': '172.21.15.148:6837/759792812', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 759792812}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 759792812}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7311, 'qdb_cluster': [7311]} 2024-09-21T07:43:33.057 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7366:up:reconnect 2024-09-21T07:43:34.058 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:43:34.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.297+0000 7faee1200640 1 -- 172.21.15.39:0/2621377627 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faec0004800 msgr2=0x7faec0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:34.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.297+0000 7faee1200640 1 --2- 172.21.15.39:0/2621377627 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faec0004800 0x7faec0004c10 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7faec4009b50 tx=0x7faec0005830 comp rx=0 tx=0).stop 2024-09-21T07:43:34.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.297+0000 7faedbe00640 1 -- 172.21.15.39:0/2621377627 reap_dead start 2024-09-21T07:43:34.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.298+0000 7faee1200640 1 -- 172.21.15.39:0/2621377627 shutdown_connections 2024-09-21T07:43:34.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.298+0000 7faee1200640 1 -- 172.21.15.39:0/2621377627 >> 172.21.15.39:0/2621377627 conn(0x7faedc0b2510 msgr2=0x7faedc0d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:34.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.298+0000 7faee1200640 1 -- 172.21.15.39:0/2621377627 shutdown_connections 2024-09-21T07:43:34.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.298+0000 7faee1200640 1 -- 172.21.15.39:0/2621377627 wait complete. 2024-09-21T07:43:34.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.300+0000 7faee1200640 1 Processor -- start 2024-09-21T07:43:34.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.300+0000 7faee1200640 1 -- start start 2024-09-21T07:43:34.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.301+0000 7faee1200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faec0004800 0x7faedc15aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:34.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.302+0000 7faee1200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faedc0c9b60 0x7faedc1f87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:34.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.302+0000 7faee1200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faedc09da00 0x7faedc1f5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:34.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.302+0000 7faee1200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7faedc073480 con 0x7faec0004800 2024-09-21T07:43:34.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.302+0000 7faee1200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7faedc073300 con 0x7faedc09da00 2024-09-21T07:43:34.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.302+0000 7faee1200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7faedc073600 con 0x7faedc0c9b60 2024-09-21T07:43:34.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.302+0000 7faedb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faec0004800 0x7faedc15aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:34.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.302+0000 7faedb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faec0004800 0x7faedc15aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50304/0 (socket says 172.21.15.39:50304) 2024-09-21T07:43:34.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.302+0000 7faedb400640 1 -- 172.21.15.39:0/1859102626 learned_addr learned my addr 172.21.15.39:0/1859102626 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:34.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.302+0000 7faed3e00640 1 --2- 172.21.15.39:0/1859102626 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faedc0c9b60 0x7faedc1f87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:34.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.302+0000 7faedbe00640 1 --2- 172.21.15.39:0/1859102626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faedc09da00 0x7faedc1f5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:34.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.302+0000 7faedb400640 1 -- 172.21.15.39:0/1859102626 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faedc0c9b60 msgr2=0x7faedc1f87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:34.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.302+0000 7faedb400640 1 --2- 172.21.15.39:0/1859102626 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faedc0c9b60 0x7faedc1f87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:34.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.302+0000 7faedb400640 1 -- 172.21.15.39:0/1859102626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faedc09da00 msgr2=0x7faedc1f5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:34.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.302+0000 7faedb400640 1 --2- 172.21.15.39:0/1859102626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faedc09da00 0x7faedc1f5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:34.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.302+0000 7faedb400640 1 -- 172.21.15.39:0/1859102626 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faedc1f9010 con 0x7faec0004800 2024-09-21T07:43:34.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.303+0000 7faedb400640 1 --2- 172.21.15.39:0/1859102626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faec0004800 0x7faedc15aa20 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7faecc006630 tx=0x7faecc0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:34.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.303+0000 7faed3400640 1 -- 172.21.15.39:0/1859102626 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc015070 con 0x7faec0004800 2024-09-21T07:43:34.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.303+0000 7faed3400640 1 -- 172.21.15.39:0/1859102626 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faecc0119c0 con 0x7faec0004800 2024-09-21T07:43:34.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.303+0000 7faed3400640 1 -- 172.21.15.39:0/1859102626 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc011e10 con 0x7faec0004800 2024-09-21T07:43:34.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.303+0000 7faee1200640 1 -- 172.21.15.39:0/1859102626 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faedc1f92f0 con 0x7faec0004800 2024-09-21T07:43:34.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.303+0000 7faee1200640 1 -- 172.21.15.39:0/1859102626 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faedc1f8d00 con 0x7faec0004800 2024-09-21T07:43:34.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.304+0000 7faee1200640 1 -- 172.21.15.39:0/1859102626 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faedc06f8a0 con 0x7faec0004800 2024-09-21T07:43:34.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.305+0000 7faed3400640 1 -- 172.21.15.39:0/1859102626 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7faecc018070 con 0x7faec0004800 2024-09-21T07:43:34.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.306+0000 7faed3400640 1 --2- 172.21.15.39:0/1859102626 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faeb8078b90 0x7faeb807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:34.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.306+0000 7faed3400640 1 -- 172.21.15.39:0/1859102626 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8162+0+0 (secure 0 0 0) 0x7faecc09c4a0 con 0x7faec0004800 2024-09-21T07:43:34.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.307+0000 7faed3e00640 1 --2- 172.21.15.39:0/1859102626 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faeb8078b90 0x7faeb807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:34.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.308+0000 7faed3e00640 1 --2- 172.21.15.39:0/1859102626 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faeb8078b90 0x7faeb807b0a0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7faec4009b50 tx=0x7faec402d040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:34.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.311+0000 7faed3400640 1 -- 172.21.15.39:0/1859102626 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7faecc0638f0 con 0x7faec0004800 2024-09-21T07:43:34.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.545+0000 7faee1200640 1 -- 172.21.15.39:0/1859102626 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faedc0cfd80 con 0x7faec0004800 2024-09-21T07:43:34.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.546+0000 7faed3400640 1 -- 172.21.15.39:0/1859102626 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 148 v148) ==== 95+0+5254 (secure 0 0 0) 0x7faecc068320 con 0x7faec0004800 2024-09-21T07:43:34.547 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:43:34.547 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":148,"btime":"2024-09-21T07:43:28:096825+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":8052,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1551311805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1551311805},{"type":"v1","addr":"172.21.15.39:6835","nonce":1551311805}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":147}],"filesystems":[{"mdsmap":{"epoch":148,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:43:27.101219+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7366},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7366":{"gid":7366,"name":"d","rank":1,"incarnation":146,"state":"up:reconnect","state_seq":14,"addr":"172.21.15.148:6837/759792812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":759792812},{"type":"v1","addr":"172.21.15.148:6837","nonce":759792812}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311]},"id":6}]} 2024-09-21T07:43:34.547 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 148 2024-09-21T07:43:34.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.548+0000 7faee1200640 1 -- 172.21.15.39:0/1859102626 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faeb8078b90 msgr2=0x7faeb807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:34.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.548+0000 7faee1200640 1 --2- 172.21.15.39:0/1859102626 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faeb8078b90 0x7faeb807b0a0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7faec4009b50 tx=0x7faec402d040 comp rx=0 tx=0).stop 2024-09-21T07:43:34.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.548+0000 7faee1200640 1 -- 172.21.15.39:0/1859102626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faec0004800 msgr2=0x7faedc15aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:34.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.548+0000 7faee1200640 1 --2- 172.21.15.39:0/1859102626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faec0004800 0x7faedc15aa20 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7faecc006630 tx=0x7faecc0039c0 comp rx=0 tx=0).stop 2024-09-21T07:43:34.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.549+0000 7faee1200640 1 -- 172.21.15.39:0/1859102626 shutdown_connections 2024-09-21T07:43:34.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.549+0000 7faee1200640 1 --2- 172.21.15.39:0/1859102626 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faeb8078b90 0x7faeb807b0a0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:34.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.549+0000 7faee1200640 1 --2- 172.21.15.39:0/1859102626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faedc09da00 0x7faedc1f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:34.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.549+0000 7faee1200640 1 --2- 172.21.15.39:0/1859102626 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faedc0c9b60 0x7faedc1f87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:34.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.549+0000 7faee1200640 1 --2- 172.21.15.39:0/1859102626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faec0004800 0x7faedc15aa20 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:34.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.549+0000 7faee1200640 1 -- 172.21.15.39:0/1859102626 >> 172.21.15.39:0/1859102626 conn(0x7faedc0b2510 msgr2=0x7faedc074b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:34.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.549+0000 7faee1200640 1 -- 172.21.15.39:0/1859102626 shutdown_connections 2024-09-21T07:43:34.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:34.549+0000 7faee1200640 1 -- 172.21.15.39:0/1859102626 wait complete. 2024-09-21T07:43:34.975 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 148, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:36:20.189287+0000', 'modified': '2024-09-21T07:43:27.101219+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 116, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 7311, 'mds_1': 7366}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7311': {'gid': 7311, 'name': 'c', 'rank': 0, 'incarnation': 119, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.39:6833/1790581246', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 1790581246}, {'type': 'v1', 'addr': '172.21.15.39:6833', 'nonce': 1790581246}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_7366': {'gid': 7366, 'name': 'd', 'rank': 1, 'incarnation': 146, 'state': 'up:reconnect', 'state_seq': 14, 'addr': '172.21.15.148:6837/759792812', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 759792812}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 759792812}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7311, 'qdb_cluster': [7311]} 2024-09-21T07:43:34.975 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7366:up:reconnect 2024-09-21T07:43:35.976 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:43:36.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.216+0000 7f99ed000640 1 -- 172.21.15.39:0/4006748176 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99e8151590 msgr2=0x7f99e81519a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:36.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.216+0000 7f99ed000640 1 --2- 172.21.15.39:0/4006748176 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99e8151590 0x7f99e81519a0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f99e8073100 tx=0x7f99e002a980 comp rx=0 tx=0).stop 2024-09-21T07:43:36.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.217+0000 7f99ed000640 1 -- 172.21.15.39:0/4006748176 shutdown_connections 2024-09-21T07:43:36.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.217+0000 7f99ed000640 1 --2- 172.21.15.39:0/4006748176 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99e8151590 0x7f99e81519a0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:36.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.217+0000 7f99ed000640 1 --2- 172.21.15.39:0/4006748176 >> v2:172.21.15.148:3300/0 conn(0x7f99e80b3370 0x7f99e80b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:36.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.217+0000 7f99ed000640 1 --2- 172.21.15.39:0/4006748176 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99e815e590 0x7f99e8070a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:36.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.217+0000 7f99ed000640 1 -- 172.21.15.39:0/4006748176 >> 172.21.15.39:0/4006748176 conn(0x7f99e806cfc0 msgr2=0x7f99e806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:36.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.217+0000 7f99ed000640 1 -- 172.21.15.39:0/4006748176 shutdown_connections 2024-09-21T07:43:36.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.217+0000 7f99ed000640 1 -- 172.21.15.39:0/4006748176 wait complete. 2024-09-21T07:43:36.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.219+0000 7f99ed000640 1 Processor -- start 2024-09-21T07:43:36.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.219+0000 7f99ed000640 1 -- start start 2024-09-21T07:43:36.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.220+0000 7f99ed000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99e80b3370 0x7f99e814e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:36.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.220+0000 7f99ed000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99e8151590 0x7f99e81577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:36.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.220+0000 7f99ed000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99e815e590 0x7f99e8157d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:36.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.220+0000 7f99ed000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f99e815b750 con 0x7f99e80b3370 2024-09-21T07:43:36.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.221+0000 7f99ed000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f99e815b5d0 con 0x7f99e815e590 2024-09-21T07:43:36.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.221+0000 7f99ed000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f99e815b8d0 con 0x7f99e8151590 2024-09-21T07:43:36.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.221+0000 7f99e5e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99e80b3370 0x7f99e814e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:36.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.221+0000 7f99e5e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99e80b3370 0x7f99e814e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50312/0 (socket says 172.21.15.39:50312) 2024-09-21T07:43:36.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.221+0000 7f99e5e00640 1 -- 172.21.15.39:0/2558951068 learned_addr learned my addr 172.21.15.39:0/2558951068 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:36.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.221+0000 7f99e6800640 1 --2- 172.21.15.39:0/2558951068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99e815e590 0x7f99e8157d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:36.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.221+0000 7f99e5400640 1 --2- 172.21.15.39:0/2558951068 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99e8151590 0x7f99e81577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:36.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.221+0000 7f99e5e00640 1 -- 172.21.15.39:0/2558951068 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99e8151590 msgr2=0x7f99e81577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:36.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.221+0000 7f99e5e00640 1 --2- 172.21.15.39:0/2558951068 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99e8151590 0x7f99e81577c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:36.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.221+0000 7f99e5e00640 1 -- 172.21.15.39:0/2558951068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99e815e590 msgr2=0x7f99e8157d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:36.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.221+0000 7f99e5e00640 1 --2- 172.21.15.39:0/2558951068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99e815e590 0x7f99e8157d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:36.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.221+0000 7f99e5e00640 1 -- 172.21.15.39:0/2558951068 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e81563c0 con 0x7f99e80b3370 2024-09-21T07:43:36.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.221+0000 7f99e5400640 1 --2- 172.21.15.39:0/2558951068 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99e8151590 0x7f99e81577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:43:36.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.222+0000 7f99e5e00640 1 --2- 172.21.15.39:0/2558951068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99e80b3370 0x7f99e814e270 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f99d800e7f0 tx=0x7f99d800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:36.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.222+0000 7f99de000640 1 -- 172.21.15.39:0/2558951068 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d800ccc0 con 0x7f99e80b3370 2024-09-21T07:43:36.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.222+0000 7f99de000640 1 -- 172.21.15.39:0/2558951068 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f99d800ce80 con 0x7f99e80b3370 2024-09-21T07:43:36.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.222+0000 7f99de000640 1 -- 172.21.15.39:0/2558951068 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d8005280 con 0x7f99e80b3370 2024-09-21T07:43:36.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.222+0000 7f99ed000640 1 -- 172.21.15.39:0/2558951068 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99e81566a0 con 0x7f99e80b3370 2024-09-21T07:43:36.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.222+0000 7f99ed000640 1 -- 172.21.15.39:0/2558951068 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99e8156c60 con 0x7f99e80b3370 2024-09-21T07:43:36.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.223+0000 7f99ed000640 1 -- 172.21.15.39:0/2558951068 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99e815bdf0 con 0x7f99e80b3370 2024-09-21T07:43:36.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.224+0000 7f99de000640 1 -- 172.21.15.39:0/2558951068 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f99d8005420 con 0x7f99e80b3370 2024-09-21T07:43:36.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.225+0000 7f99de000640 1 --2- 172.21.15.39:0/2558951068 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f99c8078b90 0x7f99c807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:36.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.225+0000 7f99de000640 1 -- 172.21.15.39:0/2558951068 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8205+0+0 (secure 0 0 0) 0x7f99d809c070 con 0x7f99e80b3370 2024-09-21T07:43:36.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.226+0000 7f99e5400640 1 --2- 172.21.15.39:0/2558951068 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f99c8078b90 0x7f99c807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:36.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.227+0000 7f99e5400640 1 --2- 172.21.15.39:0/2558951068 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f99c8078b90 0x7f99c807b0a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f99e81e7890 tx=0x7f99d400c040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:36.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.230+0000 7f99de000640 1 -- 172.21.15.39:0/2558951068 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f99d8063aa0 con 0x7f99e80b3370 2024-09-21T07:43:36.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.463+0000 7f99ed000640 1 -- 172.21.15.39:0/2558951068 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99e81599a0 con 0x7f99e80b3370 2024-09-21T07:43:36.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.464+0000 7f99de000640 1 -- 172.21.15.39:0/2558951068 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 148 v148) ==== 95+0+5254 (secure 0 0 0) 0x7f99d80633a0 con 0x7f99e80b3370 2024-09-21T07:43:36.466 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:43:36.466 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":148,"btime":"2024-09-21T07:43:28:096825+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":8052,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1551311805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1551311805},{"type":"v1","addr":"172.21.15.39:6835","nonce":1551311805}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":147}],"filesystems":[{"mdsmap":{"epoch":148,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:43:27.101219+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7366},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7366":{"gid":7366,"name":"d","rank":1,"incarnation":146,"state":"up:reconnect","state_seq":14,"addr":"172.21.15.148:6837/759792812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":759792812},{"type":"v1","addr":"172.21.15.148:6837","nonce":759792812}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311]},"id":6}]} 2024-09-21T07:43:36.466 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 148 2024-09-21T07:43:36.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.467+0000 7f99ed000640 1 -- 172.21.15.39:0/2558951068 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f99c8078b90 msgr2=0x7f99c807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:36.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.467+0000 7f99ed000640 1 --2- 172.21.15.39:0/2558951068 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f99c8078b90 0x7f99c807b0a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f99e81e7890 tx=0x7f99d400c040 comp rx=0 tx=0).stop 2024-09-21T07:43:36.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.467+0000 7f99ed000640 1 -- 172.21.15.39:0/2558951068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99e80b3370 msgr2=0x7f99e814e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:36.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.467+0000 7f99ed000640 1 --2- 172.21.15.39:0/2558951068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99e80b3370 0x7f99e814e270 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f99d800e7f0 tx=0x7f99d800ecb0 comp rx=0 tx=0).stop 2024-09-21T07:43:36.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.467+0000 7f99ed000640 1 -- 172.21.15.39:0/2558951068 shutdown_connections 2024-09-21T07:43:36.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.467+0000 7f99ed000640 1 --2- 172.21.15.39:0/2558951068 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f99c8078b90 0x7f99c807b0a0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:36.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.467+0000 7f99ed000640 1 --2- 172.21.15.39:0/2558951068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99e815e590 0x7f99e8157d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:36.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.467+0000 7f99ed000640 1 --2- 172.21.15.39:0/2558951068 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f99e8151590 0x7f99e81577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:36.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.467+0000 7f99ed000640 1 --2- 172.21.15.39:0/2558951068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99e80b3370 0x7f99e814e270 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:36.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.467+0000 7f99ed000640 1 -- 172.21.15.39:0/2558951068 >> 172.21.15.39:0/2558951068 conn(0x7f99e806cfc0 msgr2=0x7f99e8150cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:36.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.467+0000 7f99ed000640 1 -- 172.21.15.39:0/2558951068 shutdown_connections 2024-09-21T07:43:36.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:36.467+0000 7f99ed000640 1 -- 172.21.15.39:0/2558951068 wait complete. 2024-09-21T07:43:36.895 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 148, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:36:20.189287+0000', 'modified': '2024-09-21T07:43:27.101219+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 116, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 7311, 'mds_1': 7366}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7311': {'gid': 7311, 'name': 'c', 'rank': 0, 'incarnation': 119, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.39:6833/1790581246', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 1790581246}, {'type': 'v1', 'addr': '172.21.15.39:6833', 'nonce': 1790581246}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_7366': {'gid': 7366, 'name': 'd', 'rank': 1, 'incarnation': 146, 'state': 'up:reconnect', 'state_seq': 14, 'addr': '172.21.15.148:6837/759792812', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 759792812}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 759792812}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7311, 'qdb_cluster': [7311]} 2024-09-21T07:43:36.895 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7366:up:reconnect 2024-09-21T07:43:37.897 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:43:38.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.133+0000 7f5ef2e00640 1 -- 172.21.15.39:0/3178739202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ed4003fd0 msgr2=0x7f5ed4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:38.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.133+0000 7f5ef2e00640 1 --2- 172.21.15.39:0/3178739202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ed4003fd0 0x7f5ed4004800 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8009b50 tx=0x7f5ed4005830 comp rx=0 tx=0).stop 2024-09-21T07:43:38.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.133+0000 7f5ef2400640 1 -- 172.21.15.39:0/3178739202 reap_dead start 2024-09-21T07:43:38.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.133+0000 7f5ef2e00640 1 -- 172.21.15.39:0/3178739202 shutdown_connections 2024-09-21T07:43:38.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.133+0000 7f5ef2e00640 1 -- 172.21.15.39:0/3178739202 >> 172.21.15.39:0/3178739202 conn(0x7f5eec0b2510 msgr2=0x7f5eec0d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:38.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.133+0000 7f5ef2e00640 1 -- 172.21.15.39:0/3178739202 shutdown_connections 2024-09-21T07:43:38.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.133+0000 7f5ef2e00640 1 -- 172.21.15.39:0/3178739202 wait complete. 2024-09-21T07:43:38.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.135+0000 7f5ef2e00640 1 Processor -- start 2024-09-21T07:43:38.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.135+0000 7f5ef2e00640 1 -- start start 2024-09-21T07:43:38.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.137+0000 7f5ef2e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ed4003fd0 0x7f5eec14bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:38.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.137+0000 7f5ef2e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5eec1f8a50 0x7f5eec1f8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:38.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.137+0000 7f5ef2e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5eec09ac20 0x7f5eec09b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:38.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.137+0000 7f5ef2e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5eec073420 con 0x7f5ed4003fd0 2024-09-21T07:43:38.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.137+0000 7f5ef2e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5eec0732a0 con 0x7f5eec09ac20 2024-09-21T07:43:38.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.137+0000 7f5ef2e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f5eec0735a0 con 0x7f5eec1f8a50 2024-09-21T07:43:38.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.137+0000 7f5ef1a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ed4003fd0 0x7f5eec14bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:38.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.137+0000 7f5ef1a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ed4003fd0 0x7f5eec14bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50328/0 (socket says 172.21.15.39:50328) 2024-09-21T07:43:38.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.137+0000 7f5ef1a00640 1 -- 172.21.15.39:0/3365349493 learned_addr learned my addr 172.21.15.39:0/3365349493 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:38.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.137+0000 7f5ef1000640 1 --2- 172.21.15.39:0/3365349493 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5eec1f8a50 0x7f5eec1f8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:38.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.137+0000 7f5ef2400640 1 --2- 172.21.15.39:0/3365349493 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5eec09ac20 0x7f5eec09b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:38.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.138+0000 7f5ef1a00640 1 -- 172.21.15.39:0/3365349493 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5eec1f8a50 msgr2=0x7f5eec1f8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:38.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.138+0000 7f5ef1a00640 1 --2- 172.21.15.39:0/3365349493 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5eec1f8a50 0x7f5eec1f8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:38.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.138+0000 7f5ef1a00640 1 -- 172.21.15.39:0/3365349493 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5eec09ac20 msgr2=0x7f5eec09b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:38.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.138+0000 7f5ef1a00640 1 --2- 172.21.15.39:0/3365349493 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5eec09ac20 0x7f5eec09b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:38.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.138+0000 7f5ef1a00640 1 -- 172.21.15.39:0/3365349493 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eec14c280 con 0x7f5ed4003fd0 2024-09-21T07:43:38.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.138+0000 7f5ef1a00640 1 --2- 172.21.15.39:0/3365349493 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ed4003fd0 0x7f5eec14bd40 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0006660 tx=0x7f5ee00039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:38.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.138+0000 7f5ee9600640 1 -- 172.21.15.39:0/3365349493 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0015070 con 0x7f5ed4003fd0 2024-09-21T07:43:38.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.138+0000 7f5ee9600640 1 -- 172.21.15.39:0/3365349493 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ee0011920 con 0x7f5ed4003fd0 2024-09-21T07:43:38.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.138+0000 7f5ee9600640 1 -- 172.21.15.39:0/3365349493 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0011d70 con 0x7f5ed4003fd0 2024-09-21T07:43:38.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.138+0000 7f5ef2e00640 1 -- 172.21.15.39:0/3365349493 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eec1f93a0 con 0x7f5ed4003fd0 2024-09-21T07:43:38.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.138+0000 7f5ef2e00640 1 -- 172.21.15.39:0/3365349493 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eec14ca70 con 0x7f5ed4003fd0 2024-09-21T07:43:38.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.139+0000 7f5ef2e00640 1 -- 172.21.15.39:0/3365349493 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5eec06f8a0 con 0x7f5ed4003fd0 2024-09-21T07:43:38.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.141+0000 7f5ee9600640 1 -- 172.21.15.39:0/3365349493 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5ee0010420 con 0x7f5ed4003fd0 2024-09-21T07:43:38.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.142+0000 7f5ee9600640 1 --2- 172.21.15.39:0/3365349493 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5ecc078b90 0x7f5ecc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:38.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.142+0000 7f5ee9600640 1 -- 172.21.15.39:0/3365349493 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8205+0+0 (secure 0 0 0) 0x7f5ee009c490 con 0x7f5ed4003fd0 2024-09-21T07:43:38.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.142+0000 7f5ef1000640 1 --2- 172.21.15.39:0/3365349493 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5ecc078b90 0x7f5ecc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:38.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.143+0000 7f5ef1000640 1 --2- 172.21.15.39:0/3365349493 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5ecc078b90 0x7f5ecc07b0a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8035000 tx=0x7f5ed80046c0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:38.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.147+0000 7f5ee9600640 1 -- 172.21.15.39:0/3365349493 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5ee0063970 con 0x7f5ed4003fd0 2024-09-21T07:43:38.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.385+0000 7f5ef2e00640 1 -- 172.21.15.39:0/3365349493 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5eec069360 con 0x7f5ed4003fd0 2024-09-21T07:43:38.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.385+0000 7f5ee9600640 1 -- 172.21.15.39:0/3365349493 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 150 v150) ==== 95+0+5256 (secure 0 0 0) 0x7f5ee00683a0 con 0x7f5ed4003fd0 2024-09-21T07:43:38.390 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:43:38.390 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":150,"btime":"2024-09-21T07:43:38:142009+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":8052,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1551311805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1551311805},{"type":"v1","addr":"172.21.15.39:6835","nonce":1551311805}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":147}],"filesystems":[{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:43:38.141999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7366},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7366":{"gid":7366,"name":"d","rank":1,"incarnation":146,"state":"up:active","state_seq":18,"addr":"172.21.15.148:6837/759792812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":759792812},{"type":"v1","addr":"172.21.15.148:6837","nonce":759792812}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311,7366]},"id":6}]} 2024-09-21T07:43:38.391 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 150 2024-09-21T07:43:38.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.388+0000 7f5ef2e00640 1 -- 172.21.15.39:0/3365349493 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5ecc078b90 msgr2=0x7f5ecc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:38.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.388+0000 7f5ef2e00640 1 --2- 172.21.15.39:0/3365349493 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5ecc078b90 0x7f5ecc07b0a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8035000 tx=0x7f5ed80046c0 comp rx=0 tx=0).stop 2024-09-21T07:43:38.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.388+0000 7f5ef2e00640 1 -- 172.21.15.39:0/3365349493 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ed4003fd0 msgr2=0x7f5eec14bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:38.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.388+0000 7f5ef2e00640 1 --2- 172.21.15.39:0/3365349493 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ed4003fd0 0x7f5eec14bd40 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0006660 tx=0x7f5ee00039c0 comp rx=0 tx=0).stop 2024-09-21T07:43:38.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.388+0000 7f5ef2e00640 1 -- 172.21.15.39:0/3365349493 shutdown_connections 2024-09-21T07:43:38.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.388+0000 7f5ef2e00640 1 --2- 172.21.15.39:0/3365349493 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5ecc078b90 0x7f5ecc07b0a0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:38.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.388+0000 7f5ef2e00640 1 --2- 172.21.15.39:0/3365349493 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5eec09ac20 0x7f5eec09b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:38.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.388+0000 7f5ef2e00640 1 --2- 172.21.15.39:0/3365349493 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5eec1f8a50 0x7f5eec1f8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:38.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.388+0000 7f5ef2e00640 1 --2- 172.21.15.39:0/3365349493 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ed4003fd0 0x7f5eec14bd40 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:38.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.388+0000 7f5ef2e00640 1 -- 172.21.15.39:0/3365349493 >> 172.21.15.39:0/3365349493 conn(0x7f5eec0b2510 msgr2=0x7f5eec0742e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:38.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.388+0000 7f5ef2e00640 1 -- 172.21.15.39:0/3365349493 shutdown_connections 2024-09-21T07:43:38.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:38.388+0000 7f5ef2e00640 1 -- 172.21.15.39:0/3365349493 wait complete. 2024-09-21T07:43:38.817 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 150, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:36:20.189287+0000', 'modified': '2024-09-21T07:43:38.141999+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 116, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 7311, 'mds_1': 7366}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7311': {'gid': 7311, 'name': 'c', 'rank': 0, 'incarnation': 119, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.39:6833/1790581246', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 1790581246}, {'type': 'v1', 'addr': '172.21.15.39:6833', 'nonce': 1790581246}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_7366': {'gid': 7366, 'name': 'd', 'rank': 1, 'incarnation': 146, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.148:6837/759792812', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 759792812}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 759792812}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7311, 'qdb_cluster': [7311, 7366]} 2024-09-21T07:43:38.817 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2024-09-21T07:43:38.817 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T07:43:39.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.045+0000 7fdff9a00640 1 -- 172.21.15.39:0/3709573933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfd8003a30 msgr2=0x7fdfd8003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:39.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.045+0000 7fdff9a00640 1 --2- 172.21.15.39:0/3709573933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfd8003a30 0x7fdfd8003ea0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fdfe40016e0 tx=0x7fdfd8005830 comp rx=0 tx=0).stop 2024-09-21T07:43:39.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.045+0000 7fdff9000640 1 -- 172.21.15.39:0/3709573933 reap_dead start 2024-09-21T07:43:39.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.045+0000 7fdff9a00640 1 -- 172.21.15.39:0/3709573933 shutdown_connections 2024-09-21T07:43:39.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.045+0000 7fdff9a00640 1 -- 172.21.15.39:0/3709573933 >> 172.21.15.39:0/3709573933 conn(0x7fdff408bfb0 msgr2=0x7fdff409d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:39.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.045+0000 7fdff9a00640 1 -- 172.21.15.39:0/3709573933 shutdown_connections 2024-09-21T07:43:39.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.045+0000 7fdff9a00640 1 -- 172.21.15.39:0/3709573933 wait complete. 2024-09-21T07:43:39.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.047+0000 7fdff9a00640 1 Processor -- start 2024-09-21T07:43:39.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.047+0000 7fdff9a00640 1 -- start start 2024-09-21T07:43:39.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.049+0000 7fdff9a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfd8003a30 0x7fdff40cb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:39.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.049+0000 7fdff9a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdff41f8f50 0x7fdff41f9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:39.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.049+0000 7fdff9a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdff40cd050 0x7fdff40cd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:39.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.049+0000 7fdff9a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdff40d2310 con 0x7fdfd8003a30 2024-09-21T07:43:39.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.049+0000 7fdff9a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdff40d1750 con 0x7fdff40cd050 2024-09-21T07:43:39.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.049+0000 7fdff9a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fdff40d2490 con 0x7fdff41f8f50 2024-09-21T07:43:39.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.049+0000 7fdff3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfd8003a30 0x7fdff40cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:39.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.049+0000 7fdff3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfd8003a30 0x7fdff40cb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50346/0 (socket says 172.21.15.39:50346) 2024-09-21T07:43:39.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.049+0000 7fdff3e00640 1 -- 172.21.15.39:0/3878181580 learned_addr learned my addr 172.21.15.39:0/3878181580 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:39.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.049+0000 7fdfebe00640 1 --2- 172.21.15.39:0/3878181580 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdff41f8f50 0x7fdff41f9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:39.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.049+0000 7fdff9000640 1 --2- 172.21.15.39:0/3878181580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdff40cd050 0x7fdff40cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:39.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.049+0000 7fdff3e00640 1 -- 172.21.15.39:0/3878181580 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdff41f8f50 msgr2=0x7fdff41f9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:39.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.049+0000 7fdff3e00640 1 --2- 172.21.15.39:0/3878181580 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdff41f8f50 0x7fdff41f9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.049+0000 7fdff3e00640 1 -- 172.21.15.39:0/3878181580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdff40cd050 msgr2=0x7fdff40cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:39.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.050+0000 7fdff3e00640 1 --2- 172.21.15.39:0/3878181580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdff40cd050 0x7fdff40cd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.050+0000 7fdff3e00640 1 -- 172.21.15.39:0/3878181580 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdff40d2a70 con 0x7fdfd8003a30 2024-09-21T07:43:39.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.050+0000 7fdff9000640 1 --2- 172.21.15.39:0/3878181580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdff40cd050 0x7fdff40cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:43:39.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.050+0000 7fdff3e00640 1 --2- 172.21.15.39:0/3878181580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfd8003a30 0x7fdff40cb630 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fdfe402a910 tx=0x7fdfe4006310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:39.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.050+0000 7fdff0c00640 1 -- 172.21.15.39:0/3878181580 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4041070 con 0x7fdfd8003a30 2024-09-21T07:43:39.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.050+0000 7fdff0c00640 1 -- 172.21.15.39:0/3878181580 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfe4003930 con 0x7fdfd8003a30 2024-09-21T07:43:39.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.050+0000 7fdff0c00640 1 -- 172.21.15.39:0/3878181580 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4038070 con 0x7fdfd8003a30 2024-09-21T07:43:39.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.050+0000 7fdff9a00640 1 -- 172.21.15.39:0/3878181580 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdff41f98a0 con 0x7fdfd8003a30 2024-09-21T07:43:39.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.050+0000 7fdff9a00640 1 -- 172.21.15.39:0/3878181580 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdff40d2ce0 con 0x7fdfd8003a30 2024-09-21T07:43:39.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.053+0000 7fdff0c00640 1 -- 172.21.15.39:0/3878181580 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdfe4030070 con 0x7fdfd8003a30 2024-09-21T07:43:39.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.054+0000 7fdff0c00640 1 --2- 172.21.15.39:0/3878181580 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdfd0078b90 0x7fdfd007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:39.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.054+0000 7fdff0c00640 1 -- 172.21.15.39:0/3878181580 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8205+0+0 (secure 0 0 0) 0x7fdfe40bab60 con 0x7fdfd8003a30 2024-09-21T07:43:39.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.054+0000 7fdfebe00640 1 --2- 172.21.15.39:0/3878181580 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdfd0078b90 0x7fdfd007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:39.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.054+0000 7fe001fca740 1 Processor -- start 2024-09-21T07:43:39.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.055+0000 7fe001fca740 1 -- start start 2024-09-21T07:43:39.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.055+0000 7fdfebe00640 1 --2- 172.21.15.39:0/3878181580 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdfd0078b90 0x7fdfd007b0a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc001010 tx=0x7fdfdc0067a0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:39.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.056+0000 7fe001fca740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600626d9270 0x560062aa9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:39.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.056+0000 7fe001fca740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600626b03b0 0x560062aaa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:39.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.056+0000 7fe001fca740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5600626b8590 0x560062a7a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:39.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.056+0000 7fe001fca740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560062a03430 con 0x5600626b03b0 2024-09-21T07:43:39.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.056+0000 7fe001fca740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5600629f35d0 con 0x5600626d9270 2024-09-21T07:43:39.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.056+0000 7fe001fca740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x5600629e2a00 con 0x5600626b8590 2024-09-21T07:43:39.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.056+0000 7fdfebe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600626b03b0 0x560062aaa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:39.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.056+0000 7fdfebe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600626b03b0 0x560062aaa0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50358/0 (socket says 172.21.15.39:50358) 2024-09-21T07:43:39.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.056+0000 7fdfebe00640 1 -- 172.21.15.39:0/3413548558 learned_addr learned my addr 172.21.15.39:0/3413548558 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:39.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.057+0000 7fdff3e00640 1 --2- 172.21.15.39:0/3413548558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600626d9270 0x560062aa9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:39.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.057+0000 7fdff9000640 1 --2- 172.21.15.39:0/3413548558 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5600626b8590 0x560062a7a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:39.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.057+0000 7fdfebe00640 1 -- 172.21.15.39:0/3413548558 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5600626b8590 msgr2=0x560062a7a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:43:39.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.057+0000 7fdfebe00640 1 --2- 172.21.15.39:0/3413548558 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5600626b8590 0x560062a7a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.057+0000 7fdfebe00640 1 -- 172.21.15.39:0/3413548558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600626d9270 msgr2=0x560062aa9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:43:39.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.057+0000 7fdfebe00640 1 --2- 172.21.15.39:0/3413548558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600626d9270 0x560062aa9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.057+0000 7fdfebe00640 1 -- 172.21.15.39:0/3413548558 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600629ab520 con 0x5600626b03b0 2024-09-21T07:43:39.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.057+0000 7fdfebe00640 1 --2- 172.21.15.39:0/3413548558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600626b03b0 0x560062aaa0a0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc004ab0 tx=0x7fdfdc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71acd9114f0c7068 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:39.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.057+0000 7fdfe8c00640 1 -- 172.21.15.39:0/3413548558 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc012a80 con 0x5600626b03b0 2024-09-21T07:43:39.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.057+0000 7fdfe8c00640 1 -- 172.21.15.39:0/3413548558 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfdc023020 con 0x5600626b03b0 2024-09-21T07:43:39.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.057+0000 7fdfe8c00640 1 -- 172.21.15.39:0/3413548558 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc012c80 con 0x5600626b03b0 2024-09-21T07:43:39.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.058+0000 7fe001fca740 1 -- 172.21.15.39:0/3413548558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600626b03b0 msgr2=0x560062aaa0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:39.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.058+0000 7fe001fca740 1 --2- 172.21.15.39:0/3413548558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600626b03b0 0x560062aaa0a0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc004ab0 tx=0x7fdfdc015000 comp rx=0 tx=0).stop 2024-09-21T07:43:39.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.058+0000 7fe001fca740 1 -- 172.21.15.39:0/3413548558 shutdown_connections 2024-09-21T07:43:39.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.058+0000 7fe001fca740 1 --2- 172.21.15.39:0/3413548558 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5600626b8590 0x560062a7a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.058+0000 7fe001fca740 1 --2- 172.21.15.39:0/3413548558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600626b03b0 0x560062aaa0a0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.058+0000 7fe001fca740 1 --2- 172.21.15.39:0/3413548558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600626d9270 0x560062aa9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.058+0000 7fe001fca740 1 -- 172.21.15.39:0/3413548558 >> 172.21.15.39:0/3413548558 conn(0x5600626cd3c0 msgr2=0x560062a7fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:39.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.058+0000 7fe001fca740 1 -- 172.21.15.39:0/3413548558 shutdown_connections 2024-09-21T07:43:39.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.059+0000 7fe001fca740 1 -- 172.21.15.39:0/3413548558 wait complete. 2024-09-21T07:43:39.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.060+0000 7fe001fca740 1 -- start start 2024-09-21T07:43:39.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.060+0000 7fe001fca740 1 Processor -- start 2024-09-21T07:43:39.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.061+0000 7fe001fca740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5600626b03b0 0x560062abef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:39.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.061+0000 7fe001fca740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600626b8590 0x560062ad7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:39.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.061+0000 7fe001fca740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600626d9270 0x560062a9ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:39.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.061+0000 7fe001fca740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560062989330 con 0x5600626b8590 2024-09-21T07:43:39.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.061+0000 7fe001fca740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5600629e6810 con 0x5600626d9270 2024-09-21T07:43:39.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.061+0000 7fe001fca740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x560062753620 con 0x5600626b03b0 2024-09-21T07:43:39.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.062+0000 7fe001fca740 10 client.0 _ll_register_callbacks cb 0x56006298e420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:43:39.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.062+0000 7fdfebe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600626b8590 0x560062ad7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:39.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.062+0000 7fdfebe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600626b8590 0x560062ad7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50370/0 (socket says 172.21.15.39:50370) 2024-09-21T07:43:39.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.062+0000 7fdfebe00640 1 -- 172.21.15.39:0/3665204429 learned_addr learned my addr 172.21.15.39:0/3665204429 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:39.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.062+0000 7fdff9000640 1 --2- 172.21.15.39:0/3665204429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600626d9270 0x560062a9ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:39.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.062+0000 7fdff3e00640 1 --2- 172.21.15.39:0/3665204429 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5600626b03b0 0x560062abef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:39.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.062+0000 7fdfebe00640 1 -- 172.21.15.39:0/3665204429 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5600626b03b0 msgr2=0x560062abef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:43:39.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.062+0000 7fdfebe00640 1 --2- 172.21.15.39:0/3665204429 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5600626b03b0 0x560062abef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.062+0000 7fdfebe00640 1 -- 172.21.15.39:0/3665204429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600626d9270 msgr2=0x560062a9ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:43:39.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.062+0000 7fdfebe00640 1 --2- 172.21.15.39:0/3665204429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600626d9270 0x560062a9ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.062+0000 7fdfebe00640 1 -- 172.21.15.39:0/3665204429 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560062744850 con 0x5600626b8590 2024-09-21T07:43:39.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.062+0000 7fdfebe00640 1 --2- 172.21.15.39:0/3665204429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600626b8590 0x560062ad7700 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc020000 tx=0x7fdfdc01e620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f40857d8cf87f52 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:39.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.062+0000 7fdfe1600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:43:39.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.063+0000 7fdfe1600640 1 -- 172.21.15.39:0/3665204429 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc022050 con 0x5600626b8590 2024-09-21T07:43:39.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.063+0000 7fdfe1600640 1 -- 172.21.15.39:0/3665204429 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8205+0+0 (secure 0 0 0) 0x7fdfdc01ec20 con 0x5600626b8590 2024-09-21T07:43:39.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.063+0000 7fe001fca740 1 -- 172.21.15.39:0/3665204429 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfdc023020 con 0x5600626b8590 2024-09-21T07:43:39.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.063+0000 7fdfe1600640 1 -- 172.21.15.39:0/3665204429 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=150) ==== 24+0+0 (secure 0 0 0) 0x7fdfdc013e50 con 0x5600626b8590 2024-09-21T07:43:39.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.063+0000 7fe001fca740 10 client.8103 fetch_fsmap learned FSMap version 150 2024-09-21T07:43:39.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.063+0000 7fe001fca740 1 -- 172.21.15.39:0/3665204429 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=150,monmap=2+}) -- 0x5600627bb6b0 con 0x5600626b8590 2024-09-21T07:43:39.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.063+0000 7fdfe1600640 1 -- 172.21.15.39:0/3665204429 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfdc023020 con 0x5600626b8590 2024-09-21T07:43:39.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.063+0000 7fdfe1600640 1 -- 172.21.15.39:0/3665204429 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 150) ==== 3522+0+0 (secure 0 0 0) 0x7fdfdc03c070 con 0x5600626b8590 2024-09-21T07:43:39.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.063+0000 7fe001fca740 10 client.8103 fetch_fsmap finished waiting for FSMap version 150 2024-09-21T07:43:39.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.063+0000 7fe001fca740 10 client.8103 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:43:39.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.063+0000 7fe001fca740 20 client.8103 populate_metadata read hostname 'smithi039' 2024-09-21T07:43:39.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.064+0000 7fe001fca740 1 --2- 172.21.15.39:0/3665204429 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x560062697e40 0x560062aa5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:39.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.064+0000 7fe001fca740 4 client.8103 mds_command: new command op to 7311 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:43:39.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.064+0000 7fe001fca740 1 -- 172.21.15.39:0/3665204429 --> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600629f5520 con 0x560062697e40 2024-09-21T07:43:39.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.064+0000 7fdff3e00640 1 --2- 172.21.15.39:0/3665204429 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x560062697e40 0x560062aa5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:39.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.064+0000 7fdfe1600640 10 client.8103 ms_handle_connect on v2:172.21.15.39:6832/1790581246 2024-09-21T07:43:39.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.064+0000 7fdff3e00640 1 --2- 172.21.15.39:0/3665204429 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x560062697e40 0x560062aa5f30 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b355247f684c907f server_cookie=1bbf20cdd30943a9 in_seq=0 out_seq=0 2024-09-21T07:43:39.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.065+0000 7fdfe1600640 1 -- 172.21.15.39:0/3665204429 <== mds.0 v2:172.21.15.39:6832/1790581246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5600629f5520 con 0x560062697e40 2024-09-21T07:43:39.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.065+0000 7fdfe1600640 10 client.8103 handle_command_reply: tid=0 2024-09-21T07:43:39.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.065+0000 7fe001fca740 1 client.8103 shutdown 2024-09-21T07:43:39.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.065+0000 7fe001fca740 1 -- 172.21.15.39:0/3665204429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600626b8590 msgr2=0x560062ad7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:39.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.065+0000 7fe001fca740 1 --2- 172.21.15.39:0/3665204429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600626b8590 0x560062ad7700 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc020000 tx=0x7fdfdc01e620 comp rx=0 tx=0).stop 2024-09-21T07:43:39.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.065+0000 7fe001fca740 1 -- 172.21.15.39:0/3665204429 shutdown_connections 2024-09-21T07:43:39.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.065+0000 7fe001fca740 1 --2- 172.21.15.39:0/3665204429 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x560062697e40 0x560062aa5f30 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.065+0000 7fe001fca740 1 --2- 172.21.15.39:0/3665204429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600626d9270 0x560062a9ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.065+0000 7fe001fca740 1 --2- 172.21.15.39:0/3665204429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600626b8590 0x560062ad7700 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.065+0000 7fdfe1600640 0 client.8103 ms_handle_reset on v2:172.21.15.39:6832/1790581246 2024-09-21T07:43:39.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.065+0000 7fe001fca740 1 --2- 172.21.15.39:0/3665204429 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5600626b03b0 0x560062abef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.065+0000 7fe001fca740 1 -- 172.21.15.39:0/3665204429 >> 172.21.15.39:0/3665204429 conn(0x5600626cd3c0 msgr2=0x560062ac7700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:39.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.066+0000 7fe001fca740 1 -- 172.21.15.39:0/3665204429 shutdown_connections 2024-09-21T07:43:39.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.066+0000 7fe001fca740 1 -- 172.21.15.39:0/3665204429 wait complete. 2024-09-21T07:43:39.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.066+0000 7fe001fca740 20 client.8103 trim_cache size 0 max 16384 2024-09-21T07:43:39.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.085+0000 7fe001fca740 1 Processor -- start 2024-09-21T07:43:39.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.085+0000 7fe001fca740 1 -- start start 2024-09-21T07:43:39.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.087+0000 7fe001fca740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5600626b03b0 0x560062a9fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:39.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.087+0000 7fe001fca740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600626b8590 0x560062ac44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:39.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.087+0000 7fe001fca740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600626d9270 0x560062ac49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:39.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.087+0000 7fe001fca740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdff4205c10 con 0x5600626d9270 2024-09-21T07:43:39.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.087+0000 7fe001fca740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdff4203f60 con 0x5600626b8590 2024-09-21T07:43:39.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.087+0000 7fe001fca740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fdff4204550 con 0x5600626b03b0 2024-09-21T07:43:39.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.087+0000 7fdff9000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600626d9270 0x560062ac49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:39.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.087+0000 7fdff9000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600626d9270 0x560062ac49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50386/0 (socket says 172.21.15.39:50386) 2024-09-21T07:43:39.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.087+0000 7fdff9000640 1 -- 172.21.15.39:0/468992875 learned_addr learned my addr 172.21.15.39:0/468992875 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:39.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.087+0000 7fdff3e00640 1 --2- 172.21.15.39:0/468992875 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5600626b03b0 0x560062a9fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:39.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.087+0000 7fdff9000640 1 -- 172.21.15.39:0/468992875 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5600626b03b0 msgr2=0x560062a9fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:43:39.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.087+0000 7fdff9000640 1 --2- 172.21.15.39:0/468992875 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5600626b03b0 0x560062a9fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.087+0000 7fdfebe00640 1 --2- 172.21.15.39:0/468992875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600626b8590 0x560062ac44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:39.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.087+0000 7fdff9000640 1 -- 172.21.15.39:0/468992875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600626b8590 msgr2=0x560062ac44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:43:39.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.087+0000 7fdff9000640 1 --2- 172.21.15.39:0/468992875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600626b8590 0x560062ac44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.087+0000 7fdff9000640 1 -- 172.21.15.39:0/468992875 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560062aa4a80 con 0x5600626d9270 2024-09-21T07:43:39.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.087+0000 7fdfebe00640 1 --2- 172.21.15.39:0/468992875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600626b8590 0x560062ac44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:43:39.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.088+0000 7fdff9000640 1 --2- 172.21.15.39:0/468992875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600626d9270 0x560062ac49f0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fdfec0104e0 tx=0x7fdfec009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=414884067a86854b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:39.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.088+0000 7fdfe3e00640 1 -- 172.21.15.39:0/468992875 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfec003b70 con 0x5600626d9270 2024-09-21T07:43:39.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.088+0000 7fdfe3e00640 1 -- 172.21.15.39:0/468992875 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfec003d10 con 0x5600626d9270 2024-09-21T07:43:39.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.088+0000 7fdfe3e00640 1 -- 172.21.15.39:0/468992875 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfec004230 con 0x5600626d9270 2024-09-21T07:43:39.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.088+0000 7fe001fca740 1 -- 172.21.15.39:0/468992875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600626d9270 msgr2=0x560062ac49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:39.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.088+0000 7fe001fca740 1 --2- 172.21.15.39:0/468992875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600626d9270 0x560062ac49f0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fdfec0104e0 tx=0x7fdfec009500 comp rx=0 tx=0).stop 2024-09-21T07:43:39.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.089+0000 7fe001fca740 1 -- 172.21.15.39:0/468992875 shutdown_connections 2024-09-21T07:43:39.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.089+0000 7fe001fca740 1 --2- 172.21.15.39:0/468992875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600626d9270 0x560062ac49f0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.089+0000 7fe001fca740 1 --2- 172.21.15.39:0/468992875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600626b8590 0x560062ac44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.089+0000 7fe001fca740 1 --2- 172.21.15.39:0/468992875 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5600626b03b0 0x560062a9fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.089+0000 7fe001fca740 1 -- 172.21.15.39:0/468992875 >> 172.21.15.39:0/468992875 conn(0x5600626cd3c0 msgr2=0x560062a7d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:39.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.089+0000 7fe001fca740 1 -- 172.21.15.39:0/468992875 shutdown_connections 2024-09-21T07:43:39.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.089+0000 7fe001fca740 1 -- 172.21.15.39:0/468992875 wait complete. 2024-09-21T07:43:39.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.090+0000 7fe001fca740 1 -- start start 2024-09-21T07:43:39.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.090+0000 7fe001fca740 1 Processor -- start 2024-09-21T07:43:39.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.091+0000 7fe001fca740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600626b03b0 0x560062b48740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:39.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.091+0000 7fe001fca740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5600626b8590 0x560062ae6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:39.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.091+0000 7fe001fca740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600626d9270 0x560062b3acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:39.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.091+0000 7fe001fca740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560062aa33a0 con 0x5600626d9270 2024-09-21T07:43:39.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.091+0000 7fe001fca740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560062aa05d0 con 0x5600626b03b0 2024-09-21T07:43:39.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.091+0000 7fe001fca740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x560062aa39f0 con 0x5600626b8590 2024-09-21T07:43:39.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.091+0000 7fe001fca740 10 client.0 _ll_register_callbacks cb 0x7fdfdc006f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:43:39.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.091+0000 7fdff9000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600626d9270 0x560062b3acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:39.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.092+0000 7fdff9000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600626d9270 0x560062b3acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50402/0 (socket says 172.21.15.39:50402) 2024-09-21T07:43:39.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.092+0000 7fdff9000640 1 -- 172.21.15.39:0/1977831326 learned_addr learned my addr 172.21.15.39:0/1977831326 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:39.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.092+0000 7fdfebe00640 1 --2- 172.21.15.39:0/1977831326 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5600626b8590 0x560062ae6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:39.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.092+0000 7fdff3e00640 1 --2- 172.21.15.39:0/1977831326 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600626b03b0 0x560062b48740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:39.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.092+0000 7fdff9000640 1 -- 172.21.15.39:0/1977831326 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5600626b8590 msgr2=0x560062ae6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:43:39.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.092+0000 7fdff9000640 1 --2- 172.21.15.39:0/1977831326 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5600626b8590 0x560062ae6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.092+0000 7fdff9000640 1 -- 172.21.15.39:0/1977831326 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600626b03b0 msgr2=0x560062b48740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:43:39.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.092+0000 7fdff9000640 1 --2- 172.21.15.39:0/1977831326 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600626b03b0 0x560062b48740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.092+0000 7fdff9000640 1 -- 172.21.15.39:0/1977831326 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560062b48c80 con 0x5600626d9270 2024-09-21T07:43:39.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.092+0000 7fdff9000640 1 --2- 172.21.15.39:0/1977831326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600626d9270 0x560062b3acb0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fdfec005fd0 tx=0x7fdfec009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=791521d0d06dccc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:39.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.092+0000 7fdfe1600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:43:39.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.093+0000 7fdfe1600640 1 -- 172.21.15.39:0/1977831326 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfec0045f0 con 0x5600626d9270 2024-09-21T07:43:39.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.093+0000 7fdfe1600640 1 -- 172.21.15.39:0/1977831326 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8205+0+0 (secure 0 0 0) 0x7fdfec004790 con 0x5600626d9270 2024-09-21T07:43:39.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.093+0000 7fe001fca740 1 -- 172.21.15.39:0/1977831326 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfec003d10 con 0x5600626d9270 2024-09-21T07:43:39.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.093+0000 7fdfe1600640 1 -- 172.21.15.39:0/1977831326 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=150) ==== 24+0+0 (secure 0 0 0) 0x7fdfec033470 con 0x5600626d9270 2024-09-21T07:43:39.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.093+0000 7fe001fca740 10 client.8109 fetch_fsmap learned FSMap version 150 2024-09-21T07:43:39.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.093+0000 7fe001fca740 1 -- 172.21.15.39:0/1977831326 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=150,monmap=2+}) -- 0x560062aa1600 con 0x5600626d9270 2024-09-21T07:43:39.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.093+0000 7fdfe1600640 1 -- 172.21.15.39:0/1977831326 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfec003d10 con 0x5600626d9270 2024-09-21T07:43:39.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.093+0000 7fdfe1600640 1 -- 172.21.15.39:0/1977831326 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 150) ==== 3522+0+0 (secure 0 0 0) 0x7fdfec033610 con 0x5600626d9270 2024-09-21T07:43:39.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.093+0000 7fe001fca740 10 client.8109 fetch_fsmap finished waiting for FSMap version 150 2024-09-21T07:43:39.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.093+0000 7fe001fca740 10 client.8109 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:43:39.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.093+0000 7fe001fca740 20 client.8109 populate_metadata read hostname 'smithi039' 2024-09-21T07:43:39.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.093+0000 7fe001fca740 1 --2- 172.21.15.39:0/1977831326 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x560062b44f10 0x560062aa5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:39.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.093+0000 7fe001fca740 4 client.8109 mds_command: new command op to 7311 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:43:39.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.093+0000 7fe001fca740 1 -- 172.21.15.39:0/1977831326 --> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] -- command(tid 0: {"prefix": "status"}) -- 0x5600629f0fd0 con 0x560062b44f10 2024-09-21T07:43:39.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.094+0000 7fdff3e00640 1 --2- 172.21.15.39:0/1977831326 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x560062b44f10 0x560062aa5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:39.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.094+0000 7fdff3e00640 1 --2- 172.21.15.39:0/1977831326 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x560062b44f10 0x560062aa5c70 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd2272cfccb86f52 server_cookie=27a2365cf6b5f9d6 in_seq=0 out_seq=0 2024-09-21T07:43:39.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.094+0000 7fdfe1600640 10 client.8109 ms_handle_connect on v2:172.21.15.39:6832/1790581246 2024-09-21T07:43:39.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.094+0000 7fdfe1600640 1 -- 172.21.15.39:0/1977831326 <== mds.0 v2:172.21.15.39:6832/1790581246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x5600629f0fd0 con 0x560062b44f10 2024-09-21T07:43:39.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.094+0000 7fdfe1600640 10 client.8109 handle_command_reply: tid=0 2024-09-21T07:43:39.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.094+0000 7fe001fca740 1 client.8109 shutdown 2024-09-21T07:43:39.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.095+0000 7fe001fca740 1 -- 172.21.15.39:0/1977831326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600626d9270 msgr2=0x560062b3acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:39.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.095+0000 7fe001fca740 1 --2- 172.21.15.39:0/1977831326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600626d9270 0x560062b3acb0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fdfec005fd0 tx=0x7fdfec009e90 comp rx=0 tx=0).stop 2024-09-21T07:43:39.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.095+0000 7fe001fca740 1 -- 172.21.15.39:0/1977831326 shutdown_connections 2024-09-21T07:43:39.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.095+0000 7fe001fca740 1 --2- 172.21.15.39:0/1977831326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5600626d9270 0x560062b3acb0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.095+0000 7fe001fca740 1 --2- 172.21.15.39:0/1977831326 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5600626b8590 0x560062ae6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.095+0000 7fe001fca740 1 --2- 172.21.15.39:0/1977831326 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x560062b44f10 0x560062aa5c70 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.095+0000 7fe001fca740 1 --2- 172.21.15.39:0/1977831326 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5600626b03b0 0x560062b48740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.095+0000 7fdfe1600640 0 client.8109 ms_handle_reset on v2:172.21.15.39:6832/1790581246 2024-09-21T07:43:39.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.095+0000 7fe001fca740 1 -- 172.21.15.39:0/1977831326 >> 172.21.15.39:0/1977831326 conn(0x5600626cd3c0 msgr2=0x560062a757c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:39.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.095+0000 7fe001fca740 1 -- 172.21.15.39:0/1977831326 shutdown_connections 2024-09-21T07:43:39.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.095+0000 7fe001fca740 1 -- 172.21.15.39:0/1977831326 wait complete. 2024-09-21T07:43:39.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.095+0000 7fe001fca740 20 client.8109 trim_cache size 0 max 16384 2024-09-21T07:43:39.102 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:43:39.102 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:43:39.102 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-09-21T07:43:39.102 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 7311, 2024-09-21T07:43:39.102 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:43:39.102 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:43:39.102 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:43:39.102 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 161.71033276399999, 2024-09-21T07:43:39.102 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 150, 2024-09-21T07:43:39.103 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 117, 2024-09-21T07:43:39.103 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 117, 2024-09-21T07:43:39.103 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 210.21778133500001 2024-09-21T07:43:39.103 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:43:39.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.096+0000 7fdff9a00640 1 -- 172.21.15.39:0/3878181580 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdfd0078b90 msgr2=0x7fdfd007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:39.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.096+0000 7fdff9a00640 1 --2- 172.21.15.39:0/3878181580 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdfd0078b90 0x7fdfd007b0a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc001010 tx=0x7fdfdc0067a0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.096+0000 7fdff9a00640 1 -- 172.21.15.39:0/3878181580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfd8003a30 msgr2=0x7fdff40cb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:39.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.096+0000 7fdff9a00640 1 --2- 172.21.15.39:0/3878181580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfd8003a30 0x7fdff40cb630 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fdfe402a910 tx=0x7fdfe4006310 comp rx=0 tx=0).stop 2024-09-21T07:43:39.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.096+0000 7fdff9a00640 1 -- 172.21.15.39:0/3878181580 shutdown_connections 2024-09-21T07:43:39.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.096+0000 7fdff9a00640 1 --2- 172.21.15.39:0/3878181580 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdfd0078b90 0x7fdfd007b0a0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.096+0000 7fdff9a00640 1 --2- 172.21.15.39:0/3878181580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdff40cd050 0x7fdff40cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.096+0000 7fdff9a00640 1 --2- 172.21.15.39:0/3878181580 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdff41f8f50 0x7fdff41f9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.096+0000 7fdff9a00640 1 --2- 172.21.15.39:0/3878181580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfd8003a30 0x7fdff40cb630 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.096+0000 7fdff9a00640 1 -- 172.21.15.39:0/3878181580 >> 172.21.15.39:0/3878181580 conn(0x7fdff408bfb0 msgr2=0x7fdff40cecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:39.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.096+0000 7fdff9a00640 1 -- 172.21.15.39:0/3878181580 shutdown_connections 2024-09-21T07:43:39.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.096+0000 7fdff9a00640 1 -- 172.21.15.39:0/3878181580 wait complete. 2024-09-21T07:43:39.123 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-21T07:43:39.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.350+0000 7fa381c00640 1 -- 172.21.15.39:0/3791227907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa37c0d0620 msgr2=0x7fa37c0d0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:39.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.350+0000 7fa381c00640 1 --2- 172.21.15.39:0/3791227907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa37c0d0620 0x7fa37c0d0a30 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fa37c06de00 tx=0x7fa37402a820 comp rx=0 tx=0).stop 2024-09-21T07:43:39.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.350+0000 7fa381c00640 1 -- 172.21.15.39:0/3791227907 shutdown_connections 2024-09-21T07:43:39.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.350+0000 7fa381c00640 1 --2- 172.21.15.39:0/3791227907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa37c0d0620 0x7fa37c0d0a30 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.350+0000 7fa381c00640 1 --2- 172.21.15.39:0/3791227907 >> v2:172.21.15.148:3300/0 conn(0x7fa37c0cfd10 0x7fa37c0cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.350+0000 7fa381c00640 1 -- 172.21.15.39:0/3791227907 >> 172.21.15.39:0/3791227907 conn(0x7fa37c06cfc0 msgr2=0x7fa37c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:39.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.350+0000 7fa381c00640 1 -- 172.21.15.39:0/3791227907 shutdown_connections 2024-09-21T07:43:39.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.350+0000 7fa381c00640 1 -- 172.21.15.39:0/3791227907 wait complete. 2024-09-21T07:43:39.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.352+0000 7fa381c00640 1 Processor -- start 2024-09-21T07:43:39.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.353+0000 7fa381c00640 1 -- start start 2024-09-21T07:43:39.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.354+0000 7fa381c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa37c0cfd10 0x7fa37c159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:39.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.354+0000 7fa381c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa37c0d0620 0x7fa37c15b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:39.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.354+0000 7fa381c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa37c15cd30 0x7fa37c15b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:39.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.354+0000 7fa381c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c073710 con 0x7fa37c15cd30 2024-09-21T07:43:39.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.354+0000 7fa381c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c0ce030 con 0x7fa37c0d0620 2024-09-21T07:43:39.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.354+0000 7fa381c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa37c073890 con 0x7fa37c0cfd10 2024-09-21T07:43:39.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.354+0000 7fa37b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa37c15cd30 0x7fa37c15b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:39.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.354+0000 7fa37b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa37c15cd30 0x7fa37c15b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50412/0 (socket says 172.21.15.39:50412) 2024-09-21T07:43:39.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.355+0000 7fa37b400640 1 -- 172.21.15.39:0/2445326518 learned_addr learned my addr 172.21.15.39:0/2445326518 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:39.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.355+0000 7fa373e00640 1 --2- 172.21.15.39:0/2445326518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa37c0d0620 0x7fa37c15b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:39.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.355+0000 7fa37aa00640 1 --2- 172.21.15.39:0/2445326518 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa37c0cfd10 0x7fa37c159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:39.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.355+0000 7fa37b400640 1 -- 172.21.15.39:0/2445326518 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa37c0cfd10 msgr2=0x7fa37c159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:39.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.355+0000 7fa37b400640 1 --2- 172.21.15.39:0/2445326518 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa37c0cfd10 0x7fa37c159640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.355+0000 7fa37b400640 1 -- 172.21.15.39:0/2445326518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa37c0d0620 msgr2=0x7fa37c15b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:39.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.355+0000 7fa37b400640 1 --2- 172.21.15.39:0/2445326518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa37c0d0620 0x7fa37c15b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.355+0000 7fa37b400640 1 -- 172.21.15.39:0/2445326518 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa37c1f9130 con 0x7fa37c15cd30 2024-09-21T07:43:39.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.355+0000 7fa37aa00640 1 --2- 172.21.15.39:0/2445326518 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa37c0cfd10 0x7fa37c159640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:43:39.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.355+0000 7fa37b400640 1 --2- 172.21.15.39:0/2445326518 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa37c15cd30 0x7fa37c15b800 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fa37402ae10 tx=0x7fa374032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:39.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.356+0000 7fa372a00640 1 -- 172.21.15.39:0/2445326518 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa374037040 con 0x7fa37c15cd30 2024-09-21T07:43:39.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.356+0000 7fa372a00640 1 -- 172.21.15.39:0/2445326518 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa374042070 con 0x7fa37c15cd30 2024-09-21T07:43:39.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.356+0000 7fa372a00640 1 -- 172.21.15.39:0/2445326518 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3740042a0 con 0x7fa37c15cd30 2024-09-21T07:43:39.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.356+0000 7fa381c00640 1 -- 172.21.15.39:0/2445326518 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa37c1f9410 con 0x7fa37c15cd30 2024-09-21T07:43:39.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.356+0000 7fa381c00640 1 -- 172.21.15.39:0/2445326518 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa37c1f99d0 con 0x7fa37c15cd30 2024-09-21T07:43:39.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.359+0000 7fa372a00640 1 -- 172.21.15.39:0/2445326518 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa374004440 con 0x7fa37c15cd30 2024-09-21T07:43:39.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.360+0000 7fa372a00640 1 --2- 172.21.15.39:0/2445326518 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa358078b90 0x7fa35807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:39.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.360+0000 7fa372a00640 1 -- 172.21.15.39:0/2445326518 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8205+0+0 (secure 0 0 0) 0x7fa3740bc5f0 con 0x7fa37c15cd30 2024-09-21T07:43:39.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.360+0000 7fa388c7d740 1 Processor -- start 2024-09-21T07:43:39.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.361+0000 7fa388c7d740 1 -- start start 2024-09-21T07:43:39.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.361+0000 7fa37aa00640 1 --2- 172.21.15.39:0/2445326518 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa358078b90 0x7fa35807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:39.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.361+0000 7fa37aa00640 1 --2- 172.21.15.39:0/2445326518 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa358078b90 0x7fa35807b0a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa36c000f60 tx=0x7fa36c009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:39.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.362+0000 7fa388c7d740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558c13c51270 0x558c14021970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:39.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.362+0000 7fa388c7d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c13c283b0 0x558c14021eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:39.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.362+0000 7fa388c7d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c13c30590 0x558c13ff1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:39.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.362+0000 7fa388c7d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558c13f7b430 con 0x558c13c283b0 2024-09-21T07:43:39.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.362+0000 7fa388c7d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558c13f6b5d0 con 0x558c13c30590 2024-09-21T07:43:39.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.362+0000 7fa388c7d740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x558c13f5aa00 con 0x558c13c51270 2024-09-21T07:43:39.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.362+0000 7fa373e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c13c283b0 0x558c14021eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:39.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.362+0000 7fa373e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c13c283b0 0x558c14021eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50422/0 (socket says 172.21.15.39:50422) 2024-09-21T07:43:39.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.362+0000 7fa373e00640 1 -- 172.21.15.39:0/562132993 learned_addr learned my addr 172.21.15.39:0/562132993 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:39.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.363+0000 7fa37b400640 1 --2- 172.21.15.39:0/562132993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c13c30590 0x558c13ff1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:39.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.363+0000 7fa37aa00640 1 --2- 172.21.15.39:0/562132993 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558c13c51270 0x558c14021970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:39.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.363+0000 7fa373e00640 1 -- 172.21.15.39:0/562132993 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558c13c51270 msgr2=0x558c14021970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:43:39.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.363+0000 7fa373e00640 1 --2- 172.21.15.39:0/562132993 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558c13c51270 0x558c14021970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.363+0000 7fa373e00640 1 -- 172.21.15.39:0/562132993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c13c30590 msgr2=0x558c13ff1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:43:39.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.363+0000 7fa373e00640 1 --2- 172.21.15.39:0/562132993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c13c30590 0x558c13ff1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.363+0000 7fa373e00640 1 -- 172.21.15.39:0/562132993 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c13f23520 con 0x558c13c283b0 2024-09-21T07:43:39.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.363+0000 7fa373e00640 1 --2- 172.21.15.39:0/562132993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c13c283b0 0x558c14021eb0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fa36400bba0 tx=0x7fa3640107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26bce689dc93ab47 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:39.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.363+0000 7fa36b400640 1 -- 172.21.15.39:0/562132993 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa364004030 con 0x558c13c283b0 2024-09-21T07:43:39.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.363+0000 7fa36b400640 1 -- 172.21.15.39:0/562132993 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3640042c0 con 0x558c13c283b0 2024-09-21T07:43:39.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.363+0000 7fa36b400640 1 -- 172.21.15.39:0/562132993 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3640035f0 con 0x558c13c283b0 2024-09-21T07:43:39.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.364+0000 7fa388c7d740 1 -- 172.21.15.39:0/562132993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c13c283b0 msgr2=0x558c14021eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:39.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.364+0000 7fa388c7d740 1 --2- 172.21.15.39:0/562132993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c13c283b0 0x558c14021eb0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fa36400bba0 tx=0x7fa3640107a0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.364+0000 7fa388c7d740 1 -- 172.21.15.39:0/562132993 shutdown_connections 2024-09-21T07:43:39.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.364+0000 7fa388c7d740 1 --2- 172.21.15.39:0/562132993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c13c30590 0x558c13ff1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.364+0000 7fa388c7d740 1 --2- 172.21.15.39:0/562132993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c13c283b0 0x558c14021eb0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.364+0000 7fa388c7d740 1 --2- 172.21.15.39:0/562132993 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558c13c51270 0x558c14021970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.364+0000 7fa388c7d740 1 -- 172.21.15.39:0/562132993 >> 172.21.15.39:0/562132993 conn(0x558c13c453c0 msgr2=0x558c13ff7970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:39.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.364+0000 7fa388c7d740 1 -- 172.21.15.39:0/562132993 shutdown_connections 2024-09-21T07:43:39.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.365+0000 7fa388c7d740 1 -- 172.21.15.39:0/562132993 wait complete. 2024-09-21T07:43:39.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.365+0000 7fa388c7d740 1 -- start start 2024-09-21T07:43:39.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.366+0000 7fa388c7d740 1 Processor -- start 2024-09-21T07:43:39.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.367+0000 7fa388c7d740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558c13c283b0 0x558c14017ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:39.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.367+0000 7fa388c7d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c13c30590 0x558c14019b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:39.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.367+0000 7fa388c7d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c13c51270 0x558c1401a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:39.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.367+0000 7fa388c7d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558c13f01330 con 0x558c13c30590 2024-09-21T07:43:39.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.367+0000 7fa388c7d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558c13f5e810 con 0x558c13c51270 2024-09-21T07:43:39.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.367+0000 7fa388c7d740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x558c13ccb620 con 0x558c13c283b0 2024-09-21T07:43:39.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.367+0000 7fa388c7d740 10 client.0 _ll_register_callbacks cb 0x558c13f06420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:43:39.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.367+0000 7fa373e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c13c30590 0x558c14019b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:39.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.368+0000 7fa373e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c13c30590 0x558c14019b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50436/0 (socket says 172.21.15.39:50436) 2024-09-21T07:43:39.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.368+0000 7fa373e00640 1 -- 172.21.15.39:0/3299862111 learned_addr learned my addr 172.21.15.39:0/3299862111 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:39.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.368+0000 7fa37aa00640 1 --2- 172.21.15.39:0/3299862111 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558c13c283b0 0x558c14017ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:39.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.368+0000 7fa37b400640 1 --2- 172.21.15.39:0/3299862111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c13c51270 0x558c1401a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:39.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.368+0000 7fa373e00640 1 -- 172.21.15.39:0/3299862111 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558c13c283b0 msgr2=0x558c14017ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:43:39.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.368+0000 7fa373e00640 1 --2- 172.21.15.39:0/3299862111 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558c13c283b0 0x558c14017ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.368+0000 7fa373e00640 1 -- 172.21.15.39:0/3299862111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c13c51270 msgr2=0x558c1401a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:43:39.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.368+0000 7fa373e00640 1 --2- 172.21.15.39:0/3299862111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c13c51270 0x558c1401a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.368+0000 7fa373e00640 1 -- 172.21.15.39:0/3299862111 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c13cbc850 con 0x558c13c30590 2024-09-21T07:43:39.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.368+0000 7fa37aa00640 1 --2- 172.21.15.39:0/3299862111 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558c13c283b0 0x558c14017ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:43:39.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.368+0000 7fa373e00640 1 --2- 172.21.15.39:0/3299862111 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c13c30590 0x558c14019b10 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fa364011010 tx=0x7fa364004130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10dac351512f7d00 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:39.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.368+0000 7fa35fe00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:43:39.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.368+0000 7fa35fe00640 1 -- 172.21.15.39:0/3299862111 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36402b060 con 0x558c13c30590 2024-09-21T07:43:39.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.368+0000 7fa35fe00640 1 -- 172.21.15.39:0/3299862111 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8205+0+0 (secure 0 0 0) 0x7fa3640037f0 con 0x558c13c30590 2024-09-21T07:43:39.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.368+0000 7fa388c7d740 1 -- 172.21.15.39:0/3299862111 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3640042c0 con 0x558c13c30590 2024-09-21T07:43:39.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.369+0000 7fa35fe00640 1 -- 172.21.15.39:0/3299862111 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=150) ==== 24+0+0 (secure 0 0 0) 0x7fa36400bea0 con 0x558c13c30590 2024-09-21T07:43:39.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.369+0000 7fa388c7d740 10 client.8124 fetch_fsmap learned FSMap version 150 2024-09-21T07:43:39.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.369+0000 7fa388c7d740 1 -- 172.21.15.39:0/3299862111 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=150,monmap=2+}) -- 0x558c13d336b0 con 0x558c13c30590 2024-09-21T07:43:39.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.369+0000 7fa35fe00640 1 -- 172.21.15.39:0/3299862111 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3640042c0 con 0x558c13c30590 2024-09-21T07:43:39.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.369+0000 7fa35fe00640 1 -- 172.21.15.39:0/3299862111 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 150) ==== 3522+0+0 (secure 0 0 0) 0x7fa364003ae0 con 0x558c13c30590 2024-09-21T07:43:39.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.369+0000 7fa388c7d740 10 client.8124 fetch_fsmap finished waiting for FSMap version 150 2024-09-21T07:43:39.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.369+0000 7fa388c7d740 10 client.8124 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T07:43:39.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.369+0000 7fa388c7d740 20 client.8124 populate_metadata read hostname 'smithi039' 2024-09-21T07:43:39.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.369+0000 7fa388c7d740 1 --2- 172.21.15.39:0/3299862111 >> [v2:172.21.15.148:6836/759792812,v1:172.21.15.148:6837/759792812] conn(0x558c13c0fe40 0x558c1404eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:39.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.369+0000 7fa388c7d740 4 client.8124 mds_command: new command op to 7366 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:43:39.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.369+0000 7fa388c7d740 1 -- 172.21.15.39:0/3299862111 --> [v2:172.21.15.148:6836/759792812,v1:172.21.15.148:6837/759792812] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c13f6d520 con 0x558c13c0fe40 2024-09-21T07:43:39.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.370+0000 7fa37aa00640 1 --2- 172.21.15.39:0/3299862111 >> [v2:172.21.15.148:6836/759792812,v1:172.21.15.148:6837/759792812] conn(0x558c13c0fe40 0x558c1404eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:39.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.370+0000 7fa37aa00640 1 --2- 172.21.15.39:0/3299862111 >> [v2:172.21.15.148:6836/759792812,v1:172.21.15.148:6837/759792812] conn(0x558c13c0fe40 0x558c1404eb00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b19fd5eb22adda51 server_cookie=cbd91b9d85e6f59d in_seq=0 out_seq=0 2024-09-21T07:43:39.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.370+0000 7fa35fe00640 10 client.8124 ms_handle_connect on v2:172.21.15.148:6836/759792812 2024-09-21T07:43:39.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.371+0000 7fa35fe00640 1 -- 172.21.15.39:0/3299862111 <== mds.1 v2:172.21.15.148:6836/759792812 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558c13f6d520 con 0x558c13c0fe40 2024-09-21T07:43:39.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.371+0000 7fa35fe00640 10 client.8124 handle_command_reply: tid=0 2024-09-21T07:43:39.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.371+0000 7fa388c7d740 1 client.8124 shutdown 2024-09-21T07:43:39.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.372+0000 7fa388c7d740 1 -- 172.21.15.39:0/3299862111 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c13c30590 msgr2=0x558c14019b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:39.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.372+0000 7fa388c7d740 1 --2- 172.21.15.39:0/3299862111 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c13c30590 0x558c14019b10 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fa364011010 tx=0x7fa364004130 comp rx=0 tx=0).stop 2024-09-21T07:43:39.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.372+0000 7fa388c7d740 1 -- 172.21.15.39:0/3299862111 shutdown_connections 2024-09-21T07:43:39.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.372+0000 7fa388c7d740 1 --2- 172.21.15.39:0/3299862111 >> [v2:172.21.15.148:6836/759792812,v1:172.21.15.148:6837/759792812] conn(0x558c13c0fe40 0x558c1404eb00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.372+0000 7fa388c7d740 1 --2- 172.21.15.39:0/3299862111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c13c51270 0x558c1401a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.372+0000 7fa388c7d740 1 --2- 172.21.15.39:0/3299862111 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c13c30590 0x558c14019b10 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.372+0000 7fa388c7d740 1 --2- 172.21.15.39:0/3299862111 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558c13c283b0 0x558c14017ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.372+0000 7fa388c7d740 1 -- 172.21.15.39:0/3299862111 >> 172.21.15.39:0/3299862111 conn(0x558c13c453c0 msgr2=0x558c1403fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:39.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.372+0000 7fa35fe00640 0 client.8124 ms_handle_reset on v2:172.21.15.148:6836/759792812 2024-09-21T07:43:39.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.372+0000 7fa388c7d740 1 -- 172.21.15.39:0/3299862111 shutdown_connections 2024-09-21T07:43:39.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.372+0000 7fa388c7d740 1 -- 172.21.15.39:0/3299862111 wait complete. 2024-09-21T07:43:39.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.372+0000 7fa388c7d740 20 client.8124 trim_cache size 0 max 16384 2024-09-21T07:43:39.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.391+0000 7fa388c7d740 1 Processor -- start 2024-09-21T07:43:39.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.392+0000 7fa388c7d740 1 -- start start 2024-09-21T07:43:39.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.393+0000 7fa388c7d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c13c283b0 0x558c140363e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:39.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.393+0000 7fa388c7d740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558c13c30590 0x558c1405efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:39.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.393+0000 7fa388c7d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c13c51270 0x558c1405f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:39.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.393+0000 7fa388c7d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c205500 con 0x558c13c51270 2024-09-21T07:43:39.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.393+0000 7fa388c7d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c1f6420 con 0x558c13c283b0 2024-09-21T07:43:39.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.393+0000 7fa388c7d740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa37c203e90 con 0x558c13c30590 2024-09-21T07:43:39.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.393+0000 7fa37b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c13c51270 0x558c1405f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:39.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.393+0000 7fa37b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c13c51270 0x558c1405f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50452/0 (socket says 172.21.15.39:50452) 2024-09-21T07:43:39.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.393+0000 7fa37b400640 1 -- 172.21.15.39:0/1152114077 learned_addr learned my addr 172.21.15.39:0/1152114077 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:39.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.393+0000 7fa373e00640 1 --2- 172.21.15.39:0/1152114077 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558c13c30590 0x558c1405efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:39.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.393+0000 7fa37aa00640 1 --2- 172.21.15.39:0/1152114077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c13c283b0 0x558c140363e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:39.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.393+0000 7fa37b400640 1 -- 172.21.15.39:0/1152114077 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558c13c30590 msgr2=0x558c1405efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:43:39.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.394+0000 7fa37b400640 1 --2- 172.21.15.39:0/1152114077 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558c13c30590 0x558c1405efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.394+0000 7fa37b400640 1 -- 172.21.15.39:0/1152114077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c13c283b0 msgr2=0x558c140363e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:43:39.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.394+0000 7fa37b400640 1 --2- 172.21.15.39:0/1152114077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c13c283b0 0x558c140363e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.394+0000 7fa37b400640 1 -- 172.21.15.39:0/1152114077 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c1403f590 con 0x558c13c51270 2024-09-21T07:43:39.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.394+0000 7fa373e00640 1 --2- 172.21.15.39:0/1152114077 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558c13c30590 0x558c1405efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:43:39.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.394+0000 7fa37aa00640 1 --2- 172.21.15.39:0/1152114077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c13c283b0 0x558c140363e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:43:39.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.394+0000 7fa37b400640 1 --2- 172.21.15.39:0/1152114077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c13c51270 0x558c1405f500 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fa3740ba050 tx=0x7fa37402c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b9e290c8afff6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:39.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.394+0000 7fa36aa00640 1 -- 172.21.15.39:0/1152114077 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa374083440 con 0x558c13c51270 2024-09-21T07:43:39.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.394+0000 7fa36aa00640 1 -- 172.21.15.39:0/1152114077 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa374009280 con 0x558c13c51270 2024-09-21T07:43:39.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.394+0000 7fa36aa00640 1 -- 172.21.15.39:0/1152114077 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa374084030 con 0x558c13c51270 2024-09-21T07:43:39.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.395+0000 7fa388c7d740 1 -- 172.21.15.39:0/1152114077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c13c51270 msgr2=0x558c1405f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:39.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.395+0000 7fa388c7d740 1 --2- 172.21.15.39:0/1152114077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c13c51270 0x558c1405f500 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fa3740ba050 tx=0x7fa37402c040 comp rx=0 tx=0).stop 2024-09-21T07:43:39.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.395+0000 7fa388c7d740 1 -- 172.21.15.39:0/1152114077 shutdown_connections 2024-09-21T07:43:39.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.395+0000 7fa388c7d740 1 --2- 172.21.15.39:0/1152114077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c13c51270 0x558c1405f500 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.395+0000 7fa388c7d740 1 --2- 172.21.15.39:0/1152114077 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558c13c30590 0x558c1405efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.395+0000 7fa388c7d740 1 --2- 172.21.15.39:0/1152114077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c13c283b0 0x558c140363e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.395+0000 7fa388c7d740 1 -- 172.21.15.39:0/1152114077 >> 172.21.15.39:0/1152114077 conn(0x558c13c453c0 msgr2=0x558c13ff77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:39.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.395+0000 7fa388c7d740 1 -- 172.21.15.39:0/1152114077 shutdown_connections 2024-09-21T07:43:39.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.395+0000 7fa388c7d740 1 -- 172.21.15.39:0/1152114077 wait complete. 2024-09-21T07:43:39.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.396+0000 7fa388c7d740 1 -- start start 2024-09-21T07:43:39.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.396+0000 7fa388c7d740 1 Processor -- start 2024-09-21T07:43:39.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.397+0000 7fa388c7d740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558c13c283b0 0x558c140a1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:39.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.397+0000 7fa388c7d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c13c30590 0x558c13ff2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:39.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.397+0000 7fa388c7d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c13c51270 0x558c13ff31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:39.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.397+0000 7fa388c7d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558c1401b840 con 0x558c13c51270 2024-09-21T07:43:39.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.397+0000 7fa388c7d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558c1404bef0 con 0x558c13c30590 2024-09-21T07:43:39.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.397+0000 7fa388c7d740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x558c1401be90 con 0x558c13c283b0 2024-09-21T07:43:39.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.398+0000 7fa388c7d740 10 client.0 _ll_register_callbacks cb 0x7fa36400b9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:43:39.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.398+0000 7fa37b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c13c51270 0x558c13ff31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:39.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.398+0000 7fa37b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c13c51270 0x558c13ff31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50458/0 (socket says 172.21.15.39:50458) 2024-09-21T07:43:39.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.398+0000 7fa37b400640 1 -- 172.21.15.39:0/145558731 learned_addr learned my addr 172.21.15.39:0/145558731 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:39.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.398+0000 7fa37aa00640 1 --2- 172.21.15.39:0/145558731 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558c13c283b0 0x558c140a1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:39.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.398+0000 7fa373e00640 1 --2- 172.21.15.39:0/145558731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c13c30590 0x558c13ff2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:39.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.398+0000 7fa37b400640 1 -- 172.21.15.39:0/145558731 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558c13c283b0 msgr2=0x558c140a1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:43:39.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.398+0000 7fa37b400640 1 --2- 172.21.15.39:0/145558731 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558c13c283b0 0x558c140a1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.398+0000 7fa37b400640 1 -- 172.21.15.39:0/145558731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c13c30590 msgr2=0x558c13ff2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:43:39.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.398+0000 7fa37b400640 1 --2- 172.21.15.39:0/145558731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c13c30590 0x558c13ff2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.398+0000 7fa37b400640 1 -- 172.21.15.39:0/145558731 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c140bcb90 con 0x558c13c51270 2024-09-21T07:43:39.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.398+0000 7fa37b400640 1 --2- 172.21.15.39:0/145558731 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c13c51270 0x558c13ff31b0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fa374083f90 tx=0x7fa374004960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d18b2dd24ab15dc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:39.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.398+0000 7fa35fe00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:43:39.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.399+0000 7fa35fe00640 1 -- 172.21.15.39:0/145558731 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37408d030 con 0x558c13c51270 2024-09-21T07:43:39.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.399+0000 7fa35fe00640 1 -- 172.21.15.39:0/145558731 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8205+0+0 (secure 0 0 0) 0x7fa374082b90 con 0x558c13c51270 2024-09-21T07:43:39.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.399+0000 7fa388c7d740 1 -- 172.21.15.39:0/145558731 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa374009280 con 0x558c13c51270 2024-09-21T07:43:39.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.399+0000 7fa35fe00640 1 -- 172.21.15.39:0/145558731 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=150) ==== 24+0+0 (secure 0 0 0) 0x7fa374083930 con 0x558c13c51270 2024-09-21T07:43:39.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.399+0000 7fa388c7d740 10 client.8130 fetch_fsmap learned FSMap version 150 2024-09-21T07:43:39.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.399+0000 7fa388c7d740 1 -- 172.21.15.39:0/145558731 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=150,monmap=2+}) -- 0x558c13ff7de0 con 0x558c13c51270 2024-09-21T07:43:39.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.399+0000 7fa35fe00640 1 -- 172.21.15.39:0/145558731 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa374009280 con 0x558c13c51270 2024-09-21T07:43:39.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.399+0000 7fa35fe00640 1 -- 172.21.15.39:0/145558731 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 150) ==== 3522+0+0 (secure 0 0 0) 0x7fa37403a030 con 0x558c13c51270 2024-09-21T07:43:39.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.399+0000 7fa388c7d740 10 client.8130 fetch_fsmap finished waiting for FSMap version 150 2024-09-21T07:43:39.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.399+0000 7fa388c7d740 10 client.8130 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T07:43:39.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.399+0000 7fa388c7d740 20 client.8130 populate_metadata read hostname 'smithi039' 2024-09-21T07:43:39.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.399+0000 7fa388c7d740 1 --2- 172.21.15.39:0/145558731 >> [v2:172.21.15.148:6836/759792812,v1:172.21.15.148:6837/759792812] conn(0x558c140342c0 0x558c140346d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:39.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.399+0000 7fa388c7d740 4 client.8130 mds_command: new command op to 7366 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:43:39.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.399+0000 7fa388c7d740 1 -- 172.21.15.39:0/145558731 --> [v2:172.21.15.148:6836/759792812,v1:172.21.15.148:6837/759792812] -- command(tid 0: {"prefix": "status"}) -- 0x558c13f68fd0 con 0x558c140342c0 2024-09-21T07:43:39.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.400+0000 7fa37aa00640 1 --2- 172.21.15.39:0/145558731 >> [v2:172.21.15.148:6836/759792812,v1:172.21.15.148:6837/759792812] conn(0x558c140342c0 0x558c140346d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:39.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.400+0000 7fa37aa00640 1 --2- 172.21.15.39:0/145558731 >> [v2:172.21.15.148:6836/759792812,v1:172.21.15.148:6837/759792812] conn(0x558c140342c0 0x558c140346d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2f8b229af0ee05df server_cookie=57b6360d9855e8f8 in_seq=0 out_seq=0 2024-09-21T07:43:39.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.400+0000 7fa35fe00640 10 client.8130 ms_handle_connect on v2:172.21.15.148:6836/759792812 2024-09-21T07:43:39.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.401+0000 7fa35fe00640 1 -- 172.21.15.39:0/145558731 <== mds.1 v2:172.21.15.148:6836/759792812 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x558c13f68fd0 con 0x558c140342c0 2024-09-21T07:43:39.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.401+0000 7fa35fe00640 10 client.8130 handle_command_reply: tid=0 2024-09-21T07:43:39.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.401+0000 7fa388c7d740 1 client.8130 shutdown 2024-09-21T07:43:39.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.401+0000 7fa388c7d740 1 -- 172.21.15.39:0/145558731 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c13c51270 msgr2=0x558c13ff31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:39.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.401+0000 7fa388c7d740 1 --2- 172.21.15.39:0/145558731 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c13c51270 0x558c13ff31b0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fa374083f90 tx=0x7fa374004960 comp rx=0 tx=0).stop 2024-09-21T07:43:39.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.401+0000 7fa388c7d740 1 -- 172.21.15.39:0/145558731 shutdown_connections 2024-09-21T07:43:39.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.401+0000 7fa388c7d740 1 --2- 172.21.15.39:0/145558731 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c13c51270 0x558c13ff31b0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.401+0000 7fa388c7d740 1 --2- 172.21.15.39:0/145558731 >> [v2:172.21.15.148:6836/759792812,v1:172.21.15.148:6837/759792812] conn(0x558c140342c0 0x558c140346d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.401+0000 7fa388c7d740 1 --2- 172.21.15.39:0/145558731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c13c30590 0x558c13ff2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.401+0000 7fa388c7d740 1 --2- 172.21.15.39:0/145558731 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558c13c283b0 0x558c140a1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.401+0000 7fa388c7d740 1 -- 172.21.15.39:0/145558731 >> 172.21.15.39:0/145558731 conn(0x558c13c453c0 msgr2=0x558c14032ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:39.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.401+0000 7fa35fe00640 0 client.8130 ms_handle_reset on v2:172.21.15.148:6836/759792812 2024-09-21T07:43:39.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.401+0000 7fa388c7d740 1 -- 172.21.15.39:0/145558731 shutdown_connections 2024-09-21T07:43:39.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.401+0000 7fa388c7d740 1 -- 172.21.15.39:0/145558731 wait complete. 2024-09-21T07:43:39.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.401+0000 7fa388c7d740 20 client.8130 trim_cache size 0 max 16384 2024-09-21T07:43:39.407 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:43:39.407 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:43:39.407 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 1, 2024-09-21T07:43:39.407 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 7366, 2024-09-21T07:43:39.407 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:43:39.407 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:43:39.407 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:43:39.407 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 13.309277403999999, 2024-09-21T07:43:39.407 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 150, 2024-09-21T07:43:39.407 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 117, 2024-09-21T07:43:39.407 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 117, 2024-09-21T07:43:39.407 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 59.089880362999999 2024-09-21T07:43:39.408 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:43:39.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.402+0000 7fa381c00640 1 -- 172.21.15.39:0/2445326518 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa358078b90 msgr2=0x7fa35807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:39.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.402+0000 7fa381c00640 1 --2- 172.21.15.39:0/2445326518 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa358078b90 0x7fa35807b0a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa36c000f60 tx=0x7fa36c009040 comp rx=0 tx=0).stop 2024-09-21T07:43:39.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.402+0000 7fa381c00640 1 -- 172.21.15.39:0/2445326518 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa37c15cd30 msgr2=0x7fa37c15b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:39.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.402+0000 7fa381c00640 1 --2- 172.21.15.39:0/2445326518 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa37c15cd30 0x7fa37c15b800 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fa37402ae10 tx=0x7fa374032d10 comp rx=0 tx=0).stop 2024-09-21T07:43:39.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.402+0000 7fa381c00640 1 -- 172.21.15.39:0/2445326518 shutdown_connections 2024-09-21T07:43:39.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.402+0000 7fa381c00640 1 --2- 172.21.15.39:0/2445326518 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa358078b90 0x7fa35807b0a0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.402+0000 7fa381c00640 1 --2- 172.21.15.39:0/2445326518 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa37c15cd30 0x7fa37c15b800 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.402+0000 7fa381c00640 1 --2- 172.21.15.39:0/2445326518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa37c0d0620 0x7fa37c15b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.402+0000 7fa381c00640 1 --2- 172.21.15.39:0/2445326518 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa37c0cfd10 0x7fa37c159640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.402+0000 7fa381c00640 1 -- 172.21.15.39:0/2445326518 >> 172.21.15.39:0/2445326518 conn(0x7fa37c06cfc0 msgr2=0x7fa37c06f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:39.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.402+0000 7fa381c00640 1 -- 172.21.15.39:0/2445326518 shutdown_connections 2024-09-21T07:43:39.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.402+0000 7fa381c00640 1 -- 172.21.15.39:0/2445326518 wait complete. 2024-09-21T07:43:39.426 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:43:39.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:43:39.426 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-09-21T07:43:39.482 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:43:39.482 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:43:39.521 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:43:39.521 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:43:39.597 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:43:39.679 INFO:tasks.ceph.mds.c.smithi039.stderr:2024-09-21T07:43:39.677+0000 7f9202600640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:43:39.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.866+0000 7f081a000640 1 -- 172.21.15.39:0/3456643427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08140d0f20 msgr2=0x7f081415c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:39.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.866+0000 7f081a000640 1 --2- 172.21.15.39:0/3456643427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08140d0f20 0x7f081415c830 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f08140cdda0 tx=0x7f080802a560 comp rx=0 tx=0).stop 2024-09-21T07:43:39.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.866+0000 7f081a000640 1 -- 172.21.15.39:0/3456643427 shutdown_connections 2024-09-21T07:43:39.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.866+0000 7f081a000640 1 --2- 172.21.15.39:0/3456643427 >> v2:172.21.15.148:3300/0 conn(0x7f08140cbd70 0x7f0814072fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.866+0000 7f081a000640 1 --2- 172.21.15.39:0/3456643427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08140d0f20 0x7f081415c830 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.866+0000 7f081a000640 1 -- 172.21.15.39:0/3456643427 >> 172.21.15.39:0/3456643427 conn(0x7f08140b2510 msgr2=0x7f08140d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:39.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.867+0000 7f081a000640 1 -- 172.21.15.39:0/3456643427 shutdown_connections 2024-09-21T07:43:39.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.867+0000 7f081a000640 1 -- 172.21.15.39:0/3456643427 wait complete. 2024-09-21T07:43:39.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.869+0000 7f081a000640 1 Processor -- start 2024-09-21T07:43:39.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.869+0000 7f081a000640 1 -- start start 2024-09-21T07:43:39.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.870+0000 7f081a000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08140cb960 0x7f0814073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:39.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.870+0000 7f081a000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08140cbd70 0x7f08141f6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:39.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.870+0000 7f081a000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f08140d0f20 0x7f08141f73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:39.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.870+0000 7f081a000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f08140b43f0 con 0x7f08140cb960 2024-09-21T07:43:39.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.870+0000 7f081a000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0814073830 con 0x7f08140cbd70 2024-09-21T07:43:39.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.870+0000 7f081a000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f08140b4570 con 0x7f08140d0f20 2024-09-21T07:43:39.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.871+0000 7f0818c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08140cb960 0x7f0814073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:39.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.871+0000 7f0818c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08140cb960 0x7f0814073f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50486/0 (socket says 172.21.15.39:50486) 2024-09-21T07:43:39.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.871+0000 7f0818c00640 1 -- 172.21.15.39:0/2849359133 learned_addr learned my addr 172.21.15.39:0/2849359133 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:39.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.871+0000 7f0813e00640 1 --2- 172.21.15.39:0/2849359133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08140cbd70 0x7f08141f6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:39.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.871+0000 7f0819600640 1 --2- 172.21.15.39:0/2849359133 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f08140d0f20 0x7f08141f73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:39.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.871+0000 7f0818c00640 1 -- 172.21.15.39:0/2849359133 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f08140d0f20 msgr2=0x7f08141f73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:39.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.871+0000 7f0818c00640 1 --2- 172.21.15.39:0/2849359133 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f08140d0f20 0x7f08141f73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.871+0000 7f0818c00640 1 -- 172.21.15.39:0/2849359133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08140cbd70 msgr2=0x7f08141f6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:39.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.871+0000 7f0818c00640 1 --2- 172.21.15.39:0/2849359133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08140cbd70 0x7f08141f6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:39.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.871+0000 7f0818c00640 1 -- 172.21.15.39:0/2849359133 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08140ad250 con 0x7f08140cb960 2024-09-21T07:43:39.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.871+0000 7f0813e00640 1 --2- 172.21.15.39:0/2849359133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08140cbd70 0x7f08141f6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:43:39.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.871+0000 7f0819600640 1 --2- 172.21.15.39:0/2849359133 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f08140d0f20 0x7f08141f73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:43:39.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.871+0000 7f0818c00640 1 --2- 172.21.15.39:0/2849359133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08140cb960 0x7f0814073f70 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f0808034040 tx=0x7f0808003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:39.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.872+0000 7f0810c00640 1 -- 172.21.15.39:0/2849359133 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080803e030 con 0x7f08140cb960 2024-09-21T07:43:39.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.872+0000 7f0810c00640 1 -- 172.21.15.39:0/2849359133 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0808003e00 con 0x7f08140cb960 2024-09-21T07:43:39.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.872+0000 7f0810c00640 1 -- 172.21.15.39:0/2849359133 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0808035040 con 0x7f08140cb960 2024-09-21T07:43:39.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.872+0000 7f081a000640 1 -- 172.21.15.39:0/2849359133 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08140ad530 con 0x7f08140cb960 2024-09-21T07:43:39.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.872+0000 7f081a000640 1 -- 172.21.15.39:0/2849359133 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08140aed60 con 0x7f08140cb960 2024-09-21T07:43:39.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.873+0000 7f081a000640 1 -- 172.21.15.39:0/2849359133 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08141f5430 con 0x7f08140cb960 2024-09-21T07:43:39.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.874+0000 7f0810c00640 1 -- 172.21.15.39:0/2849359133 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0808041030 con 0x7f08140cb960 2024-09-21T07:43:39.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.875+0000 7f0810c00640 1 --2- 172.21.15.39:0/2849359133 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f07f4078b90 0x7f07f407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:39.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.875+0000 7f0810c00640 1 -- 172.21.15.39:0/2849359133 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8205+0+0 (secure 0 0 0) 0x7f08080bbfd0 con 0x7f08140cb960 2024-09-21T07:43:39.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.876+0000 7f0813e00640 1 --2- 172.21.15.39:0/2849359133 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f07f4078b90 0x7f07f407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:39.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.877+0000 7f0813e00640 1 --2- 172.21.15.39:0/2849359133 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f07f4078b90 0x7f07f407b0a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f0814156940 tx=0x7f0800008040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:39.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:39.880+0000 7f0810c00640 1 -- 172.21.15.39:0/2849359133 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0808083a30 con 0x7f08140cb960 2024-09-21T07:43:40.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:40.089+0000 7f081a000640 1 -- 172.21.15.39:0/2849359133 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f08140ad700 con 0x7f08140cb960 2024-09-21T07:43:40.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:40.090+0000 7f0810c00640 1 -- 172.21.15.39:0/2849359133 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 32 entries v117) ==== 81+0+1925 (secure 0 0 0) 0x7f0808083330 con 0x7f08140cb960 2024-09-21T07:43:40.092 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/1326711417 2024-09-22T07:43:26.076277+0000 2024-09-21T07:43:40.092 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/1326711417 2024-09-22T07:43:26.076277+0000 2024-09-21T07:43:40.092 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/364657398 2024-09-22T07:39:15.579613+0000 2024-09-21T07:43:40.092 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.1:0/177249410 2024-09-21T08:41:13.882767+0000 2024-09-21T07:43:40.092 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/73220122 2024-09-22T07:42:40.074859+0000 2024-09-21T07:43:40.092 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/596665086 2024-09-21T08:43:35.317820+0000 2024-09-21T07:43:40.092 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/829716143 2024-09-22T07:37:01.259219+0000 2024-09-21T07:43:40.093 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/855150775 2024-09-22T07:39:07.739045+0000 2024-09-21T07:43:40.093 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/2381504293 2024-09-22T07:38:32.207414+0000 2024-09-21T07:43:40.093 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/855150775 2024-09-22T07:39:07.739045+0000 2024-09-21T07:43:40.093 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/742594435 2024-09-21T08:42:01.225749+0000 2024-09-21T07:43:40.093 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/2821446711 2024-09-22T07:40:57.372409+0000 2024-09-21T07:43:40.093 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.1:0/90671776 2024-09-21T08:39:22.988045+0000 2024-09-21T07:43:40.093 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/1472430787 2024-09-22T07:37:58.212892+0000 2024-09-21T07:43:40.093 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/3695489158 2024-09-22T07:41:51.807018+0000 2024-09-21T07:43:40.093 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/3026827614 2024-09-22T07:37:29.781472+0000 2024-09-21T07:43:40.093 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/3026827614 2024-09-22T07:37:29.781472+0000 2024-09-21T07:43:40.093 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/829716143 2024-09-22T07:37:01.259219+0000 2024-09-21T07:43:40.093 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/3695489158 2024-09-22T07:41:51.807018+0000 2024-09-21T07:43:40.093 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/1472430787 2024-09-22T07:37:58.212892+0000 2024-09-21T07:43:40.094 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/992188984 2024-09-22T07:40:00.778049+0000 2024-09-21T07:43:40.094 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6833/364657398 2024-09-22T07:39:15.579613+0000 2024-09-21T07:43:40.094 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/992188984 2024-09-22T07:40:00.778049+0000 2024-09-21T07:43:40.094 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6833/1456343023 2024-09-22T07:40:08.637993+0000 2024-09-21T07:43:40.094 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/2381504293 2024-09-22T07:38:32.207414+0000 2024-09-21T07:43:40.094 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/2336112936 2024-09-22T07:41:05.979253+0000 2024-09-21T07:43:40.094 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/1456343023 2024-09-22T07:40:08.637993+0000 2024-09-21T07:43:40.094 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/2821446711 2024-09-22T07:40:57.372409+0000 2024-09-21T07:43:40.094 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/3356393880 2024-09-21T08:40:15.824973+0000 2024-09-21T07:43:40.094 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/2336112936 2024-09-22T07:41:05.979253+0000 2024-09-21T07:43:40.094 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/73220122 2024-09-22T07:42:40.074859+0000 2024-09-21T07:43:40.094 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1669604593 2024-09-21T08:42:52.059256+0000 2024-09-21T07:43:40.095 INFO:teuthology.orchestra.run.smithi039.stderr:listed 32 entries 2024-09-21T07:43:40.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:40.093+0000 7f081a000640 1 -- 172.21.15.39:0/2849359133 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f07f4078b90 msgr2=0x7f07f407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:40.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:40.093+0000 7f081a000640 1 --2- 172.21.15.39:0/2849359133 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f07f4078b90 0x7f07f407b0a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f0814156940 tx=0x7f0800008040 comp rx=0 tx=0).stop 2024-09-21T07:43:40.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:40.093+0000 7f081a000640 1 -- 172.21.15.39:0/2849359133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08140cb960 msgr2=0x7f0814073f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:40.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:40.093+0000 7f081a000640 1 --2- 172.21.15.39:0/2849359133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08140cb960 0x7f0814073f70 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f0808034040 tx=0x7f0808003910 comp rx=0 tx=0).stop 2024-09-21T07:43:40.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:40.093+0000 7f081a000640 1 -- 172.21.15.39:0/2849359133 shutdown_connections 2024-09-21T07:43:40.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:40.093+0000 7f081a000640 1 --2- 172.21.15.39:0/2849359133 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f07f4078b90 0x7f07f407b0a0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:40.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:40.093+0000 7f081a000640 1 --2- 172.21.15.39:0/2849359133 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f08140d0f20 0x7f08141f73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:40.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:40.093+0000 7f081a000640 1 --2- 172.21.15.39:0/2849359133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08140cbd70 0x7f08141f6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:40.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:40.093+0000 7f081a000640 1 --2- 172.21.15.39:0/2849359133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08140cb960 0x7f0814073f70 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:40.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:40.093+0000 7f081a000640 1 -- 172.21.15.39:0/2849359133 >> 172.21.15.39:0/2849359133 conn(0x7f08140b2510 msgr2=0x7f081415f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:40.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:40.093+0000 7f081a000640 1 -- 172.21.15.39:0/2849359133 shutdown_connections 2024-09-21T07:43:40.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:40.093+0000 7f081a000640 1 -- 172.21.15.39:0/2849359133 wait complete. 2024-09-21T07:43:40.121 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T07:43:40.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:43:40.121 DEBUG:teuthology.orchestra.run.smithi039:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T07:43:40.858 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T07:43:40.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:43:40.858 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:43:40.895 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:43:40.895 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:43:40.971 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:43:40.971 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:43:41.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.237+0000 7fdb7f200640 1 -- 172.21.15.39:0/2348723894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb64004d20 msgr2=0x7fdb64005170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:41.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.237+0000 7fdb7f200640 1 --2- 172.21.15.39:0/2348723894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb64004d20 0x7fdb64005170 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fdb70006f80 tx=0x7fdb64006940 comp rx=0 tx=0).stop 2024-09-21T07:43:41.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.237+0000 7fdb7e800640 1 -- 172.21.15.39:0/2348723894 reap_dead start 2024-09-21T07:43:41.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.238+0000 7fdb7f200640 1 -- 172.21.15.39:0/2348723894 shutdown_connections 2024-09-21T07:43:41.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.238+0000 7fdb7f200640 1 -- 172.21.15.39:0/2348723894 >> 172.21.15.39:0/2348723894 conn(0x7fdb800b2510 msgr2=0x7fdb800d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:41.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.238+0000 7fdb7f200640 1 -- 172.21.15.39:0/2348723894 shutdown_connections 2024-09-21T07:43:41.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.238+0000 7fdb7f200640 1 -- 172.21.15.39:0/2348723894 wait complete. 2024-09-21T07:43:41.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.240+0000 7fdb7f200640 1 Processor -- start 2024-09-21T07:43:41.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.240+0000 7fdb7f200640 1 -- start start 2024-09-21T07:43:41.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.241+0000 7fdb7f200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb64004d20 0x7fdb801f3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:41.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.241+0000 7fdb7f200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdb801f68a0 0x7fdb801e4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:41.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.241+0000 7fdb7f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb801f4420 0x7fdb801f4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:41.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.242+0000 7fdb7f200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdb800cfd20 con 0x7fdb801f4420 2024-09-21T07:43:41.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.242+0000 7fdb7f200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdb800cfba0 con 0x7fdb64004d20 2024-09-21T07:43:41.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.242+0000 7fdb7f200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fdb800cfea0 con 0x7fdb801f68a0 2024-09-21T07:43:41.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.242+0000 7fdb7e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb801f4420 0x7fdb801f4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:41.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.242+0000 7fdb7e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb801f4420 0x7fdb801f4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50494/0 (socket says 172.21.15.39:50494) 2024-09-21T07:43:41.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.242+0000 7fdb7e800640 1 -- 172.21.15.39:0/824216710 learned_addr learned my addr 172.21.15.39:0/824216710 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:41.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.242+0000 7fdb7e800640 1 -- 172.21.15.39:0/824216710 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdb801f68a0 msgr2=0x7fdb801e4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:41.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.242+0000 7fdb7e800640 1 --2- 172.21.15.39:0/824216710 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdb801f68a0 0x7fdb801e4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:41.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.242+0000 7fdb7e800640 1 -- 172.21.15.39:0/824216710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb64004d20 msgr2=0x7fdb801f3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:41.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.242+0000 7fdb7e800640 1 --2- 172.21.15.39:0/824216710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb64004d20 0x7fdb801f3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:41.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.242+0000 7fdb7e800640 1 -- 172.21.15.39:0/824216710 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb801f72c0 con 0x7fdb801f4420 2024-09-21T07:43:41.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.242+0000 7fdb7e800640 1 --2- 172.21.15.39:0/824216710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb801f4420 0x7fdb801f4830 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fdb80072da0 tx=0x7fdb78006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:41.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.243+0000 7fdb76000640 1 -- 172.21.15.39:0/824216710 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb78011900 con 0x7fdb801f4420 2024-09-21T07:43:41.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.243+0000 7fdb76000640 1 -- 172.21.15.39:0/824216710 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb78018070 con 0x7fdb801f4420 2024-09-21T07:43:41.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.243+0000 7fdb7f200640 1 -- 172.21.15.39:0/824216710 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb801f75a0 con 0x7fdb801f4420 2024-09-21T07:43:41.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.243+0000 7fdb76000640 1 -- 172.21.15.39:0/824216710 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb78011d40 con 0x7fdb801f4420 2024-09-21T07:43:41.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.243+0000 7fdb7f200640 1 -- 172.21.15.39:0/824216710 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb801e4780 con 0x7fdb801f4420 2024-09-21T07:43:41.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.244+0000 7fdb7f200640 1 -- 172.21.15.39:0/824216710 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb801f7030 con 0x7fdb801f4420 2024-09-21T07:43:41.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.245+0000 7fdb76000640 1 -- 172.21.15.39:0/824216710 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdb7801d460 con 0x7fdb801f4420 2024-09-21T07:43:41.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.246+0000 7fdb76000640 1 --2- 172.21.15.39:0/824216710 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdb60078bc0 0x7fdb6007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:41.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.246+0000 7fdb76000640 1 -- 172.21.15.39:0/824216710 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8205+0+0 (secure 0 0 0) 0x7fdb7809c130 con 0x7fdb801f4420 2024-09-21T07:43:41.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.247+0000 7fdb7de00640 1 --2- 172.21.15.39:0/824216710 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdb60078bc0 0x7fdb6007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:41.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.248+0000 7fdb7de00640 1 --2- 172.21.15.39:0/824216710 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdb60078bc0 0x7fdb6007b0d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fdb70033000 tx=0x7fdb700019b0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:41.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.251+0000 7fdb76000640 1 -- 172.21.15.39:0/824216710 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdb78063b00 con 0x7fdb801f4420 2024-09-21T07:43:41.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.489+0000 7fdb7f200640 1 -- 172.21.15.39:0/824216710 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fdb801f4240 con 0x7fdb801f4420 2024-09-21T07:43:41.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.490+0000 7fdb76000640 1 -- 172.21.15.39:0/824216710 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v150) ==== 53+0+73 (secure 0 0 0) 0x7fdb801f4240 con 0x7fdb801f4420 2024-09-21T07:43:41.492 INFO:teuthology.orchestra.run.smithi039.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:43:41.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.492+0000 7fdb7f200640 1 -- 172.21.15.39:0/824216710 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdb60078bc0 msgr2=0x7fdb6007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:41.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.492+0000 7fdb7f200640 1 --2- 172.21.15.39:0/824216710 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdb60078bc0 0x7fdb6007b0d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fdb70033000 tx=0x7fdb700019b0 comp rx=0 tx=0).stop 2024-09-21T07:43:41.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.492+0000 7fdb7f200640 1 -- 172.21.15.39:0/824216710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb801f4420 msgr2=0x7fdb801f4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:41.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.492+0000 7fdb7f200640 1 --2- 172.21.15.39:0/824216710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb801f4420 0x7fdb801f4830 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fdb80072da0 tx=0x7fdb78006430 comp rx=0 tx=0).stop 2024-09-21T07:43:41.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.492+0000 7fdb7f200640 1 -- 172.21.15.39:0/824216710 shutdown_connections 2024-09-21T07:43:41.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.493+0000 7fdb7f200640 1 --2- 172.21.15.39:0/824216710 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdb60078bc0 0x7fdb6007b0d0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:41.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.493+0000 7fdb7f200640 1 --2- 172.21.15.39:0/824216710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb801f4420 0x7fdb801f4830 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:41.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.493+0000 7fdb7f200640 1 --2- 172.21.15.39:0/824216710 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdb801f68a0 0x7fdb801e4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:41.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.493+0000 7fdb7f200640 1 --2- 172.21.15.39:0/824216710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb64004d20 0x7fdb801f3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:41.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.493+0000 7fdb7f200640 1 -- 172.21.15.39:0/824216710 >> 172.21.15.39:0/824216710 conn(0x7fdb800b2510 msgr2=0x7fdb8006f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:41.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.493+0000 7fdb7f200640 1 -- 172.21.15.39:0/824216710 shutdown_connections 2024-09-21T07:43:41.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41.493+0000 7fdb7f200640 1 -- 172.21.15.39:0/824216710 wait complete. 2024-09-21T07:43:41.521 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:43:41.521 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:43:41.521 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi039.front.sepia.ceph.com 2024-09-21T07:43:41.521 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T07:43:41.521 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:43:41.522 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:43:41.522 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:43:41.522 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:43:41.522 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:43:41.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:43:41.522 DEBUG:teuthology.orchestra.run.smithi039:> ip addr 2024-09-21T07:43:41.557 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:43:41.558 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:43:41.558 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:43:41.558 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:43:41.558 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2024-09-21T07:43:41.558 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:43:41.558 INFO:teuthology.orchestra.run.smithi039.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:43:41.558 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:43:41.558 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f0 2024-09-21T07:43:41.558 INFO:teuthology.orchestra.run.smithi039.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:43:41.558 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c7 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:43:41.558 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f1 2024-09-21T07:43:41.558 INFO:teuthology.orchestra.run.smithi039.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:43:41.558 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:70 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:43:41.559 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f0 2024-09-21T07:43:41.559 INFO:teuthology.orchestra.run.smithi039.stdout: inet 172.21.15.39/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T07:43:41.559 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft 42097sec preferred_lft 42097sec 2024-09-21T07:43:41.559 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec4:7aff:febd:1370/64 scope link 2024-09-21T07:43:41.559 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:43:41.559 INFO:teuthology.orchestra.run.smithi039.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:43:41.559 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:71 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:43:41.559 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f1 2024-09-21T07:43:41.559 INFO:teuthology.orchestra.run.smithi039.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:43:41.559 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff 2024-09-21T07:43:41.559 INFO:teuthology.orchestra.run.smithi039.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:43:41.559 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:43:41.559 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::b079:ddff:fe0e:6333/64 scope link 2024-09-21T07:43:41.560 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:43:41.560 INFO:teuthology.orchestra.run.smithi039.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:43:41.560 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:43:41.560 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::2ca7:98ff:fe8a:d0ed/64 scope link 2024-09-21T07:43:41.560 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:43:41.560 INFO:teuthology.orchestra.run.smithi039.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:43:41.560 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether ee:10:53:01:0c:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:43:41.560 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec10:53ff:fe01:c44/64 scope link 2024-09-21T07:43:41.560 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:43:41.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:43:41.560 DEBUG:teuthology.orchestra.run.smithi039:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:43:41.651 INFO:teuthology.orchestra.run.smithi039.stdout:1 2024-09-21T07:43:41.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:43:41.654 DEBUG:teuthology.orchestra.run.smithi039:> route 2024-09-21T07:43:41.690 INFO:teuthology.orchestra.run.smithi039.stdout:Kernel IP routing table 2024-09-21T07:43:41.691 INFO:teuthology.orchestra.run.smithi039.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:43:41.691 INFO:teuthology.orchestra.run.smithi039.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T07:43:41.691 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T07:43:41.691 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:43:41.691 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:43:41.691 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-21T07:43:41.691 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T07:43:41.691 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T07:43:41.691 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T07:43:41.691 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-21T07:43:41.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:43:41.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:43:41.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:43:41.889 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-09-21T07:43:41.925 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:43:41.925 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T07:43:41.926 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:43:41.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:43:41.926 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-09-21T07:43:42.021 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:43:42.022 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T07:43:42.057 INFO:teuthology.orchestra.run.smithi039.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:43:42.057 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:43:42.057 DEBUG:teuthology.orchestra.run.smithi039:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:43:42.132 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-21T07:43:42.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:43:42.132 DEBUG:teuthology.orchestra.run.smithi039:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:43:42.331 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:43:42.331 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:43:42.366 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:43:42.366 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:43:42.367 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: resolved to: "172.21.15.39:3300,172.21.15.148:3300,172.21.15.148:3301" 2024-09-21T07:43:42.367 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: trying mount with new device syntax: 0@7ab9b04a-f88d-49aa-a493-e5c076637ffb.cephfs=/ 2024-09-21T07:43:42.367 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.39:3300/172.21.15.148:3300/172.21.15.148:3301" will pass to kernel 2024-09-21T07:43:42.368 INFO:teuthology.orchestra.run.smithi039.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T07:43:42.368 INFO:teuthology.orchestra.run.smithi039.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:43:42.368 INFO:teuthology.orchestra.run.smithi039.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:43:42.368 INFO:teuthology.orchestra.run.smithi039.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:43:42.368 INFO:teuthology.orchestra.run.smithi039.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:43:42.371 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:43:42.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:43:42.371 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:43:42.427 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:43:42.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:43:42.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:43:42.520 INFO:teuthology.orchestra.run.smithi039.stdout:client7585 2024-09-21T07:43:42.520 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:43:42.521 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:43:42.556 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 flush journal 2024-09-21T07:43:42.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.824+0000 7fd943e00640 1 -- 172.21.15.39:0/1096945229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd944156d60 msgr2=0x7fd9440cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:42.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.824+0000 7fd943e00640 1 --2- 172.21.15.39:0/1096945229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd944156d60 0x7fd9440cc430 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fd9440c3870 tx=0x7fd93402a560 comp rx=0 tx=0).stop 2024-09-21T07:43:42.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.824+0000 7fd943e00640 1 -- 172.21.15.39:0/1096945229 shutdown_connections 2024-09-21T07:43:42.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.824+0000 7fd943e00640 1 --2- 172.21.15.39:0/1096945229 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd944160770 0x7fd94415cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:42.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.824+0000 7fd943e00640 1 --2- 172.21.15.39:0/1096945229 >> v2:172.21.15.148:3300/0 conn(0x7fd9440af690 0x7fd944160230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:42.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.824+0000 7fd943e00640 1 --2- 172.21.15.39:0/1096945229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd944156d60 0x7fd9440cc430 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:42.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.824+0000 7fd943e00640 1 -- 172.21.15.39:0/1096945229 >> 172.21.15.39:0/1096945229 conn(0x7fd9440b25f0 msgr2=0x7fd9440d01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:42.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.824+0000 7fd943e00640 1 -- 172.21.15.39:0/1096945229 shutdown_connections 2024-09-21T07:43:42.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.824+0000 7fd943e00640 1 -- 172.21.15.39:0/1096945229 wait complete. 2024-09-21T07:43:42.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.826+0000 7fd943e00640 1 Processor -- start 2024-09-21T07:43:42.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.827+0000 7fd943e00640 1 -- start start 2024-09-21T07:43:42.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.828+0000 7fd943e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9440af690 0x7fd9441564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:42.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.828+0000 7fd943e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd944156d60 0x7fd9440b3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:42.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.828+0000 7fd943e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd944160770 0x7fd9440ae5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:42.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.828+0000 7fd943e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd94415d500 con 0x7fd944156d60 2024-09-21T07:43:42.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.828+0000 7fd943e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd94415d380 con 0x7fd9440af690 2024-09-21T07:43:42.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.828+0000 7fd943e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd94415d680 con 0x7fd944160770 2024-09-21T07:43:42.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.828+0000 7fd942000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd944156d60 0x7fd9440b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:42.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.828+0000 7fd942000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd944156d60 0x7fd9440b3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60674/0 (socket says 172.21.15.39:60674) 2024-09-21T07:43:42.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.828+0000 7fd942000640 1 -- 172.21.15.39:0/829679710 learned_addr learned my addr 172.21.15.39:0/829679710 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:42.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.828+0000 7fd942a00640 1 --2- 172.21.15.39:0/829679710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9440af690 0x7fd9441564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:42.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.828+0000 7fd943400640 1 --2- 172.21.15.39:0/829679710 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd944160770 0x7fd9440ae5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:42.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.829+0000 7fd942000640 1 -- 172.21.15.39:0/829679710 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd944160770 msgr2=0x7fd9440ae5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:42.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.829+0000 7fd942000640 1 --2- 172.21.15.39:0/829679710 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd944160770 0x7fd9440ae5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:42.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.829+0000 7fd942000640 1 -- 172.21.15.39:0/829679710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9440af690 msgr2=0x7fd9441564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:42.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.829+0000 7fd942000640 1 --2- 172.21.15.39:0/829679710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9440af690 0x7fd9441564c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:42.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.829+0000 7fd942000640 1 -- 172.21.15.39:0/829679710 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd94415e080 con 0x7fd944156d60 2024-09-21T07:43:42.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.829+0000 7fd942000640 1 --2- 172.21.15.39:0/829679710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd944156d60 0x7fd9440b3700 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fd93000c970 tx=0x7fd93000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:42.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.829+0000 7fd93aa00640 1 -- 172.21.15.39:0/829679710 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd930007bb0 con 0x7fd944156d60 2024-09-21T07:43:42.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.829+0000 7fd93aa00640 1 -- 172.21.15.39:0/829679710 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd930007e40 con 0x7fd944156d60 2024-09-21T07:43:42.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.830+0000 7fd943e00640 1 -- 172.21.15.39:0/829679710 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd94415e360 con 0x7fd944156d60 2024-09-21T07:43:42.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.830+0000 7fd93aa00640 1 -- 172.21.15.39:0/829679710 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93000f6b0 con 0x7fd944156d60 2024-09-21T07:43:42.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.830+0000 7fd943e00640 1 -- 172.21.15.39:0/829679710 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd94415e920 con 0x7fd944156d60 2024-09-21T07:43:42.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.832+0000 7fd93aa00640 1 -- 172.21.15.39:0/829679710 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd93000f850 con 0x7fd944156d60 2024-09-21T07:43:42.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.833+0000 7fd93aa00640 1 --2- 172.21.15.39:0/829679710 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd924078bc0 0x7fd92407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:42.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.833+0000 7fd93aa00640 1 -- 172.21.15.39:0/829679710 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8205+0+0 (secure 0 0 0) 0x7fd93009bd90 con 0x7fd944156d60 2024-09-21T07:43:42.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.833+0000 7fd950822740 1 Processor -- start 2024-09-21T07:43:42.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.834+0000 7fd942a00640 1 --2- 172.21.15.39:0/829679710 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd924078bc0 0x7fd92407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:42.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.834+0000 7fd950822740 1 -- start start 2024-09-21T07:43:42.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.834+0000 7fd942a00640 1 --2- 172.21.15.39:0/829679710 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd924078bc0 0x7fd92407b0d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd934033000 tx=0x7fd934004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:42.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.835+0000 7fd950822740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557069893390 0x557069c81730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:42.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.835+0000 7fd950822740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55706986a4d0 0x557069c64400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:42.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.835+0000 7fd950822740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5570698726b0 0x557069c64940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:42.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.835+0000 7fd950822740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557069b9a1c0 con 0x55706986a4d0 2024-09-21T07:43:42.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.835+0000 7fd950822740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557069bad870 con 0x557069893390 2024-09-21T07:43:42.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.835+0000 7fd950822740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x557069b9cca0 con 0x5570698726b0 2024-09-21T07:43:42.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.835+0000 7fd942000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55706986a4d0 0x557069c64400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:42.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.835+0000 7fd942000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55706986a4d0 0x557069c64400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60686/0 (socket says 172.21.15.39:60686) 2024-09-21T07:43:42.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.835+0000 7fd942000640 1 -- 172.21.15.39:0/4012334667 learned_addr learned my addr 172.21.15.39:0/4012334667 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:42.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.835+0000 7fd942a00640 1 --2- 172.21.15.39:0/4012334667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557069893390 0x557069c81730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:42.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.835+0000 7fd943400640 1 --2- 172.21.15.39:0/4012334667 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5570698726b0 0x557069c64940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:42.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.836+0000 7fd942000640 1 -- 172.21.15.39:0/4012334667 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5570698726b0 msgr2=0x557069c64940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:43:42.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.836+0000 7fd942000640 1 --2- 172.21.15.39:0/4012334667 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5570698726b0 0x557069c64940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:42.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.836+0000 7fd942000640 1 -- 172.21.15.39:0/4012334667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557069893390 msgr2=0x557069c81730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:43:42.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.836+0000 7fd942000640 1 --2- 172.21.15.39:0/4012334667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557069893390 0x557069c81730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:42.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.836+0000 7fd942000640 1 -- 172.21.15.39:0/4012334667 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570699757d0 con 0x55706986a4d0 2024-09-21T07:43:42.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.836+0000 7fd942000640 1 --2- 172.21.15.39:0/4012334667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55706986a4d0 0x557069c64400 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fd930062c50 tx=0x7fd9300629d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5a817d486cb6873 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:42.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.836+0000 7fd92f000640 1 -- 172.21.15.39:0/4012334667 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93006a0f0 con 0x55706986a4d0 2024-09-21T07:43:42.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.836+0000 7fd92f000640 1 -- 172.21.15.39:0/4012334667 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd930067bd0 con 0x55706986a4d0 2024-09-21T07:43:42.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.836+0000 7fd92f000640 1 -- 172.21.15.39:0/4012334667 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93000fda0 con 0x55706986a4d0 2024-09-21T07:43:42.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.837+0000 7fd950822740 1 -- 172.21.15.39:0/4012334667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55706986a4d0 msgr2=0x557069c64400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:42.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.837+0000 7fd950822740 1 --2- 172.21.15.39:0/4012334667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55706986a4d0 0x557069c64400 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fd930062c50 tx=0x7fd9300629d0 comp rx=0 tx=0).stop 2024-09-21T07:43:42.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.837+0000 7fd950822740 1 -- 172.21.15.39:0/4012334667 shutdown_connections 2024-09-21T07:43:42.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.837+0000 7fd950822740 1 --2- 172.21.15.39:0/4012334667 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5570698726b0 0x557069c64940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:42.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.837+0000 7fd950822740 1 --2- 172.21.15.39:0/4012334667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55706986a4d0 0x557069c64400 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:42.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.837+0000 7fd950822740 1 --2- 172.21.15.39:0/4012334667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557069893390 0x557069c81730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:42.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.837+0000 7fd950822740 1 -- 172.21.15.39:0/4012334667 >> 172.21.15.39:0/4012334667 conn(0x5570698874e0 msgr2=0x557069c3a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:42.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.837+0000 7fd950822740 1 -- 172.21.15.39:0/4012334667 shutdown_connections 2024-09-21T07:43:42.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.837+0000 7fd950822740 1 -- 172.21.15.39:0/4012334667 wait complete. 2024-09-21T07:43:42.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.838+0000 7fd950822740 1 -- start start 2024-09-21T07:43:42.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.838+0000 7fd950822740 1 Processor -- start 2024-09-21T07:43:42.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.840+0000 7fd950822740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55706986a4d0 0x557069c51fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:42.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.840+0000 7fd950822740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5570698726b0 0x557069c39be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:42.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.840+0000 7fd950822740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557069893390 0x557069c3a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:42.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.840+0000 7fd950822740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557069b43410 con 0x5570698726b0 2024-09-21T07:43:42.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.840+0000 7fd950822740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557069ba0ab0 con 0x55706986a4d0 2024-09-21T07:43:42.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.840+0000 7fd950822740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55706990d740 con 0x557069893390 2024-09-21T07:43:42.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.840+0000 7fd950822740 10 client.0 _ll_register_callbacks cb 0x557069b20360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:43:42.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.840+0000 7fd942000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5570698726b0 0x557069c39be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:42.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.840+0000 7fd942000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5570698726b0 0x557069c39be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60694/0 (socket says 172.21.15.39:60694) 2024-09-21T07:43:42.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.840+0000 7fd942000640 1 -- 172.21.15.39:0/1649254080 learned_addr learned my addr 172.21.15.39:0/1649254080 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:42.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.840+0000 7fd943400640 1 --2- 172.21.15.39:0/1649254080 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557069893390 0x557069c3a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:42.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.840+0000 7fd942a00640 1 --2- 172.21.15.39:0/1649254080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55706986a4d0 0x557069c51fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:42.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.841+0000 7fd942000640 1 -- 172.21.15.39:0/1649254080 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557069893390 msgr2=0x557069c3a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:43:42.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.841+0000 7fd942000640 1 --2- 172.21.15.39:0/1649254080 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557069893390 0x557069c3a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:42.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.841+0000 7fd942000640 1 -- 172.21.15.39:0/1649254080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55706986a4d0 msgr2=0x557069c51fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:43:42.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.841+0000 7fd942000640 1 --2- 172.21.15.39:0/1649254080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55706986a4d0 0x557069c51fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:42.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.841+0000 7fd942000640 1 -- 172.21.15.39:0/1649254080 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570698fe970 con 0x5570698726b0 2024-09-21T07:43:42.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.841+0000 7fd942a00640 1 --2- 172.21.15.39:0/1649254080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55706986a4d0 0x557069c51fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:43:42.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.841+0000 7fd942000640 1 --2- 172.21.15.39:0/1649254080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5570698726b0 0x557069c39be0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fd93006c9f0 tx=0x7fd93006ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b47a7db559befd48 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:42.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.841+0000 7fd923400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:43:42.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.841+0000 7fd923400640 1 -- 172.21.15.39:0/1649254080 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd930089180 con 0x5570698726b0 2024-09-21T07:43:42.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.841+0000 7fd923400640 1 -- 172.21.15.39:0/1649254080 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8205+0+0 (secure 0 0 0) 0x7fd930089320 con 0x5570698726b0 2024-09-21T07:43:42.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.841+0000 7fd950822740 1 -- 172.21.15.39:0/1649254080 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd930067bd0 con 0x5570698726b0 2024-09-21T07:43:42.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.841+0000 7fd923400640 1 -- 172.21.15.39:0/1649254080 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=150) ==== 24+0+0 (secure 0 0 0) 0x7fd930062400 con 0x5570698726b0 2024-09-21T07:43:42.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.841+0000 7fd950822740 10 client.8154 fetch_fsmap learned FSMap version 150 2024-09-21T07:43:42.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.842+0000 7fd950822740 1 -- 172.21.15.39:0/1649254080 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=150,monmap=2+}) -- 0x5570698dc6a0 con 0x5570698726b0 2024-09-21T07:43:42.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.842+0000 7fd923400640 1 -- 172.21.15.39:0/1649254080 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd930067bd0 con 0x5570698726b0 2024-09-21T07:43:42.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.842+0000 7fd923400640 1 -- 172.21.15.39:0/1649254080 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 150) ==== 3522+0+0 (secure 0 0 0) 0x7fd930020030 con 0x5570698726b0 2024-09-21T07:43:42.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.842+0000 7fd950822740 10 client.8154 fetch_fsmap finished waiting for FSMap version 150 2024-09-21T07:43:42.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.842+0000 7fd950822740 10 client.8154 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-21T07:43:42.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.842+0000 7fd950822740 20 client.8154 populate_metadata read hostname 'smithi039' 2024-09-21T07:43:42.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.842+0000 7fd950822740 1 --2- 172.21.15.39:0/1649254080 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x557069851f60 0x557069c7a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:42.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.842+0000 7fd950822740 4 client.8154 mds_command: new command op to 7311 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:43:42.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.842+0000 7fd950822740 1 -- 172.21.15.39:0/1649254080 --> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557069baf7c0 con 0x557069851f60 2024-09-21T07:43:42.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.842+0000 7fd942a00640 1 --2- 172.21.15.39:0/1649254080 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x557069851f60 0x557069c7a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:42.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.843+0000 7fd942a00640 1 --2- 172.21.15.39:0/1649254080 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x557069851f60 0x557069c7a820 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eeaeed3501f67749 server_cookie=763393b50f8012cf in_seq=0 out_seq=0 2024-09-21T07:43:42.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.843+0000 7fd923400640 10 client.8154 ms_handle_connect on v2:172.21.15.39:6832/1790581246 2024-09-21T07:43:42.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.844+0000 7fd923400640 1 -- 172.21.15.39:0/1649254080 <== mds.0 v2:172.21.15.39:6832/1790581246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557069baf7c0 con 0x557069851f60 2024-09-21T07:43:42.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.844+0000 7fd923400640 10 client.8154 handle_command_reply: tid=0 2024-09-21T07:43:42.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.844+0000 7fd950822740 1 client.8154 shutdown 2024-09-21T07:43:42.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.844+0000 7fd950822740 1 -- 172.21.15.39:0/1649254080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5570698726b0 msgr2=0x557069c39be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:42.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.844+0000 7fd950822740 1 --2- 172.21.15.39:0/1649254080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5570698726b0 0x557069c39be0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fd93006c9f0 tx=0x7fd93006ca80 comp rx=0 tx=0).stop 2024-09-21T07:43:42.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.844+0000 7fd950822740 1 -- 172.21.15.39:0/1649254080 shutdown_connections 2024-09-21T07:43:42.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.844+0000 7fd950822740 1 --2- 172.21.15.39:0/1649254080 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557069893390 0x557069c3a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:42.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.844+0000 7fd950822740 1 --2- 172.21.15.39:0/1649254080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5570698726b0 0x557069c39be0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:42.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.844+0000 7fd950822740 1 --2- 172.21.15.39:0/1649254080 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x557069851f60 0x557069c7a820 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:42.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.844+0000 7fd950822740 1 --2- 172.21.15.39:0/1649254080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55706986a4d0 0x557069c51fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:42.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.844+0000 7fd950822740 1 -- 172.21.15.39:0/1649254080 >> 172.21.15.39:0/1649254080 conn(0x5570698874e0 msgr2=0x557069c82f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:42.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.844+0000 7fd923400640 0 client.8154 ms_handle_reset on v2:172.21.15.39:6832/1790581246 2024-09-21T07:43:42.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.844+0000 7fd950822740 1 -- 172.21.15.39:0/1649254080 shutdown_connections 2024-09-21T07:43:42.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.844+0000 7fd950822740 1 -- 172.21.15.39:0/1649254080 wait complete. 2024-09-21T07:43:42.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.844+0000 7fd950822740 20 client.8154 trim_cache size 0 max 16384 2024-09-21T07:43:42.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.860+0000 7fd950822740 1 Processor -- start 2024-09-21T07:43:42.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.860+0000 7fd950822740 1 -- start start 2024-09-21T07:43:42.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.861+0000 7fd950822740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55706986a4d0 0x557069c5be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:42.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.861+0000 7fd950822740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5570698726b0 0x557069ca2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:42.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.862+0000 7fd950822740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557069893390 0x557069c53bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:42.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.862+0000 7fd950822740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd944204ea0 con 0x55706986a4d0 2024-09-21T07:43:42.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.862+0000 7fd950822740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd944205570 con 0x557069893390 2024-09-21T07:43:42.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.862+0000 7fd950822740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd9442038d0 con 0x5570698726b0 2024-09-21T07:43:42.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.862+0000 7fd942a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55706986a4d0 0x557069c5be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:42.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.862+0000 7fd942a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55706986a4d0 0x557069c5be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60698/0 (socket says 172.21.15.39:60698) 2024-09-21T07:43:42.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.862+0000 7fd942a00640 1 -- 172.21.15.39:0/452337053 learned_addr learned my addr 172.21.15.39:0/452337053 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:42.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.862+0000 7fd942a00640 1 -- 172.21.15.39:0/452337053 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5570698726b0 msgr2=0x557069ca2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:43:42.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.862+0000 7fd942a00640 1 --2- 172.21.15.39:0/452337053 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5570698726b0 0x557069ca2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:42.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.862+0000 7fd942a00640 1 -- 172.21.15.39:0/452337053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557069893390 msgr2=0x557069c53bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:43:42.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.862+0000 7fd942a00640 1 --2- 172.21.15.39:0/452337053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557069893390 0x557069c53bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:42.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.862+0000 7fd942a00640 1 -- 172.21.15.39:0/452337053 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557069c7cb10 con 0x55706986a4d0 2024-09-21T07:43:42.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.862+0000 7fd942a00640 1 --2- 172.21.15.39:0/452337053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55706986a4d0 0x557069c5be10 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fd934034040 tx=0x7fd93403d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d84f32bc8dde8e5b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:42.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.863+0000 7fd92e600640 1 -- 172.21.15.39:0/452337053 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd934035040 con 0x55706986a4d0 2024-09-21T07:43:42.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.863+0000 7fd92e600640 1 -- 172.21.15.39:0/452337053 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd934003a80 con 0x55706986a4d0 2024-09-21T07:43:42.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.863+0000 7fd92e600640 1 -- 172.21.15.39:0/452337053 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd934003cb0 con 0x55706986a4d0 2024-09-21T07:43:42.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.863+0000 7fd950822740 1 -- 172.21.15.39:0/452337053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55706986a4d0 msgr2=0x557069c5be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:42.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.863+0000 7fd950822740 1 --2- 172.21.15.39:0/452337053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55706986a4d0 0x557069c5be10 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fd934034040 tx=0x7fd93403d000 comp rx=0 tx=0).stop 2024-09-21T07:43:42.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.863+0000 7fd950822740 1 -- 172.21.15.39:0/452337053 shutdown_connections 2024-09-21T07:43:42.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.863+0000 7fd950822740 1 --2- 172.21.15.39:0/452337053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557069893390 0x557069c53bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:42.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.863+0000 7fd950822740 1 --2- 172.21.15.39:0/452337053 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5570698726b0 0x557069ca2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:42.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.864+0000 7fd950822740 1 --2- 172.21.15.39:0/452337053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55706986a4d0 0x557069c5be10 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:42.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.864+0000 7fd950822740 1 -- 172.21.15.39:0/452337053 >> 172.21.15.39:0/452337053 conn(0x5570698874e0 msgr2=0x557069c3b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:42.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.864+0000 7fd950822740 1 -- 172.21.15.39:0/452337053 shutdown_connections 2024-09-21T07:43:42.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.864+0000 7fd950822740 1 -- 172.21.15.39:0/452337053 wait complete. 2024-09-21T07:43:42.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.865+0000 7fd950822740 1 -- start start 2024-09-21T07:43:42.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.865+0000 7fd950822740 1 Processor -- start 2024-09-21T07:43:42.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.866+0000 7fd950822740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55706986a4d0 0x557069cf52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:42.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.866+0000 7fd950822740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5570698726b0 0x557069c53b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:42.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.866+0000 7fd950822740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557069893390 0x557069c54050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:42.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.866+0000 7fd950822740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd944203e90 con 0x5570698726b0 2024-09-21T07:43:42.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.866+0000 7fd950822740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557069cff6a0 con 0x557069893390 2024-09-21T07:43:42.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.866+0000 7fd950822740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x557069d02930 con 0x55706986a4d0 2024-09-21T07:43:42.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.866+0000 7fd950822740 10 client.0 _ll_register_callbacks cb 0x7fd930062af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:43:42.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.866+0000 7fd942000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5570698726b0 0x557069c53b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:42.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.866+0000 7fd942000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5570698726b0 0x557069c53b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60708/0 (socket says 172.21.15.39:60708) 2024-09-21T07:43:42.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.866+0000 7fd942000640 1 -- 172.21.15.39:0/4020881655 learned_addr learned my addr 172.21.15.39:0/4020881655 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:42.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.867+0000 7fd943400640 1 --2- 172.21.15.39:0/4020881655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557069893390 0x557069c54050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:42.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.867+0000 7fd942a00640 1 --2- 172.21.15.39:0/4020881655 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55706986a4d0 0x557069cf52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:42.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.867+0000 7fd942000640 1 -- 172.21.15.39:0/4020881655 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55706986a4d0 msgr2=0x557069cf52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:43:42.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.867+0000 7fd942000640 1 --2- 172.21.15.39:0/4020881655 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55706986a4d0 0x557069cf52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:42.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.867+0000 7fd942000640 1 -- 172.21.15.39:0/4020881655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557069893390 msgr2=0x557069c54050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:43:42.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.867+0000 7fd942000640 1 --2- 172.21.15.39:0/4020881655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557069893390 0x557069c54050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:42.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.867+0000 7fd942000640 1 -- 172.21.15.39:0/4020881655 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557069c37180 con 0x5570698726b0 2024-09-21T07:43:42.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.867+0000 7fd942000640 1 --2- 172.21.15.39:0/4020881655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5570698726b0 0x557069c53b10 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fd93005f720 tx=0x7fd9300625a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29fa59dd3e984ae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:42.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.867+0000 7fd923400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:43:42.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.867+0000 7fd923400640 1 -- 172.21.15.39:0/4020881655 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93007d270 con 0x5570698726b0 2024-09-21T07:43:42.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.867+0000 7fd923400640 1 -- 172.21.15.39:0/4020881655 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8205+0+0 (secure 0 0 0) 0x7fd93004aae0 con 0x5570698726b0 2024-09-21T07:43:42.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.867+0000 7fd950822740 1 -- 172.21.15.39:0/4020881655 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd934003a80 con 0x5570698726b0 2024-09-21T07:43:42.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.868+0000 7fd923400640 1 -- 172.21.15.39:0/4020881655 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=150) ==== 24+0+0 (secure 0 0 0) 0x7fd930067bd0 con 0x5570698726b0 2024-09-21T07:43:42.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.868+0000 7fd950822740 10 client.8160 fetch_fsmap learned FSMap version 150 2024-09-21T07:43:42.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.868+0000 7fd950822740 1 -- 172.21.15.39:0/4020881655 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=150,monmap=2+}) -- 0x557069c37180 con 0x5570698726b0 2024-09-21T07:43:42.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.868+0000 7fd923400640 1 -- 172.21.15.39:0/4020881655 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd934003a80 con 0x5570698726b0 2024-09-21T07:43:42.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.868+0000 7fd923400640 1 -- 172.21.15.39:0/4020881655 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 150) ==== 3522+0+0 (secure 0 0 0) 0x7fd930020070 con 0x5570698726b0 2024-09-21T07:43:42.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.868+0000 7fd950822740 10 client.8160 fetch_fsmap finished waiting for FSMap version 150 2024-09-21T07:43:42.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.868+0000 7fd950822740 10 client.8160 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-21T07:43:42.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.868+0000 7fd950822740 20 client.8160 populate_metadata read hostname 'smithi039' 2024-09-21T07:43:42.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.868+0000 7fd950822740 1 --2- 172.21.15.39:0/4020881655 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x557069c33830 0x557069cfe830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:42.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.868+0000 7fd950822740 4 client.8160 mds_command: new command op to 7311 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:43:42.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.868+0000 7fd950822740 1 -- 172.21.15.39:0/4020881655 --> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557069bab270 con 0x557069c33830 2024-09-21T07:43:42.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.868+0000 7fd942a00640 1 --2- 172.21.15.39:0/4020881655 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x557069c33830 0x557069cfe830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:42.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.869+0000 7fd942a00640 1 --2- 172.21.15.39:0/4020881655 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x557069c33830 0x557069cfe830 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=224cee2ac81d3e44 server_cookie=27e1ff5c2d0090cd in_seq=0 out_seq=0 2024-09-21T07:43:42.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.869+0000 7fd923400640 10 client.8160 ms_handle_connect on v2:172.21.15.39:6832/1790581246 2024-09-21T07:43:42.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.883+0000 7fd923400640 1 -- 172.21.15.39:0/4020881655 <== mds.0 v2:172.21.15.39:6832/1790581246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557069bab270 con 0x557069c33830 2024-09-21T07:43:42.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.883+0000 7fd923400640 10 client.8160 handle_command_reply: tid=0 2024-09-21T07:43:42.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.883+0000 7fd950822740 1 client.8160 shutdown 2024-09-21T07:43:42.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.883+0000 7fd950822740 1 -- 172.21.15.39:0/4020881655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5570698726b0 msgr2=0x557069c53b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:42.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.883+0000 7fd950822740 1 --2- 172.21.15.39:0/4020881655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5570698726b0 0x557069c53b10 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fd93005f720 tx=0x7fd9300625a0 comp rx=0 tx=0).stop 2024-09-21T07:43:42.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.883+0000 7fd950822740 1 -- 172.21.15.39:0/4020881655 shutdown_connections 2024-09-21T07:43:42.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.883+0000 7fd950822740 1 --2- 172.21.15.39:0/4020881655 >> [v2:172.21.15.39:6832/1790581246,v1:172.21.15.39:6833/1790581246] conn(0x557069c33830 0x557069cfe830 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:42.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.883+0000 7fd950822740 1 --2- 172.21.15.39:0/4020881655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557069893390 0x557069c54050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:42.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.883+0000 7fd950822740 1 --2- 172.21.15.39:0/4020881655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5570698726b0 0x557069c53b10 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:42.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.883+0000 7fd923400640 0 client.8160 ms_handle_reset on v2:172.21.15.39:6832/1790581246 2024-09-21T07:43:42.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.883+0000 7fd950822740 1 --2- 172.21.15.39:0/4020881655 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55706986a4d0 0x557069cf52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:42.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.883+0000 7fd950822740 1 -- 172.21.15.39:0/4020881655 >> 172.21.15.39:0/4020881655 conn(0x5570698874e0 msgr2=0x557069c51380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:42.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.884+0000 7fd950822740 1 -- 172.21.15.39:0/4020881655 shutdown_connections 2024-09-21T07:43:42.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.884+0000 7fd950822740 1 -- 172.21.15.39:0/4020881655 wait complete. 2024-09-21T07:43:42.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.884+0000 7fd950822740 20 client.8160 trim_cache size 0 max 16384 2024-09-21T07:43:42.887 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:43:42.887 INFO:teuthology.orchestra.run.smithi039.stdout: "message": "", 2024-09-21T07:43:42.887 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0 2024-09-21T07:43:42.887 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:43:42.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.885+0000 7fd943e00640 1 -- 172.21.15.39:0/829679710 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd924078bc0 msgr2=0x7fd92407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:42.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.885+0000 7fd943e00640 1 --2- 172.21.15.39:0/829679710 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd924078bc0 0x7fd92407b0d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd934033000 tx=0x7fd934004730 comp rx=0 tx=0).stop 2024-09-21T07:43:42.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.885+0000 7fd943e00640 1 -- 172.21.15.39:0/829679710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd944156d60 msgr2=0x7fd9440b3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:42.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.885+0000 7fd943e00640 1 --2- 172.21.15.39:0/829679710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd944156d60 0x7fd9440b3700 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fd93000c970 tx=0x7fd93000ce30 comp rx=0 tx=0).stop 2024-09-21T07:43:42.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.885+0000 7fd943e00640 1 -- 172.21.15.39:0/829679710 shutdown_connections 2024-09-21T07:43:42.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.885+0000 7fd943e00640 1 --2- 172.21.15.39:0/829679710 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd924078bc0 0x7fd92407b0d0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:42.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.885+0000 7fd943e00640 1 --2- 172.21.15.39:0/829679710 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd944160770 0x7fd9440ae5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:42.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.885+0000 7fd943e00640 1 --2- 172.21.15.39:0/829679710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd944156d60 0x7fd9440b3700 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:42.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.885+0000 7fd943e00640 1 --2- 172.21.15.39:0/829679710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9440af690 0x7fd9441564c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:42.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.885+0000 7fd943e00640 1 -- 172.21.15.39:0/829679710 >> 172.21.15.39:0/829679710 conn(0x7fd9440b25f0 msgr2=0x7fd9440c9f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:42.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.885+0000 7fd943e00640 1 -- 172.21.15.39:0/829679710 shutdown_connections 2024-09-21T07:43:42.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:42.885+0000 7fd943e00640 1 -- 172.21.15.39:0/829679710 wait complete. 2024-09-21T07:43:42.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:43:42.909 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir d1/dir/.snap/*) 2024-09-21T07:43:42.962 INFO:tasks.cephfs.test_snapshots:testing snapclient mds_kill_mdstable_at=3, snapserver mds_kill_mdstable_at=8 2024-09-21T07:43:42.962 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:43:43.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.186+0000 7f960c000640 1 -- 172.21.15.39:0/3097725679 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f960415cd30 msgr2=0x7f96040cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:43.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.186+0000 7f960c000640 1 --2- 172.21.15.39:0/3097725679 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f960415cd30 0x7f96040cfd10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f95f8005fd0 tx=0x7f95f802a5d0 comp rx=0 tx=0).stop 2024-09-21T07:43:43.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.186+0000 7f960c000640 1 -- 172.21.15.39:0/3097725679 shutdown_connections 2024-09-21T07:43:43.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.186+0000 7f960c000640 1 --2- 172.21.15.39:0/3097725679 >> v2:172.21.15.148:3300/0 conn(0x7f960406f800 0x7f96040cd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:43.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.186+0000 7f960c000640 1 --2- 172.21.15.39:0/3097725679 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f960415cd30 0x7f96040cfd10 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:43.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.186+0000 7f960c000640 1 -- 172.21.15.39:0/3097725679 >> 172.21.15.39:0/3097725679 conn(0x7f960406cfc0 msgr2=0x7f960406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:43.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.187+0000 7f960c000640 1 -- 172.21.15.39:0/3097725679 shutdown_connections 2024-09-21T07:43:43.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.187+0000 7f960c000640 1 -- 172.21.15.39:0/3097725679 wait complete. 2024-09-21T07:43:43.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.189+0000 7f960c000640 1 Processor -- start 2024-09-21T07:43:43.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.189+0000 7f960c000640 1 -- start start 2024-09-21T07:43:43.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.190+0000 7f960c000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f960406f800 0x7f9604145270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:43.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.190+0000 7f960c000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96040d06e0 0x7f9604145c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:43.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.190+0000 7f960c000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f960415cd30 0x7f9604146190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:43.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.191+0000 7f960c000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9604073970 con 0x7f960415cd30 2024-09-21T07:43:43.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.191+0000 7f960c000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f96040737f0 con 0x7f96040d06e0 2024-09-21T07:43:43.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.191+0000 7f960c000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f9604073af0 con 0x7f960406f800 2024-09-21T07:43:43.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.191+0000 7f960a000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f960415cd30 0x7f9604146190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:43.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.191+0000 7f960a000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f960415cd30 0x7f9604146190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60716/0 (socket says 172.21.15.39:60716) 2024-09-21T07:43:43.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.191+0000 7f960a000640 1 -- 172.21.15.39:0/1724053083 learned_addr learned my addr 172.21.15.39:0/1724053083 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:43.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.191+0000 7f9609600640 1 --2- 172.21.15.39:0/1724053083 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f960406f800 0x7f9604145270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:43.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.191+0000 7f9608c00640 1 --2- 172.21.15.39:0/1724053083 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96040d06e0 0x7f9604145c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:43.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.191+0000 7f960a000640 1 -- 172.21.15.39:0/1724053083 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f960406f800 msgr2=0x7f9604145270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:43.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.191+0000 7f960a000640 1 --2- 172.21.15.39:0/1724053083 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f960406f800 0x7f9604145270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:43.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.191+0000 7f960a000640 1 -- 172.21.15.39:0/1724053083 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96040d06e0 msgr2=0x7f9604145c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:43.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.191+0000 7f960a000640 1 --2- 172.21.15.39:0/1724053083 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96040d06e0 0x7f9604145c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:43.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.191+0000 7f960a000640 1 -- 172.21.15.39:0/1724053083 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96040a9b20 con 0x7f960415cd30 2024-09-21T07:43:43.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.192+0000 7f960a000640 1 --2- 172.21.15.39:0/1724053083 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f960415cd30 0x7f9604146190 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f95fc002230 tx=0x7f95fc00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:43.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.192+0000 7f9601600640 1 -- 172.21.15.39:0/1724053083 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc003b70 con 0x7f960415cd30 2024-09-21T07:43:43.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.192+0000 7f9601600640 1 -- 172.21.15.39:0/1724053083 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95fc003d10 con 0x7f960415cd30 2024-09-21T07:43:43.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.192+0000 7f960c000640 1 -- 172.21.15.39:0/1724053083 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96040a9e00 con 0x7f960415cd30 2024-09-21T07:43:43.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.192+0000 7f9601600640 1 -- 172.21.15.39:0/1724053083 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc005330 con 0x7f960415cd30 2024-09-21T07:43:43.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.192+0000 7f960c000640 1 -- 172.21.15.39:0/1724053083 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96040aa3c0 con 0x7f960415cd30 2024-09-21T07:43:43.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.193+0000 7f960c000640 1 -- 172.21.15.39:0/1724053083 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9604073fa0 con 0x7f960415cd30 2024-09-21T07:43:43.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.194+0000 7f9601600640 1 -- 172.21.15.39:0/1724053083 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f95fc0056e0 con 0x7f960415cd30 2024-09-21T07:43:43.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.195+0000 7f9601600640 1 --2- 172.21.15.39:0/1724053083 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f95e4078b90 0x7f95e407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:43.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.195+0000 7f9601600640 1 -- 172.21.15.39:0/1724053083 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8205+0+0 (secure 0 0 0) 0x7f95fc09b5d0 con 0x7f960415cd30 2024-09-21T07:43:43.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.196+0000 7f9609600640 1 --2- 172.21.15.39:0/1724053083 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f95e4078b90 0x7f95e407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:43.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.197+0000 7f9609600640 1 --2- 172.21.15.39:0/1724053083 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f95e4078b90 0x7f95e407b0a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f95f8033000 tx=0x7f95f8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:43.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.200+0000 7f9601600640 1 -- 172.21.15.39:0/1724053083 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f95fc0629e0 con 0x7f960415cd30 2024-09-21T07:43:43.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.435+0000 7f960c000640 1 -- 172.21.15.39:0/1724053083 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f960406a7d0 con 0x7f960415cd30 2024-09-21T07:43:43.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.436+0000 7f9601600640 1 -- 172.21.15.39:0/1724053083 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 150 v150) ==== 95+0+5256 (secure 0 0 0) 0x7f95fc067410 con 0x7f960415cd30 2024-09-21T07:43:43.438 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:43:43.438 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":150,"btime":"2024-09-21T07:43:38:142009+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":8052,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1551311805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1551311805},{"type":"v1","addr":"172.21.15.39:6835","nonce":1551311805}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":147}],"filesystems":[{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:43:38.141999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7366},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7366":{"gid":7366,"name":"d","rank":1,"incarnation":146,"state":"up:active","state_seq":18,"addr":"172.21.15.148:6837/759792812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":759792812},{"type":"v1","addr":"172.21.15.148:6837","nonce":759792812}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311,7366]},"id":6}]} 2024-09-21T07:43:43.438 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 150 2024-09-21T07:43:43.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.438+0000 7f960c000640 1 -- 172.21.15.39:0/1724053083 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f95e4078b90 msgr2=0x7f95e407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:43.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.438+0000 7f960c000640 1 --2- 172.21.15.39:0/1724053083 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f95e4078b90 0x7f95e407b0a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f95f8033000 tx=0x7f95f8004730 comp rx=0 tx=0).stop 2024-09-21T07:43:43.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.438+0000 7f960c000640 1 -- 172.21.15.39:0/1724053083 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f960415cd30 msgr2=0x7f9604146190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:43.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.438+0000 7f960c000640 1 --2- 172.21.15.39:0/1724053083 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f960415cd30 0x7f9604146190 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f95fc002230 tx=0x7f95fc00cb90 comp rx=0 tx=0).stop 2024-09-21T07:43:43.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.438+0000 7f960c000640 1 -- 172.21.15.39:0/1724053083 shutdown_connections 2024-09-21T07:43:43.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.438+0000 7f960c000640 1 --2- 172.21.15.39:0/1724053083 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f95e4078b90 0x7f95e407b0a0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:43.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.438+0000 7f960c000640 1 --2- 172.21.15.39:0/1724053083 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f960415cd30 0x7f9604146190 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:43.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.438+0000 7f960c000640 1 --2- 172.21.15.39:0/1724053083 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96040d06e0 0x7f9604145c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:43.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.438+0000 7f960c000640 1 --2- 172.21.15.39:0/1724053083 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f960406f800 0x7f9604145270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:43.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.438+0000 7f960c000640 1 -- 172.21.15.39:0/1724053083 >> 172.21.15.39:0/1724053083 conn(0x7f960406cfc0 msgr2=0x7f960406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:43.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.438+0000 7f960c000640 1 -- 172.21.15.39:0/1724053083 shutdown_connections 2024-09-21T07:43:43.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.438+0000 7f960c000640 1 -- 172.21.15.39:0/1724053083 wait complete. 2024-09-21T07:43:43.462 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2024-09-21T07:43:43.706 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_snap":19,"last_created":17,"last_destroyed":18,"pending_noop":[],"snaps":[{"snapid":17,"ino":1099511627777,"stamp":"2024-09-21T07:43:06.501843+0000","name":"s37","metadata":{}}],"need_to_purge":{"16":[16,18]},"pending_update":[],"pending_destroy":[{"version":39,"removed_snap":17,"seq":19}]} 2024-09-21T07:43:43.707 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 17, "last_destroyed": 18, "last_snap": 19, "need_to_purge": { "16": [ 16, 18 ] }, "pending_destroy": [ { "removed_snap": 17, "seq": 19, "version": 39 } ], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s37", "snapid": 17, "stamp": "2024-09-21T07:43:06.501843+0000" } ] } 2024-09-21T07:43:43.707 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:43:43.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.934+0000 7f7cd2e00640 1 -- 172.21.15.39:0/187677808 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ccc15a460 msgr2=0x7f7ccc06fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:43.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.934+0000 7f7cd2e00640 1 --2- 172.21.15.39:0/187677808 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ccc15a460 0x7f7ccc06fa30 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc005fd0 tx=0x7f7cbc02a540 comp rx=0 tx=0).stop 2024-09-21T07:43:43.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.934+0000 7f7cd2e00640 1 -- 172.21.15.39:0/187677808 shutdown_connections 2024-09-21T07:43:43.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.934+0000 7f7cd2e00640 1 --2- 172.21.15.39:0/187677808 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7ccc0cd780 0x7f7ccc0cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:43.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.934+0000 7f7cd2e00640 1 --2- 172.21.15.39:0/187677808 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ccc15a460 0x7f7ccc06fa30 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:43.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.934+0000 7f7cd2e00640 1 -- 172.21.15.39:0/187677808 >> 172.21.15.39:0/187677808 conn(0x7f7ccc06cfc0 msgr2=0x7f7ccc06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:43.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.934+0000 7f7cd2e00640 1 -- 172.21.15.39:0/187677808 shutdown_connections 2024-09-21T07:43:43.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.934+0000 7f7cd2e00640 1 -- 172.21.15.39:0/187677808 wait complete. 2024-09-21T07:43:43.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.937+0000 7f7cd2e00640 1 Processor -- start 2024-09-21T07:43:43.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.937+0000 7f7cd2e00640 1 -- start start 2024-09-21T07:43:43.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.938+0000 7f7cd2e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ccc0cd780 0x7f7ccc0ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:43.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.938+0000 7f7cd2e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7ccc0d06c0 0x7f7ccc1f90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:43.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.938+0000 7f7cd2e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ccc15a460 0x7f7ccc1f9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:43.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.938+0000 7f7cd2e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7ccc15d610 con 0x7f7ccc15a460 2024-09-21T07:43:43.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.938+0000 7f7cd2e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7ccc15d490 con 0x7f7ccc0cd780 2024-09-21T07:43:43.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.938+0000 7f7cd2e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f7ccc15d790 con 0x7f7ccc0d06c0 2024-09-21T07:43:43.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.939+0000 7f7cd0e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ccc15a460 0x7f7ccc1f9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:43.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.939+0000 7f7cd0e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ccc15a460 0x7f7ccc1f9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60734/0 (socket says 172.21.15.39:60734) 2024-09-21T07:43:43.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.939+0000 7f7cd0e00640 1 -- 172.21.15.39:0/3636612817 learned_addr learned my addr 172.21.15.39:0/3636612817 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:43.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.939+0000 7f7cc3e00640 1 --2- 172.21.15.39:0/3636612817 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7ccc0d06c0 0x7f7ccc1f90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:43.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.939+0000 7f7ccbe00640 1 --2- 172.21.15.39:0/3636612817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ccc0cd780 0x7f7ccc0ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:43.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.939+0000 7f7cd0e00640 1 -- 172.21.15.39:0/3636612817 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7ccc0d06c0 msgr2=0x7f7ccc1f90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:43.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.939+0000 7f7cd0e00640 1 --2- 172.21.15.39:0/3636612817 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7ccc0d06c0 0x7f7ccc1f90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:43.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.939+0000 7f7cd0e00640 1 -- 172.21.15.39:0/3636612817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ccc0cd780 msgr2=0x7f7ccc0ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:43.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.939+0000 7f7cd0e00640 1 --2- 172.21.15.39:0/3636612817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ccc0cd780 0x7f7ccc0ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:43.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.939+0000 7f7cd0e00640 1 -- 172.21.15.39:0/3636612817 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ccc0ae190 con 0x7f7ccc15a460 2024-09-21T07:43:43.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.939+0000 7f7cd0e00640 1 --2- 172.21.15.39:0/3636612817 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ccc15a460 0x7f7ccc1f9600 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f7cc400c6e0 tx=0x7f7cc400cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:43.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.940+0000 7f7cc8c00640 1 -- 172.21.15.39:0/3636612817 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc4003b70 con 0x7f7ccc15a460 2024-09-21T07:43:43.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.940+0000 7f7cc8c00640 1 -- 172.21.15.39:0/3636612817 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7cc4003d30 con 0x7f7ccc15a460 2024-09-21T07:43:43.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.940+0000 7f7cc8c00640 1 -- 172.21.15.39:0/3636612817 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc4005330 con 0x7f7ccc15a460 2024-09-21T07:43:43.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.940+0000 7f7cd2e00640 1 -- 172.21.15.39:0/3636612817 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ccc0ae470 con 0x7f7ccc15a460 2024-09-21T07:43:43.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.940+0000 7f7cd2e00640 1 -- 172.21.15.39:0/3636612817 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ccc0aea30 con 0x7f7ccc15a460 2024-09-21T07:43:43.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.941+0000 7f7cd2e00640 1 -- 172.21.15.39:0/3636612817 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ccc069450 con 0x7f7ccc15a460 2024-09-21T07:43:43.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.942+0000 7f7cc8c00640 1 -- 172.21.15.39:0/3636612817 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7cc40054d0 con 0x7f7ccc15a460 2024-09-21T07:43:43.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.943+0000 7f7cc8c00640 1 --2- 172.21.15.39:0/3636612817 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7ca8078bc0 0x7f7ca807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:43.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.943+0000 7f7cc8c00640 1 -- 172.21.15.39:0/3636612817 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8205+0+0 (secure 0 0 0) 0x7f7cc409b400 con 0x7f7ccc15a460 2024-09-21T07:43:43.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.944+0000 7f7ccbe00640 1 --2- 172.21.15.39:0/3636612817 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7ca8078bc0 0x7f7ca807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:43.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.945+0000 7f7ccbe00640 1 --2- 172.21.15.39:0/3636612817 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7ca8078bc0 0x7f7ca807b0d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc033000 tx=0x7f7cbc004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:43.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:43.948+0000 7f7cc8c00640 1 -- 172.21.15.39:0/3636612817 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7ccc069450 con 0x7f7ccc15a460 2024-09-21T07:43:44.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.182+0000 7f7cd2e00640 1 -- 172.21.15.39:0/3636612817 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ccc0adf20 con 0x7f7ccc15a460 2024-09-21T07:43:44.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.182+0000 7f7cc8c00640 1 -- 172.21.15.39:0/3636612817 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 150 v150) ==== 95+0+5256 (secure 0 0 0) 0x7f7cc4062780 con 0x7f7ccc15a460 2024-09-21T07:43:44.184 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:43:44.184 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":150,"btime":"2024-09-21T07:43:38:142009+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":8052,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1551311805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1551311805},{"type":"v1","addr":"172.21.15.39:6835","nonce":1551311805}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":147}],"filesystems":[{"mdsmap":{"epoch":150,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:43:38.141999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7366},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7366":{"gid":7366,"name":"d","rank":1,"incarnation":146,"state":"up:active","state_seq":18,"addr":"172.21.15.148:6837/759792812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":759792812},{"type":"v1","addr":"172.21.15.148:6837","nonce":759792812}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311,7366]},"id":6}]} 2024-09-21T07:43:44.185 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 150 2024-09-21T07:43:44.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.185+0000 7f7cd2e00640 1 -- 172.21.15.39:0/3636612817 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7ca8078bc0 msgr2=0x7f7ca807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:44.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.185+0000 7f7cd2e00640 1 --2- 172.21.15.39:0/3636612817 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7ca8078bc0 0x7f7ca807b0d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc033000 tx=0x7f7cbc004730 comp rx=0 tx=0).stop 2024-09-21T07:43:44.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.185+0000 7f7cd2e00640 1 -- 172.21.15.39:0/3636612817 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ccc15a460 msgr2=0x7f7ccc1f9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:44.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.185+0000 7f7cd2e00640 1 --2- 172.21.15.39:0/3636612817 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ccc15a460 0x7f7ccc1f9600 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f7cc400c6e0 tx=0x7f7cc400cba0 comp rx=0 tx=0).stop 2024-09-21T07:43:44.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.185+0000 7f7cd2e00640 1 -- 172.21.15.39:0/3636612817 shutdown_connections 2024-09-21T07:43:44.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.185+0000 7f7cd2e00640 1 --2- 172.21.15.39:0/3636612817 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7ca8078bc0 0x7f7ca807b0d0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:44.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.185+0000 7f7cd2e00640 1 --2- 172.21.15.39:0/3636612817 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ccc15a460 0x7f7ccc1f9600 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:44.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.185+0000 7f7cd2e00640 1 --2- 172.21.15.39:0/3636612817 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7ccc0d06c0 0x7f7ccc1f90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:44.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.185+0000 7f7cd2e00640 1 --2- 172.21.15.39:0/3636612817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ccc0cd780 0x7f7ccc0ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:44.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.185+0000 7f7cd2e00640 1 -- 172.21.15.39:0/3636612817 >> 172.21.15.39:0/3636612817 conn(0x7f7ccc06cfc0 msgr2=0x7f7ccc06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:44.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.185+0000 7f7cd2e00640 1 -- 172.21.15.39:0/3636612817 shutdown_connections 2024-09-21T07:43:44.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.185+0000 7f7cd2e00640 1 -- 172.21.15.39:0/3636612817 wait complete. 2024-09-21T07:43:44.211 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:0 true 2024-09-21T07:43:44.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.439+0000 7fbaddc00640 1 -- 172.21.15.39:0/3593289744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbabc004d40 msgr2=0x7fbabc005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:44.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.439+0000 7fbaddc00640 1 --2- 172.21.15.39:0/3593289744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbabc004d40 0x7fbabc005150 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fbad0005fd0 tx=0x7fbabc005830 comp rx=0 tx=0).stop 2024-09-21T07:43:44.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.439+0000 7fbadd200640 1 -- 172.21.15.39:0/3593289744 reap_dead start 2024-09-21T07:43:44.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.439+0000 7fbaddc00640 1 -- 172.21.15.39:0/3593289744 shutdown_connections 2024-09-21T07:43:44.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.439+0000 7fbaddc00640 1 -- 172.21.15.39:0/3593289744 >> 172.21.15.39:0/3593289744 conn(0x7fbad80b25f0 msgr2=0x7fbad80d01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:44.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.439+0000 7fbaddc00640 1 -- 172.21.15.39:0/3593289744 shutdown_connections 2024-09-21T07:43:44.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.439+0000 7fbaddc00640 1 -- 172.21.15.39:0/3593289744 wait complete. 2024-09-21T07:43:44.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.442+0000 7fbaddc00640 1 Processor -- start 2024-09-21T07:43:44.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.442+0000 7fbaddc00640 1 -- start start 2024-09-21T07:43:44.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.443+0000 7fbaddc00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbabc004d40 0x7fbad80c7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:44.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.443+0000 7fbaddc00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbad8161990 0x7fbad80c9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:44.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.443+0000 7fbaddc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbad80c9760 0x7fbad8160250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:44.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.443+0000 7fbaddc00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbad80b3760 con 0x7fbad80c9760 2024-09-21T07:43:44.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.443+0000 7fbaddc00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbad80b35e0 con 0x7fbabc004d40 2024-09-21T07:43:44.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.443+0000 7fbaddc00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fbad80b38e0 con 0x7fbad8161990 2024-09-21T07:43:44.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.443+0000 7fbadd200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbad80c9760 0x7fbad8160250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:44.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.443+0000 7fbadd200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbad80c9760 0x7fbad8160250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60748/0 (socket says 172.21.15.39:60748) 2024-09-21T07:43:44.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.443+0000 7fbadd200640 1 -- 172.21.15.39:0/3576727027 learned_addr learned my addr 172.21.15.39:0/3576727027 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:44.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.443+0000 7fbad7e00640 1 --2- 172.21.15.39:0/3576727027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbabc004d40 0x7fbad80c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:44.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.444+0000 7fbacf400640 1 --2- 172.21.15.39:0/3576727027 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbad8161990 0x7fbad80c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:44.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.444+0000 7fbadd200640 1 -- 172.21.15.39:0/3576727027 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbad8161990 msgr2=0x7fbad80c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:44.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.444+0000 7fbadd200640 1 --2- 172.21.15.39:0/3576727027 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbad8161990 0x7fbad80c9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:44.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.444+0000 7fbadd200640 1 -- 172.21.15.39:0/3576727027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbabc004d40 msgr2=0x7fbad80c7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:44.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.444+0000 7fbadd200640 1 --2- 172.21.15.39:0/3576727027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbabc004d40 0x7fbad80c7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:44.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.444+0000 7fbadd200640 1 -- 172.21.15.39:0/3576727027 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad815f1b0 con 0x7fbad80c9760 2024-09-21T07:43:44.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.444+0000 7fbadd200640 1 --2- 172.21.15.39:0/3576727027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbad80c9760 0x7fbad8160250 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fbad002aec0 tx=0x7fbad0033b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:44.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.444+0000 7fbad4c00640 1 -- 172.21.15.39:0/3576727027 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad0038040 con 0x7fbad80c9760 2024-09-21T07:43:44.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.444+0000 7fbad4c00640 1 -- 172.21.15.39:0/3576727027 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbad002bc30 con 0x7fbad80c9760 2024-09-21T07:43:44.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.445+0000 7fbad4c00640 1 -- 172.21.15.39:0/3576727027 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad0009200 con 0x7fbad80c9760 2024-09-21T07:43:44.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.445+0000 7fbaddc00640 1 -- 172.21.15.39:0/3576727027 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad815f490 con 0x7fbad80c9760 2024-09-21T07:43:44.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.445+0000 7fbaddc00640 1 -- 172.21.15.39:0/3576727027 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad815fa50 con 0x7fbad80c9760 2024-09-21T07:43:44.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.446+0000 7fbaddc00640 1 -- 172.21.15.39:0/3576727027 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbad8050f60 con 0x7fbad80c9760 2024-09-21T07:43:44.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.447+0000 7fbad4c00640 1 -- 172.21.15.39:0/3576727027 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbad0044030 con 0x7fbad80c9760 2024-09-21T07:43:44.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.448+0000 7fbad4c00640 1 --2- 172.21.15.39:0/3576727027 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbab4078b90 0x7fbab407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:44.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.448+0000 7fbad4c00640 1 -- 172.21.15.39:0/3576727027 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8293+0+0 (secure 0 0 0) 0x7fbad00bb890 con 0x7fbad80c9760 2024-09-21T07:43:44.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.448+0000 7fbad7e00640 1 --2- 172.21.15.39:0/3576727027 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbab4078b90 0x7fbab407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:44.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.449+0000 7fbad7e00640 1 --2- 172.21.15.39:0/3576727027 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbab4078b90 0x7fbab407b0a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fbac8009c50 tx=0x7fbac8005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:44.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.452+0000 7fbad4c00640 1 -- 172.21.15.39:0/3576727027 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbad8050f60 con 0x7fbad80c9760 2024-09-21T07:43:44.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:44.734+0000 7fbaddc00640 1 -- 172.21.15.39:0/3576727027 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"} v 0) -- 0x7fbad80b30a0 con 0x7fbad80c9760 2024-09-21T07:43:45.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.190+0000 7fbad4c00640 1 -- 172.21.15.39:0/3576727027 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"}]=0 freezing mds.7311 v151) ==== 112+0+0 (secure 0 0 0) 0x7fbad004b060 con 0x7fbad80c9760 2024-09-21T07:43:45.193 INFO:teuthology.orchestra.run.smithi039.stderr:freezing mds.7311 2024-09-21T07:43:45.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.193+0000 7fbaddc00640 1 -- 172.21.15.39:0/3576727027 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbab4078b90 msgr2=0x7fbab407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:45.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.193+0000 7fbaddc00640 1 --2- 172.21.15.39:0/3576727027 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbab4078b90 0x7fbab407b0a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fbac8009c50 tx=0x7fbac8005f90 comp rx=0 tx=0).stop 2024-09-21T07:43:45.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.193+0000 7fbaddc00640 1 -- 172.21.15.39:0/3576727027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbad80c9760 msgr2=0x7fbad8160250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:45.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.193+0000 7fbaddc00640 1 --2- 172.21.15.39:0/3576727027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbad80c9760 0x7fbad8160250 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fbad002aec0 tx=0x7fbad0033b80 comp rx=0 tx=0).stop 2024-09-21T07:43:45.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.194+0000 7fbaddc00640 1 -- 172.21.15.39:0/3576727027 shutdown_connections 2024-09-21T07:43:45.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.194+0000 7fbaddc00640 1 --2- 172.21.15.39:0/3576727027 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbab4078b90 0x7fbab407b0a0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:45.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.194+0000 7fbaddc00640 1 --2- 172.21.15.39:0/3576727027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbad80c9760 0x7fbad8160250 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:45.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.194+0000 7fbaddc00640 1 --2- 172.21.15.39:0/3576727027 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbad8161990 0x7fbad80c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:45.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.194+0000 7fbaddc00640 1 --2- 172.21.15.39:0/3576727027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbabc004d40 0x7fbad80c7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:45.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.194+0000 7fbaddc00640 1 -- 172.21.15.39:0/3576727027 >> 172.21.15.39:0/3576727027 conn(0x7fbad80b25f0 msgr2=0x7fbad815d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:45.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.194+0000 7fbaddc00640 1 -- 172.21.15.39:0/3576727027 shutdown_connections 2024-09-21T07:43:45.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.194+0000 7fbaddc00640 1 -- 172.21.15.39:0/3576727027 wait complete. 2024-09-21T07:43:45.227 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:1 true 2024-09-21T07:43:45.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.455+0000 7f9edb200640 1 -- 172.21.15.39:0/1317987567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ec0004800 msgr2=0x7f9ec0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:45.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.455+0000 7f9edb200640 1 --2- 172.21.15.39:0/1317987567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ec0004800 0x7f9ec0004c10 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f9ec400c310 tx=0x7f9ec0005830 comp rx=0 tx=0).stop 2024-09-21T07:43:45.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.455+0000 7f9edb200640 1 -- 172.21.15.39:0/1317987567 shutdown_connections 2024-09-21T07:43:45.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.455+0000 7f9edb200640 1 --2- 172.21.15.39:0/1317987567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ec0004800 0x7f9ec0004c10 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:45.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.455+0000 7f9edb200640 1 -- 172.21.15.39:0/1317987567 >> 172.21.15.39:0/1317987567 conn(0x7f9edc0b25f0 msgr2=0x7f9edc0d01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:45.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.455+0000 7f9edb200640 1 -- 172.21.15.39:0/1317987567 shutdown_connections 2024-09-21T07:43:45.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.455+0000 7f9edb200640 1 -- 172.21.15.39:0/1317987567 wait complete. 2024-09-21T07:43:45.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.457+0000 7f9edb200640 1 Processor -- start 2024-09-21T07:43:45.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.457+0000 7f9edb200640 1 -- start start 2024-09-21T07:43:45.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.459+0000 7f9edb200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ec0004800 0x7f9edc0cf500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:45.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.459+0000 7f9edb200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9edc0b4520 0x7f9edc06fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:45.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.459+0000 7f9edb200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9edc15f630 0x7f9edc15fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:45.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.459+0000 7f9edb200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc160230 con 0x7f9edc0b4520 2024-09-21T07:43:45.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.459+0000 7f9edb200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc0cc850 con 0x7f9ec0004800 2024-09-21T07:43:45.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.459+0000 7f9edb200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f9edc1603b0 con 0x7f9edc15f630 2024-09-21T07:43:45.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.459+0000 7f9ed3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9edc0b4520 0x7f9edc06fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:45.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.459+0000 7f9ed3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9edc0b4520 0x7f9edc06fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60764/0 (socket says 172.21.15.39:60764) 2024-09-21T07:43:45.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.459+0000 7f9ed3e00640 1 -- 172.21.15.39:0/4081130330 learned_addr learned my addr 172.21.15.39:0/4081130330 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:45.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.459+0000 7f9ed9e00640 1 --2- 172.21.15.39:0/4081130330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ec0004800 0x7f9edc0cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:45.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.459+0000 7f9eda800640 1 --2- 172.21.15.39:0/4081130330 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9edc15f630 0x7f9edc15fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:45.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.460+0000 7f9ed3e00640 1 -- 172.21.15.39:0/4081130330 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9edc15f630 msgr2=0x7f9edc15fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:45.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.460+0000 7f9ed3e00640 1 --2- 172.21.15.39:0/4081130330 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9edc15f630 0x7f9edc15fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:45.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.460+0000 7f9ed3e00640 1 -- 172.21.15.39:0/4081130330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ec0004800 msgr2=0x7f9edc0cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:45.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.460+0000 7f9ed3e00640 1 --2- 172.21.15.39:0/4081130330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ec0004800 0x7f9edc0cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:45.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.460+0000 7f9ed3e00640 1 -- 172.21.15.39:0/4081130330 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9edc0cfa40 con 0x7f9edc0b4520 2024-09-21T07:43:45.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.460+0000 7f9ed3e00640 1 --2- 172.21.15.39:0/4081130330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9edc0b4520 0x7f9edc06fbb0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f9ec402b160 tx=0x7f9ec4001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:45.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.460+0000 7f9ed2000640 1 -- 172.21.15.39:0/4081130330 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4031040 con 0x7f9edc0b4520 2024-09-21T07:43:45.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.460+0000 7f9ed2000640 1 -- 172.21.15.39:0/4081130330 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ec40065e0 con 0x7f9edc0b4520 2024-09-21T07:43:45.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.460+0000 7f9edb200640 1 -- 172.21.15.39:0/4081130330 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9edc0700f0 con 0x7f9edc0b4520 2024-09-21T07:43:45.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.460+0000 7f9ed2000640 1 -- 172.21.15.39:0/4081130330 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4003a60 con 0x7f9edc0b4520 2024-09-21T07:43:45.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.460+0000 7f9edb200640 1 -- 172.21.15.39:0/4081130330 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9edc15eb50 con 0x7f9edc0b4520 2024-09-21T07:43:45.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.461+0000 7f9edb200640 1 -- 172.21.15.39:0/4081130330 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9edc160910 con 0x7f9edc0b4520 2024-09-21T07:43:45.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.463+0000 7f9ed2000640 1 -- 172.21.15.39:0/4081130330 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9ec403e030 con 0x7f9edc0b4520 2024-09-21T07:43:45.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.464+0000 7f9ed2000640 1 --2- 172.21.15.39:0/4081130330 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9eb8078b90 0x7f9eb807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:45.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.464+0000 7f9ed2000640 1 -- 172.21.15.39:0/4081130330 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8277+0+0 (secure 0 0 0) 0x7f9ec40bb570 con 0x7f9edc0b4520 2024-09-21T07:43:45.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.464+0000 7f9ed9e00640 1 --2- 172.21.15.39:0/4081130330 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9eb8078b90 0x7f9eb807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:45.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.465+0000 7f9ed9e00640 1 --2- 172.21.15.39:0/4081130330 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9eb8078b90 0x7f9eb807b0a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc009c10 tx=0x7f9ecc005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:45.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.468+0000 7f9ed2000640 1 -- 172.21.15.39:0/4081130330 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9ec4082ef0 con 0x7f9edc0b4520 2024-09-21T07:43:45.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:45.745+0000 7f9edb200640 1 -- 172.21.15.39:0/4081130330 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:1", "val": "true"} v 0) -- 0x7f9edc0aec70 con 0x7f9edc0b4520 2024-09-21T07:43:46.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:46.207+0000 7f9ed2000640 1 -- 172.21.15.39:0/4081130330 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:1", "val": "true"}]=0 freezing mds.7366 v152) ==== 112+0+0 (secure 0 0 0) 0x7f9ec40827f0 con 0x7f9edc0b4520 2024-09-21T07:43:46.209 INFO:teuthology.orchestra.run.smithi039.stderr:freezing mds.7366 2024-09-21T07:43:46.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:46.210+0000 7f9edb200640 1 -- 172.21.15.39:0/4081130330 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9eb8078b90 msgr2=0x7f9eb807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:46.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:46.210+0000 7f9edb200640 1 --2- 172.21.15.39:0/4081130330 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9eb8078b90 0x7f9eb807b0a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc009c10 tx=0x7f9ecc005f90 comp rx=0 tx=0).stop 2024-09-21T07:43:46.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:46.210+0000 7f9edb200640 1 -- 172.21.15.39:0/4081130330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9edc0b4520 msgr2=0x7f9edc06fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:46.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:46.210+0000 7f9edb200640 1 --2- 172.21.15.39:0/4081130330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9edc0b4520 0x7f9edc06fbb0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f9ec402b160 tx=0x7f9ec4001d80 comp rx=0 tx=0).stop 2024-09-21T07:43:46.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:46.211+0000 7f9edb200640 1 -- 172.21.15.39:0/4081130330 shutdown_connections 2024-09-21T07:43:46.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:46.211+0000 7f9edb200640 1 --2- 172.21.15.39:0/4081130330 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9eb8078b90 0x7f9eb807b0a0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:46.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:46.211+0000 7f9edb200640 1 --2- 172.21.15.39:0/4081130330 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9edc15f630 0x7f9edc15fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:46.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:46.211+0000 7f9edb200640 1 --2- 172.21.15.39:0/4081130330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9edc0b4520 0x7f9edc06fbb0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:46.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:46.211+0000 7f9edb200640 1 --2- 172.21.15.39:0/4081130330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ec0004800 0x7f9edc0cf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:46.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:46.211+0000 7f9edb200640 1 -- 172.21.15.39:0/4081130330 >> 172.21.15.39:0/4081130330 conn(0x7f9edc0b25f0 msgr2=0x7f9edc156120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:46.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:46.211+0000 7f9edb200640 1 -- 172.21.15.39:0/4081130330 shutdown_connections 2024-09-21T07:43:46.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:46.211+0000 7f9edb200640 1 -- 172.21.15.39:0/4081130330 wait complete. 2024-09-21T07:43:46.239 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json config set mds_kill_mdstable_at 8 2024-09-21T07:43:46.480 INFO:teuthology.orchestra.run.smithi039.stdout:{"success":"mds_kill_mdstable_at = '' (not observed, change may require restart) "} 2024-09-21T07:43:46.481 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_kill_mdstable_at = '' (not observed, change may require restart) " } 2024-09-21T07:43:46.481 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json config set mds_kill_mdstable_at 3 2024-09-21T07:43:46.728 INFO:teuthology.orchestra.run.smithi148.stdout:{"success":"mds_kill_mdstable_at = '' (not observed, change may require restart) "} 2024-09-21T07:43:46.728 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_kill_mdstable_at = '' (not observed, change may require restart) " } 2024-09-21T07:43:46.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:43:46.729 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d1/dir/.snap/s4) 2024-09-21T07:43:46.733 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-09-21T07:43:46.768 INFO:tasks.ceph.mds.d.smithi148.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::handle_request(ceph::cref_t&)' thread 7f5e4d400640 time 2024-09-21T07:43:46.767426+0000 2024-09-21T07:43:46.768 INFO:tasks.ceph.mds.d.smithi148.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: 76: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 3) 2024-09-21T07:43:46.768 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:43:46.768 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f5e5445654b] 2024-09-21T07:43:46.769 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f5e54456758] 2024-09-21T07:43:46.769 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x506) [0x55f2d3cb670a] 2024-09-21T07:43:46.769 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xc96) [0x55f2d394edd4] 2024-09-21T07:43:46.769 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55f2d3952523] 2024-09-21T07:43:46.769 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55f2d3952e4a] 2024-09-21T07:43:46.769 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55f2d3935fa4] 2024-09-21T07:43:46.769 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f5e545a57df] 2024-09-21T07:43:46.769 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: (DispatchQueue::entry()+0x60f) [0x7f5e545a60d1] 2024-09-21T07:43:46.769 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f5e54693f5b] 2024-09-21T07:43:46.769 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (Thread::entry_wrapper()+0x43) [0x7f5e5442fd57] 2024-09-21T07:43:46.769 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: (Thread::_entry_func(void*)+0xd) [0x7f5e5442fd73] 2024-09-21T07:43:46.769 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: /lib64/libc.so.6(+0x89c52) [0x7f5e53a89c52] 2024-09-21T07:43:46.769 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: /lib64/libc.so.6(+0x10ec80) [0x7f5e53b0ec80] 2024-09-21T07:43:46.769 INFO:tasks.ceph.mds.d.smithi148.stderr:*** Caught signal (Aborted) ** 2024-09-21T07:43:46.770 INFO:tasks.ceph.mds.d.smithi148.stderr: in thread 7f5e4d400640 thread_name:ms_dispatch 2024-09-21T07:43:46.770 INFO:tasks.ceph.mds.d.smithi148.stderr:2024-09-21T07:43:46.766+0000 7f5e4d400640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::handle_request(ceph::cref_t&)' thread 7f5e4d400640 time 2024-09-21T07:43:46.767426+0000 2024-09-21T07:43:46.770 INFO:tasks.ceph.mds.d.smithi148.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: 76: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 3) 2024-09-21T07:43:46.770 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:43:46.770 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:43:46.770 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f5e5445654b] 2024-09-21T07:43:46.770 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f5e54456758] 2024-09-21T07:43:46.770 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x506) [0x55f2d3cb670a] 2024-09-21T07:43:46.770 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xc96) [0x55f2d394edd4] 2024-09-21T07:43:46.770 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55f2d3952523] 2024-09-21T07:43:46.770 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55f2d3952e4a] 2024-09-21T07:43:46.770 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55f2d3935fa4] 2024-09-21T07:43:46.770 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f5e545a57df] 2024-09-21T07:43:46.771 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: (DispatchQueue::entry()+0x60f) [0x7f5e545a60d1] 2024-09-21T07:43:46.771 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f5e54693f5b] 2024-09-21T07:43:46.771 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (Thread::entry_wrapper()+0x43) [0x7f5e5442fd57] 2024-09-21T07:43:46.771 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: (Thread::_entry_func(void*)+0xd) [0x7f5e5442fd73] 2024-09-21T07:43:46.771 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: /lib64/libc.so.6(+0x89c52) [0x7f5e53a89c52] 2024-09-21T07:43:46.771 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: /lib64/libc.so.6(+0x10ec80) [0x7f5e53b0ec80] 2024-09-21T07:43:46.771 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:43:46.771 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:43:46.772 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f5e53a3e6f0] 2024-09-21T07:43:46.772 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f5e53a8b99c] 2024-09-21T07:43:46.772 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: raise() 2024-09-21T07:43:46.772 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: abort() 2024-09-21T07:43:46.772 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7f5e54456686] 2024-09-21T07:43:46.772 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f5e54456758] 2024-09-21T07:43:46.772 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x506) [0x55f2d3cb670a] 2024-09-21T07:43:46.772 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xc96) [0x55f2d394edd4] 2024-09-21T07:43:46.772 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55f2d3952523] 2024-09-21T07:43:46.772 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55f2d3952e4a] 2024-09-21T07:43:46.772 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55f2d3935fa4] 2024-09-21T07:43:46.772 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f5e545a57df] 2024-09-21T07:43:46.772 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: (DispatchQueue::entry()+0x60f) [0x7f5e545a60d1] 2024-09-21T07:43:46.772 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f5e54693f5b] 2024-09-21T07:43:46.772 INFO:tasks.ceph.mds.d.smithi148.stderr: 15: (Thread::entry_wrapper()+0x43) [0x7f5e5442fd57] 2024-09-21T07:43:46.773 INFO:tasks.ceph.mds.d.smithi148.stderr: 16: (Thread::_entry_func(void*)+0xd) [0x7f5e5442fd73] 2024-09-21T07:43:46.773 INFO:tasks.ceph.mds.d.smithi148.stderr: 17: /lib64/libc.so.6(+0x89c52) [0x7f5e53a89c52] 2024-09-21T07:43:46.773 INFO:tasks.ceph.mds.d.smithi148.stderr: 18: /lib64/libc.so.6(+0x10ec80) [0x7f5e53b0ec80] 2024-09-21T07:43:46.773 INFO:tasks.ceph.mds.d.smithi148.stderr:2024-09-21T07:43:46.767+0000 7f5e4d400640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:43:46.773 INFO:tasks.ceph.mds.d.smithi148.stderr: in thread 7f5e4d400640 thread_name:ms_dispatch 2024-09-21T07:43:46.773 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:43:46.773 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:43:46.773 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f5e53a3e6f0] 2024-09-21T07:43:46.773 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f5e53a8b99c] 2024-09-21T07:43:46.773 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: raise() 2024-09-21T07:43:46.773 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: abort() 2024-09-21T07:43:46.773 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7f5e54456686] 2024-09-21T07:43:46.773 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f5e54456758] 2024-09-21T07:43:46.773 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x506) [0x55f2d3cb670a] 2024-09-21T07:43:46.773 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xc96) [0x55f2d394edd4] 2024-09-21T07:43:46.774 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55f2d3952523] 2024-09-21T07:43:46.774 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55f2d3952e4a] 2024-09-21T07:43:46.774 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55f2d3935fa4] 2024-09-21T07:43:46.774 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f5e545a57df] 2024-09-21T07:43:46.774 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: (DispatchQueue::entry()+0x60f) [0x7f5e545a60d1] 2024-09-21T07:43:46.774 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f5e54693f5b] 2024-09-21T07:43:46.774 INFO:tasks.ceph.mds.d.smithi148.stderr: 15: (Thread::entry_wrapper()+0x43) [0x7f5e5442fd57] 2024-09-21T07:43:46.774 INFO:tasks.ceph.mds.d.smithi148.stderr: 16: (Thread::_entry_func(void*)+0xd) [0x7f5e5442fd73] 2024-09-21T07:43:46.774 INFO:tasks.ceph.mds.d.smithi148.stderr: 17: /lib64/libc.so.6(+0x89c52) [0x7f5e53a89c52] 2024-09-21T07:43:46.774 INFO:tasks.ceph.mds.d.smithi148.stderr: 18: /lib64/libc.so.6(+0x10ec80) [0x7f5e53b0ec80] 2024-09-21T07:43:46.774 INFO:tasks.ceph.mds.d.smithi148.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:43:46.774 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:43:46.782 INFO:tasks.ceph.mds.d.smithi148.stderr: -1> 2024-09-21T07:43:46.766+0000 7f5e4d400640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::handle_request(ceph::cref_t&)' thread 7f5e4d400640 time 2024-09-21T07:43:46.767426+0000 2024-09-21T07:43:46.782 INFO:tasks.ceph.mds.d.smithi148.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: 76: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 3) 2024-09-21T07:43:46.782 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:43:46.782 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:43:46.782 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f5e5445654b] 2024-09-21T07:43:46.782 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f5e54456758] 2024-09-21T07:43:46.782 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x506) [0x55f2d3cb670a] 2024-09-21T07:43:46.782 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xc96) [0x55f2d394edd4] 2024-09-21T07:43:46.782 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55f2d3952523] 2024-09-21T07:43:46.783 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55f2d3952e4a] 2024-09-21T07:43:46.783 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55f2d3935fa4] 2024-09-21T07:43:46.783 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f5e545a57df] 2024-09-21T07:43:46.783 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: (DispatchQueue::entry()+0x60f) [0x7f5e545a60d1] 2024-09-21T07:43:46.783 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f5e54693f5b] 2024-09-21T07:43:46.783 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (Thread::entry_wrapper()+0x43) [0x7f5e5442fd57] 2024-09-21T07:43:46.783 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: (Thread::_entry_func(void*)+0xd) [0x7f5e5442fd73] 2024-09-21T07:43:46.783 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: /lib64/libc.so.6(+0x89c52) [0x7f5e53a89c52] 2024-09-21T07:43:46.783 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: /lib64/libc.so.6(+0x10ec80) [0x7f5e53b0ec80] 2024-09-21T07:43:46.783 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:43:46.783 INFO:tasks.ceph.mds.d.smithi148.stderr: 0> 2024-09-21T07:43:46.767+0000 7f5e4d400640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:43:46.783 INFO:tasks.ceph.mds.d.smithi148.stderr: in thread 7f5e4d400640 thread_name:ms_dispatch 2024-09-21T07:43:46.784 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:43:46.784 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:43:46.784 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f5e53a3e6f0] 2024-09-21T07:43:46.784 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f5e53a8b99c] 2024-09-21T07:43:46.784 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: raise() 2024-09-21T07:43:46.785 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: abort() 2024-09-21T07:43:46.785 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7f5e54456686] 2024-09-21T07:43:46.785 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f5e54456758] 2024-09-21T07:43:46.785 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x506) [0x55f2d3cb670a] 2024-09-21T07:43:46.786 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xc96) [0x55f2d394edd4] 2024-09-21T07:43:46.786 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55f2d3952523] 2024-09-21T07:43:46.786 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55f2d3952e4a] 2024-09-21T07:43:46.786 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55f2d3935fa4] 2024-09-21T07:43:46.786 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f5e545a57df] 2024-09-21T07:43:46.786 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: (DispatchQueue::entry()+0x60f) [0x7f5e545a60d1] 2024-09-21T07:43:46.786 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f5e54693f5b] 2024-09-21T07:43:46.786 INFO:tasks.ceph.mds.d.smithi148.stderr: 15: (Thread::entry_wrapper()+0x43) [0x7f5e5442fd57] 2024-09-21T07:43:46.786 INFO:tasks.ceph.mds.d.smithi148.stderr: 16: (Thread::_entry_func(void*)+0xd) [0x7f5e5442fd73] 2024-09-21T07:43:46.786 INFO:tasks.ceph.mds.d.smithi148.stderr: 17: /lib64/libc.so.6(+0x89c52) [0x7f5e53a89c52] 2024-09-21T07:43:46.786 INFO:tasks.ceph.mds.d.smithi148.stderr: 18: /lib64/libc.so.6(+0x10ec80) [0x7f5e53b0ec80] 2024-09-21T07:43:46.786 INFO:tasks.ceph.mds.d.smithi148.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:43:46.786 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:43:46.789 INFO:tasks.ceph.mds.d.smithi148.stderr: -1> 2024-09-21T07:43:46.766+0000 7f5e4d400640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::handle_request(ceph::cref_t&)' thread 7f5e4d400640 time 2024-09-21T07:43:46.767426+0000 2024-09-21T07:43:46.789 INFO:tasks.ceph.mds.d.smithi148.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: 76: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 3) 2024-09-21T07:43:46.789 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:43:46.789 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:43:46.789 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f5e5445654b] 2024-09-21T07:43:46.789 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f5e54456758] 2024-09-21T07:43:46.789 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x506) [0x55f2d3cb670a] 2024-09-21T07:43:46.789 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xc96) [0x55f2d394edd4] 2024-09-21T07:43:46.789 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55f2d3952523] 2024-09-21T07:43:46.789 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55f2d3952e4a] 2024-09-21T07:43:46.789 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55f2d3935fa4] 2024-09-21T07:43:46.789 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f5e545a57df] 2024-09-21T07:43:46.789 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: (DispatchQueue::entry()+0x60f) [0x7f5e545a60d1] 2024-09-21T07:43:46.790 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f5e54693f5b] 2024-09-21T07:43:46.790 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (Thread::entry_wrapper()+0x43) [0x7f5e5442fd57] 2024-09-21T07:43:46.790 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: (Thread::_entry_func(void*)+0xd) [0x7f5e5442fd73] 2024-09-21T07:43:46.790 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: /lib64/libc.so.6(+0x89c52) [0x7f5e53a89c52] 2024-09-21T07:43:46.790 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: /lib64/libc.so.6(+0x10ec80) [0x7f5e53b0ec80] 2024-09-21T07:43:46.790 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:43:46.790 INFO:tasks.ceph.mds.d.smithi148.stderr: 0> 2024-09-21T07:43:46.767+0000 7f5e4d400640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:43:46.790 INFO:tasks.ceph.mds.d.smithi148.stderr: in thread 7f5e4d400640 thread_name:ms_dispatch 2024-09-21T07:43:46.790 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:43:46.790 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:43:46.790 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f5e53a3e6f0] 2024-09-21T07:43:46.790 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f5e53a8b99c] 2024-09-21T07:43:46.790 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: raise() 2024-09-21T07:43:46.791 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: abort() 2024-09-21T07:43:46.791 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7f5e54456686] 2024-09-21T07:43:46.791 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f5e54456758] 2024-09-21T07:43:46.791 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x506) [0x55f2d3cb670a] 2024-09-21T07:43:46.791 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xc96) [0x55f2d394edd4] 2024-09-21T07:43:46.791 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55f2d3952523] 2024-09-21T07:43:46.791 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55f2d3952e4a] 2024-09-21T07:43:46.791 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55f2d3935fa4] 2024-09-21T07:43:46.791 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f5e545a57df] 2024-09-21T07:43:46.791 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: (DispatchQueue::entry()+0x60f) [0x7f5e545a60d1] 2024-09-21T07:43:46.791 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f5e54693f5b] 2024-09-21T07:43:46.791 INFO:tasks.ceph.mds.d.smithi148.stderr: 15: (Thread::entry_wrapper()+0x43) [0x7f5e5442fd57] 2024-09-21T07:43:46.791 INFO:tasks.ceph.mds.d.smithi148.stderr: 16: (Thread::_entry_func(void*)+0xd) [0x7f5e5442fd73] 2024-09-21T07:43:46.791 INFO:tasks.ceph.mds.d.smithi148.stderr: 17: /lib64/libc.so.6(+0x89c52) [0x7f5e53a89c52] 2024-09-21T07:43:46.792 INFO:tasks.ceph.mds.d.smithi148.stderr: 18: /lib64/libc.so.6(+0x10ec80) [0x7f5e53b0ec80] 2024-09-21T07:43:46.792 INFO:tasks.ceph.mds.d.smithi148.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:43:46.792 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:43:47.008 INFO:teuthology.orchestra.run.smithi039.stdout:{"mds_beacon_grace":"15.000000"} 2024-09-21T07:43:47.009 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-09-21T07:43:47.009 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:43:47.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.235+0000 7fd06f200640 1 -- 172.21.15.39:0/2524170814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0700cbd70 msgr2=0x7fd070072fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:47.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.235+0000 7fd06f200640 1 --2- 172.21.15.39:0/2524170814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0700cbd70 0x7fd070072fa0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fd068005fd0 tx=0x7fd06802a8b0 comp rx=0 tx=0).stop 2024-09-21T07:43:47.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.235+0000 7fd06f200640 1 -- 172.21.15.39:0/2524170814 shutdown_connections 2024-09-21T07:43:47.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.235+0000 7fd06f200640 1 --2- 172.21.15.39:0/2524170814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0700cbd70 0x7fd070072fa0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:47.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.235+0000 7fd06f200640 1 --2- 172.21.15.39:0/2524170814 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd0700d0f20 0x7fd07015c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:47.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.235+0000 7fd06f200640 1 -- 172.21.15.39:0/2524170814 >> 172.21.15.39:0/2524170814 conn(0x7fd0700b2510 msgr2=0x7fd0700d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:47.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.235+0000 7fd06f200640 1 -- 172.21.15.39:0/2524170814 shutdown_connections 2024-09-21T07:43:47.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.235+0000 7fd06f200640 1 -- 172.21.15.39:0/2524170814 wait complete. 2024-09-21T07:43:47.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.238+0000 7fd06f200640 1 Processor -- start 2024-09-21T07:43:47.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.238+0000 7fd06f200640 1 -- start start 2024-09-21T07:43:47.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.239+0000 7fd06f200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0700cb960 0x7fd07009c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:47.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.239+0000 7fd06f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0700cbd70 0x7fd0701f6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:47.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.239+0000 7fd06f200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd0700d0f20 0x7fd0700c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:47.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.239+0000 7fd06f200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd0700b4570 con 0x7fd0700cbd70 2024-09-21T07:43:47.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.239+0000 7fd06f200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd0700b43f0 con 0x7fd0700cb960 2024-09-21T07:43:47.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.239+0000 7fd06f200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd0700b46f0 con 0x7fd0700d0f20 2024-09-21T07:43:47.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.240+0000 7fd065400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0700cbd70 0x7fd0701f6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:47.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.240+0000 7fd065400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0700cbd70 0x7fd0701f6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60780/0 (socket says 172.21.15.39:60780) 2024-09-21T07:43:47.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.240+0000 7fd065400640 1 -- 172.21.15.39:0/1977732212 learned_addr learned my addr 172.21.15.39:0/1977732212 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:47.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.240+0000 7fd06de00640 1 --2- 172.21.15.39:0/1977732212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0700cb960 0x7fd07009c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:47.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.240+0000 7fd06e800640 1 --2- 172.21.15.39:0/1977732212 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd0700d0f20 0x7fd0700c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:47.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.240+0000 7fd065400640 1 -- 172.21.15.39:0/1977732212 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd0700d0f20 msgr2=0x7fd0700c9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:47.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.240+0000 7fd065400640 1 --2- 172.21.15.39:0/1977732212 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd0700d0f20 0x7fd0700c9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:47.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.240+0000 7fd065400640 1 -- 172.21.15.39:0/1977732212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0700cb960 msgr2=0x7fd07009c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:47.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.240+0000 7fd065400640 1 --2- 172.21.15.39:0/1977732212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0700cb960 0x7fd07009c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:47.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.240+0000 7fd065400640 1 -- 172.21.15.39:0/1977732212 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd07009dbc0 con 0x7fd0700cbd70 2024-09-21T07:43:47.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.241+0000 7fd065400640 1 --2- 172.21.15.39:0/1977732212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0700cbd70 0x7fd0701f6a50 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fd05800c430 tx=0x7fd05800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:47.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.241+0000 7fd066a00640 1 -- 172.21.15.39:0/1977732212 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd058011790 con 0x7fd0700cbd70 2024-09-21T07:43:47.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.241+0000 7fd066a00640 1 -- 172.21.15.39:0/1977732212 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd058017070 con 0x7fd0700cbd70 2024-09-21T07:43:47.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.241+0000 7fd06f200640 1 -- 172.21.15.39:0/1977732212 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd07009cb20 con 0x7fd0700cbd70 2024-09-21T07:43:47.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.241+0000 7fd066a00640 1 -- 172.21.15.39:0/1977732212 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd058011ce0 con 0x7fd0700cbd70 2024-09-21T07:43:47.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.241+0000 7fd06f200640 1 -- 172.21.15.39:0/1977732212 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd07009d0e0 con 0x7fd0700cbd70 2024-09-21T07:43:47.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.242+0000 7fd06f200640 1 -- 172.21.15.39:0/1977732212 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd07009d7f0 con 0x7fd0700cbd70 2024-09-21T07:43:47.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.243+0000 7fd066a00640 1 -- 172.21.15.39:0/1977732212 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd05801c460 con 0x7fd0700cbd70 2024-09-21T07:43:47.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.244+0000 7fd066a00640 1 --2- 172.21.15.39:0/1977732212 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd04c078b90 0x7fd04c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:47.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.244+0000 7fd066a00640 1 -- 172.21.15.39:0/1977732212 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8277+0+0 (secure 0 0 0) 0x7fd05809b390 con 0x7fd0700cbd70 2024-09-21T07:43:47.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.245+0000 7fd06de00640 1 --2- 172.21.15.39:0/1977732212 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd04c078b90 0x7fd04c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:47.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.245+0000 7fd06de00640 1 --2- 172.21.15.39:0/1977732212 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd04c078b90 0x7fd04c07b0a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd060009b30 tx=0x7fd060005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:47.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.249+0000 7fd066a00640 1 -- 172.21.15.39:0/1977732212 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd0580627e0 con 0x7fd0700cbd70 2024-09-21T07:43:47.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.487+0000 7fd06f200640 1 -- 172.21.15.39:0/1977732212 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd07009c8b0 con 0x7fd0700cbd70 2024-09-21T07:43:47.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.487+0000 7fd066a00640 1 -- 172.21.15.39:0/1977732212 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) ==== 95+0+5256 (secure 0 0 0) 0x7fd058067210 con 0x7fd0700cbd70 2024-09-21T07:43:47.489 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:43:47.489 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":152,"btime":"2024-09-21T07:43:46:202252+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":8052,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1551311805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1551311805},{"type":"v1","addr":"172.21.15.39:6835","nonce":1551311805}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":147}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:43:45.746888+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7366},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7366":{"gid":7366,"name":"d","rank":1,"incarnation":146,"state":"up:active","state_seq":18,"addr":"172.21.15.148:6837/759792812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":759792812},{"type":"v1","addr":"172.21.15.148:6837","nonce":759792812}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311,7366]},"id":6}]} 2024-09-21T07:43:47.490 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 152 2024-09-21T07:43:47.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.490+0000 7fd06f200640 1 -- 172.21.15.39:0/1977732212 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd04c078b90 msgr2=0x7fd04c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:47.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.490+0000 7fd06f200640 1 --2- 172.21.15.39:0/1977732212 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd04c078b90 0x7fd04c07b0a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd060009b30 tx=0x7fd060005f90 comp rx=0 tx=0).stop 2024-09-21T07:43:47.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.490+0000 7fd06f200640 1 -- 172.21.15.39:0/1977732212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0700cbd70 msgr2=0x7fd0701f6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:47.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.490+0000 7fd06f200640 1 --2- 172.21.15.39:0/1977732212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0700cbd70 0x7fd0701f6a50 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fd05800c430 tx=0x7fd05800c8f0 comp rx=0 tx=0).stop 2024-09-21T07:43:47.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.490+0000 7fd06f200640 1 -- 172.21.15.39:0/1977732212 shutdown_connections 2024-09-21T07:43:47.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.490+0000 7fd06f200640 1 --2- 172.21.15.39:0/1977732212 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd04c078b90 0x7fd04c07b0a0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:47.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.490+0000 7fd06f200640 1 --2- 172.21.15.39:0/1977732212 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd0700d0f20 0x7fd0700c9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:47.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.490+0000 7fd06f200640 1 --2- 172.21.15.39:0/1977732212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0700cbd70 0x7fd0701f6a50 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:47.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.490+0000 7fd06f200640 1 --2- 172.21.15.39:0/1977732212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0700cb960 0x7fd07009c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:47.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.490+0000 7fd06f200640 1 -- 172.21.15.39:0/1977732212 >> 172.21.15.39:0/1977732212 conn(0x7fd0700b2510 msgr2=0x7fd07015b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:47.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.490+0000 7fd06f200640 1 -- 172.21.15.39:0/1977732212 shutdown_connections 2024-09-21T07:43:47.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:47.490+0000 7fd06f200640 1 -- 172.21.15.39:0/1977732212 wait complete. 2024-09-21T07:43:47.515 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:43:47.714 INFO:tasks.ceph.mds.d.smithi148.stderr:daemon-helper: command crashed with signal 6 2024-09-21T07:43:49.546 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-09-21T07:43:52.516 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:43:52.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.751+0000 7fc117200640 1 -- 172.21.15.39:0/1141019675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc11015cd30 msgr2=0x7fc1100cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:52.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.751+0000 7fc117200640 1 --2- 172.21.15.39:0/1141019675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc11015cd30 0x7fc1100cfd10 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fc1101572f0 tx=0x7fc10002a560 comp rx=0 tx=0).stop 2024-09-21T07:43:52.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.751+0000 7fc117200640 1 -- 172.21.15.39:0/1141019675 shutdown_connections 2024-09-21T07:43:52.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.751+0000 7fc117200640 1 --2- 172.21.15.39:0/1141019675 >> v2:172.21.15.148:3300/0 conn(0x7fc11006fc10 0x7fc1100d0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:52.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.751+0000 7fc117200640 1 --2- 172.21.15.39:0/1141019675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc11015cd30 0x7fc1100cfd10 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:52.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.751+0000 7fc117200640 1 -- 172.21.15.39:0/1141019675 >> 172.21.15.39:0/1141019675 conn(0x7fc11006cfc0 msgr2=0x7fc11006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:52.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.752+0000 7fc117200640 1 -- 172.21.15.39:0/1141019675 shutdown_connections 2024-09-21T07:43:52.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.752+0000 7fc117200640 1 -- 172.21.15.39:0/1141019675 wait complete. 2024-09-21T07:43:52.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.754+0000 7fc117200640 1 Processor -- start 2024-09-21T07:43:52.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.754+0000 7fc117200640 1 -- start start 2024-09-21T07:43:52.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.755+0000 7fc117200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc11006f800 0x7fc1101454d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:52.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.755+0000 7fc117200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc11006fc10 0x7fc110147d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:52.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.755+0000 7fc117200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc11015cd30 0x7fc1100b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:52.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.755+0000 7fc117200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc110073890 con 0x7fc11006f800 2024-09-21T07:43:52.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.755+0000 7fc117200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc110073710 con 0x7fc11006fc10 2024-09-21T07:43:52.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.755+0000 7fc117200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc110073a10 con 0x7fc11015cd30 2024-09-21T07:43:52.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.756+0000 7fc10fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc11006f800 0x7fc1101454d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:52.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.756+0000 7fc10fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc11006f800 0x7fc1101454d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57816/0 (socket says 172.21.15.39:57816) 2024-09-21T07:43:52.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.756+0000 7fc10fe00640 1 -- 172.21.15.39:0/1391647691 learned_addr learned my addr 172.21.15.39:0/1391647691 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:52.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.756+0000 7fc115200640 1 --2- 172.21.15.39:0/1391647691 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc11015cd30 0x7fc1100b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:52.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.756+0000 7fc10f400640 1 --2- 172.21.15.39:0/1391647691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc11006fc10 0x7fc110147d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:52.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.756+0000 7fc10fe00640 1 -- 172.21.15.39:0/1391647691 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc11015cd30 msgr2=0x7fc1100b18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:52.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.756+0000 7fc10fe00640 1 --2- 172.21.15.39:0/1391647691 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc11015cd30 0x7fc1100b18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:52.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.756+0000 7fc10fe00640 1 -- 172.21.15.39:0/1391647691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc11006fc10 msgr2=0x7fc110147d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:52.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.756+0000 7fc10fe00640 1 --2- 172.21.15.39:0/1391647691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc11006fc10 0x7fc110147d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:52.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.756+0000 7fc10fe00640 1 -- 172.21.15.39:0/1391647691 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc110145a10 con 0x7fc11006f800 2024-09-21T07:43:52.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.756+0000 7fc115200640 1 --2- 172.21.15.39:0/1391647691 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc11015cd30 0x7fc1100b18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:43:52.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.756+0000 7fc10f400640 1 --2- 172.21.15.39:0/1391647691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc11006fc10 0x7fc110147d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:43:52.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.756+0000 7fc10fe00640 1 --2- 172.21.15.39:0/1391647691 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc11006f800 0x7fc1101454d0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fc100034040 tx=0x7fc1000039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:52.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.757+0000 7fc107e00640 1 -- 172.21.15.39:0/1391647691 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10003e030 con 0x7fc11006f800 2024-09-21T07:43:52.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.757+0000 7fc107e00640 1 -- 172.21.15.39:0/1391647691 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1000047a0 con 0x7fc11006f800 2024-09-21T07:43:52.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.757+0000 7fc107e00640 1 -- 172.21.15.39:0/1391647691 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc100035040 con 0x7fc11006f800 2024-09-21T07:43:52.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.757+0000 7fc117200640 1 -- 172.21.15.39:0/1391647691 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1101482b0 con 0x7fc11006f800 2024-09-21T07:43:52.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.757+0000 7fc117200640 1 -- 172.21.15.39:0/1391647691 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc110148a10 con 0x7fc11006f800 2024-09-21T07:43:52.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.758+0000 7fc117200640 1 -- 172.21.15.39:0/1391647691 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc110073f50 con 0x7fc11006f800 2024-09-21T07:43:52.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.759+0000 7fc107e00640 1 -- 172.21.15.39:0/1391647691 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc100041030 con 0x7fc11006f800 2024-09-21T07:43:52.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.760+0000 7fc107e00640 1 --2- 172.21.15.39:0/1391647691 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc0f0078bc0 0x7fc0f007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:52.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.760+0000 7fc107e00640 1 -- 172.21.15.39:0/1391647691 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8233+0+0 (secure 0 0 0) 0x7fc1000bbf10 con 0x7fc11006f800 2024-09-21T07:43:52.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.761+0000 7fc10f400640 1 --2- 172.21.15.39:0/1391647691 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc0f0078bc0 0x7fc0f007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:52.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.761+0000 7fc10f400640 1 --2- 172.21.15.39:0/1391647691 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc0f0078bc0 0x7fc0f007b0d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc1100ccc30 tx=0x7fc0fc009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:52.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.765+0000 7fc107e00640 1 -- 172.21.15.39:0/1391647691 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc1000839e0 con 0x7fc11006f800 2024-09-21T07:43:53.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:52.999+0000 7fc117200640 1 -- 172.21.15.39:0/1391647691 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1100b3470 con 0x7fc11006f800 2024-09-21T07:43:53.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:53.000+0000 7fc107e00640 1 -- 172.21.15.39:0/1391647691 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) ==== 95+0+5256 (secure 0 0 0) 0x7fc1000832e0 con 0x7fc11006f800 2024-09-21T07:43:53.002 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:43:53.002 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":152,"btime":"2024-09-21T07:43:46:202252+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":8052,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1551311805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1551311805},{"type":"v1","addr":"172.21.15.39:6835","nonce":1551311805}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":147}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:43:45.746888+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7366},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7366":{"gid":7366,"name":"d","rank":1,"incarnation":146,"state":"up:active","state_seq":18,"addr":"172.21.15.148:6837/759792812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":759792812},{"type":"v1","addr":"172.21.15.148:6837","nonce":759792812}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311,7366]},"id":6}]} 2024-09-21T07:43:53.002 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 152 2024-09-21T07:43:53.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:53.002+0000 7fc117200640 1 -- 172.21.15.39:0/1391647691 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc0f0078bc0 msgr2=0x7fc0f007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:53.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:53.002+0000 7fc117200640 1 --2- 172.21.15.39:0/1391647691 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc0f0078bc0 0x7fc0f007b0d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc1100ccc30 tx=0x7fc0fc009040 comp rx=0 tx=0).stop 2024-09-21T07:43:53.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:53.002+0000 7fc117200640 1 -- 172.21.15.39:0/1391647691 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc11006f800 msgr2=0x7fc1101454d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:53.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:53.002+0000 7fc117200640 1 --2- 172.21.15.39:0/1391647691 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc11006f800 0x7fc1101454d0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fc100034040 tx=0x7fc1000039e0 comp rx=0 tx=0).stop 2024-09-21T07:43:53.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:53.002+0000 7fc117200640 1 -- 172.21.15.39:0/1391647691 shutdown_connections 2024-09-21T07:43:53.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:53.002+0000 7fc117200640 1 --2- 172.21.15.39:0/1391647691 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc0f0078bc0 0x7fc0f007b0d0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:53.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:53.002+0000 7fc117200640 1 --2- 172.21.15.39:0/1391647691 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc11015cd30 0x7fc1100b18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:53.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:53.002+0000 7fc117200640 1 --2- 172.21.15.39:0/1391647691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc11006fc10 0x7fc110147d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:53.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:53.002+0000 7fc117200640 1 --2- 172.21.15.39:0/1391647691 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc11006f800 0x7fc1101454d0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:53.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:53.002+0000 7fc117200640 1 -- 172.21.15.39:0/1391647691 >> 172.21.15.39:0/1391647691 conn(0x7fc11006cfc0 msgr2=0x7fc11015b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:53.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:53.002+0000 7fc117200640 1 -- 172.21.15.39:0/1391647691 shutdown_connections 2024-09-21T07:43:53.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:53.003+0000 7fc117200640 1 -- 172.21.15.39:0/1391647691 wait complete. 2024-09-21T07:43:53.027 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:43:56.159 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~7s 2024-09-21T07:43:58.029 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:43:58.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.265+0000 7f4b62e00640 1 -- 172.21.15.39:0/3015889214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b5c0d2a50 msgr2=0x7f4b5c0d2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:58.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.265+0000 7f4b62e00640 1 --2- 172.21.15.39:0/3015889214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b5c0d2a50 0x7f4b5c0d2e60 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c0b96b0 tx=0x7f4b4802a4a0 comp rx=0 tx=0).stop 2024-09-21T07:43:58.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.265+0000 7f4b62e00640 1 -- 172.21.15.39:0/3015889214 shutdown_connections 2024-09-21T07:43:58.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.265+0000 7f4b62e00640 1 --2- 172.21.15.39:0/3015889214 >> v2:172.21.15.148:3300/0 conn(0x7f4b5c162b40 0x7f4b5c162f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:58.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.265+0000 7f4b62e00640 1 --2- 172.21.15.39:0/3015889214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b5c0d2a50 0x7f4b5c0d2e60 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:58.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.265+0000 7f4b62e00640 1 -- 172.21.15.39:0/3015889214 >> 172.21.15.39:0/3015889214 conn(0x7f4b5c08bfb0 msgr2=0x7f4b5c0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:58.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.265+0000 7f4b62e00640 1 -- 172.21.15.39:0/3015889214 shutdown_connections 2024-09-21T07:43:58.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.265+0000 7f4b62e00640 1 -- 172.21.15.39:0/3015889214 wait complete. 2024-09-21T07:43:58.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.267+0000 7f4b62e00640 1 Processor -- start 2024-09-21T07:43:58.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.267+0000 7f4b62e00640 1 -- start start 2024-09-21T07:43:58.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.269+0000 7f4b62e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b5c0b4db0 0x7f4b5c1f56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:58.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.269+0000 7f4b62e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b5c0d2a50 0x7f4b5c1f9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:58.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.269+0000 7f4b62e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b5c162b40 0x7f4b5c0ca940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:58.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.269+0000 7f4b62e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c163930 con 0x7f4b5c162b40 2024-09-21T07:43:58.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.269+0000 7f4b62e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c1637b0 con 0x7f4b5c0b4db0 2024-09-21T07:43:58.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.269+0000 7f4b62e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f4b5c0cee90 con 0x7f4b5c0d2a50 2024-09-21T07:43:58.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.269+0000 7f4b62400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b5c162b40 0x7f4b5c0ca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:58.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.269+0000 7f4b62400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b5c162b40 0x7f4b5c0ca940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57838/0 (socket says 172.21.15.39:57838) 2024-09-21T07:43:58.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.269+0000 7f4b62400640 1 -- 172.21.15.39:0/4208548820 learned_addr learned my addr 172.21.15.39:0/4208548820 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:43:58.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.269+0000 7f4b61a00640 1 --2- 172.21.15.39:0/4208548820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b5c0b4db0 0x7f4b5c1f56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:58.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.269+0000 7f4b61000640 1 --2- 172.21.15.39:0/4208548820 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b5c0d2a50 0x7f4b5c1f9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:58.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.270+0000 7f4b62400640 1 -- 172.21.15.39:0/4208548820 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b5c0d2a50 msgr2=0x7f4b5c1f9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:58.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.270+0000 7f4b62400640 1 --2- 172.21.15.39:0/4208548820 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b5c0d2a50 0x7f4b5c1f9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:58.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.270+0000 7f4b62400640 1 -- 172.21.15.39:0/4208548820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b5c0b4db0 msgr2=0x7f4b5c1f56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:58.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.270+0000 7f4b62400640 1 --2- 172.21.15.39:0/4208548820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b5c0b4db0 0x7f4b5c1f56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:58.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.270+0000 7f4b62400640 1 -- 172.21.15.39:0/4208548820 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b5c1f68c0 con 0x7f4b5c162b40 2024-09-21T07:43:58.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.270+0000 7f4b62400640 1 --2- 172.21.15.39:0/4208548820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b5c162b40 0x7f4b5c0ca940 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f4b5400e9c0 tx=0x7f4b5400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:58.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.270+0000 7f4b59600640 1 -- 172.21.15.39:0/4208548820 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b54004d20 con 0x7f4b5c162b40 2024-09-21T07:43:58.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.270+0000 7f4b59600640 1 -- 172.21.15.39:0/4208548820 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b5400cb20 con 0x7f4b5c162b40 2024-09-21T07:43:58.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.270+0000 7f4b62e00640 1 -- 172.21.15.39:0/4208548820 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b5c0cae80 con 0x7f4b5c162b40 2024-09-21T07:43:58.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.270+0000 7f4b59600640 1 -- 172.21.15.39:0/4208548820 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b54005020 con 0x7f4b5c162b40 2024-09-21T07:43:58.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.270+0000 7f4b62e00640 1 -- 172.21.15.39:0/4208548820 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b5c1f61d0 con 0x7f4b5c162b40 2024-09-21T07:43:58.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.271+0000 7f4b62e00640 1 -- 172.21.15.39:0/4208548820 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b5c1f6a90 con 0x7f4b5c162b40 2024-09-21T07:43:58.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.273+0000 7f4b59600640 1 -- 172.21.15.39:0/4208548820 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4b540051c0 con 0x7f4b5c162b40 2024-09-21T07:43:58.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.274+0000 7f4b59600640 1 --2- 172.21.15.39:0/4208548820 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4b3c078b90 0x7f4b3c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:43:58.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.274+0000 7f4b59600640 1 -- 172.21.15.39:0/4208548820 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8233+0+0 (secure 0 0 0) 0x7f4b5409bf70 con 0x7f4b5c162b40 2024-09-21T07:43:58.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.274+0000 7f4b61a00640 1 --2- 172.21.15.39:0/4208548820 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4b3c078b90 0x7f4b3c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:43:58.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.275+0000 7f4b61a00640 1 --2- 172.21.15.39:0/4208548820 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4b3c078b90 0x7f4b3c07b0a0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4b50001760 tx=0x7f4b50009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:43:58.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.278+0000 7f4b59600640 1 -- 172.21.15.39:0/4208548820 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4b54016030 con 0x7f4b5c162b40 2024-09-21T07:43:58.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.513+0000 7f4b62e00640 1 -- 172.21.15.39:0/4208548820 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b5c1f5c80 con 0x7f4b5c162b40 2024-09-21T07:43:58.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.514+0000 7f4b59600640 1 -- 172.21.15.39:0/4208548820 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) ==== 95+0+5256 (secure 0 0 0) 0x7f4b540633f0 con 0x7f4b5c162b40 2024-09-21T07:43:58.516 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:43:58.516 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":152,"btime":"2024-09-21T07:43:46:202252+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":8052,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1551311805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1551311805},{"type":"v1","addr":"172.21.15.39:6835","nonce":1551311805}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":147}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:43:45.746888+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7366},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7366":{"gid":7366,"name":"d","rank":1,"incarnation":146,"state":"up:active","state_seq":18,"addr":"172.21.15.148:6837/759792812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":759792812},{"type":"v1","addr":"172.21.15.148:6837","nonce":759792812}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311,7366]},"id":6}]} 2024-09-21T07:43:58.516 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 152 2024-09-21T07:43:58.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.516+0000 7f4b62e00640 1 -- 172.21.15.39:0/4208548820 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4b3c078b90 msgr2=0x7f4b3c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:58.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.516+0000 7f4b62e00640 1 --2- 172.21.15.39:0/4208548820 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4b3c078b90 0x7f4b3c07b0a0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4b50001760 tx=0x7f4b50009040 comp rx=0 tx=0).stop 2024-09-21T07:43:58.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.516+0000 7f4b62e00640 1 -- 172.21.15.39:0/4208548820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b5c162b40 msgr2=0x7f4b5c0ca940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:43:58.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.516+0000 7f4b62e00640 1 --2- 172.21.15.39:0/4208548820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b5c162b40 0x7f4b5c0ca940 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f4b5400e9c0 tx=0x7f4b5400ee80 comp rx=0 tx=0).stop 2024-09-21T07:43:58.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.516+0000 7f4b62e00640 1 -- 172.21.15.39:0/4208548820 shutdown_connections 2024-09-21T07:43:58.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.516+0000 7f4b62e00640 1 --2- 172.21.15.39:0/4208548820 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4b3c078b90 0x7f4b3c07b0a0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:58.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.516+0000 7f4b62e00640 1 --2- 172.21.15.39:0/4208548820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b5c162b40 0x7f4b5c0ca940 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:58.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.516+0000 7f4b62e00640 1 --2- 172.21.15.39:0/4208548820 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b5c0d2a50 0x7f4b5c1f9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:58.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.516+0000 7f4b62e00640 1 --2- 172.21.15.39:0/4208548820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b5c0b4db0 0x7f4b5c1f56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:43:58.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.516+0000 7f4b62e00640 1 -- 172.21.15.39:0/4208548820 >> 172.21.15.39:0/4208548820 conn(0x7f4b5c08bfb0 msgr2=0x7f4b5c0b8b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:43:58.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.517+0000 7f4b62e00640 1 -- 172.21.15.39:0/4208548820 shutdown_connections 2024-09-21T07:43:58.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:43:58.517+0000 7f4b62e00640 1 -- 172.21.15.39:0/4208548820 wait complete. 2024-09-21T07:43:58.541 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:44:02.007 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:44:02.011 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:44:02.772 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~13s 2024-09-21T07:44:03.542 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:44:03.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:03.779+0000 7f8984400640 1 -- 172.21.15.39:0/2491285017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f897c15f360 msgr2=0x7f897c15bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:03.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:03.779+0000 7f8984400640 1 --2- 172.21.15.39:0/2491285017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f897c15f360 0x7f897c15bd70 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f8970009830 tx=0x7f897002a510 comp rx=0 tx=0).stop 2024-09-21T07:44:03.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:03.780+0000 7f8984400640 1 -- 172.21.15.39:0/2491285017 shutdown_connections 2024-09-21T07:44:03.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:03.780+0000 7f8984400640 1 --2- 172.21.15.39:0/2491285017 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f897c155e80 0x7f897c0b4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:03.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:03.780+0000 7f8984400640 1 --2- 172.21.15.39:0/2491285017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f897c15f360 0x7f897c15bd70 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:03.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:03.780+0000 7f8984400640 1 -- 172.21.15.39:0/2491285017 >> 172.21.15.39:0/2491285017 conn(0x7f897c06cfc0 msgr2=0x7f897c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:03.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:03.780+0000 7f8984400640 1 -- 172.21.15.39:0/2491285017 shutdown_connections 2024-09-21T07:44:03.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:03.780+0000 7f8984400640 1 -- 172.21.15.39:0/2491285017 wait complete. 2024-09-21T07:44:03.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:03.782+0000 7f8984400640 1 Processor -- start 2024-09-21T07:44:03.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:03.782+0000 7f8984400640 1 -- start start 2024-09-21T07:44:03.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:03.784+0000 7f8984400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f897c155a70 0x7f897c15a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:03.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:03.784+0000 7f8984400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f897c155e80 0x7f897c149010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:03.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:03.784+0000 7f8984400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f897c15f360 0x7f897c0aea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:03.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:03.784+0000 7f8984400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f897c0d0620 con 0x7f897c155a70 2024-09-21T07:44:03.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:03.784+0000 7f8984400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f897c0b4ef0 con 0x7f897c15f360 2024-09-21T07:44:03.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:03.784+0000 7f8984400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f897c0d07a0 con 0x7f897c155e80 2024-09-21T07:44:03.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:03.784+0000 7f8981a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f897c155a70 0x7f897c15a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:03.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:03.784+0000 7f8981a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f897c155a70 0x7f897c15a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37292/0 (socket says 172.21.15.39:37292) 2024-09-21T07:44:03.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:03.784+0000 7f8981a00640 1 -- 172.21.15.39:0/942172356 learned_addr learned my addr 172.21.15.39:0/942172356 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:03.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:03.784+0000 7f8981000640 1 --2- 172.21.15.39:0/942172356 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f897c155e80 0x7f897c149010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:03.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:03.784+0000 7f8982400640 1 --2- 172.21.15.39:0/942172356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f897c15f360 0x7f897c0aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:03.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:03.784+0000 7f8981a00640 1 -- 172.21.15.39:0/942172356 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f897c155e80 msgr2=0x7f897c149010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:03.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:03.784+0000 7f8981a00640 1 --2- 172.21.15.39:0/942172356 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f897c155e80 0x7f897c149010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:03.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:03.784+0000 7f8981a00640 1 -- 172.21.15.39:0/942172356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f897c15f360 msgr2=0x7f897c0aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:03.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:03.784+0000 7f8981a00640 1 --2- 172.21.15.39:0/942172356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f897c15f360 0x7f897c0aea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:03.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:03.784+0000 7f8981a00640 1 -- 172.21.15.39:0/942172356 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f897c15aed0 con 0x7f897c155a70 2024-09-21T07:44:03.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:03.785+0000 7f8981a00640 1 --2- 172.21.15.39:0/942172356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f897c155a70 0x7f897c15a630 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f897002ab00 tx=0x7f8970033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:03.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:03.785+0000 7f8979600640 1 -- 172.21.15.39:0/942172356 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8970005c90 con 0x7f897c155a70 2024-09-21T07:44:03.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:03.785+0000 7f8979600640 1 -- 172.21.15.39:0/942172356 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8970031e50 con 0x7f897c155a70 2024-09-21T07:44:03.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:03.785+0000 7f8979600640 1 -- 172.21.15.39:0/942172356 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8970005260 con 0x7f897c155a70 2024-09-21T07:44:03.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:03.785+0000 7f8984400640 1 -- 172.21.15.39:0/942172356 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f897c15b1b0 con 0x7f897c155a70 2024-09-21T07:44:03.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:03.785+0000 7f8984400640 1 -- 172.21.15.39:0/942172356 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f897c149de0 con 0x7f897c155a70 2024-09-21T07:44:03.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:03.786+0000 7f8984400640 1 -- 172.21.15.39:0/942172356 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f897c0b2d20 con 0x7f897c155a70 2024-09-21T07:44:03.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:03.788+0000 7f8979600640 1 -- 172.21.15.39:0/942172356 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8970005400 con 0x7f897c155a70 2024-09-21T07:44:03.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:03.788+0000 7f8979600640 1 --2- 172.21.15.39:0/942172356 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f895c078b90 0x7f895c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:03.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:03.789+0000 7f8979600640 1 -- 172.21.15.39:0/942172356 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8233+0+0 (secure 0 0 0) 0x7f89700bc7f0 con 0x7f897c155a70 2024-09-21T07:44:03.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:03.789+0000 7f8981000640 1 --2- 172.21.15.39:0/942172356 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f895c078b90 0x7f895c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:03.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:03.790+0000 7f8981000640 1 --2- 172.21.15.39:0/942172356 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f895c078b90 0x7f895c07b0a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f8968007fd0 tx=0x7f8968001390 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:03.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:03.793+0000 7f8979600640 1 -- 172.21.15.39:0/942172356 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f897c0b2d20 con 0x7f897c155a70 2024-09-21T07:44:04.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.032+0000 7f8984400640 1 -- 172.21.15.39:0/942172356 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f897c0d0cc0 con 0x7f897c155a70 2024-09-21T07:44:04.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.033+0000 7f8979600640 1 -- 172.21.15.39:0/942172356 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 153 v153) ==== 95+0+5304 (secure 0 0 0) 0x7f897003f030 con 0x7f897c155a70 2024-09-21T07:44:04.035 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:04.035 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":153,"btime":"2024-09-21T07:44:01:465939+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":8052,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1551311805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1551311805},{"type":"v1","addr":"172.21.15.39:6835","nonce":1551311805}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":147}],"filesystems":[{"mdsmap":{"epoch":153,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:44:01.465885+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7366},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7366":{"gid":7366,"name":"d","rank":1,"incarnation":146,"state":"up:active","state_seq":18,"addr":"172.21.15.148:6837/759792812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":759792812},{"type":"v1","addr":"172.21.15.148:6837","nonce":759792812}]},"join_fscid":-1,"laggy_since":"2024-09-21T07:44:01.465885+0000","export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7366,7311]},"id":6}]} 2024-09-21T07:44:04.035 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 153 2024-09-21T07:44:04.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.035+0000 7f8984400640 1 -- 172.21.15.39:0/942172356 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f895c078b90 msgr2=0x7f895c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:04.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.035+0000 7f8984400640 1 --2- 172.21.15.39:0/942172356 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f895c078b90 0x7f895c07b0a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f8968007fd0 tx=0x7f8968001390 comp rx=0 tx=0).stop 2024-09-21T07:44:04.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.035+0000 7f8984400640 1 -- 172.21.15.39:0/942172356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f897c155a70 msgr2=0x7f897c15a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:04.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.035+0000 7f8984400640 1 --2- 172.21.15.39:0/942172356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f897c155a70 0x7f897c15a630 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f897002ab00 tx=0x7f8970033c20 comp rx=0 tx=0).stop 2024-09-21T07:44:04.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.036+0000 7f8984400640 1 -- 172.21.15.39:0/942172356 shutdown_connections 2024-09-21T07:44:04.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.036+0000 7f8984400640 1 --2- 172.21.15.39:0/942172356 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f895c078b90 0x7f895c07b0a0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:04.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.036+0000 7f8984400640 1 --2- 172.21.15.39:0/942172356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f897c15f360 0x7f897c0aea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:04.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.036+0000 7f8984400640 1 --2- 172.21.15.39:0/942172356 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f897c155e80 0x7f897c149010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:04.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.036+0000 7f8984400640 1 --2- 172.21.15.39:0/942172356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f897c155a70 0x7f897c15a630 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:04.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.036+0000 7f8984400640 1 -- 172.21.15.39:0/942172356 >> 172.21.15.39:0/942172356 conn(0x7f897c06cfc0 msgr2=0x7f897c068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:04.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.036+0000 7f8984400640 1 -- 172.21.15.39:0/942172356 shutdown_connections 2024-09-21T07:44:04.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.036+0000 7f8984400640 1 -- 172.21.15.39:0/942172356 wait complete. 2024-09-21T07:44:04.060 DEBUG:tasks.ceph_test_case:wait_until_true: success in 15s and 0 retries 2024-09-21T07:44:04.061 DEBUG:teuthology.orchestra.run.smithi148:> sudo sysctl -n kernel.core_pattern 2024-09-21T07:44:04.120 INFO:teuthology.orchestra.run.smithi148.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T07:44:04.123 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2024-09-21T07:44:04.124 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2024-09-21T07:44:04.189 INFO:teuthology.orchestra.run.smithi148.stdout:1726904626.43971.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i d', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2024-09-21T07:44:04.193 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1726904626.43971.core'] 2024-09-21T07:44:04.193 INFO:tasks.cephfs.cephfs_test_case:Found core file 1726904626.43971.core, deleting it 2024-09-21T07:44:04.193 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1726904626.43971.core 2024-09-21T07:44:04.284 INFO:tasks.cephfs.mount:Suspending the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:44:04.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:44:04.284 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 down 2024-09-21T07:44:04.348 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:44:04.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.572+0000 7fe4e9c00640 1 -- 172.21.15.39:0/1392761524 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe4e415cd30 msgr2=0x7fe4e40cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:04.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.572+0000 7fe4e9c00640 1 --2- 172.21.15.39:0/1392761524 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe4e415cd30 0x7fe4e40cfd10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4009830 tx=0x7fe4d402a530 comp rx=0 tx=0).stop 2024-09-21T07:44:04.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.573+0000 7fe4e9c00640 1 -- 172.21.15.39:0/1392761524 shutdown_connections 2024-09-21T07:44:04.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.573+0000 7fe4e9c00640 1 --2- 172.21.15.39:0/1392761524 >> v2:172.21.15.148:3300/0 conn(0x7fe4e406f800 0x7fe4e40cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:04.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.573+0000 7fe4e9c00640 1 --2- 172.21.15.39:0/1392761524 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe4e415cd30 0x7fe4e40cfd10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:04.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.573+0000 7fe4e9c00640 1 -- 172.21.15.39:0/1392761524 >> 172.21.15.39:0/1392761524 conn(0x7fe4e406cfc0 msgr2=0x7fe4e406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:04.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.573+0000 7fe4e9c00640 1 -- 172.21.15.39:0/1392761524 shutdown_connections 2024-09-21T07:44:04.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.573+0000 7fe4e9c00640 1 -- 172.21.15.39:0/1392761524 wait complete. 2024-09-21T07:44:04.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.575+0000 7fe4e9c00640 1 Processor -- start 2024-09-21T07:44:04.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.575+0000 7fe4e9c00640 1 -- start start 2024-09-21T07:44:04.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.577+0000 7fe4e9c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4e406f800 0x7fe4e40b3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:04.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.577+0000 7fe4e9c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4e406fc10 0x7fe4e41f56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:04.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.577+0000 7fe4e9c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe4e415cd30 0x7fe4e41f5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:04.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.577+0000 7fe4e9c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e4073890 con 0x7fe4e406f800 2024-09-21T07:44:04.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.577+0000 7fe4e9c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e4073710 con 0x7fe4e406fc10 2024-09-21T07:44:04.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.577+0000 7fe4e9c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe4e4073a10 con 0x7fe4e415cd30 2024-09-21T07:44:04.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.577+0000 7fe4e2a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4e406f800 0x7fe4e40b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:04.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.577+0000 7fe4e2a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4e406f800 0x7fe4e40b3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37300/0 (socket says 172.21.15.39:37300) 2024-09-21T07:44:04.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.577+0000 7fe4e2a00640 1 -- 172.21.15.39:0/3851787341 learned_addr learned my addr 172.21.15.39:0/3851787341 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:04.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.577+0000 7fe4e3400640 1 --2- 172.21.15.39:0/3851787341 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe4e415cd30 0x7fe4e41f5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:04.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.577+0000 7fe4dbe00640 1 --2- 172.21.15.39:0/3851787341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4e406fc10 0x7fe4e41f56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:04.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.577+0000 7fe4e2a00640 1 -- 172.21.15.39:0/3851787341 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe4e415cd30 msgr2=0x7fe4e41f5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:04.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.577+0000 7fe4e2a00640 1 --2- 172.21.15.39:0/3851787341 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe4e415cd30 0x7fe4e41f5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:04.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.577+0000 7fe4e2a00640 1 -- 172.21.15.39:0/3851787341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4e406fc10 msgr2=0x7fe4e41f56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:04.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.577+0000 7fe4e2a00640 1 --2- 172.21.15.39:0/3851787341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4e406fc10 0x7fe4e41f56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:04.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.577+0000 7fe4e2a00640 1 -- 172.21.15.39:0/3851787341 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4e40a9690 con 0x7fe4e406f800 2024-09-21T07:44:04.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.578+0000 7fe4e2a00640 1 --2- 172.21.15.39:0/3851787341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4e406f800 0x7fe4e40b3470 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4009950 tx=0x7fe4d4033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:04.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.578+0000 7fe4daa00640 1 -- 172.21.15.39:0/3851787341 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4031d60 con 0x7fe4e406f800 2024-09-21T07:44:04.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.578+0000 7fe4daa00640 1 -- 172.21.15.39:0/3851787341 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4d4033e10 con 0x7fe4e406f800 2024-09-21T07:44:04.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.578+0000 7fe4daa00640 1 -- 172.21.15.39:0/3851787341 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4033310 con 0x7fe4e406f800 2024-09-21T07:44:04.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.578+0000 7fe4e9c00640 1 -- 172.21.15.39:0/3851787341 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4e40a9970 con 0x7fe4e406f800 2024-09-21T07:44:04.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.578+0000 7fe4e9c00640 1 -- 172.21.15.39:0/3851787341 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4e40aa060 con 0x7fe4e406f800 2024-09-21T07:44:04.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.579+0000 7fe4e9c00640 1 -- 172.21.15.39:0/3851787341 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4e4073ec0 con 0x7fe4e406f800 2024-09-21T07:44:04.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.581+0000 7fe4daa00640 1 -- 172.21.15.39:0/3851787341 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe4d40334b0 con 0x7fe4e406f800 2024-09-21T07:44:04.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.581+0000 7fe4daa00640 1 --2- 172.21.15.39:0/3851787341 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe4c0078b90 0x7fe4c007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:04.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.582+0000 7fe4daa00640 1 -- 172.21.15.39:0/3851787341 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8233+0+0 (secure 0 0 0) 0x7fe4d40bca30 con 0x7fe4e406f800 2024-09-21T07:44:04.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.582+0000 7fe4dbe00640 1 --2- 172.21.15.39:0/3851787341 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe4c0078b90 0x7fe4c007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:04.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.583+0000 7fe4dbe00640 1 --2- 172.21.15.39:0/3851787341 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe4c0078b90 0x7fe4c007b0a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fe4e40ccc30 tx=0x7fe4cc005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:04.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.586+0000 7fe4daa00640 1 -- 172.21.15.39:0/3851787341 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe4d403f030 con 0x7fe4e406f800 2024-09-21T07:44:04.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.827+0000 7fe4e9c00640 1 -- 172.21.15.39:0/3851787341 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4e41581d0 con 0x7fe4e406f800 2024-09-21T07:44:04.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.827+0000 7fe4daa00640 1 -- 172.21.15.39:0/3851787341 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 153 v153) ==== 95+0+5304 (secure 0 0 0) 0x7fe4d4083e20 con 0x7fe4e406f800 2024-09-21T07:44:04.829 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:04.829 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":153,"btime":"2024-09-21T07:44:01:465939+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":8052,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1551311805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1551311805},{"type":"v1","addr":"172.21.15.39:6835","nonce":1551311805}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":147}],"filesystems":[{"mdsmap":{"epoch":153,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:44:01.465885+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":7366},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7366":{"gid":7366,"name":"d","rank":1,"incarnation":146,"state":"up:active","state_seq":18,"addr":"172.21.15.148:6837/759792812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":759792812},{"type":"v1","addr":"172.21.15.148:6837","nonce":759792812}]},"join_fscid":-1,"laggy_since":"2024-09-21T07:44:01.465885+0000","export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7366,7311]},"id":6}]} 2024-09-21T07:44:04.830 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 153 2024-09-21T07:44:04.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.830+0000 7fe4e9c00640 1 -- 172.21.15.39:0/3851787341 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe4c0078b90 msgr2=0x7fe4c007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:04.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.830+0000 7fe4e9c00640 1 --2- 172.21.15.39:0/3851787341 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe4c0078b90 0x7fe4c007b0a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fe4e40ccc30 tx=0x7fe4cc005f90 comp rx=0 tx=0).stop 2024-09-21T07:44:04.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.830+0000 7fe4e9c00640 1 -- 172.21.15.39:0/3851787341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4e406f800 msgr2=0x7fe4e40b3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:04.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.830+0000 7fe4e9c00640 1 --2- 172.21.15.39:0/3851787341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4e406f800 0x7fe4e40b3470 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4009950 tx=0x7fe4d4033c20 comp rx=0 tx=0).stop 2024-09-21T07:44:04.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.830+0000 7fe4e9c00640 1 -- 172.21.15.39:0/3851787341 shutdown_connections 2024-09-21T07:44:04.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.830+0000 7fe4e9c00640 1 --2- 172.21.15.39:0/3851787341 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe4c0078b90 0x7fe4c007b0a0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:04.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.830+0000 7fe4e9c00640 1 --2- 172.21.15.39:0/3851787341 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe4e415cd30 0x7fe4e41f5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:04.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.830+0000 7fe4e9c00640 1 --2- 172.21.15.39:0/3851787341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4e406fc10 0x7fe4e41f56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:04.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.830+0000 7fe4e9c00640 1 --2- 172.21.15.39:0/3851787341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4e406f800 0x7fe4e40b3470 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:04.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.830+0000 7fe4e9c00640 1 -- 172.21.15.39:0/3851787341 >> 172.21.15.39:0/3851787341 conn(0x7fe4e406cfc0 msgr2=0x7fe4e40b2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:04.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.830+0000 7fe4e9c00640 1 -- 172.21.15.39:0/3851787341 shutdown_connections 2024-09-21T07:44:04.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:04.830+0000 7fe4e9c00640 1 -- 172.21.15.39:0/3851787341 wait complete. 2024-09-21T07:44:04.857 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2024-09-21T07:44:05.103 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_snap":20,"last_created":17,"last_destroyed":19,"pending_noop":[],"snaps":[],"need_to_purge":{},"pending_update":[{"version":44,"snapinfo":{"snapid":20,"ino":1099511627777,"stamp":"2024-09-21T07:43:46.764013+0000","name":"s4","metadata":{}}}],"pending_destroy":[]} 2024-09-21T07:44:05.103 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 17, "last_destroyed": 19, "last_snap": 20, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [ { "snapInfinityo": { "ino": 1099511627777, "metadata": {}, "name": "s4", "snapid": 20, "stamp": "2024-09-21T07:43:46.764013+0000" }, "version": 44 } ], "snaps": [] } 2024-09-21T07:44:05.104 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 6:1 2024-09-21T07:44:05.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.329+0000 7f34f7e00640 1 -- 172.21.15.39:0/3221263923 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34f015fba0 msgr2=0x7f34f00d0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:05.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.329+0000 7f34f7e00640 1 --2- 172.21.15.39:0/3221263923 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34f015fba0 0x7f34f00d0c20 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f34f0072c00 tx=0x7f34e802a560 comp rx=0 tx=0).stop 2024-09-21T07:44:05.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.329+0000 7f34f7e00640 1 -- 172.21.15.39:0/3221263923 shutdown_connections 2024-09-21T07:44:05.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.329+0000 7f34f7e00640 1 --2- 172.21.15.39:0/3221263923 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34f015fba0 0x7f34f00d0c20 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:05.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.329+0000 7f34f7e00640 1 --2- 172.21.15.39:0/3221263923 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f34f015f790 0x7f34f00d06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:05.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.329+0000 7f34f7e00640 1 --2- 172.21.15.39:0/3221263923 >> v2:172.21.15.148:3300/0 conn(0x7f34f00b47c0 0x7f34f0155800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:05.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.329+0000 7f34f7e00640 1 -- 172.21.15.39:0/3221263923 >> 172.21.15.39:0/3221263923 conn(0x7f34f006cfc0 msgr2=0x7f34f006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:05.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.330+0000 7f34f7e00640 1 -- 172.21.15.39:0/3221263923 shutdown_connections 2024-09-21T07:44:05.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.330+0000 7f34f7e00640 1 -- 172.21.15.39:0/3221263923 wait complete. 2024-09-21T07:44:05.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.332+0000 7f34f7e00640 1 Processor -- start 2024-09-21T07:44:05.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.332+0000 7f34f7e00640 1 -- start start 2024-09-21T07:44:05.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.333+0000 7f34f7e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34f015f790 0x7f34f01f4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:05.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.333+0000 7f34f7e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f34f015fba0 0x7f34f00b2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:05.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.333+0000 7f34f7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34f00b2750 0x7f34f01e9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:05.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.333+0000 7f34f7e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f34f0162b00 con 0x7f34f00b2750 2024-09-21T07:44:05.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.333+0000 7f34f7e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f34f0162980 con 0x7f34f015f790 2024-09-21T07:44:05.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.333+0000 7f34f7e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f34f0162c80 con 0x7f34f015fba0 2024-09-21T07:44:05.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.334+0000 7f34f5e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34f00b2750 0x7f34f01e9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:05.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.334+0000 7f34f5e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34f00b2750 0x7f34f01e9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37320/0 (socket says 172.21.15.39:37320) 2024-09-21T07:44:05.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.334+0000 7f34f5e00640 1 -- 172.21.15.39:0/798342037 learned_addr learned my addr 172.21.15.39:0/798342037 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:05.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.334+0000 7f34f4a00640 1 --2- 172.21.15.39:0/798342037 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f34f015fba0 0x7f34f00b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:05.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.334+0000 7f34f5400640 1 --2- 172.21.15.39:0/798342037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34f015f790 0x7f34f01f4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:05.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.334+0000 7f34f5e00640 1 -- 172.21.15.39:0/798342037 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f34f015fba0 msgr2=0x7f34f00b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:05.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.334+0000 7f34f5e00640 1 --2- 172.21.15.39:0/798342037 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f34f015fba0 0x7f34f00b2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:05.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.334+0000 7f34f5e00640 1 -- 172.21.15.39:0/798342037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34f015f790 msgr2=0x7f34f01f4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:05.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.334+0000 7f34f5e00640 1 --2- 172.21.15.39:0/798342037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34f015f790 0x7f34f01f4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:05.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.334+0000 7f34f5e00640 1 -- 172.21.15.39:0/798342037 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f0156d70 con 0x7f34f00b2750 2024-09-21T07:44:05.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.334+0000 7f34f5e00640 1 --2- 172.21.15.39:0/798342037 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34f00b2750 0x7f34f01e9df0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f34e8034040 tx=0x7f34e8003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:05.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.335+0000 7f34ed600640 1 -- 172.21.15.39:0/798342037 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e803e030 con 0x7f34f00b2750 2024-09-21T07:44:05.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.335+0000 7f34ed600640 1 -- 172.21.15.39:0/798342037 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34e8003e40 con 0x7f34f00b2750 2024-09-21T07:44:05.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.335+0000 7f34ed600640 1 -- 172.21.15.39:0/798342037 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e8035040 con 0x7f34f00b2750 2024-09-21T07:44:05.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.335+0000 7f34f7e00640 1 -- 172.21.15.39:0/798342037 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34f0157050 con 0x7f34f00b2750 2024-09-21T07:44:05.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.335+0000 7f34f7e00640 1 -- 172.21.15.39:0/798342037 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34f0157610 con 0x7f34f00b2750 2024-09-21T07:44:05.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.336+0000 7f34f7e00640 1 -- 172.21.15.39:0/798342037 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34f0163210 con 0x7f34f00b2750 2024-09-21T07:44:05.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.337+0000 7f34ed600640 1 -- 172.21.15.39:0/798342037 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f34e8041030 con 0x7f34f00b2750 2024-09-21T07:44:05.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.338+0000 7f34ed600640 1 --2- 172.21.15.39:0/798342037 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f34d0078bc0 0x7f34d007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:05.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.338+0000 7f34ed600640 1 -- 172.21.15.39:0/798342037 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8233+0+0 (secure 0 0 0) 0x7f34e80bc330 con 0x7f34f00b2750 2024-09-21T07:44:05.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.339+0000 7f34f5400640 1 --2- 172.21.15.39:0/798342037 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f34d0078bc0 0x7f34d007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:05.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.339+0000 7f34f5400640 1 --2- 172.21.15.39:0/798342037 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f34d0078bc0 0x7f34d007b0d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f34e4005fd0 tx=0x7f34e40097d0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:05.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.343+0000 7f34ed600640 1 -- 172.21.15.39:0/798342037 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f34e8083cf0 con 0x7f34f00b2750 2024-09-21T07:44:05.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.613+0000 7f34f7e00640 1 -- 172.21.15.39:0/798342037 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "6:1"} v 0) -- 0x7f34f01eb330 con 0x7f34f00b2750 2024-09-21T07:44:05.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.616+0000 7f34ed600640 1 -- 172.21.15.39:0/798342037 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "6:1"}]=0 failed mds gid 7366 v154) ==== 97+0+0 (secure 0 0 0) 0x7f34e80835f0 con 0x7f34f00b2750 2024-09-21T07:44:05.618 INFO:teuthology.orchestra.run.smithi039.stderr:failed mds gid 7366 2024-09-21T07:44:05.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.619+0000 7f34f7e00640 1 -- 172.21.15.39:0/798342037 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f34d0078bc0 msgr2=0x7f34d007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:05.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.619+0000 7f34f7e00640 1 --2- 172.21.15.39:0/798342037 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f34d0078bc0 0x7f34d007b0d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f34e4005fd0 tx=0x7f34e40097d0 comp rx=0 tx=0).stop 2024-09-21T07:44:05.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.619+0000 7f34f7e00640 1 -- 172.21.15.39:0/798342037 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34f00b2750 msgr2=0x7f34f01e9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:05.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.619+0000 7f34f7e00640 1 --2- 172.21.15.39:0/798342037 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34f00b2750 0x7f34f01e9df0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f34e8034040 tx=0x7f34e8003910 comp rx=0 tx=0).stop 2024-09-21T07:44:05.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.619+0000 7f34f7e00640 1 -- 172.21.15.39:0/798342037 shutdown_connections 2024-09-21T07:44:05.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.619+0000 7f34f7e00640 1 --2- 172.21.15.39:0/798342037 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f34d0078bc0 0x7f34d007b0d0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:05.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.619+0000 7f34f7e00640 1 --2- 172.21.15.39:0/798342037 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34f00b2750 0x7f34f01e9df0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:05.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.619+0000 7f34f7e00640 1 --2- 172.21.15.39:0/798342037 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f34f015fba0 0x7f34f00b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:05.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.619+0000 7f34f7e00640 1 --2- 172.21.15.39:0/798342037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34f015f790 0x7f34f01f4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:05.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.619+0000 7f34f7e00640 1 -- 172.21.15.39:0/798342037 >> 172.21.15.39:0/798342037 conn(0x7f34f006cfc0 msgr2=0x7f34f006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:05.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.619+0000 7f34f7e00640 1 -- 172.21.15.39:0/798342037 shutdown_connections 2024-09-21T07:44:05.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.619+0000 7f34f7e00640 1 -- 172.21.15.39:0/798342037 wait complete. 2024-09-21T07:44:05.650 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T07:44:05.650 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T07:44:05.651 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T07:44:05.651 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:44:05.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:44:05.651 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/daemon/state.py", line 146, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi148 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d' 2024-09-21T07:44:05.651 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T07:44:05.651 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T07:44:05.655 INFO:tasks.ceph.mds.d:Started 2024-09-21T07:44:05.656 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:44:05.851 INFO:tasks.ceph.mds.d.smithi148.stdout:starting mds.d at 2024-09-21T07:44:05.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.880+0000 7f1b6d000640 1 -- 172.21.15.39:0/1397833778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b68155e80 msgr2=0x7f1b680b4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:05.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.880+0000 7f1b6d000640 1 --2- 172.21.15.39:0/1397833778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b68155e80 0x7f1b680b4690 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f1b68072c00 tx=0x7f1b6002aa30 comp rx=0 tx=0).stop 2024-09-21T07:44:05.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.880+0000 7f1b6d000640 1 -- 172.21.15.39:0/1397833778 shutdown_connections 2024-09-21T07:44:05.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.880+0000 7f1b6d000640 1 --2- 172.21.15.39:0/1397833778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b68155e80 0x7f1b680b4690 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:05.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.880+0000 7f1b6d000640 1 --2- 172.21.15.39:0/1397833778 >> v2:172.21.15.148:3300/0 conn(0x7f1b6815f360 0x7f1b6815bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:05.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.880+0000 7f1b6d000640 1 -- 172.21.15.39:0/1397833778 >> 172.21.15.39:0/1397833778 conn(0x7f1b6806cfc0 msgr2=0x7f1b6806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:05.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.880+0000 7f1b6d000640 1 -- 172.21.15.39:0/1397833778 shutdown_connections 2024-09-21T07:44:05.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.880+0000 7f1b6d000640 1 -- 172.21.15.39:0/1397833778 wait complete. 2024-09-21T07:44:05.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.883+0000 7f1b6d000640 1 Processor -- start 2024-09-21T07:44:05.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.883+0000 7f1b6d000640 1 -- start start 2024-09-21T07:44:05.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.884+0000 7f1b6d000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b68155a70 0x7f1b681f9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:05.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.884+0000 7f1b6d000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1b68155e80 0x7f1b681428b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:05.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.884+0000 7f1b6d000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b6815f360 0x7f1b68145a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:05.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.884+0000 7f1b6d000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1b680b4fa0 con 0x7f1b68155a70 2024-09-21T07:44:05.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.884+0000 7f1b6d000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1b680b4e20 con 0x7f1b6815f360 2024-09-21T07:44:05.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.884+0000 7f1b6d000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f1b680d0620 con 0x7f1b68155e80 2024-09-21T07:44:05.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.885+0000 7f1b65e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b68155a70 0x7f1b681f9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:05.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.885+0000 7f1b65e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b68155a70 0x7f1b681f9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37332/0 (socket says 172.21.15.39:37332) 2024-09-21T07:44:05.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.885+0000 7f1b65e00640 1 -- 172.21.15.39:0/150049031 learned_addr learned my addr 172.21.15.39:0/150049031 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:05.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.885+0000 7f1b66800640 1 --2- 172.21.15.39:0/150049031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b6815f360 0x7f1b68145a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:05.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.885+0000 7f1b5fe00640 1 --2- 172.21.15.39:0/150049031 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1b68155e80 0x7f1b681428b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:05.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.885+0000 7f1b65e00640 1 -- 172.21.15.39:0/150049031 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1b68155e80 msgr2=0x7f1b681428b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:05.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.885+0000 7f1b65e00640 1 --2- 172.21.15.39:0/150049031 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1b68155e80 0x7f1b681428b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:05.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.885+0000 7f1b65e00640 1 -- 172.21.15.39:0/150049031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b6815f360 msgr2=0x7f1b68145a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:05.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.885+0000 7f1b65e00640 1 --2- 172.21.15.39:0/150049031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b6815f360 0x7f1b68145a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:05.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.885+0000 7f1b65e00640 1 -- 172.21.15.39:0/150049031 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b68142df0 con 0x7f1b68155a70 2024-09-21T07:44:05.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.885+0000 7f1b5fe00640 1 --2- 172.21.15.39:0/150049031 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1b68155e80 0x7f1b681428b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:44:05.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.885+0000 7f1b65e00640 1 --2- 172.21.15.39:0/150049031 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b68155a70 0x7f1b681f9110 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f1b5800d740 tx=0x7f1b5800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:05.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.886+0000 7f1b5e000640 1 -- 172.21.15.39:0/150049031 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b58017070 con 0x7f1b68155a70 2024-09-21T07:44:05.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.886+0000 7f1b5e000640 1 -- 172.21.15.39:0/150049031 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b58009d20 con 0x7f1b68155a70 2024-09-21T07:44:05.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.886+0000 7f1b5e000640 1 -- 172.21.15.39:0/150049031 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b580106c0 con 0x7f1b68155a70 2024-09-21T07:44:05.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.886+0000 7f1b6d000640 1 -- 172.21.15.39:0/150049031 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b681f9ae0 con 0x7f1b68155a70 2024-09-21T07:44:05.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.886+0000 7f1b6d000640 1 -- 172.21.15.39:0/150049031 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b68145290 con 0x7f1b68155a70 2024-09-21T07:44:05.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.887+0000 7f1b6d000640 1 -- 172.21.15.39:0/150049031 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b680b4690 con 0x7f1b68155a70 2024-09-21T07:44:05.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.888+0000 7f1b5e000640 1 -- 172.21.15.39:0/150049031 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1b5801a070 con 0x7f1b68155a70 2024-09-21T07:44:05.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.890+0000 7f1b5e000640 1 --2- 172.21.15.39:0/150049031 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1b44078b90 0x7f1b4407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:05.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.890+0000 7f1b5e000640 1 -- 172.21.15.39:0/150049031 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8291+0+0 (secure 0 0 0) 0x7f1b5809b870 con 0x7f1b68155a70 2024-09-21T07:44:05.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.890+0000 7f1b5fe00640 1 --2- 172.21.15.39:0/150049031 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1b44078b90 0x7f1b4407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:05.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.891+0000 7f1b5fe00640 1 --2- 172.21.15.39:0/150049031 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1b44078b90 0x7f1b4407b0a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f1b68152af0 tx=0x7f1b50009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:05.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:05.894+0000 7f1b5e000640 1 -- 172.21.15.39:0/150049031 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1b68142df0 con 0x7f1b68155a70 2024-09-21T07:44:06.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:06.130+0000 7f1b6d000640 1 -- 172.21.15.39:0/150049031 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b680708d0 con 0x7f1b68155a70 2024-09-21T07:44:06.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:06.131+0000 7f1b5e000640 1 -- 172.21.15.39:0/150049031 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 155 v155) ==== 95+0+4424 (secure 0 0 0) 0x7f1b58062c20 con 0x7f1b68155a70 2024-09-21T07:44:06.133 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:06.133 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":155,"btime":"2024-09-21T07:44:05:621535+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":155,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:44:05.621522+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":8052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8052":{"gid":8052,"name":"a","rank":1,"incarnation":155,"state":"up:replay","state_seq":1,"addr":"172.21.15.39:6835/1551311805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1551311805},{"type":"v1","addr":"172.21.15.39:6835","nonce":1551311805}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311]},"id":6}]} 2024-09-21T07:44:06.133 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 155 2024-09-21T07:44:06.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:06.133+0000 7f1b6d000640 1 -- 172.21.15.39:0/150049031 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1b44078b90 msgr2=0x7f1b4407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:06.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:06.133+0000 7f1b6d000640 1 --2- 172.21.15.39:0/150049031 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1b44078b90 0x7f1b4407b0a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f1b68152af0 tx=0x7f1b50009040 comp rx=0 tx=0).stop 2024-09-21T07:44:06.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:06.133+0000 7f1b6d000640 1 -- 172.21.15.39:0/150049031 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b68155a70 msgr2=0x7f1b681f9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:06.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:06.133+0000 7f1b6d000640 1 --2- 172.21.15.39:0/150049031 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b68155a70 0x7f1b681f9110 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f1b5800d740 tx=0x7f1b5800dc00 comp rx=0 tx=0).stop 2024-09-21T07:44:06.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:06.133+0000 7f1b6d000640 1 -- 172.21.15.39:0/150049031 shutdown_connections 2024-09-21T07:44:06.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:06.133+0000 7f1b6d000640 1 --2- 172.21.15.39:0/150049031 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1b44078b90 0x7f1b4407b0a0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:06.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:06.133+0000 7f1b6d000640 1 --2- 172.21.15.39:0/150049031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b6815f360 0x7f1b68145a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:06.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:06.133+0000 7f1b6d000640 1 --2- 172.21.15.39:0/150049031 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1b68155e80 0x7f1b681428b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:06.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:06.133+0000 7f1b6d000640 1 --2- 172.21.15.39:0/150049031 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b68155a70 0x7f1b681f9110 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:06.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:06.133+0000 7f1b6d000640 1 -- 172.21.15.39:0/150049031 >> 172.21.15.39:0/150049031 conn(0x7f1b6806cfc0 msgr2=0x7f1b6806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:06.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:06.133+0000 7f1b6d000640 1 -- 172.21.15.39:0/150049031 shutdown_connections 2024-09-21T07:44:06.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:06.133+0000 7f1b6d000640 1 -- 172.21.15.39:0/150049031 wait complete. 2024-09-21T07:44:06.157 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-09-21T07:44:06.642 INFO:tasks.ceph.mds.c.smithi039.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_rollback(ceph::cref_t&)' thread 7f9205800640 time 2024-09-21T07:44:06.641292+0000 2024-09-21T07:44:06.642 INFO:tasks.ceph.mds.c.smithi039.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: 202: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 8) 2024-09-21T07:44:06.642 INFO:tasks.ceph.mds.c.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:44:06.642 INFO:tasks.ceph.mds.c.smithi039.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f920c85654b] 2024-09-21T07:44:06.642 INFO:tasks.ceph.mds.c.smithi039.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f920c856758] 2024-09-21T07:44:06.642 INFO:tasks.ceph.mds.c.smithi039.stderr: 3: (MDSTableServer::handle_rollback(boost::intrusive_ptr const&)+0x24b) [0x561cfbeb4e8f] 2024-09-21T07:44:06.643 INFO:tasks.ceph.mds.c.smithi039.stderr: 4: (MDSTableServer::handle_request(boost::intrusive_ptr const&)+0xa1) [0x561cfbeb8917] 2024-09-21T07:44:06.643 INFO:tasks.ceph.mds.c.smithi039.stderr: 5: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xca5) [0x561cfbb51de3] 2024-09-21T07:44:06.643 INFO:tasks.ceph.mds.c.smithi039.stderr: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x561cfbb55523] 2024-09-21T07:44:06.643 INFO:tasks.ceph.mds.c.smithi039.stderr: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x561cfbb55e4a] 2024-09-21T07:44:06.643 INFO:tasks.ceph.mds.c.smithi039.stderr: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x561cfbb38fa4] 2024-09-21T07:44:06.643 INFO:tasks.ceph.mds.c.smithi039.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f920c9a57df] 2024-09-21T07:44:06.643 INFO:tasks.ceph.mds.c.smithi039.stderr: 10: (DispatchQueue::entry()+0x60f) [0x7f920c9a60d1] 2024-09-21T07:44:06.643 INFO:tasks.ceph.mds.c.smithi039.stderr: 11: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f920ca93f5b] 2024-09-21T07:44:06.643 INFO:tasks.ceph.mds.c.smithi039.stderr: 12: (Thread::entry_wrapper()+0x43) [0x7f920c82fd57] 2024-09-21T07:44:06.643 INFO:tasks.ceph.mds.c.smithi039.stderr: 13: (Thread::_entry_func(void*)+0xd) [0x7f920c82fd73] 2024-09-21T07:44:06.643 INFO:tasks.ceph.mds.c.smithi039.stderr: 14: /lib64/libc.so.6(+0x89c52) [0x7f920be89c52] 2024-09-21T07:44:06.643 INFO:tasks.ceph.mds.c.smithi039.stderr: 15: /lib64/libc.so.6(+0x10ec80) [0x7f920bf0ec80] 2024-09-21T07:44:06.643 INFO:tasks.ceph.mds.c.smithi039.stderr:2024-09-21T07:44:06.640+0000 7f9205800640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_rollback(ceph::cref_t&)' thread 7f9205800640 time 2024-09-21T07:44:06.641292+0000 2024-09-21T07:44:06.644 INFO:tasks.ceph.mds.c.smithi039.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: 202: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 8) 2024-09-21T07:44:06.644 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:44:06.644 INFO:tasks.ceph.mds.c.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:44:06.644 INFO:tasks.ceph.mds.c.smithi039.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f920c85654b] 2024-09-21T07:44:06.644 INFO:tasks.ceph.mds.c.smithi039.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f920c856758] 2024-09-21T07:44:06.644 INFO:tasks.ceph.mds.c.smithi039.stderr: 3: (MDSTableServer::handle_rollback(boost::intrusive_ptr const&)+0x24b) [0x561cfbeb4e8f] 2024-09-21T07:44:06.644 INFO:tasks.ceph.mds.c.smithi039.stderr: 4: (MDSTableServer::handle_request(boost::intrusive_ptr const&)+0xa1) [0x561cfbeb8917] 2024-09-21T07:44:06.644 INFO:tasks.ceph.mds.c.smithi039.stderr: 5: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xca5) [0x561cfbb51de3] 2024-09-21T07:44:06.644 INFO:tasks.ceph.mds.c.smithi039.stderr: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x561cfbb55523] 2024-09-21T07:44:06.644 INFO:tasks.ceph.mds.c.smithi039.stderr: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x561cfbb55e4a] 2024-09-21T07:44:06.644 INFO:tasks.ceph.mds.c.smithi039.stderr: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x561cfbb38fa4] 2024-09-21T07:44:06.644 INFO:tasks.ceph.mds.c.smithi039.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f920c9a57df] 2024-09-21T07:44:06.645 INFO:tasks.ceph.mds.c.smithi039.stderr: 10: (DispatchQueue::entry()+0x60f) [0x7f920c9a60d1] 2024-09-21T07:44:06.645 INFO:tasks.ceph.mds.c.smithi039.stderr: 11: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f920ca93f5b] 2024-09-21T07:44:06.645 INFO:tasks.ceph.mds.c.smithi039.stderr: 12: (Thread::entry_wrapper()+0x43) [0x7f920c82fd57] 2024-09-21T07:44:06.645 INFO:tasks.ceph.mds.c.smithi039.stderr: 13: (Thread::_entry_func(void*)+0xd) [0x7f920c82fd73] 2024-09-21T07:44:06.645 INFO:tasks.ceph.mds.c.smithi039.stderr: 14: /lib64/libc.so.6(+0x89c52) [0x7f920be89c52] 2024-09-21T07:44:06.645 INFO:tasks.ceph.mds.c.smithi039.stderr: 15: /lib64/libc.so.6(+0x10ec80) [0x7f920bf0ec80] 2024-09-21T07:44:06.645 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:44:06.645 INFO:tasks.ceph.mds.c.smithi039.stderr:*** Caught signal (Aborted) ** 2024-09-21T07:44:06.645 INFO:tasks.ceph.mds.c.smithi039.stderr: in thread 7f9205800640 thread_name:ms_dispatch 2024-09-21T07:44:06.646 INFO:tasks.ceph.mds.c.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:44:06.646 INFO:tasks.ceph.mds.c.smithi039.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f920be3e6f0] 2024-09-21T07:44:06.646 INFO:tasks.ceph.mds.c.smithi039.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f920be8b99c] 2024-09-21T07:44:06.646 INFO:tasks.ceph.mds.c.smithi039.stderr: 3: raise() 2024-09-21T07:44:06.646 INFO:tasks.ceph.mds.c.smithi039.stderr: 4: abort() 2024-09-21T07:44:06.646 INFO:tasks.ceph.mds.c.smithi039.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7f920c856686] 2024-09-21T07:44:06.646 INFO:tasks.ceph.mds.c.smithi039.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f920c856758] 2024-09-21T07:44:06.646 INFO:tasks.ceph.mds.c.smithi039.stderr: 7: (MDSTableServer::handle_rollback(boost::intrusive_ptr const&)+0x24b) [0x561cfbeb4e8f] 2024-09-21T07:44:06.646 INFO:tasks.ceph.mds.c.smithi039.stderr: 8: (MDSTableServer::handle_request(boost::intrusive_ptr const&)+0xa1) [0x561cfbeb8917] 2024-09-21T07:44:06.646 INFO:tasks.ceph.mds.c.smithi039.stderr: 9: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xca5) [0x561cfbb51de3] 2024-09-21T07:44:06.646 INFO:tasks.ceph.mds.c.smithi039.stderr: 10: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x561cfbb55523] 2024-09-21T07:44:06.646 INFO:tasks.ceph.mds.c.smithi039.stderr: 11: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x561cfbb55e4a] 2024-09-21T07:44:06.646 INFO:tasks.ceph.mds.c.smithi039.stderr: 12: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x561cfbb38fa4] 2024-09-21T07:44:06.647 INFO:tasks.ceph.mds.c.smithi039.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f920c9a57df] 2024-09-21T07:44:06.647 INFO:tasks.ceph.mds.c.smithi039.stderr: 14: (DispatchQueue::entry()+0x60f) [0x7f920c9a60d1] 2024-09-21T07:44:06.647 INFO:tasks.ceph.mds.c.smithi039.stderr: 15: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f920ca93f5b] 2024-09-21T07:44:06.647 INFO:tasks.ceph.mds.c.smithi039.stderr: 16: (Thread::entry_wrapper()+0x43) [0x7f920c82fd57] 2024-09-21T07:44:06.647 INFO:tasks.ceph.mds.c.smithi039.stderr: 17: (Thread::_entry_func(void*)+0xd) [0x7f920c82fd73] 2024-09-21T07:44:06.647 INFO:tasks.ceph.mds.c.smithi039.stderr: 18: /lib64/libc.so.6(+0x89c52) [0x7f920be89c52] 2024-09-21T07:44:06.647 INFO:tasks.ceph.mds.c.smithi039.stderr: 19: /lib64/libc.so.6(+0x10ec80) [0x7f920bf0ec80] 2024-09-21T07:44:06.647 INFO:tasks.ceph.mds.c.smithi039.stderr:2024-09-21T07:44:06.642+0000 7f9205800640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:44:06.647 INFO:tasks.ceph.mds.c.smithi039.stderr: in thread 7f9205800640 thread_name:ms_dispatch 2024-09-21T07:44:06.647 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:44:06.647 INFO:tasks.ceph.mds.c.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:44:06.647 INFO:tasks.ceph.mds.c.smithi039.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f920be3e6f0] 2024-09-21T07:44:06.647 INFO:tasks.ceph.mds.c.smithi039.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f920be8b99c] 2024-09-21T07:44:06.648 INFO:tasks.ceph.mds.c.smithi039.stderr: 3: raise() 2024-09-21T07:44:06.648 INFO:tasks.ceph.mds.c.smithi039.stderr: 4: abort() 2024-09-21T07:44:06.648 INFO:tasks.ceph.mds.c.smithi039.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7f920c856686] 2024-09-21T07:44:06.648 INFO:tasks.ceph.mds.c.smithi039.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f920c856758] 2024-09-21T07:44:06.648 INFO:tasks.ceph.mds.c.smithi039.stderr: 7: (MDSTableServer::handle_rollback(boost::intrusive_ptr const&)+0x24b) [0x561cfbeb4e8f] 2024-09-21T07:44:06.648 INFO:tasks.ceph.mds.c.smithi039.stderr: 8: (MDSTableServer::handle_request(boost::intrusive_ptr const&)+0xa1) [0x561cfbeb8917] 2024-09-21T07:44:06.648 INFO:tasks.ceph.mds.c.smithi039.stderr: 9: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xca5) [0x561cfbb51de3] 2024-09-21T07:44:06.648 INFO:tasks.ceph.mds.c.smithi039.stderr: 10: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x561cfbb55523] 2024-09-21T07:44:06.648 INFO:tasks.ceph.mds.c.smithi039.stderr: 11: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x561cfbb55e4a] 2024-09-21T07:44:06.648 INFO:tasks.ceph.mds.c.smithi039.stderr: 12: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x561cfbb38fa4] 2024-09-21T07:44:06.648 INFO:tasks.ceph.mds.c.smithi039.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f920c9a57df] 2024-09-21T07:44:06.648 INFO:tasks.ceph.mds.c.smithi039.stderr: 14: (DispatchQueue::entry()+0x60f) [0x7f920c9a60d1] 2024-09-21T07:44:06.648 INFO:tasks.ceph.mds.c.smithi039.stderr: 15: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f920ca93f5b] 2024-09-21T07:44:06.648 INFO:tasks.ceph.mds.c.smithi039.stderr: 16: (Thread::entry_wrapper()+0x43) [0x7f920c82fd57] 2024-09-21T07:44:06.648 INFO:tasks.ceph.mds.c.smithi039.stderr: 17: (Thread::_entry_func(void*)+0xd) [0x7f920c82fd73] 2024-09-21T07:44:06.649 INFO:tasks.ceph.mds.c.smithi039.stderr: 18: /lib64/libc.so.6(+0x89c52) [0x7f920be89c52] 2024-09-21T07:44:06.649 INFO:tasks.ceph.mds.c.smithi039.stderr: 19: /lib64/libc.so.6(+0x10ec80) [0x7f920bf0ec80] 2024-09-21T07:44:06.649 INFO:tasks.ceph.mds.c.smithi039.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:44:06.649 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:44:06.656 INFO:tasks.ceph.mds.c.smithi039.stderr: -8730> 2024-09-21T07:42:04.670+0000 7f9202600640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:44:06.659 INFO:tasks.ceph.mds.c.smithi039.stderr: -5583> 2024-09-21T07:42:54.674+0000 7f9202600640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:44:06.663 INFO:tasks.ceph.mds.c.smithi039.stderr: -2436> 2024-09-21T07:43:39.677+0000 7f9202600640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:44:06.665 INFO:tasks.ceph.mds.c.smithi039.stderr: -1> 2024-09-21T07:44:06.640+0000 7f9205800640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_rollback(ceph::cref_t&)' thread 7f9205800640 time 2024-09-21T07:44:06.641292+0000 2024-09-21T07:44:06.665 INFO:tasks.ceph.mds.c.smithi039.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: 202: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 8) 2024-09-21T07:44:06.666 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:44:06.666 INFO:tasks.ceph.mds.c.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:44:06.666 INFO:tasks.ceph.mds.c.smithi039.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f920c85654b] 2024-09-21T07:44:06.666 INFO:tasks.ceph.mds.c.smithi039.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f920c856758] 2024-09-21T07:44:06.666 INFO:tasks.ceph.mds.c.smithi039.stderr: 3: (MDSTableServer::handle_rollback(boost::intrusive_ptr const&)+0x24b) [0x561cfbeb4e8f] 2024-09-21T07:44:06.666 INFO:tasks.ceph.mds.c.smithi039.stderr: 4: (MDSTableServer::handle_request(boost::intrusive_ptr const&)+0xa1) [0x561cfbeb8917] 2024-09-21T07:44:06.666 INFO:tasks.ceph.mds.c.smithi039.stderr: 5: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xca5) [0x561cfbb51de3] 2024-09-21T07:44:06.666 INFO:tasks.ceph.mds.c.smithi039.stderr: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x561cfbb55523] 2024-09-21T07:44:06.666 INFO:tasks.ceph.mds.c.smithi039.stderr: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x561cfbb55e4a] 2024-09-21T07:44:06.666 INFO:tasks.ceph.mds.c.smithi039.stderr: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x561cfbb38fa4] 2024-09-21T07:44:06.666 INFO:tasks.ceph.mds.c.smithi039.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f920c9a57df] 2024-09-21T07:44:06.667 INFO:tasks.ceph.mds.c.smithi039.stderr: 10: (DispatchQueue::entry()+0x60f) [0x7f920c9a60d1] 2024-09-21T07:44:06.667 INFO:tasks.ceph.mds.c.smithi039.stderr: 11: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f920ca93f5b] 2024-09-21T07:44:06.667 INFO:tasks.ceph.mds.c.smithi039.stderr: 12: (Thread::entry_wrapper()+0x43) [0x7f920c82fd57] 2024-09-21T07:44:06.667 INFO:tasks.ceph.mds.c.smithi039.stderr: 13: (Thread::_entry_func(void*)+0xd) [0x7f920c82fd73] 2024-09-21T07:44:06.667 INFO:tasks.ceph.mds.c.smithi039.stderr: 14: /lib64/libc.so.6(+0x89c52) [0x7f920be89c52] 2024-09-21T07:44:06.667 INFO:tasks.ceph.mds.c.smithi039.stderr: 15: /lib64/libc.so.6(+0x10ec80) [0x7f920bf0ec80] 2024-09-21T07:44:06.667 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:44:06.667 INFO:tasks.ceph.mds.c.smithi039.stderr: 0> 2024-09-21T07:44:06.642+0000 7f9205800640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:44:06.667 INFO:tasks.ceph.mds.c.smithi039.stderr: in thread 7f9205800640 thread_name:ms_dispatch 2024-09-21T07:44:06.667 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:44:06.668 INFO:tasks.ceph.mds.c.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:44:06.668 INFO:tasks.ceph.mds.c.smithi039.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f920be3e6f0] 2024-09-21T07:44:06.668 INFO:tasks.ceph.mds.c.smithi039.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f920be8b99c] 2024-09-21T07:44:06.668 INFO:tasks.ceph.mds.c.smithi039.stderr: 3: raise() 2024-09-21T07:44:06.668 INFO:tasks.ceph.mds.c.smithi039.stderr: 4: abort() 2024-09-21T07:44:06.668 INFO:tasks.ceph.mds.c.smithi039.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7f920c856686] 2024-09-21T07:44:06.668 INFO:tasks.ceph.mds.c.smithi039.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f920c856758] 2024-09-21T07:44:06.668 INFO:tasks.ceph.mds.c.smithi039.stderr: 7: (MDSTableServer::handle_rollback(boost::intrusive_ptr const&)+0x24b) [0x561cfbeb4e8f] 2024-09-21T07:44:06.668 INFO:tasks.ceph.mds.c.smithi039.stderr: 8: (MDSTableServer::handle_request(boost::intrusive_ptr const&)+0xa1) [0x561cfbeb8917] 2024-09-21T07:44:06.668 INFO:tasks.ceph.mds.c.smithi039.stderr: 9: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xca5) [0x561cfbb51de3] 2024-09-21T07:44:06.668 INFO:tasks.ceph.mds.c.smithi039.stderr: 10: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x561cfbb55523] 2024-09-21T07:44:06.668 INFO:tasks.ceph.mds.c.smithi039.stderr: 11: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x561cfbb55e4a] 2024-09-21T07:44:06.668 INFO:tasks.ceph.mds.c.smithi039.stderr: 12: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x561cfbb38fa4] 2024-09-21T07:44:06.668 INFO:tasks.ceph.mds.c.smithi039.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f920c9a57df] 2024-09-21T07:44:06.669 INFO:tasks.ceph.mds.c.smithi039.stderr: 14: (DispatchQueue::entry()+0x60f) [0x7f920c9a60d1] 2024-09-21T07:44:06.669 INFO:tasks.ceph.mds.c.smithi039.stderr: 15: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f920ca93f5b] 2024-09-21T07:44:06.669 INFO:tasks.ceph.mds.c.smithi039.stderr: 16: (Thread::entry_wrapper()+0x43) [0x7f920c82fd57] 2024-09-21T07:44:06.669 INFO:tasks.ceph.mds.c.smithi039.stderr: 17: (Thread::_entry_func(void*)+0xd) [0x7f920c82fd73] 2024-09-21T07:44:06.669 INFO:tasks.ceph.mds.c.smithi039.stderr: 18: /lib64/libc.so.6(+0x89c52) [0x7f920be89c52] 2024-09-21T07:44:06.669 INFO:tasks.ceph.mds.c.smithi039.stderr: 19: /lib64/libc.so.6(+0x10ec80) [0x7f920bf0ec80] 2024-09-21T07:44:06.669 INFO:tasks.ceph.mds.c.smithi039.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:44:06.669 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:44:06.669 INFO:tasks.ceph.mds.c.smithi039.stderr: -8730> 2024-09-21T07:42:04.670+0000 7f9202600640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:44:06.671 INFO:tasks.ceph.mds.c.smithi039.stderr: -5583> 2024-09-21T07:42:54.674+0000 7f9202600640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:44:06.674 INFO:tasks.ceph.mds.c.smithi039.stderr: -2436> 2024-09-21T07:43:39.677+0000 7f9202600640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:44:06.676 INFO:tasks.ceph.mds.c.smithi039.stderr: -1> 2024-09-21T07:44:06.640+0000 7f9205800640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_rollback(ceph::cref_t&)' thread 7f9205800640 time 2024-09-21T07:44:06.641292+0000 2024-09-21T07:44:06.676 INFO:tasks.ceph.mds.c.smithi039.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableServer.cc: 202: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 8) 2024-09-21T07:44:06.676 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:44:06.676 INFO:tasks.ceph.mds.c.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:44:06.676 INFO:tasks.ceph.mds.c.smithi039.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f920c85654b] 2024-09-21T07:44:06.677 INFO:tasks.ceph.mds.c.smithi039.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f920c856758] 2024-09-21T07:44:06.677 INFO:tasks.ceph.mds.c.smithi039.stderr: 3: (MDSTableServer::handle_rollback(boost::intrusive_ptr const&)+0x24b) [0x561cfbeb4e8f] 2024-09-21T07:44:06.677 INFO:tasks.ceph.mds.c.smithi039.stderr: 4: (MDSTableServer::handle_request(boost::intrusive_ptr const&)+0xa1) [0x561cfbeb8917] 2024-09-21T07:44:06.677 INFO:tasks.ceph.mds.c.smithi039.stderr: 5: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xca5) [0x561cfbb51de3] 2024-09-21T07:44:06.677 INFO:tasks.ceph.mds.c.smithi039.stderr: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x561cfbb55523] 2024-09-21T07:44:06.677 INFO:tasks.ceph.mds.c.smithi039.stderr: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x561cfbb55e4a] 2024-09-21T07:44:06.678 INFO:tasks.ceph.mds.c.smithi039.stderr: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x561cfbb38fa4] 2024-09-21T07:44:06.678 INFO:tasks.ceph.mds.c.smithi039.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f920c9a57df] 2024-09-21T07:44:06.678 INFO:tasks.ceph.mds.c.smithi039.stderr: 10: (DispatchQueue::entry()+0x60f) [0x7f920c9a60d1] 2024-09-21T07:44:06.678 INFO:tasks.ceph.mds.c.smithi039.stderr: 11: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f920ca93f5b] 2024-09-21T07:44:06.678 INFO:tasks.ceph.mds.c.smithi039.stderr: 12: (Thread::entry_wrapper()+0x43) [0x7f920c82fd57] 2024-09-21T07:44:06.678 INFO:tasks.ceph.mds.c.smithi039.stderr: 13: (Thread::_entry_func(void*)+0xd) [0x7f920c82fd73] 2024-09-21T07:44:06.678 INFO:tasks.ceph.mds.c.smithi039.stderr: 14: /lib64/libc.so.6(+0x89c52) [0x7f920be89c52] 2024-09-21T07:44:06.679 INFO:tasks.ceph.mds.c.smithi039.stderr: 15: /lib64/libc.so.6(+0x10ec80) [0x7f920bf0ec80] 2024-09-21T07:44:06.679 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:44:06.679 INFO:tasks.ceph.mds.c.smithi039.stderr: 0> 2024-09-21T07:44:06.642+0000 7f9205800640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:44:06.679 INFO:tasks.ceph.mds.c.smithi039.stderr: in thread 7f9205800640 thread_name:ms_dispatch 2024-09-21T07:44:06.679 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:44:06.679 INFO:tasks.ceph.mds.c.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:44:06.679 INFO:tasks.ceph.mds.c.smithi039.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f920be3e6f0] 2024-09-21T07:44:06.679 INFO:tasks.ceph.mds.c.smithi039.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f920be8b99c] 2024-09-21T07:44:06.679 INFO:tasks.ceph.mds.c.smithi039.stderr: 3: raise() 2024-09-21T07:44:06.679 INFO:tasks.ceph.mds.c.smithi039.stderr: 4: abort() 2024-09-21T07:44:06.679 INFO:tasks.ceph.mds.c.smithi039.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7f920c856686] 2024-09-21T07:44:06.679 INFO:tasks.ceph.mds.c.smithi039.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f920c856758] 2024-09-21T07:44:06.680 INFO:tasks.ceph.mds.c.smithi039.stderr: 7: (MDSTableServer::handle_rollback(boost::intrusive_ptr const&)+0x24b) [0x561cfbeb4e8f] 2024-09-21T07:44:06.680 INFO:tasks.ceph.mds.c.smithi039.stderr: 8: (MDSTableServer::handle_request(boost::intrusive_ptr const&)+0xa1) [0x561cfbeb8917] 2024-09-21T07:44:06.680 INFO:tasks.ceph.mds.c.smithi039.stderr: 9: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xca5) [0x561cfbb51de3] 2024-09-21T07:44:06.680 INFO:tasks.ceph.mds.c.smithi039.stderr: 10: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x561cfbb55523] 2024-09-21T07:44:06.680 INFO:tasks.ceph.mds.c.smithi039.stderr: 11: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x561cfbb55e4a] 2024-09-21T07:44:06.680 INFO:tasks.ceph.mds.c.smithi039.stderr: 12: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x561cfbb38fa4] 2024-09-21T07:44:06.680 INFO:tasks.ceph.mds.c.smithi039.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7f920c9a57df] 2024-09-21T07:44:06.680 INFO:tasks.ceph.mds.c.smithi039.stderr: 14: (DispatchQueue::entry()+0x60f) [0x7f920c9a60d1] 2024-09-21T07:44:06.680 INFO:tasks.ceph.mds.c.smithi039.stderr: 15: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7f920ca93f5b] 2024-09-21T07:44:06.680 INFO:tasks.ceph.mds.c.smithi039.stderr: 16: (Thread::entry_wrapper()+0x43) [0x7f920c82fd57] 2024-09-21T07:44:06.680 INFO:tasks.ceph.mds.c.smithi039.stderr: 17: (Thread::_entry_func(void*)+0xd) [0x7f920c82fd73] 2024-09-21T07:44:06.681 INFO:tasks.ceph.mds.c.smithi039.stderr: 18: /lib64/libc.so.6(+0x89c52) [0x7f920be89c52] 2024-09-21T07:44:06.681 INFO:tasks.ceph.mds.c.smithi039.stderr: 19: /lib64/libc.so.6(+0x10ec80) [0x7f920bf0ec80] 2024-09-21T07:44:06.681 INFO:tasks.ceph.mds.c.smithi039.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:44:06.681 INFO:tasks.ceph.mds.c.smithi039.stderr: 2024-09-21T07:44:07.318 INFO:tasks.ceph.mds.c.smithi039.stderr:daemon-helper: command crashed with signal 6 2024-09-21T07:44:09.385 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-09-21T07:44:09.385 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-09-21T07:44:11.158 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:44:11.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.394+0000 7f31abe00640 1 -- 172.21.15.39:0/584749083 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31ac06e5a0 msgr2=0x7f31ac0b5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:11.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.394+0000 7f31abe00640 1 --2- 172.21.15.39:0/584749083 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31ac06e5a0 0x7f31ac0b5750 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f31a4005fd0 tx=0x7f31a402aa00 comp rx=0 tx=0).stop 2024-09-21T07:44:11.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.394+0000 7f31abe00640 1 -- 172.21.15.39:0/584749083 shutdown_connections 2024-09-21T07:44:11.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.394+0000 7f31abe00640 1 --2- 172.21.15.39:0/584749083 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f31ac06e190 0x7f31ac0b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:11.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.394+0000 7f31abe00640 1 --2- 172.21.15.39:0/584749083 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31ac06e5a0 0x7f31ac0b5750 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:11.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.394+0000 7f31abe00640 1 -- 172.21.15.39:0/584749083 >> 172.21.15.39:0/584749083 conn(0x7f31ac0cee40 msgr2=0x7f31ac0cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:11.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.394+0000 7f31abe00640 1 -- 172.21.15.39:0/584749083 shutdown_connections 2024-09-21T07:44:11.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.394+0000 7f31abe00640 1 -- 172.21.15.39:0/584749083 wait complete. 2024-09-21T07:44:11.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.396+0000 7f31abe00640 1 Processor -- start 2024-09-21T07:44:11.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.396+0000 7f31abe00640 1 -- start start 2024-09-21T07:44:11.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.397+0000 7f31abe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31ac06e190 0x7f31ac09c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:11.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.398+0000 7f31abe00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f31ac06e5a0 0x7f31ac15ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:11.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.398+0000 7f31abe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31ac09ba30 0x7f31ac09da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:11.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.398+0000 7f31abe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f31ac0d07f0 con 0x7f31ac09ba30 2024-09-21T07:44:11.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.398+0000 7f31abe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f31ac0d0670 con 0x7f31ac06e190 2024-09-21T07:44:11.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.398+0000 7f31abe00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f31ac0d0970 con 0x7f31ac06e5a0 2024-09-21T07:44:11.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.398+0000 7f31ab400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31ac09ba30 0x7f31ac09da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:11.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.398+0000 7f31ab400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31ac09ba30 0x7f31ac09da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37362/0 (socket says 172.21.15.39:37362) 2024-09-21T07:44:11.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.398+0000 7f31ab400640 1 -- 172.21.15.39:0/1717640291 learned_addr learned my addr 172.21.15.39:0/1717640291 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:11.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.398+0000 7f31a3e00640 1 --2- 172.21.15.39:0/1717640291 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f31ac06e5a0 0x7f31ac15ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:11.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.398+0000 7f31aaa00640 1 --2- 172.21.15.39:0/1717640291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31ac06e190 0x7f31ac09c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:11.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.398+0000 7f31ab400640 1 -- 172.21.15.39:0/1717640291 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f31ac06e5a0 msgr2=0x7f31ac15ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:11.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.398+0000 7f31ab400640 1 --2- 172.21.15.39:0/1717640291 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f31ac06e5a0 0x7f31ac15ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:11.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.398+0000 7f31ab400640 1 -- 172.21.15.39:0/1717640291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31ac06e190 msgr2=0x7f31ac09c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:11.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.398+0000 7f31ab400640 1 --2- 172.21.15.39:0/1717640291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31ac06e190 0x7f31ac09c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:11.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.398+0000 7f31ab400640 1 -- 172.21.15.39:0/1717640291 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31ac09cb80 con 0x7f31ac09ba30 2024-09-21T07:44:11.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.398+0000 7f31a3e00640 1 --2- 172.21.15.39:0/1717640291 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f31ac06e5a0 0x7f31ac15ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:44:11.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.399+0000 7f31ab400640 1 --2- 172.21.15.39:0/1717640291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31ac09ba30 0x7f31ac09da50 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f31a4006010 tx=0x7f31a4003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:11.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.399+0000 7f31a3400640 1 -- 172.21.15.39:0/1717640291 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a4046070 con 0x7f31ac09ba30 2024-09-21T07:44:11.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.399+0000 7f31a3400640 1 -- 172.21.15.39:0/1717640291 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f31a4004540 con 0x7f31ac09ba30 2024-09-21T07:44:11.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.399+0000 7f31a3400640 1 -- 172.21.15.39:0/1717640291 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a4033070 con 0x7f31ac09ba30 2024-09-21T07:44:11.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.399+0000 7f31abe00640 1 -- 172.21.15.39:0/1717640291 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31ac09ce60 con 0x7f31ac09ba30 2024-09-21T07:44:11.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.399+0000 7f31abe00640 1 -- 172.21.15.39:0/1717640291 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31ac0c9470 con 0x7f31ac09ba30 2024-09-21T07:44:11.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.400+0000 7f31abe00640 1 -- 172.21.15.39:0/1717640291 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31ac0b3c20 con 0x7f31ac09ba30 2024-09-21T07:44:11.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.401+0000 7f31a3400640 1 -- 172.21.15.39:0/1717640291 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f31a4049070 con 0x7f31ac09ba30 2024-09-21T07:44:11.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.402+0000 7f31a3400640 1 --2- 172.21.15.39:0/1717640291 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3188078b90 0x7f318807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:11.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.402+0000 7f31a3400640 1 -- 172.21.15.39:0/1717640291 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8291+0+0 (secure 0 0 0) 0x7f31a40bc940 con 0x7f31ac09ba30 2024-09-21T07:44:11.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.403+0000 7f31aaa00640 1 --2- 172.21.15.39:0/1717640291 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3188078b90 0x7f318807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:11.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.404+0000 7f31aaa00640 1 --2- 172.21.15.39:0/1717640291 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3188078b90 0x7f318807b0a0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f319c00b7f0 tx=0x7f319c008040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:11.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.407+0000 7f31a3400640 1 -- 172.21.15.39:0/1717640291 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f31ac0b3c20 con 0x7f31ac09ba30 2024-09-21T07:44:11.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.641+0000 7f31abe00640 1 -- 172.21.15.39:0/1717640291 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31ac06ec60 con 0x7f31ac09ba30 2024-09-21T07:44:11.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.642+0000 7f31a3400640 1 -- 172.21.15.39:0/1717640291 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 157 v157) ==== 95+0+5257 (secure 0 0 0) 0x7f31a403e070 con 0x7f31ac09ba30 2024-09-21T07:44:11.644 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:11.644 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":157,"btime":"2024-09-21T07:44:07:639547+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6929,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2582569093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2582569093},{"type":"v1","addr":"172.21.15.148:6837","nonce":2582569093}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":156}],"filesystems":[{"mdsmap":{"epoch":157,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:44:06.641973+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":8052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8052":{"gid":8052,"name":"a","rank":1,"incarnation":155,"state":"up:reconnect","state_seq":12,"addr":"172.21.15.39:6835/1551311805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1551311805},{"type":"v1","addr":"172.21.15.39:6835","nonce":1551311805}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311]},"id":6}]} 2024-09-21T07:44:11.644 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 157 2024-09-21T07:44:11.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.644+0000 7f31abe00640 1 -- 172.21.15.39:0/1717640291 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3188078b90 msgr2=0x7f318807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:11.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.644+0000 7f31abe00640 1 --2- 172.21.15.39:0/1717640291 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3188078b90 0x7f318807b0a0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f319c00b7f0 tx=0x7f319c008040 comp rx=0 tx=0).stop 2024-09-21T07:44:11.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.644+0000 7f31abe00640 1 -- 172.21.15.39:0/1717640291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31ac09ba30 msgr2=0x7f31ac09da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:11.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.644+0000 7f31abe00640 1 --2- 172.21.15.39:0/1717640291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31ac09ba30 0x7f31ac09da50 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f31a4006010 tx=0x7f31a4003f10 comp rx=0 tx=0).stop 2024-09-21T07:44:11.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.644+0000 7f31abe00640 1 -- 172.21.15.39:0/1717640291 shutdown_connections 2024-09-21T07:44:11.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.644+0000 7f31abe00640 1 --2- 172.21.15.39:0/1717640291 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3188078b90 0x7f318807b0a0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:11.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.644+0000 7f31abe00640 1 --2- 172.21.15.39:0/1717640291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31ac09ba30 0x7f31ac09da50 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:11.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.644+0000 7f31abe00640 1 --2- 172.21.15.39:0/1717640291 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f31ac06e5a0 0x7f31ac15ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:11.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.644+0000 7f31abe00640 1 --2- 172.21.15.39:0/1717640291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31ac06e190 0x7f31ac09c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:11.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.644+0000 7f31abe00640 1 -- 172.21.15.39:0/1717640291 >> 172.21.15.39:0/1717640291 conn(0x7f31ac0cee40 msgr2=0x7f31ac15e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:11.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.644+0000 7f31abe00640 1 -- 172.21.15.39:0/1717640291 shutdown_connections 2024-09-21T07:44:11.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:11.644+0000 7f31abe00640 1 -- 172.21.15.39:0/1717640291 wait complete. 2024-09-21T07:44:11.671 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-21T07:44:11.671 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-09-21T07:44:11.908 INFO:teuthology.orchestra.run.smithi039.stdout:{"mds_beacon_grace":"15.000000"} 2024-09-21T07:44:11.909 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-09-21T07:44:11.909 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:44:12.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.134+0000 7f4009a00640 1 -- 172.21.15.39:0/2828222430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f400406f800 msgr2=0x7f40040cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:12.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.134+0000 7f4009a00640 1 --2- 172.21.15.39:0/2828222430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f400406f800 0x7f40040cd8e0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f40040d1240 tx=0x7f3fec02a4d0 comp rx=0 tx=0).stop 2024-09-21T07:44:12.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.134+0000 7f4009a00640 1 -- 172.21.15.39:0/2828222430 shutdown_connections 2024-09-21T07:44:12.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.134+0000 7f4009a00640 1 --2- 172.21.15.39:0/2828222430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f400406f800 0x7f40040cd8e0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:12.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.134+0000 7f4009a00640 1 --2- 172.21.15.39:0/2828222430 >> v2:172.21.15.148:3300/0 conn(0x7f400415cd30 0x7f40040cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:12.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.134+0000 7f4009a00640 1 -- 172.21.15.39:0/2828222430 >> 172.21.15.39:0/2828222430 conn(0x7f400406cfc0 msgr2=0x7f400406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:12.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.134+0000 7f4009a00640 1 -- 172.21.15.39:0/2828222430 shutdown_connections 2024-09-21T07:44:12.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.134+0000 7f4009a00640 1 -- 172.21.15.39:0/2828222430 wait complete. 2024-09-21T07:44:12.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.137+0000 7f4009a00640 1 Processor -- start 2024-09-21T07:44:12.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.137+0000 7f4009a00640 1 -- start start 2024-09-21T07:44:12.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.138+0000 7f4009a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f400406f800 0x7f400414b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:12.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.138+0000 7f4009a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f400406fc10 0x7f40041504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:12.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.138+0000 7f4009a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f400415cd30 0x7f40040b4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:12.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.138+0000 7f4009a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4004073890 con 0x7f400406f800 2024-09-21T07:44:12.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.138+0000 7f4009a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4004073710 con 0x7f400415cd30 2024-09-21T07:44:12.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.138+0000 7f4009a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f4004073a10 con 0x7f400406fc10 2024-09-21T07:44:12.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.139+0000 7f4002a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f400406f800 0x7f400414b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:12.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.139+0000 7f4002a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f400406f800 0x7f400414b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59622/0 (socket says 172.21.15.39:59622) 2024-09-21T07:44:12.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.139+0000 7f4002a00640 1 -- 172.21.15.39:0/631266453 learned_addr learned my addr 172.21.15.39:0/631266453 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:12.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.139+0000 7f4002a00640 1 -- 172.21.15.39:0/631266453 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f400406fc10 msgr2=0x7f40041504f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:12.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.139+0000 7f4002a00640 1 --2- 172.21.15.39:0/631266453 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f400406fc10 0x7f40041504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:12.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.139+0000 7f4002a00640 1 -- 172.21.15.39:0/631266453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f400415cd30 msgr2=0x7f40040b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:12.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.139+0000 7f4002a00640 1 --2- 172.21.15.39:0/631266453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f400415cd30 0x7f40040b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:12.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.139+0000 7f4002a00640 1 -- 172.21.15.39:0/631266453 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4004154a20 con 0x7f400406f800 2024-09-21T07:44:12.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.139+0000 7f4002a00640 1 --2- 172.21.15.39:0/631266453 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f400406f800 0x7f400414b940 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400d6d0 tx=0x7f3ff400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:12.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.140+0000 7f3ffaa00640 1 -- 172.21.15.39:0/631266453 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4017070 con 0x7f400406f800 2024-09-21T07:44:12.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.140+0000 7f3ffaa00640 1 -- 172.21.15.39:0/631266453 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ff4005330 con 0x7f400406f800 2024-09-21T07:44:12.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.140+0000 7f3ffaa00640 1 -- 172.21.15.39:0/631266453 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4009820 con 0x7f400406f800 2024-09-21T07:44:12.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.140+0000 7f4009a00640 1 -- 172.21.15.39:0/631266453 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4004154d00 con 0x7f400406f800 2024-09-21T07:44:12.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.140+0000 7f4009a00640 1 -- 172.21.15.39:0/631266453 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40041552c0 con 0x7f400406f800 2024-09-21T07:44:12.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.141+0000 7f4009a00640 1 -- 172.21.15.39:0/631266453 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4004073ee0 con 0x7f400406f800 2024-09-21T07:44:12.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.142+0000 7f3ffaa00640 1 -- 172.21.15.39:0/631266453 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3ff40099c0 con 0x7f400406f800 2024-09-21T07:44:12.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.144+0000 7f3ffaa00640 1 --2- 172.21.15.39:0/631266453 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3fe0078b90 0x7f3fe007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:12.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.144+0000 7f3ffaa00640 1 -- 172.21.15.39:0/631266453 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8291+0+0 (secure 0 0 0) 0x7f3ff409b920 con 0x7f400406f800 2024-09-21T07:44:12.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.144+0000 7f3ffbe00640 1 --2- 172.21.15.39:0/631266453 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3fe0078b90 0x7f3fe007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:12.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.145+0000 7f3ffbe00640 1 --2- 172.21.15.39:0/631266453 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3fe0078b90 0x7f3fe007b0a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f3fec033000 tx=0x7f3fec0046c0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:12.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.149+0000 7f3ffaa00640 1 -- 172.21.15.39:0/631266453 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3ff4063290 con 0x7f400406f800 2024-09-21T07:44:12.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.384+0000 7f4009a00640 1 -- 172.21.15.39:0/631266453 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4004159880 con 0x7f400406f800 2024-09-21T07:44:12.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.385+0000 7f3ffaa00640 1 -- 172.21.15.39:0/631266453 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 157 v157) ==== 95+0+5257 (secure 0 0 0) 0x7f3ff4062b90 con 0x7f400406f800 2024-09-21T07:44:12.387 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:12.387 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":157,"btime":"2024-09-21T07:44:07:639547+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6929,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2582569093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2582569093},{"type":"v1","addr":"172.21.15.148:6837","nonce":2582569093}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":156}],"filesystems":[{"mdsmap":{"epoch":157,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:44:06.641973+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":8052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8052":{"gid":8052,"name":"a","rank":1,"incarnation":155,"state":"up:reconnect","state_seq":12,"addr":"172.21.15.39:6835/1551311805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1551311805},{"type":"v1","addr":"172.21.15.39:6835","nonce":1551311805}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311]},"id":6}]} 2024-09-21T07:44:12.387 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 157 2024-09-21T07:44:12.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.387+0000 7f4009a00640 1 -- 172.21.15.39:0/631266453 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3fe0078b90 msgr2=0x7f3fe007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:12.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.387+0000 7f4009a00640 1 --2- 172.21.15.39:0/631266453 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3fe0078b90 0x7f3fe007b0a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f3fec033000 tx=0x7f3fec0046c0 comp rx=0 tx=0).stop 2024-09-21T07:44:12.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.387+0000 7f4009a00640 1 -- 172.21.15.39:0/631266453 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f400406f800 msgr2=0x7f400414b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:12.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.387+0000 7f4009a00640 1 --2- 172.21.15.39:0/631266453 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f400406f800 0x7f400414b940 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400d6d0 tx=0x7f3ff400db90 comp rx=0 tx=0).stop 2024-09-21T07:44:12.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.387+0000 7f4009a00640 1 -- 172.21.15.39:0/631266453 shutdown_connections 2024-09-21T07:44:12.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.387+0000 7f4009a00640 1 --2- 172.21.15.39:0/631266453 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3fe0078b90 0x7f3fe007b0a0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:12.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.387+0000 7f4009a00640 1 --2- 172.21.15.39:0/631266453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f400415cd30 0x7f40040b4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:12.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.387+0000 7f4009a00640 1 --2- 172.21.15.39:0/631266453 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f400406fc10 0x7f40041504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:12.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.387+0000 7f4009a00640 1 --2- 172.21.15.39:0/631266453 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f400406f800 0x7f400414b940 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:12.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.387+0000 7f4009a00640 1 -- 172.21.15.39:0/631266453 >> 172.21.15.39:0/631266453 conn(0x7f400406cfc0 msgr2=0x7f400415a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:12.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.387+0000 7f4009a00640 1 -- 172.21.15.39:0/631266453 shutdown_connections 2024-09-21T07:44:12.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:12.387+0000 7f4009a00640 1 -- 172.21.15.39:0/631266453 wait complete. 2024-09-21T07:44:12.411 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:44:15.998 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~7s 2024-09-21T07:44:17.412 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:44:17.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.653+0000 7fb62dc00640 1 -- 172.21.15.39:0/797487362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6280cb960 msgr2=0x7fb6280cbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:17.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.653+0000 7fb62dc00640 1 --2- 172.21.15.39:0/797487362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6280cb960 0x7fb6280cbd70 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb614005fd0 tx=0x7fb61402a5d0 comp rx=0 tx=0).stop 2024-09-21T07:44:17.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.653+0000 7fb62dc00640 1 -- 172.21.15.39:0/797487362 shutdown_connections 2024-09-21T07:44:17.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.653+0000 7fb62dc00640 1 --2- 172.21.15.39:0/797487362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6280cb960 0x7fb6280cbd70 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:17.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.653+0000 7fb62dc00640 1 --2- 172.21.15.39:0/797487362 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb6280d0f20 0x7fb62815c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:17.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.653+0000 7fb62dc00640 1 -- 172.21.15.39:0/797487362 >> 172.21.15.39:0/797487362 conn(0x7fb6280b2510 msgr2=0x7fb6280d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:17.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.653+0000 7fb62dc00640 1 -- 172.21.15.39:0/797487362 shutdown_connections 2024-09-21T07:44:17.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.653+0000 7fb62dc00640 1 -- 172.21.15.39:0/797487362 wait complete. 2024-09-21T07:44:17.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.656+0000 7fb62dc00640 1 Processor -- start 2024-09-21T07:44:17.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.656+0000 7fb62dc00640 1 -- start start 2024-09-21T07:44:17.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.657+0000 7fb62dc00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb62806f8a0 0x7fb6280b13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:17.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.657+0000 7fb62dc00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6280cb960 0x7fb6281f3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:17.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.657+0000 7fb62dc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6280d0f20 0x7fb6281f3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:17.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.657+0000 7fb62dc00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb6280b4570 con 0x7fb6280d0f20 2024-09-21T07:44:17.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.657+0000 7fb62dc00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb6280b43f0 con 0x7fb6280cb960 2024-09-21T07:44:17.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.657+0000 7fb62dc00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb6280b46f0 con 0x7fb62806f8a0 2024-09-21T07:44:17.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.657+0000 7fb62d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6280d0f20 0x7fb6281f3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:17.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.658+0000 7fb627e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb62806f8a0 0x7fb6280b13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:17.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.658+0000 7fb62d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6280d0f20 0x7fb6281f3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59626/0 (socket says 172.21.15.39:59626) 2024-09-21T07:44:17.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.658+0000 7fb62d200640 1 -- 172.21.15.39:0/762081340 learned_addr learned my addr 172.21.15.39:0/762081340 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:17.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.658+0000 7fb627400640 1 --2- 172.21.15.39:0/762081340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6280cb960 0x7fb6281f3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:17.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.658+0000 7fb62d200640 1 -- 172.21.15.39:0/762081340 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb62806f8a0 msgr2=0x7fb6280b13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:17.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.658+0000 7fb62d200640 1 --2- 172.21.15.39:0/762081340 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb62806f8a0 0x7fb6280b13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:17.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.658+0000 7fb62d200640 1 -- 172.21.15.39:0/762081340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6280cb960 msgr2=0x7fb6281f3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:17.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.658+0000 7fb62d200640 1 --2- 172.21.15.39:0/762081340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6280cb960 0x7fb6281f3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:17.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.658+0000 7fb62d200640 1 -- 172.21.15.39:0/762081340 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb62814e690 con 0x7fb6280d0f20 2024-09-21T07:44:17.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.658+0000 7fb62d200640 1 --2- 172.21.15.39:0/762081340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6280d0f20 0x7fb6281f3ff0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fb62000c730 tx=0x7fb62000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:17.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.659+0000 7fb61fe00640 1 -- 172.21.15.39:0/762081340 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb620003b70 con 0x7fb6280d0f20 2024-09-21T07:44:17.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.659+0000 7fb61fe00640 1 -- 172.21.15.39:0/762081340 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb620003d30 con 0x7fb6280d0f20 2024-09-21T07:44:17.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.659+0000 7fb61fe00640 1 -- 172.21.15.39:0/762081340 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6200052d0 con 0x7fb6280d0f20 2024-09-21T07:44:17.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.659+0000 7fb62dc00640 1 -- 172.21.15.39:0/762081340 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6281f3860 con 0x7fb6280d0f20 2024-09-21T07:44:17.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.659+0000 7fb62dc00640 1 -- 172.21.15.39:0/762081340 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6280ad430 con 0x7fb6280d0f20 2024-09-21T07:44:17.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.660+0000 7fb62dc00640 1 -- 172.21.15.39:0/762081340 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6280b4c50 con 0x7fb6280d0f20 2024-09-21T07:44:17.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.661+0000 7fb61fe00640 1 -- 172.21.15.39:0/762081340 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb620005470 con 0x7fb6280d0f20 2024-09-21T07:44:17.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.662+0000 7fb61fe00640 1 --2- 172.21.15.39:0/762081340 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb608078bc0 0x7fb60807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:17.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.662+0000 7fb61fe00640 1 -- 172.21.15.39:0/762081340 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8334+0+0 (secure 0 0 0) 0x7fb62009b310 con 0x7fb6280d0f20 2024-09-21T07:44:17.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.662+0000 7fb627e00640 1 --2- 172.21.15.39:0/762081340 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb608078bc0 0x7fb60807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:17.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.663+0000 7fb627e00640 1 --2- 172.21.15.39:0/762081340 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb608078bc0 0x7fb60807b0d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb6180095f0 tx=0x7fb618005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:17.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.667+0000 7fb61fe00640 1 -- 172.21.15.39:0/762081340 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb620062730 con 0x7fb6280d0f20 2024-09-21T07:44:17.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.904+0000 7fb62dc00640 1 -- 172.21.15.39:0/762081340 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6280cbea0 con 0x7fb6280d0f20 2024-09-21T07:44:17.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.904+0000 7fb61fe00640 1 -- 172.21.15.39:0/762081340 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 157 v157) ==== 95+0+5257 (secure 0 0 0) 0x7fb620067160 con 0x7fb6280d0f20 2024-09-21T07:44:17.906 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:17.906 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":157,"btime":"2024-09-21T07:44:07:639547+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6929,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2582569093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2582569093},{"type":"v1","addr":"172.21.15.148:6837","nonce":2582569093}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":156}],"filesystems":[{"mdsmap":{"epoch":157,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:44:06.641973+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":8052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8052":{"gid":8052,"name":"a","rank":1,"incarnation":155,"state":"up:reconnect","state_seq":12,"addr":"172.21.15.39:6835/1551311805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1551311805},{"type":"v1","addr":"172.21.15.39:6835","nonce":1551311805}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311]},"id":6}]} 2024-09-21T07:44:17.907 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 157 2024-09-21T07:44:17.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.907+0000 7fb62dc00640 1 -- 172.21.15.39:0/762081340 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb608078bc0 msgr2=0x7fb60807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:17.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.907+0000 7fb62dc00640 1 --2- 172.21.15.39:0/762081340 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb608078bc0 0x7fb60807b0d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb6180095f0 tx=0x7fb618005f90 comp rx=0 tx=0).stop 2024-09-21T07:44:17.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.907+0000 7fb62dc00640 1 -- 172.21.15.39:0/762081340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6280d0f20 msgr2=0x7fb6281f3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:17.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.907+0000 7fb62dc00640 1 --2- 172.21.15.39:0/762081340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6280d0f20 0x7fb6281f3ff0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fb62000c730 tx=0x7fb62000cbf0 comp rx=0 tx=0).stop 2024-09-21T07:44:17.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.907+0000 7fb62dc00640 1 -- 172.21.15.39:0/762081340 shutdown_connections 2024-09-21T07:44:17.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.907+0000 7fb62dc00640 1 --2- 172.21.15.39:0/762081340 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb608078bc0 0x7fb60807b0d0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:17.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.907+0000 7fb62dc00640 1 --2- 172.21.15.39:0/762081340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6280d0f20 0x7fb6281f3ff0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:17.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.907+0000 7fb62dc00640 1 --2- 172.21.15.39:0/762081340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb6280cb960 0x7fb6281f3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:17.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.907+0000 7fb62dc00640 1 --2- 172.21.15.39:0/762081340 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb62806f8a0 0x7fb6280b13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:17.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.907+0000 7fb62dc00640 1 -- 172.21.15.39:0/762081340 >> 172.21.15.39:0/762081340 conn(0x7fb6280b2510 msgr2=0x7fb62815e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:17.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.907+0000 7fb62dc00640 1 -- 172.21.15.39:0/762081340 shutdown_connections 2024-09-21T07:44:17.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:17.907+0000 7fb62dc00640 1 -- 172.21.15.39:0/762081340 wait complete. 2024-09-21T07:44:17.935 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:44:22.609 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~13s 2024-09-21T07:44:22.935 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:44:23.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.172+0000 7f9bbca00640 1 -- 172.21.15.39:0/971741750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bb806fde0 msgr2=0x7f9bb8072fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:23.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.172+0000 7f9bbca00640 1 --2- 172.21.15.39:0/971741750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bb806fde0 0x7f9bb8072fa0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0005fd0 tx=0x7f9bb002a8a0 comp rx=0 tx=0).stop 2024-09-21T07:44:23.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.172+0000 7f9bbca00640 1 -- 172.21.15.39:0/971741750 shutdown_connections 2024-09-21T07:44:23.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.172+0000 7f9bbca00640 1 --2- 172.21.15.39:0/971741750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bb806fde0 0x7f9bb8072fa0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:23.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.172+0000 7f9bbca00640 1 --2- 172.21.15.39:0/971741750 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9bb815c830 0x7f9bb806f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:23.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.172+0000 7f9bbca00640 1 -- 172.21.15.39:0/971741750 >> 172.21.15.39:0/971741750 conn(0x7f9bb80b2510 msgr2=0x7f9bb80d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:23.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.172+0000 7f9bbca00640 1 -- 172.21.15.39:0/971741750 shutdown_connections 2024-09-21T07:44:23.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.172+0000 7f9bbca00640 1 -- 172.21.15.39:0/971741750 wait complete. 2024-09-21T07:44:23.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.175+0000 7f9bbca00640 1 Processor -- start 2024-09-21T07:44:23.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.175+0000 7f9bbca00640 1 -- start start 2024-09-21T07:44:23.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.176+0000 7f9bbca00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bb806fde0 0x7f9bb80aeec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:23.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.176+0000 7f9bbca00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9bb80d0f20 0x7f9bb80c9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:23.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.176+0000 7f9bbca00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9bb815c830 0x7f9bb814bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:23.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.176+0000 7f9bbca00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb80b4570 con 0x7f9bb815c830 2024-09-21T07:44:23.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.176+0000 7f9bbca00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb80b43f0 con 0x7f9bb806fde0 2024-09-21T07:44:23.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.176+0000 7f9bbca00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f9bb80b46f0 con 0x7f9bb80d0f20 2024-09-21T07:44:23.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.176+0000 7f9bb7200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9bb815c830 0x7f9bb814bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:23.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.176+0000 7f9bb7200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9bb815c830 0x7f9bb814bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35184/0 (socket says 172.21.15.39:35184) 2024-09-21T07:44:23.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.176+0000 7f9bb7200640 1 -- 172.21.15.39:0/1924566543 learned_addr learned my addr 172.21.15.39:0/1924566543 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:23.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.177+0000 7f9bade00640 1 --2- 172.21.15.39:0/1924566543 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9bb80d0f20 0x7f9bb80c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:23.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.177+0000 7f9bb6800640 1 --2- 172.21.15.39:0/1924566543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bb806fde0 0x7f9bb80aeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:23.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.177+0000 7f9bb7200640 1 -- 172.21.15.39:0/1924566543 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9bb80d0f20 msgr2=0x7f9bb80c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:23.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.177+0000 7f9bb7200640 1 --2- 172.21.15.39:0/1924566543 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9bb80d0f20 0x7f9bb80c9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:23.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.177+0000 7f9bb7200640 1 -- 172.21.15.39:0/1924566543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bb806fde0 msgr2=0x7f9bb80aeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:23.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.177+0000 7f9bb7200640 1 --2- 172.21.15.39:0/1924566543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bb806fde0 0x7f9bb80aeec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:23.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.177+0000 7f9bb7200640 1 -- 172.21.15.39:0/1924566543 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bb80ade20 con 0x7f9bb815c830 2024-09-21T07:44:23.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.177+0000 7f9bb7200640 1 --2- 172.21.15.39:0/1924566543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9bb815c830 0x7f9bb814bd60 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f9bb002ae90 tx=0x7f9bb0032d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:23.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.178+0000 7f9baf400640 1 -- 172.21.15.39:0/1924566543 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb0041070 con 0x7f9bb815c830 2024-09-21T07:44:23.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.178+0000 7f9bbca00640 1 -- 172.21.15.39:0/1924566543 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bb80ae100 con 0x7f9bb815c830 2024-09-21T07:44:23.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.178+0000 7f9baf400640 1 -- 172.21.15.39:0/1924566543 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9bb0004450 con 0x7f9bb815c830 2024-09-21T07:44:23.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.178+0000 7f9baf400640 1 -- 172.21.15.39:0/1924566543 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb0031430 con 0x7f9bb815c830 2024-09-21T07:44:23.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.178+0000 7f9bbca00640 1 -- 172.21.15.39:0/1924566543 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bb80ae6c0 con 0x7f9bb815c830 2024-09-21T07:44:23.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.179+0000 7f9bbca00640 1 -- 172.21.15.39:0/1924566543 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bb80adb70 con 0x7f9bb815c830 2024-09-21T07:44:23.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.180+0000 7f9baf400640 1 -- 172.21.15.39:0/1924566543 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9bb0044070 con 0x7f9bb815c830 2024-09-21T07:44:23.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.181+0000 7f9baf400640 1 --2- 172.21.15.39:0/1924566543 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9b94078bc0 0x7f9b9407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:23.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.181+0000 7f9baf400640 1 -- 172.21.15.39:0/1924566543 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8334+0+0 (secure 0 0 0) 0x7f9bb002e020 con 0x7f9bb815c830 2024-09-21T07:44:23.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.182+0000 7f9bb6800640 1 --2- 172.21.15.39:0/1924566543 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9b94078bc0 0x7f9b9407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:23.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.182+0000 7f9bb6800640 1 --2- 172.21.15.39:0/1924566543 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9b94078bc0 0x7f9b9407b0d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8001720 tx=0x7f9ba8009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:23.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.186+0000 7f9baf400640 1 -- 172.21.15.39:0/1924566543 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9bb004b030 con 0x7f9bb815c830 2024-09-21T07:44:23.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.421+0000 7f9bbca00640 1 -- 172.21.15.39:0/1924566543 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9bb80b4c10 con 0x7f9bb815c830 2024-09-21T07:44:23.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.422+0000 7f9baf400640 1 -- 172.21.15.39:0/1924566543 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 158 v158) ==== 95+0+5305 (secure 0 0 0) 0x7f9bb00839c0 con 0x7f9bb815c830 2024-09-21T07:44:23.423 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:23.423 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":158,"btime":"2024-09-21T07:44:21:470412+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6929,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2582569093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2582569093},{"type":"v1","addr":"172.21.15.148:6837","nonce":2582569093}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":156}],"filesystems":[{"mdsmap":{"epoch":158,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:44:21.470358+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7311,"mds_1":8052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"c","rank":0,"incarnation":119,"state":"up:active","state_seq":20,"addr":"172.21.15.39:6833/1790581246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1790581246},{"type":"v1","addr":"172.21.15.39:6833","nonce":1790581246}]},"join_fscid":-1,"laggy_since":"2024-09-21T07:44:21.470358+0000","export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8052":{"gid":8052,"name":"a","rank":1,"incarnation":155,"state":"up:reconnect","state_seq":12,"addr":"172.21.15.39:6835/1551311805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1551311805},{"type":"v1","addr":"172.21.15.39:6835","nonce":1551311805}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311]},"id":6}]} 2024-09-21T07:44:23.423 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 158 2024-09-21T07:44:23.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.424+0000 7f9bbca00640 1 -- 172.21.15.39:0/1924566543 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9b94078bc0 msgr2=0x7f9b9407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:23.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.424+0000 7f9bbca00640 1 --2- 172.21.15.39:0/1924566543 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9b94078bc0 0x7f9b9407b0d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8001720 tx=0x7f9ba8009040 comp rx=0 tx=0).stop 2024-09-21T07:44:23.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.424+0000 7f9bbca00640 1 -- 172.21.15.39:0/1924566543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9bb815c830 msgr2=0x7f9bb814bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:23.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.424+0000 7f9bbca00640 1 --2- 172.21.15.39:0/1924566543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9bb815c830 0x7f9bb814bd60 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f9bb002ae90 tx=0x7f9bb0032d30 comp rx=0 tx=0).stop 2024-09-21T07:44:23.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.424+0000 7f9bbca00640 1 -- 172.21.15.39:0/1924566543 shutdown_connections 2024-09-21T07:44:23.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.424+0000 7f9bbca00640 1 --2- 172.21.15.39:0/1924566543 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9b94078bc0 0x7f9b9407b0d0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:23.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.424+0000 7f9bbca00640 1 --2- 172.21.15.39:0/1924566543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9bb815c830 0x7f9bb814bd60 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:23.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.424+0000 7f9bbca00640 1 --2- 172.21.15.39:0/1924566543 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9bb80d0f20 0x7f9bb80c9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:23.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.424+0000 7f9bbca00640 1 --2- 172.21.15.39:0/1924566543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bb806fde0 0x7f9bb80aeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:23.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.424+0000 7f9bbca00640 1 -- 172.21.15.39:0/1924566543 >> 172.21.15.39:0/1924566543 conn(0x7f9bb80b2510 msgr2=0x7f9bb80b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:23.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.424+0000 7f9bbca00640 1 -- 172.21.15.39:0/1924566543 shutdown_connections 2024-09-21T07:44:23.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.424+0000 7f9bbca00640 1 -- 172.21.15.39:0/1924566543 wait complete. 2024-09-21T07:44:23.450 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2024-09-21T07:44:23.451 DEBUG:teuthology.orchestra.run.smithi039:> sudo sysctl -n kernel.core_pattern 2024-09-21T07:44:23.503 INFO:teuthology.orchestra.run.smithi039.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T07:44:23.506 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2024-09-21T07:44:23.506 DEBUG:teuthology.orchestra.run.smithi039:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2024-09-21T07:44:23.569 INFO:teuthology.orchestra.run.smithi039.stdout:1726904646.63781.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i c', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2024-09-21T07:44:23.572 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1726904646.63781.core'] 2024-09-21T07:44:23.572 INFO:tasks.cephfs.cephfs_test_case:Found core file 1726904646.63781.core, deleting it 2024-09-21T07:44:23.573 DEBUG:teuthology.orchestra.run.smithi039:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1726904646.63781.core 2024-09-21T07:44:23.670 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 6:0 2024-09-21T07:44:23.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.898+0000 7f3e1d000640 1 -- 172.21.15.39:0/3238402599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e180b4c20 msgr2=0x7f3e1806f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:23.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.898+0000 7f3e1d000640 1 --2- 172.21.15.39:0/3238402599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e180b4c20 0x7f3e1806f5c0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f3e10005fd0 tx=0x7f3e1002aa90 comp rx=0 tx=0).stop 2024-09-21T07:44:23.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.899+0000 7f3e1d000640 1 -- 172.21.15.39:0/3238402599 shutdown_connections 2024-09-21T07:44:23.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.899+0000 7f3e1d000640 1 --2- 172.21.15.39:0/3238402599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e180b4c20 0x7f3e1806f5c0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:23.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.899+0000 7f3e1d000640 1 --2- 172.21.15.39:0/3238402599 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3e180cc560 0x7f3e180ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:23.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.899+0000 7f3e1d000640 1 -- 172.21.15.39:0/3238402599 >> 172.21.15.39:0/3238402599 conn(0x7f3e180b2510 msgr2=0x7f3e180d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:23.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.899+0000 7f3e1d000640 1 -- 172.21.15.39:0/3238402599 shutdown_connections 2024-09-21T07:44:23.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.899+0000 7f3e1d000640 1 -- 172.21.15.39:0/3238402599 wait complete. 2024-09-21T07:44:23.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.901+0000 7f3e1d000640 1 Processor -- start 2024-09-21T07:44:23.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.901+0000 7f3e1d000640 1 -- start start 2024-09-21T07:44:23.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.902+0000 7f3e1d000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3e180b30d0 0x7f3e181f5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:23.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.902+0000 7f3e1d000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e180cc560 0x7f3e18155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:23.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.902+0000 7f3e1d000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e1815a700 0x7f3e181596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:23.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.902+0000 7f3e1d000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1806fe90 con 0x7f3e180cc560 2024-09-21T07:44:23.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.902+0000 7f3e1d000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1806fd10 con 0x7f3e1815a700 2024-09-21T07:44:23.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.902+0000 7f3e1d000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f3e18070010 con 0x7f3e180b30d0 2024-09-21T07:44:23.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.903+0000 7f3e0ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e180cc560 0x7f3e18155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:23.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.903+0000 7f3e0ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e180cc560 0x7f3e18155190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35204/0 (socket says 172.21.15.39:35204) 2024-09-21T07:44:23.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.903+0000 7f3e0ea00640 1 -- 172.21.15.39:0/3561762194 learned_addr learned my addr 172.21.15.39:0/3561762194 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:23.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.903+0000 7f3e17400640 1 --2- 172.21.15.39:0/3561762194 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3e180b30d0 0x7f3e181f5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:23.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.903+0000 7f3e17e00640 1 --2- 172.21.15.39:0/3561762194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e1815a700 0x7f3e181596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:23.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.903+0000 7f3e0ea00640 1 -- 172.21.15.39:0/3561762194 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3e180b30d0 msgr2=0x7f3e181f5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:23.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.903+0000 7f3e0ea00640 1 --2- 172.21.15.39:0/3561762194 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3e180b30d0 0x7f3e181f5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:23.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.903+0000 7f3e0ea00640 1 -- 172.21.15.39:0/3561762194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e1815a700 msgr2=0x7f3e181596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:23.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.903+0000 7f3e0ea00640 1 --2- 172.21.15.39:0/3561762194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e1815a700 0x7f3e181596e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:23.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.903+0000 7f3e0ea00640 1 -- 172.21.15.39:0/3561762194 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e18156f00 con 0x7f3e180cc560 2024-09-21T07:44:23.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.904+0000 7f3e0ea00640 1 --2- 172.21.15.39:0/3561762194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e180cc560 0x7f3e18155190 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f3e0000c390 tx=0x7f3e0000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:23.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.904+0000 7f3e0fe00640 1 -- 172.21.15.39:0/3561762194 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e00017070 con 0x7f3e180cc560 2024-09-21T07:44:23.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.904+0000 7f3e0fe00640 1 -- 172.21.15.39:0/3561762194 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e000035f0 con 0x7f3e180cc560 2024-09-21T07:44:23.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.904+0000 7f3e1d000640 1 -- 172.21.15.39:0/3561762194 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e181571e0 con 0x7f3e180cc560 2024-09-21T07:44:23.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.904+0000 7f3e0fe00640 1 -- 172.21.15.39:0/3561762194 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e00003a20 con 0x7f3e180cc560 2024-09-21T07:44:23.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.904+0000 7f3e1d000640 1 -- 172.21.15.39:0/3561762194 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e181591c0 con 0x7f3e180cc560 2024-09-21T07:44:23.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.905+0000 7f3e1d000640 1 -- 172.21.15.39:0/3561762194 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e1806f5c0 con 0x7f3e180cc560 2024-09-21T07:44:23.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.906+0000 7f3e0fe00640 1 -- 172.21.15.39:0/3561762194 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3e0001a070 con 0x7f3e180cc560 2024-09-21T07:44:23.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.907+0000 7f3e0fe00640 1 --2- 172.21.15.39:0/3561762194 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3df4078bc0 0x7f3df407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:23.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.907+0000 7f3e0fe00640 1 -- 172.21.15.39:0/3561762194 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8334+0+0 (secure 0 0 0) 0x7f3e0009ab50 con 0x7f3e180cc560 2024-09-21T07:44:23.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.908+0000 7f3e17400640 1 --2- 172.21.15.39:0/3561762194 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3df4078bc0 0x7f3df407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:23.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.908+0000 7f3e17400640 1 --2- 172.21.15.39:0/3561762194 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3df4078bc0 0x7f3df407b0d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f3e08005fd0 tx=0x7f3e08005850 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:23.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:23.912+0000 7f3e0fe00640 1 -- 172.21.15.39:0/3561762194 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3e000624b0 con 0x7f3e180cc560 2024-09-21T07:44:24.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.184+0000 7f3e1d000640 1 -- 172.21.15.39:0/3561762194 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "6:0"} v 0) -- 0x7f3e180c7090 con 0x7f3e180cc560 2024-09-21T07:44:24.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.187+0000 7f3e0fe00640 1 -- 172.21.15.39:0/3561762194 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "6:0"}]=0 failed mds gid 7311 v159) ==== 97+0+0 (secure 0 0 0) 0x7f3e00061db0 con 0x7f3e180cc560 2024-09-21T07:44:24.188 INFO:teuthology.orchestra.run.smithi039.stderr:failed mds gid 7311 2024-09-21T07:44:24.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.190+0000 7f3e1d000640 1 -- 172.21.15.39:0/3561762194 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3df4078bc0 msgr2=0x7f3df407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:24.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.190+0000 7f3e1d000640 1 --2- 172.21.15.39:0/3561762194 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3df4078bc0 0x7f3df407b0d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f3e08005fd0 tx=0x7f3e08005850 comp rx=0 tx=0).stop 2024-09-21T07:44:24.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.190+0000 7f3e1d000640 1 -- 172.21.15.39:0/3561762194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e180cc560 msgr2=0x7f3e18155190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:24.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.190+0000 7f3e1d000640 1 --2- 172.21.15.39:0/3561762194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e180cc560 0x7f3e18155190 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f3e0000c390 tx=0x7f3e0000c850 comp rx=0 tx=0).stop 2024-09-21T07:44:24.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.190+0000 7f3e1d000640 1 -- 172.21.15.39:0/3561762194 shutdown_connections 2024-09-21T07:44:24.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.190+0000 7f3e1d000640 1 --2- 172.21.15.39:0/3561762194 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3df4078bc0 0x7f3df407b0d0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:24.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.190+0000 7f3e1d000640 1 --2- 172.21.15.39:0/3561762194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e1815a700 0x7f3e181596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:24.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.190+0000 7f3e1d000640 1 --2- 172.21.15.39:0/3561762194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e180cc560 0x7f3e18155190 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:24.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.190+0000 7f3e1d000640 1 --2- 172.21.15.39:0/3561762194 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3e180b30d0 0x7f3e181f5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:24.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.190+0000 7f3e1d000640 1 -- 172.21.15.39:0/3561762194 >> 172.21.15.39:0/3561762194 conn(0x7f3e180b2510 msgr2=0x7f3e180730b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:24.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.190+0000 7f3e1d000640 1 -- 172.21.15.39:0/3561762194 shutdown_connections 2024-09-21T07:44:24.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.190+0000 7f3e1d000640 1 -- 172.21.15.39:0/3561762194 wait complete. 2024-09-21T07:44:24.222 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T07:44:24.222 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T07:44:24.223 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T07:44:24.223 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:44:24.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:44:24.223 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/daemon/state.py", line 146, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi039 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c' 2024-09-21T07:44:24.223 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T07:44:24.223 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T07:44:24.227 INFO:tasks.ceph.mds.c:Started 2024-09-21T07:44:24.227 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:44:24.422 INFO:tasks.ceph.mds.c.smithi039.stdout:starting mds.c at 2024-09-21T07:44:24.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.456+0000 7f1e69600640 1 -- 172.21.15.39:0/4177609488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e640b3370 msgr2=0x7f1e640b3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:24.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.456+0000 7f1e69600640 1 --2- 172.21.15.39:0/4177609488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e640b3370 0x7f1e640b3780 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f1e6415afa0 tx=0x7f1e5002a560 comp rx=0 tx=0).stop 2024-09-21T07:44:24.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.457+0000 7f1e69600640 1 -- 172.21.15.39:0/4177609488 shutdown_connections 2024-09-21T07:44:24.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.457+0000 7f1e69600640 1 --2- 172.21.15.39:0/4177609488 >> v2:172.21.15.148:3300/0 conn(0x7f1e64151590 0x7f1e641519a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:24.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.457+0000 7f1e69600640 1 --2- 172.21.15.39:0/4177609488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e640b3370 0x7f1e640b3780 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:24.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.457+0000 7f1e69600640 1 -- 172.21.15.39:0/4177609488 >> 172.21.15.39:0/4177609488 conn(0x7f1e6406cfc0 msgr2=0x7f1e6406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:24.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.457+0000 7f1e69600640 1 -- 172.21.15.39:0/4177609488 shutdown_connections 2024-09-21T07:44:24.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.457+0000 7f1e69600640 1 -- 172.21.15.39:0/4177609488 wait complete. 2024-09-21T07:44:24.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.459+0000 7f1e69600640 1 Processor -- start 2024-09-21T07:44:24.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.459+0000 7f1e69600640 1 -- start start 2024-09-21T07:44:24.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.460+0000 7f1e69600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e640b3370 0x7f1e64149640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:24.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.461+0000 7f1e69600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1e64151590 0x7f1e640ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:24.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.461+0000 7f1e69600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e6415e590 0x7f1e640addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:24.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.461+0000 7f1e69600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6415b750 con 0x7f1e640b3370 2024-09-21T07:44:24.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.461+0000 7f1e69600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6415b5d0 con 0x7f1e6415e590 2024-09-21T07:44:24.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.461+0000 7f1e69600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f1e6415b8d0 con 0x7f1e64151590 2024-09-21T07:44:24.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.461+0000 7f1e62a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e640b3370 0x7f1e64149640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:24.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.461+0000 7f1e62a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e640b3370 0x7f1e64149640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35230/0 (socket says 172.21.15.39:35230) 2024-09-21T07:44:24.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.461+0000 7f1e62a00640 1 -- 172.21.15.39:0/3975395882 learned_addr learned my addr 172.21.15.39:0/3975395882 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:24.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.461+0000 7f1e63400640 1 --2- 172.21.15.39:0/3975395882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e6415e590 0x7f1e640addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:24.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.461+0000 7f1e62000640 1 --2- 172.21.15.39:0/3975395882 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1e64151590 0x7f1e640ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:24.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.461+0000 7f1e62a00640 1 -- 172.21.15.39:0/3975395882 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1e64151590 msgr2=0x7f1e640ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:24.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.461+0000 7f1e62a00640 1 --2- 172.21.15.39:0/3975395882 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1e64151590 0x7f1e640ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:24.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.461+0000 7f1e62a00640 1 -- 172.21.15.39:0/3975395882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e6415e590 msgr2=0x7f1e640addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:24.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.461+0000 7f1e62a00640 1 --2- 172.21.15.39:0/3975395882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e6415e590 0x7f1e640addd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:24.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.461+0000 7f1e62a00640 1 -- 172.21.15.39:0/3975395882 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e6414f310 con 0x7f1e640b3370 2024-09-21T07:44:24.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.461+0000 7f1e63400640 1 --2- 172.21.15.39:0/3975395882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e6415e590 0x7f1e640addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:44:24.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.461+0000 7f1e62000640 1 --2- 172.21.15.39:0/3975395882 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1e64151590 0x7f1e640ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:44:24.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.462+0000 7f1e62a00640 1 --2- 172.21.15.39:0/3975395882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e640b3370 0x7f1e64149640 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f1e5400c3c0 tx=0x7f1e5400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:24.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.462+0000 7f1e5aa00640 1 -- 172.21.15.39:0/3975395882 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e54017070 con 0x7f1e640b3370 2024-09-21T07:44:24.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.462+0000 7f1e5aa00640 1 -- 172.21.15.39:0/3975395882 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e540035f0 con 0x7f1e640b3370 2024-09-21T07:44:24.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.462+0000 7f1e69600640 1 -- 172.21.15.39:0/3975395882 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e6414f5f0 con 0x7f1e640b3370 2024-09-21T07:44:24.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.462+0000 7f1e5aa00640 1 -- 172.21.15.39:0/3975395882 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e54003a40 con 0x7f1e640b3370 2024-09-21T07:44:24.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.462+0000 7f1e69600640 1 -- 172.21.15.39:0/3975395882 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e6414edc0 con 0x7f1e640b3370 2024-09-21T07:44:24.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.463+0000 7f1e69600640 1 -- 172.21.15.39:0/3975395882 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e640b3780 con 0x7f1e640b3370 2024-09-21T07:44:24.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.464+0000 7f1e5aa00640 1 -- 172.21.15.39:0/3975395882 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1e5401a070 con 0x7f1e640b3370 2024-09-21T07:44:24.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.465+0000 7f1e5aa00640 1 --2- 172.21.15.39:0/3975395882 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1e44078b90 0x7f1e4407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:24.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.465+0000 7f1e5aa00640 1 -- 172.21.15.39:0/3975395882 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8420+0+0 (secure 0 0 0) 0x7f1e5409ac00 con 0x7f1e640b3370 2024-09-21T07:44:24.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.466+0000 7f1e62000640 1 --2- 172.21.15.39:0/3975395882 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1e44078b90 0x7f1e4407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:24.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.467+0000 7f1e62000640 1 --2- 172.21.15.39:0/3975395882 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1e44078b90 0x7f1e4407b0a0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1e50033000 tx=0x7f1e50004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:24.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.470+0000 7f1e5aa00640 1 -- 172.21.15.39:0/3975395882 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1e540624f0 con 0x7f1e640b3370 2024-09-21T07:44:24.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.703+0000 7f1e69600640 1 -- 172.21.15.39:0/3975395882 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e64151370 con 0x7f1e640b3370 2024-09-21T07:44:24.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.704+0000 7f1e5aa00640 1 -- 172.21.15.39:0/3975395882 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 160 v160) ==== 95+0+4423 (secure 0 0 0) 0x7f1e54061df0 con 0x7f1e640b3370 2024-09-21T07:44:24.705 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:24.705 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":160,"btime":"2024-09-21T07:44:24:192530+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":160,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:44:24.192514+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6929,"mds_1":8052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6929":{"gid":6929,"name":"d","rank":0,"incarnation":160,"state":"up:replay","state_seq":1,"addr":"172.21.15.148:6837/2582569093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2582569093},{"type":"v1","addr":"172.21.15.148:6837","nonce":2582569093}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8052":{"gid":8052,"name":"a","rank":1,"incarnation":155,"state":"up:reconnect","state_seq":12,"addr":"172.21.15.39:6835/1551311805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1551311805},{"type":"v1","addr":"172.21.15.39:6835","nonce":1551311805}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T07:44:24.706 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 160 2024-09-21T07:44:24.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.706+0000 7f1e69600640 1 -- 172.21.15.39:0/3975395882 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1e44078b90 msgr2=0x7f1e4407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:24.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.706+0000 7f1e69600640 1 --2- 172.21.15.39:0/3975395882 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1e44078b90 0x7f1e4407b0a0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1e50033000 tx=0x7f1e50004730 comp rx=0 tx=0).stop 2024-09-21T07:44:24.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.706+0000 7f1e69600640 1 -- 172.21.15.39:0/3975395882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e640b3370 msgr2=0x7f1e64149640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:24.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.706+0000 7f1e69600640 1 --2- 172.21.15.39:0/3975395882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e640b3370 0x7f1e64149640 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f1e5400c3c0 tx=0x7f1e5400c880 comp rx=0 tx=0).stop 2024-09-21T07:44:24.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.706+0000 7f1e69600640 1 -- 172.21.15.39:0/3975395882 shutdown_connections 2024-09-21T07:44:24.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.706+0000 7f1e69600640 1 --2- 172.21.15.39:0/3975395882 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1e44078b90 0x7f1e4407b0a0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:24.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.706+0000 7f1e69600640 1 --2- 172.21.15.39:0/3975395882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e6415e590 0x7f1e640addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:24.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.706+0000 7f1e69600640 1 --2- 172.21.15.39:0/3975395882 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1e64151590 0x7f1e640ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:24.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.706+0000 7f1e69600640 1 --2- 172.21.15.39:0/3975395882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e640b3370 0x7f1e64149640 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:24.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.706+0000 7f1e69600640 1 -- 172.21.15.39:0/3975395882 >> 172.21.15.39:0/3975395882 conn(0x7f1e6406cfc0 msgr2=0x7f1e6406f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:24.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.706+0000 7f1e69600640 1 -- 172.21.15.39:0/3975395882 shutdown_connections 2024-09-21T07:44:24.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:24.707+0000 7f1e69600640 1 -- 172.21.15.39:0/3975395882 wait complete. 2024-09-21T07:44:24.732 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-09-21T07:44:29.321 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-09-21T07:44:29.734 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:44:29.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:29.969+0000 7f807ce00640 1 -- 172.21.15.39:0/698550259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f807815cd30 msgr2=0x7f80780cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:29.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:29.969+0000 7f807ce00640 1 --2- 172.21.15.39:0/698550259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f807815cd30 0x7f80780cfd10 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f807806de00 tx=0x7f806802a580 comp rx=0 tx=0).stop 2024-09-21T07:44:29.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:29.969+0000 7f807ce00640 1 -- 172.21.15.39:0/698550259 shutdown_connections 2024-09-21T07:44:29.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:29.969+0000 7f807ce00640 1 --2- 172.21.15.39:0/698550259 >> v2:172.21.15.148:3300/0 conn(0x7f807806f800 0x7f80780cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:29.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:29.969+0000 7f807ce00640 1 --2- 172.21.15.39:0/698550259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f807815cd30 0x7f80780cfd10 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:29.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:29.970+0000 7f807ce00640 1 -- 172.21.15.39:0/698550259 >> 172.21.15.39:0/698550259 conn(0x7f807806cfc0 msgr2=0x7f807806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:29.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:29.970+0000 7f807ce00640 1 -- 172.21.15.39:0/698550259 shutdown_connections 2024-09-21T07:44:29.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:29.970+0000 7f807ce00640 1 -- 172.21.15.39:0/698550259 wait complete. 2024-09-21T07:44:29.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:29.972+0000 7f807ce00640 1 Processor -- start 2024-09-21T07:44:29.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:29.972+0000 7f807ce00640 1 -- start start 2024-09-21T07:44:29.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:29.973+0000 7f807ce00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f807806f800 0x7f80780aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:29.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:29.973+0000 7f807ce00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f807806fc10 0x7f8078155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:29.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:29.973+0000 7f807ce00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f807815cd30 0x7f8078155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:29.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:29.973+0000 7f807ce00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8078073890 con 0x7f807806fc10 2024-09-21T07:44:29.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:29.973+0000 7f807ce00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8078073710 con 0x7f807815cd30 2024-09-21T07:44:29.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:29.973+0000 7f807ce00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f8078073a10 con 0x7f807806f800 2024-09-21T07:44:29.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:29.974+0000 7f806fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f807806fc10 0x7f8078155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:29.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:29.974+0000 7f806fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f807806fc10 0x7f8078155650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35242/0 (socket says 172.21.15.39:35242) 2024-09-21T07:44:29.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:29.974+0000 7f806fe00640 1 -- 172.21.15.39:0/2624748951 learned_addr learned my addr 172.21.15.39:0/2624748951 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:29.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:29.974+0000 7f8076800640 1 --2- 172.21.15.39:0/2624748951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f807815cd30 0x7f8078155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:29.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:29.974+0000 7f8075e00640 1 --2- 172.21.15.39:0/2624748951 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f807806f800 0x7f80780aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:29.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:29.974+0000 7f806fe00640 1 -- 172.21.15.39:0/2624748951 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f807806f800 msgr2=0x7f80780aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:29.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:29.974+0000 7f806fe00640 1 --2- 172.21.15.39:0/2624748951 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f807806f800 0x7f80780aefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:29.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:29.974+0000 7f806fe00640 1 -- 172.21.15.39:0/2624748951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f807815cd30 msgr2=0x7f8078155b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:29.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:29.974+0000 7f806fe00640 1 --2- 172.21.15.39:0/2624748951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f807815cd30 0x7f8078155b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:29.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:29.974+0000 7f806fe00640 1 -- 172.21.15.39:0/2624748951 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80781484c0 con 0x7f807806fc10 2024-09-21T07:44:29.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:29.974+0000 7f806fe00640 1 --2- 172.21.15.39:0/2624748951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f807806fc10 0x7f8078155650 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f806000d6a0 tx=0x7f806000db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:29.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:29.975+0000 7f806e000640 1 -- 172.21.15.39:0/2624748951 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8060017070 con 0x7f807806fc10 2024-09-21T07:44:29.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:29.975+0000 7f806e000640 1 -- 172.21.15.39:0/2624748951 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8060004310 con 0x7f807806fc10 2024-09-21T07:44:29.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:29.975+0000 7f806e000640 1 -- 172.21.15.39:0/2624748951 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8060009a80 con 0x7f807806fc10 2024-09-21T07:44:29.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:29.975+0000 7f807ce00640 1 -- 172.21.15.39:0/2624748951 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80781487a0 con 0x7f807806fc10 2024-09-21T07:44:29.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:29.975+0000 7f807ce00640 1 -- 172.21.15.39:0/2624748951 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8078147420 con 0x7f807806fc10 2024-09-21T07:44:29.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:29.976+0000 7f807ce00640 1 -- 172.21.15.39:0/2624748951 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8078148cc0 con 0x7f807806fc10 2024-09-21T07:44:29.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:29.977+0000 7f806e000640 1 -- 172.21.15.39:0/2624748951 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8060009c20 con 0x7f807806fc10 2024-09-21T07:44:29.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:29.978+0000 7f806e000640 1 --2- 172.21.15.39:0/2624748951 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8054078b90 0x7f805407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:29.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:29.978+0000 7f806e000640 1 -- 172.21.15.39:0/2624748951 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8420+0+0 (secure 0 0 0) 0x7f806009b890 con 0x7f807806fc10 2024-09-21T07:44:29.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:29.978+0000 7f8075e00640 1 --2- 172.21.15.39:0/2624748951 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8054078b90 0x7f805407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:29.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:29.979+0000 7f8075e00640 1 --2- 172.21.15.39:0/2624748951 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8054078b90 0x7f805407b0a0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8068033000 tx=0x7f8068004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:29.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:29.983+0000 7f806e000640 1 -- 172.21.15.39:0/2624748951 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8078148cc0 con 0x7f807806fc10 2024-09-21T07:44:30.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.219+0000 7f807ce00640 1 -- 172.21.15.39:0/2624748951 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8078073ec0 con 0x7f807806fc10 2024-09-21T07:44:30.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.219+0000 7f806e000640 1 -- 172.21.15.39:0/2624748951 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 164 v164) ==== 95+0+5252 (secure 0 0 0) 0x7f8060063210 con 0x7f807806fc10 2024-09-21T07:44:30.221 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:30.221 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":164,"btime":"2024-09-21T07:44:28:210229+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":8268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/67641510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":67641510},{"type":"v1","addr":"172.21.15.39:6833","nonce":67641510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":161}],"filesystems":[{"mdsmap":{"epoch":164,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:44:28.210222+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6929,"mds_1":8052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6929":{"gid":6929,"name":"d","rank":0,"incarnation":160,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2582569093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2582569093},{"type":"v1","addr":"172.21.15.148:6837","nonce":2582569093}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8052":{"gid":8052,"name":"a","rank":1,"incarnation":155,"state":"up:active","state_seq":18,"addr":"172.21.15.39:6835/1551311805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1551311805},{"type":"v1","addr":"172.21.15.39:6835","nonce":1551311805}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6929,"qdb_cluster":[6929,8052]},"id":6}]} 2024-09-21T07:44:30.222 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 164 2024-09-21T07:44:30.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.221+0000 7f807ce00640 1 -- 172.21.15.39:0/2624748951 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8054078b90 msgr2=0x7f805407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:30.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.221+0000 7f807ce00640 1 --2- 172.21.15.39:0/2624748951 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8054078b90 0x7f805407b0a0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8068033000 tx=0x7f8068004730 comp rx=0 tx=0).stop 2024-09-21T07:44:30.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.222+0000 7f807ce00640 1 -- 172.21.15.39:0/2624748951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f807806fc10 msgr2=0x7f8078155650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:30.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.222+0000 7f807ce00640 1 --2- 172.21.15.39:0/2624748951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f807806fc10 0x7f8078155650 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f806000d6a0 tx=0x7f806000db60 comp rx=0 tx=0).stop 2024-09-21T07:44:30.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.222+0000 7f807ce00640 1 -- 172.21.15.39:0/2624748951 shutdown_connections 2024-09-21T07:44:30.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.222+0000 7f807ce00640 1 --2- 172.21.15.39:0/2624748951 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8054078b90 0x7f805407b0a0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:30.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.222+0000 7f807ce00640 1 --2- 172.21.15.39:0/2624748951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f807815cd30 0x7f8078155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:30.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.222+0000 7f807ce00640 1 --2- 172.21.15.39:0/2624748951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f807806fc10 0x7f8078155650 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:30.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.222+0000 7f807ce00640 1 --2- 172.21.15.39:0/2624748951 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f807806f800 0x7f80780aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:30.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.222+0000 7f807ce00640 1 -- 172.21.15.39:0/2624748951 >> 172.21.15.39:0/2624748951 conn(0x7f807806cfc0 msgr2=0x7f80780b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:30.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.222+0000 7f807ce00640 1 -- 172.21.15.39:0/2624748951 shutdown_connections 2024-09-21T07:44:30.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.222+0000 7f807ce00640 1 -- 172.21.15.39:0/2624748951 wait complete. 2024-09-21T07:44:30.246 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-21T07:44:30.246 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:44:30.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.472+0000 7f6de6200640 1 -- 172.21.15.39:0/2342020149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6de00cbd70 msgr2=0x7f6de0072fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:30.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.472+0000 7f6de6200640 1 --2- 172.21.15.39:0/2342020149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6de00cbd70 0x7f6de0072fa0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8005fd0 tx=0x7f6dd802aaa0 comp rx=0 tx=0).stop 2024-09-21T07:44:30.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.472+0000 7f6de6200640 1 -- 172.21.15.39:0/2342020149 shutdown_connections 2024-09-21T07:44:30.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.472+0000 7f6de6200640 1 --2- 172.21.15.39:0/2342020149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6de00cbd70 0x7f6de0072fa0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:30.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.472+0000 7f6de6200640 1 --2- 172.21.15.39:0/2342020149 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6de00d0f20 0x7f6de015c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:30.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.472+0000 7f6de6200640 1 -- 172.21.15.39:0/2342020149 >> 172.21.15.39:0/2342020149 conn(0x7f6de00b2510 msgr2=0x7f6de00d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:30.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.472+0000 7f6de6200640 1 -- 172.21.15.39:0/2342020149 shutdown_connections 2024-09-21T07:44:30.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.472+0000 7f6de6200640 1 -- 172.21.15.39:0/2342020149 wait complete. 2024-09-21T07:44:30.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.474+0000 7f6de6200640 1 Processor -- start 2024-09-21T07:44:30.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.474+0000 7f6de6200640 1 -- start start 2024-09-21T07:44:30.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.476+0000 7f6de6200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6de00cb960 0x7f6de01f6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:30.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.476+0000 7f6de6200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6de00cbd70 0x7f6de00c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:30.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.476+0000 7f6de6200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6de00d0f20 0x7f6de00c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:30.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.476+0000 7f6de6200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6de00b4570 con 0x7f6de00d0f20 2024-09-21T07:44:30.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.476+0000 7f6de6200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6de00b43f0 con 0x7f6de00cb960 2024-09-21T07:44:30.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.476+0000 7f6de6200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f6de00b46f0 con 0x7f6de00cbd70 2024-09-21T07:44:30.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.476+0000 7f6de5800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6de00d0f20 0x7f6de00c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:30.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.476+0000 7f6de5800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6de00d0f20 0x7f6de00c9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35258/0 (socket says 172.21.15.39:35258) 2024-09-21T07:44:30.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.476+0000 7f6de5800640 1 -- 172.21.15.39:0/4162521018 learned_addr learned my addr 172.21.15.39:0/4162521018 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:30.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.476+0000 7f6de4e00640 1 --2- 172.21.15.39:0/4162521018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6de00cb960 0x7f6de01f6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:30.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.476+0000 7f6dd7e00640 1 --2- 172.21.15.39:0/4162521018 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6de00cbd70 0x7f6de00c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:30.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.477+0000 7f6de5800640 1 -- 172.21.15.39:0/4162521018 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6de00cbd70 msgr2=0x7f6de00c8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:30.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.477+0000 7f6de5800640 1 --2- 172.21.15.39:0/4162521018 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6de00cbd70 0x7f6de00c8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:30.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.477+0000 7f6de5800640 1 -- 172.21.15.39:0/4162521018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6de00cb960 msgr2=0x7f6de01f6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:30.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.477+0000 7f6de5800640 1 --2- 172.21.15.39:0/4162521018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6de00cb960 0x7f6de01f6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:30.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.477+0000 7f6de5800640 1 -- 172.21.15.39:0/4162521018 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de00c91e0 con 0x7f6de00d0f20 2024-09-21T07:44:30.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.477+0000 7f6de4e00640 1 --2- 172.21.15.39:0/4162521018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6de00cb960 0x7f6de01f6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:44:30.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.477+0000 7f6de5800640 1 --2- 172.21.15.39:0/4162521018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6de00d0f20 0x7f6de00c9f40 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8006010 tx=0x7f6dd8003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:30.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.477+0000 7f6ddd600640 1 -- 172.21.15.39:0/4162521018 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8044070 con 0x7f6de00d0f20 2024-09-21T07:44:30.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.477+0000 7f6ddd600640 1 -- 172.21.15.39:0/4162521018 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6dd8004040 con 0x7f6de00d0f20 2024-09-21T07:44:30.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.477+0000 7f6de6200640 1 -- 172.21.15.39:0/4162521018 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6de01f94c0 con 0x7f6de00d0f20 2024-09-21T07:44:30.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.477+0000 7f6ddd600640 1 -- 172.21.15.39:0/4162521018 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8004430 con 0x7f6de00d0f20 2024-09-21T07:44:30.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.477+0000 7f6de6200640 1 -- 172.21.15.39:0/4162521018 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6de01f9a80 con 0x7f6de00d0f20 2024-09-21T07:44:30.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.479+0000 7f6de6200640 1 -- 172.21.15.39:0/4162521018 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6de00b4c10 con 0x7f6de00d0f20 2024-09-21T07:44:30.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.480+0000 7f6ddd600640 1 -- 172.21.15.39:0/4162521018 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6dd8047070 con 0x7f6de00d0f20 2024-09-21T07:44:30.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.481+0000 7f6ddd600640 1 --2- 172.21.15.39:0/4162521018 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6dbc078b90 0x7f6dbc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:30.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.481+0000 7f6ddd600640 1 -- 172.21.15.39:0/4162521018 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8420+0+0 (secure 0 0 0) 0x7f6dd80bc680 con 0x7f6de00d0f20 2024-09-21T07:44:30.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.481+0000 7f6de4e00640 1 --2- 172.21.15.39:0/4162521018 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6dbc078b90 0x7f6dbc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:30.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.482+0000 7f6de4e00640 1 --2- 172.21.15.39:0/4162521018 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6dbc078b90 0x7f6dbc07b0a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0009b80 tx=0x7f6dd0005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:30.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.485+0000 7f6ddd600640 1 -- 172.21.15.39:0/4162521018 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6dd8032030 con 0x7f6de00d0f20 2024-09-21T07:44:30.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.721+0000 7f6de6200640 1 -- 172.21.15.39:0/4162521018 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6de00b1be0 con 0x7f6de00d0f20 2024-09-21T07:44:30.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.722+0000 7f6ddd600640 1 -- 172.21.15.39:0/4162521018 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 164 v164) ==== 95+0+5252 (secure 0 0 0) 0x7f6dd8083930 con 0x7f6de00d0f20 2024-09-21T07:44:30.723 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:30.724 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":164,"btime":"2024-09-21T07:44:28:210229+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":8268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/67641510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":67641510},{"type":"v1","addr":"172.21.15.39:6833","nonce":67641510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":161}],"filesystems":[{"mdsmap":{"epoch":164,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:44:28.210222+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6929,"mds_1":8052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6929":{"gid":6929,"name":"d","rank":0,"incarnation":160,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2582569093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2582569093},{"type":"v1","addr":"172.21.15.148:6837","nonce":2582569093}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8052":{"gid":8052,"name":"a","rank":1,"incarnation":155,"state":"up:active","state_seq":18,"addr":"172.21.15.39:6835/1551311805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1551311805},{"type":"v1","addr":"172.21.15.39:6835","nonce":1551311805}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6929,"qdb_cluster":[6929,8052]},"id":6}]} 2024-09-21T07:44:30.724 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 164 2024-09-21T07:44:30.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.724+0000 7f6de6200640 1 -- 172.21.15.39:0/4162521018 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6dbc078b90 msgr2=0x7f6dbc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:30.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.724+0000 7f6de6200640 1 --2- 172.21.15.39:0/4162521018 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6dbc078b90 0x7f6dbc07b0a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0009b80 tx=0x7f6dd0005f90 comp rx=0 tx=0).stop 2024-09-21T07:44:30.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.724+0000 7f6de6200640 1 -- 172.21.15.39:0/4162521018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6de00d0f20 msgr2=0x7f6de00c9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:30.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.724+0000 7f6de6200640 1 --2- 172.21.15.39:0/4162521018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6de00d0f20 0x7f6de00c9f40 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8006010 tx=0x7f6dd8003ce0 comp rx=0 tx=0).stop 2024-09-21T07:44:30.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.724+0000 7f6de6200640 1 -- 172.21.15.39:0/4162521018 shutdown_connections 2024-09-21T07:44:30.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.724+0000 7f6de6200640 1 --2- 172.21.15.39:0/4162521018 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6dbc078b90 0x7f6dbc07b0a0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:30.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.724+0000 7f6de6200640 1 --2- 172.21.15.39:0/4162521018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6de00d0f20 0x7f6de00c9f40 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:30.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.724+0000 7f6de6200640 1 --2- 172.21.15.39:0/4162521018 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6de00cbd70 0x7f6de00c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:30.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.724+0000 7f6de6200640 1 --2- 172.21.15.39:0/4162521018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6de00cb960 0x7f6de01f6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:30.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.724+0000 7f6de6200640 1 -- 172.21.15.39:0/4162521018 >> 172.21.15.39:0/4162521018 conn(0x7f6de00b2510 msgr2=0x7f6de015f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:30.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.724+0000 7f6de6200640 1 -- 172.21.15.39:0/4162521018 shutdown_connections 2024-09-21T07:44:30.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.724+0000 7f6de6200640 1 -- 172.21.15.39:0/4162521018 wait complete. 2024-09-21T07:44:30.751 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:active 2024-09-21T07:44:30.751 DEBUG:tasks.cephfs.filesystem:reached state 'up:active' in 0.5051195621490479s 2024-09-21T07:44:30.751 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:44:30.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.980+0000 7f183d800640 1 -- 172.21.15.39:0/3995167320 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f181c004800 msgr2=0x7f181c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:30.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.980+0000 7f183d800640 1 --2- 172.21.15.39:0/3995167320 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f181c004800 0x7f181c004c10 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f1820009b50 tx=0x7f181c005830 comp rx=0 tx=0).stop 2024-09-21T07:44:30.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.980+0000 7f183ce00640 1 -- 172.21.15.39:0/3995167320 reap_dead start 2024-09-21T07:44:30.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.981+0000 7f183d800640 1 -- 172.21.15.39:0/3995167320 shutdown_connections 2024-09-21T07:44:30.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.981+0000 7f183d800640 1 -- 172.21.15.39:0/3995167320 >> 172.21.15.39:0/3995167320 conn(0x7f1838074050 msgr2=0x7f1838074490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:30.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.981+0000 7f183d800640 1 -- 172.21.15.39:0/3995167320 shutdown_connections 2024-09-21T07:44:30.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.981+0000 7f183d800640 1 -- 172.21.15.39:0/3995167320 wait complete. 2024-09-21T07:44:30.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.983+0000 7f183d800640 1 Processor -- start 2024-09-21T07:44:30.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.983+0000 7f183d800640 1 -- start start 2024-09-21T07:44:30.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.984+0000 7f183d800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f181c004800 0x7f18381f5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:30.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.985+0000 7f183d800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f183813f450 0x7f18380a9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:30.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.985+0000 7f183d800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18380a99c0 0x7f18380ae670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:30.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.985+0000 7f183d800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f18380ce230 con 0x7f181c004800 2024-09-21T07:44:30.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.985+0000 7f183d800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f18380ce0b0 con 0x7f18380a99c0 2024-09-21T07:44:30.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.985+0000 7f183d800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f183815d290 con 0x7f183813f450 2024-09-21T07:44:30.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.985+0000 7f1837e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f181c004800 0x7f18381f5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:30.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.985+0000 7f1837e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f181c004800 0x7f18381f5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35278/0 (socket says 172.21.15.39:35278) 2024-09-21T07:44:30.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.985+0000 7f1837e00640 1 -- 172.21.15.39:0/3688491262 learned_addr learned my addr 172.21.15.39:0/3688491262 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:30.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.985+0000 7f182f400640 1 --2- 172.21.15.39:0/3688491262 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f183813f450 0x7f18380a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:30.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.985+0000 7f183ce00640 1 --2- 172.21.15.39:0/3688491262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18380a99c0 0x7f18380ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:30.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.985+0000 7f1837e00640 1 -- 172.21.15.39:0/3688491262 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f183813f450 msgr2=0x7f18380a9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:30.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.985+0000 7f1837e00640 1 --2- 172.21.15.39:0/3688491262 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f183813f450 0x7f18380a9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:30.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.985+0000 7f1837e00640 1 -- 172.21.15.39:0/3688491262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18380a99c0 msgr2=0x7f18380ae670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:30.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.985+0000 7f1837e00640 1 --2- 172.21.15.39:0/3688491262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18380a99c0 0x7f18380ae670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:30.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.985+0000 7f1837e00640 1 -- 172.21.15.39:0/3688491262 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18380a9dd0 con 0x7f181c004800 2024-09-21T07:44:30.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.985+0000 7f182f400640 1 --2- 172.21.15.39:0/3688491262 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f183813f450 0x7f18380a9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:44:30.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.985+0000 7f183ce00640 1 --2- 172.21.15.39:0/3688491262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18380a99c0 0x7f18380ae670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:44:30.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.986+0000 7f1837e00640 1 --2- 172.21.15.39:0/3688491262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f181c004800 0x7f18381f5dc0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f1828006680 tx=0x7f18280039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:30.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.986+0000 7f1834c00640 1 -- 172.21.15.39:0/3688491262 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1828015070 con 0x7f181c004800 2024-09-21T07:44:30.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.986+0000 7f1834c00640 1 -- 172.21.15.39:0/3688491262 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1828011900 con 0x7f181c004800 2024-09-21T07:44:30.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.986+0000 7f183d800640 1 -- 172.21.15.39:0/3688491262 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18381f9850 con 0x7f181c004800 2024-09-21T07:44:30.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.986+0000 7f1834c00640 1 -- 172.21.15.39:0/3688491262 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1828011d50 con 0x7f181c004800 2024-09-21T07:44:30.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.986+0000 7f183d800640 1 -- 172.21.15.39:0/3688491262 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18381f8fb0 con 0x7f181c004800 2024-09-21T07:44:30.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.987+0000 7f183d800640 1 -- 172.21.15.39:0/3688491262 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18380b4cf0 con 0x7f181c004800 2024-09-21T07:44:30.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.988+0000 7f1834c00640 1 -- 172.21.15.39:0/3688491262 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1828018070 con 0x7f181c004800 2024-09-21T07:44:30.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.989+0000 7f1834c00640 1 --2- 172.21.15.39:0/3688491262 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1814078b90 0x7f181407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:30.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.989+0000 7f1834c00640 1 -- 172.21.15.39:0/3688491262 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8420+0+0 (secure 0 0 0) 0x7f182809c510 con 0x7f181c004800 2024-09-21T07:44:30.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.990+0000 7f182f400640 1 --2- 172.21.15.39:0/3688491262 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1814078b90 0x7f181407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:30.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.991+0000 7f182f400640 1 --2- 172.21.15.39:0/3688491262 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1814078b90 0x7f181407b0a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f1820035000 tx=0x7f182002d040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:30.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:30.993+0000 7f1834c00640 1 -- 172.21.15.39:0/3688491262 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f18380a9dd0 con 0x7f181c004800 2024-09-21T07:44:31.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.232+0000 7f183d800640 1 -- 172.21.15.39:0/3688491262 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1838074740 con 0x7f181c004800 2024-09-21T07:44:31.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.233+0000 7f1834c00640 1 -- 172.21.15.39:0/3688491262 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 164 v164) ==== 95+0+5252 (secure 0 0 0) 0x7f18380ce5a0 con 0x7f181c004800 2024-09-21T07:44:31.235 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:31.235 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":164,"btime":"2024-09-21T07:44:28:210229+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":8268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/67641510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":67641510},{"type":"v1","addr":"172.21.15.39:6833","nonce":67641510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":161}],"filesystems":[{"mdsmap":{"epoch":164,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:44:28.210222+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6929,"mds_1":8052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6929":{"gid":6929,"name":"d","rank":0,"incarnation":160,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2582569093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2582569093},{"type":"v1","addr":"172.21.15.148:6837","nonce":2582569093}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8052":{"gid":8052,"name":"a","rank":1,"incarnation":155,"state":"up:active","state_seq":18,"addr":"172.21.15.39:6835/1551311805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1551311805},{"type":"v1","addr":"172.21.15.39:6835","nonce":1551311805}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6929,"qdb_cluster":[6929,8052]},"id":6}]} 2024-09-21T07:44:31.236 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 164 2024-09-21T07:44:31.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.235+0000 7f183d800640 1 -- 172.21.15.39:0/3688491262 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1814078b90 msgr2=0x7f181407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:31.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.236+0000 7f183d800640 1 --2- 172.21.15.39:0/3688491262 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1814078b90 0x7f181407b0a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f1820035000 tx=0x7f182002d040 comp rx=0 tx=0).stop 2024-09-21T07:44:31.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.236+0000 7f183d800640 1 -- 172.21.15.39:0/3688491262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f181c004800 msgr2=0x7f18381f5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:31.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.236+0000 7f183d800640 1 --2- 172.21.15.39:0/3688491262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f181c004800 0x7f18381f5dc0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f1828006680 tx=0x7f18280039c0 comp rx=0 tx=0).stop 2024-09-21T07:44:31.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.236+0000 7f183d800640 1 -- 172.21.15.39:0/3688491262 shutdown_connections 2024-09-21T07:44:31.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.236+0000 7f183d800640 1 --2- 172.21.15.39:0/3688491262 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1814078b90 0x7f181407b0a0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:31.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.236+0000 7f183d800640 1 --2- 172.21.15.39:0/3688491262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f18380a99c0 0x7f18380ae670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:31.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.236+0000 7f183d800640 1 --2- 172.21.15.39:0/3688491262 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f183813f450 0x7f18380a9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:31.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.236+0000 7f183d800640 1 --2- 172.21.15.39:0/3688491262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f181c004800 0x7f18381f5dc0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:31.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.236+0000 7f183d800640 1 -- 172.21.15.39:0/3688491262 >> 172.21.15.39:0/3688491262 conn(0x7f1838074050 msgr2=0x7f18380b45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:31.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.236+0000 7f183d800640 1 -- 172.21.15.39:0/3688491262 shutdown_connections 2024-09-21T07:44:31.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.236+0000 7f183d800640 1 -- 172.21.15.39:0/3688491262 wait complete. 2024-09-21T07:44:31.262 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json dump snaps --server 2024-09-21T07:44:31.509 INFO:teuthology.orchestra.run.smithi148.stdout:{"last_snap":20,"last_created":17,"last_destroyed":19,"pending_noop":[],"snaps":[],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2024-09-21T07:44:31.510 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 17, "last_destroyed": 19, "last_snap": 20, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [] } 2024-09-21T07:44:31.510 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T07:44:31.510 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:44:31.649 INFO:tasks.ceph.mds.d.smithi148.stderr:2024-09-21T07:44:31.647+0000 7f0a47000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:44:31.649 INFO:tasks.ceph.mds.d.smithi148.stderr:2024-09-21T07:44:31.647+0000 7f0a47000640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:44:31.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.735+0000 7fe1f8c00640 1 -- 172.21.15.39:0/3243049032 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1f415cd30 msgr2=0x7fe1f40cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:31.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.735+0000 7fe1f8c00640 1 --2- 172.21.15.39:0/3243049032 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1f415cd30 0x7fe1f40cfd10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4009830 tx=0x7fe1e402a510 comp rx=0 tx=0).stop 2024-09-21T07:44:31.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.735+0000 7fe1f8c00640 1 -- 172.21.15.39:0/3243049032 shutdown_connections 2024-09-21T07:44:31.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.735+0000 7fe1f8c00640 1 --2- 172.21.15.39:0/3243049032 >> v2:172.21.15.148:3300/0 conn(0x7fe1f406f800 0x7fe1f40cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:31.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.735+0000 7fe1f8c00640 1 --2- 172.21.15.39:0/3243049032 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1f415cd30 0x7fe1f40cfd10 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:31.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.735+0000 7fe1f8c00640 1 -- 172.21.15.39:0/3243049032 >> 172.21.15.39:0/3243049032 conn(0x7fe1f406cfc0 msgr2=0x7fe1f406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:31.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.735+0000 7fe1f8c00640 1 -- 172.21.15.39:0/3243049032 shutdown_connections 2024-09-21T07:44:31.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.736+0000 7fe1f8c00640 1 -- 172.21.15.39:0/3243049032 wait complete. 2024-09-21T07:44:31.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.738+0000 7fe1f8c00640 1 Processor -- start 2024-09-21T07:44:31.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.738+0000 7fe1f8c00640 1 -- start start 2024-09-21T07:44:31.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.739+0000 7fe1f8c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1f406f800 0x7fe1f4145690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:31.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.739+0000 7fe1f8c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1f406fc10 0x7fe1f4148200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:31.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.739+0000 7fe1f8c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1f415cd30 0x7fe1f415eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:31.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.740+0000 7fe1f8c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f4073890 con 0x7fe1f406fc10 2024-09-21T07:44:31.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.740+0000 7fe1f8c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f4073710 con 0x7fe1f415cd30 2024-09-21T07:44:31.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.740+0000 7fe1f8c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe1f4073a10 con 0x7fe1f406f800 2024-09-21T07:44:31.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.740+0000 7fe1ebe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1f406fc10 0x7fe1f4148200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:31.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.740+0000 7fe1ebe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1f406fc10 0x7fe1f4148200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35288/0 (socket says 172.21.15.39:35288) 2024-09-21T07:44:31.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.740+0000 7fe1ebe00640 1 -- 172.21.15.39:0/445473963 learned_addr learned my addr 172.21.15.39:0/445473963 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:31.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.740+0000 7fe1f1e00640 1 --2- 172.21.15.39:0/445473963 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1f406f800 0x7fe1f4145690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:31.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.740+0000 7fe1f2800640 1 --2- 172.21.15.39:0/445473963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1f415cd30 0x7fe1f415eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:31.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.740+0000 7fe1ebe00640 1 -- 172.21.15.39:0/445473963 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1f406f800 msgr2=0x7fe1f4145690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:31.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.740+0000 7fe1ebe00640 1 --2- 172.21.15.39:0/445473963 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1f406f800 0x7fe1f4145690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:31.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.740+0000 7fe1ebe00640 1 -- 172.21.15.39:0/445473963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1f415cd30 msgr2=0x7fe1f415eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:31.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.740+0000 7fe1ebe00640 1 --2- 172.21.15.39:0/445473963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1f415cd30 0x7fe1f415eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:31.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.740+0000 7fe1ebe00640 1 -- 172.21.15.39:0/445473963 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f41556f0 con 0x7fe1f406fc10 2024-09-21T07:44:31.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.740+0000 7fe1f2800640 1 --2- 172.21.15.39:0/445473963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1f415cd30 0x7fe1f415eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:44:31.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.740+0000 7fe1ebe00640 1 --2- 172.21.15.39:0/445473963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1f406fc10 0x7fe1f4148200 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc00e9f0 tx=0x7fe1dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:31.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.741+0000 7fe1ea000640 1 -- 172.21.15.39:0/445473963 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc00cd80 con 0x7fe1f406fc10 2024-09-21T07:44:31.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.741+0000 7fe1ea000640 1 -- 172.21.15.39:0/445473963 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1dc004020 con 0x7fe1f406fc10 2024-09-21T07:44:31.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.741+0000 7fe1ea000640 1 -- 172.21.15.39:0/445473963 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc004470 con 0x7fe1f406fc10 2024-09-21T07:44:31.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.741+0000 7fe1f8c00640 1 -- 172.21.15.39:0/445473963 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1f41460e0 con 0x7fe1f406fc10 2024-09-21T07:44:31.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.741+0000 7fe1f8c00640 1 -- 172.21.15.39:0/445473963 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1f41466a0 con 0x7fe1f406fc10 2024-09-21T07:44:31.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.742+0000 7fe1f8c00640 1 -- 172.21.15.39:0/445473963 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1f4148ce0 con 0x7fe1f406fc10 2024-09-21T07:44:31.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.743+0000 7fe1ea000640 1 -- 172.21.15.39:0/445473963 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe1dc0035f0 con 0x7fe1f406fc10 2024-09-21T07:44:31.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.745+0000 7fe1ea000640 1 --2- 172.21.15.39:0/445473963 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe1d0078b90 0x7fe1d007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:31.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.745+0000 7fe1ea000640 1 -- 172.21.15.39:0/445473963 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8420+0+0 (secure 0 0 0) 0x7fe1dc09c130 con 0x7fe1f406fc10 2024-09-21T07:44:31.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.745+0000 7fe1f1e00640 1 --2- 172.21.15.39:0/445473963 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe1d0078b90 0x7fe1d007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:31.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.746+0000 7fe1f1e00640 1 --2- 172.21.15.39:0/445473963 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe1d0078b90 0x7fe1d007b0a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4034000 tx=0x7fe1e4004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:31.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.750+0000 7fe1ea000640 1 -- 172.21.15.39:0/445473963 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe1dc063b10 con 0x7fe1f406fc10 2024-09-21T07:44:31.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.984+0000 7fe1f8c00640 1 -- 172.21.15.39:0/445473963 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1f40ced60 con 0x7fe1f406fc10 2024-09-21T07:44:31.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.985+0000 7fe1ea000640 1 -- 172.21.15.39:0/445473963 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 164 v164) ==== 95+0+5252 (secure 0 0 0) 0x7fe1f40ced60 con 0x7fe1f406fc10 2024-09-21T07:44:31.987 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:31.987 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":164,"btime":"2024-09-21T07:44:28:210229+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":8268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/67641510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":67641510},{"type":"v1","addr":"172.21.15.39:6833","nonce":67641510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":161}],"filesystems":[{"mdsmap":{"epoch":164,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:44:28.210222+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6929,"mds_1":8052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6929":{"gid":6929,"name":"d","rank":0,"incarnation":160,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2582569093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2582569093},{"type":"v1","addr":"172.21.15.148:6837","nonce":2582569093}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8052":{"gid":8052,"name":"a","rank":1,"incarnation":155,"state":"up:active","state_seq":18,"addr":"172.21.15.39:6835/1551311805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1551311805},{"type":"v1","addr":"172.21.15.39:6835","nonce":1551311805}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6929,"qdb_cluster":[6929,8052]},"id":6}]} 2024-09-21T07:44:31.987 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 164 2024-09-21T07:44:31.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.987+0000 7fe1f8c00640 1 -- 172.21.15.39:0/445473963 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe1d0078b90 msgr2=0x7fe1d007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:31.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.987+0000 7fe1f8c00640 1 --2- 172.21.15.39:0/445473963 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe1d0078b90 0x7fe1d007b0a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4034000 tx=0x7fe1e4004730 comp rx=0 tx=0).stop 2024-09-21T07:44:31.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.987+0000 7fe1f8c00640 1 -- 172.21.15.39:0/445473963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1f406fc10 msgr2=0x7fe1f4148200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:31.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.987+0000 7fe1f8c00640 1 --2- 172.21.15.39:0/445473963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1f406fc10 0x7fe1f4148200 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc00e9f0 tx=0x7fe1dc00eeb0 comp rx=0 tx=0).stop 2024-09-21T07:44:31.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.987+0000 7fe1f8c00640 1 -- 172.21.15.39:0/445473963 shutdown_connections 2024-09-21T07:44:31.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.987+0000 7fe1f8c00640 1 --2- 172.21.15.39:0/445473963 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe1d0078b90 0x7fe1d007b0a0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:31.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.987+0000 7fe1f8c00640 1 --2- 172.21.15.39:0/445473963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1f415cd30 0x7fe1f415eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:31.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.987+0000 7fe1f8c00640 1 --2- 172.21.15.39:0/445473963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1f406fc10 0x7fe1f4148200 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:31.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.987+0000 7fe1f8c00640 1 --2- 172.21.15.39:0/445473963 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1f406f800 0x7fe1f4145690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:31.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.987+0000 7fe1f8c00640 1 -- 172.21.15.39:0/445473963 >> 172.21.15.39:0/445473963 conn(0x7fe1f406cfc0 msgr2=0x7fe1f415b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:31.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.987+0000 7fe1f8c00640 1 -- 172.21.15.39:0/445473963 shutdown_connections 2024-09-21T07:44:31.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:31.988+0000 7fe1f8c00640 1 -- 172.21.15.39:0/445473963 wait complete. 2024-09-21T07:44:32.011 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json dump snaps --server 2024-09-21T07:44:32.104 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:44:32.108 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:44:32.253 INFO:teuthology.orchestra.run.smithi148.stdout:{"last_snap":20,"last_created":17,"last_destroyed":19,"pending_noop":[],"snaps":[],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2024-09-21T07:44:32.253 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 17, "last_destroyed": 19, "last_snap": 20, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [] } 2024-09-21T07:44:32.253 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:44:32.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:44:32.253 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-09-21T07:44:33.016 INFO:teuthology.orchestra.run.smithi039.stderr:mkdir: cannot create directory ‘d1/dir/.snap/s4’: Permission denied 2024-09-21T07:44:33.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:44:33.017 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:44:33.017 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:44:33.053 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:44:33.053 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:44:33.128 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:44:33.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.394+0000 7f5a1ae00640 1 -- 172.21.15.39:0/4173883908 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a1415f790 msgr2=0x7f5a140d0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:33.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.394+0000 7f5a1ae00640 1 --2- 172.21.15.39:0/4173883908 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a1415f790 0x7f5a140d0620 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f5a14072c00 tx=0x7f5a0002a580 comp rx=0 tx=0).stop 2024-09-21T07:44:33.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.395+0000 7f5a1ae00640 1 -- 172.21.15.39:0/4173883908 shutdown_connections 2024-09-21T07:44:33.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.395+0000 7f5a1ae00640 1 --2- 172.21.15.39:0/4173883908 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5a1415fba0 0x7f5a140d0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:33.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.395+0000 7f5a1ae00640 1 --2- 172.21.15.39:0/4173883908 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a1415f790 0x7f5a140d0620 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:33.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.395+0000 7f5a1ae00640 1 --2- 172.21.15.39:0/4173883908 >> v2:172.21.15.148:3300/0 conn(0x7f5a140b47c0 0x7f5a14155800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:33.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.395+0000 7f5a1ae00640 1 -- 172.21.15.39:0/4173883908 >> 172.21.15.39:0/4173883908 conn(0x7f5a1406cfc0 msgr2=0x7f5a1406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:33.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.395+0000 7f5a1ae00640 1 -- 172.21.15.39:0/4173883908 shutdown_connections 2024-09-21T07:44:33.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.395+0000 7f5a1ae00640 1 -- 172.21.15.39:0/4173883908 wait complete. 2024-09-21T07:44:33.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.397+0000 7f5a1ae00640 1 Processor -- start 2024-09-21T07:44:33.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.397+0000 7f5a1ae00640 1 -- start start 2024-09-21T07:44:33.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.398+0000 7f5a1ae00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a1415f790 0x7f5a14142520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:33.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.399+0000 7f5a1ae00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a1415fba0 0x7f5a140adda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:33.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.399+0000 7f5a1ae00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5a140ae2e0 0x7f5a1415bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:33.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.399+0000 7f5a1ae00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5a14162b20 con 0x7f5a1415f790 2024-09-21T07:44:33.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.399+0000 7f5a1ae00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5a141629a0 con 0x7f5a1415fba0 2024-09-21T07:44:33.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.399+0000 7f5a1ae00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f5a14162ca0 con 0x7f5a140ae2e0 2024-09-21T07:44:33.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.399+0000 7f5a13e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a1415f790 0x7f5a14142520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:33.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.399+0000 7f5a13e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a1415f790 0x7f5a14142520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54366/0 (socket says 172.21.15.39:54366) 2024-09-21T07:44:33.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.399+0000 7f5a13e00640 1 -- 172.21.15.39:0/4193520415 learned_addr learned my addr 172.21.15.39:0/4193520415 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:33.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.399+0000 7f5a18e00640 1 --2- 172.21.15.39:0/4193520415 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5a140ae2e0 0x7f5a1415bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:33.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.399+0000 7f5a13400640 1 --2- 172.21.15.39:0/4193520415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a1415fba0 0x7f5a140adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:33.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.399+0000 7f5a13e00640 1 -- 172.21.15.39:0/4193520415 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5a140ae2e0 msgr2=0x7f5a1415bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:33.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.399+0000 7f5a13e00640 1 --2- 172.21.15.39:0/4193520415 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5a140ae2e0 0x7f5a1415bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:33.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.399+0000 7f5a13e00640 1 -- 172.21.15.39:0/4193520415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a1415fba0 msgr2=0x7f5a140adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:33.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.399+0000 7f5a13e00640 1 --2- 172.21.15.39:0/4193520415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a1415fba0 0x7f5a140adda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:33.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.399+0000 7f5a13e00640 1 -- 172.21.15.39:0/4193520415 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a140ae6f0 con 0x7f5a1415f790 2024-09-21T07:44:33.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.399+0000 7f5a13e00640 1 --2- 172.21.15.39:0/4193520415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a1415f790 0x7f5a14142520 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400b3e0 tx=0x7f5a0400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:33.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.400+0000 7f5a0be00640 1 -- 172.21.15.39:0/4193520415 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a04015020 con 0x7f5a1415f790 2024-09-21T07:44:33.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.400+0000 7f5a0be00640 1 -- 172.21.15.39:0/4193520415 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a04011940 con 0x7f5a1415f790 2024-09-21T07:44:33.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.400+0000 7f5a1ae00640 1 -- 172.21.15.39:0/4193520415 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a1415b2d0 con 0x7f5a1415f790 2024-09-21T07:44:33.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.400+0000 7f5a0be00640 1 -- 172.21.15.39:0/4193520415 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a04011d90 con 0x7f5a1415f790 2024-09-21T07:44:33.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.400+0000 7f5a1ae00640 1 -- 172.21.15.39:0/4193520415 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a141598b0 con 0x7f5a1415f790 2024-09-21T07:44:33.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.401+0000 7f5a1ae00640 1 -- 172.21.15.39:0/4193520415 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a140b47c0 con 0x7f5a1415f790 2024-09-21T07:44:33.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.402+0000 7f5a0be00640 1 -- 172.21.15.39:0/4193520415 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5a04018020 con 0x7f5a1415f790 2024-09-21T07:44:33.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.403+0000 7f5a0be00640 1 --2- 172.21.15.39:0/4193520415 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f59f4078b90 0x7f59f407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:33.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.403+0000 7f5a0be00640 1 -- 172.21.15.39:0/4193520415 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8420+0+0 (secure 0 0 0) 0x7f5a0409b760 con 0x7f5a1415f790 2024-09-21T07:44:33.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.404+0000 7f5a13400640 1 --2- 172.21.15.39:0/4193520415 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f59f4078b90 0x7f59f407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:33.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.405+0000 7f5a13400640 1 --2- 172.21.15.39:0/4193520415 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f59f4078b90 0x7f59f407b0a0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f5a00033000 tx=0x7f5a00004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:33.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.408+0000 7f5a0be00640 1 -- 172.21.15.39:0/4193520415 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5a0401f020 con 0x7f5a1415f790 2024-09-21T07:44:33.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.614+0000 7f5a1ae00640 1 -- 172.21.15.39:0/4193520415 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5a14160500 con 0x7f5a1415f790 2024-09-21T07:44:33.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.615+0000 7f5a0be00640 1 -- 172.21.15.39:0/4193520415 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 37 entries v123) ==== 81+0+2227 (secure 0 0 0) 0x7f5a140ae6f0 con 0x7f5a1415f790 2024-09-21T07:44:33.617 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/1790581246 2024-09-22T07:44:24.184881+0000 2024-09-21T07:44:33.617 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1731243684 2024-09-21T08:44:16.323688+0000 2024-09-21T07:44:33.617 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/1326711417 2024-09-22T07:43:26.076277+0000 2024-09-21T07:44:33.617 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/1326711417 2024-09-22T07:43:26.076277+0000 2024-09-21T07:44:33.617 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/364657398 2024-09-22T07:39:15.579613+0000 2024-09-21T07:44:33.617 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.1:0/177249410 2024-09-21T08:41:13.882767+0000 2024-09-21T07:44:33.617 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/73220122 2024-09-22T07:42:40.074859+0000 2024-09-21T07:44:33.617 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/596665086 2024-09-21T08:43:35.317820+0000 2024-09-21T07:44:33.617 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/829716143 2024-09-22T07:37:01.259219+0000 2024-09-21T07:44:33.617 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/855150775 2024-09-22T07:39:07.739045+0000 2024-09-21T07:44:33.617 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/2381504293 2024-09-22T07:38:32.207414+0000 2024-09-21T07:44:33.617 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/855150775 2024-09-22T07:39:07.739045+0000 2024-09-21T07:44:33.618 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/742594435 2024-09-21T08:42:01.225749+0000 2024-09-21T07:44:33.618 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/2821446711 2024-09-22T07:40:57.372409+0000 2024-09-21T07:44:33.618 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.1:0/90671776 2024-09-21T08:39:22.988045+0000 2024-09-21T07:44:33.618 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/759792812 2024-09-22T07:44:05.615021+0000 2024-09-21T07:44:33.618 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/1472430787 2024-09-22T07:37:58.212892+0000 2024-09-21T07:44:33.618 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/3695489158 2024-09-22T07:41:51.807018+0000 2024-09-21T07:44:33.618 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/3026827614 2024-09-22T07:37:29.781472+0000 2024-09-21T07:44:33.618 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/3026827614 2024-09-22T07:37:29.781472+0000 2024-09-21T07:44:33.618 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/829716143 2024-09-22T07:37:01.259219+0000 2024-09-21T07:44:33.618 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/3695489158 2024-09-22T07:41:51.807018+0000 2024-09-21T07:44:33.618 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/1472430787 2024-09-22T07:37:58.212892+0000 2024-09-21T07:44:33.618 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/759792812 2024-09-22T07:44:05.615021+0000 2024-09-21T07:44:33.618 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/992188984 2024-09-22T07:40:00.778049+0000 2024-09-21T07:44:33.618 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6833/364657398 2024-09-22T07:39:15.579613+0000 2024-09-21T07:44:33.618 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/992188984 2024-09-22T07:40:00.778049+0000 2024-09-21T07:44:33.619 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6833/1790581246 2024-09-22T07:44:24.184881+0000 2024-09-21T07:44:33.619 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6833/1456343023 2024-09-22T07:40:08.637993+0000 2024-09-21T07:44:33.619 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/2381504293 2024-09-22T07:38:32.207414+0000 2024-09-21T07:44:33.619 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/2336112936 2024-09-22T07:41:05.979253+0000 2024-09-21T07:44:33.619 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/1456343023 2024-09-22T07:40:08.637993+0000 2024-09-21T07:44:33.619 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/2821446711 2024-09-22T07:40:57.372409+0000 2024-09-21T07:44:33.619 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/3356393880 2024-09-21T08:40:15.824973+0000 2024-09-21T07:44:33.619 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/2336112936 2024-09-22T07:41:05.979253+0000 2024-09-21T07:44:33.619 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/73220122 2024-09-22T07:42:40.074859+0000 2024-09-21T07:44:33.619 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1669604593 2024-09-21T08:42:52.059256+0000 2024-09-21T07:44:33.620 INFO:teuthology.orchestra.run.smithi039.stderr:listed 37 entries 2024-09-21T07:44:33.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.617+0000 7f5a1ae00640 1 -- 172.21.15.39:0/4193520415 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f59f4078b90 msgr2=0x7f59f407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:33.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.617+0000 7f5a1ae00640 1 --2- 172.21.15.39:0/4193520415 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f59f4078b90 0x7f59f407b0a0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f5a00033000 tx=0x7f5a00004730 comp rx=0 tx=0).stop 2024-09-21T07:44:33.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.617+0000 7f5a1ae00640 1 -- 172.21.15.39:0/4193520415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a1415f790 msgr2=0x7f5a14142520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:33.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.617+0000 7f5a1ae00640 1 --2- 172.21.15.39:0/4193520415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a1415f790 0x7f5a14142520 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400b3e0 tx=0x7f5a0400b8a0 comp rx=0 tx=0).stop 2024-09-21T07:44:33.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.617+0000 7f5a1ae00640 1 -- 172.21.15.39:0/4193520415 shutdown_connections 2024-09-21T07:44:33.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.617+0000 7f5a1ae00640 1 --2- 172.21.15.39:0/4193520415 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f59f4078b90 0x7f59f407b0a0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:33.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.617+0000 7f5a1ae00640 1 --2- 172.21.15.39:0/4193520415 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5a140ae2e0 0x7f5a1415bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:33.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.617+0000 7f5a1ae00640 1 --2- 172.21.15.39:0/4193520415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a1415fba0 0x7f5a140adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:33.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.617+0000 7f5a1ae00640 1 --2- 172.21.15.39:0/4193520415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a1415f790 0x7f5a14142520 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:33.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.617+0000 7f5a1ae00640 1 -- 172.21.15.39:0/4193520415 >> 172.21.15.39:0/4193520415 conn(0x7f5a1406cfc0 msgr2=0x7f5a1406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:33.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.617+0000 7f5a1ae00640 1 -- 172.21.15.39:0/4193520415 shutdown_connections 2024-09-21T07:44:33.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:33.617+0000 7f5a1ae00640 1 -- 172.21.15.39:0/4193520415 wait complete. 2024-09-21T07:44:33.645 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T07:44:33.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:44:33.646 DEBUG:teuthology.orchestra.run.smithi039:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T07:44:33.705 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T07:44:33.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:44:33.706 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:44:33.742 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:44:33.742 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:44:33.818 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:44:33.818 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:44:34.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.084+0000 7fbfdf000640 1 -- 172.21.15.39:0/54425646 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfd80d0f20 msgr2=0x7fbfd815c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:34.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.084+0000 7fbfdf000640 1 --2- 172.21.15.39:0/54425646 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfd80d0f20 0x7fbfd815c830 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc005fd0 tx=0x7fbfcc02aa90 comp rx=0 tx=0).stop 2024-09-21T07:44:34.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.084+0000 7fbfdf000640 1 -- 172.21.15.39:0/54425646 shutdown_connections 2024-09-21T07:44:34.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.084+0000 7fbfdf000640 1 --2- 172.21.15.39:0/54425646 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbfd80cbd70 0x7fbfd8072fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:34.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.084+0000 7fbfdf000640 1 --2- 172.21.15.39:0/54425646 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfd80d0f20 0x7fbfd815c830 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:34.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.084+0000 7fbfdf000640 1 -- 172.21.15.39:0/54425646 >> 172.21.15.39:0/54425646 conn(0x7fbfd80b2510 msgr2=0x7fbfd80d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:34.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.084+0000 7fbfdf000640 1 -- 172.21.15.39:0/54425646 shutdown_connections 2024-09-21T07:44:34.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.084+0000 7fbfdf000640 1 -- 172.21.15.39:0/54425646 wait complete. 2024-09-21T07:44:34.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.086+0000 7fbfdf000640 1 Processor -- start 2024-09-21T07:44:34.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.086+0000 7fbfdf000640 1 -- start start 2024-09-21T07:44:34.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.088+0000 7fbfdf000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfd80cb960 0x7fbfd80ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:34.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.088+0000 7fbfdf000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbfd80cbd70 0x7fbfd81f5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:34.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.088+0000 7fbfdf000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfd80d0f20 0x7fbfd809a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:34.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.088+0000 7fbfdf000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd80b4570 con 0x7fbfd80cb960 2024-09-21T07:44:34.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.088+0000 7fbfdf000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd80b43f0 con 0x7fbfd80d0f20 2024-09-21T07:44:34.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.088+0000 7fbfdf000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fbfd80b46f0 con 0x7fbfd80cbd70 2024-09-21T07:44:34.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.088+0000 7fbfddc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfd80cb960 0x7fbfd80ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:34.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.088+0000 7fbfddc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfd80cb960 0x7fbfd80ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54388/0 (socket says 172.21.15.39:54388) 2024-09-21T07:44:34.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.088+0000 7fbfddc00640 1 -- 172.21.15.39:0/2727931687 learned_addr learned my addr 172.21.15.39:0/2727931687 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:34.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.088+0000 7fbfddc00640 1 -- 172.21.15.39:0/2727931687 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbfd80cbd70 msgr2=0x7fbfd81f5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:34.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.088+0000 7fbfde600640 1 --2- 172.21.15.39:0/2727931687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfd80d0f20 0x7fbfd809a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:34.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.088+0000 7fbfdd200640 1 --2- 172.21.15.39:0/2727931687 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbfd80cbd70 0x7fbfd81f5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:34.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.088+0000 7fbfddc00640 1 --2- 172.21.15.39:0/2727931687 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbfd80cbd70 0x7fbfd81f5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:34.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.088+0000 7fbfddc00640 1 -- 172.21.15.39:0/2727931687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfd80d0f20 msgr2=0x7fbfd809a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:34.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.088+0000 7fbfddc00640 1 --2- 172.21.15.39:0/2727931687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfd80d0f20 0x7fbfd809a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:34.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.088+0000 7fbfddc00640 1 -- 172.21.15.39:0/2727931687 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd80cf9c0 con 0x7fbfd80cb960 2024-09-21T07:44:34.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.088+0000 7fbfde600640 1 --2- 172.21.15.39:0/2727931687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfd80d0f20 0x7fbfd809a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:44:34.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.088+0000 7fbfdd200640 1 --2- 172.21.15.39:0/2727931687 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbfd80cbd70 0x7fbfd81f5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:44:34.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.089+0000 7fbfddc00640 1 --2- 172.21.15.39:0/2727931687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfd80cb960 0x7fbfd80ce8a0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc000f80 tx=0x7fbfcc003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:34.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.089+0000 7fbfd5600640 1 -- 172.21.15.39:0/2727931687 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc044070 con 0x7fbfd80cb960 2024-09-21T07:44:34.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.089+0000 7fbfd5600640 1 -- 172.21.15.39:0/2727931687 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbfcc004250 con 0x7fbfd80cb960 2024-09-21T07:44:34.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.089+0000 7fbfdf000640 1 -- 172.21.15.39:0/2727931687 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfd80cfca0 con 0x7fbfd80cb960 2024-09-21T07:44:34.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.089+0000 7fbfd5600640 1 -- 172.21.15.39:0/2727931687 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc004660 con 0x7fbfd80cb960 2024-09-21T07:44:34.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.089+0000 7fbfdf000640 1 -- 172.21.15.39:0/2727931687 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfd80ce380 con 0x7fbfd80cb960 2024-09-21T07:44:34.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.090+0000 7fbfdf000640 1 -- 172.21.15.39:0/2727931687 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfd80b4c10 con 0x7fbfd80cb960 2024-09-21T07:44:34.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.092+0000 7fbfd5600640 1 -- 172.21.15.39:0/2727931687 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbfcc047070 con 0x7fbfd80cb960 2024-09-21T07:44:34.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.093+0000 7fbfd5600640 1 --2- 172.21.15.39:0/2727931687 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbfb8078b90 0x7fbfb807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:34.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.093+0000 7fbfd5600640 1 -- 172.21.15.39:0/2727931687 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8420+0+0 (secure 0 0 0) 0x7fbfcc0bc700 con 0x7fbfd80cb960 2024-09-21T07:44:34.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.093+0000 7fbfdd200640 1 --2- 172.21.15.39:0/2727931687 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbfb8078b90 0x7fbfb807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:34.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.094+0000 7fbfdd200640 1 --2- 172.21.15.39:0/2727931687 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbfb8078b90 0x7fbfb807b0a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8159f10 tx=0x7fbfc4009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:34.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.098+0000 7fbfd5600640 1 -- 172.21.15.39:0/2727931687 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbfcc032030 con 0x7fbfd80cb960 2024-09-21T07:44:34.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.331+0000 7fbfdf000640 1 -- 172.21.15.39:0/2727931687 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbfd815cec0 con 0x7fbfd80cb960 2024-09-21T07:44:34.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.331+0000 7fbfd5600640 1 -- 172.21.15.39:0/2727931687 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v164) ==== 53+0+73 (secure 0 0 0) 0x7fbfcc084110 con 0x7fbfd80cb960 2024-09-21T07:44:34.333 INFO:teuthology.orchestra.run.smithi039.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:44:34.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.334+0000 7fbfdf000640 1 -- 172.21.15.39:0/2727931687 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbfb8078b90 msgr2=0x7fbfb807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:34.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.334+0000 7fbfdf000640 1 --2- 172.21.15.39:0/2727931687 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbfb8078b90 0x7fbfb807b0a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8159f10 tx=0x7fbfc4009040 comp rx=0 tx=0).stop 2024-09-21T07:44:34.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.334+0000 7fbfdf000640 1 -- 172.21.15.39:0/2727931687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfd80cb960 msgr2=0x7fbfd80ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:34.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.334+0000 7fbfdf000640 1 --2- 172.21.15.39:0/2727931687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfd80cb960 0x7fbfd80ce8a0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc000f80 tx=0x7fbfcc003ce0 comp rx=0 tx=0).stop 2024-09-21T07:44:34.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.334+0000 7fbfdf000640 1 -- 172.21.15.39:0/2727931687 shutdown_connections 2024-09-21T07:44:34.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.334+0000 7fbfdf000640 1 --2- 172.21.15.39:0/2727931687 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbfb8078b90 0x7fbfb807b0a0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:34.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.334+0000 7fbfdf000640 1 --2- 172.21.15.39:0/2727931687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbfd80d0f20 0x7fbfd809a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:34.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.334+0000 7fbfdf000640 1 --2- 172.21.15.39:0/2727931687 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbfd80cbd70 0x7fbfd81f5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:34.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.334+0000 7fbfdf000640 1 --2- 172.21.15.39:0/2727931687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbfd80cb960 0x7fbfd80ce8a0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:34.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.334+0000 7fbfdf000640 1 -- 172.21.15.39:0/2727931687 >> 172.21.15.39:0/2727931687 conn(0x7fbfd80b2510 msgr2=0x7fbfd815bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:34.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.334+0000 7fbfdf000640 1 -- 172.21.15.39:0/2727931687 shutdown_connections 2024-09-21T07:44:34.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34.334+0000 7fbfdf000640 1 -- 172.21.15.39:0/2727931687 wait complete. 2024-09-21T07:44:34.361 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:44:34.361 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:44:34.361 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi039.front.sepia.ceph.com 2024-09-21T07:44:34.361 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T07:44:34.361 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:44:34.361 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:44:34.362 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:44:34.362 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:44:34.362 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:44:34.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:44:34.362 DEBUG:teuthology.orchestra.run.smithi039:> ip addr 2024-09-21T07:44:34.397 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:44:34.398 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:44:34.398 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:44:34.398 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:44:34.398 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2024-09-21T07:44:34.398 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:44:34.398 INFO:teuthology.orchestra.run.smithi039.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:44:34.398 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:44:34.398 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f0 2024-09-21T07:44:34.398 INFO:teuthology.orchestra.run.smithi039.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:44:34.398 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c7 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:44:34.398 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f1 2024-09-21T07:44:34.398 INFO:teuthology.orchestra.run.smithi039.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:44:34.399 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:70 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:44:34.399 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f0 2024-09-21T07:44:34.399 INFO:teuthology.orchestra.run.smithi039.stdout: inet 172.21.15.39/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T07:44:34.399 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft 42045sec preferred_lft 42045sec 2024-09-21T07:44:34.399 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec4:7aff:febd:1370/64 scope link 2024-09-21T07:44:34.399 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:44:34.399 INFO:teuthology.orchestra.run.smithi039.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:44:34.399 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:71 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:44:34.399 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f1 2024-09-21T07:44:34.399 INFO:teuthology.orchestra.run.smithi039.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:44:34.399 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff 2024-09-21T07:44:34.399 INFO:teuthology.orchestra.run.smithi039.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:44:34.399 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:44:34.399 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::b079:ddff:fe0e:6333/64 scope link 2024-09-21T07:44:34.399 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:44:34.400 INFO:teuthology.orchestra.run.smithi039.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:44:34.400 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:44:34.400 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::2ca7:98ff:fe8a:d0ed/64 scope link 2024-09-21T07:44:34.400 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:44:34.400 INFO:teuthology.orchestra.run.smithi039.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:44:34.400 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether ee:10:53:01:0c:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:44:34.400 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec10:53ff:fe01:c44/64 scope link 2024-09-21T07:44:34.400 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:44:34.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:44:34.401 DEBUG:teuthology.orchestra.run.smithi039:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:44:34.495 INFO:teuthology.orchestra.run.smithi039.stdout:1 2024-09-21T07:44:34.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:44:34.498 DEBUG:teuthology.orchestra.run.smithi039:> route 2024-09-21T07:44:34.533 INFO:teuthology.orchestra.run.smithi039.stdout:Kernel IP routing table 2024-09-21T07:44:34.533 INFO:teuthology.orchestra.run.smithi039.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:44:34.533 INFO:teuthology.orchestra.run.smithi039.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T07:44:34.533 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T07:44:34.533 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:44:34.534 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:44:34.534 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-21T07:44:34.534 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T07:44:34.534 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T07:44:34.534 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T07:44:34.534 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-21T07:44:34.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:44:34.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:44:34.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:44:34.731 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-09-21T07:44:34.765 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:44:34.766 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T07:44:34.766 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:44:34.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:44:34.767 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-09-21T07:44:34.863 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:44:34.863 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T07:44:34.898 INFO:teuthology.orchestra.run.smithi039.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:44:34.898 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:44:34.898 DEBUG:teuthology.orchestra.run.smithi039:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:44:34.972 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-21T07:44:34.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:44:34.972 DEBUG:teuthology.orchestra.run.smithi039:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:44:35.172 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:44:35.172 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:44:35.207 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:44:35.207 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:44:35.207 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: resolved to: "172.21.15.39:3300,172.21.15.148:3300,172.21.15.148:3301" 2024-09-21T07:44:35.208 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: trying mount with new device syntax: 0@7ab9b04a-f88d-49aa-a493-e5c076637ffb.cephfs=/ 2024-09-21T07:44:35.208 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.39:3300/172.21.15.148:3300/172.21.15.148:3301" will pass to kernel 2024-09-21T07:44:35.209 INFO:teuthology.orchestra.run.smithi039.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T07:44:35.209 INFO:teuthology.orchestra.run.smithi039.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:44:35.209 INFO:teuthology.orchestra.run.smithi039.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:44:35.209 INFO:teuthology.orchestra.run.smithi039.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:44:35.209 INFO:teuthology.orchestra.run.smithi039.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:44:35.212 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:44:35.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:44:35.212 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:44:35.268 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:44:35.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:44:35.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:44:35.359 INFO:teuthology.orchestra.run.smithi039.stdout:client6992 2024-09-21T07:44:35.359 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:44:35.359 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:44:35.395 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 flush journal 2024-09-21T07:44:35.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.663+0000 7f7f45400640 1 -- 172.21.15.39:0/2643531479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f24005270 msgr2=0x7f7f2400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:35.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.663+0000 7f7f45400640 1 --2- 172.21.15.39:0/2643531479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f24005270 0x7f7f2400b9c0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f7f38006fd0 tx=0x7f7f24005830 comp rx=0 tx=0).stop 2024-09-21T07:44:35.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.663+0000 7f7f45400640 1 -- 172.21.15.39:0/2643531479 shutdown_connections 2024-09-21T07:44:35.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.663+0000 7f7f45400640 1 --2- 172.21.15.39:0/2643531479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f24005270 0x7f7f2400b9c0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:35.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.663+0000 7f7f45400640 1 -- 172.21.15.39:0/2643531479 >> 172.21.15.39:0/2643531479 conn(0x7f7f400b25f0 msgr2=0x7f7f400d01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:35.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.663+0000 7f7f45400640 1 -- 172.21.15.39:0/2643531479 shutdown_connections 2024-09-21T07:44:35.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.663+0000 7f7f45400640 1 -- 172.21.15.39:0/2643531479 wait complete. 2024-09-21T07:44:35.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.665+0000 7f7f45400640 1 Processor -- start 2024-09-21T07:44:35.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.666+0000 7f7f45400640 1 -- start start 2024-09-21T07:44:35.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.667+0000 7f7f45400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7f24005270 0x7f7f401f8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:35.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.667+0000 7f7f45400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f400c5990 0x7f7f401f4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:35.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.667+0000 7f7f45400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f400ceb30 0x7f7f400cef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:35.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.667+0000 7f7f45400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7f40160230 con 0x7f7f400ceb30 2024-09-21T07:44:35.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.667+0000 7f7f45400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7f400cc850 con 0x7f7f400c5990 2024-09-21T07:44:35.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.667+0000 7f7f45400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f7f401603b0 con 0x7f7f24005270 2024-09-21T07:44:35.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.667+0000 7f7f44a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f400ceb30 0x7f7f400cef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:35.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.667+0000 7f7f44a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f400ceb30 0x7f7f400cef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54398/0 (socket says 172.21.15.39:54398) 2024-09-21T07:44:35.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.667+0000 7f7f44a00640 1 -- 172.21.15.39:0/28779198 learned_addr learned my addr 172.21.15.39:0/28779198 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:35.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.668+0000 7f7f3fe00640 1 --2- 172.21.15.39:0/28779198 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7f24005270 0x7f7f401f8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:35.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.668+0000 7f7f37400640 1 --2- 172.21.15.39:0/28779198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f400c5990 0x7f7f401f4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:35.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.668+0000 7f7f44a00640 1 -- 172.21.15.39:0/28779198 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7f24005270 msgr2=0x7f7f401f8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:35.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.668+0000 7f7f44a00640 1 --2- 172.21.15.39:0/28779198 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7f24005270 0x7f7f401f8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:35.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.668+0000 7f7f44a00640 1 -- 172.21.15.39:0/28779198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f400c5990 msgr2=0x7f7f401f4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:35.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.668+0000 7f7f44a00640 1 --2- 172.21.15.39:0/28779198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f400c5990 0x7f7f401f4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:35.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.668+0000 7f7f44a00640 1 -- 172.21.15.39:0/28779198 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f400cf480 con 0x7f7f400ceb30 2024-09-21T07:44:35.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.668+0000 7f7f44a00640 1 --2- 172.21.15.39:0/28779198 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f400ceb30 0x7f7f400cef40 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f7f3802b470 tx=0x7f7f38003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:35.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.668+0000 7f7f3cc00640 1 -- 172.21.15.39:0/28779198 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3802b980 con 0x7f7f400ceb30 2024-09-21T07:44:35.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.668+0000 7f7f3cc00640 1 -- 172.21.15.39:0/28779198 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f3803a070 con 0x7f7f400ceb30 2024-09-21T07:44:35.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.668+0000 7f7f45400640 1 -- 172.21.15.39:0/28779198 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f401f50d0 con 0x7f7f400ceb30 2024-09-21T07:44:35.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.669+0000 7f7f3cc00640 1 -- 172.21.15.39:0/28779198 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3802bdc0 con 0x7f7f400ceb30 2024-09-21T07:44:35.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.669+0000 7f7f45400640 1 -- 172.21.15.39:0/28779198 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f401f5690 con 0x7f7f400ceb30 2024-09-21T07:44:35.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.671+0000 7f7f3cc00640 1 -- 172.21.15.39:0/28779198 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7f3803c020 con 0x7f7f400ceb30 2024-09-21T07:44:35.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.672+0000 7f7f3cc00640 1 --2- 172.21.15.39:0/28779198 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7f1c078b90 0x7f7f1c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:35.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.672+0000 7f7f3cc00640 1 -- 172.21.15.39:0/28779198 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8420+0+0 (secure 0 0 0) 0x7f7f380bacf0 con 0x7f7f400ceb30 2024-09-21T07:44:35.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.672+0000 7f7f3fe00640 1 --2- 172.21.15.39:0/28779198 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7f1c078b90 0x7f7f1c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:35.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.673+0000 7f7f4daba740 1 Processor -- start 2024-09-21T07:44:35.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.673+0000 7f7f4daba740 1 -- start start 2024-09-21T07:44:35.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.673+0000 7f7f3fe00640 1 --2- 172.21.15.39:0/28779198 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7f1c078b90 0x7f7f1c07b0a0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f7f30009bb0 tx=0x7f7f30005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:35.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.674+0000 7f7f4daba740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555a8bd92390 0x555a8c180730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:35.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.674+0000 7f7f4daba740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a8bd694d0 0x555a8c163400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:35.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.674+0000 7f7f4daba740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a8bd716b0 0x555a8c163940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:35.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.674+0000 7f7f4daba740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x555a8c0991c0 con 0x555a8bd716b0 2024-09-21T07:44:35.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.674+0000 7f7f4daba740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x555a8c0ac870 con 0x555a8bd694d0 2024-09-21T07:44:35.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.674+0000 7f7f4daba740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x555a8c09bca0 con 0x555a8bd92390 2024-09-21T07:44:35.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.674+0000 7f7f44a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a8bd716b0 0x555a8c163940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:35.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.674+0000 7f7f44a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a8bd716b0 0x555a8c163940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54406/0 (socket says 172.21.15.39:54406) 2024-09-21T07:44:35.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.674+0000 7f7f44a00640 1 -- 172.21.15.39:0/2684506853 learned_addr learned my addr 172.21.15.39:0/2684506853 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:35.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.675+0000 7f7f37400640 1 --2- 172.21.15.39:0/2684506853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a8bd694d0 0x555a8c163400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:35.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.675+0000 7f7f3fe00640 1 --2- 172.21.15.39:0/2684506853 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555a8bd92390 0x555a8c180730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:35.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.675+0000 7f7f44a00640 1 -- 172.21.15.39:0/2684506853 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555a8bd92390 msgr2=0x555a8c180730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:44:35.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.675+0000 7f7f44a00640 1 --2- 172.21.15.39:0/2684506853 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555a8bd92390 0x555a8c180730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:35.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.675+0000 7f7f44a00640 1 -- 172.21.15.39:0/2684506853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a8bd694d0 msgr2=0x555a8c163400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:44:35.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.675+0000 7f7f44a00640 1 --2- 172.21.15.39:0/2684506853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a8bd694d0 0x555a8c163400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:35.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.675+0000 7f7f44a00640 1 -- 172.21.15.39:0/2684506853 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a8be747d0 con 0x555a8bd716b0 2024-09-21T07:44:35.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.675+0000 7f7f3fe00640 1 --2- 172.21.15.39:0/2684506853 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555a8bd92390 0x555a8c180730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:44:35.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.675+0000 7f7f44a00640 1 --2- 172.21.15.39:0/2684506853 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a8bd716b0 0x555a8c163940 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f7f3807e700 tx=0x7f7f3807dee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dabb1c7ef9d2916 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:35.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.675+0000 7f7f2fe00640 1 -- 172.21.15.39:0/2684506853 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f38081a90 con 0x555a8bd716b0 2024-09-21T07:44:35.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.675+0000 7f7f2fe00640 1 -- 172.21.15.39:0/2684506853 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f3802f070 con 0x555a8bd716b0 2024-09-21T07:44:35.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.675+0000 7f7f2fe00640 1 -- 172.21.15.39:0/2684506853 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f38081f80 con 0x555a8bd716b0 2024-09-21T07:44:35.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.676+0000 7f7f4daba740 1 -- 172.21.15.39:0/2684506853 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a8bd716b0 msgr2=0x555a8c163940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:35.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.676+0000 7f7f4daba740 1 --2- 172.21.15.39:0/2684506853 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a8bd716b0 0x555a8c163940 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f7f3807e700 tx=0x7f7f3807dee0 comp rx=0 tx=0).stop 2024-09-21T07:44:35.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.676+0000 7f7f4daba740 1 -- 172.21.15.39:0/2684506853 shutdown_connections 2024-09-21T07:44:35.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.676+0000 7f7f4daba740 1 --2- 172.21.15.39:0/2684506853 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a8bd716b0 0x555a8c163940 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:35.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.676+0000 7f7f4daba740 1 --2- 172.21.15.39:0/2684506853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a8bd694d0 0x555a8c163400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:35.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.676+0000 7f7f4daba740 1 --2- 172.21.15.39:0/2684506853 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555a8bd92390 0x555a8c180730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:35.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.676+0000 7f7f4daba740 1 -- 172.21.15.39:0/2684506853 >> 172.21.15.39:0/2684506853 conn(0x555a8bd864e0 msgr2=0x555a8c139620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:35.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.676+0000 7f7f4daba740 1 -- 172.21.15.39:0/2684506853 shutdown_connections 2024-09-21T07:44:35.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.676+0000 7f7f4daba740 1 -- 172.21.15.39:0/2684506853 wait complete. 2024-09-21T07:44:35.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.677+0000 7f7f4daba740 1 -- start start 2024-09-21T07:44:35.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.678+0000 7f7f4daba740 1 Processor -- start 2024-09-21T07:44:35.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.679+0000 7f7f4daba740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a8bd694d0 0x555a8c191540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:35.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.679+0000 7f7f4daba740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a8bd716b0 0x555a8c151a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:35.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.679+0000 7f7f4daba740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555a8bd92390 0x555a8c138b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:35.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.679+0000 7f7f4daba740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x555a8c042410 con 0x555a8bd716b0 2024-09-21T07:44:35.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.679+0000 7f7f4daba740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x555a8c09fab0 con 0x555a8bd694d0 2024-09-21T07:44:35.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.679+0000 7f7f4daba740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x555a8be0c740 con 0x555a8bd92390 2024-09-21T07:44:35.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.679+0000 7f7f4daba740 10 client.0 _ll_register_callbacks cb 0x555a8c01f360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:44:35.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.679+0000 7f7f37400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a8bd716b0 0x555a8c151a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:35.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.679+0000 7f7f37400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a8bd716b0 0x555a8c151a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54410/0 (socket says 172.21.15.39:54410) 2024-09-21T07:44:35.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.679+0000 7f7f37400640 1 -- 172.21.15.39:0/2854056739 learned_addr learned my addr 172.21.15.39:0/2854056739 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:35.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.680+0000 7f7f3fe00640 1 --2- 172.21.15.39:0/2854056739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a8bd694d0 0x555a8c191540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:35.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.680+0000 7f7f44a00640 1 --2- 172.21.15.39:0/2854056739 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555a8bd92390 0x555a8c138b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:35.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.680+0000 7f7f37400640 1 -- 172.21.15.39:0/2854056739 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555a8bd92390 msgr2=0x555a8c138b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:44:35.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.680+0000 7f7f37400640 1 --2- 172.21.15.39:0/2854056739 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555a8bd92390 0x555a8c138b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:35.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.680+0000 7f7f37400640 1 -- 172.21.15.39:0/2854056739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a8bd694d0 msgr2=0x555a8c191540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:44:35.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.680+0000 7f7f37400640 1 --2- 172.21.15.39:0/2854056739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a8bd694d0 0x555a8c191540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:35.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.680+0000 7f7f37400640 1 -- 172.21.15.39:0/2854056739 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a8bdfd970 con 0x555a8bd716b0 2024-09-21T07:44:35.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.680+0000 7f7f37400640 1 --2- 172.21.15.39:0/2854056739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a8bd716b0 0x555a8c151a90 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f7f28006650 tx=0x7f7f28003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8411e3f454d4138b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:35.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.680+0000 7f7f2cc00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:44:35.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.680+0000 7f7f2cc00640 1 -- 172.21.15.39:0/2854056739 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f28001c30 con 0x555a8bd716b0 2024-09-21T07:44:35.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.680+0000 7f7f2cc00640 1 -- 172.21.15.39:0/2854056739 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 8420+0+0 (secure 0 0 0) 0x7f7f28012440 con 0x555a8bd716b0 2024-09-21T07:44:35.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.680+0000 7f7f4daba740 1 -- 172.21.15.39:0/2854056739 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f3802f070 con 0x555a8bd716b0 2024-09-21T07:44:35.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.681+0000 7f7f2cc00640 1 -- 172.21.15.39:0/2854056739 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=164) ==== 24+0+0 (secure 0 0 0) 0x7f7f28012640 con 0x555a8bd716b0 2024-09-21T07:44:35.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.681+0000 7f7f4daba740 10 client.8331 fetch_fsmap learned FSMap version 164 2024-09-21T07:44:35.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.681+0000 7f7f4daba740 1 -- 172.21.15.39:0/2854056739 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=164,monmap=2+}) -- 0x555a8bddb6a0 con 0x555a8bd716b0 2024-09-21T07:44:35.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.681+0000 7f7f2cc00640 1 -- 172.21.15.39:0/2854056739 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f3802f070 con 0x555a8bd716b0 2024-09-21T07:44:35.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.681+0000 7f7f2cc00640 1 -- 172.21.15.39:0/2854056739 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 164) ==== 3522+0+0 (secure 0 0 0) 0x7f7f280127e0 con 0x555a8bd716b0 2024-09-21T07:44:35.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.681+0000 7f7f4daba740 10 client.8331 fetch_fsmap finished waiting for FSMap version 164 2024-09-21T07:44:35.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.681+0000 7f7f4daba740 10 client.8331 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-09-21T07:44:35.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.681+0000 7f7f4daba740 20 client.8331 populate_metadata read hostname 'smithi039' 2024-09-21T07:44:35.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.681+0000 7f7f4daba740 1 --2- 172.21.15.39:0/2854056739 >> [v2:172.21.15.148:6836/2582569093,v1:172.21.15.148:6837/2582569093] conn(0x555a8bd50f60 0x555a8c1758e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:35.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.681+0000 7f7f4daba740 4 client.8331 mds_command: new command op to 6929 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:44:35.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.681+0000 7f7f4daba740 1 -- 172.21.15.39:0/2854056739 --> [v2:172.21.15.148:6836/2582569093,v1:172.21.15.148:6837/2582569093] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a8c0ae7c0 con 0x555a8bd50f60 2024-09-21T07:44:35.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.682+0000 7f7f3fe00640 1 --2- 172.21.15.39:0/2854056739 >> [v2:172.21.15.148:6836/2582569093,v1:172.21.15.148:6837/2582569093] conn(0x555a8bd50f60 0x555a8c1758e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:35.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.682+0000 7f7f3fe00640 1 --2- 172.21.15.39:0/2854056739 >> [v2:172.21.15.148:6836/2582569093,v1:172.21.15.148:6837/2582569093] conn(0x555a8bd50f60 0x555a8c1758e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6eedd6af459d9743 server_cookie=691b74356e68ac06 in_seq=0 out_seq=0 2024-09-21T07:44:35.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.682+0000 7f7f2cc00640 10 client.8331 ms_handle_connect on v2:172.21.15.148:6836/2582569093 2024-09-21T07:44:35.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.683+0000 7f7f2cc00640 1 -- 172.21.15.39:0/2854056739 <== mds.0 v2:172.21.15.148:6836/2582569093 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555a8c0ae7c0 con 0x555a8bd50f60 2024-09-21T07:44:35.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.683+0000 7f7f2cc00640 10 client.8331 handle_command_reply: tid=0 2024-09-21T07:44:35.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.683+0000 7f7f4daba740 1 client.8331 shutdown 2024-09-21T07:44:35.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.684+0000 7f7f4daba740 1 -- 172.21.15.39:0/2854056739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a8bd716b0 msgr2=0x555a8c151a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:35.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.684+0000 7f7f4daba740 1 --2- 172.21.15.39:0/2854056739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a8bd716b0 0x555a8c151a90 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f7f28006650 tx=0x7f7f28003f30 comp rx=0 tx=0).stop 2024-09-21T07:44:35.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.684+0000 7f7f4daba740 1 -- 172.21.15.39:0/2854056739 shutdown_connections 2024-09-21T07:44:35.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.684+0000 7f7f4daba740 1 --2- 172.21.15.39:0/2854056739 >> [v2:172.21.15.148:6836/2582569093,v1:172.21.15.148:6837/2582569093] conn(0x555a8bd50f60 0x555a8c1758e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:35.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.684+0000 7f7f4daba740 1 --2- 172.21.15.39:0/2854056739 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555a8bd92390 0x555a8c138b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:35.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.684+0000 7f7f4daba740 1 --2- 172.21.15.39:0/2854056739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a8bd716b0 0x555a8c151a90 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:35.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.684+0000 7f7f4daba740 1 --2- 172.21.15.39:0/2854056739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a8bd694d0 0x555a8c191540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:35.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.684+0000 7f7f2cc00640 0 client.8331 ms_handle_reset on v2:172.21.15.148:6836/2582569093 2024-09-21T07:44:35.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.684+0000 7f7f4daba740 1 -- 172.21.15.39:0/2854056739 >> 172.21.15.39:0/2854056739 conn(0x555a8bd864e0 msgr2=0x555a8c160050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:35.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.684+0000 7f7f4daba740 1 -- 172.21.15.39:0/2854056739 shutdown_connections 2024-09-21T07:44:35.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.684+0000 7f7f4daba740 1 -- 172.21.15.39:0/2854056739 wait complete. 2024-09-21T07:44:35.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.684+0000 7f7f4daba740 20 client.8331 trim_cache size 0 max 16384 2024-09-21T07:44:35.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.699+0000 7f7f4daba740 1 Processor -- start 2024-09-21T07:44:35.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.700+0000 7f7f4daba740 1 -- start start 2024-09-21T07:44:35.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.701+0000 7f7f4daba740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a8bd694d0 0x555a8c1384b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:35.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.701+0000 7f7f4daba740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555a8bd716b0 0x555a8c1389f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:35.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.701+0000 7f7f4daba740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a8bd92390 0x555a8c181790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:35.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.701+0000 7f7f4daba740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7f40204e30 con 0x555a8bd694d0 2024-09-21T07:44:35.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.701+0000 7f7f4daba740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7f40205500 con 0x555a8bd92390 2024-09-21T07:44:35.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.701+0000 7f7f4daba740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f7f40203870 con 0x555a8bd716b0 2024-09-21T07:44:35.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.701+0000 7f7f3fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a8bd694d0 0x555a8c1384b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:35.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.701+0000 7f7f3fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a8bd694d0 0x555a8c1384b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54426/0 (socket says 172.21.15.39:54426) 2024-09-21T07:44:35.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.701+0000 7f7f3fe00640 1 -- 172.21.15.39:0/1220167760 learned_addr learned my addr 172.21.15.39:0/1220167760 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:35.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.701+0000 7f7f37400640 1 --2- 172.21.15.39:0/1220167760 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555a8bd716b0 0x555a8c1389f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:35.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.701+0000 7f7f44a00640 1 --2- 172.21.15.39:0/1220167760 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a8bd92390 0x555a8c181790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:35.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.702+0000 7f7f3fe00640 1 -- 172.21.15.39:0/1220167760 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555a8bd716b0 msgr2=0x555a8c1389f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:44:35.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.702+0000 7f7f3fe00640 1 --2- 172.21.15.39:0/1220167760 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555a8bd716b0 0x555a8c1389f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:35.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.702+0000 7f7f3fe00640 1 -- 172.21.15.39:0/1220167760 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a8bd92390 msgr2=0x555a8c181790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:44:35.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.702+0000 7f7f3fe00640 1 --2- 172.21.15.39:0/1220167760 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a8bd92390 0x555a8c181790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:35.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.702+0000 7f7f3fe00640 1 -- 172.21.15.39:0/1220167760 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a8c174a20 con 0x555a8bd694d0 2024-09-21T07:44:35.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.702+0000 7f7f3fe00640 1 --2- 172.21.15.39:0/1220167760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a8bd694d0 0x555a8c1384b0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f7f300138f0 tx=0x7f7f30016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d10f910065b69294 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:35.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.702+0000 7f7f2f400640 1 -- 172.21.15.39:0/1220167760 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f30005020 con 0x555a8bd694d0 2024-09-21T07:44:35.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.702+0000 7f7f2f400640 1 -- 172.21.15.39:0/1220167760 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f300051c0 con 0x555a8bd694d0 2024-09-21T07:44:35.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.702+0000 7f7f2f400640 1 -- 172.21.15.39:0/1220167760 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f300055d0 con 0x555a8bd694d0 2024-09-21T07:44:35.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.703+0000 7f7f4daba740 1 -- 172.21.15.39:0/1220167760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a8bd694d0 msgr2=0x555a8c1384b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:35.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.703+0000 7f7f4daba740 1 --2- 172.21.15.39:0/1220167760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a8bd694d0 0x555a8c1384b0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f7f300138f0 tx=0x7f7f30016000 comp rx=0 tx=0).stop 2024-09-21T07:44:35.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.703+0000 7f7f4daba740 1 -- 172.21.15.39:0/1220167760 shutdown_connections 2024-09-21T07:44:35.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.703+0000 7f7f4daba740 1 --2- 172.21.15.39:0/1220167760 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a8bd92390 0x555a8c181790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:35.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.703+0000 7f7f4daba740 1 --2- 172.21.15.39:0/1220167760 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555a8bd716b0 0x555a8c1389f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:35.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.703+0000 7f7f4daba740 1 --2- 172.21.15.39:0/1220167760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a8bd694d0 0x555a8c1384b0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:35.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.703+0000 7f7f4daba740 1 -- 172.21.15.39:0/1220167760 >> 172.21.15.39:0/1220167760 conn(0x555a8bd864e0 msgr2=0x555a8c174310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:35.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.703+0000 7f7f4daba740 1 -- 172.21.15.39:0/1220167760 shutdown_connections 2024-09-21T07:44:35.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.703+0000 7f7f4daba740 1 -- 172.21.15.39:0/1220167760 wait complete. 2024-09-21T07:44:35.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.704+0000 7f7f4daba740 1 -- start start 2024-09-21T07:44:35.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.704+0000 7f7f4daba740 1 Processor -- start 2024-09-21T07:44:35.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.706+0000 7f7f4daba740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a8bd694d0 0x555a8c201a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:35.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.706+0000 7f7f4daba740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a8bd716b0 0x555a8c200940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:35.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.706+0000 7f7f4daba740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555a8bd92390 0x555a8c181790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:35.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.706+0000 7f7f4daba740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x555a8c134900 con 0x555a8bd694d0 2024-09-21T07:44:35.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.706+0000 7f7f4daba740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7f40203ec0 con 0x555a8bd716b0 2024-09-21T07:44:35.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.706+0000 7f7f4daba740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x555a8c201f80 con 0x555a8bd92390 2024-09-21T07:44:35.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.706+0000 7f7f4daba740 10 client.0 _ll_register_callbacks cb 0x7f7f28003730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:44:35.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.706+0000 7f7f3fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a8bd694d0 0x555a8c201a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:35.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.706+0000 7f7f3fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a8bd694d0 0x555a8c201a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54442/0 (socket says 172.21.15.39:54442) 2024-09-21T07:44:35.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.706+0000 7f7f3fe00640 1 -- 172.21.15.39:0/23103772 learned_addr learned my addr 172.21.15.39:0/23103772 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:35.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.706+0000 7f7f37400640 1 --2- 172.21.15.39:0/23103772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a8bd716b0 0x555a8c200940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:35.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.706+0000 7f7f44a00640 1 --2- 172.21.15.39:0/23103772 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555a8bd92390 0x555a8c181790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:35.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.706+0000 7f7f3fe00640 1 -- 172.21.15.39:0/23103772 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555a8bd92390 msgr2=0x555a8c181790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:44:35.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.706+0000 7f7f3fe00640 1 --2- 172.21.15.39:0/23103772 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555a8bd92390 0x555a8c181790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:35.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.707+0000 7f7f3fe00640 1 -- 172.21.15.39:0/23103772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a8bd716b0 msgr2=0x555a8c200940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:44:35.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.707+0000 7f7f3fe00640 1 --2- 172.21.15.39:0/23103772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a8bd716b0 0x555a8c200940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:35.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.707+0000 7f7f3fe00640 1 -- 172.21.15.39:0/23103772 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a8c136180 con 0x555a8bd694d0 2024-09-21T07:44:35.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.707+0000 7f7f3fe00640 1 --2- 172.21.15.39:0/23103772 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a8bd694d0 0x555a8c201a40 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f7f300039c0 tx=0x7f7f30003e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38f950f00cb1c6c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:35.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.707+0000 7f7f2cc00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:44:35.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.707+0000 7f7f2cc00640 1 -- 172.21.15.39:0/23103772 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f300047a0 con 0x555a8bd694d0 2024-09-21T07:44:35.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.707+0000 7f7f2cc00640 1 -- 172.21.15.39:0/23103772 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 8420+0+0 (secure 0 0 0) 0x7f7f30004940 con 0x555a8bd694d0 2024-09-21T07:44:35.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.707+0000 7f7f4daba740 1 -- 172.21.15.39:0/23103772 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f300051c0 con 0x555a8bd694d0 2024-09-21T07:44:35.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.707+0000 7f7f2cc00640 1 -- 172.21.15.39:0/23103772 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=164) ==== 24+0+0 (secure 0 0 0) 0x7f7f30045050 con 0x555a8bd694d0 2024-09-21T07:44:35.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.707+0000 7f7f4daba740 10 client.8337 fetch_fsmap learned FSMap version 164 2024-09-21T07:44:35.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.707+0000 7f7f4daba740 1 -- 172.21.15.39:0/23103772 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=164,monmap=2+}) -- 0x555a8c163400 con 0x555a8bd694d0 2024-09-21T07:44:35.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.708+0000 7f7f2cc00640 1 -- 172.21.15.39:0/23103772 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f300051c0 con 0x555a8bd694d0 2024-09-21T07:44:35.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.708+0000 7f7f2cc00640 1 -- 172.21.15.39:0/23103772 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 164) ==== 3522+0+0 (secure 0 0 0) 0x7f7f30004b60 con 0x555a8bd694d0 2024-09-21T07:44:35.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.708+0000 7f7f4daba740 10 client.8337 fetch_fsmap finished waiting for FSMap version 164 2024-09-21T07:44:35.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.708+0000 7f7f4daba740 10 client.8337 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-09-21T07:44:35.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.708+0000 7f7f4daba740 20 client.8337 populate_metadata read hostname 'smithi039' 2024-09-21T07:44:35.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.708+0000 7f7f4daba740 1 --2- 172.21.15.39:0/23103772 >> [v2:172.21.15.148:6836/2582569093,v1:172.21.15.148:6837/2582569093] conn(0x555a8c181cd0 0x555a8c180180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:35.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.708+0000 7f7f4daba740 4 client.8337 mds_command: new command op to 6929 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:44:35.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.708+0000 7f7f4daba740 1 -- 172.21.15.39:0/23103772 --> [v2:172.21.15.148:6836/2582569093,v1:172.21.15.148:6837/2582569093] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555a8c0aa270 con 0x555a8c181cd0 2024-09-21T07:44:35.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.708+0000 7f7f37400640 1 --2- 172.21.15.39:0/23103772 >> [v2:172.21.15.148:6836/2582569093,v1:172.21.15.148:6837/2582569093] conn(0x555a8c181cd0 0x555a8c180180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:35.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.709+0000 7f7f37400640 1 --2- 172.21.15.39:0/23103772 >> [v2:172.21.15.148:6836/2582569093,v1:172.21.15.148:6837/2582569093] conn(0x555a8c181cd0 0x555a8c180180 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2970122253e2254 server_cookie=adce8dcc70476216 in_seq=0 out_seq=0 2024-09-21T07:44:35.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.709+0000 7f7f2cc00640 10 client.8337 ms_handle_connect on v2:172.21.15.148:6836/2582569093 2024-09-21T07:44:35.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.723+0000 7f7f2cc00640 1 -- 172.21.15.39:0/23103772 <== mds.0 v2:172.21.15.148:6836/2582569093 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555a8c0aa270 con 0x555a8c181cd0 2024-09-21T07:44:35.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.723+0000 7f7f2cc00640 10 client.8337 handle_command_reply: tid=0 2024-09-21T07:44:35.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.723+0000 7f7f4daba740 1 client.8337 shutdown 2024-09-21T07:44:35.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.723+0000 7f7f4daba740 1 -- 172.21.15.39:0/23103772 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a8bd694d0 msgr2=0x555a8c201a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:35.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.723+0000 7f7f4daba740 1 --2- 172.21.15.39:0/23103772 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a8bd694d0 0x555a8c201a40 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f7f300039c0 tx=0x7f7f30003e30 comp rx=0 tx=0).stop 2024-09-21T07:44:35.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.723+0000 7f7f4daba740 1 -- 172.21.15.39:0/23103772 shutdown_connections 2024-09-21T07:44:35.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.723+0000 7f7f4daba740 1 --2- 172.21.15.39:0/23103772 >> [v2:172.21.15.148:6836/2582569093,v1:172.21.15.148:6837/2582569093] conn(0x555a8c181cd0 0x555a8c180180 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:35.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.723+0000 7f7f4daba740 1 --2- 172.21.15.39:0/23103772 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x555a8bd92390 0x555a8c181790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:35.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.723+0000 7f7f4daba740 1 --2- 172.21.15.39:0/23103772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555a8bd716b0 0x555a8c200940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:35.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.723+0000 7f7f4daba740 1 --2- 172.21.15.39:0/23103772 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a8bd694d0 0x555a8c201a40 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:35.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.723+0000 7f7f2cc00640 0 client.8337 ms_handle_reset on v2:172.21.15.148:6836/2582569093 2024-09-21T07:44:35.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.723+0000 7f7f4daba740 1 -- 172.21.15.39:0/23103772 >> 172.21.15.39:0/23103772 conn(0x555a8bd864e0 msgr2=0x555a8c133e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:35.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.723+0000 7f7f4daba740 1 -- 172.21.15.39:0/23103772 shutdown_connections 2024-09-21T07:44:35.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.723+0000 7f7f4daba740 1 -- 172.21.15.39:0/23103772 wait complete. 2024-09-21T07:44:35.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.724+0000 7f7f4daba740 20 client.8337 trim_cache size 0 max 16384 2024-09-21T07:44:35.727 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:44:35.727 INFO:teuthology.orchestra.run.smithi039.stdout: "message": "", 2024-09-21T07:44:35.727 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0 2024-09-21T07:44:35.727 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:44:35.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.724+0000 7f7f45400640 1 -- 172.21.15.39:0/28779198 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7f1c078b90 msgr2=0x7f7f1c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:35.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.724+0000 7f7f45400640 1 --2- 172.21.15.39:0/28779198 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7f1c078b90 0x7f7f1c07b0a0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f7f30009bb0 tx=0x7f7f30005f90 comp rx=0 tx=0).stop 2024-09-21T07:44:35.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.724+0000 7f7f45400640 1 -- 172.21.15.39:0/28779198 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f400ceb30 msgr2=0x7f7f400cef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:35.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.724+0000 7f7f45400640 1 --2- 172.21.15.39:0/28779198 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f400ceb30 0x7f7f400cef40 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f7f3802b470 tx=0x7f7f38003f10 comp rx=0 tx=0).stop 2024-09-21T07:44:35.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.724+0000 7f7f45400640 1 -- 172.21.15.39:0/28779198 shutdown_connections 2024-09-21T07:44:35.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.724+0000 7f7f45400640 1 --2- 172.21.15.39:0/28779198 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7f1c078b90 0x7f7f1c07b0a0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:35.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.724+0000 7f7f45400640 1 --2- 172.21.15.39:0/28779198 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f400ceb30 0x7f7f400cef40 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:35.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.724+0000 7f7f45400640 1 --2- 172.21.15.39:0/28779198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f400c5990 0x7f7f401f4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:35.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.724+0000 7f7f45400640 1 --2- 172.21.15.39:0/28779198 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7f24005270 0x7f7f401f8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:35.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.724+0000 7f7f45400640 1 -- 172.21.15.39:0/28779198 >> 172.21.15.39:0/28779198 conn(0x7f7f400b25f0 msgr2=0x7f7f400c1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:35.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.724+0000 7f7f45400640 1 -- 172.21.15.39:0/28779198 shutdown_connections 2024-09-21T07:44:35.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:35.724+0000 7f7f45400640 1 -- 172.21.15.39:0/28779198 wait complete. 2024-09-21T07:44:35.749 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:44:35.749 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:44:35.835 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:44:35.928 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:44:35.928 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:44:35.928 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:44:35.929 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:44:35.964 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:44:35.964 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:44:36.039 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:44:36.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.304+0000 7fd9fc000640 1 -- 172.21.15.39:0/3235748503 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9f40d0dd0 msgr2=0x7fd9f40cdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:36.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.304+0000 7fd9fc000640 1 --2- 172.21.15.39:0/3235748503 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9f40d0dd0 0x7fd9f40cdb90 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4072c00 tx=0x7fd9ec02aa30 comp rx=0 tx=0).stop 2024-09-21T07:44:36.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.304+0000 7fd9fc000640 1 -- 172.21.15.39:0/3235748503 shutdown_connections 2024-09-21T07:44:36.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.304+0000 7fd9fc000640 1 --2- 172.21.15.39:0/3235748503 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9f40d0dd0 0x7fd9f40cdb90 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:36.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.304+0000 7fd9fc000640 1 --2- 172.21.15.39:0/3235748503 >> v2:172.21.15.148:3300/0 conn(0x7fd9f415a460 0x7fd9f406fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:36.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.304+0000 7fd9fc000640 1 -- 172.21.15.39:0/3235748503 >> 172.21.15.39:0/3235748503 conn(0x7fd9f406cfc0 msgr2=0x7fd9f406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:36.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.304+0000 7fd9fc000640 1 -- 172.21.15.39:0/3235748503 shutdown_connections 2024-09-21T07:44:36.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.305+0000 7fd9fc000640 1 -- 172.21.15.39:0/3235748503 wait complete. 2024-09-21T07:44:36.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.307+0000 7fd9fc000640 1 Processor -- start 2024-09-21T07:44:36.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.307+0000 7fd9fc000640 1 -- start start 2024-09-21T07:44:36.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.308+0000 7fd9fc000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9f40cd780 0x7fd9f41e7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:36.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.308+0000 7fd9fc000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9f40d0dd0 0x7fd9f4159280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:36.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.308+0000 7fd9fc000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd9f415a460 0x7fd9f41364e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:36.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.308+0000 7fd9fc000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f415d560 con 0x7fd9f40cd780 2024-09-21T07:44:36.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.308+0000 7fd9fc000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f415d3e0 con 0x7fd9f40d0dd0 2024-09-21T07:44:36.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.308+0000 7fd9fc000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd9f415d6e0 con 0x7fd9f415a460 2024-09-21T07:44:36.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.309+0000 7fd9f9600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9f40cd780 0x7fd9f41e7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:36.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.309+0000 7fd9f9600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9f40cd780 0x7fd9f41e7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54466/0 (socket says 172.21.15.39:54466) 2024-09-21T07:44:36.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.309+0000 7fd9f9600640 1 -- 172.21.15.39:0/228854901 learned_addr learned my addr 172.21.15.39:0/228854901 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:36.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.309+0000 7fd9fa000640 1 --2- 172.21.15.39:0/228854901 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd9f415a460 0x7fd9f41364e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:36.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.309+0000 7fd9f8c00640 1 --2- 172.21.15.39:0/228854901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9f40d0dd0 0x7fd9f4159280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:36.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.309+0000 7fd9f9600640 1 -- 172.21.15.39:0/228854901 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd9f415a460 msgr2=0x7fd9f41364e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:36.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.309+0000 7fd9f9600640 1 --2- 172.21.15.39:0/228854901 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd9f415a460 0x7fd9f41364e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:36.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.309+0000 7fd9f9600640 1 -- 172.21.15.39:0/228854901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9f40d0dd0 msgr2=0x7fd9f4159280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:36.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.309+0000 7fd9f9600640 1 --2- 172.21.15.39:0/228854901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9f40d0dd0 0x7fd9f4159280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:36.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.309+0000 7fd9f9600640 1 -- 172.21.15.39:0/228854901 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f41597c0 con 0x7fd9f40cd780 2024-09-21T07:44:36.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.309+0000 7fd9fa000640 1 --2- 172.21.15.39:0/228854901 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd9f415a460 0x7fd9f41364e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:44:36.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.309+0000 7fd9f8c00640 1 --2- 172.21.15.39:0/228854901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9f40d0dd0 0x7fd9f4159280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:44:36.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.309+0000 7fd9f9600640 1 --2- 172.21.15.39:0/228854901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9f40cd780 0x7fd9f41e7530 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fd9e8009fd0 tx=0x7fd9e800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:36.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.310+0000 7fd9f1600640 1 -- 172.21.15.39:0/228854901 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e800ccc0 con 0x7fd9f40cd780 2024-09-21T07:44:36.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.310+0000 7fd9f1600640 1 -- 172.21.15.39:0/228854901 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9e800ee70 con 0x7fd9f40cd780 2024-09-21T07:44:36.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.310+0000 7fd9fc000640 1 -- 172.21.15.39:0/228854901 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9f41e6490 con 0x7fd9f40cd780 2024-09-21T07:44:36.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.310+0000 7fd9f1600640 1 -- 172.21.15.39:0/228854901 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e800ce60 con 0x7fd9f40cd780 2024-09-21T07:44:36.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.310+0000 7fd9fc000640 1 -- 172.21.15.39:0/228854901 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9f41e6a50 con 0x7fd9f40cd780 2024-09-21T07:44:36.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.311+0000 7fd9fc000640 1 -- 172.21.15.39:0/228854901 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9f415db90 con 0x7fd9f40cd780 2024-09-21T07:44:36.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.312+0000 7fd9f1600640 1 -- 172.21.15.39:0/228854901 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd9e8004280 con 0x7fd9f40cd780 2024-09-21T07:44:36.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.314+0000 7fd9f1600640 1 --2- 172.21.15.39:0/228854901 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd9d4078bc0 0x7fd9d407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:36.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.314+0000 7fd9f1600640 1 -- 172.21.15.39:0/228854901 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8420+0+0 (secure 0 0 0) 0x7fd9e809c210 con 0x7fd9f40cd780 2024-09-21T07:44:36.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.314+0000 7fd9f8c00640 1 --2- 172.21.15.39:0/228854901 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd9d4078bc0 0x7fd9d407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:36.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.315+0000 7fd9f8c00640 1 --2- 172.21.15.39:0/228854901 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd9d4078bc0 0x7fd9d407b0d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd9f40b21b0 tx=0x7fd9e0009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:36.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.318+0000 7fd9f1600640 1 -- 172.21.15.39:0/228854901 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd9e8063b90 con 0x7fd9f40cd780 2024-09-21T07:44:36.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.527+0000 7fd9fc000640 1 -- 172.21.15.39:0/228854901 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd9f40b3f60 con 0x7fd9f40cd780 2024-09-21T07:44:36.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.527+0000 7fd9f1600640 1 -- 172.21.15.39:0/228854901 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 37 entries v123) ==== 81+0+2227 (secure 0 0 0) 0x7fd9e8063490 con 0x7fd9f40cd780 2024-09-21T07:44:36.529 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/1790581246 2024-09-22T07:44:24.184881+0000 2024-09-21T07:44:36.530 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1731243684 2024-09-21T08:44:16.323688+0000 2024-09-21T07:44:36.530 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/1326711417 2024-09-22T07:43:26.076277+0000 2024-09-21T07:44:36.530 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/1326711417 2024-09-22T07:43:26.076277+0000 2024-09-21T07:44:36.530 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/364657398 2024-09-22T07:39:15.579613+0000 2024-09-21T07:44:36.530 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.1:0/177249410 2024-09-21T08:41:13.882767+0000 2024-09-21T07:44:36.530 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/73220122 2024-09-22T07:42:40.074859+0000 2024-09-21T07:44:36.531 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/596665086 2024-09-21T08:43:35.317820+0000 2024-09-21T07:44:36.531 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/829716143 2024-09-22T07:37:01.259219+0000 2024-09-21T07:44:36.531 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/855150775 2024-09-22T07:39:07.739045+0000 2024-09-21T07:44:36.531 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/2381504293 2024-09-22T07:38:32.207414+0000 2024-09-21T07:44:36.531 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/855150775 2024-09-22T07:39:07.739045+0000 2024-09-21T07:44:36.531 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/742594435 2024-09-21T08:42:01.225749+0000 2024-09-21T07:44:36.531 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/2821446711 2024-09-22T07:40:57.372409+0000 2024-09-21T07:44:36.531 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.1:0/90671776 2024-09-21T08:39:22.988045+0000 2024-09-21T07:44:36.531 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/759792812 2024-09-22T07:44:05.615021+0000 2024-09-21T07:44:36.532 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/1472430787 2024-09-22T07:37:58.212892+0000 2024-09-21T07:44:36.532 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/3695489158 2024-09-22T07:41:51.807018+0000 2024-09-21T07:44:36.532 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/3026827614 2024-09-22T07:37:29.781472+0000 2024-09-21T07:44:36.532 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/3026827614 2024-09-22T07:37:29.781472+0000 2024-09-21T07:44:36.532 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/829716143 2024-09-22T07:37:01.259219+0000 2024-09-21T07:44:36.532 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/3695489158 2024-09-22T07:41:51.807018+0000 2024-09-21T07:44:36.532 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/1472430787 2024-09-22T07:37:58.212892+0000 2024-09-21T07:44:36.532 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/759792812 2024-09-22T07:44:05.615021+0000 2024-09-21T07:44:36.532 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/992188984 2024-09-22T07:40:00.778049+0000 2024-09-21T07:44:36.532 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6833/364657398 2024-09-22T07:39:15.579613+0000 2024-09-21T07:44:36.532 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/992188984 2024-09-22T07:40:00.778049+0000 2024-09-21T07:44:36.532 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6833/1790581246 2024-09-22T07:44:24.184881+0000 2024-09-21T07:44:36.532 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6833/1456343023 2024-09-22T07:40:08.637993+0000 2024-09-21T07:44:36.532 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/2381504293 2024-09-22T07:38:32.207414+0000 2024-09-21T07:44:36.533 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/2336112936 2024-09-22T07:41:05.979253+0000 2024-09-21T07:44:36.533 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/1456343023 2024-09-22T07:40:08.637993+0000 2024-09-21T07:44:36.533 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/2821446711 2024-09-22T07:40:57.372409+0000 2024-09-21T07:44:36.533 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/3356393880 2024-09-21T08:40:15.824973+0000 2024-09-21T07:44:36.533 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/2336112936 2024-09-22T07:41:05.979253+0000 2024-09-21T07:44:36.533 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/73220122 2024-09-22T07:42:40.074859+0000 2024-09-21T07:44:36.533 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1669604593 2024-09-21T08:42:52.059256+0000 2024-09-21T07:44:36.534 INFO:teuthology.orchestra.run.smithi039.stderr:listed 37 entries 2024-09-21T07:44:36.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.529+0000 7fd9fc000640 1 -- 172.21.15.39:0/228854901 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd9d4078bc0 msgr2=0x7fd9d407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:36.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.529+0000 7fd9fc000640 1 --2- 172.21.15.39:0/228854901 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd9d4078bc0 0x7fd9d407b0d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd9f40b21b0 tx=0x7fd9e0009040 comp rx=0 tx=0).stop 2024-09-21T07:44:36.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.529+0000 7fd9fc000640 1 -- 172.21.15.39:0/228854901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9f40cd780 msgr2=0x7fd9f41e7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:36.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.530+0000 7fd9fc000640 1 --2- 172.21.15.39:0/228854901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9f40cd780 0x7fd9f41e7530 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fd9e8009fd0 tx=0x7fd9e800ec20 comp rx=0 tx=0).stop 2024-09-21T07:44:36.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.530+0000 7fd9fc000640 1 -- 172.21.15.39:0/228854901 shutdown_connections 2024-09-21T07:44:36.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.530+0000 7fd9fc000640 1 --2- 172.21.15.39:0/228854901 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd9d4078bc0 0x7fd9d407b0d0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:36.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.530+0000 7fd9fc000640 1 --2- 172.21.15.39:0/228854901 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd9f415a460 0x7fd9f41364e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:36.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.530+0000 7fd9fc000640 1 --2- 172.21.15.39:0/228854901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9f40d0dd0 0x7fd9f4159280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:36.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.530+0000 7fd9fc000640 1 --2- 172.21.15.39:0/228854901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9f40cd780 0x7fd9f41e7530 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:36.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.530+0000 7fd9fc000640 1 -- 172.21.15.39:0/228854901 >> 172.21.15.39:0/228854901 conn(0x7fd9f406cfc0 msgr2=0x7fd9f40b4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:36.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.530+0000 7fd9fc000640 1 -- 172.21.15.39:0/228854901 shutdown_connections 2024-09-21T07:44:36.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.530+0000 7fd9fc000640 1 -- 172.21.15.39:0/228854901 wait complete. 2024-09-21T07:44:36.556 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T07:44:36.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:44:36.556 DEBUG:teuthology.orchestra.run.smithi039:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T07:44:36.619 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T07:44:36.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:44:36.619 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:44:36.655 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:44:36.655 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:44:36.655 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:44:36.693 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:44:36.693 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:44:36.693 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:44:36.694 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:44:36.733 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:44:36.733 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:44:36.733 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:44:36.733 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:44:36.770 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:44:36.771 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:44:37.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.997+0000 7f21f6c00640 1 -- 172.21.15.39:0/4248683506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21f006fc10 msgr2=0x7f21f00d0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:37.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.997+0000 7f21f6c00640 1 --2- 172.21.15.39:0/4248683506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21f006fc10 0x7f21f00d0620 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f21f00d1240 tx=0x7f21e802aa30 comp rx=0 tx=0).stop 2024-09-21T07:44:37.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.998+0000 7f21f6c00640 1 -- 172.21.15.39:0/4248683506 shutdown_connections 2024-09-21T07:44:37.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.998+0000 7f21f6c00640 1 --2- 172.21.15.39:0/4248683506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21f006fc10 0x7f21f00d0620 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:37.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.998+0000 7f21f6c00640 1 --2- 172.21.15.39:0/4248683506 >> v2:172.21.15.148:3300/0 conn(0x7f21f015cd30 0x7f21f00cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:37.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.998+0000 7f21f6c00640 1 -- 172.21.15.39:0/4248683506 >> 172.21.15.39:0/4248683506 conn(0x7f21f006cfc0 msgr2=0x7f21f006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:37.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.998+0000 7f21f6c00640 1 -- 172.21.15.39:0/4248683506 shutdown_connections 2024-09-21T07:44:37.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:36.998+0000 7f21f6c00640 1 -- 172.21.15.39:0/4248683506 wait complete. 2024-09-21T07:44:37.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.000+0000 7f21f6c00640 1 Processor -- start 2024-09-21T07:44:37.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.000+0000 7f21f6c00640 1 -- start start 2024-09-21T07:44:37.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.001+0000 7f21f6c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21f006f800 0x7f21f014d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:37.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.001+0000 7f21f6c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21f006fc10 0x7f21f014e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:37.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.001+0000 7f21f6c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f21f015cd30 0x7f21f01f6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:37.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.001+0000 7f21f6c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f21f0073890 con 0x7f21f006f800 2024-09-21T07:44:37.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.001+0000 7f21f6c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f21f0073710 con 0x7f21f006fc10 2024-09-21T07:44:37.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.001+0000 7f21f6c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f21f0073a10 con 0x7f21f015cd30 2024-09-21T07:44:37.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.002+0000 7f21efe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21f006f800 0x7f21f014d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:37.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.002+0000 7f21efe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21f006f800 0x7f21f014d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54484/0 (socket says 172.21.15.39:54484) 2024-09-21T07:44:37.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.002+0000 7f21efe00640 1 -- 172.21.15.39:0/882639002 learned_addr learned my addr 172.21.15.39:0/882639002 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:37.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.002+0000 7f21e7400640 1 --2- 172.21.15.39:0/882639002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21f006fc10 0x7f21f014e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:37.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.002+0000 7f21f4c00640 1 --2- 172.21.15.39:0/882639002 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f21f015cd30 0x7f21f01f6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:37.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.002+0000 7f21efe00640 1 -- 172.21.15.39:0/882639002 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f21f015cd30 msgr2=0x7f21f01f6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:37.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.002+0000 7f21efe00640 1 --2- 172.21.15.39:0/882639002 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f21f015cd30 0x7f21f01f6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:37.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.002+0000 7f21efe00640 1 -- 172.21.15.39:0/882639002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21f006fc10 msgr2=0x7f21f014e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:37.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.002+0000 7f21efe00640 1 --2- 172.21.15.39:0/882639002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21f006fc10 0x7f21f014e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:37.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.002+0000 7f21efe00640 1 -- 172.21.15.39:0/882639002 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f01e66c0 con 0x7f21f006f800 2024-09-21T07:44:37.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.002+0000 7f21e7400640 1 --2- 172.21.15.39:0/882639002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21f006fc10 0x7f21f014e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:44:37.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.002+0000 7f21efe00640 1 --2- 172.21.15.39:0/882639002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21f006f800 0x7f21f014d510 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f21e000d7a0 tx=0x7f21e000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:37.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.003+0000 7f21ecc00640 1 -- 172.21.15.39:0/882639002 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0017070 con 0x7f21f006f800 2024-09-21T07:44:37.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.003+0000 7f21ecc00640 1 -- 172.21.15.39:0/882639002 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21e0004310 con 0x7f21f006f800 2024-09-21T07:44:37.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.003+0000 7f21f6c00640 1 -- 172.21.15.39:0/882639002 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21f01e6980 con 0x7f21f006f800 2024-09-21T07:44:37.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.003+0000 7f21ecc00640 1 -- 172.21.15.39:0/882639002 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0009c30 con 0x7f21f006f800 2024-09-21T07:44:37.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.003+0000 7f21f6c00640 1 -- 172.21.15.39:0/882639002 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21f01f7a70 con 0x7f21f006f800 2024-09-21T07:44:37.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.004+0000 7f21f6c00640 1 -- 172.21.15.39:0/882639002 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21f0073f30 con 0x7f21f006f800 2024-09-21T07:44:37.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.005+0000 7f21ecc00640 1 -- 172.21.15.39:0/882639002 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f21e0005020 con 0x7f21f006f800 2024-09-21T07:44:37.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.006+0000 7f21ecc00640 1 --2- 172.21.15.39:0/882639002 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f21cc078bc0 0x7f21cc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:37.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.006+0000 7f21ecc00640 1 -- 172.21.15.39:0/882639002 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8420+0+0 (secure 0 0 0) 0x7f21e009bad0 con 0x7f21f006f800 2024-09-21T07:44:37.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.006+0000 7f21e7400640 1 --2- 172.21.15.39:0/882639002 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f21cc078bc0 0x7f21cc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:37.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.007+0000 7f21e7400640 1 --2- 172.21.15.39:0/882639002 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f21cc078bc0 0x7f21cc07b0d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f21f00cca50 tx=0x7f21d8009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:37.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.010+0000 7f21ecc00640 1 -- 172.21.15.39:0/882639002 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f21e0020020 con 0x7f21f006f800 2024-09-21T07:44:37.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.242+0000 7f21f6c00640 1 -- 172.21.15.39:0/882639002 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f21f00cd320 con 0x7f21f006f800 2024-09-21T07:44:37.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.244+0000 7f21ecc00640 1 -- 172.21.15.39:0/882639002 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 164 v164) ==== 95+0+5252 (secure 0 0 0) 0x7f21e00634e0 con 0x7f21f006f800 2024-09-21T07:44:37.246 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:37.246 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":164,"btime":"2024-09-21T07:44:28:210229+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":8268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/67641510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":67641510},{"type":"v1","addr":"172.21.15.39:6833","nonce":67641510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":161}],"filesystems":[{"mdsmap":{"epoch":164,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:44:28.210222+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6929,"mds_1":8052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6929":{"gid":6929,"name":"d","rank":0,"incarnation":160,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2582569093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2582569093},{"type":"v1","addr":"172.21.15.148:6837","nonce":2582569093}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8052":{"gid":8052,"name":"a","rank":1,"incarnation":155,"state":"up:active","state_seq":18,"addr":"172.21.15.39:6835/1551311805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1551311805},{"type":"v1","addr":"172.21.15.39:6835","nonce":1551311805}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6929,"qdb_cluster":[6929,8052]},"id":6}]} 2024-09-21T07:44:37.246 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 164 2024-09-21T07:44:37.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.246+0000 7f21f6c00640 1 -- 172.21.15.39:0/882639002 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f21cc078bc0 msgr2=0x7f21cc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:37.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.246+0000 7f21f6c00640 1 --2- 172.21.15.39:0/882639002 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f21cc078bc0 0x7f21cc07b0d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f21f00cca50 tx=0x7f21d8009040 comp rx=0 tx=0).stop 2024-09-21T07:44:37.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.246+0000 7f21f6c00640 1 -- 172.21.15.39:0/882639002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21f006f800 msgr2=0x7f21f014d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:37.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.246+0000 7f21f6c00640 1 --2- 172.21.15.39:0/882639002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21f006f800 0x7f21f014d510 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f21e000d7a0 tx=0x7f21e000dc60 comp rx=0 tx=0).stop 2024-09-21T07:44:37.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.246+0000 7f21f6c00640 1 -- 172.21.15.39:0/882639002 shutdown_connections 2024-09-21T07:44:37.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.246+0000 7f21f6c00640 1 --2- 172.21.15.39:0/882639002 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f21cc078bc0 0x7f21cc07b0d0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:37.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.246+0000 7f21f6c00640 1 --2- 172.21.15.39:0/882639002 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f21f015cd30 0x7f21f01f6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:37.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.246+0000 7f21f6c00640 1 --2- 172.21.15.39:0/882639002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21f006fc10 0x7f21f014e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:37.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.246+0000 7f21f6c00640 1 --2- 172.21.15.39:0/882639002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21f006f800 0x7f21f014d510 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:37.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.246+0000 7f21f6c00640 1 -- 172.21.15.39:0/882639002 >> 172.21.15.39:0/882639002 conn(0x7f21f006cfc0 msgr2=0x7f21f00b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:37.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.247+0000 7f21f6c00640 1 -- 172.21.15.39:0/882639002 shutdown_connections 2024-09-21T07:44:37.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.247+0000 7f21f6c00640 1 -- 172.21.15.39:0/882639002 wait complete. 2024-09-21T07:44:37.274 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:44:37.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.498+0000 7f8b2a000640 1 -- 172.21.15.39:0/2004848037 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b2406f800 msgr2=0x7f8b240cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:37.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.498+0000 7f8b2a000640 1 --2- 172.21.15.39:0/2004848037 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b2406f800 0x7f8b240cd8e0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f8b240d1240 tx=0x7f8b0c02a4a0 comp rx=0 tx=0).stop 2024-09-21T07:44:37.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.498+0000 7f8b2a000640 1 -- 172.21.15.39:0/2004848037 shutdown_connections 2024-09-21T07:44:37.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.498+0000 7f8b2a000640 1 --2- 172.21.15.39:0/2004848037 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b2406f800 0x7f8b240cd8e0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:37.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.498+0000 7f8b2a000640 1 --2- 172.21.15.39:0/2004848037 >> v2:172.21.15.148:3300/0 conn(0x7f8b2415cd30 0x7f8b240cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:37.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.498+0000 7f8b2a000640 1 -- 172.21.15.39:0/2004848037 >> 172.21.15.39:0/2004848037 conn(0x7f8b2406cfc0 msgr2=0x7f8b2406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:37.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.498+0000 7f8b2a000640 1 -- 172.21.15.39:0/2004848037 shutdown_connections 2024-09-21T07:44:37.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.498+0000 7f8b2a000640 1 -- 172.21.15.39:0/2004848037 wait complete. 2024-09-21T07:44:37.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.500+0000 7f8b2a000640 1 Processor -- start 2024-09-21T07:44:37.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.500+0000 7f8b2a000640 1 -- start start 2024-09-21T07:44:37.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.502+0000 7f8b2a000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b2406f800 0x7f8b240b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:37.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.502+0000 7f8b2a000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b2406fc10 0x7f8b240b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:37.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.502+0000 7f8b2a000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b2415cd30 0x7f8b240b2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:37.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.502+0000 7f8b2a000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8b24073890 con 0x7f8b2406fc10 2024-09-21T07:44:37.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.502+0000 7f8b2a000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8b24073710 con 0x7f8b2406f800 2024-09-21T07:44:37.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.502+0000 7f8b2a000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f8b24073a10 con 0x7f8b2415cd30 2024-09-21T07:44:37.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.502+0000 7f8b1aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b2406fc10 0x7f8b240b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:37.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.502+0000 7f8b1aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b2406fc10 0x7f8b240b1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54490/0 (socket says 172.21.15.39:54490) 2024-09-21T07:44:37.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.502+0000 7f8b1aa00640 1 -- 172.21.15.39:0/3517162366 learned_addr learned my addr 172.21.15.39:0/3517162366 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:37.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.502+0000 7f8b23400640 1 --2- 172.21.15.39:0/3517162366 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b2406f800 0x7f8b240b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:37.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.502+0000 7f8b23e00640 1 --2- 172.21.15.39:0/3517162366 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b2415cd30 0x7f8b240b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:37.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.503+0000 7f8b1aa00640 1 -- 172.21.15.39:0/3517162366 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b2415cd30 msgr2=0x7f8b240b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:37.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.503+0000 7f8b1aa00640 1 --2- 172.21.15.39:0/3517162366 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b2415cd30 0x7f8b240b2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:37.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.503+0000 7f8b1aa00640 1 -- 172.21.15.39:0/3517162366 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b2406f800 msgr2=0x7f8b240b44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:37.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.503+0000 7f8b1aa00640 1 --2- 172.21.15.39:0/3517162366 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b2406f800 0x7f8b240b44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:37.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.503+0000 7f8b1aa00640 1 -- 172.21.15.39:0/3517162366 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b240b5430 con 0x7f8b2406fc10 2024-09-21T07:44:37.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.503+0000 7f8b23e00640 1 --2- 172.21.15.39:0/3517162366 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b2415cd30 0x7f8b240b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:44:37.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.503+0000 7f8b23400640 1 --2- 172.21.15.39:0/3517162366 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b2406f800 0x7f8b240b44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:44:37.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.503+0000 7f8b1aa00640 1 --2- 172.21.15.39:0/3517162366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b2406fc10 0x7f8b240b1fe0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c02aa90 tx=0x7f8b0c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:37.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.503+0000 7f8b1be00640 1 -- 172.21.15.39:0/3517162366 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c031d60 con 0x7f8b2406fc10 2024-09-21T07:44:37.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.503+0000 7f8b1be00640 1 -- 172.21.15.39:0/3517162366 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b0c033e10 con 0x7f8b2406fc10 2024-09-21T07:44:37.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.503+0000 7f8b1be00640 1 -- 172.21.15.39:0/3517162366 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c005260 con 0x7f8b2406fc10 2024-09-21T07:44:37.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.503+0000 7f8b2a000640 1 -- 172.21.15.39:0/3517162366 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b241f5660 con 0x7f8b2406fc10 2024-09-21T07:44:37.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.503+0000 7f8b2a000640 1 -- 172.21.15.39:0/3517162366 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b241f5c20 con 0x7f8b2406fc10 2024-09-21T07:44:37.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.504+0000 7f8b2a000640 1 -- 172.21.15.39:0/3517162366 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b240b3da0 con 0x7f8b2406fc10 2024-09-21T07:44:37.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.506+0000 7f8b1be00640 1 -- 172.21.15.39:0/3517162366 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8b0c005400 con 0x7f8b2406fc10 2024-09-21T07:44:37.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.507+0000 7f8b1be00640 1 --2- 172.21.15.39:0/3517162366 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8b00078bc0 0x7f8b0007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:37.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.507+0000 7f8b1be00640 1 -- 172.21.15.39:0/3517162366 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8420+0+0 (secure 0 0 0) 0x7f8b0c0bca00 con 0x7f8b2406fc10 2024-09-21T07:44:37.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.507+0000 7f8b23400640 1 --2- 172.21.15.39:0/3517162366 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8b00078bc0 0x7f8b0007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:37.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.508+0000 7f8b23400640 1 --2- 172.21.15.39:0/3517162366 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8b00078bc0 0x7f8b0007b0d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f8b14005fd0 tx=0x7f8b1400f360 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:37.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.511+0000 7f8b1be00640 1 -- 172.21.15.39:0/3517162366 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8b0c03f030 con 0x7f8b2406fc10 2024-09-21T07:44:37.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.749+0000 7f8b2a000640 1 -- 172.21.15.39:0/3517162366 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b24073ec0 con 0x7f8b2406fc10 2024-09-21T07:44:37.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.749+0000 7f8b1be00640 1 -- 172.21.15.39:0/3517162366 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 164 v164) ==== 95+0+5252 (secure 0 0 0) 0x7f8b0c005d00 con 0x7f8b2406fc10 2024-09-21T07:44:37.751 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:37.751 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":164,"btime":"2024-09-21T07:44:28:210229+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":8268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/67641510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":67641510},{"type":"v1","addr":"172.21.15.39:6833","nonce":67641510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":161}],"filesystems":[{"mdsmap":{"epoch":164,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:44:28.210222+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6929,"mds_1":8052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6929":{"gid":6929,"name":"d","rank":0,"incarnation":160,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2582569093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2582569093},{"type":"v1","addr":"172.21.15.148:6837","nonce":2582569093}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8052":{"gid":8052,"name":"a","rank":1,"incarnation":155,"state":"up:active","state_seq":18,"addr":"172.21.15.39:6835/1551311805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1551311805},{"type":"v1","addr":"172.21.15.39:6835","nonce":1551311805}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6929,"qdb_cluster":[6929,8052]},"id":6}]} 2024-09-21T07:44:37.752 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 164 2024-09-21T07:44:37.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.751+0000 7f8b2a000640 1 -- 172.21.15.39:0/3517162366 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8b00078bc0 msgr2=0x7f8b0007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:37.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.751+0000 7f8b2a000640 1 --2- 172.21.15.39:0/3517162366 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8b00078bc0 0x7f8b0007b0d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f8b14005fd0 tx=0x7f8b1400f360 comp rx=0 tx=0).stop 2024-09-21T07:44:37.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.751+0000 7f8b2a000640 1 -- 172.21.15.39:0/3517162366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b2406fc10 msgr2=0x7f8b240b1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:37.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.751+0000 7f8b2a000640 1 --2- 172.21.15.39:0/3517162366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b2406fc10 0x7f8b240b1fe0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c02aa90 tx=0x7f8b0c033c20 comp rx=0 tx=0).stop 2024-09-21T07:44:37.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.752+0000 7f8b2a000640 1 -- 172.21.15.39:0/3517162366 shutdown_connections 2024-09-21T07:44:37.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.752+0000 7f8b2a000640 1 --2- 172.21.15.39:0/3517162366 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8b00078bc0 0x7f8b0007b0d0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:37.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.752+0000 7f8b2a000640 1 --2- 172.21.15.39:0/3517162366 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b2415cd30 0x7f8b240b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:37.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.752+0000 7f8b2a000640 1 --2- 172.21.15.39:0/3517162366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b2406fc10 0x7f8b240b1fe0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:37.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.752+0000 7f8b2a000640 1 --2- 172.21.15.39:0/3517162366 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b2406f800 0x7f8b240b44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:37.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.752+0000 7f8b2a000640 1 -- 172.21.15.39:0/3517162366 >> 172.21.15.39:0/3517162366 conn(0x7f8b2406cfc0 msgr2=0x7f8b2415b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:37.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.752+0000 7f8b2a000640 1 -- 172.21.15.39:0/3517162366 shutdown_connections 2024-09-21T07:44:37.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:37.752+0000 7f8b2a000640 1 -- 172.21.15.39:0/3517162366 wait complete. 2024-09-21T07:44:37.776 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:44:38.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.002+0000 7f7c52600640 1 -- 172.21.15.39:0/1942054696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c4c0d0f20 msgr2=0x7f7c4c15c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:38.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.002+0000 7f7c52600640 1 --2- 172.21.15.39:0/1942054696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c4c0d0f20 0x7f7c4c15c830 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f7c40005fd0 tx=0x7f7c4002a5d0 comp rx=0 tx=0).stop 2024-09-21T07:44:38.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.002+0000 7f7c52600640 1 -- 172.21.15.39:0/1942054696 shutdown_connections 2024-09-21T07:44:38.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.002+0000 7f7c52600640 1 --2- 172.21.15.39:0/1942054696 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c4c0cb960 0x7f7c4c0cbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:38.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.002+0000 7f7c52600640 1 --2- 172.21.15.39:0/1942054696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c4c0d0f20 0x7f7c4c15c830 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:38.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.002+0000 7f7c52600640 1 -- 172.21.15.39:0/1942054696 >> 172.21.15.39:0/1942054696 conn(0x7f7c4c0b2510 msgr2=0x7f7c4c0d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:38.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.002+0000 7f7c52600640 1 -- 172.21.15.39:0/1942054696 shutdown_connections 2024-09-21T07:44:38.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.002+0000 7f7c52600640 1 -- 172.21.15.39:0/1942054696 wait complete. 2024-09-21T07:44:38.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.004+0000 7f7c52600640 1 Processor -- start 2024-09-21T07:44:38.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.004+0000 7f7c52600640 1 -- start start 2024-09-21T07:44:38.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.006+0000 7f7c52600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c4c06f8a0 0x7f7c4c0af570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:38.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.006+0000 7f7c52600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c4c0cb960 0x7f7c4c155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:38.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.006+0000 7f7c52600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c4c0d0f20 0x7f7c4c0ae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:38.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.006+0000 7f7c52600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c0b4570 con 0x7f7c4c0d0f20 2024-09-21T07:44:38.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.006+0000 7f7c52600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c0b43f0 con 0x7f7c4c0cb960 2024-09-21T07:44:38.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.006+0000 7f7c52600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f7c4c0b46f0 con 0x7f7c4c06f8a0 2024-09-21T07:44:38.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.006+0000 7f7c51c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c4c0d0f20 0x7f7c4c0ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:38.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.006+0000 7f7c51c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c4c0d0f20 0x7f7c4c0ae100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54512/0 (socket says 172.21.15.39:54512) 2024-09-21T07:44:38.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.006+0000 7f7c51c00640 1 -- 172.21.15.39:0/1531118902 learned_addr learned my addr 172.21.15.39:0/1531118902 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:38.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.006+0000 7f7c4be00640 1 --2- 172.21.15.39:0/1531118902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c4c0cb960 0x7f7c4c155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:38.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.006+0000 7f7c51200640 1 --2- 172.21.15.39:0/1531118902 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c4c06f8a0 0x7f7c4c0af570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:38.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.007+0000 7f7c51c00640 1 -- 172.21.15.39:0/1531118902 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c4c06f8a0 msgr2=0x7f7c4c0af570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:38.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.007+0000 7f7c51c00640 1 --2- 172.21.15.39:0/1531118902 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c4c06f8a0 0x7f7c4c0af570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:38.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.007+0000 7f7c51c00640 1 -- 172.21.15.39:0/1531118902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c4c0cb960 msgr2=0x7f7c4c155e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:38.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.007+0000 7f7c51c00640 1 --2- 172.21.15.39:0/1531118902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c4c0cb960 0x7f7c4c155e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:38.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.007+0000 7f7c51c00640 1 -- 172.21.15.39:0/1531118902 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c4c156660 con 0x7f7c4c0d0f20 2024-09-21T07:44:38.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.007+0000 7f7c51c00640 1 --2- 172.21.15.39:0/1531118902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c4c0d0f20 0x7f7c4c0ae100 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f7c4400c730 tx=0x7f7c4400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:38.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.007+0000 7f7c48c00640 1 -- 172.21.15.39:0/1531118902 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c44003b70 con 0x7f7c4c0d0f20 2024-09-21T07:44:38.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.007+0000 7f7c48c00640 1 -- 172.21.15.39:0/1531118902 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c44003d10 con 0x7f7c4c0d0f20 2024-09-21T07:44:38.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.007+0000 7f7c52600640 1 -- 172.21.15.39:0/1531118902 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c4c156920 con 0x7f7c4c0d0f20 2024-09-21T07:44:38.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.007+0000 7f7c48c00640 1 -- 172.21.15.39:0/1531118902 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c44005330 con 0x7f7c4c0d0f20 2024-09-21T07:44:38.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.007+0000 7f7c52600640 1 -- 172.21.15.39:0/1531118902 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c4c156ee0 con 0x7f7c4c0d0f20 2024-09-21T07:44:38.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.008+0000 7f7c52600640 1 -- 172.21.15.39:0/1531118902 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c4c0b4c50 con 0x7f7c4c0d0f20 2024-09-21T07:44:38.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.010+0000 7f7c48c00640 1 -- 172.21.15.39:0/1531118902 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7c440056e0 con 0x7f7c4c0d0f20 2024-09-21T07:44:38.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.011+0000 7f7c48c00640 1 --2- 172.21.15.39:0/1531118902 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7c2c078b90 0x7f7c2c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:38.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.011+0000 7f7c48c00640 1 -- 172.21.15.39:0/1531118902 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8420+0+0 (secure 0 0 0) 0x7f7c4409b6b0 con 0x7f7c4c0d0f20 2024-09-21T07:44:38.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.011+0000 7f7c51200640 1 --2- 172.21.15.39:0/1531118902 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7c2c078b90 0x7f7c2c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:38.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.012+0000 7f7c51200640 1 --2- 172.21.15.39:0/1531118902 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7c2c078b90 0x7f7c2c07b0a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f7c40033000 tx=0x7f7c40004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:38.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.015+0000 7f7c48c00640 1 -- 172.21.15.39:0/1531118902 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7c44062950 con 0x7f7c4c0d0f20 2024-09-21T07:44:38.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.213+0000 7f7c52600640 1 -- 172.21.15.39:0/1531118902 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7c4c156390 con 0x7f7c4c0d0f20 2024-09-21T07:44:38.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.215+0000 7f7c48c00640 1 -- 172.21.15.39:0/1531118902 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) ==== 74+0+24569 (secure 0 0 0) 0x7f7c44067380 con 0x7f7c4c0d0f20 2024-09-21T07:44:38.217 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:38.218 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":123,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:44:24.184919+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:36:18.142953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-21T07:36:19.149503+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":119,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.22","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6832/1790581246":"2024-09-22T07:44:24.184881+0000","172.21.15.39:0/1731243684":"2024-09-21T08:44:16.323688+0000","172.21.15.39:6835/1326711417":"2024-09-22T07:43:26.076277+0000","172.21.15.39:6834/1326711417":"2024-09-22T07:43:26.076277+0000","172.21.15.39:6835/364657398":"2024-09-22T07:39:15.579613+0000","192.168.0.1:0/177249410":"2024-09-21T08:41:13.882767+0000","172.21.15.148:6837/73220122":"2024-09-22T07:42:40.074859+0000","172.21.15.39:0/596665086":"2024-09-21T08:43:35.317820+0000","172.21.15.39:6832/829716143":"2024-09-22T07:37:01.259219+0000","172.21.15.39:6832/855150775":"2024-09-22T07:39:07.739045+0000","172.21.15.148:6837/2381504293":"2024-09-22T07:38:32.207414+0000","172.21.15.39:6834/855150775":"2024-09-22T07:39:07.739045+0000","172.21.15.39:0/742594435":"2024-09-21T08:42:01.225749+0000","172.21.15.39:6835/2821446711":"2024-09-22T07:40:57.372409+0000","192.168.0.1:0/90671776":"2024-09-21T08:39:22.988045+0000","172.21.15.148:6836/759792812":"2024-09-22T07:44:05.615021+0000","172.21.15.39:6834/1472430787":"2024-09-22T07:37:58.212892+0000","172.21.15.39:6834/3695489158":"2024-09-22T07:41:51.807018+0000","172.21.15.148:6836/3026827614":"2024-09-22T07:37:29.781472+0000","172.21.15.148:6837/3026827614":"2024-09-22T07:37:29.781472+0000","172.21.15.39:6834/829716143":"2024-09-22T07:37:01.259219+0000","172.21.15.39:6835/3695489158":"2024-09-22T07:41:51.807018+0000","172.21.15.39:6832/1472430787":"2024-09-22T07:37:58.212892+0000","172.21.15.148:6837/759792812":"2024-09-22T07:44:05.615021+0000","172.21.15.148:6836/992188984":"2024-09-22T07:40:00.778049+0000","172.21.15.39:6833/364657398":"2024-09-22T07:39:15.579613+0000","172.21.15.148:6837/992188984":"2024-09-22T07:40:00.778049+0000","172.21.15.39:6833/1790581246":"2024-09-22T07:44:24.184881+0000","172.21.15.39:6833/1456343023":"2024-09-22T07:40:08.637993+0000","172.21.15.148:6836/2381504293":"2024-09-22T07:38:32.207414+0000","172.21.15.148:6836/2336112936":"2024-09-22T07:41:05.979253+0000","172.21.15.39:6832/1456343023":"2024-09-22T07:40:08.637993+0000","172.21.15.39:6834/2821446711":"2024-09-22T07:40:57.372409+0000","172.21.15.39:0/3356393880":"2024-09-21T08:40:15.824973+0000","172.21.15.148:6837/2336112936":"2024-09-22T07:41:05.979253+0000","172.21.15.148:6836/73220122":"2024-09-22T07:42:40.074859+0000","172.21.15.39:0/1669604593":"2024-09-21T08:42:52.059256+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:44:38.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.217+0000 7f7c52600640 1 -- 172.21.15.39:0/1531118902 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7c2c078b90 msgr2=0x7f7c2c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:38.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.217+0000 7f7c52600640 1 --2- 172.21.15.39:0/1531118902 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7c2c078b90 0x7f7c2c07b0a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f7c40033000 tx=0x7f7c40004730 comp rx=0 tx=0).stop 2024-09-21T07:44:38.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.217+0000 7f7c52600640 1 -- 172.21.15.39:0/1531118902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c4c0d0f20 msgr2=0x7f7c4c0ae100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:38.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.217+0000 7f7c52600640 1 --2- 172.21.15.39:0/1531118902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c4c0d0f20 0x7f7c4c0ae100 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f7c4400c730 tx=0x7f7c4400cbf0 comp rx=0 tx=0).stop 2024-09-21T07:44:38.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.218+0000 7f7c52600640 1 -- 172.21.15.39:0/1531118902 shutdown_connections 2024-09-21T07:44:38.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.218+0000 7f7c52600640 1 --2- 172.21.15.39:0/1531118902 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7c2c078b90 0x7f7c2c07b0a0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:38.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.218+0000 7f7c52600640 1 --2- 172.21.15.39:0/1531118902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c4c0d0f20 0x7f7c4c0ae100 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:38.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.218+0000 7f7c52600640 1 --2- 172.21.15.39:0/1531118902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c4c0cb960 0x7f7c4c155e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:38.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.218+0000 7f7c52600640 1 --2- 172.21.15.39:0/1531118902 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c4c06f8a0 0x7f7c4c0af570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:38.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.218+0000 7f7c52600640 1 -- 172.21.15.39:0/1531118902 >> 172.21.15.39:0/1531118902 conn(0x7f7c4c0b2510 msgr2=0x7f7c4c0ca190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:38.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.218+0000 7f7c52600640 1 -- 172.21.15.39:0/1531118902 shutdown_connections 2024-09-21T07:44:38.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.218+0000 7f7c52600640 1 -- 172.21.15.39:0/1531118902 wait complete. 2024-09-21T07:44:38.241 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T07:44:38.241 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:44:38.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.466+0000 7fd10ae00640 1 -- 172.21.15.39:0/359581025 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd10415cd30 msgr2=0x7fd1040cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:38.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.466+0000 7fd10ae00640 1 --2- 172.21.15.39:0/359581025 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd10415cd30 0x7fd1040cfd10 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4009830 tx=0x7fd0f402a530 comp rx=0 tx=0).stop 2024-09-21T07:44:38.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.466+0000 7fd10ae00640 1 -- 172.21.15.39:0/359581025 shutdown_connections 2024-09-21T07:44:38.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.466+0000 7fd10ae00640 1 --2- 172.21.15.39:0/359581025 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd10415cd30 0x7fd1040cfd10 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:38.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.466+0000 7fd10ae00640 1 -- 172.21.15.39:0/359581025 >> 172.21.15.39:0/359581025 conn(0x7fd10406cfc0 msgr2=0x7fd10406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:38.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.467+0000 7fd10ae00640 1 -- 172.21.15.39:0/359581025 shutdown_connections 2024-09-21T07:44:38.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.467+0000 7fd10ae00640 1 -- 172.21.15.39:0/359581025 wait complete. 2024-09-21T07:44:38.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.469+0000 7fd10ae00640 1 Processor -- start 2024-09-21T07:44:38.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.469+0000 7fd10ae00640 1 -- start start 2024-09-21T07:44:38.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.470+0000 7fd10ae00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd10406f800 0x7fd104155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:38.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.470+0000 7fd10ae00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd10406fc10 0x7fd1040a9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:38.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.470+0000 7fd10ae00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd10415cd30 0x7fd104151a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:38.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.470+0000 7fd10ae00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd104073aa0 con 0x7fd10415cd30 2024-09-21T07:44:38.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.470+0000 7fd10ae00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd104073920 con 0x7fd10406fc10 2024-09-21T07:44:38.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.470+0000 7fd10ae00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd104073c20 con 0x7fd10406f800 2024-09-21T07:44:38.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.471+0000 7fd108e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd10415cd30 0x7fd104151a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:38.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.471+0000 7fd108e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd10415cd30 0x7fd104151a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54534/0 (socket says 172.21.15.39:54534) 2024-09-21T07:44:38.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.471+0000 7fd108e00640 1 -- 172.21.15.39:0/4259596813 learned_addr learned my addr 172.21.15.39:0/4259596813 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:38.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.471+0000 7fd103e00640 1 --2- 172.21.15.39:0/4259596813 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd10406f800 0x7fd104155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:38.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.471+0000 7fd0fb400640 1 --2- 172.21.15.39:0/4259596813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd10406fc10 0x7fd1040a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:38.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.471+0000 7fd108e00640 1 -- 172.21.15.39:0/4259596813 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd10406f800 msgr2=0x7fd104155a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:38.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.471+0000 7fd108e00640 1 --2- 172.21.15.39:0/4259596813 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd10406f800 0x7fd104155a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:38.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.471+0000 7fd108e00640 1 -- 172.21.15.39:0/4259596813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd10406fc10 msgr2=0x7fd1040a9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:38.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.471+0000 7fd108e00640 1 --2- 172.21.15.39:0/4259596813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd10406fc10 0x7fd1040a9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:38.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.471+0000 7fd108e00640 1 -- 172.21.15.39:0/4259596813 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1040af8d0 con 0x7fd10415cd30 2024-09-21T07:44:38.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.472+0000 7fd108e00640 1 --2- 172.21.15.39:0/4259596813 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd10415cd30 0x7fd104151a20 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc00c730 tx=0x7fd0fc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:38.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.472+0000 7fd100c00640 1 -- 172.21.15.39:0/4259596813 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc003b70 con 0x7fd10415cd30 2024-09-21T07:44:38.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.472+0000 7fd100c00640 1 -- 172.21.15.39:0/4259596813 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd0fc003d30 con 0x7fd10415cd30 2024-09-21T07:44:38.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.472+0000 7fd10ae00640 1 -- 172.21.15.39:0/4259596813 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1041f5930 con 0x7fd10415cd30 2024-09-21T07:44:38.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.472+0000 7fd100c00640 1 -- 172.21.15.39:0/4259596813 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc0052d0 con 0x7fd10415cd30 2024-09-21T07:44:38.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.472+0000 7fd10ae00640 1 -- 172.21.15.39:0/4259596813 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1040af5f0 con 0x7fd10415cd30 2024-09-21T07:44:38.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.473+0000 7fd10ae00640 1 -- 172.21.15.39:0/4259596813 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd104157000 con 0x7fd10415cd30 2024-09-21T07:44:38.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.474+0000 7fd100c00640 1 -- 172.21.15.39:0/4259596813 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd0fc005680 con 0x7fd10415cd30 2024-09-21T07:44:38.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.475+0000 7fd100c00640 1 --2- 172.21.15.39:0/4259596813 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd0e0078b90 0x7fd0e007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:38.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.475+0000 7fd100c00640 1 -- 172.21.15.39:0/4259596813 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8420+0+0 (secure 0 0 0) 0x7fd0fc09b880 con 0x7fd10415cd30 2024-09-21T07:44:38.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.476+0000 7fd103e00640 1 --2- 172.21.15.39:0/4259596813 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd0e0078b90 0x7fd0e007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:38.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.477+0000 7fd103e00640 1 --2- 172.21.15.39:0/4259596813 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd0e0078b90 0x7fd0e007b0a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4034000 tx=0x7fd0f4004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:38.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.480+0000 7fd100c00640 1 -- 172.21.15.39:0/4259596813 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd104157000 con 0x7fd10415cd30 2024-09-21T07:44:38.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.715+0000 7fd10ae00640 1 -- 172.21.15.39:0/4259596813 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1040740a0 con 0x7fd10415cd30 2024-09-21T07:44:38.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.715+0000 7fd100c00640 1 -- 172.21.15.39:0/4259596813 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 164 v164) ==== 95+0+5252 (secure 0 0 0) 0x7fd0fc063200 con 0x7fd10415cd30 2024-09-21T07:44:38.717 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:38.717 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":164,"btime":"2024-09-21T07:44:28:210229+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":8268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/67641510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":67641510},{"type":"v1","addr":"172.21.15.39:6833","nonce":67641510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":161}],"filesystems":[{"mdsmap":{"epoch":164,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:44:28.210222+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6929,"mds_1":8052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6929":{"gid":6929,"name":"d","rank":0,"incarnation":160,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2582569093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2582569093},{"type":"v1","addr":"172.21.15.148:6837","nonce":2582569093}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8052":{"gid":8052,"name":"a","rank":1,"incarnation":155,"state":"up:active","state_seq":18,"addr":"172.21.15.39:6835/1551311805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1551311805},{"type":"v1","addr":"172.21.15.39:6835","nonce":1551311805}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6929,"qdb_cluster":[6929,8052]},"id":6}]} 2024-09-21T07:44:38.718 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 164 2024-09-21T07:44:38.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.718+0000 7fd10ae00640 1 -- 172.21.15.39:0/4259596813 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd0e0078b90 msgr2=0x7fd0e007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:38.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.718+0000 7fd10ae00640 1 --2- 172.21.15.39:0/4259596813 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd0e0078b90 0x7fd0e007b0a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4034000 tx=0x7fd0f4004730 comp rx=0 tx=0).stop 2024-09-21T07:44:38.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.718+0000 7fd10ae00640 1 -- 172.21.15.39:0/4259596813 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd10415cd30 msgr2=0x7fd104151a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:38.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.718+0000 7fd10ae00640 1 --2- 172.21.15.39:0/4259596813 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd10415cd30 0x7fd104151a20 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc00c730 tx=0x7fd0fc00cbf0 comp rx=0 tx=0).stop 2024-09-21T07:44:38.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.718+0000 7fd10ae00640 1 -- 172.21.15.39:0/4259596813 shutdown_connections 2024-09-21T07:44:38.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.718+0000 7fd10ae00640 1 --2- 172.21.15.39:0/4259596813 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd0e0078b90 0x7fd0e007b0a0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:38.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.718+0000 7fd10ae00640 1 --2- 172.21.15.39:0/4259596813 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd10415cd30 0x7fd104151a20 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:38.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.718+0000 7fd10ae00640 1 --2- 172.21.15.39:0/4259596813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd10406fc10 0x7fd1040a9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:38.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.718+0000 7fd10ae00640 1 --2- 172.21.15.39:0/4259596813 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd10406f800 0x7fd104155a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:38.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.718+0000 7fd10ae00640 1 -- 172.21.15.39:0/4259596813 >> 172.21.15.39:0/4259596813 conn(0x7fd10406cfc0 msgr2=0x7fd104068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:38.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.718+0000 7fd10ae00640 1 -- 172.21.15.39:0/4259596813 shutdown_connections 2024-09-21T07:44:38.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.718+0000 7fd10ae00640 1 -- 172.21.15.39:0/4259596813 wait complete. 2024-09-21T07:44:38.742 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:44:38.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.968+0000 7f9c58a00640 1 -- 172.21.15.39:0/821714612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c5415cd30 msgr2=0x7f9c540cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:38.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.968+0000 7f9c58a00640 1 --2- 172.21.15.39:0/821714612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c5415cd30 0x7f9c540cfd10 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f9c44005fd0 tx=0x7f9c4402a5d0 comp rx=0 tx=0).stop 2024-09-21T07:44:38.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.968+0000 7f9c58a00640 1 -- 172.21.15.39:0/821714612 shutdown_connections 2024-09-21T07:44:38.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.968+0000 7f9c58a00640 1 --2- 172.21.15.39:0/821714612 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9c5406f800 0x7f9c540cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:38.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.968+0000 7f9c58a00640 1 --2- 172.21.15.39:0/821714612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c5415cd30 0x7f9c540cfd10 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:38.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.968+0000 7f9c58a00640 1 -- 172.21.15.39:0/821714612 >> 172.21.15.39:0/821714612 conn(0x7f9c5406cfc0 msgr2=0x7f9c5406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:38.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.968+0000 7f9c58a00640 1 -- 172.21.15.39:0/821714612 shutdown_connections 2024-09-21T07:44:38.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.968+0000 7f9c58a00640 1 -- 172.21.15.39:0/821714612 wait complete. 2024-09-21T07:44:38.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.971+0000 7f9c58a00640 1 Processor -- start 2024-09-21T07:44:38.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.971+0000 7f9c58a00640 1 -- start start 2024-09-21T07:44:38.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.972+0000 7f9c58a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c5406f800 0x7f9c541456e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:38.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.972+0000 7f9c58a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c5406fc10 0x7f9c5414fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:38.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.972+0000 7f9c58a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9c5415cd30 0x7f9c54144c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:38.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.972+0000 7f9c58a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9c54073890 con 0x7f9c5406fc10 2024-09-21T07:44:38.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.972+0000 7f9c58a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9c54073710 con 0x7f9c5406f800 2024-09-21T07:44:38.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.972+0000 7f9c58a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f9c54073a10 con 0x7f9c5415cd30 2024-09-21T07:44:38.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.972+0000 7f9c49400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c5406fc10 0x7f9c5414fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:38.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.973+0000 7f9c49400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c5406fc10 0x7f9c5414fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54552/0 (socket says 172.21.15.39:54552) 2024-09-21T07:44:38.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.973+0000 7f9c49400640 1 -- 172.21.15.39:0/682279603 learned_addr learned my addr 172.21.15.39:0/682279603 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:38.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.973+0000 7f9c51e00640 1 --2- 172.21.15.39:0/682279603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c5406f800 0x7f9c541456e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:38.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.973+0000 7f9c52800640 1 --2- 172.21.15.39:0/682279603 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9c5415cd30 0x7f9c54144c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:38.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.973+0000 7f9c49400640 1 -- 172.21.15.39:0/682279603 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9c5415cd30 msgr2=0x7f9c54144c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:38.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.973+0000 7f9c49400640 1 --2- 172.21.15.39:0/682279603 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9c5415cd30 0x7f9c54144c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:38.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.973+0000 7f9c49400640 1 -- 172.21.15.39:0/682279603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c5406f800 msgr2=0x7f9c541456e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:38.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.973+0000 7f9c49400640 1 --2- 172.21.15.39:0/682279603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c5406f800 0x7f9c541456e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:38.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.973+0000 7f9c49400640 1 -- 172.21.15.39:0/682279603 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c5414ec20 con 0x7f9c5406fc10 2024-09-21T07:44:38.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.973+0000 7f9c49400640 1 --2- 172.21.15.39:0/682279603 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c5406fc10 0x7f9c5414fcc0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c00d7a0 tx=0x7f9c3c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:38.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.974+0000 7f9c4aa00640 1 -- 172.21.15.39:0/682279603 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c017070 con 0x7f9c5406fc10 2024-09-21T07:44:38.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.974+0000 7f9c4aa00640 1 -- 172.21.15.39:0/682279603 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c3c009d20 con 0x7f9c5406fc10 2024-09-21T07:44:38.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.974+0000 7f9c4aa00640 1 -- 172.21.15.39:0/682279603 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c004280 con 0x7f9c5406fc10 2024-09-21T07:44:38.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.974+0000 7f9c58a00640 1 -- 172.21.15.39:0/682279603 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c5414ef00 con 0x7f9c5406fc10 2024-09-21T07:44:38.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.974+0000 7f9c58a00640 1 -- 172.21.15.39:0/682279603 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c5414f4c0 con 0x7f9c5406fc10 2024-09-21T07:44:38.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.975+0000 7f9c58a00640 1 -- 172.21.15.39:0/682279603 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c540ce1e0 con 0x7f9c5406fc10 2024-09-21T07:44:38.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.976+0000 7f9c4aa00640 1 -- 172.21.15.39:0/682279603 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9c3c005020 con 0x7f9c5406fc10 2024-09-21T07:44:38.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.977+0000 7f9c4aa00640 1 --2- 172.21.15.39:0/682279603 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9c30078b90 0x7f9c3007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:38.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.977+0000 7f9c4aa00640 1 -- 172.21.15.39:0/682279603 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8420+0+0 (secure 0 0 0) 0x7f9c3c09bb70 con 0x7f9c5406fc10 2024-09-21T07:44:38.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.978+0000 7f9c51e00640 1 --2- 172.21.15.39:0/682279603 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9c30078b90 0x7f9c3007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:38.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.979+0000 7f9c51e00640 1 --2- 172.21.15.39:0/682279603 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9c30078b90 0x7f9c3007b0a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f9c44033000 tx=0x7f9c44004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:38.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:38.982+0000 7f9c4aa00640 1 -- 172.21.15.39:0/682279603 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9c540ce1e0 con 0x7f9c5406fc10 2024-09-21T07:44:39.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.217+0000 7f9c58a00640 1 -- 172.21.15.39:0/682279603 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c54073ec0 con 0x7f9c5406fc10 2024-09-21T07:44:39.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.217+0000 7f9c4aa00640 1 -- 172.21.15.39:0/682279603 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 164 v164) ==== 95+0+5252 (secure 0 0 0) 0x7f9c3c020020 con 0x7f9c5406fc10 2024-09-21T07:44:39.219 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:39.220 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":164,"btime":"2024-09-21T07:44:28:210229+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":8268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/67641510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":67641510},{"type":"v1","addr":"172.21.15.39:6833","nonce":67641510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":161}],"filesystems":[{"mdsmap":{"epoch":164,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:36:20.189287+0000","modified":"2024-09-21T07:44:28.210222+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6929,"mds_1":8052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6929":{"gid":6929,"name":"d","rank":0,"incarnation":160,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2582569093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2582569093},{"type":"v1","addr":"172.21.15.148:6837","nonce":2582569093}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8052":{"gid":8052,"name":"a","rank":1,"incarnation":155,"state":"up:active","state_seq":18,"addr":"172.21.15.39:6835/1551311805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1551311805},{"type":"v1","addr":"172.21.15.39:6835","nonce":1551311805}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6929,"qdb_cluster":[6929,8052]},"id":6}]} 2024-09-21T07:44:39.220 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 164 2024-09-21T07:44:39.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.220+0000 7f9c58a00640 1 -- 172.21.15.39:0/682279603 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9c30078b90 msgr2=0x7f9c3007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:39.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.220+0000 7f9c58a00640 1 --2- 172.21.15.39:0/682279603 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9c30078b90 0x7f9c3007b0a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f9c44033000 tx=0x7f9c44004730 comp rx=0 tx=0).stop 2024-09-21T07:44:39.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.220+0000 7f9c58a00640 1 -- 172.21.15.39:0/682279603 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c5406fc10 msgr2=0x7f9c5414fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:39.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.220+0000 7f9c58a00640 1 --2- 172.21.15.39:0/682279603 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c5406fc10 0x7f9c5414fcc0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c00d7a0 tx=0x7f9c3c00dc60 comp rx=0 tx=0).stop 2024-09-21T07:44:39.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.220+0000 7f9c58a00640 1 -- 172.21.15.39:0/682279603 shutdown_connections 2024-09-21T07:44:39.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.220+0000 7f9c58a00640 1 --2- 172.21.15.39:0/682279603 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9c30078b90 0x7f9c3007b0a0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:39.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.220+0000 7f9c58a00640 1 --2- 172.21.15.39:0/682279603 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9c5415cd30 0x7f9c54144c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:39.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.220+0000 7f9c58a00640 1 --2- 172.21.15.39:0/682279603 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c5406fc10 0x7f9c5414fcc0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:39.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.220+0000 7f9c58a00640 1 --2- 172.21.15.39:0/682279603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c5406f800 0x7f9c541456e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:39.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.220+0000 7f9c58a00640 1 -- 172.21.15.39:0/682279603 >> 172.21.15.39:0/682279603 conn(0x7f9c5406cfc0 msgr2=0x7f9c540b3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:39.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.220+0000 7f9c58a00640 1 -- 172.21.15.39:0/682279603 shutdown_connections 2024-09-21T07:44:39.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.220+0000 7f9c58a00640 1 -- 172.21.15.39:0/682279603 wait complete. 2024-09-21T07:44:39.246 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:44:39.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.470+0000 7f945d800640 1 -- 172.21.15.39:0/2222656477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f945815c830 msgr2=0x7f945806f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:39.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.470+0000 7f945d800640 1 --2- 172.21.15.39:0/2222656477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f945815c830 0x7f945806f8a0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f9440009830 tx=0x7f944002a510 comp rx=0 tx=0).stop 2024-09-21T07:44:39.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.470+0000 7f945d800640 1 -- 172.21.15.39:0/2222656477 shutdown_connections 2024-09-21T07:44:39.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.470+0000 7f945d800640 1 --2- 172.21.15.39:0/2222656477 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f945806fde0 0x7f9458072fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:39.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.470+0000 7f945d800640 1 --2- 172.21.15.39:0/2222656477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f945815c830 0x7f945806f8a0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:39.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.470+0000 7f945d800640 1 -- 172.21.15.39:0/2222656477 >> 172.21.15.39:0/2222656477 conn(0x7f94580b2510 msgr2=0x7f94580d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:39.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.470+0000 7f945d800640 1 -- 172.21.15.39:0/2222656477 shutdown_connections 2024-09-21T07:44:39.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.470+0000 7f945d800640 1 -- 172.21.15.39:0/2222656477 wait complete. 2024-09-21T07:44:39.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.473+0000 7f945d800640 1 Processor -- start 2024-09-21T07:44:39.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.473+0000 7f945d800640 1 -- start start 2024-09-21T07:44:39.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.474+0000 7f945d800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f945806fde0 0x7f945809a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:39.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.474+0000 7f945d800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94580d0f20 0x7f94581468c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:39.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.474+0000 7f945d800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f945815c830 0x7f945814a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:39.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.474+0000 7f945d800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f94580b4570 con 0x7f94580d0f20 2024-09-21T07:44:39.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.474+0000 7f945d800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f94580b43f0 con 0x7f945815c830 2024-09-21T07:44:39.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.474+0000 7f945d800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f94580b46f0 con 0x7f945806fde0 2024-09-21T07:44:39.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.475+0000 7f944fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94580d0f20 0x7f94581468c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:39.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.475+0000 7f944fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94580d0f20 0x7f94581468c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54582/0 (socket says 172.21.15.39:54582) 2024-09-21T07:44:39.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.475+0000 7f944fe00640 1 -- 172.21.15.39:0/1978373211 learned_addr learned my addr 172.21.15.39:0/1978373211 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:39.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.475+0000 7f9457e00640 1 --2- 172.21.15.39:0/1978373211 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f945806fde0 0x7f945809a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:39.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.475+0000 7f945ce00640 1 --2- 172.21.15.39:0/1978373211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f945815c830 0x7f945814a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:39.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.475+0000 7f944fe00640 1 -- 172.21.15.39:0/1978373211 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f945806fde0 msgr2=0x7f945809a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:39.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.475+0000 7f944fe00640 1 --2- 172.21.15.39:0/1978373211 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f945806fde0 0x7f945809a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:39.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.475+0000 7f944fe00640 1 -- 172.21.15.39:0/1978373211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f945815c830 msgr2=0x7f945814a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:39.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.475+0000 7f944fe00640 1 --2- 172.21.15.39:0/1978373211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f945815c830 0x7f945814a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:39.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.475+0000 7f944fe00640 1 -- 172.21.15.39:0/1978373211 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f945809b160 con 0x7f94580d0f20 2024-09-21T07:44:39.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.475+0000 7f945ce00640 1 --2- 172.21.15.39:0/1978373211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f945815c830 0x7f945814a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:44:39.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.476+0000 7f944fe00640 1 --2- 172.21.15.39:0/1978373211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94580d0f20 0x7f94581468c0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f9440000c00 tx=0x7f9440033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:39.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.476+0000 7f9454c00640 1 -- 172.21.15.39:0/1978373211 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9440031d60 con 0x7f94580d0f20 2024-09-21T07:44:39.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.476+0000 7f9454c00640 1 -- 172.21.15.39:0/1978373211 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9440033e10 con 0x7f94580d0f20 2024-09-21T07:44:39.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.476+0000 7f945d800640 1 -- 172.21.15.39:0/1978373211 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f945809aba0 con 0x7f94580d0f20 2024-09-21T07:44:39.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.476+0000 7f9454c00640 1 -- 172.21.15.39:0/1978373211 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9440033310 con 0x7f94580d0f20 2024-09-21T07:44:39.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.476+0000 7f945d800640 1 -- 172.21.15.39:0/1978373211 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9458149f00 con 0x7f94580d0f20 2024-09-21T07:44:39.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.477+0000 7f945d800640 1 -- 172.21.15.39:0/1978373211 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f945809a910 con 0x7f94580d0f20 2024-09-21T07:44:39.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.478+0000 7f9454c00640 1 -- 172.21.15.39:0/1978373211 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f94400334b0 con 0x7f94580d0f20 2024-09-21T07:44:39.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.479+0000 7f9454c00640 1 --2- 172.21.15.39:0/1978373211 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9434078b90 0x7f943407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:39.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.479+0000 7f9454c00640 1 -- 172.21.15.39:0/1978373211 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8420+0+0 (secure 0 0 0) 0x7f94400bc930 con 0x7f94580d0f20 2024-09-21T07:44:39.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.480+0000 7f9457e00640 1 --2- 172.21.15.39:0/1978373211 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9434078b90 0x7f943407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:39.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.480+0000 7f9457e00640 1 --2- 172.21.15.39:0/1978373211 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9434078b90 0x7f943407b0a0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9448001720 tx=0x7f9448007e90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:39.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.484+0000 7f9454c00640 1 -- 172.21.15.39:0/1978373211 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f944004a030 con 0x7f94580d0f20 2024-09-21T07:44:39.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.682+0000 7f945d800640 1 -- 172.21.15.39:0/1978373211 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f94580b4c10 con 0x7f94580d0f20 2024-09-21T07:44:39.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.684+0000 7f9454c00640 1 -- 172.21.15.39:0/1978373211 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) ==== 74+0+24569 (secure 0 0 0) 0x7f944003f030 con 0x7f94580d0f20 2024-09-21T07:44:39.686 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:39.686 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":123,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:44:24.184919+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:36:18.142953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-21T07:36:19.149503+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":119,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.22","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6832/1790581246":"2024-09-22T07:44:24.184881+0000","172.21.15.39:0/1731243684":"2024-09-21T08:44:16.323688+0000","172.21.15.39:6835/1326711417":"2024-09-22T07:43:26.076277+0000","172.21.15.39:6834/1326711417":"2024-09-22T07:43:26.076277+0000","172.21.15.39:6835/364657398":"2024-09-22T07:39:15.579613+0000","192.168.0.1:0/177249410":"2024-09-21T08:41:13.882767+0000","172.21.15.148:6837/73220122":"2024-09-22T07:42:40.074859+0000","172.21.15.39:0/596665086":"2024-09-21T08:43:35.317820+0000","172.21.15.39:6832/829716143":"2024-09-22T07:37:01.259219+0000","172.21.15.39:6832/855150775":"2024-09-22T07:39:07.739045+0000","172.21.15.148:6837/2381504293":"2024-09-22T07:38:32.207414+0000","172.21.15.39:6834/855150775":"2024-09-22T07:39:07.739045+0000","172.21.15.39:0/742594435":"2024-09-21T08:42:01.225749+0000","172.21.15.39:6835/2821446711":"2024-09-22T07:40:57.372409+0000","192.168.0.1:0/90671776":"2024-09-21T08:39:22.988045+0000","172.21.15.148:6836/759792812":"2024-09-22T07:44:05.615021+0000","172.21.15.39:6834/1472430787":"2024-09-22T07:37:58.212892+0000","172.21.15.39:6834/3695489158":"2024-09-22T07:41:51.807018+0000","172.21.15.148:6836/3026827614":"2024-09-22T07:37:29.781472+0000","172.21.15.148:6837/3026827614":"2024-09-22T07:37:29.781472+0000","172.21.15.39:6834/829716143":"2024-09-22T07:37:01.259219+0000","172.21.15.39:6835/3695489158":"2024-09-22T07:41:51.807018+0000","172.21.15.39:6832/1472430787":"2024-09-22T07:37:58.212892+0000","172.21.15.148:6837/759792812":"2024-09-22T07:44:05.615021+0000","172.21.15.148:6836/992188984":"2024-09-22T07:40:00.778049+0000","172.21.15.39:6833/364657398":"2024-09-22T07:39:15.579613+0000","172.21.15.148:6837/992188984":"2024-09-22T07:40:00.778049+0000","172.21.15.39:6833/1790581246":"2024-09-22T07:44:24.184881+0000","172.21.15.39:6833/1456343023":"2024-09-22T07:40:08.637993+0000","172.21.15.148:6836/2381504293":"2024-09-22T07:38:32.207414+0000","172.21.15.148:6836/2336112936":"2024-09-22T07:41:05.979253+0000","172.21.15.39:6832/1456343023":"2024-09-22T07:40:08.637993+0000","172.21.15.39:6834/2821446711":"2024-09-22T07:40:57.372409+0000","172.21.15.39:0/3356393880":"2024-09-21T08:40:15.824973+0000","172.21.15.148:6837/2336112936":"2024-09-22T07:41:05.979253+0000","172.21.15.148:6836/73220122":"2024-09-22T07:42:40.074859+0000","172.21.15.39:0/1669604593":"2024-09-21T08:42:52.059256+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:44:39.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.686+0000 7f945d800640 1 -- 172.21.15.39:0/1978373211 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9434078b90 msgr2=0x7f943407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:39.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.686+0000 7f945d800640 1 --2- 172.21.15.39:0/1978373211 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9434078b90 0x7f943407b0a0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9448001720 tx=0x7f9448007e90 comp rx=0 tx=0).stop 2024-09-21T07:44:39.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.686+0000 7f945d800640 1 -- 172.21.15.39:0/1978373211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94580d0f20 msgr2=0x7f94581468c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:39.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.686+0000 7f945d800640 1 --2- 172.21.15.39:0/1978373211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94580d0f20 0x7f94581468c0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f9440000c00 tx=0x7f9440033c20 comp rx=0 tx=0).stop 2024-09-21T07:44:39.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.686+0000 7f945d800640 1 -- 172.21.15.39:0/1978373211 shutdown_connections 2024-09-21T07:44:39.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.686+0000 7f945d800640 1 --2- 172.21.15.39:0/1978373211 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9434078b90 0x7f943407b0a0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:39.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.686+0000 7f945d800640 1 --2- 172.21.15.39:0/1978373211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f945815c830 0x7f945814a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:39.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.686+0000 7f945d800640 1 --2- 172.21.15.39:0/1978373211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94580d0f20 0x7f94581468c0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:39.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.686+0000 7f945d800640 1 --2- 172.21.15.39:0/1978373211 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f945806fde0 0x7f945809a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:39.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.686+0000 7f945d800640 1 -- 172.21.15.39:0/1978373211 >> 172.21.15.39:0/1978373211 conn(0x7f94580b2510 msgr2=0x7f945815d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:39.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.686+0000 7f945d800640 1 -- 172.21.15.39:0/1978373211 shutdown_connections 2024-09-21T07:44:39.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.686+0000 7f945d800640 1 -- 172.21.15.39:0/1978373211 wait complete. 2024-09-21T07:44:39.713 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T07:44:39.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.940+0000 7fc333200640 1 -- 172.21.15.39:0/3795734326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc334160a30 msgr2=0x7fc3340cb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:39.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.940+0000 7fc333200640 1 --2- 172.21.15.39:0/3795734326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc334160a30 0x7fc3340cb9b0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fc31c005fd0 tx=0x7fc31c02a540 comp rx=0 tx=0).stop 2024-09-21T07:44:39.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.941+0000 7fc333200640 1 -- 172.21.15.39:0/3795734326 shutdown_connections 2024-09-21T07:44:39.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.941+0000 7fc333200640 1 --2- 172.21.15.39:0/3795734326 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc3340cbef0 0x7fc33406f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:39.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.941+0000 7fc333200640 1 --2- 172.21.15.39:0/3795734326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc334160a30 0x7fc3340cb9b0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:39.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.941+0000 7fc333200640 1 -- 172.21.15.39:0/3795734326 >> 172.21.15.39:0/3795734326 conn(0x7fc3340b2510 msgr2=0x7fc3340d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:39.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.941+0000 7fc333200640 1 -- 172.21.15.39:0/3795734326 shutdown_connections 2024-09-21T07:44:39.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.941+0000 7fc333200640 1 -- 172.21.15.39:0/3795734326 wait complete. 2024-09-21T07:44:39.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.943+0000 7fc333200640 1 Processor -- start 2024-09-21T07:44:39.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.943+0000 7fc333200640 1 -- start start 2024-09-21T07:44:39.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.945+0000 7fc333200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3340cbef0 0x7fc3340cf390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:39.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.945+0000 7fc333200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc33415fc30 0x7fc33415d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:39.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.945+0000 7fc333200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc3340cfc90 0x7fc33415dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:39.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.945+0000 7fc333200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc33406f8d0 con 0x7fc33415fc30 2024-09-21T07:44:39.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.945+0000 7fc333200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc33406f750 con 0x7fc3340cbef0 2024-09-21T07:44:39.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.945+0000 7fc333200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc33406fa50 con 0x7fc3340cfc90 2024-09-21T07:44:39.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.945+0000 7fc32be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc33415fc30 0x7fc33415d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:39.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.945+0000 7fc32be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc33415fc30 0x7fc33415d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54614/0 (socket says 172.21.15.39:54614) 2024-09-21T07:44:39.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.945+0000 7fc32be00640 1 -- 172.21.15.39:0/589037871 learned_addr learned my addr 172.21.15.39:0/589037871 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:39.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.945+0000 7fc331e00640 1 --2- 172.21.15.39:0/589037871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3340cbef0 0x7fc3340cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:39.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.945+0000 7fc332800640 1 --2- 172.21.15.39:0/589037871 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc3340cfc90 0x7fc33415dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:39.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.945+0000 7fc32be00640 1 -- 172.21.15.39:0/589037871 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc3340cfc90 msgr2=0x7fc33415dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:39.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.945+0000 7fc32be00640 1 --2- 172.21.15.39:0/589037871 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc3340cfc90 0x7fc33415dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:39.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.945+0000 7fc32be00640 1 -- 172.21.15.39:0/589037871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3340cbef0 msgr2=0x7fc3340cf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:39.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.945+0000 7fc32be00640 1 --2- 172.21.15.39:0/589037871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3340cbef0 0x7fc3340cf390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:39.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.945+0000 7fc32be00640 1 -- 172.21.15.39:0/589037871 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3340c7500 con 0x7fc33415fc30 2024-09-21T07:44:39.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.946+0000 7fc32be00640 1 --2- 172.21.15.39:0/589037871 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc33415fc30 0x7fc33415d6a0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fc31c034040 tx=0x7fc31c003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:39.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.946+0000 7fc32a000640 1 -- 172.21.15.39:0/589037871 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c03e030 con 0x7fc33415fc30 2024-09-21T07:44:39.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.946+0000 7fc32a000640 1 -- 172.21.15.39:0/589037871 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc31c003e00 con 0x7fc33415fc30 2024-09-21T07:44:39.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.946+0000 7fc333200640 1 -- 172.21.15.39:0/589037871 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3340c77e0 con 0x7fc33415fc30 2024-09-21T07:44:39.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.946+0000 7fc32a000640 1 -- 172.21.15.39:0/589037871 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c035040 con 0x7fc33415fc30 2024-09-21T07:44:39.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.946+0000 7fc333200640 1 -- 172.21.15.39:0/589037871 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3340c93e0 con 0x7fc33415fc30 2024-09-21T07:44:39.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.948+0000 7fc333200640 1 -- 172.21.15.39:0/589037871 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3341f9d00 con 0x7fc33415fc30 2024-09-21T07:44:39.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.949+0000 7fc32a000640 1 -- 172.21.15.39:0/589037871 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc31c041030 con 0x7fc33415fc30 2024-09-21T07:44:39.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.949+0000 7fc32a000640 1 --2- 172.21.15.39:0/589037871 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc310078b90 0x7fc31007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:39.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.950+0000 7fc32a000640 1 -- 172.21.15.39:0/589037871 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8420+0+0 (secure 0 0 0) 0x7fc31c0bbf30 con 0x7fc33415fc30 2024-09-21T07:44:39.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.950+0000 7fc331e00640 1 --2- 172.21.15.39:0/589037871 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc310078b90 0x7fc31007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:39.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.951+0000 7fc331e00640 1 --2- 172.21.15.39:0/589037871 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc310078b90 0x7fc31007b0a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fc334161330 tx=0x7fc324009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:39.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:39.954+0000 7fc32a000640 1 -- 172.21.15.39:0/589037871 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc31c083260 con 0x7fc33415fc30 2024-09-21T07:44:40.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.190+0000 7fc333200640 1 -- 172.21.15.39:0/589037871 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc33406ff70 con 0x7fc33415fc30 2024-09-21T07:44:40.193 INFO:tasks.ceph.mon.a.smithi039.stderr:2024-09-21T07:44:40.191+0000 7fabab600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T07:44:40.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.193+0000 7fc32a000640 1 -- 172.21.15.39:0/589037871 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v165) ==== 161+0+0 (secure 0 0 0) 0x7fc31c088e20 con 0x7fc33415fc30 2024-09-21T07:44:40.195 INFO:teuthology.orchestra.run.smithi039.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T07:44:40.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.196+0000 7fc333200640 1 -- 172.21.15.39:0/589037871 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc310078b90 msgr2=0x7fc31007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:40.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.196+0000 7fc333200640 1 --2- 172.21.15.39:0/589037871 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc310078b90 0x7fc31007b0a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fc334161330 tx=0x7fc324009040 comp rx=0 tx=0).stop 2024-09-21T07:44:40.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.196+0000 7fc333200640 1 -- 172.21.15.39:0/589037871 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc33415fc30 msgr2=0x7fc33415d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:40.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.196+0000 7fc333200640 1 --2- 172.21.15.39:0/589037871 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc33415fc30 0x7fc33415d6a0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fc31c034040 tx=0x7fc31c003910 comp rx=0 tx=0).stop 2024-09-21T07:44:40.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.196+0000 7fc333200640 1 -- 172.21.15.39:0/589037871 shutdown_connections 2024-09-21T07:44:40.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.196+0000 7fc333200640 1 --2- 172.21.15.39:0/589037871 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc310078b90 0x7fc31007b0a0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:40.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.196+0000 7fc333200640 1 --2- 172.21.15.39:0/589037871 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc3340cfc90 0x7fc33415dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:40.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.196+0000 7fc333200640 1 --2- 172.21.15.39:0/589037871 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc33415fc30 0x7fc33415d6a0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:40.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.196+0000 7fc333200640 1 --2- 172.21.15.39:0/589037871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3340cbef0 0x7fc3340cf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:40.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.197+0000 7fc333200640 1 -- 172.21.15.39:0/589037871 >> 172.21.15.39:0/589037871 conn(0x7fc3340b2510 msgr2=0x7fc334073ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:40.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.197+0000 7fc333200640 1 -- 172.21.15.39:0/589037871 shutdown_connections 2024-09-21T07:44:40.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.198+0000 7fc333200640 1 -- 172.21.15.39:0/589037871 wait complete. 2024-09-21T07:44:40.207 INFO:tasks.ceph.mds.d.smithi148.stderr: -1113> 2024-09-21T07:44:31.647+0000 7f0a47000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:44:40.207 INFO:tasks.ceph.mds.d.smithi148.stderr: -1111> 2024-09-21T07:44:31.647+0000 7f0a47000640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:44:40.227 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T07:44:40.327 INFO:tasks.ceph.mds.d.smithi148.stdout:starting mds.d at 2024-09-21T07:44:40.330 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-21T07:44:40.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.460+0000 7f0d20a00640 1 -- 172.21.15.39:0/3844126907 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d1c156d60 msgr2=0x7f0d1c0cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:40.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.460+0000 7f0d20a00640 1 --2- 172.21.15.39:0/3844126907 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d1c156d60 0x7f0d1c0cc430 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c005fd0 tx=0x7f0d0c02a5d0 comp rx=0 tx=0).stop 2024-09-21T07:44:40.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.460+0000 7f0d20a00640 1 -- 172.21.15.39:0/3844126907 shutdown_connections 2024-09-21T07:44:40.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.460+0000 7f0d20a00640 1 --2- 172.21.15.39:0/3844126907 >> v2:172.21.15.39:3300/0 conn(0x7f0d1c160940 0x7f0d1c0ad4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:40.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.460+0000 7f0d20a00640 1 --2- 172.21.15.39:0/3844126907 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d1c156d60 0x7f0d1c0cc430 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:40.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.460+0000 7f0d20a00640 1 -- 172.21.15.39:0/3844126907 >> 172.21.15.39:0/3844126907 conn(0x7f0d1c0b25f0 msgr2=0x7f0d1c0d01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:40.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.460+0000 7f0d20a00640 1 -- 172.21.15.39:0/3844126907 shutdown_connections 2024-09-21T07:44:40.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.460+0000 7f0d20a00640 1 -- 172.21.15.39:0/3844126907 wait complete. 2024-09-21T07:44:40.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.462+0000 7f0d20a00640 1 Processor -- start 2024-09-21T07:44:40.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.462+0000 7f0d20a00640 1 -- start start 2024-09-21T07:44:40.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.464+0000 7f0d20a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d1c0af690 0x7f0d1c1f2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:40.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.464+0000 7f0d20a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d1c156d60 0x7f0d1c1f6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:40.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.464+0000 7f0d20a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d1c09c290 0x7f0d1c1f6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:40.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.464+0000 7f0d20a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1c0adf50 con 0x7f0d1c156d60 2024-09-21T07:44:40.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.464+0000 7f0d20a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1c0addd0 con 0x7f0d1c0af690 2024-09-21T07:44:40.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.464+0000 7f0d20a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f0d1c0ae0d0 con 0x7f0d1c09c290 2024-09-21T07:44:40.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.464+0000 7f0d13e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d1c156d60 0x7f0d1c1f6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:40.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.464+0000 7f0d1b400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d1c0af690 0x7f0d1c1f2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:40.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.464+0000 7f0d13e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d1c156d60 0x7f0d1c1f6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54648/0 (socket says 172.21.15.39:54648) 2024-09-21T07:44:40.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.464+0000 7f0d13e00640 1 -- 172.21.15.39:0/3840241284 learned_addr learned my addr 172.21.15.39:0/3840241284 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:40.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.464+0000 7f0d1be00640 1 --2- 172.21.15.39:0/3840241284 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d1c09c290 0x7f0d1c1f6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:40.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.465+0000 7f0d1b400640 1 -- 172.21.15.39:0/3840241284 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d1c09c290 msgr2=0x7f0d1c1f6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:40.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.465+0000 7f0d1b400640 1 --2- 172.21.15.39:0/3840241284 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d1c09c290 0x7f0d1c1f6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:40.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.465+0000 7f0d1b400640 1 -- 172.21.15.39:0/3840241284 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d1c156d60 msgr2=0x7f0d1c1f6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:40.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.465+0000 7f0d1b400640 1 --2- 172.21.15.39:0/3840241284 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d1c156d60 0x7f0d1c1f6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:40.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.465+0000 7f0d1b400640 1 -- 172.21.15.39:0/3840241284 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d1c1f3600 con 0x7f0d1c0af690 2024-09-21T07:44:40.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.465+0000 7f0d1b400640 1 --2- 172.21.15.39:0/3840241284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d1c0af690 0x7f0d1c1f2d60 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c0caf30 tx=0x7f0d0c005770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:40.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.465+0000 7f0d13400640 1 -- 172.21.15.39:0/3840241284 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c03e030 con 0x7f0d1c0af690 2024-09-21T07:44:40.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.465+0000 7f0d13400640 1 -- 172.21.15.39:0/3840241284 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d0c0039f0 con 0x7f0d1c0af690 2024-09-21T07:44:40.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.465+0000 7f0d20a00640 1 -- 172.21.15.39:0/3840241284 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d1c1f38e0 con 0x7f0d1c0af690 2024-09-21T07:44:40.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.465+0000 7f0d13400640 1 -- 172.21.15.39:0/3840241284 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c035040 con 0x7f0d1c0af690 2024-09-21T07:44:40.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.466+0000 7f0d20a00640 1 -- 172.21.15.39:0/3840241284 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d1c072dd0 con 0x7f0d1c0af690 2024-09-21T07:44:40.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.467+0000 7f0d20a00640 1 -- 172.21.15.39:0/3840241284 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d1c0ae580 con 0x7f0d1c0af690 2024-09-21T07:44:40.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.468+0000 7f0d13400640 1 -- 172.21.15.39:0/3840241284 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0d0c041030 con 0x7f0d1c0af690 2024-09-21T07:44:40.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.469+0000 7f0d13400640 1 --2- 172.21.15.39:0/3840241284 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0cf8078bc0 0x7f0cf807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:40.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.469+0000 7f0d13400640 1 -- 172.21.15.39:0/3840241284 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8592+0+0 (secure 0 0 0) 0x7f0d0c0bbc40 con 0x7f0d1c0af690 2024-09-21T07:44:40.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.469+0000 7f0d13e00640 1 --2- 172.21.15.39:0/3840241284 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0cf8078bc0 0x7f0cf807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:40.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.470+0000 7f0d13e00640 1 --2- 172.21.15.39:0/3840241284 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0cf8078bc0 0x7f0cf807b0d0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c0b1630 tx=0x7f0d04009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:40.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.474+0000 7f0d13400640 1 -- 172.21.15.39:0/3840241284 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0d0c082e60 con 0x7f0d1c0af690 2024-09-21T07:44:40.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:40.711+0000 7f0d20a00640 1 -- 172.21.15.39:0/3840241284 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f0d1c069300 con 0x7f0d1c0af690 2024-09-21T07:44:41.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.282+0000 7f0d13400640 1 -- 172.21.15.39:0/3840241284 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v166) ==== 104+0+0 (secure 0 0 0) 0x7f0d0c087890 con 0x7f0d1c0af690 2024-09-21T07:44:41.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.285+0000 7f0d20a00640 1 -- 172.21.15.39:0/3840241284 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0cf8078bc0 msgr2=0x7f0cf807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:41.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.285+0000 7f0d20a00640 1 --2- 172.21.15.39:0/3840241284 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0cf8078bc0 0x7f0cf807b0d0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c0b1630 tx=0x7f0d04009040 comp rx=0 tx=0).stop 2024-09-21T07:44:41.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.285+0000 7f0d20a00640 1 -- 172.21.15.39:0/3840241284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d1c0af690 msgr2=0x7f0d1c1f2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:41.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.285+0000 7f0d20a00640 1 --2- 172.21.15.39:0/3840241284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d1c0af690 0x7f0d1c1f2d60 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c0caf30 tx=0x7f0d0c005770 comp rx=0 tx=0).stop 2024-09-21T07:44:41.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.285+0000 7f0d20a00640 1 -- 172.21.15.39:0/3840241284 shutdown_connections 2024-09-21T07:44:41.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.285+0000 7f0d20a00640 1 --2- 172.21.15.39:0/3840241284 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0cf8078bc0 0x7f0cf807b0d0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:41.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.285+0000 7f0d20a00640 1 --2- 172.21.15.39:0/3840241284 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d1c09c290 0x7f0d1c1f6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:41.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.285+0000 7f0d20a00640 1 --2- 172.21.15.39:0/3840241284 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d1c156d60 0x7f0d1c1f6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:41.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.285+0000 7f0d20a00640 1 --2- 172.21.15.39:0/3840241284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d1c0af690 0x7f0d1c1f2d60 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:41.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.285+0000 7f0d20a00640 1 -- 172.21.15.39:0/3840241284 >> 172.21.15.39:0/3840241284 conn(0x7f0d1c0b25f0 msgr2=0x7f0d1c1601c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:41.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.285+0000 7f0d20a00640 1 -- 172.21.15.39:0/3840241284 shutdown_connections 2024-09-21T07:44:41.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.285+0000 7f0d20a00640 1 -- 172.21.15.39:0/3840241284 wait complete. 2024-09-21T07:44:41.315 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T07:44:41.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.545+0000 7f3b73e00640 1 -- 172.21.15.39:0/2381849470 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b7415fee0 msgr2=0x7f3b74159330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:41.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.545+0000 7f3b73e00640 1 --2- 172.21.15.39:0/2381849470 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b7415fee0 0x7f3b74159330 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3b64009830 tx=0x7f3b6402a530 comp rx=0 tx=0).stop 2024-09-21T07:44:41.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.545+0000 7f3b73e00640 1 -- 172.21.15.39:0/2381849470 shutdown_connections 2024-09-21T07:44:41.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.545+0000 7f3b73e00640 1 --2- 172.21.15.39:0/2381849470 >> v2:172.21.15.148:3300/0 conn(0x7f3b7415fa10 0x7f3b740b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:41.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.545+0000 7f3b73e00640 1 --2- 172.21.15.39:0/2381849470 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b7415fee0 0x7f3b74159330 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:41.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.545+0000 7f3b73e00640 1 -- 172.21.15.39:0/2381849470 >> 172.21.15.39:0/2381849470 conn(0x7f3b740b65d0 msgr2=0x7f3b740b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:41.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.545+0000 7f3b73e00640 1 -- 172.21.15.39:0/2381849470 shutdown_connections 2024-09-21T07:44:41.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.545+0000 7f3b73e00640 1 -- 172.21.15.39:0/2381849470 wait complete. 2024-09-21T07:44:41.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.547+0000 7f3b73e00640 1 Processor -- start 2024-09-21T07:44:41.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.547+0000 7f3b73e00640 1 -- start start 2024-09-21T07:44:41.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.549+0000 7f3b73e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b740b34c0 0x7f3b7414b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:41.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.549+0000 7f3b73e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b7415fee0 0x7f3b740ca430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:41.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.549+0000 7f3b73e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b7414bc70 0x7f3b7415ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:41.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.549+0000 7f3b73e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3b740b4fd0 con 0x7f3b740b34c0 2024-09-21T07:44:41.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.549+0000 7f3b73e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3b740b4e50 con 0x7f3b7414bc70 2024-09-21T07:44:41.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.549+0000 7f3b73e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f3b740b5150 con 0x7f3b7415fee0 2024-09-21T07:44:41.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.549+0000 7f3b72a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b740b34c0 0x7f3b7414b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:41.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.549+0000 7f3b72a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b740b34c0 0x7f3b7414b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54664/0 (socket says 172.21.15.39:54664) 2024-09-21T07:44:41.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.549+0000 7f3b72a00640 1 -- 172.21.15.39:0/4150233702 learned_addr learned my addr 172.21.15.39:0/4150233702 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:41.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.549+0000 7f3b6be00640 1 --2- 172.21.15.39:0/4150233702 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b7415fee0 0x7f3b740ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:41.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.549+0000 7f3b73400640 1 --2- 172.21.15.39:0/4150233702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b7414bc70 0x7f3b7415ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:41.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.550+0000 7f3b72a00640 1 -- 172.21.15.39:0/4150233702 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b7415fee0 msgr2=0x7f3b740ca430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:41.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.550+0000 7f3b72a00640 1 --2- 172.21.15.39:0/4150233702 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b7415fee0 0x7f3b740ca430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:41.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.550+0000 7f3b72a00640 1 -- 172.21.15.39:0/4150233702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b7414bc70 msgr2=0x7f3b7415ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:41.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.550+0000 7f3b72a00640 1 --2- 172.21.15.39:0/4150233702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b7414bc70 0x7f3b7415ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:41.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.550+0000 7f3b72a00640 1 -- 172.21.15.39:0/4150233702 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b740cada0 con 0x7f3b740b34c0 2024-09-21T07:44:41.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.550+0000 7f3b6be00640 1 --2- 172.21.15.39:0/4150233702 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b7415fee0 0x7f3b740ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:44:41.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.550+0000 7f3b72a00640 1 --2- 172.21.15.39:0/4150233702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b740b34c0 0x7f3b7414b570 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f3b64000c00 tx=0x7f3b64033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:41.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.550+0000 7f3b6aa00640 1 -- 172.21.15.39:0/4150233702 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b64005c90 con 0x7f3b740b34c0 2024-09-21T07:44:41.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.550+0000 7f3b6aa00640 1 -- 172.21.15.39:0/4150233702 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b64031e50 con 0x7f3b740b34c0 2024-09-21T07:44:41.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.550+0000 7f3b73e00640 1 -- 172.21.15.39:0/4150233702 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b740b0290 con 0x7f3b740b34c0 2024-09-21T07:44:41.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.550+0000 7f3b6aa00640 1 -- 172.21.15.39:0/4150233702 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b640332f0 con 0x7f3b740b34c0 2024-09-21T07:44:41.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.550+0000 7f3b73e00640 1 -- 172.21.15.39:0/4150233702 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b740b0850 con 0x7f3b740b34c0 2024-09-21T07:44:41.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.551+0000 7f3b73e00640 1 -- 172.21.15.39:0/4150233702 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b740b5670 con 0x7f3b740b34c0 2024-09-21T07:44:41.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.553+0000 7f3b6aa00640 1 -- 172.21.15.39:0/4150233702 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3b64033490 con 0x7f3b740b34c0 2024-09-21T07:44:41.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.554+0000 7f3b6aa00640 1 --2- 172.21.15.39:0/4150233702 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3b50078bc0 0x7f3b5007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:41.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.554+0000 7f3b6aa00640 1 -- 172.21.15.39:0/4150233702 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8592+0+0 (secure 0 0 0) 0x7f3b640bcc30 con 0x7f3b740b34c0 2024-09-21T07:44:41.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.554+0000 7f3b6be00640 1 --2- 172.21.15.39:0/4150233702 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3b50078bc0 0x7f3b5007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:41.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.555+0000 7f3b6be00640 1 --2- 172.21.15.39:0/4150233702 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3b50078bc0 0x7f3b5007b0d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3b74071e00 tx=0x7f3b5c009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:41.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.558+0000 7f3b6aa00640 1 -- 172.21.15.39:0/4150233702 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3b6403f030 con 0x7f3b740b34c0 2024-09-21T07:44:41.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:41.756+0000 7f3b73e00640 1 -- 172.21.15.39:0/4150233702 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3b74071e30 con 0x7f3b740b34c0 2024-09-21T07:44:42.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.288+0000 7f3b6aa00640 1 -- 172.21.15.39:0/4150233702 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v125) ==== 181+0+0 (secure 0 0 0) 0x7f3b640844a0 con 0x7f3b740b34c0 2024-09-21T07:44:42.290 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' removed 2024-09-21T07:44:42.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.291+0000 7f3b73e00640 1 -- 172.21.15.39:0/4150233702 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3b50078bc0 msgr2=0x7f3b5007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:42.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.291+0000 7f3b73e00640 1 --2- 172.21.15.39:0/4150233702 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3b50078bc0 0x7f3b5007b0d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3b74071e00 tx=0x7f3b5c009040 comp rx=0 tx=0).stop 2024-09-21T07:44:42.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.291+0000 7f3b73e00640 1 -- 172.21.15.39:0/4150233702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b740b34c0 msgr2=0x7f3b7414b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:42.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.291+0000 7f3b73e00640 1 --2- 172.21.15.39:0/4150233702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b740b34c0 0x7f3b7414b570 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f3b64000c00 tx=0x7f3b64033c20 comp rx=0 tx=0).stop 2024-09-21T07:44:42.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.291+0000 7f3b73e00640 1 -- 172.21.15.39:0/4150233702 shutdown_connections 2024-09-21T07:44:42.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.291+0000 7f3b73e00640 1 --2- 172.21.15.39:0/4150233702 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3b50078bc0 0x7f3b5007b0d0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:42.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.291+0000 7f3b73e00640 1 --2- 172.21.15.39:0/4150233702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b7414bc70 0x7f3b7415ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:42.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.291+0000 7f3b73e00640 1 --2- 172.21.15.39:0/4150233702 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b7415fee0 0x7f3b740ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:42.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.291+0000 7f3b73e00640 1 --2- 172.21.15.39:0/4150233702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b740b34c0 0x7f3b7414b570 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:42.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.291+0000 7f3b73e00640 1 -- 172.21.15.39:0/4150233702 >> 172.21.15.39:0/4150233702 conn(0x7f3b740b65d0 msgr2=0x7f3b7415f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:42.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.291+0000 7f3b73e00640 1 -- 172.21.15.39:0/4150233702 shutdown_connections 2024-09-21T07:44:42.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.291+0000 7f3b73e00640 1 -- 172.21.15.39:0/4150233702 wait complete. 2024-09-21T07:44:42.328 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T07:44:42.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.559+0000 7f2d01400640 1 -- 172.21.15.39:0/2542802056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cfc073a60 msgr2=0x7f2cfc0b3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:42.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.559+0000 7f2d01400640 1 --2- 172.21.15.39:0/2542802056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cfc073a60 0x7f2cfc0b3950 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc06f3c0 tx=0x7f2ce402a540 comp rx=0 tx=0).stop 2024-09-21T07:44:42.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.559+0000 7f2d01400640 1 -- 172.21.15.39:0/2542802056 shutdown_connections 2024-09-21T07:44:42.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.559+0000 7f2d01400640 1 --2- 172.21.15.39:0/2542802056 >> v2:172.21.15.148:3300/0 conn(0x7f2cfc0b3e90 0x7f2cfc162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:42.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.559+0000 7f2d01400640 1 --2- 172.21.15.39:0/2542802056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cfc073a60 0x7f2cfc0b3950 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:42.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.559+0000 7f2d01400640 1 --2- 172.21.15.39:0/2542802056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2cfc155c20 0x7f2cfc073520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:42.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.559+0000 7f2d01400640 1 -- 172.21.15.39:0/2542802056 >> 172.21.15.39:0/2542802056 conn(0x7f2cfc06d7a0 msgr2=0x7f2cfc0b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:42.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.559+0000 7f2d01400640 1 -- 172.21.15.39:0/2542802056 shutdown_connections 2024-09-21T07:44:42.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.559+0000 7f2d01400640 1 -- 172.21.15.39:0/2542802056 wait complete. 2024-09-21T07:44:42.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.562+0000 7f2d01400640 1 Processor -- start 2024-09-21T07:44:42.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.562+0000 7f2d01400640 1 -- start start 2024-09-21T07:44:42.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.563+0000 7f2d01400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cfc073a60 0x7f2cfc158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:42.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.563+0000 7f2d01400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cfc0b3e90 0x7f2cfc1466c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:42.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.563+0000 7f2d01400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2cfc155c20 0x7f2cfc146c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:42.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.563+0000 7f2d01400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc162f60 con 0x7f2cfc0b3e90 2024-09-21T07:44:42.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.563+0000 7f2d01400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc162de0 con 0x7f2cfc073a60 2024-09-21T07:44:42.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.563+0000 7f2d01400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f2cfc1630e0 con 0x7f2cfc155c20 2024-09-21T07:44:42.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.563+0000 7f2cf3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cfc0b3e90 0x7f2cfc1466c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:42.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.563+0000 7f2cf3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cfc0b3e90 0x7f2cfc1466c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46424/0 (socket says 172.21.15.39:46424) 2024-09-21T07:44:42.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.563+0000 7f2cf3e00640 1 -- 172.21.15.39:0/3248789872 learned_addr learned my addr 172.21.15.39:0/3248789872 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:42.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.564+0000 7f2cfb200640 1 --2- 172.21.15.39:0/3248789872 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2cfc155c20 0x7f2cfc146c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:42.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.564+0000 7f2cfa800640 1 --2- 172.21.15.39:0/3248789872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cfc073a60 0x7f2cfc158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:42.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.564+0000 7f2cf3e00640 1 -- 172.21.15.39:0/3248789872 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2cfc155c20 msgr2=0x7f2cfc146c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:42.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.564+0000 7f2cf3e00640 1 --2- 172.21.15.39:0/3248789872 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2cfc155c20 0x7f2cfc146c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:42.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.564+0000 7f2cf3e00640 1 -- 172.21.15.39:0/3248789872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cfc073a60 msgr2=0x7f2cfc158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:42.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.564+0000 7f2cf3e00640 1 --2- 172.21.15.39:0/3248789872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cfc073a60 0x7f2cfc158780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:42.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.564+0000 7f2cf3e00640 1 -- 172.21.15.39:0/3248789872 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cfc1490d0 con 0x7f2cfc0b3e90 2024-09-21T07:44:42.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.564+0000 7f2cf3e00640 1 --2- 172.21.15.39:0/3248789872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cfc0b3e90 0x7f2cfc1466c0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4000c00 tx=0x7f2ce4003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:42.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.565+0000 7f2cf2a00640 1 -- 172.21.15.39:0/3248789872 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce403e030 con 0x7f2cfc0b3e90 2024-09-21T07:44:42.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.565+0000 7f2cf2a00640 1 -- 172.21.15.39:0/3248789872 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ce4003e00 con 0x7f2cfc0b3e90 2024-09-21T07:44:42.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.565+0000 7f2cf2a00640 1 -- 172.21.15.39:0/3248789872 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4035040 con 0x7f2cfc0b3e90 2024-09-21T07:44:42.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.565+0000 7f2d01400640 1 -- 172.21.15.39:0/3248789872 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cfc149370 con 0x7f2cfc0b3e90 2024-09-21T07:44:42.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.565+0000 7f2d01400640 1 -- 172.21.15.39:0/3248789872 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cfc14b8a0 con 0x7f2cfc0b3e90 2024-09-21T07:44:42.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.566+0000 7f2d01400640 1 -- 172.21.15.39:0/3248789872 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cfc162690 con 0x7f2cfc0b3e90 2024-09-21T07:44:42.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.567+0000 7f2cf2a00640 1 -- 172.21.15.39:0/3248789872 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2ce4041030 con 0x7f2cfc0b3e90 2024-09-21T07:44:42.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.568+0000 7f2cf2a00640 1 --2- 172.21.15.39:0/3248789872 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2cd8078bc0 0x7f2cd807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:42.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.568+0000 7f2cf2a00640 1 -- 172.21.15.39:0/3248789872 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8154+0+0 (secure 0 0 0) 0x7f2ce40bbe20 con 0x7f2cfc0b3e90 2024-09-21T07:44:42.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.569+0000 7f2cfa800640 1 --2- 172.21.15.39:0/3248789872 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2cd8078bc0 0x7f2cd807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:42.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.570+0000 7f2cfa800640 1 --2- 172.21.15.39:0/3248789872 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2cd8078bc0 0x7f2cd807b0d0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f2cec004630 tx=0x7f2cec00b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:42.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.572+0000 7f2cf2a00640 1 -- 172.21.15.39:0/3248789872 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2ce4083270 con 0x7f2cfc0b3e90 2024-09-21T07:44:42.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:42.770+0000 7f2d01400640 1 -- 172.21.15.39:0/3248789872 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2cfc162050 con 0x7f2cfc0b3e90 2024-09-21T07:44:43.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.296+0000 7f2cf2a00640 1 -- 172.21.15.39:0/3248789872 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v126) ==== 169+0+0 (secure 0 0 0) 0x7f2ce4087ca0 con 0x7f2cfc0b3e90 2024-09-21T07:44:43.298 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' removed 2024-09-21T07:44:43.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.298+0000 7f2d01400640 1 -- 172.21.15.39:0/3248789872 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2cd8078bc0 msgr2=0x7f2cd807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:43.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.298+0000 7f2d01400640 1 --2- 172.21.15.39:0/3248789872 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2cd8078bc0 0x7f2cd807b0d0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f2cec004630 tx=0x7f2cec00b2e0 comp rx=0 tx=0).stop 2024-09-21T07:44:43.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.298+0000 7f2d01400640 1 -- 172.21.15.39:0/3248789872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cfc0b3e90 msgr2=0x7f2cfc1466c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:43.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.298+0000 7f2d01400640 1 --2- 172.21.15.39:0/3248789872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cfc0b3e90 0x7f2cfc1466c0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4000c00 tx=0x7f2ce4003910 comp rx=0 tx=0).stop 2024-09-21T07:44:43.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.298+0000 7f2d01400640 1 -- 172.21.15.39:0/3248789872 shutdown_connections 2024-09-21T07:44:43.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.298+0000 7f2d01400640 1 --2- 172.21.15.39:0/3248789872 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2cd8078bc0 0x7f2cd807b0d0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:43.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.298+0000 7f2d01400640 1 --2- 172.21.15.39:0/3248789872 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2cfc155c20 0x7f2cfc146c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:43.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.298+0000 7f2d01400640 1 --2- 172.21.15.39:0/3248789872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cfc0b3e90 0x7f2cfc1466c0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:43.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.298+0000 7f2d01400640 1 --2- 172.21.15.39:0/3248789872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cfc073a60 0x7f2cfc158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:43.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.298+0000 7f2d01400640 1 -- 172.21.15.39:0/3248789872 >> 172.21.15.39:0/3248789872 conn(0x7f2cfc06d7a0 msgr2=0x7f2cfc152b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:43.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.298+0000 7f2d01400640 1 -- 172.21.15.39:0/3248789872 shutdown_connections 2024-09-21T07:44:43.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.298+0000 7f2d01400640 1 -- 172.21.15.39:0/3248789872 wait complete. 2024-09-21T07:44:43.333 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T07:44:43.338 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T07:44:43.432 INFO:teuthology.orchestra.run.smithi039.stdout:[global] 2024-09-21T07:44:43.433 INFO:teuthology.orchestra.run.smithi039.stdout: chdir = "" 2024-09-21T07:44:43.433 INFO:teuthology.orchestra.run.smithi039.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T07:44:43.433 INFO:teuthology.orchestra.run.smithi039.stdout: auth supported = cephx 2024-09-21T07:44:43.433 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:43.433 INFO:teuthology.orchestra.run.smithi039.stdout: filestore xattr use omap = true 2024-09-21T07:44:43.433 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:43.433 INFO:teuthology.orchestra.run.smithi039.stdout: mon clock drift allowed = 1.000 2024-09-21T07:44:43.433 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:43.433 INFO:teuthology.orchestra.run.smithi039.stdout: osd crush chooseleaf type = 0 2024-09-21T07:44:43.433 INFO:teuthology.orchestra.run.smithi039.stdout: auth debug = true 2024-09-21T07:44:43.433 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:43.433 INFO:teuthology.orchestra.run.smithi039.stdout: ms die on old message = true 2024-09-21T07:44:43.434 INFO:teuthology.orchestra.run.smithi039.stdout: ms die on bug = true 2024-09-21T07:44:43.434 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:43.434 INFO:teuthology.orchestra.run.smithi039.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T07:44:43.434 INFO:teuthology.orchestra.run.smithi039.stdout: mon pg warn max object skew = 0 2024-09-21T07:44:43.434 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:43.434 INFO:teuthology.orchestra.run.smithi039.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T07:44:43.434 INFO:teuthology.orchestra.run.smithi039.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T07:44:43.434 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:43.434 INFO:teuthology.orchestra.run.smithi039.stdout: osd pool default size = 2 2024-09-21T07:44:43.434 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:43.434 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd allow primary affinity = true 2024-09-21T07:44:43.434 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd allow pg remap = true 2024-09-21T07:44:43.434 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on legacy crush tunables = false 2024-09-21T07:44:43.434 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on crush straw calc version zero = false 2024-09-21T07:44:43.435 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on no sortbitwise = false 2024-09-21T07:44:43.435 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on osd down out interval zero = false 2024-09-21T07:44:43.435 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on too few osds = false 2024-09-21T07:44:43.435 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T07:44:43.435 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T07:44:43.435 INFO:teuthology.orchestra.run.smithi039.stdout: mon_allow_pool_size_one = true 2024-09-21T07:44:43.435 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:43.435 INFO:teuthology.orchestra.run.smithi039.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T07:44:43.435 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:43.435 INFO:teuthology.orchestra.run.smithi039.stdout: osd default data pool replay window = 5 2024-09-21T07:44:43.435 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:43.435 INFO:teuthology.orchestra.run.smithi039.stdout: mon allow pool delete = true 2024-09-21T07:44:43.435 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:43.435 INFO:teuthology.orchestra.run.smithi039.stdout: mon cluster log file level = debug 2024-09-21T07:44:43.435 INFO:teuthology.orchestra.run.smithi039.stdout: debug asserts on shutdown = true 2024-09-21T07:44:43.436 INFO:teuthology.orchestra.run.smithi039.stdout: mon health detail to clog = false 2024-09-21T07:44:43.436 INFO:teuthology.orchestra.run.smithi039.stdout: mon host = "172.21.15.39,172.21.15.148,[v2:172.21.15.148:3301,v1:172.21.15.148:6790]" 2024-09-21T07:44:43.436 INFO:teuthology.orchestra.run.smithi039.stdout: ms die on skipped message = False 2024-09-21T07:44:43.436 INFO:teuthology.orchestra.run.smithi039.stdout: fsid = 7ab9b04a-f88d-49aa-a493-e5c076637ffb 2024-09-21T07:44:43.436 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:43.436 INFO:teuthology.orchestra.run.smithi039.stdout:[osd] 2024-09-21T07:44:43.436 INFO:teuthology.orchestra.run.smithi039.stdout: osd journal size = 100 2024-09-21T07:44:43.436 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:43.436 INFO:teuthology.orchestra.run.smithi039.stdout: osd scrub load threshold = 5.0 2024-09-21T07:44:43.436 INFO:teuthology.orchestra.run.smithi039.stdout: osd scrub max interval = 600 2024-09-21T07:44:43.436 INFO:teuthology.orchestra.run.smithi039.stdout: osd mclock profile = high_recovery_ops 2024-09-21T07:44:43.436 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:43.436 INFO:teuthology.orchestra.run.smithi039.stdout: osd recover clone overlap = true 2024-09-21T07:44:43.437 INFO:teuthology.orchestra.run.smithi039.stdout: osd recovery max chunk = 1048576 2024-09-21T07:44:43.437 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:43.437 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug shutdown = true 2024-09-21T07:44:43.437 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug op order = true 2024-09-21T07:44:43.437 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug verify stray on activate = true 2024-09-21T07:44:43.437 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug trim objects = true 2024-09-21T07:44:43.437 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:43.437 INFO:teuthology.orchestra.run.smithi039.stdout: osd open classes on start = true 2024-09-21T07:44:43.437 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug pg log writeout = true 2024-09-21T07:44:43.437 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:43.437 INFO:teuthology.orchestra.run.smithi039.stdout: osd deep scrub update digest min age = 30 2024-09-21T07:44:43.437 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:43.437 INFO:teuthology.orchestra.run.smithi039.stdout: osd map max advance = 10 2024-09-21T07:44:43.437 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:43.437 INFO:teuthology.orchestra.run.smithi039.stdout: journal zero on create = true 2024-09-21T07:44:43.438 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:43.438 INFO:teuthology.orchestra.run.smithi039.stdout: filestore ondisk finisher threads = 3 2024-09-21T07:44:43.438 INFO:teuthology.orchestra.run.smithi039.stdout: filestore apply finisher threads = 3 2024-09-21T07:44:43.438 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:43.438 INFO:teuthology.orchestra.run.smithi039.stdout: bdev debug aio = true 2024-09-21T07:44:43.438 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug misdirected ops = true 2024-09-21T07:44:43.438 INFO:teuthology.orchestra.run.smithi039.stdout: bdev async discard = True 2024-09-21T07:44:43.438 INFO:teuthology.orchestra.run.smithi039.stdout: bdev enable discard = True 2024-09-21T07:44:43.438 INFO:teuthology.orchestra.run.smithi039.stdout: bluestore allocator = bitmap 2024-09-21T07:44:43.438 INFO:teuthology.orchestra.run.smithi039.stdout: bluestore block size = 96636764160 2024-09-21T07:44:43.438 INFO:teuthology.orchestra.run.smithi039.stdout: bluestore fsck on mount = True 2024-09-21T07:44:43.438 INFO:teuthology.orchestra.run.smithi039.stdout: debug bluefs = 1/20 2024-09-21T07:44:43.438 INFO:teuthology.orchestra.run.smithi039.stdout: debug bluestore = 1/20 2024-09-21T07:44:43.438 INFO:teuthology.orchestra.run.smithi039.stdout: debug ms = 1 2024-09-21T07:44:43.439 INFO:teuthology.orchestra.run.smithi039.stdout: debug osd = 20 2024-09-21T07:44:43.439 INFO:teuthology.orchestra.run.smithi039.stdout: debug rocksdb = 4/10 2024-09-21T07:44:43.439 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T07:44:43.439 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd full ratio = 0.9 2024-09-21T07:44:43.439 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd nearfull ratio = 0.8 2024-09-21T07:44:43.439 INFO:teuthology.orchestra.run.smithi039.stdout: osd failsafe full ratio = 0.95 2024-09-21T07:44:43.439 INFO:teuthology.orchestra.run.smithi039.stdout: osd objectstore = bluestore 2024-09-21T07:44:43.439 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:43.439 INFO:teuthology.orchestra.run.smithi039.stdout:[mgr] 2024-09-21T07:44:43.439 INFO:teuthology.orchestra.run.smithi039.stdout: debug ms = 1 2024-09-21T07:44:43.439 INFO:teuthology.orchestra.run.smithi039.stdout: debug mgr = 20 2024-09-21T07:44:43.439 INFO:teuthology.orchestra.run.smithi039.stdout: debug mon = 20 2024-09-21T07:44:43.439 INFO:teuthology.orchestra.run.smithi039.stdout: debug auth = 20 2024-09-21T07:44:43.439 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min pgs per osd = 4 2024-09-21T07:44:43.440 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min bytes per osd = 10 2024-09-21T07:44:43.440 INFO:teuthology.orchestra.run.smithi039.stdout: mgr/telemetry/nag = false 2024-09-21T07:44:43.440 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:43.440 INFO:teuthology.orchestra.run.smithi039.stdout:[mon] 2024-09-21T07:44:43.440 INFO:teuthology.orchestra.run.smithi039.stdout: debug ms = 1 2024-09-21T07:44:43.440 INFO:teuthology.orchestra.run.smithi039.stdout: debug mon = 20 2024-09-21T07:44:43.440 INFO:teuthology.orchestra.run.smithi039.stdout: debug paxos = 20 2024-09-21T07:44:43.440 INFO:teuthology.orchestra.run.smithi039.stdout: debug auth = 20 2024-09-21T07:44:43.440 INFO:teuthology.orchestra.run.smithi039.stdout: mon data avail warn = 5 2024-09-21T07:44:43.440 INFO:teuthology.orchestra.run.smithi039.stdout: mon mgr mkfs grace = 240 2024-09-21T07:44:43.440 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min pgs per osd = 4 2024-09-21T07:44:43.440 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd reporter subtree level = osd 2024-09-21T07:44:43.440 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd prime pg temp = true 2024-09-21T07:44:43.440 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min bytes per osd = 10 2024-09-21T07:44:43.441 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:43.441 INFO:teuthology.orchestra.run.smithi039.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T07:44:43.441 INFO:teuthology.orchestra.run.smithi039.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T07:44:43.441 INFO:teuthology.orchestra.run.smithi039.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T07:44:43.441 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:43.441 INFO:teuthology.orchestra.run.smithi039.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T07:44:43.441 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T07:44:43.441 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T07:44:43.441 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:43.441 INFO:teuthology.orchestra.run.smithi039.stdout: # 1m isn't quite enough 2024-09-21T07:44:43.441 INFO:teuthology.orchestra.run.smithi039.stdout: mon_down_mkfs_grace = 2m 2024-09-21T07:44:43.441 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:43.441 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_filestore_osds = false 2024-09-21T07:44:43.441 INFO:teuthology.orchestra.run.smithi039.stdout: mon down mkfs grace = 300 2024-09-21T07:44:43.441 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:43.441 INFO:teuthology.orchestra.run.smithi039.stdout:[client] 2024-09-21T07:44:43.442 INFO:teuthology.orchestra.run.smithi039.stdout: rgw cache enabled = true 2024-09-21T07:44:43.442 INFO:teuthology.orchestra.run.smithi039.stdout: rgw enable ops log = true 2024-09-21T07:44:43.442 INFO:teuthology.orchestra.run.smithi039.stdout: rgw enable usage log = true 2024-09-21T07:44:43.442 INFO:teuthology.orchestra.run.smithi039.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T07:44:43.442 INFO:teuthology.orchestra.run.smithi039.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T07:44:43.442 INFO:teuthology.orchestra.run.smithi039.stdout: pid file = "" 2024-09-21T07:44:43.442 INFO:teuthology.orchestra.run.smithi039.stdout:[mon.a] 2024-09-21T07:44:43.442 INFO:teuthology.orchestra.run.smithi039.stdout:[mon.b] 2024-09-21T07:44:43.442 INFO:teuthology.orchestra.run.smithi039.stdout:[mon.c] 2024-09-21T07:44:43.442 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.a] 2024-09-21T07:44:43.442 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.c] 2024-09-21T07:44:43.442 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.b] 2024-09-21T07:44:43.442 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.d] 2024-09-21T07:44:43.442 INFO:teuthology.orchestra.run.smithi039.stdout:[mds] 2024-09-21T07:44:43.443 INFO:teuthology.orchestra.run.smithi148.stdout:[global] 2024-09-21T07:44:43.443 INFO:teuthology.orchestra.run.smithi148.stdout: chdir = "" 2024-09-21T07:44:43.443 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T07:44:43.443 INFO:teuthology.orchestra.run.smithi148.stdout: auth supported = cephx 2024-09-21T07:44:43.443 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:44:43.443 INFO:teuthology.orchestra.run.smithi148.stdout: filestore xattr use omap = true 2024-09-21T07:44:43.443 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:44:43.443 INFO:teuthology.orchestra.run.smithi148.stdout: mon clock drift allowed = 1.000 2024-09-21T07:44:43.443 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:44:43.443 INFO:teuthology.orchestra.run.smithi148.stdout: osd crush chooseleaf type = 0 2024-09-21T07:44:43.443 INFO:teuthology.orchestra.run.smithi148.stdout: auth debug = true 2024-09-21T07:44:43.444 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:44:43.444 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on old message = true 2024-09-21T07:44:43.444 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on bug = true 2024-09-21T07:44:43.444 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:44:43.444 INFO:teuthology.orchestra.run.smithi148.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T07:44:43.444 INFO:teuthology.orchestra.run.smithi148.stdout: mon pg warn max object skew = 0 2024-09-21T07:44:43.444 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:44:43.444 INFO:teuthology.orchestra.run.smithi148.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T07:44:43.444 INFO:teuthology.orchestra.run.smithi148.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T07:44:43.444 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:44:43.444 INFO:teuthology.orchestra.run.smithi148.stdout: osd pool default size = 2 2024-09-21T07:44:43.444 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:44:43.444 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd allow primary affinity = true 2024-09-21T07:44:43.444 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd allow pg remap = true 2024-09-21T07:44:43.445 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on legacy crush tunables = false 2024-09-21T07:44:43.445 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on crush straw calc version zero = false 2024-09-21T07:44:43.445 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on no sortbitwise = false 2024-09-21T07:44:43.445 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on osd down out interval zero = false 2024-09-21T07:44:43.445 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on too few osds = false 2024-09-21T07:44:43.445 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T07:44:43.445 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T07:44:43.445 INFO:teuthology.orchestra.run.smithi148.stdout: mon_allow_pool_size_one = true 2024-09-21T07:44:43.445 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:44:43.445 INFO:teuthology.orchestra.run.smithi148.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T07:44:43.445 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:44:43.445 INFO:teuthology.orchestra.run.smithi148.stdout: osd default data pool replay window = 5 2024-09-21T07:44:43.445 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:44:43.445 INFO:teuthology.orchestra.run.smithi148.stdout: mon allow pool delete = true 2024-09-21T07:44:43.445 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:44:43.446 INFO:teuthology.orchestra.run.smithi148.stdout: mon cluster log file level = debug 2024-09-21T07:44:43.446 INFO:teuthology.orchestra.run.smithi148.stdout: debug asserts on shutdown = true 2024-09-21T07:44:43.446 INFO:teuthology.orchestra.run.smithi148.stdout: mon health detail to clog = false 2024-09-21T07:44:43.446 INFO:teuthology.orchestra.run.smithi148.stdout: mon host = "172.21.15.39,172.21.15.148,[v2:172.21.15.148:3301,v1:172.21.15.148:6790]" 2024-09-21T07:44:43.446 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on skipped message = False 2024-09-21T07:44:43.446 INFO:teuthology.orchestra.run.smithi148.stdout: fsid = 7ab9b04a-f88d-49aa-a493-e5c076637ffb 2024-09-21T07:44:43.446 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:44:43.446 INFO:teuthology.orchestra.run.smithi148.stdout:[osd] 2024-09-21T07:44:43.446 INFO:teuthology.orchestra.run.smithi148.stdout: osd journal size = 100 2024-09-21T07:44:43.446 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:44:43.446 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub load threshold = 5.0 2024-09-21T07:44:43.446 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub max interval = 600 2024-09-21T07:44:43.446 INFO:teuthology.orchestra.run.smithi148.stdout: osd mclock profile = high_recovery_ops 2024-09-21T07:44:43.446 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:44:43.447 INFO:teuthology.orchestra.run.smithi148.stdout: osd recover clone overlap = true 2024-09-21T07:44:43.447 INFO:teuthology.orchestra.run.smithi148.stdout: osd recovery max chunk = 1048576 2024-09-21T07:44:43.447 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:44:43.447 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug shutdown = true 2024-09-21T07:44:43.447 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug op order = true 2024-09-21T07:44:43.447 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug verify stray on activate = true 2024-09-21T07:44:43.447 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug trim objects = true 2024-09-21T07:44:43.447 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:44:43.447 INFO:teuthology.orchestra.run.smithi148.stdout: osd open classes on start = true 2024-09-21T07:44:43.447 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug pg log writeout = true 2024-09-21T07:44:43.447 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:44:43.447 INFO:teuthology.orchestra.run.smithi148.stdout: osd deep scrub update digest min age = 30 2024-09-21T07:44:43.447 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:44:43.447 INFO:teuthology.orchestra.run.smithi148.stdout: osd map max advance = 10 2024-09-21T07:44:43.448 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:44:43.448 INFO:teuthology.orchestra.run.smithi148.stdout: journal zero on create = true 2024-09-21T07:44:43.448 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:44:43.448 INFO:teuthology.orchestra.run.smithi148.stdout: filestore ondisk finisher threads = 3 2024-09-21T07:44:43.448 INFO:teuthology.orchestra.run.smithi148.stdout: filestore apply finisher threads = 3 2024-09-21T07:44:43.448 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:44:43.448 INFO:teuthology.orchestra.run.smithi148.stdout: bdev debug aio = true 2024-09-21T07:44:43.448 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug misdirected ops = true 2024-09-21T07:44:43.448 INFO:teuthology.orchestra.run.smithi148.stdout: bdev async discard = True 2024-09-21T07:44:43.448 INFO:teuthology.orchestra.run.smithi148.stdout: bdev enable discard = True 2024-09-21T07:44:43.448 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore allocator = bitmap 2024-09-21T07:44:43.448 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore block size = 96636764160 2024-09-21T07:44:43.448 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore fsck on mount = True 2024-09-21T07:44:43.448 INFO:teuthology.orchestra.run.smithi148.stdout: debug bluefs = 1/20 2024-09-21T07:44:43.449 INFO:teuthology.orchestra.run.smithi148.stdout: debug bluestore = 1/20 2024-09-21T07:44:43.449 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-09-21T07:44:43.449 INFO:teuthology.orchestra.run.smithi148.stdout: debug osd = 20 2024-09-21T07:44:43.449 INFO:teuthology.orchestra.run.smithi148.stdout: debug rocksdb = 4/10 2024-09-21T07:44:43.449 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T07:44:43.449 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd full ratio = 0.9 2024-09-21T07:44:43.449 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd nearfull ratio = 0.8 2024-09-21T07:44:43.449 INFO:teuthology.orchestra.run.smithi148.stdout: osd failsafe full ratio = 0.95 2024-09-21T07:44:43.449 INFO:teuthology.orchestra.run.smithi148.stdout: osd objectstore = bluestore 2024-09-21T07:44:43.449 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:44:43.449 INFO:teuthology.orchestra.run.smithi148.stdout:[mgr] 2024-09-21T07:44:43.449 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-09-21T07:44:43.449 INFO:teuthology.orchestra.run.smithi148.stdout: debug mgr = 20 2024-09-21T07:44:43.449 INFO:teuthology.orchestra.run.smithi148.stdout: debug mon = 20 2024-09-21T07:44:43.449 INFO:teuthology.orchestra.run.smithi148.stdout: debug auth = 20 2024-09-21T07:44:43.450 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min pgs per osd = 4 2024-09-21T07:44:43.450 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min bytes per osd = 10 2024-09-21T07:44:43.450 INFO:teuthology.orchestra.run.smithi148.stdout: mgr/telemetry/nag = false 2024-09-21T07:44:43.450 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:44:43.450 INFO:teuthology.orchestra.run.smithi148.stdout:[mon] 2024-09-21T07:44:43.450 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-09-21T07:44:43.450 INFO:teuthology.orchestra.run.smithi148.stdout: debug mon = 20 2024-09-21T07:44:43.450 INFO:teuthology.orchestra.run.smithi148.stdout: debug paxos = 20 2024-09-21T07:44:43.450 INFO:teuthology.orchestra.run.smithi148.stdout: debug auth = 20 2024-09-21T07:44:43.450 INFO:teuthology.orchestra.run.smithi148.stdout: mon data avail warn = 5 2024-09-21T07:44:43.450 INFO:teuthology.orchestra.run.smithi148.stdout: mon mgr mkfs grace = 240 2024-09-21T07:44:43.450 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min pgs per osd = 4 2024-09-21T07:44:43.450 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd reporter subtree level = osd 2024-09-21T07:44:43.451 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd prime pg temp = true 2024-09-21T07:44:43.451 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min bytes per osd = 10 2024-09-21T07:44:43.451 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:44:43.451 INFO:teuthology.orchestra.run.smithi148.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T07:44:43.451 INFO:teuthology.orchestra.run.smithi148.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T07:44:43.451 INFO:teuthology.orchestra.run.smithi148.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T07:44:43.451 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:44:43.451 INFO:teuthology.orchestra.run.smithi148.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T07:44:43.451 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T07:44:43.451 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T07:44:43.451 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:44:43.451 INFO:teuthology.orchestra.run.smithi148.stdout: # 1m isn't quite enough 2024-09-21T07:44:43.451 INFO:teuthology.orchestra.run.smithi148.stdout: mon_down_mkfs_grace = 2m 2024-09-21T07:44:43.451 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:44:43.451 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_filestore_osds = false 2024-09-21T07:44:43.452 INFO:teuthology.orchestra.run.smithi148.stdout: mon down mkfs grace = 300 2024-09-21T07:44:43.452 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:44:43.452 INFO:teuthology.orchestra.run.smithi148.stdout:[client] 2024-09-21T07:44:43.452 INFO:teuthology.orchestra.run.smithi148.stdout: rgw cache enabled = true 2024-09-21T07:44:43.452 INFO:teuthology.orchestra.run.smithi148.stdout: rgw enable ops log = true 2024-09-21T07:44:43.452 INFO:teuthology.orchestra.run.smithi148.stdout: rgw enable usage log = true 2024-09-21T07:44:43.452 INFO:teuthology.orchestra.run.smithi148.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T07:44:43.452 INFO:teuthology.orchestra.run.smithi148.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T07:44:43.452 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = "" 2024-09-21T07:44:43.452 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.a] 2024-09-21T07:44:43.452 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.b] 2024-09-21T07:44:43.452 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.c] 2024-09-21T07:44:43.452 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.a] 2024-09-21T07:44:43.452 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.c] 2024-09-21T07:44:43.453 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.b] 2024-09-21T07:44:43.453 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.d] 2024-09-21T07:44:43.453 INFO:teuthology.orchestra.run.smithi148.stdout:[mds] 2024-09-21T07:44:43.467 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T07:44:43.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.696+0000 7f4874a00640 1 -- 172.21.15.39:0/3894345021 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48700d1580 msgr2=0x7f48700d1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:43.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.696+0000 7f4874a00640 1 --2- 172.21.15.39:0/3894345021 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48700d1580 0x7f48700d1ac0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f485c009830 tx=0x7f485c02a4b0 comp rx=0 tx=0).stop 2024-09-21T07:44:43.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.696+0000 7f4874a00640 1 -- 172.21.15.39:0/3894345021 shutdown_connections 2024-09-21T07:44:43.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.696+0000 7f4874a00640 1 --2- 172.21.15.39:0/3894345021 >> v2:172.21.15.148:3300/0 conn(0x7f48700b3b60 0x7f48700b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:43.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.696+0000 7f4874a00640 1 --2- 172.21.15.39:0/3894345021 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48700d1580 0x7f48700d1ac0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:43.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.696+0000 7f4874a00640 1 -- 172.21.15.39:0/3894345021 >> 172.21.15.39:0/3894345021 conn(0x7f487009d7b0 msgr2=0x7f48700d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:43.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.696+0000 7f4874a00640 1 -- 172.21.15.39:0/3894345021 shutdown_connections 2024-09-21T07:44:43.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.696+0000 7f4874a00640 1 -- 172.21.15.39:0/3894345021 wait complete. 2024-09-21T07:44:43.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.699+0000 7f4874a00640 1 Processor -- start 2024-09-21T07:44:43.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.699+0000 7f4874a00640 1 -- start start 2024-09-21T07:44:43.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.700+0000 7f4874a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48700b3b60 0x7f48701f91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:43.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.700+0000 7f4874a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48700b5a90 0x7f48700ca750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:43.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.700+0000 7f4874a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f487015b800 0x7f48701f8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:43.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.700+0000 7f4874a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f48700b4630 con 0x7f487015b800 2024-09-21T07:44:43.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.700+0000 7f4874a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f48700b44b0 con 0x7f48700b3b60 2024-09-21T07:44:43.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.700+0000 7f4874a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f48700b47b0 con 0x7f48700b5a90 2024-09-21T07:44:43.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.701+0000 7f486f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f487015b800 0x7f48701f8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:43.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.701+0000 7f486de00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48700b5a90 0x7f48700ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:43.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.701+0000 7f486f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f487015b800 0x7f48701f8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46428/0 (socket says 172.21.15.39:46428) 2024-09-21T07:44:43.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.701+0000 7f486e800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48700b3b60 0x7f48701f91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:43.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.701+0000 7f486f200640 1 -- 172.21.15.39:0/2714558001 learned_addr learned my addr 172.21.15.39:0/2714558001 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:43.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.701+0000 7f486f200640 1 -- 172.21.15.39:0/2714558001 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48700b5a90 msgr2=0x7f48700ca750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:43.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.701+0000 7f486f200640 1 --2- 172.21.15.39:0/2714558001 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48700b5a90 0x7f48700ca750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:43.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.701+0000 7f486f200640 1 -- 172.21.15.39:0/2714558001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48700b3b60 msgr2=0x7f48701f91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:43.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.701+0000 7f486f200640 1 --2- 172.21.15.39:0/2714558001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48700b3b60 0x7f48701f91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:43.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.701+0000 7f486f200640 1 -- 172.21.15.39:0/2714558001 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f487015bfe0 con 0x7f487015b800 2024-09-21T07:44:43.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.701+0000 7f486f200640 1 --2- 172.21.15.39:0/2714558001 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f487015b800 0x7f48701f8600 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f486800ec20 tx=0x7f486800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:43.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.702+0000 7f4866a00640 1 -- 172.21.15.39:0/2714558001 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4868003b70 con 0x7f487015b800 2024-09-21T07:44:43.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.702+0000 7f4866a00640 1 -- 172.21.15.39:0/2714558001 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4868003d10 con 0x7f487015b800 2024-09-21T07:44:43.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.702+0000 7f4866a00640 1 -- 172.21.15.39:0/2714558001 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4868004280 con 0x7f487015b800 2024-09-21T07:44:43.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.702+0000 7f4874a00640 1 -- 172.21.15.39:0/2714558001 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f487015c2c0 con 0x7f487015b800 2024-09-21T07:44:43.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.702+0000 7f4874a00640 1 -- 172.21.15.39:0/2714558001 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48700b1640 con 0x7f487015b800 2024-09-21T07:44:43.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.703+0000 7f4874a00640 1 -- 172.21.15.39:0/2714558001 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48700b4c80 con 0x7f487015b800 2024-09-21T07:44:43.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.704+0000 7f4866a00640 1 -- 172.21.15.39:0/2714558001 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4868004420 con 0x7f487015b800 2024-09-21T07:44:43.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.705+0000 7f4866a00640 1 --2- 172.21.15.39:0/2714558001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4850078bc0 0x7f485007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:43.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.705+0000 7f4866a00640 1 -- 172.21.15.39:0/2714558001 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 7711+0+0 (secure 0 0 0) 0x7f486809c0d0 con 0x7f487015b800 2024-09-21T07:44:43.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.705+0000 7f486e800640 1 --2- 172.21.15.39:0/2714558001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4850078bc0 0x7f485007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:43.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.706+0000 7f486e800640 1 --2- 172.21.15.39:0/2714558001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4850078bc0 0x7f485007b0d0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f4860001570 tx=0x7f4860009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:43.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.710+0000 7f4866a00640 1 -- 172.21.15.39:0/2714558001 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4868063650 con 0x7f487015b800 2024-09-21T07:44:43.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.910+0000 7f4874a00640 1 -- 172.21.15.39:0/2714558001 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f48700d16d0 con 0x7f487015b800 2024-09-21T07:44:43.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.914+0000 7f4866a00640 1 -- 172.21.15.39:0/2714558001 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v42) ==== 71+0+0 (secure 0 0 0) 0x7f486809b080 con 0x7f487015b800 2024-09-21T07:44:43.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.917+0000 7f4874a00640 1 -- 172.21.15.39:0/2714558001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4850078bc0 msgr2=0x7f485007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:43.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.917+0000 7f4874a00640 1 --2- 172.21.15.39:0/2714558001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4850078bc0 0x7f485007b0d0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f4860001570 tx=0x7f4860009040 comp rx=0 tx=0).stop 2024-09-21T07:44:43.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.917+0000 7f4874a00640 1 -- 172.21.15.39:0/2714558001 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f487015b800 msgr2=0x7f48701f8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:43.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.917+0000 7f4874a00640 1 --2- 172.21.15.39:0/2714558001 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f487015b800 0x7f48701f8600 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f486800ec20 tx=0x7f486800c6a0 comp rx=0 tx=0).stop 2024-09-21T07:44:43.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.917+0000 7f4874a00640 1 -- 172.21.15.39:0/2714558001 shutdown_connections 2024-09-21T07:44:43.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.917+0000 7f4874a00640 1 --2- 172.21.15.39:0/2714558001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4850078bc0 0x7f485007b0d0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:43.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.917+0000 7f4874a00640 1 --2- 172.21.15.39:0/2714558001 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f487015b800 0x7f48701f8600 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:43.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.917+0000 7f4874a00640 1 --2- 172.21.15.39:0/2714558001 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48700b5a90 0x7f48700ca750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:43.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.917+0000 7f4874a00640 1 --2- 172.21.15.39:0/2714558001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48700b3b60 0x7f48701f91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:43.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.917+0000 7f4874a00640 1 -- 172.21.15.39:0/2714558001 >> 172.21.15.39:0/2714558001 conn(0x7f487009d7b0 msgr2=0x7f487006f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:43.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.917+0000 7f4874a00640 1 -- 172.21.15.39:0/2714558001 shutdown_connections 2024-09-21T07:44:43.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:43.917+0000 7f4874a00640 1 -- 172.21.15.39:0/2714558001 wait complete. 2024-09-21T07:44:43.940 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_kill_mdstable' 2024-09-21T07:44:44.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:44.168+0000 7f0070a00640 1 -- 172.21.15.39:0/1231223768 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f006c0b4480 msgr2=0x7f006c0b4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:44.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:44.168+0000 7f0070a00640 1 --2- 172.21.15.39:0/1231223768 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f006c0b4480 0x7f006c0b4890 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f0064005fd0 tx=0x7f006402aa10 comp rx=0 tx=0).stop 2024-09-21T07:44:44.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:44.168+0000 7f0070a00640 1 -- 172.21.15.39:0/1231223768 shutdown_connections 2024-09-21T07:44:44.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:44.168+0000 7f0070a00640 1 --2- 172.21.15.39:0/1231223768 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f006c0b4480 0x7f006c0b4890 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:44.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:44.168+0000 7f0070a00640 1 --2- 172.21.15.39:0/1231223768 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f006c159010 0x7f006c15c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:44.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:44.168+0000 7f0070a00640 1 -- 172.21.15.39:0/1231223768 >> 172.21.15.39:0/1231223768 conn(0x7f006c06def0 msgr2=0x7f006c06e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:44.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:44.168+0000 7f0070a00640 1 -- 172.21.15.39:0/1231223768 shutdown_connections 2024-09-21T07:44:44.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:44.168+0000 7f0070a00640 1 -- 172.21.15.39:0/1231223768 wait complete. 2024-09-21T07:44:44.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:44.170+0000 7f0070a00640 1 Processor -- start 2024-09-21T07:44:44.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:44.171+0000 7f0070a00640 1 -- start start 2024-09-21T07:44:44.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:44.172+0000 7f0070a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f006c06fe90 0x7f006c14f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:44.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:44.172+0000 7f0070a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f006c0b4480 0x7f006c155f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:44.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:44.172+0000 7f0070a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f006c159010 0x7f006c157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:44.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:44.172+0000 7f0070a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f006c0b5160 con 0x7f006c06fe90 2024-09-21T07:44:44.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:44.172+0000 7f0070a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f006c0b4fe0 con 0x7f006c0b4480 2024-09-21T07:44:44.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:44.172+0000 7f0070a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f006c0b52e0 con 0x7f006c159010 2024-09-21T07:44:44.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:44.172+0000 7f006a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f006c06fe90 0x7f006c14f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:44.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:44.172+0000 7f006a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f006c06fe90 0x7f006c14f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46450/0 (socket says 172.21.15.39:46450) 2024-09-21T07:44:44.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:44.172+0000 7f006a800640 1 -- 172.21.15.39:0/2867638705 learned_addr learned my addr 172.21.15.39:0/2867638705 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:44.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:44.172+0000 7f006b200640 1 --2- 172.21.15.39:0/2867638705 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f006c159010 0x7f006c157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:44.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:44.172+0000 7f0063e00640 1 --2- 172.21.15.39:0/2867638705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f006c0b4480 0x7f006c155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:44.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:44.173+0000 7f006a800640 1 -- 172.21.15.39:0/2867638705 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f006c159010 msgr2=0x7f006c157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:44.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:44.173+0000 7f006a800640 1 --2- 172.21.15.39:0/2867638705 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f006c159010 0x7f006c157bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:44.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:44.173+0000 7f006a800640 1 -- 172.21.15.39:0/2867638705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f006c0b4480 msgr2=0x7f006c155f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:44.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:44.173+0000 7f006a800640 1 --2- 172.21.15.39:0/2867638705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f006c0b4480 0x7f006c155f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:44.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:44.173+0000 7f006a800640 1 -- 172.21.15.39:0/2867638705 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f006c157040 con 0x7f006c06fe90 2024-09-21T07:44:44.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:44.173+0000 7f006a800640 1 --2- 172.21.15.39:0/2867638705 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f006c06fe90 0x7f006c14f160 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f005c00d700 tx=0x7f005c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:44.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:44.173+0000 7f0062a00640 1 -- 172.21.15.39:0/2867638705 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c017070 con 0x7f006c06fe90 2024-09-21T07:44:44.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:44.173+0000 7f0062a00640 1 -- 172.21.15.39:0/2867638705 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f005c005310 con 0x7f006c06fe90 2024-09-21T07:44:44.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:44.173+0000 7f0062a00640 1 -- 172.21.15.39:0/2867638705 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c009820 con 0x7f006c06fe90 2024-09-21T07:44:44.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:44.173+0000 7f0070a00640 1 -- 172.21.15.39:0/2867638705 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f006c156a80 con 0x7f006c06fe90 2024-09-21T07:44:44.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:44.174+0000 7f0070a00640 1 -- 172.21.15.39:0/2867638705 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f006c0cf210 con 0x7f006c06fe90 2024-09-21T07:44:44.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:44.174+0000 7f0070a00640 1 -- 172.21.15.39:0/2867638705 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f006c0b4890 con 0x7f006c06fe90 2024-09-21T07:44:44.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:44.176+0000 7f0062a00640 1 -- 172.21.15.39:0/2867638705 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f005c0099c0 con 0x7f006c06fe90 2024-09-21T07:44:44.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:44.177+0000 7f0062a00640 1 --2- 172.21.15.39:0/2867638705 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0048078b90 0x7f004807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:44.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:44.177+0000 7f0062a00640 1 -- 172.21.15.39:0/2867638705 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 7711+0+0 (secure 0 0 0) 0x7f005c09b640 con 0x7f006c06fe90 2024-09-21T07:44:44.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:44.177+0000 7f0063e00640 1 --2- 172.21.15.39:0/2867638705 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0048078b90 0x7f004807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:44.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:44.178+0000 7f0063e00640 1 --2- 172.21.15.39:0/2867638705 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0048078b90 0x7f004807b0a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f006c15a980 tx=0x7f0054009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:44.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:44.182+0000 7f0062a00640 1 -- 172.21.15.39:0/2867638705 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f005c062bc0 con 0x7f006c06fe90 2024-09-21T07:44:44.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:44.452+0000 7f0070a00640 1 -- 172.21.15.39:0/2867638705 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_kill_mdstable"]} v 0) -- 0x7f006c06dcb0 con 0x7f006c06fe90 2024-09-21T07:44:45.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.305+0000 7f0062a00640 1 -- 172.21.15.39:0/2867638705 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_kill_mdstable"]}]=0 v557) ==== 139+0+0 (secure 0 0 0) 0x7f005c0675f0 con 0x7f006c06fe90 2024-09-21T07:44:45.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.307+0000 7f0070a00640 1 -- 172.21.15.39:0/2867638705 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0048078b90 msgr2=0x7f004807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:45.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.307+0000 7f0070a00640 1 --2- 172.21.15.39:0/2867638705 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0048078b90 0x7f004807b0a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f006c15a980 tx=0x7f0054009040 comp rx=0 tx=0).stop 2024-09-21T07:44:45.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.307+0000 7f0070a00640 1 -- 172.21.15.39:0/2867638705 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f006c06fe90 msgr2=0x7f006c14f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:45.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.307+0000 7f0070a00640 1 --2- 172.21.15.39:0/2867638705 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f006c06fe90 0x7f006c14f160 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f005c00d700 tx=0x7f005c00dbc0 comp rx=0 tx=0).stop 2024-09-21T07:44:45.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.308+0000 7f0070a00640 1 -- 172.21.15.39:0/2867638705 shutdown_connections 2024-09-21T07:44:45.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.308+0000 7f0070a00640 1 --2- 172.21.15.39:0/2867638705 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0048078b90 0x7f004807b0a0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:45.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.308+0000 7f0070a00640 1 --2- 172.21.15.39:0/2867638705 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f006c159010 0x7f006c157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:45.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.308+0000 7f0070a00640 1 --2- 172.21.15.39:0/2867638705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f006c0b4480 0x7f006c155f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:45.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.308+0000 7f0070a00640 1 --2- 172.21.15.39:0/2867638705 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f006c06fe90 0x7f006c14f160 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:45.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.308+0000 7f0070a00640 1 -- 172.21.15.39:0/2867638705 >> 172.21.15.39:0/2867638705 conn(0x7f006c06def0 msgr2=0x7f006c0706f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:45.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.308+0000 7f0070a00640 1 -- 172.21.15.39:0/2867638705 shutdown_connections 2024-09-21T07:44:45.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.308+0000 7f0070a00640 1 -- 172.21.15.39:0/2867638705 wait complete. 2024-09-21T07:44:45.334 INFO:tasks.cephfs_test_runner:check snaptable transcation ... ok 2024-09-21T07:44:45.334 INFO:tasks.cephfs_test_runner:Starting test: test_mds_max_snaps_per_dir_default_limit (tasks.cephfs.test_snapshots.TestSnapshots) Test the newly introudced option named mds_max_snaps_per_dir 2024-09-21T07:44:45.334 INFO:tasks.cephfs_test_runner:test_mds_max_snaps_per_dir_default_limit (tasks.cephfs.test_snapshots.TestSnapshots) 2024-09-21T07:44:45.335 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:44:45.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.563+0000 7f635dc00640 1 -- 172.21.15.39:0/614718867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f633c004800 msgr2=0x7f633c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:45.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.563+0000 7f635dc00640 1 --2- 172.21.15.39:0/614718867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f633c004800 0x7f633c004c10 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f634000c6e0 tx=0x7f633c005830 comp rx=0 tx=0).stop 2024-09-21T07:44:45.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.563+0000 7f635dc00640 1 -- 172.21.15.39:0/614718867 shutdown_connections 2024-09-21T07:44:45.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.563+0000 7f635dc00640 1 --2- 172.21.15.39:0/614718867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f633c004800 0x7f633c004c10 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:45.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.563+0000 7f635dc00640 1 -- 172.21.15.39:0/614718867 >> 172.21.15.39:0/614718867 conn(0x7f635806cfc0 msgr2=0x7f635806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:45.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.563+0000 7f635dc00640 1 -- 172.21.15.39:0/614718867 shutdown_connections 2024-09-21T07:44:45.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.563+0000 7f635dc00640 1 -- 172.21.15.39:0/614718867 wait complete. 2024-09-21T07:44:45.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.566+0000 7f635dc00640 1 Processor -- start 2024-09-21T07:44:45.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.566+0000 7f635dc00640 1 -- start start 2024-09-21T07:44:45.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.567+0000 7f635dc00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f633c004800 0x7f63581f5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:45.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.567+0000 7f635dc00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f635814e1c0 0x7f6358159560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:45.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.567+0000 7f635dc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f635815a880 0x7f635814bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:45.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.567+0000 7f635dc00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6358162880 con 0x7f635815a880 2024-09-21T07:44:45.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.567+0000 7f635dc00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6358162700 con 0x7f633c004800 2024-09-21T07:44:45.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.567+0000 7f635dc00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f6358162a00 con 0x7f635814e1c0 2024-09-21T07:44:45.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.568+0000 7f6357400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f635815a880 0x7f635814bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:45.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.568+0000 7f6357400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f635815a880 0x7f635814bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46480/0 (socket says 172.21.15.39:46480) 2024-09-21T07:44:45.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.568+0000 7f6357400640 1 -- 172.21.15.39:0/54264438 learned_addr learned my addr 172.21.15.39:0/54264438 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:45.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.568+0000 7f6356a00640 1 --2- 172.21.15.39:0/54264438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f633c004800 0x7f63581f5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:45.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.568+0000 7f634fe00640 1 --2- 172.21.15.39:0/54264438 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f635814e1c0 0x7f6358159560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:45.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.568+0000 7f6357400640 1 -- 172.21.15.39:0/54264438 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f635814e1c0 msgr2=0x7f6358159560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:45.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.568+0000 7f6357400640 1 --2- 172.21.15.39:0/54264438 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f635814e1c0 0x7f6358159560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:45.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.568+0000 7f6357400640 1 -- 172.21.15.39:0/54264438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f633c004800 msgr2=0x7f63581f5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:45.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.568+0000 7f6357400640 1 --2- 172.21.15.39:0/54264438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f633c004800 0x7f63581f5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:45.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.568+0000 7f6357400640 1 -- 172.21.15.39:0/54264438 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f635814cef0 con 0x7f635815a880 2024-09-21T07:44:45.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.568+0000 7f6357400640 1 --2- 172.21.15.39:0/54264438 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f635815a880 0x7f635814bf10 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f6358072f00 tx=0x7f6350004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:45.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.569+0000 7f634ea00640 1 -- 172.21.15.39:0/54264438 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6350004b20 con 0x7f635815a880 2024-09-21T07:44:45.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.569+0000 7f634ea00640 1 -- 172.21.15.39:0/54264438 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6350004cc0 con 0x7f635815a880 2024-09-21T07:44:45.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.569+0000 7f635dc00640 1 -- 172.21.15.39:0/54264438 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f635814d1d0 con 0x7f635815a880 2024-09-21T07:44:45.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.569+0000 7f634ea00640 1 -- 172.21.15.39:0/54264438 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6350010d10 con 0x7f635815a880 2024-09-21T07:44:45.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.569+0000 7f635dc00640 1 -- 172.21.15.39:0/54264438 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f635814b710 con 0x7f635815a880 2024-09-21T07:44:45.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.570+0000 7f635dc00640 1 -- 172.21.15.39:0/54264438 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63580d0930 con 0x7f635815a880 2024-09-21T07:44:45.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.571+0000 7f634ea00640 1 -- 172.21.15.39:0/54264438 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f635001c020 con 0x7f635815a880 2024-09-21T07:44:45.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.572+0000 7f634ea00640 1 --2- 172.21.15.39:0/54264438 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6334078b90 0x7f633407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:45.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.572+0000 7f634ea00640 1 -- 172.21.15.39:0/54264438 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 7711+0+0 (secure 0 0 0) 0x7f635009bb40 con 0x7f635815a880 2024-09-21T07:44:45.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.573+0000 7f6356a00640 1 --2- 172.21.15.39:0/54264438 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6334078b90 0x7f633407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:45.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.574+0000 7f6356a00640 1 --2- 172.21.15.39:0/54264438 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6334078b90 0x7f633407b0a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6348001050 tx=0x7f63480067a0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:45.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.577+0000 7f634ea00640 1 -- 172.21.15.39:0/54264438 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6350063150 con 0x7f635815a880 2024-09-21T07:44:45.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.773+0000 7f635dc00640 1 -- 172.21.15.39:0/54264438 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6358070fd0 con 0x7f635815a880 2024-09-21T07:44:45.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.775+0000 7f634ea00640 1 -- 172.21.15.39:0/54264438 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) ==== 74+0+21000 (secure 0 0 0) 0x7f6350067b80 con 0x7f635815a880 2024-09-21T07:44:45.777 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:45.777 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":126,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:44:43.289724+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6835/1551311805":"2024-09-22T07:44:40.192595+0000","172.21.15.39:6834/1551311805":"2024-09-22T07:44:40.192595+0000","172.21.15.148:6837/2582569093":"2024-09-22T07:44:40.192571+0000","172.21.15.39:6832/1790581246":"2024-09-22T07:44:24.184881+0000","172.21.15.39:0/1731243684":"2024-09-21T08:44:16.323688+0000","172.21.15.39:6835/1326711417":"2024-09-22T07:43:26.076277+0000","172.21.15.39:6834/1326711417":"2024-09-22T07:43:26.076277+0000","172.21.15.39:6835/364657398":"2024-09-22T07:39:15.579613+0000","192.168.0.1:0/177249410":"2024-09-21T08:41:13.882767+0000","172.21.15.148:6837/73220122":"2024-09-22T07:42:40.074859+0000","172.21.15.39:0/596665086":"2024-09-21T08:43:35.317820+0000","172.21.15.39:6832/829716143":"2024-09-22T07:37:01.259219+0000","172.21.15.39:6832/855150775":"2024-09-22T07:39:07.739045+0000","172.21.15.148:6837/2381504293":"2024-09-22T07:38:32.207414+0000","172.21.15.148:6836/2582569093":"2024-09-22T07:44:40.192571+0000","172.21.15.39:6834/855150775":"2024-09-22T07:39:07.739045+0000","172.21.15.39:0/742594435":"2024-09-21T08:42:01.225749+0000","172.21.15.39:6835/2821446711":"2024-09-22T07:40:57.372409+0000","192.168.0.1:0/90671776":"2024-09-21T08:39:22.988045+0000","172.21.15.148:6836/759792812":"2024-09-22T07:44:05.615021+0000","172.21.15.39:6834/1472430787":"2024-09-22T07:37:58.212892+0000","172.21.15.39:6834/3695489158":"2024-09-22T07:41:51.807018+0000","172.21.15.148:6836/3026827614":"2024-09-22T07:37:29.781472+0000","172.21.15.148:6837/3026827614":"2024-09-22T07:37:29.781472+0000","172.21.15.39:6834/829716143":"2024-09-22T07:37:01.259219+0000","172.21.15.39:6835/3695489158":"2024-09-22T07:41:51.807018+0000","172.21.15.39:6832/1472430787":"2024-09-22T07:37:58.212892+0000","172.21.15.148:6837/759792812":"2024-09-22T07:44:05.615021+0000","172.21.15.148:6836/992188984":"2024-09-22T07:40:00.778049+0000","172.21.15.39:6833/364657398":"2024-09-22T07:39:15.579613+0000","172.21.15.148:6837/992188984":"2024-09-22T07:40:00.778049+0000","172.21.15.39:6833/1790581246":"2024-09-22T07:44:24.184881+0000","172.21.15.39:6833/1456343023":"2024-09-22T07:40:08.637993+0000","172.21.15.148:6836/2381504293":"2024-09-22T07:38:32.207414+0000","172.21.15.148:6836/2336112936":"2024-09-22T07:41:05.979253+0000","172.21.15.39:6832/1456343023":"2024-09-22T07:40:08.637993+0000","172.21.15.39:6834/2821446711":"2024-09-22T07:40:57.372409+0000","172.21.15.39:0/3356393880":"2024-09-21T08:40:15.824973+0000","172.21.15.148:6837/2336112936":"2024-09-22T07:41:05.979253+0000","172.21.15.148:6836/73220122":"2024-09-22T07:42:40.074859+0000","172.21.15.39:0/1669604593":"2024-09-21T08:42:52.059256+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:44:45.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.777+0000 7f635dc00640 1 -- 172.21.15.39:0/54264438 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6334078b90 msgr2=0x7f633407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:45.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.777+0000 7f635dc00640 1 --2- 172.21.15.39:0/54264438 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6334078b90 0x7f633407b0a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6348001050 tx=0x7f63480067a0 comp rx=0 tx=0).stop 2024-09-21T07:44:45.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.777+0000 7f635dc00640 1 -- 172.21.15.39:0/54264438 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f635815a880 msgr2=0x7f635814bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:45.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.777+0000 7f635dc00640 1 --2- 172.21.15.39:0/54264438 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f635815a880 0x7f635814bf10 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f6358072f00 tx=0x7f6350004700 comp rx=0 tx=0).stop 2024-09-21T07:44:45.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.777+0000 7f635dc00640 1 -- 172.21.15.39:0/54264438 shutdown_connections 2024-09-21T07:44:45.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.777+0000 7f635dc00640 1 --2- 172.21.15.39:0/54264438 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6334078b90 0x7f633407b0a0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:45.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.777+0000 7f635dc00640 1 --2- 172.21.15.39:0/54264438 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f635815a880 0x7f635814bf10 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:45.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.777+0000 7f635dc00640 1 --2- 172.21.15.39:0/54264438 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f635814e1c0 0x7f6358159560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:45.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.777+0000 7f635dc00640 1 --2- 172.21.15.39:0/54264438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f633c004800 0x7f63581f5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:45.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.777+0000 7f635dc00640 1 -- 172.21.15.39:0/54264438 >> 172.21.15.39:0/54264438 conn(0x7f635806cfc0 msgr2=0x7f635815df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:45.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.777+0000 7f635dc00640 1 -- 172.21.15.39:0/54264438 shutdown_connections 2024-09-21T07:44:45.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:45.777+0000 7f635dc00640 1 -- 172.21.15.39:0/54264438 wait complete. 2024-09-21T07:44:45.801 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T07:33:01.569433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'snap-test-pool', 'create_time': '2024-09-21T07:34:22.032185+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 1, 'snap_epoch': 37, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'test_data_pool', 'create_time': '2024-09-21T07:35:25.604532+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 1, 'snap_epoch': 66, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_metadata_pool', 'create_time': '2024-09-21T07:35:26.327607+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T07:44:45.801 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T07:44:46.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.031+0000 7f9517a00640 1 -- 172.21.15.39:0/1450716465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95100b2740 msgr2=0x7f951015b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:46.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.031+0000 7f9517a00640 1 --2- 172.21.15.39:0/1450716465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95100b2740 0x7f951015b650 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f9510071e60 tx=0x7f950002a540 comp rx=0 tx=0).stop 2024-09-21T07:44:46.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.031+0000 7f9517a00640 1 -- 172.21.15.39:0/1450716465 shutdown_connections 2024-09-21T07:44:46.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.031+0000 7f9517a00640 1 --2- 172.21.15.39:0/1450716465 >> v2:172.21.15.148:3300/0 conn(0x7f951015fbc0 0x7f95100b30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:46.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.031+0000 7f9517a00640 1 --2- 172.21.15.39:0/1450716465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95100b2740 0x7f951015b650 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:46.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.031+0000 7f9517a00640 1 -- 172.21.15.39:0/1450716465 >> 172.21.15.39:0/1450716465 conn(0x7f951006d7a0 msgr2=0x7f95100b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:46.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.031+0000 7f9517a00640 1 -- 172.21.15.39:0/1450716465 shutdown_connections 2024-09-21T07:44:46.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.031+0000 7f9517a00640 1 -- 172.21.15.39:0/1450716465 wait complete. 2024-09-21T07:44:46.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.033+0000 7f9517a00640 1 Processor -- start 2024-09-21T07:44:46.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.033+0000 7f9517a00640 1 -- start start 2024-09-21T07:44:46.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.035+0000 7f9517a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9510074500 0x7f95101f2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:46.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.035+0000 7f9517a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95100b2740 0x7f95100b1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:46.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.035+0000 7f9517a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f951015fbc0 0x7f95100b3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:46.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.035+0000 7f9517a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f95100af390 con 0x7f9510074500 2024-09-21T07:44:46.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.035+0000 7f9517a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f95100af210 con 0x7f951015fbc0 2024-09-21T07:44:46.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.035+0000 7f9517a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f95100af510 con 0x7f95100b2740 2024-09-21T07:44:46.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.035+0000 7f9515000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9510074500 0x7f95101f2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:46.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.035+0000 7f9515000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9510074500 0x7f95101f2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46504/0 (socket says 172.21.15.39:46504) 2024-09-21T07:44:46.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.035+0000 7f9515000640 1 -- 172.21.15.39:0/3711058267 learned_addr learned my addr 172.21.15.39:0/3711058267 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:46.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.035+0000 7f9507e00640 1 --2- 172.21.15.39:0/3711058267 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95100b2740 0x7f95100b1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:46.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.035+0000 7f9515a00640 1 --2- 172.21.15.39:0/3711058267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f951015fbc0 0x7f95100b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:46.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.035+0000 7f9515000640 1 -- 172.21.15.39:0/3711058267 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95100b2740 msgr2=0x7f95100b1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:46.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.036+0000 7f9515000640 1 --2- 172.21.15.39:0/3711058267 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95100b2740 0x7f95100b1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:46.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.036+0000 7f9515000640 1 -- 172.21.15.39:0/3711058267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f951015fbc0 msgr2=0x7f95100b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:46.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.036+0000 7f9515000640 1 --2- 172.21.15.39:0/3711058267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f951015fbc0 0x7f95100b3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:46.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.036+0000 7f9515000640 1 -- 172.21.15.39:0/3711058267 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95100b19d0 con 0x7f9510074500 2024-09-21T07:44:46.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.036+0000 7f9507e00640 1 --2- 172.21.15.39:0/3711058267 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95100b2740 0x7f95100b1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:44:46.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.036+0000 7f9515a00640 1 --2- 172.21.15.39:0/3711058267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f951015fbc0 0x7f95100b3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:44:46.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.036+0000 7f9515000640 1 --2- 172.21.15.39:0/3711058267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9510074500 0x7f95101f2960 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f950002ab30 tx=0x7f9500033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:46.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.036+0000 7f950d600640 1 -- 172.21.15.39:0/3711058267 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9500031d60 con 0x7f9510074500 2024-09-21T07:44:46.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.036+0000 7f950d600640 1 -- 172.21.15.39:0/3711058267 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9500033e10 con 0x7f9510074500 2024-09-21T07:44:46.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.036+0000 7f9517a00640 1 -- 172.21.15.39:0/3711058267 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95100c9610 con 0x7f9510074500 2024-09-21T07:44:46.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.036+0000 7f950d600640 1 -- 172.21.15.39:0/3711058267 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9500005260 con 0x7f9510074500 2024-09-21T07:44:46.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.036+0000 7f9517a00640 1 -- 172.21.15.39:0/3711058267 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95100c9bd0 con 0x7f9510074500 2024-09-21T07:44:46.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.037+0000 7f9517a00640 1 -- 172.21.15.39:0/3711058267 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95100af9e0 con 0x7f9510074500 2024-09-21T07:44:46.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.039+0000 7f950d600640 1 -- 172.21.15.39:0/3711058267 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9500005400 con 0x7f9510074500 2024-09-21T07:44:46.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.040+0000 7f950d600640 1 --2- 172.21.15.39:0/3711058267 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f94ec078b90 0x7f94ec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:46.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.040+0000 7f950d600640 1 -- 172.21.15.39:0/3711058267 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 7711+0+0 (secure 0 0 0) 0x7f95000bc780 con 0x7f9510074500 2024-09-21T07:44:46.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.040+0000 7f9507e00640 1 --2- 172.21.15.39:0/3711058267 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f94ec078b90 0x7f94ec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:46.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.041+0000 7f9507e00640 1 --2- 172.21.15.39:0/3711058267 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f94ec078b90 0x7f94ec07b0a0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f95100cfba0 tx=0x7f94f8005dd0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:46.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.044+0000 7f950d600640 1 -- 172.21.15.39:0/3711058267 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f950003f030 con 0x7f9510074500 2024-09-21T07:44:46.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.239+0000 7f9517a00640 1 -- 172.21.15.39:0/3711058267 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f951015f820 con 0x7f9510074500 2024-09-21T07:44:46.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.239+0000 7f950d600640 1 -- 172.21.15.39:0/3711058267 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v126) ==== 93+0+10 (secure 0 0 0) 0x7f9500005d00 con 0x7f9510074500 2024-09-21T07:44:46.242 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-09-21T07:44:46.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.242+0000 7f9517a00640 1 -- 172.21.15.39:0/3711058267 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f94ec078b90 msgr2=0x7f94ec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:46.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.242+0000 7f9517a00640 1 --2- 172.21.15.39:0/3711058267 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f94ec078b90 0x7f94ec07b0a0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f95100cfba0 tx=0x7f94f8005dd0 comp rx=0 tx=0).stop 2024-09-21T07:44:46.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.242+0000 7f9517a00640 1 -- 172.21.15.39:0/3711058267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9510074500 msgr2=0x7f95101f2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:46.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.242+0000 7f9517a00640 1 --2- 172.21.15.39:0/3711058267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9510074500 0x7f95101f2960 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f950002ab30 tx=0x7f9500033c20 comp rx=0 tx=0).stop 2024-09-21T07:44:46.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.242+0000 7f9517a00640 1 -- 172.21.15.39:0/3711058267 shutdown_connections 2024-09-21T07:44:46.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.242+0000 7f9517a00640 1 --2- 172.21.15.39:0/3711058267 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f94ec078b90 0x7f94ec07b0a0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:46.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.242+0000 7f9517a00640 1 --2- 172.21.15.39:0/3711058267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f951015fbc0 0x7f95100b3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:46.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.242+0000 7f9517a00640 1 --2- 172.21.15.39:0/3711058267 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95100b2740 0x7f95100b1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:46.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.242+0000 7f9517a00640 1 --2- 172.21.15.39:0/3711058267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9510074500 0x7f95101f2960 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:46.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.242+0000 7f9517a00640 1 -- 172.21.15.39:0/3711058267 >> 172.21.15.39:0/3711058267 conn(0x7f951006d7a0 msgr2=0x7f95100cf2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:46.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.242+0000 7f9517a00640 1 -- 172.21.15.39:0/3711058267 shutdown_connections 2024-09-21T07:44:46.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.242+0000 7f9517a00640 1 -- 172.21.15.39:0/3711058267 wait complete. 2024-09-21T07:44:46.269 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2024-09-21T07:44:46.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.501+0000 7f9348800640 1 -- 172.21.15.39:0/2252155394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9328004800 msgr2=0x7f9328004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:46.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.501+0000 7f9348800640 1 --2- 172.21.15.39:0/2252155394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9328004800 0x7f9328004c10 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f933000c470 tx=0x7f9328005830 comp rx=0 tx=0).stop 2024-09-21T07:44:46.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.501+0000 7f9346800640 1 -- 172.21.15.39:0/2252155394 reap_dead start 2024-09-21T07:44:46.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.502+0000 7f9348800640 1 -- 172.21.15.39:0/2252155394 shutdown_connections 2024-09-21T07:44:46.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.502+0000 7f9348800640 1 -- 172.21.15.39:0/2252155394 >> 172.21.15.39:0/2252155394 conn(0x7f93400b2460 msgr2=0x7f934006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:46.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.502+0000 7f9348800640 1 -- 172.21.15.39:0/2252155394 shutdown_connections 2024-09-21T07:44:46.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.502+0000 7f9348800640 1 -- 172.21.15.39:0/2252155394 wait complete. 2024-09-21T07:44:46.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.504+0000 7f9348800640 1 Processor -- start 2024-09-21T07:44:46.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.504+0000 7f9348800640 1 -- start start 2024-09-21T07:44:46.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.506+0000 7f9348800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9328004800 0x7f93400c4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:46.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.506+0000 7f9348800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9340154230 0x7f9340099ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:46.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.506+0000 7f9348800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93401f48c0 0x7f93400c69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:46.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.506+0000 7f9348800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f93400af080 con 0x7f9328004800 2024-09-21T07:44:46.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.506+0000 7f9348800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f93400cd3c0 con 0x7f93401f48c0 2024-09-21T07:44:46.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.506+0000 7f9348800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f93400af200 con 0x7f9340154230 2024-09-21T07:44:46.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.506+0000 7f9345e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9328004800 0x7f93400c4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:46.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.506+0000 7f9345e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9328004800 0x7f93400c4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46520/0 (socket says 172.21.15.39:46520) 2024-09-21T07:44:46.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.506+0000 7f9345e00640 1 -- 172.21.15.39:0/336345639 learned_addr learned my addr 172.21.15.39:0/336345639 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:46.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.506+0000 7f9346800640 1 --2- 172.21.15.39:0/336345639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93401f48c0 0x7f93400c69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:46.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.506+0000 7f9345400640 1 --2- 172.21.15.39:0/336345639 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9340154230 0x7f9340099ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:46.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.506+0000 7f9345e00640 1 -- 172.21.15.39:0/336345639 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9340154230 msgr2=0x7f9340099ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:46.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.506+0000 7f9345e00640 1 --2- 172.21.15.39:0/336345639 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9340154230 0x7f9340099ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:46.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.506+0000 7f9345e00640 1 -- 172.21.15.39:0/336345639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93401f48c0 msgr2=0x7f93400c69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:46.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.506+0000 7f9345e00640 1 --2- 172.21.15.39:0/336345639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93401f48c0 0x7f93400c69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:46.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.506+0000 7f9345e00640 1 -- 172.21.15.39:0/336345639 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93401f4460 con 0x7f9328004800 2024-09-21T07:44:46.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.506+0000 7f9345400640 1 --2- 172.21.15.39:0/336345639 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9340154230 0x7f9340099ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:44:46.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.507+0000 7f9345e00640 1 --2- 172.21.15.39:0/336345639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9328004800 0x7f93400c4390 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f9334004c90 tx=0x7f9334001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:46.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.507+0000 7f933a000640 1 -- 172.21.15.39:0/336345639 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93340107d0 con 0x7f9328004800 2024-09-21T07:44:46.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.507+0000 7f933a000640 1 -- 172.21.15.39:0/336345639 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9334016070 con 0x7f9328004800 2024-09-21T07:44:46.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.507+0000 7f9348800640 1 -- 172.21.15.39:0/336345639 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93400c5f00 con 0x7f9328004800 2024-09-21T07:44:46.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.507+0000 7f933a000640 1 -- 172.21.15.39:0/336345639 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9334010ce0 con 0x7f9328004800 2024-09-21T07:44:46.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.507+0000 7f9348800640 1 -- 172.21.15.39:0/336345639 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93400c64c0 con 0x7f9328004800 2024-09-21T07:44:46.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.508+0000 7f9348800640 1 -- 172.21.15.39:0/336345639 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93400af680 con 0x7f9328004800 2024-09-21T07:44:46.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.509+0000 7f933a000640 1 -- 172.21.15.39:0/336345639 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f933401b460 con 0x7f9328004800 2024-09-21T07:44:46.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.510+0000 7f933a000640 1 --2- 172.21.15.39:0/336345639 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9324078b90 0x7f932407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:46.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.510+0000 7f933a000640 1 -- 172.21.15.39:0/336345639 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 7711+0+0 (secure 0 0 0) 0x7f933409bd90 con 0x7f9328004800 2024-09-21T07:44:46.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.511+0000 7f9345400640 1 --2- 172.21.15.39:0/336345639 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9324078b90 0x7f932407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:46.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.512+0000 7f9345400640 1 --2- 172.21.15.39:0/336345639 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9324078b90 0x7f932407b0a0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f9330033000 tx=0x7f933002d070 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:46.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.515+0000 7f933a000640 1 -- 172.21.15.39:0/336345639 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9334063a00 con 0x7f9328004800 2024-09-21T07:44:46.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.710+0000 7f9348800640 1 -- 172.21.15.39:0/336345639 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) -- 0x7f934015c970 con 0x7f9328004800 2024-09-21T07:44:46.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.711+0000 7f933a000640 1 -- 172.21.15.39:0/336345639 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v126) ==== 103+0+11 (secure 0 0 0) 0x7f9334063300 con 0x7f9328004800 2024-09-21T07:44:46.713 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-21T07:44:46.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.713+0000 7f9348800640 1 -- 172.21.15.39:0/336345639 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9324078b90 msgr2=0x7f932407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:46.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.713+0000 7f9348800640 1 --2- 172.21.15.39:0/336345639 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9324078b90 0x7f932407b0a0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f9330033000 tx=0x7f933002d070 comp rx=0 tx=0).stop 2024-09-21T07:44:46.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.713+0000 7f9348800640 1 -- 172.21.15.39:0/336345639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9328004800 msgr2=0x7f93400c4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:46.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.713+0000 7f9348800640 1 --2- 172.21.15.39:0/336345639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9328004800 0x7f93400c4390 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f9334004c90 tx=0x7f9334001800 comp rx=0 tx=0).stop 2024-09-21T07:44:46.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.713+0000 7f9348800640 1 -- 172.21.15.39:0/336345639 shutdown_connections 2024-09-21T07:44:46.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.713+0000 7f9348800640 1 --2- 172.21.15.39:0/336345639 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9324078b90 0x7f932407b0a0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:46.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.713+0000 7f9348800640 1 --2- 172.21.15.39:0/336345639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93401f48c0 0x7f93400c69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:46.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.713+0000 7f9348800640 1 --2- 172.21.15.39:0/336345639 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9340154230 0x7f9340099ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:46.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.713+0000 7f9348800640 1 --2- 172.21.15.39:0/336345639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9328004800 0x7f93400c4390 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:46.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.713+0000 7f9348800640 1 -- 172.21.15.39:0/336345639 >> 172.21.15.39:0/336345639 conn(0x7f93400b2460 msgr2=0x7f93400d0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:46.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.714+0000 7f9348800640 1 -- 172.21.15.39:0/336345639 shutdown_connections 2024-09-21T07:44:46.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.714+0000 7f9348800640 1 -- 172.21.15.39:0/336345639 wait complete. 2024-09-21T07:44:46.738 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2024-09-21T07:44:46.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.967+0000 7f87f7200640 1 -- 172.21.15.39:0/3575614638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87f80cfa20 msgr2=0x7f87f80cd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:46.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.967+0000 7f87f7200640 1 --2- 172.21.15.39:0/3575614638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87f80cfa20 0x7f87f80cd5a0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f87f806dcc0 tx=0x7f87e802a530 comp rx=0 tx=0).stop 2024-09-21T07:44:46.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.967+0000 7f87f7200640 1 -- 172.21.15.39:0/3575614638 shutdown_connections 2024-09-21T07:44:46.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.967+0000 7f87f7200640 1 --2- 172.21.15.39:0/3575614638 >> v2:172.21.15.148:3300/0 conn(0x7f87f815cc00 0x7f87f815d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:46.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.967+0000 7f87f7200640 1 --2- 172.21.15.39:0/3575614638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87f80cfa20 0x7f87f80cd5a0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:46.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.967+0000 7f87f7200640 1 -- 172.21.15.39:0/3575614638 >> 172.21.15.39:0/3575614638 conn(0x7f87f806cfc0 msgr2=0x7f87f806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:46.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.967+0000 7f87f7200640 1 -- 172.21.15.39:0/3575614638 shutdown_connections 2024-09-21T07:44:46.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.967+0000 7f87f7200640 1 -- 172.21.15.39:0/3575614638 wait complete. 2024-09-21T07:44:46.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.969+0000 7f87f7200640 1 Processor -- start 2024-09-21T07:44:46.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.969+0000 7f87f7200640 1 -- start start 2024-09-21T07:44:46.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.971+0000 7f87f7200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87f80cfa20 0x7f87f80a9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:46.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.971+0000 7f87f7200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87f80d04b0 0x7f87f80abb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:46.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.971+0000 7f87f7200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87f815cc00 0x7f87f815b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:46.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.971+0000 7f87f7200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f87f806ee70 con 0x7f87f80d04b0 2024-09-21T07:44:46.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.971+0000 7f87f7200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f87f806ecf0 con 0x7f87f815cc00 2024-09-21T07:44:46.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.971+0000 7f87f7200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f87f806eff0 con 0x7f87f80cfa20 2024-09-21T07:44:46.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.971+0000 7f87efe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87f80d04b0 0x7f87f80abb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:46.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.971+0000 7f87efe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87f80d04b0 0x7f87f80abb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46532/0 (socket says 172.21.15.39:46532) 2024-09-21T07:44:46.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.971+0000 7f87efe00640 1 -- 172.21.15.39:0/18044973 learned_addr learned my addr 172.21.15.39:0/18044973 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:46.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.971+0000 7f87f5e00640 1 --2- 172.21.15.39:0/18044973 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87f80cfa20 0x7f87f80a9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:46.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.971+0000 7f87f6800640 1 --2- 172.21.15.39:0/18044973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87f815cc00 0x7f87f815b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:46.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.972+0000 7f87efe00640 1 -- 172.21.15.39:0/18044973 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87f80cfa20 msgr2=0x7f87f80a9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:46.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.972+0000 7f87efe00640 1 --2- 172.21.15.39:0/18044973 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87f80cfa20 0x7f87f80a9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:46.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.972+0000 7f87efe00640 1 -- 172.21.15.39:0/18044973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87f815cc00 msgr2=0x7f87f815b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:46.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.972+0000 7f87efe00640 1 --2- 172.21.15.39:0/18044973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87f815cc00 0x7f87f815b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:46.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.972+0000 7f87efe00640 1 -- 172.21.15.39:0/18044973 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f80a9eb0 con 0x7f87f80d04b0 2024-09-21T07:44:46.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.972+0000 7f87efe00640 1 --2- 172.21.15.39:0/18044973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87f80d04b0 0x7f87f80abb80 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f87e000e7c0 tx=0x7f87e000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:46.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.972+0000 7f87ee000640 1 -- 172.21.15.39:0/18044973 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e000ccc0 con 0x7f87f80d04b0 2024-09-21T07:44:46.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.972+0000 7f87ee000640 1 -- 172.21.15.39:0/18044973 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87e000ce80 con 0x7f87f80d04b0 2024-09-21T07:44:46.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.972+0000 7f87ee000640 1 -- 172.21.15.39:0/18044973 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e0005280 con 0x7f87f80d04b0 2024-09-21T07:44:46.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.972+0000 7f87f7200640 1 -- 172.21.15.39:0/18044973 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87f815a6f0 con 0x7f87f80d04b0 2024-09-21T07:44:46.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.972+0000 7f87f7200640 1 -- 172.21.15.39:0/18044973 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87f815acb0 con 0x7f87f80d04b0 2024-09-21T07:44:46.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.974+0000 7f87f7200640 1 -- 172.21.15.39:0/18044973 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87f806f480 con 0x7f87f80d04b0 2024-09-21T07:44:46.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.975+0000 7f87ee000640 1 -- 172.21.15.39:0/18044973 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f87e0005420 con 0x7f87f80d04b0 2024-09-21T07:44:46.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.976+0000 7f87ee000640 1 --2- 172.21.15.39:0/18044973 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f87d4078b90 0x7f87d407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:46.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.976+0000 7f87ee000640 1 -- 172.21.15.39:0/18044973 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 7711+0+0 (secure 0 0 0) 0x7f87e009bde0 con 0x7f87f80d04b0 2024-09-21T07:44:46.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.977+0000 7f87f5e00640 1 --2- 172.21.15.39:0/18044973 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f87d4078b90 0x7f87d407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:46.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.978+0000 7f87f5e00640 1 --2- 172.21.15.39:0/18044973 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f87d4078b90 0x7f87d407b0a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f87e8034000 tx=0x7f87e8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:46.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:46.981+0000 7f87ee000640 1 -- 172.21.15.39:0/18044973 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f87e0063360 con 0x7f87f80d04b0 2024-09-21T07:44:47.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.177+0000 7f87f7200640 1 -- 172.21.15.39:0/18044973 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) -- 0x7f87f80d0c40 con 0x7f87f80d04b0 2024-09-21T07:44:47.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.178+0000 7f87ee000640 1 -- 172.21.15.39:0/18044973 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v126) ==== 103+0+11 (secure 0 0 0) 0x7f87e0067d90 con 0x7f87f80d04b0 2024-09-21T07:44:47.180 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-21T07:44:47.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.180+0000 7f87f7200640 1 -- 172.21.15.39:0/18044973 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f87d4078b90 msgr2=0x7f87d407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:47.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.180+0000 7f87f7200640 1 --2- 172.21.15.39:0/18044973 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f87d4078b90 0x7f87d407b0a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f87e8034000 tx=0x7f87e8004730 comp rx=0 tx=0).stop 2024-09-21T07:44:47.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.180+0000 7f87f7200640 1 -- 172.21.15.39:0/18044973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87f80d04b0 msgr2=0x7f87f80abb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:47.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.180+0000 7f87f7200640 1 --2- 172.21.15.39:0/18044973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87f80d04b0 0x7f87f80abb80 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f87e000e7c0 tx=0x7f87e000ec80 comp rx=0 tx=0).stop 2024-09-21T07:44:47.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.181+0000 7f87f7200640 1 -- 172.21.15.39:0/18044973 shutdown_connections 2024-09-21T07:44:47.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.181+0000 7f87f7200640 1 --2- 172.21.15.39:0/18044973 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f87d4078b90 0x7f87d407b0a0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:47.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.181+0000 7f87f7200640 1 --2- 172.21.15.39:0/18044973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87f815cc00 0x7f87f815b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:47.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.181+0000 7f87f7200640 1 --2- 172.21.15.39:0/18044973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87f80d04b0 0x7f87f80abb80 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:47.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.181+0000 7f87f7200640 1 --2- 172.21.15.39:0/18044973 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87f80cfa20 0x7f87f80a9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:47.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.181+0000 7f87f7200640 1 -- 172.21.15.39:0/18044973 >> 172.21.15.39:0/18044973 conn(0x7f87f806cfc0 msgr2=0x7f87f8154af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:47.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.181+0000 7f87f7200640 1 -- 172.21.15.39:0/18044973 shutdown_connections 2024-09-21T07:44:47.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.181+0000 7f87f7200640 1 -- 172.21.15.39:0/18044973 wait complete. 2024-09-21T07:44:47.207 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2024-09-21T07:44:47.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.434+0000 7f2127e00640 1 -- 172.21.15.39:0/2840747957 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21280b3580 msgr2=0x7f21280aed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:47.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.434+0000 7f2127e00640 1 --2- 172.21.15.39:0/2840747957 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21280b3580 0x7f21280aed10 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f2128075030 tx=0x7f212002aa30 comp rx=0 tx=0).stop 2024-09-21T07:44:47.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.434+0000 7f2127e00640 1 -- 172.21.15.39:0/2840747957 shutdown_connections 2024-09-21T07:44:47.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.434+0000 7f2127e00640 1 --2- 172.21.15.39:0/2840747957 >> v2:172.21.15.148:3300/0 conn(0x7f212815f830 0x7f21280b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:47.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.434+0000 7f2127e00640 1 --2- 172.21.15.39:0/2840747957 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21280b3580 0x7f21280aed10 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:47.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.434+0000 7f2127e00640 1 -- 172.21.15.39:0/2840747957 >> 172.21.15.39:0/2840747957 conn(0x7f212806cfc0 msgr2=0x7f212806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:47.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.434+0000 7f2127e00640 1 -- 172.21.15.39:0/2840747957 shutdown_connections 2024-09-21T07:44:47.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.434+0000 7f2127e00640 1 -- 172.21.15.39:0/2840747957 wait complete. 2024-09-21T07:44:47.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.437+0000 7f2127e00640 1 Processor -- start 2024-09-21T07:44:47.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.437+0000 7f2127e00640 1 -- start start 2024-09-21T07:44:47.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.438+0000 7f2127e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21280b3580 0x7f2128143cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:47.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.438+0000 7f2127e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2128155880 0x7f21281e8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:47.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.438+0000 7f2127e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f212815f830 0x7f21280b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:47.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.438+0000 7f2127e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f21280af5e0 con 0x7f21280b3580 2024-09-21T07:44:47.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.438+0000 7f2127e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f21280af460 con 0x7f2128155880 2024-09-21T07:44:47.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.438+0000 7f2127e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f21280af760 con 0x7f212815f830 2024-09-21T07:44:47.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.439+0000 7f2126a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21280b3580 0x7f2128143cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:47.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.439+0000 7f2126a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21280b3580 0x7f2128143cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46548/0 (socket says 172.21.15.39:46548) 2024-09-21T07:44:47.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.439+0000 7f2126a00640 1 -- 172.21.15.39:0/3844501387 learned_addr learned my addr 172.21.15.39:0/3844501387 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:47.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.439+0000 7f211fe00640 1 --2- 172.21.15.39:0/3844501387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2128155880 0x7f21281e8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:47.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.439+0000 7f2127400640 1 --2- 172.21.15.39:0/3844501387 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f212815f830 0x7f21280b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:47.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.439+0000 7f2126a00640 1 -- 172.21.15.39:0/3844501387 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f212815f830 msgr2=0x7f21280b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:47.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.439+0000 7f2126a00640 1 --2- 172.21.15.39:0/3844501387 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f212815f830 0x7f21280b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:47.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.439+0000 7f2126a00640 1 -- 172.21.15.39:0/3844501387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2128155880 msgr2=0x7f21281e8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:47.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.439+0000 7f2126a00640 1 --2- 172.21.15.39:0/3844501387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2128155880 0x7f21281e8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:47.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.439+0000 7f2126a00640 1 -- 172.21.15.39:0/3844501387 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21280ce070 con 0x7f21280b3580 2024-09-21T07:44:47.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.439+0000 7f211fe00640 1 --2- 172.21.15.39:0/3844501387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2128155880 0x7f21281e8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:44:47.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.439+0000 7f2126a00640 1 --2- 172.21.15.39:0/3844501387 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21280b3580 0x7f2128143cb0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f211800b420 tx=0x7f211800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:47.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.440+0000 7f211f400640 1 -- 172.21.15.39:0/3844501387 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2118011830 con 0x7f21280b3580 2024-09-21T07:44:47.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.440+0000 7f211f400640 1 -- 172.21.15.39:0/3844501387 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2118016070 con 0x7f21280b3580 2024-09-21T07:44:47.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.440+0000 7f211f400640 1 -- 172.21.15.39:0/3844501387 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2118011d60 con 0x7f21280b3580 2024-09-21T07:44:47.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.440+0000 7f2127e00640 1 -- 172.21.15.39:0/3844501387 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21280ce350 con 0x7f21280b3580 2024-09-21T07:44:47.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.440+0000 7f2127e00640 1 -- 172.21.15.39:0/3844501387 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21280ce910 con 0x7f21280b3580 2024-09-21T07:44:47.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.441+0000 7f2127e00640 1 -- 172.21.15.39:0/3844501387 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21280aed10 con 0x7f21280b3580 2024-09-21T07:44:47.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.442+0000 7f211f400640 1 -- 172.21.15.39:0/3844501387 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f211801b460 con 0x7f21280b3580 2024-09-21T07:44:47.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.443+0000 7f211f400640 1 --2- 172.21.15.39:0/3844501387 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2104078b90 0x7f210407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:47.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.443+0000 7f211f400640 1 -- 172.21.15.39:0/3844501387 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 7711+0+0 (secure 0 0 0) 0x7f211809b330 con 0x7f21280b3580 2024-09-21T07:44:47.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.444+0000 7f211fe00640 1 --2- 172.21.15.39:0/3844501387 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2104078b90 0x7f210407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:47.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.444+0000 7f211fe00640 1 --2- 172.21.15.39:0/3844501387 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2104078b90 0x7f210407b0a0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f21281f8f20 tx=0x7f2110005f10 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:47.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.448+0000 7f211f400640 1 -- 172.21.15.39:0/3844501387 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2118062f00 con 0x7f21280b3580 2024-09-21T07:44:47.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.642+0000 7f2127e00640 1 -- 172.21.15.39:0/3844501387 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) -- 0x7f2128075060 con 0x7f21280b3580 2024-09-21T07:44:47.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.642+0000 7f211f400640 1 -- 172.21.15.39:0/3844501387 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v126) ==== 107+0+11 (secure 0 0 0) 0x7f2118062800 con 0x7f21280b3580 2024-09-21T07:44:47.645 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-21T07:44:47.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.645+0000 7f2127e00640 1 -- 172.21.15.39:0/3844501387 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2104078b90 msgr2=0x7f210407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:47.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.645+0000 7f2127e00640 1 --2- 172.21.15.39:0/3844501387 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2104078b90 0x7f210407b0a0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f21281f8f20 tx=0x7f2110005f10 comp rx=0 tx=0).stop 2024-09-21T07:44:47.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.645+0000 7f2127e00640 1 -- 172.21.15.39:0/3844501387 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21280b3580 msgr2=0x7f2128143cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:47.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.645+0000 7f2127e00640 1 --2- 172.21.15.39:0/3844501387 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21280b3580 0x7f2128143cb0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f211800b420 tx=0x7f211800b8e0 comp rx=0 tx=0).stop 2024-09-21T07:44:47.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.645+0000 7f2127e00640 1 -- 172.21.15.39:0/3844501387 shutdown_connections 2024-09-21T07:44:47.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.645+0000 7f2127e00640 1 --2- 172.21.15.39:0/3844501387 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2104078b90 0x7f210407b0a0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:47.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.645+0000 7f2127e00640 1 --2- 172.21.15.39:0/3844501387 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f212815f830 0x7f21280b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:47.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.645+0000 7f2127e00640 1 --2- 172.21.15.39:0/3844501387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2128155880 0x7f21281e8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:47.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.645+0000 7f2127e00640 1 --2- 172.21.15.39:0/3844501387 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21280b3580 0x7f2128143cb0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:47.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.645+0000 7f2127e00640 1 -- 172.21.15.39:0/3844501387 >> 172.21.15.39:0/3844501387 conn(0x7f212806cfc0 msgr2=0x7f2128161a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:47.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.645+0000 7f2127e00640 1 -- 172.21.15.39:0/3844501387 shutdown_connections 2024-09-21T07:44:47.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.645+0000 7f2127e00640 1 -- 172.21.15.39:0/3844501387 wait complete. 2024-09-21T07:44:47.669 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_mds_max_snaps_per_dir_default_limit' 2024-09-21T07:44:47.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.895+0000 7f10dc800640 1 -- 172.21.15.39:0/1189854919 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10d40b30a0 msgr2=0x7f10d40cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:47.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.895+0000 7f10dc800640 1 --2- 172.21.15.39:0/1189854919 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10d40b30a0 0x7f10d40cb8a0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f10c8009830 tx=0x7f10c802a480 comp rx=0 tx=0).stop 2024-09-21T07:44:47.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.895+0000 7f10dc800640 1 -- 172.21.15.39:0/1189854919 shutdown_connections 2024-09-21T07:44:47.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.895+0000 7f10dc800640 1 --2- 172.21.15.39:0/1189854919 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f10d415d830 0x7f10d4156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:47.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.895+0000 7f10dc800640 1 --2- 172.21.15.39:0/1189854919 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10d40b30a0 0x7f10d40cb8a0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:47.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.895+0000 7f10dc800640 1 -- 172.21.15.39:0/1189854919 >> 172.21.15.39:0/1189854919 conn(0x7f10d40b2460 msgr2=0x7f10d406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:47.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.896+0000 7f10dc800640 1 -- 172.21.15.39:0/1189854919 shutdown_connections 2024-09-21T07:44:47.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.896+0000 7f10dc800640 1 -- 172.21.15.39:0/1189854919 wait complete. 2024-09-21T07:44:47.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.898+0000 7f10dc800640 1 Processor -- start 2024-09-21T07:44:47.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.898+0000 7f10dc800640 1 -- start start 2024-09-21T07:44:47.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.899+0000 7f10dc800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10d40b30a0 0x7f10d4155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:47.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.899+0000 7f10dc800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f10d40ccc60 0x7f10d40c4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:47.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.900+0000 7f10dc800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10d415d830 0x7f10d40c4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:47.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.900+0000 7f10dc800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f10d40af450 con 0x7f10d415d830 2024-09-21T07:44:47.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.900+0000 7f10dc800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f10d40af2d0 con 0x7f10d40b30a0 2024-09-21T07:44:47.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.900+0000 7f10dc800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f10d40af5d0 con 0x7f10d40ccc60 2024-09-21T07:44:47.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.900+0000 7f10da800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10d415d830 0x7f10d40c4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:47.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.900+0000 7f10da800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10d415d830 0x7f10d40c4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46556/0 (socket says 172.21.15.39:46556) 2024-09-21T07:44:47.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.900+0000 7f10da800640 1 -- 172.21.15.39:0/1986700519 learned_addr learned my addr 172.21.15.39:0/1986700519 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:47.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.900+0000 7f10d9e00640 1 --2- 172.21.15.39:0/1986700519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10d40b30a0 0x7f10d4155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:47.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.900+0000 7f10d9400640 1 --2- 172.21.15.39:0/1986700519 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f10d40ccc60 0x7f10d40c4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:47.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.900+0000 7f10da800640 1 -- 172.21.15.39:0/1986700519 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f10d40ccc60 msgr2=0x7f10d40c4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:47.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.900+0000 7f10da800640 1 --2- 172.21.15.39:0/1986700519 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f10d40ccc60 0x7f10d40c4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:47.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.900+0000 7f10da800640 1 -- 172.21.15.39:0/1986700519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10d40b30a0 msgr2=0x7f10d4155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:47.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.900+0000 7f10da800640 1 --2- 172.21.15.39:0/1986700519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10d40b30a0 0x7f10d4155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:47.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.900+0000 7f10da800640 1 -- 172.21.15.39:0/1986700519 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d40c50f0 con 0x7f10d415d830 2024-09-21T07:44:47.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.900+0000 7f10d9400640 1 --2- 172.21.15.39:0/1986700519 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f10d40ccc60 0x7f10d40c4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:44:47.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.900+0000 7f10d9e00640 1 --2- 172.21.15.39:0/1986700519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10d40b30a0 0x7f10d4155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:44:47.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.901+0000 7f10da800640 1 --2- 172.21.15.39:0/1986700519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10d415d830 0x7f10d40c4760 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f10d000cae0 tx=0x7f10d000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:47.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.901+0000 7f10ce000640 1 -- 172.21.15.39:0/1986700519 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d0016020 con 0x7f10d415d830 2024-09-21T07:44:47.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.901+0000 7f10ce000640 1 -- 172.21.15.39:0/1986700519 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10d000ad30 con 0x7f10d415d830 2024-09-21T07:44:47.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.901+0000 7f10ce000640 1 -- 172.21.15.39:0/1986700519 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d0012680 con 0x7f10d415d830 2024-09-21T07:44:47.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.901+0000 7f10dc800640 1 -- 172.21.15.39:0/1986700519 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10d40c5ff0 con 0x7f10d415d830 2024-09-21T07:44:47.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.901+0000 7f10dc800640 1 -- 172.21.15.39:0/1986700519 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10d40c65b0 con 0x7f10d415d830 2024-09-21T07:44:47.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.902+0000 7f10dc800640 1 -- 172.21.15.39:0/1986700519 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10d40afa30 con 0x7f10d415d830 2024-09-21T07:44:47.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.903+0000 7f10ce000640 1 -- 172.21.15.39:0/1986700519 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f10d0019020 con 0x7f10d415d830 2024-09-21T07:44:47.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.904+0000 7f10ce000640 1 --2- 172.21.15.39:0/1986700519 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f10b8078bc0 0x7f10b807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:47.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.904+0000 7f10ce000640 1 -- 172.21.15.39:0/1986700519 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 7711+0+0 (secure 0 0 0) 0x7f10d009b6c0 con 0x7f10d415d830 2024-09-21T07:44:47.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.905+0000 7f10d9e00640 1 --2- 172.21.15.39:0/1986700519 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f10b8078bc0 0x7f10b807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:47.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.906+0000 7f10d9e00640 1 --2- 172.21.15.39:0/1986700519 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f10b8078bc0 0x7f10b807b0d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f10c8034000 tx=0x7f10c8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:47.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:47.909+0000 7f10ce000640 1 -- 172.21.15.39:0/1986700519 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f10d0020030 con 0x7f10d415d830 2024-09-21T07:44:48.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.179+0000 7f10dc800640 1 -- 172.21.15.39:0/1986700519 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_mds_max_snaps_per_dir_default_limit"]} v 0) -- 0x7f10d415f850 con 0x7f10d415d830 2024-09-21T07:44:48.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.487+0000 7f10ce000640 1 -- 172.21.15.39:0/1986700519 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_mds_max_snaps_per_dir_default_limit"]}]=0 v560) ==== 164+0+0 (secure 0 0 0) 0x7f10d0063450 con 0x7f10d415d830 2024-09-21T07:44:48.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.490+0000 7f10dc800640 1 -- 172.21.15.39:0/1986700519 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f10b8078bc0 msgr2=0x7f10b807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:48.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.490+0000 7f10dc800640 1 --2- 172.21.15.39:0/1986700519 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f10b8078bc0 0x7f10b807b0d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f10c8034000 tx=0x7f10c8004730 comp rx=0 tx=0).stop 2024-09-21T07:44:48.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.490+0000 7f10dc800640 1 -- 172.21.15.39:0/1986700519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10d415d830 msgr2=0x7f10d40c4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:48.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.490+0000 7f10dc800640 1 --2- 172.21.15.39:0/1986700519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10d415d830 0x7f10d40c4760 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f10d000cae0 tx=0x7f10d000a060 comp rx=0 tx=0).stop 2024-09-21T07:44:48.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.490+0000 7f10dc800640 1 -- 172.21.15.39:0/1986700519 shutdown_connections 2024-09-21T07:44:48.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.490+0000 7f10dc800640 1 --2- 172.21.15.39:0/1986700519 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f10b8078bc0 0x7f10b807b0d0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:48.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.490+0000 7f10dc800640 1 --2- 172.21.15.39:0/1986700519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10d415d830 0x7f10d40c4760 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:48.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.490+0000 7f10dc800640 1 --2- 172.21.15.39:0/1986700519 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f10d40ccc60 0x7f10d40c4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:48.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.490+0000 7f10dc800640 1 --2- 172.21.15.39:0/1986700519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10d40b30a0 0x7f10d4155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:48.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.490+0000 7f10dc800640 1 -- 172.21.15.39:0/1986700519 >> 172.21.15.39:0/1986700519 conn(0x7f10d40b2460 msgr2=0x7f10d40ad5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:48.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.490+0000 7f10dc800640 1 -- 172.21.15.39:0/1986700519 shutdown_connections 2024-09-21T07:44:48.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.490+0000 7f10dc800640 1 -- 172.21.15.39:0/1986700519 wait complete. 2024-09-21T07:44:48.518 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T07:44:48.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.743+0000 7f5cf5200640 1 -- 172.21.15.39:0/2648107020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cf00ce690 msgr2=0x7f5cf00b2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:48.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.743+0000 7f5cf5200640 1 --2- 172.21.15.39:0/2648107020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cf00ce690 0x7f5cf00b2f70 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0009820 tx=0x7f5ce002a510 comp rx=0 tx=0).stop 2024-09-21T07:44:48.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.743+0000 7f5cf5200640 1 -- 172.21.15.39:0/2648107020 shutdown_connections 2024-09-21T07:44:48.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.743+0000 7f5cf5200640 1 --2- 172.21.15.39:0/2648107020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cf00ce690 0x7f5cf00b2f70 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:48.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.743+0000 7f5cf5200640 1 -- 172.21.15.39:0/2648107020 >> 172.21.15.39:0/2648107020 conn(0x7f5cf009b5f0 msgr2=0x7f5cf008c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:48.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.743+0000 7f5cf5200640 1 -- 172.21.15.39:0/2648107020 shutdown_connections 2024-09-21T07:44:48.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.743+0000 7f5cf5200640 1 -- 172.21.15.39:0/2648107020 wait complete. 2024-09-21T07:44:48.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.745+0000 7f5cf5200640 1 Processor -- start 2024-09-21T07:44:48.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.745+0000 7f5cf5200640 1 -- start start 2024-09-21T07:44:48.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.747+0000 7f5cf5200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5cf00ce690 0x7f5cf00c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:48.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.747+0000 7f5cf5200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cf00d0060 0x7f5cf00ca590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:48.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.747+0000 7f5cf5200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cf00d2740 0x7f5cf015e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:48.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.747+0000 7f5cf5200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf00cd5a0 con 0x7f5cf00d2740 2024-09-21T07:44:48.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.747+0000 7f5cf5200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf00d0790 con 0x7f5cf00d0060 2024-09-21T07:44:48.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.747+0000 7f5cf5200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f5cf00cd720 con 0x7f5cf00ce690 2024-09-21T07:44:48.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.747+0000 7f5cefe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cf00d2740 0x7f5cf015e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:48.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.747+0000 7f5cefe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cf00d2740 0x7f5cf015e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46578/0 (socket says 172.21.15.39:46578) 2024-09-21T07:44:48.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.747+0000 7f5cefe00640 1 -- 172.21.15.39:0/67877693 learned_addr learned my addr 172.21.15.39:0/67877693 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:48.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.747+0000 7f5cef400640 1 --2- 172.21.15.39:0/67877693 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5cf00ce690 0x7f5cf00c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:48.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.747+0000 7f5ce6a00640 1 --2- 172.21.15.39:0/67877693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cf00d0060 0x7f5cf00ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:48.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.748+0000 7f5cefe00640 1 -- 172.21.15.39:0/67877693 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5cf00ce690 msgr2=0x7f5cf00c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:48.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.748+0000 7f5cefe00640 1 --2- 172.21.15.39:0/67877693 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5cf00ce690 0x7f5cf00c86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:48.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.748+0000 7f5cefe00640 1 -- 172.21.15.39:0/67877693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cf00d0060 msgr2=0x7f5cf00ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:48.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.748+0000 7f5cefe00640 1 --2- 172.21.15.39:0/67877693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cf00d0060 0x7f5cf00ca590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:48.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.748+0000 7f5cefe00640 1 -- 172.21.15.39:0/67877693 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf00c8c00 con 0x7f5cf00d2740 2024-09-21T07:44:48.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.748+0000 7f5cefe00640 1 --2- 172.21.15.39:0/67877693 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cf00d2740 0x7f5cf015e1d0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f5ce800ba10 tx=0x7f5ce800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:48.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.748+0000 7f5ce7e00640 1 -- 172.21.15.39:0/67877693 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8019070 con 0x7f5cf00d2740 2024-09-21T07:44:48.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.748+0000 7f5ce7e00640 1 -- 172.21.15.39:0/67877693 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ce800cc30 con 0x7f5cf00d2740 2024-09-21T07:44:48.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.748+0000 7f5cf5200640 1 -- 172.21.15.39:0/67877693 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cf015f370 con 0x7f5cf00d2740 2024-09-21T07:44:48.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.748+0000 7f5ce7e00640 1 -- 172.21.15.39:0/67877693 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce800ce60 con 0x7f5cf00d2740 2024-09-21T07:44:48.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.748+0000 7f5cf5200640 1 -- 172.21.15.39:0/67877693 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cf00c9830 con 0x7f5cf00d2740 2024-09-21T07:44:48.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.749+0000 7f5cf5200640 1 -- 172.21.15.39:0/67877693 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cf00b2f70 con 0x7f5cf00d2740 2024-09-21T07:44:48.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.751+0000 7f5ce7e00640 1 -- 172.21.15.39:0/67877693 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5ce801c070 con 0x7f5cf00d2740 2024-09-21T07:44:48.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.752+0000 7f5ce7e00640 1 --2- 172.21.15.39:0/67877693 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5ccc078b90 0x7f5ccc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:48.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.752+0000 7f5ce7e00640 1 -- 172.21.15.39:0/67877693 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 7711+0+0 (secure 0 0 0) 0x7f5ce809afa0 con 0x7f5cf00d2740 2024-09-21T07:44:48.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.753+0000 7f5cef400640 1 --2- 172.21.15.39:0/67877693 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5ccc078b90 0x7f5ccc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:48.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.753+0000 7f5cef400640 1 --2- 172.21.15.39:0/67877693 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5ccc078b90 0x7f5ccc07b0a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0034000 tx=0x7f5ce0004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:48.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.757+0000 7f5ce7e00640 1 -- 172.21.15.39:0/67877693 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5ce8062640 con 0x7f5cf00d2740 2024-09-21T07:44:48.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.952+0000 7f5cf5200640 1 -- 172.21.15.39:0/67877693 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f5cf00cc7b0 con 0x7f5cf00d2740 2024-09-21T07:44:48.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.952+0000 7f5ce7e00640 1 -- 172.21.15.39:0/67877693 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v42) ==== 59+0+2294 (secure 0 0 0) 0x7f5ce8067070 con 0x7f5cf00d2740 2024-09-21T07:44:48.954 INFO:teuthology.orchestra.run.smithi039.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T07:44:48.954 INFO:teuthology.orchestra.run.smithi039.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T07:44:48.954 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced client_mount_timeout 30 2024-09-21T07:44:48.954 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_client 20/20 2024-09-21T07:44:48.954 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_mgr 20/20 2024-09-21T07:44:48.955 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_ms 1/1 2024-09-21T07:44:48.955 INFO:teuthology.orchestra.run.smithi039.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T07:44:48.955 INFO:teuthology.orchestra.run.smithi039.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T07:44:48.955 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3421.881759 2024-09-21T07:44:48.955 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3520.252978 2024-09-21T07:44:48.955 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3511.183316 2024-09-21T07:44:48.955 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3829.631847 2024-09-21T07:44:48.955 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 5474.545872 2024-09-21T07:44:48.955 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4013.628559 2024-09-21T07:44:48.955 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4230.187288 2024-09-21T07:44:48.955 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4148.983107 2024-09-21T07:44:48.955 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_mds 20/20 2024-09-21T07:44:48.955 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T07:44:48.955 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_ms 2/2 2024-09-21T07:44:48.956 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_debug_frag true 2024-09-21T07:44:48.956 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_debug_scatterstat true 2024-09-21T07:44:48.956 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T07:44:48.956 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_verify_scatter true 2024-09-21T07:44:48.956 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T07:44:48.956 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T07:44:48.956 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T07:44:48.956 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced client_mount_timeout 600 2024-09-21T07:44:48.956 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced debug_client 20/20 2024-09-21T07:44:48.956 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced debug_ms 1/1 2024-09-21T07:44:48.956 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T07:44:48.956 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T07:44:48.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.955+0000 7f5cf5200640 1 -- 172.21.15.39:0/67877693 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5ccc078b90 msgr2=0x7f5ccc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:48.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.955+0000 7f5cf5200640 1 --2- 172.21.15.39:0/67877693 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5ccc078b90 0x7f5ccc07b0a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0034000 tx=0x7f5ce0004730 comp rx=0 tx=0).stop 2024-09-21T07:44:48.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.955+0000 7f5cf5200640 1 -- 172.21.15.39:0/67877693 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cf00d2740 msgr2=0x7f5cf015e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:48.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.955+0000 7f5cf5200640 1 --2- 172.21.15.39:0/67877693 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cf00d2740 0x7f5cf015e1d0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f5ce800ba10 tx=0x7f5ce800bed0 comp rx=0 tx=0).stop 2024-09-21T07:44:48.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.955+0000 7f5cf5200640 1 -- 172.21.15.39:0/67877693 shutdown_connections 2024-09-21T07:44:48.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.955+0000 7f5cf5200640 1 --2- 172.21.15.39:0/67877693 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5ccc078b90 0x7f5ccc07b0a0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:48.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.955+0000 7f5cf5200640 1 --2- 172.21.15.39:0/67877693 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cf00d2740 0x7f5cf015e1d0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:48.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.955+0000 7f5cf5200640 1 --2- 172.21.15.39:0/67877693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cf00d0060 0x7f5cf00ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:48.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.955+0000 7f5cf5200640 1 --2- 172.21.15.39:0/67877693 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5cf00ce690 0x7f5cf00c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:48.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.955+0000 7f5cf5200640 1 -- 172.21.15.39:0/67877693 >> 172.21.15.39:0/67877693 conn(0x7f5cf009b5f0 msgr2=0x7f5cf008c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:48.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.955+0000 7f5cf5200640 1 -- 172.21.15.39:0/67877693 shutdown_connections 2024-09-21T07:44:48.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:48.955+0000 7f5cf5200640 1 -- 172.21.15.39:0/67877693 wait complete. 2024-09-21T07:44:48.979 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T07:44:49.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.205+0000 7fbae6a00640 1 -- 172.21.15.39:0/1724781074 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbae00ccc60 msgr2=0x7fbae00cd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:49.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.205+0000 7fbae6a00640 1 --2- 172.21.15.39:0/1724781074 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbae00ccc60 0x7fbae00cd070 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fbae0157440 tx=0x7fbad802a890 comp rx=0 tx=0).stop 2024-09-21T07:44:49.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.206+0000 7fbae6a00640 1 -- 172.21.15.39:0/1724781074 shutdown_connections 2024-09-21T07:44:49.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.206+0000 7fbae6a00640 1 --2- 172.21.15.39:0/1724781074 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbae00ccc60 0x7fbae00cd070 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:49.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.206+0000 7fbae6a00640 1 -- 172.21.15.39:0/1724781074 >> 172.21.15.39:0/1724781074 conn(0x7fbae00b2460 msgr2=0x7fbae006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:49.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.206+0000 7fbae6a00640 1 -- 172.21.15.39:0/1724781074 shutdown_connections 2024-09-21T07:44:49.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.206+0000 7fbae6a00640 1 -- 172.21.15.39:0/1724781074 wait complete. 2024-09-21T07:44:49.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.208+0000 7fbae6a00640 1 Processor -- start 2024-09-21T07:44:49.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.208+0000 7fbae6a00640 1 -- start start 2024-09-21T07:44:49.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.209+0000 7fbae6a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbae00b30a0 0x7fbae01559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:49.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.210+0000 7fbae6a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbae00cb8a0 0x7fbae015e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:49.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.210+0000 7fbae6a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbae00ccc60 0x7fbae00b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:49.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.210+0000 7fbae6a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbae00af550 con 0x7fbae00cb8a0 2024-09-21T07:44:49.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.210+0000 7fbae6a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbae00af3d0 con 0x7fbae00b30a0 2024-09-21T07:44:49.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.210+0000 7fbae6a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fbae00af6d0 con 0x7fbae00ccc60 2024-09-21T07:44:49.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.210+0000 7fbad7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbae00cb8a0 0x7fbae015e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:49.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.210+0000 7fbad7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbae00cb8a0 0x7fbae015e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46604/0 (socket says 172.21.15.39:46604) 2024-09-21T07:44:49.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.210+0000 7fbad7e00640 1 -- 172.21.15.39:0/671724861 learned_addr learned my addr 172.21.15.39:0/671724861 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:49.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.210+0000 7fbadfe00640 1 --2- 172.21.15.39:0/671724861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbae00b30a0 0x7fbae01559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:49.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.210+0000 7fbae4a00640 1 --2- 172.21.15.39:0/671724861 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbae00ccc60 0x7fbae00b0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:49.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.210+0000 7fbad7e00640 1 -- 172.21.15.39:0/671724861 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbae00ccc60 msgr2=0x7fbae00b0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:49.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.210+0000 7fbad7e00640 1 --2- 172.21.15.39:0/671724861 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbae00ccc60 0x7fbae00b0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:49.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.210+0000 7fbad7e00640 1 -- 172.21.15.39:0/671724861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbae00b30a0 msgr2=0x7fbae01559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:49.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.210+0000 7fbad7e00640 1 --2- 172.21.15.39:0/671724861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbae00b30a0 0x7fbae01559f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:49.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.210+0000 7fbad7e00640 1 -- 172.21.15.39:0/671724861 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae015e680 con 0x7fbae00cb8a0 2024-09-21T07:44:49.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.211+0000 7fbad7e00640 1 --2- 172.21.15.39:0/671724861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbae00cb8a0 0x7fbae015e140 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fbac800c400 tx=0x7fbac800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:49.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.211+0000 7fbadcc00640 1 -- 172.21.15.39:0/671724861 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac8011790 con 0x7fbae00cb8a0 2024-09-21T07:44:49.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.211+0000 7fbadcc00640 1 -- 172.21.15.39:0/671724861 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbac8017070 con 0x7fbae00cb8a0 2024-09-21T07:44:49.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.211+0000 7fbae6a00640 1 -- 172.21.15.39:0/671724861 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbae015f1f0 con 0x7fbae00cb8a0 2024-09-21T07:44:49.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.211+0000 7fbadcc00640 1 -- 172.21.15.39:0/671724861 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac8011ce0 con 0x7fbae00cb8a0 2024-09-21T07:44:49.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.211+0000 7fbae6a00640 1 -- 172.21.15.39:0/671724861 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbae00c42b0 con 0x7fbae00cb8a0 2024-09-21T07:44:49.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.212+0000 7fbae6a00640 1 -- 172.21.15.39:0/671724861 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbae00cd070 con 0x7fbae00cb8a0 2024-09-21T07:44:49.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.213+0000 7fbadcc00640 1 -- 172.21.15.39:0/671724861 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbac801c460 con 0x7fbae00cb8a0 2024-09-21T07:44:49.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.214+0000 7fbadcc00640 1 --2- 172.21.15.39:0/671724861 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbabc078b90 0x7fbabc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:49.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.214+0000 7fbadcc00640 1 -- 172.21.15.39:0/671724861 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 7711+0+0 (secure 0 0 0) 0x7fbac809b1f0 con 0x7fbae00cb8a0 2024-09-21T07:44:49.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.215+0000 7fbadfe00640 1 --2- 172.21.15.39:0/671724861 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbabc078b90 0x7fbabc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:49.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.216+0000 7fbadfe00640 1 --2- 172.21.15.39:0/671724861 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbabc078b90 0x7fbabc07b0a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fbad00015e0 tx=0x7fbad0009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:49.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.219+0000 7fbadcc00640 1 -- 172.21.15.39:0/671724861 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbae00cd5b0 con 0x7fbae00cb8a0 2024-09-21T07:44:49.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.415+0000 7fbae6a00640 1 -- 172.21.15.39:0/671724861 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fbae015fd70 con 0x7fbae00cb8a0 2024-09-21T07:44:49.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.420+0000 7fbadcc00640 1 -- 172.21.15.39:0/671724861 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v43)=0 v43) ==== 122+0+0 (secure 0 0 0) 0x7fbac8062f80 con 0x7fbae00cb8a0 2024-09-21T07:44:49.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.423+0000 7fbae6a00640 1 -- 172.21.15.39:0/671724861 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbabc078b90 msgr2=0x7fbabc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:49.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.423+0000 7fbae6a00640 1 --2- 172.21.15.39:0/671724861 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbabc078b90 0x7fbabc07b0a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fbad00015e0 tx=0x7fbad0009040 comp rx=0 tx=0).stop 2024-09-21T07:44:49.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.423+0000 7fbae6a00640 1 -- 172.21.15.39:0/671724861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbae00cb8a0 msgr2=0x7fbae015e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:49.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.423+0000 7fbae6a00640 1 --2- 172.21.15.39:0/671724861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbae00cb8a0 0x7fbae015e140 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fbac800c400 tx=0x7fbac800c8c0 comp rx=0 tx=0).stop 2024-09-21T07:44:49.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.423+0000 7fbae6a00640 1 -- 172.21.15.39:0/671724861 shutdown_connections 2024-09-21T07:44:49.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.423+0000 7fbae6a00640 1 --2- 172.21.15.39:0/671724861 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbabc078b90 0x7fbabc07b0a0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:49.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.423+0000 7fbae6a00640 1 --2- 172.21.15.39:0/671724861 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbae00ccc60 0x7fbae00b0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:49.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.423+0000 7fbae6a00640 1 --2- 172.21.15.39:0/671724861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbae00cb8a0 0x7fbae015e140 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:49.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.423+0000 7fbae6a00640 1 --2- 172.21.15.39:0/671724861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbae00b30a0 0x7fbae01559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:49.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.423+0000 7fbae6a00640 1 -- 172.21.15.39:0/671724861 >> 172.21.15.39:0/671724861 conn(0x7fbae00b2460 msgr2=0x7fbae00ae630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:49.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.423+0000 7fbae6a00640 1 -- 172.21.15.39:0/671724861 shutdown_connections 2024-09-21T07:44:49.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:49.423+0000 7fbae6a00640 1 -- 172.21.15.39:0/671724861 wait complete. 2024-09-21T07:44:49.446 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:44:49.447 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:44:49.550 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:44:49.635 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:44:49.636 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:44:49.636 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:44:49.671 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:44:49.671 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:44:49.671 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:44:49.707 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:44:49.708 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:44:49.708 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:44:49.745 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:44:49.745 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:44:49.746 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:44:49.785 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:44:49.785 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T07:44:49.785 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:44:50.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.011+0000 7f4711400640 1 -- 172.21.15.39:0/2786804706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46f0004800 msgr2=0x7f46f0004cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:50.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.011+0000 7f4711400640 1 --2- 172.21.15.39:0/2786804706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46f0004800 0x7f46f0004cb0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f470c0cd040 tx=0x7f46f0005830 comp rx=0 tx=0).stop 2024-09-21T07:44:50.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.011+0000 7f4711400640 1 -- 172.21.15.39:0/2786804706 shutdown_connections 2024-09-21T07:44:50.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.011+0000 7f4711400640 1 --2- 172.21.15.39:0/2786804706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46f0004800 0x7f46f0004cb0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:50.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.011+0000 7f4711400640 1 -- 172.21.15.39:0/2786804706 >> 172.21.15.39:0/2786804706 conn(0x7f470c06cfc0 msgr2=0x7f470c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:50.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.011+0000 7f4711400640 1 -- 172.21.15.39:0/2786804706 shutdown_connections 2024-09-21T07:44:50.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.011+0000 7f4711400640 1 -- 172.21.15.39:0/2786804706 wait complete. 2024-09-21T07:44:50.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.014+0000 7f4711400640 1 Processor -- start 2024-09-21T07:44:50.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.014+0000 7f4711400640 1 -- start start 2024-09-21T07:44:50.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.015+0000 7f4711400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46f0004800 0x7f470c0aee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:50.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.015+0000 7f4711400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f470c14ef00 0x7f470c0b1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:50.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.015+0000 7f4711400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f470c1539d0 0x7f470c153de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:50.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.015+0000 7f4711400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f470c0cf5e0 con 0x7f46f0004800 2024-09-21T07:44:50.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.015+0000 7f4711400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f470c0cf460 con 0x7f470c1539d0 2024-09-21T07:44:50.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.015+0000 7f4711400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f470c15b2e0 con 0x7f470c14ef00 2024-09-21T07:44:50.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.015+0000 7f470a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46f0004800 0x7f470c0aee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:50.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.016+0000 7f470a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46f0004800 0x7f470c0aee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46626/0 (socket says 172.21.15.39:46626) 2024-09-21T07:44:50.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.016+0000 7f470a800640 1 -- 172.21.15.39:0/3135554420 learned_addr learned my addr 172.21.15.39:0/3135554420 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:50.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.016+0000 7f470a800640 1 -- 172.21.15.39:0/3135554420 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f470c14ef00 msgr2=0x7f470c0b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:50.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.016+0000 7f470a800640 1 --2- 172.21.15.39:0/3135554420 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f470c14ef00 0x7f470c0b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:50.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.016+0000 7f470a800640 1 -- 172.21.15.39:0/3135554420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f470c1539d0 msgr2=0x7f470c153de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:50.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.016+0000 7f470a800640 1 --2- 172.21.15.39:0/3135554420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f470c1539d0 0x7f470c153de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:50.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.016+0000 7f470a800640 1 -- 172.21.15.39:0/3135554420 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f470c0adda0 con 0x7f46f0004800 2024-09-21T07:44:50.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.016+0000 7f470a800640 1 --2- 172.21.15.39:0/3135554420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46f0004800 0x7f470c0aee40 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f46fc011a40 tx=0x7f46fc011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:50.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.016+0000 7f4702a00640 1 -- 172.21.15.39:0/3135554420 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc004440 con 0x7f46f0004800 2024-09-21T07:44:50.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.017+0000 7f4702a00640 1 -- 172.21.15.39:0/3135554420 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46fc00f9a0 con 0x7f46f0004800 2024-09-21T07:44:50.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.017+0000 7f4711400640 1 -- 172.21.15.39:0/3135554420 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f470c0ae080 con 0x7f46f0004800 2024-09-21T07:44:50.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.017+0000 7f4702a00640 1 -- 172.21.15.39:0/3135554420 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc00fdf0 con 0x7f46f0004800 2024-09-21T07:44:50.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.017+0000 7f4711400640 1 -- 172.21.15.39:0/3135554420 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f470c0ae640 con 0x7f46f0004800 2024-09-21T07:44:50.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.018+0000 7f4711400640 1 -- 172.21.15.39:0/3135554420 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f470c0d0b00 con 0x7f46f0004800 2024-09-21T07:44:50.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.019+0000 7f4702a00640 1 -- 172.21.15.39:0/3135554420 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f46fc0035f0 con 0x7f46f0004800 2024-09-21T07:44:50.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.020+0000 7f4702a00640 1 --2- 172.21.15.39:0/3135554420 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f46e8078b90 0x7f46e807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:50.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.020+0000 7f4702a00640 1 -- 172.21.15.39:0/3135554420 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 7711+0+0 (secure 0 0 0) 0x7f46fc09c530 con 0x7f46f0004800 2024-09-21T07:44:50.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.020+0000 7f4703e00640 1 --2- 172.21.15.39:0/3135554420 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f46e8078b90 0x7f46e807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:50.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.021+0000 7f4703e00640 1 --2- 172.21.15.39:0/3135554420 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f46e8078b90 0x7f46e807b0a0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f46f4033000 tx=0x7f470c0ce660 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:50.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.024+0000 7f4702a00640 1 -- 172.21.15.39:0/3135554420 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f46fc064190 con 0x7f46f0004800 2024-09-21T07:44:50.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.257+0000 7f4711400640 1 -- 172.21.15.39:0/3135554420 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f470c0af380 con 0x7f46f0004800 2024-09-21T07:44:50.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.257+0000 7f4702a00640 1 -- 172.21.15.39:0/3135554420 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 166 v166) ==== 95+0+3915 (secure 0 0 0) 0x7f46fc063a90 con 0x7f46f0004800 2024-09-21T07:44:50.260 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:50.260 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":166,"btime":"2024-09-21T07:44:41:268223+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/2262142502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2262142502},{"type":"v1","addr":"172.21.15.148:6835","nonce":2262142502}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":7052,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/4280842618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":4280842618},{"type":"v1","addr":"172.21.15.148:6837","nonce":4280842618}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":166},{"gid":7055,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3359884438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3359884438},{"type":"v1","addr":"172.21.15.39:6835","nonce":3359884438}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":166},{"gid":8268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/67641510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":67641510},{"type":"v1","addr":"172.21.15.39:6833","nonce":67641510}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":161}],"filesystems":[]} 2024-09-21T07:44:50.260 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 166 2024-09-21T07:44:50.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.260+0000 7f4711400640 1 -- 172.21.15.39:0/3135554420 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f46e8078b90 msgr2=0x7f46e807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:50.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.260+0000 7f4711400640 1 --2- 172.21.15.39:0/3135554420 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f46e8078b90 0x7f46e807b0a0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f46f4033000 tx=0x7f470c0ce660 comp rx=0 tx=0).stop 2024-09-21T07:44:50.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.260+0000 7f4711400640 1 -- 172.21.15.39:0/3135554420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46f0004800 msgr2=0x7f470c0aee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:50.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.260+0000 7f4711400640 1 --2- 172.21.15.39:0/3135554420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46f0004800 0x7f470c0aee40 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f46fc011a40 tx=0x7f46fc011f00 comp rx=0 tx=0).stop 2024-09-21T07:44:50.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.260+0000 7f4711400640 1 -- 172.21.15.39:0/3135554420 shutdown_connections 2024-09-21T07:44:50.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.260+0000 7f4711400640 1 --2- 172.21.15.39:0/3135554420 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f46e8078b90 0x7f46e807b0a0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:50.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.260+0000 7f4711400640 1 --2- 172.21.15.39:0/3135554420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f470c1539d0 0x7f470c153de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:50.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.260+0000 7f4711400640 1 --2- 172.21.15.39:0/3135554420 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f470c14ef00 0x7f470c0b1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:50.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.260+0000 7f4711400640 1 --2- 172.21.15.39:0/3135554420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46f0004800 0x7f470c0aee40 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:50.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.260+0000 7f4711400640 1 -- 172.21.15.39:0/3135554420 >> 172.21.15.39:0/3135554420 conn(0x7f470c06cfc0 msgr2=0x7f470c073130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:50.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.260+0000 7f4711400640 1 -- 172.21.15.39:0/3135554420 shutdown_connections 2024-09-21T07:44:50.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.260+0000 7f4711400640 1 -- 172.21.15.39:0/3135554420 wait complete. 2024-09-21T07:44:50.284 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T07:44:50.284 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T07:44:50.284 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T07:44:50.285 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:44:50.285 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T07:44:50.285 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T07:44:50.285 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T07:44:50.285 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:44:50.285 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T07:44:50.285 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T07:44:50.286 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T07:44:50.286 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:44:50.286 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T07:44:50.286 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T07:44:50.286 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T07:44:50.286 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:44:50.307 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T07:44:50.307 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T07:44:50.309 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T07:44:50.309 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T07:44:50.311 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T07:44:50.311 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T07:44:50.311 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T07:44:50.311 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T07:44:50.354 INFO:tasks.ceph.mds.c:Started 2024-09-21T07:44:50.354 DEBUG:teuthology.parallel:result is None 2024-09-21T07:44:50.355 INFO:tasks.ceph.mds.b:Started 2024-09-21T07:44:50.355 DEBUG:teuthology.parallel:result is None 2024-09-21T07:44:50.357 INFO:tasks.ceph.mds.a:Started 2024-09-21T07:44:50.357 DEBUG:teuthology.parallel:result is None 2024-09-21T07:44:50.358 INFO:tasks.ceph.mds.d:Started 2024-09-21T07:44:50.358 DEBUG:teuthology.parallel:result is None 2024-09-21T07:44:50.358 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T07:44:50.551 INFO:tasks.ceph.mds.b.smithi148.stdout:starting mds.b at 2024-09-21T07:44:50.551 INFO:tasks.ceph.mds.d.smithi148.stdout:starting mds.d at 2024-09-21T07:44:50.554 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-21T07:44:50.555 INFO:tasks.ceph.mds.c.smithi039.stdout:starting mds.c at 2024-09-21T07:44:50.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.592+0000 7f7e2ac00640 1 -- 172.21.15.39:0/1969699924 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e240b28d0 msgr2=0x7f7e240cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:50.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.592+0000 7f7e2ac00640 1 --2- 172.21.15.39:0/1969699924 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e240b28d0 0x7f7e240cf070 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f7e240b7260 tx=0x7f7e1002a4a0 comp rx=0 tx=0).stop 2024-09-21T07:44:50.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.592+0000 7f7e2ac00640 1 -- 172.21.15.39:0/1969699924 shutdown_connections 2024-09-21T07:44:50.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.592+0000 7f7e2ac00640 1 --2- 172.21.15.39:0/1969699924 >> v2:172.21.15.148:3300/0 conn(0x7f7e240d32a0 0x7f7e240cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:50.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.592+0000 7f7e2ac00640 1 --2- 172.21.15.39:0/1969699924 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e240b28d0 0x7f7e240cf070 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:50.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.592+0000 7f7e2ac00640 1 -- 172.21.15.39:0/1969699924 >> 172.21.15.39:0/1969699924 conn(0x7f7e2409da40 msgr2=0x7f7e2408c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:50.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.592+0000 7f7e2ac00640 1 -- 172.21.15.39:0/1969699924 shutdown_connections 2024-09-21T07:44:50.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.592+0000 7f7e2ac00640 1 -- 172.21.15.39:0/1969699924 wait complete. 2024-09-21T07:44:50.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.595+0000 7f7e2ac00640 1 Processor -- start 2024-09-21T07:44:50.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.595+0000 7f7e2ac00640 1 -- start start 2024-09-21T07:44:50.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.596+0000 7f7e2ac00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e240b28d0 0x7f7e24151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:50.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.596+0000 7f7e2ac00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7e240b41e0 0x7f7e240ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:50.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.596+0000 7f7e2ac00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e240d32a0 0x7f7e240cd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:50.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.596+0000 7f7e2ac00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7e240cfc70 con 0x7f7e240d32a0 2024-09-21T07:44:50.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.596+0000 7f7e2ac00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7e240cfaf0 con 0x7f7e240b28d0 2024-09-21T07:44:50.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.596+0000 7f7e2ac00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f7e240cfdf0 con 0x7f7e240b41e0 2024-09-21T07:44:50.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.596+0000 7f7e28c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e240d32a0 0x7f7e240cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:50.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.596+0000 7f7e28c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e240d32a0 0x7f7e240cd000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46678/0 (socket says 172.21.15.39:46678) 2024-09-21T07:44:50.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.596+0000 7f7e28c00640 1 -- 172.21.15.39:0/312457242 learned_addr learned my addr 172.21.15.39:0/312457242 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:50.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.597+0000 7f7e23e00640 1 --2- 172.21.15.39:0/312457242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e240b28d0 0x7f7e24151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:50.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.597+0000 7f7e23400640 1 --2- 172.21.15.39:0/312457242 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7e240b41e0 0x7f7e240ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:50.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.597+0000 7f7e28c00640 1 -- 172.21.15.39:0/312457242 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7e240b41e0 msgr2=0x7f7e240ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:50.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.597+0000 7f7e28c00640 1 --2- 172.21.15.39:0/312457242 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7e240b41e0 0x7f7e240ccac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:50.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.597+0000 7f7e28c00640 1 -- 172.21.15.39:0/312457242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e240b28d0 msgr2=0x7f7e24151b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:50.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.597+0000 7f7e28c00640 1 --2- 172.21.15.39:0/312457242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e240b28d0 0x7f7e24151b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:50.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.597+0000 7f7e28c00640 1 -- 172.21.15.39:0/312457242 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2415e1c0 con 0x7f7e240d32a0 2024-09-21T07:44:50.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.597+0000 7f7e23e00640 1 --2- 172.21.15.39:0/312457242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e240b28d0 0x7f7e24151b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:44:50.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.597+0000 7f7e23400640 1 --2- 172.21.15.39:0/312457242 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7e240b41e0 0x7f7e240ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:44:50.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.597+0000 7f7e28c00640 1 --2- 172.21.15.39:0/312457242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e240d32a0 0x7f7e240cd000 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00ea70 tx=0x7f7e1c00ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:50.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.597+0000 7f7e1be00640 1 -- 172.21.15.39:0/312457242 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c00ce40 con 0x7f7e240d32a0 2024-09-21T07:44:50.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.597+0000 7f7e1be00640 1 -- 172.21.15.39:0/312457242 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e1c004c10 con 0x7f7e240d32a0 2024-09-21T07:44:50.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.598+0000 7f7e1be00640 1 -- 172.21.15.39:0/312457242 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c005020 con 0x7f7e240d32a0 2024-09-21T07:44:50.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.598+0000 7f7e2ac00640 1 -- 172.21.15.39:0/312457242 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e2409e9d0 con 0x7f7e240d32a0 2024-09-21T07:44:50.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.598+0000 7f7e2ac00640 1 -- 172.21.15.39:0/312457242 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e240a03e0 con 0x7f7e240d32a0 2024-09-21T07:44:50.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.599+0000 7f7e2ac00640 1 -- 172.21.15.39:0/312457242 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e240d0340 con 0x7f7e240d32a0 2024-09-21T07:44:50.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.600+0000 7f7e1be00640 1 -- 172.21.15.39:0/312457242 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7e1c0051c0 con 0x7f7e240d32a0 2024-09-21T07:44:50.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.601+0000 7f7e1be00640 1 --2- 172.21.15.39:0/312457242 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7e04078b90 0x7f7e0407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:50.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.601+0000 7f7e1be00640 1 -- 172.21.15.39:0/312457242 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 7711+0+0 (secure 0 0 0) 0x7f7e1c09c380 con 0x7f7e240d32a0 2024-09-21T07:44:50.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.601+0000 7f7e23e00640 1 --2- 172.21.15.39:0/312457242 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7e04078b90 0x7f7e0407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:50.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.602+0000 7f7e23e00640 1 --2- 172.21.15.39:0/312457242 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7e04078b90 0x7f7e0407b0a0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f7e240b40f0 tx=0x7f7e14009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:50.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.606+0000 7f7e1be00640 1 -- 172.21.15.39:0/312457242 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7e1c063f50 con 0x7f7e240d32a0 2024-09-21T07:44:50.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:50.813+0000 7f7e2ac00640 1 -- 172.21.15.39:0/312457242 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f7e2415e390 con 0x7f7e240d32a0 2024-09-21T07:44:51.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.516+0000 7f7e1be00640 1 -- 172.21.15.39:0/312457242 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v128) ==== 97+0+0 (secure 0 0 0) 0x7f7e1c063850 con 0x7f7e240d32a0 2024-09-21T07:44:51.518 INFO:teuthology.orchestra.run.smithi039.stderr: removed all blocklist entries 2024-09-21T07:44:51.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.518+0000 7f7e2ac00640 1 -- 172.21.15.39:0/312457242 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7e04078b90 msgr2=0x7f7e0407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:51.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.518+0000 7f7e2ac00640 1 --2- 172.21.15.39:0/312457242 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7e04078b90 0x7f7e0407b0a0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f7e240b40f0 tx=0x7f7e14009040 comp rx=0 tx=0).stop 2024-09-21T07:44:51.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.518+0000 7f7e2ac00640 1 -- 172.21.15.39:0/312457242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e240d32a0 msgr2=0x7f7e240cd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:51.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.518+0000 7f7e2ac00640 1 --2- 172.21.15.39:0/312457242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e240d32a0 0x7f7e240cd000 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00ea70 tx=0x7f7e1c00ef30 comp rx=0 tx=0).stop 2024-09-21T07:44:51.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.518+0000 7f7e2ac00640 1 -- 172.21.15.39:0/312457242 shutdown_connections 2024-09-21T07:44:51.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.518+0000 7f7e2ac00640 1 --2- 172.21.15.39:0/312457242 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7e04078b90 0x7f7e0407b0a0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:51.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.518+0000 7f7e2ac00640 1 --2- 172.21.15.39:0/312457242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e240d32a0 0x7f7e240cd000 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:51.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.518+0000 7f7e2ac00640 1 --2- 172.21.15.39:0/312457242 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7e240b41e0 0x7f7e240ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:51.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.518+0000 7f7e2ac00640 1 --2- 172.21.15.39:0/312457242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e240b28d0 0x7f7e24151b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:51.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.518+0000 7f7e2ac00640 1 -- 172.21.15.39:0/312457242 >> 172.21.15.39:0/312457242 conn(0x7f7e2409da40 msgr2=0x7f7e240b78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:51.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.519+0000 7f7e2ac00640 1 -- 172.21.15.39:0/312457242 shutdown_connections 2024-09-21T07:44:51.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.519+0000 7f7e2ac00640 1 -- 172.21.15.39:0/312457242 wait complete. 2024-09-21T07:44:51.548 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T07:44:51.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.777+0000 7f0fd7e00640 1 -- 172.21.15.39:0/2117161258 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fbc004d40 msgr2=0x7f0fbc005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:51.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.777+0000 7f0fd7e00640 1 --2- 172.21.15.39:0/2117161258 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fbc004d40 0x7f0fbc005150 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0005fd0 tx=0x7f0fbc005830 comp rx=0 tx=0).stop 2024-09-21T07:44:51.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.777+0000 7f0fd7400640 1 -- 172.21.15.39:0/2117161258 reap_dead start 2024-09-21T07:44:51.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.777+0000 7f0fd7e00640 1 -- 172.21.15.39:0/2117161258 shutdown_connections 2024-09-21T07:44:51.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.777+0000 7f0fd7e00640 1 -- 172.21.15.39:0/2117161258 >> 172.21.15.39:0/2117161258 conn(0x7f0fd806cfc0 msgr2=0x7f0fd806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:51.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.777+0000 7f0fd7e00640 1 -- 172.21.15.39:0/2117161258 shutdown_connections 2024-09-21T07:44:51.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.777+0000 7f0fd7e00640 1 -- 172.21.15.39:0/2117161258 wait complete. 2024-09-21T07:44:51.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.779+0000 7f0fd7e00640 1 Processor -- start 2024-09-21T07:44:51.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.780+0000 7f0fd7e00640 1 -- start start 2024-09-21T07:44:51.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.781+0000 7f0fd7e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0fbc004d40 0x7f0fd814c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:51.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.781+0000 7f0fd7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fd80abaa0 0x7f0fd81f6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:51.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.781+0000 7f0fd7e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0fd81f70a0 0x7f0fd8145030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:51.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.781+0000 7f0fd7e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd815d190 con 0x7f0fd80abaa0 2024-09-21T07:44:51.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.781+0000 7f0fd7e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd815d010 con 0x7f0fbc004d40 2024-09-21T07:44:51.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.781+0000 7f0fd7e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f0fd815d310 con 0x7f0fd81f70a0 2024-09-21T07:44:51.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.781+0000 7f0fcfe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fd80abaa0 0x7f0fd81f6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:51.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.782+0000 7f0fcfe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fd80abaa0 0x7f0fd81f6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46704/0 (socket says 172.21.15.39:46704) 2024-09-21T07:44:51.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.782+0000 7f0fcfe00640 1 -- 172.21.15.39:0/3253863752 learned_addr learned my addr 172.21.15.39:0/3253863752 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:51.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.782+0000 7f0fd6a00640 1 --2- 172.21.15.39:0/3253863752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0fbc004d40 0x7f0fd814c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:51.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.782+0000 7f0fd7400640 1 --2- 172.21.15.39:0/3253863752 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0fd81f70a0 0x7f0fd8145030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:51.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.782+0000 7f0fcfe00640 1 -- 172.21.15.39:0/3253863752 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0fd81f70a0 msgr2=0x7f0fd8145030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:51.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.782+0000 7f0fcfe00640 1 --2- 172.21.15.39:0/3253863752 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0fd81f70a0 0x7f0fd8145030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:51.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.782+0000 7f0fcfe00640 1 -- 172.21.15.39:0/3253863752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0fbc004d40 msgr2=0x7f0fd814c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:51.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.782+0000 7f0fcfe00640 1 --2- 172.21.15.39:0/3253863752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0fbc004d40 0x7f0fd814c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:51.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.782+0000 7f0fcfe00640 1 -- 172.21.15.39:0/3253863752 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd814c890 con 0x7f0fd80abaa0 2024-09-21T07:44:51.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.782+0000 7f0fcfe00640 1 --2- 172.21.15.39:0/3253863752 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fd80abaa0 0x7f0fd81f6b60 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0011a50 tx=0x7f0fc0011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:51.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.783+0000 7f0fcea00640 1 -- 172.21.15.39:0/3253863752 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc001f070 con 0x7f0fd80abaa0 2024-09-21T07:44:51.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.783+0000 7f0fcea00640 1 -- 172.21.15.39:0/3253863752 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0fc000fcd0 con 0x7f0fd80abaa0 2024-09-21T07:44:51.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.783+0000 7f0fcea00640 1 -- 172.21.15.39:0/3253863752 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc000fea0 con 0x7f0fd80abaa0 2024-09-21T07:44:51.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.783+0000 7f0fd7e00640 1 -- 172.21.15.39:0/3253863752 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fd80aaa20 con 0x7f0fd80abaa0 2024-09-21T07:44:51.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.783+0000 7f0fd7e00640 1 -- 172.21.15.39:0/3253863752 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fd80aafe0 con 0x7f0fd80abaa0 2024-09-21T07:44:51.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.784+0000 7f0fd7e00640 1 -- 172.21.15.39:0/3253863752 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fd80d08c0 con 0x7f0fd80abaa0 2024-09-21T07:44:51.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.785+0000 7f0fcea00640 1 -- 172.21.15.39:0/3253863752 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0fc0022070 con 0x7f0fd80abaa0 2024-09-21T07:44:51.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.786+0000 7f0fcea00640 1 --2- 172.21.15.39:0/3253863752 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0fb4078b90 0x7f0fb407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:51.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.786+0000 7f0fcea00640 1 -- 172.21.15.39:0/3253863752 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5948+0+0 (secure 0 0 0) 0x7f0fc009b800 con 0x7f0fd80abaa0 2024-09-21T07:44:51.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.787+0000 7f0fd6a00640 1 --2- 172.21.15.39:0/3253863752 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0fb4078b90 0x7f0fb407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:51.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.788+0000 7f0fd6a00640 1 --2- 172.21.15.39:0/3253863752 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0fb4078b90 0x7f0fb407b0a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8001d40 tx=0x7f0fc8006040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:51.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:51.791+0000 7f0fcea00640 1 -- 172.21.15.39:0/3253863752 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0fc0063460 con 0x7f0fd80abaa0 2024-09-21T07:44:52.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.059+0000 7f0fd7e00640 1 -- 172.21.15.39:0/3253863752 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f0fd81e4290 con 0x7f0fd80abaa0 2024-09-21T07:44:52.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.060+0000 7f0fcea00640 1 -- 172.21.15.39:0/3253863752 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) ==== 80+0+5304 (secure 0 0 0) 0x7f0fc0067e90 con 0x7f0fd80abaa0 2024-09-21T07:44:52.062 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:52.062 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:44:52.062 INFO:teuthology.orchestra.run.smithi039.stdout: "auth_dump": [ 2024-09-21T07:44:52.062 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:44:52.062 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.a", 2024-09-21T07:44:52.062 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5m+P6EERAABQqmlkrocSi1+hwc73Adrg==", 2024-09-21T07:44:52.062 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:44:52.062 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:44:52.062 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:44:52.062 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:44:52.063 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:44:52.063 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:44:52.063 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:44:52.063 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:44:52.063 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.b", 2024-09-21T07:44:52.063 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5m5/GeLRAAfE8fr5dbitQa89rNK9pZ2Q==", 2024-09-21T07:44:52.063 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:44:52.063 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:44:52.063 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:44:52.063 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:44:52.063 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:44:52.063 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:44:52.063 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:44:52.063 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:44:52.063 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.c", 2024-09-21T07:44:52.064 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5msbx6HxAALsF8ORvIoJpIAFZ5uxmOEw==", 2024-09-21T07:44:52.064 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:44:52.064 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:44:52.064 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:44:52.064 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:44:52.064 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:44:52.064 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:44:52.064 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:44:52.064 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:44:52.064 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.d", 2024-09-21T07:44:52.064 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mIrcBABAApKAB/7FkZ5glNTxlbkIsBw==", 2024-09-21T07:44:52.064 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:44:52.064 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:44:52.064 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:44:52.064 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:44:52.064 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:44:52.065 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:44:52.065 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:44:52.065 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:44:52.065 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.0", 2024-09-21T07:44:52.065 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB7du5msw81DhAAmVimqN4x9K3YpT0nbovppQ==", 2024-09-21T07:44:52.065 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:44:52.065 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:44:52.065 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:44:52.065 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:44:52.065 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:44:52.065 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:44:52.065 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:44:52.065 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.1", 2024-09-21T07:44:52.065 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB+du5mdnCIOhAAtVB4ThehQwb0zRriyKcaxA==", 2024-09-21T07:44:52.065 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:44:52.065 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:44:52.065 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:44:52.066 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:44:52.066 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:44:52.066 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:44:52.066 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:44:52.066 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.2", 2024-09-21T07:44:52.066 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCCdu5mTfWxDhAAadJCbPN5xBx/f9L0v2yTdg==", 2024-09-21T07:44:52.066 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:44:52.066 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:44:52.066 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:44:52.066 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:44:52.066 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:44:52.066 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:44:52.066 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:44:52.066 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.3", 2024-09-21T07:44:52.066 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCFdu5mL1E2HxAAyOhY97uUXEw5L1CcDeONVA==", 2024-09-21T07:44:52.067 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:44:52.067 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:44:52.067 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:44:52.067 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:44:52.067 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:44:52.067 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:44:52.067 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:44:52.067 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.4", 2024-09-21T07:44:52.067 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCLdu5m6PY/IhAAgvcTgm+6uWA7lIG4tcTIEg==", 2024-09-21T07:44:52.067 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:44:52.067 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:44:52.067 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:44:52.067 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:44:52.067 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:44:52.067 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:44:52.068 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:44:52.068 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.5", 2024-09-21T07:44:52.068 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCOdu5mE/wlMxAA3JBg4xH+N/PPC1Uv2ZVrYQ==", 2024-09-21T07:44:52.068 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:44:52.068 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:44:52.068 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:44:52.068 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:44:52.068 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:44:52.068 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:44:52.068 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:44:52.068 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.6", 2024-09-21T07:44:52.068 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCSdu5mC3TgBxAAwsArjeIdFcApY3/1OJffEg==", 2024-09-21T07:44:52.068 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:44:52.068 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:44:52.068 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:44:52.069 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:44:52.069 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:44:52.069 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:44:52.069 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:44:52.069 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.7", 2024-09-21T07:44:52.069 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCVdu5mSWHWFBAA9qPHWRdsvrB74inrp1lj4Q==", 2024-09-21T07:44:52.069 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:44:52.069 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:44:52.069 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:44:52.069 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:44:52.069 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:44:52.069 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:44:52.069 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:44:52.069 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.0", 2024-09-21T07:44:52.069 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mGfb1DBAA/aSMKR5or3xy53rB/iUjpA==", 2024-09-21T07:44:52.070 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:44:52.070 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:44:52.070 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow r", 2024-09-21T07:44:52.070 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T07:44:52.070 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:44:52.070 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:44:52.070 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:44:52.070 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.1", 2024-09-21T07:44:52.070 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mEkciIhAAid+bxJEr4P7vMnZ42PAy5w==", 2024-09-21T07:44:52.070 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:44:52.070 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:44:52.070 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow r", 2024-09-21T07:44:52.070 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow rw", 2024-09-21T07:44:52.070 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rwx" 2024-09-21T07:44:52.071 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:44:52.071 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:44:52.071 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:44:52.071 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.2", 2024-09-21T07:44:52.071 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mSu2oFxAAgjy2m/wWKh6xaoh+YDoU1Q==", 2024-09-21T07:44:52.071 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:44:52.071 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:44:52.071 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow r", 2024-09-21T07:44:52.071 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow rw", 2024-09-21T07:44:52.071 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rwx" 2024-09-21T07:44:52.071 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:44:52.071 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:44:52.071 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:44:52.071 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.3", 2024-09-21T07:44:52.072 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mCHy0LBAAZneF2jUWgfuyGehZFbOxDg==", 2024-09-21T07:44:52.072 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:44:52.072 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:44:52.072 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow r", 2024-09-21T07:44:52.072 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow rw", 2024-09-21T07:44:52.072 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rwx" 2024-09-21T07:44:52.072 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:44:52.072 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:44:52.072 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:44:52.072 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.admin", 2024-09-21T07:44:52.072 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBydu5m9LInERAAAlDpqb94f2Qb9VYJJwTBcA==", 2024-09-21T07:44:52.072 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:44:52.072 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow *", 2024-09-21T07:44:52.072 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:44:52.072 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:44:52.073 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:44:52.073 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:44:52.073 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:44:52.073 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:44:52.073 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mgr.x", 2024-09-21T07:44:52.073 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5mA2GRBhAAkyCKdgH6laHy8xNt080dXg==", 2024-09-21T07:44:52.073 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:44:52.073 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow *", 2024-09-21T07:44:52.073 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile mgr", 2024-09-21T07:44:52.073 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:44:52.073 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:44:52.073 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:44:52.073 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:44:52.073 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mgr.y", 2024-09-21T07:44:52.073 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBydu5mr+8aNxAA7TFYNms2WsXUfa4ITTJOHw==", 2024-09-21T07:44:52.073 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:44:52.073 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow *", 2024-09-21T07:44:52.074 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile mgr", 2024-09-21T07:44:52.074 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:44:52.074 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:44:52.074 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:44:52.074 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-21T07:44:52.074 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:44:52.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.062+0000 7f0fd7e00640 1 -- 172.21.15.39:0/3253863752 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0fb4078b90 msgr2=0x7f0fb407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:52.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.062+0000 7f0fd7e00640 1 --2- 172.21.15.39:0/3253863752 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0fb4078b90 0x7f0fb407b0a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8001d40 tx=0x7f0fc8006040 comp rx=0 tx=0).stop 2024-09-21T07:44:52.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.062+0000 7f0fd7e00640 1 -- 172.21.15.39:0/3253863752 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fd80abaa0 msgr2=0x7f0fd81f6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:52.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.062+0000 7f0fd7e00640 1 --2- 172.21.15.39:0/3253863752 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fd80abaa0 0x7f0fd81f6b60 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0011a50 tx=0x7f0fc0011f10 comp rx=0 tx=0).stop 2024-09-21T07:44:52.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.062+0000 7f0fd7e00640 1 -- 172.21.15.39:0/3253863752 shutdown_connections 2024-09-21T07:44:52.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.062+0000 7f0fd7e00640 1 --2- 172.21.15.39:0/3253863752 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0fb4078b90 0x7f0fb407b0a0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:52.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.062+0000 7f0fd7e00640 1 --2- 172.21.15.39:0/3253863752 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0fd81f70a0 0x7f0fd8145030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:52.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.062+0000 7f0fd7e00640 1 --2- 172.21.15.39:0/3253863752 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fd80abaa0 0x7f0fd81f6b60 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:52.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.062+0000 7f0fd7e00640 1 --2- 172.21.15.39:0/3253863752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0fbc004d40 0x7f0fd814c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:52.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.062+0000 7f0fd7e00640 1 -- 172.21.15.39:0/3253863752 >> 172.21.15.39:0/3253863752 conn(0x7f0fd806cfc0 msgr2=0x7f0fd81e2a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:52.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.062+0000 7f0fd7e00640 1 -- 172.21.15.39:0/3253863752 shutdown_connections 2024-09-21T07:44:52.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.062+0000 7f0fd7e00640 1 -- 172.21.15.39:0/3253863752 wait complete. 2024-09-21T07:44:52.089 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T07:44:52.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.314+0000 7f6522800640 1 -- 172.21.15.39:0/767966296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f651c0d0e60 msgr2=0x7f651c0d1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:52.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.314+0000 7f6522800640 1 --2- 172.21.15.39:0/767966296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f651c0d0e60 0x7f651c0d1270 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f6514005fd0 tx=0x7f651402aa90 comp rx=0 tx=0).stop 2024-09-21T07:44:52.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.315+0000 7f6522800640 1 -- 172.21.15.39:0/767966296 shutdown_connections 2024-09-21T07:44:52.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.315+0000 7f6522800640 1 --2- 172.21.15.39:0/767966296 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f651c0b96a0 0x7f651c0b59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:52.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.315+0000 7f6522800640 1 --2- 172.21.15.39:0/767966296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f651c0d0e60 0x7f651c0d1270 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:52.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.315+0000 7f6522800640 1 -- 172.21.15.39:0/767966296 >> 172.21.15.39:0/767966296 conn(0x7f651c09d7d0 msgr2=0x7f651c0d4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:52.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.315+0000 7f6522800640 1 -- 172.21.15.39:0/767966296 shutdown_connections 2024-09-21T07:44:52.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.315+0000 7f6522800640 1 -- 172.21.15.39:0/767966296 wait complete. 2024-09-21T07:44:52.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.317+0000 7f6522800640 1 Processor -- start 2024-09-21T07:44:52.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.317+0000 7f6522800640 1 -- start start 2024-09-21T07:44:52.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.318+0000 7f6522800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f651c0b96a0 0x7f651c150620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:52.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.318+0000 7f6522800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f651c0d0e60 0x7f651c15cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:52.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.318+0000 7f6522800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f651c0d5060 0x7f651c15d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:52.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.318+0000 7f6522800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f651c0d2c10 con 0x7f651c0d5060 2024-09-21T07:44:52.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.318+0000 7f6522800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f651c0d2a90 con 0x7f651c0b96a0 2024-09-21T07:44:52.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.318+0000 7f6522800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f651c0d2d90 con 0x7f651c0d0e60 2024-09-21T07:44:52.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.319+0000 7f6521e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f651c0d5060 0x7f651c15d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:52.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.319+0000 7f6521e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f651c0d5060 0x7f651c15d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47856/0 (socket says 172.21.15.39:47856) 2024-09-21T07:44:52.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.319+0000 7f6521e00640 1 -- 172.21.15.39:0/4226146965 learned_addr learned my addr 172.21.15.39:0/4226146965 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:52.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.319+0000 7f6520a00640 1 --2- 172.21.15.39:0/4226146965 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f651c0d0e60 0x7f651c15cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:52.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.319+0000 7f6521400640 1 --2- 172.21.15.39:0/4226146965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f651c0b96a0 0x7f651c150620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:52.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.319+0000 7f6521e00640 1 -- 172.21.15.39:0/4226146965 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f651c0d0e60 msgr2=0x7f651c15cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:52.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.319+0000 7f6521e00640 1 --2- 172.21.15.39:0/4226146965 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f651c0d0e60 0x7f651c15cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:52.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.319+0000 7f6521e00640 1 -- 172.21.15.39:0/4226146965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f651c0b96a0 msgr2=0x7f651c150620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:52.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.319+0000 7f6521e00640 1 --2- 172.21.15.39:0/4226146965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f651c0b96a0 0x7f651c150620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:52.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.319+0000 7f6521e00640 1 -- 172.21.15.39:0/4226146965 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f651c150fd0 con 0x7f651c0d5060 2024-09-21T07:44:52.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.319+0000 7f6521e00640 1 --2- 172.21.15.39:0/4226146965 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f651c0d5060 0x7f651c15d0c0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f6514009d90 tx=0x7f6514003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:52.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.320+0000 7f6519600640 1 -- 172.21.15.39:0/4226146965 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6514044070 con 0x7f651c0d5060 2024-09-21T07:44:52.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.320+0000 7f6519600640 1 -- 172.21.15.39:0/4226146965 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6514004040 con 0x7f651c0d5060 2024-09-21T07:44:52.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.320+0000 7f6519600640 1 -- 172.21.15.39:0/4226146965 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6514004430 con 0x7f651c0d5060 2024-09-21T07:44:52.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.320+0000 7f6522800640 1 -- 172.21.15.39:0/4226146965 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f651c15d720 con 0x7f651c0d5060 2024-09-21T07:44:52.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.320+0000 7f6522800640 1 -- 172.21.15.39:0/4226146965 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f651c15dce0 con 0x7f651c0d5060 2024-09-21T07:44:52.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.321+0000 7f6522800640 1 -- 172.21.15.39:0/4226146965 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f651c0d3240 con 0x7f651c0d5060 2024-09-21T07:44:52.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.322+0000 7f6519600640 1 -- 172.21.15.39:0/4226146965 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6514047070 con 0x7f651c0d5060 2024-09-21T07:44:52.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.323+0000 7f6519600640 1 --2- 172.21.15.39:0/4226146965 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f64fc078b90 0x7f64fc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:52.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.323+0000 7f6519600640 1 -- 172.21.15.39:0/4226146965 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5948+0+0 (secure 0 0 0) 0x7f65140bbd40 con 0x7f651c0d5060 2024-09-21T07:44:52.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.324+0000 7f6521400640 1 --2- 172.21.15.39:0/4226146965 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f64fc078b90 0x7f64fc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:52.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.324+0000 7f6521400640 1 --2- 172.21.15.39:0/4226146965 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f64fc078b90 0x7f64fc07b0a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f651c0d1a70 tx=0x7f6510009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:52.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.327+0000 7f6519600640 1 -- 172.21.15.39:0/4226146965 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f651c150fd0 con 0x7f651c0d5060 2024-09-21T07:44:52.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.525+0000 7f6522800640 1 -- 172.21.15.39:0/4226146965 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f651c0d1d40 con 0x7f651c0d5060 2024-09-21T07:44:52.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.525+0000 7f6519600640 1 -- 172.21.15.39:0/4226146965 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v130) ==== 84+0+287 (secure 0 0 0) 0x7f651c0d17b0 con 0x7f651c0d5060 2024-09-21T07:44:52.527 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:52.527 INFO:teuthology.orchestra.run.smithi039.stdout:[ 2024-09-21T07:44:52.527 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:44:52.528 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 1, 2024-09-21T07:44:52.528 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": ".mgr" 2024-09-21T07:44:52.528 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:44:52.528 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:44:52.528 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 6, 2024-09-21T07:44:52.528 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": "snap-test-pool" 2024-09-21T07:44:52.528 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:44:52.528 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:44:52.528 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 11, 2024-09-21T07:44:52.528 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": "test_data_pool" 2024-09-21T07:44:52.528 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:44:52.528 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:44:52.528 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 12, 2024-09-21T07:44:52.528 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": "test_metadata_pool" 2024-09-21T07:44:52.528 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:44:52.529 INFO:teuthology.orchestra.run.smithi039.stdout:] 2024-09-21T07:44:52.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.527+0000 7f6522800640 1 -- 172.21.15.39:0/4226146965 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f64fc078b90 msgr2=0x7f64fc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:52.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.527+0000 7f6522800640 1 --2- 172.21.15.39:0/4226146965 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f64fc078b90 0x7f64fc07b0a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f651c0d1a70 tx=0x7f6510009040 comp rx=0 tx=0).stop 2024-09-21T07:44:52.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.528+0000 7f6522800640 1 -- 172.21.15.39:0/4226146965 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f651c0d5060 msgr2=0x7f651c15d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:52.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.528+0000 7f6522800640 1 --2- 172.21.15.39:0/4226146965 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f651c0d5060 0x7f651c15d0c0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f6514009d90 tx=0x7f6514003ce0 comp rx=0 tx=0).stop 2024-09-21T07:44:52.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.528+0000 7f6522800640 1 -- 172.21.15.39:0/4226146965 shutdown_connections 2024-09-21T07:44:52.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.528+0000 7f6522800640 1 --2- 172.21.15.39:0/4226146965 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f64fc078b90 0x7f64fc07b0a0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:52.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.528+0000 7f6522800640 1 --2- 172.21.15.39:0/4226146965 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f651c0d5060 0x7f651c15d0c0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:52.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.528+0000 7f6522800640 1 --2- 172.21.15.39:0/4226146965 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f651c0d0e60 0x7f651c15cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:52.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.528+0000 7f6522800640 1 --2- 172.21.15.39:0/4226146965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f651c0b96a0 0x7f651c150620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:52.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.528+0000 7f6522800640 1 -- 172.21.15.39:0/4226146965 >> 172.21.15.39:0/4226146965 conn(0x7f651c09d7d0 msgr2=0x7f651c160ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:52.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.528+0000 7f6522800640 1 -- 172.21.15.39:0/4226146965 shutdown_connections 2024-09-21T07:44:52.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.528+0000 7f6522800640 1 -- 172.21.15.39:0/4226146965 wait complete. 2024-09-21T07:44:52.552 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T07:44:52.553 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T07:44:52.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.781+0000 7f7631600640 1 -- 172.21.15.39:0/206066583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f762c06dfc0 msgr2=0x7f762c06e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:52.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.781+0000 7f7631600640 1 --2- 172.21.15.39:0/206066583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f762c06dfc0 0x7f762c06e3d0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f7614009830 tx=0x7f761402a480 comp rx=0 tx=0).stop 2024-09-21T07:44:52.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.781+0000 7f7631600640 1 -- 172.21.15.39:0/206066583 shutdown_connections 2024-09-21T07:44:52.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.781+0000 7f7631600640 1 --2- 172.21.15.39:0/206066583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f762c06dfc0 0x7f762c06e3d0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:52.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.781+0000 7f7631600640 1 --2- 172.21.15.39:0/206066583 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f762c0b3410 0x7f762c1620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:52.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.781+0000 7f7631600640 1 -- 172.21.15.39:0/206066583 >> 172.21.15.39:0/206066583 conn(0x7f762c08c000 msgr2=0x7f762c0d4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:52.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.781+0000 7f7631600640 1 -- 172.21.15.39:0/206066583 shutdown_connections 2024-09-21T07:44:52.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.781+0000 7f7631600640 1 -- 172.21.15.39:0/206066583 wait complete. 2024-09-21T07:44:52.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.783+0000 7f7631600640 1 Processor -- start 2024-09-21T07:44:52.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.783+0000 7f7631600640 1 -- start start 2024-09-21T07:44:52.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.785+0000 7f7631600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f762c06dfc0 0x7f762c0cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:52.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.785+0000 7f7631600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f762c06e910 0x7f762c155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:52.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.785+0000 7f7631600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f762c0b3410 0x7f762c0d0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:52.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.785+0000 7f7631600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f762c0b1940 con 0x7f762c0b3410 2024-09-21T07:44:52.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.785+0000 7f7631600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f762c0d34b0 con 0x7f762c06e910 2024-09-21T07:44:52.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.785+0000 7f7631600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f762c0b1ac0 con 0x7f762c06dfc0 2024-09-21T07:44:52.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.785+0000 7f7630c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f762c0b3410 0x7f762c0d0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:52.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.785+0000 7f7630c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f762c0b3410 0x7f762c0d0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47882/0 (socket says 172.21.15.39:47882) 2024-09-21T07:44:52.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.785+0000 7f7630c00640 1 -- 172.21.15.39:0/2235308559 learned_addr learned my addr 172.21.15.39:0/2235308559 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:52.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.785+0000 7f7623e00640 1 --2- 172.21.15.39:0/2235308559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f762c06e910 0x7f762c155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:52.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.785+0000 7f762be00640 1 --2- 172.21.15.39:0/2235308559 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f762c06dfc0 0x7f762c0cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:52.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.785+0000 7f7630c00640 1 -- 172.21.15.39:0/2235308559 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f762c06dfc0 msgr2=0x7f762c0cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:52.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.785+0000 7f7630c00640 1 --2- 172.21.15.39:0/2235308559 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f762c06dfc0 0x7f762c0cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:52.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.785+0000 7f7630c00640 1 -- 172.21.15.39:0/2235308559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f762c06e910 msgr2=0x7f762c155f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:52.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.785+0000 7f7630c00640 1 --2- 172.21.15.39:0/2235308559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f762c06e910 0x7f762c155f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:52.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.786+0000 7f7630c00640 1 -- 172.21.15.39:0/2235308559 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f762c0b4300 con 0x7f762c0b3410 2024-09-21T07:44:52.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.786+0000 7f762be00640 1 --2- 172.21.15.39:0/2235308559 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f762c06dfc0 0x7f762c0cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:44:52.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.786+0000 7f7623e00640 1 --2- 172.21.15.39:0/2235308559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f762c06e910 0x7f762c155f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:44:52.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.786+0000 7f7630c00640 1 --2- 172.21.15.39:0/2235308559 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f762c0b3410 0x7f762c0d0860 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f762400c730 tx=0x7f762400cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:52.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.786+0000 7f7628c00640 1 -- 172.21.15.39:0/2235308559 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7624003b70 con 0x7f762c0b3410 2024-09-21T07:44:52.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.786+0000 7f7628c00640 1 -- 172.21.15.39:0/2235308559 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7624003d10 con 0x7f762c0b3410 2024-09-21T07:44:52.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.786+0000 7f7628c00640 1 -- 172.21.15.39:0/2235308559 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76240052d0 con 0x7f762c0b3410 2024-09-21T07:44:52.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.786+0000 7f7631600640 1 -- 172.21.15.39:0/2235308559 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f762c0b45e0 con 0x7f762c0b3410 2024-09-21T07:44:52.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.786+0000 7f7631600640 1 -- 172.21.15.39:0/2235308559 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f762c0b4ba0 con 0x7f762c0b3410 2024-09-21T07:44:52.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.787+0000 7f7631600640 1 -- 172.21.15.39:0/2235308559 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f762c0d02d0 con 0x7f762c0b3410 2024-09-21T07:44:52.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.789+0000 7f7628c00640 1 -- 172.21.15.39:0/2235308559 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7624005470 con 0x7f762c0b3410 2024-09-21T07:44:52.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.790+0000 7f7628c00640 1 --2- 172.21.15.39:0/2235308559 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7608078bc0 0x7f760807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:52.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.790+0000 7f7628c00640 1 -- 172.21.15.39:0/2235308559 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5948+0+0 (secure 0 0 0) 0x7f762409ac30 con 0x7f762c0b3410 2024-09-21T07:44:52.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.790+0000 7f762be00640 1 --2- 172.21.15.39:0/2235308559 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7608078bc0 0x7f760807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:52.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.791+0000 7f762be00640 1 --2- 172.21.15.39:0/2235308559 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7608078bc0 0x7f760807b0d0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f761c00bbb0 tx=0x7f761c005ec0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:52.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.794+0000 7f7628c00640 1 -- 172.21.15.39:0/2235308559 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7624062ef0 con 0x7f762c0b3410 2024-09-21T07:44:52.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:52.992+0000 7f7631600640 1 -- 172.21.15.39:0/2235308559 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f762c1573a0 con 0x7f762c0b3410 2024-09-21T07:44:53.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.536+0000 7f7628c00640 1 -- 172.21.15.39:0/2235308559 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v131) ==== 138+0+0 (secure 0 0 0) 0x7f76240627f0 con 0x7f762c0b3410 2024-09-21T07:44:53.538 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' created 2024-09-21T07:44:53.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.539+0000 7f7631600640 1 -- 172.21.15.39:0/2235308559 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7608078bc0 msgr2=0x7f760807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:53.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.539+0000 7f7631600640 1 --2- 172.21.15.39:0/2235308559 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7608078bc0 0x7f760807b0d0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f761c00bbb0 tx=0x7f761c005ec0 comp rx=0 tx=0).stop 2024-09-21T07:44:53.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.539+0000 7f7631600640 1 -- 172.21.15.39:0/2235308559 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f762c0b3410 msgr2=0x7f762c0d0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:53.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.539+0000 7f7631600640 1 --2- 172.21.15.39:0/2235308559 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f762c0b3410 0x7f762c0d0860 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f762400c730 tx=0x7f762400cb30 comp rx=0 tx=0).stop 2024-09-21T07:44:53.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.540+0000 7f7631600640 1 -- 172.21.15.39:0/2235308559 shutdown_connections 2024-09-21T07:44:53.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.540+0000 7f7631600640 1 --2- 172.21.15.39:0/2235308559 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7608078bc0 0x7f760807b0d0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:53.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.540+0000 7f7631600640 1 --2- 172.21.15.39:0/2235308559 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f762c0b3410 0x7f762c0d0860 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:53.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.540+0000 7f7631600640 1 --2- 172.21.15.39:0/2235308559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f762c06e910 0x7f762c155f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:53.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.540+0000 7f7631600640 1 --2- 172.21.15.39:0/2235308559 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f762c06dfc0 0x7f762c0cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:53.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.540+0000 7f7631600640 1 -- 172.21.15.39:0/2235308559 >> 172.21.15.39:0/2235308559 conn(0x7f762c08c000 msgr2=0x7f762c15f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:53.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.540+0000 7f7631600640 1 -- 172.21.15.39:0/2235308559 shutdown_connections 2024-09-21T07:44:53.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.540+0000 7f7631600640 1 -- 172.21.15.39:0/2235308559 wait complete. 2024-09-21T07:44:53.571 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T07:44:53.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.806+0000 7f272b000640 1 -- 172.21.15.39:0/914071861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2708004800 msgr2=0x7f2708004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:53.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.806+0000 7f272b000640 1 --2- 172.21.15.39:0/914071861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2708004800 0x7f2708004c10 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f270c009b50 tx=0x7f2708005830 comp rx=0 tx=0).stop 2024-09-21T07:44:53.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.806+0000 7f272b000640 1 -- 172.21.15.39:0/914071861 shutdown_connections 2024-09-21T07:44:53.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.806+0000 7f272b000640 1 --2- 172.21.15.39:0/914071861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2708004800 0x7f2708004c10 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:53.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.806+0000 7f272b000640 1 -- 172.21.15.39:0/914071861 >> 172.21.15.39:0/914071861 conn(0x7f27240b2460 msgr2=0x7f272406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:53.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.806+0000 7f272b000640 1 -- 172.21.15.39:0/914071861 shutdown_connections 2024-09-21T07:44:53.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.806+0000 7f272b000640 1 -- 172.21.15.39:0/914071861 wait complete. 2024-09-21T07:44:53.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.808+0000 7f272b000640 1 Processor -- start 2024-09-21T07:44:53.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.808+0000 7f272b000640 1 -- start start 2024-09-21T07:44:53.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.810+0000 7f272b000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2708004800 0x7f27240b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:53.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.810+0000 7f272b000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27240c7f10 0x7f27241f5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:53.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.810+0000 7f272b000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27240c5260 0x7f27240c5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:53.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.810+0000 7f272b000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f27240cd1f0 con 0x7f27240c5260 2024-09-21T07:44:53.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.810+0000 7f272b000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f27240cd070 con 0x7f2708004800 2024-09-21T07:44:53.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.810+0000 7f272b000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f27240cd370 con 0x7f27240c7f10 2024-09-21T07:44:53.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.810+0000 7f2729000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27240c5260 0x7f27240c5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:53.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.810+0000 7f271be00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27240c7f10 0x7f27241f5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:53.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.810+0000 7f271be00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27240c7f10 0x7f27241f5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.39:52076/0 (socket says 172.21.15.39:52076) 2024-09-21T07:44:53.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.810+0000 7f271be00640 1 -- 172.21.15.39:0/3139286349 learned_addr learned my addr 172.21.15.39:0/3139286349 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:53.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.810+0000 7f2723e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2708004800 0x7f27240b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:53.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.811+0000 7f271be00640 1 -- 172.21.15.39:0/3139286349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2708004800 msgr2=0x7f27240b0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:53.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.811+0000 7f271be00640 1 --2- 172.21.15.39:0/3139286349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2708004800 0x7f27240b0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:53.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.811+0000 7f271be00640 1 -- 172.21.15.39:0/3139286349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27240c5260 msgr2=0x7f27240c5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:53.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.811+0000 7f271be00640 1 --2- 172.21.15.39:0/3139286349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27240c5260 0x7f27240c5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:53.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.811+0000 7f271be00640 1 -- 172.21.15.39:0/3139286349 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27240c5bb0 con 0x7f27240c7f10 2024-09-21T07:44:53.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.811+0000 7f2729000640 1 --2- 172.21.15.39:0/3139286349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27240c5260 0x7f27240c5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:44:53.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.811+0000 7f271be00640 1 --2- 172.21.15.39:0/3139286349 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27240c7f10 0x7f27241f5ce0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f270c036040 tx=0x7f270c0336f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:53.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.811+0000 7f2720c00640 1 -- 172.21.15.39:0/3139286349 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270c038040 con 0x7f27240c7f10 2024-09-21T07:44:53.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.811+0000 7f2720c00640 1 -- 172.21.15.39:0/3139286349 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f270c041070 con 0x7f27240c7f10 2024-09-21T07:44:53.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.811+0000 7f272b000640 1 -- 172.21.15.39:0/3139286349 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27240c88b0 con 0x7f27240c7f10 2024-09-21T07:44:53.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.811+0000 7f2720c00640 1 -- 172.21.15.39:0/3139286349 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270c03c430 con 0x7f27240c7f10 2024-09-21T07:44:53.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.811+0000 7f272b000640 1 -- 172.21.15.39:0/3139286349 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f27240c7430 con 0x7f27240c7f10 2024-09-21T07:44:53.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.813+0000 7f272b000640 1 -- 172.21.15.39:0/3139286349 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27240adbd0 con 0x7f27240c7f10 2024-09-21T07:44:53.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.814+0000 7f2720c00640 1 -- 172.21.15.39:0/3139286349 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f270c03c5d0 con 0x7f27240c7f10 2024-09-21T07:44:53.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.815+0000 7f2720c00640 1 --2- 172.21.15.39:0/3139286349 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2700078b90 0x7f270007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:53.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.815+0000 7f2720c00640 1 -- 172.21.15.39:0/3139286349 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(131..131 src has 1..131) ==== 6318+0+0 (secure 0 0 0) 0x7f270c0bbaf0 con 0x7f27240c7f10 2024-09-21T07:44:53.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.815+0000 7f2723e00640 1 --2- 172.21.15.39:0/3139286349 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2700078b90 0x7f270007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:53.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.816+0000 7f2723e00640 1 --2- 172.21.15.39:0/3139286349 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2700078b90 0x7f270007b0a0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f2724052860 tx=0x7f2714006720 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:53.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:53.821+0000 7f2720c00640 1 -- 172.21.15.39:0/3139286349 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f27240c5bb0 con 0x7f27240c7f10 2024-09-21T07:44:54.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.017+0000 7f272b000640 1 -- 172.21.15.39:0/3139286349 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2724156c90 con 0x7f27240c7f10 2024-09-21T07:44:54.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.552+0000 7f2720c00640 1 -- 172.21.15.39:0/3139286349 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v132) ==== 170+0+0 (secure 0 0 0) 0x7f270c08a030 con 0x7f27240c7f10 2024-09-21T07:44:54.555 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' created 2024-09-21T07:44:54.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.561+0000 7f272b000640 1 -- 172.21.15.39:0/3139286349 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2700078b90 msgr2=0x7f270007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:54.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.561+0000 7f272b000640 1 --2- 172.21.15.39:0/3139286349 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2700078b90 0x7f270007b0a0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f2724052860 tx=0x7f2714006720 comp rx=0 tx=0).stop 2024-09-21T07:44:54.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.561+0000 7f272b000640 1 -- 172.21.15.39:0/3139286349 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27240c7f10 msgr2=0x7f27241f5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:54.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.561+0000 7f272b000640 1 --2- 172.21.15.39:0/3139286349 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27240c7f10 0x7f27241f5ce0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f270c036040 tx=0x7f270c0336f0 comp rx=0 tx=0).stop 2024-09-21T07:44:54.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.561+0000 7f272b000640 1 -- 172.21.15.39:0/3139286349 shutdown_connections 2024-09-21T07:44:54.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.561+0000 7f272b000640 1 --2- 172.21.15.39:0/3139286349 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2700078b90 0x7f270007b0a0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:54.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.561+0000 7f272b000640 1 --2- 172.21.15.39:0/3139286349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27240c5260 0x7f27240c5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:54.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.561+0000 7f272b000640 1 --2- 172.21.15.39:0/3139286349 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27240c7f10 0x7f27241f5ce0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:54.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.561+0000 7f272b000640 1 --2- 172.21.15.39:0/3139286349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2708004800 0x7f27240b0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:54.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.561+0000 7f272b000640 1 -- 172.21.15.39:0/3139286349 >> 172.21.15.39:0/3139286349 conn(0x7f27240b2460 msgr2=0x7f2724073b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:54.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.562+0000 7f272b000640 1 -- 172.21.15.39:0/3139286349 shutdown_connections 2024-09-21T07:44:54.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.563+0000 7f272b000640 1 -- 172.21.15.39:0/3139286349 wait complete. 2024-09-21T07:44:54.599 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T07:44:54.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.826+0000 7f5f59a00640 1 -- 172.21.15.39:0/2145581167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f540b30a0 msgr2=0x7f5f540cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:54.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.826+0000 7f5f59a00640 1 --2- 172.21.15.39:0/2145581167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f540b30a0 0x7f5f540cb8a0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f5f54075160 tx=0x7f5f4402a4d0 comp rx=0 tx=0).stop 2024-09-21T07:44:54.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.827+0000 7f5f59a00640 1 -- 172.21.15.39:0/2145581167 shutdown_connections 2024-09-21T07:44:54.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.827+0000 7f5f59a00640 1 --2- 172.21.15.39:0/2145581167 >> v2:172.21.15.148:3300/0 conn(0x7f5f5415d830 0x7f5f54156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:54.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.827+0000 7f5f59a00640 1 --2- 172.21.15.39:0/2145581167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f540b30a0 0x7f5f540cb8a0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:54.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.827+0000 7f5f59a00640 1 -- 172.21.15.39:0/2145581167 >> 172.21.15.39:0/2145581167 conn(0x7f5f540b2460 msgr2=0x7f5f5406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:54.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.827+0000 7f5f59a00640 1 -- 172.21.15.39:0/2145581167 shutdown_connections 2024-09-21T07:44:54.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.827+0000 7f5f59a00640 1 -- 172.21.15.39:0/2145581167 wait complete. 2024-09-21T07:44:54.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.829+0000 7f5f59a00640 1 Processor -- start 2024-09-21T07:44:54.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.829+0000 7f5f59a00640 1 -- start start 2024-09-21T07:44:54.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.831+0000 7f5f59a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f540b30a0 0x7f5f540c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:54.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.831+0000 7f5f59a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f540ccc60 0x7f5f541f54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:54.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.831+0000 7f5f59a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f5415d830 0x7f5f540cf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:54.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.831+0000 7f5f59a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5f540af410 con 0x7f5f5415d830 2024-09-21T07:44:54.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.831+0000 7f5f59a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5f540af290 con 0x7f5f540b30a0 2024-09-21T07:44:54.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.831+0000 7f5f59a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f5f540af590 con 0x7f5f540ccc60 2024-09-21T07:44:54.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.831+0000 7f5f53400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f5415d830 0x7f5f540cf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:54.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.831+0000 7f5f53400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f5415d830 0x7f5f540cf4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47912/0 (socket says 172.21.15.39:47912) 2024-09-21T07:44:54.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.831+0000 7f5f53400640 1 -- 172.21.15.39:0/1020956146 learned_addr learned my addr 172.21.15.39:0/1020956146 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:54.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.831+0000 7f5f52a00640 1 --2- 172.21.15.39:0/1020956146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f540b30a0 0x7f5f540c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:54.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.831+0000 7f5f4be00640 1 --2- 172.21.15.39:0/1020956146 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f540ccc60 0x7f5f541f54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:54.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.831+0000 7f5f53400640 1 -- 172.21.15.39:0/1020956146 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f540ccc60 msgr2=0x7f5f541f54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:54.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.831+0000 7f5f53400640 1 --2- 172.21.15.39:0/1020956146 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f540ccc60 0x7f5f541f54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:54.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.831+0000 7f5f53400640 1 -- 172.21.15.39:0/1020956146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f540b30a0 msgr2=0x7f5f540c87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:54.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.831+0000 7f5f53400640 1 --2- 172.21.15.39:0/1020956146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f540b30a0 0x7f5f540c87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:54.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.831+0000 7f5f53400640 1 -- 172.21.15.39:0/1020956146 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f541f98e0 con 0x7f5f5415d830 2024-09-21T07:44:54.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.832+0000 7f5f53400640 1 --2- 172.21.15.39:0/1020956146 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f5415d830 0x7f5f540cf4e0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c00b9e0 tx=0x7f5f4c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:54.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.832+0000 7f5f4aa00640 1 -- 172.21.15.39:0/1020956146 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c003b70 con 0x7f5f5415d830 2024-09-21T07:44:54.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.832+0000 7f5f4aa00640 1 -- 172.21.15.39:0/1020956146 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f4c003e00 con 0x7f5f5415d830 2024-09-21T07:44:54.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.832+0000 7f5f59a00640 1 -- 172.21.15.39:0/1020956146 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f541f9bc0 con 0x7f5f5415d830 2024-09-21T07:44:54.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.832+0000 7f5f4aa00640 1 -- 172.21.15.39:0/1020956146 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c00cb20 con 0x7f5f5415d830 2024-09-21T07:44:54.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.832+0000 7f5f59a00640 1 -- 172.21.15.39:0/1020956146 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f541fa180 con 0x7f5f5415d830 2024-09-21T07:44:54.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.833+0000 7f5f59a00640 1 -- 172.21.15.39:0/1020956146 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f5415c910 con 0x7f5f5415d830 2024-09-21T07:44:54.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.835+0000 7f5f4aa00640 1 -- 172.21.15.39:0/1020956146 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5f4c00ccc0 con 0x7f5f5415d830 2024-09-21T07:44:54.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.835+0000 7f5f4aa00640 1 --2- 172.21.15.39:0/1020956146 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5f30078bc0 0x7f5f3007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:54.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.836+0000 7f5f4aa00640 1 -- 172.21.15.39:0/1020956146 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 6700+0+0 (secure 0 0 0) 0x7f5f4c09afd0 con 0x7f5f5415d830 2024-09-21T07:44:54.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.836+0000 7f5f52a00640 1 --2- 172.21.15.39:0/1020956146 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5f30078bc0 0x7f5f3007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:54.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.837+0000 7f5f52a00640 1 --2- 172.21.15.39:0/1020956146 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5f30078bc0 0x7f5f3007b0d0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5f44033000 tx=0x7f5f44004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:54.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:54.841+0000 7f5f4aa00640 1 -- 172.21.15.39:0/1020956146 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5f4c062fa0 con 0x7f5f5415d830 2024-09-21T07:44:55.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.079+0000 7f5f59a00640 1 -- 172.21.15.39:0/1020956146 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f5f5415f680 con 0x7f5f5415d830 2024-09-21T07:44:55.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.546+0000 7f5f4aa00640 1 -- 172.21.15.39:0/1020956146 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v169) ==== 174+0+0 (secure 0 0 0) 0x7f5f4c0628a0 con 0x7f5f5415d830 2024-09-21T07:44:55.548 INFO:teuthology.orchestra.run.smithi039.stderr:new fs with metadata pool 17 and data pool 18 2024-09-21T07:44:55.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.549+0000 7f5f59a00640 1 -- 172.21.15.39:0/1020956146 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5f30078bc0 msgr2=0x7f5f3007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:55.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.549+0000 7f5f59a00640 1 --2- 172.21.15.39:0/1020956146 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5f30078bc0 0x7f5f3007b0d0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5f44033000 tx=0x7f5f44004730 comp rx=0 tx=0).stop 2024-09-21T07:44:55.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.549+0000 7f5f59a00640 1 -- 172.21.15.39:0/1020956146 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f5415d830 msgr2=0x7f5f540cf4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:55.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.549+0000 7f5f59a00640 1 --2- 172.21.15.39:0/1020956146 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f5415d830 0x7f5f540cf4e0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c00b9e0 tx=0x7f5f4c00bea0 comp rx=0 tx=0).stop 2024-09-21T07:44:55.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.549+0000 7f5f59a00640 1 -- 172.21.15.39:0/1020956146 shutdown_connections 2024-09-21T07:44:55.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.549+0000 7f5f59a00640 1 --2- 172.21.15.39:0/1020956146 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5f30078bc0 0x7f5f3007b0d0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:55.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.549+0000 7f5f59a00640 1 --2- 172.21.15.39:0/1020956146 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f5415d830 0x7f5f540cf4e0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:55.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.549+0000 7f5f59a00640 1 --2- 172.21.15.39:0/1020956146 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5f540ccc60 0x7f5f541f54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:55.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.549+0000 7f5f59a00640 1 --2- 172.21.15.39:0/1020956146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f540b30a0 0x7f5f540c87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:55.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.549+0000 7f5f59a00640 1 -- 172.21.15.39:0/1020956146 >> 172.21.15.39:0/1020956146 conn(0x7f5f540b2460 msgr2=0x7f5f54068df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:55.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.549+0000 7f5f59a00640 1 -- 172.21.15.39:0/1020956146 shutdown_connections 2024-09-21T07:44:55.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.549+0000 7f5f59a00640 1 -- 172.21.15.39:0/1020956146 wait complete. 2024-09-21T07:44:55.601 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:44:55.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.827+0000 7fcf2b600640 1 -- 172.21.15.39:0/2880348880 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf08004d40 msgr2=0x7fcf08005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:55.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.827+0000 7fcf2b600640 1 --2- 172.21.15.39:0/2880348880 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf08004d40 0x7fcf08005150 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fcf240cbe40 tx=0x7fcf08005830 comp rx=0 tx=0).stop 2024-09-21T07:44:55.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.827+0000 7fcf29600640 1 -- 172.21.15.39:0/2880348880 reap_dead start 2024-09-21T07:44:55.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.827+0000 7fcf2b600640 1 -- 172.21.15.39:0/2880348880 shutdown_connections 2024-09-21T07:44:55.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.827+0000 7fcf2b600640 1 -- 172.21.15.39:0/2880348880 >> 172.21.15.39:0/2880348880 conn(0x7fcf240b2460 msgr2=0x7fcf2406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:55.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.827+0000 7fcf2b600640 1 -- 172.21.15.39:0/2880348880 shutdown_connections 2024-09-21T07:44:55.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.828+0000 7fcf2b600640 1 -- 172.21.15.39:0/2880348880 wait complete. 2024-09-21T07:44:55.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.830+0000 7fcf2b600640 1 Processor -- start 2024-09-21T07:44:55.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.830+0000 7fcf2b600640 1 -- start start 2024-09-21T07:44:55.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.831+0000 7fcf2b600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcf08004d40 0x7fcf241f7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:55.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.831+0000 7fcf2b600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf240c6370 0x7fcf240c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:55.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.831+0000 7fcf2b600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf240c98f0 0x7fcf240ad060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:55.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.831+0000 7fcf2b600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcf240cbc30 con 0x7fcf240c98f0 2024-09-21T07:44:55.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.831+0000 7fcf2b600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcf240cbab0 con 0x7fcf240c6370 2024-09-21T07:44:55.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.831+0000 7fcf2b600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fcf240af080 con 0x7fcf08004d40 2024-09-21T07:44:55.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.832+0000 7fcf29600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf240c98f0 0x7fcf240ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:55.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.832+0000 7fcf29600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf240c98f0 0x7fcf240ad060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47928/0 (socket says 172.21.15.39:47928) 2024-09-21T07:44:55.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.832+0000 7fcf29600640 1 -- 172.21.15.39:0/418425501 learned_addr learned my addr 172.21.15.39:0/418425501 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:55.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.832+0000 7fcf1be00640 1 --2- 172.21.15.39:0/418425501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf240c6370 0x7fcf240c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:55.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.832+0000 7fcf28c00640 1 --2- 172.21.15.39:0/418425501 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcf08004d40 0x7fcf241f7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:55.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.832+0000 7fcf29600640 1 -- 172.21.15.39:0/418425501 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcf08004d40 msgr2=0x7fcf241f7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:55.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.832+0000 7fcf29600640 1 --2- 172.21.15.39:0/418425501 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcf08004d40 0x7fcf241f7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:55.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.832+0000 7fcf29600640 1 -- 172.21.15.39:0/418425501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf240c6370 msgr2=0x7fcf240c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:55.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.832+0000 7fcf29600640 1 --2- 172.21.15.39:0/418425501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf240c6370 0x7fcf240c7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:55.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.832+0000 7fcf29600640 1 -- 172.21.15.39:0/418425501 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf240ca1f0 con 0x7fcf240c98f0 2024-09-21T07:44:55.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.832+0000 7fcf1be00640 1 --2- 172.21.15.39:0/418425501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf240c6370 0x7fcf240c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:44:55.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.832+0000 7fcf28c00640 1 --2- 172.21.15.39:0/418425501 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcf08004d40 0x7fcf241f7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:44:55.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.832+0000 7fcf29600640 1 --2- 172.21.15.39:0/418425501 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf240c98f0 0x7fcf240ad060 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c02b430 tx=0x7fcf1c001aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:55.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.833+0000 7fcf21600640 1 -- 172.21.15.39:0/418425501 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c039070 con 0x7fcf240c98f0 2024-09-21T07:44:55.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.833+0000 7fcf21600640 1 -- 172.21.15.39:0/418425501 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf1c006590 con 0x7fcf240c98f0 2024-09-21T07:44:55.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.833+0000 7fcf21600640 1 -- 172.21.15.39:0/418425501 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c002030 con 0x7fcf240c98f0 2024-09-21T07:44:55.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.833+0000 7fcf2b600640 1 -- 172.21.15.39:0/418425501 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf240c5310 con 0x7fcf240c98f0 2024-09-21T07:44:55.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.833+0000 7fcf2b600640 1 -- 172.21.15.39:0/418425501 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf241f8450 con 0x7fcf240c98f0 2024-09-21T07:44:55.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.834+0000 7fcf2b600640 1 -- 172.21.15.39:0/418425501 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf24156bb0 con 0x7fcf240c98f0 2024-09-21T07:44:55.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.835+0000 7fcf21600640 1 -- 172.21.15.39:0/418425501 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcf1c03c020 con 0x7fcf240c98f0 2024-09-21T07:44:55.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.836+0000 7fcf21600640 1 --2- 172.21.15.39:0/418425501 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcf00078bc0 0x7fcf0007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:55.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.836+0000 7fcf21600640 1 -- 172.21.15.39:0/418425501 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 6800+0+0 (secure 0 0 0) 0x7fcf1c0bac50 con 0x7fcf240c98f0 2024-09-21T07:44:55.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.836+0000 7fcf28c00640 1 --2- 172.21.15.39:0/418425501 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcf00078bc0 0x7fcf0007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:55.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.837+0000 7fcf28c00640 1 --2- 172.21.15.39:0/418425501 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcf00078bc0 0x7fcf0007b0d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fcf14009b40 tx=0x7fcf14005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:55.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:55.840+0000 7fcf21600640 1 -- 172.21.15.39:0/418425501 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcf1c045020 con 0x7fcf240c98f0 2024-09-21T07:44:56.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.040+0000 7fcf2b600640 1 -- 172.21.15.39:0/418425501 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcf240c6780 con 0x7fcf240c98f0 2024-09-21T07:44:56.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.042+0000 7fcf21600640 1 -- 172.21.15.39:0/418425501 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) ==== 74+0+22211 (secure 0 0 0) 0x7fcf240c5310 con 0x7fcf240c98f0 2024-09-21T07:44:56.044 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:56.044 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":133,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:44:55.080893+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:44:53.006035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-09-21T07:44:54.032155+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:44:56.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.044+0000 7fcf2b600640 1 -- 172.21.15.39:0/418425501 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcf00078bc0 msgr2=0x7fcf0007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:56.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.044+0000 7fcf2b600640 1 --2- 172.21.15.39:0/418425501 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcf00078bc0 0x7fcf0007b0d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fcf14009b40 tx=0x7fcf14005f90 comp rx=0 tx=0).stop 2024-09-21T07:44:56.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.044+0000 7fcf2b600640 1 -- 172.21.15.39:0/418425501 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf240c98f0 msgr2=0x7fcf240ad060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:56.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.044+0000 7fcf2b600640 1 --2- 172.21.15.39:0/418425501 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf240c98f0 0x7fcf240ad060 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c02b430 tx=0x7fcf1c001aa0 comp rx=0 tx=0).stop 2024-09-21T07:44:56.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.044+0000 7fcf2b600640 1 -- 172.21.15.39:0/418425501 shutdown_connections 2024-09-21T07:44:56.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.044+0000 7fcf2b600640 1 --2- 172.21.15.39:0/418425501 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcf00078bc0 0x7fcf0007b0d0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:56.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.044+0000 7fcf2b600640 1 --2- 172.21.15.39:0/418425501 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf240c98f0 0x7fcf240ad060 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:56.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.044+0000 7fcf2b600640 1 --2- 172.21.15.39:0/418425501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf240c6370 0x7fcf240c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:56.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.044+0000 7fcf2b600640 1 --2- 172.21.15.39:0/418425501 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcf08004d40 0x7fcf241f7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:56.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.045+0000 7fcf2b600640 1 -- 172.21.15.39:0/418425501 >> 172.21.15.39:0/418425501 conn(0x7fcf240b2460 msgr2=0x7fcf2415f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:56.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.045+0000 7fcf2b600640 1 -- 172.21.15.39:0/418425501 shutdown_connections 2024-09-21T07:44:56.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.045+0000 7fcf2b600640 1 -- 172.21.15.39:0/418425501 wait complete. 2024-09-21T07:44:56.072 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:44:56.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.296+0000 7fe70e200640 1 -- 172.21.15.39:0/2502942988 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7080b5a90 msgr2=0x7fe7080d1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:56.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.296+0000 7fe70e200640 1 --2- 172.21.15.39:0/2502942988 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7080b5a90 0x7fe7080d1ac0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe6f80098e0 tx=0x7fe6f802a5d0 comp rx=0 tx=0).stop 2024-09-21T07:44:56.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.297+0000 7fe70e200640 1 -- 172.21.15.39:0/2502942988 shutdown_connections 2024-09-21T07:44:56.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.297+0000 7fe70e200640 1 --2- 172.21.15.39:0/2502942988 >> v2:172.21.15.148:3300/0 conn(0x7fe7080b3b60 0x7fe7080b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:56.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.297+0000 7fe70e200640 1 --2- 172.21.15.39:0/2502942988 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7080b5a90 0x7fe7080d1ac0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:56.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.297+0000 7fe70e200640 1 -- 172.21.15.39:0/2502942988 >> 172.21.15.39:0/2502942988 conn(0x7fe70809d7b0 msgr2=0x7fe7080d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:56.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.297+0000 7fe70e200640 1 -- 172.21.15.39:0/2502942988 shutdown_connections 2024-09-21T07:44:56.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.297+0000 7fe70e200640 1 -- 172.21.15.39:0/2502942988 wait complete. 2024-09-21T07:44:56.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.299+0000 7fe70e200640 1 Processor -- start 2024-09-21T07:44:56.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.299+0000 7fe70e200640 1 -- start start 2024-09-21T07:44:56.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.301+0000 7fe70e200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7080b3b60 0x7fe708154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:56.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.301+0000 7fe70e200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7080b5a90 0x7fe7080cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:56.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.301+0000 7fe70e200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7080d0e70 0x7fe7080cb400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:56.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.301+0000 7fe70e200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe7080b4630 con 0x7fe7080b3b60 2024-09-21T07:44:56.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.301+0000 7fe70e200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe7080b44b0 con 0x7fe7080d0e70 2024-09-21T07:44:56.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.301+0000 7fe70e200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe7080b47b0 con 0x7fe7080b5a90 2024-09-21T07:44:56.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.301+0000 7fe70ce00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7080b3b60 0x7fe708154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:56.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.301+0000 7fe70ce00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7080b3b60 0x7fe708154a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47944/0 (socket says 172.21.15.39:47944) 2024-09-21T07:44:56.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.301+0000 7fe70ce00640 1 -- 172.21.15.39:0/3261208415 learned_addr learned my addr 172.21.15.39:0/3261208415 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:56.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.301+0000 7fe70d800640 1 --2- 172.21.15.39:0/3261208415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7080d0e70 0x7fe7080cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:56.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.301+0000 7fe6ffe00640 1 --2- 172.21.15.39:0/3261208415 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7080b5a90 0x7fe7080cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:56.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.302+0000 7fe70ce00640 1 -- 172.21.15.39:0/3261208415 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7080b5a90 msgr2=0x7fe7080cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:56.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.302+0000 7fe70ce00640 1 --2- 172.21.15.39:0/3261208415 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7080b5a90 0x7fe7080cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:56.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.302+0000 7fe70ce00640 1 -- 172.21.15.39:0/3261208415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7080d0e70 msgr2=0x7fe7080cb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:56.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.302+0000 7fe70ce00640 1 --2- 172.21.15.39:0/3261208415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7080d0e70 0x7fe7080cb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:56.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.302+0000 7fe70ce00640 1 -- 172.21.15.39:0/3261208415 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7080cdfd0 con 0x7fe7080b3b60 2024-09-21T07:44:56.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.302+0000 7fe70ce00640 1 --2- 172.21.15.39:0/3261208415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7080b3b60 0x7fe708154a90 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fe6f802abc0 tx=0x7fe6f80316b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:56.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.302+0000 7fe705600640 1 -- 172.21.15.39:0/3261208415 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f803f070 con 0x7fe7080b3b60 2024-09-21T07:44:56.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.302+0000 7fe705600640 1 -- 172.21.15.39:0/3261208415 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6f8031950 con 0x7fe7080b3b60 2024-09-21T07:44:56.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.302+0000 7fe705600640 1 -- 172.21.15.39:0/3261208415 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8031da0 con 0x7fe7080b3b60 2024-09-21T07:44:56.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.302+0000 7fe70e200640 1 -- 172.21.15.39:0/3261208415 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7080ce2b0 con 0x7fe7080b3b60 2024-09-21T07:44:56.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.302+0000 7fe70e200640 1 -- 172.21.15.39:0/3261208415 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7080caa60 con 0x7fe7080b3b60 2024-09-21T07:44:56.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.304+0000 7fe70e200640 1 -- 172.21.15.39:0/3261208415 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7080cacc0 con 0x7fe7080b3b60 2024-09-21T07:44:56.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.305+0000 7fe705600640 1 -- 172.21.15.39:0/3261208415 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe6f8042070 con 0x7fe7080b3b60 2024-09-21T07:44:56.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.306+0000 7fe705600640 1 --2- 172.21.15.39:0/3261208415 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe6e4078b90 0x7fe6e407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:56.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.306+0000 7fe705600640 1 -- 172.21.15.39:0/3261208415 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 6800+0+0 (secure 0 0 0) 0x7fe6f80bbd20 con 0x7fe7080b3b60 2024-09-21T07:44:56.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.306+0000 7fe6ffe00640 1 --2- 172.21.15.39:0/3261208415 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe6e4078b90 0x7fe6e407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:56.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.307+0000 7fe6ffe00640 1 --2- 172.21.15.39:0/3261208415 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe6e4078b90 0x7fe6e407b0a0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe70815e1c0 tx=0x7fe6f0009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:56.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.311+0000 7fe705600640 1 -- 172.21.15.39:0/3261208415 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe6f80836e0 con 0x7fe7080b3b60 2024-09-21T07:44:56.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.512+0000 7fe70e200640 1 -- 172.21.15.39:0/3261208415 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe7080b4c60 con 0x7fe7080b3b60 2024-09-21T07:44:56.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.514+0000 7fe705600640 1 -- 172.21.15.39:0/3261208415 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) ==== 74+0+22211 (secure 0 0 0) 0x7fe6f80bb070 con 0x7fe7080b3b60 2024-09-21T07:44:56.516 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:56.516 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":133,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:44:55.080893+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:44:53.006035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-09-21T07:44:54.032155+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:44:56.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.517+0000 7fe70e200640 1 -- 172.21.15.39:0/3261208415 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe6e4078b90 msgr2=0x7fe6e407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:56.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.517+0000 7fe70e200640 1 --2- 172.21.15.39:0/3261208415 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe6e4078b90 0x7fe6e407b0a0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe70815e1c0 tx=0x7fe6f0009040 comp rx=0 tx=0).stop 2024-09-21T07:44:56.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.517+0000 7fe70e200640 1 -- 172.21.15.39:0/3261208415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7080b3b60 msgr2=0x7fe708154a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:56.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.517+0000 7fe70e200640 1 --2- 172.21.15.39:0/3261208415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7080b3b60 0x7fe708154a90 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fe6f802abc0 tx=0x7fe6f80316b0 comp rx=0 tx=0).stop 2024-09-21T07:44:56.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.517+0000 7fe70e200640 1 -- 172.21.15.39:0/3261208415 shutdown_connections 2024-09-21T07:44:56.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.517+0000 7fe70e200640 1 --2- 172.21.15.39:0/3261208415 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe6e4078b90 0x7fe6e407b0a0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:56.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.517+0000 7fe70e200640 1 --2- 172.21.15.39:0/3261208415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7080d0e70 0x7fe7080cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:56.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.517+0000 7fe70e200640 1 --2- 172.21.15.39:0/3261208415 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7080b5a90 0x7fe7080cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:56.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.517+0000 7fe70e200640 1 --2- 172.21.15.39:0/3261208415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7080b3b60 0x7fe708154a90 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:56.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.517+0000 7fe70e200640 1 -- 172.21.15.39:0/3261208415 >> 172.21.15.39:0/3261208415 conn(0x7fe70809d7b0 msgr2=0x7fe7080b8030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:56.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.517+0000 7fe70e200640 1 -- 172.21.15.39:0/3261208415 shutdown_connections 2024-09-21T07:44:56.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.517+0000 7fe70e200640 1 -- 172.21.15.39:0/3261208415 wait complete. 2024-09-21T07:44:56.544 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T07:44:56.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.775+0000 7fdce0a00640 1 -- 172.21.15.39:0/1504555861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcc0005210 msgr2=0x7fdcc000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:56.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.775+0000 7fdce0a00640 1 --2- 172.21.15.39:0/1504555861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcc0005210 0x7fdcc000b9c0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0cecb0 tx=0x7fdcc0005830 comp rx=0 tx=0).stop 2024-09-21T07:44:56.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.775+0000 7fdcda800640 1 -- 172.21.15.39:0/1504555861 reap_dead start 2024-09-21T07:44:56.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.775+0000 7fdce0a00640 1 -- 172.21.15.39:0/1504555861 shutdown_connections 2024-09-21T07:44:56.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.775+0000 7fdce0a00640 1 -- 172.21.15.39:0/1504555861 >> 172.21.15.39:0/1504555861 conn(0x7fdcdc06cfc0 msgr2=0x7fdcdc06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:56.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.775+0000 7fdce0a00640 1 -- 172.21.15.39:0/1504555861 shutdown_connections 2024-09-21T07:44:56.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.776+0000 7fdce0a00640 1 -- 172.21.15.39:0/1504555861 wait complete. 2024-09-21T07:44:56.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.778+0000 7fdce0a00640 1 Processor -- start 2024-09-21T07:44:56.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.778+0000 7fdce0a00640 1 -- start start 2024-09-21T07:44:56.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.779+0000 7fdce0a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdcc0005210 0x7fdcdc14d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:56.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.779+0000 7fdce0a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcdc155770 0x7fdcdc1601b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:56.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.779+0000 7fdce0a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcdc153e80 0x7fdcdc1f6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:56.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.779+0000 7fdce0a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc0cfbf0 con 0x7fdcdc153e80 2024-09-21T07:44:56.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.779+0000 7fdce0a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc0cfa70 con 0x7fdcdc155770 2024-09-21T07:44:56.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.779+0000 7fdce0a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fdcdc165520 con 0x7fdcc0005210 2024-09-21T07:44:56.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.780+0000 7fdcda800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcdc153e80 0x7fdcdc1f6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:56.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.780+0000 7fdcda800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcdc153e80 0x7fdcdc1f6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47952/0 (socket says 172.21.15.39:47952) 2024-09-21T07:44:56.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.780+0000 7fdcda800640 1 -- 172.21.15.39:0/1192449150 learned_addr learned my addr 172.21.15.39:0/1192449150 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:56.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.780+0000 7fdcd9e00640 1 --2- 172.21.15.39:0/1192449150 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdcc0005210 0x7fdcdc14d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:56.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.780+0000 7fdcd1400640 1 --2- 172.21.15.39:0/1192449150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcdc155770 0x7fdcdc1601b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:56.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.780+0000 7fdcda800640 1 -- 172.21.15.39:0/1192449150 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdcc0005210 msgr2=0x7fdcdc14d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:56.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.780+0000 7fdcda800640 1 --2- 172.21.15.39:0/1192449150 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdcc0005210 0x7fdcdc14d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:56.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.780+0000 7fdcda800640 1 -- 172.21.15.39:0/1192449150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcdc155770 msgr2=0x7fdcdc1601b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:56.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.780+0000 7fdcda800640 1 --2- 172.21.15.39:0/1192449150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcdc155770 0x7fdcdc1601b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:56.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.780+0000 7fdcda800640 1 -- 172.21.15.39:0/1192449150 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcdc152de0 con 0x7fdcdc153e80 2024-09-21T07:44:56.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.780+0000 7fdcd9e00640 1 --2- 172.21.15.39:0/1192449150 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdcc0005210 0x7fdcdc14d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:44:56.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.781+0000 7fdcda800640 1 --2- 172.21.15.39:0/1192449150 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcdc153e80 0x7fdcdc1f6ac0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4005fd0 tx=0x7fdcd4033bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:56.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.781+0000 7fdcd2a00640 1 -- 172.21.15.39:0/1192449150 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4043070 con 0x7fdcdc153e80 2024-09-21T07:44:56.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.781+0000 7fdcd2a00640 1 -- 172.21.15.39:0/1192449150 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdcd4003970 con 0x7fdcdc153e80 2024-09-21T07:44:56.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.781+0000 7fdcd2a00640 1 -- 172.21.15.39:0/1192449150 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4038040 con 0x7fdcdc153e80 2024-09-21T07:44:56.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.781+0000 7fdce0a00640 1 -- 172.21.15.39:0/1192449150 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcdc1530c0 con 0x7fdcdc153e80 2024-09-21T07:44:56.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.781+0000 7fdce0a00640 1 -- 172.21.15.39:0/1192449150 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcdc153680 con 0x7fdcdc153e80 2024-09-21T07:44:56.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.782+0000 7fdce0a00640 1 -- 172.21.15.39:0/1192449150 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcdc15d5b0 con 0x7fdcdc153e80 2024-09-21T07:44:56.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.783+0000 7fdcd2a00640 1 -- 172.21.15.39:0/1192449150 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdcd4046070 con 0x7fdcdc153e80 2024-09-21T07:44:56.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.784+0000 7fdcd2a00640 1 --2- 172.21.15.39:0/1192449150 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdcb8078b90 0x7fdcb807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:56.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.784+0000 7fdcd2a00640 1 -- 172.21.15.39:0/1192449150 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 6800+0+0 (secure 0 0 0) 0x7fdcd40bbcc0 con 0x7fdcdc153e80 2024-09-21T07:44:56.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.785+0000 7fdcd9e00640 1 --2- 172.21.15.39:0/1192449150 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdcb8078b90 0x7fdcb807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:56.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.786+0000 7fdcd9e00640 1 --2- 172.21.15.39:0/1192449150 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdcb8078b90 0x7fdcb807b0a0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0730e0 tx=0x7fdccc006720 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:56.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:56.789+0000 7fdcd2a00640 1 -- 172.21.15.39:0/1192449150 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdcd40c2080 con 0x7fdcdc153e80 2024-09-21T07:44:57.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.028+0000 7fdce0a00640 1 -- 172.21.15.39:0/1192449150 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fdcdc152b70 con 0x7fdcdc153e80 2024-09-21T07:44:57.520 INFO:tasks.ceph.mds.a.smithi039.stderr:2024-09-21T07:44:57.518+0000 7f6034000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:44:57.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.572+0000 7fdcd2a00640 1 -- 172.21.15.39:0/1192449150 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v171) ==== 118+0+0 (secure 0 0 0) 0x7fdcd40835f0 con 0x7fdcdc153e80 2024-09-21T07:44:57.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.575+0000 7fdce0a00640 1 -- 172.21.15.39:0/1192449150 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdcb8078b90 msgr2=0x7fdcb807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:57.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.575+0000 7fdce0a00640 1 --2- 172.21.15.39:0/1192449150 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdcb8078b90 0x7fdcb807b0a0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0730e0 tx=0x7fdccc006720 comp rx=0 tx=0).stop 2024-09-21T07:44:57.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.575+0000 7fdce0a00640 1 -- 172.21.15.39:0/1192449150 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcdc153e80 msgr2=0x7fdcdc1f6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:57.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.575+0000 7fdce0a00640 1 --2- 172.21.15.39:0/1192449150 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcdc153e80 0x7fdcdc1f6ac0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4005fd0 tx=0x7fdcd4033bc0 comp rx=0 tx=0).stop 2024-09-21T07:44:57.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.575+0000 7fdce0a00640 1 -- 172.21.15.39:0/1192449150 shutdown_connections 2024-09-21T07:44:57.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.575+0000 7fdce0a00640 1 --2- 172.21.15.39:0/1192449150 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdcb8078b90 0x7fdcb807b0a0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:57.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.575+0000 7fdce0a00640 1 --2- 172.21.15.39:0/1192449150 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcdc153e80 0x7fdcdc1f6ac0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:57.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.575+0000 7fdce0a00640 1 --2- 172.21.15.39:0/1192449150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcdc155770 0x7fdcdc1601b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:57.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.575+0000 7fdce0a00640 1 --2- 172.21.15.39:0/1192449150 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdcc0005210 0x7fdcdc14d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:57.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.575+0000 7fdce0a00640 1 -- 172.21.15.39:0/1192449150 >> 172.21.15.39:0/1192449150 conn(0x7fdcdc06cfc0 msgr2=0x7fdcdc151380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:57.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.575+0000 7fdce0a00640 1 -- 172.21.15.39:0/1192449150 shutdown_connections 2024-09-21T07:44:57.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.576+0000 7fdce0a00640 1 -- 172.21.15.39:0/1192449150 wait complete. 2024-09-21T07:44:57.604 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T07:44:57.604 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T07:44:57.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.833+0000 7f9075400640 1 -- 172.21.15.39:0/852488615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9070160250 msgr2=0x7f907015d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:57.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.833+0000 7f9075400640 1 --2- 172.21.15.39:0/852488615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9070160250 0x7f907015d570 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f9060005fd0 tx=0x7f906002a580 comp rx=0 tx=0).stop 2024-09-21T07:44:57.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.833+0000 7f9075400640 1 -- 172.21.15.39:0/852488615 shutdown_connections 2024-09-21T07:44:57.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.833+0000 7f9075400640 1 --2- 172.21.15.39:0/852488615 >> v2:172.21.15.148:3300/0 conn(0x7f90700d0390 0x7f90700d07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:57.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.833+0000 7f9075400640 1 --2- 172.21.15.39:0/852488615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9070160250 0x7f907015d570 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:57.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.833+0000 7f9075400640 1 -- 172.21.15.39:0/852488615 >> 172.21.15.39:0/852488615 conn(0x7f907006cfc0 msgr2=0x7f907006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:57.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.833+0000 7f9075400640 1 -- 172.21.15.39:0/852488615 shutdown_connections 2024-09-21T07:44:57.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.833+0000 7f9075400640 1 -- 172.21.15.39:0/852488615 wait complete. 2024-09-21T07:44:57.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.835+0000 7f9075400640 1 Processor -- start 2024-09-21T07:44:57.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.835+0000 7f9075400640 1 -- start start 2024-09-21T07:44:57.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.837+0000 7f9075400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90700d0390 0x7f9070145110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:57.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.837+0000 7f9075400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9070160250 0x7f90701430b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:57.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.837+0000 7f9075400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9070160660 0x7f90701fde50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:57.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.837+0000 7f9075400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f90700ccdc0 con 0x7f9070160660 2024-09-21T07:44:57.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.837+0000 7f9075400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f90700ccc40 con 0x7f90700d0390 2024-09-21T07:44:57.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.837+0000 7f9075400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f90700ccf40 con 0x7f9070160250 2024-09-21T07:44:57.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.837+0000 7f9074a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9070160660 0x7f90701fde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:57.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.837+0000 7f9074a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9070160660 0x7f90701fde50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47974/0 (socket says 172.21.15.39:47974) 2024-09-21T07:44:57.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.837+0000 7f9074a00640 1 -- 172.21.15.39:0/4037559181 learned_addr learned my addr 172.21.15.39:0/4037559181 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:57.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.838+0000 7f906fe00640 1 --2- 172.21.15.39:0/4037559181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90700d0390 0x7f9070145110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:57.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.838+0000 7f9067e00640 1 --2- 172.21.15.39:0/4037559181 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9070160250 0x7f90701430b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:57.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.838+0000 7f9074a00640 1 -- 172.21.15.39:0/4037559181 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9070160250 msgr2=0x7f90701430b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:57.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.838+0000 7f9074a00640 1 --2- 172.21.15.39:0/4037559181 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9070160250 0x7f90701430b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:57.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.838+0000 7f9074a00640 1 -- 172.21.15.39:0/4037559181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90700d0390 msgr2=0x7f9070145110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:57.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.838+0000 7f9074a00640 1 --2- 172.21.15.39:0/4037559181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90700d0390 0x7f9070145110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:57.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.838+0000 7f9074a00640 1 -- 172.21.15.39:0/4037559181 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9070145a40 con 0x7f9070160660 2024-09-21T07:44:57.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.838+0000 7f906fe00640 1 --2- 172.21.15.39:0/4037559181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90700d0390 0x7f9070145110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:44:57.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.838+0000 7f9074a00640 1 --2- 172.21.15.39:0/4037559181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9070160660 0x7f90701fde50 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f906800e960 tx=0x7f906800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:57.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.838+0000 7f906cc00640 1 -- 172.21.15.39:0/4037559181 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9068004d20 con 0x7f9070160660 2024-09-21T07:44:57.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.838+0000 7f906cc00640 1 -- 172.21.15.39:0/4037559181 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f906800cb20 con 0x7f9070160660 2024-09-21T07:44:57.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.838+0000 7f906cc00640 1 -- 172.21.15.39:0/4037559181 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9068005020 con 0x7f9070160660 2024-09-21T07:44:57.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.838+0000 7f9075400640 1 -- 172.21.15.39:0/4037559181 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9070145d20 con 0x7f9070160660 2024-09-21T07:44:57.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.839+0000 7f9075400640 1 -- 172.21.15.39:0/4037559181 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90701fd650 con 0x7f9070160660 2024-09-21T07:44:57.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.840+0000 7f9075400640 1 -- 172.21.15.39:0/4037559181 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90701457b0 con 0x7f9070160660 2024-09-21T07:44:57.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.841+0000 7f906cc00640 1 -- 172.21.15.39:0/4037559181 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f90680051c0 con 0x7f9070160660 2024-09-21T07:44:57.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.842+0000 7f906cc00640 1 --2- 172.21.15.39:0/4037559181 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f904c078b90 0x7f904c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:57.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.842+0000 7f906cc00640 1 -- 172.21.15.39:0/4037559181 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 6800+0+0 (secure 0 0 0) 0x7f906809bae0 con 0x7f9070160660 2024-09-21T07:44:57.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.842+0000 7f906fe00640 1 --2- 172.21.15.39:0/4037559181 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f904c078b90 0x7f904c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:57.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.843+0000 7f906fe00640 1 --2- 172.21.15.39:0/4037559181 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f904c078b90 0x7f904c07b0a0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f9060033000 tx=0x7f9060004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:57.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:57.846+0000 7f906cc00640 1 -- 172.21.15.39:0/4037559181 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9068016030 con 0x7f9070160660 2024-09-21T07:44:58.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.082+0000 7f9075400640 1 -- 172.21.15.39:0/4037559181 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f90700cd460 con 0x7f9070160660 2024-09-21T07:44:58.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.589+0000 7f906cc00640 1 -- 172.21.15.39:0/4037559181 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v172) ==== 122+0+0 (secure 0 0 0) 0x7f9068063af0 con 0x7f9070160660 2024-09-21T07:44:58.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.592+0000 7f9075400640 1 -- 172.21.15.39:0/4037559181 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f904c078b90 msgr2=0x7f904c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:58.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.592+0000 7f9075400640 1 --2- 172.21.15.39:0/4037559181 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f904c078b90 0x7f904c07b0a0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f9060033000 tx=0x7f9060004730 comp rx=0 tx=0).stop 2024-09-21T07:44:58.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.592+0000 7f9075400640 1 -- 172.21.15.39:0/4037559181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9070160660 msgr2=0x7f90701fde50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:58.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.592+0000 7f9075400640 1 --2- 172.21.15.39:0/4037559181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9070160660 0x7f90701fde50 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f906800e960 tx=0x7f906800ee20 comp rx=0 tx=0).stop 2024-09-21T07:44:58.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.592+0000 7f9075400640 1 -- 172.21.15.39:0/4037559181 shutdown_connections 2024-09-21T07:44:58.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.592+0000 7f9075400640 1 --2- 172.21.15.39:0/4037559181 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f904c078b90 0x7f904c07b0a0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:58.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.592+0000 7f9075400640 1 --2- 172.21.15.39:0/4037559181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9070160660 0x7f90701fde50 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:58.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.592+0000 7f9075400640 1 --2- 172.21.15.39:0/4037559181 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9070160250 0x7f90701430b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:58.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.592+0000 7f9075400640 1 --2- 172.21.15.39:0/4037559181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f90700d0390 0x7f9070145110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:58.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.592+0000 7f9075400640 1 -- 172.21.15.39:0/4037559181 >> 172.21.15.39:0/4037559181 conn(0x7f907006cfc0 msgr2=0x7f9070155190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:58.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.592+0000 7f9075400640 1 -- 172.21.15.39:0/4037559181 shutdown_connections 2024-09-21T07:44:58.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.592+0000 7f9075400640 1 -- 172.21.15.39:0/4037559181 wait complete. 2024-09-21T07:44:58.623 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T07:44:58.624 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:44:58.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.850+0000 7ff301e00640 1 -- 172.21.15.39:0/1374987578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2fc0d0410 msgr2=0x7ff2fc0d2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:58.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.850+0000 7ff301e00640 1 --2- 172.21.15.39:0/1374987578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2fc0d0410 0x7ff2fc0d2230 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc06f6a0 tx=0x7ff2f002a4b0 comp rx=0 tx=0).stop 2024-09-21T07:44:58.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.850+0000 7ff301e00640 1 -- 172.21.15.39:0/1374987578 shutdown_connections 2024-09-21T07:44:58.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.850+0000 7ff301e00640 1 --2- 172.21.15.39:0/1374987578 >> v2:172.21.15.148:3300/0 conn(0x7ff2fc06ec80 0x7ff2fc0cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:58.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.850+0000 7ff301e00640 1 --2- 172.21.15.39:0/1374987578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2fc0d0410 0x7ff2fc0d2230 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:58.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.850+0000 7ff301e00640 1 -- 172.21.15.39:0/1374987578 >> 172.21.15.39:0/1374987578 conn(0x7ff2fc09b5f0 msgr2=0x7ff2fc08c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:58.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.850+0000 7ff301e00640 1 -- 172.21.15.39:0/1374987578 shutdown_connections 2024-09-21T07:44:58.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.850+0000 7ff301e00640 1 -- 172.21.15.39:0/1374987578 wait complete. 2024-09-21T07:44:58.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.852+0000 7ff301e00640 1 Processor -- start 2024-09-21T07:44:58.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.852+0000 7ff301e00640 1 -- start start 2024-09-21T07:44:58.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.854+0000 7ff301e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2fc06ec80 0x7ff2fc0b9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:58.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.854+0000 7ff301e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2fc0d0410 0x7ff2fc0bc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:58.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.854+0000 7ff301e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2fc0d2770 0x7ff2fc1f06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:58.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.854+0000 7ff301e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc0cd3f0 con 0x7ff2fc0d2770 2024-09-21T07:44:58.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.854+0000 7ff301e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc0cd270 con 0x7ff2fc0d0410 2024-09-21T07:44:58.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.854+0000 7ff301e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7ff2fc0cd570 con 0x7ff2fc06ec80 2024-09-21T07:44:58.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.854+0000 7ff301400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2fc0d2770 0x7ff2fc1f06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:58.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.854+0000 7ff301400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2fc0d2770 0x7ff2fc1f06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47994/0 (socket says 172.21.15.39:47994) 2024-09-21T07:44:58.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.854+0000 7ff301400640 1 -- 172.21.15.39:0/1239527212 learned_addr learned my addr 172.21.15.39:0/1239527212 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:58.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.854+0000 7ff300a00640 1 --2- 172.21.15.39:0/1239527212 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2fc06ec80 0x7ff2fc0b9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:58.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.854+0000 7ff2fbe00640 1 --2- 172.21.15.39:0/1239527212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2fc0d0410 0x7ff2fc0bc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:58.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.855+0000 7ff301400640 1 -- 172.21.15.39:0/1239527212 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2fc06ec80 msgr2=0x7ff2fc0b9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:58.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.855+0000 7ff301400640 1 --2- 172.21.15.39:0/1239527212 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2fc06ec80 0x7ff2fc0b9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:58.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.855+0000 7ff301400640 1 -- 172.21.15.39:0/1239527212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2fc0d0410 msgr2=0x7ff2fc0bc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:58.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.855+0000 7ff301400640 1 --2- 172.21.15.39:0/1239527212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2fc0d0410 0x7ff2fc0bc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:58.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.855+0000 7ff301400640 1 -- 172.21.15.39:0/1239527212 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2fc0bb210 con 0x7ff2fc0d2770 2024-09-21T07:44:58.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.855+0000 7ff301400640 1 --2- 172.21.15.39:0/1239527212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2fc0d2770 0x7ff2fc1f06d0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400e990 tx=0x7ff2f400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:58.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.855+0000 7ff2f8c00640 1 -- 172.21.15.39:0/1239527212 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4004d20 con 0x7ff2fc0d2770 2024-09-21T07:44:58.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.855+0000 7ff2f8c00640 1 -- 172.21.15.39:0/1239527212 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2f400caa0 con 0x7ff2fc0d2770 2024-09-21T07:44:58.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.855+0000 7ff301e00640 1 -- 172.21.15.39:0/1239527212 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2fc0bb4f0 con 0x7ff2fc0d2770 2024-09-21T07:44:58.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.855+0000 7ff2f8c00640 1 -- 172.21.15.39:0/1239527212 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4005020 con 0x7ff2fc0d2770 2024-09-21T07:44:58.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.856+0000 7ff301e00640 1 -- 172.21.15.39:0/1239527212 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2fc0bbab0 con 0x7ff2fc0d2770 2024-09-21T07:44:58.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.857+0000 7ff301e00640 1 -- 172.21.15.39:0/1239527212 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2fc0cdad0 con 0x7ff2fc0d2770 2024-09-21T07:44:58.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.858+0000 7ff2f8c00640 1 -- 172.21.15.39:0/1239527212 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff2f40051c0 con 0x7ff2fc0d2770 2024-09-21T07:44:58.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.859+0000 7ff2f8c00640 1 --2- 172.21.15.39:0/1239527212 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff2dc078b90 0x7ff2dc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:58.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.859+0000 7ff2f8c00640 1 -- 172.21.15.39:0/1239527212 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 6800+0+0 (secure 0 0 0) 0x7ff2f409bc90 con 0x7ff2fc0d2770 2024-09-21T07:44:58.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.859+0000 7ff300a00640 1 --2- 172.21.15.39:0/1239527212 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff2dc078b90 0x7ff2dc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:58.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.860+0000 7ff300a00640 1 --2- 172.21.15.39:0/1239527212 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff2dc078b90 0x7ff2dc07b0a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0034000 tx=0x7ff2f0004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:58.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:58.863+0000 7ff2f8c00640 1 -- 172.21.15.39:0/1239527212 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff2f4016030 con 0x7ff2fc0d2770 2024-09-21T07:44:59.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.099+0000 7ff301e00640 1 -- 172.21.15.39:0/1239527212 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2fc162270 con 0x7ff2fc0d2770 2024-09-21T07:44:59.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.099+0000 7ff2f8c00640 1 -- 172.21.15.39:0/1239527212 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 172 v172) ==== 95+0+5237 (secure 0 0 0) 0x7ff2f40636d0 con 0x7ff2fc0d2770 2024-09-21T07:44:59.101 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:59.101 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":172,"btime":"2024-09-21T07:44:58:579257+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7127,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3876879654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3876879654},{"type":"v1","addr":"172.21.15.148:6836","nonce":3876879654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":167},{"gid":7130,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3044588529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3044588529},{"type":"v1","addr":"172.21.15.148:6837","nonce":3044588529}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":168},{"gid":8517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/3334092885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3334092885},{"type":"v1","addr":"172.21.15.39:6834","nonce":3334092885}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":168}],"filesystems":[{"mdsmap":{"epoch":172,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:44:55.080850+0000","modified":"2024-09-21T07:44:58.083710+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8520":{"gid":8520,"name":"a","rank":0,"incarnation":169,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/3547777254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3547777254},{"type":"v1","addr":"172.21.15.39:6835","nonce":3547777254}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8520,"qdb_cluster":[8520]},"id":7}]} 2024-09-21T07:44:59.102 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 172 2024-09-21T07:44:59.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.101+0000 7ff301e00640 1 -- 172.21.15.39:0/1239527212 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff2dc078b90 msgr2=0x7ff2dc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:59.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.101+0000 7ff301e00640 1 --2- 172.21.15.39:0/1239527212 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff2dc078b90 0x7ff2dc07b0a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0034000 tx=0x7ff2f0004730 comp rx=0 tx=0).stop 2024-09-21T07:44:59.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.102+0000 7ff301e00640 1 -- 172.21.15.39:0/1239527212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2fc0d2770 msgr2=0x7ff2fc1f06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:59.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.102+0000 7ff301e00640 1 --2- 172.21.15.39:0/1239527212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2fc0d2770 0x7ff2fc1f06d0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400e990 tx=0x7ff2f400ee50 comp rx=0 tx=0).stop 2024-09-21T07:44:59.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.102+0000 7ff301e00640 1 -- 172.21.15.39:0/1239527212 shutdown_connections 2024-09-21T07:44:59.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.102+0000 7ff301e00640 1 --2- 172.21.15.39:0/1239527212 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff2dc078b90 0x7ff2dc07b0a0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:59.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.102+0000 7ff301e00640 1 --2- 172.21.15.39:0/1239527212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2fc0d2770 0x7ff2fc1f06d0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:59.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.102+0000 7ff301e00640 1 --2- 172.21.15.39:0/1239527212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2fc0d0410 0x7ff2fc0bc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:59.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.102+0000 7ff301e00640 1 --2- 172.21.15.39:0/1239527212 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff2fc06ec80 0x7ff2fc0b9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:59.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.102+0000 7ff301e00640 1 -- 172.21.15.39:0/1239527212 >> 172.21.15.39:0/1239527212 conn(0x7ff2fc09b5f0 msgr2=0x7ff2fc0cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:59.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.102+0000 7ff301e00640 1 -- 172.21.15.39:0/1239527212 shutdown_connections 2024-09-21T07:44:59.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.102+0000 7ff301e00640 1 -- 172.21.15.39:0/1239527212 wait complete. 2024-09-21T07:44:59.127 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:44:59.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.353+0000 7fa4a2600640 1 -- 172.21.15.39:0/1840436136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa49c162520 msgr2=0x7fa49c162930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:59.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.353+0000 7fa4a2600640 1 --2- 172.21.15.39:0/1840436136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa49c162520 0x7fa49c162930 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa484009830 tx=0x7fa48402a5c0 comp rx=0 tx=0).stop 2024-09-21T07:44:59.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.353+0000 7fa4a2600640 1 -- 172.21.15.39:0/1840436136 shutdown_connections 2024-09-21T07:44:59.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.353+0000 7fa4a2600640 1 --2- 172.21.15.39:0/1840436136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa49c162520 0x7fa49c162930 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:59.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.353+0000 7fa4a2600640 1 -- 172.21.15.39:0/1840436136 >> 172.21.15.39:0/1840436136 conn(0x7fa49c06cfc0 msgr2=0x7fa49c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:59.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.353+0000 7fa4a2600640 1 -- 172.21.15.39:0/1840436136 shutdown_connections 2024-09-21T07:44:59.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.353+0000 7fa4a2600640 1 -- 172.21.15.39:0/1840436136 wait complete. 2024-09-21T07:44:59.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.355+0000 7fa4a2600640 1 Processor -- start 2024-09-21T07:44:59.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.355+0000 7fa4a2600640 1 -- start start 2024-09-21T07:44:59.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.357+0000 7fa4a2600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa49c0d06d0 0x7fa49c1e9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:59.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.357+0000 7fa4a2600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa49c15fb20 0x7fa49c0b1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:59.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.357+0000 7fa4a2600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa49c162520 0x7fa49c0b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:59.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.357+0000 7fa4a2600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa49c070570 con 0x7fa49c0d06d0 2024-09-21T07:44:59.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.357+0000 7fa4a2600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa49c0703f0 con 0x7fa49c15fb20 2024-09-21T07:44:59.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.357+0000 7fa4a2600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa49c0706f0 con 0x7fa49c162520 2024-09-21T07:44:59.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.357+0000 7fa49b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa49c0d06d0 0x7fa49c1e9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:59.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.357+0000 7fa49b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa49c0d06d0 0x7fa49c1e9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47998/0 (socket says 172.21.15.39:47998) 2024-09-21T07:44:59.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.357+0000 7fa49b400640 1 -- 172.21.15.39:0/402829631 learned_addr learned my addr 172.21.15.39:0/402829631 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:59.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.357+0000 7fa493e00640 1 --2- 172.21.15.39:0/402829631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa49c15fb20 0x7fa49c0b1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:59.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.357+0000 7fa49be00640 1 --2- 172.21.15.39:0/402829631 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa49c162520 0x7fa49c0b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:59.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.358+0000 7fa49b400640 1 -- 172.21.15.39:0/402829631 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa49c162520 msgr2=0x7fa49c0b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:59.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.358+0000 7fa49b400640 1 --2- 172.21.15.39:0/402829631 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa49c162520 0x7fa49c0b1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:59.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.358+0000 7fa49b400640 1 -- 172.21.15.39:0/402829631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa49c15fb20 msgr2=0x7fa49c0b1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:59.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.358+0000 7fa49b400640 1 --2- 172.21.15.39:0/402829631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa49c15fb20 0x7fa49c0b1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:59.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.358+0000 7fa49b400640 1 -- 172.21.15.39:0/402829631 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa49c1e88d0 con 0x7fa49c0d06d0 2024-09-21T07:44:59.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.358+0000 7fa49be00640 1 --2- 172.21.15.39:0/402829631 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa49c162520 0x7fa49c0b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:44:59.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.358+0000 7fa49b400640 1 --2- 172.21.15.39:0/402829631 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa49c0d06d0 0x7fa49c1e9690 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fa48c00c430 tx=0x7fa48c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:59.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.358+0000 7fa493400640 1 -- 172.21.15.39:0/402829631 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48c017070 con 0x7fa49c0d06d0 2024-09-21T07:44:59.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.358+0000 7fa493400640 1 -- 172.21.15.39:0/402829631 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa48c0053e0 con 0x7fa49c0d06d0 2024-09-21T07:44:59.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.358+0000 7fa4a2600640 1 -- 172.21.15.39:0/402829631 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa49c1e8bb0 con 0x7fa49c0d06d0 2024-09-21T07:44:59.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.358+0000 7fa493400640 1 -- 172.21.15.39:0/402829631 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48c003850 con 0x7fa49c0d06d0 2024-09-21T07:44:59.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.358+0000 7fa4a2600640 1 -- 172.21.15.39:0/402829631 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa49c158df0 con 0x7fa49c0d06d0 2024-09-21T07:44:59.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.359+0000 7fa4a2600640 1 -- 172.21.15.39:0/402829631 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa49c162930 con 0x7fa49c0d06d0 2024-09-21T07:44:59.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.361+0000 7fa493400640 1 -- 172.21.15.39:0/402829631 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa48c01a070 con 0x7fa49c0d06d0 2024-09-21T07:44:59.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.361+0000 7fa493400640 1 --2- 172.21.15.39:0/402829631 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa478078b90 0x7fa47807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:59.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.362+0000 7fa493400640 1 -- 172.21.15.39:0/402829631 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 6800+0+0 (secure 0 0 0) 0x7fa48c09a460 con 0x7fa49c0d06d0 2024-09-21T07:44:59.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.362+0000 7fa493e00640 1 --2- 172.21.15.39:0/402829631 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa478078b90 0x7fa47807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:59.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.363+0000 7fa493e00640 1 --2- 172.21.15.39:0/402829631 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa478078b90 0x7fa47807b0a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa484034000 tx=0x7fa484004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:59.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.366+0000 7fa493400640 1 -- 172.21.15.39:0/402829631 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa48c061eb0 con 0x7fa49c0d06d0 2024-09-21T07:44:59.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.565+0000 7fa4a2600640 1 -- 172.21.15.39:0/402829631 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa49c1e8580 con 0x7fa49c0d06d0 2024-09-21T07:44:59.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.566+0000 7fa493400640 1 -- 172.21.15.39:0/402829631 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) ==== 74+0+22198 (secure 0 0 0) 0x7fa48c0668e0 con 0x7fa49c0d06d0 2024-09-21T07:44:59.568 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:44:59.568 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":134,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:44:56.565966+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:44:53.006035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-09-21T07:44:54.032155+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:44:59.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.569+0000 7fa4a2600640 1 -- 172.21.15.39:0/402829631 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa478078b90 msgr2=0x7fa47807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:59.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.569+0000 7fa4a2600640 1 --2- 172.21.15.39:0/402829631 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa478078b90 0x7fa47807b0a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa484034000 tx=0x7fa484004730 comp rx=0 tx=0).stop 2024-09-21T07:44:59.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.569+0000 7fa4a2600640 1 -- 172.21.15.39:0/402829631 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa49c0d06d0 msgr2=0x7fa49c1e9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:59.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.569+0000 7fa4a2600640 1 --2- 172.21.15.39:0/402829631 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa49c0d06d0 0x7fa49c1e9690 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fa48c00c430 tx=0x7fa48c00c8f0 comp rx=0 tx=0).stop 2024-09-21T07:44:59.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.569+0000 7fa4a2600640 1 -- 172.21.15.39:0/402829631 shutdown_connections 2024-09-21T07:44:59.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.569+0000 7fa4a2600640 1 --2- 172.21.15.39:0/402829631 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa478078b90 0x7fa47807b0a0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:59.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.569+0000 7fa4a2600640 1 --2- 172.21.15.39:0/402829631 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa49c162520 0x7fa49c0b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:59.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.569+0000 7fa4a2600640 1 --2- 172.21.15.39:0/402829631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa49c15fb20 0x7fa49c0b1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:59.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.569+0000 7fa4a2600640 1 --2- 172.21.15.39:0/402829631 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa49c0d06d0 0x7fa49c1e9690 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:59.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.569+0000 7fa4a2600640 1 -- 172.21.15.39:0/402829631 >> 172.21.15.39:0/402829631 conn(0x7fa49c06cfc0 msgr2=0x7fa49c0b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:59.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.569+0000 7fa4a2600640 1 -- 172.21.15.39:0/402829631 shutdown_connections 2024-09-21T07:44:59.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.569+0000 7fa4a2600640 1 -- 172.21.15.39:0/402829631 wait complete. 2024-09-21T07:44:59.593 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T07:44:59.594 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:44:59.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.820+0000 7f3b21800640 1 -- 172.21.15.39:0/2045871622 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b000053f0 msgr2=0x7f3b0000ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:59.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.820+0000 7f3b21800640 1 --2- 172.21.15.39:0/2045871622 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b000053f0 0x7f3b0000ba80 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f3b14009e20 tx=0x7f3b00005830 comp rx=0 tx=0).stop 2024-09-21T07:44:59.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.820+0000 7f3b21800640 1 -- 172.21.15.39:0/2045871622 shutdown_connections 2024-09-21T07:44:59.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.820+0000 7f3b21800640 1 --2- 172.21.15.39:0/2045871622 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b000053f0 0x7f3b0000ba80 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:59.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.820+0000 7f3b21800640 1 -- 172.21.15.39:0/2045871622 >> 172.21.15.39:0/2045871622 conn(0x7f3b1c06cfc0 msgr2=0x7f3b1c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:44:59.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.820+0000 7f3b21800640 1 -- 172.21.15.39:0/2045871622 shutdown_connections 2024-09-21T07:44:59.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.820+0000 7f3b21800640 1 -- 172.21.15.39:0/2045871622 wait complete. 2024-09-21T07:44:59.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.822+0000 7f3b21800640 1 Processor -- start 2024-09-21T07:44:59.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.822+0000 7f3b21800640 1 -- start start 2024-09-21T07:44:59.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.824+0000 7f3b21800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b1c156cf0 0x7f3b1c0b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:59.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.824+0000 7f3b21800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b1c0b4c20 0x7f3b1c14cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:59.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.824+0000 7f3b21800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b1c14d450 0x7f3b1c0b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:59.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.824+0000 7f3b21800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1c15da50 con 0x7f3b1c156cf0 2024-09-21T07:44:59.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.824+0000 7f3b21800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1c15d8d0 con 0x7f3b1c0b4c20 2024-09-21T07:44:59.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.824+0000 7f3b21800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f3b1c15dbd0 con 0x7f3b1c14d450 2024-09-21T07:44:59.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.824+0000 7f3b1aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b1c156cf0 0x7f3b1c0b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:59.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.824+0000 7f3b1aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b1c156cf0 0x7f3b1c0b46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48014/0 (socket says 172.21.15.39:48014) 2024-09-21T07:44:59.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.824+0000 7f3b1aa00640 1 -- 172.21.15.39:0/3339191071 learned_addr learned my addr 172.21.15.39:0/3339191071 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:44:59.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.824+0000 7f3b12000640 1 --2- 172.21.15.39:0/3339191071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b1c0b4c20 0x7f3b1c14cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:59.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.824+0000 7f3b1b400640 1 --2- 172.21.15.39:0/3339191071 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b1c14d450 0x7f3b1c0b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:59.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.825+0000 7f3b1aa00640 1 -- 172.21.15.39:0/3339191071 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b1c14d450 msgr2=0x7f3b1c0b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:59.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.825+0000 7f3b1aa00640 1 --2- 172.21.15.39:0/3339191071 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b1c14d450 0x7f3b1c0b34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:59.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.825+0000 7f3b1aa00640 1 -- 172.21.15.39:0/3339191071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b1c0b4c20 msgr2=0x7f3b1c14cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:44:59.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.825+0000 7f3b1aa00640 1 --2- 172.21.15.39:0/3339191071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b1c0b4c20 0x7f3b1c14cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:44:59.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.825+0000 7f3b1aa00640 1 -- 172.21.15.39:0/3339191071 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b1c14d860 con 0x7f3b1c156cf0 2024-09-21T07:44:59.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.825+0000 7f3b1aa00640 1 --2- 172.21.15.39:0/3339191071 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b1c156cf0 0x7f3b1c0b46e0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c0066b0 tx=0x7f3b0c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:59.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.825+0000 7f3b13400640 1 -- 172.21.15.39:0/3339191071 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0c015070 con 0x7f3b1c156cf0 2024-09-21T07:44:59.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.825+0000 7f3b13400640 1 -- 172.21.15.39:0/3339191071 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b0c011840 con 0x7f3b1c156cf0 2024-09-21T07:44:59.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.825+0000 7f3b13400640 1 -- 172.21.15.39:0/3339191071 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0c011c90 con 0x7f3b1c156cf0 2024-09-21T07:44:59.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.825+0000 7f3b21800640 1 -- 172.21.15.39:0/3339191071 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b1c153fb0 con 0x7f3b1c156cf0 2024-09-21T07:44:59.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.825+0000 7f3b21800640 1 -- 172.21.15.39:0/3339191071 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b1c15b9c0 con 0x7f3b1c156cf0 2024-09-21T07:44:59.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.826+0000 7f3b21800640 1 -- 172.21.15.39:0/3339191071 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b1c0d0840 con 0x7f3b1c156cf0 2024-09-21T07:44:59.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.828+0000 7f3b13400640 1 -- 172.21.15.39:0/3339191071 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3b0c018070 con 0x7f3b1c156cf0 2024-09-21T07:44:59.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.829+0000 7f3b13400640 1 --2- 172.21.15.39:0/3339191071 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3af8078b90 0x7f3af807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:44:59.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.829+0000 7f3b13400640 1 -- 172.21.15.39:0/3339191071 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 6800+0+0 (secure 0 0 0) 0x7f3b0c09bce0 con 0x7f3b1c156cf0 2024-09-21T07:44:59.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.829+0000 7f3b12000640 1 --2- 172.21.15.39:0/3339191071 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3af8078b90 0x7f3af807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:44:59.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.830+0000 7f3b12000640 1 --2- 172.21.15.39:0/3339191071 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3af8078b90 0x7f3af807b0a0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c15e4d0 tx=0x7f3b04006b70 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:44:59.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:44:59.833+0000 7f3b13400640 1 -- 172.21.15.39:0/3339191071 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3b1c14d860 con 0x7f3b1c156cf0 2024-09-21T07:45:00.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.025+0000 7f3b21800640 1 -- 172.21.15.39:0/3339191071 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3b1c0b54f0 con 0x7f3af8078b90 2024-09-21T07:45:00.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.036+0000 7f3b13400640 1 -- 172.21.15.39:0/3339191071 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+495241 (secure 0 0 0) 0x7f3b1c0b54f0 con 0x7f3af8078b90 2024-09-21T07:45:00.044 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:45:00.048 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:45:00.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.041+0000 7f3b21800640 1 -- 172.21.15.39:0/3339191071 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3af8078b90 msgr2=0x7f3af807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:00.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.041+0000 7f3b21800640 1 --2- 172.21.15.39:0/3339191071 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3af8078b90 0x7f3af807b0a0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c15e4d0 tx=0x7f3b04006b70 comp rx=0 tx=0).stop 2024-09-21T07:45:00.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.042+0000 7f3b21800640 1 -- 172.21.15.39:0/3339191071 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b1c156cf0 msgr2=0x7f3b1c0b46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:00.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.042+0000 7f3b21800640 1 --2- 172.21.15.39:0/3339191071 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b1c156cf0 0x7f3b1c0b46e0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c0066b0 tx=0x7f3b0c0039c0 comp rx=0 tx=0).stop 2024-09-21T07:45:00.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.042+0000 7f3b21800640 1 -- 172.21.15.39:0/3339191071 shutdown_connections 2024-09-21T07:45:00.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.042+0000 7f3b21800640 1 --2- 172.21.15.39:0/3339191071 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3af8078b90 0x7f3af807b0a0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:00.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.042+0000 7f3b21800640 1 --2- 172.21.15.39:0/3339191071 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b1c14d450 0x7f3b1c0b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:00.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.042+0000 7f3b21800640 1 --2- 172.21.15.39:0/3339191071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b1c0b4c20 0x7f3b1c14cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:00.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.042+0000 7f3b21800640 1 --2- 172.21.15.39:0/3339191071 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b1c156cf0 0x7f3b1c0b46e0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:00.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.042+0000 7f3b21800640 1 -- 172.21.15.39:0/3339191071 >> 172.21.15.39:0/3339191071 conn(0x7f3b1c06cfc0 msgr2=0x7f3b1c1532f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:00.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.042+0000 7f3b21800640 1 -- 172.21.15.39:0/3339191071 shutdown_connections 2024-09-21T07:45:00.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.042+0000 7f3b21800640 1 -- 172.21.15.39:0/3339191071 wait complete. 2024-09-21T07:45:00.070 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":488,"stamp":"2024-09-21T07:44:59.773886+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":356,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":263776,"kb_used_data":14220,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754710944,"statfs":{"total":773094113280,"available":772824006656,"internally_reserved":0,"allocated":14561280,"data_stored":8055649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001933"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.578662+0000","last_change":"2024-09-21T07:44:55.577330+0000","last_active":"2024-09-21T07:44:56.578662+0000","last_peered":"2024-09-21T07:44:56.578662+0000","last_clean":"2024-09-21T07:44:56.578662+0000","last_became_active":"2024-09-21T07:44:55.576951+0000","last_became_peered":"2024-09-21T07:44:55.576951+0000","last_unstale":"2024-09-21T07:44:56.578662+0000","last_undegraded":"2024-09-21T07:44:56.578662+0000","last_fullsized":"2024-09-21T07:44:56.578662+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:48.641042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":10,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.599252+0000","last_change":"2024-09-21T07:44:55.597295+0000","last_active":"2024-09-21T07:44:55.599252+0000","last_peered":"2024-09-21T07:44:55.599252+0000","last_clean":"2024-09-21T07:44:55.599252+0000","last_became_active":"2024-09-21T07:44:55.596987+0000","last_became_peered":"2024-09-21T07:44:55.596987+0000","last_unstale":"2024-09-21T07:44:55.599252+0000","last_undegraded":"2024-09-21T07:44:55.599252+0000","last_fullsized":"2024-09-21T07:44:55.599252+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:03.040762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":10,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.596730+0000","last_change":"2024-09-21T07:44:55.583357+0000","last_active":"2024-09-21T07:44:55.596730+0000","last_peered":"2024-09-21T07:44:55.596730+0000","last_clean":"2024-09-21T07:44:55.596730+0000","last_became_active":"2024-09-21T07:44:55.583061+0000","last_became_peered":"2024-09-21T07:44:55.583061+0000","last_unstale":"2024-09-21T07:44:55.596730+0000","last_undegraded":"2024-09-21T07:44:55.596730+0000","last_fullsized":"2024-09-21T07:44:55.596730+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:25:24.765059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410297+0000","last_change":"2024-09-21T07:44:55.589391+0000","last_active":"2024-09-21T07:44:57.410297+0000","last_peered":"2024-09-21T07:44:57.410297+0000","last_clean":"2024-09-21T07:44:57.410297+0000","last_became_active":"2024-09-21T07:44:55.586615+0000","last_became_peered":"2024-09-21T07:44:55.586615+0000","last_unstale":"2024-09-21T07:44:57.410297+0000","last_undegraded":"2024-09-21T07:44:57.410297+0000","last_fullsized":"2024-09-21T07:44:57.410297+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:57:30.533423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576766+0000","last_change":"2024-09-21T07:44:55.582437+0000","last_active":"2024-09-21T07:44:56.576766+0000","last_peered":"2024-09-21T07:44:56.576766+0000","last_clean":"2024-09-21T07:44:56.576766+0000","last_became_active":"2024-09-21T07:44:55.580896+0000","last_became_peered":"2024-09-21T07:44:55.580896+0000","last_unstale":"2024-09-21T07:44:56.576766+0000","last_undegraded":"2024-09-21T07:44:56.576766+0000","last_fullsized":"2024-09-21T07:44:56.576766+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:01.412902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410284+0000","last_change":"2024-09-21T07:44:55.588345+0000","last_active":"2024-09-21T07:44:57.410284+0000","last_peered":"2024-09-21T07:44:57.410284+0000","last_clean":"2024-09-21T07:44:57.410284+0000","last_became_active":"2024-09-21T07:44:55.588019+0000","last_became_peered":"2024-09-21T07:44:55.588019+0000","last_unstale":"2024-09-21T07:44:57.410284+0000","last_undegraded":"2024-09-21T07:44:57.410284+0000","last_fullsized":"2024-09-21T07:44:57.410284+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:57.925520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409212+0000","last_change":"2024-09-21T07:44:55.589748+0000","last_active":"2024-09-21T07:44:57.409212+0000","last_peered":"2024-09-21T07:44:57.409212+0000","last_clean":"2024-09-21T07:44:57.409212+0000","last_became_active":"2024-09-21T07:44:55.586211+0000","last_became_peered":"2024-09-21T07:44:55.586211+0000","last_unstale":"2024-09-21T07:44:57.409212+0000","last_undegraded":"2024-09-21T07:44:57.409212+0000","last_fullsized":"2024-09-21T07:44:57.409212+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:42.462390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":10,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.598957+0000","last_change":"2024-09-21T07:44:55.575890+0000","last_active":"2024-09-21T07:44:55.598957+0000","last_peered":"2024-09-21T07:44:55.598957+0000","last_clean":"2024-09-21T07:44:55.598957+0000","last_became_active":"2024-09-21T07:44:55.575594+0000","last_became_peered":"2024-09-21T07:44:55.575594+0000","last_unstale":"2024-09-21T07:44:55.598957+0000","last_undegraded":"2024-09-21T07:44:55.598957+0000","last_fullsized":"2024-09-21T07:44:55.598957+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:09:15.363621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410541+0000","last_change":"2024-09-21T07:44:55.594003+0000","last_active":"2024-09-21T07:44:57.410541+0000","last_peered":"2024-09-21T07:44:57.410541+0000","last_clean":"2024-09-21T07:44:57.410541+0000","last_became_active":"2024-09-21T07:44:55.593315+0000","last_became_peered":"2024-09-21T07:44:55.593315+0000","last_unstale":"2024-09-21T07:44:57.410541+0000","last_undegraded":"2024-09-21T07:44:57.410541+0000","last_fullsized":"2024-09-21T07:44:57.410541+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:36:16.134960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.575833+0000","last_change":"2024-09-21T07:44:55.590261+0000","last_active":"2024-09-21T07:44:56.575833+0000","last_peered":"2024-09-21T07:44:56.575833+0000","last_clean":"2024-09-21T07:44:56.575833+0000","last_became_active":"2024-09-21T07:44:55.589848+0000","last_became_peered":"2024-09-21T07:44:55.589848+0000","last_unstale":"2024-09-21T07:44:56.575833+0000","last_undegraded":"2024-09-21T07:44:56.575833+0000","last_fullsized":"2024-09-21T07:44:56.575833+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:04:55.654398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.578247+0000","last_change":"2024-09-21T07:44:55.575847+0000","last_active":"2024-09-21T07:44:56.578247+0000","last_peered":"2024-09-21T07:44:56.578247+0000","last_clean":"2024-09-21T07:44:56.578247+0000","last_became_active":"2024-09-21T07:44:55.575524+0000","last_became_peered":"2024-09-21T07:44:55.575524+0000","last_unstale":"2024-09-21T07:44:56.578247+0000","last_undegraded":"2024-09-21T07:44:56.578247+0000","last_fullsized":"2024-09-21T07:44:56.578247+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:16:24.610724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410779+0000","last_change":"2024-09-21T07:44:55.576749+0000","last_active":"2024-09-21T07:44:57.410779+0000","last_peered":"2024-09-21T07:44:57.410779+0000","last_clean":"2024-09-21T07:44:57.410779+0000","last_became_active":"2024-09-21T07:44:55.576450+0000","last_became_peered":"2024-09-21T07:44:55.576450+0000","last_unstale":"2024-09-21T07:44:57.410779+0000","last_undegraded":"2024-09-21T07:44:57.410779+0000","last_fullsized":"2024-09-21T07:44:57.410779+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:17.212037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.574843+0000","last_change":"2024-09-21T07:44:55.586779+0000","last_active":"2024-09-21T07:44:56.574843+0000","last_peered":"2024-09-21T07:44:56.574843+0000","last_clean":"2024-09-21T07:44:56.574843+0000","last_became_active":"2024-09-21T07:44:55.586473+0000","last_became_peered":"2024-09-21T07:44:55.586473+0000","last_unstale":"2024-09-21T07:44:56.574843+0000","last_undegraded":"2024-09-21T07:44:56.574843+0000","last_fullsized":"2024-09-21T07:44:56.574843+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:23.423387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":10,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.597948+0000","last_change":"2024-09-21T07:44:55.587326+0000","last_active":"2024-09-21T07:44:55.597948+0000","last_peered":"2024-09-21T07:44:55.597948+0000","last_clean":"2024-09-21T07:44:55.597948+0000","last_became_active":"2024-09-21T07:44:55.582056+0000","last_became_peered":"2024-09-21T07:44:55.582056+0000","last_unstale":"2024-09-21T07:44:55.597948+0000","last_undegraded":"2024-09-21T07:44:55.597948+0000","last_fullsized":"2024-09-21T07:44:55.597948+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:53.727782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576256+0000","last_change":"2024-09-21T07:44:55.574716+0000","last_active":"2024-09-21T07:44:56.576256+0000","last_peered":"2024-09-21T07:44:56.576256+0000","last_clean":"2024-09-21T07:44:56.576256+0000","last_became_active":"2024-09-21T07:44:55.574412+0000","last_became_peered":"2024-09-21T07:44:55.574412+0000","last_unstale":"2024-09-21T07:44:56.576256+0000","last_undegraded":"2024-09-21T07:44:56.576256+0000","last_fullsized":"2024-09-21T07:44:56.576256+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:59.447663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.408754+0000","last_change":"2024-09-21T07:44:55.575384+0000","last_active":"2024-09-21T07:44:57.408754+0000","last_peered":"2024-09-21T07:44:57.408754+0000","last_clean":"2024-09-21T07:44:57.408754+0000","last_became_active":"2024-09-21T07:44:55.575069+0000","last_became_peered":"2024-09-21T07:44:55.575069+0000","last_unstale":"2024-09-21T07:44:57.408754+0000","last_undegraded":"2024-09-21T07:44:57.408754+0000","last_fullsized":"2024-09-21T07:44:57.408754+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:50.957046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.575891+0000","last_change":"2024-09-21T07:44:55.578638+0000","last_active":"2024-09-21T07:44:56.575891+0000","last_peered":"2024-09-21T07:44:56.575891+0000","last_clean":"2024-09-21T07:44:56.575891+0000","last_became_active":"2024-09-21T07:44:55.578093+0000","last_became_peered":"2024-09-21T07:44:55.578093+0000","last_unstale":"2024-09-21T07:44:56.575891+0000","last_undegraded":"2024-09-21T07:44:56.575891+0000","last_fullsized":"2024-09-21T07:44:56.575891+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:45.442291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":10,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.598751+0000","last_change":"2024-09-21T07:44:55.596708+0000","last_active":"2024-09-21T07:44:55.598751+0000","last_peered":"2024-09-21T07:44:55.598751+0000","last_clean":"2024-09-21T07:44:55.598751+0000","last_became_active":"2024-09-21T07:44:55.596318+0000","last_became_peered":"2024-09-21T07:44:55.596318+0000","last_unstale":"2024-09-21T07:44:55.598751+0000","last_undegraded":"2024-09-21T07:44:55.598751+0000","last_fullsized":"2024-09-21T07:44:55.598751+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:40.500276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.408944+0000","last_change":"2024-09-21T07:44:55.587990+0000","last_active":"2024-09-21T07:44:57.408944+0000","last_peered":"2024-09-21T07:44:57.408944+0000","last_clean":"2024-09-21T07:44:57.408944+0000","last_became_active":"2024-09-21T07:44:55.587629+0000","last_became_peered":"2024-09-21T07:44:55.587629+0000","last_unstale":"2024-09-21T07:44:57.408944+0000","last_undegraded":"2024-09-21T07:44:57.408944+0000","last_fullsized":"2024-09-21T07:44:57.408944+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:36:16.134960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.925524+0000","last_change":"2024-09-21T07:44:55.582492+0000","last_active":"2024-09-21T07:44:56.925524+0000","last_peered":"2024-09-21T07:44:56.925524+0000","last_clean":"2024-09-21T07:44:56.925524+0000","last_became_active":"2024-09-21T07:44:55.580803+0000","last_became_peered":"2024-09-21T07:44:55.580803+0000","last_unstale":"2024-09-21T07:44:56.925524+0000","last_undegraded":"2024-09-21T07:44:56.925524+0000","last_fullsized":"2024-09-21T07:44:56.925524+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:28:20.883491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410061+0000","last_change":"2024-09-21T07:44:55.589685+0000","last_active":"2024-09-21T07:44:57.410061+0000","last_peered":"2024-09-21T07:44:57.410061+0000","last_clean":"2024-09-21T07:44:57.410061+0000","last_became_active":"2024-09-21T07:44:55.589283+0000","last_became_peered":"2024-09-21T07:44:55.589283+0000","last_unstale":"2024-09-21T07:44:57.410061+0000","last_undegraded":"2024-09-21T07:44:57.410061+0000","last_fullsized":"2024-09-21T07:44:57.410061+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:45.442291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.577585+0000","last_change":"2024-09-21T07:44:55.588693+0000","last_active":"2024-09-21T07:44:56.577585+0000","last_peered":"2024-09-21T07:44:56.577585+0000","last_clean":"2024-09-21T07:44:56.577585+0000","last_became_active":"2024-09-21T07:44:55.588331+0000","last_became_peered":"2024-09-21T07:44:55.588331+0000","last_unstale":"2024-09-21T07:44:56.577585+0000","last_undegraded":"2024-09-21T07:44:56.577585+0000","last_fullsized":"2024-09-21T07:44:56.577585+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:03.374280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.408624+0000","last_change":"2024-09-21T07:44:55.589391+0000","last_active":"2024-09-21T07:44:57.408624+0000","last_peered":"2024-09-21T07:44:57.408624+0000","last_clean":"2024-09-21T07:44:57.408624+0000","last_became_active":"2024-09-21T07:44:55.586797+0000","last_became_peered":"2024-09-21T07:44:55.586797+0000","last_unstale":"2024-09-21T07:44:57.408624+0000","last_undegraded":"2024-09-21T07:44:57.408624+0000","last_fullsized":"2024-09-21T07:44:57.408624+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:27.029715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.575550+0000","last_change":"2024-09-21T07:44:55.569104+0000","last_active":"2024-09-21T07:44:56.575550+0000","last_peered":"2024-09-21T07:44:56.575550+0000","last_clean":"2024-09-21T07:44:56.575550+0000","last_became_active":"2024-09-21T07:44:55.568769+0000","last_became_peered":"2024-09-21T07:44:55.568769+0000","last_unstale":"2024-09-21T07:44:56.575550+0000","last_undegraded":"2024-09-21T07:44:56.575550+0000","last_fullsized":"2024-09-21T07:44:56.575550+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:05.902947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.577096+0000","last_change":"2024-09-21T07:44:55.569906+0000","last_active":"2024-09-21T07:44:56.577096+0000","last_peered":"2024-09-21T07:44:56.577096+0000","last_clean":"2024-09-21T07:44:56.577096+0000","last_became_active":"2024-09-21T07:44:55.569584+0000","last_became_peered":"2024-09-21T07:44:55.569584+0000","last_unstale":"2024-09-21T07:44:56.577096+0000","last_undegraded":"2024-09-21T07:44:56.577096+0000","last_fullsized":"2024-09-21T07:44:56.577096+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:39:07.663576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.925994+0000","last_change":"2024-09-21T07:44:55.577844+0000","last_active":"2024-09-21T07:44:56.925994+0000","last_peered":"2024-09-21T07:44:56.925994+0000","last_clean":"2024-09-21T07:44:56.925994+0000","last_became_active":"2024-09-21T07:44:55.577546+0000","last_became_peered":"2024-09-21T07:44:55.577546+0000","last_unstale":"2024-09-21T07:44:56.925994+0000","last_undegraded":"2024-09-21T07:44:56.925994+0000","last_fullsized":"2024-09-21T07:44:56.925994+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:51:45.788835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409555+0000","last_change":"2024-09-21T07:44:55.584249+0000","last_active":"2024-09-21T07:44:57.409555+0000","last_peered":"2024-09-21T07:44:57.409555+0000","last_clean":"2024-09-21T07:44:57.409555+0000","last_became_active":"2024-09-21T07:44:55.583386+0000","last_became_peered":"2024-09-21T07:44:55.583386+0000","last_unstale":"2024-09-21T07:44:57.409555+0000","last_undegraded":"2024-09-21T07:44:57.409555+0000","last_fullsized":"2024-09-21T07:44:57.409555+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:41.343224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410977+0000","last_change":"2024-09-21T07:44:55.580364+0000","last_active":"2024-09-21T07:44:57.410977+0000","last_peered":"2024-09-21T07:44:57.410977+0000","last_clean":"2024-09-21T07:44:57.410977+0000","last_became_active":"2024-09-21T07:44:55.579952+0000","last_became_peered":"2024-09-21T07:44:55.579952+0000","last_unstale":"2024-09-21T07:44:57.410977+0000","last_undegraded":"2024-09-21T07:44:57.410977+0000","last_fullsized":"2024-09-21T07:44:57.410977+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:38.518021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.926207+0000","last_change":"2024-09-21T07:44:55.583118+0000","last_active":"2024-09-21T07:44:56.926207+0000","last_peered":"2024-09-21T07:44:56.926207+0000","last_clean":"2024-09-21T07:44:56.926207+0000","last_became_active":"2024-09-21T07:44:55.582455+0000","last_became_peered":"2024-09-21T07:44:55.582455+0000","last_unstale":"2024-09-21T07:44:56.926207+0000","last_undegraded":"2024-09-21T07:44:56.926207+0000","last_fullsized":"2024-09-21T07:44:56.926207+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:35.307701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":10,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.597773+0000","last_change":"2024-09-21T07:44:55.585979+0000","last_active":"2024-09-21T07:44:55.597773+0000","last_peered":"2024-09-21T07:44:55.597773+0000","last_clean":"2024-09-21T07:44:55.597773+0000","last_became_active":"2024-09-21T07:44:55.583787+0000","last_became_peered":"2024-09-21T07:44:55.583787+0000","last_unstale":"2024-09-21T07:44:55.597773+0000","last_undegraded":"2024-09-21T07:44:55.597773+0000","last_fullsized":"2024-09-21T07:44:55.597773+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:20:07.013117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.575754+0000","last_change":"2024-09-21T07:44:55.576065+0000","last_active":"2024-09-21T07:44:56.575754+0000","last_peered":"2024-09-21T07:44:56.575754+0000","last_clean":"2024-09-21T07:44:56.575754+0000","last_became_active":"2024-09-21T07:44:55.575720+0000","last_became_peered":"2024-09-21T07:44:55.575720+0000","last_unstale":"2024-09-21T07:44:56.575754+0000","last_undegraded":"2024-09-21T07:44:56.575754+0000","last_fullsized":"2024-09-21T07:44:56.575754+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:54.908229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.575818+0000","last_change":"2024-09-21T07:44:55.560991+0000","last_active":"2024-09-21T07:44:56.575818+0000","last_peered":"2024-09-21T07:44:56.575818+0000","last_clean":"2024-09-21T07:44:56.575818+0000","last_became_active":"2024-09-21T07:44:55.560593+0000","last_became_peered":"2024-09-21T07:44:55.560593+0000","last_unstale":"2024-09-21T07:44:56.575818+0000","last_undegraded":"2024-09-21T07:44:56.575818+0000","last_fullsized":"2024-09-21T07:44:56.575818+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:36:51.196810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.408690+0000","last_change":"2024-09-21T07:44:55.588440+0000","last_active":"2024-09-21T07:44:57.408690+0000","last_peered":"2024-09-21T07:44:57.408690+0000","last_clean":"2024-09-21T07:44:57.408690+0000","last_became_active":"2024-09-21T07:44:55.586797+0000","last_became_peered":"2024-09-21T07:44:55.586797+0000","last_unstale":"2024-09-21T07:44:57.408690+0000","last_undegraded":"2024-09-21T07:44:57.408690+0000","last_fullsized":"2024-09-21T07:44:57.408690+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:41.936023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"133'8","reported_seq":21,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.601626+0000","last_change":"2024-09-21T07:44:54.579517+0000","last_active":"2024-09-21T07:44:55.601626+0000","last_peered":"2024-09-21T07:44:55.601626+0000","last_clean":"2024-09-21T07:44:55.601626+0000","last_became_active":"2024-09-21T07:44:54.578700+0000","last_became_peered":"2024-09-21T07:44:54.578700+0000","last_unstale":"2024-09-21T07:44:55.601626+0000","last_undegraded":"2024-09-21T07:44:55.601626+0000","last_fullsized":"2024-09-21T07:44:55.601626+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:36.690522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":147,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410130+0000","last_change":"2024-09-21T07:35:27.879605+0000","last_active":"2024-09-21T07:44:57.410130+0000","last_peered":"2024-09-21T07:44:57.410130+0000","last_clean":"2024-09-21T07:44:57.410130+0000","last_became_active":"2024-09-21T07:35:27.879274+0000","last_became_peered":"2024-09-21T07:35:27.879274+0000","last_unstale":"2024-09-21T07:44:57.410130+0000","last_undegraded":"2024-09-21T07:44:57.410130+0000","last_fullsized":"2024-09-21T07:44:57.410130+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:24.241636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":151,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.565496+0000","last_change":"2024-09-21T07:35:29.491336+0000","last_active":"2024-09-21T07:44:55.565496+0000","last_peered":"2024-09-21T07:44:55.565496+0000","last_clean":"2024-09-21T07:44:55.565496+0000","last_became_active":"2024-09-21T07:35:26.866484+0000","last_became_peered":"2024-09-21T07:35:26.866484+0000","last_unstale":"2024-09-21T07:44:55.565496+0000","last_undegraded":"2024-09-21T07:44:55.565496+0000","last_fullsized":"2024-09-21T07:44:55.565496+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:10.449665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076010100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":332,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.408176+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:44:57.408176+0000","last_peered":"2024-09-21T07:44:57.408176+0000","last_clean":"2024-09-21T07:44:57.408176+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:44:57.408176+0000","last_undegraded":"2024-09-21T07:44:57.408176+0000","last_fullsized":"2024-09-21T07:44:57.408176+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":209,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.577172+0000","last_change":"2024-09-21T07:34:24.539248+0000","last_active":"2024-09-21T07:44:56.577172+0000","last_peered":"2024-09-21T07:44:56.577172+0000","last_clean":"2024-09-21T07:44:56.577172+0000","last_became_active":"2024-09-21T07:34:23.273465+0000","last_became_peered":"2024-09-21T07:34:23.273465+0000","last_unstale":"2024-09-21T07:44:56.577172+0000","last_undegraded":"2024-09-21T07:44:56.577172+0000","last_fullsized":"2024-09-21T07:44:56.577172+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:55.517452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060627700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.597889+0000","last_change":"2024-09-21T07:44:55.586687+0000","last_active":"2024-09-21T07:44:55.597889+0000","last_peered":"2024-09-21T07:44:55.597889+0000","last_clean":"2024-09-21T07:44:55.597889+0000","last_became_active":"2024-09-21T07:44:55.586385+0000","last_became_peered":"2024-09-21T07:44:55.586385+0000","last_unstale":"2024-09-21T07:44:55.597889+0000","last_undegraded":"2024-09-21T07:44:55.597889+0000","last_fullsized":"2024-09-21T07:44:55.597889+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:30:21.718346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"133'3","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.577689+0000","last_change":"2024-09-21T07:44:54.570673+0000","last_active":"2024-09-21T07:44:56.577689+0000","last_peered":"2024-09-21T07:44:56.577689+0000","last_clean":"2024-09-21T07:44:56.577689+0000","last_became_active":"2024-09-21T07:44:54.570315+0000","last_became_peered":"2024-09-21T07:44:54.570315+0000","last_unstale":"2024-09-21T07:44:56.577689+0000","last_undegraded":"2024-09-21T07:44:56.577689+0000","last_fullsized":"2024-09-21T07:44:56.577689+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:04:26.458971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":144,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.565323+0000","last_change":"2024-09-21T07:35:27.864492+0000","last_active":"2024-09-21T07:44:55.565323+0000","last_peered":"2024-09-21T07:44:55.565323+0000","last_clean":"2024-09-21T07:44:55.565323+0000","last_became_active":"2024-09-21T07:35:27.864176+0000","last_became_peered":"2024-09-21T07:35:27.864176+0000","last_unstale":"2024-09-21T07:44:55.565323+0000","last_undegraded":"2024-09-21T07:44:55.565323+0000","last_fullsized":"2024-09-21T07:44:55.565323+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:05.153467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":154,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410622+0000","last_change":"2024-09-21T07:35:29.493678+0000","last_active":"2024-09-21T07:44:57.410622+0000","last_peered":"2024-09-21T07:44:57.410622+0000","last_clean":"2024-09-21T07:44:57.410622+0000","last_became_active":"2024-09-21T07:35:26.870749+0000","last_became_peered":"2024-09-21T07:35:26.870749+0000","last_unstale":"2024-09-21T07:44:57.410622+0000","last_undegraded":"2024-09-21T07:44:57.410622+0000","last_fullsized":"2024-09-21T07:44:57.410622+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:55.367877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010159570000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":202,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.577662+0000","last_change":"2024-09-21T07:34:24.597274+0000","last_active":"2024-09-21T07:44:56.577662+0000","last_peered":"2024-09-21T07:44:56.577662+0000","last_clean":"2024-09-21T07:44:56.577662+0000","last_became_active":"2024-09-21T07:34:23.279560+0000","last_became_peered":"2024-09-21T07:34:23.279560+0000","last_unstale":"2024-09-21T07:44:56.577662+0000","last_undegraded":"2024-09-21T07:44:56.577662+0000","last_fullsized":"2024-09-21T07:44:56.577662+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:15.845265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059699799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576442+0000","last_change":"2024-09-21T07:44:55.574395+0000","last_active":"2024-09-21T07:44:56.576442+0000","last_peered":"2024-09-21T07:44:56.576442+0000","last_clean":"2024-09-21T07:44:56.576442+0000","last_became_active":"2024-09-21T07:44:55.574031+0000","last_became_peered":"2024-09-21T07:44:55.574031+0000","last_unstale":"2024-09-21T07:44:56.576442+0000","last_undegraded":"2024-09-21T07:44:56.576442+0000","last_fullsized":"2024-09-21T07:44:56.576442+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:53.313866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"133'6","reported_seq":19,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.600582+0000","last_change":"2024-09-21T07:44:54.575574+0000","last_active":"2024-09-21T07:44:55.600582+0000","last_peered":"2024-09-21T07:44:55.600582+0000","last_clean":"2024-09-21T07:44:55.600582+0000","last_became_active":"2024-09-21T07:44:54.575263+0000","last_became_peered":"2024-09-21T07:44:54.575263+0000","last_unstale":"2024-09-21T07:44:55.600582+0000","last_undegraded":"2024-09-21T07:44:55.600582+0000","last_fullsized":"2024-09-21T07:44:55.600582+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:06.135058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":147,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.925261+0000","last_change":"2024-09-21T07:35:27.874387+0000","last_active":"2024-09-21T07:44:56.925261+0000","last_peered":"2024-09-21T07:44:56.925261+0000","last_clean":"2024-09-21T07:44:56.925261+0000","last_became_active":"2024-09-21T07:35:27.873277+0000","last_became_peered":"2024-09-21T07:35:27.873277+0000","last_unstale":"2024-09-21T07:44:56.925261+0000","last_undegraded":"2024-09-21T07:44:56.925261+0000","last_fullsized":"2024-09-21T07:44:56.925261+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:04.581463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":152,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.578248+0000","last_change":"2024-09-21T07:35:29.387094+0000","last_active":"2024-09-21T07:44:56.578248+0000","last_peered":"2024-09-21T07:44:56.578248+0000","last_clean":"2024-09-21T07:44:56.578248+0000","last_became_active":"2024-09-21T07:35:26.874757+0000","last_became_peered":"2024-09-21T07:35:26.874757+0000","last_unstale":"2024-09-21T07:44:56.578248+0000","last_undegraded":"2024-09-21T07:44:56.578248+0000","last_fullsized":"2024-09-21T07:44:56.578248+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:33.542750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061065700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":208,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409380+0000","last_change":"2024-09-21T07:34:24.259906+0000","last_active":"2024-09-21T07:44:57.409380+0000","last_peered":"2024-09-21T07:44:57.409380+0000","last_clean":"2024-09-21T07:44:57.409380+0000","last_became_active":"2024-09-21T07:34:23.635282+0000","last_became_peered":"2024-09-21T07:34:23.635282+0000","last_unstale":"2024-09-21T07:44:57.409380+0000","last_undegraded":"2024-09-21T07:44:57.409380+0000","last_fullsized":"2024-09-21T07:44:57.409380+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:00.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001527664,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409128+0000","last_change":"2024-09-21T07:44:55.583909+0000","last_active":"2024-09-21T07:44:57.409128+0000","last_peered":"2024-09-21T07:44:57.409128+0000","last_clean":"2024-09-21T07:44:57.409128+0000","last_became_active":"2024-09-21T07:44:55.583449+0000","last_became_peered":"2024-09-21T07:44:55.583449+0000","last_unstale":"2024-09-21T07:44:57.409128+0000","last_undegraded":"2024-09-21T07:44:57.409128+0000","last_fullsized":"2024-09-21T07:44:57.409128+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:09.715628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409468+0000","last_change":"2024-09-21T07:44:54.574714+0000","last_active":"2024-09-21T07:44:57.409468+0000","last_peered":"2024-09-21T07:44:57.409468+0000","last_clean":"2024-09-21T07:44:57.409468+0000","last_became_active":"2024-09-21T07:44:54.574362+0000","last_became_peered":"2024-09-21T07:44:54.574362+0000","last_unstale":"2024-09-21T07:44:57.409468+0000","last_undegraded":"2024-09-21T07:44:57.409468+0000","last_fullsized":"2024-09-21T07:44:57.409468+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:22.420605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":147,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.925407+0000","last_change":"2024-09-21T07:35:27.873645+0000","last_active":"2024-09-21T07:44:56.925407+0000","last_peered":"2024-09-21T07:44:56.925407+0000","last_clean":"2024-09-21T07:44:56.925407+0000","last_became_active":"2024-09-21T07:35:27.873245+0000","last_became_peered":"2024-09-21T07:35:27.873245+0000","last_unstale":"2024-09-21T07:44:56.925407+0000","last_undegraded":"2024-09-21T07:44:56.925407+0000","last_fullsized":"2024-09-21T07:44:56.925407+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:55.090953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":154,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.925678+0000","last_change":"2024-09-21T07:35:28.861744+0000","last_active":"2024-09-21T07:44:56.925678+0000","last_peered":"2024-09-21T07:44:56.925678+0000","last_clean":"2024-09-21T07:44:56.925678+0000","last_became_active":"2024-09-21T07:35:26.876462+0000","last_became_peered":"2024-09-21T07:35:26.876462+0000","last_unstale":"2024-09-21T07:44:56.925678+0000","last_undegraded":"2024-09-21T07:44:56.925678+0000","last_fullsized":"2024-09-21T07:44:56.925678+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:48.632568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012020920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":205,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409403+0000","last_change":"2024-09-21T07:34:25.085454+0000","last_active":"2024-09-21T07:44:57.409403+0000","last_peered":"2024-09-21T07:44:57.409403+0000","last_clean":"2024-09-21T07:44:57.409403+0000","last_became_active":"2024-09-21T07:34:23.279205+0000","last_became_peered":"2024-09-21T07:34:23.279205+0000","last_unstale":"2024-09-21T07:44:57.409403+0000","last_undegraded":"2024-09-21T07:44:57.409403+0000","last_fullsized":"2024-09-21T07:44:57.409403+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:54.231666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095320599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.577540+0000","last_change":"2024-09-21T07:44:55.587979+0000","last_active":"2024-09-21T07:44:56.577540+0000","last_peered":"2024-09-21T07:44:56.577540+0000","last_clean":"2024-09-21T07:44:56.577540+0000","last_became_active":"2024-09-21T07:44:55.587653+0000","last_became_peered":"2024-09-21T07:44:55.587653+0000","last_unstale":"2024-09-21T07:44:56.577540+0000","last_undegraded":"2024-09-21T07:44:56.577540+0000","last_fullsized":"2024-09-21T07:44:56.577540+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:42.142665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576220+0000","last_change":"2024-09-21T07:44:54.564396+0000","last_active":"2024-09-21T07:44:56.576220+0000","last_peered":"2024-09-21T07:44:56.576220+0000","last_clean":"2024-09-21T07:44:56.576220+0000","last_became_active":"2024-09-21T07:44:54.564012+0000","last_became_peered":"2024-09-21T07:44:54.564012+0000","last_unstale":"2024-09-21T07:44:56.576220+0000","last_undegraded":"2024-09-21T07:44:56.576220+0000","last_fullsized":"2024-09-21T07:44:56.576220+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:43:27.809502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":144,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.567215+0000","last_change":"2024-09-21T07:35:27.861373+0000","last_active":"2024-09-21T07:44:55.567215+0000","last_peered":"2024-09-21T07:44:55.567215+0000","last_clean":"2024-09-21T07:44:55.567215+0000","last_became_active":"2024-09-21T07:35:27.861016+0000","last_became_peered":"2024-09-21T07:35:27.861016+0000","last_unstale":"2024-09-21T07:44:55.567215+0000","last_undegraded":"2024-09-21T07:44:55.567215+0000","last_fullsized":"2024-09-21T07:44:55.567215+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:56.043763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":155,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.575383+0000","last_change":"2024-09-21T07:35:29.053677+0000","last_active":"2024-09-21T07:44:56.575383+0000","last_peered":"2024-09-21T07:44:56.575383+0000","last_clean":"2024-09-21T07:44:56.575383+0000","last_became_active":"2024-09-21T07:35:26.875496+0000","last_became_peered":"2024-09-21T07:35:26.875496+0000","last_unstale":"2024-09-21T07:44:56.575383+0000","last_undegraded":"2024-09-21T07:44:56.575383+0000","last_fullsized":"2024-09-21T07:44:56.575383+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:04.850784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065859000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":207,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.574611+0000","last_change":"2024-09-21T07:34:25.028521+0000","last_active":"2024-09-21T07:44:56.574611+0000","last_peered":"2024-09-21T07:44:56.574611+0000","last_clean":"2024-09-21T07:44:56.574611+0000","last_became_active":"2024-09-21T07:34:23.635467+0000","last_became_peered":"2024-09-21T07:34:23.635467+0000","last_unstale":"2024-09-21T07:44:56.574611+0000","last_undegraded":"2024-09-21T07:44:56.574611+0000","last_fullsized":"2024-09-21T07:44:56.574611+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:33.729123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000660558,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.925423+0000","last_change":"2024-09-21T07:44:55.573727+0000","last_active":"2024-09-21T07:44:56.925423+0000","last_peered":"2024-09-21T07:44:56.925423+0000","last_clean":"2024-09-21T07:44:56.925423+0000","last_became_active":"2024-09-21T07:44:55.573386+0000","last_became_peered":"2024-09-21T07:44:55.573386+0000","last_unstale":"2024-09-21T07:44:56.925423+0000","last_undegraded":"2024-09-21T07:44:56.925423+0000","last_fullsized":"2024-09-21T07:44:56.925423+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:15.437515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.573509+0000","last_change":"2024-09-21T07:44:54.578178+0000","last_active":"2024-09-21T07:44:55.573509+0000","last_peered":"2024-09-21T07:44:55.573509+0000","last_clean":"2024-09-21T07:44:55.573509+0000","last_became_active":"2024-09-21T07:44:54.577847+0000","last_became_peered":"2024-09-21T07:44:54.577847+0000","last_unstale":"2024-09-21T07:44:55.573509+0000","last_undegraded":"2024-09-21T07:44:55.573509+0000","last_fullsized":"2024-09-21T07:44:55.573509+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:41:12.848885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":147,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.411193+0000","last_change":"2024-09-21T07:35:27.871201+0000","last_active":"2024-09-21T07:44:57.411193+0000","last_peered":"2024-09-21T07:44:57.411193+0000","last_clean":"2024-09-21T07:44:57.411193+0000","last_became_active":"2024-09-21T07:35:27.870816+0000","last_became_peered":"2024-09-21T07:35:27.870816+0000","last_unstale":"2024-09-21T07:44:57.411193+0000","last_undegraded":"2024-09-21T07:44:57.411193+0000","last_fullsized":"2024-09-21T07:44:57.411193+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:04.787450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":151,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.567513+0000","last_change":"2024-09-21T07:35:29.491874+0000","last_active":"2024-09-21T07:44:55.567513+0000","last_peered":"2024-09-21T07:44:55.567513+0000","last_clean":"2024-09-21T07:44:55.567513+0000","last_became_active":"2024-09-21T07:35:26.866381+0000","last_became_peered":"2024-09-21T07:35:26.866381+0000","last_unstale":"2024-09-21T07:44:55.567513+0000","last_undegraded":"2024-09-21T07:44:55.567513+0000","last_fullsized":"2024-09-21T07:44:55.567513+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:36.404647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073235399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":205,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.574888+0000","last_change":"2024-09-21T07:34:24.260046+0000","last_active":"2024-09-21T07:44:55.574888+0000","last_peered":"2024-09-21T07:44:55.574888+0000","last_clean":"2024-09-21T07:44:55.574888+0000","last_became_active":"2024-09-21T07:34:23.273431+0000","last_became_peered":"2024-09-21T07:34:23.273431+0000","last_unstale":"2024-09-21T07:44:55.574888+0000","last_undegraded":"2024-09-21T07:44:55.574888+0000","last_fullsized":"2024-09-21T07:44:55.574888+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:18.876057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019803310000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.408725+0000","last_change":"2024-09-21T07:44:55.587563+0000","last_active":"2024-09-21T07:44:57.408725+0000","last_peered":"2024-09-21T07:44:57.408725+0000","last_clean":"2024-09-21T07:44:57.408725+0000","last_became_active":"2024-09-21T07:44:55.587101+0000","last_became_peered":"2024-09-21T07:44:55.587101+0000","last_unstale":"2024-09-21T07:44:57.408725+0000","last_undegraded":"2024-09-21T07:44:57.408725+0000","last_fullsized":"2024-09-21T07:44:57.408725+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:16.375239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"133'2","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.598066+0000","last_change":"2024-09-21T07:44:54.579114+0000","last_active":"2024-09-21T07:44:55.598066+0000","last_peered":"2024-09-21T07:44:55.598066+0000","last_clean":"2024-09-21T07:44:55.598066+0000","last_became_active":"2024-09-21T07:44:54.578761+0000","last_became_peered":"2024-09-21T07:44:54.578761+0000","last_unstale":"2024-09-21T07:44:55.598066+0000","last_undegraded":"2024-09-21T07:44:55.598066+0000","last_fullsized":"2024-09-21T07:44:55.598066+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:22.050230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":147,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410342+0000","last_change":"2024-09-21T07:35:27.871736+0000","last_active":"2024-09-21T07:44:57.410342+0000","last_peered":"2024-09-21T07:44:57.410342+0000","last_clean":"2024-09-21T07:44:57.410342+0000","last_became_active":"2024-09-21T07:35:27.871232+0000","last_became_peered":"2024-09-21T07:35:27.871232+0000","last_unstale":"2024-09-21T07:44:57.410342+0000","last_undegraded":"2024-09-21T07:44:57.410342+0000","last_fullsized":"2024-09-21T07:44:57.410342+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:04.419301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":151,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.573798+0000","last_change":"2024-09-21T07:35:29.491246+0000","last_active":"2024-09-21T07:44:55.573798+0000","last_peered":"2024-09-21T07:44:55.573798+0000","last_clean":"2024-09-21T07:44:55.573798+0000","last_became_active":"2024-09-21T07:35:26.866783+0000","last_became_peered":"2024-09-21T07:35:26.866783+0000","last_unstale":"2024-09-21T07:44:55.573798+0000","last_undegraded":"2024-09-21T07:44:55.573798+0000","last_fullsized":"2024-09-21T07:44:55.573798+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:28.881914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098474099999999992,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":207,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.575464+0000","last_change":"2024-09-21T07:34:25.028640+0000","last_active":"2024-09-21T07:44:56.575464+0000","last_peered":"2024-09-21T07:44:56.575464+0000","last_clean":"2024-09-21T07:44:56.575464+0000","last_became_active":"2024-09-21T07:34:23.260950+0000","last_became_peered":"2024-09-21T07:34:23.260950+0000","last_unstale":"2024-09-21T07:44:56.575464+0000","last_undegraded":"2024-09-21T07:44:56.575464+0000","last_fullsized":"2024-09-21T07:44:56.575464+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:23:42.121398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055135099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.574774+0000","last_change":"2024-09-21T07:44:55.583610+0000","last_active":"2024-09-21T07:44:56.574774+0000","last_peered":"2024-09-21T07:44:56.574774+0000","last_clean":"2024-09-21T07:44:56.574774+0000","last_became_active":"2024-09-21T07:44:55.583311+0000","last_became_peered":"2024-09-21T07:44:55.583311+0000","last_unstale":"2024-09-21T07:44:56.574774+0000","last_undegraded":"2024-09-21T07:44:56.574774+0000","last_fullsized":"2024-09-21T07:44:56.574774+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:13.056811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.574119+0000","last_change":"2024-09-21T07:44:54.577276+0000","last_active":"2024-09-21T07:44:55.574119+0000","last_peered":"2024-09-21T07:44:55.574119+0000","last_clean":"2024-09-21T07:44:55.574119+0000","last_became_active":"2024-09-21T07:44:54.576350+0000","last_became_peered":"2024-09-21T07:44:54.576350+0000","last_unstale":"2024-09-21T07:44:55.574119+0000","last_undegraded":"2024-09-21T07:44:55.574119+0000","last_fullsized":"2024-09-21T07:44:55.574119+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:42.264379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":146,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410369+0000","last_change":"2024-09-21T07:35:27.872027+0000","last_active":"2024-09-21T07:44:57.410369+0000","last_peered":"2024-09-21T07:44:57.410369+0000","last_clean":"2024-09-21T07:44:57.410369+0000","last_became_active":"2024-09-21T07:35:27.870426+0000","last_became_peered":"2024-09-21T07:35:27.870426+0000","last_unstale":"2024-09-21T07:44:57.410369+0000","last_undegraded":"2024-09-21T07:44:57.410369+0000","last_fullsized":"2024-09-21T07:44:57.410369+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":151,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.565447+0000","last_change":"2024-09-21T07:35:29.491008+0000","last_active":"2024-09-21T07:44:55.565447+0000","last_peered":"2024-09-21T07:44:55.565447+0000","last_clean":"2024-09-21T07:44:55.565447+0000","last_became_active":"2024-09-21T07:35:26.866750+0000","last_became_peered":"2024-09-21T07:35:26.866750+0000","last_unstale":"2024-09-21T07:44:55.565447+0000","last_undegraded":"2024-09-21T07:44:55.565447+0000","last_fullsized":"2024-09-21T07:44:55.565447+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:25.490406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083529700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":208,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410202+0000","last_change":"2024-09-21T07:34:24.260665+0000","last_active":"2024-09-21T07:44:57.410202+0000","last_peered":"2024-09-21T07:44:57.410202+0000","last_clean":"2024-09-21T07:44:57.410202+0000","last_became_active":"2024-09-21T07:34:23.635909+0000","last_became_peered":"2024-09-21T07:34:23.635909+0000","last_unstale":"2024-09-21T07:44:57.410202+0000","last_undegraded":"2024-09-21T07:44:57.410202+0000","last_fullsized":"2024-09-21T07:44:57.410202+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:44.536885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576073+0000","last_change":"2024-09-21T07:44:55.578791+0000","last_active":"2024-09-21T07:44:56.576073+0000","last_peered":"2024-09-21T07:44:56.576073+0000","last_clean":"2024-09-21T07:44:56.576073+0000","last_became_active":"2024-09-21T07:44:55.578451+0000","last_became_peered":"2024-09-21T07:44:55.578451+0000","last_unstale":"2024-09-21T07:44:56.576073+0000","last_undegraded":"2024-09-21T07:44:56.576073+0000","last_fullsized":"2024-09-21T07:44:56.576073+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:25:05.931499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"133'2","reported_seq":17,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409640+0000","last_change":"2024-09-21T07:44:54.576757+0000","last_active":"2024-09-21T07:44:57.409640+0000","last_peered":"2024-09-21T07:44:57.409640+0000","last_clean":"2024-09-21T07:44:57.409640+0000","last_became_active":"2024-09-21T07:44:54.576395+0000","last_became_peered":"2024-09-21T07:44:54.576395+0000","last_unstale":"2024-09-21T07:44:57.409640+0000","last_undegraded":"2024-09-21T07:44:57.409640+0000","last_fullsized":"2024-09-21T07:44:57.409640+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:29.874215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":145,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576506+0000","last_change":"2024-09-21T07:35:27.874772+0000","last_active":"2024-09-21T07:44:56.576506+0000","last_peered":"2024-09-21T07:44:56.576506+0000","last_clean":"2024-09-21T07:44:56.576506+0000","last_became_active":"2024-09-21T07:35:27.874434+0000","last_became_peered":"2024-09-21T07:35:27.874434+0000","last_unstale":"2024-09-21T07:44:56.576506+0000","last_undegraded":"2024-09-21T07:44:56.576506+0000","last_fullsized":"2024-09-21T07:44:56.576506+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:31.998755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":157,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576807+0000","last_change":"2024-09-21T07:35:28.859153+0000","last_active":"2024-09-21T07:44:56.576807+0000","last_peered":"2024-09-21T07:44:56.576807+0000","last_clean":"2024-09-21T07:44:56.576807+0000","last_became_active":"2024-09-21T07:35:26.874428+0000","last_became_peered":"2024-09-21T07:35:26.874428+0000","last_unstale":"2024-09-21T07:44:56.576807+0000","last_undegraded":"2024-09-21T07:44:56.576807+0000","last_fullsized":"2024-09-21T07:44:56.576807+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:04.487621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048557299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":209,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.577016+0000","last_change":"2024-09-21T07:34:24.538845+0000","last_active":"2024-09-21T07:44:56.577016+0000","last_peered":"2024-09-21T07:44:56.577016+0000","last_clean":"2024-09-21T07:44:56.577016+0000","last_became_active":"2024-09-21T07:34:23.636372+0000","last_became_peered":"2024-09-21T07:34:23.636372+0000","last_unstale":"2024-09-21T07:44:56.577016+0000","last_undegraded":"2024-09-21T07:44:56.577016+0000","last_fullsized":"2024-09-21T07:44:56.577016+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:29.393813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841451,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410444+0000","last_change":"2024-09-21T07:44:55.586424+0000","last_active":"2024-09-21T07:44:57.410444+0000","last_peered":"2024-09-21T07:44:57.410444+0000","last_clean":"2024-09-21T07:44:57.410444+0000","last_became_active":"2024-09-21T07:44:55.586050+0000","last_became_peered":"2024-09-21T07:44:55.586050+0000","last_unstale":"2024-09-21T07:44:57.410444+0000","last_undegraded":"2024-09-21T07:44:57.410444+0000","last_fullsized":"2024-09-21T07:44:57.410444+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:16.375239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"133'2","reported_seq":17,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.575316+0000","last_change":"2024-09-21T07:44:54.567388+0000","last_active":"2024-09-21T07:44:56.575316+0000","last_peered":"2024-09-21T07:44:56.575316+0000","last_clean":"2024-09-21T07:44:56.575316+0000","last_became_active":"2024-09-21T07:44:54.566216+0000","last_became_peered":"2024-09-21T07:44:54.566216+0000","last_unstale":"2024-09-21T07:44:56.575316+0000","last_undegraded":"2024-09-21T07:44:56.575316+0000","last_fullsized":"2024-09-21T07:44:56.575316+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:45:55.012600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":146,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.408661+0000","last_change":"2024-09-21T07:35:27.877801+0000","last_active":"2024-09-21T07:44:57.408661+0000","last_peered":"2024-09-21T07:44:57.408661+0000","last_clean":"2024-09-21T07:44:57.408661+0000","last_became_active":"2024-09-21T07:35:27.877464+0000","last_became_peered":"2024-09-21T07:35:27.877464+0000","last_unstale":"2024-09-21T07:44:57.408661+0000","last_undegraded":"2024-09-21T07:44:57.408661+0000","last_fullsized":"2024-09-21T07:44:57.408661+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:08.814929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":154,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410452+0000","last_change":"2024-09-21T07:35:28.863732+0000","last_active":"2024-09-21T07:44:57.410452+0000","last_peered":"2024-09-21T07:44:57.410452+0000","last_clean":"2024-09-21T07:44:57.410452+0000","last_became_active":"2024-09-21T07:35:26.877392+0000","last_became_peered":"2024-09-21T07:35:26.877392+0000","last_unstale":"2024-09-21T07:44:57.410452+0000","last_undegraded":"2024-09-21T07:44:57.410452+0000","last_fullsized":"2024-09-21T07:44:57.410452+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071425700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":205,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.567612+0000","last_change":"2024-09-21T07:34:24.259686+0000","last_active":"2024-09-21T07:44:55.567612+0000","last_peered":"2024-09-21T07:44:55.567612+0000","last_clean":"2024-09-21T07:44:55.567612+0000","last_became_active":"2024-09-21T07:34:23.274405+0000","last_became_peered":"2024-09-21T07:34:23.274405+0000","last_unstale":"2024-09-21T07:44:55.567612+0000","last_undegraded":"2024-09-21T07:44:55.567612+0000","last_fullsized":"2024-09-21T07:44:55.567612+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:26.304693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022009920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.575386+0000","last_change":"2024-09-21T07:44:55.589236+0000","last_active":"2024-09-21T07:44:56.575386+0000","last_peered":"2024-09-21T07:44:56.575386+0000","last_clean":"2024-09-21T07:44:56.575386+0000","last_became_active":"2024-09-21T07:44:55.588892+0000","last_became_peered":"2024-09-21T07:44:55.588892+0000","last_unstale":"2024-09-21T07:44:56.575386+0000","last_undegraded":"2024-09-21T07:44:56.575386+0000","last_fullsized":"2024-09-21T07:44:56.575386+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:39.729629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"133'4","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409771+0000","last_change":"2024-09-21T07:44:54.572798+0000","last_active":"2024-09-21T07:44:57.409771+0000","last_peered":"2024-09-21T07:44:57.409771+0000","last_clean":"2024-09-21T07:44:57.409771+0000","last_became_active":"2024-09-21T07:44:54.572390+0000","last_became_peered":"2024-09-21T07:44:54.572390+0000","last_unstale":"2024-09-21T07:44:57.409771+0000","last_undegraded":"2024-09-21T07:44:57.409771+0000","last_fullsized":"2024-09-21T07:44:57.409771+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:07:31.093396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":145,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.577180+0000","last_change":"2024-09-21T07:35:27.867336+0000","last_active":"2024-09-21T07:44:56.577180+0000","last_peered":"2024-09-21T07:44:56.577180+0000","last_clean":"2024-09-21T07:44:56.577180+0000","last_became_active":"2024-09-21T07:35:27.867015+0000","last_became_peered":"2024-09-21T07:35:27.867015+0000","last_unstale":"2024-09-21T07:44:56.577180+0000","last_undegraded":"2024-09-21T07:44:56.577180+0000","last_fullsized":"2024-09-21T07:44:56.577180+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:59.214139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":154,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409964+0000","last_change":"2024-09-21T07:35:29.493137+0000","last_active":"2024-09-21T07:44:57.409964+0000","last_peered":"2024-09-21T07:44:57.409964+0000","last_clean":"2024-09-21T07:44:57.409964+0000","last_became_active":"2024-09-21T07:35:26.872230+0000","last_became_peered":"2024-09-21T07:35:26.872230+0000","last_unstale":"2024-09-21T07:44:57.409964+0000","last_undegraded":"2024-09-21T07:44:57.409964+0000","last_fullsized":"2024-09-21T07:44:57.409964+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:24.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001197449,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":207,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.408121+0000","last_change":"2024-09-21T07:34:24.258754+0000","last_active":"2024-09-21T07:44:57.408121+0000","last_peered":"2024-09-21T07:44:57.408121+0000","last_clean":"2024-09-21T07:44:57.408121+0000","last_became_active":"2024-09-21T07:34:23.272657+0000","last_became_peered":"2024-09-21T07:34:23.272657+0000","last_unstale":"2024-09-21T07:44:57.408121+0000","last_undegraded":"2024-09-21T07:44:57.408121+0000","last_fullsized":"2024-09-21T07:44:57.408121+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:05.477099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067143200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409889+0000","last_change":"2024-09-21T07:44:55.593668+0000","last_active":"2024-09-21T07:44:57.409889+0000","last_peered":"2024-09-21T07:44:57.409889+0000","last_clean":"2024-09-21T07:44:57.409889+0000","last_became_active":"2024-09-21T07:44:55.593316+0000","last_became_peered":"2024-09-21T07:44:55.593316+0000","last_unstale":"2024-09-21T07:44:57.409889+0000","last_undegraded":"2024-09-21T07:44:57.409889+0000","last_fullsized":"2024-09-21T07:44:57.409889+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:57.156663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"133'1","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.575624+0000","last_change":"2024-09-21T07:44:54.567256+0000","last_active":"2024-09-21T07:44:56.575624+0000","last_peered":"2024-09-21T07:44:56.575624+0000","last_clean":"2024-09-21T07:44:56.575624+0000","last_became_active":"2024-09-21T07:44:54.565314+0000","last_became_peered":"2024-09-21T07:44:54.565314+0000","last_unstale":"2024-09-21T07:44:56.575624+0000","last_undegraded":"2024-09-21T07:44:56.575624+0000","last_fullsized":"2024-09-21T07:44:56.575624+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:54:06.235195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":147,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410129+0000","last_change":"2024-09-21T07:35:27.876021+0000","last_active":"2024-09-21T07:44:57.410129+0000","last_peered":"2024-09-21T07:44:57.410129+0000","last_clean":"2024-09-21T07:44:57.410129+0000","last_became_active":"2024-09-21T07:35:27.875724+0000","last_became_peered":"2024-09-21T07:35:27.875724+0000","last_unstale":"2024-09-21T07:44:57.410129+0000","last_undegraded":"2024-09-21T07:44:57.410129+0000","last_fullsized":"2024-09-21T07:44:57.410129+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:03.022812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":154,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409837+0000","last_change":"2024-09-21T07:35:29.492570+0000","last_active":"2024-09-21T07:44:57.409837+0000","last_peered":"2024-09-21T07:44:57.409837+0000","last_clean":"2024-09-21T07:44:57.409837+0000","last_became_active":"2024-09-21T07:35:26.874316+0000","last_became_peered":"2024-09-21T07:35:26.874316+0000","last_unstale":"2024-09-21T07:44:57.409837+0000","last_undegraded":"2024-09-21T07:44:57.409837+0000","last_fullsized":"2024-09-21T07:44:57.409837+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:16.512323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011990760000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":205,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409592+0000","last_change":"2024-09-21T07:34:25.085232+0000","last_active":"2024-09-21T07:44:57.409592+0000","last_peered":"2024-09-21T07:44:57.409592+0000","last_clean":"2024-09-21T07:44:57.409592+0000","last_became_active":"2024-09-21T07:34:23.277807+0000","last_became_peered":"2024-09-21T07:34:23.277807+0000","last_unstale":"2024-09-21T07:44:57.409592+0000","last_undegraded":"2024-09-21T07:44:57.409592+0000","last_fullsized":"2024-09-21T07:44:57.409592+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:36.362919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086619099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":10,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.595191+0000","last_change":"2024-09-21T07:44:55.581656+0000","last_active":"2024-09-21T07:44:55.595191+0000","last_peered":"2024-09-21T07:44:55.595191+0000","last_clean":"2024-09-21T07:44:55.595191+0000","last_became_active":"2024-09-21T07:44:55.580995+0000","last_became_peered":"2024-09-21T07:44:55.580995+0000","last_unstale":"2024-09-21T07:44:55.595191+0000","last_undegraded":"2024-09-21T07:44:55.595191+0000","last_fullsized":"2024-09-21T07:44:55.595191+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:50:34.185292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.575882+0000","last_change":"2024-09-21T07:44:54.566731+0000","last_active":"2024-09-21T07:44:56.575882+0000","last_peered":"2024-09-21T07:44:56.575882+0000","last_clean":"2024-09-21T07:44:56.575882+0000","last_became_active":"2024-09-21T07:44:54.566364+0000","last_became_peered":"2024-09-21T07:44:54.566364+0000","last_unstale":"2024-09-21T07:44:56.575882+0000","last_undegraded":"2024-09-21T07:44:56.575882+0000","last_fullsized":"2024-09-21T07:44:56.575882+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:08.662117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":146,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409289+0000","last_change":"2024-09-21T07:35:27.872798+0000","last_active":"2024-09-21T07:44:57.409289+0000","last_peered":"2024-09-21T07:44:57.409289+0000","last_clean":"2024-09-21T07:44:57.409289+0000","last_became_active":"2024-09-21T07:35:27.872483+0000","last_became_peered":"2024-09-21T07:35:27.872483+0000","last_unstale":"2024-09-21T07:44:57.409289+0000","last_undegraded":"2024-09-21T07:44:57.409289+0000","last_fullsized":"2024-09-21T07:44:57.409289+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:11.933469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":153,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.408218+0000","last_change":"2024-09-21T07:35:29.206723+0000","last_active":"2024-09-21T07:44:57.408218+0000","last_peered":"2024-09-21T07:44:57.408218+0000","last_clean":"2024-09-21T07:44:57.408218+0000","last_became_active":"2024-09-21T07:35:26.872377+0000","last_became_peered":"2024-09-21T07:35:26.872377+0000","last_unstale":"2024-09-21T07:44:57.408218+0000","last_undegraded":"2024-09-21T07:44:57.408218+0000","last_fullsized":"2024-09-21T07:44:57.408218+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:29.086862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083077200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":201,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.578425+0000","last_change":"2024-09-21T07:34:24.597672+0000","last_active":"2024-09-21T07:44:56.578425+0000","last_peered":"2024-09-21T07:44:56.578425+0000","last_clean":"2024-09-21T07:44:56.578425+0000","last_became_active":"2024-09-21T07:34:23.267770+0000","last_became_peered":"2024-09-21T07:34:23.267770+0000","last_unstale":"2024-09-21T07:44:56.578425+0000","last_undegraded":"2024-09-21T07:44:56.578425+0000","last_fullsized":"2024-09-21T07:44:56.578425+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:51.629476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086378500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.574971+0000","last_change":"2024-09-21T07:44:55.592212+0000","last_active":"2024-09-21T07:44:56.574971+0000","last_peered":"2024-09-21T07:44:56.574971+0000","last_clean":"2024-09-21T07:44:56.574971+0000","last_became_active":"2024-09-21T07:44:55.591594+0000","last_became_peered":"2024-09-21T07:44:55.591594+0000","last_unstale":"2024-09-21T07:44:56.574971+0000","last_undegraded":"2024-09-21T07:44:56.574971+0000","last_fullsized":"2024-09-21T07:44:56.574971+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:07:32.096178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409219+0000","last_change":"2024-09-21T07:44:54.573479+0000","last_active":"2024-09-21T07:44:57.409219+0000","last_peered":"2024-09-21T07:44:57.409219+0000","last_clean":"2024-09-21T07:44:57.409219+0000","last_became_active":"2024-09-21T07:44:54.572774+0000","last_became_peered":"2024-09-21T07:44:54.572774+0000","last_unstale":"2024-09-21T07:44:57.409219+0000","last_undegraded":"2024-09-21T07:44:57.409219+0000","last_fullsized":"2024-09-21T07:44:57.409219+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:51:31.141936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":146,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409146+0000","last_change":"2024-09-21T07:35:27.870121+0000","last_active":"2024-09-21T07:44:57.409146+0000","last_peered":"2024-09-21T07:44:57.409146+0000","last_clean":"2024-09-21T07:44:57.409146+0000","last_became_active":"2024-09-21T07:35:27.869811+0000","last_became_peered":"2024-09-21T07:35:27.869811+0000","last_unstale":"2024-09-21T07:44:57.409146+0000","last_undegraded":"2024-09-21T07:44:57.409146+0000","last_fullsized":"2024-09-21T07:44:57.409146+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:31.778461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":154,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.925283+0000","last_change":"2024-09-21T07:35:28.860699+0000","last_active":"2024-09-21T07:44:56.925283+0000","last_peered":"2024-09-21T07:44:56.925283+0000","last_clean":"2024-09-21T07:44:56.925283+0000","last_became_active":"2024-09-21T07:35:26.876779+0000","last_became_peered":"2024-09-21T07:35:26.876779+0000","last_unstale":"2024-09-21T07:44:56.925283+0000","last_undegraded":"2024-09-21T07:44:56.925283+0000","last_fullsized":"2024-09-21T07:44:56.925283+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:28.286403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000913188,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":206,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.926022+0000","last_change":"2024-09-21T07:34:24.260181+0000","last_active":"2024-09-21T07:44:56.926022+0000","last_peered":"2024-09-21T07:44:56.926022+0000","last_clean":"2024-09-21T07:44:56.926022+0000","last_became_active":"2024-09-21T07:34:23.275580+0000","last_became_peered":"2024-09-21T07:34:23.275580+0000","last_unstale":"2024-09-21T07:44:56.926022+0000","last_undegraded":"2024-09-21T07:44:56.926022+0000","last_fullsized":"2024-09-21T07:44:56.926022+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:49.857622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014706980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.574597+0000","last_change":"2024-09-21T07:44:55.573251+0000","last_active":"2024-09-21T07:44:56.574597+0000","last_peered":"2024-09-21T07:44:56.574597+0000","last_clean":"2024-09-21T07:44:56.574597+0000","last_became_active":"2024-09-21T07:44:55.572910+0000","last_became_peered":"2024-09-21T07:44:55.572910+0000","last_unstale":"2024-09-21T07:44:56.574597+0000","last_undegraded":"2024-09-21T07:44:56.574597+0000","last_fullsized":"2024-09-21T07:44:56.574597+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:35:39.531219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"133'1","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.578049+0000","last_change":"2024-09-21T07:44:54.582263+0000","last_active":"2024-09-21T07:44:56.578049+0000","last_peered":"2024-09-21T07:44:56.578049+0000","last_clean":"2024-09-21T07:44:56.578049+0000","last_became_active":"2024-09-21T07:44:54.581985+0000","last_became_peered":"2024-09-21T07:44:54.581985+0000","last_unstale":"2024-09-21T07:44:56.578049+0000","last_undegraded":"2024-09-21T07:44:56.578049+0000","last_fullsized":"2024-09-21T07:44:56.578049+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:48.161401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":148,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.575688+0000","last_change":"2024-09-21T07:35:27.875393+0000","last_active":"2024-09-21T07:44:56.575688+0000","last_peered":"2024-09-21T07:44:56.575688+0000","last_clean":"2024-09-21T07:44:56.575688+0000","last_became_active":"2024-09-21T07:35:27.874671+0000","last_became_peered":"2024-09-21T07:35:27.874671+0000","last_unstale":"2024-09-21T07:44:56.575688+0000","last_undegraded":"2024-09-21T07:44:56.575688+0000","last_fullsized":"2024-09-21T07:44:56.575688+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":153,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.408857+0000","last_change":"2024-09-21T07:35:29.206296+0000","last_active":"2024-09-21T07:44:57.408857+0000","last_peered":"2024-09-21T07:44:57.408857+0000","last_clean":"2024-09-21T07:44:57.408857+0000","last_became_active":"2024-09-21T07:35:26.874575+0000","last_became_peered":"2024-09-21T07:35:26.874575+0000","last_unstale":"2024-09-21T07:44:57.408857+0000","last_undegraded":"2024-09-21T07:44:57.408857+0000","last_fullsized":"2024-09-21T07:44:57.408857+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:22.771154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061011599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":205,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.571513+0000","last_change":"2024-09-21T07:34:24.259112+0000","last_active":"2024-09-21T07:44:55.571513+0000","last_peered":"2024-09-21T07:44:55.571513+0000","last_clean":"2024-09-21T07:44:55.571513+0000","last_became_active":"2024-09-21T07:34:23.635444+0000","last_became_peered":"2024-09-21T07:34:23.635444+0000","last_unstale":"2024-09-21T07:44:55.571513+0000","last_undegraded":"2024-09-21T07:44:55.571513+0000","last_fullsized":"2024-09-21T07:44:55.571513+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:46.255385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001952636,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576512+0000","last_change":"2024-09-21T07:44:55.591992+0000","last_active":"2024-09-21T07:44:56.576512+0000","last_peered":"2024-09-21T07:44:56.576512+0000","last_clean":"2024-09-21T07:44:56.576512+0000","last_became_active":"2024-09-21T07:44:55.591655+0000","last_became_peered":"2024-09-21T07:44:55.591655+0000","last_unstale":"2024-09-21T07:44:56.576512+0000","last_undegraded":"2024-09-21T07:44:56.576512+0000","last_fullsized":"2024-09-21T07:44:56.576512+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:02:53.367109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"133'1","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410264+0000","last_change":"2024-09-21T07:44:54.570542+0000","last_active":"2024-09-21T07:44:57.410264+0000","last_peered":"2024-09-21T07:44:57.410264+0000","last_clean":"2024-09-21T07:44:57.410264+0000","last_became_active":"2024-09-21T07:44:54.570143+0000","last_became_peered":"2024-09-21T07:44:54.570143+0000","last_unstale":"2024-09-21T07:44:57.410264+0000","last_undegraded":"2024-09-21T07:44:57.410264+0000","last_fullsized":"2024-09-21T07:44:57.410264+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:25:42.288412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":148,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576045+0000","last_change":"2024-09-21T07:35:27.878356+0000","last_active":"2024-09-21T07:44:56.576045+0000","last_peered":"2024-09-21T07:44:56.576045+0000","last_clean":"2024-09-21T07:44:56.576045+0000","last_became_active":"2024-09-21T07:35:27.877982+0000","last_became_peered":"2024-09-21T07:35:27.877982+0000","last_unstale":"2024-09-21T07:44:56.576045+0000","last_undegraded":"2024-09-21T07:44:56.576045+0000","last_fullsized":"2024-09-21T07:44:56.576045+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":155,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.574611+0000","last_change":"2024-09-21T07:35:29.053131+0000","last_active":"2024-09-21T07:44:56.574611+0000","last_peered":"2024-09-21T07:44:56.574611+0000","last_clean":"2024-09-21T07:44:56.574611+0000","last_became_active":"2024-09-21T07:35:26.870516+0000","last_became_peered":"2024-09-21T07:35:26.870516+0000","last_unstale":"2024-09-21T07:44:56.574611+0000","last_undegraded":"2024-09-21T07:44:56.574611+0000","last_fullsized":"2024-09-21T07:44:56.574611+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:10.924491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070705699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":209,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.577837+0000","last_change":"2024-09-21T07:34:24.539152+0000","last_active":"2024-09-21T07:44:56.577837+0000","last_peered":"2024-09-21T07:44:56.577837+0000","last_clean":"2024-09-21T07:44:56.577837+0000","last_became_active":"2024-09-21T07:34:23.259573+0000","last_became_peered":"2024-09-21T07:34:23.259573+0000","last_unstale":"2024-09-21T07:44:56.577837+0000","last_undegraded":"2024-09-21T07:44:56.577837+0000","last_fullsized":"2024-09-21T07:44:56.577837+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:26.986900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058108400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409847+0000","last_change":"2024-09-21T07:44:55.591791+0000","last_active":"2024-09-21T07:44:57.409847+0000","last_peered":"2024-09-21T07:44:57.409847+0000","last_clean":"2024-09-21T07:44:57.409847+0000","last_became_active":"2024-09-21T07:44:55.586035+0000","last_became_peered":"2024-09-21T07:44:55.586035+0000","last_unstale":"2024-09-21T07:44:57.409847+0000","last_undegraded":"2024-09-21T07:44:57.409847+0000","last_fullsized":"2024-09-21T07:44:57.409847+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:04:48.783564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409544+0000","last_change":"2024-09-21T07:44:54.579509+0000","last_active":"2024-09-21T07:44:57.409544+0000","last_peered":"2024-09-21T07:44:57.409544+0000","last_clean":"2024-09-21T07:44:57.409544+0000","last_became_active":"2024-09-21T07:44:54.579162+0000","last_became_peered":"2024-09-21T07:44:54.579162+0000","last_unstale":"2024-09-21T07:44:57.409544+0000","last_undegraded":"2024-09-21T07:44:57.409544+0000","last_fullsized":"2024-09-21T07:44:57.409544+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:53:06.530441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":147,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.574468+0000","last_change":"2024-09-21T07:35:27.873726+0000","last_active":"2024-09-21T07:44:56.574468+0000","last_peered":"2024-09-21T07:44:56.574468+0000","last_clean":"2024-09-21T07:44:56.574468+0000","last_became_active":"2024-09-21T07:35:27.871421+0000","last_became_peered":"2024-09-21T07:35:27.871421+0000","last_unstale":"2024-09-21T07:44:56.574468+0000","last_undegraded":"2024-09-21T07:44:56.574468+0000","last_fullsized":"2024-09-21T07:44:56.574468+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:00.168915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":154,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.925873+0000","last_change":"2024-09-21T07:35:28.861032+0000","last_active":"2024-09-21T07:44:56.925873+0000","last_peered":"2024-09-21T07:44:56.925873+0000","last_clean":"2024-09-21T07:44:56.925873+0000","last_became_active":"2024-09-21T07:35:26.873149+0000","last_became_peered":"2024-09-21T07:35:26.873149+0000","last_unstale":"2024-09-21T07:44:56.925873+0000","last_undegraded":"2024-09-21T07:44:56.925873+0000","last_fullsized":"2024-09-21T07:44:56.925873+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:59.435981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011988369999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":207,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409164+0000","last_change":"2024-09-21T07:34:24.259902+0000","last_active":"2024-09-21T07:44:57.409164+0000","last_peered":"2024-09-21T07:44:57.409164+0000","last_clean":"2024-09-21T07:44:57.409164+0000","last_became_active":"2024-09-21T07:34:23.271902+0000","last_became_peered":"2024-09-21T07:34:23.271902+0000","last_unstale":"2024-09-21T07:44:57.409164+0000","last_undegraded":"2024-09-21T07:44:57.409164+0000","last_fullsized":"2024-09-21T07:44:57.409164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:59.439525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001686358,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.926013+0000","last_change":"2024-09-21T07:44:55.584056+0000","last_active":"2024-09-21T07:44:56.926013+0000","last_peered":"2024-09-21T07:44:56.926013+0000","last_clean":"2024-09-21T07:44:56.926013+0000","last_became_active":"2024-09-21T07:44:55.583709+0000","last_became_peered":"2024-09-21T07:44:55.583709+0000","last_unstale":"2024-09-21T07:44:56.926013+0000","last_undegraded":"2024-09-21T07:44:56.926013+0000","last_fullsized":"2024-09-21T07:44:56.926013+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:27.776809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.411093+0000","last_change":"2024-09-21T07:44:54.571782+0000","last_active":"2024-09-21T07:44:57.411093+0000","last_peered":"2024-09-21T07:44:57.411093+0000","last_clean":"2024-09-21T07:44:57.411093+0000","last_became_active":"2024-09-21T07:44:54.571458+0000","last_became_peered":"2024-09-21T07:44:54.571458+0000","last_unstale":"2024-09-21T07:44:57.411093+0000","last_undegraded":"2024-09-21T07:44:57.411093+0000","last_fullsized":"2024-09-21T07:44:57.411093+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:13:26.409608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":146,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409316+0000","last_change":"2024-09-21T07:35:27.872637+0000","last_active":"2024-09-21T07:44:57.409316+0000","last_peered":"2024-09-21T07:44:57.409316+0000","last_clean":"2024-09-21T07:44:57.409316+0000","last_became_active":"2024-09-21T07:35:27.872149+0000","last_became_peered":"2024-09-21T07:35:27.872149+0000","last_unstale":"2024-09-21T07:44:57.409316+0000","last_undegraded":"2024-09-21T07:44:57.409316+0000","last_fullsized":"2024-09-21T07:44:57.409316+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:27.697872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":154,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409759+0000","last_change":"2024-09-21T07:35:29.493362+0000","last_active":"2024-09-21T07:44:57.409759+0000","last_peered":"2024-09-21T07:44:57.409759+0000","last_clean":"2024-09-21T07:44:57.409759+0000","last_became_active":"2024-09-21T07:35:26.873018+0000","last_became_peered":"2024-09-21T07:35:26.873018+0000","last_unstale":"2024-09-21T07:44:57.409759+0000","last_undegraded":"2024-09-21T07:44:57.409759+0000","last_fullsized":"2024-09-21T07:44:57.409759+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:02.460557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012496510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":205,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.567418+0000","last_change":"2024-09-21T07:34:24.259662+0000","last_active":"2024-09-21T07:44:55.567418+0000","last_peered":"2024-09-21T07:44:55.567418+0000","last_clean":"2024-09-21T07:44:55.567418+0000","last_became_active":"2024-09-21T07:34:23.278812+0000","last_became_peered":"2024-09-21T07:34:23.278812+0000","last_unstale":"2024-09-21T07:44:55.567418+0000","last_undegraded":"2024-09-21T07:44:55.567418+0000","last_fullsized":"2024-09-21T07:44:55.567418+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:49.829974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356463,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.411031+0000","last_change":"2024-09-21T07:44:55.580365+0000","last_active":"2024-09-21T07:44:57.411031+0000","last_peered":"2024-09-21T07:44:57.411031+0000","last_clean":"2024-09-21T07:44:57.411031+0000","last_became_active":"2024-09-21T07:44:55.580000+0000","last_became_peered":"2024-09-21T07:44:55.580000+0000","last_unstale":"2024-09-21T07:44:57.411031+0000","last_undegraded":"2024-09-21T07:44:57.411031+0000","last_fullsized":"2024-09-21T07:44:57.411031+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:56.224315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"133'2","reported_seq":17,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410640+0000","last_change":"2024-09-21T07:44:54.578311+0000","last_active":"2024-09-21T07:44:57.410640+0000","last_peered":"2024-09-21T07:44:57.410640+0000","last_clean":"2024-09-21T07:44:57.410640+0000","last_became_active":"2024-09-21T07:44:54.577821+0000","last_became_peered":"2024-09-21T07:44:54.577821+0000","last_unstale":"2024-09-21T07:44:57.410640+0000","last_undegraded":"2024-09-21T07:44:57.410640+0000","last_fullsized":"2024-09-21T07:44:57.410640+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:06.744686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":144,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576903+0000","last_change":"2024-09-21T07:35:27.868448+0000","last_active":"2024-09-21T07:44:56.576903+0000","last_peered":"2024-09-21T07:44:56.576903+0000","last_clean":"2024-09-21T07:44:56.576903+0000","last_became_active":"2024-09-21T07:35:27.868105+0000","last_became_peered":"2024-09-21T07:35:27.868105+0000","last_unstale":"2024-09-21T07:44:56.576903+0000","last_undegraded":"2024-09-21T07:44:56.576903+0000","last_fullsized":"2024-09-21T07:44:56.576903+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:41.468702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":153,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409715+0000","last_change":"2024-09-21T07:35:29.492974+0000","last_active":"2024-09-21T07:44:57.409715+0000","last_peered":"2024-09-21T07:44:57.409715+0000","last_clean":"2024-09-21T07:44:57.409715+0000","last_became_active":"2024-09-21T07:35:26.872165+0000","last_became_peered":"2024-09-21T07:35:26.872165+0000","last_unstale":"2024-09-21T07:44:57.409715+0000","last_undegraded":"2024-09-21T07:44:57.409715+0000","last_fullsized":"2024-09-21T07:44:57.409715+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:59.912520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013795179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":205,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409293+0000","last_change":"2024-09-21T07:34:25.084861+0000","last_active":"2024-09-21T07:44:57.409293+0000","last_peered":"2024-09-21T07:44:57.409293+0000","last_clean":"2024-09-21T07:44:57.409293+0000","last_became_active":"2024-09-21T07:34:23.277708+0000","last_became_peered":"2024-09-21T07:34:23.277708+0000","last_unstale":"2024-09-21T07:44:57.409293+0000","last_undegraded":"2024-09-21T07:44:57.409293+0000","last_fullsized":"2024-09-21T07:44:57.409293+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:40.816488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000597566,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.575498+0000","last_change":"2024-09-21T07:44:55.579831+0000","last_active":"2024-09-21T07:44:56.575498+0000","last_peered":"2024-09-21T07:44:56.575498+0000","last_clean":"2024-09-21T07:44:56.575498+0000","last_became_active":"2024-09-21T07:44:55.578589+0000","last_became_peered":"2024-09-21T07:44:55.578589+0000","last_unstale":"2024-09-21T07:44:56.575498+0000","last_undegraded":"2024-09-21T07:44:56.575498+0000","last_fullsized":"2024-09-21T07:44:56.575498+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:53.300887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"133'1","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410195+0000","last_change":"2024-09-21T07:44:54.575177+0000","last_active":"2024-09-21T07:44:57.410195+0000","last_peered":"2024-09-21T07:44:57.410195+0000","last_clean":"2024-09-21T07:44:57.410195+0000","last_became_active":"2024-09-21T07:44:54.574885+0000","last_became_peered":"2024-09-21T07:44:54.574885+0000","last_unstale":"2024-09-21T07:44:57.410195+0000","last_undegraded":"2024-09-21T07:44:57.410195+0000","last_fullsized":"2024-09-21T07:44:57.410195+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:12.054029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":144,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.578033+0000","last_change":"2024-09-21T07:35:27.874640+0000","last_active":"2024-09-21T07:44:56.578033+0000","last_peered":"2024-09-21T07:44:56.578033+0000","last_clean":"2024-09-21T07:44:56.578033+0000","last_became_active":"2024-09-21T07:35:27.874265+0000","last_became_peered":"2024-09-21T07:35:27.874265+0000","last_unstale":"2024-09-21T07:44:56.578033+0000","last_undegraded":"2024-09-21T07:44:56.578033+0000","last_fullsized":"2024-09-21T07:44:56.578033+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:45.754654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":152,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409564+0000","last_change":"2024-09-21T07:35:29.207392+0000","last_active":"2024-09-21T07:44:57.409564+0000","last_peered":"2024-09-21T07:44:57.409564+0000","last_clean":"2024-09-21T07:44:57.409564+0000","last_became_active":"2024-09-21T07:35:26.870917+0000","last_became_peered":"2024-09-21T07:35:26.870917+0000","last_unstale":"2024-09-21T07:44:57.409564+0000","last_undegraded":"2024-09-21T07:44:57.409564+0000","last_fullsized":"2024-09-21T07:44:57.409564+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:59.159937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059198499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":205,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.573965+0000","last_change":"2024-09-21T07:34:24.258709+0000","last_active":"2024-09-21T07:44:55.573965+0000","last_peered":"2024-09-21T07:44:55.573965+0000","last_clean":"2024-09-21T07:44:55.573965+0000","last_became_active":"2024-09-21T07:34:23.279042+0000","last_became_peered":"2024-09-21T07:34:23.279042+0000","last_unstale":"2024-09-21T07:44:55.573965+0000","last_undegraded":"2024-09-21T07:44:55.573965+0000","last_fullsized":"2024-09-21T07:44:55.573965+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:01.762117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019282100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.408953+0000","last_change":"2024-09-21T07:44:55.586454+0000","last_active":"2024-09-21T07:44:57.408953+0000","last_peered":"2024-09-21T07:44:57.408953+0000","last_clean":"2024-09-21T07:44:57.408953+0000","last_became_active":"2024-09-21T07:44:55.586082+0000","last_became_peered":"2024-09-21T07:44:55.586082+0000","last_unstale":"2024-09-21T07:44:57.408953+0000","last_undegraded":"2024-09-21T07:44:57.408953+0000","last_fullsized":"2024-09-21T07:44:57.408953+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:55.284231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410765+0000","last_change":"2024-09-21T07:44:54.571759+0000","last_active":"2024-09-21T07:44:57.410765+0000","last_peered":"2024-09-21T07:44:57.410765+0000","last_clean":"2024-09-21T07:44:57.410765+0000","last_became_active":"2024-09-21T07:44:54.571430+0000","last_became_peered":"2024-09-21T07:44:54.571430+0000","last_unstale":"2024-09-21T07:44:57.410765+0000","last_undegraded":"2024-09-21T07:44:57.410765+0000","last_fullsized":"2024-09-21T07:44:57.410765+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:22.572592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":146,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.411128+0000","last_change":"2024-09-21T07:35:27.870260+0000","last_active":"2024-09-21T07:44:57.411128+0000","last_peered":"2024-09-21T07:44:57.411128+0000","last_clean":"2024-09-21T07:44:57.411128+0000","last_became_active":"2024-09-21T07:35:27.869448+0000","last_became_peered":"2024-09-21T07:35:27.869448+0000","last_unstale":"2024-09-21T07:44:57.411128+0000","last_undegraded":"2024-09-21T07:44:57.411128+0000","last_fullsized":"2024-09-21T07:44:57.411128+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:16.946790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":151,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.577906+0000","last_change":"2024-09-21T07:35:29.386367+0000","last_active":"2024-09-21T07:44:56.577906+0000","last_peered":"2024-09-21T07:44:56.577906+0000","last_clean":"2024-09-21T07:44:56.577906+0000","last_became_active":"2024-09-21T07:35:26.879238+0000","last_became_peered":"2024-09-21T07:35:26.879238+0000","last_unstale":"2024-09-21T07:44:56.577906+0000","last_undegraded":"2024-09-21T07:44:56.577906+0000","last_fullsized":"2024-09-21T07:44:56.577906+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:16.528260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092223100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":205,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410642+0000","last_change":"2024-09-21T07:34:25.085635+0000","last_active":"2024-09-21T07:44:57.410642+0000","last_peered":"2024-09-21T07:44:57.410642+0000","last_clean":"2024-09-21T07:44:57.410642+0000","last_became_active":"2024-09-21T07:34:23.273566+0000","last_became_peered":"2024-09-21T07:34:23.273566+0000","last_unstale":"2024-09-21T07:44:57.410642+0000","last_undegraded":"2024-09-21T07:44:57.410642+0000","last_fullsized":"2024-09-21T07:44:57.410642+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:14.142272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079897300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.574972+0000","last_change":"2024-09-21T07:44:55.594648+0000","last_active":"2024-09-21T07:44:56.574972+0000","last_peered":"2024-09-21T07:44:56.574972+0000","last_clean":"2024-09-21T07:44:56.574972+0000","last_became_active":"2024-09-21T07:44:55.591974+0000","last_became_peered":"2024-09-21T07:44:55.591974+0000","last_unstale":"2024-09-21T07:44:56.574972+0000","last_undegraded":"2024-09-21T07:44:56.574972+0000","last_fullsized":"2024-09-21T07:44:56.574972+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:53:07.533223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.408008+0000","last_change":"2024-09-21T07:44:54.569509+0000","last_active":"2024-09-21T07:44:57.408008+0000","last_peered":"2024-09-21T07:44:57.408008+0000","last_clean":"2024-09-21T07:44:57.408008+0000","last_became_active":"2024-09-21T07:44:54.569173+0000","last_became_peered":"2024-09-21T07:44:54.569173+0000","last_unstale":"2024-09-21T07:44:57.408008+0000","last_undegraded":"2024-09-21T07:44:57.408008+0000","last_fullsized":"2024-09-21T07:44:57.408008+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:04.900165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":146,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410112+0000","last_change":"2024-09-21T07:35:27.878728+0000","last_active":"2024-09-21T07:44:57.410112+0000","last_peered":"2024-09-21T07:44:57.410112+0000","last_clean":"2024-09-21T07:44:57.410112+0000","last_became_active":"2024-09-21T07:35:27.878335+0000","last_became_peered":"2024-09-21T07:35:27.878335+0000","last_unstale":"2024-09-21T07:44:57.410112+0000","last_undegraded":"2024-09-21T07:44:57.410112+0000","last_fullsized":"2024-09-21T07:44:57.410112+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:25:37.731771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":156,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576819+0000","last_change":"2024-09-21T07:35:28.858418+0000","last_active":"2024-09-21T07:44:56.576819+0000","last_peered":"2024-09-21T07:44:56.576819+0000","last_clean":"2024-09-21T07:44:56.576819+0000","last_became_active":"2024-09-21T07:35:26.876944+0000","last_became_peered":"2024-09-21T07:35:26.876944+0000","last_unstale":"2024-09-21T07:44:56.576819+0000","last_undegraded":"2024-09-21T07:44:56.576819+0000","last_fullsized":"2024-09-21T07:44:56.576819+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:04.514304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090862799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":201,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.578579+0000","last_change":"2024-09-21T07:34:24.597926+0000","last_active":"2024-09-21T07:44:56.578579+0000","last_peered":"2024-09-21T07:44:56.578579+0000","last_clean":"2024-09-21T07:44:56.578579+0000","last_became_active":"2024-09-21T07:34:23.268926+0000","last_became_peered":"2024-09-21T07:34:23.268926+0000","last_unstale":"2024-09-21T07:44:56.578579+0000","last_undegraded":"2024-09-21T07:44:56.578579+0000","last_fullsized":"2024-09-21T07:44:56.578579+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:46.190752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101851,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409774+0000","last_change":"2024-09-21T07:44:55.578405+0000","last_active":"2024-09-21T07:44:57.409774+0000","last_peered":"2024-09-21T07:44:57.409774+0000","last_clean":"2024-09-21T07:44:57.409774+0000","last_became_active":"2024-09-21T07:44:55.578106+0000","last_became_peered":"2024-09-21T07:44:55.578106+0000","last_unstale":"2024-09-21T07:44:57.409774+0000","last_undegraded":"2024-09-21T07:44:57.409774+0000","last_fullsized":"2024-09-21T07:44:57.409774+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:40.279797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576910+0000","last_change":"2024-09-21T07:44:54.567157+0000","last_active":"2024-09-21T07:44:56.576910+0000","last_peered":"2024-09-21T07:44:56.576910+0000","last_clean":"2024-09-21T07:44:56.576910+0000","last_became_active":"2024-09-21T07:44:54.566497+0000","last_became_peered":"2024-09-21T07:44:54.566497+0000","last_unstale":"2024-09-21T07:44:56.576910+0000","last_undegraded":"2024-09-21T07:44:56.576910+0000","last_fullsized":"2024-09-21T07:44:56.576910+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:15.695364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":145,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.578283+0000","last_change":"2024-09-21T07:35:27.877426+0000","last_active":"2024-09-21T07:44:56.578283+0000","last_peered":"2024-09-21T07:44:56.578283+0000","last_clean":"2024-09-21T07:44:56.578283+0000","last_became_active":"2024-09-21T07:35:27.877050+0000","last_became_peered":"2024-09-21T07:35:27.877050+0000","last_unstale":"2024-09-21T07:44:56.578283+0000","last_undegraded":"2024-09-21T07:44:56.578283+0000","last_fullsized":"2024-09-21T07:44:56.578283+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:36.268334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":153,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409425+0000","last_change":"2024-09-21T07:35:28.862273+0000","last_active":"2024-09-21T07:44:57.409425+0000","last_peered":"2024-09-21T07:44:57.409425+0000","last_clean":"2024-09-21T07:44:57.409425+0000","last_became_active":"2024-09-21T07:35:26.874011+0000","last_became_peered":"2024-09-21T07:35:26.874011+0000","last_unstale":"2024-09-21T07:44:57.409425+0000","last_undegraded":"2024-09-21T07:44:57.409425+0000","last_fullsized":"2024-09-21T07:44:57.409425+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:11.986950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071644799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":206,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.925678+0000","last_change":"2024-09-21T07:34:24.259794+0000","last_active":"2024-09-21T07:44:56.925678+0000","last_peered":"2024-09-21T07:44:56.925678+0000","last_clean":"2024-09-21T07:44:56.925678+0000","last_became_active":"2024-09-21T07:34:23.265679+0000","last_became_peered":"2024-09-21T07:34:23.265679+0000","last_unstale":"2024-09-21T07:44:56.925678+0000","last_undegraded":"2024-09-21T07:44:56.925678+0000","last_fullsized":"2024-09-21T07:44:56.925678+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:02.575796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001298518,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.596670+0000","last_change":"2024-09-21T07:44:55.582406+0000","last_active":"2024-09-21T07:44:55.596670+0000","last_peered":"2024-09-21T07:44:55.596670+0000","last_clean":"2024-09-21T07:44:55.596670+0000","last_became_active":"2024-09-21T07:44:55.581499+0000","last_became_peered":"2024-09-21T07:44:55.581499+0000","last_unstale":"2024-09-21T07:44:55.596670+0000","last_undegraded":"2024-09-21T07:44:55.596670+0000","last_fullsized":"2024-09-21T07:44:55.596670+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:30.495936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576728+0000","last_change":"2024-09-21T07:44:54.568321+0000","last_active":"2024-09-21T07:44:56.576728+0000","last_peered":"2024-09-21T07:44:56.576728+0000","last_clean":"2024-09-21T07:44:56.576728+0000","last_became_active":"2024-09-21T07:44:54.566990+0000","last_became_peered":"2024-09-21T07:44:54.566990+0000","last_unstale":"2024-09-21T07:44:56.576728+0000","last_undegraded":"2024-09-21T07:44:56.576728+0000","last_fullsized":"2024-09-21T07:44:56.576728+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:10:56.227159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":146,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409554+0000","last_change":"2024-09-21T07:35:27.877725+0000","last_active":"2024-09-21T07:44:57.409554+0000","last_peered":"2024-09-21T07:44:57.409554+0000","last_clean":"2024-09-21T07:44:57.409554+0000","last_became_active":"2024-09-21T07:35:27.877398+0000","last_became_peered":"2024-09-21T07:35:27.877398+0000","last_unstale":"2024-09-21T07:44:57.409554+0000","last_undegraded":"2024-09-21T07:44:57.409554+0000","last_fullsized":"2024-09-21T07:44:57.409554+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:05.859762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":153,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.925279+0000","last_change":"2024-09-21T07:35:28.860612+0000","last_active":"2024-09-21T07:44:56.925279+0000","last_peered":"2024-09-21T07:44:56.925279+0000","last_clean":"2024-09-21T07:44:56.925279+0000","last_became_active":"2024-09-21T07:35:26.876562+0000","last_became_peered":"2024-09-21T07:35:26.876562+0000","last_unstale":"2024-09-21T07:44:56.925279+0000","last_undegraded":"2024-09-21T07:44:56.925279+0000","last_fullsized":"2024-09-21T07:44:56.925279+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:06.824331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012252470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":207,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410078+0000","last_change":"2024-09-21T07:34:24.260330+0000","last_active":"2024-09-21T07:44:57.410078+0000","last_peered":"2024-09-21T07:44:57.410078+0000","last_clean":"2024-09-21T07:44:57.410078+0000","last_became_active":"2024-09-21T07:34:23.637210+0000","last_became_peered":"2024-09-21T07:34:23.637210+0000","last_unstale":"2024-09-21T07:44:57.410078+0000","last_undegraded":"2024-09-21T07:44:57.410078+0000","last_fullsized":"2024-09-21T07:44:57.410078+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.488451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001537093,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410266+0000","last_change":"2024-09-21T07:44:55.582996+0000","last_active":"2024-09-21T07:44:57.410266+0000","last_peered":"2024-09-21T07:44:57.410266+0000","last_clean":"2024-09-21T07:44:57.410266+0000","last_became_active":"2024-09-21T07:44:55.582596+0000","last_became_peered":"2024-09-21T07:44:55.582596+0000","last_unstale":"2024-09-21T07:44:57.410266+0000","last_undegraded":"2024-09-21T07:44:57.410266+0000","last_fullsized":"2024-09-21T07:44:57.410266+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:51:28.363844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"133'1","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409762+0000","last_change":"2024-09-21T07:44:54.576648+0000","last_active":"2024-09-21T07:44:57.409762+0000","last_peered":"2024-09-21T07:44:57.409762+0000","last_clean":"2024-09-21T07:44:57.409762+0000","last_became_active":"2024-09-21T07:44:54.576284+0000","last_became_peered":"2024-09-21T07:44:54.576284+0000","last_unstale":"2024-09-21T07:44:57.409762+0000","last_undegraded":"2024-09-21T07:44:57.409762+0000","last_fullsized":"2024-09-21T07:44:57.409762+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:25:04.928717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":147,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.408896+0000","last_change":"2024-09-21T07:35:27.874757+0000","last_active":"2024-09-21T07:44:57.408896+0000","last_peered":"2024-09-21T07:44:57.408896+0000","last_clean":"2024-09-21T07:44:57.408896+0000","last_became_active":"2024-09-21T07:35:27.874360+0000","last_became_peered":"2024-09-21T07:35:27.874360+0000","last_unstale":"2024-09-21T07:44:57.408896+0000","last_undegraded":"2024-09-21T07:44:57.408896+0000","last_fullsized":"2024-09-21T07:44:57.408896+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:17.646912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":153,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410602+0000","last_change":"2024-09-21T07:35:29.493431+0000","last_active":"2024-09-21T07:44:57.410602+0000","last_peered":"2024-09-21T07:44:57.410602+0000","last_clean":"2024-09-21T07:44:57.410602+0000","last_became_active":"2024-09-21T07:35:26.873691+0000","last_became_peered":"2024-09-21T07:35:26.873691+0000","last_unstale":"2024-09-21T07:44:57.410602+0000","last_undegraded":"2024-09-21T07:44:57.410602+0000","last_fullsized":"2024-09-21T07:44:57.410602+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:51.705051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001162464,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":201,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.577291+0000","last_change":"2024-09-21T07:34:24.597286+0000","last_active":"2024-09-21T07:44:56.577291+0000","last_peered":"2024-09-21T07:44:56.577291+0000","last_clean":"2024-09-21T07:44:56.577291+0000","last_became_active":"2024-09-21T07:34:23.267133+0000","last_became_peered":"2024-09-21T07:34:23.267133+0000","last_unstale":"2024-09-21T07:44:56.577291+0000","last_undegraded":"2024-09-21T07:44:56.577291+0000","last_fullsized":"2024-09-21T07:44:56.577291+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:18.257010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576660+0000","last_change":"2024-09-21T07:44:55.584018+0000","last_active":"2024-09-21T07:44:56.576660+0000","last_peered":"2024-09-21T07:44:56.576660+0000","last_clean":"2024-09-21T07:44:56.576660+0000","last_became_active":"2024-09-21T07:44:55.583638+0000","last_became_peered":"2024-09-21T07:44:55.583638+0000","last_unstale":"2024-09-21T07:44:56.576660+0000","last_undegraded":"2024-09-21T07:44:56.576660+0000","last_fullsized":"2024-09-21T07:44:56.576660+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:48.999240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409744+0000","last_change":"2024-09-21T07:44:54.579096+0000","last_active":"2024-09-21T07:44:57.409744+0000","last_peered":"2024-09-21T07:44:57.409744+0000","last_clean":"2024-09-21T07:44:57.409744+0000","last_became_active":"2024-09-21T07:44:54.578668+0000","last_became_peered":"2024-09-21T07:44:54.578668+0000","last_unstale":"2024-09-21T07:44:57.409744+0000","last_undegraded":"2024-09-21T07:44:57.409744+0000","last_fullsized":"2024-09-21T07:44:57.409744+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:36:50.194028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":150,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576044+0000","last_change":"2024-09-21T07:35:27.864458+0000","last_active":"2024-09-21T07:44:56.576044+0000","last_peered":"2024-09-21T07:44:56.576044+0000","last_clean":"2024-09-21T07:44:56.576044+0000","last_became_active":"2024-09-21T07:35:27.864136+0000","last_became_peered":"2024-09-21T07:35:27.864136+0000","last_unstale":"2024-09-21T07:44:56.576044+0000","last_undegraded":"2024-09-21T07:44:56.576044+0000","last_fullsized":"2024-09-21T07:44:56.576044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:20.162240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":156,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576292+0000","last_change":"2024-09-21T07:35:28.858784+0000","last_active":"2024-09-21T07:44:56.576292+0000","last_peered":"2024-09-21T07:44:56.576292+0000","last_clean":"2024-09-21T07:44:56.576292+0000","last_became_active":"2024-09-21T07:35:26.876890+0000","last_became_peered":"2024-09-21T07:35:26.876890+0000","last_unstale":"2024-09-21T07:44:56.576292+0000","last_undegraded":"2024-09-21T07:44:56.576292+0000","last_fullsized":"2024-09-21T07:44:56.576292+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:40.738327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064543499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":205,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.567319+0000","last_change":"2024-09-21T07:34:24.258712+0000","last_active":"2024-09-21T07:44:55.567319+0000","last_peered":"2024-09-21T07:44:55.567319+0000","last_clean":"2024-09-21T07:44:55.567319+0000","last_became_active":"2024-09-21T07:34:23.274692+0000","last_became_peered":"2024-09-21T07:34:23.274692+0000","last_unstale":"2024-09-21T07:44:55.567319+0000","last_undegraded":"2024-09-21T07:44:55.567319+0000","last_fullsized":"2024-09-21T07:44:55.567319+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:43.279454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018051479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576870+0000","last_change":"2024-09-21T07:44:55.561016+0000","last_active":"2024-09-21T07:44:56.576870+0000","last_peered":"2024-09-21T07:44:56.576870+0000","last_clean":"2024-09-21T07:44:56.576870+0000","last_became_active":"2024-09-21T07:44:55.560410+0000","last_became_peered":"2024-09-21T07:44:55.560410+0000","last_unstale":"2024-09-21T07:44:56.576870+0000","last_undegraded":"2024-09-21T07:44:56.576870+0000","last_fullsized":"2024-09-21T07:44:56.576870+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:51.565447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"133'1","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576017+0000","last_change":"2024-09-21T07:44:54.575744+0000","last_active":"2024-09-21T07:44:56.576017+0000","last_peered":"2024-09-21T07:44:56.576017+0000","last_clean":"2024-09-21T07:44:56.576017+0000","last_became_active":"2024-09-21T07:44:54.574915+0000","last_became_peered":"2024-09-21T07:44:54.574915+0000","last_unstale":"2024-09-21T07:44:56.576017+0000","last_undegraded":"2024-09-21T07:44:56.576017+0000","last_fullsized":"2024-09-21T07:44:56.576017+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:22:22.749559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":144,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.565382+0000","last_change":"2024-09-21T07:35:27.878020+0000","last_active":"2024-09-21T07:44:55.565382+0000","last_peered":"2024-09-21T07:44:55.565382+0000","last_clean":"2024-09-21T07:44:55.565382+0000","last_became_active":"2024-09-21T07:35:27.877532+0000","last_became_peered":"2024-09-21T07:35:27.877532+0000","last_unstale":"2024-09-21T07:44:55.565382+0000","last_undegraded":"2024-09-21T07:44:55.565382+0000","last_fullsized":"2024-09-21T07:44:55.565382+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:19.756073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":153,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.925641+0000","last_change":"2024-09-21T07:35:28.861049+0000","last_active":"2024-09-21T07:44:56.925641+0000","last_peered":"2024-09-21T07:44:56.925641+0000","last_clean":"2024-09-21T07:44:56.925641+0000","last_became_active":"2024-09-21T07:35:26.876819+0000","last_became_peered":"2024-09-21T07:35:26.876819+0000","last_unstale":"2024-09-21T07:44:56.925641+0000","last_undegraded":"2024-09-21T07:44:56.925641+0000","last_fullsized":"2024-09-21T07:44:56.925641+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:19.633803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055780599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":205,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409032+0000","last_change":"2024-09-21T07:34:25.084849+0000","last_active":"2024-09-21T07:44:57.409032+0000","last_peered":"2024-09-21T07:44:57.409032+0000","last_clean":"2024-09-21T07:44:57.409032+0000","last_became_active":"2024-09-21T07:34:23.277713+0000","last_became_peered":"2024-09-21T07:34:23.277713+0000","last_unstale":"2024-09-21T07:44:57.409032+0000","last_undegraded":"2024-09-21T07:44:57.409032+0000","last_fullsized":"2024-09-21T07:44:57.409032+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:05.811810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059079499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.578298+0000","last_change":"2024-09-21T07:44:55.593875+0000","last_active":"2024-09-21T07:44:56.578298+0000","last_peered":"2024-09-21T07:44:56.578298+0000","last_clean":"2024-09-21T07:44:56.578298+0000","last_became_active":"2024-09-21T07:44:55.593403+0000","last_became_peered":"2024-09-21T07:44:55.593403+0000","last_unstale":"2024-09-21T07:44:56.578298+0000","last_undegraded":"2024-09-21T07:44:56.578298+0000","last_fullsized":"2024-09-21T07:44:56.578298+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:51.691583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"133'2","reported_seq":17,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.925831+0000","last_change":"2024-09-21T07:44:54.571773+0000","last_active":"2024-09-21T07:44:56.925831+0000","last_peered":"2024-09-21T07:44:56.925831+0000","last_clean":"2024-09-21T07:44:56.925831+0000","last_became_active":"2024-09-21T07:44:54.571422+0000","last_became_peered":"2024-09-21T07:44:54.571422+0000","last_unstale":"2024-09-21T07:44:56.925831+0000","last_undegraded":"2024-09-21T07:44:56.925831+0000","last_fullsized":"2024-09-21T07:44:56.925831+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:14:01.746987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":148,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.575055+0000","last_change":"2024-09-21T07:35:27.884248+0000","last_active":"2024-09-21T07:44:56.575055+0000","last_peered":"2024-09-21T07:44:56.575055+0000","last_clean":"2024-09-21T07:44:56.575055+0000","last_became_active":"2024-09-21T07:35:27.883861+0000","last_became_peered":"2024-09-21T07:35:27.883861+0000","last_unstale":"2024-09-21T07:44:56.575055+0000","last_undegraded":"2024-09-21T07:44:56.575055+0000","last_fullsized":"2024-09-21T07:44:56.575055+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":153,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409424+0000","last_change":"2024-09-21T07:35:29.207135+0000","last_active":"2024-09-21T07:44:57.409424+0000","last_peered":"2024-09-21T07:44:57.409424+0000","last_clean":"2024-09-21T07:44:57.409424+0000","last_became_active":"2024-09-21T07:35:26.872787+0000","last_became_peered":"2024-09-21T07:35:26.872787+0000","last_unstale":"2024-09-21T07:44:57.409424+0000","last_undegraded":"2024-09-21T07:44:57.409424+0000","last_fullsized":"2024-09-21T07:44:57.409424+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:27.543217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011157509999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":209,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576158+0000","last_change":"2024-09-21T07:34:24.538610+0000","last_active":"2024-09-21T07:44:56.576158+0000","last_peered":"2024-09-21T07:44:56.576158+0000","last_clean":"2024-09-21T07:44:56.576158+0000","last_became_active":"2024-09-21T07:34:23.259700+0000","last_became_peered":"2024-09-21T07:34:23.259700+0000","last_unstale":"2024-09-21T07:44:56.576158+0000","last_undegraded":"2024-09-21T07:44:56.576158+0000","last_fullsized":"2024-09-21T07:44:56.576158+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:57.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070522399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.596780+0000","last_change":"2024-09-21T07:44:55.582701+0000","last_active":"2024-09-21T07:44:55.596780+0000","last_peered":"2024-09-21T07:44:55.596780+0000","last_clean":"2024-09-21T07:44:55.596780+0000","last_became_active":"2024-09-21T07:44:55.581337+0000","last_became_peered":"2024-09-21T07:44:55.581337+0000","last_unstale":"2024-09-21T07:44:55.596780+0000","last_undegraded":"2024-09-21T07:44:55.596780+0000","last_fullsized":"2024-09-21T07:44:55.596780+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:12:58.407288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.571597+0000","last_change":"2024-09-21T07:44:54.578278+0000","last_active":"2024-09-21T07:44:55.571597+0000","last_peered":"2024-09-21T07:44:55.571597+0000","last_clean":"2024-09-21T07:44:55.571597+0000","last_became_active":"2024-09-21T07:44:54.577982+0000","last_became_peered":"2024-09-21T07:44:54.577982+0000","last_unstale":"2024-09-21T07:44:55.571597+0000","last_undegraded":"2024-09-21T07:44:55.571597+0000","last_fullsized":"2024-09-21T07:44:55.571597+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:26.299821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":150,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576953+0000","last_change":"2024-09-21T07:35:27.868024+0000","last_active":"2024-09-21T07:44:56.576953+0000","last_peered":"2024-09-21T07:44:56.576953+0000","last_clean":"2024-09-21T07:44:56.576953+0000","last_became_active":"2024-09-21T07:35:27.867339+0000","last_became_peered":"2024-09-21T07:35:27.867339+0000","last_unstale":"2024-09-21T07:44:56.576953+0000","last_undegraded":"2024-09-21T07:44:56.576953+0000","last_fullsized":"2024-09-21T07:44:56.576953+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":157,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576079+0000","last_change":"2024-09-21T07:35:28.858505+0000","last_active":"2024-09-21T07:44:56.576079+0000","last_peered":"2024-09-21T07:44:56.576079+0000","last_clean":"2024-09-21T07:44:56.576079+0000","last_became_active":"2024-09-21T07:35:26.877320+0000","last_became_peered":"2024-09-21T07:35:26.877320+0000","last_unstale":"2024-09-21T07:44:56.576079+0000","last_undegraded":"2024-09-21T07:44:56.576079+0000","last_fullsized":"2024-09-21T07:44:56.576079+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:53.672782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244119999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":202,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.578434+0000","last_change":"2024-09-21T07:34:24.597597+0000","last_active":"2024-09-21T07:44:56.578434+0000","last_peered":"2024-09-21T07:44:56.578434+0000","last_clean":"2024-09-21T07:44:56.578434+0000","last_became_active":"2024-09-21T07:34:23.635204+0000","last_became_peered":"2024-09-21T07:34:23.635204+0000","last_unstale":"2024-09-21T07:44:56.578434+0000","last_undegraded":"2024-09-21T07:44:56.578434+0000","last_fullsized":"2024-09-21T07:44:56.578434+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:48.644789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081139199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576000+0000","last_change":"2024-09-21T07:44:55.574364+0000","last_active":"2024-09-21T07:44:56.576000+0000","last_peered":"2024-09-21T07:44:56.576000+0000","last_clean":"2024-09-21T07:44:56.576000+0000","last_became_active":"2024-09-21T07:44:55.574006+0000","last_became_peered":"2024-09-21T07:44:55.574006+0000","last_unstale":"2024-09-21T07:44:56.576000+0000","last_undegraded":"2024-09-21T07:44:56.576000+0000","last_fullsized":"2024-09-21T07:44:56.576000+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:30.876997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.407975+0000","last_change":"2024-09-21T07:44:54.571003+0000","last_active":"2024-09-21T07:44:57.407975+0000","last_peered":"2024-09-21T07:44:57.407975+0000","last_clean":"2024-09-21T07:44:57.407975+0000","last_became_active":"2024-09-21T07:44:54.569949+0000","last_became_peered":"2024-09-21T07:44:54.569949+0000","last_unstale":"2024-09-21T07:44:57.407975+0000","last_undegraded":"2024-09-21T07:44:57.407975+0000","last_fullsized":"2024-09-21T07:44:57.407975+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:54:06.235195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":145,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409028+0000","last_change":"2024-09-21T07:35:27.877503+0000","last_active":"2024-09-21T07:44:57.409028+0000","last_peered":"2024-09-21T07:44:57.409028+0000","last_clean":"2024-09-21T07:44:57.409028+0000","last_became_active":"2024-09-21T07:35:27.875775+0000","last_became_peered":"2024-09-21T07:35:27.875775+0000","last_unstale":"2024-09-21T07:44:57.409028+0000","last_undegraded":"2024-09-21T07:44:57.409028+0000","last_fullsized":"2024-09-21T07:44:57.409028+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:49.291023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":154,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409042+0000","last_change":"2024-09-21T07:35:29.491412+0000","last_active":"2024-09-21T07:44:57.409042+0000","last_peered":"2024-09-21T07:44:57.409042+0000","last_clean":"2024-09-21T07:44:57.409042+0000","last_became_active":"2024-09-21T07:35:26.876404+0000","last_became_peered":"2024-09-21T07:35:26.876404+0000","last_unstale":"2024-09-21T07:44:57.409042+0000","last_undegraded":"2024-09-21T07:44:57.409042+0000","last_fullsized":"2024-09-21T07:44:57.409042+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063918000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":206,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.574970+0000","last_change":"2024-09-21T07:34:24.259026+0000","last_active":"2024-09-21T07:44:55.574970+0000","last_peered":"2024-09-21T07:44:55.574970+0000","last_clean":"2024-09-21T07:44:55.574970+0000","last_became_active":"2024-09-21T07:34:23.635456+0000","last_became_peered":"2024-09-21T07:34:23.635456+0000","last_unstale":"2024-09-21T07:44:55.574970+0000","last_undegraded":"2024-09-21T07:44:55.574970+0000","last_fullsized":"2024-09-21T07:44:55.574970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:09.592752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018654419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.408788+0000","last_change":"2024-09-21T07:44:55.569794+0000","last_active":"2024-09-21T07:44:57.408788+0000","last_peered":"2024-09-21T07:44:57.408788+0000","last_clean":"2024-09-21T07:44:57.408788+0000","last_became_active":"2024-09-21T07:44:55.568883+0000","last_became_peered":"2024-09-21T07:44:55.568883+0000","last_unstale":"2024-09-21T07:44:57.408788+0000","last_undegraded":"2024-09-21T07:44:57.408788+0000","last_fullsized":"2024-09-21T07:44:57.408788+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:01:20.752577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"133'2","reported_seq":17,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410005+0000","last_change":"2024-09-21T07:44:54.570827+0000","last_active":"2024-09-21T07:44:57.410005+0000","last_peered":"2024-09-21T07:44:57.410005+0000","last_clean":"2024-09-21T07:44:57.410005+0000","last_became_active":"2024-09-21T07:44:54.570516+0000","last_became_peered":"2024-09-21T07:44:54.570516+0000","last_unstale":"2024-09-21T07:44:57.410005+0000","last_undegraded":"2024-09-21T07:44:57.410005+0000","last_fullsized":"2024-09-21T07:44:57.410005+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:50.388307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":145,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410202+0000","last_change":"2024-09-21T07:35:27.871984+0000","last_active":"2024-09-21T07:44:57.410202+0000","last_peered":"2024-09-21T07:44:57.410202+0000","last_clean":"2024-09-21T07:44:57.410202+0000","last_became_active":"2024-09-21T07:35:27.870336+0000","last_became_peered":"2024-09-21T07:35:27.870336+0000","last_unstale":"2024-09-21T07:44:57.410202+0000","last_undegraded":"2024-09-21T07:44:57.410202+0000","last_fullsized":"2024-09-21T07:44:57.410202+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:57.120479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":154,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.925589+0000","last_change":"2024-09-21T07:35:28.860351+0000","last_active":"2024-09-21T07:44:56.925589+0000","last_peered":"2024-09-21T07:44:56.925589+0000","last_clean":"2024-09-21T07:44:56.925589+0000","last_became_active":"2024-09-21T07:35:26.872046+0000","last_became_peered":"2024-09-21T07:35:26.872046+0000","last_unstale":"2024-09-21T07:44:56.925589+0000","last_undegraded":"2024-09-21T07:44:56.925589+0000","last_fullsized":"2024-09-21T07:44:56.925589+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:47.074327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098384399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":207,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410020+0000","last_change":"2024-09-21T07:34:24.260300+0000","last_active":"2024-09-21T07:44:57.410020+0000","last_peered":"2024-09-21T07:44:57.410020+0000","last_clean":"2024-09-21T07:44:57.410020+0000","last_became_active":"2024-09-21T07:34:23.270128+0000","last_became_peered":"2024-09-21T07:34:23.270128+0000","last_unstale":"2024-09-21T07:44:57.410020+0000","last_undegraded":"2024-09-21T07:44:57.410020+0000","last_fullsized":"2024-09-21T07:44:57.410020+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:25.260931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00176651,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.598052+0000","last_change":"2024-09-21T07:44:55.589128+0000","last_active":"2024-09-21T07:44:55.598052+0000","last_peered":"2024-09-21T07:44:55.598052+0000","last_clean":"2024-09-21T07:44:55.598052+0000","last_became_active":"2024-09-21T07:44:55.582134+0000","last_became_peered":"2024-09-21T07:44:55.582134+0000","last_unstale":"2024-09-21T07:44:55.598052+0000","last_undegraded":"2024-09-21T07:44:55.598052+0000","last_fullsized":"2024-09-21T07:44:55.598052+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:26:27.669272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.408461+0000","last_change":"2024-09-21T07:44:54.571619+0000","last_active":"2024-09-21T07:44:57.408461+0000","last_peered":"2024-09-21T07:44:57.408461+0000","last_clean":"2024-09-21T07:44:57.408461+0000","last_became_active":"2024-09-21T07:44:54.571116+0000","last_became_peered":"2024-09-21T07:44:54.571116+0000","last_unstale":"2024-09-21T07:44:57.408461+0000","last_undegraded":"2024-09-21T07:44:57.408461+0000","last_fullsized":"2024-09-21T07:44:57.408461+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:53.905447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":145,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409831+0000","last_change":"2024-09-21T07:35:27.877861+0000","last_active":"2024-09-21T07:44:57.409831+0000","last_peered":"2024-09-21T07:44:57.409831+0000","last_clean":"2024-09-21T07:44:57.409831+0000","last_became_active":"2024-09-21T07:35:27.877434+0000","last_became_peered":"2024-09-21T07:35:27.877434+0000","last_unstale":"2024-09-21T07:44:57.409831+0000","last_undegraded":"2024-09-21T07:44:57.409831+0000","last_fullsized":"2024-09-21T07:44:57.409831+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":154,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409508+0000","last_change":"2024-09-21T07:35:28.862954+0000","last_active":"2024-09-21T07:44:57.409508+0000","last_peered":"2024-09-21T07:44:57.409508+0000","last_clean":"2024-09-21T07:44:57.409508+0000","last_became_active":"2024-09-21T07:35:26.870613+0000","last_became_peered":"2024-09-21T07:35:26.870613+0000","last_unstale":"2024-09-21T07:44:57.409508+0000","last_undegraded":"2024-09-21T07:44:57.409508+0000","last_fullsized":"2024-09-21T07:44:57.409508+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:06.901043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059842100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":206,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.575493+0000","last_change":"2024-09-21T07:34:25.028904+0000","last_active":"2024-09-21T07:44:56.575493+0000","last_peered":"2024-09-21T07:44:56.575493+0000","last_clean":"2024-09-21T07:44:56.575493+0000","last_became_active":"2024-09-21T07:34:23.260942+0000","last_became_peered":"2024-09-21T07:34:23.260942+0000","last_unstale":"2024-09-21T07:44:56.575493+0000","last_undegraded":"2024-09-21T07:44:56.575493+0000","last_fullsized":"2024-09-21T07:44:56.575493+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:22.709328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060553899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509586,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33420,"kb_used_data":2228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338420,"statfs":{"total":96636764160,"available":96602542080,"internally_reserved":0,"allocated":2281472,"data_stored":1454525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509588,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32820,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339020,"statfs":{"total":96636764160,"available":96603156480,"internally_reserved":0,"allocated":1662976,"data_stored":856572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509586,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32836,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339004,"statfs":{"total":96636764160,"available":96603140096,"internally_reserved":0,"allocated":1679360,"data_stored":863541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509587,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32816,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339024,"statfs":{"total":96636764160,"available":96603160576,"internally_reserved":0,"allocated":1658880,"data_stored":855990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509588,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32832,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339008,"statfs":{"total":96636764160,"available":96603144192,"internally_reserved":0,"allocated":1675264,"data_stored":863403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509588,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33400,"kb_used_data":2204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338440,"statfs":{"total":96636764160,"available":96602562560,"internally_reserved":0,"allocated":2256896,"data_stored":1446380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542291,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32828,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339012,"statfs":{"total":96636764160,"available":96603148288,"internally_reserved":0,"allocated":1671168,"data_stored":857602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542290,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32824,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339016,"statfs":{"total":96636764160,"available":96603152384,"internally_reserved":0,"allocated":1675264,"data_stored":857636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:45:00.078 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:45:00.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.336+0000 7fbbed600640 1 -- 172.21.15.39:0/1451201207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbe80b30a0 msgr2=0x7fbbe80cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:00.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.336+0000 7fbbed600640 1 --2- 172.21.15.39:0/1451201207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbe80b30a0 0x7fbbe80cb8a0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8075160 tx=0x7fbbd802a560 comp rx=0 tx=0).stop 2024-09-21T07:45:00.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.336+0000 7fbbed600640 1 -- 172.21.15.39:0/1451201207 shutdown_connections 2024-09-21T07:45:00.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.336+0000 7fbbed600640 1 --2- 172.21.15.39:0/1451201207 >> v2:172.21.15.148:3300/0 conn(0x7fbbe815d830 0x7fbbe8156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:00.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.336+0000 7fbbed600640 1 --2- 172.21.15.39:0/1451201207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbe80b30a0 0x7fbbe80cb8a0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:00.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.336+0000 7fbbed600640 1 -- 172.21.15.39:0/1451201207 >> 172.21.15.39:0/1451201207 conn(0x7fbbe80b2460 msgr2=0x7fbbe806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:00.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.337+0000 7fbbed600640 1 -- 172.21.15.39:0/1451201207 shutdown_connections 2024-09-21T07:45:00.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.337+0000 7fbbed600640 1 -- 172.21.15.39:0/1451201207 wait complete. 2024-09-21T07:45:00.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.339+0000 7fbbed600640 1 Processor -- start 2024-09-21T07:45:00.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.339+0000 7fbbed600640 1 -- start start 2024-09-21T07:45:00.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.340+0000 7fbbed600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbe80b30a0 0x7fbbe81f58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:00.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.341+0000 7fbbed600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbe80ccc60 0x7fbbe80c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:00.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.341+0000 7fbbed600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbbe815d830 0x7fbbe80c6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:00.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.341+0000 7fbbed600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe80af410 con 0x7fbbe80ccc60 2024-09-21T07:45:00.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.341+0000 7fbbed600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe80af290 con 0x7fbbe80b30a0 2024-09-21T07:45:00.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.341+0000 7fbbed600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fbbe80af590 con 0x7fbbe815d830 2024-09-21T07:45:00.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.341+0000 7fbbe6000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbe80ccc60 0x7fbbe80c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:00.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.341+0000 7fbbe6000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbe80ccc60 0x7fbbe80c5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48028/0 (socket says 172.21.15.39:48028) 2024-09-21T07:45:00.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.341+0000 7fbbe6000640 1 -- 172.21.15.39:0/1263993777 learned_addr learned my addr 172.21.15.39:0/1263993777 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:00.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.341+0000 7fbbe6a00640 1 --2- 172.21.15.39:0/1263993777 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbe80b30a0 0x7fbbe81f58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:00.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.341+0000 7fbbe7400640 1 --2- 172.21.15.39:0/1263993777 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbbe815d830 0x7fbbe80c6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:00.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.341+0000 7fbbe6000640 1 -- 172.21.15.39:0/1263993777 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbbe815d830 msgr2=0x7fbbe80c6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:00.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.341+0000 7fbbe6000640 1 --2- 172.21.15.39:0/1263993777 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbbe815d830 0x7fbbe80c6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:00.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.341+0000 7fbbe6000640 1 -- 172.21.15.39:0/1263993777 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbe80b30a0 msgr2=0x7fbbe81f58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:00.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.341+0000 7fbbe6000640 1 --2- 172.21.15.39:0/1263993777 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbe80b30a0 0x7fbbe81f58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:00.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.341+0000 7fbbe6000640 1 -- 172.21.15.39:0/1263993777 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe80b1970 con 0x7fbbe80ccc60 2024-09-21T07:45:00.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.341+0000 7fbbe7400640 1 --2- 172.21.15.39:0/1263993777 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbbe815d830 0x7fbbe80c6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:45:00.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.342+0000 7fbbe6000640 1 --2- 172.21.15.39:0/1263993777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbe80ccc60 0x7fbbe80c5dc0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fbbd400e7c0 tx=0x7fbbd400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:00.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.342+0000 7fbbdea00640 1 -- 172.21.15.39:0/1263993777 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd400ccc0 con 0x7fbbe80ccc60 2024-09-21T07:45:00.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.342+0000 7fbbdea00640 1 -- 172.21.15.39:0/1263993777 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbbd4010430 con 0x7fbbe80ccc60 2024-09-21T07:45:00.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.342+0000 7fbbdea00640 1 -- 172.21.15.39:0/1263993777 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4010860 con 0x7fbbe80ccc60 2024-09-21T07:45:00.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.342+0000 7fbbed600640 1 -- 172.21.15.39:0/1263993777 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbe80c4410 con 0x7fbbe80ccc60 2024-09-21T07:45:00.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.342+0000 7fbbed600640 1 -- 172.21.15.39:0/1263993777 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbe80c49d0 con 0x7fbbe80ccc60 2024-09-21T07:45:00.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.343+0000 7fbbed600640 1 -- 172.21.15.39:0/1263993777 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbe80c4180 con 0x7fbbe80ccc60 2024-09-21T07:45:00.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.344+0000 7fbbdea00640 1 -- 172.21.15.39:0/1263993777 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbbd4010a00 con 0x7fbbe80ccc60 2024-09-21T07:45:00.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.346+0000 7fbbdea00640 1 --2- 172.21.15.39:0/1263993777 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbbc8078bc0 0x7fbbc807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:00.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.346+0000 7fbbdea00640 1 -- 172.21.15.39:0/1263993777 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 6800+0+0 (secure 0 0 0) 0x7fbbd409bbc0 con 0x7fbbe80ccc60 2024-09-21T07:45:00.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.347+0000 7fbbe6a00640 1 --2- 172.21.15.39:0/1263993777 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbbc8078bc0 0x7fbbc807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:00.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.347+0000 7fbbe6a00640 1 --2- 172.21.15.39:0/1263993777 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbbc8078bc0 0x7fbbc807b0d0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8033000 tx=0x7fbbd8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:00.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.351+0000 7fbbdea00640 1 -- 172.21.15.39:0/1263993777 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbbd4063b40 con 0x7fbbe80ccc60 2024-09-21T07:45:00.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.544+0000 7fbbed600640 1 -- 172.21.15.39:0/1263993777 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbbe80700b0 con 0x7fbbc8078bc0 2024-09-21T07:45:00.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.553+0000 7fbbdea00640 1 -- 172.21.15.39:0/1263993777 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+495241 (secure 0 0 0) 0x7fbbe80700b0 con 0x7fbbc8078bc0 2024-09-21T07:45:00.561 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:45:00.564 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:45:00.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.559+0000 7fbbed600640 1 -- 172.21.15.39:0/1263993777 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbbc8078bc0 msgr2=0x7fbbc807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:00.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.559+0000 7fbbed600640 1 --2- 172.21.15.39:0/1263993777 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbbc8078bc0 0x7fbbc807b0d0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8033000 tx=0x7fbbd8004730 comp rx=0 tx=0).stop 2024-09-21T07:45:00.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.559+0000 7fbbed600640 1 -- 172.21.15.39:0/1263993777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbe80ccc60 msgr2=0x7fbbe80c5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:00.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.559+0000 7fbbed600640 1 --2- 172.21.15.39:0/1263993777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbe80ccc60 0x7fbbe80c5dc0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fbbd400e7c0 tx=0x7fbbd400ec80 comp rx=0 tx=0).stop 2024-09-21T07:45:00.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.559+0000 7fbbed600640 1 -- 172.21.15.39:0/1263993777 shutdown_connections 2024-09-21T07:45:00.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.559+0000 7fbbed600640 1 --2- 172.21.15.39:0/1263993777 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbbc8078bc0 0x7fbbc807b0d0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:00.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.559+0000 7fbbed600640 1 --2- 172.21.15.39:0/1263993777 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbbe815d830 0x7fbbe80c6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:00.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.559+0000 7fbbed600640 1 --2- 172.21.15.39:0/1263993777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbe80ccc60 0x7fbbe80c5dc0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:00.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.559+0000 7fbbed600640 1 --2- 172.21.15.39:0/1263993777 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbe80b30a0 0x7fbbe81f58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:00.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.559+0000 7fbbed600640 1 -- 172.21.15.39:0/1263993777 >> 172.21.15.39:0/1263993777 conn(0x7fbbe80b2460 msgr2=0x7fbbe806f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:00.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.560+0000 7fbbed600640 1 -- 172.21.15.39:0/1263993777 shutdown_connections 2024-09-21T07:45:00.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.560+0000 7fbbed600640 1 -- 172.21.15.39:0/1263993777 wait complete. 2024-09-21T07:45:00.587 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":488,"stamp":"2024-09-21T07:44:59.773886+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":356,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":263776,"kb_used_data":14220,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754710944,"statfs":{"total":773094113280,"available":772824006656,"internally_reserved":0,"allocated":14561280,"data_stored":8055649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001933"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.578662+0000","last_change":"2024-09-21T07:44:55.577330+0000","last_active":"2024-09-21T07:44:56.578662+0000","last_peered":"2024-09-21T07:44:56.578662+0000","last_clean":"2024-09-21T07:44:56.578662+0000","last_became_active":"2024-09-21T07:44:55.576951+0000","last_became_peered":"2024-09-21T07:44:55.576951+0000","last_unstale":"2024-09-21T07:44:56.578662+0000","last_undegraded":"2024-09-21T07:44:56.578662+0000","last_fullsized":"2024-09-21T07:44:56.578662+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:48.641042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":10,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.599252+0000","last_change":"2024-09-21T07:44:55.597295+0000","last_active":"2024-09-21T07:44:55.599252+0000","last_peered":"2024-09-21T07:44:55.599252+0000","last_clean":"2024-09-21T07:44:55.599252+0000","last_became_active":"2024-09-21T07:44:55.596987+0000","last_became_peered":"2024-09-21T07:44:55.596987+0000","last_unstale":"2024-09-21T07:44:55.599252+0000","last_undegraded":"2024-09-21T07:44:55.599252+0000","last_fullsized":"2024-09-21T07:44:55.599252+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:03.040762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":10,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.596730+0000","last_change":"2024-09-21T07:44:55.583357+0000","last_active":"2024-09-21T07:44:55.596730+0000","last_peered":"2024-09-21T07:44:55.596730+0000","last_clean":"2024-09-21T07:44:55.596730+0000","last_became_active":"2024-09-21T07:44:55.583061+0000","last_became_peered":"2024-09-21T07:44:55.583061+0000","last_unstale":"2024-09-21T07:44:55.596730+0000","last_undegraded":"2024-09-21T07:44:55.596730+0000","last_fullsized":"2024-09-21T07:44:55.596730+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:25:24.765059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410297+0000","last_change":"2024-09-21T07:44:55.589391+0000","last_active":"2024-09-21T07:44:57.410297+0000","last_peered":"2024-09-21T07:44:57.410297+0000","last_clean":"2024-09-21T07:44:57.410297+0000","last_became_active":"2024-09-21T07:44:55.586615+0000","last_became_peered":"2024-09-21T07:44:55.586615+0000","last_unstale":"2024-09-21T07:44:57.410297+0000","last_undegraded":"2024-09-21T07:44:57.410297+0000","last_fullsized":"2024-09-21T07:44:57.410297+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:57:30.533423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576766+0000","last_change":"2024-09-21T07:44:55.582437+0000","last_active":"2024-09-21T07:44:56.576766+0000","last_peered":"2024-09-21T07:44:56.576766+0000","last_clean":"2024-09-21T07:44:56.576766+0000","last_became_active":"2024-09-21T07:44:55.580896+0000","last_became_peered":"2024-09-21T07:44:55.580896+0000","last_unstale":"2024-09-21T07:44:56.576766+0000","last_undegraded":"2024-09-21T07:44:56.576766+0000","last_fullsized":"2024-09-21T07:44:56.576766+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:01.412902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410284+0000","last_change":"2024-09-21T07:44:55.588345+0000","last_active":"2024-09-21T07:44:57.410284+0000","last_peered":"2024-09-21T07:44:57.410284+0000","last_clean":"2024-09-21T07:44:57.410284+0000","last_became_active":"2024-09-21T07:44:55.588019+0000","last_became_peered":"2024-09-21T07:44:55.588019+0000","last_unstale":"2024-09-21T07:44:57.410284+0000","last_undegraded":"2024-09-21T07:44:57.410284+0000","last_fullsized":"2024-09-21T07:44:57.410284+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:57.925520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409212+0000","last_change":"2024-09-21T07:44:55.589748+0000","last_active":"2024-09-21T07:44:57.409212+0000","last_peered":"2024-09-21T07:44:57.409212+0000","last_clean":"2024-09-21T07:44:57.409212+0000","last_became_active":"2024-09-21T07:44:55.586211+0000","last_became_peered":"2024-09-21T07:44:55.586211+0000","last_unstale":"2024-09-21T07:44:57.409212+0000","last_undegraded":"2024-09-21T07:44:57.409212+0000","last_fullsized":"2024-09-21T07:44:57.409212+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:42.462390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":10,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.598957+0000","last_change":"2024-09-21T07:44:55.575890+0000","last_active":"2024-09-21T07:44:55.598957+0000","last_peered":"2024-09-21T07:44:55.598957+0000","last_clean":"2024-09-21T07:44:55.598957+0000","last_became_active":"2024-09-21T07:44:55.575594+0000","last_became_peered":"2024-09-21T07:44:55.575594+0000","last_unstale":"2024-09-21T07:44:55.598957+0000","last_undegraded":"2024-09-21T07:44:55.598957+0000","last_fullsized":"2024-09-21T07:44:55.598957+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:09:15.363621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410541+0000","last_change":"2024-09-21T07:44:55.594003+0000","last_active":"2024-09-21T07:44:57.410541+0000","last_peered":"2024-09-21T07:44:57.410541+0000","last_clean":"2024-09-21T07:44:57.410541+0000","last_became_active":"2024-09-21T07:44:55.593315+0000","last_became_peered":"2024-09-21T07:44:55.593315+0000","last_unstale":"2024-09-21T07:44:57.410541+0000","last_undegraded":"2024-09-21T07:44:57.410541+0000","last_fullsized":"2024-09-21T07:44:57.410541+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:36:16.134960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.575833+0000","last_change":"2024-09-21T07:44:55.590261+0000","last_active":"2024-09-21T07:44:56.575833+0000","last_peered":"2024-09-21T07:44:56.575833+0000","last_clean":"2024-09-21T07:44:56.575833+0000","last_became_active":"2024-09-21T07:44:55.589848+0000","last_became_peered":"2024-09-21T07:44:55.589848+0000","last_unstale":"2024-09-21T07:44:56.575833+0000","last_undegraded":"2024-09-21T07:44:56.575833+0000","last_fullsized":"2024-09-21T07:44:56.575833+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:04:55.654398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.578247+0000","last_change":"2024-09-21T07:44:55.575847+0000","last_active":"2024-09-21T07:44:56.578247+0000","last_peered":"2024-09-21T07:44:56.578247+0000","last_clean":"2024-09-21T07:44:56.578247+0000","last_became_active":"2024-09-21T07:44:55.575524+0000","last_became_peered":"2024-09-21T07:44:55.575524+0000","last_unstale":"2024-09-21T07:44:56.578247+0000","last_undegraded":"2024-09-21T07:44:56.578247+0000","last_fullsized":"2024-09-21T07:44:56.578247+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:16:24.610724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410779+0000","last_change":"2024-09-21T07:44:55.576749+0000","last_active":"2024-09-21T07:44:57.410779+0000","last_peered":"2024-09-21T07:44:57.410779+0000","last_clean":"2024-09-21T07:44:57.410779+0000","last_became_active":"2024-09-21T07:44:55.576450+0000","last_became_peered":"2024-09-21T07:44:55.576450+0000","last_unstale":"2024-09-21T07:44:57.410779+0000","last_undegraded":"2024-09-21T07:44:57.410779+0000","last_fullsized":"2024-09-21T07:44:57.410779+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:17.212037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.574843+0000","last_change":"2024-09-21T07:44:55.586779+0000","last_active":"2024-09-21T07:44:56.574843+0000","last_peered":"2024-09-21T07:44:56.574843+0000","last_clean":"2024-09-21T07:44:56.574843+0000","last_became_active":"2024-09-21T07:44:55.586473+0000","last_became_peered":"2024-09-21T07:44:55.586473+0000","last_unstale":"2024-09-21T07:44:56.574843+0000","last_undegraded":"2024-09-21T07:44:56.574843+0000","last_fullsized":"2024-09-21T07:44:56.574843+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:23.423387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":10,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.597948+0000","last_change":"2024-09-21T07:44:55.587326+0000","last_active":"2024-09-21T07:44:55.597948+0000","last_peered":"2024-09-21T07:44:55.597948+0000","last_clean":"2024-09-21T07:44:55.597948+0000","last_became_active":"2024-09-21T07:44:55.582056+0000","last_became_peered":"2024-09-21T07:44:55.582056+0000","last_unstale":"2024-09-21T07:44:55.597948+0000","last_undegraded":"2024-09-21T07:44:55.597948+0000","last_fullsized":"2024-09-21T07:44:55.597948+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:53.727782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576256+0000","last_change":"2024-09-21T07:44:55.574716+0000","last_active":"2024-09-21T07:44:56.576256+0000","last_peered":"2024-09-21T07:44:56.576256+0000","last_clean":"2024-09-21T07:44:56.576256+0000","last_became_active":"2024-09-21T07:44:55.574412+0000","last_became_peered":"2024-09-21T07:44:55.574412+0000","last_unstale":"2024-09-21T07:44:56.576256+0000","last_undegraded":"2024-09-21T07:44:56.576256+0000","last_fullsized":"2024-09-21T07:44:56.576256+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:59.447663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.408754+0000","last_change":"2024-09-21T07:44:55.575384+0000","last_active":"2024-09-21T07:44:57.408754+0000","last_peered":"2024-09-21T07:44:57.408754+0000","last_clean":"2024-09-21T07:44:57.408754+0000","last_became_active":"2024-09-21T07:44:55.575069+0000","last_became_peered":"2024-09-21T07:44:55.575069+0000","last_unstale":"2024-09-21T07:44:57.408754+0000","last_undegraded":"2024-09-21T07:44:57.408754+0000","last_fullsized":"2024-09-21T07:44:57.408754+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:50.957046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.575891+0000","last_change":"2024-09-21T07:44:55.578638+0000","last_active":"2024-09-21T07:44:56.575891+0000","last_peered":"2024-09-21T07:44:56.575891+0000","last_clean":"2024-09-21T07:44:56.575891+0000","last_became_active":"2024-09-21T07:44:55.578093+0000","last_became_peered":"2024-09-21T07:44:55.578093+0000","last_unstale":"2024-09-21T07:44:56.575891+0000","last_undegraded":"2024-09-21T07:44:56.575891+0000","last_fullsized":"2024-09-21T07:44:56.575891+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:45.442291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":10,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.598751+0000","last_change":"2024-09-21T07:44:55.596708+0000","last_active":"2024-09-21T07:44:55.598751+0000","last_peered":"2024-09-21T07:44:55.598751+0000","last_clean":"2024-09-21T07:44:55.598751+0000","last_became_active":"2024-09-21T07:44:55.596318+0000","last_became_peered":"2024-09-21T07:44:55.596318+0000","last_unstale":"2024-09-21T07:44:55.598751+0000","last_undegraded":"2024-09-21T07:44:55.598751+0000","last_fullsized":"2024-09-21T07:44:55.598751+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:40.500276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.408944+0000","last_change":"2024-09-21T07:44:55.587990+0000","last_active":"2024-09-21T07:44:57.408944+0000","last_peered":"2024-09-21T07:44:57.408944+0000","last_clean":"2024-09-21T07:44:57.408944+0000","last_became_active":"2024-09-21T07:44:55.587629+0000","last_became_peered":"2024-09-21T07:44:55.587629+0000","last_unstale":"2024-09-21T07:44:57.408944+0000","last_undegraded":"2024-09-21T07:44:57.408944+0000","last_fullsized":"2024-09-21T07:44:57.408944+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:36:16.134960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.925524+0000","last_change":"2024-09-21T07:44:55.582492+0000","last_active":"2024-09-21T07:44:56.925524+0000","last_peered":"2024-09-21T07:44:56.925524+0000","last_clean":"2024-09-21T07:44:56.925524+0000","last_became_active":"2024-09-21T07:44:55.580803+0000","last_became_peered":"2024-09-21T07:44:55.580803+0000","last_unstale":"2024-09-21T07:44:56.925524+0000","last_undegraded":"2024-09-21T07:44:56.925524+0000","last_fullsized":"2024-09-21T07:44:56.925524+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:28:20.883491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410061+0000","last_change":"2024-09-21T07:44:55.589685+0000","last_active":"2024-09-21T07:44:57.410061+0000","last_peered":"2024-09-21T07:44:57.410061+0000","last_clean":"2024-09-21T07:44:57.410061+0000","last_became_active":"2024-09-21T07:44:55.589283+0000","last_became_peered":"2024-09-21T07:44:55.589283+0000","last_unstale":"2024-09-21T07:44:57.410061+0000","last_undegraded":"2024-09-21T07:44:57.410061+0000","last_fullsized":"2024-09-21T07:44:57.410061+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:45.442291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.577585+0000","last_change":"2024-09-21T07:44:55.588693+0000","last_active":"2024-09-21T07:44:56.577585+0000","last_peered":"2024-09-21T07:44:56.577585+0000","last_clean":"2024-09-21T07:44:56.577585+0000","last_became_active":"2024-09-21T07:44:55.588331+0000","last_became_peered":"2024-09-21T07:44:55.588331+0000","last_unstale":"2024-09-21T07:44:56.577585+0000","last_undegraded":"2024-09-21T07:44:56.577585+0000","last_fullsized":"2024-09-21T07:44:56.577585+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:03.374280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.408624+0000","last_change":"2024-09-21T07:44:55.589391+0000","last_active":"2024-09-21T07:44:57.408624+0000","last_peered":"2024-09-21T07:44:57.408624+0000","last_clean":"2024-09-21T07:44:57.408624+0000","last_became_active":"2024-09-21T07:44:55.586797+0000","last_became_peered":"2024-09-21T07:44:55.586797+0000","last_unstale":"2024-09-21T07:44:57.408624+0000","last_undegraded":"2024-09-21T07:44:57.408624+0000","last_fullsized":"2024-09-21T07:44:57.408624+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:27.029715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.575550+0000","last_change":"2024-09-21T07:44:55.569104+0000","last_active":"2024-09-21T07:44:56.575550+0000","last_peered":"2024-09-21T07:44:56.575550+0000","last_clean":"2024-09-21T07:44:56.575550+0000","last_became_active":"2024-09-21T07:44:55.568769+0000","last_became_peered":"2024-09-21T07:44:55.568769+0000","last_unstale":"2024-09-21T07:44:56.575550+0000","last_undegraded":"2024-09-21T07:44:56.575550+0000","last_fullsized":"2024-09-21T07:44:56.575550+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:05.902947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.577096+0000","last_change":"2024-09-21T07:44:55.569906+0000","last_active":"2024-09-21T07:44:56.577096+0000","last_peered":"2024-09-21T07:44:56.577096+0000","last_clean":"2024-09-21T07:44:56.577096+0000","last_became_active":"2024-09-21T07:44:55.569584+0000","last_became_peered":"2024-09-21T07:44:55.569584+0000","last_unstale":"2024-09-21T07:44:56.577096+0000","last_undegraded":"2024-09-21T07:44:56.577096+0000","last_fullsized":"2024-09-21T07:44:56.577096+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:39:07.663576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.925994+0000","last_change":"2024-09-21T07:44:55.577844+0000","last_active":"2024-09-21T07:44:56.925994+0000","last_peered":"2024-09-21T07:44:56.925994+0000","last_clean":"2024-09-21T07:44:56.925994+0000","last_became_active":"2024-09-21T07:44:55.577546+0000","last_became_peered":"2024-09-21T07:44:55.577546+0000","last_unstale":"2024-09-21T07:44:56.925994+0000","last_undegraded":"2024-09-21T07:44:56.925994+0000","last_fullsized":"2024-09-21T07:44:56.925994+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:51:45.788835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409555+0000","last_change":"2024-09-21T07:44:55.584249+0000","last_active":"2024-09-21T07:44:57.409555+0000","last_peered":"2024-09-21T07:44:57.409555+0000","last_clean":"2024-09-21T07:44:57.409555+0000","last_became_active":"2024-09-21T07:44:55.583386+0000","last_became_peered":"2024-09-21T07:44:55.583386+0000","last_unstale":"2024-09-21T07:44:57.409555+0000","last_undegraded":"2024-09-21T07:44:57.409555+0000","last_fullsized":"2024-09-21T07:44:57.409555+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:41.343224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410977+0000","last_change":"2024-09-21T07:44:55.580364+0000","last_active":"2024-09-21T07:44:57.410977+0000","last_peered":"2024-09-21T07:44:57.410977+0000","last_clean":"2024-09-21T07:44:57.410977+0000","last_became_active":"2024-09-21T07:44:55.579952+0000","last_became_peered":"2024-09-21T07:44:55.579952+0000","last_unstale":"2024-09-21T07:44:57.410977+0000","last_undegraded":"2024-09-21T07:44:57.410977+0000","last_fullsized":"2024-09-21T07:44:57.410977+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:38.518021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.926207+0000","last_change":"2024-09-21T07:44:55.583118+0000","last_active":"2024-09-21T07:44:56.926207+0000","last_peered":"2024-09-21T07:44:56.926207+0000","last_clean":"2024-09-21T07:44:56.926207+0000","last_became_active":"2024-09-21T07:44:55.582455+0000","last_became_peered":"2024-09-21T07:44:55.582455+0000","last_unstale":"2024-09-21T07:44:56.926207+0000","last_undegraded":"2024-09-21T07:44:56.926207+0000","last_fullsized":"2024-09-21T07:44:56.926207+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:35.307701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":10,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.597773+0000","last_change":"2024-09-21T07:44:55.585979+0000","last_active":"2024-09-21T07:44:55.597773+0000","last_peered":"2024-09-21T07:44:55.597773+0000","last_clean":"2024-09-21T07:44:55.597773+0000","last_became_active":"2024-09-21T07:44:55.583787+0000","last_became_peered":"2024-09-21T07:44:55.583787+0000","last_unstale":"2024-09-21T07:44:55.597773+0000","last_undegraded":"2024-09-21T07:44:55.597773+0000","last_fullsized":"2024-09-21T07:44:55.597773+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:20:07.013117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.575754+0000","last_change":"2024-09-21T07:44:55.576065+0000","last_active":"2024-09-21T07:44:56.575754+0000","last_peered":"2024-09-21T07:44:56.575754+0000","last_clean":"2024-09-21T07:44:56.575754+0000","last_became_active":"2024-09-21T07:44:55.575720+0000","last_became_peered":"2024-09-21T07:44:55.575720+0000","last_unstale":"2024-09-21T07:44:56.575754+0000","last_undegraded":"2024-09-21T07:44:56.575754+0000","last_fullsized":"2024-09-21T07:44:56.575754+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:54.908229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.575818+0000","last_change":"2024-09-21T07:44:55.560991+0000","last_active":"2024-09-21T07:44:56.575818+0000","last_peered":"2024-09-21T07:44:56.575818+0000","last_clean":"2024-09-21T07:44:56.575818+0000","last_became_active":"2024-09-21T07:44:55.560593+0000","last_became_peered":"2024-09-21T07:44:55.560593+0000","last_unstale":"2024-09-21T07:44:56.575818+0000","last_undegraded":"2024-09-21T07:44:56.575818+0000","last_fullsized":"2024-09-21T07:44:56.575818+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:36:51.196810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.408690+0000","last_change":"2024-09-21T07:44:55.588440+0000","last_active":"2024-09-21T07:44:57.408690+0000","last_peered":"2024-09-21T07:44:57.408690+0000","last_clean":"2024-09-21T07:44:57.408690+0000","last_became_active":"2024-09-21T07:44:55.586797+0000","last_became_peered":"2024-09-21T07:44:55.586797+0000","last_unstale":"2024-09-21T07:44:57.408690+0000","last_undegraded":"2024-09-21T07:44:57.408690+0000","last_fullsized":"2024-09-21T07:44:57.408690+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:41.936023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"133'8","reported_seq":21,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.601626+0000","last_change":"2024-09-21T07:44:54.579517+0000","last_active":"2024-09-21T07:44:55.601626+0000","last_peered":"2024-09-21T07:44:55.601626+0000","last_clean":"2024-09-21T07:44:55.601626+0000","last_became_active":"2024-09-21T07:44:54.578700+0000","last_became_peered":"2024-09-21T07:44:54.578700+0000","last_unstale":"2024-09-21T07:44:55.601626+0000","last_undegraded":"2024-09-21T07:44:55.601626+0000","last_fullsized":"2024-09-21T07:44:55.601626+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:36.690522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":147,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410130+0000","last_change":"2024-09-21T07:35:27.879605+0000","last_active":"2024-09-21T07:44:57.410130+0000","last_peered":"2024-09-21T07:44:57.410130+0000","last_clean":"2024-09-21T07:44:57.410130+0000","last_became_active":"2024-09-21T07:35:27.879274+0000","last_became_peered":"2024-09-21T07:35:27.879274+0000","last_unstale":"2024-09-21T07:44:57.410130+0000","last_undegraded":"2024-09-21T07:44:57.410130+0000","last_fullsized":"2024-09-21T07:44:57.410130+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:24.241636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":151,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.565496+0000","last_change":"2024-09-21T07:35:29.491336+0000","last_active":"2024-09-21T07:44:55.565496+0000","last_peered":"2024-09-21T07:44:55.565496+0000","last_clean":"2024-09-21T07:44:55.565496+0000","last_became_active":"2024-09-21T07:35:26.866484+0000","last_became_peered":"2024-09-21T07:35:26.866484+0000","last_unstale":"2024-09-21T07:44:55.565496+0000","last_undegraded":"2024-09-21T07:44:55.565496+0000","last_fullsized":"2024-09-21T07:44:55.565496+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:10.449665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076010100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":332,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.408176+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:44:57.408176+0000","last_peered":"2024-09-21T07:44:57.408176+0000","last_clean":"2024-09-21T07:44:57.408176+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:44:57.408176+0000","last_undegraded":"2024-09-21T07:44:57.408176+0000","last_fullsized":"2024-09-21T07:44:57.408176+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":209,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.577172+0000","last_change":"2024-09-21T07:34:24.539248+0000","last_active":"2024-09-21T07:44:56.577172+0000","last_peered":"2024-09-21T07:44:56.577172+0000","last_clean":"2024-09-21T07:44:56.577172+0000","last_became_active":"2024-09-21T07:34:23.273465+0000","last_became_peered":"2024-09-21T07:34:23.273465+0000","last_unstale":"2024-09-21T07:44:56.577172+0000","last_undegraded":"2024-09-21T07:44:56.577172+0000","last_fullsized":"2024-09-21T07:44:56.577172+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:55.517452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060627700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.597889+0000","last_change":"2024-09-21T07:44:55.586687+0000","last_active":"2024-09-21T07:44:55.597889+0000","last_peered":"2024-09-21T07:44:55.597889+0000","last_clean":"2024-09-21T07:44:55.597889+0000","last_became_active":"2024-09-21T07:44:55.586385+0000","last_became_peered":"2024-09-21T07:44:55.586385+0000","last_unstale":"2024-09-21T07:44:55.597889+0000","last_undegraded":"2024-09-21T07:44:55.597889+0000","last_fullsized":"2024-09-21T07:44:55.597889+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:30:21.718346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"133'3","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.577689+0000","last_change":"2024-09-21T07:44:54.570673+0000","last_active":"2024-09-21T07:44:56.577689+0000","last_peered":"2024-09-21T07:44:56.577689+0000","last_clean":"2024-09-21T07:44:56.577689+0000","last_became_active":"2024-09-21T07:44:54.570315+0000","last_became_peered":"2024-09-21T07:44:54.570315+0000","last_unstale":"2024-09-21T07:44:56.577689+0000","last_undegraded":"2024-09-21T07:44:56.577689+0000","last_fullsized":"2024-09-21T07:44:56.577689+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:04:26.458971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":144,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.565323+0000","last_change":"2024-09-21T07:35:27.864492+0000","last_active":"2024-09-21T07:44:55.565323+0000","last_peered":"2024-09-21T07:44:55.565323+0000","last_clean":"2024-09-21T07:44:55.565323+0000","last_became_active":"2024-09-21T07:35:27.864176+0000","last_became_peered":"2024-09-21T07:35:27.864176+0000","last_unstale":"2024-09-21T07:44:55.565323+0000","last_undegraded":"2024-09-21T07:44:55.565323+0000","last_fullsized":"2024-09-21T07:44:55.565323+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:05.153467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":154,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410622+0000","last_change":"2024-09-21T07:35:29.493678+0000","last_active":"2024-09-21T07:44:57.410622+0000","last_peered":"2024-09-21T07:44:57.410622+0000","last_clean":"2024-09-21T07:44:57.410622+0000","last_became_active":"2024-09-21T07:35:26.870749+0000","last_became_peered":"2024-09-21T07:35:26.870749+0000","last_unstale":"2024-09-21T07:44:57.410622+0000","last_undegraded":"2024-09-21T07:44:57.410622+0000","last_fullsized":"2024-09-21T07:44:57.410622+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:55.367877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010159570000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":202,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.577662+0000","last_change":"2024-09-21T07:34:24.597274+0000","last_active":"2024-09-21T07:44:56.577662+0000","last_peered":"2024-09-21T07:44:56.577662+0000","last_clean":"2024-09-21T07:44:56.577662+0000","last_became_active":"2024-09-21T07:34:23.279560+0000","last_became_peered":"2024-09-21T07:34:23.279560+0000","last_unstale":"2024-09-21T07:44:56.577662+0000","last_undegraded":"2024-09-21T07:44:56.577662+0000","last_fullsized":"2024-09-21T07:44:56.577662+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:15.845265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059699799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576442+0000","last_change":"2024-09-21T07:44:55.574395+0000","last_active":"2024-09-21T07:44:56.576442+0000","last_peered":"2024-09-21T07:44:56.576442+0000","last_clean":"2024-09-21T07:44:56.576442+0000","last_became_active":"2024-09-21T07:44:55.574031+0000","last_became_peered":"2024-09-21T07:44:55.574031+0000","last_unstale":"2024-09-21T07:44:56.576442+0000","last_undegraded":"2024-09-21T07:44:56.576442+0000","last_fullsized":"2024-09-21T07:44:56.576442+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:53.313866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"133'6","reported_seq":19,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.600582+0000","last_change":"2024-09-21T07:44:54.575574+0000","last_active":"2024-09-21T07:44:55.600582+0000","last_peered":"2024-09-21T07:44:55.600582+0000","last_clean":"2024-09-21T07:44:55.600582+0000","last_became_active":"2024-09-21T07:44:54.575263+0000","last_became_peered":"2024-09-21T07:44:54.575263+0000","last_unstale":"2024-09-21T07:44:55.600582+0000","last_undegraded":"2024-09-21T07:44:55.600582+0000","last_fullsized":"2024-09-21T07:44:55.600582+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:06.135058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":147,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.925261+0000","last_change":"2024-09-21T07:35:27.874387+0000","last_active":"2024-09-21T07:44:56.925261+0000","last_peered":"2024-09-21T07:44:56.925261+0000","last_clean":"2024-09-21T07:44:56.925261+0000","last_became_active":"2024-09-21T07:35:27.873277+0000","last_became_peered":"2024-09-21T07:35:27.873277+0000","last_unstale":"2024-09-21T07:44:56.925261+0000","last_undegraded":"2024-09-21T07:44:56.925261+0000","last_fullsized":"2024-09-21T07:44:56.925261+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:04.581463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":152,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.578248+0000","last_change":"2024-09-21T07:35:29.387094+0000","last_active":"2024-09-21T07:44:56.578248+0000","last_peered":"2024-09-21T07:44:56.578248+0000","last_clean":"2024-09-21T07:44:56.578248+0000","last_became_active":"2024-09-21T07:35:26.874757+0000","last_became_peered":"2024-09-21T07:35:26.874757+0000","last_unstale":"2024-09-21T07:44:56.578248+0000","last_undegraded":"2024-09-21T07:44:56.578248+0000","last_fullsized":"2024-09-21T07:44:56.578248+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:33.542750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061065700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":208,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409380+0000","last_change":"2024-09-21T07:34:24.259906+0000","last_active":"2024-09-21T07:44:57.409380+0000","last_peered":"2024-09-21T07:44:57.409380+0000","last_clean":"2024-09-21T07:44:57.409380+0000","last_became_active":"2024-09-21T07:34:23.635282+0000","last_became_peered":"2024-09-21T07:34:23.635282+0000","last_unstale":"2024-09-21T07:44:57.409380+0000","last_undegraded":"2024-09-21T07:44:57.409380+0000","last_fullsized":"2024-09-21T07:44:57.409380+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:00.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001527664,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409128+0000","last_change":"2024-09-21T07:44:55.583909+0000","last_active":"2024-09-21T07:44:57.409128+0000","last_peered":"2024-09-21T07:44:57.409128+0000","last_clean":"2024-09-21T07:44:57.409128+0000","last_became_active":"2024-09-21T07:44:55.583449+0000","last_became_peered":"2024-09-21T07:44:55.583449+0000","last_unstale":"2024-09-21T07:44:57.409128+0000","last_undegraded":"2024-09-21T07:44:57.409128+0000","last_fullsized":"2024-09-21T07:44:57.409128+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:09.715628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409468+0000","last_change":"2024-09-21T07:44:54.574714+0000","last_active":"2024-09-21T07:44:57.409468+0000","last_peered":"2024-09-21T07:44:57.409468+0000","last_clean":"2024-09-21T07:44:57.409468+0000","last_became_active":"2024-09-21T07:44:54.574362+0000","last_became_peered":"2024-09-21T07:44:54.574362+0000","last_unstale":"2024-09-21T07:44:57.409468+0000","last_undegraded":"2024-09-21T07:44:57.409468+0000","last_fullsized":"2024-09-21T07:44:57.409468+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:22.420605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":147,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.925407+0000","last_change":"2024-09-21T07:35:27.873645+0000","last_active":"2024-09-21T07:44:56.925407+0000","last_peered":"2024-09-21T07:44:56.925407+0000","last_clean":"2024-09-21T07:44:56.925407+0000","last_became_active":"2024-09-21T07:35:27.873245+0000","last_became_peered":"2024-09-21T07:35:27.873245+0000","last_unstale":"2024-09-21T07:44:56.925407+0000","last_undegraded":"2024-09-21T07:44:56.925407+0000","last_fullsized":"2024-09-21T07:44:56.925407+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:55.090953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":154,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.925678+0000","last_change":"2024-09-21T07:35:28.861744+0000","last_active":"2024-09-21T07:44:56.925678+0000","last_peered":"2024-09-21T07:44:56.925678+0000","last_clean":"2024-09-21T07:44:56.925678+0000","last_became_active":"2024-09-21T07:35:26.876462+0000","last_became_peered":"2024-09-21T07:35:26.876462+0000","last_unstale":"2024-09-21T07:44:56.925678+0000","last_undegraded":"2024-09-21T07:44:56.925678+0000","last_fullsized":"2024-09-21T07:44:56.925678+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:48.632568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012020920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":205,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409403+0000","last_change":"2024-09-21T07:34:25.085454+0000","last_active":"2024-09-21T07:44:57.409403+0000","last_peered":"2024-09-21T07:44:57.409403+0000","last_clean":"2024-09-21T07:44:57.409403+0000","last_became_active":"2024-09-21T07:34:23.279205+0000","last_became_peered":"2024-09-21T07:34:23.279205+0000","last_unstale":"2024-09-21T07:44:57.409403+0000","last_undegraded":"2024-09-21T07:44:57.409403+0000","last_fullsized":"2024-09-21T07:44:57.409403+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:54.231666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095320599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.577540+0000","last_change":"2024-09-21T07:44:55.587979+0000","last_active":"2024-09-21T07:44:56.577540+0000","last_peered":"2024-09-21T07:44:56.577540+0000","last_clean":"2024-09-21T07:44:56.577540+0000","last_became_active":"2024-09-21T07:44:55.587653+0000","last_became_peered":"2024-09-21T07:44:55.587653+0000","last_unstale":"2024-09-21T07:44:56.577540+0000","last_undegraded":"2024-09-21T07:44:56.577540+0000","last_fullsized":"2024-09-21T07:44:56.577540+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:42.142665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576220+0000","last_change":"2024-09-21T07:44:54.564396+0000","last_active":"2024-09-21T07:44:56.576220+0000","last_peered":"2024-09-21T07:44:56.576220+0000","last_clean":"2024-09-21T07:44:56.576220+0000","last_became_active":"2024-09-21T07:44:54.564012+0000","last_became_peered":"2024-09-21T07:44:54.564012+0000","last_unstale":"2024-09-21T07:44:56.576220+0000","last_undegraded":"2024-09-21T07:44:56.576220+0000","last_fullsized":"2024-09-21T07:44:56.576220+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:43:27.809502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":144,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.567215+0000","last_change":"2024-09-21T07:35:27.861373+0000","last_active":"2024-09-21T07:44:55.567215+0000","last_peered":"2024-09-21T07:44:55.567215+0000","last_clean":"2024-09-21T07:44:55.567215+0000","last_became_active":"2024-09-21T07:35:27.861016+0000","last_became_peered":"2024-09-21T07:35:27.861016+0000","last_unstale":"2024-09-21T07:44:55.567215+0000","last_undegraded":"2024-09-21T07:44:55.567215+0000","last_fullsized":"2024-09-21T07:44:55.567215+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:56.043763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":155,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.575383+0000","last_change":"2024-09-21T07:35:29.053677+0000","last_active":"2024-09-21T07:44:56.575383+0000","last_peered":"2024-09-21T07:44:56.575383+0000","last_clean":"2024-09-21T07:44:56.575383+0000","last_became_active":"2024-09-21T07:35:26.875496+0000","last_became_peered":"2024-09-21T07:35:26.875496+0000","last_unstale":"2024-09-21T07:44:56.575383+0000","last_undegraded":"2024-09-21T07:44:56.575383+0000","last_fullsized":"2024-09-21T07:44:56.575383+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:04.850784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065859000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":207,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.574611+0000","last_change":"2024-09-21T07:34:25.028521+0000","last_active":"2024-09-21T07:44:56.574611+0000","last_peered":"2024-09-21T07:44:56.574611+0000","last_clean":"2024-09-21T07:44:56.574611+0000","last_became_active":"2024-09-21T07:34:23.635467+0000","last_became_peered":"2024-09-21T07:34:23.635467+0000","last_unstale":"2024-09-21T07:44:56.574611+0000","last_undegraded":"2024-09-21T07:44:56.574611+0000","last_fullsized":"2024-09-21T07:44:56.574611+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:33.729123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000660558,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.925423+0000","last_change":"2024-09-21T07:44:55.573727+0000","last_active":"2024-09-21T07:44:56.925423+0000","last_peered":"2024-09-21T07:44:56.925423+0000","last_clean":"2024-09-21T07:44:56.925423+0000","last_became_active":"2024-09-21T07:44:55.573386+0000","last_became_peered":"2024-09-21T07:44:55.573386+0000","last_unstale":"2024-09-21T07:44:56.925423+0000","last_undegraded":"2024-09-21T07:44:56.925423+0000","last_fullsized":"2024-09-21T07:44:56.925423+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:15.437515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.573509+0000","last_change":"2024-09-21T07:44:54.578178+0000","last_active":"2024-09-21T07:44:55.573509+0000","last_peered":"2024-09-21T07:44:55.573509+0000","last_clean":"2024-09-21T07:44:55.573509+0000","last_became_active":"2024-09-21T07:44:54.577847+0000","last_became_peered":"2024-09-21T07:44:54.577847+0000","last_unstale":"2024-09-21T07:44:55.573509+0000","last_undegraded":"2024-09-21T07:44:55.573509+0000","last_fullsized":"2024-09-21T07:44:55.573509+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:41:12.848885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":147,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.411193+0000","last_change":"2024-09-21T07:35:27.871201+0000","last_active":"2024-09-21T07:44:57.411193+0000","last_peered":"2024-09-21T07:44:57.411193+0000","last_clean":"2024-09-21T07:44:57.411193+0000","last_became_active":"2024-09-21T07:35:27.870816+0000","last_became_peered":"2024-09-21T07:35:27.870816+0000","last_unstale":"2024-09-21T07:44:57.411193+0000","last_undegraded":"2024-09-21T07:44:57.411193+0000","last_fullsized":"2024-09-21T07:44:57.411193+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:04.787450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":151,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.567513+0000","last_change":"2024-09-21T07:35:29.491874+0000","last_active":"2024-09-21T07:44:55.567513+0000","last_peered":"2024-09-21T07:44:55.567513+0000","last_clean":"2024-09-21T07:44:55.567513+0000","last_became_active":"2024-09-21T07:35:26.866381+0000","last_became_peered":"2024-09-21T07:35:26.866381+0000","last_unstale":"2024-09-21T07:44:55.567513+0000","last_undegraded":"2024-09-21T07:44:55.567513+0000","last_fullsized":"2024-09-21T07:44:55.567513+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:36.404647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073235399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":205,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.574888+0000","last_change":"2024-09-21T07:34:24.260046+0000","last_active":"2024-09-21T07:44:55.574888+0000","last_peered":"2024-09-21T07:44:55.574888+0000","last_clean":"2024-09-21T07:44:55.574888+0000","last_became_active":"2024-09-21T07:34:23.273431+0000","last_became_peered":"2024-09-21T07:34:23.273431+0000","last_unstale":"2024-09-21T07:44:55.574888+0000","last_undegraded":"2024-09-21T07:44:55.574888+0000","last_fullsized":"2024-09-21T07:44:55.574888+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:18.876057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019803310000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.408725+0000","last_change":"2024-09-21T07:44:55.587563+0000","last_active":"2024-09-21T07:44:57.408725+0000","last_peered":"2024-09-21T07:44:57.408725+0000","last_clean":"2024-09-21T07:44:57.408725+0000","last_became_active":"2024-09-21T07:44:55.587101+0000","last_became_peered":"2024-09-21T07:44:55.587101+0000","last_unstale":"2024-09-21T07:44:57.408725+0000","last_undegraded":"2024-09-21T07:44:57.408725+0000","last_fullsized":"2024-09-21T07:44:57.408725+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:16.375239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"133'2","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.598066+0000","last_change":"2024-09-21T07:44:54.579114+0000","last_active":"2024-09-21T07:44:55.598066+0000","last_peered":"2024-09-21T07:44:55.598066+0000","last_clean":"2024-09-21T07:44:55.598066+0000","last_became_active":"2024-09-21T07:44:54.578761+0000","last_became_peered":"2024-09-21T07:44:54.578761+0000","last_unstale":"2024-09-21T07:44:55.598066+0000","last_undegraded":"2024-09-21T07:44:55.598066+0000","last_fullsized":"2024-09-21T07:44:55.598066+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:22.050230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":147,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410342+0000","last_change":"2024-09-21T07:35:27.871736+0000","last_active":"2024-09-21T07:44:57.410342+0000","last_peered":"2024-09-21T07:44:57.410342+0000","last_clean":"2024-09-21T07:44:57.410342+0000","last_became_active":"2024-09-21T07:35:27.871232+0000","last_became_peered":"2024-09-21T07:35:27.871232+0000","last_unstale":"2024-09-21T07:44:57.410342+0000","last_undegraded":"2024-09-21T07:44:57.410342+0000","last_fullsized":"2024-09-21T07:44:57.410342+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:04.419301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":151,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.573798+0000","last_change":"2024-09-21T07:35:29.491246+0000","last_active":"2024-09-21T07:44:55.573798+0000","last_peered":"2024-09-21T07:44:55.573798+0000","last_clean":"2024-09-21T07:44:55.573798+0000","last_became_active":"2024-09-21T07:35:26.866783+0000","last_became_peered":"2024-09-21T07:35:26.866783+0000","last_unstale":"2024-09-21T07:44:55.573798+0000","last_undegraded":"2024-09-21T07:44:55.573798+0000","last_fullsized":"2024-09-21T07:44:55.573798+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:28.881914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098474099999999992,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":207,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.575464+0000","last_change":"2024-09-21T07:34:25.028640+0000","last_active":"2024-09-21T07:44:56.575464+0000","last_peered":"2024-09-21T07:44:56.575464+0000","last_clean":"2024-09-21T07:44:56.575464+0000","last_became_active":"2024-09-21T07:34:23.260950+0000","last_became_peered":"2024-09-21T07:34:23.260950+0000","last_unstale":"2024-09-21T07:44:56.575464+0000","last_undegraded":"2024-09-21T07:44:56.575464+0000","last_fullsized":"2024-09-21T07:44:56.575464+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:23:42.121398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055135099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.574774+0000","last_change":"2024-09-21T07:44:55.583610+0000","last_active":"2024-09-21T07:44:56.574774+0000","last_peered":"2024-09-21T07:44:56.574774+0000","last_clean":"2024-09-21T07:44:56.574774+0000","last_became_active":"2024-09-21T07:44:55.583311+0000","last_became_peered":"2024-09-21T07:44:55.583311+0000","last_unstale":"2024-09-21T07:44:56.574774+0000","last_undegraded":"2024-09-21T07:44:56.574774+0000","last_fullsized":"2024-09-21T07:44:56.574774+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:13.056811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.574119+0000","last_change":"2024-09-21T07:44:54.577276+0000","last_active":"2024-09-21T07:44:55.574119+0000","last_peered":"2024-09-21T07:44:55.574119+0000","last_clean":"2024-09-21T07:44:55.574119+0000","last_became_active":"2024-09-21T07:44:54.576350+0000","last_became_peered":"2024-09-21T07:44:54.576350+0000","last_unstale":"2024-09-21T07:44:55.574119+0000","last_undegraded":"2024-09-21T07:44:55.574119+0000","last_fullsized":"2024-09-21T07:44:55.574119+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:42.264379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":146,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410369+0000","last_change":"2024-09-21T07:35:27.872027+0000","last_active":"2024-09-21T07:44:57.410369+0000","last_peered":"2024-09-21T07:44:57.410369+0000","last_clean":"2024-09-21T07:44:57.410369+0000","last_became_active":"2024-09-21T07:35:27.870426+0000","last_became_peered":"2024-09-21T07:35:27.870426+0000","last_unstale":"2024-09-21T07:44:57.410369+0000","last_undegraded":"2024-09-21T07:44:57.410369+0000","last_fullsized":"2024-09-21T07:44:57.410369+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":151,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.565447+0000","last_change":"2024-09-21T07:35:29.491008+0000","last_active":"2024-09-21T07:44:55.565447+0000","last_peered":"2024-09-21T07:44:55.565447+0000","last_clean":"2024-09-21T07:44:55.565447+0000","last_became_active":"2024-09-21T07:35:26.866750+0000","last_became_peered":"2024-09-21T07:35:26.866750+0000","last_unstale":"2024-09-21T07:44:55.565447+0000","last_undegraded":"2024-09-21T07:44:55.565447+0000","last_fullsized":"2024-09-21T07:44:55.565447+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:25.490406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083529700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":208,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410202+0000","last_change":"2024-09-21T07:34:24.260665+0000","last_active":"2024-09-21T07:44:57.410202+0000","last_peered":"2024-09-21T07:44:57.410202+0000","last_clean":"2024-09-21T07:44:57.410202+0000","last_became_active":"2024-09-21T07:34:23.635909+0000","last_became_peered":"2024-09-21T07:34:23.635909+0000","last_unstale":"2024-09-21T07:44:57.410202+0000","last_undegraded":"2024-09-21T07:44:57.410202+0000","last_fullsized":"2024-09-21T07:44:57.410202+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:44.536885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576073+0000","last_change":"2024-09-21T07:44:55.578791+0000","last_active":"2024-09-21T07:44:56.576073+0000","last_peered":"2024-09-21T07:44:56.576073+0000","last_clean":"2024-09-21T07:44:56.576073+0000","last_became_active":"2024-09-21T07:44:55.578451+0000","last_became_peered":"2024-09-21T07:44:55.578451+0000","last_unstale":"2024-09-21T07:44:56.576073+0000","last_undegraded":"2024-09-21T07:44:56.576073+0000","last_fullsized":"2024-09-21T07:44:56.576073+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:25:05.931499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"133'2","reported_seq":17,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409640+0000","last_change":"2024-09-21T07:44:54.576757+0000","last_active":"2024-09-21T07:44:57.409640+0000","last_peered":"2024-09-21T07:44:57.409640+0000","last_clean":"2024-09-21T07:44:57.409640+0000","last_became_active":"2024-09-21T07:44:54.576395+0000","last_became_peered":"2024-09-21T07:44:54.576395+0000","last_unstale":"2024-09-21T07:44:57.409640+0000","last_undegraded":"2024-09-21T07:44:57.409640+0000","last_fullsized":"2024-09-21T07:44:57.409640+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:29.874215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":145,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576506+0000","last_change":"2024-09-21T07:35:27.874772+0000","last_active":"2024-09-21T07:44:56.576506+0000","last_peered":"2024-09-21T07:44:56.576506+0000","last_clean":"2024-09-21T07:44:56.576506+0000","last_became_active":"2024-09-21T07:35:27.874434+0000","last_became_peered":"2024-09-21T07:35:27.874434+0000","last_unstale":"2024-09-21T07:44:56.576506+0000","last_undegraded":"2024-09-21T07:44:56.576506+0000","last_fullsized":"2024-09-21T07:44:56.576506+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:31.998755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":157,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576807+0000","last_change":"2024-09-21T07:35:28.859153+0000","last_active":"2024-09-21T07:44:56.576807+0000","last_peered":"2024-09-21T07:44:56.576807+0000","last_clean":"2024-09-21T07:44:56.576807+0000","last_became_active":"2024-09-21T07:35:26.874428+0000","last_became_peered":"2024-09-21T07:35:26.874428+0000","last_unstale":"2024-09-21T07:44:56.576807+0000","last_undegraded":"2024-09-21T07:44:56.576807+0000","last_fullsized":"2024-09-21T07:44:56.576807+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:04.487621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048557299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":209,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.577016+0000","last_change":"2024-09-21T07:34:24.538845+0000","last_active":"2024-09-21T07:44:56.577016+0000","last_peered":"2024-09-21T07:44:56.577016+0000","last_clean":"2024-09-21T07:44:56.577016+0000","last_became_active":"2024-09-21T07:34:23.636372+0000","last_became_peered":"2024-09-21T07:34:23.636372+0000","last_unstale":"2024-09-21T07:44:56.577016+0000","last_undegraded":"2024-09-21T07:44:56.577016+0000","last_fullsized":"2024-09-21T07:44:56.577016+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:29.393813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841451,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410444+0000","last_change":"2024-09-21T07:44:55.586424+0000","last_active":"2024-09-21T07:44:57.410444+0000","last_peered":"2024-09-21T07:44:57.410444+0000","last_clean":"2024-09-21T07:44:57.410444+0000","last_became_active":"2024-09-21T07:44:55.586050+0000","last_became_peered":"2024-09-21T07:44:55.586050+0000","last_unstale":"2024-09-21T07:44:57.410444+0000","last_undegraded":"2024-09-21T07:44:57.410444+0000","last_fullsized":"2024-09-21T07:44:57.410444+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:16.375239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"133'2","reported_seq":17,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.575316+0000","last_change":"2024-09-21T07:44:54.567388+0000","last_active":"2024-09-21T07:44:56.575316+0000","last_peered":"2024-09-21T07:44:56.575316+0000","last_clean":"2024-09-21T07:44:56.575316+0000","last_became_active":"2024-09-21T07:44:54.566216+0000","last_became_peered":"2024-09-21T07:44:54.566216+0000","last_unstale":"2024-09-21T07:44:56.575316+0000","last_undegraded":"2024-09-21T07:44:56.575316+0000","last_fullsized":"2024-09-21T07:44:56.575316+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:45:55.012600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":146,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.408661+0000","last_change":"2024-09-21T07:35:27.877801+0000","last_active":"2024-09-21T07:44:57.408661+0000","last_peered":"2024-09-21T07:44:57.408661+0000","last_clean":"2024-09-21T07:44:57.408661+0000","last_became_active":"2024-09-21T07:35:27.877464+0000","last_became_peered":"2024-09-21T07:35:27.877464+0000","last_unstale":"2024-09-21T07:44:57.408661+0000","last_undegraded":"2024-09-21T07:44:57.408661+0000","last_fullsized":"2024-09-21T07:44:57.408661+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:08.814929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":154,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410452+0000","last_change":"2024-09-21T07:35:28.863732+0000","last_active":"2024-09-21T07:44:57.410452+0000","last_peered":"2024-09-21T07:44:57.410452+0000","last_clean":"2024-09-21T07:44:57.410452+0000","last_became_active":"2024-09-21T07:35:26.877392+0000","last_became_peered":"2024-09-21T07:35:26.877392+0000","last_unstale":"2024-09-21T07:44:57.410452+0000","last_undegraded":"2024-09-21T07:44:57.410452+0000","last_fullsized":"2024-09-21T07:44:57.410452+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071425700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":205,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.567612+0000","last_change":"2024-09-21T07:34:24.259686+0000","last_active":"2024-09-21T07:44:55.567612+0000","last_peered":"2024-09-21T07:44:55.567612+0000","last_clean":"2024-09-21T07:44:55.567612+0000","last_became_active":"2024-09-21T07:34:23.274405+0000","last_became_peered":"2024-09-21T07:34:23.274405+0000","last_unstale":"2024-09-21T07:44:55.567612+0000","last_undegraded":"2024-09-21T07:44:55.567612+0000","last_fullsized":"2024-09-21T07:44:55.567612+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:26.304693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022009920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.575386+0000","last_change":"2024-09-21T07:44:55.589236+0000","last_active":"2024-09-21T07:44:56.575386+0000","last_peered":"2024-09-21T07:44:56.575386+0000","last_clean":"2024-09-21T07:44:56.575386+0000","last_became_active":"2024-09-21T07:44:55.588892+0000","last_became_peered":"2024-09-21T07:44:55.588892+0000","last_unstale":"2024-09-21T07:44:56.575386+0000","last_undegraded":"2024-09-21T07:44:56.575386+0000","last_fullsized":"2024-09-21T07:44:56.575386+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:39.729629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"133'4","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409771+0000","last_change":"2024-09-21T07:44:54.572798+0000","last_active":"2024-09-21T07:44:57.409771+0000","last_peered":"2024-09-21T07:44:57.409771+0000","last_clean":"2024-09-21T07:44:57.409771+0000","last_became_active":"2024-09-21T07:44:54.572390+0000","last_became_peered":"2024-09-21T07:44:54.572390+0000","last_unstale":"2024-09-21T07:44:57.409771+0000","last_undegraded":"2024-09-21T07:44:57.409771+0000","last_fullsized":"2024-09-21T07:44:57.409771+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:07:31.093396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":145,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.577180+0000","last_change":"2024-09-21T07:35:27.867336+0000","last_active":"2024-09-21T07:44:56.577180+0000","last_peered":"2024-09-21T07:44:56.577180+0000","last_clean":"2024-09-21T07:44:56.577180+0000","last_became_active":"2024-09-21T07:35:27.867015+0000","last_became_peered":"2024-09-21T07:35:27.867015+0000","last_unstale":"2024-09-21T07:44:56.577180+0000","last_undegraded":"2024-09-21T07:44:56.577180+0000","last_fullsized":"2024-09-21T07:44:56.577180+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:59.214139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":154,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409964+0000","last_change":"2024-09-21T07:35:29.493137+0000","last_active":"2024-09-21T07:44:57.409964+0000","last_peered":"2024-09-21T07:44:57.409964+0000","last_clean":"2024-09-21T07:44:57.409964+0000","last_became_active":"2024-09-21T07:35:26.872230+0000","last_became_peered":"2024-09-21T07:35:26.872230+0000","last_unstale":"2024-09-21T07:44:57.409964+0000","last_undegraded":"2024-09-21T07:44:57.409964+0000","last_fullsized":"2024-09-21T07:44:57.409964+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:24.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001197449,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":207,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.408121+0000","last_change":"2024-09-21T07:34:24.258754+0000","last_active":"2024-09-21T07:44:57.408121+0000","last_peered":"2024-09-21T07:44:57.408121+0000","last_clean":"2024-09-21T07:44:57.408121+0000","last_became_active":"2024-09-21T07:34:23.272657+0000","last_became_peered":"2024-09-21T07:34:23.272657+0000","last_unstale":"2024-09-21T07:44:57.408121+0000","last_undegraded":"2024-09-21T07:44:57.408121+0000","last_fullsized":"2024-09-21T07:44:57.408121+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:05.477099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067143200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409889+0000","last_change":"2024-09-21T07:44:55.593668+0000","last_active":"2024-09-21T07:44:57.409889+0000","last_peered":"2024-09-21T07:44:57.409889+0000","last_clean":"2024-09-21T07:44:57.409889+0000","last_became_active":"2024-09-21T07:44:55.593316+0000","last_became_peered":"2024-09-21T07:44:55.593316+0000","last_unstale":"2024-09-21T07:44:57.409889+0000","last_undegraded":"2024-09-21T07:44:57.409889+0000","last_fullsized":"2024-09-21T07:44:57.409889+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:57.156663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"133'1","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.575624+0000","last_change":"2024-09-21T07:44:54.567256+0000","last_active":"2024-09-21T07:44:56.575624+0000","last_peered":"2024-09-21T07:44:56.575624+0000","last_clean":"2024-09-21T07:44:56.575624+0000","last_became_active":"2024-09-21T07:44:54.565314+0000","last_became_peered":"2024-09-21T07:44:54.565314+0000","last_unstale":"2024-09-21T07:44:56.575624+0000","last_undegraded":"2024-09-21T07:44:56.575624+0000","last_fullsized":"2024-09-21T07:44:56.575624+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:54:06.235195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":147,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410129+0000","last_change":"2024-09-21T07:35:27.876021+0000","last_active":"2024-09-21T07:44:57.410129+0000","last_peered":"2024-09-21T07:44:57.410129+0000","last_clean":"2024-09-21T07:44:57.410129+0000","last_became_active":"2024-09-21T07:35:27.875724+0000","last_became_peered":"2024-09-21T07:35:27.875724+0000","last_unstale":"2024-09-21T07:44:57.410129+0000","last_undegraded":"2024-09-21T07:44:57.410129+0000","last_fullsized":"2024-09-21T07:44:57.410129+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:03.022812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":154,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409837+0000","last_change":"2024-09-21T07:35:29.492570+0000","last_active":"2024-09-21T07:44:57.409837+0000","last_peered":"2024-09-21T07:44:57.409837+0000","last_clean":"2024-09-21T07:44:57.409837+0000","last_became_active":"2024-09-21T07:35:26.874316+0000","last_became_peered":"2024-09-21T07:35:26.874316+0000","last_unstale":"2024-09-21T07:44:57.409837+0000","last_undegraded":"2024-09-21T07:44:57.409837+0000","last_fullsized":"2024-09-21T07:44:57.409837+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:16.512323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011990760000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":205,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409592+0000","last_change":"2024-09-21T07:34:25.085232+0000","last_active":"2024-09-21T07:44:57.409592+0000","last_peered":"2024-09-21T07:44:57.409592+0000","last_clean":"2024-09-21T07:44:57.409592+0000","last_became_active":"2024-09-21T07:34:23.277807+0000","last_became_peered":"2024-09-21T07:34:23.277807+0000","last_unstale":"2024-09-21T07:44:57.409592+0000","last_undegraded":"2024-09-21T07:44:57.409592+0000","last_fullsized":"2024-09-21T07:44:57.409592+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:36.362919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086619099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":10,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.595191+0000","last_change":"2024-09-21T07:44:55.581656+0000","last_active":"2024-09-21T07:44:55.595191+0000","last_peered":"2024-09-21T07:44:55.595191+0000","last_clean":"2024-09-21T07:44:55.595191+0000","last_became_active":"2024-09-21T07:44:55.580995+0000","last_became_peered":"2024-09-21T07:44:55.580995+0000","last_unstale":"2024-09-21T07:44:55.595191+0000","last_undegraded":"2024-09-21T07:44:55.595191+0000","last_fullsized":"2024-09-21T07:44:55.595191+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:50:34.185292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.575882+0000","last_change":"2024-09-21T07:44:54.566731+0000","last_active":"2024-09-21T07:44:56.575882+0000","last_peered":"2024-09-21T07:44:56.575882+0000","last_clean":"2024-09-21T07:44:56.575882+0000","last_became_active":"2024-09-21T07:44:54.566364+0000","last_became_peered":"2024-09-21T07:44:54.566364+0000","last_unstale":"2024-09-21T07:44:56.575882+0000","last_undegraded":"2024-09-21T07:44:56.575882+0000","last_fullsized":"2024-09-21T07:44:56.575882+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:08.662117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":146,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409289+0000","last_change":"2024-09-21T07:35:27.872798+0000","last_active":"2024-09-21T07:44:57.409289+0000","last_peered":"2024-09-21T07:44:57.409289+0000","last_clean":"2024-09-21T07:44:57.409289+0000","last_became_active":"2024-09-21T07:35:27.872483+0000","last_became_peered":"2024-09-21T07:35:27.872483+0000","last_unstale":"2024-09-21T07:44:57.409289+0000","last_undegraded":"2024-09-21T07:44:57.409289+0000","last_fullsized":"2024-09-21T07:44:57.409289+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:11.933469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":153,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.408218+0000","last_change":"2024-09-21T07:35:29.206723+0000","last_active":"2024-09-21T07:44:57.408218+0000","last_peered":"2024-09-21T07:44:57.408218+0000","last_clean":"2024-09-21T07:44:57.408218+0000","last_became_active":"2024-09-21T07:35:26.872377+0000","last_became_peered":"2024-09-21T07:35:26.872377+0000","last_unstale":"2024-09-21T07:44:57.408218+0000","last_undegraded":"2024-09-21T07:44:57.408218+0000","last_fullsized":"2024-09-21T07:44:57.408218+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:29.086862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083077200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":201,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.578425+0000","last_change":"2024-09-21T07:34:24.597672+0000","last_active":"2024-09-21T07:44:56.578425+0000","last_peered":"2024-09-21T07:44:56.578425+0000","last_clean":"2024-09-21T07:44:56.578425+0000","last_became_active":"2024-09-21T07:34:23.267770+0000","last_became_peered":"2024-09-21T07:34:23.267770+0000","last_unstale":"2024-09-21T07:44:56.578425+0000","last_undegraded":"2024-09-21T07:44:56.578425+0000","last_fullsized":"2024-09-21T07:44:56.578425+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:51.629476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086378500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.574971+0000","last_change":"2024-09-21T07:44:55.592212+0000","last_active":"2024-09-21T07:44:56.574971+0000","last_peered":"2024-09-21T07:44:56.574971+0000","last_clean":"2024-09-21T07:44:56.574971+0000","last_became_active":"2024-09-21T07:44:55.591594+0000","last_became_peered":"2024-09-21T07:44:55.591594+0000","last_unstale":"2024-09-21T07:44:56.574971+0000","last_undegraded":"2024-09-21T07:44:56.574971+0000","last_fullsized":"2024-09-21T07:44:56.574971+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:07:32.096178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409219+0000","last_change":"2024-09-21T07:44:54.573479+0000","last_active":"2024-09-21T07:44:57.409219+0000","last_peered":"2024-09-21T07:44:57.409219+0000","last_clean":"2024-09-21T07:44:57.409219+0000","last_became_active":"2024-09-21T07:44:54.572774+0000","last_became_peered":"2024-09-21T07:44:54.572774+0000","last_unstale":"2024-09-21T07:44:57.409219+0000","last_undegraded":"2024-09-21T07:44:57.409219+0000","last_fullsized":"2024-09-21T07:44:57.409219+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:51:31.141936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":146,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409146+0000","last_change":"2024-09-21T07:35:27.870121+0000","last_active":"2024-09-21T07:44:57.409146+0000","last_peered":"2024-09-21T07:44:57.409146+0000","last_clean":"2024-09-21T07:44:57.409146+0000","last_became_active":"2024-09-21T07:35:27.869811+0000","last_became_peered":"2024-09-21T07:35:27.869811+0000","last_unstale":"2024-09-21T07:44:57.409146+0000","last_undegraded":"2024-09-21T07:44:57.409146+0000","last_fullsized":"2024-09-21T07:44:57.409146+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:31.778461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":154,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.925283+0000","last_change":"2024-09-21T07:35:28.860699+0000","last_active":"2024-09-21T07:44:56.925283+0000","last_peered":"2024-09-21T07:44:56.925283+0000","last_clean":"2024-09-21T07:44:56.925283+0000","last_became_active":"2024-09-21T07:35:26.876779+0000","last_became_peered":"2024-09-21T07:35:26.876779+0000","last_unstale":"2024-09-21T07:44:56.925283+0000","last_undegraded":"2024-09-21T07:44:56.925283+0000","last_fullsized":"2024-09-21T07:44:56.925283+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:28.286403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000913188,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":206,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.926022+0000","last_change":"2024-09-21T07:34:24.260181+0000","last_active":"2024-09-21T07:44:56.926022+0000","last_peered":"2024-09-21T07:44:56.926022+0000","last_clean":"2024-09-21T07:44:56.926022+0000","last_became_active":"2024-09-21T07:34:23.275580+0000","last_became_peered":"2024-09-21T07:34:23.275580+0000","last_unstale":"2024-09-21T07:44:56.926022+0000","last_undegraded":"2024-09-21T07:44:56.926022+0000","last_fullsized":"2024-09-21T07:44:56.926022+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:49.857622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014706980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.574597+0000","last_change":"2024-09-21T07:44:55.573251+0000","last_active":"2024-09-21T07:44:56.574597+0000","last_peered":"2024-09-21T07:44:56.574597+0000","last_clean":"2024-09-21T07:44:56.574597+0000","last_became_active":"2024-09-21T07:44:55.572910+0000","last_became_peered":"2024-09-21T07:44:55.572910+0000","last_unstale":"2024-09-21T07:44:56.574597+0000","last_undegraded":"2024-09-21T07:44:56.574597+0000","last_fullsized":"2024-09-21T07:44:56.574597+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:35:39.531219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"133'1","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.578049+0000","last_change":"2024-09-21T07:44:54.582263+0000","last_active":"2024-09-21T07:44:56.578049+0000","last_peered":"2024-09-21T07:44:56.578049+0000","last_clean":"2024-09-21T07:44:56.578049+0000","last_became_active":"2024-09-21T07:44:54.581985+0000","last_became_peered":"2024-09-21T07:44:54.581985+0000","last_unstale":"2024-09-21T07:44:56.578049+0000","last_undegraded":"2024-09-21T07:44:56.578049+0000","last_fullsized":"2024-09-21T07:44:56.578049+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:48.161401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":148,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.575688+0000","last_change":"2024-09-21T07:35:27.875393+0000","last_active":"2024-09-21T07:44:56.575688+0000","last_peered":"2024-09-21T07:44:56.575688+0000","last_clean":"2024-09-21T07:44:56.575688+0000","last_became_active":"2024-09-21T07:35:27.874671+0000","last_became_peered":"2024-09-21T07:35:27.874671+0000","last_unstale":"2024-09-21T07:44:56.575688+0000","last_undegraded":"2024-09-21T07:44:56.575688+0000","last_fullsized":"2024-09-21T07:44:56.575688+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":153,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.408857+0000","last_change":"2024-09-21T07:35:29.206296+0000","last_active":"2024-09-21T07:44:57.408857+0000","last_peered":"2024-09-21T07:44:57.408857+0000","last_clean":"2024-09-21T07:44:57.408857+0000","last_became_active":"2024-09-21T07:35:26.874575+0000","last_became_peered":"2024-09-21T07:35:26.874575+0000","last_unstale":"2024-09-21T07:44:57.408857+0000","last_undegraded":"2024-09-21T07:44:57.408857+0000","last_fullsized":"2024-09-21T07:44:57.408857+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:22.771154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061011599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":205,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.571513+0000","last_change":"2024-09-21T07:34:24.259112+0000","last_active":"2024-09-21T07:44:55.571513+0000","last_peered":"2024-09-21T07:44:55.571513+0000","last_clean":"2024-09-21T07:44:55.571513+0000","last_became_active":"2024-09-21T07:34:23.635444+0000","last_became_peered":"2024-09-21T07:34:23.635444+0000","last_unstale":"2024-09-21T07:44:55.571513+0000","last_undegraded":"2024-09-21T07:44:55.571513+0000","last_fullsized":"2024-09-21T07:44:55.571513+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:46.255385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001952636,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576512+0000","last_change":"2024-09-21T07:44:55.591992+0000","last_active":"2024-09-21T07:44:56.576512+0000","last_peered":"2024-09-21T07:44:56.576512+0000","last_clean":"2024-09-21T07:44:56.576512+0000","last_became_active":"2024-09-21T07:44:55.591655+0000","last_became_peered":"2024-09-21T07:44:55.591655+0000","last_unstale":"2024-09-21T07:44:56.576512+0000","last_undegraded":"2024-09-21T07:44:56.576512+0000","last_fullsized":"2024-09-21T07:44:56.576512+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:02:53.367109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"133'1","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410264+0000","last_change":"2024-09-21T07:44:54.570542+0000","last_active":"2024-09-21T07:44:57.410264+0000","last_peered":"2024-09-21T07:44:57.410264+0000","last_clean":"2024-09-21T07:44:57.410264+0000","last_became_active":"2024-09-21T07:44:54.570143+0000","last_became_peered":"2024-09-21T07:44:54.570143+0000","last_unstale":"2024-09-21T07:44:57.410264+0000","last_undegraded":"2024-09-21T07:44:57.410264+0000","last_fullsized":"2024-09-21T07:44:57.410264+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:25:42.288412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":148,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576045+0000","last_change":"2024-09-21T07:35:27.878356+0000","last_active":"2024-09-21T07:44:56.576045+0000","last_peered":"2024-09-21T07:44:56.576045+0000","last_clean":"2024-09-21T07:44:56.576045+0000","last_became_active":"2024-09-21T07:35:27.877982+0000","last_became_peered":"2024-09-21T07:35:27.877982+0000","last_unstale":"2024-09-21T07:44:56.576045+0000","last_undegraded":"2024-09-21T07:44:56.576045+0000","last_fullsized":"2024-09-21T07:44:56.576045+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":155,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.574611+0000","last_change":"2024-09-21T07:35:29.053131+0000","last_active":"2024-09-21T07:44:56.574611+0000","last_peered":"2024-09-21T07:44:56.574611+0000","last_clean":"2024-09-21T07:44:56.574611+0000","last_became_active":"2024-09-21T07:35:26.870516+0000","last_became_peered":"2024-09-21T07:35:26.870516+0000","last_unstale":"2024-09-21T07:44:56.574611+0000","last_undegraded":"2024-09-21T07:44:56.574611+0000","last_fullsized":"2024-09-21T07:44:56.574611+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:10.924491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070705699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":209,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.577837+0000","last_change":"2024-09-21T07:34:24.539152+0000","last_active":"2024-09-21T07:44:56.577837+0000","last_peered":"2024-09-21T07:44:56.577837+0000","last_clean":"2024-09-21T07:44:56.577837+0000","last_became_active":"2024-09-21T07:34:23.259573+0000","last_became_peered":"2024-09-21T07:34:23.259573+0000","last_unstale":"2024-09-21T07:44:56.577837+0000","last_undegraded":"2024-09-21T07:44:56.577837+0000","last_fullsized":"2024-09-21T07:44:56.577837+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:26.986900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058108400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409847+0000","last_change":"2024-09-21T07:44:55.591791+0000","last_active":"2024-09-21T07:44:57.409847+0000","last_peered":"2024-09-21T07:44:57.409847+0000","last_clean":"2024-09-21T07:44:57.409847+0000","last_became_active":"2024-09-21T07:44:55.586035+0000","last_became_peered":"2024-09-21T07:44:55.586035+0000","last_unstale":"2024-09-21T07:44:57.409847+0000","last_undegraded":"2024-09-21T07:44:57.409847+0000","last_fullsized":"2024-09-21T07:44:57.409847+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:04:48.783564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409544+0000","last_change":"2024-09-21T07:44:54.579509+0000","last_active":"2024-09-21T07:44:57.409544+0000","last_peered":"2024-09-21T07:44:57.409544+0000","last_clean":"2024-09-21T07:44:57.409544+0000","last_became_active":"2024-09-21T07:44:54.579162+0000","last_became_peered":"2024-09-21T07:44:54.579162+0000","last_unstale":"2024-09-21T07:44:57.409544+0000","last_undegraded":"2024-09-21T07:44:57.409544+0000","last_fullsized":"2024-09-21T07:44:57.409544+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:53:06.530441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":147,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.574468+0000","last_change":"2024-09-21T07:35:27.873726+0000","last_active":"2024-09-21T07:44:56.574468+0000","last_peered":"2024-09-21T07:44:56.574468+0000","last_clean":"2024-09-21T07:44:56.574468+0000","last_became_active":"2024-09-21T07:35:27.871421+0000","last_became_peered":"2024-09-21T07:35:27.871421+0000","last_unstale":"2024-09-21T07:44:56.574468+0000","last_undegraded":"2024-09-21T07:44:56.574468+0000","last_fullsized":"2024-09-21T07:44:56.574468+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:00.168915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":154,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.925873+0000","last_change":"2024-09-21T07:35:28.861032+0000","last_active":"2024-09-21T07:44:56.925873+0000","last_peered":"2024-09-21T07:44:56.925873+0000","last_clean":"2024-09-21T07:44:56.925873+0000","last_became_active":"2024-09-21T07:35:26.873149+0000","last_became_peered":"2024-09-21T07:35:26.873149+0000","last_unstale":"2024-09-21T07:44:56.925873+0000","last_undegraded":"2024-09-21T07:44:56.925873+0000","last_fullsized":"2024-09-21T07:44:56.925873+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:59.435981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011988369999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":207,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409164+0000","last_change":"2024-09-21T07:34:24.259902+0000","last_active":"2024-09-21T07:44:57.409164+0000","last_peered":"2024-09-21T07:44:57.409164+0000","last_clean":"2024-09-21T07:44:57.409164+0000","last_became_active":"2024-09-21T07:34:23.271902+0000","last_became_peered":"2024-09-21T07:34:23.271902+0000","last_unstale":"2024-09-21T07:44:57.409164+0000","last_undegraded":"2024-09-21T07:44:57.409164+0000","last_fullsized":"2024-09-21T07:44:57.409164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:59.439525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001686358,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.926013+0000","last_change":"2024-09-21T07:44:55.584056+0000","last_active":"2024-09-21T07:44:56.926013+0000","last_peered":"2024-09-21T07:44:56.926013+0000","last_clean":"2024-09-21T07:44:56.926013+0000","last_became_active":"2024-09-21T07:44:55.583709+0000","last_became_peered":"2024-09-21T07:44:55.583709+0000","last_unstale":"2024-09-21T07:44:56.926013+0000","last_undegraded":"2024-09-21T07:44:56.926013+0000","last_fullsized":"2024-09-21T07:44:56.926013+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:27.776809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.411093+0000","last_change":"2024-09-21T07:44:54.571782+0000","last_active":"2024-09-21T07:44:57.411093+0000","last_peered":"2024-09-21T07:44:57.411093+0000","last_clean":"2024-09-21T07:44:57.411093+0000","last_became_active":"2024-09-21T07:44:54.571458+0000","last_became_peered":"2024-09-21T07:44:54.571458+0000","last_unstale":"2024-09-21T07:44:57.411093+0000","last_undegraded":"2024-09-21T07:44:57.411093+0000","last_fullsized":"2024-09-21T07:44:57.411093+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:13:26.409608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":146,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409316+0000","last_change":"2024-09-21T07:35:27.872637+0000","last_active":"2024-09-21T07:44:57.409316+0000","last_peered":"2024-09-21T07:44:57.409316+0000","last_clean":"2024-09-21T07:44:57.409316+0000","last_became_active":"2024-09-21T07:35:27.872149+0000","last_became_peered":"2024-09-21T07:35:27.872149+0000","last_unstale":"2024-09-21T07:44:57.409316+0000","last_undegraded":"2024-09-21T07:44:57.409316+0000","last_fullsized":"2024-09-21T07:44:57.409316+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:27.697872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":154,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409759+0000","last_change":"2024-09-21T07:35:29.493362+0000","last_active":"2024-09-21T07:44:57.409759+0000","last_peered":"2024-09-21T07:44:57.409759+0000","last_clean":"2024-09-21T07:44:57.409759+0000","last_became_active":"2024-09-21T07:35:26.873018+0000","last_became_peered":"2024-09-21T07:35:26.873018+0000","last_unstale":"2024-09-21T07:44:57.409759+0000","last_undegraded":"2024-09-21T07:44:57.409759+0000","last_fullsized":"2024-09-21T07:44:57.409759+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:02.460557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012496510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":205,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.567418+0000","last_change":"2024-09-21T07:34:24.259662+0000","last_active":"2024-09-21T07:44:55.567418+0000","last_peered":"2024-09-21T07:44:55.567418+0000","last_clean":"2024-09-21T07:44:55.567418+0000","last_became_active":"2024-09-21T07:34:23.278812+0000","last_became_peered":"2024-09-21T07:34:23.278812+0000","last_unstale":"2024-09-21T07:44:55.567418+0000","last_undegraded":"2024-09-21T07:44:55.567418+0000","last_fullsized":"2024-09-21T07:44:55.567418+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:49.829974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356463,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.411031+0000","last_change":"2024-09-21T07:44:55.580365+0000","last_active":"2024-09-21T07:44:57.411031+0000","last_peered":"2024-09-21T07:44:57.411031+0000","last_clean":"2024-09-21T07:44:57.411031+0000","last_became_active":"2024-09-21T07:44:55.580000+0000","last_became_peered":"2024-09-21T07:44:55.580000+0000","last_unstale":"2024-09-21T07:44:57.411031+0000","last_undegraded":"2024-09-21T07:44:57.411031+0000","last_fullsized":"2024-09-21T07:44:57.411031+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:57:56.224315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"133'2","reported_seq":17,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410640+0000","last_change":"2024-09-21T07:44:54.578311+0000","last_active":"2024-09-21T07:44:57.410640+0000","last_peered":"2024-09-21T07:44:57.410640+0000","last_clean":"2024-09-21T07:44:57.410640+0000","last_became_active":"2024-09-21T07:44:54.577821+0000","last_became_peered":"2024-09-21T07:44:54.577821+0000","last_unstale":"2024-09-21T07:44:57.410640+0000","last_undegraded":"2024-09-21T07:44:57.410640+0000","last_fullsized":"2024-09-21T07:44:57.410640+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:06.744686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":144,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576903+0000","last_change":"2024-09-21T07:35:27.868448+0000","last_active":"2024-09-21T07:44:56.576903+0000","last_peered":"2024-09-21T07:44:56.576903+0000","last_clean":"2024-09-21T07:44:56.576903+0000","last_became_active":"2024-09-21T07:35:27.868105+0000","last_became_peered":"2024-09-21T07:35:27.868105+0000","last_unstale":"2024-09-21T07:44:56.576903+0000","last_undegraded":"2024-09-21T07:44:56.576903+0000","last_fullsized":"2024-09-21T07:44:56.576903+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:41.468702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":153,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409715+0000","last_change":"2024-09-21T07:35:29.492974+0000","last_active":"2024-09-21T07:44:57.409715+0000","last_peered":"2024-09-21T07:44:57.409715+0000","last_clean":"2024-09-21T07:44:57.409715+0000","last_became_active":"2024-09-21T07:35:26.872165+0000","last_became_peered":"2024-09-21T07:35:26.872165+0000","last_unstale":"2024-09-21T07:44:57.409715+0000","last_undegraded":"2024-09-21T07:44:57.409715+0000","last_fullsized":"2024-09-21T07:44:57.409715+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:59.912520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013795179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":205,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409293+0000","last_change":"2024-09-21T07:34:25.084861+0000","last_active":"2024-09-21T07:44:57.409293+0000","last_peered":"2024-09-21T07:44:57.409293+0000","last_clean":"2024-09-21T07:44:57.409293+0000","last_became_active":"2024-09-21T07:34:23.277708+0000","last_became_peered":"2024-09-21T07:34:23.277708+0000","last_unstale":"2024-09-21T07:44:57.409293+0000","last_undegraded":"2024-09-21T07:44:57.409293+0000","last_fullsized":"2024-09-21T07:44:57.409293+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:40.816488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000597566,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.575498+0000","last_change":"2024-09-21T07:44:55.579831+0000","last_active":"2024-09-21T07:44:56.575498+0000","last_peered":"2024-09-21T07:44:56.575498+0000","last_clean":"2024-09-21T07:44:56.575498+0000","last_became_active":"2024-09-21T07:44:55.578589+0000","last_became_peered":"2024-09-21T07:44:55.578589+0000","last_unstale":"2024-09-21T07:44:56.575498+0000","last_undegraded":"2024-09-21T07:44:56.575498+0000","last_fullsized":"2024-09-21T07:44:56.575498+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:53.300887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"133'1","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410195+0000","last_change":"2024-09-21T07:44:54.575177+0000","last_active":"2024-09-21T07:44:57.410195+0000","last_peered":"2024-09-21T07:44:57.410195+0000","last_clean":"2024-09-21T07:44:57.410195+0000","last_became_active":"2024-09-21T07:44:54.574885+0000","last_became_peered":"2024-09-21T07:44:54.574885+0000","last_unstale":"2024-09-21T07:44:57.410195+0000","last_undegraded":"2024-09-21T07:44:57.410195+0000","last_fullsized":"2024-09-21T07:44:57.410195+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:12.054029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":144,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.578033+0000","last_change":"2024-09-21T07:35:27.874640+0000","last_active":"2024-09-21T07:44:56.578033+0000","last_peered":"2024-09-21T07:44:56.578033+0000","last_clean":"2024-09-21T07:44:56.578033+0000","last_became_active":"2024-09-21T07:35:27.874265+0000","last_became_peered":"2024-09-21T07:35:27.874265+0000","last_unstale":"2024-09-21T07:44:56.578033+0000","last_undegraded":"2024-09-21T07:44:56.578033+0000","last_fullsized":"2024-09-21T07:44:56.578033+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:45.754654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":152,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409564+0000","last_change":"2024-09-21T07:35:29.207392+0000","last_active":"2024-09-21T07:44:57.409564+0000","last_peered":"2024-09-21T07:44:57.409564+0000","last_clean":"2024-09-21T07:44:57.409564+0000","last_became_active":"2024-09-21T07:35:26.870917+0000","last_became_peered":"2024-09-21T07:35:26.870917+0000","last_unstale":"2024-09-21T07:44:57.409564+0000","last_undegraded":"2024-09-21T07:44:57.409564+0000","last_fullsized":"2024-09-21T07:44:57.409564+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:59.159937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059198499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":205,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.573965+0000","last_change":"2024-09-21T07:34:24.258709+0000","last_active":"2024-09-21T07:44:55.573965+0000","last_peered":"2024-09-21T07:44:55.573965+0000","last_clean":"2024-09-21T07:44:55.573965+0000","last_became_active":"2024-09-21T07:34:23.279042+0000","last_became_peered":"2024-09-21T07:34:23.279042+0000","last_unstale":"2024-09-21T07:44:55.573965+0000","last_undegraded":"2024-09-21T07:44:55.573965+0000","last_fullsized":"2024-09-21T07:44:55.573965+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:01.762117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019282100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.408953+0000","last_change":"2024-09-21T07:44:55.586454+0000","last_active":"2024-09-21T07:44:57.408953+0000","last_peered":"2024-09-21T07:44:57.408953+0000","last_clean":"2024-09-21T07:44:57.408953+0000","last_became_active":"2024-09-21T07:44:55.586082+0000","last_became_peered":"2024-09-21T07:44:55.586082+0000","last_unstale":"2024-09-21T07:44:57.408953+0000","last_undegraded":"2024-09-21T07:44:57.408953+0000","last_fullsized":"2024-09-21T07:44:57.408953+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:55.284231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410765+0000","last_change":"2024-09-21T07:44:54.571759+0000","last_active":"2024-09-21T07:44:57.410765+0000","last_peered":"2024-09-21T07:44:57.410765+0000","last_clean":"2024-09-21T07:44:57.410765+0000","last_became_active":"2024-09-21T07:44:54.571430+0000","last_became_peered":"2024-09-21T07:44:54.571430+0000","last_unstale":"2024-09-21T07:44:57.410765+0000","last_undegraded":"2024-09-21T07:44:57.410765+0000","last_fullsized":"2024-09-21T07:44:57.410765+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:22.572592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":146,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.411128+0000","last_change":"2024-09-21T07:35:27.870260+0000","last_active":"2024-09-21T07:44:57.411128+0000","last_peered":"2024-09-21T07:44:57.411128+0000","last_clean":"2024-09-21T07:44:57.411128+0000","last_became_active":"2024-09-21T07:35:27.869448+0000","last_became_peered":"2024-09-21T07:35:27.869448+0000","last_unstale":"2024-09-21T07:44:57.411128+0000","last_undegraded":"2024-09-21T07:44:57.411128+0000","last_fullsized":"2024-09-21T07:44:57.411128+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:16.946790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":151,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.577906+0000","last_change":"2024-09-21T07:35:29.386367+0000","last_active":"2024-09-21T07:44:56.577906+0000","last_peered":"2024-09-21T07:44:56.577906+0000","last_clean":"2024-09-21T07:44:56.577906+0000","last_became_active":"2024-09-21T07:35:26.879238+0000","last_became_peered":"2024-09-21T07:35:26.879238+0000","last_unstale":"2024-09-21T07:44:56.577906+0000","last_undegraded":"2024-09-21T07:44:56.577906+0000","last_fullsized":"2024-09-21T07:44:56.577906+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:16.528260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092223100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":205,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410642+0000","last_change":"2024-09-21T07:34:25.085635+0000","last_active":"2024-09-21T07:44:57.410642+0000","last_peered":"2024-09-21T07:44:57.410642+0000","last_clean":"2024-09-21T07:44:57.410642+0000","last_became_active":"2024-09-21T07:34:23.273566+0000","last_became_peered":"2024-09-21T07:34:23.273566+0000","last_unstale":"2024-09-21T07:44:57.410642+0000","last_undegraded":"2024-09-21T07:44:57.410642+0000","last_fullsized":"2024-09-21T07:44:57.410642+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:14.142272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079897300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.574972+0000","last_change":"2024-09-21T07:44:55.594648+0000","last_active":"2024-09-21T07:44:56.574972+0000","last_peered":"2024-09-21T07:44:56.574972+0000","last_clean":"2024-09-21T07:44:56.574972+0000","last_became_active":"2024-09-21T07:44:55.591974+0000","last_became_peered":"2024-09-21T07:44:55.591974+0000","last_unstale":"2024-09-21T07:44:56.574972+0000","last_undegraded":"2024-09-21T07:44:56.574972+0000","last_fullsized":"2024-09-21T07:44:56.574972+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:53:07.533223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.408008+0000","last_change":"2024-09-21T07:44:54.569509+0000","last_active":"2024-09-21T07:44:57.408008+0000","last_peered":"2024-09-21T07:44:57.408008+0000","last_clean":"2024-09-21T07:44:57.408008+0000","last_became_active":"2024-09-21T07:44:54.569173+0000","last_became_peered":"2024-09-21T07:44:54.569173+0000","last_unstale":"2024-09-21T07:44:57.408008+0000","last_undegraded":"2024-09-21T07:44:57.408008+0000","last_fullsized":"2024-09-21T07:44:57.408008+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:04.900165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":146,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410112+0000","last_change":"2024-09-21T07:35:27.878728+0000","last_active":"2024-09-21T07:44:57.410112+0000","last_peered":"2024-09-21T07:44:57.410112+0000","last_clean":"2024-09-21T07:44:57.410112+0000","last_became_active":"2024-09-21T07:35:27.878335+0000","last_became_peered":"2024-09-21T07:35:27.878335+0000","last_unstale":"2024-09-21T07:44:57.410112+0000","last_undegraded":"2024-09-21T07:44:57.410112+0000","last_fullsized":"2024-09-21T07:44:57.410112+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:25:37.731771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":156,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576819+0000","last_change":"2024-09-21T07:35:28.858418+0000","last_active":"2024-09-21T07:44:56.576819+0000","last_peered":"2024-09-21T07:44:56.576819+0000","last_clean":"2024-09-21T07:44:56.576819+0000","last_became_active":"2024-09-21T07:35:26.876944+0000","last_became_peered":"2024-09-21T07:35:26.876944+0000","last_unstale":"2024-09-21T07:44:56.576819+0000","last_undegraded":"2024-09-21T07:44:56.576819+0000","last_fullsized":"2024-09-21T07:44:56.576819+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:04.514304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090862799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":201,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.578579+0000","last_change":"2024-09-21T07:34:24.597926+0000","last_active":"2024-09-21T07:44:56.578579+0000","last_peered":"2024-09-21T07:44:56.578579+0000","last_clean":"2024-09-21T07:44:56.578579+0000","last_became_active":"2024-09-21T07:34:23.268926+0000","last_became_peered":"2024-09-21T07:34:23.268926+0000","last_unstale":"2024-09-21T07:44:56.578579+0000","last_undegraded":"2024-09-21T07:44:56.578579+0000","last_fullsized":"2024-09-21T07:44:56.578579+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:46.190752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101851,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409774+0000","last_change":"2024-09-21T07:44:55.578405+0000","last_active":"2024-09-21T07:44:57.409774+0000","last_peered":"2024-09-21T07:44:57.409774+0000","last_clean":"2024-09-21T07:44:57.409774+0000","last_became_active":"2024-09-21T07:44:55.578106+0000","last_became_peered":"2024-09-21T07:44:55.578106+0000","last_unstale":"2024-09-21T07:44:57.409774+0000","last_undegraded":"2024-09-21T07:44:57.409774+0000","last_fullsized":"2024-09-21T07:44:57.409774+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:40.279797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576910+0000","last_change":"2024-09-21T07:44:54.567157+0000","last_active":"2024-09-21T07:44:56.576910+0000","last_peered":"2024-09-21T07:44:56.576910+0000","last_clean":"2024-09-21T07:44:56.576910+0000","last_became_active":"2024-09-21T07:44:54.566497+0000","last_became_peered":"2024-09-21T07:44:54.566497+0000","last_unstale":"2024-09-21T07:44:56.576910+0000","last_undegraded":"2024-09-21T07:44:56.576910+0000","last_fullsized":"2024-09-21T07:44:56.576910+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:15.695364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":145,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.578283+0000","last_change":"2024-09-21T07:35:27.877426+0000","last_active":"2024-09-21T07:44:56.578283+0000","last_peered":"2024-09-21T07:44:56.578283+0000","last_clean":"2024-09-21T07:44:56.578283+0000","last_became_active":"2024-09-21T07:35:27.877050+0000","last_became_peered":"2024-09-21T07:35:27.877050+0000","last_unstale":"2024-09-21T07:44:56.578283+0000","last_undegraded":"2024-09-21T07:44:56.578283+0000","last_fullsized":"2024-09-21T07:44:56.578283+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:36.268334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":153,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409425+0000","last_change":"2024-09-21T07:35:28.862273+0000","last_active":"2024-09-21T07:44:57.409425+0000","last_peered":"2024-09-21T07:44:57.409425+0000","last_clean":"2024-09-21T07:44:57.409425+0000","last_became_active":"2024-09-21T07:35:26.874011+0000","last_became_peered":"2024-09-21T07:35:26.874011+0000","last_unstale":"2024-09-21T07:44:57.409425+0000","last_undegraded":"2024-09-21T07:44:57.409425+0000","last_fullsized":"2024-09-21T07:44:57.409425+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:11.986950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071644799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":206,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.925678+0000","last_change":"2024-09-21T07:34:24.259794+0000","last_active":"2024-09-21T07:44:56.925678+0000","last_peered":"2024-09-21T07:44:56.925678+0000","last_clean":"2024-09-21T07:44:56.925678+0000","last_became_active":"2024-09-21T07:34:23.265679+0000","last_became_peered":"2024-09-21T07:34:23.265679+0000","last_unstale":"2024-09-21T07:44:56.925678+0000","last_undegraded":"2024-09-21T07:44:56.925678+0000","last_fullsized":"2024-09-21T07:44:56.925678+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:02.575796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001298518,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.596670+0000","last_change":"2024-09-21T07:44:55.582406+0000","last_active":"2024-09-21T07:44:55.596670+0000","last_peered":"2024-09-21T07:44:55.596670+0000","last_clean":"2024-09-21T07:44:55.596670+0000","last_became_active":"2024-09-21T07:44:55.581499+0000","last_became_peered":"2024-09-21T07:44:55.581499+0000","last_unstale":"2024-09-21T07:44:55.596670+0000","last_undegraded":"2024-09-21T07:44:55.596670+0000","last_fullsized":"2024-09-21T07:44:55.596670+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:30.495936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576728+0000","last_change":"2024-09-21T07:44:54.568321+0000","last_active":"2024-09-21T07:44:56.576728+0000","last_peered":"2024-09-21T07:44:56.576728+0000","last_clean":"2024-09-21T07:44:56.576728+0000","last_became_active":"2024-09-21T07:44:54.566990+0000","last_became_peered":"2024-09-21T07:44:54.566990+0000","last_unstale":"2024-09-21T07:44:56.576728+0000","last_undegraded":"2024-09-21T07:44:56.576728+0000","last_fullsized":"2024-09-21T07:44:56.576728+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:10:56.227159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":146,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409554+0000","last_change":"2024-09-21T07:35:27.877725+0000","last_active":"2024-09-21T07:44:57.409554+0000","last_peered":"2024-09-21T07:44:57.409554+0000","last_clean":"2024-09-21T07:44:57.409554+0000","last_became_active":"2024-09-21T07:35:27.877398+0000","last_became_peered":"2024-09-21T07:35:27.877398+0000","last_unstale":"2024-09-21T07:44:57.409554+0000","last_undegraded":"2024-09-21T07:44:57.409554+0000","last_fullsized":"2024-09-21T07:44:57.409554+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:05.859762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":153,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.925279+0000","last_change":"2024-09-21T07:35:28.860612+0000","last_active":"2024-09-21T07:44:56.925279+0000","last_peered":"2024-09-21T07:44:56.925279+0000","last_clean":"2024-09-21T07:44:56.925279+0000","last_became_active":"2024-09-21T07:35:26.876562+0000","last_became_peered":"2024-09-21T07:35:26.876562+0000","last_unstale":"2024-09-21T07:44:56.925279+0000","last_undegraded":"2024-09-21T07:44:56.925279+0000","last_fullsized":"2024-09-21T07:44:56.925279+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:06.824331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012252470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":207,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410078+0000","last_change":"2024-09-21T07:34:24.260330+0000","last_active":"2024-09-21T07:44:57.410078+0000","last_peered":"2024-09-21T07:44:57.410078+0000","last_clean":"2024-09-21T07:44:57.410078+0000","last_became_active":"2024-09-21T07:34:23.637210+0000","last_became_peered":"2024-09-21T07:34:23.637210+0000","last_unstale":"2024-09-21T07:44:57.410078+0000","last_undegraded":"2024-09-21T07:44:57.410078+0000","last_fullsized":"2024-09-21T07:44:57.410078+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.488451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001537093,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410266+0000","last_change":"2024-09-21T07:44:55.582996+0000","last_active":"2024-09-21T07:44:57.410266+0000","last_peered":"2024-09-21T07:44:57.410266+0000","last_clean":"2024-09-21T07:44:57.410266+0000","last_became_active":"2024-09-21T07:44:55.582596+0000","last_became_peered":"2024-09-21T07:44:55.582596+0000","last_unstale":"2024-09-21T07:44:57.410266+0000","last_undegraded":"2024-09-21T07:44:57.410266+0000","last_fullsized":"2024-09-21T07:44:57.410266+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:51:28.363844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"133'1","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409762+0000","last_change":"2024-09-21T07:44:54.576648+0000","last_active":"2024-09-21T07:44:57.409762+0000","last_peered":"2024-09-21T07:44:57.409762+0000","last_clean":"2024-09-21T07:44:57.409762+0000","last_became_active":"2024-09-21T07:44:54.576284+0000","last_became_peered":"2024-09-21T07:44:54.576284+0000","last_unstale":"2024-09-21T07:44:57.409762+0000","last_undegraded":"2024-09-21T07:44:57.409762+0000","last_fullsized":"2024-09-21T07:44:57.409762+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:25:04.928717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":147,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.408896+0000","last_change":"2024-09-21T07:35:27.874757+0000","last_active":"2024-09-21T07:44:57.408896+0000","last_peered":"2024-09-21T07:44:57.408896+0000","last_clean":"2024-09-21T07:44:57.408896+0000","last_became_active":"2024-09-21T07:35:27.874360+0000","last_became_peered":"2024-09-21T07:35:27.874360+0000","last_unstale":"2024-09-21T07:44:57.408896+0000","last_undegraded":"2024-09-21T07:44:57.408896+0000","last_fullsized":"2024-09-21T07:44:57.408896+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:17.646912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":153,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410602+0000","last_change":"2024-09-21T07:35:29.493431+0000","last_active":"2024-09-21T07:44:57.410602+0000","last_peered":"2024-09-21T07:44:57.410602+0000","last_clean":"2024-09-21T07:44:57.410602+0000","last_became_active":"2024-09-21T07:35:26.873691+0000","last_became_peered":"2024-09-21T07:35:26.873691+0000","last_unstale":"2024-09-21T07:44:57.410602+0000","last_undegraded":"2024-09-21T07:44:57.410602+0000","last_fullsized":"2024-09-21T07:44:57.410602+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:51.705051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001162464,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":201,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.577291+0000","last_change":"2024-09-21T07:34:24.597286+0000","last_active":"2024-09-21T07:44:56.577291+0000","last_peered":"2024-09-21T07:44:56.577291+0000","last_clean":"2024-09-21T07:44:56.577291+0000","last_became_active":"2024-09-21T07:34:23.267133+0000","last_became_peered":"2024-09-21T07:34:23.267133+0000","last_unstale":"2024-09-21T07:44:56.577291+0000","last_undegraded":"2024-09-21T07:44:56.577291+0000","last_fullsized":"2024-09-21T07:44:56.577291+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:18.257010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576660+0000","last_change":"2024-09-21T07:44:55.584018+0000","last_active":"2024-09-21T07:44:56.576660+0000","last_peered":"2024-09-21T07:44:56.576660+0000","last_clean":"2024-09-21T07:44:56.576660+0000","last_became_active":"2024-09-21T07:44:55.583638+0000","last_became_peered":"2024-09-21T07:44:55.583638+0000","last_unstale":"2024-09-21T07:44:56.576660+0000","last_undegraded":"2024-09-21T07:44:56.576660+0000","last_fullsized":"2024-09-21T07:44:56.576660+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:48.999240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409744+0000","last_change":"2024-09-21T07:44:54.579096+0000","last_active":"2024-09-21T07:44:57.409744+0000","last_peered":"2024-09-21T07:44:57.409744+0000","last_clean":"2024-09-21T07:44:57.409744+0000","last_became_active":"2024-09-21T07:44:54.578668+0000","last_became_peered":"2024-09-21T07:44:54.578668+0000","last_unstale":"2024-09-21T07:44:57.409744+0000","last_undegraded":"2024-09-21T07:44:57.409744+0000","last_fullsized":"2024-09-21T07:44:57.409744+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:36:50.194028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":150,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576044+0000","last_change":"2024-09-21T07:35:27.864458+0000","last_active":"2024-09-21T07:44:56.576044+0000","last_peered":"2024-09-21T07:44:56.576044+0000","last_clean":"2024-09-21T07:44:56.576044+0000","last_became_active":"2024-09-21T07:35:27.864136+0000","last_became_peered":"2024-09-21T07:35:27.864136+0000","last_unstale":"2024-09-21T07:44:56.576044+0000","last_undegraded":"2024-09-21T07:44:56.576044+0000","last_fullsized":"2024-09-21T07:44:56.576044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:20.162240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":156,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576292+0000","last_change":"2024-09-21T07:35:28.858784+0000","last_active":"2024-09-21T07:44:56.576292+0000","last_peered":"2024-09-21T07:44:56.576292+0000","last_clean":"2024-09-21T07:44:56.576292+0000","last_became_active":"2024-09-21T07:35:26.876890+0000","last_became_peered":"2024-09-21T07:35:26.876890+0000","last_unstale":"2024-09-21T07:44:56.576292+0000","last_undegraded":"2024-09-21T07:44:56.576292+0000","last_fullsized":"2024-09-21T07:44:56.576292+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:40.738327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064543499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":205,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.567319+0000","last_change":"2024-09-21T07:34:24.258712+0000","last_active":"2024-09-21T07:44:55.567319+0000","last_peered":"2024-09-21T07:44:55.567319+0000","last_clean":"2024-09-21T07:44:55.567319+0000","last_became_active":"2024-09-21T07:34:23.274692+0000","last_became_peered":"2024-09-21T07:34:23.274692+0000","last_unstale":"2024-09-21T07:44:55.567319+0000","last_undegraded":"2024-09-21T07:44:55.567319+0000","last_fullsized":"2024-09-21T07:44:55.567319+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:43.279454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018051479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576870+0000","last_change":"2024-09-21T07:44:55.561016+0000","last_active":"2024-09-21T07:44:56.576870+0000","last_peered":"2024-09-21T07:44:56.576870+0000","last_clean":"2024-09-21T07:44:56.576870+0000","last_became_active":"2024-09-21T07:44:55.560410+0000","last_became_peered":"2024-09-21T07:44:55.560410+0000","last_unstale":"2024-09-21T07:44:56.576870+0000","last_undegraded":"2024-09-21T07:44:56.576870+0000","last_fullsized":"2024-09-21T07:44:56.576870+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:51.565447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"133'1","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576017+0000","last_change":"2024-09-21T07:44:54.575744+0000","last_active":"2024-09-21T07:44:56.576017+0000","last_peered":"2024-09-21T07:44:56.576017+0000","last_clean":"2024-09-21T07:44:56.576017+0000","last_became_active":"2024-09-21T07:44:54.574915+0000","last_became_peered":"2024-09-21T07:44:54.574915+0000","last_unstale":"2024-09-21T07:44:56.576017+0000","last_undegraded":"2024-09-21T07:44:56.576017+0000","last_fullsized":"2024-09-21T07:44:56.576017+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:22:22.749559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":144,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.565382+0000","last_change":"2024-09-21T07:35:27.878020+0000","last_active":"2024-09-21T07:44:55.565382+0000","last_peered":"2024-09-21T07:44:55.565382+0000","last_clean":"2024-09-21T07:44:55.565382+0000","last_became_active":"2024-09-21T07:35:27.877532+0000","last_became_peered":"2024-09-21T07:35:27.877532+0000","last_unstale":"2024-09-21T07:44:55.565382+0000","last_undegraded":"2024-09-21T07:44:55.565382+0000","last_fullsized":"2024-09-21T07:44:55.565382+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:19.756073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":153,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.925641+0000","last_change":"2024-09-21T07:35:28.861049+0000","last_active":"2024-09-21T07:44:56.925641+0000","last_peered":"2024-09-21T07:44:56.925641+0000","last_clean":"2024-09-21T07:44:56.925641+0000","last_became_active":"2024-09-21T07:35:26.876819+0000","last_became_peered":"2024-09-21T07:35:26.876819+0000","last_unstale":"2024-09-21T07:44:56.925641+0000","last_undegraded":"2024-09-21T07:44:56.925641+0000","last_fullsized":"2024-09-21T07:44:56.925641+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:19.633803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055780599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":205,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409032+0000","last_change":"2024-09-21T07:34:25.084849+0000","last_active":"2024-09-21T07:44:57.409032+0000","last_peered":"2024-09-21T07:44:57.409032+0000","last_clean":"2024-09-21T07:44:57.409032+0000","last_became_active":"2024-09-21T07:34:23.277713+0000","last_became_peered":"2024-09-21T07:34:23.277713+0000","last_unstale":"2024-09-21T07:44:57.409032+0000","last_undegraded":"2024-09-21T07:44:57.409032+0000","last_fullsized":"2024-09-21T07:44:57.409032+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:05.811810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059079499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.578298+0000","last_change":"2024-09-21T07:44:55.593875+0000","last_active":"2024-09-21T07:44:56.578298+0000","last_peered":"2024-09-21T07:44:56.578298+0000","last_clean":"2024-09-21T07:44:56.578298+0000","last_became_active":"2024-09-21T07:44:55.593403+0000","last_became_peered":"2024-09-21T07:44:55.593403+0000","last_unstale":"2024-09-21T07:44:56.578298+0000","last_undegraded":"2024-09-21T07:44:56.578298+0000","last_fullsized":"2024-09-21T07:44:56.578298+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:51.691583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"133'2","reported_seq":17,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.925831+0000","last_change":"2024-09-21T07:44:54.571773+0000","last_active":"2024-09-21T07:44:56.925831+0000","last_peered":"2024-09-21T07:44:56.925831+0000","last_clean":"2024-09-21T07:44:56.925831+0000","last_became_active":"2024-09-21T07:44:54.571422+0000","last_became_peered":"2024-09-21T07:44:54.571422+0000","last_unstale":"2024-09-21T07:44:56.925831+0000","last_undegraded":"2024-09-21T07:44:56.925831+0000","last_fullsized":"2024-09-21T07:44:56.925831+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:14:01.746987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":148,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.575055+0000","last_change":"2024-09-21T07:35:27.884248+0000","last_active":"2024-09-21T07:44:56.575055+0000","last_peered":"2024-09-21T07:44:56.575055+0000","last_clean":"2024-09-21T07:44:56.575055+0000","last_became_active":"2024-09-21T07:35:27.883861+0000","last_became_peered":"2024-09-21T07:35:27.883861+0000","last_unstale":"2024-09-21T07:44:56.575055+0000","last_undegraded":"2024-09-21T07:44:56.575055+0000","last_fullsized":"2024-09-21T07:44:56.575055+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":153,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409424+0000","last_change":"2024-09-21T07:35:29.207135+0000","last_active":"2024-09-21T07:44:57.409424+0000","last_peered":"2024-09-21T07:44:57.409424+0000","last_clean":"2024-09-21T07:44:57.409424+0000","last_became_active":"2024-09-21T07:35:26.872787+0000","last_became_peered":"2024-09-21T07:35:26.872787+0000","last_unstale":"2024-09-21T07:44:57.409424+0000","last_undegraded":"2024-09-21T07:44:57.409424+0000","last_fullsized":"2024-09-21T07:44:57.409424+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:27.543217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011157509999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":209,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576158+0000","last_change":"2024-09-21T07:34:24.538610+0000","last_active":"2024-09-21T07:44:56.576158+0000","last_peered":"2024-09-21T07:44:56.576158+0000","last_clean":"2024-09-21T07:44:56.576158+0000","last_became_active":"2024-09-21T07:34:23.259700+0000","last_became_peered":"2024-09-21T07:34:23.259700+0000","last_unstale":"2024-09-21T07:44:56.576158+0000","last_undegraded":"2024-09-21T07:44:56.576158+0000","last_fullsized":"2024-09-21T07:44:56.576158+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:57.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070522399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.596780+0000","last_change":"2024-09-21T07:44:55.582701+0000","last_active":"2024-09-21T07:44:55.596780+0000","last_peered":"2024-09-21T07:44:55.596780+0000","last_clean":"2024-09-21T07:44:55.596780+0000","last_became_active":"2024-09-21T07:44:55.581337+0000","last_became_peered":"2024-09-21T07:44:55.581337+0000","last_unstale":"2024-09-21T07:44:55.596780+0000","last_undegraded":"2024-09-21T07:44:55.596780+0000","last_fullsized":"2024-09-21T07:44:55.596780+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:12:58.407288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.571597+0000","last_change":"2024-09-21T07:44:54.578278+0000","last_active":"2024-09-21T07:44:55.571597+0000","last_peered":"2024-09-21T07:44:55.571597+0000","last_clean":"2024-09-21T07:44:55.571597+0000","last_became_active":"2024-09-21T07:44:54.577982+0000","last_became_peered":"2024-09-21T07:44:54.577982+0000","last_unstale":"2024-09-21T07:44:55.571597+0000","last_undegraded":"2024-09-21T07:44:55.571597+0000","last_fullsized":"2024-09-21T07:44:55.571597+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:26.299821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":150,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576953+0000","last_change":"2024-09-21T07:35:27.868024+0000","last_active":"2024-09-21T07:44:56.576953+0000","last_peered":"2024-09-21T07:44:56.576953+0000","last_clean":"2024-09-21T07:44:56.576953+0000","last_became_active":"2024-09-21T07:35:27.867339+0000","last_became_peered":"2024-09-21T07:35:27.867339+0000","last_unstale":"2024-09-21T07:44:56.576953+0000","last_undegraded":"2024-09-21T07:44:56.576953+0000","last_fullsized":"2024-09-21T07:44:56.576953+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":157,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576079+0000","last_change":"2024-09-21T07:35:28.858505+0000","last_active":"2024-09-21T07:44:56.576079+0000","last_peered":"2024-09-21T07:44:56.576079+0000","last_clean":"2024-09-21T07:44:56.576079+0000","last_became_active":"2024-09-21T07:35:26.877320+0000","last_became_peered":"2024-09-21T07:35:26.877320+0000","last_unstale":"2024-09-21T07:44:56.576079+0000","last_undegraded":"2024-09-21T07:44:56.576079+0000","last_fullsized":"2024-09-21T07:44:56.576079+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:53.672782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244119999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":202,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.578434+0000","last_change":"2024-09-21T07:34:24.597597+0000","last_active":"2024-09-21T07:44:56.578434+0000","last_peered":"2024-09-21T07:44:56.578434+0000","last_clean":"2024-09-21T07:44:56.578434+0000","last_became_active":"2024-09-21T07:34:23.635204+0000","last_became_peered":"2024-09-21T07:34:23.635204+0000","last_unstale":"2024-09-21T07:44:56.578434+0000","last_undegraded":"2024-09-21T07:44:56.578434+0000","last_fullsized":"2024-09-21T07:44:56.578434+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:48.644789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081139199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.576000+0000","last_change":"2024-09-21T07:44:55.574364+0000","last_active":"2024-09-21T07:44:56.576000+0000","last_peered":"2024-09-21T07:44:56.576000+0000","last_clean":"2024-09-21T07:44:56.576000+0000","last_became_active":"2024-09-21T07:44:55.574006+0000","last_became_peered":"2024-09-21T07:44:55.574006+0000","last_unstale":"2024-09-21T07:44:56.576000+0000","last_undegraded":"2024-09-21T07:44:56.576000+0000","last_fullsized":"2024-09-21T07:44:56.576000+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:30.876997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.407975+0000","last_change":"2024-09-21T07:44:54.571003+0000","last_active":"2024-09-21T07:44:57.407975+0000","last_peered":"2024-09-21T07:44:57.407975+0000","last_clean":"2024-09-21T07:44:57.407975+0000","last_became_active":"2024-09-21T07:44:54.569949+0000","last_became_peered":"2024-09-21T07:44:54.569949+0000","last_unstale":"2024-09-21T07:44:57.407975+0000","last_undegraded":"2024-09-21T07:44:57.407975+0000","last_fullsized":"2024-09-21T07:44:57.407975+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:54:06.235195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":145,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409028+0000","last_change":"2024-09-21T07:35:27.877503+0000","last_active":"2024-09-21T07:44:57.409028+0000","last_peered":"2024-09-21T07:44:57.409028+0000","last_clean":"2024-09-21T07:44:57.409028+0000","last_became_active":"2024-09-21T07:35:27.875775+0000","last_became_peered":"2024-09-21T07:35:27.875775+0000","last_unstale":"2024-09-21T07:44:57.409028+0000","last_undegraded":"2024-09-21T07:44:57.409028+0000","last_fullsized":"2024-09-21T07:44:57.409028+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:49.291023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":154,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409042+0000","last_change":"2024-09-21T07:35:29.491412+0000","last_active":"2024-09-21T07:44:57.409042+0000","last_peered":"2024-09-21T07:44:57.409042+0000","last_clean":"2024-09-21T07:44:57.409042+0000","last_became_active":"2024-09-21T07:35:26.876404+0000","last_became_peered":"2024-09-21T07:35:26.876404+0000","last_unstale":"2024-09-21T07:44:57.409042+0000","last_undegraded":"2024-09-21T07:44:57.409042+0000","last_fullsized":"2024-09-21T07:44:57.409042+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063918000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":206,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.574970+0000","last_change":"2024-09-21T07:34:24.259026+0000","last_active":"2024-09-21T07:44:55.574970+0000","last_peered":"2024-09-21T07:44:55.574970+0000","last_clean":"2024-09-21T07:44:55.574970+0000","last_became_active":"2024-09-21T07:34:23.635456+0000","last_became_peered":"2024-09-21T07:34:23.635456+0000","last_unstale":"2024-09-21T07:44:55.574970+0000","last_undegraded":"2024-09-21T07:44:55.574970+0000","last_fullsized":"2024-09-21T07:44:55.574970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:09.592752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018654419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.408788+0000","last_change":"2024-09-21T07:44:55.569794+0000","last_active":"2024-09-21T07:44:57.408788+0000","last_peered":"2024-09-21T07:44:57.408788+0000","last_clean":"2024-09-21T07:44:57.408788+0000","last_became_active":"2024-09-21T07:44:55.568883+0000","last_became_peered":"2024-09-21T07:44:55.568883+0000","last_unstale":"2024-09-21T07:44:57.408788+0000","last_undegraded":"2024-09-21T07:44:57.408788+0000","last_fullsized":"2024-09-21T07:44:57.408788+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:01:20.752577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"133'2","reported_seq":17,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410005+0000","last_change":"2024-09-21T07:44:54.570827+0000","last_active":"2024-09-21T07:44:57.410005+0000","last_peered":"2024-09-21T07:44:57.410005+0000","last_clean":"2024-09-21T07:44:57.410005+0000","last_became_active":"2024-09-21T07:44:54.570516+0000","last_became_peered":"2024-09-21T07:44:54.570516+0000","last_unstale":"2024-09-21T07:44:57.410005+0000","last_undegraded":"2024-09-21T07:44:57.410005+0000","last_fullsized":"2024-09-21T07:44:57.410005+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:50.388307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":145,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410202+0000","last_change":"2024-09-21T07:35:27.871984+0000","last_active":"2024-09-21T07:44:57.410202+0000","last_peered":"2024-09-21T07:44:57.410202+0000","last_clean":"2024-09-21T07:44:57.410202+0000","last_became_active":"2024-09-21T07:35:27.870336+0000","last_became_peered":"2024-09-21T07:35:27.870336+0000","last_unstale":"2024-09-21T07:44:57.410202+0000","last_undegraded":"2024-09-21T07:44:57.410202+0000","last_fullsized":"2024-09-21T07:44:57.410202+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:57.120479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":154,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.925589+0000","last_change":"2024-09-21T07:35:28.860351+0000","last_active":"2024-09-21T07:44:56.925589+0000","last_peered":"2024-09-21T07:44:56.925589+0000","last_clean":"2024-09-21T07:44:56.925589+0000","last_became_active":"2024-09-21T07:35:26.872046+0000","last_became_peered":"2024-09-21T07:35:26.872046+0000","last_unstale":"2024-09-21T07:44:56.925589+0000","last_undegraded":"2024-09-21T07:44:56.925589+0000","last_fullsized":"2024-09-21T07:44:56.925589+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:47.074327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098384399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":207,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.410020+0000","last_change":"2024-09-21T07:34:24.260300+0000","last_active":"2024-09-21T07:44:57.410020+0000","last_peered":"2024-09-21T07:44:57.410020+0000","last_clean":"2024-09-21T07:44:57.410020+0000","last_became_active":"2024-09-21T07:34:23.270128+0000","last_became_peered":"2024-09-21T07:34:23.270128+0000","last_unstale":"2024-09-21T07:44:57.410020+0000","last_undegraded":"2024-09-21T07:44:57.410020+0000","last_fullsized":"2024-09-21T07:44:57.410020+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:25.260931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00176651,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-21T07:44:55.598052+0000","last_change":"2024-09-21T07:44:55.589128+0000","last_active":"2024-09-21T07:44:55.598052+0000","last_peered":"2024-09-21T07:44:55.598052+0000","last_clean":"2024-09-21T07:44:55.598052+0000","last_became_active":"2024-09-21T07:44:55.582134+0000","last_became_peered":"2024-09-21T07:44:55.582134+0000","last_unstale":"2024-09-21T07:44:55.598052+0000","last_undegraded":"2024-09-21T07:44:55.598052+0000","last_fullsized":"2024-09-21T07:44:55.598052+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:54.538063+0000","last_clean_scrub_stamp":"2024-09-21T07:44:54.538063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:26:27.669272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.408461+0000","last_change":"2024-09-21T07:44:54.571619+0000","last_active":"2024-09-21T07:44:57.408461+0000","last_peered":"2024-09-21T07:44:57.408461+0000","last_clean":"2024-09-21T07:44:57.408461+0000","last_became_active":"2024-09-21T07:44:54.571116+0000","last_became_peered":"2024-09-21T07:44:54.571116+0000","last_unstale":"2024-09-21T07:44:57.408461+0000","last_undegraded":"2024-09-21T07:44:57.408461+0000","last_fullsized":"2024-09-21T07:44:57.408461+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:44:53.535281+0000","last_clean_scrub_stamp":"2024-09-21T07:44:53.535281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:53.905447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":145,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409831+0000","last_change":"2024-09-21T07:35:27.877861+0000","last_active":"2024-09-21T07:44:57.409831+0000","last_peered":"2024-09-21T07:44:57.409831+0000","last_clean":"2024-09-21T07:44:57.409831+0000","last_became_active":"2024-09-21T07:35:27.877434+0000","last_became_peered":"2024-09-21T07:35:27.877434+0000","last_unstale":"2024-09-21T07:44:57.409831+0000","last_undegraded":"2024-09-21T07:44:57.409831+0000","last_fullsized":"2024-09-21T07:44:57.409831+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":154,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:57.409508+0000","last_change":"2024-09-21T07:35:28.862954+0000","last_active":"2024-09-21T07:44:57.409508+0000","last_peered":"2024-09-21T07:44:57.409508+0000","last_clean":"2024-09-21T07:44:57.409508+0000","last_became_active":"2024-09-21T07:35:26.870613+0000","last_became_peered":"2024-09-21T07:35:26.870613+0000","last_unstale":"2024-09-21T07:44:57.409508+0000","last_undegraded":"2024-09-21T07:44:57.409508+0000","last_fullsized":"2024-09-21T07:44:57.409508+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:06.901043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059842100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":206,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-21T07:44:56.575493+0000","last_change":"2024-09-21T07:34:25.028904+0000","last_active":"2024-09-21T07:44:56.575493+0000","last_peered":"2024-09-21T07:44:56.575493+0000","last_clean":"2024-09-21T07:44:56.575493+0000","last_became_active":"2024-09-21T07:34:23.260942+0000","last_became_peered":"2024-09-21T07:34:23.260942+0000","last_unstale":"2024-09-21T07:44:56.575493+0000","last_undegraded":"2024-09-21T07:44:56.575493+0000","last_fullsized":"2024-09-21T07:44:56.575493+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:22.709328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060553899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509586,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33420,"kb_used_data":2228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338420,"statfs":{"total":96636764160,"available":96602542080,"internally_reserved":0,"allocated":2281472,"data_stored":1454525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509588,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32820,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339020,"statfs":{"total":96636764160,"available":96603156480,"internally_reserved":0,"allocated":1662976,"data_stored":856572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509586,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32836,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339004,"statfs":{"total":96636764160,"available":96603140096,"internally_reserved":0,"allocated":1679360,"data_stored":863541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509587,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32816,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339024,"statfs":{"total":96636764160,"available":96603160576,"internally_reserved":0,"allocated":1658880,"data_stored":855990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509588,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32832,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339008,"statfs":{"total":96636764160,"available":96603144192,"internally_reserved":0,"allocated":1675264,"data_stored":863403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509588,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33400,"kb_used_data":2204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338440,"statfs":{"total":96636764160,"available":96602562560,"internally_reserved":0,"allocated":2256896,"data_stored":1446380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542291,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32828,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339012,"statfs":{"total":96636764160,"available":96603148288,"internally_reserved":0,"allocated":1671168,"data_stored":857602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542290,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32824,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339016,"statfs":{"total":96636764160,"available":96603152384,"internally_reserved":0,"allocated":1675264,"data_stored":857636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:45:00.594 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T07:45:00.594 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T07:45:00.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.861+0000 7f1196c00640 1 -- 172.21.15.39:0/4112406187 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11900d0390 msgr2=0x7f11900d07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:00.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.861+0000 7f1196c00640 1 --2- 172.21.15.39:0/4112406187 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11900d0390 0x7f11900d07a0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f1188009be0 tx=0x7f118802a970 comp rx=0 tx=0).stop 2024-09-21T07:45:00.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.861+0000 7f1196c00640 1 -- 172.21.15.39:0/4112406187 shutdown_connections 2024-09-21T07:45:00.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.861+0000 7f1196c00640 1 --2- 172.21.15.39:0/4112406187 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11900d0390 0x7f11900d07a0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:00.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.861+0000 7f1196c00640 1 -- 172.21.15.39:0/4112406187 >> 172.21.15.39:0/4112406187 conn(0x7f119006cfc0 msgr2=0x7f119006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:00.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.861+0000 7f1196c00640 1 -- 172.21.15.39:0/4112406187 shutdown_connections 2024-09-21T07:45:00.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.861+0000 7f1196c00640 1 -- 172.21.15.39:0/4112406187 wait complete. 2024-09-21T07:45:00.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.863+0000 7f1196c00640 1 Processor -- start 2024-09-21T07:45:00.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.863+0000 7f1196c00640 1 -- start start 2024-09-21T07:45:00.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.865+0000 7f1196c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11900d0390 0x7f119015a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:00.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.865+0000 7f1196c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1190160250 0x7f1190145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:00.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.865+0000 7f1196c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1190160660 0x7f11901484a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:00.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.865+0000 7f1196c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f11900ccf30 con 0x7f1190160660 2024-09-21T07:45:00.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.865+0000 7f1196c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f11900ccdb0 con 0x7f1190160250 2024-09-21T07:45:00.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.865+0000 7f1196c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f11900cd0b0 con 0x7f11900d0390 2024-09-21T07:45:00.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.865+0000 7f1196200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1190160660 0x7f11901484a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:00.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.865+0000 7f1196200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1190160660 0x7f11901484a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48044/0 (socket says 172.21.15.39:48044) 2024-09-21T07:45:00.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.865+0000 7f1196200640 1 -- 172.21.15.39:0/2723014616 learned_addr learned my addr 172.21.15.39:0/2723014616 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:00.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.865+0000 7f1195800640 1 --2- 172.21.15.39:0/2723014616 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11900d0390 0x7f119015a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:00.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.865+0000 7f1194e00640 1 --2- 172.21.15.39:0/2723014616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1190160250 0x7f1190145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:00.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.865+0000 7f1196200640 1 -- 172.21.15.39:0/2723014616 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11900d0390 msgr2=0x7f119015a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:00.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.865+0000 7f1196200640 1 --2- 172.21.15.39:0/2723014616 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11900d0390 0x7f119015a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:00.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.865+0000 7f1196200640 1 -- 172.21.15.39:0/2723014616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1190160250 msgr2=0x7f1190145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:00.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.865+0000 7f1196200640 1 --2- 172.21.15.39:0/2723014616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1190160250 0x7f1190145bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:00.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.865+0000 7f1196200640 1 -- 172.21.15.39:0/2723014616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11901460f0 con 0x7f1190160660 2024-09-21T07:45:00.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.865+0000 7f1194e00640 1 --2- 172.21.15.39:0/2723014616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1190160250 0x7f1190145bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:45:00.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.866+0000 7f1196200640 1 --2- 172.21.15.39:0/2723014616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1190160660 0x7f11901484a0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f1188005fd0 tx=0x7f1188003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:00.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.866+0000 7f118d600640 1 -- 172.21.15.39:0/2723014616 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1188030c60 con 0x7f1190160660 2024-09-21T07:45:00.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.866+0000 7f118d600640 1 -- 172.21.15.39:0/2723014616 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f118803e070 con 0x7f1190160660 2024-09-21T07:45:00.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.866+0000 7f1196c00640 1 -- 172.21.15.39:0/2723014616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1190157c90 con 0x7f1190160660 2024-09-21T07:45:00.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.866+0000 7f118d600640 1 -- 172.21.15.39:0/2723014616 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11880322f0 con 0x7f1190160660 2024-09-21T07:45:00.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.866+0000 7f1196c00640 1 -- 172.21.15.39:0/2723014616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1190146870 con 0x7f1190160660 2024-09-21T07:45:00.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.867+0000 7f1196c00640 1 -- 172.21.15.39:0/2723014616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11900ce090 con 0x7f1190160660 2024-09-21T07:45:00.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.869+0000 7f118d600640 1 -- 172.21.15.39:0/2723014616 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1188032490 con 0x7f1190160660 2024-09-21T07:45:00.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.870+0000 7f118d600640 1 --2- 172.21.15.39:0/2723014616 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1170078b90 0x7f117007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:00.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.870+0000 7f118d600640 1 -- 172.21.15.39:0/2723014616 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 6800+0+0 (secure 0 0 0) 0x7f11880bc6c0 con 0x7f1190160660 2024-09-21T07:45:00.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.871+0000 7f1195800640 1 --2- 172.21.15.39:0/2723014616 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1170078b90 0x7f117007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:00.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.871+0000 7f1195800640 1 --2- 172.21.15.39:0/2723014616 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1170078b90 0x7f117007b0a0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f1184000f60 tx=0x7f1184009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:00.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:00.875+0000 7f118d600640 1 -- 172.21.15.39:0/2723014616 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f11900ce090 con 0x7f1190160660 2024-09-21T07:45:01.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.145+0000 7f1196c00640 1 -- 172.21.15.39:0/2723014616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f11900d07a0 con 0x7f1190160660 2024-09-21T07:45:01.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.153+0000 7f118d600640 1 -- 172.21.15.39:0/2723014616 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) ==== 190+0+147 (secure 0 0 0) 0x7f118808be20 con 0x7f1190160660 2024-09-21T07:45:01.155 INFO:teuthology.orchestra.run.smithi039.stdout:[client.0] 2024-09-21T07:45:01.155 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQB0du5mGfb1DBAA/aSMKR5or3xy53rB/iUjpA== 2024-09-21T07:45:01.155 INFO:teuthology.orchestra.run.smithi039.stdout: caps mds = "allow" 2024-09-21T07:45:01.155 INFO:teuthology.orchestra.run.smithi039.stdout: caps mon = "allow r" 2024-09-21T07:45:01.155 INFO:teuthology.orchestra.run.smithi039.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T07:45:01.155 INFO:teuthology.orchestra.run.smithi039.stderr:updated caps for client.0 2024-09-21T07:45:01.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.155+0000 7f1196c00640 1 -- 172.21.15.39:0/2723014616 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1170078b90 msgr2=0x7f117007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:01.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.155+0000 7f1196c00640 1 --2- 172.21.15.39:0/2723014616 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1170078b90 0x7f117007b0a0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f1184000f60 tx=0x7f1184009040 comp rx=0 tx=0).stop 2024-09-21T07:45:01.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.155+0000 7f1196c00640 1 -- 172.21.15.39:0/2723014616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1190160660 msgr2=0x7f11901484a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:01.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.155+0000 7f1196c00640 1 --2- 172.21.15.39:0/2723014616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1190160660 0x7f11901484a0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f1188005fd0 tx=0x7f1188003970 comp rx=0 tx=0).stop 2024-09-21T07:45:01.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.155+0000 7f1196c00640 1 -- 172.21.15.39:0/2723014616 shutdown_connections 2024-09-21T07:45:01.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.155+0000 7f1196c00640 1 --2- 172.21.15.39:0/2723014616 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1170078b90 0x7f117007b0a0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:01.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.155+0000 7f1196c00640 1 --2- 172.21.15.39:0/2723014616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1190160660 0x7f11901484a0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:01.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.155+0000 7f1196c00640 1 --2- 172.21.15.39:0/2723014616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1190160250 0x7f1190145bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:01.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.155+0000 7f1196c00640 1 --2- 172.21.15.39:0/2723014616 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11900d0390 0x7f119015a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:01.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.155+0000 7f1196c00640 1 -- 172.21.15.39:0/2723014616 >> 172.21.15.39:0/2723014616 conn(0x7f119006cfc0 msgr2=0x7f11900cf710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:01.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.155+0000 7f1196c00640 1 -- 172.21.15.39:0/2723014616 shutdown_connections 2024-09-21T07:45:01.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.156+0000 7f1196c00640 1 -- 172.21.15.39:0/2723014616 wait complete. 2024-09-21T07:45:01.184 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:45:01.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.408+0000 7f5afe000640 1 -- 172.21.15.39:0/163574486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5af815cc00 msgr2=0x7f5af815d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:01.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.408+0000 7f5afe000640 1 --2- 172.21.15.39:0/163574486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5af815cc00 0x7f5af815d010 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4009830 tx=0x7f5ae402a540 comp rx=0 tx=0).stop 2024-09-21T07:45:01.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.408+0000 7f5afe000640 1 -- 172.21.15.39:0/163574486 shutdown_connections 2024-09-21T07:45:01.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.408+0000 7f5afe000640 1 --2- 172.21.15.39:0/163574486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5af815cc00 0x7f5af815d010 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:01.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.408+0000 7f5afe000640 1 -- 172.21.15.39:0/163574486 >> 172.21.15.39:0/163574486 conn(0x7f5af806cfc0 msgr2=0x7f5af806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:01.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.409+0000 7f5afe000640 1 -- 172.21.15.39:0/163574486 shutdown_connections 2024-09-21T07:45:01.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.409+0000 7f5afe000640 1 -- 172.21.15.39:0/163574486 wait complete. 2024-09-21T07:45:01.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.411+0000 7f5afe000640 1 Processor -- start 2024-09-21T07:45:01.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.411+0000 7f5afe000640 1 -- start start 2024-09-21T07:45:01.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.412+0000 7f5afe000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5af80cfa20 0x7f5af814ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:01.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.412+0000 7f5afe000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5af80d04b0 0x7f5af81e94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:01.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.412+0000 7f5afe000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5af815cc00 0x7f5af81e9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:01.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.412+0000 7f5afe000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5af806f080 con 0x7f5af815cc00 2024-09-21T07:45:01.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.412+0000 7f5afe000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5af806ef00 con 0x7f5af80d04b0 2024-09-21T07:45:01.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.412+0000 7f5afe000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f5af806f200 con 0x7f5af80cfa20 2024-09-21T07:45:01.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.413+0000 7f5afd600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5af815cc00 0x7f5af81e9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:01.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.413+0000 7f5afd600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5af815cc00 0x7f5af81e9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48054/0 (socket says 172.21.15.39:48054) 2024-09-21T07:45:01.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.413+0000 7f5afd600640 1 -- 172.21.15.39:0/3095193158 learned_addr learned my addr 172.21.15.39:0/3095193158 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:01.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.413+0000 7f5af7e00640 1 --2- 172.21.15.39:0/3095193158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5af80d04b0 0x7f5af81e94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:01.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.413+0000 7f5afcc00640 1 --2- 172.21.15.39:0/3095193158 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5af80cfa20 0x7f5af814ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:01.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.413+0000 7f5afd600640 1 -- 172.21.15.39:0/3095193158 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5af80cfa20 msgr2=0x7f5af814ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:01.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.413+0000 7f5afd600640 1 --2- 172.21.15.39:0/3095193158 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5af80cfa20 0x7f5af814ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:01.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.413+0000 7f5afd600640 1 -- 172.21.15.39:0/3095193158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5af80d04b0 msgr2=0x7f5af81e94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:01.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.413+0000 7f5afd600640 1 --2- 172.21.15.39:0/3095193158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5af80d04b0 0x7f5af81e94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:01.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.413+0000 7f5afd600640 1 -- 172.21.15.39:0/3095193158 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af81e7030 con 0x7f5af815cc00 2024-09-21T07:45:01.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.413+0000 7f5afcc00640 1 --2- 172.21.15.39:0/3095193158 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5af80cfa20 0x7f5af814ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:45:01.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.413+0000 7f5afd600640 1 --2- 172.21.15.39:0/3095193158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5af815cc00 0x7f5af81e9a10 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f5af0001f40 tx=0x7f5af000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:01.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.414+0000 7f5af4c00640 1 -- 172.21.15.39:0/3095193158 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af0004a20 con 0x7f5af815cc00 2024-09-21T07:45:01.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.414+0000 7f5af4c00640 1 -- 172.21.15.39:0/3095193158 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5af0004cd0 con 0x7f5af815cc00 2024-09-21T07:45:01.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.414+0000 7f5afe000640 1 -- 172.21.15.39:0/3095193158 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5af8155850 con 0x7f5af815cc00 2024-09-21T07:45:01.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.414+0000 7f5af4c00640 1 -- 172.21.15.39:0/3095193158 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af0005290 con 0x7f5af815cc00 2024-09-21T07:45:01.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.414+0000 7f5afe000640 1 -- 172.21.15.39:0/3095193158 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5af8155e10 con 0x7f5af815cc00 2024-09-21T07:45:01.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.415+0000 7f5afe000640 1 -- 172.21.15.39:0/3095193158 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5af81e7200 con 0x7f5af815cc00 2024-09-21T07:45:01.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.416+0000 7f5af4c00640 1 -- 172.21.15.39:0/3095193158 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5af0005430 con 0x7f5af815cc00 2024-09-21T07:45:01.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.417+0000 7f5af4c00640 1 --2- 172.21.15.39:0/3095193158 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5ad8078b90 0x7f5ad807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:01.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.417+0000 7f5af4c00640 1 -- 172.21.15.39:0/3095193158 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 6800+0+0 (secure 0 0 0) 0x7f5af009bd00 con 0x7f5af815cc00 2024-09-21T07:45:01.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.418+0000 7f5afcc00640 1 --2- 172.21.15.39:0/3095193158 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5ad8078b90 0x7f5ad807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:01.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.418+0000 7f5afcc00640 1 --2- 172.21.15.39:0/3095193158 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5ad8078b90 0x7f5ad807b0a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f5aec000f60 tx=0x7f5aec009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:01.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.422+0000 7f5af4c00640 1 -- 172.21.15.39:0/3095193158 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5af0063d20 con 0x7f5af815cc00 2024-09-21T07:45:01.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.658+0000 7f5afe000640 1 -- 172.21.15.39:0/3095193158 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5af806f6f0 con 0x7f5af815cc00 2024-09-21T07:45:01.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.658+0000 7f5af4c00640 1 -- 172.21.15.39:0/3095193158 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 172 v172) ==== 95+0+5237 (secure 0 0 0) 0x7f5af0063620 con 0x7f5af815cc00 2024-09-21T07:45:01.660 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:45:01.661 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":172,"btime":"2024-09-21T07:44:58:579257+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7127,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3876879654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3876879654},{"type":"v1","addr":"172.21.15.148:6836","nonce":3876879654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":167},{"gid":7130,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3044588529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3044588529},{"type":"v1","addr":"172.21.15.148:6837","nonce":3044588529}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":168},{"gid":8517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/3334092885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3334092885},{"type":"v1","addr":"172.21.15.39:6834","nonce":3334092885}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":168}],"filesystems":[{"mdsmap":{"epoch":172,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:44:55.080850+0000","modified":"2024-09-21T07:44:58.083710+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8520":{"gid":8520,"name":"a","rank":0,"incarnation":169,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/3547777254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3547777254},{"type":"v1","addr":"172.21.15.39:6835","nonce":3547777254}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8520,"qdb_cluster":[8520]},"id":7}]} 2024-09-21T07:45:01.661 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 172 2024-09-21T07:45:01.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.661+0000 7f5afe000640 1 -- 172.21.15.39:0/3095193158 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5ad8078b90 msgr2=0x7f5ad807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:01.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.661+0000 7f5afe000640 1 --2- 172.21.15.39:0/3095193158 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5ad8078b90 0x7f5ad807b0a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f5aec000f60 tx=0x7f5aec009040 comp rx=0 tx=0).stop 2024-09-21T07:45:01.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.661+0000 7f5afe000640 1 -- 172.21.15.39:0/3095193158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5af815cc00 msgr2=0x7f5af81e9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:01.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.661+0000 7f5afe000640 1 --2- 172.21.15.39:0/3095193158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5af815cc00 0x7f5af81e9a10 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f5af0001f40 tx=0x7f5af000a430 comp rx=0 tx=0).stop 2024-09-21T07:45:01.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.661+0000 7f5afe000640 1 -- 172.21.15.39:0/3095193158 shutdown_connections 2024-09-21T07:45:01.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.661+0000 7f5afe000640 1 --2- 172.21.15.39:0/3095193158 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5ad8078b90 0x7f5ad807b0a0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:01.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.661+0000 7f5afe000640 1 --2- 172.21.15.39:0/3095193158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5af815cc00 0x7f5af81e9a10 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:01.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.661+0000 7f5afe000640 1 --2- 172.21.15.39:0/3095193158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5af80d04b0 0x7f5af81e94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:01.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.661+0000 7f5afe000640 1 --2- 172.21.15.39:0/3095193158 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5af80cfa20 0x7f5af814ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:01.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.661+0000 7f5afe000640 1 -- 172.21.15.39:0/3095193158 >> 172.21.15.39:0/3095193158 conn(0x7f5af806cfc0 msgr2=0x7f5af81ea270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:01.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.661+0000 7f5afe000640 1 -- 172.21.15.39:0/3095193158 shutdown_connections 2024-09-21T07:45:01.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:01.661+0000 7f5afe000640 1 -- 172.21.15.39:0/3095193158 wait complete. 2024-09-21T07:45:02.088 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 172, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T07:44:55.080850+0000', 'modified': '2024-09-21T07:44:58.083710+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8520}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8520': {'gid': 8520, 'name': 'a', 'rank': 0, 'incarnation': 169, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.39:6835/3547777254', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 3547777254}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 3547777254}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8520, 'qdb_cluster': [8520]} 2024-09-21T07:45:02.088 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T07:45:02.088 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T07:45:02.202 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:45:02.206 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:45:02.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.317+0000 7fb90a000640 1 -- 172.21.15.39:0/960860107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb90406ef90 msgr2=0x7fb90406f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:02.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.317+0000 7fb90a000640 1 --2- 172.21.15.39:0/960860107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb90406ef90 0x7fb90406f3a0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc005fd0 tx=0x7fb8fc02a8b0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.317+0000 7fb90a000640 1 -- 172.21.15.39:0/960860107 shutdown_connections 2024-09-21T07:45:02.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.317+0000 7fb90a000640 1 --2- 172.21.15.39:0/960860107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb90406ef90 0x7fb90406f3a0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.317+0000 7fb90a000640 1 --2- 172.21.15.39:0/960860107 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb9040b3d80 0x7fb9040ce2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.317+0000 7fb90a000640 1 -- 172.21.15.39:0/960860107 >> 172.21.15.39:0/960860107 conn(0x7fb9040b2460 msgr2=0x7fb90406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:02.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.317+0000 7fb90a000640 1 -- 172.21.15.39:0/960860107 shutdown_connections 2024-09-21T07:45:02.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.317+0000 7fb90a000640 1 -- 172.21.15.39:0/960860107 wait complete. 2024-09-21T07:45:02.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.319+0000 7fb90a000640 1 Processor -- start 2024-09-21T07:45:02.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.319+0000 7fb90a000640 1 -- start start 2024-09-21T07:45:02.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.321+0000 7fb90a000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb90406ef90 0x7fb9041f4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:02.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.321+0000 7fb90a000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9040b4c40 0x7fb9040b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:02.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.321+0000 7fb90a000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9040b4330 0x7fb904154fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:02.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.321+0000 7fb90a000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb90406fa60 con 0x7fb9040b4330 2024-09-21T07:45:02.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.321+0000 7fb90a000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb90406f8e0 con 0x7fb9040b4c40 2024-09-21T07:45:02.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.321+0000 7fb90a000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb90406fbe0 con 0x7fb90406ef90 2024-09-21T07:45:02.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.321+0000 7fb903e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9040b4330 0x7fb904154fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:02.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.321+0000 7fb903e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9040b4330 0x7fb904154fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53378/0 (socket says 172.21.15.39:53378) 2024-09-21T07:45:02.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.321+0000 7fb903e00640 1 -- 172.21.15.39:0/4200612222 learned_addr learned my addr 172.21.15.39:0/4200612222 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:02.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.321+0000 7fb903400640 1 --2- 172.21.15.39:0/4200612222 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb90406ef90 0x7fb9041f4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:02.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.321+0000 7fb8faa00640 1 --2- 172.21.15.39:0/4200612222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9040b4c40 0x7fb9040b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:02.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.322+0000 7fb903e00640 1 -- 172.21.15.39:0/4200612222 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb90406ef90 msgr2=0x7fb9041f4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:02.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.322+0000 7fb903e00640 1 --2- 172.21.15.39:0/4200612222 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb90406ef90 0x7fb9041f4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.322+0000 7fb903e00640 1 -- 172.21.15.39:0/4200612222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9040b4c40 msgr2=0x7fb9040b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:02.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.322+0000 7fb903e00640 1 --2- 172.21.15.39:0/4200612222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9040b4c40 0x7fb9040b3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.322+0000 7fb903e00640 1 -- 172.21.15.39:0/4200612222 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9041f1260 con 0x7fb9040b4330 2024-09-21T07:45:02.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.322+0000 7fb903e00640 1 --2- 172.21.15.39:0/4200612222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9040b4330 0x7fb904154fb0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc005fd0 tx=0x7fb8fc031760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:02.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.322+0000 7fb8fbe00640 1 -- 172.21.15.39:0/4200612222 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc031990 con 0x7fb9040b4330 2024-09-21T07:45:02.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.322+0000 7fb8fbe00640 1 -- 172.21.15.39:0/4200612222 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8fc031c20 con 0x7fb9040b4330 2024-09-21T07:45:02.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.322+0000 7fb90a000640 1 -- 172.21.15.39:0/4200612222 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb904153f10 con 0x7fb9040b4330 2024-09-21T07:45:02.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.322+0000 7fb8fbe00640 1 -- 172.21.15.39:0/4200612222 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc0330d0 con 0x7fb9040b4330 2024-09-21T07:45:02.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.322+0000 7fb90a000640 1 -- 172.21.15.39:0/4200612222 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9041544d0 con 0x7fb9040b4330 2024-09-21T07:45:02.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.325+0000 7fb8fbe00640 1 -- 172.21.15.39:0/4200612222 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb8fc033270 con 0x7fb9040b4330 2024-09-21T07:45:02.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.326+0000 7fb8fbe00640 1 --2- 172.21.15.39:0/4200612222 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb8e0078b90 0x7fb8e007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:02.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.326+0000 7fb8fbe00640 1 -- 172.21.15.39:0/4200612222 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 6800+0+0 (secure 0 0 0) 0x7fb8fc0bc5e0 con 0x7fb9040b4330 2024-09-21T07:45:02.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.326+0000 7fb903400640 1 --2- 172.21.15.39:0/4200612222 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb8e0078b90 0x7fb8e007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:02.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.327+0000 7fb9111da740 1 Processor -- start 2024-09-21T07:45:02.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.327+0000 7fb9111da740 1 -- start start 2024-09-21T07:45:02.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.327+0000 7fb903400640 1 --2- 172.21.15.39:0/4200612222 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb8e0078b90 0x7fb8e007b0a0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fb8f40046b0 tx=0x7fb8f4007eb0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:02.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.328+0000 7fb9111da740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5577e54b0270 0x5577e589ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:02.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.328+0000 7fb9111da740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5577e54873b0 0x5577e5850f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:02.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.328+0000 7fb9111da740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577e548f590 0x5577e58514a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:02.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.328+0000 7fb9111da740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5577e57da430 con 0x5577e548f590 2024-09-21T07:45:02.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.328+0000 7fb9111da740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5577e57ca5d0 con 0x5577e54873b0 2024-09-21T07:45:02.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.328+0000 7fb9111da740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x5577e57b9a00 con 0x5577e54b0270 2024-09-21T07:45:02.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.328+0000 7fb903e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577e548f590 0x5577e58514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:02.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.329+0000 7fb8faa00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5577e54873b0 0x5577e5850f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:02.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.329+0000 7fb903400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5577e54b0270 0x5577e589ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:02.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.329+0000 7fb8faa00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5577e54873b0 0x5577e5850f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.39:41790/0 (socket says 172.21.15.39:41790) 2024-09-21T07:45:02.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.329+0000 7fb903e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577e548f590 0x5577e58514a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53388/0 (socket says 172.21.15.39:53388) 2024-09-21T07:45:02.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.329+0000 7fb8faa00640 1 -- 172.21.15.39:0/1727580729 learned_addr learned my addr 172.21.15.39:0/1727580729 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:02.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.329+0000 7fb8faa00640 1 -- 172.21.15.39:0/1727580729 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5577e54b0270 msgr2=0x5577e589ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:45:02.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.329+0000 7fb8faa00640 1 --2- 172.21.15.39:0/1727580729 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5577e54b0270 0x5577e589ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.329+0000 7fb8faa00640 1 -- 172.21.15.39:0/1727580729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577e548f590 msgr2=0x5577e58514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:45:02.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.329+0000 7fb8faa00640 1 --2- 172.21.15.39:0/1727580729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577e548f590 0x5577e58514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.329+0000 7fb8faa00640 1 -- 172.21.15.39:0/1727580729 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577e5782520 con 0x5577e54873b0 2024-09-21T07:45:02.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.329+0000 7fb903e00640 1 --2- 172.21.15.39:0/1727580729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577e548f590 0x5577e58514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:45:02.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.329+0000 7fb8faa00640 1 --2- 172.21.15.39:0/1727580729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5577e54873b0 0x5577e5850f60 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00bca0 tx=0x7fb8ec0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e16dbf8c52d1607 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:02.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.330+0000 7fb8f3400640 1 -- 172.21.15.39:0/1727580729 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec004030 con 0x5577e54873b0 2024-09-21T07:45:02.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.330+0000 7fb8f3400640 1 -- 172.21.15.39:0/1727580729 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8ec0042c0 con 0x5577e54873b0 2024-09-21T07:45:02.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.330+0000 7fb8f3400640 1 -- 172.21.15.39:0/1727580729 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec003850 con 0x5577e54873b0 2024-09-21T07:45:02.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.330+0000 7fb9111da740 1 -- 172.21.15.39:0/1727580729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5577e54873b0 msgr2=0x5577e5850f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:02.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.330+0000 7fb9111da740 1 --2- 172.21.15.39:0/1727580729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5577e54873b0 0x5577e5850f60 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00bca0 tx=0x7fb8ec0107a0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.331+0000 7fb9111da740 1 -- 172.21.15.39:0/1727580729 shutdown_connections 2024-09-21T07:45:02.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.331+0000 7fb9111da740 1 --2- 172.21.15.39:0/1727580729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577e548f590 0x5577e58514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.331+0000 7fb9111da740 1 --2- 172.21.15.39:0/1727580729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5577e54873b0 0x5577e5850f60 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.331+0000 7fb9111da740 1 --2- 172.21.15.39:0/1727580729 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5577e54b0270 0x5577e589ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.331+0000 7fb9111da740 1 -- 172.21.15.39:0/1727580729 >> 172.21.15.39:0/1727580729 conn(0x5577e54a43c0 msgr2=0x5577e5856970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:02.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.331+0000 7fb9111da740 1 -- 172.21.15.39:0/1727580729 shutdown_connections 2024-09-21T07:45:02.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.331+0000 7fb9111da740 1 -- 172.21.15.39:0/1727580729 wait complete. 2024-09-21T07:45:02.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.332+0000 7fb9111da740 1 -- start start 2024-09-21T07:45:02.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.332+0000 7fb9111da740 1 Processor -- start 2024-09-21T07:45:02.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.334+0000 7fb9111da740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577e54873b0 0x5577e58943d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:02.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.334+0000 7fb9111da740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5577e548f590 0x5577e5878050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:02.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.334+0000 7fb9111da740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5577e54b0270 0x5577e58807c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:02.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.334+0000 7fb9111da740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5577e5760330 con 0x5577e54873b0 2024-09-21T07:45:02.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.334+0000 7fb9111da740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5577e57bd810 con 0x5577e54b0270 2024-09-21T07:45:02.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.334+0000 7fb9111da740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x5577e552a620 con 0x5577e548f590 2024-09-21T07:45:02.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.334+0000 7fb9111da740 10 client.0 _ll_register_callbacks cb 0x5577e5765420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:45:02.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.334+0000 7fb903400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577e54873b0 0x5577e58943d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:02.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.334+0000 7fb903400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577e54873b0 0x5577e58943d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53400/0 (socket says 172.21.15.39:53400) 2024-09-21T07:45:02.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.334+0000 7fb903400640 1 -- 172.21.15.39:0/2238358381 learned_addr learned my addr 172.21.15.39:0/2238358381 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:02.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.334+0000 7fb8faa00640 1 --2- 172.21.15.39:0/2238358381 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5577e548f590 0x5577e5878050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:02.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.334+0000 7fb903e00640 1 --2- 172.21.15.39:0/2238358381 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5577e54b0270 0x5577e58807c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:02.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.334+0000 7fb903400640 1 -- 172.21.15.39:0/2238358381 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5577e548f590 msgr2=0x5577e5878050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:45:02.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.334+0000 7fb903400640 1 --2- 172.21.15.39:0/2238358381 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5577e548f590 0x5577e5878050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.334+0000 7fb903400640 1 -- 172.21.15.39:0/2238358381 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5577e54b0270 msgr2=0x5577e58807c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:45:02.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.334+0000 7fb903400640 1 --2- 172.21.15.39:0/2238358381 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5577e54b0270 0x5577e58807c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.334+0000 7fb903400640 1 -- 172.21.15.39:0/2238358381 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577e551b850 con 0x5577e54873b0 2024-09-21T07:45:02.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.334+0000 7fb8faa00640 1 --2- 172.21.15.39:0/2238358381 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5577e548f590 0x5577e5878050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:45:02.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.334+0000 7fb903e00640 1 --2- 172.21.15.39:0/2238358381 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5577e54b0270 0x5577e58807c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:45:02.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.335+0000 7fb903400640 1 --2- 172.21.15.39:0/2238358381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577e54873b0 0x5577e58943d0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fb8f40146f0 tx=0x7fb8f4004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1848ae77b64a3140 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:02.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.335+0000 7fb8e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:45:02.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.335+0000 7fb8e7e00640 1 -- 172.21.15.39:0/2238358381 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f4025070 con 0x5577e54873b0 2024-09-21T07:45:02.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.335+0000 7fb8e7e00640 1 -- 172.21.15.39:0/2238358381 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 6800+0+0 (secure 0 0 0) 0x7fb8f4015850 con 0x5577e54873b0 2024-09-21T07:45:02.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.335+0000 7fb9111da740 1 -- 172.21.15.39:0/2238358381 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8ec0042c0 con 0x5577e54873b0 2024-09-21T07:45:02.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.335+0000 7fb8e7e00640 1 -- 172.21.15.39:0/2238358381 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=172) ==== 24+0+0 (secure 0 0 0) 0x7fb8f4030050 con 0x5577e54873b0 2024-09-21T07:45:02.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.335+0000 7fb9111da740 10 client.8637 fetch_fsmap learned FSMap version 172 2024-09-21T07:45:02.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.335+0000 7fb9111da740 1 -- 172.21.15.39:0/2238358381 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=172,monmap=2+}) -- 0x5577e55926b0 con 0x5577e54873b0 2024-09-21T07:45:02.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.335+0000 7fb8e7e00640 1 -- 172.21.15.39:0/2238358381 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8ec0042c0 con 0x5577e54873b0 2024-09-21T07:45:02.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.336+0000 7fb8e7e00640 1 -- 172.21.15.39:0/2238358381 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 172) ==== 3502+0+0 (secure 0 0 0) 0x7fb8f4033020 con 0x5577e54873b0 2024-09-21T07:45:02.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.336+0000 7fb9111da740 10 client.8637 fetch_fsmap finished waiting for FSMap version 172 2024-09-21T07:45:02.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.336+0000 7fb9111da740 10 client.8637 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:45:02.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.336+0000 7fb9111da740 20 client.8637 populate_metadata read hostname 'smithi039' 2024-09-21T07:45:02.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.336+0000 7fb9111da740 1 --2- 172.21.15.39:0/2238358381 >> [v2:172.21.15.39:6832/3547777254,v1:172.21.15.39:6835/3547777254] conn(0x5577e546ee40 0x5577e58b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:02.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.336+0000 7fb9111da740 4 client.8637 mds_command: new command op to 8520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:45:02.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.336+0000 7fb9111da740 1 -- 172.21.15.39:0/2238358381 --> [v2:172.21.15.39:6832/3547777254,v1:172.21.15.39:6835/3547777254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577e57cc520 con 0x5577e546ee40 2024-09-21T07:45:02.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.336+0000 7fb8faa00640 1 --2- 172.21.15.39:0/2238358381 >> [v2:172.21.15.39:6832/3547777254,v1:172.21.15.39:6835/3547777254] conn(0x5577e546ee40 0x5577e58b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:02.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.337+0000 7fb8e7e00640 10 client.8637 ms_handle_connect on v2:172.21.15.39:6832/3547777254 2024-09-21T07:45:02.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.337+0000 7fb8faa00640 1 --2- 172.21.15.39:0/2238358381 >> [v2:172.21.15.39:6832/3547777254,v1:172.21.15.39:6835/3547777254] conn(0x5577e546ee40 0x5577e58b2ff0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=383f7872bcfc673d server_cookie=6acfa42d210a31d0 in_seq=0 out_seq=0 2024-09-21T07:45:02.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.337+0000 7fb8e7e00640 1 -- 172.21.15.39:0/2238358381 <== mds.0 v2:172.21.15.39:6832/3547777254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5577e57cc520 con 0x5577e546ee40 2024-09-21T07:45:02.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.337+0000 7fb8e7e00640 10 client.8637 handle_command_reply: tid=0 2024-09-21T07:45:02.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.338+0000 7fb9111da740 1 client.8637 shutdown 2024-09-21T07:45:02.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.338+0000 7fb9111da740 1 -- 172.21.15.39:0/2238358381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577e54873b0 msgr2=0x5577e58943d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:02.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.338+0000 7fb9111da740 1 --2- 172.21.15.39:0/2238358381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577e54873b0 0x5577e58943d0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fb8f40146f0 tx=0x7fb8f4004620 comp rx=0 tx=0).stop 2024-09-21T07:45:02.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.338+0000 7fb9111da740 1 -- 172.21.15.39:0/2238358381 shutdown_connections 2024-09-21T07:45:02.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.338+0000 7fb9111da740 1 --2- 172.21.15.39:0/2238358381 >> [v2:172.21.15.39:6832/3547777254,v1:172.21.15.39:6835/3547777254] conn(0x5577e546ee40 0x5577e58b2ff0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.338+0000 7fb9111da740 1 --2- 172.21.15.39:0/2238358381 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5577e54b0270 0x5577e58807c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.338+0000 7fb9111da740 1 --2- 172.21.15.39:0/2238358381 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5577e548f590 0x5577e5878050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.338+0000 7fb9111da740 1 --2- 172.21.15.39:0/2238358381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577e54873b0 0x5577e58943d0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.338+0000 7fb8e7e00640 0 client.8637 ms_handle_reset on v2:172.21.15.39:6832/3547777254 2024-09-21T07:45:02.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.338+0000 7fb9111da740 1 -- 172.21.15.39:0/2238358381 >> 172.21.15.39:0/2238358381 conn(0x5577e54a43c0 msgr2=0x5577e5892010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:02.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.338+0000 7fb9111da740 1 -- 172.21.15.39:0/2238358381 shutdown_connections 2024-09-21T07:45:02.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.338+0000 7fb9111da740 1 -- 172.21.15.39:0/2238358381 wait complete. 2024-09-21T07:45:02.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.338+0000 7fb9111da740 20 client.8637 trim_cache size 0 max 16384 2024-09-21T07:45:02.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.357+0000 7fb9111da740 1 Processor -- start 2024-09-21T07:45:02.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.358+0000 7fb9111da740 1 -- start start 2024-09-21T07:45:02.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.359+0000 7fb9111da740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5577e54873b0 0x5577e5851160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:02.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.359+0000 7fb9111da740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577e548f590 0x5577e58c4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:02.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.359+0000 7fb9111da740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5577e54b0270 0x5577e5898440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:02.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.359+0000 7fb9111da740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb9041ff4e0 con 0x5577e548f590 2024-09-21T07:45:02.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.359+0000 7fb9111da740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb9041f7bf0 con 0x5577e54b0270 2024-09-21T07:45:02.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.359+0000 7fb9111da740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb9040aef90 con 0x5577e54873b0 2024-09-21T07:45:02.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.359+0000 7fb8faa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577e548f590 0x5577e58c4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:02.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.359+0000 7fb8faa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577e548f590 0x5577e58c4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53416/0 (socket says 172.21.15.39:53416) 2024-09-21T07:45:02.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.359+0000 7fb8faa00640 1 -- 172.21.15.39:0/707334105 learned_addr learned my addr 172.21.15.39:0/707334105 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:02.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.359+0000 7fb903400640 1 --2- 172.21.15.39:0/707334105 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5577e54873b0 0x5577e5851160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:02.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.359+0000 7fb903e00640 1 --2- 172.21.15.39:0/707334105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5577e54b0270 0x5577e5898440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:02.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.360+0000 7fb8faa00640 1 -- 172.21.15.39:0/707334105 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5577e54873b0 msgr2=0x5577e5851160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:45:02.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.360+0000 7fb8faa00640 1 --2- 172.21.15.39:0/707334105 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5577e54873b0 0x5577e5851160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.360+0000 7fb8faa00640 1 -- 172.21.15.39:0/707334105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5577e54b0270 msgr2=0x5577e5898440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:45:02.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.360+0000 7fb8faa00640 1 --2- 172.21.15.39:0/707334105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5577e54b0270 0x5577e5898440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.360+0000 7fb8faa00640 1 -- 172.21.15.39:0/707334105 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577e5894740 con 0x5577e548f590 2024-09-21T07:45:02.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.360+0000 7fb8faa00640 1 --2- 172.21.15.39:0/707334105 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577e548f590 0x5577e58c4440 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x5577e5852910 tx=0x7fb8ec00b2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd31c810c9c7a93c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:02.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.360+0000 7fb8f2a00640 1 -- 172.21.15.39:0/707334105 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec032070 con 0x5577e548f590 2024-09-21T07:45:02.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.360+0000 7fb8f2a00640 1 -- 172.21.15.39:0/707334105 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8ec0042c0 con 0x5577e548f590 2024-09-21T07:45:02.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.360+0000 7fb8f2a00640 1 -- 172.21.15.39:0/707334105 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec003850 con 0x5577e548f590 2024-09-21T07:45:02.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.361+0000 7fb9111da740 1 -- 172.21.15.39:0/707334105 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577e548f590 msgr2=0x5577e58c4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:02.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.361+0000 7fb9111da740 1 --2- 172.21.15.39:0/707334105 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577e548f590 0x5577e58c4440 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x5577e5852910 tx=0x7fb8ec00b2e0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.361+0000 7fb9111da740 1 -- 172.21.15.39:0/707334105 shutdown_connections 2024-09-21T07:45:02.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.361+0000 7fb9111da740 1 --2- 172.21.15.39:0/707334105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5577e54b0270 0x5577e5898440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.361+0000 7fb9111da740 1 --2- 172.21.15.39:0/707334105 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577e548f590 0x5577e58c4440 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.361+0000 7fb9111da740 1 --2- 172.21.15.39:0/707334105 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5577e54873b0 0x5577e5851160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.361+0000 7fb9111da740 1 -- 172.21.15.39:0/707334105 >> 172.21.15.39:0/707334105 conn(0x5577e54a43c0 msgr2=0x5577e58805e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:02.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.361+0000 7fb9111da740 1 -- 172.21.15.39:0/707334105 shutdown_connections 2024-09-21T07:45:02.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.361+0000 7fb9111da740 1 -- 172.21.15.39:0/707334105 wait complete. 2024-09-21T07:45:02.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.362+0000 7fb9111da740 1 -- start start 2024-09-21T07:45:02.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.362+0000 7fb9111da740 1 Processor -- start 2024-09-21T07:45:02.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.364+0000 7fb9111da740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5577e54873b0 0x5577e59167b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:02.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.364+0000 7fb9111da740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5577e548f590 0x5577e59237d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:02.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.364+0000 7fb9111da740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577e54b0270 0x5577e5923d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:02.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.364+0000 7fb9111da740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5577e587e570 con 0x5577e54b0270 2024-09-21T07:45:02.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.364+0000 7fb9111da740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5577e5880d00 con 0x5577e548f590 2024-09-21T07:45:02.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.364+0000 7fb9111da740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x5577e587eb30 con 0x5577e54873b0 2024-09-21T07:45:02.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.364+0000 7fb9111da740 10 client.0 _ll_register_callbacks cb 0x7fb8f400fca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:45:02.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.364+0000 7fb903e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577e54b0270 0x5577e5923d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:02.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.364+0000 7fb903e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577e54b0270 0x5577e5923d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53420/0 (socket says 172.21.15.39:53420) 2024-09-21T07:45:02.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.364+0000 7fb903e00640 1 -- 172.21.15.39:0/1138239041 learned_addr learned my addr 172.21.15.39:0/1138239041 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:02.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.364+0000 7fb903400640 1 --2- 172.21.15.39:0/1138239041 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5577e54873b0 0x5577e59167b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:02.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.364+0000 7fb8faa00640 1 --2- 172.21.15.39:0/1138239041 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5577e548f590 0x5577e59237d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:02.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.364+0000 7fb903e00640 1 -- 172.21.15.39:0/1138239041 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5577e54873b0 msgr2=0x5577e59167b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:45:02.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.364+0000 7fb903e00640 1 --2- 172.21.15.39:0/1138239041 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5577e54873b0 0x5577e59167b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.364+0000 7fb903e00640 1 -- 172.21.15.39:0/1138239041 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5577e548f590 msgr2=0x5577e59237d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:45:02.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.364+0000 7fb903e00640 1 --2- 172.21.15.39:0/1138239041 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5577e548f590 0x5577e59237d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.364+0000 7fb903e00640 1 -- 172.21.15.39:0/1138239041 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577e5850920 con 0x5577e54b0270 2024-09-21T07:45:02.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.364+0000 7fb903400640 1 --2- 172.21.15.39:0/1138239041 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5577e54873b0 0x5577e59167b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:45:02.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.365+0000 7fb8faa00640 1 --2- 172.21.15.39:0/1138239041 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5577e548f590 0x5577e59237d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:45:02.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.365+0000 7fb903e00640 1 --2- 172.21.15.39:0/1138239041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577e54b0270 0x5577e5923d10 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc009420 tx=0x7fb8fc080450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fccc6ae5ade1265 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:02.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.365+0000 7fb8e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:45:02.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.365+0000 7fb8e7e00640 1 -- 172.21.15.39:0/1138239041 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc088940 con 0x5577e54b0270 2024-09-21T07:45:02.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.365+0000 7fb8e7e00640 1 -- 172.21.15.39:0/1138239041 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 6800+0+0 (secure 0 0 0) 0x7fb8fc08d3f0 con 0x5577e54b0270 2024-09-21T07:45:02.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.365+0000 7fb9111da740 1 -- 172.21.15.39:0/1138239041 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8ec0042c0 con 0x5577e54b0270 2024-09-21T07:45:02.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.365+0000 7fb8e7e00640 1 -- 172.21.15.39:0/1138239041 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=172) ==== 24+0+0 (secure 0 0 0) 0x7fb8fc0a5050 con 0x5577e54b0270 2024-09-21T07:45:02.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.365+0000 7fb9111da740 10 client.8643 fetch_fsmap learned FSMap version 172 2024-09-21T07:45:02.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.365+0000 7fb9111da740 1 -- 172.21.15.39:0/1138239041 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=172,monmap=2+}) -- 0x5577e5924250 con 0x5577e54b0270 2024-09-21T07:45:02.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.365+0000 7fb8e7e00640 1 -- 172.21.15.39:0/1138239041 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8ec0042c0 con 0x5577e54b0270 2024-09-21T07:45:02.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.366+0000 7fb8e7e00640 1 -- 172.21.15.39:0/1138239041 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 172) ==== 3502+0+0 (secure 0 0 0) 0x7fb8fc04b030 con 0x5577e54b0270 2024-09-21T07:45:02.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.366+0000 7fb9111da740 10 client.8643 fetch_fsmap finished waiting for FSMap version 172 2024-09-21T07:45:02.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.366+0000 7fb9111da740 10 client.8643 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:45:02.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.366+0000 7fb9111da740 20 client.8643 populate_metadata read hostname 'smithi039' 2024-09-21T07:45:02.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.366+0000 7fb9111da740 1 --2- 172.21.15.39:0/1138239041 >> [v2:172.21.15.39:6832/3547777254,v1:172.21.15.39:6835/3547777254] conn(0x5577e587ddb0 0x5577e58c2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:02.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.366+0000 7fb9111da740 4 client.8643 mds_command: new command op to 8520 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:45:02.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.366+0000 7fb9111da740 1 -- 172.21.15.39:0/1138239041 --> [v2:172.21.15.39:6832/3547777254,v1:172.21.15.39:6835/3547777254] -- command(tid 0: {"prefix": "status"}) -- 0x5577e57c7fd0 con 0x5577e587ddb0 2024-09-21T07:45:02.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.366+0000 7fb903400640 1 --2- 172.21.15.39:0/1138239041 >> [v2:172.21.15.39:6832/3547777254,v1:172.21.15.39:6835/3547777254] conn(0x5577e587ddb0 0x5577e58c2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:02.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.366+0000 7fb903400640 1 --2- 172.21.15.39:0/1138239041 >> [v2:172.21.15.39:6832/3547777254,v1:172.21.15.39:6835/3547777254] conn(0x5577e587ddb0 0x5577e58c2e50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=386213527c3fd96 server_cookie=8afb967beabd4df3 in_seq=0 out_seq=0 2024-09-21T07:45:02.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.366+0000 7fb8e7e00640 10 client.8643 ms_handle_connect on v2:172.21.15.39:6832/3547777254 2024-09-21T07:45:02.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.367+0000 7fb8e7e00640 1 -- 172.21.15.39:0/1138239041 <== mds.0 v2:172.21.15.39:6832/3547777254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x5577e57c7fd0 con 0x5577e587ddb0 2024-09-21T07:45:02.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.367+0000 7fb8e7e00640 10 client.8643 handle_command_reply: tid=0 2024-09-21T07:45:02.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.367+0000 7fb9111da740 1 client.8643 shutdown 2024-09-21T07:45:02.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.367+0000 7fb9111da740 1 -- 172.21.15.39:0/1138239041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577e54b0270 msgr2=0x5577e5923d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:02.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.367+0000 7fb9111da740 1 --2- 172.21.15.39:0/1138239041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577e54b0270 0x5577e5923d10 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc009420 tx=0x7fb8fc080450 comp rx=0 tx=0).stop 2024-09-21T07:45:02.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.367+0000 7fb9111da740 1 -- 172.21.15.39:0/1138239041 shutdown_connections 2024-09-21T07:45:02.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.367+0000 7fb9111da740 1 --2- 172.21.15.39:0/1138239041 >> [v2:172.21.15.39:6832/3547777254,v1:172.21.15.39:6835/3547777254] conn(0x5577e587ddb0 0x5577e58c2e50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.367+0000 7fb9111da740 1 --2- 172.21.15.39:0/1138239041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5577e54b0270 0x5577e5923d10 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.367+0000 7fb9111da740 1 --2- 172.21.15.39:0/1138239041 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5577e548f590 0x5577e59237d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.367+0000 7fb9111da740 1 --2- 172.21.15.39:0/1138239041 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5577e54873b0 0x5577e59167b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.367+0000 7fb8e7e00640 0 client.8643 ms_handle_reset on v2:172.21.15.39:6832/3547777254 2024-09-21T07:45:02.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.367+0000 7fb9111da740 1 -- 172.21.15.39:0/1138239041 >> 172.21.15.39:0/1138239041 conn(0x5577e54a43c0 msgr2=0x5577e584c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:02.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.367+0000 7fb9111da740 1 -- 172.21.15.39:0/1138239041 shutdown_connections 2024-09-21T07:45:02.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.367+0000 7fb9111da740 1 -- 172.21.15.39:0/1138239041 wait complete. 2024-09-21T07:45:02.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.367+0000 7fb9111da740 20 client.8643 trim_cache size 0 max 16384 2024-09-21T07:45:02.373 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:45:02.373 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:45:02.373 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-09-21T07:45:02.373 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 8520, 2024-09-21T07:45:02.373 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:45:02.374 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:45:02.374 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:45:02.374 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 6.8196185829999996, 2024-09-21T07:45:02.374 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 172, 2024-09-21T07:45:02.374 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 134, 2024-09-21T07:45:02.374 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 134, 2024-09-21T07:45:02.374 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 11.808430226 2024-09-21T07:45:02.374 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:45:02.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.368+0000 7fb90a000640 1 -- 172.21.15.39:0/4200612222 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb8e0078b90 msgr2=0x7fb8e007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:02.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.368+0000 7fb90a000640 1 --2- 172.21.15.39:0/4200612222 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb8e0078b90 0x7fb8e007b0a0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fb8f40046b0 tx=0x7fb8f4007eb0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.368+0000 7fb90a000640 1 -- 172.21.15.39:0/4200612222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9040b4330 msgr2=0x7fb904154fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:02.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.368+0000 7fb90a000640 1 --2- 172.21.15.39:0/4200612222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9040b4330 0x7fb904154fb0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc005fd0 tx=0x7fb8fc031760 comp rx=0 tx=0).stop 2024-09-21T07:45:02.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.368+0000 7fb90a000640 1 -- 172.21.15.39:0/4200612222 shutdown_connections 2024-09-21T07:45:02.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.368+0000 7fb90a000640 1 --2- 172.21.15.39:0/4200612222 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb8e0078b90 0x7fb8e007b0a0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.368+0000 7fb90a000640 1 --2- 172.21.15.39:0/4200612222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9040b4330 0x7fb904154fb0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.368+0000 7fb90a000640 1 --2- 172.21.15.39:0/4200612222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9040b4c40 0x7fb9040b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.368+0000 7fb90a000640 1 --2- 172.21.15.39:0/4200612222 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb90406ef90 0x7fb9041f4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.368+0000 7fb90a000640 1 -- 172.21.15.39:0/4200612222 >> 172.21.15.39:0/4200612222 conn(0x7fb9040b2460 msgr2=0x7fb9040d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:02.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.368+0000 7fb90a000640 1 -- 172.21.15.39:0/4200612222 shutdown_connections 2024-09-21T07:45:02.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.368+0000 7fb90a000640 1 -- 172.21.15.39:0/4200612222 wait complete. 2024-09-21T07:45:02.394 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:45:02.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.663+0000 7f80efe00640 1 -- 172.21.15.39:0/1172405922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80f00cd630 msgr2=0x7f80f015ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:02.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.663+0000 7f80efe00640 1 --2- 172.21.15.39:0/1172405922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80f00cd630 0x7f80f015ccb0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f80e8005fd0 tx=0x7f80e802aa90 comp rx=0 tx=0).stop 2024-09-21T07:45:02.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.663+0000 7f80efe00640 1 -- 172.21.15.39:0/1172405922 shutdown_connections 2024-09-21T07:45:02.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.663+0000 7f80efe00640 1 --2- 172.21.15.39:0/1172405922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80f00cd630 0x7f80f015ccb0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.663+0000 7f80efe00640 1 --2- 172.21.15.39:0/1172405922 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80f00b5ac0 0x7f80f0153870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.663+0000 7f80efe00640 1 -- 172.21.15.39:0/1172405922 >> 172.21.15.39:0/1172405922 conn(0x7f80f00d0c70 msgr2=0x7f80f00d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:02.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.663+0000 7f80efe00640 1 -- 172.21.15.39:0/1172405922 shutdown_connections 2024-09-21T07:45:02.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.663+0000 7f80efe00640 1 -- 172.21.15.39:0/1172405922 wait complete. 2024-09-21T07:45:02.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.665+0000 7f80efe00640 1 Processor -- start 2024-09-21T07:45:02.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.665+0000 7f80efe00640 1 -- start start 2024-09-21T07:45:02.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.667+0000 7f80efe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80f00b5ac0 0x7f80f0152240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:02.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.667+0000 7f80efe00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80f00cde20 0x7f80f015bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:02.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.667+0000 7f80efe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80f015c0a0 0x7f80f01f8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:02.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.667+0000 7f80efe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f80f015d580 con 0x7f80f00b5ac0 2024-09-21T07:45:02.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.667+0000 7f80efe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f80f015d400 con 0x7f80f015c0a0 2024-09-21T07:45:02.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.667+0000 7f80efe00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f80f015d700 con 0x7f80f00cde20 2024-09-21T07:45:02.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.667+0000 7f80eea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80f00b5ac0 0x7f80f0152240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:02.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.667+0000 7f80eea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80f00b5ac0 0x7f80f0152240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53432/0 (socket says 172.21.15.39:53432) 2024-09-21T07:45:02.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.667+0000 7f80eea00640 1 -- 172.21.15.39:0/219247692 learned_addr learned my addr 172.21.15.39:0/219247692 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:02.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.667+0000 7f80ef400640 1 --2- 172.21.15.39:0/219247692 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80f015c0a0 0x7f80f01f8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:02.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.667+0000 7f80ee000640 1 --2- 172.21.15.39:0/219247692 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80f00cde20 0x7f80f015bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:02.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.668+0000 7f80eea00640 1 -- 172.21.15.39:0/219247692 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80f00cde20 msgr2=0x7f80f015bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:02.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.668+0000 7f80eea00640 1 --2- 172.21.15.39:0/219247692 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80f00cde20 0x7f80f015bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.668+0000 7f80eea00640 1 -- 172.21.15.39:0/219247692 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80f015c0a0 msgr2=0x7f80f01f8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:02.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.668+0000 7f80eea00640 1 --2- 172.21.15.39:0/219247692 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80f015c0a0 0x7f80f01f8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.668+0000 7f80eea00640 1 -- 172.21.15.39:0/219247692 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f01f5070 con 0x7f80f00b5ac0 2024-09-21T07:45:02.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.668+0000 7f80ef400640 1 --2- 172.21.15.39:0/219247692 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80f015c0a0 0x7f80f01f8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:45:02.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.668+0000 7f80ee000640 1 --2- 172.21.15.39:0/219247692 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80f00cde20 0x7f80f015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:45:02.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.668+0000 7f80eea00640 1 --2- 172.21.15.39:0/219247692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80f00b5ac0 0x7f80f0152240 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f80e000d740 tx=0x7f80e000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:02.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.668+0000 7f80e6a00640 1 -- 172.21.15.39:0/219247692 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e0017070 con 0x7f80f00b5ac0 2024-09-21T07:45:02.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.668+0000 7f80e6a00640 1 -- 172.21.15.39:0/219247692 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80e0009d20 con 0x7f80f00b5ac0 2024-09-21T07:45:02.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.668+0000 7f80efe00640 1 -- 172.21.15.39:0/219247692 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80f01f5350 con 0x7f80f00b5ac0 2024-09-21T07:45:02.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.668+0000 7f80e6a00640 1 -- 172.21.15.39:0/219247692 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e00106c0 con 0x7f80f00b5ac0 2024-09-21T07:45:02.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.668+0000 7f80efe00640 1 -- 172.21.15.39:0/219247692 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80f01f5910 con 0x7f80f00b5ac0 2024-09-21T07:45:02.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.669+0000 7f80efe00640 1 -- 172.21.15.39:0/219247692 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80f00d0450 con 0x7f80f00b5ac0 2024-09-21T07:45:02.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.671+0000 7f80e6a00640 1 -- 172.21.15.39:0/219247692 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f80e001a070 con 0x7f80f00b5ac0 2024-09-21T07:45:02.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.672+0000 7f80e6a00640 1 --2- 172.21.15.39:0/219247692 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f80d0078b90 0x7f80d007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:02.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.672+0000 7f80e6a00640 1 -- 172.21.15.39:0/219247692 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 6800+0+0 (secure 0 0 0) 0x7f80e009b3d0 con 0x7f80f00b5ac0 2024-09-21T07:45:02.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.672+0000 7f80ee000640 1 --2- 172.21.15.39:0/219247692 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f80d0078b90 0x7f80d007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:02.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.673+0000 7f80ee000640 1 --2- 172.21.15.39:0/219247692 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f80d0078b90 0x7f80d007b0a0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f80f01521d0 tx=0x7f80dc009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:02.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.677+0000 7f80e6a00640 1 -- 172.21.15.39:0/219247692 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f80e0063350 con 0x7f80f00b5ac0 2024-09-21T07:45:02.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.914+0000 7f80efe00640 1 -- 172.21.15.39:0/219247692 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f80f0069fc0 con 0x7f80f00b5ac0 2024-09-21T07:45:02.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.915+0000 7f80e6a00640 1 -- 172.21.15.39:0/219247692 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v172) ==== 53+0+73 (secure 0 0 0) 0x7f80e0062c50 con 0x7f80f00b5ac0 2024-09-21T07:45:02.917 INFO:teuthology.orchestra.run.smithi039.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:45:02.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.917+0000 7f80efe00640 1 -- 172.21.15.39:0/219247692 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f80d0078b90 msgr2=0x7f80d007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:02.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.917+0000 7f80efe00640 1 --2- 172.21.15.39:0/219247692 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f80d0078b90 0x7f80d007b0a0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f80f01521d0 tx=0x7f80dc009040 comp rx=0 tx=0).stop 2024-09-21T07:45:02.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.917+0000 7f80efe00640 1 -- 172.21.15.39:0/219247692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80f00b5ac0 msgr2=0x7f80f0152240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:02.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.917+0000 7f80efe00640 1 --2- 172.21.15.39:0/219247692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80f00b5ac0 0x7f80f0152240 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f80e000d740 tx=0x7f80e000dc00 comp rx=0 tx=0).stop 2024-09-21T07:45:02.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.917+0000 7f80efe00640 1 -- 172.21.15.39:0/219247692 shutdown_connections 2024-09-21T07:45:02.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.917+0000 7f80efe00640 1 --2- 172.21.15.39:0/219247692 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f80d0078b90 0x7f80d007b0a0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.917+0000 7f80efe00640 1 --2- 172.21.15.39:0/219247692 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80f015c0a0 0x7f80f01f8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.917+0000 7f80efe00640 1 --2- 172.21.15.39:0/219247692 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80f00cde20 0x7f80f015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.917+0000 7f80efe00640 1 --2- 172.21.15.39:0/219247692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80f00b5ac0 0x7f80f0152240 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:02.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.917+0000 7f80efe00640 1 -- 172.21.15.39:0/219247692 >> 172.21.15.39:0/219247692 conn(0x7f80f00d0c70 msgr2=0x7f80f006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:02.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.917+0000 7f80efe00640 1 -- 172.21.15.39:0/219247692 shutdown_connections 2024-09-21T07:45:02.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:02.917+0000 7f80efe00640 1 -- 172.21.15.39:0/219247692 wait complete. 2024-09-21T07:45:02.941 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:45:02.941 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:45:02.941 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi039.front.sepia.ceph.com 2024-09-21T07:45:02.941 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T07:45:02.941 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:45:02.941 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:45:02.941 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:45:02.941 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:45:02.941 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:45:02.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:02.942 DEBUG:teuthology.orchestra.run.smithi039:> ip addr 2024-09-21T07:45:02.978 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:45:02.978 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:45:02.978 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:45:02.978 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:45:02.978 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2024-09-21T07:45:02.978 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:45:02.978 INFO:teuthology.orchestra.run.smithi039.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:45:02.978 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:45:02.978 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f0 2024-09-21T07:45:02.978 INFO:teuthology.orchestra.run.smithi039.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:45:02.978 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c7 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:45:02.978 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f1 2024-09-21T07:45:02.978 INFO:teuthology.orchestra.run.smithi039.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:45:02.979 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:70 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:45:02.979 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f0 2024-09-21T07:45:02.979 INFO:teuthology.orchestra.run.smithi039.stdout: inet 172.21.15.39/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T07:45:02.979 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft 42016sec preferred_lft 42016sec 2024-09-21T07:45:02.979 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec4:7aff:febd:1370/64 scope link 2024-09-21T07:45:02.979 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:45:02.979 INFO:teuthology.orchestra.run.smithi039.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:45:02.979 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:71 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:45:02.979 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f1 2024-09-21T07:45:02.979 INFO:teuthology.orchestra.run.smithi039.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:45:02.979 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff 2024-09-21T07:45:02.979 INFO:teuthology.orchestra.run.smithi039.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:45:02.979 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:45:02.979 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::b079:ddff:fe0e:6333/64 scope link 2024-09-21T07:45:02.980 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:45:02.980 INFO:teuthology.orchestra.run.smithi039.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:45:02.980 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:45:02.980 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::2ca7:98ff:fe8a:d0ed/64 scope link 2024-09-21T07:45:02.980 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:45:02.980 INFO:teuthology.orchestra.run.smithi039.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:45:02.980 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether ee:10:53:01:0c:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:45:02.980 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec10:53ff:fe01:c44/64 scope link 2024-09-21T07:45:02.980 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:45:02.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:02.981 DEBUG:teuthology.orchestra.run.smithi039:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:45:03.071 INFO:teuthology.orchestra.run.smithi039.stdout:1 2024-09-21T07:45:03.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:03.074 DEBUG:teuthology.orchestra.run.smithi039:> route 2024-09-21T07:45:03.110 INFO:teuthology.orchestra.run.smithi039.stdout:Kernel IP routing table 2024-09-21T07:45:03.110 INFO:teuthology.orchestra.run.smithi039.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:45:03.110 INFO:teuthology.orchestra.run.smithi039.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T07:45:03.110 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T07:45:03.110 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:45:03.111 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:45:03.111 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-21T07:45:03.111 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T07:45:03.111 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T07:45:03.111 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T07:45:03.111 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-21T07:45:03.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:45:03.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:45:03.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:03.310 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-09-21T07:45:03.346 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:45:03.346 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T07:45:03.346 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:45:03.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:03.347 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-09-21T07:45:03.442 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:45:03.442 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T07:45:03.514 INFO:teuthology.orchestra.run.smithi039.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:45:03.514 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:45:03.514 DEBUG:teuthology.orchestra.run.smithi039:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:45:03.549 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-21T07:45:03.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:03.550 DEBUG:teuthology.orchestra.run.smithi039:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:45:03.748 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:45:03.748 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:45:03.783 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:45:03.783 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:45:03.783 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: resolved to: "172.21.15.39:3300,172.21.15.148:3300,172.21.15.148:3301" 2024-09-21T07:45:03.783 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: trying mount with new device syntax: 0@7ab9b04a-f88d-49aa-a493-e5c076637ffb.cephfs=/ 2024-09-21T07:45:03.783 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.39:3300/172.21.15.148:3300/172.21.15.148:3301" will pass to kernel 2024-09-21T07:45:03.784 INFO:teuthology.orchestra.run.smithi039.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T07:45:03.785 INFO:teuthology.orchestra.run.smithi039.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:45:03.785 INFO:teuthology.orchestra.run.smithi039.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:45:03.785 INFO:teuthology.orchestra.run.smithi039.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:45:03.785 INFO:teuthology.orchestra.run.smithi039.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:45:03.788 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:45:03.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:03.788 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:45:03.844 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:45:03.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:45:03.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:45:03.943 INFO:teuthology.orchestra.run.smithi039.stdout:client7211 2024-09-21T07:45:03.943 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:45:03.943 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:45:03.979 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get mds_max_snaps_per_dir 2024-09-21T07:45:04.261 INFO:teuthology.orchestra.run.smithi039.stdout:{"mds_max_snaps_per_dir":"100"} 2024-09-21T07:45:04.261 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_max_snaps_per_dir": "100" } 2024-09-21T07:45:04.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:04.261 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts) 2024-09-21T07:45:04.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:04.304 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_1) 2024-09-21T07:45:04.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:04.407 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_2) 2024-09-21T07:45:04.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:04.469 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_3) 2024-09-21T07:45:04.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:04.532 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_4) 2024-09-21T07:45:04.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:04.594 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_5) 2024-09-21T07:45:04.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:04.656 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_6) 2024-09-21T07:45:04.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:04.718 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_7) 2024-09-21T07:45:04.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:04.782 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_8) 2024-09-21T07:45:04.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:04.844 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_9) 2024-09-21T07:45:04.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:04.908 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_10) 2024-09-21T07:45:04.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:04.970 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_11) 2024-09-21T07:45:05.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:05.021 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_12) 2024-09-21T07:45:05.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:05.072 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_13) 2024-09-21T07:45:05.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:05.123 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_14) 2024-09-21T07:45:05.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:05.175 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_15) 2024-09-21T07:45:05.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:05.227 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_16) 2024-09-21T07:45:05.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:05.278 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_17) 2024-09-21T07:45:05.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:05.329 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_18) 2024-09-21T07:45:05.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:05.381 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_19) 2024-09-21T07:45:05.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:05.431 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_20) 2024-09-21T07:45:05.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:05.482 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_21) 2024-09-21T07:45:05.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:05.533 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_22) 2024-09-21T07:45:05.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:05.586 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_23) 2024-09-21T07:45:05.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:05.637 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_24) 2024-09-21T07:45:05.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:05.688 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_25) 2024-09-21T07:45:05.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:05.740 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_26) 2024-09-21T07:45:05.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:05.791 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_27) 2024-09-21T07:45:05.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:05.842 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_28) 2024-09-21T07:45:05.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:05.893 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_29) 2024-09-21T07:45:05.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:05.944 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_30) 2024-09-21T07:45:05.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:05.996 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_31) 2024-09-21T07:45:06.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:06.048 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_32) 2024-09-21T07:45:06.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:06.100 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_33) 2024-09-21T07:45:06.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:06.151 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_34) 2024-09-21T07:45:06.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:06.203 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_35) 2024-09-21T07:45:06.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:06.253 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_36) 2024-09-21T07:45:06.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:06.305 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_37) 2024-09-21T07:45:06.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:06.356 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_38) 2024-09-21T07:45:06.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:06.408 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_39) 2024-09-21T07:45:06.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:06.460 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_40) 2024-09-21T07:45:06.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:06.512 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_41) 2024-09-21T07:45:06.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:06.563 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_42) 2024-09-21T07:45:06.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:06.614 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_43) 2024-09-21T07:45:06.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:06.666 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_44) 2024-09-21T07:45:06.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:06.718 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_45) 2024-09-21T07:45:06.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:06.769 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_46) 2024-09-21T07:45:06.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:06.821 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_47) 2024-09-21T07:45:06.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:06.872 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_48) 2024-09-21T07:45:06.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:06.924 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_49) 2024-09-21T07:45:06.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:06.976 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_50) 2024-09-21T07:45:07.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:07.029 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_51) 2024-09-21T07:45:07.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:07.081 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_52) 2024-09-21T07:45:07.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:07.134 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_53) 2024-09-21T07:45:07.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:07.185 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_54) 2024-09-21T07:45:07.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:07.237 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_55) 2024-09-21T07:45:07.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:07.289 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_56) 2024-09-21T07:45:07.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:07.341 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_57) 2024-09-21T07:45:07.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:07.393 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_58) 2024-09-21T07:45:07.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:07.445 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_59) 2024-09-21T07:45:07.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:07.497 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_60) 2024-09-21T07:45:07.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:07.549 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_61) 2024-09-21T07:45:07.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:07.601 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_62) 2024-09-21T07:45:07.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:07.655 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_63) 2024-09-21T07:45:07.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:07.708 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_64) 2024-09-21T07:45:07.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:07.761 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_65) 2024-09-21T07:45:07.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:07.813 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_66) 2024-09-21T07:45:07.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:07.866 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_67) 2024-09-21T07:45:07.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:07.918 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_68) 2024-09-21T07:45:07.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:07.970 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_69) 2024-09-21T07:45:08.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:08.023 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_70) 2024-09-21T07:45:08.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:08.077 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_71) 2024-09-21T07:45:08.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:08.132 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_72) 2024-09-21T07:45:08.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:08.185 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_73) 2024-09-21T07:45:08.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:08.237 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_74) 2024-09-21T07:45:08.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:08.290 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_75) 2024-09-21T07:45:08.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:08.343 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_76) 2024-09-21T07:45:08.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:08.395 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_77) 2024-09-21T07:45:08.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:08.448 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_78) 2024-09-21T07:45:08.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:08.501 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_79) 2024-09-21T07:45:08.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:08.553 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_80) 2024-09-21T07:45:08.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:08.605 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_81) 2024-09-21T07:45:08.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:08.658 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_82) 2024-09-21T07:45:08.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:08.710 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_83) 2024-09-21T07:45:08.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:08.763 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_84) 2024-09-21T07:45:08.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:08.816 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_85) 2024-09-21T07:45:08.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:08.869 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_86) 2024-09-21T07:45:08.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:08.922 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_87) 2024-09-21T07:45:08.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:08.974 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_88) 2024-09-21T07:45:09.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:09.029 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_89) 2024-09-21T07:45:09.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:09.085 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_90) 2024-09-21T07:45:09.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:09.138 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_91) 2024-09-21T07:45:09.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:09.190 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_92) 2024-09-21T07:45:09.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:09.243 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_93) 2024-09-21T07:45:09.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:09.297 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_94) 2024-09-21T07:45:09.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:09.350 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_95) 2024-09-21T07:45:09.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:09.419 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_96) 2024-09-21T07:45:09.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:09.483 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_97) 2024-09-21T07:45:09.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:09.548 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_98) 2024-09-21T07:45:09.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:09.613 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_99) 2024-09-21T07:45:09.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:09.678 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_100) 2024-09-21T07:45:09.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:09.743 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_1) 2024-09-21T07:45:09.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:09.789 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_2) 2024-09-21T07:45:09.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:09.873 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_3) 2024-09-21T07:45:09.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:09.957 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_4) 2024-09-21T07:45:10.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:10.043 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_5) 2024-09-21T07:45:10.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:10.128 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_6) 2024-09-21T07:45:10.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:10.212 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_7) 2024-09-21T07:45:10.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:10.297 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_8) 2024-09-21T07:45:10.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:10.381 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_9) 2024-09-21T07:45:10.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:10.466 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_10) 2024-09-21T07:45:10.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:10.550 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_11) 2024-09-21T07:45:10.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:10.636 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_12) 2024-09-21T07:45:10.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:10.726 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_13) 2024-09-21T07:45:10.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:10.777 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_14) 2024-09-21T07:45:10.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:10.821 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_15) 2024-09-21T07:45:10.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:10.905 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_16) 2024-09-21T07:45:11.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:11.000 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_17) 2024-09-21T07:45:11.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:11.046 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_18) 2024-09-21T07:45:11.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:11.147 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_19) 2024-09-21T07:45:11.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:11.191 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_20) 2024-09-21T07:45:11.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:11.276 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_21) 2024-09-21T07:45:11.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:11.361 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_22) 2024-09-21T07:45:11.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:11.445 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_23) 2024-09-21T07:45:11.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:11.530 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_24) 2024-09-21T07:45:11.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:11.616 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_25) 2024-09-21T07:45:11.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:11.700 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_26) 2024-09-21T07:45:11.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:11.783 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_27) 2024-09-21T07:45:11.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:11.875 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_28) 2024-09-21T07:45:11.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:11.919 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_29) 2024-09-21T07:45:12.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:12.005 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_30) 2024-09-21T07:45:12.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:12.088 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_31) 2024-09-21T07:45:12.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:12.170 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_32) 2024-09-21T07:45:12.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:12.255 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_33) 2024-09-21T07:45:12.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:12.338 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_34) 2024-09-21T07:45:12.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:12.420 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_35) 2024-09-21T07:45:12.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:12.502 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_36) 2024-09-21T07:45:12.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:12.587 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_37) 2024-09-21T07:45:12.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:12.702 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_38) 2024-09-21T07:45:12.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:12.767 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_39) 2024-09-21T07:45:12.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:12.809 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_40) 2024-09-21T07:45:12.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:12.892 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_41) 2024-09-21T07:45:12.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:12.990 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_42) 2024-09-21T07:45:13.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:13.032 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_43) 2024-09-21T07:45:13.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:13.115 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_44) 2024-09-21T07:45:13.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:13.208 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_45) 2024-09-21T07:45:13.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:13.260 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_46) 2024-09-21T07:45:13.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:13.303 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_47) 2024-09-21T07:45:13.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:13.385 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_48) 2024-09-21T07:45:13.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:13.467 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_49) 2024-09-21T07:45:13.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:13.550 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_50) 2024-09-21T07:45:13.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:13.633 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_51) 2024-09-21T07:45:13.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:13.745 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_52) 2024-09-21T07:45:13.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:13.801 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_53) 2024-09-21T07:45:13.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:13.843 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_54) 2024-09-21T07:45:13.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:13.950 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_55) 2024-09-21T07:45:14.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:14.003 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_56) 2024-09-21T07:45:14.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:14.045 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_57) 2024-09-21T07:45:14.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:14.127 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_58) 2024-09-21T07:45:14.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:14.210 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_59) 2024-09-21T07:45:14.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:14.292 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_60) 2024-09-21T07:45:14.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:14.375 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_61) 2024-09-21T07:45:14.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:14.460 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_62) 2024-09-21T07:45:14.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:14.541 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_63) 2024-09-21T07:45:14.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:14.624 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_64) 2024-09-21T07:45:14.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:14.753 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_65) 2024-09-21T07:45:14.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:14.847 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_66) 2024-09-21T07:45:14.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:14.896 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_67) 2024-09-21T07:45:14.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:14.938 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_68) 2024-09-21T07:45:15.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:15.024 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_69) 2024-09-21T07:45:15.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:15.110 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_70) 2024-09-21T07:45:15.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:15.193 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_71) 2024-09-21T07:45:15.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:15.275 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_72) 2024-09-21T07:45:15.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:15.358 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_73) 2024-09-21T07:45:15.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:15.440 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_74) 2024-09-21T07:45:15.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:15.523 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_75) 2024-09-21T07:45:15.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:15.605 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_76) 2024-09-21T07:45:15.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:15.716 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_77) 2024-09-21T07:45:15.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:15.800 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_78) 2024-09-21T07:45:15.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:15.862 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_79) 2024-09-21T07:45:15.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:15.907 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_80) 2024-09-21T07:45:16.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:16.003 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_81) 2024-09-21T07:45:16.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:16.063 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_82) 2024-09-21T07:45:16.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:16.135 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_83) 2024-09-21T07:45:16.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:16.198 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_84) 2024-09-21T07:45:16.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:16.240 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_85) 2024-09-21T07:45:16.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:16.322 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_86) 2024-09-21T07:45:16.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:16.404 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_87) 2024-09-21T07:45:16.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:16.487 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_88) 2024-09-21T07:45:16.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:16.575 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_89) 2024-09-21T07:45:16.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:16.618 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_90) 2024-09-21T07:45:16.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:16.701 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_91) 2024-09-21T07:45:16.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:16.784 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_92) 2024-09-21T07:45:16.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:16.868 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_93) 2024-09-21T07:45:16.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:16.950 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_94) 2024-09-21T07:45:17.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:17.032 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_95) 2024-09-21T07:45:17.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:17.116 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_96) 2024-09-21T07:45:17.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:17.198 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_97) 2024-09-21T07:45:17.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:17.282 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_98) 2024-09-21T07:45:17.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:17.365 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_99) 2024-09-21T07:45:17.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:17.448 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_100) 2024-09-21T07:45:17.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:17.550 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts) 2024-09-21T07:45:17.588 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:45:17.588 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:45:17.730 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:45:17.807 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:45:17.807 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:45:17.808 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:45:17.808 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:45:17.843 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:45:17.843 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:45:17.938 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:45:18.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.197+0000 7fbcf1c00640 1 -- 172.21.15.39:0/1023226734 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcec0d5060 msgr2=0x7fbcec0b96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:18.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.197+0000 7fbcf1c00640 1 --2- 172.21.15.39:0/1023226734 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcec0d5060 0x7fbcec0b96a0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc009820 tx=0x7fbcdc02a500 comp rx=0 tx=0).stop 2024-09-21T07:45:18.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.198+0000 7fbcf1c00640 1 -- 172.21.15.39:0/1023226734 shutdown_connections 2024-09-21T07:45:18.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.198+0000 7fbcf1c00640 1 --2- 172.21.15.39:0/1023226734 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbcec0d0e60 0x7fbcec0d1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:18.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.198+0000 7fbcf1c00640 1 --2- 172.21.15.39:0/1023226734 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcec0d5060 0x7fbcec0b96a0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:18.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.198+0000 7fbcf1c00640 1 -- 172.21.15.39:0/1023226734 >> 172.21.15.39:0/1023226734 conn(0x7fbcec09d7d0 msgr2=0x7fbcec0d4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:18.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.198+0000 7fbcf1c00640 1 -- 172.21.15.39:0/1023226734 shutdown_connections 2024-09-21T07:45:18.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.198+0000 7fbcf1c00640 1 -- 172.21.15.39:0/1023226734 wait complete. 2024-09-21T07:45:18.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.200+0000 7fbcf1c00640 1 Processor -- start 2024-09-21T07:45:18.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.200+0000 7fbcf1c00640 1 -- start start 2024-09-21T07:45:18.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.201+0000 7fbcf1c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcec0b59e0 0x7fbcec09f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:18.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.201+0000 7fbcf1c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcec0d0e60 0x7fbcec152770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:18.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.201+0000 7fbcf1c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbcec0d5060 0x7fbcec1e7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:18.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.201+0000 7fbcf1c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec0d2c10 con 0x7fbcec0d0e60 2024-09-21T07:45:18.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.201+0000 7fbcf1c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec0d2a90 con 0x7fbcec0b59e0 2024-09-21T07:45:18.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.201+0000 7fbcf1c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fbcec0d2d90 con 0x7fbcec0d5060 2024-09-21T07:45:18.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.202+0000 7fbce3400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcec0d0e60 0x7fbcec152770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:18.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.202+0000 7fbce3400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcec0d0e60 0x7fbcec152770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46732/0 (socket says 172.21.15.39:46732) 2024-09-21T07:45:18.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.202+0000 7fbce3400640 1 -- 172.21.15.39:0/1478675051 learned_addr learned my addr 172.21.15.39:0/1478675051 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:18.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.202+0000 7fbce3400640 1 -- 172.21.15.39:0/1478675051 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbcec0d5060 msgr2=0x7fbcec1e7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:18.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.202+0000 7fbce3400640 1 --2- 172.21.15.39:0/1478675051 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbcec0d5060 0x7fbcec1e7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:18.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.202+0000 7fbcebe00640 1 --2- 172.21.15.39:0/1478675051 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcec0b59e0 0x7fbcec09f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:18.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.202+0000 7fbce3400640 1 -- 172.21.15.39:0/1478675051 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcec0b59e0 msgr2=0x7fbcec09f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:18.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.202+0000 7fbce3400640 1 --2- 172.21.15.39:0/1478675051 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcec0b59e0 0x7fbcec09f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:18.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.202+0000 7fbce3400640 1 -- 172.21.15.39:0/1478675051 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcec09f7d0 con 0x7fbcec0d0e60 2024-09-21T07:45:18.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.202+0000 7fbcebe00640 1 --2- 172.21.15.39:0/1478675051 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcec0b59e0 0x7fbcec09f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:45:18.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.202+0000 7fbce3400640 1 --2- 172.21.15.39:0/1478675051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcec0d0e60 0x7fbcec152770 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400c370 tx=0x7fbcd400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:18.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.203+0000 7fbce8c00640 1 -- 172.21.15.39:0/1478675051 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4006890 con 0x7fbcec0d0e60 2024-09-21T07:45:18.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.203+0000 7fbce8c00640 1 -- 172.21.15.39:0/1478675051 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbcd4006b20 con 0x7fbcec0d0e60 2024-09-21T07:45:18.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.203+0000 7fbcf1c00640 1 -- 172.21.15.39:0/1478675051 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcec0a18a0 con 0x7fbcec0d0e60 2024-09-21T07:45:18.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.203+0000 7fbce8c00640 1 -- 172.21.15.39:0/1478675051 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4007020 con 0x7fbcec0d0e60 2024-09-21T07:45:18.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.203+0000 7fbcf1c00640 1 -- 172.21.15.39:0/1478675051 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcec0a1ff0 con 0x7fbcec0d0e60 2024-09-21T07:45:18.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.204+0000 7fbcf1c00640 1 -- 172.21.15.39:0/1478675051 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcec0d3240 con 0x7fbcec0d0e60 2024-09-21T07:45:18.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.205+0000 7fbce8c00640 1 -- 172.21.15.39:0/1478675051 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbcd4019020 con 0x7fbcec0d0e60 2024-09-21T07:45:18.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.207+0000 7fbce8c00640 1 --2- 172.21.15.39:0/1478675051 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbcc8078b90 0x7fbcc807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:18.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.207+0000 7fbce8c00640 1 -- 172.21.15.39:0/1478675051 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 6888+0+0 (secure 0 0 0) 0x7fbcd409a900 con 0x7fbcec0d0e60 2024-09-21T07:45:18.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.207+0000 7fbcebe00640 1 --2- 172.21.15.39:0/1478675051 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbcc8078b90 0x7fbcc807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:18.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.208+0000 7fbcebe00640 1 --2- 172.21.15.39:0/1478675051 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbcc8078b90 0x7fbcc807b0a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc034000 tx=0x7fbcdc004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:18.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.212+0000 7fbce8c00640 1 -- 172.21.15.39:0/1478675051 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbcd4062930 con 0x7fbcec0d0e60 2024-09-21T07:45:18.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.419+0000 7fbcf1c00640 1 -- 172.21.15.39:0/1478675051 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbcec0d1d40 con 0x7fbcec0d0e60 2024-09-21T07:45:18.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.419+0000 7fbce8c00640 1 -- 172.21.15.39:0/1478675051 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v142) ==== 80+0+0 (secure 0 0 0) 0x7fbcd4062230 con 0x7fbcec0d0e60 2024-09-21T07:45:18.420 INFO:teuthology.orchestra.run.smithi039.stderr:listed 0 entries 2024-09-21T07:45:18.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.422+0000 7fbcf1c00640 1 -- 172.21.15.39:0/1478675051 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbcc8078b90 msgr2=0x7fbcc807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:18.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.422+0000 7fbcf1c00640 1 --2- 172.21.15.39:0/1478675051 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbcc8078b90 0x7fbcc807b0a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc034000 tx=0x7fbcdc004730 comp rx=0 tx=0).stop 2024-09-21T07:45:18.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.422+0000 7fbcf1c00640 1 -- 172.21.15.39:0/1478675051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcec0d0e60 msgr2=0x7fbcec152770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:18.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.422+0000 7fbcf1c00640 1 --2- 172.21.15.39:0/1478675051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcec0d0e60 0x7fbcec152770 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400c370 tx=0x7fbcd400c830 comp rx=0 tx=0).stop 2024-09-21T07:45:18.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.422+0000 7fbcf1c00640 1 -- 172.21.15.39:0/1478675051 shutdown_connections 2024-09-21T07:45:18.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.422+0000 7fbcf1c00640 1 --2- 172.21.15.39:0/1478675051 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbcc8078b90 0x7fbcc807b0a0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:18.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.422+0000 7fbcf1c00640 1 --2- 172.21.15.39:0/1478675051 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbcec0d5060 0x7fbcec1e7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:18.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.422+0000 7fbcf1c00640 1 --2- 172.21.15.39:0/1478675051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcec0d0e60 0x7fbcec152770 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:18.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.422+0000 7fbcf1c00640 1 --2- 172.21.15.39:0/1478675051 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbcec0b59e0 0x7fbcec09f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:18.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.422+0000 7fbcf1c00640 1 -- 172.21.15.39:0/1478675051 >> 172.21.15.39:0/1478675051 conn(0x7fbcec09d7d0 msgr2=0x7fbcec0b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:18.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.422+0000 7fbcf1c00640 1 -- 172.21.15.39:0/1478675051 shutdown_connections 2024-09-21T07:45:18.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.422+0000 7fbcf1c00640 1 -- 172.21.15.39:0/1478675051 wait complete. 2024-09-21T07:45:18.443 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T07:45:18.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:18.444 DEBUG:teuthology.orchestra.run.smithi039:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T07:45:18.540 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T07:45:18.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:18.540 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:45:18.599 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:45:18.600 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:45:18.600 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:45:18.637 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:45:18.638 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:45:18.638 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:45:18.638 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:45:18.674 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:45:18.675 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:45:18.675 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:45:18.675 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:45:18.713 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:45:18.714 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:45:18.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.962+0000 7f550ea00640 1 -- 172.21.15.39:0/850352620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55080b3670 msgr2=0x7f55080b3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:18.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.962+0000 7f550ea00640 1 --2- 172.21.15.39:0/850352620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55080b3670 0x7f55080b3a80 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f54f4005fd0 tx=0x7f54f402a530 comp rx=0 tx=0).stop 2024-09-21T07:45:18.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.962+0000 7f550ea00640 1 -- 172.21.15.39:0/850352620 shutdown_connections 2024-09-21T07:45:18.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.962+0000 7f550ea00640 1 --2- 172.21.15.39:0/850352620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55080b3670 0x7f55080b3a80 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:18.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.962+0000 7f550ea00640 1 --2- 172.21.15.39:0/850352620 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f55080b4c40 0x7f55080ce2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:18.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.962+0000 7f550ea00640 1 -- 172.21.15.39:0/850352620 >> 172.21.15.39:0/850352620 conn(0x7f55080b2460 msgr2=0x7f550806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:18.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.962+0000 7f550ea00640 1 -- 172.21.15.39:0/850352620 shutdown_connections 2024-09-21T07:45:18.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.962+0000 7f550ea00640 1 -- 172.21.15.39:0/850352620 wait complete. 2024-09-21T07:45:18.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.965+0000 7f550ea00640 1 Processor -- start 2024-09-21T07:45:18.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.965+0000 7f550ea00640 1 -- start start 2024-09-21T07:45:18.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.966+0000 7f550ea00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f55080b3670 0x7f55080994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:18.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.966+0000 7f550ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55080b4c40 0x7f55081f5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:18.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.966+0000 7f550ea00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55080ca8a0 0x7f55081e43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:18.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.966+0000 7f550ea00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5508070da0 con 0x7f55080b4c40 2024-09-21T07:45:18.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.966+0000 7f550ea00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5508070c20 con 0x7f55080ca8a0 2024-09-21T07:45:18.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.966+0000 7f550ea00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f55080cb3c0 con 0x7f55080b3670 2024-09-21T07:45:18.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.966+0000 7f5507400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55080b4c40 0x7f55081f5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:18.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.966+0000 7f5507400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55080b4c40 0x7f55081f5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46754/0 (socket says 172.21.15.39:46754) 2024-09-21T07:45:18.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.966+0000 7f5507400640 1 -- 172.21.15.39:0/2119559271 learned_addr learned my addr 172.21.15.39:0/2119559271 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:18.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.967+0000 7f5507e00640 1 --2- 172.21.15.39:0/2119559271 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f55080b3670 0x7f55080994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:18.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.967+0000 7f550ca00640 1 --2- 172.21.15.39:0/2119559271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55080ca8a0 0x7f55081e43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:18.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.967+0000 7f5507400640 1 -- 172.21.15.39:0/2119559271 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f55080b3670 msgr2=0x7f55080994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:18.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.967+0000 7f5507400640 1 --2- 172.21.15.39:0/2119559271 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f55080b3670 0x7f55080994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:18.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.967+0000 7f5507400640 1 -- 172.21.15.39:0/2119559271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55080ca8a0 msgr2=0x7f55081e43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:18.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.967+0000 7f5507400640 1 --2- 172.21.15.39:0/2119559271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55080ca8a0 0x7f55081e43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:18.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.967+0000 7f5507400640 1 -- 172.21.15.39:0/2119559271 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55081e5160 con 0x7f55080b4c40 2024-09-21T07:45:18.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.967+0000 7f5507e00640 1 --2- 172.21.15.39:0/2119559271 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f55080b3670 0x7f55080994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:45:18.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.967+0000 7f5507400640 1 --2- 172.21.15.39:0/2119559271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55080b4c40 0x7f55081f5750 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f54f4034040 tx=0x7f54f40039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:18.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.968+0000 7f54ffe00640 1 -- 172.21.15.39:0/2119559271 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f403e030 con 0x7f55080b4c40 2024-09-21T07:45:18.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.968+0000 7f54ffe00640 1 -- 172.21.15.39:0/2119559271 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54f40047a0 con 0x7f55080b4c40 2024-09-21T07:45:18.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.968+0000 7f54ffe00640 1 -- 172.21.15.39:0/2119559271 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f4035040 con 0x7f55080b4c40 2024-09-21T07:45:18.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.968+0000 7f550ea00640 1 -- 172.21.15.39:0/2119559271 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55081e5440 con 0x7f55080b4c40 2024-09-21T07:45:18.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.968+0000 7f550ea00640 1 -- 172.21.15.39:0/2119559271 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55081e5a00 con 0x7f55080b4c40 2024-09-21T07:45:18.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.969+0000 7f550ea00640 1 -- 172.21.15.39:0/2119559271 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55080b3a80 con 0x7f55080b4c40 2024-09-21T07:45:18.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.970+0000 7f54ffe00640 1 -- 172.21.15.39:0/2119559271 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f54f4041030 con 0x7f55080b4c40 2024-09-21T07:45:18.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.971+0000 7f54ffe00640 1 --2- 172.21.15.39:0/2119559271 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f54e8078b90 0x7f54e807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:18.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.971+0000 7f54ffe00640 1 -- 172.21.15.39:0/2119559271 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 6888+0+0 (secure 0 0 0) 0x7f54f40bba00 con 0x7f55080b4c40 2024-09-21T07:45:18.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.972+0000 7f5507e00640 1 --2- 172.21.15.39:0/2119559271 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f54e8078b90 0x7f54e807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:18.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.972+0000 7f5507e00640 1 --2- 172.21.15.39:0/2119559271 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f54e8078b90 0x7f54e807b0a0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f54f800ab40 tx=0x7f54f800a520 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:18.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:18.976+0000 7f54ffe00640 1 -- 172.21.15.39:0/2119559271 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f55080b3fc0 con 0x7f55080b4c40 2024-09-21T07:45:19.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.234+0000 7f550ea00640 1 -- 172.21.15.39:0/2119559271 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f55081e4ef0 con 0x7f55080b4c40 2024-09-21T07:45:19.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.235+0000 7f54ffe00640 1 -- 172.21.15.39:0/2119559271 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 172 v172) ==== 95+0+5237 (secure 0 0 0) 0x7f54f40832b0 con 0x7f55080b4c40 2024-09-21T07:45:19.236 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:45:19.236 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":172,"btime":"2024-09-21T07:44:58:579257+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7127,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3876879654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3876879654},{"type":"v1","addr":"172.21.15.148:6836","nonce":3876879654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":167},{"gid":7130,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3044588529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3044588529},{"type":"v1","addr":"172.21.15.148:6837","nonce":3044588529}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":168},{"gid":8517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/3334092885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3334092885},{"type":"v1","addr":"172.21.15.39:6834","nonce":3334092885}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":168}],"filesystems":[{"mdsmap":{"epoch":172,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:44:55.080850+0000","modified":"2024-09-21T07:44:58.083710+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8520":{"gid":8520,"name":"a","rank":0,"incarnation":169,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/3547777254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3547777254},{"type":"v1","addr":"172.21.15.39:6835","nonce":3547777254}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8520,"qdb_cluster":[8520]},"id":7}]} 2024-09-21T07:45:19.236 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 172 2024-09-21T07:45:19.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.237+0000 7f550ea00640 1 -- 172.21.15.39:0/2119559271 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f54e8078b90 msgr2=0x7f54e807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:19.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.237+0000 7f550ea00640 1 --2- 172.21.15.39:0/2119559271 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f54e8078b90 0x7f54e807b0a0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f54f800ab40 tx=0x7f54f800a520 comp rx=0 tx=0).stop 2024-09-21T07:45:19.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.237+0000 7f550ea00640 1 -- 172.21.15.39:0/2119559271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55080b4c40 msgr2=0x7f55081f5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:19.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.237+0000 7f550ea00640 1 --2- 172.21.15.39:0/2119559271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55080b4c40 0x7f55081f5750 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f54f4034040 tx=0x7f54f40039e0 comp rx=0 tx=0).stop 2024-09-21T07:45:19.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.237+0000 7f550ea00640 1 -- 172.21.15.39:0/2119559271 shutdown_connections 2024-09-21T07:45:19.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.237+0000 7f550ea00640 1 --2- 172.21.15.39:0/2119559271 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f54e8078b90 0x7f54e807b0a0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:19.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.237+0000 7f550ea00640 1 --2- 172.21.15.39:0/2119559271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55080ca8a0 0x7f55081e43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:19.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.237+0000 7f550ea00640 1 --2- 172.21.15.39:0/2119559271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55080b4c40 0x7f55081f5750 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:19.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.237+0000 7f550ea00640 1 --2- 172.21.15.39:0/2119559271 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f55080b3670 0x7f55080994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:19.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.237+0000 7f550ea00640 1 -- 172.21.15.39:0/2119559271 >> 172.21.15.39:0/2119559271 conn(0x7f55080b2460 msgr2=0x7f5508068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:19.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.238+0000 7f550ea00640 1 -- 172.21.15.39:0/2119559271 shutdown_connections 2024-09-21T07:45:19.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.238+0000 7f550ea00640 1 -- 172.21.15.39:0/2119559271 wait complete. 2024-09-21T07:45:19.263 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:45:19.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.488+0000 7ff434c00640 1 -- 172.21.15.39:0/1107553151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4300b30a0 msgr2=0x7ff4300cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:19.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.488+0000 7ff434c00640 1 --2- 172.21.15.39:0/1107553151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4300b30a0 0x7ff4300cb8a0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7ff420005fd0 tx=0x7ff42002a5f0 comp rx=0 tx=0).stop 2024-09-21T07:45:19.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.488+0000 7ff434c00640 1 -- 172.21.15.39:0/1107553151 shutdown_connections 2024-09-21T07:45:19.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.488+0000 7ff434c00640 1 --2- 172.21.15.39:0/1107553151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4300b30a0 0x7ff4300cb8a0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:19.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.488+0000 7ff434c00640 1 -- 172.21.15.39:0/1107553151 >> 172.21.15.39:0/1107553151 conn(0x7ff4300b2460 msgr2=0x7ff43006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:19.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.488+0000 7ff434c00640 1 -- 172.21.15.39:0/1107553151 shutdown_connections 2024-09-21T07:45:19.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.488+0000 7ff434c00640 1 -- 172.21.15.39:0/1107553151 wait complete. 2024-09-21T07:45:19.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.490+0000 7ff434c00640 1 Processor -- start 2024-09-21T07:45:19.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.491+0000 7ff434c00640 1 -- start start 2024-09-21T07:45:19.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.492+0000 7ff434c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4300b30a0 0x7ff4301f8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:19.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.492+0000 7ff434c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4300ccc60 0x7ff4301f8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:19.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.492+0000 7ff434c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff43015d830 0x7ff4300ce590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:19.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.492+0000 7ff434c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff4300af4f0 con 0x7ff4300b30a0 2024-09-21T07:45:19.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.492+0000 7ff434c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff4300af370 con 0x7ff4300ccc60 2024-09-21T07:45:19.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.492+0000 7ff434c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7ff4300af670 con 0x7ff43015d830 2024-09-21T07:45:19.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.492+0000 7ff42de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4300b30a0 0x7ff4301f8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:19.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.492+0000 7ff42de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4300b30a0 0x7ff4301f8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46758/0 (socket says 172.21.15.39:46758) 2024-09-21T07:45:19.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.492+0000 7ff42de00640 1 -- 172.21.15.39:0/3477718819 learned_addr learned my addr 172.21.15.39:0/3477718819 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:19.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.493+0000 7ff427e00640 1 --2- 172.21.15.39:0/3477718819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4300ccc60 0x7ff4301f8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:19.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.493+0000 7ff42e800640 1 --2- 172.21.15.39:0/3477718819 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff43015d830 0x7ff4300ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:19.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.493+0000 7ff42de00640 1 -- 172.21.15.39:0/3477718819 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff43015d830 msgr2=0x7ff4300ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:19.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.493+0000 7ff42de00640 1 --2- 172.21.15.39:0/3477718819 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff43015d830 0x7ff4300ce590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:19.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.493+0000 7ff42de00640 1 -- 172.21.15.39:0/3477718819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4300ccc60 msgr2=0x7ff4301f8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:19.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.493+0000 7ff42de00640 1 --2- 172.21.15.39:0/3477718819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4300ccc60 0x7ff4301f8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:19.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.493+0000 7ff42de00640 1 -- 172.21.15.39:0/3477718819 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4300d08b0 con 0x7ff4300b30a0 2024-09-21T07:45:19.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.493+0000 7ff427e00640 1 --2- 172.21.15.39:0/3477718819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4300ccc60 0x7ff4301f8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:45:19.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.493+0000 7ff42de00640 1 --2- 172.21.15.39:0/3477718819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4300b30a0 0x7ff4301f8860 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7ff420034040 tx=0x7ff420003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:19.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.493+0000 7ff426000640 1 -- 172.21.15.39:0/3477718819 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42003e030 con 0x7ff4300b30a0 2024-09-21T07:45:19.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.493+0000 7ff426000640 1 -- 172.21.15.39:0/3477718819 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff420003e00 con 0x7ff4300b30a0 2024-09-21T07:45:19.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.493+0000 7ff426000640 1 -- 172.21.15.39:0/3477718819 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420035040 con 0x7ff4300b30a0 2024-09-21T07:45:19.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.493+0000 7ff434c00640 1 -- 172.21.15.39:0/3477718819 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff430074bb0 con 0x7ff4300b30a0 2024-09-21T07:45:19.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.494+0000 7ff434c00640 1 -- 172.21.15.39:0/3477718819 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4300d0b70 con 0x7ff4300b30a0 2024-09-21T07:45:19.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.495+0000 7ff434c00640 1 -- 172.21.15.39:0/3477718819 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4300d0f50 con 0x7ff4300b30a0 2024-09-21T07:45:19.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.496+0000 7ff426000640 1 -- 172.21.15.39:0/3477718819 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff420041030 con 0x7ff4300b30a0 2024-09-21T07:45:19.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.497+0000 7ff426000640 1 --2- 172.21.15.39:0/3477718819 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff40c078bc0 0x7ff40c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:19.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.497+0000 7ff426000640 1 -- 172.21.15.39:0/3477718819 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 6888+0+0 (secure 0 0 0) 0x7ff4200bba80 con 0x7ff4300b30a0 2024-09-21T07:45:19.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.497+0000 7ff427e00640 1 --2- 172.21.15.39:0/3477718819 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff40c078bc0 0x7ff40c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:19.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.498+0000 7ff427e00640 1 --2- 172.21.15.39:0/3477718819 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff40c078bc0 0x7ff40c07b0d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff418007fd0 tx=0x7ff418009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:19.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.502+0000 7ff426000640 1 -- 172.21.15.39:0/3477718819 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff4300d0f50 con 0x7ff4300b30a0 2024-09-21T07:45:19.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.758+0000 7ff434c00640 1 -- 172.21.15.39:0/3477718819 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4300cb8a0 con 0x7ff4300b30a0 2024-09-21T07:45:19.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.759+0000 7ff426000640 1 -- 172.21.15.39:0/3477718819 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 172 v172) ==== 95+0+5237 (secure 0 0 0) 0x7ff4200bde20 con 0x7ff4300b30a0 2024-09-21T07:45:19.762 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:45:19.762 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":172,"btime":"2024-09-21T07:44:58:579257+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7127,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3876879654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3876879654},{"type":"v1","addr":"172.21.15.148:6836","nonce":3876879654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":167},{"gid":7130,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3044588529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3044588529},{"type":"v1","addr":"172.21.15.148:6837","nonce":3044588529}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":168},{"gid":8517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/3334092885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3334092885},{"type":"v1","addr":"172.21.15.39:6834","nonce":3334092885}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":168}],"filesystems":[{"mdsmap":{"epoch":172,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:44:55.080850+0000","modified":"2024-09-21T07:44:58.083710+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8520":{"gid":8520,"name":"a","rank":0,"incarnation":169,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/3547777254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3547777254},{"type":"v1","addr":"172.21.15.39:6835","nonce":3547777254}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8520,"qdb_cluster":[8520]},"id":7}]} 2024-09-21T07:45:19.762 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 172 2024-09-21T07:45:19.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.762+0000 7ff434c00640 1 -- 172.21.15.39:0/3477718819 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff40c078bc0 msgr2=0x7ff40c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:19.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.762+0000 7ff434c00640 1 --2- 172.21.15.39:0/3477718819 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff40c078bc0 0x7ff40c07b0d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff418007fd0 tx=0x7ff418009040 comp rx=0 tx=0).stop 2024-09-21T07:45:19.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.762+0000 7ff434c00640 1 -- 172.21.15.39:0/3477718819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4300b30a0 msgr2=0x7ff4301f8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:19.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.762+0000 7ff434c00640 1 --2- 172.21.15.39:0/3477718819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4300b30a0 0x7ff4301f8860 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7ff420034040 tx=0x7ff420003910 comp rx=0 tx=0).stop 2024-09-21T07:45:19.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.762+0000 7ff434c00640 1 -- 172.21.15.39:0/3477718819 shutdown_connections 2024-09-21T07:45:19.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.762+0000 7ff434c00640 1 --2- 172.21.15.39:0/3477718819 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff40c078bc0 0x7ff40c07b0d0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:19.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.762+0000 7ff434c00640 1 --2- 172.21.15.39:0/3477718819 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff43015d830 0x7ff4300ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:19.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.762+0000 7ff434c00640 1 --2- 172.21.15.39:0/3477718819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4300ccc60 0x7ff4301f8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:19.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.762+0000 7ff434c00640 1 --2- 172.21.15.39:0/3477718819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4300b30a0 0x7ff4301f8860 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:19.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.762+0000 7ff434c00640 1 -- 172.21.15.39:0/3477718819 >> 172.21.15.39:0/3477718819 conn(0x7ff4300b2460 msgr2=0x7ff43015f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:19.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.762+0000 7ff434c00640 1 -- 172.21.15.39:0/3477718819 shutdown_connections 2024-09-21T07:45:19.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:19.762+0000 7ff434c00640 1 -- 172.21.15.39:0/3477718819 wait complete. 2024-09-21T07:45:19.785 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:45:20.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.011+0000 7f3abe800640 1 -- 172.21.15.39:0/303183867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ab80cf450 msgr2=0x7f3ab80cf860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:20.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.011+0000 7f3abe800640 1 --2- 172.21.15.39:0/303183867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ab80cf450 0x7f3ab80cf860 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f3ab80cece0 tx=0x7f3aa002a560 comp rx=0 tx=0).stop 2024-09-21T07:45:20.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.011+0000 7f3abe800640 1 -- 172.21.15.39:0/303183867 shutdown_connections 2024-09-21T07:45:20.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.011+0000 7f3abe800640 1 --2- 172.21.15.39:0/303183867 >> v2:172.21.15.148:3300/0 conn(0x7f3ab80d0630 0x7f3ab80d0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:20.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.011+0000 7f3abe800640 1 --2- 172.21.15.39:0/303183867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ab80cf450 0x7f3ab80cf860 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:20.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.011+0000 7f3abe800640 1 -- 172.21.15.39:0/303183867 >> 172.21.15.39:0/303183867 conn(0x7f3ab806cfc0 msgr2=0x7f3ab806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:20.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.011+0000 7f3abe800640 1 -- 172.21.15.39:0/303183867 shutdown_connections 2024-09-21T07:45:20.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.011+0000 7f3abe800640 1 -- 172.21.15.39:0/303183867 wait complete. 2024-09-21T07:45:20.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.014+0000 7f3abe800640 1 Processor -- start 2024-09-21T07:45:20.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.014+0000 7f3abe800640 1 -- start start 2024-09-21T07:45:20.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.015+0000 7f3abe800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ab8073dd0 0x7f3ab814d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:20.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.015+0000 7f3abe800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ab80cf450 0x7f3ab815ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:20.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.015+0000 7f3abe800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ab80d0630 0x7f3ab815bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:20.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.015+0000 7f3abe800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab80cd1a0 con 0x7f3ab8073dd0 2024-09-21T07:45:20.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.015+0000 7f3abe800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab80cd020 con 0x7f3ab80d0630 2024-09-21T07:45:20.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.015+0000 7f3abe800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f3ab80cd320 con 0x7f3ab80cf450 2024-09-21T07:45:20.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.015+0000 7f3ab7400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ab8073dd0 0x7f3ab814d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:20.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.015+0000 7f3ab7400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ab8073dd0 0x7f3ab814d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46768/0 (socket says 172.21.15.39:46768) 2024-09-21T07:45:20.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.015+0000 7f3ab7400640 1 -- 172.21.15.39:0/3668810900 learned_addr learned my addr 172.21.15.39:0/3668810900 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:20.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.016+0000 7f3ab7e00640 1 --2- 172.21.15.39:0/3668810900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ab80d0630 0x7f3ab815bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:20.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.016+0000 7f3aaea00640 1 --2- 172.21.15.39:0/3668810900 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ab80cf450 0x7f3ab815ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:20.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.016+0000 7f3ab7400640 1 -- 172.21.15.39:0/3668810900 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ab80cf450 msgr2=0x7f3ab815ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:20.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.016+0000 7f3ab7400640 1 --2- 172.21.15.39:0/3668810900 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ab80cf450 0x7f3ab815ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:20.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.016+0000 7f3ab7400640 1 -- 172.21.15.39:0/3668810900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ab80d0630 msgr2=0x7f3ab815bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:20.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.016+0000 7f3ab7400640 1 --2- 172.21.15.39:0/3668810900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ab80d0630 0x7f3ab815bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:20.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.016+0000 7f3ab7400640 1 -- 172.21.15.39:0/3668810900 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab81f3660 con 0x7f3ab8073dd0 2024-09-21T07:45:20.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.016+0000 7f3ab7400640 1 --2- 172.21.15.39:0/3668810900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ab8073dd0 0x7f3ab814d7e0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f3aa800c340 tx=0x7f3aa800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:20.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.016+0000 7f3aafe00640 1 -- 172.21.15.39:0/3668810900 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa80052d0 con 0x7f3ab8073dd0 2024-09-21T07:45:20.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.017+0000 7f3aafe00640 1 -- 172.21.15.39:0/3668810900 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3aa8007020 con 0x7f3ab8073dd0 2024-09-21T07:45:20.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.017+0000 7f3aafe00640 1 -- 172.21.15.39:0/3668810900 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa8007470 con 0x7f3ab8073dd0 2024-09-21T07:45:20.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.017+0000 7f3abe800640 1 -- 172.21.15.39:0/3668810900 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ab81f3920 con 0x7f3ab8073dd0 2024-09-21T07:45:20.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.017+0000 7f3abe800640 1 -- 172.21.15.39:0/3668810900 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ab81f3ee0 con 0x7f3ab8073dd0 2024-09-21T07:45:20.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.018+0000 7f3abe800640 1 -- 172.21.15.39:0/3668810900 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ab80cd840 con 0x7f3ab8073dd0 2024-09-21T07:45:20.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.019+0000 7f3aafe00640 1 -- 172.21.15.39:0/3668810900 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3aa80067d0 con 0x7f3ab8073dd0 2024-09-21T07:45:20.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.020+0000 7f3aafe00640 1 --2- 172.21.15.39:0/3668810900 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3a94078b90 0x7f3a9407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:20.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.020+0000 7f3aafe00640 1 -- 172.21.15.39:0/3668810900 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 6888+0+0 (secure 0 0 0) 0x7f3aa809a690 con 0x7f3ab8073dd0 2024-09-21T07:45:20.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.020+0000 7f3aaea00640 1 --2- 172.21.15.39:0/3668810900 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3a94078b90 0x7f3a9407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:20.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.022+0000 7f3aaea00640 1 --2- 172.21.15.39:0/3668810900 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3a94078b90 0x7f3a9407b0a0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0033000 tx=0x7f3aa0004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:20.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.024+0000 7f3aafe00640 1 -- 172.21.15.39:0/3668810900 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3ab80d0f80 con 0x7f3ab8073dd0 2024-09-21T07:45:20.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.225+0000 7f3abe800640 1 -- 172.21.15.39:0/3668810900 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3ab80b0280 con 0x7f3ab8073dd0 2024-09-21T07:45:20.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.227+0000 7f3aafe00640 1 -- 172.21.15.39:0/3668810900 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) ==== 74+0+22367 (secure 0 0 0) 0x7f3aa8061f40 con 0x7f3ab8073dd0 2024-09-21T07:45:20.229 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:45:20.229 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":144,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:45:19.496820+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:44:53.006035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-09-21T07:44:54.032155+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":200,"snap_epoch":144,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":11,"length":90},{"begin":111,"length":90}]}],"new_removed_snaps":[{"pool":18,"snaps":[{"begin":100,"length":1},{"begin":200,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:45:20.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.230+0000 7f3abe800640 1 -- 172.21.15.39:0/3668810900 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3a94078b90 msgr2=0x7f3a9407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:20.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.230+0000 7f3abe800640 1 --2- 172.21.15.39:0/3668810900 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3a94078b90 0x7f3a9407b0a0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0033000 tx=0x7f3aa0004730 comp rx=0 tx=0).stop 2024-09-21T07:45:20.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.230+0000 7f3abe800640 1 -- 172.21.15.39:0/3668810900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ab8073dd0 msgr2=0x7f3ab814d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:20.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.230+0000 7f3abe800640 1 --2- 172.21.15.39:0/3668810900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ab8073dd0 0x7f3ab814d7e0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f3aa800c340 tx=0x7f3aa800c800 comp rx=0 tx=0).stop 2024-09-21T07:45:20.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.230+0000 7f3abe800640 1 -- 172.21.15.39:0/3668810900 shutdown_connections 2024-09-21T07:45:20.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.230+0000 7f3abe800640 1 --2- 172.21.15.39:0/3668810900 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3a94078b90 0x7f3a9407b0a0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:20.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.230+0000 7f3abe800640 1 --2- 172.21.15.39:0/3668810900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ab80d0630 0x7f3ab815bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:20.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.230+0000 7f3abe800640 1 --2- 172.21.15.39:0/3668810900 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3ab80cf450 0x7f3ab815ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:20.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.230+0000 7f3abe800640 1 --2- 172.21.15.39:0/3668810900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ab8073dd0 0x7f3ab814d7e0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:20.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.230+0000 7f3abe800640 1 -- 172.21.15.39:0/3668810900 >> 172.21.15.39:0/3668810900 conn(0x7f3ab806cfc0 msgr2=0x7f3ab806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:20.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.230+0000 7f3abe800640 1 -- 172.21.15.39:0/3668810900 shutdown_connections 2024-09-21T07:45:20.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.230+0000 7f3abe800640 1 -- 172.21.15.39:0/3668810900 wait complete. 2024-09-21T07:45:20.255 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T07:45:20.255 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:45:20.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.480+0000 7fa47a200640 1 -- 172.21.15.39:0/3587456514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa458004800 msgr2=0x7fa458004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:20.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.480+0000 7fa47a200640 1 --2- 172.21.15.39:0/3587456514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa458004800 0x7fa458004c10 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fa460006f70 tx=0x7fa458005830 comp rx=0 tx=0).stop 2024-09-21T07:45:20.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.480+0000 7fa47a200640 1 -- 172.21.15.39:0/3587456514 shutdown_connections 2024-09-21T07:45:20.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.480+0000 7fa47a200640 1 --2- 172.21.15.39:0/3587456514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa458004800 0x7fa458004c10 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:20.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.480+0000 7fa47a200640 1 -- 172.21.15.39:0/3587456514 >> 172.21.15.39:0/3587456514 conn(0x7fa4740b2460 msgr2=0x7fa47406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:20.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.481+0000 7fa47a200640 1 -- 172.21.15.39:0/3587456514 shutdown_connections 2024-09-21T07:45:20.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.481+0000 7fa47a200640 1 -- 172.21.15.39:0/3587456514 wait complete. 2024-09-21T07:45:20.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.483+0000 7fa47a200640 1 Processor -- start 2024-09-21T07:45:20.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.483+0000 7fa47a200640 1 -- start start 2024-09-21T07:45:20.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.485+0000 7fa47a200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa458004800 0x7fa4741f54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:20.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.485+0000 7fa47a200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4740ad060 0x7fa4740c64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:20.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.485+0000 7fa47a200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4740ad470 0x7fa4740c6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:20.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.485+0000 7fa47a200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa474156bb0 con 0x7fa4740ad060 2024-09-21T07:45:20.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.485+0000 7fa47a200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa4740cbb00 con 0x7fa458004800 2024-09-21T07:45:20.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.485+0000 7fa47a200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa474156d30 con 0x7fa4740ad470 2024-09-21T07:45:20.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.485+0000 7fa472a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4740ad060 0x7fa4740c64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:20.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.485+0000 7fa472a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4740ad060 0x7fa4740c64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46790/0 (socket says 172.21.15.39:46790) 2024-09-21T07:45:20.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.485+0000 7fa472a00640 1 -- 172.21.15.39:0/2216107734 learned_addr learned my addr 172.21.15.39:0/2216107734 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:20.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.485+0000 7fa472a00640 1 -- 172.21.15.39:0/2216107734 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4740ad470 msgr2=0x7fa4740c6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:20.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.485+0000 7fa472a00640 1 --2- 172.21.15.39:0/2216107734 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4740ad470 0x7fa4740c6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:20.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.485+0000 7fa472a00640 1 -- 172.21.15.39:0/2216107734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa458004800 msgr2=0x7fa4741f54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:20.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.485+0000 7fa472a00640 1 --2- 172.21.15.39:0/2216107734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa458004800 0x7fa4741f54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:20.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.485+0000 7fa472a00640 1 -- 172.21.15.39:0/2216107734 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4741f5a20 con 0x7fa4740ad060 2024-09-21T07:45:20.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.486+0000 7fa472a00640 1 --2- 172.21.15.39:0/2216107734 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4740ad060 0x7fa4740c64f0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fa460031040 tx=0x7fa460004f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:20.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.486+0000 7fa46b400640 1 -- 172.21.15.39:0/2216107734 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa460001bf0 con 0x7fa4740ad060 2024-09-21T07:45:20.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.486+0000 7fa46b400640 1 -- 172.21.15.39:0/2216107734 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4600060c0 con 0x7fa4740ad060 2024-09-21T07:45:20.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.486+0000 7fa46b400640 1 -- 172.21.15.39:0/2216107734 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa460042680 con 0x7fa4740ad060 2024-09-21T07:45:20.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.486+0000 7fa47a200640 1 -- 172.21.15.39:0/2216107734 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa47414ce20 con 0x7fa4740ad060 2024-09-21T07:45:20.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.486+0000 7fa47a200640 1 -- 172.21.15.39:0/2216107734 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa47414d3e0 con 0x7fa4740ad060 2024-09-21T07:45:20.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.487+0000 7fa47a200640 1 -- 172.21.15.39:0/2216107734 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4740cd070 con 0x7fa4740ad060 2024-09-21T07:45:20.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.488+0000 7fa46b400640 1 -- 172.21.15.39:0/2216107734 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa460035020 con 0x7fa4740ad060 2024-09-21T07:45:20.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.489+0000 7fa46b400640 1 --2- 172.21.15.39:0/2216107734 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa454078bc0 0x7fa45407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:20.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.489+0000 7fa46b400640 1 -- 172.21.15.39:0/2216107734 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 6888+0+0 (secure 0 0 0) 0x7fa4600bb930 con 0x7fa4740ad060 2024-09-21T07:45:20.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.490+0000 7fa473400640 1 --2- 172.21.15.39:0/2216107734 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa454078bc0 0x7fa45407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:20.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.491+0000 7fa473400640 1 --2- 172.21.15.39:0/2216107734 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa454078bc0 0x7fa45407b0d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa464009cc0 tx=0x7fa464011810 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:20.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.494+0000 7fa46b400640 1 -- 172.21.15.39:0/2216107734 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa4600838c0 con 0x7fa4740ad060 2024-09-21T07:45:20.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.732+0000 7fa47a200640 1 -- 172.21.15.39:0/2216107734 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4740ad890 con 0x7fa4740ad060 2024-09-21T07:45:20.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.733+0000 7fa46b400640 1 -- 172.21.15.39:0/2216107734 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 172 v172) ==== 95+0+5237 (secure 0 0 0) 0x7fa4600831c0 con 0x7fa4740ad060 2024-09-21T07:45:20.735 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:45:20.735 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":172,"btime":"2024-09-21T07:44:58:579257+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7127,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3876879654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3876879654},{"type":"v1","addr":"172.21.15.148:6836","nonce":3876879654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":167},{"gid":7130,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3044588529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3044588529},{"type":"v1","addr":"172.21.15.148:6837","nonce":3044588529}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":168},{"gid":8517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/3334092885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3334092885},{"type":"v1","addr":"172.21.15.39:6834","nonce":3334092885}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":168}],"filesystems":[{"mdsmap":{"epoch":172,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:44:55.080850+0000","modified":"2024-09-21T07:44:58.083710+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8520":{"gid":8520,"name":"a","rank":0,"incarnation":169,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/3547777254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3547777254},{"type":"v1","addr":"172.21.15.39:6835","nonce":3547777254}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8520,"qdb_cluster":[8520]},"id":7}]} 2024-09-21T07:45:20.735 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 172 2024-09-21T07:45:20.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.736+0000 7fa47a200640 1 -- 172.21.15.39:0/2216107734 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa454078bc0 msgr2=0x7fa45407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:20.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.736+0000 7fa47a200640 1 --2- 172.21.15.39:0/2216107734 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa454078bc0 0x7fa45407b0d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa464009cc0 tx=0x7fa464011810 comp rx=0 tx=0).stop 2024-09-21T07:45:20.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.736+0000 7fa47a200640 1 -- 172.21.15.39:0/2216107734 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4740ad060 msgr2=0x7fa4740c64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:20.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.736+0000 7fa47a200640 1 --2- 172.21.15.39:0/2216107734 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4740ad060 0x7fa4740c64f0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fa460031040 tx=0x7fa460004f70 comp rx=0 tx=0).stop 2024-09-21T07:45:20.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.736+0000 7fa47a200640 1 -- 172.21.15.39:0/2216107734 shutdown_connections 2024-09-21T07:45:20.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.736+0000 7fa47a200640 1 --2- 172.21.15.39:0/2216107734 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa454078bc0 0x7fa45407b0d0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:20.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.736+0000 7fa47a200640 1 --2- 172.21.15.39:0/2216107734 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4740ad470 0x7fa4740c6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:20.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.736+0000 7fa47a200640 1 --2- 172.21.15.39:0/2216107734 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4740ad060 0x7fa4740c64f0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:20.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.736+0000 7fa47a200640 1 --2- 172.21.15.39:0/2216107734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa458004800 0x7fa4741f54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:20.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.736+0000 7fa47a200640 1 -- 172.21.15.39:0/2216107734 >> 172.21.15.39:0/2216107734 conn(0x7fa4740b2460 msgr2=0x7fa474073830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:20.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.736+0000 7fa47a200640 1 -- 172.21.15.39:0/2216107734 shutdown_connections 2024-09-21T07:45:20.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.736+0000 7fa47a200640 1 -- 172.21.15.39:0/2216107734 wait complete. 2024-09-21T07:45:20.758 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:45:20.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.990+0000 7f6e8d400640 1 -- 172.21.15.39:0/473864656 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e880b3da0 msgr2=0x7f6e880b4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:20.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.990+0000 7f6e8d400640 1 --2- 172.21.15.39:0/473864656 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e880b3da0 0x7f6e880b4990 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f6e880b2e40 tx=0x7f6e8002aa30 comp rx=0 tx=0).stop 2024-09-21T07:45:20.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.990+0000 7f6e8d400640 1 -- 172.21.15.39:0/473864656 shutdown_connections 2024-09-21T07:45:20.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.990+0000 7f6e8d400640 1 --2- 172.21.15.39:0/473864656 >> v2:172.21.15.148:3300/0 conn(0x7f6e880b3990 0x7f6e880b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:20.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.990+0000 7f6e8d400640 1 --2- 172.21.15.39:0/473864656 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e880b3da0 0x7f6e880b4990 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:20.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.990+0000 7f6e8d400640 1 -- 172.21.15.39:0/473864656 >> 172.21.15.39:0/473864656 conn(0x7f6e880b2460 msgr2=0x7f6e8806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:20.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.990+0000 7f6e8d400640 1 -- 172.21.15.39:0/473864656 shutdown_connections 2024-09-21T07:45:20.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.990+0000 7f6e8d400640 1 -- 172.21.15.39:0/473864656 wait complete. 2024-09-21T07:45:20.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.992+0000 7f6e8d400640 1 Processor -- start 2024-09-21T07:45:20.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.992+0000 7f6e8d400640 1 -- start start 2024-09-21T07:45:20.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.994+0000 7f6e8d400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e880b3990 0x7f6e8809d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:20.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.994+0000 7f6e8d400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e880b3da0 0x7f6e88073e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:20.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.994+0000 7f6e8d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e88162f10 0x7f6e88074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:20.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.994+0000 7f6e8d400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6e880af200 con 0x7f6e88162f10 2024-09-21T07:45:20.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.994+0000 7f6e8d400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6e880af080 con 0x7f6e880b3990 2024-09-21T07:45:20.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.994+0000 7f6e8d400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f6e880af380 con 0x7f6e880b3da0 2024-09-21T07:45:20.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.994+0000 7f6e87200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e88162f10 0x7f6e88074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:20.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.994+0000 7f6e87200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e88162f10 0x7f6e88074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46800/0 (socket says 172.21.15.39:46800) 2024-09-21T07:45:20.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.994+0000 7f6e87200640 1 -- 172.21.15.39:0/766040756 learned_addr learned my addr 172.21.15.39:0/766040756 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:20.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.995+0000 7f6e87200640 1 -- 172.21.15.39:0/766040756 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e880b3da0 msgr2=0x7f6e88073e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:20.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.995+0000 7f6e87200640 1 --2- 172.21.15.39:0/766040756 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e880b3da0 0x7f6e88073e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:20.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.995+0000 7f6e87200640 1 -- 172.21.15.39:0/766040756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e880b3990 msgr2=0x7f6e8809d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:20.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.995+0000 7f6e87200640 1 --2- 172.21.15.39:0/766040756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e880b3990 0x7f6e8809d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:20.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.995+0000 7f6e87200640 1 -- 172.21.15.39:0/766040756 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e8815bfa0 con 0x7f6e88162f10 2024-09-21T07:45:20.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.995+0000 7f6e87200640 1 --2- 172.21.15.39:0/766040756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e88162f10 0x7f6e88074340 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f6e80009430 tx=0x7f6e80003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:20.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.995+0000 7f6e7ea00640 1 -- 172.21.15.39:0/766040756 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e80033050 con 0x7f6e88162f10 2024-09-21T07:45:20.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.995+0000 7f6e7ea00640 1 -- 172.21.15.39:0/766040756 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e800044d0 con 0x7f6e88162f10 2024-09-21T07:45:20.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.995+0000 7f6e8d400640 1 -- 172.21.15.39:0/766040756 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e8815e080 con 0x7f6e88162f10 2024-09-21T07:45:20.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.995+0000 7f6e7ea00640 1 -- 172.21.15.39:0/766040756 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e80038450 con 0x7f6e88162f10 2024-09-21T07:45:20.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.996+0000 7f6e8d400640 1 -- 172.21.15.39:0/766040756 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e8815c520 con 0x7f6e88162f10 2024-09-21T07:45:20.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.997+0000 7f6e8d400640 1 -- 172.21.15.39:0/766040756 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e880cdb70 con 0x7f6e88162f10 2024-09-21T07:45:21.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.998+0000 7f6e7ea00640 1 -- 172.21.15.39:0/766040756 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6e80036070 con 0x7f6e88162f10 2024-09-21T07:45:21.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.999+0000 7f6e7ea00640 1 --2- 172.21.15.39:0/766040756 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6e64078bc0 0x7f6e6407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:21.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:20.999+0000 7f6e7ea00640 1 -- 172.21.15.39:0/766040756 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 6888+0+0 (secure 0 0 0) 0x7f6e800bc770 con 0x7f6e88162f10 2024-09-21T07:45:21.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.000+0000 7f6e86800640 1 --2- 172.21.15.39:0/766040756 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6e64078bc0 0x7f6e6407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:21.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.001+0000 7f6e86800640 1 --2- 172.21.15.39:0/766040756 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6e64078bc0 0x7f6e6407b0d0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6e78006fd0 tx=0x7f6e78006bc0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:21.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.003+0000 7f6e7ea00640 1 -- 172.21.15.39:0/766040756 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6e80084700 con 0x7f6e88162f10 2024-09-21T07:45:21.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.240+0000 7f6e8d400640 1 -- 172.21.15.39:0/766040756 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e8806da00 con 0x7f6e88162f10 2024-09-21T07:45:21.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.241+0000 7f6e7ea00640 1 -- 172.21.15.39:0/766040756 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 172 v172) ==== 95+0+5237 (secure 0 0 0) 0x7f6e80084000 con 0x7f6e88162f10 2024-09-21T07:45:21.243 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:45:21.243 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":172,"btime":"2024-09-21T07:44:58:579257+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7127,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3876879654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3876879654},{"type":"v1","addr":"172.21.15.148:6836","nonce":3876879654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":167},{"gid":7130,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3044588529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3044588529},{"type":"v1","addr":"172.21.15.148:6837","nonce":3044588529}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":168},{"gid":8517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/3334092885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3334092885},{"type":"v1","addr":"172.21.15.39:6834","nonce":3334092885}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":168}],"filesystems":[{"mdsmap":{"epoch":172,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:44:55.080850+0000","modified":"2024-09-21T07:44:58.083710+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8520":{"gid":8520,"name":"a","rank":0,"incarnation":169,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/3547777254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3547777254},{"type":"v1","addr":"172.21.15.39:6835","nonce":3547777254}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8520,"qdb_cluster":[8520]},"id":7}]} 2024-09-21T07:45:21.243 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 172 2024-09-21T07:45:21.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.244+0000 7f6e8d400640 1 -- 172.21.15.39:0/766040756 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6e64078bc0 msgr2=0x7f6e6407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:21.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.244+0000 7f6e8d400640 1 --2- 172.21.15.39:0/766040756 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6e64078bc0 0x7f6e6407b0d0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6e78006fd0 tx=0x7f6e78006bc0 comp rx=0 tx=0).stop 2024-09-21T07:45:21.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.244+0000 7f6e8d400640 1 -- 172.21.15.39:0/766040756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e88162f10 msgr2=0x7f6e88074340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:21.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.244+0000 7f6e8d400640 1 --2- 172.21.15.39:0/766040756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e88162f10 0x7f6e88074340 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f6e80009430 tx=0x7f6e80003f10 comp rx=0 tx=0).stop 2024-09-21T07:45:21.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.244+0000 7f6e8d400640 1 -- 172.21.15.39:0/766040756 shutdown_connections 2024-09-21T07:45:21.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.244+0000 7f6e8d400640 1 --2- 172.21.15.39:0/766040756 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6e64078bc0 0x7f6e6407b0d0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:21.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.244+0000 7f6e8d400640 1 --2- 172.21.15.39:0/766040756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e88162f10 0x7f6e88074340 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:21.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.244+0000 7f6e8d400640 1 --2- 172.21.15.39:0/766040756 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e880b3da0 0x7f6e88073e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:21.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.244+0000 7f6e8d400640 1 --2- 172.21.15.39:0/766040756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e880b3990 0x7f6e8809d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:21.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.244+0000 7f6e8d400640 1 -- 172.21.15.39:0/766040756 >> 172.21.15.39:0/766040756 conn(0x7f6e880b2460 msgr2=0x7f6e8806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:21.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.244+0000 7f6e8d400640 1 -- 172.21.15.39:0/766040756 shutdown_connections 2024-09-21T07:45:21.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.244+0000 7f6e8d400640 1 -- 172.21.15.39:0/766040756 wait complete. 2024-09-21T07:45:21.267 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:45:21.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.492+0000 7f190a200640 1 -- 172.21.15.39:0/1327238981 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19040ce690 msgr2=0x7f19040b2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:21.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.492+0000 7f190a200640 1 --2- 172.21.15.39:0/1327238981 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19040ce690 0x7f19040b2f70 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f19040b2c60 tx=0x7f18f802a5d0 comp rx=0 tx=0).stop 2024-09-21T07:45:21.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.492+0000 7f190a200640 1 -- 172.21.15.39:0/1327238981 shutdown_connections 2024-09-21T07:45:21.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.492+0000 7f190a200640 1 --2- 172.21.15.39:0/1327238981 >> v2:172.21.15.148:3300/0 conn(0x7f19040d0060 0x7f19040d0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:21.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.492+0000 7f190a200640 1 --2- 172.21.15.39:0/1327238981 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19040ce690 0x7f19040b2f70 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:21.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.492+0000 7f190a200640 1 -- 172.21.15.39:0/1327238981 >> 172.21.15.39:0/1327238981 conn(0x7f190409b5f0 msgr2=0x7f190408c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:21.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.492+0000 7f190a200640 1 -- 172.21.15.39:0/1327238981 shutdown_connections 2024-09-21T07:45:21.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.492+0000 7f190a200640 1 -- 172.21.15.39:0/1327238981 wait complete. 2024-09-21T07:45:21.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.495+0000 7f190a200640 1 Processor -- start 2024-09-21T07:45:21.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.495+0000 7f190a200640 1 -- start start 2024-09-21T07:45:21.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.496+0000 7f190a200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f19040ce690 0x7f190415b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:21.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.496+0000 7f190a200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19040d0060 0x7f19040b6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:21.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.496+0000 7f190a200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19040d2030 0x7f19040b6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:21.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.496+0000 7f190a200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f19040cd260 con 0x7f19040d2030 2024-09-21T07:45:21.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.496+0000 7f190a200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f19040cd0e0 con 0x7f19040d0060 2024-09-21T07:45:21.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.497+0000 7f190a200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f19040cd3e0 con 0x7f19040ce690 2024-09-21T07:45:21.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.497+0000 7f1909800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19040d2030 0x7f19040b6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:21.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.497+0000 7f1909800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19040d2030 0x7f19040b6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46804/0 (socket says 172.21.15.39:46804) 2024-09-21T07:45:21.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.497+0000 7f1909800640 1 -- 172.21.15.39:0/3294739758 learned_addr learned my addr 172.21.15.39:0/3294739758 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:21.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.497+0000 7f1908e00640 1 --2- 172.21.15.39:0/3294739758 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f19040ce690 0x7f190415b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:21.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.497+0000 7f1903e00640 1 --2- 172.21.15.39:0/3294739758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19040d0060 0x7f19040b6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:21.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.497+0000 7f1909800640 1 -- 172.21.15.39:0/3294739758 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f19040ce690 msgr2=0x7f190415b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:21.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.497+0000 7f1909800640 1 --2- 172.21.15.39:0/3294739758 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f19040ce690 0x7f190415b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:21.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.497+0000 7f1909800640 1 -- 172.21.15.39:0/3294739758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19040d0060 msgr2=0x7f19040b6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:21.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.497+0000 7f1909800640 1 --2- 172.21.15.39:0/3294739758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19040d0060 0x7f19040b6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:21.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.497+0000 7f1909800640 1 -- 172.21.15.39:0/3294739758 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19041f7c70 con 0x7f19040d2030 2024-09-21T07:45:21.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.498+0000 7f1909800640 1 --2- 172.21.15.39:0/3294739758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19040d2030 0x7f19040b6770 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00c730 tx=0x7f18fc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:21.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.498+0000 7f1900c00640 1 -- 172.21.15.39:0/3294739758 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc003b70 con 0x7f19040d2030 2024-09-21T07:45:21.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.498+0000 7f1900c00640 1 -- 172.21.15.39:0/3294739758 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18fc003d10 con 0x7f19040d2030 2024-09-21T07:45:21.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.498+0000 7f1900c00640 1 -- 172.21.15.39:0/3294739758 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc00aab0 con 0x7f19040d2030 2024-09-21T07:45:21.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.498+0000 7f190a200640 1 -- 172.21.15.39:0/3294739758 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19040c9860 con 0x7f19040d2030 2024-09-21T07:45:21.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.498+0000 7f190a200640 1 -- 172.21.15.39:0/3294739758 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19040cbce0 con 0x7f19040d2030 2024-09-21T07:45:21.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.499+0000 7f190a200640 1 -- 172.21.15.39:0/3294739758 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19041e7fa0 con 0x7f19040d2030 2024-09-21T07:45:21.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.500+0000 7f1900c00640 1 -- 172.21.15.39:0/3294739758 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f18fc00ac50 con 0x7f19040d2030 2024-09-21T07:45:21.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.501+0000 7f1900c00640 1 --2- 172.21.15.39:0/3294739758 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f18e8078bc0 0x7f18e807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:21.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.501+0000 7f1900c00640 1 -- 172.21.15.39:0/3294739758 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6888+0+0 (secure 0 0 0) 0x7f18fc09b020 con 0x7f19040d2030 2024-09-21T07:45:21.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.502+0000 7f1908e00640 1 --2- 172.21.15.39:0/3294739758 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f18e8078bc0 0x7f18e807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:21.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.503+0000 7f1908e00640 1 --2- 172.21.15.39:0/3294739758 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f18e8078bc0 0x7f18e807b0d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f18f8034000 tx=0x7f18f8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:21.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.506+0000 7f1900c00640 1 -- 172.21.15.39:0/3294739758 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f18fc062fc0 con 0x7f19040d2030 2024-09-21T07:45:21.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.702+0000 7f190a200640 1 -- 172.21.15.39:0/3294739758 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f190415fe30 con 0x7f19040d2030 2024-09-21T07:45:21.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.703+0000 7f1900c00640 1 -- 172.21.15.39:0/3294739758 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) ==== 74+0+22368 (secure 0 0 0) 0x7f190415fe30 con 0x7f19040d2030 2024-09-21T07:45:21.706 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:45:21.706 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":146,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:45:21.484742+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:44:53.006035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-09-21T07:44:54.032155+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":201,"snap_epoch":145,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":18,"snaps":[{"begin":11,"length":64},{"begin":111,"length":64}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:45:21.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.706+0000 7f190a200640 1 -- 172.21.15.39:0/3294739758 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f18e8078bc0 msgr2=0x7f18e807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:21.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.706+0000 7f190a200640 1 --2- 172.21.15.39:0/3294739758 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f18e8078bc0 0x7f18e807b0d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f18f8034000 tx=0x7f18f8004730 comp rx=0 tx=0).stop 2024-09-21T07:45:21.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.706+0000 7f190a200640 1 -- 172.21.15.39:0/3294739758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19040d2030 msgr2=0x7f19040b6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:21.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.706+0000 7f190a200640 1 --2- 172.21.15.39:0/3294739758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19040d2030 0x7f19040b6770 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00c730 tx=0x7f18fc00cbf0 comp rx=0 tx=0).stop 2024-09-21T07:45:21.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.706+0000 7f190a200640 1 -- 172.21.15.39:0/3294739758 shutdown_connections 2024-09-21T07:45:21.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.706+0000 7f190a200640 1 --2- 172.21.15.39:0/3294739758 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f18e8078bc0 0x7f18e807b0d0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:21.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.706+0000 7f190a200640 1 --2- 172.21.15.39:0/3294739758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19040d2030 0x7f19040b6770 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:21.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.706+0000 7f190a200640 1 --2- 172.21.15.39:0/3294739758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f19040d0060 0x7f19040b6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:21.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.706+0000 7f190a200640 1 --2- 172.21.15.39:0/3294739758 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f19040ce690 0x7f190415b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:21.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.706+0000 7f190a200640 1 -- 172.21.15.39:0/3294739758 >> 172.21.15.39:0/3294739758 conn(0x7f190409b5f0 msgr2=0x7f1904071fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:21.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.706+0000 7f190a200640 1 -- 172.21.15.39:0/3294739758 shutdown_connections 2024-09-21T07:45:21.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.706+0000 7f190a200640 1 -- 172.21.15.39:0/3294739758 wait complete. 2024-09-21T07:45:21.729 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T07:45:21.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.955+0000 7f1881c00640 1 -- 172.21.15.39:0/201859289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f187c0cf450 msgr2=0x7f187c0cf860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:21.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.955+0000 7f1881c00640 1 --2- 172.21.15.39:0/201859289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f187c0cf450 0x7f187c0cf860 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f187c0cece0 tx=0x7f186402a4a0 comp rx=0 tx=0).stop 2024-09-21T07:45:21.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.955+0000 7f1881c00640 1 -- 172.21.15.39:0/201859289 shutdown_connections 2024-09-21T07:45:21.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.955+0000 7f1881c00640 1 --2- 172.21.15.39:0/201859289 >> v2:172.21.15.148:3300/0 conn(0x7f187c0d0630 0x7f187c0d0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:21.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.955+0000 7f1881c00640 1 --2- 172.21.15.39:0/201859289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f187c0cf450 0x7f187c0cf860 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:21.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.955+0000 7f1881c00640 1 -- 172.21.15.39:0/201859289 >> 172.21.15.39:0/201859289 conn(0x7f187c06cfc0 msgr2=0x7f187c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:21.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.956+0000 7f1881c00640 1 -- 172.21.15.39:0/201859289 shutdown_connections 2024-09-21T07:45:21.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.956+0000 7f1881c00640 1 -- 172.21.15.39:0/201859289 wait complete. 2024-09-21T07:45:21.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.958+0000 7f1881c00640 1 Processor -- start 2024-09-21T07:45:21.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.958+0000 7f1881c00640 1 -- start start 2024-09-21T07:45:21.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.959+0000 7f1881c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f187c073dd0 0x7f187c140f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:21.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.960+0000 7f1881c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f187c0cf450 0x7f187c151bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:21.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.960+0000 7f1881c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f187c0d0630 0x7f187c1f6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:21.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.960+0000 7f1881c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f187c0cd1a0 con 0x7f187c0d0630 2024-09-21T07:45:21.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.960+0000 7f1881c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f187c0cd020 con 0x7f187c0cf450 2024-09-21T07:45:21.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.960+0000 7f1881c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f187c0cd320 con 0x7f187c073dd0 2024-09-21T07:45:21.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.960+0000 7f187b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f187c0d0630 0x7f187c1f6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:21.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.960+0000 7f187b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f187c0d0630 0x7f187c1f6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46826/0 (socket says 172.21.15.39:46826) 2024-09-21T07:45:21.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.960+0000 7f187b400640 1 -- 172.21.15.39:0/1270654661 learned_addr learned my addr 172.21.15.39:0/1270654661 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:21.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.960+0000 7f1873e00640 1 --2- 172.21.15.39:0/1270654661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f187c0cf450 0x7f187c151bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:21.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.960+0000 7f187aa00640 1 --2- 172.21.15.39:0/1270654661 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f187c073dd0 0x7f187c140f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:21.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.960+0000 7f187b400640 1 -- 172.21.15.39:0/1270654661 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f187c073dd0 msgr2=0x7f187c140f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:21.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.960+0000 7f187b400640 1 --2- 172.21.15.39:0/1270654661 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f187c073dd0 0x7f187c140f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:21.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.960+0000 7f187b400640 1 -- 172.21.15.39:0/1270654661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f187c0cf450 msgr2=0x7f187c151bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:21.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.960+0000 7f187b400640 1 --2- 172.21.15.39:0/1270654661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f187c0cf450 0x7f187c151bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:21.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.961+0000 7f187b400640 1 -- 172.21.15.39:0/1270654661 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f187c1574d0 con 0x7f187c0d0630 2024-09-21T07:45:21.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.961+0000 7f187b400640 1 --2- 172.21.15.39:0/1270654661 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f187c0d0630 0x7f187c1f6d80 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f187400ea70 tx=0x7f187400ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:21.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.961+0000 7f1872a00640 1 -- 172.21.15.39:0/1270654661 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187400cde0 con 0x7f187c0d0630 2024-09-21T07:45:21.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.961+0000 7f1872a00640 1 -- 172.21.15.39:0/1270654661 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1874004c10 con 0x7f187c0d0630 2024-09-21T07:45:21.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.961+0000 7f1881c00640 1 -- 172.21.15.39:0/1270654661 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f187c1577b0 con 0x7f187c0d0630 2024-09-21T07:45:21.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.961+0000 7f1872a00640 1 -- 172.21.15.39:0/1270654661 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1874005020 con 0x7f187c0d0630 2024-09-21T07:45:21.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.961+0000 7f1881c00640 1 -- 172.21.15.39:0/1270654661 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f187c157d70 con 0x7f187c0d0630 2024-09-21T07:45:21.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.962+0000 7f1881c00640 1 -- 172.21.15.39:0/1270654661 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f187c0cd840 con 0x7f187c0d0630 2024-09-21T07:45:21.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.964+0000 7f1872a00640 1 -- 172.21.15.39:0/1270654661 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f18740051c0 con 0x7f187c0d0630 2024-09-21T07:45:21.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.965+0000 7f1872a00640 1 --2- 172.21.15.39:0/1270654661 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1858078b90 0x7f185807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:21.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.965+0000 7f1872a00640 1 -- 172.21.15.39:0/1270654661 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 6888+0+0 (secure 0 0 0) 0x7f187409be40 con 0x7f187c0d0630 2024-09-21T07:45:21.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.966+0000 7f187aa00640 1 --2- 172.21.15.39:0/1270654661 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1858078b90 0x7f185807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:21.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.967+0000 7f187aa00640 1 --2- 172.21.15.39:0/1270654661 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1858078b90 0x7f185807b0a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f186c007d60 tx=0x7f186c007cf0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:21.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:21.970+0000 7f1872a00640 1 -- 172.21.15.39:0/1270654661 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1874063820 con 0x7f187c0d0630 2024-09-21T07:45:22.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.211+0000 7f1881c00640 1 -- 172.21.15.39:0/1270654661 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f187c0b5880 con 0x7f187c0d0630 2024-09-21T07:45:22.213 INFO:tasks.ceph.mon.a.smithi039.stderr:2024-09-21T07:45:22.211+0000 7fabab600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T07:45:22.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.214+0000 7f1872a00640 1 -- 172.21.15.39:0/1270654661 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v173) ==== 161+0+0 (secure 0 0 0) 0x7f1874068250 con 0x7f187c0d0630 2024-09-21T07:45:22.215 INFO:teuthology.orchestra.run.smithi039.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T07:45:22.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.217+0000 7f1881c00640 1 -- 172.21.15.39:0/1270654661 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1858078b90 msgr2=0x7f185807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:22.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.217+0000 7f1881c00640 1 --2- 172.21.15.39:0/1270654661 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1858078b90 0x7f185807b0a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f186c007d60 tx=0x7f186c007cf0 comp rx=0 tx=0).stop 2024-09-21T07:45:22.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.217+0000 7f1881c00640 1 -- 172.21.15.39:0/1270654661 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f187c0d0630 msgr2=0x7f187c1f6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:22.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.217+0000 7f1881c00640 1 --2- 172.21.15.39:0/1270654661 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f187c0d0630 0x7f187c1f6d80 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f187400ea70 tx=0x7f187400ef30 comp rx=0 tx=0).stop 2024-09-21T07:45:22.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.217+0000 7f1881c00640 1 -- 172.21.15.39:0/1270654661 shutdown_connections 2024-09-21T07:45:22.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.217+0000 7f1881c00640 1 --2- 172.21.15.39:0/1270654661 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1858078b90 0x7f185807b0a0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:22.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.217+0000 7f1881c00640 1 --2- 172.21.15.39:0/1270654661 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f187c0d0630 0x7f187c1f6d80 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:22.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.217+0000 7f1881c00640 1 --2- 172.21.15.39:0/1270654661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f187c0cf450 0x7f187c151bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:22.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.217+0000 7f1881c00640 1 --2- 172.21.15.39:0/1270654661 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f187c073dd0 0x7f187c140f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:22.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.217+0000 7f1881c00640 1 -- 172.21.15.39:0/1270654661 >> 172.21.15.39:0/1270654661 conn(0x7f187c06cfc0 msgr2=0x7f187c0b2430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:22.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.217+0000 7f1881c00640 1 -- 172.21.15.39:0/1270654661 shutdown_connections 2024-09-21T07:45:22.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.217+0000 7f1881c00640 1 -- 172.21.15.39:0/1270654661 wait complete. 2024-09-21T07:45:22.245 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T07:45:22.359 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-21T07:45:22.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.478+0000 7f85cbe00640 1 -- 172.21.15.39:0/930913811 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85b0004800 msgr2=0x7f85b0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:22.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.478+0000 7f85cbe00640 1 --2- 172.21.15.39:0/930913811 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85b0004800 0x7f85b0004c10 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f85b4006f90 tx=0x7f85b0005830 comp rx=0 tx=0).stop 2024-09-21T07:45:22.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.478+0000 7f85cb400640 1 -- 172.21.15.39:0/930913811 reap_dead start 2024-09-21T07:45:22.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.478+0000 7f85cbe00640 1 -- 172.21.15.39:0/930913811 shutdown_connections 2024-09-21T07:45:22.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.478+0000 7f85cbe00640 1 -- 172.21.15.39:0/930913811 >> 172.21.15.39:0/930913811 conn(0x7f85cc06cfc0 msgr2=0x7f85cc06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:22.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.478+0000 7f85cbe00640 1 -- 172.21.15.39:0/930913811 shutdown_connections 2024-09-21T07:45:22.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.478+0000 7f85cbe00640 1 -- 172.21.15.39:0/930913811 wait complete. 2024-09-21T07:45:22.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.481+0000 7f85cbe00640 1 Processor -- start 2024-09-21T07:45:22.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.481+0000 7f85cbe00640 1 -- start start 2024-09-21T07:45:22.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.482+0000 7f85cbe00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f85b0004800 0x7f85cc1f6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:22.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.482+0000 7f85cbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85cc142e40 0x7f85cc15bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:22.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.482+0000 7f85cbe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85cc1453c0 0x7f85cc1457d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:22.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.482+0000 7f85cbe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f85cc0cd9b0 con 0x7f85cc142e40 2024-09-21T07:45:22.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.482+0000 7f85cbe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f85cc0cd830 con 0x7f85cc1453c0 2024-09-21T07:45:22.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.482+0000 7f85cbe00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f85cc0cdb30 con 0x7f85b0004800 2024-09-21T07:45:22.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.482+0000 7f85c3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85cc142e40 0x7f85cc15bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:22.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.482+0000 7f85c3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85cc142e40 0x7f85cc15bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45766/0 (socket says 172.21.15.39:45766) 2024-09-21T07:45:22.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.482+0000 7f85c3e00640 1 -- 172.21.15.39:0/2488291028 learned_addr learned my addr 172.21.15.39:0/2488291028 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:22.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.483+0000 7f85caa00640 1 --2- 172.21.15.39:0/2488291028 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f85b0004800 0x7f85cc1f6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:22.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.483+0000 7f85cb400640 1 --2- 172.21.15.39:0/2488291028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85cc1453c0 0x7f85cc1457d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:22.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.483+0000 7f85c3e00640 1 -- 172.21.15.39:0/2488291028 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f85b0004800 msgr2=0x7f85cc1f6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:22.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.483+0000 7f85c3e00640 1 --2- 172.21.15.39:0/2488291028 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f85b0004800 0x7f85cc1f6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:22.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.483+0000 7f85c3e00640 1 -- 172.21.15.39:0/2488291028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85cc1453c0 msgr2=0x7f85cc1457d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:22.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.483+0000 7f85c3e00640 1 --2- 172.21.15.39:0/2488291028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85cc1453c0 0x7f85cc1457d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:22.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.483+0000 7f85c3e00640 1 -- 172.21.15.39:0/2488291028 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85cc145d10 con 0x7f85cc142e40 2024-09-21T07:45:22.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.483+0000 7f85cb400640 1 --2- 172.21.15.39:0/2488291028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85cc1453c0 0x7f85cc1457d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:45:22.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.483+0000 7f85c3e00640 1 --2- 172.21.15.39:0/2488291028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85cc142e40 0x7f85cc15bdd0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f85b402b0a0 tx=0x7f85b4001ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:22.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.483+0000 7f85c2a00640 1 -- 172.21.15.39:0/2488291028 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b4003800 con 0x7f85cc142e40 2024-09-21T07:45:22.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.484+0000 7f85c2a00640 1 -- 172.21.15.39:0/2488291028 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f85b40039a0 con 0x7f85cc142e40 2024-09-21T07:45:22.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.484+0000 7f85cbe00640 1 -- 172.21.15.39:0/2488291028 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85cc13c4e0 con 0x7f85cc142e40 2024-09-21T07:45:22.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.484+0000 7f85c2a00640 1 -- 172.21.15.39:0/2488291028 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b4003dd0 con 0x7f85cc142e40 2024-09-21T07:45:22.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.484+0000 7f85cbe00640 1 -- 172.21.15.39:0/2488291028 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85cc1f75f0 con 0x7f85cc142e40 2024-09-21T07:45:22.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.485+0000 7f85cbe00640 1 -- 172.21.15.39:0/2488291028 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85cc15d010 con 0x7f85cc142e40 2024-09-21T07:45:22.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.486+0000 7f85c2a00640 1 -- 172.21.15.39:0/2488291028 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f85b4035020 con 0x7f85cc142e40 2024-09-21T07:45:22.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.487+0000 7f85c2a00640 1 --2- 172.21.15.39:0/2488291028 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f85a8078bc0 0x7f85a807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:22.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.487+0000 7f85c2a00640 1 -- 172.21.15.39:0/2488291028 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 6930+0+0 (secure 0 0 0) 0x7f85b40bbde0 con 0x7f85cc142e40 2024-09-21T07:45:22.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.487+0000 7f85caa00640 1 --2- 172.21.15.39:0/2488291028 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f85a8078bc0 0x7f85a807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:22.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.488+0000 7f85caa00640 1 --2- 172.21.15.39:0/2488291028 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f85a8078bc0 0x7f85a807b0d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f85bc009b30 tx=0x7f85bc005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:22.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.491+0000 7f85c2a00640 1 -- 172.21.15.39:0/2488291028 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f85cc145d10 con 0x7f85cc142e40 2024-09-21T07:45:22.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:22.729+0000 7f85cbe00640 1 -- 172.21.15.39:0/2488291028 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f85cc0b1960 con 0x7f85cc142e40 2024-09-21T07:45:23.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.216+0000 7f85c2a00640 1 -- 172.21.15.39:0/2488291028 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v174) ==== 104+0+0 (secure 0 0 0) 0x7f85b4083d40 con 0x7f85cc142e40 2024-09-21T07:45:23.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.218+0000 7f85cbe00640 1 -- 172.21.15.39:0/2488291028 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f85a8078bc0 msgr2=0x7f85a807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:23.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.218+0000 7f85cbe00640 1 --2- 172.21.15.39:0/2488291028 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f85a8078bc0 0x7f85a807b0d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f85bc009b30 tx=0x7f85bc005f90 comp rx=0 tx=0).stop 2024-09-21T07:45:23.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.218+0000 7f85cbe00640 1 -- 172.21.15.39:0/2488291028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85cc142e40 msgr2=0x7f85cc15bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:23.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.218+0000 7f85cbe00640 1 --2- 172.21.15.39:0/2488291028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85cc142e40 0x7f85cc15bdd0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f85b402b0a0 tx=0x7f85b4001ca0 comp rx=0 tx=0).stop 2024-09-21T07:45:23.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.218+0000 7f85cbe00640 1 -- 172.21.15.39:0/2488291028 shutdown_connections 2024-09-21T07:45:23.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.218+0000 7f85cbe00640 1 --2- 172.21.15.39:0/2488291028 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f85a8078bc0 0x7f85a807b0d0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:23.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.218+0000 7f85cbe00640 1 --2- 172.21.15.39:0/2488291028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85cc1453c0 0x7f85cc1457d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:23.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.218+0000 7f85cbe00640 1 --2- 172.21.15.39:0/2488291028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85cc142e40 0x7f85cc15bdd0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:23.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.218+0000 7f85cbe00640 1 --2- 172.21.15.39:0/2488291028 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f85b0004800 0x7f85cc1f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:23.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.218+0000 7f85cbe00640 1 -- 172.21.15.39:0/2488291028 >> 172.21.15.39:0/2488291028 conn(0x7f85cc06cfc0 msgr2=0x7f85cc1e4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:23.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.219+0000 7f85cbe00640 1 -- 172.21.15.39:0/2488291028 shutdown_connections 2024-09-21T07:45:23.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.219+0000 7f85cbe00640 1 -- 172.21.15.39:0/2488291028 wait complete. 2024-09-21T07:45:23.245 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T07:45:23.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.474+0000 7f785ce00640 1 -- 172.21.15.39:0/2216892258 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78580cef80 msgr2=0x7f7858070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:23.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.474+0000 7f785ce00640 1 --2- 172.21.15.39:0/2216892258 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78580cef80 0x7f7858070350 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7848009830 tx=0x7f784802a480 comp rx=0 tx=0).stop 2024-09-21T07:45:23.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.474+0000 7f785ce00640 1 -- 172.21.15.39:0/2216892258 shutdown_connections 2024-09-21T07:45:23.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.474+0000 7f785ce00640 1 --2- 172.21.15.39:0/2216892258 >> v2:172.21.15.148:3300/0 conn(0x7f78580cd230 0x7f78580ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:23.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.474+0000 7f785ce00640 1 --2- 172.21.15.39:0/2216892258 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78580cef80 0x7f7858070350 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:23.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.474+0000 7f785ce00640 1 -- 172.21.15.39:0/2216892258 >> 172.21.15.39:0/2216892258 conn(0x7f78580b1e60 msgr2=0x7f78580b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:23.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.474+0000 7f785ce00640 1 -- 172.21.15.39:0/2216892258 shutdown_connections 2024-09-21T07:45:23.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.474+0000 7f785ce00640 1 -- 172.21.15.39:0/2216892258 wait complete. 2024-09-21T07:45:23.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.477+0000 7f785ce00640 1 Processor -- start 2024-09-21T07:45:23.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.477+0000 7f785ce00640 1 -- start start 2024-09-21T07:45:23.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.478+0000 7f785ce00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7858072e80 0x7f78580d0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:23.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.478+0000 7f785ce00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78580cef80 0x7f78580d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:23.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.478+0000 7f785ce00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f785809b760 0x7f785809bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:23.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.478+0000 7f785ce00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f78580ce710 con 0x7f785809b760 2024-09-21T07:45:23.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.478+0000 7f785ce00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f78580ce590 con 0x7f7858072e80 2024-09-21T07:45:23.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.478+0000 7f785ce00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f78580ce890 con 0x7f78580cef80 2024-09-21T07:45:23.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.478+0000 7f7857e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f785809b760 0x7f785809bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:23.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.479+0000 7f7857e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f785809b760 0x7f785809bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45772/0 (socket says 172.21.15.39:45772) 2024-09-21T07:45:23.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.479+0000 7f7857e00640 1 -- 172.21.15.39:0/3363187430 learned_addr learned my addr 172.21.15.39:0/3363187430 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:23.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.479+0000 7f7857400640 1 --2- 172.21.15.39:0/3363187430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7858072e80 0x7f78580d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:23.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.479+0000 7f784ea00640 1 --2- 172.21.15.39:0/3363187430 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78580cef80 0x7f78580d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:23.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.479+0000 7f7857e00640 1 -- 172.21.15.39:0/3363187430 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78580cef80 msgr2=0x7f78580d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:23.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.479+0000 7f7857e00640 1 --2- 172.21.15.39:0/3363187430 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78580cef80 0x7f78580d05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:23.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.479+0000 7f7857e00640 1 -- 172.21.15.39:0/3363187430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7858072e80 msgr2=0x7f78580d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:23.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.479+0000 7f7857e00640 1 --2- 172.21.15.39:0/3363187430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7858072e80 0x7f78580d0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:23.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.479+0000 7f7857e00640 1 -- 172.21.15.39:0/3363187430 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78580c3de0 con 0x7f785809b760 2024-09-21T07:45:23.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.479+0000 7f784ea00640 1 --2- 172.21.15.39:0/3363187430 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78580cef80 0x7f78580d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:45:23.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.479+0000 7f7857e00640 1 --2- 172.21.15.39:0/3363187430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f785809b760 0x7f785809bb70 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f785000ea70 tx=0x7f785000ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:23.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.480+0000 7f784fe00640 1 -- 172.21.15.39:0/3363187430 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785000ce40 con 0x7f785809b760 2024-09-21T07:45:23.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.480+0000 7f784fe00640 1 -- 172.21.15.39:0/3363187430 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7850004c10 con 0x7f785809b760 2024-09-21T07:45:23.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.480+0000 7f784fe00640 1 -- 172.21.15.39:0/3363187430 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850005020 con 0x7f785809b760 2024-09-21T07:45:23.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.480+0000 7f785ce00640 1 -- 172.21.15.39:0/3363187430 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78580c4360 con 0x7f785809b760 2024-09-21T07:45:23.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.480+0000 7f785ce00640 1 -- 172.21.15.39:0/3363187430 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78580c4080 con 0x7f785809b760 2024-09-21T07:45:23.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.481+0000 7f785ce00640 1 -- 172.21.15.39:0/3363187430 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78580ce050 con 0x7f785809b760 2024-09-21T07:45:23.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.482+0000 7f784fe00640 1 -- 172.21.15.39:0/3363187430 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f78500051c0 con 0x7f785809b760 2024-09-21T07:45:23.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.484+0000 7f784fe00640 1 --2- 172.21.15.39:0/3363187430 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7834078b90 0x7f783407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:23.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.484+0000 7f784fe00640 1 -- 172.21.15.39:0/3363187430 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 6930+0+0 (secure 0 0 0) 0x7f785009c0a0 con 0x7f785809b760 2024-09-21T07:45:23.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.484+0000 7f7857400640 1 --2- 172.21.15.39:0/3363187430 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7834078b90 0x7f783407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:23.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.485+0000 7f7857400640 1 --2- 172.21.15.39:0/3363187430 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7834078b90 0x7f783407b0a0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f7848034000 tx=0x7f7848004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:23.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.488+0000 7f784fe00640 1 -- 172.21.15.39:0/3363187430 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7850063f70 con 0x7f785809b760 2024-09-21T07:45:23.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:23.684+0000 7f785ce00640 1 -- 172.21.15.39:0/3363187430 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7858161bd0 con 0x7f785809b760 2024-09-21T07:45:24.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.543+0000 7f784fe00640 1 -- 172.21.15.39:0/3363187430 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v148) ==== 181+0+0 (secure 0 0 0) 0x7f7850063870 con 0x7f785809b760 2024-09-21T07:45:24.545 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' removed 2024-09-21T07:45:24.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.547+0000 7f785ce00640 1 -- 172.21.15.39:0/3363187430 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7834078b90 msgr2=0x7f783407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:24.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.547+0000 7f785ce00640 1 --2- 172.21.15.39:0/3363187430 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7834078b90 0x7f783407b0a0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f7848034000 tx=0x7f7848004730 comp rx=0 tx=0).stop 2024-09-21T07:45:24.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.547+0000 7f785ce00640 1 -- 172.21.15.39:0/3363187430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f785809b760 msgr2=0x7f785809bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:24.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.547+0000 7f785ce00640 1 --2- 172.21.15.39:0/3363187430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f785809b760 0x7f785809bb70 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f785000ea70 tx=0x7f785000ef30 comp rx=0 tx=0).stop 2024-09-21T07:45:24.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.548+0000 7f785ce00640 1 -- 172.21.15.39:0/3363187430 shutdown_connections 2024-09-21T07:45:24.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.548+0000 7f785ce00640 1 --2- 172.21.15.39:0/3363187430 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7834078b90 0x7f783407b0a0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:24.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.548+0000 7f785ce00640 1 --2- 172.21.15.39:0/3363187430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f785809b760 0x7f785809bb70 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:24.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.548+0000 7f785ce00640 1 --2- 172.21.15.39:0/3363187430 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78580cef80 0x7f78580d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:24.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.548+0000 7f785ce00640 1 --2- 172.21.15.39:0/3363187430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7858072e80 0x7f78580d0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:24.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.548+0000 7f785ce00640 1 -- 172.21.15.39:0/3363187430 >> 172.21.15.39:0/3363187430 conn(0x7f78580b1e60 msgr2=0x7f78580b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:24.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.548+0000 7f785ce00640 1 -- 172.21.15.39:0/3363187430 shutdown_connections 2024-09-21T07:45:24.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.548+0000 7f785ce00640 1 -- 172.21.15.39:0/3363187430 wait complete. 2024-09-21T07:45:24.581 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T07:45:24.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.813+0000 7f60f8c00640 1 -- 172.21.15.39:0/843700004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60f415cc30 msgr2=0x7f60f415d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:24.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.813+0000 7f60f8c00640 1 --2- 172.21.15.39:0/843700004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60f415cc30 0x7f60f415d040 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f60f40cb1e0 tx=0x7f60ec02aa30 comp rx=0 tx=0).stop 2024-09-21T07:45:24.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.813+0000 7f60f8c00640 1 -- 172.21.15.39:0/843700004 shutdown_connections 2024-09-21T07:45:24.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.813+0000 7f60f8c00640 1 --2- 172.21.15.39:0/843700004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60f415cc30 0x7f60f415d040 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:24.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.813+0000 7f60f8c00640 1 --2- 172.21.15.39:0/843700004 >> v2:172.21.15.148:3300/0 conn(0x7f60f406f560 0x7f60f406f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:24.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.813+0000 7f60f8c00640 1 --2- 172.21.15.39:0/843700004 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f60f4070bc0 0x7f60f406f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:24.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.813+0000 7f60f8c00640 1 -- 172.21.15.39:0/843700004 >> 172.21.15.39:0/843700004 conn(0x7f60f40b2640 msgr2=0x7f60f40d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:24.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.813+0000 7f60f8c00640 1 -- 172.21.15.39:0/843700004 shutdown_connections 2024-09-21T07:45:24.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.813+0000 7f60f8c00640 1 -- 172.21.15.39:0/843700004 wait complete. 2024-09-21T07:45:24.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.815+0000 7f60f8c00640 1 Processor -- start 2024-09-21T07:45:24.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.815+0000 7f60f8c00640 1 -- start start 2024-09-21T07:45:24.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.817+0000 7f60f8c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60f406f560 0x7f60f40cf220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:24.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.817+0000 7f60f8c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f60f4070bc0 0x7f60f40c6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:24.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.817+0000 7f60f8c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60f415cc30 0x7f60f40c6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:24.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.817+0000 7f60f8c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f60f415d700 con 0x7f60f415cc30 2024-09-21T07:45:24.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.817+0000 7f60f8c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f60f415d580 con 0x7f60f406f560 2024-09-21T07:45:24.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.817+0000 7f60f8c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f60f415d880 con 0x7f60f4070bc0 2024-09-21T07:45:24.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.817+0000 7f60f2800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60f415cc30 0x7f60f40c6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:24.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.817+0000 7f60f1e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60f406f560 0x7f60f40cf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:24.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.817+0000 7f60f2800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60f415cc30 0x7f60f40c6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45790/0 (socket says 172.21.15.39:45790) 2024-09-21T07:45:24.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.817+0000 7f60f2800640 1 -- 172.21.15.39:0/3918596937 learned_addr learned my addr 172.21.15.39:0/3918596937 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:24.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.817+0000 7f60e9400640 1 --2- 172.21.15.39:0/3918596937 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f60f4070bc0 0x7f60f40c6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:24.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.818+0000 7f60f2800640 1 -- 172.21.15.39:0/3918596937 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f60f4070bc0 msgr2=0x7f60f40c6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:24.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.818+0000 7f60f2800640 1 --2- 172.21.15.39:0/3918596937 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f60f4070bc0 0x7f60f40c6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:24.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.818+0000 7f60f2800640 1 -- 172.21.15.39:0/3918596937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60f406f560 msgr2=0x7f60f40cf220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:24.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.818+0000 7f60f2800640 1 --2- 172.21.15.39:0/3918596937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60f406f560 0x7f60f40cf220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:24.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.818+0000 7f60f2800640 1 -- 172.21.15.39:0/3918596937 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f41519d0 con 0x7f60f415cc30 2024-09-21T07:45:24.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.818+0000 7f60f2800640 1 --2- 172.21.15.39:0/3918596937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60f415cc30 0x7f60f40c6980 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f60ec009d60 tx=0x7f60ec003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:24.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.818+0000 7f60eaa00640 1 -- 172.21.15.39:0/3918596937 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec033050 con 0x7f60f415cc30 2024-09-21T07:45:24.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.818+0000 7f60eaa00640 1 -- 172.21.15.39:0/3918596937 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60ec004450 con 0x7f60f415cc30 2024-09-21T07:45:24.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.818+0000 7f60eaa00640 1 -- 172.21.15.39:0/3918596937 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec038450 con 0x7f60f415cc30 2024-09-21T07:45:24.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.818+0000 7f60f8c00640 1 -- 172.21.15.39:0/3918596937 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60f4151cb0 con 0x7f60f415cc30 2024-09-21T07:45:24.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.819+0000 7f60f8c00640 1 -- 172.21.15.39:0/3918596937 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60f40c7830 con 0x7f60f415cc30 2024-09-21T07:45:24.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.819+0000 7f60f8c00640 1 -- 172.21.15.39:0/3918596937 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60f40c7340 con 0x7f60f415cc30 2024-09-21T07:45:24.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.821+0000 7f60eaa00640 1 -- 172.21.15.39:0/3918596937 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f60ec036070 con 0x7f60f415cc30 2024-09-21T07:45:24.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.822+0000 7f60eaa00640 1 --2- 172.21.15.39:0/3918596937 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f60d0078bc0 0x7f60d007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:24.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.822+0000 7f60eaa00640 1 -- 172.21.15.39:0/3918596937 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 6492+0+0 (secure 0 0 0) 0x7f60ec0bc310 con 0x7f60f415cc30 2024-09-21T07:45:24.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.822+0000 7f60f1e00640 1 --2- 172.21.15.39:0/3918596937 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f60d0078bc0 0x7f60d007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:24.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.823+0000 7f60f1e00640 1 --2- 172.21.15.39:0/3918596937 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f60d0078bc0 0x7f60d007b0d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f60e4009b30 tx=0x7f60e4005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:24.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:24.826+0000 7f60eaa00640 1 -- 172.21.15.39:0/3918596937 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f60ec083df0 con 0x7f60f415cc30 2024-09-21T07:45:25.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.024+0000 7f60f8c00640 1 -- 172.21.15.39:0/3918596937 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f60f406f020 con 0x7f60f415cc30 2024-09-21T07:45:25.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.545+0000 7f60eaa00640 1 -- 172.21.15.39:0/3918596937 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v149) ==== 169+0+0 (secure 0 0 0) 0x7f60ec088820 con 0x7f60f415cc30 2024-09-21T07:45:25.547 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' removed 2024-09-21T07:45:25.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.548+0000 7f60f8c00640 1 -- 172.21.15.39:0/3918596937 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f60d0078bc0 msgr2=0x7f60d007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:25.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.548+0000 7f60f8c00640 1 --2- 172.21.15.39:0/3918596937 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f60d0078bc0 0x7f60d007b0d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f60e4009b30 tx=0x7f60e4005f90 comp rx=0 tx=0).stop 2024-09-21T07:45:25.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.548+0000 7f60f8c00640 1 -- 172.21.15.39:0/3918596937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60f415cc30 msgr2=0x7f60f40c6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:25.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.548+0000 7f60f8c00640 1 --2- 172.21.15.39:0/3918596937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60f415cc30 0x7f60f40c6980 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f60ec009d60 tx=0x7f60ec003f10 comp rx=0 tx=0).stop 2024-09-21T07:45:25.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.548+0000 7f60f8c00640 1 -- 172.21.15.39:0/3918596937 shutdown_connections 2024-09-21T07:45:25.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.548+0000 7f60f8c00640 1 --2- 172.21.15.39:0/3918596937 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f60d0078bc0 0x7f60d007b0d0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:25.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.548+0000 7f60f8c00640 1 --2- 172.21.15.39:0/3918596937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60f415cc30 0x7f60f40c6980 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:25.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.548+0000 7f60f8c00640 1 --2- 172.21.15.39:0/3918596937 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f60f4070bc0 0x7f60f40c6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:25.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.548+0000 7f60f8c00640 1 --2- 172.21.15.39:0/3918596937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60f406f560 0x7f60f40cf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:25.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.548+0000 7f60f8c00640 1 -- 172.21.15.39:0/3918596937 >> 172.21.15.39:0/3918596937 conn(0x7f60f40b2640 msgr2=0x7f60f4070240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:25.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.549+0000 7f60f8c00640 1 -- 172.21.15.39:0/3918596937 shutdown_connections 2024-09-21T07:45:25.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.549+0000 7f60f8c00640 1 -- 172.21.15.39:0/3918596937 wait complete. 2024-09-21T07:45:25.583 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T07:45:25.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.809+0000 7f8177e00640 1 -- 172.21.15.39:0/4140793029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f815c003a00 msgr2=0x7f815c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:25.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.809+0000 7f8177e00640 1 --2- 172.21.15.39:0/4140793029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f815c003a00 0x7f815c003e70 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f8168009b40 tx=0x7f815c005830 comp rx=0 tx=0).stop 2024-09-21T07:45:25.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.809+0000 7f8177400640 1 -- 172.21.15.39:0/4140793029 reap_dead start 2024-09-21T07:45:25.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.809+0000 7f8177e00640 1 -- 172.21.15.39:0/4140793029 shutdown_connections 2024-09-21T07:45:25.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.809+0000 7f8177e00640 1 -- 172.21.15.39:0/4140793029 >> 172.21.15.39:0/4140793029 conn(0x7f817809d7d0 msgr2=0x7f81780d4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:25.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.809+0000 7f8177e00640 1 -- 172.21.15.39:0/4140793029 shutdown_connections 2024-09-21T07:45:25.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.809+0000 7f8177e00640 1 -- 172.21.15.39:0/4140793029 wait complete. 2024-09-21T07:45:25.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.811+0000 7f8177e00640 1 Processor -- start 2024-09-21T07:45:25.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.812+0000 7f8177e00640 1 -- start start 2024-09-21T07:45:25.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.813+0000 7f8177e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f815c003a00 0x7f81780ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:25.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.813+0000 7f8177e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8178158760 0x7f81781508c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:25.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.813+0000 7f8177e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f817815eeb0 0x7f817815f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:25.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.813+0000 7f8177e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f81780b3fb0 con 0x7f815c003a00 2024-09-21T07:45:25.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.813+0000 7f8177e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f81780b3e30 con 0x7f817815eeb0 2024-09-21T07:45:25.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.813+0000 7f8177e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f81780b4130 con 0x7f8178158760 2024-09-21T07:45:25.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.813+0000 7f8176a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f815c003a00 0x7f81780ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:25.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.813+0000 7f8176a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f815c003a00 0x7f81780ce2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45806/0 (socket says 172.21.15.39:45806) 2024-09-21T07:45:25.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.813+0000 7f8176a00640 1 -- 172.21.15.39:0/3192047051 learned_addr learned my addr 172.21.15.39:0/3192047051 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:25.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.814+0000 7f8177400640 1 --2- 172.21.15.39:0/3192047051 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f817815eeb0 0x7f817815f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:25.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.814+0000 7f816e000640 1 --2- 172.21.15.39:0/3192047051 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8178158760 0x7f81781508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:25.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.814+0000 7f8176a00640 1 -- 172.21.15.39:0/3192047051 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8178158760 msgr2=0x7f81781508c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:25.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.814+0000 7f8176a00640 1 --2- 172.21.15.39:0/3192047051 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8178158760 0x7f81781508c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:25.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.814+0000 7f8176a00640 1 -- 172.21.15.39:0/3192047051 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f817815eeb0 msgr2=0x7f817815f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:25.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.814+0000 7f8176a00640 1 --2- 172.21.15.39:0/3192047051 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f817815eeb0 0x7f817815f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:25.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.814+0000 7f8176a00640 1 -- 172.21.15.39:0/3192047051 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8178159000 con 0x7f815c003a00 2024-09-21T07:45:25.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.814+0000 7f816e000640 1 --2- 172.21.15.39:0/3192047051 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8178158760 0x7f81781508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:45:25.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.814+0000 7f8177400640 1 --2- 172.21.15.39:0/3192047051 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f817815eeb0 0x7f817815f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:45:25.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.814+0000 7f8176a00640 1 --2- 172.21.15.39:0/3192047051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f815c003a00 0x7f81780ce2e0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f8168009a60 tx=0x7f8168033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:25.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.814+0000 7f816fe00640 1 -- 172.21.15.39:0/3192047051 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8168043070 con 0x7f815c003a00 2024-09-21T07:45:25.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.814+0000 7f816fe00640 1 -- 172.21.15.39:0/3192047051 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8168004120 con 0x7f815c003a00 2024-09-21T07:45:25.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.814+0000 7f816fe00640 1 -- 172.21.15.39:0/3192047051 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8168038040 con 0x7f815c003a00 2024-09-21T07:45:25.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.814+0000 7f8177e00640 1 -- 172.21.15.39:0/3192047051 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f817815f800 con 0x7f815c003a00 2024-09-21T07:45:25.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.815+0000 7f8177e00640 1 -- 172.21.15.39:0/3192047051 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81780cc770 con 0x7f815c003a00 2024-09-21T07:45:25.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.816+0000 7f8177e00640 1 -- 172.21.15.39:0/3192047051 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81780b96a0 con 0x7f815c003a00 2024-09-21T07:45:25.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.817+0000 7f816fe00640 1 -- 172.21.15.39:0/3192047051 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8168033860 con 0x7f815c003a00 2024-09-21T07:45:25.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.818+0000 7f816fe00640 1 --2- 172.21.15.39:0/3192047051 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8154078bc0 0x7f815407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:25.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.818+0000 7f816fe00640 1 -- 172.21.15.39:0/3192047051 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6078+0+0 (secure 0 0 0) 0x7f81680bb420 con 0x7f815c003a00 2024-09-21T07:45:25.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.818+0000 7f816e000640 1 --2- 172.21.15.39:0/3192047051 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8154078bc0 0x7f815407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:25.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.819+0000 7f816e000640 1 --2- 172.21.15.39:0/3192047051 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8154078bc0 0x7f815407b0d0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f8178158bd0 tx=0x7f8160005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:25.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:25.822+0000 7f816fe00640 1 -- 172.21.15.39:0/3192047051 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f817815f800 con 0x7f815c003a00 2024-09-21T07:45:26.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.018+0000 7f8177e00640 1 -- 172.21.15.39:0/3192047051 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f81780cf0f0 con 0x7f815c003a00 2024-09-21T07:45:26.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.023+0000 7f816fe00640 1 -- 172.21.15.39:0/3192047051 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v44) ==== 71+0+0 (secure 0 0 0) 0x7f8168083130 con 0x7f815c003a00 2024-09-21T07:45:26.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.025+0000 7f8177e00640 1 -- 172.21.15.39:0/3192047051 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8154078bc0 msgr2=0x7f815407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:26.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.025+0000 7f8177e00640 1 --2- 172.21.15.39:0/3192047051 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8154078bc0 0x7f815407b0d0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f8178158bd0 tx=0x7f8160005f90 comp rx=0 tx=0).stop 2024-09-21T07:45:26.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.025+0000 7f8177e00640 1 -- 172.21.15.39:0/3192047051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f815c003a00 msgr2=0x7f81780ce2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:26.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.025+0000 7f8177e00640 1 --2- 172.21.15.39:0/3192047051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f815c003a00 0x7f81780ce2e0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f8168009a60 tx=0x7f8168033670 comp rx=0 tx=0).stop 2024-09-21T07:45:26.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.025+0000 7f8177e00640 1 -- 172.21.15.39:0/3192047051 shutdown_connections 2024-09-21T07:45:26.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.025+0000 7f8177e00640 1 --2- 172.21.15.39:0/3192047051 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8154078bc0 0x7f815407b0d0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:26.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.025+0000 7f8177e00640 1 --2- 172.21.15.39:0/3192047051 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f817815eeb0 0x7f817815f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:26.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.025+0000 7f8177e00640 1 --2- 172.21.15.39:0/3192047051 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8178158760 0x7f81781508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:26.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.026+0000 7f8177e00640 1 --2- 172.21.15.39:0/3192047051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f815c003a00 0x7f81780ce2e0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:26.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.026+0000 7f8177e00640 1 -- 172.21.15.39:0/3192047051 >> 172.21.15.39:0/3192047051 conn(0x7f817809d7d0 msgr2=0x7f8178068f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:26.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.026+0000 7f8177e00640 1 -- 172.21.15.39:0/3192047051 shutdown_connections 2024-09-21T07:45:26.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.026+0000 7f8177e00640 1 -- 172.21.15.39:0/3192047051 wait complete. 2024-09-21T07:45:26.051 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_mds_max_snaps_per_dir_default_limit' 2024-09-21T07:45:26.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.277+0000 7f15b9e00640 1 -- 172.21.15.39:0/4121728643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1598004800 msgr2=0x7f1598004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:26.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.277+0000 7f15b9e00640 1 --2- 172.21.15.39:0/4121728643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1598004800 0x7f1598004c10 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f159c009b50 tx=0x7f1598005830 comp rx=0 tx=0).stop 2024-09-21T07:45:26.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.278+0000 7f15b3400640 1 -- 172.21.15.39:0/4121728643 reap_dead start 2024-09-21T07:45:26.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.278+0000 7f15b9e00640 1 -- 172.21.15.39:0/4121728643 shutdown_connections 2024-09-21T07:45:26.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.278+0000 7f15b9e00640 1 -- 172.21.15.39:0/4121728643 >> 172.21.15.39:0/4121728643 conn(0x7f15b40b2460 msgr2=0x7f15b406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:26.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.278+0000 7f15b9e00640 1 -- 172.21.15.39:0/4121728643 shutdown_connections 2024-09-21T07:45:26.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.278+0000 7f15b9e00640 1 -- 172.21.15.39:0/4121728643 wait complete. 2024-09-21T07:45:26.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.280+0000 7f15b9e00640 1 Processor -- start 2024-09-21T07:45:26.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.280+0000 7f15b9e00640 1 -- start start 2024-09-21T07:45:26.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.282+0000 7f15b9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1598004800 0x7f15b40b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:26.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.282+0000 7f15b9e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15b41f6390 0x7f15b414de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:26.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.282+0000 7f15b9e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15b414e360 0x7f15b41f8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:26.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.282+0000 7f15b9e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f15b40b40b0 con 0x7f1598004800 2024-09-21T07:45:26.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.282+0000 7f15b9e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f15b4071230 con 0x7f15b41f6390 2024-09-21T07:45:26.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.282+0000 7f15b9e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f15b40b4230 con 0x7f15b414e360 2024-09-21T07:45:26.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.282+0000 7f15b2a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1598004800 0x7f15b40b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:26.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.282+0000 7f15b2a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1598004800 0x7f15b40b13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45836/0 (socket says 172.21.15.39:45836) 2024-09-21T07:45:26.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.282+0000 7f15b2a00640 1 -- 172.21.15.39:0/488011842 learned_addr learned my addr 172.21.15.39:0/488011842 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:26.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.282+0000 7f15b2a00640 1 -- 172.21.15.39:0/488011842 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15b414e360 msgr2=0x7f15b41f8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:26.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.282+0000 7f15b2a00640 1 --2- 172.21.15.39:0/488011842 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15b414e360 0x7f15b41f8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:26.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.282+0000 7f15b2a00640 1 -- 172.21.15.39:0/488011842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15b41f6390 msgr2=0x7f15b414de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:26.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.282+0000 7f15b2a00640 1 --2- 172.21.15.39:0/488011842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15b41f6390 0x7f15b414de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:26.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.282+0000 7f15b2a00640 1 -- 172.21.15.39:0/488011842 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b414e770 con 0x7f1598004800 2024-09-21T07:45:26.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.283+0000 7f15b2a00640 1 --2- 172.21.15.39:0/488011842 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1598004800 0x7f15b40b13f0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f15a4006660 tx=0x7f15a40039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:26.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.283+0000 7f15ab400640 1 -- 172.21.15.39:0/488011842 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a4015070 con 0x7f1598004800 2024-09-21T07:45:26.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.283+0000 7f15ab400640 1 -- 172.21.15.39:0/488011842 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15a40119c0 con 0x7f1598004800 2024-09-21T07:45:26.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.283+0000 7f15ab400640 1 -- 172.21.15.39:0/488011842 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a4011e10 con 0x7f1598004800 2024-09-21T07:45:26.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.283+0000 7f15b9e00640 1 -- 172.21.15.39:0/488011842 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15b40c0030 con 0x7f1598004800 2024-09-21T07:45:26.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.283+0000 7f15b9e00640 1 -- 172.21.15.39:0/488011842 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15b40c05f0 con 0x7f1598004800 2024-09-21T07:45:26.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.284+0000 7f15b9e00640 1 -- 172.21.15.39:0/488011842 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15b40cf450 con 0x7f1598004800 2024-09-21T07:45:26.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.285+0000 7f15ab400640 1 -- 172.21.15.39:0/488011842 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f15a4018070 con 0x7f1598004800 2024-09-21T07:45:26.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.286+0000 7f15ab400640 1 --2- 172.21.15.39:0/488011842 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1590078bc0 0x7f159007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:26.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.287+0000 7f15ab400640 1 -- 172.21.15.39:0/488011842 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6078+0+0 (secure 0 0 0) 0x7f15a409bb80 con 0x7f1598004800 2024-09-21T07:45:26.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.287+0000 7f15abe00640 1 --2- 172.21.15.39:0/488011842 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1590078bc0 0x7f159007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:26.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.288+0000 7f15abe00640 1 --2- 172.21.15.39:0/488011842 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1590078bc0 0x7f159007b0d0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f159c009b50 tx=0x7f159c02d040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:26.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.291+0000 7f15ab400640 1 -- 172.21.15.39:0/488011842 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f15b414e770 con 0x7f1598004800 2024-09-21T07:45:26.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:26.564+0000 7f15b9e00640 1 -- 172.21.15.39:0/488011842 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_mds_max_snaps_per_dir_default_limit"]} v 0) -- 0x7f15b4167560 con 0x7f1598004800 2024-09-21T07:45:27.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.568+0000 7f15ab400640 1 -- 172.21.15.39:0/488011842 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_mds_max_snaps_per_dir_default_limit"]}]=0 v593) ==== 161+0+0 (secure 0 0 0) 0x7f15a4063ef0 con 0x7f1598004800 2024-09-21T07:45:27.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.571+0000 7f15b9e00640 1 -- 172.21.15.39:0/488011842 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1590078bc0 msgr2=0x7f159007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:27.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.571+0000 7f15b9e00640 1 --2- 172.21.15.39:0/488011842 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1590078bc0 0x7f159007b0d0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f159c009b50 tx=0x7f159c02d040 comp rx=0 tx=0).stop 2024-09-21T07:45:27.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.571+0000 7f15b9e00640 1 -- 172.21.15.39:0/488011842 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1598004800 msgr2=0x7f15b40b13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:27.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.571+0000 7f15b9e00640 1 --2- 172.21.15.39:0/488011842 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1598004800 0x7f15b40b13f0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f15a4006660 tx=0x7f15a40039c0 comp rx=0 tx=0).stop 2024-09-21T07:45:27.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.571+0000 7f15b9e00640 1 -- 172.21.15.39:0/488011842 shutdown_connections 2024-09-21T07:45:27.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.571+0000 7f15b9e00640 1 --2- 172.21.15.39:0/488011842 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1590078bc0 0x7f159007b0d0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:27.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.571+0000 7f15b9e00640 1 --2- 172.21.15.39:0/488011842 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15b414e360 0x7f15b41f8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:27.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.571+0000 7f15b9e00640 1 --2- 172.21.15.39:0/488011842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15b41f6390 0x7f15b414de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:27.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.571+0000 7f15b9e00640 1 --2- 172.21.15.39:0/488011842 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1598004800 0x7f15b40b13f0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:27.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.571+0000 7f15b9e00640 1 -- 172.21.15.39:0/488011842 >> 172.21.15.39:0/488011842 conn(0x7f15b40b2460 msgr2=0x7f15b415a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:27.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.571+0000 7f15b9e00640 1 -- 172.21.15.39:0/488011842 shutdown_connections 2024-09-21T07:45:27.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.571+0000 7f15b9e00640 1 -- 172.21.15.39:0/488011842 wait complete. 2024-09-21T07:45:27.598 INFO:tasks.cephfs_test_runner:Test the newly introudced option named mds_max_snaps_per_dir ... ok 2024-09-21T07:45:27.598 INFO:tasks.cephfs_test_runner:Starting test: test_mds_max_snaps_per_dir_with_increased_limit (tasks.cephfs.test_snapshots.TestSnapshots) Test the newly introudced option named mds_max_snaps_per_dir 2024-09-21T07:45:27.598 INFO:tasks.cephfs_test_runner:test_mds_max_snaps_per_dir_with_increased_limit (tasks.cephfs.test_snapshots.TestSnapshots) 2024-09-21T07:45:27.598 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:45:27.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.825+0000 7f9409600640 1 -- 172.21.15.39:0/378623787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9404160660 msgr2=0x7f94040cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:27.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.825+0000 7f9409600640 1 --2- 172.21.15.39:0/378623787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9404160660 0x7f94040cf710 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f940406dd50 tx=0x7f93ec02a560 comp rx=0 tx=0).stop 2024-09-21T07:45:27.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.825+0000 7f9409600640 1 -- 172.21.15.39:0/378623787 shutdown_connections 2024-09-21T07:45:27.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.825+0000 7f9409600640 1 --2- 172.21.15.39:0/378623787 >> v2:172.21.15.148:3300/0 conn(0x7f94040cfc50 0x7f94040d0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:27.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.825+0000 7f9409600640 1 --2- 172.21.15.39:0/378623787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9404160660 0x7f94040cf710 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:27.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.825+0000 7f9409600640 1 --2- 172.21.15.39:0/378623787 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9404160250 0x7f940415d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:27.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.825+0000 7f9409600640 1 -- 172.21.15.39:0/378623787 >> 172.21.15.39:0/378623787 conn(0x7f940406cfc0 msgr2=0x7f940406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:27.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.825+0000 7f9409600640 1 -- 172.21.15.39:0/378623787 shutdown_connections 2024-09-21T07:45:27.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.825+0000 7f9409600640 1 -- 172.21.15.39:0/378623787 wait complete. 2024-09-21T07:45:27.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.827+0000 7f9409600640 1 Processor -- start 2024-09-21T07:45:27.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.827+0000 7f9409600640 1 -- start start 2024-09-21T07:45:27.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.829+0000 7f9409600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94040cfc50 0x7f940415a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:27.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.829+0000 7f9409600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9404160250 0x7f94041f92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:27.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.829+0000 7f9409600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9404160660 0x7f94040b1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:27.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.829+0000 7f9409600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f94040ccbb0 con 0x7f9404160250 2024-09-21T07:45:27.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.829+0000 7f9409600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f94040cca30 con 0x7f94040cfc50 2024-09-21T07:45:27.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.829+0000 7f9409600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f94040ccd30 con 0x7f9404160660 2024-09-21T07:45:27.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.829+0000 7f93fbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9404160250 0x7f94041f92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:27.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.829+0000 7f93fbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9404160250 0x7f94041f92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45850/0 (socket says 172.21.15.39:45850) 2024-09-21T07:45:27.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.829+0000 7f93fbe00640 1 -- 172.21.15.39:0/3889593932 learned_addr learned my addr 172.21.15.39:0/3889593932 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:27.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.829+0000 7f9403e00640 1 --2- 172.21.15.39:0/3889593932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94040cfc50 0x7f940415a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:27.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.829+0000 7f9408c00640 1 --2- 172.21.15.39:0/3889593932 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9404160660 0x7f94040b1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:27.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.829+0000 7f93fbe00640 1 -- 172.21.15.39:0/3889593932 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9404160660 msgr2=0x7f94040b1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:27.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.829+0000 7f93fbe00640 1 --2- 172.21.15.39:0/3889593932 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9404160660 0x7f94040b1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:27.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.829+0000 7f93fbe00640 1 -- 172.21.15.39:0/3889593932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94040cfc50 msgr2=0x7f940415a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:27.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.829+0000 7f93fbe00640 1 --2- 172.21.15.39:0/3889593932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94040cfc50 0x7f940415a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:27.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.829+0000 7f93fbe00640 1 -- 172.21.15.39:0/3889593932 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94041f68c0 con 0x7f9404160250 2024-09-21T07:45:27.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.830+0000 7f93fbe00640 1 --2- 172.21.15.39:0/3889593932 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9404160250 0x7f94041f92a0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f93ec034040 tx=0x7f93ec003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:27.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.830+0000 7f9400c00640 1 -- 172.21.15.39:0/3889593932 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec03e030 con 0x7f9404160250 2024-09-21T07:45:27.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.830+0000 7f9400c00640 1 -- 172.21.15.39:0/3889593932 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93ec003e00 con 0x7f9404160250 2024-09-21T07:45:27.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.830+0000 7f9400c00640 1 -- 172.21.15.39:0/3889593932 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec035040 con 0x7f9404160250 2024-09-21T07:45:27.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.830+0000 7f9409600640 1 -- 172.21.15.39:0/3889593932 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94040b12a0 con 0x7f9404160250 2024-09-21T07:45:27.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.830+0000 7f9409600640 1 -- 172.21.15.39:0/3889593932 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94040b1860 con 0x7f9404160250 2024-09-21T07:45:27.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.831+0000 7f9409600640 1 -- 172.21.15.39:0/3889593932 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94041f6630 con 0x7f9404160250 2024-09-21T07:45:27.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.832+0000 7f9400c00640 1 -- 172.21.15.39:0/3889593932 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f93ec041030 con 0x7f9404160250 2024-09-21T07:45:27.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.833+0000 7f9400c00640 1 --2- 172.21.15.39:0/3889593932 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f93e0078b90 0x7f93e007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:27.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.833+0000 7f9400c00640 1 -- 172.21.15.39:0/3889593932 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6078+0+0 (secure 0 0 0) 0x7f93ec0bb760 con 0x7f9404160250 2024-09-21T07:45:27.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.834+0000 7f9403e00640 1 --2- 172.21.15.39:0/3889593932 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f93e0078b90 0x7f93e007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:27.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.834+0000 7f9403e00640 1 --2- 172.21.15.39:0/3889593932 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f93e0078b90 0x7f93e007b0a0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f93f4005fd0 tx=0x7f93f4005850 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:27.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:27.838+0000 7f9400c00640 1 -- 172.21.15.39:0/3889593932 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f93ec083a20 con 0x7f9404160250 2024-09-21T07:45:28.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.038+0000 7f9409600640 1 -- 172.21.15.39:0/3889593932 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f94040b1090 con 0x7f9404160250 2024-09-21T07:45:28.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.039+0000 7f9400c00640 1 -- 172.21.15.39:0/3889593932 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) ==== 74+0+18574 (secure 0 0 0) 0x7f93ec083320 con 0x7f9404160250 2024-09-21T07:45:28.041 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:45:28.042 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":149,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:45:25.544345+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6835/3547777254":"2024-09-22T07:45:22.212679+0000","172.21.15.39:6832/3547777254":"2024-09-22T07:45:22.212679+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:45:28.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.041+0000 7f9409600640 1 -- 172.21.15.39:0/3889593932 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f93e0078b90 msgr2=0x7f93e007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:28.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.041+0000 7f9409600640 1 --2- 172.21.15.39:0/3889593932 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f93e0078b90 0x7f93e007b0a0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f93f4005fd0 tx=0x7f93f4005850 comp rx=0 tx=0).stop 2024-09-21T07:45:28.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.041+0000 7f9409600640 1 -- 172.21.15.39:0/3889593932 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9404160250 msgr2=0x7f94041f92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:28.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.041+0000 7f9409600640 1 --2- 172.21.15.39:0/3889593932 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9404160250 0x7f94041f92a0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f93ec034040 tx=0x7f93ec003910 comp rx=0 tx=0).stop 2024-09-21T07:45:28.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.042+0000 7f9409600640 1 -- 172.21.15.39:0/3889593932 shutdown_connections 2024-09-21T07:45:28.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.042+0000 7f9409600640 1 --2- 172.21.15.39:0/3889593932 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f93e0078b90 0x7f93e007b0a0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:28.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.042+0000 7f9409600640 1 --2- 172.21.15.39:0/3889593932 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9404160660 0x7f94040b1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:28.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.042+0000 7f9409600640 1 --2- 172.21.15.39:0/3889593932 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9404160250 0x7f94041f92a0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:28.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.042+0000 7f9409600640 1 --2- 172.21.15.39:0/3889593932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94040cfc50 0x7f940415a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:28.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.042+0000 7f9409600640 1 -- 172.21.15.39:0/3889593932 >> 172.21.15.39:0/3889593932 conn(0x7f940406cfc0 msgr2=0x7f9404073f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:28.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.042+0000 7f9409600640 1 -- 172.21.15.39:0/3889593932 shutdown_connections 2024-09-21T07:45:28.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.042+0000 7f9409600640 1 -- 172.21.15.39:0/3889593932 wait complete. 2024-09-21T07:45:28.066 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T07:33:01.569433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'snap-test-pool', 'create_time': '2024-09-21T07:34:22.032185+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 1, 'snap_epoch': 37, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'test_data_pool', 'create_time': '2024-09-21T07:35:25.604532+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 1, 'snap_epoch': 66, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_metadata_pool', 'create_time': '2024-09-21T07:35:26.327607+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T07:45:28.066 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T07:45:28.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.292+0000 7f202c600640 1 -- 172.21.15.39:0/1694696479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20240af830 msgr2=0x7f202406ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:28.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.292+0000 7f202c600640 1 --2- 172.21.15.39:0/1694696479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20240af830 0x7f202406ef90 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f2010005fd0 tx=0x7f201002a5f0 comp rx=0 tx=0).stop 2024-09-21T07:45:28.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.292+0000 7f202c600640 1 -- 172.21.15.39:0/1694696479 shutdown_connections 2024-09-21T07:45:28.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.292+0000 7f202c600640 1 --2- 172.21.15.39:0/1694696479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20240af830 0x7f202406ef90 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:28.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.292+0000 7f202c600640 1 -- 172.21.15.39:0/1694696479 >> 172.21.15.39:0/1694696479 conn(0x7f202406d7a0 msgr2=0x7f20240b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:28.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.292+0000 7f202c600640 1 -- 172.21.15.39:0/1694696479 shutdown_connections 2024-09-21T07:45:28.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.292+0000 7f202c600640 1 -- 172.21.15.39:0/1694696479 wait complete. 2024-09-21T07:45:28.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.295+0000 7f202c600640 1 Processor -- start 2024-09-21T07:45:28.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.295+0000 7f202c600640 1 -- start start 2024-09-21T07:45:28.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.296+0000 7f202c600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20240706f0 0x7f202414e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:28.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.296+0000 7f202c600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20240af830 0x7f2024152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:28.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.296+0000 7f202c600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f202415d180 0x7f2024155430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:28.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.296+0000 7f202c600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f20240b3910 con 0x7f20240af830 2024-09-21T07:45:28.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.296+0000 7f202c600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f20240b3790 con 0x7f202415d180 2024-09-21T07:45:28.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.296+0000 7f202c600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f20240b3a90 con 0x7f20240706f0 2024-09-21T07:45:28.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.297+0000 7f2029200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20240af830 0x7f2024152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:28.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.297+0000 7f2029200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20240af830 0x7f2024152a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45858/0 (socket says 172.21.15.39:45858) 2024-09-21T07:45:28.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.297+0000 7f2029200640 1 -- 172.21.15.39:0/2799085548 learned_addr learned my addr 172.21.15.39:0/2799085548 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:28.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.297+0000 7f2029c00640 1 --2- 172.21.15.39:0/2799085548 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20240706f0 0x7f202414e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:28.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.297+0000 7f202a600640 1 --2- 172.21.15.39:0/2799085548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f202415d180 0x7f2024155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:28.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.297+0000 7f2029200640 1 -- 172.21.15.39:0/2799085548 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20240706f0 msgr2=0x7f202414e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:28.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.297+0000 7f2029200640 1 --2- 172.21.15.39:0/2799085548 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20240706f0 0x7f202414e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:28.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.297+0000 7f2029200640 1 -- 172.21.15.39:0/2799085548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f202415d180 msgr2=0x7f2024155430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:28.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.297+0000 7f2029200640 1 --2- 172.21.15.39:0/2799085548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f202415d180 0x7f2024155430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:28.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.297+0000 7f2029200640 1 -- 172.21.15.39:0/2799085548 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2024157330 con 0x7f20240af830 2024-09-21T07:45:28.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.297+0000 7f2029c00640 1 --2- 172.21.15.39:0/2799085548 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20240706f0 0x7f202414e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:45:28.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.297+0000 7f202a600640 1 --2- 172.21.15.39:0/2799085548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f202415d180 0x7f2024155430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:45:28.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.298+0000 7f2029200640 1 --2- 172.21.15.39:0/2799085548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20240af830 0x7f2024152a00 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f2010006010 tx=0x7f20100039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:28.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.298+0000 7f2021600640 1 -- 172.21.15.39:0/2799085548 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201003e030 con 0x7f20240af830 2024-09-21T07:45:28.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.298+0000 7f2021600640 1 -- 172.21.15.39:0/2799085548 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20100047a0 con 0x7f20240af830 2024-09-21T07:45:28.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.298+0000 7f2021600640 1 -- 172.21.15.39:0/2799085548 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2010035040 con 0x7f20240af830 2024-09-21T07:45:28.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.298+0000 7f202c600640 1 -- 172.21.15.39:0/2799085548 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2024157860 con 0x7f20240af830 2024-09-21T07:45:28.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.298+0000 7f202c600640 1 -- 172.21.15.39:0/2799085548 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2024157e20 con 0x7f20240af830 2024-09-21T07:45:28.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.299+0000 7f202c600640 1 -- 172.21.15.39:0/2799085548 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f202406ef90 con 0x7f20240af830 2024-09-21T07:45:28.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.301+0000 7f2021600640 1 -- 172.21.15.39:0/2799085548 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2010041030 con 0x7f20240af830 2024-09-21T07:45:28.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.302+0000 7f2021600640 1 --2- 172.21.15.39:0/2799085548 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2004078bc0 0x7f200407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:28.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.302+0000 7f2021600640 1 -- 172.21.15.39:0/2799085548 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6078+0+0 (secure 0 0 0) 0x7f20100bb7e0 con 0x7f20240af830 2024-09-21T07:45:28.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.302+0000 7f2029c00640 1 --2- 172.21.15.39:0/2799085548 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2004078bc0 0x7f200407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:28.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.303+0000 7f2029c00640 1 --2- 172.21.15.39:0/2799085548 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2004078bc0 0x7f200407b0d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f2018007fd0 tx=0x7f2018009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:28.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.306+0000 7f2021600640 1 -- 172.21.15.39:0/2799085548 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2010083a10 con 0x7f20240af830 2024-09-21T07:45:28.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.500+0000 7f202c600640 1 -- 172.21.15.39:0/2799085548 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f20241575f0 con 0x7f20240af830 2024-09-21T07:45:28.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.500+0000 7f2021600640 1 -- 172.21.15.39:0/2799085548 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v149) ==== 93+0+10 (secure 0 0 0) 0x7f2010083310 con 0x7f20240af830 2024-09-21T07:45:28.502 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-09-21T07:45:28.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.503+0000 7f202c600640 1 -- 172.21.15.39:0/2799085548 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2004078bc0 msgr2=0x7f200407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:28.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.503+0000 7f202c600640 1 --2- 172.21.15.39:0/2799085548 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2004078bc0 0x7f200407b0d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f2018007fd0 tx=0x7f2018009040 comp rx=0 tx=0).stop 2024-09-21T07:45:28.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.503+0000 7f202c600640 1 -- 172.21.15.39:0/2799085548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20240af830 msgr2=0x7f2024152a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:28.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.503+0000 7f202c600640 1 --2- 172.21.15.39:0/2799085548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20240af830 0x7f2024152a00 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f2010006010 tx=0x7f20100039e0 comp rx=0 tx=0).stop 2024-09-21T07:45:28.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.503+0000 7f202c600640 1 -- 172.21.15.39:0/2799085548 shutdown_connections 2024-09-21T07:45:28.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.503+0000 7f202c600640 1 --2- 172.21.15.39:0/2799085548 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2004078bc0 0x7f200407b0d0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:28.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.503+0000 7f202c600640 1 --2- 172.21.15.39:0/2799085548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f202415d180 0x7f2024155430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:28.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.503+0000 7f202c600640 1 --2- 172.21.15.39:0/2799085548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20240af830 0x7f2024152a00 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:28.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.503+0000 7f202c600640 1 --2- 172.21.15.39:0/2799085548 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20240706f0 0x7f202414e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:28.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.503+0000 7f202c600640 1 -- 172.21.15.39:0/2799085548 >> 172.21.15.39:0/2799085548 conn(0x7f202406d7a0 msgr2=0x7f202415a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:28.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.503+0000 7f202c600640 1 -- 172.21.15.39:0/2799085548 shutdown_connections 2024-09-21T07:45:28.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.503+0000 7f202c600640 1 -- 172.21.15.39:0/2799085548 wait complete. 2024-09-21T07:45:28.529 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2024-09-21T07:45:28.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.760+0000 7f5213200640 1 -- 172.21.15.39:0/1702205521 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52140d25a0 msgr2=0x7f52140d29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:28.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.760+0000 7f5213200640 1 --2- 172.21.15.39:0/1702205521 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52140d25a0 0x7f52140d29b0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f52140b5490 tx=0x7f520c02a8f0 comp rx=0 tx=0).stop 2024-09-21T07:45:28.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.760+0000 7f5213200640 1 -- 172.21.15.39:0/1702205521 shutdown_connections 2024-09-21T07:45:28.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.760+0000 7f5213200640 1 --2- 172.21.15.39:0/1702205521 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52140d25a0 0x7f52140d29b0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:28.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.760+0000 7f5213200640 1 --2- 172.21.15.39:0/1702205521 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52140b5f60 0x7f52140b6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:28.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.760+0000 7f5213200640 1 --2- 172.21.15.39:0/1702205521 >> v2:172.21.15.148:3300/0 conn(0x7f52140b6c90 0x7f52140cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:28.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.760+0000 7f5213200640 1 -- 172.21.15.39:0/1702205521 >> 172.21.15.39:0/1702205521 conn(0x7f521409d7b0 msgr2=0x7f52140d44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:28.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.760+0000 7f5213200640 1 -- 172.21.15.39:0/1702205521 shutdown_connections 2024-09-21T07:45:28.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.760+0000 7f5213200640 1 -- 172.21.15.39:0/1702205521 wait complete. 2024-09-21T07:45:28.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.762+0000 7f5213200640 1 Processor -- start 2024-09-21T07:45:28.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.763+0000 7f5213200640 1 -- start start 2024-09-21T07:45:28.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.764+0000 7f5213200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52140b5f60 0x7f52140cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:28.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.764+0000 7f5213200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52140b6c90 0x7f52141f8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:28.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.764+0000 7f5213200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52140d25a0 0x7f52140b11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:28.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.764+0000 7f5213200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f52140cfea0 con 0x7f52140b5f60 2024-09-21T07:45:28.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.764+0000 7f5213200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f52140cfd20 con 0x7f52140d25a0 2024-09-21T07:45:28.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.764+0000 7f5213200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f52140b9e60 con 0x7f52140b6c90 2024-09-21T07:45:28.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.764+0000 7f5211e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52140b5f60 0x7f52140cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:28.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.764+0000 7f5211e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52140b5f60 0x7f52140cdb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45884/0 (socket says 172.21.15.39:45884) 2024-09-21T07:45:28.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.764+0000 7f5211e00640 1 -- 172.21.15.39:0/3395850643 learned_addr learned my addr 172.21.15.39:0/3395850643 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:28.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.765+0000 7f520be00640 1 --2- 172.21.15.39:0/3395850643 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52140b6c90 0x7f52141f8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:28.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.765+0000 7f5212800640 1 --2- 172.21.15.39:0/3395850643 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52140d25a0 0x7f52140b11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:28.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.765+0000 7f5211e00640 1 -- 172.21.15.39:0/3395850643 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52140b6c90 msgr2=0x7f52141f8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:28.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.765+0000 7f5211e00640 1 --2- 172.21.15.39:0/3395850643 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52140b6c90 0x7f52141f8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:28.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.765+0000 7f5211e00640 1 -- 172.21.15.39:0/3395850643 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52140d25a0 msgr2=0x7f52140b11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:28.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.765+0000 7f5211e00640 1 --2- 172.21.15.39:0/3395850643 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52140d25a0 0x7f52140b11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:28.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.765+0000 7f5211e00640 1 -- 172.21.15.39:0/3395850643 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52141f53d0 con 0x7f52140b5f60 2024-09-21T07:45:28.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.765+0000 7f5211e00640 1 --2- 172.21.15.39:0/3395850643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52140b5f60 0x7f52140cdb20 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f5204009fd0 tx=0x7f520400ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:28.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.765+0000 7f520a000640 1 -- 172.21.15.39:0/3395850643 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520400ccc0 con 0x7f52140b5f60 2024-09-21T07:45:28.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.765+0000 7f520a000640 1 -- 172.21.15.39:0/3395850643 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f520400ee70 con 0x7f52140b5f60 2024-09-21T07:45:28.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.766+0000 7f5213200640 1 -- 172.21.15.39:0/3395850643 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52141f56b0 con 0x7f52140b5f60 2024-09-21T07:45:28.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.766+0000 7f520a000640 1 -- 172.21.15.39:0/3395850643 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520400ce60 con 0x7f52140b5f60 2024-09-21T07:45:28.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.766+0000 7f5213200640 1 -- 172.21.15.39:0/3395850643 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52141f5c70 con 0x7f52140b5f60 2024-09-21T07:45:28.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.767+0000 7f5213200640 1 -- 172.21.15.39:0/3395850643 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52140cf7e0 con 0x7f52140b5f60 2024-09-21T07:45:28.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.768+0000 7f520a000640 1 -- 172.21.15.39:0/3395850643 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5204004280 con 0x7f52140b5f60 2024-09-21T07:45:28.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.769+0000 7f520a000640 1 --2- 172.21.15.39:0/3395850643 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f51f0078b90 0x7f51f007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:28.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.769+0000 7f520a000640 1 -- 172.21.15.39:0/3395850643 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6078+0+0 (secure 0 0 0) 0x7f520409b8f0 con 0x7f52140b5f60 2024-09-21T07:45:28.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.769+0000 7f520be00640 1 --2- 172.21.15.39:0/3395850643 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f51f0078b90 0x7f51f007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:28.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.770+0000 7f520be00640 1 --2- 172.21.15.39:0/3395850643 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f51f0078b90 0x7f51f007b0a0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f52140b8e10 tx=0x7f51fc005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:28.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.773+0000 7f520a000640 1 -- 172.21.15.39:0/3395850643 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5204063b20 con 0x7f52140b5f60 2024-09-21T07:45:28.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.971+0000 7f5213200640 1 -- 172.21.15.39:0/3395850643 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) -- 0x7f52140d2ae0 con 0x7f52140b5f60 2024-09-21T07:45:28.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.972+0000 7f520a000640 1 -- 172.21.15.39:0/3395850643 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v149) ==== 103+0+11 (secure 0 0 0) 0x7f5204063420 con 0x7f52140b5f60 2024-09-21T07:45:28.974 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-21T07:45:28.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.974+0000 7f5213200640 1 -- 172.21.15.39:0/3395850643 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f51f0078b90 msgr2=0x7f51f007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:28.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.974+0000 7f5213200640 1 --2- 172.21.15.39:0/3395850643 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f51f0078b90 0x7f51f007b0a0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f52140b8e10 tx=0x7f51fc005f90 comp rx=0 tx=0).stop 2024-09-21T07:45:28.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.974+0000 7f5213200640 1 -- 172.21.15.39:0/3395850643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52140b5f60 msgr2=0x7f52140cdb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:28.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.974+0000 7f5213200640 1 --2- 172.21.15.39:0/3395850643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52140b5f60 0x7f52140cdb20 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f5204009fd0 tx=0x7f520400ec20 comp rx=0 tx=0).stop 2024-09-21T07:45:28.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.974+0000 7f5213200640 1 -- 172.21.15.39:0/3395850643 shutdown_connections 2024-09-21T07:45:28.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.974+0000 7f5213200640 1 --2- 172.21.15.39:0/3395850643 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f51f0078b90 0x7f51f007b0a0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:28.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.974+0000 7f5213200640 1 --2- 172.21.15.39:0/3395850643 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52140d25a0 0x7f52140b11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:28.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.974+0000 7f5213200640 1 --2- 172.21.15.39:0/3395850643 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52140b6c90 0x7f52141f8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:28.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.974+0000 7f5213200640 1 --2- 172.21.15.39:0/3395850643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52140b5f60 0x7f52140cdb20 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:28.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.974+0000 7f5213200640 1 -- 172.21.15.39:0/3395850643 >> 172.21.15.39:0/3395850643 conn(0x7f521409d7b0 msgr2=0x7f52140b8e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:28.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.974+0000 7f5213200640 1 -- 172.21.15.39:0/3395850643 shutdown_connections 2024-09-21T07:45:28.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:28.974+0000 7f5213200640 1 -- 172.21.15.39:0/3395850643 wait complete. 2024-09-21T07:45:29.001 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2024-09-21T07:45:29.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.229+0000 7f6bca600640 1 -- 172.21.15.39:0/2217373639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6bc40d0390 msgr2=0x7f6bc40d07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:29.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.229+0000 7f6bca600640 1 --2- 172.21.15.39:0/2217373639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6bc40d0390 0x7f6bc40d07a0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc005fd0 tx=0x7f6bbc02aa00 comp rx=0 tx=0).stop 2024-09-21T07:45:29.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.229+0000 7f6bca600640 1 -- 172.21.15.39:0/2217373639 shutdown_connections 2024-09-21T07:45:29.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.229+0000 7f6bca600640 1 --2- 172.21.15.39:0/2217373639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6bc40d0390 0x7f6bc40d07a0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:29.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.229+0000 7f6bca600640 1 --2- 172.21.15.39:0/2217373639 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6bc4160250 0x7f6bc415d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:29.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.229+0000 7f6bca600640 1 -- 172.21.15.39:0/2217373639 >> 172.21.15.39:0/2217373639 conn(0x7f6bc406cfc0 msgr2=0x7f6bc406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:29.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.229+0000 7f6bca600640 1 -- 172.21.15.39:0/2217373639 shutdown_connections 2024-09-21T07:45:29.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.229+0000 7f6bca600640 1 -- 172.21.15.39:0/2217373639 wait complete. 2024-09-21T07:45:29.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.231+0000 7f6bca600640 1 Processor -- start 2024-09-21T07:45:29.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.231+0000 7f6bca600640 1 -- start start 2024-09-21T07:45:29.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.233+0000 7f6bca600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6bc40d0390 0x7f6bc4146160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:29.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.233+0000 7f6bca600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6bc4160250 0x7f6bc41f9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:29.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.233+0000 7f6bca600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6bc4160660 0x7f6bc40b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:29.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.233+0000 7f6bca600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc40ccdc0 con 0x7f6bc4160660 2024-09-21T07:45:29.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.233+0000 7f6bca600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc40ccc40 con 0x7f6bc40d0390 2024-09-21T07:45:29.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.233+0000 7f6bca600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f6bc40ccf40 con 0x7f6bc4160250 2024-09-21T07:45:29.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.233+0000 7f6bc9c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6bc4160660 0x7f6bc40b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:29.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.233+0000 7f6bc9c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6bc4160660 0x7f6bc40b1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45910/0 (socket says 172.21.15.39:45910) 2024-09-21T07:45:29.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.233+0000 7f6bc9200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6bc40d0390 0x7f6bc4146160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:29.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.233+0000 7f6bc9c00640 1 -- 172.21.15.39:0/3032585465 learned_addr learned my addr 172.21.15.39:0/3032585465 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:29.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.233+0000 7f6bc3e00640 1 --2- 172.21.15.39:0/3032585465 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6bc4160250 0x7f6bc41f9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:29.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.234+0000 7f6bc9c00640 1 -- 172.21.15.39:0/3032585465 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6bc4160250 msgr2=0x7f6bc41f9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:29.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.234+0000 7f6bc9c00640 1 --2- 172.21.15.39:0/3032585465 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6bc4160250 0x7f6bc41f9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:29.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.234+0000 7f6bc9c00640 1 -- 172.21.15.39:0/3032585465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6bc40d0390 msgr2=0x7f6bc4146160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:29.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.234+0000 7f6bc9c00640 1 --2- 172.21.15.39:0/3032585465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6bc40d0390 0x7f6bc4146160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:29.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.234+0000 7f6bc9c00640 1 -- 172.21.15.39:0/3032585465 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc4148940 con 0x7f6bc4160660 2024-09-21T07:45:29.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.234+0000 7f6bc9c00640 1 --2- 172.21.15.39:0/3032585465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6bc4160660 0x7f6bc40b1be0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc006010 tx=0x7f6bbc003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:29.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.234+0000 7f6bc0c00640 1 -- 172.21.15.39:0/3032585465 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc046070 con 0x7f6bc4160660 2024-09-21T07:45:29.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.234+0000 7f6bc0c00640 1 -- 172.21.15.39:0/3032585465 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6bbc004540 con 0x7f6bc4160660 2024-09-21T07:45:29.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.234+0000 7f6bca600640 1 -- 172.21.15.39:0/3032585465 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bc4148c20 con 0x7f6bc4160660 2024-09-21T07:45:29.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.234+0000 7f6bc0c00640 1 -- 172.21.15.39:0/3032585465 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc033070 con 0x7f6bc4160660 2024-09-21T07:45:29.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.235+0000 7f6bca600640 1 -- 172.21.15.39:0/3032585465 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bc41467a0 con 0x7f6bc4160660 2024-09-21T07:45:29.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.236+0000 7f6bca600640 1 -- 172.21.15.39:0/3032585465 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bc4149140 con 0x7f6bc4160660 2024-09-21T07:45:29.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.237+0000 7f6bc0c00640 1 -- 172.21.15.39:0/3032585465 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6bbc049070 con 0x7f6bc4160660 2024-09-21T07:45:29.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.238+0000 7f6bc0c00640 1 --2- 172.21.15.39:0/3032585465 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6ba4078b90 0x7f6ba407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:29.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.238+0000 7f6bc0c00640 1 -- 172.21.15.39:0/3032585465 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6078+0+0 (secure 0 0 0) 0x7f6bbc0bbfd0 con 0x7f6bc4160660 2024-09-21T07:45:29.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.238+0000 7f6bc9200640 1 --2- 172.21.15.39:0/3032585465 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6ba4078b90 0x7f6ba407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:29.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.239+0000 7f6bc9200640 1 --2- 172.21.15.39:0/3032585465 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6ba4078b90 0x7f6ba407b0a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8004630 tx=0x7f6bb800b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:29.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.242+0000 7f6bc0c00640 1 -- 172.21.15.39:0/3032585465 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6bc4149140 con 0x7f6bc4160660 2024-09-21T07:45:29.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.437+0000 7f6bca600640 1 -- 172.21.15.39:0/3032585465 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) -- 0x7f6bc40ce970 con 0x7f6bc4160660 2024-09-21T07:45:29.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.438+0000 7f6bc0c00640 1 -- 172.21.15.39:0/3032585465 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v149) ==== 103+0+11 (secure 0 0 0) 0x7f6bc40ce970 con 0x7f6bc4160660 2024-09-21T07:45:29.440 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-21T07:45:29.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.440+0000 7f6bca600640 1 -- 172.21.15.39:0/3032585465 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6ba4078b90 msgr2=0x7f6ba407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:29.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.440+0000 7f6bca600640 1 --2- 172.21.15.39:0/3032585465 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6ba4078b90 0x7f6ba407b0a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8004630 tx=0x7f6bb800b2e0 comp rx=0 tx=0).stop 2024-09-21T07:45:29.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.440+0000 7f6bca600640 1 -- 172.21.15.39:0/3032585465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6bc4160660 msgr2=0x7f6bc40b1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:29.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.440+0000 7f6bca600640 1 --2- 172.21.15.39:0/3032585465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6bc4160660 0x7f6bc40b1be0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc006010 tx=0x7f6bbc003f10 comp rx=0 tx=0).stop 2024-09-21T07:45:29.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.440+0000 7f6bca600640 1 -- 172.21.15.39:0/3032585465 shutdown_connections 2024-09-21T07:45:29.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.440+0000 7f6bca600640 1 --2- 172.21.15.39:0/3032585465 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6ba4078b90 0x7f6ba407b0a0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:29.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.440+0000 7f6bca600640 1 --2- 172.21.15.39:0/3032585465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6bc4160660 0x7f6bc40b1be0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:29.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.440+0000 7f6bca600640 1 --2- 172.21.15.39:0/3032585465 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6bc4160250 0x7f6bc41f9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:29.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.440+0000 7f6bca600640 1 --2- 172.21.15.39:0/3032585465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6bc40d0390 0x7f6bc4146160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:29.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.440+0000 7f6bca600640 1 -- 172.21.15.39:0/3032585465 >> 172.21.15.39:0/3032585465 conn(0x7f6bc406cfc0 msgr2=0x7f6bc406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:29.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.440+0000 7f6bca600640 1 -- 172.21.15.39:0/3032585465 shutdown_connections 2024-09-21T07:45:29.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.440+0000 7f6bca600640 1 -- 172.21.15.39:0/3032585465 wait complete. 2024-09-21T07:45:29.466 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2024-09-21T07:45:29.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.697+0000 7fb43aa00640 1 -- 172.21.15.39:0/419889694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4340ccc60 msgr2=0x7fb4340cd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:29.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.697+0000 7fb43aa00640 1 --2- 172.21.15.39:0/419889694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4340ccc60 0x7fb4340cd070 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fb434075160 tx=0x7fb42c02a980 comp rx=0 tx=0).stop 2024-09-21T07:45:29.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.697+0000 7fb43aa00640 1 -- 172.21.15.39:0/419889694 shutdown_connections 2024-09-21T07:45:29.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.697+0000 7fb43aa00640 1 --2- 172.21.15.39:0/419889694 >> v2:172.21.15.148:3300/0 conn(0x7fb4340cb8a0 0x7fb434156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:29.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.697+0000 7fb43aa00640 1 --2- 172.21.15.39:0/419889694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4340ccc60 0x7fb4340cd070 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:29.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.697+0000 7fb43aa00640 1 -- 172.21.15.39:0/419889694 >> 172.21.15.39:0/419889694 conn(0x7fb4340b2460 msgr2=0x7fb43406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:29.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.698+0000 7fb43aa00640 1 -- 172.21.15.39:0/419889694 shutdown_connections 2024-09-21T07:45:29.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.698+0000 7fb43aa00640 1 -- 172.21.15.39:0/419889694 wait complete. 2024-09-21T07:45:29.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.700+0000 7fb43aa00640 1 Processor -- start 2024-09-21T07:45:29.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.700+0000 7fb43aa00640 1 -- start start 2024-09-21T07:45:29.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.701+0000 7fb43aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4340b30a0 0x7fb4341f4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:29.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.701+0000 7fb43aa00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4340cb8a0 0x7fb4341f8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:29.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.702+0000 7fb43aa00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4340ccc60 0x7fb4341f9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:29.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.702+0000 7fb43aa00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb4340af410 con 0x7fb4340b30a0 2024-09-21T07:45:29.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.702+0000 7fb43aa00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb4340af290 con 0x7fb4340cb8a0 2024-09-21T07:45:29.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.702+0000 7fb43aa00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb4340af590 con 0x7fb4340ccc60 2024-09-21T07:45:29.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.702+0000 7fb433e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4340b30a0 0x7fb4341f4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:29.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.702+0000 7fb433e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4340b30a0 0x7fb4341f4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45924/0 (socket says 172.21.15.39:45924) 2024-09-21T07:45:29.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.702+0000 7fb433e00640 1 -- 172.21.15.39:0/3743530658 learned_addr learned my addr 172.21.15.39:0/3743530658 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:29.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.702+0000 7fb433400640 1 --2- 172.21.15.39:0/3743530658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4340cb8a0 0x7fb4341f8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:29.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.702+0000 7fb438a00640 1 --2- 172.21.15.39:0/3743530658 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4340ccc60 0x7fb4341f9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:29.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.702+0000 7fb433e00640 1 -- 172.21.15.39:0/3743530658 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4340ccc60 msgr2=0x7fb4341f9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:29.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.702+0000 7fb433e00640 1 --2- 172.21.15.39:0/3743530658 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4340ccc60 0x7fb4341f9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:29.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.702+0000 7fb433e00640 1 -- 172.21.15.39:0/3743530658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4340cb8a0 msgr2=0x7fb4341f8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:29.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.702+0000 7fb433e00640 1 --2- 172.21.15.39:0/3743530658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4340cb8a0 0x7fb4341f8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:29.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.702+0000 7fb433e00640 1 -- 172.21.15.39:0/3743530658 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4340c5890 con 0x7fb4340b30a0 2024-09-21T07:45:29.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.702+0000 7fb433400640 1 --2- 172.21.15.39:0/3743530658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4340cb8a0 0x7fb4341f8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:45:29.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.703+0000 7fb433e00640 1 --2- 172.21.15.39:0/3743530658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4340b30a0 0x7fb4341f4f90 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fb42400b420 tx=0x7fb42400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:29.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.703+0000 7fb42be00640 1 -- 172.21.15.39:0/3743530658 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb424011830 con 0x7fb4340b30a0 2024-09-21T07:45:29.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.703+0000 7fb42be00640 1 -- 172.21.15.39:0/3743530658 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb424016070 con 0x7fb4340b30a0 2024-09-21T07:45:29.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.703+0000 7fb43aa00640 1 -- 172.21.15.39:0/3743530658 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4341f98a0 con 0x7fb4340b30a0 2024-09-21T07:45:29.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.703+0000 7fb42be00640 1 -- 172.21.15.39:0/3743530658 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb424011d60 con 0x7fb4340b30a0 2024-09-21T07:45:29.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.703+0000 7fb43aa00640 1 -- 172.21.15.39:0/3743530658 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4341f54d0 con 0x7fb4340b30a0 2024-09-21T07:45:29.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.704+0000 7fb43aa00640 1 -- 172.21.15.39:0/3743530658 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4340008d0 con 0x7fb4340b30a0 2024-09-21T07:45:29.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.705+0000 7fb42be00640 1 -- 172.21.15.39:0/3743530658 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb42401b460 con 0x7fb4340b30a0 2024-09-21T07:45:29.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.706+0000 7fb42be00640 1 --2- 172.21.15.39:0/3743530658 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb414078b90 0x7fb41407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:29.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.706+0000 7fb42be00640 1 -- 172.21.15.39:0/3743530658 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6078+0+0 (secure 0 0 0) 0x7fb42409aa50 con 0x7fb4340b30a0 2024-09-21T07:45:29.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.707+0000 7fb433400640 1 --2- 172.21.15.39:0/3743530658 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb414078b90 0x7fb41407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:29.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.708+0000 7fb433400640 1 --2- 172.21.15.39:0/3743530658 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb414078b90 0x7fb41407b0a0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fb4340c6be0 tx=0x7fb420005f10 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:29.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.711+0000 7fb42be00640 1 -- 172.21.15.39:0/3743530658 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb424020020 con 0x7fb4340b30a0 2024-09-21T07:45:29.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.909+0000 7fb43aa00640 1 -- 172.21.15.39:0/3743530658 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) -- 0x7fb4340c6050 con 0x7fb4340b30a0 2024-09-21T07:45:29.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.909+0000 7fb42be00640 1 -- 172.21.15.39:0/3743530658 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v149) ==== 107+0+11 (secure 0 0 0) 0x7fb424062d10 con 0x7fb4340b30a0 2024-09-21T07:45:29.911 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-21T07:45:29.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.911+0000 7fb43aa00640 1 -- 172.21.15.39:0/3743530658 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb414078b90 msgr2=0x7fb41407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:29.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.911+0000 7fb43aa00640 1 --2- 172.21.15.39:0/3743530658 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb414078b90 0x7fb41407b0a0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fb4340c6be0 tx=0x7fb420005f10 comp rx=0 tx=0).stop 2024-09-21T07:45:29.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.911+0000 7fb43aa00640 1 -- 172.21.15.39:0/3743530658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4340b30a0 msgr2=0x7fb4341f4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:29.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.911+0000 7fb43aa00640 1 --2- 172.21.15.39:0/3743530658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4340b30a0 0x7fb4341f4f90 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fb42400b420 tx=0x7fb42400b8e0 comp rx=0 tx=0).stop 2024-09-21T07:45:29.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.912+0000 7fb43aa00640 1 -- 172.21.15.39:0/3743530658 shutdown_connections 2024-09-21T07:45:29.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.912+0000 7fb43aa00640 1 --2- 172.21.15.39:0/3743530658 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb414078b90 0x7fb41407b0a0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:29.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.912+0000 7fb43aa00640 1 --2- 172.21.15.39:0/3743530658 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4340ccc60 0x7fb4341f9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:29.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.912+0000 7fb43aa00640 1 --2- 172.21.15.39:0/3743530658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4340cb8a0 0x7fb4341f8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:29.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.912+0000 7fb43aa00640 1 --2- 172.21.15.39:0/3743530658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4340b30a0 0x7fb4341f4f90 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:29.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.912+0000 7fb43aa00640 1 -- 172.21.15.39:0/3743530658 >> 172.21.15.39:0/3743530658 conn(0x7fb4340b2460 msgr2=0x7fb43415c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:29.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.912+0000 7fb43aa00640 1 -- 172.21.15.39:0/3743530658 shutdown_connections 2024-09-21T07:45:29.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:29.912+0000 7fb43aa00640 1 -- 172.21.15.39:0/3743530658 wait complete. 2024-09-21T07:45:29.936 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_mds_max_snaps_per_dir_with_increased_limit' 2024-09-21T07:45:30.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.162+0000 7fc65b200640 1 -- 172.21.15.39:0/2008685994 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc63c004d40 msgr2=0x7fc63c005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:30.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.162+0000 7fc65b200640 1 --2- 172.21.15.39:0/2008685994 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc63c004d40 0x7fc63c005150 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fc650009ed0 tx=0x7fc63c005830 comp rx=0 tx=0).stop 2024-09-21T07:45:30.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.162+0000 7fc65a800640 1 -- 172.21.15.39:0/2008685994 reap_dead start 2024-09-21T07:45:30.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.162+0000 7fc65b200640 1 -- 172.21.15.39:0/2008685994 shutdown_connections 2024-09-21T07:45:30.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.162+0000 7fc65b200640 1 -- 172.21.15.39:0/2008685994 >> 172.21.15.39:0/2008685994 conn(0x7fc65406cfc0 msgr2=0x7fc65406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:30.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.162+0000 7fc65b200640 1 -- 172.21.15.39:0/2008685994 shutdown_connections 2024-09-21T07:45:30.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.162+0000 7fc65b200640 1 -- 172.21.15.39:0/2008685994 wait complete. 2024-09-21T07:45:30.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.164+0000 7fc65b200640 1 Processor -- start 2024-09-21T07:45:30.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.165+0000 7fc65b200640 1 -- start start 2024-09-21T07:45:30.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.166+0000 7fc65b200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc63c004d40 0x7fc654140930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:30.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.166+0000 7fc65b200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc654142ff0 0x7fc654141330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:30.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.166+0000 7fc65b200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc654141870 0x7fc6541494b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:30.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.166+0000 7fc65b200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc65406f630 con 0x7fc63c004d40 2024-09-21T07:45:30.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.166+0000 7fc65b200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc65406f4b0 con 0x7fc654142ff0 2024-09-21T07:45:30.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.166+0000 7fc65b200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc65406f7b0 con 0x7fc654141870 2024-09-21T07:45:30.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.166+0000 7fc659e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc63c004d40 0x7fc654140930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:30.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.166+0000 7fc659e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc63c004d40 0x7fc654140930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45938/0 (socket says 172.21.15.39:45938) 2024-09-21T07:45:30.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.166+0000 7fc659e00640 1 -- 172.21.15.39:0/2997421868 learned_addr learned my addr 172.21.15.39:0/2997421868 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:30.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.166+0000 7fc659400640 1 --2- 172.21.15.39:0/2997421868 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc654142ff0 0x7fc654141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:30.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.166+0000 7fc65a800640 1 --2- 172.21.15.39:0/2997421868 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc654141870 0x7fc6541494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:30.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.167+0000 7fc659e00640 1 -- 172.21.15.39:0/2997421868 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc654141870 msgr2=0x7fc6541494b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:30.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.167+0000 7fc659e00640 1 --2- 172.21.15.39:0/2997421868 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc654141870 0x7fc6541494b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:30.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.167+0000 7fc659e00640 1 -- 172.21.15.39:0/2997421868 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc654142ff0 msgr2=0x7fc654141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:30.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.167+0000 7fc659e00640 1 --2- 172.21.15.39:0/2997421868 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc654142ff0 0x7fc654141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:30.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.167+0000 7fc659e00640 1 -- 172.21.15.39:0/2997421868 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc65413fb70 con 0x7fc63c004d40 2024-09-21T07:45:30.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.167+0000 7fc659e00640 1 --2- 172.21.15.39:0/2997421868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc63c004d40 0x7fc654140930 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fc648011a50 tx=0x7fc648011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:30.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.167+0000 7fc64e000640 1 -- 172.21.15.39:0/2997421868 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64801f070 con 0x7fc63c004d40 2024-09-21T07:45:30.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.167+0000 7fc64e000640 1 -- 172.21.15.39:0/2997421868 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc64800fe80 con 0x7fc63c004d40 2024-09-21T07:45:30.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.167+0000 7fc64e000640 1 -- 172.21.15.39:0/2997421868 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc648012690 con 0x7fc63c004d40 2024-09-21T07:45:30.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.167+0000 7fc65b200640 1 -- 172.21.15.39:0/2997421868 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc65413fe50 con 0x7fc63c004d40 2024-09-21T07:45:30.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.168+0000 7fc65b200640 1 -- 172.21.15.39:0/2997421868 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc654140410 con 0x7fc63c004d40 2024-09-21T07:45:30.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.169+0000 7fc65b200640 1 -- 172.21.15.39:0/2997421868 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc654073f30 con 0x7fc63c004d40 2024-09-21T07:45:30.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.170+0000 7fc64e000640 1 -- 172.21.15.39:0/2997421868 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc648022070 con 0x7fc63c004d40 2024-09-21T07:45:30.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.171+0000 7fc64e000640 1 --2- 172.21.15.39:0/2997421868 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc638078b90 0x7fc63807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:30.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.171+0000 7fc64e000640 1 -- 172.21.15.39:0/2997421868 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6078+0+0 (secure 0 0 0) 0x7fc64809b990 con 0x7fc63c004d40 2024-09-21T07:45:30.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.171+0000 7fc659400640 1 --2- 172.21.15.39:0/2997421868 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc638078b90 0x7fc63807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:30.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.172+0000 7fc659400640 1 --2- 172.21.15.39:0/2997421868 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc638078b90 0x7fc63807b0a0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fc65414d5b0 tx=0x7fc644006040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:30.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.175+0000 7fc64e000640 1 -- 172.21.15.39:0/2997421868 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc648063600 con 0x7fc63c004d40 2024-09-21T07:45:30.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.446+0000 7fc65b200640 1 -- 172.21.15.39:0/2997421868 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_mds_max_snaps_per_dir_with_increased_limit"]} v 0) -- 0x7fc654167270 con 0x7fc63c004d40 2024-09-21T07:45:30.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.588+0000 7fc64e000640 1 -- 172.21.15.39:0/2997421868 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_mds_max_snaps_per_dir_with_increased_limit"]}]=0 v596) ==== 171+0+0 (secure 0 0 0) 0x7fc648068030 con 0x7fc63c004d40 2024-09-21T07:45:30.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.591+0000 7fc65b200640 1 -- 172.21.15.39:0/2997421868 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc638078b90 msgr2=0x7fc63807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:30.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.591+0000 7fc65b200640 1 --2- 172.21.15.39:0/2997421868 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc638078b90 0x7fc63807b0a0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fc65414d5b0 tx=0x7fc644006040 comp rx=0 tx=0).stop 2024-09-21T07:45:30.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.591+0000 7fc65b200640 1 -- 172.21.15.39:0/2997421868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc63c004d40 msgr2=0x7fc654140930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:30.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.591+0000 7fc65b200640 1 --2- 172.21.15.39:0/2997421868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc63c004d40 0x7fc654140930 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fc648011a50 tx=0x7fc648011f10 comp rx=0 tx=0).stop 2024-09-21T07:45:30.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.591+0000 7fc65b200640 1 -- 172.21.15.39:0/2997421868 shutdown_connections 2024-09-21T07:45:30.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.591+0000 7fc65b200640 1 --2- 172.21.15.39:0/2997421868 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc638078b90 0x7fc63807b0a0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:30.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.591+0000 7fc65b200640 1 --2- 172.21.15.39:0/2997421868 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc654141870 0x7fc6541494b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:30.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.591+0000 7fc65b200640 1 --2- 172.21.15.39:0/2997421868 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc654142ff0 0x7fc654141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:30.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.591+0000 7fc65b200640 1 --2- 172.21.15.39:0/2997421868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc63c004d40 0x7fc654140930 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:30.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.591+0000 7fc65b200640 1 -- 172.21.15.39:0/2997421868 >> 172.21.15.39:0/2997421868 conn(0x7fc65406cfc0 msgr2=0x7fc6540b43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:30.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.592+0000 7fc65b200640 1 -- 172.21.15.39:0/2997421868 shutdown_connections 2024-09-21T07:45:30.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.592+0000 7fc65b200640 1 -- 172.21.15.39:0/2997421868 wait complete. 2024-09-21T07:45:30.615 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T07:45:30.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.839+0000 7f6798c00640 1 -- 172.21.15.39:0/2679043790 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f679415d830 msgr2=0x7f6794156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:30.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.839+0000 7f6798c00640 1 --2- 172.21.15.39:0/2679043790 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f679415d830 0x7f6794156bb0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f6794071e60 tx=0x7f678002a560 comp rx=0 tx=0).stop 2024-09-21T07:45:30.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.839+0000 7f6798c00640 1 -- 172.21.15.39:0/2679043790 shutdown_connections 2024-09-21T07:45:30.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.839+0000 7f6798c00640 1 --2- 172.21.15.39:0/2679043790 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f679415d830 0x7f6794156bb0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:30.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.839+0000 7f6798c00640 1 --2- 172.21.15.39:0/2679043790 >> v2:172.21.15.148:3300/0 conn(0x7f67940b30a0 0x7f67940cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:30.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.839+0000 7f6798c00640 1 -- 172.21.15.39:0/2679043790 >> 172.21.15.39:0/2679043790 conn(0x7f67940b2460 msgr2=0x7f679406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:30.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.839+0000 7f6798c00640 1 -- 172.21.15.39:0/2679043790 shutdown_connections 2024-09-21T07:45:30.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.840+0000 7f6798c00640 1 -- 172.21.15.39:0/2679043790 wait complete. 2024-09-21T07:45:30.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.842+0000 7f6798c00640 1 Processor -- start 2024-09-21T07:45:30.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.842+0000 7f6798c00640 1 -- start start 2024-09-21T07:45:30.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.843+0000 7f6798c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67940b30a0 0x7f67940c5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:30.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.843+0000 7f6798c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67940ccc60 0x7f67940c7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:30.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.843+0000 7f6798c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f679415d830 0x7f67940c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:30.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.843+0000 7f6798c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f67940af410 con 0x7f67940ccc60 2024-09-21T07:45:30.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.843+0000 7f6798c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f67940af290 con 0x7f67940b30a0 2024-09-21T07:45:30.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.843+0000 7f6798c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f67940af590 con 0x7f679415d830 2024-09-21T07:45:30.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.843+0000 7f6791400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67940ccc60 0x7f67940c7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:30.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.844+0000 7f6791400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67940ccc60 0x7f67940c7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45954/0 (socket says 172.21.15.39:45954) 2024-09-21T07:45:30.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.844+0000 7f6791400640 1 -- 172.21.15.39:0/1778794394 learned_addr learned my addr 172.21.15.39:0/1778794394 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:30.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.844+0000 7f6792800640 1 --2- 172.21.15.39:0/1778794394 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f679415d830 0x7f67940c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:30.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.844+0000 7f6791e00640 1 --2- 172.21.15.39:0/1778794394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67940b30a0 0x7f67940c5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:30.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.844+0000 7f6791400640 1 -- 172.21.15.39:0/1778794394 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f679415d830 msgr2=0x7f67940c8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:30.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.844+0000 7f6791400640 1 --2- 172.21.15.39:0/1778794394 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f679415d830 0x7f67940c8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:30.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.844+0000 7f6791400640 1 -- 172.21.15.39:0/1778794394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67940b30a0 msgr2=0x7f67940c5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:30.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.844+0000 7f6791400640 1 --2- 172.21.15.39:0/1778794394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67940b30a0 0x7f67940c5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:30.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.844+0000 7f6791400640 1 -- 172.21.15.39:0/1778794394 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6794155450 con 0x7f67940ccc60 2024-09-21T07:45:30.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.844+0000 7f6792800640 1 --2- 172.21.15.39:0/1778794394 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f679415d830 0x7f67940c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:45:30.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.844+0000 7f6791400640 1 --2- 172.21.15.39:0/1778794394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67940ccc60 0x7f67940c7d10 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f6780034040 tx=0x7f6780003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:30.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.845+0000 7f678a000640 1 -- 172.21.15.39:0/1778794394 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678003e030 con 0x7f67940ccc60 2024-09-21T07:45:30.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.845+0000 7f678a000640 1 -- 172.21.15.39:0/1778794394 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6780003e00 con 0x7f67940ccc60 2024-09-21T07:45:30.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.845+0000 7f678a000640 1 -- 172.21.15.39:0/1778794394 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6780035040 con 0x7f67940ccc60 2024-09-21T07:45:30.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.845+0000 7f6798c00640 1 -- 172.21.15.39:0/1778794394 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6794155730 con 0x7f67940ccc60 2024-09-21T07:45:30.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.845+0000 7f6798c00640 1 -- 172.21.15.39:0/1778794394 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6794155cf0 con 0x7f67940ccc60 2024-09-21T07:45:30.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.846+0000 7f6798c00640 1 -- 172.21.15.39:0/1778794394 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67940b0bc0 con 0x7f67940ccc60 2024-09-21T07:45:30.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.847+0000 7f678a000640 1 -- 172.21.15.39:0/1778794394 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6780041030 con 0x7f67940ccc60 2024-09-21T07:45:30.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.848+0000 7f678a000640 1 --2- 172.21.15.39:0/1778794394 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6774078b90 0x7f677407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:30.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.848+0000 7f678a000640 1 -- 172.21.15.39:0/1778794394 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6078+0+0 (secure 0 0 0) 0x7f67800bb6b0 con 0x7f67940ccc60 2024-09-21T07:45:30.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.848+0000 7f6791e00640 1 --2- 172.21.15.39:0/1778794394 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6774078b90 0x7f677407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:30.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.849+0000 7f6791e00640 1 --2- 172.21.15.39:0/1778794394 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6774078b90 0x7f677407b0a0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f6784004630 tx=0x7f678400b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:30.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:30.852+0000 7f678a000640 1 -- 172.21.15.39:0/1778794394 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6780083290 con 0x7f67940ccc60 2024-09-21T07:45:31.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.047+0000 7f6798c00640 1 -- 172.21.15.39:0/1778794394 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f679406df10 con 0x7f67940ccc60 2024-09-21T07:45:31.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.048+0000 7f678a000640 1 -- 172.21.15.39:0/1778794394 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v44) ==== 59+0+2294 (secure 0 0 0) 0x7f6780087cc0 con 0x7f67940ccc60 2024-09-21T07:45:31.050 INFO:teuthology.orchestra.run.smithi039.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T07:45:31.050 INFO:teuthology.orchestra.run.smithi039.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T07:45:31.050 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced client_mount_timeout 30 2024-09-21T07:45:31.050 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_client 20/20 2024-09-21T07:45:31.050 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_mgr 20/20 2024-09-21T07:45:31.050 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_ms 1/1 2024-09-21T07:45:31.050 INFO:teuthology.orchestra.run.smithi039.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T07:45:31.051 INFO:teuthology.orchestra.run.smithi039.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T07:45:31.051 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3421.881759 2024-09-21T07:45:31.051 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3520.252978 2024-09-21T07:45:31.051 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3511.183316 2024-09-21T07:45:31.051 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3829.631847 2024-09-21T07:45:31.051 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 5474.545872 2024-09-21T07:45:31.051 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4013.628559 2024-09-21T07:45:31.051 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4230.187288 2024-09-21T07:45:31.051 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4148.983107 2024-09-21T07:45:31.051 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_mds 20/20 2024-09-21T07:45:31.051 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T07:45:31.051 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_ms 2/2 2024-09-21T07:45:31.051 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_debug_frag true 2024-09-21T07:45:31.052 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_debug_scatterstat true 2024-09-21T07:45:31.052 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T07:45:31.052 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_verify_scatter true 2024-09-21T07:45:31.052 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T07:45:31.052 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T07:45:31.052 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T07:45:31.052 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced client_mount_timeout 600 2024-09-21T07:45:31.052 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced debug_client 20/20 2024-09-21T07:45:31.052 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced debug_ms 1/1 2024-09-21T07:45:31.052 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T07:45:31.052 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T07:45:31.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.050+0000 7f6798c00640 1 -- 172.21.15.39:0/1778794394 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6774078b90 msgr2=0x7f677407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:31.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.050+0000 7f6798c00640 1 --2- 172.21.15.39:0/1778794394 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6774078b90 0x7f677407b0a0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f6784004630 tx=0x7f678400b2e0 comp rx=0 tx=0).stop 2024-09-21T07:45:31.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.050+0000 7f6798c00640 1 -- 172.21.15.39:0/1778794394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67940ccc60 msgr2=0x7f67940c7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:31.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.050+0000 7f6798c00640 1 --2- 172.21.15.39:0/1778794394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67940ccc60 0x7f67940c7d10 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f6780034040 tx=0x7f6780003910 comp rx=0 tx=0).stop 2024-09-21T07:45:31.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.050+0000 7f6798c00640 1 -- 172.21.15.39:0/1778794394 shutdown_connections 2024-09-21T07:45:31.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.050+0000 7f6798c00640 1 --2- 172.21.15.39:0/1778794394 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6774078b90 0x7f677407b0a0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:31.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.050+0000 7f6798c00640 1 --2- 172.21.15.39:0/1778794394 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f679415d830 0x7f67940c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:31.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.050+0000 7f6798c00640 1 --2- 172.21.15.39:0/1778794394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67940ccc60 0x7f67940c7d10 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:31.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.050+0000 7f6798c00640 1 --2- 172.21.15.39:0/1778794394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67940b30a0 0x7f67940c5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:31.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.050+0000 7f6798c00640 1 -- 172.21.15.39:0/1778794394 >> 172.21.15.39:0/1778794394 conn(0x7f67940b2460 msgr2=0x7f679406f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:31.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.050+0000 7f6798c00640 1 -- 172.21.15.39:0/1778794394 shutdown_connections 2024-09-21T07:45:31.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.050+0000 7f6798c00640 1 -- 172.21.15.39:0/1778794394 wait complete. 2024-09-21T07:45:31.073 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T07:45:31.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.300+0000 7f87d2e00640 1 -- 172.21.15.39:0/4090567324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87cc0743a0 msgr2=0x7f87cc0d09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:31.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.300+0000 7f87d2e00640 1 --2- 172.21.15.39:0/4090567324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87cc0743a0 0x7f87cc0d09f0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f87cc06ffc0 tx=0x7f87c402a9a0 comp rx=0 tx=0).stop 2024-09-21T07:45:31.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.300+0000 7f87d2e00640 1 -- 172.21.15.39:0/4090567324 shutdown_connections 2024-09-21T07:45:31.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.300+0000 7f87d2e00640 1 --2- 172.21.15.39:0/4090567324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87cc0743a0 0x7f87cc0d09f0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:31.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.300+0000 7f87d2e00640 1 --2- 172.21.15.39:0/4090567324 >> v2:172.21.15.148:3300/0 conn(0x7f87cc073f90 0x7f87cc0d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:31.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.300+0000 7f87d2e00640 1 --2- 172.21.15.39:0/4090567324 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87cc0b3b50 0x7f87cc15c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:31.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.300+0000 7f87d2e00640 1 -- 172.21.15.39:0/4090567324 >> 172.21.15.39:0/4090567324 conn(0x7f87cc06cfc0 msgr2=0x7f87cc06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:31.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.300+0000 7f87d2e00640 1 -- 172.21.15.39:0/4090567324 shutdown_connections 2024-09-21T07:45:31.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.300+0000 7f87d2e00640 1 -- 172.21.15.39:0/4090567324 wait complete. 2024-09-21T07:45:31.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.302+0000 7f87d2e00640 1 Processor -- start 2024-09-21T07:45:31.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.303+0000 7f87d2e00640 1 -- start start 2024-09-21T07:45:31.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.304+0000 7f87d2e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87cc073f90 0x7f87cc145a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:31.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.304+0000 7f87d2e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87cc0743a0 0x7f87cc1439f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:31.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.304+0000 7f87d2e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87cc0b3b50 0x7f87cc15a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:31.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.304+0000 7f87d2e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc155620 con 0x7f87cc0743a0 2024-09-21T07:45:31.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.304+0000 7f87d2e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc1554a0 con 0x7f87cc073f90 2024-09-21T07:45:31.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.304+0000 7f87d2e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f87cc1557a0 con 0x7f87cc0b3b50 2024-09-21T07:45:31.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.304+0000 7f87d1000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87cc0743a0 0x7f87cc1439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:31.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.304+0000 7f87d1000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87cc0743a0 0x7f87cc1439f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45966/0 (socket says 172.21.15.39:45966) 2024-09-21T07:45:31.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.304+0000 7f87d1000640 1 -- 172.21.15.39:0/1646373271 learned_addr learned my addr 172.21.15.39:0/1646373271 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:31.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.305+0000 7f87d2400640 1 --2- 172.21.15.39:0/1646373271 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87cc0b3b50 0x7f87cc15a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:31.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.305+0000 7f87d1a00640 1 --2- 172.21.15.39:0/1646373271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87cc073f90 0x7f87cc145a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:31.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.305+0000 7f87d1000640 1 -- 172.21.15.39:0/1646373271 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87cc0b3b50 msgr2=0x7f87cc15a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:31.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.305+0000 7f87d1000640 1 --2- 172.21.15.39:0/1646373271 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87cc0b3b50 0x7f87cc15a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:31.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.305+0000 7f87d1000640 1 -- 172.21.15.39:0/1646373271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87cc073f90 msgr2=0x7f87cc145a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:31.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.305+0000 7f87d1000640 1 --2- 172.21.15.39:0/1646373271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87cc073f90 0x7f87cc145a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:31.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.305+0000 7f87d1000640 1 -- 172.21.15.39:0/1646373271 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87cc1f8970 con 0x7f87cc0743a0 2024-09-21T07:45:31.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.305+0000 7f87d1000640 1 --2- 172.21.15.39:0/1646373271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87cc0743a0 0x7f87cc1439f0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f87b800d750 tx=0x7f87b800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:31.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.305+0000 7f87c9600640 1 -- 172.21.15.39:0/1646373271 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b8017070 con 0x7f87cc0743a0 2024-09-21T07:45:31.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.305+0000 7f87c9600640 1 -- 172.21.15.39:0/1646373271 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87b8009cc0 con 0x7f87cc0743a0 2024-09-21T07:45:31.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.305+0000 7f87c9600640 1 -- 172.21.15.39:0/1646373271 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b80106c0 con 0x7f87cc0743a0 2024-09-21T07:45:31.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.305+0000 7f87d2e00640 1 -- 172.21.15.39:0/1646373271 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87cc1462d0 con 0x7f87cc0743a0 2024-09-21T07:45:31.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.306+0000 7f87d2e00640 1 -- 172.21.15.39:0/1646373271 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87cc146890 con 0x7f87cc0743a0 2024-09-21T07:45:31.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.307+0000 7f87d2e00640 1 -- 172.21.15.39:0/1646373271 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87cc146040 con 0x7f87cc0743a0 2024-09-21T07:45:31.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.308+0000 7f87c9600640 1 -- 172.21.15.39:0/1646373271 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f87b801a070 con 0x7f87cc0743a0 2024-09-21T07:45:31.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.309+0000 7f87c9600640 1 --2- 172.21.15.39:0/1646373271 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f87ac078b90 0x7f87ac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:31.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.309+0000 7f87c9600640 1 -- 172.21.15.39:0/1646373271 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6078+0+0 (secure 0 0 0) 0x7f87b809afc0 con 0x7f87cc0743a0 2024-09-21T07:45:31.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.310+0000 7f87d1a00640 1 --2- 172.21.15.39:0/1646373271 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f87ac078b90 0x7f87ac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:31.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.310+0000 7f87d1a00640 1 --2- 172.21.15.39:0/1646373271 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f87ac078b90 0x7f87ac07b0a0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f87c0005fd0 tx=0x7f87c0005850 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:31.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.314+0000 7f87c9600640 1 -- 172.21.15.39:0/1646373271 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f87b800de60 con 0x7f87cc0743a0 2024-09-21T07:45:31.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.510+0000 7f87d2e00640 1 -- 172.21.15.39:0/1646373271 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f87cc15c510 con 0x7f87cc0743a0 2024-09-21T07:45:31.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.514+0000 7f87c9600640 1 -- 172.21.15.39:0/1646373271 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v45)=0 v45) ==== 122+0+0 (secure 0 0 0) 0x7f87b8063280 con 0x7f87cc0743a0 2024-09-21T07:45:31.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.517+0000 7f87d2e00640 1 -- 172.21.15.39:0/1646373271 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f87ac078b90 msgr2=0x7f87ac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:31.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.517+0000 7f87d2e00640 1 --2- 172.21.15.39:0/1646373271 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f87ac078b90 0x7f87ac07b0a0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f87c0005fd0 tx=0x7f87c0005850 comp rx=0 tx=0).stop 2024-09-21T07:45:31.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.517+0000 7f87d2e00640 1 -- 172.21.15.39:0/1646373271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87cc0743a0 msgr2=0x7f87cc1439f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:31.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.517+0000 7f87d2e00640 1 --2- 172.21.15.39:0/1646373271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87cc0743a0 0x7f87cc1439f0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f87b800d750 tx=0x7f87b800dc10 comp rx=0 tx=0).stop 2024-09-21T07:45:31.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.517+0000 7f87d2e00640 1 -- 172.21.15.39:0/1646373271 shutdown_connections 2024-09-21T07:45:31.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.517+0000 7f87d2e00640 1 --2- 172.21.15.39:0/1646373271 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f87ac078b90 0x7f87ac07b0a0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:31.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.517+0000 7f87d2e00640 1 --2- 172.21.15.39:0/1646373271 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87cc0b3b50 0x7f87cc15a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:31.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.517+0000 7f87d2e00640 1 --2- 172.21.15.39:0/1646373271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87cc0743a0 0x7f87cc1439f0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:31.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.517+0000 7f87d2e00640 1 --2- 172.21.15.39:0/1646373271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87cc073f90 0x7f87cc145a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:31.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.517+0000 7f87d2e00640 1 -- 172.21.15.39:0/1646373271 >> 172.21.15.39:0/1646373271 conn(0x7f87cc06cfc0 msgr2=0x7f87cc15cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:31.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.517+0000 7f87d2e00640 1 -- 172.21.15.39:0/1646373271 shutdown_connections 2024-09-21T07:45:31.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:31.517+0000 7f87d2e00640 1 -- 172.21.15.39:0/1646373271 wait complete. 2024-09-21T07:45:31.543 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:45:31.543 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:45:31.620 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:45:31.708 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:45:31.708 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:45:31.709 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:45:31.743 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:45:31.744 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:45:31.744 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:45:31.780 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:45:31.780 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:45:31.781 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:45:31.819 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:45:31.820 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:45:31.820 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:45:31.857 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:45:31.857 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T07:45:31.858 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:45:32.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.085+0000 7fe9dba00640 1 -- 172.21.15.39:0/2551440055 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9d415d830 msgr2=0x7fe9d4156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:32.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.085+0000 7fe9dba00640 1 --2- 172.21.15.39:0/2551440055 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9d415d830 0x7fe9d4156bb0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4071e60 tx=0x7fe9c002a4d0 comp rx=0 tx=0).stop 2024-09-21T07:45:32.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.085+0000 7fe9dba00640 1 -- 172.21.15.39:0/2551440055 shutdown_connections 2024-09-21T07:45:32.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.085+0000 7fe9dba00640 1 --2- 172.21.15.39:0/2551440055 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9d40ccc60 0x7fe9d40cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:32.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.085+0000 7fe9dba00640 1 --2- 172.21.15.39:0/2551440055 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9d415d830 0x7fe9d4156bb0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:32.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.085+0000 7fe9dba00640 1 --2- 172.21.15.39:0/2551440055 >> v2:172.21.15.148:3300/0 conn(0x7fe9d40b30a0 0x7fe9d40cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:32.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.085+0000 7fe9dba00640 1 -- 172.21.15.39:0/2551440055 >> 172.21.15.39:0/2551440055 conn(0x7fe9d40b2460 msgr2=0x7fe9d406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:32.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.085+0000 7fe9dba00640 1 -- 172.21.15.39:0/2551440055 shutdown_connections 2024-09-21T07:45:32.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.085+0000 7fe9dba00640 1 -- 172.21.15.39:0/2551440055 wait complete. 2024-09-21T07:45:32.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.088+0000 7fe9dba00640 1 Processor -- start 2024-09-21T07:45:32.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.088+0000 7fe9dba00640 1 -- start start 2024-09-21T07:45:32.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.089+0000 7fe9dba00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9d40b30a0 0x7fe9d41f52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:32.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.089+0000 7fe9dba00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9d40ccc60 0x7fe9d414da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:32.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.089+0000 7fe9dba00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9d415d830 0x7fe9d414df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:32.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.089+0000 7fe9dba00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d40af200 con 0x7fe9d40b30a0 2024-09-21T07:45:32.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.089+0000 7fe9dba00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d40af080 con 0x7fe9d415d830 2024-09-21T07:45:32.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.089+0000 7fe9dba00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe9d40af380 con 0x7fe9d40ccc60 2024-09-21T07:45:32.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.089+0000 7fe9d9000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9d40b30a0 0x7fe9d41f52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:32.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.090+0000 7fe9d9000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9d40b30a0 0x7fe9d41f52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52054/0 (socket says 172.21.15.39:52054) 2024-09-21T07:45:32.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.090+0000 7fe9d9000640 1 -- 172.21.15.39:0/517787920 learned_addr learned my addr 172.21.15.39:0/517787920 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:32.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.090+0000 7fe9d3e00640 1 --2- 172.21.15.39:0/517787920 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9d40ccc60 0x7fe9d414da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:32.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.090+0000 7fe9d9a00640 1 --2- 172.21.15.39:0/517787920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9d415d830 0x7fe9d414df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:32.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.090+0000 7fe9d9000640 1 -- 172.21.15.39:0/517787920 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9d40ccc60 msgr2=0x7fe9d414da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:32.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.090+0000 7fe9d9000640 1 --2- 172.21.15.39:0/517787920 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9d40ccc60 0x7fe9d414da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:32.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.090+0000 7fe9d9000640 1 -- 172.21.15.39:0/517787920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9d415d830 msgr2=0x7fe9d414df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:32.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.090+0000 7fe9d9000640 1 --2- 172.21.15.39:0/517787920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9d415d830 0x7fe9d414df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:32.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.090+0000 7fe9d9000640 1 -- 172.21.15.39:0/517787920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d40c4120 con 0x7fe9d40b30a0 2024-09-21T07:45:32.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.090+0000 7fe9d9000640 1 --2- 172.21.15.39:0/517787920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9d40b30a0 0x7fe9d41f52b0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fe9c800d680 tx=0x7fe9c800db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:32.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.091+0000 7fe9d0c00640 1 -- 172.21.15.39:0/517787920 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8017070 con 0x7fe9d40b30a0 2024-09-21T07:45:32.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.091+0000 7fe9d0c00640 1 -- 172.21.15.39:0/517787920 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe9c8004310 con 0x7fe9d40b30a0 2024-09-21T07:45:32.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.091+0000 7fe9dba00640 1 -- 172.21.15.39:0/517787920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9d40c4400 con 0x7fe9d40b30a0 2024-09-21T07:45:32.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.091+0000 7fe9d0c00640 1 -- 172.21.15.39:0/517787920 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8005280 con 0x7fe9d40b30a0 2024-09-21T07:45:32.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.091+0000 7fe9dba00640 1 -- 172.21.15.39:0/517787920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9d40c6ac0 con 0x7fe9d40b30a0 2024-09-21T07:45:32.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.092+0000 7fe9dba00640 1 -- 172.21.15.39:0/517787920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9d40af840 con 0x7fe9d40b30a0 2024-09-21T07:45:32.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.093+0000 7fe9d0c00640 1 -- 172.21.15.39:0/517787920 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe9c8005420 con 0x7fe9d40b30a0 2024-09-21T07:45:32.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.094+0000 7fe9d0c00640 1 --2- 172.21.15.39:0/517787920 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe9b4078bc0 0x7fe9b407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:32.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.094+0000 7fe9d0c00640 1 -- 172.21.15.39:0/517787920 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6078+0+0 (secure 0 0 0) 0x7fe9c809aff0 con 0x7fe9d40b30a0 2024-09-21T07:45:32.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.094+0000 7fe9d3e00640 1 --2- 172.21.15.39:0/517787920 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe9b4078bc0 0x7fe9b407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:32.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.095+0000 7fe9d3e00640 1 --2- 172.21.15.39:0/517787920 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe9b4078bc0 0x7fe9b407b0d0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0033000 tx=0x7fe9c0004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:32.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.099+0000 7fe9d0c00640 1 -- 172.21.15.39:0/517787920 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe9c8020020 con 0x7fe9d40b30a0 2024-09-21T07:45:32.300 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:45:32.304 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:45:32.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.335+0000 7fe9dba00640 1 -- 172.21.15.39:0/517787920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9d415c9a0 con 0x7fe9d40b30a0 2024-09-21T07:45:32.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.336+0000 7fe9d0c00640 1 -- 172.21.15.39:0/517787920 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 174 v174) ==== 95+0+3922 (secure 0 0 0) 0x7fe9c8062d10 con 0x7fe9d40b30a0 2024-09-21T07:45:32.337 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:45:32.338 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":174,"btime":"2024-09-21T07:45:23:215238+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7127,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3876879654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3876879654},{"type":"v1","addr":"172.21.15.148:6836","nonce":3876879654}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":167},{"gid":7130,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3044588529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":3044588529},{"type":"v1","addr":"172.21.15.148:6837","nonce":3044588529}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":168},{"gid":8517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/3334092885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3334092885},{"type":"v1","addr":"172.21.15.39:6834","nonce":3334092885}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":168},{"gid":8715,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3507166267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3507166267},{"type":"v1","addr":"172.21.15.39:6835","nonce":3507166267}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":174}],"filesystems":[]} 2024-09-21T07:45:32.338 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 174 2024-09-21T07:45:32.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.338+0000 7fe9dba00640 1 -- 172.21.15.39:0/517787920 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe9b4078bc0 msgr2=0x7fe9b407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:32.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.338+0000 7fe9dba00640 1 --2- 172.21.15.39:0/517787920 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe9b4078bc0 0x7fe9b407b0d0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0033000 tx=0x7fe9c0004730 comp rx=0 tx=0).stop 2024-09-21T07:45:32.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.338+0000 7fe9dba00640 1 -- 172.21.15.39:0/517787920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9d40b30a0 msgr2=0x7fe9d41f52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:32.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.338+0000 7fe9dba00640 1 --2- 172.21.15.39:0/517787920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9d40b30a0 0x7fe9d41f52b0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fe9c800d680 tx=0x7fe9c800db40 comp rx=0 tx=0).stop 2024-09-21T07:45:32.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.338+0000 7fe9dba00640 1 -- 172.21.15.39:0/517787920 shutdown_connections 2024-09-21T07:45:32.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.338+0000 7fe9dba00640 1 --2- 172.21.15.39:0/517787920 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe9b4078bc0 0x7fe9b407b0d0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:32.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.338+0000 7fe9dba00640 1 --2- 172.21.15.39:0/517787920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9d415d830 0x7fe9d414df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:32.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.338+0000 7fe9dba00640 1 --2- 172.21.15.39:0/517787920 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9d40ccc60 0x7fe9d414da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:32.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.338+0000 7fe9dba00640 1 --2- 172.21.15.39:0/517787920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9d40b30a0 0x7fe9d41f52b0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:32.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.338+0000 7fe9dba00640 1 -- 172.21.15.39:0/517787920 >> 172.21.15.39:0/517787920 conn(0x7fe9d40b2460 msgr2=0x7fe9d40ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:32.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.338+0000 7fe9dba00640 1 -- 172.21.15.39:0/517787920 shutdown_connections 2024-09-21T07:45:32.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.338+0000 7fe9dba00640 1 -- 172.21.15.39:0/517787920 wait complete. 2024-09-21T07:45:32.362 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T07:45:32.362 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T07:45:32.363 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T07:45:32.363 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:45:32.363 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T07:45:32.363 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T07:45:32.363 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T07:45:32.363 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:45:32.363 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T07:45:32.364 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T07:45:32.364 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T07:45:32.364 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:45:32.364 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T07:45:32.364 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T07:45:32.364 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T07:45:32.364 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:45:32.387 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T07:45:32.387 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T07:45:32.388 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T07:45:32.388 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T07:45:32.423 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T07:45:32.424 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T07:45:32.427 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T07:45:32.427 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T07:45:32.431 INFO:tasks.ceph.mds.d:Started 2024-09-21T07:45:32.432 DEBUG:teuthology.parallel:result is None 2024-09-21T07:45:32.434 INFO:tasks.ceph.mds.b:Started 2024-09-21T07:45:32.434 DEBUG:teuthology.parallel:result is None 2024-09-21T07:45:32.471 INFO:tasks.ceph.mds.a:Started 2024-09-21T07:45:32.471 DEBUG:teuthology.parallel:result is None 2024-09-21T07:45:32.474 INFO:tasks.ceph.mds.c:Started 2024-09-21T07:45:32.474 DEBUG:teuthology.parallel:result is None 2024-09-21T07:45:32.474 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T07:45:32.626 INFO:tasks.ceph.mds.d.smithi148.stdout:starting mds.d at 2024-09-21T07:45:32.626 INFO:tasks.ceph.mds.b.smithi148.stdout:starting mds.b at 2024-09-21T07:45:32.672 INFO:tasks.ceph.mds.c.smithi039.stdout:starting mds.c at 2024-09-21T07:45:32.672 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-21T07:45:32.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.708+0000 7f543d400640 1 -- 172.21.15.39:0/156136423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f541c004db0 msgr2=0x7f541c005280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:32.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.708+0000 7f543d400640 1 --2- 172.21.15.39:0/156136423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f541c004db0 0x7f541c005280 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f5430009ed0 tx=0x7f541c005830 comp rx=0 tx=0).stop 2024-09-21T07:45:32.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.708+0000 7f5437200640 1 -- 172.21.15.39:0/156136423 reap_dead start 2024-09-21T07:45:32.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.708+0000 7f543d400640 1 -- 172.21.15.39:0/156136423 shutdown_connections 2024-09-21T07:45:32.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.708+0000 7f543d400640 1 -- 172.21.15.39:0/156136423 >> 172.21.15.39:0/156136423 conn(0x7f543806cfc0 msgr2=0x7f543806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:32.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.708+0000 7f543d400640 1 -- 172.21.15.39:0/156136423 shutdown_connections 2024-09-21T07:45:32.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.709+0000 7f543d400640 1 -- 172.21.15.39:0/156136423 wait complete. 2024-09-21T07:45:32.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.711+0000 7f543d400640 1 Processor -- start 2024-09-21T07:45:32.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.711+0000 7f543d400640 1 -- start start 2024-09-21T07:45:32.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.712+0000 7f543d400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f543815c6d0 0x7f54381f88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:32.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.712+0000 7f543d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54380aeae0 0x7f54381f59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:32.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.713+0000 7f543d400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54381f5f30 0x7f54380adbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:32.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.713+0000 7f543d400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5438151c60 con 0x7f54380aeae0 2024-09-21T07:45:32.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.713+0000 7f543d400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f54380d0270 con 0x7f54381f5f30 2024-09-21T07:45:32.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.713+0000 7f543d400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f5438151de0 con 0x7f543815c6d0 2024-09-21T07:45:32.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.713+0000 7f5435e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54380aeae0 0x7f54381f59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:32.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.713+0000 7f5435e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54380aeae0 0x7f54381f59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52084/0 (socket says 172.21.15.39:52084) 2024-09-21T07:45:32.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.713+0000 7f5435e00640 1 -- 172.21.15.39:0/3594164071 learned_addr learned my addr 172.21.15.39:0/3594164071 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:32.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.713+0000 7f5436800640 1 --2- 172.21.15.39:0/3594164071 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f543815c6d0 0x7f54381f88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:32.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.713+0000 7f5437200640 1 --2- 172.21.15.39:0/3594164071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54381f5f30 0x7f54380adbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:32.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.713+0000 7f5435e00640 1 -- 172.21.15.39:0/3594164071 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f543815c6d0 msgr2=0x7f54381f88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:32.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.713+0000 7f5435e00640 1 --2- 172.21.15.39:0/3594164071 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f543815c6d0 0x7f54381f88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:32.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.713+0000 7f5435e00640 1 -- 172.21.15.39:0/3594164071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54381f5f30 msgr2=0x7f54380adbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:32.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.713+0000 7f5435e00640 1 --2- 172.21.15.39:0/3594164071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54381f5f30 0x7f54380adbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:32.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.713+0000 7f5435e00640 1 -- 172.21.15.39:0/3594164071 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54381f8fb0 con 0x7f54380aeae0 2024-09-21T07:45:32.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.714+0000 7f5435e00640 1 --2- 172.21.15.39:0/3594164071 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54380aeae0 0x7f54381f59f0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f54240017e0 tx=0x7f5424003aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:32.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.714+0000 7f542ea00640 1 -- 172.21.15.39:0/3594164071 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5424015070 con 0x7f54380aeae0 2024-09-21T07:45:32.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.714+0000 7f542ea00640 1 -- 172.21.15.39:0/3594164071 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5424011900 con 0x7f54380aeae0 2024-09-21T07:45:32.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.714+0000 7f542ea00640 1 -- 172.21.15.39:0/3594164071 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5424011d50 con 0x7f54380aeae0 2024-09-21T07:45:32.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.714+0000 7f543d400640 1 -- 172.21.15.39:0/3594164071 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54381f9290 con 0x7f54380aeae0 2024-09-21T07:45:32.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.714+0000 7f543d400640 1 -- 172.21.15.39:0/3594164071 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54380aeef0 con 0x7f54380aeae0 2024-09-21T07:45:32.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.715+0000 7f543d400640 1 -- 172.21.15.39:0/3594164071 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54381f52e0 con 0x7f54380aeae0 2024-09-21T07:45:32.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.717+0000 7f542ea00640 1 -- 172.21.15.39:0/3594164071 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5424018070 con 0x7f54380aeae0 2024-09-21T07:45:32.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.717+0000 7f542ea00640 1 --2- 172.21.15.39:0/3594164071 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5418078b90 0x7f541807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:32.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.718+0000 7f542ea00640 1 -- 172.21.15.39:0/3594164071 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6078+0+0 (secure 0 0 0) 0x7f542409ba40 con 0x7f54380aeae0 2024-09-21T07:45:32.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.718+0000 7f5436800640 1 --2- 172.21.15.39:0/3594164071 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5418078b90 0x7f541807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:32.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.719+0000 7f5436800640 1 --2- 172.21.15.39:0/3594164071 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5418078b90 0x7f541807b0a0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f543806d440 tx=0x7f5428006b10 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:32.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.722+0000 7f542ea00640 1 -- 172.21.15.39:0/3594164071 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f54240637e0 con 0x7f54380aeae0 2024-09-21T07:45:32.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:32.932+0000 7f543d400640 1 -- 172.21.15.39:0/3594164071 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f54380cf750 con 0x7f54380aeae0 2024-09-21T07:45:33.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.619+0000 7f542ea00640 1 -- 172.21.15.39:0/3594164071 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v151) ==== 97+0+0 (secure 0 0 0) 0x7f542409be20 con 0x7f54380aeae0 2024-09-21T07:45:33.621 INFO:teuthology.orchestra.run.smithi039.stderr: removed all blocklist entries 2024-09-21T07:45:33.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.622+0000 7f543d400640 1 -- 172.21.15.39:0/3594164071 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5418078b90 msgr2=0x7f541807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:33.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.622+0000 7f543d400640 1 --2- 172.21.15.39:0/3594164071 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5418078b90 0x7f541807b0a0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f543806d440 tx=0x7f5428006b10 comp rx=0 tx=0).stop 2024-09-21T07:45:33.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.622+0000 7f543d400640 1 -- 172.21.15.39:0/3594164071 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54380aeae0 msgr2=0x7f54381f59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:33.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.622+0000 7f543d400640 1 --2- 172.21.15.39:0/3594164071 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54380aeae0 0x7f54381f59f0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f54240017e0 tx=0x7f5424003aa0 comp rx=0 tx=0).stop 2024-09-21T07:45:33.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.622+0000 7f543d400640 1 -- 172.21.15.39:0/3594164071 shutdown_connections 2024-09-21T07:45:33.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.622+0000 7f543d400640 1 --2- 172.21.15.39:0/3594164071 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5418078b90 0x7f541807b0a0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:33.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.622+0000 7f543d400640 1 --2- 172.21.15.39:0/3594164071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54381f5f30 0x7f54380adbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:33.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.622+0000 7f543d400640 1 --2- 172.21.15.39:0/3594164071 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54380aeae0 0x7f54381f59f0 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:33.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.622+0000 7f543d400640 1 --2- 172.21.15.39:0/3594164071 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f543815c6d0 0x7f54381f88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:33.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.622+0000 7f543d400640 1 -- 172.21.15.39:0/3594164071 >> 172.21.15.39:0/3594164071 conn(0x7f543806cfc0 msgr2=0x7f543815e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:33.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.622+0000 7f543d400640 1 -- 172.21.15.39:0/3594164071 shutdown_connections 2024-09-21T07:45:33.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.622+0000 7f543d400640 1 -- 172.21.15.39:0/3594164071 wait complete. 2024-09-21T07:45:33.655 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T07:45:33.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.881+0000 7fa45ca00640 1 -- 172.21.15.39:0/2021948358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa45815cc00 msgr2=0x7fa45815d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:33.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.881+0000 7fa45ca00640 1 --2- 172.21.15.39:0/2021948358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa45815cc00 0x7fa45815d010 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fa440009830 tx=0x7fa44002a5c0 comp rx=0 tx=0).stop 2024-09-21T07:45:33.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.881+0000 7fa45ca00640 1 -- 172.21.15.39:0/2021948358 shutdown_connections 2024-09-21T07:45:33.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.881+0000 7fa45ca00640 1 --2- 172.21.15.39:0/2021948358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa45815cc00 0x7fa45815d010 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:33.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.881+0000 7fa45ca00640 1 -- 172.21.15.39:0/2021948358 >> 172.21.15.39:0/2021948358 conn(0x7fa45806cfc0 msgr2=0x7fa45806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:33.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.881+0000 7fa45ca00640 1 -- 172.21.15.39:0/2021948358 shutdown_connections 2024-09-21T07:45:33.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.881+0000 7fa45ca00640 1 -- 172.21.15.39:0/2021948358 wait complete. 2024-09-21T07:45:33.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.883+0000 7fa45ca00640 1 Processor -- start 2024-09-21T07:45:33.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.883+0000 7fa45ca00640 1 -- start start 2024-09-21T07:45:33.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.885+0000 7fa45ca00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4580cfa20 0x7fa458157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:33.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.885+0000 7fa45ca00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4580d04b0 0x7fa4581e5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:33.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.885+0000 7fa45ca00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa45815cc00 0x7fa4581f3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:33.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.885+0000 7fa45ca00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa45806f080 con 0x7fa4580d04b0 2024-09-21T07:45:33.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.885+0000 7fa45ca00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa45806ef00 con 0x7fa45815cc00 2024-09-21T07:45:33.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.885+0000 7fa45ca00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa45806f200 con 0x7fa4580cfa20 2024-09-21T07:45:33.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.885+0000 7fa44de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4580d04b0 0x7fa4581e5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:33.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.885+0000 7fa44de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4580d04b0 0x7fa4581e5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52102/0 (socket says 172.21.15.39:52102) 2024-09-21T07:45:33.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.885+0000 7fa44de00640 1 -- 172.21.15.39:0/4059223341 learned_addr learned my addr 172.21.15.39:0/4059223341 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:33.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.885+0000 7fa456800640 1 --2- 172.21.15.39:0/4059223341 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4580cfa20 0x7fa458157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:33.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.885+0000 7fa457200640 1 --2- 172.21.15.39:0/4059223341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa45815cc00 0x7fa4581f3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:33.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.885+0000 7fa44de00640 1 -- 172.21.15.39:0/4059223341 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4580cfa20 msgr2=0x7fa458157230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:33.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.885+0000 7fa44de00640 1 --2- 172.21.15.39:0/4059223341 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4580cfa20 0x7fa458157230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:33.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.885+0000 7fa44de00640 1 -- 172.21.15.39:0/4059223341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa45815cc00 msgr2=0x7fa4581f3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:33.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.885+0000 7fa44de00640 1 --2- 172.21.15.39:0/4059223341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa45815cc00 0x7fa4581f3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:33.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.886+0000 7fa44de00640 1 -- 172.21.15.39:0/4059223341 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4581f4280 con 0x7fa4580d04b0 2024-09-21T07:45:33.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.886+0000 7fa457200640 1 --2- 172.21.15.39:0/4059223341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa45815cc00 0x7fa4581f3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:45:33.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.886+0000 7fa456800640 1 --2- 172.21.15.39:0/4059223341 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4580cfa20 0x7fa458157230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:45:33.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.886+0000 7fa44de00640 1 --2- 172.21.15.39:0/4059223341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4580d04b0 0x7fa4581e5050 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fa44002abb0 tx=0x7fa440033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:33.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.886+0000 7fa44f400640 1 -- 172.21.15.39:0/4059223341 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa440031b60 con 0x7fa4580d04b0 2024-09-21T07:45:33.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.886+0000 7fa44f400640 1 -- 172.21.15.39:0/4059223341 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa440031df0 con 0x7fa4580d04b0 2024-09-21T07:45:33.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.886+0000 7fa45ca00640 1 -- 172.21.15.39:0/4059223341 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4581f4560 con 0x7fa4580d04b0 2024-09-21T07:45:33.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.886+0000 7fa44f400640 1 -- 172.21.15.39:0/4059223341 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa440005280 con 0x7fa4580d04b0 2024-09-21T07:45:33.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.886+0000 7fa45ca00640 1 -- 172.21.15.39:0/4059223341 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa45815adf0 con 0x7fa4580d04b0 2024-09-21T07:45:33.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.887+0000 7fa45ca00640 1 -- 172.21.15.39:0/4059223341 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4581f3ff0 con 0x7fa4580d04b0 2024-09-21T07:45:33.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.889+0000 7fa44f400640 1 -- 172.21.15.39:0/4059223341 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa440005420 con 0x7fa4580d04b0 2024-09-21T07:45:33.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.889+0000 7fa44f400640 1 --2- 172.21.15.39:0/4059223341 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa434078b90 0x7fa43407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:33.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.890+0000 7fa44f400640 1 -- 172.21.15.39:0/4059223341 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5992+0+0 (secure 0 0 0) 0x7fa4400bc110 con 0x7fa4580d04b0 2024-09-21T07:45:33.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.890+0000 7fa456800640 1 --2- 172.21.15.39:0/4059223341 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa434078b90 0x7fa43407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:33.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.891+0000 7fa456800640 1 --2- 172.21.15.39:0/4059223341 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa434078b90 0x7fa43407b0a0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa4480012e0 tx=0x7fa448009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:33.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:33.894+0000 7fa44f400640 1 -- 172.21.15.39:0/4059223341 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa44003f030 con 0x7fa4580d04b0 2024-09-21T07:45:34.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.164+0000 7fa45ca00640 1 -- 172.21.15.39:0/4059223341 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fa458153a70 con 0x7fa4580d04b0 2024-09-21T07:45:34.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.165+0000 7fa44f400640 1 -- 172.21.15.39:0/4059223341 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5304 (secure 0 0 0) 0x7fa458153a70 con 0x7fa4580d04b0 2024-09-21T07:45:34.167 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:45:34.167 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:45:34.167 INFO:teuthology.orchestra.run.smithi039.stdout: "auth_dump": [ 2024-09-21T07:45:34.167 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:45:34.168 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.a", 2024-09-21T07:45:34.168 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5m+P6EERAABQqmlkrocSi1+hwc73Adrg==", 2024-09-21T07:45:34.168 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:45:34.168 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:45:34.168 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:45:34.168 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:45:34.168 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:45:34.168 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:45:34.168 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:45:34.168 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:45:34.169 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.b", 2024-09-21T07:45:34.169 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5m5/GeLRAAfE8fr5dbitQa89rNK9pZ2Q==", 2024-09-21T07:45:34.169 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:45:34.169 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:45:34.169 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:45:34.169 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:45:34.169 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:45:34.169 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:45:34.169 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:45:34.170 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:45:34.170 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.c", 2024-09-21T07:45:34.170 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5msbx6HxAALsF8ORvIoJpIAFZ5uxmOEw==", 2024-09-21T07:45:34.170 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:45:34.170 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:45:34.170 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:45:34.170 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:45:34.170 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:45:34.170 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:45:34.170 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:45:34.170 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:45:34.170 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.d", 2024-09-21T07:45:34.171 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mIrcBABAApKAB/7FkZ5glNTxlbkIsBw==", 2024-09-21T07:45:34.171 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:45:34.171 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:45:34.171 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:45:34.171 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:45:34.171 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:45:34.171 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:45:34.171 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:45:34.171 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:45:34.171 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.0", 2024-09-21T07:45:34.171 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB7du5msw81DhAAmVimqN4x9K3YpT0nbovppQ==", 2024-09-21T07:45:34.171 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:45:34.171 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:45:34.172 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:45:34.172 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:45:34.172 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:45:34.172 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:45:34.172 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:45:34.172 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.1", 2024-09-21T07:45:34.172 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB+du5mdnCIOhAAtVB4ThehQwb0zRriyKcaxA==", 2024-09-21T07:45:34.172 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:45:34.172 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:45:34.172 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:45:34.172 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:45:34.173 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:45:34.173 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:45:34.173 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:45:34.173 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.2", 2024-09-21T07:45:34.173 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCCdu5mTfWxDhAAadJCbPN5xBx/f9L0v2yTdg==", 2024-09-21T07:45:34.173 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:45:34.173 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:45:34.173 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:45:34.173 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:45:34.173 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:45:34.173 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:45:34.173 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:45:34.173 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.3", 2024-09-21T07:45:34.174 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCFdu5mL1E2HxAAyOhY97uUXEw5L1CcDeONVA==", 2024-09-21T07:45:34.174 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:45:34.174 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:45:34.174 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:45:34.174 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:45:34.174 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:45:34.174 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:45:34.174 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:45:34.174 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.4", 2024-09-21T07:45:34.174 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCLdu5m6PY/IhAAgvcTgm+6uWA7lIG4tcTIEg==", 2024-09-21T07:45:34.174 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:45:34.174 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:45:34.174 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:45:34.175 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:45:34.175 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:45:34.175 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:45:34.175 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:45:34.175 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.5", 2024-09-21T07:45:34.175 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCOdu5mE/wlMxAA3JBg4xH+N/PPC1Uv2ZVrYQ==", 2024-09-21T07:45:34.175 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:45:34.175 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:45:34.175 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:45:34.175 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:45:34.175 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:45:34.175 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:45:34.176 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:45:34.176 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.6", 2024-09-21T07:45:34.176 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCSdu5mC3TgBxAAwsArjeIdFcApY3/1OJffEg==", 2024-09-21T07:45:34.176 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:45:34.176 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:45:34.176 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:45:34.176 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:45:34.176 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:45:34.176 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:45:34.176 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:45:34.176 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.7", 2024-09-21T07:45:34.176 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCVdu5mSWHWFBAA9qPHWRdsvrB74inrp1lj4Q==", 2024-09-21T07:45:34.177 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:45:34.177 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:45:34.177 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:45:34.177 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:45:34.177 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:45:34.177 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:45:34.177 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:45:34.177 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.0", 2024-09-21T07:45:34.177 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mGfb1DBAA/aSMKR5or3xy53rB/iUjpA==", 2024-09-21T07:45:34.177 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:45:34.177 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:45:34.177 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow r", 2024-09-21T07:45:34.177 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T07:45:34.178 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:45:34.178 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:45:34.178 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:45:34.178 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.1", 2024-09-21T07:45:34.178 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mEkciIhAAid+bxJEr4P7vMnZ42PAy5w==", 2024-09-21T07:45:34.178 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:45:34.178 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:45:34.178 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow r", 2024-09-21T07:45:34.178 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow rw", 2024-09-21T07:45:34.178 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rwx" 2024-09-21T07:45:34.178 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:45:34.178 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:45:34.179 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:45:34.179 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.2", 2024-09-21T07:45:34.179 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mSu2oFxAAgjy2m/wWKh6xaoh+YDoU1Q==", 2024-09-21T07:45:34.179 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:45:34.179 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:45:34.179 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow r", 2024-09-21T07:45:34.179 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow rw", 2024-09-21T07:45:34.179 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rwx" 2024-09-21T07:45:34.179 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:45:34.179 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:45:34.179 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:45:34.179 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.3", 2024-09-21T07:45:34.180 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mCHy0LBAAZneF2jUWgfuyGehZFbOxDg==", 2024-09-21T07:45:34.180 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:45:34.180 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:45:34.180 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow r", 2024-09-21T07:45:34.180 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow rw", 2024-09-21T07:45:34.180 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rwx" 2024-09-21T07:45:34.180 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:45:34.180 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:45:34.180 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:45:34.180 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.admin", 2024-09-21T07:45:34.180 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBydu5m9LInERAAAlDpqb94f2Qb9VYJJwTBcA==", 2024-09-21T07:45:34.180 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:45:34.181 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow *", 2024-09-21T07:45:34.181 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:45:34.181 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:45:34.181 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:45:34.181 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:45:34.181 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:45:34.181 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:45:34.181 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mgr.x", 2024-09-21T07:45:34.181 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5mA2GRBhAAkyCKdgH6laHy8xNt080dXg==", 2024-09-21T07:45:34.181 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:45:34.181 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow *", 2024-09-21T07:45:34.181 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile mgr", 2024-09-21T07:45:34.181 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:45:34.182 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:45:34.182 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:45:34.182 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:45:34.182 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mgr.y", 2024-09-21T07:45:34.182 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBydu5mr+8aNxAA7TFYNms2WsXUfa4ITTJOHw==", 2024-09-21T07:45:34.182 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:45:34.182 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow *", 2024-09-21T07:45:34.182 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile mgr", 2024-09-21T07:45:34.182 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:45:34.182 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:45:34.182 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:45:34.182 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-21T07:45:34.183 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:45:34.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.167+0000 7fa45ca00640 1 -- 172.21.15.39:0/4059223341 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa434078b90 msgr2=0x7fa43407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:34.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.167+0000 7fa45ca00640 1 --2- 172.21.15.39:0/4059223341 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa434078b90 0x7fa43407b0a0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa4480012e0 tx=0x7fa448009040 comp rx=0 tx=0).stop 2024-09-21T07:45:34.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.167+0000 7fa45ca00640 1 -- 172.21.15.39:0/4059223341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4580d04b0 msgr2=0x7fa4581e5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:34.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.167+0000 7fa45ca00640 1 --2- 172.21.15.39:0/4059223341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4580d04b0 0x7fa4581e5050 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fa44002abb0 tx=0x7fa440033c10 comp rx=0 tx=0).stop 2024-09-21T07:45:34.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.167+0000 7fa45ca00640 1 -- 172.21.15.39:0/4059223341 shutdown_connections 2024-09-21T07:45:34.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.167+0000 7fa45ca00640 1 --2- 172.21.15.39:0/4059223341 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa434078b90 0x7fa43407b0a0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:34.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.167+0000 7fa45ca00640 1 --2- 172.21.15.39:0/4059223341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa45815cc00 0x7fa4581f3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:34.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.167+0000 7fa45ca00640 1 --2- 172.21.15.39:0/4059223341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4580d04b0 0x7fa4581e5050 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:34.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.167+0000 7fa45ca00640 1 --2- 172.21.15.39:0/4059223341 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4580cfa20 0x7fa458157230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:34.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.167+0000 7fa45ca00640 1 -- 172.21.15.39:0/4059223341 >> 172.21.15.39:0/4059223341 conn(0x7fa45806cfc0 msgr2=0x7fa4580b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:34.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.168+0000 7fa45ca00640 1 -- 172.21.15.39:0/4059223341 shutdown_connections 2024-09-21T07:45:34.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.168+0000 7fa45ca00640 1 -- 172.21.15.39:0/4059223341 wait complete. 2024-09-21T07:45:34.192 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T07:45:34.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.419+0000 7fb48a400640 1 -- 172.21.15.39:0/1001928800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4840b3b60 msgr2=0x7fb4840b3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:34.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.419+0000 7fb48a400640 1 --2- 172.21.15.39:0/1001928800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4840b3b60 0x7fb4840b3f70 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fb4840d5560 tx=0x7fb47c02a8b0 comp rx=0 tx=0).stop 2024-09-21T07:45:34.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.419+0000 7fb48a400640 1 -- 172.21.15.39:0/1001928800 shutdown_connections 2024-09-21T07:45:34.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.419+0000 7fb48a400640 1 --2- 172.21.15.39:0/1001928800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4840b3b60 0x7fb4840b3f70 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:34.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.419+0000 7fb48a400640 1 --2- 172.21.15.39:0/1001928800 >> v2:172.21.15.148:3300/0 conn(0x7fb4840d1580 0x7fb4840d1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:34.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.419+0000 7fb48a400640 1 -- 172.21.15.39:0/1001928800 >> 172.21.15.39:0/1001928800 conn(0x7fb48409d7b0 msgr2=0x7fb4840d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:34.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.419+0000 7fb48a400640 1 -- 172.21.15.39:0/1001928800 shutdown_connections 2024-09-21T07:45:34.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.419+0000 7fb48a400640 1 -- 172.21.15.39:0/1001928800 wait complete. 2024-09-21T07:45:34.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.421+0000 7fb48a400640 1 Processor -- start 2024-09-21T07:45:34.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.421+0000 7fb48a400640 1 -- start start 2024-09-21T07:45:34.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.423+0000 7fb48a400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4840b3b60 0x7fb4840c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:34.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.423+0000 7fb48a400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4840b5a90 0x7fb4841f8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:34.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.423+0000 7fb48a400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4841f8970 0x7fb48414a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:34.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.423+0000 7fb48a400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb4840b4630 con 0x7fb4841f8970 2024-09-21T07:45:34.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.423+0000 7fb48a400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb4840b44b0 con 0x7fb4840b3b60 2024-09-21T07:45:34.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.423+0000 7fb48a400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb4840b47b0 con 0x7fb4840b5a90 2024-09-21T07:45:34.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.423+0000 7fb489a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4841f8970 0x7fb48414a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:34.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.423+0000 7fb489a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4841f8970 0x7fb48414a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52122/0 (socket says 172.21.15.39:52122) 2024-09-21T07:45:34.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.423+0000 7fb489a00640 1 -- 172.21.15.39:0/139124041 learned_addr learned my addr 172.21.15.39:0/139124041 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:34.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.423+0000 7fb489000640 1 --2- 172.21.15.39:0/139124041 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4840b3b60 0x7fb4840c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:34.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.423+0000 7fb483e00640 1 --2- 172.21.15.39:0/139124041 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4840b5a90 0x7fb4841f8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:34.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.424+0000 7fb489a00640 1 -- 172.21.15.39:0/139124041 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4840b5a90 msgr2=0x7fb4841f8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:34.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.424+0000 7fb489a00640 1 --2- 172.21.15.39:0/139124041 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4840b5a90 0x7fb4841f8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:34.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.424+0000 7fb489a00640 1 -- 172.21.15.39:0/139124041 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4840b3b60 msgr2=0x7fb4840c84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:34.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.424+0000 7fb489a00640 1 --2- 172.21.15.39:0/139124041 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4840b3b60 0x7fb4840c84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:34.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.424+0000 7fb489a00640 1 -- 172.21.15.39:0/139124041 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4841f8d80 con 0x7fb4841f8970 2024-09-21T07:45:34.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.424+0000 7fb489a00640 1 --2- 172.21.15.39:0/139124041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4841f8970 0x7fb48414a650 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fb47c02aea0 tx=0x7fb47c031760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:34.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.424+0000 7fb480c00640 1 -- 172.21.15.39:0/139124041 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c031990 con 0x7fb4841f8970 2024-09-21T07:45:34.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.424+0000 7fb480c00640 1 -- 172.21.15.39:0/139124041 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb47c031c20 con 0x7fb4841f8970 2024-09-21T07:45:34.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.424+0000 7fb48a400640 1 -- 172.21.15.39:0/139124041 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4840cd1f0 con 0x7fb4841f8970 2024-09-21T07:45:34.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.424+0000 7fb480c00640 1 -- 172.21.15.39:0/139124041 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c0330d0 con 0x7fb4841f8970 2024-09-21T07:45:34.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.424+0000 7fb48a400640 1 -- 172.21.15.39:0/139124041 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4840cb7f0 con 0x7fb4841f8970 2024-09-21T07:45:34.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.426+0000 7fb48a400640 1 -- 172.21.15.39:0/139124041 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4840c8ad0 con 0x7fb4841f8970 2024-09-21T07:45:34.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.427+0000 7fb480c00640 1 -- 172.21.15.39:0/139124041 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb47c033270 con 0x7fb4841f8970 2024-09-21T07:45:34.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.428+0000 7fb480c00640 1 --2- 172.21.15.39:0/139124041 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb464078bc0 0x7fb46407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:34.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.428+0000 7fb480c00640 1 -- 172.21.15.39:0/139124041 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5992+0+0 (secure 0 0 0) 0x7fb47c0bc290 con 0x7fb4841f8970 2024-09-21T07:45:34.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.428+0000 7fb489000640 1 --2- 172.21.15.39:0/139124041 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb464078bc0 0x7fb46407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:34.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.429+0000 7fb489000640 1 --2- 172.21.15.39:0/139124041 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb464078bc0 0x7fb46407b0d0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb4780016f0 tx=0x7fb478009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:34.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.432+0000 7fb480c00640 1 -- 172.21.15.39:0/139124041 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb4841f8d80 con 0x7fb4841f8970 2024-09-21T07:45:34.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.637+0000 7fb48a400640 1 -- 172.21.15.39:0/139124041 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb4840ce750 con 0x7fb4841f8970 2024-09-21T07:45:34.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.637+0000 7fb480c00640 1 -- 172.21.15.39:0/139124041 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v153) ==== 84+0+287 (secure 0 0 0) 0x7fb47c03f030 con 0x7fb4841f8970 2024-09-21T07:45:34.639 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:45:34.640 INFO:teuthology.orchestra.run.smithi039.stdout:[ 2024-09-21T07:45:34.640 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:45:34.640 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 1, 2024-09-21T07:45:34.640 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": ".mgr" 2024-09-21T07:45:34.640 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:45:34.640 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:45:34.640 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 6, 2024-09-21T07:45:34.640 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": "snap-test-pool" 2024-09-21T07:45:34.640 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:45:34.640 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:45:34.640 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 11, 2024-09-21T07:45:34.640 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": "test_data_pool" 2024-09-21T07:45:34.640 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:45:34.640 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:45:34.641 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 12, 2024-09-21T07:45:34.641 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": "test_metadata_pool" 2024-09-21T07:45:34.641 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:45:34.641 INFO:teuthology.orchestra.run.smithi039.stdout:] 2024-09-21T07:45:34.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.640+0000 7fb48a400640 1 -- 172.21.15.39:0/139124041 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb464078bc0 msgr2=0x7fb46407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:34.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.640+0000 7fb48a400640 1 --2- 172.21.15.39:0/139124041 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb464078bc0 0x7fb46407b0d0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb4780016f0 tx=0x7fb478009040 comp rx=0 tx=0).stop 2024-09-21T07:45:34.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.640+0000 7fb48a400640 1 -- 172.21.15.39:0/139124041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4841f8970 msgr2=0x7fb48414a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:34.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.640+0000 7fb48a400640 1 --2- 172.21.15.39:0/139124041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4841f8970 0x7fb48414a650 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fb47c02aea0 tx=0x7fb47c031760 comp rx=0 tx=0).stop 2024-09-21T07:45:34.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.640+0000 7fb48a400640 1 -- 172.21.15.39:0/139124041 shutdown_connections 2024-09-21T07:45:34.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.640+0000 7fb48a400640 1 --2- 172.21.15.39:0/139124041 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb464078bc0 0x7fb46407b0d0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:34.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.640+0000 7fb48a400640 1 --2- 172.21.15.39:0/139124041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4841f8970 0x7fb48414a650 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:34.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.640+0000 7fb48a400640 1 --2- 172.21.15.39:0/139124041 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4840b5a90 0x7fb4841f8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:34.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.640+0000 7fb48a400640 1 --2- 172.21.15.39:0/139124041 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4840b3b60 0x7fb4840c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:34.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.640+0000 7fb48a400640 1 -- 172.21.15.39:0/139124041 >> 172.21.15.39:0/139124041 conn(0x7fb48409d7b0 msgr2=0x7fb48406f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:34.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.640+0000 7fb48a400640 1 -- 172.21.15.39:0/139124041 shutdown_connections 2024-09-21T07:45:34.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.640+0000 7fb48a400640 1 -- 172.21.15.39:0/139124041 wait complete. 2024-09-21T07:45:34.664 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T07:45:34.664 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T07:45:34.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.893+0000 7fc2b3200640 1 -- 172.21.15.39:0/3065158301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2b4160340 msgr2=0x7fc2b406ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:34.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.893+0000 7fc2b3200640 1 --2- 172.21.15.39:0/3065158301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2b4160340 0x7fc2b406ef70 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0009860 tx=0x7fc2a002a5a0 comp rx=0 tx=0).stop 2024-09-21T07:45:34.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.893+0000 7fc2b3200640 1 -- 172.21.15.39:0/3065158301 shutdown_connections 2024-09-21T07:45:34.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.893+0000 7fc2b3200640 1 --2- 172.21.15.39:0/3065158301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2b4160340 0x7fc2b406ef70 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:34.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.893+0000 7fc2b3200640 1 --2- 172.21.15.39:0/3065158301 >> v2:172.21.15.148:3300/0 conn(0x7fc2b41539f0 0x7fc2b40b4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:34.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.893+0000 7fc2b3200640 1 -- 172.21.15.39:0/3065158301 >> 172.21.15.39:0/3065158301 conn(0x7fc2b406d660 msgr2=0x7fc2b40b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:34.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.893+0000 7fc2b3200640 1 -- 172.21.15.39:0/3065158301 shutdown_connections 2024-09-21T07:45:34.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.893+0000 7fc2b3200640 1 -- 172.21.15.39:0/3065158301 wait complete. 2024-09-21T07:45:34.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.895+0000 7fc2b3200640 1 Processor -- start 2024-09-21T07:45:34.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.895+0000 7fc2b3200640 1 -- start start 2024-09-21T07:45:34.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.897+0000 7fc2b3200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2b406f4b0 0x7fc2b41fdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:34.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.897+0000 7fc2b3200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2b41539f0 0x7fc2b4074750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:34.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.897+0000 7fc2b3200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc2b4160340 0x7fc2b41f0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:34.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.897+0000 7fc2b3200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b40ad670 con 0x7fc2b41539f0 2024-09-21T07:45:34.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.897+0000 7fc2b3200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b40ad4f0 con 0x7fc2b406f4b0 2024-09-21T07:45:34.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.897+0000 7fc2b3200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc2b40ad7f0 con 0x7fc2b4160340 2024-09-21T07:45:34.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.897+0000 7fc2b1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2b41539f0 0x7fc2b4074750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:34.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.897+0000 7fc2b1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2b41539f0 0x7fc2b4074750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52146/0 (socket says 172.21.15.39:52146) 2024-09-21T07:45:34.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.897+0000 7fc2b1e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2b406f4b0 0x7fc2b41fdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:34.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.897+0000 7fc2b1400640 1 -- 172.21.15.39:0/2302647305 learned_addr learned my addr 172.21.15.39:0/2302647305 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:34.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.897+0000 7fc2b2800640 1 --2- 172.21.15.39:0/2302647305 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc2b4160340 0x7fc2b41f0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:34.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.898+0000 7fc2b1400640 1 -- 172.21.15.39:0/2302647305 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc2b4160340 msgr2=0x7fc2b41f0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:34.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.898+0000 7fc2b1400640 1 --2- 172.21.15.39:0/2302647305 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc2b4160340 0x7fc2b41f0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:34.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.898+0000 7fc2b1400640 1 -- 172.21.15.39:0/2302647305 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2b406f4b0 msgr2=0x7fc2b41fdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:34.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.898+0000 7fc2b1400640 1 --2- 172.21.15.39:0/2302647305 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2b406f4b0 0x7fc2b41fdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:34.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.898+0000 7fc2b1400640 1 -- 172.21.15.39:0/2302647305 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2b41fa430 con 0x7fc2b41539f0 2024-09-21T07:45:34.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.898+0000 7fc2b1400640 1 --2- 172.21.15.39:0/2302647305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2b41539f0 0x7fc2b4074750 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0009980 tx=0x7fc2a0033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:34.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.898+0000 7fc2aa000640 1 -- 172.21.15.39:0/2302647305 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0031b50 con 0x7fc2b41539f0 2024-09-21T07:45:34.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.898+0000 7fc2aa000640 1 -- 172.21.15.39:0/2302647305 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2a0031de0 con 0x7fc2b41539f0 2024-09-21T07:45:34.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.898+0000 7fc2b3200640 1 -- 172.21.15.39:0/2302647305 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2b41fa710 con 0x7fc2b41539f0 2024-09-21T07:45:34.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.898+0000 7fc2aa000640 1 -- 172.21.15.39:0/2302647305 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0005280 con 0x7fc2b41539f0 2024-09-21T07:45:34.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.899+0000 7fc2b3200640 1 -- 172.21.15.39:0/2302647305 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2b4155a90 con 0x7fc2b41539f0 2024-09-21T07:45:34.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.900+0000 7fc2b3200640 1 -- 172.21.15.39:0/2302647305 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2b41fa1a0 con 0x7fc2b41539f0 2024-09-21T07:45:34.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.901+0000 7fc2aa000640 1 -- 172.21.15.39:0/2302647305 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc2a0005420 con 0x7fc2b41539f0 2024-09-21T07:45:34.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.902+0000 7fc2aa000640 1 --2- 172.21.15.39:0/2302647305 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc294078bc0 0x7fc29407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:34.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.902+0000 7fc2aa000640 1 -- 172.21.15.39:0/2302647305 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5992+0+0 (secure 0 0 0) 0x7fc2a00bc1f0 con 0x7fc2b41539f0 2024-09-21T07:45:34.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.902+0000 7fc2b1e00640 1 --2- 172.21.15.39:0/2302647305 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc294078bc0 0x7fc29407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:34.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.903+0000 7fc2b1e00640 1 --2- 172.21.15.39:0/2302647305 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc294078bc0 0x7fc29407b0d0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009b30 tx=0x7fc2a4005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:34.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:34.906+0000 7fc2aa000640 1 -- 172.21.15.39:0/2302647305 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc2a003f030 con 0x7fc2b41539f0 2024-09-21T07:45:35.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.101+0000 7fc2b3200640 1 -- 172.21.15.39:0/2302647305 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc2b40ae1f0 con 0x7fc2b41539f0 2024-09-21T07:45:35.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.647+0000 7fc2aa000640 1 -- 172.21.15.39:0/2302647305 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v154) ==== 138+0+0 (secure 0 0 0) 0x7fc2b40ae1f0 con 0x7fc2b41539f0 2024-09-21T07:45:35.649 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' created 2024-09-21T07:45:35.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.649+0000 7fc2b3200640 1 -- 172.21.15.39:0/2302647305 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc294078bc0 msgr2=0x7fc29407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:35.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.649+0000 7fc2b3200640 1 --2- 172.21.15.39:0/2302647305 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc294078bc0 0x7fc29407b0d0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009b30 tx=0x7fc2a4005f90 comp rx=0 tx=0).stop 2024-09-21T07:45:35.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.650+0000 7fc2b3200640 1 -- 172.21.15.39:0/2302647305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2b41539f0 msgr2=0x7fc2b4074750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:35.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.650+0000 7fc2b3200640 1 --2- 172.21.15.39:0/2302647305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2b41539f0 0x7fc2b4074750 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0009980 tx=0x7fc2a0033ba0 comp rx=0 tx=0).stop 2024-09-21T07:45:35.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.650+0000 7fc2b3200640 1 -- 172.21.15.39:0/2302647305 shutdown_connections 2024-09-21T07:45:35.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.650+0000 7fc2b3200640 1 --2- 172.21.15.39:0/2302647305 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc294078bc0 0x7fc29407b0d0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:35.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.650+0000 7fc2b3200640 1 --2- 172.21.15.39:0/2302647305 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc2b4160340 0x7fc2b41f0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:35.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.650+0000 7fc2b3200640 1 --2- 172.21.15.39:0/2302647305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2b41539f0 0x7fc2b4074750 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:35.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.650+0000 7fc2b3200640 1 --2- 172.21.15.39:0/2302647305 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2b406f4b0 0x7fc2b41fdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:35.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.650+0000 7fc2b3200640 1 -- 172.21.15.39:0/2302647305 >> 172.21.15.39:0/2302647305 conn(0x7fc2b406d660 msgr2=0x7fc2b40b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:35.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.650+0000 7fc2b3200640 1 -- 172.21.15.39:0/2302647305 shutdown_connections 2024-09-21T07:45:35.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.650+0000 7fc2b3200640 1 -- 172.21.15.39:0/2302647305 wait complete. 2024-09-21T07:45:35.683 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T07:45:35.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.918+0000 7f10e3200640 1 -- 172.21.15.39:0/2432874866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10e4160250 msgr2=0x7f10e415d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:35.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.918+0000 7f10e3200640 1 --2- 172.21.15.39:0/2432874866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10e4160250 0x7f10e415d570 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f10e4161040 tx=0x7f10d402a540 comp rx=0 tx=0).stop 2024-09-21T07:45:35.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.918+0000 7f10e3200640 1 -- 172.21.15.39:0/2432874866 shutdown_connections 2024-09-21T07:45:35.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.918+0000 7f10e3200640 1 --2- 172.21.15.39:0/2432874866 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f10e40cfc50 0x7f10e40d0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:35.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.918+0000 7f10e3200640 1 --2- 172.21.15.39:0/2432874866 >> v2:172.21.15.148:3300/0 conn(0x7f10e4160660 0x7f10e40cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:35.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.918+0000 7f10e3200640 1 --2- 172.21.15.39:0/2432874866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10e4160250 0x7f10e415d570 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:35.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.918+0000 7f10e3200640 1 -- 172.21.15.39:0/2432874866 >> 172.21.15.39:0/2432874866 conn(0x7f10e406cfc0 msgr2=0x7f10e406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:35.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.918+0000 7f10e3200640 1 -- 172.21.15.39:0/2432874866 shutdown_connections 2024-09-21T07:45:35.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.918+0000 7f10e3200640 1 -- 172.21.15.39:0/2432874866 wait complete. 2024-09-21T07:45:35.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.920+0000 7f10e3200640 1 Processor -- start 2024-09-21T07:45:35.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.920+0000 7f10e3200640 1 -- start start 2024-09-21T07:45:35.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.922+0000 7f10e3200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f10e40cfc50 0x7f10e41f60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:35.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.922+0000 7f10e3200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10e4160250 0x7f10e40ad910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:35.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.922+0000 7f10e3200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10e4160660 0x7f10e40b3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:35.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.922+0000 7f10e3200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f10e40ccbb0 con 0x7f10e4160250 2024-09-21T07:45:35.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.922+0000 7f10e3200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f10e40cca30 con 0x7f10e4160660 2024-09-21T07:45:35.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.922+0000 7f10e3200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f10e40ccd30 con 0x7f10e40cfc50 2024-09-21T07:45:35.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.922+0000 7f10e1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10e4160250 0x7f10e40ad910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:35.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.922+0000 7f10e1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10e4160250 0x7f10e40ad910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52156/0 (socket says 172.21.15.39:52156) 2024-09-21T07:45:35.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.922+0000 7f10e1400640 1 -- 172.21.15.39:0/3114917205 learned_addr learned my addr 172.21.15.39:0/3114917205 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:35.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.922+0000 7f10e1e00640 1 --2- 172.21.15.39:0/3114917205 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f10e40cfc50 0x7f10e41f60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:35.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.922+0000 7f10e2800640 1 --2- 172.21.15.39:0/3114917205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10e4160660 0x7f10e40b3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:35.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.922+0000 7f10e1400640 1 -- 172.21.15.39:0/3114917205 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f10e40cfc50 msgr2=0x7f10e41f60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:35.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.922+0000 7f10e1400640 1 --2- 172.21.15.39:0/3114917205 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f10e40cfc50 0x7f10e41f60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:35.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.922+0000 7f10e1400640 1 -- 172.21.15.39:0/3114917205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10e4160660 msgr2=0x7f10e40b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:35.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.922+0000 7f10e1400640 1 --2- 172.21.15.39:0/3114917205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10e4160660 0x7f10e40b3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:35.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.922+0000 7f10e1400640 1 -- 172.21.15.39:0/3114917205 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e40ae520 con 0x7f10e4160250 2024-09-21T07:45:35.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.922+0000 7f10e1e00640 1 --2- 172.21.15.39:0/3114917205 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f10e40cfc50 0x7f10e41f60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:45:35.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.923+0000 7f10e1400640 1 --2- 172.21.15.39:0/3114917205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10e4160250 0x7f10e40ad910 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f10d000d750 tx=0x7f10d000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:35.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.923+0000 7f10da000640 1 -- 172.21.15.39:0/3114917205 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d0017070 con 0x7f10e4160250 2024-09-21T07:45:35.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.923+0000 7f10da000640 1 -- 172.21.15.39:0/3114917205 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10d0004310 con 0x7f10e4160250 2024-09-21T07:45:35.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.923+0000 7f10da000640 1 -- 172.21.15.39:0/3114917205 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d000de60 con 0x7f10e4160250 2024-09-21T07:45:35.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.923+0000 7f10e3200640 1 -- 172.21.15.39:0/3114917205 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10e40ae800 con 0x7f10e4160250 2024-09-21T07:45:35.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.923+0000 7f10e3200640 1 -- 172.21.15.39:0/3114917205 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10e40aedc0 con 0x7f10e4160250 2024-09-21T07:45:35.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.924+0000 7f10e3200640 1 -- 172.21.15.39:0/3114917205 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10e40ae290 con 0x7f10e4160250 2024-09-21T07:45:35.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.926+0000 7f10da000640 1 -- 172.21.15.39:0/3114917205 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f10d0009c30 con 0x7f10e4160250 2024-09-21T07:45:35.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.927+0000 7f10da000640 1 --2- 172.21.15.39:0/3114917205 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f10c4078bc0 0x7f10c407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:35.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.927+0000 7f10da000640 1 -- 172.21.15.39:0/3114917205 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 6362+0+0 (secure 0 0 0) 0x7f10d009b9e0 con 0x7f10e4160250 2024-09-21T07:45:35.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.927+0000 7f10e1e00640 1 --2- 172.21.15.39:0/3114917205 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f10c4078bc0 0x7f10c407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:35.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.928+0000 7f10e1e00640 1 --2- 172.21.15.39:0/3114917205 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f10c4078bc0 0x7f10c407b0d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f10d4033000 tx=0x7f10d4004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:35.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:35.930+0000 7f10da000640 1 -- 172.21.15.39:0/3114917205 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f10d0020020 con 0x7f10e4160250 2024-09-21T07:45:36.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.128+0000 7f10e3200640 1 -- 172.21.15.39:0/3114917205 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f10e40cda10 con 0x7f10e4160250 2024-09-21T07:45:36.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.659+0000 7f10da000640 1 -- 172.21.15.39:0/3114917205 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v155) ==== 170+0+0 (secure 0 0 0) 0x7f10d0063c00 con 0x7f10e4160250 2024-09-21T07:45:36.661 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' created 2024-09-21T07:45:36.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.661+0000 7f10e3200640 1 -- 172.21.15.39:0/3114917205 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f10c4078bc0 msgr2=0x7f10c407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:36.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.661+0000 7f10e3200640 1 --2- 172.21.15.39:0/3114917205 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f10c4078bc0 0x7f10c407b0d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f10d4033000 tx=0x7f10d4004730 comp rx=0 tx=0).stop 2024-09-21T07:45:36.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.661+0000 7f10e3200640 1 -- 172.21.15.39:0/3114917205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10e4160250 msgr2=0x7f10e40ad910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:36.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.661+0000 7f10e3200640 1 --2- 172.21.15.39:0/3114917205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10e4160250 0x7f10e40ad910 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f10d000d750 tx=0x7f10d000dc10 comp rx=0 tx=0).stop 2024-09-21T07:45:36.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.661+0000 7f10e3200640 1 -- 172.21.15.39:0/3114917205 shutdown_connections 2024-09-21T07:45:36.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.661+0000 7f10e3200640 1 --2- 172.21.15.39:0/3114917205 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f10c4078bc0 0x7f10c407b0d0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:36.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.661+0000 7f10e3200640 1 --2- 172.21.15.39:0/3114917205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10e4160660 0x7f10e40b3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:36.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.661+0000 7f10e3200640 1 --2- 172.21.15.39:0/3114917205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10e4160250 0x7f10e40ad910 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:36.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.661+0000 7f10e3200640 1 --2- 172.21.15.39:0/3114917205 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f10e40cfc50 0x7f10e41f60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:36.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.661+0000 7f10e3200640 1 -- 172.21.15.39:0/3114917205 >> 172.21.15.39:0/3114917205 conn(0x7f10e406cfc0 msgr2=0x7f10e415f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:36.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.662+0000 7f10e3200640 1 -- 172.21.15.39:0/3114917205 shutdown_connections 2024-09-21T07:45:36.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.662+0000 7f10e3200640 1 -- 172.21.15.39:0/3114917205 wait complete. 2024-09-21T07:45:36.704 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T07:45:36.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.934+0000 7f98e3c00640 1 -- 172.21.15.39:0/1365303103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98dc0cf750 msgr2=0x7f98dc0cfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:36.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.934+0000 7f98e3c00640 1 --2- 172.21.15.39:0/1365303103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98dc0cf750 0x7f98dc0cfb60 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f98d400afd0 tx=0x7f98d402b480 comp rx=0 tx=0).stop 2024-09-21T07:45:36.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.934+0000 7f98e3c00640 1 -- 172.21.15.39:0/1365303103 shutdown_connections 2024-09-21T07:45:36.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.934+0000 7f98e3c00640 1 --2- 172.21.15.39:0/1365303103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98dc0cf750 0x7f98dc0cfb60 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:36.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.934+0000 7f98e3c00640 1 --2- 172.21.15.39:0/1365303103 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f98dc151640 0x7f98dc151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:36.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.934+0000 7f98e3c00640 1 -- 172.21.15.39:0/1365303103 >> 172.21.15.39:0/1365303103 conn(0x7f98dc06cfc0 msgr2=0x7f98dc06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:36.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.934+0000 7f98e3c00640 1 -- 172.21.15.39:0/1365303103 shutdown_connections 2024-09-21T07:45:36.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.934+0000 7f98e3c00640 1 -- 172.21.15.39:0/1365303103 wait complete. 2024-09-21T07:45:36.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.936+0000 7f98e3c00640 1 Processor -- start 2024-09-21T07:45:36.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.936+0000 7f98e3c00640 1 -- start start 2024-09-21T07:45:36.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.938+0000 7f98e3c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98dc070570 0x7f98dc1f5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:36.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.938+0000 7f98e3c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f98dc0cf750 0x7f98dc154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:36.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.938+0000 7f98e3c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98dc151640 0x7f98dc1e6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:36.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.938+0000 7f98e3c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc15eb60 con 0x7f98dc151640 2024-09-21T07:45:36.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.938+0000 7f98e3c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc15e9e0 con 0x7f98dc070570 2024-09-21T07:45:36.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.938+0000 7f98e3c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f98dc15ece0 con 0x7f98dc0cf750 2024-09-21T07:45:36.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.938+0000 7f98e1c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98dc151640 0x7f98dc1e6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:36.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.938+0000 7f98e1c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98dc151640 0x7f98dc1e6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52162/0 (socket says 172.21.15.39:52162) 2024-09-21T07:45:36.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.938+0000 7f98e1c00640 1 -- 172.21.15.39:0/1031093690 learned_addr learned my addr 172.21.15.39:0/1031093690 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:36.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.938+0000 7f98e1200640 1 --2- 172.21.15.39:0/1031093690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98dc070570 0x7f98dc1f5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:36.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.938+0000 7f98d3e00640 1 --2- 172.21.15.39:0/1031093690 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f98dc0cf750 0x7f98dc154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:36.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.939+0000 7f98e1c00640 1 -- 172.21.15.39:0/1031093690 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f98dc0cf750 msgr2=0x7f98dc154c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:36.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.939+0000 7f98e1c00640 1 --2- 172.21.15.39:0/1031093690 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f98dc0cf750 0x7f98dc154c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:36.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.939+0000 7f98e1c00640 1 -- 172.21.15.39:0/1031093690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98dc070570 msgr2=0x7f98dc1f5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:36.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.939+0000 7f98e1c00640 1 --2- 172.21.15.39:0/1031093690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98dc070570 0x7f98dc1f5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:36.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.939+0000 7f98e1c00640 1 -- 172.21.15.39:0/1031093690 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98dc1f5cf0 con 0x7f98dc151640 2024-09-21T07:45:36.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.939+0000 7f98e1200640 1 --2- 172.21.15.39:0/1031093690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98dc070570 0x7f98dc1f5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:45:36.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.939+0000 7f98e1c00640 1 --2- 172.21.15.39:0/1031093690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98dc151640 0x7f98dc1e6810 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f98d4035040 tx=0x7f98d40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:36.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.939+0000 7f98d9600640 1 -- 172.21.15.39:0/1031093690 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d4037040 con 0x7f98dc151640 2024-09-21T07:45:36.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.939+0000 7f98d9600640 1 -- 172.21.15.39:0/1031093690 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98d4007a50 con 0x7f98dc151640 2024-09-21T07:45:36.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.939+0000 7f98d9600640 1 -- 172.21.15.39:0/1031093690 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d400a370 con 0x7f98dc151640 2024-09-21T07:45:36.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.939+0000 7f98e3c00640 1 -- 172.21.15.39:0/1031093690 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98dc1f5fd0 con 0x7f98dc151640 2024-09-21T07:45:36.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.940+0000 7f98e3c00640 1 -- 172.21.15.39:0/1031093690 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98dc154760 con 0x7f98dc151640 2024-09-21T07:45:36.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.941+0000 7f98e3c00640 1 -- 172.21.15.39:0/1031093690 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98dc15f190 con 0x7f98dc151640 2024-09-21T07:45:36.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.942+0000 7f98d9600640 1 -- 172.21.15.39:0/1031093690 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f98d400a510 con 0x7f98dc151640 2024-09-21T07:45:36.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.943+0000 7f98d9600640 1 --2- 172.21.15.39:0/1031093690 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f98b8078b90 0x7f98b807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:36.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.943+0000 7f98d9600640 1 -- 172.21.15.39:0/1031093690 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 6744+0+0 (secure 0 0 0) 0x7f98d40bc0d0 con 0x7f98dc151640 2024-09-21T07:45:36.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.944+0000 7f98e1200640 1 --2- 172.21.15.39:0/1031093690 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f98b8078b90 0x7f98b807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:36.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.944+0000 7f98e1200640 1 --2- 172.21.15.39:0/1031093690 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f98b8078b90 0x7f98b807b0a0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0097a0 tx=0x7f98cc009260 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:36.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:36.947+0000 7f98d9600640 1 -- 172.21.15.39:0/1031093690 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f98d4083a90 con 0x7f98dc151640 2024-09-21T07:45:37.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.184+0000 7f98e3c00640 1 -- 172.21.15.39:0/1031093690 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f98dc1f5770 con 0x7f98dc151640 2024-09-21T07:45:37.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.661+0000 7f98d9600640 1 -- 172.21.15.39:0/1031093690 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 19 and data pool 20 v177) ==== 174+0+0 (secure 0 0 0) 0x7f98d40884c0 con 0x7f98dc151640 2024-09-21T07:45:37.663 INFO:teuthology.orchestra.run.smithi039.stderr:new fs with metadata pool 19 and data pool 20 2024-09-21T07:45:37.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.664+0000 7f98e3c00640 1 -- 172.21.15.39:0/1031093690 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f98b8078b90 msgr2=0x7f98b807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:37.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.664+0000 7f98e3c00640 1 --2- 172.21.15.39:0/1031093690 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f98b8078b90 0x7f98b807b0a0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0097a0 tx=0x7f98cc009260 comp rx=0 tx=0).stop 2024-09-21T07:45:37.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.664+0000 7f98e3c00640 1 -- 172.21.15.39:0/1031093690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98dc151640 msgr2=0x7f98dc1e6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:37.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.664+0000 7f98e3c00640 1 --2- 172.21.15.39:0/1031093690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98dc151640 0x7f98dc1e6810 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f98d4035040 tx=0x7f98d40045e0 comp rx=0 tx=0).stop 2024-09-21T07:45:37.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.665+0000 7f98e3c00640 1 -- 172.21.15.39:0/1031093690 shutdown_connections 2024-09-21T07:45:37.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.665+0000 7f98e3c00640 1 --2- 172.21.15.39:0/1031093690 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f98b8078b90 0x7f98b807b0a0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:37.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.665+0000 7f98e3c00640 1 --2- 172.21.15.39:0/1031093690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98dc151640 0x7f98dc1e6810 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:37.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.665+0000 7f98e3c00640 1 --2- 172.21.15.39:0/1031093690 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f98dc0cf750 0x7f98dc154c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:37.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.665+0000 7f98e3c00640 1 --2- 172.21.15.39:0/1031093690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98dc070570 0x7f98dc1f5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:37.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.665+0000 7f98e3c00640 1 -- 172.21.15.39:0/1031093690 >> 172.21.15.39:0/1031093690 conn(0x7f98dc06cfc0 msgr2=0x7f98dc06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:37.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.665+0000 7f98e3c00640 1 -- 172.21.15.39:0/1031093690 shutdown_connections 2024-09-21T07:45:37.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.665+0000 7f98e3c00640 1 -- 172.21.15.39:0/1031093690 wait complete. 2024-09-21T07:45:37.716 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:45:37.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.940+0000 7f6e8d200640 1 -- 172.21.15.39:0/763013357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e88160660 msgr2=0x7f6e880cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:37.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.940+0000 7f6e8d200640 1 --2- 172.21.15.39:0/763013357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e88160660 0x7f6e880cf710 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f6e74009830 tx=0x7f6e7402a510 comp rx=0 tx=0).stop 2024-09-21T07:45:37.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.940+0000 7f6e8d200640 1 -- 172.21.15.39:0/763013357 shutdown_connections 2024-09-21T07:45:37.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.940+0000 7f6e8d200640 1 --2- 172.21.15.39:0/763013357 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e880cfc50 0x7f6e880d0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:37.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.940+0000 7f6e8d200640 1 --2- 172.21.15.39:0/763013357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e88160660 0x7f6e880cf710 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:37.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.940+0000 7f6e8d200640 1 -- 172.21.15.39:0/763013357 >> 172.21.15.39:0/763013357 conn(0x7f6e8806cfc0 msgr2=0x7f6e8806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:37.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.940+0000 7f6e8d200640 1 -- 172.21.15.39:0/763013357 shutdown_connections 2024-09-21T07:45:37.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.940+0000 7f6e8d200640 1 -- 172.21.15.39:0/763013357 wait complete. 2024-09-21T07:45:37.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.942+0000 7f6e8d200640 1 Processor -- start 2024-09-21T07:45:37.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.942+0000 7f6e8d200640 1 -- start start 2024-09-21T07:45:37.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.943+0000 7f6e8d200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e880cfc50 0x7f6e880aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:37.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.943+0000 7f6e8d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e88160250 0x7f6e881342f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:37.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.944+0000 7f6e8d200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e88160660 0x7f6e88134830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:37.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.944+0000 7f6e8d200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6e880ccdc0 con 0x7f6e88160250 2024-09-21T07:45:37.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.944+0000 7f6e8d200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6e880ccc40 con 0x7f6e88160660 2024-09-21T07:45:37.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.944+0000 7f6e8d200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f6e880ccf40 con 0x7f6e880cfc50 2024-09-21T07:45:37.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.944+0000 7f6e86a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e88160250 0x7f6e881342f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:37.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.944+0000 7f6e86a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e88160250 0x7f6e881342f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52168/0 (socket says 172.21.15.39:52168) 2024-09-21T07:45:37.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.944+0000 7f6e86a00640 1 -- 172.21.15.39:0/1454001708 learned_addr learned my addr 172.21.15.39:0/1454001708 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:37.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.944+0000 7f6e86a00640 1 -- 172.21.15.39:0/1454001708 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e880cfc50 msgr2=0x7f6e880aec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:37.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.944+0000 7f6e86a00640 1 --2- 172.21.15.39:0/1454001708 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e880cfc50 0x7f6e880aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:37.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.944+0000 7f6e86a00640 1 -- 172.21.15.39:0/1454001708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e88160660 msgr2=0x7f6e88134830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:37.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.944+0000 7f6e86a00640 1 --2- 172.21.15.39:0/1454001708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e88160660 0x7f6e88134830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:37.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.944+0000 7f6e86a00640 1 -- 172.21.15.39:0/1454001708 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e88135090 con 0x7f6e88160250 2024-09-21T07:45:37.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.945+0000 7f6e86a00640 1 --2- 172.21.15.39:0/1454001708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e88160250 0x7f6e881342f0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f6e74009950 tx=0x7f6e74033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:37.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.945+0000 7f6e7f400640 1 -- 172.21.15.39:0/1454001708 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e74031d60 con 0x7f6e88160250 2024-09-21T07:45:37.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.945+0000 7f6e7f400640 1 -- 172.21.15.39:0/1454001708 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e74033e10 con 0x7f6e88160250 2024-09-21T07:45:37.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.945+0000 7f6e7f400640 1 -- 172.21.15.39:0/1454001708 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e74005260 con 0x7f6e88160250 2024-09-21T07:45:37.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.945+0000 7f6e8d200640 1 -- 172.21.15.39:0/1454001708 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e880b0cc0 con 0x7f6e88160250 2024-09-21T07:45:37.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.945+0000 7f6e8d200640 1 -- 172.21.15.39:0/1454001708 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e880b1280 con 0x7f6e88160250 2024-09-21T07:45:37.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.946+0000 7f6e8d200640 1 -- 172.21.15.39:0/1454001708 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e88134e00 con 0x7f6e88160250 2024-09-21T07:45:37.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.947+0000 7f6e7f400640 1 -- 172.21.15.39:0/1454001708 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6e74005400 con 0x7f6e88160250 2024-09-21T07:45:37.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.948+0000 7f6e7f400640 1 --2- 172.21.15.39:0/1454001708 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6e68078b90 0x7f6e6807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:37.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.948+0000 7f6e7f400640 1 -- 172.21.15.39:0/1454001708 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6844+0+0 (secure 0 0 0) 0x7f6e740bc480 con 0x7f6e88160250 2024-09-21T07:45:37.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.949+0000 7f6e87400640 1 --2- 172.21.15.39:0/1454001708 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6e68078b90 0x7f6e6807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:37.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.949+0000 7f6e87400640 1 --2- 172.21.15.39:0/1454001708 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6e68078b90 0x7f6e6807b0a0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f6e78001510 tx=0x7f6e78007930 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:37.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:37.953+0000 7f6e7f400640 1 -- 172.21.15.39:0/1454001708 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6e7403f030 con 0x7f6e88160250 2024-09-21T07:45:38.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.151+0000 7f6e8d200640 1 -- 172.21.15.39:0/1454001708 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6e8806fdb0 con 0x7f6e88160250 2024-09-21T07:45:38.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.153+0000 7f6e7f400640 1 -- 172.21.15.39:0/1454001708 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) ==== 74+0+22232 (secure 0 0 0) 0x7f6e8806fdb0 con 0x7f6e88160250 2024-09-21T07:45:38.155 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:45:38.155 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":156,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:45:37.186097+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:45:35.115582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-09-21T07:45:36.142685+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:45:38.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.155+0000 7f6e8d200640 1 -- 172.21.15.39:0/1454001708 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6e68078b90 msgr2=0x7f6e6807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:38.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.155+0000 7f6e8d200640 1 --2- 172.21.15.39:0/1454001708 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6e68078b90 0x7f6e6807b0a0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f6e78001510 tx=0x7f6e78007930 comp rx=0 tx=0).stop 2024-09-21T07:45:38.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.155+0000 7f6e8d200640 1 -- 172.21.15.39:0/1454001708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e88160250 msgr2=0x7f6e881342f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:38.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.155+0000 7f6e8d200640 1 --2- 172.21.15.39:0/1454001708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e88160250 0x7f6e881342f0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f6e74009950 tx=0x7f6e74033c20 comp rx=0 tx=0).stop 2024-09-21T07:45:38.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.155+0000 7f6e8d200640 1 -- 172.21.15.39:0/1454001708 shutdown_connections 2024-09-21T07:45:38.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.155+0000 7f6e8d200640 1 --2- 172.21.15.39:0/1454001708 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6e68078b90 0x7f6e6807b0a0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:38.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.155+0000 7f6e8d200640 1 --2- 172.21.15.39:0/1454001708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e88160660 0x7f6e88134830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:38.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.155+0000 7f6e8d200640 1 --2- 172.21.15.39:0/1454001708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e88160250 0x7f6e881342f0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:38.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.155+0000 7f6e8d200640 1 --2- 172.21.15.39:0/1454001708 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e880cfc50 0x7f6e880aec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:38.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.155+0000 7f6e8d200640 1 -- 172.21.15.39:0/1454001708 >> 172.21.15.39:0/1454001708 conn(0x7f6e8806cfc0 msgr2=0x7f6e8815ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:38.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.156+0000 7f6e8d200640 1 -- 172.21.15.39:0/1454001708 shutdown_connections 2024-09-21T07:45:38.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.156+0000 7f6e8d200640 1 -- 172.21.15.39:0/1454001708 wait complete. 2024-09-21T07:45:38.179 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:45:38.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.402+0000 7f3e9f800640 1 -- 172.21.15.39:0/1701661818 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e98151640 msgr2=0x7f3e98151a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:38.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.402+0000 7f3e9f800640 1 --2- 172.21.15.39:0/1701661818 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e98151640 0x7f3e98151a50 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c009830 tx=0x7f3e8c02a4d0 comp rx=0 tx=0).stop 2024-09-21T07:45:38.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.402+0000 7f3e9f800640 1 -- 172.21.15.39:0/1701661818 shutdown_connections 2024-09-21T07:45:38.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.402+0000 7f3e9f800640 1 --2- 172.21.15.39:0/1701661818 >> v2:172.21.15.148:3300/0 conn(0x7f3e98070570 0x7f3e98070980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:38.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.402+0000 7f3e9f800640 1 --2- 172.21.15.39:0/1701661818 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e98151640 0x7f3e98151a50 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:38.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.402+0000 7f3e9f800640 1 -- 172.21.15.39:0/1701661818 >> 172.21.15.39:0/1701661818 conn(0x7f3e9806cfc0 msgr2=0x7f3e9806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:38.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.402+0000 7f3e9f800640 1 -- 172.21.15.39:0/1701661818 shutdown_connections 2024-09-21T07:45:38.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.402+0000 7f3e9f800640 1 -- 172.21.15.39:0/1701661818 wait complete. 2024-09-21T07:45:38.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.404+0000 7f3e9f800640 1 Processor -- start 2024-09-21T07:45:38.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.405+0000 7f3e9f800640 1 -- start start 2024-09-21T07:45:38.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.406+0000 7f3e9f800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e98070570 0x7f3e981f5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:38.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.406+0000 7f3e9f800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3e980cf750 0x7f3e980a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:38.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.406+0000 7f3e9f800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e98151640 0x7f3e980b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:38.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.406+0000 7f3e9f800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9815eb60 con 0x7f3e98070570 2024-09-21T07:45:38.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.406+0000 7f3e9f800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9815e9e0 con 0x7f3e98151640 2024-09-21T07:45:38.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.406+0000 7f3e9f800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f3e9815ece0 con 0x7f3e980cf750 2024-09-21T07:45:38.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.406+0000 7f3e9ce00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e98070570 0x7f3e981f5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:38.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.406+0000 7f3e9ce00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e98070570 0x7f3e981f5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52194/0 (socket says 172.21.15.39:52194) 2024-09-21T07:45:38.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.406+0000 7f3e9ce00640 1 -- 172.21.15.39:0/4134313788 learned_addr learned my addr 172.21.15.39:0/4134313788 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:38.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.407+0000 7f3e9ce00640 1 -- 172.21.15.39:0/4134313788 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3e980cf750 msgr2=0x7f3e980a94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:38.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.407+0000 7f3e9ce00640 1 --2- 172.21.15.39:0/4134313788 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3e980cf750 0x7f3e980a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:38.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.407+0000 7f3e9ce00640 1 -- 172.21.15.39:0/4134313788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e98151640 msgr2=0x7f3e980b0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:38.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.407+0000 7f3e9ce00640 1 --2- 172.21.15.39:0/4134313788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e98151640 0x7f3e980b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:38.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.407+0000 7f3e9ce00640 1 -- 172.21.15.39:0/4134313788 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e980a9f90 con 0x7f3e98070570 2024-09-21T07:45:38.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.407+0000 7f3e9ce00640 1 --2- 172.21.15.39:0/4134313788 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e98070570 0x7f3e981f5f50 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c009950 tx=0x7f3e8c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:38.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.407+0000 7f3e94c00640 1 -- 172.21.15.39:0/4134313788 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c031d60 con 0x7f3e98070570 2024-09-21T07:45:38.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.407+0000 7f3e94c00640 1 -- 172.21.15.39:0/4134313788 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e8c033e10 con 0x7f3e98070570 2024-09-21T07:45:38.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.407+0000 7f3e9f800640 1 -- 172.21.15.39:0/4134313788 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e98159cd0 con 0x7f3e98070570 2024-09-21T07:45:38.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.407+0000 7f3e94c00640 1 -- 172.21.15.39:0/4134313788 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c005260 con 0x7f3e98070570 2024-09-21T07:45:38.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.408+0000 7f3e9f800640 1 -- 172.21.15.39:0/4134313788 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e980a9a10 con 0x7f3e98070570 2024-09-21T07:45:38.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.409+0000 7f3e9f800640 1 -- 172.21.15.39:0/4134313788 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e9815f200 con 0x7f3e98070570 2024-09-21T07:45:38.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.410+0000 7f3e94c00640 1 -- 172.21.15.39:0/4134313788 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3e8c005400 con 0x7f3e98070570 2024-09-21T07:45:38.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.411+0000 7f3e94c00640 1 --2- 172.21.15.39:0/4134313788 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3e78078b90 0x7f3e7807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:38.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.411+0000 7f3e94c00640 1 -- 172.21.15.39:0/4134313788 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6844+0+0 (secure 0 0 0) 0x7f3e8c0bc260 con 0x7f3e98070570 2024-09-21T07:45:38.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.411+0000 7f3e97e00640 1 --2- 172.21.15.39:0/4134313788 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3e78078b90 0x7f3e7807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:38.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.412+0000 7f3e97e00640 1 --2- 172.21.15.39:0/4134313788 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3e78078b90 0x7f3e7807b0a0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f3e8400afd0 tx=0x7f3e8400c040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:38.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.416+0000 7f3e94c00640 1 -- 172.21.15.39:0/4134313788 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3e8c03f030 con 0x7f3e98070570 2024-09-21T07:45:38.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.613+0000 7f3e9f800640 1 -- 172.21.15.39:0/4134313788 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3e9815a960 con 0x7f3e98070570 2024-09-21T07:45:38.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.614+0000 7f3e94c00640 1 -- 172.21.15.39:0/4134313788 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) ==== 74+0+22232 (secure 0 0 0) 0x7f3e8c0842c0 con 0x7f3e98070570 2024-09-21T07:45:38.617 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:45:38.617 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":156,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:45:37.186097+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:45:35.115582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-09-21T07:45:36.142685+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:45:38.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.617+0000 7f3e9f800640 1 -- 172.21.15.39:0/4134313788 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3e78078b90 msgr2=0x7f3e7807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:38.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.617+0000 7f3e9f800640 1 --2- 172.21.15.39:0/4134313788 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3e78078b90 0x7f3e7807b0a0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f3e8400afd0 tx=0x7f3e8400c040 comp rx=0 tx=0).stop 2024-09-21T07:45:38.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.617+0000 7f3e9f800640 1 -- 172.21.15.39:0/4134313788 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e98070570 msgr2=0x7f3e981f5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:38.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.617+0000 7f3e9f800640 1 --2- 172.21.15.39:0/4134313788 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e98070570 0x7f3e981f5f50 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c009950 tx=0x7f3e8c033c20 comp rx=0 tx=0).stop 2024-09-21T07:45:38.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.617+0000 7f3e9f800640 1 -- 172.21.15.39:0/4134313788 shutdown_connections 2024-09-21T07:45:38.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.617+0000 7f3e9f800640 1 --2- 172.21.15.39:0/4134313788 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3e78078b90 0x7f3e7807b0a0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:38.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.617+0000 7f3e9f800640 1 --2- 172.21.15.39:0/4134313788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e98151640 0x7f3e980b0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:38.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.617+0000 7f3e9f800640 1 --2- 172.21.15.39:0/4134313788 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3e980cf750 0x7f3e980a94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:38.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.617+0000 7f3e9f800640 1 --2- 172.21.15.39:0/4134313788 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e98070570 0x7f3e981f5f50 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:38.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.617+0000 7f3e9f800640 1 -- 172.21.15.39:0/4134313788 >> 172.21.15.39:0/4134313788 conn(0x7f3e9806cfc0 msgr2=0x7f3e980cfb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:38.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.617+0000 7f3e9f800640 1 -- 172.21.15.39:0/4134313788 shutdown_connections 2024-09-21T07:45:38.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.617+0000 7f3e9f800640 1 -- 172.21.15.39:0/4134313788 wait complete. 2024-09-21T07:45:38.644 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T07:45:38.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.873+0000 7fe6c5600640 1 -- 172.21.15.39:0/1213180635 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6c00b32d0 msgr2=0x7fe6c0074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:38.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.873+0000 7fe6c5600640 1 --2- 172.21.15.39:0/1213180635 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6c00b32d0 0x7fe6c0074490 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0005fd0 tx=0x7fe6b002a4a0 comp rx=0 tx=0).stop 2024-09-21T07:45:38.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.873+0000 7fe6c5600640 1 -- 172.21.15.39:0/1213180635 shutdown_connections 2024-09-21T07:45:38.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.873+0000 7fe6c5600640 1 --2- 172.21.15.39:0/1213180635 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe6c00749d0 0x7fe6c0167030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:38.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.873+0000 7fe6c5600640 1 --2- 172.21.15.39:0/1213180635 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6c00b32d0 0x7fe6c0074490 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:38.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.873+0000 7fe6c5600640 1 -- 172.21.15.39:0/1213180635 >> 172.21.15.39:0/1213180635 conn(0x7fe6c00b1f70 msgr2=0x7fe6c00b2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:38.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.873+0000 7fe6c5600640 1 -- 172.21.15.39:0/1213180635 shutdown_connections 2024-09-21T07:45:38.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.873+0000 7fe6c5600640 1 -- 172.21.15.39:0/1213180635 wait complete. 2024-09-21T07:45:38.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.875+0000 7fe6c5600640 1 Processor -- start 2024-09-21T07:45:38.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.875+0000 7fe6c5600640 1 -- start start 2024-09-21T07:45:38.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.877+0000 7fe6c5600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe6c00749d0 0x7fe6c0156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:38.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.877+0000 7fe6c5600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6c00b32d0 0x7fe6c01e7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:38.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.877+0000 7fe6c5600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6c00b36e0 0x7fe6c015b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:38.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.877+0000 7fe6c5600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c00b5e00 con 0x7fe6c00b36e0 2024-09-21T07:45:38.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.877+0000 7fe6c5600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c006e960 con 0x7fe6c00b32d0 2024-09-21T07:45:38.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.877+0000 7fe6c5600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe6c00b5f80 con 0x7fe6c00749d0 2024-09-21T07:45:38.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.877+0000 7fe6c4c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6c00b36e0 0x7fe6c015b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:38.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.877+0000 7fe6c4c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6c00b36e0 0x7fe6c015b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52202/0 (socket says 172.21.15.39:52202) 2024-09-21T07:45:38.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.877+0000 7fe6c4c00640 1 -- 172.21.15.39:0/2172127097 learned_addr learned my addr 172.21.15.39:0/2172127097 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:38.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.877+0000 7fe6b7e00640 1 --2- 172.21.15.39:0/2172127097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6c00b32d0 0x7fe6c01e7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:38.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.877+0000 7fe6bfe00640 1 --2- 172.21.15.39:0/2172127097 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe6c00749d0 0x7fe6c0156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:38.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.878+0000 7fe6c4c00640 1 -- 172.21.15.39:0/2172127097 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe6c00749d0 msgr2=0x7fe6c0156200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:38.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.878+0000 7fe6c4c00640 1 --2- 172.21.15.39:0/2172127097 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe6c00749d0 0x7fe6c0156200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:38.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.878+0000 7fe6c4c00640 1 -- 172.21.15.39:0/2172127097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6c00b32d0 msgr2=0x7fe6c01e7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:38.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.878+0000 7fe6c4c00640 1 --2- 172.21.15.39:0/2172127097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6c00b32d0 0x7fe6c01e7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:38.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.878+0000 7fe6c4c00640 1 -- 172.21.15.39:0/2172127097 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6c01e67c0 con 0x7fe6c00b36e0 2024-09-21T07:45:38.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.878+0000 7fe6c4c00640 1 --2- 172.21.15.39:0/2172127097 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6c00b36e0 0x7fe6c015b9c0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fe6b800caa0 tx=0x7fe6b800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:38.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.878+0000 7fe6bcc00640 1 -- 172.21.15.39:0/2172127097 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b800adf0 con 0x7fe6c00b36e0 2024-09-21T07:45:38.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.878+0000 7fe6bcc00640 1 -- 172.21.15.39:0/2172127097 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6b800f540 con 0x7fe6c00b36e0 2024-09-21T07:45:38.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.878+0000 7fe6bcc00640 1 -- 172.21.15.39:0/2172127097 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b800f990 con 0x7fe6c00b36e0 2024-09-21T07:45:38.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.878+0000 7fe6c5600640 1 -- 172.21.15.39:0/2172127097 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6c01e6aa0 con 0x7fe6c00b36e0 2024-09-21T07:45:38.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.878+0000 7fe6c5600640 1 -- 172.21.15.39:0/2172127097 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6c01e7060 con 0x7fe6c00b36e0 2024-09-21T07:45:38.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.879+0000 7fe6c5600640 1 -- 172.21.15.39:0/2172127097 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6c01e6530 con 0x7fe6c00b36e0 2024-09-21T07:45:38.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.881+0000 7fe6bcc00640 1 -- 172.21.15.39:0/2172127097 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe6b800fb30 con 0x7fe6c00b36e0 2024-09-21T07:45:38.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.882+0000 7fe6bcc00640 1 --2- 172.21.15.39:0/2172127097 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe69c078b90 0x7fe69c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:38.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.882+0000 7fe6bcc00640 1 -- 172.21.15.39:0/2172127097 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 6844+0+0 (secure 0 0 0) 0x7fe6b809bc30 con 0x7fe6c00b36e0 2024-09-21T07:45:38.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.882+0000 7fe6bfe00640 1 --2- 172.21.15.39:0/2172127097 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe69c078b90 0x7fe69c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:38.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.883+0000 7fe6bfe00640 1 --2- 172.21.15.39:0/2172127097 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe69c078b90 0x7fe69c07b0a0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0033000 tx=0x7fe6b0004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:38.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:38.886+0000 7fe6bcc00640 1 -- 172.21.15.39:0/2172127097 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe6b8014030 con 0x7fe6c00b36e0 2024-09-21T07:45:39.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.123+0000 7fe6c5600640 1 -- 172.21.15.39:0/2172127097 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe6c00b1670 con 0x7fe6c00b36e0 2024-09-21T07:45:39.626 INFO:tasks.ceph.mds.a.smithi039.stderr:2024-09-21T07:45:39.625+0000 7fb8efc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:45:39.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.694+0000 7fe6bcc00640 1 -- 172.21.15.39:0/2172127097 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v180) ==== 118+0+0 (secure 0 0 0) 0x7fe6b800fe20 con 0x7fe6c00b36e0 2024-09-21T07:45:39.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.696+0000 7fe6c5600640 1 -- 172.21.15.39:0/2172127097 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe69c078b90 msgr2=0x7fe69c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:39.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.696+0000 7fe6c5600640 1 --2- 172.21.15.39:0/2172127097 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe69c078b90 0x7fe69c07b0a0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0033000 tx=0x7fe6b0004730 comp rx=0 tx=0).stop 2024-09-21T07:45:39.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.696+0000 7fe6c5600640 1 -- 172.21.15.39:0/2172127097 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6c00b36e0 msgr2=0x7fe6c015b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:39.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.696+0000 7fe6c5600640 1 --2- 172.21.15.39:0/2172127097 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6c00b36e0 0x7fe6c015b9c0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fe6b800caa0 tx=0x7fe6b800cf60 comp rx=0 tx=0).stop 2024-09-21T07:45:39.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.696+0000 7fe6c5600640 1 -- 172.21.15.39:0/2172127097 shutdown_connections 2024-09-21T07:45:39.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.696+0000 7fe6c5600640 1 --2- 172.21.15.39:0/2172127097 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe69c078b90 0x7fe69c07b0a0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.696+0000 7fe6c5600640 1 --2- 172.21.15.39:0/2172127097 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6c00b36e0 0x7fe6c015b9c0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.696+0000 7fe6c5600640 1 --2- 172.21.15.39:0/2172127097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6c00b32d0 0x7fe6c01e7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.696+0000 7fe6c5600640 1 --2- 172.21.15.39:0/2172127097 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe6c00749d0 0x7fe6c0156200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.696+0000 7fe6c5600640 1 -- 172.21.15.39:0/2172127097 >> 172.21.15.39:0/2172127097 conn(0x7fe6c00b1f70 msgr2=0x7fe6c00b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:39.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.697+0000 7fe6c5600640 1 -- 172.21.15.39:0/2172127097 shutdown_connections 2024-09-21T07:45:39.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.697+0000 7fe6c5600640 1 -- 172.21.15.39:0/2172127097 wait complete. 2024-09-21T07:45:39.725 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T07:45:39.725 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T07:45:39.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.954+0000 7f8b10800640 1 -- 172.21.15.39:0/3815069509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b080cd260 msgr2=0x7f8b080cf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:39.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.954+0000 7f8b10800640 1 --2- 172.21.15.39:0/3815069509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b080cd260 0x7f8b080cf450 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f8af8009830 tx=0x7f8af802a510 comp rx=0 tx=0).stop 2024-09-21T07:45:39.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.954+0000 7f8b10800640 1 -- 172.21.15.39:0/3815069509 shutdown_connections 2024-09-21T07:45:39.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.954+0000 7f8b10800640 1 --2- 172.21.15.39:0/3815069509 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b080cf990 0x7f8b080b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.954+0000 7f8b10800640 1 --2- 172.21.15.39:0/3815069509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b080cd260 0x7f8b080cf450 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.954+0000 7f8b10800640 1 -- 172.21.15.39:0/3815069509 >> 172.21.15.39:0/3815069509 conn(0x7f8b080b2460 msgr2=0x7f8b0806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:39.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.955+0000 7f8b10800640 1 -- 172.21.15.39:0/3815069509 shutdown_connections 2024-09-21T07:45:39.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.955+0000 7f8b10800640 1 -- 172.21.15.39:0/3815069509 wait complete. 2024-09-21T07:45:39.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.957+0000 7f8b10800640 1 Processor -- start 2024-09-21T07:45:39.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.957+0000 7f8b10800640 1 -- start start 2024-09-21T07:45:39.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.958+0000 7f8b10800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b080af430 0x7f8b08098de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:39.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.958+0000 7f8b10800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b080cd260 0x7f8b081584c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:39.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.958+0000 7f8b10800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b080cf990 0x7f8b081f9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:39.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.958+0000 7f8b10800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8b080b45f0 con 0x7f8b080af430 2024-09-21T07:45:39.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.958+0000 7f8b10800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8b08071230 con 0x7f8b080cf990 2024-09-21T07:45:39.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.958+0000 7f8b10800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f8b080b4770 con 0x7f8b080cd260 2024-09-21T07:45:39.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.959+0000 7f8b0de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b080af430 0x7f8b08098de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:39.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.959+0000 7f8b0de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b080af430 0x7f8b08098de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52216/0 (socket says 172.21.15.39:52216) 2024-09-21T07:45:39.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.959+0000 7f8b0de00640 1 -- 172.21.15.39:0/547941109 learned_addr learned my addr 172.21.15.39:0/547941109 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:39.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.959+0000 7f8b0d400640 1 --2- 172.21.15.39:0/547941109 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b080cd260 0x7f8b081584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:39.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.959+0000 7f8b0e800640 1 --2- 172.21.15.39:0/547941109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b080cf990 0x7f8b081f9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:39.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.959+0000 7f8b0de00640 1 -- 172.21.15.39:0/547941109 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b080cd260 msgr2=0x7f8b081584c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:39.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.959+0000 7f8b0de00640 1 --2- 172.21.15.39:0/547941109 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b080cd260 0x7f8b081584c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.959+0000 7f8b0de00640 1 -- 172.21.15.39:0/547941109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b080cf990 msgr2=0x7f8b081f9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:39.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.959+0000 7f8b0de00640 1 --2- 172.21.15.39:0/547941109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b080cf990 0x7f8b081f9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.959+0000 7f8b0de00640 1 -- 172.21.15.39:0/547941109 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b08158a00 con 0x7f8b080af430 2024-09-21T07:45:39.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.959+0000 7f8b0d400640 1 --2- 172.21.15.39:0/547941109 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b080cd260 0x7f8b081584c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:45:39.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.959+0000 7f8b0e800640 1 --2- 172.21.15.39:0/547941109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b080cf990 0x7f8b081f9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:45:39.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.959+0000 7f8b0de00640 1 --2- 172.21.15.39:0/547941109 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b080af430 0x7f8b08098de0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f8afc00c3f0 tx=0x7f8afc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:39.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.960+0000 7f8b02000640 1 -- 172.21.15.39:0/547941109 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc017070 con 0x7f8b080af430 2024-09-21T07:45:39.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.960+0000 7f8b02000640 1 -- 172.21.15.39:0/547941109 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8afc0035f0 con 0x7f8b080af430 2024-09-21T07:45:39.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.960+0000 7f8b10800640 1 -- 172.21.15.39:0/547941109 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b080cd060 con 0x7f8b080af430 2024-09-21T07:45:39.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.960+0000 7f8b02000640 1 -- 172.21.15.39:0/547941109 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc003a40 con 0x7f8b080af430 2024-09-21T07:45:39.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.960+0000 7f8b10800640 1 -- 172.21.15.39:0/547941109 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b0815d890 con 0x7f8b080af430 2024-09-21T07:45:39.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.961+0000 7f8b10800640 1 -- 172.21.15.39:0/547941109 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b080cf450 con 0x7f8b080af430 2024-09-21T07:45:39.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.962+0000 7f8b02000640 1 -- 172.21.15.39:0/547941109 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8afc01a070 con 0x7f8b080af430 2024-09-21T07:45:39.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.963+0000 7f8b02000640 1 --2- 172.21.15.39:0/547941109 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8aec078bc0 0x7f8aec07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:39.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.963+0000 7f8b02000640 1 -- 172.21.15.39:0/547941109 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 6844+0+0 (secure 0 0 0) 0x7f8afc09a5b0 con 0x7f8b080af430 2024-09-21T07:45:39.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.964+0000 7f8b0d400640 1 --2- 172.21.15.39:0/547941109 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8aec078bc0 0x7f8aec07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:39.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.965+0000 7f8b0d400640 1 --2- 172.21.15.39:0/547941109 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8aec078bc0 0x7f8aec07b0d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f8af8034000 tx=0x7f8af8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:39.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:39.968+0000 7f8b02000640 1 -- 172.21.15.39:0/547941109 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8afc062570 con 0x7f8b080af430 2024-09-21T07:45:40.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.205+0000 7f8b10800640 1 -- 172.21.15.39:0/547941109 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f8b081582a0 con 0x7f8b080af430 2024-09-21T07:45:40.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.696+0000 7f8b02000640 1 -- 172.21.15.39:0/547941109 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v181) ==== 122+0+0 (secure 0 0 0) 0x7f8afc061e70 con 0x7f8b080af430 2024-09-21T07:45:40.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.699+0000 7f8b10800640 1 -- 172.21.15.39:0/547941109 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8aec078bc0 msgr2=0x7f8aec07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.699+0000 7f8b10800640 1 --2- 172.21.15.39:0/547941109 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8aec078bc0 0x7f8aec07b0d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f8af8034000 tx=0x7f8af8004730 comp rx=0 tx=0).stop 2024-09-21T07:45:40.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.699+0000 7f8b10800640 1 -- 172.21.15.39:0/547941109 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b080af430 msgr2=0x7f8b08098de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.699+0000 7f8b10800640 1 --2- 172.21.15.39:0/547941109 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b080af430 0x7f8b08098de0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f8afc00c3f0 tx=0x7f8afc00c8b0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.699+0000 7f8b10800640 1 -- 172.21.15.39:0/547941109 shutdown_connections 2024-09-21T07:45:40.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.699+0000 7f8b10800640 1 --2- 172.21.15.39:0/547941109 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8aec078bc0 0x7f8aec07b0d0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.699+0000 7f8b10800640 1 --2- 172.21.15.39:0/547941109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b080cf990 0x7f8b081f9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.699+0000 7f8b10800640 1 --2- 172.21.15.39:0/547941109 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b080cd260 0x7f8b081584c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.699+0000 7f8b10800640 1 --2- 172.21.15.39:0/547941109 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b080af430 0x7f8b08098de0 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.699+0000 7f8b10800640 1 -- 172.21.15.39:0/547941109 >> 172.21.15.39:0/547941109 conn(0x7f8b080b2460 msgr2=0x7f8b0815a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:40.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.699+0000 7f8b10800640 1 -- 172.21.15.39:0/547941109 shutdown_connections 2024-09-21T07:45:40.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.699+0000 7f8b10800640 1 -- 172.21.15.39:0/547941109 wait complete. 2024-09-21T07:45:40.725 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T07:45:40.725 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:45:40.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.951+0000 7f7d6de00640 1 -- 172.21.15.39:0/1879222917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d680b30a0 msgr2=0x7f7d680cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.951+0000 7f7d6de00640 1 --2- 172.21.15.39:0/1879222917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d680b30a0 0x7f7d680cb8a0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f7d68075160 tx=0x7f7d5802a4d0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.951+0000 7f7d6de00640 1 -- 172.21.15.39:0/1879222917 shutdown_connections 2024-09-21T07:45:40.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.951+0000 7f7d6de00640 1 --2- 172.21.15.39:0/1879222917 >> v2:172.21.15.148:3300/0 conn(0x7f7d6815d830 0x7f7d68156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.951+0000 7f7d6de00640 1 --2- 172.21.15.39:0/1879222917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d680b30a0 0x7f7d680cb8a0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.951+0000 7f7d6de00640 1 -- 172.21.15.39:0/1879222917 >> 172.21.15.39:0/1879222917 conn(0x7f7d680b2460 msgr2=0x7f7d6806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:40.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.951+0000 7f7d6de00640 1 -- 172.21.15.39:0/1879222917 shutdown_connections 2024-09-21T07:45:40.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.952+0000 7f7d6de00640 1 -- 172.21.15.39:0/1879222917 wait complete. 2024-09-21T07:45:40.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.954+0000 7f7d6de00640 1 Processor -- start 2024-09-21T07:45:40.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.954+0000 7f7d6de00640 1 -- start start 2024-09-21T07:45:40.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.956+0000 7f7d6de00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d680b30a0 0x7f7d681563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:40.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.956+0000 7f7d6de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d680ccc60 0x7f7d680cea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:40.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.956+0000 7f7d6de00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7d6815d830 0x7f7d681e93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:40.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.956+0000 7f7d6de00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7d680af410 con 0x7f7d680ccc60 2024-09-21T07:45:40.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.956+0000 7f7d6de00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7d680af290 con 0x7f7d680b30a0 2024-09-21T07:45:40.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.956+0000 7f7d6de00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f7d680af590 con 0x7f7d6815d830 2024-09-21T07:45:40.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.956+0000 7f7d66000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d680ccc60 0x7f7d680cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:40.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.956+0000 7f7d66000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d680ccc60 0x7f7d680cea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52240/0 (socket says 172.21.15.39:52240) 2024-09-21T07:45:40.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.956+0000 7f7d66000640 1 -- 172.21.15.39:0/2357079507 learned_addr learned my addr 172.21.15.39:0/2357079507 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:40.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.956+0000 7f7d66a00640 1 --2- 172.21.15.39:0/2357079507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d680b30a0 0x7f7d681563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:40.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.956+0000 7f7d67400640 1 --2- 172.21.15.39:0/2357079507 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7d6815d830 0x7f7d681e93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:40.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.956+0000 7f7d66000640 1 -- 172.21.15.39:0/2357079507 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7d6815d830 msgr2=0x7f7d681e93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.956+0000 7f7d66000640 1 --2- 172.21.15.39:0/2357079507 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7d6815d830 0x7f7d681e93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.956+0000 7f7d66000640 1 -- 172.21.15.39:0/2357079507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d680b30a0 msgr2=0x7f7d681563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.956+0000 7f7d66000640 1 --2- 172.21.15.39:0/2357079507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d680b30a0 0x7f7d681563b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.956+0000 7f7d66000640 1 -- 172.21.15.39:0/2357079507 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d680cfd10 con 0x7f7d680ccc60 2024-09-21T07:45:40.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.957+0000 7f7d66000640 1 --2- 172.21.15.39:0/2357079507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d680ccc60 0x7f7d680cea20 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f7d5400c970 tx=0x7f7d5400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:40.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.957+0000 7f7d5f400640 1 -- 172.21.15.39:0/2357079507 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d54007b50 con 0x7f7d680ccc60 2024-09-21T07:45:40.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.957+0000 7f7d5f400640 1 -- 172.21.15.39:0/2357079507 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d54007de0 con 0x7f7d680ccc60 2024-09-21T07:45:40.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.957+0000 7f7d5f400640 1 -- 172.21.15.39:0/2357079507 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5400f6b0 con 0x7f7d680ccc60 2024-09-21T07:45:40.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.957+0000 7f7d6de00640 1 -- 172.21.15.39:0/2357079507 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d680c4f70 con 0x7f7d680ccc60 2024-09-21T07:45:40.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.957+0000 7f7d6de00640 1 -- 172.21.15.39:0/2357079507 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d680c5530 con 0x7f7d680ccc60 2024-09-21T07:45:40.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.958+0000 7f7d6de00640 1 -- 172.21.15.39:0/2357079507 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d680b0bc0 con 0x7f7d680ccc60 2024-09-21T07:45:40.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.959+0000 7f7d5f400640 1 -- 172.21.15.39:0/2357079507 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7d5400f850 con 0x7f7d680ccc60 2024-09-21T07:45:40.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.960+0000 7f7d5f400640 1 --2- 172.21.15.39:0/2357079507 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7d48078b90 0x7f7d4807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:40.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.960+0000 7f7d5f400640 1 -- 172.21.15.39:0/2357079507 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 6844+0+0 (secure 0 0 0) 0x7f7d5409b960 con 0x7f7d680ccc60 2024-09-21T07:45:40.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.961+0000 7f7d66a00640 1 --2- 172.21.15.39:0/2357079507 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7d48078b90 0x7f7d4807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:40.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.962+0000 7f7d66a00640 1 --2- 172.21.15.39:0/2357079507 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7d48078b90 0x7f7d4807b0a0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f7d58033000 tx=0x7f7d58004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:40.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:40.965+0000 7f7d5f400640 1 -- 172.21.15.39:0/2357079507 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7d54014030 con 0x7f7d680ccc60 2024-09-21T07:45:41.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.200+0000 7f7d6de00640 1 -- 172.21.15.39:0/2357079507 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d6815cb20 con 0x7f7d680ccc60 2024-09-21T07:45:41.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.200+0000 7f7d5f400640 1 -- 172.21.15.39:0/2357079507 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 181 v181) ==== 95+0+5234 (secure 0 0 0) 0x7f7d54063920 con 0x7f7d680ccc60 2024-09-21T07:45:41.202 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:45:41.203 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":181,"btime":"2024-09-21T07:45:40:695099+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7310,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2823960253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2823960253},{"type":"v1","addr":"172.21.15.39:6835","nonce":2823960253}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":176},{"gid":8155,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/603157278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":603157278},{"type":"v1","addr":"172.21.15.148:6836","nonce":603157278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":175},{"gid":8158,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2610135999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2610135999},{"type":"v1","addr":"172.21.15.148:6837","nonce":2610135999}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":176}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:45:37.186043+0000","modified":"2024-09-21T07:45:40.206770+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8829},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8829":{"gid":8829,"name":"a","rank":0,"incarnation":177,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/2389975689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2389975689},{"type":"v1","addr":"172.21.15.39:6834","nonce":2389975689}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8829,"qdb_cluster":[8829]},"id":8}]} 2024-09-21T07:45:41.203 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 181 2024-09-21T07:45:41.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.202+0000 7f7d6de00640 1 -- 172.21.15.39:0/2357079507 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7d48078b90 msgr2=0x7f7d4807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:41.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.202+0000 7f7d6de00640 1 --2- 172.21.15.39:0/2357079507 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7d48078b90 0x7f7d4807b0a0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f7d58033000 tx=0x7f7d58004730 comp rx=0 tx=0).stop 2024-09-21T07:45:41.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.202+0000 7f7d6de00640 1 -- 172.21.15.39:0/2357079507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d680ccc60 msgr2=0x7f7d680cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:41.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.202+0000 7f7d6de00640 1 --2- 172.21.15.39:0/2357079507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d680ccc60 0x7f7d680cea20 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f7d5400c970 tx=0x7f7d5400ce30 comp rx=0 tx=0).stop 2024-09-21T07:45:41.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.203+0000 7f7d6de00640 1 -- 172.21.15.39:0/2357079507 shutdown_connections 2024-09-21T07:45:41.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.203+0000 7f7d6de00640 1 --2- 172.21.15.39:0/2357079507 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7d48078b90 0x7f7d4807b0a0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:41.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.203+0000 7f7d6de00640 1 --2- 172.21.15.39:0/2357079507 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7d6815d830 0x7f7d681e93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:41.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.203+0000 7f7d6de00640 1 --2- 172.21.15.39:0/2357079507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d680ccc60 0x7f7d680cea20 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:41.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.203+0000 7f7d6de00640 1 --2- 172.21.15.39:0/2357079507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d680b30a0 0x7f7d681563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:41.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.203+0000 7f7d6de00640 1 -- 172.21.15.39:0/2357079507 >> 172.21.15.39:0/2357079507 conn(0x7f7d680b2460 msgr2=0x7f7d680b34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:41.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.203+0000 7f7d6de00640 1 -- 172.21.15.39:0/2357079507 shutdown_connections 2024-09-21T07:45:41.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.203+0000 7f7d6de00640 1 -- 172.21.15.39:0/2357079507 wait complete. 2024-09-21T07:45:41.227 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:45:41.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.451+0000 7fd493a00640 1 -- 172.21.15.39:0/1293687962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd48c0d0b70 msgr2=0x7fd48c0b43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:41.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.451+0000 7fd493a00640 1 --2- 172.21.15.39:0/1293687962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd48c0d0b70 0x7fd48c0b43d0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fd48c0b0000 tx=0x7fd48402a950 comp rx=0 tx=0).stop 2024-09-21T07:45:41.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.451+0000 7fd493a00640 1 -- 172.21.15.39:0/1293687962 shutdown_connections 2024-09-21T07:45:41.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.451+0000 7fd493a00640 1 --2- 172.21.15.39:0/1293687962 >> v2:172.21.15.148:3300/0 conn(0x7fd48c15c690 0x7fd48c0d0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:41.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.451+0000 7fd493a00640 1 --2- 172.21.15.39:0/1293687962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd48c0d0b70 0x7fd48c0b43d0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:41.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.451+0000 7fd493a00640 1 -- 172.21.15.39:0/1293687962 >> 172.21.15.39:0/1293687962 conn(0x7fd48c06cfc0 msgr2=0x7fd48c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:41.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.451+0000 7fd493a00640 1 -- 172.21.15.39:0/1293687962 shutdown_connections 2024-09-21T07:45:41.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.451+0000 7fd493a00640 1 -- 172.21.15.39:0/1293687962 wait complete. 2024-09-21T07:45:41.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.453+0000 7fd493a00640 1 Processor -- start 2024-09-21T07:45:41.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.454+0000 7fd493a00640 1 -- start start 2024-09-21T07:45:41.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.455+0000 7fd493a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd48c0b5240 0x7fd48c14f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:41.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.455+0000 7fd493a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd48c0d0b70 0x7fd48c0b0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:41.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.455+0000 7fd493a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd48c15c690 0x7fd48c15a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:41.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.455+0000 7fd493a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c070090 con 0x7fd48c0b5240 2024-09-21T07:45:41.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.455+0000 7fd493a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c06ff10 con 0x7fd48c0d0b70 2024-09-21T07:45:41.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.455+0000 7fd493a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd48c0b4910 con 0x7fd48c15c690 2024-09-21T07:45:41.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.455+0000 7fd491000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd48c0b5240 0x7fd48c14f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:41.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.455+0000 7fd491000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd48c0b5240 0x7fd48c14f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52252/0 (socket says 172.21.15.39:52252) 2024-09-21T07:45:41.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.455+0000 7fd491000640 1 -- 172.21.15.39:0/4226079182 learned_addr learned my addr 172.21.15.39:0/4226079182 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:41.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.456+0000 7fd483e00640 1 --2- 172.21.15.39:0/4226079182 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd48c0d0b70 0x7fd48c0b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:41.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.456+0000 7fd491a00640 1 --2- 172.21.15.39:0/4226079182 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd48c15c690 0x7fd48c15a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:41.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.456+0000 7fd491000640 1 -- 172.21.15.39:0/4226079182 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd48c15c690 msgr2=0x7fd48c15a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:41.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.456+0000 7fd491000640 1 --2- 172.21.15.39:0/4226079182 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd48c15c690 0x7fd48c15a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:41.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.456+0000 7fd491000640 1 -- 172.21.15.39:0/4226079182 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd48c0d0b70 msgr2=0x7fd48c0b0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:41.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.456+0000 7fd491000640 1 --2- 172.21.15.39:0/4226079182 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd48c0d0b70 0x7fd48c0b0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:41.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.456+0000 7fd491000640 1 -- 172.21.15.39:0/4226079182 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48c14f7d0 con 0x7fd48c0b5240 2024-09-21T07:45:41.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.456+0000 7fd483e00640 1 --2- 172.21.15.39:0/4226079182 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd48c0d0b70 0x7fd48c0b0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:45:41.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.456+0000 7fd491a00640 1 --2- 172.21.15.39:0/4226079182 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd48c15c690 0x7fd48c15a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:45:41.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.456+0000 7fd491000640 1 --2- 172.21.15.39:0/4226079182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd48c0b5240 0x7fd48c14f290 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fd47c00b420 tx=0x7fd47c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:41.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.456+0000 7fd489600640 1 -- 172.21.15.39:0/4226079182 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c011830 con 0x7fd48c0b5240 2024-09-21T07:45:41.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.456+0000 7fd489600640 1 -- 172.21.15.39:0/4226079182 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd47c016070 con 0x7fd48c0b5240 2024-09-21T07:45:41.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.456+0000 7fd489600640 1 -- 172.21.15.39:0/4226079182 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c011d60 con 0x7fd48c0b5240 2024-09-21T07:45:41.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.456+0000 7fd493a00640 1 -- 172.21.15.39:0/4226079182 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd48c1f5990 con 0x7fd48c0b5240 2024-09-21T07:45:41.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.457+0000 7fd493a00640 1 -- 172.21.15.39:0/4226079182 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd48c1f5f50 con 0x7fd48c0b5240 2024-09-21T07:45:41.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.458+0000 7fd493a00640 1 -- 172.21.15.39:0/4226079182 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd48c06a690 con 0x7fd48c0b5240 2024-09-21T07:45:41.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.459+0000 7fd489600640 1 -- 172.21.15.39:0/4226079182 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd47c01b460 con 0x7fd48c0b5240 2024-09-21T07:45:41.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.460+0000 7fd489600640 1 --2- 172.21.15.39:0/4226079182 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd468078bc0 0x7fd46807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:41.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.460+0000 7fd489600640 1 -- 172.21.15.39:0/4226079182 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 6844+0+0 (secure 0 0 0) 0x7fd47c09acb0 con 0x7fd48c0b5240 2024-09-21T07:45:41.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.460+0000 7fd483e00640 1 --2- 172.21.15.39:0/4226079182 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd468078bc0 0x7fd46807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:41.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.461+0000 7fd483e00640 1 --2- 172.21.15.39:0/4226079182 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd468078bc0 0x7fd46807b0d0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fd474009aa0 tx=0x7fd474005db0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:41.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.465+0000 7fd489600640 1 -- 172.21.15.39:0/4226079182 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd47c020020 con 0x7fd48c0b5240 2024-09-21T07:45:41.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.662+0000 7fd493a00640 1 -- 172.21.15.39:0/4226079182 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd48c0b4ea0 con 0x7fd48c0b5240 2024-09-21T07:45:41.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.664+0000 7fd489600640 1 -- 172.21.15.39:0/4226079182 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v157) ==== 74+0+22219 (secure 0 0 0) 0x7fd47c062be0 con 0x7fd48c0b5240 2024-09-21T07:45:41.666 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:45:41.666 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":157,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:45:38.675485+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:45:35.115582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-09-21T07:45:36.142685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:45:41.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.666+0000 7fd493a00640 1 -- 172.21.15.39:0/4226079182 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd468078bc0 msgr2=0x7fd46807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:41.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.666+0000 7fd493a00640 1 --2- 172.21.15.39:0/4226079182 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd468078bc0 0x7fd46807b0d0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fd474009aa0 tx=0x7fd474005db0 comp rx=0 tx=0).stop 2024-09-21T07:45:41.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.666+0000 7fd493a00640 1 -- 172.21.15.39:0/4226079182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd48c0b5240 msgr2=0x7fd48c14f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:41.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.666+0000 7fd493a00640 1 --2- 172.21.15.39:0/4226079182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd48c0b5240 0x7fd48c14f290 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fd47c00b420 tx=0x7fd47c00b8e0 comp rx=0 tx=0).stop 2024-09-21T07:45:41.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.667+0000 7fd493a00640 1 -- 172.21.15.39:0/4226079182 shutdown_connections 2024-09-21T07:45:41.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.667+0000 7fd493a00640 1 --2- 172.21.15.39:0/4226079182 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd468078bc0 0x7fd46807b0d0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:41.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.667+0000 7fd493a00640 1 --2- 172.21.15.39:0/4226079182 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd48c15c690 0x7fd48c15a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:41.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.667+0000 7fd493a00640 1 --2- 172.21.15.39:0/4226079182 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd48c0d0b70 0x7fd48c0b0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:41.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.667+0000 7fd493a00640 1 --2- 172.21.15.39:0/4226079182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd48c0b5240 0x7fd48c14f290 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:41.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.667+0000 7fd493a00640 1 -- 172.21.15.39:0/4226079182 >> 172.21.15.39:0/4226079182 conn(0x7fd48c06cfc0 msgr2=0x7fd48c0b2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:41.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.667+0000 7fd493a00640 1 -- 172.21.15.39:0/4226079182 shutdown_connections 2024-09-21T07:45:41.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.667+0000 7fd493a00640 1 -- 172.21.15.39:0/4226079182 wait complete. 2024-09-21T07:45:41.691 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T07:45:41.691 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:45:41.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.918+0000 7f4993200640 1 -- 172.21.15.39:0/2633812395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4974005150 msgr2=0x7f497400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:41.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.918+0000 7f4993200640 1 --2- 172.21.15.39:0/2633812395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4974005150 0x7f497400b9c0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f4988009ed0 tx=0x7f4974005830 comp rx=0 tx=0).stop 2024-09-21T07:45:41.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.918+0000 7f4992800640 1 -- 172.21.15.39:0/2633812395 reap_dead start 2024-09-21T07:45:41.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.918+0000 7f4993200640 1 -- 172.21.15.39:0/2633812395 shutdown_connections 2024-09-21T07:45:41.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.918+0000 7f4993200640 1 -- 172.21.15.39:0/2633812395 >> 172.21.15.39:0/2633812395 conn(0x7f498c0d0cb0 msgr2=0x7f498c0d10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:41.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.918+0000 7f4993200640 1 -- 172.21.15.39:0/2633812395 shutdown_connections 2024-09-21T07:45:41.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.918+0000 7f4993200640 1 -- 172.21.15.39:0/2633812395 wait complete. 2024-09-21T07:45:41.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.921+0000 7f4993200640 1 Processor -- start 2024-09-21T07:45:41.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.921+0000 7f4993200640 1 -- start start 2024-09-21T07:45:41.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.922+0000 7f4993200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4974005150 0x7f498c15abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:41.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.922+0000 7f4993200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f498c14e670 0x7f498c06cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:41.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.922+0000 7f4993200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f498c06d320 0x7f498c1563a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:41.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.922+0000 7f4993200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f498c06f8c0 con 0x7f498c14e670 2024-09-21T07:45:41.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.922+0000 7f4993200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f498c0b5d00 con 0x7f4974005150 2024-09-21T07:45:41.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.922+0000 7f4993200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f498c06fa40 con 0x7f498c06d320 2024-09-21T07:45:41.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.923+0000 7f4991400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f498c14e670 0x7f498c06cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:41.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.923+0000 7f4991400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f498c14e670 0x7f498c06cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52268/0 (socket says 172.21.15.39:52268) 2024-09-21T07:45:41.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.923+0000 7f4991400640 1 -- 172.21.15.39:0/3895613672 learned_addr learned my addr 172.21.15.39:0/3895613672 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:41.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.923+0000 7f4991e00640 1 --2- 172.21.15.39:0/3895613672 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4974005150 0x7f498c15abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:41.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.923+0000 7f4992800640 1 --2- 172.21.15.39:0/3895613672 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f498c06d320 0x7f498c1563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:41.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.923+0000 7f4991400640 1 -- 172.21.15.39:0/3895613672 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f498c06d320 msgr2=0x7f498c1563a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:41.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.923+0000 7f4991400640 1 --2- 172.21.15.39:0/3895613672 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f498c06d320 0x7f498c1563a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:41.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.923+0000 7f4991400640 1 -- 172.21.15.39:0/3895613672 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4974005150 msgr2=0x7f498c15abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:41.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.923+0000 7f4991400640 1 --2- 172.21.15.39:0/3895613672 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4974005150 0x7f498c15abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:41.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.923+0000 7f4991400640 1 -- 172.21.15.39:0/3895613672 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f498c06d730 con 0x7f498c14e670 2024-09-21T07:45:41.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.923+0000 7f4991400640 1 --2- 172.21.15.39:0/3895613672 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f498c14e670 0x7f498c06cde0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f497c00e6f0 tx=0x7f497c00eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:41.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.924+0000 7f4986000640 1 -- 172.21.15.39:0/3895613672 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497c004500 con 0x7f498c14e670 2024-09-21T07:45:41.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.924+0000 7f4986000640 1 -- 172.21.15.39:0/3895613672 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f497c016070 con 0x7f498c14e670 2024-09-21T07:45:41.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.924+0000 7f4993200640 1 -- 172.21.15.39:0/3895613672 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f498c1f6950 con 0x7f498c14e670 2024-09-21T07:45:41.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.924+0000 7f4986000640 1 -- 172.21.15.39:0/3895613672 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497c003980 con 0x7f498c14e670 2024-09-21T07:45:41.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.924+0000 7f4993200640 1 -- 172.21.15.39:0/3895613672 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f498c154f10 con 0x7f498c14e670 2024-09-21T07:45:41.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.925+0000 7f4993200640 1 -- 172.21.15.39:0/3895613672 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f498c0d0540 con 0x7f498c14e670 2024-09-21T07:45:41.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.926+0000 7f4986000640 1 -- 172.21.15.39:0/3895613672 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f497c003b20 con 0x7f498c14e670 2024-09-21T07:45:41.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.927+0000 7f4986000640 1 --2- 172.21.15.39:0/3895613672 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4970078bc0 0x7f497007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:41.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.927+0000 7f4986000640 1 -- 172.21.15.39:0/3895613672 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 6844+0+0 (secure 0 0 0) 0x7f497c09b640 con 0x7f498c14e670 2024-09-21T07:45:41.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.928+0000 7f4991e00640 1 --2- 172.21.15.39:0/3895613672 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4970078bc0 0x7f497007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:41.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.928+0000 7f4991e00640 1 --2- 172.21.15.39:0/3895613672 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4970078bc0 0x7f497007b0d0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f4980009c50 tx=0x7f4980005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:41.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:41.932+0000 7f4986000640 1 -- 172.21.15.39:0/3895613672 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f498c06d730 con 0x7f498c14e670 2024-09-21T07:45:42.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.125+0000 7f4993200640 1 -- 172.21.15.39:0/3895613672 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f498c1f6c10 con 0x7f4970078bc0 2024-09-21T07:45:42.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.134+0000 7f4986000640 1 -- 172.21.15.39:0/3895613672 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+490976 (secure 0 0 0) 0x7f498c1f6c10 con 0x7f4970078bc0 2024-09-21T07:45:42.142 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:45:42.145 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:45:42.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.139+0000 7f4993200640 1 -- 172.21.15.39:0/3895613672 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4970078bc0 msgr2=0x7f497007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:42.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.139+0000 7f4993200640 1 --2- 172.21.15.39:0/3895613672 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4970078bc0 0x7f497007b0d0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f4980009c50 tx=0x7f4980005f90 comp rx=0 tx=0).stop 2024-09-21T07:45:42.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.139+0000 7f4993200640 1 -- 172.21.15.39:0/3895613672 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f498c14e670 msgr2=0x7f498c06cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:42.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.139+0000 7f4993200640 1 --2- 172.21.15.39:0/3895613672 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f498c14e670 0x7f498c06cde0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f497c00e6f0 tx=0x7f497c00eaf0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.139+0000 7f4993200640 1 -- 172.21.15.39:0/3895613672 shutdown_connections 2024-09-21T07:45:42.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.139+0000 7f4993200640 1 --2- 172.21.15.39:0/3895613672 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4970078bc0 0x7f497007b0d0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.139+0000 7f4993200640 1 --2- 172.21.15.39:0/3895613672 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f498c06d320 0x7f498c1563a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.139+0000 7f4993200640 1 --2- 172.21.15.39:0/3895613672 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f498c14e670 0x7f498c06cde0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.139+0000 7f4993200640 1 --2- 172.21.15.39:0/3895613672 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4974005150 0x7f498c15abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.140+0000 7f4993200640 1 -- 172.21.15.39:0/3895613672 >> 172.21.15.39:0/3895613672 conn(0x7f498c0d0cb0 msgr2=0x7f498c15d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:42.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.140+0000 7f4993200640 1 -- 172.21.15.39:0/3895613672 shutdown_connections 2024-09-21T07:45:42.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.140+0000 7f4993200640 1 -- 172.21.15.39:0/3895613672 wait complete. 2024-09-21T07:45:42.165 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":532,"stamp":"2024-09-21T07:45:41.786929+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":285,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":303332,"kb_used_data":16360,"kb_used_omap":12,"kb_used_meta":286771,"kb_avail":754671388,"statfs":{"total":773094113280,"available":772783501312,"internally_reserved":0,"allocated":16752640,"data_stored":9322596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":293654064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001700"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.689192+0000","last_change":"2024-09-21T07:45:36.689192+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.689192+0000","last_undegraded":"2024-09-21T07:45:36.689192+0000","last_fullsized":"2024-09-21T07:45:36.689192+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.665560+0000","last_change":"2024-09-21T07:45:36.665560+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.665560+0000","last_undegraded":"2024-09-21T07:45:36.665560+0000","last_fullsized":"2024-09-21T07:45:36.665560+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.675392+0000","last_change":"2024-09-21T07:45:36.675392+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.675392+0000","last_undegraded":"2024-09-21T07:45:36.675392+0000","last_fullsized":"2024-09-21T07:45:36.675392+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.664845+0000","last_change":"2024-09-21T07:45:36.664845+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.664845+0000","last_undegraded":"2024-09-21T07:45:36.664845+0000","last_fullsized":"2024-09-21T07:45:36.664845+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689578+0000","last_change":"2024-09-21T07:45:37.706582+0000","last_active":"2024-09-21T07:45:38.689578+0000","last_peered":"2024-09-21T07:45:38.689578+0000","last_clean":"2024-09-21T07:45:38.689578+0000","last_became_active":"2024-09-21T07:45:37.706243+0000","last_became_peered":"2024-09-21T07:45:37.706243+0000","last_unstale":"2024-09-21T07:45:38.689578+0000","last_undegraded":"2024-09-21T07:45:38.689578+0000","last_fullsized":"2024-09-21T07:45:38.689578+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:20:35.055835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.676932+0000","last_change":"2024-09-21T07:45:36.676932+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.676932+0000","last_undegraded":"2024-09-21T07:45:36.676932+0000","last_fullsized":"2024-09-21T07:45:36.676932+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.674875+0000","last_change":"2024-09-21T07:45:36.674875+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.674875+0000","last_undegraded":"2024-09-21T07:45:36.674875+0000","last_fullsized":"2024-09-21T07:45:36.674875+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.687461+0000","last_change":"2024-09-21T07:45:37.694980+0000","last_active":"2024-09-21T07:45:38.687461+0000","last_peered":"2024-09-21T07:45:38.687461+0000","last_clean":"2024-09-21T07:45:38.687461+0000","last_became_active":"2024-09-21T07:45:37.689627+0000","last_became_peered":"2024-09-21T07:45:37.689627+0000","last_unstale":"2024-09-21T07:45:38.687461+0000","last_undegraded":"2024-09-21T07:45:38.687461+0000","last_fullsized":"2024-09-21T07:45:38.687461+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:32:36.418482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.677810+0000","last_change":"2024-09-21T07:45:36.677810+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.677810+0000","last_undegraded":"2024-09-21T07:45:36.677810+0000","last_fullsized":"2024-09-21T07:45:36.677810+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.680253+0000","last_change":"2024-09-21T07:45:36.680253+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.680253+0000","last_undegraded":"2024-09-21T07:45:36.680253+0000","last_fullsized":"2024-09-21T07:45:36.680253+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.664838+0000","last_change":"2024-09-21T07:45:36.664838+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.664838+0000","last_undegraded":"2024-09-21T07:45:36.664838+0000","last_fullsized":"2024-09-21T07:45:36.664838+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.688646+0000","last_change":"2024-09-21T07:45:36.688646+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.688646+0000","last_undegraded":"2024-09-21T07:45:36.688646+0000","last_fullsized":"2024-09-21T07:45:36.688646+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.676713+0000","last_change":"2024-09-21T07:45:36.676713+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.676713+0000","last_undegraded":"2024-09-21T07:45:36.676713+0000","last_fullsized":"2024-09-21T07:45:36.676713+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.687739+0000","last_change":"2024-09-21T07:45:36.687739+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.687739+0000","last_undegraded":"2024-09-21T07:45:36.687739+0000","last_fullsized":"2024-09-21T07:45:36.687739+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.684481+0000","last_change":"2024-09-21T07:45:36.684481+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.684481+0000","last_undegraded":"2024-09-21T07:45:36.684481+0000","last_fullsized":"2024-09-21T07:45:36.684481+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.677447+0000","last_change":"2024-09-21T07:45:36.677447+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.677447+0000","last_undegraded":"2024-09-21T07:45:36.677447+0000","last_fullsized":"2024-09-21T07:45:36.677447+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.677254+0000","last_change":"2024-09-21T07:45:36.677254+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.677254+0000","last_undegraded":"2024-09-21T07:45:36.677254+0000","last_fullsized":"2024-09-21T07:45:36.677254+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689982+0000","last_change":"2024-09-21T07:45:37.704608+0000","last_active":"2024-09-21T07:45:38.689982+0000","last_peered":"2024-09-21T07:45:38.689982+0000","last_clean":"2024-09-21T07:45:38.689982+0000","last_became_active":"2024-09-21T07:45:37.703051+0000","last_became_peered":"2024-09-21T07:45:37.703051+0000","last_unstale":"2024-09-21T07:45:38.689982+0000","last_undegraded":"2024-09-21T07:45:38.689982+0000","last_fullsized":"2024-09-21T07:45:38.689982+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:33.889448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.675086+0000","last_change":"2024-09-21T07:45:36.675086+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.675086+0000","last_undegraded":"2024-09-21T07:45:36.675086+0000","last_fullsized":"2024-09-21T07:45:36.675086+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689772+0000","last_change":"2024-09-21T07:45:37.698151+0000","last_active":"2024-09-21T07:45:38.689772+0000","last_peered":"2024-09-21T07:45:38.689772+0000","last_clean":"2024-09-21T07:45:38.689772+0000","last_became_active":"2024-09-21T07:45:37.697818+0000","last_became_peered":"2024-09-21T07:45:37.697818+0000","last_unstale":"2024-09-21T07:45:38.689772+0000","last_undegraded":"2024-09-21T07:45:38.689772+0000","last_fullsized":"2024-09-21T07:45:38.689772+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:57.867727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.664937+0000","last_change":"2024-09-21T07:45:36.664937+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.664937+0000","last_undegraded":"2024-09-21T07:45:36.664937+0000","last_fullsized":"2024-09-21T07:45:36.664937+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.691525+0000","last_change":"2024-09-21T07:45:36.691525+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.691525+0000","last_undegraded":"2024-09-21T07:45:36.691525+0000","last_fullsized":"2024-09-21T07:45:36.691525+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.666008+0000","last_change":"2024-09-21T07:45:36.666008+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.666008+0000","last_undegraded":"2024-09-21T07:45:36.666008+0000","last_fullsized":"2024-09-21T07:45:36.666008+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.676435+0000","last_change":"2024-09-21T07:45:36.676435+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.676435+0000","last_undegraded":"2024-09-21T07:45:36.676435+0000","last_fullsized":"2024-09-21T07:45:36.676435+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.675973+0000","last_change":"2024-09-21T07:45:36.675973+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.675973+0000","last_undegraded":"2024-09-21T07:45:36.675973+0000","last_fullsized":"2024-09-21T07:45:36.675973+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.671245+0000","last_change":"2024-09-21T07:45:36.671245+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.671245+0000","last_undegraded":"2024-09-21T07:45:36.671245+0000","last_fullsized":"2024-09-21T07:45:36.671245+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.671879+0000","last_change":"2024-09-21T07:45:36.671879+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.671879+0000","last_undegraded":"2024-09-21T07:45:36.671879+0000","last_fullsized":"2024-09-21T07:45:36.671879+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.681606+0000","last_change":"2024-09-21T07:45:36.681606+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.681606+0000","last_undegraded":"2024-09-21T07:45:36.681606+0000","last_fullsized":"2024-09-21T07:45:36.681606+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.676579+0000","last_change":"2024-09-21T07:45:36.676579+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.676579+0000","last_undegraded":"2024-09-21T07:45:36.676579+0000","last_fullsized":"2024-09-21T07:45:36.676579+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.675479+0000","last_change":"2024-09-21T07:45:36.675479+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.675479+0000","last_undegraded":"2024-09-21T07:45:36.675479+0000","last_fullsized":"2024-09-21T07:45:36.675479+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.687575+0000","last_change":"2024-09-21T07:45:37.685098+0000","last_active":"2024-09-21T07:45:38.687575+0000","last_peered":"2024-09-21T07:45:38.687575+0000","last_clean":"2024-09-21T07:45:38.687575+0000","last_became_active":"2024-09-21T07:45:37.684748+0000","last_became_peered":"2024-09-21T07:45:37.684748+0000","last_unstale":"2024-09-21T07:45:38.687575+0000","last_undegraded":"2024-09-21T07:45:38.687575+0000","last_fullsized":"2024-09-21T07:45:38.687575+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:51:22.048251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.681462+0000","last_change":"2024-09-21T07:45:36.681462+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.681462+0000","last_undegraded":"2024-09-21T07:45:36.681462+0000","last_fullsized":"2024-09-21T07:45:36.681462+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.671410+0000","last_change":"2024-09-21T07:45:36.671410+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.671410+0000","last_undegraded":"2024-09-21T07:45:36.671410+0000","last_fullsized":"2024-09-21T07:45:36.671410+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.699039+0000","last_change":"2024-09-21T07:45:36.697679+0000","last_active":"2024-09-21T07:45:36.699039+0000","last_peered":"2024-09-21T07:45:36.699039+0000","last_clean":"2024-09-21T07:45:36.699039+0000","last_became_active":"2024-09-21T07:45:36.697396+0000","last_became_peered":"2024-09-21T07:45:36.697396+0000","last_unstale":"2024-09-21T07:45:36.699039+0000","last_undegraded":"2024-09-21T07:45:36.699039+0000","last_fullsized":"2024-09-21T07:45:36.699039+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:43:37.693480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":187,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.679060+0000","last_change":"2024-09-21T07:35:27.879605+0000","last_active":"2024-09-21T07:45:36.679060+0000","last_peered":"2024-09-21T07:45:36.679060+0000","last_clean":"2024-09-21T07:45:36.679060+0000","last_became_active":"2024-09-21T07:35:27.879274+0000","last_became_peered":"2024-09-21T07:35:27.879274+0000","last_unstale":"2024-09-21T07:45:36.679060+0000","last_undegraded":"2024-09-21T07:45:36.679060+0000","last_fullsized":"2024-09-21T07:45:36.679060+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:24.241636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":197,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689112+0000","last_change":"2024-09-21T07:35:29.491336+0000","last_active":"2024-09-21T07:45:38.689112+0000","last_peered":"2024-09-21T07:45:38.689112+0000","last_clean":"2024-09-21T07:45:38.689112+0000","last_became_active":"2024-09-21T07:35:26.866484+0000","last_became_peered":"2024-09-21T07:35:26.866484+0000","last_unstale":"2024-09-21T07:45:38.689112+0000","last_undegraded":"2024-09-21T07:45:38.689112+0000","last_fullsized":"2024-09-21T07:45:38.689112+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:10.449665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076010100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":373,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.679314+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:45:36.679314+0000","last_peered":"2024-09-21T07:45:36.679314+0000","last_clean":"2024-09-21T07:45:36.679314+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:45:36.679314+0000","last_undegraded":"2024-09-21T07:45:36.679314+0000","last_fullsized":"2024-09-21T07:45:36.679314+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":252,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.670156+0000","last_change":"2024-09-21T07:34:24.539248+0000","last_active":"2024-09-21T07:45:36.670156+0000","last_peered":"2024-09-21T07:45:36.670156+0000","last_clean":"2024-09-21T07:45:36.670156+0000","last_became_active":"2024-09-21T07:34:23.273465+0000","last_became_peered":"2024-09-21T07:34:23.273465+0000","last_unstale":"2024-09-21T07:45:36.670156+0000","last_undegraded":"2024-09-21T07:45:36.670156+0000","last_fullsized":"2024-09-21T07:45:36.670156+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:55.517452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060627700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.686033+0000","last_change":"2024-09-21T07:45:36.686033+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.686033+0000","last_undegraded":"2024-09-21T07:45:36.686033+0000","last_fullsized":"2024-09-21T07:45:36.686033+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.701635+0000","last_change":"2024-09-21T07:45:36.700703+0000","last_active":"2024-09-21T07:45:36.701635+0000","last_peered":"2024-09-21T07:45:36.701635+0000","last_clean":"2024-09-21T07:45:36.701635+0000","last_became_active":"2024-09-21T07:45:36.700344+0000","last_became_peered":"2024-09-21T07:45:36.700344+0000","last_unstale":"2024-09-21T07:45:36.701635+0000","last_undegraded":"2024-09-21T07:45:36.701635+0000","last_fullsized":"2024-09-21T07:45:36.701635+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:59.539139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689081+0000","last_change":"2024-09-21T07:35:27.864492+0000","last_active":"2024-09-21T07:45:38.689081+0000","last_peered":"2024-09-21T07:45:38.689081+0000","last_clean":"2024-09-21T07:45:38.689081+0000","last_became_active":"2024-09-21T07:35:27.864176+0000","last_became_peered":"2024-09-21T07:35:27.864176+0000","last_unstale":"2024-09-21T07:45:38.689081+0000","last_undegraded":"2024-09-21T07:45:38.689081+0000","last_fullsized":"2024-09-21T07:45:38.689081+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:05.153467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.680405+0000","last_change":"2024-09-21T07:35:29.493678+0000","last_active":"2024-09-21T07:45:36.680405+0000","last_peered":"2024-09-21T07:45:36.680405+0000","last_clean":"2024-09-21T07:45:36.680405+0000","last_became_active":"2024-09-21T07:35:26.870749+0000","last_became_peered":"2024-09-21T07:35:26.870749+0000","last_unstale":"2024-09-21T07:45:36.680405+0000","last_undegraded":"2024-09-21T07:45:36.680405+0000","last_fullsized":"2024-09-21T07:45:36.680405+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:55.367877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010159570000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":246,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688084+0000","last_change":"2024-09-21T07:34:24.597274+0000","last_active":"2024-09-21T07:45:38.688084+0000","last_peered":"2024-09-21T07:45:38.688084+0000","last_clean":"2024-09-21T07:45:38.688084+0000","last_became_active":"2024-09-21T07:34:23.279560+0000","last_became_peered":"2024-09-21T07:34:23.279560+0000","last_unstale":"2024-09-21T07:45:38.688084+0000","last_undegraded":"2024-09-21T07:45:38.688084+0000","last_fullsized":"2024-09-21T07:45:38.688084+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:15.845265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059699799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.676108+0000","last_change":"2024-09-21T07:45:36.676108+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.676108+0000","last_undegraded":"2024-09-21T07:45:36.676108+0000","last_fullsized":"2024-09-21T07:45:36.676108+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.698291+0000","last_change":"2024-09-21T07:45:36.695689+0000","last_active":"2024-09-21T07:45:36.698291+0000","last_peered":"2024-09-21T07:45:36.698291+0000","last_clean":"2024-09-21T07:45:36.698291+0000","last_became_active":"2024-09-21T07:45:36.695386+0000","last_became_peered":"2024-09-21T07:45:36.695386+0000","last_unstale":"2024-09-21T07:45:36.698291+0000","last_undegraded":"2024-09-21T07:45:36.698291+0000","last_fullsized":"2024-09-21T07:45:36.698291+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:13.324095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":187,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.693357+0000","last_change":"2024-09-21T07:35:27.874387+0000","last_active":"2024-09-21T07:45:36.693357+0000","last_peered":"2024-09-21T07:45:36.693357+0000","last_clean":"2024-09-21T07:45:36.693357+0000","last_became_active":"2024-09-21T07:35:27.873277+0000","last_became_peered":"2024-09-21T07:35:27.873277+0000","last_unstale":"2024-09-21T07:45:36.693357+0000","last_undegraded":"2024-09-21T07:45:36.693357+0000","last_fullsized":"2024-09-21T07:45:36.693357+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:04.581463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":196,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688673+0000","last_change":"2024-09-21T07:35:29.387094+0000","last_active":"2024-09-21T07:45:38.688673+0000","last_peered":"2024-09-21T07:45:38.688673+0000","last_clean":"2024-09-21T07:45:38.688673+0000","last_became_active":"2024-09-21T07:35:26.874757+0000","last_became_peered":"2024-09-21T07:35:26.874757+0000","last_unstale":"2024-09-21T07:45:38.688673+0000","last_undegraded":"2024-09-21T07:45:38.688673+0000","last_fullsized":"2024-09-21T07:45:38.688673+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:33.542750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061065700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":248,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.678663+0000","last_change":"2024-09-21T07:34:24.259906+0000","last_active":"2024-09-21T07:45:36.678663+0000","last_peered":"2024-09-21T07:45:36.678663+0000","last_clean":"2024-09-21T07:45:36.678663+0000","last_became_active":"2024-09-21T07:34:23.635282+0000","last_became_peered":"2024-09-21T07:34:23.635282+0000","last_unstale":"2024-09-21T07:45:36.678663+0000","last_undegraded":"2024-09-21T07:45:36.678663+0000","last_fullsized":"2024-09-21T07:45:36.678663+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:00.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001527664,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.670461+0000","last_change":"2024-09-21T07:45:36.670461+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.670461+0000","last_undegraded":"2024-09-21T07:45:36.670461+0000","last_fullsized":"2024-09-21T07:45:36.670461+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.689405+0000","last_change":"2024-09-21T07:45:36.688037+0000","last_active":"2024-09-21T07:45:36.689405+0000","last_peered":"2024-09-21T07:45:36.689405+0000","last_clean":"2024-09-21T07:45:36.689405+0000","last_became_active":"2024-09-21T07:45:36.687731+0000","last_became_peered":"2024-09-21T07:45:36.687731+0000","last_unstale":"2024-09-21T07:45:36.689405+0000","last_undegraded":"2024-09-21T07:45:36.689405+0000","last_fullsized":"2024-09-21T07:45:36.689405+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:31.034405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":187,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.696504+0000","last_change":"2024-09-21T07:35:27.873645+0000","last_active":"2024-09-21T07:45:36.696504+0000","last_peered":"2024-09-21T07:45:36.696504+0000","last_clean":"2024-09-21T07:45:36.696504+0000","last_became_active":"2024-09-21T07:35:27.873245+0000","last_became_peered":"2024-09-21T07:35:27.873245+0000","last_unstale":"2024-09-21T07:45:36.696504+0000","last_undegraded":"2024-09-21T07:45:36.696504+0000","last_fullsized":"2024-09-21T07:45:36.696504+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:55.090953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.698882+0000","last_change":"2024-09-21T07:35:28.861744+0000","last_active":"2024-09-21T07:45:36.698882+0000","last_peered":"2024-09-21T07:45:36.698882+0000","last_clean":"2024-09-21T07:45:36.698882+0000","last_became_active":"2024-09-21T07:35:26.876462+0000","last_became_peered":"2024-09-21T07:35:26.876462+0000","last_unstale":"2024-09-21T07:45:36.698882+0000","last_undegraded":"2024-09-21T07:45:36.698882+0000","last_fullsized":"2024-09-21T07:45:36.698882+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:48.632568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012020920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":245,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.672276+0000","last_change":"2024-09-21T07:34:25.085454+0000","last_active":"2024-09-21T07:45:36.672276+0000","last_peered":"2024-09-21T07:45:36.672276+0000","last_clean":"2024-09-21T07:45:36.672276+0000","last_became_active":"2024-09-21T07:34:23.279205+0000","last_became_peered":"2024-09-21T07:34:23.279205+0000","last_unstale":"2024-09-21T07:45:36.672276+0000","last_undegraded":"2024-09-21T07:45:36.672276+0000","last_fullsized":"2024-09-21T07:45:36.672276+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:54.231666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095320599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.675896+0000","last_change":"2024-09-21T07:45:36.675896+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.675896+0000","last_undegraded":"2024-09-21T07:45:36.675896+0000","last_fullsized":"2024-09-21T07:45:36.675896+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.701499+0000","last_change":"2024-09-21T07:45:36.700878+0000","last_active":"2024-09-21T07:45:36.701499+0000","last_peered":"2024-09-21T07:45:36.701499+0000","last_clean":"2024-09-21T07:45:36.701499+0000","last_became_active":"2024-09-21T07:45:36.700484+0000","last_became_peered":"2024-09-21T07:45:36.700484+0000","last_unstale":"2024-09-21T07:45:36.701499+0000","last_undegraded":"2024-09-21T07:45:36.701499+0000","last_fullsized":"2024-09-21T07:45:36.701499+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:52:12.375777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688964+0000","last_change":"2024-09-21T07:35:27.861373+0000","last_active":"2024-09-21T07:45:38.688964+0000","last_peered":"2024-09-21T07:45:38.688964+0000","last_clean":"2024-09-21T07:45:38.688964+0000","last_became_active":"2024-09-21T07:35:27.861016+0000","last_became_peered":"2024-09-21T07:35:27.861016+0000","last_unstale":"2024-09-21T07:45:38.688964+0000","last_undegraded":"2024-09-21T07:45:38.688964+0000","last_fullsized":"2024-09-21T07:45:38.688964+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:56.043763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":195,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.673133+0000","last_change":"2024-09-21T07:35:29.053677+0000","last_active":"2024-09-21T07:45:36.673133+0000","last_peered":"2024-09-21T07:45:36.673133+0000","last_clean":"2024-09-21T07:45:36.673133+0000","last_became_active":"2024-09-21T07:35:26.875496+0000","last_became_peered":"2024-09-21T07:35:26.875496+0000","last_unstale":"2024-09-21T07:45:36.673133+0000","last_undegraded":"2024-09-21T07:45:36.673133+0000","last_fullsized":"2024-09-21T07:45:36.673133+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:04.850784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065859000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":247,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.672104+0000","last_change":"2024-09-21T07:34:25.028521+0000","last_active":"2024-09-21T07:45:36.672104+0000","last_peered":"2024-09-21T07:45:36.672104+0000","last_clean":"2024-09-21T07:45:36.672104+0000","last_became_active":"2024-09-21T07:34:23.635467+0000","last_became_peered":"2024-09-21T07:34:23.635467+0000","last_unstale":"2024-09-21T07:45:36.672104+0000","last_undegraded":"2024-09-21T07:45:36.672104+0000","last_fullsized":"2024-09-21T07:45:36.672104+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:33.729123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000660558,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.676454+0000","last_change":"2024-09-21T07:45:36.676454+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.676454+0000","last_undegraded":"2024-09-21T07:45:36.676454+0000","last_fullsized":"2024-09-21T07:45:36.676454+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.699376+0000","last_change":"2024-09-21T07:45:36.697573+0000","last_active":"2024-09-21T07:45:36.699376+0000","last_peered":"2024-09-21T07:45:36.699376+0000","last_clean":"2024-09-21T07:45:36.699376+0000","last_became_active":"2024-09-21T07:45:36.697341+0000","last_became_peered":"2024-09-21T07:45:36.697341+0000","last_unstale":"2024-09-21T07:45:36.699376+0000","last_undegraded":"2024-09-21T07:45:36.699376+0000","last_fullsized":"2024-09-21T07:45:36.699376+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:18.613601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":187,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.676705+0000","last_change":"2024-09-21T07:35:27.871201+0000","last_active":"2024-09-21T07:45:36.676705+0000","last_peered":"2024-09-21T07:45:36.676705+0000","last_clean":"2024-09-21T07:45:36.676705+0000","last_became_active":"2024-09-21T07:35:27.870816+0000","last_became_peered":"2024-09-21T07:35:27.870816+0000","last_unstale":"2024-09-21T07:45:36.676705+0000","last_undegraded":"2024-09-21T07:45:36.676705+0000","last_fullsized":"2024-09-21T07:45:36.676705+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:04.787450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":197,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689594+0000","last_change":"2024-09-21T07:35:29.491874+0000","last_active":"2024-09-21T07:45:38.689594+0000","last_peered":"2024-09-21T07:45:38.689594+0000","last_clean":"2024-09-21T07:45:38.689594+0000","last_became_active":"2024-09-21T07:35:26.866381+0000","last_became_peered":"2024-09-21T07:35:26.866381+0000","last_unstale":"2024-09-21T07:45:38.689594+0000","last_undegraded":"2024-09-21T07:45:38.689594+0000","last_fullsized":"2024-09-21T07:45:38.689594+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:36.404647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073235399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":252,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690345+0000","last_change":"2024-09-21T07:34:24.260046+0000","last_active":"2024-09-21T07:45:38.690345+0000","last_peered":"2024-09-21T07:45:38.690345+0000","last_clean":"2024-09-21T07:45:38.690345+0000","last_became_active":"2024-09-21T07:34:23.273431+0000","last_became_peered":"2024-09-21T07:34:23.273431+0000","last_unstale":"2024-09-21T07:45:38.690345+0000","last_undegraded":"2024-09-21T07:45:38.690345+0000","last_fullsized":"2024-09-21T07:45:38.690345+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:18.876057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019803310000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.674849+0000","last_change":"2024-09-21T07:45:36.674849+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.674849+0000","last_undegraded":"2024-09-21T07:45:36.674849+0000","last_fullsized":"2024-09-21T07:45:36.674849+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.687046+0000","last_change":"2024-09-21T07:45:36.684575+0000","last_active":"2024-09-21T07:45:36.687046+0000","last_peered":"2024-09-21T07:45:36.687046+0000","last_clean":"2024-09-21T07:45:36.687046+0000","last_became_active":"2024-09-21T07:45:36.683843+0000","last_became_peered":"2024-09-21T07:45:36.683843+0000","last_unstale":"2024-09-21T07:45:36.687046+0000","last_undegraded":"2024-09-21T07:45:36.687046+0000","last_fullsized":"2024-09-21T07:45:36.687046+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:56.463705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":187,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.676204+0000","last_change":"2024-09-21T07:35:27.871736+0000","last_active":"2024-09-21T07:45:36.676204+0000","last_peered":"2024-09-21T07:45:36.676204+0000","last_clean":"2024-09-21T07:45:36.676204+0000","last_became_active":"2024-09-21T07:35:27.871232+0000","last_became_peered":"2024-09-21T07:35:27.871232+0000","last_unstale":"2024-09-21T07:45:36.676204+0000","last_undegraded":"2024-09-21T07:45:36.676204+0000","last_fullsized":"2024-09-21T07:45:36.676204+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:04.419301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":197,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688913+0000","last_change":"2024-09-21T07:35:29.491246+0000","last_active":"2024-09-21T07:45:38.688913+0000","last_peered":"2024-09-21T07:45:38.688913+0000","last_clean":"2024-09-21T07:45:38.688913+0000","last_became_active":"2024-09-21T07:35:26.866783+0000","last_became_peered":"2024-09-21T07:35:26.866783+0000","last_unstale":"2024-09-21T07:45:38.688913+0000","last_undegraded":"2024-09-21T07:45:38.688913+0000","last_fullsized":"2024-09-21T07:45:38.688913+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:28.881914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098474099999999992,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":247,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.673249+0000","last_change":"2024-09-21T07:34:25.028640+0000","last_active":"2024-09-21T07:45:36.673249+0000","last_peered":"2024-09-21T07:45:36.673249+0000","last_clean":"2024-09-21T07:45:36.673249+0000","last_became_active":"2024-09-21T07:34:23.260950+0000","last_became_peered":"2024-09-21T07:34:23.260950+0000","last_unstale":"2024-09-21T07:45:36.673249+0000","last_undegraded":"2024-09-21T07:45:36.673249+0000","last_fullsized":"2024-09-21T07:45:36.673249+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:23:42.121398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055135099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689869+0000","last_change":"2024-09-21T07:45:37.693385+0000","last_active":"2024-09-21T07:45:38.689869+0000","last_peered":"2024-09-21T07:45:38.689869+0000","last_clean":"2024-09-21T07:45:38.689869+0000","last_became_active":"2024-09-21T07:45:37.691927+0000","last_became_peered":"2024-09-21T07:45:37.691927+0000","last_unstale":"2024-09-21T07:45:38.689869+0000","last_undegraded":"2024-09-21T07:45:38.689869+0000","last_fullsized":"2024-09-21T07:45:38.689869+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:29.208442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.684490+0000","last_change":"2024-09-21T07:45:36.681956+0000","last_active":"2024-09-21T07:45:36.684490+0000","last_peered":"2024-09-21T07:45:36.684490+0000","last_clean":"2024-09-21T07:45:36.684490+0000","last_became_active":"2024-09-21T07:45:36.681656+0000","last_became_peered":"2024-09-21T07:45:36.681656+0000","last_unstale":"2024-09-21T07:45:36.684490+0000","last_undegraded":"2024-09-21T07:45:36.684490+0000","last_fullsized":"2024-09-21T07:45:36.684490+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:04:54.765401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":186,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.680978+0000","last_change":"2024-09-21T07:35:27.872027+0000","last_active":"2024-09-21T07:45:36.680978+0000","last_peered":"2024-09-21T07:45:36.680978+0000","last_clean":"2024-09-21T07:45:36.680978+0000","last_became_active":"2024-09-21T07:35:27.870426+0000","last_became_peered":"2024-09-21T07:35:27.870426+0000","last_unstale":"2024-09-21T07:45:36.680978+0000","last_undegraded":"2024-09-21T07:45:36.680978+0000","last_fullsized":"2024-09-21T07:45:36.680978+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":197,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689126+0000","last_change":"2024-09-21T07:35:29.491008+0000","last_active":"2024-09-21T07:45:38.689126+0000","last_peered":"2024-09-21T07:45:38.689126+0000","last_clean":"2024-09-21T07:45:38.689126+0000","last_became_active":"2024-09-21T07:35:26.866750+0000","last_became_peered":"2024-09-21T07:35:26.866750+0000","last_unstale":"2024-09-21T07:45:38.689126+0000","last_undegraded":"2024-09-21T07:45:38.689126+0000","last_fullsized":"2024-09-21T07:45:38.689126+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:25.490406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083529700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":248,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.679720+0000","last_change":"2024-09-21T07:34:24.260665+0000","last_active":"2024-09-21T07:45:36.679720+0000","last_peered":"2024-09-21T07:45:36.679720+0000","last_clean":"2024-09-21T07:45:36.679720+0000","last_became_active":"2024-09-21T07:34:23.635909+0000","last_became_peered":"2024-09-21T07:34:23.635909+0000","last_unstale":"2024-09-21T07:45:36.679720+0000","last_undegraded":"2024-09-21T07:45:36.679720+0000","last_fullsized":"2024-09-21T07:45:36.679720+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:44.536885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689417+0000","last_change":"2024-09-21T07:45:37.686269+0000","last_active":"2024-09-21T07:45:38.689417+0000","last_peered":"2024-09-21T07:45:38.689417+0000","last_clean":"2024-09-21T07:45:38.689417+0000","last_became_active":"2024-09-21T07:45:37.685971+0000","last_became_peered":"2024-09-21T07:45:37.685971+0000","last_unstale":"2024-09-21T07:45:38.689417+0000","last_undegraded":"2024-09-21T07:45:38.689417+0000","last_fullsized":"2024-09-21T07:45:38.689417+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:23.017715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.696537+0000","last_change":"2024-09-21T07:45:36.695413+0000","last_active":"2024-09-21T07:45:36.696537+0000","last_peered":"2024-09-21T07:45:36.696537+0000","last_clean":"2024-09-21T07:45:36.696537+0000","last_became_active":"2024-09-21T07:45:36.695045+0000","last_became_peered":"2024-09-21T07:45:36.695045+0000","last_unstale":"2024-09-21T07:45:36.696537+0000","last_undegraded":"2024-09-21T07:45:36.696537+0000","last_fullsized":"2024-09-21T07:45:36.696537+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:32.061491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689492+0000","last_change":"2024-09-21T07:35:27.874772+0000","last_active":"2024-09-21T07:45:38.689492+0000","last_peered":"2024-09-21T07:45:38.689492+0000","last_clean":"2024-09-21T07:45:38.689492+0000","last_became_active":"2024-09-21T07:35:27.874434+0000","last_became_peered":"2024-09-21T07:35:27.874434+0000","last_unstale":"2024-09-21T07:45:38.689492+0000","last_undegraded":"2024-09-21T07:45:38.689492+0000","last_fullsized":"2024-09-21T07:45:38.689492+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:31.998755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":199,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.669713+0000","last_change":"2024-09-21T07:35:28.859153+0000","last_active":"2024-09-21T07:45:36.669713+0000","last_peered":"2024-09-21T07:45:36.669713+0000","last_clean":"2024-09-21T07:45:36.669713+0000","last_became_active":"2024-09-21T07:35:26.874428+0000","last_became_peered":"2024-09-21T07:35:26.874428+0000","last_unstale":"2024-09-21T07:45:36.669713+0000","last_undegraded":"2024-09-21T07:45:36.669713+0000","last_fullsized":"2024-09-21T07:45:36.669713+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:04.487621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048557299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":252,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.669176+0000","last_change":"2024-09-21T07:34:24.538845+0000","last_active":"2024-09-21T07:45:36.669176+0000","last_peered":"2024-09-21T07:45:36.669176+0000","last_clean":"2024-09-21T07:45:36.669176+0000","last_became_active":"2024-09-21T07:34:23.636372+0000","last_became_peered":"2024-09-21T07:34:23.636372+0000","last_unstale":"2024-09-21T07:45:36.669176+0000","last_undegraded":"2024-09-21T07:45:36.669176+0000","last_fullsized":"2024-09-21T07:45:36.669176+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:29.393813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841451,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.683953+0000","last_change":"2024-09-21T07:45:36.683953+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.683953+0000","last_undegraded":"2024-09-21T07:45:36.683953+0000","last_fullsized":"2024-09-21T07:45:36.683953+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"156'1","reported_seq":16,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690230+0000","last_change":"2024-09-21T07:45:36.698933+0000","last_active":"2024-09-21T07:45:38.690230+0000","last_peered":"2024-09-21T07:45:38.690230+0000","last_clean":"2024-09-21T07:45:38.690230+0000","last_became_active":"2024-09-21T07:45:36.698651+0000","last_became_peered":"2024-09-21T07:45:36.698651+0000","last_unstale":"2024-09-21T07:45:38.690230+0000","last_undegraded":"2024-09-21T07:45:38.690230+0000","last_fullsized":"2024-09-21T07:45:38.690230+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:17:31.377323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":186,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.677861+0000","last_change":"2024-09-21T07:35:27.877801+0000","last_active":"2024-09-21T07:45:36.677861+0000","last_peered":"2024-09-21T07:45:36.677861+0000","last_clean":"2024-09-21T07:45:36.677861+0000","last_became_active":"2024-09-21T07:35:27.877464+0000","last_became_peered":"2024-09-21T07:35:27.877464+0000","last_unstale":"2024-09-21T07:45:36.677861+0000","last_undegraded":"2024-09-21T07:45:36.677861+0000","last_fullsized":"2024-09-21T07:45:36.677861+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:08.814929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.676325+0000","last_change":"2024-09-21T07:35:28.863732+0000","last_active":"2024-09-21T07:45:36.676325+0000","last_peered":"2024-09-21T07:45:36.676325+0000","last_clean":"2024-09-21T07:45:36.676325+0000","last_became_active":"2024-09-21T07:35:26.877392+0000","last_became_peered":"2024-09-21T07:35:26.877392+0000","last_unstale":"2024-09-21T07:45:36.676325+0000","last_undegraded":"2024-09-21T07:45:36.676325+0000","last_fullsized":"2024-09-21T07:45:36.676325+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071425700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":252,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689803+0000","last_change":"2024-09-21T07:34:24.259686+0000","last_active":"2024-09-21T07:45:38.689803+0000","last_peered":"2024-09-21T07:45:38.689803+0000","last_clean":"2024-09-21T07:45:38.689803+0000","last_became_active":"2024-09-21T07:34:23.274405+0000","last_became_peered":"2024-09-21T07:34:23.274405+0000","last_unstale":"2024-09-21T07:45:38.689803+0000","last_undegraded":"2024-09-21T07:45:38.689803+0000","last_fullsized":"2024-09-21T07:45:38.689803+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:26.304693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022009920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.664831+0000","last_change":"2024-09-21T07:45:36.664831+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.664831+0000","last_undegraded":"2024-09-21T07:45:36.664831+0000","last_fullsized":"2024-09-21T07:45:36.664831+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.692374+0000","last_change":"2024-09-21T07:45:36.689252+0000","last_active":"2024-09-21T07:45:36.692374+0000","last_peered":"2024-09-21T07:45:36.692374+0000","last_clean":"2024-09-21T07:45:36.692374+0000","last_became_active":"2024-09-21T07:45:36.688915+0000","last_became_peered":"2024-09-21T07:45:36.688915+0000","last_unstale":"2024-09-21T07:45:36.692374+0000","last_undegraded":"2024-09-21T07:45:36.692374+0000","last_fullsized":"2024-09-21T07:45:36.692374+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:41:51.910757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":189,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.687721+0000","last_change":"2024-09-21T07:35:27.867336+0000","last_active":"2024-09-21T07:45:38.687721+0000","last_peered":"2024-09-21T07:45:38.687721+0000","last_clean":"2024-09-21T07:45:38.687721+0000","last_became_active":"2024-09-21T07:35:27.867015+0000","last_became_peered":"2024-09-21T07:35:27.867015+0000","last_unstale":"2024-09-21T07:45:38.687721+0000","last_undegraded":"2024-09-21T07:45:38.687721+0000","last_fullsized":"2024-09-21T07:45:38.687721+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:59.214139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.679176+0000","last_change":"2024-09-21T07:35:29.493137+0000","last_active":"2024-09-21T07:45:36.679176+0000","last_peered":"2024-09-21T07:45:36.679176+0000","last_clean":"2024-09-21T07:45:36.679176+0000","last_became_active":"2024-09-21T07:35:26.872230+0000","last_became_peered":"2024-09-21T07:35:26.872230+0000","last_unstale":"2024-09-21T07:45:36.679176+0000","last_undegraded":"2024-09-21T07:45:36.679176+0000","last_fullsized":"2024-09-21T07:45:36.679176+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:24.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001197449,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":248,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.679175+0000","last_change":"2024-09-21T07:34:24.258754+0000","last_active":"2024-09-21T07:45:36.679175+0000","last_peered":"2024-09-21T07:45:36.679175+0000","last_clean":"2024-09-21T07:45:36.679175+0000","last_became_active":"2024-09-21T07:34:23.272657+0000","last_became_peered":"2024-09-21T07:34:23.272657+0000","last_unstale":"2024-09-21T07:45:36.679175+0000","last_undegraded":"2024-09-21T07:45:36.679175+0000","last_fullsized":"2024-09-21T07:45:36.679175+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:05.477099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067143200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.692417+0000","last_change":"2024-09-21T07:45:36.692417+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.692417+0000","last_undegraded":"2024-09-21T07:45:36.692417+0000","last_fullsized":"2024-09-21T07:45:36.692417+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.689641+0000","last_change":"2024-09-21T07:45:36.687920+0000","last_active":"2024-09-21T07:45:36.689641+0000","last_peered":"2024-09-21T07:45:36.689641+0000","last_clean":"2024-09-21T07:45:36.689641+0000","last_became_active":"2024-09-21T07:45:36.687561+0000","last_became_peered":"2024-09-21T07:45:36.687561+0000","last_unstale":"2024-09-21T07:45:36.689641+0000","last_undegraded":"2024-09-21T07:45:36.689641+0000","last_fullsized":"2024-09-21T07:45:36.689641+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:47:40.418087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":187,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.674812+0000","last_change":"2024-09-21T07:35:27.876021+0000","last_active":"2024-09-21T07:45:36.674812+0000","last_peered":"2024-09-21T07:45:36.674812+0000","last_clean":"2024-09-21T07:45:36.674812+0000","last_became_active":"2024-09-21T07:35:27.875724+0000","last_became_peered":"2024-09-21T07:35:27.875724+0000","last_unstale":"2024-09-21T07:45:36.674812+0000","last_undegraded":"2024-09-21T07:45:36.674812+0000","last_fullsized":"2024-09-21T07:45:36.674812+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:03.022812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.679122+0000","last_change":"2024-09-21T07:35:29.492570+0000","last_active":"2024-09-21T07:45:36.679122+0000","last_peered":"2024-09-21T07:45:36.679122+0000","last_clean":"2024-09-21T07:45:36.679122+0000","last_became_active":"2024-09-21T07:35:26.874316+0000","last_became_peered":"2024-09-21T07:35:26.874316+0000","last_unstale":"2024-09-21T07:45:36.679122+0000","last_undegraded":"2024-09-21T07:45:36.679122+0000","last_fullsized":"2024-09-21T07:45:36.679122+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:16.512323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011990760000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":246,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.674075+0000","last_change":"2024-09-21T07:34:25.085232+0000","last_active":"2024-09-21T07:45:36.674075+0000","last_peered":"2024-09-21T07:45:36.674075+0000","last_clean":"2024-09-21T07:45:36.674075+0000","last_became_active":"2024-09-21T07:34:23.277807+0000","last_became_peered":"2024-09-21T07:34:23.277807+0000","last_unstale":"2024-09-21T07:45:36.674075+0000","last_undegraded":"2024-09-21T07:45:36.674075+0000","last_fullsized":"2024-09-21T07:45:36.674075+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:36.362919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086619099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689375+0000","last_change":"2024-09-21T07:45:37.685148+0000","last_active":"2024-09-21T07:45:38.689375+0000","last_peered":"2024-09-21T07:45:38.689375+0000","last_clean":"2024-09-21T07:45:38.689375+0000","last_became_active":"2024-09-21T07:45:37.684774+0000","last_became_peered":"2024-09-21T07:45:37.684774+0000","last_unstale":"2024-09-21T07:45:38.689375+0000","last_undegraded":"2024-09-21T07:45:38.689375+0000","last_fullsized":"2024-09-21T07:45:38.689375+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:25.616419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"156'1","reported_seq":16,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.687687+0000","last_change":"2024-09-21T07:45:36.686891+0000","last_active":"2024-09-21T07:45:38.687687+0000","last_peered":"2024-09-21T07:45:38.687687+0000","last_clean":"2024-09-21T07:45:38.687687+0000","last_became_active":"2024-09-21T07:45:36.684124+0000","last_became_peered":"2024-09-21T07:45:36.684124+0000","last_unstale":"2024-09-21T07:45:38.687687+0000","last_undegraded":"2024-09-21T07:45:38.687687+0000","last_fullsized":"2024-09-21T07:45:38.687687+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:00.150983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":186,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.679954+0000","last_change":"2024-09-21T07:35:27.872798+0000","last_active":"2024-09-21T07:45:36.679954+0000","last_peered":"2024-09-21T07:45:36.679954+0000","last_clean":"2024-09-21T07:45:36.679954+0000","last_became_active":"2024-09-21T07:35:27.872483+0000","last_became_peered":"2024-09-21T07:35:27.872483+0000","last_unstale":"2024-09-21T07:45:36.679954+0000","last_undegraded":"2024-09-21T07:45:36.679954+0000","last_fullsized":"2024-09-21T07:45:36.679954+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:11.933469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":193,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.679781+0000","last_change":"2024-09-21T07:35:29.206723+0000","last_active":"2024-09-21T07:45:36.679781+0000","last_peered":"2024-09-21T07:45:36.679781+0000","last_clean":"2024-09-21T07:45:36.679781+0000","last_became_active":"2024-09-21T07:35:26.872377+0000","last_became_peered":"2024-09-21T07:35:26.872377+0000","last_unstale":"2024-09-21T07:45:36.679781+0000","last_undegraded":"2024-09-21T07:45:36.679781+0000","last_fullsized":"2024-09-21T07:45:36.679781+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:29.086862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083077200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":246,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689123+0000","last_change":"2024-09-21T07:34:24.597672+0000","last_active":"2024-09-21T07:45:38.689123+0000","last_peered":"2024-09-21T07:45:38.689123+0000","last_clean":"2024-09-21T07:45:38.689123+0000","last_became_active":"2024-09-21T07:34:23.267770+0000","last_became_peered":"2024-09-21T07:34:23.267770+0000","last_unstale":"2024-09-21T07:45:38.689123+0000","last_undegraded":"2024-09-21T07:45:38.689123+0000","last_fullsized":"2024-09-21T07:45:38.689123+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:51.629476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086378500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689895+0000","last_change":"2024-09-21T07:45:37.691898+0000","last_active":"2024-09-21T07:45:38.689895+0000","last_peered":"2024-09-21T07:45:38.689895+0000","last_clean":"2024-09-21T07:45:38.689895+0000","last_became_active":"2024-09-21T07:45:37.691507+0000","last_became_peered":"2024-09-21T07:45:37.691507+0000","last_unstale":"2024-09-21T07:45:38.689895+0000","last_undegraded":"2024-09-21T07:45:38.689895+0000","last_fullsized":"2024-09-21T07:45:38.689895+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:14:43.900850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"156'1","reported_seq":16,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689228+0000","last_change":"2024-09-21T07:45:36.694668+0000","last_active":"2024-09-21T07:45:38.689228+0000","last_peered":"2024-09-21T07:45:38.689228+0000","last_clean":"2024-09-21T07:45:38.689228+0000","last_became_active":"2024-09-21T07:45:36.694405+0000","last_became_peered":"2024-09-21T07:45:36.694405+0000","last_unstale":"2024-09-21T07:45:38.689228+0000","last_undegraded":"2024-09-21T07:45:38.689228+0000","last_fullsized":"2024-09-21T07:45:38.689228+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:08.552085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":186,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.680841+0000","last_change":"2024-09-21T07:35:27.870121+0000","last_active":"2024-09-21T07:45:36.680841+0000","last_peered":"2024-09-21T07:45:36.680841+0000","last_clean":"2024-09-21T07:45:36.680841+0000","last_became_active":"2024-09-21T07:35:27.869811+0000","last_became_peered":"2024-09-21T07:35:27.869811+0000","last_unstale":"2024-09-21T07:45:36.680841+0000","last_undegraded":"2024-09-21T07:45:36.680841+0000","last_fullsized":"2024-09-21T07:45:36.680841+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:31.778461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.696395+0000","last_change":"2024-09-21T07:35:28.860699+0000","last_active":"2024-09-21T07:45:36.696395+0000","last_peered":"2024-09-21T07:45:36.696395+0000","last_clean":"2024-09-21T07:45:36.696395+0000","last_became_active":"2024-09-21T07:35:26.876779+0000","last_became_peered":"2024-09-21T07:35:26.876779+0000","last_unstale":"2024-09-21T07:45:36.696395+0000","last_undegraded":"2024-09-21T07:45:36.696395+0000","last_fullsized":"2024-09-21T07:45:36.696395+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:28.286403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000913188,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":247,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.700060+0000","last_change":"2024-09-21T07:34:24.260181+0000","last_active":"2024-09-21T07:45:36.700060+0000","last_peered":"2024-09-21T07:45:36.700060+0000","last_clean":"2024-09-21T07:45:36.700060+0000","last_became_active":"2024-09-21T07:34:23.275580+0000","last_became_peered":"2024-09-21T07:34:23.275580+0000","last_unstale":"2024-09-21T07:45:36.700060+0000","last_undegraded":"2024-09-21T07:45:36.700060+0000","last_fullsized":"2024-09-21T07:45:36.700060+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:49.857622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014706980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689850+0000","last_change":"2024-09-21T07:45:37.698246+0000","last_active":"2024-09-21T07:45:38.689850+0000","last_peered":"2024-09-21T07:45:38.689850+0000","last_clean":"2024-09-21T07:45:38.689850+0000","last_became_active":"2024-09-21T07:45:37.697792+0000","last_became_peered":"2024-09-21T07:45:37.697792+0000","last_unstale":"2024-09-21T07:45:38.689850+0000","last_undegraded":"2024-09-21T07:45:38.689850+0000","last_fullsized":"2024-09-21T07:45:38.689850+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:06.139752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.694910+0000","last_change":"2024-09-21T07:45:36.693696+0000","last_active":"2024-09-21T07:45:36.694910+0000","last_peered":"2024-09-21T07:45:36.694910+0000","last_clean":"2024-09-21T07:45:36.694910+0000","last_became_active":"2024-09-21T07:45:36.693211+0000","last_became_peered":"2024-09-21T07:45:36.693211+0000","last_unstale":"2024-09-21T07:45:36.694910+0000","last_undegraded":"2024-09-21T07:45:36.694910+0000","last_fullsized":"2024-09-21T07:45:36.694910+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:15.970506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":188,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.672760+0000","last_change":"2024-09-21T07:35:27.875393+0000","last_active":"2024-09-21T07:45:36.672760+0000","last_peered":"2024-09-21T07:45:36.672760+0000","last_clean":"2024-09-21T07:45:36.672760+0000","last_became_active":"2024-09-21T07:35:27.874671+0000","last_became_peered":"2024-09-21T07:35:27.874671+0000","last_unstale":"2024-09-21T07:45:36.672760+0000","last_undegraded":"2024-09-21T07:45:36.672760+0000","last_fullsized":"2024-09-21T07:45:36.672760+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":193,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.678090+0000","last_change":"2024-09-21T07:35:29.206296+0000","last_active":"2024-09-21T07:45:36.678090+0000","last_peered":"2024-09-21T07:45:36.678090+0000","last_clean":"2024-09-21T07:45:36.678090+0000","last_became_active":"2024-09-21T07:35:26.874575+0000","last_became_peered":"2024-09-21T07:35:26.874575+0000","last_unstale":"2024-09-21T07:45:36.678090+0000","last_undegraded":"2024-09-21T07:45:36.678090+0000","last_fullsized":"2024-09-21T07:45:36.678090+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:22.771154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061011599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":252,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690260+0000","last_change":"2024-09-21T07:34:24.259112+0000","last_active":"2024-09-21T07:45:38.690260+0000","last_peered":"2024-09-21T07:45:38.690260+0000","last_clean":"2024-09-21T07:45:38.690260+0000","last_became_active":"2024-09-21T07:34:23.635444+0000","last_became_peered":"2024-09-21T07:34:23.635444+0000","last_unstale":"2024-09-21T07:45:38.690260+0000","last_undegraded":"2024-09-21T07:45:38.690260+0000","last_fullsized":"2024-09-21T07:45:38.690260+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:46.255385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001952636,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.672205+0000","last_change":"2024-09-21T07:45:36.672205+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.672205+0000","last_undegraded":"2024-09-21T07:45:36.672205+0000","last_fullsized":"2024-09-21T07:45:36.672205+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.696954+0000","last_change":"2024-09-21T07:45:36.696284+0000","last_active":"2024-09-21T07:45:36.696954+0000","last_peered":"2024-09-21T07:45:36.696954+0000","last_clean":"2024-09-21T07:45:36.696954+0000","last_became_active":"2024-09-21T07:45:36.695952+0000","last_became_peered":"2024-09-21T07:45:36.695952+0000","last_unstale":"2024-09-21T07:45:36.696954+0000","last_undegraded":"2024-09-21T07:45:36.696954+0000","last_fullsized":"2024-09-21T07:45:36.696954+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:01.857023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":188,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.673558+0000","last_change":"2024-09-21T07:35:27.878356+0000","last_active":"2024-09-21T07:45:36.673558+0000","last_peered":"2024-09-21T07:45:36.673558+0000","last_clean":"2024-09-21T07:45:36.673558+0000","last_became_active":"2024-09-21T07:35:27.877982+0000","last_became_peered":"2024-09-21T07:35:27.877982+0000","last_unstale":"2024-09-21T07:45:36.673558+0000","last_undegraded":"2024-09-21T07:45:36.673558+0000","last_fullsized":"2024-09-21T07:45:36.673558+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":196,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.671879+0000","last_change":"2024-09-21T07:35:29.053131+0000","last_active":"2024-09-21T07:45:36.671879+0000","last_peered":"2024-09-21T07:45:36.671879+0000","last_clean":"2024-09-21T07:45:36.671879+0000","last_became_active":"2024-09-21T07:35:26.870516+0000","last_became_peered":"2024-09-21T07:35:26.870516+0000","last_unstale":"2024-09-21T07:45:36.671879+0000","last_undegraded":"2024-09-21T07:45:36.671879+0000","last_fullsized":"2024-09-21T07:45:36.671879+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:10.924491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070705699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":252,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.669780+0000","last_change":"2024-09-21T07:34:24.539152+0000","last_active":"2024-09-21T07:45:36.669780+0000","last_peered":"2024-09-21T07:45:36.669780+0000","last_clean":"2024-09-21T07:45:36.669780+0000","last_became_active":"2024-09-21T07:34:23.259573+0000","last_became_peered":"2024-09-21T07:34:23.259573+0000","last_unstale":"2024-09-21T07:45:36.669780+0000","last_undegraded":"2024-09-21T07:45:36.669780+0000","last_fullsized":"2024-09-21T07:45:36.669780+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:26.986900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058108400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689007+0000","last_change":"2024-09-21T07:45:37.689304+0000","last_active":"2024-09-21T07:45:38.689007+0000","last_peered":"2024-09-21T07:45:38.689007+0000","last_clean":"2024-09-21T07:45:38.689007+0000","last_became_active":"2024-09-21T07:45:37.688985+0000","last_became_peered":"2024-09-21T07:45:37.688985+0000","last_unstale":"2024-09-21T07:45:38.689007+0000","last_undegraded":"2024-09-21T07:45:38.689007+0000","last_fullsized":"2024-09-21T07:45:38.689007+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:03.596216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.700099+0000","last_change":"2024-09-21T07:45:36.697335+0000","last_active":"2024-09-21T07:45:36.700099+0000","last_peered":"2024-09-21T07:45:36.700099+0000","last_clean":"2024-09-21T07:45:36.700099+0000","last_became_active":"2024-09-21T07:45:36.697036+0000","last_became_peered":"2024-09-21T07:45:36.697036+0000","last_unstale":"2024-09-21T07:45:36.700099+0000","last_undegraded":"2024-09-21T07:45:36.700099+0000","last_fullsized":"2024-09-21T07:45:36.700099+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:16.366485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":188,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.671824+0000","last_change":"2024-09-21T07:35:27.873726+0000","last_active":"2024-09-21T07:45:36.671824+0000","last_peered":"2024-09-21T07:45:36.671824+0000","last_clean":"2024-09-21T07:45:36.671824+0000","last_became_active":"2024-09-21T07:35:27.871421+0000","last_became_peered":"2024-09-21T07:35:27.871421+0000","last_unstale":"2024-09-21T07:45:36.671824+0000","last_undegraded":"2024-09-21T07:45:36.671824+0000","last_fullsized":"2024-09-21T07:45:36.671824+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:00.168915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.699581+0000","last_change":"2024-09-21T07:35:28.861032+0000","last_active":"2024-09-21T07:45:36.699581+0000","last_peered":"2024-09-21T07:45:36.699581+0000","last_clean":"2024-09-21T07:45:36.699581+0000","last_became_active":"2024-09-21T07:35:26.873149+0000","last_became_peered":"2024-09-21T07:35:26.873149+0000","last_unstale":"2024-09-21T07:45:36.699581+0000","last_undegraded":"2024-09-21T07:45:36.699581+0000","last_fullsized":"2024-09-21T07:45:36.699581+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:59.435981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011988369999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":247,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.679365+0000","last_change":"2024-09-21T07:34:24.259902+0000","last_active":"2024-09-21T07:45:36.679365+0000","last_peered":"2024-09-21T07:45:36.679365+0000","last_clean":"2024-09-21T07:45:36.679365+0000","last_became_active":"2024-09-21T07:34:23.271902+0000","last_became_peered":"2024-09-21T07:34:23.271902+0000","last_unstale":"2024-09-21T07:45:36.679365+0000","last_undegraded":"2024-09-21T07:45:36.679365+0000","last_fullsized":"2024-09-21T07:45:36.679365+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:59.439525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001686358,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.671064+0000","last_change":"2024-09-21T07:45:36.671064+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.671064+0000","last_undegraded":"2024-09-21T07:45:36.671064+0000","last_fullsized":"2024-09-21T07:45:36.671064+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"157'1","reported_seq":16,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689021+0000","last_change":"2024-09-21T07:45:36.673147+0000","last_active":"2024-09-21T07:45:38.689021+0000","last_peered":"2024-09-21T07:45:38.689021+0000","last_clean":"2024-09-21T07:45:38.689021+0000","last_became_active":"2024-09-21T07:45:36.672816+0000","last_became_peered":"2024-09-21T07:45:36.672816+0000","last_unstale":"2024-09-21T07:45:38.689021+0000","last_undegraded":"2024-09-21T07:45:38.689021+0000","last_fullsized":"2024-09-21T07:45:38.689021+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:04:18.857870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":187,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.673815+0000","last_change":"2024-09-21T07:35:27.872637+0000","last_active":"2024-09-21T07:45:36.673815+0000","last_peered":"2024-09-21T07:45:36.673815+0000","last_clean":"2024-09-21T07:45:36.673815+0000","last_became_active":"2024-09-21T07:35:27.872149+0000","last_became_peered":"2024-09-21T07:35:27.872149+0000","last_unstale":"2024-09-21T07:45:36.673815+0000","last_undegraded":"2024-09-21T07:45:36.673815+0000","last_fullsized":"2024-09-21T07:45:36.673815+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:27.697872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":195,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.678962+0000","last_change":"2024-09-21T07:35:29.493362+0000","last_active":"2024-09-21T07:45:36.678962+0000","last_peered":"2024-09-21T07:45:36.678962+0000","last_clean":"2024-09-21T07:45:36.678962+0000","last_became_active":"2024-09-21T07:35:26.873018+0000","last_became_peered":"2024-09-21T07:35:26.873018+0000","last_unstale":"2024-09-21T07:45:36.678962+0000","last_undegraded":"2024-09-21T07:45:36.678962+0000","last_fullsized":"2024-09-21T07:45:36.678962+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:02.460557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012496510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":251,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689489+0000","last_change":"2024-09-21T07:34:24.259662+0000","last_active":"2024-09-21T07:45:38.689489+0000","last_peered":"2024-09-21T07:45:38.689489+0000","last_clean":"2024-09-21T07:45:38.689489+0000","last_became_active":"2024-09-21T07:34:23.278812+0000","last_became_peered":"2024-09-21T07:34:23.278812+0000","last_unstale":"2024-09-21T07:45:38.689489+0000","last_undegraded":"2024-09-21T07:45:38.689489+0000","last_fullsized":"2024-09-21T07:45:38.689489+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:49.829974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356463,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.675060+0000","last_change":"2024-09-21T07:45:36.675060+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.675060+0000","last_undegraded":"2024-09-21T07:45:36.675060+0000","last_fullsized":"2024-09-21T07:45:36.675060+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689740+0000","last_change":"2024-09-21T07:45:36.693420+0000","last_active":"2024-09-21T07:45:38.689740+0000","last_peered":"2024-09-21T07:45:38.689740+0000","last_clean":"2024-09-21T07:45:38.689740+0000","last_became_active":"2024-09-21T07:45:36.692969+0000","last_became_peered":"2024-09-21T07:45:36.692969+0000","last_unstale":"2024-09-21T07:45:38.689740+0000","last_undegraded":"2024-09-21T07:45:38.689740+0000","last_fullsized":"2024-09-21T07:45:38.689740+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:09:22.417758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":189,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.687613+0000","last_change":"2024-09-21T07:35:27.868448+0000","last_active":"2024-09-21T07:45:38.687613+0000","last_peered":"2024-09-21T07:45:38.687613+0000","last_clean":"2024-09-21T07:45:38.687613+0000","last_became_active":"2024-09-21T07:35:27.868105+0000","last_became_peered":"2024-09-21T07:35:27.868105+0000","last_unstale":"2024-09-21T07:45:38.687613+0000","last_undegraded":"2024-09-21T07:45:38.687613+0000","last_fullsized":"2024-09-21T07:45:38.687613+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:41.468702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.678932+0000","last_change":"2024-09-21T07:35:29.492974+0000","last_active":"2024-09-21T07:45:36.678932+0000","last_peered":"2024-09-21T07:45:36.678932+0000","last_clean":"2024-09-21T07:45:36.678932+0000","last_became_active":"2024-09-21T07:35:26.872165+0000","last_became_peered":"2024-09-21T07:35:26.872165+0000","last_unstale":"2024-09-21T07:45:36.678932+0000","last_undegraded":"2024-09-21T07:45:36.678932+0000","last_fullsized":"2024-09-21T07:45:36.678932+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:59.912520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013795179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":245,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.672175+0000","last_change":"2024-09-21T07:34:25.084861+0000","last_active":"2024-09-21T07:45:36.672175+0000","last_peered":"2024-09-21T07:45:36.672175+0000","last_clean":"2024-09-21T07:45:36.672175+0000","last_became_active":"2024-09-21T07:34:23.277708+0000","last_became_peered":"2024-09-21T07:34:23.277708+0000","last_unstale":"2024-09-21T07:45:36.672175+0000","last_undegraded":"2024-09-21T07:45:36.672175+0000","last_fullsized":"2024-09-21T07:45:36.672175+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:40.816488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000597566,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.677283+0000","last_change":"2024-09-21T07:45:36.677283+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.677283+0000","last_undegraded":"2024-09-21T07:45:36.677283+0000","last_fullsized":"2024-09-21T07:45:36.677283+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.688738+0000","last_change":"2024-09-21T07:45:36.687709+0000","last_active":"2024-09-21T07:45:36.688738+0000","last_peered":"2024-09-21T07:45:36.688738+0000","last_clean":"2024-09-21T07:45:36.688738+0000","last_became_active":"2024-09-21T07:45:36.684703+0000","last_became_peered":"2024-09-21T07:45:36.684703+0000","last_unstale":"2024-09-21T07:45:36.688738+0000","last_undegraded":"2024-09-21T07:45:36.688738+0000","last_fullsized":"2024-09-21T07:45:36.688738+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:59.601883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":189,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688381+0000","last_change":"2024-09-21T07:35:27.874640+0000","last_active":"2024-09-21T07:45:38.688381+0000","last_peered":"2024-09-21T07:45:38.688381+0000","last_clean":"2024-09-21T07:45:38.688381+0000","last_became_active":"2024-09-21T07:35:27.874265+0000","last_became_peered":"2024-09-21T07:35:27.874265+0000","last_unstale":"2024-09-21T07:45:38.688381+0000","last_undegraded":"2024-09-21T07:45:38.688381+0000","last_fullsized":"2024-09-21T07:45:38.688381+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:45.754654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":193,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.680159+0000","last_change":"2024-09-21T07:35:29.207392+0000","last_active":"2024-09-21T07:45:36.680159+0000","last_peered":"2024-09-21T07:45:36.680159+0000","last_clean":"2024-09-21T07:45:36.680159+0000","last_became_active":"2024-09-21T07:35:26.870917+0000","last_became_peered":"2024-09-21T07:35:26.870917+0000","last_unstale":"2024-09-21T07:45:36.680159+0000","last_undegraded":"2024-09-21T07:45:36.680159+0000","last_fullsized":"2024-09-21T07:45:36.680159+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:59.159937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059198499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":251,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689049+0000","last_change":"2024-09-21T07:34:24.258709+0000","last_active":"2024-09-21T07:45:38.689049+0000","last_peered":"2024-09-21T07:45:38.689049+0000","last_clean":"2024-09-21T07:45:38.689049+0000","last_became_active":"2024-09-21T07:34:23.279042+0000","last_became_peered":"2024-09-21T07:34:23.279042+0000","last_unstale":"2024-09-21T07:45:38.689049+0000","last_undegraded":"2024-09-21T07:45:38.689049+0000","last_fullsized":"2024-09-21T07:45:38.689049+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:01.762117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019282100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.676888+0000","last_change":"2024-09-21T07:45:36.676888+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.676888+0000","last_undegraded":"2024-09-21T07:45:36.676888+0000","last_fullsized":"2024-09-21T07:45:36.676888+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.699451+0000","last_change":"2024-09-21T07:45:36.698592+0000","last_active":"2024-09-21T07:45:36.699451+0000","last_peered":"2024-09-21T07:45:36.699451+0000","last_clean":"2024-09-21T07:45:36.699451+0000","last_became_active":"2024-09-21T07:45:36.698011+0000","last_became_peered":"2024-09-21T07:45:36.698011+0000","last_unstale":"2024-09-21T07:45:36.699451+0000","last_undegraded":"2024-09-21T07:45:36.699451+0000","last_fullsized":"2024-09-21T07:45:36.699451+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:00.979027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":187,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.676080+0000","last_change":"2024-09-21T07:35:27.870260+0000","last_active":"2024-09-21T07:45:36.676080+0000","last_peered":"2024-09-21T07:45:36.676080+0000","last_clean":"2024-09-21T07:45:36.676080+0000","last_became_active":"2024-09-21T07:35:27.869448+0000","last_became_peered":"2024-09-21T07:35:27.869448+0000","last_unstale":"2024-09-21T07:45:36.676080+0000","last_undegraded":"2024-09-21T07:45:36.676080+0000","last_fullsized":"2024-09-21T07:45:36.676080+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:16.946790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":196,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688161+0000","last_change":"2024-09-21T07:35:29.386367+0000","last_active":"2024-09-21T07:45:38.688161+0000","last_peered":"2024-09-21T07:45:38.688161+0000","last_clean":"2024-09-21T07:45:38.688161+0000","last_became_active":"2024-09-21T07:35:26.879238+0000","last_became_peered":"2024-09-21T07:35:26.879238+0000","last_unstale":"2024-09-21T07:45:38.688161+0000","last_undegraded":"2024-09-21T07:45:38.688161+0000","last_fullsized":"2024-09-21T07:45:38.688161+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:16.528260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092223100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":245,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.677233+0000","last_change":"2024-09-21T07:34:25.085635+0000","last_active":"2024-09-21T07:45:36.677233+0000","last_peered":"2024-09-21T07:45:36.677233+0000","last_clean":"2024-09-21T07:45:36.677233+0000","last_became_active":"2024-09-21T07:34:23.273566+0000","last_became_peered":"2024-09-21T07:34:23.273566+0000","last_unstale":"2024-09-21T07:45:36.677233+0000","last_undegraded":"2024-09-21T07:45:36.677233+0000","last_fullsized":"2024-09-21T07:45:36.677233+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:14.142272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079897300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.676058+0000","last_change":"2024-09-21T07:45:36.676058+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.676058+0000","last_undegraded":"2024-09-21T07:45:36.676058+0000","last_fullsized":"2024-09-21T07:45:36.676058+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.689075+0000","last_change":"2024-09-21T07:45:36.685900+0000","last_active":"2024-09-21T07:45:36.689075+0000","last_peered":"2024-09-21T07:45:36.689075+0000","last_clean":"2024-09-21T07:45:36.689075+0000","last_became_active":"2024-09-21T07:45:36.685578+0000","last_became_peered":"2024-09-21T07:45:36.685578+0000","last_unstale":"2024-09-21T07:45:36.689075+0000","last_undegraded":"2024-09-21T07:45:36.689075+0000","last_fullsized":"2024-09-21T07:45:36.689075+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:03.369431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":187,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.677035+0000","last_change":"2024-09-21T07:35:27.878728+0000","last_active":"2024-09-21T07:45:36.677035+0000","last_peered":"2024-09-21T07:45:36.677035+0000","last_clean":"2024-09-21T07:45:36.677035+0000","last_became_active":"2024-09-21T07:35:27.878335+0000","last_became_peered":"2024-09-21T07:35:27.878335+0000","last_unstale":"2024-09-21T07:45:36.677035+0000","last_undegraded":"2024-09-21T07:45:36.677035+0000","last_fullsized":"2024-09-21T07:45:36.677035+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:25:37.731771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":199,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.668981+0000","last_change":"2024-09-21T07:35:28.858418+0000","last_active":"2024-09-21T07:45:36.668981+0000","last_peered":"2024-09-21T07:45:36.668981+0000","last_clean":"2024-09-21T07:45:36.668981+0000","last_became_active":"2024-09-21T07:35:26.876944+0000","last_became_peered":"2024-09-21T07:35:26.876944+0000","last_unstale":"2024-09-21T07:45:36.668981+0000","last_undegraded":"2024-09-21T07:45:36.668981+0000","last_fullsized":"2024-09-21T07:45:36.668981+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:04.514304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090862799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":245,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688906+0000","last_change":"2024-09-21T07:34:24.597926+0000","last_active":"2024-09-21T07:45:38.688906+0000","last_peered":"2024-09-21T07:45:38.688906+0000","last_clean":"2024-09-21T07:45:38.688906+0000","last_became_active":"2024-09-21T07:34:23.268926+0000","last_became_peered":"2024-09-21T07:34:23.268926+0000","last_unstale":"2024-09-21T07:45:38.688906+0000","last_undegraded":"2024-09-21T07:45:38.688906+0000","last_fullsized":"2024-09-21T07:45:38.688906+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:46.190752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101851,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.676259+0000","last_change":"2024-09-21T07:45:36.676259+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.676259+0000","last_undegraded":"2024-09-21T07:45:36.676259+0000","last_fullsized":"2024-09-21T07:45:36.676259+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.685429+0000","last_change":"2024-09-21T07:45:36.683898+0000","last_active":"2024-09-21T07:45:36.685429+0000","last_peered":"2024-09-21T07:45:36.685429+0000","last_clean":"2024-09-21T07:45:36.685429+0000","last_became_active":"2024-09-21T07:45:36.683536+0000","last_became_peered":"2024-09-21T07:45:36.683536+0000","last_unstale":"2024-09-21T07:45:36.685429+0000","last_undegraded":"2024-09-21T07:45:36.685429+0000","last_fullsized":"2024-09-21T07:45:36.685429+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:47.039079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688581+0000","last_change":"2024-09-21T07:35:27.877426+0000","last_active":"2024-09-21T07:45:38.688581+0000","last_peered":"2024-09-21T07:45:38.688581+0000","last_clean":"2024-09-21T07:45:38.688581+0000","last_became_active":"2024-09-21T07:35:27.877050+0000","last_became_peered":"2024-09-21T07:35:27.877050+0000","last_unstale":"2024-09-21T07:45:38.688581+0000","last_undegraded":"2024-09-21T07:45:38.688581+0000","last_fullsized":"2024-09-21T07:45:38.688581+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:36.268334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.673914+0000","last_change":"2024-09-21T07:35:28.862273+0000","last_active":"2024-09-21T07:45:36.673914+0000","last_peered":"2024-09-21T07:45:36.673914+0000","last_clean":"2024-09-21T07:45:36.673914+0000","last_became_active":"2024-09-21T07:35:26.874011+0000","last_became_peered":"2024-09-21T07:35:26.874011+0000","last_unstale":"2024-09-21T07:45:36.673914+0000","last_undegraded":"2024-09-21T07:45:36.673914+0000","last_fullsized":"2024-09-21T07:45:36.673914+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:11.986950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071644799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":246,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.697773+0000","last_change":"2024-09-21T07:34:24.259794+0000","last_active":"2024-09-21T07:45:36.697773+0000","last_peered":"2024-09-21T07:45:36.697773+0000","last_clean":"2024-09-21T07:45:36.697773+0000","last_became_active":"2024-09-21T07:34:23.265679+0000","last_became_peered":"2024-09-21T07:34:23.265679+0000","last_unstale":"2024-09-21T07:45:36.697773+0000","last_undegraded":"2024-09-21T07:45:36.697773+0000","last_fullsized":"2024-09-21T07:45:36.697773+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:02.575796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001298518,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.673113+0000","last_change":"2024-09-21T07:45:36.673113+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.673113+0000","last_undegraded":"2024-09-21T07:45:36.673113+0000","last_fullsized":"2024-09-21T07:45:36.673113+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689224+0000","last_change":"2024-09-21T07:45:36.695633+0000","last_active":"2024-09-21T07:45:38.689224+0000","last_peered":"2024-09-21T07:45:38.689224+0000","last_clean":"2024-09-21T07:45:38.689224+0000","last_became_active":"2024-09-21T07:45:36.695319+0000","last_became_peered":"2024-09-21T07:45:36.695319+0000","last_unstale":"2024-09-21T07:45:38.689224+0000","last_undegraded":"2024-09-21T07:45:38.689224+0000","last_fullsized":"2024-09-21T07:45:38.689224+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:36.781831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":186,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.680833+0000","last_change":"2024-09-21T07:35:27.877725+0000","last_active":"2024-09-21T07:45:36.680833+0000","last_peered":"2024-09-21T07:45:36.680833+0000","last_clean":"2024-09-21T07:45:36.680833+0000","last_became_active":"2024-09-21T07:35:27.877398+0000","last_became_peered":"2024-09-21T07:35:27.877398+0000","last_unstale":"2024-09-21T07:45:36.680833+0000","last_undegraded":"2024-09-21T07:45:36.680833+0000","last_fullsized":"2024-09-21T07:45:36.680833+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:05.859762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.693485+0000","last_change":"2024-09-21T07:35:28.860612+0000","last_active":"2024-09-21T07:45:36.693485+0000","last_peered":"2024-09-21T07:45:36.693485+0000","last_clean":"2024-09-21T07:45:36.693485+0000","last_became_active":"2024-09-21T07:35:26.876562+0000","last_became_peered":"2024-09-21T07:35:26.876562+0000","last_unstale":"2024-09-21T07:45:36.693485+0000","last_undegraded":"2024-09-21T07:45:36.693485+0000","last_fullsized":"2024-09-21T07:45:36.693485+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:06.824331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012252470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":247,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.679653+0000","last_change":"2024-09-21T07:34:24.260330+0000","last_active":"2024-09-21T07:45:36.679653+0000","last_peered":"2024-09-21T07:45:36.679653+0000","last_clean":"2024-09-21T07:45:36.679653+0000","last_became_active":"2024-09-21T07:34:23.637210+0000","last_became_peered":"2024-09-21T07:34:23.637210+0000","last_unstale":"2024-09-21T07:45:36.679653+0000","last_undegraded":"2024-09-21T07:45:36.679653+0000","last_fullsized":"2024-09-21T07:45:36.679653+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.488451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001537093,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.673352+0000","last_change":"2024-09-21T07:45:36.673352+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.673352+0000","last_undegraded":"2024-09-21T07:45:36.673352+0000","last_fullsized":"2024-09-21T07:45:36.673352+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.677935+0000","last_change":"2024-09-21T07:45:36.676063+0000","last_active":"2024-09-21T07:45:36.677935+0000","last_peered":"2024-09-21T07:45:36.677935+0000","last_clean":"2024-09-21T07:45:36.677935+0000","last_became_active":"2024-09-21T07:45:36.675750+0000","last_became_peered":"2024-09-21T07:45:36.675750+0000","last_unstale":"2024-09-21T07:45:36.677935+0000","last_undegraded":"2024-09-21T07:45:36.677935+0000","last_fullsized":"2024-09-21T07:45:36.677935+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:20:48.069405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":187,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.673680+0000","last_change":"2024-09-21T07:35:27.874757+0000","last_active":"2024-09-21T07:45:36.673680+0000","last_peered":"2024-09-21T07:45:36.673680+0000","last_clean":"2024-09-21T07:45:36.673680+0000","last_became_active":"2024-09-21T07:35:27.874360+0000","last_became_peered":"2024-09-21T07:35:27.874360+0000","last_unstale":"2024-09-21T07:45:36.673680+0000","last_undegraded":"2024-09-21T07:45:36.673680+0000","last_fullsized":"2024-09-21T07:45:36.673680+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:17.646912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.680294+0000","last_change":"2024-09-21T07:35:29.493431+0000","last_active":"2024-09-21T07:45:36.680294+0000","last_peered":"2024-09-21T07:45:36.680294+0000","last_clean":"2024-09-21T07:45:36.680294+0000","last_became_active":"2024-09-21T07:35:26.873691+0000","last_became_peered":"2024-09-21T07:35:26.873691+0000","last_unstale":"2024-09-21T07:45:36.680294+0000","last_undegraded":"2024-09-21T07:45:36.680294+0000","last_fullsized":"2024-09-21T07:45:36.680294+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:51.705051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001162464,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":245,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.687901+0000","last_change":"2024-09-21T07:34:24.597286+0000","last_active":"2024-09-21T07:45:38.687901+0000","last_peered":"2024-09-21T07:45:38.687901+0000","last_clean":"2024-09-21T07:45:38.687901+0000","last_became_active":"2024-09-21T07:34:23.267133+0000","last_became_peered":"2024-09-21T07:34:23.267133+0000","last_unstale":"2024-09-21T07:45:38.687901+0000","last_undegraded":"2024-09-21T07:45:38.687901+0000","last_fullsized":"2024-09-21T07:45:38.687901+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:18.257010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.670174+0000","last_change":"2024-09-21T07:45:36.670174+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.670174+0000","last_undegraded":"2024-09-21T07:45:36.670174+0000","last_fullsized":"2024-09-21T07:45:36.670174+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.694069+0000","last_change":"2024-09-21T07:45:36.693348+0000","last_active":"2024-09-21T07:45:36.694069+0000","last_peered":"2024-09-21T07:45:36.694069+0000","last_clean":"2024-09-21T07:45:36.694069+0000","last_became_active":"2024-09-21T07:45:36.692821+0000","last_became_peered":"2024-09-21T07:45:36.692821+0000","last_unstale":"2024-09-21T07:45:36.694069+0000","last_undegraded":"2024-09-21T07:45:36.694069+0000","last_fullsized":"2024-09-21T07:45:36.694069+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:15:15.557087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":192,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.668792+0000","last_change":"2024-09-21T07:35:27.864458+0000","last_active":"2024-09-21T07:45:36.668792+0000","last_peered":"2024-09-21T07:45:36.668792+0000","last_clean":"2024-09-21T07:45:36.668792+0000","last_became_active":"2024-09-21T07:35:27.864136+0000","last_became_peered":"2024-09-21T07:35:27.864136+0000","last_unstale":"2024-09-21T07:45:36.668792+0000","last_undegraded":"2024-09-21T07:45:36.668792+0000","last_fullsized":"2024-09-21T07:45:36.668792+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:20.162240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":199,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.669445+0000","last_change":"2024-09-21T07:35:28.858784+0000","last_active":"2024-09-21T07:45:36.669445+0000","last_peered":"2024-09-21T07:45:36.669445+0000","last_clean":"2024-09-21T07:45:36.669445+0000","last_became_active":"2024-09-21T07:35:26.876890+0000","last_became_peered":"2024-09-21T07:35:26.876890+0000","last_unstale":"2024-09-21T07:45:36.669445+0000","last_undegraded":"2024-09-21T07:45:36.669445+0000","last_fullsized":"2024-09-21T07:45:36.669445+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:40.738327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064543499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":251,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689383+0000","last_change":"2024-09-21T07:34:24.258712+0000","last_active":"2024-09-21T07:45:38.689383+0000","last_peered":"2024-09-21T07:45:38.689383+0000","last_clean":"2024-09-21T07:45:38.689383+0000","last_became_active":"2024-09-21T07:34:23.274692+0000","last_became_peered":"2024-09-21T07:34:23.274692+0000","last_unstale":"2024-09-21T07:45:38.689383+0000","last_undegraded":"2024-09-21T07:45:38.689383+0000","last_fullsized":"2024-09-21T07:45:38.689383+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:43.279454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018051479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.673682+0000","last_change":"2024-09-21T07:45:36.673682+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.673682+0000","last_undegraded":"2024-09-21T07:45:36.673682+0000","last_fullsized":"2024-09-21T07:45:36.673682+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.702127+0000","last_change":"2024-09-21T07:45:36.700873+0000","last_active":"2024-09-21T07:45:36.702127+0000","last_peered":"2024-09-21T07:45:36.702127+0000","last_clean":"2024-09-21T07:45:36.702127+0000","last_became_active":"2024-09-21T07:45:36.700526+0000","last_became_peered":"2024-09-21T07:45:36.700526+0000","last_unstale":"2024-09-21T07:45:36.702127+0000","last_undegraded":"2024-09-21T07:45:36.702127+0000","last_fullsized":"2024-09-21T07:45:36.702127+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:08:53.416498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689061+0000","last_change":"2024-09-21T07:35:27.878020+0000","last_active":"2024-09-21T07:45:38.689061+0000","last_peered":"2024-09-21T07:45:38.689061+0000","last_clean":"2024-09-21T07:45:38.689061+0000","last_became_active":"2024-09-21T07:35:27.877532+0000","last_became_peered":"2024-09-21T07:35:27.877532+0000","last_unstale":"2024-09-21T07:45:38.689061+0000","last_undegraded":"2024-09-21T07:45:38.689061+0000","last_fullsized":"2024-09-21T07:45:38.689061+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:19.756073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.698780+0000","last_change":"2024-09-21T07:35:28.861049+0000","last_active":"2024-09-21T07:45:36.698780+0000","last_peered":"2024-09-21T07:45:36.698780+0000","last_clean":"2024-09-21T07:45:36.698780+0000","last_became_active":"2024-09-21T07:35:26.876819+0000","last_became_peered":"2024-09-21T07:35:26.876819+0000","last_unstale":"2024-09-21T07:45:36.698780+0000","last_undegraded":"2024-09-21T07:45:36.698780+0000","last_fullsized":"2024-09-21T07:45:36.698780+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:19.633803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055780599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":245,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.672049+0000","last_change":"2024-09-21T07:34:25.084849+0000","last_active":"2024-09-21T07:45:36.672049+0000","last_peered":"2024-09-21T07:45:36.672049+0000","last_clean":"2024-09-21T07:45:36.672049+0000","last_became_active":"2024-09-21T07:34:23.277713+0000","last_became_peered":"2024-09-21T07:34:23.277713+0000","last_unstale":"2024-09-21T07:45:36.672049+0000","last_undegraded":"2024-09-21T07:45:36.672049+0000","last_fullsized":"2024-09-21T07:45:36.672049+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:05.811810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059079499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.677740+0000","last_change":"2024-09-21T07:45:36.677740+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.677740+0000","last_undegraded":"2024-09-21T07:45:36.677740+0000","last_fullsized":"2024-09-21T07:45:36.677740+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690040+0000","last_change":"2024-09-21T07:45:36.693932+0000","last_active":"2024-09-21T07:45:38.690040+0000","last_peered":"2024-09-21T07:45:38.690040+0000","last_clean":"2024-09-21T07:45:38.690040+0000","last_became_active":"2024-09-21T07:45:36.693640+0000","last_became_peered":"2024-09-21T07:45:36.693640+0000","last_unstale":"2024-09-21T07:45:38.690040+0000","last_undegraded":"2024-09-21T07:45:38.690040+0000","last_fullsized":"2024-09-21T07:45:38.690040+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:04.701463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":188,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.672603+0000","last_change":"2024-09-21T07:35:27.884248+0000","last_active":"2024-09-21T07:45:36.672603+0000","last_peered":"2024-09-21T07:45:36.672603+0000","last_clean":"2024-09-21T07:45:36.672603+0000","last_became_active":"2024-09-21T07:35:27.883861+0000","last_became_peered":"2024-09-21T07:35:27.883861+0000","last_unstale":"2024-09-21T07:45:36.672603+0000","last_undegraded":"2024-09-21T07:45:36.672603+0000","last_fullsized":"2024-09-21T07:45:36.672603+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":193,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.680061+0000","last_change":"2024-09-21T07:35:29.207135+0000","last_active":"2024-09-21T07:45:36.680061+0000","last_peered":"2024-09-21T07:45:36.680061+0000","last_clean":"2024-09-21T07:45:36.680061+0000","last_became_active":"2024-09-21T07:35:26.872787+0000","last_became_peered":"2024-09-21T07:35:26.872787+0000","last_unstale":"2024-09-21T07:45:36.680061+0000","last_undegraded":"2024-09-21T07:45:36.680061+0000","last_fullsized":"2024-09-21T07:45:36.680061+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:27.543217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011157509999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":251,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.668930+0000","last_change":"2024-09-21T07:34:24.538610+0000","last_active":"2024-09-21T07:45:36.668930+0000","last_peered":"2024-09-21T07:45:36.668930+0000","last_clean":"2024-09-21T07:45:36.668930+0000","last_became_active":"2024-09-21T07:34:23.259700+0000","last_became_peered":"2024-09-21T07:34:23.259700+0000","last_unstale":"2024-09-21T07:45:36.668930+0000","last_undegraded":"2024-09-21T07:45:36.668930+0000","last_fullsized":"2024-09-21T07:45:36.668930+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:57.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070522399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690121+0000","last_change":"2024-09-21T07:45:37.714568+0000","last_active":"2024-09-21T07:45:38.690121+0000","last_peered":"2024-09-21T07:45:38.690121+0000","last_clean":"2024-09-21T07:45:38.690121+0000","last_became_active":"2024-09-21T07:45:37.714235+0000","last_became_peered":"2024-09-21T07:45:37.714235+0000","last_unstale":"2024-09-21T07:45:38.690121+0000","last_undegraded":"2024-09-21T07:45:38.690121+0000","last_fullsized":"2024-09-21T07:45:38.690121+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:29.318374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"156'2","reported_seq":17,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.687809+0000","last_change":"2024-09-21T07:45:36.685978+0000","last_active":"2024-09-21T07:45:38.687809+0000","last_peered":"2024-09-21T07:45:38.687809+0000","last_clean":"2024-09-21T07:45:38.687809+0000","last_became_active":"2024-09-21T07:45:36.685307+0000","last_became_peered":"2024-09-21T07:45:36.685307+0000","last_unstale":"2024-09-21T07:45:38.687809+0000","last_undegraded":"2024-09-21T07:45:38.687809+0000","last_fullsized":"2024-09-21T07:45:38.687809+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:18.642482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":192,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.669313+0000","last_change":"2024-09-21T07:35:27.868024+0000","last_active":"2024-09-21T07:45:36.669313+0000","last_peered":"2024-09-21T07:45:36.669313+0000","last_clean":"2024-09-21T07:45:36.669313+0000","last_became_active":"2024-09-21T07:35:27.867339+0000","last_became_peered":"2024-09-21T07:35:27.867339+0000","last_unstale":"2024-09-21T07:45:36.669313+0000","last_undegraded":"2024-09-21T07:45:36.669313+0000","last_fullsized":"2024-09-21T07:45:36.669313+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":199,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.669113+0000","last_change":"2024-09-21T07:35:28.858505+0000","last_active":"2024-09-21T07:45:36.669113+0000","last_peered":"2024-09-21T07:45:36.669113+0000","last_clean":"2024-09-21T07:45:36.669113+0000","last_became_active":"2024-09-21T07:35:26.877320+0000","last_became_peered":"2024-09-21T07:35:26.877320+0000","last_unstale":"2024-09-21T07:45:36.669113+0000","last_undegraded":"2024-09-21T07:45:36.669113+0000","last_fullsized":"2024-09-21T07:45:36.669113+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:53.672782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244119999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":246,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689180+0000","last_change":"2024-09-21T07:34:24.597597+0000","last_active":"2024-09-21T07:45:38.689180+0000","last_peered":"2024-09-21T07:45:38.689180+0000","last_clean":"2024-09-21T07:45:38.689180+0000","last_became_active":"2024-09-21T07:34:23.635204+0000","last_became_peered":"2024-09-21T07:34:23.635204+0000","last_unstale":"2024-09-21T07:45:38.689180+0000","last_undegraded":"2024-09-21T07:45:38.689180+0000","last_fullsized":"2024-09-21T07:45:38.689180+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:48.644789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081139199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.689181+0000","last_change":"2024-09-21T07:45:36.689181+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.689181+0000","last_undegraded":"2024-09-21T07:45:36.689181+0000","last_fullsized":"2024-09-21T07:45:36.689181+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.696649+0000","last_change":"2024-09-21T07:45:36.693812+0000","last_active":"2024-09-21T07:45:36.696649+0000","last_peered":"2024-09-21T07:45:36.696649+0000","last_clean":"2024-09-21T07:45:36.696649+0000","last_became_active":"2024-09-21T07:45:36.692616+0000","last_became_peered":"2024-09-21T07:45:36.692616+0000","last_unstale":"2024-09-21T07:45:36.696649+0000","last_undegraded":"2024-09-21T07:45:36.696649+0000","last_fullsized":"2024-09-21T07:45:36.696649+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:12:40.958669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":186,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.680482+0000","last_change":"2024-09-21T07:35:27.877503+0000","last_active":"2024-09-21T07:45:36.680482+0000","last_peered":"2024-09-21T07:45:36.680482+0000","last_clean":"2024-09-21T07:45:36.680482+0000","last_became_active":"2024-09-21T07:35:27.875775+0000","last_became_peered":"2024-09-21T07:35:27.875775+0000","last_unstale":"2024-09-21T07:45:36.680482+0000","last_undegraded":"2024-09-21T07:45:36.680482+0000","last_fullsized":"2024-09-21T07:45:36.680482+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:49.291023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.678647+0000","last_change":"2024-09-21T07:35:29.491412+0000","last_active":"2024-09-21T07:45:36.678647+0000","last_peered":"2024-09-21T07:45:36.678647+0000","last_clean":"2024-09-21T07:45:36.678647+0000","last_became_active":"2024-09-21T07:35:26.876404+0000","last_became_peered":"2024-09-21T07:35:26.876404+0000","last_unstale":"2024-09-21T07:45:36.678647+0000","last_undegraded":"2024-09-21T07:45:36.678647+0000","last_fullsized":"2024-09-21T07:45:36.678647+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063918000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":252,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689996+0000","last_change":"2024-09-21T07:34:24.259026+0000","last_active":"2024-09-21T07:45:38.689996+0000","last_peered":"2024-09-21T07:45:38.689996+0000","last_clean":"2024-09-21T07:45:38.689996+0000","last_became_active":"2024-09-21T07:34:23.635456+0000","last_became_peered":"2024-09-21T07:34:23.635456+0000","last_unstale":"2024-09-21T07:45:38.689996+0000","last_undegraded":"2024-09-21T07:45:38.689996+0000","last_fullsized":"2024-09-21T07:45:38.689996+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:09.592752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018654419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690345+0000","last_change":"2024-09-21T07:45:37.700588+0000","last_active":"2024-09-21T07:45:38.690345+0000","last_peered":"2024-09-21T07:45:38.690345+0000","last_clean":"2024-09-21T07:45:38.690345+0000","last_became_active":"2024-09-21T07:45:37.700288+0000","last_became_peered":"2024-09-21T07:45:37.700288+0000","last_unstale":"2024-09-21T07:45:38.690345+0000","last_undegraded":"2024-09-21T07:45:38.690345+0000","last_fullsized":"2024-09-21T07:45:38.690345+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:36:25.586374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.696399+0000","last_change":"2024-09-21T07:45:36.695336+0000","last_active":"2024-09-21T07:45:36.696399+0000","last_peered":"2024-09-21T07:45:36.696399+0000","last_clean":"2024-09-21T07:45:36.696399+0000","last_became_active":"2024-09-21T07:45:36.694952+0000","last_became_peered":"2024-09-21T07:45:36.694952+0000","last_unstale":"2024-09-21T07:45:36.696399+0000","last_undegraded":"2024-09-21T07:45:36.696399+0000","last_fullsized":"2024-09-21T07:45:36.696399+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:52.278657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":186,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.680709+0000","last_change":"2024-09-21T07:35:27.871984+0000","last_active":"2024-09-21T07:45:36.680709+0000","last_peered":"2024-09-21T07:45:36.680709+0000","last_clean":"2024-09-21T07:45:36.680709+0000","last_became_active":"2024-09-21T07:35:27.870336+0000","last_became_peered":"2024-09-21T07:35:27.870336+0000","last_unstale":"2024-09-21T07:45:36.680709+0000","last_undegraded":"2024-09-21T07:45:36.680709+0000","last_fullsized":"2024-09-21T07:45:36.680709+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:57.120479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.693795+0000","last_change":"2024-09-21T07:35:28.860351+0000","last_active":"2024-09-21T07:45:36.693795+0000","last_peered":"2024-09-21T07:45:36.693795+0000","last_clean":"2024-09-21T07:45:36.693795+0000","last_became_active":"2024-09-21T07:35:26.872046+0000","last_became_peered":"2024-09-21T07:35:26.872046+0000","last_unstale":"2024-09-21T07:45:36.693795+0000","last_undegraded":"2024-09-21T07:45:36.693795+0000","last_fullsized":"2024-09-21T07:45:36.693795+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:47.074327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098384399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":247,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.679336+0000","last_change":"2024-09-21T07:34:24.260300+0000","last_active":"2024-09-21T07:45:36.679336+0000","last_peered":"2024-09-21T07:45:36.679336+0000","last_clean":"2024-09-21T07:45:36.679336+0000","last_became_active":"2024-09-21T07:34:23.270128+0000","last_became_peered":"2024-09-21T07:34:23.270128+0000","last_unstale":"2024-09-21T07:45:36.679336+0000","last_undegraded":"2024-09-21T07:45:36.679336+0000","last_fullsized":"2024-09-21T07:45:36.679336+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:25.260931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00176651,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.664838+0000","last_change":"2024-09-21T07:45:36.664838+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.664838+0000","last_undegraded":"2024-09-21T07:45:36.664838+0000","last_fullsized":"2024-09-21T07:45:36.664838+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.685247+0000","last_change":"2024-09-21T07:45:36.683495+0000","last_active":"2024-09-21T07:45:36.685247+0000","last_peered":"2024-09-21T07:45:36.685247+0000","last_clean":"2024-09-21T07:45:36.685247+0000","last_became_active":"2024-09-21T07:45:36.683165+0000","last_became_peered":"2024-09-21T07:45:36.683165+0000","last_unstale":"2024-09-21T07:45:36.685247+0000","last_undegraded":"2024-09-21T07:45:36.685247+0000","last_fullsized":"2024-09-21T07:45:36.685247+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:53:20.269457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":186,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.680954+0000","last_change":"2024-09-21T07:35:27.877861+0000","last_active":"2024-09-21T07:45:36.680954+0000","last_peered":"2024-09-21T07:45:36.680954+0000","last_clean":"2024-09-21T07:45:36.680954+0000","last_became_active":"2024-09-21T07:35:27.877434+0000","last_became_peered":"2024-09-21T07:35:27.877434+0000","last_unstale":"2024-09-21T07:45:36.680954+0000","last_undegraded":"2024-09-21T07:45:36.680954+0000","last_fullsized":"2024-09-21T07:45:36.680954+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.672375+0000","last_change":"2024-09-21T07:35:28.862954+0000","last_active":"2024-09-21T07:45:36.672375+0000","last_peered":"2024-09-21T07:45:36.672375+0000","last_clean":"2024-09-21T07:45:36.672375+0000","last_became_active":"2024-09-21T07:35:26.870613+0000","last_became_peered":"2024-09-21T07:35:26.870613+0000","last_unstale":"2024-09-21T07:45:36.672375+0000","last_undegraded":"2024-09-21T07:45:36.672375+0000","last_fullsized":"2024-09-21T07:45:36.672375+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:06.901043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059842100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":246,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.673328+0000","last_change":"2024-09-21T07:34:25.028904+0000","last_active":"2024-09-21T07:45:36.673328+0000","last_peered":"2024-09-21T07:45:36.673328+0000","last_clean":"2024-09-21T07:45:36.673328+0000","last_became_active":"2024-09-21T07:34:23.260942+0000","last_became_peered":"2024-09-21T07:34:23.260942+0000","last_unstale":"2024-09-21T07:45:36.673328+0000","last_undegraded":"2024-09-21T07:45:36.673328+0000","last_fullsized":"2024-09-21T07:45:36.673328+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:22.709328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060553899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509594,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37824,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334016,"statfs":{"total":96636764160,"available":96598032384,"internally_reserved":0,"allocated":2531328,"data_stored":1604054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509596,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37244,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334596,"statfs":{"total":96636764160,"available":96598626304,"internally_reserved":0,"allocated":1937408,"data_stored":1013686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509594,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37240,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334600,"statfs":{"total":96636764160,"available":96598630400,"internally_reserved":0,"allocated":1937408,"data_stored":1013686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509595,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37244,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334596,"statfs":{"total":96636764160,"available":96598626304,"internally_reserved":0,"allocated":1937408,"data_stored":1013686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509596,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37240,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334600,"statfs":{"total":96636764160,"available":96598630400,"internally_reserved":0,"allocated":1937408,"data_stored":1013686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509596,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37820,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334020,"statfs":{"total":96636764160,"available":96598036480,"internally_reserved":0,"allocated":2531328,"data_stored":1604054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542300,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37272,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334568,"statfs":{"total":96636764160,"available":96598597632,"internally_reserved":0,"allocated":1961984,"data_stored":1029031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542299,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41448,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330392,"statfs":{"total":96636764160,"available":96594321408,"internally_reserved":0,"allocated":1978368,"data_stored":1030713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:45:42.173 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:45:42.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.430+0000 7f65d1800640 1 -- 172.21.15.39:0/231820970 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65cc15cc00 msgr2=0x7f65cc15d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:42.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.430+0000 7f65d1800640 1 --2- 172.21.15.39:0/231820970 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65cc15cc00 0x7f65cc15d010 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f65cc073490 tx=0x7f65b802a560 comp rx=0 tx=0).stop 2024-09-21T07:45:42.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.431+0000 7f65d1800640 1 -- 172.21.15.39:0/231820970 shutdown_connections 2024-09-21T07:45:42.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.431+0000 7f65d1800640 1 --2- 172.21.15.39:0/231820970 >> v2:172.21.15.148:3300/0 conn(0x7f65cc0d04b0 0x7f65cc0d08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.431+0000 7f65d1800640 1 --2- 172.21.15.39:0/231820970 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65cc15cc00 0x7f65cc15d010 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.431+0000 7f65d1800640 1 --2- 172.21.15.39:0/231820970 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f65cc0cfa20 0x7f65cc0cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.431+0000 7f65d1800640 1 -- 172.21.15.39:0/231820970 >> 172.21.15.39:0/231820970 conn(0x7f65cc06cfc0 msgr2=0x7f65cc06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:42.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.431+0000 7f65d1800640 1 -- 172.21.15.39:0/231820970 shutdown_connections 2024-09-21T07:45:42.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.431+0000 7f65d1800640 1 -- 172.21.15.39:0/231820970 wait complete. 2024-09-21T07:45:42.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.433+0000 7f65d1800640 1 Processor -- start 2024-09-21T07:45:42.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.433+0000 7f65d1800640 1 -- start start 2024-09-21T07:45:42.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.435+0000 7f65d1800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f65cc0cfa20 0x7f65cc0cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:42.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.435+0000 7f65d1800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65cc0d04b0 0x7f65cc0c6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:42.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.435+0000 7f65d1800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65cc15cc00 0x7f65cc0ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:42.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.435+0000 7f65d1800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc06ecf0 con 0x7f65cc0d04b0 2024-09-21T07:45:42.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.435+0000 7f65d1800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f65cc074f60 con 0x7f65cc15cc00 2024-09-21T07:45:42.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.435+0000 7f65d1800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f65cc06ee70 con 0x7f65cc0cfa20 2024-09-21T07:45:42.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.435+0000 7f65cb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65cc0d04b0 0x7f65cc0c6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:42.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.435+0000 7f65cb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65cc0d04b0 0x7f65cc0c6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53836/0 (socket says 172.21.15.39:53836) 2024-09-21T07:45:42.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.435+0000 7f65cb400640 1 -- 172.21.15.39:0/370144392 learned_addr learned my addr 172.21.15.39:0/370144392 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:42.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.435+0000 7f65cbe00640 1 --2- 172.21.15.39:0/370144392 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f65cc0cfa20 0x7f65cc0cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:42.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.435+0000 7f65d0e00640 1 --2- 172.21.15.39:0/370144392 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65cc15cc00 0x7f65cc0ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:42.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.435+0000 7f65cb400640 1 -- 172.21.15.39:0/370144392 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f65cc0cfa20 msgr2=0x7f65cc0cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:42.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.435+0000 7f65cb400640 1 --2- 172.21.15.39:0/370144392 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f65cc0cfa20 0x7f65cc0cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.435+0000 7f65cb400640 1 -- 172.21.15.39:0/370144392 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65cc15cc00 msgr2=0x7f65cc0ae930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:42.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.435+0000 7f65cb400640 1 --2- 172.21.15.39:0/370144392 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65cc15cc00 0x7f65cc0ae930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.435+0000 7f65cb400640 1 -- 172.21.15.39:0/370144392 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65cc0cc0d0 con 0x7f65cc0d04b0 2024-09-21T07:45:42.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.435+0000 7f65d0e00640 1 --2- 172.21.15.39:0/370144392 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65cc15cc00 0x7f65cc0ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:45:42.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.436+0000 7f65cb400640 1 --2- 172.21.15.39:0/370144392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65cc0d04b0 0x7f65cc0c6fc0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f65b8034040 tx=0x7f65b8003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:42.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.436+0000 7f65c3e00640 1 -- 172.21.15.39:0/370144392 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b803e030 con 0x7f65cc0d04b0 2024-09-21T07:45:42.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.436+0000 7f65c3e00640 1 -- 172.21.15.39:0/370144392 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65b8003d10 con 0x7f65cc0d04b0 2024-09-21T07:45:42.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.436+0000 7f65c3e00640 1 -- 172.21.15.39:0/370144392 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b8035040 con 0x7f65cc0d04b0 2024-09-21T07:45:42.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.436+0000 7f65d1800640 1 -- 172.21.15.39:0/370144392 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65cc1f54e0 con 0x7f65cc0d04b0 2024-09-21T07:45:42.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.436+0000 7f65d1800640 1 -- 172.21.15.39:0/370144392 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65cc1f5aa0 con 0x7f65cc0d04b0 2024-09-21T07:45:42.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.437+0000 7f65d1800640 1 -- 172.21.15.39:0/370144392 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65cc0d08c0 con 0x7f65cc0d04b0 2024-09-21T07:45:42.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.439+0000 7f65c3e00640 1 -- 172.21.15.39:0/370144392 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f65b8041030 con 0x7f65cc0d04b0 2024-09-21T07:45:42.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.440+0000 7f65c3e00640 1 --2- 172.21.15.39:0/370144392 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f65ac078bc0 0x7f65ac07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:42.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.440+0000 7f65c3e00640 1 -- 172.21.15.39:0/370144392 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 6844+0+0 (secure 0 0 0) 0x7f65b80bb8e0 con 0x7f65cc0d04b0 2024-09-21T07:45:42.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.441+0000 7f65cbe00640 1 --2- 172.21.15.39:0/370144392 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f65ac078bc0 0x7f65ac07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:42.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.441+0000 7f65cbe00640 1 --2- 172.21.15.39:0/370144392 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f65ac078bc0 0x7f65ac07b0d0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f65bc004590 tx=0x7f65bc00b390 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:42.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.444+0000 7f65c3e00640 1 -- 172.21.15.39:0/370144392 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f65b8083260 con 0x7f65cc0d04b0 2024-09-21T07:45:42.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.639+0000 7f65d1800640 1 -- 172.21.15.39:0/370144392 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f65cc158a80 con 0x7f65ac078bc0 2024-09-21T07:45:42.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.648+0000 7f65c3e00640 1 -- 172.21.15.39:0/370144392 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+490976 (secure 0 0 0) 0x7f65cc158a80 con 0x7f65ac078bc0 2024-09-21T07:45:42.657 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:45:42.660 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:45:42.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.655+0000 7f65d1800640 1 -- 172.21.15.39:0/370144392 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f65ac078bc0 msgr2=0x7f65ac07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:42.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.655+0000 7f65d1800640 1 --2- 172.21.15.39:0/370144392 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f65ac078bc0 0x7f65ac07b0d0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f65bc004590 tx=0x7f65bc00b390 comp rx=0 tx=0).stop 2024-09-21T07:45:42.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.655+0000 7f65d1800640 1 -- 172.21.15.39:0/370144392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65cc0d04b0 msgr2=0x7f65cc0c6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:42.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.655+0000 7f65d1800640 1 --2- 172.21.15.39:0/370144392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65cc0d04b0 0x7f65cc0c6fc0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f65b8034040 tx=0x7f65b8003820 comp rx=0 tx=0).stop 2024-09-21T07:45:42.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.655+0000 7f65d1800640 1 -- 172.21.15.39:0/370144392 shutdown_connections 2024-09-21T07:45:42.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.655+0000 7f65d1800640 1 --2- 172.21.15.39:0/370144392 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f65ac078bc0 0x7f65ac07b0d0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.655+0000 7f65d1800640 1 --2- 172.21.15.39:0/370144392 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65cc15cc00 0x7f65cc0ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.655+0000 7f65d1800640 1 --2- 172.21.15.39:0/370144392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65cc0d04b0 0x7f65cc0c6fc0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.655+0000 7f65d1800640 1 --2- 172.21.15.39:0/370144392 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f65cc0cfa20 0x7f65cc0cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.655+0000 7f65d1800640 1 -- 172.21.15.39:0/370144392 >> 172.21.15.39:0/370144392 conn(0x7f65cc06cfc0 msgr2=0x7f65cc157db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:42.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.655+0000 7f65d1800640 1 -- 172.21.15.39:0/370144392 shutdown_connections 2024-09-21T07:45:42.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.655+0000 7f65d1800640 1 -- 172.21.15.39:0/370144392 wait complete. 2024-09-21T07:45:42.681 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":532,"stamp":"2024-09-21T07:45:41.786929+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":285,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":303332,"kb_used_data":16360,"kb_used_omap":12,"kb_used_meta":286771,"kb_avail":754671388,"statfs":{"total":773094113280,"available":772783501312,"internally_reserved":0,"allocated":16752640,"data_stored":9322596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":293654064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001700"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.689192+0000","last_change":"2024-09-21T07:45:36.689192+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.689192+0000","last_undegraded":"2024-09-21T07:45:36.689192+0000","last_fullsized":"2024-09-21T07:45:36.689192+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.665560+0000","last_change":"2024-09-21T07:45:36.665560+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.665560+0000","last_undegraded":"2024-09-21T07:45:36.665560+0000","last_fullsized":"2024-09-21T07:45:36.665560+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.675392+0000","last_change":"2024-09-21T07:45:36.675392+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.675392+0000","last_undegraded":"2024-09-21T07:45:36.675392+0000","last_fullsized":"2024-09-21T07:45:36.675392+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.664845+0000","last_change":"2024-09-21T07:45:36.664845+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.664845+0000","last_undegraded":"2024-09-21T07:45:36.664845+0000","last_fullsized":"2024-09-21T07:45:36.664845+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689578+0000","last_change":"2024-09-21T07:45:37.706582+0000","last_active":"2024-09-21T07:45:38.689578+0000","last_peered":"2024-09-21T07:45:38.689578+0000","last_clean":"2024-09-21T07:45:38.689578+0000","last_became_active":"2024-09-21T07:45:37.706243+0000","last_became_peered":"2024-09-21T07:45:37.706243+0000","last_unstale":"2024-09-21T07:45:38.689578+0000","last_undegraded":"2024-09-21T07:45:38.689578+0000","last_fullsized":"2024-09-21T07:45:38.689578+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:20:35.055835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.676932+0000","last_change":"2024-09-21T07:45:36.676932+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.676932+0000","last_undegraded":"2024-09-21T07:45:36.676932+0000","last_fullsized":"2024-09-21T07:45:36.676932+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.674875+0000","last_change":"2024-09-21T07:45:36.674875+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.674875+0000","last_undegraded":"2024-09-21T07:45:36.674875+0000","last_fullsized":"2024-09-21T07:45:36.674875+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.687461+0000","last_change":"2024-09-21T07:45:37.694980+0000","last_active":"2024-09-21T07:45:38.687461+0000","last_peered":"2024-09-21T07:45:38.687461+0000","last_clean":"2024-09-21T07:45:38.687461+0000","last_became_active":"2024-09-21T07:45:37.689627+0000","last_became_peered":"2024-09-21T07:45:37.689627+0000","last_unstale":"2024-09-21T07:45:38.687461+0000","last_undegraded":"2024-09-21T07:45:38.687461+0000","last_fullsized":"2024-09-21T07:45:38.687461+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:32:36.418482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.677810+0000","last_change":"2024-09-21T07:45:36.677810+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.677810+0000","last_undegraded":"2024-09-21T07:45:36.677810+0000","last_fullsized":"2024-09-21T07:45:36.677810+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.680253+0000","last_change":"2024-09-21T07:45:36.680253+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.680253+0000","last_undegraded":"2024-09-21T07:45:36.680253+0000","last_fullsized":"2024-09-21T07:45:36.680253+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.664838+0000","last_change":"2024-09-21T07:45:36.664838+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.664838+0000","last_undegraded":"2024-09-21T07:45:36.664838+0000","last_fullsized":"2024-09-21T07:45:36.664838+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.688646+0000","last_change":"2024-09-21T07:45:36.688646+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.688646+0000","last_undegraded":"2024-09-21T07:45:36.688646+0000","last_fullsized":"2024-09-21T07:45:36.688646+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.676713+0000","last_change":"2024-09-21T07:45:36.676713+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.676713+0000","last_undegraded":"2024-09-21T07:45:36.676713+0000","last_fullsized":"2024-09-21T07:45:36.676713+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.687739+0000","last_change":"2024-09-21T07:45:36.687739+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.687739+0000","last_undegraded":"2024-09-21T07:45:36.687739+0000","last_fullsized":"2024-09-21T07:45:36.687739+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.684481+0000","last_change":"2024-09-21T07:45:36.684481+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.684481+0000","last_undegraded":"2024-09-21T07:45:36.684481+0000","last_fullsized":"2024-09-21T07:45:36.684481+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.677447+0000","last_change":"2024-09-21T07:45:36.677447+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.677447+0000","last_undegraded":"2024-09-21T07:45:36.677447+0000","last_fullsized":"2024-09-21T07:45:36.677447+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.677254+0000","last_change":"2024-09-21T07:45:36.677254+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.677254+0000","last_undegraded":"2024-09-21T07:45:36.677254+0000","last_fullsized":"2024-09-21T07:45:36.677254+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689982+0000","last_change":"2024-09-21T07:45:37.704608+0000","last_active":"2024-09-21T07:45:38.689982+0000","last_peered":"2024-09-21T07:45:38.689982+0000","last_clean":"2024-09-21T07:45:38.689982+0000","last_became_active":"2024-09-21T07:45:37.703051+0000","last_became_peered":"2024-09-21T07:45:37.703051+0000","last_unstale":"2024-09-21T07:45:38.689982+0000","last_undegraded":"2024-09-21T07:45:38.689982+0000","last_fullsized":"2024-09-21T07:45:38.689982+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:33.889448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.675086+0000","last_change":"2024-09-21T07:45:36.675086+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.675086+0000","last_undegraded":"2024-09-21T07:45:36.675086+0000","last_fullsized":"2024-09-21T07:45:36.675086+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689772+0000","last_change":"2024-09-21T07:45:37.698151+0000","last_active":"2024-09-21T07:45:38.689772+0000","last_peered":"2024-09-21T07:45:38.689772+0000","last_clean":"2024-09-21T07:45:38.689772+0000","last_became_active":"2024-09-21T07:45:37.697818+0000","last_became_peered":"2024-09-21T07:45:37.697818+0000","last_unstale":"2024-09-21T07:45:38.689772+0000","last_undegraded":"2024-09-21T07:45:38.689772+0000","last_fullsized":"2024-09-21T07:45:38.689772+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:57.867727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.664937+0000","last_change":"2024-09-21T07:45:36.664937+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.664937+0000","last_undegraded":"2024-09-21T07:45:36.664937+0000","last_fullsized":"2024-09-21T07:45:36.664937+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.691525+0000","last_change":"2024-09-21T07:45:36.691525+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.691525+0000","last_undegraded":"2024-09-21T07:45:36.691525+0000","last_fullsized":"2024-09-21T07:45:36.691525+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.666008+0000","last_change":"2024-09-21T07:45:36.666008+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.666008+0000","last_undegraded":"2024-09-21T07:45:36.666008+0000","last_fullsized":"2024-09-21T07:45:36.666008+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.676435+0000","last_change":"2024-09-21T07:45:36.676435+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.676435+0000","last_undegraded":"2024-09-21T07:45:36.676435+0000","last_fullsized":"2024-09-21T07:45:36.676435+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.675973+0000","last_change":"2024-09-21T07:45:36.675973+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.675973+0000","last_undegraded":"2024-09-21T07:45:36.675973+0000","last_fullsized":"2024-09-21T07:45:36.675973+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.671245+0000","last_change":"2024-09-21T07:45:36.671245+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.671245+0000","last_undegraded":"2024-09-21T07:45:36.671245+0000","last_fullsized":"2024-09-21T07:45:36.671245+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.671879+0000","last_change":"2024-09-21T07:45:36.671879+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.671879+0000","last_undegraded":"2024-09-21T07:45:36.671879+0000","last_fullsized":"2024-09-21T07:45:36.671879+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.681606+0000","last_change":"2024-09-21T07:45:36.681606+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.681606+0000","last_undegraded":"2024-09-21T07:45:36.681606+0000","last_fullsized":"2024-09-21T07:45:36.681606+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.676579+0000","last_change":"2024-09-21T07:45:36.676579+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.676579+0000","last_undegraded":"2024-09-21T07:45:36.676579+0000","last_fullsized":"2024-09-21T07:45:36.676579+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.675479+0000","last_change":"2024-09-21T07:45:36.675479+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.675479+0000","last_undegraded":"2024-09-21T07:45:36.675479+0000","last_fullsized":"2024-09-21T07:45:36.675479+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.687575+0000","last_change":"2024-09-21T07:45:37.685098+0000","last_active":"2024-09-21T07:45:38.687575+0000","last_peered":"2024-09-21T07:45:38.687575+0000","last_clean":"2024-09-21T07:45:38.687575+0000","last_became_active":"2024-09-21T07:45:37.684748+0000","last_became_peered":"2024-09-21T07:45:37.684748+0000","last_unstale":"2024-09-21T07:45:38.687575+0000","last_undegraded":"2024-09-21T07:45:38.687575+0000","last_fullsized":"2024-09-21T07:45:38.687575+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:51:22.048251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.681462+0000","last_change":"2024-09-21T07:45:36.681462+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.681462+0000","last_undegraded":"2024-09-21T07:45:36.681462+0000","last_fullsized":"2024-09-21T07:45:36.681462+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.671410+0000","last_change":"2024-09-21T07:45:36.671410+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.671410+0000","last_undegraded":"2024-09-21T07:45:36.671410+0000","last_fullsized":"2024-09-21T07:45:36.671410+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.699039+0000","last_change":"2024-09-21T07:45:36.697679+0000","last_active":"2024-09-21T07:45:36.699039+0000","last_peered":"2024-09-21T07:45:36.699039+0000","last_clean":"2024-09-21T07:45:36.699039+0000","last_became_active":"2024-09-21T07:45:36.697396+0000","last_became_peered":"2024-09-21T07:45:36.697396+0000","last_unstale":"2024-09-21T07:45:36.699039+0000","last_undegraded":"2024-09-21T07:45:36.699039+0000","last_fullsized":"2024-09-21T07:45:36.699039+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:43:37.693480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":187,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.679060+0000","last_change":"2024-09-21T07:35:27.879605+0000","last_active":"2024-09-21T07:45:36.679060+0000","last_peered":"2024-09-21T07:45:36.679060+0000","last_clean":"2024-09-21T07:45:36.679060+0000","last_became_active":"2024-09-21T07:35:27.879274+0000","last_became_peered":"2024-09-21T07:35:27.879274+0000","last_unstale":"2024-09-21T07:45:36.679060+0000","last_undegraded":"2024-09-21T07:45:36.679060+0000","last_fullsized":"2024-09-21T07:45:36.679060+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:24.241636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":197,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689112+0000","last_change":"2024-09-21T07:35:29.491336+0000","last_active":"2024-09-21T07:45:38.689112+0000","last_peered":"2024-09-21T07:45:38.689112+0000","last_clean":"2024-09-21T07:45:38.689112+0000","last_became_active":"2024-09-21T07:35:26.866484+0000","last_became_peered":"2024-09-21T07:35:26.866484+0000","last_unstale":"2024-09-21T07:45:38.689112+0000","last_undegraded":"2024-09-21T07:45:38.689112+0000","last_fullsized":"2024-09-21T07:45:38.689112+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:10.449665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076010100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":373,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.679314+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:45:36.679314+0000","last_peered":"2024-09-21T07:45:36.679314+0000","last_clean":"2024-09-21T07:45:36.679314+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:45:36.679314+0000","last_undegraded":"2024-09-21T07:45:36.679314+0000","last_fullsized":"2024-09-21T07:45:36.679314+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":252,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.670156+0000","last_change":"2024-09-21T07:34:24.539248+0000","last_active":"2024-09-21T07:45:36.670156+0000","last_peered":"2024-09-21T07:45:36.670156+0000","last_clean":"2024-09-21T07:45:36.670156+0000","last_became_active":"2024-09-21T07:34:23.273465+0000","last_became_peered":"2024-09-21T07:34:23.273465+0000","last_unstale":"2024-09-21T07:45:36.670156+0000","last_undegraded":"2024-09-21T07:45:36.670156+0000","last_fullsized":"2024-09-21T07:45:36.670156+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:55.517452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060627700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.686033+0000","last_change":"2024-09-21T07:45:36.686033+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.686033+0000","last_undegraded":"2024-09-21T07:45:36.686033+0000","last_fullsized":"2024-09-21T07:45:36.686033+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.701635+0000","last_change":"2024-09-21T07:45:36.700703+0000","last_active":"2024-09-21T07:45:36.701635+0000","last_peered":"2024-09-21T07:45:36.701635+0000","last_clean":"2024-09-21T07:45:36.701635+0000","last_became_active":"2024-09-21T07:45:36.700344+0000","last_became_peered":"2024-09-21T07:45:36.700344+0000","last_unstale":"2024-09-21T07:45:36.701635+0000","last_undegraded":"2024-09-21T07:45:36.701635+0000","last_fullsized":"2024-09-21T07:45:36.701635+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:59.539139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689081+0000","last_change":"2024-09-21T07:35:27.864492+0000","last_active":"2024-09-21T07:45:38.689081+0000","last_peered":"2024-09-21T07:45:38.689081+0000","last_clean":"2024-09-21T07:45:38.689081+0000","last_became_active":"2024-09-21T07:35:27.864176+0000","last_became_peered":"2024-09-21T07:35:27.864176+0000","last_unstale":"2024-09-21T07:45:38.689081+0000","last_undegraded":"2024-09-21T07:45:38.689081+0000","last_fullsized":"2024-09-21T07:45:38.689081+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:05.153467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.680405+0000","last_change":"2024-09-21T07:35:29.493678+0000","last_active":"2024-09-21T07:45:36.680405+0000","last_peered":"2024-09-21T07:45:36.680405+0000","last_clean":"2024-09-21T07:45:36.680405+0000","last_became_active":"2024-09-21T07:35:26.870749+0000","last_became_peered":"2024-09-21T07:35:26.870749+0000","last_unstale":"2024-09-21T07:45:36.680405+0000","last_undegraded":"2024-09-21T07:45:36.680405+0000","last_fullsized":"2024-09-21T07:45:36.680405+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:55.367877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010159570000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":246,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688084+0000","last_change":"2024-09-21T07:34:24.597274+0000","last_active":"2024-09-21T07:45:38.688084+0000","last_peered":"2024-09-21T07:45:38.688084+0000","last_clean":"2024-09-21T07:45:38.688084+0000","last_became_active":"2024-09-21T07:34:23.279560+0000","last_became_peered":"2024-09-21T07:34:23.279560+0000","last_unstale":"2024-09-21T07:45:38.688084+0000","last_undegraded":"2024-09-21T07:45:38.688084+0000","last_fullsized":"2024-09-21T07:45:38.688084+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:15.845265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059699799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.676108+0000","last_change":"2024-09-21T07:45:36.676108+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.676108+0000","last_undegraded":"2024-09-21T07:45:36.676108+0000","last_fullsized":"2024-09-21T07:45:36.676108+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.698291+0000","last_change":"2024-09-21T07:45:36.695689+0000","last_active":"2024-09-21T07:45:36.698291+0000","last_peered":"2024-09-21T07:45:36.698291+0000","last_clean":"2024-09-21T07:45:36.698291+0000","last_became_active":"2024-09-21T07:45:36.695386+0000","last_became_peered":"2024-09-21T07:45:36.695386+0000","last_unstale":"2024-09-21T07:45:36.698291+0000","last_undegraded":"2024-09-21T07:45:36.698291+0000","last_fullsized":"2024-09-21T07:45:36.698291+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:13.324095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":187,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.693357+0000","last_change":"2024-09-21T07:35:27.874387+0000","last_active":"2024-09-21T07:45:36.693357+0000","last_peered":"2024-09-21T07:45:36.693357+0000","last_clean":"2024-09-21T07:45:36.693357+0000","last_became_active":"2024-09-21T07:35:27.873277+0000","last_became_peered":"2024-09-21T07:35:27.873277+0000","last_unstale":"2024-09-21T07:45:36.693357+0000","last_undegraded":"2024-09-21T07:45:36.693357+0000","last_fullsized":"2024-09-21T07:45:36.693357+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:04.581463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":196,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688673+0000","last_change":"2024-09-21T07:35:29.387094+0000","last_active":"2024-09-21T07:45:38.688673+0000","last_peered":"2024-09-21T07:45:38.688673+0000","last_clean":"2024-09-21T07:45:38.688673+0000","last_became_active":"2024-09-21T07:35:26.874757+0000","last_became_peered":"2024-09-21T07:35:26.874757+0000","last_unstale":"2024-09-21T07:45:38.688673+0000","last_undegraded":"2024-09-21T07:45:38.688673+0000","last_fullsized":"2024-09-21T07:45:38.688673+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:33.542750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061065700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":248,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.678663+0000","last_change":"2024-09-21T07:34:24.259906+0000","last_active":"2024-09-21T07:45:36.678663+0000","last_peered":"2024-09-21T07:45:36.678663+0000","last_clean":"2024-09-21T07:45:36.678663+0000","last_became_active":"2024-09-21T07:34:23.635282+0000","last_became_peered":"2024-09-21T07:34:23.635282+0000","last_unstale":"2024-09-21T07:45:36.678663+0000","last_undegraded":"2024-09-21T07:45:36.678663+0000","last_fullsized":"2024-09-21T07:45:36.678663+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:00.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001527664,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.670461+0000","last_change":"2024-09-21T07:45:36.670461+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.670461+0000","last_undegraded":"2024-09-21T07:45:36.670461+0000","last_fullsized":"2024-09-21T07:45:36.670461+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.689405+0000","last_change":"2024-09-21T07:45:36.688037+0000","last_active":"2024-09-21T07:45:36.689405+0000","last_peered":"2024-09-21T07:45:36.689405+0000","last_clean":"2024-09-21T07:45:36.689405+0000","last_became_active":"2024-09-21T07:45:36.687731+0000","last_became_peered":"2024-09-21T07:45:36.687731+0000","last_unstale":"2024-09-21T07:45:36.689405+0000","last_undegraded":"2024-09-21T07:45:36.689405+0000","last_fullsized":"2024-09-21T07:45:36.689405+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:31.034405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":187,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.696504+0000","last_change":"2024-09-21T07:35:27.873645+0000","last_active":"2024-09-21T07:45:36.696504+0000","last_peered":"2024-09-21T07:45:36.696504+0000","last_clean":"2024-09-21T07:45:36.696504+0000","last_became_active":"2024-09-21T07:35:27.873245+0000","last_became_peered":"2024-09-21T07:35:27.873245+0000","last_unstale":"2024-09-21T07:45:36.696504+0000","last_undegraded":"2024-09-21T07:45:36.696504+0000","last_fullsized":"2024-09-21T07:45:36.696504+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:55.090953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.698882+0000","last_change":"2024-09-21T07:35:28.861744+0000","last_active":"2024-09-21T07:45:36.698882+0000","last_peered":"2024-09-21T07:45:36.698882+0000","last_clean":"2024-09-21T07:45:36.698882+0000","last_became_active":"2024-09-21T07:35:26.876462+0000","last_became_peered":"2024-09-21T07:35:26.876462+0000","last_unstale":"2024-09-21T07:45:36.698882+0000","last_undegraded":"2024-09-21T07:45:36.698882+0000","last_fullsized":"2024-09-21T07:45:36.698882+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:48.632568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012020920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":245,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.672276+0000","last_change":"2024-09-21T07:34:25.085454+0000","last_active":"2024-09-21T07:45:36.672276+0000","last_peered":"2024-09-21T07:45:36.672276+0000","last_clean":"2024-09-21T07:45:36.672276+0000","last_became_active":"2024-09-21T07:34:23.279205+0000","last_became_peered":"2024-09-21T07:34:23.279205+0000","last_unstale":"2024-09-21T07:45:36.672276+0000","last_undegraded":"2024-09-21T07:45:36.672276+0000","last_fullsized":"2024-09-21T07:45:36.672276+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:54.231666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095320599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.675896+0000","last_change":"2024-09-21T07:45:36.675896+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.675896+0000","last_undegraded":"2024-09-21T07:45:36.675896+0000","last_fullsized":"2024-09-21T07:45:36.675896+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.701499+0000","last_change":"2024-09-21T07:45:36.700878+0000","last_active":"2024-09-21T07:45:36.701499+0000","last_peered":"2024-09-21T07:45:36.701499+0000","last_clean":"2024-09-21T07:45:36.701499+0000","last_became_active":"2024-09-21T07:45:36.700484+0000","last_became_peered":"2024-09-21T07:45:36.700484+0000","last_unstale":"2024-09-21T07:45:36.701499+0000","last_undegraded":"2024-09-21T07:45:36.701499+0000","last_fullsized":"2024-09-21T07:45:36.701499+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:52:12.375777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688964+0000","last_change":"2024-09-21T07:35:27.861373+0000","last_active":"2024-09-21T07:45:38.688964+0000","last_peered":"2024-09-21T07:45:38.688964+0000","last_clean":"2024-09-21T07:45:38.688964+0000","last_became_active":"2024-09-21T07:35:27.861016+0000","last_became_peered":"2024-09-21T07:35:27.861016+0000","last_unstale":"2024-09-21T07:45:38.688964+0000","last_undegraded":"2024-09-21T07:45:38.688964+0000","last_fullsized":"2024-09-21T07:45:38.688964+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:56.043763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":195,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.673133+0000","last_change":"2024-09-21T07:35:29.053677+0000","last_active":"2024-09-21T07:45:36.673133+0000","last_peered":"2024-09-21T07:45:36.673133+0000","last_clean":"2024-09-21T07:45:36.673133+0000","last_became_active":"2024-09-21T07:35:26.875496+0000","last_became_peered":"2024-09-21T07:35:26.875496+0000","last_unstale":"2024-09-21T07:45:36.673133+0000","last_undegraded":"2024-09-21T07:45:36.673133+0000","last_fullsized":"2024-09-21T07:45:36.673133+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:04.850784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065859000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":247,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.672104+0000","last_change":"2024-09-21T07:34:25.028521+0000","last_active":"2024-09-21T07:45:36.672104+0000","last_peered":"2024-09-21T07:45:36.672104+0000","last_clean":"2024-09-21T07:45:36.672104+0000","last_became_active":"2024-09-21T07:34:23.635467+0000","last_became_peered":"2024-09-21T07:34:23.635467+0000","last_unstale":"2024-09-21T07:45:36.672104+0000","last_undegraded":"2024-09-21T07:45:36.672104+0000","last_fullsized":"2024-09-21T07:45:36.672104+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:33.729123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000660558,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.676454+0000","last_change":"2024-09-21T07:45:36.676454+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.676454+0000","last_undegraded":"2024-09-21T07:45:36.676454+0000","last_fullsized":"2024-09-21T07:45:36.676454+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.699376+0000","last_change":"2024-09-21T07:45:36.697573+0000","last_active":"2024-09-21T07:45:36.699376+0000","last_peered":"2024-09-21T07:45:36.699376+0000","last_clean":"2024-09-21T07:45:36.699376+0000","last_became_active":"2024-09-21T07:45:36.697341+0000","last_became_peered":"2024-09-21T07:45:36.697341+0000","last_unstale":"2024-09-21T07:45:36.699376+0000","last_undegraded":"2024-09-21T07:45:36.699376+0000","last_fullsized":"2024-09-21T07:45:36.699376+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:18.613601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":187,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.676705+0000","last_change":"2024-09-21T07:35:27.871201+0000","last_active":"2024-09-21T07:45:36.676705+0000","last_peered":"2024-09-21T07:45:36.676705+0000","last_clean":"2024-09-21T07:45:36.676705+0000","last_became_active":"2024-09-21T07:35:27.870816+0000","last_became_peered":"2024-09-21T07:35:27.870816+0000","last_unstale":"2024-09-21T07:45:36.676705+0000","last_undegraded":"2024-09-21T07:45:36.676705+0000","last_fullsized":"2024-09-21T07:45:36.676705+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:04.787450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":197,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689594+0000","last_change":"2024-09-21T07:35:29.491874+0000","last_active":"2024-09-21T07:45:38.689594+0000","last_peered":"2024-09-21T07:45:38.689594+0000","last_clean":"2024-09-21T07:45:38.689594+0000","last_became_active":"2024-09-21T07:35:26.866381+0000","last_became_peered":"2024-09-21T07:35:26.866381+0000","last_unstale":"2024-09-21T07:45:38.689594+0000","last_undegraded":"2024-09-21T07:45:38.689594+0000","last_fullsized":"2024-09-21T07:45:38.689594+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:36.404647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073235399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":252,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690345+0000","last_change":"2024-09-21T07:34:24.260046+0000","last_active":"2024-09-21T07:45:38.690345+0000","last_peered":"2024-09-21T07:45:38.690345+0000","last_clean":"2024-09-21T07:45:38.690345+0000","last_became_active":"2024-09-21T07:34:23.273431+0000","last_became_peered":"2024-09-21T07:34:23.273431+0000","last_unstale":"2024-09-21T07:45:38.690345+0000","last_undegraded":"2024-09-21T07:45:38.690345+0000","last_fullsized":"2024-09-21T07:45:38.690345+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:18.876057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019803310000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.674849+0000","last_change":"2024-09-21T07:45:36.674849+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.674849+0000","last_undegraded":"2024-09-21T07:45:36.674849+0000","last_fullsized":"2024-09-21T07:45:36.674849+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.687046+0000","last_change":"2024-09-21T07:45:36.684575+0000","last_active":"2024-09-21T07:45:36.687046+0000","last_peered":"2024-09-21T07:45:36.687046+0000","last_clean":"2024-09-21T07:45:36.687046+0000","last_became_active":"2024-09-21T07:45:36.683843+0000","last_became_peered":"2024-09-21T07:45:36.683843+0000","last_unstale":"2024-09-21T07:45:36.687046+0000","last_undegraded":"2024-09-21T07:45:36.687046+0000","last_fullsized":"2024-09-21T07:45:36.687046+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:56.463705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":187,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.676204+0000","last_change":"2024-09-21T07:35:27.871736+0000","last_active":"2024-09-21T07:45:36.676204+0000","last_peered":"2024-09-21T07:45:36.676204+0000","last_clean":"2024-09-21T07:45:36.676204+0000","last_became_active":"2024-09-21T07:35:27.871232+0000","last_became_peered":"2024-09-21T07:35:27.871232+0000","last_unstale":"2024-09-21T07:45:36.676204+0000","last_undegraded":"2024-09-21T07:45:36.676204+0000","last_fullsized":"2024-09-21T07:45:36.676204+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:04.419301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":197,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688913+0000","last_change":"2024-09-21T07:35:29.491246+0000","last_active":"2024-09-21T07:45:38.688913+0000","last_peered":"2024-09-21T07:45:38.688913+0000","last_clean":"2024-09-21T07:45:38.688913+0000","last_became_active":"2024-09-21T07:35:26.866783+0000","last_became_peered":"2024-09-21T07:35:26.866783+0000","last_unstale":"2024-09-21T07:45:38.688913+0000","last_undegraded":"2024-09-21T07:45:38.688913+0000","last_fullsized":"2024-09-21T07:45:38.688913+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:28.881914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098474099999999992,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":247,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.673249+0000","last_change":"2024-09-21T07:34:25.028640+0000","last_active":"2024-09-21T07:45:36.673249+0000","last_peered":"2024-09-21T07:45:36.673249+0000","last_clean":"2024-09-21T07:45:36.673249+0000","last_became_active":"2024-09-21T07:34:23.260950+0000","last_became_peered":"2024-09-21T07:34:23.260950+0000","last_unstale":"2024-09-21T07:45:36.673249+0000","last_undegraded":"2024-09-21T07:45:36.673249+0000","last_fullsized":"2024-09-21T07:45:36.673249+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:23:42.121398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055135099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689869+0000","last_change":"2024-09-21T07:45:37.693385+0000","last_active":"2024-09-21T07:45:38.689869+0000","last_peered":"2024-09-21T07:45:38.689869+0000","last_clean":"2024-09-21T07:45:38.689869+0000","last_became_active":"2024-09-21T07:45:37.691927+0000","last_became_peered":"2024-09-21T07:45:37.691927+0000","last_unstale":"2024-09-21T07:45:38.689869+0000","last_undegraded":"2024-09-21T07:45:38.689869+0000","last_fullsized":"2024-09-21T07:45:38.689869+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:29.208442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.684490+0000","last_change":"2024-09-21T07:45:36.681956+0000","last_active":"2024-09-21T07:45:36.684490+0000","last_peered":"2024-09-21T07:45:36.684490+0000","last_clean":"2024-09-21T07:45:36.684490+0000","last_became_active":"2024-09-21T07:45:36.681656+0000","last_became_peered":"2024-09-21T07:45:36.681656+0000","last_unstale":"2024-09-21T07:45:36.684490+0000","last_undegraded":"2024-09-21T07:45:36.684490+0000","last_fullsized":"2024-09-21T07:45:36.684490+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:04:54.765401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":186,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.680978+0000","last_change":"2024-09-21T07:35:27.872027+0000","last_active":"2024-09-21T07:45:36.680978+0000","last_peered":"2024-09-21T07:45:36.680978+0000","last_clean":"2024-09-21T07:45:36.680978+0000","last_became_active":"2024-09-21T07:35:27.870426+0000","last_became_peered":"2024-09-21T07:35:27.870426+0000","last_unstale":"2024-09-21T07:45:36.680978+0000","last_undegraded":"2024-09-21T07:45:36.680978+0000","last_fullsized":"2024-09-21T07:45:36.680978+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":197,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689126+0000","last_change":"2024-09-21T07:35:29.491008+0000","last_active":"2024-09-21T07:45:38.689126+0000","last_peered":"2024-09-21T07:45:38.689126+0000","last_clean":"2024-09-21T07:45:38.689126+0000","last_became_active":"2024-09-21T07:35:26.866750+0000","last_became_peered":"2024-09-21T07:35:26.866750+0000","last_unstale":"2024-09-21T07:45:38.689126+0000","last_undegraded":"2024-09-21T07:45:38.689126+0000","last_fullsized":"2024-09-21T07:45:38.689126+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:25.490406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083529700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":248,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.679720+0000","last_change":"2024-09-21T07:34:24.260665+0000","last_active":"2024-09-21T07:45:36.679720+0000","last_peered":"2024-09-21T07:45:36.679720+0000","last_clean":"2024-09-21T07:45:36.679720+0000","last_became_active":"2024-09-21T07:34:23.635909+0000","last_became_peered":"2024-09-21T07:34:23.635909+0000","last_unstale":"2024-09-21T07:45:36.679720+0000","last_undegraded":"2024-09-21T07:45:36.679720+0000","last_fullsized":"2024-09-21T07:45:36.679720+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:44.536885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689417+0000","last_change":"2024-09-21T07:45:37.686269+0000","last_active":"2024-09-21T07:45:38.689417+0000","last_peered":"2024-09-21T07:45:38.689417+0000","last_clean":"2024-09-21T07:45:38.689417+0000","last_became_active":"2024-09-21T07:45:37.685971+0000","last_became_peered":"2024-09-21T07:45:37.685971+0000","last_unstale":"2024-09-21T07:45:38.689417+0000","last_undegraded":"2024-09-21T07:45:38.689417+0000","last_fullsized":"2024-09-21T07:45:38.689417+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:23.017715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.696537+0000","last_change":"2024-09-21T07:45:36.695413+0000","last_active":"2024-09-21T07:45:36.696537+0000","last_peered":"2024-09-21T07:45:36.696537+0000","last_clean":"2024-09-21T07:45:36.696537+0000","last_became_active":"2024-09-21T07:45:36.695045+0000","last_became_peered":"2024-09-21T07:45:36.695045+0000","last_unstale":"2024-09-21T07:45:36.696537+0000","last_undegraded":"2024-09-21T07:45:36.696537+0000","last_fullsized":"2024-09-21T07:45:36.696537+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:32.061491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689492+0000","last_change":"2024-09-21T07:35:27.874772+0000","last_active":"2024-09-21T07:45:38.689492+0000","last_peered":"2024-09-21T07:45:38.689492+0000","last_clean":"2024-09-21T07:45:38.689492+0000","last_became_active":"2024-09-21T07:35:27.874434+0000","last_became_peered":"2024-09-21T07:35:27.874434+0000","last_unstale":"2024-09-21T07:45:38.689492+0000","last_undegraded":"2024-09-21T07:45:38.689492+0000","last_fullsized":"2024-09-21T07:45:38.689492+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:31.998755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":199,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.669713+0000","last_change":"2024-09-21T07:35:28.859153+0000","last_active":"2024-09-21T07:45:36.669713+0000","last_peered":"2024-09-21T07:45:36.669713+0000","last_clean":"2024-09-21T07:45:36.669713+0000","last_became_active":"2024-09-21T07:35:26.874428+0000","last_became_peered":"2024-09-21T07:35:26.874428+0000","last_unstale":"2024-09-21T07:45:36.669713+0000","last_undegraded":"2024-09-21T07:45:36.669713+0000","last_fullsized":"2024-09-21T07:45:36.669713+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:04.487621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048557299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":252,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.669176+0000","last_change":"2024-09-21T07:34:24.538845+0000","last_active":"2024-09-21T07:45:36.669176+0000","last_peered":"2024-09-21T07:45:36.669176+0000","last_clean":"2024-09-21T07:45:36.669176+0000","last_became_active":"2024-09-21T07:34:23.636372+0000","last_became_peered":"2024-09-21T07:34:23.636372+0000","last_unstale":"2024-09-21T07:45:36.669176+0000","last_undegraded":"2024-09-21T07:45:36.669176+0000","last_fullsized":"2024-09-21T07:45:36.669176+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:29.393813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841451,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.683953+0000","last_change":"2024-09-21T07:45:36.683953+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.683953+0000","last_undegraded":"2024-09-21T07:45:36.683953+0000","last_fullsized":"2024-09-21T07:45:36.683953+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"156'1","reported_seq":16,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690230+0000","last_change":"2024-09-21T07:45:36.698933+0000","last_active":"2024-09-21T07:45:38.690230+0000","last_peered":"2024-09-21T07:45:38.690230+0000","last_clean":"2024-09-21T07:45:38.690230+0000","last_became_active":"2024-09-21T07:45:36.698651+0000","last_became_peered":"2024-09-21T07:45:36.698651+0000","last_unstale":"2024-09-21T07:45:38.690230+0000","last_undegraded":"2024-09-21T07:45:38.690230+0000","last_fullsized":"2024-09-21T07:45:38.690230+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:17:31.377323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":186,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.677861+0000","last_change":"2024-09-21T07:35:27.877801+0000","last_active":"2024-09-21T07:45:36.677861+0000","last_peered":"2024-09-21T07:45:36.677861+0000","last_clean":"2024-09-21T07:45:36.677861+0000","last_became_active":"2024-09-21T07:35:27.877464+0000","last_became_peered":"2024-09-21T07:35:27.877464+0000","last_unstale":"2024-09-21T07:45:36.677861+0000","last_undegraded":"2024-09-21T07:45:36.677861+0000","last_fullsized":"2024-09-21T07:45:36.677861+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:08.814929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.676325+0000","last_change":"2024-09-21T07:35:28.863732+0000","last_active":"2024-09-21T07:45:36.676325+0000","last_peered":"2024-09-21T07:45:36.676325+0000","last_clean":"2024-09-21T07:45:36.676325+0000","last_became_active":"2024-09-21T07:35:26.877392+0000","last_became_peered":"2024-09-21T07:35:26.877392+0000","last_unstale":"2024-09-21T07:45:36.676325+0000","last_undegraded":"2024-09-21T07:45:36.676325+0000","last_fullsized":"2024-09-21T07:45:36.676325+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071425700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":252,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689803+0000","last_change":"2024-09-21T07:34:24.259686+0000","last_active":"2024-09-21T07:45:38.689803+0000","last_peered":"2024-09-21T07:45:38.689803+0000","last_clean":"2024-09-21T07:45:38.689803+0000","last_became_active":"2024-09-21T07:34:23.274405+0000","last_became_peered":"2024-09-21T07:34:23.274405+0000","last_unstale":"2024-09-21T07:45:38.689803+0000","last_undegraded":"2024-09-21T07:45:38.689803+0000","last_fullsized":"2024-09-21T07:45:38.689803+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:26.304693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022009920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.664831+0000","last_change":"2024-09-21T07:45:36.664831+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.664831+0000","last_undegraded":"2024-09-21T07:45:36.664831+0000","last_fullsized":"2024-09-21T07:45:36.664831+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.692374+0000","last_change":"2024-09-21T07:45:36.689252+0000","last_active":"2024-09-21T07:45:36.692374+0000","last_peered":"2024-09-21T07:45:36.692374+0000","last_clean":"2024-09-21T07:45:36.692374+0000","last_became_active":"2024-09-21T07:45:36.688915+0000","last_became_peered":"2024-09-21T07:45:36.688915+0000","last_unstale":"2024-09-21T07:45:36.692374+0000","last_undegraded":"2024-09-21T07:45:36.692374+0000","last_fullsized":"2024-09-21T07:45:36.692374+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:41:51.910757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":189,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.687721+0000","last_change":"2024-09-21T07:35:27.867336+0000","last_active":"2024-09-21T07:45:38.687721+0000","last_peered":"2024-09-21T07:45:38.687721+0000","last_clean":"2024-09-21T07:45:38.687721+0000","last_became_active":"2024-09-21T07:35:27.867015+0000","last_became_peered":"2024-09-21T07:35:27.867015+0000","last_unstale":"2024-09-21T07:45:38.687721+0000","last_undegraded":"2024-09-21T07:45:38.687721+0000","last_fullsized":"2024-09-21T07:45:38.687721+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:59.214139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.679176+0000","last_change":"2024-09-21T07:35:29.493137+0000","last_active":"2024-09-21T07:45:36.679176+0000","last_peered":"2024-09-21T07:45:36.679176+0000","last_clean":"2024-09-21T07:45:36.679176+0000","last_became_active":"2024-09-21T07:35:26.872230+0000","last_became_peered":"2024-09-21T07:35:26.872230+0000","last_unstale":"2024-09-21T07:45:36.679176+0000","last_undegraded":"2024-09-21T07:45:36.679176+0000","last_fullsized":"2024-09-21T07:45:36.679176+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:24.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001197449,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":248,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.679175+0000","last_change":"2024-09-21T07:34:24.258754+0000","last_active":"2024-09-21T07:45:36.679175+0000","last_peered":"2024-09-21T07:45:36.679175+0000","last_clean":"2024-09-21T07:45:36.679175+0000","last_became_active":"2024-09-21T07:34:23.272657+0000","last_became_peered":"2024-09-21T07:34:23.272657+0000","last_unstale":"2024-09-21T07:45:36.679175+0000","last_undegraded":"2024-09-21T07:45:36.679175+0000","last_fullsized":"2024-09-21T07:45:36.679175+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:05.477099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067143200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.692417+0000","last_change":"2024-09-21T07:45:36.692417+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.692417+0000","last_undegraded":"2024-09-21T07:45:36.692417+0000","last_fullsized":"2024-09-21T07:45:36.692417+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.689641+0000","last_change":"2024-09-21T07:45:36.687920+0000","last_active":"2024-09-21T07:45:36.689641+0000","last_peered":"2024-09-21T07:45:36.689641+0000","last_clean":"2024-09-21T07:45:36.689641+0000","last_became_active":"2024-09-21T07:45:36.687561+0000","last_became_peered":"2024-09-21T07:45:36.687561+0000","last_unstale":"2024-09-21T07:45:36.689641+0000","last_undegraded":"2024-09-21T07:45:36.689641+0000","last_fullsized":"2024-09-21T07:45:36.689641+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:47:40.418087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":187,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.674812+0000","last_change":"2024-09-21T07:35:27.876021+0000","last_active":"2024-09-21T07:45:36.674812+0000","last_peered":"2024-09-21T07:45:36.674812+0000","last_clean":"2024-09-21T07:45:36.674812+0000","last_became_active":"2024-09-21T07:35:27.875724+0000","last_became_peered":"2024-09-21T07:35:27.875724+0000","last_unstale":"2024-09-21T07:45:36.674812+0000","last_undegraded":"2024-09-21T07:45:36.674812+0000","last_fullsized":"2024-09-21T07:45:36.674812+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:03.022812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.679122+0000","last_change":"2024-09-21T07:35:29.492570+0000","last_active":"2024-09-21T07:45:36.679122+0000","last_peered":"2024-09-21T07:45:36.679122+0000","last_clean":"2024-09-21T07:45:36.679122+0000","last_became_active":"2024-09-21T07:35:26.874316+0000","last_became_peered":"2024-09-21T07:35:26.874316+0000","last_unstale":"2024-09-21T07:45:36.679122+0000","last_undegraded":"2024-09-21T07:45:36.679122+0000","last_fullsized":"2024-09-21T07:45:36.679122+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:16.512323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011990760000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":246,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.674075+0000","last_change":"2024-09-21T07:34:25.085232+0000","last_active":"2024-09-21T07:45:36.674075+0000","last_peered":"2024-09-21T07:45:36.674075+0000","last_clean":"2024-09-21T07:45:36.674075+0000","last_became_active":"2024-09-21T07:34:23.277807+0000","last_became_peered":"2024-09-21T07:34:23.277807+0000","last_unstale":"2024-09-21T07:45:36.674075+0000","last_undegraded":"2024-09-21T07:45:36.674075+0000","last_fullsized":"2024-09-21T07:45:36.674075+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:36.362919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086619099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689375+0000","last_change":"2024-09-21T07:45:37.685148+0000","last_active":"2024-09-21T07:45:38.689375+0000","last_peered":"2024-09-21T07:45:38.689375+0000","last_clean":"2024-09-21T07:45:38.689375+0000","last_became_active":"2024-09-21T07:45:37.684774+0000","last_became_peered":"2024-09-21T07:45:37.684774+0000","last_unstale":"2024-09-21T07:45:38.689375+0000","last_undegraded":"2024-09-21T07:45:38.689375+0000","last_fullsized":"2024-09-21T07:45:38.689375+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:25.616419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"156'1","reported_seq":16,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.687687+0000","last_change":"2024-09-21T07:45:36.686891+0000","last_active":"2024-09-21T07:45:38.687687+0000","last_peered":"2024-09-21T07:45:38.687687+0000","last_clean":"2024-09-21T07:45:38.687687+0000","last_became_active":"2024-09-21T07:45:36.684124+0000","last_became_peered":"2024-09-21T07:45:36.684124+0000","last_unstale":"2024-09-21T07:45:38.687687+0000","last_undegraded":"2024-09-21T07:45:38.687687+0000","last_fullsized":"2024-09-21T07:45:38.687687+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:00.150983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":186,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.679954+0000","last_change":"2024-09-21T07:35:27.872798+0000","last_active":"2024-09-21T07:45:36.679954+0000","last_peered":"2024-09-21T07:45:36.679954+0000","last_clean":"2024-09-21T07:45:36.679954+0000","last_became_active":"2024-09-21T07:35:27.872483+0000","last_became_peered":"2024-09-21T07:35:27.872483+0000","last_unstale":"2024-09-21T07:45:36.679954+0000","last_undegraded":"2024-09-21T07:45:36.679954+0000","last_fullsized":"2024-09-21T07:45:36.679954+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:11.933469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":193,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.679781+0000","last_change":"2024-09-21T07:35:29.206723+0000","last_active":"2024-09-21T07:45:36.679781+0000","last_peered":"2024-09-21T07:45:36.679781+0000","last_clean":"2024-09-21T07:45:36.679781+0000","last_became_active":"2024-09-21T07:35:26.872377+0000","last_became_peered":"2024-09-21T07:35:26.872377+0000","last_unstale":"2024-09-21T07:45:36.679781+0000","last_undegraded":"2024-09-21T07:45:36.679781+0000","last_fullsized":"2024-09-21T07:45:36.679781+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:29.086862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083077200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":246,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689123+0000","last_change":"2024-09-21T07:34:24.597672+0000","last_active":"2024-09-21T07:45:38.689123+0000","last_peered":"2024-09-21T07:45:38.689123+0000","last_clean":"2024-09-21T07:45:38.689123+0000","last_became_active":"2024-09-21T07:34:23.267770+0000","last_became_peered":"2024-09-21T07:34:23.267770+0000","last_unstale":"2024-09-21T07:45:38.689123+0000","last_undegraded":"2024-09-21T07:45:38.689123+0000","last_fullsized":"2024-09-21T07:45:38.689123+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:51.629476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086378500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689895+0000","last_change":"2024-09-21T07:45:37.691898+0000","last_active":"2024-09-21T07:45:38.689895+0000","last_peered":"2024-09-21T07:45:38.689895+0000","last_clean":"2024-09-21T07:45:38.689895+0000","last_became_active":"2024-09-21T07:45:37.691507+0000","last_became_peered":"2024-09-21T07:45:37.691507+0000","last_unstale":"2024-09-21T07:45:38.689895+0000","last_undegraded":"2024-09-21T07:45:38.689895+0000","last_fullsized":"2024-09-21T07:45:38.689895+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:14:43.900850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"156'1","reported_seq":16,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689228+0000","last_change":"2024-09-21T07:45:36.694668+0000","last_active":"2024-09-21T07:45:38.689228+0000","last_peered":"2024-09-21T07:45:38.689228+0000","last_clean":"2024-09-21T07:45:38.689228+0000","last_became_active":"2024-09-21T07:45:36.694405+0000","last_became_peered":"2024-09-21T07:45:36.694405+0000","last_unstale":"2024-09-21T07:45:38.689228+0000","last_undegraded":"2024-09-21T07:45:38.689228+0000","last_fullsized":"2024-09-21T07:45:38.689228+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:08.552085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":186,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.680841+0000","last_change":"2024-09-21T07:35:27.870121+0000","last_active":"2024-09-21T07:45:36.680841+0000","last_peered":"2024-09-21T07:45:36.680841+0000","last_clean":"2024-09-21T07:45:36.680841+0000","last_became_active":"2024-09-21T07:35:27.869811+0000","last_became_peered":"2024-09-21T07:35:27.869811+0000","last_unstale":"2024-09-21T07:45:36.680841+0000","last_undegraded":"2024-09-21T07:45:36.680841+0000","last_fullsized":"2024-09-21T07:45:36.680841+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:31.778461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.696395+0000","last_change":"2024-09-21T07:35:28.860699+0000","last_active":"2024-09-21T07:45:36.696395+0000","last_peered":"2024-09-21T07:45:36.696395+0000","last_clean":"2024-09-21T07:45:36.696395+0000","last_became_active":"2024-09-21T07:35:26.876779+0000","last_became_peered":"2024-09-21T07:35:26.876779+0000","last_unstale":"2024-09-21T07:45:36.696395+0000","last_undegraded":"2024-09-21T07:45:36.696395+0000","last_fullsized":"2024-09-21T07:45:36.696395+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:28.286403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000913188,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":247,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.700060+0000","last_change":"2024-09-21T07:34:24.260181+0000","last_active":"2024-09-21T07:45:36.700060+0000","last_peered":"2024-09-21T07:45:36.700060+0000","last_clean":"2024-09-21T07:45:36.700060+0000","last_became_active":"2024-09-21T07:34:23.275580+0000","last_became_peered":"2024-09-21T07:34:23.275580+0000","last_unstale":"2024-09-21T07:45:36.700060+0000","last_undegraded":"2024-09-21T07:45:36.700060+0000","last_fullsized":"2024-09-21T07:45:36.700060+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:49.857622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014706980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689850+0000","last_change":"2024-09-21T07:45:37.698246+0000","last_active":"2024-09-21T07:45:38.689850+0000","last_peered":"2024-09-21T07:45:38.689850+0000","last_clean":"2024-09-21T07:45:38.689850+0000","last_became_active":"2024-09-21T07:45:37.697792+0000","last_became_peered":"2024-09-21T07:45:37.697792+0000","last_unstale":"2024-09-21T07:45:38.689850+0000","last_undegraded":"2024-09-21T07:45:38.689850+0000","last_fullsized":"2024-09-21T07:45:38.689850+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:06.139752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.694910+0000","last_change":"2024-09-21T07:45:36.693696+0000","last_active":"2024-09-21T07:45:36.694910+0000","last_peered":"2024-09-21T07:45:36.694910+0000","last_clean":"2024-09-21T07:45:36.694910+0000","last_became_active":"2024-09-21T07:45:36.693211+0000","last_became_peered":"2024-09-21T07:45:36.693211+0000","last_unstale":"2024-09-21T07:45:36.694910+0000","last_undegraded":"2024-09-21T07:45:36.694910+0000","last_fullsized":"2024-09-21T07:45:36.694910+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:15.970506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":188,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.672760+0000","last_change":"2024-09-21T07:35:27.875393+0000","last_active":"2024-09-21T07:45:36.672760+0000","last_peered":"2024-09-21T07:45:36.672760+0000","last_clean":"2024-09-21T07:45:36.672760+0000","last_became_active":"2024-09-21T07:35:27.874671+0000","last_became_peered":"2024-09-21T07:35:27.874671+0000","last_unstale":"2024-09-21T07:45:36.672760+0000","last_undegraded":"2024-09-21T07:45:36.672760+0000","last_fullsized":"2024-09-21T07:45:36.672760+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":193,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.678090+0000","last_change":"2024-09-21T07:35:29.206296+0000","last_active":"2024-09-21T07:45:36.678090+0000","last_peered":"2024-09-21T07:45:36.678090+0000","last_clean":"2024-09-21T07:45:36.678090+0000","last_became_active":"2024-09-21T07:35:26.874575+0000","last_became_peered":"2024-09-21T07:35:26.874575+0000","last_unstale":"2024-09-21T07:45:36.678090+0000","last_undegraded":"2024-09-21T07:45:36.678090+0000","last_fullsized":"2024-09-21T07:45:36.678090+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:22.771154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061011599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":252,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690260+0000","last_change":"2024-09-21T07:34:24.259112+0000","last_active":"2024-09-21T07:45:38.690260+0000","last_peered":"2024-09-21T07:45:38.690260+0000","last_clean":"2024-09-21T07:45:38.690260+0000","last_became_active":"2024-09-21T07:34:23.635444+0000","last_became_peered":"2024-09-21T07:34:23.635444+0000","last_unstale":"2024-09-21T07:45:38.690260+0000","last_undegraded":"2024-09-21T07:45:38.690260+0000","last_fullsized":"2024-09-21T07:45:38.690260+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:46.255385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001952636,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.672205+0000","last_change":"2024-09-21T07:45:36.672205+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.672205+0000","last_undegraded":"2024-09-21T07:45:36.672205+0000","last_fullsized":"2024-09-21T07:45:36.672205+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.696954+0000","last_change":"2024-09-21T07:45:36.696284+0000","last_active":"2024-09-21T07:45:36.696954+0000","last_peered":"2024-09-21T07:45:36.696954+0000","last_clean":"2024-09-21T07:45:36.696954+0000","last_became_active":"2024-09-21T07:45:36.695952+0000","last_became_peered":"2024-09-21T07:45:36.695952+0000","last_unstale":"2024-09-21T07:45:36.696954+0000","last_undegraded":"2024-09-21T07:45:36.696954+0000","last_fullsized":"2024-09-21T07:45:36.696954+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:01.857023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":188,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.673558+0000","last_change":"2024-09-21T07:35:27.878356+0000","last_active":"2024-09-21T07:45:36.673558+0000","last_peered":"2024-09-21T07:45:36.673558+0000","last_clean":"2024-09-21T07:45:36.673558+0000","last_became_active":"2024-09-21T07:35:27.877982+0000","last_became_peered":"2024-09-21T07:35:27.877982+0000","last_unstale":"2024-09-21T07:45:36.673558+0000","last_undegraded":"2024-09-21T07:45:36.673558+0000","last_fullsized":"2024-09-21T07:45:36.673558+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":196,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.671879+0000","last_change":"2024-09-21T07:35:29.053131+0000","last_active":"2024-09-21T07:45:36.671879+0000","last_peered":"2024-09-21T07:45:36.671879+0000","last_clean":"2024-09-21T07:45:36.671879+0000","last_became_active":"2024-09-21T07:35:26.870516+0000","last_became_peered":"2024-09-21T07:35:26.870516+0000","last_unstale":"2024-09-21T07:45:36.671879+0000","last_undegraded":"2024-09-21T07:45:36.671879+0000","last_fullsized":"2024-09-21T07:45:36.671879+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:10.924491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070705699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":252,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.669780+0000","last_change":"2024-09-21T07:34:24.539152+0000","last_active":"2024-09-21T07:45:36.669780+0000","last_peered":"2024-09-21T07:45:36.669780+0000","last_clean":"2024-09-21T07:45:36.669780+0000","last_became_active":"2024-09-21T07:34:23.259573+0000","last_became_peered":"2024-09-21T07:34:23.259573+0000","last_unstale":"2024-09-21T07:45:36.669780+0000","last_undegraded":"2024-09-21T07:45:36.669780+0000","last_fullsized":"2024-09-21T07:45:36.669780+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:26.986900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058108400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689007+0000","last_change":"2024-09-21T07:45:37.689304+0000","last_active":"2024-09-21T07:45:38.689007+0000","last_peered":"2024-09-21T07:45:38.689007+0000","last_clean":"2024-09-21T07:45:38.689007+0000","last_became_active":"2024-09-21T07:45:37.688985+0000","last_became_peered":"2024-09-21T07:45:37.688985+0000","last_unstale":"2024-09-21T07:45:38.689007+0000","last_undegraded":"2024-09-21T07:45:38.689007+0000","last_fullsized":"2024-09-21T07:45:38.689007+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:03.596216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.700099+0000","last_change":"2024-09-21T07:45:36.697335+0000","last_active":"2024-09-21T07:45:36.700099+0000","last_peered":"2024-09-21T07:45:36.700099+0000","last_clean":"2024-09-21T07:45:36.700099+0000","last_became_active":"2024-09-21T07:45:36.697036+0000","last_became_peered":"2024-09-21T07:45:36.697036+0000","last_unstale":"2024-09-21T07:45:36.700099+0000","last_undegraded":"2024-09-21T07:45:36.700099+0000","last_fullsized":"2024-09-21T07:45:36.700099+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:16.366485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":188,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.671824+0000","last_change":"2024-09-21T07:35:27.873726+0000","last_active":"2024-09-21T07:45:36.671824+0000","last_peered":"2024-09-21T07:45:36.671824+0000","last_clean":"2024-09-21T07:45:36.671824+0000","last_became_active":"2024-09-21T07:35:27.871421+0000","last_became_peered":"2024-09-21T07:35:27.871421+0000","last_unstale":"2024-09-21T07:45:36.671824+0000","last_undegraded":"2024-09-21T07:45:36.671824+0000","last_fullsized":"2024-09-21T07:45:36.671824+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:00.168915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.699581+0000","last_change":"2024-09-21T07:35:28.861032+0000","last_active":"2024-09-21T07:45:36.699581+0000","last_peered":"2024-09-21T07:45:36.699581+0000","last_clean":"2024-09-21T07:45:36.699581+0000","last_became_active":"2024-09-21T07:35:26.873149+0000","last_became_peered":"2024-09-21T07:35:26.873149+0000","last_unstale":"2024-09-21T07:45:36.699581+0000","last_undegraded":"2024-09-21T07:45:36.699581+0000","last_fullsized":"2024-09-21T07:45:36.699581+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:59.435981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011988369999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":247,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.679365+0000","last_change":"2024-09-21T07:34:24.259902+0000","last_active":"2024-09-21T07:45:36.679365+0000","last_peered":"2024-09-21T07:45:36.679365+0000","last_clean":"2024-09-21T07:45:36.679365+0000","last_became_active":"2024-09-21T07:34:23.271902+0000","last_became_peered":"2024-09-21T07:34:23.271902+0000","last_unstale":"2024-09-21T07:45:36.679365+0000","last_undegraded":"2024-09-21T07:45:36.679365+0000","last_fullsized":"2024-09-21T07:45:36.679365+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:59.439525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001686358,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.671064+0000","last_change":"2024-09-21T07:45:36.671064+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.671064+0000","last_undegraded":"2024-09-21T07:45:36.671064+0000","last_fullsized":"2024-09-21T07:45:36.671064+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"157'1","reported_seq":16,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689021+0000","last_change":"2024-09-21T07:45:36.673147+0000","last_active":"2024-09-21T07:45:38.689021+0000","last_peered":"2024-09-21T07:45:38.689021+0000","last_clean":"2024-09-21T07:45:38.689021+0000","last_became_active":"2024-09-21T07:45:36.672816+0000","last_became_peered":"2024-09-21T07:45:36.672816+0000","last_unstale":"2024-09-21T07:45:38.689021+0000","last_undegraded":"2024-09-21T07:45:38.689021+0000","last_fullsized":"2024-09-21T07:45:38.689021+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:04:18.857870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":187,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.673815+0000","last_change":"2024-09-21T07:35:27.872637+0000","last_active":"2024-09-21T07:45:36.673815+0000","last_peered":"2024-09-21T07:45:36.673815+0000","last_clean":"2024-09-21T07:45:36.673815+0000","last_became_active":"2024-09-21T07:35:27.872149+0000","last_became_peered":"2024-09-21T07:35:27.872149+0000","last_unstale":"2024-09-21T07:45:36.673815+0000","last_undegraded":"2024-09-21T07:45:36.673815+0000","last_fullsized":"2024-09-21T07:45:36.673815+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:27.697872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":195,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.678962+0000","last_change":"2024-09-21T07:35:29.493362+0000","last_active":"2024-09-21T07:45:36.678962+0000","last_peered":"2024-09-21T07:45:36.678962+0000","last_clean":"2024-09-21T07:45:36.678962+0000","last_became_active":"2024-09-21T07:35:26.873018+0000","last_became_peered":"2024-09-21T07:35:26.873018+0000","last_unstale":"2024-09-21T07:45:36.678962+0000","last_undegraded":"2024-09-21T07:45:36.678962+0000","last_fullsized":"2024-09-21T07:45:36.678962+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:02.460557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012496510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":251,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689489+0000","last_change":"2024-09-21T07:34:24.259662+0000","last_active":"2024-09-21T07:45:38.689489+0000","last_peered":"2024-09-21T07:45:38.689489+0000","last_clean":"2024-09-21T07:45:38.689489+0000","last_became_active":"2024-09-21T07:34:23.278812+0000","last_became_peered":"2024-09-21T07:34:23.278812+0000","last_unstale":"2024-09-21T07:45:38.689489+0000","last_undegraded":"2024-09-21T07:45:38.689489+0000","last_fullsized":"2024-09-21T07:45:38.689489+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:49.829974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356463,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.675060+0000","last_change":"2024-09-21T07:45:36.675060+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.675060+0000","last_undegraded":"2024-09-21T07:45:36.675060+0000","last_fullsized":"2024-09-21T07:45:36.675060+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689740+0000","last_change":"2024-09-21T07:45:36.693420+0000","last_active":"2024-09-21T07:45:38.689740+0000","last_peered":"2024-09-21T07:45:38.689740+0000","last_clean":"2024-09-21T07:45:38.689740+0000","last_became_active":"2024-09-21T07:45:36.692969+0000","last_became_peered":"2024-09-21T07:45:36.692969+0000","last_unstale":"2024-09-21T07:45:38.689740+0000","last_undegraded":"2024-09-21T07:45:38.689740+0000","last_fullsized":"2024-09-21T07:45:38.689740+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:09:22.417758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":189,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.687613+0000","last_change":"2024-09-21T07:35:27.868448+0000","last_active":"2024-09-21T07:45:38.687613+0000","last_peered":"2024-09-21T07:45:38.687613+0000","last_clean":"2024-09-21T07:45:38.687613+0000","last_became_active":"2024-09-21T07:35:27.868105+0000","last_became_peered":"2024-09-21T07:35:27.868105+0000","last_unstale":"2024-09-21T07:45:38.687613+0000","last_undegraded":"2024-09-21T07:45:38.687613+0000","last_fullsized":"2024-09-21T07:45:38.687613+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:41.468702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.678932+0000","last_change":"2024-09-21T07:35:29.492974+0000","last_active":"2024-09-21T07:45:36.678932+0000","last_peered":"2024-09-21T07:45:36.678932+0000","last_clean":"2024-09-21T07:45:36.678932+0000","last_became_active":"2024-09-21T07:35:26.872165+0000","last_became_peered":"2024-09-21T07:35:26.872165+0000","last_unstale":"2024-09-21T07:45:36.678932+0000","last_undegraded":"2024-09-21T07:45:36.678932+0000","last_fullsized":"2024-09-21T07:45:36.678932+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:59.912520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013795179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":245,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.672175+0000","last_change":"2024-09-21T07:34:25.084861+0000","last_active":"2024-09-21T07:45:36.672175+0000","last_peered":"2024-09-21T07:45:36.672175+0000","last_clean":"2024-09-21T07:45:36.672175+0000","last_became_active":"2024-09-21T07:34:23.277708+0000","last_became_peered":"2024-09-21T07:34:23.277708+0000","last_unstale":"2024-09-21T07:45:36.672175+0000","last_undegraded":"2024-09-21T07:45:36.672175+0000","last_fullsized":"2024-09-21T07:45:36.672175+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:40.816488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000597566,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.677283+0000","last_change":"2024-09-21T07:45:36.677283+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.677283+0000","last_undegraded":"2024-09-21T07:45:36.677283+0000","last_fullsized":"2024-09-21T07:45:36.677283+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.688738+0000","last_change":"2024-09-21T07:45:36.687709+0000","last_active":"2024-09-21T07:45:36.688738+0000","last_peered":"2024-09-21T07:45:36.688738+0000","last_clean":"2024-09-21T07:45:36.688738+0000","last_became_active":"2024-09-21T07:45:36.684703+0000","last_became_peered":"2024-09-21T07:45:36.684703+0000","last_unstale":"2024-09-21T07:45:36.688738+0000","last_undegraded":"2024-09-21T07:45:36.688738+0000","last_fullsized":"2024-09-21T07:45:36.688738+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:59.601883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":189,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688381+0000","last_change":"2024-09-21T07:35:27.874640+0000","last_active":"2024-09-21T07:45:38.688381+0000","last_peered":"2024-09-21T07:45:38.688381+0000","last_clean":"2024-09-21T07:45:38.688381+0000","last_became_active":"2024-09-21T07:35:27.874265+0000","last_became_peered":"2024-09-21T07:35:27.874265+0000","last_unstale":"2024-09-21T07:45:38.688381+0000","last_undegraded":"2024-09-21T07:45:38.688381+0000","last_fullsized":"2024-09-21T07:45:38.688381+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:45.754654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":193,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.680159+0000","last_change":"2024-09-21T07:35:29.207392+0000","last_active":"2024-09-21T07:45:36.680159+0000","last_peered":"2024-09-21T07:45:36.680159+0000","last_clean":"2024-09-21T07:45:36.680159+0000","last_became_active":"2024-09-21T07:35:26.870917+0000","last_became_peered":"2024-09-21T07:35:26.870917+0000","last_unstale":"2024-09-21T07:45:36.680159+0000","last_undegraded":"2024-09-21T07:45:36.680159+0000","last_fullsized":"2024-09-21T07:45:36.680159+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:59.159937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059198499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":251,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689049+0000","last_change":"2024-09-21T07:34:24.258709+0000","last_active":"2024-09-21T07:45:38.689049+0000","last_peered":"2024-09-21T07:45:38.689049+0000","last_clean":"2024-09-21T07:45:38.689049+0000","last_became_active":"2024-09-21T07:34:23.279042+0000","last_became_peered":"2024-09-21T07:34:23.279042+0000","last_unstale":"2024-09-21T07:45:38.689049+0000","last_undegraded":"2024-09-21T07:45:38.689049+0000","last_fullsized":"2024-09-21T07:45:38.689049+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:01.762117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019282100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.676888+0000","last_change":"2024-09-21T07:45:36.676888+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.676888+0000","last_undegraded":"2024-09-21T07:45:36.676888+0000","last_fullsized":"2024-09-21T07:45:36.676888+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.699451+0000","last_change":"2024-09-21T07:45:36.698592+0000","last_active":"2024-09-21T07:45:36.699451+0000","last_peered":"2024-09-21T07:45:36.699451+0000","last_clean":"2024-09-21T07:45:36.699451+0000","last_became_active":"2024-09-21T07:45:36.698011+0000","last_became_peered":"2024-09-21T07:45:36.698011+0000","last_unstale":"2024-09-21T07:45:36.699451+0000","last_undegraded":"2024-09-21T07:45:36.699451+0000","last_fullsized":"2024-09-21T07:45:36.699451+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:00.979027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":187,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.676080+0000","last_change":"2024-09-21T07:35:27.870260+0000","last_active":"2024-09-21T07:45:36.676080+0000","last_peered":"2024-09-21T07:45:36.676080+0000","last_clean":"2024-09-21T07:45:36.676080+0000","last_became_active":"2024-09-21T07:35:27.869448+0000","last_became_peered":"2024-09-21T07:35:27.869448+0000","last_unstale":"2024-09-21T07:45:36.676080+0000","last_undegraded":"2024-09-21T07:45:36.676080+0000","last_fullsized":"2024-09-21T07:45:36.676080+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:16.946790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":196,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688161+0000","last_change":"2024-09-21T07:35:29.386367+0000","last_active":"2024-09-21T07:45:38.688161+0000","last_peered":"2024-09-21T07:45:38.688161+0000","last_clean":"2024-09-21T07:45:38.688161+0000","last_became_active":"2024-09-21T07:35:26.879238+0000","last_became_peered":"2024-09-21T07:35:26.879238+0000","last_unstale":"2024-09-21T07:45:38.688161+0000","last_undegraded":"2024-09-21T07:45:38.688161+0000","last_fullsized":"2024-09-21T07:45:38.688161+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:16.528260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092223100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":245,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.677233+0000","last_change":"2024-09-21T07:34:25.085635+0000","last_active":"2024-09-21T07:45:36.677233+0000","last_peered":"2024-09-21T07:45:36.677233+0000","last_clean":"2024-09-21T07:45:36.677233+0000","last_became_active":"2024-09-21T07:34:23.273566+0000","last_became_peered":"2024-09-21T07:34:23.273566+0000","last_unstale":"2024-09-21T07:45:36.677233+0000","last_undegraded":"2024-09-21T07:45:36.677233+0000","last_fullsized":"2024-09-21T07:45:36.677233+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:14.142272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079897300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.676058+0000","last_change":"2024-09-21T07:45:36.676058+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.676058+0000","last_undegraded":"2024-09-21T07:45:36.676058+0000","last_fullsized":"2024-09-21T07:45:36.676058+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.689075+0000","last_change":"2024-09-21T07:45:36.685900+0000","last_active":"2024-09-21T07:45:36.689075+0000","last_peered":"2024-09-21T07:45:36.689075+0000","last_clean":"2024-09-21T07:45:36.689075+0000","last_became_active":"2024-09-21T07:45:36.685578+0000","last_became_peered":"2024-09-21T07:45:36.685578+0000","last_unstale":"2024-09-21T07:45:36.689075+0000","last_undegraded":"2024-09-21T07:45:36.689075+0000","last_fullsized":"2024-09-21T07:45:36.689075+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:03.369431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":187,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.677035+0000","last_change":"2024-09-21T07:35:27.878728+0000","last_active":"2024-09-21T07:45:36.677035+0000","last_peered":"2024-09-21T07:45:36.677035+0000","last_clean":"2024-09-21T07:45:36.677035+0000","last_became_active":"2024-09-21T07:35:27.878335+0000","last_became_peered":"2024-09-21T07:35:27.878335+0000","last_unstale":"2024-09-21T07:45:36.677035+0000","last_undegraded":"2024-09-21T07:45:36.677035+0000","last_fullsized":"2024-09-21T07:45:36.677035+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:25:37.731771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":199,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.668981+0000","last_change":"2024-09-21T07:35:28.858418+0000","last_active":"2024-09-21T07:45:36.668981+0000","last_peered":"2024-09-21T07:45:36.668981+0000","last_clean":"2024-09-21T07:45:36.668981+0000","last_became_active":"2024-09-21T07:35:26.876944+0000","last_became_peered":"2024-09-21T07:35:26.876944+0000","last_unstale":"2024-09-21T07:45:36.668981+0000","last_undegraded":"2024-09-21T07:45:36.668981+0000","last_fullsized":"2024-09-21T07:45:36.668981+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:04.514304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090862799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":245,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688906+0000","last_change":"2024-09-21T07:34:24.597926+0000","last_active":"2024-09-21T07:45:38.688906+0000","last_peered":"2024-09-21T07:45:38.688906+0000","last_clean":"2024-09-21T07:45:38.688906+0000","last_became_active":"2024-09-21T07:34:23.268926+0000","last_became_peered":"2024-09-21T07:34:23.268926+0000","last_unstale":"2024-09-21T07:45:38.688906+0000","last_undegraded":"2024-09-21T07:45:38.688906+0000","last_fullsized":"2024-09-21T07:45:38.688906+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:46.190752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101851,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.676259+0000","last_change":"2024-09-21T07:45:36.676259+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.676259+0000","last_undegraded":"2024-09-21T07:45:36.676259+0000","last_fullsized":"2024-09-21T07:45:36.676259+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.685429+0000","last_change":"2024-09-21T07:45:36.683898+0000","last_active":"2024-09-21T07:45:36.685429+0000","last_peered":"2024-09-21T07:45:36.685429+0000","last_clean":"2024-09-21T07:45:36.685429+0000","last_became_active":"2024-09-21T07:45:36.683536+0000","last_became_peered":"2024-09-21T07:45:36.683536+0000","last_unstale":"2024-09-21T07:45:36.685429+0000","last_undegraded":"2024-09-21T07:45:36.685429+0000","last_fullsized":"2024-09-21T07:45:36.685429+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:47.039079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688581+0000","last_change":"2024-09-21T07:35:27.877426+0000","last_active":"2024-09-21T07:45:38.688581+0000","last_peered":"2024-09-21T07:45:38.688581+0000","last_clean":"2024-09-21T07:45:38.688581+0000","last_became_active":"2024-09-21T07:35:27.877050+0000","last_became_peered":"2024-09-21T07:35:27.877050+0000","last_unstale":"2024-09-21T07:45:38.688581+0000","last_undegraded":"2024-09-21T07:45:38.688581+0000","last_fullsized":"2024-09-21T07:45:38.688581+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:36.268334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.673914+0000","last_change":"2024-09-21T07:35:28.862273+0000","last_active":"2024-09-21T07:45:36.673914+0000","last_peered":"2024-09-21T07:45:36.673914+0000","last_clean":"2024-09-21T07:45:36.673914+0000","last_became_active":"2024-09-21T07:35:26.874011+0000","last_became_peered":"2024-09-21T07:35:26.874011+0000","last_unstale":"2024-09-21T07:45:36.673914+0000","last_undegraded":"2024-09-21T07:45:36.673914+0000","last_fullsized":"2024-09-21T07:45:36.673914+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:11.986950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071644799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":246,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.697773+0000","last_change":"2024-09-21T07:34:24.259794+0000","last_active":"2024-09-21T07:45:36.697773+0000","last_peered":"2024-09-21T07:45:36.697773+0000","last_clean":"2024-09-21T07:45:36.697773+0000","last_became_active":"2024-09-21T07:34:23.265679+0000","last_became_peered":"2024-09-21T07:34:23.265679+0000","last_unstale":"2024-09-21T07:45:36.697773+0000","last_undegraded":"2024-09-21T07:45:36.697773+0000","last_fullsized":"2024-09-21T07:45:36.697773+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:02.575796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001298518,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.673113+0000","last_change":"2024-09-21T07:45:36.673113+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.673113+0000","last_undegraded":"2024-09-21T07:45:36.673113+0000","last_fullsized":"2024-09-21T07:45:36.673113+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689224+0000","last_change":"2024-09-21T07:45:36.695633+0000","last_active":"2024-09-21T07:45:38.689224+0000","last_peered":"2024-09-21T07:45:38.689224+0000","last_clean":"2024-09-21T07:45:38.689224+0000","last_became_active":"2024-09-21T07:45:36.695319+0000","last_became_peered":"2024-09-21T07:45:36.695319+0000","last_unstale":"2024-09-21T07:45:38.689224+0000","last_undegraded":"2024-09-21T07:45:38.689224+0000","last_fullsized":"2024-09-21T07:45:38.689224+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:36.781831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":186,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.680833+0000","last_change":"2024-09-21T07:35:27.877725+0000","last_active":"2024-09-21T07:45:36.680833+0000","last_peered":"2024-09-21T07:45:36.680833+0000","last_clean":"2024-09-21T07:45:36.680833+0000","last_became_active":"2024-09-21T07:35:27.877398+0000","last_became_peered":"2024-09-21T07:35:27.877398+0000","last_unstale":"2024-09-21T07:45:36.680833+0000","last_undegraded":"2024-09-21T07:45:36.680833+0000","last_fullsized":"2024-09-21T07:45:36.680833+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:05.859762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.693485+0000","last_change":"2024-09-21T07:35:28.860612+0000","last_active":"2024-09-21T07:45:36.693485+0000","last_peered":"2024-09-21T07:45:36.693485+0000","last_clean":"2024-09-21T07:45:36.693485+0000","last_became_active":"2024-09-21T07:35:26.876562+0000","last_became_peered":"2024-09-21T07:35:26.876562+0000","last_unstale":"2024-09-21T07:45:36.693485+0000","last_undegraded":"2024-09-21T07:45:36.693485+0000","last_fullsized":"2024-09-21T07:45:36.693485+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:06.824331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012252470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":247,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.679653+0000","last_change":"2024-09-21T07:34:24.260330+0000","last_active":"2024-09-21T07:45:36.679653+0000","last_peered":"2024-09-21T07:45:36.679653+0000","last_clean":"2024-09-21T07:45:36.679653+0000","last_became_active":"2024-09-21T07:34:23.637210+0000","last_became_peered":"2024-09-21T07:34:23.637210+0000","last_unstale":"2024-09-21T07:45:36.679653+0000","last_undegraded":"2024-09-21T07:45:36.679653+0000","last_fullsized":"2024-09-21T07:45:36.679653+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.488451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001537093,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.673352+0000","last_change":"2024-09-21T07:45:36.673352+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.673352+0000","last_undegraded":"2024-09-21T07:45:36.673352+0000","last_fullsized":"2024-09-21T07:45:36.673352+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.677935+0000","last_change":"2024-09-21T07:45:36.676063+0000","last_active":"2024-09-21T07:45:36.677935+0000","last_peered":"2024-09-21T07:45:36.677935+0000","last_clean":"2024-09-21T07:45:36.677935+0000","last_became_active":"2024-09-21T07:45:36.675750+0000","last_became_peered":"2024-09-21T07:45:36.675750+0000","last_unstale":"2024-09-21T07:45:36.677935+0000","last_undegraded":"2024-09-21T07:45:36.677935+0000","last_fullsized":"2024-09-21T07:45:36.677935+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:20:48.069405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":187,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.673680+0000","last_change":"2024-09-21T07:35:27.874757+0000","last_active":"2024-09-21T07:45:36.673680+0000","last_peered":"2024-09-21T07:45:36.673680+0000","last_clean":"2024-09-21T07:45:36.673680+0000","last_became_active":"2024-09-21T07:35:27.874360+0000","last_became_peered":"2024-09-21T07:35:27.874360+0000","last_unstale":"2024-09-21T07:45:36.673680+0000","last_undegraded":"2024-09-21T07:45:36.673680+0000","last_fullsized":"2024-09-21T07:45:36.673680+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:17.646912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.680294+0000","last_change":"2024-09-21T07:35:29.493431+0000","last_active":"2024-09-21T07:45:36.680294+0000","last_peered":"2024-09-21T07:45:36.680294+0000","last_clean":"2024-09-21T07:45:36.680294+0000","last_became_active":"2024-09-21T07:35:26.873691+0000","last_became_peered":"2024-09-21T07:35:26.873691+0000","last_unstale":"2024-09-21T07:45:36.680294+0000","last_undegraded":"2024-09-21T07:45:36.680294+0000","last_fullsized":"2024-09-21T07:45:36.680294+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:51.705051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001162464,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":245,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.687901+0000","last_change":"2024-09-21T07:34:24.597286+0000","last_active":"2024-09-21T07:45:38.687901+0000","last_peered":"2024-09-21T07:45:38.687901+0000","last_clean":"2024-09-21T07:45:38.687901+0000","last_became_active":"2024-09-21T07:34:23.267133+0000","last_became_peered":"2024-09-21T07:34:23.267133+0000","last_unstale":"2024-09-21T07:45:38.687901+0000","last_undegraded":"2024-09-21T07:45:38.687901+0000","last_fullsized":"2024-09-21T07:45:38.687901+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:18.257010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.670174+0000","last_change":"2024-09-21T07:45:36.670174+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.670174+0000","last_undegraded":"2024-09-21T07:45:36.670174+0000","last_fullsized":"2024-09-21T07:45:36.670174+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.694069+0000","last_change":"2024-09-21T07:45:36.693348+0000","last_active":"2024-09-21T07:45:36.694069+0000","last_peered":"2024-09-21T07:45:36.694069+0000","last_clean":"2024-09-21T07:45:36.694069+0000","last_became_active":"2024-09-21T07:45:36.692821+0000","last_became_peered":"2024-09-21T07:45:36.692821+0000","last_unstale":"2024-09-21T07:45:36.694069+0000","last_undegraded":"2024-09-21T07:45:36.694069+0000","last_fullsized":"2024-09-21T07:45:36.694069+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:15:15.557087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":192,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.668792+0000","last_change":"2024-09-21T07:35:27.864458+0000","last_active":"2024-09-21T07:45:36.668792+0000","last_peered":"2024-09-21T07:45:36.668792+0000","last_clean":"2024-09-21T07:45:36.668792+0000","last_became_active":"2024-09-21T07:35:27.864136+0000","last_became_peered":"2024-09-21T07:35:27.864136+0000","last_unstale":"2024-09-21T07:45:36.668792+0000","last_undegraded":"2024-09-21T07:45:36.668792+0000","last_fullsized":"2024-09-21T07:45:36.668792+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:20.162240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":199,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.669445+0000","last_change":"2024-09-21T07:35:28.858784+0000","last_active":"2024-09-21T07:45:36.669445+0000","last_peered":"2024-09-21T07:45:36.669445+0000","last_clean":"2024-09-21T07:45:36.669445+0000","last_became_active":"2024-09-21T07:35:26.876890+0000","last_became_peered":"2024-09-21T07:35:26.876890+0000","last_unstale":"2024-09-21T07:45:36.669445+0000","last_undegraded":"2024-09-21T07:45:36.669445+0000","last_fullsized":"2024-09-21T07:45:36.669445+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:40.738327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064543499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":251,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689383+0000","last_change":"2024-09-21T07:34:24.258712+0000","last_active":"2024-09-21T07:45:38.689383+0000","last_peered":"2024-09-21T07:45:38.689383+0000","last_clean":"2024-09-21T07:45:38.689383+0000","last_became_active":"2024-09-21T07:34:23.274692+0000","last_became_peered":"2024-09-21T07:34:23.274692+0000","last_unstale":"2024-09-21T07:45:38.689383+0000","last_undegraded":"2024-09-21T07:45:38.689383+0000","last_fullsized":"2024-09-21T07:45:38.689383+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:43.279454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018051479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.673682+0000","last_change":"2024-09-21T07:45:36.673682+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.673682+0000","last_undegraded":"2024-09-21T07:45:36.673682+0000","last_fullsized":"2024-09-21T07:45:36.673682+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.702127+0000","last_change":"2024-09-21T07:45:36.700873+0000","last_active":"2024-09-21T07:45:36.702127+0000","last_peered":"2024-09-21T07:45:36.702127+0000","last_clean":"2024-09-21T07:45:36.702127+0000","last_became_active":"2024-09-21T07:45:36.700526+0000","last_became_peered":"2024-09-21T07:45:36.700526+0000","last_unstale":"2024-09-21T07:45:36.702127+0000","last_undegraded":"2024-09-21T07:45:36.702127+0000","last_fullsized":"2024-09-21T07:45:36.702127+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:08:53.416498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689061+0000","last_change":"2024-09-21T07:35:27.878020+0000","last_active":"2024-09-21T07:45:38.689061+0000","last_peered":"2024-09-21T07:45:38.689061+0000","last_clean":"2024-09-21T07:45:38.689061+0000","last_became_active":"2024-09-21T07:35:27.877532+0000","last_became_peered":"2024-09-21T07:35:27.877532+0000","last_unstale":"2024-09-21T07:45:38.689061+0000","last_undegraded":"2024-09-21T07:45:38.689061+0000","last_fullsized":"2024-09-21T07:45:38.689061+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:19.756073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.698780+0000","last_change":"2024-09-21T07:35:28.861049+0000","last_active":"2024-09-21T07:45:36.698780+0000","last_peered":"2024-09-21T07:45:36.698780+0000","last_clean":"2024-09-21T07:45:36.698780+0000","last_became_active":"2024-09-21T07:35:26.876819+0000","last_became_peered":"2024-09-21T07:35:26.876819+0000","last_unstale":"2024-09-21T07:45:36.698780+0000","last_undegraded":"2024-09-21T07:45:36.698780+0000","last_fullsized":"2024-09-21T07:45:36.698780+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:19.633803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055780599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":245,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.672049+0000","last_change":"2024-09-21T07:34:25.084849+0000","last_active":"2024-09-21T07:45:36.672049+0000","last_peered":"2024-09-21T07:45:36.672049+0000","last_clean":"2024-09-21T07:45:36.672049+0000","last_became_active":"2024-09-21T07:34:23.277713+0000","last_became_peered":"2024-09-21T07:34:23.277713+0000","last_unstale":"2024-09-21T07:45:36.672049+0000","last_undegraded":"2024-09-21T07:45:36.672049+0000","last_fullsized":"2024-09-21T07:45:36.672049+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:05.811810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059079499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.677740+0000","last_change":"2024-09-21T07:45:36.677740+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.677740+0000","last_undegraded":"2024-09-21T07:45:36.677740+0000","last_fullsized":"2024-09-21T07:45:36.677740+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690040+0000","last_change":"2024-09-21T07:45:36.693932+0000","last_active":"2024-09-21T07:45:38.690040+0000","last_peered":"2024-09-21T07:45:38.690040+0000","last_clean":"2024-09-21T07:45:38.690040+0000","last_became_active":"2024-09-21T07:45:36.693640+0000","last_became_peered":"2024-09-21T07:45:36.693640+0000","last_unstale":"2024-09-21T07:45:38.690040+0000","last_undegraded":"2024-09-21T07:45:38.690040+0000","last_fullsized":"2024-09-21T07:45:38.690040+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:04.701463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":188,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.672603+0000","last_change":"2024-09-21T07:35:27.884248+0000","last_active":"2024-09-21T07:45:36.672603+0000","last_peered":"2024-09-21T07:45:36.672603+0000","last_clean":"2024-09-21T07:45:36.672603+0000","last_became_active":"2024-09-21T07:35:27.883861+0000","last_became_peered":"2024-09-21T07:35:27.883861+0000","last_unstale":"2024-09-21T07:45:36.672603+0000","last_undegraded":"2024-09-21T07:45:36.672603+0000","last_fullsized":"2024-09-21T07:45:36.672603+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":193,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.680061+0000","last_change":"2024-09-21T07:35:29.207135+0000","last_active":"2024-09-21T07:45:36.680061+0000","last_peered":"2024-09-21T07:45:36.680061+0000","last_clean":"2024-09-21T07:45:36.680061+0000","last_became_active":"2024-09-21T07:35:26.872787+0000","last_became_peered":"2024-09-21T07:35:26.872787+0000","last_unstale":"2024-09-21T07:45:36.680061+0000","last_undegraded":"2024-09-21T07:45:36.680061+0000","last_fullsized":"2024-09-21T07:45:36.680061+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:27.543217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011157509999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":251,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.668930+0000","last_change":"2024-09-21T07:34:24.538610+0000","last_active":"2024-09-21T07:45:36.668930+0000","last_peered":"2024-09-21T07:45:36.668930+0000","last_clean":"2024-09-21T07:45:36.668930+0000","last_became_active":"2024-09-21T07:34:23.259700+0000","last_became_peered":"2024-09-21T07:34:23.259700+0000","last_unstale":"2024-09-21T07:45:36.668930+0000","last_undegraded":"2024-09-21T07:45:36.668930+0000","last_fullsized":"2024-09-21T07:45:36.668930+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:57.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070522399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690121+0000","last_change":"2024-09-21T07:45:37.714568+0000","last_active":"2024-09-21T07:45:38.690121+0000","last_peered":"2024-09-21T07:45:38.690121+0000","last_clean":"2024-09-21T07:45:38.690121+0000","last_became_active":"2024-09-21T07:45:37.714235+0000","last_became_peered":"2024-09-21T07:45:37.714235+0000","last_unstale":"2024-09-21T07:45:38.690121+0000","last_undegraded":"2024-09-21T07:45:38.690121+0000","last_fullsized":"2024-09-21T07:45:38.690121+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:29.318374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"156'2","reported_seq":17,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.687809+0000","last_change":"2024-09-21T07:45:36.685978+0000","last_active":"2024-09-21T07:45:38.687809+0000","last_peered":"2024-09-21T07:45:38.687809+0000","last_clean":"2024-09-21T07:45:38.687809+0000","last_became_active":"2024-09-21T07:45:36.685307+0000","last_became_peered":"2024-09-21T07:45:36.685307+0000","last_unstale":"2024-09-21T07:45:38.687809+0000","last_undegraded":"2024-09-21T07:45:38.687809+0000","last_fullsized":"2024-09-21T07:45:38.687809+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:18.642482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":192,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.669313+0000","last_change":"2024-09-21T07:35:27.868024+0000","last_active":"2024-09-21T07:45:36.669313+0000","last_peered":"2024-09-21T07:45:36.669313+0000","last_clean":"2024-09-21T07:45:36.669313+0000","last_became_active":"2024-09-21T07:35:27.867339+0000","last_became_peered":"2024-09-21T07:35:27.867339+0000","last_unstale":"2024-09-21T07:45:36.669313+0000","last_undegraded":"2024-09-21T07:45:36.669313+0000","last_fullsized":"2024-09-21T07:45:36.669313+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":199,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.669113+0000","last_change":"2024-09-21T07:35:28.858505+0000","last_active":"2024-09-21T07:45:36.669113+0000","last_peered":"2024-09-21T07:45:36.669113+0000","last_clean":"2024-09-21T07:45:36.669113+0000","last_became_active":"2024-09-21T07:35:26.877320+0000","last_became_peered":"2024-09-21T07:35:26.877320+0000","last_unstale":"2024-09-21T07:45:36.669113+0000","last_undegraded":"2024-09-21T07:45:36.669113+0000","last_fullsized":"2024-09-21T07:45:36.669113+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:53.672782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244119999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":246,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689180+0000","last_change":"2024-09-21T07:34:24.597597+0000","last_active":"2024-09-21T07:45:38.689180+0000","last_peered":"2024-09-21T07:45:38.689180+0000","last_clean":"2024-09-21T07:45:38.689180+0000","last_became_active":"2024-09-21T07:34:23.635204+0000","last_became_peered":"2024-09-21T07:34:23.635204+0000","last_unstale":"2024-09-21T07:45:38.689180+0000","last_undegraded":"2024-09-21T07:45:38.689180+0000","last_fullsized":"2024-09-21T07:45:38.689180+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:48.644789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081139199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.689181+0000","last_change":"2024-09-21T07:45:36.689181+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.689181+0000","last_undegraded":"2024-09-21T07:45:36.689181+0000","last_fullsized":"2024-09-21T07:45:36.689181+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.696649+0000","last_change":"2024-09-21T07:45:36.693812+0000","last_active":"2024-09-21T07:45:36.696649+0000","last_peered":"2024-09-21T07:45:36.696649+0000","last_clean":"2024-09-21T07:45:36.696649+0000","last_became_active":"2024-09-21T07:45:36.692616+0000","last_became_peered":"2024-09-21T07:45:36.692616+0000","last_unstale":"2024-09-21T07:45:36.696649+0000","last_undegraded":"2024-09-21T07:45:36.696649+0000","last_fullsized":"2024-09-21T07:45:36.696649+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:12:40.958669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":186,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.680482+0000","last_change":"2024-09-21T07:35:27.877503+0000","last_active":"2024-09-21T07:45:36.680482+0000","last_peered":"2024-09-21T07:45:36.680482+0000","last_clean":"2024-09-21T07:45:36.680482+0000","last_became_active":"2024-09-21T07:35:27.875775+0000","last_became_peered":"2024-09-21T07:35:27.875775+0000","last_unstale":"2024-09-21T07:45:36.680482+0000","last_undegraded":"2024-09-21T07:45:36.680482+0000","last_fullsized":"2024-09-21T07:45:36.680482+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:49.291023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.678647+0000","last_change":"2024-09-21T07:35:29.491412+0000","last_active":"2024-09-21T07:45:36.678647+0000","last_peered":"2024-09-21T07:45:36.678647+0000","last_clean":"2024-09-21T07:45:36.678647+0000","last_became_active":"2024-09-21T07:35:26.876404+0000","last_became_peered":"2024-09-21T07:35:26.876404+0000","last_unstale":"2024-09-21T07:45:36.678647+0000","last_undegraded":"2024-09-21T07:45:36.678647+0000","last_fullsized":"2024-09-21T07:45:36.678647+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063918000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":252,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689996+0000","last_change":"2024-09-21T07:34:24.259026+0000","last_active":"2024-09-21T07:45:38.689996+0000","last_peered":"2024-09-21T07:45:38.689996+0000","last_clean":"2024-09-21T07:45:38.689996+0000","last_became_active":"2024-09-21T07:34:23.635456+0000","last_became_peered":"2024-09-21T07:34:23.635456+0000","last_unstale":"2024-09-21T07:45:38.689996+0000","last_undegraded":"2024-09-21T07:45:38.689996+0000","last_fullsized":"2024-09-21T07:45:38.689996+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:09.592752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018654419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690345+0000","last_change":"2024-09-21T07:45:37.700588+0000","last_active":"2024-09-21T07:45:38.690345+0000","last_peered":"2024-09-21T07:45:38.690345+0000","last_clean":"2024-09-21T07:45:38.690345+0000","last_became_active":"2024-09-21T07:45:37.700288+0000","last_became_peered":"2024-09-21T07:45:37.700288+0000","last_unstale":"2024-09-21T07:45:38.690345+0000","last_undegraded":"2024-09-21T07:45:38.690345+0000","last_fullsized":"2024-09-21T07:45:38.690345+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:36:25.586374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.696399+0000","last_change":"2024-09-21T07:45:36.695336+0000","last_active":"2024-09-21T07:45:36.696399+0000","last_peered":"2024-09-21T07:45:36.696399+0000","last_clean":"2024-09-21T07:45:36.696399+0000","last_became_active":"2024-09-21T07:45:36.694952+0000","last_became_peered":"2024-09-21T07:45:36.694952+0000","last_unstale":"2024-09-21T07:45:36.696399+0000","last_undegraded":"2024-09-21T07:45:36.696399+0000","last_fullsized":"2024-09-21T07:45:36.696399+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:52.278657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":186,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.680709+0000","last_change":"2024-09-21T07:35:27.871984+0000","last_active":"2024-09-21T07:45:36.680709+0000","last_peered":"2024-09-21T07:45:36.680709+0000","last_clean":"2024-09-21T07:45:36.680709+0000","last_became_active":"2024-09-21T07:35:27.870336+0000","last_became_peered":"2024-09-21T07:35:27.870336+0000","last_unstale":"2024-09-21T07:45:36.680709+0000","last_undegraded":"2024-09-21T07:45:36.680709+0000","last_fullsized":"2024-09-21T07:45:36.680709+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:57.120479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.693795+0000","last_change":"2024-09-21T07:35:28.860351+0000","last_active":"2024-09-21T07:45:36.693795+0000","last_peered":"2024-09-21T07:45:36.693795+0000","last_clean":"2024-09-21T07:45:36.693795+0000","last_became_active":"2024-09-21T07:35:26.872046+0000","last_became_peered":"2024-09-21T07:35:26.872046+0000","last_unstale":"2024-09-21T07:45:36.693795+0000","last_undegraded":"2024-09-21T07:45:36.693795+0000","last_fullsized":"2024-09-21T07:45:36.693795+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:47.074327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098384399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":247,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.679336+0000","last_change":"2024-09-21T07:34:24.260300+0000","last_active":"2024-09-21T07:45:36.679336+0000","last_peered":"2024-09-21T07:45:36.679336+0000","last_clean":"2024-09-21T07:45:36.679336+0000","last_became_active":"2024-09-21T07:34:23.270128+0000","last_became_peered":"2024-09-21T07:34:23.270128+0000","last_unstale":"2024-09-21T07:45:36.679336+0000","last_undegraded":"2024-09-21T07:45:36.679336+0000","last_fullsized":"2024-09-21T07:45:36.679336+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:25.260931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00176651,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.664838+0000","last_change":"2024-09-21T07:45:36.664838+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.664838+0000","last_undegraded":"2024-09-21T07:45:36.664838+0000","last_fullsized":"2024-09-21T07:45:36.664838+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.685247+0000","last_change":"2024-09-21T07:45:36.683495+0000","last_active":"2024-09-21T07:45:36.685247+0000","last_peered":"2024-09-21T07:45:36.685247+0000","last_clean":"2024-09-21T07:45:36.685247+0000","last_became_active":"2024-09-21T07:45:36.683165+0000","last_became_peered":"2024-09-21T07:45:36.683165+0000","last_unstale":"2024-09-21T07:45:36.685247+0000","last_undegraded":"2024-09-21T07:45:36.685247+0000","last_fullsized":"2024-09-21T07:45:36.685247+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:53:20.269457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":186,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.680954+0000","last_change":"2024-09-21T07:35:27.877861+0000","last_active":"2024-09-21T07:45:36.680954+0000","last_peered":"2024-09-21T07:45:36.680954+0000","last_clean":"2024-09-21T07:45:36.680954+0000","last_became_active":"2024-09-21T07:35:27.877434+0000","last_became_peered":"2024-09-21T07:35:27.877434+0000","last_unstale":"2024-09-21T07:45:36.680954+0000","last_undegraded":"2024-09-21T07:45:36.680954+0000","last_fullsized":"2024-09-21T07:45:36.680954+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.672375+0000","last_change":"2024-09-21T07:35:28.862954+0000","last_active":"2024-09-21T07:45:36.672375+0000","last_peered":"2024-09-21T07:45:36.672375+0000","last_clean":"2024-09-21T07:45:36.672375+0000","last_became_active":"2024-09-21T07:35:26.870613+0000","last_became_peered":"2024-09-21T07:35:26.870613+0000","last_unstale":"2024-09-21T07:45:36.672375+0000","last_undegraded":"2024-09-21T07:45:36.672375+0000","last_fullsized":"2024-09-21T07:45:36.672375+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:06.901043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059842100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":246,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.673328+0000","last_change":"2024-09-21T07:34:25.028904+0000","last_active":"2024-09-21T07:45:36.673328+0000","last_peered":"2024-09-21T07:45:36.673328+0000","last_clean":"2024-09-21T07:45:36.673328+0000","last_became_active":"2024-09-21T07:34:23.260942+0000","last_became_peered":"2024-09-21T07:34:23.260942+0000","last_unstale":"2024-09-21T07:45:36.673328+0000","last_undegraded":"2024-09-21T07:45:36.673328+0000","last_fullsized":"2024-09-21T07:45:36.673328+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:22.709328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060553899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509594,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37824,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334016,"statfs":{"total":96636764160,"available":96598032384,"internally_reserved":0,"allocated":2531328,"data_stored":1604054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509596,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37244,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334596,"statfs":{"total":96636764160,"available":96598626304,"internally_reserved":0,"allocated":1937408,"data_stored":1013686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509594,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37240,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334600,"statfs":{"total":96636764160,"available":96598630400,"internally_reserved":0,"allocated":1937408,"data_stored":1013686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509595,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37244,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334596,"statfs":{"total":96636764160,"available":96598626304,"internally_reserved":0,"allocated":1937408,"data_stored":1013686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509596,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37240,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334600,"statfs":{"total":96636764160,"available":96598630400,"internally_reserved":0,"allocated":1937408,"data_stored":1013686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509596,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37820,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334020,"statfs":{"total":96636764160,"available":96598036480,"internally_reserved":0,"allocated":2531328,"data_stored":1604054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542300,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37272,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334568,"statfs":{"total":96636764160,"available":96598597632,"internally_reserved":0,"allocated":1961984,"data_stored":1029031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542299,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41448,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330392,"statfs":{"total":96636764160,"available":96594321408,"internally_reserved":0,"allocated":1978368,"data_stored":1030713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:45:42.688 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:45:42.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.947+0000 7f7076a00640 1 -- 172.21.15.39:0/768533590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f70700cb8a0 msgr2=0x7f7070156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:42.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.947+0000 7f7076a00640 1 --2- 172.21.15.39:0/768533590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f70700cb8a0 0x7f7070156bb0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f7058009830 tx=0x7f705802a480 comp rx=0 tx=0).stop 2024-09-21T07:45:42.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.947+0000 7f7076a00640 1 -- 172.21.15.39:0/768533590 shutdown_connections 2024-09-21T07:45:42.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.947+0000 7f7076a00640 1 --2- 172.21.15.39:0/768533590 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f70700ccc60 0x7f70700cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.947+0000 7f7076a00640 1 --2- 172.21.15.39:0/768533590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f70700cb8a0 0x7f7070156bb0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.947+0000 7f7076a00640 1 -- 172.21.15.39:0/768533590 >> 172.21.15.39:0/768533590 conn(0x7f70700b2460 msgr2=0x7f707006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:42.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.947+0000 7f7076a00640 1 -- 172.21.15.39:0/768533590 shutdown_connections 2024-09-21T07:45:42.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.947+0000 7f7076a00640 1 -- 172.21.15.39:0/768533590 wait complete. 2024-09-21T07:45:42.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.949+0000 7f7076a00640 1 Processor -- start 2024-09-21T07:45:42.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.950+0000 7f7076a00640 1 -- start start 2024-09-21T07:45:42.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.951+0000 7f7076a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f70700b30a0 0x7f707015f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:42.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.951+0000 7f7076a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f70700cb8a0 0x7f70701f5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:42.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.951+0000 7f7076a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70700ccc60 0x7f707015e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:42.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.951+0000 7f7076a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f70700af410 con 0x7f70700b30a0 2024-09-21T07:45:42.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.951+0000 7f7076a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f70700af290 con 0x7f70700ccc60 2024-09-21T07:45:42.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.951+0000 7f7076a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f70700af590 con 0x7f70700cb8a0 2024-09-21T07:45:42.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.951+0000 7f706fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f70700b30a0 0x7f707015f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:42.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.951+0000 7f706fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f70700b30a0 0x7f707015f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53856/0 (socket says 172.21.15.39:53856) 2024-09-21T07:45:42.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.951+0000 7f706fe00640 1 -- 172.21.15.39:0/3436872158 learned_addr learned my addr 172.21.15.39:0/3436872158 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:42.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.951+0000 7f7067e00640 1 --2- 172.21.15.39:0/3436872158 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f70700cb8a0 0x7f70701f5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:42.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.952+0000 7f7074a00640 1 --2- 172.21.15.39:0/3436872158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70700ccc60 0x7f707015e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:42.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.952+0000 7f706fe00640 1 -- 172.21.15.39:0/3436872158 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f70700cb8a0 msgr2=0x7f70701f5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:42.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.952+0000 7f706fe00640 1 --2- 172.21.15.39:0/3436872158 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f70700cb8a0 0x7f70701f5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.952+0000 7f706fe00640 1 -- 172.21.15.39:0/3436872158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70700ccc60 msgr2=0x7f707015e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:42.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.952+0000 7f706fe00640 1 --2- 172.21.15.39:0/3436872158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70700ccc60 0x7f707015e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.952+0000 7f706fe00640 1 -- 172.21.15.39:0/3436872158 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70700c5470 con 0x7f70700b30a0 2024-09-21T07:45:42.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.952+0000 7f706fe00640 1 --2- 172.21.15.39:0/3436872158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f70700b30a0 0x7f707015f5e0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f706000c3a0 tx=0x7f706000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:42.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.952+0000 7f706cc00640 1 -- 172.21.15.39:0/3436872158 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7060011790 con 0x7f70700b30a0 2024-09-21T07:45:42.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.953+0000 7f706cc00640 1 -- 172.21.15.39:0/3436872158 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7060017070 con 0x7f70700b30a0 2024-09-21T07:45:42.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.953+0000 7f706cc00640 1 -- 172.21.15.39:0/3436872158 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7060011cc0 con 0x7f70700b30a0 2024-09-21T07:45:42.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.953+0000 7f7076a00640 1 -- 172.21.15.39:0/3436872158 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70700c6510 con 0x7f70700b30a0 2024-09-21T07:45:42.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.953+0000 7f7076a00640 1 -- 172.21.15.39:0/3436872158 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70700c5a30 con 0x7f70700b30a0 2024-09-21T07:45:42.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.954+0000 7f7076a00640 1 -- 172.21.15.39:0/3436872158 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70700c6140 con 0x7f70700b30a0 2024-09-21T07:45:42.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.955+0000 7f706cc00640 1 -- 172.21.15.39:0/3436872158 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f706001c460 con 0x7f70700b30a0 2024-09-21T07:45:42.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.956+0000 7f706cc00640 1 --2- 172.21.15.39:0/3436872158 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f704c078b90 0x7f704c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:42.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.957+0000 7f706cc00640 1 -- 172.21.15.39:0/3436872158 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 6844+0+0 (secure 0 0 0) 0x7f706009aef0 con 0x7f70700b30a0 2024-09-21T07:45:42.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.957+0000 7f7067e00640 1 --2- 172.21.15.39:0/3436872158 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f704c078b90 0x7f704c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:42.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.958+0000 7f7067e00640 1 --2- 172.21.15.39:0/3436872158 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f704c078b90 0x7f704c07b0a0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f7058034000 tx=0x7f7058004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:42.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:42.961+0000 7f706cc00640 1 -- 172.21.15.39:0/3436872158 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f70700cd5b0 con 0x7f70700b30a0 2024-09-21T07:45:43.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.153+0000 7f7076a00640 1 -- 172.21.15.39:0/3436872158 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f70700afae0 con 0x7f704c078b90 2024-09-21T07:45:43.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.163+0000 7f706cc00640 1 -- 172.21.15.39:0/3436872158 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+490976 (secure 0 0 0) 0x7f70700afae0 con 0x7f704c078b90 2024-09-21T07:45:43.170 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:45:43.173 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:45:43.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.169+0000 7f7076a00640 1 -- 172.21.15.39:0/3436872158 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f704c078b90 msgr2=0x7f704c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:43.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.169+0000 7f7076a00640 1 --2- 172.21.15.39:0/3436872158 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f704c078b90 0x7f704c07b0a0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f7058034000 tx=0x7f7058004730 comp rx=0 tx=0).stop 2024-09-21T07:45:43.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.169+0000 7f7076a00640 1 -- 172.21.15.39:0/3436872158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f70700b30a0 msgr2=0x7f707015f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:43.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.169+0000 7f7076a00640 1 --2- 172.21.15.39:0/3436872158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f70700b30a0 0x7f707015f5e0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f706000c3a0 tx=0x7f706000c860 comp rx=0 tx=0).stop 2024-09-21T07:45:43.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.169+0000 7f7076a00640 1 -- 172.21.15.39:0/3436872158 shutdown_connections 2024-09-21T07:45:43.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.169+0000 7f7076a00640 1 --2- 172.21.15.39:0/3436872158 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f704c078b90 0x7f704c07b0a0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:43.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.169+0000 7f7076a00640 1 --2- 172.21.15.39:0/3436872158 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70700ccc60 0x7f707015e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:43.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.169+0000 7f7076a00640 1 --2- 172.21.15.39:0/3436872158 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f70700cb8a0 0x7f70701f5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:43.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.169+0000 7f7076a00640 1 --2- 172.21.15.39:0/3436872158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f70700b30a0 0x7f707015f5e0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:43.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.169+0000 7f7076a00640 1 -- 172.21.15.39:0/3436872158 >> 172.21.15.39:0/3436872158 conn(0x7f70700b2460 msgr2=0x7f70700b34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:43.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.169+0000 7f7076a00640 1 -- 172.21.15.39:0/3436872158 shutdown_connections 2024-09-21T07:45:43.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.169+0000 7f7076a00640 1 -- 172.21.15.39:0/3436872158 wait complete. 2024-09-21T07:45:43.193 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":532,"stamp":"2024-09-21T07:45:41.786929+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":285,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":303332,"kb_used_data":16360,"kb_used_omap":12,"kb_used_meta":286771,"kb_avail":754671388,"statfs":{"total":773094113280,"available":772783501312,"internally_reserved":0,"allocated":16752640,"data_stored":9322596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":293654064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001700"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.689192+0000","last_change":"2024-09-21T07:45:36.689192+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.689192+0000","last_undegraded":"2024-09-21T07:45:36.689192+0000","last_fullsized":"2024-09-21T07:45:36.689192+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.665560+0000","last_change":"2024-09-21T07:45:36.665560+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.665560+0000","last_undegraded":"2024-09-21T07:45:36.665560+0000","last_fullsized":"2024-09-21T07:45:36.665560+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.675392+0000","last_change":"2024-09-21T07:45:36.675392+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.675392+0000","last_undegraded":"2024-09-21T07:45:36.675392+0000","last_fullsized":"2024-09-21T07:45:36.675392+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.664845+0000","last_change":"2024-09-21T07:45:36.664845+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.664845+0000","last_undegraded":"2024-09-21T07:45:36.664845+0000","last_fullsized":"2024-09-21T07:45:36.664845+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689578+0000","last_change":"2024-09-21T07:45:37.706582+0000","last_active":"2024-09-21T07:45:38.689578+0000","last_peered":"2024-09-21T07:45:38.689578+0000","last_clean":"2024-09-21T07:45:38.689578+0000","last_became_active":"2024-09-21T07:45:37.706243+0000","last_became_peered":"2024-09-21T07:45:37.706243+0000","last_unstale":"2024-09-21T07:45:38.689578+0000","last_undegraded":"2024-09-21T07:45:38.689578+0000","last_fullsized":"2024-09-21T07:45:38.689578+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:20:35.055835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.676932+0000","last_change":"2024-09-21T07:45:36.676932+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.676932+0000","last_undegraded":"2024-09-21T07:45:36.676932+0000","last_fullsized":"2024-09-21T07:45:36.676932+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.674875+0000","last_change":"2024-09-21T07:45:36.674875+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.674875+0000","last_undegraded":"2024-09-21T07:45:36.674875+0000","last_fullsized":"2024-09-21T07:45:36.674875+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.687461+0000","last_change":"2024-09-21T07:45:37.694980+0000","last_active":"2024-09-21T07:45:38.687461+0000","last_peered":"2024-09-21T07:45:38.687461+0000","last_clean":"2024-09-21T07:45:38.687461+0000","last_became_active":"2024-09-21T07:45:37.689627+0000","last_became_peered":"2024-09-21T07:45:37.689627+0000","last_unstale":"2024-09-21T07:45:38.687461+0000","last_undegraded":"2024-09-21T07:45:38.687461+0000","last_fullsized":"2024-09-21T07:45:38.687461+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:32:36.418482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.677810+0000","last_change":"2024-09-21T07:45:36.677810+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.677810+0000","last_undegraded":"2024-09-21T07:45:36.677810+0000","last_fullsized":"2024-09-21T07:45:36.677810+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.680253+0000","last_change":"2024-09-21T07:45:36.680253+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.680253+0000","last_undegraded":"2024-09-21T07:45:36.680253+0000","last_fullsized":"2024-09-21T07:45:36.680253+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.664838+0000","last_change":"2024-09-21T07:45:36.664838+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.664838+0000","last_undegraded":"2024-09-21T07:45:36.664838+0000","last_fullsized":"2024-09-21T07:45:36.664838+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.688646+0000","last_change":"2024-09-21T07:45:36.688646+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.688646+0000","last_undegraded":"2024-09-21T07:45:36.688646+0000","last_fullsized":"2024-09-21T07:45:36.688646+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.676713+0000","last_change":"2024-09-21T07:45:36.676713+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.676713+0000","last_undegraded":"2024-09-21T07:45:36.676713+0000","last_fullsized":"2024-09-21T07:45:36.676713+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.687739+0000","last_change":"2024-09-21T07:45:36.687739+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.687739+0000","last_undegraded":"2024-09-21T07:45:36.687739+0000","last_fullsized":"2024-09-21T07:45:36.687739+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.684481+0000","last_change":"2024-09-21T07:45:36.684481+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.684481+0000","last_undegraded":"2024-09-21T07:45:36.684481+0000","last_fullsized":"2024-09-21T07:45:36.684481+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.677447+0000","last_change":"2024-09-21T07:45:36.677447+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.677447+0000","last_undegraded":"2024-09-21T07:45:36.677447+0000","last_fullsized":"2024-09-21T07:45:36.677447+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.677254+0000","last_change":"2024-09-21T07:45:36.677254+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.677254+0000","last_undegraded":"2024-09-21T07:45:36.677254+0000","last_fullsized":"2024-09-21T07:45:36.677254+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689982+0000","last_change":"2024-09-21T07:45:37.704608+0000","last_active":"2024-09-21T07:45:38.689982+0000","last_peered":"2024-09-21T07:45:38.689982+0000","last_clean":"2024-09-21T07:45:38.689982+0000","last_became_active":"2024-09-21T07:45:37.703051+0000","last_became_peered":"2024-09-21T07:45:37.703051+0000","last_unstale":"2024-09-21T07:45:38.689982+0000","last_undegraded":"2024-09-21T07:45:38.689982+0000","last_fullsized":"2024-09-21T07:45:38.689982+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:33.889448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.675086+0000","last_change":"2024-09-21T07:45:36.675086+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.675086+0000","last_undegraded":"2024-09-21T07:45:36.675086+0000","last_fullsized":"2024-09-21T07:45:36.675086+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689772+0000","last_change":"2024-09-21T07:45:37.698151+0000","last_active":"2024-09-21T07:45:38.689772+0000","last_peered":"2024-09-21T07:45:38.689772+0000","last_clean":"2024-09-21T07:45:38.689772+0000","last_became_active":"2024-09-21T07:45:37.697818+0000","last_became_peered":"2024-09-21T07:45:37.697818+0000","last_unstale":"2024-09-21T07:45:38.689772+0000","last_undegraded":"2024-09-21T07:45:38.689772+0000","last_fullsized":"2024-09-21T07:45:38.689772+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:57.867727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.664937+0000","last_change":"2024-09-21T07:45:36.664937+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.664937+0000","last_undegraded":"2024-09-21T07:45:36.664937+0000","last_fullsized":"2024-09-21T07:45:36.664937+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.691525+0000","last_change":"2024-09-21T07:45:36.691525+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.691525+0000","last_undegraded":"2024-09-21T07:45:36.691525+0000","last_fullsized":"2024-09-21T07:45:36.691525+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.666008+0000","last_change":"2024-09-21T07:45:36.666008+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.666008+0000","last_undegraded":"2024-09-21T07:45:36.666008+0000","last_fullsized":"2024-09-21T07:45:36.666008+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.676435+0000","last_change":"2024-09-21T07:45:36.676435+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.676435+0000","last_undegraded":"2024-09-21T07:45:36.676435+0000","last_fullsized":"2024-09-21T07:45:36.676435+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.675973+0000","last_change":"2024-09-21T07:45:36.675973+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.675973+0000","last_undegraded":"2024-09-21T07:45:36.675973+0000","last_fullsized":"2024-09-21T07:45:36.675973+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.671245+0000","last_change":"2024-09-21T07:45:36.671245+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.671245+0000","last_undegraded":"2024-09-21T07:45:36.671245+0000","last_fullsized":"2024-09-21T07:45:36.671245+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.671879+0000","last_change":"2024-09-21T07:45:36.671879+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.671879+0000","last_undegraded":"2024-09-21T07:45:36.671879+0000","last_fullsized":"2024-09-21T07:45:36.671879+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.681606+0000","last_change":"2024-09-21T07:45:36.681606+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.681606+0000","last_undegraded":"2024-09-21T07:45:36.681606+0000","last_fullsized":"2024-09-21T07:45:36.681606+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.676579+0000","last_change":"2024-09-21T07:45:36.676579+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.676579+0000","last_undegraded":"2024-09-21T07:45:36.676579+0000","last_fullsized":"2024-09-21T07:45:36.676579+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.675479+0000","last_change":"2024-09-21T07:45:36.675479+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.675479+0000","last_undegraded":"2024-09-21T07:45:36.675479+0000","last_fullsized":"2024-09-21T07:45:36.675479+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.687575+0000","last_change":"2024-09-21T07:45:37.685098+0000","last_active":"2024-09-21T07:45:38.687575+0000","last_peered":"2024-09-21T07:45:38.687575+0000","last_clean":"2024-09-21T07:45:38.687575+0000","last_became_active":"2024-09-21T07:45:37.684748+0000","last_became_peered":"2024-09-21T07:45:37.684748+0000","last_unstale":"2024-09-21T07:45:38.687575+0000","last_undegraded":"2024-09-21T07:45:38.687575+0000","last_fullsized":"2024-09-21T07:45:38.687575+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:51:22.048251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.681462+0000","last_change":"2024-09-21T07:45:36.681462+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.681462+0000","last_undegraded":"2024-09-21T07:45:36.681462+0000","last_fullsized":"2024-09-21T07:45:36.681462+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.671410+0000","last_change":"2024-09-21T07:45:36.671410+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.671410+0000","last_undegraded":"2024-09-21T07:45:36.671410+0000","last_fullsized":"2024-09-21T07:45:36.671410+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.699039+0000","last_change":"2024-09-21T07:45:36.697679+0000","last_active":"2024-09-21T07:45:36.699039+0000","last_peered":"2024-09-21T07:45:36.699039+0000","last_clean":"2024-09-21T07:45:36.699039+0000","last_became_active":"2024-09-21T07:45:36.697396+0000","last_became_peered":"2024-09-21T07:45:36.697396+0000","last_unstale":"2024-09-21T07:45:36.699039+0000","last_undegraded":"2024-09-21T07:45:36.699039+0000","last_fullsized":"2024-09-21T07:45:36.699039+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:43:37.693480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":187,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.679060+0000","last_change":"2024-09-21T07:35:27.879605+0000","last_active":"2024-09-21T07:45:36.679060+0000","last_peered":"2024-09-21T07:45:36.679060+0000","last_clean":"2024-09-21T07:45:36.679060+0000","last_became_active":"2024-09-21T07:35:27.879274+0000","last_became_peered":"2024-09-21T07:35:27.879274+0000","last_unstale":"2024-09-21T07:45:36.679060+0000","last_undegraded":"2024-09-21T07:45:36.679060+0000","last_fullsized":"2024-09-21T07:45:36.679060+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:24.241636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":197,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689112+0000","last_change":"2024-09-21T07:35:29.491336+0000","last_active":"2024-09-21T07:45:38.689112+0000","last_peered":"2024-09-21T07:45:38.689112+0000","last_clean":"2024-09-21T07:45:38.689112+0000","last_became_active":"2024-09-21T07:35:26.866484+0000","last_became_peered":"2024-09-21T07:35:26.866484+0000","last_unstale":"2024-09-21T07:45:38.689112+0000","last_undegraded":"2024-09-21T07:45:38.689112+0000","last_fullsized":"2024-09-21T07:45:38.689112+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:10.449665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076010100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":373,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.679314+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:45:36.679314+0000","last_peered":"2024-09-21T07:45:36.679314+0000","last_clean":"2024-09-21T07:45:36.679314+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:45:36.679314+0000","last_undegraded":"2024-09-21T07:45:36.679314+0000","last_fullsized":"2024-09-21T07:45:36.679314+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":252,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.670156+0000","last_change":"2024-09-21T07:34:24.539248+0000","last_active":"2024-09-21T07:45:36.670156+0000","last_peered":"2024-09-21T07:45:36.670156+0000","last_clean":"2024-09-21T07:45:36.670156+0000","last_became_active":"2024-09-21T07:34:23.273465+0000","last_became_peered":"2024-09-21T07:34:23.273465+0000","last_unstale":"2024-09-21T07:45:36.670156+0000","last_undegraded":"2024-09-21T07:45:36.670156+0000","last_fullsized":"2024-09-21T07:45:36.670156+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:55.517452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060627700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.686033+0000","last_change":"2024-09-21T07:45:36.686033+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.686033+0000","last_undegraded":"2024-09-21T07:45:36.686033+0000","last_fullsized":"2024-09-21T07:45:36.686033+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.701635+0000","last_change":"2024-09-21T07:45:36.700703+0000","last_active":"2024-09-21T07:45:36.701635+0000","last_peered":"2024-09-21T07:45:36.701635+0000","last_clean":"2024-09-21T07:45:36.701635+0000","last_became_active":"2024-09-21T07:45:36.700344+0000","last_became_peered":"2024-09-21T07:45:36.700344+0000","last_unstale":"2024-09-21T07:45:36.701635+0000","last_undegraded":"2024-09-21T07:45:36.701635+0000","last_fullsized":"2024-09-21T07:45:36.701635+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:59.539139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689081+0000","last_change":"2024-09-21T07:35:27.864492+0000","last_active":"2024-09-21T07:45:38.689081+0000","last_peered":"2024-09-21T07:45:38.689081+0000","last_clean":"2024-09-21T07:45:38.689081+0000","last_became_active":"2024-09-21T07:35:27.864176+0000","last_became_peered":"2024-09-21T07:35:27.864176+0000","last_unstale":"2024-09-21T07:45:38.689081+0000","last_undegraded":"2024-09-21T07:45:38.689081+0000","last_fullsized":"2024-09-21T07:45:38.689081+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:05.153467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.680405+0000","last_change":"2024-09-21T07:35:29.493678+0000","last_active":"2024-09-21T07:45:36.680405+0000","last_peered":"2024-09-21T07:45:36.680405+0000","last_clean":"2024-09-21T07:45:36.680405+0000","last_became_active":"2024-09-21T07:35:26.870749+0000","last_became_peered":"2024-09-21T07:35:26.870749+0000","last_unstale":"2024-09-21T07:45:36.680405+0000","last_undegraded":"2024-09-21T07:45:36.680405+0000","last_fullsized":"2024-09-21T07:45:36.680405+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:55.367877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010159570000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":246,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688084+0000","last_change":"2024-09-21T07:34:24.597274+0000","last_active":"2024-09-21T07:45:38.688084+0000","last_peered":"2024-09-21T07:45:38.688084+0000","last_clean":"2024-09-21T07:45:38.688084+0000","last_became_active":"2024-09-21T07:34:23.279560+0000","last_became_peered":"2024-09-21T07:34:23.279560+0000","last_unstale":"2024-09-21T07:45:38.688084+0000","last_undegraded":"2024-09-21T07:45:38.688084+0000","last_fullsized":"2024-09-21T07:45:38.688084+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:15.845265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059699799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.676108+0000","last_change":"2024-09-21T07:45:36.676108+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.676108+0000","last_undegraded":"2024-09-21T07:45:36.676108+0000","last_fullsized":"2024-09-21T07:45:36.676108+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.698291+0000","last_change":"2024-09-21T07:45:36.695689+0000","last_active":"2024-09-21T07:45:36.698291+0000","last_peered":"2024-09-21T07:45:36.698291+0000","last_clean":"2024-09-21T07:45:36.698291+0000","last_became_active":"2024-09-21T07:45:36.695386+0000","last_became_peered":"2024-09-21T07:45:36.695386+0000","last_unstale":"2024-09-21T07:45:36.698291+0000","last_undegraded":"2024-09-21T07:45:36.698291+0000","last_fullsized":"2024-09-21T07:45:36.698291+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:13.324095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":187,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.693357+0000","last_change":"2024-09-21T07:35:27.874387+0000","last_active":"2024-09-21T07:45:36.693357+0000","last_peered":"2024-09-21T07:45:36.693357+0000","last_clean":"2024-09-21T07:45:36.693357+0000","last_became_active":"2024-09-21T07:35:27.873277+0000","last_became_peered":"2024-09-21T07:35:27.873277+0000","last_unstale":"2024-09-21T07:45:36.693357+0000","last_undegraded":"2024-09-21T07:45:36.693357+0000","last_fullsized":"2024-09-21T07:45:36.693357+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:04.581463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":196,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688673+0000","last_change":"2024-09-21T07:35:29.387094+0000","last_active":"2024-09-21T07:45:38.688673+0000","last_peered":"2024-09-21T07:45:38.688673+0000","last_clean":"2024-09-21T07:45:38.688673+0000","last_became_active":"2024-09-21T07:35:26.874757+0000","last_became_peered":"2024-09-21T07:35:26.874757+0000","last_unstale":"2024-09-21T07:45:38.688673+0000","last_undegraded":"2024-09-21T07:45:38.688673+0000","last_fullsized":"2024-09-21T07:45:38.688673+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:33.542750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061065700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":248,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.678663+0000","last_change":"2024-09-21T07:34:24.259906+0000","last_active":"2024-09-21T07:45:36.678663+0000","last_peered":"2024-09-21T07:45:36.678663+0000","last_clean":"2024-09-21T07:45:36.678663+0000","last_became_active":"2024-09-21T07:34:23.635282+0000","last_became_peered":"2024-09-21T07:34:23.635282+0000","last_unstale":"2024-09-21T07:45:36.678663+0000","last_undegraded":"2024-09-21T07:45:36.678663+0000","last_fullsized":"2024-09-21T07:45:36.678663+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:00.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001527664,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.670461+0000","last_change":"2024-09-21T07:45:36.670461+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.670461+0000","last_undegraded":"2024-09-21T07:45:36.670461+0000","last_fullsized":"2024-09-21T07:45:36.670461+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.689405+0000","last_change":"2024-09-21T07:45:36.688037+0000","last_active":"2024-09-21T07:45:36.689405+0000","last_peered":"2024-09-21T07:45:36.689405+0000","last_clean":"2024-09-21T07:45:36.689405+0000","last_became_active":"2024-09-21T07:45:36.687731+0000","last_became_peered":"2024-09-21T07:45:36.687731+0000","last_unstale":"2024-09-21T07:45:36.689405+0000","last_undegraded":"2024-09-21T07:45:36.689405+0000","last_fullsized":"2024-09-21T07:45:36.689405+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:31.034405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":187,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.696504+0000","last_change":"2024-09-21T07:35:27.873645+0000","last_active":"2024-09-21T07:45:36.696504+0000","last_peered":"2024-09-21T07:45:36.696504+0000","last_clean":"2024-09-21T07:45:36.696504+0000","last_became_active":"2024-09-21T07:35:27.873245+0000","last_became_peered":"2024-09-21T07:35:27.873245+0000","last_unstale":"2024-09-21T07:45:36.696504+0000","last_undegraded":"2024-09-21T07:45:36.696504+0000","last_fullsized":"2024-09-21T07:45:36.696504+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:55.090953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.698882+0000","last_change":"2024-09-21T07:35:28.861744+0000","last_active":"2024-09-21T07:45:36.698882+0000","last_peered":"2024-09-21T07:45:36.698882+0000","last_clean":"2024-09-21T07:45:36.698882+0000","last_became_active":"2024-09-21T07:35:26.876462+0000","last_became_peered":"2024-09-21T07:35:26.876462+0000","last_unstale":"2024-09-21T07:45:36.698882+0000","last_undegraded":"2024-09-21T07:45:36.698882+0000","last_fullsized":"2024-09-21T07:45:36.698882+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:48.632568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012020920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":245,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.672276+0000","last_change":"2024-09-21T07:34:25.085454+0000","last_active":"2024-09-21T07:45:36.672276+0000","last_peered":"2024-09-21T07:45:36.672276+0000","last_clean":"2024-09-21T07:45:36.672276+0000","last_became_active":"2024-09-21T07:34:23.279205+0000","last_became_peered":"2024-09-21T07:34:23.279205+0000","last_unstale":"2024-09-21T07:45:36.672276+0000","last_undegraded":"2024-09-21T07:45:36.672276+0000","last_fullsized":"2024-09-21T07:45:36.672276+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:54.231666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095320599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.675896+0000","last_change":"2024-09-21T07:45:36.675896+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.675896+0000","last_undegraded":"2024-09-21T07:45:36.675896+0000","last_fullsized":"2024-09-21T07:45:36.675896+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.701499+0000","last_change":"2024-09-21T07:45:36.700878+0000","last_active":"2024-09-21T07:45:36.701499+0000","last_peered":"2024-09-21T07:45:36.701499+0000","last_clean":"2024-09-21T07:45:36.701499+0000","last_became_active":"2024-09-21T07:45:36.700484+0000","last_became_peered":"2024-09-21T07:45:36.700484+0000","last_unstale":"2024-09-21T07:45:36.701499+0000","last_undegraded":"2024-09-21T07:45:36.701499+0000","last_fullsized":"2024-09-21T07:45:36.701499+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:52:12.375777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688964+0000","last_change":"2024-09-21T07:35:27.861373+0000","last_active":"2024-09-21T07:45:38.688964+0000","last_peered":"2024-09-21T07:45:38.688964+0000","last_clean":"2024-09-21T07:45:38.688964+0000","last_became_active":"2024-09-21T07:35:27.861016+0000","last_became_peered":"2024-09-21T07:35:27.861016+0000","last_unstale":"2024-09-21T07:45:38.688964+0000","last_undegraded":"2024-09-21T07:45:38.688964+0000","last_fullsized":"2024-09-21T07:45:38.688964+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:56.043763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":195,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.673133+0000","last_change":"2024-09-21T07:35:29.053677+0000","last_active":"2024-09-21T07:45:36.673133+0000","last_peered":"2024-09-21T07:45:36.673133+0000","last_clean":"2024-09-21T07:45:36.673133+0000","last_became_active":"2024-09-21T07:35:26.875496+0000","last_became_peered":"2024-09-21T07:35:26.875496+0000","last_unstale":"2024-09-21T07:45:36.673133+0000","last_undegraded":"2024-09-21T07:45:36.673133+0000","last_fullsized":"2024-09-21T07:45:36.673133+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:04.850784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065859000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":247,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.672104+0000","last_change":"2024-09-21T07:34:25.028521+0000","last_active":"2024-09-21T07:45:36.672104+0000","last_peered":"2024-09-21T07:45:36.672104+0000","last_clean":"2024-09-21T07:45:36.672104+0000","last_became_active":"2024-09-21T07:34:23.635467+0000","last_became_peered":"2024-09-21T07:34:23.635467+0000","last_unstale":"2024-09-21T07:45:36.672104+0000","last_undegraded":"2024-09-21T07:45:36.672104+0000","last_fullsized":"2024-09-21T07:45:36.672104+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:33.729123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000660558,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.676454+0000","last_change":"2024-09-21T07:45:36.676454+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.676454+0000","last_undegraded":"2024-09-21T07:45:36.676454+0000","last_fullsized":"2024-09-21T07:45:36.676454+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.699376+0000","last_change":"2024-09-21T07:45:36.697573+0000","last_active":"2024-09-21T07:45:36.699376+0000","last_peered":"2024-09-21T07:45:36.699376+0000","last_clean":"2024-09-21T07:45:36.699376+0000","last_became_active":"2024-09-21T07:45:36.697341+0000","last_became_peered":"2024-09-21T07:45:36.697341+0000","last_unstale":"2024-09-21T07:45:36.699376+0000","last_undegraded":"2024-09-21T07:45:36.699376+0000","last_fullsized":"2024-09-21T07:45:36.699376+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:18.613601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":187,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.676705+0000","last_change":"2024-09-21T07:35:27.871201+0000","last_active":"2024-09-21T07:45:36.676705+0000","last_peered":"2024-09-21T07:45:36.676705+0000","last_clean":"2024-09-21T07:45:36.676705+0000","last_became_active":"2024-09-21T07:35:27.870816+0000","last_became_peered":"2024-09-21T07:35:27.870816+0000","last_unstale":"2024-09-21T07:45:36.676705+0000","last_undegraded":"2024-09-21T07:45:36.676705+0000","last_fullsized":"2024-09-21T07:45:36.676705+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:04.787450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":197,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689594+0000","last_change":"2024-09-21T07:35:29.491874+0000","last_active":"2024-09-21T07:45:38.689594+0000","last_peered":"2024-09-21T07:45:38.689594+0000","last_clean":"2024-09-21T07:45:38.689594+0000","last_became_active":"2024-09-21T07:35:26.866381+0000","last_became_peered":"2024-09-21T07:35:26.866381+0000","last_unstale":"2024-09-21T07:45:38.689594+0000","last_undegraded":"2024-09-21T07:45:38.689594+0000","last_fullsized":"2024-09-21T07:45:38.689594+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:36.404647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073235399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":252,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690345+0000","last_change":"2024-09-21T07:34:24.260046+0000","last_active":"2024-09-21T07:45:38.690345+0000","last_peered":"2024-09-21T07:45:38.690345+0000","last_clean":"2024-09-21T07:45:38.690345+0000","last_became_active":"2024-09-21T07:34:23.273431+0000","last_became_peered":"2024-09-21T07:34:23.273431+0000","last_unstale":"2024-09-21T07:45:38.690345+0000","last_undegraded":"2024-09-21T07:45:38.690345+0000","last_fullsized":"2024-09-21T07:45:38.690345+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:18.876057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019803310000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.674849+0000","last_change":"2024-09-21T07:45:36.674849+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.674849+0000","last_undegraded":"2024-09-21T07:45:36.674849+0000","last_fullsized":"2024-09-21T07:45:36.674849+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.687046+0000","last_change":"2024-09-21T07:45:36.684575+0000","last_active":"2024-09-21T07:45:36.687046+0000","last_peered":"2024-09-21T07:45:36.687046+0000","last_clean":"2024-09-21T07:45:36.687046+0000","last_became_active":"2024-09-21T07:45:36.683843+0000","last_became_peered":"2024-09-21T07:45:36.683843+0000","last_unstale":"2024-09-21T07:45:36.687046+0000","last_undegraded":"2024-09-21T07:45:36.687046+0000","last_fullsized":"2024-09-21T07:45:36.687046+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:56.463705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":187,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.676204+0000","last_change":"2024-09-21T07:35:27.871736+0000","last_active":"2024-09-21T07:45:36.676204+0000","last_peered":"2024-09-21T07:45:36.676204+0000","last_clean":"2024-09-21T07:45:36.676204+0000","last_became_active":"2024-09-21T07:35:27.871232+0000","last_became_peered":"2024-09-21T07:35:27.871232+0000","last_unstale":"2024-09-21T07:45:36.676204+0000","last_undegraded":"2024-09-21T07:45:36.676204+0000","last_fullsized":"2024-09-21T07:45:36.676204+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:04.419301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":197,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688913+0000","last_change":"2024-09-21T07:35:29.491246+0000","last_active":"2024-09-21T07:45:38.688913+0000","last_peered":"2024-09-21T07:45:38.688913+0000","last_clean":"2024-09-21T07:45:38.688913+0000","last_became_active":"2024-09-21T07:35:26.866783+0000","last_became_peered":"2024-09-21T07:35:26.866783+0000","last_unstale":"2024-09-21T07:45:38.688913+0000","last_undegraded":"2024-09-21T07:45:38.688913+0000","last_fullsized":"2024-09-21T07:45:38.688913+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:28.881914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098474099999999992,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":247,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.673249+0000","last_change":"2024-09-21T07:34:25.028640+0000","last_active":"2024-09-21T07:45:36.673249+0000","last_peered":"2024-09-21T07:45:36.673249+0000","last_clean":"2024-09-21T07:45:36.673249+0000","last_became_active":"2024-09-21T07:34:23.260950+0000","last_became_peered":"2024-09-21T07:34:23.260950+0000","last_unstale":"2024-09-21T07:45:36.673249+0000","last_undegraded":"2024-09-21T07:45:36.673249+0000","last_fullsized":"2024-09-21T07:45:36.673249+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:23:42.121398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055135099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689869+0000","last_change":"2024-09-21T07:45:37.693385+0000","last_active":"2024-09-21T07:45:38.689869+0000","last_peered":"2024-09-21T07:45:38.689869+0000","last_clean":"2024-09-21T07:45:38.689869+0000","last_became_active":"2024-09-21T07:45:37.691927+0000","last_became_peered":"2024-09-21T07:45:37.691927+0000","last_unstale":"2024-09-21T07:45:38.689869+0000","last_undegraded":"2024-09-21T07:45:38.689869+0000","last_fullsized":"2024-09-21T07:45:38.689869+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:29.208442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.684490+0000","last_change":"2024-09-21T07:45:36.681956+0000","last_active":"2024-09-21T07:45:36.684490+0000","last_peered":"2024-09-21T07:45:36.684490+0000","last_clean":"2024-09-21T07:45:36.684490+0000","last_became_active":"2024-09-21T07:45:36.681656+0000","last_became_peered":"2024-09-21T07:45:36.681656+0000","last_unstale":"2024-09-21T07:45:36.684490+0000","last_undegraded":"2024-09-21T07:45:36.684490+0000","last_fullsized":"2024-09-21T07:45:36.684490+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:04:54.765401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":186,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.680978+0000","last_change":"2024-09-21T07:35:27.872027+0000","last_active":"2024-09-21T07:45:36.680978+0000","last_peered":"2024-09-21T07:45:36.680978+0000","last_clean":"2024-09-21T07:45:36.680978+0000","last_became_active":"2024-09-21T07:35:27.870426+0000","last_became_peered":"2024-09-21T07:35:27.870426+0000","last_unstale":"2024-09-21T07:45:36.680978+0000","last_undegraded":"2024-09-21T07:45:36.680978+0000","last_fullsized":"2024-09-21T07:45:36.680978+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":197,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689126+0000","last_change":"2024-09-21T07:35:29.491008+0000","last_active":"2024-09-21T07:45:38.689126+0000","last_peered":"2024-09-21T07:45:38.689126+0000","last_clean":"2024-09-21T07:45:38.689126+0000","last_became_active":"2024-09-21T07:35:26.866750+0000","last_became_peered":"2024-09-21T07:35:26.866750+0000","last_unstale":"2024-09-21T07:45:38.689126+0000","last_undegraded":"2024-09-21T07:45:38.689126+0000","last_fullsized":"2024-09-21T07:45:38.689126+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:25.490406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083529700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":248,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.679720+0000","last_change":"2024-09-21T07:34:24.260665+0000","last_active":"2024-09-21T07:45:36.679720+0000","last_peered":"2024-09-21T07:45:36.679720+0000","last_clean":"2024-09-21T07:45:36.679720+0000","last_became_active":"2024-09-21T07:34:23.635909+0000","last_became_peered":"2024-09-21T07:34:23.635909+0000","last_unstale":"2024-09-21T07:45:36.679720+0000","last_undegraded":"2024-09-21T07:45:36.679720+0000","last_fullsized":"2024-09-21T07:45:36.679720+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:44.536885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689417+0000","last_change":"2024-09-21T07:45:37.686269+0000","last_active":"2024-09-21T07:45:38.689417+0000","last_peered":"2024-09-21T07:45:38.689417+0000","last_clean":"2024-09-21T07:45:38.689417+0000","last_became_active":"2024-09-21T07:45:37.685971+0000","last_became_peered":"2024-09-21T07:45:37.685971+0000","last_unstale":"2024-09-21T07:45:38.689417+0000","last_undegraded":"2024-09-21T07:45:38.689417+0000","last_fullsized":"2024-09-21T07:45:38.689417+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:23.017715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.696537+0000","last_change":"2024-09-21T07:45:36.695413+0000","last_active":"2024-09-21T07:45:36.696537+0000","last_peered":"2024-09-21T07:45:36.696537+0000","last_clean":"2024-09-21T07:45:36.696537+0000","last_became_active":"2024-09-21T07:45:36.695045+0000","last_became_peered":"2024-09-21T07:45:36.695045+0000","last_unstale":"2024-09-21T07:45:36.696537+0000","last_undegraded":"2024-09-21T07:45:36.696537+0000","last_fullsized":"2024-09-21T07:45:36.696537+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:32.061491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689492+0000","last_change":"2024-09-21T07:35:27.874772+0000","last_active":"2024-09-21T07:45:38.689492+0000","last_peered":"2024-09-21T07:45:38.689492+0000","last_clean":"2024-09-21T07:45:38.689492+0000","last_became_active":"2024-09-21T07:35:27.874434+0000","last_became_peered":"2024-09-21T07:35:27.874434+0000","last_unstale":"2024-09-21T07:45:38.689492+0000","last_undegraded":"2024-09-21T07:45:38.689492+0000","last_fullsized":"2024-09-21T07:45:38.689492+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:31.998755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":199,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.669713+0000","last_change":"2024-09-21T07:35:28.859153+0000","last_active":"2024-09-21T07:45:36.669713+0000","last_peered":"2024-09-21T07:45:36.669713+0000","last_clean":"2024-09-21T07:45:36.669713+0000","last_became_active":"2024-09-21T07:35:26.874428+0000","last_became_peered":"2024-09-21T07:35:26.874428+0000","last_unstale":"2024-09-21T07:45:36.669713+0000","last_undegraded":"2024-09-21T07:45:36.669713+0000","last_fullsized":"2024-09-21T07:45:36.669713+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:04.487621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048557299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":252,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.669176+0000","last_change":"2024-09-21T07:34:24.538845+0000","last_active":"2024-09-21T07:45:36.669176+0000","last_peered":"2024-09-21T07:45:36.669176+0000","last_clean":"2024-09-21T07:45:36.669176+0000","last_became_active":"2024-09-21T07:34:23.636372+0000","last_became_peered":"2024-09-21T07:34:23.636372+0000","last_unstale":"2024-09-21T07:45:36.669176+0000","last_undegraded":"2024-09-21T07:45:36.669176+0000","last_fullsized":"2024-09-21T07:45:36.669176+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:29.393813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841451,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.683953+0000","last_change":"2024-09-21T07:45:36.683953+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.683953+0000","last_undegraded":"2024-09-21T07:45:36.683953+0000","last_fullsized":"2024-09-21T07:45:36.683953+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"156'1","reported_seq":16,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690230+0000","last_change":"2024-09-21T07:45:36.698933+0000","last_active":"2024-09-21T07:45:38.690230+0000","last_peered":"2024-09-21T07:45:38.690230+0000","last_clean":"2024-09-21T07:45:38.690230+0000","last_became_active":"2024-09-21T07:45:36.698651+0000","last_became_peered":"2024-09-21T07:45:36.698651+0000","last_unstale":"2024-09-21T07:45:38.690230+0000","last_undegraded":"2024-09-21T07:45:38.690230+0000","last_fullsized":"2024-09-21T07:45:38.690230+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:17:31.377323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":186,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.677861+0000","last_change":"2024-09-21T07:35:27.877801+0000","last_active":"2024-09-21T07:45:36.677861+0000","last_peered":"2024-09-21T07:45:36.677861+0000","last_clean":"2024-09-21T07:45:36.677861+0000","last_became_active":"2024-09-21T07:35:27.877464+0000","last_became_peered":"2024-09-21T07:35:27.877464+0000","last_unstale":"2024-09-21T07:45:36.677861+0000","last_undegraded":"2024-09-21T07:45:36.677861+0000","last_fullsized":"2024-09-21T07:45:36.677861+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:08.814929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.676325+0000","last_change":"2024-09-21T07:35:28.863732+0000","last_active":"2024-09-21T07:45:36.676325+0000","last_peered":"2024-09-21T07:45:36.676325+0000","last_clean":"2024-09-21T07:45:36.676325+0000","last_became_active":"2024-09-21T07:35:26.877392+0000","last_became_peered":"2024-09-21T07:35:26.877392+0000","last_unstale":"2024-09-21T07:45:36.676325+0000","last_undegraded":"2024-09-21T07:45:36.676325+0000","last_fullsized":"2024-09-21T07:45:36.676325+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071425700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":252,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689803+0000","last_change":"2024-09-21T07:34:24.259686+0000","last_active":"2024-09-21T07:45:38.689803+0000","last_peered":"2024-09-21T07:45:38.689803+0000","last_clean":"2024-09-21T07:45:38.689803+0000","last_became_active":"2024-09-21T07:34:23.274405+0000","last_became_peered":"2024-09-21T07:34:23.274405+0000","last_unstale":"2024-09-21T07:45:38.689803+0000","last_undegraded":"2024-09-21T07:45:38.689803+0000","last_fullsized":"2024-09-21T07:45:38.689803+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:26.304693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022009920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.664831+0000","last_change":"2024-09-21T07:45:36.664831+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.664831+0000","last_undegraded":"2024-09-21T07:45:36.664831+0000","last_fullsized":"2024-09-21T07:45:36.664831+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.692374+0000","last_change":"2024-09-21T07:45:36.689252+0000","last_active":"2024-09-21T07:45:36.692374+0000","last_peered":"2024-09-21T07:45:36.692374+0000","last_clean":"2024-09-21T07:45:36.692374+0000","last_became_active":"2024-09-21T07:45:36.688915+0000","last_became_peered":"2024-09-21T07:45:36.688915+0000","last_unstale":"2024-09-21T07:45:36.692374+0000","last_undegraded":"2024-09-21T07:45:36.692374+0000","last_fullsized":"2024-09-21T07:45:36.692374+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:41:51.910757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":189,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.687721+0000","last_change":"2024-09-21T07:35:27.867336+0000","last_active":"2024-09-21T07:45:38.687721+0000","last_peered":"2024-09-21T07:45:38.687721+0000","last_clean":"2024-09-21T07:45:38.687721+0000","last_became_active":"2024-09-21T07:35:27.867015+0000","last_became_peered":"2024-09-21T07:35:27.867015+0000","last_unstale":"2024-09-21T07:45:38.687721+0000","last_undegraded":"2024-09-21T07:45:38.687721+0000","last_fullsized":"2024-09-21T07:45:38.687721+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:59.214139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.679176+0000","last_change":"2024-09-21T07:35:29.493137+0000","last_active":"2024-09-21T07:45:36.679176+0000","last_peered":"2024-09-21T07:45:36.679176+0000","last_clean":"2024-09-21T07:45:36.679176+0000","last_became_active":"2024-09-21T07:35:26.872230+0000","last_became_peered":"2024-09-21T07:35:26.872230+0000","last_unstale":"2024-09-21T07:45:36.679176+0000","last_undegraded":"2024-09-21T07:45:36.679176+0000","last_fullsized":"2024-09-21T07:45:36.679176+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:24.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001197449,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":248,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.679175+0000","last_change":"2024-09-21T07:34:24.258754+0000","last_active":"2024-09-21T07:45:36.679175+0000","last_peered":"2024-09-21T07:45:36.679175+0000","last_clean":"2024-09-21T07:45:36.679175+0000","last_became_active":"2024-09-21T07:34:23.272657+0000","last_became_peered":"2024-09-21T07:34:23.272657+0000","last_unstale":"2024-09-21T07:45:36.679175+0000","last_undegraded":"2024-09-21T07:45:36.679175+0000","last_fullsized":"2024-09-21T07:45:36.679175+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:05.477099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067143200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.692417+0000","last_change":"2024-09-21T07:45:36.692417+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.692417+0000","last_undegraded":"2024-09-21T07:45:36.692417+0000","last_fullsized":"2024-09-21T07:45:36.692417+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.689641+0000","last_change":"2024-09-21T07:45:36.687920+0000","last_active":"2024-09-21T07:45:36.689641+0000","last_peered":"2024-09-21T07:45:36.689641+0000","last_clean":"2024-09-21T07:45:36.689641+0000","last_became_active":"2024-09-21T07:45:36.687561+0000","last_became_peered":"2024-09-21T07:45:36.687561+0000","last_unstale":"2024-09-21T07:45:36.689641+0000","last_undegraded":"2024-09-21T07:45:36.689641+0000","last_fullsized":"2024-09-21T07:45:36.689641+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:47:40.418087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":187,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.674812+0000","last_change":"2024-09-21T07:35:27.876021+0000","last_active":"2024-09-21T07:45:36.674812+0000","last_peered":"2024-09-21T07:45:36.674812+0000","last_clean":"2024-09-21T07:45:36.674812+0000","last_became_active":"2024-09-21T07:35:27.875724+0000","last_became_peered":"2024-09-21T07:35:27.875724+0000","last_unstale":"2024-09-21T07:45:36.674812+0000","last_undegraded":"2024-09-21T07:45:36.674812+0000","last_fullsized":"2024-09-21T07:45:36.674812+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:03.022812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.679122+0000","last_change":"2024-09-21T07:35:29.492570+0000","last_active":"2024-09-21T07:45:36.679122+0000","last_peered":"2024-09-21T07:45:36.679122+0000","last_clean":"2024-09-21T07:45:36.679122+0000","last_became_active":"2024-09-21T07:35:26.874316+0000","last_became_peered":"2024-09-21T07:35:26.874316+0000","last_unstale":"2024-09-21T07:45:36.679122+0000","last_undegraded":"2024-09-21T07:45:36.679122+0000","last_fullsized":"2024-09-21T07:45:36.679122+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:16.512323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011990760000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":246,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.674075+0000","last_change":"2024-09-21T07:34:25.085232+0000","last_active":"2024-09-21T07:45:36.674075+0000","last_peered":"2024-09-21T07:45:36.674075+0000","last_clean":"2024-09-21T07:45:36.674075+0000","last_became_active":"2024-09-21T07:34:23.277807+0000","last_became_peered":"2024-09-21T07:34:23.277807+0000","last_unstale":"2024-09-21T07:45:36.674075+0000","last_undegraded":"2024-09-21T07:45:36.674075+0000","last_fullsized":"2024-09-21T07:45:36.674075+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:36.362919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086619099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689375+0000","last_change":"2024-09-21T07:45:37.685148+0000","last_active":"2024-09-21T07:45:38.689375+0000","last_peered":"2024-09-21T07:45:38.689375+0000","last_clean":"2024-09-21T07:45:38.689375+0000","last_became_active":"2024-09-21T07:45:37.684774+0000","last_became_peered":"2024-09-21T07:45:37.684774+0000","last_unstale":"2024-09-21T07:45:38.689375+0000","last_undegraded":"2024-09-21T07:45:38.689375+0000","last_fullsized":"2024-09-21T07:45:38.689375+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:25.616419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"156'1","reported_seq":16,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.687687+0000","last_change":"2024-09-21T07:45:36.686891+0000","last_active":"2024-09-21T07:45:38.687687+0000","last_peered":"2024-09-21T07:45:38.687687+0000","last_clean":"2024-09-21T07:45:38.687687+0000","last_became_active":"2024-09-21T07:45:36.684124+0000","last_became_peered":"2024-09-21T07:45:36.684124+0000","last_unstale":"2024-09-21T07:45:38.687687+0000","last_undegraded":"2024-09-21T07:45:38.687687+0000","last_fullsized":"2024-09-21T07:45:38.687687+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:00.150983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":186,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.679954+0000","last_change":"2024-09-21T07:35:27.872798+0000","last_active":"2024-09-21T07:45:36.679954+0000","last_peered":"2024-09-21T07:45:36.679954+0000","last_clean":"2024-09-21T07:45:36.679954+0000","last_became_active":"2024-09-21T07:35:27.872483+0000","last_became_peered":"2024-09-21T07:35:27.872483+0000","last_unstale":"2024-09-21T07:45:36.679954+0000","last_undegraded":"2024-09-21T07:45:36.679954+0000","last_fullsized":"2024-09-21T07:45:36.679954+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:11.933469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":193,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.679781+0000","last_change":"2024-09-21T07:35:29.206723+0000","last_active":"2024-09-21T07:45:36.679781+0000","last_peered":"2024-09-21T07:45:36.679781+0000","last_clean":"2024-09-21T07:45:36.679781+0000","last_became_active":"2024-09-21T07:35:26.872377+0000","last_became_peered":"2024-09-21T07:35:26.872377+0000","last_unstale":"2024-09-21T07:45:36.679781+0000","last_undegraded":"2024-09-21T07:45:36.679781+0000","last_fullsized":"2024-09-21T07:45:36.679781+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:29.086862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083077200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":246,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689123+0000","last_change":"2024-09-21T07:34:24.597672+0000","last_active":"2024-09-21T07:45:38.689123+0000","last_peered":"2024-09-21T07:45:38.689123+0000","last_clean":"2024-09-21T07:45:38.689123+0000","last_became_active":"2024-09-21T07:34:23.267770+0000","last_became_peered":"2024-09-21T07:34:23.267770+0000","last_unstale":"2024-09-21T07:45:38.689123+0000","last_undegraded":"2024-09-21T07:45:38.689123+0000","last_fullsized":"2024-09-21T07:45:38.689123+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:51.629476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086378500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689895+0000","last_change":"2024-09-21T07:45:37.691898+0000","last_active":"2024-09-21T07:45:38.689895+0000","last_peered":"2024-09-21T07:45:38.689895+0000","last_clean":"2024-09-21T07:45:38.689895+0000","last_became_active":"2024-09-21T07:45:37.691507+0000","last_became_peered":"2024-09-21T07:45:37.691507+0000","last_unstale":"2024-09-21T07:45:38.689895+0000","last_undegraded":"2024-09-21T07:45:38.689895+0000","last_fullsized":"2024-09-21T07:45:38.689895+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:14:43.900850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"156'1","reported_seq":16,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689228+0000","last_change":"2024-09-21T07:45:36.694668+0000","last_active":"2024-09-21T07:45:38.689228+0000","last_peered":"2024-09-21T07:45:38.689228+0000","last_clean":"2024-09-21T07:45:38.689228+0000","last_became_active":"2024-09-21T07:45:36.694405+0000","last_became_peered":"2024-09-21T07:45:36.694405+0000","last_unstale":"2024-09-21T07:45:38.689228+0000","last_undegraded":"2024-09-21T07:45:38.689228+0000","last_fullsized":"2024-09-21T07:45:38.689228+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:08.552085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":186,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.680841+0000","last_change":"2024-09-21T07:35:27.870121+0000","last_active":"2024-09-21T07:45:36.680841+0000","last_peered":"2024-09-21T07:45:36.680841+0000","last_clean":"2024-09-21T07:45:36.680841+0000","last_became_active":"2024-09-21T07:35:27.869811+0000","last_became_peered":"2024-09-21T07:35:27.869811+0000","last_unstale":"2024-09-21T07:45:36.680841+0000","last_undegraded":"2024-09-21T07:45:36.680841+0000","last_fullsized":"2024-09-21T07:45:36.680841+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:31.778461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.696395+0000","last_change":"2024-09-21T07:35:28.860699+0000","last_active":"2024-09-21T07:45:36.696395+0000","last_peered":"2024-09-21T07:45:36.696395+0000","last_clean":"2024-09-21T07:45:36.696395+0000","last_became_active":"2024-09-21T07:35:26.876779+0000","last_became_peered":"2024-09-21T07:35:26.876779+0000","last_unstale":"2024-09-21T07:45:36.696395+0000","last_undegraded":"2024-09-21T07:45:36.696395+0000","last_fullsized":"2024-09-21T07:45:36.696395+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:28.286403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000913188,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":247,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.700060+0000","last_change":"2024-09-21T07:34:24.260181+0000","last_active":"2024-09-21T07:45:36.700060+0000","last_peered":"2024-09-21T07:45:36.700060+0000","last_clean":"2024-09-21T07:45:36.700060+0000","last_became_active":"2024-09-21T07:34:23.275580+0000","last_became_peered":"2024-09-21T07:34:23.275580+0000","last_unstale":"2024-09-21T07:45:36.700060+0000","last_undegraded":"2024-09-21T07:45:36.700060+0000","last_fullsized":"2024-09-21T07:45:36.700060+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:49.857622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014706980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689850+0000","last_change":"2024-09-21T07:45:37.698246+0000","last_active":"2024-09-21T07:45:38.689850+0000","last_peered":"2024-09-21T07:45:38.689850+0000","last_clean":"2024-09-21T07:45:38.689850+0000","last_became_active":"2024-09-21T07:45:37.697792+0000","last_became_peered":"2024-09-21T07:45:37.697792+0000","last_unstale":"2024-09-21T07:45:38.689850+0000","last_undegraded":"2024-09-21T07:45:38.689850+0000","last_fullsized":"2024-09-21T07:45:38.689850+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:06.139752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.694910+0000","last_change":"2024-09-21T07:45:36.693696+0000","last_active":"2024-09-21T07:45:36.694910+0000","last_peered":"2024-09-21T07:45:36.694910+0000","last_clean":"2024-09-21T07:45:36.694910+0000","last_became_active":"2024-09-21T07:45:36.693211+0000","last_became_peered":"2024-09-21T07:45:36.693211+0000","last_unstale":"2024-09-21T07:45:36.694910+0000","last_undegraded":"2024-09-21T07:45:36.694910+0000","last_fullsized":"2024-09-21T07:45:36.694910+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:15.970506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":188,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.672760+0000","last_change":"2024-09-21T07:35:27.875393+0000","last_active":"2024-09-21T07:45:36.672760+0000","last_peered":"2024-09-21T07:45:36.672760+0000","last_clean":"2024-09-21T07:45:36.672760+0000","last_became_active":"2024-09-21T07:35:27.874671+0000","last_became_peered":"2024-09-21T07:35:27.874671+0000","last_unstale":"2024-09-21T07:45:36.672760+0000","last_undegraded":"2024-09-21T07:45:36.672760+0000","last_fullsized":"2024-09-21T07:45:36.672760+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":193,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.678090+0000","last_change":"2024-09-21T07:35:29.206296+0000","last_active":"2024-09-21T07:45:36.678090+0000","last_peered":"2024-09-21T07:45:36.678090+0000","last_clean":"2024-09-21T07:45:36.678090+0000","last_became_active":"2024-09-21T07:35:26.874575+0000","last_became_peered":"2024-09-21T07:35:26.874575+0000","last_unstale":"2024-09-21T07:45:36.678090+0000","last_undegraded":"2024-09-21T07:45:36.678090+0000","last_fullsized":"2024-09-21T07:45:36.678090+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:22.771154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061011599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":252,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690260+0000","last_change":"2024-09-21T07:34:24.259112+0000","last_active":"2024-09-21T07:45:38.690260+0000","last_peered":"2024-09-21T07:45:38.690260+0000","last_clean":"2024-09-21T07:45:38.690260+0000","last_became_active":"2024-09-21T07:34:23.635444+0000","last_became_peered":"2024-09-21T07:34:23.635444+0000","last_unstale":"2024-09-21T07:45:38.690260+0000","last_undegraded":"2024-09-21T07:45:38.690260+0000","last_fullsized":"2024-09-21T07:45:38.690260+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:46.255385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001952636,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.672205+0000","last_change":"2024-09-21T07:45:36.672205+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.672205+0000","last_undegraded":"2024-09-21T07:45:36.672205+0000","last_fullsized":"2024-09-21T07:45:36.672205+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.696954+0000","last_change":"2024-09-21T07:45:36.696284+0000","last_active":"2024-09-21T07:45:36.696954+0000","last_peered":"2024-09-21T07:45:36.696954+0000","last_clean":"2024-09-21T07:45:36.696954+0000","last_became_active":"2024-09-21T07:45:36.695952+0000","last_became_peered":"2024-09-21T07:45:36.695952+0000","last_unstale":"2024-09-21T07:45:36.696954+0000","last_undegraded":"2024-09-21T07:45:36.696954+0000","last_fullsized":"2024-09-21T07:45:36.696954+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:01.857023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":188,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.673558+0000","last_change":"2024-09-21T07:35:27.878356+0000","last_active":"2024-09-21T07:45:36.673558+0000","last_peered":"2024-09-21T07:45:36.673558+0000","last_clean":"2024-09-21T07:45:36.673558+0000","last_became_active":"2024-09-21T07:35:27.877982+0000","last_became_peered":"2024-09-21T07:35:27.877982+0000","last_unstale":"2024-09-21T07:45:36.673558+0000","last_undegraded":"2024-09-21T07:45:36.673558+0000","last_fullsized":"2024-09-21T07:45:36.673558+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":196,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.671879+0000","last_change":"2024-09-21T07:35:29.053131+0000","last_active":"2024-09-21T07:45:36.671879+0000","last_peered":"2024-09-21T07:45:36.671879+0000","last_clean":"2024-09-21T07:45:36.671879+0000","last_became_active":"2024-09-21T07:35:26.870516+0000","last_became_peered":"2024-09-21T07:35:26.870516+0000","last_unstale":"2024-09-21T07:45:36.671879+0000","last_undegraded":"2024-09-21T07:45:36.671879+0000","last_fullsized":"2024-09-21T07:45:36.671879+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:10.924491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070705699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":252,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.669780+0000","last_change":"2024-09-21T07:34:24.539152+0000","last_active":"2024-09-21T07:45:36.669780+0000","last_peered":"2024-09-21T07:45:36.669780+0000","last_clean":"2024-09-21T07:45:36.669780+0000","last_became_active":"2024-09-21T07:34:23.259573+0000","last_became_peered":"2024-09-21T07:34:23.259573+0000","last_unstale":"2024-09-21T07:45:36.669780+0000","last_undegraded":"2024-09-21T07:45:36.669780+0000","last_fullsized":"2024-09-21T07:45:36.669780+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:26.986900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058108400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689007+0000","last_change":"2024-09-21T07:45:37.689304+0000","last_active":"2024-09-21T07:45:38.689007+0000","last_peered":"2024-09-21T07:45:38.689007+0000","last_clean":"2024-09-21T07:45:38.689007+0000","last_became_active":"2024-09-21T07:45:37.688985+0000","last_became_peered":"2024-09-21T07:45:37.688985+0000","last_unstale":"2024-09-21T07:45:38.689007+0000","last_undegraded":"2024-09-21T07:45:38.689007+0000","last_fullsized":"2024-09-21T07:45:38.689007+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:03.596216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.700099+0000","last_change":"2024-09-21T07:45:36.697335+0000","last_active":"2024-09-21T07:45:36.700099+0000","last_peered":"2024-09-21T07:45:36.700099+0000","last_clean":"2024-09-21T07:45:36.700099+0000","last_became_active":"2024-09-21T07:45:36.697036+0000","last_became_peered":"2024-09-21T07:45:36.697036+0000","last_unstale":"2024-09-21T07:45:36.700099+0000","last_undegraded":"2024-09-21T07:45:36.700099+0000","last_fullsized":"2024-09-21T07:45:36.700099+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:16.366485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":188,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.671824+0000","last_change":"2024-09-21T07:35:27.873726+0000","last_active":"2024-09-21T07:45:36.671824+0000","last_peered":"2024-09-21T07:45:36.671824+0000","last_clean":"2024-09-21T07:45:36.671824+0000","last_became_active":"2024-09-21T07:35:27.871421+0000","last_became_peered":"2024-09-21T07:35:27.871421+0000","last_unstale":"2024-09-21T07:45:36.671824+0000","last_undegraded":"2024-09-21T07:45:36.671824+0000","last_fullsized":"2024-09-21T07:45:36.671824+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:00.168915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.699581+0000","last_change":"2024-09-21T07:35:28.861032+0000","last_active":"2024-09-21T07:45:36.699581+0000","last_peered":"2024-09-21T07:45:36.699581+0000","last_clean":"2024-09-21T07:45:36.699581+0000","last_became_active":"2024-09-21T07:35:26.873149+0000","last_became_peered":"2024-09-21T07:35:26.873149+0000","last_unstale":"2024-09-21T07:45:36.699581+0000","last_undegraded":"2024-09-21T07:45:36.699581+0000","last_fullsized":"2024-09-21T07:45:36.699581+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:59.435981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011988369999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":247,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.679365+0000","last_change":"2024-09-21T07:34:24.259902+0000","last_active":"2024-09-21T07:45:36.679365+0000","last_peered":"2024-09-21T07:45:36.679365+0000","last_clean":"2024-09-21T07:45:36.679365+0000","last_became_active":"2024-09-21T07:34:23.271902+0000","last_became_peered":"2024-09-21T07:34:23.271902+0000","last_unstale":"2024-09-21T07:45:36.679365+0000","last_undegraded":"2024-09-21T07:45:36.679365+0000","last_fullsized":"2024-09-21T07:45:36.679365+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:59.439525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001686358,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.671064+0000","last_change":"2024-09-21T07:45:36.671064+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.671064+0000","last_undegraded":"2024-09-21T07:45:36.671064+0000","last_fullsized":"2024-09-21T07:45:36.671064+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"157'1","reported_seq":16,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689021+0000","last_change":"2024-09-21T07:45:36.673147+0000","last_active":"2024-09-21T07:45:38.689021+0000","last_peered":"2024-09-21T07:45:38.689021+0000","last_clean":"2024-09-21T07:45:38.689021+0000","last_became_active":"2024-09-21T07:45:36.672816+0000","last_became_peered":"2024-09-21T07:45:36.672816+0000","last_unstale":"2024-09-21T07:45:38.689021+0000","last_undegraded":"2024-09-21T07:45:38.689021+0000","last_fullsized":"2024-09-21T07:45:38.689021+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:04:18.857870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":187,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.673815+0000","last_change":"2024-09-21T07:35:27.872637+0000","last_active":"2024-09-21T07:45:36.673815+0000","last_peered":"2024-09-21T07:45:36.673815+0000","last_clean":"2024-09-21T07:45:36.673815+0000","last_became_active":"2024-09-21T07:35:27.872149+0000","last_became_peered":"2024-09-21T07:35:27.872149+0000","last_unstale":"2024-09-21T07:45:36.673815+0000","last_undegraded":"2024-09-21T07:45:36.673815+0000","last_fullsized":"2024-09-21T07:45:36.673815+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:27.697872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":195,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.678962+0000","last_change":"2024-09-21T07:35:29.493362+0000","last_active":"2024-09-21T07:45:36.678962+0000","last_peered":"2024-09-21T07:45:36.678962+0000","last_clean":"2024-09-21T07:45:36.678962+0000","last_became_active":"2024-09-21T07:35:26.873018+0000","last_became_peered":"2024-09-21T07:35:26.873018+0000","last_unstale":"2024-09-21T07:45:36.678962+0000","last_undegraded":"2024-09-21T07:45:36.678962+0000","last_fullsized":"2024-09-21T07:45:36.678962+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:02.460557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012496510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":251,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689489+0000","last_change":"2024-09-21T07:34:24.259662+0000","last_active":"2024-09-21T07:45:38.689489+0000","last_peered":"2024-09-21T07:45:38.689489+0000","last_clean":"2024-09-21T07:45:38.689489+0000","last_became_active":"2024-09-21T07:34:23.278812+0000","last_became_peered":"2024-09-21T07:34:23.278812+0000","last_unstale":"2024-09-21T07:45:38.689489+0000","last_undegraded":"2024-09-21T07:45:38.689489+0000","last_fullsized":"2024-09-21T07:45:38.689489+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:49.829974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356463,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.675060+0000","last_change":"2024-09-21T07:45:36.675060+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.675060+0000","last_undegraded":"2024-09-21T07:45:36.675060+0000","last_fullsized":"2024-09-21T07:45:36.675060+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689740+0000","last_change":"2024-09-21T07:45:36.693420+0000","last_active":"2024-09-21T07:45:38.689740+0000","last_peered":"2024-09-21T07:45:38.689740+0000","last_clean":"2024-09-21T07:45:38.689740+0000","last_became_active":"2024-09-21T07:45:36.692969+0000","last_became_peered":"2024-09-21T07:45:36.692969+0000","last_unstale":"2024-09-21T07:45:38.689740+0000","last_undegraded":"2024-09-21T07:45:38.689740+0000","last_fullsized":"2024-09-21T07:45:38.689740+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:09:22.417758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":189,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.687613+0000","last_change":"2024-09-21T07:35:27.868448+0000","last_active":"2024-09-21T07:45:38.687613+0000","last_peered":"2024-09-21T07:45:38.687613+0000","last_clean":"2024-09-21T07:45:38.687613+0000","last_became_active":"2024-09-21T07:35:27.868105+0000","last_became_peered":"2024-09-21T07:35:27.868105+0000","last_unstale":"2024-09-21T07:45:38.687613+0000","last_undegraded":"2024-09-21T07:45:38.687613+0000","last_fullsized":"2024-09-21T07:45:38.687613+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:41.468702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.678932+0000","last_change":"2024-09-21T07:35:29.492974+0000","last_active":"2024-09-21T07:45:36.678932+0000","last_peered":"2024-09-21T07:45:36.678932+0000","last_clean":"2024-09-21T07:45:36.678932+0000","last_became_active":"2024-09-21T07:35:26.872165+0000","last_became_peered":"2024-09-21T07:35:26.872165+0000","last_unstale":"2024-09-21T07:45:36.678932+0000","last_undegraded":"2024-09-21T07:45:36.678932+0000","last_fullsized":"2024-09-21T07:45:36.678932+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:59.912520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013795179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":245,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.672175+0000","last_change":"2024-09-21T07:34:25.084861+0000","last_active":"2024-09-21T07:45:36.672175+0000","last_peered":"2024-09-21T07:45:36.672175+0000","last_clean":"2024-09-21T07:45:36.672175+0000","last_became_active":"2024-09-21T07:34:23.277708+0000","last_became_peered":"2024-09-21T07:34:23.277708+0000","last_unstale":"2024-09-21T07:45:36.672175+0000","last_undegraded":"2024-09-21T07:45:36.672175+0000","last_fullsized":"2024-09-21T07:45:36.672175+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:40.816488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000597566,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.677283+0000","last_change":"2024-09-21T07:45:36.677283+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.677283+0000","last_undegraded":"2024-09-21T07:45:36.677283+0000","last_fullsized":"2024-09-21T07:45:36.677283+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.688738+0000","last_change":"2024-09-21T07:45:36.687709+0000","last_active":"2024-09-21T07:45:36.688738+0000","last_peered":"2024-09-21T07:45:36.688738+0000","last_clean":"2024-09-21T07:45:36.688738+0000","last_became_active":"2024-09-21T07:45:36.684703+0000","last_became_peered":"2024-09-21T07:45:36.684703+0000","last_unstale":"2024-09-21T07:45:36.688738+0000","last_undegraded":"2024-09-21T07:45:36.688738+0000","last_fullsized":"2024-09-21T07:45:36.688738+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:59.601883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":189,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688381+0000","last_change":"2024-09-21T07:35:27.874640+0000","last_active":"2024-09-21T07:45:38.688381+0000","last_peered":"2024-09-21T07:45:38.688381+0000","last_clean":"2024-09-21T07:45:38.688381+0000","last_became_active":"2024-09-21T07:35:27.874265+0000","last_became_peered":"2024-09-21T07:35:27.874265+0000","last_unstale":"2024-09-21T07:45:38.688381+0000","last_undegraded":"2024-09-21T07:45:38.688381+0000","last_fullsized":"2024-09-21T07:45:38.688381+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:45.754654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":193,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.680159+0000","last_change":"2024-09-21T07:35:29.207392+0000","last_active":"2024-09-21T07:45:36.680159+0000","last_peered":"2024-09-21T07:45:36.680159+0000","last_clean":"2024-09-21T07:45:36.680159+0000","last_became_active":"2024-09-21T07:35:26.870917+0000","last_became_peered":"2024-09-21T07:35:26.870917+0000","last_unstale":"2024-09-21T07:45:36.680159+0000","last_undegraded":"2024-09-21T07:45:36.680159+0000","last_fullsized":"2024-09-21T07:45:36.680159+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:59.159937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059198499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":251,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689049+0000","last_change":"2024-09-21T07:34:24.258709+0000","last_active":"2024-09-21T07:45:38.689049+0000","last_peered":"2024-09-21T07:45:38.689049+0000","last_clean":"2024-09-21T07:45:38.689049+0000","last_became_active":"2024-09-21T07:34:23.279042+0000","last_became_peered":"2024-09-21T07:34:23.279042+0000","last_unstale":"2024-09-21T07:45:38.689049+0000","last_undegraded":"2024-09-21T07:45:38.689049+0000","last_fullsized":"2024-09-21T07:45:38.689049+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:01.762117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019282100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.676888+0000","last_change":"2024-09-21T07:45:36.676888+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.676888+0000","last_undegraded":"2024-09-21T07:45:36.676888+0000","last_fullsized":"2024-09-21T07:45:36.676888+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.699451+0000","last_change":"2024-09-21T07:45:36.698592+0000","last_active":"2024-09-21T07:45:36.699451+0000","last_peered":"2024-09-21T07:45:36.699451+0000","last_clean":"2024-09-21T07:45:36.699451+0000","last_became_active":"2024-09-21T07:45:36.698011+0000","last_became_peered":"2024-09-21T07:45:36.698011+0000","last_unstale":"2024-09-21T07:45:36.699451+0000","last_undegraded":"2024-09-21T07:45:36.699451+0000","last_fullsized":"2024-09-21T07:45:36.699451+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:00.979027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":187,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.676080+0000","last_change":"2024-09-21T07:35:27.870260+0000","last_active":"2024-09-21T07:45:36.676080+0000","last_peered":"2024-09-21T07:45:36.676080+0000","last_clean":"2024-09-21T07:45:36.676080+0000","last_became_active":"2024-09-21T07:35:27.869448+0000","last_became_peered":"2024-09-21T07:35:27.869448+0000","last_unstale":"2024-09-21T07:45:36.676080+0000","last_undegraded":"2024-09-21T07:45:36.676080+0000","last_fullsized":"2024-09-21T07:45:36.676080+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:16.946790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":196,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688161+0000","last_change":"2024-09-21T07:35:29.386367+0000","last_active":"2024-09-21T07:45:38.688161+0000","last_peered":"2024-09-21T07:45:38.688161+0000","last_clean":"2024-09-21T07:45:38.688161+0000","last_became_active":"2024-09-21T07:35:26.879238+0000","last_became_peered":"2024-09-21T07:35:26.879238+0000","last_unstale":"2024-09-21T07:45:38.688161+0000","last_undegraded":"2024-09-21T07:45:38.688161+0000","last_fullsized":"2024-09-21T07:45:38.688161+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:16.528260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092223100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":245,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.677233+0000","last_change":"2024-09-21T07:34:25.085635+0000","last_active":"2024-09-21T07:45:36.677233+0000","last_peered":"2024-09-21T07:45:36.677233+0000","last_clean":"2024-09-21T07:45:36.677233+0000","last_became_active":"2024-09-21T07:34:23.273566+0000","last_became_peered":"2024-09-21T07:34:23.273566+0000","last_unstale":"2024-09-21T07:45:36.677233+0000","last_undegraded":"2024-09-21T07:45:36.677233+0000","last_fullsized":"2024-09-21T07:45:36.677233+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:14.142272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079897300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.676058+0000","last_change":"2024-09-21T07:45:36.676058+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.676058+0000","last_undegraded":"2024-09-21T07:45:36.676058+0000","last_fullsized":"2024-09-21T07:45:36.676058+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.689075+0000","last_change":"2024-09-21T07:45:36.685900+0000","last_active":"2024-09-21T07:45:36.689075+0000","last_peered":"2024-09-21T07:45:36.689075+0000","last_clean":"2024-09-21T07:45:36.689075+0000","last_became_active":"2024-09-21T07:45:36.685578+0000","last_became_peered":"2024-09-21T07:45:36.685578+0000","last_unstale":"2024-09-21T07:45:36.689075+0000","last_undegraded":"2024-09-21T07:45:36.689075+0000","last_fullsized":"2024-09-21T07:45:36.689075+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:03.369431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":187,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.677035+0000","last_change":"2024-09-21T07:35:27.878728+0000","last_active":"2024-09-21T07:45:36.677035+0000","last_peered":"2024-09-21T07:45:36.677035+0000","last_clean":"2024-09-21T07:45:36.677035+0000","last_became_active":"2024-09-21T07:35:27.878335+0000","last_became_peered":"2024-09-21T07:35:27.878335+0000","last_unstale":"2024-09-21T07:45:36.677035+0000","last_undegraded":"2024-09-21T07:45:36.677035+0000","last_fullsized":"2024-09-21T07:45:36.677035+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:25:37.731771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":199,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.668981+0000","last_change":"2024-09-21T07:35:28.858418+0000","last_active":"2024-09-21T07:45:36.668981+0000","last_peered":"2024-09-21T07:45:36.668981+0000","last_clean":"2024-09-21T07:45:36.668981+0000","last_became_active":"2024-09-21T07:35:26.876944+0000","last_became_peered":"2024-09-21T07:35:26.876944+0000","last_unstale":"2024-09-21T07:45:36.668981+0000","last_undegraded":"2024-09-21T07:45:36.668981+0000","last_fullsized":"2024-09-21T07:45:36.668981+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:04.514304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090862799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":245,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688906+0000","last_change":"2024-09-21T07:34:24.597926+0000","last_active":"2024-09-21T07:45:38.688906+0000","last_peered":"2024-09-21T07:45:38.688906+0000","last_clean":"2024-09-21T07:45:38.688906+0000","last_became_active":"2024-09-21T07:34:23.268926+0000","last_became_peered":"2024-09-21T07:34:23.268926+0000","last_unstale":"2024-09-21T07:45:38.688906+0000","last_undegraded":"2024-09-21T07:45:38.688906+0000","last_fullsized":"2024-09-21T07:45:38.688906+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:46.190752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101851,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.676259+0000","last_change":"2024-09-21T07:45:36.676259+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.676259+0000","last_undegraded":"2024-09-21T07:45:36.676259+0000","last_fullsized":"2024-09-21T07:45:36.676259+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.685429+0000","last_change":"2024-09-21T07:45:36.683898+0000","last_active":"2024-09-21T07:45:36.685429+0000","last_peered":"2024-09-21T07:45:36.685429+0000","last_clean":"2024-09-21T07:45:36.685429+0000","last_became_active":"2024-09-21T07:45:36.683536+0000","last_became_peered":"2024-09-21T07:45:36.683536+0000","last_unstale":"2024-09-21T07:45:36.685429+0000","last_undegraded":"2024-09-21T07:45:36.685429+0000","last_fullsized":"2024-09-21T07:45:36.685429+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:47.039079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688581+0000","last_change":"2024-09-21T07:35:27.877426+0000","last_active":"2024-09-21T07:45:38.688581+0000","last_peered":"2024-09-21T07:45:38.688581+0000","last_clean":"2024-09-21T07:45:38.688581+0000","last_became_active":"2024-09-21T07:35:27.877050+0000","last_became_peered":"2024-09-21T07:35:27.877050+0000","last_unstale":"2024-09-21T07:45:38.688581+0000","last_undegraded":"2024-09-21T07:45:38.688581+0000","last_fullsized":"2024-09-21T07:45:38.688581+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:36.268334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.673914+0000","last_change":"2024-09-21T07:35:28.862273+0000","last_active":"2024-09-21T07:45:36.673914+0000","last_peered":"2024-09-21T07:45:36.673914+0000","last_clean":"2024-09-21T07:45:36.673914+0000","last_became_active":"2024-09-21T07:35:26.874011+0000","last_became_peered":"2024-09-21T07:35:26.874011+0000","last_unstale":"2024-09-21T07:45:36.673914+0000","last_undegraded":"2024-09-21T07:45:36.673914+0000","last_fullsized":"2024-09-21T07:45:36.673914+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:11.986950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071644799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":246,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.697773+0000","last_change":"2024-09-21T07:34:24.259794+0000","last_active":"2024-09-21T07:45:36.697773+0000","last_peered":"2024-09-21T07:45:36.697773+0000","last_clean":"2024-09-21T07:45:36.697773+0000","last_became_active":"2024-09-21T07:34:23.265679+0000","last_became_peered":"2024-09-21T07:34:23.265679+0000","last_unstale":"2024-09-21T07:45:36.697773+0000","last_undegraded":"2024-09-21T07:45:36.697773+0000","last_fullsized":"2024-09-21T07:45:36.697773+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:02.575796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001298518,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.673113+0000","last_change":"2024-09-21T07:45:36.673113+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.673113+0000","last_undegraded":"2024-09-21T07:45:36.673113+0000","last_fullsized":"2024-09-21T07:45:36.673113+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689224+0000","last_change":"2024-09-21T07:45:36.695633+0000","last_active":"2024-09-21T07:45:38.689224+0000","last_peered":"2024-09-21T07:45:38.689224+0000","last_clean":"2024-09-21T07:45:38.689224+0000","last_became_active":"2024-09-21T07:45:36.695319+0000","last_became_peered":"2024-09-21T07:45:36.695319+0000","last_unstale":"2024-09-21T07:45:38.689224+0000","last_undegraded":"2024-09-21T07:45:38.689224+0000","last_fullsized":"2024-09-21T07:45:38.689224+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:36.781831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":186,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.680833+0000","last_change":"2024-09-21T07:35:27.877725+0000","last_active":"2024-09-21T07:45:36.680833+0000","last_peered":"2024-09-21T07:45:36.680833+0000","last_clean":"2024-09-21T07:45:36.680833+0000","last_became_active":"2024-09-21T07:35:27.877398+0000","last_became_peered":"2024-09-21T07:35:27.877398+0000","last_unstale":"2024-09-21T07:45:36.680833+0000","last_undegraded":"2024-09-21T07:45:36.680833+0000","last_fullsized":"2024-09-21T07:45:36.680833+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:05.859762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.693485+0000","last_change":"2024-09-21T07:35:28.860612+0000","last_active":"2024-09-21T07:45:36.693485+0000","last_peered":"2024-09-21T07:45:36.693485+0000","last_clean":"2024-09-21T07:45:36.693485+0000","last_became_active":"2024-09-21T07:35:26.876562+0000","last_became_peered":"2024-09-21T07:35:26.876562+0000","last_unstale":"2024-09-21T07:45:36.693485+0000","last_undegraded":"2024-09-21T07:45:36.693485+0000","last_fullsized":"2024-09-21T07:45:36.693485+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:06.824331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012252470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":247,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.679653+0000","last_change":"2024-09-21T07:34:24.260330+0000","last_active":"2024-09-21T07:45:36.679653+0000","last_peered":"2024-09-21T07:45:36.679653+0000","last_clean":"2024-09-21T07:45:36.679653+0000","last_became_active":"2024-09-21T07:34:23.637210+0000","last_became_peered":"2024-09-21T07:34:23.637210+0000","last_unstale":"2024-09-21T07:45:36.679653+0000","last_undegraded":"2024-09-21T07:45:36.679653+0000","last_fullsized":"2024-09-21T07:45:36.679653+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.488451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001537093,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.673352+0000","last_change":"2024-09-21T07:45:36.673352+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.673352+0000","last_undegraded":"2024-09-21T07:45:36.673352+0000","last_fullsized":"2024-09-21T07:45:36.673352+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.677935+0000","last_change":"2024-09-21T07:45:36.676063+0000","last_active":"2024-09-21T07:45:36.677935+0000","last_peered":"2024-09-21T07:45:36.677935+0000","last_clean":"2024-09-21T07:45:36.677935+0000","last_became_active":"2024-09-21T07:45:36.675750+0000","last_became_peered":"2024-09-21T07:45:36.675750+0000","last_unstale":"2024-09-21T07:45:36.677935+0000","last_undegraded":"2024-09-21T07:45:36.677935+0000","last_fullsized":"2024-09-21T07:45:36.677935+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:20:48.069405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":187,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.673680+0000","last_change":"2024-09-21T07:35:27.874757+0000","last_active":"2024-09-21T07:45:36.673680+0000","last_peered":"2024-09-21T07:45:36.673680+0000","last_clean":"2024-09-21T07:45:36.673680+0000","last_became_active":"2024-09-21T07:35:27.874360+0000","last_became_peered":"2024-09-21T07:35:27.874360+0000","last_unstale":"2024-09-21T07:45:36.673680+0000","last_undegraded":"2024-09-21T07:45:36.673680+0000","last_fullsized":"2024-09-21T07:45:36.673680+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:17.646912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.680294+0000","last_change":"2024-09-21T07:35:29.493431+0000","last_active":"2024-09-21T07:45:36.680294+0000","last_peered":"2024-09-21T07:45:36.680294+0000","last_clean":"2024-09-21T07:45:36.680294+0000","last_became_active":"2024-09-21T07:35:26.873691+0000","last_became_peered":"2024-09-21T07:35:26.873691+0000","last_unstale":"2024-09-21T07:45:36.680294+0000","last_undegraded":"2024-09-21T07:45:36.680294+0000","last_fullsized":"2024-09-21T07:45:36.680294+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:51.705051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001162464,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":245,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.687901+0000","last_change":"2024-09-21T07:34:24.597286+0000","last_active":"2024-09-21T07:45:38.687901+0000","last_peered":"2024-09-21T07:45:38.687901+0000","last_clean":"2024-09-21T07:45:38.687901+0000","last_became_active":"2024-09-21T07:34:23.267133+0000","last_became_peered":"2024-09-21T07:34:23.267133+0000","last_unstale":"2024-09-21T07:45:38.687901+0000","last_undegraded":"2024-09-21T07:45:38.687901+0000","last_fullsized":"2024-09-21T07:45:38.687901+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:18.257010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.670174+0000","last_change":"2024-09-21T07:45:36.670174+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.670174+0000","last_undegraded":"2024-09-21T07:45:36.670174+0000","last_fullsized":"2024-09-21T07:45:36.670174+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.694069+0000","last_change":"2024-09-21T07:45:36.693348+0000","last_active":"2024-09-21T07:45:36.694069+0000","last_peered":"2024-09-21T07:45:36.694069+0000","last_clean":"2024-09-21T07:45:36.694069+0000","last_became_active":"2024-09-21T07:45:36.692821+0000","last_became_peered":"2024-09-21T07:45:36.692821+0000","last_unstale":"2024-09-21T07:45:36.694069+0000","last_undegraded":"2024-09-21T07:45:36.694069+0000","last_fullsized":"2024-09-21T07:45:36.694069+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:15:15.557087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":192,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.668792+0000","last_change":"2024-09-21T07:35:27.864458+0000","last_active":"2024-09-21T07:45:36.668792+0000","last_peered":"2024-09-21T07:45:36.668792+0000","last_clean":"2024-09-21T07:45:36.668792+0000","last_became_active":"2024-09-21T07:35:27.864136+0000","last_became_peered":"2024-09-21T07:35:27.864136+0000","last_unstale":"2024-09-21T07:45:36.668792+0000","last_undegraded":"2024-09-21T07:45:36.668792+0000","last_fullsized":"2024-09-21T07:45:36.668792+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:20.162240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":199,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.669445+0000","last_change":"2024-09-21T07:35:28.858784+0000","last_active":"2024-09-21T07:45:36.669445+0000","last_peered":"2024-09-21T07:45:36.669445+0000","last_clean":"2024-09-21T07:45:36.669445+0000","last_became_active":"2024-09-21T07:35:26.876890+0000","last_became_peered":"2024-09-21T07:35:26.876890+0000","last_unstale":"2024-09-21T07:45:36.669445+0000","last_undegraded":"2024-09-21T07:45:36.669445+0000","last_fullsized":"2024-09-21T07:45:36.669445+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:40.738327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064543499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":251,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689383+0000","last_change":"2024-09-21T07:34:24.258712+0000","last_active":"2024-09-21T07:45:38.689383+0000","last_peered":"2024-09-21T07:45:38.689383+0000","last_clean":"2024-09-21T07:45:38.689383+0000","last_became_active":"2024-09-21T07:34:23.274692+0000","last_became_peered":"2024-09-21T07:34:23.274692+0000","last_unstale":"2024-09-21T07:45:38.689383+0000","last_undegraded":"2024-09-21T07:45:38.689383+0000","last_fullsized":"2024-09-21T07:45:38.689383+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:43.279454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018051479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.673682+0000","last_change":"2024-09-21T07:45:36.673682+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.673682+0000","last_undegraded":"2024-09-21T07:45:36.673682+0000","last_fullsized":"2024-09-21T07:45:36.673682+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.702127+0000","last_change":"2024-09-21T07:45:36.700873+0000","last_active":"2024-09-21T07:45:36.702127+0000","last_peered":"2024-09-21T07:45:36.702127+0000","last_clean":"2024-09-21T07:45:36.702127+0000","last_became_active":"2024-09-21T07:45:36.700526+0000","last_became_peered":"2024-09-21T07:45:36.700526+0000","last_unstale":"2024-09-21T07:45:36.702127+0000","last_undegraded":"2024-09-21T07:45:36.702127+0000","last_fullsized":"2024-09-21T07:45:36.702127+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:08:53.416498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689061+0000","last_change":"2024-09-21T07:35:27.878020+0000","last_active":"2024-09-21T07:45:38.689061+0000","last_peered":"2024-09-21T07:45:38.689061+0000","last_clean":"2024-09-21T07:45:38.689061+0000","last_became_active":"2024-09-21T07:35:27.877532+0000","last_became_peered":"2024-09-21T07:35:27.877532+0000","last_unstale":"2024-09-21T07:45:38.689061+0000","last_undegraded":"2024-09-21T07:45:38.689061+0000","last_fullsized":"2024-09-21T07:45:38.689061+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:19.756073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.698780+0000","last_change":"2024-09-21T07:35:28.861049+0000","last_active":"2024-09-21T07:45:36.698780+0000","last_peered":"2024-09-21T07:45:36.698780+0000","last_clean":"2024-09-21T07:45:36.698780+0000","last_became_active":"2024-09-21T07:35:26.876819+0000","last_became_peered":"2024-09-21T07:35:26.876819+0000","last_unstale":"2024-09-21T07:45:36.698780+0000","last_undegraded":"2024-09-21T07:45:36.698780+0000","last_fullsized":"2024-09-21T07:45:36.698780+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:19.633803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055780599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":245,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.672049+0000","last_change":"2024-09-21T07:34:25.084849+0000","last_active":"2024-09-21T07:45:36.672049+0000","last_peered":"2024-09-21T07:45:36.672049+0000","last_clean":"2024-09-21T07:45:36.672049+0000","last_became_active":"2024-09-21T07:34:23.277713+0000","last_became_peered":"2024-09-21T07:34:23.277713+0000","last_unstale":"2024-09-21T07:45:36.672049+0000","last_undegraded":"2024-09-21T07:45:36.672049+0000","last_fullsized":"2024-09-21T07:45:36.672049+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:05.811810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059079499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.677740+0000","last_change":"2024-09-21T07:45:36.677740+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.677740+0000","last_undegraded":"2024-09-21T07:45:36.677740+0000","last_fullsized":"2024-09-21T07:45:36.677740+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690040+0000","last_change":"2024-09-21T07:45:36.693932+0000","last_active":"2024-09-21T07:45:38.690040+0000","last_peered":"2024-09-21T07:45:38.690040+0000","last_clean":"2024-09-21T07:45:38.690040+0000","last_became_active":"2024-09-21T07:45:36.693640+0000","last_became_peered":"2024-09-21T07:45:36.693640+0000","last_unstale":"2024-09-21T07:45:38.690040+0000","last_undegraded":"2024-09-21T07:45:38.690040+0000","last_fullsized":"2024-09-21T07:45:38.690040+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:04.701463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":188,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.672603+0000","last_change":"2024-09-21T07:35:27.884248+0000","last_active":"2024-09-21T07:45:36.672603+0000","last_peered":"2024-09-21T07:45:36.672603+0000","last_clean":"2024-09-21T07:45:36.672603+0000","last_became_active":"2024-09-21T07:35:27.883861+0000","last_became_peered":"2024-09-21T07:35:27.883861+0000","last_unstale":"2024-09-21T07:45:36.672603+0000","last_undegraded":"2024-09-21T07:45:36.672603+0000","last_fullsized":"2024-09-21T07:45:36.672603+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":193,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.680061+0000","last_change":"2024-09-21T07:35:29.207135+0000","last_active":"2024-09-21T07:45:36.680061+0000","last_peered":"2024-09-21T07:45:36.680061+0000","last_clean":"2024-09-21T07:45:36.680061+0000","last_became_active":"2024-09-21T07:35:26.872787+0000","last_became_peered":"2024-09-21T07:35:26.872787+0000","last_unstale":"2024-09-21T07:45:36.680061+0000","last_undegraded":"2024-09-21T07:45:36.680061+0000","last_fullsized":"2024-09-21T07:45:36.680061+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:27.543217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011157509999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":251,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.668930+0000","last_change":"2024-09-21T07:34:24.538610+0000","last_active":"2024-09-21T07:45:36.668930+0000","last_peered":"2024-09-21T07:45:36.668930+0000","last_clean":"2024-09-21T07:45:36.668930+0000","last_became_active":"2024-09-21T07:34:23.259700+0000","last_became_peered":"2024-09-21T07:34:23.259700+0000","last_unstale":"2024-09-21T07:45:36.668930+0000","last_undegraded":"2024-09-21T07:45:36.668930+0000","last_fullsized":"2024-09-21T07:45:36.668930+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:57.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070522399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690121+0000","last_change":"2024-09-21T07:45:37.714568+0000","last_active":"2024-09-21T07:45:38.690121+0000","last_peered":"2024-09-21T07:45:38.690121+0000","last_clean":"2024-09-21T07:45:38.690121+0000","last_became_active":"2024-09-21T07:45:37.714235+0000","last_became_peered":"2024-09-21T07:45:37.714235+0000","last_unstale":"2024-09-21T07:45:38.690121+0000","last_undegraded":"2024-09-21T07:45:38.690121+0000","last_fullsized":"2024-09-21T07:45:38.690121+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:29.318374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"156'2","reported_seq":17,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.687809+0000","last_change":"2024-09-21T07:45:36.685978+0000","last_active":"2024-09-21T07:45:38.687809+0000","last_peered":"2024-09-21T07:45:38.687809+0000","last_clean":"2024-09-21T07:45:38.687809+0000","last_became_active":"2024-09-21T07:45:36.685307+0000","last_became_peered":"2024-09-21T07:45:36.685307+0000","last_unstale":"2024-09-21T07:45:38.687809+0000","last_undegraded":"2024-09-21T07:45:38.687809+0000","last_fullsized":"2024-09-21T07:45:38.687809+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:18.642482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":192,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.669313+0000","last_change":"2024-09-21T07:35:27.868024+0000","last_active":"2024-09-21T07:45:36.669313+0000","last_peered":"2024-09-21T07:45:36.669313+0000","last_clean":"2024-09-21T07:45:36.669313+0000","last_became_active":"2024-09-21T07:35:27.867339+0000","last_became_peered":"2024-09-21T07:35:27.867339+0000","last_unstale":"2024-09-21T07:45:36.669313+0000","last_undegraded":"2024-09-21T07:45:36.669313+0000","last_fullsized":"2024-09-21T07:45:36.669313+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":199,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.669113+0000","last_change":"2024-09-21T07:35:28.858505+0000","last_active":"2024-09-21T07:45:36.669113+0000","last_peered":"2024-09-21T07:45:36.669113+0000","last_clean":"2024-09-21T07:45:36.669113+0000","last_became_active":"2024-09-21T07:35:26.877320+0000","last_became_peered":"2024-09-21T07:35:26.877320+0000","last_unstale":"2024-09-21T07:45:36.669113+0000","last_undegraded":"2024-09-21T07:45:36.669113+0000","last_fullsized":"2024-09-21T07:45:36.669113+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:53.672782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244119999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":246,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689180+0000","last_change":"2024-09-21T07:34:24.597597+0000","last_active":"2024-09-21T07:45:38.689180+0000","last_peered":"2024-09-21T07:45:38.689180+0000","last_clean":"2024-09-21T07:45:38.689180+0000","last_became_active":"2024-09-21T07:34:23.635204+0000","last_became_peered":"2024-09-21T07:34:23.635204+0000","last_unstale":"2024-09-21T07:45:38.689180+0000","last_undegraded":"2024-09-21T07:45:38.689180+0000","last_fullsized":"2024-09-21T07:45:38.689180+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:48.644789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081139199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.689181+0000","last_change":"2024-09-21T07:45:36.689181+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.689181+0000","last_undegraded":"2024-09-21T07:45:36.689181+0000","last_fullsized":"2024-09-21T07:45:36.689181+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.696649+0000","last_change":"2024-09-21T07:45:36.693812+0000","last_active":"2024-09-21T07:45:36.696649+0000","last_peered":"2024-09-21T07:45:36.696649+0000","last_clean":"2024-09-21T07:45:36.696649+0000","last_became_active":"2024-09-21T07:45:36.692616+0000","last_became_peered":"2024-09-21T07:45:36.692616+0000","last_unstale":"2024-09-21T07:45:36.696649+0000","last_undegraded":"2024-09-21T07:45:36.696649+0000","last_fullsized":"2024-09-21T07:45:36.696649+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:12:40.958669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":186,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.680482+0000","last_change":"2024-09-21T07:35:27.877503+0000","last_active":"2024-09-21T07:45:36.680482+0000","last_peered":"2024-09-21T07:45:36.680482+0000","last_clean":"2024-09-21T07:45:36.680482+0000","last_became_active":"2024-09-21T07:35:27.875775+0000","last_became_peered":"2024-09-21T07:35:27.875775+0000","last_unstale":"2024-09-21T07:45:36.680482+0000","last_undegraded":"2024-09-21T07:45:36.680482+0000","last_fullsized":"2024-09-21T07:45:36.680482+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:49.291023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.678647+0000","last_change":"2024-09-21T07:35:29.491412+0000","last_active":"2024-09-21T07:45:36.678647+0000","last_peered":"2024-09-21T07:45:36.678647+0000","last_clean":"2024-09-21T07:45:36.678647+0000","last_became_active":"2024-09-21T07:35:26.876404+0000","last_became_peered":"2024-09-21T07:35:26.876404+0000","last_unstale":"2024-09-21T07:45:36.678647+0000","last_undegraded":"2024-09-21T07:45:36.678647+0000","last_fullsized":"2024-09-21T07:45:36.678647+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063918000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":252,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689996+0000","last_change":"2024-09-21T07:34:24.259026+0000","last_active":"2024-09-21T07:45:38.689996+0000","last_peered":"2024-09-21T07:45:38.689996+0000","last_clean":"2024-09-21T07:45:38.689996+0000","last_became_active":"2024-09-21T07:34:23.635456+0000","last_became_peered":"2024-09-21T07:34:23.635456+0000","last_unstale":"2024-09-21T07:45:38.689996+0000","last_undegraded":"2024-09-21T07:45:38.689996+0000","last_fullsized":"2024-09-21T07:45:38.689996+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:09.592752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018654419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690345+0000","last_change":"2024-09-21T07:45:37.700588+0000","last_active":"2024-09-21T07:45:38.690345+0000","last_peered":"2024-09-21T07:45:38.690345+0000","last_clean":"2024-09-21T07:45:38.690345+0000","last_became_active":"2024-09-21T07:45:37.700288+0000","last_became_peered":"2024-09-21T07:45:37.700288+0000","last_unstale":"2024-09-21T07:45:38.690345+0000","last_undegraded":"2024-09-21T07:45:38.690345+0000","last_fullsized":"2024-09-21T07:45:38.690345+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:36:25.586374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.696399+0000","last_change":"2024-09-21T07:45:36.695336+0000","last_active":"2024-09-21T07:45:36.696399+0000","last_peered":"2024-09-21T07:45:36.696399+0000","last_clean":"2024-09-21T07:45:36.696399+0000","last_became_active":"2024-09-21T07:45:36.694952+0000","last_became_peered":"2024-09-21T07:45:36.694952+0000","last_unstale":"2024-09-21T07:45:36.696399+0000","last_undegraded":"2024-09-21T07:45:36.696399+0000","last_fullsized":"2024-09-21T07:45:36.696399+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:52.278657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":186,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.680709+0000","last_change":"2024-09-21T07:35:27.871984+0000","last_active":"2024-09-21T07:45:36.680709+0000","last_peered":"2024-09-21T07:45:36.680709+0000","last_clean":"2024-09-21T07:45:36.680709+0000","last_became_active":"2024-09-21T07:35:27.870336+0000","last_became_peered":"2024-09-21T07:35:27.870336+0000","last_unstale":"2024-09-21T07:45:36.680709+0000","last_undegraded":"2024-09-21T07:45:36.680709+0000","last_fullsized":"2024-09-21T07:45:36.680709+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:57.120479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.693795+0000","last_change":"2024-09-21T07:35:28.860351+0000","last_active":"2024-09-21T07:45:36.693795+0000","last_peered":"2024-09-21T07:45:36.693795+0000","last_clean":"2024-09-21T07:45:36.693795+0000","last_became_active":"2024-09-21T07:35:26.872046+0000","last_became_peered":"2024-09-21T07:35:26.872046+0000","last_unstale":"2024-09-21T07:45:36.693795+0000","last_undegraded":"2024-09-21T07:45:36.693795+0000","last_fullsized":"2024-09-21T07:45:36.693795+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:47.074327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098384399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":247,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.679336+0000","last_change":"2024-09-21T07:34:24.260300+0000","last_active":"2024-09-21T07:45:36.679336+0000","last_peered":"2024-09-21T07:45:36.679336+0000","last_clean":"2024-09-21T07:45:36.679336+0000","last_became_active":"2024-09-21T07:34:23.270128+0000","last_became_peered":"2024-09-21T07:34:23.270128+0000","last_unstale":"2024-09-21T07:45:36.679336+0000","last_undegraded":"2024-09-21T07:45:36.679336+0000","last_fullsized":"2024-09-21T07:45:36.679336+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:25.260931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00176651,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-21T07:45:36.664838+0000","last_change":"2024-09-21T07:45:36.664838+0000","last_active":"2024-09-21T07:45:36.648974+0000","last_peered":"2024-09-21T07:45:36.648974+0000","last_clean":"2024-09-21T07:45:36.648974+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:45:36.664838+0000","last_undegraded":"2024-09-21T07:45:36.664838+0000","last_fullsized":"2024-09-21T07:45:36.664838+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":10,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.685247+0000","last_change":"2024-09-21T07:45:36.683495+0000","last_active":"2024-09-21T07:45:36.685247+0000","last_peered":"2024-09-21T07:45:36.685247+0000","last_clean":"2024-09-21T07:45:36.685247+0000","last_became_active":"2024-09-21T07:45:36.683165+0000","last_became_peered":"2024-09-21T07:45:36.683165+0000","last_unstale":"2024-09-21T07:45:36.685247+0000","last_undegraded":"2024-09-21T07:45:36.685247+0000","last_fullsized":"2024-09-21T07:45:36.685247+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:53:20.269457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":186,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.680954+0000","last_change":"2024-09-21T07:35:27.877861+0000","last_active":"2024-09-21T07:45:36.680954+0000","last_peered":"2024-09-21T07:45:36.680954+0000","last_clean":"2024-09-21T07:45:36.680954+0000","last_became_active":"2024-09-21T07:35:27.877434+0000","last_became_peered":"2024-09-21T07:35:27.877434+0000","last_unstale":"2024-09-21T07:45:36.680954+0000","last_undegraded":"2024-09-21T07:45:36.680954+0000","last_fullsized":"2024-09-21T07:45:36.680954+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":194,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.672375+0000","last_change":"2024-09-21T07:35:28.862954+0000","last_active":"2024-09-21T07:45:36.672375+0000","last_peered":"2024-09-21T07:45:36.672375+0000","last_clean":"2024-09-21T07:45:36.672375+0000","last_became_active":"2024-09-21T07:35:26.870613+0000","last_became_peered":"2024-09-21T07:35:26.870613+0000","last_unstale":"2024-09-21T07:45:36.672375+0000","last_undegraded":"2024-09-21T07:45:36.672375+0000","last_fullsized":"2024-09-21T07:45:36.672375+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:06.901043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059842100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":246,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-21T07:45:36.673328+0000","last_change":"2024-09-21T07:34:25.028904+0000","last_active":"2024-09-21T07:45:36.673328+0000","last_peered":"2024-09-21T07:45:36.673328+0000","last_clean":"2024-09-21T07:45:36.673328+0000","last_became_active":"2024-09-21T07:34:23.260942+0000","last_became_peered":"2024-09-21T07:34:23.260942+0000","last_unstale":"2024-09-21T07:45:36.673328+0000","last_undegraded":"2024-09-21T07:45:36.673328+0000","last_fullsized":"2024-09-21T07:45:36.673328+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:22.709328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060553899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509594,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37824,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334016,"statfs":{"total":96636764160,"available":96598032384,"internally_reserved":0,"allocated":2531328,"data_stored":1604054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509596,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37244,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334596,"statfs":{"total":96636764160,"available":96598626304,"internally_reserved":0,"allocated":1937408,"data_stored":1013686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509594,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37240,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334600,"statfs":{"total":96636764160,"available":96598630400,"internally_reserved":0,"allocated":1937408,"data_stored":1013686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509595,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37244,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334596,"statfs":{"total":96636764160,"available":96598626304,"internally_reserved":0,"allocated":1937408,"data_stored":1013686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509596,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37240,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334600,"statfs":{"total":96636764160,"available":96598630400,"internally_reserved":0,"allocated":1937408,"data_stored":1013686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509596,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37820,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334020,"statfs":{"total":96636764160,"available":96598036480,"internally_reserved":0,"allocated":2531328,"data_stored":1604054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542300,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37272,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334568,"statfs":{"total":96636764160,"available":96598597632,"internally_reserved":0,"allocated":1961984,"data_stored":1029031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542299,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41448,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330392,"statfs":{"total":96636764160,"available":96594321408,"internally_reserved":0,"allocated":1978368,"data_stored":1030713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:45:43.206 INFO:tasks.ceph.ceph_manager.ceph:PG 20.3f is not active+clean 2024-09-21T07:45:43.206 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.689192+0000', 'last_change': '2024-09-21T07:45:36.689192+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.689192+0000', 'last_undegraded': '2024-09-21T07:45:36.689192+0000', 'last_fullsized': '2024-09-21T07:45:36.689192+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T07:45:43.206 INFO:tasks.ceph.ceph_manager.ceph:PG 20.3e is not active+clean 2024-09-21T07:45:43.206 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.665560+0000', 'last_change': '2024-09-21T07:45:36.665560+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.665560+0000', 'last_undegraded': '2024-09-21T07:45:36.665560+0000', 'last_fullsized': '2024-09-21T07:45:36.665560+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T07:45:43.207 INFO:tasks.ceph.ceph_manager.ceph:PG 20.3d is not active+clean 2024-09-21T07:45:43.207 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.675392+0000', 'last_change': '2024-09-21T07:45:36.675392+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.675392+0000', 'last_undegraded': '2024-09-21T07:45:36.675392+0000', 'last_fullsized': '2024-09-21T07:45:36.675392+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T07:45:43.207 INFO:tasks.ceph.ceph_manager.ceph:PG 20.3c is not active+clean 2024-09-21T07:45:43.207 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.664845+0000', 'last_change': '2024-09-21T07:45:36.664845+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.664845+0000', 'last_undegraded': '2024-09-21T07:45:36.664845+0000', 'last_fullsized': '2024-09-21T07:45:36.664845+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T07:45:43.207 INFO:tasks.ceph.ceph_manager.ceph:PG 20.3a is not active+clean 2024-09-21T07:45:43.207 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.676932+0000', 'last_change': '2024-09-21T07:45:36.676932+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.676932+0000', 'last_undegraded': '2024-09-21T07:45:36.676932+0000', 'last_fullsized': '2024-09-21T07:45:36.676932+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T07:45:43.207 INFO:tasks.ceph.ceph_manager.ceph:PG 20.39 is not active+clean 2024-09-21T07:45:43.207 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.674875+0000', 'last_change': '2024-09-21T07:45:36.674875+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.674875+0000', 'last_undegraded': '2024-09-21T07:45:36.674875+0000', 'last_fullsized': '2024-09-21T07:45:36.674875+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T07:45:43.207 INFO:tasks.ceph.ceph_manager.ceph:PG 20.37 is not active+clean 2024-09-21T07:45:43.208 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.677810+0000', 'last_change': '2024-09-21T07:45:36.677810+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.677810+0000', 'last_undegraded': '2024-09-21T07:45:36.677810+0000', 'last_fullsized': '2024-09-21T07:45:36.677810+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T07:45:43.208 INFO:tasks.ceph.ceph_manager.ceph:PG 20.36 is not active+clean 2024-09-21T07:45:43.208 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.680253+0000', 'last_change': '2024-09-21T07:45:36.680253+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.680253+0000', 'last_undegraded': '2024-09-21T07:45:36.680253+0000', 'last_fullsized': '2024-09-21T07:45:36.680253+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T07:45:43.208 INFO:tasks.ceph.ceph_manager.ceph:PG 20.35 is not active+clean 2024-09-21T07:45:43.208 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.664838+0000', 'last_change': '2024-09-21T07:45:36.664838+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.664838+0000', 'last_undegraded': '2024-09-21T07:45:36.664838+0000', 'last_fullsized': '2024-09-21T07:45:36.664838+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T07:45:43.208 INFO:tasks.ceph.ceph_manager.ceph:PG 20.34 is not active+clean 2024-09-21T07:45:43.208 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.688646+0000', 'last_change': '2024-09-21T07:45:36.688646+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.688646+0000', 'last_undegraded': '2024-09-21T07:45:36.688646+0000', 'last_fullsized': '2024-09-21T07:45:36.688646+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T07:45:43.208 INFO:tasks.ceph.ceph_manager.ceph:PG 20.33 is not active+clean 2024-09-21T07:45:43.208 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.676713+0000', 'last_change': '2024-09-21T07:45:36.676713+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.676713+0000', 'last_undegraded': '2024-09-21T07:45:36.676713+0000', 'last_fullsized': '2024-09-21T07:45:36.676713+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T07:45:43.208 INFO:tasks.ceph.ceph_manager.ceph:PG 20.32 is not active+clean 2024-09-21T07:45:43.209 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.687739+0000', 'last_change': '2024-09-21T07:45:36.687739+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.687739+0000', 'last_undegraded': '2024-09-21T07:45:36.687739+0000', 'last_fullsized': '2024-09-21T07:45:36.687739+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T07:45:43.209 INFO:tasks.ceph.ceph_manager.ceph:PG 20.31 is not active+clean 2024-09-21T07:45:43.209 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.684481+0000', 'last_change': '2024-09-21T07:45:36.684481+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.684481+0000', 'last_undegraded': '2024-09-21T07:45:36.684481+0000', 'last_fullsized': '2024-09-21T07:45:36.684481+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T07:45:43.209 INFO:tasks.ceph.ceph_manager.ceph:PG 20.30 is not active+clean 2024-09-21T07:45:43.209 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.677447+0000', 'last_change': '2024-09-21T07:45:36.677447+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.677447+0000', 'last_undegraded': '2024-09-21T07:45:36.677447+0000', 'last_fullsized': '2024-09-21T07:45:36.677447+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T07:45:43.209 INFO:tasks.ceph.ceph_manager.ceph:PG 20.2f is not active+clean 2024-09-21T07:45:43.209 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.677254+0000', 'last_change': '2024-09-21T07:45:36.677254+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.677254+0000', 'last_undegraded': '2024-09-21T07:45:36.677254+0000', 'last_fullsized': '2024-09-21T07:45:36.677254+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T07:45:43.209 INFO:tasks.ceph.ceph_manager.ceph:PG 20.2d is not active+clean 2024-09-21T07:45:43.209 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.675086+0000', 'last_change': '2024-09-21T07:45:36.675086+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.675086+0000', 'last_undegraded': '2024-09-21T07:45:36.675086+0000', 'last_fullsized': '2024-09-21T07:45:36.675086+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T07:45:43.209 INFO:tasks.ceph.ceph_manager.ceph:PG 20.2b is not active+clean 2024-09-21T07:45:43.209 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.664937+0000', 'last_change': '2024-09-21T07:45:36.664937+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.664937+0000', 'last_undegraded': '2024-09-21T07:45:36.664937+0000', 'last_fullsized': '2024-09-21T07:45:36.664937+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T07:45:43.210 INFO:tasks.ceph.ceph_manager.ceph:PG 20.2a is not active+clean 2024-09-21T07:45:43.210 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.691525+0000', 'last_change': '2024-09-21T07:45:36.691525+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.691525+0000', 'last_undegraded': '2024-09-21T07:45:36.691525+0000', 'last_fullsized': '2024-09-21T07:45:36.691525+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T07:45:43.210 INFO:tasks.ceph.ceph_manager.ceph:PG 20.29 is not active+clean 2024-09-21T07:45:43.210 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.666008+0000', 'last_change': '2024-09-21T07:45:36.666008+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.666008+0000', 'last_undegraded': '2024-09-21T07:45:36.666008+0000', 'last_fullsized': '2024-09-21T07:45:36.666008+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T07:45:43.210 INFO:tasks.ceph.ceph_manager.ceph:PG 20.28 is not active+clean 2024-09-21T07:45:43.210 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.676435+0000', 'last_change': '2024-09-21T07:45:36.676435+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.676435+0000', 'last_undegraded': '2024-09-21T07:45:36.676435+0000', 'last_fullsized': '2024-09-21T07:45:36.676435+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T07:45:43.210 INFO:tasks.ceph.ceph_manager.ceph:PG 20.27 is not active+clean 2024-09-21T07:45:43.210 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.675973+0000', 'last_change': '2024-09-21T07:45:36.675973+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.675973+0000', 'last_undegraded': '2024-09-21T07:45:36.675973+0000', 'last_fullsized': '2024-09-21T07:45:36.675973+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T07:45:43.210 INFO:tasks.ceph.ceph_manager.ceph:PG 20.26 is not active+clean 2024-09-21T07:45:43.211 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.671245+0000', 'last_change': '2024-09-21T07:45:36.671245+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.671245+0000', 'last_undegraded': '2024-09-21T07:45:36.671245+0000', 'last_fullsized': '2024-09-21T07:45:36.671245+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T07:45:43.211 INFO:tasks.ceph.ceph_manager.ceph:PG 20.25 is not active+clean 2024-09-21T07:45:43.211 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.671879+0000', 'last_change': '2024-09-21T07:45:36.671879+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.671879+0000', 'last_undegraded': '2024-09-21T07:45:36.671879+0000', 'last_fullsized': '2024-09-21T07:45:36.671879+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T07:45:43.211 INFO:tasks.ceph.ceph_manager.ceph:PG 20.24 is not active+clean 2024-09-21T07:45:43.211 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.681606+0000', 'last_change': '2024-09-21T07:45:36.681606+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.681606+0000', 'last_undegraded': '2024-09-21T07:45:36.681606+0000', 'last_fullsized': '2024-09-21T07:45:36.681606+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T07:45:43.211 INFO:tasks.ceph.ceph_manager.ceph:PG 20.23 is not active+clean 2024-09-21T07:45:43.211 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.676579+0000', 'last_change': '2024-09-21T07:45:36.676579+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.676579+0000', 'last_undegraded': '2024-09-21T07:45:36.676579+0000', 'last_fullsized': '2024-09-21T07:45:36.676579+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T07:45:43.211 INFO:tasks.ceph.ceph_manager.ceph:PG 20.22 is not active+clean 2024-09-21T07:45:43.211 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.675479+0000', 'last_change': '2024-09-21T07:45:36.675479+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.675479+0000', 'last_undegraded': '2024-09-21T07:45:36.675479+0000', 'last_fullsized': '2024-09-21T07:45:36.675479+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T07:45:43.211 INFO:tasks.ceph.ceph_manager.ceph:PG 20.20 is not active+clean 2024-09-21T07:45:43.212 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.681462+0000', 'last_change': '2024-09-21T07:45:36.681462+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.681462+0000', 'last_undegraded': '2024-09-21T07:45:36.681462+0000', 'last_fullsized': '2024-09-21T07:45:36.681462+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T07:45:43.212 INFO:tasks.ceph.ceph_manager.ceph:PG 20.15 is not active+clean 2024-09-21T07:45:43.212 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.671410+0000', 'last_change': '2024-09-21T07:45:36.671410+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.671410+0000', 'last_undegraded': '2024-09-21T07:45:36.671410+0000', 'last_fullsized': '2024-09-21T07:45:36.671410+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T07:45:43.212 INFO:tasks.ceph.ceph_manager.ceph:PG 20.10 is not active+clean 2024-09-21T07:45:43.212 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.686033+0000', 'last_change': '2024-09-21T07:45:36.686033+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.686033+0000', 'last_undegraded': '2024-09-21T07:45:36.686033+0000', 'last_fullsized': '2024-09-21T07:45:36.686033+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T07:45:43.212 INFO:tasks.ceph.ceph_manager.ceph:PG 20.11 is not active+clean 2024-09-21T07:45:43.212 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.676108+0000', 'last_change': '2024-09-21T07:45:36.676108+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.676108+0000', 'last_undegraded': '2024-09-21T07:45:36.676108+0000', 'last_fullsized': '2024-09-21T07:45:36.676108+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T07:45:43.213 INFO:tasks.ceph.ceph_manager.ceph:PG 20.12 is not active+clean 2024-09-21T07:45:43.213 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.670461+0000', 'last_change': '2024-09-21T07:45:36.670461+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.670461+0000', 'last_undegraded': '2024-09-21T07:45:36.670461+0000', 'last_fullsized': '2024-09-21T07:45:36.670461+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T07:45:43.213 INFO:tasks.ceph.ceph_manager.ceph:PG 20.13 is not active+clean 2024-09-21T07:45:43.213 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.675896+0000', 'last_change': '2024-09-21T07:45:36.675896+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.675896+0000', 'last_undegraded': '2024-09-21T07:45:36.675896+0000', 'last_fullsized': '2024-09-21T07:45:36.675896+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T07:45:43.213 INFO:tasks.ceph.ceph_manager.ceph:PG 20.14 is not active+clean 2024-09-21T07:45:43.213 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.676454+0000', 'last_change': '2024-09-21T07:45:36.676454+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.676454+0000', 'last_undegraded': '2024-09-21T07:45:36.676454+0000', 'last_fullsized': '2024-09-21T07:45:36.676454+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T07:45:43.213 INFO:tasks.ceph.ceph_manager.ceph:PG 20.16 is not active+clean 2024-09-21T07:45:43.213 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.674849+0000', 'last_change': '2024-09-21T07:45:36.674849+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.674849+0000', 'last_undegraded': '2024-09-21T07:45:36.674849+0000', 'last_fullsized': '2024-09-21T07:45:36.674849+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T07:45:43.213 INFO:tasks.ceph.ceph_manager.ceph:PG 20.19 is not active+clean 2024-09-21T07:45:43.213 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.683953+0000', 'last_change': '2024-09-21T07:45:36.683953+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.683953+0000', 'last_undegraded': '2024-09-21T07:45:36.683953+0000', 'last_fullsized': '2024-09-21T07:45:36.683953+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T07:45:43.214 INFO:tasks.ceph.ceph_manager.ceph:PG 20.1a is not active+clean 2024-09-21T07:45:43.214 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.664831+0000', 'last_change': '2024-09-21T07:45:36.664831+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.664831+0000', 'last_undegraded': '2024-09-21T07:45:36.664831+0000', 'last_fullsized': '2024-09-21T07:45:36.664831+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T07:45:43.214 INFO:tasks.ceph.ceph_manager.ceph:PG 20.1b is not active+clean 2024-09-21T07:45:43.214 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.692417+0000', 'last_change': '2024-09-21T07:45:36.692417+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.692417+0000', 'last_undegraded': '2024-09-21T07:45:36.692417+0000', 'last_fullsized': '2024-09-21T07:45:36.692417+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T07:45:43.214 INFO:tasks.ceph.ceph_manager.ceph:PG 20.1f is not active+clean 2024-09-21T07:45:43.214 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.672205+0000', 'last_change': '2024-09-21T07:45:36.672205+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.672205+0000', 'last_undegraded': '2024-09-21T07:45:36.672205+0000', 'last_fullsized': '2024-09-21T07:45:36.672205+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T07:45:43.214 INFO:tasks.ceph.ceph_manager.ceph:PG 20.1 is not active+clean 2024-09-21T07:45:43.214 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.671064+0000', 'last_change': '2024-09-21T07:45:36.671064+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.671064+0000', 'last_undegraded': '2024-09-21T07:45:36.671064+0000', 'last_fullsized': '2024-09-21T07:45:36.671064+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T07:45:43.214 INFO:tasks.ceph.ceph_manager.ceph:PG 20.2 is not active+clean 2024-09-21T07:45:43.214 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.675060+0000', 'last_change': '2024-09-21T07:45:36.675060+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.675060+0000', 'last_undegraded': '2024-09-21T07:45:36.675060+0000', 'last_fullsized': '2024-09-21T07:45:36.675060+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T07:45:43.214 INFO:tasks.ceph.ceph_manager.ceph:PG 20.3 is not active+clean 2024-09-21T07:45:43.215 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.677283+0000', 'last_change': '2024-09-21T07:45:36.677283+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.677283+0000', 'last_undegraded': '2024-09-21T07:45:36.677283+0000', 'last_fullsized': '2024-09-21T07:45:36.677283+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T07:45:43.215 INFO:tasks.ceph.ceph_manager.ceph:PG 20.4 is not active+clean 2024-09-21T07:45:43.215 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.676888+0000', 'last_change': '2024-09-21T07:45:36.676888+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.676888+0000', 'last_undegraded': '2024-09-21T07:45:36.676888+0000', 'last_fullsized': '2024-09-21T07:45:36.676888+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T07:45:43.215 INFO:tasks.ceph.ceph_manager.ceph:PG 20.5 is not active+clean 2024-09-21T07:45:43.215 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.676058+0000', 'last_change': '2024-09-21T07:45:36.676058+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.676058+0000', 'last_undegraded': '2024-09-21T07:45:36.676058+0000', 'last_fullsized': '2024-09-21T07:45:36.676058+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T07:45:43.215 INFO:tasks.ceph.ceph_manager.ceph:PG 20.6 is not active+clean 2024-09-21T07:45:43.215 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.676259+0000', 'last_change': '2024-09-21T07:45:36.676259+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.676259+0000', 'last_undegraded': '2024-09-21T07:45:36.676259+0000', 'last_fullsized': '2024-09-21T07:45:36.676259+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T07:45:43.215 INFO:tasks.ceph.ceph_manager.ceph:PG 20.7 is not active+clean 2024-09-21T07:45:43.215 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.673113+0000', 'last_change': '2024-09-21T07:45:36.673113+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.673113+0000', 'last_undegraded': '2024-09-21T07:45:36.673113+0000', 'last_fullsized': '2024-09-21T07:45:36.673113+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T07:45:43.215 INFO:tasks.ceph.ceph_manager.ceph:PG 20.8 is not active+clean 2024-09-21T07:45:43.215 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.673352+0000', 'last_change': '2024-09-21T07:45:36.673352+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.673352+0000', 'last_undegraded': '2024-09-21T07:45:36.673352+0000', 'last_fullsized': '2024-09-21T07:45:36.673352+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T07:45:43.215 INFO:tasks.ceph.ceph_manager.ceph:PG 20.9 is not active+clean 2024-09-21T07:45:43.216 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.670174+0000', 'last_change': '2024-09-21T07:45:36.670174+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.670174+0000', 'last_undegraded': '2024-09-21T07:45:36.670174+0000', 'last_fullsized': '2024-09-21T07:45:36.670174+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T07:45:43.216 INFO:tasks.ceph.ceph_manager.ceph:PG 20.a is not active+clean 2024-09-21T07:45:43.216 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.673682+0000', 'last_change': '2024-09-21T07:45:36.673682+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.673682+0000', 'last_undegraded': '2024-09-21T07:45:36.673682+0000', 'last_fullsized': '2024-09-21T07:45:36.673682+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T07:45:43.216 INFO:tasks.ceph.ceph_manager.ceph:PG 20.b is not active+clean 2024-09-21T07:45:43.216 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.677740+0000', 'last_change': '2024-09-21T07:45:36.677740+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.677740+0000', 'last_undegraded': '2024-09-21T07:45:36.677740+0000', 'last_fullsized': '2024-09-21T07:45:36.677740+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T07:45:43.216 INFO:tasks.ceph.ceph_manager.ceph:PG 20.d is not active+clean 2024-09-21T07:45:43.216 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.689181+0000', 'last_change': '2024-09-21T07:45:36.689181+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.689181+0000', 'last_undegraded': '2024-09-21T07:45:36.689181+0000', 'last_fullsized': '2024-09-21T07:45:36.689181+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T07:45:43.216 INFO:tasks.ceph.ceph_manager.ceph:PG 20.f is not active+clean 2024-09-21T07:45:43.216 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '20.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 155, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:45:36.664838+0000', 'last_change': '2024-09-21T07:45:36.664838+0000', 'last_active': '2024-09-21T07:45:36.648974+0000', 'last_peered': '2024-09-21T07:45:36.648974+0000', 'last_clean': '2024-09-21T07:45:36.648974+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:45:36.664838+0000', 'last_undegraded': '2024-09-21T07:45:36.664838+0000', 'last_fullsized': '2024-09-21T07:45:36.664838+0000', 'mapping_epoch': 155, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 155, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'last_clean_scrub_stamp': '2024-09-21T07:45:36.648974+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T07:45:43.217 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T07:45:43.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.459+0000 7fd5b3800640 1 -- 172.21.15.39:0/3028661167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5ac0d0b70 msgr2=0x7fd5ac0b43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:43.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.459+0000 7fd5b3800640 1 --2- 172.21.15.39:0/3028661167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5ac0d0b70 0x7fd5ac0b43d0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac06dd90 tx=0x7fd5a402a830 comp rx=0 tx=0).stop 2024-09-21T07:45:43.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.459+0000 7fd5b3800640 1 -- 172.21.15.39:0/3028661167 shutdown_connections 2024-09-21T07:45:43.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.459+0000 7fd5b3800640 1 --2- 172.21.15.39:0/3028661167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5ac0d0b70 0x7fd5ac0b43d0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:43.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.459+0000 7fd5b3800640 1 --2- 172.21.15.39:0/3028661167 >> v2:172.21.15.148:3300/0 conn(0x7fd5ac15c690 0x7fd5ac0d0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:43.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.459+0000 7fd5b3800640 1 -- 172.21.15.39:0/3028661167 >> 172.21.15.39:0/3028661167 conn(0x7fd5ac06cfc0 msgr2=0x7fd5ac06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:43.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.459+0000 7fd5b3800640 1 -- 172.21.15.39:0/3028661167 shutdown_connections 2024-09-21T07:45:43.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.459+0000 7fd5b3800640 1 -- 172.21.15.39:0/3028661167 wait complete. 2024-09-21T07:45:43.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.461+0000 7fd5b3800640 1 Processor -- start 2024-09-21T07:45:43.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.461+0000 7fd5b3800640 1 -- start start 2024-09-21T07:45:43.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.463+0000 7fd5b3800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5ac0b5240 0x7fd5ac0af240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:43.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.463+0000 7fd5b3800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5ac0d0b70 0x7fd5ac0b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:43.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.463+0000 7fd5b3800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5ac15c690 0x7fd5ac0ae820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:43.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.463+0000 7fd5b3800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac070090 con 0x7fd5ac0d0b70 2024-09-21T07:45:43.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.463+0000 7fd5b3800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac06ff10 con 0x7fd5ac15c690 2024-09-21T07:45:43.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.463+0000 7fd5b3800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd5ac0b4910 con 0x7fd5ac0b5240 2024-09-21T07:45:43.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.463+0000 7fd5abe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5ac0d0b70 0x7fd5ac0b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:43.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.463+0000 7fd5abe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5ac0d0b70 0x7fd5ac0b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53866/0 (socket says 172.21.15.39:53866) 2024-09-21T07:45:43.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.463+0000 7fd5abe00640 1 -- 172.21.15.39:0/1382559369 learned_addr learned my addr 172.21.15.39:0/1382559369 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:43.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.463+0000 7fd5b1800640 1 --2- 172.21.15.39:0/1382559369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5ac15c690 0x7fd5ac0ae820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:43.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.463+0000 7fd5b0e00640 1 --2- 172.21.15.39:0/1382559369 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5ac0b5240 0x7fd5ac0af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:43.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.463+0000 7fd5abe00640 1 -- 172.21.15.39:0/1382559369 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5ac0b5240 msgr2=0x7fd5ac0af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:43.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.463+0000 7fd5abe00640 1 --2- 172.21.15.39:0/1382559369 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5ac0b5240 0x7fd5ac0af240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:43.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.463+0000 7fd5abe00640 1 -- 172.21.15.39:0/1382559369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5ac15c690 msgr2=0x7fd5ac0ae820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:43.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.463+0000 7fd5abe00640 1 --2- 172.21.15.39:0/1382559369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5ac15c690 0x7fd5ac0ae820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:43.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.463+0000 7fd5abe00640 1 -- 172.21.15.39:0/1382559369 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5ac0ada60 con 0x7fd5ac0d0b70 2024-09-21T07:45:43.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.464+0000 7fd5abe00640 1 --2- 172.21.15.39:0/1382559369 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5ac0d0b70 0x7fd5ac0b1e80 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fd59800d740 tx=0x7fd59800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:43.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.464+0000 7fd5a8c00640 1 -- 172.21.15.39:0/1382559369 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd598017070 con 0x7fd5ac0d0b70 2024-09-21T07:45:43.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.464+0000 7fd5a8c00640 1 -- 172.21.15.39:0/1382559369 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd598009d20 con 0x7fd5ac0d0b70 2024-09-21T07:45:43.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.464+0000 7fd5a8c00640 1 -- 172.21.15.39:0/1382559369 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5980106c0 con 0x7fd5ac0d0b70 2024-09-21T07:45:43.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.464+0000 7fd5b3800640 1 -- 172.21.15.39:0/1382559369 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5ac0add40 con 0x7fd5ac0d0b70 2024-09-21T07:45:43.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.464+0000 7fd5b3800640 1 -- 172.21.15.39:0/1382559369 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5ac144160 con 0x7fd5ac0d0b70 2024-09-21T07:45:43.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.465+0000 7fd5b3800640 1 -- 172.21.15.39:0/1382559369 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5ac0af800 con 0x7fd5ac0d0b70 2024-09-21T07:45:43.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.466+0000 7fd5a8c00640 1 -- 172.21.15.39:0/1382559369 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd59801a070 con 0x7fd5ac0d0b70 2024-09-21T07:45:43.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.467+0000 7fd5a8c00640 1 --2- 172.21.15.39:0/1382559369 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd58c078bc0 0x7fd58c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:43.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.468+0000 7fd5a8c00640 1 -- 172.21.15.39:0/1382559369 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 6844+0+0 (secure 0 0 0) 0x7fd59809b1f0 con 0x7fd5ac0d0b70 2024-09-21T07:45:43.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.468+0000 7fd5b0e00640 1 --2- 172.21.15.39:0/1382559369 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd58c078bc0 0x7fd58c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:43.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.469+0000 7fd5b0e00640 1 --2- 172.21.15.39:0/1382559369 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd58c078bc0 0x7fd58c07b0d0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0001410 tx=0x7fd5a0009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:43.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.472+0000 7fd5a8c00640 1 -- 172.21.15.39:0/1382559369 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd598062b70 con 0x7fd5ac0d0b70 2024-09-21T07:45:43.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.745+0000 7fd5b3800640 1 -- 172.21.15.39:0/1382559369 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fd5ac0b4e50 con 0x7fd5ac0d0b70 2024-09-21T07:45:43.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.745+0000 7fd5a8c00640 1 -- 172.21.15.39:0/1382559369 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1303 (secure 0 0 0) 0x7fd5980675a0 con 0x7fd5ac0d0b70 2024-09-21T07:45:43.747 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:45:43.748 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":777,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":157,"num_osds":8,"num_up_osds":8,"osd_up_since":1726903982,"num_in_osds":8,"osd_in_since":1726903972,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":142},{"state_name":"creating+peering","count":51}],"num_pgs":193,"num_pools":6,"num_objects":4,"data_bytes":590368,"bytes_used":310611968,"bytes_avail":772783501312,"bytes_total":773094113280,"inactive_pgs_ratio":0.26424869894981384,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":181,"btime":"2024-09-21T07:45:40:695099+0000","id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":8829}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-09-21T07:44:05.755620+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-21T07:45:43.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.748+0000 7fd5b3800640 1 -- 172.21.15.39:0/1382559369 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd58c078bc0 msgr2=0x7fd58c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:43.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.748+0000 7fd5b3800640 1 --2- 172.21.15.39:0/1382559369 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd58c078bc0 0x7fd58c07b0d0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0001410 tx=0x7fd5a0009040 comp rx=0 tx=0).stop 2024-09-21T07:45:43.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.748+0000 7fd5b3800640 1 -- 172.21.15.39:0/1382559369 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5ac0d0b70 msgr2=0x7fd5ac0b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:43.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.748+0000 7fd5b3800640 1 --2- 172.21.15.39:0/1382559369 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5ac0d0b70 0x7fd5ac0b1e80 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fd59800d740 tx=0x7fd59800dc00 comp rx=0 tx=0).stop 2024-09-21T07:45:43.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.748+0000 7fd5b3800640 1 -- 172.21.15.39:0/1382559369 shutdown_connections 2024-09-21T07:45:43.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.748+0000 7fd5b3800640 1 --2- 172.21.15.39:0/1382559369 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd58c078bc0 0x7fd58c07b0d0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:43.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.748+0000 7fd5b3800640 1 --2- 172.21.15.39:0/1382559369 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5ac15c690 0x7fd5ac0ae820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:43.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.748+0000 7fd5b3800640 1 --2- 172.21.15.39:0/1382559369 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5ac0d0b70 0x7fd5ac0b1e80 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:43.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.748+0000 7fd5b3800640 1 --2- 172.21.15.39:0/1382559369 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5ac0b5240 0x7fd5ac0af240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:43.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.748+0000 7fd5b3800640 1 -- 172.21.15.39:0/1382559369 >> 172.21.15.39:0/1382559369 conn(0x7fd5ac06cfc0 msgr2=0x7fd5ac0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:43.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.748+0000 7fd5b3800640 1 -- 172.21.15.39:0/1382559369 shutdown_connections 2024-09-21T07:45:43.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:43.748+0000 7fd5b3800640 1 -- 172.21.15.39:0/1382559369 wait complete. 2024-09-21T07:45:43.774 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-21T07:45:43.775 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:45:44.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.003+0000 7fb256e00640 1 -- 172.21.15.39:0/3781298704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb238003fd0 msgr2=0x7fb238004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:44.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.003+0000 7fb256e00640 1 --2- 172.21.15.39:0/3781298704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb238003fd0 0x7fb238004800 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fb23c009b50 tx=0x7fb238005830 comp rx=0 tx=0).stop 2024-09-21T07:45:44.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.003+0000 7fb256400640 1 -- 172.21.15.39:0/3781298704 reap_dead start 2024-09-21T07:45:44.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.003+0000 7fb256e00640 1 -- 172.21.15.39:0/3781298704 shutdown_connections 2024-09-21T07:45:44.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.003+0000 7fb256e00640 1 -- 172.21.15.39:0/3781298704 >> 172.21.15.39:0/3781298704 conn(0x7fb25006cfc0 msgr2=0x7fb25006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:44.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.003+0000 7fb256e00640 1 -- 172.21.15.39:0/3781298704 shutdown_connections 2024-09-21T07:45:44.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.003+0000 7fb256e00640 1 -- 172.21.15.39:0/3781298704 wait complete. 2024-09-21T07:45:44.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.006+0000 7fb256e00640 1 Processor -- start 2024-09-21T07:45:44.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.006+0000 7fb256e00640 1 -- start start 2024-09-21T07:45:44.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.007+0000 7fb256e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb238003fd0 0x7fb2501f9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:44.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.007+0000 7fb256e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb250152b00 0x7fb2501522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:44.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.007+0000 7fb256e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2500b3140 0x7fb2500b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:44.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.007+0000 7fb256e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb25015da90 con 0x7fb2500b3140 2024-09-21T07:45:44.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.007+0000 7fb256e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb25015d910 con 0x7fb238003fd0 2024-09-21T07:45:44.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.007+0000 7fb256e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb25015dc10 con 0x7fb250152b00 2024-09-21T07:45:44.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.007+0000 7fb256400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2500b3140 0x7fb2500b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:44.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.007+0000 7fb256400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2500b3140 0x7fb2500b3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53872/0 (socket says 172.21.15.39:53872) 2024-09-21T07:45:44.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.007+0000 7fb256400640 1 -- 172.21.15.39:0/2614565097 learned_addr learned my addr 172.21.15.39:0/2614565097 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:44.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.008+0000 7fb255a00640 1 --2- 172.21.15.39:0/2614565097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb238003fd0 0x7fb2501f9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:44.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.008+0000 7fb255000640 1 --2- 172.21.15.39:0/2614565097 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb250152b00 0x7fb2501522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:44.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.008+0000 7fb256400640 1 -- 172.21.15.39:0/2614565097 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb250152b00 msgr2=0x7fb2501522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:44.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.008+0000 7fb256400640 1 --2- 172.21.15.39:0/2614565097 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb250152b00 0x7fb2501522a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:44.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.008+0000 7fb256400640 1 -- 172.21.15.39:0/2614565097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb238003fd0 msgr2=0x7fb2501f9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:44.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.008+0000 7fb256400640 1 --2- 172.21.15.39:0/2614565097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb238003fd0 0x7fb2501f9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:44.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.008+0000 7fb256400640 1 -- 172.21.15.39:0/2614565097 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb25013ca90 con 0x7fb2500b3140 2024-09-21T07:45:44.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.008+0000 7fb256400640 1 --2- 172.21.15.39:0/2614565097 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2500b3140 0x7fb2500b3550 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fb2500cdd50 tx=0x7fb24800ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:44.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.008+0000 7fb24d600640 1 -- 172.21.15.39:0/2614565097 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb248010890 con 0x7fb2500b3140 2024-09-21T07:45:44.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.009+0000 7fb24d600640 1 -- 172.21.15.39:0/2614565097 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb248018070 con 0x7fb2500b3140 2024-09-21T07:45:44.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.009+0000 7fb256e00640 1 -- 172.21.15.39:0/2614565097 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb25013d2a0 con 0x7fb2500b3140 2024-09-21T07:45:44.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.009+0000 7fb24d600640 1 -- 172.21.15.39:0/2614565097 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb248010de0 con 0x7fb2500b3140 2024-09-21T07:45:44.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.009+0000 7fb256e00640 1 -- 172.21.15.39:0/2614565097 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2501f92c0 con 0x7fb2500b3140 2024-09-21T07:45:44.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.010+0000 7fb256e00640 1 -- 172.21.15.39:0/2614565097 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2500cf710 con 0x7fb2500b3140 2024-09-21T07:45:44.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.011+0000 7fb24d600640 1 -- 172.21.15.39:0/2614565097 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb24801d460 con 0x7fb2500b3140 2024-09-21T07:45:44.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.012+0000 7fb24d600640 1 --2- 172.21.15.39:0/2614565097 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb230078b90 0x7fb23007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:44.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.013+0000 7fb24d600640 1 -- 172.21.15.39:0/2614565097 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 6844+0+0 (secure 0 0 0) 0x7fb24809b4f0 con 0x7fb2500b3140 2024-09-21T07:45:44.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.013+0000 7fb255a00640 1 --2- 172.21.15.39:0/2614565097 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb230078b90 0x7fb23007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:44.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.014+0000 7fb255a00640 1 --2- 172.21.15.39:0/2614565097 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb230078b90 0x7fb23007b0a0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fb244009bb0 tx=0x7fb244005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:44.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.017+0000 7fb24d600640 1 -- 172.21.15.39:0/2614565097 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb248063420 con 0x7fb2500b3140 2024-09-21T07:45:44.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.214+0000 7fb256e00640 1 -- 172.21.15.39:0/2614565097 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb25013d7c0 con 0x7fb230078b90 2024-09-21T07:45:44.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.223+0000 7fb24d600640 1 -- 172.21.15.39:0/2614565097 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+495245 (secure 0 0 0) 0x7fb244099ea0 con 0x7fb230078b90 2024-09-21T07:45:44.232 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:45:44.235 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:45:44.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.228+0000 7fb256e00640 1 -- 172.21.15.39:0/2614565097 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb230078b90 msgr2=0x7fb23007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:44.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.229+0000 7fb256e00640 1 --2- 172.21.15.39:0/2614565097 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb230078b90 0x7fb23007b0a0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fb244009bb0 tx=0x7fb244005f90 comp rx=0 tx=0).stop 2024-09-21T07:45:44.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.229+0000 7fb256e00640 1 -- 172.21.15.39:0/2614565097 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2500b3140 msgr2=0x7fb2500b3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:44.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.229+0000 7fb256e00640 1 --2- 172.21.15.39:0/2614565097 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2500b3140 0x7fb2500b3550 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fb2500cdd50 tx=0x7fb24800ea20 comp rx=0 tx=0).stop 2024-09-21T07:45:44.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.229+0000 7fb256e00640 1 -- 172.21.15.39:0/2614565097 shutdown_connections 2024-09-21T07:45:44.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.229+0000 7fb256e00640 1 --2- 172.21.15.39:0/2614565097 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb230078b90 0x7fb23007b0a0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:44.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.229+0000 7fb256e00640 1 --2- 172.21.15.39:0/2614565097 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2500b3140 0x7fb2500b3550 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:44.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.229+0000 7fb256e00640 1 --2- 172.21.15.39:0/2614565097 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb250152b00 0x7fb2501522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:44.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.229+0000 7fb256e00640 1 --2- 172.21.15.39:0/2614565097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb238003fd0 0x7fb2501f9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:44.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.229+0000 7fb256e00640 1 -- 172.21.15.39:0/2614565097 >> 172.21.15.39:0/2614565097 conn(0x7fb25006cfc0 msgr2=0x7fb25006ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:44.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.229+0000 7fb256e00640 1 -- 172.21.15.39:0/2614565097 shutdown_connections 2024-09-21T07:45:44.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:44.229+0000 7fb256e00640 1 -- 172.21.15.39:0/2614565097 wait complete. 2024-09-21T07:45:44.252 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":533,"stamp":"2024-09-21T07:45:43.787646+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":379,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":303528,"kb_used_data":16552,"kb_used_omap":12,"kb_used_meta":286771,"kb_avail":754671192,"statfs":{"total":773094113280,"available":772783300608,"internally_reserved":0,"allocated":16949248,"data_stored":9417804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":293654064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.125972"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259902+0000","last_change":"2024-09-21T07:45:37.693693+0000","last_active":"2024-09-21T07:45:39.259902+0000","last_peered":"2024-09-21T07:45:39.259902+0000","last_clean":"2024-09-21T07:45:39.259902+0000","last_became_active":"2024-09-21T07:45:37.693371+0000","last_became_peered":"2024-09-21T07:45:37.693371+0000","last_unstale":"2024-09-21T07:45:39.259902+0000","last_undegraded":"2024-09-21T07:45:39.259902+0000","last_fullsized":"2024-09-21T07:45:39.259902+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:38.801281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.815832+0000","last_change":"2024-09-21T07:45:37.676938+0000","last_active":"2024-09-21T07:45:38.815832+0000","last_peered":"2024-09-21T07:45:38.815832+0000","last_clean":"2024-09-21T07:45:38.815832+0000","last_became_active":"2024-09-21T07:45:37.676598+0000","last_became_peered":"2024-09-21T07:45:37.676598+0000","last_unstale":"2024-09-21T07:45:38.815832+0000","last_undegraded":"2024-09-21T07:45:38.815832+0000","last_fullsized":"2024-09-21T07:45:38.815832+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:32.304761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259838+0000","last_change":"2024-09-21T07:45:37.697566+0000","last_active":"2024-09-21T07:45:39.259838+0000","last_peered":"2024-09-21T07:45:39.259838+0000","last_clean":"2024-09-21T07:45:39.259838+0000","last_became_active":"2024-09-21T07:45:37.697108+0000","last_became_peered":"2024-09-21T07:45:37.697108+0000","last_unstale":"2024-09-21T07:45:39.259838+0000","last_undegraded":"2024-09-21T07:45:39.259838+0000","last_fullsized":"2024-09-21T07:45:39.259838+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:32.684544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.815383+0000","last_change":"2024-09-21T07:45:37.693927+0000","last_active":"2024-09-21T07:45:38.815383+0000","last_peered":"2024-09-21T07:45:38.815383+0000","last_clean":"2024-09-21T07:45:38.815383+0000","last_became_active":"2024-09-21T07:45:37.691663+0000","last_became_peered":"2024-09-21T07:45:37.691663+0000","last_unstale":"2024-09-21T07:45:38.815383+0000","last_undegraded":"2024-09-21T07:45:38.815383+0000","last_fullsized":"2024-09-21T07:45:38.815383+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:49.194272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689578+0000","last_change":"2024-09-21T07:45:37.706582+0000","last_active":"2024-09-21T07:45:38.689578+0000","last_peered":"2024-09-21T07:45:38.689578+0000","last_clean":"2024-09-21T07:45:38.689578+0000","last_became_active":"2024-09-21T07:45:37.706243+0000","last_became_peered":"2024-09-21T07:45:37.706243+0000","last_unstale":"2024-09-21T07:45:38.689578+0000","last_undegraded":"2024-09-21T07:45:38.689578+0000","last_fullsized":"2024-09-21T07:45:38.689578+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:20:35.055835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.257960+0000","last_change":"2024-09-21T07:45:37.696514+0000","last_active":"2024-09-21T07:45:39.257960+0000","last_peered":"2024-09-21T07:45:39.257960+0000","last_clean":"2024-09-21T07:45:39.257960+0000","last_became_active":"2024-09-21T07:45:37.694092+0000","last_became_peered":"2024-09-21T07:45:37.694092+0000","last_unstale":"2024-09-21T07:45:39.257960+0000","last_undegraded":"2024-09-21T07:45:39.257960+0000","last_fullsized":"2024-09-21T07:45:39.257960+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:01:39.096631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688574+0000","last_change":"2024-09-21T07:45:37.695651+0000","last_active":"2024-09-21T07:45:38.688574+0000","last_peered":"2024-09-21T07:45:38.688574+0000","last_clean":"2024-09-21T07:45:38.688574+0000","last_became_active":"2024-09-21T07:45:37.695288+0000","last_became_peered":"2024-09-21T07:45:37.695288+0000","last_unstale":"2024-09-21T07:45:38.688574+0000","last_undegraded":"2024-09-21T07:45:38.688574+0000","last_fullsized":"2024-09-21T07:45:38.688574+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:05.514879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.687461+0000","last_change":"2024-09-21T07:45:37.694980+0000","last_active":"2024-09-21T07:45:38.687461+0000","last_peered":"2024-09-21T07:45:38.687461+0000","last_clean":"2024-09-21T07:45:38.687461+0000","last_became_active":"2024-09-21T07:45:37.689627+0000","last_became_peered":"2024-09-21T07:45:37.689627+0000","last_unstale":"2024-09-21T07:45:38.687461+0000","last_undegraded":"2024-09-21T07:45:38.687461+0000","last_fullsized":"2024-09-21T07:45:38.687461+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:32:36.418482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259474+0000","last_change":"2024-09-21T07:45:37.681699+0000","last_active":"2024-09-21T07:45:39.259474+0000","last_peered":"2024-09-21T07:45:39.259474+0000","last_clean":"2024-09-21T07:45:39.259474+0000","last_became_active":"2024-09-21T07:45:37.681397+0000","last_became_peered":"2024-09-21T07:45:37.681397+0000","last_unstale":"2024-09-21T07:45:39.259474+0000","last_undegraded":"2024-09-21T07:45:39.259474+0000","last_fullsized":"2024-09-21T07:45:39.259474+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:39:14.136174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.830238+0000","last_change":"2024-09-21T07:45:37.693068+0000","last_active":"2024-09-21T07:45:38.830238+0000","last_peered":"2024-09-21T07:45:38.830238+0000","last_clean":"2024-09-21T07:45:38.830238+0000","last_became_active":"2024-09-21T07:45:37.692707+0000","last_became_peered":"2024-09-21T07:45:37.692707+0000","last_unstale":"2024-09-21T07:45:38.830238+0000","last_undegraded":"2024-09-21T07:45:38.830238+0000","last_fullsized":"2024-09-21T07:45:38.830238+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:23.625237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.815466+0000","last_change":"2024-09-21T07:45:37.693691+0000","last_active":"2024-09-21T07:45:38.815466+0000","last_peered":"2024-09-21T07:45:38.815466+0000","last_clean":"2024-09-21T07:45:38.815466+0000","last_became_active":"2024-09-21T07:45:37.693332+0000","last_became_peered":"2024-09-21T07:45:37.693332+0000","last_unstale":"2024-09-21T07:45:38.815466+0000","last_undegraded":"2024-09-21T07:45:38.815466+0000","last_fullsized":"2024-09-21T07:45:38.815466+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:38.976486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258353+0000","last_change":"2024-09-21T07:45:37.708847+0000","last_active":"2024-09-21T07:45:39.258353+0000","last_peered":"2024-09-21T07:45:39.258353+0000","last_clean":"2024-09-21T07:45:39.258353+0000","last_became_active":"2024-09-21T07:45:37.708379+0000","last_became_peered":"2024-09-21T07:45:37.708379+0000","last_unstale":"2024-09-21T07:45:39.258353+0000","last_undegraded":"2024-09-21T07:45:39.258353+0000","last_fullsized":"2024-09-21T07:45:39.258353+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:11:00.052134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259131+0000","last_change":"2024-09-21T07:45:37.699762+0000","last_active":"2024-09-21T07:45:39.259131+0000","last_peered":"2024-09-21T07:45:39.259131+0000","last_clean":"2024-09-21T07:45:39.259131+0000","last_became_active":"2024-09-21T07:45:37.698405+0000","last_became_peered":"2024-09-21T07:45:37.698405+0000","last_unstale":"2024-09-21T07:45:39.259131+0000","last_undegraded":"2024-09-21T07:45:39.259131+0000","last_fullsized":"2024-09-21T07:45:39.259131+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:40.095743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258837+0000","last_change":"2024-09-21T07:45:37.704433+0000","last_active":"2024-09-21T07:45:39.258837+0000","last_peered":"2024-09-21T07:45:39.258837+0000","last_clean":"2024-09-21T07:45:39.258837+0000","last_became_active":"2024-09-21T07:45:37.704117+0000","last_became_peered":"2024-09-21T07:45:37.704117+0000","last_unstale":"2024-09-21T07:45:39.258837+0000","last_undegraded":"2024-09-21T07:45:39.258837+0000","last_fullsized":"2024-09-21T07:45:39.258837+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:50.140124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258124+0000","last_change":"2024-09-21T07:45:37.706791+0000","last_active":"2024-09-21T07:45:39.258124+0000","last_peered":"2024-09-21T07:45:39.258124+0000","last_clean":"2024-09-21T07:45:39.258124+0000","last_became_active":"2024-09-21T07:45:37.706461+0000","last_became_peered":"2024-09-21T07:45:37.706461+0000","last_unstale":"2024-09-21T07:45:39.258124+0000","last_undegraded":"2024-09-21T07:45:39.258124+0000","last_fullsized":"2024-09-21T07:45:39.258124+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:09.660927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.257805+0000","last_change":"2024-09-21T07:45:37.690378+0000","last_active":"2024-09-21T07:45:39.257805+0000","last_peered":"2024-09-21T07:45:39.257805+0000","last_clean":"2024-09-21T07:45:39.257805+0000","last_became_active":"2024-09-21T07:45:37.683092+0000","last_became_peered":"2024-09-21T07:45:37.683092+0000","last_unstale":"2024-09-21T07:45:39.257805+0000","last_undegraded":"2024-09-21T07:45:39.257805+0000","last_fullsized":"2024-09-21T07:45:39.257805+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:40.878568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259706+0000","last_change":"2024-09-21T07:45:37.685191+0000","last_active":"2024-09-21T07:45:39.259706+0000","last_peered":"2024-09-21T07:45:39.259706+0000","last_clean":"2024-09-21T07:45:39.259706+0000","last_became_active":"2024-09-21T07:45:37.684765+0000","last_became_peered":"2024-09-21T07:45:37.684765+0000","last_unstale":"2024-09-21T07:45:39.259706+0000","last_undegraded":"2024-09-21T07:45:39.259706+0000","last_fullsized":"2024-09-21T07:45:39.259706+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:32.102165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689982+0000","last_change":"2024-09-21T07:45:37.704608+0000","last_active":"2024-09-21T07:45:38.689982+0000","last_peered":"2024-09-21T07:45:38.689982+0000","last_clean":"2024-09-21T07:45:38.689982+0000","last_became_active":"2024-09-21T07:45:37.703051+0000","last_became_peered":"2024-09-21T07:45:37.703051+0000","last_unstale":"2024-09-21T07:45:38.689982+0000","last_undegraded":"2024-09-21T07:45:38.689982+0000","last_fullsized":"2024-09-21T07:45:38.689982+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:33.889448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690451+0000","last_change":"2024-09-21T07:45:37.707249+0000","last_active":"2024-09-21T07:45:38.690451+0000","last_peered":"2024-09-21T07:45:38.690451+0000","last_clean":"2024-09-21T07:45:38.690451+0000","last_became_active":"2024-09-21T07:45:37.706656+0000","last_became_peered":"2024-09-21T07:45:37.706656+0000","last_unstale":"2024-09-21T07:45:38.690451+0000","last_undegraded":"2024-09-21T07:45:38.690451+0000","last_fullsized":"2024-09-21T07:45:38.690451+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:07.329504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689772+0000","last_change":"2024-09-21T07:45:37.698151+0000","last_active":"2024-09-21T07:45:38.689772+0000","last_peered":"2024-09-21T07:45:38.689772+0000","last_clean":"2024-09-21T07:45:38.689772+0000","last_became_active":"2024-09-21T07:45:37.697818+0000","last_became_peered":"2024-09-21T07:45:37.697818+0000","last_unstale":"2024-09-21T07:45:38.689772+0000","last_undegraded":"2024-09-21T07:45:38.689772+0000","last_fullsized":"2024-09-21T07:45:38.689772+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:57.867727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.815627+0000","last_change":"2024-09-21T07:45:37.675287+0000","last_active":"2024-09-21T07:45:38.815627+0000","last_peered":"2024-09-21T07:45:38.815627+0000","last_clean":"2024-09-21T07:45:38.815627+0000","last_became_active":"2024-09-21T07:45:37.674891+0000","last_became_peered":"2024-09-21T07:45:37.674891+0000","last_unstale":"2024-09-21T07:45:38.815627+0000","last_undegraded":"2024-09-21T07:45:38.815627+0000","last_fullsized":"2024-09-21T07:45:38.815627+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:45.842877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258544+0000","last_change":"2024-09-21T07:45:37.695767+0000","last_active":"2024-09-21T07:45:39.258544+0000","last_peered":"2024-09-21T07:45:39.258544+0000","last_clean":"2024-09-21T07:45:39.258544+0000","last_became_active":"2024-09-21T07:45:37.695487+0000","last_became_peered":"2024-09-21T07:45:37.695487+0000","last_unstale":"2024-09-21T07:45:39.258544+0000","last_undegraded":"2024-09-21T07:45:39.258544+0000","last_fullsized":"2024-09-21T07:45:39.258544+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:17.845096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.815270+0000","last_change":"2024-09-21T07:45:37.678052+0000","last_active":"2024-09-21T07:45:38.815270+0000","last_peered":"2024-09-21T07:45:38.815270+0000","last_clean":"2024-09-21T07:45:38.815270+0000","last_became_active":"2024-09-21T07:45:37.677738+0000","last_became_peered":"2024-09-21T07:45:37.677738+0000","last_unstale":"2024-09-21T07:45:38.815270+0000","last_undegraded":"2024-09-21T07:45:38.815270+0000","last_fullsized":"2024-09-21T07:45:38.815270+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:43:53.088002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.257910+0000","last_change":"2024-09-21T07:45:37.678436+0000","last_active":"2024-09-21T07:45:39.257910+0000","last_peered":"2024-09-21T07:45:39.257910+0000","last_clean":"2024-09-21T07:45:39.257910+0000","last_became_active":"2024-09-21T07:45:37.678086+0000","last_became_peered":"2024-09-21T07:45:37.678086+0000","last_unstale":"2024-09-21T07:45:39.257910+0000","last_undegraded":"2024-09-21T07:45:39.257910+0000","last_fullsized":"2024-09-21T07:45:39.257910+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:06:54.903590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.260024+0000","last_change":"2024-09-21T07:45:37.701506+0000","last_active":"2024-09-21T07:45:39.260024+0000","last_peered":"2024-09-21T07:45:39.260024+0000","last_clean":"2024-09-21T07:45:39.260024+0000","last_became_active":"2024-09-21T07:45:37.700952+0000","last_became_peered":"2024-09-21T07:45:37.700952+0000","last_unstale":"2024-09-21T07:45:39.260024+0000","last_undegraded":"2024-09-21T07:45:39.260024+0000","last_fullsized":"2024-09-21T07:45:39.260024+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:13.756567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259704+0000","last_change":"2024-09-21T07:45:37.697193+0000","last_active":"2024-09-21T07:45:39.259704+0000","last_peered":"2024-09-21T07:45:39.259704+0000","last_clean":"2024-09-21T07:45:39.259704+0000","last_became_active":"2024-09-21T07:45:37.696657+0000","last_became_peered":"2024-09-21T07:45:37.696657+0000","last_unstale":"2024-09-21T07:45:39.259704+0000","last_undegraded":"2024-09-21T07:45:39.259704+0000","last_fullsized":"2024-09-21T07:45:39.259704+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:14.864957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690561+0000","last_change":"2024-09-21T07:45:37.706976+0000","last_active":"2024-09-21T07:45:38.690561+0000","last_peered":"2024-09-21T07:45:38.690561+0000","last_clean":"2024-09-21T07:45:38.690561+0000","last_became_active":"2024-09-21T07:45:37.706115+0000","last_became_peered":"2024-09-21T07:45:37.706115+0000","last_unstale":"2024-09-21T07:45:38.690561+0000","last_undegraded":"2024-09-21T07:45:38.690561+0000","last_fullsized":"2024-09-21T07:45:38.690561+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:32.304761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.829565+0000","last_change":"2024-09-21T07:45:37.695282+0000","last_active":"2024-09-21T07:45:38.829565+0000","last_peered":"2024-09-21T07:45:38.829565+0000","last_clean":"2024-09-21T07:45:38.829565+0000","last_became_active":"2024-09-21T07:45:37.694929+0000","last_became_peered":"2024-09-21T07:45:37.694929+0000","last_unstale":"2024-09-21T07:45:38.829565+0000","last_undegraded":"2024-09-21T07:45:38.829565+0000","last_fullsized":"2024-09-21T07:45:38.829565+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:06:57.817032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258976+0000","last_change":"2024-09-21T07:45:37.694377+0000","last_active":"2024-09-21T07:45:39.258976+0000","last_peered":"2024-09-21T07:45:39.258976+0000","last_clean":"2024-09-21T07:45:39.258976+0000","last_became_active":"2024-09-21T07:45:37.693987+0000","last_became_peered":"2024-09-21T07:45:37.693987+0000","last_unstale":"2024-09-21T07:45:39.258976+0000","last_undegraded":"2024-09-21T07:45:39.258976+0000","last_fullsized":"2024-09-21T07:45:39.258976+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:15.146497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259076+0000","last_change":"2024-09-21T07:45:37.700110+0000","last_active":"2024-09-21T07:45:39.259076+0000","last_peered":"2024-09-21T07:45:39.259076+0000","last_clean":"2024-09-21T07:45:39.259076+0000","last_became_active":"2024-09-21T07:45:37.699820+0000","last_became_peered":"2024-09-21T07:45:37.699820+0000","last_unstale":"2024-09-21T07:45:39.259076+0000","last_undegraded":"2024-09-21T07:45:39.259076+0000","last_fullsized":"2024-09-21T07:45:39.259076+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:44.462069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.687575+0000","last_change":"2024-09-21T07:45:37.685098+0000","last_active":"2024-09-21T07:45:38.687575+0000","last_peered":"2024-09-21T07:45:38.687575+0000","last_clean":"2024-09-21T07:45:38.687575+0000","last_became_active":"2024-09-21T07:45:37.684748+0000","last_became_peered":"2024-09-21T07:45:37.684748+0000","last_unstale":"2024-09-21T07:45:38.687575+0000","last_undegraded":"2024-09-21T07:45:38.687575+0000","last_fullsized":"2024-09-21T07:45:38.687575+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:51:22.048251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.829291+0000","last_change":"2024-09-21T07:45:37.686973+0000","last_active":"2024-09-21T07:45:38.829291+0000","last_peered":"2024-09-21T07:45:38.829291+0000","last_clean":"2024-09-21T07:45:38.829291+0000","last_became_active":"2024-09-21T07:45:37.686592+0000","last_became_peered":"2024-09-21T07:45:37.686592+0000","last_unstale":"2024-09-21T07:45:38.829291+0000","last_undegraded":"2024-09-21T07:45:38.829291+0000","last_fullsized":"2024-09-21T07:45:38.829291+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:32.366993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259156+0000","last_change":"2024-09-21T07:45:37.695865+0000","last_active":"2024-09-21T07:45:39.259156+0000","last_peered":"2024-09-21T07:45:39.259156+0000","last_clean":"2024-09-21T07:45:39.259156+0000","last_became_active":"2024-09-21T07:45:37.695558+0000","last_became_peered":"2024-09-21T07:45:37.695558+0000","last_unstale":"2024-09-21T07:45:39.259156+0000","last_undegraded":"2024-09-21T07:45:39.259156+0000","last_fullsized":"2024-09-21T07:45:39.259156+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:35.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688636+0000","last_change":"2024-09-21T07:45:36.697679+0000","last_active":"2024-09-21T07:45:38.688636+0000","last_peered":"2024-09-21T07:45:38.688636+0000","last_clean":"2024-09-21T07:45:38.688636+0000","last_became_active":"2024-09-21T07:45:36.697396+0000","last_became_peered":"2024-09-21T07:45:36.697396+0000","last_unstale":"2024-09-21T07:45:38.688636+0000","last_undegraded":"2024-09-21T07:45:38.688636+0000","last_fullsized":"2024-09-21T07:45:38.688636+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:04:54.765401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":191,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690879+0000","last_change":"2024-09-21T07:35:27.879605+0000","last_active":"2024-09-21T07:45:38.690879+0000","last_peered":"2024-09-21T07:45:38.690879+0000","last_clean":"2024-09-21T07:45:38.690879+0000","last_became_active":"2024-09-21T07:35:27.879274+0000","last_became_peered":"2024-09-21T07:35:27.879274+0000","last_unstale":"2024-09-21T07:45:38.690879+0000","last_undegraded":"2024-09-21T07:45:38.690879+0000","last_fullsized":"2024-09-21T07:45:38.690879+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:24.241636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":197,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689112+0000","last_change":"2024-09-21T07:35:29.491336+0000","last_active":"2024-09-21T07:45:38.689112+0000","last_peered":"2024-09-21T07:45:38.689112+0000","last_clean":"2024-09-21T07:45:38.689112+0000","last_became_active":"2024-09-21T07:35:26.866484+0000","last_became_peered":"2024-09-21T07:35:26.866484+0000","last_unstale":"2024-09-21T07:45:38.689112+0000","last_undegraded":"2024-09-21T07:45:38.689112+0000","last_fullsized":"2024-09-21T07:45:38.689112+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:10.449665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076010100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":377,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688135+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:45:38.688135+0000","last_peered":"2024-09-21T07:45:38.688135+0000","last_clean":"2024-09-21T07:45:38.688135+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:45:38.688135+0000","last_undegraded":"2024-09-21T07:45:38.688135+0000","last_fullsized":"2024-09-21T07:45:38.688135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":256,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.816174+0000","last_change":"2024-09-21T07:34:24.539248+0000","last_active":"2024-09-21T07:45:38.816174+0000","last_peered":"2024-09-21T07:45:38.816174+0000","last_clean":"2024-09-21T07:45:38.816174+0000","last_became_active":"2024-09-21T07:34:23.273465+0000","last_became_peered":"2024-09-21T07:34:23.273465+0000","last_unstale":"2024-09-21T07:45:38.816174+0000","last_undegraded":"2024-09-21T07:45:38.816174+0000","last_fullsized":"2024-09-21T07:45:38.816174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:55.517452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060627700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258677+0000","last_change":"2024-09-21T07:45:37.695318+0000","last_active":"2024-09-21T07:45:39.258677+0000","last_peered":"2024-09-21T07:45:39.258677+0000","last_clean":"2024-09-21T07:45:39.258677+0000","last_became_active":"2024-09-21T07:45:37.695081+0000","last_became_peered":"2024-09-21T07:45:37.695081+0000","last_unstale":"2024-09-21T07:45:39.258677+0000","last_undegraded":"2024-09-21T07:45:39.258677+0000","last_fullsized":"2024-09-21T07:45:39.258677+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:56.672995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.815955+0000","last_change":"2024-09-21T07:45:36.700703+0000","last_active":"2024-09-21T07:45:38.815955+0000","last_peered":"2024-09-21T07:45:38.815955+0000","last_clean":"2024-09-21T07:45:38.815955+0000","last_became_active":"2024-09-21T07:45:36.700344+0000","last_became_peered":"2024-09-21T07:45:36.700344+0000","last_unstale":"2024-09-21T07:45:38.815955+0000","last_undegraded":"2024-09-21T07:45:38.815955+0000","last_fullsized":"2024-09-21T07:45:38.815955+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:42.885001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689081+0000","last_change":"2024-09-21T07:35:27.864492+0000","last_active":"2024-09-21T07:45:38.689081+0000","last_peered":"2024-09-21T07:45:38.689081+0000","last_clean":"2024-09-21T07:45:38.689081+0000","last_became_active":"2024-09-21T07:35:27.864176+0000","last_became_peered":"2024-09-21T07:35:27.864176+0000","last_unstale":"2024-09-21T07:45:38.689081+0000","last_undegraded":"2024-09-21T07:45:38.689081+0000","last_fullsized":"2024-09-21T07:45:38.689081+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:05.153467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":198,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.691142+0000","last_change":"2024-09-21T07:35:29.493678+0000","last_active":"2024-09-21T07:45:38.691142+0000","last_peered":"2024-09-21T07:45:38.691142+0000","last_clean":"2024-09-21T07:45:38.691142+0000","last_became_active":"2024-09-21T07:35:26.870749+0000","last_became_peered":"2024-09-21T07:35:26.870749+0000","last_unstale":"2024-09-21T07:45:38.691142+0000","last_undegraded":"2024-09-21T07:45:38.691142+0000","last_fullsized":"2024-09-21T07:45:38.691142+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:55.367877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010159570000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":246,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688084+0000","last_change":"2024-09-21T07:34:24.597274+0000","last_active":"2024-09-21T07:45:38.688084+0000","last_peered":"2024-09-21T07:45:38.688084+0000","last_clean":"2024-09-21T07:45:38.688084+0000","last_became_active":"2024-09-21T07:34:23.279560+0000","last_became_peered":"2024-09-21T07:34:23.279560+0000","last_unstale":"2024-09-21T07:45:38.688084+0000","last_undegraded":"2024-09-21T07:45:38.688084+0000","last_fullsized":"2024-09-21T07:45:38.688084+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:15.845265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059699799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.257960+0000","last_change":"2024-09-21T07:45:37.688438+0000","last_active":"2024-09-21T07:45:39.257960+0000","last_peered":"2024-09-21T07:45:39.257960+0000","last_clean":"2024-09-21T07:45:39.257960+0000","last_became_active":"2024-09-21T07:45:37.688141+0000","last_became_peered":"2024-09-21T07:45:37.688141+0000","last_unstale":"2024-09-21T07:45:39.257960+0000","last_undegraded":"2024-09-21T07:45:39.257960+0000","last_fullsized":"2024-09-21T07:45:39.257960+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:33.326739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258474+0000","last_change":"2024-09-21T07:45:36.695689+0000","last_active":"2024-09-21T07:45:39.258474+0000","last_peered":"2024-09-21T07:45:39.258474+0000","last_clean":"2024-09-21T07:45:39.258474+0000","last_became_active":"2024-09-21T07:45:36.695386+0000","last_became_peered":"2024-09-21T07:45:36.695386+0000","last_unstale":"2024-09-21T07:45:39.258474+0000","last_undegraded":"2024-09-21T07:45:39.258474+0000","last_fullsized":"2024-09-21T07:45:39.258474+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:46:14.029499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":191,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258322+0000","last_change":"2024-09-21T07:35:27.874387+0000","last_active":"2024-09-21T07:45:39.258322+0000","last_peered":"2024-09-21T07:45:39.258322+0000","last_clean":"2024-09-21T07:45:39.258322+0000","last_became_active":"2024-09-21T07:35:27.873277+0000","last_became_peered":"2024-09-21T07:35:27.873277+0000","last_unstale":"2024-09-21T07:45:39.258322+0000","last_undegraded":"2024-09-21T07:45:39.258322+0000","last_fullsized":"2024-09-21T07:45:39.258322+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:04.581463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":196,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688673+0000","last_change":"2024-09-21T07:35:29.387094+0000","last_active":"2024-09-21T07:45:38.688673+0000","last_peered":"2024-09-21T07:45:38.688673+0000","last_clean":"2024-09-21T07:45:38.688673+0000","last_became_active":"2024-09-21T07:35:26.874757+0000","last_became_peered":"2024-09-21T07:35:26.874757+0000","last_unstale":"2024-09-21T07:45:38.688673+0000","last_undegraded":"2024-09-21T07:45:38.688673+0000","last_fullsized":"2024-09-21T07:45:38.688673+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:33.542750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061065700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":252,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690132+0000","last_change":"2024-09-21T07:34:24.259906+0000","last_active":"2024-09-21T07:45:38.690132+0000","last_peered":"2024-09-21T07:45:38.690132+0000","last_clean":"2024-09-21T07:45:38.690132+0000","last_became_active":"2024-09-21T07:34:23.635282+0000","last_became_peered":"2024-09-21T07:34:23.635282+0000","last_unstale":"2024-09-21T07:45:38.690132+0000","last_undegraded":"2024-09-21T07:45:38.690132+0000","last_fullsized":"2024-09-21T07:45:38.690132+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:00.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001527664,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258930+0000","last_change":"2024-09-21T07:45:37.688779+0000","last_active":"2024-09-21T07:45:39.258930+0000","last_peered":"2024-09-21T07:45:39.258930+0000","last_clean":"2024-09-21T07:45:39.258930+0000","last_became_active":"2024-09-21T07:45:37.688359+0000","last_became_peered":"2024-09-21T07:45:37.688359+0000","last_unstale":"2024-09-21T07:45:39.258930+0000","last_undegraded":"2024-09-21T07:45:39.258930+0000","last_fullsized":"2024-09-21T07:45:39.258930+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:29:36.750167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"156'3","reported_seq":18,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.691053+0000","last_change":"2024-09-21T07:45:36.688037+0000","last_active":"2024-09-21T07:45:38.691053+0000","last_peered":"2024-09-21T07:45:38.691053+0000","last_clean":"2024-09-21T07:45:38.691053+0000","last_became_active":"2024-09-21T07:45:36.687731+0000","last_became_peered":"2024-09-21T07:45:36.687731+0000","last_unstale":"2024-09-21T07:45:38.691053+0000","last_undegraded":"2024-09-21T07:45:38.691053+0000","last_fullsized":"2024-09-21T07:45:38.691053+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:12:07.289510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":191,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259189+0000","last_change":"2024-09-21T07:35:27.873645+0000","last_active":"2024-09-21T07:45:39.259189+0000","last_peered":"2024-09-21T07:45:39.259189+0000","last_clean":"2024-09-21T07:45:39.259189+0000","last_became_active":"2024-09-21T07:35:27.873245+0000","last_became_peered":"2024-09-21T07:35:27.873245+0000","last_unstale":"2024-09-21T07:45:39.259189+0000","last_undegraded":"2024-09-21T07:45:39.259189+0000","last_fullsized":"2024-09-21T07:45:39.259189+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:55.090953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":198,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259570+0000","last_change":"2024-09-21T07:35:28.861744+0000","last_active":"2024-09-21T07:45:39.259570+0000","last_peered":"2024-09-21T07:45:39.259570+0000","last_clean":"2024-09-21T07:45:39.259570+0000","last_became_active":"2024-09-21T07:35:26.876462+0000","last_became_peered":"2024-09-21T07:35:26.876462+0000","last_unstale":"2024-09-21T07:45:39.259570+0000","last_undegraded":"2024-09-21T07:45:39.259570+0000","last_fullsized":"2024-09-21T07:45:39.259570+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:48.632568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012020920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":249,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.829502+0000","last_change":"2024-09-21T07:34:25.085454+0000","last_active":"2024-09-21T07:45:38.829502+0000","last_peered":"2024-09-21T07:45:38.829502+0000","last_clean":"2024-09-21T07:45:38.829502+0000","last_became_active":"2024-09-21T07:34:23.279205+0000","last_became_peered":"2024-09-21T07:34:23.279205+0000","last_unstale":"2024-09-21T07:45:38.829502+0000","last_undegraded":"2024-09-21T07:45:38.829502+0000","last_fullsized":"2024-09-21T07:45:38.829502+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:54.231666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095320599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259623+0000","last_change":"2024-09-21T07:45:37.693546+0000","last_active":"2024-09-21T07:45:39.259623+0000","last_peered":"2024-09-21T07:45:39.259623+0000","last_clean":"2024-09-21T07:45:39.259623+0000","last_became_active":"2024-09-21T07:45:37.693193+0000","last_became_peered":"2024-09-21T07:45:37.693193+0000","last_unstale":"2024-09-21T07:45:39.259623+0000","last_undegraded":"2024-09-21T07:45:39.259623+0000","last_fullsized":"2024-09-21T07:45:39.259623+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:13.178725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"156'6","reported_seq":21,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690072+0000","last_change":"2024-09-21T07:45:36.700878+0000","last_active":"2024-09-21T07:45:38.690072+0000","last_peered":"2024-09-21T07:45:38.690072+0000","last_clean":"2024-09-21T07:45:38.690072+0000","last_became_active":"2024-09-21T07:45:36.700484+0000","last_became_peered":"2024-09-21T07:45:36.700484+0000","last_unstale":"2024-09-21T07:45:38.690072+0000","last_undegraded":"2024-09-21T07:45:38.690072+0000","last_fullsized":"2024-09-21T07:45:38.690072+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:47.039079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688964+0000","last_change":"2024-09-21T07:35:27.861373+0000","last_active":"2024-09-21T07:45:38.688964+0000","last_peered":"2024-09-21T07:45:38.688964+0000","last_clean":"2024-09-21T07:45:38.688964+0000","last_became_active":"2024-09-21T07:35:27.861016+0000","last_became_peered":"2024-09-21T07:35:27.861016+0000","last_unstale":"2024-09-21T07:45:38.688964+0000","last_undegraded":"2024-09-21T07:45:38.688964+0000","last_fullsized":"2024-09-21T07:45:38.688964+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:56.043763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":199,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259347+0000","last_change":"2024-09-21T07:35:29.053677+0000","last_active":"2024-09-21T07:45:39.259347+0000","last_peered":"2024-09-21T07:45:39.259347+0000","last_clean":"2024-09-21T07:45:39.259347+0000","last_became_active":"2024-09-21T07:35:26.875496+0000","last_became_peered":"2024-09-21T07:35:26.875496+0000","last_unstale":"2024-09-21T07:45:39.259347+0000","last_undegraded":"2024-09-21T07:45:39.259347+0000","last_fullsized":"2024-09-21T07:45:39.259347+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:04.850784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065859000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":251,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258115+0000","last_change":"2024-09-21T07:34:25.028521+0000","last_active":"2024-09-21T07:45:39.258115+0000","last_peered":"2024-09-21T07:45:39.258115+0000","last_clean":"2024-09-21T07:45:39.258115+0000","last_became_active":"2024-09-21T07:34:23.635467+0000","last_became_peered":"2024-09-21T07:34:23.635467+0000","last_unstale":"2024-09-21T07:45:39.258115+0000","last_undegraded":"2024-09-21T07:45:39.258115+0000","last_fullsized":"2024-09-21T07:45:39.258115+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:33.729123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000660558,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.829911+0000","last_change":"2024-09-21T07:45:37.709721+0000","last_active":"2024-09-21T07:45:38.829911+0000","last_peered":"2024-09-21T07:45:38.829911+0000","last_clean":"2024-09-21T07:45:38.829911+0000","last_became_active":"2024-09-21T07:45:37.709405+0000","last_became_peered":"2024-09-21T07:45:37.709405+0000","last_unstale":"2024-09-21T07:45:38.829911+0000","last_undegraded":"2024-09-21T07:45:38.829911+0000","last_fullsized":"2024-09-21T07:45:38.829911+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:45.673393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"156'2","reported_seq":17,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259501+0000","last_change":"2024-09-21T07:45:36.697573+0000","last_active":"2024-09-21T07:45:39.259501+0000","last_peered":"2024-09-21T07:45:39.259501+0000","last_clean":"2024-09-21T07:45:39.259501+0000","last_became_active":"2024-09-21T07:45:36.697341+0000","last_became_peered":"2024-09-21T07:45:36.697341+0000","last_unstale":"2024-09-21T07:45:39.259501+0000","last_undegraded":"2024-09-21T07:45:39.259501+0000","last_fullsized":"2024-09-21T07:45:39.259501+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:48.026132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":191,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.829937+0000","last_change":"2024-09-21T07:35:27.871201+0000","last_active":"2024-09-21T07:45:38.829937+0000","last_peered":"2024-09-21T07:45:38.829937+0000","last_clean":"2024-09-21T07:45:38.829937+0000","last_became_active":"2024-09-21T07:35:27.870816+0000","last_became_peered":"2024-09-21T07:35:27.870816+0000","last_unstale":"2024-09-21T07:45:38.829937+0000","last_undegraded":"2024-09-21T07:45:38.829937+0000","last_fullsized":"2024-09-21T07:45:38.829937+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:04.787450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":197,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689594+0000","last_change":"2024-09-21T07:35:29.491874+0000","last_active":"2024-09-21T07:45:38.689594+0000","last_peered":"2024-09-21T07:45:38.689594+0000","last_clean":"2024-09-21T07:45:38.689594+0000","last_became_active":"2024-09-21T07:35:26.866381+0000","last_became_peered":"2024-09-21T07:35:26.866381+0000","last_unstale":"2024-09-21T07:45:38.689594+0000","last_undegraded":"2024-09-21T07:45:38.689594+0000","last_fullsized":"2024-09-21T07:45:38.689594+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:36.404647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073235399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":252,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690345+0000","last_change":"2024-09-21T07:34:24.260046+0000","last_active":"2024-09-21T07:45:38.690345+0000","last_peered":"2024-09-21T07:45:38.690345+0000","last_clean":"2024-09-21T07:45:38.690345+0000","last_became_active":"2024-09-21T07:34:23.273431+0000","last_became_peered":"2024-09-21T07:34:23.273431+0000","last_unstale":"2024-09-21T07:45:38.690345+0000","last_undegraded":"2024-09-21T07:45:38.690345+0000","last_fullsized":"2024-09-21T07:45:38.690345+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:18.876057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019803310000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.691105+0000","last_change":"2024-09-21T07:45:37.695290+0000","last_active":"2024-09-21T07:45:38.691105+0000","last_peered":"2024-09-21T07:45:38.691105+0000","last_clean":"2024-09-21T07:45:38.691105+0000","last_became_active":"2024-09-21T07:45:37.694954+0000","last_became_peered":"2024-09-21T07:45:37.694954+0000","last_unstale":"2024-09-21T07:45:38.691105+0000","last_undegraded":"2024-09-21T07:45:38.691105+0000","last_fullsized":"2024-09-21T07:45:38.691105+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:01:39.096631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259065+0000","last_change":"2024-09-21T07:45:36.684575+0000","last_active":"2024-09-21T07:45:39.259065+0000","last_peered":"2024-09-21T07:45:39.259065+0000","last_clean":"2024-09-21T07:45:39.259065+0000","last_became_active":"2024-09-21T07:45:36.683843+0000","last_became_peered":"2024-09-21T07:45:36.683843+0000","last_unstale":"2024-09-21T07:45:39.259065+0000","last_undegraded":"2024-09-21T07:45:39.259065+0000","last_fullsized":"2024-09-21T07:45:39.259065+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:39.029966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":191,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.830300+0000","last_change":"2024-09-21T07:35:27.871736+0000","last_active":"2024-09-21T07:45:38.830300+0000","last_peered":"2024-09-21T07:45:38.830300+0000","last_clean":"2024-09-21T07:45:38.830300+0000","last_became_active":"2024-09-21T07:35:27.871232+0000","last_became_peered":"2024-09-21T07:35:27.871232+0000","last_unstale":"2024-09-21T07:45:38.830300+0000","last_undegraded":"2024-09-21T07:45:38.830300+0000","last_fullsized":"2024-09-21T07:45:38.830300+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:04.419301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":197,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688913+0000","last_change":"2024-09-21T07:35:29.491246+0000","last_active":"2024-09-21T07:45:38.688913+0000","last_peered":"2024-09-21T07:45:38.688913+0000","last_clean":"2024-09-21T07:45:38.688913+0000","last_became_active":"2024-09-21T07:35:26.866783+0000","last_became_peered":"2024-09-21T07:35:26.866783+0000","last_unstale":"2024-09-21T07:45:38.688913+0000","last_undegraded":"2024-09-21T07:45:38.688913+0000","last_fullsized":"2024-09-21T07:45:38.688913+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:28.881914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098474099999999992,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":251,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259402+0000","last_change":"2024-09-21T07:34:25.028640+0000","last_active":"2024-09-21T07:45:39.259402+0000","last_peered":"2024-09-21T07:45:39.259402+0000","last_clean":"2024-09-21T07:45:39.259402+0000","last_became_active":"2024-09-21T07:34:23.260950+0000","last_became_peered":"2024-09-21T07:34:23.260950+0000","last_unstale":"2024-09-21T07:45:39.259402+0000","last_undegraded":"2024-09-21T07:45:39.259402+0000","last_fullsized":"2024-09-21T07:45:39.259402+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:23:42.121398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055135099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689869+0000","last_change":"2024-09-21T07:45:37.693385+0000","last_active":"2024-09-21T07:45:38.689869+0000","last_peered":"2024-09-21T07:45:38.689869+0000","last_clean":"2024-09-21T07:45:38.689869+0000","last_became_active":"2024-09-21T07:45:37.691927+0000","last_became_peered":"2024-09-21T07:45:37.691927+0000","last_unstale":"2024-09-21T07:45:38.689869+0000","last_undegraded":"2024-09-21T07:45:38.689869+0000","last_fullsized":"2024-09-21T07:45:38.689869+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:29.208442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"156'8","reported_seq":23,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258001+0000","last_change":"2024-09-21T07:45:36.681956+0000","last_active":"2024-09-21T07:45:39.258001+0000","last_peered":"2024-09-21T07:45:39.258001+0000","last_clean":"2024-09-21T07:45:39.258001+0000","last_became_active":"2024-09-21T07:45:36.681656+0000","last_became_peered":"2024-09-21T07:45:36.681656+0000","last_unstale":"2024-09-21T07:45:39.258001+0000","last_undegraded":"2024-09-21T07:45:39.258001+0000","last_fullsized":"2024-09-21T07:45:39.258001+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:17.358434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259762+0000","last_change":"2024-09-21T07:35:27.872027+0000","last_active":"2024-09-21T07:45:39.259762+0000","last_peered":"2024-09-21T07:45:39.259762+0000","last_clean":"2024-09-21T07:45:39.259762+0000","last_became_active":"2024-09-21T07:35:27.870426+0000","last_became_peered":"2024-09-21T07:35:27.870426+0000","last_unstale":"2024-09-21T07:45:39.259762+0000","last_undegraded":"2024-09-21T07:45:39.259762+0000","last_fullsized":"2024-09-21T07:45:39.259762+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":197,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689126+0000","last_change":"2024-09-21T07:35:29.491008+0000","last_active":"2024-09-21T07:45:38.689126+0000","last_peered":"2024-09-21T07:45:38.689126+0000","last_clean":"2024-09-21T07:45:38.689126+0000","last_became_active":"2024-09-21T07:35:26.866750+0000","last_became_peered":"2024-09-21T07:35:26.866750+0000","last_unstale":"2024-09-21T07:45:38.689126+0000","last_undegraded":"2024-09-21T07:45:38.689126+0000","last_fullsized":"2024-09-21T07:45:38.689126+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:25.490406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083529700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":252,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.691222+0000","last_change":"2024-09-21T07:34:24.260665+0000","last_active":"2024-09-21T07:45:38.691222+0000","last_peered":"2024-09-21T07:45:38.691222+0000","last_clean":"2024-09-21T07:45:38.691222+0000","last_became_active":"2024-09-21T07:34:23.635909+0000","last_became_peered":"2024-09-21T07:34:23.635909+0000","last_unstale":"2024-09-21T07:45:38.691222+0000","last_undegraded":"2024-09-21T07:45:38.691222+0000","last_fullsized":"2024-09-21T07:45:38.691222+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:44.536885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689417+0000","last_change":"2024-09-21T07:45:37.686269+0000","last_active":"2024-09-21T07:45:38.689417+0000","last_peered":"2024-09-21T07:45:38.689417+0000","last_clean":"2024-09-21T07:45:38.689417+0000","last_became_active":"2024-09-21T07:45:37.685971+0000","last_became_peered":"2024-09-21T07:45:37.685971+0000","last_unstale":"2024-09-21T07:45:38.689417+0000","last_undegraded":"2024-09-21T07:45:38.689417+0000","last_fullsized":"2024-09-21T07:45:38.689417+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:23.017715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"156'4","reported_seq":19,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259943+0000","last_change":"2024-09-21T07:45:36.695413+0000","last_active":"2024-09-21T07:45:39.259943+0000","last_peered":"2024-09-21T07:45:39.259943+0000","last_clean":"2024-09-21T07:45:39.259943+0000","last_became_active":"2024-09-21T07:45:36.695045+0000","last_became_peered":"2024-09-21T07:45:36.695045+0000","last_unstale":"2024-09-21T07:45:39.259943+0000","last_undegraded":"2024-09-21T07:45:39.259943+0000","last_fullsized":"2024-09-21T07:45:39.259943+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:58:37.328760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689492+0000","last_change":"2024-09-21T07:35:27.874772+0000","last_active":"2024-09-21T07:45:38.689492+0000","last_peered":"2024-09-21T07:45:38.689492+0000","last_clean":"2024-09-21T07:45:38.689492+0000","last_became_active":"2024-09-21T07:35:27.874434+0000","last_became_peered":"2024-09-21T07:35:27.874434+0000","last_unstale":"2024-09-21T07:45:38.689492+0000","last_undegraded":"2024-09-21T07:45:38.689492+0000","last_fullsized":"2024-09-21T07:45:38.689492+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:31.998755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":203,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.816113+0000","last_change":"2024-09-21T07:35:28.859153+0000","last_active":"2024-09-21T07:45:38.816113+0000","last_peered":"2024-09-21T07:45:38.816113+0000","last_clean":"2024-09-21T07:45:38.816113+0000","last_became_active":"2024-09-21T07:35:26.874428+0000","last_became_peered":"2024-09-21T07:35:26.874428+0000","last_unstale":"2024-09-21T07:45:38.816113+0000","last_undegraded":"2024-09-21T07:45:38.816113+0000","last_fullsized":"2024-09-21T07:45:38.816113+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:04.487621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048557299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":256,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.815840+0000","last_change":"2024-09-21T07:34:24.538845+0000","last_active":"2024-09-21T07:45:38.815840+0000","last_peered":"2024-09-21T07:45:38.815840+0000","last_clean":"2024-09-21T07:45:38.815840+0000","last_became_active":"2024-09-21T07:34:23.636372+0000","last_became_peered":"2024-09-21T07:34:23.636372+0000","last_unstale":"2024-09-21T07:45:38.815840+0000","last_undegraded":"2024-09-21T07:45:38.815840+0000","last_fullsized":"2024-09-21T07:45:38.815840+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:29.393813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841451,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.260055+0000","last_change":"2024-09-21T07:45:37.701732+0000","last_active":"2024-09-21T07:45:39.260055+0000","last_peered":"2024-09-21T07:45:39.260055+0000","last_clean":"2024-09-21T07:45:39.260055+0000","last_became_active":"2024-09-21T07:45:37.701407+0000","last_became_peered":"2024-09-21T07:45:37.701407+0000","last_unstale":"2024-09-21T07:45:39.260055+0000","last_undegraded":"2024-09-21T07:45:39.260055+0000","last_fullsized":"2024-09-21T07:45:39.260055+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:14.520707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"156'1","reported_seq":16,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690230+0000","last_change":"2024-09-21T07:45:36.698933+0000","last_active":"2024-09-21T07:45:38.690230+0000","last_peered":"2024-09-21T07:45:38.690230+0000","last_clean":"2024-09-21T07:45:38.690230+0000","last_became_active":"2024-09-21T07:45:36.698651+0000","last_became_peered":"2024-09-21T07:45:36.698651+0000","last_unstale":"2024-09-21T07:45:38.690230+0000","last_undegraded":"2024-09-21T07:45:38.690230+0000","last_fullsized":"2024-09-21T07:45:38.690230+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:17:31.377323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258710+0000","last_change":"2024-09-21T07:35:27.877801+0000","last_active":"2024-09-21T07:45:39.258710+0000","last_peered":"2024-09-21T07:45:39.258710+0000","last_clean":"2024-09-21T07:45:39.258710+0000","last_became_active":"2024-09-21T07:35:27.877464+0000","last_became_peered":"2024-09-21T07:35:27.877464+0000","last_unstale":"2024-09-21T07:45:39.258710+0000","last_undegraded":"2024-09-21T07:45:39.258710+0000","last_fullsized":"2024-09-21T07:45:39.258710+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:08.814929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":198,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.830348+0000","last_change":"2024-09-21T07:35:28.863732+0000","last_active":"2024-09-21T07:45:38.830348+0000","last_peered":"2024-09-21T07:45:38.830348+0000","last_clean":"2024-09-21T07:45:38.830348+0000","last_became_active":"2024-09-21T07:35:26.877392+0000","last_became_peered":"2024-09-21T07:35:26.877392+0000","last_unstale":"2024-09-21T07:45:38.830348+0000","last_undegraded":"2024-09-21T07:45:38.830348+0000","last_fullsized":"2024-09-21T07:45:38.830348+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071425700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":252,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689803+0000","last_change":"2024-09-21T07:34:24.259686+0000","last_active":"2024-09-21T07:45:38.689803+0000","last_peered":"2024-09-21T07:45:38.689803+0000","last_clean":"2024-09-21T07:45:38.689803+0000","last_became_active":"2024-09-21T07:34:23.274405+0000","last_became_peered":"2024-09-21T07:34:23.274405+0000","last_unstale":"2024-09-21T07:45:38.689803+0000","last_undegraded":"2024-09-21T07:45:38.689803+0000","last_fullsized":"2024-09-21T07:45:38.689803+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:26.304693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022009920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.815464+0000","last_change":"2024-09-21T07:45:37.678620+0000","last_active":"2024-09-21T07:45:38.815464+0000","last_peered":"2024-09-21T07:45:38.815464+0000","last_clean":"2024-09-21T07:45:38.815464+0000","last_became_active":"2024-09-21T07:45:37.678277+0000","last_became_peered":"2024-09-21T07:45:37.678277+0000","last_unstale":"2024-09-21T07:45:38.815464+0000","last_undegraded":"2024-09-21T07:45:38.815464+0000","last_fullsized":"2024-09-21T07:45:38.815464+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:50:43.432893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"156'2","reported_seq":17,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690904+0000","last_change":"2024-09-21T07:45:36.689252+0000","last_active":"2024-09-21T07:45:38.690904+0000","last_peered":"2024-09-21T07:45:38.690904+0000","last_clean":"2024-09-21T07:45:38.690904+0000","last_became_active":"2024-09-21T07:45:36.688915+0000","last_became_peered":"2024-09-21T07:45:36.688915+0000","last_unstale":"2024-09-21T07:45:38.690904+0000","last_undegraded":"2024-09-21T07:45:38.690904+0000","last_fullsized":"2024-09-21T07:45:38.690904+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:42.885001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":189,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.687721+0000","last_change":"2024-09-21T07:35:27.867336+0000","last_active":"2024-09-21T07:45:38.687721+0000","last_peered":"2024-09-21T07:45:38.687721+0000","last_clean":"2024-09-21T07:45:38.687721+0000","last_became_active":"2024-09-21T07:35:27.867015+0000","last_became_peered":"2024-09-21T07:35:27.867015+0000","last_unstale":"2024-09-21T07:45:38.687721+0000","last_undegraded":"2024-09-21T07:45:38.687721+0000","last_fullsized":"2024-09-21T07:45:38.687721+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:59.214139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":198,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.691008+0000","last_change":"2024-09-21T07:35:29.493137+0000","last_active":"2024-09-21T07:45:38.691008+0000","last_peered":"2024-09-21T07:45:38.691008+0000","last_clean":"2024-09-21T07:45:38.691008+0000","last_became_active":"2024-09-21T07:35:26.872230+0000","last_became_peered":"2024-09-21T07:35:26.872230+0000","last_unstale":"2024-09-21T07:45:38.691008+0000","last_undegraded":"2024-09-21T07:45:38.691008+0000","last_fullsized":"2024-09-21T07:45:38.691008+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:24.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001197449,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":252,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688024+0000","last_change":"2024-09-21T07:34:24.258754+0000","last_active":"2024-09-21T07:45:38.688024+0000","last_peered":"2024-09-21T07:45:38.688024+0000","last_clean":"2024-09-21T07:45:38.688024+0000","last_became_active":"2024-09-21T07:34:23.272657+0000","last_became_peered":"2024-09-21T07:34:23.272657+0000","last_unstale":"2024-09-21T07:45:38.688024+0000","last_undegraded":"2024-09-21T07:45:38.688024+0000","last_fullsized":"2024-09-21T07:45:38.688024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:05.477099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067143200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259381+0000","last_change":"2024-09-21T07:45:37.704353+0000","last_active":"2024-09-21T07:45:39.259381+0000","last_peered":"2024-09-21T07:45:39.259381+0000","last_clean":"2024-09-21T07:45:39.259381+0000","last_became_active":"2024-09-21T07:45:37.703997+0000","last_became_peered":"2024-09-21T07:45:37.703997+0000","last_unstale":"2024-09-21T07:45:39.259381+0000","last_undegraded":"2024-09-21T07:45:39.259381+0000","last_fullsized":"2024-09-21T07:45:39.259381+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:54:12.637245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"156'2","reported_seq":17,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690236+0000","last_change":"2024-09-21T07:45:36.687920+0000","last_active":"2024-09-21T07:45:38.690236+0000","last_peered":"2024-09-21T07:45:38.690236+0000","last_clean":"2024-09-21T07:45:38.690236+0000","last_became_active":"2024-09-21T07:45:36.687561+0000","last_became_peered":"2024-09-21T07:45:36.687561+0000","last_unstale":"2024-09-21T07:45:38.690236+0000","last_undegraded":"2024-09-21T07:45:38.690236+0000","last_fullsized":"2024-09-21T07:45:38.690236+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:59.601883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":191,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.830034+0000","last_change":"2024-09-21T07:35:27.876021+0000","last_active":"2024-09-21T07:45:38.830034+0000","last_peered":"2024-09-21T07:45:38.830034+0000","last_clean":"2024-09-21T07:45:38.830034+0000","last_became_active":"2024-09-21T07:35:27.875724+0000","last_became_peered":"2024-09-21T07:35:27.875724+0000","last_unstale":"2024-09-21T07:45:38.830034+0000","last_undegraded":"2024-09-21T07:45:38.830034+0000","last_fullsized":"2024-09-21T07:45:38.830034+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:03.022812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":198,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690256+0000","last_change":"2024-09-21T07:35:29.492570+0000","last_active":"2024-09-21T07:45:38.690256+0000","last_peered":"2024-09-21T07:45:38.690256+0000","last_clean":"2024-09-21T07:45:38.690256+0000","last_became_active":"2024-09-21T07:35:26.874316+0000","last_became_peered":"2024-09-21T07:35:26.874316+0000","last_unstale":"2024-09-21T07:45:38.690256+0000","last_undegraded":"2024-09-21T07:45:38.690256+0000","last_fullsized":"2024-09-21T07:45:38.690256+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:16.512323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011990760000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":250,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.829567+0000","last_change":"2024-09-21T07:34:25.085232+0000","last_active":"2024-09-21T07:45:38.829567+0000","last_peered":"2024-09-21T07:45:38.829567+0000","last_clean":"2024-09-21T07:45:38.829567+0000","last_became_active":"2024-09-21T07:34:23.277807+0000","last_became_peered":"2024-09-21T07:34:23.277807+0000","last_unstale":"2024-09-21T07:45:38.829567+0000","last_undegraded":"2024-09-21T07:45:38.829567+0000","last_fullsized":"2024-09-21T07:45:38.829567+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:36.362919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086619099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689375+0000","last_change":"2024-09-21T07:45:37.685148+0000","last_active":"2024-09-21T07:45:38.689375+0000","last_peered":"2024-09-21T07:45:38.689375+0000","last_clean":"2024-09-21T07:45:38.689375+0000","last_became_active":"2024-09-21T07:45:37.684774+0000","last_became_peered":"2024-09-21T07:45:37.684774+0000","last_unstale":"2024-09-21T07:45:38.689375+0000","last_undegraded":"2024-09-21T07:45:38.689375+0000","last_fullsized":"2024-09-21T07:45:38.689375+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:25.616419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"156'1","reported_seq":16,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.687687+0000","last_change":"2024-09-21T07:45:36.686891+0000","last_active":"2024-09-21T07:45:38.687687+0000","last_peered":"2024-09-21T07:45:38.687687+0000","last_clean":"2024-09-21T07:45:38.687687+0000","last_became_active":"2024-09-21T07:45:36.684124+0000","last_became_peered":"2024-09-21T07:45:36.684124+0000","last_unstale":"2024-09-21T07:45:38.687687+0000","last_undegraded":"2024-09-21T07:45:38.687687+0000","last_fullsized":"2024-09-21T07:45:38.687687+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:00.150983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258873+0000","last_change":"2024-09-21T07:35:27.872798+0000","last_active":"2024-09-21T07:45:39.258873+0000","last_peered":"2024-09-21T07:45:39.258873+0000","last_clean":"2024-09-21T07:45:39.258873+0000","last_became_active":"2024-09-21T07:35:27.872483+0000","last_became_peered":"2024-09-21T07:35:27.872483+0000","last_unstale":"2024-09-21T07:45:39.258873+0000","last_undegraded":"2024-09-21T07:45:39.258873+0000","last_fullsized":"2024-09-21T07:45:39.258873+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:11.933469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":197,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259383+0000","last_change":"2024-09-21T07:35:29.206723+0000","last_active":"2024-09-21T07:45:39.259383+0000","last_peered":"2024-09-21T07:45:39.259383+0000","last_clean":"2024-09-21T07:45:39.259383+0000","last_became_active":"2024-09-21T07:35:26.872377+0000","last_became_peered":"2024-09-21T07:35:26.872377+0000","last_unstale":"2024-09-21T07:45:39.259383+0000","last_undegraded":"2024-09-21T07:45:39.259383+0000","last_fullsized":"2024-09-21T07:45:39.259383+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:29.086862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083077200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":246,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689123+0000","last_change":"2024-09-21T07:34:24.597672+0000","last_active":"2024-09-21T07:45:38.689123+0000","last_peered":"2024-09-21T07:45:38.689123+0000","last_clean":"2024-09-21T07:45:38.689123+0000","last_became_active":"2024-09-21T07:34:23.267770+0000","last_became_peered":"2024-09-21T07:34:23.267770+0000","last_unstale":"2024-09-21T07:45:38.689123+0000","last_undegraded":"2024-09-21T07:45:38.689123+0000","last_fullsized":"2024-09-21T07:45:38.689123+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:51.629476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086378500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689895+0000","last_change":"2024-09-21T07:45:37.691898+0000","last_active":"2024-09-21T07:45:38.689895+0000","last_peered":"2024-09-21T07:45:38.689895+0000","last_clean":"2024-09-21T07:45:38.689895+0000","last_became_active":"2024-09-21T07:45:37.691507+0000","last_became_peered":"2024-09-21T07:45:37.691507+0000","last_unstale":"2024-09-21T07:45:38.689895+0000","last_undegraded":"2024-09-21T07:45:38.689895+0000","last_fullsized":"2024-09-21T07:45:38.689895+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:14:43.900850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"156'1","reported_seq":16,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689228+0000","last_change":"2024-09-21T07:45:36.694668+0000","last_active":"2024-09-21T07:45:38.689228+0000","last_peered":"2024-09-21T07:45:38.689228+0000","last_clean":"2024-09-21T07:45:38.689228+0000","last_became_active":"2024-09-21T07:45:36.694405+0000","last_became_peered":"2024-09-21T07:45:36.694405+0000","last_unstale":"2024-09-21T07:45:38.689228+0000","last_undegraded":"2024-09-21T07:45:38.689228+0000","last_fullsized":"2024-09-21T07:45:38.689228+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:08.552085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259996+0000","last_change":"2024-09-21T07:35:27.870121+0000","last_active":"2024-09-21T07:45:39.259996+0000","last_peered":"2024-09-21T07:45:39.259996+0000","last_clean":"2024-09-21T07:45:39.259996+0000","last_became_active":"2024-09-21T07:35:27.869811+0000","last_became_peered":"2024-09-21T07:35:27.869811+0000","last_unstale":"2024-09-21T07:45:39.259996+0000","last_undegraded":"2024-09-21T07:45:39.259996+0000","last_fullsized":"2024-09-21T07:45:39.259996+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:31.778461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":198,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259121+0000","last_change":"2024-09-21T07:35:28.860699+0000","last_active":"2024-09-21T07:45:39.259121+0000","last_peered":"2024-09-21T07:45:39.259121+0000","last_clean":"2024-09-21T07:45:39.259121+0000","last_became_active":"2024-09-21T07:35:26.876779+0000","last_became_peered":"2024-09-21T07:35:26.876779+0000","last_unstale":"2024-09-21T07:45:39.259121+0000","last_undegraded":"2024-09-21T07:45:39.259121+0000","last_fullsized":"2024-09-21T07:45:39.259121+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:28.286403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000913188,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":251,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259901+0000","last_change":"2024-09-21T07:34:24.260181+0000","last_active":"2024-09-21T07:45:39.259901+0000","last_peered":"2024-09-21T07:45:39.259901+0000","last_clean":"2024-09-21T07:45:39.259901+0000","last_became_active":"2024-09-21T07:34:23.275580+0000","last_became_peered":"2024-09-21T07:34:23.275580+0000","last_unstale":"2024-09-21T07:45:39.259901+0000","last_undegraded":"2024-09-21T07:45:39.259901+0000","last_fullsized":"2024-09-21T07:45:39.259901+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:49.857622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014706980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689850+0000","last_change":"2024-09-21T07:45:37.698246+0000","last_active":"2024-09-21T07:45:38.689850+0000","last_peered":"2024-09-21T07:45:38.689850+0000","last_clean":"2024-09-21T07:45:38.689850+0000","last_became_active":"2024-09-21T07:45:37.697792+0000","last_became_peered":"2024-09-21T07:45:37.697792+0000","last_unstale":"2024-09-21T07:45:38.689850+0000","last_undegraded":"2024-09-21T07:45:38.689850+0000","last_fullsized":"2024-09-21T07:45:38.689850+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:06.139752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.257812+0000","last_change":"2024-09-21T07:45:36.693696+0000","last_active":"2024-09-21T07:45:39.257812+0000","last_peered":"2024-09-21T07:45:39.257812+0000","last_clean":"2024-09-21T07:45:39.257812+0000","last_became_active":"2024-09-21T07:45:36.693211+0000","last_became_peered":"2024-09-21T07:45:36.693211+0000","last_unstale":"2024-09-21T07:45:39.257812+0000","last_undegraded":"2024-09-21T07:45:39.257812+0000","last_fullsized":"2024-09-21T07:45:39.257812+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:12:07.289510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":192,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258799+0000","last_change":"2024-09-21T07:35:27.875393+0000","last_active":"2024-09-21T07:45:39.258799+0000","last_peered":"2024-09-21T07:45:39.258799+0000","last_clean":"2024-09-21T07:45:39.258799+0000","last_became_active":"2024-09-21T07:35:27.874671+0000","last_became_peered":"2024-09-21T07:35:27.874671+0000","last_unstale":"2024-09-21T07:45:39.258799+0000","last_undegraded":"2024-09-21T07:45:39.258799+0000","last_fullsized":"2024-09-21T07:45:39.258799+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":197,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259239+0000","last_change":"2024-09-21T07:35:29.206296+0000","last_active":"2024-09-21T07:45:39.259239+0000","last_peered":"2024-09-21T07:45:39.259239+0000","last_clean":"2024-09-21T07:45:39.259239+0000","last_became_active":"2024-09-21T07:35:26.874575+0000","last_became_peered":"2024-09-21T07:35:26.874575+0000","last_unstale":"2024-09-21T07:45:39.259239+0000","last_undegraded":"2024-09-21T07:45:39.259239+0000","last_fullsized":"2024-09-21T07:45:39.259239+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:22.771154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061011599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":252,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690260+0000","last_change":"2024-09-21T07:34:24.259112+0000","last_active":"2024-09-21T07:45:38.690260+0000","last_peered":"2024-09-21T07:45:38.690260+0000","last_clean":"2024-09-21T07:45:38.690260+0000","last_became_active":"2024-09-21T07:34:23.635444+0000","last_became_peered":"2024-09-21T07:34:23.635444+0000","last_unstale":"2024-09-21T07:45:38.690260+0000","last_undegraded":"2024-09-21T07:45:38.690260+0000","last_fullsized":"2024-09-21T07:45:38.690260+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:46.255385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001952636,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690949+0000","last_change":"2024-09-21T07:45:37.689391+0000","last_active":"2024-09-21T07:45:38.690949+0000","last_peered":"2024-09-21T07:45:38.690949+0000","last_clean":"2024-09-21T07:45:38.690949+0000","last_became_active":"2024-09-21T07:45:37.688994+0000","last_became_peered":"2024-09-21T07:45:37.688994+0000","last_unstale":"2024-09-21T07:45:38.690949+0000","last_undegraded":"2024-09-21T07:45:38.690949+0000","last_fullsized":"2024-09-21T07:45:38.690949+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:06:54.903590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258563+0000","last_change":"2024-09-21T07:45:36.696284+0000","last_active":"2024-09-21T07:45:39.258563+0000","last_peered":"2024-09-21T07:45:39.258563+0000","last_clean":"2024-09-21T07:45:39.258563+0000","last_became_active":"2024-09-21T07:45:36.695952+0000","last_became_peered":"2024-09-21T07:45:36.695952+0000","last_unstale":"2024-09-21T07:45:39.258563+0000","last_undegraded":"2024-09-21T07:45:39.258563+0000","last_fullsized":"2024-09-21T07:45:39.258563+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:38.261108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":192,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259873+0000","last_change":"2024-09-21T07:35:27.878356+0000","last_active":"2024-09-21T07:45:39.259873+0000","last_peered":"2024-09-21T07:45:39.259873+0000","last_clean":"2024-09-21T07:45:39.259873+0000","last_became_active":"2024-09-21T07:35:27.877982+0000","last_became_peered":"2024-09-21T07:35:27.877982+0000","last_unstale":"2024-09-21T07:45:39.259873+0000","last_undegraded":"2024-09-21T07:45:39.259873+0000","last_fullsized":"2024-09-21T07:45:39.259873+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":200,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258236+0000","last_change":"2024-09-21T07:35:29.053131+0000","last_active":"2024-09-21T07:45:39.258236+0000","last_peered":"2024-09-21T07:45:39.258236+0000","last_clean":"2024-09-21T07:45:39.258236+0000","last_became_active":"2024-09-21T07:35:26.870516+0000","last_became_peered":"2024-09-21T07:35:26.870516+0000","last_unstale":"2024-09-21T07:45:39.258236+0000","last_undegraded":"2024-09-21T07:45:39.258236+0000","last_fullsized":"2024-09-21T07:45:39.258236+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:10.924491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070705699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":256,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.815898+0000","last_change":"2024-09-21T07:34:24.539152+0000","last_active":"2024-09-21T07:45:38.815898+0000","last_peered":"2024-09-21T07:45:38.815898+0000","last_clean":"2024-09-21T07:45:38.815898+0000","last_became_active":"2024-09-21T07:34:23.259573+0000","last_became_peered":"2024-09-21T07:34:23.259573+0000","last_unstale":"2024-09-21T07:45:38.815898+0000","last_undegraded":"2024-09-21T07:45:38.815898+0000","last_fullsized":"2024-09-21T07:45:38.815898+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:26.986900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058108400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689007+0000","last_change":"2024-09-21T07:45:37.689304+0000","last_active":"2024-09-21T07:45:38.689007+0000","last_peered":"2024-09-21T07:45:38.689007+0000","last_clean":"2024-09-21T07:45:38.689007+0000","last_became_active":"2024-09-21T07:45:37.688985+0000","last_became_peered":"2024-09-21T07:45:37.688985+0000","last_unstale":"2024-09-21T07:45:38.689007+0000","last_undegraded":"2024-09-21T07:45:38.689007+0000","last_fullsized":"2024-09-21T07:45:38.689007+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:03.596216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"156'2","reported_seq":17,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259672+0000","last_change":"2024-09-21T07:45:36.697335+0000","last_active":"2024-09-21T07:45:39.259672+0000","last_peered":"2024-09-21T07:45:39.259672+0000","last_clean":"2024-09-21T07:45:39.259672+0000","last_became_active":"2024-09-21T07:45:36.697036+0000","last_became_peered":"2024-09-21T07:45:36.697036+0000","last_unstale":"2024-09-21T07:45:39.259672+0000","last_undegraded":"2024-09-21T07:45:39.259672+0000","last_fullsized":"2024-09-21T07:45:39.259672+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:10:38.762615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":192,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258124+0000","last_change":"2024-09-21T07:35:27.873726+0000","last_active":"2024-09-21T07:45:39.258124+0000","last_peered":"2024-09-21T07:45:39.258124+0000","last_clean":"2024-09-21T07:45:39.258124+0000","last_became_active":"2024-09-21T07:35:27.871421+0000","last_became_peered":"2024-09-21T07:35:27.871421+0000","last_unstale":"2024-09-21T07:45:39.258124+0000","last_undegraded":"2024-09-21T07:45:39.258124+0000","last_fullsized":"2024-09-21T07:45:39.258124+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:00.168915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":198,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259737+0000","last_change":"2024-09-21T07:35:28.861032+0000","last_active":"2024-09-21T07:45:39.259737+0000","last_peered":"2024-09-21T07:45:39.259737+0000","last_clean":"2024-09-21T07:45:39.259737+0000","last_became_active":"2024-09-21T07:35:26.873149+0000","last_became_peered":"2024-09-21T07:35:26.873149+0000","last_unstale":"2024-09-21T07:45:39.259737+0000","last_undegraded":"2024-09-21T07:45:39.259737+0000","last_fullsized":"2024-09-21T07:45:39.259737+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:59.435981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011988369999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":251,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689271+0000","last_change":"2024-09-21T07:34:24.259902+0000","last_active":"2024-09-21T07:45:38.689271+0000","last_peered":"2024-09-21T07:45:38.689271+0000","last_clean":"2024-09-21T07:45:38.689271+0000","last_became_active":"2024-09-21T07:34:23.271902+0000","last_became_peered":"2024-09-21T07:34:23.271902+0000","last_unstale":"2024-09-21T07:45:38.689271+0000","last_undegraded":"2024-09-21T07:45:38.689271+0000","last_fullsized":"2024-09-21T07:45:38.689271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:59.439525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001686358,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258605+0000","last_change":"2024-09-21T07:45:37.691171+0000","last_active":"2024-09-21T07:45:39.258605+0000","last_peered":"2024-09-21T07:45:39.258605+0000","last_clean":"2024-09-21T07:45:39.258605+0000","last_became_active":"2024-09-21T07:45:37.690825+0000","last_became_peered":"2024-09-21T07:45:37.690825+0000","last_unstale":"2024-09-21T07:45:39.258605+0000","last_undegraded":"2024-09-21T07:45:39.258605+0000","last_fullsized":"2024-09-21T07:45:39.258605+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:34.131587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"157'1","reported_seq":16,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689021+0000","last_change":"2024-09-21T07:45:36.673147+0000","last_active":"2024-09-21T07:45:38.689021+0000","last_peered":"2024-09-21T07:45:38.689021+0000","last_clean":"2024-09-21T07:45:38.689021+0000","last_became_active":"2024-09-21T07:45:36.672816+0000","last_became_peered":"2024-09-21T07:45:36.672816+0000","last_unstale":"2024-09-21T07:45:38.689021+0000","last_undegraded":"2024-09-21T07:45:38.689021+0000","last_fullsized":"2024-09-21T07:45:38.689021+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:04:18.857870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":191,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.829343+0000","last_change":"2024-09-21T07:35:27.872637+0000","last_active":"2024-09-21T07:45:38.829343+0000","last_peered":"2024-09-21T07:45:38.829343+0000","last_clean":"2024-09-21T07:45:38.829343+0000","last_became_active":"2024-09-21T07:35:27.872149+0000","last_became_peered":"2024-09-21T07:35:27.872149+0000","last_unstale":"2024-09-21T07:45:38.829343+0000","last_undegraded":"2024-09-21T07:45:38.829343+0000","last_fullsized":"2024-09-21T07:45:38.829343+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:27.697872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":199,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690975+0000","last_change":"2024-09-21T07:35:29.493362+0000","last_active":"2024-09-21T07:45:38.690975+0000","last_peered":"2024-09-21T07:45:38.690975+0000","last_clean":"2024-09-21T07:45:38.690975+0000","last_became_active":"2024-09-21T07:35:26.873018+0000","last_became_peered":"2024-09-21T07:35:26.873018+0000","last_unstale":"2024-09-21T07:45:38.690975+0000","last_undegraded":"2024-09-21T07:45:38.690975+0000","last_fullsized":"2024-09-21T07:45:38.690975+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:02.460557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012496510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":251,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689489+0000","last_change":"2024-09-21T07:34:24.259662+0000","last_active":"2024-09-21T07:45:38.689489+0000","last_peered":"2024-09-21T07:45:38.689489+0000","last_clean":"2024-09-21T07:45:38.689489+0000","last_became_active":"2024-09-21T07:34:23.278812+0000","last_became_peered":"2024-09-21T07:34:23.278812+0000","last_unstale":"2024-09-21T07:45:38.689489+0000","last_undegraded":"2024-09-21T07:45:38.689489+0000","last_fullsized":"2024-09-21T07:45:38.689489+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:49.829974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356463,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688331+0000","last_change":"2024-09-21T07:45:37.701467+0000","last_active":"2024-09-21T07:45:38.688331+0000","last_peered":"2024-09-21T07:45:38.688331+0000","last_clean":"2024-09-21T07:45:38.688331+0000","last_became_active":"2024-09-21T07:45:37.696605+0000","last_became_peered":"2024-09-21T07:45:37.696605+0000","last_unstale":"2024-09-21T07:45:38.688331+0000","last_undegraded":"2024-09-21T07:45:38.688331+0000","last_fullsized":"2024-09-21T07:45:38.688331+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:08:37.797721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689740+0000","last_change":"2024-09-21T07:45:36.693420+0000","last_active":"2024-09-21T07:45:38.689740+0000","last_peered":"2024-09-21T07:45:38.689740+0000","last_clean":"2024-09-21T07:45:38.689740+0000","last_became_active":"2024-09-21T07:45:36.692969+0000","last_became_peered":"2024-09-21T07:45:36.692969+0000","last_unstale":"2024-09-21T07:45:38.689740+0000","last_undegraded":"2024-09-21T07:45:38.689740+0000","last_fullsized":"2024-09-21T07:45:38.689740+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:09:22.417758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":189,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.687613+0000","last_change":"2024-09-21T07:35:27.868448+0000","last_active":"2024-09-21T07:45:38.687613+0000","last_peered":"2024-09-21T07:45:38.687613+0000","last_clean":"2024-09-21T07:45:38.687613+0000","last_became_active":"2024-09-21T07:35:27.868105+0000","last_became_peered":"2024-09-21T07:35:27.868105+0000","last_unstale":"2024-09-21T07:45:38.687613+0000","last_undegraded":"2024-09-21T07:45:38.687613+0000","last_fullsized":"2024-09-21T07:45:38.687613+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:41.468702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":198,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690782+0000","last_change":"2024-09-21T07:35:29.492974+0000","last_active":"2024-09-21T07:45:38.690782+0000","last_peered":"2024-09-21T07:45:38.690782+0000","last_clean":"2024-09-21T07:45:38.690782+0000","last_became_active":"2024-09-21T07:35:26.872165+0000","last_became_peered":"2024-09-21T07:35:26.872165+0000","last_unstale":"2024-09-21T07:45:38.690782+0000","last_undegraded":"2024-09-21T07:45:38.690782+0000","last_fullsized":"2024-09-21T07:45:38.690782+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:59.912520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013795179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":249,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.829490+0000","last_change":"2024-09-21T07:34:25.084861+0000","last_active":"2024-09-21T07:45:38.829490+0000","last_peered":"2024-09-21T07:45:38.829490+0000","last_clean":"2024-09-21T07:45:38.829490+0000","last_became_active":"2024-09-21T07:34:23.277708+0000","last_became_peered":"2024-09-21T07:34:23.277708+0000","last_unstale":"2024-09-21T07:45:38.829490+0000","last_undegraded":"2024-09-21T07:45:38.829490+0000","last_fullsized":"2024-09-21T07:45:38.829490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:40.816488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000597566,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.829769+0000","last_change":"2024-09-21T07:45:37.691290+0000","last_active":"2024-09-21T07:45:38.829769+0000","last_peered":"2024-09-21T07:45:38.829769+0000","last_clean":"2024-09-21T07:45:38.829769+0000","last_became_active":"2024-09-21T07:45:37.690017+0000","last_became_peered":"2024-09-21T07:45:37.690017+0000","last_unstale":"2024-09-21T07:45:38.829769+0000","last_undegraded":"2024-09-21T07:45:38.829769+0000","last_fullsized":"2024-09-21T07:45:38.829769+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:14.769595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259263+0000","last_change":"2024-09-21T07:45:36.687709+0000","last_active":"2024-09-21T07:45:39.259263+0000","last_peered":"2024-09-21T07:45:39.259263+0000","last_clean":"2024-09-21T07:45:39.259263+0000","last_became_active":"2024-09-21T07:45:36.684703+0000","last_became_peered":"2024-09-21T07:45:36.684703+0000","last_unstale":"2024-09-21T07:45:39.259263+0000","last_undegraded":"2024-09-21T07:45:39.259263+0000","last_fullsized":"2024-09-21T07:45:39.259263+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:59:04.017206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":189,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688381+0000","last_change":"2024-09-21T07:35:27.874640+0000","last_active":"2024-09-21T07:45:38.688381+0000","last_peered":"2024-09-21T07:45:38.688381+0000","last_clean":"2024-09-21T07:45:38.688381+0000","last_became_active":"2024-09-21T07:35:27.874265+0000","last_became_peered":"2024-09-21T07:35:27.874265+0000","last_unstale":"2024-09-21T07:45:38.688381+0000","last_undegraded":"2024-09-21T07:45:38.688381+0000","last_fullsized":"2024-09-21T07:45:38.688381+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:45.754654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":197,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259024+0000","last_change":"2024-09-21T07:35:29.207392+0000","last_active":"2024-09-21T07:45:39.259024+0000","last_peered":"2024-09-21T07:45:39.259024+0000","last_clean":"2024-09-21T07:45:39.259024+0000","last_became_active":"2024-09-21T07:35:26.870917+0000","last_became_peered":"2024-09-21T07:35:26.870917+0000","last_unstale":"2024-09-21T07:45:39.259024+0000","last_undegraded":"2024-09-21T07:45:39.259024+0000","last_fullsized":"2024-09-21T07:45:39.259024+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:59.159937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059198499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":251,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689049+0000","last_change":"2024-09-21T07:34:24.258709+0000","last_active":"2024-09-21T07:45:38.689049+0000","last_peered":"2024-09-21T07:45:38.689049+0000","last_clean":"2024-09-21T07:45:38.689049+0000","last_became_active":"2024-09-21T07:34:23.279042+0000","last_became_peered":"2024-09-21T07:34:23.279042+0000","last_unstale":"2024-09-21T07:45:38.689049+0000","last_undegraded":"2024-09-21T07:45:38.689049+0000","last_fullsized":"2024-09-21T07:45:38.689049+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:01.762117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019282100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259047+0000","last_change":"2024-09-21T07:45:37.697340+0000","last_active":"2024-09-21T07:45:39.259047+0000","last_peered":"2024-09-21T07:45:39.259047+0000","last_clean":"2024-09-21T07:45:39.259047+0000","last_became_active":"2024-09-21T07:45:37.696970+0000","last_became_peered":"2024-09-21T07:45:37.696970+0000","last_unstale":"2024-09-21T07:45:39.259047+0000","last_undegraded":"2024-09-21T07:45:39.259047+0000","last_fullsized":"2024-09-21T07:45:39.259047+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:19:22.593298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.815240+0000","last_change":"2024-09-21T07:45:36.698592+0000","last_active":"2024-09-21T07:45:38.815240+0000","last_peered":"2024-09-21T07:45:38.815240+0000","last_clean":"2024-09-21T07:45:38.815240+0000","last_became_active":"2024-09-21T07:45:36.698011+0000","last_became_peered":"2024-09-21T07:45:36.698011+0000","last_unstale":"2024-09-21T07:45:38.815240+0000","last_undegraded":"2024-09-21T07:45:38.815240+0000","last_fullsized":"2024-09-21T07:45:38.815240+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:12:07.289510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":191,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.829818+0000","last_change":"2024-09-21T07:35:27.870260+0000","last_active":"2024-09-21T07:45:38.829818+0000","last_peered":"2024-09-21T07:45:38.829818+0000","last_clean":"2024-09-21T07:45:38.829818+0000","last_became_active":"2024-09-21T07:35:27.869448+0000","last_became_peered":"2024-09-21T07:35:27.869448+0000","last_unstale":"2024-09-21T07:45:38.829818+0000","last_undegraded":"2024-09-21T07:45:38.829818+0000","last_fullsized":"2024-09-21T07:45:38.829818+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:16.946790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":196,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688161+0000","last_change":"2024-09-21T07:35:29.386367+0000","last_active":"2024-09-21T07:45:38.688161+0000","last_peered":"2024-09-21T07:45:38.688161+0000","last_clean":"2024-09-21T07:45:38.688161+0000","last_became_active":"2024-09-21T07:35:26.879238+0000","last_became_peered":"2024-09-21T07:35:26.879238+0000","last_unstale":"2024-09-21T07:45:38.688161+0000","last_undegraded":"2024-09-21T07:45:38.688161+0000","last_fullsized":"2024-09-21T07:45:38.688161+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:16.528260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092223100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":249,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.830404+0000","last_change":"2024-09-21T07:34:25.085635+0000","last_active":"2024-09-21T07:45:38.830404+0000","last_peered":"2024-09-21T07:45:38.830404+0000","last_clean":"2024-09-21T07:45:38.830404+0000","last_became_active":"2024-09-21T07:34:23.273566+0000","last_became_peered":"2024-09-21T07:34:23.273566+0000","last_unstale":"2024-09-21T07:45:38.830404+0000","last_undegraded":"2024-09-21T07:45:38.830404+0000","last_fullsized":"2024-09-21T07:45:38.830404+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:14.142272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079897300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259060+0000","last_change":"2024-09-21T07:45:37.696449+0000","last_active":"2024-09-21T07:45:39.259060+0000","last_peered":"2024-09-21T07:45:39.259060+0000","last_clean":"2024-09-21T07:45:39.259060+0000","last_became_active":"2024-09-21T07:45:37.692664+0000","last_became_peered":"2024-09-21T07:45:37.692664+0000","last_unstale":"2024-09-21T07:45:39.259060+0000","last_undegraded":"2024-09-21T07:45:39.259060+0000","last_fullsized":"2024-09-21T07:45:39.259060+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:52.828679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689020+0000","last_change":"2024-09-21T07:45:36.685900+0000","last_active":"2024-09-21T07:45:38.689020+0000","last_peered":"2024-09-21T07:45:38.689020+0000","last_clean":"2024-09-21T07:45:38.689020+0000","last_became_active":"2024-09-21T07:45:36.685578+0000","last_became_peered":"2024-09-21T07:45:36.685578+0000","last_unstale":"2024-09-21T07:45:38.689020+0000","last_undegraded":"2024-09-21T07:45:38.689020+0000","last_fullsized":"2024-09-21T07:45:38.689020+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:15.970506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":191,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.830083+0000","last_change":"2024-09-21T07:35:27.878728+0000","last_active":"2024-09-21T07:45:38.830083+0000","last_peered":"2024-09-21T07:45:38.830083+0000","last_clean":"2024-09-21T07:45:38.830083+0000","last_became_active":"2024-09-21T07:35:27.878335+0000","last_became_peered":"2024-09-21T07:35:27.878335+0000","last_unstale":"2024-09-21T07:45:38.830083+0000","last_undegraded":"2024-09-21T07:45:38.830083+0000","last_fullsized":"2024-09-21T07:45:38.830083+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:25:37.731771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":203,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.815572+0000","last_change":"2024-09-21T07:35:28.858418+0000","last_active":"2024-09-21T07:45:38.815572+0000","last_peered":"2024-09-21T07:45:38.815572+0000","last_clean":"2024-09-21T07:45:38.815572+0000","last_became_active":"2024-09-21T07:35:26.876944+0000","last_became_peered":"2024-09-21T07:35:26.876944+0000","last_unstale":"2024-09-21T07:45:38.815572+0000","last_undegraded":"2024-09-21T07:45:38.815572+0000","last_fullsized":"2024-09-21T07:45:38.815572+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:04.514304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090862799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":245,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688906+0000","last_change":"2024-09-21T07:34:24.597926+0000","last_active":"2024-09-21T07:45:38.688906+0000","last_peered":"2024-09-21T07:45:38.688906+0000","last_clean":"2024-09-21T07:45:38.688906+0000","last_became_active":"2024-09-21T07:34:23.268926+0000","last_became_peered":"2024-09-21T07:34:23.268926+0000","last_unstale":"2024-09-21T07:45:38.688906+0000","last_undegraded":"2024-09-21T07:45:38.688906+0000","last_fullsized":"2024-09-21T07:45:38.688906+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:46.190752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101851,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258976+0000","last_change":"2024-09-21T07:45:37.690580+0000","last_active":"2024-09-21T07:45:39.258976+0000","last_peered":"2024-09-21T07:45:39.258976+0000","last_clean":"2024-09-21T07:45:39.258976+0000","last_became_active":"2024-09-21T07:45:37.690274+0000","last_became_peered":"2024-09-21T07:45:37.690274+0000","last_unstale":"2024-09-21T07:45:39.258976+0000","last_undegraded":"2024-09-21T07:45:39.258976+0000","last_fullsized":"2024-09-21T07:45:39.258976+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:07.141785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.257933+0000","last_change":"2024-09-21T07:45:36.683898+0000","last_active":"2024-09-21T07:45:39.257933+0000","last_peered":"2024-09-21T07:45:39.257933+0000","last_clean":"2024-09-21T07:45:39.257933+0000","last_became_active":"2024-09-21T07:45:36.683536+0000","last_became_peered":"2024-09-21T07:45:36.683536+0000","last_unstale":"2024-09-21T07:45:39.257933+0000","last_undegraded":"2024-09-21T07:45:39.257933+0000","last_fullsized":"2024-09-21T07:45:39.257933+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:42.885001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688581+0000","last_change":"2024-09-21T07:35:27.877426+0000","last_active":"2024-09-21T07:45:38.688581+0000","last_peered":"2024-09-21T07:45:38.688581+0000","last_clean":"2024-09-21T07:45:38.688581+0000","last_became_active":"2024-09-21T07:35:27.877050+0000","last_became_peered":"2024-09-21T07:35:27.877050+0000","last_unstale":"2024-09-21T07:45:38.688581+0000","last_undegraded":"2024-09-21T07:45:38.688581+0000","last_fullsized":"2024-09-21T07:45:38.688581+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:36.268334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":198,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.829462+0000","last_change":"2024-09-21T07:35:28.862273+0000","last_active":"2024-09-21T07:45:38.829462+0000","last_peered":"2024-09-21T07:45:38.829462+0000","last_clean":"2024-09-21T07:45:38.829462+0000","last_became_active":"2024-09-21T07:35:26.874011+0000","last_became_peered":"2024-09-21T07:35:26.874011+0000","last_unstale":"2024-09-21T07:45:38.829462+0000","last_undegraded":"2024-09-21T07:45:38.829462+0000","last_fullsized":"2024-09-21T07:45:38.829462+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:11.986950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071644799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":250,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258568+0000","last_change":"2024-09-21T07:34:24.259794+0000","last_active":"2024-09-21T07:45:39.258568+0000","last_peered":"2024-09-21T07:45:39.258568+0000","last_clean":"2024-09-21T07:45:39.258568+0000","last_became_active":"2024-09-21T07:34:23.265679+0000","last_became_peered":"2024-09-21T07:34:23.265679+0000","last_unstale":"2024-09-21T07:45:39.258568+0000","last_undegraded":"2024-09-21T07:45:39.258568+0000","last_fullsized":"2024-09-21T07:45:39.258568+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:02.575796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001298518,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690698+0000","last_change":"2024-09-21T07:45:37.703118+0000","last_active":"2024-09-21T07:45:38.690698+0000","last_peered":"2024-09-21T07:45:38.690698+0000","last_clean":"2024-09-21T07:45:38.690698+0000","last_became_active":"2024-09-21T07:45:37.702812+0000","last_became_peered":"2024-09-21T07:45:37.702812+0000","last_unstale":"2024-09-21T07:45:38.690698+0000","last_undegraded":"2024-09-21T07:45:38.690698+0000","last_fullsized":"2024-09-21T07:45:38.690698+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:40.878568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689224+0000","last_change":"2024-09-21T07:45:36.695633+0000","last_active":"2024-09-21T07:45:38.689224+0000","last_peered":"2024-09-21T07:45:38.689224+0000","last_clean":"2024-09-21T07:45:38.689224+0000","last_became_active":"2024-09-21T07:45:36.695319+0000","last_became_peered":"2024-09-21T07:45:36.695319+0000","last_unstale":"2024-09-21T07:45:38.689224+0000","last_undegraded":"2024-09-21T07:45:38.689224+0000","last_fullsized":"2024-09-21T07:45:38.689224+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:36.781831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259525+0000","last_change":"2024-09-21T07:35:27.877725+0000","last_active":"2024-09-21T07:45:39.259525+0000","last_peered":"2024-09-21T07:45:39.259525+0000","last_clean":"2024-09-21T07:45:39.259525+0000","last_became_active":"2024-09-21T07:35:27.877398+0000","last_became_peered":"2024-09-21T07:35:27.877398+0000","last_unstale":"2024-09-21T07:45:39.259525+0000","last_undegraded":"2024-09-21T07:45:39.259525+0000","last_fullsized":"2024-09-21T07:45:39.259525+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:05.859762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":198,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258909+0000","last_change":"2024-09-21T07:35:28.860612+0000","last_active":"2024-09-21T07:45:39.258909+0000","last_peered":"2024-09-21T07:45:39.258909+0000","last_clean":"2024-09-21T07:45:39.258909+0000","last_became_active":"2024-09-21T07:35:26.876562+0000","last_became_peered":"2024-09-21T07:35:26.876562+0000","last_unstale":"2024-09-21T07:45:39.258909+0000","last_undegraded":"2024-09-21T07:45:39.258909+0000","last_fullsized":"2024-09-21T07:45:39.258909+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:06.824331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012252470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":251,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.691149+0000","last_change":"2024-09-21T07:34:24.260330+0000","last_active":"2024-09-21T07:45:38.691149+0000","last_peered":"2024-09-21T07:45:38.691149+0000","last_clean":"2024-09-21T07:45:38.691149+0000","last_became_active":"2024-09-21T07:34:23.637210+0000","last_became_peered":"2024-09-21T07:34:23.637210+0000","last_unstale":"2024-09-21T07:45:38.691149+0000","last_undegraded":"2024-09-21T07:45:38.691149+0000","last_fullsized":"2024-09-21T07:45:38.691149+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.488451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001537093,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258595+0000","last_change":"2024-09-21T07:45:37.686719+0000","last_active":"2024-09-21T07:45:39.258595+0000","last_peered":"2024-09-21T07:45:39.258595+0000","last_clean":"2024-09-21T07:45:39.258595+0000","last_became_active":"2024-09-21T07:45:37.686379+0000","last_became_peered":"2024-09-21T07:45:37.686379+0000","last_unstale":"2024-09-21T07:45:39.258595+0000","last_undegraded":"2024-09-21T07:45:39.258595+0000","last_fullsized":"2024-09-21T07:45:39.258595+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:20.522083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"156'1","reported_seq":16,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.816015+0000","last_change":"2024-09-21T07:45:36.676063+0000","last_active":"2024-09-21T07:45:38.816015+0000","last_peered":"2024-09-21T07:45:38.816015+0000","last_clean":"2024-09-21T07:45:38.816015+0000","last_became_active":"2024-09-21T07:45:36.675750+0000","last_became_peered":"2024-09-21T07:45:36.675750+0000","last_unstale":"2024-09-21T07:45:38.816015+0000","last_undegraded":"2024-09-21T07:45:38.816015+0000","last_fullsized":"2024-09-21T07:45:38.816015+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:17.358434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":191,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.829276+0000","last_change":"2024-09-21T07:35:27.874757+0000","last_active":"2024-09-21T07:45:38.829276+0000","last_peered":"2024-09-21T07:45:38.829276+0000","last_clean":"2024-09-21T07:45:38.829276+0000","last_became_active":"2024-09-21T07:35:27.874360+0000","last_became_peered":"2024-09-21T07:35:27.874360+0000","last_unstale":"2024-09-21T07:45:38.829276+0000","last_undegraded":"2024-09-21T07:45:38.829276+0000","last_fullsized":"2024-09-21T07:45:38.829276+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:17.646912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":198,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.691038+0000","last_change":"2024-09-21T07:35:29.493431+0000","last_active":"2024-09-21T07:45:38.691038+0000","last_peered":"2024-09-21T07:45:38.691038+0000","last_clean":"2024-09-21T07:45:38.691038+0000","last_became_active":"2024-09-21T07:35:26.873691+0000","last_became_peered":"2024-09-21T07:35:26.873691+0000","last_unstale":"2024-09-21T07:45:38.691038+0000","last_undegraded":"2024-09-21T07:45:38.691038+0000","last_fullsized":"2024-09-21T07:45:38.691038+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:51.705051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001162464,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":245,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.687901+0000","last_change":"2024-09-21T07:34:24.597286+0000","last_active":"2024-09-21T07:45:38.687901+0000","last_peered":"2024-09-21T07:45:38.687901+0000","last_clean":"2024-09-21T07:45:38.687901+0000","last_became_active":"2024-09-21T07:34:23.267133+0000","last_became_peered":"2024-09-21T07:34:23.267133+0000","last_unstale":"2024-09-21T07:45:38.687901+0000","last_undegraded":"2024-09-21T07:45:38.687901+0000","last_fullsized":"2024-09-21T07:45:38.687901+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:18.257010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688522+0000","last_change":"2024-09-21T07:45:37.705412+0000","last_active":"2024-09-21T07:45:38.688522+0000","last_peered":"2024-09-21T07:45:38.688522+0000","last_clean":"2024-09-21T07:45:38.688522+0000","last_became_active":"2024-09-21T07:45:37.702124+0000","last_became_peered":"2024-09-21T07:45:37.702124+0000","last_unstale":"2024-09-21T07:45:38.688522+0000","last_undegraded":"2024-09-21T07:45:38.688522+0000","last_fullsized":"2024-09-21T07:45:38.688522+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:42:19.473518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"156'2","reported_seq":17,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259093+0000","last_change":"2024-09-21T07:45:36.693348+0000","last_active":"2024-09-21T07:45:39.259093+0000","last_peered":"2024-09-21T07:45:39.259093+0000","last_clean":"2024-09-21T07:45:39.259093+0000","last_became_active":"2024-09-21T07:45:36.692821+0000","last_became_peered":"2024-09-21T07:45:36.692821+0000","last_unstale":"2024-09-21T07:45:39.259093+0000","last_undegraded":"2024-09-21T07:45:39.259093+0000","last_fullsized":"2024-09-21T07:45:39.259093+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:19.622466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":196,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.815289+0000","last_change":"2024-09-21T07:35:27.864458+0000","last_active":"2024-09-21T07:45:38.815289+0000","last_peered":"2024-09-21T07:45:38.815289+0000","last_clean":"2024-09-21T07:45:38.815289+0000","last_became_active":"2024-09-21T07:35:27.864136+0000","last_became_peered":"2024-09-21T07:35:27.864136+0000","last_unstale":"2024-09-21T07:45:38.815289+0000","last_undegraded":"2024-09-21T07:45:38.815289+0000","last_fullsized":"2024-09-21T07:45:38.815289+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:20.162240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":203,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.815836+0000","last_change":"2024-09-21T07:35:28.858784+0000","last_active":"2024-09-21T07:45:38.815836+0000","last_peered":"2024-09-21T07:45:38.815836+0000","last_clean":"2024-09-21T07:45:38.815836+0000","last_became_active":"2024-09-21T07:35:26.876890+0000","last_became_peered":"2024-09-21T07:35:26.876890+0000","last_unstale":"2024-09-21T07:45:38.815836+0000","last_undegraded":"2024-09-21T07:45:38.815836+0000","last_fullsized":"2024-09-21T07:45:38.815836+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:40.738327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064543499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":251,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689383+0000","last_change":"2024-09-21T07:34:24.258712+0000","last_active":"2024-09-21T07:45:38.689383+0000","last_peered":"2024-09-21T07:45:38.689383+0000","last_clean":"2024-09-21T07:45:38.689383+0000","last_became_active":"2024-09-21T07:34:23.274692+0000","last_became_peered":"2024-09-21T07:34:23.274692+0000","last_unstale":"2024-09-21T07:45:38.689383+0000","last_undegraded":"2024-09-21T07:45:38.689383+0000","last_fullsized":"2024-09-21T07:45:38.689383+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:43.279454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018051479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688844+0000","last_change":"2024-09-21T07:45:37.694775+0000","last_active":"2024-09-21T07:45:38.688844+0000","last_peered":"2024-09-21T07:45:38.688844+0000","last_clean":"2024-09-21T07:45:38.688844+0000","last_became_active":"2024-09-21T07:45:37.694023+0000","last_became_peered":"2024-09-21T07:45:37.694023+0000","last_unstale":"2024-09-21T07:45:38.688844+0000","last_undegraded":"2024-09-21T07:45:38.688844+0000","last_fullsized":"2024-09-21T07:45:38.688844+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:46:14.530791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"156'1","reported_seq":16,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259260+0000","last_change":"2024-09-21T07:45:36.700873+0000","last_active":"2024-09-21T07:45:39.259260+0000","last_peered":"2024-09-21T07:45:39.259260+0000","last_clean":"2024-09-21T07:45:39.259260+0000","last_became_active":"2024-09-21T07:45:36.700526+0000","last_became_peered":"2024-09-21T07:45:36.700526+0000","last_unstale":"2024-09-21T07:45:39.259260+0000","last_undegraded":"2024-09-21T07:45:39.259260+0000","last_fullsized":"2024-09-21T07:45:39.259260+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:34.374418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689061+0000","last_change":"2024-09-21T07:35:27.878020+0000","last_active":"2024-09-21T07:45:38.689061+0000","last_peered":"2024-09-21T07:45:38.689061+0000","last_clean":"2024-09-21T07:45:38.689061+0000","last_became_active":"2024-09-21T07:35:27.877532+0000","last_became_peered":"2024-09-21T07:35:27.877532+0000","last_unstale":"2024-09-21T07:45:38.689061+0000","last_undegraded":"2024-09-21T07:45:38.689061+0000","last_fullsized":"2024-09-21T07:45:38.689061+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:19.756073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":198,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259515+0000","last_change":"2024-09-21T07:35:28.861049+0000","last_active":"2024-09-21T07:45:39.259515+0000","last_peered":"2024-09-21T07:45:39.259515+0000","last_clean":"2024-09-21T07:45:39.259515+0000","last_became_active":"2024-09-21T07:35:26.876819+0000","last_became_peered":"2024-09-21T07:35:26.876819+0000","last_unstale":"2024-09-21T07:45:39.259515+0000","last_undegraded":"2024-09-21T07:45:39.259515+0000","last_fullsized":"2024-09-21T07:45:39.259515+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:19.633803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055780599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":249,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.829340+0000","last_change":"2024-09-21T07:34:25.084849+0000","last_active":"2024-09-21T07:45:38.829340+0000","last_peered":"2024-09-21T07:45:38.829340+0000","last_clean":"2024-09-21T07:45:38.829340+0000","last_became_active":"2024-09-21T07:34:23.277713+0000","last_became_peered":"2024-09-21T07:34:23.277713+0000","last_unstale":"2024-09-21T07:45:38.829340+0000","last_undegraded":"2024-09-21T07:45:38.829340+0000","last_fullsized":"2024-09-21T07:45:38.829340+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:05.811810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059079499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.829858+0000","last_change":"2024-09-21T07:45:37.700946+0000","last_active":"2024-09-21T07:45:38.829858+0000","last_peered":"2024-09-21T07:45:38.829858+0000","last_clean":"2024-09-21T07:45:38.829858+0000","last_became_active":"2024-09-21T07:45:37.700358+0000","last_became_peered":"2024-09-21T07:45:37.700358+0000","last_unstale":"2024-09-21T07:45:38.829858+0000","last_undegraded":"2024-09-21T07:45:38.829858+0000","last_fullsized":"2024-09-21T07:45:38.829858+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:58:14.141300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690040+0000","last_change":"2024-09-21T07:45:36.693932+0000","last_active":"2024-09-21T07:45:38.690040+0000","last_peered":"2024-09-21T07:45:38.690040+0000","last_clean":"2024-09-21T07:45:38.690040+0000","last_became_active":"2024-09-21T07:45:36.693640+0000","last_became_peered":"2024-09-21T07:45:36.693640+0000","last_unstale":"2024-09-21T07:45:38.690040+0000","last_undegraded":"2024-09-21T07:45:38.690040+0000","last_fullsized":"2024-09-21T07:45:38.690040+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:04.701463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":192,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258955+0000","last_change":"2024-09-21T07:35:27.884248+0000","last_active":"2024-09-21T07:45:39.258955+0000","last_peered":"2024-09-21T07:45:39.258955+0000","last_clean":"2024-09-21T07:45:39.258955+0000","last_became_active":"2024-09-21T07:35:27.883861+0000","last_became_peered":"2024-09-21T07:35:27.883861+0000","last_unstale":"2024-09-21T07:45:39.258955+0000","last_undegraded":"2024-09-21T07:45:39.258955+0000","last_fullsized":"2024-09-21T07:45:39.258955+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":197,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258922+0000","last_change":"2024-09-21T07:35:29.207135+0000","last_active":"2024-09-21T07:45:39.258922+0000","last_peered":"2024-09-21T07:45:39.258922+0000","last_clean":"2024-09-21T07:45:39.258922+0000","last_became_active":"2024-09-21T07:35:26.872787+0000","last_became_peered":"2024-09-21T07:35:26.872787+0000","last_unstale":"2024-09-21T07:45:39.258922+0000","last_undegraded":"2024-09-21T07:45:39.258922+0000","last_fullsized":"2024-09-21T07:45:39.258922+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:27.543217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011157509999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":255,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.815416+0000","last_change":"2024-09-21T07:34:24.538610+0000","last_active":"2024-09-21T07:45:38.815416+0000","last_peered":"2024-09-21T07:45:38.815416+0000","last_clean":"2024-09-21T07:45:38.815416+0000","last_became_active":"2024-09-21T07:34:23.259700+0000","last_became_peered":"2024-09-21T07:34:23.259700+0000","last_unstale":"2024-09-21T07:45:38.815416+0000","last_undegraded":"2024-09-21T07:45:38.815416+0000","last_fullsized":"2024-09-21T07:45:38.815416+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:57.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070522399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690121+0000","last_change":"2024-09-21T07:45:37.714568+0000","last_active":"2024-09-21T07:45:38.690121+0000","last_peered":"2024-09-21T07:45:38.690121+0000","last_clean":"2024-09-21T07:45:38.690121+0000","last_became_active":"2024-09-21T07:45:37.714235+0000","last_became_peered":"2024-09-21T07:45:37.714235+0000","last_unstale":"2024-09-21T07:45:38.690121+0000","last_undegraded":"2024-09-21T07:45:38.690121+0000","last_fullsized":"2024-09-21T07:45:38.690121+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:29.318374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"156'2","reported_seq":17,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.687809+0000","last_change":"2024-09-21T07:45:36.685978+0000","last_active":"2024-09-21T07:45:38.687809+0000","last_peered":"2024-09-21T07:45:38.687809+0000","last_clean":"2024-09-21T07:45:38.687809+0000","last_became_active":"2024-09-21T07:45:36.685307+0000","last_became_peered":"2024-09-21T07:45:36.685307+0000","last_unstale":"2024-09-21T07:45:38.687809+0000","last_undegraded":"2024-09-21T07:45:38.687809+0000","last_fullsized":"2024-09-21T07:45:38.687809+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:18.642482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":196,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.815520+0000","last_change":"2024-09-21T07:35:27.868024+0000","last_active":"2024-09-21T07:45:38.815520+0000","last_peered":"2024-09-21T07:45:38.815520+0000","last_clean":"2024-09-21T07:45:38.815520+0000","last_became_active":"2024-09-21T07:35:27.867339+0000","last_became_peered":"2024-09-21T07:35:27.867339+0000","last_unstale":"2024-09-21T07:45:38.815520+0000","last_undegraded":"2024-09-21T07:45:38.815520+0000","last_fullsized":"2024-09-21T07:45:38.815520+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":203,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.815424+0000","last_change":"2024-09-21T07:35:28.858505+0000","last_active":"2024-09-21T07:45:38.815424+0000","last_peered":"2024-09-21T07:45:38.815424+0000","last_clean":"2024-09-21T07:45:38.815424+0000","last_became_active":"2024-09-21T07:35:26.877320+0000","last_became_peered":"2024-09-21T07:35:26.877320+0000","last_unstale":"2024-09-21T07:45:38.815424+0000","last_undegraded":"2024-09-21T07:45:38.815424+0000","last_fullsized":"2024-09-21T07:45:38.815424+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:53.672782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244119999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":246,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689180+0000","last_change":"2024-09-21T07:34:24.597597+0000","last_active":"2024-09-21T07:45:38.689180+0000","last_peered":"2024-09-21T07:45:38.689180+0000","last_clean":"2024-09-21T07:45:38.689180+0000","last_became_active":"2024-09-21T07:34:23.635204+0000","last_became_peered":"2024-09-21T07:34:23.635204+0000","last_unstale":"2024-09-21T07:45:38.689180+0000","last_undegraded":"2024-09-21T07:45:38.689180+0000","last_fullsized":"2024-09-21T07:45:38.689180+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:48.644789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081139199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258252+0000","last_change":"2024-09-21T07:45:37.707585+0000","last_active":"2024-09-21T07:45:39.258252+0000","last_peered":"2024-09-21T07:45:39.258252+0000","last_clean":"2024-09-21T07:45:39.258252+0000","last_became_active":"2024-09-21T07:45:37.706725+0000","last_became_peered":"2024-09-21T07:45:37.706725+0000","last_unstale":"2024-09-21T07:45:39.258252+0000","last_undegraded":"2024-09-21T07:45:39.258252+0000","last_fullsized":"2024-09-21T07:45:39.258252+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:16.829578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259009+0000","last_change":"2024-09-21T07:45:36.693812+0000","last_active":"2024-09-21T07:45:39.259009+0000","last_peered":"2024-09-21T07:45:39.259009+0000","last_clean":"2024-09-21T07:45:39.259009+0000","last_became_active":"2024-09-21T07:45:36.692616+0000","last_became_peered":"2024-09-21T07:45:36.692616+0000","last_unstale":"2024-09-21T07:45:39.259009+0000","last_undegraded":"2024-09-21T07:45:39.259009+0000","last_fullsized":"2024-09-21T07:45:39.259009+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:39.176262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259810+0000","last_change":"2024-09-21T07:35:27.877503+0000","last_active":"2024-09-21T07:45:39.259810+0000","last_peered":"2024-09-21T07:45:39.259810+0000","last_clean":"2024-09-21T07:45:39.259810+0000","last_became_active":"2024-09-21T07:35:27.875775+0000","last_became_peered":"2024-09-21T07:35:27.875775+0000","last_unstale":"2024-09-21T07:45:39.259810+0000","last_undegraded":"2024-09-21T07:45:39.259810+0000","last_fullsized":"2024-09-21T07:45:39.259810+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:49.291023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":198,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689157+0000","last_change":"2024-09-21T07:35:29.491412+0000","last_active":"2024-09-21T07:45:38.689157+0000","last_peered":"2024-09-21T07:45:38.689157+0000","last_clean":"2024-09-21T07:45:38.689157+0000","last_became_active":"2024-09-21T07:35:26.876404+0000","last_became_peered":"2024-09-21T07:35:26.876404+0000","last_unstale":"2024-09-21T07:45:38.689157+0000","last_undegraded":"2024-09-21T07:45:38.689157+0000","last_fullsized":"2024-09-21T07:45:38.689157+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063918000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":252,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689996+0000","last_change":"2024-09-21T07:34:24.259026+0000","last_active":"2024-09-21T07:45:38.689996+0000","last_peered":"2024-09-21T07:45:38.689996+0000","last_clean":"2024-09-21T07:45:38.689996+0000","last_became_active":"2024-09-21T07:34:23.635456+0000","last_became_peered":"2024-09-21T07:34:23.635456+0000","last_unstale":"2024-09-21T07:45:38.689996+0000","last_undegraded":"2024-09-21T07:45:38.689996+0000","last_fullsized":"2024-09-21T07:45:38.689996+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:09.592752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018654419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690345+0000","last_change":"2024-09-21T07:45:37.700588+0000","last_active":"2024-09-21T07:45:38.690345+0000","last_peered":"2024-09-21T07:45:38.690345+0000","last_clean":"2024-09-21T07:45:38.690345+0000","last_became_active":"2024-09-21T07:45:37.700288+0000","last_became_peered":"2024-09-21T07:45:37.700288+0000","last_unstale":"2024-09-21T07:45:38.690345+0000","last_undegraded":"2024-09-21T07:45:38.690345+0000","last_fullsized":"2024-09-21T07:45:38.690345+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:36:25.586374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688671+0000","last_change":"2024-09-21T07:45:36.695336+0000","last_active":"2024-09-21T07:45:38.688671+0000","last_peered":"2024-09-21T07:45:38.688671+0000","last_clean":"2024-09-21T07:45:38.688671+0000","last_became_active":"2024-09-21T07:45:36.694952+0000","last_became_peered":"2024-09-21T07:45:36.694952+0000","last_unstale":"2024-09-21T07:45:38.688671+0000","last_undegraded":"2024-09-21T07:45:38.688671+0000","last_fullsized":"2024-09-21T07:45:38.688671+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:53:20.269457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259465+0000","last_change":"2024-09-21T07:35:27.871984+0000","last_active":"2024-09-21T07:45:39.259465+0000","last_peered":"2024-09-21T07:45:39.259465+0000","last_clean":"2024-09-21T07:45:39.259465+0000","last_became_active":"2024-09-21T07:35:27.870336+0000","last_became_peered":"2024-09-21T07:35:27.870336+0000","last_unstale":"2024-09-21T07:45:39.259465+0000","last_undegraded":"2024-09-21T07:45:39.259465+0000","last_fullsized":"2024-09-21T07:45:39.259465+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:57.120479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":198,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258561+0000","last_change":"2024-09-21T07:35:28.860351+0000","last_active":"2024-09-21T07:45:39.258561+0000","last_peered":"2024-09-21T07:45:39.258561+0000","last_clean":"2024-09-21T07:45:39.258561+0000","last_became_active":"2024-09-21T07:35:26.872046+0000","last_became_peered":"2024-09-21T07:35:26.872046+0000","last_unstale":"2024-09-21T07:45:39.258561+0000","last_undegraded":"2024-09-21T07:45:39.258561+0000","last_fullsized":"2024-09-21T07:45:39.258561+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:47.074327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098384399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":251,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690460+0000","last_change":"2024-09-21T07:34:24.260300+0000","last_active":"2024-09-21T07:45:38.690460+0000","last_peered":"2024-09-21T07:45:38.690460+0000","last_clean":"2024-09-21T07:45:38.690460+0000","last_became_active":"2024-09-21T07:34:23.270128+0000","last_became_peered":"2024-09-21T07:34:23.270128+0000","last_unstale":"2024-09-21T07:45:38.690460+0000","last_undegraded":"2024-09-21T07:45:38.690460+0000","last_fullsized":"2024-09-21T07:45:38.690460+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:25.260931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00176651,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.816185+0000","last_change":"2024-09-21T07:45:37.698965+0000","last_active":"2024-09-21T07:45:38.816185+0000","last_peered":"2024-09-21T07:45:38.816185+0000","last_clean":"2024-09-21T07:45:38.816185+0000","last_became_active":"2024-09-21T07:45:37.698631+0000","last_became_peered":"2024-09-21T07:45:37.698631+0000","last_unstale":"2024-09-21T07:45:38.816185+0000","last_undegraded":"2024-09-21T07:45:38.816185+0000","last_fullsized":"2024-09-21T07:45:38.816185+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:04:04.127234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258099+0000","last_change":"2024-09-21T07:45:36.683495+0000","last_active":"2024-09-21T07:45:39.258099+0000","last_peered":"2024-09-21T07:45:39.258099+0000","last_clean":"2024-09-21T07:45:39.258099+0000","last_became_active":"2024-09-21T07:45:36.683165+0000","last_became_peered":"2024-09-21T07:45:36.683165+0000","last_unstale":"2024-09-21T07:45:39.258099+0000","last_undegraded":"2024-09-21T07:45:39.258099+0000","last_fullsized":"2024-09-21T07:45:39.258099+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:35:14.065677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259743+0000","last_change":"2024-09-21T07:35:27.877861+0000","last_active":"2024-09-21T07:45:39.259743+0000","last_peered":"2024-09-21T07:45:39.259743+0000","last_clean":"2024-09-21T07:45:39.259743+0000","last_became_active":"2024-09-21T07:35:27.877434+0000","last_became_peered":"2024-09-21T07:35:27.877434+0000","last_unstale":"2024-09-21T07:45:39.259743+0000","last_undegraded":"2024-09-21T07:45:39.259743+0000","last_fullsized":"2024-09-21T07:45:39.259743+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":198,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.829608+0000","last_change":"2024-09-21T07:35:28.862954+0000","last_active":"2024-09-21T07:45:38.829608+0000","last_peered":"2024-09-21T07:45:38.829608+0000","last_clean":"2024-09-21T07:45:38.829608+0000","last_became_active":"2024-09-21T07:35:26.870613+0000","last_became_peered":"2024-09-21T07:35:26.870613+0000","last_unstale":"2024-09-21T07:45:38.829608+0000","last_undegraded":"2024-09-21T07:45:38.829608+0000","last_fullsized":"2024-09-21T07:45:38.829608+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:06.901043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059842100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":250,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259583+0000","last_change":"2024-09-21T07:34:25.028904+0000","last_active":"2024-09-21T07:45:39.259583+0000","last_peered":"2024-09-21T07:45:39.259583+0000","last_clean":"2024-09-21T07:45:39.259583+0000","last_became_active":"2024-09-21T07:34:23.260942+0000","last_became_peered":"2024-09-21T07:34:23.260942+0000","last_unstale":"2024-09-21T07:45:39.259583+0000","last_undegraded":"2024-09-21T07:45:39.259583+0000","last_fullsized":"2024-09-21T07:45:39.259583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:22.709328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060553899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509595,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37860,"kb_used_data":2512,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333980,"statfs":{"total":96636764160,"available":96597995520,"internally_reserved":0,"allocated":2572288,"data_stored":1621101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509597,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37272,"kb_used_data":1924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334568,"statfs":{"total":96636764160,"available":96598597632,"internally_reserved":0,"allocated":1970176,"data_stored":1029703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509595,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37276,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334564,"statfs":{"total":96636764160,"available":96598593536,"internally_reserved":0,"allocated":1966080,"data_stored":1029053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509596,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37272,"kb_used_data":1924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334568,"statfs":{"total":96636764160,"available":96598597632,"internally_reserved":0,"allocated":1970176,"data_stored":1029647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509597,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37268,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334572,"statfs":{"total":96636764160,"available":96598601728,"internally_reserved":0,"allocated":1966080,"data_stored":1029101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509597,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37860,"kb_used_data":2504,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333980,"statfs":{"total":96636764160,"available":96597995520,"internally_reserved":0,"allocated":2564096,"data_stored":1619455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542300,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37272,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334568,"statfs":{"total":96636764160,"available":96598597632,"internally_reserved":0,"allocated":1961984,"data_stored":1029031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542299,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41448,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330392,"statfs":{"total":96636764160,"available":96594321408,"internally_reserved":0,"allocated":1978368,"data_stored":1030713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:45:47.260 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:45:47.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.497+0000 7f2d52400640 1 -- 172.21.15.39:0/1013217582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d30005250 msgr2=0x7f2d3000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:47.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.497+0000 7f2d52400640 1 --2- 172.21.15.39:0/1013217582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d30005250 0x7f2d3000b9c0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f2d4400c630 tx=0x7f2d30005830 comp rx=0 tx=0).stop 2024-09-21T07:45:47.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.497+0000 7f2d52400640 1 -- 172.21.15.39:0/1013217582 shutdown_connections 2024-09-21T07:45:47.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.497+0000 7f2d52400640 1 --2- 172.21.15.39:0/1013217582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d30005250 0x7f2d3000b9c0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:47.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.497+0000 7f2d52400640 1 -- 172.21.15.39:0/1013217582 >> 172.21.15.39:0/1013217582 conn(0x7f2d4c09d7d0 msgr2=0x7f2d4c0d4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:47.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.497+0000 7f2d52400640 1 -- 172.21.15.39:0/1013217582 shutdown_connections 2024-09-21T07:45:47.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.497+0000 7f2d52400640 1 -- 172.21.15.39:0/1013217582 wait complete. 2024-09-21T07:45:47.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.499+0000 7f2d52400640 1 Processor -- start 2024-09-21T07:45:47.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.499+0000 7f2d52400640 1 -- start start 2024-09-21T07:45:47.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.501+0000 7f2d52400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d30005250 0x7f2d4c150950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:47.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.501+0000 7f2d52400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d4c15f4c0 0x7f2d4c15b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:47.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.501+0000 7f2d52400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d4c15b8d0 0x7f2d4c1f9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:47.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.501+0000 7f2d52400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c0b5d70 con 0x7f2d4c15b8d0 2024-09-21T07:45:47.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.501+0000 7f2d52400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c0b5bf0 con 0x7f2d4c15f4c0 2024-09-21T07:45:47.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.501+0000 7f2d52400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f2d4c0d1270 con 0x7f2d30005250 2024-09-21T07:45:47.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.501+0000 7f2d51a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d4c15b8d0 0x7f2d4c1f9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:47.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.501+0000 7f2d51a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d4c15b8d0 0x7f2d4c1f9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53896/0 (socket says 172.21.15.39:53896) 2024-09-21T07:45:47.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.501+0000 7f2d51a00640 1 -- 172.21.15.39:0/1980641324 learned_addr learned my addr 172.21.15.39:0/1980641324 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:47.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.501+0000 7f2d51000640 1 --2- 172.21.15.39:0/1980641324 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d30005250 0x7f2d4c150950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:47.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.501+0000 7f2d43e00640 1 --2- 172.21.15.39:0/1980641324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d4c15f4c0 0x7f2d4c15b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:47.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.501+0000 7f2d51a00640 1 -- 172.21.15.39:0/1980641324 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d30005250 msgr2=0x7f2d4c150950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:47.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.501+0000 7f2d51a00640 1 --2- 172.21.15.39:0/1980641324 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d30005250 0x7f2d4c150950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:47.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.501+0000 7f2d51a00640 1 -- 172.21.15.39:0/1980641324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d4c15f4c0 msgr2=0x7f2d4c15b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:47.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.501+0000 7f2d51a00640 1 --2- 172.21.15.39:0/1980641324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d4c15f4c0 0x7f2d4c15b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:47.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.501+0000 7f2d51a00640 1 -- 172.21.15.39:0/1980641324 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d4c15bce0 con 0x7f2d4c15b8d0 2024-09-21T07:45:47.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.501+0000 7f2d51000640 1 --2- 172.21.15.39:0/1980641324 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d30005250 0x7f2d4c150950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:45:47.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.501+0000 7f2d43e00640 1 --2- 172.21.15.39:0/1980641324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d4c15f4c0 0x7f2d4c15b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:45:47.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.502+0000 7f2d51a00640 1 --2- 172.21.15.39:0/1980641324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d4c15b8d0 0x7f2d4c1f9070 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f2d4400c750 tx=0x7f2d44003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:47.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.502+0000 7f2d49600640 1 -- 172.21.15.39:0/1980641324 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d44039070 con 0x7f2d4c15b8d0 2024-09-21T07:45:47.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.502+0000 7f2d49600640 1 -- 172.21.15.39:0/1980641324 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d4402bae0 con 0x7f2d4c15b8d0 2024-09-21T07:45:47.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.502+0000 7f2d49600640 1 -- 172.21.15.39:0/1980641324 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d44002080 con 0x7f2d4c15b8d0 2024-09-21T07:45:47.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.502+0000 7f2d52400640 1 -- 172.21.15.39:0/1980641324 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d4c15a670 con 0x7f2d4c15b8d0 2024-09-21T07:45:47.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.502+0000 7f2d52400640 1 -- 172.21.15.39:0/1980641324 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d4c15ccf0 con 0x7f2d4c15b8d0 2024-09-21T07:45:47.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.503+0000 7f2d52400640 1 -- 172.21.15.39:0/1980641324 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d4c069250 con 0x7f2d4c15b8d0 2024-09-21T07:45:47.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.505+0000 7f2d49600640 1 -- 172.21.15.39:0/1980641324 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2d44006140 con 0x7f2d4c15b8d0 2024-09-21T07:45:47.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.506+0000 7f2d49600640 1 --2- 172.21.15.39:0/1980641324 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2d28078b90 0x7f2d2807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:47.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.506+0000 7f2d49600640 1 -- 172.21.15.39:0/1980641324 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 6844+0+0 (secure 0 0 0) 0x7f2d440bacf0 con 0x7f2d4c15b8d0 2024-09-21T07:45:47.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.507+0000 7f2d51000640 1 --2- 172.21.15.39:0/1980641324 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2d28078b90 0x7f2d2807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:47.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.507+0000 7f2d51000640 1 --2- 172.21.15.39:0/1980641324 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2d28078b90 0x7f2d2807b0a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c006fd0 tx=0x7f2d3c006bc0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:47.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.511+0000 7f2d49600640 1 -- 172.21.15.39:0/1980641324 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2d440bf070 con 0x7f2d4c15b8d0 2024-09-21T07:45:47.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.706+0000 7f2d52400640 1 -- 172.21.15.39:0/1980641324 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2d4c0d31b0 con 0x7f2d28078b90 2024-09-21T07:45:47.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.715+0000 7f2d49600640 1 -- 172.21.15.39:0/1980641324 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+495245 (secure 0 0 0) 0x7f2d3c011650 con 0x7f2d28078b90 2024-09-21T07:45:47.723 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:45:47.726 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:45:47.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.720+0000 7f2d52400640 1 -- 172.21.15.39:0/1980641324 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2d28078b90 msgr2=0x7f2d2807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:47.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.720+0000 7f2d52400640 1 --2- 172.21.15.39:0/1980641324 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2d28078b90 0x7f2d2807b0a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c006fd0 tx=0x7f2d3c006bc0 comp rx=0 tx=0).stop 2024-09-21T07:45:47.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.720+0000 7f2d52400640 1 -- 172.21.15.39:0/1980641324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d4c15b8d0 msgr2=0x7f2d4c1f9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:47.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.720+0000 7f2d52400640 1 --2- 172.21.15.39:0/1980641324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d4c15b8d0 0x7f2d4c1f9070 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f2d4400c750 tx=0x7f2d44003ba0 comp rx=0 tx=0).stop 2024-09-21T07:45:47.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.720+0000 7f2d52400640 1 -- 172.21.15.39:0/1980641324 shutdown_connections 2024-09-21T07:45:47.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.720+0000 7f2d52400640 1 --2- 172.21.15.39:0/1980641324 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2d28078b90 0x7f2d2807b0a0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:47.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.720+0000 7f2d52400640 1 --2- 172.21.15.39:0/1980641324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d4c15b8d0 0x7f2d4c1f9070 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:47.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.720+0000 7f2d52400640 1 --2- 172.21.15.39:0/1980641324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d4c15f4c0 0x7f2d4c15b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:47.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.720+0000 7f2d52400640 1 --2- 172.21.15.39:0/1980641324 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d30005250 0x7f2d4c150950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:47.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.720+0000 7f2d52400640 1 -- 172.21.15.39:0/1980641324 >> 172.21.15.39:0/1980641324 conn(0x7f2d4c09d7d0 msgr2=0x7f2d4c06dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:47.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.721+0000 7f2d52400640 1 -- 172.21.15.39:0/1980641324 shutdown_connections 2024-09-21T07:45:47.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:47.721+0000 7f2d52400640 1 -- 172.21.15.39:0/1980641324 wait complete. 2024-09-21T07:45:47.746 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":534,"stamp":"2024-09-21T07:45:45.788495+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":379,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":303528,"kb_used_data":16552,"kb_used_omap":12,"kb_used_meta":286771,"kb_avail":754671192,"statfs":{"total":773094113280,"available":772783300608,"internally_reserved":0,"allocated":16949248,"data_stored":9417804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":293654064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.123580"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259902+0000","last_change":"2024-09-21T07:45:37.693693+0000","last_active":"2024-09-21T07:45:39.259902+0000","last_peered":"2024-09-21T07:45:39.259902+0000","last_clean":"2024-09-21T07:45:39.259902+0000","last_became_active":"2024-09-21T07:45:37.693371+0000","last_became_peered":"2024-09-21T07:45:37.693371+0000","last_unstale":"2024-09-21T07:45:39.259902+0000","last_undegraded":"2024-09-21T07:45:39.259902+0000","last_fullsized":"2024-09-21T07:45:39.259902+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:38.801281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.815832+0000","last_change":"2024-09-21T07:45:37.676938+0000","last_active":"2024-09-21T07:45:38.815832+0000","last_peered":"2024-09-21T07:45:38.815832+0000","last_clean":"2024-09-21T07:45:38.815832+0000","last_became_active":"2024-09-21T07:45:37.676598+0000","last_became_peered":"2024-09-21T07:45:37.676598+0000","last_unstale":"2024-09-21T07:45:38.815832+0000","last_undegraded":"2024-09-21T07:45:38.815832+0000","last_fullsized":"2024-09-21T07:45:38.815832+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:32.304761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259838+0000","last_change":"2024-09-21T07:45:37.697566+0000","last_active":"2024-09-21T07:45:39.259838+0000","last_peered":"2024-09-21T07:45:39.259838+0000","last_clean":"2024-09-21T07:45:39.259838+0000","last_became_active":"2024-09-21T07:45:37.697108+0000","last_became_peered":"2024-09-21T07:45:37.697108+0000","last_unstale":"2024-09-21T07:45:39.259838+0000","last_undegraded":"2024-09-21T07:45:39.259838+0000","last_fullsized":"2024-09-21T07:45:39.259838+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:32.684544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.815383+0000","last_change":"2024-09-21T07:45:37.693927+0000","last_active":"2024-09-21T07:45:38.815383+0000","last_peered":"2024-09-21T07:45:38.815383+0000","last_clean":"2024-09-21T07:45:38.815383+0000","last_became_active":"2024-09-21T07:45:37.691663+0000","last_became_peered":"2024-09-21T07:45:37.691663+0000","last_unstale":"2024-09-21T07:45:38.815383+0000","last_undegraded":"2024-09-21T07:45:38.815383+0000","last_fullsized":"2024-09-21T07:45:38.815383+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:49.194272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689578+0000","last_change":"2024-09-21T07:45:37.706582+0000","last_active":"2024-09-21T07:45:38.689578+0000","last_peered":"2024-09-21T07:45:38.689578+0000","last_clean":"2024-09-21T07:45:38.689578+0000","last_became_active":"2024-09-21T07:45:37.706243+0000","last_became_peered":"2024-09-21T07:45:37.706243+0000","last_unstale":"2024-09-21T07:45:38.689578+0000","last_undegraded":"2024-09-21T07:45:38.689578+0000","last_fullsized":"2024-09-21T07:45:38.689578+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:20:35.055835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.257960+0000","last_change":"2024-09-21T07:45:37.696514+0000","last_active":"2024-09-21T07:45:39.257960+0000","last_peered":"2024-09-21T07:45:39.257960+0000","last_clean":"2024-09-21T07:45:39.257960+0000","last_became_active":"2024-09-21T07:45:37.694092+0000","last_became_peered":"2024-09-21T07:45:37.694092+0000","last_unstale":"2024-09-21T07:45:39.257960+0000","last_undegraded":"2024-09-21T07:45:39.257960+0000","last_fullsized":"2024-09-21T07:45:39.257960+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:01:39.096631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688574+0000","last_change":"2024-09-21T07:45:37.695651+0000","last_active":"2024-09-21T07:45:38.688574+0000","last_peered":"2024-09-21T07:45:38.688574+0000","last_clean":"2024-09-21T07:45:38.688574+0000","last_became_active":"2024-09-21T07:45:37.695288+0000","last_became_peered":"2024-09-21T07:45:37.695288+0000","last_unstale":"2024-09-21T07:45:38.688574+0000","last_undegraded":"2024-09-21T07:45:38.688574+0000","last_fullsized":"2024-09-21T07:45:38.688574+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:05.514879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.687461+0000","last_change":"2024-09-21T07:45:37.694980+0000","last_active":"2024-09-21T07:45:38.687461+0000","last_peered":"2024-09-21T07:45:38.687461+0000","last_clean":"2024-09-21T07:45:38.687461+0000","last_became_active":"2024-09-21T07:45:37.689627+0000","last_became_peered":"2024-09-21T07:45:37.689627+0000","last_unstale":"2024-09-21T07:45:38.687461+0000","last_undegraded":"2024-09-21T07:45:38.687461+0000","last_fullsized":"2024-09-21T07:45:38.687461+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:32:36.418482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259474+0000","last_change":"2024-09-21T07:45:37.681699+0000","last_active":"2024-09-21T07:45:39.259474+0000","last_peered":"2024-09-21T07:45:39.259474+0000","last_clean":"2024-09-21T07:45:39.259474+0000","last_became_active":"2024-09-21T07:45:37.681397+0000","last_became_peered":"2024-09-21T07:45:37.681397+0000","last_unstale":"2024-09-21T07:45:39.259474+0000","last_undegraded":"2024-09-21T07:45:39.259474+0000","last_fullsized":"2024-09-21T07:45:39.259474+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:39:14.136174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.830238+0000","last_change":"2024-09-21T07:45:37.693068+0000","last_active":"2024-09-21T07:45:38.830238+0000","last_peered":"2024-09-21T07:45:38.830238+0000","last_clean":"2024-09-21T07:45:38.830238+0000","last_became_active":"2024-09-21T07:45:37.692707+0000","last_became_peered":"2024-09-21T07:45:37.692707+0000","last_unstale":"2024-09-21T07:45:38.830238+0000","last_undegraded":"2024-09-21T07:45:38.830238+0000","last_fullsized":"2024-09-21T07:45:38.830238+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:23.625237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.815466+0000","last_change":"2024-09-21T07:45:37.693691+0000","last_active":"2024-09-21T07:45:38.815466+0000","last_peered":"2024-09-21T07:45:38.815466+0000","last_clean":"2024-09-21T07:45:38.815466+0000","last_became_active":"2024-09-21T07:45:37.693332+0000","last_became_peered":"2024-09-21T07:45:37.693332+0000","last_unstale":"2024-09-21T07:45:38.815466+0000","last_undegraded":"2024-09-21T07:45:38.815466+0000","last_fullsized":"2024-09-21T07:45:38.815466+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:38.976486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258353+0000","last_change":"2024-09-21T07:45:37.708847+0000","last_active":"2024-09-21T07:45:39.258353+0000","last_peered":"2024-09-21T07:45:39.258353+0000","last_clean":"2024-09-21T07:45:39.258353+0000","last_became_active":"2024-09-21T07:45:37.708379+0000","last_became_peered":"2024-09-21T07:45:37.708379+0000","last_unstale":"2024-09-21T07:45:39.258353+0000","last_undegraded":"2024-09-21T07:45:39.258353+0000","last_fullsized":"2024-09-21T07:45:39.258353+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:11:00.052134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259131+0000","last_change":"2024-09-21T07:45:37.699762+0000","last_active":"2024-09-21T07:45:39.259131+0000","last_peered":"2024-09-21T07:45:39.259131+0000","last_clean":"2024-09-21T07:45:39.259131+0000","last_became_active":"2024-09-21T07:45:37.698405+0000","last_became_peered":"2024-09-21T07:45:37.698405+0000","last_unstale":"2024-09-21T07:45:39.259131+0000","last_undegraded":"2024-09-21T07:45:39.259131+0000","last_fullsized":"2024-09-21T07:45:39.259131+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:40.095743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258837+0000","last_change":"2024-09-21T07:45:37.704433+0000","last_active":"2024-09-21T07:45:39.258837+0000","last_peered":"2024-09-21T07:45:39.258837+0000","last_clean":"2024-09-21T07:45:39.258837+0000","last_became_active":"2024-09-21T07:45:37.704117+0000","last_became_peered":"2024-09-21T07:45:37.704117+0000","last_unstale":"2024-09-21T07:45:39.258837+0000","last_undegraded":"2024-09-21T07:45:39.258837+0000","last_fullsized":"2024-09-21T07:45:39.258837+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:50.140124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258124+0000","last_change":"2024-09-21T07:45:37.706791+0000","last_active":"2024-09-21T07:45:39.258124+0000","last_peered":"2024-09-21T07:45:39.258124+0000","last_clean":"2024-09-21T07:45:39.258124+0000","last_became_active":"2024-09-21T07:45:37.706461+0000","last_became_peered":"2024-09-21T07:45:37.706461+0000","last_unstale":"2024-09-21T07:45:39.258124+0000","last_undegraded":"2024-09-21T07:45:39.258124+0000","last_fullsized":"2024-09-21T07:45:39.258124+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:09.660927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.257805+0000","last_change":"2024-09-21T07:45:37.690378+0000","last_active":"2024-09-21T07:45:39.257805+0000","last_peered":"2024-09-21T07:45:39.257805+0000","last_clean":"2024-09-21T07:45:39.257805+0000","last_became_active":"2024-09-21T07:45:37.683092+0000","last_became_peered":"2024-09-21T07:45:37.683092+0000","last_unstale":"2024-09-21T07:45:39.257805+0000","last_undegraded":"2024-09-21T07:45:39.257805+0000","last_fullsized":"2024-09-21T07:45:39.257805+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:40.878568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259706+0000","last_change":"2024-09-21T07:45:37.685191+0000","last_active":"2024-09-21T07:45:39.259706+0000","last_peered":"2024-09-21T07:45:39.259706+0000","last_clean":"2024-09-21T07:45:39.259706+0000","last_became_active":"2024-09-21T07:45:37.684765+0000","last_became_peered":"2024-09-21T07:45:37.684765+0000","last_unstale":"2024-09-21T07:45:39.259706+0000","last_undegraded":"2024-09-21T07:45:39.259706+0000","last_fullsized":"2024-09-21T07:45:39.259706+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:32.102165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689982+0000","last_change":"2024-09-21T07:45:37.704608+0000","last_active":"2024-09-21T07:45:38.689982+0000","last_peered":"2024-09-21T07:45:38.689982+0000","last_clean":"2024-09-21T07:45:38.689982+0000","last_became_active":"2024-09-21T07:45:37.703051+0000","last_became_peered":"2024-09-21T07:45:37.703051+0000","last_unstale":"2024-09-21T07:45:38.689982+0000","last_undegraded":"2024-09-21T07:45:38.689982+0000","last_fullsized":"2024-09-21T07:45:38.689982+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:33.889448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690451+0000","last_change":"2024-09-21T07:45:37.707249+0000","last_active":"2024-09-21T07:45:38.690451+0000","last_peered":"2024-09-21T07:45:38.690451+0000","last_clean":"2024-09-21T07:45:38.690451+0000","last_became_active":"2024-09-21T07:45:37.706656+0000","last_became_peered":"2024-09-21T07:45:37.706656+0000","last_unstale":"2024-09-21T07:45:38.690451+0000","last_undegraded":"2024-09-21T07:45:38.690451+0000","last_fullsized":"2024-09-21T07:45:38.690451+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:07.329504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689772+0000","last_change":"2024-09-21T07:45:37.698151+0000","last_active":"2024-09-21T07:45:38.689772+0000","last_peered":"2024-09-21T07:45:38.689772+0000","last_clean":"2024-09-21T07:45:38.689772+0000","last_became_active":"2024-09-21T07:45:37.697818+0000","last_became_peered":"2024-09-21T07:45:37.697818+0000","last_unstale":"2024-09-21T07:45:38.689772+0000","last_undegraded":"2024-09-21T07:45:38.689772+0000","last_fullsized":"2024-09-21T07:45:38.689772+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:57.867727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.815627+0000","last_change":"2024-09-21T07:45:37.675287+0000","last_active":"2024-09-21T07:45:38.815627+0000","last_peered":"2024-09-21T07:45:38.815627+0000","last_clean":"2024-09-21T07:45:38.815627+0000","last_became_active":"2024-09-21T07:45:37.674891+0000","last_became_peered":"2024-09-21T07:45:37.674891+0000","last_unstale":"2024-09-21T07:45:38.815627+0000","last_undegraded":"2024-09-21T07:45:38.815627+0000","last_fullsized":"2024-09-21T07:45:38.815627+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:45.842877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258544+0000","last_change":"2024-09-21T07:45:37.695767+0000","last_active":"2024-09-21T07:45:39.258544+0000","last_peered":"2024-09-21T07:45:39.258544+0000","last_clean":"2024-09-21T07:45:39.258544+0000","last_became_active":"2024-09-21T07:45:37.695487+0000","last_became_peered":"2024-09-21T07:45:37.695487+0000","last_unstale":"2024-09-21T07:45:39.258544+0000","last_undegraded":"2024-09-21T07:45:39.258544+0000","last_fullsized":"2024-09-21T07:45:39.258544+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:17.845096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.815270+0000","last_change":"2024-09-21T07:45:37.678052+0000","last_active":"2024-09-21T07:45:38.815270+0000","last_peered":"2024-09-21T07:45:38.815270+0000","last_clean":"2024-09-21T07:45:38.815270+0000","last_became_active":"2024-09-21T07:45:37.677738+0000","last_became_peered":"2024-09-21T07:45:37.677738+0000","last_unstale":"2024-09-21T07:45:38.815270+0000","last_undegraded":"2024-09-21T07:45:38.815270+0000","last_fullsized":"2024-09-21T07:45:38.815270+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:43:53.088002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.257910+0000","last_change":"2024-09-21T07:45:37.678436+0000","last_active":"2024-09-21T07:45:39.257910+0000","last_peered":"2024-09-21T07:45:39.257910+0000","last_clean":"2024-09-21T07:45:39.257910+0000","last_became_active":"2024-09-21T07:45:37.678086+0000","last_became_peered":"2024-09-21T07:45:37.678086+0000","last_unstale":"2024-09-21T07:45:39.257910+0000","last_undegraded":"2024-09-21T07:45:39.257910+0000","last_fullsized":"2024-09-21T07:45:39.257910+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:06:54.903590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.260024+0000","last_change":"2024-09-21T07:45:37.701506+0000","last_active":"2024-09-21T07:45:39.260024+0000","last_peered":"2024-09-21T07:45:39.260024+0000","last_clean":"2024-09-21T07:45:39.260024+0000","last_became_active":"2024-09-21T07:45:37.700952+0000","last_became_peered":"2024-09-21T07:45:37.700952+0000","last_unstale":"2024-09-21T07:45:39.260024+0000","last_undegraded":"2024-09-21T07:45:39.260024+0000","last_fullsized":"2024-09-21T07:45:39.260024+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:13.756567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259704+0000","last_change":"2024-09-21T07:45:37.697193+0000","last_active":"2024-09-21T07:45:39.259704+0000","last_peered":"2024-09-21T07:45:39.259704+0000","last_clean":"2024-09-21T07:45:39.259704+0000","last_became_active":"2024-09-21T07:45:37.696657+0000","last_became_peered":"2024-09-21T07:45:37.696657+0000","last_unstale":"2024-09-21T07:45:39.259704+0000","last_undegraded":"2024-09-21T07:45:39.259704+0000","last_fullsized":"2024-09-21T07:45:39.259704+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:14.864957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690561+0000","last_change":"2024-09-21T07:45:37.706976+0000","last_active":"2024-09-21T07:45:38.690561+0000","last_peered":"2024-09-21T07:45:38.690561+0000","last_clean":"2024-09-21T07:45:38.690561+0000","last_became_active":"2024-09-21T07:45:37.706115+0000","last_became_peered":"2024-09-21T07:45:37.706115+0000","last_unstale":"2024-09-21T07:45:38.690561+0000","last_undegraded":"2024-09-21T07:45:38.690561+0000","last_fullsized":"2024-09-21T07:45:38.690561+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:32.304761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.829565+0000","last_change":"2024-09-21T07:45:37.695282+0000","last_active":"2024-09-21T07:45:38.829565+0000","last_peered":"2024-09-21T07:45:38.829565+0000","last_clean":"2024-09-21T07:45:38.829565+0000","last_became_active":"2024-09-21T07:45:37.694929+0000","last_became_peered":"2024-09-21T07:45:37.694929+0000","last_unstale":"2024-09-21T07:45:38.829565+0000","last_undegraded":"2024-09-21T07:45:38.829565+0000","last_fullsized":"2024-09-21T07:45:38.829565+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:06:57.817032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258976+0000","last_change":"2024-09-21T07:45:37.694377+0000","last_active":"2024-09-21T07:45:39.258976+0000","last_peered":"2024-09-21T07:45:39.258976+0000","last_clean":"2024-09-21T07:45:39.258976+0000","last_became_active":"2024-09-21T07:45:37.693987+0000","last_became_peered":"2024-09-21T07:45:37.693987+0000","last_unstale":"2024-09-21T07:45:39.258976+0000","last_undegraded":"2024-09-21T07:45:39.258976+0000","last_fullsized":"2024-09-21T07:45:39.258976+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:15.146497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259076+0000","last_change":"2024-09-21T07:45:37.700110+0000","last_active":"2024-09-21T07:45:39.259076+0000","last_peered":"2024-09-21T07:45:39.259076+0000","last_clean":"2024-09-21T07:45:39.259076+0000","last_became_active":"2024-09-21T07:45:37.699820+0000","last_became_peered":"2024-09-21T07:45:37.699820+0000","last_unstale":"2024-09-21T07:45:39.259076+0000","last_undegraded":"2024-09-21T07:45:39.259076+0000","last_fullsized":"2024-09-21T07:45:39.259076+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:44.462069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.687575+0000","last_change":"2024-09-21T07:45:37.685098+0000","last_active":"2024-09-21T07:45:38.687575+0000","last_peered":"2024-09-21T07:45:38.687575+0000","last_clean":"2024-09-21T07:45:38.687575+0000","last_became_active":"2024-09-21T07:45:37.684748+0000","last_became_peered":"2024-09-21T07:45:37.684748+0000","last_unstale":"2024-09-21T07:45:38.687575+0000","last_undegraded":"2024-09-21T07:45:38.687575+0000","last_fullsized":"2024-09-21T07:45:38.687575+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:51:22.048251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.829291+0000","last_change":"2024-09-21T07:45:37.686973+0000","last_active":"2024-09-21T07:45:38.829291+0000","last_peered":"2024-09-21T07:45:38.829291+0000","last_clean":"2024-09-21T07:45:38.829291+0000","last_became_active":"2024-09-21T07:45:37.686592+0000","last_became_peered":"2024-09-21T07:45:37.686592+0000","last_unstale":"2024-09-21T07:45:38.829291+0000","last_undegraded":"2024-09-21T07:45:38.829291+0000","last_fullsized":"2024-09-21T07:45:38.829291+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:32.366993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259156+0000","last_change":"2024-09-21T07:45:37.695865+0000","last_active":"2024-09-21T07:45:39.259156+0000","last_peered":"2024-09-21T07:45:39.259156+0000","last_clean":"2024-09-21T07:45:39.259156+0000","last_became_active":"2024-09-21T07:45:37.695558+0000","last_became_peered":"2024-09-21T07:45:37.695558+0000","last_unstale":"2024-09-21T07:45:39.259156+0000","last_undegraded":"2024-09-21T07:45:39.259156+0000","last_fullsized":"2024-09-21T07:45:39.259156+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:35.011339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688636+0000","last_change":"2024-09-21T07:45:36.697679+0000","last_active":"2024-09-21T07:45:38.688636+0000","last_peered":"2024-09-21T07:45:38.688636+0000","last_clean":"2024-09-21T07:45:38.688636+0000","last_became_active":"2024-09-21T07:45:36.697396+0000","last_became_peered":"2024-09-21T07:45:36.697396+0000","last_unstale":"2024-09-21T07:45:38.688636+0000","last_undegraded":"2024-09-21T07:45:38.688636+0000","last_fullsized":"2024-09-21T07:45:38.688636+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:04:54.765401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":191,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690879+0000","last_change":"2024-09-21T07:35:27.879605+0000","last_active":"2024-09-21T07:45:38.690879+0000","last_peered":"2024-09-21T07:45:38.690879+0000","last_clean":"2024-09-21T07:45:38.690879+0000","last_became_active":"2024-09-21T07:35:27.879274+0000","last_became_peered":"2024-09-21T07:35:27.879274+0000","last_unstale":"2024-09-21T07:45:38.690879+0000","last_undegraded":"2024-09-21T07:45:38.690879+0000","last_fullsized":"2024-09-21T07:45:38.690879+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:24.241636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":197,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689112+0000","last_change":"2024-09-21T07:35:29.491336+0000","last_active":"2024-09-21T07:45:38.689112+0000","last_peered":"2024-09-21T07:45:38.689112+0000","last_clean":"2024-09-21T07:45:38.689112+0000","last_became_active":"2024-09-21T07:35:26.866484+0000","last_became_peered":"2024-09-21T07:35:26.866484+0000","last_unstale":"2024-09-21T07:45:38.689112+0000","last_undegraded":"2024-09-21T07:45:38.689112+0000","last_fullsized":"2024-09-21T07:45:38.689112+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:10.449665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076010100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":377,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688135+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:45:38.688135+0000","last_peered":"2024-09-21T07:45:38.688135+0000","last_clean":"2024-09-21T07:45:38.688135+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:45:38.688135+0000","last_undegraded":"2024-09-21T07:45:38.688135+0000","last_fullsized":"2024-09-21T07:45:38.688135+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":256,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.816174+0000","last_change":"2024-09-21T07:34:24.539248+0000","last_active":"2024-09-21T07:45:38.816174+0000","last_peered":"2024-09-21T07:45:38.816174+0000","last_clean":"2024-09-21T07:45:38.816174+0000","last_became_active":"2024-09-21T07:34:23.273465+0000","last_became_peered":"2024-09-21T07:34:23.273465+0000","last_unstale":"2024-09-21T07:45:38.816174+0000","last_undegraded":"2024-09-21T07:45:38.816174+0000","last_fullsized":"2024-09-21T07:45:38.816174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:55.517452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060627700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258677+0000","last_change":"2024-09-21T07:45:37.695318+0000","last_active":"2024-09-21T07:45:39.258677+0000","last_peered":"2024-09-21T07:45:39.258677+0000","last_clean":"2024-09-21T07:45:39.258677+0000","last_became_active":"2024-09-21T07:45:37.695081+0000","last_became_peered":"2024-09-21T07:45:37.695081+0000","last_unstale":"2024-09-21T07:45:39.258677+0000","last_undegraded":"2024-09-21T07:45:39.258677+0000","last_fullsized":"2024-09-21T07:45:39.258677+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:56.672995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.815955+0000","last_change":"2024-09-21T07:45:36.700703+0000","last_active":"2024-09-21T07:45:38.815955+0000","last_peered":"2024-09-21T07:45:38.815955+0000","last_clean":"2024-09-21T07:45:38.815955+0000","last_became_active":"2024-09-21T07:45:36.700344+0000","last_became_peered":"2024-09-21T07:45:36.700344+0000","last_unstale":"2024-09-21T07:45:38.815955+0000","last_undegraded":"2024-09-21T07:45:38.815955+0000","last_fullsized":"2024-09-21T07:45:38.815955+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:42.885001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689081+0000","last_change":"2024-09-21T07:35:27.864492+0000","last_active":"2024-09-21T07:45:38.689081+0000","last_peered":"2024-09-21T07:45:38.689081+0000","last_clean":"2024-09-21T07:45:38.689081+0000","last_became_active":"2024-09-21T07:35:27.864176+0000","last_became_peered":"2024-09-21T07:35:27.864176+0000","last_unstale":"2024-09-21T07:45:38.689081+0000","last_undegraded":"2024-09-21T07:45:38.689081+0000","last_fullsized":"2024-09-21T07:45:38.689081+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:05.153467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":198,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.691142+0000","last_change":"2024-09-21T07:35:29.493678+0000","last_active":"2024-09-21T07:45:38.691142+0000","last_peered":"2024-09-21T07:45:38.691142+0000","last_clean":"2024-09-21T07:45:38.691142+0000","last_became_active":"2024-09-21T07:35:26.870749+0000","last_became_peered":"2024-09-21T07:35:26.870749+0000","last_unstale":"2024-09-21T07:45:38.691142+0000","last_undegraded":"2024-09-21T07:45:38.691142+0000","last_fullsized":"2024-09-21T07:45:38.691142+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:55.367877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010159570000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":246,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688084+0000","last_change":"2024-09-21T07:34:24.597274+0000","last_active":"2024-09-21T07:45:38.688084+0000","last_peered":"2024-09-21T07:45:38.688084+0000","last_clean":"2024-09-21T07:45:38.688084+0000","last_became_active":"2024-09-21T07:34:23.279560+0000","last_became_peered":"2024-09-21T07:34:23.279560+0000","last_unstale":"2024-09-21T07:45:38.688084+0000","last_undegraded":"2024-09-21T07:45:38.688084+0000","last_fullsized":"2024-09-21T07:45:38.688084+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:15.845265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059699799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.257960+0000","last_change":"2024-09-21T07:45:37.688438+0000","last_active":"2024-09-21T07:45:39.257960+0000","last_peered":"2024-09-21T07:45:39.257960+0000","last_clean":"2024-09-21T07:45:39.257960+0000","last_became_active":"2024-09-21T07:45:37.688141+0000","last_became_peered":"2024-09-21T07:45:37.688141+0000","last_unstale":"2024-09-21T07:45:39.257960+0000","last_undegraded":"2024-09-21T07:45:39.257960+0000","last_fullsized":"2024-09-21T07:45:39.257960+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:33.326739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258474+0000","last_change":"2024-09-21T07:45:36.695689+0000","last_active":"2024-09-21T07:45:39.258474+0000","last_peered":"2024-09-21T07:45:39.258474+0000","last_clean":"2024-09-21T07:45:39.258474+0000","last_became_active":"2024-09-21T07:45:36.695386+0000","last_became_peered":"2024-09-21T07:45:36.695386+0000","last_unstale":"2024-09-21T07:45:39.258474+0000","last_undegraded":"2024-09-21T07:45:39.258474+0000","last_fullsized":"2024-09-21T07:45:39.258474+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:46:14.029499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":191,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258322+0000","last_change":"2024-09-21T07:35:27.874387+0000","last_active":"2024-09-21T07:45:39.258322+0000","last_peered":"2024-09-21T07:45:39.258322+0000","last_clean":"2024-09-21T07:45:39.258322+0000","last_became_active":"2024-09-21T07:35:27.873277+0000","last_became_peered":"2024-09-21T07:35:27.873277+0000","last_unstale":"2024-09-21T07:45:39.258322+0000","last_undegraded":"2024-09-21T07:45:39.258322+0000","last_fullsized":"2024-09-21T07:45:39.258322+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:04.581463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":196,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688673+0000","last_change":"2024-09-21T07:35:29.387094+0000","last_active":"2024-09-21T07:45:38.688673+0000","last_peered":"2024-09-21T07:45:38.688673+0000","last_clean":"2024-09-21T07:45:38.688673+0000","last_became_active":"2024-09-21T07:35:26.874757+0000","last_became_peered":"2024-09-21T07:35:26.874757+0000","last_unstale":"2024-09-21T07:45:38.688673+0000","last_undegraded":"2024-09-21T07:45:38.688673+0000","last_fullsized":"2024-09-21T07:45:38.688673+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:33.542750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061065700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":252,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690132+0000","last_change":"2024-09-21T07:34:24.259906+0000","last_active":"2024-09-21T07:45:38.690132+0000","last_peered":"2024-09-21T07:45:38.690132+0000","last_clean":"2024-09-21T07:45:38.690132+0000","last_became_active":"2024-09-21T07:34:23.635282+0000","last_became_peered":"2024-09-21T07:34:23.635282+0000","last_unstale":"2024-09-21T07:45:38.690132+0000","last_undegraded":"2024-09-21T07:45:38.690132+0000","last_fullsized":"2024-09-21T07:45:38.690132+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:00.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001527664,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258930+0000","last_change":"2024-09-21T07:45:37.688779+0000","last_active":"2024-09-21T07:45:39.258930+0000","last_peered":"2024-09-21T07:45:39.258930+0000","last_clean":"2024-09-21T07:45:39.258930+0000","last_became_active":"2024-09-21T07:45:37.688359+0000","last_became_peered":"2024-09-21T07:45:37.688359+0000","last_unstale":"2024-09-21T07:45:39.258930+0000","last_undegraded":"2024-09-21T07:45:39.258930+0000","last_fullsized":"2024-09-21T07:45:39.258930+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:29:36.750167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"156'3","reported_seq":18,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.691053+0000","last_change":"2024-09-21T07:45:36.688037+0000","last_active":"2024-09-21T07:45:38.691053+0000","last_peered":"2024-09-21T07:45:38.691053+0000","last_clean":"2024-09-21T07:45:38.691053+0000","last_became_active":"2024-09-21T07:45:36.687731+0000","last_became_peered":"2024-09-21T07:45:36.687731+0000","last_unstale":"2024-09-21T07:45:38.691053+0000","last_undegraded":"2024-09-21T07:45:38.691053+0000","last_fullsized":"2024-09-21T07:45:38.691053+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:12:07.289510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":191,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259189+0000","last_change":"2024-09-21T07:35:27.873645+0000","last_active":"2024-09-21T07:45:39.259189+0000","last_peered":"2024-09-21T07:45:39.259189+0000","last_clean":"2024-09-21T07:45:39.259189+0000","last_became_active":"2024-09-21T07:35:27.873245+0000","last_became_peered":"2024-09-21T07:35:27.873245+0000","last_unstale":"2024-09-21T07:45:39.259189+0000","last_undegraded":"2024-09-21T07:45:39.259189+0000","last_fullsized":"2024-09-21T07:45:39.259189+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:55.090953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":198,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259570+0000","last_change":"2024-09-21T07:35:28.861744+0000","last_active":"2024-09-21T07:45:39.259570+0000","last_peered":"2024-09-21T07:45:39.259570+0000","last_clean":"2024-09-21T07:45:39.259570+0000","last_became_active":"2024-09-21T07:35:26.876462+0000","last_became_peered":"2024-09-21T07:35:26.876462+0000","last_unstale":"2024-09-21T07:45:39.259570+0000","last_undegraded":"2024-09-21T07:45:39.259570+0000","last_fullsized":"2024-09-21T07:45:39.259570+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:48.632568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012020920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":249,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.829502+0000","last_change":"2024-09-21T07:34:25.085454+0000","last_active":"2024-09-21T07:45:38.829502+0000","last_peered":"2024-09-21T07:45:38.829502+0000","last_clean":"2024-09-21T07:45:38.829502+0000","last_became_active":"2024-09-21T07:34:23.279205+0000","last_became_peered":"2024-09-21T07:34:23.279205+0000","last_unstale":"2024-09-21T07:45:38.829502+0000","last_undegraded":"2024-09-21T07:45:38.829502+0000","last_fullsized":"2024-09-21T07:45:38.829502+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:54.231666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095320599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259623+0000","last_change":"2024-09-21T07:45:37.693546+0000","last_active":"2024-09-21T07:45:39.259623+0000","last_peered":"2024-09-21T07:45:39.259623+0000","last_clean":"2024-09-21T07:45:39.259623+0000","last_became_active":"2024-09-21T07:45:37.693193+0000","last_became_peered":"2024-09-21T07:45:37.693193+0000","last_unstale":"2024-09-21T07:45:39.259623+0000","last_undegraded":"2024-09-21T07:45:39.259623+0000","last_fullsized":"2024-09-21T07:45:39.259623+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:13.178725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"156'6","reported_seq":21,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690072+0000","last_change":"2024-09-21T07:45:36.700878+0000","last_active":"2024-09-21T07:45:38.690072+0000","last_peered":"2024-09-21T07:45:38.690072+0000","last_clean":"2024-09-21T07:45:38.690072+0000","last_became_active":"2024-09-21T07:45:36.700484+0000","last_became_peered":"2024-09-21T07:45:36.700484+0000","last_unstale":"2024-09-21T07:45:38.690072+0000","last_undegraded":"2024-09-21T07:45:38.690072+0000","last_fullsized":"2024-09-21T07:45:38.690072+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:47.039079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688964+0000","last_change":"2024-09-21T07:35:27.861373+0000","last_active":"2024-09-21T07:45:38.688964+0000","last_peered":"2024-09-21T07:45:38.688964+0000","last_clean":"2024-09-21T07:45:38.688964+0000","last_became_active":"2024-09-21T07:35:27.861016+0000","last_became_peered":"2024-09-21T07:35:27.861016+0000","last_unstale":"2024-09-21T07:45:38.688964+0000","last_undegraded":"2024-09-21T07:45:38.688964+0000","last_fullsized":"2024-09-21T07:45:38.688964+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:56.043763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":199,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259347+0000","last_change":"2024-09-21T07:35:29.053677+0000","last_active":"2024-09-21T07:45:39.259347+0000","last_peered":"2024-09-21T07:45:39.259347+0000","last_clean":"2024-09-21T07:45:39.259347+0000","last_became_active":"2024-09-21T07:35:26.875496+0000","last_became_peered":"2024-09-21T07:35:26.875496+0000","last_unstale":"2024-09-21T07:45:39.259347+0000","last_undegraded":"2024-09-21T07:45:39.259347+0000","last_fullsized":"2024-09-21T07:45:39.259347+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:04.850784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065859000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":251,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258115+0000","last_change":"2024-09-21T07:34:25.028521+0000","last_active":"2024-09-21T07:45:39.258115+0000","last_peered":"2024-09-21T07:45:39.258115+0000","last_clean":"2024-09-21T07:45:39.258115+0000","last_became_active":"2024-09-21T07:34:23.635467+0000","last_became_peered":"2024-09-21T07:34:23.635467+0000","last_unstale":"2024-09-21T07:45:39.258115+0000","last_undegraded":"2024-09-21T07:45:39.258115+0000","last_fullsized":"2024-09-21T07:45:39.258115+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:33.729123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000660558,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.829911+0000","last_change":"2024-09-21T07:45:37.709721+0000","last_active":"2024-09-21T07:45:38.829911+0000","last_peered":"2024-09-21T07:45:38.829911+0000","last_clean":"2024-09-21T07:45:38.829911+0000","last_became_active":"2024-09-21T07:45:37.709405+0000","last_became_peered":"2024-09-21T07:45:37.709405+0000","last_unstale":"2024-09-21T07:45:38.829911+0000","last_undegraded":"2024-09-21T07:45:38.829911+0000","last_fullsized":"2024-09-21T07:45:38.829911+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:45.673393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"156'2","reported_seq":17,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259501+0000","last_change":"2024-09-21T07:45:36.697573+0000","last_active":"2024-09-21T07:45:39.259501+0000","last_peered":"2024-09-21T07:45:39.259501+0000","last_clean":"2024-09-21T07:45:39.259501+0000","last_became_active":"2024-09-21T07:45:36.697341+0000","last_became_peered":"2024-09-21T07:45:36.697341+0000","last_unstale":"2024-09-21T07:45:39.259501+0000","last_undegraded":"2024-09-21T07:45:39.259501+0000","last_fullsized":"2024-09-21T07:45:39.259501+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:48.026132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":191,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.829937+0000","last_change":"2024-09-21T07:35:27.871201+0000","last_active":"2024-09-21T07:45:38.829937+0000","last_peered":"2024-09-21T07:45:38.829937+0000","last_clean":"2024-09-21T07:45:38.829937+0000","last_became_active":"2024-09-21T07:35:27.870816+0000","last_became_peered":"2024-09-21T07:35:27.870816+0000","last_unstale":"2024-09-21T07:45:38.829937+0000","last_undegraded":"2024-09-21T07:45:38.829937+0000","last_fullsized":"2024-09-21T07:45:38.829937+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:04.787450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":197,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689594+0000","last_change":"2024-09-21T07:35:29.491874+0000","last_active":"2024-09-21T07:45:38.689594+0000","last_peered":"2024-09-21T07:45:38.689594+0000","last_clean":"2024-09-21T07:45:38.689594+0000","last_became_active":"2024-09-21T07:35:26.866381+0000","last_became_peered":"2024-09-21T07:35:26.866381+0000","last_unstale":"2024-09-21T07:45:38.689594+0000","last_undegraded":"2024-09-21T07:45:38.689594+0000","last_fullsized":"2024-09-21T07:45:38.689594+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:36.404647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073235399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":252,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690345+0000","last_change":"2024-09-21T07:34:24.260046+0000","last_active":"2024-09-21T07:45:38.690345+0000","last_peered":"2024-09-21T07:45:38.690345+0000","last_clean":"2024-09-21T07:45:38.690345+0000","last_became_active":"2024-09-21T07:34:23.273431+0000","last_became_peered":"2024-09-21T07:34:23.273431+0000","last_unstale":"2024-09-21T07:45:38.690345+0000","last_undegraded":"2024-09-21T07:45:38.690345+0000","last_fullsized":"2024-09-21T07:45:38.690345+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:18.876057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019803310000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.691105+0000","last_change":"2024-09-21T07:45:37.695290+0000","last_active":"2024-09-21T07:45:38.691105+0000","last_peered":"2024-09-21T07:45:38.691105+0000","last_clean":"2024-09-21T07:45:38.691105+0000","last_became_active":"2024-09-21T07:45:37.694954+0000","last_became_peered":"2024-09-21T07:45:37.694954+0000","last_unstale":"2024-09-21T07:45:38.691105+0000","last_undegraded":"2024-09-21T07:45:38.691105+0000","last_fullsized":"2024-09-21T07:45:38.691105+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:01:39.096631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259065+0000","last_change":"2024-09-21T07:45:36.684575+0000","last_active":"2024-09-21T07:45:39.259065+0000","last_peered":"2024-09-21T07:45:39.259065+0000","last_clean":"2024-09-21T07:45:39.259065+0000","last_became_active":"2024-09-21T07:45:36.683843+0000","last_became_peered":"2024-09-21T07:45:36.683843+0000","last_unstale":"2024-09-21T07:45:39.259065+0000","last_undegraded":"2024-09-21T07:45:39.259065+0000","last_fullsized":"2024-09-21T07:45:39.259065+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:39.029966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":191,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.830300+0000","last_change":"2024-09-21T07:35:27.871736+0000","last_active":"2024-09-21T07:45:38.830300+0000","last_peered":"2024-09-21T07:45:38.830300+0000","last_clean":"2024-09-21T07:45:38.830300+0000","last_became_active":"2024-09-21T07:35:27.871232+0000","last_became_peered":"2024-09-21T07:35:27.871232+0000","last_unstale":"2024-09-21T07:45:38.830300+0000","last_undegraded":"2024-09-21T07:45:38.830300+0000","last_fullsized":"2024-09-21T07:45:38.830300+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:04.419301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":197,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688913+0000","last_change":"2024-09-21T07:35:29.491246+0000","last_active":"2024-09-21T07:45:38.688913+0000","last_peered":"2024-09-21T07:45:38.688913+0000","last_clean":"2024-09-21T07:45:38.688913+0000","last_became_active":"2024-09-21T07:35:26.866783+0000","last_became_peered":"2024-09-21T07:35:26.866783+0000","last_unstale":"2024-09-21T07:45:38.688913+0000","last_undegraded":"2024-09-21T07:45:38.688913+0000","last_fullsized":"2024-09-21T07:45:38.688913+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:28.881914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098474099999999992,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":251,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259402+0000","last_change":"2024-09-21T07:34:25.028640+0000","last_active":"2024-09-21T07:45:39.259402+0000","last_peered":"2024-09-21T07:45:39.259402+0000","last_clean":"2024-09-21T07:45:39.259402+0000","last_became_active":"2024-09-21T07:34:23.260950+0000","last_became_peered":"2024-09-21T07:34:23.260950+0000","last_unstale":"2024-09-21T07:45:39.259402+0000","last_undegraded":"2024-09-21T07:45:39.259402+0000","last_fullsized":"2024-09-21T07:45:39.259402+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:23:42.121398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055135099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689869+0000","last_change":"2024-09-21T07:45:37.693385+0000","last_active":"2024-09-21T07:45:38.689869+0000","last_peered":"2024-09-21T07:45:38.689869+0000","last_clean":"2024-09-21T07:45:38.689869+0000","last_became_active":"2024-09-21T07:45:37.691927+0000","last_became_peered":"2024-09-21T07:45:37.691927+0000","last_unstale":"2024-09-21T07:45:38.689869+0000","last_undegraded":"2024-09-21T07:45:38.689869+0000","last_fullsized":"2024-09-21T07:45:38.689869+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:29.208442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"156'8","reported_seq":23,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258001+0000","last_change":"2024-09-21T07:45:36.681956+0000","last_active":"2024-09-21T07:45:39.258001+0000","last_peered":"2024-09-21T07:45:39.258001+0000","last_clean":"2024-09-21T07:45:39.258001+0000","last_became_active":"2024-09-21T07:45:36.681656+0000","last_became_peered":"2024-09-21T07:45:36.681656+0000","last_unstale":"2024-09-21T07:45:39.258001+0000","last_undegraded":"2024-09-21T07:45:39.258001+0000","last_fullsized":"2024-09-21T07:45:39.258001+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:17.358434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259762+0000","last_change":"2024-09-21T07:35:27.872027+0000","last_active":"2024-09-21T07:45:39.259762+0000","last_peered":"2024-09-21T07:45:39.259762+0000","last_clean":"2024-09-21T07:45:39.259762+0000","last_became_active":"2024-09-21T07:35:27.870426+0000","last_became_peered":"2024-09-21T07:35:27.870426+0000","last_unstale":"2024-09-21T07:45:39.259762+0000","last_undegraded":"2024-09-21T07:45:39.259762+0000","last_fullsized":"2024-09-21T07:45:39.259762+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":197,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689126+0000","last_change":"2024-09-21T07:35:29.491008+0000","last_active":"2024-09-21T07:45:38.689126+0000","last_peered":"2024-09-21T07:45:38.689126+0000","last_clean":"2024-09-21T07:45:38.689126+0000","last_became_active":"2024-09-21T07:35:26.866750+0000","last_became_peered":"2024-09-21T07:35:26.866750+0000","last_unstale":"2024-09-21T07:45:38.689126+0000","last_undegraded":"2024-09-21T07:45:38.689126+0000","last_fullsized":"2024-09-21T07:45:38.689126+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:25.490406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083529700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":252,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.691222+0000","last_change":"2024-09-21T07:34:24.260665+0000","last_active":"2024-09-21T07:45:38.691222+0000","last_peered":"2024-09-21T07:45:38.691222+0000","last_clean":"2024-09-21T07:45:38.691222+0000","last_became_active":"2024-09-21T07:34:23.635909+0000","last_became_peered":"2024-09-21T07:34:23.635909+0000","last_unstale":"2024-09-21T07:45:38.691222+0000","last_undegraded":"2024-09-21T07:45:38.691222+0000","last_fullsized":"2024-09-21T07:45:38.691222+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:44.536885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689417+0000","last_change":"2024-09-21T07:45:37.686269+0000","last_active":"2024-09-21T07:45:38.689417+0000","last_peered":"2024-09-21T07:45:38.689417+0000","last_clean":"2024-09-21T07:45:38.689417+0000","last_became_active":"2024-09-21T07:45:37.685971+0000","last_became_peered":"2024-09-21T07:45:37.685971+0000","last_unstale":"2024-09-21T07:45:38.689417+0000","last_undegraded":"2024-09-21T07:45:38.689417+0000","last_fullsized":"2024-09-21T07:45:38.689417+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:23.017715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"156'4","reported_seq":19,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259943+0000","last_change":"2024-09-21T07:45:36.695413+0000","last_active":"2024-09-21T07:45:39.259943+0000","last_peered":"2024-09-21T07:45:39.259943+0000","last_clean":"2024-09-21T07:45:39.259943+0000","last_became_active":"2024-09-21T07:45:36.695045+0000","last_became_peered":"2024-09-21T07:45:36.695045+0000","last_unstale":"2024-09-21T07:45:39.259943+0000","last_undegraded":"2024-09-21T07:45:39.259943+0000","last_fullsized":"2024-09-21T07:45:39.259943+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:58:37.328760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689492+0000","last_change":"2024-09-21T07:35:27.874772+0000","last_active":"2024-09-21T07:45:38.689492+0000","last_peered":"2024-09-21T07:45:38.689492+0000","last_clean":"2024-09-21T07:45:38.689492+0000","last_became_active":"2024-09-21T07:35:27.874434+0000","last_became_peered":"2024-09-21T07:35:27.874434+0000","last_unstale":"2024-09-21T07:45:38.689492+0000","last_undegraded":"2024-09-21T07:45:38.689492+0000","last_fullsized":"2024-09-21T07:45:38.689492+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:31.998755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":203,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.816113+0000","last_change":"2024-09-21T07:35:28.859153+0000","last_active":"2024-09-21T07:45:38.816113+0000","last_peered":"2024-09-21T07:45:38.816113+0000","last_clean":"2024-09-21T07:45:38.816113+0000","last_became_active":"2024-09-21T07:35:26.874428+0000","last_became_peered":"2024-09-21T07:35:26.874428+0000","last_unstale":"2024-09-21T07:45:38.816113+0000","last_undegraded":"2024-09-21T07:45:38.816113+0000","last_fullsized":"2024-09-21T07:45:38.816113+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:04.487621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048557299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":256,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.815840+0000","last_change":"2024-09-21T07:34:24.538845+0000","last_active":"2024-09-21T07:45:38.815840+0000","last_peered":"2024-09-21T07:45:38.815840+0000","last_clean":"2024-09-21T07:45:38.815840+0000","last_became_active":"2024-09-21T07:34:23.636372+0000","last_became_peered":"2024-09-21T07:34:23.636372+0000","last_unstale":"2024-09-21T07:45:38.815840+0000","last_undegraded":"2024-09-21T07:45:38.815840+0000","last_fullsized":"2024-09-21T07:45:38.815840+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:29.393813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841451,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.260055+0000","last_change":"2024-09-21T07:45:37.701732+0000","last_active":"2024-09-21T07:45:39.260055+0000","last_peered":"2024-09-21T07:45:39.260055+0000","last_clean":"2024-09-21T07:45:39.260055+0000","last_became_active":"2024-09-21T07:45:37.701407+0000","last_became_peered":"2024-09-21T07:45:37.701407+0000","last_unstale":"2024-09-21T07:45:39.260055+0000","last_undegraded":"2024-09-21T07:45:39.260055+0000","last_fullsized":"2024-09-21T07:45:39.260055+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:14.520707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"156'1","reported_seq":16,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690230+0000","last_change":"2024-09-21T07:45:36.698933+0000","last_active":"2024-09-21T07:45:38.690230+0000","last_peered":"2024-09-21T07:45:38.690230+0000","last_clean":"2024-09-21T07:45:38.690230+0000","last_became_active":"2024-09-21T07:45:36.698651+0000","last_became_peered":"2024-09-21T07:45:36.698651+0000","last_unstale":"2024-09-21T07:45:38.690230+0000","last_undegraded":"2024-09-21T07:45:38.690230+0000","last_fullsized":"2024-09-21T07:45:38.690230+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:17:31.377323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258710+0000","last_change":"2024-09-21T07:35:27.877801+0000","last_active":"2024-09-21T07:45:39.258710+0000","last_peered":"2024-09-21T07:45:39.258710+0000","last_clean":"2024-09-21T07:45:39.258710+0000","last_became_active":"2024-09-21T07:35:27.877464+0000","last_became_peered":"2024-09-21T07:35:27.877464+0000","last_unstale":"2024-09-21T07:45:39.258710+0000","last_undegraded":"2024-09-21T07:45:39.258710+0000","last_fullsized":"2024-09-21T07:45:39.258710+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:08.814929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":198,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.830348+0000","last_change":"2024-09-21T07:35:28.863732+0000","last_active":"2024-09-21T07:45:38.830348+0000","last_peered":"2024-09-21T07:45:38.830348+0000","last_clean":"2024-09-21T07:45:38.830348+0000","last_became_active":"2024-09-21T07:35:26.877392+0000","last_became_peered":"2024-09-21T07:35:26.877392+0000","last_unstale":"2024-09-21T07:45:38.830348+0000","last_undegraded":"2024-09-21T07:45:38.830348+0000","last_fullsized":"2024-09-21T07:45:38.830348+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071425700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":252,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689803+0000","last_change":"2024-09-21T07:34:24.259686+0000","last_active":"2024-09-21T07:45:38.689803+0000","last_peered":"2024-09-21T07:45:38.689803+0000","last_clean":"2024-09-21T07:45:38.689803+0000","last_became_active":"2024-09-21T07:34:23.274405+0000","last_became_peered":"2024-09-21T07:34:23.274405+0000","last_unstale":"2024-09-21T07:45:38.689803+0000","last_undegraded":"2024-09-21T07:45:38.689803+0000","last_fullsized":"2024-09-21T07:45:38.689803+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:26.304693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022009920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.815464+0000","last_change":"2024-09-21T07:45:37.678620+0000","last_active":"2024-09-21T07:45:38.815464+0000","last_peered":"2024-09-21T07:45:38.815464+0000","last_clean":"2024-09-21T07:45:38.815464+0000","last_became_active":"2024-09-21T07:45:37.678277+0000","last_became_peered":"2024-09-21T07:45:37.678277+0000","last_unstale":"2024-09-21T07:45:38.815464+0000","last_undegraded":"2024-09-21T07:45:38.815464+0000","last_fullsized":"2024-09-21T07:45:38.815464+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:50:43.432893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"156'2","reported_seq":17,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690904+0000","last_change":"2024-09-21T07:45:36.689252+0000","last_active":"2024-09-21T07:45:38.690904+0000","last_peered":"2024-09-21T07:45:38.690904+0000","last_clean":"2024-09-21T07:45:38.690904+0000","last_became_active":"2024-09-21T07:45:36.688915+0000","last_became_peered":"2024-09-21T07:45:36.688915+0000","last_unstale":"2024-09-21T07:45:38.690904+0000","last_undegraded":"2024-09-21T07:45:38.690904+0000","last_fullsized":"2024-09-21T07:45:38.690904+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:42.885001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":189,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.687721+0000","last_change":"2024-09-21T07:35:27.867336+0000","last_active":"2024-09-21T07:45:38.687721+0000","last_peered":"2024-09-21T07:45:38.687721+0000","last_clean":"2024-09-21T07:45:38.687721+0000","last_became_active":"2024-09-21T07:35:27.867015+0000","last_became_peered":"2024-09-21T07:35:27.867015+0000","last_unstale":"2024-09-21T07:45:38.687721+0000","last_undegraded":"2024-09-21T07:45:38.687721+0000","last_fullsized":"2024-09-21T07:45:38.687721+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:59.214139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":198,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.691008+0000","last_change":"2024-09-21T07:35:29.493137+0000","last_active":"2024-09-21T07:45:38.691008+0000","last_peered":"2024-09-21T07:45:38.691008+0000","last_clean":"2024-09-21T07:45:38.691008+0000","last_became_active":"2024-09-21T07:35:26.872230+0000","last_became_peered":"2024-09-21T07:35:26.872230+0000","last_unstale":"2024-09-21T07:45:38.691008+0000","last_undegraded":"2024-09-21T07:45:38.691008+0000","last_fullsized":"2024-09-21T07:45:38.691008+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:24.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001197449,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":252,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688024+0000","last_change":"2024-09-21T07:34:24.258754+0000","last_active":"2024-09-21T07:45:38.688024+0000","last_peered":"2024-09-21T07:45:38.688024+0000","last_clean":"2024-09-21T07:45:38.688024+0000","last_became_active":"2024-09-21T07:34:23.272657+0000","last_became_peered":"2024-09-21T07:34:23.272657+0000","last_unstale":"2024-09-21T07:45:38.688024+0000","last_undegraded":"2024-09-21T07:45:38.688024+0000","last_fullsized":"2024-09-21T07:45:38.688024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:05.477099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067143200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259381+0000","last_change":"2024-09-21T07:45:37.704353+0000","last_active":"2024-09-21T07:45:39.259381+0000","last_peered":"2024-09-21T07:45:39.259381+0000","last_clean":"2024-09-21T07:45:39.259381+0000","last_became_active":"2024-09-21T07:45:37.703997+0000","last_became_peered":"2024-09-21T07:45:37.703997+0000","last_unstale":"2024-09-21T07:45:39.259381+0000","last_undegraded":"2024-09-21T07:45:39.259381+0000","last_fullsized":"2024-09-21T07:45:39.259381+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:54:12.637245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"156'2","reported_seq":17,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690236+0000","last_change":"2024-09-21T07:45:36.687920+0000","last_active":"2024-09-21T07:45:38.690236+0000","last_peered":"2024-09-21T07:45:38.690236+0000","last_clean":"2024-09-21T07:45:38.690236+0000","last_became_active":"2024-09-21T07:45:36.687561+0000","last_became_peered":"2024-09-21T07:45:36.687561+0000","last_unstale":"2024-09-21T07:45:38.690236+0000","last_undegraded":"2024-09-21T07:45:38.690236+0000","last_fullsized":"2024-09-21T07:45:38.690236+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:59.601883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":191,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.830034+0000","last_change":"2024-09-21T07:35:27.876021+0000","last_active":"2024-09-21T07:45:38.830034+0000","last_peered":"2024-09-21T07:45:38.830034+0000","last_clean":"2024-09-21T07:45:38.830034+0000","last_became_active":"2024-09-21T07:35:27.875724+0000","last_became_peered":"2024-09-21T07:35:27.875724+0000","last_unstale":"2024-09-21T07:45:38.830034+0000","last_undegraded":"2024-09-21T07:45:38.830034+0000","last_fullsized":"2024-09-21T07:45:38.830034+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:03.022812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":198,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690256+0000","last_change":"2024-09-21T07:35:29.492570+0000","last_active":"2024-09-21T07:45:38.690256+0000","last_peered":"2024-09-21T07:45:38.690256+0000","last_clean":"2024-09-21T07:45:38.690256+0000","last_became_active":"2024-09-21T07:35:26.874316+0000","last_became_peered":"2024-09-21T07:35:26.874316+0000","last_unstale":"2024-09-21T07:45:38.690256+0000","last_undegraded":"2024-09-21T07:45:38.690256+0000","last_fullsized":"2024-09-21T07:45:38.690256+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:16.512323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011990760000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":250,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.829567+0000","last_change":"2024-09-21T07:34:25.085232+0000","last_active":"2024-09-21T07:45:38.829567+0000","last_peered":"2024-09-21T07:45:38.829567+0000","last_clean":"2024-09-21T07:45:38.829567+0000","last_became_active":"2024-09-21T07:34:23.277807+0000","last_became_peered":"2024-09-21T07:34:23.277807+0000","last_unstale":"2024-09-21T07:45:38.829567+0000","last_undegraded":"2024-09-21T07:45:38.829567+0000","last_fullsized":"2024-09-21T07:45:38.829567+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:36.362919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086619099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689375+0000","last_change":"2024-09-21T07:45:37.685148+0000","last_active":"2024-09-21T07:45:38.689375+0000","last_peered":"2024-09-21T07:45:38.689375+0000","last_clean":"2024-09-21T07:45:38.689375+0000","last_became_active":"2024-09-21T07:45:37.684774+0000","last_became_peered":"2024-09-21T07:45:37.684774+0000","last_unstale":"2024-09-21T07:45:38.689375+0000","last_undegraded":"2024-09-21T07:45:38.689375+0000","last_fullsized":"2024-09-21T07:45:38.689375+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:25.616419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"156'1","reported_seq":16,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.687687+0000","last_change":"2024-09-21T07:45:36.686891+0000","last_active":"2024-09-21T07:45:38.687687+0000","last_peered":"2024-09-21T07:45:38.687687+0000","last_clean":"2024-09-21T07:45:38.687687+0000","last_became_active":"2024-09-21T07:45:36.684124+0000","last_became_peered":"2024-09-21T07:45:36.684124+0000","last_unstale":"2024-09-21T07:45:38.687687+0000","last_undegraded":"2024-09-21T07:45:38.687687+0000","last_fullsized":"2024-09-21T07:45:38.687687+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:00.150983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258873+0000","last_change":"2024-09-21T07:35:27.872798+0000","last_active":"2024-09-21T07:45:39.258873+0000","last_peered":"2024-09-21T07:45:39.258873+0000","last_clean":"2024-09-21T07:45:39.258873+0000","last_became_active":"2024-09-21T07:35:27.872483+0000","last_became_peered":"2024-09-21T07:35:27.872483+0000","last_unstale":"2024-09-21T07:45:39.258873+0000","last_undegraded":"2024-09-21T07:45:39.258873+0000","last_fullsized":"2024-09-21T07:45:39.258873+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:11.933469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":197,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259383+0000","last_change":"2024-09-21T07:35:29.206723+0000","last_active":"2024-09-21T07:45:39.259383+0000","last_peered":"2024-09-21T07:45:39.259383+0000","last_clean":"2024-09-21T07:45:39.259383+0000","last_became_active":"2024-09-21T07:35:26.872377+0000","last_became_peered":"2024-09-21T07:35:26.872377+0000","last_unstale":"2024-09-21T07:45:39.259383+0000","last_undegraded":"2024-09-21T07:45:39.259383+0000","last_fullsized":"2024-09-21T07:45:39.259383+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:29.086862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083077200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":246,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689123+0000","last_change":"2024-09-21T07:34:24.597672+0000","last_active":"2024-09-21T07:45:38.689123+0000","last_peered":"2024-09-21T07:45:38.689123+0000","last_clean":"2024-09-21T07:45:38.689123+0000","last_became_active":"2024-09-21T07:34:23.267770+0000","last_became_peered":"2024-09-21T07:34:23.267770+0000","last_unstale":"2024-09-21T07:45:38.689123+0000","last_undegraded":"2024-09-21T07:45:38.689123+0000","last_fullsized":"2024-09-21T07:45:38.689123+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:51.629476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086378500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689895+0000","last_change":"2024-09-21T07:45:37.691898+0000","last_active":"2024-09-21T07:45:38.689895+0000","last_peered":"2024-09-21T07:45:38.689895+0000","last_clean":"2024-09-21T07:45:38.689895+0000","last_became_active":"2024-09-21T07:45:37.691507+0000","last_became_peered":"2024-09-21T07:45:37.691507+0000","last_unstale":"2024-09-21T07:45:38.689895+0000","last_undegraded":"2024-09-21T07:45:38.689895+0000","last_fullsized":"2024-09-21T07:45:38.689895+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:14:43.900850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"156'1","reported_seq":16,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689228+0000","last_change":"2024-09-21T07:45:36.694668+0000","last_active":"2024-09-21T07:45:38.689228+0000","last_peered":"2024-09-21T07:45:38.689228+0000","last_clean":"2024-09-21T07:45:38.689228+0000","last_became_active":"2024-09-21T07:45:36.694405+0000","last_became_peered":"2024-09-21T07:45:36.694405+0000","last_unstale":"2024-09-21T07:45:38.689228+0000","last_undegraded":"2024-09-21T07:45:38.689228+0000","last_fullsized":"2024-09-21T07:45:38.689228+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:08.552085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259996+0000","last_change":"2024-09-21T07:35:27.870121+0000","last_active":"2024-09-21T07:45:39.259996+0000","last_peered":"2024-09-21T07:45:39.259996+0000","last_clean":"2024-09-21T07:45:39.259996+0000","last_became_active":"2024-09-21T07:35:27.869811+0000","last_became_peered":"2024-09-21T07:35:27.869811+0000","last_unstale":"2024-09-21T07:45:39.259996+0000","last_undegraded":"2024-09-21T07:45:39.259996+0000","last_fullsized":"2024-09-21T07:45:39.259996+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:31.778461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":198,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259121+0000","last_change":"2024-09-21T07:35:28.860699+0000","last_active":"2024-09-21T07:45:39.259121+0000","last_peered":"2024-09-21T07:45:39.259121+0000","last_clean":"2024-09-21T07:45:39.259121+0000","last_became_active":"2024-09-21T07:35:26.876779+0000","last_became_peered":"2024-09-21T07:35:26.876779+0000","last_unstale":"2024-09-21T07:45:39.259121+0000","last_undegraded":"2024-09-21T07:45:39.259121+0000","last_fullsized":"2024-09-21T07:45:39.259121+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:28.286403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000913188,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":251,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259901+0000","last_change":"2024-09-21T07:34:24.260181+0000","last_active":"2024-09-21T07:45:39.259901+0000","last_peered":"2024-09-21T07:45:39.259901+0000","last_clean":"2024-09-21T07:45:39.259901+0000","last_became_active":"2024-09-21T07:34:23.275580+0000","last_became_peered":"2024-09-21T07:34:23.275580+0000","last_unstale":"2024-09-21T07:45:39.259901+0000","last_undegraded":"2024-09-21T07:45:39.259901+0000","last_fullsized":"2024-09-21T07:45:39.259901+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:49.857622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014706980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689850+0000","last_change":"2024-09-21T07:45:37.698246+0000","last_active":"2024-09-21T07:45:38.689850+0000","last_peered":"2024-09-21T07:45:38.689850+0000","last_clean":"2024-09-21T07:45:38.689850+0000","last_became_active":"2024-09-21T07:45:37.697792+0000","last_became_peered":"2024-09-21T07:45:37.697792+0000","last_unstale":"2024-09-21T07:45:38.689850+0000","last_undegraded":"2024-09-21T07:45:38.689850+0000","last_fullsized":"2024-09-21T07:45:38.689850+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:06.139752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.257812+0000","last_change":"2024-09-21T07:45:36.693696+0000","last_active":"2024-09-21T07:45:39.257812+0000","last_peered":"2024-09-21T07:45:39.257812+0000","last_clean":"2024-09-21T07:45:39.257812+0000","last_became_active":"2024-09-21T07:45:36.693211+0000","last_became_peered":"2024-09-21T07:45:36.693211+0000","last_unstale":"2024-09-21T07:45:39.257812+0000","last_undegraded":"2024-09-21T07:45:39.257812+0000","last_fullsized":"2024-09-21T07:45:39.257812+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:12:07.289510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":192,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258799+0000","last_change":"2024-09-21T07:35:27.875393+0000","last_active":"2024-09-21T07:45:39.258799+0000","last_peered":"2024-09-21T07:45:39.258799+0000","last_clean":"2024-09-21T07:45:39.258799+0000","last_became_active":"2024-09-21T07:35:27.874671+0000","last_became_peered":"2024-09-21T07:35:27.874671+0000","last_unstale":"2024-09-21T07:45:39.258799+0000","last_undegraded":"2024-09-21T07:45:39.258799+0000","last_fullsized":"2024-09-21T07:45:39.258799+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":197,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259239+0000","last_change":"2024-09-21T07:35:29.206296+0000","last_active":"2024-09-21T07:45:39.259239+0000","last_peered":"2024-09-21T07:45:39.259239+0000","last_clean":"2024-09-21T07:45:39.259239+0000","last_became_active":"2024-09-21T07:35:26.874575+0000","last_became_peered":"2024-09-21T07:35:26.874575+0000","last_unstale":"2024-09-21T07:45:39.259239+0000","last_undegraded":"2024-09-21T07:45:39.259239+0000","last_fullsized":"2024-09-21T07:45:39.259239+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:22.771154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061011599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":252,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690260+0000","last_change":"2024-09-21T07:34:24.259112+0000","last_active":"2024-09-21T07:45:38.690260+0000","last_peered":"2024-09-21T07:45:38.690260+0000","last_clean":"2024-09-21T07:45:38.690260+0000","last_became_active":"2024-09-21T07:34:23.635444+0000","last_became_peered":"2024-09-21T07:34:23.635444+0000","last_unstale":"2024-09-21T07:45:38.690260+0000","last_undegraded":"2024-09-21T07:45:38.690260+0000","last_fullsized":"2024-09-21T07:45:38.690260+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:46.255385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001952636,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690949+0000","last_change":"2024-09-21T07:45:37.689391+0000","last_active":"2024-09-21T07:45:38.690949+0000","last_peered":"2024-09-21T07:45:38.690949+0000","last_clean":"2024-09-21T07:45:38.690949+0000","last_became_active":"2024-09-21T07:45:37.688994+0000","last_became_peered":"2024-09-21T07:45:37.688994+0000","last_unstale":"2024-09-21T07:45:38.690949+0000","last_undegraded":"2024-09-21T07:45:38.690949+0000","last_fullsized":"2024-09-21T07:45:38.690949+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:06:54.903590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258563+0000","last_change":"2024-09-21T07:45:36.696284+0000","last_active":"2024-09-21T07:45:39.258563+0000","last_peered":"2024-09-21T07:45:39.258563+0000","last_clean":"2024-09-21T07:45:39.258563+0000","last_became_active":"2024-09-21T07:45:36.695952+0000","last_became_peered":"2024-09-21T07:45:36.695952+0000","last_unstale":"2024-09-21T07:45:39.258563+0000","last_undegraded":"2024-09-21T07:45:39.258563+0000","last_fullsized":"2024-09-21T07:45:39.258563+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:38.261108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":192,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259873+0000","last_change":"2024-09-21T07:35:27.878356+0000","last_active":"2024-09-21T07:45:39.259873+0000","last_peered":"2024-09-21T07:45:39.259873+0000","last_clean":"2024-09-21T07:45:39.259873+0000","last_became_active":"2024-09-21T07:35:27.877982+0000","last_became_peered":"2024-09-21T07:35:27.877982+0000","last_unstale":"2024-09-21T07:45:39.259873+0000","last_undegraded":"2024-09-21T07:45:39.259873+0000","last_fullsized":"2024-09-21T07:45:39.259873+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":200,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258236+0000","last_change":"2024-09-21T07:35:29.053131+0000","last_active":"2024-09-21T07:45:39.258236+0000","last_peered":"2024-09-21T07:45:39.258236+0000","last_clean":"2024-09-21T07:45:39.258236+0000","last_became_active":"2024-09-21T07:35:26.870516+0000","last_became_peered":"2024-09-21T07:35:26.870516+0000","last_unstale":"2024-09-21T07:45:39.258236+0000","last_undegraded":"2024-09-21T07:45:39.258236+0000","last_fullsized":"2024-09-21T07:45:39.258236+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:10.924491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070705699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":256,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.815898+0000","last_change":"2024-09-21T07:34:24.539152+0000","last_active":"2024-09-21T07:45:38.815898+0000","last_peered":"2024-09-21T07:45:38.815898+0000","last_clean":"2024-09-21T07:45:38.815898+0000","last_became_active":"2024-09-21T07:34:23.259573+0000","last_became_peered":"2024-09-21T07:34:23.259573+0000","last_unstale":"2024-09-21T07:45:38.815898+0000","last_undegraded":"2024-09-21T07:45:38.815898+0000","last_fullsized":"2024-09-21T07:45:38.815898+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:26.986900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058108400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689007+0000","last_change":"2024-09-21T07:45:37.689304+0000","last_active":"2024-09-21T07:45:38.689007+0000","last_peered":"2024-09-21T07:45:38.689007+0000","last_clean":"2024-09-21T07:45:38.689007+0000","last_became_active":"2024-09-21T07:45:37.688985+0000","last_became_peered":"2024-09-21T07:45:37.688985+0000","last_unstale":"2024-09-21T07:45:38.689007+0000","last_undegraded":"2024-09-21T07:45:38.689007+0000","last_fullsized":"2024-09-21T07:45:38.689007+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:03.596216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"156'2","reported_seq":17,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259672+0000","last_change":"2024-09-21T07:45:36.697335+0000","last_active":"2024-09-21T07:45:39.259672+0000","last_peered":"2024-09-21T07:45:39.259672+0000","last_clean":"2024-09-21T07:45:39.259672+0000","last_became_active":"2024-09-21T07:45:36.697036+0000","last_became_peered":"2024-09-21T07:45:36.697036+0000","last_unstale":"2024-09-21T07:45:39.259672+0000","last_undegraded":"2024-09-21T07:45:39.259672+0000","last_fullsized":"2024-09-21T07:45:39.259672+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:10:38.762615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":192,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258124+0000","last_change":"2024-09-21T07:35:27.873726+0000","last_active":"2024-09-21T07:45:39.258124+0000","last_peered":"2024-09-21T07:45:39.258124+0000","last_clean":"2024-09-21T07:45:39.258124+0000","last_became_active":"2024-09-21T07:35:27.871421+0000","last_became_peered":"2024-09-21T07:35:27.871421+0000","last_unstale":"2024-09-21T07:45:39.258124+0000","last_undegraded":"2024-09-21T07:45:39.258124+0000","last_fullsized":"2024-09-21T07:45:39.258124+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:00.168915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":198,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259737+0000","last_change":"2024-09-21T07:35:28.861032+0000","last_active":"2024-09-21T07:45:39.259737+0000","last_peered":"2024-09-21T07:45:39.259737+0000","last_clean":"2024-09-21T07:45:39.259737+0000","last_became_active":"2024-09-21T07:35:26.873149+0000","last_became_peered":"2024-09-21T07:35:26.873149+0000","last_unstale":"2024-09-21T07:45:39.259737+0000","last_undegraded":"2024-09-21T07:45:39.259737+0000","last_fullsized":"2024-09-21T07:45:39.259737+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:59.435981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011988369999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":251,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689271+0000","last_change":"2024-09-21T07:34:24.259902+0000","last_active":"2024-09-21T07:45:38.689271+0000","last_peered":"2024-09-21T07:45:38.689271+0000","last_clean":"2024-09-21T07:45:38.689271+0000","last_became_active":"2024-09-21T07:34:23.271902+0000","last_became_peered":"2024-09-21T07:34:23.271902+0000","last_unstale":"2024-09-21T07:45:38.689271+0000","last_undegraded":"2024-09-21T07:45:38.689271+0000","last_fullsized":"2024-09-21T07:45:38.689271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:59.439525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001686358,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258605+0000","last_change":"2024-09-21T07:45:37.691171+0000","last_active":"2024-09-21T07:45:39.258605+0000","last_peered":"2024-09-21T07:45:39.258605+0000","last_clean":"2024-09-21T07:45:39.258605+0000","last_became_active":"2024-09-21T07:45:37.690825+0000","last_became_peered":"2024-09-21T07:45:37.690825+0000","last_unstale":"2024-09-21T07:45:39.258605+0000","last_undegraded":"2024-09-21T07:45:39.258605+0000","last_fullsized":"2024-09-21T07:45:39.258605+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:34.131587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"157'1","reported_seq":16,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689021+0000","last_change":"2024-09-21T07:45:36.673147+0000","last_active":"2024-09-21T07:45:38.689021+0000","last_peered":"2024-09-21T07:45:38.689021+0000","last_clean":"2024-09-21T07:45:38.689021+0000","last_became_active":"2024-09-21T07:45:36.672816+0000","last_became_peered":"2024-09-21T07:45:36.672816+0000","last_unstale":"2024-09-21T07:45:38.689021+0000","last_undegraded":"2024-09-21T07:45:38.689021+0000","last_fullsized":"2024-09-21T07:45:38.689021+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:04:18.857870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":191,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.829343+0000","last_change":"2024-09-21T07:35:27.872637+0000","last_active":"2024-09-21T07:45:38.829343+0000","last_peered":"2024-09-21T07:45:38.829343+0000","last_clean":"2024-09-21T07:45:38.829343+0000","last_became_active":"2024-09-21T07:35:27.872149+0000","last_became_peered":"2024-09-21T07:35:27.872149+0000","last_unstale":"2024-09-21T07:45:38.829343+0000","last_undegraded":"2024-09-21T07:45:38.829343+0000","last_fullsized":"2024-09-21T07:45:38.829343+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:27.697872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":199,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690975+0000","last_change":"2024-09-21T07:35:29.493362+0000","last_active":"2024-09-21T07:45:38.690975+0000","last_peered":"2024-09-21T07:45:38.690975+0000","last_clean":"2024-09-21T07:45:38.690975+0000","last_became_active":"2024-09-21T07:35:26.873018+0000","last_became_peered":"2024-09-21T07:35:26.873018+0000","last_unstale":"2024-09-21T07:45:38.690975+0000","last_undegraded":"2024-09-21T07:45:38.690975+0000","last_fullsized":"2024-09-21T07:45:38.690975+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:02.460557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012496510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":251,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689489+0000","last_change":"2024-09-21T07:34:24.259662+0000","last_active":"2024-09-21T07:45:38.689489+0000","last_peered":"2024-09-21T07:45:38.689489+0000","last_clean":"2024-09-21T07:45:38.689489+0000","last_became_active":"2024-09-21T07:34:23.278812+0000","last_became_peered":"2024-09-21T07:34:23.278812+0000","last_unstale":"2024-09-21T07:45:38.689489+0000","last_undegraded":"2024-09-21T07:45:38.689489+0000","last_fullsized":"2024-09-21T07:45:38.689489+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:49.829974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356463,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688331+0000","last_change":"2024-09-21T07:45:37.701467+0000","last_active":"2024-09-21T07:45:38.688331+0000","last_peered":"2024-09-21T07:45:38.688331+0000","last_clean":"2024-09-21T07:45:38.688331+0000","last_became_active":"2024-09-21T07:45:37.696605+0000","last_became_peered":"2024-09-21T07:45:37.696605+0000","last_unstale":"2024-09-21T07:45:38.688331+0000","last_undegraded":"2024-09-21T07:45:38.688331+0000","last_fullsized":"2024-09-21T07:45:38.688331+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:08:37.797721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689740+0000","last_change":"2024-09-21T07:45:36.693420+0000","last_active":"2024-09-21T07:45:38.689740+0000","last_peered":"2024-09-21T07:45:38.689740+0000","last_clean":"2024-09-21T07:45:38.689740+0000","last_became_active":"2024-09-21T07:45:36.692969+0000","last_became_peered":"2024-09-21T07:45:36.692969+0000","last_unstale":"2024-09-21T07:45:38.689740+0000","last_undegraded":"2024-09-21T07:45:38.689740+0000","last_fullsized":"2024-09-21T07:45:38.689740+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:09:22.417758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":189,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.687613+0000","last_change":"2024-09-21T07:35:27.868448+0000","last_active":"2024-09-21T07:45:38.687613+0000","last_peered":"2024-09-21T07:45:38.687613+0000","last_clean":"2024-09-21T07:45:38.687613+0000","last_became_active":"2024-09-21T07:35:27.868105+0000","last_became_peered":"2024-09-21T07:35:27.868105+0000","last_unstale":"2024-09-21T07:45:38.687613+0000","last_undegraded":"2024-09-21T07:45:38.687613+0000","last_fullsized":"2024-09-21T07:45:38.687613+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:41.468702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":198,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690782+0000","last_change":"2024-09-21T07:35:29.492974+0000","last_active":"2024-09-21T07:45:38.690782+0000","last_peered":"2024-09-21T07:45:38.690782+0000","last_clean":"2024-09-21T07:45:38.690782+0000","last_became_active":"2024-09-21T07:35:26.872165+0000","last_became_peered":"2024-09-21T07:35:26.872165+0000","last_unstale":"2024-09-21T07:45:38.690782+0000","last_undegraded":"2024-09-21T07:45:38.690782+0000","last_fullsized":"2024-09-21T07:45:38.690782+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:59.912520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013795179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":249,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.829490+0000","last_change":"2024-09-21T07:34:25.084861+0000","last_active":"2024-09-21T07:45:38.829490+0000","last_peered":"2024-09-21T07:45:38.829490+0000","last_clean":"2024-09-21T07:45:38.829490+0000","last_became_active":"2024-09-21T07:34:23.277708+0000","last_became_peered":"2024-09-21T07:34:23.277708+0000","last_unstale":"2024-09-21T07:45:38.829490+0000","last_undegraded":"2024-09-21T07:45:38.829490+0000","last_fullsized":"2024-09-21T07:45:38.829490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:40.816488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000597566,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.829769+0000","last_change":"2024-09-21T07:45:37.691290+0000","last_active":"2024-09-21T07:45:38.829769+0000","last_peered":"2024-09-21T07:45:38.829769+0000","last_clean":"2024-09-21T07:45:38.829769+0000","last_became_active":"2024-09-21T07:45:37.690017+0000","last_became_peered":"2024-09-21T07:45:37.690017+0000","last_unstale":"2024-09-21T07:45:38.829769+0000","last_undegraded":"2024-09-21T07:45:38.829769+0000","last_fullsized":"2024-09-21T07:45:38.829769+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:14.769595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259263+0000","last_change":"2024-09-21T07:45:36.687709+0000","last_active":"2024-09-21T07:45:39.259263+0000","last_peered":"2024-09-21T07:45:39.259263+0000","last_clean":"2024-09-21T07:45:39.259263+0000","last_became_active":"2024-09-21T07:45:36.684703+0000","last_became_peered":"2024-09-21T07:45:36.684703+0000","last_unstale":"2024-09-21T07:45:39.259263+0000","last_undegraded":"2024-09-21T07:45:39.259263+0000","last_fullsized":"2024-09-21T07:45:39.259263+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:59:04.017206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":189,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688381+0000","last_change":"2024-09-21T07:35:27.874640+0000","last_active":"2024-09-21T07:45:38.688381+0000","last_peered":"2024-09-21T07:45:38.688381+0000","last_clean":"2024-09-21T07:45:38.688381+0000","last_became_active":"2024-09-21T07:35:27.874265+0000","last_became_peered":"2024-09-21T07:35:27.874265+0000","last_unstale":"2024-09-21T07:45:38.688381+0000","last_undegraded":"2024-09-21T07:45:38.688381+0000","last_fullsized":"2024-09-21T07:45:38.688381+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:45.754654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":197,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259024+0000","last_change":"2024-09-21T07:35:29.207392+0000","last_active":"2024-09-21T07:45:39.259024+0000","last_peered":"2024-09-21T07:45:39.259024+0000","last_clean":"2024-09-21T07:45:39.259024+0000","last_became_active":"2024-09-21T07:35:26.870917+0000","last_became_peered":"2024-09-21T07:35:26.870917+0000","last_unstale":"2024-09-21T07:45:39.259024+0000","last_undegraded":"2024-09-21T07:45:39.259024+0000","last_fullsized":"2024-09-21T07:45:39.259024+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:59.159937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059198499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":251,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689049+0000","last_change":"2024-09-21T07:34:24.258709+0000","last_active":"2024-09-21T07:45:38.689049+0000","last_peered":"2024-09-21T07:45:38.689049+0000","last_clean":"2024-09-21T07:45:38.689049+0000","last_became_active":"2024-09-21T07:34:23.279042+0000","last_became_peered":"2024-09-21T07:34:23.279042+0000","last_unstale":"2024-09-21T07:45:38.689049+0000","last_undegraded":"2024-09-21T07:45:38.689049+0000","last_fullsized":"2024-09-21T07:45:38.689049+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:01.762117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019282100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259047+0000","last_change":"2024-09-21T07:45:37.697340+0000","last_active":"2024-09-21T07:45:39.259047+0000","last_peered":"2024-09-21T07:45:39.259047+0000","last_clean":"2024-09-21T07:45:39.259047+0000","last_became_active":"2024-09-21T07:45:37.696970+0000","last_became_peered":"2024-09-21T07:45:37.696970+0000","last_unstale":"2024-09-21T07:45:39.259047+0000","last_undegraded":"2024-09-21T07:45:39.259047+0000","last_fullsized":"2024-09-21T07:45:39.259047+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:19:22.593298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.815240+0000","last_change":"2024-09-21T07:45:36.698592+0000","last_active":"2024-09-21T07:45:38.815240+0000","last_peered":"2024-09-21T07:45:38.815240+0000","last_clean":"2024-09-21T07:45:38.815240+0000","last_became_active":"2024-09-21T07:45:36.698011+0000","last_became_peered":"2024-09-21T07:45:36.698011+0000","last_unstale":"2024-09-21T07:45:38.815240+0000","last_undegraded":"2024-09-21T07:45:38.815240+0000","last_fullsized":"2024-09-21T07:45:38.815240+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:12:07.289510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":191,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.829818+0000","last_change":"2024-09-21T07:35:27.870260+0000","last_active":"2024-09-21T07:45:38.829818+0000","last_peered":"2024-09-21T07:45:38.829818+0000","last_clean":"2024-09-21T07:45:38.829818+0000","last_became_active":"2024-09-21T07:35:27.869448+0000","last_became_peered":"2024-09-21T07:35:27.869448+0000","last_unstale":"2024-09-21T07:45:38.829818+0000","last_undegraded":"2024-09-21T07:45:38.829818+0000","last_fullsized":"2024-09-21T07:45:38.829818+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:16.946790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":196,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688161+0000","last_change":"2024-09-21T07:35:29.386367+0000","last_active":"2024-09-21T07:45:38.688161+0000","last_peered":"2024-09-21T07:45:38.688161+0000","last_clean":"2024-09-21T07:45:38.688161+0000","last_became_active":"2024-09-21T07:35:26.879238+0000","last_became_peered":"2024-09-21T07:35:26.879238+0000","last_unstale":"2024-09-21T07:45:38.688161+0000","last_undegraded":"2024-09-21T07:45:38.688161+0000","last_fullsized":"2024-09-21T07:45:38.688161+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:16.528260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092223100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":249,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.830404+0000","last_change":"2024-09-21T07:34:25.085635+0000","last_active":"2024-09-21T07:45:38.830404+0000","last_peered":"2024-09-21T07:45:38.830404+0000","last_clean":"2024-09-21T07:45:38.830404+0000","last_became_active":"2024-09-21T07:34:23.273566+0000","last_became_peered":"2024-09-21T07:34:23.273566+0000","last_unstale":"2024-09-21T07:45:38.830404+0000","last_undegraded":"2024-09-21T07:45:38.830404+0000","last_fullsized":"2024-09-21T07:45:38.830404+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:14.142272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079897300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259060+0000","last_change":"2024-09-21T07:45:37.696449+0000","last_active":"2024-09-21T07:45:39.259060+0000","last_peered":"2024-09-21T07:45:39.259060+0000","last_clean":"2024-09-21T07:45:39.259060+0000","last_became_active":"2024-09-21T07:45:37.692664+0000","last_became_peered":"2024-09-21T07:45:37.692664+0000","last_unstale":"2024-09-21T07:45:39.259060+0000","last_undegraded":"2024-09-21T07:45:39.259060+0000","last_fullsized":"2024-09-21T07:45:39.259060+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:52.828679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689020+0000","last_change":"2024-09-21T07:45:36.685900+0000","last_active":"2024-09-21T07:45:38.689020+0000","last_peered":"2024-09-21T07:45:38.689020+0000","last_clean":"2024-09-21T07:45:38.689020+0000","last_became_active":"2024-09-21T07:45:36.685578+0000","last_became_peered":"2024-09-21T07:45:36.685578+0000","last_unstale":"2024-09-21T07:45:38.689020+0000","last_undegraded":"2024-09-21T07:45:38.689020+0000","last_fullsized":"2024-09-21T07:45:38.689020+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:15.970506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":191,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.830083+0000","last_change":"2024-09-21T07:35:27.878728+0000","last_active":"2024-09-21T07:45:38.830083+0000","last_peered":"2024-09-21T07:45:38.830083+0000","last_clean":"2024-09-21T07:45:38.830083+0000","last_became_active":"2024-09-21T07:35:27.878335+0000","last_became_peered":"2024-09-21T07:35:27.878335+0000","last_unstale":"2024-09-21T07:45:38.830083+0000","last_undegraded":"2024-09-21T07:45:38.830083+0000","last_fullsized":"2024-09-21T07:45:38.830083+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:25:37.731771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":203,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.815572+0000","last_change":"2024-09-21T07:35:28.858418+0000","last_active":"2024-09-21T07:45:38.815572+0000","last_peered":"2024-09-21T07:45:38.815572+0000","last_clean":"2024-09-21T07:45:38.815572+0000","last_became_active":"2024-09-21T07:35:26.876944+0000","last_became_peered":"2024-09-21T07:35:26.876944+0000","last_unstale":"2024-09-21T07:45:38.815572+0000","last_undegraded":"2024-09-21T07:45:38.815572+0000","last_fullsized":"2024-09-21T07:45:38.815572+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:04.514304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090862799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":245,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688906+0000","last_change":"2024-09-21T07:34:24.597926+0000","last_active":"2024-09-21T07:45:38.688906+0000","last_peered":"2024-09-21T07:45:38.688906+0000","last_clean":"2024-09-21T07:45:38.688906+0000","last_became_active":"2024-09-21T07:34:23.268926+0000","last_became_peered":"2024-09-21T07:34:23.268926+0000","last_unstale":"2024-09-21T07:45:38.688906+0000","last_undegraded":"2024-09-21T07:45:38.688906+0000","last_fullsized":"2024-09-21T07:45:38.688906+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:46.190752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101851,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258976+0000","last_change":"2024-09-21T07:45:37.690580+0000","last_active":"2024-09-21T07:45:39.258976+0000","last_peered":"2024-09-21T07:45:39.258976+0000","last_clean":"2024-09-21T07:45:39.258976+0000","last_became_active":"2024-09-21T07:45:37.690274+0000","last_became_peered":"2024-09-21T07:45:37.690274+0000","last_unstale":"2024-09-21T07:45:39.258976+0000","last_undegraded":"2024-09-21T07:45:39.258976+0000","last_fullsized":"2024-09-21T07:45:39.258976+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:07.141785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.257933+0000","last_change":"2024-09-21T07:45:36.683898+0000","last_active":"2024-09-21T07:45:39.257933+0000","last_peered":"2024-09-21T07:45:39.257933+0000","last_clean":"2024-09-21T07:45:39.257933+0000","last_became_active":"2024-09-21T07:45:36.683536+0000","last_became_peered":"2024-09-21T07:45:36.683536+0000","last_unstale":"2024-09-21T07:45:39.257933+0000","last_undegraded":"2024-09-21T07:45:39.257933+0000","last_fullsized":"2024-09-21T07:45:39.257933+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:42.885001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688581+0000","last_change":"2024-09-21T07:35:27.877426+0000","last_active":"2024-09-21T07:45:38.688581+0000","last_peered":"2024-09-21T07:45:38.688581+0000","last_clean":"2024-09-21T07:45:38.688581+0000","last_became_active":"2024-09-21T07:35:27.877050+0000","last_became_peered":"2024-09-21T07:35:27.877050+0000","last_unstale":"2024-09-21T07:45:38.688581+0000","last_undegraded":"2024-09-21T07:45:38.688581+0000","last_fullsized":"2024-09-21T07:45:38.688581+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:36.268334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":198,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.829462+0000","last_change":"2024-09-21T07:35:28.862273+0000","last_active":"2024-09-21T07:45:38.829462+0000","last_peered":"2024-09-21T07:45:38.829462+0000","last_clean":"2024-09-21T07:45:38.829462+0000","last_became_active":"2024-09-21T07:35:26.874011+0000","last_became_peered":"2024-09-21T07:35:26.874011+0000","last_unstale":"2024-09-21T07:45:38.829462+0000","last_undegraded":"2024-09-21T07:45:38.829462+0000","last_fullsized":"2024-09-21T07:45:38.829462+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:11.986950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071644799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":250,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258568+0000","last_change":"2024-09-21T07:34:24.259794+0000","last_active":"2024-09-21T07:45:39.258568+0000","last_peered":"2024-09-21T07:45:39.258568+0000","last_clean":"2024-09-21T07:45:39.258568+0000","last_became_active":"2024-09-21T07:34:23.265679+0000","last_became_peered":"2024-09-21T07:34:23.265679+0000","last_unstale":"2024-09-21T07:45:39.258568+0000","last_undegraded":"2024-09-21T07:45:39.258568+0000","last_fullsized":"2024-09-21T07:45:39.258568+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:02.575796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001298518,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690698+0000","last_change":"2024-09-21T07:45:37.703118+0000","last_active":"2024-09-21T07:45:38.690698+0000","last_peered":"2024-09-21T07:45:38.690698+0000","last_clean":"2024-09-21T07:45:38.690698+0000","last_became_active":"2024-09-21T07:45:37.702812+0000","last_became_peered":"2024-09-21T07:45:37.702812+0000","last_unstale":"2024-09-21T07:45:38.690698+0000","last_undegraded":"2024-09-21T07:45:38.690698+0000","last_fullsized":"2024-09-21T07:45:38.690698+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:40.878568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689224+0000","last_change":"2024-09-21T07:45:36.695633+0000","last_active":"2024-09-21T07:45:38.689224+0000","last_peered":"2024-09-21T07:45:38.689224+0000","last_clean":"2024-09-21T07:45:38.689224+0000","last_became_active":"2024-09-21T07:45:36.695319+0000","last_became_peered":"2024-09-21T07:45:36.695319+0000","last_unstale":"2024-09-21T07:45:38.689224+0000","last_undegraded":"2024-09-21T07:45:38.689224+0000","last_fullsized":"2024-09-21T07:45:38.689224+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:36.781831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259525+0000","last_change":"2024-09-21T07:35:27.877725+0000","last_active":"2024-09-21T07:45:39.259525+0000","last_peered":"2024-09-21T07:45:39.259525+0000","last_clean":"2024-09-21T07:45:39.259525+0000","last_became_active":"2024-09-21T07:35:27.877398+0000","last_became_peered":"2024-09-21T07:35:27.877398+0000","last_unstale":"2024-09-21T07:45:39.259525+0000","last_undegraded":"2024-09-21T07:45:39.259525+0000","last_fullsized":"2024-09-21T07:45:39.259525+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:05.859762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":198,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258909+0000","last_change":"2024-09-21T07:35:28.860612+0000","last_active":"2024-09-21T07:45:39.258909+0000","last_peered":"2024-09-21T07:45:39.258909+0000","last_clean":"2024-09-21T07:45:39.258909+0000","last_became_active":"2024-09-21T07:35:26.876562+0000","last_became_peered":"2024-09-21T07:35:26.876562+0000","last_unstale":"2024-09-21T07:45:39.258909+0000","last_undegraded":"2024-09-21T07:45:39.258909+0000","last_fullsized":"2024-09-21T07:45:39.258909+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:06.824331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012252470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":251,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.691149+0000","last_change":"2024-09-21T07:34:24.260330+0000","last_active":"2024-09-21T07:45:38.691149+0000","last_peered":"2024-09-21T07:45:38.691149+0000","last_clean":"2024-09-21T07:45:38.691149+0000","last_became_active":"2024-09-21T07:34:23.637210+0000","last_became_peered":"2024-09-21T07:34:23.637210+0000","last_unstale":"2024-09-21T07:45:38.691149+0000","last_undegraded":"2024-09-21T07:45:38.691149+0000","last_fullsized":"2024-09-21T07:45:38.691149+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.488451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001537093,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258595+0000","last_change":"2024-09-21T07:45:37.686719+0000","last_active":"2024-09-21T07:45:39.258595+0000","last_peered":"2024-09-21T07:45:39.258595+0000","last_clean":"2024-09-21T07:45:39.258595+0000","last_became_active":"2024-09-21T07:45:37.686379+0000","last_became_peered":"2024-09-21T07:45:37.686379+0000","last_unstale":"2024-09-21T07:45:39.258595+0000","last_undegraded":"2024-09-21T07:45:39.258595+0000","last_fullsized":"2024-09-21T07:45:39.258595+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:20.522083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"156'1","reported_seq":16,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.816015+0000","last_change":"2024-09-21T07:45:36.676063+0000","last_active":"2024-09-21T07:45:38.816015+0000","last_peered":"2024-09-21T07:45:38.816015+0000","last_clean":"2024-09-21T07:45:38.816015+0000","last_became_active":"2024-09-21T07:45:36.675750+0000","last_became_peered":"2024-09-21T07:45:36.675750+0000","last_unstale":"2024-09-21T07:45:38.816015+0000","last_undegraded":"2024-09-21T07:45:38.816015+0000","last_fullsized":"2024-09-21T07:45:38.816015+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:17.358434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":191,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.829276+0000","last_change":"2024-09-21T07:35:27.874757+0000","last_active":"2024-09-21T07:45:38.829276+0000","last_peered":"2024-09-21T07:45:38.829276+0000","last_clean":"2024-09-21T07:45:38.829276+0000","last_became_active":"2024-09-21T07:35:27.874360+0000","last_became_peered":"2024-09-21T07:35:27.874360+0000","last_unstale":"2024-09-21T07:45:38.829276+0000","last_undegraded":"2024-09-21T07:45:38.829276+0000","last_fullsized":"2024-09-21T07:45:38.829276+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:17.646912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":198,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.691038+0000","last_change":"2024-09-21T07:35:29.493431+0000","last_active":"2024-09-21T07:45:38.691038+0000","last_peered":"2024-09-21T07:45:38.691038+0000","last_clean":"2024-09-21T07:45:38.691038+0000","last_became_active":"2024-09-21T07:35:26.873691+0000","last_became_peered":"2024-09-21T07:35:26.873691+0000","last_unstale":"2024-09-21T07:45:38.691038+0000","last_undegraded":"2024-09-21T07:45:38.691038+0000","last_fullsized":"2024-09-21T07:45:38.691038+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:51.705051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001162464,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":245,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.687901+0000","last_change":"2024-09-21T07:34:24.597286+0000","last_active":"2024-09-21T07:45:38.687901+0000","last_peered":"2024-09-21T07:45:38.687901+0000","last_clean":"2024-09-21T07:45:38.687901+0000","last_became_active":"2024-09-21T07:34:23.267133+0000","last_became_peered":"2024-09-21T07:34:23.267133+0000","last_unstale":"2024-09-21T07:45:38.687901+0000","last_undegraded":"2024-09-21T07:45:38.687901+0000","last_fullsized":"2024-09-21T07:45:38.687901+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:18.257010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688522+0000","last_change":"2024-09-21T07:45:37.705412+0000","last_active":"2024-09-21T07:45:38.688522+0000","last_peered":"2024-09-21T07:45:38.688522+0000","last_clean":"2024-09-21T07:45:38.688522+0000","last_became_active":"2024-09-21T07:45:37.702124+0000","last_became_peered":"2024-09-21T07:45:37.702124+0000","last_unstale":"2024-09-21T07:45:38.688522+0000","last_undegraded":"2024-09-21T07:45:38.688522+0000","last_fullsized":"2024-09-21T07:45:38.688522+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:42:19.473518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"156'2","reported_seq":17,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259093+0000","last_change":"2024-09-21T07:45:36.693348+0000","last_active":"2024-09-21T07:45:39.259093+0000","last_peered":"2024-09-21T07:45:39.259093+0000","last_clean":"2024-09-21T07:45:39.259093+0000","last_became_active":"2024-09-21T07:45:36.692821+0000","last_became_peered":"2024-09-21T07:45:36.692821+0000","last_unstale":"2024-09-21T07:45:39.259093+0000","last_undegraded":"2024-09-21T07:45:39.259093+0000","last_fullsized":"2024-09-21T07:45:39.259093+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:19.622466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":196,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.815289+0000","last_change":"2024-09-21T07:35:27.864458+0000","last_active":"2024-09-21T07:45:38.815289+0000","last_peered":"2024-09-21T07:45:38.815289+0000","last_clean":"2024-09-21T07:45:38.815289+0000","last_became_active":"2024-09-21T07:35:27.864136+0000","last_became_peered":"2024-09-21T07:35:27.864136+0000","last_unstale":"2024-09-21T07:45:38.815289+0000","last_undegraded":"2024-09-21T07:45:38.815289+0000","last_fullsized":"2024-09-21T07:45:38.815289+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:20.162240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":203,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.815836+0000","last_change":"2024-09-21T07:35:28.858784+0000","last_active":"2024-09-21T07:45:38.815836+0000","last_peered":"2024-09-21T07:45:38.815836+0000","last_clean":"2024-09-21T07:45:38.815836+0000","last_became_active":"2024-09-21T07:35:26.876890+0000","last_became_peered":"2024-09-21T07:35:26.876890+0000","last_unstale":"2024-09-21T07:45:38.815836+0000","last_undegraded":"2024-09-21T07:45:38.815836+0000","last_fullsized":"2024-09-21T07:45:38.815836+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:40.738327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064543499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":251,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689383+0000","last_change":"2024-09-21T07:34:24.258712+0000","last_active":"2024-09-21T07:45:38.689383+0000","last_peered":"2024-09-21T07:45:38.689383+0000","last_clean":"2024-09-21T07:45:38.689383+0000","last_became_active":"2024-09-21T07:34:23.274692+0000","last_became_peered":"2024-09-21T07:34:23.274692+0000","last_unstale":"2024-09-21T07:45:38.689383+0000","last_undegraded":"2024-09-21T07:45:38.689383+0000","last_fullsized":"2024-09-21T07:45:38.689383+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:43.279454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018051479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688844+0000","last_change":"2024-09-21T07:45:37.694775+0000","last_active":"2024-09-21T07:45:38.688844+0000","last_peered":"2024-09-21T07:45:38.688844+0000","last_clean":"2024-09-21T07:45:38.688844+0000","last_became_active":"2024-09-21T07:45:37.694023+0000","last_became_peered":"2024-09-21T07:45:37.694023+0000","last_unstale":"2024-09-21T07:45:38.688844+0000","last_undegraded":"2024-09-21T07:45:38.688844+0000","last_fullsized":"2024-09-21T07:45:38.688844+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:46:14.530791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"156'1","reported_seq":16,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259260+0000","last_change":"2024-09-21T07:45:36.700873+0000","last_active":"2024-09-21T07:45:39.259260+0000","last_peered":"2024-09-21T07:45:39.259260+0000","last_clean":"2024-09-21T07:45:39.259260+0000","last_became_active":"2024-09-21T07:45:36.700526+0000","last_became_peered":"2024-09-21T07:45:36.700526+0000","last_unstale":"2024-09-21T07:45:39.259260+0000","last_undegraded":"2024-09-21T07:45:39.259260+0000","last_fullsized":"2024-09-21T07:45:39.259260+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:34.374418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689061+0000","last_change":"2024-09-21T07:35:27.878020+0000","last_active":"2024-09-21T07:45:38.689061+0000","last_peered":"2024-09-21T07:45:38.689061+0000","last_clean":"2024-09-21T07:45:38.689061+0000","last_became_active":"2024-09-21T07:35:27.877532+0000","last_became_peered":"2024-09-21T07:35:27.877532+0000","last_unstale":"2024-09-21T07:45:38.689061+0000","last_undegraded":"2024-09-21T07:45:38.689061+0000","last_fullsized":"2024-09-21T07:45:38.689061+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:19.756073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":198,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259515+0000","last_change":"2024-09-21T07:35:28.861049+0000","last_active":"2024-09-21T07:45:39.259515+0000","last_peered":"2024-09-21T07:45:39.259515+0000","last_clean":"2024-09-21T07:45:39.259515+0000","last_became_active":"2024-09-21T07:35:26.876819+0000","last_became_peered":"2024-09-21T07:35:26.876819+0000","last_unstale":"2024-09-21T07:45:39.259515+0000","last_undegraded":"2024-09-21T07:45:39.259515+0000","last_fullsized":"2024-09-21T07:45:39.259515+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:19.633803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055780599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":249,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.829340+0000","last_change":"2024-09-21T07:34:25.084849+0000","last_active":"2024-09-21T07:45:38.829340+0000","last_peered":"2024-09-21T07:45:38.829340+0000","last_clean":"2024-09-21T07:45:38.829340+0000","last_became_active":"2024-09-21T07:34:23.277713+0000","last_became_peered":"2024-09-21T07:34:23.277713+0000","last_unstale":"2024-09-21T07:45:38.829340+0000","last_undegraded":"2024-09-21T07:45:38.829340+0000","last_fullsized":"2024-09-21T07:45:38.829340+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:05.811810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059079499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.829858+0000","last_change":"2024-09-21T07:45:37.700946+0000","last_active":"2024-09-21T07:45:38.829858+0000","last_peered":"2024-09-21T07:45:38.829858+0000","last_clean":"2024-09-21T07:45:38.829858+0000","last_became_active":"2024-09-21T07:45:37.700358+0000","last_became_peered":"2024-09-21T07:45:37.700358+0000","last_unstale":"2024-09-21T07:45:38.829858+0000","last_undegraded":"2024-09-21T07:45:38.829858+0000","last_fullsized":"2024-09-21T07:45:38.829858+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:58:14.141300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690040+0000","last_change":"2024-09-21T07:45:36.693932+0000","last_active":"2024-09-21T07:45:38.690040+0000","last_peered":"2024-09-21T07:45:38.690040+0000","last_clean":"2024-09-21T07:45:38.690040+0000","last_became_active":"2024-09-21T07:45:36.693640+0000","last_became_peered":"2024-09-21T07:45:36.693640+0000","last_unstale":"2024-09-21T07:45:38.690040+0000","last_undegraded":"2024-09-21T07:45:38.690040+0000","last_fullsized":"2024-09-21T07:45:38.690040+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:04.701463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":192,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258955+0000","last_change":"2024-09-21T07:35:27.884248+0000","last_active":"2024-09-21T07:45:39.258955+0000","last_peered":"2024-09-21T07:45:39.258955+0000","last_clean":"2024-09-21T07:45:39.258955+0000","last_became_active":"2024-09-21T07:35:27.883861+0000","last_became_peered":"2024-09-21T07:35:27.883861+0000","last_unstale":"2024-09-21T07:45:39.258955+0000","last_undegraded":"2024-09-21T07:45:39.258955+0000","last_fullsized":"2024-09-21T07:45:39.258955+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":197,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258922+0000","last_change":"2024-09-21T07:35:29.207135+0000","last_active":"2024-09-21T07:45:39.258922+0000","last_peered":"2024-09-21T07:45:39.258922+0000","last_clean":"2024-09-21T07:45:39.258922+0000","last_became_active":"2024-09-21T07:35:26.872787+0000","last_became_peered":"2024-09-21T07:35:26.872787+0000","last_unstale":"2024-09-21T07:45:39.258922+0000","last_undegraded":"2024-09-21T07:45:39.258922+0000","last_fullsized":"2024-09-21T07:45:39.258922+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:27.543217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011157509999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":255,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.815416+0000","last_change":"2024-09-21T07:34:24.538610+0000","last_active":"2024-09-21T07:45:38.815416+0000","last_peered":"2024-09-21T07:45:38.815416+0000","last_clean":"2024-09-21T07:45:38.815416+0000","last_became_active":"2024-09-21T07:34:23.259700+0000","last_became_peered":"2024-09-21T07:34:23.259700+0000","last_unstale":"2024-09-21T07:45:38.815416+0000","last_undegraded":"2024-09-21T07:45:38.815416+0000","last_fullsized":"2024-09-21T07:45:38.815416+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:57.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070522399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690121+0000","last_change":"2024-09-21T07:45:37.714568+0000","last_active":"2024-09-21T07:45:38.690121+0000","last_peered":"2024-09-21T07:45:38.690121+0000","last_clean":"2024-09-21T07:45:38.690121+0000","last_became_active":"2024-09-21T07:45:37.714235+0000","last_became_peered":"2024-09-21T07:45:37.714235+0000","last_unstale":"2024-09-21T07:45:38.690121+0000","last_undegraded":"2024-09-21T07:45:38.690121+0000","last_fullsized":"2024-09-21T07:45:38.690121+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:29.318374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"156'2","reported_seq":17,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.687809+0000","last_change":"2024-09-21T07:45:36.685978+0000","last_active":"2024-09-21T07:45:38.687809+0000","last_peered":"2024-09-21T07:45:38.687809+0000","last_clean":"2024-09-21T07:45:38.687809+0000","last_became_active":"2024-09-21T07:45:36.685307+0000","last_became_peered":"2024-09-21T07:45:36.685307+0000","last_unstale":"2024-09-21T07:45:38.687809+0000","last_undegraded":"2024-09-21T07:45:38.687809+0000","last_fullsized":"2024-09-21T07:45:38.687809+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:18.642482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":196,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.815520+0000","last_change":"2024-09-21T07:35:27.868024+0000","last_active":"2024-09-21T07:45:38.815520+0000","last_peered":"2024-09-21T07:45:38.815520+0000","last_clean":"2024-09-21T07:45:38.815520+0000","last_became_active":"2024-09-21T07:35:27.867339+0000","last_became_peered":"2024-09-21T07:35:27.867339+0000","last_unstale":"2024-09-21T07:45:38.815520+0000","last_undegraded":"2024-09-21T07:45:38.815520+0000","last_fullsized":"2024-09-21T07:45:38.815520+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":203,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.815424+0000","last_change":"2024-09-21T07:35:28.858505+0000","last_active":"2024-09-21T07:45:38.815424+0000","last_peered":"2024-09-21T07:45:38.815424+0000","last_clean":"2024-09-21T07:45:38.815424+0000","last_became_active":"2024-09-21T07:35:26.877320+0000","last_became_peered":"2024-09-21T07:35:26.877320+0000","last_unstale":"2024-09-21T07:45:38.815424+0000","last_undegraded":"2024-09-21T07:45:38.815424+0000","last_fullsized":"2024-09-21T07:45:38.815424+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:53.672782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244119999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":246,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689180+0000","last_change":"2024-09-21T07:34:24.597597+0000","last_active":"2024-09-21T07:45:38.689180+0000","last_peered":"2024-09-21T07:45:38.689180+0000","last_clean":"2024-09-21T07:45:38.689180+0000","last_became_active":"2024-09-21T07:34:23.635204+0000","last_became_peered":"2024-09-21T07:34:23.635204+0000","last_unstale":"2024-09-21T07:45:38.689180+0000","last_undegraded":"2024-09-21T07:45:38.689180+0000","last_fullsized":"2024-09-21T07:45:38.689180+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:48.644789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081139199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258252+0000","last_change":"2024-09-21T07:45:37.707585+0000","last_active":"2024-09-21T07:45:39.258252+0000","last_peered":"2024-09-21T07:45:39.258252+0000","last_clean":"2024-09-21T07:45:39.258252+0000","last_became_active":"2024-09-21T07:45:37.706725+0000","last_became_peered":"2024-09-21T07:45:37.706725+0000","last_unstale":"2024-09-21T07:45:39.258252+0000","last_undegraded":"2024-09-21T07:45:39.258252+0000","last_fullsized":"2024-09-21T07:45:39.258252+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:16.829578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259009+0000","last_change":"2024-09-21T07:45:36.693812+0000","last_active":"2024-09-21T07:45:39.259009+0000","last_peered":"2024-09-21T07:45:39.259009+0000","last_clean":"2024-09-21T07:45:39.259009+0000","last_became_active":"2024-09-21T07:45:36.692616+0000","last_became_peered":"2024-09-21T07:45:36.692616+0000","last_unstale":"2024-09-21T07:45:39.259009+0000","last_undegraded":"2024-09-21T07:45:39.259009+0000","last_fullsized":"2024-09-21T07:45:39.259009+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:39.176262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259810+0000","last_change":"2024-09-21T07:35:27.877503+0000","last_active":"2024-09-21T07:45:39.259810+0000","last_peered":"2024-09-21T07:45:39.259810+0000","last_clean":"2024-09-21T07:45:39.259810+0000","last_became_active":"2024-09-21T07:35:27.875775+0000","last_became_peered":"2024-09-21T07:35:27.875775+0000","last_unstale":"2024-09-21T07:45:39.259810+0000","last_undegraded":"2024-09-21T07:45:39.259810+0000","last_fullsized":"2024-09-21T07:45:39.259810+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:49.291023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":198,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689157+0000","last_change":"2024-09-21T07:35:29.491412+0000","last_active":"2024-09-21T07:45:38.689157+0000","last_peered":"2024-09-21T07:45:38.689157+0000","last_clean":"2024-09-21T07:45:38.689157+0000","last_became_active":"2024-09-21T07:35:26.876404+0000","last_became_peered":"2024-09-21T07:35:26.876404+0000","last_unstale":"2024-09-21T07:45:38.689157+0000","last_undegraded":"2024-09-21T07:45:38.689157+0000","last_fullsized":"2024-09-21T07:45:38.689157+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063918000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":252,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.689996+0000","last_change":"2024-09-21T07:34:24.259026+0000","last_active":"2024-09-21T07:45:38.689996+0000","last_peered":"2024-09-21T07:45:38.689996+0000","last_clean":"2024-09-21T07:45:38.689996+0000","last_became_active":"2024-09-21T07:34:23.635456+0000","last_became_peered":"2024-09-21T07:34:23.635456+0000","last_unstale":"2024-09-21T07:45:38.689996+0000","last_undegraded":"2024-09-21T07:45:38.689996+0000","last_fullsized":"2024-09-21T07:45:38.689996+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:09.592752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018654419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690345+0000","last_change":"2024-09-21T07:45:37.700588+0000","last_active":"2024-09-21T07:45:38.690345+0000","last_peered":"2024-09-21T07:45:38.690345+0000","last_clean":"2024-09-21T07:45:38.690345+0000","last_became_active":"2024-09-21T07:45:37.700288+0000","last_became_peered":"2024-09-21T07:45:37.700288+0000","last_unstale":"2024-09-21T07:45:38.690345+0000","last_undegraded":"2024-09-21T07:45:38.690345+0000","last_fullsized":"2024-09-21T07:45:38.690345+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:36:25.586374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.688671+0000","last_change":"2024-09-21T07:45:36.695336+0000","last_active":"2024-09-21T07:45:38.688671+0000","last_peered":"2024-09-21T07:45:38.688671+0000","last_clean":"2024-09-21T07:45:38.688671+0000","last_became_active":"2024-09-21T07:45:36.694952+0000","last_became_peered":"2024-09-21T07:45:36.694952+0000","last_unstale":"2024-09-21T07:45:38.688671+0000","last_undegraded":"2024-09-21T07:45:38.688671+0000","last_fullsized":"2024-09-21T07:45:38.688671+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:53:20.269457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259465+0000","last_change":"2024-09-21T07:35:27.871984+0000","last_active":"2024-09-21T07:45:39.259465+0000","last_peered":"2024-09-21T07:45:39.259465+0000","last_clean":"2024-09-21T07:45:39.259465+0000","last_became_active":"2024-09-21T07:35:27.870336+0000","last_became_peered":"2024-09-21T07:35:27.870336+0000","last_unstale":"2024-09-21T07:45:39.259465+0000","last_undegraded":"2024-09-21T07:45:39.259465+0000","last_fullsized":"2024-09-21T07:45:39.259465+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:57.120479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":198,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258561+0000","last_change":"2024-09-21T07:35:28.860351+0000","last_active":"2024-09-21T07:45:39.258561+0000","last_peered":"2024-09-21T07:45:39.258561+0000","last_clean":"2024-09-21T07:45:39.258561+0000","last_became_active":"2024-09-21T07:35:26.872046+0000","last_became_peered":"2024-09-21T07:35:26.872046+0000","last_unstale":"2024-09-21T07:45:39.258561+0000","last_undegraded":"2024-09-21T07:45:39.258561+0000","last_fullsized":"2024-09-21T07:45:39.258561+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:47.074327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098384399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":251,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.690460+0000","last_change":"2024-09-21T07:34:24.260300+0000","last_active":"2024-09-21T07:45:38.690460+0000","last_peered":"2024-09-21T07:45:38.690460+0000","last_clean":"2024-09-21T07:45:38.690460+0000","last_became_active":"2024-09-21T07:34:23.270128+0000","last_became_peered":"2024-09-21T07:34:23.270128+0000","last_unstale":"2024-09-21T07:45:38.690460+0000","last_undegraded":"2024-09-21T07:45:38.690460+0000","last_fullsized":"2024-09-21T07:45:38.690460+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:25.260931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00176651,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":13,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.816185+0000","last_change":"2024-09-21T07:45:37.698965+0000","last_active":"2024-09-21T07:45:38.816185+0000","last_peered":"2024-09-21T07:45:38.816185+0000","last_clean":"2024-09-21T07:45:38.816185+0000","last_became_active":"2024-09-21T07:45:37.698631+0000","last_became_peered":"2024-09-21T07:45:37.698631+0000","last_unstale":"2024-09-21T07:45:38.816185+0000","last_undegraded":"2024-09-21T07:45:38.816185+0000","last_fullsized":"2024-09-21T07:45:38.816185+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:36.648974+0000","last_clean_scrub_stamp":"2024-09-21T07:45:36.648974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:04:04.127234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":15,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.258099+0000","last_change":"2024-09-21T07:45:36.683495+0000","last_active":"2024-09-21T07:45:39.258099+0000","last_peered":"2024-09-21T07:45:39.258099+0000","last_clean":"2024-09-21T07:45:39.258099+0000","last_became_active":"2024-09-21T07:45:36.683165+0000","last_became_peered":"2024-09-21T07:45:36.683165+0000","last_unstale":"2024-09-21T07:45:39.258099+0000","last_undegraded":"2024-09-21T07:45:39.258099+0000","last_fullsized":"2024-09-21T07:45:39.258099+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:35.642508+0000","last_clean_scrub_stamp":"2024-09-21T07:45:35.642508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:35:14.065677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":190,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259743+0000","last_change":"2024-09-21T07:35:27.877861+0000","last_active":"2024-09-21T07:45:39.259743+0000","last_peered":"2024-09-21T07:45:39.259743+0000","last_clean":"2024-09-21T07:45:39.259743+0000","last_became_active":"2024-09-21T07:35:27.877434+0000","last_became_peered":"2024-09-21T07:35:27.877434+0000","last_unstale":"2024-09-21T07:45:39.259743+0000","last_undegraded":"2024-09-21T07:45:39.259743+0000","last_fullsized":"2024-09-21T07:45:39.259743+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":198,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:38.829608+0000","last_change":"2024-09-21T07:35:28.862954+0000","last_active":"2024-09-21T07:45:38.829608+0000","last_peered":"2024-09-21T07:45:38.829608+0000","last_clean":"2024-09-21T07:45:38.829608+0000","last_became_active":"2024-09-21T07:35:26.870613+0000","last_became_peered":"2024-09-21T07:35:26.870613+0000","last_unstale":"2024-09-21T07:45:38.829608+0000","last_undegraded":"2024-09-21T07:45:38.829608+0000","last_fullsized":"2024-09-21T07:45:38.829608+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:06.901043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059842100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":250,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-21T07:45:39.259583+0000","last_change":"2024-09-21T07:34:25.028904+0000","last_active":"2024-09-21T07:45:39.259583+0000","last_peered":"2024-09-21T07:45:39.259583+0000","last_clean":"2024-09-21T07:45:39.259583+0000","last_became_active":"2024-09-21T07:34:23.260942+0000","last_became_peered":"2024-09-21T07:34:23.260942+0000","last_unstale":"2024-09-21T07:45:39.259583+0000","last_undegraded":"2024-09-21T07:45:39.259583+0000","last_fullsized":"2024-09-21T07:45:39.259583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:22.709328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060553899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509595,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37860,"kb_used_data":2512,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333980,"statfs":{"total":96636764160,"available":96597995520,"internally_reserved":0,"allocated":2572288,"data_stored":1621101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509597,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37272,"kb_used_data":1924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334568,"statfs":{"total":96636764160,"available":96598597632,"internally_reserved":0,"allocated":1970176,"data_stored":1029703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509595,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37276,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334564,"statfs":{"total":96636764160,"available":96598593536,"internally_reserved":0,"allocated":1966080,"data_stored":1029053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509596,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37272,"kb_used_data":1924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334568,"statfs":{"total":96636764160,"available":96598597632,"internally_reserved":0,"allocated":1970176,"data_stored":1029647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509597,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37268,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334572,"statfs":{"total":96636764160,"available":96598601728,"internally_reserved":0,"allocated":1966080,"data_stored":1029101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509597,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37860,"kb_used_data":2504,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333980,"statfs":{"total":96636764160,"available":96597995520,"internally_reserved":0,"allocated":2564096,"data_stored":1619455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542300,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37272,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334568,"statfs":{"total":96636764160,"available":96598597632,"internally_reserved":0,"allocated":1961984,"data_stored":1029031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542299,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41448,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330392,"statfs":{"total":96636764160,"available":96594321408,"internally_reserved":0,"allocated":1978368,"data_stored":1030713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:45:47.754 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T07:45:47.754 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T07:45:48.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.020+0000 7fc2fa200640 1 -- 172.21.15.39:0/3314253007 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2dc003a80 msgr2=0x7fc2dc003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:48.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.020+0000 7fc2fa200640 1 --2- 172.21.15.39:0/3314253007 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2dc003a80 0x7fc2dc003ef0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fc2e80016c0 tx=0x7fc2dc005830 comp rx=0 tx=0).stop 2024-09-21T07:45:48.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.020+0000 7fc2fa200640 1 -- 172.21.15.39:0/3314253007 shutdown_connections 2024-09-21T07:45:48.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.020+0000 7fc2fa200640 1 --2- 172.21.15.39:0/3314253007 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2dc003a80 0x7fc2dc003ef0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:48.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.020+0000 7fc2fa200640 1 -- 172.21.15.39:0/3314253007 >> 172.21.15.39:0/3314253007 conn(0x7fc2f406cfc0 msgr2=0x7fc2f406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:48.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.021+0000 7fc2fa200640 1 -- 172.21.15.39:0/3314253007 shutdown_connections 2024-09-21T07:45:48.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.021+0000 7fc2fa200640 1 -- 172.21.15.39:0/3314253007 wait complete. 2024-09-21T07:45:48.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.023+0000 7fc2fa200640 1 Processor -- start 2024-09-21T07:45:48.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.023+0000 7fc2fa200640 1 -- start start 2024-09-21T07:45:48.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.024+0000 7fc2fa200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2dc003a80 0x7fc2f41f6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:48.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.024+0000 7fc2fa200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2f40c5930 0x7fc2f4157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:48.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.024+0000 7fc2fa200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc2f4144dc0 0x7fc2f40b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:48.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.024+0000 7fc2fa200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f415d900 con 0x7fc2dc003a80 2024-09-21T07:45:48.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.024+0000 7fc2fa200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f415d780 con 0x7fc2f40c5930 2024-09-21T07:45:48.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.024+0000 7fc2fa200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc2f415da80 con 0x7fc2f4144dc0 2024-09-21T07:45:48.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.025+0000 7fc2f8e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2dc003a80 0x7fc2f41f6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:48.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.025+0000 7fc2f8e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2dc003a80 0x7fc2f41f6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53928/0 (socket says 172.21.15.39:53928) 2024-09-21T07:45:48.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.025+0000 7fc2f8e00640 1 -- 172.21.15.39:0/4033906672 learned_addr learned my addr 172.21.15.39:0/4033906672 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:48.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.025+0000 7fc2f9800640 1 --2- 172.21.15.39:0/4033906672 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc2f4144dc0 0x7fc2f40b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:48.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.025+0000 7fc2f3e00640 1 --2- 172.21.15.39:0/4033906672 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2f40c5930 0x7fc2f4157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:48.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.025+0000 7fc2f8e00640 1 -- 172.21.15.39:0/4033906672 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc2f4144dc0 msgr2=0x7fc2f40b1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:48.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.025+0000 7fc2f8e00640 1 --2- 172.21.15.39:0/4033906672 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc2f4144dc0 0x7fc2f40b1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:48.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.025+0000 7fc2f8e00640 1 -- 172.21.15.39:0/4033906672 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2f40c5930 msgr2=0x7fc2f4157cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:48.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.025+0000 7fc2f8e00640 1 --2- 172.21.15.39:0/4033906672 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2f40c5930 0x7fc2f4157cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:48.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.025+0000 7fc2f8e00640 1 -- 172.21.15.39:0/4033906672 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f40c6080 con 0x7fc2dc003a80 2024-09-21T07:45:48.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.025+0000 7fc2f9800640 1 --2- 172.21.15.39:0/4033906672 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc2f4144dc0 0x7fc2f40b1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:45:48.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.025+0000 7fc2f3e00640 1 --2- 172.21.15.39:0/4033906672 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2f40c5930 0x7fc2f4157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:45:48.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.026+0000 7fc2f8e00640 1 --2- 172.21.15.39:0/4033906672 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2dc003a80 0x7fc2f41f6870 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fc2e80017e0 tx=0x7fc2e80063a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:48.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.026+0000 7fc2f0c00640 1 -- 172.21.15.39:0/4033906672 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e8042070 con 0x7fc2dc003a80 2024-09-21T07:45:48.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.026+0000 7fc2f0c00640 1 -- 172.21.15.39:0/4033906672 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2e8039070 con 0x7fc2dc003a80 2024-09-21T07:45:48.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.026+0000 7fc2f0c00640 1 -- 172.21.15.39:0/4033906672 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e8003d70 con 0x7fc2dc003a80 2024-09-21T07:45:48.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.026+0000 7fc2fa200640 1 -- 172.21.15.39:0/4033906672 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2f40c6360 con 0x7fc2dc003a80 2024-09-21T07:45:48.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.026+0000 7fc2fa200640 1 -- 172.21.15.39:0/4033906672 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2f41577a0 con 0x7fc2dc003a80 2024-09-21T07:45:48.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.027+0000 7fc2fa200640 1 -- 172.21.15.39:0/4033906672 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2f40cf920 con 0x7fc2dc003a80 2024-09-21T07:45:48.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.028+0000 7fc2f0c00640 1 -- 172.21.15.39:0/4033906672 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc2e803b070 con 0x7fc2dc003a80 2024-09-21T07:45:48.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.030+0000 7fc2f0c00640 1 --2- 172.21.15.39:0/4033906672 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc2d4078bc0 0x7fc2d407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:48.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.030+0000 7fc2f0c00640 1 -- 172.21.15.39:0/4033906672 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 6844+0+0 (secure 0 0 0) 0x7fc2e80ba800 con 0x7fc2dc003a80 2024-09-21T07:45:48.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.030+0000 7fc2f3e00640 1 --2- 172.21.15.39:0/4033906672 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc2d4078bc0 0x7fc2d407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:48.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.031+0000 7fc2f3e00640 1 --2- 172.21.15.39:0/4033906672 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc2d4078bc0 0x7fc2d407b0d0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0001a30 tx=0x7fc2e0001110 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:48.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.034+0000 7fc2f0c00640 1 -- 172.21.15.39:0/4033906672 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc2e80827c0 con 0x7fc2dc003a80 2024-09-21T07:45:48.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.303+0000 7fc2fa200640 1 -- 172.21.15.39:0/4033906672 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fc2f40c6530 con 0x7fc2dc003a80 2024-09-21T07:45:48.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.308+0000 7fc2f0c00640 1 -- 172.21.15.39:0/4033906672 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7fc2e80890c0 con 0x7fc2dc003a80 2024-09-21T07:45:48.310 INFO:teuthology.orchestra.run.smithi039.stdout:[client.0] 2024-09-21T07:45:48.310 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQB0du5mGfb1DBAA/aSMKR5or3xy53rB/iUjpA== 2024-09-21T07:45:48.310 INFO:teuthology.orchestra.run.smithi039.stdout: caps mds = "allow" 2024-09-21T07:45:48.310 INFO:teuthology.orchestra.run.smithi039.stdout: caps mon = "allow r" 2024-09-21T07:45:48.310 INFO:teuthology.orchestra.run.smithi039.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T07:45:48.311 INFO:teuthology.orchestra.run.smithi039.stderr:updated caps for client.0 2024-09-21T07:45:48.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.310+0000 7fc2fa200640 1 -- 172.21.15.39:0/4033906672 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc2d4078bc0 msgr2=0x7fc2d407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:48.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.310+0000 7fc2fa200640 1 --2- 172.21.15.39:0/4033906672 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc2d4078bc0 0x7fc2d407b0d0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0001a30 tx=0x7fc2e0001110 comp rx=0 tx=0).stop 2024-09-21T07:45:48.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.310+0000 7fc2fa200640 1 -- 172.21.15.39:0/4033906672 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2dc003a80 msgr2=0x7fc2f41f6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:48.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.310+0000 7fc2fa200640 1 --2- 172.21.15.39:0/4033906672 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2dc003a80 0x7fc2f41f6870 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fc2e80017e0 tx=0x7fc2e80063a0 comp rx=0 tx=0).stop 2024-09-21T07:45:48.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.310+0000 7fc2fa200640 1 -- 172.21.15.39:0/4033906672 shutdown_connections 2024-09-21T07:45:48.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.310+0000 7fc2fa200640 1 --2- 172.21.15.39:0/4033906672 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc2d4078bc0 0x7fc2d407b0d0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:48.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.310+0000 7fc2fa200640 1 --2- 172.21.15.39:0/4033906672 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc2f4144dc0 0x7fc2f40b1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:48.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.310+0000 7fc2fa200640 1 --2- 172.21.15.39:0/4033906672 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2f40c5930 0x7fc2f4157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:48.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.310+0000 7fc2fa200640 1 --2- 172.21.15.39:0/4033906672 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2dc003a80 0x7fc2f41f6870 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:48.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.310+0000 7fc2fa200640 1 -- 172.21.15.39:0/4033906672 >> 172.21.15.39:0/4033906672 conn(0x7fc2f406cfc0 msgr2=0x7fc2f40735b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:48.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.311+0000 7fc2fa200640 1 -- 172.21.15.39:0/4033906672 shutdown_connections 2024-09-21T07:45:48.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.311+0000 7fc2fa200640 1 -- 172.21.15.39:0/4033906672 wait complete. 2024-09-21T07:45:48.338 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:45:48.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.562+0000 7f9cce600640 1 -- 172.21.15.39:0/3932296785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cc80b4c40 msgr2=0x7f9cc80ce2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:48.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.562+0000 7f9cce600640 1 --2- 172.21.15.39:0/3932296785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cc80b4c40 0x7f9cc80ce2b0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8009830 tx=0x7f9cb802a510 comp rx=0 tx=0).stop 2024-09-21T07:45:48.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.562+0000 7f9cce600640 1 -- 172.21.15.39:0/3932296785 shutdown_connections 2024-09-21T07:45:48.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.562+0000 7f9cce600640 1 --2- 172.21.15.39:0/3932296785 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9cc80b3670 0x7f9cc80b3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:48.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.562+0000 7f9cce600640 1 --2- 172.21.15.39:0/3932296785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cc80b4c40 0x7f9cc80ce2b0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:48.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.562+0000 7f9cce600640 1 -- 172.21.15.39:0/3932296785 >> 172.21.15.39:0/3932296785 conn(0x7f9cc80b2460 msgr2=0x7f9cc806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:48.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.562+0000 7f9cce600640 1 -- 172.21.15.39:0/3932296785 shutdown_connections 2024-09-21T07:45:48.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.562+0000 7f9cce600640 1 -- 172.21.15.39:0/3932296785 wait complete. 2024-09-21T07:45:48.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.565+0000 7f9cce600640 1 Processor -- start 2024-09-21T07:45:48.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.565+0000 7f9cce600640 1 -- start start 2024-09-21T07:45:48.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.566+0000 7f9cce600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9cc80b3670 0x7f9cc80994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:48.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.566+0000 7f9cce600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9cc80b4c40 0x7f9cc81f9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:48.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.566+0000 7f9cce600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cc80ca8a0 0x7f9cc81f99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:48.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.566+0000 7f9cce600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc8070c20 con 0x7f9cc80ca8a0 2024-09-21T07:45:48.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.566+0000 7f9cce600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc8070da0 con 0x7f9cc80b3670 2024-09-21T07:45:48.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.566+0000 7f9cce600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f9cc80cb3c0 con 0x7f9cc80b4c40 2024-09-21T07:45:48.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.567+0000 7f9cc7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cc80ca8a0 0x7f9cc81f99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:48.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.567+0000 7f9cc7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cc80ca8a0 0x7f9cc81f99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53948/0 (socket says 172.21.15.39:53948) 2024-09-21T07:45:48.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.567+0000 7f9cc7e00640 1 -- 172.21.15.39:0/1996305828 learned_addr learned my addr 172.21.15.39:0/1996305828 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:48.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.567+0000 7f9cc7e00640 1 -- 172.21.15.39:0/1996305828 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9cc80b4c40 msgr2=0x7f9cc81f9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:48.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.567+0000 7f9cbfe00640 1 --2- 172.21.15.39:0/1996305828 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9cc80b4c40 0x7f9cc81f9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:48.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.567+0000 7f9cc7400640 1 --2- 172.21.15.39:0/1996305828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9cc80b3670 0x7f9cc80994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:48.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.567+0000 7f9cc7e00640 1 --2- 172.21.15.39:0/1996305828 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9cc80b4c40 0x7f9cc81f9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:48.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.567+0000 7f9cc7e00640 1 -- 172.21.15.39:0/1996305828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9cc80b3670 msgr2=0x7f9cc80994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:48.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.567+0000 7f9cc7e00640 1 --2- 172.21.15.39:0/1996305828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9cc80b3670 0x7f9cc80994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:48.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.567+0000 7f9cc7e00640 1 -- 172.21.15.39:0/1996305828 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc81e9820 con 0x7f9cc80ca8a0 2024-09-21T07:45:48.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.567+0000 7f9cbfe00640 1 --2- 172.21.15.39:0/1996305828 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9cc80b4c40 0x7f9cc81f9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:45:48.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.567+0000 7f9cc7e00640 1 --2- 172.21.15.39:0/1996305828 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cc80ca8a0 0x7f9cc81f99c0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f9cc000c730 tx=0x7f9cc000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:48.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.568+0000 7f9cbf400640 1 -- 172.21.15.39:0/1996305828 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc0017070 con 0x7f9cc80ca8a0 2024-09-21T07:45:48.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.568+0000 7f9cbf400640 1 -- 172.21.15.39:0/1996305828 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9cc0003c60 con 0x7f9cc80ca8a0 2024-09-21T07:45:48.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.568+0000 7f9cbf400640 1 -- 172.21.15.39:0/1996305828 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc00052d0 con 0x7f9cc80ca8a0 2024-09-21T07:45:48.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.568+0000 7f9cce600640 1 -- 172.21.15.39:0/1996305828 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cc81e9b00 con 0x7f9cc80ca8a0 2024-09-21T07:45:48.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.568+0000 7f9cce600640 1 -- 172.21.15.39:0/1996305828 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cc81ea0c0 con 0x7f9cc80ca8a0 2024-09-21T07:45:48.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.569+0000 7f9cce600640 1 -- 172.21.15.39:0/1996305828 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cc80b3a80 con 0x7f9cc80ca8a0 2024-09-21T07:45:48.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.570+0000 7f9cbf400640 1 -- 172.21.15.39:0/1996305828 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9cc001a070 con 0x7f9cc80ca8a0 2024-09-21T07:45:48.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.571+0000 7f9cbf400640 1 --2- 172.21.15.39:0/1996305828 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9ca4078b90 0x7f9ca407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:48.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.571+0000 7f9cbf400640 1 -- 172.21.15.39:0/1996305828 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 6844+0+0 (secure 0 0 0) 0x7f9cc009a930 con 0x7f9cc80ca8a0 2024-09-21T07:45:48.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.572+0000 7f9cc7400640 1 --2- 172.21.15.39:0/1996305828 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9ca4078b90 0x7f9ca407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:48.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.572+0000 7f9cc7400640 1 --2- 172.21.15.39:0/1996305828 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9ca4078b90 0x7f9ca407b0a0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8034000 tx=0x7f9cb8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:48.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.575+0000 7f9cbf400640 1 -- 172.21.15.39:0/1996305828 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9cc80b3fc0 con 0x7f9cc80ca8a0 2024-09-21T07:45:48.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.812+0000 7f9cce600640 1 -- 172.21.15.39:0/1996305828 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9cc80b3130 con 0x7f9cc80ca8a0 2024-09-21T07:45:48.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.812+0000 7f9cbf400640 1 -- 172.21.15.39:0/1996305828 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 181 v181) ==== 95+0+5234 (secure 0 0 0) 0x7f9cc0062990 con 0x7f9cc80ca8a0 2024-09-21T07:45:48.814 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:45:48.814 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":181,"btime":"2024-09-21T07:45:40:695099+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7310,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2823960253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2823960253},{"type":"v1","addr":"172.21.15.39:6835","nonce":2823960253}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":176},{"gid":8155,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/603157278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":603157278},{"type":"v1","addr":"172.21.15.148:6836","nonce":603157278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":175},{"gid":8158,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2610135999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2610135999},{"type":"v1","addr":"172.21.15.148:6837","nonce":2610135999}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":176}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:45:37.186043+0000","modified":"2024-09-21T07:45:40.206770+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8829},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8829":{"gid":8829,"name":"a","rank":0,"incarnation":177,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/2389975689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2389975689},{"type":"v1","addr":"172.21.15.39:6834","nonce":2389975689}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8829,"qdb_cluster":[8829]},"id":8}]} 2024-09-21T07:45:48.815 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 181 2024-09-21T07:45:48.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.815+0000 7f9cce600640 1 -- 172.21.15.39:0/1996305828 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9ca4078b90 msgr2=0x7f9ca407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:48.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.815+0000 7f9cce600640 1 --2- 172.21.15.39:0/1996305828 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9ca4078b90 0x7f9ca407b0a0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8034000 tx=0x7f9cb8004730 comp rx=0 tx=0).stop 2024-09-21T07:45:48.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.815+0000 7f9cce600640 1 -- 172.21.15.39:0/1996305828 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cc80ca8a0 msgr2=0x7f9cc81f99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:48.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.815+0000 7f9cce600640 1 --2- 172.21.15.39:0/1996305828 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cc80ca8a0 0x7f9cc81f99c0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f9cc000c730 tx=0x7f9cc000cbf0 comp rx=0 tx=0).stop 2024-09-21T07:45:48.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.815+0000 7f9cce600640 1 -- 172.21.15.39:0/1996305828 shutdown_connections 2024-09-21T07:45:48.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.815+0000 7f9cce600640 1 --2- 172.21.15.39:0/1996305828 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9ca4078b90 0x7f9ca407b0a0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:48.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.815+0000 7f9cce600640 1 --2- 172.21.15.39:0/1996305828 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cc80ca8a0 0x7f9cc81f99c0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:48.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.815+0000 7f9cce600640 1 --2- 172.21.15.39:0/1996305828 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9cc80b4c40 0x7f9cc81f9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:48.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.815+0000 7f9cce600640 1 --2- 172.21.15.39:0/1996305828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9cc80b3670 0x7f9cc80994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:48.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.815+0000 7f9cce600640 1 -- 172.21.15.39:0/1996305828 >> 172.21.15.39:0/1996305828 conn(0x7f9cc80b2460 msgr2=0x7f9cc8074d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:48.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.815+0000 7f9cce600640 1 -- 172.21.15.39:0/1996305828 shutdown_connections 2024-09-21T07:45:48.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:48.815+0000 7f9cce600640 1 -- 172.21.15.39:0/1996305828 wait complete. 2024-09-21T07:45:49.242 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 181, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T07:45:37.186043+0000', 'modified': '2024-09-21T07:45:40.206770+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8829}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8829': {'gid': 8829, 'name': 'a', 'rank': 0, 'incarnation': 177, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.39:6834/2389975689', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 2389975689}, {'type': 'v1', 'addr': '172.21.15.39:6834', 'nonce': 2389975689}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8829, 'qdb_cluster': [8829]} 2024-09-21T07:45:49.242 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T07:45:49.242 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T07:45:49.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.469+0000 7fb3b1800640 1 -- 172.21.15.39:0/539456633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3ac0b5240 msgr2=0x7fb3ac15c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:49.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.469+0000 7fb3b1800640 1 --2- 172.21.15.39:0/539456633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3ac0b5240 0x7fb3ac15c690 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac06dd90 tx=0x7fb39c02a4a0 comp rx=0 tx=0).stop 2024-09-21T07:45:49.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.469+0000 7fb3b1800640 1 -- 172.21.15.39:0/539456633 shutdown_connections 2024-09-21T07:45:49.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.469+0000 7fb3b1800640 1 --2- 172.21.15.39:0/539456633 >> v2:172.21.15.148:3300/0 conn(0x7fb3ac0cf920 0x7fb3ac0d0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:49.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.469+0000 7fb3b1800640 1 --2- 172.21.15.39:0/539456633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3ac0b5240 0x7fb3ac15c690 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:49.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.469+0000 7fb3b1800640 1 -- 172.21.15.39:0/539456633 >> 172.21.15.39:0/539456633 conn(0x7fb3ac06cfc0 msgr2=0x7fb3ac06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:49.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.469+0000 7fb3b1800640 1 -- 172.21.15.39:0/539456633 shutdown_connections 2024-09-21T07:45:49.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.469+0000 7fb3b1800640 1 -- 172.21.15.39:0/539456633 wait complete. 2024-09-21T07:45:49.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.471+0000 7fb3b1800640 1 Processor -- start 2024-09-21T07:45:49.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.471+0000 7fb3b1800640 1 -- start start 2024-09-21T07:45:49.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.473+0000 7fb3b1800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3ac0b5240 0x7fb3ac0b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:49.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.473+0000 7fb3b1800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb3ac0cf920 0x7fb3ac14f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:49.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.473+0000 7fb3b1800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3ac0cfd30 0x7fb3ac1e5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:49.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.473+0000 7fb3b1800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ac0d0cf0 con 0x7fb3ac0cfd30 2024-09-21T07:45:49.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.473+0000 7fb3b1800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ac0d0b70 con 0x7fb3ac0b5240 2024-09-21T07:45:49.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.473+0000 7fb3b1800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb3ac0d0e70 con 0x7fb3ac0cf920 2024-09-21T07:45:49.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.473+0000 7fb3ab400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3ac0cfd30 0x7fb3ac1e5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:49.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.473+0000 7fb3ab400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3ac0cfd30 0x7fb3ac1e5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53966/0 (socket says 172.21.15.39:53966) 2024-09-21T07:45:49.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.473+0000 7fb3ab400640 1 -- 172.21.15.39:0/3123688616 learned_addr learned my addr 172.21.15.39:0/3123688616 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:49.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.473+0000 7fb3a3e00640 1 --2- 172.21.15.39:0/3123688616 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb3ac0cf920 0x7fb3ac14f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:49.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.473+0000 7fb3aaa00640 1 --2- 172.21.15.39:0/3123688616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3ac0b5240 0x7fb3ac0b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:49.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.473+0000 7fb3ab400640 1 -- 172.21.15.39:0/3123688616 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb3ac0cf920 msgr2=0x7fb3ac14f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:49.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.473+0000 7fb3ab400640 1 --2- 172.21.15.39:0/3123688616 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb3ac0cf920 0x7fb3ac14f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:49.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.473+0000 7fb3ab400640 1 -- 172.21.15.39:0/3123688616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3ac0b5240 msgr2=0x7fb3ac0b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:49.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.473+0000 7fb3ab400640 1 --2- 172.21.15.39:0/3123688616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3ac0b5240 0x7fb3ac0b1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:49.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.473+0000 7fb3ab400640 1 -- 172.21.15.39:0/3123688616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3ac0b01b0 con 0x7fb3ac0cfd30 2024-09-21T07:45:49.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.474+0000 7fb3ab400640 1 --2- 172.21.15.39:0/3123688616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3ac0cfd30 0x7fb3ac1e5d00 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fb3a400c760 tx=0x7fb3a400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:49.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.474+0000 7fb3a2a00640 1 -- 172.21.15.39:0/3123688616 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4017070 con 0x7fb3ac0cfd30 2024-09-21T07:45:49.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.474+0000 7fb3a2a00640 1 -- 172.21.15.39:0/3123688616 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb3a400ce10 con 0x7fb3ac0cfd30 2024-09-21T07:45:49.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.474+0000 7fb3a2a00640 1 -- 172.21.15.39:0/3123688616 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a40052b0 con 0x7fb3ac0cfd30 2024-09-21T07:45:49.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.474+0000 7fb3b1800640 1 -- 172.21.15.39:0/3123688616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3ac0b0490 con 0x7fb3ac0cfd30 2024-09-21T07:45:49.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.474+0000 7fb3b1800640 1 -- 172.21.15.39:0/3123688616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3ac153a10 con 0x7fb3ac0cfd30 2024-09-21T07:45:49.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.477+0000 7fb3a2a00640 1 -- 172.21.15.39:0/3123688616 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb3a401a070 con 0x7fb3ac0cfd30 2024-09-21T07:45:49.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.478+0000 7fb3a2a00640 1 --2- 172.21.15.39:0/3123688616 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb388078b90 0x7fb38807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:49.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.478+0000 7fb3a2a00640 1 -- 172.21.15.39:0/3123688616 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 6844+0+0 (secure 0 0 0) 0x7fb3a409a840 con 0x7fb3ac0cfd30 2024-09-21T07:45:49.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.478+0000 7fb3aaa00640 1 --2- 172.21.15.39:0/3123688616 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb388078b90 0x7fb38807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:49.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.478+0000 7fb3b89bb740 1 Processor -- start 2024-09-21T07:45:49.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.478+0000 7fb3b89bb740 1 -- start start 2024-09-21T07:45:49.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.479+0000 7fb3aaa00640 1 --2- 172.21.15.39:0/3123688616 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb388078b90 0x7fb38807b0a0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb39c034000 tx=0x7fb39c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:49.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.480+0000 7fb3b89bb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55924817e270 0x55924854e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:49.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.480+0000 7fb3b89bb740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5592481553b0 0x55924854eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:49.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.480+0000 7fb3b89bb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55924815d590 0x55924851ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:49.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.480+0000 7fb3b89bb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5592484a8430 con 0x55924817e270 2024-09-21T07:45:49.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.480+0000 7fb3b89bb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5592484985d0 con 0x55924815d590 2024-09-21T07:45:49.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.480+0000 7fb3b89bb740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x559248487a00 con 0x5592481553b0 2024-09-21T07:45:49.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.480+0000 7fb3aaa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55924817e270 0x55924854e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:49.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.480+0000 7fb3aaa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55924817e270 0x55924854e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53978/0 (socket says 172.21.15.39:53978) 2024-09-21T07:45:49.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.480+0000 7fb3aaa00640 1 -- 172.21.15.39:0/3736065534 learned_addr learned my addr 172.21.15.39:0/3736065534 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:49.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.480+0000 7fb3a3e00640 1 --2- 172.21.15.39:0/3736065534 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5592481553b0 0x55924854eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:49.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.480+0000 7fb3ab400640 1 --2- 172.21.15.39:0/3736065534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55924815d590 0x55924851ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:49.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.480+0000 7fb3aaa00640 1 -- 172.21.15.39:0/3736065534 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5592481553b0 msgr2=0x55924854eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:45:49.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.480+0000 7fb3aaa00640 1 --2- 172.21.15.39:0/3736065534 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5592481553b0 0x55924854eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:49.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.481+0000 7fb3aaa00640 1 -- 172.21.15.39:0/3736065534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55924815d590 msgr2=0x55924851ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:45:49.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.481+0000 7fb3aaa00640 1 --2- 172.21.15.39:0/3736065534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55924815d590 0x55924851ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:49.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.481+0000 7fb3aaa00640 1 -- 172.21.15.39:0/3736065534 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559248450520 con 0x55924817e270 2024-09-21T07:45:49.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.481+0000 7fb3aaa00640 1 --2- 172.21.15.39:0/3736065534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55924817e270 0x55924854e970 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fb39c009780 tx=0x7fb39c03d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e0236bc4ed076 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:49.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.481+0000 7fb39b400640 1 -- 172.21.15.39:0/3736065534 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39c003a80 con 0x55924817e270 2024-09-21T07:45:49.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.481+0000 7fb39b400640 1 -- 172.21.15.39:0/3736065534 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb39c003d10 con 0x55924817e270 2024-09-21T07:45:49.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.481+0000 7fb39b400640 1 -- 172.21.15.39:0/3736065534 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39c048660 con 0x55924817e270 2024-09-21T07:45:49.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.482+0000 7fb3b89bb740 1 -- 172.21.15.39:0/3736065534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55924817e270 msgr2=0x55924854e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:49.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.482+0000 7fb3b89bb740 1 --2- 172.21.15.39:0/3736065534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55924817e270 0x55924854e970 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fb39c009780 tx=0x7fb39c03d070 comp rx=0 tx=0).stop 2024-09-21T07:45:49.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.482+0000 7fb3b89bb740 1 -- 172.21.15.39:0/3736065534 shutdown_connections 2024-09-21T07:45:49.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.482+0000 7fb3b89bb740 1 --2- 172.21.15.39:0/3736065534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55924815d590 0x55924851ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:49.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.482+0000 7fb3b89bb740 1 --2- 172.21.15.39:0/3736065534 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5592481553b0 0x55924854eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:49.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.482+0000 7fb3b89bb740 1 --2- 172.21.15.39:0/3736065534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55924817e270 0x55924854e970 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:49.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.482+0000 7fb3b89bb740 1 -- 172.21.15.39:0/3736065534 >> 172.21.15.39:0/3736065534 conn(0x5592481723c0 msgr2=0x559248524970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:49.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.482+0000 7fb3b89bb740 1 -- 172.21.15.39:0/3736065534 shutdown_connections 2024-09-21T07:45:49.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.482+0000 7fb3b89bb740 1 -- 172.21.15.39:0/3736065534 wait complete. 2024-09-21T07:45:49.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.483+0000 7fb3b89bb740 1 -- start start 2024-09-21T07:45:49.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.483+0000 7fb3b89bb740 1 Processor -- start 2024-09-21T07:45:49.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.485+0000 7fb3b89bb740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5592481553b0 0x55924857d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:49.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.485+0000 7fb3b89bb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55924815d590 0x55924854b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:49.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.485+0000 7fb3b89bb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55924817e270 0x559248561f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:49.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.485+0000 7fb3b89bb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55924842e330 con 0x55924815d590 2024-09-21T07:45:49.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.485+0000 7fb3b89bb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55924848b810 con 0x55924817e270 2024-09-21T07:45:49.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.485+0000 7fb3b89bb740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x5592481f8620 con 0x5592481553b0 2024-09-21T07:45:49.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.485+0000 7fb3b89bb740 10 client.0 _ll_register_callbacks cb 0x559248433420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:45:49.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.485+0000 7fb3a3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55924815d590 0x55924854b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:49.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.485+0000 7fb3a3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55924815d590 0x55924854b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53994/0 (socket says 172.21.15.39:53994) 2024-09-21T07:45:49.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.485+0000 7fb3a3e00640 1 -- 172.21.15.39:0/484792370 learned_addr learned my addr 172.21.15.39:0/484792370 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:49.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.485+0000 7fb3aaa00640 1 --2- 172.21.15.39:0/484792370 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5592481553b0 0x55924857d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:49.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.485+0000 7fb3ab400640 1 --2- 172.21.15.39:0/484792370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55924817e270 0x559248561f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:49.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.486+0000 7fb3a3e00640 1 -- 172.21.15.39:0/484792370 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5592481553b0 msgr2=0x55924857d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:45:49.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.486+0000 7fb3a3e00640 1 --2- 172.21.15.39:0/484792370 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5592481553b0 0x55924857d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:49.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.486+0000 7fb3a3e00640 1 -- 172.21.15.39:0/484792370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55924817e270 msgr2=0x559248561f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:45:49.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.486+0000 7fb3a3e00640 1 --2- 172.21.15.39:0/484792370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55924817e270 0x559248561f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:49.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.486+0000 7fb3a3e00640 1 -- 172.21.15.39:0/484792370 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592481e9850 con 0x55924815d590 2024-09-21T07:45:49.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.486+0000 7fb3a3e00640 1 --2- 172.21.15.39:0/484792370 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55924815d590 0x55924854b830 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fb394013500 tx=0x7fb3940139c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e5d1b5b8bea6dce server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:49.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.486+0000 7fb38fe00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:45:49.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.486+0000 7fb38fe00640 1 -- 172.21.15.39:0/484792370 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3940119f0 con 0x55924815d590 2024-09-21T07:45:49.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.486+0000 7fb38fe00640 1 -- 172.21.15.39:0/484792370 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(157..157 src has 1..157) ==== 6844+0+0 (secure 0 0 0) 0x7fb394011b90 con 0x55924815d590 2024-09-21T07:45:49.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.486+0000 7fb3b89bb740 1 -- 172.21.15.39:0/484792370 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb39c003d10 con 0x55924815d590 2024-09-21T07:45:49.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.487+0000 7fb38fe00640 1 -- 172.21.15.39:0/484792370 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=181) ==== 24+0+0 (secure 0 0 0) 0x7fb394027050 con 0x55924815d590 2024-09-21T07:45:49.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.487+0000 7fb3b89bb740 10 client.8991 fetch_fsmap learned FSMap version 181 2024-09-21T07:45:49.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.487+0000 7fb3b89bb740 1 -- 172.21.15.39:0/484792370 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=181,monmap=2+}) -- 0x5592482606b0 con 0x55924815d590 2024-09-21T07:45:49.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.487+0000 7fb38fe00640 1 -- 172.21.15.39:0/484792370 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb39c003d10 con 0x55924815d590 2024-09-21T07:45:49.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.487+0000 7fb38fe00640 1 -- 172.21.15.39:0/484792370 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 181) ==== 3502+0+0 (secure 0 0 0) 0x7fb39402a020 con 0x55924815d590 2024-09-21T07:45:49.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.487+0000 7fb3b89bb740 10 client.8991 fetch_fsmap finished waiting for FSMap version 181 2024-09-21T07:45:49.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.487+0000 7fb3b89bb740 10 client.8991 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:45:49.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.487+0000 7fb3b89bb740 20 client.8991 populate_metadata read hostname 'smithi039' 2024-09-21T07:45:49.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.487+0000 7fb3b89bb740 1 --2- 172.21.15.39:0/484792370 >> [v2:172.21.15.39:6832/2389975689,v1:172.21.15.39:6834/2389975689] conn(0x55924813ce40 0x5592485794c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:49.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.487+0000 7fb3b89bb740 4 client.8991 mds_command: new command op to 8829 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:45:49.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.487+0000 7fb3b89bb740 1 -- 172.21.15.39:0/484792370 --> [v2:172.21.15.39:6832/2389975689,v1:172.21.15.39:6834/2389975689] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55924849a520 con 0x55924813ce40 2024-09-21T07:45:49.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.487+0000 7fb3aaa00640 1 --2- 172.21.15.39:0/484792370 >> [v2:172.21.15.39:6832/2389975689,v1:172.21.15.39:6834/2389975689] conn(0x55924813ce40 0x5592485794c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:49.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.488+0000 7fb38fe00640 10 client.8991 ms_handle_connect on v2:172.21.15.39:6832/2389975689 2024-09-21T07:45:49.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.488+0000 7fb3aaa00640 1 --2- 172.21.15.39:0/484792370 >> [v2:172.21.15.39:6832/2389975689,v1:172.21.15.39:6834/2389975689] conn(0x55924813ce40 0x5592485794c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d78d79cb343bc4eb server_cookie=36b2351000ab4207 in_seq=0 out_seq=0 2024-09-21T07:45:49.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.489+0000 7fb38fe00640 1 -- 172.21.15.39:0/484792370 <== mds.0 v2:172.21.15.39:6832/2389975689 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55924849a520 con 0x55924813ce40 2024-09-21T07:45:49.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.489+0000 7fb38fe00640 10 client.8991 handle_command_reply: tid=0 2024-09-21T07:45:49.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.489+0000 7fb3b89bb740 1 client.8991 shutdown 2024-09-21T07:45:49.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.489+0000 7fb3b89bb740 1 -- 172.21.15.39:0/484792370 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55924815d590 msgr2=0x55924854b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:49.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.489+0000 7fb3b89bb740 1 --2- 172.21.15.39:0/484792370 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55924815d590 0x55924854b830 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fb394013500 tx=0x7fb3940139c0 comp rx=0 tx=0).stop 2024-09-21T07:45:49.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.489+0000 7fb3b89bb740 1 -- 172.21.15.39:0/484792370 shutdown_connections 2024-09-21T07:45:49.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.489+0000 7fb3b89bb740 1 --2- 172.21.15.39:0/484792370 >> [v2:172.21.15.39:6832/2389975689,v1:172.21.15.39:6834/2389975689] conn(0x55924813ce40 0x5592485794c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:49.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.489+0000 7fb3b89bb740 1 --2- 172.21.15.39:0/484792370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55924817e270 0x559248561f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:49.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.489+0000 7fb3b89bb740 1 --2- 172.21.15.39:0/484792370 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55924815d590 0x55924854b830 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:49.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.489+0000 7fb38fe00640 0 client.8991 ms_handle_reset on v2:172.21.15.39:6832/2389975689 2024-09-21T07:45:49.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.489+0000 7fb3b89bb740 1 --2- 172.21.15.39:0/484792370 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5592481553b0 0x55924857d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:49.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.489+0000 7fb3b89bb740 1 -- 172.21.15.39:0/484792370 >> 172.21.15.39:0/484792370 conn(0x5592481723c0 msgr2=0x55924854e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:49.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.490+0000 7fb3b89bb740 1 -- 172.21.15.39:0/484792370 shutdown_connections 2024-09-21T07:45:49.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.490+0000 7fb3b89bb740 1 -- 172.21.15.39:0/484792370 wait complete. 2024-09-21T07:45:49.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.490+0000 7fb3b89bb740 20 client.8991 trim_cache size 0 max 16384 2024-09-21T07:45:49.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.509+0000 7fb3b89bb740 1 Processor -- start 2024-09-21T07:45:49.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.509+0000 7fb3b89bb740 1 -- start start 2024-09-21T07:45:49.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.510+0000 7fb3b89bb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592481553b0 0x55924854ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:49.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.510+0000 7fb3b89bb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55924815d590 0x55924856c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:49.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.510+0000 7fb3b89bb740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55924817e270 0x55924856c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:49.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.511+0000 7fb3b89bb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ac2036d0 con 0x5592481553b0 2024-09-21T07:45:49.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.511+0000 7fb3b89bb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ac202a60 con 0x55924815d590 2024-09-21T07:45:49.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.511+0000 7fb3b89bb740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb3ac203080 con 0x55924817e270 2024-09-21T07:45:49.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.511+0000 7fb3aaa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592481553b0 0x55924854ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:49.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.511+0000 7fb3aaa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592481553b0 0x55924854ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54002/0 (socket says 172.21.15.39:54002) 2024-09-21T07:45:49.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.511+0000 7fb3aaa00640 1 -- 172.21.15.39:0/233376873 learned_addr learned my addr 172.21.15.39:0/233376873 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:49.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.511+0000 7fb3aaa00640 1 -- 172.21.15.39:0/233376873 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55924817e270 msgr2=0x55924856c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:45:49.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.511+0000 7fb3aaa00640 1 --2- 172.21.15.39:0/233376873 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55924817e270 0x55924856c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:49.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.511+0000 7fb3aaa00640 1 -- 172.21.15.39:0/233376873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55924815d590 msgr2=0x55924856c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:45:49.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.511+0000 7fb3aaa00640 1 --2- 172.21.15.39:0/233376873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55924815d590 0x55924856c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:49.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.511+0000 7fb3aaa00640 1 -- 172.21.15.39:0/233376873 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559248525650 con 0x5592481553b0 2024-09-21T07:45:49.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.511+0000 7fb3aaa00640 1 --2- 172.21.15.39:0/233376873 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592481553b0 0x55924854ba00 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fb39c009920 tx=0x7fb39c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec7e7320917aad9d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:49.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.512+0000 7fb39aa00640 1 -- 172.21.15.39:0/233376873 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39c003a80 con 0x5592481553b0 2024-09-21T07:45:49.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.512+0000 7fb39aa00640 1 -- 172.21.15.39:0/233376873 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb39c05a440 con 0x5592481553b0 2024-09-21T07:45:49.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.512+0000 7fb39aa00640 1 -- 172.21.15.39:0/233376873 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39c05a7f0 con 0x5592481553b0 2024-09-21T07:45:49.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.512+0000 7fb3b89bb740 1 -- 172.21.15.39:0/233376873 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592481553b0 msgr2=0x55924854ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:49.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.512+0000 7fb3b89bb740 1 --2- 172.21.15.39:0/233376873 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592481553b0 0x55924854ba00 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fb39c009920 tx=0x7fb39c005f90 comp rx=0 tx=0).stop 2024-09-21T07:45:49.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.512+0000 7fb3b89bb740 1 -- 172.21.15.39:0/233376873 shutdown_connections 2024-09-21T07:45:49.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.512+0000 7fb3b89bb740 1 --2- 172.21.15.39:0/233376873 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55924817e270 0x55924856c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:49.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.512+0000 7fb3b89bb740 1 --2- 172.21.15.39:0/233376873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55924815d590 0x55924856c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:49.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.512+0000 7fb3b89bb740 1 --2- 172.21.15.39:0/233376873 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592481553b0 0x55924854ba00 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:49.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.512+0000 7fb3b89bb740 1 -- 172.21.15.39:0/233376873 >> 172.21.15.39:0/233376873 conn(0x5592481723c0 msgr2=0x559248562ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:49.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.512+0000 7fb3b89bb740 1 -- 172.21.15.39:0/233376873 shutdown_connections 2024-09-21T07:45:49.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.513+0000 7fb3b89bb740 1 -- 172.21.15.39:0/233376873 wait complete. 2024-09-21T07:45:49.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.513+0000 7fb3b89bb740 1 -- start start 2024-09-21T07:45:49.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.514+0000 7fb3b89bb740 1 Processor -- start 2024-09-21T07:45:49.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.515+0000 7fb3b89bb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592481553b0 0x5592485ec810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:49.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.515+0000 7fb3b89bb740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55924815d590 0x559248545f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:49.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.515+0000 7fb3b89bb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55924817e270 0x559248546440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:49.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.515+0000 7fb3b89bb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559248546fe0 con 0x5592481553b0 2024-09-21T07:45:49.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.515+0000 7fb3b89bb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559248562460 con 0x55924817e270 2024-09-21T07:45:49.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.515+0000 7fb3b89bb740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x559248548a40 con 0x55924815d590 2024-09-21T07:45:49.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.515+0000 7fb3b89bb740 10 client.0 _ll_register_callbacks cb 0x7fb394009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:45:49.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.515+0000 7fb3aaa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592481553b0 0x5592485ec810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:49.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.515+0000 7fb3aaa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592481553b0 0x5592485ec810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54006/0 (socket says 172.21.15.39:54006) 2024-09-21T07:45:49.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.515+0000 7fb3aaa00640 1 -- 172.21.15.39:0/2205029985 learned_addr learned my addr 172.21.15.39:0/2205029985 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:49.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.516+0000 7fb3aaa00640 1 -- 172.21.15.39:0/2205029985 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55924815d590 msgr2=0x559248545f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:45:49.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.516+0000 7fb3aaa00640 1 --2- 172.21.15.39:0/2205029985 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55924815d590 0x559248545f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:49.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.516+0000 7fb3aaa00640 1 -- 172.21.15.39:0/2205029985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55924817e270 msgr2=0x559248546440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:45:49.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.516+0000 7fb3aaa00640 1 --2- 172.21.15.39:0/2205029985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55924817e270 0x559248546440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:49.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.516+0000 7fb3aaa00640 1 -- 172.21.15.39:0/2205029985 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55924856bf20 con 0x5592481553b0 2024-09-21T07:45:49.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.516+0000 7fb3aaa00640 1 --2- 172.21.15.39:0/2205029985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592481553b0 0x5592485ec810 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fb39c009780 tx=0x7fb39c003fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf84a5013cca4893 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:49.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.516+0000 7fb38fe00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:45:49.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.516+0000 7fb38fe00640 1 -- 172.21.15.39:0/2205029985 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39c05a7f0 con 0x5592481553b0 2024-09-21T07:45:49.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.516+0000 7fb38fe00640 1 -- 172.21.15.39:0/2205029985 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(157..157 src has 1..157) ==== 6844+0+0 (secure 0 0 0) 0x7fb39c05a990 con 0x5592481553b0 2024-09-21T07:45:49.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.516+0000 7fb3b89bb740 1 -- 172.21.15.39:0/2205029985 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb39c05a440 con 0x5592481553b0 2024-09-21T07:45:49.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.517+0000 7fb38fe00640 1 -- 172.21.15.39:0/2205029985 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=181) ==== 24+0+0 (secure 0 0 0) 0x7fb39c073050 con 0x5592481553b0 2024-09-21T07:45:49.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.517+0000 7fb3b89bb740 10 client.8997 fetch_fsmap learned FSMap version 181 2024-09-21T07:45:49.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.517+0000 7fb3b89bb740 1 -- 172.21.15.39:0/2205029985 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=181,monmap=2+}) -- 0x5592485edd40 con 0x5592481553b0 2024-09-21T07:45:49.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.517+0000 7fb38fe00640 1 -- 172.21.15.39:0/2205029985 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb39c05a440 con 0x5592481553b0 2024-09-21T07:45:49.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.517+0000 7fb38fe00640 1 -- 172.21.15.39:0/2205029985 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 181) ==== 3502+0+0 (secure 0 0 0) 0x7fb39c076020 con 0x5592481553b0 2024-09-21T07:45:49.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.517+0000 7fb3b89bb740 10 client.8997 fetch_fsmap finished waiting for FSMap version 181 2024-09-21T07:45:49.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.517+0000 7fb3b89bb740 10 client.8997 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:45:49.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.517+0000 7fb3b89bb740 20 client.8997 populate_metadata read hostname 'smithi039' 2024-09-21T07:45:49.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.517+0000 7fb3b89bb740 1 --2- 172.21.15.39:0/2205029985 >> [v2:172.21.15.39:6832/2389975689,v1:172.21.15.39:6834/2389975689] conn(0x5592485ed230 0x5592485ed640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:49.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.517+0000 7fb3b89bb740 4 client.8997 mds_command: new command op to 8829 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:45:49.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.517+0000 7fb3b89bb740 1 -- 172.21.15.39:0/2205029985 --> [v2:172.21.15.39:6832/2389975689,v1:172.21.15.39:6834/2389975689] -- command(tid 0: {"prefix": "status"}) -- 0x559248495fd0 con 0x5592485ed230 2024-09-21T07:45:49.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.517+0000 7fb3a3e00640 1 --2- 172.21.15.39:0/2205029985 >> [v2:172.21.15.39:6832/2389975689,v1:172.21.15.39:6834/2389975689] conn(0x5592485ed230 0x5592485ed640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.518+0000 7fb3a3e00640 1 --2- 172.21.15.39:0/2205029985 >> [v2:172.21.15.39:6832/2389975689,v1:172.21.15.39:6834/2389975689] conn(0x5592485ed230 0x5592485ed640 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd76339f409960bb server_cookie=51318bd752e30cad in_seq=0 out_seq=0 2024-09-21T07:45:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.518+0000 7fb38fe00640 10 client.8997 ms_handle_connect on v2:172.21.15.39:6832/2389975689 2024-09-21T07:45:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.518+0000 7fb38fe00640 1 -- 172.21.15.39:0/2205029985 <== mds.0 v2:172.21.15.39:6832/2389975689 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x559248495fd0 con 0x5592485ed230 2024-09-21T07:45:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.518+0000 7fb38fe00640 10 client.8997 handle_command_reply: tid=0 2024-09-21T07:45:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.518+0000 7fb3b89bb740 1 client.8997 shutdown 2024-09-21T07:45:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.518+0000 7fb3b89bb740 1 -- 172.21.15.39:0/2205029985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592481553b0 msgr2=0x5592485ec810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.518+0000 7fb3b89bb740 1 --2- 172.21.15.39:0/2205029985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592481553b0 0x5592485ec810 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fb39c009780 tx=0x7fb39c003fd0 comp rx=0 tx=0).stop 2024-09-21T07:45:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.518+0000 7fb3b89bb740 1 -- 172.21.15.39:0/2205029985 shutdown_connections 2024-09-21T07:45:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.518+0000 7fb3b89bb740 1 --2- 172.21.15.39:0/2205029985 >> [v2:172.21.15.39:6832/2389975689,v1:172.21.15.39:6834/2389975689] conn(0x5592485ed230 0x5592485ed640 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.518+0000 7fb3b89bb740 1 --2- 172.21.15.39:0/2205029985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55924817e270 0x559248546440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.518+0000 7fb3b89bb740 1 --2- 172.21.15.39:0/2205029985 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55924815d590 0x559248545f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.518+0000 7fb3b89bb740 1 --2- 172.21.15.39:0/2205029985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592481553b0 0x5592485ec810 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.518+0000 7fb38fe00640 0 client.8997 ms_handle_reset on v2:172.21.15.39:6832/2389975689 2024-09-21T07:45:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.518+0000 7fb3b89bb740 1 -- 172.21.15.39:0/2205029985 >> 172.21.15.39:0/2205029985 conn(0x5592481723c0 msgr2=0x559248525820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.519+0000 7fb3b89bb740 1 -- 172.21.15.39:0/2205029985 shutdown_connections 2024-09-21T07:45:49.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.519+0000 7fb3b89bb740 1 -- 172.21.15.39:0/2205029985 wait complete. 2024-09-21T07:45:49.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.519+0000 7fb3b89bb740 20 client.8997 trim_cache size 0 max 16384 2024-09-21T07:45:49.523 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:45:49.523 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:45:49.523 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-09-21T07:45:49.523 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 8829, 2024-09-21T07:45:49.523 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:45:49.523 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:45:49.523 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:45:49.523 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 11.856125671999999, 2024-09-21T07:45:49.523 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 181, 2024-09-21T07:45:49.524 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 157, 2024-09-21T07:45:49.524 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 157, 2024-09-21T07:45:49.524 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 16.844296867000001 2024-09-21T07:45:49.524 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:45:49.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.519+0000 7fb3b1800640 1 -- 172.21.15.39:0/3123688616 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb388078b90 msgr2=0x7fb38807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:49.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.519+0000 7fb3b1800640 1 --2- 172.21.15.39:0/3123688616 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb388078b90 0x7fb38807b0a0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb39c034000 tx=0x7fb39c004730 comp rx=0 tx=0).stop 2024-09-21T07:45:49.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.519+0000 7fb3b1800640 1 -- 172.21.15.39:0/3123688616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3ac0cfd30 msgr2=0x7fb3ac1e5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:49.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.519+0000 7fb3b1800640 1 --2- 172.21.15.39:0/3123688616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3ac0cfd30 0x7fb3ac1e5d00 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fb3a400c760 tx=0x7fb3a400cc20 comp rx=0 tx=0).stop 2024-09-21T07:45:49.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.519+0000 7fb3b1800640 1 -- 172.21.15.39:0/3123688616 shutdown_connections 2024-09-21T07:45:49.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.519+0000 7fb3b1800640 1 --2- 172.21.15.39:0/3123688616 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb388078b90 0x7fb38807b0a0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:49.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.519+0000 7fb3b1800640 1 --2- 172.21.15.39:0/3123688616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3ac0cfd30 0x7fb3ac1e5d00 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:49.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.519+0000 7fb3b1800640 1 --2- 172.21.15.39:0/3123688616 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb3ac0cf920 0x7fb3ac14f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:49.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.519+0000 7fb3b1800640 1 --2- 172.21.15.39:0/3123688616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3ac0b5240 0x7fb3ac0b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:49.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.519+0000 7fb3b1800640 1 -- 172.21.15.39:0/3123688616 >> 172.21.15.39:0/3123688616 conn(0x7fb3ac06cfc0 msgr2=0x7fb3ac15e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:49.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.520+0000 7fb3b1800640 1 -- 172.21.15.39:0/3123688616 shutdown_connections 2024-09-21T07:45:49.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.520+0000 7fb3b1800640 1 -- 172.21.15.39:0/3123688616 wait complete. 2024-09-21T07:45:49.543 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:45:49.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.768+0000 7f85e6a00640 1 -- 172.21.15.39:0/716699871 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85e00b43f0 msgr2=0x7f85e015b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:49.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.768+0000 7f85e6a00640 1 --2- 172.21.15.39:0/716699871 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85e00b43f0 0x7f85e015b640 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f85cc009830 tx=0x7f85cc02a480 comp rx=0 tx=0).stop 2024-09-21T07:45:49.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.768+0000 7f85e6a00640 1 -- 172.21.15.39:0/716699871 shutdown_connections 2024-09-21T07:45:49.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.768+0000 7f85e6a00640 1 --2- 172.21.15.39:0/716699871 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85e00b43f0 0x7f85e015b640 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:49.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.768+0000 7f85e6a00640 1 -- 172.21.15.39:0/716699871 >> 172.21.15.39:0/716699871 conn(0x7f85e00d0c70 msgr2=0x7f85e00d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:49.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.768+0000 7f85e6a00640 1 -- 172.21.15.39:0/716699871 shutdown_connections 2024-09-21T07:45:49.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.768+0000 7f85e6a00640 1 -- 172.21.15.39:0/716699871 wait complete. 2024-09-21T07:45:49.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.771+0000 7f85e6a00640 1 Processor -- start 2024-09-21T07:45:49.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.771+0000 7f85e6a00640 1 -- start start 2024-09-21T07:45:49.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.772+0000 7f85e6a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f85e006f4e0 0x7f85e0154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:49.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.772+0000 7f85e6a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85e00b43f0 0x7f85e014e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:49.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.772+0000 7f85e6a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85e00d0400 0x7f85e01560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:49.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.772+0000 7f85e6a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f85e00b5580 con 0x7f85e00b43f0 2024-09-21T07:45:49.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.772+0000 7f85e6a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f85e00d0260 con 0x7f85e00d0400 2024-09-21T07:45:49.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.772+0000 7f85e6a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f85e00b5700 con 0x7f85e006f4e0 2024-09-21T07:45:49.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.773+0000 7f85e4c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85e00b43f0 0x7f85e014e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:49.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.773+0000 7f85e4c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85e00b43f0 0x7f85e014e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54030/0 (socket says 172.21.15.39:54030) 2024-09-21T07:45:49.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.773+0000 7f85e4c00640 1 -- 172.21.15.39:0/1454442976 learned_addr learned my addr 172.21.15.39:0/1454442976 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:49.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.773+0000 7f85e6000640 1 --2- 172.21.15.39:0/1454442976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85e00d0400 0x7f85e01560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:49.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.773+0000 7f85e5600640 1 --2- 172.21.15.39:0/1454442976 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f85e006f4e0 0x7f85e0154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:49.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.773+0000 7f85e4c00640 1 -- 172.21.15.39:0/1454442976 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f85e006f4e0 msgr2=0x7f85e0154f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:49.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.773+0000 7f85e4c00640 1 --2- 172.21.15.39:0/1454442976 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f85e006f4e0 0x7f85e0154f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:49.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.773+0000 7f85e4c00640 1 -- 172.21.15.39:0/1454442976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85e00d0400 msgr2=0x7f85e01560e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:49.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.773+0000 7f85e4c00640 1 --2- 172.21.15.39:0/1454442976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85e00d0400 0x7f85e01560e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:49.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.773+0000 7f85e4c00640 1 -- 172.21.15.39:0/1454442976 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e01f5b30 con 0x7f85e00b43f0 2024-09-21T07:45:49.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.774+0000 7f85e4c00640 1 --2- 172.21.15.39:0/1454442976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85e00b43f0 0x7f85e014e2d0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f85cc000c00 tx=0x7f85cc033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:49.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.774+0000 7f85dd600640 1 -- 172.21.15.39:0/1454442976 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc031d60 con 0x7f85e00b43f0 2024-09-21T07:45:49.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.774+0000 7f85dd600640 1 -- 172.21.15.39:0/1454442976 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f85cc033070 con 0x7f85e00b43f0 2024-09-21T07:45:49.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.774+0000 7f85e6a00640 1 -- 172.21.15.39:0/1454442976 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85e01f5e10 con 0x7f85e00b43f0 2024-09-21T07:45:49.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.774+0000 7f85dd600640 1 -- 172.21.15.39:0/1454442976 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc0334c0 con 0x7f85e00b43f0 2024-09-21T07:45:49.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.774+0000 7f85e6a00640 1 -- 172.21.15.39:0/1454442976 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85e01f63d0 con 0x7f85e00b43f0 2024-09-21T07:45:49.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.775+0000 7f85e6a00640 1 -- 172.21.15.39:0/1454442976 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85e00b5c40 con 0x7f85e00b43f0 2024-09-21T07:45:49.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.776+0000 7f85dd600640 1 -- 172.21.15.39:0/1454442976 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f85cc033660 con 0x7f85e00b43f0 2024-09-21T07:45:49.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.777+0000 7f85dd600640 1 --2- 172.21.15.39:0/1454442976 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f85c0078b90 0x7f85c007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:49.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.777+0000 7f85dd600640 1 -- 172.21.15.39:0/1454442976 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 6844+0+0 (secure 0 0 0) 0x7f85cc0bc620 con 0x7f85e00b43f0 2024-09-21T07:45:49.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.778+0000 7f85e5600640 1 --2- 172.21.15.39:0/1454442976 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f85c0078b90 0x7f85c007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:49.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.779+0000 7f85e5600640 1 --2- 172.21.15.39:0/1454442976 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f85c0078b90 0x7f85c007b0a0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f85d4001720 tx=0x7f85d4007e90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:49.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:49.782+0000 7f85dd600640 1 -- 172.21.15.39:0/1454442976 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f85cc03f030 con 0x7f85e00b43f0 2024-09-21T07:45:50.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:50.016+0000 7f85e6a00640 1 -- 172.21.15.39:0/1454442976 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f85e014ee60 con 0x7f85e00b43f0 2024-09-21T07:45:50.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:50.017+0000 7f85dd600640 1 -- 172.21.15.39:0/1454442976 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v181) ==== 53+0+73 (secure 0 0 0) 0x7f85cc083f90 con 0x7f85e00b43f0 2024-09-21T07:45:50.019 INFO:teuthology.orchestra.run.smithi039.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:45:50.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:50.019+0000 7f85e6a00640 1 -- 172.21.15.39:0/1454442976 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f85c0078b90 msgr2=0x7f85c007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:50.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:50.019+0000 7f85e6a00640 1 --2- 172.21.15.39:0/1454442976 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f85c0078b90 0x7f85c007b0a0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f85d4001720 tx=0x7f85d4007e90 comp rx=0 tx=0).stop 2024-09-21T07:45:50.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:50.019+0000 7f85e6a00640 1 -- 172.21.15.39:0/1454442976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85e00b43f0 msgr2=0x7f85e014e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:50.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:50.019+0000 7f85e6a00640 1 --2- 172.21.15.39:0/1454442976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85e00b43f0 0x7f85e014e2d0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f85cc000c00 tx=0x7f85cc033d60 comp rx=0 tx=0).stop 2024-09-21T07:45:50.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:50.019+0000 7f85e6a00640 1 -- 172.21.15.39:0/1454442976 shutdown_connections 2024-09-21T07:45:50.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:50.019+0000 7f85e6a00640 1 --2- 172.21.15.39:0/1454442976 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f85c0078b90 0x7f85c007b0a0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:50.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:50.019+0000 7f85e6a00640 1 --2- 172.21.15.39:0/1454442976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f85e00d0400 0x7f85e01560e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:50.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:50.019+0000 7f85e6a00640 1 --2- 172.21.15.39:0/1454442976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85e00b43f0 0x7f85e014e2d0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:50.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:50.019+0000 7f85e6a00640 1 --2- 172.21.15.39:0/1454442976 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f85e006f4e0 0x7f85e0154f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:50.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:50.019+0000 7f85e6a00640 1 -- 172.21.15.39:0/1454442976 >> 172.21.15.39:0/1454442976 conn(0x7f85e00d0c70 msgr2=0x7f85e00b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:50.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:50.019+0000 7f85e6a00640 1 -- 172.21.15.39:0/1454442976 shutdown_connections 2024-09-21T07:45:50.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:50.019+0000 7f85e6a00640 1 -- 172.21.15.39:0/1454442976 wait complete. 2024-09-21T07:45:50.042 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:45:50.042 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:45:50.043 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi039.front.sepia.ceph.com 2024-09-21T07:45:50.043 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T07:45:50.043 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:45:50.043 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:45:50.043 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:45:50.043 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:45:50.043 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:45:50.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:50.043 DEBUG:teuthology.orchestra.run.smithi039:> ip addr 2024-09-21T07:45:50.079 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:45:50.080 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:45:50.080 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:45:50.080 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:45:50.080 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2024-09-21T07:45:50.080 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:45:50.080 INFO:teuthology.orchestra.run.smithi039.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:45:50.080 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:45:50.080 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f0 2024-09-21T07:45:50.080 INFO:teuthology.orchestra.run.smithi039.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:45:50.080 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c7 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:45:50.080 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f1 2024-09-21T07:45:50.080 INFO:teuthology.orchestra.run.smithi039.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:45:50.080 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:70 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:45:50.081 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f0 2024-09-21T07:45:50.081 INFO:teuthology.orchestra.run.smithi039.stdout: inet 172.21.15.39/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T07:45:50.081 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft 41969sec preferred_lft 41969sec 2024-09-21T07:45:50.081 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec4:7aff:febd:1370/64 scope link 2024-09-21T07:45:50.081 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:45:50.081 INFO:teuthology.orchestra.run.smithi039.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:45:50.081 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:71 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:45:50.081 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f1 2024-09-21T07:45:50.081 INFO:teuthology.orchestra.run.smithi039.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:45:50.081 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff 2024-09-21T07:45:50.081 INFO:teuthology.orchestra.run.smithi039.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:45:50.081 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:45:50.081 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::b079:ddff:fe0e:6333/64 scope link 2024-09-21T07:45:50.081 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:45:50.082 INFO:teuthology.orchestra.run.smithi039.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:45:50.082 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:45:50.082 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::2ca7:98ff:fe8a:d0ed/64 scope link 2024-09-21T07:45:50.082 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:45:50.082 INFO:teuthology.orchestra.run.smithi039.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:45:50.082 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether ee:10:53:01:0c:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:45:50.082 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec10:53ff:fe01:c44/64 scope link 2024-09-21T07:45:50.082 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:45:50.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:50.082 DEBUG:teuthology.orchestra.run.smithi039:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:45:50.173 INFO:teuthology.orchestra.run.smithi039.stdout:1 2024-09-21T07:45:50.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:50.176 DEBUG:teuthology.orchestra.run.smithi039:> route 2024-09-21T07:45:50.212 INFO:teuthology.orchestra.run.smithi039.stdout:Kernel IP routing table 2024-09-21T07:45:50.212 INFO:teuthology.orchestra.run.smithi039.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:45:50.212 INFO:teuthology.orchestra.run.smithi039.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T07:45:50.212 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T07:45:50.212 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:45:50.213 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:45:50.213 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-21T07:45:50.213 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T07:45:50.213 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T07:45:50.213 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T07:45:50.213 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-21T07:45:50.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:45:50.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:45:50.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:50.411 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-09-21T07:45:50.447 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:45:50.447 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T07:45:50.447 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:45:50.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:50.447 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-09-21T07:45:50.542 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:45:50.542 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T07:45:50.577 INFO:teuthology.orchestra.run.smithi039.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:45:50.578 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:45:50.578 DEBUG:teuthology.orchestra.run.smithi039:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:45:50.652 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-21T07:45:50.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:50.652 DEBUG:teuthology.orchestra.run.smithi039:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:45:50.850 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:45:50.850 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:45:50.885 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:45:50.885 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:45:50.885 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: resolved to: "172.21.15.39:3300,172.21.15.148:3300,172.21.15.148:3301" 2024-09-21T07:45:50.885 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: trying mount with new device syntax: 0@7ab9b04a-f88d-49aa-a493-e5c076637ffb.cephfs=/ 2024-09-21T07:45:50.885 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.39:3300/172.21.15.148:3300/172.21.15.148:3301" will pass to kernel 2024-09-21T07:45:50.886 INFO:teuthology.orchestra.run.smithi039.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T07:45:50.886 INFO:teuthology.orchestra.run.smithi039.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:45:50.886 INFO:teuthology.orchestra.run.smithi039.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:45:50.886 INFO:teuthology.orchestra.run.smithi039.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:45:50.887 INFO:teuthology.orchestra.run.smithi039.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:45:50.889 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:45:50.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:50.889 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:45:50.946 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:45:51.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:45:51.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:45:51.045 INFO:teuthology.orchestra.run.smithi039.stdout:client8278 2024-09-21T07:45:51.045 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:45:51.045 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:45:51.080 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get mds_max_snaps_per_dir 2024-09-21T07:45:51.364 INFO:teuthology.orchestra.run.smithi039.stdout:{"mds_max_snaps_per_dir":"100"} 2024-09-21T07:45:51.365 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_max_snaps_per_dir": "100" } 2024-09-21T07:45:51.365 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:45:51.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.590+0000 7f3d37000640 1 -- 172.21.15.39:0/970421605 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d30162f10 msgr2=0x7f3d300b30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:51.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.590+0000 7f3d37000640 1 --2- 172.21.15.39:0/970421605 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d30162f10 0x7f3d300b30a0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f3d20005fd0 tx=0x7f3d2002a5d0 comp rx=0 tx=0).stop 2024-09-21T07:45:51.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.590+0000 7f3d37000640 1 -- 172.21.15.39:0/970421605 shutdown_connections 2024-09-21T07:45:51.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.590+0000 7f3d37000640 1 --2- 172.21.15.39:0/970421605 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d300b3990 0x7f3d300b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:51.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.590+0000 7f3d37000640 1 --2- 172.21.15.39:0/970421605 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d30162f10 0x7f3d300b30a0 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:51.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.590+0000 7f3d37000640 1 -- 172.21.15.39:0/970421605 >> 172.21.15.39:0/970421605 conn(0x7f3d300b2460 msgr2=0x7f3d3006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:51.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.590+0000 7f3d37000640 1 -- 172.21.15.39:0/970421605 shutdown_connections 2024-09-21T07:45:51.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.590+0000 7f3d37000640 1 -- 172.21.15.39:0/970421605 wait complete. 2024-09-21T07:45:51.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.592+0000 7f3d37000640 1 Processor -- start 2024-09-21T07:45:51.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.592+0000 7f3d37000640 1 -- start start 2024-09-21T07:45:51.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.594+0000 7f3d37000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d300b3990 0x7f3d300c7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:51.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.594+0000 7f3d37000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d300b3da0 0x7f3d30155280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:51.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.594+0000 7f3d37000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d30162f10 0x7f3d301f6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:51.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.594+0000 7f3d37000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3d300af910 con 0x7f3d300b3990 2024-09-21T07:45:51.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.594+0000 7f3d37000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3d300af790 con 0x7f3d30162f10 2024-09-21T07:45:51.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.594+0000 7f3d37000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f3d300afa90 con 0x7f3d300b3da0 2024-09-21T07:45:51.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.594+0000 7f3d2fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d300b3990 0x7f3d300c7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:51.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.594+0000 7f3d2fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d300b3990 0x7f3d300c7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54058/0 (socket says 172.21.15.39:54058) 2024-09-21T07:45:51.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.594+0000 7f3d2fe00640 1 -- 172.21.15.39:0/2050831962 learned_addr learned my addr 172.21.15.39:0/2050831962 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:51.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.594+0000 7f3d27400640 1 --2- 172.21.15.39:0/2050831962 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d300b3da0 0x7f3d30155280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:51.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.594+0000 7f3d35000640 1 --2- 172.21.15.39:0/2050831962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d30162f10 0x7f3d301f6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:51.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.595+0000 7f3d2fe00640 1 -- 172.21.15.39:0/2050831962 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d300b3da0 msgr2=0x7f3d30155280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:51.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.595+0000 7f3d2fe00640 1 --2- 172.21.15.39:0/2050831962 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d300b3da0 0x7f3d30155280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:51.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.595+0000 7f3d2fe00640 1 -- 172.21.15.39:0/2050831962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d30162f10 msgr2=0x7f3d301f6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:51.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.595+0000 7f3d2fe00640 1 --2- 172.21.15.39:0/2050831962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d30162f10 0x7f3d301f6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:51.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.595+0000 7f3d2fe00640 1 -- 172.21.15.39:0/2050831962 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d300c9c00 con 0x7f3d300b3990 2024-09-21T07:45:51.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.595+0000 7f3d2fe00640 1 --2- 172.21.15.39:0/2050831962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d300b3990 0x7f3d300c7b60 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f3d20034040 tx=0x7f3d200038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:51.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.595+0000 7f3d2cc00640 1 -- 172.21.15.39:0/2050831962 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2003e030 con 0x7f3d300b3990 2024-09-21T07:45:51.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.595+0000 7f3d2cc00640 1 -- 172.21.15.39:0/2050831962 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d20003ca0 con 0x7f3d300b3990 2024-09-21T07:45:51.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.595+0000 7f3d37000640 1 -- 172.21.15.39:0/2050831962 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d300c9ee0 con 0x7f3d300b3990 2024-09-21T07:45:51.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.595+0000 7f3d2cc00640 1 -- 172.21.15.39:0/2050831962 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d20035040 con 0x7f3d300b3990 2024-09-21T07:45:51.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.595+0000 7f3d37000640 1 -- 172.21.15.39:0/2050831962 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d300ca4a0 con 0x7f3d300b3990 2024-09-21T07:45:51.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.596+0000 7f3d37000640 1 -- 172.21.15.39:0/2050831962 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d300b4b00 con 0x7f3d300b3990 2024-09-21T07:45:51.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.598+0000 7f3d2cc00640 1 -- 172.21.15.39:0/2050831962 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3d20041030 con 0x7f3d300b3990 2024-09-21T07:45:51.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.598+0000 7f3d2cc00640 1 --2- 172.21.15.39:0/2050831962 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3d0c078b90 0x7f3d0c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:51.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.599+0000 7f3d2cc00640 1 -- 172.21.15.39:0/2050831962 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 6844+0+0 (secure 0 0 0) 0x7f3d200bb710 con 0x7f3d300b3990 2024-09-21T07:45:51.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.599+0000 7f3d27400640 1 --2- 172.21.15.39:0/2050831962 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3d0c078b90 0x7f3d0c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:51.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.600+0000 7f3d27400640 1 --2- 172.21.15.39:0/2050831962 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3d0c078b90 0x7f3d0c07b0a0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f3d301f9180 tx=0x7f3d1800b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:51.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.603+0000 7f3d2cc00640 1 -- 172.21.15.39:0/2050831962 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3d20083080 con 0x7f3d300b3990 2024-09-21T07:45:51.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.839+0000 7f3d37000640 1 -- 172.21.15.39:0/2050831962 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d300cdb70 con 0x7f3d300b3990 2024-09-21T07:45:51.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.840+0000 7f3d2cc00640 1 -- 172.21.15.39:0/2050831962 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 181 v181) ==== 95+0+5234 (secure 0 0 0) 0x7f3d20087ab0 con 0x7f3d300b3990 2024-09-21T07:45:51.842 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:45:51.842 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":181,"btime":"2024-09-21T07:45:40:695099+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7310,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2823960253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2823960253},{"type":"v1","addr":"172.21.15.39:6835","nonce":2823960253}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":176},{"gid":8155,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/603157278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":603157278},{"type":"v1","addr":"172.21.15.148:6836","nonce":603157278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":175},{"gid":8158,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2610135999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2610135999},{"type":"v1","addr":"172.21.15.148:6837","nonce":2610135999}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":176}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:45:37.186043+0000","modified":"2024-09-21T07:45:40.206770+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8829},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8829":{"gid":8829,"name":"a","rank":0,"incarnation":177,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/2389975689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2389975689},{"type":"v1","addr":"172.21.15.39:6834","nonce":2389975689}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8829,"qdb_cluster":[8829]},"id":8}]} 2024-09-21T07:45:51.842 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 181 2024-09-21T07:45:51.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.843+0000 7f3d37000640 1 -- 172.21.15.39:0/2050831962 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3d0c078b90 msgr2=0x7f3d0c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:51.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.843+0000 7f3d37000640 1 --2- 172.21.15.39:0/2050831962 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3d0c078b90 0x7f3d0c07b0a0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f3d301f9180 tx=0x7f3d1800b2e0 comp rx=0 tx=0).stop 2024-09-21T07:45:51.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.843+0000 7f3d37000640 1 -- 172.21.15.39:0/2050831962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d300b3990 msgr2=0x7f3d300c7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:51.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.843+0000 7f3d37000640 1 --2- 172.21.15.39:0/2050831962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d300b3990 0x7f3d300c7b60 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f3d20034040 tx=0x7f3d200038a0 comp rx=0 tx=0).stop 2024-09-21T07:45:51.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.843+0000 7f3d37000640 1 -- 172.21.15.39:0/2050831962 shutdown_connections 2024-09-21T07:45:51.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.843+0000 7f3d37000640 1 --2- 172.21.15.39:0/2050831962 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3d0c078b90 0x7f3d0c07b0a0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:51.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.843+0000 7f3d37000640 1 --2- 172.21.15.39:0/2050831962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d30162f10 0x7f3d301f6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:51.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.843+0000 7f3d37000640 1 --2- 172.21.15.39:0/2050831962 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d300b3da0 0x7f3d30155280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:51.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.843+0000 7f3d37000640 1 --2- 172.21.15.39:0/2050831962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d300b3990 0x7f3d300c7b60 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:51.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.843+0000 7f3d37000640 1 -- 172.21.15.39:0/2050831962 >> 172.21.15.39:0/2050831962 conn(0x7f3d300b2460 msgr2=0x7f3d3006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:51.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.843+0000 7f3d37000640 1 -- 172.21.15.39:0/2050831962 shutdown_connections 2024-09-21T07:45:51.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:51.843+0000 7f3d37000640 1 -- 172.21.15.39:0/2050831962 wait complete. 2024-09-21T07:45:51.866 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_max_snaps_per_dir 100 2024-09-21T07:45:52.113 INFO:teuthology.orchestra.run.smithi039.stdout:{"success":"mds_max_snaps_per_dir = '' "} 2024-09-21T07:45:52.113 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_max_snaps_per_dir = '' " } 2024-09-21T07:45:52.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:52.113 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts) 2024-09-21T07:45:52.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:52.155 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_1) 2024-09-21T07:45:52.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:52.258 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_2) 2024-09-21T07:45:52.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:52.320 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_3) 2024-09-21T07:45:52.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:52.381 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_4) 2024-09-21T07:45:52.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:52.443 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_5) 2024-09-21T07:45:52.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:52.505 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_6) 2024-09-21T07:45:52.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:52.567 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_7) 2024-09-21T07:45:52.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:52.629 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_8) 2024-09-21T07:45:52.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:52.692 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_9) 2024-09-21T07:45:52.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:52.755 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_10) 2024-09-21T07:45:52.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:52.818 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_11) 2024-09-21T07:45:52.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:52.869 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_12) 2024-09-21T07:45:52.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:52.921 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_13) 2024-09-21T07:45:52.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:52.973 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_14) 2024-09-21T07:45:53.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:53.025 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_15) 2024-09-21T07:45:53.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:53.076 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_16) 2024-09-21T07:45:53.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:53.128 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_17) 2024-09-21T07:45:53.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:53.179 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_18) 2024-09-21T07:45:53.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:53.230 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_19) 2024-09-21T07:45:53.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:53.281 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_20) 2024-09-21T07:45:53.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:53.332 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_21) 2024-09-21T07:45:53.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:53.384 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_22) 2024-09-21T07:45:53.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:53.435 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_23) 2024-09-21T07:45:53.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:53.487 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_24) 2024-09-21T07:45:53.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:53.538 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_25) 2024-09-21T07:45:53.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:53.589 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_26) 2024-09-21T07:45:53.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:53.640 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_27) 2024-09-21T07:45:53.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:53.691 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_28) 2024-09-21T07:45:53.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:53.744 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_29) 2024-09-21T07:45:53.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:53.796 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_30) 2024-09-21T07:45:53.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:53.847 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_31) 2024-09-21T07:45:53.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:53.899 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_32) 2024-09-21T07:45:53.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:53.951 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_33) 2024-09-21T07:45:54.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:54.003 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_34) 2024-09-21T07:45:54.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:54.055 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_35) 2024-09-21T07:45:54.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:54.107 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_36) 2024-09-21T07:45:54.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:54.158 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_37) 2024-09-21T07:45:54.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:54.210 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_38) 2024-09-21T07:45:54.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:54.262 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_39) 2024-09-21T07:45:54.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:54.314 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_40) 2024-09-21T07:45:54.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:54.366 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_41) 2024-09-21T07:45:54.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:54.419 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_42) 2024-09-21T07:45:54.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:54.470 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_43) 2024-09-21T07:45:54.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:54.523 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_44) 2024-09-21T07:45:54.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:54.575 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_45) 2024-09-21T07:45:54.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:54.627 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_46) 2024-09-21T07:45:54.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:54.679 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_47) 2024-09-21T07:45:54.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:54.731 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_48) 2024-09-21T07:45:54.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:54.783 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_49) 2024-09-21T07:45:54.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:54.835 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_50) 2024-09-21T07:45:54.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:54.888 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_51) 2024-09-21T07:45:54.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:54.940 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_52) 2024-09-21T07:45:54.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:54.991 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_53) 2024-09-21T07:45:55.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:55.043 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_54) 2024-09-21T07:45:55.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:55.096 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_55) 2024-09-21T07:45:55.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:55.148 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_56) 2024-09-21T07:45:55.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:55.201 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_57) 2024-09-21T07:45:55.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:55.254 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_58) 2024-09-21T07:45:55.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:55.306 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_59) 2024-09-21T07:45:55.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:55.359 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_60) 2024-09-21T07:45:55.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:55.411 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_61) 2024-09-21T07:45:55.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:55.464 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_62) 2024-09-21T07:45:55.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:55.516 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_63) 2024-09-21T07:45:55.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:55.568 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_64) 2024-09-21T07:45:55.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:55.621 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_65) 2024-09-21T07:45:55.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:55.673 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_66) 2024-09-21T07:45:55.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:55.725 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_67) 2024-09-21T07:45:55.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:55.777 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_68) 2024-09-21T07:45:55.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:55.829 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_69) 2024-09-21T07:45:55.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:55.881 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_70) 2024-09-21T07:45:55.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:55.934 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_71) 2024-09-21T07:45:55.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:55.986 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_72) 2024-09-21T07:45:56.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:56.039 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_73) 2024-09-21T07:45:56.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:56.092 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_74) 2024-09-21T07:45:56.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:56.145 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_75) 2024-09-21T07:45:56.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:56.198 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_76) 2024-09-21T07:45:56.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:56.250 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_77) 2024-09-21T07:45:56.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:56.303 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_78) 2024-09-21T07:45:56.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:56.356 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_79) 2024-09-21T07:45:56.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:56.409 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_80) 2024-09-21T07:45:56.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:56.462 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_81) 2024-09-21T07:45:56.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:56.515 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_82) 2024-09-21T07:45:56.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:56.569 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_83) 2024-09-21T07:45:56.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:56.621 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_84) 2024-09-21T07:45:56.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:56.675 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_85) 2024-09-21T07:45:56.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:56.728 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_86) 2024-09-21T07:45:56.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:56.782 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_87) 2024-09-21T07:45:56.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:56.835 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_88) 2024-09-21T07:45:56.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:56.889 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_89) 2024-09-21T07:45:56.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:56.944 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_90) 2024-09-21T07:45:56.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:56.998 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_91) 2024-09-21T07:45:57.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:57.052 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_92) 2024-09-21T07:45:57.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:57.105 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_93) 2024-09-21T07:45:57.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:57.159 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_94) 2024-09-21T07:45:57.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:57.213 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_95) 2024-09-21T07:45:57.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:57.282 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_96) 2024-09-21T07:45:57.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:57.346 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_97) 2024-09-21T07:45:57.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:57.410 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_98) 2024-09-21T07:45:57.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:57.475 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_99) 2024-09-21T07:45:57.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:57.539 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_100) 2024-09-21T07:45:57.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:57.603 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_101) 2024-09-21T07:45:57.639 INFO:teuthology.orchestra.run.smithi039.stderr:mkdir: cannot create directory ‘accounts/.snap/s_101’: Too many links 2024-09-21T07:45:57.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:45:57.640 INFO:tasks.cephfs.test_snapshots:failed while creating snap #101: CommandFailedError() 2024-09-21T07:45:57.641 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:45:57.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:57.906+0000 7f82e2a00640 1 -- 172.21.15.39:0/3394927531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82dc0d5060 msgr2=0x7f82dc0b96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:57.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:57.906+0000 7f82e2a00640 1 --2- 172.21.15.39:0/3394927531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82dc0d5060 0x7f82dc0b96a0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f82d0009820 tx=0x7f82d002a510 comp rx=0 tx=0).stop 2024-09-21T07:45:57.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:57.906+0000 7f82e2a00640 1 -- 172.21.15.39:0/3394927531 shutdown_connections 2024-09-21T07:45:57.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:57.906+0000 7f82e2a00640 1 --2- 172.21.15.39:0/3394927531 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82dc0b59e0 0x7f82dc0b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:57.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:57.906+0000 7f82e2a00640 1 --2- 172.21.15.39:0/3394927531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82dc0d5060 0x7f82dc0b96a0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:57.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:57.906+0000 7f82e2a00640 1 -- 172.21.15.39:0/3394927531 >> 172.21.15.39:0/3394927531 conn(0x7f82dc09d7d0 msgr2=0x7f82dc0d4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:57.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:57.906+0000 7f82e2a00640 1 -- 172.21.15.39:0/3394927531 shutdown_connections 2024-09-21T07:45:57.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:57.906+0000 7f82e2a00640 1 -- 172.21.15.39:0/3394927531 wait complete. 2024-09-21T07:45:57.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:57.908+0000 7f82e2a00640 1 Processor -- start 2024-09-21T07:45:57.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:57.908+0000 7f82e2a00640 1 -- start start 2024-09-21T07:45:57.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:57.910+0000 7f82e2a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82dc0b59e0 0x7f82dc151a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:57.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:57.910+0000 7f82e2a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82dc0d0e60 0x7f82dc0c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:57.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:57.910+0000 7f82e2a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82dc0d5060 0x7f82dc0cad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:57.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:57.910+0000 7f82e2a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc0d2c10 con 0x7f82dc0d5060 2024-09-21T07:45:57.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:57.910+0000 7f82e2a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc0d2a90 con 0x7f82dc0d0e60 2024-09-21T07:45:57.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:57.910+0000 7f82e2a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f82dc0d2d90 con 0x7f82dc0b59e0 2024-09-21T07:45:57.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:57.910+0000 7f82e2000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82dc0d5060 0x7f82dc0cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:57.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:57.910+0000 7f82e2000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82dc0d5060 0x7f82dc0cad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49420/0 (socket says 172.21.15.39:49420) 2024-09-21T07:45:57.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:57.910+0000 7f82e2000640 1 -- 172.21.15.39:0/2912872001 learned_addr learned my addr 172.21.15.39:0/2912872001 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:45:57.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:57.910+0000 7f82e2000640 1 -- 172.21.15.39:0/2912872001 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82dc0b59e0 msgr2=0x7f82dc151a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:57.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:57.910+0000 7f82e2000640 1 --2- 172.21.15.39:0/2912872001 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82dc0b59e0 0x7f82dc151a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:57.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:57.910+0000 7f82e2000640 1 -- 172.21.15.39:0/2912872001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82dc0d0e60 msgr2=0x7f82dc0c85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:57.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:57.910+0000 7f82e2000640 1 --2- 172.21.15.39:0/2912872001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82dc0d0e60 0x7f82dc0c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:57.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:57.911+0000 7f82e2000640 1 -- 172.21.15.39:0/2912872001 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82dc1f9b60 con 0x7f82dc0d5060 2024-09-21T07:45:57.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:57.911+0000 7f82e2000640 1 --2- 172.21.15.39:0/2912872001 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82dc0d5060 0x7f82dc0cad20 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f82d400ca50 tx=0x7f82d400cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:57.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:57.911+0000 7f82d9600640 1 -- 172.21.15.39:0/2912872001 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d4016020 con 0x7f82dc0d5060 2024-09-21T07:45:57.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:57.911+0000 7f82d9600640 1 -- 172.21.15.39:0/2912872001 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82d400ac00 con 0x7f82dc0d5060 2024-09-21T07:45:57.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:57.911+0000 7f82e2a00640 1 -- 172.21.15.39:0/2912872001 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82dc0c9f60 con 0x7f82dc0d5060 2024-09-21T07:45:57.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:57.911+0000 7f82d9600640 1 -- 172.21.15.39:0/2912872001 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d4012680 con 0x7f82dc0d5060 2024-09-21T07:45:57.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:57.911+0000 7f82e2a00640 1 -- 172.21.15.39:0/2912872001 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82dc0ca520 con 0x7f82dc0d5060 2024-09-21T07:45:57.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:57.912+0000 7f82e2a00640 1 -- 172.21.15.39:0/2912872001 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82dc0d3240 con 0x7f82dc0d5060 2024-09-21T07:45:57.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:57.914+0000 7f82d9600640 1 -- 172.21.15.39:0/2912872001 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f82d4019020 con 0x7f82dc0d5060 2024-09-21T07:45:57.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:57.915+0000 7f82d9600640 1 --2- 172.21.15.39:0/2912872001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f82bc078bc0 0x7f82bc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:57.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:57.915+0000 7f82d9600640 1 -- 172.21.15.39:0/2912872001 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 6844+0+0 (secure 0 0 0) 0x7f82d409b7d0 con 0x7f82dc0d5060 2024-09-21T07:45:57.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:57.915+0000 7f82e1600640 1 --2- 172.21.15.39:0/2912872001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f82bc078bc0 0x7f82bc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:57.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:57.916+0000 7f82e1600640 1 --2- 172.21.15.39:0/2912872001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f82bc078bc0 0x7f82bc07b0d0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f82d0034000 tx=0x7f82d00046c0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:57.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:57.919+0000 7f82d9600640 1 -- 172.21.15.39:0/2912872001 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f82d4020020 con 0x7f82dc0d5060 2024-09-21T07:45:58.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:58.155+0000 7f82e2a00640 1 -- 172.21.15.39:0/2912872001 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82dc0d1d40 con 0x7f82dc0d5060 2024-09-21T07:45:58.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:58.156+0000 7f82d9600640 1 -- 172.21.15.39:0/2912872001 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 181 v181) ==== 95+0+5234 (secure 0 0 0) 0x7f82d4063700 con 0x7f82dc0d5060 2024-09-21T07:45:58.158 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:45:58.158 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":181,"btime":"2024-09-21T07:45:40:695099+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7310,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2823960253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2823960253},{"type":"v1","addr":"172.21.15.39:6835","nonce":2823960253}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":176},{"gid":8155,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/603157278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":603157278},{"type":"v1","addr":"172.21.15.148:6836","nonce":603157278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":175},{"gid":8158,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2610135999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2610135999},{"type":"v1","addr":"172.21.15.148:6837","nonce":2610135999}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":176}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:45:37.186043+0000","modified":"2024-09-21T07:45:40.206770+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8829},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8829":{"gid":8829,"name":"a","rank":0,"incarnation":177,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/2389975689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2389975689},{"type":"v1","addr":"172.21.15.39:6834","nonce":2389975689}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8829,"qdb_cluster":[8829]},"id":8}]} 2024-09-21T07:45:58.159 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 181 2024-09-21T07:45:58.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:58.159+0000 7f82e2a00640 1 -- 172.21.15.39:0/2912872001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f82bc078bc0 msgr2=0x7f82bc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:58.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:58.159+0000 7f82e2a00640 1 --2- 172.21.15.39:0/2912872001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f82bc078bc0 0x7f82bc07b0d0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f82d0034000 tx=0x7f82d00046c0 comp rx=0 tx=0).stop 2024-09-21T07:45:58.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:58.159+0000 7f82e2a00640 1 -- 172.21.15.39:0/2912872001 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82dc0d5060 msgr2=0x7f82dc0cad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:58.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:58.159+0000 7f82e2a00640 1 --2- 172.21.15.39:0/2912872001 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82dc0d5060 0x7f82dc0cad20 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f82d400ca50 tx=0x7f82d400cf10 comp rx=0 tx=0).stop 2024-09-21T07:45:58.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:58.159+0000 7f82e2a00640 1 -- 172.21.15.39:0/2912872001 shutdown_connections 2024-09-21T07:45:58.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:58.159+0000 7f82e2a00640 1 --2- 172.21.15.39:0/2912872001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f82bc078bc0 0x7f82bc07b0d0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:58.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:58.159+0000 7f82e2a00640 1 --2- 172.21.15.39:0/2912872001 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82dc0d5060 0x7f82dc0cad20 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:58.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:58.159+0000 7f82e2a00640 1 --2- 172.21.15.39:0/2912872001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82dc0d0e60 0x7f82dc0c85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:58.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:58.159+0000 7f82e2a00640 1 --2- 172.21.15.39:0/2912872001 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82dc0b59e0 0x7f82dc151a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:58.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:58.159+0000 7f82e2a00640 1 -- 172.21.15.39:0/2912872001 >> 172.21.15.39:0/2912872001 conn(0x7f82dc09d7d0 msgr2=0x7f82dc0cf050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:58.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:58.159+0000 7f82e2a00640 1 -- 172.21.15.39:0/2912872001 shutdown_connections 2024-09-21T07:45:58.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:45:58.159+0000 7f82e2a00640 1 -- 172.21.15.39:0/2912872001 wait complete. 2024-09-21T07:45:58.185 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_max_snaps_per_dir 101 2024-09-21T07:45:58.429 INFO:teuthology.orchestra.run.smithi039.stdout:{"success":"mds_max_snaps_per_dir = '' "} 2024-09-21T07:45:58.430 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_max_snaps_per_dir = '' " } 2024-09-21T07:45:58.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:58.430 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_101) 2024-09-21T07:45:58.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:58.495 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_1) 2024-09-21T07:45:58.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:58.541 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_2) 2024-09-21T07:45:58.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:58.625 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_3) 2024-09-21T07:45:58.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:58.710 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_4) 2024-09-21T07:45:58.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:58.793 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_5) 2024-09-21T07:45:58.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:58.877 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_6) 2024-09-21T07:45:58.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:58.960 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_7) 2024-09-21T07:45:59.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:59.045 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_8) 2024-09-21T07:45:59.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:59.128 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_9) 2024-09-21T07:45:59.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:59.212 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_10) 2024-09-21T07:45:59.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:59.296 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_11) 2024-09-21T07:45:59.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:59.381 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_12) 2024-09-21T07:45:59.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:59.465 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_13) 2024-09-21T07:45:59.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:59.549 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_14) 2024-09-21T07:45:59.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:59.633 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_15) 2024-09-21T07:45:59.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:59.716 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_16) 2024-09-21T07:45:59.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:59.800 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_17) 2024-09-21T07:45:59.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:59.884 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_18) 2024-09-21T07:45:59.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:45:59.969 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_19) 2024-09-21T07:46:00.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:00.054 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_20) 2024-09-21T07:46:00.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:00.139 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_21) 2024-09-21T07:46:00.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:00.222 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_22) 2024-09-21T07:46:00.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:00.307 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_23) 2024-09-21T07:46:00.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:00.390 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_24) 2024-09-21T07:46:00.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:00.474 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_25) 2024-09-21T07:46:00.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:00.557 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_26) 2024-09-21T07:46:00.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:00.641 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_27) 2024-09-21T07:46:00.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:00.727 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_28) 2024-09-21T07:46:00.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:00.811 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_29) 2024-09-21T07:46:00.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:00.896 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_30) 2024-09-21T07:46:00.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:00.979 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_31) 2024-09-21T07:46:01.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:01.063 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_32) 2024-09-21T07:46:01.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:01.147 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_33) 2024-09-21T07:46:01.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:01.231 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_34) 2024-09-21T07:46:01.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:01.314 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_35) 2024-09-21T07:46:01.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:01.397 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_36) 2024-09-21T07:46:01.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:01.481 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_37) 2024-09-21T07:46:01.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:01.564 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_38) 2024-09-21T07:46:01.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:01.648 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_39) 2024-09-21T07:46:01.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:01.733 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_40) 2024-09-21T07:46:01.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:01.816 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_41) 2024-09-21T07:46:01.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:01.901 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_42) 2024-09-21T07:46:01.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:01.985 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_43) 2024-09-21T07:46:02.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:02.069 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_44) 2024-09-21T07:46:02.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:02.153 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_45) 2024-09-21T07:46:02.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:02.236 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_46) 2024-09-21T07:46:02.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:02.320 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_47) 2024-09-21T07:46:02.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:02.404 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_48) 2024-09-21T07:46:02.408 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:46:02.451 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:46:02.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:02.489 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_49) 2024-09-21T07:46:02.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:02.573 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_50) 2024-09-21T07:46:02.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:02.657 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_51) 2024-09-21T07:46:02.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:02.739 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_52) 2024-09-21T07:46:02.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:02.822 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_53) 2024-09-21T07:46:02.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:02.930 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_54) 2024-09-21T07:46:03.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:03.032 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_55) 2024-09-21T07:46:03.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:03.094 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_56) 2024-09-21T07:46:03.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:03.153 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_57) 2024-09-21T07:46:03.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:03.223 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_58) 2024-09-21T07:46:03.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:03.287 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_59) 2024-09-21T07:46:03.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:03.337 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_60) 2024-09-21T07:46:03.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:03.414 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_61) 2024-09-21T07:46:03.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:03.485 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_62) 2024-09-21T07:46:03.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:03.551 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_63) 2024-09-21T07:46:03.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:03.602 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_64) 2024-09-21T07:46:03.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:03.675 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_65) 2024-09-21T07:46:03.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:03.739 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_66) 2024-09-21T07:46:03.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:03.806 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_67) 2024-09-21T07:46:03.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:03.853 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_68) 2024-09-21T07:46:03.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:03.949 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_69) 2024-09-21T07:46:03.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:03.992 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_70) 2024-09-21T07:46:04.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:04.075 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_71) 2024-09-21T07:46:04.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:04.157 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_72) 2024-09-21T07:46:04.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:04.243 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_73) 2024-09-21T07:46:04.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:04.327 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_74) 2024-09-21T07:46:04.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:04.411 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_75) 2024-09-21T07:46:04.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:04.495 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_76) 2024-09-21T07:46:04.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:04.580 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_77) 2024-09-21T07:46:04.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:04.665 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_78) 2024-09-21T07:46:04.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:04.748 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_79) 2024-09-21T07:46:04.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:04.832 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_80) 2024-09-21T07:46:04.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:04.953 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_81) 2024-09-21T07:46:05.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:05.014 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_82) 2024-09-21T07:46:05.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:05.057 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_83) 2024-09-21T07:46:05.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:05.140 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_84) 2024-09-21T07:46:05.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:05.224 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_85) 2024-09-21T07:46:05.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:05.309 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_86) 2024-09-21T07:46:05.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:05.435 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_87) 2024-09-21T07:46:05.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:05.478 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_88) 2024-09-21T07:46:05.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:05.562 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_89) 2024-09-21T07:46:05.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:05.644 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_90) 2024-09-21T07:46:05.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:05.727 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_91) 2024-09-21T07:46:05.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:05.811 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_92) 2024-09-21T07:46:05.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:05.894 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_93) 2024-09-21T07:46:05.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:05.990 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_94) 2024-09-21T07:46:06.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:06.049 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_95) 2024-09-21T07:46:06.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:06.092 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_96) 2024-09-21T07:46:06.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:06.175 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_97) 2024-09-21T07:46:06.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:06.288 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_98) 2024-09-21T07:46:06.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:06.343 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_99) 2024-09-21T07:46:06.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:06.386 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_100) 2024-09-21T07:46:06.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:06.544 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_101) 2024-09-21T07:46:06.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:06.587 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts) 2024-09-21T07:46:06.665 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:46:06.665 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:46:06.789 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:46:06.885 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:46:06.885 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:46:06.885 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:46:06.885 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:46:06.935 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:46:06.935 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:46:06.974 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:46:07.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.262+0000 7fd90e200640 1 -- 172.21.15.39:0/1635475342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd908160660 msgr2=0x7fd9080cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:07.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.262+0000 7fd90e200640 1 --2- 172.21.15.39:0/1635475342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd908160660 0x7fd9080cf710 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4009860 tx=0x7fd8f402a5f0 comp rx=0 tx=0).stop 2024-09-21T07:46:07.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.262+0000 7fd90e200640 1 -- 172.21.15.39:0/1635475342 shutdown_connections 2024-09-21T07:46:07.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.262+0000 7fd90e200640 1 --2- 172.21.15.39:0/1635475342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd908160660 0x7fd9080cf710 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:07.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.262+0000 7fd90e200640 1 -- 172.21.15.39:0/1635475342 >> 172.21.15.39:0/1635475342 conn(0x7fd90806cfc0 msgr2=0x7fd90806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:07.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.262+0000 7fd90e200640 1 -- 172.21.15.39:0/1635475342 shutdown_connections 2024-09-21T07:46:07.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.262+0000 7fd90e200640 1 -- 172.21.15.39:0/1635475342 wait complete. 2024-09-21T07:46:07.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.265+0000 7fd90e200640 1 Processor -- start 2024-09-21T07:46:07.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.265+0000 7fd90e200640 1 -- start start 2024-09-21T07:46:07.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.266+0000 7fd90e200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9080cfc50 0x7fd9081e6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:07.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.266+0000 7fd90e200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd908160250 0x7fd9081efa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:07.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.266+0000 7fd90e200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd908160660 0x7fd9081f9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:07.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.266+0000 7fd90e200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd9080cced0 con 0x7fd908160660 2024-09-21T07:46:07.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.266+0000 7fd90e200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd9080ccd50 con 0x7fd9080cfc50 2024-09-21T07:46:07.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.266+0000 7fd90e200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd9080cd050 con 0x7fd908160250 2024-09-21T07:46:07.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.266+0000 7fd90d800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd908160660 0x7fd9081f9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:07.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.267+0000 7fd90d800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd908160660 0x7fd9081f9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51056/0 (socket says 172.21.15.39:51056) 2024-09-21T07:46:07.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.267+0000 7fd90d800640 1 -- 172.21.15.39:0/430162934 learned_addr learned my addr 172.21.15.39:0/430162934 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:07.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.267+0000 7fd907e00640 1 --2- 172.21.15.39:0/430162934 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd908160250 0x7fd9081efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:07.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.267+0000 7fd90d800640 1 -- 172.21.15.39:0/430162934 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd908160250 msgr2=0x7fd9081efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:07.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.267+0000 7fd90ce00640 1 --2- 172.21.15.39:0/430162934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9080cfc50 0x7fd9081e6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:07.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.267+0000 7fd90d800640 1 --2- 172.21.15.39:0/430162934 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd908160250 0x7fd9081efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:07.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.267+0000 7fd90d800640 1 -- 172.21.15.39:0/430162934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9080cfc50 msgr2=0x7fd9081e6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:07.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.267+0000 7fd90d800640 1 --2- 172.21.15.39:0/430162934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9080cfc50 0x7fd9081e6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:07.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.267+0000 7fd90d800640 1 -- 172.21.15.39:0/430162934 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9080aafe0 con 0x7fd908160660 2024-09-21T07:46:07.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.267+0000 7fd907e00640 1 --2- 172.21.15.39:0/430162934 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd908160250 0x7fd9081efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:46:07.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.267+0000 7fd90ce00640 1 --2- 172.21.15.39:0/430162934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9080cfc50 0x7fd9081e6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:46:07.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.267+0000 7fd90d800640 1 --2- 172.21.15.39:0/430162934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd908160660 0x7fd9081f9230 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fd90000caa0 tx=0x7fd90000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:07.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.268+0000 7fd904c00640 1 -- 172.21.15.39:0/430162934 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd900016020 con 0x7fd908160660 2024-09-21T07:46:07.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.268+0000 7fd904c00640 1 -- 172.21.15.39:0/430162934 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd90000ac50 con 0x7fd908160660 2024-09-21T07:46:07.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.268+0000 7fd904c00640 1 -- 172.21.15.39:0/430162934 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd900012680 con 0x7fd908160660 2024-09-21T07:46:07.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.268+0000 7fd90e200640 1 -- 172.21.15.39:0/430162934 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9080ab520 con 0x7fd908160660 2024-09-21T07:46:07.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.268+0000 7fd90e200640 1 -- 172.21.15.39:0/430162934 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9080abae0 con 0x7fd908160660 2024-09-21T07:46:07.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.269+0000 7fd90e200640 1 -- 172.21.15.39:0/430162934 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9080cf710 con 0x7fd908160660 2024-09-21T07:46:07.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.270+0000 7fd904c00640 1 -- 172.21.15.39:0/430162934 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd900019020 con 0x7fd908160660 2024-09-21T07:46:07.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.271+0000 7fd904c00640 1 --2- 172.21.15.39:0/430162934 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd8e8078b90 0x7fd8e807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:07.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.271+0000 7fd904c00640 1 -- 172.21.15.39:0/430162934 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 6932+0+0 (secure 0 0 0) 0x7fd90009b840 con 0x7fd908160660 2024-09-21T07:46:07.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.272+0000 7fd90ce00640 1 --2- 172.21.15.39:0/430162934 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd8e8078b90 0x7fd8e807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:07.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.273+0000 7fd90ce00640 1 --2- 172.21.15.39:0/430162934 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd8e8078b90 0x7fd8e807b0a0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc007fd0 tx=0x7fd8fc009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:07.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.276+0000 7fd904c00640 1 -- 172.21.15.39:0/430162934 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd900020020 con 0x7fd908160660 2024-09-21T07:46:07.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.513+0000 7fd90e200640 1 -- 172.21.15.39:0/430162934 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd9080ab2b0 con 0x7fd908160660 2024-09-21T07:46:07.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.514+0000 7fd904c00640 1 -- 172.21.15.39:0/430162934 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v162) ==== 80+0+0 (secure 0 0 0) 0x7fd9000637a0 con 0x7fd908160660 2024-09-21T07:46:07.515 INFO:teuthology.orchestra.run.smithi039.stderr:listed 0 entries 2024-09-21T07:46:07.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.516+0000 7fd90e200640 1 -- 172.21.15.39:0/430162934 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd8e8078b90 msgr2=0x7fd8e807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:07.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.516+0000 7fd90e200640 1 --2- 172.21.15.39:0/430162934 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd8e8078b90 0x7fd8e807b0a0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc007fd0 tx=0x7fd8fc009040 comp rx=0 tx=0).stop 2024-09-21T07:46:07.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.516+0000 7fd90e200640 1 -- 172.21.15.39:0/430162934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd908160660 msgr2=0x7fd9081f9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:07.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.516+0000 7fd90e200640 1 --2- 172.21.15.39:0/430162934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd908160660 0x7fd9081f9230 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fd90000caa0 tx=0x7fd90000cf60 comp rx=0 tx=0).stop 2024-09-21T07:46:07.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.516+0000 7fd90e200640 1 -- 172.21.15.39:0/430162934 shutdown_connections 2024-09-21T07:46:07.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.516+0000 7fd90e200640 1 --2- 172.21.15.39:0/430162934 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd8e8078b90 0x7fd8e807b0a0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:07.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.516+0000 7fd90e200640 1 --2- 172.21.15.39:0/430162934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd908160660 0x7fd9081f9230 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:07.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.516+0000 7fd90e200640 1 --2- 172.21.15.39:0/430162934 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd908160250 0x7fd9081efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:07.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.516+0000 7fd90e200640 1 --2- 172.21.15.39:0/430162934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9080cfc50 0x7fd9081e6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:07.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.516+0000 7fd90e200640 1 -- 172.21.15.39:0/430162934 >> 172.21.15.39:0/430162934 conn(0x7fd90806cfc0 msgr2=0x7fd9080cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:07.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.516+0000 7fd90e200640 1 -- 172.21.15.39:0/430162934 shutdown_connections 2024-09-21T07:46:07.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:07.516+0000 7fd90e200640 1 -- 172.21.15.39:0/430162934 wait complete. 2024-09-21T07:46:07.540 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T07:46:07.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:07.541 DEBUG:teuthology.orchestra.run.smithi039:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T07:46:07.635 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T07:46:07.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:07.635 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:46:07.671 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:46:07.672 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:46:07.672 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:46:07.707 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:46:07.708 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:46:07.708 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:46:07.708 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:46:07.746 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:46:07.746 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:46:07.747 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:46:07.747 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:46:07.783 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:46:07.783 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:46:08.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.044+0000 7f251d200640 1 -- 172.21.15.39:0/1328005710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24fc003c30 msgr2=0x7f24fc004080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:08.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.044+0000 7f251d200640 1 --2- 172.21.15.39:0/1328005710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24fc003c30 0x7f24fc004080 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f2508001940 tx=0x7f24fc005830 comp rx=0 tx=0).stop 2024-09-21T07:46:08.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.046+0000 7f251d200640 1 -- 172.21.15.39:0/1328005710 shutdown_connections 2024-09-21T07:46:08.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.046+0000 7f251d200640 1 --2- 172.21.15.39:0/1328005710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f24fc003c30 0x7f24fc004080 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:08.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.046+0000 7f251d200640 1 -- 172.21.15.39:0/1328005710 >> 172.21.15.39:0/1328005710 conn(0x7f251806cde0 msgr2=0x7f251806d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:08.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.046+0000 7f251d200640 1 -- 172.21.15.39:0/1328005710 shutdown_connections 2024-09-21T07:46:08.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.046+0000 7f251d200640 1 -- 172.21.15.39:0/1328005710 wait complete. 2024-09-21T07:46:08.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.052+0000 7f251d200640 1 Processor -- start 2024-09-21T07:46:08.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.052+0000 7f251d200640 1 -- start start 2024-09-21T07:46:08.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.055+0000 7f251d200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f24fc003c30 0x7f2518150b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:08.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.055+0000 7f251d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25181f5110 0x7f25180b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:08.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.055+0000 7f251d200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2518159520 0x7f2518159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:08.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.055+0000 7f251d200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f251815b7c0 con 0x7f25181f5110 2024-09-21T07:46:08.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.055+0000 7f251d200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f251815b640 con 0x7f2518159520 2024-09-21T07:46:08.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.055+0000 7f251d200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f2518151870 con 0x7f24fc003c30 2024-09-21T07:46:08.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.056+0000 7f2517e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2518159520 0x7f2518159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:08.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.056+0000 7f2517400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f24fc003c30 0x7f2518150b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:08.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.056+0000 7f2517e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2518159520 0x7f2518159930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.39:45718/0 (socket says 172.21.15.39:45718) 2024-09-21T07:46:08.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.056+0000 7f2517e00640 1 -- 172.21.15.39:0/246054153 learned_addr learned my addr 172.21.15.39:0/246054153 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:08.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.056+0000 7f2516a00640 1 --2- 172.21.15.39:0/246054153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25181f5110 0x7f25180b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:08.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.058+0000 7f2517e00640 1 -- 172.21.15.39:0/246054153 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f24fc003c30 msgr2=0x7f2518150b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:08.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.058+0000 7f2517e00640 1 --2- 172.21.15.39:0/246054153 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f24fc003c30 0x7f2518150b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:08.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.058+0000 7f2517e00640 1 -- 172.21.15.39:0/246054153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25181f5110 msgr2=0x7f25180b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:08.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.059+0000 7f2517e00640 1 --2- 172.21.15.39:0/246054153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25181f5110 0x7f25180b1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:08.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.059+0000 7f2517e00640 1 -- 172.21.15.39:0/246054153 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2518159e70 con 0x7f2518159520 2024-09-21T07:46:08.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.059+0000 7f2516a00640 1 --2- 172.21.15.39:0/246054153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25181f5110 0x7f25180b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:46:08.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.060+0000 7f2517e00640 1 --2- 172.21.15.39:0/246054153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2518159520 0x7f2518159930 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f251815e440 tx=0x7f251000dd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:08.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.061+0000 7f250f400640 1 -- 172.21.15.39:0/246054153 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2510019070 con 0x7f2518159520 2024-09-21T07:46:08.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.061+0000 7f250f400640 1 -- 172.21.15.39:0/246054153 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2510008d60 con 0x7f2518159520 2024-09-21T07:46:08.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.061+0000 7f250f400640 1 -- 172.21.15.39:0/246054153 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2504006ae0 con 0x7f2518159520 2024-09-21T07:46:08.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.062+0000 7f251d200640 1 -- 172.21.15.39:0/246054153 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25180af5d0 con 0x7f2518159520 2024-09-21T07:46:08.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.063+0000 7f251d200640 1 -- 172.21.15.39:0/246054153 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25180b26e0 con 0x7f2518159520 2024-09-21T07:46:08.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.066+0000 7f251d200640 1 -- 172.21.15.39:0/246054153 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2518151cf0 con 0x7f2518159520 2024-09-21T07:46:08.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.067+0000 7f250f400640 1 -- 172.21.15.39:0/246054153 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f251001c070 con 0x7f2518159520 2024-09-21T07:46:08.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.069+0000 7f250f400640 1 --2- 172.21.15.39:0/246054153 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f24f8078be0 0x7f24f807b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:08.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.069+0000 7f250f400640 1 -- 172.21.15.39:0/246054153 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 6916+0+0 (secure 0 0 0) 0x7f251009a180 con 0x7f2518159520 2024-09-21T07:46:08.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.074+0000 7f2517400640 1 --2- 172.21.15.39:0/246054153 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f24f8078be0 0x7f24f807b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:08.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.076+0000 7f2517400640 1 --2- 172.21.15.39:0/246054153 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f24f8078be0 0x7f24f807b0f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f2508032000 tx=0x7f2508001380 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:08.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.080+0000 7f250f400640 1 -- 172.21.15.39:0/246054153 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2518159e70 con 0x7f2518159520 2024-09-21T07:46:08.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.337+0000 7f251d200640 1 -- 172.21.15.39:0/246054153 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f251806f6e0 con 0x7f2518159520 2024-09-21T07:46:08.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.338+0000 7f250f400640 1 -- 172.21.15.39:0/246054153 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 181 v181) ==== 95+0+5234 (secure 0 0 0) 0x7f2510061cb0 con 0x7f2518159520 2024-09-21T07:46:08.339 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:46:08.340 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":181,"btime":"2024-09-21T07:45:40:695099+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7310,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2823960253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2823960253},{"type":"v1","addr":"172.21.15.39:6835","nonce":2823960253}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":176},{"gid":8155,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/603157278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":603157278},{"type":"v1","addr":"172.21.15.148:6836","nonce":603157278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":175},{"gid":8158,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2610135999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2610135999},{"type":"v1","addr":"172.21.15.148:6837","nonce":2610135999}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":176}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:45:37.186043+0000","modified":"2024-09-21T07:45:40.206770+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8829},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8829":{"gid":8829,"name":"a","rank":0,"incarnation":177,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/2389975689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2389975689},{"type":"v1","addr":"172.21.15.39:6834","nonce":2389975689}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8829,"qdb_cluster":[8829]},"id":8}]} 2024-09-21T07:46:08.340 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 181 2024-09-21T07:46:08.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.341+0000 7f251d200640 1 -- 172.21.15.39:0/246054153 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f24f8078be0 msgr2=0x7f24f807b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:08.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.341+0000 7f251d200640 1 --2- 172.21.15.39:0/246054153 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f24f8078be0 0x7f24f807b0f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f2508032000 tx=0x7f2508001380 comp rx=0 tx=0).stop 2024-09-21T07:46:08.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.341+0000 7f251d200640 1 -- 172.21.15.39:0/246054153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2518159520 msgr2=0x7f2518159930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:08.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.341+0000 7f251d200640 1 --2- 172.21.15.39:0/246054153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2518159520 0x7f2518159930 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f251815e440 tx=0x7f251000dd70 comp rx=0 tx=0).stop 2024-09-21T07:46:08.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.341+0000 7f251d200640 1 -- 172.21.15.39:0/246054153 shutdown_connections 2024-09-21T07:46:08.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.341+0000 7f251d200640 1 --2- 172.21.15.39:0/246054153 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f24f8078be0 0x7f24f807b0f0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:08.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.341+0000 7f251d200640 1 --2- 172.21.15.39:0/246054153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2518159520 0x7f2518159930 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:08.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.341+0000 7f251d200640 1 --2- 172.21.15.39:0/246054153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25181f5110 0x7f25180b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:08.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.341+0000 7f251d200640 1 --2- 172.21.15.39:0/246054153 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f24fc003c30 0x7f2518150b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:08.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.341+0000 7f251d200640 1 -- 172.21.15.39:0/246054153 >> 172.21.15.39:0/246054153 conn(0x7f251806cde0 msgr2=0x7f2518151f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:08.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.341+0000 7f251d200640 1 -- 172.21.15.39:0/246054153 shutdown_connections 2024-09-21T07:46:08.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.341+0000 7f251d200640 1 -- 172.21.15.39:0/246054153 wait complete. 2024-09-21T07:46:08.365 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:46:08.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.592+0000 7fcccbe00640 1 -- 172.21.15.39:0/1383815111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcccc0d04b0 msgr2=0x7fcccc0d08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:08.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.592+0000 7fcccbe00640 1 --2- 172.21.15.39:0/1383815111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcccc0d04b0 0x7fcccc0d08c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fccc4005fd0 tx=0x7fccc402aaa0 comp rx=0 tx=0).stop 2024-09-21T07:46:08.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.592+0000 7fcccbe00640 1 -- 172.21.15.39:0/1383815111 shutdown_connections 2024-09-21T07:46:08.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.592+0000 7fcccbe00640 1 --2- 172.21.15.39:0/1383815111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcccc0d04b0 0x7fcccc0d08c0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:08.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.592+0000 7fcccbe00640 1 --2- 172.21.15.39:0/1383815111 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcccc15cc00 0x7fcccc15d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:08.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.592+0000 7fcccbe00640 1 -- 172.21.15.39:0/1383815111 >> 172.21.15.39:0/1383815111 conn(0x7fcccc06cfc0 msgr2=0x7fcccc06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:08.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.592+0000 7fcccbe00640 1 -- 172.21.15.39:0/1383815111 shutdown_connections 2024-09-21T07:46:08.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.592+0000 7fcccbe00640 1 -- 172.21.15.39:0/1383815111 wait complete. 2024-09-21T07:46:08.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.594+0000 7fcccbe00640 1 Processor -- start 2024-09-21T07:46:08.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.594+0000 7fcccbe00640 1 -- start start 2024-09-21T07:46:08.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.596+0000 7fcccbe00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcccc0cfa20 0x7fcccc152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:08.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.596+0000 7fcccbe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcccc0d04b0 0x7fcccc1556f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:08.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.596+0000 7fcccbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcccc15cc00 0x7fcccc155c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:08.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.596+0000 7fcccbe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcccc06ee70 con 0x7fcccc15cc00 2024-09-21T07:46:08.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.596+0000 7fcccbe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcccc06ecf0 con 0x7fcccc0d04b0 2024-09-21T07:46:08.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.596+0000 7fcccbe00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fcccc06eff0 con 0x7fcccc0cfa20 2024-09-21T07:46:08.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.596+0000 7fcccb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcccc15cc00 0x7fcccc155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:08.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.596+0000 7fcccb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcccc15cc00 0x7fcccc155c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51072/0 (socket says 172.21.15.39:51072) 2024-09-21T07:46:08.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.596+0000 7fccc3e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcccc0d04b0 0x7fcccc1556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:08.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.596+0000 7fcccb400640 1 -- 172.21.15.39:0/3833635720 learned_addr learned my addr 172.21.15.39:0/3833635720 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:08.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.596+0000 7fcccaa00640 1 --2- 172.21.15.39:0/3833635720 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcccc0cfa20 0x7fcccc152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:08.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.597+0000 7fccc3e00640 1 -- 172.21.15.39:0/3833635720 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcccc0cfa20 msgr2=0x7fcccc152d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:08.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.597+0000 7fccc3e00640 1 --2- 172.21.15.39:0/3833635720 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcccc0cfa20 0x7fcccc152d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:08.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.597+0000 7fccc3e00640 1 -- 172.21.15.39:0/3833635720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcccc15cc00 msgr2=0x7fcccc155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:08.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.597+0000 7fccc3e00640 1 --2- 172.21.15.39:0/3833635720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcccc15cc00 0x7fcccc155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:08.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.597+0000 7fccc3e00640 1 -- 172.21.15.39:0/3833635720 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcccc0ae0c0 con 0x7fcccc0d04b0 2024-09-21T07:46:08.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.597+0000 7fcccb400640 1 --2- 172.21.15.39:0/3833635720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcccc15cc00 0x7fcccc155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:46:08.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.597+0000 7fccc3e00640 1 --2- 172.21.15.39:0/3833635720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcccc0d04b0 0x7fcccc1556f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fccb400e700 tx=0x7fccb400ebc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:08.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.597+0000 7fccc3400640 1 -- 172.21.15.39:0/3833635720 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb400cab0 con 0x7fcccc0d04b0 2024-09-21T07:46:08.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.597+0000 7fccc3400640 1 -- 172.21.15.39:0/3833635720 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fccb400cd40 con 0x7fcccc0d04b0 2024-09-21T07:46:08.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.597+0000 7fccc3400640 1 -- 172.21.15.39:0/3833635720 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb4010690 con 0x7fcccc0d04b0 2024-09-21T07:46:08.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.597+0000 7fcccbe00640 1 -- 172.21.15.39:0/3833635720 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcccc0ae3a0 con 0x7fcccc0d04b0 2024-09-21T07:46:08.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.597+0000 7fcccbe00640 1 -- 172.21.15.39:0/3833635720 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcccc0ae960 con 0x7fcccc0d04b0 2024-09-21T07:46:08.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.599+0000 7fcccbe00640 1 -- 172.21.15.39:0/3833635720 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcccc06f510 con 0x7fcccc0d04b0 2024-09-21T07:46:08.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.600+0000 7fccc3400640 1 -- 172.21.15.39:0/3833635720 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fccb4010830 con 0x7fcccc0d04b0 2024-09-21T07:46:08.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.601+0000 7fccc3400640 1 --2- 172.21.15.39:0/3833635720 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcca8078b90 0x7fcca807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:08.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.601+0000 7fccc3400640 1 -- 172.21.15.39:0/3833635720 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 6916+0+0 (secure 0 0 0) 0x7fccb409bbd0 con 0x7fcccc0d04b0 2024-09-21T07:46:08.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.602+0000 7fcccaa00640 1 --2- 172.21.15.39:0/3833635720 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcca8078b90 0x7fcca807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:08.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.602+0000 7fcccaa00640 1 --2- 172.21.15.39:0/3833635720 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcca8078b90 0x7fcca807b0a0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fccbc001720 tx=0x7fccbc009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:08.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.606+0000 7fccc3400640 1 -- 172.21.15.39:0/3833635720 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fccb4063460 con 0x7fcccc0d04b0 2024-09-21T07:46:08.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.844+0000 7fcccbe00640 1 -- 172.21.15.39:0/3833635720 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcccc15d330 con 0x7fcccc0d04b0 2024-09-21T07:46:08.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.845+0000 7fccc3400640 1 -- 172.21.15.39:0/3833635720 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 181 v181) ==== 95+0+5234 (secure 0 0 0) 0x7fccb4067e90 con 0x7fcccc0d04b0 2024-09-21T07:46:08.846 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:46:08.847 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":181,"btime":"2024-09-21T07:45:40:695099+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7310,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2823960253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2823960253},{"type":"v1","addr":"172.21.15.39:6835","nonce":2823960253}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":176},{"gid":8155,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/603157278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":603157278},{"type":"v1","addr":"172.21.15.148:6836","nonce":603157278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":175},{"gid":8158,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2610135999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2610135999},{"type":"v1","addr":"172.21.15.148:6837","nonce":2610135999}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":176}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:45:37.186043+0000","modified":"2024-09-21T07:45:40.206770+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8829},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8829":{"gid":8829,"name":"a","rank":0,"incarnation":177,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/2389975689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2389975689},{"type":"v1","addr":"172.21.15.39:6834","nonce":2389975689}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8829,"qdb_cluster":[8829]},"id":8}]} 2024-09-21T07:46:08.847 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 181 2024-09-21T07:46:08.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.848+0000 7fcccbe00640 1 -- 172.21.15.39:0/3833635720 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcca8078b90 msgr2=0x7fcca807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:08.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.848+0000 7fcccbe00640 1 --2- 172.21.15.39:0/3833635720 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcca8078b90 0x7fcca807b0a0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fccbc001720 tx=0x7fccbc009040 comp rx=0 tx=0).stop 2024-09-21T07:46:08.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.848+0000 7fcccbe00640 1 -- 172.21.15.39:0/3833635720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcccc0d04b0 msgr2=0x7fcccc1556f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:08.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.848+0000 7fcccbe00640 1 --2- 172.21.15.39:0/3833635720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcccc0d04b0 0x7fcccc1556f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fccb400e700 tx=0x7fccb400ebc0 comp rx=0 tx=0).stop 2024-09-21T07:46:08.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.848+0000 7fcccbe00640 1 -- 172.21.15.39:0/3833635720 shutdown_connections 2024-09-21T07:46:08.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.848+0000 7fcccbe00640 1 --2- 172.21.15.39:0/3833635720 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcca8078b90 0x7fcca807b0a0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:08.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.848+0000 7fcccbe00640 1 --2- 172.21.15.39:0/3833635720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcccc15cc00 0x7fcccc155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:08.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.848+0000 7fcccbe00640 1 --2- 172.21.15.39:0/3833635720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcccc0d04b0 0x7fcccc1556f0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:08.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.848+0000 7fcccbe00640 1 --2- 172.21.15.39:0/3833635720 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcccc0cfa20 0x7fcccc152d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:08.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.848+0000 7fcccbe00640 1 -- 172.21.15.39:0/3833635720 >> 172.21.15.39:0/3833635720 conn(0x7fcccc06cfc0 msgr2=0x7fcccc0cc3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:08.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.848+0000 7fcccbe00640 1 -- 172.21.15.39:0/3833635720 shutdown_connections 2024-09-21T07:46:08.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:08.848+0000 7fcccbe00640 1 -- 172.21.15.39:0/3833635720 wait complete. 2024-09-21T07:46:08.870 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:46:09.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.096+0000 7f2d73800640 1 -- 172.21.15.39:0/368148235 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d6c0b5240 msgr2=0x7f2d6c15c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:09.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.096+0000 7f2d73800640 1 --2- 172.21.15.39:0/368148235 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d6c0b5240 0x7f2d6c15c690 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f2d60009830 tx=0x7f2d6002a510 comp rx=0 tx=0).stop 2024-09-21T07:46:09.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.097+0000 7f2d73800640 1 -- 172.21.15.39:0/368148235 shutdown_connections 2024-09-21T07:46:09.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.097+0000 7f2d73800640 1 --2- 172.21.15.39:0/368148235 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d6c0d0d40 0x7f2d6c0b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:09.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.097+0000 7f2d73800640 1 --2- 172.21.15.39:0/368148235 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d6c0b5240 0x7f2d6c15c690 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:09.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.097+0000 7f2d73800640 1 -- 172.21.15.39:0/368148235 >> 172.21.15.39:0/368148235 conn(0x7f2d6c06cfc0 msgr2=0x7f2d6c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:09.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.097+0000 7f2d73800640 1 -- 172.21.15.39:0/368148235 shutdown_connections 2024-09-21T07:46:09.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.097+0000 7f2d73800640 1 -- 172.21.15.39:0/368148235 wait complete. 2024-09-21T07:46:09.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.099+0000 7f2d73800640 1 Processor -- start 2024-09-21T07:46:09.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.099+0000 7f2d73800640 1 -- start start 2024-09-21T07:46:09.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.100+0000 7f2d73800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d6c0b5240 0x7f2d6c0b29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:09.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.100+0000 7f2d73800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d6c0cf920 0x7f2d6c1f89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:09.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.100+0000 7f2d73800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d6c1f8f20 0x7f2d6c14a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:09.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.101+0000 7f2d73800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c0b4910 con 0x7f2d6c0b5240 2024-09-21T07:46:09.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.101+0000 7f2d73800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c0cff40 con 0x7f2d6c0cf920 2024-09-21T07:46:09.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.101+0000 7f2d73800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f2d6c0b4a90 con 0x7f2d6c1f8f20 2024-09-21T07:46:09.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.101+0000 7f2d70e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d6c0b5240 0x7f2d6c0b29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:09.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.101+0000 7f2d70e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d6c0b5240 0x7f2d6c0b29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51088/0 (socket says 172.21.15.39:51088) 2024-09-21T07:46:09.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.101+0000 7f2d70e00640 1 -- 172.21.15.39:0/4176171085 learned_addr learned my addr 172.21.15.39:0/4176171085 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:09.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.101+0000 7f2d6be00640 1 --2- 172.21.15.39:0/4176171085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d6c0cf920 0x7f2d6c1f89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:09.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.101+0000 7f2d71800640 1 --2- 172.21.15.39:0/4176171085 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d6c1f8f20 0x7f2d6c14a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:09.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.101+0000 7f2d70e00640 1 -- 172.21.15.39:0/4176171085 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d6c1f8f20 msgr2=0x7f2d6c14a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:09.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.101+0000 7f2d70e00640 1 --2- 172.21.15.39:0/4176171085 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d6c1f8f20 0x7f2d6c14a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:09.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.101+0000 7f2d70e00640 1 -- 172.21.15.39:0/4176171085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d6c0cf920 msgr2=0x7f2d6c1f89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:09.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.101+0000 7f2d70e00640 1 --2- 172.21.15.39:0/4176171085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d6c0cf920 0x7f2d6c1f89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:09.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.101+0000 7f2d70e00640 1 -- 172.21.15.39:0/4176171085 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d6c1f9330 con 0x7f2d6c0b5240 2024-09-21T07:46:09.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.101+0000 7f2d6be00640 1 --2- 172.21.15.39:0/4176171085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d6c0cf920 0x7f2d6c1f89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:46:09.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.101+0000 7f2d71800640 1 --2- 172.21.15.39:0/4176171085 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d6c1f8f20 0x7f2d6c14a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:46:09.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.101+0000 7f2d70e00640 1 --2- 172.21.15.39:0/4176171085 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d6c0b5240 0x7f2d6c0b29f0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f2d60009950 tx=0x7f2d60033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:09.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.102+0000 7f2d68c00640 1 -- 172.21.15.39:0/4176171085 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d60031d60 con 0x7f2d6c0b5240 2024-09-21T07:46:09.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.102+0000 7f2d68c00640 1 -- 172.21.15.39:0/4176171085 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d60033e10 con 0x7f2d6c0b5240 2024-09-21T07:46:09.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.102+0000 7f2d68c00640 1 -- 172.21.15.39:0/4176171085 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d60005260 con 0x7f2d6c0b5240 2024-09-21T07:46:09.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.102+0000 7f2d73800640 1 -- 172.21.15.39:0/4176171085 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d6c0b2f30 con 0x7f2d6c0b5240 2024-09-21T07:46:09.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.102+0000 7f2d73800640 1 -- 172.21.15.39:0/4176171085 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d6c150d70 con 0x7f2d6c0b5240 2024-09-21T07:46:09.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.103+0000 7f2d73800640 1 -- 172.21.15.39:0/4176171085 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d6c0cd960 con 0x7f2d6c0b5240 2024-09-21T07:46:09.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.104+0000 7f2d68c00640 1 -- 172.21.15.39:0/4176171085 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2d60005400 con 0x7f2d6c0b5240 2024-09-21T07:46:09.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.105+0000 7f2d68c00640 1 --2- 172.21.15.39:0/4176171085 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2d4c078bc0 0x7f2d4c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:09.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.105+0000 7f2d68c00640 1 -- 172.21.15.39:0/4176171085 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 6916+0+0 (secure 0 0 0) 0x7f2d600bc5a0 con 0x7f2d6c0b5240 2024-09-21T07:46:09.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.106+0000 7f2d6be00640 1 --2- 172.21.15.39:0/4176171085 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2d4c078bc0 0x7f2d4c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:09.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.107+0000 7f2d6be00640 1 --2- 172.21.15.39:0/4176171085 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2d4c078bc0 0x7f2d4c07b0d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c14e8a0 tx=0x7f2d58009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:09.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.109+0000 7f2d68c00640 1 -- 172.21.15.39:0/4176171085 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2d6c1f9330 con 0x7f2d6c0b5240 2024-09-21T07:46:09.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.305+0000 7f2d73800640 1 -- 172.21.15.39:0/4176171085 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2d6c15c690 con 0x7f2d6c0b5240 2024-09-21T07:46:09.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.307+0000 7f2d68c00640 1 -- 172.21.15.39:0/4176171085 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v163) ==== 74+0+22363 (secure 0 0 0) 0x7f2d6003f030 con 0x7f2d6c0b5240 2024-09-21T07:46:09.308 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:46:09.308 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":163,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:46:07.903428+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:45:35.115582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-09-21T07:45:36.142685+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":203,"snap_epoch":163,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]},{"pool":20,"snaps":[{"begin":2,"length":202}]}],"new_removed_snaps":[{"pool":20,"snaps":[{"begin":93,"length":10},{"begin":194,"length":10}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:46:09.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.309+0000 7f2d73800640 1 -- 172.21.15.39:0/4176171085 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2d4c078bc0 msgr2=0x7f2d4c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:09.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.309+0000 7f2d73800640 1 --2- 172.21.15.39:0/4176171085 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2d4c078bc0 0x7f2d4c07b0d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c14e8a0 tx=0x7f2d58009040 comp rx=0 tx=0).stop 2024-09-21T07:46:09.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.309+0000 7f2d73800640 1 -- 172.21.15.39:0/4176171085 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d6c0b5240 msgr2=0x7f2d6c0b29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:09.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.309+0000 7f2d73800640 1 --2- 172.21.15.39:0/4176171085 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d6c0b5240 0x7f2d6c0b29f0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f2d60009950 tx=0x7f2d60033c20 comp rx=0 tx=0).stop 2024-09-21T07:46:09.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.309+0000 7f2d73800640 1 -- 172.21.15.39:0/4176171085 shutdown_connections 2024-09-21T07:46:09.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.309+0000 7f2d73800640 1 --2- 172.21.15.39:0/4176171085 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2d4c078bc0 0x7f2d4c07b0d0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:09.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.309+0000 7f2d73800640 1 --2- 172.21.15.39:0/4176171085 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d6c1f8f20 0x7f2d6c14a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:09.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.309+0000 7f2d73800640 1 --2- 172.21.15.39:0/4176171085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d6c0cf920 0x7f2d6c1f89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:09.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.309+0000 7f2d73800640 1 --2- 172.21.15.39:0/4176171085 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d6c0b5240 0x7f2d6c0b29f0 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:09.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.309+0000 7f2d73800640 1 -- 172.21.15.39:0/4176171085 >> 172.21.15.39:0/4176171085 conn(0x7f2d6c06cfc0 msgr2=0x7f2d6c0d05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:09.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.309+0000 7f2d73800640 1 -- 172.21.15.39:0/4176171085 shutdown_connections 2024-09-21T07:46:09.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.309+0000 7f2d73800640 1 -- 172.21.15.39:0/4176171085 wait complete. 2024-09-21T07:46:09.334 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T07:46:09.334 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:46:09.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.559+0000 7fc393000640 1 -- 172.21.15.39:0/3658776552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc38c0b3b50 msgr2=0x7fc38c15c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:09.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.559+0000 7fc393000640 1 --2- 172.21.15.39:0/3658776552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc38c0b3b50 0x7fc38c15c510 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fc380005fd0 tx=0x7fc38002a4f0 comp rx=0 tx=0).stop 2024-09-21T07:46:09.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.559+0000 7fc393000640 1 -- 172.21.15.39:0/3658776552 shutdown_connections 2024-09-21T07:46:09.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.559+0000 7fc393000640 1 --2- 172.21.15.39:0/3658776552 >> v2:172.21.15.148:3300/0 conn(0x7fc38c0d0bc0 0x7fc38c1554a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:09.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.559+0000 7fc393000640 1 --2- 172.21.15.39:0/3658776552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc38c0b3b50 0x7fc38c15c510 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:09.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.559+0000 7fc393000640 1 -- 172.21.15.39:0/3658776552 >> 172.21.15.39:0/3658776552 conn(0x7fc38c06cfc0 msgr2=0x7fc38c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:09.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.559+0000 7fc393000640 1 -- 172.21.15.39:0/3658776552 shutdown_connections 2024-09-21T07:46:09.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.559+0000 7fc393000640 1 -- 172.21.15.39:0/3658776552 wait complete. 2024-09-21T07:46:09.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.562+0000 7fc393000640 1 Processor -- start 2024-09-21T07:46:09.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.562+0000 7fc393000640 1 -- start start 2024-09-21T07:46:09.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.563+0000 7fc393000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc38c073f90 0x7fc38c1f8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:09.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.563+0000 7fc393000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc38c0b3b50 0x7fc38c1f9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:09.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.563+0000 7fc393000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc38c0d0bc0 0x7fc38c1497c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:09.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.563+0000 7fc393000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc38c074520 con 0x7fc38c073f90 2024-09-21T07:46:09.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.563+0000 7fc393000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc38c0743a0 con 0x7fc38c0b3b50 2024-09-21T07:46:09.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.563+0000 7fc393000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc38c0746a0 con 0x7fc38c0d0bc0 2024-09-21T07:46:09.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.563+0000 7fc391c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc38c073f90 0x7fc38c1f8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:09.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.563+0000 7fc391c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc38c073f90 0x7fc38c1f8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51106/0 (socket says 172.21.15.39:51106) 2024-09-21T07:46:09.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.563+0000 7fc391200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc38c0b3b50 0x7fc38c1f9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:09.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.563+0000 7fc391c00640 1 -- 172.21.15.39:0/218902026 learned_addr learned my addr 172.21.15.39:0/218902026 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:09.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.564+0000 7fc392600640 1 --2- 172.21.15.39:0/218902026 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc38c0d0bc0 0x7fc38c1497c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:09.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.564+0000 7fc391c00640 1 -- 172.21.15.39:0/218902026 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc38c0d0bc0 msgr2=0x7fc38c1497c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:09.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.564+0000 7fc391c00640 1 --2- 172.21.15.39:0/218902026 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc38c0d0bc0 0x7fc38c1497c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:09.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.564+0000 7fc391c00640 1 -- 172.21.15.39:0/218902026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc38c0b3b50 msgr2=0x7fc38c1f9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:09.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.564+0000 7fc391c00640 1 --2- 172.21.15.39:0/218902026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc38c0b3b50 0x7fc38c1f9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:09.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.564+0000 7fc391c00640 1 -- 172.21.15.39:0/218902026 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc38c157d90 con 0x7fc38c073f90 2024-09-21T07:46:09.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.564+0000 7fc392600640 1 --2- 172.21.15.39:0/218902026 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc38c0d0bc0 0x7fc38c1497c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:09.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.564+0000 7fc391c00640 1 --2- 172.21.15.39:0/218902026 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc38c073f90 0x7fc38c1f8bf0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fc380034040 tx=0x7fc380003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:09.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.565+0000 7fc389600640 1 -- 172.21.15.39:0/218902026 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc38003e030 con 0x7fc38c073f90 2024-09-21T07:46:09.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.565+0000 7fc389600640 1 -- 172.21.15.39:0/218902026 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc3800047a0 con 0x7fc38c073f90 2024-09-21T07:46:09.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.565+0000 7fc393000640 1 -- 172.21.15.39:0/218902026 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc38c158070 con 0x7fc38c073f90 2024-09-21T07:46:09.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.565+0000 7fc389600640 1 -- 172.21.15.39:0/218902026 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc380035040 con 0x7fc38c073f90 2024-09-21T07:46:09.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.565+0000 7fc393000640 1 -- 172.21.15.39:0/218902026 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc38c0c94c0 con 0x7fc38c073f90 2024-09-21T07:46:09.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.566+0000 7fc393000640 1 -- 172.21.15.39:0/218902026 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc38c15c510 con 0x7fc38c073f90 2024-09-21T07:46:09.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.567+0000 7fc389600640 1 -- 172.21.15.39:0/218902026 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc380041030 con 0x7fc38c073f90 2024-09-21T07:46:09.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.568+0000 7fc389600640 1 --2- 172.21.15.39:0/218902026 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc36c078b90 0x7fc36c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:09.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.568+0000 7fc389600640 1 -- 172.21.15.39:0/218902026 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 6916+0+0 (secure 0 0 0) 0x7fc3800bba80 con 0x7fc38c073f90 2024-09-21T07:46:09.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.568+0000 7fc391200640 1 --2- 172.21.15.39:0/218902026 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc36c078b90 0x7fc36c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:09.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.569+0000 7fc391200640 1 --2- 172.21.15.39:0/218902026 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc36c078b90 0x7fc36c07b0a0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc378006fd0 tx=0x7fc378009060 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:09.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.573+0000 7fc389600640 1 -- 172.21.15.39:0/218902026 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc3800bde20 con 0x7fc38c073f90 2024-09-21T07:46:09.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.813+0000 7fc393000640 1 -- 172.21.15.39:0/218902026 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc38c0b4780 con 0x7fc38c073f90 2024-09-21T07:46:09.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.814+0000 7fc389600640 1 -- 172.21.15.39:0/218902026 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 181 v181) ==== 95+0+5234 (secure 0 0 0) 0x7fc380083310 con 0x7fc38c073f90 2024-09-21T07:46:09.815 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:46:09.815 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":181,"btime":"2024-09-21T07:45:40:695099+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7310,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2823960253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2823960253},{"type":"v1","addr":"172.21.15.39:6835","nonce":2823960253}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":176},{"gid":8155,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/603157278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":603157278},{"type":"v1","addr":"172.21.15.148:6836","nonce":603157278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":175},{"gid":8158,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2610135999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2610135999},{"type":"v1","addr":"172.21.15.148:6837","nonce":2610135999}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":176}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:45:37.186043+0000","modified":"2024-09-21T07:45:40.206770+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8829},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8829":{"gid":8829,"name":"a","rank":0,"incarnation":177,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/2389975689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2389975689},{"type":"v1","addr":"172.21.15.39:6834","nonce":2389975689}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8829,"qdb_cluster":[8829]},"id":8}]} 2024-09-21T07:46:09.815 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 181 2024-09-21T07:46:09.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.816+0000 7fc393000640 1 -- 172.21.15.39:0/218902026 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc36c078b90 msgr2=0x7fc36c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:09.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.816+0000 7fc393000640 1 --2- 172.21.15.39:0/218902026 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc36c078b90 0x7fc36c07b0a0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc378006fd0 tx=0x7fc378009060 comp rx=0 tx=0).stop 2024-09-21T07:46:09.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.816+0000 7fc393000640 1 -- 172.21.15.39:0/218902026 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc38c073f90 msgr2=0x7fc38c1f8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:09.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.816+0000 7fc393000640 1 --2- 172.21.15.39:0/218902026 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc38c073f90 0x7fc38c1f8bf0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fc380034040 tx=0x7fc380003970 comp rx=0 tx=0).stop 2024-09-21T07:46:09.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.816+0000 7fc393000640 1 -- 172.21.15.39:0/218902026 shutdown_connections 2024-09-21T07:46:09.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.816+0000 7fc393000640 1 --2- 172.21.15.39:0/218902026 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc36c078b90 0x7fc36c07b0a0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:09.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.816+0000 7fc393000640 1 --2- 172.21.15.39:0/218902026 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc38c0d0bc0 0x7fc38c1497c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:09.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.816+0000 7fc393000640 1 --2- 172.21.15.39:0/218902026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc38c0b3b50 0x7fc38c1f9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:09.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.816+0000 7fc393000640 1 --2- 172.21.15.39:0/218902026 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc38c073f90 0x7fc38c1f8bf0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:09.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.816+0000 7fc393000640 1 -- 172.21.15.39:0/218902026 >> 172.21.15.39:0/218902026 conn(0x7fc38c06cfc0 msgr2=0x7fc38c15d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:09.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.816+0000 7fc393000640 1 -- 172.21.15.39:0/218902026 shutdown_connections 2024-09-21T07:46:09.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:09.816+0000 7fc393000640 1 -- 172.21.15.39:0/218902026 wait complete. 2024-09-21T07:46:09.841 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:46:10.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.068+0000 7f5c5fe00640 1 -- 172.21.15.39:0/4043512362 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c580d0630 msgr2=0x7f5c580d0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:10.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.068+0000 7f5c5fe00640 1 --2- 172.21.15.39:0/4043512362 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c580d0630 0x7f5c580d0a40 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f5c580cefc0 tx=0x7f5c5002a820 comp rx=0 tx=0).stop 2024-09-21T07:46:10.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.068+0000 7f5c5fe00640 1 -- 172.21.15.39:0/4043512362 shutdown_connections 2024-09-21T07:46:10.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.068+0000 7f5c5fe00640 1 --2- 172.21.15.39:0/4043512362 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c580d0630 0x7f5c580d0a40 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:10.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.068+0000 7f5c5fe00640 1 --2- 172.21.15.39:0/4043512362 >> v2:172.21.15.148:3300/0 conn(0x7f5c580cf450 0x7f5c580cf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:10.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.068+0000 7f5c5fe00640 1 -- 172.21.15.39:0/4043512362 >> 172.21.15.39:0/4043512362 conn(0x7f5c5806cfc0 msgr2=0x7f5c5806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:10.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.068+0000 7f5c5fe00640 1 -- 172.21.15.39:0/4043512362 shutdown_connections 2024-09-21T07:46:10.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.068+0000 7f5c5fe00640 1 -- 172.21.15.39:0/4043512362 wait complete. 2024-09-21T07:46:10.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.070+0000 7f5c5fe00640 1 Processor -- start 2024-09-21T07:46:10.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.071+0000 7f5c5fe00640 1 -- start start 2024-09-21T07:46:10.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.072+0000 7f5c5fe00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c58073dd0 0x7f5c58140f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:10.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.072+0000 7f5c5fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c580cf450 0x7f5c581f3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:10.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.072+0000 7f5c5fe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c580d0630 0x7f5c581e4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:10.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.072+0000 7f5c5fe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5c580cd170 con 0x7f5c580cf450 2024-09-21T07:46:10.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.072+0000 7f5c5fe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5c580ccff0 con 0x7f5c580d0630 2024-09-21T07:46:10.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.072+0000 7f5c5fe00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f5c580cd2f0 con 0x7f5c58073dd0 2024-09-21T07:46:10.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.072+0000 7f5c5ca00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c580cf450 0x7f5c581f3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:10.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.072+0000 7f5c5ca00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c580cf450 0x7f5c581f3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51116/0 (socket says 172.21.15.39:51116) 2024-09-21T07:46:10.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.072+0000 7f5c5ca00640 1 -- 172.21.15.39:0/4292658053 learned_addr learned my addr 172.21.15.39:0/4292658053 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:10.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.072+0000 7f5c5d400640 1 --2- 172.21.15.39:0/4292658053 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c58073dd0 0x7f5c58140f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:10.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.073+0000 7f5c5de00640 1 --2- 172.21.15.39:0/4292658053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c580d0630 0x7f5c581e4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:10.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.073+0000 7f5c5ca00640 1 -- 172.21.15.39:0/4292658053 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c58073dd0 msgr2=0x7f5c58140f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:10.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.073+0000 7f5c5ca00640 1 --2- 172.21.15.39:0/4292658053 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c58073dd0 0x7f5c58140f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:10.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.073+0000 7f5c5ca00640 1 -- 172.21.15.39:0/4292658053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c580d0630 msgr2=0x7f5c581e4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:10.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.073+0000 7f5c5ca00640 1 --2- 172.21.15.39:0/4292658053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c580d0630 0x7f5c581e4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:10.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.073+0000 7f5c5ca00640 1 -- 172.21.15.39:0/4292658053 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c581e5b40 con 0x7f5c580cf450 2024-09-21T07:46:10.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.073+0000 7f5c5ca00640 1 --2- 172.21.15.39:0/4292658053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c580cf450 0x7f5c581f3820 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f5c4400d6d0 tx=0x7f5c4400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:10.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.073+0000 7f5c55600640 1 -- 172.21.15.39:0/4292658053 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c44017070 con 0x7f5c580cf450 2024-09-21T07:46:10.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.073+0000 7f5c55600640 1 -- 172.21.15.39:0/4292658053 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c44009b10 con 0x7f5c580cf450 2024-09-21T07:46:10.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.073+0000 7f5c55600640 1 -- 172.21.15.39:0/4292658053 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c44010460 con 0x7f5c580cf450 2024-09-21T07:46:10.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.073+0000 7f5c5fe00640 1 -- 172.21.15.39:0/4292658053 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c581e5e20 con 0x7f5c580cf450 2024-09-21T07:46:10.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.074+0000 7f5c5fe00640 1 -- 172.21.15.39:0/4292658053 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c581e63e0 con 0x7f5c580cf450 2024-09-21T07:46:10.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.075+0000 7f5c5fe00640 1 -- 172.21.15.39:0/4292658053 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c581e58b0 con 0x7f5c580cf450 2024-09-21T07:46:10.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.076+0000 7f5c55600640 1 -- 172.21.15.39:0/4292658053 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5c4401a070 con 0x7f5c580cf450 2024-09-21T07:46:10.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.077+0000 7f5c55600640 1 --2- 172.21.15.39:0/4292658053 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5c38078bc0 0x7f5c3807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:10.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.077+0000 7f5c55600640 1 -- 172.21.15.39:0/4292658053 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 6916+0+0 (secure 0 0 0) 0x7f5c4409b280 con 0x7f5c580cf450 2024-09-21T07:46:10.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.077+0000 7f5c5d400640 1 --2- 172.21.15.39:0/4292658053 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5c38078bc0 0x7f5c3807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:10.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.078+0000 7f5c5d400640 1 --2- 172.21.15.39:0/4292658053 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5c38078bc0 0x7f5c3807b0d0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c0046b0 tx=0x7f5c4c007eb0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:10.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.082+0000 7f5c55600640 1 -- 172.21.15.39:0/4292658053 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5c44062b20 con 0x7f5c580cf450 2024-09-21T07:46:10.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.320+0000 7f5c5fe00640 1 -- 172.21.15.39:0/4292658053 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c580ce4b0 con 0x7f5c580cf450 2024-09-21T07:46:10.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.320+0000 7f5c55600640 1 -- 172.21.15.39:0/4292658053 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 181 v181) ==== 95+0+5234 (secure 0 0 0) 0x7f5c580ce4b0 con 0x7f5c580cf450 2024-09-21T07:46:10.321 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:46:10.322 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":181,"btime":"2024-09-21T07:45:40:695099+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7310,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2823960253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2823960253},{"type":"v1","addr":"172.21.15.39:6835","nonce":2823960253}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":176},{"gid":8155,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/603157278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":603157278},{"type":"v1","addr":"172.21.15.148:6836","nonce":603157278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":175},{"gid":8158,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2610135999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2610135999},{"type":"v1","addr":"172.21.15.148:6837","nonce":2610135999}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":176}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:45:37.186043+0000","modified":"2024-09-21T07:45:40.206770+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8829},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8829":{"gid":8829,"name":"a","rank":0,"incarnation":177,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/2389975689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2389975689},{"type":"v1","addr":"172.21.15.39:6834","nonce":2389975689}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8829,"qdb_cluster":[8829]},"id":8}]} 2024-09-21T07:46:10.322 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 181 2024-09-21T07:46:10.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.323+0000 7f5c5fe00640 1 -- 172.21.15.39:0/4292658053 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5c38078bc0 msgr2=0x7f5c3807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:10.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.323+0000 7f5c5fe00640 1 --2- 172.21.15.39:0/4292658053 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5c38078bc0 0x7f5c3807b0d0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c0046b0 tx=0x7f5c4c007eb0 comp rx=0 tx=0).stop 2024-09-21T07:46:10.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.323+0000 7f5c5fe00640 1 -- 172.21.15.39:0/4292658053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c580cf450 msgr2=0x7f5c581f3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:10.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.323+0000 7f5c5fe00640 1 --2- 172.21.15.39:0/4292658053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c580cf450 0x7f5c581f3820 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f5c4400d6d0 tx=0x7f5c4400db90 comp rx=0 tx=0).stop 2024-09-21T07:46:10.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.323+0000 7f5c5fe00640 1 -- 172.21.15.39:0/4292658053 shutdown_connections 2024-09-21T07:46:10.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.323+0000 7f5c5fe00640 1 --2- 172.21.15.39:0/4292658053 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5c38078bc0 0x7f5c3807b0d0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:10.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.323+0000 7f5c5fe00640 1 --2- 172.21.15.39:0/4292658053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c580d0630 0x7f5c581e4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:10.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.323+0000 7f5c5fe00640 1 --2- 172.21.15.39:0/4292658053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c580cf450 0x7f5c581f3820 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:10.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.323+0000 7f5c5fe00640 1 --2- 172.21.15.39:0/4292658053 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c58073dd0 0x7f5c58140f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:10.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.323+0000 7f5c5fe00640 1 -- 172.21.15.39:0/4292658053 >> 172.21.15.39:0/4292658053 conn(0x7f5c5806cfc0 msgr2=0x7f5c5806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:10.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.323+0000 7f5c5fe00640 1 -- 172.21.15.39:0/4292658053 shutdown_connections 2024-09-21T07:46:10.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.323+0000 7f5c5fe00640 1 -- 172.21.15.39:0/4292658053 wait complete. 2024-09-21T07:46:10.348 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:46:10.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.575+0000 7f1abbc00640 1 -- 172.21.15.39:0/2712152207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ab40b30a0 msgr2=0x7f1ab40cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:10.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.575+0000 7f1abbc00640 1 --2- 172.21.15.39:0/2712152207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ab40b30a0 0x7f1ab40cb8a0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8005fd0 tx=0x7f1aa802a4a0 comp rx=0 tx=0).stop 2024-09-21T07:46:10.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.575+0000 7f1abbc00640 1 -- 172.21.15.39:0/2712152207 shutdown_connections 2024-09-21T07:46:10.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.575+0000 7f1abbc00640 1 --2- 172.21.15.39:0/2712152207 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1ab40ccc60 0x7f1ab40cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:10.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.575+0000 7f1abbc00640 1 --2- 172.21.15.39:0/2712152207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ab40b30a0 0x7f1ab40cb8a0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:10.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.575+0000 7f1abbc00640 1 -- 172.21.15.39:0/2712152207 >> 172.21.15.39:0/2712152207 conn(0x7f1ab40b2460 msgr2=0x7f1ab406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:10.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.575+0000 7f1abbc00640 1 -- 172.21.15.39:0/2712152207 shutdown_connections 2024-09-21T07:46:10.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.575+0000 7f1abbc00640 1 -- 172.21.15.39:0/2712152207 wait complete. 2024-09-21T07:46:10.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.578+0000 7f1abbc00640 1 Processor -- start 2024-09-21T07:46:10.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.578+0000 7f1abbc00640 1 -- start start 2024-09-21T07:46:10.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.579+0000 7f1abbc00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ab40b30a0 0x7f1ab40c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:10.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.579+0000 7f1abbc00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1ab40ccc60 0x7f1ab40c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:10.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.579+0000 7f1abbc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ab415d830 0x7f1ab41f9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:10.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.579+0000 7f1abbc00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab40af410 con 0x7f1ab415d830 2024-09-21T07:46:10.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.579+0000 7f1abbc00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab40af290 con 0x7f1ab40b30a0 2024-09-21T07:46:10.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.579+0000 7f1abbc00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f1ab40af590 con 0x7f1ab40ccc60 2024-09-21T07:46:10.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.580+0000 7f1ab9c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ab415d830 0x7f1ab41f9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:10.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.580+0000 7f1ab9c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ab415d830 0x7f1ab41f9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51128/0 (socket says 172.21.15.39:51128) 2024-09-21T07:46:10.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.580+0000 7f1ab9c00640 1 -- 172.21.15.39:0/3652858535 learned_addr learned my addr 172.21.15.39:0/3652858535 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:10.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.580+0000 7f1ab9200640 1 --2- 172.21.15.39:0/3652858535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ab40b30a0 0x7f1ab40c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:10.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.580+0000 7f1ab3e00640 1 --2- 172.21.15.39:0/3652858535 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1ab40ccc60 0x7f1ab40c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:10.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.580+0000 7f1ab9c00640 1 -- 172.21.15.39:0/3652858535 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1ab40ccc60 msgr2=0x7f1ab40c65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:10.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.580+0000 7f1ab9c00640 1 --2- 172.21.15.39:0/3652858535 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1ab40ccc60 0x7f1ab40c65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:10.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.580+0000 7f1ab9c00640 1 -- 172.21.15.39:0/3652858535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ab40b30a0 msgr2=0x7f1ab40c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:10.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.580+0000 7f1ab9c00640 1 --2- 172.21.15.39:0/3652858535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ab40b30a0 0x7f1ab40c7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:10.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.580+0000 7f1ab9c00640 1 -- 172.21.15.39:0/3652858535 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab4074760 con 0x7f1ab415d830 2024-09-21T07:46:10.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.580+0000 7f1ab9200640 1 --2- 172.21.15.39:0/3652858535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ab40b30a0 0x7f1ab40c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:10.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.580+0000 7f1ab3e00640 1 --2- 172.21.15.39:0/3652858535 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1ab40ccc60 0x7f1ab40c65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:10.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.580+0000 7f1ab9c00640 1 --2- 172.21.15.39:0/3652858535 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ab415d830 0x7f1ab41f9080 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f1aac00ba10 tx=0x7f1aac00bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:10.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.581+0000 7f1ab0c00640 1 -- 172.21.15.39:0/3652858535 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac019070 con 0x7f1ab415d830 2024-09-21T07:46:10.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.581+0000 7f1ab0c00640 1 -- 172.21.15.39:0/3652858535 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1aac00cbd0 con 0x7f1ab415d830 2024-09-21T07:46:10.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.581+0000 7f1ab0c00640 1 -- 172.21.15.39:0/3652858535 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac014440 con 0x7f1ab415d830 2024-09-21T07:46:10.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.581+0000 7f1abbc00640 1 -- 172.21.15.39:0/3652858535 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab4074a40 con 0x7f1ab415d830 2024-09-21T07:46:10.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.581+0000 7f1abbc00640 1 -- 172.21.15.39:0/3652858535 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab40c9bd0 con 0x7f1ab415d830 2024-09-21T07:46:10.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.582+0000 7f1abbc00640 1 -- 172.21.15.39:0/3652858535 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ab40afa70 con 0x7f1ab415d830 2024-09-21T07:46:10.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.583+0000 7f1ab0c00640 1 -- 172.21.15.39:0/3652858535 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1aac01c070 con 0x7f1ab415d830 2024-09-21T07:46:10.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.584+0000 7f1ab0c00640 1 --2- 172.21.15.39:0/3652858535 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1a94078b90 0x7f1a9407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:10.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.584+0000 7f1ab0c00640 1 -- 172.21.15.39:0/3652858535 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 6916+0+0 (secure 0 0 0) 0x7f1aac09aec0 con 0x7f1ab415d830 2024-09-21T07:46:10.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.584+0000 7f1ab9200640 1 --2- 172.21.15.39:0/3652858535 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1a94078b90 0x7f1a9407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:10.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.585+0000 7f1ab9200640 1 --2- 172.21.15.39:0/3652858535 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1a94078b90 0x7f1a9407b0a0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8033000 tx=0x7f1aa8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:10.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.588+0000 7f1ab0c00640 1 -- 172.21.15.39:0/3652858535 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1aac062e30 con 0x7f1ab415d830 2024-09-21T07:46:10.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.784+0000 7f1abbc00640 1 -- 172.21.15.39:0/3652858535 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ab40ad700 con 0x7f1ab415d830 2024-09-21T07:46:10.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.785+0000 7f1ab0c00640 1 -- 172.21.15.39:0/3652858535 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v163) ==== 74+0+22363 (secure 0 0 0) 0x7f1aac062730 con 0x7f1ab415d830 2024-09-21T07:46:10.787 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:46:10.787 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":163,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:46:07.903428+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:45:35.115582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-09-21T07:45:36.142685+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":203,"snap_epoch":163,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]},{"pool":20,"snaps":[{"begin":2,"length":202}]}],"new_removed_snaps":[{"pool":20,"snaps":[{"begin":93,"length":10},{"begin":194,"length":10}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:46:10.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.788+0000 7f1abbc00640 1 -- 172.21.15.39:0/3652858535 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1a94078b90 msgr2=0x7f1a9407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:10.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.788+0000 7f1abbc00640 1 --2- 172.21.15.39:0/3652858535 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1a94078b90 0x7f1a9407b0a0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8033000 tx=0x7f1aa8004730 comp rx=0 tx=0).stop 2024-09-21T07:46:10.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.788+0000 7f1abbc00640 1 -- 172.21.15.39:0/3652858535 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ab415d830 msgr2=0x7f1ab41f9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:10.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.788+0000 7f1abbc00640 1 --2- 172.21.15.39:0/3652858535 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ab415d830 0x7f1ab41f9080 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f1aac00ba10 tx=0x7f1aac00bed0 comp rx=0 tx=0).stop 2024-09-21T07:46:10.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.788+0000 7f1abbc00640 1 -- 172.21.15.39:0/3652858535 shutdown_connections 2024-09-21T07:46:10.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.788+0000 7f1abbc00640 1 --2- 172.21.15.39:0/3652858535 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1a94078b90 0x7f1a9407b0a0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:10.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.788+0000 7f1abbc00640 1 --2- 172.21.15.39:0/3652858535 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ab415d830 0x7f1ab41f9080 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:10.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.788+0000 7f1abbc00640 1 --2- 172.21.15.39:0/3652858535 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1ab40ccc60 0x7f1ab40c65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:10.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.788+0000 7f1abbc00640 1 --2- 172.21.15.39:0/3652858535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ab40b30a0 0x7f1ab40c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:10.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.788+0000 7f1abbc00640 1 -- 172.21.15.39:0/3652858535 >> 172.21.15.39:0/3652858535 conn(0x7f1ab40b2460 msgr2=0x7f1ab40adec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:10.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.788+0000 7f1abbc00640 1 -- 172.21.15.39:0/3652858535 shutdown_connections 2024-09-21T07:46:10.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:10.788+0000 7f1abbc00640 1 -- 172.21.15.39:0/3652858535 wait complete. 2024-09-21T07:46:10.810 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T07:46:11.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.036+0000 7ff1f5e00640 1 -- 172.21.15.39:0/1360116099 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1f015d830 msgr2=0x7ff1f0156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:11.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.036+0000 7ff1f5e00640 1 --2- 172.21.15.39:0/1360116099 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1f015d830 0x7ff1f0156bb0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7ff1f006d880 tx=0x7ff1d802a4d0 comp rx=0 tx=0).stop 2024-09-21T07:46:11.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.036+0000 7ff1f5e00640 1 -- 172.21.15.39:0/1360116099 shutdown_connections 2024-09-21T07:46:11.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.036+0000 7ff1f5e00640 1 --2- 172.21.15.39:0/1360116099 >> v2:172.21.15.148:3300/0 conn(0x7ff1f00ccc60 0x7ff1f00cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:11.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.036+0000 7ff1f5e00640 1 --2- 172.21.15.39:0/1360116099 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1f015d830 0x7ff1f0156bb0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:11.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.036+0000 7ff1f5e00640 1 --2- 172.21.15.39:0/1360116099 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff1f00b30a0 0x7ff1f00cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:11.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.036+0000 7ff1f5e00640 1 -- 172.21.15.39:0/1360116099 >> 172.21.15.39:0/1360116099 conn(0x7ff1f00b2460 msgr2=0x7ff1f006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:11.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.036+0000 7ff1f5e00640 1 -- 172.21.15.39:0/1360116099 shutdown_connections 2024-09-21T07:46:11.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.036+0000 7ff1f5e00640 1 -- 172.21.15.39:0/1360116099 wait complete. 2024-09-21T07:46:11.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.038+0000 7ff1f5e00640 1 Processor -- start 2024-09-21T07:46:11.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.038+0000 7ff1f5e00640 1 -- start start 2024-09-21T07:46:11.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.040+0000 7ff1f5e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1f00b30a0 0x7ff1f0074850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:11.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.040+0000 7ff1f5e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1f00ccc60 0x7ff1f01f9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:11.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.040+0000 7ff1f5e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff1f015d830 0x7ff1f01f9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:11.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.040+0000 7ff1f5e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f00af200 con 0x7ff1f00b30a0 2024-09-21T07:46:11.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.040+0000 7ff1f5e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f00af080 con 0x7ff1f00ccc60 2024-09-21T07:46:11.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.040+0000 7ff1f5e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7ff1f00af380 con 0x7ff1f015d830 2024-09-21T07:46:11.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.040+0000 7ff1eea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1f00b30a0 0x7ff1f0074850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:11.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.040+0000 7ff1eea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1f00b30a0 0x7ff1f0074850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51132/0 (socket says 172.21.15.39:51132) 2024-09-21T07:46:11.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.040+0000 7ff1eea00640 1 -- 172.21.15.39:0/1447220973 learned_addr learned my addr 172.21.15.39:0/1447220973 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:11.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.040+0000 7ff1e7e00640 1 --2- 172.21.15.39:0/1447220973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1f00ccc60 0x7ff1f01f9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:11.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.040+0000 7ff1ef400640 1 --2- 172.21.15.39:0/1447220973 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff1f015d830 0x7ff1f01f9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:11.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.040+0000 7ff1eea00640 1 -- 172.21.15.39:0/1447220973 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff1f015d830 msgr2=0x7ff1f01f9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:11.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.040+0000 7ff1eea00640 1 --2- 172.21.15.39:0/1447220973 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff1f015d830 0x7ff1f01f9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:11.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.041+0000 7ff1eea00640 1 -- 172.21.15.39:0/1447220973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1f00ccc60 msgr2=0x7ff1f01f9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:11.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.041+0000 7ff1eea00640 1 --2- 172.21.15.39:0/1447220973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1f00ccc60 0x7ff1f01f9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:11.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.041+0000 7ff1eea00640 1 -- 172.21.15.39:0/1447220973 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f00c9ae0 con 0x7ff1f00b30a0 2024-09-21T07:46:11.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.041+0000 7ff1eea00640 1 --2- 172.21.15.39:0/1447220973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1f00b30a0 0x7ff1f0074850 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7ff1e000d720 tx=0x7ff1e000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:11.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.041+0000 7ff1e7400640 1 -- 172.21.15.39:0/1447220973 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0017070 con 0x7ff1f00b30a0 2024-09-21T07:46:11.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.041+0000 7ff1e7400640 1 -- 172.21.15.39:0/1447220973 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1e0009ab0 con 0x7ff1f00b30a0 2024-09-21T07:46:11.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.041+0000 7ff1f5e00640 1 -- 172.21.15.39:0/1447220973 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f00c9dc0 con 0x7ff1f00b30a0 2024-09-21T07:46:11.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.041+0000 7ff1e7400640 1 -- 172.21.15.39:0/1447220973 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0010460 con 0x7ff1f00b30a0 2024-09-21T07:46:11.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.041+0000 7ff1f5e00640 1 -- 172.21.15.39:0/1447220973 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f00ca380 con 0x7ff1f00b30a0 2024-09-21T07:46:11.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.042+0000 7ff1f5e00640 1 -- 172.21.15.39:0/1447220973 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1f00af840 con 0x7ff1f00b30a0 2024-09-21T07:46:11.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.043+0000 7ff1e7400640 1 -- 172.21.15.39:0/1447220973 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff1e001a070 con 0x7ff1f00b30a0 2024-09-21T07:46:11.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.044+0000 7ff1e7400640 1 --2- 172.21.15.39:0/1447220973 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff1cc078b90 0x7ff1cc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:11.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.045+0000 7ff1e7400640 1 -- 172.21.15.39:0/1447220973 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 6916+0+0 (secure 0 0 0) 0x7ff1e009b3c0 con 0x7ff1f00b30a0 2024-09-21T07:46:11.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.045+0000 7ff1e7e00640 1 --2- 172.21.15.39:0/1447220973 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff1cc078b90 0x7ff1cc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:11.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.046+0000 7ff1e7e00640 1 --2- 172.21.15.39:0/1447220973 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff1cc078b90 0x7ff1cc07b0a0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8033000 tx=0x7ff1d8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:11.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.049+0000 7ff1e7400640 1 -- 172.21.15.39:0/1447220973 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff1e00632a0 con 0x7ff1f00b30a0 2024-09-21T07:46:11.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.281+0000 7ff1f5e00640 1 -- 172.21.15.39:0/1447220973 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff1f0156ed0 con 0x7ff1f00b30a0 2024-09-21T07:46:11.284 INFO:tasks.ceph.mon.a.smithi039.stderr:2024-09-21T07:46:11.282+0000 7fabab600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T07:46:11.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.285+0000 7ff1e7400640 1 -- 172.21.15.39:0/1447220973 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v182) ==== 161+0+0 (secure 0 0 0) 0x7ff1e0062ba0 con 0x7ff1f00b30a0 2024-09-21T07:46:11.287 INFO:teuthology.orchestra.run.smithi039.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T07:46:11.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.288+0000 7ff1f5e00640 1 -- 172.21.15.39:0/1447220973 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff1cc078b90 msgr2=0x7ff1cc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:11.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.288+0000 7ff1f5e00640 1 --2- 172.21.15.39:0/1447220973 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff1cc078b90 0x7ff1cc07b0a0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8033000 tx=0x7ff1d8004730 comp rx=0 tx=0).stop 2024-09-21T07:46:11.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.288+0000 7ff1f5e00640 1 -- 172.21.15.39:0/1447220973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1f00b30a0 msgr2=0x7ff1f0074850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:11.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.288+0000 7ff1f5e00640 1 --2- 172.21.15.39:0/1447220973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1f00b30a0 0x7ff1f0074850 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7ff1e000d720 tx=0x7ff1e000dbe0 comp rx=0 tx=0).stop 2024-09-21T07:46:11.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.289+0000 7ff1f5e00640 1 -- 172.21.15.39:0/1447220973 shutdown_connections 2024-09-21T07:46:11.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.289+0000 7ff1f5e00640 1 --2- 172.21.15.39:0/1447220973 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff1cc078b90 0x7ff1cc07b0a0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:11.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.289+0000 7ff1f5e00640 1 --2- 172.21.15.39:0/1447220973 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff1f015d830 0x7ff1f01f9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:11.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.289+0000 7ff1f5e00640 1 --2- 172.21.15.39:0/1447220973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1f00ccc60 0x7ff1f01f9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:11.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.289+0000 7ff1f5e00640 1 --2- 172.21.15.39:0/1447220973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1f00b30a0 0x7ff1f0074850 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:11.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.289+0000 7ff1f5e00640 1 -- 172.21.15.39:0/1447220973 >> 172.21.15.39:0/1447220973 conn(0x7ff1f00b2460 msgr2=0x7ff1f006f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:11.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.290+0000 7ff1f5e00640 1 -- 172.21.15.39:0/1447220973 shutdown_connections 2024-09-21T07:46:11.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.290+0000 7ff1f5e00640 1 -- 172.21.15.39:0/1447220973 wait complete. 2024-09-21T07:46:11.317 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T07:46:11.431 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-21T07:46:11.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.551+0000 7f17f9200640 1 -- 172.21.15.39:0/1996071759 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17f415d830 msgr2=0x7f17f4156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:11.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.551+0000 7f17f9200640 1 --2- 172.21.15.39:0/1996071759 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17f415d830 0x7f17f4156bb0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f17f406d880 tx=0x7f17e002a4d0 comp rx=0 tx=0).stop 2024-09-21T07:46:11.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.552+0000 7f17f9200640 1 -- 172.21.15.39:0/1996071759 shutdown_connections 2024-09-21T07:46:11.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.552+0000 7f17f9200640 1 --2- 172.21.15.39:0/1996071759 >> v2:172.21.15.148:3300/0 conn(0x7f17f40ccc60 0x7f17f40cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:11.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.552+0000 7f17f9200640 1 --2- 172.21.15.39:0/1996071759 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17f415d830 0x7f17f4156bb0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:11.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.552+0000 7f17f9200640 1 --2- 172.21.15.39:0/1996071759 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f17f40b30a0 0x7f17f40cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:11.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.552+0000 7f17f9200640 1 -- 172.21.15.39:0/1996071759 >> 172.21.15.39:0/1996071759 conn(0x7f17f40b2460 msgr2=0x7f17f406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:11.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.552+0000 7f17f9200640 1 -- 172.21.15.39:0/1996071759 shutdown_connections 2024-09-21T07:46:11.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.552+0000 7f17f9200640 1 -- 172.21.15.39:0/1996071759 wait complete. 2024-09-21T07:46:11.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.555+0000 7f17f9200640 1 Processor -- start 2024-09-21T07:46:11.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.555+0000 7f17f9200640 1 -- start start 2024-09-21T07:46:11.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.556+0000 7f17f9200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17f40b30a0 0x7f17f41f9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:11.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.556+0000 7f17f9200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17f40ccc60 0x7f17f409d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:11.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.556+0000 7f17f9200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f17f415d830 0x7f17f41e93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:11.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.556+0000 7f17f9200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f17f40af200 con 0x7f17f40b30a0 2024-09-21T07:46:11.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.556+0000 7f17f9200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f17f40af080 con 0x7f17f40ccc60 2024-09-21T07:46:11.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.556+0000 7f17f9200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f17f40af380 con 0x7f17f415d830 2024-09-21T07:46:11.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.557+0000 7f17f1e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17f40b30a0 0x7f17f41f9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:11.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.557+0000 7f17f1e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17f40b30a0 0x7f17f41f9940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51158/0 (socket says 172.21.15.39:51158) 2024-09-21T07:46:11.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.557+0000 7f17f1e00640 1 -- 172.21.15.39:0/2779626520 learned_addr learned my addr 172.21.15.39:0/2779626520 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:11.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.557+0000 7f17f2800640 1 --2- 172.21.15.39:0/2779626520 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f17f415d830 0x7f17f41e93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:11.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.557+0000 7f17f1400640 1 --2- 172.21.15.39:0/2779626520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17f40ccc60 0x7f17f409d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:11.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.557+0000 7f17f1e00640 1 -- 172.21.15.39:0/2779626520 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f17f415d830 msgr2=0x7f17f41e93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:11.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.557+0000 7f17f1e00640 1 --2- 172.21.15.39:0/2779626520 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f17f415d830 0x7f17f41e93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:11.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.557+0000 7f17f1e00640 1 -- 172.21.15.39:0/2779626520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17f40ccc60 msgr2=0x7f17f409d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:11.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.557+0000 7f17f1e00640 1 --2- 172.21.15.39:0/2779626520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17f40ccc60 0x7f17f409d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:11.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.557+0000 7f17f1e00640 1 -- 172.21.15.39:0/2779626520 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f40c4250 con 0x7f17f40b30a0 2024-09-21T07:46:11.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.557+0000 7f17f2800640 1 --2- 172.21.15.39:0/2779626520 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f17f415d830 0x7f17f41e93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:11.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.557+0000 7f17f1400640 1 --2- 172.21.15.39:0/2779626520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17f40ccc60 0x7f17f409d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:11.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.557+0000 7f17f1e00640 1 --2- 172.21.15.39:0/2779626520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17f40b30a0 0x7f17f41f9940 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f17e400d690 tx=0x7f17e400db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:11.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.558+0000 7f17ea000640 1 -- 172.21.15.39:0/2779626520 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e4017070 con 0x7f17f40b30a0 2024-09-21T07:46:11.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.558+0000 7f17ea000640 1 -- 172.21.15.39:0/2779626520 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17e4005310 con 0x7f17f40b30a0 2024-09-21T07:46:11.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.558+0000 7f17ea000640 1 -- 172.21.15.39:0/2779626520 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e4009820 con 0x7f17f40b30a0 2024-09-21T07:46:11.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.558+0000 7f17f9200640 1 -- 172.21.15.39:0/2779626520 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17f40c4530 con 0x7f17f40b30a0 2024-09-21T07:46:11.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.558+0000 7f17f9200640 1 -- 172.21.15.39:0/2779626520 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17f409daf0 con 0x7f17f40b30a0 2024-09-21T07:46:11.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.559+0000 7f17f9200640 1 -- 172.21.15.39:0/2779626520 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17f40af870 con 0x7f17f40b30a0 2024-09-21T07:46:11.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.560+0000 7f17ea000640 1 -- 172.21.15.39:0/2779626520 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f17e40099c0 con 0x7f17f40b30a0 2024-09-21T07:46:11.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.561+0000 7f17ea000640 1 --2- 172.21.15.39:0/2779626520 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f17d4078bc0 0x7f17d407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:11.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.561+0000 7f17ea000640 1 -- 172.21.15.39:0/2779626520 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 7018+0+0 (secure 0 0 0) 0x7f17e409b340 con 0x7f17f40b30a0 2024-09-21T07:46:11.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.562+0000 7f17f1400640 1 --2- 172.21.15.39:0/2779626520 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f17d4078bc0 0x7f17d407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:11.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.562+0000 7f17f1400640 1 --2- 172.21.15.39:0/2779626520 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f17d4078bc0 0x7f17d407b0d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f17e0033000 tx=0x7f17e0004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:11.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.566+0000 7f17ea000640 1 -- 172.21.15.39:0/2779626520 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f17e40631d0 con 0x7f17f40b30a0 2024-09-21T07:46:11.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:11.807+0000 7f17f9200640 1 -- 172.21.15.39:0/2779626520 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f17f40cb860 con 0x7f17f40b30a0 2024-09-21T07:46:12.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.287+0000 7f17ea000640 1 -- 172.21.15.39:0/2779626520 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v183) ==== 104+0+0 (secure 0 0 0) 0x7f17e4062ad0 con 0x7f17f40b30a0 2024-09-21T07:46:12.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.290+0000 7f17f9200640 1 -- 172.21.15.39:0/2779626520 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f17d4078bc0 msgr2=0x7f17d407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:12.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.290+0000 7f17f9200640 1 --2- 172.21.15.39:0/2779626520 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f17d4078bc0 0x7f17d407b0d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f17e0033000 tx=0x7f17e0004730 comp rx=0 tx=0).stop 2024-09-21T07:46:12.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.290+0000 7f17f9200640 1 -- 172.21.15.39:0/2779626520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17f40b30a0 msgr2=0x7f17f41f9940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:12.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.290+0000 7f17f9200640 1 --2- 172.21.15.39:0/2779626520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17f40b30a0 0x7f17f41f9940 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f17e400d690 tx=0x7f17e400db50 comp rx=0 tx=0).stop 2024-09-21T07:46:12.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.290+0000 7f17f9200640 1 -- 172.21.15.39:0/2779626520 shutdown_connections 2024-09-21T07:46:12.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.290+0000 7f17f9200640 1 --2- 172.21.15.39:0/2779626520 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f17d4078bc0 0x7f17d407b0d0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:12.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.290+0000 7f17f9200640 1 --2- 172.21.15.39:0/2779626520 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f17f415d830 0x7f17f41e93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:12.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.290+0000 7f17f9200640 1 --2- 172.21.15.39:0/2779626520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17f40ccc60 0x7f17f409d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:12.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.290+0000 7f17f9200640 1 --2- 172.21.15.39:0/2779626520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17f40b30a0 0x7f17f41f9940 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:12.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.290+0000 7f17f9200640 1 -- 172.21.15.39:0/2779626520 >> 172.21.15.39:0/2779626520 conn(0x7f17f40b2460 msgr2=0x7f17f406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:12.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.290+0000 7f17f9200640 1 -- 172.21.15.39:0/2779626520 shutdown_connections 2024-09-21T07:46:12.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.290+0000 7f17f9200640 1 -- 172.21.15.39:0/2779626520 wait complete. 2024-09-21T07:46:12.317 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T07:46:12.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.548+0000 7f1475e00640 1 -- 172.21.15.39:0/3269464364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1458004000 msgr2=0x7f1458004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:12.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.548+0000 7f1475e00640 1 --2- 172.21.15.39:0/3269464364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1458004000 0x7f1458004800 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f14701602a0 tx=0x7f1458005830 comp rx=0 tx=0).stop 2024-09-21T07:46:12.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.548+0000 7f1475400640 1 -- 172.21.15.39:0/3269464364 reap_dead start 2024-09-21T07:46:12.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.548+0000 7f1475e00640 1 -- 172.21.15.39:0/3269464364 shutdown_connections 2024-09-21T07:46:12.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.548+0000 7f1475e00640 1 -- 172.21.15.39:0/3269464364 >> 172.21.15.39:0/3269464364 conn(0x7f147006d660 msgr2=0x7f14700b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:12.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.548+0000 7f1475e00640 1 -- 172.21.15.39:0/3269464364 shutdown_connections 2024-09-21T07:46:12.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.548+0000 7f1475e00640 1 -- 172.21.15.39:0/3269464364 wait complete. 2024-09-21T07:46:12.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.550+0000 7f1475e00640 1 Processor -- start 2024-09-21T07:46:12.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.551+0000 7f1475e00640 1 -- start start 2024-09-21T07:46:12.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.552+0000 7f1475e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1458004000 0x7f147014d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:12.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.552+0000 7f1475e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f14701546b0 0x7f1470156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:12.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.552+0000 7f1475e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1470154ac0 0x7f1470156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:12.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.552+0000 7f1475e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1470160340 con 0x7f1458004000 2024-09-21T07:46:12.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.552+0000 7f1475e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f14700adc50 con 0x7f1470154ac0 2024-09-21T07:46:12.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.552+0000 7f1475e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f14701604c0 con 0x7f14701546b0 2024-09-21T07:46:12.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.552+0000 7f1474a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1458004000 0x7f147014d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:12.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.552+0000 7f1474a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1458004000 0x7f147014d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52998/0 (socket says 172.21.15.39:52998) 2024-09-21T07:46:12.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.552+0000 7f1474a00640 1 -- 172.21.15.39:0/3504193940 learned_addr learned my addr 172.21.15.39:0/3504193940 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:12.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.552+0000 7f146fe00640 1 --2- 172.21.15.39:0/3504193940 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f14701546b0 0x7f1470156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:12.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.553+0000 7f1475400640 1 --2- 172.21.15.39:0/3504193940 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1470154ac0 0x7f1470156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:12.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.553+0000 7f1474a00640 1 -- 172.21.15.39:0/3504193940 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f14701546b0 msgr2=0x7f1470156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:12.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.553+0000 7f1474a00640 1 --2- 172.21.15.39:0/3504193940 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f14701546b0 0x7f1470156230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:12.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.553+0000 7f1474a00640 1 -- 172.21.15.39:0/3504193940 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1470154ac0 msgr2=0x7f1470156770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:12.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.553+0000 7f1474a00640 1 --2- 172.21.15.39:0/3504193940 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1470154ac0 0x7f1470156770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:12.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.553+0000 7f1474a00640 1 -- 172.21.15.39:0/3504193940 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14701f5650 con 0x7f1458004000 2024-09-21T07:46:12.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.553+0000 7f146fe00640 1 --2- 172.21.15.39:0/3504193940 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f14701546b0 0x7f1470156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:46:12.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.553+0000 7f1475400640 1 --2- 172.21.15.39:0/3504193940 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1470154ac0 0x7f1470156770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:46:12.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.553+0000 7f1474a00640 1 --2- 172.21.15.39:0/3504193940 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1458004000 0x7f147014d4c0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f1464001480 tx=0x7f14640039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:12.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.553+0000 7f146cc00640 1 -- 172.21.15.39:0/3504193940 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1464015070 con 0x7f1458004000 2024-09-21T07:46:12.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.553+0000 7f146cc00640 1 -- 172.21.15.39:0/3504193940 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1464011920 con 0x7f1458004000 2024-09-21T07:46:12.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.553+0000 7f146cc00640 1 -- 172.21.15.39:0/3504193940 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1464011d70 con 0x7f1458004000 2024-09-21T07:46:12.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.553+0000 7f1475e00640 1 -- 172.21.15.39:0/3504193940 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14701f58f0 con 0x7f1458004000 2024-09-21T07:46:12.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.553+0000 7f1475e00640 1 -- 172.21.15.39:0/3504193940 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14701f5eb0 con 0x7f1458004000 2024-09-21T07:46:12.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.554+0000 7f1475e00640 1 -- 172.21.15.39:0/3504193940 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14700d0e20 con 0x7f1458004000 2024-09-21T07:46:12.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.556+0000 7f146cc00640 1 -- 172.21.15.39:0/3504193940 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1464018070 con 0x7f1458004000 2024-09-21T07:46:12.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.557+0000 7f146cc00640 1 --2- 172.21.15.39:0/3504193940 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1450078bc0 0x7f145007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:12.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.557+0000 7f146cc00640 1 -- 172.21.15.39:0/3504193940 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 7018+0+0 (secure 0 0 0) 0x7f146409bfc0 con 0x7f1458004000 2024-09-21T07:46:12.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.557+0000 7f146fe00640 1 --2- 172.21.15.39:0/3504193940 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1450078bc0 0x7f145007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:12.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.558+0000 7f146fe00640 1 --2- 172.21.15.39:0/3504193940 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1450078bc0 0x7f145007b0d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f145c035000 tx=0x7f145c0046c0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:12.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.561+0000 7f146cc00640 1 -- 172.21.15.39:0/3504193940 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f14701f5650 con 0x7f1458004000 2024-09-21T07:46:12.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:12.759+0000 7f1475e00640 1 -- 172.21.15.39:0/3504193940 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f147006df10 con 0x7f1458004000 2024-09-21T07:46:13.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.501+0000 7f146cc00640 1 -- 172.21.15.39:0/3504193940 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v166) ==== 181+0+0 (secure 0 0 0) 0x7f1464063ee0 con 0x7f1458004000 2024-09-21T07:46:13.503 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' removed 2024-09-21T07:46:13.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.503+0000 7f1475e00640 1 -- 172.21.15.39:0/3504193940 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1450078bc0 msgr2=0x7f145007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:13.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.503+0000 7f1475e00640 1 --2- 172.21.15.39:0/3504193940 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1450078bc0 0x7f145007b0d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f145c035000 tx=0x7f145c0046c0 comp rx=0 tx=0).stop 2024-09-21T07:46:13.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.503+0000 7f1475e00640 1 -- 172.21.15.39:0/3504193940 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1458004000 msgr2=0x7f147014d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:13.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.503+0000 7f1475e00640 1 --2- 172.21.15.39:0/3504193940 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1458004000 0x7f147014d4c0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f1464001480 tx=0x7f14640039c0 comp rx=0 tx=0).stop 2024-09-21T07:46:13.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.503+0000 7f1475e00640 1 -- 172.21.15.39:0/3504193940 shutdown_connections 2024-09-21T07:46:13.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.503+0000 7f1475e00640 1 --2- 172.21.15.39:0/3504193940 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1450078bc0 0x7f145007b0d0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:13.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.503+0000 7f1475e00640 1 --2- 172.21.15.39:0/3504193940 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1470154ac0 0x7f1470156770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:13.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.503+0000 7f1475e00640 1 --2- 172.21.15.39:0/3504193940 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f14701546b0 0x7f1470156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:13.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.503+0000 7f1475e00640 1 --2- 172.21.15.39:0/3504193940 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1458004000 0x7f147014d4c0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:13.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.503+0000 7f1475e00640 1 -- 172.21.15.39:0/3504193940 >> 172.21.15.39:0/3504193940 conn(0x7f147006d660 msgr2=0x7f14700ad2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:13.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.504+0000 7f1475e00640 1 -- 172.21.15.39:0/3504193940 shutdown_connections 2024-09-21T07:46:13.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.504+0000 7f1475e00640 1 -- 172.21.15.39:0/3504193940 wait complete. 2024-09-21T07:46:13.534 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T07:46:13.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.766+0000 7f5abd200640 1 -- 172.21.15.39:0/853612750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ab80cd230 msgr2=0x7f5ab80b3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:13.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.766+0000 7f5abd200640 1 --2- 172.21.15.39:0/853612750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ab80cd230 0x7f5ab80b3830 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0005fd0 tx=0x7f5ab002a820 comp rx=0 tx=0).stop 2024-09-21T07:46:13.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.766+0000 7f5abd200640 1 -- 172.21.15.39:0/853612750 shutdown_connections 2024-09-21T07:46:13.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.766+0000 7f5abd200640 1 --2- 172.21.15.39:0/853612750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ab80cd230 0x7f5ab80b3830 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:13.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.766+0000 7f5abd200640 1 --2- 172.21.15.39:0/853612750 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5ab815a440 0x7f5ab80cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:13.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.766+0000 7f5abd200640 1 -- 172.21.15.39:0/853612750 >> 172.21.15.39:0/853612750 conn(0x7f5ab80b2650 msgr2=0x7f5ab80d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:13.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.766+0000 7f5abd200640 1 -- 172.21.15.39:0/853612750 shutdown_connections 2024-09-21T07:46:13.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.766+0000 7f5abd200640 1 -- 172.21.15.39:0/853612750 wait complete. 2024-09-21T07:46:13.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.768+0000 7f5abd200640 1 Processor -- start 2024-09-21T07:46:13.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.768+0000 7f5abd200640 1 -- start start 2024-09-21T07:46:13.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.770+0000 7f5abd200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ab80cd230 0x7f5ab81f3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:13.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.770+0000 7f5abd200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5ab815a440 0x7f5ab80c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:13.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.770+0000 7f5abd200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ab815da30 0x7f5ab815f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:13.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.770+0000 7f5abd200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab80b42f0 con 0x7f5ab815da30 2024-09-21T07:46:13.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.770+0000 7f5abd200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab80b4170 con 0x7f5ab80cd230 2024-09-21T07:46:13.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.770+0000 7f5abd200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f5ab80b4470 con 0x7f5ab815a440 2024-09-21T07:46:13.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.770+0000 7f5ab6800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ab815da30 0x7f5ab815f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:13.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.770+0000 7f5ab6800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ab815da30 0x7f5ab815f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53020/0 (socket says 172.21.15.39:53020) 2024-09-21T07:46:13.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.770+0000 7f5ab6800640 1 -- 172.21.15.39:0/3135125170 learned_addr learned my addr 172.21.15.39:0/3135125170 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:13.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.770+0000 7f5aafe00640 1 --2- 172.21.15.39:0/3135125170 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5ab815a440 0x7f5ab80c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:13.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.770+0000 7f5ab5e00640 1 --2- 172.21.15.39:0/3135125170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ab80cd230 0x7f5ab81f3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:13.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.770+0000 7f5ab6800640 1 -- 172.21.15.39:0/3135125170 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5ab815a440 msgr2=0x7f5ab80c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:13.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.770+0000 7f5ab6800640 1 --2- 172.21.15.39:0/3135125170 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5ab815a440 0x7f5ab80c89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:13.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.770+0000 7f5ab6800640 1 -- 172.21.15.39:0/3135125170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ab80cd230 msgr2=0x7f5ab81f3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:13.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.770+0000 7f5ab6800640 1 --2- 172.21.15.39:0/3135125170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ab80cd230 0x7f5ab81f3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:13.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.770+0000 7f5ab6800640 1 -- 172.21.15.39:0/3135125170 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab80c7930 con 0x7f5ab815da30 2024-09-21T07:46:13.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.770+0000 7f5aafe00640 1 --2- 172.21.15.39:0/3135125170 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5ab815a440 0x7f5ab80c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:46:13.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.770+0000 7f5ab5e00640 1 --2- 172.21.15.39:0/3135125170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ab80cd230 0x7f5ab81f3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:46:13.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.771+0000 7f5ab6800640 1 --2- 172.21.15.39:0/3135125170 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ab815da30 0x7f5ab815f1f0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0006010 tx=0x7f5ab0032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:13.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.771+0000 7f5aae000640 1 -- 172.21.15.39:0/3135125170 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0037040 con 0x7f5ab815da30 2024-09-21T07:46:13.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.771+0000 7f5aae000640 1 -- 172.21.15.39:0/3135125170 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ab0042070 con 0x7f5ab815da30 2024-09-21T07:46:13.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.771+0000 7f5aae000640 1 -- 172.21.15.39:0/3135125170 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab00042a0 con 0x7f5ab815da30 2024-09-21T07:46:13.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.771+0000 7f5abd200640 1 -- 172.21.15.39:0/3135125170 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ab80c7c10 con 0x7f5ab815da30 2024-09-21T07:46:13.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.771+0000 7f5abd200640 1 -- 172.21.15.39:0/3135125170 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ab80c81d0 con 0x7f5ab815da30 2024-09-21T07:46:13.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.772+0000 7f5abd200640 1 -- 172.21.15.39:0/3135125170 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ab80b4a00 con 0x7f5ab815da30 2024-09-21T07:46:13.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.773+0000 7f5aae000640 1 -- 172.21.15.39:0/3135125170 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5ab0004440 con 0x7f5ab815da30 2024-09-21T07:46:13.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.774+0000 7f5aae000640 1 --2- 172.21.15.39:0/3135125170 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5a94078b90 0x7f5a9407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:13.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.774+0000 7f5aae000640 1 -- 172.21.15.39:0/3135125170 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 6536+0+0 (secure 0 0 0) 0x7f5ab00bbe80 con 0x7f5ab815da30 2024-09-21T07:46:13.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.775+0000 7f5ab5e00640 1 --2- 172.21.15.39:0/3135125170 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5a94078b90 0x7f5a9407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:13.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.776+0000 7f5ab5e00640 1 --2- 172.21.15.39:0/3135125170 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5a94078b90 0x7f5a9407b0a0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8007fd0 tx=0x7f5aa8009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:13.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.779+0000 7f5aae000640 1 -- 172.21.15.39:0/3135125170 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5ab0083f70 con 0x7f5ab815da30 2024-09-21T07:46:13.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:13.985+0000 7f5abd200640 1 -- 172.21.15.39:0/3135125170 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5ab80d11f0 con 0x7f5ab815da30 2024-09-21T07:46:14.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.512+0000 7f5aae000640 1 -- 172.21.15.39:0/3135125170 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v167) ==== 169+0+0 (secure 0 0 0) 0x7f5ab0083870 con 0x7f5ab815da30 2024-09-21T07:46:14.514 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' removed 2024-09-21T07:46:14.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.516+0000 7f5abd200640 1 -- 172.21.15.39:0/3135125170 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5a94078b90 msgr2=0x7f5a9407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.516+0000 7f5abd200640 1 --2- 172.21.15.39:0/3135125170 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5a94078b90 0x7f5a9407b0a0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8007fd0 tx=0x7f5aa8009040 comp rx=0 tx=0).stop 2024-09-21T07:46:14.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.516+0000 7f5abd200640 1 -- 172.21.15.39:0/3135125170 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ab815da30 msgr2=0x7f5ab815f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.516+0000 7f5abd200640 1 --2- 172.21.15.39:0/3135125170 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ab815da30 0x7f5ab815f1f0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0006010 tx=0x7f5ab0032c90 comp rx=0 tx=0).stop 2024-09-21T07:46:14.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.517+0000 7f5abd200640 1 -- 172.21.15.39:0/3135125170 shutdown_connections 2024-09-21T07:46:14.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.517+0000 7f5abd200640 1 --2- 172.21.15.39:0/3135125170 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5a94078b90 0x7f5a9407b0a0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.517+0000 7f5abd200640 1 --2- 172.21.15.39:0/3135125170 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ab815da30 0x7f5ab815f1f0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.517+0000 7f5abd200640 1 --2- 172.21.15.39:0/3135125170 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5ab815a440 0x7f5ab80c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.517+0000 7f5abd200640 1 --2- 172.21.15.39:0/3135125170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ab80cd230 0x7f5ab81f3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.517+0000 7f5abd200640 1 -- 172.21.15.39:0/3135125170 >> 172.21.15.39:0/3135125170 conn(0x7f5ab80b2650 msgr2=0x7f5ab8072e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:14.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.518+0000 7f5abd200640 1 -- 172.21.15.39:0/3135125170 shutdown_connections 2024-09-21T07:46:14.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.518+0000 7f5abd200640 1 -- 172.21.15.39:0/3135125170 wait complete. 2024-09-21T07:46:14.543 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T07:46:14.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.771+0000 7fe7d9000640 1 -- 172.21.15.39:0/2340249661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7d40b30a0 msgr2=0x7fe7d40cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.771+0000 7fe7d9000640 1 --2- 172.21.15.39:0/2340249661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7d40b30a0 0x7fe7d40cb8a0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4009830 tx=0x7fe7c402a480 comp rx=0 tx=0).stop 2024-09-21T07:46:14.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.771+0000 7fe7d9000640 1 -- 172.21.15.39:0/2340249661 shutdown_connections 2024-09-21T07:46:14.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.771+0000 7fe7d9000640 1 --2- 172.21.15.39:0/2340249661 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7d415d830 0x7fe7d4156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.771+0000 7fe7d9000640 1 --2- 172.21.15.39:0/2340249661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7d40b30a0 0x7fe7d40cb8a0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.771+0000 7fe7d9000640 1 -- 172.21.15.39:0/2340249661 >> 172.21.15.39:0/2340249661 conn(0x7fe7d40b2460 msgr2=0x7fe7d406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:14.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.771+0000 7fe7d9000640 1 -- 172.21.15.39:0/2340249661 shutdown_connections 2024-09-21T07:46:14.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.771+0000 7fe7d9000640 1 -- 172.21.15.39:0/2340249661 wait complete. 2024-09-21T07:46:14.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.773+0000 7fe7d9000640 1 Processor -- start 2024-09-21T07:46:14.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.773+0000 7fe7d9000640 1 -- start start 2024-09-21T07:46:14.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.775+0000 7fe7d9000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7d40b30a0 0x7fe7d41f50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.775+0000 7fe7d9000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7d40ccc60 0x7fe7d40c7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.775+0000 7fe7d9000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7d415d830 0x7fe7d40cf270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.775+0000 7fe7d9000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d40af450 con 0x7fe7d40b30a0 2024-09-21T07:46:14.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.775+0000 7fe7d9000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d40af2d0 con 0x7fe7d40ccc60 2024-09-21T07:46:14.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.775+0000 7fe7d9000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe7d40af5d0 con 0x7fe7d415d830 2024-09-21T07:46:14.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.775+0000 7fe7d1e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7d40b30a0 0x7fe7d41f50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.775+0000 7fe7d1e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7d40b30a0 0x7fe7d41f50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53024/0 (socket says 172.21.15.39:53024) 2024-09-21T07:46:14.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.775+0000 7fe7d1e00640 1 -- 172.21.15.39:0/1155878309 learned_addr learned my addr 172.21.15.39:0/1155878309 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:14.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.775+0000 7fe7d1400640 1 --2- 172.21.15.39:0/1155878309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7d40ccc60 0x7fe7d40c7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.775+0000 7fe7d2800640 1 --2- 172.21.15.39:0/1155878309 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7d415d830 0x7fe7d40cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.776+0000 7fe7d1e00640 1 -- 172.21.15.39:0/1155878309 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7d415d830 msgr2=0x7fe7d40cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.776+0000 7fe7d1e00640 1 --2- 172.21.15.39:0/1155878309 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7d415d830 0x7fe7d40cf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.776+0000 7fe7d1e00640 1 -- 172.21.15.39:0/1155878309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7d40ccc60 msgr2=0x7fe7d40c7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.776+0000 7fe7d1e00640 1 --2- 172.21.15.39:0/1155878309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7d40ccc60 0x7fe7d40c7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.776+0000 7fe7d1e00640 1 -- 172.21.15.39:0/1155878309 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d41f5610 con 0x7fe7d40b30a0 2024-09-21T07:46:14.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.776+0000 7fe7d1e00640 1 --2- 172.21.15.39:0/1155878309 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7d40b30a0 0x7fe7d41f50d0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fe7c402aa70 tx=0x7fe7c4033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:14.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.776+0000 7fe7ca000640 1 -- 172.21.15.39:0/1155878309 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4005c90 con 0x7fe7d40b30a0 2024-09-21T07:46:14.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.776+0000 7fe7ca000640 1 -- 172.21.15.39:0/1155878309 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe7c4031e50 con 0x7fe7d40b30a0 2024-09-21T07:46:14.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.776+0000 7fe7ca000640 1 -- 172.21.15.39:0/1155878309 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4005260 con 0x7fe7d40b30a0 2024-09-21T07:46:14.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.776+0000 7fe7d9000640 1 -- 172.21.15.39:0/1155878309 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7d40ce1d0 con 0x7fe7d40b30a0 2024-09-21T07:46:14.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.776+0000 7fe7d9000640 1 -- 172.21.15.39:0/1155878309 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7d40ce790 con 0x7fe7d40b30a0 2024-09-21T07:46:14.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.777+0000 7fe7d9000640 1 -- 172.21.15.39:0/1155878309 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7d40afa30 con 0x7fe7d40b30a0 2024-09-21T07:46:14.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.779+0000 7fe7ca000640 1 -- 172.21.15.39:0/1155878309 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe7c4005400 con 0x7fe7d40b30a0 2024-09-21T07:46:14.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.780+0000 7fe7ca000640 1 --2- 172.21.15.39:0/1155878309 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe7b4078b90 0x7fe7b407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.781+0000 7fe7ca000640 1 -- 172.21.15.39:0/1155878309 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 6122+0+0 (secure 0 0 0) 0x7fe7c40bbe90 con 0x7fe7d40b30a0 2024-09-21T07:46:14.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.781+0000 7fe7d1400640 1 --2- 172.21.15.39:0/1155878309 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe7b4078b90 0x7fe7b407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.782+0000 7fe7d1400640 1 --2- 172.21.15.39:0/1155878309 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe7b4078b90 0x7fe7b407b0a0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fe7c00096c0 tx=0x7fe7c0009260 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:14.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.785+0000 7fe7ca000640 1 -- 172.21.15.39:0/1155878309 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe7c403f030 con 0x7fe7d40b30a0 2024-09-21T07:46:14.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.992+0000 7fe7d9000640 1 -- 172.21.15.39:0/1155878309 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7fe7d415d610 con 0x7fe7d40b30a0 2024-09-21T07:46:14.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.996+0000 7fe7ca000640 1 -- 172.21.15.39:0/1155878309 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v46) ==== 71+0+0 (secure 0 0 0) 0x7fe7c4083b70 con 0x7fe7d40b30a0 2024-09-21T07:46:15.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.999+0000 7fe7d9000640 1 -- 172.21.15.39:0/1155878309 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe7b4078b90 msgr2=0x7fe7b407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:15.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.999+0000 7fe7d9000640 1 --2- 172.21.15.39:0/1155878309 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe7b4078b90 0x7fe7b407b0a0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fe7c00096c0 tx=0x7fe7c0009260 comp rx=0 tx=0).stop 2024-09-21T07:46:15.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.999+0000 7fe7d9000640 1 -- 172.21.15.39:0/1155878309 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7d40b30a0 msgr2=0x7fe7d41f50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:15.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.999+0000 7fe7d9000640 1 --2- 172.21.15.39:0/1155878309 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7d40b30a0 0x7fe7d41f50d0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fe7c402aa70 tx=0x7fe7c4033c20 comp rx=0 tx=0).stop 2024-09-21T07:46:15.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.999+0000 7fe7d9000640 1 -- 172.21.15.39:0/1155878309 shutdown_connections 2024-09-21T07:46:15.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.999+0000 7fe7d9000640 1 --2- 172.21.15.39:0/1155878309 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe7b4078b90 0x7fe7b407b0a0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.999+0000 7fe7d9000640 1 --2- 172.21.15.39:0/1155878309 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7d415d830 0x7fe7d40cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.999+0000 7fe7d9000640 1 --2- 172.21.15.39:0/1155878309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7d40ccc60 0x7fe7d40c7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.999+0000 7fe7d9000640 1 --2- 172.21.15.39:0/1155878309 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7d40b30a0 0x7fe7d41f50d0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.999+0000 7fe7d9000640 1 -- 172.21.15.39:0/1155878309 >> 172.21.15.39:0/1155878309 conn(0x7fe7d40b2460 msgr2=0x7fe7d40ad540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:15.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.999+0000 7fe7d9000640 1 -- 172.21.15.39:0/1155878309 shutdown_connections 2024-09-21T07:46:15.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:14.999+0000 7fe7d9000640 1 -- 172.21.15.39:0/1155878309 wait complete. 2024-09-21T07:46:15.022 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_mds_max_snaps_per_dir_with_increased_limit' 2024-09-21T07:46:15.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:15.248+0000 7ffa51600640 1 -- 172.21.15.39:0/3945100787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa4c0cf450 msgr2=0x7ffa4c073130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:15.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:15.248+0000 7ffa51600640 1 --2- 172.21.15.39:0/3945100787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa4c0cf450 0x7ffa4c073130 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0ceee0 tx=0x7ffa3c02a580 comp rx=0 tx=0).stop 2024-09-21T07:46:15.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:15.248+0000 7ffa51600640 1 -- 172.21.15.39:0/3945100787 shutdown_connections 2024-09-21T07:46:15.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:15.248+0000 7ffa51600640 1 --2- 172.21.15.39:0/3945100787 >> v2:172.21.15.148:3300/0 conn(0x7ffa4c073670 0x7ffa4c15d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:15.248+0000 7ffa51600640 1 --2- 172.21.15.39:0/3945100787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa4c0cf450 0x7ffa4c073130 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:15.248+0000 7ffa51600640 1 -- 172.21.15.39:0/3945100787 >> 172.21.15.39:0/3945100787 conn(0x7ffa4c06cfc0 msgr2=0x7ffa4c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:15.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:15.248+0000 7ffa51600640 1 -- 172.21.15.39:0/3945100787 shutdown_connections 2024-09-21T07:46:15.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:15.248+0000 7ffa51600640 1 -- 172.21.15.39:0/3945100787 wait complete. 2024-09-21T07:46:15.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:15.250+0000 7ffa51600640 1 Processor -- start 2024-09-21T07:46:15.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:15.250+0000 7ffa51600640 1 -- start start 2024-09-21T07:46:15.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:15.252+0000 7ffa51600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffa4c073670 0x7ffa4c147260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:15.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:15.252+0000 7ffa51600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa4c0cf450 0x7ffa4c1f9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:15.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:15.252+0000 7ffa51600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa4c15d8e0 0x7ffa4c1f97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:15.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:15.252+0000 7ffa51600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c0cd6b0 con 0x7ffa4c15d8e0 2024-09-21T07:46:15.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:15.252+0000 7ffa51600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c0cd530 con 0x7ffa4c0cf450 2024-09-21T07:46:15.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:15.252+0000 7ffa51600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7ffa4c0cd830 con 0x7ffa4c073670 2024-09-21T07:46:15.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:15.252+0000 7ffa4b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa4c15d8e0 0x7ffa4c1f97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:15.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:15.252+0000 7ffa4b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa4c15d8e0 0x7ffa4c1f97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53040/0 (socket says 172.21.15.39:53040) 2024-09-21T07:46:15.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:15.252+0000 7ffa4b400640 1 -- 172.21.15.39:0/2117844486 learned_addr learned my addr 172.21.15.39:0/2117844486 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:15.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:15.252+0000 7ffa4b400640 1 -- 172.21.15.39:0/2117844486 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffa4c073670 msgr2=0x7ffa4c147260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:15.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:15.252+0000 7ffa4b400640 1 --2- 172.21.15.39:0/2117844486 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffa4c073670 0x7ffa4c147260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:15.252+0000 7ffa4b400640 1 -- 172.21.15.39:0/2117844486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa4c0cf450 msgr2=0x7ffa4c1f9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:15.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:15.252+0000 7ffa42000640 1 --2- 172.21.15.39:0/2117844486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa4c0cf450 0x7ffa4c1f9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:15.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:15.253+0000 7ffa4b400640 1 --2- 172.21.15.39:0/2117844486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa4c0cf450 0x7ffa4c1f9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:15.253+0000 7ffa4b400640 1 -- 172.21.15.39:0/2117844486 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa4c0b2400 con 0x7ffa4c15d8e0 2024-09-21T07:46:15.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:15.253+0000 7ffa42000640 1 --2- 172.21.15.39:0/2117844486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa4c0cf450 0x7ffa4c1f9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:46:15.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:15.253+0000 7ffa4b400640 1 --2- 172.21.15.39:0/2117844486 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa4c15d8e0 0x7ffa4c1f97b0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7ffa4400ba10 tx=0x7ffa4400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:15.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:15.253+0000 7ffa43400640 1 -- 172.21.15.39:0/2117844486 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44019070 con 0x7ffa4c15d8e0 2024-09-21T07:46:15.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:15.253+0000 7ffa43400640 1 -- 172.21.15.39:0/2117844486 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa4400cc90 con 0x7ffa4c15d8e0 2024-09-21T07:46:15.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:15.253+0000 7ffa43400640 1 -- 172.21.15.39:0/2117844486 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa440146a0 con 0x7ffa4c15d8e0 2024-09-21T07:46:15.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:15.253+0000 7ffa51600640 1 -- 172.21.15.39:0/2117844486 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa4c0b26e0 con 0x7ffa4c15d8e0 2024-09-21T07:46:15.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:15.253+0000 7ffa51600640 1 -- 172.21.15.39:0/2117844486 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa4c0b2ca0 con 0x7ffa4c15d8e0 2024-09-21T07:46:15.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:15.254+0000 7ffa51600640 1 -- 172.21.15.39:0/2117844486 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa4c0cdde0 con 0x7ffa4c15d8e0 2024-09-21T07:46:15.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:15.255+0000 7ffa43400640 1 -- 172.21.15.39:0/2117844486 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ffa4401c070 con 0x7ffa4c15d8e0 2024-09-21T07:46:15.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:15.256+0000 7ffa43400640 1 --2- 172.21.15.39:0/2117844486 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ffa28078b90 0x7ffa2807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:15.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:15.257+0000 7ffa43400640 1 -- 172.21.15.39:0/2117844486 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 6122+0+0 (secure 0 0 0) 0x7ffa4409acc0 con 0x7ffa4c15d8e0 2024-09-21T07:46:15.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:15.257+0000 7ffa4aa00640 1 --2- 172.21.15.39:0/2117844486 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ffa28078b90 0x7ffa2807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:15.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:15.258+0000 7ffa4aa00640 1 --2- 172.21.15.39:0/2117844486 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ffa28078b90 0x7ffa2807b0a0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c033000 tx=0x7ffa3c0046c0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:15.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:15.261+0000 7ffa43400640 1 -- 172.21.15.39:0/2117844486 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ffa440a1080 con 0x7ffa4c15d8e0 2024-09-21T07:46:15.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:15.530+0000 7ffa51600640 1 -- 172.21.15.39:0/2117844486 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_mds_max_snaps_per_dir_with_increased_limit"]} v 0) -- 0x7ffa4c0b55b0 con 0x7ffa4c15d8e0 2024-09-21T07:46:16.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.521+0000 7ffa43400640 1 -- 172.21.15.39:0/2117844486 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_mds_max_snaps_per_dir_with_increased_limit"]}]=0 v632) ==== 168+0+0 (secure 0 0 0) 0x7ffa44062f50 con 0x7ffa4c15d8e0 2024-09-21T07:46:16.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.524+0000 7ffa51600640 1 -- 172.21.15.39:0/2117844486 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ffa28078b90 msgr2=0x7ffa2807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:16.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.524+0000 7ffa51600640 1 --2- 172.21.15.39:0/2117844486 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ffa28078b90 0x7ffa2807b0a0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c033000 tx=0x7ffa3c0046c0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.524+0000 7ffa51600640 1 -- 172.21.15.39:0/2117844486 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa4c15d8e0 msgr2=0x7ffa4c1f97b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:16.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.524+0000 7ffa51600640 1 --2- 172.21.15.39:0/2117844486 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa4c15d8e0 0x7ffa4c1f97b0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7ffa4400ba10 tx=0x7ffa4400bed0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.524+0000 7ffa51600640 1 -- 172.21.15.39:0/2117844486 shutdown_connections 2024-09-21T07:46:16.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.524+0000 7ffa51600640 1 --2- 172.21.15.39:0/2117844486 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ffa28078b90 0x7ffa2807b0a0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.524+0000 7ffa51600640 1 --2- 172.21.15.39:0/2117844486 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa4c15d8e0 0x7ffa4c1f97b0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.524+0000 7ffa51600640 1 --2- 172.21.15.39:0/2117844486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa4c0cf450 0x7ffa4c1f9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.524+0000 7ffa51600640 1 --2- 172.21.15.39:0/2117844486 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffa4c073670 0x7ffa4c147260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.524+0000 7ffa51600640 1 -- 172.21.15.39:0/2117844486 >> 172.21.15.39:0/2117844486 conn(0x7ffa4c06cfc0 msgr2=0x7ffa4c068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:16.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.525+0000 7ffa51600640 1 -- 172.21.15.39:0/2117844486 shutdown_connections 2024-09-21T07:46:16.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.525+0000 7ffa51600640 1 -- 172.21.15.39:0/2117844486 wait complete. 2024-09-21T07:46:16.555 INFO:tasks.cephfs_test_runner:Test the newly introudced option named mds_max_snaps_per_dir ... ok 2024-09-21T07:46:16.555 INFO:tasks.cephfs_test_runner:Starting test: test_mds_max_snaps_per_dir_with_reduced_limit (tasks.cephfs.test_snapshots.TestSnapshots) Test the newly introudced option named mds_max_snaps_per_dir 2024-09-21T07:46:16.555 INFO:tasks.cephfs_test_runner:test_mds_max_snaps_per_dir_with_reduced_limit (tasks.cephfs.test_snapshots.TestSnapshots) 2024-09-21T07:46:16.555 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:46:16.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.785+0000 7fea8be00640 1 -- 172.21.15.39:0/4173456897 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea840cd260 msgr2=0x7fea840cf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:16.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.785+0000 7fea8be00640 1 --2- 172.21.15.39:0/4173456897 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea840cd260 0x7fea840cf450 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fea70007350 tx=0x7fea7002b1a0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.785+0000 7fea8be00640 1 -- 172.21.15.39:0/4173456897 shutdown_connections 2024-09-21T07:46:16.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.785+0000 7fea8be00640 1 --2- 172.21.15.39:0/4173456897 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea840cf990 0x7fea840b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.785+0000 7fea8be00640 1 --2- 172.21.15.39:0/4173456897 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea840cd260 0x7fea840cf450 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.785+0000 7fea8be00640 1 -- 172.21.15.39:0/4173456897 >> 172.21.15.39:0/4173456897 conn(0x7fea840b2460 msgr2=0x7fea8406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:16.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.785+0000 7fea8be00640 1 -- 172.21.15.39:0/4173456897 shutdown_connections 2024-09-21T07:46:16.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.785+0000 7fea8be00640 1 -- 172.21.15.39:0/4173456897 wait complete. 2024-09-21T07:46:16.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.787+0000 7fea8be00640 1 Processor -- start 2024-09-21T07:46:16.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.787+0000 7fea8be00640 1 -- start start 2024-09-21T07:46:16.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.789+0000 7fea8be00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea840af430 0x7fea841f5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:16.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.789+0000 7fea8be00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea840cd260 0x7fea841f9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:16.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.789+0000 7fea8be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea840cf990 0x7fea841f97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:16.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.789+0000 7fea8be00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fea840b45f0 con 0x7fea840cf990 2024-09-21T07:46:16.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.789+0000 7fea8be00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fea84071230 con 0x7fea840af430 2024-09-21T07:46:16.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.789+0000 7fea8be00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fea840b4770 con 0x7fea840cd260 2024-09-21T07:46:16.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.789+0000 7fea89e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea840cf990 0x7fea841f97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:16.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.789+0000 7fea89e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea840cf990 0x7fea841f97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53046/0 (socket says 172.21.15.39:53046) 2024-09-21T07:46:16.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.789+0000 7fea89e00640 1 -- 172.21.15.39:0/2127487939 learned_addr learned my addr 172.21.15.39:0/2127487939 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:16.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.789+0000 7fea89400640 1 --2- 172.21.15.39:0/2127487939 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea840af430 0x7fea841f5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:16.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.789+0000 7fea88a00640 1 --2- 172.21.15.39:0/2127487939 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea840cd260 0x7fea841f9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:16.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.790+0000 7fea89e00640 1 -- 172.21.15.39:0/2127487939 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea840cd260 msgr2=0x7fea841f9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:16.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.790+0000 7fea89e00640 1 --2- 172.21.15.39:0/2127487939 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea840cd260 0x7fea841f9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.790+0000 7fea89e00640 1 -- 172.21.15.39:0/2127487939 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea840af430 msgr2=0x7fea841f5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:16.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.790+0000 7fea89e00640 1 --2- 172.21.15.39:0/2127487939 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea840af430 0x7fea841f5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.790+0000 7fea89e00640 1 -- 172.21.15.39:0/2127487939 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8406f930 con 0x7fea840cf990 2024-09-21T07:46:16.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.790+0000 7fea89e00640 1 --2- 172.21.15.39:0/2127487939 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea840cf990 0x7fea841f97d0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fea7c00ec50 tx=0x7fea7c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:16.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.790+0000 7fea81600640 1 -- 172.21.15.39:0/2127487939 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c003b70 con 0x7fea840cf990 2024-09-21T07:46:16.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.790+0000 7fea81600640 1 -- 172.21.15.39:0/2127487939 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea7c003d10 con 0x7fea840cf990 2024-09-21T07:46:16.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.790+0000 7fea81600640 1 -- 172.21.15.39:0/2127487939 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c004280 con 0x7fea840cf990 2024-09-21T07:46:16.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.790+0000 7fea8be00640 1 -- 172.21.15.39:0/2127487939 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea840af9f0 con 0x7fea840cf990 2024-09-21T07:46:16.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.790+0000 7fea8be00640 1 -- 172.21.15.39:0/2127487939 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea840affb0 con 0x7fea840cf990 2024-09-21T07:46:16.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.791+0000 7fea8be00640 1 -- 172.21.15.39:0/2127487939 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea841549c0 con 0x7fea840cf990 2024-09-21T07:46:16.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.793+0000 7fea81600640 1 -- 172.21.15.39:0/2127487939 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fea7c004420 con 0x7fea840cf990 2024-09-21T07:46:16.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.794+0000 7fea81600640 1 --2- 172.21.15.39:0/2127487939 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fea64078b90 0x7fea6407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:16.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.794+0000 7fea81600640 1 -- 172.21.15.39:0/2127487939 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 6122+0+0 (secure 0 0 0) 0x7fea7c09bc90 con 0x7fea840cf990 2024-09-21T07:46:16.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.794+0000 7fea89400640 1 --2- 172.21.15.39:0/2127487939 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fea64078b90 0x7fea6407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:16.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.795+0000 7fea89400640 1 --2- 172.21.15.39:0/2127487939 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fea64078b90 0x7fea6407b0a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fea78007fd0 tx=0x7fea78001370 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:16.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.798+0000 7fea81600640 1 -- 172.21.15.39:0/2127487939 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fea841549c0 con 0x7fea840cf990 2024-09-21T07:46:16.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.997+0000 7fea8be00640 1 -- 172.21.15.39:0/2127487939 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fea840cf450 con 0x7fea840cf990 2024-09-21T07:46:17.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:16.998+0000 7fea81600640 1 -- 172.21.15.39:0/2127487939 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v167) ==== 74+0+18647 (secure 0 0 0) 0x7fea7c063840 con 0x7fea840cf990 2024-09-21T07:46:17.000 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:46:17.000 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":167,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:46:14.502614+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6834/2389975689":"2024-09-22T07:46:11.283655+0000","172.21.15.39:6832/2389975689":"2024-09-22T07:46:11.283655+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]},{"pool":20,"snaps":[{"begin":80,"length":23},{"begin":181,"length":23}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:46:17.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.000+0000 7fea8be00640 1 -- 172.21.15.39:0/2127487939 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fea64078b90 msgr2=0x7fea6407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:17.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.000+0000 7fea8be00640 1 --2- 172.21.15.39:0/2127487939 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fea64078b90 0x7fea6407b0a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fea78007fd0 tx=0x7fea78001370 comp rx=0 tx=0).stop 2024-09-21T07:46:17.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.001+0000 7fea8be00640 1 -- 172.21.15.39:0/2127487939 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea840cf990 msgr2=0x7fea841f97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:17.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.001+0000 7fea8be00640 1 --2- 172.21.15.39:0/2127487939 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea840cf990 0x7fea841f97d0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fea7c00ec50 tx=0x7fea7c00c6a0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.001+0000 7fea8be00640 1 -- 172.21.15.39:0/2127487939 shutdown_connections 2024-09-21T07:46:17.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.001+0000 7fea8be00640 1 --2- 172.21.15.39:0/2127487939 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fea64078b90 0x7fea6407b0a0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.001+0000 7fea8be00640 1 --2- 172.21.15.39:0/2127487939 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea840cf990 0x7fea841f97d0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.001+0000 7fea8be00640 1 --2- 172.21.15.39:0/2127487939 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea840cd260 0x7fea841f9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.001+0000 7fea8be00640 1 --2- 172.21.15.39:0/2127487939 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea840af430 0x7fea841f5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.001+0000 7fea8be00640 1 -- 172.21.15.39:0/2127487939 >> 172.21.15.39:0/2127487939 conn(0x7fea840b2460 msgr2=0x7fea840ad4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:17.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.001+0000 7fea8be00640 1 -- 172.21.15.39:0/2127487939 shutdown_connections 2024-09-21T07:46:17.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.001+0000 7fea8be00640 1 -- 172.21.15.39:0/2127487939 wait complete. 2024-09-21T07:46:17.024 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T07:33:01.569433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'snap-test-pool', 'create_time': '2024-09-21T07:34:22.032185+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 1, 'snap_epoch': 37, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'test_data_pool', 'create_time': '2024-09-21T07:35:25.604532+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 1, 'snap_epoch': 66, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_metadata_pool', 'create_time': '2024-09-21T07:35:26.327607+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T07:46:17.024 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T07:46:17.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.252+0000 7f0ee3e00640 1 -- 172.21.15.39:0/3402734638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ec8003c10 msgr2=0x7f0ec8004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:17.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.252+0000 7f0ee3e00640 1 --2- 172.21.15.39:0/3402734638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ec8003c10 0x7f0ec8004060 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4001830 tx=0x7f0ec8005830 comp rx=0 tx=0).stop 2024-09-21T07:46:17.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.252+0000 7f0ee3400640 1 -- 172.21.15.39:0/3402734638 reap_dead start 2024-09-21T07:46:17.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.252+0000 7f0ee3e00640 1 -- 172.21.15.39:0/3402734638 shutdown_connections 2024-09-21T07:46:17.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.252+0000 7f0ee3e00640 1 -- 172.21.15.39:0/3402734638 >> 172.21.15.39:0/3402734638 conn(0x7f0ee406cde0 msgr2=0x7f0ee406d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:17.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.252+0000 7f0ee3e00640 1 -- 172.21.15.39:0/3402734638 shutdown_connections 2024-09-21T07:46:17.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.252+0000 7f0ee3e00640 1 -- 172.21.15.39:0/3402734638 wait complete. 2024-09-21T07:46:17.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.255+0000 7f0ee3e00640 1 Processor -- start 2024-09-21T07:46:17.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.255+0000 7f0ee3e00640 1 -- start start 2024-09-21T07:46:17.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.256+0000 7f0ee3e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ec8003c10 0x7f0ee41f5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:17.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.256+0000 7f0ee3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ee40b2a90 0x7f0ee40b1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:17.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.256+0000 7f0ee3e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0ee4158df0 0x7f0ee4159200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:17.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.256+0000 7f0ee3e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee40d0a60 con 0x7f0ee40b2a90 2024-09-21T07:46:17.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.256+0000 7f0ee3e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee40d08e0 con 0x7f0ec8003c10 2024-09-21T07:46:17.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.256+0000 7f0ee3e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f0ee40d0be0 con 0x7f0ee4158df0 2024-09-21T07:46:17.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.257+0000 7f0edbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ee40b2a90 0x7f0ee40b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:17.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.257+0000 7f0edbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ee40b2a90 0x7f0ee40b1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53062/0 (socket says 172.21.15.39:53062) 2024-09-21T07:46:17.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.257+0000 7f0edbe00640 1 -- 172.21.15.39:0/431542210 learned_addr learned my addr 172.21.15.39:0/431542210 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:17.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.257+0000 7f0ee2a00640 1 --2- 172.21.15.39:0/431542210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ec8003c10 0x7f0ee41f5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:17.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.257+0000 7f0ee3400640 1 --2- 172.21.15.39:0/431542210 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0ee4158df0 0x7f0ee4159200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:17.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.257+0000 7f0edbe00640 1 -- 172.21.15.39:0/431542210 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0ee4158df0 msgr2=0x7f0ee4159200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:17.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.257+0000 7f0edbe00640 1 --2- 172.21.15.39:0/431542210 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0ee4158df0 0x7f0ee4159200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.257+0000 7f0edbe00640 1 -- 172.21.15.39:0/431542210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ec8003c10 msgr2=0x7f0ee41f5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:17.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.257+0000 7f0edbe00640 1 --2- 172.21.15.39:0/431542210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ec8003c10 0x7f0ee41f5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.257+0000 7f0edbe00640 1 -- 172.21.15.39:0/431542210 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ee4159740 con 0x7f0ee40b2a90 2024-09-21T07:46:17.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.257+0000 7f0edbe00640 1 --2- 172.21.15.39:0/431542210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ee40b2a90 0x7f0ee40b1820 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc011ae0 tx=0x7f0ecc009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:17.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.258+0000 7f0edaa00640 1 -- 172.21.15.39:0/431542210 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc01e070 con 0x7f0ee40b2a90 2024-09-21T07:46:17.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.258+0000 7f0edaa00640 1 -- 172.21.15.39:0/431542210 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ecc004460 con 0x7f0ee40b2a90 2024-09-21T07:46:17.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.258+0000 7f0edaa00640 1 -- 172.21.15.39:0/431542210 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc00fbe0 con 0x7f0ee40b2a90 2024-09-21T07:46:17.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.258+0000 7f0ee3e00640 1 -- 172.21.15.39:0/431542210 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ee41574e0 con 0x7f0ee40b2a90 2024-09-21T07:46:17.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.258+0000 7f0ee3e00640 1 -- 172.21.15.39:0/431542210 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ee4157aa0 con 0x7f0ee40b2a90 2024-09-21T07:46:17.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.259+0000 7f0ee3e00640 1 -- 172.21.15.39:0/431542210 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ee406ecf0 con 0x7f0ee40b2a90 2024-09-21T07:46:17.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.260+0000 7f0edaa00640 1 -- 172.21.15.39:0/431542210 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0ecc021070 con 0x7f0ee40b2a90 2024-09-21T07:46:17.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.261+0000 7f0edaa00640 1 --2- 172.21.15.39:0/431542210 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0ec0078bc0 0x7f0ec007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:17.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.261+0000 7f0edaa00640 1 -- 172.21.15.39:0/431542210 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 6122+0+0 (secure 0 0 0) 0x7f0ecc09b750 con 0x7f0ee40b2a90 2024-09-21T07:46:17.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.262+0000 7f0ee2a00640 1 --2- 172.21.15.39:0/431542210 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0ec0078bc0 0x7f0ec007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:17.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.263+0000 7f0ee2a00640 1 --2- 172.21.15.39:0/431542210 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0ec0078bc0 0x7f0ec007b0d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4032000 tx=0x7f0ed4001360 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:17.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.266+0000 7f0edaa00640 1 -- 172.21.15.39:0/431542210 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0ee4159740 con 0x7f0ee40b2a90 2024-09-21T07:46:17.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.467+0000 7f0ee3e00640 1 -- 172.21.15.39:0/431542210 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0ee4157210 con 0x7f0ee40b2a90 2024-09-21T07:46:17.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.468+0000 7f0edaa00640 1 -- 172.21.15.39:0/431542210 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v167) ==== 93+0+10 (secure 0 0 0) 0x7f0ecc063430 con 0x7f0ee40b2a90 2024-09-21T07:46:17.469 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-09-21T07:46:17.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.470+0000 7f0ee3e00640 1 -- 172.21.15.39:0/431542210 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0ec0078bc0 msgr2=0x7f0ec007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:17.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.470+0000 7f0ee3e00640 1 --2- 172.21.15.39:0/431542210 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0ec0078bc0 0x7f0ec007b0d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4032000 tx=0x7f0ed4001360 comp rx=0 tx=0).stop 2024-09-21T07:46:17.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.470+0000 7f0ee3e00640 1 -- 172.21.15.39:0/431542210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ee40b2a90 msgr2=0x7f0ee40b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:17.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.470+0000 7f0ee3e00640 1 --2- 172.21.15.39:0/431542210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ee40b2a90 0x7f0ee40b1820 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc011ae0 tx=0x7f0ecc009e40 comp rx=0 tx=0).stop 2024-09-21T07:46:17.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.470+0000 7f0ee3e00640 1 -- 172.21.15.39:0/431542210 shutdown_connections 2024-09-21T07:46:17.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.470+0000 7f0ee3e00640 1 --2- 172.21.15.39:0/431542210 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0ec0078bc0 0x7f0ec007b0d0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.470+0000 7f0ee3e00640 1 --2- 172.21.15.39:0/431542210 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0ee4158df0 0x7f0ee4159200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.470+0000 7f0ee3e00640 1 --2- 172.21.15.39:0/431542210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ee40b2a90 0x7f0ee40b1820 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.470+0000 7f0ee3e00640 1 --2- 172.21.15.39:0/431542210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ec8003c10 0x7f0ee41f5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.470+0000 7f0ee3e00640 1 -- 172.21.15.39:0/431542210 >> 172.21.15.39:0/431542210 conn(0x7f0ee406cde0 msgr2=0x7f0ee415bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:17.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.470+0000 7f0ee3e00640 1 -- 172.21.15.39:0/431542210 shutdown_connections 2024-09-21T07:46:17.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.471+0000 7f0ee3e00640 1 -- 172.21.15.39:0/431542210 wait complete. 2024-09-21T07:46:17.494 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2024-09-21T07:46:17.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.723+0000 7fe6a8400640 1 -- 172.21.15.39:0/2229867433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6a015d830 msgr2=0x7fe6a0156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:17.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.723+0000 7fe6a8400640 1 --2- 172.21.15.39:0/2229867433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6a015d830 0x7fe6a0156bb0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0071e60 tx=0x7fe68c02a4d0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.723+0000 7fe6a8400640 1 -- 172.21.15.39:0/2229867433 shutdown_connections 2024-09-21T07:46:17.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.723+0000 7fe6a8400640 1 --2- 172.21.15.39:0/2229867433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6a015d830 0x7fe6a0156bb0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.723+0000 7fe6a8400640 1 --2- 172.21.15.39:0/2229867433 >> v2:172.21.15.148:3300/0 conn(0x7fe6a00b30a0 0x7fe6a00cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.723+0000 7fe6a8400640 1 -- 172.21.15.39:0/2229867433 >> 172.21.15.39:0/2229867433 conn(0x7fe6a00b2460 msgr2=0x7fe6a006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:17.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.724+0000 7fe6a8400640 1 -- 172.21.15.39:0/2229867433 shutdown_connections 2024-09-21T07:46:17.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.724+0000 7fe6a8400640 1 -- 172.21.15.39:0/2229867433 wait complete. 2024-09-21T07:46:17.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.726+0000 7fe6a8400640 1 Processor -- start 2024-09-21T07:46:17.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.726+0000 7fe6a8400640 1 -- start start 2024-09-21T07:46:17.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.727+0000 7fe6a8400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6a00b30a0 0x7fe6a00cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:17.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.727+0000 7fe6a8400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe6a00ccc60 0x7fe6a01559c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:17.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.727+0000 7fe6a8400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6a015d830 0x7fe6a01e93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:17.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.728+0000 7fe6a8400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a00af410 con 0x7fe6a00b30a0 2024-09-21T07:46:17.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.728+0000 7fe6a8400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a00af290 con 0x7fe6a015d830 2024-09-21T07:46:17.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.728+0000 7fe6a8400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe6a00af590 con 0x7fe6a00ccc60 2024-09-21T07:46:17.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.728+0000 7fe6a5a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6a00b30a0 0x7fe6a00cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:17.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.728+0000 7fe6a5a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6a00b30a0 0x7fe6a00cfc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53072/0 (socket says 172.21.15.39:53072) 2024-09-21T07:46:17.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.728+0000 7fe6a5a00640 1 -- 172.21.15.39:0/3978252383 learned_addr learned my addr 172.21.15.39:0/3978252383 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:17.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.728+0000 7fe6a5000640 1 --2- 172.21.15.39:0/3978252383 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe6a00ccc60 0x7fe6a01559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:17.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.728+0000 7fe6a6400640 1 --2- 172.21.15.39:0/3978252383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6a015d830 0x7fe6a01e93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:17.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.728+0000 7fe6a5a00640 1 -- 172.21.15.39:0/3978252383 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe6a00ccc60 msgr2=0x7fe6a01559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:17.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.728+0000 7fe6a5a00640 1 --2- 172.21.15.39:0/3978252383 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe6a00ccc60 0x7fe6a01559c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.728+0000 7fe6a5a00640 1 -- 172.21.15.39:0/3978252383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6a015d830 msgr2=0x7fe6a01e93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:17.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.728+0000 7fe6a5a00640 1 --2- 172.21.15.39:0/3978252383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6a015d830 0x7fe6a01e93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.728+0000 7fe6a5a00640 1 -- 172.21.15.39:0/3978252383 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a0155f00 con 0x7fe6a00b30a0 2024-09-21T07:46:17.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.728+0000 7fe6a6400640 1 --2- 172.21.15.39:0/3978252383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6a015d830 0x7fe6a01e93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:17.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.729+0000 7fe6a5a00640 1 --2- 172.21.15.39:0/3978252383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6a00b30a0 0x7fe6a00cfc40 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fe69400c970 tx=0x7fe69400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:17.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.729+0000 7fe69d600640 1 -- 172.21.15.39:0/3978252383 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe694007bb0 con 0x7fe6a00b30a0 2024-09-21T07:46:17.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.729+0000 7fe69d600640 1 -- 172.21.15.39:0/3978252383 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe694007e40 con 0x7fe6a00b30a0 2024-09-21T07:46:17.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.729+0000 7fe6a8400640 1 -- 172.21.15.39:0/3978252383 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6a01fa060 con 0x7fe6a00b30a0 2024-09-21T07:46:17.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.729+0000 7fe69d600640 1 -- 172.21.15.39:0/3978252383 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69400f690 con 0x7fe6a00b30a0 2024-09-21T07:46:17.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.729+0000 7fe6a8400640 1 -- 172.21.15.39:0/3978252383 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6a00c6150 con 0x7fe6a00b30a0 2024-09-21T07:46:17.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.730+0000 7fe6a8400640 1 -- 172.21.15.39:0/3978252383 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6a00b0bc0 con 0x7fe6a00b30a0 2024-09-21T07:46:17.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.731+0000 7fe69d600640 1 -- 172.21.15.39:0/3978252383 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe69400f830 con 0x7fe6a00b30a0 2024-09-21T07:46:17.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.732+0000 7fe69d600640 1 --2- 172.21.15.39:0/3978252383 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe680078b90 0x7fe68007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:17.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.732+0000 7fe69d600640 1 -- 172.21.15.39:0/3978252383 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 6122+0+0 (secure 0 0 0) 0x7fe69409b740 con 0x7fe6a00b30a0 2024-09-21T07:46:17.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.733+0000 7fe6a5000640 1 --2- 172.21.15.39:0/3978252383 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe680078b90 0x7fe68007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:17.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.734+0000 7fe6a5000640 1 --2- 172.21.15.39:0/3978252383 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe680078b90 0x7fe68007b0a0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fe68c033000 tx=0x7fe68c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:17.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.737+0000 7fe69d600640 1 -- 172.21.15.39:0/3978252383 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe694014030 con 0x7fe6a00b30a0 2024-09-21T07:46:17.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.933+0000 7fe6a8400640 1 -- 172.21.15.39:0/3978252383 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) -- 0x7fe6a015cb20 con 0x7fe6a00b30a0 2024-09-21T07:46:17.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.934+0000 7fe69d600640 1 -- 172.21.15.39:0/3978252383 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v167) ==== 103+0+11 (secure 0 0 0) 0x7fe6940638c0 con 0x7fe6a00b30a0 2024-09-21T07:46:17.937 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-21T07:46:17.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.936+0000 7fe6a8400640 1 -- 172.21.15.39:0/3978252383 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe680078b90 msgr2=0x7fe68007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:17.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.936+0000 7fe6a8400640 1 --2- 172.21.15.39:0/3978252383 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe680078b90 0x7fe68007b0a0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fe68c033000 tx=0x7fe68c004730 comp rx=0 tx=0).stop 2024-09-21T07:46:17.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.936+0000 7fe6a8400640 1 -- 172.21.15.39:0/3978252383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6a00b30a0 msgr2=0x7fe6a00cfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:17.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.936+0000 7fe6a8400640 1 --2- 172.21.15.39:0/3978252383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6a00b30a0 0x7fe6a00cfc40 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fe69400c970 tx=0x7fe69400ce30 comp rx=0 tx=0).stop 2024-09-21T07:46:17.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.936+0000 7fe6a8400640 1 -- 172.21.15.39:0/3978252383 shutdown_connections 2024-09-21T07:46:17.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.936+0000 7fe6a8400640 1 --2- 172.21.15.39:0/3978252383 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe680078b90 0x7fe68007b0a0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.936+0000 7fe6a8400640 1 --2- 172.21.15.39:0/3978252383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6a015d830 0x7fe6a01e93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.936+0000 7fe6a8400640 1 --2- 172.21.15.39:0/3978252383 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe6a00ccc60 0x7fe6a01559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.936+0000 7fe6a8400640 1 --2- 172.21.15.39:0/3978252383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6a00b30a0 0x7fe6a00cfc40 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.936+0000 7fe6a8400640 1 -- 172.21.15.39:0/3978252383 >> 172.21.15.39:0/3978252383 conn(0x7fe6a00b2460 msgr2=0x7fe6a015dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:17.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.937+0000 7fe6a8400640 1 -- 172.21.15.39:0/3978252383 shutdown_connections 2024-09-21T07:46:17.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:17.937+0000 7fe6a8400640 1 -- 172.21.15.39:0/3978252383 wait complete. 2024-09-21T07:46:17.962 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2024-09-21T07:46:18.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.189+0000 7f8feb200640 1 -- 172.21.15.39:0/905216422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fec15cc00 msgr2=0x7f8fec15d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:18.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.189+0000 7f8feb200640 1 --2- 172.21.15.39:0/905216422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fec15cc00 0x7f8fec15d010 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4005fd0 tx=0x7f8fd402a5d0 comp rx=0 tx=0).stop 2024-09-21T07:46:18.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.189+0000 7f8feb200640 1 -- 172.21.15.39:0/905216422 shutdown_connections 2024-09-21T07:46:18.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.189+0000 7f8feb200640 1 --2- 172.21.15.39:0/905216422 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8fec0d04b0 0x7f8fec0d08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:18.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.189+0000 7f8feb200640 1 --2- 172.21.15.39:0/905216422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fec15cc00 0x7f8fec15d010 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:18.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.189+0000 7f8feb200640 1 -- 172.21.15.39:0/905216422 >> 172.21.15.39:0/905216422 conn(0x7f8fec06cfc0 msgr2=0x7f8fec06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:18.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.189+0000 7f8feb200640 1 -- 172.21.15.39:0/905216422 shutdown_connections 2024-09-21T07:46:18.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.189+0000 7f8feb200640 1 -- 172.21.15.39:0/905216422 wait complete. 2024-09-21T07:46:18.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.192+0000 7f8feb200640 1 Processor -- start 2024-09-21T07:46:18.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.192+0000 7f8feb200640 1 -- start start 2024-09-21T07:46:18.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.193+0000 7f8feb200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8fec0cfa20 0x7f8fec13f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:18.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.193+0000 7f8feb200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fec0d04b0 0x7f8fec1f8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:18.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.193+0000 7f8feb200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8fec15cc00 0x7f8fec1f8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:18.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.193+0000 7f8feb200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8fec06f080 con 0x7f8fec0d04b0 2024-09-21T07:46:18.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.193+0000 7f8feb200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8fec06ef00 con 0x7f8fec15cc00 2024-09-21T07:46:18.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.193+0000 7f8feb200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f8fec06f200 con 0x7f8fec0cfa20 2024-09-21T07:46:18.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.193+0000 7f8fe1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fec0d04b0 0x7f8fec1f8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:18.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.194+0000 7f8fe1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fec0d04b0 0x7f8fec1f8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53100/0 (socket says 172.21.15.39:53100) 2024-09-21T07:46:18.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.194+0000 7f8fe1400640 1 -- 172.21.15.39:0/1482482102 learned_addr learned my addr 172.21.15.39:0/1482482102 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:18.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.194+0000 7f8fe9e00640 1 --2- 172.21.15.39:0/1482482102 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8fec0cfa20 0x7f8fec13f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:18.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.194+0000 7f8fea800640 1 --2- 172.21.15.39:0/1482482102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8fec15cc00 0x7f8fec1f8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:18.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.194+0000 7f8fe1400640 1 -- 172.21.15.39:0/1482482102 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8fec0cfa20 msgr2=0x7f8fec13f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:18.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.194+0000 7f8fe1400640 1 --2- 172.21.15.39:0/1482482102 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8fec0cfa20 0x7f8fec13f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:18.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.194+0000 7f8fe1400640 1 -- 172.21.15.39:0/1482482102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8fec15cc00 msgr2=0x7f8fec1f8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:18.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.194+0000 7f8fe1400640 1 --2- 172.21.15.39:0/1482482102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8fec15cc00 0x7f8fec1f8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:18.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.194+0000 7f8fe1400640 1 -- 172.21.15.39:0/1482482102 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fec153c90 con 0x7f8fec0d04b0 2024-09-21T07:46:18.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.194+0000 7f8fe9e00640 1 --2- 172.21.15.39:0/1482482102 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8fec0cfa20 0x7f8fec13f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:18.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.194+0000 7f8fea800640 1 --2- 172.21.15.39:0/1482482102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8fec15cc00 0x7f8fec1f8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:18.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.194+0000 7f8fe1400640 1 --2- 172.21.15.39:0/1482482102 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fec0d04b0 0x7f8fec1f8900 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4034040 tx=0x7f8fd40038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:18.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.195+0000 7f8fe2a00640 1 -- 172.21.15.39:0/1482482102 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd403e030 con 0x7f8fec0d04b0 2024-09-21T07:46:18.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.195+0000 7f8fe2a00640 1 -- 172.21.15.39:0/1482482102 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8fd4003ca0 con 0x7f8fec0d04b0 2024-09-21T07:46:18.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.195+0000 7f8fe2a00640 1 -- 172.21.15.39:0/1482482102 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd4035040 con 0x7f8fec0d04b0 2024-09-21T07:46:18.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.195+0000 7f8feb200640 1 -- 172.21.15.39:0/1482482102 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fec1e95b0 con 0x7f8fec0d04b0 2024-09-21T07:46:18.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.195+0000 7f8feb200640 1 -- 172.21.15.39:0/1482482102 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fec1e9b70 con 0x7f8fec0d04b0 2024-09-21T07:46:18.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.196+0000 7f8feb200640 1 -- 172.21.15.39:0/1482482102 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fec06f6b0 con 0x7f8fec0d04b0 2024-09-21T07:46:18.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.197+0000 7f8fe2a00640 1 -- 172.21.15.39:0/1482482102 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8fd4041030 con 0x7f8fec0d04b0 2024-09-21T07:46:18.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.199+0000 7f8fe2a00640 1 --2- 172.21.15.39:0/1482482102 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8fc8078bc0 0x7f8fc807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:18.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.199+0000 7f8fe2a00640 1 -- 172.21.15.39:0/1482482102 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 6122+0+0 (secure 0 0 0) 0x7f8fd40bb4f0 con 0x7f8fec0d04b0 2024-09-21T07:46:18.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.199+0000 7f8fe9e00640 1 --2- 172.21.15.39:0/1482482102 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8fc8078bc0 0x7f8fc807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:18.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.200+0000 7f8fe9e00640 1 --2- 172.21.15.39:0/1482482102 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8fc8078bc0 0x7f8fc807b0d0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc007fd0 tx=0x7f8fdc009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:18.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.203+0000 7f8fe2a00640 1 -- 172.21.15.39:0/1482482102 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8fd4083780 con 0x7f8fec0d04b0 2024-09-21T07:46:18.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.399+0000 7f8feb200640 1 -- 172.21.15.39:0/1482482102 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) -- 0x7f8fec06ddf0 con 0x7f8fec0d04b0 2024-09-21T07:46:18.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.399+0000 7f8fe2a00640 1 -- 172.21.15.39:0/1482482102 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v167) ==== 103+0+11 (secure 0 0 0) 0x7f8fd4083080 con 0x7f8fec0d04b0 2024-09-21T07:46:18.401 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-21T07:46:18.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.401+0000 7f8feb200640 1 -- 172.21.15.39:0/1482482102 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8fc8078bc0 msgr2=0x7f8fc807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:18.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.402+0000 7f8feb200640 1 --2- 172.21.15.39:0/1482482102 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8fc8078bc0 0x7f8fc807b0d0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc007fd0 tx=0x7f8fdc009040 comp rx=0 tx=0).stop 2024-09-21T07:46:18.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.402+0000 7f8feb200640 1 -- 172.21.15.39:0/1482482102 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fec0d04b0 msgr2=0x7f8fec1f8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:18.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.402+0000 7f8feb200640 1 --2- 172.21.15.39:0/1482482102 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fec0d04b0 0x7f8fec1f8900 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4034040 tx=0x7f8fd40038a0 comp rx=0 tx=0).stop 2024-09-21T07:46:18.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.402+0000 7f8feb200640 1 -- 172.21.15.39:0/1482482102 shutdown_connections 2024-09-21T07:46:18.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.402+0000 7f8feb200640 1 --2- 172.21.15.39:0/1482482102 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8fc8078bc0 0x7f8fc807b0d0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:18.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.402+0000 7f8feb200640 1 --2- 172.21.15.39:0/1482482102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8fec15cc00 0x7f8fec1f8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:18.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.402+0000 7f8feb200640 1 --2- 172.21.15.39:0/1482482102 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fec0d04b0 0x7f8fec1f8900 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:18.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.402+0000 7f8feb200640 1 --2- 172.21.15.39:0/1482482102 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8fec0cfa20 0x7f8fec13f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:18.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.402+0000 7f8feb200640 1 -- 172.21.15.39:0/1482482102 >> 172.21.15.39:0/1482482102 conn(0x7f8fec06cfc0 msgr2=0x7f8fec06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:18.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.402+0000 7f8feb200640 1 -- 172.21.15.39:0/1482482102 shutdown_connections 2024-09-21T07:46:18.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.402+0000 7f8feb200640 1 -- 172.21.15.39:0/1482482102 wait complete. 2024-09-21T07:46:18.428 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2024-09-21T07:46:18.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.657+0000 7fb121200640 1 -- 172.21.15.39:0/87341056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb100005350 msgr2=0x7fb10000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:18.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.657+0000 7fb121200640 1 --2- 172.21.15.39:0/87341056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb100005350 0x7fb10000b9c0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fb11400c610 tx=0x7fb100005830 comp rx=0 tx=0).stop 2024-09-21T07:46:18.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.657+0000 7fb121200640 1 -- 172.21.15.39:0/87341056 shutdown_connections 2024-09-21T07:46:18.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.657+0000 7fb121200640 1 --2- 172.21.15.39:0/87341056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb100005350 0x7fb10000b9c0 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:18.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.657+0000 7fb121200640 1 -- 172.21.15.39:0/87341056 >> 172.21.15.39:0/87341056 conn(0x7fb11c0b1f90 msgr2=0x7fb11c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:18.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.657+0000 7fb121200640 1 -- 172.21.15.39:0/87341056 shutdown_connections 2024-09-21T07:46:18.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.657+0000 7fb121200640 1 -- 172.21.15.39:0/87341056 wait complete. 2024-09-21T07:46:18.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.660+0000 7fb121200640 1 Processor -- start 2024-09-21T07:46:18.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.660+0000 7fb121200640 1 -- start start 2024-09-21T07:46:18.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.661+0000 7fb121200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb100005350 0x7fb11c1e4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:18.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.661+0000 7fb121200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb11c0c49e0 0x7fb11c0cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:18.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.661+0000 7fb121200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb11c0cc3f0 0x7fb11c1df9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:18.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.661+0000 7fb121200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb11c0cae80 con 0x7fb11c0c49e0 2024-09-21T07:46:18.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.661+0000 7fb121200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb11c0cad00 con 0x7fb11c0cc3f0 2024-09-21T07:46:18.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.661+0000 7fb121200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb11c074630 con 0x7fb100005350 2024-09-21T07:46:18.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.661+0000 7fb112a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb11c0c49e0 0x7fb11c0cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:18.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.662+0000 7fb112a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb11c0c49e0 0x7fb11c0cbeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53130/0 (socket says 172.21.15.39:53130) 2024-09-21T07:46:18.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.662+0000 7fb112a00640 1 -- 172.21.15.39:0/2681184441 learned_addr learned my addr 172.21.15.39:0/2681184441 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:18.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.662+0000 7fb11b400640 1 --2- 172.21.15.39:0/2681184441 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb100005350 0x7fb11c1e4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:18.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.662+0000 7fb11be00640 1 --2- 172.21.15.39:0/2681184441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb11c0cc3f0 0x7fb11c1df9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:18.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.662+0000 7fb112a00640 1 -- 172.21.15.39:0/2681184441 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb100005350 msgr2=0x7fb11c1e4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:18.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.662+0000 7fb112a00640 1 --2- 172.21.15.39:0/2681184441 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb100005350 0x7fb11c1e4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:18.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.662+0000 7fb112a00640 1 -- 172.21.15.39:0/2681184441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb11c0cc3f0 msgr2=0x7fb11c1df9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:18.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.662+0000 7fb112a00640 1 --2- 172.21.15.39:0/2681184441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb11c0cc3f0 0x7fb11c1df9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:18.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.662+0000 7fb112a00640 1 -- 172.21.15.39:0/2681184441 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb11c0c63d0 con 0x7fb11c0c49e0 2024-09-21T07:46:18.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.662+0000 7fb11be00640 1 --2- 172.21.15.39:0/2681184441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb11c0cc3f0 0x7fb11c1df9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:18.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.662+0000 7fb112a00640 1 --2- 172.21.15.39:0/2681184441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb11c0c49e0 0x7fb11c0cbeb0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fb104011ab0 tx=0x7fb104011eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:18.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.662+0000 7fb113e00640 1 -- 172.21.15.39:0/2681184441 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10401e070 con 0x7fb11c0c49e0 2024-09-21T07:46:18.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.663+0000 7fb113e00640 1 -- 172.21.15.39:0/2681184441 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb10400fbb0 con 0x7fb11c0c49e0 2024-09-21T07:46:18.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.663+0000 7fb113e00640 1 -- 172.21.15.39:0/2681184441 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb104019410 con 0x7fb11c0c49e0 2024-09-21T07:46:18.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.663+0000 7fb121200640 1 -- 172.21.15.39:0/2681184441 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb11c0c66b0 con 0x7fb11c0c49e0 2024-09-21T07:46:18.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.663+0000 7fb121200640 1 -- 172.21.15.39:0/2681184441 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb11c0c6f50 con 0x7fb11c0c49e0 2024-09-21T07:46:18.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.664+0000 7fb121200640 1 -- 172.21.15.39:0/2681184441 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb11c074b90 con 0x7fb11c0c49e0 2024-09-21T07:46:18.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.665+0000 7fb113e00640 1 -- 172.21.15.39:0/2681184441 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb104021070 con 0x7fb11c0c49e0 2024-09-21T07:46:18.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.666+0000 7fb113e00640 1 --2- 172.21.15.39:0/2681184441 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb0f8078b90 0x7fb0f807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:18.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.666+0000 7fb113e00640 1 -- 172.21.15.39:0/2681184441 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 6122+0+0 (secure 0 0 0) 0x7fb10409b640 con 0x7fb11c0c49e0 2024-09-21T07:46:18.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.666+0000 7fb11b400640 1 --2- 172.21.15.39:0/2681184441 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb0f8078b90 0x7fb0f807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:18.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.667+0000 7fb11b400640 1 --2- 172.21.15.39:0/2681184441 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb0f8078b90 0x7fb0f807b0a0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fb10c006680 tx=0x7fb10c006b70 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:18.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.671+0000 7fb113e00640 1 -- 172.21.15.39:0/2681184441 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb104063280 con 0x7fb11c0c49e0 2024-09-21T07:46:18.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.864+0000 7fb121200640 1 -- 172.21.15.39:0/2681184441 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) -- 0x7fb11c0c7810 con 0x7fb11c0c49e0 2024-09-21T07:46:18.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.865+0000 7fb113e00640 1 -- 172.21.15.39:0/2681184441 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v167) ==== 107+0+11 (secure 0 0 0) 0x7fb104067cb0 con 0x7fb11c0c49e0 2024-09-21T07:46:18.867 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-21T07:46:18.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.868+0000 7fb121200640 1 -- 172.21.15.39:0/2681184441 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb0f8078b90 msgr2=0x7fb0f807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:18.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.868+0000 7fb121200640 1 --2- 172.21.15.39:0/2681184441 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb0f8078b90 0x7fb0f807b0a0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fb10c006680 tx=0x7fb10c006b70 comp rx=0 tx=0).stop 2024-09-21T07:46:18.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.868+0000 7fb121200640 1 -- 172.21.15.39:0/2681184441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb11c0c49e0 msgr2=0x7fb11c0cbeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:18.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.868+0000 7fb121200640 1 --2- 172.21.15.39:0/2681184441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb11c0c49e0 0x7fb11c0cbeb0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fb104011ab0 tx=0x7fb104011eb0 comp rx=0 tx=0).stop 2024-09-21T07:46:18.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.868+0000 7fb121200640 1 -- 172.21.15.39:0/2681184441 shutdown_connections 2024-09-21T07:46:18.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.868+0000 7fb121200640 1 --2- 172.21.15.39:0/2681184441 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb0f8078b90 0x7fb0f807b0a0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:18.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.868+0000 7fb121200640 1 --2- 172.21.15.39:0/2681184441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb11c0cc3f0 0x7fb11c1df9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:18.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.868+0000 7fb121200640 1 --2- 172.21.15.39:0/2681184441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb11c0c49e0 0x7fb11c0cbeb0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:18.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.868+0000 7fb121200640 1 --2- 172.21.15.39:0/2681184441 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb100005350 0x7fb11c1e4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:18.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.868+0000 7fb121200640 1 -- 172.21.15.39:0/2681184441 >> 172.21.15.39:0/2681184441 conn(0x7fb11c0b1f90 msgr2=0x7fb11c0b3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:18.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.868+0000 7fb121200640 1 -- 172.21.15.39:0/2681184441 shutdown_connections 2024-09-21T07:46:18.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:18.868+0000 7fb121200640 1 -- 172.21.15.39:0/2681184441 wait complete. 2024-09-21T07:46:18.891 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_mds_max_snaps_per_dir_with_reduced_limit' 2024-09-21T07:46:19.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.117+0000 7ffa4b200640 1 -- 172.21.15.39:0/2266704581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa2c0051c0 msgr2=0x7ffa2c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:19.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.117+0000 7ffa4b200640 1 --2- 172.21.15.39:0/2266704581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa2c0051c0 0x7ffa2c00b9c0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7ffa40001dc0 tx=0x7ffa2c005830 comp rx=0 tx=0).stop 2024-09-21T07:46:19.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.118+0000 7ffa4b200640 1 -- 172.21.15.39:0/2266704581 shutdown_connections 2024-09-21T07:46:19.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.118+0000 7ffa4b200640 1 --2- 172.21.15.39:0/2266704581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa2c0051c0 0x7ffa2c00b9c0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:19.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.118+0000 7ffa4b200640 1 -- 172.21.15.39:0/2266704581 >> 172.21.15.39:0/2266704581 conn(0x7ffa4406cfc0 msgr2=0x7ffa4406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:19.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.118+0000 7ffa4b200640 1 -- 172.21.15.39:0/2266704581 shutdown_connections 2024-09-21T07:46:19.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.118+0000 7ffa4b200640 1 -- 172.21.15.39:0/2266704581 wait complete. 2024-09-21T07:46:19.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.120+0000 7ffa4b200640 1 Processor -- start 2024-09-21T07:46:19.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.120+0000 7ffa4b200640 1 -- start start 2024-09-21T07:46:19.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.122+0000 7ffa4b200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffa2c0051c0 0x7ffa441f7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:19.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.122+0000 7ffa4b200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa440b26c0 0x7ffa4415b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:19.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.122+0000 7ffa4b200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa4415bdb0 0x7ffa440afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:19.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.122+0000 7ffa4b200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4415d900 con 0x7ffa440b26c0 2024-09-21T07:46:19.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.122+0000 7ffa4b200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4415d780 con 0x7ffa4415bdb0 2024-09-21T07:46:19.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.122+0000 7ffa4b200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7ffa4415da80 con 0x7ffa2c0051c0 2024-09-21T07:46:19.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.122+0000 7ffa49400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa440b26c0 0x7ffa4415b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:19.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.122+0000 7ffa49400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa440b26c0 0x7ffa4415b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53144/0 (socket says 172.21.15.39:53144) 2024-09-21T07:46:19.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.122+0000 7ffa49400640 1 -- 172.21.15.39:0/1622317108 learned_addr learned my addr 172.21.15.39:0/1622317108 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:19.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.122+0000 7ffa49400640 1 -- 172.21.15.39:0/1622317108 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffa2c0051c0 msgr2=0x7ffa441f7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:19.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.122+0000 7ffa49e00640 1 --2- 172.21.15.39:0/1622317108 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffa2c0051c0 0x7ffa441f7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:19.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.122+0000 7ffa4a800640 1 --2- 172.21.15.39:0/1622317108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa4415bdb0 0x7ffa440afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:19.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.122+0000 7ffa49400640 1 --2- 172.21.15.39:0/1622317108 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffa2c0051c0 0x7ffa441f7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:19.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.122+0000 7ffa49400640 1 -- 172.21.15.39:0/1622317108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa4415bdb0 msgr2=0x7ffa440afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:19.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.122+0000 7ffa49400640 1 --2- 172.21.15.39:0/1622317108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa4415bdb0 0x7ffa440afdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:19.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.122+0000 7ffa49400640 1 -- 172.21.15.39:0/1622317108 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa441f7ca0 con 0x7ffa440b26c0 2024-09-21T07:46:19.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.122+0000 7ffa49e00640 1 --2- 172.21.15.39:0/1622317108 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffa2c0051c0 0x7ffa441f7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:46:19.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.122+0000 7ffa4a800640 1 --2- 172.21.15.39:0/1622317108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa4415bdb0 0x7ffa440afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:46:19.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.123+0000 7ffa49400640 1 --2- 172.21.15.39:0/1622317108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa440b26c0 0x7ffa4415b870 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7ffa34011ab0 tx=0x7ffa34011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:19.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.123+0000 7ffa3e000640 1 -- 172.21.15.39:0/1622317108 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3401e070 con 0x7ffa440b26c0 2024-09-21T07:46:19.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.123+0000 7ffa3e000640 1 -- 172.21.15.39:0/1622317108 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa3400fe70 con 0x7ffa440b26c0 2024-09-21T07:46:19.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.123+0000 7ffa4b200640 1 -- 172.21.15.39:0/1622317108 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa4415c1c0 con 0x7ffa440b26c0 2024-09-21T07:46:19.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.123+0000 7ffa3e000640 1 -- 172.21.15.39:0/1622317108 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa340042b0 con 0x7ffa440b26c0 2024-09-21T07:46:19.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.123+0000 7ffa4b200640 1 -- 172.21.15.39:0/1622317108 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa440a99d0 con 0x7ffa440b26c0 2024-09-21T07:46:19.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.124+0000 7ffa4b200640 1 -- 172.21.15.39:0/1622317108 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa440a9740 con 0x7ffa440b26c0 2024-09-21T07:46:19.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.125+0000 7ffa3e000640 1 -- 172.21.15.39:0/1622317108 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ffa34021070 con 0x7ffa440b26c0 2024-09-21T07:46:19.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.127+0000 7ffa3e000640 1 --2- 172.21.15.39:0/1622317108 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ffa28078b90 0x7ffa2807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:19.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.127+0000 7ffa3e000640 1 -- 172.21.15.39:0/1622317108 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 6122+0+0 (secure 0 0 0) 0x7ffa3409ba00 con 0x7ffa440b26c0 2024-09-21T07:46:19.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.127+0000 7ffa49e00640 1 --2- 172.21.15.39:0/1622317108 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ffa28078b90 0x7ffa2807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:19.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.128+0000 7ffa49e00640 1 --2- 172.21.15.39:0/1622317108 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ffa28078b90 0x7ffa2807b0a0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ffa38001050 tx=0x7ffa38001dc0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:19.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.131+0000 7ffa3e000640 1 -- 172.21.15.39:0/1622317108 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ffa4415c1c0 con 0x7ffa440b26c0 2024-09-21T07:46:19.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.402+0000 7ffa4b200640 1 -- 172.21.15.39:0/1622317108 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_mds_max_snaps_per_dir_with_reduced_limit"]} v 0) -- 0x7ffa440d0610 con 0x7ffa440b26c0 2024-09-21T07:46:19.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.533+0000 7ffa3e000640 1 -- 172.21.15.39:0/1622317108 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_mds_max_snaps_per_dir_with_reduced_limit"]}]=0 v635) ==== 169+0+0 (secure 0 0 0) 0x7ffa441f7ca0 con 0x7ffa440b26c0 2024-09-21T07:46:19.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.535+0000 7ffa4b200640 1 -- 172.21.15.39:0/1622317108 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ffa28078b90 msgr2=0x7ffa2807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:19.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.535+0000 7ffa4b200640 1 --2- 172.21.15.39:0/1622317108 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ffa28078b90 0x7ffa2807b0a0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ffa38001050 tx=0x7ffa38001dc0 comp rx=0 tx=0).stop 2024-09-21T07:46:19.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.535+0000 7ffa4b200640 1 -- 172.21.15.39:0/1622317108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa440b26c0 msgr2=0x7ffa4415b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:19.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.535+0000 7ffa4b200640 1 --2- 172.21.15.39:0/1622317108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa440b26c0 0x7ffa4415b870 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7ffa34011ab0 tx=0x7ffa34011f70 comp rx=0 tx=0).stop 2024-09-21T07:46:19.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.535+0000 7ffa4b200640 1 -- 172.21.15.39:0/1622317108 shutdown_connections 2024-09-21T07:46:19.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.535+0000 7ffa4b200640 1 --2- 172.21.15.39:0/1622317108 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ffa28078b90 0x7ffa2807b0a0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:19.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.535+0000 7ffa4b200640 1 --2- 172.21.15.39:0/1622317108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa4415bdb0 0x7ffa440afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:19.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.535+0000 7ffa4b200640 1 --2- 172.21.15.39:0/1622317108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa440b26c0 0x7ffa4415b870 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:19.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.535+0000 7ffa4b200640 1 --2- 172.21.15.39:0/1622317108 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ffa2c0051c0 0x7ffa441f7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:19.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.535+0000 7ffa4b200640 1 -- 172.21.15.39:0/1622317108 >> 172.21.15.39:0/1622317108 conn(0x7ffa4406cfc0 msgr2=0x7ffa441428f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:19.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.535+0000 7ffa4b200640 1 -- 172.21.15.39:0/1622317108 shutdown_connections 2024-09-21T07:46:19.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.535+0000 7ffa4b200640 1 -- 172.21.15.39:0/1622317108 wait complete. 2024-09-21T07:46:19.558 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T07:46:19.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.782+0000 7fe14a800640 1 -- 172.21.15.39:0/2880474696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1440b30a0 msgr2=0x7fe1440cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:19.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.782+0000 7fe14a800640 1 --2- 172.21.15.39:0/2880474696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1440b30a0 0x7fe1440cb8a0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fe14406d880 tx=0x7fe13402a4d0 comp rx=0 tx=0).stop 2024-09-21T07:46:19.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.782+0000 7fe14a800640 1 -- 172.21.15.39:0/2880474696 shutdown_connections 2024-09-21T07:46:19.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.782+0000 7fe14a800640 1 --2- 172.21.15.39:0/2880474696 >> v2:172.21.15.148:3300/0 conn(0x7fe1440ccc60 0x7fe1440cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:19.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.782+0000 7fe14a800640 1 --2- 172.21.15.39:0/2880474696 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe14415d830 0x7fe144156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:19.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.782+0000 7fe14a800640 1 --2- 172.21.15.39:0/2880474696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1440b30a0 0x7fe1440cb8a0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:19.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.782+0000 7fe14a800640 1 -- 172.21.15.39:0/2880474696 >> 172.21.15.39:0/2880474696 conn(0x7fe1440b2460 msgr2=0x7fe14406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:19.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.782+0000 7fe14a800640 1 -- 172.21.15.39:0/2880474696 shutdown_connections 2024-09-21T07:46:19.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.782+0000 7fe14a800640 1 -- 172.21.15.39:0/2880474696 wait complete. 2024-09-21T07:46:19.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.784+0000 7fe14a800640 1 Processor -- start 2024-09-21T07:46:19.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.784+0000 7fe14a800640 1 -- start start 2024-09-21T07:46:19.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.786+0000 7fe14a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1440b30a0 0x7fe1441f97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:19.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.786+0000 7fe14a800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1440ccc60 0x7fe14409d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:19.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.786+0000 7fe14a800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe14415d830 0x7fe1441f5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:19.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.786+0000 7fe14a800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe1440af200 con 0x7fe1440b30a0 2024-09-21T07:46:19.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.786+0000 7fe14a800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe1440af080 con 0x7fe1440ccc60 2024-09-21T07:46:19.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.786+0000 7fe14a800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe1440af380 con 0x7fe14415d830 2024-09-21T07:46:19.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.786+0000 7fe143400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1440b30a0 0x7fe1441f97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:19.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.786+0000 7fe143400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1440b30a0 0x7fe1441f97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53166/0 (socket says 172.21.15.39:53166) 2024-09-21T07:46:19.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.786+0000 7fe143400640 1 -- 172.21.15.39:0/1408824936 learned_addr learned my addr 172.21.15.39:0/1408824936 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:19.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.786+0000 7fe13be00640 1 --2- 172.21.15.39:0/1408824936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1440ccc60 0x7fe14409d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:19.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.787+0000 7fe143e00640 1 --2- 172.21.15.39:0/1408824936 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe14415d830 0x7fe1441f5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:19.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.787+0000 7fe143400640 1 -- 172.21.15.39:0/1408824936 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe14415d830 msgr2=0x7fe1441f5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:19.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.787+0000 7fe143400640 1 --2- 172.21.15.39:0/1408824936 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe14415d830 0x7fe1441f5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:19.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.787+0000 7fe143400640 1 -- 172.21.15.39:0/1408824936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1440ccc60 msgr2=0x7fe14409d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:19.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.787+0000 7fe143400640 1 --2- 172.21.15.39:0/1408824936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1440ccc60 0x7fe14409d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:19.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.787+0000 7fe143400640 1 -- 172.21.15.39:0/1408824936 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1441f65d0 con 0x7fe1440b30a0 2024-09-21T07:46:19.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.787+0000 7fe13be00640 1 --2- 172.21.15.39:0/1408824936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1440ccc60 0x7fe14409d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:19.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.787+0000 7fe143400640 1 --2- 172.21.15.39:0/1408824936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1440b30a0 0x7fe1441f97f0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fe134034040 tx=0x7fe134003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:19.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.787+0000 7fe13b400640 1 -- 172.21.15.39:0/1408824936 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13403e030 con 0x7fe1440b30a0 2024-09-21T07:46:19.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.787+0000 7fe13b400640 1 -- 172.21.15.39:0/1408824936 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe134003e00 con 0x7fe1440b30a0 2024-09-21T07:46:19.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.787+0000 7fe13b400640 1 -- 172.21.15.39:0/1408824936 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe134035040 con 0x7fe1440b30a0 2024-09-21T07:46:19.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.787+0000 7fe14a800640 1 -- 172.21.15.39:0/1408824936 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1441f62f0 con 0x7fe1440b30a0 2024-09-21T07:46:19.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.788+0000 7fe14a800640 1 -- 172.21.15.39:0/1408824936 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1441f8ff0 con 0x7fe1440b30a0 2024-09-21T07:46:19.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.789+0000 7fe14a800640 1 -- 172.21.15.39:0/1408824936 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1440af870 con 0x7fe1440b30a0 2024-09-21T07:46:19.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.790+0000 7fe13b400640 1 -- 172.21.15.39:0/1408824936 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe134041030 con 0x7fe1440b30a0 2024-09-21T07:46:19.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.791+0000 7fe13b400640 1 --2- 172.21.15.39:0/1408824936 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe120078b90 0x7fe12007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:19.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.791+0000 7fe13b400640 1 -- 172.21.15.39:0/1408824936 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 6122+0+0 (secure 0 0 0) 0x7fe1340bb690 con 0x7fe1440b30a0 2024-09-21T07:46:19.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.791+0000 7fe13be00640 1 --2- 172.21.15.39:0/1408824936 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe120078b90 0x7fe12007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:19.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.792+0000 7fe13be00640 1 --2- 172.21.15.39:0/1408824936 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe120078b90 0x7fe12007b0a0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fe1440d0160 tx=0x7fe12c005f10 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:19.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.795+0000 7fe13b400640 1 -- 172.21.15.39:0/1408824936 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe1340839b0 con 0x7fe1440b30a0 2024-09-21T07:46:19.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.992+0000 7fe14a800640 1 -- 172.21.15.39:0/1408824936 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fe14415d280 con 0x7fe1440b30a0 2024-09-21T07:46:19.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.992+0000 7fe13b400640 1 -- 172.21.15.39:0/1408824936 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v46) ==== 59+0+2294 (secure 0 0 0) 0x7fe1340832b0 con 0x7fe1440b30a0 2024-09-21T07:46:19.995 INFO:teuthology.orchestra.run.smithi039.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T07:46:19.995 INFO:teuthology.orchestra.run.smithi039.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T07:46:19.995 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced client_mount_timeout 30 2024-09-21T07:46:19.995 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_client 20/20 2024-09-21T07:46:19.995 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_mgr 20/20 2024-09-21T07:46:19.995 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_ms 1/1 2024-09-21T07:46:19.995 INFO:teuthology.orchestra.run.smithi039.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T07:46:19.996 INFO:teuthology.orchestra.run.smithi039.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T07:46:19.996 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3421.881759 2024-09-21T07:46:19.996 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3520.252978 2024-09-21T07:46:19.996 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3511.183316 2024-09-21T07:46:19.996 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3829.631847 2024-09-21T07:46:19.996 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 5474.545872 2024-09-21T07:46:19.996 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4013.628559 2024-09-21T07:46:19.996 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4230.187288 2024-09-21T07:46:19.997 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4148.983107 2024-09-21T07:46:19.997 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_mds 20/20 2024-09-21T07:46:19.997 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T07:46:19.997 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_ms 2/2 2024-09-21T07:46:19.997 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_debug_frag true 2024-09-21T07:46:19.997 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_debug_scatterstat true 2024-09-21T07:46:19.997 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T07:46:19.997 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_verify_scatter true 2024-09-21T07:46:19.997 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T07:46:19.998 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T07:46:19.998 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T07:46:19.998 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced client_mount_timeout 600 2024-09-21T07:46:19.998 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced debug_client 20/20 2024-09-21T07:46:19.998 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced debug_ms 1/1 2024-09-21T07:46:19.998 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T07:46:19.998 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T07:46:19.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.995+0000 7fe14a800640 1 -- 172.21.15.39:0/1408824936 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe120078b90 msgr2=0x7fe12007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:20.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.995+0000 7fe14a800640 1 --2- 172.21.15.39:0/1408824936 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe120078b90 0x7fe12007b0a0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fe1440d0160 tx=0x7fe12c005f10 comp rx=0 tx=0).stop 2024-09-21T07:46:20.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.995+0000 7fe14a800640 1 -- 172.21.15.39:0/1408824936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1440b30a0 msgr2=0x7fe1441f97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:20.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.995+0000 7fe14a800640 1 --2- 172.21.15.39:0/1408824936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1440b30a0 0x7fe1441f97f0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fe134034040 tx=0x7fe134003910 comp rx=0 tx=0).stop 2024-09-21T07:46:20.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.995+0000 7fe14a800640 1 -- 172.21.15.39:0/1408824936 shutdown_connections 2024-09-21T07:46:20.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.995+0000 7fe14a800640 1 --2- 172.21.15.39:0/1408824936 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe120078b90 0x7fe12007b0a0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:20.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.995+0000 7fe14a800640 1 --2- 172.21.15.39:0/1408824936 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe14415d830 0x7fe1441f5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:20.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.995+0000 7fe14a800640 1 --2- 172.21.15.39:0/1408824936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1440ccc60 0x7fe14409d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:20.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.995+0000 7fe14a800640 1 --2- 172.21.15.39:0/1408824936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1440b30a0 0x7fe1441f97f0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:20.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.995+0000 7fe14a800640 1 -- 172.21.15.39:0/1408824936 >> 172.21.15.39:0/1408824936 conn(0x7fe1440b2460 msgr2=0x7fe14406fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:20.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.995+0000 7fe14a800640 1 -- 172.21.15.39:0/1408824936 shutdown_connections 2024-09-21T07:46:20.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:19.995+0000 7fe14a800640 1 -- 172.21.15.39:0/1408824936 wait complete. 2024-09-21T07:46:20.019 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T07:46:20.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.247+0000 7f5940c00640 1 -- 172.21.15.39:0/608689546 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f593c0d0630 msgr2=0x7f593c0d0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:20.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.247+0000 7f5940c00640 1 --2- 172.21.15.39:0/608689546 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f593c0d0630 0x7f593c0d0a40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5934005fd0 tx=0x7f593402aa90 comp rx=0 tx=0).stop 2024-09-21T07:46:20.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.247+0000 7f5940c00640 1 -- 172.21.15.39:0/608689546 shutdown_connections 2024-09-21T07:46:20.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.247+0000 7f5940c00640 1 --2- 172.21.15.39:0/608689546 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f593c0d0630 0x7f593c0d0a40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:20.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.247+0000 7f5940c00640 1 --2- 172.21.15.39:0/608689546 >> v2:172.21.15.148:3300/0 conn(0x7f593c073dd0 0x7f593c15d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:20.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.247+0000 7f5940c00640 1 -- 172.21.15.39:0/608689546 >> 172.21.15.39:0/608689546 conn(0x7f593c06cfc0 msgr2=0x7f593c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:20.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.247+0000 7f5940c00640 1 -- 172.21.15.39:0/608689546 shutdown_connections 2024-09-21T07:46:20.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.247+0000 7f5940c00640 1 -- 172.21.15.39:0/608689546 wait complete. 2024-09-21T07:46:20.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.249+0000 7f5940c00640 1 Processor -- start 2024-09-21T07:46:20.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.250+0000 7f5940c00640 1 -- start start 2024-09-21T07:46:20.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.251+0000 7f5940c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f593c073dd0 0x7f593c155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:20.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.251+0000 7f5940c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f593c0d0630 0x7f593c1e92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:20.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.251+0000 7f5940c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f593c15daf0 0x7f593c1e9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:20.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.251+0000 7f5940c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f593c0cd220 con 0x7f593c15daf0 2024-09-21T07:46:20.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.251+0000 7f5940c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f593c0cd0a0 con 0x7f593c073dd0 2024-09-21T07:46:20.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.251+0000 7f5940c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f593c0cd3a0 con 0x7f593c0d0630 2024-09-21T07:46:20.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.251+0000 7f593a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f593c15daf0 0x7f593c1e9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:20.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.251+0000 7f593a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f593c15daf0 0x7f593c1e9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53168/0 (socket says 172.21.15.39:53168) 2024-09-21T07:46:20.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.251+0000 7f593a800640 1 -- 172.21.15.39:0/1433411138 learned_addr learned my addr 172.21.15.39:0/1433411138 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:20.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.251+0000 7f5939e00640 1 --2- 172.21.15.39:0/1433411138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f593c073dd0 0x7f593c155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:20.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.251+0000 7f5931400640 1 --2- 172.21.15.39:0/1433411138 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f593c0d0630 0x7f593c1e92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:20.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.252+0000 7f593a800640 1 -- 172.21.15.39:0/1433411138 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f593c0d0630 msgr2=0x7f593c1e92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:20.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.252+0000 7f593a800640 1 --2- 172.21.15.39:0/1433411138 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f593c0d0630 0x7f593c1e92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:20.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.252+0000 7f593a800640 1 -- 172.21.15.39:0/1433411138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f593c073dd0 msgr2=0x7f593c155bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:20.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.252+0000 7f593a800640 1 --2- 172.21.15.39:0/1433411138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f593c073dd0 0x7f593c155bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:20.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.252+0000 7f593a800640 1 -- 172.21.15.39:0/1433411138 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f593c0a94d0 con 0x7f593c15daf0 2024-09-21T07:46:20.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.252+0000 7f593a800640 1 --2- 172.21.15.39:0/1433411138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f593c15daf0 0x7f593c1e9810 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f5934009d90 tx=0x7f5934003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:20.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.252+0000 7f5932a00640 1 -- 172.21.15.39:0/1433411138 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5934044070 con 0x7f593c15daf0 2024-09-21T07:46:20.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.253+0000 7f5932a00640 1 -- 172.21.15.39:0/1433411138 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5934004040 con 0x7f593c15daf0 2024-09-21T07:46:20.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.253+0000 7f5940c00640 1 -- 172.21.15.39:0/1433411138 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f593c1f60e0 con 0x7f593c15daf0 2024-09-21T07:46:20.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.253+0000 7f5932a00640 1 -- 172.21.15.39:0/1433411138 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5934004450 con 0x7f593c15daf0 2024-09-21T07:46:20.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.253+0000 7f5940c00640 1 -- 172.21.15.39:0/1433411138 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f593c1f66a0 con 0x7f593c15daf0 2024-09-21T07:46:20.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.254+0000 7f5940c00640 1 -- 172.21.15.39:0/1433411138 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f593c0a96a0 con 0x7f593c15daf0 2024-09-21T07:46:20.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.255+0000 7f5932a00640 1 -- 172.21.15.39:0/1433411138 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5934047070 con 0x7f593c15daf0 2024-09-21T07:46:20.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.256+0000 7f5932a00640 1 --2- 172.21.15.39:0/1433411138 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5918078b90 0x7f591807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:20.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.256+0000 7f5932a00640 1 -- 172.21.15.39:0/1433411138 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 6122+0+0 (secure 0 0 0) 0x7f59340bbc80 con 0x7f593c15daf0 2024-09-21T07:46:20.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.257+0000 7f5939e00640 1 --2- 172.21.15.39:0/1433411138 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5918078b90 0x7f591807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:20.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.257+0000 7f5939e00640 1 --2- 172.21.15.39:0/1433411138 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5918078b90 0x7f591807b0a0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f592c0095f0 tx=0x7f592c005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:20.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.261+0000 7f5932a00640 1 -- 172.21.15.39:0/1433411138 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5934032030 con 0x7f593c15daf0 2024-09-21T07:46:20.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.457+0000 7f5940c00640 1 -- 172.21.15.39:0/1433411138 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f593c0cd8c0 con 0x7f593c15daf0 2024-09-21T07:46:20.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.463+0000 7f5932a00640 1 -- 172.21.15.39:0/1433411138 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v47)=0 v47) ==== 122+0+0 (secure 0 0 0) 0x7f59340838e0 con 0x7f593c15daf0 2024-09-21T07:46:20.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.465+0000 7f5940c00640 1 -- 172.21.15.39:0/1433411138 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5918078b90 msgr2=0x7f591807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:20.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.465+0000 7f5940c00640 1 --2- 172.21.15.39:0/1433411138 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5918078b90 0x7f591807b0a0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f592c0095f0 tx=0x7f592c005f90 comp rx=0 tx=0).stop 2024-09-21T07:46:20.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.465+0000 7f5940c00640 1 -- 172.21.15.39:0/1433411138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f593c15daf0 msgr2=0x7f593c1e9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:20.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.465+0000 7f5940c00640 1 --2- 172.21.15.39:0/1433411138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f593c15daf0 0x7f593c1e9810 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f5934009d90 tx=0x7f5934003ba0 comp rx=0 tx=0).stop 2024-09-21T07:46:20.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.465+0000 7f5940c00640 1 -- 172.21.15.39:0/1433411138 shutdown_connections 2024-09-21T07:46:20.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.465+0000 7f5940c00640 1 --2- 172.21.15.39:0/1433411138 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5918078b90 0x7f591807b0a0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:20.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.465+0000 7f5940c00640 1 --2- 172.21.15.39:0/1433411138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f593c15daf0 0x7f593c1e9810 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:20.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.465+0000 7f5940c00640 1 --2- 172.21.15.39:0/1433411138 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f593c0d0630 0x7f593c1e92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:20.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.465+0000 7f5940c00640 1 --2- 172.21.15.39:0/1433411138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f593c073dd0 0x7f593c155bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:20.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.465+0000 7f5940c00640 1 -- 172.21.15.39:0/1433411138 >> 172.21.15.39:0/1433411138 conn(0x7f593c06cfc0 msgr2=0x7f593c0b0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:20.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.465+0000 7f5940c00640 1 -- 172.21.15.39:0/1433411138 shutdown_connections 2024-09-21T07:46:20.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:20.465+0000 7f5940c00640 1 -- 172.21.15.39:0/1433411138 wait complete. 2024-09-21T07:46:20.488 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:46:20.488 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:46:20.577 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:46:20.664 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:46:20.665 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:46:20.665 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:46:20.700 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:46:20.700 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:46:20.700 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:46:20.737 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:46:20.737 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:46:20.737 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:46:20.776 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:46:20.776 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:46:20.776 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:46:20.814 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:46:20.814 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T07:46:20.814 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:46:21.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.043+0000 7f618e400640 1 -- 172.21.15.39:0/2795880211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f616c004800 msgr2=0x7f616c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:21.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.043+0000 7f618e400640 1 --2- 172.21.15.39:0/2795880211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f616c004800 0x7f616c004c10 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f617000c310 tx=0x7f616c005830 comp rx=0 tx=0).stop 2024-09-21T07:46:21.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.043+0000 7f618e400640 1 -- 172.21.15.39:0/2795880211 shutdown_connections 2024-09-21T07:46:21.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.043+0000 7f618e400640 1 --2- 172.21.15.39:0/2795880211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f616c004800 0x7f616c004c10 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.043+0000 7f618e400640 1 -- 172.21.15.39:0/2795880211 >> 172.21.15.39:0/2795880211 conn(0x7f618809d7b0 msgr2=0x7f61880d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:21.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.043+0000 7f618e400640 1 -- 172.21.15.39:0/2795880211 shutdown_connections 2024-09-21T07:46:21.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.043+0000 7f618e400640 1 -- 172.21.15.39:0/2795880211 wait complete. 2024-09-21T07:46:21.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.045+0000 7f618e400640 1 Processor -- start 2024-09-21T07:46:21.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.045+0000 7f618e400640 1 -- start start 2024-09-21T07:46:21.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.047+0000 7f618e400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f616c004800 0x7f61880cc020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:21.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.047+0000 7f618e400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61880cdc40 0x7f61880ce050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:21.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.047+0000 7f618e400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f618815abb0 0x7f618815afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:21.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.047+0000 7f618e400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f61880d0e70 con 0x7f61880cdc40 2024-09-21T07:46:21.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.047+0000 7f618e400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f61880d2240 con 0x7f616c004800 2024-09-21T07:46:21.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.047+0000 7f618e400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f61880d0ff0 con 0x7f618815abb0 2024-09-21T07:46:21.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.047+0000 7f617fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61880cdc40 0x7f61880ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:21.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.047+0000 7f617fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61880cdc40 0x7f61880ce050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53186/0 (socket says 172.21.15.39:53186) 2024-09-21T07:46:21.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.047+0000 7f617fe00640 1 -- 172.21.15.39:0/788207345 learned_addr learned my addr 172.21.15.39:0/788207345 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:21.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.047+0000 7f618d000640 1 --2- 172.21.15.39:0/788207345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f616c004800 0x7f61880cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:21.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.047+0000 7f618da00640 1 --2- 172.21.15.39:0/788207345 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f618815abb0 0x7f618815afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:21.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.047+0000 7f617fe00640 1 -- 172.21.15.39:0/788207345 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f618815abb0 msgr2=0x7f618815afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:21.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.047+0000 7f617fe00640 1 --2- 172.21.15.39:0/788207345 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f618815abb0 0x7f618815afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.048+0000 7f617fe00640 1 -- 172.21.15.39:0/788207345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f616c004800 msgr2=0x7f61880cc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:21.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.048+0000 7f617fe00640 1 --2- 172.21.15.39:0/788207345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f616c004800 0x7f61880cc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.048+0000 7f617fe00640 1 -- 172.21.15.39:0/788207345 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f618815b500 con 0x7f61880cdc40 2024-09-21T07:46:21.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.048+0000 7f617fe00640 1 --2- 172.21.15.39:0/788207345 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61880cdc40 0x7f61880ce050 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f617002b160 tx=0x7f6170001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:21.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.048+0000 7f6185600640 1 -- 172.21.15.39:0/788207345 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6170031040 con 0x7f61880cdc40 2024-09-21T07:46:21.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.048+0000 7f6185600640 1 -- 172.21.15.39:0/788207345 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61700065e0 con 0x7f61880cdc40 2024-09-21T07:46:21.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.048+0000 7f618e400640 1 -- 172.21.15.39:0/788207345 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61881f61a0 con 0x7f61880cdc40 2024-09-21T07:46:21.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.048+0000 7f6185600640 1 -- 172.21.15.39:0/788207345 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6170003a60 con 0x7f61880cdc40 2024-09-21T07:46:21.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.048+0000 7f618e400640 1 -- 172.21.15.39:0/788207345 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61881f8ca0 con 0x7f61880cdc40 2024-09-21T07:46:21.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.049+0000 7f618e400640 1 -- 172.21.15.39:0/788207345 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61880ce670 con 0x7f61880cdc40 2024-09-21T07:46:21.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.051+0000 7f6185600640 1 -- 172.21.15.39:0/788207345 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f617003e030 con 0x7f61880cdc40 2024-09-21T07:46:21.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.052+0000 7f6185600640 1 --2- 172.21.15.39:0/788207345 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6164078b90 0x7f616407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:21.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.052+0000 7f6185600640 1 -- 172.21.15.39:0/788207345 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 6122+0+0 (secure 0 0 0) 0x7f61700baba0 con 0x7f61880cdc40 2024-09-21T07:46:21.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.053+0000 7f618d000640 1 --2- 172.21.15.39:0/788207345 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6164078b90 0x7f616407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:21.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.053+0000 7f618d000640 1 --2- 172.21.15.39:0/788207345 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6164078b90 0x7f616407b0a0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f6188052910 tx=0x7f6178006bc0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:21.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.057+0000 7f6185600640 1 -- 172.21.15.39:0/788207345 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f618815b500 con 0x7f61880cdc40 2024-09-21T07:46:21.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.288+0000 7f618e400640 1 -- 172.21.15.39:0/788207345 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61880d14f0 con 0x7f61880cdc40 2024-09-21T07:46:21.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.289+0000 7f6185600640 1 -- 172.21.15.39:0/788207345 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 183 v183) ==== 95+0+3916 (secure 0 0 0) 0x7f6170082870 con 0x7f61880cdc40 2024-09-21T07:46:21.291 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:46:21.291 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":183,"btime":"2024-09-21T07:46:12:286624+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7310,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2823960253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2823960253},{"type":"v1","addr":"172.21.15.39:6835","nonce":2823960253}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":176},{"gid":8155,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/603157278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":603157278},{"type":"v1","addr":"172.21.15.148:6836","nonce":603157278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":175},{"gid":8158,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2610135999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":2610135999},{"type":"v1","addr":"172.21.15.148:6837","nonce":2610135999}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":176},{"gid":9078,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/375753346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":375753346},{"type":"v1","addr":"172.21.15.39:6834","nonce":375753346}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":183}],"filesystems":[]} 2024-09-21T07:46:21.291 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 183 2024-09-21T07:46:21.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.291+0000 7f618e400640 1 -- 172.21.15.39:0/788207345 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6164078b90 msgr2=0x7f616407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:21.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.291+0000 7f618e400640 1 --2- 172.21.15.39:0/788207345 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6164078b90 0x7f616407b0a0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f6188052910 tx=0x7f6178006bc0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.291+0000 7f618e400640 1 -- 172.21.15.39:0/788207345 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61880cdc40 msgr2=0x7f61880ce050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:21.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.291+0000 7f618e400640 1 --2- 172.21.15.39:0/788207345 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61880cdc40 0x7f61880ce050 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f617002b160 tx=0x7f6170001d80 comp rx=0 tx=0).stop 2024-09-21T07:46:21.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.291+0000 7f618e400640 1 -- 172.21.15.39:0/788207345 shutdown_connections 2024-09-21T07:46:21.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.291+0000 7f618e400640 1 --2- 172.21.15.39:0/788207345 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6164078b90 0x7f616407b0a0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.291+0000 7f618e400640 1 --2- 172.21.15.39:0/788207345 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f618815abb0 0x7f618815afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.291+0000 7f618e400640 1 --2- 172.21.15.39:0/788207345 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61880cdc40 0x7f61880ce050 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.291+0000 7f618e400640 1 --2- 172.21.15.39:0/788207345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f616c004800 0x7f61880cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.291+0000 7f618e400640 1 -- 172.21.15.39:0/788207345 >> 172.21.15.39:0/788207345 conn(0x7f618809d7b0 msgr2=0x7f61881585a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:21.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.291+0000 7f618e400640 1 -- 172.21.15.39:0/788207345 shutdown_connections 2024-09-21T07:46:21.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.291+0000 7f618e400640 1 -- 172.21.15.39:0/788207345 wait complete. 2024-09-21T07:46:21.318 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T07:46:21.318 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T07:46:21.318 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T07:46:21.318 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:46:21.318 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T07:46:21.318 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T07:46:21.319 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T07:46:21.319 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:46:21.319 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T07:46:21.319 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T07:46:21.319 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T07:46:21.319 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:46:21.319 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T07:46:21.319 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T07:46:21.320 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T07:46:21.320 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:46:21.341 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T07:46:21.341 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T07:46:21.343 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T07:46:21.343 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T07:46:21.344 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T07:46:21.344 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T07:46:21.345 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T07:46:21.345 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T07:46:21.386 INFO:tasks.ceph.mds.a:Started 2024-09-21T07:46:21.386 DEBUG:teuthology.parallel:result is None 2024-09-21T07:46:21.389 INFO:tasks.ceph.mds.c:Started 2024-09-21T07:46:21.389 DEBUG:teuthology.parallel:result is None 2024-09-21T07:46:21.390 INFO:tasks.ceph.mds.d:Started 2024-09-21T07:46:21.390 DEBUG:teuthology.parallel:result is None 2024-09-21T07:46:21.392 INFO:tasks.ceph.mds.b:Started 2024-09-21T07:46:21.392 DEBUG:teuthology.parallel:result is None 2024-09-21T07:46:21.392 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T07:46:21.585 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-21T07:46:21.587 INFO:tasks.ceph.mds.c.smithi039.stdout:starting mds.c at 2024-09-21T07:46:21.588 INFO:tasks.ceph.mds.d.smithi148.stdout:starting mds.d at 2024-09-21T07:46:21.588 INFO:tasks.ceph.mds.b.smithi148.stdout:starting mds.b at 2024-09-21T07:46:21.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.627+0000 7f2ff6200640 1 -- 172.21.15.39:0/4118154645 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ff00744b0 msgr2=0x7f2ff015fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:21.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.627+0000 7f2ff6200640 1 --2- 172.21.15.39:0/4118154645 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ff00744b0 0x7f2ff015fb10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc005fd0 tx=0x7f2fdc02a540 comp rx=0 tx=0).stop 2024-09-21T07:46:21.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.627+0000 7f2ff6200640 1 -- 172.21.15.39:0/4118154645 shutdown_connections 2024-09-21T07:46:21.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.627+0000 7f2ff6200640 1 --2- 172.21.15.39:0/4118154645 >> v2:172.21.15.148:3300/0 conn(0x7f2ff00b4cb0 0x7f2ff00b50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.627+0000 7f2ff6200640 1 --2- 172.21.15.39:0/4118154645 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ff00744b0 0x7f2ff015fb10 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.627+0000 7f2ff6200640 1 -- 172.21.15.39:0/4118154645 >> 172.21.15.39:0/4118154645 conn(0x7f2ff00d1450 msgr2=0x7f2ff0073690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:21.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.627+0000 7f2ff6200640 1 -- 172.21.15.39:0/4118154645 shutdown_connections 2024-09-21T07:46:21.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.628+0000 7f2ff6200640 1 -- 172.21.15.39:0/4118154645 wait complete. 2024-09-21T07:46:21.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.630+0000 7f2ff6200640 1 Processor -- start 2024-09-21T07:46:21.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.630+0000 7f2ff6200640 1 -- start start 2024-09-21T07:46:21.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.631+0000 7f2ff6200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ff00718e0 0x7f2ff013bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:21.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.631+0000 7f2ff6200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ff00b4cb0 0x7f2ff01e9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:21.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.631+0000 7f2ff6200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ff013e8c0 0x7f2ff01f4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:21.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.631+0000 7f2ff6200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff00b5600 con 0x7f2ff013e8c0 2024-09-21T07:46:21.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.631+0000 7f2ff6200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff015ca80 con 0x7f2ff00b4cb0 2024-09-21T07:46:21.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.631+0000 7f2ff6200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f2ff00b5780 con 0x7f2ff00718e0 2024-09-21T07:46:21.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.632+0000 7f2fefe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ff013e8c0 0x7f2ff01f4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:21.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.632+0000 7f2fefe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ff013e8c0 0x7f2ff01f4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53244/0 (socket says 172.21.15.39:53244) 2024-09-21T07:46:21.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.632+0000 7f2fefe00640 1 -- 172.21.15.39:0/1271960673 learned_addr learned my addr 172.21.15.39:0/1271960673 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:21.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.632+0000 7f2feea00640 1 --2- 172.21.15.39:0/1271960673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ff00b4cb0 0x7f2ff01e9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:21.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.632+0000 7f2fef400640 1 --2- 172.21.15.39:0/1271960673 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ff00718e0 0x7f2ff013bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:21.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.632+0000 7f2fefe00640 1 -- 172.21.15.39:0/1271960673 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ff00718e0 msgr2=0x7f2ff013bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:21.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.632+0000 7f2fefe00640 1 --2- 172.21.15.39:0/1271960673 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ff00718e0 0x7f2ff013bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.632+0000 7f2fefe00640 1 -- 172.21.15.39:0/1271960673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ff00b4cb0 msgr2=0x7f2ff01e9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:21.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.632+0000 7f2fefe00640 1 --2- 172.21.15.39:0/1271960673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ff00b4cb0 0x7f2ff01e9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.632+0000 7f2fefe00640 1 -- 172.21.15.39:0/1271960673 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff01f5280 con 0x7f2ff013e8c0 2024-09-21T07:46:21.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.632+0000 7f2feea00640 1 --2- 172.21.15.39:0/1271960673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ff00b4cb0 0x7f2ff01e9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:46:21.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.632+0000 7f2fefe00640 1 --2- 172.21.15.39:0/1271960673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ff013e8c0 0x7f2ff01f4d40 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800ea40 tx=0x7f2fe800ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:21.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.633+0000 7f2fe7400640 1 -- 172.21.15.39:0/1271960673 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe800ce40 con 0x7f2ff013e8c0 2024-09-21T07:46:21.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.633+0000 7f2fe7400640 1 -- 172.21.15.39:0/1271960673 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2fe8004c10 con 0x7f2ff013e8c0 2024-09-21T07:46:21.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.633+0000 7f2fe7400640 1 -- 172.21.15.39:0/1271960673 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8005020 con 0x7f2ff013e8c0 2024-09-21T07:46:21.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.633+0000 7f2ff6200640 1 -- 172.21.15.39:0/1271960673 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ff0150600 con 0x7f2ff013e8c0 2024-09-21T07:46:21.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.633+0000 7f2ff6200640 1 -- 172.21.15.39:0/1271960673 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ff0150bc0 con 0x7f2ff013e8c0 2024-09-21T07:46:21.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.634+0000 7f2ff6200640 1 -- 172.21.15.39:0/1271960673 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ff01ea190 con 0x7f2ff013e8c0 2024-09-21T07:46:21.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.635+0000 7f2fe7400640 1 -- 172.21.15.39:0/1271960673 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2fe80051c0 con 0x7f2ff013e8c0 2024-09-21T07:46:21.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.636+0000 7f2fe7400640 1 --2- 172.21.15.39:0/1271960673 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2fd0078b90 0x7f2fd007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:21.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.637+0000 7f2fe7400640 1 -- 172.21.15.39:0/1271960673 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 6122+0+0 (secure 0 0 0) 0x7f2fe809bb70 con 0x7f2ff013e8c0 2024-09-21T07:46:21.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.637+0000 7f2fef400640 1 --2- 172.21.15.39:0/1271960673 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2fd0078b90 0x7f2fd007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:21.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.637+0000 7f2fef400640 1 --2- 172.21.15.39:0/1271960673 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2fd0078b90 0x7f2fd007b0a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f2ff015f060 tx=0x7f2fe0009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:21.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.641+0000 7f2fe7400640 1 -- 172.21.15.39:0/1271960673 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2fe80637c0 con 0x7f2ff013e8c0 2024-09-21T07:46:21.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:21.848+0000 7f2ff6200640 1 -- 172.21.15.39:0/1271960673 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f2ff015b2d0 con 0x7f2ff013e8c0 2024-09-21T07:46:22.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.559+0000 7f2fe7400640 1 -- 172.21.15.39:0/1271960673 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v169) ==== 97+0+0 (secure 0 0 0) 0x7f2fe80681f0 con 0x7f2ff013e8c0 2024-09-21T07:46:22.561 INFO:teuthology.orchestra.run.smithi039.stderr: removed all blocklist entries 2024-09-21T07:46:22.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.562+0000 7f2ff6200640 1 -- 172.21.15.39:0/1271960673 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2fd0078b90 msgr2=0x7f2fd007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:22.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.562+0000 7f2ff6200640 1 --2- 172.21.15.39:0/1271960673 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2fd0078b90 0x7f2fd007b0a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f2ff015f060 tx=0x7f2fe0009040 comp rx=0 tx=0).stop 2024-09-21T07:46:22.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.562+0000 7f2ff6200640 1 -- 172.21.15.39:0/1271960673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ff013e8c0 msgr2=0x7f2ff01f4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:22.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.562+0000 7f2ff6200640 1 --2- 172.21.15.39:0/1271960673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ff013e8c0 0x7f2ff01f4d40 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800ea40 tx=0x7f2fe800ef00 comp rx=0 tx=0).stop 2024-09-21T07:46:22.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.562+0000 7f2ff6200640 1 -- 172.21.15.39:0/1271960673 shutdown_connections 2024-09-21T07:46:22.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.562+0000 7f2ff6200640 1 --2- 172.21.15.39:0/1271960673 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2fd0078b90 0x7f2fd007b0a0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:22.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.562+0000 7f2ff6200640 1 --2- 172.21.15.39:0/1271960673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ff013e8c0 0x7f2ff01f4d40 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:22.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.562+0000 7f2ff6200640 1 --2- 172.21.15.39:0/1271960673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ff00b4cb0 0x7f2ff01e9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:22.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.562+0000 7f2ff6200640 1 --2- 172.21.15.39:0/1271960673 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ff00718e0 0x7f2ff013bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:22.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.563+0000 7f2ff6200640 1 -- 172.21.15.39:0/1271960673 >> 172.21.15.39:0/1271960673 conn(0x7f2ff00d1450 msgr2=0x7f2ff015a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:22.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.563+0000 7f2ff6200640 1 -- 172.21.15.39:0/1271960673 shutdown_connections 2024-09-21T07:46:22.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.563+0000 7f2ff6200640 1 -- 172.21.15.39:0/1271960673 wait complete. 2024-09-21T07:46:22.596 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T07:46:22.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.825+0000 7f51a4c00640 1 -- 172.21.15.39:0/4083671228 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5184004800 msgr2=0x7f5184004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:22.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.825+0000 7f51a4c00640 1 --2- 172.21.15.39:0/4083671228 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5184004800 0x7f5184004c10 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f5188009b50 tx=0x7f5184005830 comp rx=0 tx=0).stop 2024-09-21T07:46:22.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.825+0000 7f51a4c00640 1 -- 172.21.15.39:0/4083671228 shutdown_connections 2024-09-21T07:46:22.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.825+0000 7f51a4c00640 1 --2- 172.21.15.39:0/4083671228 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5184004800 0x7f5184004c10 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:22.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.825+0000 7f51a4c00640 1 -- 172.21.15.39:0/4083671228 >> 172.21.15.39:0/4083671228 conn(0x7f51a006cfc0 msgr2=0x7f51a006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:22.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.825+0000 7f51a4c00640 1 -- 172.21.15.39:0/4083671228 shutdown_connections 2024-09-21T07:46:22.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.825+0000 7f51a4c00640 1 -- 172.21.15.39:0/4083671228 wait complete. 2024-09-21T07:46:22.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.827+0000 7f51a4c00640 1 Processor -- start 2024-09-21T07:46:22.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.828+0000 7f51a4c00640 1 -- start start 2024-09-21T07:46:22.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.829+0000 7f51a4c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5184004800 0x7f51a013e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:22.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.829+0000 7f51a4c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51a01f8eb0 0x7f51a00b3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:22.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.829+0000 7f51a4c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51a00b2580 0x7f51a01f6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:22.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.829+0000 7f51a4c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f51a015d900 con 0x7f5184004800 2024-09-21T07:46:22.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.829+0000 7f51a4c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f51a015d780 con 0x7f51a01f8eb0 2024-09-21T07:46:22.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.829+0000 7f51a4c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f51a015da80 con 0x7f51a00b2580 2024-09-21T07:46:22.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.829+0000 7f519f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5184004800 0x7f51a013e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:22.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.829+0000 7f519f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5184004800 0x7f51a013e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35446/0 (socket says 172.21.15.39:35446) 2024-09-21T07:46:22.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.829+0000 7f519f400640 1 -- 172.21.15.39:0/1893740943 learned_addr learned my addr 172.21.15.39:0/1893740943 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:22.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.830+0000 7f5197e00640 1 --2- 172.21.15.39:0/1893740943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51a01f8eb0 0x7f51a00b3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:22.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.830+0000 7f519fe00640 1 --2- 172.21.15.39:0/1893740943 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51a00b2580 0x7f51a01f6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:22.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.830+0000 7f519f400640 1 -- 172.21.15.39:0/1893740943 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51a00b2580 msgr2=0x7f51a01f6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:22.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.830+0000 7f519f400640 1 --2- 172.21.15.39:0/1893740943 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51a00b2580 0x7f51a01f6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:22.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.830+0000 7f519f400640 1 -- 172.21.15.39:0/1893740943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51a01f8eb0 msgr2=0x7f51a00b3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:22.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.830+0000 7f519f400640 1 --2- 172.21.15.39:0/1893740943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51a01f8eb0 0x7f51a00b3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:22.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.830+0000 7f519f400640 1 -- 172.21.15.39:0/1893740943 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a01f58c0 con 0x7f5184004800 2024-09-21T07:46:22.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.830+0000 7f519fe00640 1 --2- 172.21.15.39:0/1893740943 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51a00b2580 0x7f51a01f6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:46:22.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.830+0000 7f519f400640 1 --2- 172.21.15.39:0/1893740943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5184004800 0x7f51a013e630 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f5190003ed0 tx=0x7f5190001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:22.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.830+0000 7f5197400640 1 -- 172.21.15.39:0/1893740943 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5190015070 con 0x7f5184004800 2024-09-21T07:46:22.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.830+0000 7f5197400640 1 -- 172.21.15.39:0/1893740943 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5190011840 con 0x7f5184004800 2024-09-21T07:46:22.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.830+0000 7f51a4c00640 1 -- 172.21.15.39:0/1893740943 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51a01f9690 con 0x7f5184004800 2024-09-21T07:46:22.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.830+0000 7f5197400640 1 -- 172.21.15.39:0/1893740943 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5190011c50 con 0x7f5184004800 2024-09-21T07:46:22.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.831+0000 7f51a4c00640 1 -- 172.21.15.39:0/1893740943 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51a01f9c50 con 0x7f5184004800 2024-09-21T07:46:22.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.832+0000 7f51a4c00640 1 -- 172.21.15.39:0/1893740943 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51a00d0680 con 0x7f5184004800 2024-09-21T07:46:22.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.833+0000 7f5197400640 1 -- 172.21.15.39:0/1893740943 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5190018070 con 0x7f5184004800 2024-09-21T07:46:22.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.834+0000 7f5197400640 1 --2- 172.21.15.39:0/1893740943 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f517c078b90 0x7f517c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:22.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.834+0000 7f5197400640 1 -- 172.21.15.39:0/1893740943 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 6036+0+0 (secure 0 0 0) 0x7f519009ba60 con 0x7f5184004800 2024-09-21T07:46:22.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.834+0000 7f5197e00640 1 --2- 172.21.15.39:0/1893740943 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f517c078b90 0x7f517c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:22.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.835+0000 7f5197e00640 1 --2- 172.21.15.39:0/1893740943 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f517c078b90 0x7f517c07b0a0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f5188035000 tx=0x7f51880046c0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:22.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:22.839+0000 7f5197400640 1 -- 172.21.15.39:0/1893740943 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f51a01f58c0 con 0x7f5184004800 2024-09-21T07:46:23.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.108+0000 7f51a4c00640 1 -- 172.21.15.39:0/1893740943 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f51a0073190 con 0x7f5184004800 2024-09-21T07:46:23.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.108+0000 7f5197400640 1 -- 172.21.15.39:0/1893740943 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) ==== 80+0+5304 (secure 0 0 0) 0x7f5190063cb0 con 0x7f5184004800 2024-09-21T07:46:23.110 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:46:23.110 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:46:23.110 INFO:teuthology.orchestra.run.smithi039.stdout: "auth_dump": [ 2024-09-21T07:46:23.111 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:46:23.111 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.a", 2024-09-21T07:46:23.111 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5m+P6EERAABQqmlkrocSi1+hwc73Adrg==", 2024-09-21T07:46:23.111 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:46:23.111 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:46:23.111 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:46:23.111 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:46:23.111 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:46:23.111 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:46:23.111 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:46:23.111 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:46:23.111 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.b", 2024-09-21T07:46:23.111 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5m5/GeLRAAfE8fr5dbitQa89rNK9pZ2Q==", 2024-09-21T07:46:23.111 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:46:23.112 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:46:23.112 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:46:23.112 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:46:23.112 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:46:23.112 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:46:23.112 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:46:23.112 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:46:23.112 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.c", 2024-09-21T07:46:23.112 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5msbx6HxAALsF8ORvIoJpIAFZ5uxmOEw==", 2024-09-21T07:46:23.112 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:46:23.112 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:46:23.112 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:46:23.112 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:46:23.112 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:46:23.113 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:46:23.113 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:46:23.113 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:46:23.113 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.d", 2024-09-21T07:46:23.113 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mIrcBABAApKAB/7FkZ5glNTxlbkIsBw==", 2024-09-21T07:46:23.113 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:46:23.113 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:46:23.113 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:46:23.113 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:46:23.113 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:46:23.113 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:46:23.113 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:46:23.113 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:46:23.114 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.0", 2024-09-21T07:46:23.114 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB7du5msw81DhAAmVimqN4x9K3YpT0nbovppQ==", 2024-09-21T07:46:23.114 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:46:23.114 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:46:23.114 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:46:23.114 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:46:23.114 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:46:23.114 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:46:23.114 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:46:23.114 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.1", 2024-09-21T07:46:23.114 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB+du5mdnCIOhAAtVB4ThehQwb0zRriyKcaxA==", 2024-09-21T07:46:23.114 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:46:23.114 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:46:23.114 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:46:23.114 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:46:23.115 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:46:23.115 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:46:23.115 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:46:23.115 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.2", 2024-09-21T07:46:23.115 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCCdu5mTfWxDhAAadJCbPN5xBx/f9L0v2yTdg==", 2024-09-21T07:46:23.115 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:46:23.115 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:46:23.115 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:46:23.115 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:46:23.115 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:46:23.115 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:46:23.115 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:46:23.115 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.3", 2024-09-21T07:46:23.115 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCFdu5mL1E2HxAAyOhY97uUXEw5L1CcDeONVA==", 2024-09-21T07:46:23.116 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:46:23.116 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:46:23.116 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:46:23.116 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:46:23.116 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:46:23.116 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:46:23.116 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:46:23.116 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.4", 2024-09-21T07:46:23.116 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCLdu5m6PY/IhAAgvcTgm+6uWA7lIG4tcTIEg==", 2024-09-21T07:46:23.116 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:46:23.116 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:46:23.116 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:46:23.116 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:46:23.116 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:46:23.116 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:46:23.117 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:46:23.117 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.5", 2024-09-21T07:46:23.117 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCOdu5mE/wlMxAA3JBg4xH+N/PPC1Uv2ZVrYQ==", 2024-09-21T07:46:23.117 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:46:23.117 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:46:23.117 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:46:23.118 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:46:23.118 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:46:23.118 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:46:23.118 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:46:23.118 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.6", 2024-09-21T07:46:23.118 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCSdu5mC3TgBxAAwsArjeIdFcApY3/1OJffEg==", 2024-09-21T07:46:23.118 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:46:23.118 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:46:23.118 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:46:23.118 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:46:23.118 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:46:23.118 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:46:23.118 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:46:23.118 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.7", 2024-09-21T07:46:23.119 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCVdu5mSWHWFBAA9qPHWRdsvrB74inrp1lj4Q==", 2024-09-21T07:46:23.119 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:46:23.119 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:46:23.119 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:46:23.119 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:46:23.119 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:46:23.119 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:46:23.119 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:46:23.119 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.0", 2024-09-21T07:46:23.119 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mGfb1DBAA/aSMKR5or3xy53rB/iUjpA==", 2024-09-21T07:46:23.119 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:46:23.119 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:46:23.119 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow r", 2024-09-21T07:46:23.119 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T07:46:23.119 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:46:23.120 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:46:23.120 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:46:23.120 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.1", 2024-09-21T07:46:23.120 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mEkciIhAAid+bxJEr4P7vMnZ42PAy5w==", 2024-09-21T07:46:23.120 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:46:23.120 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:46:23.120 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow r", 2024-09-21T07:46:23.120 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow rw", 2024-09-21T07:46:23.120 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rwx" 2024-09-21T07:46:23.120 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:46:23.120 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:46:23.120 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:46:23.120 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.2", 2024-09-21T07:46:23.120 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mSu2oFxAAgjy2m/wWKh6xaoh+YDoU1Q==", 2024-09-21T07:46:23.120 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:46:23.121 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:46:23.121 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow r", 2024-09-21T07:46:23.121 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow rw", 2024-09-21T07:46:23.121 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rwx" 2024-09-21T07:46:23.121 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:46:23.121 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:46:23.121 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:46:23.121 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.3", 2024-09-21T07:46:23.121 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mCHy0LBAAZneF2jUWgfuyGehZFbOxDg==", 2024-09-21T07:46:23.121 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:46:23.121 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:46:23.121 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow r", 2024-09-21T07:46:23.121 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow rw", 2024-09-21T07:46:23.121 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rwx" 2024-09-21T07:46:23.121 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:46:23.122 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:46:23.122 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:46:23.122 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.admin", 2024-09-21T07:46:23.122 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBydu5m9LInERAAAlDpqb94f2Qb9VYJJwTBcA==", 2024-09-21T07:46:23.122 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:46:23.122 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow *", 2024-09-21T07:46:23.122 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:46:23.122 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:46:23.122 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:46:23.122 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:46:23.122 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:46:23.122 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:46:23.122 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mgr.x", 2024-09-21T07:46:23.122 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5mA2GRBhAAkyCKdgH6laHy8xNt080dXg==", 2024-09-21T07:46:23.122 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:46:23.123 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow *", 2024-09-21T07:46:23.123 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile mgr", 2024-09-21T07:46:23.123 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:46:23.123 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:46:23.123 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:46:23.123 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:46:23.123 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mgr.y", 2024-09-21T07:46:23.123 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBydu5mr+8aNxAA7TFYNms2WsXUfa4ITTJOHw==", 2024-09-21T07:46:23.123 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:46:23.123 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow *", 2024-09-21T07:46:23.123 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile mgr", 2024-09-21T07:46:23.123 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:46:23.123 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:46:23.123 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:46:23.123 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-21T07:46:23.124 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:46:23.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.110+0000 7f51a4c00640 1 -- 172.21.15.39:0/1893740943 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f517c078b90 msgr2=0x7f517c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:23.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.110+0000 7f51a4c00640 1 --2- 172.21.15.39:0/1893740943 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f517c078b90 0x7f517c07b0a0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f5188035000 tx=0x7f51880046c0 comp rx=0 tx=0).stop 2024-09-21T07:46:23.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.110+0000 7f51a4c00640 1 -- 172.21.15.39:0/1893740943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5184004800 msgr2=0x7f51a013e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:23.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.110+0000 7f51a4c00640 1 --2- 172.21.15.39:0/1893740943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5184004800 0x7f51a013e630 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f5190003ed0 tx=0x7f5190001630 comp rx=0 tx=0).stop 2024-09-21T07:46:23.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.111+0000 7f51a4c00640 1 -- 172.21.15.39:0/1893740943 shutdown_connections 2024-09-21T07:46:23.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.111+0000 7f51a4c00640 1 --2- 172.21.15.39:0/1893740943 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f517c078b90 0x7f517c07b0a0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:23.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.111+0000 7f51a4c00640 1 --2- 172.21.15.39:0/1893740943 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51a00b2580 0x7f51a01f6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:23.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.111+0000 7f51a4c00640 1 --2- 172.21.15.39:0/1893740943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51a01f8eb0 0x7f51a00b3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:23.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.111+0000 7f51a4c00640 1 --2- 172.21.15.39:0/1893740943 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5184004800 0x7f51a013e630 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:23.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.111+0000 7f51a4c00640 1 -- 172.21.15.39:0/1893740943 >> 172.21.15.39:0/1893740943 conn(0x7f51a006cfc0 msgr2=0x7f51a015eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:23.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.111+0000 7f51a4c00640 1 -- 172.21.15.39:0/1893740943 shutdown_connections 2024-09-21T07:46:23.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.111+0000 7f51a4c00640 1 -- 172.21.15.39:0/1893740943 wait complete. 2024-09-21T07:46:23.134 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T07:46:23.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.364+0000 7f3011800640 1 -- 172.21.15.39:0/3845681184 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f300c073dd0 msgr2=0x7f300c15d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:23.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.364+0000 7f3011800640 1 --2- 172.21.15.39:0/3845681184 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f300c073dd0 0x7f300c15d5b0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc005fd0 tx=0x7f2ffc02a5d0 comp rx=0 tx=0).stop 2024-09-21T07:46:23.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.364+0000 7f3011800640 1 -- 172.21.15.39:0/3845681184 shutdown_connections 2024-09-21T07:46:23.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.364+0000 7f3011800640 1 --2- 172.21.15.39:0/3845681184 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f300c15daf0 0x7f300c0cf450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:23.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.364+0000 7f3011800640 1 --2- 172.21.15.39:0/3845681184 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f300c073dd0 0x7f300c15d5b0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:23.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.364+0000 7f3011800640 1 -- 172.21.15.39:0/3845681184 >> 172.21.15.39:0/3845681184 conn(0x7f300c06cfc0 msgr2=0x7f300c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:23.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.364+0000 7f3011800640 1 -- 172.21.15.39:0/3845681184 shutdown_connections 2024-09-21T07:46:23.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.364+0000 7f3011800640 1 -- 172.21.15.39:0/3845681184 wait complete. 2024-09-21T07:46:23.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.367+0000 7f3011800640 1 Processor -- start 2024-09-21T07:46:23.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.367+0000 7f3011800640 1 -- start start 2024-09-21T07:46:23.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.368+0000 7f3011800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f300c073dd0 0x7f300c1f99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:23.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.368+0000 7f3011800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f300c0cf990 0x7f300c142d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:23.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.368+0000 7f3011800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f300c15daf0 0x7f300c15b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:23.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.368+0000 7f3011800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f300c0cd1d0 con 0x7f300c073dd0 2024-09-21T07:46:23.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.368+0000 7f3011800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f300c0cd050 con 0x7f300c0cf990 2024-09-21T07:46:23.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.368+0000 7f3011800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f300c0cd350 con 0x7f300c15daf0 2024-09-21T07:46:23.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.368+0000 7f300aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f300c073dd0 0x7f300c1f99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:23.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.368+0000 7f300aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f300c073dd0 0x7f300c1f99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35466/0 (socket says 172.21.15.39:35466) 2024-09-21T07:46:23.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.368+0000 7f300aa00640 1 -- 172.21.15.39:0/226564090 learned_addr learned my addr 172.21.15.39:0/226564090 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:23.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.369+0000 7f3003e00640 1 --2- 172.21.15.39:0/226564090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f300c0cf990 0x7f300c142d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:23.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.369+0000 7f300b400640 1 --2- 172.21.15.39:0/226564090 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f300c15daf0 0x7f300c15b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:23.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.369+0000 7f300aa00640 1 -- 172.21.15.39:0/226564090 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f300c15daf0 msgr2=0x7f300c15b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:23.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.369+0000 7f300aa00640 1 --2- 172.21.15.39:0/226564090 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f300c15daf0 0x7f300c15b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:23.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.369+0000 7f300aa00640 1 -- 172.21.15.39:0/226564090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f300c0cf990 msgr2=0x7f300c142d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:23.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.369+0000 7f300aa00640 1 --2- 172.21.15.39:0/226564090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f300c0cf990 0x7f300c142d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:23.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.369+0000 7f300aa00640 1 -- 172.21.15.39:0/226564090 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f300c1e5310 con 0x7f300c073dd0 2024-09-21T07:46:23.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.369+0000 7f300aa00640 1 --2- 172.21.15.39:0/226564090 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f300c073dd0 0x7f300c1f99f0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc034040 tx=0x7f2ffc0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:23.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.370+0000 7f3002a00640 1 -- 172.21.15.39:0/226564090 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc03e030 con 0x7f300c073dd0 2024-09-21T07:46:23.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.370+0000 7f3002a00640 1 -- 172.21.15.39:0/226564090 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ffc003ca0 con 0x7f300c073dd0 2024-09-21T07:46:23.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.370+0000 7f3002a00640 1 -- 172.21.15.39:0/226564090 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc035040 con 0x7f300c073dd0 2024-09-21T07:46:23.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.370+0000 7f3011800640 1 -- 172.21.15.39:0/226564090 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f300c1e55f0 con 0x7f300c073dd0 2024-09-21T07:46:23.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.370+0000 7f3011800640 1 -- 172.21.15.39:0/226564090 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f300c150d10 con 0x7f300c073dd0 2024-09-21T07:46:23.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.371+0000 7f3011800640 1 -- 172.21.15.39:0/226564090 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f300c0cd800 con 0x7f300c073dd0 2024-09-21T07:46:23.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.372+0000 7f3002a00640 1 -- 172.21.15.39:0/226564090 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2ffc041030 con 0x7f300c073dd0 2024-09-21T07:46:23.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.373+0000 7f3002a00640 1 --2- 172.21.15.39:0/226564090 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2fe8078b90 0x7f2fe807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:23.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.373+0000 7f3002a00640 1 -- 172.21.15.39:0/226564090 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 6036+0+0 (secure 0 0 0) 0x7f2ffc0bb500 con 0x7f300c073dd0 2024-09-21T07:46:23.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.374+0000 7f3003e00640 1 --2- 172.21.15.39:0/226564090 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2fe8078b90 0x7f2fe807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:23.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.374+0000 7f3003e00640 1 --2- 172.21.15.39:0/226564090 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2fe8078b90 0x7f2fe807b0a0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f300c15fbb0 tx=0x7f2ff400b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:23.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.378+0000 7f3002a00640 1 -- 172.21.15.39:0/226564090 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2ffc083100 con 0x7f300c073dd0 2024-09-21T07:46:23.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.580+0000 7f3011800640 1 -- 172.21.15.39:0/226564090 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f300c0cdcf0 con 0x7f300c073dd0 2024-09-21T07:46:23.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.580+0000 7f3002a00640 1 -- 172.21.15.39:0/226564090 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v171) ==== 84+0+287 (secure 0 0 0) 0x7f2ffc087b30 con 0x7f300c073dd0 2024-09-21T07:46:23.583 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:46:23.583 INFO:teuthology.orchestra.run.smithi039.stdout:[ 2024-09-21T07:46:23.583 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:46:23.583 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 1, 2024-09-21T07:46:23.583 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": ".mgr" 2024-09-21T07:46:23.583 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:46:23.583 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:46:23.583 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 6, 2024-09-21T07:46:23.583 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": "snap-test-pool" 2024-09-21T07:46:23.583 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:46:23.583 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:46:23.583 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 11, 2024-09-21T07:46:23.583 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": "test_data_pool" 2024-09-21T07:46:23.584 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:46:23.584 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:46:23.584 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 12, 2024-09-21T07:46:23.584 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": "test_metadata_pool" 2024-09-21T07:46:23.584 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:46:23.584 INFO:teuthology.orchestra.run.smithi039.stdout:] 2024-09-21T07:46:23.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.583+0000 7f3011800640 1 -- 172.21.15.39:0/226564090 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2fe8078b90 msgr2=0x7f2fe807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:23.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.583+0000 7f3011800640 1 --2- 172.21.15.39:0/226564090 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2fe8078b90 0x7f2fe807b0a0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f300c15fbb0 tx=0x7f2ff400b2e0 comp rx=0 tx=0).stop 2024-09-21T07:46:23.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.583+0000 7f3011800640 1 -- 172.21.15.39:0/226564090 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f300c073dd0 msgr2=0x7f300c1f99f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:23.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.583+0000 7f3011800640 1 --2- 172.21.15.39:0/226564090 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f300c073dd0 0x7f300c1f99f0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc034040 tx=0x7f2ffc0038a0 comp rx=0 tx=0).stop 2024-09-21T07:46:23.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.583+0000 7f3011800640 1 -- 172.21.15.39:0/226564090 shutdown_connections 2024-09-21T07:46:23.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.583+0000 7f3011800640 1 --2- 172.21.15.39:0/226564090 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2fe8078b90 0x7f2fe807b0a0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:23.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.583+0000 7f3011800640 1 --2- 172.21.15.39:0/226564090 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f300c15daf0 0x7f300c15b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:23.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.583+0000 7f3011800640 1 --2- 172.21.15.39:0/226564090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f300c0cf990 0x7f300c142d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:23.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.583+0000 7f3011800640 1 --2- 172.21.15.39:0/226564090 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f300c073dd0 0x7f300c1f99f0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:23.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.583+0000 7f3011800640 1 -- 172.21.15.39:0/226564090 >> 172.21.15.39:0/226564090 conn(0x7f300c06cfc0 msgr2=0x7f300c15c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:23.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.583+0000 7f3011800640 1 -- 172.21.15.39:0/226564090 shutdown_connections 2024-09-21T07:46:23.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.583+0000 7f3011800640 1 -- 172.21.15.39:0/226564090 wait complete. 2024-09-21T07:46:23.608 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T07:46:23.608 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T07:46:23.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.837+0000 7fd88b200640 1 -- 172.21.15.39:0/2570087251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd88415d440 msgr2=0x7fd88415d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:23.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.837+0000 7fd88b200640 1 --2- 172.21.15.39:0/2570087251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd88415d440 0x7fd88415d850 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fd880005fd0 tx=0x7fd88002aa00 comp rx=0 tx=0).stop 2024-09-21T07:46:23.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.837+0000 7fd88b200640 1 -- 172.21.15.39:0/2570087251 shutdown_connections 2024-09-21T07:46:23.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.837+0000 7fd88b200640 1 --2- 172.21.15.39:0/2570087251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd88415d440 0x7fd88415d850 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:23.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.837+0000 7fd88b200640 1 --2- 172.21.15.39:0/2570087251 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd8841539f0 0x7fd8840b4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:23.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.837+0000 7fd88b200640 1 -- 172.21.15.39:0/2570087251 >> 172.21.15.39:0/2570087251 conn(0x7fd88406d660 msgr2=0x7fd8840b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:23.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.837+0000 7fd88b200640 1 -- 172.21.15.39:0/2570087251 shutdown_connections 2024-09-21T07:46:23.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.837+0000 7fd88b200640 1 -- 172.21.15.39:0/2570087251 wait complete. 2024-09-21T07:46:23.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.839+0000 7fd88b200640 1 Processor -- start 2024-09-21T07:46:23.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.840+0000 7fd88b200640 1 -- start start 2024-09-21T07:46:23.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.841+0000 7fd88b200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8841539f0 0x7fd88414ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:23.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.841+0000 7fd88b200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd88415d440 0x7fd8840cef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:23.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.841+0000 7fd88b200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd884160340 0x7fd8841f3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:23.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.841+0000 7fd88b200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd8840ad670 con 0x7fd8841539f0 2024-09-21T07:46:23.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.841+0000 7fd88b200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd8840ad4f0 con 0x7fd884160340 2024-09-21T07:46:23.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.841+0000 7fd88b200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd8840ad7f0 con 0x7fd88415d440 2024-09-21T07:46:23.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.841+0000 7fd889e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8841539f0 0x7fd88414ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:23.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.841+0000 7fd889e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8841539f0 0x7fd88414ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35478/0 (socket says 172.21.15.39:35478) 2024-09-21T07:46:23.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.841+0000 7fd889e00640 1 -- 172.21.15.39:0/154543622 learned_addr learned my addr 172.21.15.39:0/154543622 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:23.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.842+0000 7fd88a800640 1 --2- 172.21.15.39:0/154543622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd884160340 0x7fd8841f3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:23.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.842+0000 7fd889400640 1 --2- 172.21.15.39:0/154543622 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd88415d440 0x7fd8840cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:23.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.842+0000 7fd889e00640 1 -- 172.21.15.39:0/154543622 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd88415d440 msgr2=0x7fd8840cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:23.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.842+0000 7fd889e00640 1 --2- 172.21.15.39:0/154543622 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd88415d440 0x7fd8840cef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:23.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.842+0000 7fd889e00640 1 -- 172.21.15.39:0/154543622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd884160340 msgr2=0x7fd8841f3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:23.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.842+0000 7fd889e00640 1 --2- 172.21.15.39:0/154543622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd884160340 0x7fd8841f3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:23.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.842+0000 7fd889e00640 1 -- 172.21.15.39:0/154543622 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd884154050 con 0x7fd8841539f0 2024-09-21T07:46:23.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.842+0000 7fd889e00640 1 --2- 172.21.15.39:0/154543622 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8841539f0 0x7fd88414ccc0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fd87800d6d0 tx=0x7fd87800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:23.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.843+0000 7fd87e000640 1 -- 172.21.15.39:0/154543622 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd878017070 con 0x7fd8841539f0 2024-09-21T07:46:23.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.843+0000 7fd87e000640 1 -- 172.21.15.39:0/154543622 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd878005330 con 0x7fd8841539f0 2024-09-21T07:46:23.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.843+0000 7fd87e000640 1 -- 172.21.15.39:0/154543622 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8780097c0 con 0x7fd8841539f0 2024-09-21T07:46:23.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.843+0000 7fd88b200640 1 -- 172.21.15.39:0/154543622 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd884154330 con 0x7fd8841539f0 2024-09-21T07:46:23.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.843+0000 7fd88b200640 1 -- 172.21.15.39:0/154543622 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8841f4540 con 0x7fd8841539f0 2024-09-21T07:46:23.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.844+0000 7fd88b200640 1 -- 172.21.15.39:0/154543622 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8840b4c40 con 0x7fd8841539f0 2024-09-21T07:46:23.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.845+0000 7fd87e000640 1 -- 172.21.15.39:0/154543622 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd878009b70 con 0x7fd8841539f0 2024-09-21T07:46:23.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.846+0000 7fd87e000640 1 --2- 172.21.15.39:0/154543622 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd868078b90 0x7fd86807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:23.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.846+0000 7fd87e000640 1 -- 172.21.15.39:0/154543622 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 6036+0+0 (secure 0 0 0) 0x7fd87809b0a0 con 0x7fd8841539f0 2024-09-21T07:46:23.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.847+0000 7fd889400640 1 --2- 172.21.15.39:0/154543622 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd868078b90 0x7fd86807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:23.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.848+0000 7fd889400640 1 --2- 172.21.15.39:0/154543622 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd868078b90 0x7fd86807b0a0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fd8841601f0 tx=0x7fd874009060 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:23.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:23.850+0000 7fd87e000640 1 -- 172.21.15.39:0/154543622 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd878062d30 con 0x7fd8841539f0 2024-09-21T07:46:24.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.053+0000 7fd88b200640 1 -- 172.21.15.39:0/154543622 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fd8841f4800 con 0x7fd8841539f0 2024-09-21T07:46:24.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.563+0000 7fd87e000640 1 -- 172.21.15.39:0/154543622 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v172) ==== 138+0+0 (secure 0 0 0) 0x7fd878067760 con 0x7fd8841539f0 2024-09-21T07:46:24.565 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' created 2024-09-21T07:46:24.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.566+0000 7fd88b200640 1 -- 172.21.15.39:0/154543622 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd868078b90 msgr2=0x7fd86807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:24.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.566+0000 7fd88b200640 1 --2- 172.21.15.39:0/154543622 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd868078b90 0x7fd86807b0a0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fd8841601f0 tx=0x7fd874009060 comp rx=0 tx=0).stop 2024-09-21T07:46:24.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.566+0000 7fd88b200640 1 -- 172.21.15.39:0/154543622 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8841539f0 msgr2=0x7fd88414ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:24.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.566+0000 7fd88b200640 1 --2- 172.21.15.39:0/154543622 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8841539f0 0x7fd88414ccc0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fd87800d6d0 tx=0x7fd87800db90 comp rx=0 tx=0).stop 2024-09-21T07:46:24.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.567+0000 7fd88b200640 1 -- 172.21.15.39:0/154543622 shutdown_connections 2024-09-21T07:46:24.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.567+0000 7fd88b200640 1 --2- 172.21.15.39:0/154543622 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd868078b90 0x7fd86807b0a0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:24.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.567+0000 7fd88b200640 1 --2- 172.21.15.39:0/154543622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd884160340 0x7fd8841f3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:24.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.567+0000 7fd88b200640 1 --2- 172.21.15.39:0/154543622 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd88415d440 0x7fd8840cef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:24.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.567+0000 7fd88b200640 1 --2- 172.21.15.39:0/154543622 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8841539f0 0x7fd88414ccc0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:24.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.567+0000 7fd88b200640 1 -- 172.21.15.39:0/154543622 >> 172.21.15.39:0/154543622 conn(0x7fd88406d660 msgr2=0x7fd884151350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:24.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.567+0000 7fd88b200640 1 -- 172.21.15.39:0/154543622 shutdown_connections 2024-09-21T07:46:24.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.567+0000 7fd88b200640 1 -- 172.21.15.39:0/154543622 wait complete. 2024-09-21T07:46:24.596 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T07:46:24.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.829+0000 7f6fc6a00640 1 -- 172.21.15.39:0/498644224 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6fc0160250 msgr2=0x7f6fc015d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:24.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.829+0000 7f6fc6a00640 1 --2- 172.21.15.39:0/498644224 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6fc0160250 0x7f6fc015d570 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4005fd0 tx=0x7f6fb402a5d0 comp rx=0 tx=0).stop 2024-09-21T07:46:24.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.829+0000 7f6fc6a00640 1 -- 172.21.15.39:0/498644224 shutdown_connections 2024-09-21T07:46:24.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.829+0000 7f6fc6a00640 1 --2- 172.21.15.39:0/498644224 >> v2:172.21.15.148:3300/0 conn(0x7f6fc0160660 0x7f6fc00cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:24.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.829+0000 7f6fc6a00640 1 --2- 172.21.15.39:0/498644224 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6fc0160250 0x7f6fc015d570 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:24.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.829+0000 7f6fc6a00640 1 -- 172.21.15.39:0/498644224 >> 172.21.15.39:0/498644224 conn(0x7f6fc006cfc0 msgr2=0x7f6fc006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:24.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.829+0000 7f6fc6a00640 1 -- 172.21.15.39:0/498644224 shutdown_connections 2024-09-21T07:46:24.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.829+0000 7f6fc6a00640 1 -- 172.21.15.39:0/498644224 wait complete. 2024-09-21T07:46:24.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.831+0000 7f6fc6a00640 1 Processor -- start 2024-09-21T07:46:24.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.831+0000 7f6fc6a00640 1 -- start start 2024-09-21T07:46:24.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.833+0000 7f6fc6a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6fc00cfc50 0x7f6fc00b1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:24.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.833+0000 7f6fc6a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6fc0160250 0x7f6fc0141d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:24.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.833+0000 7f6fc6a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6fc0160660 0x7f6fc0142260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:24.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.833+0000 7f6fc6a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc00cce00 con 0x7f6fc00cfc50 2024-09-21T07:46:24.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.833+0000 7f6fc6a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc00ccc80 con 0x7f6fc0160660 2024-09-21T07:46:24.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.833+0000 7f6fc6a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f6fc00ccf80 con 0x7f6fc0160250 2024-09-21T07:46:24.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.833+0000 7f6fc5600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6fc00cfc50 0x7f6fc00b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:24.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.833+0000 7f6fc5600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6fc00cfc50 0x7f6fc00b1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35494/0 (socket says 172.21.15.39:35494) 2024-09-21T07:46:24.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.833+0000 7f6fc5600640 1 -- 172.21.15.39:0/2153024989 learned_addr learned my addr 172.21.15.39:0/2153024989 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:24.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.833+0000 7f6fc4c00640 1 --2- 172.21.15.39:0/2153024989 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6fc0160250 0x7f6fc0141d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:24.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.833+0000 7f6fc6000640 1 --2- 172.21.15.39:0/2153024989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6fc0160660 0x7f6fc0142260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:24.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.833+0000 7f6fc5600640 1 -- 172.21.15.39:0/2153024989 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6fc0160250 msgr2=0x7f6fc0141d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:24.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.833+0000 7f6fc5600640 1 --2- 172.21.15.39:0/2153024989 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6fc0160250 0x7f6fc0141d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:24.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.833+0000 7f6fc5600640 1 -- 172.21.15.39:0/2153024989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6fc0160660 msgr2=0x7f6fc0142260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:24.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.833+0000 7f6fc5600640 1 --2- 172.21.15.39:0/2153024989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6fc0160660 0x7f6fc0142260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:24.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.833+0000 7f6fc5600640 1 -- 172.21.15.39:0/2153024989 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fc015a600 con 0x7f6fc00cfc50 2024-09-21T07:46:24.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.834+0000 7f6fc5600640 1 --2- 172.21.15.39:0/2153024989 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6fc00cfc50 0x7f6fc00b1dc0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4006010 tx=0x7f6fb40035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:24.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.834+0000 7f6fbd600640 1 -- 172.21.15.39:0/2153024989 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb403e030 con 0x7f6fc00cfc50 2024-09-21T07:46:24.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.834+0000 7f6fbd600640 1 -- 172.21.15.39:0/2153024989 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fb4003ae0 con 0x7f6fc00cfc50 2024-09-21T07:46:24.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.834+0000 7f6fbd600640 1 -- 172.21.15.39:0/2153024989 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb4035040 con 0x7f6fc00cfc50 2024-09-21T07:46:24.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.834+0000 7f6fc6a00640 1 -- 172.21.15.39:0/2153024989 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fc0159840 con 0x7f6fc00cfc50 2024-09-21T07:46:24.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.834+0000 7f6fc6a00640 1 -- 172.21.15.39:0/2153024989 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fc0159e00 con 0x7f6fc00cfc50 2024-09-21T07:46:24.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.835+0000 7f6fc6a00640 1 -- 172.21.15.39:0/2153024989 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fc00cd430 con 0x7f6fc00cfc50 2024-09-21T07:46:24.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.837+0000 7f6fbd600640 1 -- 172.21.15.39:0/2153024989 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6fb4041030 con 0x7f6fc00cfc50 2024-09-21T07:46:24.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.838+0000 7f6fbd600640 1 --2- 172.21.15.39:0/2153024989 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6fa0078b90 0x7f6fa007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:24.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.839+0000 7f6fbd600640 1 -- 172.21.15.39:0/2153024989 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(172..172 src has 1..172) ==== 6406+0+0 (secure 0 0 0) 0x7f6fb40bba10 con 0x7f6fc00cfc50 2024-09-21T07:46:24.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.839+0000 7f6fc4c00640 1 --2- 172.21.15.39:0/2153024989 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6fa0078b90 0x7f6fa007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:24.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.840+0000 7f6fc4c00640 1 --2- 172.21.15.39:0/2153024989 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6fa0078b90 0x7f6fa007b0a0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6fac0095f0 tx=0x7f6fac005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:24.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:24.843+0000 7f6fbd600640 1 -- 172.21.15.39:0/2153024989 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6fb40834a0 con 0x7f6fc00cfc50 2024-09-21T07:46:25.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.040+0000 7f6fc6a00640 1 -- 172.21.15.39:0/2153024989 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f6fc006d570 con 0x7f6fc00cfc50 2024-09-21T07:46:25.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.572+0000 7f6fbd600640 1 -- 172.21.15.39:0/2153024989 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v173) ==== 170+0+0 (secure 0 0 0) 0x7f6fb4087ed0 con 0x7f6fc00cfc50 2024-09-21T07:46:25.574 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' created 2024-09-21T07:46:25.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.575+0000 7f6fc6a00640 1 -- 172.21.15.39:0/2153024989 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6fa0078b90 msgr2=0x7f6fa007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:25.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.575+0000 7f6fc6a00640 1 --2- 172.21.15.39:0/2153024989 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6fa0078b90 0x7f6fa007b0a0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6fac0095f0 tx=0x7f6fac005f90 comp rx=0 tx=0).stop 2024-09-21T07:46:25.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.575+0000 7f6fc6a00640 1 -- 172.21.15.39:0/2153024989 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6fc00cfc50 msgr2=0x7f6fc00b1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:25.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.575+0000 7f6fc6a00640 1 --2- 172.21.15.39:0/2153024989 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6fc00cfc50 0x7f6fc00b1dc0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4006010 tx=0x7f6fb40035f0 comp rx=0 tx=0).stop 2024-09-21T07:46:25.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.576+0000 7f6fc6a00640 1 -- 172.21.15.39:0/2153024989 shutdown_connections 2024-09-21T07:46:25.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.576+0000 7f6fc6a00640 1 --2- 172.21.15.39:0/2153024989 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6fa0078b90 0x7f6fa007b0a0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:25.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.576+0000 7f6fc6a00640 1 --2- 172.21.15.39:0/2153024989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6fc0160660 0x7f6fc0142260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:25.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.576+0000 7f6fc6a00640 1 --2- 172.21.15.39:0/2153024989 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6fc0160250 0x7f6fc0141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:25.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.576+0000 7f6fc6a00640 1 --2- 172.21.15.39:0/2153024989 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6fc00cfc50 0x7f6fc00b1dc0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:25.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.576+0000 7f6fc6a00640 1 -- 172.21.15.39:0/2153024989 >> 172.21.15.39:0/2153024989 conn(0x7f6fc006cfc0 msgr2=0x7f6fc0068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:25.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.576+0000 7f6fc6a00640 1 -- 172.21.15.39:0/2153024989 shutdown_connections 2024-09-21T07:46:25.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.578+0000 7f6fc6a00640 1 -- 172.21.15.39:0/2153024989 wait complete. 2024-09-21T07:46:25.618 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T07:46:25.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.844+0000 7fde3b200640 1 -- 172.21.15.39:0/2111711397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde340b4c40 msgr2=0x7fde340ce2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:25.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.844+0000 7fde3b200640 1 --2- 172.21.15.39:0/2111711397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde340b4c40 0x7fde340ce2b0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fde340b2df0 tx=0x7fde2402a510 comp rx=0 tx=0).stop 2024-09-21T07:46:25.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.844+0000 7fde3b200640 1 -- 172.21.15.39:0/2111711397 shutdown_connections 2024-09-21T07:46:25.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.844+0000 7fde3b200640 1 --2- 172.21.15.39:0/2111711397 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde3406ef90 0x7fde3406f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:25.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.844+0000 7fde3b200640 1 --2- 172.21.15.39:0/2111711397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde340b4c40 0x7fde340ce2b0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:25.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.844+0000 7fde3b200640 1 -- 172.21.15.39:0/2111711397 >> 172.21.15.39:0/2111711397 conn(0x7fde340b2460 msgr2=0x7fde3406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:25.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.844+0000 7fde3b200640 1 -- 172.21.15.39:0/2111711397 shutdown_connections 2024-09-21T07:46:25.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.844+0000 7fde3b200640 1 -- 172.21.15.39:0/2111711397 wait complete. 2024-09-21T07:46:25.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.846+0000 7fde3b200640 1 Processor -- start 2024-09-21T07:46:25.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.847+0000 7fde3b200640 1 -- start start 2024-09-21T07:46:25.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.848+0000 7fde3b200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde3406ef90 0x7fde34092270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:25.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.848+0000 7fde3b200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde340b3670 0x7fde340927b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:25.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.848+0000 7fde3b200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde340b4c40 0x7fde341f6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:25.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.848+0000 7fde3b200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fde3406fa60 con 0x7fde340b3670 2024-09-21T07:46:25.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.848+0000 7fde3b200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fde3406f8e0 con 0x7fde3406ef90 2024-09-21T07:46:25.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.848+0000 7fde3b200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fde3406fbe0 con 0x7fde340b4c40 2024-09-21T07:46:25.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.848+0000 7fde2be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde340b3670 0x7fde340927b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:25.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.848+0000 7fde2be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde340b3670 0x7fde340927b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35502/0 (socket says 172.21.15.39:35502) 2024-09-21T07:46:25.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.848+0000 7fde2be00640 1 -- 172.21.15.39:0/270471401 learned_addr learned my addr 172.21.15.39:0/270471401 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:25.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.849+0000 7fde39200640 1 --2- 172.21.15.39:0/270471401 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde340b4c40 0x7fde341f6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:25.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.849+0000 7fde33e00640 1 --2- 172.21.15.39:0/270471401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde3406ef90 0x7fde34092270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:25.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.849+0000 7fde2be00640 1 -- 172.21.15.39:0/270471401 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde340b4c40 msgr2=0x7fde341f6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:25.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.849+0000 7fde2be00640 1 --2- 172.21.15.39:0/270471401 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde340b4c40 0x7fde341f6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:25.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.849+0000 7fde2be00640 1 -- 172.21.15.39:0/270471401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde3406ef90 msgr2=0x7fde34092270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:25.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.849+0000 7fde2be00640 1 --2- 172.21.15.39:0/270471401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde3406ef90 0x7fde34092270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:25.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.849+0000 7fde2be00640 1 -- 172.21.15.39:0/270471401 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde34159a00 con 0x7fde340b3670 2024-09-21T07:46:25.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.849+0000 7fde2be00640 1 --2- 172.21.15.39:0/270471401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde340b3670 0x7fde340927b0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fde1c00b420 tx=0x7fde1c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:25.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.850+0000 7fde30c00640 1 -- 172.21.15.39:0/270471401 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1c011830 con 0x7fde340b3670 2024-09-21T07:46:25.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.850+0000 7fde30c00640 1 -- 172.21.15.39:0/270471401 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde1c016070 con 0x7fde340b3670 2024-09-21T07:46:25.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.850+0000 7fde30c00640 1 -- 172.21.15.39:0/270471401 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1c011d60 con 0x7fde340b3670 2024-09-21T07:46:25.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.850+0000 7fde3b200640 1 -- 172.21.15.39:0/270471401 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde34159ce0 con 0x7fde340b3670 2024-09-21T07:46:25.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.850+0000 7fde3b200640 1 -- 172.21.15.39:0/270471401 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde3415a2a0 con 0x7fde340b3670 2024-09-21T07:46:25.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.851+0000 7fde3b200640 1 -- 172.21.15.39:0/270471401 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde3406f3a0 con 0x7fde340b3670 2024-09-21T07:46:25.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.852+0000 7fde30c00640 1 -- 172.21.15.39:0/270471401 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fde1c01b460 con 0x7fde340b3670 2024-09-21T07:46:25.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.853+0000 7fde30c00640 1 --2- 172.21.15.39:0/270471401 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fde10078bc0 0x7fde1007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:25.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.853+0000 7fde30c00640 1 -- 172.21.15.39:0/270471401 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 6788+0+0 (secure 0 0 0) 0x7fde1c09ab60 con 0x7fde340b3670 2024-09-21T07:46:25.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.854+0000 7fde33e00640 1 --2- 172.21.15.39:0/270471401 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fde10078bc0 0x7fde1007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:25.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.855+0000 7fde33e00640 1 --2- 172.21.15.39:0/270471401 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fde10078bc0 0x7fde1007b0d0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fde24033000 tx=0x7fde24004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:25.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:25.858+0000 7fde30c00640 1 -- 172.21.15.39:0/270471401 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fde1c020020 con 0x7fde340b3670 2024-09-21T07:46:26.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.098+0000 7fde3b200640 1 -- 172.21.15.39:0/270471401 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fde34159790 con 0x7fde340b3670 2024-09-21T07:46:26.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.588+0000 7fde30c00640 1 -- 172.21.15.39:0/270471401 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 21 and data pool 22 v186) ==== 174+0+0 (secure 0 0 0) 0x7fde1c062500 con 0x7fde340b3670 2024-09-21T07:46:26.591 INFO:teuthology.orchestra.run.smithi039.stderr:new fs with metadata pool 21 and data pool 22 2024-09-21T07:46:26.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.601+0000 7fde3b200640 1 -- 172.21.15.39:0/270471401 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fde10078bc0 msgr2=0x7fde1007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:26.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.601+0000 7fde3b200640 1 --2- 172.21.15.39:0/270471401 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fde10078bc0 0x7fde1007b0d0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fde24033000 tx=0x7fde24004730 comp rx=0 tx=0).stop 2024-09-21T07:46:26.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.601+0000 7fde3b200640 1 -- 172.21.15.39:0/270471401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde340b3670 msgr2=0x7fde340927b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:26.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.601+0000 7fde3b200640 1 --2- 172.21.15.39:0/270471401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde340b3670 0x7fde340927b0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fde1c00b420 tx=0x7fde1c00b8e0 comp rx=0 tx=0).stop 2024-09-21T07:46:26.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.601+0000 7fde3b200640 1 -- 172.21.15.39:0/270471401 shutdown_connections 2024-09-21T07:46:26.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.601+0000 7fde3b200640 1 --2- 172.21.15.39:0/270471401 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fde10078bc0 0x7fde1007b0d0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:26.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.601+0000 7fde3b200640 1 --2- 172.21.15.39:0/270471401 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde340b4c40 0x7fde341f6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:26.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.601+0000 7fde3b200640 1 --2- 172.21.15.39:0/270471401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde340b3670 0x7fde340927b0 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:26.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.601+0000 7fde3b200640 1 --2- 172.21.15.39:0/270471401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde3406ef90 0x7fde34092270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:26.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.601+0000 7fde3b200640 1 -- 172.21.15.39:0/270471401 >> 172.21.15.39:0/270471401 conn(0x7fde340b2460 msgr2=0x7fde340d0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:26.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.602+0000 7fde3b200640 1 -- 172.21.15.39:0/270471401 shutdown_connections 2024-09-21T07:46:26.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.603+0000 7fde3b200640 1 -- 172.21.15.39:0/270471401 wait complete. 2024-09-21T07:46:26.643 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:46:26.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.869+0000 7f726ea00640 1 -- 172.21.15.39:0/2927957022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72680cb8a0 msgr2=0x7f7268156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:26.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.869+0000 7f726ea00640 1 --2- 172.21.15.39:0/2927957022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72680cb8a0 0x7f7268156bb0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f7254005fd0 tx=0x7f725402a540 comp rx=0 tx=0).stop 2024-09-21T07:46:26.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.869+0000 7f726ea00640 1 -- 172.21.15.39:0/2927957022 shutdown_connections 2024-09-21T07:46:26.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.869+0000 7f726ea00640 1 --2- 172.21.15.39:0/2927957022 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72680ccc60 0x7f72680cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:26.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.869+0000 7f726ea00640 1 --2- 172.21.15.39:0/2927957022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72680cb8a0 0x7f7268156bb0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:26.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.869+0000 7f726ea00640 1 -- 172.21.15.39:0/2927957022 >> 172.21.15.39:0/2927957022 conn(0x7f72680b2460 msgr2=0x7f726806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:26.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.869+0000 7f726ea00640 1 -- 172.21.15.39:0/2927957022 shutdown_connections 2024-09-21T07:46:26.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.869+0000 7f726ea00640 1 -- 172.21.15.39:0/2927957022 wait complete. 2024-09-21T07:46:26.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.871+0000 7f726ea00640 1 Processor -- start 2024-09-21T07:46:26.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.871+0000 7f726ea00640 1 -- start start 2024-09-21T07:46:26.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.873+0000 7f726ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72680b30a0 0x7f72680c5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:26.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.873+0000 7f726ea00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72680cb8a0 0x7f72681f3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:26.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.873+0000 7f726ea00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72680ccc60 0x7f726815eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:26.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.873+0000 7f726ea00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f72680af410 con 0x7f72680b30a0 2024-09-21T07:46:26.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.873+0000 7f726ea00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f72680af290 con 0x7f72680ccc60 2024-09-21T07:46:26.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.873+0000 7f726ea00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f72680af590 con 0x7f72680cb8a0 2024-09-21T07:46:26.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.873+0000 7f7267e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72680b30a0 0x7f72680c5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:26.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.873+0000 7f7267e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72680b30a0 0x7f72680c5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35522/0 (socket says 172.21.15.39:35522) 2024-09-21T07:46:26.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.873+0000 7f7267e00640 1 -- 172.21.15.39:0/307372171 learned_addr learned my addr 172.21.15.39:0/307372171 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:26.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.873+0000 7f7267400640 1 --2- 172.21.15.39:0/307372171 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72680cb8a0 0x7f72681f3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:26.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.873+0000 7f726ca00640 1 --2- 172.21.15.39:0/307372171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72680ccc60 0x7f726815eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:26.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.873+0000 7f7267e00640 1 -- 172.21.15.39:0/307372171 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72680cb8a0 msgr2=0x7f72681f3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:26.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.873+0000 7f7267e00640 1 --2- 172.21.15.39:0/307372171 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72680cb8a0 0x7f72681f3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:26.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.873+0000 7f7267e00640 1 -- 172.21.15.39:0/307372171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72680ccc60 msgr2=0x7f726815eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:26.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.873+0000 7f7267e00640 1 --2- 172.21.15.39:0/307372171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72680ccc60 0x7f726815eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:26.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.874+0000 7f7267e00640 1 -- 172.21.15.39:0/307372171 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72681f72a0 con 0x7f72680b30a0 2024-09-21T07:46:26.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.874+0000 7f726ca00640 1 --2- 172.21.15.39:0/307372171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72680ccc60 0x7f726815eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:26.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.874+0000 7f7267e00640 1 --2- 172.21.15.39:0/307372171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72680b30a0 0x7f72680c5660 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f725800c3c0 tx=0x7f725800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:26.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.874+0000 7f725fe00640 1 -- 172.21.15.39:0/307372171 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7258017070 con 0x7f72680b30a0 2024-09-21T07:46:26.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.874+0000 7f725fe00640 1 -- 172.21.15.39:0/307372171 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f72580035f0 con 0x7f72680b30a0 2024-09-21T07:46:26.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.874+0000 7f725fe00640 1 -- 172.21.15.39:0/307372171 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7258003a40 con 0x7f72680b30a0 2024-09-21T07:46:26.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.874+0000 7f726ea00640 1 -- 172.21.15.39:0/307372171 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72681f7580 con 0x7f72680b30a0 2024-09-21T07:46:26.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.874+0000 7f726ea00640 1 -- 172.21.15.39:0/307372171 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72681f7b40 con 0x7f72680b30a0 2024-09-21T07:46:26.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.875+0000 7f726ea00640 1 -- 172.21.15.39:0/307372171 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72680b0c00 con 0x7f72680b30a0 2024-09-21T07:46:26.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.877+0000 7f725fe00640 1 -- 172.21.15.39:0/307372171 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f725801a070 con 0x7f72680b30a0 2024-09-21T07:46:26.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.878+0000 7f725fe00640 1 --2- 172.21.15.39:0/307372171 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7248078b90 0x7f724807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:26.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.878+0000 7f725fe00640 1 -- 172.21.15.39:0/307372171 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(174..174 src has 1..174) ==== 6888+0+0 (secure 0 0 0) 0x7f725809a410 con 0x7f72680b30a0 2024-09-21T07:46:26.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.878+0000 7f7267400640 1 --2- 172.21.15.39:0/307372171 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7248078b90 0x7f724807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:26.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.879+0000 7f7267400640 1 --2- 172.21.15.39:0/307372171 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7248078b90 0x7f724807b0a0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f7254033000 tx=0x7f7254004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:26.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:26.883+0000 7f725fe00640 1 -- 172.21.15.39:0/307372171 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f72680b0c00 con 0x7f72680b30a0 2024-09-21T07:46:27.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.084+0000 7f726ea00640 1 -- 172.21.15.39:0/307372171 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f72680afae0 con 0x7f72680b30a0 2024-09-21T07:46:27.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.085+0000 7f725fe00640 1 -- 172.21.15.39:0/307372171 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v174) ==== 74+0+22297 (secure 0 0 0) 0x7f72680cd5b0 con 0x7f72680b30a0 2024-09-21T07:46:27.087 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:46:27.088 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":174,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:46:26.100557+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:46:24.067206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-09-21T07:46:25.054230+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]},{"pool":20,"snaps":[{"begin":80,"length":23},{"begin":181,"length":23}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:46:27.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.088+0000 7f726ea00640 1 -- 172.21.15.39:0/307372171 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7248078b90 msgr2=0x7f724807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:27.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.088+0000 7f726ea00640 1 --2- 172.21.15.39:0/307372171 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7248078b90 0x7f724807b0a0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f7254033000 tx=0x7f7254004730 comp rx=0 tx=0).stop 2024-09-21T07:46:27.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.088+0000 7f726ea00640 1 -- 172.21.15.39:0/307372171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72680b30a0 msgr2=0x7f72680c5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:27.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.088+0000 7f726ea00640 1 --2- 172.21.15.39:0/307372171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72680b30a0 0x7f72680c5660 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f725800c3c0 tx=0x7f725800c880 comp rx=0 tx=0).stop 2024-09-21T07:46:27.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.088+0000 7f726ea00640 1 -- 172.21.15.39:0/307372171 shutdown_connections 2024-09-21T07:46:27.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.088+0000 7f726ea00640 1 --2- 172.21.15.39:0/307372171 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7248078b90 0x7f724807b0a0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:27.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.088+0000 7f726ea00640 1 --2- 172.21.15.39:0/307372171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72680ccc60 0x7f726815eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:27.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.088+0000 7f726ea00640 1 --2- 172.21.15.39:0/307372171 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72680cb8a0 0x7f72681f3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:27.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.088+0000 7f726ea00640 1 --2- 172.21.15.39:0/307372171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72680b30a0 0x7f72680c5660 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:27.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.088+0000 7f726ea00640 1 -- 172.21.15.39:0/307372171 >> 172.21.15.39:0/307372171 conn(0x7f72680b2460 msgr2=0x7f72680b04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:27.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.088+0000 7f726ea00640 1 -- 172.21.15.39:0/307372171 shutdown_connections 2024-09-21T07:46:27.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.088+0000 7f726ea00640 1 -- 172.21.15.39:0/307372171 wait complete. 2024-09-21T07:46:27.114 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:46:27.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.341+0000 7f8811600640 1 -- 172.21.15.39:0/2160267636 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f880c0b3b50 msgr2=0x7f880c15c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:27.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.341+0000 7f8811600640 1 --2- 172.21.15.39:0/2160267636 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f880c0b3b50 0x7f880c15c510 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f87fc009830 tx=0x7f87fc02a410 comp rx=0 tx=0).stop 2024-09-21T07:46:27.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.341+0000 7f8811600640 1 -- 172.21.15.39:0/2160267636 shutdown_connections 2024-09-21T07:46:27.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.341+0000 7f8811600640 1 --2- 172.21.15.39:0/2160267636 >> v2:172.21.15.148:3300/0 conn(0x7f880c073f90 0x7f880c0d04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:27.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.341+0000 7f8811600640 1 --2- 172.21.15.39:0/2160267636 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f880c0b3b50 0x7f880c15c510 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:27.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.341+0000 7f8811600640 1 -- 172.21.15.39:0/2160267636 >> 172.21.15.39:0/2160267636 conn(0x7f880c06cfc0 msgr2=0x7f880c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:27.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.341+0000 7f8811600640 1 -- 172.21.15.39:0/2160267636 shutdown_connections 2024-09-21T07:46:27.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.341+0000 7f8811600640 1 -- 172.21.15.39:0/2160267636 wait complete. 2024-09-21T07:46:27.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.343+0000 7f8811600640 1 Processor -- start 2024-09-21T07:46:27.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.343+0000 7f8811600640 1 -- start start 2024-09-21T07:46:27.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.345+0000 7f8811600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f880c073f90 0x7f880c15add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:27.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.345+0000 7f8811600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f880c0743a0 0x7f880c13f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:27.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.345+0000 7f8811600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f880c0b3b50 0x7f880c13f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:27.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.345+0000 7f8811600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f880c155d30 con 0x7f880c0b3b50 2024-09-21T07:46:27.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.345+0000 7f8811600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f880c155bb0 con 0x7f880c073f90 2024-09-21T07:46:27.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.345+0000 7f8811600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f880c155eb0 con 0x7f880c0743a0 2024-09-21T07:46:27.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.345+0000 7f8810c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f880c0b3b50 0x7f880c13f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:27.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.345+0000 7f8810c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f880c0b3b50 0x7f880c13f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35526/0 (socket says 172.21.15.39:35526) 2024-09-21T07:46:27.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.345+0000 7f8810c00640 1 -- 172.21.15.39:0/3995322032 learned_addr learned my addr 172.21.15.39:0/3995322032 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:27.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.345+0000 7f8803e00640 1 --2- 172.21.15.39:0/3995322032 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f880c0743a0 0x7f880c13f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:27.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.345+0000 7f880be00640 1 --2- 172.21.15.39:0/3995322032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f880c073f90 0x7f880c15add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:27.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.345+0000 7f8810c00640 1 -- 172.21.15.39:0/3995322032 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f880c0743a0 msgr2=0x7f880c13f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:27.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.345+0000 7f8810c00640 1 --2- 172.21.15.39:0/3995322032 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f880c0743a0 0x7f880c13f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:27.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.345+0000 7f8810c00640 1 -- 172.21.15.39:0/3995322032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f880c073f90 msgr2=0x7f880c15add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:27.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.345+0000 7f8810c00640 1 --2- 172.21.15.39:0/3995322032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f880c073f90 0x7f880c15add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:27.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.345+0000 7f8810c00640 1 -- 172.21.15.39:0/3995322032 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f880c13eb20 con 0x7f880c0b3b50 2024-09-21T07:46:27.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.346+0000 7f8803e00640 1 --2- 172.21.15.39:0/3995322032 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f880c0743a0 0x7f880c13f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:27.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.346+0000 7f8810c00640 1 --2- 172.21.15.39:0/3995322032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f880c0b3b50 0x7f880c13f820 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f880400cbe0 tx=0x7f880400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:27.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.346+0000 7f8808c00640 1 -- 172.21.15.39:0/3995322032 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8804016020 con 0x7f880c0b3b50 2024-09-21T07:46:27.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.346+0000 7f8808c00640 1 -- 172.21.15.39:0/3995322032 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f880400ae50 con 0x7f880c0b3b50 2024-09-21T07:46:27.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.346+0000 7f8808c00640 1 -- 172.21.15.39:0/3995322032 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8804012680 con 0x7f880c0b3b50 2024-09-21T07:46:27.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.346+0000 7f8811600640 1 -- 172.21.15.39:0/3995322032 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f880c14fb40 con 0x7f880c0b3b50 2024-09-21T07:46:27.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.346+0000 7f8811600640 1 -- 172.21.15.39:0/3995322032 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f880c150100 con 0x7f880c0b3b50 2024-09-21T07:46:27.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.347+0000 7f8811600640 1 -- 172.21.15.39:0/3995322032 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f880c0d04b0 con 0x7f880c0b3b50 2024-09-21T07:46:27.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.349+0000 7f8808c00640 1 -- 172.21.15.39:0/3995322032 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8804019020 con 0x7f880c0b3b50 2024-09-21T07:46:27.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.350+0000 7f8808c00640 1 --2- 172.21.15.39:0/3995322032 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f87e8078b90 0x7f87e807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:27.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.350+0000 7f8808c00640 1 -- 172.21.15.39:0/3995322032 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(174..174 src has 1..174) ==== 6888+0+0 (secure 0 0 0) 0x7f880409b780 con 0x7f880c0b3b50 2024-09-21T07:46:27.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.350+0000 7f880be00640 1 --2- 172.21.15.39:0/3995322032 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f87e8078b90 0x7f87e807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:27.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.351+0000 7f880be00640 1 --2- 172.21.15.39:0/3995322032 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f87e8078b90 0x7f87e807b0a0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f87fc034000 tx=0x7f87fc004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:27.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.354+0000 7f8808c00640 1 -- 172.21.15.39:0/3995322032 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8804020020 con 0x7f880c0b3b50 2024-09-21T07:46:27.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.553+0000 7f8811600640 1 -- 172.21.15.39:0/3995322032 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f880c06ffc0 con 0x7f880c0b3b50 2024-09-21T07:46:27.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.555+0000 7f8808c00640 1 -- 172.21.15.39:0/3995322032 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v174) ==== 74+0+22297 (secure 0 0 0) 0x7f88040637b0 con 0x7f880c0b3b50 2024-09-21T07:46:27.557 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:46:27.557 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":174,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:46:26.100557+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:46:24.067206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-09-21T07:46:25.054230+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]},{"pool":20,"snaps":[{"begin":80,"length":23},{"begin":181,"length":23}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:46:27.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.557+0000 7f8811600640 1 -- 172.21.15.39:0/3995322032 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f87e8078b90 msgr2=0x7f87e807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:27.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.557+0000 7f8811600640 1 --2- 172.21.15.39:0/3995322032 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f87e8078b90 0x7f87e807b0a0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f87fc034000 tx=0x7f87fc004730 comp rx=0 tx=0).stop 2024-09-21T07:46:27.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.557+0000 7f8811600640 1 -- 172.21.15.39:0/3995322032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f880c0b3b50 msgr2=0x7f880c13f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:27.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.557+0000 7f8811600640 1 --2- 172.21.15.39:0/3995322032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f880c0b3b50 0x7f880c13f820 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f880400cbe0 tx=0x7f880400a430 comp rx=0 tx=0).stop 2024-09-21T07:46:27.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.557+0000 7f8811600640 1 -- 172.21.15.39:0/3995322032 shutdown_connections 2024-09-21T07:46:27.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.557+0000 7f8811600640 1 --2- 172.21.15.39:0/3995322032 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f87e8078b90 0x7f87e807b0a0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:27.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.557+0000 7f8811600640 1 --2- 172.21.15.39:0/3995322032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f880c0b3b50 0x7f880c13f820 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:27.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.557+0000 7f8811600640 1 --2- 172.21.15.39:0/3995322032 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f880c0743a0 0x7f880c13f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:27.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.557+0000 7f8811600640 1 --2- 172.21.15.39:0/3995322032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f880c073f90 0x7f880c15add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:27.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.557+0000 7f8811600640 1 -- 172.21.15.39:0/3995322032 >> 172.21.15.39:0/3995322032 conn(0x7f880c06cfc0 msgr2=0x7f880c155480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:27.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.557+0000 7f8811600640 1 -- 172.21.15.39:0/3995322032 shutdown_connections 2024-09-21T07:46:27.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.557+0000 7f8811600640 1 -- 172.21.15.39:0/3995322032 wait complete. 2024-09-21T07:46:27.585 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T07:46:27.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.816+0000 7fb017e00640 1 -- 172.21.15.39:0/2825271470 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0180d0230 msgr2=0x7fb01815d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:27.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.816+0000 7fb017e00640 1 --2- 172.21.15.39:0/2825271470 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0180d0230 0x7fb01815d700 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fb008005fd0 tx=0x7fb00802a5f0 comp rx=0 tx=0).stop 2024-09-21T07:46:27.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.817+0000 7fb017e00640 1 -- 172.21.15.39:0/2825271470 shutdown_connections 2024-09-21T07:46:27.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.817+0000 7fb017e00640 1 --2- 172.21.15.39:0/2825271470 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0180d0230 0x7fb01815d700 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:27.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.817+0000 7fb017e00640 1 -- 172.21.15.39:0/2825271470 >> 172.21.15.39:0/2825271470 conn(0x7fb01806def0 msgr2=0x7fb01806e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:27.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.817+0000 7fb017e00640 1 -- 172.21.15.39:0/2825271470 shutdown_connections 2024-09-21T07:46:27.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.817+0000 7fb017e00640 1 -- 172.21.15.39:0/2825271470 wait complete. 2024-09-21T07:46:27.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.819+0000 7fb017e00640 1 Processor -- start 2024-09-21T07:46:27.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.819+0000 7fb017e00640 1 -- start start 2024-09-21T07:46:27.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.820+0000 7fb017e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb0180cdb00 0x7fb0181413c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:27.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.820+0000 7fb017e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0180d0230 0x7fb0180ae200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:27.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.821+0000 7fb017e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb018156600 0x7fb018155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:27.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.821+0000 7fb017e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb0180ce2a0 con 0x7fb0180d0230 2024-09-21T07:46:27.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.821+0000 7fb017e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb0180ce120 con 0x7fb018156600 2024-09-21T07:46:27.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.821+0000 7fb017e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb018074c10 con 0x7fb0180cdb00 2024-09-21T07:46:27.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.821+0000 7fb00fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0180d0230 0x7fb0180ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:27.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.821+0000 7fb00fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0180d0230 0x7fb0180ae200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35532/0 (socket says 172.21.15.39:35532) 2024-09-21T07:46:27.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.821+0000 7fb00fe00640 1 -- 172.21.15.39:0/2337638823 learned_addr learned my addr 172.21.15.39:0/2337638823 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:27.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.821+0000 7fb016a00640 1 --2- 172.21.15.39:0/2337638823 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb0180cdb00 0x7fb0181413c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:27.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.821+0000 7fb017400640 1 --2- 172.21.15.39:0/2337638823 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb018156600 0x7fb018155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:27.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.821+0000 7fb00fe00640 1 -- 172.21.15.39:0/2337638823 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb0180cdb00 msgr2=0x7fb0181413c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:27.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.821+0000 7fb00fe00640 1 --2- 172.21.15.39:0/2337638823 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb0180cdb00 0x7fb0181413c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:27.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.821+0000 7fb00fe00640 1 -- 172.21.15.39:0/2337638823 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb018156600 msgr2=0x7fb018155d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:27.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.821+0000 7fb00fe00640 1 --2- 172.21.15.39:0/2337638823 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb018156600 0x7fb018155d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:27.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.821+0000 7fb00fe00640 1 -- 172.21.15.39:0/2337638823 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0181562a0 con 0x7fb0180d0230 2024-09-21T07:46:27.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.822+0000 7fb00fe00640 1 --2- 172.21.15.39:0/2337638823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0180d0230 0x7fb0180ae200 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fb00000c3a0 tx=0x7fb00000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:27.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.822+0000 7fb00ea00640 1 -- 172.21.15.39:0/2337638823 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb000017070 con 0x7fb0180d0230 2024-09-21T07:46:27.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.822+0000 7fb00ea00640 1 -- 172.21.15.39:0/2337638823 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0000035f0 con 0x7fb0180d0230 2024-09-21T07:46:27.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.822+0000 7fb017e00640 1 -- 172.21.15.39:0/2337638823 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb018140ae0 con 0x7fb0180d0230 2024-09-21T07:46:27.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.822+0000 7fb00ea00640 1 -- 172.21.15.39:0/2337638823 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb000003a40 con 0x7fb0180d0230 2024-09-21T07:46:27.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.822+0000 7fb017e00640 1 -- 172.21.15.39:0/2337638823 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0180ae7d0 con 0x7fb0180d0230 2024-09-21T07:46:27.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.823+0000 7fb017e00640 1 -- 172.21.15.39:0/2337638823 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0180b1ac0 con 0x7fb0180d0230 2024-09-21T07:46:27.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.825+0000 7fb00ea00640 1 -- 172.21.15.39:0/2337638823 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb00001a070 con 0x7fb0180d0230 2024-09-21T07:46:27.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.826+0000 7fb00ea00640 1 --2- 172.21.15.39:0/2337638823 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faff4078bc0 0x7faff407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:27.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.826+0000 7fb00ea00640 1 -- 172.21.15.39:0/2337638823 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 6888+0+0 (secure 0 0 0) 0x7fb00009a3f0 con 0x7fb0180d0230 2024-09-21T07:46:27.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.826+0000 7fb016a00640 1 --2- 172.21.15.39:0/2337638823 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faff4078bc0 0x7faff407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:27.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.827+0000 7fb016a00640 1 --2- 172.21.15.39:0/2337638823 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faff4078bc0 0x7faff407b0d0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fb008033000 tx=0x7fb008004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:27.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:27.830+0000 7fb00ea00640 1 -- 172.21.15.39:0/2337638823 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb0180b1ac0 con 0x7fb0180d0230 2024-09-21T07:46:28.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.066+0000 7fb017e00640 1 -- 172.21.15.39:0/2337638823 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb018069450 con 0x7fb0180d0230 2024-09-21T07:46:28.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.606+0000 7fb00ea00640 1 -- 172.21.15.39:0/2337638823 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v189) ==== 118+0+0 (secure 0 0 0) 0x7fb018069450 con 0x7fb0180d0230 2024-09-21T07:46:28.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.609+0000 7fb017e00640 1 -- 172.21.15.39:0/2337638823 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faff4078bc0 msgr2=0x7faff407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:28.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.609+0000 7fb017e00640 1 --2- 172.21.15.39:0/2337638823 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faff4078bc0 0x7faff407b0d0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fb008033000 tx=0x7fb008004730 comp rx=0 tx=0).stop 2024-09-21T07:46:28.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.609+0000 7fb017e00640 1 -- 172.21.15.39:0/2337638823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0180d0230 msgr2=0x7fb0180ae200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:28.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.609+0000 7fb017e00640 1 --2- 172.21.15.39:0/2337638823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0180d0230 0x7fb0180ae200 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fb00000c3a0 tx=0x7fb00000c860 comp rx=0 tx=0).stop 2024-09-21T07:46:28.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.609+0000 7fb017e00640 1 -- 172.21.15.39:0/2337638823 shutdown_connections 2024-09-21T07:46:28.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.609+0000 7fb017e00640 1 --2- 172.21.15.39:0/2337638823 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faff4078bc0 0x7faff407b0d0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:28.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.609+0000 7fb017e00640 1 --2- 172.21.15.39:0/2337638823 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb018156600 0x7fb018155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:28.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.609+0000 7fb017e00640 1 --2- 172.21.15.39:0/2337638823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0180d0230 0x7fb0180ae200 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:28.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.609+0000 7fb017e00640 1 --2- 172.21.15.39:0/2337638823 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb0180cdb00 0x7fb0181413c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:28.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.609+0000 7fb017e00640 1 -- 172.21.15.39:0/2337638823 >> 172.21.15.39:0/2337638823 conn(0x7fb01806def0 msgr2=0x7fb0180b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:28.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.609+0000 7fb017e00640 1 -- 172.21.15.39:0/2337638823 shutdown_connections 2024-09-21T07:46:28.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.609+0000 7fb017e00640 1 -- 172.21.15.39:0/2337638823 wait complete. 2024-09-21T07:46:28.635 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T07:46:28.635 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T07:46:28.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.864+0000 7fbe29c00640 1 -- 172.21.15.39:0/1916685435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe24160660 msgr2=0x7fbe240cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:28.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.864+0000 7fbe29c00640 1 --2- 172.21.15.39:0/1916685435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe24160660 0x7fbe240cf710 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c005fd0 tx=0x7fbe0c02a540 comp rx=0 tx=0).stop 2024-09-21T07:46:28.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.864+0000 7fbe29c00640 1 -- 172.21.15.39:0/1916685435 shutdown_connections 2024-09-21T07:46:28.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.864+0000 7fbe29c00640 1 --2- 172.21.15.39:0/1916685435 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe240cfc50 0x7fbe240d0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:28.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.864+0000 7fbe29c00640 1 --2- 172.21.15.39:0/1916685435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe24160660 0x7fbe240cf710 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:28.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.864+0000 7fbe29c00640 1 -- 172.21.15.39:0/1916685435 >> 172.21.15.39:0/1916685435 conn(0x7fbe2406cfc0 msgr2=0x7fbe2406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:28.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.864+0000 7fbe29c00640 1 -- 172.21.15.39:0/1916685435 shutdown_connections 2024-09-21T07:46:28.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.864+0000 7fbe29c00640 1 -- 172.21.15.39:0/1916685435 wait complete. 2024-09-21T07:46:28.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.866+0000 7fbe29c00640 1 Processor -- start 2024-09-21T07:46:28.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.866+0000 7fbe29c00640 1 -- start start 2024-09-21T07:46:28.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.868+0000 7fbe29c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe240cfc50 0x7fbe24144990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:28.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.868+0000 7fbe29c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe24160250 0x7fbe240b4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:28.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.868+0000 7fbe29c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe24160660 0x7fbe240b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:28.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.868+0000 7fbe29c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbe240ccdc0 con 0x7fbe24160660 2024-09-21T07:46:28.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.868+0000 7fbe29c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbe240ccc40 con 0x7fbe24160250 2024-09-21T07:46:28.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.868+0000 7fbe29c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fbe240ccf40 con 0x7fbe240cfc50 2024-09-21T07:46:28.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.868+0000 7fbe29200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe24160660 0x7fbe240b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:28.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.868+0000 7fbe29200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe24160660 0x7fbe240b4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35540/0 (socket says 172.21.15.39:35540) 2024-09-21T07:46:28.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.868+0000 7fbe29200640 1 -- 172.21.15.39:0/2846752823 learned_addr learned my addr 172.21.15.39:0/2846752823 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:28.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.869+0000 7fbe23e00640 1 --2- 172.21.15.39:0/2846752823 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe240cfc50 0x7fbe24144990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:28.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.869+0000 7fbe1b400640 1 --2- 172.21.15.39:0/2846752823 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe24160250 0x7fbe240b4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:28.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.869+0000 7fbe29200640 1 -- 172.21.15.39:0/2846752823 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe240cfc50 msgr2=0x7fbe24144990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:28.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.869+0000 7fbe29200640 1 --2- 172.21.15.39:0/2846752823 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe240cfc50 0x7fbe24144990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:28.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.869+0000 7fbe29200640 1 -- 172.21.15.39:0/2846752823 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe24160250 msgr2=0x7fbe240b4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:28.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.869+0000 7fbe29200640 1 --2- 172.21.15.39:0/2846752823 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe24160250 0x7fbe240b4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:28.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.869+0000 7fbe29200640 1 -- 172.21.15.39:0/2846752823 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe241f9fe0 con 0x7fbe24160660 2024-09-21T07:46:28.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.869+0000 7fbe29200640 1 --2- 172.21.15.39:0/2846752823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe24160660 0x7fbe240b4ba0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c00cad0 tx=0x7fbe1c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:28.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.869+0000 7fbe20c00640 1 -- 172.21.15.39:0/2846752823 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c00ae20 con 0x7fbe24160660 2024-09-21T07:46:28.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.869+0000 7fbe20c00640 1 -- 172.21.15.39:0/2846752823 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe1c00f450 con 0x7fbe24160660 2024-09-21T07:46:28.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.869+0000 7fbe20c00640 1 -- 172.21.15.39:0/2846752823 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c00f8c0 con 0x7fbe24160660 2024-09-21T07:46:28.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.869+0000 7fbe29c00640 1 -- 172.21.15.39:0/2846752823 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe2415c130 con 0x7fbe24160660 2024-09-21T07:46:28.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.869+0000 7fbe29c00640 1 -- 172.21.15.39:0/2846752823 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe241f9b00 con 0x7fbe24160660 2024-09-21T07:46:28.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.870+0000 7fbe29c00640 1 -- 172.21.15.39:0/2846752823 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe241f9d30 con 0x7fbe24160660 2024-09-21T07:46:28.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.872+0000 7fbe20c00640 1 -- 172.21.15.39:0/2846752823 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbe1c00fa60 con 0x7fbe24160660 2024-09-21T07:46:28.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.873+0000 7fbe20c00640 1 --2- 172.21.15.39:0/2846752823 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbe00078bc0 0x7fbe0007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:28.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.873+0000 7fbe20c00640 1 -- 172.21.15.39:0/2846752823 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 6888+0+0 (secure 0 0 0) 0x7fbe1c09be00 con 0x7fbe24160660 2024-09-21T07:46:28.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.873+0000 7fbe23e00640 1 --2- 172.21.15.39:0/2846752823 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbe00078bc0 0x7fbe0007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:28.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.874+0000 7fbe23e00640 1 --2- 172.21.15.39:0/2846752823 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbe00078bc0 0x7fbe0007b0d0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fbe14006fd0 tx=0x7fbe14008040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:28.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:28.877+0000 7fbe20c00640 1 -- 172.21.15.39:0/2846752823 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbe1c014030 con 0x7fbe24160660 2024-09-21T07:46:29.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.113+0000 7fbe29c00640 1 -- 172.21.15.39:0/2846752823 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fbe2415f3e0 con 0x7fbe24160660 2024-09-21T07:46:29.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.625+0000 7fbe20c00640 1 -- 172.21.15.39:0/2846752823 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v190) ==== 122+0+0 (secure 0 0 0) 0x7fbe1c063d90 con 0x7fbe24160660 2024-09-21T07:46:29.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.628+0000 7fbe29c00640 1 -- 172.21.15.39:0/2846752823 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbe00078bc0 msgr2=0x7fbe0007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:29.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.628+0000 7fbe29c00640 1 --2- 172.21.15.39:0/2846752823 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbe00078bc0 0x7fbe0007b0d0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fbe14006fd0 tx=0x7fbe14008040 comp rx=0 tx=0).stop 2024-09-21T07:46:29.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.628+0000 7fbe29c00640 1 -- 172.21.15.39:0/2846752823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe24160660 msgr2=0x7fbe240b4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:29.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.628+0000 7fbe29c00640 1 --2- 172.21.15.39:0/2846752823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe24160660 0x7fbe240b4ba0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c00cad0 tx=0x7fbe1c00cf90 comp rx=0 tx=0).stop 2024-09-21T07:46:29.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.628+0000 7fbe29c00640 1 -- 172.21.15.39:0/2846752823 shutdown_connections 2024-09-21T07:46:29.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.628+0000 7fbe29c00640 1 --2- 172.21.15.39:0/2846752823 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbe00078bc0 0x7fbe0007b0d0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:29.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.628+0000 7fbe29c00640 1 --2- 172.21.15.39:0/2846752823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe24160660 0x7fbe240b4ba0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:29.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.628+0000 7fbe29c00640 1 --2- 172.21.15.39:0/2846752823 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe24160250 0x7fbe240b4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:29.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.628+0000 7fbe29c00640 1 --2- 172.21.15.39:0/2846752823 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe240cfc50 0x7fbe24144990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:29.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.628+0000 7fbe29c00640 1 -- 172.21.15.39:0/2846752823 >> 172.21.15.39:0/2846752823 conn(0x7fbe2406cfc0 msgr2=0x7fbe2406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:29.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.628+0000 7fbe29c00640 1 -- 172.21.15.39:0/2846752823 shutdown_connections 2024-09-21T07:46:29.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.628+0000 7fbe29c00640 1 -- 172.21.15.39:0/2846752823 wait complete. 2024-09-21T07:46:29.654 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T07:46:29.655 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:46:29.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.881+0000 7fded8200640 1 -- 172.21.15.39:0/3364507807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fded00cb8a0 msgr2=0x7fded0156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:29.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.881+0000 7fded8200640 1 --2- 172.21.15.39:0/3364507807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fded00cb8a0 0x7fded0156bb0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fdebc005fd0 tx=0x7fdebc02a540 comp rx=0 tx=0).stop 2024-09-21T07:46:29.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.881+0000 7fded8200640 1 -- 172.21.15.39:0/3364507807 shutdown_connections 2024-09-21T07:46:29.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.881+0000 7fded8200640 1 --2- 172.21.15.39:0/3364507807 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fded00ccc60 0x7fded00cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:29.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.881+0000 7fded8200640 1 --2- 172.21.15.39:0/3364507807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fded00cb8a0 0x7fded0156bb0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:29.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.881+0000 7fded8200640 1 -- 172.21.15.39:0/3364507807 >> 172.21.15.39:0/3364507807 conn(0x7fded00b2460 msgr2=0x7fded006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:29.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.881+0000 7fded8200640 1 -- 172.21.15.39:0/3364507807 shutdown_connections 2024-09-21T07:46:29.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.881+0000 7fded8200640 1 -- 172.21.15.39:0/3364507807 wait complete. 2024-09-21T07:46:29.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.883+0000 7fded8200640 1 Processor -- start 2024-09-21T07:46:29.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.883+0000 7fded8200640 1 -- start start 2024-09-21T07:46:29.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.885+0000 7fded8200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fded00b30a0 0x7fded00c4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:29.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.885+0000 7fded8200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fded00cb8a0 0x7fded01f54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:29.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.885+0000 7fded8200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fded00ccc60 0x7fded015ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:29.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.885+0000 7fded8200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fded00af410 con 0x7fded00ccc60 2024-09-21T07:46:29.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.885+0000 7fded8200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fded00af290 con 0x7fded00b30a0 2024-09-21T07:46:29.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.885+0000 7fded8200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fded00af590 con 0x7fded00cb8a0 2024-09-21T07:46:29.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.885+0000 7fded6200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fded00ccc60 0x7fded015ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:29.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.885+0000 7fded6200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fded00ccc60 0x7fded015ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35548/0 (socket says 172.21.15.39:35548) 2024-09-21T07:46:29.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.885+0000 7fded6200640 1 -- 172.21.15.39:0/2677273049 learned_addr learned my addr 172.21.15.39:0/2677273049 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:29.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.885+0000 7fded5800640 1 --2- 172.21.15.39:0/2677273049 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fded00b30a0 0x7fded00c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:29.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.885+0000 7fded4e00640 1 --2- 172.21.15.39:0/2677273049 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fded00cb8a0 0x7fded01f54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:29.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.886+0000 7fded6200640 1 -- 172.21.15.39:0/2677273049 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fded00cb8a0 msgr2=0x7fded01f54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:29.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.886+0000 7fded6200640 1 --2- 172.21.15.39:0/2677273049 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fded00cb8a0 0x7fded01f54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:29.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.886+0000 7fded6200640 1 -- 172.21.15.39:0/2677273049 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fded00b30a0 msgr2=0x7fded00c4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:29.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.886+0000 7fded6200640 1 --2- 172.21.15.39:0/2677273049 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fded00b30a0 0x7fded00c4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:29.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.886+0000 7fded6200640 1 -- 172.21.15.39:0/2677273049 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fded01f5e90 con 0x7fded00ccc60 2024-09-21T07:46:29.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.886+0000 7fded4e00640 1 --2- 172.21.15.39:0/2677273049 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fded00cb8a0 0x7fded01f54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:29.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.886+0000 7fded6200640 1 --2- 172.21.15.39:0/2677273049 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fded00ccc60 0x7fded015ed20 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fdec800ec20 tx=0x7fdec800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:29.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.886+0000 7fdecd600640 1 -- 172.21.15.39:0/2677273049 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec8003b70 con 0x7fded00ccc60 2024-09-21T07:46:29.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.886+0000 7fdecd600640 1 -- 172.21.15.39:0/2677273049 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdec8003e00 con 0x7fded00ccc60 2024-09-21T07:46:29.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.886+0000 7fded8200640 1 -- 172.21.15.39:0/2677273049 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fded01f6170 con 0x7fded00ccc60 2024-09-21T07:46:29.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.886+0000 7fdecd600640 1 -- 172.21.15.39:0/2677273049 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec8004280 con 0x7fded00ccc60 2024-09-21T07:46:29.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.886+0000 7fded8200640 1 -- 172.21.15.39:0/2677273049 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fded01f6730 con 0x7fded00ccc60 2024-09-21T07:46:29.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.888+0000 7fded8200640 1 -- 172.21.15.39:0/2677273049 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fded00afae0 con 0x7fded00ccc60 2024-09-21T07:46:29.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.889+0000 7fdecd600640 1 -- 172.21.15.39:0/2677273049 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdec8004420 con 0x7fded00ccc60 2024-09-21T07:46:29.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.890+0000 7fdecd600640 1 --2- 172.21.15.39:0/2677273049 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdeb0078b90 0x7fdeb007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:29.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.890+0000 7fdecd600640 1 -- 172.21.15.39:0/2677273049 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 6888+0+0 (secure 0 0 0) 0x7fdec809bff0 con 0x7fded00ccc60 2024-09-21T07:46:29.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.890+0000 7fded5800640 1 --2- 172.21.15.39:0/2677273049 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdeb0078b90 0x7fdeb007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:29.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.891+0000 7fded5800640 1 --2- 172.21.15.39:0/2677273049 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdeb0078b90 0x7fdeb007b0a0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fdec4001720 tx=0x7fdec4009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:29.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:29.894+0000 7fdecd600640 1 -- 172.21.15.39:0/2677273049 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdec8064020 con 0x7fded00ccc60 2024-09-21T07:46:30.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.130+0000 7fded8200640 1 -- 172.21.15.39:0/2677273049 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fded00b1860 con 0x7fded00ccc60 2024-09-21T07:46:30.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.130+0000 7fdecd600640 1 -- 172.21.15.39:0/2677273049 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 190 v190) ==== 95+0+5234 (secure 0 0 0) 0x7fdec8063920 con 0x7fded00ccc60 2024-09-21T07:46:30.132 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:46:30.132 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":190,"btime":"2024-09-21T07:46:29:608115+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7493,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/4129342618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4129342618},{"type":"v1","addr":"172.21.15.39:6834","nonce":4129342618}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":185},{"gid":7499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1125302960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1125302960},{"type":"v1","addr":"172.21.15.148:6836","nonce":1125302960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":185},{"gid":8431,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/585545787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":585545787},{"type":"v1","addr":"172.21.15.148:6837","nonce":585545787}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":185}],"filesystems":[{"mdsmap":{"epoch":190,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:46:26.100496+0000","modified":"2024-09-21T07:46:29.115340+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9180},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9180":{"gid":9180,"name":"c","rank":0,"incarnation":186,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/3000703195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3000703195},{"type":"v1","addr":"172.21.15.39:6835","nonce":3000703195}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9180,"qdb_cluster":[9180]},"id":9}]} 2024-09-21T07:46:30.132 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 190 2024-09-21T07:46:30.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.133+0000 7fded8200640 1 -- 172.21.15.39:0/2677273049 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdeb0078b90 msgr2=0x7fdeb007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:30.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.133+0000 7fded8200640 1 --2- 172.21.15.39:0/2677273049 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdeb0078b90 0x7fdeb007b0a0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fdec4001720 tx=0x7fdec4009040 comp rx=0 tx=0).stop 2024-09-21T07:46:30.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.133+0000 7fded8200640 1 -- 172.21.15.39:0/2677273049 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fded00ccc60 msgr2=0x7fded015ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:30.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.133+0000 7fded8200640 1 --2- 172.21.15.39:0/2677273049 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fded00ccc60 0x7fded015ed20 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fdec800ec20 tx=0x7fdec800c6a0 comp rx=0 tx=0).stop 2024-09-21T07:46:30.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.133+0000 7fded8200640 1 -- 172.21.15.39:0/2677273049 shutdown_connections 2024-09-21T07:46:30.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.133+0000 7fded8200640 1 --2- 172.21.15.39:0/2677273049 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdeb0078b90 0x7fdeb007b0a0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:30.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.133+0000 7fded8200640 1 --2- 172.21.15.39:0/2677273049 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fded00ccc60 0x7fded015ed20 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:30.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.133+0000 7fded8200640 1 --2- 172.21.15.39:0/2677273049 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fded00cb8a0 0x7fded01f54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:30.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.133+0000 7fded8200640 1 --2- 172.21.15.39:0/2677273049 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fded00b30a0 0x7fded00c4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:30.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.133+0000 7fded8200640 1 -- 172.21.15.39:0/2677273049 >> 172.21.15.39:0/2677273049 conn(0x7fded00b2460 msgr2=0x7fded00ada20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:30.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.133+0000 7fded8200640 1 -- 172.21.15.39:0/2677273049 shutdown_connections 2024-09-21T07:46:30.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.133+0000 7fded8200640 1 -- 172.21.15.39:0/2677273049 wait complete. 2024-09-21T07:46:30.159 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:46:30.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.385+0000 7f8962a00640 1 -- 172.21.15.39:0/3540350800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f895c073dd0 msgr2=0x7f895c15d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:30.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.385+0000 7f8962a00640 1 --2- 172.21.15.39:0/3540350800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f895c073dd0 0x7f895c15d5b0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f894c005fd0 tx=0x7f894c02a5d0 comp rx=0 tx=0).stop 2024-09-21T07:46:30.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.385+0000 7f8962a00640 1 -- 172.21.15.39:0/3540350800 shutdown_connections 2024-09-21T07:46:30.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.385+0000 7f8962a00640 1 --2- 172.21.15.39:0/3540350800 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f895c15daf0 0x7f895c0cf450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:30.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.385+0000 7f8962a00640 1 --2- 172.21.15.39:0/3540350800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f895c073dd0 0x7f895c15d5b0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:30.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.385+0000 7f8962a00640 1 -- 172.21.15.39:0/3540350800 >> 172.21.15.39:0/3540350800 conn(0x7f895c06cfc0 msgr2=0x7f895c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:30.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.385+0000 7f8962a00640 1 -- 172.21.15.39:0/3540350800 shutdown_connections 2024-09-21T07:46:30.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.385+0000 7f8962a00640 1 -- 172.21.15.39:0/3540350800 wait complete. 2024-09-21T07:46:30.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.387+0000 7f8962a00640 1 Processor -- start 2024-09-21T07:46:30.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.387+0000 7f8962a00640 1 -- start start 2024-09-21T07:46:30.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.389+0000 7f8962a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f895c073dd0 0x7f895c1f5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:30.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.389+0000 7f8962a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f895c0cf990 0x7f895c1ef210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:30.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.389+0000 7f8962a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f895c15daf0 0x7f895c1e5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:30.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.389+0000 7f8962a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f895c0cd1d0 con 0x7f895c15daf0 2024-09-21T07:46:30.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.389+0000 7f8962a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f895c0cd050 con 0x7f895c073dd0 2024-09-21T07:46:30.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.389+0000 7f8962a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f895c0cd350 con 0x7f895c0cf990 2024-09-21T07:46:30.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.389+0000 7f8960a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f895c15daf0 0x7f895c1e5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:30.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.389+0000 7f8960a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f895c15daf0 0x7f895c1e5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35578/0 (socket says 172.21.15.39:35578) 2024-09-21T07:46:30.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.389+0000 7f8960a00640 1 -- 172.21.15.39:0/2130355970 learned_addr learned my addr 172.21.15.39:0/2130355970 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:30.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.389+0000 7f895be00640 1 --2- 172.21.15.39:0/2130355970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f895c073dd0 0x7f895c1f5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:30.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.389+0000 7f8953400640 1 --2- 172.21.15.39:0/2130355970 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f895c0cf990 0x7f895c1ef210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:30.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.389+0000 7f8960a00640 1 -- 172.21.15.39:0/2130355970 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f895c0cf990 msgr2=0x7f895c1ef210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:30.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.390+0000 7f8960a00640 1 --2- 172.21.15.39:0/2130355970 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f895c0cf990 0x7f895c1ef210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:30.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.390+0000 7f8960a00640 1 -- 172.21.15.39:0/2130355970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f895c073dd0 msgr2=0x7f895c1f5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:30.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.390+0000 7f8960a00640 1 --2- 172.21.15.39:0/2130355970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f895c073dd0 0x7f895c1f5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:30.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.390+0000 7f8960a00640 1 -- 172.21.15.39:0/2130355970 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f895c0ae280 con 0x7f895c15daf0 2024-09-21T07:46:30.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.390+0000 7f8960a00640 1 --2- 172.21.15.39:0/2130355970 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f895c15daf0 0x7f895c1e5ac0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f895400c760 tx=0x7f895400cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:30.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.390+0000 7f8958c00640 1 -- 172.21.15.39:0/2130355970 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8954016020 con 0x7f895c15daf0 2024-09-21T07:46:30.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.390+0000 7f8958c00640 1 -- 172.21.15.39:0/2130355970 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8954003990 con 0x7f895c15daf0 2024-09-21T07:46:30.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.390+0000 7f8958c00640 1 -- 172.21.15.39:0/2130355970 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8954006e60 con 0x7f895c15daf0 2024-09-21T07:46:30.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.390+0000 7f8962a00640 1 -- 172.21.15.39:0/2130355970 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f895c0ae560 con 0x7f895c15daf0 2024-09-21T07:46:30.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.390+0000 7f8962a00640 1 -- 172.21.15.39:0/2130355970 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f895c0aeb20 con 0x7f895c15daf0 2024-09-21T07:46:30.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.391+0000 7f8962a00640 1 -- 172.21.15.39:0/2130355970 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f895c15d5b0 con 0x7f895c15daf0 2024-09-21T07:46:30.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.393+0000 7f8958c00640 1 -- 172.21.15.39:0/2130355970 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f89540050d0 con 0x7f895c15daf0 2024-09-21T07:46:30.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.393+0000 7f8958c00640 1 --2- 172.21.15.39:0/2130355970 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8938078bc0 0x7f893807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:30.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.394+0000 7f8958c00640 1 -- 172.21.15.39:0/2130355970 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 6888+0+0 (secure 0 0 0) 0x7f895409ac50 con 0x7f895c15daf0 2024-09-21T07:46:30.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.394+0000 7f895be00640 1 --2- 172.21.15.39:0/2130355970 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8938078bc0 0x7f893807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:30.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.395+0000 7f895be00640 1 --2- 172.21.15.39:0/2130355970 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8938078bc0 0x7f893807b0d0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f894c033000 tx=0x7f894c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:30.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.398+0000 7f8958c00640 1 -- 172.21.15.39:0/2130355970 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f895c15d5b0 con 0x7f895c15daf0 2024-09-21T07:46:30.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.594+0000 7f8962a00640 1 -- 172.21.15.39:0/2130355970 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f895c0cd800 con 0x7f895c15daf0 2024-09-21T07:46:30.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.596+0000 7f8958c00640 1 -- 172.21.15.39:0/2130355970 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v175) ==== 74+0+22284 (secure 0 0 0) 0x7f89540a0020 con 0x7f895c15daf0 2024-09-21T07:46:30.598 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:46:30.598 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":175,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:46:27.588503+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:46:24.067206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-09-21T07:46:25.054230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"175","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]},{"pool":20,"snaps":[{"begin":80,"length":23},{"begin":181,"length":23}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:46:30.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.598+0000 7f8962a00640 1 -- 172.21.15.39:0/2130355970 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8938078bc0 msgr2=0x7f893807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:30.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.598+0000 7f8962a00640 1 --2- 172.21.15.39:0/2130355970 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8938078bc0 0x7f893807b0d0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f894c033000 tx=0x7f894c004730 comp rx=0 tx=0).stop 2024-09-21T07:46:30.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.598+0000 7f8962a00640 1 -- 172.21.15.39:0/2130355970 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f895c15daf0 msgr2=0x7f895c1e5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:30.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.598+0000 7f8962a00640 1 --2- 172.21.15.39:0/2130355970 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f895c15daf0 0x7f895c1e5ac0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f895400c760 tx=0x7f895400cb60 comp rx=0 tx=0).stop 2024-09-21T07:46:30.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.598+0000 7f8962a00640 1 -- 172.21.15.39:0/2130355970 shutdown_connections 2024-09-21T07:46:30.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.598+0000 7f8962a00640 1 --2- 172.21.15.39:0/2130355970 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8938078bc0 0x7f893807b0d0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:30.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.598+0000 7f8962a00640 1 --2- 172.21.15.39:0/2130355970 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f895c15daf0 0x7f895c1e5ac0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:30.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.598+0000 7f8962a00640 1 --2- 172.21.15.39:0/2130355970 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f895c0cf990 0x7f895c1ef210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:30.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.598+0000 7f8962a00640 1 --2- 172.21.15.39:0/2130355970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f895c073dd0 0x7f895c1f5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:30.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.598+0000 7f8962a00640 1 -- 172.21.15.39:0/2130355970 >> 172.21.15.39:0/2130355970 conn(0x7f895c06cfc0 msgr2=0x7f895c0b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:30.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.599+0000 7f8962a00640 1 -- 172.21.15.39:0/2130355970 shutdown_connections 2024-09-21T07:46:30.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.599+0000 7f8962a00640 1 -- 172.21.15.39:0/2130355970 wait complete. 2024-09-21T07:46:30.623 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T07:46:30.623 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:46:30.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.849+0000 7fc34e800640 1 -- 172.21.15.39:0/219680116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc32c003a50 msgr2=0x7fc32c003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:30.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.849+0000 7fc34e800640 1 --2- 172.21.15.39:0/219680116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc32c003a50 0x7fc32c003ec0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fc33800c470 tx=0x7fc32c005830 comp rx=0 tx=0).stop 2024-09-21T07:46:30.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.849+0000 7fc34e800640 1 -- 172.21.15.39:0/219680116 shutdown_connections 2024-09-21T07:46:30.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.849+0000 7fc34e800640 1 --2- 172.21.15.39:0/219680116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc32c003a50 0x7fc32c003ec0 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:30.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.849+0000 7fc34e800640 1 -- 172.21.15.39:0/219680116 >> 172.21.15.39:0/219680116 conn(0x7fc3480b2460 msgr2=0x7fc34806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:30.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.849+0000 7fc34e800640 1 -- 172.21.15.39:0/219680116 shutdown_connections 2024-09-21T07:46:30.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.849+0000 7fc34e800640 1 -- 172.21.15.39:0/219680116 wait complete. 2024-09-21T07:46:30.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.851+0000 7fc34e800640 1 Processor -- start 2024-09-21T07:46:30.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.852+0000 7fc34e800640 1 -- start start 2024-09-21T07:46:30.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.853+0000 7fc34e800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc32c003a50 0x7fc3481f46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:30.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.853+0000 7fc34e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3481f1cc0 0x7fc348074970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:30.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.853+0000 7fc34e800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc348074eb0 0x7fc3481552c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:30.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.853+0000 7fc34e800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc3480b3670 con 0x7fc3481f1cc0 2024-09-21T07:46:30.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.853+0000 7fc34e800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc3480cc060 con 0x7fc348074eb0 2024-09-21T07:46:30.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.853+0000 7fc34e800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc3480b37f0 con 0x7fc32c003a50 2024-09-21T07:46:30.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.853+0000 7fc33fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3481f1cc0 0x7fc348074970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:30.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.853+0000 7fc33fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3481f1cc0 0x7fc348074970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35596/0 (socket says 172.21.15.39:35596) 2024-09-21T07:46:30.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.853+0000 7fc33fe00640 1 -- 172.21.15.39:0/15361147 learned_addr learned my addr 172.21.15.39:0/15361147 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:30.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.853+0000 7fc347e00640 1 --2- 172.21.15.39:0/15361147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc348074eb0 0x7fc3481552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:30.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.853+0000 7fc347400640 1 --2- 172.21.15.39:0/15361147 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc32c003a50 0x7fc3481f46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:30.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.854+0000 7fc33fe00640 1 -- 172.21.15.39:0/15361147 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc32c003a50 msgr2=0x7fc3481f46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:30.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.854+0000 7fc33fe00640 1 --2- 172.21.15.39:0/15361147 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc32c003a50 0x7fc3481f46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:30.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.854+0000 7fc33fe00640 1 -- 172.21.15.39:0/15361147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc348074eb0 msgr2=0x7fc3481552c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:30.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.854+0000 7fc33fe00640 1 --2- 172.21.15.39:0/15361147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc348074eb0 0x7fc3481552c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:30.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.854+0000 7fc33fe00640 1 -- 172.21.15.39:0/15361147 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3481f51c0 con 0x7fc3481f1cc0 2024-09-21T07:46:30.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.854+0000 7fc33fe00640 1 --2- 172.21.15.39:0/15361147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3481f1cc0 0x7fc348074970 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fc330003f00 tx=0x7fc330001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:30.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.854+0000 7fc33f400640 1 -- 172.21.15.39:0/15361147 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc330016030 con 0x7fc3481f1cc0 2024-09-21T07:46:30.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.854+0000 7fc33f400640 1 -- 172.21.15.39:0/15361147 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc330011640 con 0x7fc3481f1cc0 2024-09-21T07:46:30.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.854+0000 7fc34e800640 1 -- 172.21.15.39:0/15361147 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3481f11e0 con 0x7fc3481f1cc0 2024-09-21T07:46:30.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.854+0000 7fc33f400640 1 -- 172.21.15.39:0/15361147 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc330011a90 con 0x7fc3481f1cc0 2024-09-21T07:46:30.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.855+0000 7fc34e800640 1 -- 172.21.15.39:0/15361147 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3481f17a0 con 0x7fc3481f1cc0 2024-09-21T07:46:30.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.855+0000 7fc34e800640 1 -- 172.21.15.39:0/15361147 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3481f4f30 con 0x7fc3481f1cc0 2024-09-21T07:46:30.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.857+0000 7fc33f400640 1 -- 172.21.15.39:0/15361147 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc330019070 con 0x7fc3481f1cc0 2024-09-21T07:46:30.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.858+0000 7fc33f400640 1 --2- 172.21.15.39:0/15361147 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc324078b90 0x7fc32407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:30.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.858+0000 7fc33f400640 1 -- 172.21.15.39:0/15361147 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 6888+0+0 (secure 0 0 0) 0x7fc33009b730 con 0x7fc3481f1cc0 2024-09-21T07:46:30.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.858+0000 7fc347400640 1 --2- 172.21.15.39:0/15361147 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc324078b90 0x7fc32407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:30.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.859+0000 7fc347400640 1 --2- 172.21.15.39:0/15361147 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc324078b90 0x7fc32407b0a0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fc338001050 tx=0x7fc338001bc0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:30.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:30.862+0000 7fc33f400640 1 -- 172.21.15.39:0/15361147 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc3300636c0 con 0x7fc3481f1cc0 2024-09-21T07:46:31.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.056+0000 7fc34e800640 1 -- 172.21.15.39:0/15361147 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc3480b3ce0 con 0x7fc324078b90 2024-09-21T07:46:31.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.065+0000 7fc33f400640 1 -- 172.21.15.39:0/15361147 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+495246 (secure 0 0 0) 0x7fc3480b3ce0 con 0x7fc324078b90 2024-09-21T07:46:31.073 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:46:31.077 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:46:31.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.071+0000 7fc34e800640 1 -- 172.21.15.39:0/15361147 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc324078b90 msgr2=0x7fc32407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:31.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.071+0000 7fc34e800640 1 --2- 172.21.15.39:0/15361147 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc324078b90 0x7fc32407b0a0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fc338001050 tx=0x7fc338001bc0 comp rx=0 tx=0).stop 2024-09-21T07:46:31.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.071+0000 7fc34e800640 1 -- 172.21.15.39:0/15361147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3481f1cc0 msgr2=0x7fc348074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:31.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.071+0000 7fc34e800640 1 --2- 172.21.15.39:0/15361147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3481f1cc0 0x7fc348074970 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fc330003f00 tx=0x7fc330001630 comp rx=0 tx=0).stop 2024-09-21T07:46:31.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.071+0000 7fc34e800640 1 -- 172.21.15.39:0/15361147 shutdown_connections 2024-09-21T07:46:31.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.071+0000 7fc34e800640 1 --2- 172.21.15.39:0/15361147 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc324078b90 0x7fc32407b0a0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:31.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.071+0000 7fc34e800640 1 --2- 172.21.15.39:0/15361147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc348074eb0 0x7fc3481552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:31.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.071+0000 7fc34e800640 1 --2- 172.21.15.39:0/15361147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3481f1cc0 0x7fc348074970 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:31.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.071+0000 7fc34e800640 1 --2- 172.21.15.39:0/15361147 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc32c003a50 0x7fc3481f46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:31.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.071+0000 7fc34e800640 1 -- 172.21.15.39:0/15361147 >> 172.21.15.39:0/15361147 conn(0x7fc3480b2460 msgr2=0x7fc3480b0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:31.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.071+0000 7fc34e800640 1 -- 172.21.15.39:0/15361147 shutdown_connections 2024-09-21T07:46:31.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.071+0000 7fc34e800640 1 -- 172.21.15.39:0/15361147 wait complete. 2024-09-21T07:46:31.094 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":574,"stamp":"2024-09-21T07:46:29.803154+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":252,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":321832,"kb_used_data":18204,"kb_used_omap":12,"kb_used_meta":303411,"kb_avail":754652888,"statfs":{"total":773094113280,"available":772764557312,"internally_reserved":0,"allocated":18640896,"data_stored":10370244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":310693424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.231649"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.600815+0000","last_change":"2024-09-21T07:46:26.603718+0000","last_active":"2024-09-21T07:46:27.600815+0000","last_peered":"2024-09-21T07:46:27.600815+0000","last_clean":"2024-09-21T07:46:27.600815+0000","last_became_active":"2024-09-21T07:46:26.602807+0000","last_became_peered":"2024-09-21T07:46:26.602807+0000","last_unstale":"2024-09-21T07:46:27.600815+0000","last_undegraded":"2024-09-21T07:46:27.600815+0000","last_fullsized":"2024-09-21T07:46:27.600815+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:44:29.488485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.623985+0000","last_change":"2024-09-21T07:46:26.620149+0000","last_active":"2024-09-21T07:46:26.623985+0000","last_peered":"2024-09-21T07:46:26.623985+0000","last_clean":"2024-09-21T07:46:26.623985+0000","last_became_active":"2024-09-21T07:46:26.615071+0000","last_became_peered":"2024-09-21T07:46:26.615071+0000","last_unstale":"2024-09-21T07:46:26.623985+0000","last_undegraded":"2024-09-21T07:46:26.623985+0000","last_fullsized":"2024-09-21T07:46:26.623985+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:28.468270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.623621+0000","last_change":"2024-09-21T07:46:26.615131+0000","last_active":"2024-09-21T07:46:26.623621+0000","last_peered":"2024-09-21T07:46:26.623621+0000","last_clean":"2024-09-21T07:46:26.623621+0000","last_became_active":"2024-09-21T07:46:26.614788+0000","last_became_peered":"2024-09-21T07:46:26.614788+0000","last_unstale":"2024-09-21T07:46:26.623621+0000","last_undegraded":"2024-09-21T07:46:26.623621+0000","last_fullsized":"2024-09-21T07:46:26.623621+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:23.450255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.610407+0000","last_change":"2024-09-21T07:46:26.601917+0000","last_active":"2024-09-21T07:46:26.610407+0000","last_peered":"2024-09-21T07:46:26.610407+0000","last_clean":"2024-09-21T07:46:26.610407+0000","last_became_active":"2024-09-21T07:46:26.601581+0000","last_became_peered":"2024-09-21T07:46:26.601581+0000","last_unstale":"2024-09-21T07:46:26.610407+0000","last_undegraded":"2024-09-21T07:46:26.610407+0000","last_fullsized":"2024-09-21T07:46:26.610407+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:15.141341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.610303+0000","last_change":"2024-09-21T07:46:26.596795+0000","last_active":"2024-09-21T07:46:26.610303+0000","last_peered":"2024-09-21T07:46:26.610303+0000","last_clean":"2024-09-21T07:46:26.610303+0000","last_became_active":"2024-09-21T07:46:26.596512+0000","last_became_peered":"2024-09-21T07:46:26.596512+0000","last_unstale":"2024-09-21T07:46:26.610303+0000","last_undegraded":"2024-09-21T07:46:26.610303+0000","last_fullsized":"2024-09-21T07:46:26.610303+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:23:34.648456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.608364+0000","last_change":"2024-09-21T07:46:26.599492+0000","last_active":"2024-09-21T07:46:26.608364+0000","last_peered":"2024-09-21T07:46:26.608364+0000","last_clean":"2024-09-21T07:46:26.608364+0000","last_became_active":"2024-09-21T07:46:26.597014+0000","last_became_peered":"2024-09-21T07:46:26.597014+0000","last_unstale":"2024-09-21T07:46:26.608364+0000","last_undegraded":"2024-09-21T07:46:26.608364+0000","last_fullsized":"2024-09-21T07:46:26.608364+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:41:30.878490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599769+0000","last_change":"2024-09-21T07:46:26.604204+0000","last_active":"2024-09-21T07:46:27.599769+0000","last_peered":"2024-09-21T07:46:27.599769+0000","last_clean":"2024-09-21T07:46:27.599769+0000","last_became_active":"2024-09-21T07:46:26.603837+0000","last_became_peered":"2024-09-21T07:46:26.603837+0000","last_unstale":"2024-09-21T07:46:27.599769+0000","last_undegraded":"2024-09-21T07:46:27.599769+0000","last_fullsized":"2024-09-21T07:46:27.599769+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:28.695927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.626154+0000","last_change":"2024-09-21T07:46:26.617954+0000","last_active":"2024-09-21T07:46:26.626154+0000","last_peered":"2024-09-21T07:46:26.626154+0000","last_clean":"2024-09-21T07:46:26.626154+0000","last_became_active":"2024-09-21T07:46:26.612003+0000","last_became_peered":"2024-09-21T07:46:26.612003+0000","last_unstale":"2024-09-21T07:46:26.626154+0000","last_undegraded":"2024-09-21T07:46:26.626154+0000","last_fullsized":"2024-09-21T07:46:26.626154+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:15.980531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.613636+0000","last_change":"2024-09-21T07:46:26.610922+0000","last_active":"2024-09-21T07:46:26.613636+0000","last_peered":"2024-09-21T07:46:26.613636+0000","last_clean":"2024-09-21T07:46:26.613636+0000","last_became_active":"2024-09-21T07:46:26.610591+0000","last_became_peered":"2024-09-21T07:46:26.610591+0000","last_unstale":"2024-09-21T07:46:26.613636+0000","last_undegraded":"2024-09-21T07:46:26.613636+0000","last_fullsized":"2024-09-21T07:46:26.613636+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:33.939458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.607039+0000","last_change":"2024-09-21T07:46:26.592170+0000","last_active":"2024-09-21T07:46:26.607039+0000","last_peered":"2024-09-21T07:46:26.607039+0000","last_clean":"2024-09-21T07:46:26.607039+0000","last_became_active":"2024-09-21T07:46:26.591847+0000","last_became_peered":"2024-09-21T07:46:26.591847+0000","last_unstale":"2024-09-21T07:46:26.607039+0000","last_undegraded":"2024-09-21T07:46:26.607039+0000","last_fullsized":"2024-09-21T07:46:26.607039+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:41.885676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.601124+0000","last_change":"2024-09-21T07:46:26.604500+0000","last_active":"2024-09-21T07:46:27.601124+0000","last_peered":"2024-09-21T07:46:27.601124+0000","last_clean":"2024-09-21T07:46:27.601124+0000","last_became_active":"2024-09-21T07:46:26.603764+0000","last_became_peered":"2024-09-21T07:46:26.603764+0000","last_unstale":"2024-09-21T07:46:27.601124+0000","last_undegraded":"2024-09-21T07:46:27.601124+0000","last_fullsized":"2024-09-21T07:46:27.601124+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:08.389634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.618690+0000","last_change":"2024-09-21T07:46:26.614039+0000","last_active":"2024-09-21T07:46:26.618690+0000","last_peered":"2024-09-21T07:46:26.618690+0000","last_clean":"2024-09-21T07:46:26.618690+0000","last_became_active":"2024-09-21T07:46:26.613725+0000","last_became_peered":"2024-09-21T07:46:26.613725+0000","last_unstale":"2024-09-21T07:46:26.618690+0000","last_undegraded":"2024-09-21T07:46:26.618690+0000","last_fullsized":"2024-09-21T07:46:26.618690+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:23:35.677383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.610275+0000","last_change":"2024-09-21T07:46:26.596094+0000","last_active":"2024-09-21T07:46:26.610275+0000","last_peered":"2024-09-21T07:46:26.610275+0000","last_clean":"2024-09-21T07:46:26.610275+0000","last_became_active":"2024-09-21T07:46:26.594210+0000","last_became_peered":"2024-09-21T07:46:26.594210+0000","last_unstale":"2024-09-21T07:46:26.610275+0000","last_undegraded":"2024-09-21T07:46:26.610275+0000","last_fullsized":"2024-09-21T07:46:26.610275+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:57.950433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.617456+0000","last_change":"2024-09-21T07:46:26.612264+0000","last_active":"2024-09-21T07:46:26.617456+0000","last_peered":"2024-09-21T07:46:26.617456+0000","last_clean":"2024-09-21T07:46:26.617456+0000","last_became_active":"2024-09-21T07:46:26.611939+0000","last_became_peered":"2024-09-21T07:46:26.611939+0000","last_unstale":"2024-09-21T07:46:26.617456+0000","last_undegraded":"2024-09-21T07:46:26.617456+0000","last_fullsized":"2024-09-21T07:46:26.617456+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:57:46.032193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.624257+0000","last_change":"2024-09-21T07:46:26.615565+0000","last_active":"2024-09-21T07:46:26.624257+0000","last_peered":"2024-09-21T07:46:26.624257+0000","last_clean":"2024-09-21T07:46:26.624257+0000","last_became_active":"2024-09-21T07:46:26.615236+0000","last_became_peered":"2024-09-21T07:46:26.615236+0000","last_unstale":"2024-09-21T07:46:26.624257+0000","last_undegraded":"2024-09-21T07:46:26.624257+0000","last_fullsized":"2024-09-21T07:46:26.624257+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:42:11.471492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.613544+0000","last_change":"2024-09-21T07:46:26.610453+0000","last_active":"2024-09-21T07:46:26.613544+0000","last_peered":"2024-09-21T07:46:26.613544+0000","last_clean":"2024-09-21T07:46:26.613544+0000","last_became_active":"2024-09-21T07:46:26.605865+0000","last_became_peered":"2024-09-21T07:46:26.605865+0000","last_unstale":"2024-09-21T07:46:26.613544+0000","last_undegraded":"2024-09-21T07:46:26.613544+0000","last_fullsized":"2024-09-21T07:46:26.613544+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:29.896851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.618999+0000","last_change":"2024-09-21T07:46:26.610826+0000","last_active":"2024-09-21T07:46:26.618999+0000","last_peered":"2024-09-21T07:46:26.618999+0000","last_clean":"2024-09-21T07:46:26.618999+0000","last_became_active":"2024-09-21T07:46:26.610439+0000","last_became_peered":"2024-09-21T07:46:26.610439+0000","last_unstale":"2024-09-21T07:46:26.618999+0000","last_undegraded":"2024-09-21T07:46:26.618999+0000","last_fullsized":"2024-09-21T07:46:26.618999+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:43:13.569707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.601229+0000","last_change":"2024-09-21T07:46:26.595755+0000","last_active":"2024-09-21T07:46:27.601229+0000","last_peered":"2024-09-21T07:46:27.601229+0000","last_clean":"2024-09-21T07:46:27.601229+0000","last_became_active":"2024-09-21T07:46:26.595051+0000","last_became_peered":"2024-09-21T07:46:26.595051+0000","last_unstale":"2024-09-21T07:46:27.601229+0000","last_undegraded":"2024-09-21T07:46:27.601229+0000","last_fullsized":"2024-09-21T07:46:27.601229+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:54.430995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.604603+0000","last_change":"2024-09-21T07:46:26.625028+0000","last_active":"2024-09-21T07:46:27.604603+0000","last_peered":"2024-09-21T07:46:27.604603+0000","last_clean":"2024-09-21T07:46:27.604603+0000","last_became_active":"2024-09-21T07:46:26.624829+0000","last_became_peered":"2024-09-21T07:46:26.624829+0000","last_unstale":"2024-09-21T07:46:27.604603+0000","last_undegraded":"2024-09-21T07:46:27.604603+0000","last_fullsized":"2024-09-21T07:46:27.604603+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:44.374583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599614+0000","last_change":"2024-09-21T07:46:26.593853+0000","last_active":"2024-09-21T07:46:27.599614+0000","last_peered":"2024-09-21T07:46:27.599614+0000","last_clean":"2024-09-21T07:46:27.599614+0000","last_became_active":"2024-09-21T07:46:26.593566+0000","last_became_peered":"2024-09-21T07:46:26.593566+0000","last_unstale":"2024-09-21T07:46:27.599614+0000","last_undegraded":"2024-09-21T07:46:27.599614+0000","last_fullsized":"2024-09-21T07:46:27.599614+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:20.557353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.613098+0000","last_change":"2024-09-21T07:46:26.609318+0000","last_active":"2024-09-21T07:46:26.613098+0000","last_peered":"2024-09-21T07:46:26.613098+0000","last_clean":"2024-09-21T07:46:26.613098+0000","last_became_active":"2024-09-21T07:46:26.603129+0000","last_became_peered":"2024-09-21T07:46:26.603129+0000","last_unstale":"2024-09-21T07:46:26.613098+0000","last_undegraded":"2024-09-21T07:46:26.613098+0000","last_fullsized":"2024-09-21T07:46:26.613098+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:11:02.101471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599926+0000","last_change":"2024-09-21T07:46:26.603485+0000","last_active":"2024-09-21T07:46:27.599926+0000","last_peered":"2024-09-21T07:46:27.599926+0000","last_clean":"2024-09-21T07:46:27.599926+0000","last_became_active":"2024-09-21T07:46:26.602566+0000","last_became_peered":"2024-09-21T07:46:26.602566+0000","last_unstale":"2024-09-21T07:46:27.599926+0000","last_undegraded":"2024-09-21T07:46:27.599926+0000","last_fullsized":"2024-09-21T07:46:27.599926+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:51:32.172552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.618630+0000","last_change":"2024-09-21T07:46:26.608329+0000","last_active":"2024-09-21T07:46:26.618630+0000","last_peered":"2024-09-21T07:46:26.618630+0000","last_clean":"2024-09-21T07:46:26.618630+0000","last_became_active":"2024-09-21T07:46:26.606179+0000","last_became_peered":"2024-09-21T07:46:26.606179+0000","last_unstale":"2024-09-21T07:46:26.618630+0000","last_undegraded":"2024-09-21T07:46:26.618630+0000","last_fullsized":"2024-09-21T07:46:26.618630+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:13.966123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599917+0000","last_change":"2024-09-21T07:46:26.612396+0000","last_active":"2024-09-21T07:46:27.599917+0000","last_peered":"2024-09-21T07:46:27.599917+0000","last_clean":"2024-09-21T07:46:27.599917+0000","last_became_active":"2024-09-21T07:46:26.611432+0000","last_became_peered":"2024-09-21T07:46:26.611432+0000","last_unstale":"2024-09-21T07:46:27.599917+0000","last_undegraded":"2024-09-21T07:46:27.599917+0000","last_fullsized":"2024-09-21T07:46:27.599917+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:47.852203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.600950+0000","last_change":"2024-09-21T07:46:26.594621+0000","last_active":"2024-09-21T07:46:27.600950+0000","last_peered":"2024-09-21T07:46:27.600950+0000","last_clean":"2024-09-21T07:46:27.600950+0000","last_became_active":"2024-09-21T07:46:26.594214+0000","last_became_peered":"2024-09-21T07:46:26.594214+0000","last_unstale":"2024-09-21T07:46:27.600950+0000","last_undegraded":"2024-09-21T07:46:27.600950+0000","last_fullsized":"2024-09-21T07:46:27.600950+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:34.463326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.600964+0000","last_change":"2024-09-21T07:46:26.601442+0000","last_active":"2024-09-21T07:46:27.600964+0000","last_peered":"2024-09-21T07:46:27.600964+0000","last_clean":"2024-09-21T07:46:27.600964+0000","last_became_active":"2024-09-21T07:46:26.601045+0000","last_became_peered":"2024-09-21T07:46:26.601045+0000","last_unstale":"2024-09-21T07:46:27.600964+0000","last_undegraded":"2024-09-21T07:46:27.600964+0000","last_fullsized":"2024-09-21T07:46:27.600964+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:41:14.294772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.612569+0000","last_change":"2024-09-21T07:46:26.608915+0000","last_active":"2024-09-21T07:46:26.612569+0000","last_peered":"2024-09-21T07:46:26.612569+0000","last_clean":"2024-09-21T07:46:26.612569+0000","last_became_active":"2024-09-21T07:46:26.608630+0000","last_became_peered":"2024-09-21T07:46:26.608630+0000","last_unstale":"2024-09-21T07:46:26.612569+0000","last_undegraded":"2024-09-21T07:46:26.612569+0000","last_fullsized":"2024-09-21T07:46:26.612569+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:15.406096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.600406+0000","last_change":"2024-09-21T07:46:26.597210+0000","last_active":"2024-09-21T07:46:27.600406+0000","last_peered":"2024-09-21T07:46:27.600406+0000","last_clean":"2024-09-21T07:46:27.600406+0000","last_became_active":"2024-09-21T07:46:26.596513+0000","last_became_peered":"2024-09-21T07:46:26.596513+0000","last_unstale":"2024-09-21T07:46:27.600406+0000","last_undegraded":"2024-09-21T07:46:27.600406+0000","last_fullsized":"2024-09-21T07:46:27.600406+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:20.818984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.626081+0000","last_change":"2024-09-21T07:46:26.621506+0000","last_active":"2024-09-21T07:46:26.626081+0000","last_peered":"2024-09-21T07:46:26.626081+0000","last_clean":"2024-09-21T07:46:26.626081+0000","last_became_active":"2024-09-21T07:46:26.618147+0000","last_became_peered":"2024-09-21T07:46:26.618147+0000","last_unstale":"2024-09-21T07:46:26.626081+0000","last_undegraded":"2024-09-21T07:46:26.626081+0000","last_fullsized":"2024-09-21T07:46:26.626081+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:29:28.176807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.600588+0000","last_change":"2024-09-21T07:46:26.608633+0000","last_active":"2024-09-21T07:46:27.600588+0000","last_peered":"2024-09-21T07:46:27.600588+0000","last_clean":"2024-09-21T07:46:27.600588+0000","last_became_active":"2024-09-21T07:46:26.606778+0000","last_became_peered":"2024-09-21T07:46:26.606778+0000","last_unstale":"2024-09-21T07:46:27.600588+0000","last_undegraded":"2024-09-21T07:46:27.600588+0000","last_fullsized":"2024-09-21T07:46:27.600588+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:57.377169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.611874+0000","last_change":"2024-09-21T07:46:26.607976+0000","last_active":"2024-09-21T07:46:26.611874+0000","last_peered":"2024-09-21T07:46:26.611874+0000","last_clean":"2024-09-21T07:46:26.611874+0000","last_became_active":"2024-09-21T07:46:26.607452+0000","last_became_peered":"2024-09-21T07:46:26.607452+0000","last_unstale":"2024-09-21T07:46:26.611874+0000","last_undegraded":"2024-09-21T07:46:26.611874+0000","last_fullsized":"2024-09-21T07:46:26.611874+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:51.495597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.622036+0000","last_change":"2024-09-21T07:46:26.611007+0000","last_active":"2024-09-21T07:46:26.622036+0000","last_peered":"2024-09-21T07:46:26.622036+0000","last_clean":"2024-09-21T07:46:26.622036+0000","last_became_active":"2024-09-21T07:46:26.610179+0000","last_became_peered":"2024-09-21T07:46:26.610179+0000","last_unstale":"2024-09-21T07:46:26.622036+0000","last_undegraded":"2024-09-21T07:46:26.622036+0000","last_fullsized":"2024-09-21T07:46:26.622036+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:54:05.047873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.628897+0000","last_change":"2024-09-21T07:46:26.622360+0000","last_active":"2024-09-21T07:46:26.628897+0000","last_peered":"2024-09-21T07:46:26.628897+0000","last_clean":"2024-09-21T07:46:26.628897+0000","last_became_active":"2024-09-21T07:46:26.619462+0000","last_became_peered":"2024-09-21T07:46:26.619462+0000","last_unstale":"2024-09-21T07:46:26.628897+0000","last_undegraded":"2024-09-21T07:46:26.628897+0000","last_fullsized":"2024-09-21T07:46:26.628897+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:23:15.779122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"174'6","reported_seq":21,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599594+0000","last_change":"2024-09-21T07:46:25.596989+0000","last_active":"2024-09-21T07:46:27.599594+0000","last_peered":"2024-09-21T07:46:27.599594+0000","last_clean":"2024-09-21T07:46:27.599594+0000","last_became_active":"2024-09-21T07:46:25.596030+0000","last_became_peered":"2024-09-21T07:46:25.596030+0000","last_unstale":"2024-09-21T07:46:27.599594+0000","last_undegraded":"2024-09-21T07:46:27.599594+0000","last_fullsized":"2024-09-21T07:46:27.599594+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:08:26.795197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":224,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.604830+0000","last_change":"2024-09-21T07:35:27.879605+0000","last_active":"2024-09-21T07:46:27.604830+0000","last_peered":"2024-09-21T07:46:27.604830+0000","last_clean":"2024-09-21T07:46:27.604830+0000","last_became_active":"2024-09-21T07:35:27.879274+0000","last_became_peered":"2024-09-21T07:35:27.879274+0000","last_unstale":"2024-09-21T07:46:27.604830+0000","last_undegraded":"2024-09-21T07:46:27.604830+0000","last_fullsized":"2024-09-21T07:46:27.604830+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:24.241636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":230,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.588630+0000","last_change":"2024-09-21T07:35:29.491336+0000","last_active":"2024-09-21T07:46:26.588630+0000","last_peered":"2024-09-21T07:46:26.588630+0000","last_clean":"2024-09-21T07:46:26.588630+0000","last_became_active":"2024-09-21T07:35:26.866484+0000","last_became_peered":"2024-09-21T07:35:26.866484+0000","last_unstale":"2024-09-21T07:46:26.588630+0000","last_undegraded":"2024-09-21T07:46:26.588630+0000","last_fullsized":"2024-09-21T07:46:26.588630+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:10.449665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076010100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":410,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.603920+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:46:27.603920+0000","last_peered":"2024-09-21T07:46:27.603920+0000","last_clean":"2024-09-21T07:46:27.603920+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:46:27.603920+0000","last_undegraded":"2024-09-21T07:46:27.603920+0000","last_fullsized":"2024-09-21T07:46:27.603920+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":291,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.601133+0000","last_change":"2024-09-21T07:34:24.539248+0000","last_active":"2024-09-21T07:46:27.601133+0000","last_peered":"2024-09-21T07:46:27.601133+0000","last_clean":"2024-09-21T07:46:27.601133+0000","last_became_active":"2024-09-21T07:34:23.273465+0000","last_became_peered":"2024-09-21T07:34:23.273465+0000","last_unstale":"2024-09-21T07:46:27.601133+0000","last_undegraded":"2024-09-21T07:46:27.601133+0000","last_fullsized":"2024-09-21T07:46:27.601133+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:55.517452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060627700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.611787+0000","last_change":"2024-09-21T07:46:26.605575+0000","last_active":"2024-09-21T07:46:26.611787+0000","last_peered":"2024-09-21T07:46:26.611787+0000","last_clean":"2024-09-21T07:46:26.611787+0000","last_became_active":"2024-09-21T07:46:26.597517+0000","last_became_peered":"2024-09-21T07:46:26.597517+0000","last_unstale":"2024-09-21T07:46:26.611787+0000","last_undegraded":"2024-09-21T07:46:26.611787+0000","last_fullsized":"2024-09-21T07:46:26.611787+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:50.946973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":13,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.605339+0000","last_change":"2024-09-21T07:46:25.600542+0000","last_active":"2024-09-21T07:46:26.605339+0000","last_peered":"2024-09-21T07:46:26.605339+0000","last_clean":"2024-09-21T07:46:26.605339+0000","last_became_active":"2024-09-21T07:46:25.600103+0000","last_became_peered":"2024-09-21T07:46:25.600103+0000","last_unstale":"2024-09-21T07:46:26.605339+0000","last_undegraded":"2024-09-21T07:46:26.605339+0000","last_fullsized":"2024-09-21T07:46:26.605339+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:24.714172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":223,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.587593+0000","last_change":"2024-09-21T07:35:27.864492+0000","last_active":"2024-09-21T07:46:26.587593+0000","last_peered":"2024-09-21T07:46:26.587593+0000","last_clean":"2024-09-21T07:46:26.587593+0000","last_became_active":"2024-09-21T07:35:27.864176+0000","last_became_peered":"2024-09-21T07:35:27.864176+0000","last_unstale":"2024-09-21T07:46:26.587593+0000","last_undegraded":"2024-09-21T07:46:26.587593+0000","last_fullsized":"2024-09-21T07:46:26.587593+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:05.153467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":231,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.605060+0000","last_change":"2024-09-21T07:35:29.493678+0000","last_active":"2024-09-21T07:46:27.605060+0000","last_peered":"2024-09-21T07:46:27.605060+0000","last_clean":"2024-09-21T07:46:27.605060+0000","last_became_active":"2024-09-21T07:35:26.870749+0000","last_became_peered":"2024-09-21T07:35:26.870749+0000","last_unstale":"2024-09-21T07:46:27.605060+0000","last_undegraded":"2024-09-21T07:46:27.605060+0000","last_fullsized":"2024-09-21T07:46:27.605060+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:55.367877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010159570000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":276,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.607914+0000","last_change":"2024-09-21T07:34:24.597274+0000","last_active":"2024-09-21T07:46:26.607914+0000","last_peered":"2024-09-21T07:46:26.607914+0000","last_clean":"2024-09-21T07:46:26.607914+0000","last_became_active":"2024-09-21T07:34:23.279560+0000","last_became_peered":"2024-09-21T07:34:23.279560+0000","last_unstale":"2024-09-21T07:46:26.607914+0000","last_undegraded":"2024-09-21T07:46:26.607914+0000","last_fullsized":"2024-09-21T07:46:26.607914+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:15.845265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059699799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.615217+0000","last_change":"2024-09-21T07:46:26.600249+0000","last_active":"2024-09-21T07:46:26.615217+0000","last_peered":"2024-09-21T07:46:26.615217+0000","last_clean":"2024-09-21T07:46:26.615217+0000","last_became_active":"2024-09-21T07:46:26.599921+0000","last_became_peered":"2024-09-21T07:46:26.599921+0000","last_unstale":"2024-09-21T07:46:26.615217+0000","last_undegraded":"2024-09-21T07:46:26.615217+0000","last_fullsized":"2024-09-21T07:46:26.615217+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:20.298916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"174'8","reported_seq":23,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599410+0000","last_change":"2024-09-21T07:46:25.597983+0000","last_active":"2024-09-21T07:46:27.599410+0000","last_peered":"2024-09-21T07:46:27.599410+0000","last_clean":"2024-09-21T07:46:27.599410+0000","last_became_active":"2024-09-21T07:46:25.597074+0000","last_became_peered":"2024-09-21T07:46:25.597074+0000","last_unstale":"2024-09-21T07:46:27.599410+0000","last_undegraded":"2024-09-21T07:46:27.599410+0000","last_fullsized":"2024-09-21T07:46:27.599410+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:15.858457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":225,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.600271+0000","last_change":"2024-09-21T07:35:27.874387+0000","last_active":"2024-09-21T07:46:27.600271+0000","last_peered":"2024-09-21T07:46:27.600271+0000","last_clean":"2024-09-21T07:46:27.600271+0000","last_became_active":"2024-09-21T07:35:27.873277+0000","last_became_peered":"2024-09-21T07:35:27.873277+0000","last_unstale":"2024-09-21T07:46:27.600271+0000","last_undegraded":"2024-09-21T07:46:27.600271+0000","last_fullsized":"2024-09-21T07:46:27.600271+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:04.581463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":226,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.615398+0000","last_change":"2024-09-21T07:35:29.387094+0000","last_active":"2024-09-21T07:46:26.615398+0000","last_peered":"2024-09-21T07:46:26.615398+0000","last_clean":"2024-09-21T07:46:26.615398+0000","last_became_active":"2024-09-21T07:35:26.874757+0000","last_became_peered":"2024-09-21T07:35:26.874757+0000","last_unstale":"2024-09-21T07:46:26.615398+0000","last_undegraded":"2024-09-21T07:46:26.615398+0000","last_fullsized":"2024-09-21T07:46:26.615398+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:33.542750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061065700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":285,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.604258+0000","last_change":"2024-09-21T07:34:24.259906+0000","last_active":"2024-09-21T07:46:27.604258+0000","last_peered":"2024-09-21T07:46:27.604258+0000","last_clean":"2024-09-21T07:46:27.604258+0000","last_became_active":"2024-09-21T07:34:23.635282+0000","last_became_peered":"2024-09-21T07:34:23.635282+0000","last_unstale":"2024-09-21T07:46:27.604258+0000","last_undegraded":"2024-09-21T07:46:27.604258+0000","last_fullsized":"2024-09-21T07:46:27.604258+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:00.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001527664,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.611241+0000","last_change":"2024-09-21T07:46:26.603275+0000","last_active":"2024-09-21T07:46:26.611241+0000","last_peered":"2024-09-21T07:46:26.611241+0000","last_clean":"2024-09-21T07:46:26.611241+0000","last_became_active":"2024-09-21T07:46:26.602801+0000","last_became_peered":"2024-09-21T07:46:26.602801+0000","last_unstale":"2024-09-21T07:46:26.611241+0000","last_undegraded":"2024-09-21T07:46:26.611241+0000","last_fullsized":"2024-09-21T07:46:26.611241+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:37:59.308165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"174'2","reported_seq":15,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.626821+0000","last_change":"2024-09-21T07:46:25.588970+0000","last_active":"2024-09-21T07:46:26.626821+0000","last_peered":"2024-09-21T07:46:26.626821+0000","last_clean":"2024-09-21T07:46:26.626821+0000","last_became_active":"2024-09-21T07:46:25.588571+0000","last_became_peered":"2024-09-21T07:46:25.588571+0000","last_unstale":"2024-09-21T07:46:26.626821+0000","last_undegraded":"2024-09-21T07:46:26.626821+0000","last_fullsized":"2024-09-21T07:46:26.626821+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:26:02.967925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":225,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.600772+0000","last_change":"2024-09-21T07:35:27.873645+0000","last_active":"2024-09-21T07:46:27.600772+0000","last_peered":"2024-09-21T07:46:27.600772+0000","last_clean":"2024-09-21T07:46:27.600772+0000","last_became_active":"2024-09-21T07:35:27.873245+0000","last_became_peered":"2024-09-21T07:35:27.873245+0000","last_unstale":"2024-09-21T07:46:27.600772+0000","last_undegraded":"2024-09-21T07:46:27.600772+0000","last_fullsized":"2024-09-21T07:46:27.600772+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:55.090953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":231,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.601211+0000","last_change":"2024-09-21T07:35:28.861744+0000","last_active":"2024-09-21T07:46:27.601211+0000","last_peered":"2024-09-21T07:46:27.601211+0000","last_clean":"2024-09-21T07:46:27.601211+0000","last_became_active":"2024-09-21T07:35:26.876462+0000","last_became_peered":"2024-09-21T07:35:26.876462+0000","last_unstale":"2024-09-21T07:46:27.601211+0000","last_undegraded":"2024-09-21T07:46:27.601211+0000","last_fullsized":"2024-09-21T07:46:27.601211+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:48.632568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012020920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":282,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.598475+0000","last_change":"2024-09-21T07:34:25.085454+0000","last_active":"2024-09-21T07:46:27.598475+0000","last_peered":"2024-09-21T07:46:27.598475+0000","last_clean":"2024-09-21T07:46:27.598475+0000","last_became_active":"2024-09-21T07:34:23.279205+0000","last_became_peered":"2024-09-21T07:34:23.279205+0000","last_unstale":"2024-09-21T07:46:27.598475+0000","last_undegraded":"2024-09-21T07:46:27.598475+0000","last_fullsized":"2024-09-21T07:46:27.598475+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:54.231666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095320599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.620627+0000","last_change":"2024-09-21T07:46:26.601788+0000","last_active":"2024-09-21T07:46:26.620627+0000","last_peered":"2024-09-21T07:46:26.620627+0000","last_clean":"2024-09-21T07:46:26.620627+0000","last_became_active":"2024-09-21T07:46:26.601484+0000","last_became_peered":"2024-09-21T07:46:26.601484+0000","last_unstale":"2024-09-21T07:46:26.620627+0000","last_undegraded":"2024-09-21T07:46:26.620627+0000","last_fullsized":"2024-09-21T07:46:26.620627+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:55:53.588662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":13,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.606233+0000","last_change":"2024-09-21T07:46:25.604131+0000","last_active":"2024-09-21T07:46:26.606233+0000","last_peered":"2024-09-21T07:46:26.606233+0000","last_clean":"2024-09-21T07:46:26.606233+0000","last_became_active":"2024-09-21T07:46:25.603782+0000","last_became_peered":"2024-09-21T07:46:25.603782+0000","last_unstale":"2024-09-21T07:46:26.606233+0000","last_undegraded":"2024-09-21T07:46:26.606233+0000","last_fullsized":"2024-09-21T07:46:26.606233+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:22:08.526888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":222,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.590401+0000","last_change":"2024-09-21T07:35:27.861373+0000","last_active":"2024-09-21T07:46:26.590401+0000","last_peered":"2024-09-21T07:46:26.590401+0000","last_clean":"2024-09-21T07:46:26.590401+0000","last_became_active":"2024-09-21T07:35:27.861016+0000","last_became_peered":"2024-09-21T07:35:27.861016+0000","last_unstale":"2024-09-21T07:46:26.590401+0000","last_undegraded":"2024-09-21T07:46:26.590401+0000","last_fullsized":"2024-09-21T07:46:26.590401+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:56.043763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":231,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.583172+0000","last_change":"2024-09-21T07:35:29.053677+0000","last_active":"2024-09-21T07:46:26.583172+0000","last_peered":"2024-09-21T07:46:26.583172+0000","last_clean":"2024-09-21T07:46:26.583172+0000","last_became_active":"2024-09-21T07:35:26.875496+0000","last_became_peered":"2024-09-21T07:35:26.875496+0000","last_unstale":"2024-09-21T07:46:26.583172+0000","last_undegraded":"2024-09-21T07:46:26.583172+0000","last_fullsized":"2024-09-21T07:46:26.583172+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:04.850784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065859000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":283,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.586083+0000","last_change":"2024-09-21T07:34:25.028521+0000","last_active":"2024-09-21T07:46:26.586083+0000","last_peered":"2024-09-21T07:46:26.586083+0000","last_clean":"2024-09-21T07:46:26.586083+0000","last_became_active":"2024-09-21T07:34:23.635467+0000","last_became_peered":"2024-09-21T07:34:23.635467+0000","last_unstale":"2024-09-21T07:46:26.586083+0000","last_undegraded":"2024-09-21T07:46:26.586083+0000","last_fullsized":"2024-09-21T07:46:26.586083+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:33.729123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000660558,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.601219+0000","last_change":"2024-09-21T07:46:26.596074+0000","last_active":"2024-09-21T07:46:27.601219+0000","last_peered":"2024-09-21T07:46:27.601219+0000","last_clean":"2024-09-21T07:46:27.601219+0000","last_became_active":"2024-09-21T07:46:26.595466+0000","last_became_peered":"2024-09-21T07:46:26.595466+0000","last_unstale":"2024-09-21T07:46:27.601219+0000","last_undegraded":"2024-09-21T07:46:27.601219+0000","last_fullsized":"2024-09-21T07:46:27.601219+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:51:32.349009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"174'3","reported_seq":18,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.600880+0000","last_change":"2024-09-21T07:46:25.594804+0000","last_active":"2024-09-21T07:46:27.600880+0000","last_peered":"2024-09-21T07:46:27.600880+0000","last_clean":"2024-09-21T07:46:27.600880+0000","last_became_active":"2024-09-21T07:46:25.593032+0000","last_became_peered":"2024-09-21T07:46:25.593032+0000","last_unstale":"2024-09-21T07:46:27.600880+0000","last_undegraded":"2024-09-21T07:46:27.600880+0000","last_fullsized":"2024-09-21T07:46:27.600880+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:49.893169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":223,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599681+0000","last_change":"2024-09-21T07:35:27.871201+0000","last_active":"2024-09-21T07:46:27.599681+0000","last_peered":"2024-09-21T07:46:27.599681+0000","last_clean":"2024-09-21T07:46:27.599681+0000","last_became_active":"2024-09-21T07:35:27.870816+0000","last_became_peered":"2024-09-21T07:35:27.870816+0000","last_unstale":"2024-09-21T07:46:27.599681+0000","last_undegraded":"2024-09-21T07:46:27.599681+0000","last_fullsized":"2024-09-21T07:46:27.599681+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:04.787450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":229,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.590646+0000","last_change":"2024-09-21T07:35:29.491874+0000","last_active":"2024-09-21T07:46:26.590646+0000","last_peered":"2024-09-21T07:46:26.590646+0000","last_clean":"2024-09-21T07:46:26.590646+0000","last_became_active":"2024-09-21T07:35:26.866381+0000","last_became_peered":"2024-09-21T07:35:26.866381+0000","last_unstale":"2024-09-21T07:46:26.590646+0000","last_undegraded":"2024-09-21T07:46:26.590646+0000","last_fullsized":"2024-09-21T07:46:26.590646+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:36.404647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073235399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":284,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.589509+0000","last_change":"2024-09-21T07:34:24.260046+0000","last_active":"2024-09-21T07:46:26.589509+0000","last_peered":"2024-09-21T07:46:26.589509+0000","last_clean":"2024-09-21T07:46:26.589509+0000","last_became_active":"2024-09-21T07:34:23.273431+0000","last_became_peered":"2024-09-21T07:34:23.273431+0000","last_unstale":"2024-09-21T07:46:26.589509+0000","last_undegraded":"2024-09-21T07:46:26.589509+0000","last_fullsized":"2024-09-21T07:46:26.589509+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:18.876057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019803310000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.625832+0000","last_change":"2024-09-21T07:46:26.620066+0000","last_active":"2024-09-21T07:46:26.625832+0000","last_peered":"2024-09-21T07:46:26.625832+0000","last_clean":"2024-09-21T07:46:26.625832+0000","last_became_active":"2024-09-21T07:46:26.619764+0000","last_became_peered":"2024-09-21T07:46:26.619764+0000","last_unstale":"2024-09-21T07:46:26.625832+0000","last_undegraded":"2024-09-21T07:46:26.625832+0000","last_fullsized":"2024-09-21T07:46:26.625832+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:39:53.865815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":15,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.604962+0000","last_change":"2024-09-21T07:46:25.600862+0000","last_active":"2024-09-21T07:46:27.604962+0000","last_peered":"2024-09-21T07:46:27.604962+0000","last_clean":"2024-09-21T07:46:27.604962+0000","last_became_active":"2024-09-21T07:46:25.600256+0000","last_became_peered":"2024-09-21T07:46:25.600256+0000","last_unstale":"2024-09-21T07:46:27.604962+0000","last_undegraded":"2024-09-21T07:46:27.604962+0000","last_fullsized":"2024-09-21T07:46:27.604962+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:44.271091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":223,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.598597+0000","last_change":"2024-09-21T07:35:27.871736+0000","last_active":"2024-09-21T07:46:27.598597+0000","last_peered":"2024-09-21T07:46:27.598597+0000","last_clean":"2024-09-21T07:46:27.598597+0000","last_became_active":"2024-09-21T07:35:27.871232+0000","last_became_peered":"2024-09-21T07:35:27.871232+0000","last_unstale":"2024-09-21T07:46:27.598597+0000","last_undegraded":"2024-09-21T07:46:27.598597+0000","last_fullsized":"2024-09-21T07:46:27.598597+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:04.419301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":230,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.586888+0000","last_change":"2024-09-21T07:35:29.491246+0000","last_active":"2024-09-21T07:46:26.586888+0000","last_peered":"2024-09-21T07:46:26.586888+0000","last_clean":"2024-09-21T07:46:26.586888+0000","last_became_active":"2024-09-21T07:35:26.866783+0000","last_became_peered":"2024-09-21T07:35:26.866783+0000","last_unstale":"2024-09-21T07:46:26.586888+0000","last_undegraded":"2024-09-21T07:46:26.586888+0000","last_fullsized":"2024-09-21T07:46:26.586888+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:28.881914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098474099999999992,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":283,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.583293+0000","last_change":"2024-09-21T07:34:25.028640+0000","last_active":"2024-09-21T07:46:26.583293+0000","last_peered":"2024-09-21T07:46:26.583293+0000","last_clean":"2024-09-21T07:46:26.583293+0000","last_became_active":"2024-09-21T07:34:23.260950+0000","last_became_peered":"2024-09-21T07:34:23.260950+0000","last_unstale":"2024-09-21T07:46:26.583293+0000","last_undegraded":"2024-09-21T07:46:26.583293+0000","last_fullsized":"2024-09-21T07:46:26.583293+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:23:42.121398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055135099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.600980+0000","last_change":"2024-09-21T07:46:26.604123+0000","last_active":"2024-09-21T07:46:27.600980+0000","last_peered":"2024-09-21T07:46:27.600980+0000","last_clean":"2024-09-21T07:46:27.600980+0000","last_became_active":"2024-09-21T07:46:26.603295+0000","last_became_peered":"2024-09-21T07:46:26.603295+0000","last_unstale":"2024-09-21T07:46:27.600980+0000","last_undegraded":"2024-09-21T07:46:27.600980+0000","last_fullsized":"2024-09-21T07:46:27.600980+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:09:26.713837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":15,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.601204+0000","last_change":"2024-09-21T07:46:25.605408+0000","last_active":"2024-09-21T07:46:27.601204+0000","last_peered":"2024-09-21T07:46:27.601204+0000","last_clean":"2024-09-21T07:46:27.601204+0000","last_became_active":"2024-09-21T07:46:25.602154+0000","last_became_peered":"2024-09-21T07:46:25.602154+0000","last_unstale":"2024-09-21T07:46:27.601204+0000","last_undegraded":"2024-09-21T07:46:27.601204+0000","last_fullsized":"2024-09-21T07:46:27.601204+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:57.518833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":223,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.606912+0000","last_change":"2024-09-21T07:35:27.872027+0000","last_active":"2024-09-21T07:46:26.606912+0000","last_peered":"2024-09-21T07:46:26.606912+0000","last_clean":"2024-09-21T07:46:26.606912+0000","last_became_active":"2024-09-21T07:35:27.870426+0000","last_became_peered":"2024-09-21T07:35:27.870426+0000","last_unstale":"2024-09-21T07:46:26.606912+0000","last_undegraded":"2024-09-21T07:46:26.606912+0000","last_fullsized":"2024-09-21T07:46:26.606912+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":230,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.587705+0000","last_change":"2024-09-21T07:35:29.491008+0000","last_active":"2024-09-21T07:46:26.587705+0000","last_peered":"2024-09-21T07:46:26.587705+0000","last_clean":"2024-09-21T07:46:26.587705+0000","last_became_active":"2024-09-21T07:35:26.866750+0000","last_became_peered":"2024-09-21T07:35:26.866750+0000","last_unstale":"2024-09-21T07:46:26.587705+0000","last_undegraded":"2024-09-21T07:46:26.587705+0000","last_fullsized":"2024-09-21T07:46:26.587705+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:25.490406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083529700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":285,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.605074+0000","last_change":"2024-09-21T07:34:24.260665+0000","last_active":"2024-09-21T07:46:27.605074+0000","last_peered":"2024-09-21T07:46:27.605074+0000","last_clean":"2024-09-21T07:46:27.605074+0000","last_became_active":"2024-09-21T07:34:23.635909+0000","last_became_peered":"2024-09-21T07:34:23.635909+0000","last_unstale":"2024-09-21T07:46:27.605074+0000","last_undegraded":"2024-09-21T07:46:27.605074+0000","last_fullsized":"2024-09-21T07:46:27.605074+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:44.536885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.602115+0000","last_change":"2024-09-21T07:46:26.597685+0000","last_active":"2024-09-21T07:46:26.602115+0000","last_peered":"2024-09-21T07:46:26.602115+0000","last_clean":"2024-09-21T07:46:26.602115+0000","last_became_active":"2024-09-21T07:46:26.595053+0000","last_became_peered":"2024-09-21T07:46:26.595053+0000","last_unstale":"2024-09-21T07:46:26.602115+0000","last_undegraded":"2024-09-21T07:46:26.602115+0000","last_fullsized":"2024-09-21T07:46:26.602115+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:51:31.780109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":15,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599463+0000","last_change":"2024-09-21T07:46:25.599088+0000","last_active":"2024-09-21T07:46:27.599463+0000","last_peered":"2024-09-21T07:46:27.599463+0000","last_clean":"2024-09-21T07:46:27.599463+0000","last_became_active":"2024-09-21T07:46:25.598661+0000","last_became_peered":"2024-09-21T07:46:25.598661+0000","last_unstale":"2024-09-21T07:46:27.599463+0000","last_undegraded":"2024-09-21T07:46:27.599463+0000","last_fullsized":"2024-09-21T07:46:27.599463+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:45:29.686688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":220,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.606126+0000","last_change":"2024-09-21T07:35:27.874772+0000","last_active":"2024-09-21T07:46:26.606126+0000","last_peered":"2024-09-21T07:46:26.606126+0000","last_clean":"2024-09-21T07:46:26.606126+0000","last_became_active":"2024-09-21T07:35:27.874434+0000","last_became_peered":"2024-09-21T07:35:27.874434+0000","last_unstale":"2024-09-21T07:46:26.606126+0000","last_undegraded":"2024-09-21T07:46:26.606126+0000","last_fullsized":"2024-09-21T07:46:26.606126+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:31.998755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":239,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.601025+0000","last_change":"2024-09-21T07:35:28.859153+0000","last_active":"2024-09-21T07:46:27.601025+0000","last_peered":"2024-09-21T07:46:27.601025+0000","last_clean":"2024-09-21T07:46:27.601025+0000","last_became_active":"2024-09-21T07:35:26.874428+0000","last_became_peered":"2024-09-21T07:35:26.874428+0000","last_unstale":"2024-09-21T07:46:27.601025+0000","last_undegraded":"2024-09-21T07:46:27.601025+0000","last_fullsized":"2024-09-21T07:46:27.601025+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:04.487621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048557299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":291,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.600173+0000","last_change":"2024-09-21T07:34:24.538845+0000","last_active":"2024-09-21T07:46:27.600173+0000","last_peered":"2024-09-21T07:46:27.600173+0000","last_clean":"2024-09-21T07:46:27.600173+0000","last_became_active":"2024-09-21T07:34:23.636372+0000","last_became_peered":"2024-09-21T07:34:23.636372+0000","last_unstale":"2024-09-21T07:46:27.600173+0000","last_undegraded":"2024-09-21T07:46:27.600173+0000","last_fullsized":"2024-09-21T07:46:27.600173+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:29.393813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841451,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599953+0000","last_change":"2024-09-21T07:46:26.604373+0000","last_active":"2024-09-21T07:46:27.599953+0000","last_peered":"2024-09-21T07:46:27.599953+0000","last_clean":"2024-09-21T07:46:27.599953+0000","last_became_active":"2024-09-21T07:46:26.602850+0000","last_became_peered":"2024-09-21T07:46:26.602850+0000","last_unstale":"2024-09-21T07:46:27.599953+0000","last_undegraded":"2024-09-21T07:46:27.599953+0000","last_fullsized":"2024-09-21T07:46:27.599953+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:13.124773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":13,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.587729+0000","last_change":"2024-09-21T07:46:25.596109+0000","last_active":"2024-09-21T07:46:26.587729+0000","last_peered":"2024-09-21T07:46:26.587729+0000","last_clean":"2024-09-21T07:46:26.587729+0000","last_became_active":"2024-09-21T07:46:25.595814+0000","last_became_peered":"2024-09-21T07:46:25.595814+0000","last_unstale":"2024-09-21T07:46:26.587729+0000","last_undegraded":"2024-09-21T07:46:26.587729+0000","last_fullsized":"2024-09-21T07:46:26.587729+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:39:32.880265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":224,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.605537+0000","last_change":"2024-09-21T07:35:27.877801+0000","last_active":"2024-09-21T07:46:26.605537+0000","last_peered":"2024-09-21T07:46:26.605537+0000","last_clean":"2024-09-21T07:46:26.605537+0000","last_became_active":"2024-09-21T07:35:27.877464+0000","last_became_peered":"2024-09-21T07:35:27.877464+0000","last_unstale":"2024-09-21T07:46:26.605537+0000","last_undegraded":"2024-09-21T07:46:26.605537+0000","last_fullsized":"2024-09-21T07:46:26.605537+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:08.814929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":231,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.598652+0000","last_change":"2024-09-21T07:35:28.863732+0000","last_active":"2024-09-21T07:46:27.598652+0000","last_peered":"2024-09-21T07:46:27.598652+0000","last_clean":"2024-09-21T07:46:27.598652+0000","last_became_active":"2024-09-21T07:35:26.877392+0000","last_became_peered":"2024-09-21T07:35:26.877392+0000","last_unstale":"2024-09-21T07:46:27.598652+0000","last_undegraded":"2024-09-21T07:46:27.598652+0000","last_fullsized":"2024-09-21T07:46:27.598652+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071425700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":284,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.590754+0000","last_change":"2024-09-21T07:34:24.259686+0000","last_active":"2024-09-21T07:46:26.590754+0000","last_peered":"2024-09-21T07:46:26.590754+0000","last_clean":"2024-09-21T07:46:26.590754+0000","last_became_active":"2024-09-21T07:34:23.274405+0000","last_became_peered":"2024-09-21T07:34:23.274405+0000","last_unstale":"2024-09-21T07:46:26.590754+0000","last_undegraded":"2024-09-21T07:46:26.590754+0000","last_fullsized":"2024-09-21T07:46:26.590754+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:26.304693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022009920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599039+0000","last_change":"2024-09-21T07:46:26.593329+0000","last_active":"2024-09-21T07:46:27.599039+0000","last_peered":"2024-09-21T07:46:27.599039+0000","last_clean":"2024-09-21T07:46:27.599039+0000","last_became_active":"2024-09-21T07:46:26.592688+0000","last_became_peered":"2024-09-21T07:46:26.592688+0000","last_unstale":"2024-09-21T07:46:27.599039+0000","last_undegraded":"2024-09-21T07:46:27.599039+0000","last_fullsized":"2024-09-21T07:46:27.599039+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:53.049143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"174'1","reported_seq":14,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.628422+0000","last_change":"2024-09-21T07:46:25.603666+0000","last_active":"2024-09-21T07:46:26.628422+0000","last_peered":"2024-09-21T07:46:26.628422+0000","last_clean":"2024-09-21T07:46:26.628422+0000","last_became_active":"2024-09-21T07:46:25.603334+0000","last_became_peered":"2024-09-21T07:46:25.603334+0000","last_unstale":"2024-09-21T07:46:26.628422+0000","last_undegraded":"2024-09-21T07:46:26.628422+0000","last_fullsized":"2024-09-21T07:46:26.628422+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:29.725512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":220,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.607793+0000","last_change":"2024-09-21T07:35:27.867336+0000","last_active":"2024-09-21T07:46:26.607793+0000","last_peered":"2024-09-21T07:46:26.607793+0000","last_clean":"2024-09-21T07:46:26.607793+0000","last_became_active":"2024-09-21T07:35:27.867015+0000","last_became_peered":"2024-09-21T07:35:27.867015+0000","last_unstale":"2024-09-21T07:46:26.607793+0000","last_undegraded":"2024-09-21T07:46:26.607793+0000","last_fullsized":"2024-09-21T07:46:26.607793+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:59.214139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":232,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.604942+0000","last_change":"2024-09-21T07:35:29.493137+0000","last_active":"2024-09-21T07:46:27.604942+0000","last_peered":"2024-09-21T07:46:27.604942+0000","last_clean":"2024-09-21T07:46:27.604942+0000","last_became_active":"2024-09-21T07:35:26.872230+0000","last_became_peered":"2024-09-21T07:35:26.872230+0000","last_unstale":"2024-09-21T07:46:27.604942+0000","last_undegraded":"2024-09-21T07:46:27.604942+0000","last_fullsized":"2024-09-21T07:46:27.604942+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:24.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001197449,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":285,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.603882+0000","last_change":"2024-09-21T07:34:24.258754+0000","last_active":"2024-09-21T07:46:27.603882+0000","last_peered":"2024-09-21T07:46:27.603882+0000","last_clean":"2024-09-21T07:46:27.603882+0000","last_became_active":"2024-09-21T07:34:23.272657+0000","last_became_peered":"2024-09-21T07:34:23.272657+0000","last_unstale":"2024-09-21T07:46:27.603882+0000","last_undegraded":"2024-09-21T07:46:27.603882+0000","last_fullsized":"2024-09-21T07:46:27.603882+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:05.477099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067143200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.621977+0000","last_change":"2024-09-21T07:46:26.616838+0000","last_active":"2024-09-21T07:46:26.621977+0000","last_peered":"2024-09-21T07:46:26.621977+0000","last_clean":"2024-09-21T07:46:26.621977+0000","last_became_active":"2024-09-21T07:46:26.616318+0000","last_became_peered":"2024-09-21T07:46:26.616318+0000","last_unstale":"2024-09-21T07:46:26.621977+0000","last_undegraded":"2024-09-21T07:46:26.621977+0000","last_fullsized":"2024-09-21T07:46:26.621977+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:10.469068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"174'1","reported_seq":14,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.626072+0000","last_change":"2024-09-21T07:46:25.593521+0000","last_active":"2024-09-21T07:46:26.626072+0000","last_peered":"2024-09-21T07:46:26.626072+0000","last_clean":"2024-09-21T07:46:26.626072+0000","last_became_active":"2024-09-21T07:46:25.592303+0000","last_became_peered":"2024-09-21T07:46:25.592303+0000","last_unstale":"2024-09-21T07:46:26.626072+0000","last_undegraded":"2024-09-21T07:46:26.626072+0000","last_fullsized":"2024-09-21T07:46:26.626072+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:25:51.090251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":224,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599255+0000","last_change":"2024-09-21T07:35:27.876021+0000","last_active":"2024-09-21T07:46:27.599255+0000","last_peered":"2024-09-21T07:46:27.599255+0000","last_clean":"2024-09-21T07:46:27.599255+0000","last_became_active":"2024-09-21T07:35:27.875724+0000","last_became_peered":"2024-09-21T07:35:27.875724+0000","last_unstale":"2024-09-21T07:46:27.599255+0000","last_undegraded":"2024-09-21T07:46:27.599255+0000","last_fullsized":"2024-09-21T07:46:27.599255+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:03.022812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":232,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.604201+0000","last_change":"2024-09-21T07:35:29.492570+0000","last_active":"2024-09-21T07:46:27.604201+0000","last_peered":"2024-09-21T07:46:27.604201+0000","last_clean":"2024-09-21T07:46:27.604201+0000","last_became_active":"2024-09-21T07:35:26.874316+0000","last_became_peered":"2024-09-21T07:35:26.874316+0000","last_unstale":"2024-09-21T07:46:27.604201+0000","last_undegraded":"2024-09-21T07:46:27.604201+0000","last_fullsized":"2024-09-21T07:46:27.604201+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:16.512323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011990760000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":282,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599291+0000","last_change":"2024-09-21T07:34:25.085232+0000","last_active":"2024-09-21T07:46:27.599291+0000","last_peered":"2024-09-21T07:46:27.599291+0000","last_clean":"2024-09-21T07:46:27.599291+0000","last_became_active":"2024-09-21T07:34:23.277807+0000","last_became_peered":"2024-09-21T07:34:23.277807+0000","last_unstale":"2024-09-21T07:46:27.599291+0000","last_undegraded":"2024-09-21T07:46:27.599291+0000","last_fullsized":"2024-09-21T07:46:27.599291+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:36.362919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086619099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.625960+0000","last_change":"2024-09-21T07:46:26.620027+0000","last_active":"2024-09-21T07:46:26.625960+0000","last_peered":"2024-09-21T07:46:26.625960+0000","last_clean":"2024-09-21T07:46:26.625960+0000","last_became_active":"2024-09-21T07:46:26.619485+0000","last_became_peered":"2024-09-21T07:46:26.619485+0000","last_unstale":"2024-09-21T07:46:26.625960+0000","last_undegraded":"2024-09-21T07:46:26.625960+0000","last_fullsized":"2024-09-21T07:46:26.625960+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:59:28.471988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"174'2","reported_seq":17,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.604494+0000","last_change":"2024-09-21T07:46:25.606943+0000","last_active":"2024-09-21T07:46:27.604494+0000","last_peered":"2024-09-21T07:46:27.604494+0000","last_clean":"2024-09-21T07:46:27.604494+0000","last_became_active":"2024-09-21T07:46:25.606617+0000","last_became_peered":"2024-09-21T07:46:25.606617+0000","last_unstale":"2024-09-21T07:46:27.604494+0000","last_undegraded":"2024-09-21T07:46:27.604494+0000","last_fullsized":"2024-09-21T07:46:27.604494+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:27.687486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":224,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.608132+0000","last_change":"2024-09-21T07:35:27.872798+0000","last_active":"2024-09-21T07:46:26.608132+0000","last_peered":"2024-09-21T07:46:26.608132+0000","last_clean":"2024-09-21T07:46:26.608132+0000","last_became_active":"2024-09-21T07:35:27.872483+0000","last_became_peered":"2024-09-21T07:35:27.872483+0000","last_unstale":"2024-09-21T07:46:26.608132+0000","last_undegraded":"2024-09-21T07:46:26.608132+0000","last_fullsized":"2024-09-21T07:46:26.608132+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:11.933469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":231,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.607440+0000","last_change":"2024-09-21T07:35:29.206723+0000","last_active":"2024-09-21T07:46:26.607440+0000","last_peered":"2024-09-21T07:46:26.607440+0000","last_clean":"2024-09-21T07:46:26.607440+0000","last_became_active":"2024-09-21T07:35:26.872377+0000","last_became_peered":"2024-09-21T07:35:26.872377+0000","last_unstale":"2024-09-21T07:46:26.607440+0000","last_undegraded":"2024-09-21T07:46:26.607440+0000","last_fullsized":"2024-09-21T07:46:26.607440+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:29.086862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083077200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":276,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.615675+0000","last_change":"2024-09-21T07:34:24.597672+0000","last_active":"2024-09-21T07:46:26.615675+0000","last_peered":"2024-09-21T07:46:26.615675+0000","last_clean":"2024-09-21T07:46:26.615675+0000","last_became_active":"2024-09-21T07:34:23.267770+0000","last_became_peered":"2024-09-21T07:34:23.267770+0000","last_unstale":"2024-09-21T07:46:26.615675+0000","last_undegraded":"2024-09-21T07:46:26.615675+0000","last_fullsized":"2024-09-21T07:46:26.615675+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:51.629476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086378500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.604975+0000","last_change":"2024-09-21T07:46:26.620299+0000","last_active":"2024-09-21T07:46:27.604975+0000","last_peered":"2024-09-21T07:46:27.604975+0000","last_clean":"2024-09-21T07:46:27.604975+0000","last_became_active":"2024-09-21T07:46:26.619941+0000","last_became_peered":"2024-09-21T07:46:26.619941+0000","last_unstale":"2024-09-21T07:46:27.604975+0000","last_undegraded":"2024-09-21T07:46:27.604975+0000","last_fullsized":"2024-09-21T07:46:27.604975+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:55:53.588662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"174'2","reported_seq":15,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.631050+0000","last_change":"2024-09-21T07:46:25.600068+0000","last_active":"2024-09-21T07:46:26.631050+0000","last_peered":"2024-09-21T07:46:26.631050+0000","last_clean":"2024-09-21T07:46:26.631050+0000","last_became_active":"2024-09-21T07:46:25.598819+0000","last_became_peered":"2024-09-21T07:46:25.598819+0000","last_unstale":"2024-09-21T07:46:26.631050+0000","last_undegraded":"2024-09-21T07:46:26.631050+0000","last_fullsized":"2024-09-21T07:46:26.631050+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:26:06.303080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":224,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.607343+0000","last_change":"2024-09-21T07:35:27.870121+0000","last_active":"2024-09-21T07:46:26.607343+0000","last_peered":"2024-09-21T07:46:26.607343+0000","last_clean":"2024-09-21T07:46:26.607343+0000","last_became_active":"2024-09-21T07:35:27.869811+0000","last_became_peered":"2024-09-21T07:35:27.869811+0000","last_unstale":"2024-09-21T07:46:26.607343+0000","last_undegraded":"2024-09-21T07:46:26.607343+0000","last_fullsized":"2024-09-21T07:46:26.607343+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:31.778461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":232,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.600765+0000","last_change":"2024-09-21T07:35:28.860699+0000","last_active":"2024-09-21T07:46:27.600765+0000","last_peered":"2024-09-21T07:46:27.600765+0000","last_clean":"2024-09-21T07:46:27.600765+0000","last_became_active":"2024-09-21T07:35:26.876779+0000","last_became_peered":"2024-09-21T07:35:26.876779+0000","last_unstale":"2024-09-21T07:46:27.600765+0000","last_undegraded":"2024-09-21T07:46:27.600765+0000","last_fullsized":"2024-09-21T07:46:27.600765+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:28.286403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000913188,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":284,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.601364+0000","last_change":"2024-09-21T07:34:24.260181+0000","last_active":"2024-09-21T07:46:27.601364+0000","last_peered":"2024-09-21T07:46:27.601364+0000","last_clean":"2024-09-21T07:46:27.601364+0000","last_became_active":"2024-09-21T07:34:23.275580+0000","last_became_peered":"2024-09-21T07:34:23.275580+0000","last_unstale":"2024-09-21T07:46:27.601364+0000","last_undegraded":"2024-09-21T07:46:27.601364+0000","last_fullsized":"2024-09-21T07:46:27.601364+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:49.857622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014706980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.622749+0000","last_change":"2024-09-21T07:46:26.614494+0000","last_active":"2024-09-21T07:46:26.622749+0000","last_peered":"2024-09-21T07:46:26.622749+0000","last_clean":"2024-09-21T07:46:26.622749+0000","last_became_active":"2024-09-21T07:46:26.601276+0000","last_became_peered":"2024-09-21T07:46:26.601276+0000","last_unstale":"2024-09-21T07:46:26.622749+0000","last_undegraded":"2024-09-21T07:46:26.622749+0000","last_fullsized":"2024-09-21T07:46:26.622749+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:49:25.294386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"174'4","reported_seq":17,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.627490+0000","last_change":"2024-09-21T07:46:25.590100+0000","last_active":"2024-09-21T07:46:26.627490+0000","last_peered":"2024-09-21T07:46:26.627490+0000","last_clean":"2024-09-21T07:46:26.627490+0000","last_became_active":"2024-09-21T07:46:25.589489+0000","last_became_peered":"2024-09-21T07:46:25.589489+0000","last_unstale":"2024-09-21T07:46:26.627490+0000","last_undegraded":"2024-09-21T07:46:26.627490+0000","last_fullsized":"2024-09-21T07:46:26.627490+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:15.158504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":225,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.583893+0000","last_change":"2024-09-21T07:35:27.875393+0000","last_active":"2024-09-21T07:46:26.583893+0000","last_peered":"2024-09-21T07:46:26.583893+0000","last_clean":"2024-09-21T07:46:26.583893+0000","last_became_active":"2024-09-21T07:35:27.874671+0000","last_became_peered":"2024-09-21T07:35:27.874671+0000","last_unstale":"2024-09-21T07:46:26.583893+0000","last_undegraded":"2024-09-21T07:46:26.583893+0000","last_fullsized":"2024-09-21T07:46:26.583893+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":231,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.606543+0000","last_change":"2024-09-21T07:35:29.206296+0000","last_active":"2024-09-21T07:46:26.606543+0000","last_peered":"2024-09-21T07:46:26.606543+0000","last_clean":"2024-09-21T07:46:26.606543+0000","last_became_active":"2024-09-21T07:35:26.874575+0000","last_became_peered":"2024-09-21T07:35:26.874575+0000","last_unstale":"2024-09-21T07:46:26.606543+0000","last_undegraded":"2024-09-21T07:46:26.606543+0000","last_fullsized":"2024-09-21T07:46:26.606543+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:22.771154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061011599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":284,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.590311+0000","last_change":"2024-09-21T07:34:24.259112+0000","last_active":"2024-09-21T07:46:26.590311+0000","last_peered":"2024-09-21T07:46:26.590311+0000","last_clean":"2024-09-21T07:46:26.590311+0000","last_became_active":"2024-09-21T07:34:23.635444+0000","last_became_peered":"2024-09-21T07:34:23.635444+0000","last_unstale":"2024-09-21T07:46:26.590311+0000","last_undegraded":"2024-09-21T07:46:26.590311+0000","last_fullsized":"2024-09-21T07:46:26.590311+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:46.255385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001952636,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.623924+0000","last_change":"2024-09-21T07:46:26.615601+0000","last_active":"2024-09-21T07:46:26.623924+0000","last_peered":"2024-09-21T07:46:26.623924+0000","last_clean":"2024-09-21T07:46:26.623924+0000","last_became_active":"2024-09-21T07:46:26.615269+0000","last_became_peered":"2024-09-21T07:46:26.615269+0000","last_unstale":"2024-09-21T07:46:26.623924+0000","last_undegraded":"2024-09-21T07:46:26.623924+0000","last_fullsized":"2024-09-21T07:46:26.623924+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:06.717177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"174'1","reported_seq":16,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.604583+0000","last_change":"2024-09-21T07:46:25.608137+0000","last_active":"2024-09-21T07:46:27.604583+0000","last_peered":"2024-09-21T07:46:27.604583+0000","last_clean":"2024-09-21T07:46:27.604583+0000","last_became_active":"2024-09-21T07:46:25.607728+0000","last_became_peered":"2024-09-21T07:46:25.607728+0000","last_unstale":"2024-09-21T07:46:27.604583+0000","last_undegraded":"2024-09-21T07:46:27.604583+0000","last_fullsized":"2024-09-21T07:46:27.604583+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:43:30.481435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":225,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.582087+0000","last_change":"2024-09-21T07:35:27.878356+0000","last_active":"2024-09-21T07:46:26.582087+0000","last_peered":"2024-09-21T07:46:26.582087+0000","last_clean":"2024-09-21T07:46:26.582087+0000","last_became_active":"2024-09-21T07:35:27.877982+0000","last_became_peered":"2024-09-21T07:35:27.877982+0000","last_unstale":"2024-09-21T07:46:26.582087+0000","last_undegraded":"2024-09-21T07:46:26.582087+0000","last_fullsized":"2024-09-21T07:46:26.582087+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":232,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.582045+0000","last_change":"2024-09-21T07:35:29.053131+0000","last_active":"2024-09-21T07:46:26.582045+0000","last_peered":"2024-09-21T07:46:26.582045+0000","last_clean":"2024-09-21T07:46:26.582045+0000","last_became_active":"2024-09-21T07:35:26.870516+0000","last_became_peered":"2024-09-21T07:35:26.870516+0000","last_unstale":"2024-09-21T07:46:26.582045+0000","last_undegraded":"2024-09-21T07:46:26.582045+0000","last_fullsized":"2024-09-21T07:46:26.582045+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:10.924491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070705699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":291,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.600658+0000","last_change":"2024-09-21T07:34:24.539152+0000","last_active":"2024-09-21T07:46:27.600658+0000","last_peered":"2024-09-21T07:46:27.600658+0000","last_clean":"2024-09-21T07:46:27.600658+0000","last_became_active":"2024-09-21T07:34:23.259573+0000","last_became_peered":"2024-09-21T07:34:23.259573+0000","last_unstale":"2024-09-21T07:46:27.600658+0000","last_undegraded":"2024-09-21T07:46:27.600658+0000","last_fullsized":"2024-09-21T07:46:27.600658+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:26.986900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058108400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.624237+0000","last_change":"2024-09-21T07:46:26.620012+0000","last_active":"2024-09-21T07:46:26.624237+0000","last_peered":"2024-09-21T07:46:26.624237+0000","last_clean":"2024-09-21T07:46:26.624237+0000","last_became_active":"2024-09-21T07:46:26.615667+0000","last_became_peered":"2024-09-21T07:46:26.615667+0000","last_unstale":"2024-09-21T07:46:26.624237+0000","last_undegraded":"2024-09-21T07:46:26.624237+0000","last_fullsized":"2024-09-21T07:46:26.624237+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:45:57.255990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":15,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599600+0000","last_change":"2024-09-21T07:46:25.598178+0000","last_active":"2024-09-21T07:46:27.599600+0000","last_peered":"2024-09-21T07:46:27.599600+0000","last_clean":"2024-09-21T07:46:27.599600+0000","last_became_active":"2024-09-21T07:46:25.597252+0000","last_became_peered":"2024-09-21T07:46:25.597252+0000","last_unstale":"2024-09-21T07:46:27.599600+0000","last_undegraded":"2024-09-21T07:46:27.599600+0000","last_fullsized":"2024-09-21T07:46:27.599600+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:09.902385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":224,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.581917+0000","last_change":"2024-09-21T07:35:27.873726+0000","last_active":"2024-09-21T07:46:26.581917+0000","last_peered":"2024-09-21T07:46:26.581917+0000","last_clean":"2024-09-21T07:46:26.581917+0000","last_became_active":"2024-09-21T07:35:27.871421+0000","last_became_peered":"2024-09-21T07:35:27.871421+0000","last_unstale":"2024-09-21T07:46:26.581917+0000","last_undegraded":"2024-09-21T07:46:26.581917+0000","last_fullsized":"2024-09-21T07:46:26.581917+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:00.168915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":232,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.601191+0000","last_change":"2024-09-21T07:35:28.861032+0000","last_active":"2024-09-21T07:46:27.601191+0000","last_peered":"2024-09-21T07:46:27.601191+0000","last_clean":"2024-09-21T07:46:27.601191+0000","last_became_active":"2024-09-21T07:35:26.873149+0000","last_became_peered":"2024-09-21T07:35:26.873149+0000","last_unstale":"2024-09-21T07:46:27.601191+0000","last_undegraded":"2024-09-21T07:46:27.601191+0000","last_fullsized":"2024-09-21T07:46:27.601191+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:59.435981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011988369999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":285,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.604002+0000","last_change":"2024-09-21T07:34:24.259902+0000","last_active":"2024-09-21T07:46:27.604002+0000","last_peered":"2024-09-21T07:46:27.604002+0000","last_clean":"2024-09-21T07:46:27.604002+0000","last_became_active":"2024-09-21T07:34:23.271902+0000","last_became_peered":"2024-09-21T07:34:23.271902+0000","last_unstale":"2024-09-21T07:46:27.604002+0000","last_undegraded":"2024-09-21T07:46:27.604002+0000","last_fullsized":"2024-09-21T07:46:27.604002+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:59.439525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001686358,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.626050+0000","last_change":"2024-09-21T07:46:26.618986+0000","last_active":"2024-09-21T07:46:26.626050+0000","last_peered":"2024-09-21T07:46:26.626050+0000","last_clean":"2024-09-21T07:46:26.626050+0000","last_became_active":"2024-09-21T07:46:26.618685+0000","last_became_peered":"2024-09-21T07:46:26.618685+0000","last_unstale":"2024-09-21T07:46:26.626050+0000","last_undegraded":"2024-09-21T07:46:26.626050+0000","last_fullsized":"2024-09-21T07:46:26.626050+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:43:05.973019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":15,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.603794+0000","last_change":"2024-09-21T07:46:25.599666+0000","last_active":"2024-09-21T07:46:27.603794+0000","last_peered":"2024-09-21T07:46:27.603794+0000","last_clean":"2024-09-21T07:46:27.603794+0000","last_became_active":"2024-09-21T07:46:25.598879+0000","last_became_peered":"2024-09-21T07:46:25.598879+0000","last_unstale":"2024-09-21T07:46:27.603794+0000","last_undegraded":"2024-09-21T07:46:27.603794+0000","last_fullsized":"2024-09-21T07:46:27.603794+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:46.170670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":223,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.598903+0000","last_change":"2024-09-21T07:35:27.872637+0000","last_active":"2024-09-21T07:46:27.598903+0000","last_peered":"2024-09-21T07:46:27.598903+0000","last_clean":"2024-09-21T07:46:27.598903+0000","last_became_active":"2024-09-21T07:35:27.872149+0000","last_became_peered":"2024-09-21T07:35:27.872149+0000","last_unstale":"2024-09-21T07:46:27.598903+0000","last_undegraded":"2024-09-21T07:46:27.598903+0000","last_fullsized":"2024-09-21T07:46:27.598903+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:27.697872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":232,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.604683+0000","last_change":"2024-09-21T07:35:29.493362+0000","last_active":"2024-09-21T07:46:27.604683+0000","last_peered":"2024-09-21T07:46:27.604683+0000","last_clean":"2024-09-21T07:46:27.604683+0000","last_became_active":"2024-09-21T07:35:26.873018+0000","last_became_peered":"2024-09-21T07:35:26.873018+0000","last_unstale":"2024-09-21T07:46:27.604683+0000","last_undegraded":"2024-09-21T07:46:27.604683+0000","last_fullsized":"2024-09-21T07:46:27.604683+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:02.460557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012496510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":284,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.590535+0000","last_change":"2024-09-21T07:34:24.259662+0000","last_active":"2024-09-21T07:46:26.590535+0000","last_peered":"2024-09-21T07:46:26.590535+0000","last_clean":"2024-09-21T07:46:26.590535+0000","last_became_active":"2024-09-21T07:34:23.278812+0000","last_became_peered":"2024-09-21T07:34:23.278812+0000","last_unstale":"2024-09-21T07:46:26.590535+0000","last_undegraded":"2024-09-21T07:46:26.590535+0000","last_fullsized":"2024-09-21T07:46:26.590535+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:49.829974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356463,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.622065+0000","last_change":"2024-09-21T07:46:26.611463+0000","last_active":"2024-09-21T07:46:26.622065+0000","last_peered":"2024-09-21T07:46:26.622065+0000","last_clean":"2024-09-21T07:46:26.622065+0000","last_became_active":"2024-09-21T07:46:26.611043+0000","last_became_peered":"2024-09-21T07:46:26.611043+0000","last_unstale":"2024-09-21T07:46:26.622065+0000","last_undegraded":"2024-09-21T07:46:26.622065+0000","last_fullsized":"2024-09-21T07:46:26.622065+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:11.692802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":15,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599832+0000","last_change":"2024-09-21T07:46:25.596861+0000","last_active":"2024-09-21T07:46:27.599832+0000","last_peered":"2024-09-21T07:46:27.599832+0000","last_clean":"2024-09-21T07:46:27.599832+0000","last_became_active":"2024-09-21T07:46:25.596502+0000","last_became_peered":"2024-09-21T07:46:25.596502+0000","last_unstale":"2024-09-21T07:46:27.599832+0000","last_undegraded":"2024-09-21T07:46:27.599832+0000","last_fullsized":"2024-09-21T07:46:27.599832+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:03:08.985516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":219,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.607681+0000","last_change":"2024-09-21T07:35:27.868448+0000","last_active":"2024-09-21T07:46:26.607681+0000","last_peered":"2024-09-21T07:46:26.607681+0000","last_clean":"2024-09-21T07:46:26.607681+0000","last_became_active":"2024-09-21T07:35:27.868105+0000","last_became_peered":"2024-09-21T07:35:27.868105+0000","last_unstale":"2024-09-21T07:46:26.607681+0000","last_undegraded":"2024-09-21T07:46:26.607681+0000","last_fullsized":"2024-09-21T07:46:26.607681+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:41.468702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":231,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.604712+0000","last_change":"2024-09-21T07:35:29.492974+0000","last_active":"2024-09-21T07:46:27.604712+0000","last_peered":"2024-09-21T07:46:27.604712+0000","last_clean":"2024-09-21T07:46:27.604712+0000","last_became_active":"2024-09-21T07:35:26.872165+0000","last_became_peered":"2024-09-21T07:35:26.872165+0000","last_unstale":"2024-09-21T07:46:27.604712+0000","last_undegraded":"2024-09-21T07:46:27.604712+0000","last_fullsized":"2024-09-21T07:46:27.604712+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:59.912520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013795179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":282,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.598450+0000","last_change":"2024-09-21T07:34:25.084861+0000","last_active":"2024-09-21T07:46:27.598450+0000","last_peered":"2024-09-21T07:46:27.598450+0000","last_clean":"2024-09-21T07:46:27.598450+0000","last_became_active":"2024-09-21T07:34:23.277708+0000","last_became_peered":"2024-09-21T07:34:23.277708+0000","last_unstale":"2024-09-21T07:46:27.598450+0000","last_undegraded":"2024-09-21T07:46:27.598450+0000","last_fullsized":"2024-09-21T07:46:27.598450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:40.816488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000597566,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.618746+0000","last_change":"2024-09-21T07:46:26.615012+0000","last_active":"2024-09-21T07:46:26.618746+0000","last_peered":"2024-09-21T07:46:26.618746+0000","last_clean":"2024-09-21T07:46:26.618746+0000","last_became_active":"2024-09-21T07:46:26.614673+0000","last_became_peered":"2024-09-21T07:46:26.614673+0000","last_unstale":"2024-09-21T07:46:26.618746+0000","last_undegraded":"2024-09-21T07:46:26.618746+0000","last_fullsized":"2024-09-21T07:46:26.618746+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:28.357288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":15,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.600651+0000","last_change":"2024-09-21T07:46:25.599045+0000","last_active":"2024-09-21T07:46:27.600651+0000","last_peered":"2024-09-21T07:46:27.600651+0000","last_clean":"2024-09-21T07:46:27.600651+0000","last_became_active":"2024-09-21T07:46:25.596844+0000","last_became_peered":"2024-09-21T07:46:25.596844+0000","last_unstale":"2024-09-21T07:46:27.600651+0000","last_undegraded":"2024-09-21T07:46:27.600651+0000","last_fullsized":"2024-09-21T07:46:27.600651+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:21:36.983547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":219,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.609922+0000","last_change":"2024-09-21T07:35:27.874640+0000","last_active":"2024-09-21T07:46:26.609922+0000","last_peered":"2024-09-21T07:46:26.609922+0000","last_clean":"2024-09-21T07:46:26.609922+0000","last_became_active":"2024-09-21T07:35:27.874265+0000","last_became_peered":"2024-09-21T07:35:27.874265+0000","last_unstale":"2024-09-21T07:46:26.609922+0000","last_undegraded":"2024-09-21T07:46:26.609922+0000","last_fullsized":"2024-09-21T07:46:26.609922+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:45.754654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":230,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.608366+0000","last_change":"2024-09-21T07:35:29.207392+0000","last_active":"2024-09-21T07:46:26.608366+0000","last_peered":"2024-09-21T07:46:26.608366+0000","last_clean":"2024-09-21T07:46:26.608366+0000","last_became_active":"2024-09-21T07:35:26.870917+0000","last_became_peered":"2024-09-21T07:35:26.870917+0000","last_unstale":"2024-09-21T07:46:26.608366+0000","last_undegraded":"2024-09-21T07:46:26.608366+0000","last_fullsized":"2024-09-21T07:46:26.608366+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:59.159937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059198499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":284,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.587942+0000","last_change":"2024-09-21T07:34:24.258709+0000","last_active":"2024-09-21T07:46:26.587942+0000","last_peered":"2024-09-21T07:46:26.587942+0000","last_clean":"2024-09-21T07:46:26.587942+0000","last_became_active":"2024-09-21T07:34:23.279042+0000","last_became_peered":"2024-09-21T07:34:23.279042+0000","last_unstale":"2024-09-21T07:46:26.587942+0000","last_undegraded":"2024-09-21T07:46:26.587942+0000","last_fullsized":"2024-09-21T07:46:26.587942+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:01.762117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019282100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.601198+0000","last_change":"2024-09-21T07:46:26.609601+0000","last_active":"2024-09-21T07:46:27.601198+0000","last_peered":"2024-09-21T07:46:27.601198+0000","last_clean":"2024-09-21T07:46:27.601198+0000","last_became_active":"2024-09-21T07:46:26.608772+0000","last_became_peered":"2024-09-21T07:46:26.608772+0000","last_unstale":"2024-09-21T07:46:27.601198+0000","last_undegraded":"2024-09-21T07:46:27.601198+0000","last_fullsized":"2024-09-21T07:46:27.601198+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:08:40.114009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":13,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.582709+0000","last_change":"2024-09-21T07:46:25.601134+0000","last_active":"2024-09-21T07:46:26.582709+0000","last_peered":"2024-09-21T07:46:26.582709+0000","last_clean":"2024-09-21T07:46:26.582709+0000","last_became_active":"2024-09-21T07:46:25.599488+0000","last_became_peered":"2024-09-21T07:46:25.599488+0000","last_unstale":"2024-09-21T07:46:26.582709+0000","last_undegraded":"2024-09-21T07:46:26.582709+0000","last_fullsized":"2024-09-21T07:46:26.582709+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:08:39.105569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":223,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599652+0000","last_change":"2024-09-21T07:35:27.870260+0000","last_active":"2024-09-21T07:46:27.599652+0000","last_peered":"2024-09-21T07:46:27.599652+0000","last_clean":"2024-09-21T07:46:27.599652+0000","last_became_active":"2024-09-21T07:35:27.869448+0000","last_became_peered":"2024-09-21T07:35:27.869448+0000","last_unstale":"2024-09-21T07:46:27.599652+0000","last_undegraded":"2024-09-21T07:46:27.599652+0000","last_fullsized":"2024-09-21T07:46:27.599652+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:16.946790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":226,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.609808+0000","last_change":"2024-09-21T07:35:29.386367+0000","last_active":"2024-09-21T07:46:26.609808+0000","last_peered":"2024-09-21T07:46:26.609808+0000","last_clean":"2024-09-21T07:46:26.609808+0000","last_became_active":"2024-09-21T07:35:26.879238+0000","last_became_peered":"2024-09-21T07:35:26.879238+0000","last_unstale":"2024-09-21T07:46:26.609808+0000","last_undegraded":"2024-09-21T07:46:26.609808+0000","last_fullsized":"2024-09-21T07:46:26.609808+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:16.528260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092223100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":282,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.598833+0000","last_change":"2024-09-21T07:34:25.085635+0000","last_active":"2024-09-21T07:46:27.598833+0000","last_peered":"2024-09-21T07:46:27.598833+0000","last_clean":"2024-09-21T07:46:27.598833+0000","last_became_active":"2024-09-21T07:34:23.273566+0000","last_became_peered":"2024-09-21T07:34:23.273566+0000","last_unstale":"2024-09-21T07:46:27.598833+0000","last_undegraded":"2024-09-21T07:46:27.598833+0000","last_fullsized":"2024-09-21T07:46:27.598833+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:14.142272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079897300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.613609+0000","last_change":"2024-09-21T07:46:26.611680+0000","last_active":"2024-09-21T07:46:26.613609+0000","last_peered":"2024-09-21T07:46:26.613609+0000","last_clean":"2024-09-21T07:46:26.613609+0000","last_became_active":"2024-09-21T07:46:26.611369+0000","last_became_peered":"2024-09-21T07:46:26.611369+0000","last_unstale":"2024-09-21T07:46:26.613609+0000","last_undegraded":"2024-09-21T07:46:26.613609+0000","last_fullsized":"2024-09-21T07:46:26.613609+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:05:44.283123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":13,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.590091+0000","last_change":"2024-09-21T07:46:25.595917+0000","last_active":"2024-09-21T07:46:26.590091+0000","last_peered":"2024-09-21T07:46:26.590091+0000","last_clean":"2024-09-21T07:46:26.590091+0000","last_became_active":"2024-09-21T07:46:25.595403+0000","last_became_peered":"2024-09-21T07:46:25.595403+0000","last_unstale":"2024-09-21T07:46:26.590091+0000","last_undegraded":"2024-09-21T07:46:26.590091+0000","last_fullsized":"2024-09-21T07:46:26.590091+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:25:41.466314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":223,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.598756+0000","last_change":"2024-09-21T07:35:27.878728+0000","last_active":"2024-09-21T07:46:27.598756+0000","last_peered":"2024-09-21T07:46:27.598756+0000","last_clean":"2024-09-21T07:46:27.598756+0000","last_became_active":"2024-09-21T07:35:27.878335+0000","last_became_peered":"2024-09-21T07:35:27.878335+0000","last_unstale":"2024-09-21T07:46:27.598756+0000","last_undegraded":"2024-09-21T07:46:27.598756+0000","last_fullsized":"2024-09-21T07:46:27.598756+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:25:37.731771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":238,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.600021+0000","last_change":"2024-09-21T07:35:28.858418+0000","last_active":"2024-09-21T07:46:27.600021+0000","last_peered":"2024-09-21T07:46:27.600021+0000","last_clean":"2024-09-21T07:46:27.600021+0000","last_became_active":"2024-09-21T07:35:26.876944+0000","last_became_peered":"2024-09-21T07:35:26.876944+0000","last_unstale":"2024-09-21T07:46:27.600021+0000","last_undegraded":"2024-09-21T07:46:27.600021+0000","last_fullsized":"2024-09-21T07:46:27.600021+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:04.514304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090862799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":275,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.610848+0000","last_change":"2024-09-21T07:34:24.597926+0000","last_active":"2024-09-21T07:46:26.610848+0000","last_peered":"2024-09-21T07:46:26.610848+0000","last_clean":"2024-09-21T07:46:26.610848+0000","last_became_active":"2024-09-21T07:34:23.268926+0000","last_became_peered":"2024-09-21T07:34:23.268926+0000","last_unstale":"2024-09-21T07:46:26.610848+0000","last_undegraded":"2024-09-21T07:46:26.610848+0000","last_fullsized":"2024-09-21T07:46:26.610848+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:46.190752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101851,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.609925+0000","last_change":"2024-09-21T07:46:26.600244+0000","last_active":"2024-09-21T07:46:26.609925+0000","last_peered":"2024-09-21T07:46:26.609925+0000","last_clean":"2024-09-21T07:46:26.609925+0000","last_became_active":"2024-09-21T07:46:26.599918+0000","last_became_peered":"2024-09-21T07:46:26.599918+0000","last_unstale":"2024-09-21T07:46:26.609925+0000","last_undegraded":"2024-09-21T07:46:26.609925+0000","last_fullsized":"2024-09-21T07:46:26.609925+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:34:39.369981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"174'2","reported_seq":15,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.629374+0000","last_change":"2024-09-21T07:46:25.604769+0000","last_active":"2024-09-21T07:46:26.629374+0000","last_peered":"2024-09-21T07:46:26.629374+0000","last_clean":"2024-09-21T07:46:26.629374+0000","last_became_active":"2024-09-21T07:46:25.603758+0000","last_became_peered":"2024-09-21T07:46:25.603758+0000","last_unstale":"2024-09-21T07:46:26.629374+0000","last_undegraded":"2024-09-21T07:46:26.629374+0000","last_fullsized":"2024-09-21T07:46:26.629374+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:10.336493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":220,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.611594+0000","last_change":"2024-09-21T07:35:27.877426+0000","last_active":"2024-09-21T07:46:26.611594+0000","last_peered":"2024-09-21T07:46:26.611594+0000","last_clean":"2024-09-21T07:46:26.611594+0000","last_became_active":"2024-09-21T07:35:27.877050+0000","last_became_peered":"2024-09-21T07:35:27.877050+0000","last_unstale":"2024-09-21T07:46:26.611594+0000","last_undegraded":"2024-09-21T07:46:26.611594+0000","last_fullsized":"2024-09-21T07:46:26.611594+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:36.268334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":230,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599007+0000","last_change":"2024-09-21T07:35:28.862273+0000","last_active":"2024-09-21T07:46:27.599007+0000","last_peered":"2024-09-21T07:46:27.599007+0000","last_clean":"2024-09-21T07:46:27.599007+0000","last_became_active":"2024-09-21T07:35:26.874011+0000","last_became_peered":"2024-09-21T07:35:26.874011+0000","last_unstale":"2024-09-21T07:46:27.599007+0000","last_undegraded":"2024-09-21T07:46:27.599007+0000","last_fullsized":"2024-09-21T07:46:27.599007+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:11.986950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071644799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":284,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.600814+0000","last_change":"2024-09-21T07:34:24.259794+0000","last_active":"2024-09-21T07:46:27.600814+0000","last_peered":"2024-09-21T07:46:27.600814+0000","last_clean":"2024-09-21T07:46:27.600814+0000","last_became_active":"2024-09-21T07:34:23.265679+0000","last_became_peered":"2024-09-21T07:34:23.265679+0000","last_unstale":"2024-09-21T07:46:27.600814+0000","last_undegraded":"2024-09-21T07:46:27.600814+0000","last_fullsized":"2024-09-21T07:46:27.600814+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:02.575796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001298518,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.618643+0000","last_change":"2024-09-21T07:46:26.613402+0000","last_active":"2024-09-21T07:46:26.618643+0000","last_peered":"2024-09-21T07:46:26.618643+0000","last_clean":"2024-09-21T07:46:26.618643+0000","last_became_active":"2024-09-21T07:46:26.611549+0000","last_became_peered":"2024-09-21T07:46:26.611549+0000","last_unstale":"2024-09-21T07:46:26.618643+0000","last_undegraded":"2024-09-21T07:46:26.618643+0000","last_fullsized":"2024-09-21T07:46:26.618643+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:01.491090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"175'1","reported_seq":16,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.603115+0000","last_change":"2024-09-21T07:46:25.606263+0000","last_active":"2024-09-21T07:46:27.603115+0000","last_peered":"2024-09-21T07:46:27.603115+0000","last_clean":"2024-09-21T07:46:27.603115+0000","last_became_active":"2024-09-21T07:46:25.605327+0000","last_became_peered":"2024-09-21T07:46:25.605327+0000","last_unstale":"2024-09-21T07:46:27.603115+0000","last_undegraded":"2024-09-21T07:46:27.603115+0000","last_fullsized":"2024-09-21T07:46:27.603115+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:28.394690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":224,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.606690+0000","last_change":"2024-09-21T07:35:27.877725+0000","last_active":"2024-09-21T07:46:26.606690+0000","last_peered":"2024-09-21T07:46:26.606690+0000","last_clean":"2024-09-21T07:46:26.606690+0000","last_became_active":"2024-09-21T07:35:27.877398+0000","last_became_peered":"2024-09-21T07:35:27.877398+0000","last_unstale":"2024-09-21T07:46:26.606690+0000","last_undegraded":"2024-09-21T07:46:26.606690+0000","last_fullsized":"2024-09-21T07:46:26.606690+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:05.859762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":231,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.600206+0000","last_change":"2024-09-21T07:35:28.860612+0000","last_active":"2024-09-21T07:46:27.600206+0000","last_peered":"2024-09-21T07:46:27.600206+0000","last_clean":"2024-09-21T07:46:27.600206+0000","last_became_active":"2024-09-21T07:35:26.876562+0000","last_became_peered":"2024-09-21T07:35:26.876562+0000","last_unstale":"2024-09-21T07:46:27.600206+0000","last_undegraded":"2024-09-21T07:46:27.600206+0000","last_fullsized":"2024-09-21T07:46:27.600206+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:06.824331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012252470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":285,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.605010+0000","last_change":"2024-09-21T07:34:24.260330+0000","last_active":"2024-09-21T07:46:27.605010+0000","last_peered":"2024-09-21T07:46:27.605010+0000","last_clean":"2024-09-21T07:46:27.605010+0000","last_became_active":"2024-09-21T07:34:23.637210+0000","last_became_peered":"2024-09-21T07:34:23.637210+0000","last_unstale":"2024-09-21T07:46:27.605010+0000","last_undegraded":"2024-09-21T07:46:27.605010+0000","last_fullsized":"2024-09-21T07:46:27.605010+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.488451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001537093,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.603987+0000","last_change":"2024-09-21T07:46:26.614288+0000","last_active":"2024-09-21T07:46:27.603987+0000","last_peered":"2024-09-21T07:46:27.603987+0000","last_clean":"2024-09-21T07:46:27.603987+0000","last_became_active":"2024-09-21T07:46:26.613851+0000","last_became_peered":"2024-09-21T07:46:26.613851+0000","last_unstale":"2024-09-21T07:46:27.603987+0000","last_undegraded":"2024-09-21T07:46:27.603987+0000","last_fullsized":"2024-09-21T07:46:27.603987+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:39.078877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":15,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599234+0000","last_change":"2024-09-21T07:46:25.598295+0000","last_active":"2024-09-21T07:46:27.599234+0000","last_peered":"2024-09-21T07:46:27.599234+0000","last_clean":"2024-09-21T07:46:27.599234+0000","last_became_active":"2024-09-21T07:46:25.596166+0000","last_became_peered":"2024-09-21T07:46:25.596166+0000","last_unstale":"2024-09-21T07:46:27.599234+0000","last_undegraded":"2024-09-21T07:46:27.599234+0000","last_fullsized":"2024-09-21T07:46:27.599234+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:10.207013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":223,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.598267+0000","last_change":"2024-09-21T07:35:27.874757+0000","last_active":"2024-09-21T07:46:27.598267+0000","last_peered":"2024-09-21T07:46:27.598267+0000","last_clean":"2024-09-21T07:46:27.598267+0000","last_became_active":"2024-09-21T07:35:27.874360+0000","last_became_peered":"2024-09-21T07:35:27.874360+0000","last_unstale":"2024-09-21T07:46:27.598267+0000","last_undegraded":"2024-09-21T07:46:27.598267+0000","last_fullsized":"2024-09-21T07:46:27.598267+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:17.646912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":231,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.605022+0000","last_change":"2024-09-21T07:35:29.493431+0000","last_active":"2024-09-21T07:46:27.605022+0000","last_peered":"2024-09-21T07:46:27.605022+0000","last_clean":"2024-09-21T07:46:27.605022+0000","last_became_active":"2024-09-21T07:35:26.873691+0000","last_became_peered":"2024-09-21T07:35:26.873691+0000","last_unstale":"2024-09-21T07:46:27.605022+0000","last_undegraded":"2024-09-21T07:46:27.605022+0000","last_fullsized":"2024-09-21T07:46:27.605022+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:51.705051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001162464,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":275,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.607966+0000","last_change":"2024-09-21T07:34:24.597286+0000","last_active":"2024-09-21T07:46:26.607966+0000","last_peered":"2024-09-21T07:46:26.607966+0000","last_clean":"2024-09-21T07:46:26.607966+0000","last_became_active":"2024-09-21T07:34:23.267133+0000","last_became_peered":"2024-09-21T07:34:23.267133+0000","last_unstale":"2024-09-21T07:46:26.607966+0000","last_undegraded":"2024-09-21T07:46:26.607966+0000","last_fullsized":"2024-09-21T07:46:26.607966+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:18.257010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.623910+0000","last_change":"2024-09-21T07:46:26.615395+0000","last_active":"2024-09-21T07:46:26.623910+0000","last_peered":"2024-09-21T07:46:26.623910+0000","last_clean":"2024-09-21T07:46:26.623910+0000","last_became_active":"2024-09-21T07:46:26.614999+0000","last_became_peered":"2024-09-21T07:46:26.614999+0000","last_unstale":"2024-09-21T07:46:26.623910+0000","last_undegraded":"2024-09-21T07:46:26.623910+0000","last_fullsized":"2024-09-21T07:46:26.623910+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:44.279162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":13,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.605011+0000","last_change":"2024-09-21T07:46:25.594840+0000","last_active":"2024-09-21T07:46:26.605011+0000","last_peered":"2024-09-21T07:46:26.605011+0000","last_clean":"2024-09-21T07:46:26.605011+0000","last_became_active":"2024-09-21T07:46:25.594415+0000","last_became_peered":"2024-09-21T07:46:25.594415+0000","last_unstale":"2024-09-21T07:46:26.605011+0000","last_undegraded":"2024-09-21T07:46:26.605011+0000","last_fullsized":"2024-09-21T07:46:26.605011+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:49.617381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":231,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599772+0000","last_change":"2024-09-21T07:35:27.864458+0000","last_active":"2024-09-21T07:46:27.599772+0000","last_peered":"2024-09-21T07:46:27.599772+0000","last_clean":"2024-09-21T07:46:27.599772+0000","last_became_active":"2024-09-21T07:35:27.864136+0000","last_became_peered":"2024-09-21T07:35:27.864136+0000","last_unstale":"2024-09-21T07:46:27.599772+0000","last_undegraded":"2024-09-21T07:46:27.599772+0000","last_fullsized":"2024-09-21T07:46:27.599772+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:20.162240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":238,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.601274+0000","last_change":"2024-09-21T07:35:28.858784+0000","last_active":"2024-09-21T07:46:27.601274+0000","last_peered":"2024-09-21T07:46:27.601274+0000","last_clean":"2024-09-21T07:46:27.601274+0000","last_became_active":"2024-09-21T07:35:26.876890+0000","last_became_peered":"2024-09-21T07:35:26.876890+0000","last_unstale":"2024-09-21T07:46:27.601274+0000","last_undegraded":"2024-09-21T07:46:27.601274+0000","last_fullsized":"2024-09-21T07:46:27.601274+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:40.738327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064543499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":283,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.590547+0000","last_change":"2024-09-21T07:34:24.258712+0000","last_active":"2024-09-21T07:46:26.590547+0000","last_peered":"2024-09-21T07:46:26.590547+0000","last_clean":"2024-09-21T07:46:26.590547+0000","last_became_active":"2024-09-21T07:34:23.274692+0000","last_became_peered":"2024-09-21T07:34:23.274692+0000","last_unstale":"2024-09-21T07:46:26.590547+0000","last_undegraded":"2024-09-21T07:46:26.590547+0000","last_fullsized":"2024-09-21T07:46:26.590547+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:43.279454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018051479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.603775+0000","last_change":"2024-09-21T07:46:26.607882+0000","last_active":"2024-09-21T07:46:27.603775+0000","last_peered":"2024-09-21T07:46:27.603775+0000","last_clean":"2024-09-21T07:46:27.603775+0000","last_became_active":"2024-09-21T07:46:26.607510+0000","last_became_peered":"2024-09-21T07:46:26.607510+0000","last_unstale":"2024-09-21T07:46:27.603775+0000","last_undegraded":"2024-09-21T07:46:27.603775+0000","last_fullsized":"2024-09-21T07:46:27.603775+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:26:07.311520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"174'2","reported_seq":17,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599128+0000","last_change":"2024-09-21T07:46:25.603718+0000","last_active":"2024-09-21T07:46:27.599128+0000","last_peered":"2024-09-21T07:46:27.599128+0000","last_clean":"2024-09-21T07:46:27.599128+0000","last_became_active":"2024-09-21T07:46:25.603388+0000","last_became_peered":"2024-09-21T07:46:25.603388+0000","last_unstale":"2024-09-21T07:46:27.599128+0000","last_undegraded":"2024-09-21T07:46:27.599128+0000","last_fullsized":"2024-09-21T07:46:27.599128+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:43:18.533795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":222,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.588528+0000","last_change":"2024-09-21T07:35:27.878020+0000","last_active":"2024-09-21T07:46:26.588528+0000","last_peered":"2024-09-21T07:46:26.588528+0000","last_clean":"2024-09-21T07:46:26.588528+0000","last_became_active":"2024-09-21T07:35:27.877532+0000","last_became_peered":"2024-09-21T07:35:27.877532+0000","last_unstale":"2024-09-21T07:46:26.588528+0000","last_undegraded":"2024-09-21T07:46:26.588528+0000","last_fullsized":"2024-09-21T07:46:26.588528+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:19.756073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":231,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.601177+0000","last_change":"2024-09-21T07:35:28.861049+0000","last_active":"2024-09-21T07:46:27.601177+0000","last_peered":"2024-09-21T07:46:27.601177+0000","last_clean":"2024-09-21T07:46:27.601177+0000","last_became_active":"2024-09-21T07:35:26.876819+0000","last_became_peered":"2024-09-21T07:35:26.876819+0000","last_unstale":"2024-09-21T07:46:27.601177+0000","last_undegraded":"2024-09-21T07:46:27.601177+0000","last_fullsized":"2024-09-21T07:46:27.601177+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:19.633803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055780599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":282,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.598386+0000","last_change":"2024-09-21T07:34:25.084849+0000","last_active":"2024-09-21T07:46:27.598386+0000","last_peered":"2024-09-21T07:46:27.598386+0000","last_clean":"2024-09-21T07:46:27.598386+0000","last_became_active":"2024-09-21T07:34:23.277713+0000","last_became_peered":"2024-09-21T07:34:23.277713+0000","last_unstale":"2024-09-21T07:46:27.598386+0000","last_undegraded":"2024-09-21T07:46:27.598386+0000","last_fullsized":"2024-09-21T07:46:27.598386+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:05.811810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059079499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.592232+0000","last_change":"2024-09-21T07:46:26.591457+0000","last_active":"2024-09-21T07:46:26.592232+0000","last_peered":"2024-09-21T07:46:26.592232+0000","last_clean":"2024-09-21T07:46:26.592232+0000","last_became_active":"2024-09-21T07:46:26.591098+0000","last_became_peered":"2024-09-21T07:46:26.591098+0000","last_unstale":"2024-09-21T07:46:26.592232+0000","last_undegraded":"2024-09-21T07:46:26.592232+0000","last_fullsized":"2024-09-21T07:46:26.592232+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:44.374583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":13,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.607171+0000","last_change":"2024-09-21T07:46:25.596493+0000","last_active":"2024-09-21T07:46:26.607171+0000","last_peered":"2024-09-21T07:46:26.607171+0000","last_clean":"2024-09-21T07:46:26.607171+0000","last_became_active":"2024-09-21T07:46:25.593838+0000","last_became_peered":"2024-09-21T07:46:25.593838+0000","last_unstale":"2024-09-21T07:46:26.607171+0000","last_undegraded":"2024-09-21T07:46:26.607171+0000","last_fullsized":"2024-09-21T07:46:26.607171+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:39:08.550666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":224,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.582290+0000","last_change":"2024-09-21T07:35:27.884248+0000","last_active":"2024-09-21T07:46:26.582290+0000","last_peered":"2024-09-21T07:46:26.582290+0000","last_clean":"2024-09-21T07:46:26.582290+0000","last_became_active":"2024-09-21T07:35:27.883861+0000","last_became_peered":"2024-09-21T07:35:27.883861+0000","last_unstale":"2024-09-21T07:46:26.582290+0000","last_undegraded":"2024-09-21T07:46:26.582290+0000","last_fullsized":"2024-09-21T07:46:26.582290+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":230,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.608256+0000","last_change":"2024-09-21T07:35:29.207135+0000","last_active":"2024-09-21T07:46:26.608256+0000","last_peered":"2024-09-21T07:46:26.608256+0000","last_clean":"2024-09-21T07:46:26.608256+0000","last_became_active":"2024-09-21T07:35:26.872787+0000","last_became_peered":"2024-09-21T07:35:26.872787+0000","last_unstale":"2024-09-21T07:46:26.608256+0000","last_undegraded":"2024-09-21T07:46:26.608256+0000","last_fullsized":"2024-09-21T07:46:26.608256+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:27.543217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011157509999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":291,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.600051+0000","last_change":"2024-09-21T07:34:24.538610+0000","last_active":"2024-09-21T07:46:27.600051+0000","last_peered":"2024-09-21T07:46:27.600051+0000","last_clean":"2024-09-21T07:46:27.600051+0000","last_became_active":"2024-09-21T07:34:23.259700+0000","last_became_peered":"2024-09-21T07:34:23.259700+0000","last_unstale":"2024-09-21T07:46:27.600051+0000","last_undegraded":"2024-09-21T07:46:27.600051+0000","last_fullsized":"2024-09-21T07:46:27.600051+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:57.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070522399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.600915+0000","last_change":"2024-09-21T07:46:26.591714+0000","last_active":"2024-09-21T07:46:27.600915+0000","last_peered":"2024-09-21T07:46:27.600915+0000","last_clean":"2024-09-21T07:46:27.600915+0000","last_became_active":"2024-09-21T07:46:26.591384+0000","last_became_peered":"2024-09-21T07:46:26.591384+0000","last_unstale":"2024-09-21T07:46:27.600915+0000","last_undegraded":"2024-09-21T07:46:27.600915+0000","last_fullsized":"2024-09-21T07:46:27.600915+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:01:25.928166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"174'1","reported_seq":14,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.630143+0000","last_change":"2024-09-21T07:46:25.600252+0000","last_active":"2024-09-21T07:46:26.630143+0000","last_peered":"2024-09-21T07:46:26.630143+0000","last_clean":"2024-09-21T07:46:26.630143+0000","last_became_active":"2024-09-21T07:46:25.599931+0000","last_became_peered":"2024-09-21T07:46:25.599931+0000","last_unstale":"2024-09-21T07:46:26.630143+0000","last_undegraded":"2024-09-21T07:46:26.630143+0000","last_fullsized":"2024-09-21T07:46:26.630143+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:38.070437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":231,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599790+0000","last_change":"2024-09-21T07:35:27.868024+0000","last_active":"2024-09-21T07:46:27.599790+0000","last_peered":"2024-09-21T07:46:27.599790+0000","last_clean":"2024-09-21T07:46:27.599790+0000","last_became_active":"2024-09-21T07:35:27.867339+0000","last_became_peered":"2024-09-21T07:35:27.867339+0000","last_unstale":"2024-09-21T07:46:27.599790+0000","last_undegraded":"2024-09-21T07:46:27.599790+0000","last_fullsized":"2024-09-21T07:46:27.599790+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":238,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599904+0000","last_change":"2024-09-21T07:35:28.858505+0000","last_active":"2024-09-21T07:46:27.599904+0000","last_peered":"2024-09-21T07:46:27.599904+0000","last_clean":"2024-09-21T07:46:27.599904+0000","last_became_active":"2024-09-21T07:35:26.877320+0000","last_became_peered":"2024-09-21T07:35:26.877320+0000","last_unstale":"2024-09-21T07:46:27.599904+0000","last_undegraded":"2024-09-21T07:46:27.599904+0000","last_fullsized":"2024-09-21T07:46:27.599904+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:53.672782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244119999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":277,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.615821+0000","last_change":"2024-09-21T07:34:24.597597+0000","last_active":"2024-09-21T07:46:26.615821+0000","last_peered":"2024-09-21T07:46:26.615821+0000","last_clean":"2024-09-21T07:46:26.615821+0000","last_became_active":"2024-09-21T07:34:23.635204+0000","last_became_peered":"2024-09-21T07:34:23.635204+0000","last_unstale":"2024-09-21T07:46:26.615821+0000","last_undegraded":"2024-09-21T07:46:26.615821+0000","last_fullsized":"2024-09-21T07:46:26.615821+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:48.644789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081139199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.601188+0000","last_change":"2024-09-21T07:46:26.607268+0000","last_active":"2024-09-21T07:46:27.601188+0000","last_peered":"2024-09-21T07:46:27.601188+0000","last_clean":"2024-09-21T07:46:27.601188+0000","last_became_active":"2024-09-21T07:46:26.606937+0000","last_became_peered":"2024-09-21T07:46:26.606937+0000","last_unstale":"2024-09-21T07:46:27.601188+0000","last_undegraded":"2024-09-21T07:46:27.601188+0000","last_fullsized":"2024-09-21T07:46:27.601188+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:49:37.916811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":13,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.608932+0000","last_change":"2024-09-21T07:46:25.603927+0000","last_active":"2024-09-21T07:46:26.608932+0000","last_peered":"2024-09-21T07:46:26.608932+0000","last_clean":"2024-09-21T07:46:26.608932+0000","last_became_active":"2024-09-21T07:46:25.603544+0000","last_became_peered":"2024-09-21T07:46:25.603544+0000","last_unstale":"2024-09-21T07:46:26.608932+0000","last_undegraded":"2024-09-21T07:46:26.608932+0000","last_fullsized":"2024-09-21T07:46:26.608932+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:02.105173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":223,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.606701+0000","last_change":"2024-09-21T07:35:27.877503+0000","last_active":"2024-09-21T07:46:26.606701+0000","last_peered":"2024-09-21T07:46:26.606701+0000","last_clean":"2024-09-21T07:46:26.606701+0000","last_became_active":"2024-09-21T07:35:27.875775+0000","last_became_peered":"2024-09-21T07:35:27.875775+0000","last_unstale":"2024-09-21T07:46:26.606701+0000","last_undegraded":"2024-09-21T07:46:26.606701+0000","last_fullsized":"2024-09-21T07:46:26.606701+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:49.291023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":231,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.603889+0000","last_change":"2024-09-21T07:35:29.491412+0000","last_active":"2024-09-21T07:46:27.603889+0000","last_peered":"2024-09-21T07:46:27.603889+0000","last_clean":"2024-09-21T07:46:27.603889+0000","last_became_active":"2024-09-21T07:35:26.876404+0000","last_became_peered":"2024-09-21T07:35:26.876404+0000","last_unstale":"2024-09-21T07:46:27.603889+0000","last_undegraded":"2024-09-21T07:46:27.603889+0000","last_fullsized":"2024-09-21T07:46:27.603889+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063918000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":284,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.590052+0000","last_change":"2024-09-21T07:34:24.259026+0000","last_active":"2024-09-21T07:46:26.590052+0000","last_peered":"2024-09-21T07:46:26.590052+0000","last_clean":"2024-09-21T07:46:26.590052+0000","last_became_active":"2024-09-21T07:34:23.635456+0000","last_became_peered":"2024-09-21T07:34:23.635456+0000","last_unstale":"2024-09-21T07:46:26.590052+0000","last_undegraded":"2024-09-21T07:46:26.590052+0000","last_fullsized":"2024-09-21T07:46:26.590052+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:09.592752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018654419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.604683+0000","last_change":"2024-09-21T07:46:26.614197+0000","last_active":"2024-09-21T07:46:27.604683+0000","last_peered":"2024-09-21T07:46:27.604683+0000","last_clean":"2024-09-21T07:46:27.604683+0000","last_became_active":"2024-09-21T07:46:26.613885+0000","last_became_peered":"2024-09-21T07:46:26.613885+0000","last_unstale":"2024-09-21T07:46:27.604683+0000","last_undegraded":"2024-09-21T07:46:27.604683+0000","last_fullsized":"2024-09-21T07:46:27.604683+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:51:31.780109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"174'1","reported_seq":16,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.600953+0000","last_change":"2024-09-21T07:46:25.600533+0000","last_active":"2024-09-21T07:46:27.600953+0000","last_peered":"2024-09-21T07:46:27.600953+0000","last_clean":"2024-09-21T07:46:27.600953+0000","last_became_active":"2024-09-21T07:46:25.600183+0000","last_became_peered":"2024-09-21T07:46:25.600183+0000","last_unstale":"2024-09-21T07:46:27.600953+0000","last_undegraded":"2024-09-21T07:46:27.600953+0000","last_fullsized":"2024-09-21T07:46:27.600953+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:48.453281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":223,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.606099+0000","last_change":"2024-09-21T07:35:27.871984+0000","last_active":"2024-09-21T07:46:26.606099+0000","last_peered":"2024-09-21T07:46:26.606099+0000","last_clean":"2024-09-21T07:46:26.606099+0000","last_became_active":"2024-09-21T07:35:27.870336+0000","last_became_peered":"2024-09-21T07:35:27.870336+0000","last_unstale":"2024-09-21T07:46:26.606099+0000","last_undegraded":"2024-09-21T07:46:26.606099+0000","last_fullsized":"2024-09-21T07:46:26.606099+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:57.120479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":231,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.600376+0000","last_change":"2024-09-21T07:35:28.860351+0000","last_active":"2024-09-21T07:46:27.600376+0000","last_peered":"2024-09-21T07:46:27.600376+0000","last_clean":"2024-09-21T07:46:27.600376+0000","last_became_active":"2024-09-21T07:35:26.872046+0000","last_became_peered":"2024-09-21T07:35:26.872046+0000","last_unstale":"2024-09-21T07:46:27.600376+0000","last_undegraded":"2024-09-21T07:46:27.600376+0000","last_fullsized":"2024-09-21T07:46:27.600376+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:47.074327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098384399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":284,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.604442+0000","last_change":"2024-09-21T07:34:24.260300+0000","last_active":"2024-09-21T07:46:27.604442+0000","last_peered":"2024-09-21T07:46:27.604442+0000","last_clean":"2024-09-21T07:46:27.604442+0000","last_became_active":"2024-09-21T07:34:23.270128+0000","last_became_peered":"2024-09-21T07:34:23.270128+0000","last_unstale":"2024-09-21T07:46:27.604442+0000","last_undegraded":"2024-09-21T07:46:27.604442+0000","last_fullsized":"2024-09-21T07:46:27.604442+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:25.260931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00176651,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.604878+0000","last_change":"2024-09-21T07:46:26.624447+0000","last_active":"2024-09-21T07:46:27.604878+0000","last_peered":"2024-09-21T07:46:27.604878+0000","last_clean":"2024-09-21T07:46:27.604878+0000","last_became_active":"2024-09-21T07:46:26.624148+0000","last_became_peered":"2024-09-21T07:46:26.624148+0000","last_unstale":"2024-09-21T07:46:27.604878+0000","last_undegraded":"2024-09-21T07:46:27.604878+0000","last_fullsized":"2024-09-21T07:46:27.604878+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:20.298916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"174'2","reported_seq":17,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.604608+0000","last_change":"2024-09-21T07:46:25.600776+0000","last_active":"2024-09-21T07:46:27.604608+0000","last_peered":"2024-09-21T07:46:27.604608+0000","last_clean":"2024-09-21T07:46:27.604608+0000","last_became_active":"2024-09-21T07:46:25.600433+0000","last_became_peered":"2024-09-21T07:46:25.600433+0000","last_unstale":"2024-09-21T07:46:27.604608+0000","last_undegraded":"2024-09-21T07:46:27.604608+0000","last_fullsized":"2024-09-21T07:46:27.604608+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:51:31.164111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":223,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.606806+0000","last_change":"2024-09-21T07:35:27.877861+0000","last_active":"2024-09-21T07:46:26.606806+0000","last_peered":"2024-09-21T07:46:26.606806+0000","last_clean":"2024-09-21T07:46:26.606806+0000","last_became_active":"2024-09-21T07:35:27.877434+0000","last_became_peered":"2024-09-21T07:35:27.877434+0000","last_unstale":"2024-09-21T07:46:26.606806+0000","last_undegraded":"2024-09-21T07:46:26.606806+0000","last_fullsized":"2024-09-21T07:46:26.606806+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":230,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.598861+0000","last_change":"2024-09-21T07:35:28.862954+0000","last_active":"2024-09-21T07:46:27.598861+0000","last_peered":"2024-09-21T07:46:27.598861+0000","last_clean":"2024-09-21T07:46:27.598861+0000","last_became_active":"2024-09-21T07:35:26.870613+0000","last_became_peered":"2024-09-21T07:35:26.870613+0000","last_unstale":"2024-09-21T07:46:27.598861+0000","last_undegraded":"2024-09-21T07:46:27.598861+0000","last_fullsized":"2024-09-21T07:46:27.598861+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:06.901043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059842100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":282,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.583019+0000","last_change":"2024-09-21T07:34:25.028904+0000","last_active":"2024-09-21T07:46:26.583019+0000","last_peered":"2024-09-21T07:46:26.583019+0000","last_clean":"2024-09-21T07:46:26.583019+0000","last_became_active":"2024-09-21T07:34:23.260942+0000","last_became_peered":"2024-09-21T07:34:23.260942+0000","last_unstale":"2024-09-21T07:46:26.583019+0000","last_undegraded":"2024-09-21T07:46:26.583019+0000","last_fullsized":"2024-09-21T07:46:26.583019+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:22.709328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060553899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509604,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42224,"kb_used_data":2712,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329616,"statfs":{"total":96636764160,"available":96593526784,"internally_reserved":0,"allocated":2777088,"data_stored":1740743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509606,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41648,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330192,"statfs":{"total":96636764160,"available":96594116608,"internally_reserved":0,"allocated":2183168,"data_stored":1150423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509604,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41644,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330196,"statfs":{"total":96636764160,"available":96594120704,"internally_reserved":0,"allocated":2179072,"data_stored":1150353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509605,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41652,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330188,"statfs":{"total":96636764160,"available":96594112512,"internally_reserved":0,"allocated":2187264,"data_stored":1150465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509606,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41648,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330192,"statfs":{"total":96636764160,"available":96594116608,"internally_reserved":0,"allocated":2183168,"data_stored":1150423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509606,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42244,"kb_used_data":2732,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329596,"statfs":{"total":96636764160,"available":96593506304,"internally_reserved":0,"allocated":2797568,"data_stored":1743039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542309,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41628,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330212,"statfs":{"total":96636764160,"available":96594137088,"internally_reserved":0,"allocated":2166784,"data_stored":1142399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542308,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29144,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342696,"statfs":{"total":96636764160,"available":96606920704,"internally_reserved":0,"allocated":2166784,"data_stored":1142399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:46:31.102 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:46:31.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.358+0000 7efc3dc00640 1 -- 172.21.15.39:0/1436597207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc380cf450 msgr2=0x7efc380cf860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:31.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.358+0000 7efc3dc00640 1 --2- 172.21.15.39:0/1436597207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc380cf450 0x7efc380cf860 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7efc380cff20 tx=0x7efc2002a530 comp rx=0 tx=0).stop 2024-09-21T07:46:31.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.358+0000 7efc3dc00640 1 -- 172.21.15.39:0/1436597207 shutdown_connections 2024-09-21T07:46:31.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.358+0000 7efc3dc00640 1 --2- 172.21.15.39:0/1436597207 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc380d0630 0x7efc380d0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:31.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.358+0000 7efc3dc00640 1 --2- 172.21.15.39:0/1436597207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc380cf450 0x7efc380cf860 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:31.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.358+0000 7efc3dc00640 1 -- 172.21.15.39:0/1436597207 >> 172.21.15.39:0/1436597207 conn(0x7efc3806cfc0 msgr2=0x7efc3806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:31.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.358+0000 7efc3dc00640 1 -- 172.21.15.39:0/1436597207 shutdown_connections 2024-09-21T07:46:31.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.358+0000 7efc3dc00640 1 -- 172.21.15.39:0/1436597207 wait complete. 2024-09-21T07:46:31.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.361+0000 7efc3dc00640 1 Processor -- start 2024-09-21T07:46:31.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.361+0000 7efc3dc00640 1 -- start start 2024-09-21T07:46:31.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.362+0000 7efc3dc00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc38073dd0 0x7efc38156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:31.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.362+0000 7efc3dc00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc380cf450 0x7efc38155080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:31.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.362+0000 7efc3dc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc380d0630 0x7efc380aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:31.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.362+0000 7efc3dc00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efc380cd1f0 con 0x7efc380d0630 2024-09-21T07:46:31.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.362+0000 7efc3dc00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efc380cd070 con 0x7efc380cf450 2024-09-21T07:46:31.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.362+0000 7efc3dc00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7efc380cd370 con 0x7efc38073dd0 2024-09-21T07:46:31.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.362+0000 7efc37400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc380d0630 0x7efc380aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:31.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.363+0000 7efc37400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc380d0630 0x7efc380aeb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35612/0 (socket says 172.21.15.39:35612) 2024-09-21T07:46:31.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.363+0000 7efc37400640 1 -- 172.21.15.39:0/2513592232 learned_addr learned my addr 172.21.15.39:0/2513592232 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:31.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.363+0000 7efc2e000640 1 --2- 172.21.15.39:0/2513592232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc380cf450 0x7efc38155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:31.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.363+0000 7efc36a00640 1 --2- 172.21.15.39:0/2513592232 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc38073dd0 0x7efc38156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:31.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.363+0000 7efc37400640 1 -- 172.21.15.39:0/2513592232 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc38073dd0 msgr2=0x7efc38156850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:31.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.363+0000 7efc37400640 1 --2- 172.21.15.39:0/2513592232 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc38073dd0 0x7efc38156850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:31.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.363+0000 7efc37400640 1 -- 172.21.15.39:0/2513592232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc380cf450 msgr2=0x7efc38155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:31.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.363+0000 7efc37400640 1 --2- 172.21.15.39:0/2513592232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc380cf450 0x7efc38155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:31.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.363+0000 7efc37400640 1 -- 172.21.15.39:0/2513592232 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc381557b0 con 0x7efc380d0630 2024-09-21T07:46:31.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.363+0000 7efc37400640 1 --2- 172.21.15.39:0/2513592232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc380d0630 0x7efc380aeb50 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7efc3000e990 tx=0x7efc3000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:31.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.364+0000 7efc2f400640 1 -- 172.21.15.39:0/2513592232 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc30004d20 con 0x7efc380d0630 2024-09-21T07:46:31.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.364+0000 7efc2f400640 1 -- 172.21.15.39:0/2513592232 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc3000caa0 con 0x7efc380d0630 2024-09-21T07:46:31.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.364+0000 7efc2f400640 1 -- 172.21.15.39:0/2513592232 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc30005020 con 0x7efc380d0630 2024-09-21T07:46:31.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.364+0000 7efc3dc00640 1 -- 172.21.15.39:0/2513592232 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc38155a90 con 0x7efc380d0630 2024-09-21T07:46:31.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.364+0000 7efc3dc00640 1 -- 172.21.15.39:0/2513592232 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc38156050 con 0x7efc380d0630 2024-09-21T07:46:31.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.365+0000 7efc3dc00640 1 -- 172.21.15.39:0/2513592232 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc3806f4f0 con 0x7efc380d0630 2024-09-21T07:46:31.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.366+0000 7efc2f400640 1 -- 172.21.15.39:0/2513592232 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7efc300051c0 con 0x7efc380d0630 2024-09-21T07:46:31.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.367+0000 7efc2f400640 1 --2- 172.21.15.39:0/2513592232 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efc14078bc0 0x7efc1407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:31.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.367+0000 7efc2f400640 1 -- 172.21.15.39:0/2513592232 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 6888+0+0 (secure 0 0 0) 0x7efc3009bb60 con 0x7efc380d0630 2024-09-21T07:46:31.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.368+0000 7efc36a00640 1 --2- 172.21.15.39:0/2513592232 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efc14078bc0 0x7efc1407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:31.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.368+0000 7efc36a00640 1 --2- 172.21.15.39:0/2513592232 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efc14078bc0 0x7efc1407b0d0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7efc28007fd0 tx=0x7efc28009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:31.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.371+0000 7efc2f400640 1 -- 172.21.15.39:0/2513592232 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7efc3806f4f0 con 0x7efc380d0630 2024-09-21T07:46:31.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.565+0000 7efc3dc00640 1 -- 172.21.15.39:0/2513592232 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efc380cd820 con 0x7efc14078bc0 2024-09-21T07:46:31.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.574+0000 7efc2f400640 1 -- 172.21.15.39:0/2513592232 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+495246 (secure 0 0 0) 0x7efc380cd820 con 0x7efc14078bc0 2024-09-21T07:46:31.581 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:46:31.585 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:46:31.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.580+0000 7efc3dc00640 1 -- 172.21.15.39:0/2513592232 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efc14078bc0 msgr2=0x7efc1407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:31.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.580+0000 7efc3dc00640 1 --2- 172.21.15.39:0/2513592232 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efc14078bc0 0x7efc1407b0d0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7efc28007fd0 tx=0x7efc28009040 comp rx=0 tx=0).stop 2024-09-21T07:46:31.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.580+0000 7efc3dc00640 1 -- 172.21.15.39:0/2513592232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc380d0630 msgr2=0x7efc380aeb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:31.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.580+0000 7efc3dc00640 1 --2- 172.21.15.39:0/2513592232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc380d0630 0x7efc380aeb50 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7efc3000e990 tx=0x7efc3000ee50 comp rx=0 tx=0).stop 2024-09-21T07:46:31.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.580+0000 7efc3dc00640 1 -- 172.21.15.39:0/2513592232 shutdown_connections 2024-09-21T07:46:31.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.580+0000 7efc3dc00640 1 --2- 172.21.15.39:0/2513592232 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efc14078bc0 0x7efc1407b0d0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:31.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.580+0000 7efc3dc00640 1 --2- 172.21.15.39:0/2513592232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc380d0630 0x7efc380aeb50 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:31.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.580+0000 7efc3dc00640 1 --2- 172.21.15.39:0/2513592232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efc380cf450 0x7efc38155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:31.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.580+0000 7efc3dc00640 1 --2- 172.21.15.39:0/2513592232 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efc38073dd0 0x7efc38156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:31.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.580+0000 7efc3dc00640 1 -- 172.21.15.39:0/2513592232 >> 172.21.15.39:0/2513592232 conn(0x7efc3806cfc0 msgr2=0x7efc3806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:31.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.580+0000 7efc3dc00640 1 -- 172.21.15.39:0/2513592232 shutdown_connections 2024-09-21T07:46:31.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.580+0000 7efc3dc00640 1 -- 172.21.15.39:0/2513592232 wait complete. 2024-09-21T07:46:31.605 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":574,"stamp":"2024-09-21T07:46:29.803154+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":252,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":321832,"kb_used_data":18204,"kb_used_omap":12,"kb_used_meta":303411,"kb_avail":754652888,"statfs":{"total":773094113280,"available":772764557312,"internally_reserved":0,"allocated":18640896,"data_stored":10370244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":310693424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.231649"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.600815+0000","last_change":"2024-09-21T07:46:26.603718+0000","last_active":"2024-09-21T07:46:27.600815+0000","last_peered":"2024-09-21T07:46:27.600815+0000","last_clean":"2024-09-21T07:46:27.600815+0000","last_became_active":"2024-09-21T07:46:26.602807+0000","last_became_peered":"2024-09-21T07:46:26.602807+0000","last_unstale":"2024-09-21T07:46:27.600815+0000","last_undegraded":"2024-09-21T07:46:27.600815+0000","last_fullsized":"2024-09-21T07:46:27.600815+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:44:29.488485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.623985+0000","last_change":"2024-09-21T07:46:26.620149+0000","last_active":"2024-09-21T07:46:26.623985+0000","last_peered":"2024-09-21T07:46:26.623985+0000","last_clean":"2024-09-21T07:46:26.623985+0000","last_became_active":"2024-09-21T07:46:26.615071+0000","last_became_peered":"2024-09-21T07:46:26.615071+0000","last_unstale":"2024-09-21T07:46:26.623985+0000","last_undegraded":"2024-09-21T07:46:26.623985+0000","last_fullsized":"2024-09-21T07:46:26.623985+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:28.468270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.623621+0000","last_change":"2024-09-21T07:46:26.615131+0000","last_active":"2024-09-21T07:46:26.623621+0000","last_peered":"2024-09-21T07:46:26.623621+0000","last_clean":"2024-09-21T07:46:26.623621+0000","last_became_active":"2024-09-21T07:46:26.614788+0000","last_became_peered":"2024-09-21T07:46:26.614788+0000","last_unstale":"2024-09-21T07:46:26.623621+0000","last_undegraded":"2024-09-21T07:46:26.623621+0000","last_fullsized":"2024-09-21T07:46:26.623621+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:23.450255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.610407+0000","last_change":"2024-09-21T07:46:26.601917+0000","last_active":"2024-09-21T07:46:26.610407+0000","last_peered":"2024-09-21T07:46:26.610407+0000","last_clean":"2024-09-21T07:46:26.610407+0000","last_became_active":"2024-09-21T07:46:26.601581+0000","last_became_peered":"2024-09-21T07:46:26.601581+0000","last_unstale":"2024-09-21T07:46:26.610407+0000","last_undegraded":"2024-09-21T07:46:26.610407+0000","last_fullsized":"2024-09-21T07:46:26.610407+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:15.141341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.610303+0000","last_change":"2024-09-21T07:46:26.596795+0000","last_active":"2024-09-21T07:46:26.610303+0000","last_peered":"2024-09-21T07:46:26.610303+0000","last_clean":"2024-09-21T07:46:26.610303+0000","last_became_active":"2024-09-21T07:46:26.596512+0000","last_became_peered":"2024-09-21T07:46:26.596512+0000","last_unstale":"2024-09-21T07:46:26.610303+0000","last_undegraded":"2024-09-21T07:46:26.610303+0000","last_fullsized":"2024-09-21T07:46:26.610303+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:23:34.648456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.608364+0000","last_change":"2024-09-21T07:46:26.599492+0000","last_active":"2024-09-21T07:46:26.608364+0000","last_peered":"2024-09-21T07:46:26.608364+0000","last_clean":"2024-09-21T07:46:26.608364+0000","last_became_active":"2024-09-21T07:46:26.597014+0000","last_became_peered":"2024-09-21T07:46:26.597014+0000","last_unstale":"2024-09-21T07:46:26.608364+0000","last_undegraded":"2024-09-21T07:46:26.608364+0000","last_fullsized":"2024-09-21T07:46:26.608364+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:41:30.878490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599769+0000","last_change":"2024-09-21T07:46:26.604204+0000","last_active":"2024-09-21T07:46:27.599769+0000","last_peered":"2024-09-21T07:46:27.599769+0000","last_clean":"2024-09-21T07:46:27.599769+0000","last_became_active":"2024-09-21T07:46:26.603837+0000","last_became_peered":"2024-09-21T07:46:26.603837+0000","last_unstale":"2024-09-21T07:46:27.599769+0000","last_undegraded":"2024-09-21T07:46:27.599769+0000","last_fullsized":"2024-09-21T07:46:27.599769+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:28.695927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.626154+0000","last_change":"2024-09-21T07:46:26.617954+0000","last_active":"2024-09-21T07:46:26.626154+0000","last_peered":"2024-09-21T07:46:26.626154+0000","last_clean":"2024-09-21T07:46:26.626154+0000","last_became_active":"2024-09-21T07:46:26.612003+0000","last_became_peered":"2024-09-21T07:46:26.612003+0000","last_unstale":"2024-09-21T07:46:26.626154+0000","last_undegraded":"2024-09-21T07:46:26.626154+0000","last_fullsized":"2024-09-21T07:46:26.626154+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:15.980531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.613636+0000","last_change":"2024-09-21T07:46:26.610922+0000","last_active":"2024-09-21T07:46:26.613636+0000","last_peered":"2024-09-21T07:46:26.613636+0000","last_clean":"2024-09-21T07:46:26.613636+0000","last_became_active":"2024-09-21T07:46:26.610591+0000","last_became_peered":"2024-09-21T07:46:26.610591+0000","last_unstale":"2024-09-21T07:46:26.613636+0000","last_undegraded":"2024-09-21T07:46:26.613636+0000","last_fullsized":"2024-09-21T07:46:26.613636+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:33.939458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.607039+0000","last_change":"2024-09-21T07:46:26.592170+0000","last_active":"2024-09-21T07:46:26.607039+0000","last_peered":"2024-09-21T07:46:26.607039+0000","last_clean":"2024-09-21T07:46:26.607039+0000","last_became_active":"2024-09-21T07:46:26.591847+0000","last_became_peered":"2024-09-21T07:46:26.591847+0000","last_unstale":"2024-09-21T07:46:26.607039+0000","last_undegraded":"2024-09-21T07:46:26.607039+0000","last_fullsized":"2024-09-21T07:46:26.607039+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:41.885676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.601124+0000","last_change":"2024-09-21T07:46:26.604500+0000","last_active":"2024-09-21T07:46:27.601124+0000","last_peered":"2024-09-21T07:46:27.601124+0000","last_clean":"2024-09-21T07:46:27.601124+0000","last_became_active":"2024-09-21T07:46:26.603764+0000","last_became_peered":"2024-09-21T07:46:26.603764+0000","last_unstale":"2024-09-21T07:46:27.601124+0000","last_undegraded":"2024-09-21T07:46:27.601124+0000","last_fullsized":"2024-09-21T07:46:27.601124+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:08.389634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.618690+0000","last_change":"2024-09-21T07:46:26.614039+0000","last_active":"2024-09-21T07:46:26.618690+0000","last_peered":"2024-09-21T07:46:26.618690+0000","last_clean":"2024-09-21T07:46:26.618690+0000","last_became_active":"2024-09-21T07:46:26.613725+0000","last_became_peered":"2024-09-21T07:46:26.613725+0000","last_unstale":"2024-09-21T07:46:26.618690+0000","last_undegraded":"2024-09-21T07:46:26.618690+0000","last_fullsized":"2024-09-21T07:46:26.618690+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:23:35.677383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.610275+0000","last_change":"2024-09-21T07:46:26.596094+0000","last_active":"2024-09-21T07:46:26.610275+0000","last_peered":"2024-09-21T07:46:26.610275+0000","last_clean":"2024-09-21T07:46:26.610275+0000","last_became_active":"2024-09-21T07:46:26.594210+0000","last_became_peered":"2024-09-21T07:46:26.594210+0000","last_unstale":"2024-09-21T07:46:26.610275+0000","last_undegraded":"2024-09-21T07:46:26.610275+0000","last_fullsized":"2024-09-21T07:46:26.610275+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:57.950433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.617456+0000","last_change":"2024-09-21T07:46:26.612264+0000","last_active":"2024-09-21T07:46:26.617456+0000","last_peered":"2024-09-21T07:46:26.617456+0000","last_clean":"2024-09-21T07:46:26.617456+0000","last_became_active":"2024-09-21T07:46:26.611939+0000","last_became_peered":"2024-09-21T07:46:26.611939+0000","last_unstale":"2024-09-21T07:46:26.617456+0000","last_undegraded":"2024-09-21T07:46:26.617456+0000","last_fullsized":"2024-09-21T07:46:26.617456+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:57:46.032193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.624257+0000","last_change":"2024-09-21T07:46:26.615565+0000","last_active":"2024-09-21T07:46:26.624257+0000","last_peered":"2024-09-21T07:46:26.624257+0000","last_clean":"2024-09-21T07:46:26.624257+0000","last_became_active":"2024-09-21T07:46:26.615236+0000","last_became_peered":"2024-09-21T07:46:26.615236+0000","last_unstale":"2024-09-21T07:46:26.624257+0000","last_undegraded":"2024-09-21T07:46:26.624257+0000","last_fullsized":"2024-09-21T07:46:26.624257+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:42:11.471492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.613544+0000","last_change":"2024-09-21T07:46:26.610453+0000","last_active":"2024-09-21T07:46:26.613544+0000","last_peered":"2024-09-21T07:46:26.613544+0000","last_clean":"2024-09-21T07:46:26.613544+0000","last_became_active":"2024-09-21T07:46:26.605865+0000","last_became_peered":"2024-09-21T07:46:26.605865+0000","last_unstale":"2024-09-21T07:46:26.613544+0000","last_undegraded":"2024-09-21T07:46:26.613544+0000","last_fullsized":"2024-09-21T07:46:26.613544+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:29.896851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.618999+0000","last_change":"2024-09-21T07:46:26.610826+0000","last_active":"2024-09-21T07:46:26.618999+0000","last_peered":"2024-09-21T07:46:26.618999+0000","last_clean":"2024-09-21T07:46:26.618999+0000","last_became_active":"2024-09-21T07:46:26.610439+0000","last_became_peered":"2024-09-21T07:46:26.610439+0000","last_unstale":"2024-09-21T07:46:26.618999+0000","last_undegraded":"2024-09-21T07:46:26.618999+0000","last_fullsized":"2024-09-21T07:46:26.618999+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:43:13.569707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.601229+0000","last_change":"2024-09-21T07:46:26.595755+0000","last_active":"2024-09-21T07:46:27.601229+0000","last_peered":"2024-09-21T07:46:27.601229+0000","last_clean":"2024-09-21T07:46:27.601229+0000","last_became_active":"2024-09-21T07:46:26.595051+0000","last_became_peered":"2024-09-21T07:46:26.595051+0000","last_unstale":"2024-09-21T07:46:27.601229+0000","last_undegraded":"2024-09-21T07:46:27.601229+0000","last_fullsized":"2024-09-21T07:46:27.601229+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:54.430995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.604603+0000","last_change":"2024-09-21T07:46:26.625028+0000","last_active":"2024-09-21T07:46:27.604603+0000","last_peered":"2024-09-21T07:46:27.604603+0000","last_clean":"2024-09-21T07:46:27.604603+0000","last_became_active":"2024-09-21T07:46:26.624829+0000","last_became_peered":"2024-09-21T07:46:26.624829+0000","last_unstale":"2024-09-21T07:46:27.604603+0000","last_undegraded":"2024-09-21T07:46:27.604603+0000","last_fullsized":"2024-09-21T07:46:27.604603+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:44.374583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599614+0000","last_change":"2024-09-21T07:46:26.593853+0000","last_active":"2024-09-21T07:46:27.599614+0000","last_peered":"2024-09-21T07:46:27.599614+0000","last_clean":"2024-09-21T07:46:27.599614+0000","last_became_active":"2024-09-21T07:46:26.593566+0000","last_became_peered":"2024-09-21T07:46:26.593566+0000","last_unstale":"2024-09-21T07:46:27.599614+0000","last_undegraded":"2024-09-21T07:46:27.599614+0000","last_fullsized":"2024-09-21T07:46:27.599614+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:20.557353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.613098+0000","last_change":"2024-09-21T07:46:26.609318+0000","last_active":"2024-09-21T07:46:26.613098+0000","last_peered":"2024-09-21T07:46:26.613098+0000","last_clean":"2024-09-21T07:46:26.613098+0000","last_became_active":"2024-09-21T07:46:26.603129+0000","last_became_peered":"2024-09-21T07:46:26.603129+0000","last_unstale":"2024-09-21T07:46:26.613098+0000","last_undegraded":"2024-09-21T07:46:26.613098+0000","last_fullsized":"2024-09-21T07:46:26.613098+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:11:02.101471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599926+0000","last_change":"2024-09-21T07:46:26.603485+0000","last_active":"2024-09-21T07:46:27.599926+0000","last_peered":"2024-09-21T07:46:27.599926+0000","last_clean":"2024-09-21T07:46:27.599926+0000","last_became_active":"2024-09-21T07:46:26.602566+0000","last_became_peered":"2024-09-21T07:46:26.602566+0000","last_unstale":"2024-09-21T07:46:27.599926+0000","last_undegraded":"2024-09-21T07:46:27.599926+0000","last_fullsized":"2024-09-21T07:46:27.599926+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:51:32.172552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.618630+0000","last_change":"2024-09-21T07:46:26.608329+0000","last_active":"2024-09-21T07:46:26.618630+0000","last_peered":"2024-09-21T07:46:26.618630+0000","last_clean":"2024-09-21T07:46:26.618630+0000","last_became_active":"2024-09-21T07:46:26.606179+0000","last_became_peered":"2024-09-21T07:46:26.606179+0000","last_unstale":"2024-09-21T07:46:26.618630+0000","last_undegraded":"2024-09-21T07:46:26.618630+0000","last_fullsized":"2024-09-21T07:46:26.618630+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:13.966123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599917+0000","last_change":"2024-09-21T07:46:26.612396+0000","last_active":"2024-09-21T07:46:27.599917+0000","last_peered":"2024-09-21T07:46:27.599917+0000","last_clean":"2024-09-21T07:46:27.599917+0000","last_became_active":"2024-09-21T07:46:26.611432+0000","last_became_peered":"2024-09-21T07:46:26.611432+0000","last_unstale":"2024-09-21T07:46:27.599917+0000","last_undegraded":"2024-09-21T07:46:27.599917+0000","last_fullsized":"2024-09-21T07:46:27.599917+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:47.852203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.600950+0000","last_change":"2024-09-21T07:46:26.594621+0000","last_active":"2024-09-21T07:46:27.600950+0000","last_peered":"2024-09-21T07:46:27.600950+0000","last_clean":"2024-09-21T07:46:27.600950+0000","last_became_active":"2024-09-21T07:46:26.594214+0000","last_became_peered":"2024-09-21T07:46:26.594214+0000","last_unstale":"2024-09-21T07:46:27.600950+0000","last_undegraded":"2024-09-21T07:46:27.600950+0000","last_fullsized":"2024-09-21T07:46:27.600950+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:34.463326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.600964+0000","last_change":"2024-09-21T07:46:26.601442+0000","last_active":"2024-09-21T07:46:27.600964+0000","last_peered":"2024-09-21T07:46:27.600964+0000","last_clean":"2024-09-21T07:46:27.600964+0000","last_became_active":"2024-09-21T07:46:26.601045+0000","last_became_peered":"2024-09-21T07:46:26.601045+0000","last_unstale":"2024-09-21T07:46:27.600964+0000","last_undegraded":"2024-09-21T07:46:27.600964+0000","last_fullsized":"2024-09-21T07:46:27.600964+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:41:14.294772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.612569+0000","last_change":"2024-09-21T07:46:26.608915+0000","last_active":"2024-09-21T07:46:26.612569+0000","last_peered":"2024-09-21T07:46:26.612569+0000","last_clean":"2024-09-21T07:46:26.612569+0000","last_became_active":"2024-09-21T07:46:26.608630+0000","last_became_peered":"2024-09-21T07:46:26.608630+0000","last_unstale":"2024-09-21T07:46:26.612569+0000","last_undegraded":"2024-09-21T07:46:26.612569+0000","last_fullsized":"2024-09-21T07:46:26.612569+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:15.406096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.600406+0000","last_change":"2024-09-21T07:46:26.597210+0000","last_active":"2024-09-21T07:46:27.600406+0000","last_peered":"2024-09-21T07:46:27.600406+0000","last_clean":"2024-09-21T07:46:27.600406+0000","last_became_active":"2024-09-21T07:46:26.596513+0000","last_became_peered":"2024-09-21T07:46:26.596513+0000","last_unstale":"2024-09-21T07:46:27.600406+0000","last_undegraded":"2024-09-21T07:46:27.600406+0000","last_fullsized":"2024-09-21T07:46:27.600406+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:20.818984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.626081+0000","last_change":"2024-09-21T07:46:26.621506+0000","last_active":"2024-09-21T07:46:26.626081+0000","last_peered":"2024-09-21T07:46:26.626081+0000","last_clean":"2024-09-21T07:46:26.626081+0000","last_became_active":"2024-09-21T07:46:26.618147+0000","last_became_peered":"2024-09-21T07:46:26.618147+0000","last_unstale":"2024-09-21T07:46:26.626081+0000","last_undegraded":"2024-09-21T07:46:26.626081+0000","last_fullsized":"2024-09-21T07:46:26.626081+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:29:28.176807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.600588+0000","last_change":"2024-09-21T07:46:26.608633+0000","last_active":"2024-09-21T07:46:27.600588+0000","last_peered":"2024-09-21T07:46:27.600588+0000","last_clean":"2024-09-21T07:46:27.600588+0000","last_became_active":"2024-09-21T07:46:26.606778+0000","last_became_peered":"2024-09-21T07:46:26.606778+0000","last_unstale":"2024-09-21T07:46:27.600588+0000","last_undegraded":"2024-09-21T07:46:27.600588+0000","last_fullsized":"2024-09-21T07:46:27.600588+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:57.377169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.611874+0000","last_change":"2024-09-21T07:46:26.607976+0000","last_active":"2024-09-21T07:46:26.611874+0000","last_peered":"2024-09-21T07:46:26.611874+0000","last_clean":"2024-09-21T07:46:26.611874+0000","last_became_active":"2024-09-21T07:46:26.607452+0000","last_became_peered":"2024-09-21T07:46:26.607452+0000","last_unstale":"2024-09-21T07:46:26.611874+0000","last_undegraded":"2024-09-21T07:46:26.611874+0000","last_fullsized":"2024-09-21T07:46:26.611874+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:51.495597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.622036+0000","last_change":"2024-09-21T07:46:26.611007+0000","last_active":"2024-09-21T07:46:26.622036+0000","last_peered":"2024-09-21T07:46:26.622036+0000","last_clean":"2024-09-21T07:46:26.622036+0000","last_became_active":"2024-09-21T07:46:26.610179+0000","last_became_peered":"2024-09-21T07:46:26.610179+0000","last_unstale":"2024-09-21T07:46:26.622036+0000","last_undegraded":"2024-09-21T07:46:26.622036+0000","last_fullsized":"2024-09-21T07:46:26.622036+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:54:05.047873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.628897+0000","last_change":"2024-09-21T07:46:26.622360+0000","last_active":"2024-09-21T07:46:26.628897+0000","last_peered":"2024-09-21T07:46:26.628897+0000","last_clean":"2024-09-21T07:46:26.628897+0000","last_became_active":"2024-09-21T07:46:26.619462+0000","last_became_peered":"2024-09-21T07:46:26.619462+0000","last_unstale":"2024-09-21T07:46:26.628897+0000","last_undegraded":"2024-09-21T07:46:26.628897+0000","last_fullsized":"2024-09-21T07:46:26.628897+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:23:15.779122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"174'6","reported_seq":21,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599594+0000","last_change":"2024-09-21T07:46:25.596989+0000","last_active":"2024-09-21T07:46:27.599594+0000","last_peered":"2024-09-21T07:46:27.599594+0000","last_clean":"2024-09-21T07:46:27.599594+0000","last_became_active":"2024-09-21T07:46:25.596030+0000","last_became_peered":"2024-09-21T07:46:25.596030+0000","last_unstale":"2024-09-21T07:46:27.599594+0000","last_undegraded":"2024-09-21T07:46:27.599594+0000","last_fullsized":"2024-09-21T07:46:27.599594+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:08:26.795197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":224,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.604830+0000","last_change":"2024-09-21T07:35:27.879605+0000","last_active":"2024-09-21T07:46:27.604830+0000","last_peered":"2024-09-21T07:46:27.604830+0000","last_clean":"2024-09-21T07:46:27.604830+0000","last_became_active":"2024-09-21T07:35:27.879274+0000","last_became_peered":"2024-09-21T07:35:27.879274+0000","last_unstale":"2024-09-21T07:46:27.604830+0000","last_undegraded":"2024-09-21T07:46:27.604830+0000","last_fullsized":"2024-09-21T07:46:27.604830+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:24.241636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":230,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.588630+0000","last_change":"2024-09-21T07:35:29.491336+0000","last_active":"2024-09-21T07:46:26.588630+0000","last_peered":"2024-09-21T07:46:26.588630+0000","last_clean":"2024-09-21T07:46:26.588630+0000","last_became_active":"2024-09-21T07:35:26.866484+0000","last_became_peered":"2024-09-21T07:35:26.866484+0000","last_unstale":"2024-09-21T07:46:26.588630+0000","last_undegraded":"2024-09-21T07:46:26.588630+0000","last_fullsized":"2024-09-21T07:46:26.588630+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:10.449665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076010100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":410,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.603920+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:46:27.603920+0000","last_peered":"2024-09-21T07:46:27.603920+0000","last_clean":"2024-09-21T07:46:27.603920+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:46:27.603920+0000","last_undegraded":"2024-09-21T07:46:27.603920+0000","last_fullsized":"2024-09-21T07:46:27.603920+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":291,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.601133+0000","last_change":"2024-09-21T07:34:24.539248+0000","last_active":"2024-09-21T07:46:27.601133+0000","last_peered":"2024-09-21T07:46:27.601133+0000","last_clean":"2024-09-21T07:46:27.601133+0000","last_became_active":"2024-09-21T07:34:23.273465+0000","last_became_peered":"2024-09-21T07:34:23.273465+0000","last_unstale":"2024-09-21T07:46:27.601133+0000","last_undegraded":"2024-09-21T07:46:27.601133+0000","last_fullsized":"2024-09-21T07:46:27.601133+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:55.517452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060627700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.611787+0000","last_change":"2024-09-21T07:46:26.605575+0000","last_active":"2024-09-21T07:46:26.611787+0000","last_peered":"2024-09-21T07:46:26.611787+0000","last_clean":"2024-09-21T07:46:26.611787+0000","last_became_active":"2024-09-21T07:46:26.597517+0000","last_became_peered":"2024-09-21T07:46:26.597517+0000","last_unstale":"2024-09-21T07:46:26.611787+0000","last_undegraded":"2024-09-21T07:46:26.611787+0000","last_fullsized":"2024-09-21T07:46:26.611787+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:50.946973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":13,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.605339+0000","last_change":"2024-09-21T07:46:25.600542+0000","last_active":"2024-09-21T07:46:26.605339+0000","last_peered":"2024-09-21T07:46:26.605339+0000","last_clean":"2024-09-21T07:46:26.605339+0000","last_became_active":"2024-09-21T07:46:25.600103+0000","last_became_peered":"2024-09-21T07:46:25.600103+0000","last_unstale":"2024-09-21T07:46:26.605339+0000","last_undegraded":"2024-09-21T07:46:26.605339+0000","last_fullsized":"2024-09-21T07:46:26.605339+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:24.714172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":223,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.587593+0000","last_change":"2024-09-21T07:35:27.864492+0000","last_active":"2024-09-21T07:46:26.587593+0000","last_peered":"2024-09-21T07:46:26.587593+0000","last_clean":"2024-09-21T07:46:26.587593+0000","last_became_active":"2024-09-21T07:35:27.864176+0000","last_became_peered":"2024-09-21T07:35:27.864176+0000","last_unstale":"2024-09-21T07:46:26.587593+0000","last_undegraded":"2024-09-21T07:46:26.587593+0000","last_fullsized":"2024-09-21T07:46:26.587593+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:05.153467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":231,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.605060+0000","last_change":"2024-09-21T07:35:29.493678+0000","last_active":"2024-09-21T07:46:27.605060+0000","last_peered":"2024-09-21T07:46:27.605060+0000","last_clean":"2024-09-21T07:46:27.605060+0000","last_became_active":"2024-09-21T07:35:26.870749+0000","last_became_peered":"2024-09-21T07:35:26.870749+0000","last_unstale":"2024-09-21T07:46:27.605060+0000","last_undegraded":"2024-09-21T07:46:27.605060+0000","last_fullsized":"2024-09-21T07:46:27.605060+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:55.367877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010159570000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":276,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.607914+0000","last_change":"2024-09-21T07:34:24.597274+0000","last_active":"2024-09-21T07:46:26.607914+0000","last_peered":"2024-09-21T07:46:26.607914+0000","last_clean":"2024-09-21T07:46:26.607914+0000","last_became_active":"2024-09-21T07:34:23.279560+0000","last_became_peered":"2024-09-21T07:34:23.279560+0000","last_unstale":"2024-09-21T07:46:26.607914+0000","last_undegraded":"2024-09-21T07:46:26.607914+0000","last_fullsized":"2024-09-21T07:46:26.607914+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:15.845265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059699799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.615217+0000","last_change":"2024-09-21T07:46:26.600249+0000","last_active":"2024-09-21T07:46:26.615217+0000","last_peered":"2024-09-21T07:46:26.615217+0000","last_clean":"2024-09-21T07:46:26.615217+0000","last_became_active":"2024-09-21T07:46:26.599921+0000","last_became_peered":"2024-09-21T07:46:26.599921+0000","last_unstale":"2024-09-21T07:46:26.615217+0000","last_undegraded":"2024-09-21T07:46:26.615217+0000","last_fullsized":"2024-09-21T07:46:26.615217+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:20.298916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"174'8","reported_seq":23,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599410+0000","last_change":"2024-09-21T07:46:25.597983+0000","last_active":"2024-09-21T07:46:27.599410+0000","last_peered":"2024-09-21T07:46:27.599410+0000","last_clean":"2024-09-21T07:46:27.599410+0000","last_became_active":"2024-09-21T07:46:25.597074+0000","last_became_peered":"2024-09-21T07:46:25.597074+0000","last_unstale":"2024-09-21T07:46:27.599410+0000","last_undegraded":"2024-09-21T07:46:27.599410+0000","last_fullsized":"2024-09-21T07:46:27.599410+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:15.858457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":225,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.600271+0000","last_change":"2024-09-21T07:35:27.874387+0000","last_active":"2024-09-21T07:46:27.600271+0000","last_peered":"2024-09-21T07:46:27.600271+0000","last_clean":"2024-09-21T07:46:27.600271+0000","last_became_active":"2024-09-21T07:35:27.873277+0000","last_became_peered":"2024-09-21T07:35:27.873277+0000","last_unstale":"2024-09-21T07:46:27.600271+0000","last_undegraded":"2024-09-21T07:46:27.600271+0000","last_fullsized":"2024-09-21T07:46:27.600271+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:04.581463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":226,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.615398+0000","last_change":"2024-09-21T07:35:29.387094+0000","last_active":"2024-09-21T07:46:26.615398+0000","last_peered":"2024-09-21T07:46:26.615398+0000","last_clean":"2024-09-21T07:46:26.615398+0000","last_became_active":"2024-09-21T07:35:26.874757+0000","last_became_peered":"2024-09-21T07:35:26.874757+0000","last_unstale":"2024-09-21T07:46:26.615398+0000","last_undegraded":"2024-09-21T07:46:26.615398+0000","last_fullsized":"2024-09-21T07:46:26.615398+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:33.542750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061065700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":285,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.604258+0000","last_change":"2024-09-21T07:34:24.259906+0000","last_active":"2024-09-21T07:46:27.604258+0000","last_peered":"2024-09-21T07:46:27.604258+0000","last_clean":"2024-09-21T07:46:27.604258+0000","last_became_active":"2024-09-21T07:34:23.635282+0000","last_became_peered":"2024-09-21T07:34:23.635282+0000","last_unstale":"2024-09-21T07:46:27.604258+0000","last_undegraded":"2024-09-21T07:46:27.604258+0000","last_fullsized":"2024-09-21T07:46:27.604258+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:00.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001527664,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.611241+0000","last_change":"2024-09-21T07:46:26.603275+0000","last_active":"2024-09-21T07:46:26.611241+0000","last_peered":"2024-09-21T07:46:26.611241+0000","last_clean":"2024-09-21T07:46:26.611241+0000","last_became_active":"2024-09-21T07:46:26.602801+0000","last_became_peered":"2024-09-21T07:46:26.602801+0000","last_unstale":"2024-09-21T07:46:26.611241+0000","last_undegraded":"2024-09-21T07:46:26.611241+0000","last_fullsized":"2024-09-21T07:46:26.611241+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:37:59.308165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"174'2","reported_seq":15,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.626821+0000","last_change":"2024-09-21T07:46:25.588970+0000","last_active":"2024-09-21T07:46:26.626821+0000","last_peered":"2024-09-21T07:46:26.626821+0000","last_clean":"2024-09-21T07:46:26.626821+0000","last_became_active":"2024-09-21T07:46:25.588571+0000","last_became_peered":"2024-09-21T07:46:25.588571+0000","last_unstale":"2024-09-21T07:46:26.626821+0000","last_undegraded":"2024-09-21T07:46:26.626821+0000","last_fullsized":"2024-09-21T07:46:26.626821+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:26:02.967925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":225,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.600772+0000","last_change":"2024-09-21T07:35:27.873645+0000","last_active":"2024-09-21T07:46:27.600772+0000","last_peered":"2024-09-21T07:46:27.600772+0000","last_clean":"2024-09-21T07:46:27.600772+0000","last_became_active":"2024-09-21T07:35:27.873245+0000","last_became_peered":"2024-09-21T07:35:27.873245+0000","last_unstale":"2024-09-21T07:46:27.600772+0000","last_undegraded":"2024-09-21T07:46:27.600772+0000","last_fullsized":"2024-09-21T07:46:27.600772+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:55.090953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":231,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.601211+0000","last_change":"2024-09-21T07:35:28.861744+0000","last_active":"2024-09-21T07:46:27.601211+0000","last_peered":"2024-09-21T07:46:27.601211+0000","last_clean":"2024-09-21T07:46:27.601211+0000","last_became_active":"2024-09-21T07:35:26.876462+0000","last_became_peered":"2024-09-21T07:35:26.876462+0000","last_unstale":"2024-09-21T07:46:27.601211+0000","last_undegraded":"2024-09-21T07:46:27.601211+0000","last_fullsized":"2024-09-21T07:46:27.601211+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:48.632568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012020920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":282,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.598475+0000","last_change":"2024-09-21T07:34:25.085454+0000","last_active":"2024-09-21T07:46:27.598475+0000","last_peered":"2024-09-21T07:46:27.598475+0000","last_clean":"2024-09-21T07:46:27.598475+0000","last_became_active":"2024-09-21T07:34:23.279205+0000","last_became_peered":"2024-09-21T07:34:23.279205+0000","last_unstale":"2024-09-21T07:46:27.598475+0000","last_undegraded":"2024-09-21T07:46:27.598475+0000","last_fullsized":"2024-09-21T07:46:27.598475+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:54.231666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095320599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.620627+0000","last_change":"2024-09-21T07:46:26.601788+0000","last_active":"2024-09-21T07:46:26.620627+0000","last_peered":"2024-09-21T07:46:26.620627+0000","last_clean":"2024-09-21T07:46:26.620627+0000","last_became_active":"2024-09-21T07:46:26.601484+0000","last_became_peered":"2024-09-21T07:46:26.601484+0000","last_unstale":"2024-09-21T07:46:26.620627+0000","last_undegraded":"2024-09-21T07:46:26.620627+0000","last_fullsized":"2024-09-21T07:46:26.620627+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:55:53.588662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":13,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.606233+0000","last_change":"2024-09-21T07:46:25.604131+0000","last_active":"2024-09-21T07:46:26.606233+0000","last_peered":"2024-09-21T07:46:26.606233+0000","last_clean":"2024-09-21T07:46:26.606233+0000","last_became_active":"2024-09-21T07:46:25.603782+0000","last_became_peered":"2024-09-21T07:46:25.603782+0000","last_unstale":"2024-09-21T07:46:26.606233+0000","last_undegraded":"2024-09-21T07:46:26.606233+0000","last_fullsized":"2024-09-21T07:46:26.606233+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:22:08.526888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":222,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.590401+0000","last_change":"2024-09-21T07:35:27.861373+0000","last_active":"2024-09-21T07:46:26.590401+0000","last_peered":"2024-09-21T07:46:26.590401+0000","last_clean":"2024-09-21T07:46:26.590401+0000","last_became_active":"2024-09-21T07:35:27.861016+0000","last_became_peered":"2024-09-21T07:35:27.861016+0000","last_unstale":"2024-09-21T07:46:26.590401+0000","last_undegraded":"2024-09-21T07:46:26.590401+0000","last_fullsized":"2024-09-21T07:46:26.590401+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:56.043763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":231,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.583172+0000","last_change":"2024-09-21T07:35:29.053677+0000","last_active":"2024-09-21T07:46:26.583172+0000","last_peered":"2024-09-21T07:46:26.583172+0000","last_clean":"2024-09-21T07:46:26.583172+0000","last_became_active":"2024-09-21T07:35:26.875496+0000","last_became_peered":"2024-09-21T07:35:26.875496+0000","last_unstale":"2024-09-21T07:46:26.583172+0000","last_undegraded":"2024-09-21T07:46:26.583172+0000","last_fullsized":"2024-09-21T07:46:26.583172+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:04.850784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065859000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":283,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.586083+0000","last_change":"2024-09-21T07:34:25.028521+0000","last_active":"2024-09-21T07:46:26.586083+0000","last_peered":"2024-09-21T07:46:26.586083+0000","last_clean":"2024-09-21T07:46:26.586083+0000","last_became_active":"2024-09-21T07:34:23.635467+0000","last_became_peered":"2024-09-21T07:34:23.635467+0000","last_unstale":"2024-09-21T07:46:26.586083+0000","last_undegraded":"2024-09-21T07:46:26.586083+0000","last_fullsized":"2024-09-21T07:46:26.586083+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:33.729123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000660558,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.601219+0000","last_change":"2024-09-21T07:46:26.596074+0000","last_active":"2024-09-21T07:46:27.601219+0000","last_peered":"2024-09-21T07:46:27.601219+0000","last_clean":"2024-09-21T07:46:27.601219+0000","last_became_active":"2024-09-21T07:46:26.595466+0000","last_became_peered":"2024-09-21T07:46:26.595466+0000","last_unstale":"2024-09-21T07:46:27.601219+0000","last_undegraded":"2024-09-21T07:46:27.601219+0000","last_fullsized":"2024-09-21T07:46:27.601219+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:51:32.349009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"174'3","reported_seq":18,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.600880+0000","last_change":"2024-09-21T07:46:25.594804+0000","last_active":"2024-09-21T07:46:27.600880+0000","last_peered":"2024-09-21T07:46:27.600880+0000","last_clean":"2024-09-21T07:46:27.600880+0000","last_became_active":"2024-09-21T07:46:25.593032+0000","last_became_peered":"2024-09-21T07:46:25.593032+0000","last_unstale":"2024-09-21T07:46:27.600880+0000","last_undegraded":"2024-09-21T07:46:27.600880+0000","last_fullsized":"2024-09-21T07:46:27.600880+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:49.893169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":223,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599681+0000","last_change":"2024-09-21T07:35:27.871201+0000","last_active":"2024-09-21T07:46:27.599681+0000","last_peered":"2024-09-21T07:46:27.599681+0000","last_clean":"2024-09-21T07:46:27.599681+0000","last_became_active":"2024-09-21T07:35:27.870816+0000","last_became_peered":"2024-09-21T07:35:27.870816+0000","last_unstale":"2024-09-21T07:46:27.599681+0000","last_undegraded":"2024-09-21T07:46:27.599681+0000","last_fullsized":"2024-09-21T07:46:27.599681+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:04.787450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":229,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.590646+0000","last_change":"2024-09-21T07:35:29.491874+0000","last_active":"2024-09-21T07:46:26.590646+0000","last_peered":"2024-09-21T07:46:26.590646+0000","last_clean":"2024-09-21T07:46:26.590646+0000","last_became_active":"2024-09-21T07:35:26.866381+0000","last_became_peered":"2024-09-21T07:35:26.866381+0000","last_unstale":"2024-09-21T07:46:26.590646+0000","last_undegraded":"2024-09-21T07:46:26.590646+0000","last_fullsized":"2024-09-21T07:46:26.590646+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:36.404647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073235399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":284,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.589509+0000","last_change":"2024-09-21T07:34:24.260046+0000","last_active":"2024-09-21T07:46:26.589509+0000","last_peered":"2024-09-21T07:46:26.589509+0000","last_clean":"2024-09-21T07:46:26.589509+0000","last_became_active":"2024-09-21T07:34:23.273431+0000","last_became_peered":"2024-09-21T07:34:23.273431+0000","last_unstale":"2024-09-21T07:46:26.589509+0000","last_undegraded":"2024-09-21T07:46:26.589509+0000","last_fullsized":"2024-09-21T07:46:26.589509+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:18.876057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019803310000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.625832+0000","last_change":"2024-09-21T07:46:26.620066+0000","last_active":"2024-09-21T07:46:26.625832+0000","last_peered":"2024-09-21T07:46:26.625832+0000","last_clean":"2024-09-21T07:46:26.625832+0000","last_became_active":"2024-09-21T07:46:26.619764+0000","last_became_peered":"2024-09-21T07:46:26.619764+0000","last_unstale":"2024-09-21T07:46:26.625832+0000","last_undegraded":"2024-09-21T07:46:26.625832+0000","last_fullsized":"2024-09-21T07:46:26.625832+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:39:53.865815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":15,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.604962+0000","last_change":"2024-09-21T07:46:25.600862+0000","last_active":"2024-09-21T07:46:27.604962+0000","last_peered":"2024-09-21T07:46:27.604962+0000","last_clean":"2024-09-21T07:46:27.604962+0000","last_became_active":"2024-09-21T07:46:25.600256+0000","last_became_peered":"2024-09-21T07:46:25.600256+0000","last_unstale":"2024-09-21T07:46:27.604962+0000","last_undegraded":"2024-09-21T07:46:27.604962+0000","last_fullsized":"2024-09-21T07:46:27.604962+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:44.271091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":223,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.598597+0000","last_change":"2024-09-21T07:35:27.871736+0000","last_active":"2024-09-21T07:46:27.598597+0000","last_peered":"2024-09-21T07:46:27.598597+0000","last_clean":"2024-09-21T07:46:27.598597+0000","last_became_active":"2024-09-21T07:35:27.871232+0000","last_became_peered":"2024-09-21T07:35:27.871232+0000","last_unstale":"2024-09-21T07:46:27.598597+0000","last_undegraded":"2024-09-21T07:46:27.598597+0000","last_fullsized":"2024-09-21T07:46:27.598597+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:04.419301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":230,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.586888+0000","last_change":"2024-09-21T07:35:29.491246+0000","last_active":"2024-09-21T07:46:26.586888+0000","last_peered":"2024-09-21T07:46:26.586888+0000","last_clean":"2024-09-21T07:46:26.586888+0000","last_became_active":"2024-09-21T07:35:26.866783+0000","last_became_peered":"2024-09-21T07:35:26.866783+0000","last_unstale":"2024-09-21T07:46:26.586888+0000","last_undegraded":"2024-09-21T07:46:26.586888+0000","last_fullsized":"2024-09-21T07:46:26.586888+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:28.881914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098474099999999992,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":283,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.583293+0000","last_change":"2024-09-21T07:34:25.028640+0000","last_active":"2024-09-21T07:46:26.583293+0000","last_peered":"2024-09-21T07:46:26.583293+0000","last_clean":"2024-09-21T07:46:26.583293+0000","last_became_active":"2024-09-21T07:34:23.260950+0000","last_became_peered":"2024-09-21T07:34:23.260950+0000","last_unstale":"2024-09-21T07:46:26.583293+0000","last_undegraded":"2024-09-21T07:46:26.583293+0000","last_fullsized":"2024-09-21T07:46:26.583293+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:23:42.121398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055135099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.600980+0000","last_change":"2024-09-21T07:46:26.604123+0000","last_active":"2024-09-21T07:46:27.600980+0000","last_peered":"2024-09-21T07:46:27.600980+0000","last_clean":"2024-09-21T07:46:27.600980+0000","last_became_active":"2024-09-21T07:46:26.603295+0000","last_became_peered":"2024-09-21T07:46:26.603295+0000","last_unstale":"2024-09-21T07:46:27.600980+0000","last_undegraded":"2024-09-21T07:46:27.600980+0000","last_fullsized":"2024-09-21T07:46:27.600980+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:09:26.713837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":15,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.601204+0000","last_change":"2024-09-21T07:46:25.605408+0000","last_active":"2024-09-21T07:46:27.601204+0000","last_peered":"2024-09-21T07:46:27.601204+0000","last_clean":"2024-09-21T07:46:27.601204+0000","last_became_active":"2024-09-21T07:46:25.602154+0000","last_became_peered":"2024-09-21T07:46:25.602154+0000","last_unstale":"2024-09-21T07:46:27.601204+0000","last_undegraded":"2024-09-21T07:46:27.601204+0000","last_fullsized":"2024-09-21T07:46:27.601204+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:57.518833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":223,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.606912+0000","last_change":"2024-09-21T07:35:27.872027+0000","last_active":"2024-09-21T07:46:26.606912+0000","last_peered":"2024-09-21T07:46:26.606912+0000","last_clean":"2024-09-21T07:46:26.606912+0000","last_became_active":"2024-09-21T07:35:27.870426+0000","last_became_peered":"2024-09-21T07:35:27.870426+0000","last_unstale":"2024-09-21T07:46:26.606912+0000","last_undegraded":"2024-09-21T07:46:26.606912+0000","last_fullsized":"2024-09-21T07:46:26.606912+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":230,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.587705+0000","last_change":"2024-09-21T07:35:29.491008+0000","last_active":"2024-09-21T07:46:26.587705+0000","last_peered":"2024-09-21T07:46:26.587705+0000","last_clean":"2024-09-21T07:46:26.587705+0000","last_became_active":"2024-09-21T07:35:26.866750+0000","last_became_peered":"2024-09-21T07:35:26.866750+0000","last_unstale":"2024-09-21T07:46:26.587705+0000","last_undegraded":"2024-09-21T07:46:26.587705+0000","last_fullsized":"2024-09-21T07:46:26.587705+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:25.490406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083529700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":285,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.605074+0000","last_change":"2024-09-21T07:34:24.260665+0000","last_active":"2024-09-21T07:46:27.605074+0000","last_peered":"2024-09-21T07:46:27.605074+0000","last_clean":"2024-09-21T07:46:27.605074+0000","last_became_active":"2024-09-21T07:34:23.635909+0000","last_became_peered":"2024-09-21T07:34:23.635909+0000","last_unstale":"2024-09-21T07:46:27.605074+0000","last_undegraded":"2024-09-21T07:46:27.605074+0000","last_fullsized":"2024-09-21T07:46:27.605074+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:44.536885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.602115+0000","last_change":"2024-09-21T07:46:26.597685+0000","last_active":"2024-09-21T07:46:26.602115+0000","last_peered":"2024-09-21T07:46:26.602115+0000","last_clean":"2024-09-21T07:46:26.602115+0000","last_became_active":"2024-09-21T07:46:26.595053+0000","last_became_peered":"2024-09-21T07:46:26.595053+0000","last_unstale":"2024-09-21T07:46:26.602115+0000","last_undegraded":"2024-09-21T07:46:26.602115+0000","last_fullsized":"2024-09-21T07:46:26.602115+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:51:31.780109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":15,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599463+0000","last_change":"2024-09-21T07:46:25.599088+0000","last_active":"2024-09-21T07:46:27.599463+0000","last_peered":"2024-09-21T07:46:27.599463+0000","last_clean":"2024-09-21T07:46:27.599463+0000","last_became_active":"2024-09-21T07:46:25.598661+0000","last_became_peered":"2024-09-21T07:46:25.598661+0000","last_unstale":"2024-09-21T07:46:27.599463+0000","last_undegraded":"2024-09-21T07:46:27.599463+0000","last_fullsized":"2024-09-21T07:46:27.599463+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:45:29.686688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":220,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.606126+0000","last_change":"2024-09-21T07:35:27.874772+0000","last_active":"2024-09-21T07:46:26.606126+0000","last_peered":"2024-09-21T07:46:26.606126+0000","last_clean":"2024-09-21T07:46:26.606126+0000","last_became_active":"2024-09-21T07:35:27.874434+0000","last_became_peered":"2024-09-21T07:35:27.874434+0000","last_unstale":"2024-09-21T07:46:26.606126+0000","last_undegraded":"2024-09-21T07:46:26.606126+0000","last_fullsized":"2024-09-21T07:46:26.606126+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:31.998755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":239,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.601025+0000","last_change":"2024-09-21T07:35:28.859153+0000","last_active":"2024-09-21T07:46:27.601025+0000","last_peered":"2024-09-21T07:46:27.601025+0000","last_clean":"2024-09-21T07:46:27.601025+0000","last_became_active":"2024-09-21T07:35:26.874428+0000","last_became_peered":"2024-09-21T07:35:26.874428+0000","last_unstale":"2024-09-21T07:46:27.601025+0000","last_undegraded":"2024-09-21T07:46:27.601025+0000","last_fullsized":"2024-09-21T07:46:27.601025+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:04.487621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048557299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":291,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.600173+0000","last_change":"2024-09-21T07:34:24.538845+0000","last_active":"2024-09-21T07:46:27.600173+0000","last_peered":"2024-09-21T07:46:27.600173+0000","last_clean":"2024-09-21T07:46:27.600173+0000","last_became_active":"2024-09-21T07:34:23.636372+0000","last_became_peered":"2024-09-21T07:34:23.636372+0000","last_unstale":"2024-09-21T07:46:27.600173+0000","last_undegraded":"2024-09-21T07:46:27.600173+0000","last_fullsized":"2024-09-21T07:46:27.600173+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:29.393813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841451,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599953+0000","last_change":"2024-09-21T07:46:26.604373+0000","last_active":"2024-09-21T07:46:27.599953+0000","last_peered":"2024-09-21T07:46:27.599953+0000","last_clean":"2024-09-21T07:46:27.599953+0000","last_became_active":"2024-09-21T07:46:26.602850+0000","last_became_peered":"2024-09-21T07:46:26.602850+0000","last_unstale":"2024-09-21T07:46:27.599953+0000","last_undegraded":"2024-09-21T07:46:27.599953+0000","last_fullsized":"2024-09-21T07:46:27.599953+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:13.124773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":13,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.587729+0000","last_change":"2024-09-21T07:46:25.596109+0000","last_active":"2024-09-21T07:46:26.587729+0000","last_peered":"2024-09-21T07:46:26.587729+0000","last_clean":"2024-09-21T07:46:26.587729+0000","last_became_active":"2024-09-21T07:46:25.595814+0000","last_became_peered":"2024-09-21T07:46:25.595814+0000","last_unstale":"2024-09-21T07:46:26.587729+0000","last_undegraded":"2024-09-21T07:46:26.587729+0000","last_fullsized":"2024-09-21T07:46:26.587729+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:39:32.880265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":224,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.605537+0000","last_change":"2024-09-21T07:35:27.877801+0000","last_active":"2024-09-21T07:46:26.605537+0000","last_peered":"2024-09-21T07:46:26.605537+0000","last_clean":"2024-09-21T07:46:26.605537+0000","last_became_active":"2024-09-21T07:35:27.877464+0000","last_became_peered":"2024-09-21T07:35:27.877464+0000","last_unstale":"2024-09-21T07:46:26.605537+0000","last_undegraded":"2024-09-21T07:46:26.605537+0000","last_fullsized":"2024-09-21T07:46:26.605537+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:08.814929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":231,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.598652+0000","last_change":"2024-09-21T07:35:28.863732+0000","last_active":"2024-09-21T07:46:27.598652+0000","last_peered":"2024-09-21T07:46:27.598652+0000","last_clean":"2024-09-21T07:46:27.598652+0000","last_became_active":"2024-09-21T07:35:26.877392+0000","last_became_peered":"2024-09-21T07:35:26.877392+0000","last_unstale":"2024-09-21T07:46:27.598652+0000","last_undegraded":"2024-09-21T07:46:27.598652+0000","last_fullsized":"2024-09-21T07:46:27.598652+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071425700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":284,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.590754+0000","last_change":"2024-09-21T07:34:24.259686+0000","last_active":"2024-09-21T07:46:26.590754+0000","last_peered":"2024-09-21T07:46:26.590754+0000","last_clean":"2024-09-21T07:46:26.590754+0000","last_became_active":"2024-09-21T07:34:23.274405+0000","last_became_peered":"2024-09-21T07:34:23.274405+0000","last_unstale":"2024-09-21T07:46:26.590754+0000","last_undegraded":"2024-09-21T07:46:26.590754+0000","last_fullsized":"2024-09-21T07:46:26.590754+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:26.304693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022009920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599039+0000","last_change":"2024-09-21T07:46:26.593329+0000","last_active":"2024-09-21T07:46:27.599039+0000","last_peered":"2024-09-21T07:46:27.599039+0000","last_clean":"2024-09-21T07:46:27.599039+0000","last_became_active":"2024-09-21T07:46:26.592688+0000","last_became_peered":"2024-09-21T07:46:26.592688+0000","last_unstale":"2024-09-21T07:46:27.599039+0000","last_undegraded":"2024-09-21T07:46:27.599039+0000","last_fullsized":"2024-09-21T07:46:27.599039+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:53.049143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"174'1","reported_seq":14,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.628422+0000","last_change":"2024-09-21T07:46:25.603666+0000","last_active":"2024-09-21T07:46:26.628422+0000","last_peered":"2024-09-21T07:46:26.628422+0000","last_clean":"2024-09-21T07:46:26.628422+0000","last_became_active":"2024-09-21T07:46:25.603334+0000","last_became_peered":"2024-09-21T07:46:25.603334+0000","last_unstale":"2024-09-21T07:46:26.628422+0000","last_undegraded":"2024-09-21T07:46:26.628422+0000","last_fullsized":"2024-09-21T07:46:26.628422+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:29.725512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":220,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.607793+0000","last_change":"2024-09-21T07:35:27.867336+0000","last_active":"2024-09-21T07:46:26.607793+0000","last_peered":"2024-09-21T07:46:26.607793+0000","last_clean":"2024-09-21T07:46:26.607793+0000","last_became_active":"2024-09-21T07:35:27.867015+0000","last_became_peered":"2024-09-21T07:35:27.867015+0000","last_unstale":"2024-09-21T07:46:26.607793+0000","last_undegraded":"2024-09-21T07:46:26.607793+0000","last_fullsized":"2024-09-21T07:46:26.607793+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:59.214139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":232,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.604942+0000","last_change":"2024-09-21T07:35:29.493137+0000","last_active":"2024-09-21T07:46:27.604942+0000","last_peered":"2024-09-21T07:46:27.604942+0000","last_clean":"2024-09-21T07:46:27.604942+0000","last_became_active":"2024-09-21T07:35:26.872230+0000","last_became_peered":"2024-09-21T07:35:26.872230+0000","last_unstale":"2024-09-21T07:46:27.604942+0000","last_undegraded":"2024-09-21T07:46:27.604942+0000","last_fullsized":"2024-09-21T07:46:27.604942+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:24.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001197449,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":285,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.603882+0000","last_change":"2024-09-21T07:34:24.258754+0000","last_active":"2024-09-21T07:46:27.603882+0000","last_peered":"2024-09-21T07:46:27.603882+0000","last_clean":"2024-09-21T07:46:27.603882+0000","last_became_active":"2024-09-21T07:34:23.272657+0000","last_became_peered":"2024-09-21T07:34:23.272657+0000","last_unstale":"2024-09-21T07:46:27.603882+0000","last_undegraded":"2024-09-21T07:46:27.603882+0000","last_fullsized":"2024-09-21T07:46:27.603882+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:05.477099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067143200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.621977+0000","last_change":"2024-09-21T07:46:26.616838+0000","last_active":"2024-09-21T07:46:26.621977+0000","last_peered":"2024-09-21T07:46:26.621977+0000","last_clean":"2024-09-21T07:46:26.621977+0000","last_became_active":"2024-09-21T07:46:26.616318+0000","last_became_peered":"2024-09-21T07:46:26.616318+0000","last_unstale":"2024-09-21T07:46:26.621977+0000","last_undegraded":"2024-09-21T07:46:26.621977+0000","last_fullsized":"2024-09-21T07:46:26.621977+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:10.469068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"174'1","reported_seq":14,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.626072+0000","last_change":"2024-09-21T07:46:25.593521+0000","last_active":"2024-09-21T07:46:26.626072+0000","last_peered":"2024-09-21T07:46:26.626072+0000","last_clean":"2024-09-21T07:46:26.626072+0000","last_became_active":"2024-09-21T07:46:25.592303+0000","last_became_peered":"2024-09-21T07:46:25.592303+0000","last_unstale":"2024-09-21T07:46:26.626072+0000","last_undegraded":"2024-09-21T07:46:26.626072+0000","last_fullsized":"2024-09-21T07:46:26.626072+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:25:51.090251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":224,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599255+0000","last_change":"2024-09-21T07:35:27.876021+0000","last_active":"2024-09-21T07:46:27.599255+0000","last_peered":"2024-09-21T07:46:27.599255+0000","last_clean":"2024-09-21T07:46:27.599255+0000","last_became_active":"2024-09-21T07:35:27.875724+0000","last_became_peered":"2024-09-21T07:35:27.875724+0000","last_unstale":"2024-09-21T07:46:27.599255+0000","last_undegraded":"2024-09-21T07:46:27.599255+0000","last_fullsized":"2024-09-21T07:46:27.599255+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:03.022812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":232,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.604201+0000","last_change":"2024-09-21T07:35:29.492570+0000","last_active":"2024-09-21T07:46:27.604201+0000","last_peered":"2024-09-21T07:46:27.604201+0000","last_clean":"2024-09-21T07:46:27.604201+0000","last_became_active":"2024-09-21T07:35:26.874316+0000","last_became_peered":"2024-09-21T07:35:26.874316+0000","last_unstale":"2024-09-21T07:46:27.604201+0000","last_undegraded":"2024-09-21T07:46:27.604201+0000","last_fullsized":"2024-09-21T07:46:27.604201+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:16.512323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011990760000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":282,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599291+0000","last_change":"2024-09-21T07:34:25.085232+0000","last_active":"2024-09-21T07:46:27.599291+0000","last_peered":"2024-09-21T07:46:27.599291+0000","last_clean":"2024-09-21T07:46:27.599291+0000","last_became_active":"2024-09-21T07:34:23.277807+0000","last_became_peered":"2024-09-21T07:34:23.277807+0000","last_unstale":"2024-09-21T07:46:27.599291+0000","last_undegraded":"2024-09-21T07:46:27.599291+0000","last_fullsized":"2024-09-21T07:46:27.599291+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:36.362919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086619099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.625960+0000","last_change":"2024-09-21T07:46:26.620027+0000","last_active":"2024-09-21T07:46:26.625960+0000","last_peered":"2024-09-21T07:46:26.625960+0000","last_clean":"2024-09-21T07:46:26.625960+0000","last_became_active":"2024-09-21T07:46:26.619485+0000","last_became_peered":"2024-09-21T07:46:26.619485+0000","last_unstale":"2024-09-21T07:46:26.625960+0000","last_undegraded":"2024-09-21T07:46:26.625960+0000","last_fullsized":"2024-09-21T07:46:26.625960+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:59:28.471988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"174'2","reported_seq":17,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.604494+0000","last_change":"2024-09-21T07:46:25.606943+0000","last_active":"2024-09-21T07:46:27.604494+0000","last_peered":"2024-09-21T07:46:27.604494+0000","last_clean":"2024-09-21T07:46:27.604494+0000","last_became_active":"2024-09-21T07:46:25.606617+0000","last_became_peered":"2024-09-21T07:46:25.606617+0000","last_unstale":"2024-09-21T07:46:27.604494+0000","last_undegraded":"2024-09-21T07:46:27.604494+0000","last_fullsized":"2024-09-21T07:46:27.604494+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:27.687486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":224,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.608132+0000","last_change":"2024-09-21T07:35:27.872798+0000","last_active":"2024-09-21T07:46:26.608132+0000","last_peered":"2024-09-21T07:46:26.608132+0000","last_clean":"2024-09-21T07:46:26.608132+0000","last_became_active":"2024-09-21T07:35:27.872483+0000","last_became_peered":"2024-09-21T07:35:27.872483+0000","last_unstale":"2024-09-21T07:46:26.608132+0000","last_undegraded":"2024-09-21T07:46:26.608132+0000","last_fullsized":"2024-09-21T07:46:26.608132+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:11.933469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":231,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.607440+0000","last_change":"2024-09-21T07:35:29.206723+0000","last_active":"2024-09-21T07:46:26.607440+0000","last_peered":"2024-09-21T07:46:26.607440+0000","last_clean":"2024-09-21T07:46:26.607440+0000","last_became_active":"2024-09-21T07:35:26.872377+0000","last_became_peered":"2024-09-21T07:35:26.872377+0000","last_unstale":"2024-09-21T07:46:26.607440+0000","last_undegraded":"2024-09-21T07:46:26.607440+0000","last_fullsized":"2024-09-21T07:46:26.607440+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:29.086862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083077200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":276,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.615675+0000","last_change":"2024-09-21T07:34:24.597672+0000","last_active":"2024-09-21T07:46:26.615675+0000","last_peered":"2024-09-21T07:46:26.615675+0000","last_clean":"2024-09-21T07:46:26.615675+0000","last_became_active":"2024-09-21T07:34:23.267770+0000","last_became_peered":"2024-09-21T07:34:23.267770+0000","last_unstale":"2024-09-21T07:46:26.615675+0000","last_undegraded":"2024-09-21T07:46:26.615675+0000","last_fullsized":"2024-09-21T07:46:26.615675+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:51.629476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086378500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.604975+0000","last_change":"2024-09-21T07:46:26.620299+0000","last_active":"2024-09-21T07:46:27.604975+0000","last_peered":"2024-09-21T07:46:27.604975+0000","last_clean":"2024-09-21T07:46:27.604975+0000","last_became_active":"2024-09-21T07:46:26.619941+0000","last_became_peered":"2024-09-21T07:46:26.619941+0000","last_unstale":"2024-09-21T07:46:27.604975+0000","last_undegraded":"2024-09-21T07:46:27.604975+0000","last_fullsized":"2024-09-21T07:46:27.604975+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:55:53.588662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"174'2","reported_seq":15,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.631050+0000","last_change":"2024-09-21T07:46:25.600068+0000","last_active":"2024-09-21T07:46:26.631050+0000","last_peered":"2024-09-21T07:46:26.631050+0000","last_clean":"2024-09-21T07:46:26.631050+0000","last_became_active":"2024-09-21T07:46:25.598819+0000","last_became_peered":"2024-09-21T07:46:25.598819+0000","last_unstale":"2024-09-21T07:46:26.631050+0000","last_undegraded":"2024-09-21T07:46:26.631050+0000","last_fullsized":"2024-09-21T07:46:26.631050+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:26:06.303080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":224,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.607343+0000","last_change":"2024-09-21T07:35:27.870121+0000","last_active":"2024-09-21T07:46:26.607343+0000","last_peered":"2024-09-21T07:46:26.607343+0000","last_clean":"2024-09-21T07:46:26.607343+0000","last_became_active":"2024-09-21T07:35:27.869811+0000","last_became_peered":"2024-09-21T07:35:27.869811+0000","last_unstale":"2024-09-21T07:46:26.607343+0000","last_undegraded":"2024-09-21T07:46:26.607343+0000","last_fullsized":"2024-09-21T07:46:26.607343+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:31.778461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":232,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.600765+0000","last_change":"2024-09-21T07:35:28.860699+0000","last_active":"2024-09-21T07:46:27.600765+0000","last_peered":"2024-09-21T07:46:27.600765+0000","last_clean":"2024-09-21T07:46:27.600765+0000","last_became_active":"2024-09-21T07:35:26.876779+0000","last_became_peered":"2024-09-21T07:35:26.876779+0000","last_unstale":"2024-09-21T07:46:27.600765+0000","last_undegraded":"2024-09-21T07:46:27.600765+0000","last_fullsized":"2024-09-21T07:46:27.600765+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:28.286403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000913188,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":284,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.601364+0000","last_change":"2024-09-21T07:34:24.260181+0000","last_active":"2024-09-21T07:46:27.601364+0000","last_peered":"2024-09-21T07:46:27.601364+0000","last_clean":"2024-09-21T07:46:27.601364+0000","last_became_active":"2024-09-21T07:34:23.275580+0000","last_became_peered":"2024-09-21T07:34:23.275580+0000","last_unstale":"2024-09-21T07:46:27.601364+0000","last_undegraded":"2024-09-21T07:46:27.601364+0000","last_fullsized":"2024-09-21T07:46:27.601364+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:49.857622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014706980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.622749+0000","last_change":"2024-09-21T07:46:26.614494+0000","last_active":"2024-09-21T07:46:26.622749+0000","last_peered":"2024-09-21T07:46:26.622749+0000","last_clean":"2024-09-21T07:46:26.622749+0000","last_became_active":"2024-09-21T07:46:26.601276+0000","last_became_peered":"2024-09-21T07:46:26.601276+0000","last_unstale":"2024-09-21T07:46:26.622749+0000","last_undegraded":"2024-09-21T07:46:26.622749+0000","last_fullsized":"2024-09-21T07:46:26.622749+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:49:25.294386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"174'4","reported_seq":17,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.627490+0000","last_change":"2024-09-21T07:46:25.590100+0000","last_active":"2024-09-21T07:46:26.627490+0000","last_peered":"2024-09-21T07:46:26.627490+0000","last_clean":"2024-09-21T07:46:26.627490+0000","last_became_active":"2024-09-21T07:46:25.589489+0000","last_became_peered":"2024-09-21T07:46:25.589489+0000","last_unstale":"2024-09-21T07:46:26.627490+0000","last_undegraded":"2024-09-21T07:46:26.627490+0000","last_fullsized":"2024-09-21T07:46:26.627490+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:15.158504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":225,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.583893+0000","last_change":"2024-09-21T07:35:27.875393+0000","last_active":"2024-09-21T07:46:26.583893+0000","last_peered":"2024-09-21T07:46:26.583893+0000","last_clean":"2024-09-21T07:46:26.583893+0000","last_became_active":"2024-09-21T07:35:27.874671+0000","last_became_peered":"2024-09-21T07:35:27.874671+0000","last_unstale":"2024-09-21T07:46:26.583893+0000","last_undegraded":"2024-09-21T07:46:26.583893+0000","last_fullsized":"2024-09-21T07:46:26.583893+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":231,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.606543+0000","last_change":"2024-09-21T07:35:29.206296+0000","last_active":"2024-09-21T07:46:26.606543+0000","last_peered":"2024-09-21T07:46:26.606543+0000","last_clean":"2024-09-21T07:46:26.606543+0000","last_became_active":"2024-09-21T07:35:26.874575+0000","last_became_peered":"2024-09-21T07:35:26.874575+0000","last_unstale":"2024-09-21T07:46:26.606543+0000","last_undegraded":"2024-09-21T07:46:26.606543+0000","last_fullsized":"2024-09-21T07:46:26.606543+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:22.771154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061011599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":284,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.590311+0000","last_change":"2024-09-21T07:34:24.259112+0000","last_active":"2024-09-21T07:46:26.590311+0000","last_peered":"2024-09-21T07:46:26.590311+0000","last_clean":"2024-09-21T07:46:26.590311+0000","last_became_active":"2024-09-21T07:34:23.635444+0000","last_became_peered":"2024-09-21T07:34:23.635444+0000","last_unstale":"2024-09-21T07:46:26.590311+0000","last_undegraded":"2024-09-21T07:46:26.590311+0000","last_fullsized":"2024-09-21T07:46:26.590311+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:46.255385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001952636,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.623924+0000","last_change":"2024-09-21T07:46:26.615601+0000","last_active":"2024-09-21T07:46:26.623924+0000","last_peered":"2024-09-21T07:46:26.623924+0000","last_clean":"2024-09-21T07:46:26.623924+0000","last_became_active":"2024-09-21T07:46:26.615269+0000","last_became_peered":"2024-09-21T07:46:26.615269+0000","last_unstale":"2024-09-21T07:46:26.623924+0000","last_undegraded":"2024-09-21T07:46:26.623924+0000","last_fullsized":"2024-09-21T07:46:26.623924+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:06.717177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"174'1","reported_seq":16,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.604583+0000","last_change":"2024-09-21T07:46:25.608137+0000","last_active":"2024-09-21T07:46:27.604583+0000","last_peered":"2024-09-21T07:46:27.604583+0000","last_clean":"2024-09-21T07:46:27.604583+0000","last_became_active":"2024-09-21T07:46:25.607728+0000","last_became_peered":"2024-09-21T07:46:25.607728+0000","last_unstale":"2024-09-21T07:46:27.604583+0000","last_undegraded":"2024-09-21T07:46:27.604583+0000","last_fullsized":"2024-09-21T07:46:27.604583+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:43:30.481435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":225,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.582087+0000","last_change":"2024-09-21T07:35:27.878356+0000","last_active":"2024-09-21T07:46:26.582087+0000","last_peered":"2024-09-21T07:46:26.582087+0000","last_clean":"2024-09-21T07:46:26.582087+0000","last_became_active":"2024-09-21T07:35:27.877982+0000","last_became_peered":"2024-09-21T07:35:27.877982+0000","last_unstale":"2024-09-21T07:46:26.582087+0000","last_undegraded":"2024-09-21T07:46:26.582087+0000","last_fullsized":"2024-09-21T07:46:26.582087+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":232,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.582045+0000","last_change":"2024-09-21T07:35:29.053131+0000","last_active":"2024-09-21T07:46:26.582045+0000","last_peered":"2024-09-21T07:46:26.582045+0000","last_clean":"2024-09-21T07:46:26.582045+0000","last_became_active":"2024-09-21T07:35:26.870516+0000","last_became_peered":"2024-09-21T07:35:26.870516+0000","last_unstale":"2024-09-21T07:46:26.582045+0000","last_undegraded":"2024-09-21T07:46:26.582045+0000","last_fullsized":"2024-09-21T07:46:26.582045+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:10.924491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070705699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":291,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.600658+0000","last_change":"2024-09-21T07:34:24.539152+0000","last_active":"2024-09-21T07:46:27.600658+0000","last_peered":"2024-09-21T07:46:27.600658+0000","last_clean":"2024-09-21T07:46:27.600658+0000","last_became_active":"2024-09-21T07:34:23.259573+0000","last_became_peered":"2024-09-21T07:34:23.259573+0000","last_unstale":"2024-09-21T07:46:27.600658+0000","last_undegraded":"2024-09-21T07:46:27.600658+0000","last_fullsized":"2024-09-21T07:46:27.600658+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:26.986900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058108400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.624237+0000","last_change":"2024-09-21T07:46:26.620012+0000","last_active":"2024-09-21T07:46:26.624237+0000","last_peered":"2024-09-21T07:46:26.624237+0000","last_clean":"2024-09-21T07:46:26.624237+0000","last_became_active":"2024-09-21T07:46:26.615667+0000","last_became_peered":"2024-09-21T07:46:26.615667+0000","last_unstale":"2024-09-21T07:46:26.624237+0000","last_undegraded":"2024-09-21T07:46:26.624237+0000","last_fullsized":"2024-09-21T07:46:26.624237+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:45:57.255990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":15,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599600+0000","last_change":"2024-09-21T07:46:25.598178+0000","last_active":"2024-09-21T07:46:27.599600+0000","last_peered":"2024-09-21T07:46:27.599600+0000","last_clean":"2024-09-21T07:46:27.599600+0000","last_became_active":"2024-09-21T07:46:25.597252+0000","last_became_peered":"2024-09-21T07:46:25.597252+0000","last_unstale":"2024-09-21T07:46:27.599600+0000","last_undegraded":"2024-09-21T07:46:27.599600+0000","last_fullsized":"2024-09-21T07:46:27.599600+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:09.902385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":224,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.581917+0000","last_change":"2024-09-21T07:35:27.873726+0000","last_active":"2024-09-21T07:46:26.581917+0000","last_peered":"2024-09-21T07:46:26.581917+0000","last_clean":"2024-09-21T07:46:26.581917+0000","last_became_active":"2024-09-21T07:35:27.871421+0000","last_became_peered":"2024-09-21T07:35:27.871421+0000","last_unstale":"2024-09-21T07:46:26.581917+0000","last_undegraded":"2024-09-21T07:46:26.581917+0000","last_fullsized":"2024-09-21T07:46:26.581917+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:00.168915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":232,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.601191+0000","last_change":"2024-09-21T07:35:28.861032+0000","last_active":"2024-09-21T07:46:27.601191+0000","last_peered":"2024-09-21T07:46:27.601191+0000","last_clean":"2024-09-21T07:46:27.601191+0000","last_became_active":"2024-09-21T07:35:26.873149+0000","last_became_peered":"2024-09-21T07:35:26.873149+0000","last_unstale":"2024-09-21T07:46:27.601191+0000","last_undegraded":"2024-09-21T07:46:27.601191+0000","last_fullsized":"2024-09-21T07:46:27.601191+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:59.435981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011988369999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":285,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.604002+0000","last_change":"2024-09-21T07:34:24.259902+0000","last_active":"2024-09-21T07:46:27.604002+0000","last_peered":"2024-09-21T07:46:27.604002+0000","last_clean":"2024-09-21T07:46:27.604002+0000","last_became_active":"2024-09-21T07:34:23.271902+0000","last_became_peered":"2024-09-21T07:34:23.271902+0000","last_unstale":"2024-09-21T07:46:27.604002+0000","last_undegraded":"2024-09-21T07:46:27.604002+0000","last_fullsized":"2024-09-21T07:46:27.604002+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:59.439525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001686358,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.626050+0000","last_change":"2024-09-21T07:46:26.618986+0000","last_active":"2024-09-21T07:46:26.626050+0000","last_peered":"2024-09-21T07:46:26.626050+0000","last_clean":"2024-09-21T07:46:26.626050+0000","last_became_active":"2024-09-21T07:46:26.618685+0000","last_became_peered":"2024-09-21T07:46:26.618685+0000","last_unstale":"2024-09-21T07:46:26.626050+0000","last_undegraded":"2024-09-21T07:46:26.626050+0000","last_fullsized":"2024-09-21T07:46:26.626050+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:43:05.973019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":15,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.603794+0000","last_change":"2024-09-21T07:46:25.599666+0000","last_active":"2024-09-21T07:46:27.603794+0000","last_peered":"2024-09-21T07:46:27.603794+0000","last_clean":"2024-09-21T07:46:27.603794+0000","last_became_active":"2024-09-21T07:46:25.598879+0000","last_became_peered":"2024-09-21T07:46:25.598879+0000","last_unstale":"2024-09-21T07:46:27.603794+0000","last_undegraded":"2024-09-21T07:46:27.603794+0000","last_fullsized":"2024-09-21T07:46:27.603794+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:46.170670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":223,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.598903+0000","last_change":"2024-09-21T07:35:27.872637+0000","last_active":"2024-09-21T07:46:27.598903+0000","last_peered":"2024-09-21T07:46:27.598903+0000","last_clean":"2024-09-21T07:46:27.598903+0000","last_became_active":"2024-09-21T07:35:27.872149+0000","last_became_peered":"2024-09-21T07:35:27.872149+0000","last_unstale":"2024-09-21T07:46:27.598903+0000","last_undegraded":"2024-09-21T07:46:27.598903+0000","last_fullsized":"2024-09-21T07:46:27.598903+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:27.697872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":232,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.604683+0000","last_change":"2024-09-21T07:35:29.493362+0000","last_active":"2024-09-21T07:46:27.604683+0000","last_peered":"2024-09-21T07:46:27.604683+0000","last_clean":"2024-09-21T07:46:27.604683+0000","last_became_active":"2024-09-21T07:35:26.873018+0000","last_became_peered":"2024-09-21T07:35:26.873018+0000","last_unstale":"2024-09-21T07:46:27.604683+0000","last_undegraded":"2024-09-21T07:46:27.604683+0000","last_fullsized":"2024-09-21T07:46:27.604683+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:02.460557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012496510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":284,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.590535+0000","last_change":"2024-09-21T07:34:24.259662+0000","last_active":"2024-09-21T07:46:26.590535+0000","last_peered":"2024-09-21T07:46:26.590535+0000","last_clean":"2024-09-21T07:46:26.590535+0000","last_became_active":"2024-09-21T07:34:23.278812+0000","last_became_peered":"2024-09-21T07:34:23.278812+0000","last_unstale":"2024-09-21T07:46:26.590535+0000","last_undegraded":"2024-09-21T07:46:26.590535+0000","last_fullsized":"2024-09-21T07:46:26.590535+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:49.829974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356463,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.622065+0000","last_change":"2024-09-21T07:46:26.611463+0000","last_active":"2024-09-21T07:46:26.622065+0000","last_peered":"2024-09-21T07:46:26.622065+0000","last_clean":"2024-09-21T07:46:26.622065+0000","last_became_active":"2024-09-21T07:46:26.611043+0000","last_became_peered":"2024-09-21T07:46:26.611043+0000","last_unstale":"2024-09-21T07:46:26.622065+0000","last_undegraded":"2024-09-21T07:46:26.622065+0000","last_fullsized":"2024-09-21T07:46:26.622065+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:11.692802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":15,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599832+0000","last_change":"2024-09-21T07:46:25.596861+0000","last_active":"2024-09-21T07:46:27.599832+0000","last_peered":"2024-09-21T07:46:27.599832+0000","last_clean":"2024-09-21T07:46:27.599832+0000","last_became_active":"2024-09-21T07:46:25.596502+0000","last_became_peered":"2024-09-21T07:46:25.596502+0000","last_unstale":"2024-09-21T07:46:27.599832+0000","last_undegraded":"2024-09-21T07:46:27.599832+0000","last_fullsized":"2024-09-21T07:46:27.599832+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:03:08.985516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":219,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.607681+0000","last_change":"2024-09-21T07:35:27.868448+0000","last_active":"2024-09-21T07:46:26.607681+0000","last_peered":"2024-09-21T07:46:26.607681+0000","last_clean":"2024-09-21T07:46:26.607681+0000","last_became_active":"2024-09-21T07:35:27.868105+0000","last_became_peered":"2024-09-21T07:35:27.868105+0000","last_unstale":"2024-09-21T07:46:26.607681+0000","last_undegraded":"2024-09-21T07:46:26.607681+0000","last_fullsized":"2024-09-21T07:46:26.607681+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:41.468702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":231,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.604712+0000","last_change":"2024-09-21T07:35:29.492974+0000","last_active":"2024-09-21T07:46:27.604712+0000","last_peered":"2024-09-21T07:46:27.604712+0000","last_clean":"2024-09-21T07:46:27.604712+0000","last_became_active":"2024-09-21T07:35:26.872165+0000","last_became_peered":"2024-09-21T07:35:26.872165+0000","last_unstale":"2024-09-21T07:46:27.604712+0000","last_undegraded":"2024-09-21T07:46:27.604712+0000","last_fullsized":"2024-09-21T07:46:27.604712+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:59.912520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013795179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":282,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.598450+0000","last_change":"2024-09-21T07:34:25.084861+0000","last_active":"2024-09-21T07:46:27.598450+0000","last_peered":"2024-09-21T07:46:27.598450+0000","last_clean":"2024-09-21T07:46:27.598450+0000","last_became_active":"2024-09-21T07:34:23.277708+0000","last_became_peered":"2024-09-21T07:34:23.277708+0000","last_unstale":"2024-09-21T07:46:27.598450+0000","last_undegraded":"2024-09-21T07:46:27.598450+0000","last_fullsized":"2024-09-21T07:46:27.598450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:40.816488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000597566,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.618746+0000","last_change":"2024-09-21T07:46:26.615012+0000","last_active":"2024-09-21T07:46:26.618746+0000","last_peered":"2024-09-21T07:46:26.618746+0000","last_clean":"2024-09-21T07:46:26.618746+0000","last_became_active":"2024-09-21T07:46:26.614673+0000","last_became_peered":"2024-09-21T07:46:26.614673+0000","last_unstale":"2024-09-21T07:46:26.618746+0000","last_undegraded":"2024-09-21T07:46:26.618746+0000","last_fullsized":"2024-09-21T07:46:26.618746+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:28.357288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":15,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.600651+0000","last_change":"2024-09-21T07:46:25.599045+0000","last_active":"2024-09-21T07:46:27.600651+0000","last_peered":"2024-09-21T07:46:27.600651+0000","last_clean":"2024-09-21T07:46:27.600651+0000","last_became_active":"2024-09-21T07:46:25.596844+0000","last_became_peered":"2024-09-21T07:46:25.596844+0000","last_unstale":"2024-09-21T07:46:27.600651+0000","last_undegraded":"2024-09-21T07:46:27.600651+0000","last_fullsized":"2024-09-21T07:46:27.600651+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:21:36.983547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":219,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.609922+0000","last_change":"2024-09-21T07:35:27.874640+0000","last_active":"2024-09-21T07:46:26.609922+0000","last_peered":"2024-09-21T07:46:26.609922+0000","last_clean":"2024-09-21T07:46:26.609922+0000","last_became_active":"2024-09-21T07:35:27.874265+0000","last_became_peered":"2024-09-21T07:35:27.874265+0000","last_unstale":"2024-09-21T07:46:26.609922+0000","last_undegraded":"2024-09-21T07:46:26.609922+0000","last_fullsized":"2024-09-21T07:46:26.609922+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:45.754654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":230,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.608366+0000","last_change":"2024-09-21T07:35:29.207392+0000","last_active":"2024-09-21T07:46:26.608366+0000","last_peered":"2024-09-21T07:46:26.608366+0000","last_clean":"2024-09-21T07:46:26.608366+0000","last_became_active":"2024-09-21T07:35:26.870917+0000","last_became_peered":"2024-09-21T07:35:26.870917+0000","last_unstale":"2024-09-21T07:46:26.608366+0000","last_undegraded":"2024-09-21T07:46:26.608366+0000","last_fullsized":"2024-09-21T07:46:26.608366+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:59.159937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059198499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":284,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.587942+0000","last_change":"2024-09-21T07:34:24.258709+0000","last_active":"2024-09-21T07:46:26.587942+0000","last_peered":"2024-09-21T07:46:26.587942+0000","last_clean":"2024-09-21T07:46:26.587942+0000","last_became_active":"2024-09-21T07:34:23.279042+0000","last_became_peered":"2024-09-21T07:34:23.279042+0000","last_unstale":"2024-09-21T07:46:26.587942+0000","last_undegraded":"2024-09-21T07:46:26.587942+0000","last_fullsized":"2024-09-21T07:46:26.587942+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:01.762117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019282100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.601198+0000","last_change":"2024-09-21T07:46:26.609601+0000","last_active":"2024-09-21T07:46:27.601198+0000","last_peered":"2024-09-21T07:46:27.601198+0000","last_clean":"2024-09-21T07:46:27.601198+0000","last_became_active":"2024-09-21T07:46:26.608772+0000","last_became_peered":"2024-09-21T07:46:26.608772+0000","last_unstale":"2024-09-21T07:46:27.601198+0000","last_undegraded":"2024-09-21T07:46:27.601198+0000","last_fullsized":"2024-09-21T07:46:27.601198+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:08:40.114009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":13,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.582709+0000","last_change":"2024-09-21T07:46:25.601134+0000","last_active":"2024-09-21T07:46:26.582709+0000","last_peered":"2024-09-21T07:46:26.582709+0000","last_clean":"2024-09-21T07:46:26.582709+0000","last_became_active":"2024-09-21T07:46:25.599488+0000","last_became_peered":"2024-09-21T07:46:25.599488+0000","last_unstale":"2024-09-21T07:46:26.582709+0000","last_undegraded":"2024-09-21T07:46:26.582709+0000","last_fullsized":"2024-09-21T07:46:26.582709+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:08:39.105569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":223,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599652+0000","last_change":"2024-09-21T07:35:27.870260+0000","last_active":"2024-09-21T07:46:27.599652+0000","last_peered":"2024-09-21T07:46:27.599652+0000","last_clean":"2024-09-21T07:46:27.599652+0000","last_became_active":"2024-09-21T07:35:27.869448+0000","last_became_peered":"2024-09-21T07:35:27.869448+0000","last_unstale":"2024-09-21T07:46:27.599652+0000","last_undegraded":"2024-09-21T07:46:27.599652+0000","last_fullsized":"2024-09-21T07:46:27.599652+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:16.946790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":226,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.609808+0000","last_change":"2024-09-21T07:35:29.386367+0000","last_active":"2024-09-21T07:46:26.609808+0000","last_peered":"2024-09-21T07:46:26.609808+0000","last_clean":"2024-09-21T07:46:26.609808+0000","last_became_active":"2024-09-21T07:35:26.879238+0000","last_became_peered":"2024-09-21T07:35:26.879238+0000","last_unstale":"2024-09-21T07:46:26.609808+0000","last_undegraded":"2024-09-21T07:46:26.609808+0000","last_fullsized":"2024-09-21T07:46:26.609808+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:16.528260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092223100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":282,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.598833+0000","last_change":"2024-09-21T07:34:25.085635+0000","last_active":"2024-09-21T07:46:27.598833+0000","last_peered":"2024-09-21T07:46:27.598833+0000","last_clean":"2024-09-21T07:46:27.598833+0000","last_became_active":"2024-09-21T07:34:23.273566+0000","last_became_peered":"2024-09-21T07:34:23.273566+0000","last_unstale":"2024-09-21T07:46:27.598833+0000","last_undegraded":"2024-09-21T07:46:27.598833+0000","last_fullsized":"2024-09-21T07:46:27.598833+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:14.142272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079897300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.613609+0000","last_change":"2024-09-21T07:46:26.611680+0000","last_active":"2024-09-21T07:46:26.613609+0000","last_peered":"2024-09-21T07:46:26.613609+0000","last_clean":"2024-09-21T07:46:26.613609+0000","last_became_active":"2024-09-21T07:46:26.611369+0000","last_became_peered":"2024-09-21T07:46:26.611369+0000","last_unstale":"2024-09-21T07:46:26.613609+0000","last_undegraded":"2024-09-21T07:46:26.613609+0000","last_fullsized":"2024-09-21T07:46:26.613609+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:05:44.283123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":13,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.590091+0000","last_change":"2024-09-21T07:46:25.595917+0000","last_active":"2024-09-21T07:46:26.590091+0000","last_peered":"2024-09-21T07:46:26.590091+0000","last_clean":"2024-09-21T07:46:26.590091+0000","last_became_active":"2024-09-21T07:46:25.595403+0000","last_became_peered":"2024-09-21T07:46:25.595403+0000","last_unstale":"2024-09-21T07:46:26.590091+0000","last_undegraded":"2024-09-21T07:46:26.590091+0000","last_fullsized":"2024-09-21T07:46:26.590091+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:25:41.466314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":223,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.598756+0000","last_change":"2024-09-21T07:35:27.878728+0000","last_active":"2024-09-21T07:46:27.598756+0000","last_peered":"2024-09-21T07:46:27.598756+0000","last_clean":"2024-09-21T07:46:27.598756+0000","last_became_active":"2024-09-21T07:35:27.878335+0000","last_became_peered":"2024-09-21T07:35:27.878335+0000","last_unstale":"2024-09-21T07:46:27.598756+0000","last_undegraded":"2024-09-21T07:46:27.598756+0000","last_fullsized":"2024-09-21T07:46:27.598756+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:25:37.731771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":238,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.600021+0000","last_change":"2024-09-21T07:35:28.858418+0000","last_active":"2024-09-21T07:46:27.600021+0000","last_peered":"2024-09-21T07:46:27.600021+0000","last_clean":"2024-09-21T07:46:27.600021+0000","last_became_active":"2024-09-21T07:35:26.876944+0000","last_became_peered":"2024-09-21T07:35:26.876944+0000","last_unstale":"2024-09-21T07:46:27.600021+0000","last_undegraded":"2024-09-21T07:46:27.600021+0000","last_fullsized":"2024-09-21T07:46:27.600021+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:04.514304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090862799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":275,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.610848+0000","last_change":"2024-09-21T07:34:24.597926+0000","last_active":"2024-09-21T07:46:26.610848+0000","last_peered":"2024-09-21T07:46:26.610848+0000","last_clean":"2024-09-21T07:46:26.610848+0000","last_became_active":"2024-09-21T07:34:23.268926+0000","last_became_peered":"2024-09-21T07:34:23.268926+0000","last_unstale":"2024-09-21T07:46:26.610848+0000","last_undegraded":"2024-09-21T07:46:26.610848+0000","last_fullsized":"2024-09-21T07:46:26.610848+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:46.190752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101851,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.609925+0000","last_change":"2024-09-21T07:46:26.600244+0000","last_active":"2024-09-21T07:46:26.609925+0000","last_peered":"2024-09-21T07:46:26.609925+0000","last_clean":"2024-09-21T07:46:26.609925+0000","last_became_active":"2024-09-21T07:46:26.599918+0000","last_became_peered":"2024-09-21T07:46:26.599918+0000","last_unstale":"2024-09-21T07:46:26.609925+0000","last_undegraded":"2024-09-21T07:46:26.609925+0000","last_fullsized":"2024-09-21T07:46:26.609925+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:34:39.369981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"174'2","reported_seq":15,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.629374+0000","last_change":"2024-09-21T07:46:25.604769+0000","last_active":"2024-09-21T07:46:26.629374+0000","last_peered":"2024-09-21T07:46:26.629374+0000","last_clean":"2024-09-21T07:46:26.629374+0000","last_became_active":"2024-09-21T07:46:25.603758+0000","last_became_peered":"2024-09-21T07:46:25.603758+0000","last_unstale":"2024-09-21T07:46:26.629374+0000","last_undegraded":"2024-09-21T07:46:26.629374+0000","last_fullsized":"2024-09-21T07:46:26.629374+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:10.336493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":220,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.611594+0000","last_change":"2024-09-21T07:35:27.877426+0000","last_active":"2024-09-21T07:46:26.611594+0000","last_peered":"2024-09-21T07:46:26.611594+0000","last_clean":"2024-09-21T07:46:26.611594+0000","last_became_active":"2024-09-21T07:35:27.877050+0000","last_became_peered":"2024-09-21T07:35:27.877050+0000","last_unstale":"2024-09-21T07:46:26.611594+0000","last_undegraded":"2024-09-21T07:46:26.611594+0000","last_fullsized":"2024-09-21T07:46:26.611594+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:36.268334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":230,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599007+0000","last_change":"2024-09-21T07:35:28.862273+0000","last_active":"2024-09-21T07:46:27.599007+0000","last_peered":"2024-09-21T07:46:27.599007+0000","last_clean":"2024-09-21T07:46:27.599007+0000","last_became_active":"2024-09-21T07:35:26.874011+0000","last_became_peered":"2024-09-21T07:35:26.874011+0000","last_unstale":"2024-09-21T07:46:27.599007+0000","last_undegraded":"2024-09-21T07:46:27.599007+0000","last_fullsized":"2024-09-21T07:46:27.599007+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:11.986950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071644799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":284,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.600814+0000","last_change":"2024-09-21T07:34:24.259794+0000","last_active":"2024-09-21T07:46:27.600814+0000","last_peered":"2024-09-21T07:46:27.600814+0000","last_clean":"2024-09-21T07:46:27.600814+0000","last_became_active":"2024-09-21T07:34:23.265679+0000","last_became_peered":"2024-09-21T07:34:23.265679+0000","last_unstale":"2024-09-21T07:46:27.600814+0000","last_undegraded":"2024-09-21T07:46:27.600814+0000","last_fullsized":"2024-09-21T07:46:27.600814+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:02.575796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001298518,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.618643+0000","last_change":"2024-09-21T07:46:26.613402+0000","last_active":"2024-09-21T07:46:26.618643+0000","last_peered":"2024-09-21T07:46:26.618643+0000","last_clean":"2024-09-21T07:46:26.618643+0000","last_became_active":"2024-09-21T07:46:26.611549+0000","last_became_peered":"2024-09-21T07:46:26.611549+0000","last_unstale":"2024-09-21T07:46:26.618643+0000","last_undegraded":"2024-09-21T07:46:26.618643+0000","last_fullsized":"2024-09-21T07:46:26.618643+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:01.491090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"175'1","reported_seq":16,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.603115+0000","last_change":"2024-09-21T07:46:25.606263+0000","last_active":"2024-09-21T07:46:27.603115+0000","last_peered":"2024-09-21T07:46:27.603115+0000","last_clean":"2024-09-21T07:46:27.603115+0000","last_became_active":"2024-09-21T07:46:25.605327+0000","last_became_peered":"2024-09-21T07:46:25.605327+0000","last_unstale":"2024-09-21T07:46:27.603115+0000","last_undegraded":"2024-09-21T07:46:27.603115+0000","last_fullsized":"2024-09-21T07:46:27.603115+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:28.394690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":224,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.606690+0000","last_change":"2024-09-21T07:35:27.877725+0000","last_active":"2024-09-21T07:46:26.606690+0000","last_peered":"2024-09-21T07:46:26.606690+0000","last_clean":"2024-09-21T07:46:26.606690+0000","last_became_active":"2024-09-21T07:35:27.877398+0000","last_became_peered":"2024-09-21T07:35:27.877398+0000","last_unstale":"2024-09-21T07:46:26.606690+0000","last_undegraded":"2024-09-21T07:46:26.606690+0000","last_fullsized":"2024-09-21T07:46:26.606690+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:05.859762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":231,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.600206+0000","last_change":"2024-09-21T07:35:28.860612+0000","last_active":"2024-09-21T07:46:27.600206+0000","last_peered":"2024-09-21T07:46:27.600206+0000","last_clean":"2024-09-21T07:46:27.600206+0000","last_became_active":"2024-09-21T07:35:26.876562+0000","last_became_peered":"2024-09-21T07:35:26.876562+0000","last_unstale":"2024-09-21T07:46:27.600206+0000","last_undegraded":"2024-09-21T07:46:27.600206+0000","last_fullsized":"2024-09-21T07:46:27.600206+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:06.824331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012252470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":285,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.605010+0000","last_change":"2024-09-21T07:34:24.260330+0000","last_active":"2024-09-21T07:46:27.605010+0000","last_peered":"2024-09-21T07:46:27.605010+0000","last_clean":"2024-09-21T07:46:27.605010+0000","last_became_active":"2024-09-21T07:34:23.637210+0000","last_became_peered":"2024-09-21T07:34:23.637210+0000","last_unstale":"2024-09-21T07:46:27.605010+0000","last_undegraded":"2024-09-21T07:46:27.605010+0000","last_fullsized":"2024-09-21T07:46:27.605010+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.488451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001537093,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.603987+0000","last_change":"2024-09-21T07:46:26.614288+0000","last_active":"2024-09-21T07:46:27.603987+0000","last_peered":"2024-09-21T07:46:27.603987+0000","last_clean":"2024-09-21T07:46:27.603987+0000","last_became_active":"2024-09-21T07:46:26.613851+0000","last_became_peered":"2024-09-21T07:46:26.613851+0000","last_unstale":"2024-09-21T07:46:27.603987+0000","last_undegraded":"2024-09-21T07:46:27.603987+0000","last_fullsized":"2024-09-21T07:46:27.603987+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:39.078877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":15,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599234+0000","last_change":"2024-09-21T07:46:25.598295+0000","last_active":"2024-09-21T07:46:27.599234+0000","last_peered":"2024-09-21T07:46:27.599234+0000","last_clean":"2024-09-21T07:46:27.599234+0000","last_became_active":"2024-09-21T07:46:25.596166+0000","last_became_peered":"2024-09-21T07:46:25.596166+0000","last_unstale":"2024-09-21T07:46:27.599234+0000","last_undegraded":"2024-09-21T07:46:27.599234+0000","last_fullsized":"2024-09-21T07:46:27.599234+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:10.207013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":223,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.598267+0000","last_change":"2024-09-21T07:35:27.874757+0000","last_active":"2024-09-21T07:46:27.598267+0000","last_peered":"2024-09-21T07:46:27.598267+0000","last_clean":"2024-09-21T07:46:27.598267+0000","last_became_active":"2024-09-21T07:35:27.874360+0000","last_became_peered":"2024-09-21T07:35:27.874360+0000","last_unstale":"2024-09-21T07:46:27.598267+0000","last_undegraded":"2024-09-21T07:46:27.598267+0000","last_fullsized":"2024-09-21T07:46:27.598267+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:17.646912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":231,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.605022+0000","last_change":"2024-09-21T07:35:29.493431+0000","last_active":"2024-09-21T07:46:27.605022+0000","last_peered":"2024-09-21T07:46:27.605022+0000","last_clean":"2024-09-21T07:46:27.605022+0000","last_became_active":"2024-09-21T07:35:26.873691+0000","last_became_peered":"2024-09-21T07:35:26.873691+0000","last_unstale":"2024-09-21T07:46:27.605022+0000","last_undegraded":"2024-09-21T07:46:27.605022+0000","last_fullsized":"2024-09-21T07:46:27.605022+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:51.705051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001162464,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":275,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.607966+0000","last_change":"2024-09-21T07:34:24.597286+0000","last_active":"2024-09-21T07:46:26.607966+0000","last_peered":"2024-09-21T07:46:26.607966+0000","last_clean":"2024-09-21T07:46:26.607966+0000","last_became_active":"2024-09-21T07:34:23.267133+0000","last_became_peered":"2024-09-21T07:34:23.267133+0000","last_unstale":"2024-09-21T07:46:26.607966+0000","last_undegraded":"2024-09-21T07:46:26.607966+0000","last_fullsized":"2024-09-21T07:46:26.607966+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:18.257010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.623910+0000","last_change":"2024-09-21T07:46:26.615395+0000","last_active":"2024-09-21T07:46:26.623910+0000","last_peered":"2024-09-21T07:46:26.623910+0000","last_clean":"2024-09-21T07:46:26.623910+0000","last_became_active":"2024-09-21T07:46:26.614999+0000","last_became_peered":"2024-09-21T07:46:26.614999+0000","last_unstale":"2024-09-21T07:46:26.623910+0000","last_undegraded":"2024-09-21T07:46:26.623910+0000","last_fullsized":"2024-09-21T07:46:26.623910+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:44.279162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":13,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.605011+0000","last_change":"2024-09-21T07:46:25.594840+0000","last_active":"2024-09-21T07:46:26.605011+0000","last_peered":"2024-09-21T07:46:26.605011+0000","last_clean":"2024-09-21T07:46:26.605011+0000","last_became_active":"2024-09-21T07:46:25.594415+0000","last_became_peered":"2024-09-21T07:46:25.594415+0000","last_unstale":"2024-09-21T07:46:26.605011+0000","last_undegraded":"2024-09-21T07:46:26.605011+0000","last_fullsized":"2024-09-21T07:46:26.605011+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:49.617381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":231,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599772+0000","last_change":"2024-09-21T07:35:27.864458+0000","last_active":"2024-09-21T07:46:27.599772+0000","last_peered":"2024-09-21T07:46:27.599772+0000","last_clean":"2024-09-21T07:46:27.599772+0000","last_became_active":"2024-09-21T07:35:27.864136+0000","last_became_peered":"2024-09-21T07:35:27.864136+0000","last_unstale":"2024-09-21T07:46:27.599772+0000","last_undegraded":"2024-09-21T07:46:27.599772+0000","last_fullsized":"2024-09-21T07:46:27.599772+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:20.162240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":238,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.601274+0000","last_change":"2024-09-21T07:35:28.858784+0000","last_active":"2024-09-21T07:46:27.601274+0000","last_peered":"2024-09-21T07:46:27.601274+0000","last_clean":"2024-09-21T07:46:27.601274+0000","last_became_active":"2024-09-21T07:35:26.876890+0000","last_became_peered":"2024-09-21T07:35:26.876890+0000","last_unstale":"2024-09-21T07:46:27.601274+0000","last_undegraded":"2024-09-21T07:46:27.601274+0000","last_fullsized":"2024-09-21T07:46:27.601274+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:40.738327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064543499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":283,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.590547+0000","last_change":"2024-09-21T07:34:24.258712+0000","last_active":"2024-09-21T07:46:26.590547+0000","last_peered":"2024-09-21T07:46:26.590547+0000","last_clean":"2024-09-21T07:46:26.590547+0000","last_became_active":"2024-09-21T07:34:23.274692+0000","last_became_peered":"2024-09-21T07:34:23.274692+0000","last_unstale":"2024-09-21T07:46:26.590547+0000","last_undegraded":"2024-09-21T07:46:26.590547+0000","last_fullsized":"2024-09-21T07:46:26.590547+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:43.279454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018051479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.603775+0000","last_change":"2024-09-21T07:46:26.607882+0000","last_active":"2024-09-21T07:46:27.603775+0000","last_peered":"2024-09-21T07:46:27.603775+0000","last_clean":"2024-09-21T07:46:27.603775+0000","last_became_active":"2024-09-21T07:46:26.607510+0000","last_became_peered":"2024-09-21T07:46:26.607510+0000","last_unstale":"2024-09-21T07:46:27.603775+0000","last_undegraded":"2024-09-21T07:46:27.603775+0000","last_fullsized":"2024-09-21T07:46:27.603775+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:26:07.311520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"174'2","reported_seq":17,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599128+0000","last_change":"2024-09-21T07:46:25.603718+0000","last_active":"2024-09-21T07:46:27.599128+0000","last_peered":"2024-09-21T07:46:27.599128+0000","last_clean":"2024-09-21T07:46:27.599128+0000","last_became_active":"2024-09-21T07:46:25.603388+0000","last_became_peered":"2024-09-21T07:46:25.603388+0000","last_unstale":"2024-09-21T07:46:27.599128+0000","last_undegraded":"2024-09-21T07:46:27.599128+0000","last_fullsized":"2024-09-21T07:46:27.599128+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:43:18.533795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":222,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.588528+0000","last_change":"2024-09-21T07:35:27.878020+0000","last_active":"2024-09-21T07:46:26.588528+0000","last_peered":"2024-09-21T07:46:26.588528+0000","last_clean":"2024-09-21T07:46:26.588528+0000","last_became_active":"2024-09-21T07:35:27.877532+0000","last_became_peered":"2024-09-21T07:35:27.877532+0000","last_unstale":"2024-09-21T07:46:26.588528+0000","last_undegraded":"2024-09-21T07:46:26.588528+0000","last_fullsized":"2024-09-21T07:46:26.588528+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:19.756073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":231,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.601177+0000","last_change":"2024-09-21T07:35:28.861049+0000","last_active":"2024-09-21T07:46:27.601177+0000","last_peered":"2024-09-21T07:46:27.601177+0000","last_clean":"2024-09-21T07:46:27.601177+0000","last_became_active":"2024-09-21T07:35:26.876819+0000","last_became_peered":"2024-09-21T07:35:26.876819+0000","last_unstale":"2024-09-21T07:46:27.601177+0000","last_undegraded":"2024-09-21T07:46:27.601177+0000","last_fullsized":"2024-09-21T07:46:27.601177+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:19.633803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055780599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":282,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.598386+0000","last_change":"2024-09-21T07:34:25.084849+0000","last_active":"2024-09-21T07:46:27.598386+0000","last_peered":"2024-09-21T07:46:27.598386+0000","last_clean":"2024-09-21T07:46:27.598386+0000","last_became_active":"2024-09-21T07:34:23.277713+0000","last_became_peered":"2024-09-21T07:34:23.277713+0000","last_unstale":"2024-09-21T07:46:27.598386+0000","last_undegraded":"2024-09-21T07:46:27.598386+0000","last_fullsized":"2024-09-21T07:46:27.598386+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:05.811810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059079499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":10,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.592232+0000","last_change":"2024-09-21T07:46:26.591457+0000","last_active":"2024-09-21T07:46:26.592232+0000","last_peered":"2024-09-21T07:46:26.592232+0000","last_clean":"2024-09-21T07:46:26.592232+0000","last_became_active":"2024-09-21T07:46:26.591098+0000","last_became_peered":"2024-09-21T07:46:26.591098+0000","last_unstale":"2024-09-21T07:46:26.592232+0000","last_undegraded":"2024-09-21T07:46:26.592232+0000","last_fullsized":"2024-09-21T07:46:26.592232+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:44.374583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":13,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.607171+0000","last_change":"2024-09-21T07:46:25.596493+0000","last_active":"2024-09-21T07:46:26.607171+0000","last_peered":"2024-09-21T07:46:26.607171+0000","last_clean":"2024-09-21T07:46:26.607171+0000","last_became_active":"2024-09-21T07:46:25.593838+0000","last_became_peered":"2024-09-21T07:46:25.593838+0000","last_unstale":"2024-09-21T07:46:26.607171+0000","last_undegraded":"2024-09-21T07:46:26.607171+0000","last_fullsized":"2024-09-21T07:46:26.607171+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:39:08.550666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":224,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.582290+0000","last_change":"2024-09-21T07:35:27.884248+0000","last_active":"2024-09-21T07:46:26.582290+0000","last_peered":"2024-09-21T07:46:26.582290+0000","last_clean":"2024-09-21T07:46:26.582290+0000","last_became_active":"2024-09-21T07:35:27.883861+0000","last_became_peered":"2024-09-21T07:35:27.883861+0000","last_unstale":"2024-09-21T07:46:26.582290+0000","last_undegraded":"2024-09-21T07:46:26.582290+0000","last_fullsized":"2024-09-21T07:46:26.582290+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":230,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.608256+0000","last_change":"2024-09-21T07:35:29.207135+0000","last_active":"2024-09-21T07:46:26.608256+0000","last_peered":"2024-09-21T07:46:26.608256+0000","last_clean":"2024-09-21T07:46:26.608256+0000","last_became_active":"2024-09-21T07:35:26.872787+0000","last_became_peered":"2024-09-21T07:35:26.872787+0000","last_unstale":"2024-09-21T07:46:26.608256+0000","last_undegraded":"2024-09-21T07:46:26.608256+0000","last_fullsized":"2024-09-21T07:46:26.608256+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:27.543217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011157509999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":291,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.600051+0000","last_change":"2024-09-21T07:34:24.538610+0000","last_active":"2024-09-21T07:46:27.600051+0000","last_peered":"2024-09-21T07:46:27.600051+0000","last_clean":"2024-09-21T07:46:27.600051+0000","last_became_active":"2024-09-21T07:34:23.259700+0000","last_became_peered":"2024-09-21T07:34:23.259700+0000","last_unstale":"2024-09-21T07:46:27.600051+0000","last_undegraded":"2024-09-21T07:46:27.600051+0000","last_fullsized":"2024-09-21T07:46:27.600051+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:57.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070522399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.600915+0000","last_change":"2024-09-21T07:46:26.591714+0000","last_active":"2024-09-21T07:46:27.600915+0000","last_peered":"2024-09-21T07:46:27.600915+0000","last_clean":"2024-09-21T07:46:27.600915+0000","last_became_active":"2024-09-21T07:46:26.591384+0000","last_became_peered":"2024-09-21T07:46:26.591384+0000","last_unstale":"2024-09-21T07:46:27.600915+0000","last_undegraded":"2024-09-21T07:46:27.600915+0000","last_fullsized":"2024-09-21T07:46:27.600915+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:01:25.928166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"174'1","reported_seq":14,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.630143+0000","last_change":"2024-09-21T07:46:25.600252+0000","last_active":"2024-09-21T07:46:26.630143+0000","last_peered":"2024-09-21T07:46:26.630143+0000","last_clean":"2024-09-21T07:46:26.630143+0000","last_became_active":"2024-09-21T07:46:25.599931+0000","last_became_peered":"2024-09-21T07:46:25.599931+0000","last_unstale":"2024-09-21T07:46:26.630143+0000","last_undegraded":"2024-09-21T07:46:26.630143+0000","last_fullsized":"2024-09-21T07:46:26.630143+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:38.070437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":231,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599790+0000","last_change":"2024-09-21T07:35:27.868024+0000","last_active":"2024-09-21T07:46:27.599790+0000","last_peered":"2024-09-21T07:46:27.599790+0000","last_clean":"2024-09-21T07:46:27.599790+0000","last_became_active":"2024-09-21T07:35:27.867339+0000","last_became_peered":"2024-09-21T07:35:27.867339+0000","last_unstale":"2024-09-21T07:46:27.599790+0000","last_undegraded":"2024-09-21T07:46:27.599790+0000","last_fullsized":"2024-09-21T07:46:27.599790+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":238,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.599904+0000","last_change":"2024-09-21T07:35:28.858505+0000","last_active":"2024-09-21T07:46:27.599904+0000","last_peered":"2024-09-21T07:46:27.599904+0000","last_clean":"2024-09-21T07:46:27.599904+0000","last_became_active":"2024-09-21T07:35:26.877320+0000","last_became_peered":"2024-09-21T07:35:26.877320+0000","last_unstale":"2024-09-21T07:46:27.599904+0000","last_undegraded":"2024-09-21T07:46:27.599904+0000","last_fullsized":"2024-09-21T07:46:27.599904+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:53.672782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244119999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":277,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.615821+0000","last_change":"2024-09-21T07:34:24.597597+0000","last_active":"2024-09-21T07:46:26.615821+0000","last_peered":"2024-09-21T07:46:26.615821+0000","last_clean":"2024-09-21T07:46:26.615821+0000","last_became_active":"2024-09-21T07:34:23.635204+0000","last_became_peered":"2024-09-21T07:34:23.635204+0000","last_unstale":"2024-09-21T07:46:26.615821+0000","last_undegraded":"2024-09-21T07:46:26.615821+0000","last_fullsized":"2024-09-21T07:46:26.615821+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:48.644789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081139199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.601188+0000","last_change":"2024-09-21T07:46:26.607268+0000","last_active":"2024-09-21T07:46:27.601188+0000","last_peered":"2024-09-21T07:46:27.601188+0000","last_clean":"2024-09-21T07:46:27.601188+0000","last_became_active":"2024-09-21T07:46:26.606937+0000","last_became_peered":"2024-09-21T07:46:26.606937+0000","last_unstale":"2024-09-21T07:46:27.601188+0000","last_undegraded":"2024-09-21T07:46:27.601188+0000","last_fullsized":"2024-09-21T07:46:27.601188+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:49:37.916811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":13,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.608932+0000","last_change":"2024-09-21T07:46:25.603927+0000","last_active":"2024-09-21T07:46:26.608932+0000","last_peered":"2024-09-21T07:46:26.608932+0000","last_clean":"2024-09-21T07:46:26.608932+0000","last_became_active":"2024-09-21T07:46:25.603544+0000","last_became_peered":"2024-09-21T07:46:25.603544+0000","last_unstale":"2024-09-21T07:46:26.608932+0000","last_undegraded":"2024-09-21T07:46:26.608932+0000","last_fullsized":"2024-09-21T07:46:26.608932+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:02.105173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":223,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.606701+0000","last_change":"2024-09-21T07:35:27.877503+0000","last_active":"2024-09-21T07:46:26.606701+0000","last_peered":"2024-09-21T07:46:26.606701+0000","last_clean":"2024-09-21T07:46:26.606701+0000","last_became_active":"2024-09-21T07:35:27.875775+0000","last_became_peered":"2024-09-21T07:35:27.875775+0000","last_unstale":"2024-09-21T07:46:26.606701+0000","last_undegraded":"2024-09-21T07:46:26.606701+0000","last_fullsized":"2024-09-21T07:46:26.606701+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:49.291023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":231,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.603889+0000","last_change":"2024-09-21T07:35:29.491412+0000","last_active":"2024-09-21T07:46:27.603889+0000","last_peered":"2024-09-21T07:46:27.603889+0000","last_clean":"2024-09-21T07:46:27.603889+0000","last_became_active":"2024-09-21T07:35:26.876404+0000","last_became_peered":"2024-09-21T07:35:26.876404+0000","last_unstale":"2024-09-21T07:46:27.603889+0000","last_undegraded":"2024-09-21T07:46:27.603889+0000","last_fullsized":"2024-09-21T07:46:27.603889+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063918000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":284,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.590052+0000","last_change":"2024-09-21T07:34:24.259026+0000","last_active":"2024-09-21T07:46:26.590052+0000","last_peered":"2024-09-21T07:46:26.590052+0000","last_clean":"2024-09-21T07:46:26.590052+0000","last_became_active":"2024-09-21T07:34:23.635456+0000","last_became_peered":"2024-09-21T07:34:23.635456+0000","last_unstale":"2024-09-21T07:46:26.590052+0000","last_undegraded":"2024-09-21T07:46:26.590052+0000","last_fullsized":"2024-09-21T07:46:26.590052+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:09.592752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018654419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.604683+0000","last_change":"2024-09-21T07:46:26.614197+0000","last_active":"2024-09-21T07:46:27.604683+0000","last_peered":"2024-09-21T07:46:27.604683+0000","last_clean":"2024-09-21T07:46:27.604683+0000","last_became_active":"2024-09-21T07:46:26.613885+0000","last_became_peered":"2024-09-21T07:46:26.613885+0000","last_unstale":"2024-09-21T07:46:27.604683+0000","last_undegraded":"2024-09-21T07:46:27.604683+0000","last_fullsized":"2024-09-21T07:46:27.604683+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:51:31.780109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"174'1","reported_seq":16,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.600953+0000","last_change":"2024-09-21T07:46:25.600533+0000","last_active":"2024-09-21T07:46:27.600953+0000","last_peered":"2024-09-21T07:46:27.600953+0000","last_clean":"2024-09-21T07:46:27.600953+0000","last_became_active":"2024-09-21T07:46:25.600183+0000","last_became_peered":"2024-09-21T07:46:25.600183+0000","last_unstale":"2024-09-21T07:46:27.600953+0000","last_undegraded":"2024-09-21T07:46:27.600953+0000","last_fullsized":"2024-09-21T07:46:27.600953+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:48.453281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":223,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.606099+0000","last_change":"2024-09-21T07:35:27.871984+0000","last_active":"2024-09-21T07:46:26.606099+0000","last_peered":"2024-09-21T07:46:26.606099+0000","last_clean":"2024-09-21T07:46:26.606099+0000","last_became_active":"2024-09-21T07:35:27.870336+0000","last_became_peered":"2024-09-21T07:35:27.870336+0000","last_unstale":"2024-09-21T07:46:26.606099+0000","last_undegraded":"2024-09-21T07:46:26.606099+0000","last_fullsized":"2024-09-21T07:46:26.606099+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:57.120479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":231,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.600376+0000","last_change":"2024-09-21T07:35:28.860351+0000","last_active":"2024-09-21T07:46:27.600376+0000","last_peered":"2024-09-21T07:46:27.600376+0000","last_clean":"2024-09-21T07:46:27.600376+0000","last_became_active":"2024-09-21T07:35:26.872046+0000","last_became_peered":"2024-09-21T07:35:26.872046+0000","last_unstale":"2024-09-21T07:46:27.600376+0000","last_undegraded":"2024-09-21T07:46:27.600376+0000","last_fullsized":"2024-09-21T07:46:27.600376+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:47.074327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098384399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":284,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.604442+0000","last_change":"2024-09-21T07:34:24.260300+0000","last_active":"2024-09-21T07:46:27.604442+0000","last_peered":"2024-09-21T07:46:27.604442+0000","last_clean":"2024-09-21T07:46:27.604442+0000","last_became_active":"2024-09-21T07:34:23.270128+0000","last_became_peered":"2024-09-21T07:34:23.270128+0000","last_unstale":"2024-09-21T07:46:27.604442+0000","last_undegraded":"2024-09-21T07:46:27.604442+0000","last_fullsized":"2024-09-21T07:46:27.604442+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:25.260931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00176651,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":13,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.604878+0000","last_change":"2024-09-21T07:46:26.624447+0000","last_active":"2024-09-21T07:46:27.604878+0000","last_peered":"2024-09-21T07:46:27.604878+0000","last_clean":"2024-09-21T07:46:27.604878+0000","last_became_active":"2024-09-21T07:46:26.624148+0000","last_became_peered":"2024-09-21T07:46:26.624148+0000","last_unstale":"2024-09-21T07:46:27.604878+0000","last_undegraded":"2024-09-21T07:46:27.604878+0000","last_fullsized":"2024-09-21T07:46:27.604878+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":173,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:25.565090+0000","last_clean_scrub_stamp":"2024-09-21T07:46:25.565090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:20.298916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"174'2","reported_seq":17,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.604608+0000","last_change":"2024-09-21T07:46:25.600776+0000","last_active":"2024-09-21T07:46:27.604608+0000","last_peered":"2024-09-21T07:46:27.604608+0000","last_clean":"2024-09-21T07:46:27.604608+0000","last_became_active":"2024-09-21T07:46:25.600433+0000","last_became_peered":"2024-09-21T07:46:25.600433+0000","last_unstale":"2024-09-21T07:46:27.604608+0000","last_undegraded":"2024-09-21T07:46:27.604608+0000","last_fullsized":"2024-09-21T07:46:27.604608+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":172,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:24.556650+0000","last_clean_scrub_stamp":"2024-09-21T07:46:24.556650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:51:31.164111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":223,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.606806+0000","last_change":"2024-09-21T07:35:27.877861+0000","last_active":"2024-09-21T07:46:26.606806+0000","last_peered":"2024-09-21T07:46:26.606806+0000","last_clean":"2024-09-21T07:46:26.606806+0000","last_became_active":"2024-09-21T07:35:27.877434+0000","last_became_peered":"2024-09-21T07:35:27.877434+0000","last_unstale":"2024-09-21T07:46:26.606806+0000","last_undegraded":"2024-09-21T07:46:26.606806+0000","last_fullsized":"2024-09-21T07:46:26.606806+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":230,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-21T07:46:27.598861+0000","last_change":"2024-09-21T07:35:28.862954+0000","last_active":"2024-09-21T07:46:27.598861+0000","last_peered":"2024-09-21T07:46:27.598861+0000","last_clean":"2024-09-21T07:46:27.598861+0000","last_became_active":"2024-09-21T07:35:26.870613+0000","last_became_peered":"2024-09-21T07:35:26.870613+0000","last_unstale":"2024-09-21T07:46:27.598861+0000","last_undegraded":"2024-09-21T07:46:27.598861+0000","last_fullsized":"2024-09-21T07:46:27.598861+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:06.901043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059842100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":282,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-21T07:46:26.583019+0000","last_change":"2024-09-21T07:34:25.028904+0000","last_active":"2024-09-21T07:46:26.583019+0000","last_peered":"2024-09-21T07:46:26.583019+0000","last_clean":"2024-09-21T07:46:26.583019+0000","last_became_active":"2024-09-21T07:34:23.260942+0000","last_became_peered":"2024-09-21T07:34:23.260942+0000","last_unstale":"2024-09-21T07:46:26.583019+0000","last_undegraded":"2024-09-21T07:46:26.583019+0000","last_fullsized":"2024-09-21T07:46:26.583019+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:22.709328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060553899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509604,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42224,"kb_used_data":2712,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329616,"statfs":{"total":96636764160,"available":96593526784,"internally_reserved":0,"allocated":2777088,"data_stored":1740743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509606,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41648,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330192,"statfs":{"total":96636764160,"available":96594116608,"internally_reserved":0,"allocated":2183168,"data_stored":1150423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509604,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41644,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330196,"statfs":{"total":96636764160,"available":96594120704,"internally_reserved":0,"allocated":2179072,"data_stored":1150353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509605,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41652,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330188,"statfs":{"total":96636764160,"available":96594112512,"internally_reserved":0,"allocated":2187264,"data_stored":1150465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509606,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41648,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330192,"statfs":{"total":96636764160,"available":96594116608,"internally_reserved":0,"allocated":2183168,"data_stored":1150423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509606,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42244,"kb_used_data":2732,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329596,"statfs":{"total":96636764160,"available":96593506304,"internally_reserved":0,"allocated":2797568,"data_stored":1743039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542309,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41628,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330212,"statfs":{"total":96636764160,"available":96594137088,"internally_reserved":0,"allocated":2166784,"data_stored":1142399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542308,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29144,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342696,"statfs":{"total":96636764160,"available":96606920704,"internally_reserved":0,"allocated":2166784,"data_stored":1142399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:46:31.613 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T07:46:31.613 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T07:46:31.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.878+0000 7f9f59c00640 1 -- 172.21.15.39:0/3836989318 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f54160250 msgr2=0x7f9f5415d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:31.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.878+0000 7f9f59c00640 1 --2- 172.21.15.39:0/3836989318 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f54160250 0x7f9f5415d570 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9f44009830 tx=0x7f9f4402a510 comp rx=0 tx=0).stop 2024-09-21T07:46:31.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.878+0000 7f9f59c00640 1 -- 172.21.15.39:0/3836989318 shutdown_connections 2024-09-21T07:46:31.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.878+0000 7f9f59c00640 1 --2- 172.21.15.39:0/3836989318 >> v2:172.21.15.148:3300/0 conn(0x7f9f54160660 0x7f9f540cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:31.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.878+0000 7f9f59c00640 1 --2- 172.21.15.39:0/3836989318 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f54160250 0x7f9f5415d570 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:31.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.878+0000 7f9f59c00640 1 -- 172.21.15.39:0/3836989318 >> 172.21.15.39:0/3836989318 conn(0x7f9f5406cfc0 msgr2=0x7f9f5406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:31.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.878+0000 7f9f59c00640 1 -- 172.21.15.39:0/3836989318 shutdown_connections 2024-09-21T07:46:31.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.878+0000 7f9f59c00640 1 -- 172.21.15.39:0/3836989318 wait complete. 2024-09-21T07:46:31.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.880+0000 7f9f59c00640 1 Processor -- start 2024-09-21T07:46:31.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.880+0000 7f9f59c00640 1 -- start start 2024-09-21T07:46:31.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.882+0000 7f9f59c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f540cfc50 0x7f9f540b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:31.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.882+0000 7f9f59c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f54160250 0x7f9f5413baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:31.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.882+0000 7f9f59c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f54160660 0x7f9f540b2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:31.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.882+0000 7f9f59c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9f540cce00 con 0x7f9f54160660 2024-09-21T07:46:31.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.882+0000 7f9f59c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9f540ccc80 con 0x7f9f54160250 2024-09-21T07:46:31.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.882+0000 7f9f59c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f9f540ccf80 con 0x7f9f540cfc50 2024-09-21T07:46:31.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.882+0000 7f9f59200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f54160660 0x7f9f540b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:31.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.882+0000 7f9f59200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f54160660 0x7f9f540b2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35618/0 (socket says 172.21.15.39:35618) 2024-09-21T07:46:31.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.882+0000 7f9f59200640 1 -- 172.21.15.39:0/805478525 learned_addr learned my addr 172.21.15.39:0/805478525 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:31.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.882+0000 7f9f4be00640 1 --2- 172.21.15.39:0/805478525 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f54160250 0x7f9f5413baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:31.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.882+0000 7f9f53e00640 1 --2- 172.21.15.39:0/805478525 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f540cfc50 0x7f9f540b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:31.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.883+0000 7f9f4be00640 1 -- 172.21.15.39:0/805478525 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f540cfc50 msgr2=0x7f9f540b4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:31.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.883+0000 7f9f4be00640 1 --2- 172.21.15.39:0/805478525 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f540cfc50 0x7f9f540b4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:31.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.883+0000 7f9f4be00640 1 -- 172.21.15.39:0/805478525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f54160660 msgr2=0x7f9f540b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:31.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.883+0000 7f9f4be00640 1 --2- 172.21.15.39:0/805478525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f54160660 0x7f9f540b2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:31.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.883+0000 7f9f4be00640 1 -- 172.21.15.39:0/805478525 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f540b27d0 con 0x7f9f54160250 2024-09-21T07:46:31.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.883+0000 7f9f59200640 1 --2- 172.21.15.39:0/805478525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f54160660 0x7f9f540b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:46:31.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.883+0000 7f9f4be00640 1 --2- 172.21.15.39:0/805478525 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f54160250 0x7f9f5413baa0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c00e6c0 tx=0x7f9f3c00eb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:31.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.883+0000 7f9f50c00640 1 -- 172.21.15.39:0/805478525 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c004220 con 0x7f9f54160250 2024-09-21T07:46:31.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.884+0000 7f9f59c00640 1 -- 172.21.15.39:0/805478525 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f540b3d00 con 0x7f9f54160250 2024-09-21T07:46:31.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.884+0000 7f9f50c00640 1 -- 172.21.15.39:0/805478525 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f3c00c8b0 con 0x7f9f54160250 2024-09-21T07:46:31.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.884+0000 7f9f50c00640 1 -- 172.21.15.39:0/805478525 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c00cd00 con 0x7f9f54160250 2024-09-21T07:46:31.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.884+0000 7f9f59c00640 1 -- 172.21.15.39:0/805478525 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f540b42c0 con 0x7f9f54160250 2024-09-21T07:46:31.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.885+0000 7f9f59c00640 1 -- 172.21.15.39:0/805478525 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f540cd430 con 0x7f9f54160250 2024-09-21T07:46:31.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.886+0000 7f9f50c00640 1 -- 172.21.15.39:0/805478525 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9f3c0035f0 con 0x7f9f54160250 2024-09-21T07:46:31.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.887+0000 7f9f50c00640 1 --2- 172.21.15.39:0/805478525 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9f30078b90 0x7f9f3007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:31.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.887+0000 7f9f50c00640 1 -- 172.21.15.39:0/805478525 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 6888+0+0 (secure 0 0 0) 0x7f9f3c09b6e0 con 0x7f9f54160250 2024-09-21T07:46:31.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.888+0000 7f9f53e00640 1 --2- 172.21.15.39:0/805478525 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9f30078b90 0x7f9f3007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:31.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.889+0000 7f9f53e00640 1 --2- 172.21.15.39:0/805478525 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9f30078b90 0x7f9f3007b0a0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f9f44034000 tx=0x7f9f44004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:31.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:31.894+0000 7f9f50c00640 1 -- 172.21.15.39:0/805478525 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9f3c063030 con 0x7f9f54160250 2024-09-21T07:46:32.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.169+0000 7f9f59c00640 1 -- 172.21.15.39:0/805478525 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f9f540b2a70 con 0x7f9f54160250 2024-09-21T07:46:32.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.177+0000 7f9f50c00640 1 -- 172.21.15.39:0/805478525 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) ==== 190+0+147 (secure 0 0 0) 0x7f9f3c067a60 con 0x7f9f54160250 2024-09-21T07:46:32.179 INFO:teuthology.orchestra.run.smithi039.stdout:[client.0] 2024-09-21T07:46:32.179 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQB0du5mGfb1DBAA/aSMKR5or3xy53rB/iUjpA== 2024-09-21T07:46:32.179 INFO:teuthology.orchestra.run.smithi039.stdout: caps mds = "allow" 2024-09-21T07:46:32.179 INFO:teuthology.orchestra.run.smithi039.stdout: caps mon = "allow r" 2024-09-21T07:46:32.179 INFO:teuthology.orchestra.run.smithi039.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T07:46:32.179 INFO:teuthology.orchestra.run.smithi039.stderr:updated caps for client.0 2024-09-21T07:46:32.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.179+0000 7f9f59c00640 1 -- 172.21.15.39:0/805478525 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9f30078b90 msgr2=0x7f9f3007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:32.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.179+0000 7f9f59c00640 1 --2- 172.21.15.39:0/805478525 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9f30078b90 0x7f9f3007b0a0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f9f44034000 tx=0x7f9f44004730 comp rx=0 tx=0).stop 2024-09-21T07:46:32.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.179+0000 7f9f59c00640 1 -- 172.21.15.39:0/805478525 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f54160250 msgr2=0x7f9f5413baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:32.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.179+0000 7f9f59c00640 1 --2- 172.21.15.39:0/805478525 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f54160250 0x7f9f5413baa0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c00e6c0 tx=0x7f9f3c00eb80 comp rx=0 tx=0).stop 2024-09-21T07:46:32.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.179+0000 7f9f59c00640 1 -- 172.21.15.39:0/805478525 shutdown_connections 2024-09-21T07:46:32.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.179+0000 7f9f59c00640 1 --2- 172.21.15.39:0/805478525 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9f30078b90 0x7f9f3007b0a0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:32.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.179+0000 7f9f59c00640 1 --2- 172.21.15.39:0/805478525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f54160660 0x7f9f540b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:32.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.179+0000 7f9f59c00640 1 --2- 172.21.15.39:0/805478525 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f54160250 0x7f9f5413baa0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:32.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.179+0000 7f9f59c00640 1 --2- 172.21.15.39:0/805478525 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f540cfc50 0x7f9f540b4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:32.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.179+0000 7f9f59c00640 1 -- 172.21.15.39:0/805478525 >> 172.21.15.39:0/805478525 conn(0x7f9f5406cfc0 msgr2=0x7f9f5406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:32.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.179+0000 7f9f59c00640 1 -- 172.21.15.39:0/805478525 shutdown_connections 2024-09-21T07:46:32.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.180+0000 7f9f59c00640 1 -- 172.21.15.39:0/805478525 wait complete. 2024-09-21T07:46:32.206 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:46:32.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.430+0000 7f5ea6200640 1 -- 172.21.15.39:0/3160525320 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ea00b30a0 msgr2=0x7f5ea00cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:32.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.430+0000 7f5ea6200640 1 --2- 172.21.15.39:0/3160525320 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ea00b30a0 0x7f5ea00cb8a0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0075160 tx=0x7f5e9002a580 comp rx=0 tx=0).stop 2024-09-21T07:46:32.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.430+0000 7f5ea6200640 1 -- 172.21.15.39:0/3160525320 shutdown_connections 2024-09-21T07:46:32.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.430+0000 7f5ea6200640 1 --2- 172.21.15.39:0/3160525320 >> v2:172.21.15.148:3300/0 conn(0x7f5ea015d830 0x7f5ea0156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:32.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.430+0000 7f5ea6200640 1 --2- 172.21.15.39:0/3160525320 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ea00b30a0 0x7f5ea00cb8a0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:32.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.430+0000 7f5ea6200640 1 -- 172.21.15.39:0/3160525320 >> 172.21.15.39:0/3160525320 conn(0x7f5ea00b2460 msgr2=0x7f5ea006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:32.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.430+0000 7f5ea6200640 1 -- 172.21.15.39:0/3160525320 shutdown_connections 2024-09-21T07:46:32.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.430+0000 7f5ea6200640 1 -- 172.21.15.39:0/3160525320 wait complete. 2024-09-21T07:46:32.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.432+0000 7f5ea6200640 1 Processor -- start 2024-09-21T07:46:32.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.432+0000 7f5ea6200640 1 -- start start 2024-09-21T07:46:32.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.434+0000 7f5ea6200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ea00b30a0 0x7f5ea00c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:32.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.434+0000 7f5ea6200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5ea00ccc60 0x7f5ea01f52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:32.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.434+0000 7f5ea6200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ea015d830 0x7f5ea00c6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:32.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.434+0000 7f5ea6200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea00af410 con 0x7f5ea015d830 2024-09-21T07:46:32.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.434+0000 7f5ea6200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea00af290 con 0x7f5ea00b30a0 2024-09-21T07:46:32.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.434+0000 7f5ea6200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f5ea00af590 con 0x7f5ea00ccc60 2024-09-21T07:46:32.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.434+0000 7f5e9fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ea015d830 0x7f5ea00c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:32.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.434+0000 7f5e9fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ea015d830 0x7f5ea00c6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38722/0 (socket says 172.21.15.39:38722) 2024-09-21T07:46:32.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.434+0000 7f5e9fe00640 1 -- 172.21.15.39:0/837691839 learned_addr learned my addr 172.21.15.39:0/837691839 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:32.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.434+0000 7f5e97e00640 1 --2- 172.21.15.39:0/837691839 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5ea00ccc60 0x7f5ea01f52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:32.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.434+0000 7f5e9f400640 1 --2- 172.21.15.39:0/837691839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ea00b30a0 0x7f5ea00c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:32.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.435+0000 7f5e9fe00640 1 -- 172.21.15.39:0/837691839 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5ea00ccc60 msgr2=0x7f5ea01f52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:32.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.435+0000 7f5e9fe00640 1 --2- 172.21.15.39:0/837691839 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5ea00ccc60 0x7f5ea01f52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:32.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.435+0000 7f5e9fe00640 1 -- 172.21.15.39:0/837691839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ea00b30a0 msgr2=0x7f5ea00c9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:32.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.435+0000 7f5e9fe00640 1 --2- 172.21.15.39:0/837691839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ea00b30a0 0x7f5ea00c9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:32.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.435+0000 7f5e9fe00640 1 -- 172.21.15.39:0/837691839 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea00c73a0 con 0x7f5ea015d830 2024-09-21T07:46:32.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.435+0000 7f5e9f400640 1 --2- 172.21.15.39:0/837691839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ea00b30a0 0x7f5ea00c9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:32.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.435+0000 7f5e9fe00640 1 --2- 172.21.15.39:0/837691839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ea015d830 0x7f5ea00c6180 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f5e9800c760 tx=0x7f5e9800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:32.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.435+0000 7f5e97400640 1 -- 172.21.15.39:0/837691839 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e98017070 con 0x7f5ea015d830 2024-09-21T07:46:32.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.435+0000 7f5e97400640 1 -- 172.21.15.39:0/837691839 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e9800ce10 con 0x7f5ea015d830 2024-09-21T07:46:32.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.435+0000 7f5ea6200640 1 -- 172.21.15.39:0/837691839 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ea00c7680 con 0x7f5ea015d830 2024-09-21T07:46:32.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.435+0000 7f5e97400640 1 -- 172.21.15.39:0/837691839 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e980052b0 con 0x7f5ea015d830 2024-09-21T07:46:32.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.435+0000 7f5ea6200640 1 -- 172.21.15.39:0/837691839 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ea00c66c0 con 0x7f5ea015d830 2024-09-21T07:46:32.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.436+0000 7f5ea6200640 1 -- 172.21.15.39:0/837691839 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ea0074f40 con 0x7f5ea015d830 2024-09-21T07:46:32.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.438+0000 7f5e97400640 1 -- 172.21.15.39:0/837691839 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5e9801a070 con 0x7f5ea015d830 2024-09-21T07:46:32.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.438+0000 7f5e97400640 1 --2- 172.21.15.39:0/837691839 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5e7c078b90 0x7f5e7c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:32.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.439+0000 7f5e97400640 1 -- 172.21.15.39:0/837691839 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 6888+0+0 (secure 0 0 0) 0x7f5e9809a800 con 0x7f5ea015d830 2024-09-21T07:46:32.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.439+0000 7f5e9f400640 1 --2- 172.21.15.39:0/837691839 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5e7c078b90 0x7f5e7c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:32.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.440+0000 7f5e9f400640 1 --2- 172.21.15.39:0/837691839 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5e7c078b90 0x7f5e7c07b0a0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5e90033000 tx=0x7f5e90004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:32.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.443+0000 7f5e97400640 1 -- 172.21.15.39:0/837691839 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5e98062790 con 0x7f5ea015d830 2024-09-21T07:46:32.561 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:46:32.565 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:46:32.566 INFO:tasks.ceph.mds.c.smithi039.stderr:2024-09-21T07:46:32.564+0000 7f5c87800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:46:32.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.681+0000 7f5ea6200640 1 -- 172.21.15.39:0/837691839 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ea00afa40 con 0x7f5ea015d830 2024-09-21T07:46:32.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.682+0000 7f5e97400640 1 -- 172.21.15.39:0/837691839 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 190 v190) ==== 95+0+5234 (secure 0 0 0) 0x7f5e98062090 con 0x7f5ea015d830 2024-09-21T07:46:32.684 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:46:32.684 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":190,"btime":"2024-09-21T07:46:29:608115+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7493,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/4129342618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4129342618},{"type":"v1","addr":"172.21.15.39:6834","nonce":4129342618}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":185},{"gid":7499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1125302960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1125302960},{"type":"v1","addr":"172.21.15.148:6836","nonce":1125302960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":185},{"gid":8431,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/585545787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":585545787},{"type":"v1","addr":"172.21.15.148:6837","nonce":585545787}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":185}],"filesystems":[{"mdsmap":{"epoch":190,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:46:26.100496+0000","modified":"2024-09-21T07:46:29.115340+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9180},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9180":{"gid":9180,"name":"c","rank":0,"incarnation":186,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/3000703195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3000703195},{"type":"v1","addr":"172.21.15.39:6835","nonce":3000703195}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9180,"qdb_cluster":[9180]},"id":9}]} 2024-09-21T07:46:32.684 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 190 2024-09-21T07:46:32.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.685+0000 7f5ea6200640 1 -- 172.21.15.39:0/837691839 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5e7c078b90 msgr2=0x7f5e7c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:32.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.685+0000 7f5ea6200640 1 --2- 172.21.15.39:0/837691839 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5e7c078b90 0x7f5e7c07b0a0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5e90033000 tx=0x7f5e90004730 comp rx=0 tx=0).stop 2024-09-21T07:46:32.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.685+0000 7f5ea6200640 1 -- 172.21.15.39:0/837691839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ea015d830 msgr2=0x7f5ea00c6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:32.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.685+0000 7f5ea6200640 1 --2- 172.21.15.39:0/837691839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ea015d830 0x7f5ea00c6180 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f5e9800c760 tx=0x7f5e9800cc20 comp rx=0 tx=0).stop 2024-09-21T07:46:32.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.685+0000 7f5ea6200640 1 -- 172.21.15.39:0/837691839 shutdown_connections 2024-09-21T07:46:32.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.685+0000 7f5ea6200640 1 --2- 172.21.15.39:0/837691839 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5e7c078b90 0x7f5e7c07b0a0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:32.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.685+0000 7f5ea6200640 1 --2- 172.21.15.39:0/837691839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ea015d830 0x7f5ea00c6180 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:32.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.685+0000 7f5ea6200640 1 --2- 172.21.15.39:0/837691839 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5ea00ccc60 0x7f5ea01f52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:32.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.685+0000 7f5ea6200640 1 --2- 172.21.15.39:0/837691839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ea00b30a0 0x7f5ea00c9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:32.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.685+0000 7f5ea6200640 1 -- 172.21.15.39:0/837691839 >> 172.21.15.39:0/837691839 conn(0x7f5ea00b2460 msgr2=0x7f5ea0068f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:32.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.685+0000 7f5ea6200640 1 -- 172.21.15.39:0/837691839 shutdown_connections 2024-09-21T07:46:32.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:32.685+0000 7f5ea6200640 1 -- 172.21.15.39:0/837691839 wait complete. 2024-09-21T07:46:33.113 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 190, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T07:46:26.100496+0000', 'modified': '2024-09-21T07:46:29.115340+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9180}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9180': {'gid': 9180, 'name': 'c', 'rank': 0, 'incarnation': 186, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.39:6835/3000703195', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 3000703195}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 3000703195}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9180, 'qdb_cluster': [9180]} 2024-09-21T07:46:33.113 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T07:46:33.113 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T07:46:33.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.342+0000 7fbe6f000640 1 -- 172.21.15.39:0/3417622468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe680cb8a0 msgr2=0x7fbe68156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:33.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.342+0000 7fbe6f000640 1 --2- 172.21.15.39:0/3417622468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe680cb8a0 0x7fbe68156bb0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fbe680cb3e0 tx=0x7fbe5402a560 comp rx=0 tx=0).stop 2024-09-21T07:46:33.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.342+0000 7fbe6f000640 1 -- 172.21.15.39:0/3417622468 shutdown_connections 2024-09-21T07:46:33.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.342+0000 7fbe6f000640 1 --2- 172.21.15.39:0/3417622468 >> v2:172.21.15.148:3300/0 conn(0x7fbe680ccc60 0x7fbe680cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.342+0000 7fbe6f000640 1 --2- 172.21.15.39:0/3417622468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe680cb8a0 0x7fbe68156bb0 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.342+0000 7fbe6f000640 1 -- 172.21.15.39:0/3417622468 >> 172.21.15.39:0/3417622468 conn(0x7fbe680b2460 msgr2=0x7fbe6806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:33.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.343+0000 7fbe6f000640 1 -- 172.21.15.39:0/3417622468 shutdown_connections 2024-09-21T07:46:33.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.343+0000 7fbe6f000640 1 -- 172.21.15.39:0/3417622468 wait complete. 2024-09-21T07:46:33.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.345+0000 7fbe6f000640 1 Processor -- start 2024-09-21T07:46:33.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.345+0000 7fbe6f000640 1 -- start start 2024-09-21T07:46:33.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.346+0000 7fbe6f000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe680b30a0 0x7fbe680c4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:33.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.346+0000 7fbe6f000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe680cb8a0 0x7fbe6814e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:33.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.346+0000 7fbe6f000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe680ccc60 0x7fbe680c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:33.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.346+0000 7fbe6f000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbe680af410 con 0x7fbe680ccc60 2024-09-21T07:46:33.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.346+0000 7fbe6f000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbe680af290 con 0x7fbe680cb8a0 2024-09-21T07:46:33.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.346+0000 7fbe6f000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fbe680af590 con 0x7fbe680b30a0 2024-09-21T07:46:33.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.347+0000 7fbe6d000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe680ccc60 0x7fbe680c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:33.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.347+0000 7fbe6d000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe680ccc60 0x7fbe680c59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38740/0 (socket says 172.21.15.39:38740) 2024-09-21T07:46:33.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.347+0000 7fbe6d000640 1 -- 172.21.15.39:0/1589234118 learned_addr learned my addr 172.21.15.39:0/1589234118 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:33.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.347+0000 7fbe67e00640 1 --2- 172.21.15.39:0/1589234118 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe680b30a0 0x7fbe680c4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:33.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.347+0000 7fbe67400640 1 --2- 172.21.15.39:0/1589234118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe680cb8a0 0x7fbe6814e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:33.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.347+0000 7fbe6d000640 1 -- 172.21.15.39:0/1589234118 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe680b30a0 msgr2=0x7fbe680c4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:33.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.347+0000 7fbe6d000640 1 --2- 172.21.15.39:0/1589234118 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe680b30a0 0x7fbe680c4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.347+0000 7fbe6d000640 1 -- 172.21.15.39:0/1589234118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe680cb8a0 msgr2=0x7fbe6814e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:33.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.347+0000 7fbe6d000640 1 --2- 172.21.15.39:0/1589234118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe680cb8a0 0x7fbe6814e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.347+0000 7fbe6d000640 1 -- 172.21.15.39:0/1589234118 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe680c4ec0 con 0x7fbe680ccc60 2024-09-21T07:46:33.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.347+0000 7fbe6d000640 1 --2- 172.21.15.39:0/1589234118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe680ccc60 0x7fbe680c59f0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fbe6000e960 tx=0x7fbe6000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:33.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.348+0000 7fbe5fe00640 1 -- 172.21.15.39:0/1589234118 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe60004d20 con 0x7fbe680ccc60 2024-09-21T07:46:33.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.348+0000 7fbe5fe00640 1 -- 172.21.15.39:0/1589234118 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe6000cb20 con 0x7fbe680ccc60 2024-09-21T07:46:33.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.348+0000 7fbe5fe00640 1 -- 172.21.15.39:0/1589234118 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe60005020 con 0x7fbe680ccc60 2024-09-21T07:46:33.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.348+0000 7fbe6f000640 1 -- 172.21.15.39:0/1589234118 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe680c51a0 con 0x7fbe680ccc60 2024-09-21T07:46:33.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.348+0000 7fbe6f000640 1 -- 172.21.15.39:0/1589234118 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe680c86d0 con 0x7fbe680ccc60 2024-09-21T07:46:33.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.350+0000 7fbe5fe00640 1 -- 172.21.15.39:0/1589234118 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbe600051c0 con 0x7fbe680ccc60 2024-09-21T07:46:33.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.351+0000 7fbe5fe00640 1 --2- 172.21.15.39:0/1589234118 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbe48078bc0 0x7fbe4807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:33.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.351+0000 7fbe5fe00640 1 -- 172.21.15.39:0/1589234118 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 6888+0+0 (secure 0 0 0) 0x7fbe6009bab0 con 0x7fbe680ccc60 2024-09-21T07:46:33.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.352+0000 7fbe67e00640 1 --2- 172.21.15.39:0/1589234118 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbe48078bc0 0x7fbe4807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:33.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.352+0000 7fbe76094740 1 Processor -- start 2024-09-21T07:46:33.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.352+0000 7fbe76094740 1 -- start start 2024-09-21T07:46:33.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.353+0000 7fbe67e00640 1 --2- 172.21.15.39:0/1589234118 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbe48078bc0 0x7fbe4807b0d0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fbe58000f60 tx=0x7fbe58009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:33.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.353+0000 7fbe76094740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55acfeb6d270 0x55acfef58a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:33.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.353+0000 7fbe76094740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acfeb443b0 0x55acfef0df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:33.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.353+0000 7fbe76094740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acfeb4c590 0x55acfef0e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:33.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.353+0000 7fbe76094740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55acfee97430 con 0x55acfeb6d270 2024-09-21T07:46:33.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.353+0000 7fbe76094740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55acfee875d0 con 0x55acfeb4c590 2024-09-21T07:46:33.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.353+0000 7fbe76094740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55acfee76a00 con 0x55acfeb443b0 2024-09-21T07:46:33.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.353+0000 7fbe67e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55acfeb6d270 0x55acfef58a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:33.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.353+0000 7fbe67e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55acfeb6d270 0x55acfef58a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38742/0 (socket says 172.21.15.39:38742) 2024-09-21T07:46:33.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.353+0000 7fbe67e00640 1 -- 172.21.15.39:0/1643679544 learned_addr learned my addr 172.21.15.39:0/1643679544 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:33.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.354+0000 7fbe67400640 1 --2- 172.21.15.39:0/1643679544 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acfeb443b0 0x55acfef0df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:33.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.354+0000 7fbe6d000640 1 --2- 172.21.15.39:0/1643679544 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acfeb4c590 0x55acfef0e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:33.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.354+0000 7fbe67e00640 1 -- 172.21.15.39:0/1643679544 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acfeb443b0 msgr2=0x55acfef0df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:46:33.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.354+0000 7fbe67e00640 1 --2- 172.21.15.39:0/1643679544 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acfeb443b0 0x55acfef0df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.354+0000 7fbe67e00640 1 -- 172.21.15.39:0/1643679544 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acfeb4c590 msgr2=0x55acfef0e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:46:33.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.354+0000 7fbe67e00640 1 --2- 172.21.15.39:0/1643679544 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acfeb4c590 0x55acfef0e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.354+0000 7fbe67e00640 1 -- 172.21.15.39:0/1643679544 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acfee3f520 con 0x55acfeb6d270 2024-09-21T07:46:33.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.354+0000 7fbe67e00640 1 --2- 172.21.15.39:0/1643679544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55acfeb6d270 0x55acfef58a20 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fbe580015b0 tx=0x7fbe58012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=512fb158df8b289a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:33.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.354+0000 7fbe5cc00640 1 -- 172.21.15.39:0/1643679544 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe58025070 con 0x55acfeb6d270 2024-09-21T07:46:33.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.354+0000 7fbe5cc00640 1 -- 172.21.15.39:0/1643679544 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe5801b050 con 0x55acfeb6d270 2024-09-21T07:46:33.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.354+0000 7fbe5cc00640 1 -- 172.21.15.39:0/1643679544 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe58014e00 con 0x55acfeb6d270 2024-09-21T07:46:33.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.355+0000 7fbe76094740 1 -- 172.21.15.39:0/1643679544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55acfeb6d270 msgr2=0x55acfef58a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:33.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.355+0000 7fbe76094740 1 --2- 172.21.15.39:0/1643679544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55acfeb6d270 0x55acfef58a20 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fbe580015b0 tx=0x7fbe58012000 comp rx=0 tx=0).stop 2024-09-21T07:46:33.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.355+0000 7fbe76094740 1 -- 172.21.15.39:0/1643679544 shutdown_connections 2024-09-21T07:46:33.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.355+0000 7fbe76094740 1 --2- 172.21.15.39:0/1643679544 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acfeb4c590 0x55acfef0e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.355+0000 7fbe76094740 1 --2- 172.21.15.39:0/1643679544 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acfeb443b0 0x55acfef0df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.355+0000 7fbe76094740 1 --2- 172.21.15.39:0/1643679544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55acfeb6d270 0x55acfef58a20 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.355+0000 7fbe76094740 1 -- 172.21.15.39:0/1643679544 >> 172.21.15.39:0/1643679544 conn(0x55acfeb613c0 msgr2=0x55acfef13970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:33.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.355+0000 7fbe76094740 1 -- 172.21.15.39:0/1643679544 shutdown_connections 2024-09-21T07:46:33.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.355+0000 7fbe76094740 1 -- 172.21.15.39:0/1643679544 wait complete. 2024-09-21T07:46:33.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.356+0000 7fbe76094740 1 -- start start 2024-09-21T07:46:33.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.357+0000 7fbe76094740 1 Processor -- start 2024-09-21T07:46:33.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.358+0000 7fbe76094740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acfeb443b0 0x55acfef6d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:33.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.358+0000 7fbe76094740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55acfeb4c590 0x55acfef35290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:33.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.358+0000 7fbe76094740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acfeb6d270 0x55acfef53c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:33.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.358+0000 7fbe76094740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55acfee1d330 con 0x55acfeb4c590 2024-09-21T07:46:33.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.358+0000 7fbe76094740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55acfee7a810 con 0x55acfeb6d270 2024-09-21T07:46:33.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.358+0000 7fbe76094740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55acfebe7620 con 0x55acfeb443b0 2024-09-21T07:46:33.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.358+0000 7fbe76094740 10 client.0 _ll_register_callbacks cb 0x55acfee22420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:46:33.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.358+0000 7fbe67400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55acfeb4c590 0x55acfef35290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:33.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.358+0000 7fbe67400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55acfeb4c590 0x55acfef35290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38754/0 (socket says 172.21.15.39:38754) 2024-09-21T07:46:33.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.358+0000 7fbe67400640 1 -- 172.21.15.39:0/4071357042 learned_addr learned my addr 172.21.15.39:0/4071357042 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:33.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.359+0000 7fbe67e00640 1 --2- 172.21.15.39:0/4071357042 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acfeb443b0 0x55acfef6d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:33.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.359+0000 7fbe6d000640 1 --2- 172.21.15.39:0/4071357042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acfeb6d270 0x55acfef53c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:33.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.359+0000 7fbe67400640 1 -- 172.21.15.39:0/4071357042 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acfeb443b0 msgr2=0x55acfef6d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:46:33.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.359+0000 7fbe67400640 1 --2- 172.21.15.39:0/4071357042 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acfeb443b0 0x55acfef6d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.359+0000 7fbe67400640 1 -- 172.21.15.39:0/4071357042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acfeb6d270 msgr2=0x55acfef53c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:46:33.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.359+0000 7fbe67400640 1 --2- 172.21.15.39:0/4071357042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acfeb6d270 0x55acfef53c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.359+0000 7fbe67400640 1 -- 172.21.15.39:0/4071357042 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acfebd8850 con 0x55acfeb4c590 2024-09-21T07:46:33.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.359+0000 7fbe6d000640 1 --2- 172.21.15.39:0/4071357042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acfeb6d270 0x55acfef53c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:33.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.359+0000 7fbe67400640 1 --2- 172.21.15.39:0/4071357042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55acfeb4c590 0x55acfef35290 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fbe54009230 tx=0x7fbe54005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76921a3109a46708 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:33.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.359+0000 7fbe51600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:46:33.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.360+0000 7fbe51600640 1 -- 172.21.15.39:0/4071357042 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe54035040 con 0x55acfeb4c590 2024-09-21T07:46:33.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.360+0000 7fbe76094740 1 -- 172.21.15.39:0/4071357042 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe5801b050 con 0x55acfeb4c590 2024-09-21T07:46:33.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.360+0000 7fbe51600640 1 -- 172.21.15.39:0/4071357042 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(175..175 src has 1..175) ==== 6888+0+0 (secure 0 0 0) 0x7fbe540043c0 con 0x55acfeb4c590 2024-09-21T07:46:33.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.360+0000 7fbe51600640 1 -- 172.21.15.39:0/4071357042 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=190) ==== 24+0+0 (secure 0 0 0) 0x7fbe5404d050 con 0x55acfeb4c590 2024-09-21T07:46:33.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.360+0000 7fbe76094740 10 client.9297 fetch_fsmap learned FSMap version 190 2024-09-21T07:46:33.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.360+0000 7fbe76094740 1 -- 172.21.15.39:0/4071357042 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=190,monmap=2+}) -- 0x55acfec4f6b0 con 0x55acfeb4c590 2024-09-21T07:46:33.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.360+0000 7fbe51600640 1 -- 172.21.15.39:0/4071357042 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe5801b050 con 0x55acfeb4c590 2024-09-21T07:46:33.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.360+0000 7fbe51600640 1 -- 172.21.15.39:0/4071357042 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 190) ==== 3502+0+0 (secure 0 0 0) 0x7fbe54050020 con 0x55acfeb4c590 2024-09-21T07:46:33.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.360+0000 7fbe76094740 10 client.9297 fetch_fsmap finished waiting for FSMap version 190 2024-09-21T07:46:33.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.360+0000 7fbe76094740 10 client.9297 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:46:33.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.360+0000 7fbe76094740 20 client.9297 populate_metadata read hostname 'smithi039' 2024-09-21T07:46:33.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.360+0000 7fbe76094740 1 --2- 172.21.15.39:0/4071357042 >> [v2:172.21.15.39:6833/3000703195,v1:172.21.15.39:6835/3000703195] conn(0x55acfeb2be40 0x55acfef59a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:33.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.360+0000 7fbe76094740 4 client.9297 mds_command: new command op to 9180 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:46:33.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.360+0000 7fbe76094740 1 -- 172.21.15.39:0/4071357042 --> [v2:172.21.15.39:6833/3000703195,v1:172.21.15.39:6835/3000703195] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55acfee89520 con 0x55acfeb2be40 2024-09-21T07:46:33.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.361+0000 7fbe67e00640 1 --2- 172.21.15.39:0/4071357042 >> [v2:172.21.15.39:6833/3000703195,v1:172.21.15.39:6835/3000703195] conn(0x55acfeb2be40 0x55acfef59a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:33.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.361+0000 7fbe67e00640 1 --2- 172.21.15.39:0/4071357042 >> [v2:172.21.15.39:6833/3000703195,v1:172.21.15.39:6835/3000703195] conn(0x55acfeb2be40 0x55acfef59a50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6aa69b1f158b699a server_cookie=efad9c9e1f57e7a8 in_seq=0 out_seq=0 2024-09-21T07:46:33.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.361+0000 7fbe51600640 10 client.9297 ms_handle_connect on v2:172.21.15.39:6833/3000703195 2024-09-21T07:46:33.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.362+0000 7fbe51600640 1 -- 172.21.15.39:0/4071357042 <== mds.0 v2:172.21.15.39:6833/3000703195 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55acfee89520 con 0x55acfeb2be40 2024-09-21T07:46:33.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.362+0000 7fbe51600640 10 client.9297 handle_command_reply: tid=0 2024-09-21T07:46:33.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.362+0000 7fbe76094740 1 client.9297 shutdown 2024-09-21T07:46:33.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.362+0000 7fbe76094740 1 -- 172.21.15.39:0/4071357042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55acfeb4c590 msgr2=0x55acfef35290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:33.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.362+0000 7fbe76094740 1 --2- 172.21.15.39:0/4071357042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55acfeb4c590 0x55acfef35290 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fbe54009230 tx=0x7fbe54005730 comp rx=0 tx=0).stop 2024-09-21T07:46:33.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.362+0000 7fbe76094740 1 -- 172.21.15.39:0/4071357042 shutdown_connections 2024-09-21T07:46:33.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.362+0000 7fbe76094740 1 --2- 172.21.15.39:0/4071357042 >> [v2:172.21.15.39:6833/3000703195,v1:172.21.15.39:6835/3000703195] conn(0x55acfeb2be40 0x55acfef59a50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.362+0000 7fbe76094740 1 --2- 172.21.15.39:0/4071357042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acfeb6d270 0x55acfef53c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.362+0000 7fbe76094740 1 --2- 172.21.15.39:0/4071357042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55acfeb4c590 0x55acfef35290 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.362+0000 7fbe76094740 1 --2- 172.21.15.39:0/4071357042 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acfeb443b0 0x55acfef6d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.362+0000 7fbe51600640 0 client.9297 ms_handle_reset on v2:172.21.15.39:6833/3000703195 2024-09-21T07:46:33.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.362+0000 7fbe76094740 1 -- 172.21.15.39:0/4071357042 >> 172.21.15.39:0/4071357042 conn(0x55acfeb613c0 msgr2=0x55acfef3a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:33.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.363+0000 7fbe76094740 1 -- 172.21.15.39:0/4071357042 shutdown_connections 2024-09-21T07:46:33.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.363+0000 7fbe76094740 1 -- 172.21.15.39:0/4071357042 wait complete. 2024-09-21T07:46:33.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.363+0000 7fbe76094740 20 client.9297 trim_cache size 0 max 16384 2024-09-21T07:46:33.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.382+0000 7fbe76094740 1 Processor -- start 2024-09-21T07:46:33.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.382+0000 7fbe76094740 1 -- start start 2024-09-21T07:46:33.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.383+0000 7fbe76094740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55acfeb443b0 0x55acfef13af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:33.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.383+0000 7fbe76094740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acfeb4c590 0x55acfef35150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:33.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.383+0000 7fbe76094740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acfeb6d270 0x55acfef35690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:33.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.384+0000 7fbe76094740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbe681ff5e0 con 0x55acfeb443b0 2024-09-21T07:46:33.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.384+0000 7fbe76094740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbe680cfab0 con 0x55acfeb6d270 2024-09-21T07:46:33.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.384+0000 7fbe76094740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fbe681ff010 con 0x55acfeb4c590 2024-09-21T07:46:33.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.384+0000 7fbe67e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55acfeb443b0 0x55acfef13af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:33.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.384+0000 7fbe67e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55acfeb443b0 0x55acfef13af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38764/0 (socket says 172.21.15.39:38764) 2024-09-21T07:46:33.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.384+0000 7fbe67e00640 1 -- 172.21.15.39:0/2629833096 learned_addr learned my addr 172.21.15.39:0/2629833096 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:33.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.384+0000 7fbe6d000640 1 --2- 172.21.15.39:0/2629833096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acfeb6d270 0x55acfef35690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:33.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.384+0000 7fbe67400640 1 --2- 172.21.15.39:0/2629833096 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acfeb4c590 0x55acfef35150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:33.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.384+0000 7fbe67e00640 1 -- 172.21.15.39:0/2629833096 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acfeb4c590 msgr2=0x55acfef35150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:46:33.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.384+0000 7fbe67e00640 1 --2- 172.21.15.39:0/2629833096 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acfeb4c590 0x55acfef35150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.384+0000 7fbe67e00640 1 -- 172.21.15.39:0/2629833096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acfeb6d270 msgr2=0x55acfef35690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:46:33.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.384+0000 7fbe67e00640 1 --2- 172.21.15.39:0/2629833096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acfeb6d270 0x55acfef35690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.384+0000 7fbe67e00640 1 -- 172.21.15.39:0/2629833096 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acfef5ad60 con 0x55acfeb443b0 2024-09-21T07:46:33.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.384+0000 7fbe6d000640 1 --2- 172.21.15.39:0/2629833096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acfeb6d270 0x55acfef35690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:46:33.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.384+0000 7fbe67400640 1 --2- 172.21.15.39:0/2629833096 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acfeb4c590 0x55acfef35150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:46:33.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.384+0000 7fbe67e00640 1 --2- 172.21.15.39:0/2629833096 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55acfeb443b0 0x55acfef13af0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800f5a0 tx=0x7fbe58001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ebc5933e444b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:33.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.385+0000 7fbe53e00640 1 -- 172.21.15.39:0/2629833096 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe58031070 con 0x55acfeb443b0 2024-09-21T07:46:33.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.385+0000 7fbe53e00640 1 -- 172.21.15.39:0/2629833096 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe5801b070 con 0x55acfeb443b0 2024-09-21T07:46:33.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.385+0000 7fbe53e00640 1 -- 172.21.15.39:0/2629833096 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe58025040 con 0x55acfeb443b0 2024-09-21T07:46:33.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.385+0000 7fbe76094740 1 -- 172.21.15.39:0/2629833096 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55acfeb443b0 msgr2=0x55acfef13af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:33.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.385+0000 7fbe76094740 1 --2- 172.21.15.39:0/2629833096 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55acfeb443b0 0x55acfef13af0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800f5a0 tx=0x7fbe58001080 comp rx=0 tx=0).stop 2024-09-21T07:46:33.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.385+0000 7fbe76094740 1 -- 172.21.15.39:0/2629833096 shutdown_connections 2024-09-21T07:46:33.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.386+0000 7fbe76094740 1 --2- 172.21.15.39:0/2629833096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acfeb6d270 0x55acfef35690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.386+0000 7fbe76094740 1 --2- 172.21.15.39:0/2629833096 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acfeb4c590 0x55acfef35150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.386+0000 7fbe76094740 1 --2- 172.21.15.39:0/2629833096 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55acfeb443b0 0x55acfef13af0 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.386+0000 7fbe76094740 1 -- 172.21.15.39:0/2629833096 >> 172.21.15.39:0/2629833096 conn(0x55acfeb613c0 msgr2=0x55acfef6cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:33.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.386+0000 7fbe76094740 1 -- 172.21.15.39:0/2629833096 shutdown_connections 2024-09-21T07:46:33.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.386+0000 7fbe76094740 1 -- 172.21.15.39:0/2629833096 wait complete. 2024-09-21T07:46:33.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.387+0000 7fbe76094740 1 -- start start 2024-09-21T07:46:33.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.387+0000 7fbe76094740 1 Processor -- start 2024-09-21T07:46:33.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.388+0000 7fbe76094740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acfeb443b0 0x55acfefdd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:33.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.388+0000 7fbe76094740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55acfeb4c590 0x55acfef34fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:33.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.388+0000 7fbe76094740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acfeb6d270 0x55acfef354e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:33.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.388+0000 7fbe76094740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55acfef5c760 con 0x55acfeb4c590 2024-09-21T07:46:33.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.388+0000 7fbe76094740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55acfef5bd20 con 0x55acfeb6d270 2024-09-21T07:46:33.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.388+0000 7fbe76094740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55acfef3d460 con 0x55acfeb443b0 2024-09-21T07:46:33.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.388+0000 7fbe76094740 10 client.0 _ll_register_callbacks cb 0x7fbe54005310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:46:33.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.389+0000 7fbe67400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55acfeb4c590 0x55acfef34fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:33.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.389+0000 7fbe67400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55acfeb4c590 0x55acfef34fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38772/0 (socket says 172.21.15.39:38772) 2024-09-21T07:46:33.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.389+0000 7fbe67400640 1 -- 172.21.15.39:0/3243952742 learned_addr learned my addr 172.21.15.39:0/3243952742 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:33.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.389+0000 7fbe67e00640 1 --2- 172.21.15.39:0/3243952742 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acfeb443b0 0x55acfefdd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:33.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.389+0000 7fbe6d000640 1 --2- 172.21.15.39:0/3243952742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acfeb6d270 0x55acfef354e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:33.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.389+0000 7fbe67400640 1 -- 172.21.15.39:0/3243952742 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acfeb443b0 msgr2=0x55acfefdd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:46:33.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.389+0000 7fbe67400640 1 --2- 172.21.15.39:0/3243952742 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acfeb443b0 0x55acfefdd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.389+0000 7fbe67400640 1 -- 172.21.15.39:0/3243952742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acfeb6d270 msgr2=0x55acfef354e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:46:33.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.389+0000 7fbe67400640 1 --2- 172.21.15.39:0/3243952742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acfeb6d270 0x55acfef354e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.389+0000 7fbe67400640 1 -- 172.21.15.39:0/3243952742 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acfef33c90 con 0x55acfeb4c590 2024-09-21T07:46:33.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.389+0000 7fbe67e00640 1 --2- 172.21.15.39:0/3243952742 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acfeb443b0 0x55acfefdd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:33.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.389+0000 7fbe67400640 1 --2- 172.21.15.39:0/3243952742 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55acfeb4c590 0x55acfef34fa0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fbe540092a0 tx=0x7fbe54003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=659fa2779049cd87 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:33.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.389+0000 7fbe51600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:46:33.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.389+0000 7fbe51600640 1 -- 172.21.15.39:0/3243952742 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe54035040 con 0x55acfeb4c590 2024-09-21T07:46:33.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.390+0000 7fbe51600640 1 -- 172.21.15.39:0/3243952742 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(175..175 src has 1..175) ==== 6888+0+0 (secure 0 0 0) 0x7fbe540047a0 con 0x55acfeb4c590 2024-09-21T07:46:33.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.390+0000 7fbe76094740 1 -- 172.21.15.39:0/3243952742 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe5801b070 con 0x55acfeb4c590 2024-09-21T07:46:33.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.390+0000 7fbe51600640 1 -- 172.21.15.39:0/3243952742 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=190) ==== 24+0+0 (secure 0 0 0) 0x7fbe540049a0 con 0x55acfeb4c590 2024-09-21T07:46:33.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.390+0000 7fbe76094740 10 client.9303 fetch_fsmap learned FSMap version 190 2024-09-21T07:46:33.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.390+0000 7fbe76094740 1 -- 172.21.15.39:0/3243952742 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=190,monmap=2+}) -- 0x55acfef69570 con 0x55acfeb4c590 2024-09-21T07:46:33.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.390+0000 7fbe51600640 1 -- 172.21.15.39:0/3243952742 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe5801b070 con 0x55acfeb4c590 2024-09-21T07:46:33.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.390+0000 7fbe51600640 1 -- 172.21.15.39:0/3243952742 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 190) ==== 3502+0+0 (secure 0 0 0) 0x7fbe54059050 con 0x55acfeb4c590 2024-09-21T07:46:33.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.390+0000 7fbe76094740 10 client.9303 fetch_fsmap finished waiting for FSMap version 190 2024-09-21T07:46:33.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.390+0000 7fbe76094740 10 client.9303 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:46:33.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.390+0000 7fbe76094740 20 client.9303 populate_metadata read hostname 'smithi039' 2024-09-21T07:46:33.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.390+0000 7fbe76094740 1 --2- 172.21.15.39:0/3243952742 >> [v2:172.21.15.39:6833/3000703195,v1:172.21.15.39:6835/3000703195] conn(0x55acfef6ce00 0x55acfefe0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:33.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.390+0000 7fbe76094740 4 client.9303 mds_command: new command op to 9180 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:46:33.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.390+0000 7fbe76094740 1 -- 172.21.15.39:0/3243952742 --> [v2:172.21.15.39:6833/3000703195,v1:172.21.15.39:6835/3000703195] -- command(tid 0: {"prefix": "status"}) -- 0x55acfee84fd0 con 0x55acfef6ce00 2024-09-21T07:46:33.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.391+0000 7fbe67e00640 1 --2- 172.21.15.39:0/3243952742 >> [v2:172.21.15.39:6833/3000703195,v1:172.21.15.39:6835/3000703195] conn(0x55acfef6ce00 0x55acfefe0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:33.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.391+0000 7fbe67e00640 1 --2- 172.21.15.39:0/3243952742 >> [v2:172.21.15.39:6833/3000703195,v1:172.21.15.39:6835/3000703195] conn(0x55acfef6ce00 0x55acfefe0760 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2cc5abca1846ac3 server_cookie=d6243f0a72c0466 in_seq=0 out_seq=0 2024-09-21T07:46:33.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.391+0000 7fbe51600640 10 client.9303 ms_handle_connect on v2:172.21.15.39:6833/3000703195 2024-09-21T07:46:33.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.391+0000 7fbe51600640 1 -- 172.21.15.39:0/3243952742 <== mds.0 v2:172.21.15.39:6833/3000703195 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55acfee84fd0 con 0x55acfef6ce00 2024-09-21T07:46:33.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.391+0000 7fbe51600640 10 client.9303 handle_command_reply: tid=0 2024-09-21T07:46:33.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.391+0000 7fbe76094740 1 client.9303 shutdown 2024-09-21T07:46:33.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.392+0000 7fbe76094740 1 -- 172.21.15.39:0/3243952742 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55acfeb4c590 msgr2=0x55acfef34fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:33.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.392+0000 7fbe76094740 1 --2- 172.21.15.39:0/3243952742 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55acfeb4c590 0x55acfef34fa0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fbe540092a0 tx=0x7fbe54003eb0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.392+0000 7fbe76094740 1 -- 172.21.15.39:0/3243952742 shutdown_connections 2024-09-21T07:46:33.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.392+0000 7fbe76094740 1 --2- 172.21.15.39:0/3243952742 >> [v2:172.21.15.39:6833/3000703195,v1:172.21.15.39:6835/3000703195] conn(0x55acfef6ce00 0x55acfefe0760 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.392+0000 7fbe76094740 1 --2- 172.21.15.39:0/3243952742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55acfeb6d270 0x55acfef354e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.392+0000 7fbe76094740 1 --2- 172.21.15.39:0/3243952742 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55acfeb4c590 0x55acfef34fa0 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.392+0000 7fbe51600640 0 client.9303 ms_handle_reset on v2:172.21.15.39:6833/3000703195 2024-09-21T07:46:33.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.392+0000 7fbe76094740 1 --2- 172.21.15.39:0/3243952742 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55acfeb443b0 0x55acfefdd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.392+0000 7fbe76094740 1 -- 172.21.15.39:0/3243952742 >> 172.21.15.39:0/3243952742 conn(0x55acfeb613c0 msgr2=0x55acfef3a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:33.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.392+0000 7fbe76094740 1 -- 172.21.15.39:0/3243952742 shutdown_connections 2024-09-21T07:46:33.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.392+0000 7fbe76094740 1 -- 172.21.15.39:0/3243952742 wait complete. 2024-09-21T07:46:33.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.392+0000 7fbe76094740 20 client.9303 trim_cache size 0 max 16384 2024-09-21T07:46:33.397 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:46:33.397 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:46:33.397 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-09-21T07:46:33.397 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 9180, 2024-09-21T07:46:33.397 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:46:33.397 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:46:33.397 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:46:33.397 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 6.8033922750000002, 2024-09-21T07:46:33.397 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 190, 2024-09-21T07:46:33.397 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 175, 2024-09-21T07:46:33.397 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 175, 2024-09-21T07:46:33.397 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 11.802636315000001 2024-09-21T07:46:33.398 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:46:33.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.393+0000 7fbe6f000640 1 -- 172.21.15.39:0/1589234118 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbe48078bc0 msgr2=0x7fbe4807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:33.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.393+0000 7fbe6f000640 1 --2- 172.21.15.39:0/1589234118 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbe48078bc0 0x7fbe4807b0d0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fbe58000f60 tx=0x7fbe58009040 comp rx=0 tx=0).stop 2024-09-21T07:46:33.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.393+0000 7fbe6f000640 1 -- 172.21.15.39:0/1589234118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe680ccc60 msgr2=0x7fbe680c59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:33.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.393+0000 7fbe6f000640 1 --2- 172.21.15.39:0/1589234118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe680ccc60 0x7fbe680c59f0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fbe6000e960 tx=0x7fbe6000ee20 comp rx=0 tx=0).stop 2024-09-21T07:46:33.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.393+0000 7fbe6f000640 1 -- 172.21.15.39:0/1589234118 shutdown_connections 2024-09-21T07:46:33.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.393+0000 7fbe6f000640 1 --2- 172.21.15.39:0/1589234118 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbe48078bc0 0x7fbe4807b0d0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.393+0000 7fbe6f000640 1 --2- 172.21.15.39:0/1589234118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe680ccc60 0x7fbe680c59f0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.393+0000 7fbe6f000640 1 --2- 172.21.15.39:0/1589234118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe680cb8a0 0x7fbe6814e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.393+0000 7fbe6f000640 1 --2- 172.21.15.39:0/1589234118 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbe680b30a0 0x7fbe680c4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.393+0000 7fbe6f000640 1 -- 172.21.15.39:0/1589234118 >> 172.21.15.39:0/1589234118 conn(0x7fbe680b2460 msgr2=0x7fbe680ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:33.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.393+0000 7fbe6f000640 1 -- 172.21.15.39:0/1589234118 shutdown_connections 2024-09-21T07:46:33.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.393+0000 7fbe6f000640 1 -- 172.21.15.39:0/1589234118 wait complete. 2024-09-21T07:46:33.419 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:46:33.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.646+0000 7fc582600640 1 -- 172.21.15.39:0/2283426428 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc57c0b5240 msgr2=0x7fc57c15c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:33.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.646+0000 7fc582600640 1 --2- 172.21.15.39:0/2283426428 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc57c0b5240 0x7fc57c15c690 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fc56c005fd0 tx=0x7fc56c02a5d0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.646+0000 7fc582600640 1 -- 172.21.15.39:0/2283426428 shutdown_connections 2024-09-21T07:46:33.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.646+0000 7fc582600640 1 --2- 172.21.15.39:0/2283426428 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc57c0cf920 0x7fc57c0d0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.646+0000 7fc582600640 1 --2- 172.21.15.39:0/2283426428 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc57c0b5240 0x7fc57c15c690 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.646+0000 7fc582600640 1 -- 172.21.15.39:0/2283426428 >> 172.21.15.39:0/2283426428 conn(0x7fc57c06cfc0 msgr2=0x7fc57c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:33.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.646+0000 7fc582600640 1 -- 172.21.15.39:0/2283426428 shutdown_connections 2024-09-21T07:46:33.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.646+0000 7fc582600640 1 -- 172.21.15.39:0/2283426428 wait complete. 2024-09-21T07:46:33.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.648+0000 7fc582600640 1 Processor -- start 2024-09-21T07:46:33.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.648+0000 7fc582600640 1 -- start start 2024-09-21T07:46:33.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.649+0000 7fc582600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc57c0b5240 0x7fc57c0b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:33.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.649+0000 7fc582600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc57c0cf920 0x7fc57c140e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:33.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.650+0000 7fc582600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc57c0cfd30 0x7fc57c0ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:33.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.650+0000 7fc582600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc57c0d0f00 con 0x7fc57c0cfd30 2024-09-21T07:46:33.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.650+0000 7fc582600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc57c0d0d80 con 0x7fc57c0b5240 2024-09-21T07:46:33.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.650+0000 7fc582600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc57c14e640 con 0x7fc57c0cf920 2024-09-21T07:46:33.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.650+0000 7fc57be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc57c0cfd30 0x7fc57c0ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:33.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.650+0000 7fc57be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc57c0cfd30 0x7fc57c0ae930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38790/0 (socket says 172.21.15.39:38790) 2024-09-21T07:46:33.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.650+0000 7fc57be00640 1 -- 172.21.15.39:0/604603891 learned_addr learned my addr 172.21.15.39:0/604603891 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:33.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.650+0000 7fc57aa00640 1 --2- 172.21.15.39:0/604603891 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc57c0cf920 0x7fc57c140e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:33.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.650+0000 7fc57b400640 1 --2- 172.21.15.39:0/604603891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc57c0b5240 0x7fc57c0b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:33.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.650+0000 7fc57be00640 1 -- 172.21.15.39:0/604603891 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc57c0cf920 msgr2=0x7fc57c140e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:33.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.650+0000 7fc57be00640 1 --2- 172.21.15.39:0/604603891 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc57c0cf920 0x7fc57c140e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.650+0000 7fc57be00640 1 -- 172.21.15.39:0/604603891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc57c0b5240 msgr2=0x7fc57c0b1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:33.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.650+0000 7fc57be00640 1 --2- 172.21.15.39:0/604603891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc57c0b5240 0x7fc57c0b1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.650+0000 7fc57be00640 1 -- 172.21.15.39:0/604603891 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc57c141360 con 0x7fc57c0cfd30 2024-09-21T07:46:33.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.651+0000 7fc57be00640 1 --2- 172.21.15.39:0/604603891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc57c0cfd30 0x7fc57c0ae930 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fc57400c6e0 tx=0x7fc57400cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:33.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.651+0000 7fc573400640 1 -- 172.21.15.39:0/604603891 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc574003b70 con 0x7fc57c0cfd30 2024-09-21T07:46:33.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.651+0000 7fc582600640 1 -- 172.21.15.39:0/604603891 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc57c1f5df0 con 0x7fc57c0cfd30 2024-09-21T07:46:33.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.651+0000 7fc573400640 1 -- 172.21.15.39:0/604603891 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc574003d30 con 0x7fc57c0cfd30 2024-09-21T07:46:33.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.651+0000 7fc573400640 1 -- 172.21.15.39:0/604603891 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc574005330 con 0x7fc57c0cfd30 2024-09-21T07:46:33.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.651+0000 7fc582600640 1 -- 172.21.15.39:0/604603891 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc57c1f63b0 con 0x7fc57c0cfd30 2024-09-21T07:46:33.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.652+0000 7fc582600640 1 -- 172.21.15.39:0/604603891 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc57c071110 con 0x7fc57c0cfd30 2024-09-21T07:46:33.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.653+0000 7fc573400640 1 -- 172.21.15.39:0/604603891 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc5740054d0 con 0x7fc57c0cfd30 2024-09-21T07:46:33.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.654+0000 7fc573400640 1 --2- 172.21.15.39:0/604603891 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc55c078b90 0x7fc55c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:33.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.654+0000 7fc573400640 1 -- 172.21.15.39:0/604603891 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 6888+0+0 (secure 0 0 0) 0x7fc57409af00 con 0x7fc57c0cfd30 2024-09-21T07:46:33.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.655+0000 7fc57b400640 1 --2- 172.21.15.39:0/604603891 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc55c078b90 0x7fc55c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:33.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.656+0000 7fc57b400640 1 --2- 172.21.15.39:0/604603891 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc55c078b90 0x7fc55c07b0a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fc56c033000 tx=0x7fc56c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:33.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.659+0000 7fc573400640 1 -- 172.21.15.39:0/604603891 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc57c071110 con 0x7fc57c0cfd30 2024-09-21T07:46:33.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.893+0000 7fc582600640 1 -- 172.21.15.39:0/604603891 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc57c0b4e10 con 0x7fc57c0cfd30 2024-09-21T07:46:33.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.893+0000 7fc573400640 1 -- 172.21.15.39:0/604603891 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v190) ==== 53+0+73 (secure 0 0 0) 0x7fc57c141360 con 0x7fc57c0cfd30 2024-09-21T07:46:33.895 INFO:teuthology.orchestra.run.smithi039.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:46:33.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.896+0000 7fc582600640 1 -- 172.21.15.39:0/604603891 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc55c078b90 msgr2=0x7fc55c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:33.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.896+0000 7fc582600640 1 --2- 172.21.15.39:0/604603891 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc55c078b90 0x7fc55c07b0a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fc56c033000 tx=0x7fc56c004730 comp rx=0 tx=0).stop 2024-09-21T07:46:33.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.896+0000 7fc582600640 1 -- 172.21.15.39:0/604603891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc57c0cfd30 msgr2=0x7fc57c0ae930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:33.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.896+0000 7fc582600640 1 --2- 172.21.15.39:0/604603891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc57c0cfd30 0x7fc57c0ae930 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fc57400c6e0 tx=0x7fc57400cba0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.896+0000 7fc582600640 1 -- 172.21.15.39:0/604603891 shutdown_connections 2024-09-21T07:46:33.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.896+0000 7fc582600640 1 --2- 172.21.15.39:0/604603891 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc55c078b90 0x7fc55c07b0a0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.896+0000 7fc582600640 1 --2- 172.21.15.39:0/604603891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc57c0cfd30 0x7fc57c0ae930 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.896+0000 7fc582600640 1 --2- 172.21.15.39:0/604603891 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc57c0cf920 0x7fc57c140e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.896+0000 7fc582600640 1 --2- 172.21.15.39:0/604603891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc57c0b5240 0x7fc57c0b1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:33.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.896+0000 7fc582600640 1 -- 172.21.15.39:0/604603891 >> 172.21.15.39:0/604603891 conn(0x7fc57c06cfc0 msgr2=0x7fc57c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:33.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.896+0000 7fc582600640 1 -- 172.21.15.39:0/604603891 shutdown_connections 2024-09-21T07:46:33.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:33.896+0000 7fc582600640 1 -- 172.21.15.39:0/604603891 wait complete. 2024-09-21T07:46:33.919 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:46:33.919 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:46:33.919 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi039.front.sepia.ceph.com 2024-09-21T07:46:33.919 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T07:46:33.920 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:46:33.920 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:46:33.920 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:46:33.920 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:46:33.920 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:46:33.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:33.920 DEBUG:teuthology.orchestra.run.smithi039:> ip addr 2024-09-21T07:46:33.956 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:46:33.956 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:46:33.956 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:46:33.957 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:33.957 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2024-09-21T07:46:33.957 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:33.957 INFO:teuthology.orchestra.run.smithi039.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:46:33.957 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:46:33.957 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f0 2024-09-21T07:46:33.957 INFO:teuthology.orchestra.run.smithi039.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:46:33.957 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c7 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:46:33.957 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f1 2024-09-21T07:46:33.957 INFO:teuthology.orchestra.run.smithi039.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:46:33.957 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:70 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:46:33.957 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f0 2024-09-21T07:46:33.957 INFO:teuthology.orchestra.run.smithi039.stdout: inet 172.21.15.39/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T07:46:33.957 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft 41925sec preferred_lft 41925sec 2024-09-21T07:46:33.958 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec4:7aff:febd:1370/64 scope link 2024-09-21T07:46:33.958 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:33.958 INFO:teuthology.orchestra.run.smithi039.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:46:33.958 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:71 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:46:33.958 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f1 2024-09-21T07:46:33.958 INFO:teuthology.orchestra.run.smithi039.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:46:33.958 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff 2024-09-21T07:46:33.958 INFO:teuthology.orchestra.run.smithi039.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:46:33.958 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:33.958 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::b079:ddff:fe0e:6333/64 scope link 2024-09-21T07:46:33.958 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:33.958 INFO:teuthology.orchestra.run.smithi039.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:46:33.958 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:46:33.958 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::2ca7:98ff:fe8a:d0ed/64 scope link 2024-09-21T07:46:33.959 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:33.959 INFO:teuthology.orchestra.run.smithi039.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:46:33.959 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether ee:10:53:01:0c:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:46:33.959 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec10:53ff:fe01:c44/64 scope link 2024-09-21T07:46:33.959 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:33.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:33.959 DEBUG:teuthology.orchestra.run.smithi039:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:46:34.050 INFO:teuthology.orchestra.run.smithi039.stdout:1 2024-09-21T07:46:34.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:34.053 DEBUG:teuthology.orchestra.run.smithi039:> route 2024-09-21T07:46:34.089 INFO:teuthology.orchestra.run.smithi039.stdout:Kernel IP routing table 2024-09-21T07:46:34.089 INFO:teuthology.orchestra.run.smithi039.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:46:34.089 INFO:teuthology.orchestra.run.smithi039.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T07:46:34.089 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T07:46:34.089 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:46:34.089 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:46:34.089 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-21T07:46:34.089 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T07:46:34.089 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T07:46:34.089 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T07:46:34.090 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-21T07:46:34.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:46:34.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:46:34.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:34.286 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-09-21T07:46:34.322 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:46:34.322 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T07:46:34.322 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:46:34.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:34.322 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-09-21T07:46:34.419 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:46:34.420 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T07:46:34.455 INFO:teuthology.orchestra.run.smithi039.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:46:34.455 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:46:34.455 DEBUG:teuthology.orchestra.run.smithi039:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:46:34.530 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-21T07:46:34.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:34.531 DEBUG:teuthology.orchestra.run.smithi039:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:46:34.731 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:46:34.731 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:46:34.766 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:46:34.766 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:46:34.766 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: resolved to: "172.21.15.39:3300,172.21.15.148:3300,172.21.15.148:3301" 2024-09-21T07:46:34.767 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: trying mount with new device syntax: 0@7ab9b04a-f88d-49aa-a493-e5c076637ffb.cephfs=/ 2024-09-21T07:46:34.767 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.39:3300/172.21.15.148:3300/172.21.15.148:3301" will pass to kernel 2024-09-21T07:46:34.768 INFO:teuthology.orchestra.run.smithi039.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T07:46:34.768 INFO:teuthology.orchestra.run.smithi039.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:46:34.768 INFO:teuthology.orchestra.run.smithi039.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:46:34.768 INFO:teuthology.orchestra.run.smithi039.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:46:34.768 INFO:teuthology.orchestra.run.smithi039.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:46:34.771 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:46:34.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:34.771 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:46:34.828 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:46:34.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:46:34.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:46:34.925 INFO:teuthology.orchestra.run.smithi039.stdout:client9315 2024-09-21T07:46:34.925 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:46:34.925 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:46:34.961 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get mds_max_snaps_per_dir 2024-09-21T07:46:35.242 INFO:teuthology.orchestra.run.smithi039.stdout:{"mds_max_snaps_per_dir":"100"} 2024-09-21T07:46:35.243 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_max_snaps_per_dir": "100" } 2024-09-21T07:46:35.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:35.243 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts) 2024-09-21T07:46:35.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:35.285 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_1) 2024-09-21T07:46:35.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:35.387 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_2) 2024-09-21T07:46:35.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:35.449 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_3) 2024-09-21T07:46:35.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:35.510 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_4) 2024-09-21T07:46:35.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:35.572 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_5) 2024-09-21T07:46:35.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:35.633 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_6) 2024-09-21T07:46:35.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:35.695 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_7) 2024-09-21T07:46:35.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:35.757 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_8) 2024-09-21T07:46:35.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:35.819 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_9) 2024-09-21T07:46:35.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:35.881 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_10) 2024-09-21T07:46:35.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:35.944 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_11) 2024-09-21T07:46:35.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:35.995 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_12) 2024-09-21T07:46:36.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:36.046 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_13) 2024-09-21T07:46:36.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:36.098 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_14) 2024-09-21T07:46:36.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:36.150 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_15) 2024-09-21T07:46:36.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:36.201 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_16) 2024-09-21T07:46:36.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:36.253 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_17) 2024-09-21T07:46:36.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:36.304 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_18) 2024-09-21T07:46:36.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:36.357 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_19) 2024-09-21T07:46:36.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:36.409 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_20) 2024-09-21T07:46:36.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:36.460 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_21) 2024-09-21T07:46:36.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:36.512 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_22) 2024-09-21T07:46:36.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:36.563 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_23) 2024-09-21T07:46:36.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:36.615 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_24) 2024-09-21T07:46:36.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:36.666 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_25) 2024-09-21T07:46:36.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:36.718 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_26) 2024-09-21T07:46:36.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:36.770 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_27) 2024-09-21T07:46:36.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:36.821 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_28) 2024-09-21T07:46:36.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:36.873 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_29) 2024-09-21T07:46:36.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:36.924 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_30) 2024-09-21T07:46:36.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:36.976 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_31) 2024-09-21T07:46:37.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:37.027 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_32) 2024-09-21T07:46:37.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:37.078 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_33) 2024-09-21T07:46:37.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:37.130 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_34) 2024-09-21T07:46:37.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:37.181 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_35) 2024-09-21T07:46:37.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:37.233 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_36) 2024-09-21T07:46:37.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:37.286 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_37) 2024-09-21T07:46:37.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:37.337 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_38) 2024-09-21T07:46:37.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:37.389 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_39) 2024-09-21T07:46:37.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:37.441 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_40) 2024-09-21T07:46:37.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:37.492 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_41) 2024-09-21T07:46:37.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:37.543 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_42) 2024-09-21T07:46:37.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:37.595 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_43) 2024-09-21T07:46:37.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:37.646 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_44) 2024-09-21T07:46:37.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:37.698 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_45) 2024-09-21T07:46:37.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:37.790 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_46) 2024-09-21T07:46:37.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:37.842 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_47) 2024-09-21T07:46:37.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:37.894 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_48) 2024-09-21T07:46:37.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:37.946 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_49) 2024-09-21T07:46:37.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:37.998 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_50) 2024-09-21T07:46:38.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:38.051 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_51) 2024-09-21T07:46:38.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:38.103 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_52) 2024-09-21T07:46:38.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:38.155 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_53) 2024-09-21T07:46:38.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:38.207 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_54) 2024-09-21T07:46:38.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:38.259 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_55) 2024-09-21T07:46:38.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:38.312 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_56) 2024-09-21T07:46:38.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:38.364 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_57) 2024-09-21T07:46:38.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:38.416 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_58) 2024-09-21T07:46:38.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:38.468 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_59) 2024-09-21T07:46:38.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:38.520 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_60) 2024-09-21T07:46:38.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:38.573 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_61) 2024-09-21T07:46:38.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:38.624 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_62) 2024-09-21T07:46:38.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:38.676 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_63) 2024-09-21T07:46:38.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:38.729 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_64) 2024-09-21T07:46:38.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:38.781 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_65) 2024-09-21T07:46:38.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:38.833 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_66) 2024-09-21T07:46:38.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:38.886 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_67) 2024-09-21T07:46:38.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:38.940 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_68) 2024-09-21T07:46:38.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:38.993 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_69) 2024-09-21T07:46:39.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:39.046 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_70) 2024-09-21T07:46:39.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:39.099 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_71) 2024-09-21T07:46:39.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:39.152 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_72) 2024-09-21T07:46:39.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:39.204 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_73) 2024-09-21T07:46:39.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:39.257 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_74) 2024-09-21T07:46:39.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:39.310 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_75) 2024-09-21T07:46:39.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:39.363 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_76) 2024-09-21T07:46:39.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:39.415 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_77) 2024-09-21T07:46:39.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:39.468 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_78) 2024-09-21T07:46:39.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:39.521 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_79) 2024-09-21T07:46:39.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:39.573 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_80) 2024-09-21T07:46:39.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:39.626 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_81) 2024-09-21T07:46:39.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:39.679 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_82) 2024-09-21T07:46:39.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:39.732 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_83) 2024-09-21T07:46:39.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:39.784 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_84) 2024-09-21T07:46:39.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:39.837 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_85) 2024-09-21T07:46:39.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:39.890 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_86) 2024-09-21T07:46:39.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:39.943 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_87) 2024-09-21T07:46:39.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:39.995 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_88) 2024-09-21T07:46:40.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:40.048 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_89) 2024-09-21T07:46:40.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:40.101 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_90) 2024-09-21T07:46:40.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:40.153 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_91) 2024-09-21T07:46:40.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:40.207 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_92) 2024-09-21T07:46:40.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:40.259 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_93) 2024-09-21T07:46:40.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:40.311 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_94) 2024-09-21T07:46:40.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:40.364 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_95) 2024-09-21T07:46:40.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:40.432 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_96) 2024-09-21T07:46:40.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:40.496 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_97) 2024-09-21T07:46:40.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:40.559 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_98) 2024-09-21T07:46:40.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:40.623 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_99) 2024-09-21T07:46:40.687 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:46:40.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.913+0000 7f177a600640 1 -- 172.21.15.39:0/1330096433 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f17740d0430 msgr2=0x7f17740d0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:40.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.913+0000 7f177a600640 1 --2- 172.21.15.39:0/1330096433 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f17740d0430 0x7f17740d0840 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1760009830 tx=0x7f176002a420 comp rx=0 tx=0).stop 2024-09-21T07:46:40.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.913+0000 7f177a600640 1 -- 172.21.15.39:0/1330096433 shutdown_connections 2024-09-21T07:46:40.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.913+0000 7f177a600640 1 --2- 172.21.15.39:0/1330096433 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f17740d0430 0x7f17740d0840 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:40.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.913+0000 7f177a600640 1 --2- 172.21.15.39:0/1330096433 >> v2:172.21.15.148:3300/0 conn(0x7f1774160110 0x7f17740ccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:40.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.913+0000 7f177a600640 1 -- 172.21.15.39:0/1330096433 >> 172.21.15.39:0/1330096433 conn(0x7f177406cfc0 msgr2=0x7f177406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:40.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.913+0000 7f177a600640 1 -- 172.21.15.39:0/1330096433 shutdown_connections 2024-09-21T07:46:40.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.913+0000 7f177a600640 1 -- 172.21.15.39:0/1330096433 wait complete. 2024-09-21T07:46:40.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.915+0000 7f177a600640 1 Processor -- start 2024-09-21T07:46:40.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.915+0000 7f177a600640 1 -- start start 2024-09-21T07:46:40.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.917+0000 7f177a600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f17740cf5e0 0x7f1774148e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:40.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.917+0000 7f177a600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17740d0430 0x7f1774155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:40.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.917+0000 7f177a600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1774160110 0x7f17740af750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:40.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.917+0000 7f177a600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f177415d440 con 0x7f17740d0430 2024-09-21T07:46:40.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.917+0000 7f177a600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f177415d2c0 con 0x7f1774160110 2024-09-21T07:46:40.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.917+0000 7f177a600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f177415d5c0 con 0x7f17740cf5e0 2024-09-21T07:46:40.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.917+0000 7f1773e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17740d0430 0x7f1774155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:40.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.917+0000 7f1773e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17740d0430 0x7f1774155e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38804/0 (socket says 172.21.15.39:38804) 2024-09-21T07:46:40.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.917+0000 7f1773e00640 1 -- 172.21.15.39:0/539826289 learned_addr learned my addr 172.21.15.39:0/539826289 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:40.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.917+0000 7f1779200640 1 --2- 172.21.15.39:0/539826289 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f17740cf5e0 0x7f1774148e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:40.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.917+0000 7f1779c00640 1 --2- 172.21.15.39:0/539826289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1774160110 0x7f17740af750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:40.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.918+0000 7f1773e00640 1 -- 172.21.15.39:0/539826289 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f17740cf5e0 msgr2=0x7f1774148e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:40.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.918+0000 7f1773e00640 1 --2- 172.21.15.39:0/539826289 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f17740cf5e0 0x7f1774148e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:40.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.918+0000 7f1773e00640 1 -- 172.21.15.39:0/539826289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1774160110 msgr2=0x7f17740af750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:40.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.918+0000 7f1773e00640 1 --2- 172.21.15.39:0/539826289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1774160110 0x7f17740af750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:40.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.918+0000 7f1773e00640 1 -- 172.21.15.39:0/539826289 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1774158180 con 0x7f17740d0430 2024-09-21T07:46:40.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.918+0000 7f1779c00640 1 --2- 172.21.15.39:0/539826289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1774160110 0x7f17740af750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:46:40.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.918+0000 7f1779200640 1 --2- 172.21.15.39:0/539826289 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f17740cf5e0 0x7f1774148e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:40.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.918+0000 7f1773e00640 1 --2- 172.21.15.39:0/539826289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17740d0430 0x7f1774155e90 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f1760009950 tx=0x7f1760003b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:40.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.918+0000 7f1770c00640 1 -- 172.21.15.39:0/539826289 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176003f030 con 0x7f17740d0430 2024-09-21T07:46:40.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.918+0000 7f1770c00640 1 -- 172.21.15.39:0/539826289 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1760004510 con 0x7f17740d0430 2024-09-21T07:46:40.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.918+0000 7f1770c00640 1 -- 172.21.15.39:0/539826289 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1760037040 con 0x7f17740d0430 2024-09-21T07:46:40.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.918+0000 7f177a600640 1 -- 172.21.15.39:0/539826289 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1774158460 con 0x7f17740d0430 2024-09-21T07:46:40.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.918+0000 7f177a600640 1 -- 172.21.15.39:0/539826289 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1774158a20 con 0x7f17740d0430 2024-09-21T07:46:40.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.919+0000 7f177a600640 1 -- 172.21.15.39:0/539826289 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f177415da70 con 0x7f17740d0430 2024-09-21T07:46:40.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.921+0000 7f1770c00640 1 -- 172.21.15.39:0/539826289 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1760042070 con 0x7f17740d0430 2024-09-21T07:46:40.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.922+0000 7f1770c00640 1 --2- 172.21.15.39:0/539826289 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1754078b90 0x7f175407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:40.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.922+0000 7f1770c00640 1 -- 172.21.15.39:0/539826289 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 6888+0+0 (secure 0 0 0) 0x7f17600baef0 con 0x7f17740d0430 2024-09-21T07:46:40.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.922+0000 7f1779200640 1 --2- 172.21.15.39:0/539826289 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1754078b90 0x7f175407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:40.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.923+0000 7f1779200640 1 --2- 172.21.15.39:0/539826289 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1754078b90 0x7f175407b0a0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f1768009b80 tx=0x7f1768005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:40.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:40.926+0000 7f1770c00640 1 -- 172.21.15.39:0/539826289 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1760049070 con 0x7f17740d0430 2024-09-21T07:46:41.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:41.161+0000 7f177a600640 1 -- 172.21.15.39:0/539826289 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1774156870 con 0x7f17740d0430 2024-09-21T07:46:41.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:41.161+0000 7f1770c00640 1 -- 172.21.15.39:0/539826289 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 190 v190) ==== 95+0+5234 (secure 0 0 0) 0x7f1760082df0 con 0x7f17740d0430 2024-09-21T07:46:41.163 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:46:41.164 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":190,"btime":"2024-09-21T07:46:29:608115+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7493,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/4129342618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4129342618},{"type":"v1","addr":"172.21.15.39:6834","nonce":4129342618}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":185},{"gid":7499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1125302960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1125302960},{"type":"v1","addr":"172.21.15.148:6836","nonce":1125302960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":185},{"gid":8431,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/585545787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":585545787},{"type":"v1","addr":"172.21.15.148:6837","nonce":585545787}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":185}],"filesystems":[{"mdsmap":{"epoch":190,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:46:26.100496+0000","modified":"2024-09-21T07:46:29.115340+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9180},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9180":{"gid":9180,"name":"c","rank":0,"incarnation":186,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/3000703195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3000703195},{"type":"v1","addr":"172.21.15.39:6835","nonce":3000703195}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9180,"qdb_cluster":[9180]},"id":9}]} 2024-09-21T07:46:41.164 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 190 2024-09-21T07:46:41.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:41.164+0000 7f177a600640 1 -- 172.21.15.39:0/539826289 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1754078b90 msgr2=0x7f175407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:41.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:41.164+0000 7f177a600640 1 --2- 172.21.15.39:0/539826289 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1754078b90 0x7f175407b0a0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f1768009b80 tx=0x7f1768005f90 comp rx=0 tx=0).stop 2024-09-21T07:46:41.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:41.164+0000 7f177a600640 1 -- 172.21.15.39:0/539826289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17740d0430 msgr2=0x7f1774155e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:41.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:41.164+0000 7f177a600640 1 --2- 172.21.15.39:0/539826289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17740d0430 0x7f1774155e90 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f1760009950 tx=0x7f1760003b60 comp rx=0 tx=0).stop 2024-09-21T07:46:41.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:41.164+0000 7f177a600640 1 -- 172.21.15.39:0/539826289 shutdown_connections 2024-09-21T07:46:41.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:41.164+0000 7f177a600640 1 --2- 172.21.15.39:0/539826289 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1754078b90 0x7f175407b0a0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:41.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:41.164+0000 7f177a600640 1 --2- 172.21.15.39:0/539826289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1774160110 0x7f17740af750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:41.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:41.164+0000 7f177a600640 1 --2- 172.21.15.39:0/539826289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17740d0430 0x7f1774155e90 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:41.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:41.164+0000 7f177a600640 1 --2- 172.21.15.39:0/539826289 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f17740cf5e0 0x7f1774148e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:41.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:41.164+0000 7f177a600640 1 -- 172.21.15.39:0/539826289 >> 172.21.15.39:0/539826289 conn(0x7f177406cfc0 msgr2=0x7f177415e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:41.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:41.164+0000 7f177a600640 1 -- 172.21.15.39:0/539826289 shutdown_connections 2024-09-21T07:46:41.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:41.164+0000 7f177a600640 1 -- 172.21.15.39:0/539826289 wait complete. 2024-09-21T07:46:41.188 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json config set mds_max_snaps_per_dir 98 2024-09-21T07:46:41.430 INFO:teuthology.orchestra.run.smithi039.stdout:{"success":"mds_max_snaps_per_dir = '' "} 2024-09-21T07:46:41.431 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_max_snaps_per_dir = '' " } 2024-09-21T07:46:41.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:41.431 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir accounts/.snap/s_100) 2024-09-21T07:46:41.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:46:41.470 INFO:teuthology.orchestra.run.smithi039.stderr:mkdir: cannot create directory ‘accounts/.snap/s_100’: Too many links 2024-09-21T07:46:41.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:41.471 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_1) 2024-09-21T07:46:41.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:41.556 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_2) 2024-09-21T07:46:41.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:41.641 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_3) 2024-09-21T07:46:41.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:41.726 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_4) 2024-09-21T07:46:41.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:41.810 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_5) 2024-09-21T07:46:41.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:41.894 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_6) 2024-09-21T07:46:41.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:41.979 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_7) 2024-09-21T07:46:42.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:42.063 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_8) 2024-09-21T07:46:42.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:42.149 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_9) 2024-09-21T07:46:42.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:42.233 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_10) 2024-09-21T07:46:42.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:42.317 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_11) 2024-09-21T07:46:42.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:42.400 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_12) 2024-09-21T07:46:42.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:42.484 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_13) 2024-09-21T07:46:42.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:42.569 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_14) 2024-09-21T07:46:42.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:42.654 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_15) 2024-09-21T07:46:42.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:42.738 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_16) 2024-09-21T07:46:42.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:42.822 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_17) 2024-09-21T07:46:42.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:42.907 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_18) 2024-09-21T07:46:42.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:42.991 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_19) 2024-09-21T07:46:43.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:43.075 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_20) 2024-09-21T07:46:43.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:43.160 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_21) 2024-09-21T07:46:43.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:43.243 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_22) 2024-09-21T07:46:43.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:43.329 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_23) 2024-09-21T07:46:43.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:43.413 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_24) 2024-09-21T07:46:43.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:43.498 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_25) 2024-09-21T07:46:43.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:43.583 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_26) 2024-09-21T07:46:43.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:43.667 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_27) 2024-09-21T07:46:43.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:43.750 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_28) 2024-09-21T07:46:43.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:43.835 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_29) 2024-09-21T07:46:43.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:43.919 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_30) 2024-09-21T07:46:44.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:44.007 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_31) 2024-09-21T07:46:44.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:44.051 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_32) 2024-09-21T07:46:44.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:44.136 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_33) 2024-09-21T07:46:44.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:44.220 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_34) 2024-09-21T07:46:44.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:44.304 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_35) 2024-09-21T07:46:44.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:44.387 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_36) 2024-09-21T07:46:44.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:44.471 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_37) 2024-09-21T07:46:44.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:44.555 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_38) 2024-09-21T07:46:44.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:44.639 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_39) 2024-09-21T07:46:44.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:44.724 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_40) 2024-09-21T07:46:44.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:44.807 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_41) 2024-09-21T07:46:44.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:44.891 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_42) 2024-09-21T07:46:44.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:44.975 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_43) 2024-09-21T07:46:45.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:45.059 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_44) 2024-09-21T07:46:45.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:45.141 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_45) 2024-09-21T07:46:45.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:45.225 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_46) 2024-09-21T07:46:45.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:45.308 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_47) 2024-09-21T07:46:45.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:45.391 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_48) 2024-09-21T07:46:45.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:45.474 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_49) 2024-09-21T07:46:45.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:45.558 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_50) 2024-09-21T07:46:45.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:45.642 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_51) 2024-09-21T07:46:45.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:45.727 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_52) 2024-09-21T07:46:45.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:45.810 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_53) 2024-09-21T07:46:45.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:45.895 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_54) 2024-09-21T07:46:45.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:45.979 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_55) 2024-09-21T07:46:46.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:46.062 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_56) 2024-09-21T07:46:46.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:46.145 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_57) 2024-09-21T07:46:46.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:46.227 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_58) 2024-09-21T07:46:46.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:46.310 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_59) 2024-09-21T07:46:46.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:46.394 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_60) 2024-09-21T07:46:46.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:46.477 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_61) 2024-09-21T07:46:46.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:46.561 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_62) 2024-09-21T07:46:46.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:46.645 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_63) 2024-09-21T07:46:46.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:46.729 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_64) 2024-09-21T07:46:46.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:46.812 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_65) 2024-09-21T07:46:46.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:46.897 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_66) 2024-09-21T07:46:47.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:47.027 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_67) 2024-09-21T07:46:47.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:47.108 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_68) 2024-09-21T07:46:47.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:47.170 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_69) 2024-09-21T07:46:47.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:47.226 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_70) 2024-09-21T07:46:47.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:47.303 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_71) 2024-09-21T07:46:47.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:47.371 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_72) 2024-09-21T07:46:47.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:47.445 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_73) 2024-09-21T07:46:47.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:47.538 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_74) 2024-09-21T07:46:47.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:47.629 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_75) 2024-09-21T07:46:47.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:47.710 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_76) 2024-09-21T07:46:47.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:47.775 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_77) 2024-09-21T07:46:47.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:47.836 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_78) 2024-09-21T07:46:47.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:47.878 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_79) 2024-09-21T07:46:47.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:47.976 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_80) 2024-09-21T07:46:48.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:48.019 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_81) 2024-09-21T07:46:48.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:48.101 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_82) 2024-09-21T07:46:48.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:48.184 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_83) 2024-09-21T07:46:48.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:48.275 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_84) 2024-09-21T07:46:48.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:48.317 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_85) 2024-09-21T07:46:48.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:48.400 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_86) 2024-09-21T07:46:48.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:48.484 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_87) 2024-09-21T07:46:48.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:48.566 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_88) 2024-09-21T07:46:48.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:48.648 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_89) 2024-09-21T07:46:48.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:48.731 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_90) 2024-09-21T07:46:48.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:48.813 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_91) 2024-09-21T07:46:48.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:48.897 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_92) 2024-09-21T07:46:49.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:49.005 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_93) 2024-09-21T07:46:49.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:49.050 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_94) 2024-09-21T07:46:49.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:49.134 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_95) 2024-09-21T07:46:49.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:49.217 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_96) 2024-09-21T07:46:49.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:49.300 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_97) 2024-09-21T07:46:49.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:49.385 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_98) 2024-09-21T07:46:49.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:49.468 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts/.snap/s_99) 2024-09-21T07:46:49.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:49.554 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir accounts) 2024-09-21T07:46:49.632 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:46:49.632 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:46:49.764 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:46:49.854 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:46:49.854 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:46:49.854 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:46:49.854 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:46:49.890 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:46:49.891 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:46:49.997 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:46:50.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.254+0000 7f3002a00640 1 -- 172.21.15.39:0/3768136365 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ffc0cd590 msgr2=0x7f2ffc0cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:50.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.254+0000 7f3002a00640 1 --2- 172.21.15.39:0/3768136365 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ffc0cd590 0x7f2ffc0cd9a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4009900 tx=0x7f2fe402a600 comp rx=0 tx=0).stop 2024-09-21T07:46:50.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.255+0000 7f3002a00640 1 -- 172.21.15.39:0/3768136365 shutdown_connections 2024-09-21T07:46:50.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.255+0000 7f3002a00640 1 --2- 172.21.15.39:0/3768136365 >> v2:172.21.15.39:3300/0 conn(0x7f2ffc072220 0x7f2ffc0d0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:50.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.255+0000 7f3002a00640 1 --2- 172.21.15.39:0/3768136365 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ffc0cd590 0x7f2ffc0cd9a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:50.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.255+0000 7f3002a00640 1 -- 172.21.15.39:0/3768136365 >> 172.21.15.39:0/3768136365 conn(0x7f2ffc0cefe0 msgr2=0x7f2ffc0cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:50.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.255+0000 7f3002a00640 1 -- 172.21.15.39:0/3768136365 shutdown_connections 2024-09-21T07:46:50.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.255+0000 7f3002a00640 1 -- 172.21.15.39:0/3768136365 wait complete. 2024-09-21T07:46:50.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.258+0000 7f3002a00640 1 Processor -- start 2024-09-21T07:46:50.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.259+0000 7f3002a00640 1 -- start start 2024-09-21T07:46:50.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.263+0000 7f3002a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ffc072220 0x7f2ffc166050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:50.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.263+0000 7f3002a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ffc0cd590 0x7f2ffc1fd830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:50.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.263+0000 7f3002a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ffc0d0130 0x7f2ffc09b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:50.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.263+0000 7f3002a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc0c5d70 con 0x7f2ffc0cd590 2024-09-21T07:46:50.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.263+0000 7f3002a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc0c5bf0 con 0x7f2ffc0d0130 2024-09-21T07:46:50.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.263+0000 7f3002a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f2ffc167230 con 0x7f2ffc072220 2024-09-21T07:46:50.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.263+0000 7f2ff3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ffc0cd590 0x7f2ffc1fd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:50.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.263+0000 7f2ff3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ffc0cd590 0x7f2ffc1fd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46618/0 (socket says 172.21.15.39:46618) 2024-09-21T07:46:50.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.263+0000 7f2ff3e00640 1 -- 172.21.15.39:0/1801712544 learned_addr learned my addr 172.21.15.39:0/1801712544 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:50.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.264+0000 7f2ffbe00640 1 --2- 172.21.15.39:0/1801712544 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ffc072220 0x7f2ffc166050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:50.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.264+0000 7f2ff3e00640 1 -- 172.21.15.39:0/1801712544 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ffc072220 msgr2=0x7f2ffc166050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:50.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.264+0000 7f2ff3e00640 1 --2- 172.21.15.39:0/1801712544 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ffc072220 0x7f2ffc166050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:50.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.264+0000 7f2ff3e00640 1 -- 172.21.15.39:0/1801712544 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ffc0d0130 msgr2=0x7f2ffc09b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:50.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.264+0000 7f3000a00640 1 --2- 172.21.15.39:0/1801712544 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ffc0d0130 0x7f2ffc09b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:50.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.265+0000 7f2ff3e00640 1 --2- 172.21.15.39:0/1801712544 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ffc0d0130 0x7f2ffc09b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:50.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.265+0000 7f2ff3e00640 1 -- 172.21.15.39:0/1801712544 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ffc1fe560 con 0x7f2ffc0cd590 2024-09-21T07:46:50.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.265+0000 7f3000a00640 1 --2- 172.21.15.39:0/1801712544 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ffc0d0130 0x7f2ffc09b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:46:50.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.265+0000 7f2ff3e00640 1 --2- 172.21.15.39:0/1801712544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ffc0cd590 0x7f2ffc1fd830 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00c3a0 tx=0x7f2fec00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:50.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.265+0000 7f2ff8c00640 1 -- 172.21.15.39:0/1801712544 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec017070 con 0x7f2ffc0cd590 2024-09-21T07:46:50.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.266+0000 7f2ff8c00640 1 -- 172.21.15.39:0/1801712544 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2fec0053c0 con 0x7f2ffc0cd590 2024-09-21T07:46:50.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.266+0000 7f2ff8c00640 1 -- 172.21.15.39:0/1801712544 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec003850 con 0x7f2ffc0cd590 2024-09-21T07:46:50.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.266+0000 7f3002a00640 1 -- 172.21.15.39:0/1801712544 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ffc0b5cd0 con 0x7f2ffc0cd590 2024-09-21T07:46:50.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.267+0000 7f3002a00640 1 -- 172.21.15.39:0/1801712544 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ffc0b6290 con 0x7f2ffc0cd590 2024-09-21T07:46:50.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.269+0000 7f3002a00640 1 -- 172.21.15.39:0/1801712544 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ffc0cd9a0 con 0x7f2ffc0cd590 2024-09-21T07:46:50.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.270+0000 7f2ff8c00640 1 -- 172.21.15.39:0/1801712544 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2fec01a070 con 0x7f2ffc0cd590 2024-09-21T07:46:50.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.273+0000 7f2ff8c00640 1 --2- 172.21.15.39:0/1801712544 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2fd8078be0 0x7f2fd807b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:50.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.273+0000 7f2ff8c00640 1 -- 172.21.15.39:0/1801712544 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(179..179 src has 1..179) ==== 6976+0+0 (secure 0 0 0) 0x7f2fec09a2e0 con 0x7f2ffc0cd590 2024-09-21T07:46:50.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.274+0000 7f2ffbe00640 1 --2- 172.21.15.39:0/1801712544 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2fd8078be0 0x7f2fd807b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:50.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.276+0000 7f2ffbe00640 1 --2- 172.21.15.39:0/1801712544 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2fd8078be0 0x7f2fd807b0f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4033000 tx=0x7f2fe4004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:50.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.282+0000 7f2ff8c00640 1 -- 172.21.15.39:0/1801712544 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2fec061e00 con 0x7f2ffc0cd590 2024-09-21T07:46:50.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.508+0000 7f3002a00640 1 -- 172.21.15.39:0/1801712544 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2ffc073000 con 0x7f2ffc0cd590 2024-09-21T07:46:50.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.509+0000 7f2ff8c00640 1 -- 172.21.15.39:0/1801712544 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v179) ==== 80+0+0 (secure 0 0 0) 0x7f2fec066830 con 0x7f2ffc0cd590 2024-09-21T07:46:50.511 INFO:teuthology.orchestra.run.smithi039.stderr:listed 0 entries 2024-09-21T07:46:50.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.511+0000 7f3002a00640 1 -- 172.21.15.39:0/1801712544 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2fd8078be0 msgr2=0x7f2fd807b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:50.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.511+0000 7f3002a00640 1 --2- 172.21.15.39:0/1801712544 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2fd8078be0 0x7f2fd807b0f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4033000 tx=0x7f2fe4004730 comp rx=0 tx=0).stop 2024-09-21T07:46:50.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.511+0000 7f3002a00640 1 -- 172.21.15.39:0/1801712544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ffc0cd590 msgr2=0x7f2ffc1fd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:50.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.511+0000 7f3002a00640 1 --2- 172.21.15.39:0/1801712544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ffc0cd590 0x7f2ffc1fd830 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00c3a0 tx=0x7f2fec00c860 comp rx=0 tx=0).stop 2024-09-21T07:46:50.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.512+0000 7f3002a00640 1 -- 172.21.15.39:0/1801712544 shutdown_connections 2024-09-21T07:46:50.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.512+0000 7f3002a00640 1 --2- 172.21.15.39:0/1801712544 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2fd8078be0 0x7f2fd807b0f0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:50.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.512+0000 7f3002a00640 1 --2- 172.21.15.39:0/1801712544 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ffc0d0130 0x7f2ffc09b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:50.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.512+0000 7f3002a00640 1 --2- 172.21.15.39:0/1801712544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ffc0cd590 0x7f2ffc1fd830 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:50.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.512+0000 7f3002a00640 1 --2- 172.21.15.39:0/1801712544 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ffc072220 0x7f2ffc166050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:50.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.512+0000 7f3002a00640 1 -- 172.21.15.39:0/1801712544 >> 172.21.15.39:0/1801712544 conn(0x7f2ffc0cefe0 msgr2=0x7f2ffc0b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:50.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.512+0000 7f3002a00640 1 -- 172.21.15.39:0/1801712544 shutdown_connections 2024-09-21T07:46:50.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.512+0000 7f3002a00640 1 -- 172.21.15.39:0/1801712544 wait complete. 2024-09-21T07:46:50.537 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T07:46:50.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:50.537 DEBUG:teuthology.orchestra.run.smithi039:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T07:46:50.604 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T07:46:50.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:50.604 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:46:50.641 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:46:50.641 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:46:50.641 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:46:50.676 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:46:50.677 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:46:50.677 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:46:50.677 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:46:50.717 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:46:50.717 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:46:50.717 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:46:50.717 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:46:50.752 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:46:50.752 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:46:50.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.991+0000 7f50af200640 1 -- 172.21.15.39:0/2932721161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50b00cef70 msgr2=0x7f50b00cf380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:50.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.991+0000 7f50af200640 1 --2- 172.21.15.39:0/2932721161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50b00cef70 0x7f50b00cf380 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f50b008c880 tx=0x7f50a802aa30 comp rx=0 tx=0).stop 2024-09-21T07:46:50.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.991+0000 7f50af200640 1 -- 172.21.15.39:0/2932721161 shutdown_connections 2024-09-21T07:46:50.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.991+0000 7f50af200640 1 --2- 172.21.15.39:0/2932721161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50b00cef70 0x7f50b00cf380 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:50.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.991+0000 7f50af200640 1 --2- 172.21.15.39:0/2932721161 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f50b00d0e70 0x7f50b00d1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:50.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.991+0000 7f50af200640 1 --2- 172.21.15.39:0/2932721161 >> v2:172.21.15.148:3300/0 conn(0x7f50b00b5a90 0x7f50b00d1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:50.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.991+0000 7f50af200640 1 -- 172.21.15.39:0/2932721161 >> 172.21.15.39:0/2932721161 conn(0x7f50b009d7b0 msgr2=0x7f50b00d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:50.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.991+0000 7f50af200640 1 -- 172.21.15.39:0/2932721161 shutdown_connections 2024-09-21T07:46:50.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.991+0000 7f50af200640 1 -- 172.21.15.39:0/2932721161 wait complete. 2024-09-21T07:46:50.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.994+0000 7f50af200640 1 Processor -- start 2024-09-21T07:46:50.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.994+0000 7f50af200640 1 -- start start 2024-09-21T07:46:50.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.995+0000 7f50af200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50b00b5a90 0x7f50b01f56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:50.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.995+0000 7f50af200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f50b00cef70 0x7f50b015caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:50.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.995+0000 7f50af200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50b00d0e70 0x7f50b00ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:50.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.995+0000 7f50af200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f50b00cf8c0 con 0x7f50b00d0e70 2024-09-21T07:46:50.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.995+0000 7f50af200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f50b00d2240 con 0x7f50b00b5a90 2024-09-21T07:46:50.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.995+0000 7f50af200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f50b00cfa40 con 0x7f50b00cef70 2024-09-21T07:46:50.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.996+0000 7f50ae800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50b00d0e70 0x7f50b00ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:50.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.996+0000 7f50a7e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f50b00cef70 0x7f50b015caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:50.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.996+0000 7f50ae800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50b00d0e70 0x7f50b00ccd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46640/0 (socket says 172.21.15.39:46640) 2024-09-21T07:46:50.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.996+0000 7f50ae800640 1 -- 172.21.15.39:0/3122308877 learned_addr learned my addr 172.21.15.39:0/3122308877 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:50.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.996+0000 7f50ade00640 1 --2- 172.21.15.39:0/3122308877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50b00b5a90 0x7f50b01f56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:50.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.996+0000 7f50ae800640 1 -- 172.21.15.39:0/3122308877 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f50b00cef70 msgr2=0x7f50b015caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:50.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.996+0000 7f50ae800640 1 --2- 172.21.15.39:0/3122308877 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f50b00cef70 0x7f50b015caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:50.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.996+0000 7f50ae800640 1 -- 172.21.15.39:0/3122308877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50b00b5a90 msgr2=0x7f50b01f56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:51.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.996+0000 7f50ae800640 1 --2- 172.21.15.39:0/3122308877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50b00b5a90 0x7f50b01f56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:51.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.996+0000 7f50ae800640 1 -- 172.21.15.39:0/3122308877 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50b01e9de0 con 0x7f50b00d0e70 2024-09-21T07:46:51.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.996+0000 7f50ae800640 1 --2- 172.21.15.39:0/3122308877 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50b00d0e70 0x7f50b00ccd50 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f50a802b020 tx=0x7f50a8003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:51.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.997+0000 7f50a6000640 1 -- 172.21.15.39:0/3122308877 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a8033050 con 0x7f50b00d0e70 2024-09-21T07:46:51.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.997+0000 7f50a6000640 1 -- 172.21.15.39:0/3122308877 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50a8004450 con 0x7f50b00d0e70 2024-09-21T07:46:51.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.997+0000 7f50af200640 1 -- 172.21.15.39:0/3122308877 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50b00cbf90 con 0x7f50b00d0e70 2024-09-21T07:46:51.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.997+0000 7f50a6000640 1 -- 172.21.15.39:0/3122308877 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a8004840 con 0x7f50b00d0e70 2024-09-21T07:46:51.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.997+0000 7f50af200640 1 -- 172.21.15.39:0/3122308877 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50b00cc550 con 0x7f50b00d0e70 2024-09-21T07:46:51.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.998+0000 7f50af200640 1 -- 172.21.15.39:0/3122308877 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50b00cf380 con 0x7f50b00d0e70 2024-09-21T07:46:51.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:50.999+0000 7f50a6000640 1 -- 172.21.15.39:0/3122308877 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f50a8036070 con 0x7f50b00d0e70 2024-09-21T07:46:51.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.000+0000 7f50a6000640 1 --2- 172.21.15.39:0/3122308877 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f508c078bc0 0x7f508c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:51.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.000+0000 7f50a6000640 1 -- 172.21.15.39:0/3122308877 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(180..180 src has 1..180) ==== 6976+0+0 (secure 0 0 0) 0x7f50a80bc630 con 0x7f50b00d0e70 2024-09-21T07:46:51.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.001+0000 7f50ade00640 1 --2- 172.21.15.39:0/3122308877 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f508c078bc0 0x7f508c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:51.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.001+0000 7f50ade00640 1 --2- 172.21.15.39:0/3122308877 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f508c078bc0 0x7f508c07b0d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f50a000bb30 tx=0x7f50a000b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:51.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.005+0000 7f50a6000640 1 -- 172.21.15.39:0/3122308877 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f50a8083eb0 con 0x7f50b00d0e70 2024-09-21T07:46:51.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.260+0000 7f50af200640 1 -- 172.21.15.39:0/3122308877 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50b01e9fe0 con 0x7f50b00d0e70 2024-09-21T07:46:51.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.260+0000 7f50a6000640 1 -- 172.21.15.39:0/3122308877 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 190 v190) ==== 95+0+5234 (secure 0 0 0) 0x7f50a80888e0 con 0x7f50b00d0e70 2024-09-21T07:46:51.263 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:46:51.263 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":190,"btime":"2024-09-21T07:46:29:608115+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7493,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/4129342618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4129342618},{"type":"v1","addr":"172.21.15.39:6834","nonce":4129342618}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":185},{"gid":7499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1125302960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1125302960},{"type":"v1","addr":"172.21.15.148:6836","nonce":1125302960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":185},{"gid":8431,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/585545787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":585545787},{"type":"v1","addr":"172.21.15.148:6837","nonce":585545787}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":185}],"filesystems":[{"mdsmap":{"epoch":190,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:46:26.100496+0000","modified":"2024-09-21T07:46:29.115340+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9180},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9180":{"gid":9180,"name":"c","rank":0,"incarnation":186,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/3000703195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3000703195},{"type":"v1","addr":"172.21.15.39:6835","nonce":3000703195}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9180,"qdb_cluster":[9180]},"id":9}]} 2024-09-21T07:46:51.263 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 190 2024-09-21T07:46:51.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.263+0000 7f50af200640 1 -- 172.21.15.39:0/3122308877 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f508c078bc0 msgr2=0x7f508c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:51.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.263+0000 7f50af200640 1 --2- 172.21.15.39:0/3122308877 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f508c078bc0 0x7f508c07b0d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f50a000bb30 tx=0x7f50a000b2e0 comp rx=0 tx=0).stop 2024-09-21T07:46:51.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.263+0000 7f50af200640 1 -- 172.21.15.39:0/3122308877 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50b00d0e70 msgr2=0x7f50b00ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:51.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.263+0000 7f50af200640 1 --2- 172.21.15.39:0/3122308877 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50b00d0e70 0x7f50b00ccd50 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f50a802b020 tx=0x7f50a8003f10 comp rx=0 tx=0).stop 2024-09-21T07:46:51.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.263+0000 7f50af200640 1 -- 172.21.15.39:0/3122308877 shutdown_connections 2024-09-21T07:46:51.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.263+0000 7f50af200640 1 --2- 172.21.15.39:0/3122308877 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f508c078bc0 0x7f508c07b0d0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:51.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.263+0000 7f50af200640 1 --2- 172.21.15.39:0/3122308877 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50b00d0e70 0x7f50b00ccd50 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:51.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.263+0000 7f50af200640 1 --2- 172.21.15.39:0/3122308877 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f50b00cef70 0x7f50b015caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:51.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.263+0000 7f50af200640 1 --2- 172.21.15.39:0/3122308877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50b00b5a90 0x7f50b01f56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:51.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.263+0000 7f50af200640 1 -- 172.21.15.39:0/3122308877 >> 172.21.15.39:0/3122308877 conn(0x7f50b009d7b0 msgr2=0x7f50b00c7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:51.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.263+0000 7f50af200640 1 -- 172.21.15.39:0/3122308877 shutdown_connections 2024-09-21T07:46:51.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.263+0000 7f50af200640 1 -- 172.21.15.39:0/3122308877 wait complete. 2024-09-21T07:46:51.296 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:46:51.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.527+0000 7f9151200640 1 -- 172.21.15.39:0/2588337268 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9130004d40 msgr2=0x7f9130005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:51.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.527+0000 7f9151200640 1 --2- 172.21.15.39:0/2588337268 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9130004d40 0x7f9130005190 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f9144009ed0 tx=0x7f9130005830 comp rx=0 tx=0).stop 2024-09-21T07:46:51.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.527+0000 7f914be00640 1 -- 172.21.15.39:0/2588337268 reap_dead start 2024-09-21T07:46:51.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.527+0000 7f9151200640 1 -- 172.21.15.39:0/2588337268 shutdown_connections 2024-09-21T07:46:51.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.527+0000 7f9151200640 1 -- 172.21.15.39:0/2588337268 >> 172.21.15.39:0/2588337268 conn(0x7f914c06cfc0 msgr2=0x7f914c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:51.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.527+0000 7f9151200640 1 -- 172.21.15.39:0/2588337268 shutdown_connections 2024-09-21T07:46:51.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.527+0000 7f9151200640 1 -- 172.21.15.39:0/2588337268 wait complete. 2024-09-21T07:46:51.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.529+0000 7f9151200640 1 Processor -- start 2024-09-21T07:46:51.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.529+0000 7f9151200640 1 -- start start 2024-09-21T07:46:51.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.531+0000 7f9151200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9130004d40 0x7f914c1e6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:51.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.531+0000 7f9151200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f914c1f5070 0x7f914c14ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:51.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.531+0000 7f9151200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f914c1e4670 0x7f914c1e4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:51.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.531+0000 7f9151200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f914c15d3a0 con 0x7f914c1e4670 2024-09-21T07:46:51.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.531+0000 7f9151200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f914c15d220 con 0x7f9130004d40 2024-09-21T07:46:51.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.531+0000 7f9151200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f914c06ecf0 con 0x7f914c1f5070 2024-09-21T07:46:51.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.531+0000 7f914be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f914c1e4670 0x7f914c1e4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:51.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.531+0000 7f914b400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9130004d40 0x7f914c1e6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:51.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.531+0000 7f9143e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f914c1f5070 0x7f914c14ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:51.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.531+0000 7f914b400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9130004d40 0x7f914c1e6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.39:33036/0 (socket says 172.21.15.39:33036) 2024-09-21T07:46:51.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.531+0000 7f914b400640 1 -- 172.21.15.39:0/5309090 learned_addr learned my addr 172.21.15.39:0/5309090 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:51.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.531+0000 7f9143e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f914c1f5070 0x7f914c14ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.39:59728/0 (socket says 172.21.15.39:59728) 2024-09-21T07:46:51.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.532+0000 7f914be00640 1 -- 172.21.15.39:0/5309090 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f914c1f5070 msgr2=0x7f914c14ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:51.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.532+0000 7f914be00640 1 --2- 172.21.15.39:0/5309090 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f914c1f5070 0x7f914c14ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:51.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.532+0000 7f914be00640 1 -- 172.21.15.39:0/5309090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9130004d40 msgr2=0x7f914c1e6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:51.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.532+0000 7f914be00640 1 --2- 172.21.15.39:0/5309090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9130004d40 0x7f914c1e6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:51.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.532+0000 7f914be00640 1 -- 172.21.15.39:0/5309090 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f914c1e4fc0 con 0x7f914c1e4670 2024-09-21T07:46:51.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.532+0000 7f914be00640 1 --2- 172.21.15.39:0/5309090 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f914c1e4670 0x7f914c1e4a80 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f9144032000 tx=0x7f9144033c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:51.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.532+0000 7f9143400640 1 -- 172.21.15.39:0/5309090 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9144043070 con 0x7f914c1e4670 2024-09-21T07:46:51.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.532+0000 7f9143400640 1 -- 172.21.15.39:0/5309090 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9144033ce0 con 0x7f914c1e4670 2024-09-21T07:46:51.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.532+0000 7f9151200640 1 -- 172.21.15.39:0/5309090 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f914c1e6d50 con 0x7f914c1e4670 2024-09-21T07:46:51.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.532+0000 7f9143400640 1 -- 172.21.15.39:0/5309090 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9144038040 con 0x7f914c1e4670 2024-09-21T07:46:51.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.533+0000 7f9151200640 1 -- 172.21.15.39:0/5309090 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f914c1f59d0 con 0x7f914c1e4670 2024-09-21T07:46:51.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.534+0000 7f9151200640 1 -- 172.21.15.39:0/5309090 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f914c06f250 con 0x7f914c1e4670 2024-09-21T07:46:51.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.535+0000 7f9143400640 1 -- 172.21.15.39:0/5309090 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9144003bd0 con 0x7f914c1e4670 2024-09-21T07:46:51.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.536+0000 7f9143400640 1 --2- 172.21.15.39:0/5309090 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9128078b90 0x7f912807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:51.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.537+0000 7f9143400640 1 -- 172.21.15.39:0/5309090 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(180..180 src has 1..180) ==== 6976+0+0 (secure 0 0 0) 0x7f91440bb160 con 0x7f914c1e4670 2024-09-21T07:46:51.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.537+0000 7f914b400640 1 --2- 172.21.15.39:0/5309090 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9128078b90 0x7f912807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:51.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.538+0000 7f914b400640 1 --2- 172.21.15.39:0/5309090 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9128078b90 0x7f912807b0a0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f913c003790 tx=0x7f913c006b10 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:51.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.541+0000 7f9143400640 1 -- 172.21.15.39:0/5309090 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f914c1e4fc0 con 0x7f914c1e4670 2024-09-21T07:46:51.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.796+0000 7f9151200640 1 -- 172.21.15.39:0/5309090 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f914c1f5480 con 0x7f914c1e4670 2024-09-21T07:46:51.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.796+0000 7f9143400640 1 -- 172.21.15.39:0/5309090 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 190 v190) ==== 95+0+5234 (secure 0 0 0) 0x7f9144082a60 con 0x7f914c1e4670 2024-09-21T07:46:51.798 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:46:51.798 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":190,"btime":"2024-09-21T07:46:29:608115+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7493,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/4129342618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4129342618},{"type":"v1","addr":"172.21.15.39:6834","nonce":4129342618}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":185},{"gid":7499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1125302960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1125302960},{"type":"v1","addr":"172.21.15.148:6836","nonce":1125302960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":185},{"gid":8431,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/585545787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":585545787},{"type":"v1","addr":"172.21.15.148:6837","nonce":585545787}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":185}],"filesystems":[{"mdsmap":{"epoch":190,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:46:26.100496+0000","modified":"2024-09-21T07:46:29.115340+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9180},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9180":{"gid":9180,"name":"c","rank":0,"incarnation":186,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/3000703195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3000703195},{"type":"v1","addr":"172.21.15.39:6835","nonce":3000703195}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9180,"qdb_cluster":[9180]},"id":9}]} 2024-09-21T07:46:51.799 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 190 2024-09-21T07:46:51.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.799+0000 7f9151200640 1 -- 172.21.15.39:0/5309090 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9128078b90 msgr2=0x7f912807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:51.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.799+0000 7f9151200640 1 --2- 172.21.15.39:0/5309090 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9128078b90 0x7f912807b0a0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f913c003790 tx=0x7f913c006b10 comp rx=0 tx=0).stop 2024-09-21T07:46:51.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.799+0000 7f9151200640 1 -- 172.21.15.39:0/5309090 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f914c1e4670 msgr2=0x7f914c1e4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:51.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.799+0000 7f9151200640 1 --2- 172.21.15.39:0/5309090 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f914c1e4670 0x7f914c1e4a80 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f9144032000 tx=0x7f9144033c70 comp rx=0 tx=0).stop 2024-09-21T07:46:51.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.799+0000 7f9151200640 1 -- 172.21.15.39:0/5309090 shutdown_connections 2024-09-21T07:46:51.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.799+0000 7f9151200640 1 --2- 172.21.15.39:0/5309090 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9128078b90 0x7f912807b0a0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:51.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.799+0000 7f9151200640 1 --2- 172.21.15.39:0/5309090 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f914c1e4670 0x7f914c1e4a80 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:51.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.799+0000 7f9151200640 1 --2- 172.21.15.39:0/5309090 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f914c1f5070 0x7f914c14ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:51.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.799+0000 7f9151200640 1 --2- 172.21.15.39:0/5309090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9130004d40 0x7f914c1e6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:51.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.799+0000 7f9151200640 1 -- 172.21.15.39:0/5309090 >> 172.21.15.39:0/5309090 conn(0x7f914c06cfc0 msgr2=0x7f914c072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:51.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.799+0000 7f9151200640 1 -- 172.21.15.39:0/5309090 shutdown_connections 2024-09-21T07:46:51.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:51.799+0000 7f9151200640 1 -- 172.21.15.39:0/5309090 wait complete. 2024-09-21T07:46:51.822 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:46:52.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.052+0000 7fe623200640 1 -- 172.21.15.39:0/2803119664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe61c0d0630 msgr2=0x7fe61c0d0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:52.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.052+0000 7fe623200640 1 --2- 172.21.15.39:0/2803119664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe61c0d0630 0x7fe61c0d0a40 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe614005fd0 tx=0x7fe61402a5d0 comp rx=0 tx=0).stop 2024-09-21T07:46:52.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.052+0000 7fe623200640 1 -- 172.21.15.39:0/2803119664 shutdown_connections 2024-09-21T07:46:52.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.052+0000 7fe623200640 1 --2- 172.21.15.39:0/2803119664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe61c0d0630 0x7fe61c0d0a40 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:52.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.052+0000 7fe623200640 1 --2- 172.21.15.39:0/2803119664 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe61c0cf450 0x7fe61c0cf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:52.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.052+0000 7fe623200640 1 -- 172.21.15.39:0/2803119664 >> 172.21.15.39:0/2803119664 conn(0x7fe61c06cfc0 msgr2=0x7fe61c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:52.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.052+0000 7fe623200640 1 -- 172.21.15.39:0/2803119664 shutdown_connections 2024-09-21T07:46:52.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.052+0000 7fe623200640 1 -- 172.21.15.39:0/2803119664 wait complete. 2024-09-21T07:46:52.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.054+0000 7fe623200640 1 Processor -- start 2024-09-21T07:46:52.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.054+0000 7fe623200640 1 -- start start 2024-09-21T07:46:52.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.056+0000 7fe623200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe61c073dd0 0x7fe61c148e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:52.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.056+0000 7fe623200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe61c0cf450 0x7fe61c145970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:52.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.056+0000 7fe623200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe61c0d0630 0x7fe61c148230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:52.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.056+0000 7fe623200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c0cd1a0 con 0x7fe61c0cf450 2024-09-21T07:46:52.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.056+0000 7fe623200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c0cd020 con 0x7fe61c073dd0 2024-09-21T07:46:52.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.056+0000 7fe623200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe61c0cd320 con 0x7fe61c0d0630 2024-09-21T07:46:52.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.056+0000 7fe613400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe61c0cf450 0x7fe61c145970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:52.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.056+0000 7fe613400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe61c0cf450 0x7fe61c145970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34724/0 (socket says 172.21.15.39:34724) 2024-09-21T07:46:52.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.056+0000 7fe613400640 1 -- 172.21.15.39:0/3342117515 learned_addr learned my addr 172.21.15.39:0/3342117515 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:52.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.056+0000 7fe621200640 1 --2- 172.21.15.39:0/3342117515 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe61c0d0630 0x7fe61c148230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:52.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.056+0000 7fe61be00640 1 --2- 172.21.15.39:0/3342117515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe61c073dd0 0x7fe61c148e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:52.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.056+0000 7fe613400640 1 -- 172.21.15.39:0/3342117515 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe61c0d0630 msgr2=0x7fe61c148230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:52.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.056+0000 7fe613400640 1 --2- 172.21.15.39:0/3342117515 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe61c0d0630 0x7fe61c148230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:52.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.056+0000 7fe613400640 1 -- 172.21.15.39:0/3342117515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe61c073dd0 msgr2=0x7fe61c148e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:52.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.056+0000 7fe613400640 1 --2- 172.21.15.39:0/3342117515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe61c073dd0 0x7fe61c148e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:52.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.056+0000 7fe613400640 1 -- 172.21.15.39:0/3342117515 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe61c1461d0 con 0x7fe61c0cf450 2024-09-21T07:46:52.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.057+0000 7fe613400640 1 --2- 172.21.15.39:0/3342117515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe61c0cf450 0x7fe61c145970 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fe60400ea20 tx=0x7fe60400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:52.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.057+0000 7fe618c00640 1 -- 172.21.15.39:0/3342117515 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60400cd50 con 0x7fe61c0cf450 2024-09-21T07:46:52.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.057+0000 7fe618c00640 1 -- 172.21.15.39:0/3342117515 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe604004020 con 0x7fe61c0cf450 2024-09-21T07:46:52.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.057+0000 7fe618c00640 1 -- 172.21.15.39:0/3342117515 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe604004450 con 0x7fe61c0cf450 2024-09-21T07:46:52.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.057+0000 7fe623200640 1 -- 172.21.15.39:0/3342117515 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe61c145eb0 con 0x7fe61c0cf450 2024-09-21T07:46:52.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.057+0000 7fe623200640 1 -- 172.21.15.39:0/3342117515 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe61c154c10 con 0x7fe61c0cf450 2024-09-21T07:46:52.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.058+0000 7fe623200640 1 -- 172.21.15.39:0/3342117515 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe61c0af0d0 con 0x7fe61c0cf450 2024-09-21T07:46:52.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.060+0000 7fe618c00640 1 -- 172.21.15.39:0/3342117515 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe6040035f0 con 0x7fe61c0cf450 2024-09-21T07:46:52.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.061+0000 7fe618c00640 1 --2- 172.21.15.39:0/3342117515 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe5f8078bc0 0x7fe5f807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:52.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.061+0000 7fe618c00640 1 -- 172.21.15.39:0/3342117515 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(181..181 src has 1..181) ==== 6960+0+0 (secure 0 0 0) 0x7fe60409bb00 con 0x7fe61c0cf450 2024-09-21T07:46:52.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.061+0000 7fe61be00640 1 --2- 172.21.15.39:0/3342117515 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe5f8078bc0 0x7fe5f807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:52.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.062+0000 7fe61be00640 1 --2- 172.21.15.39:0/3342117515 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe5f8078bc0 0x7fe5f807b0d0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe60c007fd0 tx=0x7fe60c009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:52.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.066+0000 7fe618c00640 1 -- 172.21.15.39:0/3342117515 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe604063390 con 0x7fe61c0cf450 2024-09-21T07:46:52.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.265+0000 7fe623200640 1 -- 172.21.15.39:0/3342117515 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe61c0cd7d0 con 0x7fe61c0cf450 2024-09-21T07:46:52.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.266+0000 7fe618c00640 1 -- 172.21.15.39:0/3342117515 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v181) ==== 74+0+22427 (secure 0 0 0) 0x7fe60409be20 con 0x7fe61c0cf450 2024-09-21T07:46:52.268 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:46:52.269 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":181,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:46:51.928525+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:46:24.067206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-09-21T07:46:25.054230+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":199,"snap_epoch":181,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]},{"pool":20,"snaps":[{"begin":80,"length":23},{"begin":181,"length":23}]},{"pool":22,"snaps":[{"begin":2,"length":198}]}],"new_removed_snaps":[{"pool":22,"snaps":[{"begin":100,"length":1},{"begin":199,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:46:52.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.269+0000 7fe623200640 1 -- 172.21.15.39:0/3342117515 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe5f8078bc0 msgr2=0x7fe5f807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:52.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.269+0000 7fe623200640 1 --2- 172.21.15.39:0/3342117515 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe5f8078bc0 0x7fe5f807b0d0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe60c007fd0 tx=0x7fe60c009040 comp rx=0 tx=0).stop 2024-09-21T07:46:52.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.269+0000 7fe623200640 1 -- 172.21.15.39:0/3342117515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe61c0cf450 msgr2=0x7fe61c145970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:52.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.269+0000 7fe623200640 1 --2- 172.21.15.39:0/3342117515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe61c0cf450 0x7fe61c145970 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fe60400ea20 tx=0x7fe60400eee0 comp rx=0 tx=0).stop 2024-09-21T07:46:52.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.269+0000 7fe623200640 1 -- 172.21.15.39:0/3342117515 shutdown_connections 2024-09-21T07:46:52.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.269+0000 7fe623200640 1 --2- 172.21.15.39:0/3342117515 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe5f8078bc0 0x7fe5f807b0d0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:52.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.269+0000 7fe623200640 1 --2- 172.21.15.39:0/3342117515 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe61c0d0630 0x7fe61c148230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:52.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.269+0000 7fe623200640 1 --2- 172.21.15.39:0/3342117515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe61c0cf450 0x7fe61c145970 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:52.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.269+0000 7fe623200640 1 --2- 172.21.15.39:0/3342117515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe61c073dd0 0x7fe61c148e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:52.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.269+0000 7fe623200640 1 -- 172.21.15.39:0/3342117515 >> 172.21.15.39:0/3342117515 conn(0x7fe61c06cfc0 msgr2=0x7fe61c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:52.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.269+0000 7fe623200640 1 -- 172.21.15.39:0/3342117515 shutdown_connections 2024-09-21T07:46:52.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.269+0000 7fe623200640 1 -- 172.21.15.39:0/3342117515 wait complete. 2024-09-21T07:46:52.293 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T07:46:52.294 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:46:52.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.523+0000 7f7e00200640 1 -- 172.21.15.39:0/1798511163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7df8073dd0 msgr2=0x7f7df815d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:52.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.523+0000 7f7e00200640 1 --2- 172.21.15.39:0/1798511163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7df8073dd0 0x7f7df815d5b0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f7df80ce9d0 tx=0x7f7dec02a560 comp rx=0 tx=0).stop 2024-09-21T07:46:52.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.524+0000 7f7e00200640 1 -- 172.21.15.39:0/1798511163 shutdown_connections 2024-09-21T07:46:52.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.524+0000 7f7e00200640 1 --2- 172.21.15.39:0/1798511163 >> v2:172.21.15.148:3300/0 conn(0x7f7df80cf990 0x7f7df80d0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:52.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.524+0000 7f7e00200640 1 --2- 172.21.15.39:0/1798511163 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7df815daf0 0x7f7df80cf450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:52.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.524+0000 7f7e00200640 1 --2- 172.21.15.39:0/1798511163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7df8073dd0 0x7f7df815d5b0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:52.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.524+0000 7f7e00200640 1 -- 172.21.15.39:0/1798511163 >> 172.21.15.39:0/1798511163 conn(0x7f7df806cfc0 msgr2=0x7f7df806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:52.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.524+0000 7f7e00200640 1 -- 172.21.15.39:0/1798511163 shutdown_connections 2024-09-21T07:46:52.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.524+0000 7f7e00200640 1 -- 172.21.15.39:0/1798511163 wait complete. 2024-09-21T07:46:52.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.526+0000 7f7e00200640 1 Processor -- start 2024-09-21T07:46:52.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.526+0000 7f7e00200640 1 -- start start 2024-09-21T07:46:52.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.528+0000 7f7e00200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7df8073dd0 0x7f7df8151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:52.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.528+0000 7f7e00200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7df80cf990 0x7f7df815b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:52.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.528+0000 7f7e00200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7df815daf0 0x7f7df815bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:52.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.528+0000 7f7e00200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7df8160740 con 0x7f7df8073dd0 2024-09-21T07:46:52.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.528+0000 7f7e00200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7df81605c0 con 0x7f7df815daf0 2024-09-21T07:46:52.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.528+0000 7f7e00200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f7df81608c0 con 0x7f7df80cf990 2024-09-21T07:46:52.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.528+0000 7f7dfd800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7df8073dd0 0x7f7df8151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:52.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.528+0000 7f7dfd800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7df8073dd0 0x7f7df8151ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34750/0 (socket says 172.21.15.39:34750) 2024-09-21T07:46:52.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.528+0000 7f7dfd800640 1 -- 172.21.15.39:0/2697984629 learned_addr learned my addr 172.21.15.39:0/2697984629 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:52.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.528+0000 7f7dfe200640 1 --2- 172.21.15.39:0/2697984629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7df815daf0 0x7f7df815bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:52.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.528+0000 7f7dfce00640 1 --2- 172.21.15.39:0/2697984629 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7df80cf990 0x7f7df815b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:52.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.528+0000 7f7dfd800640 1 -- 172.21.15.39:0/2697984629 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7df80cf990 msgr2=0x7f7df815b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:52.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.528+0000 7f7dfd800640 1 --2- 172.21.15.39:0/2697984629 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7df80cf990 0x7f7df815b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:52.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.528+0000 7f7dfd800640 1 -- 172.21.15.39:0/2697984629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7df815daf0 msgr2=0x7f7df815bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:52.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.528+0000 7f7dfd800640 1 --2- 172.21.15.39:0/2697984629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7df815daf0 0x7f7df815bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:52.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.528+0000 7f7dfd800640 1 -- 172.21.15.39:0/2697984629 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df81f3a30 con 0x7f7df8073dd0 2024-09-21T07:46:52.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.529+0000 7f7dfd800640 1 --2- 172.21.15.39:0/2697984629 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7df8073dd0 0x7f7df8151ea0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f7dec034040 tx=0x7f7dec0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:52.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.529+0000 7f7df5600640 1 -- 172.21.15.39:0/2697984629 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec03e030 con 0x7f7df8073dd0 2024-09-21T07:46:52.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.529+0000 7f7df5600640 1 -- 172.21.15.39:0/2697984629 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7dec003ca0 con 0x7f7df8073dd0 2024-09-21T07:46:52.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.529+0000 7f7df5600640 1 -- 172.21.15.39:0/2697984629 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec035040 con 0x7f7df8073dd0 2024-09-21T07:46:52.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.529+0000 7f7e00200640 1 -- 172.21.15.39:0/2697984629 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df81f3d10 con 0x7f7df8073dd0 2024-09-21T07:46:52.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.529+0000 7f7e00200640 1 -- 172.21.15.39:0/2697984629 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df81f42d0 con 0x7f7df8073dd0 2024-09-21T07:46:52.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.530+0000 7f7e00200640 1 -- 172.21.15.39:0/2697984629 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7df80d0630 con 0x7f7df8073dd0 2024-09-21T07:46:52.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.532+0000 7f7df5600640 1 -- 172.21.15.39:0/2697984629 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7dec041030 con 0x7f7df8073dd0 2024-09-21T07:46:52.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.532+0000 7f7df5600640 1 --2- 172.21.15.39:0/2697984629 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7dd8078bc0 0x7f7dd807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:52.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.533+0000 7f7df5600640 1 -- 172.21.15.39:0/2697984629 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(181..181 src has 1..181) ==== 6960+0+0 (secure 0 0 0) 0x7f7dec0bbaf0 con 0x7f7df8073dd0 2024-09-21T07:46:52.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.533+0000 7f7dfce00640 1 --2- 172.21.15.39:0/2697984629 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7dd8078bc0 0x7f7dd807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:52.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.534+0000 7f7dfce00640 1 --2- 172.21.15.39:0/2697984629 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7dd8078bc0 0x7f7dd807b0d0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f7df80aca40 tx=0x7f7de4005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:52.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.537+0000 7f7df5600640 1 -- 172.21.15.39:0/2697984629 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7dec083410 con 0x7f7df8073dd0 2024-09-21T07:46:52.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.774+0000 7f7e00200640 1 -- 172.21.15.39:0/2697984629 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7df80b2e40 con 0x7f7df8073dd0 2024-09-21T07:46:52.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.775+0000 7f7df5600640 1 -- 172.21.15.39:0/2697984629 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 190 v190) ==== 95+0+5234 (secure 0 0 0) 0x7f7dec087e40 con 0x7f7df8073dd0 2024-09-21T07:46:52.777 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:46:52.777 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":190,"btime":"2024-09-21T07:46:29:608115+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7493,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/4129342618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4129342618},{"type":"v1","addr":"172.21.15.39:6834","nonce":4129342618}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":185},{"gid":7499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1125302960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1125302960},{"type":"v1","addr":"172.21.15.148:6836","nonce":1125302960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":185},{"gid":8431,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/585545787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":585545787},{"type":"v1","addr":"172.21.15.148:6837","nonce":585545787}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":185}],"filesystems":[{"mdsmap":{"epoch":190,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:46:26.100496+0000","modified":"2024-09-21T07:46:29.115340+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9180},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9180":{"gid":9180,"name":"c","rank":0,"incarnation":186,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/3000703195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3000703195},{"type":"v1","addr":"172.21.15.39:6835","nonce":3000703195}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9180,"qdb_cluster":[9180]},"id":9}]} 2024-09-21T07:46:52.777 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 190 2024-09-21T07:46:52.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.778+0000 7f7e00200640 1 -- 172.21.15.39:0/2697984629 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7dd8078bc0 msgr2=0x7f7dd807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:52.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.778+0000 7f7e00200640 1 --2- 172.21.15.39:0/2697984629 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7dd8078bc0 0x7f7dd807b0d0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f7df80aca40 tx=0x7f7de4005f90 comp rx=0 tx=0).stop 2024-09-21T07:46:52.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.778+0000 7f7e00200640 1 -- 172.21.15.39:0/2697984629 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7df8073dd0 msgr2=0x7f7df8151ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:52.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.778+0000 7f7e00200640 1 --2- 172.21.15.39:0/2697984629 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7df8073dd0 0x7f7df8151ea0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f7dec034040 tx=0x7f7dec0038a0 comp rx=0 tx=0).stop 2024-09-21T07:46:52.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.778+0000 7f7e00200640 1 -- 172.21.15.39:0/2697984629 shutdown_connections 2024-09-21T07:46:52.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.778+0000 7f7e00200640 1 --2- 172.21.15.39:0/2697984629 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7dd8078bc0 0x7f7dd807b0d0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:52.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.778+0000 7f7e00200640 1 --2- 172.21.15.39:0/2697984629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7df815daf0 0x7f7df815bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:52.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.778+0000 7f7e00200640 1 --2- 172.21.15.39:0/2697984629 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7df80cf990 0x7f7df815b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:52.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.778+0000 7f7e00200640 1 --2- 172.21.15.39:0/2697984629 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7df8073dd0 0x7f7df8151ea0 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:52.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.778+0000 7f7e00200640 1 -- 172.21.15.39:0/2697984629 >> 172.21.15.39:0/2697984629 conn(0x7f7df806cfc0 msgr2=0x7f7df8154190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:52.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.778+0000 7f7e00200640 1 -- 172.21.15.39:0/2697984629 shutdown_connections 2024-09-21T07:46:52.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:52.778+0000 7f7e00200640 1 -- 172.21.15.39:0/2697984629 wait complete. 2024-09-21T07:46:52.801 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:46:53.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.026+0000 7f477fe00640 1 -- 172.21.15.39:0/2334040727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47800b3b60 msgr2=0x7f47800b3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:53.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.026+0000 7f477fe00640 1 --2- 172.21.15.39:0/2334040727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47800b3b60 0x7f47800b3f70 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f478008c880 tx=0x7f477802aac0 comp rx=0 tx=0).stop 2024-09-21T07:46:53.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.026+0000 7f477fe00640 1 -- 172.21.15.39:0/2334040727 shutdown_connections 2024-09-21T07:46:53.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.026+0000 7f477fe00640 1 --2- 172.21.15.39:0/2334040727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47800b3b60 0x7f47800b3f70 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:53.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.026+0000 7f477fe00640 1 --2- 172.21.15.39:0/2334040727 >> v2:172.21.15.148:3300/0 conn(0x7f47800b5a90 0x7f47800d1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:53.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.026+0000 7f477fe00640 1 -- 172.21.15.39:0/2334040727 >> 172.21.15.39:0/2334040727 conn(0x7f478009d7b0 msgr2=0x7f47800d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:53.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.026+0000 7f477fe00640 1 -- 172.21.15.39:0/2334040727 shutdown_connections 2024-09-21T07:46:53.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.026+0000 7f477fe00640 1 -- 172.21.15.39:0/2334040727 wait complete. 2024-09-21T07:46:53.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.028+0000 7f477fe00640 1 Processor -- start 2024-09-21T07:46:53.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.029+0000 7f477fe00640 1 -- start start 2024-09-21T07:46:53.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.030+0000 7f477fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47800b3b60 0x7f47800cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:53.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.030+0000 7f477fe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47800b5a90 0x7f47801508c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:53.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.030+0000 7f477fe00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47800d0e70 0x7f47800cd060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:53.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.030+0000 7f477fe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f47800b44b0 con 0x7f47800b3b60 2024-09-21T07:46:53.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.030+0000 7f477fe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f47800d2280 con 0x7f47800b5a90 2024-09-21T07:46:53.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.030+0000 7f477fe00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f47800b4630 con 0x7f47800d0e70 2024-09-21T07:46:53.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.030+0000 7f477ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47800b3b60 0x7f47800cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:53.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.030+0000 7f477ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47800b3b60 0x7f47800cbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34772/0 (socket says 172.21.15.39:34772) 2024-09-21T07:46:53.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.030+0000 7f477ea00640 1 -- 172.21.15.39:0/3292048063 learned_addr learned my addr 172.21.15.39:0/3292048063 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:53.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.031+0000 7f477f400640 1 --2- 172.21.15.39:0/3292048063 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47800d0e70 0x7f47800cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:53.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.031+0000 7f477ea00640 1 -- 172.21.15.39:0/3292048063 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47800d0e70 msgr2=0x7f47800cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:53.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.031+0000 7f4776000640 1 --2- 172.21.15.39:0/3292048063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47800b5a90 0x7f47801508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:53.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.031+0000 7f477ea00640 1 --2- 172.21.15.39:0/3292048063 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47800d0e70 0x7f47800cd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:53.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.031+0000 7f477ea00640 1 -- 172.21.15.39:0/3292048063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47800b5a90 msgr2=0x7f47801508c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:53.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.031+0000 7f477ea00640 1 --2- 172.21.15.39:0/3292048063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47800b5a90 0x7f47801508c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:53.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.031+0000 7f477ea00640 1 -- 172.21.15.39:0/3292048063 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4780150e00 con 0x7f47800b3b60 2024-09-21T07:46:53.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.031+0000 7f4776000640 1 --2- 172.21.15.39:0/3292048063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47800b5a90 0x7f47801508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:46:53.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.031+0000 7f477ea00640 1 --2- 172.21.15.39:0/3292048063 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47800b3b60 0x7f47800cbf20 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f477000d980 tx=0x7f477000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:53.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.031+0000 7f4777400640 1 -- 172.21.15.39:0/3292048063 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4770009c30 con 0x7f47800b3b60 2024-09-21T07:46:53.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.031+0000 7f4777400640 1 -- 172.21.15.39:0/3292048063 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4770004020 con 0x7f47800b3b60 2024-09-21T07:46:53.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.031+0000 7f477fe00640 1 -- 172.21.15.39:0/3292048063 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f478015b960 con 0x7f47800b3b60 2024-09-21T07:46:53.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.031+0000 7f4777400640 1 -- 172.21.15.39:0/3292048063 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4770004450 con 0x7f47800b3b60 2024-09-21T07:46:53.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.031+0000 7f477fe00640 1 -- 172.21.15.39:0/3292048063 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47800cba00 con 0x7f47800b3b60 2024-09-21T07:46:53.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.032+0000 7f477fe00640 1 -- 172.21.15.39:0/3292048063 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47800cb510 con 0x7f47800b3b60 2024-09-21T07:46:53.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.034+0000 7f4777400640 1 -- 172.21.15.39:0/3292048063 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4770005030 con 0x7f47800b3b60 2024-09-21T07:46:53.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.035+0000 7f4777400640 1 --2- 172.21.15.39:0/3292048063 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f475c078b90 0x7f475c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:53.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.035+0000 7f4777400640 1 -- 172.21.15.39:0/3292048063 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(181..181 src has 1..181) ==== 6960+0+0 (secure 0 0 0) 0x7f477009b860 con 0x7f47800b3b60 2024-09-21T07:46:53.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.035+0000 7f4776000640 1 --2- 172.21.15.39:0/3292048063 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f475c078b90 0x7f475c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:53.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.036+0000 7f4776000640 1 --2- 172.21.15.39:0/3292048063 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f475c078b90 0x7f475c07b0a0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f47800a23e0 tx=0x7f4768009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:53.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.039+0000 7f4777400640 1 -- 172.21.15.39:0/3292048063 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4770020020 con 0x7f47800b3b60 2024-09-21T07:46:53.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.272+0000 7f477fe00640 1 -- 172.21.15.39:0/3292048063 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47800b3f70 con 0x7f47800b3b60 2024-09-21T07:46:53.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.272+0000 7f4777400640 1 -- 172.21.15.39:0/3292048063 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 190 v190) ==== 95+0+5234 (secure 0 0 0) 0x7f4780150e00 con 0x7f47800b3b60 2024-09-21T07:46:53.274 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:46:53.275 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":190,"btime":"2024-09-21T07:46:29:608115+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7493,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/4129342618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4129342618},{"type":"v1","addr":"172.21.15.39:6834","nonce":4129342618}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":185},{"gid":7499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1125302960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1125302960},{"type":"v1","addr":"172.21.15.148:6836","nonce":1125302960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":185},{"gid":8431,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/585545787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":585545787},{"type":"v1","addr":"172.21.15.148:6837","nonce":585545787}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":185}],"filesystems":[{"mdsmap":{"epoch":190,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:46:26.100496+0000","modified":"2024-09-21T07:46:29.115340+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9180},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9180":{"gid":9180,"name":"c","rank":0,"incarnation":186,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/3000703195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3000703195},{"type":"v1","addr":"172.21.15.39:6835","nonce":3000703195}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9180,"qdb_cluster":[9180]},"id":9}]} 2024-09-21T07:46:53.275 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 190 2024-09-21T07:46:53.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.275+0000 7f477fe00640 1 -- 172.21.15.39:0/3292048063 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f475c078b90 msgr2=0x7f475c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:53.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.275+0000 7f477fe00640 1 --2- 172.21.15.39:0/3292048063 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f475c078b90 0x7f475c07b0a0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f47800a23e0 tx=0x7f4768009040 comp rx=0 tx=0).stop 2024-09-21T07:46:53.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.275+0000 7f477fe00640 1 -- 172.21.15.39:0/3292048063 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47800b3b60 msgr2=0x7f47800cbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:53.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.275+0000 7f477fe00640 1 --2- 172.21.15.39:0/3292048063 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47800b3b60 0x7f47800cbf20 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f477000d980 tx=0x7f477000de40 comp rx=0 tx=0).stop 2024-09-21T07:46:53.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.275+0000 7f477fe00640 1 -- 172.21.15.39:0/3292048063 shutdown_connections 2024-09-21T07:46:53.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.275+0000 7f477fe00640 1 --2- 172.21.15.39:0/3292048063 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f475c078b90 0x7f475c07b0a0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:53.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.275+0000 7f477fe00640 1 --2- 172.21.15.39:0/3292048063 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47800d0e70 0x7f47800cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:53.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.275+0000 7f477fe00640 1 --2- 172.21.15.39:0/3292048063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47800b5a90 0x7f47801508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:53.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.275+0000 7f477fe00640 1 --2- 172.21.15.39:0/3292048063 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47800b3b60 0x7f47800cbf20 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:53.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.275+0000 7f477fe00640 1 -- 172.21.15.39:0/3292048063 >> 172.21.15.39:0/3292048063 conn(0x7f478009d7b0 msgr2=0x7f47800b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:53.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.275+0000 7f477fe00640 1 -- 172.21.15.39:0/3292048063 shutdown_connections 2024-09-21T07:46:53.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.275+0000 7f477fe00640 1 -- 172.21.15.39:0/3292048063 wait complete. 2024-09-21T07:46:53.299 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:46:53.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.524+0000 7fd814a00640 1 -- 172.21.15.39:0/1677422954 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8100b30a0 msgr2=0x7fd8100cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:53.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.524+0000 7fd814a00640 1 --2- 172.21.15.39:0/1677422954 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8100b30a0 0x7fd8100cb8a0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fd810075160 tx=0x7fd80002a4a0 comp rx=0 tx=0).stop 2024-09-21T07:46:53.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.524+0000 7fd814a00640 1 -- 172.21.15.39:0/1677422954 shutdown_connections 2024-09-21T07:46:53.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.524+0000 7fd814a00640 1 --2- 172.21.15.39:0/1677422954 >> v2:172.21.15.148:3300/0 conn(0x7fd81015d830 0x7fd810156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:53.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.524+0000 7fd814a00640 1 --2- 172.21.15.39:0/1677422954 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8100b30a0 0x7fd8100cb8a0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:53.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.524+0000 7fd814a00640 1 -- 172.21.15.39:0/1677422954 >> 172.21.15.39:0/1677422954 conn(0x7fd8100b2460 msgr2=0x7fd81006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:53.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.524+0000 7fd814a00640 1 -- 172.21.15.39:0/1677422954 shutdown_connections 2024-09-21T07:46:53.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.524+0000 7fd814a00640 1 -- 172.21.15.39:0/1677422954 wait complete. 2024-09-21T07:46:53.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.526+0000 7fd814a00640 1 Processor -- start 2024-09-21T07:46:53.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.526+0000 7fd814a00640 1 -- start start 2024-09-21T07:46:53.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.528+0000 7fd814a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8100b30a0 0x7fd8101559c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:53.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.528+0000 7fd814a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd8100ccc60 0x7fd8101f6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:53.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.528+0000 7fd814a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd81015d830 0x7fd8101f7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:53.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.528+0000 7fd814a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd8100af410 con 0x7fd8100b30a0 2024-09-21T07:46:53.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.528+0000 7fd814a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd8100af290 con 0x7fd81015d830 2024-09-21T07:46:53.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.528+0000 7fd814a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd8100af590 con 0x7fd8100ccc60 2024-09-21T07:46:53.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.528+0000 7fd80de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8100b30a0 0x7fd8101559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:53.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.528+0000 7fd80de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8100b30a0 0x7fd8101559c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34790/0 (socket says 172.21.15.39:34790) 2024-09-21T07:46:53.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.528+0000 7fd80de00640 1 -- 172.21.15.39:0/674096407 learned_addr learned my addr 172.21.15.39:0/674096407 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:53.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.528+0000 7fd807e00640 1 --2- 172.21.15.39:0/674096407 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd8100ccc60 0x7fd8101f6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:53.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.528+0000 7fd80e800640 1 --2- 172.21.15.39:0/674096407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd81015d830 0x7fd8101f7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:53.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.529+0000 7fd80de00640 1 -- 172.21.15.39:0/674096407 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd8100ccc60 msgr2=0x7fd8101f6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:53.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.529+0000 7fd80de00640 1 --2- 172.21.15.39:0/674096407 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd8100ccc60 0x7fd8101f6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:53.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.529+0000 7fd80de00640 1 -- 172.21.15.39:0/674096407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd81015d830 msgr2=0x7fd8101f7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:53.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.529+0000 7fd80de00640 1 --2- 172.21.15.39:0/674096407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd81015d830 0x7fd8101f7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:53.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.529+0000 7fd80de00640 1 -- 172.21.15.39:0/674096407 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8100c67a0 con 0x7fd8100b30a0 2024-09-21T07:46:53.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.529+0000 7fd80de00640 1 --2- 172.21.15.39:0/674096407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8100b30a0 0x7fd8101559c0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fd800009920 tx=0x7fd800033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:53.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.529+0000 7fd806000640 1 -- 172.21.15.39:0/674096407 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd800031d60 con 0x7fd8100b30a0 2024-09-21T07:46:53.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.529+0000 7fd806000640 1 -- 172.21.15.39:0/674096407 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd800033e10 con 0x7fd8100b30a0 2024-09-21T07:46:53.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.529+0000 7fd806000640 1 -- 172.21.15.39:0/674096407 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd800005260 con 0x7fd8100b30a0 2024-09-21T07:46:53.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.529+0000 7fd814a00640 1 -- 172.21.15.39:0/674096407 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd810074f60 con 0x7fd8100b30a0 2024-09-21T07:46:53.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.530+0000 7fd814a00640 1 -- 172.21.15.39:0/674096407 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8100c8e80 con 0x7fd8100b30a0 2024-09-21T07:46:53.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.531+0000 7fd814a00640 1 -- 172.21.15.39:0/674096407 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8100afa40 con 0x7fd8100b30a0 2024-09-21T07:46:53.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.532+0000 7fd806000640 1 -- 172.21.15.39:0/674096407 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd800005400 con 0x7fd8100b30a0 2024-09-21T07:46:53.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.533+0000 7fd806000640 1 --2- 172.21.15.39:0/674096407 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd7ec078bc0 0x7fd7ec07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:53.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.533+0000 7fd806000640 1 -- 172.21.15.39:0/674096407 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(181..181 src has 1..181) ==== 6960+0+0 (secure 0 0 0) 0x7fd8000bc4c0 con 0x7fd8100b30a0 2024-09-21T07:46:53.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.534+0000 7fd807e00640 1 --2- 172.21.15.39:0/674096407 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd7ec078bc0 0x7fd7ec07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:53.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.535+0000 7fd807e00640 1 --2- 172.21.15.39:0/674096407 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd7ec078bc0 0x7fd7ec07b0d0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fd8100ad7f0 tx=0x7fd7f800b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:53.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.538+0000 7fd806000640 1 -- 172.21.15.39:0/674096407 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd80003f030 con 0x7fd8100b30a0 2024-09-21T07:46:53.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.734+0000 7fd814a00640 1 -- 172.21.15.39:0/674096407 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd8100ada00 con 0x7fd8100b30a0 2024-09-21T07:46:53.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.736+0000 7fd806000640 1 -- 172.21.15.39:0/674096407 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v181) ==== 74+0+22427 (secure 0 0 0) 0x7fd800083d50 con 0x7fd8100b30a0 2024-09-21T07:46:53.738 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:46:53.738 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":181,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:46:51.928525+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:46:24.067206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-09-21T07:46:25.054230+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":199,"snap_epoch":181,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]},{"pool":20,"snaps":[{"begin":80,"length":23},{"begin":181,"length":23}]},{"pool":22,"snaps":[{"begin":2,"length":198}]}],"new_removed_snaps":[{"pool":22,"snaps":[{"begin":100,"length":1},{"begin":199,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:46:53.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.738+0000 7fd814a00640 1 -- 172.21.15.39:0/674096407 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd7ec078bc0 msgr2=0x7fd7ec07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:53.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.738+0000 7fd814a00640 1 --2- 172.21.15.39:0/674096407 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd7ec078bc0 0x7fd7ec07b0d0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fd8100ad7f0 tx=0x7fd7f800b2e0 comp rx=0 tx=0).stop 2024-09-21T07:46:53.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.738+0000 7fd814a00640 1 -- 172.21.15.39:0/674096407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8100b30a0 msgr2=0x7fd8101559c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:53.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.738+0000 7fd814a00640 1 --2- 172.21.15.39:0/674096407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8100b30a0 0x7fd8101559c0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fd800009920 tx=0x7fd800033c20 comp rx=0 tx=0).stop 2024-09-21T07:46:53.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.738+0000 7fd814a00640 1 -- 172.21.15.39:0/674096407 shutdown_connections 2024-09-21T07:46:53.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.738+0000 7fd814a00640 1 --2- 172.21.15.39:0/674096407 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd7ec078bc0 0x7fd7ec07b0d0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:53.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.738+0000 7fd814a00640 1 --2- 172.21.15.39:0/674096407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd81015d830 0x7fd8101f7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:53.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.738+0000 7fd814a00640 1 --2- 172.21.15.39:0/674096407 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd8100ccc60 0x7fd8101f6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:53.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.738+0000 7fd814a00640 1 --2- 172.21.15.39:0/674096407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8100b30a0 0x7fd8101559c0 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:53.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.738+0000 7fd814a00640 1 -- 172.21.15.39:0/674096407 >> 172.21.15.39:0/674096407 conn(0x7fd8100b2460 msgr2=0x7fd810068f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:53.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.738+0000 7fd814a00640 1 -- 172.21.15.39:0/674096407 shutdown_connections 2024-09-21T07:46:53.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.738+0000 7fd814a00640 1 -- 172.21.15.39:0/674096407 wait complete. 2024-09-21T07:46:53.762 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T07:46:53.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.990+0000 7f39e1200640 1 -- 172.21.15.39:0/2890533729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39c0004d40 msgr2=0x7f39c0005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:53.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.990+0000 7f39e1200640 1 --2- 172.21.15.39:0/2890533729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39c0004d40 0x7f39c0005150 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f39d4009ed0 tx=0x7f39c0005830 comp rx=0 tx=0).stop 2024-09-21T07:46:53.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.990+0000 7f39da800640 1 -- 172.21.15.39:0/2890533729 reap_dead start 2024-09-21T07:46:53.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.990+0000 7f39e1200640 1 -- 172.21.15.39:0/2890533729 shutdown_connections 2024-09-21T07:46:53.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.990+0000 7f39e1200640 1 -- 172.21.15.39:0/2890533729 >> 172.21.15.39:0/2890533729 conn(0x7f39dc06cfc0 msgr2=0x7f39dc06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:53.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.990+0000 7f39e1200640 1 -- 172.21.15.39:0/2890533729 shutdown_connections 2024-09-21T07:46:53.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.990+0000 7f39e1200640 1 -- 172.21.15.39:0/2890533729 wait complete. 2024-09-21T07:46:53.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.993+0000 7f39e1200640 1 Processor -- start 2024-09-21T07:46:53.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.993+0000 7f39e1200640 1 -- start start 2024-09-21T07:46:53.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.994+0000 7f39e1200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39c0004d40 0x7f39dc145d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:53.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.994+0000 7f39e1200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39dc150c90 0x7f39dc158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:53.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.994+0000 7f39e1200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39dc1511e0 0x7f39dc1586b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:53.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.994+0000 7f39e1200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc0ccda0 con 0x7f39dc150c90 2024-09-21T07:46:53.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.994+0000 7f39e1200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc0ccc20 con 0x7f39c0004d40 2024-09-21T07:46:53.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.994+0000 7f39e1200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f39dc0ccf20 con 0x7f39dc1511e0 2024-09-21T07:46:53.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.995+0000 7f39d1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39dc150c90 0x7f39dc158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:53.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.995+0000 7f39d1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39dc150c90 0x7f39dc158170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34812/0 (socket says 172.21.15.39:34812) 2024-09-21T07:46:53.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.995+0000 7f39d1400640 1 -- 172.21.15.39:0/2964582571 learned_addr learned my addr 172.21.15.39:0/2964582571 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:53.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.995+0000 7f39da800640 1 --2- 172.21.15.39:0/2964582571 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39dc1511e0 0x7f39dc1586b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:53.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.995+0000 7f39d9e00640 1 --2- 172.21.15.39:0/2964582571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39c0004d40 0x7f39dc145d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:53.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.995+0000 7f39d1400640 1 -- 172.21.15.39:0/2964582571 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39dc1511e0 msgr2=0x7f39dc1586b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:53.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.995+0000 7f39d1400640 1 --2- 172.21.15.39:0/2964582571 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39dc1511e0 0x7f39dc1586b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:53.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.995+0000 7f39d1400640 1 -- 172.21.15.39:0/2964582571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39c0004d40 msgr2=0x7f39dc145d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:53.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.995+0000 7f39d1400640 1 --2- 172.21.15.39:0/2964582571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39c0004d40 0x7f39dc145d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:53.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.995+0000 7f39d1400640 1 -- 172.21.15.39:0/2964582571 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39dc15a1c0 con 0x7f39dc150c90 2024-09-21T07:46:53.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.995+0000 7f39d9e00640 1 --2- 172.21.15.39:0/2964582571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39c0004d40 0x7f39dc145d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:53.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.995+0000 7f39d1400640 1 --2- 172.21.15.39:0/2964582571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39dc150c90 0x7f39dc158170 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f39c4006600 tx=0x7f39c40039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:53.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.996+0000 7f39d2a00640 1 -- 172.21.15.39:0/2964582571 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c4015070 con 0x7f39dc150c90 2024-09-21T07:46:53.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.996+0000 7f39d2a00640 1 -- 172.21.15.39:0/2964582571 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39c4011920 con 0x7f39dc150c90 2024-09-21T07:46:53.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.996+0000 7f39d2a00640 1 -- 172.21.15.39:0/2964582571 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c4011d70 con 0x7f39dc150c90 2024-09-21T07:46:53.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.996+0000 7f39e1200640 1 -- 172.21.15.39:0/2964582571 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39dc15a4a0 con 0x7f39dc150c90 2024-09-21T07:46:53.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.996+0000 7f39e1200640 1 -- 172.21.15.39:0/2964582571 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39dc15aa60 con 0x7f39dc150c90 2024-09-21T07:46:53.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.997+0000 7f39e1200640 1 -- 172.21.15.39:0/2964582571 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39dc15cae0 con 0x7f39dc150c90 2024-09-21T07:46:54.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.998+0000 7f39d2a00640 1 -- 172.21.15.39:0/2964582571 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f39c4010420 con 0x7f39dc150c90 2024-09-21T07:46:54.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.999+0000 7f39d2a00640 1 --2- 172.21.15.39:0/2964582571 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f39b8078bc0 0x7f39b807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:54.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:53.999+0000 7f39d2a00640 1 -- 172.21.15.39:0/2964582571 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(182..182 src has 1..182) ==== 7003+0+0 (secure 0 0 0) 0x7f39c409bec0 con 0x7f39dc150c90 2024-09-21T07:46:54.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.000+0000 7f39d9e00640 1 --2- 172.21.15.39:0/2964582571 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f39b8078bc0 0x7f39b807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:54.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.001+0000 7f39d9e00640 1 --2- 172.21.15.39:0/2964582571 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f39b8078bc0 0x7f39b807b0d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f39dc0730e0 tx=0x7f39cc005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:54.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.004+0000 7f39d2a00640 1 -- 172.21.15.39:0/2964582571 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f39dc15cae0 con 0x7f39dc150c90 2024-09-21T07:46:54.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.242+0000 7f39e1200640 1 -- 172.21.15.39:0/2964582571 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f39dc0697d0 con 0x7f39dc150c90 2024-09-21T07:46:54.244 INFO:tasks.ceph.mon.a.smithi039.stderr:2024-09-21T07:46:54.243+0000 7fabab600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T07:46:54.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.245+0000 7f39d2a00640 1 -- 172.21.15.39:0/2964582571 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v191) ==== 161+0+0 (secure 0 0 0) 0x7f39dc0697d0 con 0x7f39dc150c90 2024-09-21T07:46:54.247 INFO:teuthology.orchestra.run.smithi039.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T07:46:54.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.248+0000 7f39e1200640 1 -- 172.21.15.39:0/2964582571 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f39b8078bc0 msgr2=0x7f39b807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:54.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.248+0000 7f39e1200640 1 --2- 172.21.15.39:0/2964582571 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f39b8078bc0 0x7f39b807b0d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f39dc0730e0 tx=0x7f39cc005f90 comp rx=0 tx=0).stop 2024-09-21T07:46:54.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.248+0000 7f39e1200640 1 -- 172.21.15.39:0/2964582571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39dc150c90 msgr2=0x7f39dc158170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:54.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.248+0000 7f39e1200640 1 --2- 172.21.15.39:0/2964582571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39dc150c90 0x7f39dc158170 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f39c4006600 tx=0x7f39c40039c0 comp rx=0 tx=0).stop 2024-09-21T07:46:54.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.248+0000 7f39e1200640 1 -- 172.21.15.39:0/2964582571 shutdown_connections 2024-09-21T07:46:54.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.248+0000 7f39e1200640 1 --2- 172.21.15.39:0/2964582571 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f39b8078bc0 0x7f39b807b0d0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:54.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.248+0000 7f39e1200640 1 --2- 172.21.15.39:0/2964582571 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39dc1511e0 0x7f39dc1586b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:54.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.248+0000 7f39e1200640 1 --2- 172.21.15.39:0/2964582571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39dc150c90 0x7f39dc158170 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:54.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.248+0000 7f39e1200640 1 --2- 172.21.15.39:0/2964582571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39c0004d40 0x7f39dc145d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:54.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.248+0000 7f39e1200640 1 -- 172.21.15.39:0/2964582571 >> 172.21.15.39:0/2964582571 conn(0x7f39dc06cfc0 msgr2=0x7f39dc0b2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:54.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.249+0000 7f39e1200640 1 -- 172.21.15.39:0/2964582571 shutdown_connections 2024-09-21T07:46:54.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.249+0000 7f39e1200640 1 -- 172.21.15.39:0/2964582571 wait complete. 2024-09-21T07:46:54.277 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T07:46:54.384 INFO:tasks.ceph.mds.c.smithi039.stdout:starting mds.c at 2024-09-21T07:46:54.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.507+0000 7fe954800640 1 -- 172.21.15.39:0/3531241387 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe94c0cfb60 msgr2=0x7fe94c1674a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:54.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.507+0000 7fe954800640 1 --2- 172.21.15.39:0/3531241387 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe94c0cfb60 0x7fe94c1674a0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fe948005fd0 tx=0x7fe94802a940 comp rx=0 tx=0).stop 2024-09-21T07:46:54.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.507+0000 7fe954800640 1 -- 172.21.15.39:0/3531241387 shutdown_connections 2024-09-21T07:46:54.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.507+0000 7fe954800640 1 --2- 172.21.15.39:0/3531241387 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe94c0cfb60 0x7fe94c1674a0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:54.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.507+0000 7fe954800640 1 -- 172.21.15.39:0/3531241387 >> 172.21.15.39:0/3531241387 conn(0x7fe94c06cfc0 msgr2=0x7fe94c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:54.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.507+0000 7fe954800640 1 -- 172.21.15.39:0/3531241387 shutdown_connections 2024-09-21T07:46:54.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.507+0000 7fe954800640 1 -- 172.21.15.39:0/3531241387 wait complete. 2024-09-21T07:46:54.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.510+0000 7fe954800640 1 Processor -- start 2024-09-21T07:46:54.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.510+0000 7fe954800640 1 -- start start 2024-09-21T07:46:54.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.511+0000 7fe954800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe94c070bc0 0x7fe94c143290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:54.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.511+0000 7fe954800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe94c0cfb60 0x7fe94c1f76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:54.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.511+0000 7fe954800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe94c0d0dc0 0x7fe94c14fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:54.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.511+0000 7fe954800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c167bf0 con 0x7fe94c0d0dc0 2024-09-21T07:46:54.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.511+0000 7fe954800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c15d830 con 0x7fe94c070bc0 2024-09-21T07:46:54.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.511+0000 7fe954800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe94c167d70 con 0x7fe94c0cfb60 2024-09-21T07:46:54.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.511+0000 7fe952800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe94c0d0dc0 0x7fe94c14fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:54.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.512+0000 7fe952800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe94c0d0dc0 0x7fe94c14fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34854/0 (socket says 172.21.15.39:34854) 2024-09-21T07:46:54.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.512+0000 7fe952800640 1 -- 172.21.15.39:0/2416711174 learned_addr learned my addr 172.21.15.39:0/2416711174 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:54.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.512+0000 7fe951400640 1 --2- 172.21.15.39:0/2416711174 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe94c0cfb60 0x7fe94c1f76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:54.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.512+0000 7fe951e00640 1 --2- 172.21.15.39:0/2416711174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe94c070bc0 0x7fe94c143290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:54.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.512+0000 7fe952800640 1 -- 172.21.15.39:0/2416711174 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe94c0cfb60 msgr2=0x7fe94c1f76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:54.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.512+0000 7fe952800640 1 --2- 172.21.15.39:0/2416711174 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe94c0cfb60 0x7fe94c1f76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:54.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.512+0000 7fe952800640 1 -- 172.21.15.39:0/2416711174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe94c070bc0 msgr2=0x7fe94c143290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:54.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.512+0000 7fe952800640 1 --2- 172.21.15.39:0/2416711174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe94c070bc0 0x7fe94c143290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:54.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.512+0000 7fe952800640 1 -- 172.21.15.39:0/2416711174 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe94c154ec0 con 0x7fe94c0d0dc0 2024-09-21T07:46:54.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.512+0000 7fe951400640 1 --2- 172.21.15.39:0/2416711174 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe94c0cfb60 0x7fe94c1f76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:54.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.512+0000 7fe951e00640 1 --2- 172.21.15.39:0/2416711174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe94c070bc0 0x7fe94c143290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:54.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.512+0000 7fe952800640 1 --2- 172.21.15.39:0/2416711174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe94c0d0dc0 0x7fe94c14fa50 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fe94802af30 tx=0x7fe948031ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:54.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.513+0000 7fe946000640 1 -- 172.21.15.39:0/2416711174 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94803e070 con 0x7fe94c0d0dc0 2024-09-21T07:46:54.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.513+0000 7fe946000640 1 -- 172.21.15.39:0/2416711174 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe948004560 con 0x7fe94c0d0dc0 2024-09-21T07:46:54.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.513+0000 7fe954800640 1 -- 172.21.15.39:0/2416711174 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe94c150d20 con 0x7fe94c0d0dc0 2024-09-21T07:46:54.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.513+0000 7fe946000640 1 -- 172.21.15.39:0/2416711174 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe948043460 con 0x7fe94c0d0dc0 2024-09-21T07:46:54.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.513+0000 7fe954800640 1 -- 172.21.15.39:0/2416711174 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe94c150fc0 con 0x7fe94c0d0dc0 2024-09-21T07:46:54.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.514+0000 7fe954800640 1 -- 172.21.15.39:0/2416711174 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe94c14e820 con 0x7fe94c0d0dc0 2024-09-21T07:46:54.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.516+0000 7fe946000640 1 -- 172.21.15.39:0/2416711174 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe948043600 con 0x7fe94c0d0dc0 2024-09-21T07:46:54.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.516+0000 7fe946000640 1 --2- 172.21.15.39:0/2416711174 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe930078bc0 0x7fe93007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:54.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.516+0000 7fe946000640 1 -- 172.21.15.39:0/2416711174 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 7089+0+0 (secure 0 0 0) 0x7fe9480bd0c0 con 0x7fe94c0d0dc0 2024-09-21T07:46:54.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.517+0000 7fe951e00640 1 --2- 172.21.15.39:0/2416711174 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe930078bc0 0x7fe93007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:54.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.518+0000 7fe951e00640 1 --2- 172.21.15.39:0/2416711174 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe930078bc0 0x7fe93007b0d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fe940007fd0 tx=0x7fe940009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:54.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.521+0000 7fe946000640 1 -- 172.21.15.39:0/2416711174 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe948085090 con 0x7fe94c0d0dc0 2024-09-21T07:46:54.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:54.761+0000 7fe954800640 1 -- 172.21.15.39:0/2416711174 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fe94c155090 con 0x7fe94c0d0dc0 2024-09-21T07:46:55.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.249+0000 7fe946000640 1 -- 172.21.15.39:0/2416711174 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v192) ==== 104+0+0 (secure 0 0 0) 0x7fe948084990 con 0x7fe94c0d0dc0 2024-09-21T07:46:55.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.251+0000 7fe954800640 1 -- 172.21.15.39:0/2416711174 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe930078bc0 msgr2=0x7fe93007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:55.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.251+0000 7fe954800640 1 --2- 172.21.15.39:0/2416711174 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe930078bc0 0x7fe93007b0d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fe940007fd0 tx=0x7fe940009040 comp rx=0 tx=0).stop 2024-09-21T07:46:55.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.251+0000 7fe954800640 1 -- 172.21.15.39:0/2416711174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe94c0d0dc0 msgr2=0x7fe94c14fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:55.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.251+0000 7fe954800640 1 --2- 172.21.15.39:0/2416711174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe94c0d0dc0 0x7fe94c14fa50 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fe94802af30 tx=0x7fe948031ea0 comp rx=0 tx=0).stop 2024-09-21T07:46:55.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.251+0000 7fe954800640 1 -- 172.21.15.39:0/2416711174 shutdown_connections 2024-09-21T07:46:55.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.251+0000 7fe954800640 1 --2- 172.21.15.39:0/2416711174 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe930078bc0 0x7fe93007b0d0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:55.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.251+0000 7fe954800640 1 --2- 172.21.15.39:0/2416711174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe94c0d0dc0 0x7fe94c14fa50 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:55.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.251+0000 7fe954800640 1 --2- 172.21.15.39:0/2416711174 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe94c0cfb60 0x7fe94c1f76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:55.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.251+0000 7fe954800640 1 --2- 172.21.15.39:0/2416711174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe94c070bc0 0x7fe94c143290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:55.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.251+0000 7fe954800640 1 -- 172.21.15.39:0/2416711174 >> 172.21.15.39:0/2416711174 conn(0x7fe94c06cfc0 msgr2=0x7fe94c15de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:55.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.252+0000 7fe954800640 1 -- 172.21.15.39:0/2416711174 shutdown_connections 2024-09-21T07:46:55.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.252+0000 7fe954800640 1 -- 172.21.15.39:0/2416711174 wait complete. 2024-09-21T07:46:55.277 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T07:46:55.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.510+0000 7f780de00640 1 -- 172.21.15.39:0/657374835 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7808070bc0 msgr2=0x7f780806f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:55.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.510+0000 7f780de00640 1 --2- 172.21.15.39:0/657374835 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7808070bc0 0x7f780806f020 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f77f8009830 tx=0x7f77f802a520 comp rx=0 tx=0).stop 2024-09-21T07:46:55.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.510+0000 7f780de00640 1 -- 172.21.15.39:0/657374835 shutdown_connections 2024-09-21T07:46:55.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.510+0000 7f780de00640 1 --2- 172.21.15.39:0/657374835 >> v2:172.21.15.148:3300/0 conn(0x7f780806f970 0x7f780815d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:55.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.510+0000 7f780de00640 1 --2- 172.21.15.39:0/657374835 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7808070bc0 0x7f780806f020 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:55.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.510+0000 7f780de00640 1 -- 172.21.15.39:0/657374835 >> 172.21.15.39:0/657374835 conn(0x7f78080b2640 msgr2=0x7f78080d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:55.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.510+0000 7f780de00640 1 -- 172.21.15.39:0/657374835 shutdown_connections 2024-09-21T07:46:55.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.510+0000 7f780de00640 1 -- 172.21.15.39:0/657374835 wait complete. 2024-09-21T07:46:55.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.512+0000 7f780de00640 1 Processor -- start 2024-09-21T07:46:55.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.512+0000 7f780de00640 1 -- start start 2024-09-21T07:46:55.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.514+0000 7f780de00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f780806f560 0x7f78080ad0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:55.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.514+0000 7f780de00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f780806f970 0x7f7808099800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:55.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.514+0000 7f780de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7808070bc0 0x7f78080c5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:55.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.514+0000 7f780de00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f78080cd670 con 0x7f7808070bc0 2024-09-21T07:46:55.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.514+0000 7f780de00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f78080cd4f0 con 0x7f780806f560 2024-09-21T07:46:55.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.514+0000 7f780de00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f780815d880 con 0x7f780806f970 2024-09-21T07:46:55.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.514+0000 7f7807400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7808070bc0 0x7f78080c5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:55.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.514+0000 7f7807400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7808070bc0 0x7f78080c5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34868/0 (socket says 172.21.15.39:34868) 2024-09-21T07:46:55.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.514+0000 7f7807400640 1 -- 172.21.15.39:0/3155619586 learned_addr learned my addr 172.21.15.39:0/3155619586 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:55.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.514+0000 7f7806a00640 1 --2- 172.21.15.39:0/3155619586 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f780806f560 0x7f78080ad0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:55.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.514+0000 7f7806000640 1 --2- 172.21.15.39:0/3155619586 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f780806f970 0x7f7808099800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:55.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.515+0000 7f7807400640 1 -- 172.21.15.39:0/3155619586 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f780806f970 msgr2=0x7f7808099800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:55.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.515+0000 7f7807400640 1 --2- 172.21.15.39:0/3155619586 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f780806f970 0x7f7808099800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:55.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.515+0000 7f7807400640 1 -- 172.21.15.39:0/3155619586 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f780806f560 msgr2=0x7f78080ad0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:55.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.515+0000 7f7807400640 1 --2- 172.21.15.39:0/3155619586 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f780806f560 0x7f78080ad0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:55.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.515+0000 7f7807400640 1 -- 172.21.15.39:0/3155619586 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78081551f0 con 0x7f7808070bc0 2024-09-21T07:46:55.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.515+0000 7f7807400640 1 --2- 172.21.15.39:0/3155619586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7808070bc0 0x7f78080c5360 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f780000ea10 tx=0x7f780000eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:55.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.515+0000 7f77ff400640 1 -- 172.21.15.39:0/3155619586 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780000cde0 con 0x7f7808070bc0 2024-09-21T07:46:55.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.515+0000 7f77ff400640 1 -- 172.21.15.39:0/3155619586 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7800004c10 con 0x7f7808070bc0 2024-09-21T07:46:55.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.515+0000 7f780de00640 1 -- 172.21.15.39:0/3155619586 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78081f4670 con 0x7f7808070bc0 2024-09-21T07:46:55.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.515+0000 7f77ff400640 1 -- 172.21.15.39:0/3155619586 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7800005020 con 0x7f7808070bc0 2024-09-21T07:46:55.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.515+0000 7f780de00640 1 -- 172.21.15.39:0/3155619586 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78080c4b60 con 0x7f7808070bc0 2024-09-21T07:46:55.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.516+0000 7f780de00640 1 -- 172.21.15.39:0/3155619586 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7808155460 con 0x7f7808070bc0 2024-09-21T07:46:55.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.518+0000 7f77ff400640 1 -- 172.21.15.39:0/3155619586 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f78000051c0 con 0x7f7808070bc0 2024-09-21T07:46:55.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.519+0000 7f77ff400640 1 --2- 172.21.15.39:0/3155619586 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f77e8078bc0 0x7f77e807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:55.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.519+0000 7f77ff400640 1 -- 172.21.15.39:0/3155619586 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 7089+0+0 (secure 0 0 0) 0x7f780009c580 con 0x7f7808070bc0 2024-09-21T07:46:55.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.519+0000 7f7806a00640 1 --2- 172.21.15.39:0/3155619586 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f77e8078bc0 0x7f77e807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:55.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.520+0000 7f7806a00640 1 --2- 172.21.15.39:0/3155619586 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f77e8078bc0 0x7f77e807b0d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f77f8034000 tx=0x7f77f8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:55.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.522+0000 7f77ff400640 1 -- 172.21.15.39:0/3155619586 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7800026020 con 0x7f7808070bc0 2024-09-21T07:46:55.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:55.719+0000 7f780de00640 1 -- 172.21.15.39:0/3155619586 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f780806f020 con 0x7f7808070bc0 2024-09-21T07:46:56.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.255+0000 7f77ff400640 1 -- 172.21.15.39:0/3155619586 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v184) ==== 181+0+0 (secure 0 0 0) 0x7f7800063e70 con 0x7f7808070bc0 2024-09-21T07:46:56.257 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' removed 2024-09-21T07:46:56.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.259+0000 7f780de00640 1 -- 172.21.15.39:0/3155619586 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f77e8078bc0 msgr2=0x7f77e807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:56.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.259+0000 7f780de00640 1 --2- 172.21.15.39:0/3155619586 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f77e8078bc0 0x7f77e807b0d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f77f8034000 tx=0x7f77f8004730 comp rx=0 tx=0).stop 2024-09-21T07:46:56.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.259+0000 7f780de00640 1 -- 172.21.15.39:0/3155619586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7808070bc0 msgr2=0x7f78080c5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:56.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.259+0000 7f780de00640 1 --2- 172.21.15.39:0/3155619586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7808070bc0 0x7f78080c5360 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f780000ea10 tx=0x7f780000eed0 comp rx=0 tx=0).stop 2024-09-21T07:46:56.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.259+0000 7f780de00640 1 -- 172.21.15.39:0/3155619586 shutdown_connections 2024-09-21T07:46:56.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.259+0000 7f780de00640 1 --2- 172.21.15.39:0/3155619586 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f77e8078bc0 0x7f77e807b0d0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:56.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.259+0000 7f780de00640 1 --2- 172.21.15.39:0/3155619586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7808070bc0 0x7f78080c5360 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:56.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.259+0000 7f780de00640 1 --2- 172.21.15.39:0/3155619586 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f780806f970 0x7f7808099800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:56.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.259+0000 7f780de00640 1 --2- 172.21.15.39:0/3155619586 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f780806f560 0x7f78080ad0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:56.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.259+0000 7f780de00640 1 -- 172.21.15.39:0/3155619586 >> 172.21.15.39:0/3155619586 conn(0x7f78080b2640 msgr2=0x7f78080b0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:56.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.259+0000 7f780de00640 1 -- 172.21.15.39:0/3155619586 shutdown_connections 2024-09-21T07:46:56.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.259+0000 7f780de00640 1 -- 172.21.15.39:0/3155619586 wait complete. 2024-09-21T07:46:56.289 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T07:46:56.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.527+0000 7f1e5bc00640 1 -- 172.21.15.39:0/287520127 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1e540af3f0 msgr2=0x7f1e540707b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:56.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.527+0000 7f1e5bc00640 1 --2- 172.21.15.39:0/287520127 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1e540af3f0 0x7f1e540707b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c00acf0 tx=0x7f1e4c02bc40 comp rx=0 tx=0).stop 2024-09-21T07:46:56.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.528+0000 7f1e5bc00640 1 -- 172.21.15.39:0/287520127 shutdown_connections 2024-09-21T07:46:56.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.528+0000 7f1e5bc00640 1 --2- 172.21.15.39:0/287520127 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1e540af3f0 0x7f1e540707b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:56.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.528+0000 7f1e5bc00640 1 -- 172.21.15.39:0/287520127 >> 172.21.15.39:0/287520127 conn(0x7f1e540b2640 msgr2=0x7f1e540d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:56.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.528+0000 7f1e5bc00640 1 -- 172.21.15.39:0/287520127 shutdown_connections 2024-09-21T07:46:56.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.528+0000 7f1e5bc00640 1 -- 172.21.15.39:0/287520127 wait complete. 2024-09-21T07:46:56.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.530+0000 7f1e5bc00640 1 Processor -- start 2024-09-21T07:46:56.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.530+0000 7f1e5bc00640 1 -- start start 2024-09-21T07:46:56.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.531+0000 7f1e5bc00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e5406f540 0x7f1e5409ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:56.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.531+0000 7f1e5bc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e540af3f0 0x7f1e5415a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:56.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.532+0000 7f1e5bc00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1e5415aa80 0x7f1e541f3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:56.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.532+0000 7f1e5bc00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1e540cf610 con 0x7f1e540af3f0 2024-09-21T07:46:56.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.532+0000 7f1e5bc00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1e54071340 con 0x7f1e5406f540 2024-09-21T07:46:56.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.532+0000 7f1e5bc00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f1e540cf790 con 0x7f1e5415aa80 2024-09-21T07:46:56.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.532+0000 7f1e53e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e540af3f0 0x7f1e5415a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:56.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.532+0000 7f1e53e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e540af3f0 0x7f1e5415a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34876/0 (socket says 172.21.15.39:34876) 2024-09-21T07:46:56.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.532+0000 7f1e53e00640 1 -- 172.21.15.39:0/3529042894 learned_addr learned my addr 172.21.15.39:0/3529042894 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:56.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.532+0000 7f1e59200640 1 --2- 172.21.15.39:0/3529042894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e5406f540 0x7f1e5409ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:56.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.532+0000 7f1e59c00640 1 --2- 172.21.15.39:0/3529042894 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1e5415aa80 0x7f1e541f3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:56.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.532+0000 7f1e53e00640 1 -- 172.21.15.39:0/3529042894 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1e5415aa80 msgr2=0x7f1e541f3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:56.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.532+0000 7f1e53e00640 1 --2- 172.21.15.39:0/3529042894 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1e5415aa80 0x7f1e541f3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:56.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.532+0000 7f1e53e00640 1 -- 172.21.15.39:0/3529042894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e5406f540 msgr2=0x7f1e5409ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:56.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.532+0000 7f1e53e00640 1 --2- 172.21.15.39:0/3529042894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e5406f540 0x7f1e5409ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:56.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.532+0000 7f1e53e00640 1 -- 172.21.15.39:0/3529042894 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e54074970 con 0x7f1e540af3f0 2024-09-21T07:46:56.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.533+0000 7f1e53e00640 1 --2- 172.21.15.39:0/3529042894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e540af3f0 0x7f1e5415a2d0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f1e4400c390 tx=0x7f1e4400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:56.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.533+0000 7f1e50c00640 1 -- 172.21.15.39:0/3529042894 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e44011790 con 0x7f1e540af3f0 2024-09-21T07:46:56.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.533+0000 7f1e50c00640 1 -- 172.21.15.39:0/3529042894 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e44017070 con 0x7f1e540af3f0 2024-09-21T07:46:56.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.533+0000 7f1e5bc00640 1 -- 172.21.15.39:0/3529042894 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e541f43d0 con 0x7f1e540af3f0 2024-09-21T07:46:56.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.533+0000 7f1e50c00640 1 -- 172.21.15.39:0/3529042894 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e44011d00 con 0x7f1e540af3f0 2024-09-21T07:46:56.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.533+0000 7f1e5bc00640 1 -- 172.21.15.39:0/3529042894 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e5409d5d0 con 0x7f1e540af3f0 2024-09-21T07:46:56.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.534+0000 7f1e5bc00640 1 -- 172.21.15.39:0/3529042894 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e541f4140 con 0x7f1e540af3f0 2024-09-21T07:46:56.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.536+0000 7f1e50c00640 1 -- 172.21.15.39:0/3529042894 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1e4401c460 con 0x7f1e540af3f0 2024-09-21T07:46:56.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.536+0000 7f1e50c00640 1 --2- 172.21.15.39:0/3529042894 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1e34078bc0 0x7f1e3407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:56.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.537+0000 7f1e50c00640 1 -- 172.21.15.39:0/3529042894 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(185..185 src has 1..185) ==== 6711+0+0 (secure 0 0 0) 0x7f1e4409acc0 con 0x7f1e540af3f0 2024-09-21T07:46:56.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.537+0000 7f1e59200640 1 --2- 172.21.15.39:0/3529042894 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1e34078bc0 0x7f1e3407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:56.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.538+0000 7f1e59200640 1 --2- 172.21.15.39:0/3529042894 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1e34078bc0 0x7f1e3407b0d0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c035000 tx=0x7f1e4c004ab0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:56.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.542+0000 7f1e50c00640 1 -- 172.21.15.39:0/3529042894 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1e44062da0 con 0x7f1e540af3f0 2024-09-21T07:46:56.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:56.740+0000 7f1e5bc00640 1 -- 172.21.15.39:0/3529042894 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1e540cfd40 con 0x7f1e540af3f0 2024-09-21T07:46:57.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.513+0000 7f1e50c00640 1 -- 172.21.15.39:0/3529042894 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v186) ==== 169+0+0 (secure 0 0 0) 0x7f1e440626a0 con 0x7f1e540af3f0 2024-09-21T07:46:57.514 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' removed 2024-09-21T07:46:57.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.517+0000 7f1e5bc00640 1 -- 172.21.15.39:0/3529042894 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1e34078bc0 msgr2=0x7f1e3407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:57.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.517+0000 7f1e5bc00640 1 --2- 172.21.15.39:0/3529042894 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1e34078bc0 0x7f1e3407b0d0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c035000 tx=0x7f1e4c004ab0 comp rx=0 tx=0).stop 2024-09-21T07:46:57.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.517+0000 7f1e5bc00640 1 -- 172.21.15.39:0/3529042894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e540af3f0 msgr2=0x7f1e5415a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:57.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.517+0000 7f1e5bc00640 1 --2- 172.21.15.39:0/3529042894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e540af3f0 0x7f1e5415a2d0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f1e4400c390 tx=0x7f1e4400c850 comp rx=0 tx=0).stop 2024-09-21T07:46:57.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.517+0000 7f1e5bc00640 1 -- 172.21.15.39:0/3529042894 shutdown_connections 2024-09-21T07:46:57.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.517+0000 7f1e5bc00640 1 --2- 172.21.15.39:0/3529042894 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1e34078bc0 0x7f1e3407b0d0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:57.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.517+0000 7f1e5bc00640 1 --2- 172.21.15.39:0/3529042894 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1e5415aa80 0x7f1e541f3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:57.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.517+0000 7f1e5bc00640 1 --2- 172.21.15.39:0/3529042894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e540af3f0 0x7f1e5415a2d0 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:57.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.517+0000 7f1e5bc00640 1 --2- 172.21.15.39:0/3529042894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e5406f540 0x7f1e5409ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:57.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.517+0000 7f1e5bc00640 1 -- 172.21.15.39:0/3529042894 >> 172.21.15.39:0/3529042894 conn(0x7f1e540b2640 msgr2=0x7f1e54074da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:57.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.517+0000 7f1e5bc00640 1 -- 172.21.15.39:0/3529042894 shutdown_connections 2024-09-21T07:46:57.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.517+0000 7f1e5bc00640 1 -- 172.21.15.39:0/3529042894 wait complete. 2024-09-21T07:46:57.551 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T07:46:57.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.786+0000 7f9ec0a00640 1 -- 172.21.15.39:0/2996990391 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ea0003f90 msgr2=0x7f9ea0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:57.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.786+0000 7f9ec0a00640 1 --2- 172.21.15.39:0/2996990391 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ea0003f90 0x7f9ea0004800 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f9ea400c2e0 tx=0x7f9ea0005830 comp rx=0 tx=0).stop 2024-09-21T07:46:57.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.786+0000 7f9ebbe00640 1 -- 172.21.15.39:0/2996990391 reap_dead start 2024-09-21T07:46:57.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.787+0000 7f9ec0a00640 1 -- 172.21.15.39:0/2996990391 shutdown_connections 2024-09-21T07:46:57.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.787+0000 7f9ec0a00640 1 -- 172.21.15.39:0/2996990391 >> 172.21.15.39:0/2996990391 conn(0x7f9ebc06cfc0 msgr2=0x7f9ebc06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:57.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.787+0000 7f9ec0a00640 1 -- 172.21.15.39:0/2996990391 shutdown_connections 2024-09-21T07:46:57.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.787+0000 7f9ec0a00640 1 -- 172.21.15.39:0/2996990391 wait complete. 2024-09-21T07:46:57.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.789+0000 7f9ec0a00640 1 Processor -- start 2024-09-21T07:46:57.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.789+0000 7f9ec0a00640 1 -- start start 2024-09-21T07:46:57.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.790+0000 7f9ec0a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9ea0003f90 0x7f9ebc1f3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:57.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.790+0000 7f9ec0a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ebc0ae370 0x7f9ebc0ab580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:57.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.790+0000 7f9ec0a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ebc0c50b0 0x7f9ebc0c54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:57.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.790+0000 7f9ec0a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9ebc15d3a0 con 0x7f9ebc0c50b0 2024-09-21T07:46:57.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.790+0000 7f9ec0a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9ebc15d220 con 0x7f9ebc0ae370 2024-09-21T07:46:57.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.790+0000 7f9ec0a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f9ebc06ecf0 con 0x7f9ea0003f90 2024-09-21T07:46:57.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.791+0000 7f9ebbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ebc0c50b0 0x7f9ebc0c54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:57.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.791+0000 7f9ebbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ebc0c50b0 0x7f9ebc0c54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34894/0 (socket says 172.21.15.39:34894) 2024-09-21T07:46:57.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.791+0000 7f9ebbe00640 1 -- 172.21.15.39:0/3592512036 learned_addr learned my addr 172.21.15.39:0/3592512036 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:57.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.791+0000 7f9ebb400640 1 --2- 172.21.15.39:0/3592512036 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9ea0003f90 0x7f9ebc1f3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:57.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.791+0000 7f9eb3e00640 1 --2- 172.21.15.39:0/3592512036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ebc0ae370 0x7f9ebc0ab580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:57.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.791+0000 7f9ebbe00640 1 -- 172.21.15.39:0/3592512036 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9ea0003f90 msgr2=0x7f9ebc1f3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:57.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.791+0000 7f9ebbe00640 1 --2- 172.21.15.39:0/3592512036 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9ea0003f90 0x7f9ebc1f3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:57.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.791+0000 7f9ebbe00640 1 -- 172.21.15.39:0/3592512036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ebc0ae370 msgr2=0x7f9ebc0ab580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:57.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.791+0000 7f9ebbe00640 1 --2- 172.21.15.39:0/3592512036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ebc0ae370 0x7f9ebc0ab580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:57.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.791+0000 7f9ebbe00640 1 -- 172.21.15.39:0/3592512036 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ebc1f4320 con 0x7f9ebc0c50b0 2024-09-21T07:46:57.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.792+0000 7f9ebbe00640 1 --2- 172.21.15.39:0/3592512036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ebc0c50b0 0x7f9ebc0c54c0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4005fd0 tx=0x7f9eb4010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:57.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.792+0000 7f9eb3400640 1 -- 172.21.15.39:0/3592512036 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4017070 con 0x7f9ebc0c50b0 2024-09-21T07:46:57.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.792+0000 7f9eb3400640 1 -- 172.21.15.39:0/3592512036 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9eb4015070 con 0x7f9ebc0c50b0 2024-09-21T07:46:57.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.792+0000 7f9eb3400640 1 -- 172.21.15.39:0/3592512036 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4004d80 con 0x7f9ebc0c50b0 2024-09-21T07:46:57.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.792+0000 7f9ec0a00640 1 -- 172.21.15.39:0/3592512036 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ebc0c5a00 con 0x7f9ebc0c50b0 2024-09-21T07:46:57.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.792+0000 7f9ec0a00640 1 -- 172.21.15.39:0/3592512036 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ebc0ad5b0 con 0x7f9ebc0c50b0 2024-09-21T07:46:57.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.793+0000 7f9ec0a00640 1 -- 172.21.15.39:0/3592512036 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ebc0cd5a0 con 0x7f9ebc0c50b0 2024-09-21T07:46:57.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.794+0000 7f9eb3400640 1 -- 172.21.15.39:0/3592512036 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9eb4003970 con 0x7f9ebc0c50b0 2024-09-21T07:46:57.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.796+0000 7f9eb3400640 1 --2- 172.21.15.39:0/3592512036 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9e98078b90 0x7f9e9807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:57.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.796+0000 7f9eb3400640 1 -- 172.21.15.39:0/3592512036 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 6166+0+0 (secure 0 0 0) 0x7f9eb409b560 con 0x7f9ebc0c50b0 2024-09-21T07:46:57.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.796+0000 7f9ebb400640 1 --2- 172.21.15.39:0/3592512036 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9e98078b90 0x7f9e9807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:57.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.797+0000 7f9ebb400640 1 --2- 172.21.15.39:0/3592512036 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9e98078b90 0x7f9e9807b0a0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f9eac006fd0 tx=0x7f9eac006b10 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:57.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.800+0000 7f9eb3400640 1 -- 172.21.15.39:0/3592512036 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9ebc0c5a00 con 0x7f9ebc0c50b0 2024-09-21T07:46:57.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:57.997+0000 7f9ec0a00640 1 -- 172.21.15.39:0/3592512036 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f9ebc0af200 con 0x7f9ebc0c50b0 2024-09-21T07:46:58.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.001+0000 7f9eb3400640 1 -- 172.21.15.39:0/3592512036 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v48) ==== 71+0+0 (secure 0 0 0) 0x7f9ebc1f4320 con 0x7f9ebc0c50b0 2024-09-21T07:46:58.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.004+0000 7f9ec0a00640 1 -- 172.21.15.39:0/3592512036 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9e98078b90 msgr2=0x7f9e9807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:58.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.004+0000 7f9ec0a00640 1 --2- 172.21.15.39:0/3592512036 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9e98078b90 0x7f9e9807b0a0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f9eac006fd0 tx=0x7f9eac006b10 comp rx=0 tx=0).stop 2024-09-21T07:46:58.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.004+0000 7f9ec0a00640 1 -- 172.21.15.39:0/3592512036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ebc0c50b0 msgr2=0x7f9ebc0c54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:58.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.004+0000 7f9ec0a00640 1 --2- 172.21.15.39:0/3592512036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ebc0c50b0 0x7f9ebc0c54c0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4005fd0 tx=0x7f9eb4010f40 comp rx=0 tx=0).stop 2024-09-21T07:46:58.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.004+0000 7f9ec0a00640 1 -- 172.21.15.39:0/3592512036 shutdown_connections 2024-09-21T07:46:58.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.004+0000 7f9ec0a00640 1 --2- 172.21.15.39:0/3592512036 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9e98078b90 0x7f9e9807b0a0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:58.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.004+0000 7f9ec0a00640 1 --2- 172.21.15.39:0/3592512036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ebc0c50b0 0x7f9ebc0c54c0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:58.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.004+0000 7f9ec0a00640 1 --2- 172.21.15.39:0/3592512036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ebc0ae370 0x7f9ebc0ab580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:58.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.004+0000 7f9ec0a00640 1 --2- 172.21.15.39:0/3592512036 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9ea0003f90 0x7f9ebc1f3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:58.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.004+0000 7f9ec0a00640 1 -- 172.21.15.39:0/3592512036 >> 172.21.15.39:0/3592512036 conn(0x7f9ebc06cfc0 msgr2=0x7f9ebc06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:58.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.004+0000 7f9ec0a00640 1 -- 172.21.15.39:0/3592512036 shutdown_connections 2024-09-21T07:46:58.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.004+0000 7f9ec0a00640 1 -- 172.21.15.39:0/3592512036 wait complete. 2024-09-21T07:46:58.026 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_mds_max_snaps_per_dir_with_reduced_limit' 2024-09-21T07:46:58.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.255+0000 7f7860400640 1 -- 172.21.15.39:0/4144156527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78580cf920 msgr2=0x7f78580d0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:58.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.255+0000 7f7860400640 1 --2- 172.21.15.39:0/4144156527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78580cf920 0x7f78580d0630 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f78580b5a10 tx=0x7f784402a540 comp rx=0 tx=0).stop 2024-09-21T07:46:58.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.255+0000 7f7860400640 1 -- 172.21.15.39:0/4144156527 shutdown_connections 2024-09-21T07:46:58.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.255+0000 7f7860400640 1 --2- 172.21.15.39:0/4144156527 >> v2:172.21.15.148:3300/0 conn(0x7f78580cfd30 0x7f78580d0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:58.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.255+0000 7f7860400640 1 --2- 172.21.15.39:0/4144156527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78580cf920 0x7f78580d0630 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:58.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.255+0000 7f7860400640 1 --2- 172.21.15.39:0/4144156527 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78580b5240 0x7f785815c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:58.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.255+0000 7f7860400640 1 -- 172.21.15.39:0/4144156527 >> 172.21.15.39:0/4144156527 conn(0x7f785806cfc0 msgr2=0x7f785806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:58.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.255+0000 7f7860400640 1 -- 172.21.15.39:0/4144156527 shutdown_connections 2024-09-21T07:46:58.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.255+0000 7f7860400640 1 -- 172.21.15.39:0/4144156527 wait complete. 2024-09-21T07:46:58.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.257+0000 7f7860400640 1 Processor -- start 2024-09-21T07:46:58.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.257+0000 7f7860400640 1 -- start start 2024-09-21T07:46:58.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.259+0000 7f7860400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78580b5240 0x7f7858146770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:58.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.259+0000 7f7860400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78580cf920 0x7f785814df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:58.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.259+0000 7f7860400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78580cfd30 0x7f7858073130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:58.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.259+0000 7f7860400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f78580b4760 con 0x7f78580cfd30 2024-09-21T07:46:58.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.259+0000 7f7860400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f78580b45e0 con 0x7f78580cf920 2024-09-21T07:46:58.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.259+0000 7f7860400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f78580b48e0 con 0x7f78580b5240 2024-09-21T07:46:58.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.259+0000 7f785e400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78580cfd30 0x7f7858073130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:58.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.259+0000 7f785e400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78580cfd30 0x7f7858073130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34908/0 (socket says 172.21.15.39:34908) 2024-09-21T07:46:58.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.259+0000 7f785e400640 1 -- 172.21.15.39:0/1167520178 learned_addr learned my addr 172.21.15.39:0/1167520178 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:58.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.259+0000 7f785da00640 1 --2- 172.21.15.39:0/1167520178 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78580b5240 0x7f7858146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:58.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.259+0000 7f785d000640 1 --2- 172.21.15.39:0/1167520178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78580cf920 0x7f785814df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:58.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.259+0000 7f785e400640 1 -- 172.21.15.39:0/1167520178 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78580b5240 msgr2=0x7f7858146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:58.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.259+0000 7f785e400640 1 --2- 172.21.15.39:0/1167520178 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78580b5240 0x7f7858146770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:58.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.259+0000 7f785e400640 1 -- 172.21.15.39:0/1167520178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78580cf920 msgr2=0x7f785814df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:58.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.259+0000 7f785e400640 1 --2- 172.21.15.39:0/1167520178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78580cf920 0x7f785814df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:58.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.259+0000 7f785e400640 1 -- 172.21.15.39:0/1167520178 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f785814cb90 con 0x7f78580cfd30 2024-09-21T07:46:58.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.260+0000 7f785e400640 1 --2- 172.21.15.39:0/1167520178 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78580cfd30 0x7f7858073130 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f785000e960 tx=0x7f785000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:58.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.260+0000 7f7855600640 1 -- 172.21.15.39:0/1167520178 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850004d20 con 0x7f78580cfd30 2024-09-21T07:46:58.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.260+0000 7f7855600640 1 -- 172.21.15.39:0/1167520178 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f785000cb20 con 0x7f78580cfd30 2024-09-21T07:46:58.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.260+0000 7f7855600640 1 -- 172.21.15.39:0/1167520178 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850005020 con 0x7f78580cfd30 2024-09-21T07:46:58.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.260+0000 7f7860400640 1 -- 172.21.15.39:0/1167520178 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f785814ce70 con 0x7f78580cfd30 2024-09-21T07:46:58.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.260+0000 7f7860400640 1 -- 172.21.15.39:0/1167520178 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f785814d950 con 0x7f78580cfd30 2024-09-21T07:46:58.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.261+0000 7f7860400640 1 -- 172.21.15.39:0/1167520178 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78580b4e00 con 0x7f78580cfd30 2024-09-21T07:46:58.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.262+0000 7f7855600640 1 -- 172.21.15.39:0/1167520178 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f78500051c0 con 0x7f78580cfd30 2024-09-21T07:46:58.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.263+0000 7f7855600640 1 --2- 172.21.15.39:0/1167520178 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7838078b90 0x7f783807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:58.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.263+0000 7f7855600640 1 -- 172.21.15.39:0/1167520178 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 6166+0+0 (secure 0 0 0) 0x7f785009b770 con 0x7f78580cfd30 2024-09-21T07:46:58.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.264+0000 7f785da00640 1 --2- 172.21.15.39:0/1167520178 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7838078b90 0x7f783807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:58.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.265+0000 7f785da00640 1 --2- 172.21.15.39:0/1167520178 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7838078b90 0x7f783807b0a0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f784c004630 tx=0x7f784c00b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:58.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.268+0000 7f7855600640 1 -- 172.21.15.39:0/1167520178 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7850016030 con 0x7f78580cfd30 2024-09-21T07:46:58.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:58.538+0000 7f7860400640 1 -- 172.21.15.39:0/1167520178 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_mds_max_snaps_per_dir_with_reduced_limit"]} v 0) -- 0x7f7858154470 con 0x7f78580cfd30 2024-09-21T07:46:59.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.533+0000 7f7855600640 1 -- 172.21.15.39:0/1167520178 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_mds_max_snaps_per_dir_with_reduced_limit"]}]=0 v669) ==== 166+0+0 (secure 0 0 0) 0x7f7850063380 con 0x7f78580cfd30 2024-09-21T07:46:59.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.536+0000 7f7860400640 1 -- 172.21.15.39:0/1167520178 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7838078b90 msgr2=0x7f783807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:59.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.536+0000 7f7860400640 1 --2- 172.21.15.39:0/1167520178 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7838078b90 0x7f783807b0a0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f784c004630 tx=0x7f784c00b2e0 comp rx=0 tx=0).stop 2024-09-21T07:46:59.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.536+0000 7f7860400640 1 -- 172.21.15.39:0/1167520178 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78580cfd30 msgr2=0x7f7858073130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:59.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.536+0000 7f7860400640 1 --2- 172.21.15.39:0/1167520178 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78580cfd30 0x7f7858073130 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f785000e960 tx=0x7f785000ee20 comp rx=0 tx=0).stop 2024-09-21T07:46:59.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.536+0000 7f7860400640 1 -- 172.21.15.39:0/1167520178 shutdown_connections 2024-09-21T07:46:59.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.536+0000 7f7860400640 1 --2- 172.21.15.39:0/1167520178 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7838078b90 0x7f783807b0a0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:59.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.536+0000 7f7860400640 1 --2- 172.21.15.39:0/1167520178 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78580cfd30 0x7f7858073130 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:59.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.536+0000 7f7860400640 1 --2- 172.21.15.39:0/1167520178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78580cf920 0x7f785814df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:59.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.536+0000 7f7860400640 1 --2- 172.21.15.39:0/1167520178 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78580b5240 0x7f7858146770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:59.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.536+0000 7f7860400640 1 -- 172.21.15.39:0/1167520178 >> 172.21.15.39:0/1167520178 conn(0x7f785806cfc0 msgr2=0x7f785815af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:59.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.536+0000 7f7860400640 1 -- 172.21.15.39:0/1167520178 shutdown_connections 2024-09-21T07:46:59.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.536+0000 7f7860400640 1 -- 172.21.15.39:0/1167520178 wait complete. 2024-09-21T07:46:59.564 INFO:tasks.cephfs_test_runner:Test the newly introudced option named mds_max_snaps_per_dir ... ok 2024-09-21T07:46:59.565 INFO:tasks.cephfs_test_runner:Starting test: test_multimds_hardlink (tasks.cephfs.test_snapshots.TestSnapshots) check if hardlink snapshot works in multimds setup 2024-09-21T07:46:59.565 INFO:tasks.cephfs_test_runner:test_multimds_hardlink (tasks.cephfs.test_snapshots.TestSnapshots) 2024-09-21T07:46:59.565 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:46:59.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.794+0000 7f3426400640 1 -- 172.21.15.39:0/2485979647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34200b30a0 msgr2=0x7f34200cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:59.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.794+0000 7f3426400640 1 --2- 172.21.15.39:0/2485979647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34200b30a0 0x7f34200cb8a0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f3410005fd0 tx=0x7f341002a4a0 comp rx=0 tx=0).stop 2024-09-21T07:46:59.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.794+0000 7f3426400640 1 -- 172.21.15.39:0/2485979647 shutdown_connections 2024-09-21T07:46:59.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.794+0000 7f3426400640 1 --2- 172.21.15.39:0/2485979647 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f34200ccc60 0x7f34200cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:59.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.794+0000 7f3426400640 1 --2- 172.21.15.39:0/2485979647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34200b30a0 0x7f34200cb8a0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:59.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.794+0000 7f3426400640 1 -- 172.21.15.39:0/2485979647 >> 172.21.15.39:0/2485979647 conn(0x7f34200b2460 msgr2=0x7f342006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:59.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.794+0000 7f3426400640 1 -- 172.21.15.39:0/2485979647 shutdown_connections 2024-09-21T07:46:59.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.794+0000 7f3426400640 1 -- 172.21.15.39:0/2485979647 wait complete. 2024-09-21T07:46:59.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.796+0000 7f3426400640 1 Processor -- start 2024-09-21T07:46:59.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.796+0000 7f3426400640 1 -- start start 2024-09-21T07:46:59.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.798+0000 7f3426400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34200b30a0 0x7f34201f62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:59.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.798+0000 7f3426400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f34200ccc60 0x7f34200c8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:59.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.798+0000 7f3426400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f342015d830 0x7f34201558f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:59.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.798+0000 7f3426400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f34200af410 con 0x7f342015d830 2024-09-21T07:46:59.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.798+0000 7f3426400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f34200af290 con 0x7f34200b30a0 2024-09-21T07:46:59.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.798+0000 7f3426400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f34200af590 con 0x7f34200ccc60 2024-09-21T07:46:59.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.798+0000 7f341fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f342015d830 0x7f34201558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:59.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.798+0000 7f341fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f342015d830 0x7f34201558f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34930/0 (socket says 172.21.15.39:34930) 2024-09-21T07:46:59.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.798+0000 7f341fe00640 1 -- 172.21.15.39:0/3978197821 learned_addr learned my addr 172.21.15.39:0/3978197821 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:46:59.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.798+0000 7f341f400640 1 --2- 172.21.15.39:0/3978197821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34200b30a0 0x7f34201f62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:59.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.798+0000 7f3417e00640 1 --2- 172.21.15.39:0/3978197821 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f34200ccc60 0x7f34200c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:59.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.799+0000 7f341fe00640 1 -- 172.21.15.39:0/3978197821 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f34200ccc60 msgr2=0x7f34200c8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:59.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.799+0000 7f341fe00640 1 --2- 172.21.15.39:0/3978197821 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f34200ccc60 0x7f34200c8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:59.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.799+0000 7f341fe00640 1 -- 172.21.15.39:0/3978197821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34200b30a0 msgr2=0x7f34201f62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:59.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.799+0000 7f341fe00640 1 --2- 172.21.15.39:0/3978197821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34200b30a0 0x7f34201f62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:59.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.799+0000 7f341fe00640 1 -- 172.21.15.39:0/3978197821 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34200ce2d0 con 0x7f342015d830 2024-09-21T07:46:59.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.799+0000 7f3417e00640 1 --2- 172.21.15.39:0/3978197821 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f34200ccc60 0x7f34200c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:59.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.799+0000 7f341fe00640 1 --2- 172.21.15.39:0/3978197821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f342015d830 0x7f34201558f0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f341800e990 tx=0x7f341800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:59.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.799+0000 7f3417400640 1 -- 172.21.15.39:0/3978197821 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3418004d20 con 0x7f342015d830 2024-09-21T07:46:59.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.799+0000 7f3417400640 1 -- 172.21.15.39:0/3978197821 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f341800caa0 con 0x7f342015d830 2024-09-21T07:46:59.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.799+0000 7f3417400640 1 -- 172.21.15.39:0/3978197821 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3418005020 con 0x7f342015d830 2024-09-21T07:46:59.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.799+0000 7f3426400640 1 -- 172.21.15.39:0/3978197821 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34200ce5b0 con 0x7f342015d830 2024-09-21T07:46:59.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.799+0000 7f3426400640 1 -- 172.21.15.39:0/3978197821 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34200ceb70 con 0x7f342015d830 2024-09-21T07:46:59.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.801+0000 7f3426400640 1 -- 172.21.15.39:0/3978197821 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34200af9d0 con 0x7f342015d830 2024-09-21T07:46:59.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.802+0000 7f3417400640 1 -- 172.21.15.39:0/3978197821 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f34180051c0 con 0x7f342015d830 2024-09-21T07:46:59.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.803+0000 7f3417400640 1 --2- 172.21.15.39:0/3978197821 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f33fc078bc0 0x7f33fc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:59.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.803+0000 7f3417400640 1 -- 172.21.15.39:0/3978197821 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 6166+0+0 (secure 0 0 0) 0x7f341809b820 con 0x7f342015d830 2024-09-21T07:46:59.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.803+0000 7f341f400640 1 --2- 172.21.15.39:0/3978197821 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f33fc078bc0 0x7f33fc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:59.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.804+0000 7f341f400640 1 --2- 172.21.15.39:0/3978197821 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f33fc078bc0 0x7f33fc07b0d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f3410033000 tx=0x7f3410004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:59.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:46:59.808+0000 7f3417400640 1 -- 172.21.15.39:0/3978197821 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3418016030 con 0x7f342015d830 2024-09-21T07:47:00.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.006+0000 7f3426400640 1 -- 172.21.15.39:0/3978197821 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f34200ce060 con 0x7f342015d830 2024-09-21T07:47:00.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.007+0000 7f3417400640 1 -- 172.21.15.39:0/3978197821 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v186) ==== 74+0+18719 (secure 0 0 0) 0x7f34180633a0 con 0x7f342015d830 2024-09-21T07:47:00.008 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:47:00.009 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":186,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:46:57.511424+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6835/3000703195":"2024-09-22T07:46:54.243903+0000","172.21.15.39:6833/3000703195":"2024-09-22T07:46:54.243903+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]},{"pool":20,"snaps":[{"begin":80,"length":23},{"begin":181,"length":23}]},{"pool":22,"snaps":[{"begin":100,"length":1},{"begin":199,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:47:00.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.010+0000 7f3426400640 1 -- 172.21.15.39:0/3978197821 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f33fc078bc0 msgr2=0x7f33fc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:00.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.010+0000 7f3426400640 1 --2- 172.21.15.39:0/3978197821 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f33fc078bc0 0x7f33fc07b0d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f3410033000 tx=0x7f3410004730 comp rx=0 tx=0).stop 2024-09-21T07:47:00.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.010+0000 7f3426400640 1 -- 172.21.15.39:0/3978197821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f342015d830 msgr2=0x7f34201558f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:00.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.010+0000 7f3426400640 1 --2- 172.21.15.39:0/3978197821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f342015d830 0x7f34201558f0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f341800e990 tx=0x7f341800ee50 comp rx=0 tx=0).stop 2024-09-21T07:47:00.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.010+0000 7f3426400640 1 -- 172.21.15.39:0/3978197821 shutdown_connections 2024-09-21T07:47:00.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.010+0000 7f3426400640 1 --2- 172.21.15.39:0/3978197821 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f33fc078bc0 0x7f33fc07b0d0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:00.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.010+0000 7f3426400640 1 --2- 172.21.15.39:0/3978197821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f342015d830 0x7f34201558f0 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:00.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.010+0000 7f3426400640 1 --2- 172.21.15.39:0/3978197821 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f34200ccc60 0x7f34200c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:00.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.010+0000 7f3426400640 1 --2- 172.21.15.39:0/3978197821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34200b30a0 0x7f34201f62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:00.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.010+0000 7f3426400640 1 -- 172.21.15.39:0/3978197821 >> 172.21.15.39:0/3978197821 conn(0x7f34200b2460 msgr2=0x7f34200b34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:00.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.010+0000 7f3426400640 1 -- 172.21.15.39:0/3978197821 shutdown_connections 2024-09-21T07:47:00.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.010+0000 7f3426400640 1 -- 172.21.15.39:0/3978197821 wait complete. 2024-09-21T07:47:00.037 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T07:33:01.569433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'snap-test-pool', 'create_time': '2024-09-21T07:34:22.032185+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 1, 'snap_epoch': 37, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'test_data_pool', 'create_time': '2024-09-21T07:35:25.604532+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 1, 'snap_epoch': 66, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_metadata_pool', 'create_time': '2024-09-21T07:35:26.327607+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T07:47:00.038 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T07:47:00.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.269+0000 7fe587c00640 1 -- 172.21.15.39:0/3244735697 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5800706f0 msgr2=0x7fe58006ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:00.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.269+0000 7fe587c00640 1 --2- 172.21.15.39:0/3244735697 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5800706f0 0x7fe58006ef90 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fe574009830 tx=0x7fe57402a530 comp rx=0 tx=0).stop 2024-09-21T07:47:00.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.269+0000 7fe587c00640 1 -- 172.21.15.39:0/3244735697 shutdown_connections 2024-09-21T07:47:00.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.269+0000 7fe587c00640 1 --2- 172.21.15.39:0/3244735697 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5800706f0 0x7fe58006ef90 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:00.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.269+0000 7fe587c00640 1 -- 172.21.15.39:0/3244735697 >> 172.21.15.39:0/3244735697 conn(0x7fe58006d7a0 msgr2=0x7fe5800b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:00.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.269+0000 7fe587c00640 1 -- 172.21.15.39:0/3244735697 shutdown_connections 2024-09-21T07:47:00.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.269+0000 7fe587c00640 1 -- 172.21.15.39:0/3244735697 wait complete. 2024-09-21T07:47:00.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.271+0000 7fe587c00640 1 Processor -- start 2024-09-21T07:47:00.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.271+0000 7fe587c00640 1 -- start start 2024-09-21T07:47:00.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.272+0000 7fe587c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5800706f0 0x7fe5800d0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:00.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.272+0000 7fe587c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe5800af120 0x7fe58009caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:00.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.273+0000 7fe587c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe58015d180 0x7fe58009cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:00.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.273+0000 7fe587c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe5800b3910 con 0x7fe5800706f0 2024-09-21T07:47:00.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.273+0000 7fe587c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe5800b3790 con 0x7fe58015d180 2024-09-21T07:47:00.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.273+0000 7fe587c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe5800b3a90 con 0x7fe5800af120 2024-09-21T07:47:00.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.273+0000 7fe585200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5800706f0 0x7fe5800d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:00.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.273+0000 7fe585200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5800706f0 0x7fe5800d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34948/0 (socket says 172.21.15.39:34948) 2024-09-21T07:47:00.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.273+0000 7fe585200640 1 -- 172.21.15.39:0/458181728 learned_addr learned my addr 172.21.15.39:0/458181728 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:00.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.273+0000 7fe585c00640 1 --2- 172.21.15.39:0/458181728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe58015d180 0x7fe58009cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:00.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.273+0000 7fe57fe00640 1 --2- 172.21.15.39:0/458181728 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe5800af120 0x7fe58009caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:00.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.273+0000 7fe585200640 1 -- 172.21.15.39:0/458181728 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe5800af120 msgr2=0x7fe58009caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:00.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.273+0000 7fe585200640 1 --2- 172.21.15.39:0/458181728 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe5800af120 0x7fe58009caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:00.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.273+0000 7fe585200640 1 -- 172.21.15.39:0/458181728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe58015d180 msgr2=0x7fe58009cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:00.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.273+0000 7fe585200640 1 --2- 172.21.15.39:0/458181728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe58015d180 0x7fe58009cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:00.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.273+0000 7fe585200640 1 -- 172.21.15.39:0/458181728 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5800cea40 con 0x7fe5800706f0 2024-09-21T07:47:00.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.273+0000 7fe585c00640 1 --2- 172.21.15.39:0/458181728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe58015d180 0x7fe58009cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:47:00.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.274+0000 7fe585200640 1 --2- 172.21.15.39:0/458181728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5800706f0 0x7fe5800d0280 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fe574000c00 tx=0x7fe574033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:00.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.274+0000 7fe57cc00640 1 -- 172.21.15.39:0/458181728 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe574031d60 con 0x7fe5800706f0 2024-09-21T07:47:00.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.274+0000 7fe57cc00640 1 -- 172.21.15.39:0/458181728 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe574033e10 con 0x7fe5800706f0 2024-09-21T07:47:00.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.274+0000 7fe57cc00640 1 -- 172.21.15.39:0/458181728 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe574033310 con 0x7fe5800706f0 2024-09-21T07:47:00.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.274+0000 7fe587c00640 1 -- 172.21.15.39:0/458181728 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5800ced20 con 0x7fe5800706f0 2024-09-21T07:47:00.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.274+0000 7fe587c00640 1 -- 172.21.15.39:0/458181728 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5800cf2e0 con 0x7fe5800706f0 2024-09-21T07:47:00.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.275+0000 7fe587c00640 1 -- 172.21.15.39:0/458181728 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5800b2b40 con 0x7fe5800706f0 2024-09-21T07:47:00.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.276+0000 7fe57cc00640 1 -- 172.21.15.39:0/458181728 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe5740334b0 con 0x7fe5800706f0 2024-09-21T07:47:00.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.277+0000 7fe57cc00640 1 --2- 172.21.15.39:0/458181728 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe560078bc0 0x7fe56007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:00.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.277+0000 7fe57cc00640 1 -- 172.21.15.39:0/458181728 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 6166+0+0 (secure 0 0 0) 0x7fe5740bc2c0 con 0x7fe5800706f0 2024-09-21T07:47:00.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.278+0000 7fe57fe00640 1 --2- 172.21.15.39:0/458181728 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe560078bc0 0x7fe56007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:00.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.279+0000 7fe57fe00640 1 --2- 172.21.15.39:0/458181728 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe560078bc0 0x7fe56007b0d0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fe5801571c0 tx=0x7fe56c009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:00.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.282+0000 7fe57cc00640 1 -- 172.21.15.39:0/458181728 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe57403f030 con 0x7fe5800706f0 2024-09-21T07:47:00.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.480+0000 7fe587c00640 1 -- 172.21.15.39:0/458181728 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe5800b0440 con 0x7fe5800706f0 2024-09-21T07:47:00.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.480+0000 7fe57cc00640 1 -- 172.21.15.39:0/458181728 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v186) ==== 93+0+10 (secure 0 0 0) 0x7fe574084520 con 0x7fe5800706f0 2024-09-21T07:47:00.481 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-09-21T07:47:00.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.483+0000 7fe587c00640 1 -- 172.21.15.39:0/458181728 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe560078bc0 msgr2=0x7fe56007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:00.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.483+0000 7fe587c00640 1 --2- 172.21.15.39:0/458181728 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe560078bc0 0x7fe56007b0d0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fe5801571c0 tx=0x7fe56c009040 comp rx=0 tx=0).stop 2024-09-21T07:47:00.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.483+0000 7fe587c00640 1 -- 172.21.15.39:0/458181728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5800706f0 msgr2=0x7fe5800d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:00.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.483+0000 7fe587c00640 1 --2- 172.21.15.39:0/458181728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5800706f0 0x7fe5800d0280 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fe574000c00 tx=0x7fe574033c20 comp rx=0 tx=0).stop 2024-09-21T07:47:00.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.483+0000 7fe587c00640 1 -- 172.21.15.39:0/458181728 shutdown_connections 2024-09-21T07:47:00.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.483+0000 7fe587c00640 1 --2- 172.21.15.39:0/458181728 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe560078bc0 0x7fe56007b0d0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:00.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.483+0000 7fe587c00640 1 --2- 172.21.15.39:0/458181728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe58015d180 0x7fe58009cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:00.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.483+0000 7fe587c00640 1 --2- 172.21.15.39:0/458181728 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe5800af120 0x7fe58009caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:00.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.483+0000 7fe587c00640 1 --2- 172.21.15.39:0/458181728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5800706f0 0x7fe5800d0280 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:00.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.483+0000 7fe587c00640 1 -- 172.21.15.39:0/458181728 >> 172.21.15.39:0/458181728 conn(0x7fe58006d7a0 msgr2=0x7fe58009a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:00.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.483+0000 7fe587c00640 1 -- 172.21.15.39:0/458181728 shutdown_connections 2024-09-21T07:47:00.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.483+0000 7fe587c00640 1 -- 172.21.15.39:0/458181728 wait complete. 2024-09-21T07:47:00.505 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2024-09-21T07:47:00.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.736+0000 7f62ef200640 1 -- 172.21.15.39:0/2505817365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62f0160660 msgr2=0x7f62f00cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:00.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.736+0000 7f62ef200640 1 --2- 172.21.15.39:0/2505817365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62f0160660 0x7f62f00cf710 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f62d8005fd0 tx=0x7f62d802a620 comp rx=0 tx=0).stop 2024-09-21T07:47:00.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.736+0000 7f62ef200640 1 -- 172.21.15.39:0/2505817365 shutdown_connections 2024-09-21T07:47:00.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.736+0000 7f62ef200640 1 --2- 172.21.15.39:0/2505817365 >> v2:172.21.15.148:3300/0 conn(0x7f62f00cfc50 0x7f62f00d0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:00.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.736+0000 7f62ef200640 1 --2- 172.21.15.39:0/2505817365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62f0160660 0x7f62f00cf710 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:00.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.736+0000 7f62ef200640 1 -- 172.21.15.39:0/2505817365 >> 172.21.15.39:0/2505817365 conn(0x7f62f006cfc0 msgr2=0x7f62f006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:00.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.736+0000 7f62ef200640 1 -- 172.21.15.39:0/2505817365 shutdown_connections 2024-09-21T07:47:00.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.736+0000 7f62ef200640 1 -- 172.21.15.39:0/2505817365 wait complete. 2024-09-21T07:47:00.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.738+0000 7f62ef200640 1 Processor -- start 2024-09-21T07:47:00.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.739+0000 7f62ef200640 1 -- start start 2024-09-21T07:47:00.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.740+0000 7f62ef200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f62f00cfc50 0x7f62f014b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:00.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.740+0000 7f62ef200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62f0160250 0x7f62f0148310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:00.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.740+0000 7f62ef200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62f0160660 0x7f62f01f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:00.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.740+0000 7f62ef200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f62f00ccdc0 con 0x7f62f0160660 2024-09-21T07:47:00.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.740+0000 7f62ef200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f62f00ccc40 con 0x7f62f0160250 2024-09-21T07:47:00.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.740+0000 7f62ef200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f62f00ccf40 con 0x7f62f00cfc50 2024-09-21T07:47:00.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.740+0000 7f62ee800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62f0160660 0x7f62f01f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:00.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.741+0000 7f62ee800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62f0160660 0x7f62f01f89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34976/0 (socket says 172.21.15.39:34976) 2024-09-21T07:47:00.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.741+0000 7f62ee800640 1 -- 172.21.15.39:0/2000431569 learned_addr learned my addr 172.21.15.39:0/2000431569 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:00.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.741+0000 7f62e5400640 1 --2- 172.21.15.39:0/2000431569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62f0160250 0x7f62f0148310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:00.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.741+0000 7f62ede00640 1 --2- 172.21.15.39:0/2000431569 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f62f00cfc50 0x7f62f014b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:00.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.741+0000 7f62ee800640 1 -- 172.21.15.39:0/2000431569 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f62f00cfc50 msgr2=0x7f62f014b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:00.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.741+0000 7f62ee800640 1 --2- 172.21.15.39:0/2000431569 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f62f00cfc50 0x7f62f014b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:00.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.741+0000 7f62ee800640 1 -- 172.21.15.39:0/2000431569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62f0160250 msgr2=0x7f62f0148310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:00.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.741+0000 7f62ee800640 1 --2- 172.21.15.39:0/2000431569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62f0160250 0x7f62f0148310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:00.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.741+0000 7f62ee800640 1 -- 172.21.15.39:0/2000431569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62f00b15e0 con 0x7f62f0160660 2024-09-21T07:47:00.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.741+0000 7f62ee800640 1 --2- 172.21.15.39:0/2000431569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62f0160660 0x7f62f01f89b0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f62e800e960 tx=0x7f62e800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:00.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.742+0000 7f62e6a00640 1 -- 172.21.15.39:0/2000431569 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e8004d20 con 0x7f62f0160660 2024-09-21T07:47:00.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.742+0000 7f62e6a00640 1 -- 172.21.15.39:0/2000431569 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f62e800cb20 con 0x7f62f0160660 2024-09-21T07:47:00.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.742+0000 7f62e6a00640 1 -- 172.21.15.39:0/2000431569 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e8005020 con 0x7f62f0160660 2024-09-21T07:47:00.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.742+0000 7f62ef200640 1 -- 172.21.15.39:0/2000431569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62f00b18c0 con 0x7f62f0160660 2024-09-21T07:47:00.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.742+0000 7f62ef200640 1 -- 172.21.15.39:0/2000431569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62f00b1e80 con 0x7f62f0160660 2024-09-21T07:47:00.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.743+0000 7f62ef200640 1 -- 172.21.15.39:0/2000431569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62f00b1350 con 0x7f62f0160660 2024-09-21T07:47:00.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.744+0000 7f62e6a00640 1 -- 172.21.15.39:0/2000431569 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f62e80051c0 con 0x7f62f0160660 2024-09-21T07:47:00.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.745+0000 7f62e6a00640 1 --2- 172.21.15.39:0/2000431569 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f62cc078bc0 0x7f62cc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:00.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.745+0000 7f62e6a00640 1 -- 172.21.15.39:0/2000431569 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 6166+0+0 (secure 0 0 0) 0x7f62e809b820 con 0x7f62f0160660 2024-09-21T07:47:00.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.746+0000 7f62ede00640 1 --2- 172.21.15.39:0/2000431569 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f62cc078bc0 0x7f62cc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:00.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.746+0000 7f62ede00640 1 --2- 172.21.15.39:0/2000431569 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f62cc078bc0 0x7f62cc07b0d0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f62e0001720 tx=0x7f62e0009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:00.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.750+0000 7f62e6a00640 1 -- 172.21.15.39:0/2000431569 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f62e8016030 con 0x7f62f0160660 2024-09-21T07:47:00.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.946+0000 7f62ef200640 1 -- 172.21.15.39:0/2000431569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) -- 0x7f62f00cd460 con 0x7f62f0160660 2024-09-21T07:47:00.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.947+0000 7f62e6a00640 1 -- 172.21.15.39:0/2000431569 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v186) ==== 103+0+11 (secure 0 0 0) 0x7f62e80633a0 con 0x7f62f0160660 2024-09-21T07:47:00.948 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-21T07:47:00.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.949+0000 7f62ef200640 1 -- 172.21.15.39:0/2000431569 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f62cc078bc0 msgr2=0x7f62cc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:00.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.949+0000 7f62ef200640 1 --2- 172.21.15.39:0/2000431569 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f62cc078bc0 0x7f62cc07b0d0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f62e0001720 tx=0x7f62e0009040 comp rx=0 tx=0).stop 2024-09-21T07:47:00.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.949+0000 7f62ef200640 1 -- 172.21.15.39:0/2000431569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62f0160660 msgr2=0x7f62f01f89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:00.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.949+0000 7f62ef200640 1 --2- 172.21.15.39:0/2000431569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62f0160660 0x7f62f01f89b0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f62e800e960 tx=0x7f62e800ee20 comp rx=0 tx=0).stop 2024-09-21T07:47:00.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.949+0000 7f62ef200640 1 -- 172.21.15.39:0/2000431569 shutdown_connections 2024-09-21T07:47:00.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.949+0000 7f62ef200640 1 --2- 172.21.15.39:0/2000431569 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f62cc078bc0 0x7f62cc07b0d0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:00.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.949+0000 7f62ef200640 1 --2- 172.21.15.39:0/2000431569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62f0160660 0x7f62f01f89b0 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:00.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.949+0000 7f62ef200640 1 --2- 172.21.15.39:0/2000431569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62f0160250 0x7f62f0148310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:00.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.949+0000 7f62ef200640 1 --2- 172.21.15.39:0/2000431569 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f62f00cfc50 0x7f62f014b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:00.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.949+0000 7f62ef200640 1 -- 172.21.15.39:0/2000431569 >> 172.21.15.39:0/2000431569 conn(0x7f62f006cfc0 msgr2=0x7f62f015a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:00.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.949+0000 7f62ef200640 1 -- 172.21.15.39:0/2000431569 shutdown_connections 2024-09-21T07:47:00.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:00.949+0000 7f62ef200640 1 -- 172.21.15.39:0/2000431569 wait complete. 2024-09-21T07:47:00.971 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2024-09-21T07:47:01.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.199+0000 7f480ee00640 1 -- 172.21.15.39:0/734260185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48080705f0 msgr2=0x7f480815f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:01.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.199+0000 7f480ee00640 1 --2- 172.21.15.39:0/734260185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48080705f0 0x7f480815f580 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f4800005fd0 tx=0x7f480002a820 comp rx=0 tx=0).stop 2024-09-21T07:47:01.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.199+0000 7f480ee00640 1 -- 172.21.15.39:0/734260185 shutdown_connections 2024-09-21T07:47:01.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.199+0000 7f480ee00640 1 --2- 172.21.15.39:0/734260185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48080705f0 0x7f480815f580 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:01.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.199+0000 7f480ee00640 1 --2- 172.21.15.39:0/734260185 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f480815c570 0x7f48080b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:01.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.199+0000 7f480ee00640 1 -- 172.21.15.39:0/734260185 >> 172.21.15.39:0/734260185 conn(0x7f48080b6f80 msgr2=0x7f480806def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:01.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.199+0000 7f480ee00640 1 -- 172.21.15.39:0/734260185 shutdown_connections 2024-09-21T07:47:01.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.199+0000 7f480ee00640 1 -- 172.21.15.39:0/734260185 wait complete. 2024-09-21T07:47:01.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.201+0000 7f480ee00640 1 Processor -- start 2024-09-21T07:47:01.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.201+0000 7f480ee00640 1 -- start start 2024-09-21T07:47:01.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.203+0000 7f480ee00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f480815c570 0x7f48080b05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:01.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.203+0000 7f480ee00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f480815f170 0x7f4808146bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:01.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.203+0000 7f480ee00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48080b3280 0x7f48080b3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:01.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.203+0000 7f480ee00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f48080cd4c0 con 0x7f48080b3280 2024-09-21T07:47:01.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.203+0000 7f480ee00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f48080cd340 con 0x7f480815f170 2024-09-21T07:47:01.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.203+0000 7f480ee00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f48080cd640 con 0x7f480815c570 2024-09-21T07:47:01.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.203+0000 7f480e400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48080b3280 0x7f48080b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:01.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.203+0000 7f480e400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48080b3280 0x7f48080b3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34992/0 (socket says 172.21.15.39:34992) 2024-09-21T07:47:01.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.203+0000 7f480e400640 1 -- 172.21.15.39:0/2995941001 learned_addr learned my addr 172.21.15.39:0/2995941001 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:01.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.203+0000 7f480d000640 1 --2- 172.21.15.39:0/2995941001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f480815f170 0x7f4808146bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:01.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.203+0000 7f480da00640 1 --2- 172.21.15.39:0/2995941001 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f480815c570 0x7f48080b05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:01.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.203+0000 7f480e400640 1 -- 172.21.15.39:0/2995941001 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f480815c570 msgr2=0x7f48080b05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:01.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.203+0000 7f480e400640 1 --2- 172.21.15.39:0/2995941001 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f480815c570 0x7f48080b05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:01.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.203+0000 7f480e400640 1 -- 172.21.15.39:0/2995941001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f480815f170 msgr2=0x7f4808146bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:01.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.203+0000 7f480e400640 1 --2- 172.21.15.39:0/2995941001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f480815f170 0x7f4808146bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:01.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.203+0000 7f480e400640 1 -- 172.21.15.39:0/2995941001 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48080b3bd0 con 0x7f48080b3280 2024-09-21T07:47:01.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.204+0000 7f480e400640 1 --2- 172.21.15.39:0/2995941001 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48080b3280 0x7f48080b3690 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f4800006010 tx=0x7f4800032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:01.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.204+0000 7f4805600640 1 -- 172.21.15.39:0/2995941001 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4800037040 con 0x7f48080b3280 2024-09-21T07:47:01.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.204+0000 7f4805600640 1 -- 172.21.15.39:0/2995941001 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4800042070 con 0x7f48080b3280 2024-09-21T07:47:01.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.204+0000 7f4805600640 1 -- 172.21.15.39:0/2995941001 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48000042a0 con 0x7f48080b3280 2024-09-21T07:47:01.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.204+0000 7f480ee00640 1 -- 172.21.15.39:0/2995941001 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f480815a500 con 0x7f48080b3280 2024-09-21T07:47:01.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.204+0000 7f480ee00640 1 -- 172.21.15.39:0/2995941001 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f480815aac0 con 0x7f48080b3280 2024-09-21T07:47:01.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.205+0000 7f480ee00640 1 -- 172.21.15.39:0/2995941001 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48080cdaf0 con 0x7f48080b3280 2024-09-21T07:47:01.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.207+0000 7f4805600640 1 -- 172.21.15.39:0/2995941001 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4800004440 con 0x7f48080b3280 2024-09-21T07:47:01.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.208+0000 7f4805600640 1 --2- 172.21.15.39:0/2995941001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f47e8078b90 0x7f47e807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:01.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.208+0000 7f4805600640 1 -- 172.21.15.39:0/2995941001 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 6166+0+0 (secure 0 0 0) 0x7f48000bbc70 con 0x7f48080b3280 2024-09-21T07:47:01.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.208+0000 7f480da00640 1 --2- 172.21.15.39:0/2995941001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f47e8078b90 0x7f47e807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:01.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.209+0000 7f480da00640 1 --2- 172.21.15.39:0/2995941001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f47e8078b90 0x7f47e807b0a0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f48080cee70 tx=0x7f47fc00b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:01.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.212+0000 7f4805600640 1 -- 172.21.15.39:0/2995941001 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f48080b3bd0 con 0x7f48080b3280 2024-09-21T07:47:01.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.408+0000 7f480ee00640 1 -- 172.21.15.39:0/2995941001 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) -- 0x7f480815a250 con 0x7f48080b3280 2024-09-21T07:47:01.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.408+0000 7f4805600640 1 -- 172.21.15.39:0/2995941001 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v186) ==== 103+0+11 (secure 0 0 0) 0x7f48000837f0 con 0x7f48080b3280 2024-09-21T07:47:01.409 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-21T07:47:01.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.411+0000 7f480ee00640 1 -- 172.21.15.39:0/2995941001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f47e8078b90 msgr2=0x7f47e807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:01.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.411+0000 7f480ee00640 1 --2- 172.21.15.39:0/2995941001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f47e8078b90 0x7f47e807b0a0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f48080cee70 tx=0x7f47fc00b2e0 comp rx=0 tx=0).stop 2024-09-21T07:47:01.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.411+0000 7f480ee00640 1 -- 172.21.15.39:0/2995941001 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48080b3280 msgr2=0x7f48080b3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:01.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.411+0000 7f480ee00640 1 --2- 172.21.15.39:0/2995941001 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48080b3280 0x7f48080b3690 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f4800006010 tx=0x7f4800032c90 comp rx=0 tx=0).stop 2024-09-21T07:47:01.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.411+0000 7f480ee00640 1 -- 172.21.15.39:0/2995941001 shutdown_connections 2024-09-21T07:47:01.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.411+0000 7f480ee00640 1 --2- 172.21.15.39:0/2995941001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f47e8078b90 0x7f47e807b0a0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:01.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.411+0000 7f480ee00640 1 --2- 172.21.15.39:0/2995941001 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48080b3280 0x7f48080b3690 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:01.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.411+0000 7f480ee00640 1 --2- 172.21.15.39:0/2995941001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f480815f170 0x7f4808146bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:01.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.411+0000 7f480ee00640 1 --2- 172.21.15.39:0/2995941001 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f480815c570 0x7f48080b05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:01.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.411+0000 7f480ee00640 1 -- 172.21.15.39:0/2995941001 >> 172.21.15.39:0/2995941001 conn(0x7f48080b6f80 msgr2=0x7f4808151c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:01.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.411+0000 7f480ee00640 1 -- 172.21.15.39:0/2995941001 shutdown_connections 2024-09-21T07:47:01.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.411+0000 7f480ee00640 1 -- 172.21.15.39:0/2995941001 wait complete. 2024-09-21T07:47:01.437 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2024-09-21T07:47:01.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.666+0000 7fad67200640 1 -- 172.21.15.39:0/3965859014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad60160250 msgr2=0x7fad6015d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:01.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.666+0000 7fad67200640 1 --2- 172.21.15.39:0/3965859014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad60160250 0x7fad6015d570 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fad54009860 tx=0x7fad5402a580 comp rx=0 tx=0).stop 2024-09-21T07:47:01.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.666+0000 7fad67200640 1 -- 172.21.15.39:0/3965859014 shutdown_connections 2024-09-21T07:47:01.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.666+0000 7fad67200640 1 --2- 172.21.15.39:0/3965859014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad60160250 0x7fad6015d570 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:01.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.666+0000 7fad67200640 1 -- 172.21.15.39:0/3965859014 >> 172.21.15.39:0/3965859014 conn(0x7fad6006cfc0 msgr2=0x7fad6006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:01.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.666+0000 7fad67200640 1 -- 172.21.15.39:0/3965859014 shutdown_connections 2024-09-21T07:47:01.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.666+0000 7fad67200640 1 -- 172.21.15.39:0/3965859014 wait complete. 2024-09-21T07:47:01.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.668+0000 7fad67200640 1 Processor -- start 2024-09-21T07:47:01.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.668+0000 7fad67200640 1 -- start start 2024-09-21T07:47:01.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.670+0000 7fad67200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad600d0390 0x7fad601f9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:01.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.670+0000 7fad67200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad60160250 0x7fad601f90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:01.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.670+0000 7fad67200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fad60160660 0x7fad6013b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:01.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.670+0000 7fad67200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fad600ccf00 con 0x7fad60160250 2024-09-21T07:47:01.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.670+0000 7fad67200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fad600ccd80 con 0x7fad600d0390 2024-09-21T07:47:01.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.670+0000 7fad67200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fad600cd080 con 0x7fad60160660 2024-09-21T07:47:01.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.670+0000 7fad65400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad60160250 0x7fad601f90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:01.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.670+0000 7fad65400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad60160250 0x7fad601f90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34994/0 (socket says 172.21.15.39:34994) 2024-09-21T07:47:01.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.670+0000 7fad65400640 1 -- 172.21.15.39:0/1783612649 learned_addr learned my addr 172.21.15.39:0/1783612649 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:01.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.670+0000 7fad66800640 1 --2- 172.21.15.39:0/1783612649 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fad60160660 0x7fad6013b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:01.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.671+0000 7fad65e00640 1 --2- 172.21.15.39:0/1783612649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad600d0390 0x7fad601f9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:01.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.671+0000 7fad65400640 1 -- 172.21.15.39:0/1783612649 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fad60160660 msgr2=0x7fad6013b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:01.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.671+0000 7fad65400640 1 --2- 172.21.15.39:0/1783612649 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fad60160660 0x7fad6013b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:01.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.671+0000 7fad65400640 1 -- 172.21.15.39:0/1783612649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad600d0390 msgr2=0x7fad601f9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:01.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.671+0000 7fad65400640 1 --2- 172.21.15.39:0/1783612649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad600d0390 0x7fad601f9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:01.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.671+0000 7fad65400640 1 -- 172.21.15.39:0/1783612649 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad600ad8e0 con 0x7fad60160250 2024-09-21T07:47:01.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.671+0000 7fad65e00640 1 --2- 172.21.15.39:0/1783612649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad600d0390 0x7fad601f9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:01.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.671+0000 7fad66800640 1 --2- 172.21.15.39:0/1783612649 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fad60160660 0x7fad6013b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:01.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.671+0000 7fad65400640 1 --2- 172.21.15.39:0/1783612649 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad60160250 0x7fad601f90c0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fad5000c430 tx=0x7fad5000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:01.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.671+0000 7fad5a000640 1 -- 172.21.15.39:0/1783612649 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad50017070 con 0x7fad60160250 2024-09-21T07:47:01.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.671+0000 7fad5a000640 1 -- 172.21.15.39:0/1783612649 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad500053e0 con 0x7fad60160250 2024-09-21T07:47:01.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.671+0000 7fad67200640 1 -- 172.21.15.39:0/1783612649 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad600b24b0 con 0x7fad60160250 2024-09-21T07:47:01.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.671+0000 7fad5a000640 1 -- 172.21.15.39:0/1783612649 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad50003850 con 0x7fad60160250 2024-09-21T07:47:01.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.672+0000 7fad67200640 1 -- 172.21.15.39:0/1783612649 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad600adba0 con 0x7fad60160250 2024-09-21T07:47:01.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.673+0000 7fad67200640 1 -- 172.21.15.39:0/1783612649 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad6015d570 con 0x7fad60160250 2024-09-21T07:47:01.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.674+0000 7fad5a000640 1 -- 172.21.15.39:0/1783612649 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fad5001a070 con 0x7fad60160250 2024-09-21T07:47:01.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.675+0000 7fad5a000640 1 --2- 172.21.15.39:0/1783612649 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fad44078b90 0x7fad4407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:01.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.676+0000 7fad5a000640 1 -- 172.21.15.39:0/1783612649 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 6166+0+0 (secure 0 0 0) 0x7fad5009a430 con 0x7fad60160250 2024-09-21T07:47:01.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.676+0000 7fad65e00640 1 --2- 172.21.15.39:0/1783612649 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fad44078b90 0x7fad4407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:01.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.677+0000 7fad65e00640 1 --2- 172.21.15.39:0/1783612649 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fad44078b90 0x7fad4407b0a0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fad54034000 tx=0x7fad54033040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:01.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.680+0000 7fad5a000640 1 -- 172.21.15.39:0/1783612649 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fad50062580 con 0x7fad60160250 2024-09-21T07:47:01.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.874+0000 7fad67200640 1 -- 172.21.15.39:0/1783612649 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) -- 0x7fad60073190 con 0x7fad60160250 2024-09-21T07:47:01.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.874+0000 7fad5a000640 1 -- 172.21.15.39:0/1783612649 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v186) ==== 107+0+11 (secure 0 0 0) 0x7fad50061e80 con 0x7fad60160250 2024-09-21T07:47:01.875 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-21T07:47:01.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.877+0000 7fad67200640 1 -- 172.21.15.39:0/1783612649 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fad44078b90 msgr2=0x7fad4407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:01.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.877+0000 7fad67200640 1 --2- 172.21.15.39:0/1783612649 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fad44078b90 0x7fad4407b0a0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fad54034000 tx=0x7fad54033040 comp rx=0 tx=0).stop 2024-09-21T07:47:01.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.877+0000 7fad67200640 1 -- 172.21.15.39:0/1783612649 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad60160250 msgr2=0x7fad601f90c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:01.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.877+0000 7fad67200640 1 --2- 172.21.15.39:0/1783612649 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad60160250 0x7fad601f90c0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fad5000c430 tx=0x7fad5000c8f0 comp rx=0 tx=0).stop 2024-09-21T07:47:01.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.877+0000 7fad67200640 1 -- 172.21.15.39:0/1783612649 shutdown_connections 2024-09-21T07:47:01.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.877+0000 7fad67200640 1 --2- 172.21.15.39:0/1783612649 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fad44078b90 0x7fad4407b0a0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:01.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.877+0000 7fad67200640 1 --2- 172.21.15.39:0/1783612649 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fad60160660 0x7fad6013b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:01.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.877+0000 7fad67200640 1 --2- 172.21.15.39:0/1783612649 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad60160250 0x7fad601f90c0 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:01.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.877+0000 7fad67200640 1 --2- 172.21.15.39:0/1783612649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad600d0390 0x7fad601f9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:01.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.877+0000 7fad67200640 1 -- 172.21.15.39:0/1783612649 >> 172.21.15.39:0/1783612649 conn(0x7fad6006cfc0 msgr2=0x7fad6015ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:01.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.877+0000 7fad67200640 1 -- 172.21.15.39:0/1783612649 shutdown_connections 2024-09-21T07:47:01.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:01.877+0000 7fad67200640 1 -- 172.21.15.39:0/1783612649 wait complete. 2024-09-21T07:47:01.899 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_multimds_hardlink' 2024-09-21T07:47:02.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.124+0000 7f959d800640 1 -- 172.21.15.39:0/2496662995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95980b5240 msgr2=0x7f959815c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:02.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.124+0000 7f959d800640 1 --2- 172.21.15.39:0/2496662995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95980b5240 0x7f959815c690 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f95980b5a10 tx=0x7f958802a540 comp rx=0 tx=0).stop 2024-09-21T07:47:02.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.125+0000 7f959d800640 1 -- 172.21.15.39:0/2496662995 shutdown_connections 2024-09-21T07:47:02.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.125+0000 7f959d800640 1 --2- 172.21.15.39:0/2496662995 >> v2:172.21.15.148:3300/0 conn(0x7f95980cfd30 0x7f95980d0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:02.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.125+0000 7f959d800640 1 --2- 172.21.15.39:0/2496662995 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95980cf920 0x7f95980d0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:02.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.125+0000 7f959d800640 1 --2- 172.21.15.39:0/2496662995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95980b5240 0x7f959815c690 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:02.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.125+0000 7f959d800640 1 -- 172.21.15.39:0/2496662995 >> 172.21.15.39:0/2496662995 conn(0x7f959806cfc0 msgr2=0x7f959806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:02.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.125+0000 7f959d800640 1 -- 172.21.15.39:0/2496662995 shutdown_connections 2024-09-21T07:47:02.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.125+0000 7f959d800640 1 -- 172.21.15.39:0/2496662995 wait complete. 2024-09-21T07:47:02.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.127+0000 7f959d800640 1 Processor -- start 2024-09-21T07:47:02.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.127+0000 7f959d800640 1 -- start start 2024-09-21T07:47:02.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.128+0000 7f959d800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95980b5240 0x7f95981f6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:02.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.128+0000 7f959d800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95980cf920 0x7f95981f6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:02.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.128+0000 7f959d800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95980cfd30 0x7f95981f7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:02.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.129+0000 7f959d800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f95980b4760 con 0x7f95980b5240 2024-09-21T07:47:02.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.129+0000 7f959d800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f95980b45e0 con 0x7f95980cf920 2024-09-21T07:47:02.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.129+0000 7f959d800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f95980b48e0 con 0x7f95980cfd30 2024-09-21T07:47:02.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.129+0000 7f9596a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95980b5240 0x7f95981f6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:02.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.129+0000 7f9596a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95980b5240 0x7f95981f6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59930/0 (socket says 172.21.15.39:59930) 2024-09-21T07:47:02.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.129+0000 7f9596a00640 1 -- 172.21.15.39:0/4007143920 learned_addr learned my addr 172.21.15.39:0/4007143920 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:02.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.129+0000 7f9596a00640 1 -- 172.21.15.39:0/4007143920 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95980cfd30 msgr2=0x7f95981f7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:02.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.129+0000 7f9597400640 1 --2- 172.21.15.39:0/4007143920 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95980cfd30 0x7f95981f7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:02.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.129+0000 7f9596000640 1 --2- 172.21.15.39:0/4007143920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95980cf920 0x7f95981f6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:02.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.129+0000 7f9596a00640 1 --2- 172.21.15.39:0/4007143920 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95980cfd30 0x7f95981f7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:02.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.129+0000 7f9596a00640 1 -- 172.21.15.39:0/4007143920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95980cf920 msgr2=0x7f95981f6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:02.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.129+0000 7f9596a00640 1 --2- 172.21.15.39:0/4007143920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95980cf920 0x7f95981f6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:02.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.129+0000 7f9596a00640 1 -- 172.21.15.39:0/4007143920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9598155950 con 0x7f95980b5240 2024-09-21T07:47:02.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.129+0000 7f9597400640 1 --2- 172.21.15.39:0/4007143920 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95980cfd30 0x7f95981f7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:47:02.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.129+0000 7f9596000640 1 --2- 172.21.15.39:0/4007143920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95980cf920 0x7f95981f6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:47:02.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.130+0000 7f9596a00640 1 --2- 172.21.15.39:0/4007143920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95980b5240 0x7f95981f6430 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f9588000c00 tx=0x7f9588003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:02.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.130+0000 7f958ea00640 1 -- 172.21.15.39:0/4007143920 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958803e030 con 0x7f95980b5240 2024-09-21T07:47:02.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.130+0000 7f958ea00640 1 -- 172.21.15.39:0/4007143920 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9588003e00 con 0x7f95980b5240 2024-09-21T07:47:02.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.130+0000 7f958ea00640 1 -- 172.21.15.39:0/4007143920 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9588035040 con 0x7f95980b5240 2024-09-21T07:47:02.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.130+0000 7f959d800640 1 -- 172.21.15.39:0/4007143920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9598152e20 con 0x7f95980b5240 2024-09-21T07:47:02.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.130+0000 7f959d800640 1 -- 172.21.15.39:0/4007143920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95981f7650 con 0x7f95980b5240 2024-09-21T07:47:02.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.131+0000 7f959d800640 1 -- 172.21.15.39:0/4007143920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95980cf430 con 0x7f95980b5240 2024-09-21T07:47:02.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.132+0000 7f958ea00640 1 -- 172.21.15.39:0/4007143920 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9588041030 con 0x7f95980b5240 2024-09-21T07:47:02.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.133+0000 7f958ea00640 1 --2- 172.21.15.39:0/4007143920 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9578078bf0 0x7f957807b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:02.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.133+0000 7f958ea00640 1 -- 172.21.15.39:0/4007143920 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 6166+0+0 (secure 0 0 0) 0x7f95880bb780 con 0x7f95980b5240 2024-09-21T07:47:02.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.134+0000 7f9596000640 1 --2- 172.21.15.39:0/4007143920 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9578078bf0 0x7f957807b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:02.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.135+0000 7f9596000640 1 --2- 172.21.15.39:0/4007143920 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9578078bf0 0x7f957807b100 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f959814d330 tx=0x7f9584005dd0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:02.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.138+0000 7f958ea00640 1 -- 172.21.15.39:0/4007143920 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f95980cf430 con 0x7f95980b5240 2024-09-21T07:47:02.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.411+0000 7f959d800640 1 -- 172.21.15.39:0/4007143920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_multimds_hardlink"]} v 0) -- 0x7f95980b4e00 con 0x7f95980b5240 2024-09-21T07:47:02.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.549+0000 7f958ea00640 1 -- 172.21.15.39:0/4007143920 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_multimds_hardlink"]}]=0 v672) ==== 146+0+0 (secure 0 0 0) 0x7f9588083950 con 0x7f95980b5240 2024-09-21T07:47:02.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.551+0000 7f959d800640 1 -- 172.21.15.39:0/4007143920 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9578078bf0 msgr2=0x7f957807b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:02.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.551+0000 7f959d800640 1 --2- 172.21.15.39:0/4007143920 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9578078bf0 0x7f957807b100 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f959814d330 tx=0x7f9584005dd0 comp rx=0 tx=0).stop 2024-09-21T07:47:02.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.551+0000 7f959d800640 1 -- 172.21.15.39:0/4007143920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95980b5240 msgr2=0x7f95981f6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:02.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.551+0000 7f959d800640 1 --2- 172.21.15.39:0/4007143920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95980b5240 0x7f95981f6430 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f9588000c00 tx=0x7f9588003910 comp rx=0 tx=0).stop 2024-09-21T07:47:02.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.551+0000 7f959d800640 1 -- 172.21.15.39:0/4007143920 shutdown_connections 2024-09-21T07:47:02.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.551+0000 7f959d800640 1 --2- 172.21.15.39:0/4007143920 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9578078bf0 0x7f957807b100 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:02.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.551+0000 7f959d800640 1 --2- 172.21.15.39:0/4007143920 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f95980cfd30 0x7f95981f7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:02.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.551+0000 7f959d800640 1 --2- 172.21.15.39:0/4007143920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f95980cf920 0x7f95981f6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:02.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.551+0000 7f959d800640 1 --2- 172.21.15.39:0/4007143920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95980b5240 0x7f95981f6430 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:02.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.551+0000 7f959d800640 1 -- 172.21.15.39:0/4007143920 >> 172.21.15.39:0/4007143920 conn(0x7f959806cfc0 msgr2=0x7f959815ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:02.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.552+0000 7f959d800640 1 -- 172.21.15.39:0/4007143920 shutdown_connections 2024-09-21T07:47:02.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.552+0000 7f959d800640 1 -- 172.21.15.39:0/4007143920 wait complete. 2024-09-21T07:47:02.575 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T07:47:02.659 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:47:02.663 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:47:02.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.802+0000 7f2089a00640 1 -- 172.21.15.39:0/1941256363 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20841519d0 msgr2=0x7f2084073f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:02.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.802+0000 7f2089a00640 1 --2- 172.21.15.39:0/1941256363 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20841519d0 0x7f2084073f30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2074005fd0 tx=0x7f207402a5d0 comp rx=0 tx=0).stop 2024-09-21T07:47:02.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.802+0000 7f2089a00640 1 -- 172.21.15.39:0/1941256363 shutdown_connections 2024-09-21T07:47:02.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.802+0000 7f2089a00640 1 --2- 172.21.15.39:0/1941256363 >> v2:172.21.15.148:3300/0 conn(0x7f208406ee90 0x7f208406f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:02.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.802+0000 7f2089a00640 1 --2- 172.21.15.39:0/1941256363 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20841519d0 0x7f2084073f30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:02.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.802+0000 7f2089a00640 1 -- 172.21.15.39:0/1941256363 >> 172.21.15.39:0/1941256363 conn(0x7f208406cfc0 msgr2=0x7f208406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:02.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.802+0000 7f2089a00640 1 -- 172.21.15.39:0/1941256363 shutdown_connections 2024-09-21T07:47:02.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.802+0000 7f2089a00640 1 -- 172.21.15.39:0/1941256363 wait complete. 2024-09-21T07:47:02.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.804+0000 7f2089a00640 1 Processor -- start 2024-09-21T07:47:02.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.805+0000 7f2089a00640 1 -- start start 2024-09-21T07:47:02.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.806+0000 7f2089a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f208406ee90 0x7f20841382d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:02.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.806+0000 7f2089a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20840d04a0 0x7f20841f3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:02.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.806+0000 7f2089a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20841519d0 0x7f2084154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:02.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.806+0000 7f2089a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f20840b3ac0 con 0x7f208406ee90 2024-09-21T07:47:02.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.806+0000 7f2089a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f20840b3940 con 0x7f20840d04a0 2024-09-21T07:47:02.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.806+0000 7f2089a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f20840b3c40 con 0x7f20841519d0 2024-09-21T07:47:02.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.806+0000 7f2083e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f208406ee90 0x7f20841382d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:02.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.806+0000 7f2083e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f208406ee90 0x7f20841382d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59932/0 (socket says 172.21.15.39:59932) 2024-09-21T07:47:02.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.806+0000 7f2083e00640 1 -- 172.21.15.39:0/154017878 learned_addr learned my addr 172.21.15.39:0/154017878 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:02.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.807+0000 7f207b400640 1 --2- 172.21.15.39:0/154017878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20840d04a0 0x7f20841f3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:02.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.807+0000 7f2089000640 1 --2- 172.21.15.39:0/154017878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20841519d0 0x7f2084154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:02.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.807+0000 7f2083e00640 1 -- 172.21.15.39:0/154017878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20841519d0 msgr2=0x7f2084154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:02.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.807+0000 7f2083e00640 1 --2- 172.21.15.39:0/154017878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20841519d0 0x7f2084154050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:02.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.807+0000 7f2083e00640 1 -- 172.21.15.39:0/154017878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20840d04a0 msgr2=0x7f20841f3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:02.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.807+0000 7f2083e00640 1 --2- 172.21.15.39:0/154017878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20840d04a0 0x7f20841f3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:02.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.807+0000 7f2083e00640 1 -- 172.21.15.39:0/154017878 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2084138810 con 0x7f208406ee90 2024-09-21T07:47:02.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.807+0000 7f207b400640 1 --2- 172.21.15.39:0/154017878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20840d04a0 0x7f20841f3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:02.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.807+0000 7f2083e00640 1 --2- 172.21.15.39:0/154017878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f208406ee90 0x7f20841382d0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f2074034040 tx=0x7f20740035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:02.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.808+0000 7f2080c00640 1 -- 172.21.15.39:0/154017878 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207403e030 con 0x7f208406ee90 2024-09-21T07:47:02.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.808+0000 7f2089a00640 1 -- 172.21.15.39:0/154017878 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20841f41e0 con 0x7f208406ee90 2024-09-21T07:47:02.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.808+0000 7f2080c00640 1 -- 172.21.15.39:0/154017878 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2074003ae0 con 0x7f208406ee90 2024-09-21T07:47:02.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.808+0000 7f2080c00640 1 -- 172.21.15.39:0/154017878 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2074035040 con 0x7f208406ee90 2024-09-21T07:47:02.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.808+0000 7f2089a00640 1 -- 172.21.15.39:0/154017878 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20841f6f70 con 0x7f208406ee90 2024-09-21T07:47:02.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.809+0000 7f2089a00640 1 -- 172.21.15.39:0/154017878 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20841f6a90 con 0x7f208406ee90 2024-09-21T07:47:02.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.810+0000 7f2080c00640 1 -- 172.21.15.39:0/154017878 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2074041030 con 0x7f208406ee90 2024-09-21T07:47:02.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.811+0000 7f2080c00640 1 --2- 172.21.15.39:0/154017878 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2060078b90 0x7f206007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:02.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.811+0000 7f2080c00640 1 -- 172.21.15.39:0/154017878 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 6166+0+0 (secure 0 0 0) 0x7f20740bb9c0 con 0x7f208406ee90 2024-09-21T07:47:02.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.812+0000 7f207b400640 1 --2- 172.21.15.39:0/154017878 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2060078b90 0x7f206007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:02.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.812+0000 7f207b400640 1 --2- 172.21.15.39:0/154017878 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2060078b90 0x7f206007b0a0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f208413f250 tx=0x7f206c009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:02.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:02.815+0000 7f2080c00640 1 -- 172.21.15.39:0/154017878 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2074083b10 con 0x7f208406ee90 2024-09-21T07:47:03.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.014+0000 7f2089a00640 1 -- 172.21.15.39:0/154017878 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f208406f2a0 con 0x7f208406ee90 2024-09-21T07:47:03.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.015+0000 7f2080c00640 1 -- 172.21.15.39:0/154017878 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v48) ==== 59+0+2294 (secure 0 0 0) 0x7f2074083410 con 0x7f208406ee90 2024-09-21T07:47:03.017 INFO:teuthology.orchestra.run.smithi039.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T07:47:03.017 INFO:teuthology.orchestra.run.smithi039.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T07:47:03.017 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced client_mount_timeout 30 2024-09-21T07:47:03.017 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_client 20/20 2024-09-21T07:47:03.017 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_mgr 20/20 2024-09-21T07:47:03.017 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_ms 1/1 2024-09-21T07:47:03.018 INFO:teuthology.orchestra.run.smithi039.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T07:47:03.018 INFO:teuthology.orchestra.run.smithi039.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T07:47:03.018 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3421.881759 2024-09-21T07:47:03.018 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3520.252978 2024-09-21T07:47:03.018 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3511.183316 2024-09-21T07:47:03.018 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3829.631847 2024-09-21T07:47:03.018 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 5474.545872 2024-09-21T07:47:03.018 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4013.628559 2024-09-21T07:47:03.018 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4230.187288 2024-09-21T07:47:03.018 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4148.983107 2024-09-21T07:47:03.018 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_mds 20/20 2024-09-21T07:47:03.018 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T07:47:03.018 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_ms 2/2 2024-09-21T07:47:03.018 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_debug_frag true 2024-09-21T07:47:03.019 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_debug_scatterstat true 2024-09-21T07:47:03.019 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T07:47:03.019 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_verify_scatter true 2024-09-21T07:47:03.019 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T07:47:03.019 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T07:47:03.019 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T07:47:03.019 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced client_mount_timeout 600 2024-09-21T07:47:03.019 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced debug_client 20/20 2024-09-21T07:47:03.019 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced debug_ms 1/1 2024-09-21T07:47:03.019 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T07:47:03.019 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T07:47:03.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.018+0000 7f2089a00640 1 -- 172.21.15.39:0/154017878 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2060078b90 msgr2=0x7f206007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:03.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.018+0000 7f2089a00640 1 --2- 172.21.15.39:0/154017878 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2060078b90 0x7f206007b0a0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f208413f250 tx=0x7f206c009040 comp rx=0 tx=0).stop 2024-09-21T07:47:03.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.018+0000 7f2089a00640 1 -- 172.21.15.39:0/154017878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f208406ee90 msgr2=0x7f20841382d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:03.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.018+0000 7f2089a00640 1 --2- 172.21.15.39:0/154017878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f208406ee90 0x7f20841382d0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f2074034040 tx=0x7f20740035f0 comp rx=0 tx=0).stop 2024-09-21T07:47:03.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.018+0000 7f2089a00640 1 -- 172.21.15.39:0/154017878 shutdown_connections 2024-09-21T07:47:03.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.018+0000 7f2089a00640 1 --2- 172.21.15.39:0/154017878 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2060078b90 0x7f206007b0a0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:03.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.018+0000 7f2089a00640 1 --2- 172.21.15.39:0/154017878 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20841519d0 0x7f2084154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:03.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.018+0000 7f2089a00640 1 --2- 172.21.15.39:0/154017878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20840d04a0 0x7f20841f3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:03.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.018+0000 7f2089a00640 1 --2- 172.21.15.39:0/154017878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f208406ee90 0x7f20841382d0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:03.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.018+0000 7f2089a00640 1 -- 172.21.15.39:0/154017878 >> 172.21.15.39:0/154017878 conn(0x7f208406cfc0 msgr2=0x7f20840b16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:03.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.018+0000 7f2089a00640 1 -- 172.21.15.39:0/154017878 shutdown_connections 2024-09-21T07:47:03.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.018+0000 7f2089a00640 1 -- 172.21.15.39:0/154017878 wait complete. 2024-09-21T07:47:03.043 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T07:47:03.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.271+0000 7f967a200640 1 -- 172.21.15.39:0/1073819876 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f96740d1b70 msgr2=0x7f96740d1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:03.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.271+0000 7f967a200640 1 --2- 172.21.15.39:0/1073819876 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f96740d1b70 0x7f96740d1f80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f965c009890 tx=0x7f965c02a440 comp rx=0 tx=0).stop 2024-09-21T07:47:03.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.271+0000 7f967a200640 1 -- 172.21.15.39:0/1073819876 shutdown_connections 2024-09-21T07:47:03.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.271+0000 7f967a200640 1 --2- 172.21.15.39:0/1073819876 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f96740d1b70 0x7f96740d1f80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:03.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.271+0000 7f967a200640 1 -- 172.21.15.39:0/1073819876 >> 172.21.15.39:0/1073819876 conn(0x7f96740cf000 msgr2=0x7f96740cf410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:03.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.271+0000 7f967a200640 1 -- 172.21.15.39:0/1073819876 shutdown_connections 2024-09-21T07:47:03.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.272+0000 7f967a200640 1 -- 172.21.15.39:0/1073819876 wait complete. 2024-09-21T07:47:03.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.274+0000 7f967a200640 1 Processor -- start 2024-09-21T07:47:03.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.274+0000 7f967a200640 1 -- start start 2024-09-21T07:47:03.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.275+0000 7f967a200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96740b5ae0 0x7f9674157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:03.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.275+0000 7f967a200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96740b7500 0x7f967415e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:03.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.275+0000 7f967a200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f96740d1b70 0x7f967415eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:03.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.275+0000 7f967a200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f967416a280 con 0x7f96740b5ae0 2024-09-21T07:47:03.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.275+0000 7f967a200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f96740cff40 con 0x7f96740b7500 2024-09-21T07:47:03.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.275+0000 7f967a200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f967416a400 con 0x7f96740d1b70 2024-09-21T07:47:03.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.275+0000 7f9673400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96740b5ae0 0x7f9674157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:03.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.275+0000 7f9673400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96740b5ae0 0x7f9674157020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59940/0 (socket says 172.21.15.39:59940) 2024-09-21T07:47:03.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.276+0000 7f9673400640 1 -- 172.21.15.39:0/4022666335 learned_addr learned my addr 172.21.15.39:0/4022666335 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:03.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.276+0000 7f966aa00640 1 --2- 172.21.15.39:0/4022666335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96740b7500 0x7f967415e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:03.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.276+0000 7f9673e00640 1 --2- 172.21.15.39:0/4022666335 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f96740d1b70 0x7f967415eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:03.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.276+0000 7f9673400640 1 -- 172.21.15.39:0/4022666335 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f96740d1b70 msgr2=0x7f967415eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:03.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.276+0000 7f9673400640 1 --2- 172.21.15.39:0/4022666335 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f96740d1b70 0x7f967415eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:03.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.276+0000 7f9673400640 1 -- 172.21.15.39:0/4022666335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96740b7500 msgr2=0x7f967415e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:03.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.276+0000 7f9673400640 1 --2- 172.21.15.39:0/4022666335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96740b7500 0x7f967415e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:03.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.276+0000 7f9673400640 1 -- 172.21.15.39:0/4022666335 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9674158cf0 con 0x7f96740b5ae0 2024-09-21T07:47:03.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.276+0000 7f9673e00640 1 --2- 172.21.15.39:0/4022666335 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f96740d1b70 0x7f967415eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:47:03.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.276+0000 7f9673400640 1 --2- 172.21.15.39:0/4022666335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96740b5ae0 0x7f9674157020 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f966400c3c0 tx=0x7f966400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:03.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.276+0000 7f966be00640 1 -- 172.21.15.39:0/4022666335 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9664017070 con 0x7f96740b5ae0 2024-09-21T07:47:03.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.276+0000 7f966be00640 1 -- 172.21.15.39:0/4022666335 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96640053c0 con 0x7f96740b5ae0 2024-09-21T07:47:03.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.276+0000 7f966be00640 1 -- 172.21.15.39:0/4022666335 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9664003850 con 0x7f96740b5ae0 2024-09-21T07:47:03.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.277+0000 7f967a200640 1 -- 172.21.15.39:0/4022666335 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9674169b50 con 0x7f96740b5ae0 2024-09-21T07:47:03.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.277+0000 7f967a200640 1 -- 172.21.15.39:0/4022666335 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9674169e30 con 0x7f96740b5ae0 2024-09-21T07:47:03.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.278+0000 7f967a200640 1 -- 172.21.15.39:0/4022666335 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f967406a3e0 con 0x7f96740b5ae0 2024-09-21T07:47:03.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.279+0000 7f966be00640 1 -- 172.21.15.39:0/4022666335 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f966401a070 con 0x7f96740b5ae0 2024-09-21T07:47:03.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.280+0000 7f966be00640 1 --2- 172.21.15.39:0/4022666335 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9650078b90 0x7f965007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:03.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.280+0000 7f966be00640 1 -- 172.21.15.39:0/4022666335 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 6166+0+0 (secure 0 0 0) 0x7f966409a320 con 0x7f96740b5ae0 2024-09-21T07:47:03.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.280+0000 7f966aa00640 1 --2- 172.21.15.39:0/4022666335 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9650078b90 0x7f965007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:03.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.281+0000 7f966aa00640 1 --2- 172.21.15.39:0/4022666335 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9650078b90 0x7f965007b0a0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f965c034000 tx=0x7f965c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:03.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.284+0000 7f966be00640 1 -- 172.21.15.39:0/4022666335 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f967406a3e0 con 0x7f96740b5ae0 2024-09-21T07:47:03.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.482+0000 7f967a200640 1 -- 172.21.15.39:0/4022666335 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f96740d1f80 con 0x7f96740b5ae0 2024-09-21T07:47:03.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.487+0000 7f966be00640 1 -- 172.21.15.39:0/4022666335 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v49)=0 v49) ==== 122+0+0 (secure 0 0 0) 0x7f9664062500 con 0x7f96740b5ae0 2024-09-21T07:47:03.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.490+0000 7f967a200640 1 -- 172.21.15.39:0/4022666335 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9650078b90 msgr2=0x7f965007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:03.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.490+0000 7f967a200640 1 --2- 172.21.15.39:0/4022666335 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9650078b90 0x7f965007b0a0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f965c034000 tx=0x7f965c004730 comp rx=0 tx=0).stop 2024-09-21T07:47:03.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.490+0000 7f967a200640 1 -- 172.21.15.39:0/4022666335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96740b5ae0 msgr2=0x7f9674157020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:03.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.490+0000 7f967a200640 1 --2- 172.21.15.39:0/4022666335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96740b5ae0 0x7f9674157020 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f966400c3c0 tx=0x7f966400c880 comp rx=0 tx=0).stop 2024-09-21T07:47:03.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.490+0000 7f967a200640 1 -- 172.21.15.39:0/4022666335 shutdown_connections 2024-09-21T07:47:03.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.490+0000 7f967a200640 1 --2- 172.21.15.39:0/4022666335 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9650078b90 0x7f965007b0a0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:03.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.490+0000 7f967a200640 1 --2- 172.21.15.39:0/4022666335 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f96740d1b70 0x7f967415eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:03.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.490+0000 7f967a200640 1 --2- 172.21.15.39:0/4022666335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96740b7500 0x7f967415e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:03.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.490+0000 7f967a200640 1 --2- 172.21.15.39:0/4022666335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96740b5ae0 0x7f9674157020 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:03.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.490+0000 7f967a200640 1 -- 172.21.15.39:0/4022666335 >> 172.21.15.39:0/4022666335 conn(0x7f96740cf000 msgr2=0x7f9674165ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:03.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.490+0000 7f967a200640 1 -- 172.21.15.39:0/4022666335 shutdown_connections 2024-09-21T07:47:03.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:03.490+0000 7f967a200640 1 -- 172.21.15.39:0/4022666335 wait complete. 2024-09-21T07:47:03.515 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:47:03.516 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:47:03.609 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:47:03.713 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:47:03.714 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:47:03.714 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:47:03.749 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:47:03.749 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:47:03.750 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:47:03.787 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:47:03.787 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:47:03.787 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:47:03.824 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:47:03.825 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:47:03.825 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:47:03.863 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:47:03.863 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T07:47:03.863 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:47:04.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.089+0000 7fe2f8c00640 1 -- 172.21.15.39:0/2408143726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2f40b30a0 msgr2=0x7fe2f40cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:04.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.089+0000 7fe2f8c00640 1 --2- 172.21.15.39:0/2408143726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2f40b30a0 0x7fe2f40cb8a0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4075160 tx=0x7fe2e402a4d0 comp rx=0 tx=0).stop 2024-09-21T07:47:04.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.089+0000 7fe2f8c00640 1 -- 172.21.15.39:0/2408143726 shutdown_connections 2024-09-21T07:47:04.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.089+0000 7fe2f8c00640 1 --2- 172.21.15.39:0/2408143726 >> v2:172.21.15.148:3300/0 conn(0x7fe2f415d830 0x7fe2f4156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:04.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.089+0000 7fe2f8c00640 1 --2- 172.21.15.39:0/2408143726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2f40b30a0 0x7fe2f40cb8a0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:04.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.089+0000 7fe2f8c00640 1 -- 172.21.15.39:0/2408143726 >> 172.21.15.39:0/2408143726 conn(0x7fe2f40b2460 msgr2=0x7fe2f406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:04.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.090+0000 7fe2f8c00640 1 -- 172.21.15.39:0/2408143726 shutdown_connections 2024-09-21T07:47:04.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.090+0000 7fe2f8c00640 1 -- 172.21.15.39:0/2408143726 wait complete. 2024-09-21T07:47:04.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.092+0000 7fe2f8c00640 1 Processor -- start 2024-09-21T07:47:04.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.092+0000 7fe2f8c00640 1 -- start start 2024-09-21T07:47:04.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.093+0000 7fe2f8c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2f40b30a0 0x7fe2f40c3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:04.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.093+0000 7fe2f8c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2f40ccc60 0x7fe2f41f5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:04.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.093+0000 7fe2f8c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe2f415d830 0x7fe2f4074500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:04.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.093+0000 7fe2f8c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f40af410 con 0x7fe2f40ccc60 2024-09-21T07:47:04.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.093+0000 7fe2f8c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f40af290 con 0x7fe2f40b30a0 2024-09-21T07:47:04.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.093+0000 7fe2f8c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe2f40af590 con 0x7fe2f415d830 2024-09-21T07:47:04.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.094+0000 7fe2f1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2f40ccc60 0x7fe2f41f5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:04.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.094+0000 7fe2f1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2f40ccc60 0x7fe2f41f5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59972/0 (socket says 172.21.15.39:59972) 2024-09-21T07:47:04.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.094+0000 7fe2f1400640 1 -- 172.21.15.39:0/1575665586 learned_addr learned my addr 172.21.15.39:0/1575665586 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:04.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.094+0000 7fe2f1e00640 1 --2- 172.21.15.39:0/1575665586 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2f40b30a0 0x7fe2f40c3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:04.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.094+0000 7fe2f1400640 1 -- 172.21.15.39:0/1575665586 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe2f415d830 msgr2=0x7fe2f4074500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:04.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.094+0000 7fe2f2800640 1 --2- 172.21.15.39:0/1575665586 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe2f415d830 0x7fe2f4074500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:04.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.094+0000 7fe2f1400640 1 --2- 172.21.15.39:0/1575665586 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe2f415d830 0x7fe2f4074500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:04.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.094+0000 7fe2f1400640 1 -- 172.21.15.39:0/1575665586 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2f40b30a0 msgr2=0x7fe2f40c3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:04.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.094+0000 7fe2f1400640 1 --2- 172.21.15.39:0/1575665586 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2f40b30a0 0x7fe2f40c3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:04.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.094+0000 7fe2f1400640 1 -- 172.21.15.39:0/1575665586 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f40c4de0 con 0x7fe2f40ccc60 2024-09-21T07:47:04.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.094+0000 7fe2f2800640 1 --2- 172.21.15.39:0/1575665586 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe2f415d830 0x7fe2f4074500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:47:04.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.095+0000 7fe2f1400640 1 --2- 172.21.15.39:0/1575665586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2f40ccc60 0x7fe2f41f5570 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fe2e000b440 tx=0x7fe2e000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:04.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.095+0000 7fe2ea000640 1 -- 172.21.15.39:0/1575665586 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0015020 con 0x7fe2f40ccc60 2024-09-21T07:47:04.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.095+0000 7fe2ea000640 1 -- 172.21.15.39:0/1575665586 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe2e0011980 con 0x7fe2f40ccc60 2024-09-21T07:47:04.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.095+0000 7fe2f8c00640 1 -- 172.21.15.39:0/1575665586 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2f41f9400 con 0x7fe2f40ccc60 2024-09-21T07:47:04.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.095+0000 7fe2ea000640 1 -- 172.21.15.39:0/1575665586 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0011dd0 con 0x7fe2f40ccc60 2024-09-21T07:47:04.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.095+0000 7fe2f8c00640 1 -- 172.21.15.39:0/1575665586 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2f41f99c0 con 0x7fe2f40ccc60 2024-09-21T07:47:04.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.096+0000 7fe2f8c00640 1 -- 172.21.15.39:0/1575665586 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2f40ad7e0 con 0x7fe2f40ccc60 2024-09-21T07:47:04.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.097+0000 7fe2ea000640 1 -- 172.21.15.39:0/1575665586 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe2e0018020 con 0x7fe2f40ccc60 2024-09-21T07:47:04.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.099+0000 7fe2ea000640 1 --2- 172.21.15.39:0/1575665586 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe2d4078bc0 0x7fe2d407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:04.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.099+0000 7fe2ea000640 1 -- 172.21.15.39:0/1575665586 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 6166+0+0 (secure 0 0 0) 0x7fe2e009aff0 con 0x7fe2f40ccc60 2024-09-21T07:47:04.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.099+0000 7fe2f1e00640 1 --2- 172.21.15.39:0/1575665586 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe2d4078bc0 0x7fe2d407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:04.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.100+0000 7fe2f1e00640 1 --2- 172.21.15.39:0/1575665586 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe2d4078bc0 0x7fe2d407b0d0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4033000 tx=0x7fe2e4004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:04.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.103+0000 7fe2ea000640 1 -- 172.21.15.39:0/1575665586 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe2e001f020 con 0x7fe2f40ccc60 2024-09-21T07:47:04.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.339+0000 7fe2f8c00640 1 -- 172.21.15.39:0/1575665586 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2f406df10 con 0x7fe2f40ccc60 2024-09-21T07:47:04.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.340+0000 7fe2ea000640 1 -- 172.21.15.39:0/1575665586 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 192 v192) ==== 95+0+3919 (secure 0 0 0) 0x7fe2e000d040 con 0x7fe2f40ccc60 2024-09-21T07:47:04.342 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:47:04.342 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":192,"btime":"2024-09-21T07:46:55:246427+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7493,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/4129342618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4129342618},{"type":"v1","addr":"172.21.15.39:6834","nonce":4129342618}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":185},{"gid":7499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1125302960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1125302960},{"type":"v1","addr":"172.21.15.148:6836","nonce":1125302960}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":185},{"gid":8431,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/585545787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":585545787},{"type":"v1","addr":"172.21.15.148:6837","nonce":585545787}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":185},{"gid":9378,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2296137283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2296137283},{"type":"v1","addr":"172.21.15.39:6835","nonce":2296137283}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":192}],"filesystems":[]} 2024-09-21T07:47:04.343 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 192 2024-09-21T07:47:04.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.342+0000 7fe2f8c00640 1 -- 172.21.15.39:0/1575665586 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe2d4078bc0 msgr2=0x7fe2d407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:04.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.342+0000 7fe2f8c00640 1 --2- 172.21.15.39:0/1575665586 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe2d4078bc0 0x7fe2d407b0d0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4033000 tx=0x7fe2e4004730 comp rx=0 tx=0).stop 2024-09-21T07:47:04.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.343+0000 7fe2f8c00640 1 -- 172.21.15.39:0/1575665586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2f40ccc60 msgr2=0x7fe2f41f5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:04.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.343+0000 7fe2f8c00640 1 --2- 172.21.15.39:0/1575665586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2f40ccc60 0x7fe2f41f5570 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fe2e000b440 tx=0x7fe2e000b900 comp rx=0 tx=0).stop 2024-09-21T07:47:04.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.343+0000 7fe2f8c00640 1 -- 172.21.15.39:0/1575665586 shutdown_connections 2024-09-21T07:47:04.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.343+0000 7fe2f8c00640 1 --2- 172.21.15.39:0/1575665586 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe2d4078bc0 0x7fe2d407b0d0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:04.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.343+0000 7fe2f8c00640 1 --2- 172.21.15.39:0/1575665586 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe2f415d830 0x7fe2f4074500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:04.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.343+0000 7fe2f8c00640 1 --2- 172.21.15.39:0/1575665586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2f40ccc60 0x7fe2f41f5570 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:04.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.343+0000 7fe2f8c00640 1 --2- 172.21.15.39:0/1575665586 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2f40b30a0 0x7fe2f40c3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:04.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.343+0000 7fe2f8c00640 1 -- 172.21.15.39:0/1575665586 >> 172.21.15.39:0/1575665586 conn(0x7fe2f40b2460 msgr2=0x7fe2f415e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:04.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.343+0000 7fe2f8c00640 1 -- 172.21.15.39:0/1575665586 shutdown_connections 2024-09-21T07:47:04.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.343+0000 7fe2f8c00640 1 -- 172.21.15.39:0/1575665586 wait complete. 2024-09-21T07:47:04.367 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T07:47:04.367 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T07:47:04.367 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T07:47:04.367 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:47:04.367 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T07:47:04.368 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T07:47:04.368 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T07:47:04.368 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:47:04.368 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T07:47:04.368 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T07:47:04.368 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T07:47:04.368 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:47:04.369 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T07:47:04.369 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T07:47:04.369 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T07:47:04.369 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:47:04.389 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T07:47:04.389 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T07:47:04.392 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T07:47:04.392 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T07:47:04.394 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T07:47:04.394 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T07:47:04.395 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T07:47:04.395 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T07:47:04.435 INFO:tasks.ceph.mds.a:Started 2024-09-21T07:47:04.436 DEBUG:teuthology.parallel:result is None 2024-09-21T07:47:04.438 INFO:tasks.ceph.mds.c:Started 2024-09-21T07:47:04.438 DEBUG:teuthology.parallel:result is None 2024-09-21T07:47:04.439 INFO:tasks.ceph.mds.b:Started 2024-09-21T07:47:04.439 DEBUG:teuthology.parallel:result is None 2024-09-21T07:47:04.441 INFO:tasks.ceph.mds.d:Started 2024-09-21T07:47:04.441 DEBUG:teuthology.parallel:result is None 2024-09-21T07:47:04.442 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T07:47:04.634 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-21T07:47:04.634 INFO:tasks.ceph.mds.d.smithi148.stdout:starting mds.d at 2024-09-21T07:47:04.634 INFO:tasks.ceph.mds.b.smithi148.stdout:starting mds.b at 2024-09-21T07:47:04.644 INFO:tasks.ceph.mds.c.smithi039.stdout:starting mds.c at 2024-09-21T07:47:04.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.677+0000 7f2eee200640 1 -- 172.21.15.39:0/1837811821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ee80b5a90 msgr2=0x7f2ee80d1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:04.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.677+0000 7f2eee200640 1 --2- 172.21.15.39:0/1837811821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ee80b5a90 0x7f2ee80d1ac0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2edc0098e0 tx=0x7f2edc02a5d0 comp rx=0 tx=0).stop 2024-09-21T07:47:04.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.677+0000 7f2eee200640 1 -- 172.21.15.39:0/1837811821 shutdown_connections 2024-09-21T07:47:04.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.677+0000 7f2eee200640 1 --2- 172.21.15.39:0/1837811821 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ee80d0e70 0x7f2ee80d1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:04.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.677+0000 7f2eee200640 1 --2- 172.21.15.39:0/1837811821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ee80b5a90 0x7f2ee80d1ac0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:04.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.677+0000 7f2eee200640 1 -- 172.21.15.39:0/1837811821 >> 172.21.15.39:0/1837811821 conn(0x7f2ee809d7b0 msgr2=0x7f2ee80d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:04.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.677+0000 7f2eee200640 1 -- 172.21.15.39:0/1837811821 shutdown_connections 2024-09-21T07:47:04.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.677+0000 7f2eee200640 1 -- 172.21.15.39:0/1837811821 wait complete. 2024-09-21T07:47:04.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.679+0000 7f2eee200640 1 Processor -- start 2024-09-21T07:47:04.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.679+0000 7f2eee200640 1 -- start start 2024-09-21T07:47:04.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.681+0000 7f2eee200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ee80b5a90 0x7f2ee81600d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:04.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.681+0000 7f2eee200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ee80cef70 0x7f2ee809e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:04.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.681+0000 7f2eee200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ee80d0e70 0x7f2ee80cd410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:04.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.681+0000 7f2eee200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee80b4b70 con 0x7f2ee80d0e70 2024-09-21T07:47:04.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.681+0000 7f2eee200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee80d2240 con 0x7f2ee80cef70 2024-09-21T07:47:04.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.681+0000 7f2eee200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f2ee80b4cf0 con 0x7f2ee80b5a90 2024-09-21T07:47:04.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.681+0000 7f2eed800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ee80d0e70 0x7f2ee80cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:04.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.681+0000 7f2eece00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ee80b5a90 0x7f2ee81600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:04.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.681+0000 7f2eed800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ee80d0e70 0x7f2ee80cd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60012/0 (socket says 172.21.15.39:60012) 2024-09-21T07:47:04.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.681+0000 7f2ee7e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ee80cef70 0x7f2ee809e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:04.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.681+0000 7f2eed800640 1 -- 172.21.15.39:0/1731872333 learned_addr learned my addr 172.21.15.39:0/1731872333 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:04.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.682+0000 7f2eed800640 1 -- 172.21.15.39:0/1731872333 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ee80b5a90 msgr2=0x7f2ee81600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:04.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.682+0000 7f2eed800640 1 --2- 172.21.15.39:0/1731872333 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ee80b5a90 0x7f2ee81600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:04.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.682+0000 7f2eed800640 1 -- 172.21.15.39:0/1731872333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ee80cef70 msgr2=0x7f2ee809e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:04.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.682+0000 7f2eed800640 1 --2- 172.21.15.39:0/1731872333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ee80cef70 0x7f2ee809e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:04.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.682+0000 7f2eed800640 1 -- 172.21.15.39:0/1731872333 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee81f62b0 con 0x7f2ee80d0e70 2024-09-21T07:47:04.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.682+0000 7f2ee7e00640 1 --2- 172.21.15.39:0/1731872333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ee80cef70 0x7f2ee809e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:47:04.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.682+0000 7f2eed800640 1 --2- 172.21.15.39:0/1731872333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ee80d0e70 0x7f2ee80cd410 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f2ee000c730 tx=0x7f2ee000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:04.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.682+0000 7f2ee4c00640 1 -- 172.21.15.39:0/1731872333 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0017070 con 0x7f2ee80d0e70 2024-09-21T07:47:04.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.682+0000 7f2ee4c00640 1 -- 172.21.15.39:0/1731872333 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ee0003c60 con 0x7f2ee80d0e70 2024-09-21T07:47:04.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.682+0000 7f2ee4c00640 1 -- 172.21.15.39:0/1731872333 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee00052d0 con 0x7f2ee80d0e70 2024-09-21T07:47:04.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.682+0000 7f2eee200640 1 -- 172.21.15.39:0/1731872333 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee81f5730 con 0x7f2ee80d0e70 2024-09-21T07:47:04.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.683+0000 7f2eee200640 1 -- 172.21.15.39:0/1731872333 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee81f5cf0 con 0x7f2ee80d0e70 2024-09-21T07:47:04.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.684+0000 7f2eee200640 1 -- 172.21.15.39:0/1731872333 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ee80cfca0 con 0x7f2ee80d0e70 2024-09-21T07:47:04.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.685+0000 7f2ee4c00640 1 -- 172.21.15.39:0/1731872333 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2ee001a070 con 0x7f2ee80d0e70 2024-09-21T07:47:04.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.686+0000 7f2ee4c00640 1 --2- 172.21.15.39:0/1731872333 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2ec8078b90 0x7f2ec807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:04.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.686+0000 7f2ee4c00640 1 -- 172.21.15.39:0/1731872333 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 6166+0+0 (secure 0 0 0) 0x7f2ee009a7d0 con 0x7f2ee80d0e70 2024-09-21T07:47:04.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.686+0000 7f2eece00640 1 --2- 172.21.15.39:0/1731872333 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2ec8078b90 0x7f2ec807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:04.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.687+0000 7f2eece00640 1 --2- 172.21.15.39:0/1731872333 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2ec8078b90 0x7f2ec807b0a0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f2edc034000 tx=0x7f2edc004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:04.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.690+0000 7f2ee4c00640 1 -- 172.21.15.39:0/1731872333 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2ee80d17c0 con 0x7f2ee80d0e70 2024-09-21T07:47:04.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:04.898+0000 7f2eee200640 1 -- 172.21.15.39:0/1731872333 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f2ee806f6d0 con 0x7f2ee80d0e70 2024-09-21T07:47:05.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.574+0000 7f2ee4c00640 1 -- 172.21.15.39:0/1731872333 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v188) ==== 97+0+0 (secure 0 0 0) 0x7f2ee00622d0 con 0x7f2ee80d0e70 2024-09-21T07:47:05.576 INFO:teuthology.orchestra.run.smithi039.stderr: removed all blocklist entries 2024-09-21T07:47:05.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.577+0000 7f2eee200640 1 -- 172.21.15.39:0/1731872333 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2ec8078b90 msgr2=0x7f2ec807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:05.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.577+0000 7f2eee200640 1 --2- 172.21.15.39:0/1731872333 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2ec8078b90 0x7f2ec807b0a0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f2edc034000 tx=0x7f2edc004730 comp rx=0 tx=0).stop 2024-09-21T07:47:05.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.577+0000 7f2eee200640 1 -- 172.21.15.39:0/1731872333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ee80d0e70 msgr2=0x7f2ee80cd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:05.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.577+0000 7f2eee200640 1 --2- 172.21.15.39:0/1731872333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ee80d0e70 0x7f2ee80cd410 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f2ee000c730 tx=0x7f2ee000cbf0 comp rx=0 tx=0).stop 2024-09-21T07:47:05.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.577+0000 7f2eee200640 1 -- 172.21.15.39:0/1731872333 shutdown_connections 2024-09-21T07:47:05.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.577+0000 7f2eee200640 1 --2- 172.21.15.39:0/1731872333 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2ec8078b90 0x7f2ec807b0a0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:05.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.577+0000 7f2eee200640 1 --2- 172.21.15.39:0/1731872333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ee80d0e70 0x7f2ee80cd410 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:05.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.577+0000 7f2eee200640 1 --2- 172.21.15.39:0/1731872333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ee80cef70 0x7f2ee809e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:05.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.577+0000 7f2eee200640 1 --2- 172.21.15.39:0/1731872333 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2ee80b5a90 0x7f2ee81600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:05.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.577+0000 7f2eee200640 1 -- 172.21.15.39:0/1731872333 >> 172.21.15.39:0/1731872333 conn(0x7f2ee809d7b0 msgr2=0x7f2ee80d4360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:05.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.577+0000 7f2eee200640 1 -- 172.21.15.39:0/1731872333 shutdown_connections 2024-09-21T07:47:05.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.577+0000 7f2eee200640 1 -- 172.21.15.39:0/1731872333 wait complete. 2024-09-21T07:47:05.611 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T07:47:05.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.842+0000 7fd5d4600640 1 -- 172.21.15.39:0/1275590223 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5cc15fc20 msgr2=0x7fd5cc0d06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:05.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.842+0000 7fd5d4600640 1 --2- 172.21.15.39:0/1275590223 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5cc15fc20 0x7fd5cc0d06b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0005fd0 tx=0x7fd5c002aa90 comp rx=0 tx=0).stop 2024-09-21T07:47:05.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.842+0000 7fd5d4600640 1 -- 172.21.15.39:0/1275590223 shutdown_connections 2024-09-21T07:47:05.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.842+0000 7fd5d4600640 1 --2- 172.21.15.39:0/1275590223 >> v2:172.21.15.148:3300/0 conn(0x7fd5cc155a90 0x7fd5cc0d0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:05.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.842+0000 7fd5d4600640 1 --2- 172.21.15.39:0/1275590223 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5cc15fc20 0x7fd5cc0d06b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:05.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.842+0000 7fd5d4600640 1 -- 172.21.15.39:0/1275590223 >> 172.21.15.39:0/1275590223 conn(0x7fd5cc06cfc0 msgr2=0x7fd5cc06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:05.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.842+0000 7fd5d4600640 1 -- 172.21.15.39:0/1275590223 shutdown_connections 2024-09-21T07:47:05.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.842+0000 7fd5d4600640 1 -- 172.21.15.39:0/1275590223 wait complete. 2024-09-21T07:47:05.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.844+0000 7fd5d4600640 1 Processor -- start 2024-09-21T07:47:05.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.844+0000 7fd5d4600640 1 -- start start 2024-09-21T07:47:05.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.846+0000 7fd5d4600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5cc155a90 0x7fd5cc155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:05.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.846+0000 7fd5d4600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5cc15fc20 0x7fd5cc14ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:05.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.846+0000 7fd5d4600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5cc1624f0 0x7fd5cc14dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:05.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.846+0000 7fd5d4600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd5cc070490 con 0x7fd5cc1624f0 2024-09-21T07:47:05.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.846+0000 7fd5d4600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd5cc070310 con 0x7fd5cc15fc20 2024-09-21T07:47:05.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.846+0000 7fd5d4600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd5cc070610 con 0x7fd5cc155a90 2024-09-21T07:47:05.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.846+0000 7fd5d2600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5cc1624f0 0x7fd5cc14dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:05.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.846+0000 7fd5d2600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5cc1624f0 0x7fd5cc14dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60018/0 (socket says 172.21.15.39:60018) 2024-09-21T07:47:05.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.846+0000 7fd5d2600640 1 -- 172.21.15.39:0/3470076094 learned_addr learned my addr 172.21.15.39:0/3470076094 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:05.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.846+0000 7fd5d1c00640 1 --2- 172.21.15.39:0/3470076094 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5cc155a90 0x7fd5cc155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:05.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.846+0000 7fd5d1200640 1 --2- 172.21.15.39:0/3470076094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5cc15fc20 0x7fd5cc14ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:05.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.847+0000 7fd5d2600640 1 -- 172.21.15.39:0/3470076094 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5cc155a90 msgr2=0x7fd5cc155fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:05.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.847+0000 7fd5d2600640 1 --2- 172.21.15.39:0/3470076094 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5cc155a90 0x7fd5cc155fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:05.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.847+0000 7fd5d2600640 1 -- 172.21.15.39:0/3470076094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5cc15fc20 msgr2=0x7fd5cc14ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:05.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.847+0000 7fd5d2600640 1 --2- 172.21.15.39:0/3470076094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5cc15fc20 0x7fd5cc14ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:05.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.847+0000 7fd5d2600640 1 -- 172.21.15.39:0/3470076094 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5cc1f6880 con 0x7fd5cc1624f0 2024-09-21T07:47:05.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.847+0000 7fd5d2600640 1 --2- 172.21.15.39:0/3470076094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5cc1624f0 0x7fd5cc14dff0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fd5c400c390 tx=0x7fd5c400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:05.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.847+0000 7fd5c9600640 1 -- 172.21.15.39:0/3470076094 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4017070 con 0x7fd5cc1624f0 2024-09-21T07:47:05.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.847+0000 7fd5c9600640 1 -- 172.21.15.39:0/3470076094 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd5c40053e0 con 0x7fd5cc1624f0 2024-09-21T07:47:05.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.847+0000 7fd5d4600640 1 -- 172.21.15.39:0/3470076094 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5cc1f6b20 con 0x7fd5cc1624f0 2024-09-21T07:47:05.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.847+0000 7fd5c9600640 1 -- 172.21.15.39:0/3470076094 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4003850 con 0x7fd5cc1624f0 2024-09-21T07:47:05.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.847+0000 7fd5d4600640 1 -- 172.21.15.39:0/3470076094 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5cc1567f0 con 0x7fd5cc1624f0 2024-09-21T07:47:05.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.848+0000 7fd5d4600640 1 -- 172.21.15.39:0/3470076094 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5cc156a50 con 0x7fd5cc1624f0 2024-09-21T07:47:05.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.850+0000 7fd5c9600640 1 -- 172.21.15.39:0/3470076094 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd5c401a070 con 0x7fd5cc1624f0 2024-09-21T07:47:05.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.851+0000 7fd5c9600640 1 --2- 172.21.15.39:0/3470076094 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd5ac078bc0 0x7fd5ac07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:05.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.851+0000 7fd5c9600640 1 -- 172.21.15.39:0/3470076094 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 6080+0+0 (secure 0 0 0) 0x7fd5c409a0e0 con 0x7fd5cc1624f0 2024-09-21T07:47:05.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.851+0000 7fd5d1c00640 1 --2- 172.21.15.39:0/3470076094 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd5ac078bc0 0x7fd5ac07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:05.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.852+0000 7fd5d1c00640 1 --2- 172.21.15.39:0/3470076094 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd5ac078bc0 0x7fd5ac07b0d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0035000 tx=0x7fd5c0038040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:05.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:05.855+0000 7fd5c9600640 1 -- 172.21.15.39:0/3470076094 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd5c4007e20 con 0x7fd5cc1624f0 2024-09-21T07:47:06.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.124+0000 7fd5d4600640 1 -- 172.21.15.39:0/3470076094 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fd5cc070ac0 con 0x7fd5cc1624f0 2024-09-21T07:47:06.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.125+0000 7fd5c9600640 1 -- 172.21.15.39:0/3470076094 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) ==== 80+0+5304 (secure 0 0 0) 0x7fd5c4061d70 con 0x7fd5cc1624f0 2024-09-21T07:47:06.127 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:47:06.127 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:47:06.127 INFO:teuthology.orchestra.run.smithi039.stdout: "auth_dump": [ 2024-09-21T07:47:06.127 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:06.127 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.a", 2024-09-21T07:47:06.127 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5m+P6EERAABQqmlkrocSi1+hwc73Adrg==", 2024-09-21T07:47:06.127 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:47:06.127 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:47:06.128 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:47:06.128 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:47:06.128 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:47:06.128 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:06.128 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:06.128 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:06.128 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.b", 2024-09-21T07:47:06.128 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5m5/GeLRAAfE8fr5dbitQa89rNK9pZ2Q==", 2024-09-21T07:47:06.128 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:47:06.128 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:47:06.128 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:47:06.128 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:47:06.128 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:47:06.129 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:06.129 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:06.129 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:06.129 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.c", 2024-09-21T07:47:06.129 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5msbx6HxAALsF8ORvIoJpIAFZ5uxmOEw==", 2024-09-21T07:47:06.129 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:47:06.129 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:47:06.129 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:47:06.129 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:47:06.129 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:47:06.129 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:06.129 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:06.129 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:06.129 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.d", 2024-09-21T07:47:06.130 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mIrcBABAApKAB/7FkZ5glNTxlbkIsBw==", 2024-09-21T07:47:06.130 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:47:06.130 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:47:06.130 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:47:06.130 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:47:06.130 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:47:06.130 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:06.130 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:06.130 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:06.130 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.0", 2024-09-21T07:47:06.130 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB7du5msw81DhAAmVimqN4x9K3YpT0nbovppQ==", 2024-09-21T07:47:06.130 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:47:06.130 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:47:06.130 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:47:06.131 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:47:06.131 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:06.131 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:06.131 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:06.131 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.1", 2024-09-21T07:47:06.131 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB+du5mdnCIOhAAtVB4ThehQwb0zRriyKcaxA==", 2024-09-21T07:47:06.131 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:47:06.131 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:47:06.131 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:47:06.131 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:47:06.131 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:06.131 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:06.131 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:06.131 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.2", 2024-09-21T07:47:06.131 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCCdu5mTfWxDhAAadJCbPN5xBx/f9L0v2yTdg==", 2024-09-21T07:47:06.132 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:47:06.132 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:47:06.132 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:47:06.132 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:47:06.132 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:06.132 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:06.132 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:06.132 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.3", 2024-09-21T07:47:06.132 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCFdu5mL1E2HxAAyOhY97uUXEw5L1CcDeONVA==", 2024-09-21T07:47:06.132 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:47:06.132 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:47:06.132 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:47:06.132 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:47:06.133 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:06.133 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:06.133 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:06.133 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.4", 2024-09-21T07:47:06.133 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCLdu5m6PY/IhAAgvcTgm+6uWA7lIG4tcTIEg==", 2024-09-21T07:47:06.133 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:47:06.133 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:47:06.133 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:47:06.133 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:47:06.133 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:06.133 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:06.133 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:06.133 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.5", 2024-09-21T07:47:06.133 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCOdu5mE/wlMxAA3JBg4xH+N/PPC1Uv2ZVrYQ==", 2024-09-21T07:47:06.133 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:47:06.133 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:47:06.134 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:47:06.134 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:47:06.134 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:06.134 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:06.134 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:06.134 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.6", 2024-09-21T07:47:06.134 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCSdu5mC3TgBxAAwsArjeIdFcApY3/1OJffEg==", 2024-09-21T07:47:06.134 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:47:06.134 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:47:06.134 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:47:06.134 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:47:06.134 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:06.134 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:06.134 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:06.134 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.7", 2024-09-21T07:47:06.135 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCVdu5mSWHWFBAA9qPHWRdsvrB74inrp1lj4Q==", 2024-09-21T07:47:06.135 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:47:06.135 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:47:06.135 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:47:06.135 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:47:06.135 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:06.135 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:06.135 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:06.135 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.0", 2024-09-21T07:47:06.135 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mGfb1DBAA/aSMKR5or3xy53rB/iUjpA==", 2024-09-21T07:47:06.135 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:47:06.135 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:47:06.135 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow r", 2024-09-21T07:47:06.135 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T07:47:06.136 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:06.136 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:06.136 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:06.136 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.1", 2024-09-21T07:47:06.136 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mEkciIhAAid+bxJEr4P7vMnZ42PAy5w==", 2024-09-21T07:47:06.136 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:47:06.136 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:47:06.136 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow r", 2024-09-21T07:47:06.136 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow rw", 2024-09-21T07:47:06.136 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rwx" 2024-09-21T07:47:06.136 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:06.136 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:06.136 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:06.136 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.2", 2024-09-21T07:47:06.136 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mSu2oFxAAgjy2m/wWKh6xaoh+YDoU1Q==", 2024-09-21T07:47:06.136 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:47:06.136 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:47:06.137 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow r", 2024-09-21T07:47:06.137 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow rw", 2024-09-21T07:47:06.137 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rwx" 2024-09-21T07:47:06.137 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:06.137 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:06.137 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:06.137 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.3", 2024-09-21T07:47:06.137 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mCHy0LBAAZneF2jUWgfuyGehZFbOxDg==", 2024-09-21T07:47:06.137 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:47:06.137 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:47:06.137 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow r", 2024-09-21T07:47:06.137 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow rw", 2024-09-21T07:47:06.137 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rwx" 2024-09-21T07:47:06.138 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:06.138 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:06.138 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:06.138 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.admin", 2024-09-21T07:47:06.138 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBydu5m9LInERAAAlDpqb94f2Qb9VYJJwTBcA==", 2024-09-21T07:47:06.138 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:47:06.138 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow *", 2024-09-21T07:47:06.138 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:47:06.138 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:47:06.138 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:47:06.138 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:06.138 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:06.138 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:06.138 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mgr.x", 2024-09-21T07:47:06.138 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5mA2GRBhAAkyCKdgH6laHy8xNt080dXg==", 2024-09-21T07:47:06.139 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:47:06.139 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow *", 2024-09-21T07:47:06.139 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile mgr", 2024-09-21T07:47:06.139 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:47:06.139 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:06.139 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:06.139 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:06.139 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mgr.y", 2024-09-21T07:47:06.139 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBydu5mr+8aNxAA7TFYNms2WsXUfa4ITTJOHw==", 2024-09-21T07:47:06.139 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:47:06.139 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow *", 2024-09-21T07:47:06.139 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile mgr", 2024-09-21T07:47:06.139 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:47:06.139 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:06.139 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:06.139 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-21T07:47:06.140 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:47:06.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.127+0000 7fd5d4600640 1 -- 172.21.15.39:0/3470076094 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd5ac078bc0 msgr2=0x7fd5ac07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:06.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.127+0000 7fd5d4600640 1 --2- 172.21.15.39:0/3470076094 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd5ac078bc0 0x7fd5ac07b0d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0035000 tx=0x7fd5c0038040 comp rx=0 tx=0).stop 2024-09-21T07:47:06.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.127+0000 7fd5d4600640 1 -- 172.21.15.39:0/3470076094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5cc1624f0 msgr2=0x7fd5cc14dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:06.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.127+0000 7fd5d4600640 1 --2- 172.21.15.39:0/3470076094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5cc1624f0 0x7fd5cc14dff0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fd5c400c390 tx=0x7fd5c400c850 comp rx=0 tx=0).stop 2024-09-21T07:47:06.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.127+0000 7fd5d4600640 1 -- 172.21.15.39:0/3470076094 shutdown_connections 2024-09-21T07:47:06.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.127+0000 7fd5d4600640 1 --2- 172.21.15.39:0/3470076094 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd5ac078bc0 0x7fd5ac07b0d0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:06.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.127+0000 7fd5d4600640 1 --2- 172.21.15.39:0/3470076094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5cc1624f0 0x7fd5cc14dff0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:06.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.127+0000 7fd5d4600640 1 --2- 172.21.15.39:0/3470076094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5cc15fc20 0x7fd5cc14ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:06.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.127+0000 7fd5d4600640 1 --2- 172.21.15.39:0/3470076094 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5cc155a90 0x7fd5cc155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:06.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.127+0000 7fd5d4600640 1 -- 172.21.15.39:0/3470076094 >> 172.21.15.39:0/3470076094 conn(0x7fd5cc06cfc0 msgr2=0x7fd5cc0b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:06.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.127+0000 7fd5d4600640 1 -- 172.21.15.39:0/3470076094 shutdown_connections 2024-09-21T07:47:06.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.127+0000 7fd5d4600640 1 -- 172.21.15.39:0/3470076094 wait complete. 2024-09-21T07:47:06.153 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T07:47:06.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.379+0000 7fec32400640 1 -- 172.21.15.39:0/1888890406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec2c0cfb60 msgr2=0x7fec2c073130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:06.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.379+0000 7fec32400640 1 --2- 172.21.15.39:0/1888890406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec2c0cfb60 0x7fec2c073130 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fec24005fd0 tx=0x7fec2402aa00 comp rx=0 tx=0).stop 2024-09-21T07:47:06.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.379+0000 7fec32400640 1 -- 172.21.15.39:0/1888890406 shutdown_connections 2024-09-21T07:47:06.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.379+0000 7fec32400640 1 --2- 172.21.15.39:0/1888890406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec2c0cfb60 0x7fec2c073130 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:06.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.379+0000 7fec32400640 1 --2- 172.21.15.39:0/1888890406 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec2c070bc0 0x7fec2c0d06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:06.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.379+0000 7fec32400640 1 -- 172.21.15.39:0/1888890406 >> 172.21.15.39:0/1888890406 conn(0x7fec2c06cfc0 msgr2=0x7fec2c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:06.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.379+0000 7fec32400640 1 -- 172.21.15.39:0/1888890406 shutdown_connections 2024-09-21T07:47:06.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.379+0000 7fec32400640 1 -- 172.21.15.39:0/1888890406 wait complete. 2024-09-21T07:47:06.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.382+0000 7fec32400640 1 Processor -- start 2024-09-21T07:47:06.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.382+0000 7fec32400640 1 -- start start 2024-09-21T07:47:06.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.383+0000 7fec32400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec2c070bc0 0x7fec2c141da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:06.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.383+0000 7fec32400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec2c0cfb60 0x7fec2c147b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:06.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.383+0000 7fec32400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec2c0d0bf0 0x7fec2c1e9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:06.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.383+0000 7fec32400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fec2c0737f0 con 0x7fec2c0d0bf0 2024-09-21T07:47:06.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.383+0000 7fec32400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fec2c073670 con 0x7fec2c070bc0 2024-09-21T07:47:06.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.383+0000 7fec32400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fec2c073970 con 0x7fec2c0cfb60 2024-09-21T07:47:06.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.383+0000 7fec2be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec2c0d0bf0 0x7fec2c1e9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:06.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.384+0000 7fec2be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec2c0d0bf0 0x7fec2c1e9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60042/0 (socket says 172.21.15.39:60042) 2024-09-21T07:47:06.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.384+0000 7fec2be00640 1 -- 172.21.15.39:0/607963774 learned_addr learned my addr 172.21.15.39:0/607963774 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:06.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.384+0000 7fec2b400640 1 --2- 172.21.15.39:0/607963774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec2c070bc0 0x7fec2c141da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:06.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.384+0000 7fec23e00640 1 --2- 172.21.15.39:0/607963774 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec2c0cfb60 0x7fec2c147b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:06.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.384+0000 7fec2be00640 1 -- 172.21.15.39:0/607963774 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec2c0cfb60 msgr2=0x7fec2c147b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:06.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.384+0000 7fec2be00640 1 --2- 172.21.15.39:0/607963774 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec2c0cfb60 0x7fec2c147b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:06.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.384+0000 7fec2be00640 1 -- 172.21.15.39:0/607963774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec2c070bc0 msgr2=0x7fec2c141da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:06.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.384+0000 7fec2be00640 1 --2- 172.21.15.39:0/607963774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec2c070bc0 0x7fec2c141da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:06.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.384+0000 7fec2be00640 1 -- 172.21.15.39:0/607963774 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec2c148760 con 0x7fec2c0d0bf0 2024-09-21T07:47:06.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.384+0000 7fec23e00640 1 --2- 172.21.15.39:0/607963774 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec2c0cfb60 0x7fec2c147b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:06.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.384+0000 7fec2b400640 1 --2- 172.21.15.39:0/607963774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec2c070bc0 0x7fec2c141da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:06.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.384+0000 7fec2be00640 1 --2- 172.21.15.39:0/607963774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec2c0d0bf0 0x7fec2c1e9360 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fec24009d90 tx=0x7fec24003f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:06.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.385+0000 7fec23400640 1 -- 172.21.15.39:0/607963774 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec24046070 con 0x7fec2c0d0bf0 2024-09-21T07:47:06.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.385+0000 7fec23400640 1 -- 172.21.15.39:0/607963774 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec24004540 con 0x7fec2c0d0bf0 2024-09-21T07:47:06.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.385+0000 7fec23400640 1 -- 172.21.15.39:0/607963774 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec24033070 con 0x7fec2c0d0bf0 2024-09-21T07:47:06.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.385+0000 7fec32400640 1 -- 172.21.15.39:0/607963774 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec2c1f8ed0 con 0x7fec2c0d0bf0 2024-09-21T07:47:06.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.385+0000 7fec32400640 1 -- 172.21.15.39:0/607963774 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec2c153980 con 0x7fec2c0d0bf0 2024-09-21T07:47:06.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.386+0000 7fec32400640 1 -- 172.21.15.39:0/607963774 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec2c1484d0 con 0x7fec2c0d0bf0 2024-09-21T07:47:06.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.387+0000 7fec23400640 1 -- 172.21.15.39:0/607963774 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fec24049070 con 0x7fec2c0d0bf0 2024-09-21T07:47:06.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.388+0000 7fec23400640 1 --2- 172.21.15.39:0/607963774 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fec08078b90 0x7fec0807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:06.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.388+0000 7fec23400640 1 -- 172.21.15.39:0/607963774 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 6080+0+0 (secure 0 0 0) 0x7fec240bc100 con 0x7fec2c0d0bf0 2024-09-21T07:47:06.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.389+0000 7fec2b400640 1 --2- 172.21.15.39:0/607963774 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fec08078b90 0x7fec0807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:06.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.389+0000 7fec2b400640 1 --2- 172.21.15.39:0/607963774 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fec08078b90 0x7fec0807b0a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fec1c005fd0 tx=0x7fec1c00f360 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:06.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.392+0000 7fec23400640 1 -- 172.21.15.39:0/607963774 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fec2403e070 con 0x7fec2c0d0bf0 2024-09-21T07:47:06.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.593+0000 7fec32400640 1 -- 172.21.15.39:0/607963774 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fec2c073ef0 con 0x7fec2c0d0bf0 2024-09-21T07:47:06.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.595+0000 7fec23400640 1 -- 172.21.15.39:0/607963774 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v190) ==== 84+0+287 (secure 0 0 0) 0x7fec24084350 con 0x7fec2c0d0bf0 2024-09-21T07:47:06.597 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:47:06.597 INFO:teuthology.orchestra.run.smithi039.stdout:[ 2024-09-21T07:47:06.597 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:06.597 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 1, 2024-09-21T07:47:06.597 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": ".mgr" 2024-09-21T07:47:06.597 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:06.597 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:06.597 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 6, 2024-09-21T07:47:06.597 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": "snap-test-pool" 2024-09-21T07:47:06.597 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:06.597 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:06.597 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 11, 2024-09-21T07:47:06.598 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": "test_data_pool" 2024-09-21T07:47:06.598 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:06.598 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:06.598 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 12, 2024-09-21T07:47:06.598 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": "test_metadata_pool" 2024-09-21T07:47:06.598 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:06.598 INFO:teuthology.orchestra.run.smithi039.stdout:] 2024-09-21T07:47:06.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.597+0000 7fec32400640 1 -- 172.21.15.39:0/607963774 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fec08078b90 msgr2=0x7fec0807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:06.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.597+0000 7fec32400640 1 --2- 172.21.15.39:0/607963774 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fec08078b90 0x7fec0807b0a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fec1c005fd0 tx=0x7fec1c00f360 comp rx=0 tx=0).stop 2024-09-21T07:47:06.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.597+0000 7fec32400640 1 -- 172.21.15.39:0/607963774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec2c0d0bf0 msgr2=0x7fec2c1e9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:06.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.597+0000 7fec32400640 1 --2- 172.21.15.39:0/607963774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec2c0d0bf0 0x7fec2c1e9360 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fec24009d90 tx=0x7fec24003f20 comp rx=0 tx=0).stop 2024-09-21T07:47:06.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.597+0000 7fec32400640 1 -- 172.21.15.39:0/607963774 shutdown_connections 2024-09-21T07:47:06.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.597+0000 7fec32400640 1 --2- 172.21.15.39:0/607963774 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fec08078b90 0x7fec0807b0a0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:06.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.597+0000 7fec32400640 1 --2- 172.21.15.39:0/607963774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec2c0d0bf0 0x7fec2c1e9360 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:06.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.597+0000 7fec32400640 1 --2- 172.21.15.39:0/607963774 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec2c0cfb60 0x7fec2c147b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:06.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.597+0000 7fec32400640 1 --2- 172.21.15.39:0/607963774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec2c070bc0 0x7fec2c141da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:06.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.597+0000 7fec32400640 1 -- 172.21.15.39:0/607963774 >> 172.21.15.39:0/607963774 conn(0x7fec2c06cfc0 msgr2=0x7fec2c15f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:06.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.597+0000 7fec32400640 1 -- 172.21.15.39:0/607963774 shutdown_connections 2024-09-21T07:47:06.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.597+0000 7fec32400640 1 -- 172.21.15.39:0/607963774 wait complete. 2024-09-21T07:47:06.621 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T07:47:06.622 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T07:47:06.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.853+0000 7fb72f200640 1 -- 172.21.15.39:0/2052061049 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb72815cc30 msgr2=0x7fb72815d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:06.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.853+0000 7fb72f200640 1 --2- 172.21.15.39:0/2052061049 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb72815cc30 0x7fb72815d040 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fb7280cb1e0 tx=0x7fb72002aa30 comp rx=0 tx=0).stop 2024-09-21T07:47:06.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.853+0000 7fb72f200640 1 -- 172.21.15.39:0/2052061049 shutdown_connections 2024-09-21T07:47:06.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.853+0000 7fb72f200640 1 --2- 172.21.15.39:0/2052061049 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb72815cc30 0x7fb72815d040 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:06.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.853+0000 7fb72f200640 1 --2- 172.21.15.39:0/2052061049 >> v2:172.21.15.148:3300/0 conn(0x7fb72806f560 0x7fb72806f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:06.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.853+0000 7fb72f200640 1 --2- 172.21.15.39:0/2052061049 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb728070bc0 0x7fb72806f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:06.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.853+0000 7fb72f200640 1 -- 172.21.15.39:0/2052061049 >> 172.21.15.39:0/2052061049 conn(0x7fb7280b2640 msgr2=0x7fb7280d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:06.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.853+0000 7fb72f200640 1 -- 172.21.15.39:0/2052061049 shutdown_connections 2024-09-21T07:47:06.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.853+0000 7fb72f200640 1 -- 172.21.15.39:0/2052061049 wait complete. 2024-09-21T07:47:06.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.855+0000 7fb72f200640 1 Processor -- start 2024-09-21T07:47:06.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.855+0000 7fb72f200640 1 -- start start 2024-09-21T07:47:06.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.857+0000 7fb72f200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb72806f560 0x7fb7280c6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:06.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.857+0000 7fb72f200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb728070bc0 0x7fb7281f3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:06.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.857+0000 7fb72f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb72815cc30 0x7fb7280c3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:06.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.857+0000 7fb72f200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb72815d700 con 0x7fb72815cc30 2024-09-21T07:47:06.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.857+0000 7fb72f200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb72815d580 con 0x7fb728070bc0 2024-09-21T07:47:06.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.857+0000 7fb72f200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb72815d880 con 0x7fb72806f560 2024-09-21T07:47:06.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.857+0000 7fb72d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb72815cc30 0x7fb7280c3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:06.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.857+0000 7fb72d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb72815cc30 0x7fb7280c3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60062/0 (socket says 172.21.15.39:60062) 2024-09-21T07:47:06.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.857+0000 7fb72d200640 1 -- 172.21.15.39:0/1727190212 learned_addr learned my addr 172.21.15.39:0/1727190212 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:06.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.857+0000 7fb727e00640 1 --2- 172.21.15.39:0/1727190212 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb72806f560 0x7fb7280c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:06.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.857+0000 7fb727400640 1 --2- 172.21.15.39:0/1727190212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb728070bc0 0x7fb7281f3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:06.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.857+0000 7fb72d200640 1 -- 172.21.15.39:0/1727190212 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb72806f560 msgr2=0x7fb7280c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:06.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.857+0000 7fb72d200640 1 --2- 172.21.15.39:0/1727190212 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb72806f560 0x7fb7280c6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:06.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.857+0000 7fb72d200640 1 -- 172.21.15.39:0/1727190212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb728070bc0 msgr2=0x7fb7281f3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:06.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.857+0000 7fb72d200640 1 --2- 172.21.15.39:0/1727190212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb728070bc0 0x7fb7281f3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:06.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.857+0000 7fb72d200640 1 -- 172.21.15.39:0/1727190212 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7280c7340 con 0x7fb72815cc30 2024-09-21T07:47:06.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.858+0000 7fb72d200640 1 --2- 172.21.15.39:0/1727190212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb72815cc30 0x7fb7280c3dd0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fb72002b020 tx=0x7fb720003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:06.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.858+0000 7fb71fe00640 1 -- 172.21.15.39:0/1727190212 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb720033050 con 0x7fb72815cc30 2024-09-21T07:47:06.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.858+0000 7fb71fe00640 1 -- 172.21.15.39:0/1727190212 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb720004450 con 0x7fb72815cc30 2024-09-21T07:47:06.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.858+0000 7fb72f200640 1 -- 172.21.15.39:0/1727190212 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7280c7620 con 0x7fb72815cc30 2024-09-21T07:47:06.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.858+0000 7fb71fe00640 1 -- 172.21.15.39:0/1727190212 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb720038450 con 0x7fb72815cc30 2024-09-21T07:47:06.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.858+0000 7fb72f200640 1 -- 172.21.15.39:0/1727190212 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7281f3840 con 0x7fb72815cc30 2024-09-21T07:47:06.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.859+0000 7fb72f200640 1 -- 172.21.15.39:0/1727190212 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7281f42a0 con 0x7fb72815cc30 2024-09-21T07:47:06.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.862+0000 7fb71fe00640 1 -- 172.21.15.39:0/1727190212 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb720036070 con 0x7fb72815cc30 2024-09-21T07:47:06.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.863+0000 7fb71fe00640 1 --2- 172.21.15.39:0/1727190212 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb708078b90 0x7fb70807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:06.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.863+0000 7fb71fe00640 1 -- 172.21.15.39:0/1727190212 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 6080+0+0 (secure 0 0 0) 0x7fb7200bc560 con 0x7fb72815cc30 2024-09-21T07:47:06.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.863+0000 7fb727e00640 1 --2- 172.21.15.39:0/1727190212 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb708078b90 0x7fb70807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:06.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.864+0000 7fb727e00640 1 --2- 172.21.15.39:0/1727190212 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb708078b90 0x7fb70807b0a0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fb718004630 tx=0x7fb71800b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:06.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:06.866+0000 7fb71fe00640 1 -- 172.21.15.39:0/1727190212 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb7281f42a0 con 0x7fb72815cc30 2024-09-21T07:47:07.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.063+0000 7fb72f200640 1 -- 172.21.15.39:0/1727190212 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb72806f020 con 0x7fb72815cc30 2024-09-21T07:47:07.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.598+0000 7fb71fe00640 1 -- 172.21.15.39:0/1727190212 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v191) ==== 138+0+0 (secure 0 0 0) 0x7fb7200841f0 con 0x7fb72815cc30 2024-09-21T07:47:07.600 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' created 2024-09-21T07:47:07.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.601+0000 7fb72f200640 1 -- 172.21.15.39:0/1727190212 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb708078b90 msgr2=0x7fb70807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:07.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.601+0000 7fb72f200640 1 --2- 172.21.15.39:0/1727190212 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb708078b90 0x7fb70807b0a0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fb718004630 tx=0x7fb71800b2e0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.601+0000 7fb72f200640 1 -- 172.21.15.39:0/1727190212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb72815cc30 msgr2=0x7fb7280c3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:07.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.601+0000 7fb72f200640 1 --2- 172.21.15.39:0/1727190212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb72815cc30 0x7fb7280c3dd0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fb72002b020 tx=0x7fb720003f10 comp rx=0 tx=0).stop 2024-09-21T07:47:07.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.602+0000 7fb72f200640 1 -- 172.21.15.39:0/1727190212 shutdown_connections 2024-09-21T07:47:07.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.602+0000 7fb72f200640 1 --2- 172.21.15.39:0/1727190212 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb708078b90 0x7fb70807b0a0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.602+0000 7fb72f200640 1 --2- 172.21.15.39:0/1727190212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb72815cc30 0x7fb7280c3dd0 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.602+0000 7fb72f200640 1 --2- 172.21.15.39:0/1727190212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb728070bc0 0x7fb7281f3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.602+0000 7fb72f200640 1 --2- 172.21.15.39:0/1727190212 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb72806f560 0x7fb7280c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.602+0000 7fb72f200640 1 -- 172.21.15.39:0/1727190212 >> 172.21.15.39:0/1727190212 conn(0x7fb7280b2640 msgr2=0x7fb72815c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:07.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.602+0000 7fb72f200640 1 -- 172.21.15.39:0/1727190212 shutdown_connections 2024-09-21T07:47:07.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.603+0000 7fb72f200640 1 -- 172.21.15.39:0/1727190212 wait complete. 2024-09-21T07:47:07.638 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T07:47:07.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.870+0000 7fb417e00640 1 -- 172.21.15.39:0/3867006785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4180d0bc0 msgr2=0x7fb41815d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:07.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.870+0000 7fb417e00640 1 --2- 172.21.15.39:0/3867006785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4180d0bc0 0x7fb41815d010 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fb410005fd0 tx=0x7fb41002aaa0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.870+0000 7fb417e00640 1 -- 172.21.15.39:0/3867006785 shutdown_connections 2024-09-21T07:47:07.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.870+0000 7fb417e00640 1 --2- 172.21.15.39:0/3867006785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4180d0bc0 0x7fb41815d010 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.870+0000 7fb417e00640 1 -- 172.21.15.39:0/3867006785 >> 172.21.15.39:0/3867006785 conn(0x7fb41806cfc0 msgr2=0x7fb41806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:07.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.870+0000 7fb417e00640 1 -- 172.21.15.39:0/3867006785 shutdown_connections 2024-09-21T07:47:07.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.870+0000 7fb417e00640 1 -- 172.21.15.39:0/3867006785 wait complete. 2024-09-21T07:47:07.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.873+0000 7fb417e00640 1 Processor -- start 2024-09-21T07:47:07.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.873+0000 7fb417e00640 1 -- start start 2024-09-21T07:47:07.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.874+0000 7fb417e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4180cfa20 0x7fb4181e3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:07.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.874+0000 7fb417e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4180d0bc0 0x7fb418139c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:07.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.874+0000 7fb417e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb41815cc00 0x7fb4180b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:07.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.874+0000 7fb417e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb41806f080 con 0x7fb41815cc00 2024-09-21T07:47:07.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.874+0000 7fb417e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb41806ef00 con 0x7fb4180d0bc0 2024-09-21T07:47:07.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.874+0000 7fb417e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb41806f200 con 0x7fb4180cfa20 2024-09-21T07:47:07.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.875+0000 7fb417400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb41815cc00 0x7fb4180b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:07.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.875+0000 7fb417400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb41815cc00 0x7fb4180b0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60076/0 (socket says 172.21.15.39:60076) 2024-09-21T07:47:07.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.875+0000 7fb417400640 1 -- 172.21.15.39:0/3528403081 learned_addr learned my addr 172.21.15.39:0/3528403081 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:07.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.875+0000 7fb40fe00640 1 --2- 172.21.15.39:0/3528403081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4180d0bc0 0x7fb418139c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:07.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.875+0000 7fb416a00640 1 --2- 172.21.15.39:0/3528403081 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4180cfa20 0x7fb4181e3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:07.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.875+0000 7fb417400640 1 -- 172.21.15.39:0/3528403081 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4180cfa20 msgr2=0x7fb4181e3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:07.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.875+0000 7fb417400640 1 --2- 172.21.15.39:0/3528403081 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4180cfa20 0x7fb4181e3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.875+0000 7fb417400640 1 -- 172.21.15.39:0/3528403081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4180d0bc0 msgr2=0x7fb418139c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:07.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.875+0000 7fb417400640 1 --2- 172.21.15.39:0/3528403081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4180d0bc0 0x7fb418139c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.875+0000 7fb417400640 1 -- 172.21.15.39:0/3528403081 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb41813a180 con 0x7fb41815cc00 2024-09-21T07:47:07.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.875+0000 7fb417400640 1 --2- 172.21.15.39:0/3528403081 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb41815cc00 0x7fb4180b0f30 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fb410006010 tx=0x7fb410003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:07.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.876+0000 7fb40f400640 1 -- 172.21.15.39:0/3528403081 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb410044070 con 0x7fb41815cc00 2024-09-21T07:47:07.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.876+0000 7fb40f400640 1 -- 172.21.15.39:0/3528403081 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb410004250 con 0x7fb41815cc00 2024-09-21T07:47:07.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.876+0000 7fb40f400640 1 -- 172.21.15.39:0/3528403081 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb410004680 con 0x7fb41815cc00 2024-09-21T07:47:07.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.876+0000 7fb417e00640 1 -- 172.21.15.39:0/3528403081 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb418145150 con 0x7fb41815cc00 2024-09-21T07:47:07.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.876+0000 7fb417e00640 1 -- 172.21.15.39:0/3528403081 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4181f9520 con 0x7fb41815cc00 2024-09-21T07:47:07.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.877+0000 7fb417e00640 1 -- 172.21.15.39:0/3528403081 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb41815d010 con 0x7fb41815cc00 2024-09-21T07:47:07.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.879+0000 7fb40f400640 1 -- 172.21.15.39:0/3528403081 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb410047070 con 0x7fb41815cc00 2024-09-21T07:47:07.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.880+0000 7fb40f400640 1 --2- 172.21.15.39:0/3528403081 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb3f4078bc0 0x7fb3f407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:07.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.880+0000 7fb40f400640 1 -- 172.21.15.39:0/3528403081 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(191..191 src has 1..191) ==== 6450+0+0 (secure 0 0 0) 0x7fb4100bc170 con 0x7fb41815cc00 2024-09-21T07:47:07.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.880+0000 7fb416a00640 1 --2- 172.21.15.39:0/3528403081 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb3f4078bc0 0x7fb3f407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:07.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.881+0000 7fb416a00640 1 --2- 172.21.15.39:0/3528403081 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb3f4078bc0 0x7fb3f407b0d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fb408006fd0 tx=0x7fb408008040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:07.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:07.885+0000 7fb40f400640 1 -- 172.21.15.39:0/3528403081 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb410032030 con 0x7fb41815cc00 2024-09-21T07:47:08.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.081+0000 7fb417e00640 1 -- 172.21.15.39:0/3528403081 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb4180b2640 con 0x7fb41815cc00 2024-09-21T07:47:08.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.607+0000 7fb40f400640 1 -- 172.21.15.39:0/3528403081 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v192) ==== 170+0+0 (secure 0 0 0) 0x7fb410084220 con 0x7fb41815cc00 2024-09-21T07:47:08.609 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' created 2024-09-21T07:47:08.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.610+0000 7fb417e00640 1 -- 172.21.15.39:0/3528403081 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb3f4078bc0 msgr2=0x7fb3f407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:08.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.610+0000 7fb417e00640 1 --2- 172.21.15.39:0/3528403081 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb3f4078bc0 0x7fb3f407b0d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fb408006fd0 tx=0x7fb408008040 comp rx=0 tx=0).stop 2024-09-21T07:47:08.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.610+0000 7fb417e00640 1 -- 172.21.15.39:0/3528403081 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb41815cc00 msgr2=0x7fb4180b0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:08.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.610+0000 7fb417e00640 1 --2- 172.21.15.39:0/3528403081 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb41815cc00 0x7fb4180b0f30 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fb410006010 tx=0x7fb410003f10 comp rx=0 tx=0).stop 2024-09-21T07:47:08.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.610+0000 7fb417e00640 1 -- 172.21.15.39:0/3528403081 shutdown_connections 2024-09-21T07:47:08.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.610+0000 7fb417e00640 1 --2- 172.21.15.39:0/3528403081 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb3f4078bc0 0x7fb3f407b0d0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:08.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.610+0000 7fb417e00640 1 --2- 172.21.15.39:0/3528403081 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb41815cc00 0x7fb4180b0f30 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:08.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.610+0000 7fb417e00640 1 --2- 172.21.15.39:0/3528403081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb4180d0bc0 0x7fb418139c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:08.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.610+0000 7fb417e00640 1 --2- 172.21.15.39:0/3528403081 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb4180cfa20 0x7fb4181e3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:08.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.610+0000 7fb417e00640 1 -- 172.21.15.39:0/3528403081 >> 172.21.15.39:0/3528403081 conn(0x7fb41806cfc0 msgr2=0x7fb4181e8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:08.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.611+0000 7fb417e00640 1 -- 172.21.15.39:0/3528403081 shutdown_connections 2024-09-21T07:47:08.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.611+0000 7fb417e00640 1 -- 172.21.15.39:0/3528403081 wait complete. 2024-09-21T07:47:08.654 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T07:47:08.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.888+0000 7ff140a00640 1 -- 172.21.15.39:0/2973896545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff13c0cfb60 msgr2=0x7ff13c073130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:08.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.888+0000 7ff140a00640 1 --2- 172.21.15.39:0/2973896545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff13c0cfb60 0x7ff13c073130 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7ff134005fd0 tx=0x7ff13402aa10 comp rx=0 tx=0).stop 2024-09-21T07:47:08.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.888+0000 7ff140a00640 1 -- 172.21.15.39:0/2973896545 shutdown_connections 2024-09-21T07:47:08.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.888+0000 7ff140a00640 1 --2- 172.21.15.39:0/2973896545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff13c0cfb60 0x7ff13c073130 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:08.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.888+0000 7ff140a00640 1 --2- 172.21.15.39:0/2973896545 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff13c070c30 0x7ff13c0d0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:08.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.888+0000 7ff140a00640 1 -- 172.21.15.39:0/2973896545 >> 172.21.15.39:0/2973896545 conn(0x7ff13c06cfc0 msgr2=0x7ff13c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:08.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.888+0000 7ff140a00640 1 -- 172.21.15.39:0/2973896545 shutdown_connections 2024-09-21T07:47:08.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.888+0000 7ff140a00640 1 -- 172.21.15.39:0/2973896545 wait complete. 2024-09-21T07:47:08.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.890+0000 7ff140a00640 1 Processor -- start 2024-09-21T07:47:08.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.890+0000 7ff140a00640 1 -- start start 2024-09-21T07:47:08.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.892+0000 7ff140a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff13c070c30 0x7ff13c146df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:08.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.892+0000 7ff140a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff13c0cfb60 0x7ff13c0aa260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:08.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.892+0000 7ff140a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff13c0d0bd0 0x7ff13c0aa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:08.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.892+0000 7ff140a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c073a00 con 0x7ff13c070c30 2024-09-21T07:47:08.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.892+0000 7ff140a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c073880 con 0x7ff13c0d0bd0 2024-09-21T07:47:08.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.892+0000 7ff140a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7ff13c073b80 con 0x7ff13c0cfb60 2024-09-21T07:47:08.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.892+0000 7ff139e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff13c070c30 0x7ff13c146df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:08.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.892+0000 7ff139e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff13c070c30 0x7ff13c146df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60104/0 (socket says 172.21.15.39:60104) 2024-09-21T07:47:08.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.892+0000 7ff139e00640 1 -- 172.21.15.39:0/3753291588 learned_addr learned my addr 172.21.15.39:0/3753291588 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:08.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.892+0000 7ff13a800640 1 --2- 172.21.15.39:0/3753291588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff13c0d0bd0 0x7ff13c0aa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:08.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.892+0000 7ff139400640 1 --2- 172.21.15.39:0/3753291588 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff13c0cfb60 0x7ff13c0aa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:08.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.892+0000 7ff139e00640 1 -- 172.21.15.39:0/3753291588 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff13c0cfb60 msgr2=0x7ff13c0aa260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:08.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.892+0000 7ff139e00640 1 --2- 172.21.15.39:0/3753291588 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff13c0cfb60 0x7ff13c0aa260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:08.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.892+0000 7ff139e00640 1 -- 172.21.15.39:0/3753291588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff13c0d0bd0 msgr2=0x7ff13c0aa7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:08.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.892+0000 7ff139e00640 1 --2- 172.21.15.39:0/3753291588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff13c0d0bd0 0x7ff13c0aa7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:08.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.892+0000 7ff139e00640 1 -- 172.21.15.39:0/3753291588 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff13c150ef0 con 0x7ff13c070c30 2024-09-21T07:47:08.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.893+0000 7ff139e00640 1 --2- 172.21.15.39:0/3753291588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff13c070c30 0x7ff13c146df0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00e790 tx=0x7ff12c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:08.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.893+0000 7ff132000640 1 -- 172.21.15.39:0/3753291588 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c00eea0 con 0x7ff13c070c30 2024-09-21T07:47:08.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.893+0000 7ff132000640 1 -- 172.21.15.39:0/3753291588 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff12c00cdb0 con 0x7ff13c070c30 2024-09-21T07:47:08.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.893+0000 7ff132000640 1 -- 172.21.15.39:0/3753291588 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c010690 con 0x7ff13c070c30 2024-09-21T07:47:08.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.893+0000 7ff140a00640 1 -- 172.21.15.39:0/3753291588 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff13c1511d0 con 0x7ff13c070c30 2024-09-21T07:47:08.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.893+0000 7ff140a00640 1 -- 172.21.15.39:0/3753291588 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff13c151790 con 0x7ff13c070c30 2024-09-21T07:47:08.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.894+0000 7ff140a00640 1 -- 172.21.15.39:0/3753291588 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff13c074030 con 0x7ff13c070c30 2024-09-21T07:47:08.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.896+0000 7ff132000640 1 -- 172.21.15.39:0/3753291588 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff12c010830 con 0x7ff13c070c30 2024-09-21T07:47:08.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.897+0000 7ff132000640 1 --2- 172.21.15.39:0/3753291588 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff11c078b90 0x7ff11c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:08.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.897+0000 7ff132000640 1 -- 172.21.15.39:0/3753291588 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(192..192 src has 1..192) ==== 6832+0+0 (secure 0 0 0) 0x7ff12c09c330 con 0x7ff13c070c30 2024-09-21T07:47:08.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.897+0000 7ff139400640 1 --2- 172.21.15.39:0/3753291588 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff11c078b90 0x7ff11c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:08.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.898+0000 7ff139400640 1 --2- 172.21.15.39:0/3753291588 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff11c078b90 0x7ff11c07b0a0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7ff128000f60 tx=0x7ff128009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:08.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:08.901+0000 7ff132000640 1 -- 172.21.15.39:0/3753291588 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff12c063d50 con 0x7ff13c070c30 2024-09-21T07:47:09.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.138+0000 7ff140a00640 1 -- 172.21.15.39:0/3753291588 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7ff13c0d0970 con 0x7ff13c070c30 2024-09-21T07:47:09.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.610+0000 7ff132000640 1 -- 172.21.15.39:0/3753291588 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 23 and data pool 24 v195) ==== 174+0+0 (secure 0 0 0) 0x7ff12c09c110 con 0x7ff13c070c30 2024-09-21T07:47:09.612 INFO:teuthology.orchestra.run.smithi039.stderr:new fs with metadata pool 23 and data pool 24 2024-09-21T07:47:09.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.613+0000 7ff140a00640 1 -- 172.21.15.39:0/3753291588 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff11c078b90 msgr2=0x7ff11c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:09.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.613+0000 7ff140a00640 1 --2- 172.21.15.39:0/3753291588 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff11c078b90 0x7ff11c07b0a0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7ff128000f60 tx=0x7ff128009040 comp rx=0 tx=0).stop 2024-09-21T07:47:09.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.613+0000 7ff140a00640 1 -- 172.21.15.39:0/3753291588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff13c070c30 msgr2=0x7ff13c146df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:09.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.613+0000 7ff140a00640 1 --2- 172.21.15.39:0/3753291588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff13c070c30 0x7ff13c146df0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00e790 tx=0x7ff12c00ec50 comp rx=0 tx=0).stop 2024-09-21T07:47:09.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.613+0000 7ff140a00640 1 -- 172.21.15.39:0/3753291588 shutdown_connections 2024-09-21T07:47:09.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.613+0000 7ff140a00640 1 --2- 172.21.15.39:0/3753291588 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff11c078b90 0x7ff11c07b0a0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:09.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.613+0000 7ff140a00640 1 --2- 172.21.15.39:0/3753291588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff13c0d0bd0 0x7ff13c0aa7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:09.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.613+0000 7ff140a00640 1 --2- 172.21.15.39:0/3753291588 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff13c0cfb60 0x7ff13c0aa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:09.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.613+0000 7ff140a00640 1 --2- 172.21.15.39:0/3753291588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff13c070c30 0x7ff13c146df0 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:09.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.613+0000 7ff140a00640 1 -- 172.21.15.39:0/3753291588 >> 172.21.15.39:0/3753291588 conn(0x7ff13c06cfc0 msgr2=0x7ff13c15c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:09.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.613+0000 7ff140a00640 1 -- 172.21.15.39:0/3753291588 shutdown_connections 2024-09-21T07:47:09.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.613+0000 7ff140a00640 1 -- 172.21.15.39:0/3753291588 wait complete. 2024-09-21T07:47:09.667 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:47:09.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.893+0000 7fb285a00640 1 -- 172.21.15.39:0/2529791696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2800cfc50 msgr2=0x7fb2800d0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:09.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.893+0000 7fb285a00640 1 --2- 172.21.15.39:0/2529791696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2800cfc50 0x7fb2800d0390 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb278005fd0 tx=0x7fb27802a8b0 comp rx=0 tx=0).stop 2024-09-21T07:47:09.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.893+0000 7fb285a00640 1 -- 172.21.15.39:0/2529791696 shutdown_connections 2024-09-21T07:47:09.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.893+0000 7fb285a00640 1 --2- 172.21.15.39:0/2529791696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2800cfc50 0x7fb2800d0390 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:09.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.893+0000 7fb285a00640 1 --2- 172.21.15.39:0/2529791696 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb280160660 0x7fb2800cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:09.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.893+0000 7fb285a00640 1 -- 172.21.15.39:0/2529791696 >> 172.21.15.39:0/2529791696 conn(0x7fb28006cfc0 msgr2=0x7fb28006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:09.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.893+0000 7fb285a00640 1 -- 172.21.15.39:0/2529791696 shutdown_connections 2024-09-21T07:47:09.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.893+0000 7fb285a00640 1 -- 172.21.15.39:0/2529791696 wait complete. 2024-09-21T07:47:09.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.895+0000 7fb285a00640 1 Processor -- start 2024-09-21T07:47:09.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.895+0000 7fb285a00640 1 -- start start 2024-09-21T07:47:09.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.897+0000 7fb285a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb2800cfc50 0x7fb280156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:09.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.897+0000 7fb285a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb280160250 0x7fb28015ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:09.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.897+0000 7fb285a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb280160660 0x7fb28015b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:09.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.897+0000 7fb285a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb2800ccdc0 con 0x7fb280160250 2024-09-21T07:47:09.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.897+0000 7fb285a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb2800ccc40 con 0x7fb280160660 2024-09-21T07:47:09.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.897+0000 7fb285a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb2800ccf40 con 0x7fb2800cfc50 2024-09-21T07:47:09.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.897+0000 7fb277e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb280160250 0x7fb28015ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:09.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.897+0000 7fb277e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb280160250 0x7fb28015ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60118/0 (socket says 172.21.15.39:60118) 2024-09-21T07:47:09.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.897+0000 7fb277e00640 1 -- 172.21.15.39:0/2118162625 learned_addr learned my addr 172.21.15.39:0/2118162625 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:09.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.897+0000 7fb285000640 1 --2- 172.21.15.39:0/2118162625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb280160660 0x7fb28015b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:09.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.897+0000 7fb27fe00640 1 --2- 172.21.15.39:0/2118162625 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb2800cfc50 0x7fb280156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:09.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.897+0000 7fb277e00640 1 -- 172.21.15.39:0/2118162625 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb2800cfc50 msgr2=0x7fb280156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:09.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.897+0000 7fb277e00640 1 --2- 172.21.15.39:0/2118162625 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb2800cfc50 0x7fb280156060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:09.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.897+0000 7fb277e00640 1 -- 172.21.15.39:0/2118162625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb280160660 msgr2=0x7fb28015b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:09.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.897+0000 7fb277e00640 1 --2- 172.21.15.39:0/2118162625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb280160660 0x7fb28015b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:09.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.897+0000 7fb277e00640 1 -- 172.21.15.39:0/2118162625 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2800af020 con 0x7fb280160250 2024-09-21T07:47:09.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.898+0000 7fb277e00640 1 --2- 172.21.15.39:0/2118162625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb280160250 0x7fb28015ad50 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fb26800e9f0 tx=0x7fb26800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:09.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.898+0000 7fb27cc00640 1 -- 172.21.15.39:0/2118162625 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb26800cd80 con 0x7fb280160250 2024-09-21T07:47:09.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.898+0000 7fb27cc00640 1 -- 172.21.15.39:0/2118162625 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb268004020 con 0x7fb280160250 2024-09-21T07:47:09.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.898+0000 7fb285a00640 1 -- 172.21.15.39:0/2118162625 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb28015a270 con 0x7fb280160250 2024-09-21T07:47:09.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.898+0000 7fb27cc00640 1 -- 172.21.15.39:0/2118162625 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb268004450 con 0x7fb280160250 2024-09-21T07:47:09.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.898+0000 7fb285a00640 1 -- 172.21.15.39:0/2118162625 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb28015a830 con 0x7fb280160250 2024-09-21T07:47:09.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.899+0000 7fb285a00640 1 -- 172.21.15.39:0/2118162625 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2800cd3f0 con 0x7fb280160250 2024-09-21T07:47:09.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.901+0000 7fb27cc00640 1 -- 172.21.15.39:0/2118162625 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb2680035f0 con 0x7fb280160250 2024-09-21T07:47:09.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.902+0000 7fb27cc00640 1 --2- 172.21.15.39:0/2118162625 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb25c078b90 0x7fb25c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:09.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.902+0000 7fb27cc00640 1 -- 172.21.15.39:0/2118162625 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(193..193 src has 1..193) ==== 6932+0+0 (secure 0 0 0) 0x7fb26809ba20 con 0x7fb280160250 2024-09-21T07:47:09.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.902+0000 7fb27fe00640 1 --2- 172.21.15.39:0/2118162625 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb25c078b90 0x7fb25c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:09.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.903+0000 7fb27fe00640 1 --2- 172.21.15.39:0/2118162625 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb25c078b90 0x7fb25c07b0a0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fb270000f60 tx=0x7fb270009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:09.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:09.907+0000 7fb27cc00640 1 -- 172.21.15.39:0/2118162625 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb268063330 con 0x7fb280160250 2024-09-21T07:47:10.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.104+0000 7fb285a00640 1 -- 172.21.15.39:0/2118162625 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb2800aedb0 con 0x7fb280160250 2024-09-21T07:47:10.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.106+0000 7fb27cc00640 1 -- 172.21.15.39:0/2118162625 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v193) ==== 74+0+22361 (secure 0 0 0) 0x7fb268067d60 con 0x7fb280160250 2024-09-21T07:47:10.109 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:47:10.109 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":193,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:47:09.140271+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:47:07.077866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"193","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-09-21T07:47:08.095979+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"193","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]},{"pool":20,"snaps":[{"begin":80,"length":23},{"begin":181,"length":23}]},{"pool":22,"snaps":[{"begin":100,"length":1},{"begin":199,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:47:10.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.109+0000 7fb285a00640 1 -- 172.21.15.39:0/2118162625 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb25c078b90 msgr2=0x7fb25c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:10.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.109+0000 7fb285a00640 1 --2- 172.21.15.39:0/2118162625 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb25c078b90 0x7fb25c07b0a0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fb270000f60 tx=0x7fb270009040 comp rx=0 tx=0).stop 2024-09-21T07:47:10.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.109+0000 7fb285a00640 1 -- 172.21.15.39:0/2118162625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb280160250 msgr2=0x7fb28015ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:10.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.109+0000 7fb285a00640 1 --2- 172.21.15.39:0/2118162625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb280160250 0x7fb28015ad50 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fb26800e9f0 tx=0x7fb26800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:47:10.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.109+0000 7fb285a00640 1 -- 172.21.15.39:0/2118162625 shutdown_connections 2024-09-21T07:47:10.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.109+0000 7fb285a00640 1 --2- 172.21.15.39:0/2118162625 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb25c078b90 0x7fb25c07b0a0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:10.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.109+0000 7fb285a00640 1 --2- 172.21.15.39:0/2118162625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb280160660 0x7fb28015b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:10.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.109+0000 7fb285a00640 1 --2- 172.21.15.39:0/2118162625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb280160250 0x7fb28015ad50 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:10.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.109+0000 7fb285a00640 1 --2- 172.21.15.39:0/2118162625 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb2800cfc50 0x7fb280156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:10.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.109+0000 7fb285a00640 1 -- 172.21.15.39:0/2118162625 >> 172.21.15.39:0/2118162625 conn(0x7fb28006cfc0 msgr2=0x7fb2800b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:10.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.109+0000 7fb285a00640 1 -- 172.21.15.39:0/2118162625 shutdown_connections 2024-09-21T07:47:10.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.109+0000 7fb285a00640 1 -- 172.21.15.39:0/2118162625 wait complete. 2024-09-21T07:47:10.134 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:47:10.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.358+0000 7f9dfba00640 1 -- 172.21.15.39:0/2591570509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9df406ef90 msgr2=0x7f9df406f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:10.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.358+0000 7f9dfba00640 1 --2- 172.21.15.39:0/2591570509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9df406ef90 0x7f9df406f3a0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f9dec005fd0 tx=0x7f9dec02aaa0 comp rx=0 tx=0).stop 2024-09-21T07:47:10.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.358+0000 7f9dfba00640 1 -- 172.21.15.39:0/2591570509 shutdown_connections 2024-09-21T07:47:10.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.358+0000 7f9dfba00640 1 --2- 172.21.15.39:0/2591570509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9df406ef90 0x7f9df406f3a0 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:10.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.358+0000 7f9dfba00640 1 --2- 172.21.15.39:0/2591570509 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9df40b4c40 0x7f9df40ce2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:10.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.358+0000 7f9dfba00640 1 -- 172.21.15.39:0/2591570509 >> 172.21.15.39:0/2591570509 conn(0x7f9df40b2460 msgr2=0x7f9df406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:10.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.359+0000 7f9dfba00640 1 -- 172.21.15.39:0/2591570509 shutdown_connections 2024-09-21T07:47:10.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.359+0000 7f9dfba00640 1 -- 172.21.15.39:0/2591570509 wait complete. 2024-09-21T07:47:10.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.361+0000 7f9dfba00640 1 Processor -- start 2024-09-21T07:47:10.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.361+0000 7f9dfba00640 1 -- start start 2024-09-21T07:47:10.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.362+0000 7f9dfba00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9df406ef90 0x7f9df41f8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:10.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.362+0000 7f9dfba00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9df40b3670 0x7f9df4154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:10.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.362+0000 7f9dfba00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9df40b4c40 0x7f9df41546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:10.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.362+0000 7f9dfba00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9df406fa60 con 0x7f9df406ef90 2024-09-21T07:47:10.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.362+0000 7f9dfba00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9df406f8e0 con 0x7f9df40b3670 2024-09-21T07:47:10.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.362+0000 7f9dfba00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f9df406fbe0 con 0x7f9df40b4c40 2024-09-21T07:47:10.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.363+0000 7f9df9000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9df406ef90 0x7f9df41f8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:10.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.363+0000 7f9df9000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9df406ef90 0x7f9df41f8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60146/0 (socket says 172.21.15.39:60146) 2024-09-21T07:47:10.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.363+0000 7f9df9000640 1 -- 172.21.15.39:0/284397088 learned_addr learned my addr 172.21.15.39:0/284397088 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:10.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.363+0000 7f9df3e00640 1 --2- 172.21.15.39:0/284397088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9df40b3670 0x7f9df4154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:10.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.363+0000 7f9df9a00640 1 --2- 172.21.15.39:0/284397088 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9df40b4c40 0x7f9df41546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:10.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.363+0000 7f9df9000640 1 -- 172.21.15.39:0/284397088 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9df40b4c40 msgr2=0x7f9df41546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:10.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.363+0000 7f9df9000640 1 --2- 172.21.15.39:0/284397088 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9df40b4c40 0x7f9df41546a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:10.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.363+0000 7f9df9000640 1 -- 172.21.15.39:0/284397088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9df40b3670 msgr2=0x7f9df4154160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:10.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.363+0000 7f9df9000640 1 --2- 172.21.15.39:0/284397088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9df40b3670 0x7f9df4154160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:10.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.363+0000 7f9df9000640 1 -- 172.21.15.39:0/284397088 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9df40c9e00 con 0x7f9df406ef90 2024-09-21T07:47:10.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.363+0000 7f9df9a00640 1 --2- 172.21.15.39:0/284397088 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9df40b4c40 0x7f9df41546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:10.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.363+0000 7f9df3e00640 1 --2- 172.21.15.39:0/284397088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9df40b3670 0x7f9df4154160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:10.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.363+0000 7f9df9000640 1 --2- 172.21.15.39:0/284397088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9df406ef90 0x7f9df41f8a60 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f9de800d700 tx=0x7f9de800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:10.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.364+0000 7f9df0c00640 1 -- 172.21.15.39:0/284397088 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de8017070 con 0x7f9df406ef90 2024-09-21T07:47:10.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.364+0000 7f9df0c00640 1 -- 172.21.15.39:0/284397088 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9de8005310 con 0x7f9df406ef90 2024-09-21T07:47:10.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.364+0000 7f9df0c00640 1 -- 172.21.15.39:0/284397088 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de8009820 con 0x7f9df406ef90 2024-09-21T07:47:10.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.364+0000 7f9dfba00640 1 -- 172.21.15.39:0/284397088 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9df40ad220 con 0x7f9df406ef90 2024-09-21T07:47:10.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.364+0000 7f9dfba00640 1 -- 172.21.15.39:0/284397088 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9df41f8fa0 con 0x7f9df406ef90 2024-09-21T07:47:10.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.365+0000 7f9dfba00640 1 -- 172.21.15.39:0/284397088 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9df406f3a0 con 0x7f9df406ef90 2024-09-21T07:47:10.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.366+0000 7f9df0c00640 1 -- 172.21.15.39:0/284397088 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9de80099c0 con 0x7f9df406ef90 2024-09-21T07:47:10.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.367+0000 7f9df0c00640 1 --2- 172.21.15.39:0/284397088 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9dd4078b90 0x7f9dd407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:10.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.367+0000 7f9df0c00640 1 -- 172.21.15.39:0/284397088 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(193..193 src has 1..193) ==== 6932+0+0 (secure 0 0 0) 0x7f9de809b360 con 0x7f9df406ef90 2024-09-21T07:47:10.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.368+0000 7f9df3e00640 1 --2- 172.21.15.39:0/284397088 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9dd4078b90 0x7f9dd407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:10.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.369+0000 7f9df3e00640 1 --2- 172.21.15.39:0/284397088 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9dd4078b90 0x7f9dd407b0a0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f9df40cb5c0 tx=0x7f9de0009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:10.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.372+0000 7f9df0c00640 1 -- 172.21.15.39:0/284397088 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9de80632c0 con 0x7f9df406ef90 2024-09-21T07:47:10.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.570+0000 7f9dfba00640 1 -- 172.21.15.39:0/284397088 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9df40b3130 con 0x7f9df406ef90 2024-09-21T07:47:10.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.571+0000 7f9df0c00640 1 -- 172.21.15.39:0/284397088 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v193) ==== 74+0+22361 (secure 0 0 0) 0x7f9de8062bc0 con 0x7f9df406ef90 2024-09-21T07:47:10.574 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:47:10.574 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":193,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:47:09.140271+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:47:07.077866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"193","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-09-21T07:47:08.095979+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"193","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]},{"pool":20,"snaps":[{"begin":80,"length":23},{"begin":181,"length":23}]},{"pool":22,"snaps":[{"begin":100,"length":1},{"begin":199,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:47:10.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.574+0000 7f9dfba00640 1 -- 172.21.15.39:0/284397088 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9dd4078b90 msgr2=0x7f9dd407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:10.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.574+0000 7f9dfba00640 1 --2- 172.21.15.39:0/284397088 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9dd4078b90 0x7f9dd407b0a0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f9df40cb5c0 tx=0x7f9de0009040 comp rx=0 tx=0).stop 2024-09-21T07:47:10.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.574+0000 7f9dfba00640 1 -- 172.21.15.39:0/284397088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9df406ef90 msgr2=0x7f9df41f8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:10.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.574+0000 7f9dfba00640 1 --2- 172.21.15.39:0/284397088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9df406ef90 0x7f9df41f8a60 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f9de800d700 tx=0x7f9de800dbc0 comp rx=0 tx=0).stop 2024-09-21T07:47:10.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.574+0000 7f9dfba00640 1 -- 172.21.15.39:0/284397088 shutdown_connections 2024-09-21T07:47:10.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.574+0000 7f9dfba00640 1 --2- 172.21.15.39:0/284397088 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9dd4078b90 0x7f9dd407b0a0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:10.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.574+0000 7f9dfba00640 1 --2- 172.21.15.39:0/284397088 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9df40b4c40 0x7f9df41546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:10.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.574+0000 7f9dfba00640 1 --2- 172.21.15.39:0/284397088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9df40b3670 0x7f9df4154160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:10.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.574+0000 7f9dfba00640 1 --2- 172.21.15.39:0/284397088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9df406ef90 0x7f9df41f8a60 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:10.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.574+0000 7f9dfba00640 1 -- 172.21.15.39:0/284397088 >> 172.21.15.39:0/284397088 conn(0x7f9df40b2460 msgr2=0x7f9df40af8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:10.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.574+0000 7f9dfba00640 1 -- 172.21.15.39:0/284397088 shutdown_connections 2024-09-21T07:47:10.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.574+0000 7f9dfba00640 1 -- 172.21.15.39:0/284397088 wait complete. 2024-09-21T07:47:10.598 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T07:47:10.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.827+0000 7f1c07800640 1 -- 172.21.15.39:0/4198812897 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c000b30a0 msgr2=0x7f1c000cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:10.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.827+0000 7f1c07800640 1 --2- 172.21.15.39:0/4198812897 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c000b30a0 0x7f1c000cb8a0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f1bf4009830 tx=0x7f1bf402a530 comp rx=0 tx=0).stop 2024-09-21T07:47:10.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.827+0000 7f1c07800640 1 -- 172.21.15.39:0/4198812897 shutdown_connections 2024-09-21T07:47:10.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.827+0000 7f1c07800640 1 --2- 172.21.15.39:0/4198812897 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c000b30a0 0x7f1c000cb8a0 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:10.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.827+0000 7f1c07800640 1 -- 172.21.15.39:0/4198812897 >> 172.21.15.39:0/4198812897 conn(0x7f1c000b2460 msgr2=0x7f1c0006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:10.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.827+0000 7f1c07800640 1 -- 172.21.15.39:0/4198812897 shutdown_connections 2024-09-21T07:47:10.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.827+0000 7f1c07800640 1 -- 172.21.15.39:0/4198812897 wait complete. 2024-09-21T07:47:10.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.829+0000 7f1c07800640 1 Processor -- start 2024-09-21T07:47:10.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.829+0000 7f1c07800640 1 -- start start 2024-09-21T07:47:10.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.831+0000 7f1c07800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c000b30a0 0x7f1c00073f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:10.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.831+0000 7f1c07800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c000ccc60 0x7f1c000cf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:10.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.831+0000 7f1c07800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1c0015d830 0x7f1c00155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:10.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.831+0000 7f1c07800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1c000af520 con 0x7f1c000b30a0 2024-09-21T07:47:10.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.831+0000 7f1c07800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1c000af3a0 con 0x7f1c000ccc60 2024-09-21T07:47:10.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.831+0000 7f1c07800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f1c000af6a0 con 0x7f1c0015d830 2024-09-21T07:47:10.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.831+0000 7f1c04e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c000b30a0 0x7f1c00073f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:10.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.831+0000 7f1c04e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c000b30a0 0x7f1c00073f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60164/0 (socket says 172.21.15.39:60164) 2024-09-21T07:47:10.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.831+0000 7f1c04e00640 1 -- 172.21.15.39:0/3403994826 learned_addr learned my addr 172.21.15.39:0/3403994826 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:10.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.831+0000 7f1bffe00640 1 --2- 172.21.15.39:0/3403994826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c000ccc60 0x7f1c000cf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:10.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.831+0000 7f1c05800640 1 --2- 172.21.15.39:0/3403994826 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1c0015d830 0x7f1c00155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:10.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.831+0000 7f1c04e00640 1 -- 172.21.15.39:0/3403994826 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1c0015d830 msgr2=0x7f1c00155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:10.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.831+0000 7f1c04e00640 1 --2- 172.21.15.39:0/3403994826 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1c0015d830 0x7f1c00155e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:10.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.831+0000 7f1c04e00640 1 -- 172.21.15.39:0/3403994826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c000ccc60 msgr2=0x7f1c000cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:10.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.831+0000 7f1c04e00640 1 --2- 172.21.15.39:0/3403994826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c000ccc60 0x7f1c000cf1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:10.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.831+0000 7f1c04e00640 1 -- 172.21.15.39:0/3403994826 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c001f5820 con 0x7f1c000b30a0 2024-09-21T07:47:10.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.832+0000 7f1c04e00640 1 --2- 172.21.15.39:0/3403994826 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c000b30a0 0x7f1c00073f20 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f1bf4000c00 tx=0x7f1bf4033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:10.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.832+0000 7f1bfcc00640 1 -- 172.21.15.39:0/3403994826 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf4031d60 con 0x7f1c000b30a0 2024-09-21T07:47:10.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.832+0000 7f1bfcc00640 1 -- 172.21.15.39:0/3403994826 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1bf40330d0 con 0x7f1c000b30a0 2024-09-21T07:47:10.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.832+0000 7f1bfcc00640 1 -- 172.21.15.39:0/3403994826 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf4033520 con 0x7f1c000b30a0 2024-09-21T07:47:10.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.832+0000 7f1c07800640 1 -- 172.21.15.39:0/3403994826 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c001f5b00 con 0x7f1c000b30a0 2024-09-21T07:47:10.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.832+0000 7f1c07800640 1 -- 172.21.15.39:0/3403994826 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c001f60c0 con 0x7f1c000b30a0 2024-09-21T07:47:10.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.833+0000 7f1c07800640 1 -- 172.21.15.39:0/3403994826 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c000d1160 con 0x7f1c000b30a0 2024-09-21T07:47:10.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.834+0000 7f1bfcc00640 1 -- 172.21.15.39:0/3403994826 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1bf40035f0 con 0x7f1c000b30a0 2024-09-21T07:47:10.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.835+0000 7f1bfcc00640 1 --2- 172.21.15.39:0/3403994826 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1be0078bc0 0x7f1be007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:10.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.836+0000 7f1bfcc00640 1 -- 172.21.15.39:0/3403994826 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 6932+0+0 (secure 0 0 0) 0x7f1bf40bc790 con 0x7f1c000b30a0 2024-09-21T07:47:10.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.836+0000 7f1bffe00640 1 --2- 172.21.15.39:0/3403994826 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1be0078bc0 0x7f1be007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:10.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.837+0000 7f1bffe00640 1 --2- 172.21.15.39:0/3403994826 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1be0078bc0 0x7f1be007b0d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f1c00073eb0 tx=0x7f1bec009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:10.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:10.840+0000 7f1bfcc00640 1 -- 172.21.15.39:0/3403994826 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1bf403f030 con 0x7f1c000b30a0 2024-09-21T07:47:11.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.080+0000 7f1c07800640 1 -- 172.21.15.39:0/3403994826 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f1c000cb8a0 con 0x7f1c000b30a0 2024-09-21T07:47:11.584 INFO:tasks.ceph.mds.c.smithi039.stderr:2024-09-21T07:47:11.582+0000 7fab94200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:47:11.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.640+0000 7f1bfcc00640 1 -- 172.21.15.39:0/3403994826 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v198) ==== 118+0+0 (secure 0 0 0) 0x7f1bf40840a0 con 0x7f1c000b30a0 2024-09-21T07:47:11.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.644+0000 7f1c07800640 1 -- 172.21.15.39:0/3403994826 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1be0078bc0 msgr2=0x7f1be007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:11.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.644+0000 7f1c07800640 1 --2- 172.21.15.39:0/3403994826 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1be0078bc0 0x7f1be007b0d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f1c00073eb0 tx=0x7f1bec009040 comp rx=0 tx=0).stop 2024-09-21T07:47:11.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.644+0000 7f1c07800640 1 -- 172.21.15.39:0/3403994826 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c000b30a0 msgr2=0x7f1c00073f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:11.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.644+0000 7f1c07800640 1 --2- 172.21.15.39:0/3403994826 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c000b30a0 0x7f1c00073f20 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f1bf4000c00 tx=0x7f1bf4033d60 comp rx=0 tx=0).stop 2024-09-21T07:47:11.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.644+0000 7f1c07800640 1 -- 172.21.15.39:0/3403994826 shutdown_connections 2024-09-21T07:47:11.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.644+0000 7f1c07800640 1 --2- 172.21.15.39:0/3403994826 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1be0078bc0 0x7f1be007b0d0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:11.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.644+0000 7f1c07800640 1 --2- 172.21.15.39:0/3403994826 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1c0015d830 0x7f1c00155e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:11.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.644+0000 7f1c07800640 1 --2- 172.21.15.39:0/3403994826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c000ccc60 0x7f1c000cf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:11.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.644+0000 7f1c07800640 1 --2- 172.21.15.39:0/3403994826 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c000b30a0 0x7f1c00073f20 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:11.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.644+0000 7f1c07800640 1 -- 172.21.15.39:0/3403994826 >> 172.21.15.39:0/3403994826 conn(0x7f1c000b2460 msgr2=0x7f1c000b4910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:11.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.644+0000 7f1c07800640 1 -- 172.21.15.39:0/3403994826 shutdown_connections 2024-09-21T07:47:11.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.644+0000 7f1c07800640 1 -- 172.21.15.39:0/3403994826 wait complete. 2024-09-21T07:47:11.672 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T07:47:11.672 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T07:47:11.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.902+0000 7f009ee00640 1 -- 172.21.15.39:0/1332470581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f007c005250 msgr2=0x7f007c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:11.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.902+0000 7f009ee00640 1 --2- 172.21.15.39:0/1332470581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f007c005250 0x7f007c00b9c0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f0090005fd0 tx=0x7f007c005830 comp rx=0 tx=0).stop 2024-09-21T07:47:11.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.902+0000 7f009ee00640 1 -- 172.21.15.39:0/1332470581 shutdown_connections 2024-09-21T07:47:11.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.902+0000 7f009ee00640 1 --2- 172.21.15.39:0/1332470581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f007c005250 0x7f007c00b9c0 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:11.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.902+0000 7f009ee00640 1 -- 172.21.15.39:0/1332470581 >> 172.21.15.39:0/1332470581 conn(0x7f009806cfc0 msgr2=0x7f009806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:11.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.902+0000 7f009ee00640 1 -- 172.21.15.39:0/1332470581 shutdown_connections 2024-09-21T07:47:11.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.902+0000 7f009ee00640 1 -- 172.21.15.39:0/1332470581 wait complete. 2024-09-21T07:47:11.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.904+0000 7f009ee00640 1 Processor -- start 2024-09-21T07:47:11.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.905+0000 7f009ee00640 1 -- start start 2024-09-21T07:47:11.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.906+0000 7f009ee00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f007c005250 0x7f009814e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:11.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.906+0000 7f009ee00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00981f8c10 0x7f009806f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:11.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.906+0000 7f009ee00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00981f9020 0x7f009806ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:11.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.906+0000 7f009ee00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f00980734c0 con 0x7f00981f8c10 2024-09-21T07:47:11.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.906+0000 7f009ee00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0098073340 con 0x7f00981f9020 2024-09-21T07:47:11.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.906+0000 7f009ee00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f0098073640 con 0x7f007c005250 2024-09-21T07:47:11.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.906+0000 7f008fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00981f8c10 0x7f009806f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:11.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.906+0000 7f008fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00981f8c10 0x7f009806f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60196/0 (socket says 172.21.15.39:60196) 2024-09-21T07:47:11.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.906+0000 7f008fe00640 1 -- 172.21.15.39:0/3017412432 learned_addr learned my addr 172.21.15.39:0/3017412432 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:11.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.907+0000 7f0097e00640 1 --2- 172.21.15.39:0/3017412432 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f007c005250 0x7f009814e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:11.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.907+0000 7f009ce00640 1 --2- 172.21.15.39:0/3017412432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00981f9020 0x7f009806ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:11.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.907+0000 7f008fe00640 1 -- 172.21.15.39:0/3017412432 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f007c005250 msgr2=0x7f009814e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:11.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.907+0000 7f008fe00640 1 --2- 172.21.15.39:0/3017412432 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f007c005250 0x7f009814e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:11.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.907+0000 7f008fe00640 1 -- 172.21.15.39:0/3017412432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00981f9020 msgr2=0x7f009806ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:11.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.907+0000 7f008fe00640 1 --2- 172.21.15.39:0/3017412432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00981f9020 0x7f009806ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:11.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.907+0000 7f008fe00640 1 -- 172.21.15.39:0/3017412432 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00981f97b0 con 0x7f00981f8c10 2024-09-21T07:47:11.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.907+0000 7f009ce00640 1 --2- 172.21.15.39:0/3017412432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00981f9020 0x7f009806ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:47:11.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.907+0000 7f008fe00640 1 --2- 172.21.15.39:0/3017412432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00981f8c10 0x7f009806f9e0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f0080003e90 tx=0x7f0080001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:11.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.907+0000 7f0094c00640 1 -- 172.21.15.39:0/3017412432 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0080015070 con 0x7f00981f8c10 2024-09-21T07:47:11.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.907+0000 7f0094c00640 1 -- 172.21.15.39:0/3017412432 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0080011840 con 0x7f00981f8c10 2024-09-21T07:47:11.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.907+0000 7f009ee00640 1 -- 172.21.15.39:0/3017412432 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00981f61a0 con 0x7f00981f8c10 2024-09-21T07:47:11.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.907+0000 7f0094c00640 1 -- 172.21.15.39:0/3017412432 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0080011c90 con 0x7f00981f8c10 2024-09-21T07:47:11.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.908+0000 7f009ee00640 1 -- 172.21.15.39:0/3017412432 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00981f56c0 con 0x7f00981f8c10 2024-09-21T07:47:11.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.909+0000 7f009ee00640 1 -- 172.21.15.39:0/3017412432 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00980d06d0 con 0x7f00981f8c10 2024-09-21T07:47:11.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.910+0000 7f0094c00640 1 -- 172.21.15.39:0/3017412432 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0080018070 con 0x7f00981f8c10 2024-09-21T07:47:11.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.911+0000 7f0094c00640 1 --2- 172.21.15.39:0/3017412432 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0074078b90 0x7f007407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:11.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.911+0000 7f0094c00640 1 -- 172.21.15.39:0/3017412432 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 6932+0+0 (secure 0 0 0) 0x7f008009bfb0 con 0x7f00981f8c10 2024-09-21T07:47:11.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.911+0000 7f0097e00640 1 --2- 172.21.15.39:0/3017412432 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0074078b90 0x7f007407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:11.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.912+0000 7f0097e00640 1 --2- 172.21.15.39:0/3017412432 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0074078b90 0x7f007407b0a0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f0088000fb0 tx=0x7f00880067a0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:11.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:11.915+0000 7f0094c00640 1 -- 172.21.15.39:0/3017412432 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0080063e00 con 0x7f00981f8c10 2024-09-21T07:47:12.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.152+0000 7f009ee00640 1 -- 172.21.15.39:0/3017412432 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f0098153930 con 0x7f00981f8c10 2024-09-21T07:47:12.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.652+0000 7f0094c00640 1 -- 172.21.15.39:0/3017412432 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v199) ==== 122+0+0 (secure 0 0 0) 0x7f0080063700 con 0x7f00981f8c10 2024-09-21T07:47:12.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.654+0000 7f009ee00640 1 -- 172.21.15.39:0/3017412432 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0074078b90 msgr2=0x7f007407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:12.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.654+0000 7f009ee00640 1 --2- 172.21.15.39:0/3017412432 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0074078b90 0x7f007407b0a0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f0088000fb0 tx=0x7f00880067a0 comp rx=0 tx=0).stop 2024-09-21T07:47:12.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.654+0000 7f009ee00640 1 -- 172.21.15.39:0/3017412432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00981f8c10 msgr2=0x7f009806f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:12.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.654+0000 7f009ee00640 1 --2- 172.21.15.39:0/3017412432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00981f8c10 0x7f009806f9e0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f0080003e90 tx=0x7f0080001630 comp rx=0 tx=0).stop 2024-09-21T07:47:12.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.654+0000 7f009ee00640 1 -- 172.21.15.39:0/3017412432 shutdown_connections 2024-09-21T07:47:12.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.654+0000 7f009ee00640 1 --2- 172.21.15.39:0/3017412432 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0074078b90 0x7f007407b0a0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:12.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.654+0000 7f009ee00640 1 --2- 172.21.15.39:0/3017412432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00981f9020 0x7f009806ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:12.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.654+0000 7f009ee00640 1 --2- 172.21.15.39:0/3017412432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00981f8c10 0x7f009806f9e0 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:12.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.654+0000 7f009ee00640 1 --2- 172.21.15.39:0/3017412432 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f007c005250 0x7f009814e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:12.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.654+0000 7f009ee00640 1 -- 172.21.15.39:0/3017412432 >> 172.21.15.39:0/3017412432 conn(0x7f009806cfc0 msgr2=0x7f00980b0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:12.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.654+0000 7f009ee00640 1 -- 172.21.15.39:0/3017412432 shutdown_connections 2024-09-21T07:47:12.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.655+0000 7f009ee00640 1 -- 172.21.15.39:0/3017412432 wait complete. 2024-09-21T07:47:12.679 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T07:47:12.679 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:47:12.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.904+0000 7f3695e00640 1 -- 172.21.15.39:0/164419344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36900b30a0 msgr2=0x7f36900cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:12.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.904+0000 7f3695e00640 1 --2- 172.21.15.39:0/164419344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36900b30a0 0x7f36900cb8a0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f36900cbde0 tx=0x7f368002a4d0 comp rx=0 tx=0).stop 2024-09-21T07:47:12.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.904+0000 7f3695e00640 1 -- 172.21.15.39:0/164419344 shutdown_connections 2024-09-21T07:47:12.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.904+0000 7f3695e00640 1 --2- 172.21.15.39:0/164419344 >> v2:172.21.15.148:3300/0 conn(0x7f36900ccc60 0x7f36900cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:12.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.904+0000 7f3695e00640 1 --2- 172.21.15.39:0/164419344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36900b30a0 0x7f36900cb8a0 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:12.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.904+0000 7f3695e00640 1 -- 172.21.15.39:0/164419344 >> 172.21.15.39:0/164419344 conn(0x7f36900b2460 msgr2=0x7f369006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:12.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.904+0000 7f3695e00640 1 -- 172.21.15.39:0/164419344 shutdown_connections 2024-09-21T07:47:12.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.904+0000 7f3695e00640 1 -- 172.21.15.39:0/164419344 wait complete. 2024-09-21T07:47:12.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.907+0000 7f3695e00640 1 Processor -- start 2024-09-21T07:47:12.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.907+0000 7f3695e00640 1 -- start start 2024-09-21T07:47:12.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.908+0000 7f3695e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36900b30a0 0x7f36900c5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:12.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.908+0000 7f3695e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36900ccc60 0x7f36900c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:12.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.908+0000 7f3695e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f369015d830 0x7f36901f5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:12.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.908+0000 7f3695e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f36900af410 con 0x7f36900b30a0 2024-09-21T07:47:12.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.908+0000 7f3695e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f36900af290 con 0x7f36900ccc60 2024-09-21T07:47:12.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.908+0000 7f3695e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f36900af590 con 0x7f369015d830 2024-09-21T07:47:12.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.908+0000 7f368ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36900b30a0 0x7f36900c5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:12.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.909+0000 7f368ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36900b30a0 0x7f36900c5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48304/0 (socket says 172.21.15.39:48304) 2024-09-21T07:47:12.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.909+0000 7f368ea00640 1 -- 172.21.15.39:0/1760381457 learned_addr learned my addr 172.21.15.39:0/1760381457 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:12.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.909+0000 7f368f400640 1 --2- 172.21.15.39:0/1760381457 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f369015d830 0x7f36901f5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:12.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.909+0000 7f368e000640 1 --2- 172.21.15.39:0/1760381457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36900ccc60 0x7f36900c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:12.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.909+0000 7f368ea00640 1 -- 172.21.15.39:0/1760381457 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f369015d830 msgr2=0x7f36901f5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:12.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.909+0000 7f368ea00640 1 --2- 172.21.15.39:0/1760381457 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f369015d830 0x7f36901f5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:12.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.909+0000 7f368ea00640 1 -- 172.21.15.39:0/1760381457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36900ccc60 msgr2=0x7f36900c60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:12.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.909+0000 7f368ea00640 1 --2- 172.21.15.39:0/1760381457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36900ccc60 0x7f36900c60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:12.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.909+0000 7f368ea00640 1 -- 172.21.15.39:0/1760381457 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36900c40f0 con 0x7f36900b30a0 2024-09-21T07:47:12.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.909+0000 7f368ea00640 1 --2- 172.21.15.39:0/1760381457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36900b30a0 0x7f36900c5b60 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f3680034040 tx=0x7f3680003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:12.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.910+0000 7f3687400640 1 -- 172.21.15.39:0/1760381457 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368003e030 con 0x7f36900b30a0 2024-09-21T07:47:12.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.910+0000 7f3687400640 1 -- 172.21.15.39:0/1760381457 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3680003e00 con 0x7f36900b30a0 2024-09-21T07:47:12.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.910+0000 7f3687400640 1 -- 172.21.15.39:0/1760381457 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3680035040 con 0x7f36900b30a0 2024-09-21T07:47:12.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.910+0000 7f3695e00640 1 -- 172.21.15.39:0/1760381457 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36900c43d0 con 0x7f36900b30a0 2024-09-21T07:47:12.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.910+0000 7f3695e00640 1 -- 172.21.15.39:0/1760381457 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36900c4990 con 0x7f36900b30a0 2024-09-21T07:47:12.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.911+0000 7f3695e00640 1 -- 172.21.15.39:0/1760381457 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36900afa40 con 0x7f36900b30a0 2024-09-21T07:47:12.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.912+0000 7f3687400640 1 -- 172.21.15.39:0/1760381457 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3680041030 con 0x7f36900b30a0 2024-09-21T07:47:12.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.913+0000 7f3687400640 1 --2- 172.21.15.39:0/1760381457 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3670078b90 0x7f367007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:12.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.913+0000 7f3687400640 1 -- 172.21.15.39:0/1760381457 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 6932+0+0 (secure 0 0 0) 0x7f36800bba70 con 0x7f36900b30a0 2024-09-21T07:47:12.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.913+0000 7f368e000640 1 --2- 172.21.15.39:0/1760381457 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3670078b90 0x7f367007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:12.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.914+0000 7f368e000640 1 --2- 172.21.15.39:0/1760381457 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3670078b90 0x7f367007b0a0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f367c006fd0 tx=0x7f367c008040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:12.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:12.917+0000 7f3687400640 1 -- 172.21.15.39:0/1760381457 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f36900cd5b0 con 0x7f36900b30a0 2024-09-21T07:47:13.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.151+0000 7f3695e00640 1 -- 172.21.15.39:0/1760381457 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f369015d280 con 0x7f36900b30a0 2024-09-21T07:47:13.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.152+0000 7f3687400640 1 -- 172.21.15.39:0/1760381457 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 199 v199) ==== 95+0+5230 (secure 0 0 0) 0x7f36800839e0 con 0x7f36900b30a0 2024-09-21T07:47:13.154 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:47:13.154 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":199,"btime":"2024-09-21T07:47:12:642015+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7715,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3057062733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3057062733},{"type":"v1","addr":"172.21.15.148:6836","nonce":3057062733}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":194},{"gid":8689,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/8652512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":8652512},{"type":"v1","addr":"172.21.15.148:6837","nonce":8652512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":194},{"gid":9483,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/3199732683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3199732683},{"type":"v1","addr":"172.21.15.39:6833","nonce":3199732683}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":193}],"filesystems":[{"mdsmap":{"epoch":199,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:47:09.140224+0000","modified":"2024-09-21T07:47:12.153995+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9492},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9492":{"gid":9492,"name":"c","rank":0,"incarnation":195,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/3348864735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3348864735},{"type":"v1","addr":"172.21.15.39:6835","nonce":3348864735}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9492,"qdb_cluster":[9492]},"id":10}]} 2024-09-21T07:47:13.154 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 199 2024-09-21T07:47:13.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.154+0000 7f3695e00640 1 -- 172.21.15.39:0/1760381457 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3670078b90 msgr2=0x7f367007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:13.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.154+0000 7f3695e00640 1 --2- 172.21.15.39:0/1760381457 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3670078b90 0x7f367007b0a0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f367c006fd0 tx=0x7f367c008040 comp rx=0 tx=0).stop 2024-09-21T07:47:13.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.154+0000 7f3695e00640 1 -- 172.21.15.39:0/1760381457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36900b30a0 msgr2=0x7f36900c5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:13.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.154+0000 7f3695e00640 1 --2- 172.21.15.39:0/1760381457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36900b30a0 0x7f36900c5b60 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f3680034040 tx=0x7f3680003910 comp rx=0 tx=0).stop 2024-09-21T07:47:13.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.154+0000 7f3695e00640 1 -- 172.21.15.39:0/1760381457 shutdown_connections 2024-09-21T07:47:13.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.154+0000 7f3695e00640 1 --2- 172.21.15.39:0/1760381457 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3670078b90 0x7f367007b0a0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:13.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.154+0000 7f3695e00640 1 --2- 172.21.15.39:0/1760381457 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f369015d830 0x7f36901f5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:13.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.154+0000 7f3695e00640 1 --2- 172.21.15.39:0/1760381457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36900ccc60 0x7f36900c60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:13.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.154+0000 7f3695e00640 1 --2- 172.21.15.39:0/1760381457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36900b30a0 0x7f36900c5b60 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:13.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.154+0000 7f3695e00640 1 -- 172.21.15.39:0/1760381457 >> 172.21.15.39:0/1760381457 conn(0x7f36900b2460 msgr2=0x7f3690156bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:13.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.154+0000 7f3695e00640 1 -- 172.21.15.39:0/1760381457 shutdown_connections 2024-09-21T07:47:13.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.154+0000 7f3695e00640 1 -- 172.21.15.39:0/1760381457 wait complete. 2024-09-21T07:47:13.179 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:47:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.404+0000 7f2491600640 1 -- 172.21.15.39:0/2948660818 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f248c0d0b70 msgr2=0x7f248c0b43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.404+0000 7f2491600640 1 --2- 172.21.15.39:0/2948660818 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f248c0d0b70 0x7f248c0b43d0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f248c06dd90 tx=0x7f248402aa30 comp rx=0 tx=0).stop 2024-09-21T07:47:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.405+0000 7f2491600640 1 -- 172.21.15.39:0/2948660818 shutdown_connections 2024-09-21T07:47:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.405+0000 7f2491600640 1 --2- 172.21.15.39:0/2948660818 >> v2:172.21.15.148:3300/0 conn(0x7f248c15c690 0x7f248c0d0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.405+0000 7f2491600640 1 --2- 172.21.15.39:0/2948660818 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f248c0d0b70 0x7f248c0b43d0 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.405+0000 7f2491600640 1 -- 172.21.15.39:0/2948660818 >> 172.21.15.39:0/2948660818 conn(0x7f248c06cfc0 msgr2=0x7f248c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.405+0000 7f2491600640 1 -- 172.21.15.39:0/2948660818 shutdown_connections 2024-09-21T07:47:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.405+0000 7f2491600640 1 -- 172.21.15.39:0/2948660818 wait complete. 2024-09-21T07:47:13.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.407+0000 7f2491600640 1 Processor -- start 2024-09-21T07:47:13.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.407+0000 7f2491600640 1 -- start start 2024-09-21T07:47:13.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.408+0000 7f2491600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f248c0b5240 0x7f248c149620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:13.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.408+0000 7f2491600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f248c0d0b70 0x7f248c0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:13.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.408+0000 7f2491600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f248c15c690 0x7f248c0cb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:13.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.408+0000 7f2491600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f248c070090 con 0x7f248c15c690 2024-09-21T07:47:13.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.408+0000 7f2491600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f248c06ff10 con 0x7f248c0d0b70 2024-09-21T07:47:13.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.408+0000 7f2491600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f248c0b4910 con 0x7f248c0b5240 2024-09-21T07:47:13.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.409+0000 7f248b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f248c15c690 0x7f248c0cb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:13.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.409+0000 7f248b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f248c15c690 0x7f248c0cb3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48322/0 (socket says 172.21.15.39:48322) 2024-09-21T07:47:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.409+0000 7f248b400640 1 -- 172.21.15.39:0/2447106971 learned_addr learned my addr 172.21.15.39:0/2447106971 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.409+0000 7f248aa00640 1 --2- 172.21.15.39:0/2447106971 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f248c0b5240 0x7f248c149620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.409+0000 7f248a000640 1 --2- 172.21.15.39:0/2447106971 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f248c0d0b70 0x7f248c0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.409+0000 7f248b400640 1 -- 172.21.15.39:0/2447106971 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f248c0b5240 msgr2=0x7f248c149620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.409+0000 7f248b400640 1 --2- 172.21.15.39:0/2447106971 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f248c0b5240 0x7f248c149620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.409+0000 7f248b400640 1 -- 172.21.15.39:0/2447106971 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f248c0d0b70 msgr2=0x7f248c0c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.409+0000 7f248b400640 1 --2- 172.21.15.39:0/2447106971 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f248c0d0b70 0x7f248c0c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.409+0000 7f248b400640 1 -- 172.21.15.39:0/2447106971 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f248c151890 con 0x7f248c15c690 2024-09-21T07:47:13.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.410+0000 7f248b400640 1 --2- 172.21.15.39:0/2447106971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f248c15c690 0x7f248c0cb3d0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f248402b020 tx=0x7f2484003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:13.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.410+0000 7f2482a00640 1 -- 172.21.15.39:0/2447106971 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2484033050 con 0x7f248c15c690 2024-09-21T07:47:13.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.410+0000 7f2482a00640 1 -- 172.21.15.39:0/2447106971 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2484004450 con 0x7f248c15c690 2024-09-21T07:47:13.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.410+0000 7f2482a00640 1 -- 172.21.15.39:0/2447106971 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2484038450 con 0x7f248c15c690 2024-09-21T07:47:13.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.410+0000 7f2491600640 1 -- 172.21.15.39:0/2447106971 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f248c151b70 con 0x7f248c15c690 2024-09-21T07:47:13.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.410+0000 7f2491600640 1 -- 172.21.15.39:0/2447106971 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f248c146e80 con 0x7f248c15c690 2024-09-21T07:47:13.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.411+0000 7f2491600640 1 -- 172.21.15.39:0/2447106971 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f248c0b4ea0 con 0x7f248c15c690 2024-09-21T07:47:13.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.412+0000 7f2482a00640 1 -- 172.21.15.39:0/2447106971 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2484036070 con 0x7f248c15c690 2024-09-21T07:47:13.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.413+0000 7f2482a00640 1 --2- 172.21.15.39:0/2447106971 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f246c078b90 0x7f246c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:13.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.413+0000 7f2482a00640 1 -- 172.21.15.39:0/2447106971 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 6932+0+0 (secure 0 0 0) 0x7f24840bc6a0 con 0x7f248c15c690 2024-09-21T07:47:13.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.414+0000 7f248aa00640 1 --2- 172.21.15.39:0/2447106971 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f246c078b90 0x7f246c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:13.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.414+0000 7f248aa00640 1 --2- 172.21.15.39:0/2447106971 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f246c078b90 0x7f246c07b0a0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f248c15bc50 tx=0x7f247c009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:13.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.418+0000 7f2482a00640 1 -- 172.21.15.39:0/2447106971 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2484084040 con 0x7f248c15c690 2024-09-21T07:47:13.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.615+0000 7f2491600640 1 -- 172.21.15.39:0/2447106971 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f248c151d40 con 0x7f248c15c690 2024-09-21T07:47:13.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.617+0000 7f2482a00640 1 -- 172.21.15.39:0/2447106971 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v194) ==== 74+0+22348 (secure 0 0 0) 0x7f2484088a70 con 0x7f248c15c690 2024-09-21T07:47:13.619 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:47:13.619 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":194,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:47:10.626955+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:47:07.077866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"193","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-09-21T07:47:08.095979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"194","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]},{"pool":20,"snaps":[{"begin":80,"length":23},{"begin":181,"length":23}]},{"pool":22,"snaps":[{"begin":100,"length":1},{"begin":199,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:47:13.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.619+0000 7f2491600640 1 -- 172.21.15.39:0/2447106971 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f246c078b90 msgr2=0x7f246c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:13.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.619+0000 7f2491600640 1 --2- 172.21.15.39:0/2447106971 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f246c078b90 0x7f246c07b0a0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f248c15bc50 tx=0x7f247c009040 comp rx=0 tx=0).stop 2024-09-21T07:47:13.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.619+0000 7f2491600640 1 -- 172.21.15.39:0/2447106971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f248c15c690 msgr2=0x7f248c0cb3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:13.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.619+0000 7f2491600640 1 --2- 172.21.15.39:0/2447106971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f248c15c690 0x7f248c0cb3d0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f248402b020 tx=0x7f2484003f90 comp rx=0 tx=0).stop 2024-09-21T07:47:13.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.619+0000 7f2491600640 1 -- 172.21.15.39:0/2447106971 shutdown_connections 2024-09-21T07:47:13.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.619+0000 7f2491600640 1 --2- 172.21.15.39:0/2447106971 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f246c078b90 0x7f246c07b0a0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:13.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.619+0000 7f2491600640 1 --2- 172.21.15.39:0/2447106971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f248c15c690 0x7f248c0cb3d0 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:13.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.619+0000 7f2491600640 1 --2- 172.21.15.39:0/2447106971 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f248c0d0b70 0x7f248c0c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:13.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.619+0000 7f2491600640 1 --2- 172.21.15.39:0/2447106971 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f248c0b5240 0x7f248c149620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:13.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.619+0000 7f2491600640 1 -- 172.21.15.39:0/2447106971 >> 172.21.15.39:0/2447106971 conn(0x7f248c06cfc0 msgr2=0x7f248c15e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:13.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.619+0000 7f2491600640 1 -- 172.21.15.39:0/2447106971 shutdown_connections 2024-09-21T07:47:13.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.619+0000 7f2491600640 1 -- 172.21.15.39:0/2447106971 wait complete. 2024-09-21T07:47:13.644 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T07:47:13.644 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:47:13.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.867+0000 7f3d59400640 1 -- 172.21.15.39:0/2435017824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d540d1ac0 msgr2=0x7f3d540b4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:13.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.867+0000 7f3d59400640 1 --2- 172.21.15.39:0/2435017824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d540d1ac0 0x7f3d540b4270 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c005fd0 tx=0x7f3d4c02a940 comp rx=0 tx=0).stop 2024-09-21T07:47:13.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.867+0000 7f3d59400640 1 -- 172.21.15.39:0/2435017824 shutdown_connections 2024-09-21T07:47:13.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.867+0000 7f3d59400640 1 --2- 172.21.15.39:0/2435017824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d540d1ac0 0x7f3d540b4270 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:13.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.867+0000 7f3d59400640 1 -- 172.21.15.39:0/2435017824 >> 172.21.15.39:0/2435017824 conn(0x7f3d5409d7b0 msgr2=0x7f3d540d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:13.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.867+0000 7f3d59400640 1 -- 172.21.15.39:0/2435017824 shutdown_connections 2024-09-21T07:47:13.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.867+0000 7f3d59400640 1 -- 172.21.15.39:0/2435017824 wait complete. 2024-09-21T07:47:13.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.869+0000 7f3d59400640 1 Processor -- start 2024-09-21T07:47:13.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.869+0000 7f3d59400640 1 -- start start 2024-09-21T07:47:13.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.871+0000 7f3d59400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d540b5a90 0x7f3d540cde10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:13.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.871+0000 7f3d59400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d540d1ac0 0x7f3d540b8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:13.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.871+0000 7f3d59400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d540b8cc0 0x7f3d541e8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:13.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.871+0000 7f3d59400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3d540b47b0 con 0x7f3d540d1ac0 2024-09-21T07:47:13.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.871+0000 7f3d59400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3d540d2240 con 0x7f3d540b5a90 2024-09-21T07:47:13.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.871+0000 7f3d59400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f3d540b4930 con 0x7f3d540b8cc0 2024-09-21T07:47:13.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.871+0000 7f3d4be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d540d1ac0 0x7f3d540b8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:13.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.871+0000 7f3d4be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d540d1ac0 0x7f3d540b8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48326/0 (socket says 172.21.15.39:48326) 2024-09-21T07:47:13.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.871+0000 7f3d4be00640 1 -- 172.21.15.39:0/4076171979 learned_addr learned my addr 172.21.15.39:0/4076171979 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:13.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.871+0000 7f3d53e00640 1 --2- 172.21.15.39:0/4076171979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d540b5a90 0x7f3d540cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:13.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.871+0000 7f3d58a00640 1 --2- 172.21.15.39:0/4076171979 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d540b8cc0 0x7f3d541e8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:13.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.871+0000 7f3d4be00640 1 -- 172.21.15.39:0/4076171979 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d540b8cc0 msgr2=0x7f3d541e8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:13.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.871+0000 7f3d4be00640 1 --2- 172.21.15.39:0/4076171979 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d540b8cc0 0x7f3d541e8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:13.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.871+0000 7f3d4be00640 1 -- 172.21.15.39:0/4076171979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d540b5a90 msgr2=0x7f3d540cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:13.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.871+0000 7f3d4be00640 1 --2- 172.21.15.39:0/4076171979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d540b5a90 0x7f3d540cde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:13.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.871+0000 7f3d4be00640 1 -- 172.21.15.39:0/4076171979 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d541f5190 con 0x7f3d540d1ac0 2024-09-21T07:47:13.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.872+0000 7f3d4be00640 1 --2- 172.21.15.39:0/4076171979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d540d1ac0 0x7f3d540b8780 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c00c370 tx=0x7f3d3c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:13.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.872+0000 7f3d50c00640 1 -- 172.21.15.39:0/4076171979 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c017070 con 0x7f3d540d1ac0 2024-09-21T07:47:13.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.872+0000 7f3d50c00640 1 -- 172.21.15.39:0/4076171979 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d3c0035f0 con 0x7f3d540d1ac0 2024-09-21T07:47:13.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.872+0000 7f3d59400640 1 -- 172.21.15.39:0/4076171979 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d540cc700 con 0x7f3d540d1ac0 2024-09-21T07:47:13.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.872+0000 7f3d50c00640 1 -- 172.21.15.39:0/4076171979 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c003a40 con 0x7f3d540d1ac0 2024-09-21T07:47:13.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.872+0000 7f3d59400640 1 -- 172.21.15.39:0/4076171979 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d540cccc0 con 0x7f3d540d1ac0 2024-09-21T07:47:13.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.873+0000 7f3d59400640 1 -- 172.21.15.39:0/4076171979 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d540b4d80 con 0x7f3d540d1ac0 2024-09-21T07:47:13.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.874+0000 7f3d50c00640 1 -- 172.21.15.39:0/4076171979 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3d3c01a070 con 0x7f3d540d1ac0 2024-09-21T07:47:13.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.875+0000 7f3d50c00640 1 --2- 172.21.15.39:0/4076171979 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3d30078bc0 0x7f3d3007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:13.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.875+0000 7f3d50c00640 1 -- 172.21.15.39:0/4076171979 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 6932+0+0 (secure 0 0 0) 0x7f3d3c09a4b0 con 0x7f3d540d1ac0 2024-09-21T07:47:13.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.876+0000 7f3d53e00640 1 --2- 172.21.15.39:0/4076171979 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3d30078bc0 0x7f3d3007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:13.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.877+0000 7f3d53e00640 1 --2- 172.21.15.39:0/4076171979 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3d30078bc0 0x7f3d3007b0d0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f3d44000fb0 tx=0x7f3d44009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:13.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:13.880+0000 7f3d50c00640 1 -- 172.21.15.39:0/4076171979 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3d3c062410 con 0x7f3d540d1ac0 2024-09-21T07:47:14.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.073+0000 7f3d59400640 1 -- 172.21.15.39:0/4076171979 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3d540c7d10 con 0x7f3d30078bc0 2024-09-21T07:47:14.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.082+0000 7f3d50c00640 1 -- 172.21.15.39:0/4076171979 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+495240 (secure 0 0 0) 0x7f3d540c7d10 con 0x7f3d30078bc0 2024-09-21T07:47:14.090 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:47:14.093 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:47:14.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.087+0000 7f3d59400640 1 -- 172.21.15.39:0/4076171979 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3d30078bc0 msgr2=0x7f3d3007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:14.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.087+0000 7f3d59400640 1 --2- 172.21.15.39:0/4076171979 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3d30078bc0 0x7f3d3007b0d0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f3d44000fb0 tx=0x7f3d44009040 comp rx=0 tx=0).stop 2024-09-21T07:47:14.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.087+0000 7f3d59400640 1 -- 172.21.15.39:0/4076171979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d540d1ac0 msgr2=0x7f3d540b8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:14.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.087+0000 7f3d59400640 1 --2- 172.21.15.39:0/4076171979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d540d1ac0 0x7f3d540b8780 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c00c370 tx=0x7f3d3c00c830 comp rx=0 tx=0).stop 2024-09-21T07:47:14.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.088+0000 7f3d59400640 1 -- 172.21.15.39:0/4076171979 shutdown_connections 2024-09-21T07:47:14.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.088+0000 7f3d59400640 1 --2- 172.21.15.39:0/4076171979 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3d30078bc0 0x7f3d3007b0d0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:14.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.088+0000 7f3d59400640 1 --2- 172.21.15.39:0/4076171979 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d540b8cc0 0x7f3d541e8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:14.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.088+0000 7f3d59400640 1 --2- 172.21.15.39:0/4076171979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d540d1ac0 0x7f3d540b8780 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:14.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.088+0000 7f3d59400640 1 --2- 172.21.15.39:0/4076171979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d540b5a90 0x7f3d540cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:14.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.088+0000 7f3d59400640 1 -- 172.21.15.39:0/4076171979 >> 172.21.15.39:0/4076171979 conn(0x7f3d5409d7b0 msgr2=0x7f3d5406dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:14.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.088+0000 7f3d59400640 1 -- 172.21.15.39:0/4076171979 shutdown_connections 2024-09-21T07:47:14.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.088+0000 7f3d59400640 1 -- 172.21.15.39:0/4076171979 wait complete. 2024-09-21T07:47:14.111 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":615,"stamp":"2024-09-21T07:47:13.817505+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":378,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":412748,"kb_used_data":20232,"kb_used_omap":12,"kb_used_meta":392307,"kb_avail":754561972,"statfs":{"total":773094113280,"available":772671459328,"internally_reserved":0,"allocated":20717568,"data_stored":11542268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":401722928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002028"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.639616+0000","last_change":"2024-09-21T07:47:09.644043+0000","last_active":"2024-09-21T07:47:10.639616+0000","last_peered":"2024-09-21T07:47:10.639616+0000","last_clean":"2024-09-21T07:47:10.639616+0000","last_became_active":"2024-09-21T07:47:09.642096+0000","last_became_peered":"2024-09-21T07:47:09.642096+0000","last_unstale":"2024-09-21T07:47:10.639616+0000","last_undegraded":"2024-09-21T07:47:10.639616+0000","last_fullsized":"2024-09-21T07:47:10.639616+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:05:21.255629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.640744+0000","last_change":"2024-09-21T07:47:09.640763+0000","last_active":"2024-09-21T07:47:10.640744+0000","last_peered":"2024-09-21T07:47:10.640744+0000","last_clean":"2024-09-21T07:47:10.640744+0000","last_became_active":"2024-09-21T07:47:09.640441+0000","last_became_peered":"2024-09-21T07:47:09.640441+0000","last_unstale":"2024-09-21T07:47:10.640744+0000","last_undegraded":"2024-09-21T07:47:10.640744+0000","last_fullsized":"2024-09-21T07:47:10.640744+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:37.931310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638820+0000","last_change":"2024-09-21T07:47:09.638610+0000","last_active":"2024-09-21T07:47:10.638820+0000","last_peered":"2024-09-21T07:47:10.638820+0000","last_clean":"2024-09-21T07:47:10.638820+0000","last_became_active":"2024-09-21T07:47:09.638166+0000","last_became_peered":"2024-09-21T07:47:09.638166+0000","last_unstale":"2024-09-21T07:47:10.638820+0000","last_undegraded":"2024-09-21T07:47:10.638820+0000","last_fullsized":"2024-09-21T07:47:10.638820+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:03.254832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.639031+0000","last_change":"2024-09-21T07:47:09.658247+0000","last_active":"2024-09-21T07:47:10.639031+0000","last_peered":"2024-09-21T07:47:10.639031+0000","last_clean":"2024-09-21T07:47:10.639031+0000","last_became_active":"2024-09-21T07:47:09.657914+0000","last_became_peered":"2024-09-21T07:47:09.657914+0000","last_unstale":"2024-09-21T07:47:10.639031+0000","last_undegraded":"2024-09-21T07:47:10.639031+0000","last_fullsized":"2024-09-21T07:47:10.639031+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:59.525973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869671+0000","last_change":"2024-09-21T07:47:09.650494+0000","last_active":"2024-09-21T07:47:10.869671+0000","last_peered":"2024-09-21T07:47:10.869671+0000","last_clean":"2024-09-21T07:47:10.869671+0000","last_became_active":"2024-09-21T07:47:09.649970+0000","last_became_peered":"2024-09-21T07:47:09.649970+0000","last_unstale":"2024-09-21T07:47:10.869671+0000","last_undegraded":"2024-09-21T07:47:10.869671+0000","last_fullsized":"2024-09-21T07:47:10.869671+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:42.077459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.639681+0000","last_change":"2024-09-21T07:47:09.643421+0000","last_active":"2024-09-21T07:47:10.639681+0000","last_peered":"2024-09-21T07:47:10.639681+0000","last_clean":"2024-09-21T07:47:10.639681+0000","last_became_active":"2024-09-21T07:47:09.643134+0000","last_became_peered":"2024-09-21T07:47:09.643134+0000","last_unstale":"2024-09-21T07:47:10.639681+0000","last_undegraded":"2024-09-21T07:47:10.639681+0000","last_fullsized":"2024-09-21T07:47:10.639681+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:29.033001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.872323+0000","last_change":"2024-09-21T07:47:09.641703+0000","last_active":"2024-09-21T07:47:10.872323+0000","last_peered":"2024-09-21T07:47:10.872323+0000","last_clean":"2024-09-21T07:47:10.872323+0000","last_became_active":"2024-09-21T07:47:09.641393+0000","last_became_peered":"2024-09-21T07:47:09.641393+0000","last_unstale":"2024-09-21T07:47:10.872323+0000","last_undegraded":"2024-09-21T07:47:10.872323+0000","last_fullsized":"2024-09-21T07:47:10.872323+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:29:26.742108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870667+0000","last_change":"2024-09-21T07:47:09.641426+0000","last_active":"2024-09-21T07:47:10.870667+0000","last_peered":"2024-09-21T07:47:10.870667+0000","last_clean":"2024-09-21T07:47:10.870667+0000","last_became_active":"2024-09-21T07:47:09.640738+0000","last_became_peered":"2024-09-21T07:47:09.640738+0000","last_unstale":"2024-09-21T07:47:10.870667+0000","last_undegraded":"2024-09-21T07:47:10.870667+0000","last_fullsized":"2024-09-21T07:47:10.870667+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:54.389102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.874339+0000","last_change":"2024-09-21T07:47:09.652547+0000","last_active":"2024-09-21T07:47:10.874339+0000","last_peered":"2024-09-21T07:47:10.874339+0000","last_clean":"2024-09-21T07:47:10.874339+0000","last_became_active":"2024-09-21T07:47:09.651506+0000","last_became_peered":"2024-09-21T07:47:09.651506+0000","last_unstale":"2024-09-21T07:47:10.874339+0000","last_undegraded":"2024-09-21T07:47:10.874339+0000","last_fullsized":"2024-09-21T07:47:10.874339+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:29.255108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.874294+0000","last_change":"2024-09-21T07:47:09.648987+0000","last_active":"2024-09-21T07:47:10.874294+0000","last_peered":"2024-09-21T07:47:10.874294+0000","last_clean":"2024-09-21T07:47:10.874294+0000","last_became_active":"2024-09-21T07:47:09.648668+0000","last_became_peered":"2024-09-21T07:47:09.648668+0000","last_unstale":"2024-09-21T07:47:10.874294+0000","last_undegraded":"2024-09-21T07:47:10.874294+0000","last_fullsized":"2024-09-21T07:47:10.874294+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:24:02.928287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869990+0000","last_change":"2024-09-21T07:47:09.642891+0000","last_active":"2024-09-21T07:47:10.869990+0000","last_peered":"2024-09-21T07:47:10.869990+0000","last_clean":"2024-09-21T07:47:10.869990+0000","last_became_active":"2024-09-21T07:47:09.642411+0000","last_became_peered":"2024-09-21T07:47:09.642411+0000","last_unstale":"2024-09-21T07:47:10.869990+0000","last_undegraded":"2024-09-21T07:47:10.869990+0000","last_fullsized":"2024-09-21T07:47:10.869990+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:59.597050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.640344+0000","last_change":"2024-09-21T07:47:09.646034+0000","last_active":"2024-09-21T07:47:10.640344+0000","last_peered":"2024-09-21T07:47:10.640344+0000","last_clean":"2024-09-21T07:47:10.640344+0000","last_became_active":"2024-09-21T07:47:09.645505+0000","last_became_peered":"2024-09-21T07:47:09.645505+0000","last_unstale":"2024-09-21T07:47:10.640344+0000","last_undegraded":"2024-09-21T07:47:10.640344+0000","last_fullsized":"2024-09-21T07:47:10.640344+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:26:46.220617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638074+0000","last_change":"2024-09-21T07:47:09.643246+0000","last_active":"2024-09-21T07:47:10.638074+0000","last_peered":"2024-09-21T07:47:10.638074+0000","last_clean":"2024-09-21T07:47:10.638074+0000","last_became_active":"2024-09-21T07:47:09.642941+0000","last_became_peered":"2024-09-21T07:47:09.642941+0000","last_unstale":"2024-09-21T07:47:10.638074+0000","last_undegraded":"2024-09-21T07:47:10.638074+0000","last_fullsized":"2024-09-21T07:47:10.638074+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:41.557690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870038+0000","last_change":"2024-09-21T07:47:09.645793+0000","last_active":"2024-09-21T07:47:10.870038+0000","last_peered":"2024-09-21T07:47:10.870038+0000","last_clean":"2024-09-21T07:47:10.870038+0000","last_became_active":"2024-09-21T07:47:09.645333+0000","last_became_peered":"2024-09-21T07:47:09.645333+0000","last_unstale":"2024-09-21T07:47:10.870038+0000","last_undegraded":"2024-09-21T07:47:10.870038+0000","last_fullsized":"2024-09-21T07:47:10.870038+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:28.245429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871248+0000","last_change":"2024-09-21T07:47:09.639290+0000","last_active":"2024-09-21T07:47:10.871248+0000","last_peered":"2024-09-21T07:47:10.871248+0000","last_clean":"2024-09-21T07:47:10.871248+0000","last_became_active":"2024-09-21T07:47:09.638942+0000","last_became_peered":"2024-09-21T07:47:09.638942+0000","last_unstale":"2024-09-21T07:47:10.871248+0000","last_undegraded":"2024-09-21T07:47:10.871248+0000","last_fullsized":"2024-09-21T07:47:10.871248+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:58.656671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638009+0000","last_change":"2024-09-21T07:47:09.659865+0000","last_active":"2024-09-21T07:47:10.638009+0000","last_peered":"2024-09-21T07:47:10.638009+0000","last_clean":"2024-09-21T07:47:10.638009+0000","last_became_active":"2024-09-21T07:47:09.659413+0000","last_became_peered":"2024-09-21T07:47:09.659413+0000","last_unstale":"2024-09-21T07:47:10.638009+0000","last_undegraded":"2024-09-21T07:47:10.638009+0000","last_fullsized":"2024-09-21T07:47:10.638009+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:26:30.060687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.874319+0000","last_change":"2024-09-21T07:47:09.651816+0000","last_active":"2024-09-21T07:47:10.874319+0000","last_peered":"2024-09-21T07:47:10.874319+0000","last_clean":"2024-09-21T07:47:10.874319+0000","last_became_active":"2024-09-21T07:47:09.651524+0000","last_became_peered":"2024-09-21T07:47:09.651524+0000","last_unstale":"2024-09-21T07:47:10.874319+0000","last_undegraded":"2024-09-21T07:47:10.874319+0000","last_fullsized":"2024-09-21T07:47:10.874319+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:52.579196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870619+0000","last_change":"2024-09-21T07:47:09.649812+0000","last_active":"2024-09-21T07:47:10.870619+0000","last_peered":"2024-09-21T07:47:10.870619+0000","last_clean":"2024-09-21T07:47:10.870619+0000","last_became_active":"2024-09-21T07:47:09.649139+0000","last_became_peered":"2024-09-21T07:47:09.649139+0000","last_unstale":"2024-09-21T07:47:10.870619+0000","last_undegraded":"2024-09-21T07:47:10.870619+0000","last_fullsized":"2024-09-21T07:47:10.870619+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:07:13.269043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870793+0000","last_change":"2024-09-21T07:47:09.652349+0000","last_active":"2024-09-21T07:47:10.870793+0000","last_peered":"2024-09-21T07:47:10.870793+0000","last_clean":"2024-09-21T07:47:10.870793+0000","last_became_active":"2024-09-21T07:47:09.652023+0000","last_became_peered":"2024-09-21T07:47:09.652023+0000","last_unstale":"2024-09-21T07:47:10.870793+0000","last_undegraded":"2024-09-21T07:47:10.870793+0000","last_fullsized":"2024-09-21T07:47:10.870793+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:53.414209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871755+0000","last_change":"2024-09-21T07:47:09.643833+0000","last_active":"2024-09-21T07:47:10.871755+0000","last_peered":"2024-09-21T07:47:10.871755+0000","last_clean":"2024-09-21T07:47:10.871755+0000","last_became_active":"2024-09-21T07:47:09.643548+0000","last_became_peered":"2024-09-21T07:47:09.643548+0000","last_unstale":"2024-09-21T07:47:10.871755+0000","last_undegraded":"2024-09-21T07:47:10.871755+0000","last_fullsized":"2024-09-21T07:47:10.871755+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:00.737900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638095+0000","last_change":"2024-09-21T07:47:09.643968+0000","last_active":"2024-09-21T07:47:10.638095+0000","last_peered":"2024-09-21T07:47:10.638095+0000","last_clean":"2024-09-21T07:47:10.638095+0000","last_became_active":"2024-09-21T07:47:09.643671+0000","last_became_peered":"2024-09-21T07:47:09.643671+0000","last_unstale":"2024-09-21T07:47:10.638095+0000","last_undegraded":"2024-09-21T07:47:10.638095+0000","last_fullsized":"2024-09-21T07:47:10.638095+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:49.250392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.639647+0000","last_change":"2024-09-21T07:47:09.660236+0000","last_active":"2024-09-21T07:47:10.639647+0000","last_peered":"2024-09-21T07:47:10.639647+0000","last_clean":"2024-09-21T07:47:10.639647+0000","last_became_active":"2024-09-21T07:47:09.659923+0000","last_became_peered":"2024-09-21T07:47:09.659923+0000","last_unstale":"2024-09-21T07:47:10.639647+0000","last_undegraded":"2024-09-21T07:47:10.639647+0000","last_fullsized":"2024-09-21T07:47:10.639647+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:24.557148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869853+0000","last_change":"2024-09-21T07:47:09.639086+0000","last_active":"2024-09-21T07:47:10.869853+0000","last_peered":"2024-09-21T07:47:10.869853+0000","last_clean":"2024-09-21T07:47:10.869853+0000","last_became_active":"2024-09-21T07:47:09.636177+0000","last_became_peered":"2024-09-21T07:47:09.636177+0000","last_unstale":"2024-09-21T07:47:10.869853+0000","last_undegraded":"2024-09-21T07:47:10.869853+0000","last_fullsized":"2024-09-21T07:47:10.869853+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:02.806011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.637648+0000","last_change":"2024-09-21T07:47:09.660424+0000","last_active":"2024-09-21T07:47:10.637648+0000","last_peered":"2024-09-21T07:47:10.637648+0000","last_clean":"2024-09-21T07:47:10.637648+0000","last_became_active":"2024-09-21T07:47:09.659909+0000","last_became_peered":"2024-09-21T07:47:09.659909+0000","last_unstale":"2024-09-21T07:47:10.637648+0000","last_undegraded":"2024-09-21T07:47:10.637648+0000","last_fullsized":"2024-09-21T07:47:10.637648+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:08.371997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638773+0000","last_change":"2024-09-21T07:47:09.640189+0000","last_active":"2024-09-21T07:47:10.638773+0000","last_peered":"2024-09-21T07:47:10.638773+0000","last_clean":"2024-09-21T07:47:10.638773+0000","last_became_active":"2024-09-21T07:47:09.639778+0000","last_became_peered":"2024-09-21T07:47:09.639778+0000","last_unstale":"2024-09-21T07:47:10.638773+0000","last_undegraded":"2024-09-21T07:47:10.638773+0000","last_fullsized":"2024-09-21T07:47:10.638773+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:19.385094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.873348+0000","last_change":"2024-09-21T07:47:09.639182+0000","last_active":"2024-09-21T07:47:10.873348+0000","last_peered":"2024-09-21T07:47:10.873348+0000","last_clean":"2024-09-21T07:47:10.873348+0000","last_became_active":"2024-09-21T07:47:09.638324+0000","last_became_peered":"2024-09-21T07:47:09.638324+0000","last_unstale":"2024-09-21T07:47:10.873348+0000","last_undegraded":"2024-09-21T07:47:10.873348+0000","last_fullsized":"2024-09-21T07:47:10.873348+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:16.359936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.640595+0000","last_change":"2024-09-21T07:47:09.660017+0000","last_active":"2024-09-21T07:47:10.640595+0000","last_peered":"2024-09-21T07:47:10.640595+0000","last_clean":"2024-09-21T07:47:10.640595+0000","last_became_active":"2024-09-21T07:47:09.659724+0000","last_became_peered":"2024-09-21T07:47:09.659724+0000","last_unstale":"2024-09-21T07:47:10.640595+0000","last_undegraded":"2024-09-21T07:47:10.640595+0000","last_fullsized":"2024-09-21T07:47:10.640595+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:43.831537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638622+0000","last_change":"2024-09-21T07:47:09.646270+0000","last_active":"2024-09-21T07:47:10.638622+0000","last_peered":"2024-09-21T07:47:10.638622+0000","last_clean":"2024-09-21T07:47:10.638622+0000","last_became_active":"2024-09-21T07:47:09.645562+0000","last_became_peered":"2024-09-21T07:47:09.645562+0000","last_unstale":"2024-09-21T07:47:10.638622+0000","last_undegraded":"2024-09-21T07:47:10.638622+0000","last_fullsized":"2024-09-21T07:47:10.638622+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:43:03.738757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.639854+0000","last_change":"2024-09-21T07:47:09.652846+0000","last_active":"2024-09-21T07:47:10.639854+0000","last_peered":"2024-09-21T07:47:10.639854+0000","last_clean":"2024-09-21T07:47:10.639854+0000","last_became_active":"2024-09-21T07:47:09.652473+0000","last_became_peered":"2024-09-21T07:47:09.652473+0000","last_unstale":"2024-09-21T07:47:10.639854+0000","last_undegraded":"2024-09-21T07:47:10.639854+0000","last_fullsized":"2024-09-21T07:47:10.639854+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:41:20.324009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869892+0000","last_change":"2024-09-21T07:47:09.639974+0000","last_active":"2024-09-21T07:47:10.869892+0000","last_peered":"2024-09-21T07:47:10.869892+0000","last_clean":"2024-09-21T07:47:10.869892+0000","last_became_active":"2024-09-21T07:47:09.639679+0000","last_became_peered":"2024-09-21T07:47:09.639679+0000","last_unstale":"2024-09-21T07:47:10.869892+0000","last_undegraded":"2024-09-21T07:47:10.869892+0000","last_fullsized":"2024-09-21T07:47:10.869892+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:42.066652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871724+0000","last_change":"2024-09-21T07:47:09.632419+0000","last_active":"2024-09-21T07:47:10.871724+0000","last_peered":"2024-09-21T07:47:10.871724+0000","last_clean":"2024-09-21T07:47:10.871724+0000","last_became_active":"2024-09-21T07:47:09.632088+0000","last_became_peered":"2024-09-21T07:47:09.632088+0000","last_unstale":"2024-09-21T07:47:10.871724+0000","last_undegraded":"2024-09-21T07:47:10.871724+0000","last_fullsized":"2024-09-21T07:47:10.871724+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:05:05.685632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869492+0000","last_change":"2024-09-21T07:47:09.635477+0000","last_active":"2024-09-21T07:47:10.869492+0000","last_peered":"2024-09-21T07:47:10.869492+0000","last_clean":"2024-09-21T07:47:10.869492+0000","last_became_active":"2024-09-21T07:47:09.635112+0000","last_became_peered":"2024-09-21T07:47:09.635112+0000","last_unstale":"2024-09-21T07:47:10.869492+0000","last_undegraded":"2024-09-21T07:47:10.869492+0000","last_fullsized":"2024-09-21T07:47:10.869492+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:03:13.741801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.637755+0000","last_change":"2024-09-21T07:47:09.639625+0000","last_active":"2024-09-21T07:47:10.637755+0000","last_peered":"2024-09-21T07:47:10.637755+0000","last_clean":"2024-09-21T07:47:10.637755+0000","last_became_active":"2024-09-21T07:47:09.639314+0000","last_became_peered":"2024-09-21T07:47:09.639314+0000","last_unstale":"2024-09-21T07:47:10.637755+0000","last_undegraded":"2024-09-21T07:47:10.637755+0000","last_fullsized":"2024-09-21T07:47:10.637755+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:20.277990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":15,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638870+0000","last_change":"2024-09-21T07:47:08.640591+0000","last_active":"2024-09-21T07:47:10.638870+0000","last_peered":"2024-09-21T07:47:10.638870+0000","last_clean":"2024-09-21T07:47:10.638870+0000","last_became_active":"2024-09-21T07:47:08.639759+0000","last_became_peered":"2024-09-21T07:47:08.639759+0000","last_unstale":"2024-09-21T07:47:10.638870+0000","last_undegraded":"2024-09-21T07:47:10.638870+0000","last_fullsized":"2024-09-21T07:47:10.638870+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:59.250592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":262,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.640502+0000","last_change":"2024-09-21T07:35:27.879605+0000","last_active":"2024-09-21T07:47:10.640502+0000","last_peered":"2024-09-21T07:47:10.640502+0000","last_clean":"2024-09-21T07:47:10.640502+0000","last_became_active":"2024-09-21T07:35:27.879274+0000","last_became_peered":"2024-09-21T07:35:27.879274+0000","last_unstale":"2024-09-21T07:47:10.640502+0000","last_undegraded":"2024-09-21T07:47:10.640502+0000","last_fullsized":"2024-09-21T07:47:10.640502+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:24.241636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":270,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.637911+0000","last_change":"2024-09-21T07:35:29.491336+0000","last_active":"2024-09-21T07:47:10.637911+0000","last_peered":"2024-09-21T07:47:10.637911+0000","last_clean":"2024-09-21T07:47:10.637911+0000","last_became_active":"2024-09-21T07:35:26.866484+0000","last_became_peered":"2024-09-21T07:35:26.866484+0000","last_unstale":"2024-09-21T07:47:10.637911+0000","last_undegraded":"2024-09-21T07:47:10.637911+0000","last_fullsized":"2024-09-21T07:47:10.637911+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:10.449665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076010100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":448,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638246+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:47:10.638246+0000","last_peered":"2024-09-21T07:47:10.638246+0000","last_clean":"2024-09-21T07:47:10.638246+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:47:10.638246+0000","last_undegraded":"2024-09-21T07:47:10.638246+0000","last_fullsized":"2024-09-21T07:47:10.638246+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":328,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871259+0000","last_change":"2024-09-21T07:34:24.539248+0000","last_active":"2024-09-21T07:47:10.871259+0000","last_peered":"2024-09-21T07:47:10.871259+0000","last_clean":"2024-09-21T07:47:10.871259+0000","last_became_active":"2024-09-21T07:34:23.273465+0000","last_became_peered":"2024-09-21T07:34:23.273465+0000","last_unstale":"2024-09-21T07:47:10.871259+0000","last_undegraded":"2024-09-21T07:47:10.871259+0000","last_fullsized":"2024-09-21T07:47:10.871259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:55.517452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060627700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870268+0000","last_change":"2024-09-21T07:47:09.641109+0000","last_active":"2024-09-21T07:47:10.870268+0000","last_peered":"2024-09-21T07:47:10.870268+0000","last_clean":"2024-09-21T07:47:10.870268+0000","last_became_active":"2024-09-21T07:47:09.640791+0000","last_became_peered":"2024-09-21T07:47:09.640791+0000","last_unstale":"2024-09-21T07:47:10.870268+0000","last_undegraded":"2024-09-21T07:47:10.870268+0000","last_fullsized":"2024-09-21T07:47:10.870268+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:09:45.917662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"193'2","reported_seq":17,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869870+0000","last_change":"2024-09-21T07:47:08.633962+0000","last_active":"2024-09-21T07:47:10.869870+0000","last_peered":"2024-09-21T07:47:10.869870+0000","last_clean":"2024-09-21T07:47:10.869870+0000","last_became_active":"2024-09-21T07:47:08.633574+0000","last_became_peered":"2024-09-21T07:47:08.633574+0000","last_unstale":"2024-09-21T07:47:10.869870+0000","last_undegraded":"2024-09-21T07:47:10.869870+0000","last_fullsized":"2024-09-21T07:47:10.869870+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:26:45.213939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":263,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.637421+0000","last_change":"2024-09-21T07:35:27.864492+0000","last_active":"2024-09-21T07:47:10.637421+0000","last_peered":"2024-09-21T07:47:10.637421+0000","last_clean":"2024-09-21T07:47:10.637421+0000","last_became_active":"2024-09-21T07:35:27.864176+0000","last_became_peered":"2024-09-21T07:35:27.864176+0000","last_unstale":"2024-09-21T07:47:10.637421+0000","last_undegraded":"2024-09-21T07:47:10.637421+0000","last_fullsized":"2024-09-21T07:47:10.637421+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:05.153467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":269,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.640987+0000","last_change":"2024-09-21T07:35:29.493678+0000","last_active":"2024-09-21T07:47:10.640987+0000","last_peered":"2024-09-21T07:47:10.640987+0000","last_clean":"2024-09-21T07:47:10.640987+0000","last_became_active":"2024-09-21T07:35:26.870749+0000","last_became_peered":"2024-09-21T07:35:26.870749+0000","last_unstale":"2024-09-21T07:47:10.640987+0000","last_undegraded":"2024-09-21T07:47:10.640987+0000","last_fullsized":"2024-09-21T07:47:10.640987+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:55.367877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010159570000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":315,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638552+0000","last_change":"2024-09-21T07:34:24.597274+0000","last_active":"2024-09-21T07:47:10.638552+0000","last_peered":"2024-09-21T07:47:10.638552+0000","last_clean":"2024-09-21T07:47:10.638552+0000","last_became_active":"2024-09-21T07:34:23.279560+0000","last_became_peered":"2024-09-21T07:34:23.279560+0000","last_unstale":"2024-09-21T07:47:10.638552+0000","last_undegraded":"2024-09-21T07:47:10.638552+0000","last_fullsized":"2024-09-21T07:47:10.638552+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:15.845265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059699799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.873833+0000","last_change":"2024-09-21T07:47:09.657050+0000","last_active":"2024-09-21T07:47:10.873833+0000","last_peered":"2024-09-21T07:47:10.873833+0000","last_clean":"2024-09-21T07:47:10.873833+0000","last_became_active":"2024-09-21T07:47:09.656487+0000","last_became_peered":"2024-09-21T07:47:09.656487+0000","last_unstale":"2024-09-21T07:47:10.873833+0000","last_undegraded":"2024-09-21T07:47:10.873833+0000","last_fullsized":"2024-09-21T07:47:10.873833+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:19:33.447462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":15,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.637524+0000","last_change":"2024-09-21T07:47:08.636622+0000","last_active":"2024-09-21T07:47:10.637524+0000","last_peered":"2024-09-21T07:47:10.637524+0000","last_clean":"2024-09-21T07:47:10.637524+0000","last_became_active":"2024-09-21T07:47:08.636249+0000","last_became_peered":"2024-09-21T07:47:08.636249+0000","last_unstale":"2024-09-21T07:47:10.637524+0000","last_undegraded":"2024-09-21T07:47:10.637524+0000","last_fullsized":"2024-09-21T07:47:10.637524+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:32.062377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":262,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870804+0000","last_change":"2024-09-21T07:35:27.874387+0000","last_active":"2024-09-21T07:47:10.870804+0000","last_peered":"2024-09-21T07:47:10.870804+0000","last_clean":"2024-09-21T07:47:10.870804+0000","last_became_active":"2024-09-21T07:35:27.873277+0000","last_became_peered":"2024-09-21T07:35:27.873277+0000","last_unstale":"2024-09-21T07:47:10.870804+0000","last_undegraded":"2024-09-21T07:47:10.870804+0000","last_fullsized":"2024-09-21T07:47:10.870804+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:04.581463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":265,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.639200+0000","last_change":"2024-09-21T07:35:29.387094+0000","last_active":"2024-09-21T07:47:10.639200+0000","last_peered":"2024-09-21T07:47:10.639200+0000","last_clean":"2024-09-21T07:47:10.639200+0000","last_became_active":"2024-09-21T07:35:26.874757+0000","last_became_peered":"2024-09-21T07:35:26.874757+0000","last_unstale":"2024-09-21T07:47:10.639200+0000","last_undegraded":"2024-09-21T07:47:10.639200+0000","last_fullsized":"2024-09-21T07:47:10.639200+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:33.542750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061065700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":322,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.640391+0000","last_change":"2024-09-21T07:34:24.259906+0000","last_active":"2024-09-21T07:47:10.640391+0000","last_peered":"2024-09-21T07:47:10.640391+0000","last_clean":"2024-09-21T07:47:10.640391+0000","last_became_active":"2024-09-21T07:34:23.635282+0000","last_became_peered":"2024-09-21T07:34:23.635282+0000","last_unstale":"2024-09-21T07:47:10.640391+0000","last_undegraded":"2024-09-21T07:47:10.640391+0000","last_fullsized":"2024-09-21T07:47:10.640391+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:00.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001527664,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871182+0000","last_change":"2024-09-21T07:47:09.636769+0000","last_active":"2024-09-21T07:47:10.871182+0000","last_peered":"2024-09-21T07:47:10.871182+0000","last_clean":"2024-09-21T07:47:10.871182+0000","last_became_active":"2024-09-21T07:47:09.636440+0000","last_became_peered":"2024-09-21T07:47:09.636440+0000","last_unstale":"2024-09-21T07:47:10.871182+0000","last_undegraded":"2024-09-21T07:47:10.871182+0000","last_fullsized":"2024-09-21T07:47:10.871182+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:07.036416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":15,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871331+0000","last_change":"2024-09-21T07:47:08.623433+0000","last_active":"2024-09-21T07:47:10.871331+0000","last_peered":"2024-09-21T07:47:10.871331+0000","last_clean":"2024-09-21T07:47:10.871331+0000","last_became_active":"2024-09-21T07:47:08.622500+0000","last_became_peered":"2024-09-21T07:47:08.622500+0000","last_unstale":"2024-09-21T07:47:10.871331+0000","last_undegraded":"2024-09-21T07:47:10.871331+0000","last_fullsized":"2024-09-21T07:47:10.871331+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:18:34.856576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":262,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871303+0000","last_change":"2024-09-21T07:35:27.873645+0000","last_active":"2024-09-21T07:47:10.871303+0000","last_peered":"2024-09-21T07:47:10.871303+0000","last_clean":"2024-09-21T07:47:10.871303+0000","last_became_active":"2024-09-21T07:35:27.873245+0000","last_became_peered":"2024-09-21T07:35:27.873245+0000","last_unstale":"2024-09-21T07:47:10.871303+0000","last_undegraded":"2024-09-21T07:47:10.871303+0000","last_fullsized":"2024-09-21T07:47:10.871303+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:55.090953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":269,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.872919+0000","last_change":"2024-09-21T07:35:28.861744+0000","last_active":"2024-09-21T07:47:10.872919+0000","last_peered":"2024-09-21T07:47:10.872919+0000","last_clean":"2024-09-21T07:47:10.872919+0000","last_became_active":"2024-09-21T07:35:26.876462+0000","last_became_peered":"2024-09-21T07:35:26.876462+0000","last_unstale":"2024-09-21T07:47:10.872919+0000","last_undegraded":"2024-09-21T07:47:10.872919+0000","last_fullsized":"2024-09-21T07:47:10.872919+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:48.632568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012020920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":318,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869381+0000","last_change":"2024-09-21T07:34:25.085454+0000","last_active":"2024-09-21T07:47:10.869381+0000","last_peered":"2024-09-21T07:47:10.869381+0000","last_clean":"2024-09-21T07:47:10.869381+0000","last_became_active":"2024-09-21T07:34:23.279205+0000","last_became_peered":"2024-09-21T07:34:23.279205+0000","last_unstale":"2024-09-21T07:47:10.869381+0000","last_undegraded":"2024-09-21T07:47:10.869381+0000","last_fullsized":"2024-09-21T07:47:10.869381+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:54.231666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095320599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.873469+0000","last_change":"2024-09-21T07:47:09.641653+0000","last_active":"2024-09-21T07:47:10.873469+0000","last_peered":"2024-09-21T07:47:10.873469+0000","last_clean":"2024-09-21T07:47:10.873469+0000","last_became_active":"2024-09-21T07:47:09.641352+0000","last_became_peered":"2024-09-21T07:47:09.641352+0000","last_unstale":"2024-09-21T07:47:10.873469+0000","last_undegraded":"2024-09-21T07:47:10.873469+0000","last_fullsized":"2024-09-21T07:47:10.873469+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:17:19.290168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"193'8","reported_seq":23,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871191+0000","last_change":"2024-09-21T07:47:08.624573+0000","last_active":"2024-09-21T07:47:10.871191+0000","last_peered":"2024-09-21T07:47:10.871191+0000","last_clean":"2024-09-21T07:47:10.871191+0000","last_became_active":"2024-09-21T07:47:08.624259+0000","last_became_peered":"2024-09-21T07:47:08.624259+0000","last_unstale":"2024-09-21T07:47:10.871191+0000","last_undegraded":"2024-09-21T07:47:10.871191+0000","last_fullsized":"2024-09-21T07:47:10.871191+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:15:23.264287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":263,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638182+0000","last_change":"2024-09-21T07:35:27.861373+0000","last_active":"2024-09-21T07:47:10.638182+0000","last_peered":"2024-09-21T07:47:10.638182+0000","last_clean":"2024-09-21T07:47:10.638182+0000","last_became_active":"2024-09-21T07:35:27.861016+0000","last_became_peered":"2024-09-21T07:35:27.861016+0000","last_unstale":"2024-09-21T07:47:10.638182+0000","last_undegraded":"2024-09-21T07:47:10.638182+0000","last_fullsized":"2024-09-21T07:47:10.638182+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:56.043763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":269,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.872726+0000","last_change":"2024-09-21T07:35:29.053677+0000","last_active":"2024-09-21T07:47:10.872726+0000","last_peered":"2024-09-21T07:47:10.872726+0000","last_clean":"2024-09-21T07:47:10.872726+0000","last_became_active":"2024-09-21T07:35:26.875496+0000","last_became_peered":"2024-09-21T07:35:26.875496+0000","last_unstale":"2024-09-21T07:47:10.872726+0000","last_undegraded":"2024-09-21T07:47:10.872726+0000","last_fullsized":"2024-09-21T07:47:10.872726+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:04.850784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065859000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":321,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.872395+0000","last_change":"2024-09-21T07:34:25.028521+0000","last_active":"2024-09-21T07:47:10.872395+0000","last_peered":"2024-09-21T07:47:10.872395+0000","last_clean":"2024-09-21T07:47:10.872395+0000","last_became_active":"2024-09-21T07:34:23.635467+0000","last_became_peered":"2024-09-21T07:34:23.635467+0000","last_unstale":"2024-09-21T07:47:10.872395+0000","last_undegraded":"2024-09-21T07:47:10.872395+0000","last_fullsized":"2024-09-21T07:47:10.872395+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:33.729123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000660558,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.637984+0000","last_change":"2024-09-21T07:47:09.660051+0000","last_active":"2024-09-21T07:47:10.637984+0000","last_peered":"2024-09-21T07:47:10.637984+0000","last_clean":"2024-09-21T07:47:10.637984+0000","last_became_active":"2024-09-21T07:47:09.659683+0000","last_became_peered":"2024-09-21T07:47:09.659683+0000","last_unstale":"2024-09-21T07:47:10.637984+0000","last_undegraded":"2024-09-21T07:47:10.637984+0000","last_fullsized":"2024-09-21T07:47:10.637984+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:38:28.343809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":15,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638605+0000","last_change":"2024-09-21T07:47:08.648146+0000","last_active":"2024-09-21T07:47:10.638605+0000","last_peered":"2024-09-21T07:47:10.638605+0000","last_clean":"2024-09-21T07:47:10.638605+0000","last_became_active":"2024-09-21T07:47:08.647806+0000","last_became_peered":"2024-09-21T07:47:08.647806+0000","last_unstale":"2024-09-21T07:47:10.638605+0000","last_undegraded":"2024-09-21T07:47:10.638605+0000","last_fullsized":"2024-09-21T07:47:10.638605+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:00.150181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":260,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870147+0000","last_change":"2024-09-21T07:35:27.871201+0000","last_active":"2024-09-21T07:47:10.870147+0000","last_peered":"2024-09-21T07:47:10.870147+0000","last_clean":"2024-09-21T07:47:10.870147+0000","last_became_active":"2024-09-21T07:35:27.870816+0000","last_became_peered":"2024-09-21T07:35:27.870816+0000","last_unstale":"2024-09-21T07:47:10.870147+0000","last_undegraded":"2024-09-21T07:47:10.870147+0000","last_fullsized":"2024-09-21T07:47:10.870147+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:04.787450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":270,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638312+0000","last_change":"2024-09-21T07:35:29.491874+0000","last_active":"2024-09-21T07:47:10.638312+0000","last_peered":"2024-09-21T07:47:10.638312+0000","last_clean":"2024-09-21T07:47:10.638312+0000","last_became_active":"2024-09-21T07:35:26.866381+0000","last_became_peered":"2024-09-21T07:35:26.866381+0000","last_unstale":"2024-09-21T07:47:10.638312+0000","last_undegraded":"2024-09-21T07:47:10.638312+0000","last_fullsized":"2024-09-21T07:47:10.638312+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:36.404647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073235399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":324,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638735+0000","last_change":"2024-09-21T07:34:24.260046+0000","last_active":"2024-09-21T07:47:10.638735+0000","last_peered":"2024-09-21T07:47:10.638735+0000","last_clean":"2024-09-21T07:47:10.638735+0000","last_became_active":"2024-09-21T07:34:23.273431+0000","last_became_peered":"2024-09-21T07:34:23.273431+0000","last_unstale":"2024-09-21T07:47:10.638735+0000","last_undegraded":"2024-09-21T07:47:10.638735+0000","last_fullsized":"2024-09-21T07:47:10.638735+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:18.876057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019803310000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.637588+0000","last_change":"2024-09-21T07:47:09.654242+0000","last_active":"2024-09-21T07:47:10.637588+0000","last_peered":"2024-09-21T07:47:10.637588+0000","last_clean":"2024-09-21T07:47:10.637588+0000","last_became_active":"2024-09-21T07:47:09.653937+0000","last_became_peered":"2024-09-21T07:47:09.653937+0000","last_unstale":"2024-09-21T07:47:10.637588+0000","last_undegraded":"2024-09-21T07:47:10.637588+0000","last_fullsized":"2024-09-21T07:47:10.637588+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:33.116186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"193'3","reported_seq":18,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638849+0000","last_change":"2024-09-21T07:47:08.630406+0000","last_active":"2024-09-21T07:47:10.638849+0000","last_peered":"2024-09-21T07:47:10.638849+0000","last_clean":"2024-09-21T07:47:10.638849+0000","last_became_active":"2024-09-21T07:47:08.629896+0000","last_became_peered":"2024-09-21T07:47:08.629896+0000","last_unstale":"2024-09-21T07:47:10.638849+0000","last_undegraded":"2024-09-21T07:47:10.638849+0000","last_fullsized":"2024-09-21T07:47:10.638849+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:26:34.127156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":260,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.872652+0000","last_change":"2024-09-21T07:35:27.871736+0000","last_active":"2024-09-21T07:47:10.872652+0000","last_peered":"2024-09-21T07:47:10.872652+0000","last_clean":"2024-09-21T07:47:10.872652+0000","last_became_active":"2024-09-21T07:35:27.871232+0000","last_became_peered":"2024-09-21T07:35:27.871232+0000","last_unstale":"2024-09-21T07:47:10.872652+0000","last_undegraded":"2024-09-21T07:47:10.872652+0000","last_fullsized":"2024-09-21T07:47:10.872652+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:04.419301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":270,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.637602+0000","last_change":"2024-09-21T07:35:29.491246+0000","last_active":"2024-09-21T07:47:10.637602+0000","last_peered":"2024-09-21T07:47:10.637602+0000","last_clean":"2024-09-21T07:47:10.637602+0000","last_became_active":"2024-09-21T07:35:26.866783+0000","last_became_peered":"2024-09-21T07:35:26.866783+0000","last_unstale":"2024-09-21T07:47:10.637602+0000","last_undegraded":"2024-09-21T07:47:10.637602+0000","last_fullsized":"2024-09-21T07:47:10.637602+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:28.881914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098474099999999992,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":321,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.872774+0000","last_change":"2024-09-21T07:34:25.028640+0000","last_active":"2024-09-21T07:47:10.872774+0000","last_peered":"2024-09-21T07:47:10.872774+0000","last_clean":"2024-09-21T07:47:10.872774+0000","last_became_active":"2024-09-21T07:34:23.260950+0000","last_became_peered":"2024-09-21T07:34:23.260950+0000","last_unstale":"2024-09-21T07:47:10.872774+0000","last_undegraded":"2024-09-21T07:47:10.872774+0000","last_fullsized":"2024-09-21T07:47:10.872774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:23:42.121398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055135099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870971+0000","last_change":"2024-09-21T07:47:09.642435+0000","last_active":"2024-09-21T07:47:10.870971+0000","last_peered":"2024-09-21T07:47:10.870971+0000","last_clean":"2024-09-21T07:47:10.870971+0000","last_became_active":"2024-09-21T07:47:09.642088+0000","last_became_peered":"2024-09-21T07:47:09.642088+0000","last_unstale":"2024-09-21T07:47:10.870971+0000","last_undegraded":"2024-09-21T07:47:10.870971+0000","last_fullsized":"2024-09-21T07:47:10.870971+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:23.496379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"193'6","reported_seq":21,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638301+0000","last_change":"2024-09-21T07:47:08.645704+0000","last_active":"2024-09-21T07:47:10.638301+0000","last_peered":"2024-09-21T07:47:10.638301+0000","last_clean":"2024-09-21T07:47:10.638301+0000","last_became_active":"2024-09-21T07:47:08.645094+0000","last_became_peered":"2024-09-21T07:47:08.645094+0000","last_unstale":"2024-09-21T07:47:10.638301+0000","last_undegraded":"2024-09-21T07:47:10.638301+0000","last_fullsized":"2024-09-21T07:47:10.638301+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:29:23.257974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":262,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871517+0000","last_change":"2024-09-21T07:35:27.872027+0000","last_active":"2024-09-21T07:47:10.871517+0000","last_peered":"2024-09-21T07:47:10.871517+0000","last_clean":"2024-09-21T07:47:10.871517+0000","last_became_active":"2024-09-21T07:35:27.870426+0000","last_became_peered":"2024-09-21T07:35:27.870426+0000","last_unstale":"2024-09-21T07:47:10.871517+0000","last_undegraded":"2024-09-21T07:47:10.871517+0000","last_fullsized":"2024-09-21T07:47:10.871517+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":270,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.637481+0000","last_change":"2024-09-21T07:35:29.491008+0000","last_active":"2024-09-21T07:47:10.637481+0000","last_peered":"2024-09-21T07:47:10.637481+0000","last_clean":"2024-09-21T07:47:10.637481+0000","last_became_active":"2024-09-21T07:35:26.866750+0000","last_became_peered":"2024-09-21T07:35:26.866750+0000","last_unstale":"2024-09-21T07:47:10.637481+0000","last_undegraded":"2024-09-21T07:47:10.637481+0000","last_fullsized":"2024-09-21T07:47:10.637481+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:25.490406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083529700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":323,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.640741+0000","last_change":"2024-09-21T07:34:24.260665+0000","last_active":"2024-09-21T07:47:10.640741+0000","last_peered":"2024-09-21T07:47:10.640741+0000","last_clean":"2024-09-21T07:47:10.640741+0000","last_became_active":"2024-09-21T07:34:23.635909+0000","last_became_peered":"2024-09-21T07:34:23.635909+0000","last_unstale":"2024-09-21T07:47:10.640741+0000","last_undegraded":"2024-09-21T07:47:10.640741+0000","last_fullsized":"2024-09-21T07:47:10.640741+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:44.536885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.872447+0000","last_change":"2024-09-21T07:47:09.648922+0000","last_active":"2024-09-21T07:47:10.872447+0000","last_peered":"2024-09-21T07:47:10.872447+0000","last_clean":"2024-09-21T07:47:10.872447+0000","last_became_active":"2024-09-21T07:47:09.648605+0000","last_became_peered":"2024-09-21T07:47:09.648605+0000","last_unstale":"2024-09-21T07:47:10.872447+0000","last_undegraded":"2024-09-21T07:47:10.872447+0000","last_fullsized":"2024-09-21T07:47:10.872447+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:02:32.488578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"193'1","reported_seq":16,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.873609+0000","last_change":"2024-09-21T07:47:08.642374+0000","last_active":"2024-09-21T07:47:10.873609+0000","last_peered":"2024-09-21T07:47:10.873609+0000","last_clean":"2024-09-21T07:47:10.873609+0000","last_became_active":"2024-09-21T07:47:08.642036+0000","last_became_peered":"2024-09-21T07:47:08.642036+0000","last_unstale":"2024-09-21T07:47:10.873609+0000","last_undegraded":"2024-09-21T07:47:10.873609+0000","last_fullsized":"2024-09-21T07:47:10.873609+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:43.823469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":258,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.637985+0000","last_change":"2024-09-21T07:35:27.874772+0000","last_active":"2024-09-21T07:47:10.637985+0000","last_peered":"2024-09-21T07:47:10.637985+0000","last_clean":"2024-09-21T07:47:10.637985+0000","last_became_active":"2024-09-21T07:35:27.874434+0000","last_became_peered":"2024-09-21T07:35:27.874434+0000","last_unstale":"2024-09-21T07:47:10.637985+0000","last_undegraded":"2024-09-21T07:47:10.637985+0000","last_fullsized":"2024-09-21T07:47:10.637985+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:31.998755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":276,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871147+0000","last_change":"2024-09-21T07:35:28.859153+0000","last_active":"2024-09-21T07:47:10.871147+0000","last_peered":"2024-09-21T07:47:10.871147+0000","last_clean":"2024-09-21T07:47:10.871147+0000","last_became_active":"2024-09-21T07:35:26.874428+0000","last_became_peered":"2024-09-21T07:35:26.874428+0000","last_unstale":"2024-09-21T07:47:10.871147+0000","last_undegraded":"2024-09-21T07:47:10.871147+0000","last_fullsized":"2024-09-21T07:47:10.871147+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:04.487621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048557299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":328,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870726+0000","last_change":"2024-09-21T07:34:24.538845+0000","last_active":"2024-09-21T07:47:10.870726+0000","last_peered":"2024-09-21T07:47:10.870726+0000","last_clean":"2024-09-21T07:47:10.870726+0000","last_became_active":"2024-09-21T07:34:23.636372+0000","last_became_peered":"2024-09-21T07:34:23.636372+0000","last_unstale":"2024-09-21T07:47:10.870726+0000","last_undegraded":"2024-09-21T07:47:10.870726+0000","last_fullsized":"2024-09-21T07:47:10.870726+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:29.393813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841451,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.872564+0000","last_change":"2024-09-21T07:47:09.655269+0000","last_active":"2024-09-21T07:47:10.872564+0000","last_peered":"2024-09-21T07:47:10.872564+0000","last_clean":"2024-09-21T07:47:10.872564+0000","last_became_active":"2024-09-21T07:47:09.654964+0000","last_became_peered":"2024-09-21T07:47:09.654964+0000","last_unstale":"2024-09-21T07:47:10.872564+0000","last_undegraded":"2024-09-21T07:47:10.872564+0000","last_fullsized":"2024-09-21T07:47:10.872564+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:12:05.101124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"193'1","reported_seq":16,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870249+0000","last_change":"2024-09-21T07:47:08.645839+0000","last_active":"2024-09-21T07:47:10.870249+0000","last_peered":"2024-09-21T07:47:10.870249+0000","last_clean":"2024-09-21T07:47:10.870249+0000","last_became_active":"2024-09-21T07:47:08.645549+0000","last_became_peered":"2024-09-21T07:47:08.645549+0000","last_unstale":"2024-09-21T07:47:10.870249+0000","last_undegraded":"2024-09-21T07:47:10.870249+0000","last_fullsized":"2024-09-21T07:47:10.870249+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:01:03.949942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":262,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869781+0000","last_change":"2024-09-21T07:35:27.877801+0000","last_active":"2024-09-21T07:47:10.869781+0000","last_peered":"2024-09-21T07:47:10.869781+0000","last_clean":"2024-09-21T07:47:10.869781+0000","last_became_active":"2024-09-21T07:35:27.877464+0000","last_became_peered":"2024-09-21T07:35:27.877464+0000","last_unstale":"2024-09-21T07:47:10.869781+0000","last_undegraded":"2024-09-21T07:47:10.869781+0000","last_fullsized":"2024-09-21T07:47:10.869781+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:08.814929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":267,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.872393+0000","last_change":"2024-09-21T07:35:28.863732+0000","last_active":"2024-09-21T07:47:10.872393+0000","last_peered":"2024-09-21T07:47:10.872393+0000","last_clean":"2024-09-21T07:47:10.872393+0000","last_became_active":"2024-09-21T07:35:26.877392+0000","last_became_peered":"2024-09-21T07:35:26.877392+0000","last_unstale":"2024-09-21T07:47:10.872393+0000","last_undegraded":"2024-09-21T07:47:10.872393+0000","last_fullsized":"2024-09-21T07:47:10.872393+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071425700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":324,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638424+0000","last_change":"2024-09-21T07:34:24.259686+0000","last_active":"2024-09-21T07:47:10.638424+0000","last_peered":"2024-09-21T07:47:10.638424+0000","last_clean":"2024-09-21T07:47:10.638424+0000","last_became_active":"2024-09-21T07:34:23.274405+0000","last_became_peered":"2024-09-21T07:34:23.274405+0000","last_unstale":"2024-09-21T07:47:10.638424+0000","last_undegraded":"2024-09-21T07:47:10.638424+0000","last_fullsized":"2024-09-21T07:47:10.638424+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:26.304693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022009920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.873340+0000","last_change":"2024-09-21T07:47:09.631732+0000","last_active":"2024-09-21T07:47:10.873340+0000","last_peered":"2024-09-21T07:47:10.873340+0000","last_clean":"2024-09-21T07:47:10.873340+0000","last_became_active":"2024-09-21T07:47:09.631428+0000","last_became_peered":"2024-09-21T07:47:09.631428+0000","last_unstale":"2024-09-21T07:47:10.873340+0000","last_undegraded":"2024-09-21T07:47:10.873340+0000","last_fullsized":"2024-09-21T07:47:10.873340+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:46.092559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":15,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870495+0000","last_change":"2024-09-21T07:47:08.630888+0000","last_active":"2024-09-21T07:47:10.870495+0000","last_peered":"2024-09-21T07:47:10.870495+0000","last_clean":"2024-09-21T07:47:10.870495+0000","last_became_active":"2024-09-21T07:47:08.630558+0000","last_became_peered":"2024-09-21T07:47:08.630558+0000","last_unstale":"2024-09-21T07:47:10.870495+0000","last_undegraded":"2024-09-21T07:47:10.870495+0000","last_fullsized":"2024-09-21T07:47:10.870495+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:24.480307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":258,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638322+0000","last_change":"2024-09-21T07:35:27.867336+0000","last_active":"2024-09-21T07:47:10.638322+0000","last_peered":"2024-09-21T07:47:10.638322+0000","last_clean":"2024-09-21T07:47:10.638322+0000","last_became_active":"2024-09-21T07:35:27.867015+0000","last_became_peered":"2024-09-21T07:35:27.867015+0000","last_unstale":"2024-09-21T07:47:10.638322+0000","last_undegraded":"2024-09-21T07:47:10.638322+0000","last_fullsized":"2024-09-21T07:47:10.638322+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:59.214139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":269,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.640634+0000","last_change":"2024-09-21T07:35:29.493137+0000","last_active":"2024-09-21T07:47:10.640634+0000","last_peered":"2024-09-21T07:47:10.640634+0000","last_clean":"2024-09-21T07:47:10.640634+0000","last_became_active":"2024-09-21T07:35:26.872230+0000","last_became_peered":"2024-09-21T07:35:26.872230+0000","last_unstale":"2024-09-21T07:47:10.640634+0000","last_undegraded":"2024-09-21T07:47:10.640634+0000","last_fullsized":"2024-09-21T07:47:10.640634+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:24.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001197449,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":322,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638215+0000","last_change":"2024-09-21T07:34:24.258754+0000","last_active":"2024-09-21T07:47:10.638215+0000","last_peered":"2024-09-21T07:47:10.638215+0000","last_clean":"2024-09-21T07:47:10.638215+0000","last_became_active":"2024-09-21T07:34:23.272657+0000","last_became_peered":"2024-09-21T07:34:23.272657+0000","last_unstale":"2024-09-21T07:47:10.638215+0000","last_undegraded":"2024-09-21T07:47:10.638215+0000","last_fullsized":"2024-09-21T07:47:10.638215+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:05.477099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067143200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.640029+0000","last_change":"2024-09-21T07:47:09.637896+0000","last_active":"2024-09-21T07:47:10.640029+0000","last_peered":"2024-09-21T07:47:10.640029+0000","last_clean":"2024-09-21T07:47:10.640029+0000","last_became_active":"2024-09-21T07:47:09.637523+0000","last_became_peered":"2024-09-21T07:47:09.637523+0000","last_unstale":"2024-09-21T07:47:10.640029+0000","last_undegraded":"2024-09-21T07:47:10.640029+0000","last_fullsized":"2024-09-21T07:47:10.640029+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:43.508202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":15,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870737+0000","last_change":"2024-09-21T07:47:08.637339+0000","last_active":"2024-09-21T07:47:10.870737+0000","last_peered":"2024-09-21T07:47:10.870737+0000","last_clean":"2024-09-21T07:47:10.870737+0000","last_became_active":"2024-09-21T07:47:08.634751+0000","last_became_peered":"2024-09-21T07:47:08.634751+0000","last_unstale":"2024-09-21T07:47:10.870737+0000","last_undegraded":"2024-09-21T07:47:10.870737+0000","last_fullsized":"2024-09-21T07:47:10.870737+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:00:08.412196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":260,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870207+0000","last_change":"2024-09-21T07:35:27.876021+0000","last_active":"2024-09-21T07:47:10.870207+0000","last_peered":"2024-09-21T07:47:10.870207+0000","last_clean":"2024-09-21T07:47:10.870207+0000","last_became_active":"2024-09-21T07:35:27.875724+0000","last_became_peered":"2024-09-21T07:35:27.875724+0000","last_unstale":"2024-09-21T07:47:10.870207+0000","last_undegraded":"2024-09-21T07:47:10.870207+0000","last_fullsized":"2024-09-21T07:47:10.870207+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:03.022812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":269,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.640211+0000","last_change":"2024-09-21T07:35:29.492570+0000","last_active":"2024-09-21T07:47:10.640211+0000","last_peered":"2024-09-21T07:47:10.640211+0000","last_clean":"2024-09-21T07:47:10.640211+0000","last_became_active":"2024-09-21T07:35:26.874316+0000","last_became_peered":"2024-09-21T07:35:26.874316+0000","last_unstale":"2024-09-21T07:47:10.640211+0000","last_undegraded":"2024-09-21T07:47:10.640211+0000","last_fullsized":"2024-09-21T07:47:10.640211+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:16.512323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011990760000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":318,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869756+0000","last_change":"2024-09-21T07:34:25.085232+0000","last_active":"2024-09-21T07:47:10.869756+0000","last_peered":"2024-09-21T07:47:10.869756+0000","last_clean":"2024-09-21T07:47:10.869756+0000","last_became_active":"2024-09-21T07:34:23.277807+0000","last_became_peered":"2024-09-21T07:34:23.277807+0000","last_unstale":"2024-09-21T07:47:10.869756+0000","last_undegraded":"2024-09-21T07:47:10.869756+0000","last_fullsized":"2024-09-21T07:47:10.869756+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:36.362919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086619099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638069+0000","last_change":"2024-09-21T07:47:09.659567+0000","last_active":"2024-09-21T07:47:10.638069+0000","last_peered":"2024-09-21T07:47:10.638069+0000","last_clean":"2024-09-21T07:47:10.638069+0000","last_became_active":"2024-09-21T07:47:09.659195+0000","last_became_peered":"2024-09-21T07:47:09.659195+0000","last_unstale":"2024-09-21T07:47:10.638069+0000","last_undegraded":"2024-09-21T07:47:10.638069+0000","last_fullsized":"2024-09-21T07:47:10.638069+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:43:36.808598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"193'4","reported_seq":19,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.639804+0000","last_change":"2024-09-21T07:47:08.641875+0000","last_active":"2024-09-21T07:47:10.639804+0000","last_peered":"2024-09-21T07:47:10.639804+0000","last_clean":"2024-09-21T07:47:10.639804+0000","last_became_active":"2024-09-21T07:47:08.641574+0000","last_became_peered":"2024-09-21T07:47:08.641574+0000","last_unstale":"2024-09-21T07:47:10.639804+0000","last_undegraded":"2024-09-21T07:47:10.639804+0000","last_fullsized":"2024-09-21T07:47:10.639804+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:26:46.004831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":262,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870409+0000","last_change":"2024-09-21T07:35:27.872798+0000","last_active":"2024-09-21T07:47:10.870409+0000","last_peered":"2024-09-21T07:47:10.870409+0000","last_clean":"2024-09-21T07:47:10.870409+0000","last_became_active":"2024-09-21T07:35:27.872483+0000","last_became_peered":"2024-09-21T07:35:27.872483+0000","last_unstale":"2024-09-21T07:47:10.870409+0000","last_undegraded":"2024-09-21T07:47:10.870409+0000","last_fullsized":"2024-09-21T07:47:10.870409+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:11.933469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":269,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869680+0000","last_change":"2024-09-21T07:35:29.206723+0000","last_active":"2024-09-21T07:47:10.869680+0000","last_peered":"2024-09-21T07:47:10.869680+0000","last_clean":"2024-09-21T07:47:10.869680+0000","last_became_active":"2024-09-21T07:35:26.872377+0000","last_became_peered":"2024-09-21T07:35:26.872377+0000","last_unstale":"2024-09-21T07:47:10.869680+0000","last_undegraded":"2024-09-21T07:47:10.869680+0000","last_fullsized":"2024-09-21T07:47:10.869680+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:29.086862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083077200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":314,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.639394+0000","last_change":"2024-09-21T07:34:24.597672+0000","last_active":"2024-09-21T07:47:10.639394+0000","last_peered":"2024-09-21T07:47:10.639394+0000","last_clean":"2024-09-21T07:47:10.639394+0000","last_became_active":"2024-09-21T07:34:23.267770+0000","last_became_peered":"2024-09-21T07:34:23.267770+0000","last_unstale":"2024-09-21T07:47:10.639394+0000","last_undegraded":"2024-09-21T07:47:10.639394+0000","last_fullsized":"2024-09-21T07:47:10.639394+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:51.629476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086378500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.637484+0000","last_change":"2024-09-21T07:47:09.653648+0000","last_active":"2024-09-21T07:47:10.637484+0000","last_peered":"2024-09-21T07:47:10.637484+0000","last_clean":"2024-09-21T07:47:10.637484+0000","last_became_active":"2024-09-21T07:47:09.653308+0000","last_became_peered":"2024-09-21T07:47:09.653308+0000","last_unstale":"2024-09-21T07:47:10.637484+0000","last_undegraded":"2024-09-21T07:47:10.637484+0000","last_fullsized":"2024-09-21T07:47:10.637484+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:54:51.139860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"193'1","reported_seq":16,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870599+0000","last_change":"2024-09-21T07:47:08.649376+0000","last_active":"2024-09-21T07:47:10.870599+0000","last_peered":"2024-09-21T07:47:10.870599+0000","last_clean":"2024-09-21T07:47:10.870599+0000","last_became_active":"2024-09-21T07:47:08.649012+0000","last_became_peered":"2024-09-21T07:47:08.649012+0000","last_unstale":"2024-09-21T07:47:10.870599+0000","last_undegraded":"2024-09-21T07:47:10.870599+0000","last_fullsized":"2024-09-21T07:47:10.870599+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:58:31.201708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":262,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870318+0000","last_change":"2024-09-21T07:35:27.870121+0000","last_active":"2024-09-21T07:47:10.870318+0000","last_peered":"2024-09-21T07:47:10.870318+0000","last_clean":"2024-09-21T07:47:10.870318+0000","last_became_active":"2024-09-21T07:35:27.869811+0000","last_became_peered":"2024-09-21T07:35:27.869811+0000","last_unstale":"2024-09-21T07:47:10.870318+0000","last_undegraded":"2024-09-21T07:47:10.870318+0000","last_fullsized":"2024-09-21T07:47:10.870318+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:31.778461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":269,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871134+0000","last_change":"2024-09-21T07:35:28.860699+0000","last_active":"2024-09-21T07:47:10.871134+0000","last_peered":"2024-09-21T07:47:10.871134+0000","last_clean":"2024-09-21T07:47:10.871134+0000","last_became_active":"2024-09-21T07:35:26.876779+0000","last_became_peered":"2024-09-21T07:35:26.876779+0000","last_unstale":"2024-09-21T07:47:10.871134+0000","last_undegraded":"2024-09-21T07:47:10.871134+0000","last_fullsized":"2024-09-21T07:47:10.871134+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:28.286403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000913188,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":321,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.873714+0000","last_change":"2024-09-21T07:34:24.260181+0000","last_active":"2024-09-21T07:47:10.873714+0000","last_peered":"2024-09-21T07:47:10.873714+0000","last_clean":"2024-09-21T07:47:10.873714+0000","last_became_active":"2024-09-21T07:34:23.275580+0000","last_became_peered":"2024-09-21T07:34:23.275580+0000","last_unstale":"2024-09-21T07:47:10.873714+0000","last_undegraded":"2024-09-21T07:47:10.873714+0000","last_fullsized":"2024-09-21T07:47:10.873714+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:49.857622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014706980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870577+0000","last_change":"2024-09-21T07:47:09.645794+0000","last_active":"2024-09-21T07:47:10.870577+0000","last_peered":"2024-09-21T07:47:10.870577+0000","last_clean":"2024-09-21T07:47:10.870577+0000","last_became_active":"2024-09-21T07:47:09.644072+0000","last_became_peered":"2024-09-21T07:47:09.644072+0000","last_unstale":"2024-09-21T07:47:10.870577+0000","last_undegraded":"2024-09-21T07:47:10.870577+0000","last_fullsized":"2024-09-21T07:47:10.870577+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:35:07.815426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"193'2","reported_seq":17,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870781+0000","last_change":"2024-09-21T07:47:08.629617+0000","last_active":"2024-09-21T07:47:10.870781+0000","last_peered":"2024-09-21T07:47:10.870781+0000","last_clean":"2024-09-21T07:47:10.870781+0000","last_became_active":"2024-09-21T07:47:08.629299+0000","last_became_peered":"2024-09-21T07:47:08.629299+0000","last_unstale":"2024-09-21T07:47:10.870781+0000","last_undegraded":"2024-09-21T07:47:10.870781+0000","last_fullsized":"2024-09-21T07:47:10.870781+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:51:56.256805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":262,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.874413+0000","last_change":"2024-09-21T07:35:27.875393+0000","last_active":"2024-09-21T07:47:10.874413+0000","last_peered":"2024-09-21T07:47:10.874413+0000","last_clean":"2024-09-21T07:47:10.874413+0000","last_became_active":"2024-09-21T07:35:27.874671+0000","last_became_peered":"2024-09-21T07:35:27.874671+0000","last_unstale":"2024-09-21T07:47:10.874413+0000","last_undegraded":"2024-09-21T07:47:10.874413+0000","last_fullsized":"2024-09-21T07:47:10.874413+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":269,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870060+0000","last_change":"2024-09-21T07:35:29.206296+0000","last_active":"2024-09-21T07:47:10.870060+0000","last_peered":"2024-09-21T07:47:10.870060+0000","last_clean":"2024-09-21T07:47:10.870060+0000","last_became_active":"2024-09-21T07:35:26.874575+0000","last_became_peered":"2024-09-21T07:35:26.874575+0000","last_unstale":"2024-09-21T07:47:10.870060+0000","last_undegraded":"2024-09-21T07:47:10.870060+0000","last_fullsized":"2024-09-21T07:47:10.870060+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:22.771154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061011599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":324,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638788+0000","last_change":"2024-09-21T07:34:24.259112+0000","last_active":"2024-09-21T07:47:10.638788+0000","last_peered":"2024-09-21T07:47:10.638788+0000","last_clean":"2024-09-21T07:47:10.638788+0000","last_became_active":"2024-09-21T07:34:23.635444+0000","last_became_peered":"2024-09-21T07:34:23.635444+0000","last_unstale":"2024-09-21T07:47:10.638788+0000","last_undegraded":"2024-09-21T07:47:10.638788+0000","last_fullsized":"2024-09-21T07:47:10.638788+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:46.255385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001952636,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.640174+0000","last_change":"2024-09-21T07:47:09.661164+0000","last_active":"2024-09-21T07:47:10.640174+0000","last_peered":"2024-09-21T07:47:10.640174+0000","last_clean":"2024-09-21T07:47:10.640174+0000","last_became_active":"2024-09-21T07:47:09.660874+0000","last_became_peered":"2024-09-21T07:47:09.660874+0000","last_unstale":"2024-09-21T07:47:10.640174+0000","last_undegraded":"2024-09-21T07:47:10.640174+0000","last_fullsized":"2024-09-21T07:47:10.640174+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:58:32.208386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"193'2","reported_seq":17,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871829+0000","last_change":"2024-09-21T07:47:08.631059+0000","last_active":"2024-09-21T07:47:10.871829+0000","last_peered":"2024-09-21T07:47:10.871829+0000","last_clean":"2024-09-21T07:47:10.871829+0000","last_became_active":"2024-09-21T07:47:08.630725+0000","last_became_peered":"2024-09-21T07:47:08.630725+0000","last_unstale":"2024-09-21T07:47:10.871829+0000","last_undegraded":"2024-09-21T07:47:10.871829+0000","last_fullsized":"2024-09-21T07:47:10.871829+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:37.178741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":262,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.874498+0000","last_change":"2024-09-21T07:35:27.878356+0000","last_active":"2024-09-21T07:47:10.874498+0000","last_peered":"2024-09-21T07:47:10.874498+0000","last_clean":"2024-09-21T07:47:10.874498+0000","last_became_active":"2024-09-21T07:35:27.877982+0000","last_became_peered":"2024-09-21T07:35:27.877982+0000","last_unstale":"2024-09-21T07:47:10.874498+0000","last_undegraded":"2024-09-21T07:47:10.874498+0000","last_fullsized":"2024-09-21T07:47:10.874498+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":269,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871675+0000","last_change":"2024-09-21T07:35:29.053131+0000","last_active":"2024-09-21T07:47:10.871675+0000","last_peered":"2024-09-21T07:47:10.871675+0000","last_clean":"2024-09-21T07:47:10.871675+0000","last_became_active":"2024-09-21T07:35:26.870516+0000","last_became_peered":"2024-09-21T07:35:26.870516+0000","last_unstale":"2024-09-21T07:47:10.871675+0000","last_undegraded":"2024-09-21T07:47:10.871675+0000","last_fullsized":"2024-09-21T07:47:10.871675+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:10.924491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070705699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":328,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870942+0000","last_change":"2024-09-21T07:34:24.539152+0000","last_active":"2024-09-21T07:47:10.870942+0000","last_peered":"2024-09-21T07:47:10.870942+0000","last_clean":"2024-09-21T07:47:10.870942+0000","last_became_active":"2024-09-21T07:34:23.259573+0000","last_became_peered":"2024-09-21T07:34:23.259573+0000","last_unstale":"2024-09-21T07:47:10.870942+0000","last_undegraded":"2024-09-21T07:47:10.870942+0000","last_fullsized":"2024-09-21T07:47:10.870942+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:26.986900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058108400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.639087+0000","last_change":"2024-09-21T07:47:09.644863+0000","last_active":"2024-09-21T07:47:10.639087+0000","last_peered":"2024-09-21T07:47:10.639087+0000","last_clean":"2024-09-21T07:47:10.639087+0000","last_became_active":"2024-09-21T07:47:09.644435+0000","last_became_peered":"2024-09-21T07:47:09.644435+0000","last_unstale":"2024-09-21T07:47:10.639087+0000","last_undegraded":"2024-09-21T07:47:10.639087+0000","last_fullsized":"2024-09-21T07:47:10.639087+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:02.301757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":15,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638629+0000","last_change":"2024-09-21T07:47:08.644109+0000","last_active":"2024-09-21T07:47:10.638629+0000","last_peered":"2024-09-21T07:47:10.638629+0000","last_clean":"2024-09-21T07:47:10.638629+0000","last_became_active":"2024-09-21T07:47:08.643774+0000","last_became_peered":"2024-09-21T07:47:08.643774+0000","last_unstale":"2024-09-21T07:47:10.638629+0000","last_undegraded":"2024-09-21T07:47:10.638629+0000","last_fullsized":"2024-09-21T07:47:10.638629+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:15.917170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":261,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871244+0000","last_change":"2024-09-21T07:35:27.873726+0000","last_active":"2024-09-21T07:47:10.871244+0000","last_peered":"2024-09-21T07:47:10.871244+0000","last_clean":"2024-09-21T07:47:10.871244+0000","last_became_active":"2024-09-21T07:35:27.871421+0000","last_became_peered":"2024-09-21T07:35:27.871421+0000","last_unstale":"2024-09-21T07:47:10.871244+0000","last_undegraded":"2024-09-21T07:47:10.871244+0000","last_fullsized":"2024-09-21T07:47:10.871244+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:00.168915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":269,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.873564+0000","last_change":"2024-09-21T07:35:28.861032+0000","last_active":"2024-09-21T07:47:10.873564+0000","last_peered":"2024-09-21T07:47:10.873564+0000","last_clean":"2024-09-21T07:47:10.873564+0000","last_became_active":"2024-09-21T07:35:26.873149+0000","last_became_peered":"2024-09-21T07:35:26.873149+0000","last_unstale":"2024-09-21T07:47:10.873564+0000","last_undegraded":"2024-09-21T07:47:10.873564+0000","last_fullsized":"2024-09-21T07:47:10.873564+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:59.435981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011988369999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":322,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.639760+0000","last_change":"2024-09-21T07:34:24.259902+0000","last_active":"2024-09-21T07:47:10.639760+0000","last_peered":"2024-09-21T07:47:10.639760+0000","last_clean":"2024-09-21T07:47:10.639760+0000","last_became_active":"2024-09-21T07:34:23.271902+0000","last_became_peered":"2024-09-21T07:34:23.271902+0000","last_unstale":"2024-09-21T07:47:10.639760+0000","last_undegraded":"2024-09-21T07:47:10.639760+0000","last_fullsized":"2024-09-21T07:47:10.639760+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:59.439525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001686358,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.640274+0000","last_change":"2024-09-21T07:47:09.660252+0000","last_active":"2024-09-21T07:47:10.640274+0000","last_peered":"2024-09-21T07:47:10.640274+0000","last_clean":"2024-09-21T07:47:10.640274+0000","last_became_active":"2024-09-21T07:47:09.659225+0000","last_became_peered":"2024-09-21T07:47:09.659225+0000","last_unstale":"2024-09-21T07:47:10.640274+0000","last_undegraded":"2024-09-21T07:47:10.640274+0000","last_fullsized":"2024-09-21T07:47:10.640274+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:37.104542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":15,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.637768+0000","last_change":"2024-09-21T07:47:08.643231+0000","last_active":"2024-09-21T07:47:10.637768+0000","last_peered":"2024-09-21T07:47:10.637768+0000","last_clean":"2024-09-21T07:47:10.637768+0000","last_became_active":"2024-09-21T07:47:08.642744+0000","last_became_peered":"2024-09-21T07:47:08.642744+0000","last_unstale":"2024-09-21T07:47:10.637768+0000","last_undegraded":"2024-09-21T07:47:10.637768+0000","last_fullsized":"2024-09-21T07:47:10.637768+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:12:37.799404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":259,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869429+0000","last_change":"2024-09-21T07:35:27.872637+0000","last_active":"2024-09-21T07:47:10.869429+0000","last_peered":"2024-09-21T07:47:10.869429+0000","last_clean":"2024-09-21T07:47:10.869429+0000","last_became_active":"2024-09-21T07:35:27.872149+0000","last_became_peered":"2024-09-21T07:35:27.872149+0000","last_unstale":"2024-09-21T07:47:10.869429+0000","last_undegraded":"2024-09-21T07:47:10.869429+0000","last_fullsized":"2024-09-21T07:47:10.869429+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:27.697872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":269,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.640559+0000","last_change":"2024-09-21T07:35:29.493362+0000","last_active":"2024-09-21T07:47:10.640559+0000","last_peered":"2024-09-21T07:47:10.640559+0000","last_clean":"2024-09-21T07:47:10.640559+0000","last_became_active":"2024-09-21T07:35:26.873018+0000","last_became_peered":"2024-09-21T07:35:26.873018+0000","last_unstale":"2024-09-21T07:47:10.640559+0000","last_undegraded":"2024-09-21T07:47:10.640559+0000","last_fullsized":"2024-09-21T07:47:10.640559+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:02.460557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012496510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":324,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638309+0000","last_change":"2024-09-21T07:34:24.259662+0000","last_active":"2024-09-21T07:47:10.638309+0000","last_peered":"2024-09-21T07:47:10.638309+0000","last_clean":"2024-09-21T07:47:10.638309+0000","last_became_active":"2024-09-21T07:34:23.278812+0000","last_became_peered":"2024-09-21T07:34:23.278812+0000","last_unstale":"2024-09-21T07:47:10.638309+0000","last_undegraded":"2024-09-21T07:47:10.638309+0000","last_fullsized":"2024-09-21T07:47:10.638309+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:49.829974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356463,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638295+0000","last_change":"2024-09-21T07:47:09.646277+0000","last_active":"2024-09-21T07:47:10.638295+0000","last_peered":"2024-09-21T07:47:10.638295+0000","last_clean":"2024-09-21T07:47:10.638295+0000","last_became_active":"2024-09-21T07:47:09.646018+0000","last_became_peered":"2024-09-21T07:47:09.646018+0000","last_unstale":"2024-09-21T07:47:10.638295+0000","last_undegraded":"2024-09-21T07:47:10.638295+0000","last_fullsized":"2024-09-21T07:47:10.638295+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:09:52.658128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":15,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871446+0000","last_change":"2024-09-21T07:47:08.628798+0000","last_active":"2024-09-21T07:47:10.871446+0000","last_peered":"2024-09-21T07:47:10.871446+0000","last_clean":"2024-09-21T07:47:10.871446+0000","last_became_active":"2024-09-21T07:47:08.626396+0000","last_became_peered":"2024-09-21T07:47:08.626396+0000","last_unstale":"2024-09-21T07:47:10.871446+0000","last_undegraded":"2024-09-21T07:47:10.871446+0000","last_fullsized":"2024-09-21T07:47:10.871446+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:37.215849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":257,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638213+0000","last_change":"2024-09-21T07:35:27.868448+0000","last_active":"2024-09-21T07:47:10.638213+0000","last_peered":"2024-09-21T07:47:10.638213+0000","last_clean":"2024-09-21T07:47:10.638213+0000","last_became_active":"2024-09-21T07:35:27.868105+0000","last_became_peered":"2024-09-21T07:35:27.868105+0000","last_unstale":"2024-09-21T07:47:10.638213+0000","last_undegraded":"2024-09-21T07:47:10.638213+0000","last_fullsized":"2024-09-21T07:47:10.638213+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:41.468702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":268,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.640367+0000","last_change":"2024-09-21T07:35:29.492974+0000","last_active":"2024-09-21T07:47:10.640367+0000","last_peered":"2024-09-21T07:47:10.640367+0000","last_clean":"2024-09-21T07:47:10.640367+0000","last_became_active":"2024-09-21T07:35:26.872165+0000","last_became_peered":"2024-09-21T07:35:26.872165+0000","last_unstale":"2024-09-21T07:47:10.640367+0000","last_undegraded":"2024-09-21T07:47:10.640367+0000","last_fullsized":"2024-09-21T07:47:10.640367+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:59.912520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013795179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":318,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869348+0000","last_change":"2024-09-21T07:34:25.084861+0000","last_active":"2024-09-21T07:47:10.869348+0000","last_peered":"2024-09-21T07:47:10.869348+0000","last_clean":"2024-09-21T07:47:10.869348+0000","last_became_active":"2024-09-21T07:34:23.277708+0000","last_became_peered":"2024-09-21T07:34:23.277708+0000","last_unstale":"2024-09-21T07:47:10.869348+0000","last_undegraded":"2024-09-21T07:47:10.869348+0000","last_fullsized":"2024-09-21T07:47:10.869348+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:40.816488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000597566,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.873480+0000","last_change":"2024-09-21T07:47:09.642462+0000","last_active":"2024-09-21T07:47:10.873480+0000","last_peered":"2024-09-21T07:47:10.873480+0000","last_clean":"2024-09-21T07:47:10.873480+0000","last_became_active":"2024-09-21T07:47:09.642106+0000","last_became_peered":"2024-09-21T07:47:09.642106+0000","last_unstale":"2024-09-21T07:47:10.873480+0000","last_undegraded":"2024-09-21T07:47:10.873480+0000","last_fullsized":"2024-09-21T07:47:10.873480+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:14.002845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":15,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869092+0000","last_change":"2024-09-21T07:47:08.631894+0000","last_active":"2024-09-21T07:47:10.869092+0000","last_peered":"2024-09-21T07:47:10.869092+0000","last_clean":"2024-09-21T07:47:10.869092+0000","last_became_active":"2024-09-21T07:47:08.630134+0000","last_became_peered":"2024-09-21T07:47:08.630134+0000","last_unstale":"2024-09-21T07:47:10.869092+0000","last_undegraded":"2024-09-21T07:47:10.869092+0000","last_fullsized":"2024-09-21T07:47:10.869092+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:17.848186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":257,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638686+0000","last_change":"2024-09-21T07:35:27.874640+0000","last_active":"2024-09-21T07:47:10.638686+0000","last_peered":"2024-09-21T07:47:10.638686+0000","last_clean":"2024-09-21T07:47:10.638686+0000","last_became_active":"2024-09-21T07:35:27.874265+0000","last_became_peered":"2024-09-21T07:35:27.874265+0000","last_unstale":"2024-09-21T07:47:10.638686+0000","last_undegraded":"2024-09-21T07:47:10.638686+0000","last_fullsized":"2024-09-21T07:47:10.638686+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:45.754654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":268,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870515+0000","last_change":"2024-09-21T07:35:29.207392+0000","last_active":"2024-09-21T07:47:10.870515+0000","last_peered":"2024-09-21T07:47:10.870515+0000","last_clean":"2024-09-21T07:47:10.870515+0000","last_became_active":"2024-09-21T07:35:26.870917+0000","last_became_peered":"2024-09-21T07:35:26.870917+0000","last_unstale":"2024-09-21T07:47:10.870515+0000","last_undegraded":"2024-09-21T07:47:10.870515+0000","last_fullsized":"2024-09-21T07:47:10.870515+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:59.159937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059198499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":324,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.637715+0000","last_change":"2024-09-21T07:34:24.258709+0000","last_active":"2024-09-21T07:47:10.637715+0000","last_peered":"2024-09-21T07:47:10.637715+0000","last_clean":"2024-09-21T07:47:10.637715+0000","last_became_active":"2024-09-21T07:34:23.279042+0000","last_became_peered":"2024-09-21T07:34:23.279042+0000","last_unstale":"2024-09-21T07:47:10.637715+0000","last_undegraded":"2024-09-21T07:47:10.637715+0000","last_fullsized":"2024-09-21T07:47:10.637715+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:01.762117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019282100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870020+0000","last_change":"2024-09-21T07:47:09.625256+0000","last_active":"2024-09-21T07:47:10.870020+0000","last_peered":"2024-09-21T07:47:10.870020+0000","last_clean":"2024-09-21T07:47:10.870020+0000","last_became_active":"2024-09-21T07:47:09.624902+0000","last_became_peered":"2024-09-21T07:47:09.624902+0000","last_unstale":"2024-09-21T07:47:10.870020+0000","last_undegraded":"2024-09-21T07:47:10.870020+0000","last_fullsized":"2024-09-21T07:47:10.870020+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:43:19.340627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"193'2","reported_seq":17,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870403+0000","last_change":"2024-09-21T07:47:08.631365+0000","last_active":"2024-09-21T07:47:10.870403+0000","last_peered":"2024-09-21T07:47:10.870403+0000","last_clean":"2024-09-21T07:47:10.870403+0000","last_became_active":"2024-09-21T07:47:08.631036+0000","last_became_peered":"2024-09-21T07:47:08.631036+0000","last_unstale":"2024-09-21T07:47:10.870403+0000","last_undegraded":"2024-09-21T07:47:10.870403+0000","last_fullsized":"2024-09-21T07:47:10.870403+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:36.924632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":259,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869872+0000","last_change":"2024-09-21T07:35:27.870260+0000","last_active":"2024-09-21T07:47:10.869872+0000","last_peered":"2024-09-21T07:47:10.869872+0000","last_clean":"2024-09-21T07:47:10.869872+0000","last_became_active":"2024-09-21T07:35:27.869448+0000","last_became_peered":"2024-09-21T07:35:27.869448+0000","last_unstale":"2024-09-21T07:47:10.869872+0000","last_undegraded":"2024-09-21T07:47:10.869872+0000","last_fullsized":"2024-09-21T07:47:10.869872+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:16.946790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":264,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638648+0000","last_change":"2024-09-21T07:35:29.386367+0000","last_active":"2024-09-21T07:47:10.638648+0000","last_peered":"2024-09-21T07:47:10.638648+0000","last_clean":"2024-09-21T07:47:10.638648+0000","last_became_active":"2024-09-21T07:35:26.879238+0000","last_became_peered":"2024-09-21T07:35:26.879238+0000","last_unstale":"2024-09-21T07:47:10.638648+0000","last_undegraded":"2024-09-21T07:47:10.638648+0000","last_fullsized":"2024-09-21T07:47:10.638648+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:16.528260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092223100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":318,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.872551+0000","last_change":"2024-09-21T07:34:25.085635+0000","last_active":"2024-09-21T07:47:10.872551+0000","last_peered":"2024-09-21T07:47:10.872551+0000","last_clean":"2024-09-21T07:47:10.872551+0000","last_became_active":"2024-09-21T07:34:23.273566+0000","last_became_peered":"2024-09-21T07:34:23.273566+0000","last_unstale":"2024-09-21T07:47:10.872551+0000","last_undegraded":"2024-09-21T07:47:10.872551+0000","last_fullsized":"2024-09-21T07:47:10.872551+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:14.142272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079897300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871363+0000","last_change":"2024-09-21T07:47:09.656317+0000","last_active":"2024-09-21T07:47:10.871363+0000","last_peered":"2024-09-21T07:47:10.871363+0000","last_clean":"2024-09-21T07:47:10.871363+0000","last_became_active":"2024-09-21T07:47:09.655954+0000","last_became_peered":"2024-09-21T07:47:09.655954+0000","last_unstale":"2024-09-21T07:47:10.871363+0000","last_undegraded":"2024-09-21T07:47:10.871363+0000","last_fullsized":"2024-09-21T07:47:10.871363+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:03.400838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"194'1","reported_seq":16,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:11.618663+0000","last_change":"2024-09-21T07:47:08.645742+0000","last_active":"2024-09-21T07:47:11.618663+0000","last_peered":"2024-09-21T07:47:11.618663+0000","last_clean":"2024-09-21T07:47:11.618663+0000","last_became_active":"2024-09-21T07:47:08.645220+0000","last_became_peered":"2024-09-21T07:47:08.645220+0000","last_unstale":"2024-09-21T07:47:11.618663+0000","last_undegraded":"2024-09-21T07:47:11.618663+0000","last_fullsized":"2024-09-21T07:47:11.618663+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:08:59.514757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":259,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.873010+0000","last_change":"2024-09-21T07:35:27.878728+0000","last_active":"2024-09-21T07:47:10.873010+0000","last_peered":"2024-09-21T07:47:10.873010+0000","last_clean":"2024-09-21T07:47:10.873010+0000","last_became_active":"2024-09-21T07:35:27.878335+0000","last_became_peered":"2024-09-21T07:35:27.878335+0000","last_unstale":"2024-09-21T07:47:10.873010+0000","last_undegraded":"2024-09-21T07:47:10.873010+0000","last_fullsized":"2024-09-21T07:47:10.873010+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:25:37.731771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":275,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870640+0000","last_change":"2024-09-21T07:35:28.858418+0000","last_active":"2024-09-21T07:47:10.870640+0000","last_peered":"2024-09-21T07:47:10.870640+0000","last_clean":"2024-09-21T07:47:10.870640+0000","last_became_active":"2024-09-21T07:35:26.876944+0000","last_became_peered":"2024-09-21T07:35:26.876944+0000","last_unstale":"2024-09-21T07:47:10.870640+0000","last_undegraded":"2024-09-21T07:47:10.870640+0000","last_fullsized":"2024-09-21T07:47:10.870640+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:04.514304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090862799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":314,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.640105+0000","last_change":"2024-09-21T07:34:24.597926+0000","last_active":"2024-09-21T07:47:10.640105+0000","last_peered":"2024-09-21T07:47:10.640105+0000","last_clean":"2024-09-21T07:47:10.640105+0000","last_became_active":"2024-09-21T07:34:23.268926+0000","last_became_peered":"2024-09-21T07:34:23.268926+0000","last_unstale":"2024-09-21T07:47:10.640105+0000","last_undegraded":"2024-09-21T07:47:10.640105+0000","last_fullsized":"2024-09-21T07:47:10.640105+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:46.190752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101851,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870946+0000","last_change":"2024-09-21T07:47:09.626390+0000","last_active":"2024-09-21T07:47:10.870946+0000","last_peered":"2024-09-21T07:47:10.870946+0000","last_clean":"2024-09-21T07:47:10.870946+0000","last_became_active":"2024-09-21T07:47:09.626041+0000","last_became_peered":"2024-09-21T07:47:09.626041+0000","last_unstale":"2024-09-21T07:47:10.870946+0000","last_undegraded":"2024-09-21T07:47:10.870946+0000","last_fullsized":"2024-09-21T07:47:10.870946+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:54:53.082878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":15,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638366+0000","last_change":"2024-09-21T07:47:08.635992+0000","last_active":"2024-09-21T07:47:10.638366+0000","last_peered":"2024-09-21T07:47:10.638366+0000","last_clean":"2024-09-21T07:47:10.638366+0000","last_became_active":"2024-09-21T07:47:08.635692+0000","last_became_peered":"2024-09-21T07:47:08.635692+0000","last_unstale":"2024-09-21T07:47:10.638366+0000","last_undegraded":"2024-09-21T07:47:10.638366+0000","last_fullsized":"2024-09-21T07:47:10.638366+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:10:08.663465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":258,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.639081+0000","last_change":"2024-09-21T07:35:27.877426+0000","last_active":"2024-09-21T07:47:10.639081+0000","last_peered":"2024-09-21T07:47:10.639081+0000","last_clean":"2024-09-21T07:47:10.639081+0000","last_became_active":"2024-09-21T07:35:27.877050+0000","last_became_peered":"2024-09-21T07:35:27.877050+0000","last_unstale":"2024-09-21T07:47:10.639081+0000","last_undegraded":"2024-09-21T07:47:10.639081+0000","last_fullsized":"2024-09-21T07:47:10.639081+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:36.268334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":266,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869544+0000","last_change":"2024-09-21T07:35:28.862273+0000","last_active":"2024-09-21T07:47:10.869544+0000","last_peered":"2024-09-21T07:47:10.869544+0000","last_clean":"2024-09-21T07:47:10.869544+0000","last_became_active":"2024-09-21T07:35:26.874011+0000","last_became_peered":"2024-09-21T07:35:26.874011+0000","last_unstale":"2024-09-21T07:47:10.869544+0000","last_undegraded":"2024-09-21T07:47:10.869544+0000","last_fullsized":"2024-09-21T07:47:10.869544+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:11.986950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071644799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":321,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.872044+0000","last_change":"2024-09-21T07:34:24.259794+0000","last_active":"2024-09-21T07:47:10.872044+0000","last_peered":"2024-09-21T07:47:10.872044+0000","last_clean":"2024-09-21T07:47:10.872044+0000","last_became_active":"2024-09-21T07:34:23.265679+0000","last_became_peered":"2024-09-21T07:34:23.265679+0000","last_unstale":"2024-09-21T07:47:10.872044+0000","last_undegraded":"2024-09-21T07:47:10.872044+0000","last_fullsized":"2024-09-21T07:47:10.872044+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:02.575796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001298518,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.873523+0000","last_change":"2024-09-21T07:47:09.647670+0000","last_active":"2024-09-21T07:47:10.873523+0000","last_peered":"2024-09-21T07:47:10.873523+0000","last_clean":"2024-09-21T07:47:10.873523+0000","last_became_active":"2024-09-21T07:47:09.647386+0000","last_became_peered":"2024-09-21T07:47:09.647386+0000","last_unstale":"2024-09-21T07:47:10.873523+0000","last_undegraded":"2024-09-21T07:47:10.873523+0000","last_fullsized":"2024-09-21T07:47:10.873523+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:40:35.420329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":15,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871854+0000","last_change":"2024-09-21T07:47:08.634935+0000","last_active":"2024-09-21T07:47:10.871854+0000","last_peered":"2024-09-21T07:47:10.871854+0000","last_clean":"2024-09-21T07:47:10.871854+0000","last_became_active":"2024-09-21T07:47:08.634627+0000","last_became_peered":"2024-09-21T07:47:08.634627+0000","last_unstale":"2024-09-21T07:47:10.871854+0000","last_undegraded":"2024-09-21T07:47:10.871854+0000","last_fullsized":"2024-09-21T07:47:10.871854+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:17.077512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":262,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871229+0000","last_change":"2024-09-21T07:35:27.877725+0000","last_active":"2024-09-21T07:47:10.871229+0000","last_peered":"2024-09-21T07:47:10.871229+0000","last_clean":"2024-09-21T07:47:10.871229+0000","last_became_active":"2024-09-21T07:35:27.877398+0000","last_became_peered":"2024-09-21T07:35:27.877398+0000","last_unstale":"2024-09-21T07:47:10.871229+0000","last_undegraded":"2024-09-21T07:47:10.871229+0000","last_fullsized":"2024-09-21T07:47:10.871229+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:05.859762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":268,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869613+0000","last_change":"2024-09-21T07:35:28.860612+0000","last_active":"2024-09-21T07:47:10.869613+0000","last_peered":"2024-09-21T07:47:10.869613+0000","last_clean":"2024-09-21T07:47:10.869613+0000","last_became_active":"2024-09-21T07:35:26.876562+0000","last_became_peered":"2024-09-21T07:35:26.876562+0000","last_unstale":"2024-09-21T07:47:10.869613+0000","last_undegraded":"2024-09-21T07:47:10.869613+0000","last_fullsized":"2024-09-21T07:47:10.869613+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:06.824331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012252470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":322,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.640664+0000","last_change":"2024-09-21T07:34:24.260330+0000","last_active":"2024-09-21T07:47:10.640664+0000","last_peered":"2024-09-21T07:47:10.640664+0000","last_clean":"2024-09-21T07:47:10.640664+0000","last_became_active":"2024-09-21T07:34:23.637210+0000","last_became_peered":"2024-09-21T07:34:23.637210+0000","last_unstale":"2024-09-21T07:47:10.640664+0000","last_undegraded":"2024-09-21T07:47:10.640664+0000","last_fullsized":"2024-09-21T07:47:10.640664+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.488451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001537093,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638496+0000","last_change":"2024-09-21T07:47:09.646594+0000","last_active":"2024-09-21T07:47:10.638496+0000","last_peered":"2024-09-21T07:47:10.638496+0000","last_clean":"2024-09-21T07:47:10.638496+0000","last_became_active":"2024-09-21T07:47:09.645970+0000","last_became_peered":"2024-09-21T07:47:09.645970+0000","last_unstale":"2024-09-21T07:47:10.638496+0000","last_undegraded":"2024-09-21T07:47:10.638496+0000","last_fullsized":"2024-09-21T07:47:10.638496+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:52.093212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"193'2","reported_seq":17,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870067+0000","last_change":"2024-09-21T07:47:08.639174+0000","last_active":"2024-09-21T07:47:10.870067+0000","last_peered":"2024-09-21T07:47:10.870067+0000","last_clean":"2024-09-21T07:47:10.870067+0000","last_became_active":"2024-09-21T07:47:08.638830+0000","last_became_peered":"2024-09-21T07:47:08.638830+0000","last_unstale":"2024-09-21T07:47:10.870067+0000","last_undegraded":"2024-09-21T07:47:10.870067+0000","last_fullsized":"2024-09-21T07:47:10.870067+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:42.824859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":260,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869100+0000","last_change":"2024-09-21T07:35:27.874757+0000","last_active":"2024-09-21T07:47:10.869100+0000","last_peered":"2024-09-21T07:47:10.869100+0000","last_clean":"2024-09-21T07:47:10.869100+0000","last_became_active":"2024-09-21T07:35:27.874360+0000","last_became_peered":"2024-09-21T07:35:27.874360+0000","last_unstale":"2024-09-21T07:47:10.869100+0000","last_undegraded":"2024-09-21T07:47:10.869100+0000","last_fullsized":"2024-09-21T07:47:10.869100+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:17.646912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":268,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.640925+0000","last_change":"2024-09-21T07:35:29.493431+0000","last_active":"2024-09-21T07:47:10.640925+0000","last_peered":"2024-09-21T07:47:10.640925+0000","last_clean":"2024-09-21T07:47:10.640925+0000","last_became_active":"2024-09-21T07:35:26.873691+0000","last_became_peered":"2024-09-21T07:35:26.873691+0000","last_unstale":"2024-09-21T07:47:10.640925+0000","last_undegraded":"2024-09-21T07:47:10.640925+0000","last_fullsized":"2024-09-21T07:47:10.640925+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:51.705051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001162464,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":314,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638440+0000","last_change":"2024-09-21T07:34:24.597286+0000","last_active":"2024-09-21T07:47:10.638440+0000","last_peered":"2024-09-21T07:47:10.638440+0000","last_clean":"2024-09-21T07:47:10.638440+0000","last_became_active":"2024-09-21T07:34:23.267133+0000","last_became_peered":"2024-09-21T07:34:23.267133+0000","last_unstale":"2024-09-21T07:47:10.638440+0000","last_undegraded":"2024-09-21T07:47:10.638440+0000","last_fullsized":"2024-09-21T07:47:10.638440+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:18.257010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.872293+0000","last_change":"2024-09-21T07:47:09.664200+0000","last_active":"2024-09-21T07:47:10.872293+0000","last_peered":"2024-09-21T07:47:10.872293+0000","last_clean":"2024-09-21T07:47:10.872293+0000","last_became_active":"2024-09-21T07:47:09.663850+0000","last_became_peered":"2024-09-21T07:47:09.663850+0000","last_unstale":"2024-09-21T07:47:10.872293+0000","last_undegraded":"2024-09-21T07:47:10.872293+0000","last_fullsized":"2024-09-21T07:47:10.872293+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:13:19.001858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":15,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870512+0000","last_change":"2024-09-21T07:47:08.638721+0000","last_active":"2024-09-21T07:47:10.870512+0000","last_peered":"2024-09-21T07:47:10.870512+0000","last_clean":"2024-09-21T07:47:10.870512+0000","last_became_active":"2024-09-21T07:47:08.638320+0000","last_became_peered":"2024-09-21T07:47:08.638320+0000","last_unstale":"2024-09-21T07:47:10.870512+0000","last_undegraded":"2024-09-21T07:47:10.870512+0000","last_fullsized":"2024-09-21T07:47:10.870512+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:00.571043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":269,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871600+0000","last_change":"2024-09-21T07:35:27.864458+0000","last_active":"2024-09-21T07:47:10.871600+0000","last_peered":"2024-09-21T07:47:10.871600+0000","last_clean":"2024-09-21T07:47:10.871600+0000","last_became_active":"2024-09-21T07:35:27.864136+0000","last_became_peered":"2024-09-21T07:35:27.864136+0000","last_unstale":"2024-09-21T07:47:10.871600+0000","last_undegraded":"2024-09-21T07:47:10.871600+0000","last_fullsized":"2024-09-21T07:47:10.871600+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:20.162240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":275,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871257+0000","last_change":"2024-09-21T07:35:28.858784+0000","last_active":"2024-09-21T07:47:10.871257+0000","last_peered":"2024-09-21T07:47:10.871257+0000","last_clean":"2024-09-21T07:47:10.871257+0000","last_became_active":"2024-09-21T07:35:26.876890+0000","last_became_peered":"2024-09-21T07:35:26.876890+0000","last_unstale":"2024-09-21T07:47:10.871257+0000","last_undegraded":"2024-09-21T07:47:10.871257+0000","last_fullsized":"2024-09-21T07:47:10.871257+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:40.738327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064543499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":324,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638180+0000","last_change":"2024-09-21T07:34:24.258712+0000","last_active":"2024-09-21T07:47:10.638180+0000","last_peered":"2024-09-21T07:47:10.638180+0000","last_clean":"2024-09-21T07:47:10.638180+0000","last_became_active":"2024-09-21T07:34:23.274692+0000","last_became_peered":"2024-09-21T07:34:23.274692+0000","last_unstale":"2024-09-21T07:47:10.638180+0000","last_undegraded":"2024-09-21T07:47:10.638180+0000","last_fullsized":"2024-09-21T07:47:10.638180+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:43.279454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018051479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870518+0000","last_change":"2024-09-21T07:47:09.650215+0000","last_active":"2024-09-21T07:47:10.870518+0000","last_peered":"2024-09-21T07:47:10.870518+0000","last_clean":"2024-09-21T07:47:10.870518+0000","last_became_active":"2024-09-21T07:47:09.649552+0000","last_became_peered":"2024-09-21T07:47:09.649552+0000","last_unstale":"2024-09-21T07:47:10.870518+0000","last_undegraded":"2024-09-21T07:47:10.870518+0000","last_fullsized":"2024-09-21T07:47:10.870518+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:39.943614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":15,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870698+0000","last_change":"2024-09-21T07:47:08.626468+0000","last_active":"2024-09-21T07:47:10.870698+0000","last_peered":"2024-09-21T07:47:10.870698+0000","last_clean":"2024-09-21T07:47:10.870698+0000","last_became_active":"2024-09-21T07:47:08.626131+0000","last_became_peered":"2024-09-21T07:47:08.626131+0000","last_unstale":"2024-09-21T07:47:10.870698+0000","last_undegraded":"2024-09-21T07:47:10.870698+0000","last_fullsized":"2024-09-21T07:47:10.870698+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:18.852388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":263,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.637904+0000","last_change":"2024-09-21T07:35:27.878020+0000","last_active":"2024-09-21T07:47:10.637904+0000","last_peered":"2024-09-21T07:47:10.637904+0000","last_clean":"2024-09-21T07:47:10.637904+0000","last_became_active":"2024-09-21T07:35:27.877532+0000","last_became_peered":"2024-09-21T07:35:27.877532+0000","last_unstale":"2024-09-21T07:47:10.637904+0000","last_undegraded":"2024-09-21T07:47:10.637904+0000","last_fullsized":"2024-09-21T07:47:10.637904+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:19.756073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":268,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.872738+0000","last_change":"2024-09-21T07:35:28.861049+0000","last_active":"2024-09-21T07:47:10.872738+0000","last_peered":"2024-09-21T07:47:10.872738+0000","last_clean":"2024-09-21T07:47:10.872738+0000","last_became_active":"2024-09-21T07:35:26.876819+0000","last_became_peered":"2024-09-21T07:35:26.876819+0000","last_unstale":"2024-09-21T07:47:10.872738+0000","last_undegraded":"2024-09-21T07:47:10.872738+0000","last_fullsized":"2024-09-21T07:47:10.872738+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:19.633803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055780599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":318,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869280+0000","last_change":"2024-09-21T07:34:25.084849+0000","last_active":"2024-09-21T07:47:10.869280+0000","last_peered":"2024-09-21T07:47:10.869280+0000","last_clean":"2024-09-21T07:47:10.869280+0000","last_became_active":"2024-09-21T07:34:23.277713+0000","last_became_peered":"2024-09-21T07:34:23.277713+0000","last_unstale":"2024-09-21T07:47:10.869280+0000","last_undegraded":"2024-09-21T07:47:10.869280+0000","last_fullsized":"2024-09-21T07:47:10.869280+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:05.811810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059079499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.640785+0000","last_change":"2024-09-21T07:47:09.643720+0000","last_active":"2024-09-21T07:47:10.640785+0000","last_peered":"2024-09-21T07:47:10.640785+0000","last_clean":"2024-09-21T07:47:10.640785+0000","last_became_active":"2024-09-21T07:47:09.643421+0000","last_became_peered":"2024-09-21T07:47:09.643421+0000","last_unstale":"2024-09-21T07:47:10.640785+0000","last_undegraded":"2024-09-21T07:47:10.640785+0000","last_fullsized":"2024-09-21T07:47:10.640785+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:01:04.956620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":15,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638057+0000","last_change":"2024-09-21T07:47:08.628390+0000","last_active":"2024-09-21T07:47:10.638057+0000","last_peered":"2024-09-21T07:47:10.638057+0000","last_clean":"2024-09-21T07:47:10.638057+0000","last_became_active":"2024-09-21T07:47:08.628015+0000","last_became_peered":"2024-09-21T07:47:08.628015+0000","last_unstale":"2024-09-21T07:47:10.638057+0000","last_undegraded":"2024-09-21T07:47:10.638057+0000","last_fullsized":"2024-09-21T07:47:10.638057+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:15.112979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":262,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.873712+0000","last_change":"2024-09-21T07:35:27.884248+0000","last_active":"2024-09-21T07:47:10.873712+0000","last_peered":"2024-09-21T07:47:10.873712+0000","last_clean":"2024-09-21T07:47:10.873712+0000","last_became_active":"2024-09-21T07:35:27.883861+0000","last_became_peered":"2024-09-21T07:35:27.883861+0000","last_unstale":"2024-09-21T07:47:10.873712+0000","last_undegraded":"2024-09-21T07:47:10.873712+0000","last_fullsized":"2024-09-21T07:47:10.873712+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":269,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870468+0000","last_change":"2024-09-21T07:35:29.207135+0000","last_active":"2024-09-21T07:47:10.870468+0000","last_peered":"2024-09-21T07:47:10.870468+0000","last_clean":"2024-09-21T07:47:10.870468+0000","last_became_active":"2024-09-21T07:35:26.872787+0000","last_became_peered":"2024-09-21T07:35:26.872787+0000","last_unstale":"2024-09-21T07:47:10.870468+0000","last_undegraded":"2024-09-21T07:47:10.870468+0000","last_fullsized":"2024-09-21T07:47:10.870468+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:27.543217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011157509999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":328,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871681+0000","last_change":"2024-09-21T07:34:24.538610+0000","last_active":"2024-09-21T07:47:10.871681+0000","last_peered":"2024-09-21T07:47:10.871681+0000","last_clean":"2024-09-21T07:47:10.871681+0000","last_became_active":"2024-09-21T07:34:23.259700+0000","last_became_peered":"2024-09-21T07:34:23.259700+0000","last_unstale":"2024-09-21T07:47:10.871681+0000","last_undegraded":"2024-09-21T07:47:10.871681+0000","last_fullsized":"2024-09-21T07:47:10.871681+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:57.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070522399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.873170+0000","last_change":"2024-09-21T07:47:09.641729+0000","last_active":"2024-09-21T07:47:10.873170+0000","last_peered":"2024-09-21T07:47:10.873170+0000","last_clean":"2024-09-21T07:47:10.873170+0000","last_became_active":"2024-09-21T07:47:09.641130+0000","last_became_peered":"2024-09-21T07:47:09.641130+0000","last_unstale":"2024-09-21T07:47:10.873170+0000","last_undegraded":"2024-09-21T07:47:10.873170+0000","last_fullsized":"2024-09-21T07:47:10.873170+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:00:03.886850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"193'1","reported_seq":16,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638773+0000","last_change":"2024-09-21T07:47:08.644619+0000","last_active":"2024-09-21T07:47:10.638773+0000","last_peered":"2024-09-21T07:47:10.638773+0000","last_clean":"2024-09-21T07:47:10.638773+0000","last_became_active":"2024-09-21T07:47:08.642613+0000","last_became_peered":"2024-09-21T07:47:08.642613+0000","last_unstale":"2024-09-21T07:47:10.638773+0000","last_undegraded":"2024-09-21T07:47:10.638773+0000","last_fullsized":"2024-09-21T07:47:10.638773+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:02:30.516252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":269,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870288+0000","last_change":"2024-09-21T07:35:27.868024+0000","last_active":"2024-09-21T07:47:10.870288+0000","last_peered":"2024-09-21T07:47:10.870288+0000","last_clean":"2024-09-21T07:47:10.870288+0000","last_became_active":"2024-09-21T07:35:27.867339+0000","last_became_peered":"2024-09-21T07:35:27.867339+0000","last_unstale":"2024-09-21T07:47:10.870288+0000","last_undegraded":"2024-09-21T07:47:10.870288+0000","last_fullsized":"2024-09-21T07:47:10.870288+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":276,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870718+0000","last_change":"2024-09-21T07:35:28.858505+0000","last_active":"2024-09-21T07:47:10.870718+0000","last_peered":"2024-09-21T07:47:10.870718+0000","last_clean":"2024-09-21T07:47:10.870718+0000","last_became_active":"2024-09-21T07:35:26.877320+0000","last_became_peered":"2024-09-21T07:35:26.877320+0000","last_unstale":"2024-09-21T07:47:10.870718+0000","last_undegraded":"2024-09-21T07:47:10.870718+0000","last_fullsized":"2024-09-21T07:47:10.870718+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:53.672782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244119999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":315,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.639400+0000","last_change":"2024-09-21T07:34:24.597597+0000","last_active":"2024-09-21T07:47:10.639400+0000","last_peered":"2024-09-21T07:47:10.639400+0000","last_clean":"2024-09-21T07:47:10.639400+0000","last_became_active":"2024-09-21T07:34:23.635204+0000","last_became_peered":"2024-09-21T07:34:23.635204+0000","last_unstale":"2024-09-21T07:47:10.639400+0000","last_undegraded":"2024-09-21T07:47:10.639400+0000","last_fullsized":"2024-09-21T07:47:10.639400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:48.644789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081139199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869986+0000","last_change":"2024-09-21T07:47:09.635405+0000","last_active":"2024-09-21T07:47:10.869986+0000","last_peered":"2024-09-21T07:47:10.869986+0000","last_clean":"2024-09-21T07:47:10.869986+0000","last_became_active":"2024-09-21T07:47:09.634326+0000","last_became_peered":"2024-09-21T07:47:09.634326+0000","last_unstale":"2024-09-21T07:47:10.869986+0000","last_undegraded":"2024-09-21T07:47:10.869986+0000","last_fullsized":"2024-09-21T07:47:10.869986+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:46.468467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"193'2","reported_seq":17,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.874377+0000","last_change":"2024-09-21T07:47:08.633421+0000","last_active":"2024-09-21T07:47:10.874377+0000","last_peered":"2024-09-21T07:47:10.874377+0000","last_clean":"2024-09-21T07:47:10.874377+0000","last_became_active":"2024-09-21T07:47:08.633080+0000","last_became_peered":"2024-09-21T07:47:08.633080+0000","last_unstale":"2024-09-21T07:47:10.874377+0000","last_undegraded":"2024-09-21T07:47:10.874377+0000","last_fullsized":"2024-09-21T07:47:10.874377+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:19.537094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":261,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870241+0000","last_change":"2024-09-21T07:35:27.877503+0000","last_active":"2024-09-21T07:47:10.870241+0000","last_peered":"2024-09-21T07:47:10.870241+0000","last_clean":"2024-09-21T07:47:10.870241+0000","last_became_active":"2024-09-21T07:35:27.875775+0000","last_became_peered":"2024-09-21T07:35:27.875775+0000","last_unstale":"2024-09-21T07:47:10.870241+0000","last_undegraded":"2024-09-21T07:47:10.870241+0000","last_fullsized":"2024-09-21T07:47:10.870241+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:49.291023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":269,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.639729+0000","last_change":"2024-09-21T07:35:29.491412+0000","last_active":"2024-09-21T07:47:10.639729+0000","last_peered":"2024-09-21T07:47:10.639729+0000","last_clean":"2024-09-21T07:47:10.639729+0000","last_became_active":"2024-09-21T07:35:26.876404+0000","last_became_peered":"2024-09-21T07:35:26.876404+0000","last_unstale":"2024-09-21T07:47:10.639729+0000","last_undegraded":"2024-09-21T07:47:10.639729+0000","last_fullsized":"2024-09-21T07:47:10.639729+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063918000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":325,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638838+0000","last_change":"2024-09-21T07:34:24.259026+0000","last_active":"2024-09-21T07:47:10.638838+0000","last_peered":"2024-09-21T07:47:10.638838+0000","last_clean":"2024-09-21T07:47:10.638838+0000","last_became_active":"2024-09-21T07:34:23.635456+0000","last_became_peered":"2024-09-21T07:34:23.635456+0000","last_unstale":"2024-09-21T07:47:10.638838+0000","last_undegraded":"2024-09-21T07:47:10.638838+0000","last_fullsized":"2024-09-21T07:47:10.638838+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:09.592752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018654419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870438+0000","last_change":"2024-09-21T07:47:09.635440+0000","last_active":"2024-09-21T07:47:10.870438+0000","last_peered":"2024-09-21T07:47:10.870438+0000","last_clean":"2024-09-21T07:47:10.870438+0000","last_became_active":"2024-09-21T07:47:09.635009+0000","last_became_peered":"2024-09-21T07:47:09.635009+0000","last_unstale":"2024-09-21T07:47:10.870438+0000","last_undegraded":"2024-09-21T07:47:10.870438+0000","last_fullsized":"2024-09-21T07:47:10.870438+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:53.955407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"193'1","reported_seq":16,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.872853+0000","last_change":"2024-09-21T07:47:08.644049+0000","last_active":"2024-09-21T07:47:10.872853+0000","last_peered":"2024-09-21T07:47:10.872853+0000","last_clean":"2024-09-21T07:47:10.872853+0000","last_became_active":"2024-09-21T07:47:08.643739+0000","last_became_peered":"2024-09-21T07:47:08.643739+0000","last_unstale":"2024-09-21T07:47:10.872853+0000","last_undegraded":"2024-09-21T07:47:10.872853+0000","last_fullsized":"2024-09-21T07:47:10.872853+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:36.097864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":261,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870867+0000","last_change":"2024-09-21T07:35:27.871984+0000","last_active":"2024-09-21T07:47:10.870867+0000","last_peered":"2024-09-21T07:47:10.870867+0000","last_clean":"2024-09-21T07:47:10.870867+0000","last_became_active":"2024-09-21T07:35:27.870336+0000","last_became_peered":"2024-09-21T07:35:27.870336+0000","last_unstale":"2024-09-21T07:47:10.870867+0000","last_undegraded":"2024-09-21T07:47:10.870867+0000","last_fullsized":"2024-09-21T07:47:10.870867+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:57.120479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":269,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870906+0000","last_change":"2024-09-21T07:35:28.860351+0000","last_active":"2024-09-21T07:47:10.870906+0000","last_peered":"2024-09-21T07:47:10.870906+0000","last_clean":"2024-09-21T07:47:10.870906+0000","last_became_active":"2024-09-21T07:35:26.872046+0000","last_became_peered":"2024-09-21T07:35:26.872046+0000","last_unstale":"2024-09-21T07:47:10.870906+0000","last_undegraded":"2024-09-21T07:47:10.870906+0000","last_fullsized":"2024-09-21T07:47:10.870906+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:47.074327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098384399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":322,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.640432+0000","last_change":"2024-09-21T07:34:24.260300+0000","last_active":"2024-09-21T07:47:10.640432+0000","last_peered":"2024-09-21T07:47:10.640432+0000","last_clean":"2024-09-21T07:47:10.640432+0000","last_became_active":"2024-09-21T07:34:23.270128+0000","last_became_peered":"2024-09-21T07:34:23.270128+0000","last_unstale":"2024-09-21T07:47:10.640432+0000","last_undegraded":"2024-09-21T07:47:10.640432+0000","last_fullsized":"2024-09-21T07:47:10.640432+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:25.260931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00176651,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869988+0000","last_change":"2024-09-21T07:47:09.641302+0000","last_active":"2024-09-21T07:47:10.869988+0000","last_peered":"2024-09-21T07:47:10.869988+0000","last_clean":"2024-09-21T07:47:10.869988+0000","last_became_active":"2024-09-21T07:47:09.640989+0000","last_became_peered":"2024-09-21T07:47:09.640989+0000","last_unstale":"2024-09-21T07:47:10.869988+0000","last_undegraded":"2024-09-21T07:47:10.869988+0000","last_fullsized":"2024-09-21T07:47:10.869988+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:54:27.135545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":15,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.872669+0000","last_change":"2024-09-21T07:47:08.641672+0000","last_active":"2024-09-21T07:47:10.872669+0000","last_peered":"2024-09-21T07:47:10.872669+0000","last_clean":"2024-09-21T07:47:10.872669+0000","last_became_active":"2024-09-21T07:47:08.641351+0000","last_became_peered":"2024-09-21T07:47:08.641351+0000","last_unstale":"2024-09-21T07:47:10.872669+0000","last_undegraded":"2024-09-21T07:47:10.872669+0000","last_fullsized":"2024-09-21T07:47:10.872669+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:02:42.466658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":261,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871333+0000","last_change":"2024-09-21T07:35:27.877861+0000","last_active":"2024-09-21T07:47:10.871333+0000","last_peered":"2024-09-21T07:47:10.871333+0000","last_clean":"2024-09-21T07:47:10.871333+0000","last_became_active":"2024-09-21T07:35:27.877434+0000","last_became_peered":"2024-09-21T07:35:27.877434+0000","last_unstale":"2024-09-21T07:47:10.871333+0000","last_undegraded":"2024-09-21T07:47:10.871333+0000","last_fullsized":"2024-09-21T07:47:10.871333+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":267,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869461+0000","last_change":"2024-09-21T07:35:28.862954+0000","last_active":"2024-09-21T07:47:10.869461+0000","last_peered":"2024-09-21T07:47:10.869461+0000","last_clean":"2024-09-21T07:47:10.869461+0000","last_became_active":"2024-09-21T07:35:26.870613+0000","last_became_peered":"2024-09-21T07:35:26.870613+0000","last_unstale":"2024-09-21T07:47:10.869461+0000","last_undegraded":"2024-09-21T07:47:10.869461+0000","last_fullsized":"2024-09-21T07:47:10.869461+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:06.901043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059842100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":320,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.874114+0000","last_change":"2024-09-21T07:34:25.028904+0000","last_active":"2024-09-21T07:47:10.874114+0000","last_peered":"2024-09-21T07:47:10.874114+0000","last_clean":"2024-09-21T07:47:10.874114+0000","last_became_active":"2024-09-21T07:34:23.260942+0000","last_became_peered":"2024-09-21T07:34:23.260942+0000","last_unstale":"2024-09-21T07:47:10.874114+0000","last_undegraded":"2024-09-21T07:47:10.874114+0000","last_fullsized":"2024-09-21T07:47:10.874114+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:22.709328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060553899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509613,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66080,"kb_used_data":2956,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94305760,"statfs":{"total":96636764160,"available":96569098240,"internally_reserved":0,"allocated":3026944,"data_stored":1884957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509615,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51616,"kb_used_data":2376,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320224,"statfs":{"total":96636764160,"available":96583909376,"internally_reserved":0,"allocated":2433024,"data_stored":1294589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509613,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29416,"kb_used_data":2384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342424,"statfs":{"total":96636764160,"available":96606642176,"internally_reserved":0,"allocated":2441216,"data_stored":1294713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509614,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46048,"kb_used_data":2376,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94325792,"statfs":{"total":96636764160,"available":96589611008,"internally_reserved":0,"allocated":2433024,"data_stored":1294589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509615,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41900,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329940,"statfs":{"total":96636764160,"available":96593858560,"internally_reserved":0,"allocated":2445312,"data_stored":1296201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509615,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52208,"kb_used_data":2964,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319632,"statfs":{"total":96636764160,"available":96583303168,"internally_reserved":0,"allocated":3035136,"data_stored":1885061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542318,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51636,"kb_used_data":2392,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320204,"statfs":{"total":96636764160,"available":96583888896,"internally_reserved":0,"allocated":2449408,"data_stored":1295353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542317,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73844,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297996,"statfs":{"total":96636764160,"available":96561147904,"internally_reserved":0,"allocated":2453504,"data_stored":1296805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:47:14.119 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:47:14.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.378+0000 7fdbb3a00640 1 -- 172.21.15.39:0/2942945873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbac0d0b70 msgr2=0x7fdbac0b43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:14.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.378+0000 7fdbb3a00640 1 --2- 172.21.15.39:0/2942945873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbac0d0b70 0x7fdbac0b43d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fdba4005fd0 tx=0x7fdba402a8b0 comp rx=0 tx=0).stop 2024-09-21T07:47:14.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.378+0000 7fdbb3a00640 1 -- 172.21.15.39:0/2942945873 shutdown_connections 2024-09-21T07:47:14.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.378+0000 7fdbb3a00640 1 --2- 172.21.15.39:0/2942945873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbac0d0b70 0x7fdbac0b43d0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:14.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.378+0000 7fdbb3a00640 1 --2- 172.21.15.39:0/2942945873 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdbac15c690 0x7fdbac0d0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:14.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.378+0000 7fdbb3a00640 1 -- 172.21.15.39:0/2942945873 >> 172.21.15.39:0/2942945873 conn(0x7fdbac06cfc0 msgr2=0x7fdbac06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:14.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.378+0000 7fdbb3a00640 1 -- 172.21.15.39:0/2942945873 shutdown_connections 2024-09-21T07:47:14.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.378+0000 7fdbb3a00640 1 -- 172.21.15.39:0/2942945873 wait complete. 2024-09-21T07:47:14.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.380+0000 7fdbb3a00640 1 Processor -- start 2024-09-21T07:47:14.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.381+0000 7fdbb3a00640 1 -- start start 2024-09-21T07:47:14.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.382+0000 7fdbb3a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbac0b5240 0x7fdbac14b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:14.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.382+0000 7fdbb3a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdbac0d0b70 0x7fdbac0aff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:14.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.382+0000 7fdbb3a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbac15c690 0x7fdbac1e9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:14.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.382+0000 7fdbb3a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdbac070090 con 0x7fdbac15c690 2024-09-21T07:47:14.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.382+0000 7fdbb3a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdbac06ff10 con 0x7fdbac0b5240 2024-09-21T07:47:14.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.382+0000 7fdbb3a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fdbac0b4910 con 0x7fdbac0d0b70 2024-09-21T07:47:14.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.382+0000 7fdbb1a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbac15c690 0x7fdbac1e9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:14.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.382+0000 7fdbb1000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbac0b5240 0x7fdbac14b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:14.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.382+0000 7fdbb1a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbac15c690 0x7fdbac1e9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48334/0 (socket says 172.21.15.39:48334) 2024-09-21T07:47:14.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.382+0000 7fdbb1a00640 1 -- 172.21.15.39:0/3324921120 learned_addr learned my addr 172.21.15.39:0/3324921120 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:14.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.382+0000 7fdbabe00640 1 --2- 172.21.15.39:0/3324921120 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdbac0d0b70 0x7fdbac0aff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:14.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.383+0000 7fdbb1a00640 1 -- 172.21.15.39:0/3324921120 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdbac0d0b70 msgr2=0x7fdbac0aff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:14.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.383+0000 7fdbb1a00640 1 --2- 172.21.15.39:0/3324921120 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdbac0d0b70 0x7fdbac0aff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:14.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.383+0000 7fdbb1a00640 1 -- 172.21.15.39:0/3324921120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbac0b5240 msgr2=0x7fdbac14b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:14.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.383+0000 7fdbb1a00640 1 --2- 172.21.15.39:0/3324921120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbac0b5240 0x7fdbac14b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:14.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.383+0000 7fdbb1a00640 1 -- 172.21.15.39:0/3324921120 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbac0b0480 con 0x7fdbac15c690 2024-09-21T07:47:14.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.383+0000 7fdbb1000640 1 --2- 172.21.15.39:0/3324921120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbac0b5240 0x7fdbac14b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:47:14.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.383+0000 7fdbb1a00640 1 --2- 172.21.15.39:0/3324921120 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbac15c690 0x7fdbac1e9da0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fdba4006010 tx=0x7fdba4031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:14.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.383+0000 7fdba8c00640 1 -- 172.21.15.39:0/3324921120 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba4031990 con 0x7fdbac15c690 2024-09-21T07:47:14.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.384+0000 7fdba8c00640 1 -- 172.21.15.39:0/3324921120 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdba4031c20 con 0x7fdbac15c690 2024-09-21T07:47:14.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.384+0000 7fdba8c00640 1 -- 172.21.15.39:0/3324921120 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba40330d0 con 0x7fdbac15c690 2024-09-21T07:47:14.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.384+0000 7fdbb3a00640 1 -- 172.21.15.39:0/3324921120 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbac14c260 con 0x7fdbac15c690 2024-09-21T07:47:14.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.384+0000 7fdbb3a00640 1 -- 172.21.15.39:0/3324921120 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbac1e92c0 con 0x7fdbac15c690 2024-09-21T07:47:14.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.385+0000 7fdbb3a00640 1 -- 172.21.15.39:0/3324921120 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbac0b4e60 con 0x7fdbac15c690 2024-09-21T07:47:14.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.386+0000 7fdba8c00640 1 -- 172.21.15.39:0/3324921120 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdba4033270 con 0x7fdbac15c690 2024-09-21T07:47:14.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.387+0000 7fdba8c00640 1 --2- 172.21.15.39:0/3324921120 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdb8c078bc0 0x7fdb8c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:14.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.387+0000 7fdba8c00640 1 -- 172.21.15.39:0/3324921120 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 6932+0+0 (secure 0 0 0) 0x7fdba40bc610 con 0x7fdbac15c690 2024-09-21T07:47:14.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.388+0000 7fdbb1000640 1 --2- 172.21.15.39:0/3324921120 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdb8c078bc0 0x7fdb8c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:14.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.388+0000 7fdbb1000640 1 --2- 172.21.15.39:0/3324921120 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdb8c078bc0 0x7fdb8c07b0d0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fdba0001b10 tx=0x7fdba0009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:14.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.392+0000 7fdba8c00640 1 -- 172.21.15.39:0/3324921120 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdba403f030 con 0x7fdbac15c690 2024-09-21T07:47:14.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.586+0000 7fdbb3a00640 1 -- 172.21.15.39:0/3324921120 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdbac15b020 con 0x7fdb8c078bc0 2024-09-21T07:47:14.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.595+0000 7fdba8c00640 1 -- 172.21.15.39:0/3324921120 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+495240 (secure 0 0 0) 0x7fdbac15b020 con 0x7fdb8c078bc0 2024-09-21T07:47:14.603 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:47:14.606 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:47:14.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.600+0000 7fdbb3a00640 1 -- 172.21.15.39:0/3324921120 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdb8c078bc0 msgr2=0x7fdb8c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:14.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.600+0000 7fdbb3a00640 1 --2- 172.21.15.39:0/3324921120 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdb8c078bc0 0x7fdb8c07b0d0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fdba0001b10 tx=0x7fdba0009040 comp rx=0 tx=0).stop 2024-09-21T07:47:14.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.600+0000 7fdbb3a00640 1 -- 172.21.15.39:0/3324921120 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbac15c690 msgr2=0x7fdbac1e9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:14.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.600+0000 7fdbb3a00640 1 --2- 172.21.15.39:0/3324921120 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbac15c690 0x7fdbac1e9da0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fdba4006010 tx=0x7fdba4031700 comp rx=0 tx=0).stop 2024-09-21T07:47:14.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.600+0000 7fdbb3a00640 1 -- 172.21.15.39:0/3324921120 shutdown_connections 2024-09-21T07:47:14.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.600+0000 7fdbb3a00640 1 --2- 172.21.15.39:0/3324921120 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdb8c078bc0 0x7fdb8c07b0d0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:14.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.600+0000 7fdbb3a00640 1 --2- 172.21.15.39:0/3324921120 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbac15c690 0x7fdbac1e9da0 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:14.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.600+0000 7fdbb3a00640 1 --2- 172.21.15.39:0/3324921120 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdbac0d0b70 0x7fdbac0aff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:14.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.600+0000 7fdbb3a00640 1 --2- 172.21.15.39:0/3324921120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbac0b5240 0x7fdbac14b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:14.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.600+0000 7fdbb3a00640 1 -- 172.21.15.39:0/3324921120 >> 172.21.15.39:0/3324921120 conn(0x7fdbac06cfc0 msgr2=0x7fdbac0b33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:14.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.601+0000 7fdbb3a00640 1 -- 172.21.15.39:0/3324921120 shutdown_connections 2024-09-21T07:47:14.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.601+0000 7fdbb3a00640 1 -- 172.21.15.39:0/3324921120 wait complete. 2024-09-21T07:47:14.624 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":615,"stamp":"2024-09-21T07:47:13.817505+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":378,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":412748,"kb_used_data":20232,"kb_used_omap":12,"kb_used_meta":392307,"kb_avail":754561972,"statfs":{"total":773094113280,"available":772671459328,"internally_reserved":0,"allocated":20717568,"data_stored":11542268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":401722928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002028"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.639616+0000","last_change":"2024-09-21T07:47:09.644043+0000","last_active":"2024-09-21T07:47:10.639616+0000","last_peered":"2024-09-21T07:47:10.639616+0000","last_clean":"2024-09-21T07:47:10.639616+0000","last_became_active":"2024-09-21T07:47:09.642096+0000","last_became_peered":"2024-09-21T07:47:09.642096+0000","last_unstale":"2024-09-21T07:47:10.639616+0000","last_undegraded":"2024-09-21T07:47:10.639616+0000","last_fullsized":"2024-09-21T07:47:10.639616+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:05:21.255629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.640744+0000","last_change":"2024-09-21T07:47:09.640763+0000","last_active":"2024-09-21T07:47:10.640744+0000","last_peered":"2024-09-21T07:47:10.640744+0000","last_clean":"2024-09-21T07:47:10.640744+0000","last_became_active":"2024-09-21T07:47:09.640441+0000","last_became_peered":"2024-09-21T07:47:09.640441+0000","last_unstale":"2024-09-21T07:47:10.640744+0000","last_undegraded":"2024-09-21T07:47:10.640744+0000","last_fullsized":"2024-09-21T07:47:10.640744+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:37.931310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638820+0000","last_change":"2024-09-21T07:47:09.638610+0000","last_active":"2024-09-21T07:47:10.638820+0000","last_peered":"2024-09-21T07:47:10.638820+0000","last_clean":"2024-09-21T07:47:10.638820+0000","last_became_active":"2024-09-21T07:47:09.638166+0000","last_became_peered":"2024-09-21T07:47:09.638166+0000","last_unstale":"2024-09-21T07:47:10.638820+0000","last_undegraded":"2024-09-21T07:47:10.638820+0000","last_fullsized":"2024-09-21T07:47:10.638820+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:03.254832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.639031+0000","last_change":"2024-09-21T07:47:09.658247+0000","last_active":"2024-09-21T07:47:10.639031+0000","last_peered":"2024-09-21T07:47:10.639031+0000","last_clean":"2024-09-21T07:47:10.639031+0000","last_became_active":"2024-09-21T07:47:09.657914+0000","last_became_peered":"2024-09-21T07:47:09.657914+0000","last_unstale":"2024-09-21T07:47:10.639031+0000","last_undegraded":"2024-09-21T07:47:10.639031+0000","last_fullsized":"2024-09-21T07:47:10.639031+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:59.525973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869671+0000","last_change":"2024-09-21T07:47:09.650494+0000","last_active":"2024-09-21T07:47:10.869671+0000","last_peered":"2024-09-21T07:47:10.869671+0000","last_clean":"2024-09-21T07:47:10.869671+0000","last_became_active":"2024-09-21T07:47:09.649970+0000","last_became_peered":"2024-09-21T07:47:09.649970+0000","last_unstale":"2024-09-21T07:47:10.869671+0000","last_undegraded":"2024-09-21T07:47:10.869671+0000","last_fullsized":"2024-09-21T07:47:10.869671+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:42.077459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.639681+0000","last_change":"2024-09-21T07:47:09.643421+0000","last_active":"2024-09-21T07:47:10.639681+0000","last_peered":"2024-09-21T07:47:10.639681+0000","last_clean":"2024-09-21T07:47:10.639681+0000","last_became_active":"2024-09-21T07:47:09.643134+0000","last_became_peered":"2024-09-21T07:47:09.643134+0000","last_unstale":"2024-09-21T07:47:10.639681+0000","last_undegraded":"2024-09-21T07:47:10.639681+0000","last_fullsized":"2024-09-21T07:47:10.639681+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:29.033001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.872323+0000","last_change":"2024-09-21T07:47:09.641703+0000","last_active":"2024-09-21T07:47:10.872323+0000","last_peered":"2024-09-21T07:47:10.872323+0000","last_clean":"2024-09-21T07:47:10.872323+0000","last_became_active":"2024-09-21T07:47:09.641393+0000","last_became_peered":"2024-09-21T07:47:09.641393+0000","last_unstale":"2024-09-21T07:47:10.872323+0000","last_undegraded":"2024-09-21T07:47:10.872323+0000","last_fullsized":"2024-09-21T07:47:10.872323+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:29:26.742108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870667+0000","last_change":"2024-09-21T07:47:09.641426+0000","last_active":"2024-09-21T07:47:10.870667+0000","last_peered":"2024-09-21T07:47:10.870667+0000","last_clean":"2024-09-21T07:47:10.870667+0000","last_became_active":"2024-09-21T07:47:09.640738+0000","last_became_peered":"2024-09-21T07:47:09.640738+0000","last_unstale":"2024-09-21T07:47:10.870667+0000","last_undegraded":"2024-09-21T07:47:10.870667+0000","last_fullsized":"2024-09-21T07:47:10.870667+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:54.389102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.874339+0000","last_change":"2024-09-21T07:47:09.652547+0000","last_active":"2024-09-21T07:47:10.874339+0000","last_peered":"2024-09-21T07:47:10.874339+0000","last_clean":"2024-09-21T07:47:10.874339+0000","last_became_active":"2024-09-21T07:47:09.651506+0000","last_became_peered":"2024-09-21T07:47:09.651506+0000","last_unstale":"2024-09-21T07:47:10.874339+0000","last_undegraded":"2024-09-21T07:47:10.874339+0000","last_fullsized":"2024-09-21T07:47:10.874339+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:29.255108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.874294+0000","last_change":"2024-09-21T07:47:09.648987+0000","last_active":"2024-09-21T07:47:10.874294+0000","last_peered":"2024-09-21T07:47:10.874294+0000","last_clean":"2024-09-21T07:47:10.874294+0000","last_became_active":"2024-09-21T07:47:09.648668+0000","last_became_peered":"2024-09-21T07:47:09.648668+0000","last_unstale":"2024-09-21T07:47:10.874294+0000","last_undegraded":"2024-09-21T07:47:10.874294+0000","last_fullsized":"2024-09-21T07:47:10.874294+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:24:02.928287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869990+0000","last_change":"2024-09-21T07:47:09.642891+0000","last_active":"2024-09-21T07:47:10.869990+0000","last_peered":"2024-09-21T07:47:10.869990+0000","last_clean":"2024-09-21T07:47:10.869990+0000","last_became_active":"2024-09-21T07:47:09.642411+0000","last_became_peered":"2024-09-21T07:47:09.642411+0000","last_unstale":"2024-09-21T07:47:10.869990+0000","last_undegraded":"2024-09-21T07:47:10.869990+0000","last_fullsized":"2024-09-21T07:47:10.869990+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:59.597050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.640344+0000","last_change":"2024-09-21T07:47:09.646034+0000","last_active":"2024-09-21T07:47:10.640344+0000","last_peered":"2024-09-21T07:47:10.640344+0000","last_clean":"2024-09-21T07:47:10.640344+0000","last_became_active":"2024-09-21T07:47:09.645505+0000","last_became_peered":"2024-09-21T07:47:09.645505+0000","last_unstale":"2024-09-21T07:47:10.640344+0000","last_undegraded":"2024-09-21T07:47:10.640344+0000","last_fullsized":"2024-09-21T07:47:10.640344+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:26:46.220617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638074+0000","last_change":"2024-09-21T07:47:09.643246+0000","last_active":"2024-09-21T07:47:10.638074+0000","last_peered":"2024-09-21T07:47:10.638074+0000","last_clean":"2024-09-21T07:47:10.638074+0000","last_became_active":"2024-09-21T07:47:09.642941+0000","last_became_peered":"2024-09-21T07:47:09.642941+0000","last_unstale":"2024-09-21T07:47:10.638074+0000","last_undegraded":"2024-09-21T07:47:10.638074+0000","last_fullsized":"2024-09-21T07:47:10.638074+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:41.557690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870038+0000","last_change":"2024-09-21T07:47:09.645793+0000","last_active":"2024-09-21T07:47:10.870038+0000","last_peered":"2024-09-21T07:47:10.870038+0000","last_clean":"2024-09-21T07:47:10.870038+0000","last_became_active":"2024-09-21T07:47:09.645333+0000","last_became_peered":"2024-09-21T07:47:09.645333+0000","last_unstale":"2024-09-21T07:47:10.870038+0000","last_undegraded":"2024-09-21T07:47:10.870038+0000","last_fullsized":"2024-09-21T07:47:10.870038+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:28.245429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871248+0000","last_change":"2024-09-21T07:47:09.639290+0000","last_active":"2024-09-21T07:47:10.871248+0000","last_peered":"2024-09-21T07:47:10.871248+0000","last_clean":"2024-09-21T07:47:10.871248+0000","last_became_active":"2024-09-21T07:47:09.638942+0000","last_became_peered":"2024-09-21T07:47:09.638942+0000","last_unstale":"2024-09-21T07:47:10.871248+0000","last_undegraded":"2024-09-21T07:47:10.871248+0000","last_fullsized":"2024-09-21T07:47:10.871248+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:58.656671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638009+0000","last_change":"2024-09-21T07:47:09.659865+0000","last_active":"2024-09-21T07:47:10.638009+0000","last_peered":"2024-09-21T07:47:10.638009+0000","last_clean":"2024-09-21T07:47:10.638009+0000","last_became_active":"2024-09-21T07:47:09.659413+0000","last_became_peered":"2024-09-21T07:47:09.659413+0000","last_unstale":"2024-09-21T07:47:10.638009+0000","last_undegraded":"2024-09-21T07:47:10.638009+0000","last_fullsized":"2024-09-21T07:47:10.638009+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:26:30.060687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.874319+0000","last_change":"2024-09-21T07:47:09.651816+0000","last_active":"2024-09-21T07:47:10.874319+0000","last_peered":"2024-09-21T07:47:10.874319+0000","last_clean":"2024-09-21T07:47:10.874319+0000","last_became_active":"2024-09-21T07:47:09.651524+0000","last_became_peered":"2024-09-21T07:47:09.651524+0000","last_unstale":"2024-09-21T07:47:10.874319+0000","last_undegraded":"2024-09-21T07:47:10.874319+0000","last_fullsized":"2024-09-21T07:47:10.874319+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:52.579196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870619+0000","last_change":"2024-09-21T07:47:09.649812+0000","last_active":"2024-09-21T07:47:10.870619+0000","last_peered":"2024-09-21T07:47:10.870619+0000","last_clean":"2024-09-21T07:47:10.870619+0000","last_became_active":"2024-09-21T07:47:09.649139+0000","last_became_peered":"2024-09-21T07:47:09.649139+0000","last_unstale":"2024-09-21T07:47:10.870619+0000","last_undegraded":"2024-09-21T07:47:10.870619+0000","last_fullsized":"2024-09-21T07:47:10.870619+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:07:13.269043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870793+0000","last_change":"2024-09-21T07:47:09.652349+0000","last_active":"2024-09-21T07:47:10.870793+0000","last_peered":"2024-09-21T07:47:10.870793+0000","last_clean":"2024-09-21T07:47:10.870793+0000","last_became_active":"2024-09-21T07:47:09.652023+0000","last_became_peered":"2024-09-21T07:47:09.652023+0000","last_unstale":"2024-09-21T07:47:10.870793+0000","last_undegraded":"2024-09-21T07:47:10.870793+0000","last_fullsized":"2024-09-21T07:47:10.870793+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:53.414209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871755+0000","last_change":"2024-09-21T07:47:09.643833+0000","last_active":"2024-09-21T07:47:10.871755+0000","last_peered":"2024-09-21T07:47:10.871755+0000","last_clean":"2024-09-21T07:47:10.871755+0000","last_became_active":"2024-09-21T07:47:09.643548+0000","last_became_peered":"2024-09-21T07:47:09.643548+0000","last_unstale":"2024-09-21T07:47:10.871755+0000","last_undegraded":"2024-09-21T07:47:10.871755+0000","last_fullsized":"2024-09-21T07:47:10.871755+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:00.737900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638095+0000","last_change":"2024-09-21T07:47:09.643968+0000","last_active":"2024-09-21T07:47:10.638095+0000","last_peered":"2024-09-21T07:47:10.638095+0000","last_clean":"2024-09-21T07:47:10.638095+0000","last_became_active":"2024-09-21T07:47:09.643671+0000","last_became_peered":"2024-09-21T07:47:09.643671+0000","last_unstale":"2024-09-21T07:47:10.638095+0000","last_undegraded":"2024-09-21T07:47:10.638095+0000","last_fullsized":"2024-09-21T07:47:10.638095+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:49.250392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.639647+0000","last_change":"2024-09-21T07:47:09.660236+0000","last_active":"2024-09-21T07:47:10.639647+0000","last_peered":"2024-09-21T07:47:10.639647+0000","last_clean":"2024-09-21T07:47:10.639647+0000","last_became_active":"2024-09-21T07:47:09.659923+0000","last_became_peered":"2024-09-21T07:47:09.659923+0000","last_unstale":"2024-09-21T07:47:10.639647+0000","last_undegraded":"2024-09-21T07:47:10.639647+0000","last_fullsized":"2024-09-21T07:47:10.639647+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:24.557148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869853+0000","last_change":"2024-09-21T07:47:09.639086+0000","last_active":"2024-09-21T07:47:10.869853+0000","last_peered":"2024-09-21T07:47:10.869853+0000","last_clean":"2024-09-21T07:47:10.869853+0000","last_became_active":"2024-09-21T07:47:09.636177+0000","last_became_peered":"2024-09-21T07:47:09.636177+0000","last_unstale":"2024-09-21T07:47:10.869853+0000","last_undegraded":"2024-09-21T07:47:10.869853+0000","last_fullsized":"2024-09-21T07:47:10.869853+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:02.806011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.637648+0000","last_change":"2024-09-21T07:47:09.660424+0000","last_active":"2024-09-21T07:47:10.637648+0000","last_peered":"2024-09-21T07:47:10.637648+0000","last_clean":"2024-09-21T07:47:10.637648+0000","last_became_active":"2024-09-21T07:47:09.659909+0000","last_became_peered":"2024-09-21T07:47:09.659909+0000","last_unstale":"2024-09-21T07:47:10.637648+0000","last_undegraded":"2024-09-21T07:47:10.637648+0000","last_fullsized":"2024-09-21T07:47:10.637648+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:08.371997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638773+0000","last_change":"2024-09-21T07:47:09.640189+0000","last_active":"2024-09-21T07:47:10.638773+0000","last_peered":"2024-09-21T07:47:10.638773+0000","last_clean":"2024-09-21T07:47:10.638773+0000","last_became_active":"2024-09-21T07:47:09.639778+0000","last_became_peered":"2024-09-21T07:47:09.639778+0000","last_unstale":"2024-09-21T07:47:10.638773+0000","last_undegraded":"2024-09-21T07:47:10.638773+0000","last_fullsized":"2024-09-21T07:47:10.638773+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:19.385094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.873348+0000","last_change":"2024-09-21T07:47:09.639182+0000","last_active":"2024-09-21T07:47:10.873348+0000","last_peered":"2024-09-21T07:47:10.873348+0000","last_clean":"2024-09-21T07:47:10.873348+0000","last_became_active":"2024-09-21T07:47:09.638324+0000","last_became_peered":"2024-09-21T07:47:09.638324+0000","last_unstale":"2024-09-21T07:47:10.873348+0000","last_undegraded":"2024-09-21T07:47:10.873348+0000","last_fullsized":"2024-09-21T07:47:10.873348+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:16.359936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.640595+0000","last_change":"2024-09-21T07:47:09.660017+0000","last_active":"2024-09-21T07:47:10.640595+0000","last_peered":"2024-09-21T07:47:10.640595+0000","last_clean":"2024-09-21T07:47:10.640595+0000","last_became_active":"2024-09-21T07:47:09.659724+0000","last_became_peered":"2024-09-21T07:47:09.659724+0000","last_unstale":"2024-09-21T07:47:10.640595+0000","last_undegraded":"2024-09-21T07:47:10.640595+0000","last_fullsized":"2024-09-21T07:47:10.640595+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:43.831537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638622+0000","last_change":"2024-09-21T07:47:09.646270+0000","last_active":"2024-09-21T07:47:10.638622+0000","last_peered":"2024-09-21T07:47:10.638622+0000","last_clean":"2024-09-21T07:47:10.638622+0000","last_became_active":"2024-09-21T07:47:09.645562+0000","last_became_peered":"2024-09-21T07:47:09.645562+0000","last_unstale":"2024-09-21T07:47:10.638622+0000","last_undegraded":"2024-09-21T07:47:10.638622+0000","last_fullsized":"2024-09-21T07:47:10.638622+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:43:03.738757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.639854+0000","last_change":"2024-09-21T07:47:09.652846+0000","last_active":"2024-09-21T07:47:10.639854+0000","last_peered":"2024-09-21T07:47:10.639854+0000","last_clean":"2024-09-21T07:47:10.639854+0000","last_became_active":"2024-09-21T07:47:09.652473+0000","last_became_peered":"2024-09-21T07:47:09.652473+0000","last_unstale":"2024-09-21T07:47:10.639854+0000","last_undegraded":"2024-09-21T07:47:10.639854+0000","last_fullsized":"2024-09-21T07:47:10.639854+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:41:20.324009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869892+0000","last_change":"2024-09-21T07:47:09.639974+0000","last_active":"2024-09-21T07:47:10.869892+0000","last_peered":"2024-09-21T07:47:10.869892+0000","last_clean":"2024-09-21T07:47:10.869892+0000","last_became_active":"2024-09-21T07:47:09.639679+0000","last_became_peered":"2024-09-21T07:47:09.639679+0000","last_unstale":"2024-09-21T07:47:10.869892+0000","last_undegraded":"2024-09-21T07:47:10.869892+0000","last_fullsized":"2024-09-21T07:47:10.869892+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:42.066652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871724+0000","last_change":"2024-09-21T07:47:09.632419+0000","last_active":"2024-09-21T07:47:10.871724+0000","last_peered":"2024-09-21T07:47:10.871724+0000","last_clean":"2024-09-21T07:47:10.871724+0000","last_became_active":"2024-09-21T07:47:09.632088+0000","last_became_peered":"2024-09-21T07:47:09.632088+0000","last_unstale":"2024-09-21T07:47:10.871724+0000","last_undegraded":"2024-09-21T07:47:10.871724+0000","last_fullsized":"2024-09-21T07:47:10.871724+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:05:05.685632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869492+0000","last_change":"2024-09-21T07:47:09.635477+0000","last_active":"2024-09-21T07:47:10.869492+0000","last_peered":"2024-09-21T07:47:10.869492+0000","last_clean":"2024-09-21T07:47:10.869492+0000","last_became_active":"2024-09-21T07:47:09.635112+0000","last_became_peered":"2024-09-21T07:47:09.635112+0000","last_unstale":"2024-09-21T07:47:10.869492+0000","last_undegraded":"2024-09-21T07:47:10.869492+0000","last_fullsized":"2024-09-21T07:47:10.869492+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:03:13.741801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.637755+0000","last_change":"2024-09-21T07:47:09.639625+0000","last_active":"2024-09-21T07:47:10.637755+0000","last_peered":"2024-09-21T07:47:10.637755+0000","last_clean":"2024-09-21T07:47:10.637755+0000","last_became_active":"2024-09-21T07:47:09.639314+0000","last_became_peered":"2024-09-21T07:47:09.639314+0000","last_unstale":"2024-09-21T07:47:10.637755+0000","last_undegraded":"2024-09-21T07:47:10.637755+0000","last_fullsized":"2024-09-21T07:47:10.637755+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:20.277990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":15,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638870+0000","last_change":"2024-09-21T07:47:08.640591+0000","last_active":"2024-09-21T07:47:10.638870+0000","last_peered":"2024-09-21T07:47:10.638870+0000","last_clean":"2024-09-21T07:47:10.638870+0000","last_became_active":"2024-09-21T07:47:08.639759+0000","last_became_peered":"2024-09-21T07:47:08.639759+0000","last_unstale":"2024-09-21T07:47:10.638870+0000","last_undegraded":"2024-09-21T07:47:10.638870+0000","last_fullsized":"2024-09-21T07:47:10.638870+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:59.250592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":262,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.640502+0000","last_change":"2024-09-21T07:35:27.879605+0000","last_active":"2024-09-21T07:47:10.640502+0000","last_peered":"2024-09-21T07:47:10.640502+0000","last_clean":"2024-09-21T07:47:10.640502+0000","last_became_active":"2024-09-21T07:35:27.879274+0000","last_became_peered":"2024-09-21T07:35:27.879274+0000","last_unstale":"2024-09-21T07:47:10.640502+0000","last_undegraded":"2024-09-21T07:47:10.640502+0000","last_fullsized":"2024-09-21T07:47:10.640502+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:24.241636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":270,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.637911+0000","last_change":"2024-09-21T07:35:29.491336+0000","last_active":"2024-09-21T07:47:10.637911+0000","last_peered":"2024-09-21T07:47:10.637911+0000","last_clean":"2024-09-21T07:47:10.637911+0000","last_became_active":"2024-09-21T07:35:26.866484+0000","last_became_peered":"2024-09-21T07:35:26.866484+0000","last_unstale":"2024-09-21T07:47:10.637911+0000","last_undegraded":"2024-09-21T07:47:10.637911+0000","last_fullsized":"2024-09-21T07:47:10.637911+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:10.449665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076010100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":448,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638246+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:47:10.638246+0000","last_peered":"2024-09-21T07:47:10.638246+0000","last_clean":"2024-09-21T07:47:10.638246+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:47:10.638246+0000","last_undegraded":"2024-09-21T07:47:10.638246+0000","last_fullsized":"2024-09-21T07:47:10.638246+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":328,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871259+0000","last_change":"2024-09-21T07:34:24.539248+0000","last_active":"2024-09-21T07:47:10.871259+0000","last_peered":"2024-09-21T07:47:10.871259+0000","last_clean":"2024-09-21T07:47:10.871259+0000","last_became_active":"2024-09-21T07:34:23.273465+0000","last_became_peered":"2024-09-21T07:34:23.273465+0000","last_unstale":"2024-09-21T07:47:10.871259+0000","last_undegraded":"2024-09-21T07:47:10.871259+0000","last_fullsized":"2024-09-21T07:47:10.871259+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:55.517452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060627700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870268+0000","last_change":"2024-09-21T07:47:09.641109+0000","last_active":"2024-09-21T07:47:10.870268+0000","last_peered":"2024-09-21T07:47:10.870268+0000","last_clean":"2024-09-21T07:47:10.870268+0000","last_became_active":"2024-09-21T07:47:09.640791+0000","last_became_peered":"2024-09-21T07:47:09.640791+0000","last_unstale":"2024-09-21T07:47:10.870268+0000","last_undegraded":"2024-09-21T07:47:10.870268+0000","last_fullsized":"2024-09-21T07:47:10.870268+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:09:45.917662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"193'2","reported_seq":17,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869870+0000","last_change":"2024-09-21T07:47:08.633962+0000","last_active":"2024-09-21T07:47:10.869870+0000","last_peered":"2024-09-21T07:47:10.869870+0000","last_clean":"2024-09-21T07:47:10.869870+0000","last_became_active":"2024-09-21T07:47:08.633574+0000","last_became_peered":"2024-09-21T07:47:08.633574+0000","last_unstale":"2024-09-21T07:47:10.869870+0000","last_undegraded":"2024-09-21T07:47:10.869870+0000","last_fullsized":"2024-09-21T07:47:10.869870+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:26:45.213939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":263,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.637421+0000","last_change":"2024-09-21T07:35:27.864492+0000","last_active":"2024-09-21T07:47:10.637421+0000","last_peered":"2024-09-21T07:47:10.637421+0000","last_clean":"2024-09-21T07:47:10.637421+0000","last_became_active":"2024-09-21T07:35:27.864176+0000","last_became_peered":"2024-09-21T07:35:27.864176+0000","last_unstale":"2024-09-21T07:47:10.637421+0000","last_undegraded":"2024-09-21T07:47:10.637421+0000","last_fullsized":"2024-09-21T07:47:10.637421+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:05.153467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":269,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.640987+0000","last_change":"2024-09-21T07:35:29.493678+0000","last_active":"2024-09-21T07:47:10.640987+0000","last_peered":"2024-09-21T07:47:10.640987+0000","last_clean":"2024-09-21T07:47:10.640987+0000","last_became_active":"2024-09-21T07:35:26.870749+0000","last_became_peered":"2024-09-21T07:35:26.870749+0000","last_unstale":"2024-09-21T07:47:10.640987+0000","last_undegraded":"2024-09-21T07:47:10.640987+0000","last_fullsized":"2024-09-21T07:47:10.640987+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:55.367877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010159570000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":315,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638552+0000","last_change":"2024-09-21T07:34:24.597274+0000","last_active":"2024-09-21T07:47:10.638552+0000","last_peered":"2024-09-21T07:47:10.638552+0000","last_clean":"2024-09-21T07:47:10.638552+0000","last_became_active":"2024-09-21T07:34:23.279560+0000","last_became_peered":"2024-09-21T07:34:23.279560+0000","last_unstale":"2024-09-21T07:47:10.638552+0000","last_undegraded":"2024-09-21T07:47:10.638552+0000","last_fullsized":"2024-09-21T07:47:10.638552+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:15.845265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059699799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.873833+0000","last_change":"2024-09-21T07:47:09.657050+0000","last_active":"2024-09-21T07:47:10.873833+0000","last_peered":"2024-09-21T07:47:10.873833+0000","last_clean":"2024-09-21T07:47:10.873833+0000","last_became_active":"2024-09-21T07:47:09.656487+0000","last_became_peered":"2024-09-21T07:47:09.656487+0000","last_unstale":"2024-09-21T07:47:10.873833+0000","last_undegraded":"2024-09-21T07:47:10.873833+0000","last_fullsized":"2024-09-21T07:47:10.873833+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:19:33.447462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":15,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.637524+0000","last_change":"2024-09-21T07:47:08.636622+0000","last_active":"2024-09-21T07:47:10.637524+0000","last_peered":"2024-09-21T07:47:10.637524+0000","last_clean":"2024-09-21T07:47:10.637524+0000","last_became_active":"2024-09-21T07:47:08.636249+0000","last_became_peered":"2024-09-21T07:47:08.636249+0000","last_unstale":"2024-09-21T07:47:10.637524+0000","last_undegraded":"2024-09-21T07:47:10.637524+0000","last_fullsized":"2024-09-21T07:47:10.637524+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:32.062377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":262,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870804+0000","last_change":"2024-09-21T07:35:27.874387+0000","last_active":"2024-09-21T07:47:10.870804+0000","last_peered":"2024-09-21T07:47:10.870804+0000","last_clean":"2024-09-21T07:47:10.870804+0000","last_became_active":"2024-09-21T07:35:27.873277+0000","last_became_peered":"2024-09-21T07:35:27.873277+0000","last_unstale":"2024-09-21T07:47:10.870804+0000","last_undegraded":"2024-09-21T07:47:10.870804+0000","last_fullsized":"2024-09-21T07:47:10.870804+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:04.581463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":265,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.639200+0000","last_change":"2024-09-21T07:35:29.387094+0000","last_active":"2024-09-21T07:47:10.639200+0000","last_peered":"2024-09-21T07:47:10.639200+0000","last_clean":"2024-09-21T07:47:10.639200+0000","last_became_active":"2024-09-21T07:35:26.874757+0000","last_became_peered":"2024-09-21T07:35:26.874757+0000","last_unstale":"2024-09-21T07:47:10.639200+0000","last_undegraded":"2024-09-21T07:47:10.639200+0000","last_fullsized":"2024-09-21T07:47:10.639200+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:33.542750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061065700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":322,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.640391+0000","last_change":"2024-09-21T07:34:24.259906+0000","last_active":"2024-09-21T07:47:10.640391+0000","last_peered":"2024-09-21T07:47:10.640391+0000","last_clean":"2024-09-21T07:47:10.640391+0000","last_became_active":"2024-09-21T07:34:23.635282+0000","last_became_peered":"2024-09-21T07:34:23.635282+0000","last_unstale":"2024-09-21T07:47:10.640391+0000","last_undegraded":"2024-09-21T07:47:10.640391+0000","last_fullsized":"2024-09-21T07:47:10.640391+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:00.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001527664,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871182+0000","last_change":"2024-09-21T07:47:09.636769+0000","last_active":"2024-09-21T07:47:10.871182+0000","last_peered":"2024-09-21T07:47:10.871182+0000","last_clean":"2024-09-21T07:47:10.871182+0000","last_became_active":"2024-09-21T07:47:09.636440+0000","last_became_peered":"2024-09-21T07:47:09.636440+0000","last_unstale":"2024-09-21T07:47:10.871182+0000","last_undegraded":"2024-09-21T07:47:10.871182+0000","last_fullsized":"2024-09-21T07:47:10.871182+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:07.036416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":15,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871331+0000","last_change":"2024-09-21T07:47:08.623433+0000","last_active":"2024-09-21T07:47:10.871331+0000","last_peered":"2024-09-21T07:47:10.871331+0000","last_clean":"2024-09-21T07:47:10.871331+0000","last_became_active":"2024-09-21T07:47:08.622500+0000","last_became_peered":"2024-09-21T07:47:08.622500+0000","last_unstale":"2024-09-21T07:47:10.871331+0000","last_undegraded":"2024-09-21T07:47:10.871331+0000","last_fullsized":"2024-09-21T07:47:10.871331+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:18:34.856576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":262,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871303+0000","last_change":"2024-09-21T07:35:27.873645+0000","last_active":"2024-09-21T07:47:10.871303+0000","last_peered":"2024-09-21T07:47:10.871303+0000","last_clean":"2024-09-21T07:47:10.871303+0000","last_became_active":"2024-09-21T07:35:27.873245+0000","last_became_peered":"2024-09-21T07:35:27.873245+0000","last_unstale":"2024-09-21T07:47:10.871303+0000","last_undegraded":"2024-09-21T07:47:10.871303+0000","last_fullsized":"2024-09-21T07:47:10.871303+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:55.090953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":269,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.872919+0000","last_change":"2024-09-21T07:35:28.861744+0000","last_active":"2024-09-21T07:47:10.872919+0000","last_peered":"2024-09-21T07:47:10.872919+0000","last_clean":"2024-09-21T07:47:10.872919+0000","last_became_active":"2024-09-21T07:35:26.876462+0000","last_became_peered":"2024-09-21T07:35:26.876462+0000","last_unstale":"2024-09-21T07:47:10.872919+0000","last_undegraded":"2024-09-21T07:47:10.872919+0000","last_fullsized":"2024-09-21T07:47:10.872919+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:48.632568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012020920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":318,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869381+0000","last_change":"2024-09-21T07:34:25.085454+0000","last_active":"2024-09-21T07:47:10.869381+0000","last_peered":"2024-09-21T07:47:10.869381+0000","last_clean":"2024-09-21T07:47:10.869381+0000","last_became_active":"2024-09-21T07:34:23.279205+0000","last_became_peered":"2024-09-21T07:34:23.279205+0000","last_unstale":"2024-09-21T07:47:10.869381+0000","last_undegraded":"2024-09-21T07:47:10.869381+0000","last_fullsized":"2024-09-21T07:47:10.869381+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:54.231666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095320599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.873469+0000","last_change":"2024-09-21T07:47:09.641653+0000","last_active":"2024-09-21T07:47:10.873469+0000","last_peered":"2024-09-21T07:47:10.873469+0000","last_clean":"2024-09-21T07:47:10.873469+0000","last_became_active":"2024-09-21T07:47:09.641352+0000","last_became_peered":"2024-09-21T07:47:09.641352+0000","last_unstale":"2024-09-21T07:47:10.873469+0000","last_undegraded":"2024-09-21T07:47:10.873469+0000","last_fullsized":"2024-09-21T07:47:10.873469+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:17:19.290168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"193'8","reported_seq":23,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871191+0000","last_change":"2024-09-21T07:47:08.624573+0000","last_active":"2024-09-21T07:47:10.871191+0000","last_peered":"2024-09-21T07:47:10.871191+0000","last_clean":"2024-09-21T07:47:10.871191+0000","last_became_active":"2024-09-21T07:47:08.624259+0000","last_became_peered":"2024-09-21T07:47:08.624259+0000","last_unstale":"2024-09-21T07:47:10.871191+0000","last_undegraded":"2024-09-21T07:47:10.871191+0000","last_fullsized":"2024-09-21T07:47:10.871191+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:15:23.264287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":263,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638182+0000","last_change":"2024-09-21T07:35:27.861373+0000","last_active":"2024-09-21T07:47:10.638182+0000","last_peered":"2024-09-21T07:47:10.638182+0000","last_clean":"2024-09-21T07:47:10.638182+0000","last_became_active":"2024-09-21T07:35:27.861016+0000","last_became_peered":"2024-09-21T07:35:27.861016+0000","last_unstale":"2024-09-21T07:47:10.638182+0000","last_undegraded":"2024-09-21T07:47:10.638182+0000","last_fullsized":"2024-09-21T07:47:10.638182+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:56.043763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":269,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.872726+0000","last_change":"2024-09-21T07:35:29.053677+0000","last_active":"2024-09-21T07:47:10.872726+0000","last_peered":"2024-09-21T07:47:10.872726+0000","last_clean":"2024-09-21T07:47:10.872726+0000","last_became_active":"2024-09-21T07:35:26.875496+0000","last_became_peered":"2024-09-21T07:35:26.875496+0000","last_unstale":"2024-09-21T07:47:10.872726+0000","last_undegraded":"2024-09-21T07:47:10.872726+0000","last_fullsized":"2024-09-21T07:47:10.872726+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:04.850784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065859000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":321,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.872395+0000","last_change":"2024-09-21T07:34:25.028521+0000","last_active":"2024-09-21T07:47:10.872395+0000","last_peered":"2024-09-21T07:47:10.872395+0000","last_clean":"2024-09-21T07:47:10.872395+0000","last_became_active":"2024-09-21T07:34:23.635467+0000","last_became_peered":"2024-09-21T07:34:23.635467+0000","last_unstale":"2024-09-21T07:47:10.872395+0000","last_undegraded":"2024-09-21T07:47:10.872395+0000","last_fullsized":"2024-09-21T07:47:10.872395+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:33.729123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000660558,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.637984+0000","last_change":"2024-09-21T07:47:09.660051+0000","last_active":"2024-09-21T07:47:10.637984+0000","last_peered":"2024-09-21T07:47:10.637984+0000","last_clean":"2024-09-21T07:47:10.637984+0000","last_became_active":"2024-09-21T07:47:09.659683+0000","last_became_peered":"2024-09-21T07:47:09.659683+0000","last_unstale":"2024-09-21T07:47:10.637984+0000","last_undegraded":"2024-09-21T07:47:10.637984+0000","last_fullsized":"2024-09-21T07:47:10.637984+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:38:28.343809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":15,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638605+0000","last_change":"2024-09-21T07:47:08.648146+0000","last_active":"2024-09-21T07:47:10.638605+0000","last_peered":"2024-09-21T07:47:10.638605+0000","last_clean":"2024-09-21T07:47:10.638605+0000","last_became_active":"2024-09-21T07:47:08.647806+0000","last_became_peered":"2024-09-21T07:47:08.647806+0000","last_unstale":"2024-09-21T07:47:10.638605+0000","last_undegraded":"2024-09-21T07:47:10.638605+0000","last_fullsized":"2024-09-21T07:47:10.638605+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:00.150181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":260,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870147+0000","last_change":"2024-09-21T07:35:27.871201+0000","last_active":"2024-09-21T07:47:10.870147+0000","last_peered":"2024-09-21T07:47:10.870147+0000","last_clean":"2024-09-21T07:47:10.870147+0000","last_became_active":"2024-09-21T07:35:27.870816+0000","last_became_peered":"2024-09-21T07:35:27.870816+0000","last_unstale":"2024-09-21T07:47:10.870147+0000","last_undegraded":"2024-09-21T07:47:10.870147+0000","last_fullsized":"2024-09-21T07:47:10.870147+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:04.787450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":270,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638312+0000","last_change":"2024-09-21T07:35:29.491874+0000","last_active":"2024-09-21T07:47:10.638312+0000","last_peered":"2024-09-21T07:47:10.638312+0000","last_clean":"2024-09-21T07:47:10.638312+0000","last_became_active":"2024-09-21T07:35:26.866381+0000","last_became_peered":"2024-09-21T07:35:26.866381+0000","last_unstale":"2024-09-21T07:47:10.638312+0000","last_undegraded":"2024-09-21T07:47:10.638312+0000","last_fullsized":"2024-09-21T07:47:10.638312+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:36.404647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073235399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":324,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638735+0000","last_change":"2024-09-21T07:34:24.260046+0000","last_active":"2024-09-21T07:47:10.638735+0000","last_peered":"2024-09-21T07:47:10.638735+0000","last_clean":"2024-09-21T07:47:10.638735+0000","last_became_active":"2024-09-21T07:34:23.273431+0000","last_became_peered":"2024-09-21T07:34:23.273431+0000","last_unstale":"2024-09-21T07:47:10.638735+0000","last_undegraded":"2024-09-21T07:47:10.638735+0000","last_fullsized":"2024-09-21T07:47:10.638735+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:18.876057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019803310000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.637588+0000","last_change":"2024-09-21T07:47:09.654242+0000","last_active":"2024-09-21T07:47:10.637588+0000","last_peered":"2024-09-21T07:47:10.637588+0000","last_clean":"2024-09-21T07:47:10.637588+0000","last_became_active":"2024-09-21T07:47:09.653937+0000","last_became_peered":"2024-09-21T07:47:09.653937+0000","last_unstale":"2024-09-21T07:47:10.637588+0000","last_undegraded":"2024-09-21T07:47:10.637588+0000","last_fullsized":"2024-09-21T07:47:10.637588+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:33.116186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"193'3","reported_seq":18,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638849+0000","last_change":"2024-09-21T07:47:08.630406+0000","last_active":"2024-09-21T07:47:10.638849+0000","last_peered":"2024-09-21T07:47:10.638849+0000","last_clean":"2024-09-21T07:47:10.638849+0000","last_became_active":"2024-09-21T07:47:08.629896+0000","last_became_peered":"2024-09-21T07:47:08.629896+0000","last_unstale":"2024-09-21T07:47:10.638849+0000","last_undegraded":"2024-09-21T07:47:10.638849+0000","last_fullsized":"2024-09-21T07:47:10.638849+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:26:34.127156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":260,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.872652+0000","last_change":"2024-09-21T07:35:27.871736+0000","last_active":"2024-09-21T07:47:10.872652+0000","last_peered":"2024-09-21T07:47:10.872652+0000","last_clean":"2024-09-21T07:47:10.872652+0000","last_became_active":"2024-09-21T07:35:27.871232+0000","last_became_peered":"2024-09-21T07:35:27.871232+0000","last_unstale":"2024-09-21T07:47:10.872652+0000","last_undegraded":"2024-09-21T07:47:10.872652+0000","last_fullsized":"2024-09-21T07:47:10.872652+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:04.419301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":270,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.637602+0000","last_change":"2024-09-21T07:35:29.491246+0000","last_active":"2024-09-21T07:47:10.637602+0000","last_peered":"2024-09-21T07:47:10.637602+0000","last_clean":"2024-09-21T07:47:10.637602+0000","last_became_active":"2024-09-21T07:35:26.866783+0000","last_became_peered":"2024-09-21T07:35:26.866783+0000","last_unstale":"2024-09-21T07:47:10.637602+0000","last_undegraded":"2024-09-21T07:47:10.637602+0000","last_fullsized":"2024-09-21T07:47:10.637602+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:28.881914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098474099999999992,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":321,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.872774+0000","last_change":"2024-09-21T07:34:25.028640+0000","last_active":"2024-09-21T07:47:10.872774+0000","last_peered":"2024-09-21T07:47:10.872774+0000","last_clean":"2024-09-21T07:47:10.872774+0000","last_became_active":"2024-09-21T07:34:23.260950+0000","last_became_peered":"2024-09-21T07:34:23.260950+0000","last_unstale":"2024-09-21T07:47:10.872774+0000","last_undegraded":"2024-09-21T07:47:10.872774+0000","last_fullsized":"2024-09-21T07:47:10.872774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:23:42.121398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055135099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870971+0000","last_change":"2024-09-21T07:47:09.642435+0000","last_active":"2024-09-21T07:47:10.870971+0000","last_peered":"2024-09-21T07:47:10.870971+0000","last_clean":"2024-09-21T07:47:10.870971+0000","last_became_active":"2024-09-21T07:47:09.642088+0000","last_became_peered":"2024-09-21T07:47:09.642088+0000","last_unstale":"2024-09-21T07:47:10.870971+0000","last_undegraded":"2024-09-21T07:47:10.870971+0000","last_fullsized":"2024-09-21T07:47:10.870971+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:23.496379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"193'6","reported_seq":21,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638301+0000","last_change":"2024-09-21T07:47:08.645704+0000","last_active":"2024-09-21T07:47:10.638301+0000","last_peered":"2024-09-21T07:47:10.638301+0000","last_clean":"2024-09-21T07:47:10.638301+0000","last_became_active":"2024-09-21T07:47:08.645094+0000","last_became_peered":"2024-09-21T07:47:08.645094+0000","last_unstale":"2024-09-21T07:47:10.638301+0000","last_undegraded":"2024-09-21T07:47:10.638301+0000","last_fullsized":"2024-09-21T07:47:10.638301+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:29:23.257974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":262,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871517+0000","last_change":"2024-09-21T07:35:27.872027+0000","last_active":"2024-09-21T07:47:10.871517+0000","last_peered":"2024-09-21T07:47:10.871517+0000","last_clean":"2024-09-21T07:47:10.871517+0000","last_became_active":"2024-09-21T07:35:27.870426+0000","last_became_peered":"2024-09-21T07:35:27.870426+0000","last_unstale":"2024-09-21T07:47:10.871517+0000","last_undegraded":"2024-09-21T07:47:10.871517+0000","last_fullsized":"2024-09-21T07:47:10.871517+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":270,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.637481+0000","last_change":"2024-09-21T07:35:29.491008+0000","last_active":"2024-09-21T07:47:10.637481+0000","last_peered":"2024-09-21T07:47:10.637481+0000","last_clean":"2024-09-21T07:47:10.637481+0000","last_became_active":"2024-09-21T07:35:26.866750+0000","last_became_peered":"2024-09-21T07:35:26.866750+0000","last_unstale":"2024-09-21T07:47:10.637481+0000","last_undegraded":"2024-09-21T07:47:10.637481+0000","last_fullsized":"2024-09-21T07:47:10.637481+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:25.490406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083529700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":323,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.640741+0000","last_change":"2024-09-21T07:34:24.260665+0000","last_active":"2024-09-21T07:47:10.640741+0000","last_peered":"2024-09-21T07:47:10.640741+0000","last_clean":"2024-09-21T07:47:10.640741+0000","last_became_active":"2024-09-21T07:34:23.635909+0000","last_became_peered":"2024-09-21T07:34:23.635909+0000","last_unstale":"2024-09-21T07:47:10.640741+0000","last_undegraded":"2024-09-21T07:47:10.640741+0000","last_fullsized":"2024-09-21T07:47:10.640741+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:44.536885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.872447+0000","last_change":"2024-09-21T07:47:09.648922+0000","last_active":"2024-09-21T07:47:10.872447+0000","last_peered":"2024-09-21T07:47:10.872447+0000","last_clean":"2024-09-21T07:47:10.872447+0000","last_became_active":"2024-09-21T07:47:09.648605+0000","last_became_peered":"2024-09-21T07:47:09.648605+0000","last_unstale":"2024-09-21T07:47:10.872447+0000","last_undegraded":"2024-09-21T07:47:10.872447+0000","last_fullsized":"2024-09-21T07:47:10.872447+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:02:32.488578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"193'1","reported_seq":16,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.873609+0000","last_change":"2024-09-21T07:47:08.642374+0000","last_active":"2024-09-21T07:47:10.873609+0000","last_peered":"2024-09-21T07:47:10.873609+0000","last_clean":"2024-09-21T07:47:10.873609+0000","last_became_active":"2024-09-21T07:47:08.642036+0000","last_became_peered":"2024-09-21T07:47:08.642036+0000","last_unstale":"2024-09-21T07:47:10.873609+0000","last_undegraded":"2024-09-21T07:47:10.873609+0000","last_fullsized":"2024-09-21T07:47:10.873609+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:43.823469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":258,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.637985+0000","last_change":"2024-09-21T07:35:27.874772+0000","last_active":"2024-09-21T07:47:10.637985+0000","last_peered":"2024-09-21T07:47:10.637985+0000","last_clean":"2024-09-21T07:47:10.637985+0000","last_became_active":"2024-09-21T07:35:27.874434+0000","last_became_peered":"2024-09-21T07:35:27.874434+0000","last_unstale":"2024-09-21T07:47:10.637985+0000","last_undegraded":"2024-09-21T07:47:10.637985+0000","last_fullsized":"2024-09-21T07:47:10.637985+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:31.998755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":276,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871147+0000","last_change":"2024-09-21T07:35:28.859153+0000","last_active":"2024-09-21T07:47:10.871147+0000","last_peered":"2024-09-21T07:47:10.871147+0000","last_clean":"2024-09-21T07:47:10.871147+0000","last_became_active":"2024-09-21T07:35:26.874428+0000","last_became_peered":"2024-09-21T07:35:26.874428+0000","last_unstale":"2024-09-21T07:47:10.871147+0000","last_undegraded":"2024-09-21T07:47:10.871147+0000","last_fullsized":"2024-09-21T07:47:10.871147+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:04.487621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048557299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":328,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870726+0000","last_change":"2024-09-21T07:34:24.538845+0000","last_active":"2024-09-21T07:47:10.870726+0000","last_peered":"2024-09-21T07:47:10.870726+0000","last_clean":"2024-09-21T07:47:10.870726+0000","last_became_active":"2024-09-21T07:34:23.636372+0000","last_became_peered":"2024-09-21T07:34:23.636372+0000","last_unstale":"2024-09-21T07:47:10.870726+0000","last_undegraded":"2024-09-21T07:47:10.870726+0000","last_fullsized":"2024-09-21T07:47:10.870726+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:29.393813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841451,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.872564+0000","last_change":"2024-09-21T07:47:09.655269+0000","last_active":"2024-09-21T07:47:10.872564+0000","last_peered":"2024-09-21T07:47:10.872564+0000","last_clean":"2024-09-21T07:47:10.872564+0000","last_became_active":"2024-09-21T07:47:09.654964+0000","last_became_peered":"2024-09-21T07:47:09.654964+0000","last_unstale":"2024-09-21T07:47:10.872564+0000","last_undegraded":"2024-09-21T07:47:10.872564+0000","last_fullsized":"2024-09-21T07:47:10.872564+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:12:05.101124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"193'1","reported_seq":16,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870249+0000","last_change":"2024-09-21T07:47:08.645839+0000","last_active":"2024-09-21T07:47:10.870249+0000","last_peered":"2024-09-21T07:47:10.870249+0000","last_clean":"2024-09-21T07:47:10.870249+0000","last_became_active":"2024-09-21T07:47:08.645549+0000","last_became_peered":"2024-09-21T07:47:08.645549+0000","last_unstale":"2024-09-21T07:47:10.870249+0000","last_undegraded":"2024-09-21T07:47:10.870249+0000","last_fullsized":"2024-09-21T07:47:10.870249+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:01:03.949942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":262,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869781+0000","last_change":"2024-09-21T07:35:27.877801+0000","last_active":"2024-09-21T07:47:10.869781+0000","last_peered":"2024-09-21T07:47:10.869781+0000","last_clean":"2024-09-21T07:47:10.869781+0000","last_became_active":"2024-09-21T07:35:27.877464+0000","last_became_peered":"2024-09-21T07:35:27.877464+0000","last_unstale":"2024-09-21T07:47:10.869781+0000","last_undegraded":"2024-09-21T07:47:10.869781+0000","last_fullsized":"2024-09-21T07:47:10.869781+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:08.814929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":267,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.872393+0000","last_change":"2024-09-21T07:35:28.863732+0000","last_active":"2024-09-21T07:47:10.872393+0000","last_peered":"2024-09-21T07:47:10.872393+0000","last_clean":"2024-09-21T07:47:10.872393+0000","last_became_active":"2024-09-21T07:35:26.877392+0000","last_became_peered":"2024-09-21T07:35:26.877392+0000","last_unstale":"2024-09-21T07:47:10.872393+0000","last_undegraded":"2024-09-21T07:47:10.872393+0000","last_fullsized":"2024-09-21T07:47:10.872393+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071425700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":324,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638424+0000","last_change":"2024-09-21T07:34:24.259686+0000","last_active":"2024-09-21T07:47:10.638424+0000","last_peered":"2024-09-21T07:47:10.638424+0000","last_clean":"2024-09-21T07:47:10.638424+0000","last_became_active":"2024-09-21T07:34:23.274405+0000","last_became_peered":"2024-09-21T07:34:23.274405+0000","last_unstale":"2024-09-21T07:47:10.638424+0000","last_undegraded":"2024-09-21T07:47:10.638424+0000","last_fullsized":"2024-09-21T07:47:10.638424+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:26.304693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022009920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.873340+0000","last_change":"2024-09-21T07:47:09.631732+0000","last_active":"2024-09-21T07:47:10.873340+0000","last_peered":"2024-09-21T07:47:10.873340+0000","last_clean":"2024-09-21T07:47:10.873340+0000","last_became_active":"2024-09-21T07:47:09.631428+0000","last_became_peered":"2024-09-21T07:47:09.631428+0000","last_unstale":"2024-09-21T07:47:10.873340+0000","last_undegraded":"2024-09-21T07:47:10.873340+0000","last_fullsized":"2024-09-21T07:47:10.873340+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:46.092559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":15,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870495+0000","last_change":"2024-09-21T07:47:08.630888+0000","last_active":"2024-09-21T07:47:10.870495+0000","last_peered":"2024-09-21T07:47:10.870495+0000","last_clean":"2024-09-21T07:47:10.870495+0000","last_became_active":"2024-09-21T07:47:08.630558+0000","last_became_peered":"2024-09-21T07:47:08.630558+0000","last_unstale":"2024-09-21T07:47:10.870495+0000","last_undegraded":"2024-09-21T07:47:10.870495+0000","last_fullsized":"2024-09-21T07:47:10.870495+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:24.480307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":258,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638322+0000","last_change":"2024-09-21T07:35:27.867336+0000","last_active":"2024-09-21T07:47:10.638322+0000","last_peered":"2024-09-21T07:47:10.638322+0000","last_clean":"2024-09-21T07:47:10.638322+0000","last_became_active":"2024-09-21T07:35:27.867015+0000","last_became_peered":"2024-09-21T07:35:27.867015+0000","last_unstale":"2024-09-21T07:47:10.638322+0000","last_undegraded":"2024-09-21T07:47:10.638322+0000","last_fullsized":"2024-09-21T07:47:10.638322+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:59.214139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":269,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.640634+0000","last_change":"2024-09-21T07:35:29.493137+0000","last_active":"2024-09-21T07:47:10.640634+0000","last_peered":"2024-09-21T07:47:10.640634+0000","last_clean":"2024-09-21T07:47:10.640634+0000","last_became_active":"2024-09-21T07:35:26.872230+0000","last_became_peered":"2024-09-21T07:35:26.872230+0000","last_unstale":"2024-09-21T07:47:10.640634+0000","last_undegraded":"2024-09-21T07:47:10.640634+0000","last_fullsized":"2024-09-21T07:47:10.640634+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:24.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001197449,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":322,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638215+0000","last_change":"2024-09-21T07:34:24.258754+0000","last_active":"2024-09-21T07:47:10.638215+0000","last_peered":"2024-09-21T07:47:10.638215+0000","last_clean":"2024-09-21T07:47:10.638215+0000","last_became_active":"2024-09-21T07:34:23.272657+0000","last_became_peered":"2024-09-21T07:34:23.272657+0000","last_unstale":"2024-09-21T07:47:10.638215+0000","last_undegraded":"2024-09-21T07:47:10.638215+0000","last_fullsized":"2024-09-21T07:47:10.638215+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:05.477099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067143200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.640029+0000","last_change":"2024-09-21T07:47:09.637896+0000","last_active":"2024-09-21T07:47:10.640029+0000","last_peered":"2024-09-21T07:47:10.640029+0000","last_clean":"2024-09-21T07:47:10.640029+0000","last_became_active":"2024-09-21T07:47:09.637523+0000","last_became_peered":"2024-09-21T07:47:09.637523+0000","last_unstale":"2024-09-21T07:47:10.640029+0000","last_undegraded":"2024-09-21T07:47:10.640029+0000","last_fullsized":"2024-09-21T07:47:10.640029+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:43.508202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":15,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870737+0000","last_change":"2024-09-21T07:47:08.637339+0000","last_active":"2024-09-21T07:47:10.870737+0000","last_peered":"2024-09-21T07:47:10.870737+0000","last_clean":"2024-09-21T07:47:10.870737+0000","last_became_active":"2024-09-21T07:47:08.634751+0000","last_became_peered":"2024-09-21T07:47:08.634751+0000","last_unstale":"2024-09-21T07:47:10.870737+0000","last_undegraded":"2024-09-21T07:47:10.870737+0000","last_fullsized":"2024-09-21T07:47:10.870737+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:00:08.412196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":260,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870207+0000","last_change":"2024-09-21T07:35:27.876021+0000","last_active":"2024-09-21T07:47:10.870207+0000","last_peered":"2024-09-21T07:47:10.870207+0000","last_clean":"2024-09-21T07:47:10.870207+0000","last_became_active":"2024-09-21T07:35:27.875724+0000","last_became_peered":"2024-09-21T07:35:27.875724+0000","last_unstale":"2024-09-21T07:47:10.870207+0000","last_undegraded":"2024-09-21T07:47:10.870207+0000","last_fullsized":"2024-09-21T07:47:10.870207+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:03.022812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":269,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.640211+0000","last_change":"2024-09-21T07:35:29.492570+0000","last_active":"2024-09-21T07:47:10.640211+0000","last_peered":"2024-09-21T07:47:10.640211+0000","last_clean":"2024-09-21T07:47:10.640211+0000","last_became_active":"2024-09-21T07:35:26.874316+0000","last_became_peered":"2024-09-21T07:35:26.874316+0000","last_unstale":"2024-09-21T07:47:10.640211+0000","last_undegraded":"2024-09-21T07:47:10.640211+0000","last_fullsized":"2024-09-21T07:47:10.640211+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:16.512323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011990760000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":318,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869756+0000","last_change":"2024-09-21T07:34:25.085232+0000","last_active":"2024-09-21T07:47:10.869756+0000","last_peered":"2024-09-21T07:47:10.869756+0000","last_clean":"2024-09-21T07:47:10.869756+0000","last_became_active":"2024-09-21T07:34:23.277807+0000","last_became_peered":"2024-09-21T07:34:23.277807+0000","last_unstale":"2024-09-21T07:47:10.869756+0000","last_undegraded":"2024-09-21T07:47:10.869756+0000","last_fullsized":"2024-09-21T07:47:10.869756+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:36.362919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086619099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638069+0000","last_change":"2024-09-21T07:47:09.659567+0000","last_active":"2024-09-21T07:47:10.638069+0000","last_peered":"2024-09-21T07:47:10.638069+0000","last_clean":"2024-09-21T07:47:10.638069+0000","last_became_active":"2024-09-21T07:47:09.659195+0000","last_became_peered":"2024-09-21T07:47:09.659195+0000","last_unstale":"2024-09-21T07:47:10.638069+0000","last_undegraded":"2024-09-21T07:47:10.638069+0000","last_fullsized":"2024-09-21T07:47:10.638069+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:43:36.808598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"193'4","reported_seq":19,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.639804+0000","last_change":"2024-09-21T07:47:08.641875+0000","last_active":"2024-09-21T07:47:10.639804+0000","last_peered":"2024-09-21T07:47:10.639804+0000","last_clean":"2024-09-21T07:47:10.639804+0000","last_became_active":"2024-09-21T07:47:08.641574+0000","last_became_peered":"2024-09-21T07:47:08.641574+0000","last_unstale":"2024-09-21T07:47:10.639804+0000","last_undegraded":"2024-09-21T07:47:10.639804+0000","last_fullsized":"2024-09-21T07:47:10.639804+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:26:46.004831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":262,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870409+0000","last_change":"2024-09-21T07:35:27.872798+0000","last_active":"2024-09-21T07:47:10.870409+0000","last_peered":"2024-09-21T07:47:10.870409+0000","last_clean":"2024-09-21T07:47:10.870409+0000","last_became_active":"2024-09-21T07:35:27.872483+0000","last_became_peered":"2024-09-21T07:35:27.872483+0000","last_unstale":"2024-09-21T07:47:10.870409+0000","last_undegraded":"2024-09-21T07:47:10.870409+0000","last_fullsized":"2024-09-21T07:47:10.870409+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:11.933469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":269,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869680+0000","last_change":"2024-09-21T07:35:29.206723+0000","last_active":"2024-09-21T07:47:10.869680+0000","last_peered":"2024-09-21T07:47:10.869680+0000","last_clean":"2024-09-21T07:47:10.869680+0000","last_became_active":"2024-09-21T07:35:26.872377+0000","last_became_peered":"2024-09-21T07:35:26.872377+0000","last_unstale":"2024-09-21T07:47:10.869680+0000","last_undegraded":"2024-09-21T07:47:10.869680+0000","last_fullsized":"2024-09-21T07:47:10.869680+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:29.086862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083077200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":314,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.639394+0000","last_change":"2024-09-21T07:34:24.597672+0000","last_active":"2024-09-21T07:47:10.639394+0000","last_peered":"2024-09-21T07:47:10.639394+0000","last_clean":"2024-09-21T07:47:10.639394+0000","last_became_active":"2024-09-21T07:34:23.267770+0000","last_became_peered":"2024-09-21T07:34:23.267770+0000","last_unstale":"2024-09-21T07:47:10.639394+0000","last_undegraded":"2024-09-21T07:47:10.639394+0000","last_fullsized":"2024-09-21T07:47:10.639394+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:51.629476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086378500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.637484+0000","last_change":"2024-09-21T07:47:09.653648+0000","last_active":"2024-09-21T07:47:10.637484+0000","last_peered":"2024-09-21T07:47:10.637484+0000","last_clean":"2024-09-21T07:47:10.637484+0000","last_became_active":"2024-09-21T07:47:09.653308+0000","last_became_peered":"2024-09-21T07:47:09.653308+0000","last_unstale":"2024-09-21T07:47:10.637484+0000","last_undegraded":"2024-09-21T07:47:10.637484+0000","last_fullsized":"2024-09-21T07:47:10.637484+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:54:51.139860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"193'1","reported_seq":16,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870599+0000","last_change":"2024-09-21T07:47:08.649376+0000","last_active":"2024-09-21T07:47:10.870599+0000","last_peered":"2024-09-21T07:47:10.870599+0000","last_clean":"2024-09-21T07:47:10.870599+0000","last_became_active":"2024-09-21T07:47:08.649012+0000","last_became_peered":"2024-09-21T07:47:08.649012+0000","last_unstale":"2024-09-21T07:47:10.870599+0000","last_undegraded":"2024-09-21T07:47:10.870599+0000","last_fullsized":"2024-09-21T07:47:10.870599+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:58:31.201708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":262,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870318+0000","last_change":"2024-09-21T07:35:27.870121+0000","last_active":"2024-09-21T07:47:10.870318+0000","last_peered":"2024-09-21T07:47:10.870318+0000","last_clean":"2024-09-21T07:47:10.870318+0000","last_became_active":"2024-09-21T07:35:27.869811+0000","last_became_peered":"2024-09-21T07:35:27.869811+0000","last_unstale":"2024-09-21T07:47:10.870318+0000","last_undegraded":"2024-09-21T07:47:10.870318+0000","last_fullsized":"2024-09-21T07:47:10.870318+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:31.778461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":269,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871134+0000","last_change":"2024-09-21T07:35:28.860699+0000","last_active":"2024-09-21T07:47:10.871134+0000","last_peered":"2024-09-21T07:47:10.871134+0000","last_clean":"2024-09-21T07:47:10.871134+0000","last_became_active":"2024-09-21T07:35:26.876779+0000","last_became_peered":"2024-09-21T07:35:26.876779+0000","last_unstale":"2024-09-21T07:47:10.871134+0000","last_undegraded":"2024-09-21T07:47:10.871134+0000","last_fullsized":"2024-09-21T07:47:10.871134+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:28.286403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000913188,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":321,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.873714+0000","last_change":"2024-09-21T07:34:24.260181+0000","last_active":"2024-09-21T07:47:10.873714+0000","last_peered":"2024-09-21T07:47:10.873714+0000","last_clean":"2024-09-21T07:47:10.873714+0000","last_became_active":"2024-09-21T07:34:23.275580+0000","last_became_peered":"2024-09-21T07:34:23.275580+0000","last_unstale":"2024-09-21T07:47:10.873714+0000","last_undegraded":"2024-09-21T07:47:10.873714+0000","last_fullsized":"2024-09-21T07:47:10.873714+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:49.857622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014706980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870577+0000","last_change":"2024-09-21T07:47:09.645794+0000","last_active":"2024-09-21T07:47:10.870577+0000","last_peered":"2024-09-21T07:47:10.870577+0000","last_clean":"2024-09-21T07:47:10.870577+0000","last_became_active":"2024-09-21T07:47:09.644072+0000","last_became_peered":"2024-09-21T07:47:09.644072+0000","last_unstale":"2024-09-21T07:47:10.870577+0000","last_undegraded":"2024-09-21T07:47:10.870577+0000","last_fullsized":"2024-09-21T07:47:10.870577+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:35:07.815426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"193'2","reported_seq":17,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870781+0000","last_change":"2024-09-21T07:47:08.629617+0000","last_active":"2024-09-21T07:47:10.870781+0000","last_peered":"2024-09-21T07:47:10.870781+0000","last_clean":"2024-09-21T07:47:10.870781+0000","last_became_active":"2024-09-21T07:47:08.629299+0000","last_became_peered":"2024-09-21T07:47:08.629299+0000","last_unstale":"2024-09-21T07:47:10.870781+0000","last_undegraded":"2024-09-21T07:47:10.870781+0000","last_fullsized":"2024-09-21T07:47:10.870781+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:51:56.256805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":262,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.874413+0000","last_change":"2024-09-21T07:35:27.875393+0000","last_active":"2024-09-21T07:47:10.874413+0000","last_peered":"2024-09-21T07:47:10.874413+0000","last_clean":"2024-09-21T07:47:10.874413+0000","last_became_active":"2024-09-21T07:35:27.874671+0000","last_became_peered":"2024-09-21T07:35:27.874671+0000","last_unstale":"2024-09-21T07:47:10.874413+0000","last_undegraded":"2024-09-21T07:47:10.874413+0000","last_fullsized":"2024-09-21T07:47:10.874413+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":269,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870060+0000","last_change":"2024-09-21T07:35:29.206296+0000","last_active":"2024-09-21T07:47:10.870060+0000","last_peered":"2024-09-21T07:47:10.870060+0000","last_clean":"2024-09-21T07:47:10.870060+0000","last_became_active":"2024-09-21T07:35:26.874575+0000","last_became_peered":"2024-09-21T07:35:26.874575+0000","last_unstale":"2024-09-21T07:47:10.870060+0000","last_undegraded":"2024-09-21T07:47:10.870060+0000","last_fullsized":"2024-09-21T07:47:10.870060+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:22.771154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061011599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":324,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638788+0000","last_change":"2024-09-21T07:34:24.259112+0000","last_active":"2024-09-21T07:47:10.638788+0000","last_peered":"2024-09-21T07:47:10.638788+0000","last_clean":"2024-09-21T07:47:10.638788+0000","last_became_active":"2024-09-21T07:34:23.635444+0000","last_became_peered":"2024-09-21T07:34:23.635444+0000","last_unstale":"2024-09-21T07:47:10.638788+0000","last_undegraded":"2024-09-21T07:47:10.638788+0000","last_fullsized":"2024-09-21T07:47:10.638788+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:46.255385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001952636,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.640174+0000","last_change":"2024-09-21T07:47:09.661164+0000","last_active":"2024-09-21T07:47:10.640174+0000","last_peered":"2024-09-21T07:47:10.640174+0000","last_clean":"2024-09-21T07:47:10.640174+0000","last_became_active":"2024-09-21T07:47:09.660874+0000","last_became_peered":"2024-09-21T07:47:09.660874+0000","last_unstale":"2024-09-21T07:47:10.640174+0000","last_undegraded":"2024-09-21T07:47:10.640174+0000","last_fullsized":"2024-09-21T07:47:10.640174+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:58:32.208386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"193'2","reported_seq":17,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871829+0000","last_change":"2024-09-21T07:47:08.631059+0000","last_active":"2024-09-21T07:47:10.871829+0000","last_peered":"2024-09-21T07:47:10.871829+0000","last_clean":"2024-09-21T07:47:10.871829+0000","last_became_active":"2024-09-21T07:47:08.630725+0000","last_became_peered":"2024-09-21T07:47:08.630725+0000","last_unstale":"2024-09-21T07:47:10.871829+0000","last_undegraded":"2024-09-21T07:47:10.871829+0000","last_fullsized":"2024-09-21T07:47:10.871829+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:37.178741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":262,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.874498+0000","last_change":"2024-09-21T07:35:27.878356+0000","last_active":"2024-09-21T07:47:10.874498+0000","last_peered":"2024-09-21T07:47:10.874498+0000","last_clean":"2024-09-21T07:47:10.874498+0000","last_became_active":"2024-09-21T07:35:27.877982+0000","last_became_peered":"2024-09-21T07:35:27.877982+0000","last_unstale":"2024-09-21T07:47:10.874498+0000","last_undegraded":"2024-09-21T07:47:10.874498+0000","last_fullsized":"2024-09-21T07:47:10.874498+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":269,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871675+0000","last_change":"2024-09-21T07:35:29.053131+0000","last_active":"2024-09-21T07:47:10.871675+0000","last_peered":"2024-09-21T07:47:10.871675+0000","last_clean":"2024-09-21T07:47:10.871675+0000","last_became_active":"2024-09-21T07:35:26.870516+0000","last_became_peered":"2024-09-21T07:35:26.870516+0000","last_unstale":"2024-09-21T07:47:10.871675+0000","last_undegraded":"2024-09-21T07:47:10.871675+0000","last_fullsized":"2024-09-21T07:47:10.871675+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:10.924491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070705699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":328,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870942+0000","last_change":"2024-09-21T07:34:24.539152+0000","last_active":"2024-09-21T07:47:10.870942+0000","last_peered":"2024-09-21T07:47:10.870942+0000","last_clean":"2024-09-21T07:47:10.870942+0000","last_became_active":"2024-09-21T07:34:23.259573+0000","last_became_peered":"2024-09-21T07:34:23.259573+0000","last_unstale":"2024-09-21T07:47:10.870942+0000","last_undegraded":"2024-09-21T07:47:10.870942+0000","last_fullsized":"2024-09-21T07:47:10.870942+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:26.986900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058108400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.639087+0000","last_change":"2024-09-21T07:47:09.644863+0000","last_active":"2024-09-21T07:47:10.639087+0000","last_peered":"2024-09-21T07:47:10.639087+0000","last_clean":"2024-09-21T07:47:10.639087+0000","last_became_active":"2024-09-21T07:47:09.644435+0000","last_became_peered":"2024-09-21T07:47:09.644435+0000","last_unstale":"2024-09-21T07:47:10.639087+0000","last_undegraded":"2024-09-21T07:47:10.639087+0000","last_fullsized":"2024-09-21T07:47:10.639087+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:02.301757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":15,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638629+0000","last_change":"2024-09-21T07:47:08.644109+0000","last_active":"2024-09-21T07:47:10.638629+0000","last_peered":"2024-09-21T07:47:10.638629+0000","last_clean":"2024-09-21T07:47:10.638629+0000","last_became_active":"2024-09-21T07:47:08.643774+0000","last_became_peered":"2024-09-21T07:47:08.643774+0000","last_unstale":"2024-09-21T07:47:10.638629+0000","last_undegraded":"2024-09-21T07:47:10.638629+0000","last_fullsized":"2024-09-21T07:47:10.638629+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:15.917170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":261,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871244+0000","last_change":"2024-09-21T07:35:27.873726+0000","last_active":"2024-09-21T07:47:10.871244+0000","last_peered":"2024-09-21T07:47:10.871244+0000","last_clean":"2024-09-21T07:47:10.871244+0000","last_became_active":"2024-09-21T07:35:27.871421+0000","last_became_peered":"2024-09-21T07:35:27.871421+0000","last_unstale":"2024-09-21T07:47:10.871244+0000","last_undegraded":"2024-09-21T07:47:10.871244+0000","last_fullsized":"2024-09-21T07:47:10.871244+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:00.168915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":269,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.873564+0000","last_change":"2024-09-21T07:35:28.861032+0000","last_active":"2024-09-21T07:47:10.873564+0000","last_peered":"2024-09-21T07:47:10.873564+0000","last_clean":"2024-09-21T07:47:10.873564+0000","last_became_active":"2024-09-21T07:35:26.873149+0000","last_became_peered":"2024-09-21T07:35:26.873149+0000","last_unstale":"2024-09-21T07:47:10.873564+0000","last_undegraded":"2024-09-21T07:47:10.873564+0000","last_fullsized":"2024-09-21T07:47:10.873564+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:59.435981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011988369999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":322,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.639760+0000","last_change":"2024-09-21T07:34:24.259902+0000","last_active":"2024-09-21T07:47:10.639760+0000","last_peered":"2024-09-21T07:47:10.639760+0000","last_clean":"2024-09-21T07:47:10.639760+0000","last_became_active":"2024-09-21T07:34:23.271902+0000","last_became_peered":"2024-09-21T07:34:23.271902+0000","last_unstale":"2024-09-21T07:47:10.639760+0000","last_undegraded":"2024-09-21T07:47:10.639760+0000","last_fullsized":"2024-09-21T07:47:10.639760+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:59.439525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001686358,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.640274+0000","last_change":"2024-09-21T07:47:09.660252+0000","last_active":"2024-09-21T07:47:10.640274+0000","last_peered":"2024-09-21T07:47:10.640274+0000","last_clean":"2024-09-21T07:47:10.640274+0000","last_became_active":"2024-09-21T07:47:09.659225+0000","last_became_peered":"2024-09-21T07:47:09.659225+0000","last_unstale":"2024-09-21T07:47:10.640274+0000","last_undegraded":"2024-09-21T07:47:10.640274+0000","last_fullsized":"2024-09-21T07:47:10.640274+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:37.104542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":15,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.637768+0000","last_change":"2024-09-21T07:47:08.643231+0000","last_active":"2024-09-21T07:47:10.637768+0000","last_peered":"2024-09-21T07:47:10.637768+0000","last_clean":"2024-09-21T07:47:10.637768+0000","last_became_active":"2024-09-21T07:47:08.642744+0000","last_became_peered":"2024-09-21T07:47:08.642744+0000","last_unstale":"2024-09-21T07:47:10.637768+0000","last_undegraded":"2024-09-21T07:47:10.637768+0000","last_fullsized":"2024-09-21T07:47:10.637768+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:12:37.799404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":259,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869429+0000","last_change":"2024-09-21T07:35:27.872637+0000","last_active":"2024-09-21T07:47:10.869429+0000","last_peered":"2024-09-21T07:47:10.869429+0000","last_clean":"2024-09-21T07:47:10.869429+0000","last_became_active":"2024-09-21T07:35:27.872149+0000","last_became_peered":"2024-09-21T07:35:27.872149+0000","last_unstale":"2024-09-21T07:47:10.869429+0000","last_undegraded":"2024-09-21T07:47:10.869429+0000","last_fullsized":"2024-09-21T07:47:10.869429+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:27.697872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":269,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.640559+0000","last_change":"2024-09-21T07:35:29.493362+0000","last_active":"2024-09-21T07:47:10.640559+0000","last_peered":"2024-09-21T07:47:10.640559+0000","last_clean":"2024-09-21T07:47:10.640559+0000","last_became_active":"2024-09-21T07:35:26.873018+0000","last_became_peered":"2024-09-21T07:35:26.873018+0000","last_unstale":"2024-09-21T07:47:10.640559+0000","last_undegraded":"2024-09-21T07:47:10.640559+0000","last_fullsized":"2024-09-21T07:47:10.640559+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:02.460557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012496510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":324,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638309+0000","last_change":"2024-09-21T07:34:24.259662+0000","last_active":"2024-09-21T07:47:10.638309+0000","last_peered":"2024-09-21T07:47:10.638309+0000","last_clean":"2024-09-21T07:47:10.638309+0000","last_became_active":"2024-09-21T07:34:23.278812+0000","last_became_peered":"2024-09-21T07:34:23.278812+0000","last_unstale":"2024-09-21T07:47:10.638309+0000","last_undegraded":"2024-09-21T07:47:10.638309+0000","last_fullsized":"2024-09-21T07:47:10.638309+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:49.829974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356463,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638295+0000","last_change":"2024-09-21T07:47:09.646277+0000","last_active":"2024-09-21T07:47:10.638295+0000","last_peered":"2024-09-21T07:47:10.638295+0000","last_clean":"2024-09-21T07:47:10.638295+0000","last_became_active":"2024-09-21T07:47:09.646018+0000","last_became_peered":"2024-09-21T07:47:09.646018+0000","last_unstale":"2024-09-21T07:47:10.638295+0000","last_undegraded":"2024-09-21T07:47:10.638295+0000","last_fullsized":"2024-09-21T07:47:10.638295+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:09:52.658128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":15,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871446+0000","last_change":"2024-09-21T07:47:08.628798+0000","last_active":"2024-09-21T07:47:10.871446+0000","last_peered":"2024-09-21T07:47:10.871446+0000","last_clean":"2024-09-21T07:47:10.871446+0000","last_became_active":"2024-09-21T07:47:08.626396+0000","last_became_peered":"2024-09-21T07:47:08.626396+0000","last_unstale":"2024-09-21T07:47:10.871446+0000","last_undegraded":"2024-09-21T07:47:10.871446+0000","last_fullsized":"2024-09-21T07:47:10.871446+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:37.215849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":257,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638213+0000","last_change":"2024-09-21T07:35:27.868448+0000","last_active":"2024-09-21T07:47:10.638213+0000","last_peered":"2024-09-21T07:47:10.638213+0000","last_clean":"2024-09-21T07:47:10.638213+0000","last_became_active":"2024-09-21T07:35:27.868105+0000","last_became_peered":"2024-09-21T07:35:27.868105+0000","last_unstale":"2024-09-21T07:47:10.638213+0000","last_undegraded":"2024-09-21T07:47:10.638213+0000","last_fullsized":"2024-09-21T07:47:10.638213+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:41.468702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":268,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.640367+0000","last_change":"2024-09-21T07:35:29.492974+0000","last_active":"2024-09-21T07:47:10.640367+0000","last_peered":"2024-09-21T07:47:10.640367+0000","last_clean":"2024-09-21T07:47:10.640367+0000","last_became_active":"2024-09-21T07:35:26.872165+0000","last_became_peered":"2024-09-21T07:35:26.872165+0000","last_unstale":"2024-09-21T07:47:10.640367+0000","last_undegraded":"2024-09-21T07:47:10.640367+0000","last_fullsized":"2024-09-21T07:47:10.640367+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:59.912520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013795179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":318,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869348+0000","last_change":"2024-09-21T07:34:25.084861+0000","last_active":"2024-09-21T07:47:10.869348+0000","last_peered":"2024-09-21T07:47:10.869348+0000","last_clean":"2024-09-21T07:47:10.869348+0000","last_became_active":"2024-09-21T07:34:23.277708+0000","last_became_peered":"2024-09-21T07:34:23.277708+0000","last_unstale":"2024-09-21T07:47:10.869348+0000","last_undegraded":"2024-09-21T07:47:10.869348+0000","last_fullsized":"2024-09-21T07:47:10.869348+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:40.816488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000597566,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.873480+0000","last_change":"2024-09-21T07:47:09.642462+0000","last_active":"2024-09-21T07:47:10.873480+0000","last_peered":"2024-09-21T07:47:10.873480+0000","last_clean":"2024-09-21T07:47:10.873480+0000","last_became_active":"2024-09-21T07:47:09.642106+0000","last_became_peered":"2024-09-21T07:47:09.642106+0000","last_unstale":"2024-09-21T07:47:10.873480+0000","last_undegraded":"2024-09-21T07:47:10.873480+0000","last_fullsized":"2024-09-21T07:47:10.873480+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:14.002845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":15,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869092+0000","last_change":"2024-09-21T07:47:08.631894+0000","last_active":"2024-09-21T07:47:10.869092+0000","last_peered":"2024-09-21T07:47:10.869092+0000","last_clean":"2024-09-21T07:47:10.869092+0000","last_became_active":"2024-09-21T07:47:08.630134+0000","last_became_peered":"2024-09-21T07:47:08.630134+0000","last_unstale":"2024-09-21T07:47:10.869092+0000","last_undegraded":"2024-09-21T07:47:10.869092+0000","last_fullsized":"2024-09-21T07:47:10.869092+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:17.848186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":257,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638686+0000","last_change":"2024-09-21T07:35:27.874640+0000","last_active":"2024-09-21T07:47:10.638686+0000","last_peered":"2024-09-21T07:47:10.638686+0000","last_clean":"2024-09-21T07:47:10.638686+0000","last_became_active":"2024-09-21T07:35:27.874265+0000","last_became_peered":"2024-09-21T07:35:27.874265+0000","last_unstale":"2024-09-21T07:47:10.638686+0000","last_undegraded":"2024-09-21T07:47:10.638686+0000","last_fullsized":"2024-09-21T07:47:10.638686+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:45.754654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":268,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870515+0000","last_change":"2024-09-21T07:35:29.207392+0000","last_active":"2024-09-21T07:47:10.870515+0000","last_peered":"2024-09-21T07:47:10.870515+0000","last_clean":"2024-09-21T07:47:10.870515+0000","last_became_active":"2024-09-21T07:35:26.870917+0000","last_became_peered":"2024-09-21T07:35:26.870917+0000","last_unstale":"2024-09-21T07:47:10.870515+0000","last_undegraded":"2024-09-21T07:47:10.870515+0000","last_fullsized":"2024-09-21T07:47:10.870515+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:59.159937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059198499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":324,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.637715+0000","last_change":"2024-09-21T07:34:24.258709+0000","last_active":"2024-09-21T07:47:10.637715+0000","last_peered":"2024-09-21T07:47:10.637715+0000","last_clean":"2024-09-21T07:47:10.637715+0000","last_became_active":"2024-09-21T07:34:23.279042+0000","last_became_peered":"2024-09-21T07:34:23.279042+0000","last_unstale":"2024-09-21T07:47:10.637715+0000","last_undegraded":"2024-09-21T07:47:10.637715+0000","last_fullsized":"2024-09-21T07:47:10.637715+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:01.762117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019282100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870020+0000","last_change":"2024-09-21T07:47:09.625256+0000","last_active":"2024-09-21T07:47:10.870020+0000","last_peered":"2024-09-21T07:47:10.870020+0000","last_clean":"2024-09-21T07:47:10.870020+0000","last_became_active":"2024-09-21T07:47:09.624902+0000","last_became_peered":"2024-09-21T07:47:09.624902+0000","last_unstale":"2024-09-21T07:47:10.870020+0000","last_undegraded":"2024-09-21T07:47:10.870020+0000","last_fullsized":"2024-09-21T07:47:10.870020+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:43:19.340627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"193'2","reported_seq":17,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870403+0000","last_change":"2024-09-21T07:47:08.631365+0000","last_active":"2024-09-21T07:47:10.870403+0000","last_peered":"2024-09-21T07:47:10.870403+0000","last_clean":"2024-09-21T07:47:10.870403+0000","last_became_active":"2024-09-21T07:47:08.631036+0000","last_became_peered":"2024-09-21T07:47:08.631036+0000","last_unstale":"2024-09-21T07:47:10.870403+0000","last_undegraded":"2024-09-21T07:47:10.870403+0000","last_fullsized":"2024-09-21T07:47:10.870403+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:36.924632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":259,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869872+0000","last_change":"2024-09-21T07:35:27.870260+0000","last_active":"2024-09-21T07:47:10.869872+0000","last_peered":"2024-09-21T07:47:10.869872+0000","last_clean":"2024-09-21T07:47:10.869872+0000","last_became_active":"2024-09-21T07:35:27.869448+0000","last_became_peered":"2024-09-21T07:35:27.869448+0000","last_unstale":"2024-09-21T07:47:10.869872+0000","last_undegraded":"2024-09-21T07:47:10.869872+0000","last_fullsized":"2024-09-21T07:47:10.869872+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:16.946790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":264,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638648+0000","last_change":"2024-09-21T07:35:29.386367+0000","last_active":"2024-09-21T07:47:10.638648+0000","last_peered":"2024-09-21T07:47:10.638648+0000","last_clean":"2024-09-21T07:47:10.638648+0000","last_became_active":"2024-09-21T07:35:26.879238+0000","last_became_peered":"2024-09-21T07:35:26.879238+0000","last_unstale":"2024-09-21T07:47:10.638648+0000","last_undegraded":"2024-09-21T07:47:10.638648+0000","last_fullsized":"2024-09-21T07:47:10.638648+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:16.528260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092223100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":318,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.872551+0000","last_change":"2024-09-21T07:34:25.085635+0000","last_active":"2024-09-21T07:47:10.872551+0000","last_peered":"2024-09-21T07:47:10.872551+0000","last_clean":"2024-09-21T07:47:10.872551+0000","last_became_active":"2024-09-21T07:34:23.273566+0000","last_became_peered":"2024-09-21T07:34:23.273566+0000","last_unstale":"2024-09-21T07:47:10.872551+0000","last_undegraded":"2024-09-21T07:47:10.872551+0000","last_fullsized":"2024-09-21T07:47:10.872551+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:14.142272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079897300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871363+0000","last_change":"2024-09-21T07:47:09.656317+0000","last_active":"2024-09-21T07:47:10.871363+0000","last_peered":"2024-09-21T07:47:10.871363+0000","last_clean":"2024-09-21T07:47:10.871363+0000","last_became_active":"2024-09-21T07:47:09.655954+0000","last_became_peered":"2024-09-21T07:47:09.655954+0000","last_unstale":"2024-09-21T07:47:10.871363+0000","last_undegraded":"2024-09-21T07:47:10.871363+0000","last_fullsized":"2024-09-21T07:47:10.871363+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:03.400838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"194'1","reported_seq":16,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:11.618663+0000","last_change":"2024-09-21T07:47:08.645742+0000","last_active":"2024-09-21T07:47:11.618663+0000","last_peered":"2024-09-21T07:47:11.618663+0000","last_clean":"2024-09-21T07:47:11.618663+0000","last_became_active":"2024-09-21T07:47:08.645220+0000","last_became_peered":"2024-09-21T07:47:08.645220+0000","last_unstale":"2024-09-21T07:47:11.618663+0000","last_undegraded":"2024-09-21T07:47:11.618663+0000","last_fullsized":"2024-09-21T07:47:11.618663+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:08:59.514757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":259,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.873010+0000","last_change":"2024-09-21T07:35:27.878728+0000","last_active":"2024-09-21T07:47:10.873010+0000","last_peered":"2024-09-21T07:47:10.873010+0000","last_clean":"2024-09-21T07:47:10.873010+0000","last_became_active":"2024-09-21T07:35:27.878335+0000","last_became_peered":"2024-09-21T07:35:27.878335+0000","last_unstale":"2024-09-21T07:47:10.873010+0000","last_undegraded":"2024-09-21T07:47:10.873010+0000","last_fullsized":"2024-09-21T07:47:10.873010+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:25:37.731771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":275,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870640+0000","last_change":"2024-09-21T07:35:28.858418+0000","last_active":"2024-09-21T07:47:10.870640+0000","last_peered":"2024-09-21T07:47:10.870640+0000","last_clean":"2024-09-21T07:47:10.870640+0000","last_became_active":"2024-09-21T07:35:26.876944+0000","last_became_peered":"2024-09-21T07:35:26.876944+0000","last_unstale":"2024-09-21T07:47:10.870640+0000","last_undegraded":"2024-09-21T07:47:10.870640+0000","last_fullsized":"2024-09-21T07:47:10.870640+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:04.514304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090862799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":314,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.640105+0000","last_change":"2024-09-21T07:34:24.597926+0000","last_active":"2024-09-21T07:47:10.640105+0000","last_peered":"2024-09-21T07:47:10.640105+0000","last_clean":"2024-09-21T07:47:10.640105+0000","last_became_active":"2024-09-21T07:34:23.268926+0000","last_became_peered":"2024-09-21T07:34:23.268926+0000","last_unstale":"2024-09-21T07:47:10.640105+0000","last_undegraded":"2024-09-21T07:47:10.640105+0000","last_fullsized":"2024-09-21T07:47:10.640105+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:46.190752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101851,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870946+0000","last_change":"2024-09-21T07:47:09.626390+0000","last_active":"2024-09-21T07:47:10.870946+0000","last_peered":"2024-09-21T07:47:10.870946+0000","last_clean":"2024-09-21T07:47:10.870946+0000","last_became_active":"2024-09-21T07:47:09.626041+0000","last_became_peered":"2024-09-21T07:47:09.626041+0000","last_unstale":"2024-09-21T07:47:10.870946+0000","last_undegraded":"2024-09-21T07:47:10.870946+0000","last_fullsized":"2024-09-21T07:47:10.870946+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:54:53.082878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":15,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638366+0000","last_change":"2024-09-21T07:47:08.635992+0000","last_active":"2024-09-21T07:47:10.638366+0000","last_peered":"2024-09-21T07:47:10.638366+0000","last_clean":"2024-09-21T07:47:10.638366+0000","last_became_active":"2024-09-21T07:47:08.635692+0000","last_became_peered":"2024-09-21T07:47:08.635692+0000","last_unstale":"2024-09-21T07:47:10.638366+0000","last_undegraded":"2024-09-21T07:47:10.638366+0000","last_fullsized":"2024-09-21T07:47:10.638366+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:10:08.663465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":258,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.639081+0000","last_change":"2024-09-21T07:35:27.877426+0000","last_active":"2024-09-21T07:47:10.639081+0000","last_peered":"2024-09-21T07:47:10.639081+0000","last_clean":"2024-09-21T07:47:10.639081+0000","last_became_active":"2024-09-21T07:35:27.877050+0000","last_became_peered":"2024-09-21T07:35:27.877050+0000","last_unstale":"2024-09-21T07:47:10.639081+0000","last_undegraded":"2024-09-21T07:47:10.639081+0000","last_fullsized":"2024-09-21T07:47:10.639081+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:36.268334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":266,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869544+0000","last_change":"2024-09-21T07:35:28.862273+0000","last_active":"2024-09-21T07:47:10.869544+0000","last_peered":"2024-09-21T07:47:10.869544+0000","last_clean":"2024-09-21T07:47:10.869544+0000","last_became_active":"2024-09-21T07:35:26.874011+0000","last_became_peered":"2024-09-21T07:35:26.874011+0000","last_unstale":"2024-09-21T07:47:10.869544+0000","last_undegraded":"2024-09-21T07:47:10.869544+0000","last_fullsized":"2024-09-21T07:47:10.869544+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:11.986950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071644799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":321,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.872044+0000","last_change":"2024-09-21T07:34:24.259794+0000","last_active":"2024-09-21T07:47:10.872044+0000","last_peered":"2024-09-21T07:47:10.872044+0000","last_clean":"2024-09-21T07:47:10.872044+0000","last_became_active":"2024-09-21T07:34:23.265679+0000","last_became_peered":"2024-09-21T07:34:23.265679+0000","last_unstale":"2024-09-21T07:47:10.872044+0000","last_undegraded":"2024-09-21T07:47:10.872044+0000","last_fullsized":"2024-09-21T07:47:10.872044+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:02.575796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001298518,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.873523+0000","last_change":"2024-09-21T07:47:09.647670+0000","last_active":"2024-09-21T07:47:10.873523+0000","last_peered":"2024-09-21T07:47:10.873523+0000","last_clean":"2024-09-21T07:47:10.873523+0000","last_became_active":"2024-09-21T07:47:09.647386+0000","last_became_peered":"2024-09-21T07:47:09.647386+0000","last_unstale":"2024-09-21T07:47:10.873523+0000","last_undegraded":"2024-09-21T07:47:10.873523+0000","last_fullsized":"2024-09-21T07:47:10.873523+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:40:35.420329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":15,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871854+0000","last_change":"2024-09-21T07:47:08.634935+0000","last_active":"2024-09-21T07:47:10.871854+0000","last_peered":"2024-09-21T07:47:10.871854+0000","last_clean":"2024-09-21T07:47:10.871854+0000","last_became_active":"2024-09-21T07:47:08.634627+0000","last_became_peered":"2024-09-21T07:47:08.634627+0000","last_unstale":"2024-09-21T07:47:10.871854+0000","last_undegraded":"2024-09-21T07:47:10.871854+0000","last_fullsized":"2024-09-21T07:47:10.871854+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:17.077512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":262,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871229+0000","last_change":"2024-09-21T07:35:27.877725+0000","last_active":"2024-09-21T07:47:10.871229+0000","last_peered":"2024-09-21T07:47:10.871229+0000","last_clean":"2024-09-21T07:47:10.871229+0000","last_became_active":"2024-09-21T07:35:27.877398+0000","last_became_peered":"2024-09-21T07:35:27.877398+0000","last_unstale":"2024-09-21T07:47:10.871229+0000","last_undegraded":"2024-09-21T07:47:10.871229+0000","last_fullsized":"2024-09-21T07:47:10.871229+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:05.859762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":268,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869613+0000","last_change":"2024-09-21T07:35:28.860612+0000","last_active":"2024-09-21T07:47:10.869613+0000","last_peered":"2024-09-21T07:47:10.869613+0000","last_clean":"2024-09-21T07:47:10.869613+0000","last_became_active":"2024-09-21T07:35:26.876562+0000","last_became_peered":"2024-09-21T07:35:26.876562+0000","last_unstale":"2024-09-21T07:47:10.869613+0000","last_undegraded":"2024-09-21T07:47:10.869613+0000","last_fullsized":"2024-09-21T07:47:10.869613+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:06.824331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012252470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":322,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.640664+0000","last_change":"2024-09-21T07:34:24.260330+0000","last_active":"2024-09-21T07:47:10.640664+0000","last_peered":"2024-09-21T07:47:10.640664+0000","last_clean":"2024-09-21T07:47:10.640664+0000","last_became_active":"2024-09-21T07:34:23.637210+0000","last_became_peered":"2024-09-21T07:34:23.637210+0000","last_unstale":"2024-09-21T07:47:10.640664+0000","last_undegraded":"2024-09-21T07:47:10.640664+0000","last_fullsized":"2024-09-21T07:47:10.640664+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.488451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001537093,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638496+0000","last_change":"2024-09-21T07:47:09.646594+0000","last_active":"2024-09-21T07:47:10.638496+0000","last_peered":"2024-09-21T07:47:10.638496+0000","last_clean":"2024-09-21T07:47:10.638496+0000","last_became_active":"2024-09-21T07:47:09.645970+0000","last_became_peered":"2024-09-21T07:47:09.645970+0000","last_unstale":"2024-09-21T07:47:10.638496+0000","last_undegraded":"2024-09-21T07:47:10.638496+0000","last_fullsized":"2024-09-21T07:47:10.638496+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:52.093212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"193'2","reported_seq":17,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870067+0000","last_change":"2024-09-21T07:47:08.639174+0000","last_active":"2024-09-21T07:47:10.870067+0000","last_peered":"2024-09-21T07:47:10.870067+0000","last_clean":"2024-09-21T07:47:10.870067+0000","last_became_active":"2024-09-21T07:47:08.638830+0000","last_became_peered":"2024-09-21T07:47:08.638830+0000","last_unstale":"2024-09-21T07:47:10.870067+0000","last_undegraded":"2024-09-21T07:47:10.870067+0000","last_fullsized":"2024-09-21T07:47:10.870067+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:42.824859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":260,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869100+0000","last_change":"2024-09-21T07:35:27.874757+0000","last_active":"2024-09-21T07:47:10.869100+0000","last_peered":"2024-09-21T07:47:10.869100+0000","last_clean":"2024-09-21T07:47:10.869100+0000","last_became_active":"2024-09-21T07:35:27.874360+0000","last_became_peered":"2024-09-21T07:35:27.874360+0000","last_unstale":"2024-09-21T07:47:10.869100+0000","last_undegraded":"2024-09-21T07:47:10.869100+0000","last_fullsized":"2024-09-21T07:47:10.869100+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:17.646912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":268,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.640925+0000","last_change":"2024-09-21T07:35:29.493431+0000","last_active":"2024-09-21T07:47:10.640925+0000","last_peered":"2024-09-21T07:47:10.640925+0000","last_clean":"2024-09-21T07:47:10.640925+0000","last_became_active":"2024-09-21T07:35:26.873691+0000","last_became_peered":"2024-09-21T07:35:26.873691+0000","last_unstale":"2024-09-21T07:47:10.640925+0000","last_undegraded":"2024-09-21T07:47:10.640925+0000","last_fullsized":"2024-09-21T07:47:10.640925+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:51.705051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001162464,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":314,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638440+0000","last_change":"2024-09-21T07:34:24.597286+0000","last_active":"2024-09-21T07:47:10.638440+0000","last_peered":"2024-09-21T07:47:10.638440+0000","last_clean":"2024-09-21T07:47:10.638440+0000","last_became_active":"2024-09-21T07:34:23.267133+0000","last_became_peered":"2024-09-21T07:34:23.267133+0000","last_unstale":"2024-09-21T07:47:10.638440+0000","last_undegraded":"2024-09-21T07:47:10.638440+0000","last_fullsized":"2024-09-21T07:47:10.638440+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:18.257010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.872293+0000","last_change":"2024-09-21T07:47:09.664200+0000","last_active":"2024-09-21T07:47:10.872293+0000","last_peered":"2024-09-21T07:47:10.872293+0000","last_clean":"2024-09-21T07:47:10.872293+0000","last_became_active":"2024-09-21T07:47:09.663850+0000","last_became_peered":"2024-09-21T07:47:09.663850+0000","last_unstale":"2024-09-21T07:47:10.872293+0000","last_undegraded":"2024-09-21T07:47:10.872293+0000","last_fullsized":"2024-09-21T07:47:10.872293+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:13:19.001858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":15,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870512+0000","last_change":"2024-09-21T07:47:08.638721+0000","last_active":"2024-09-21T07:47:10.870512+0000","last_peered":"2024-09-21T07:47:10.870512+0000","last_clean":"2024-09-21T07:47:10.870512+0000","last_became_active":"2024-09-21T07:47:08.638320+0000","last_became_peered":"2024-09-21T07:47:08.638320+0000","last_unstale":"2024-09-21T07:47:10.870512+0000","last_undegraded":"2024-09-21T07:47:10.870512+0000","last_fullsized":"2024-09-21T07:47:10.870512+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:00.571043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":269,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871600+0000","last_change":"2024-09-21T07:35:27.864458+0000","last_active":"2024-09-21T07:47:10.871600+0000","last_peered":"2024-09-21T07:47:10.871600+0000","last_clean":"2024-09-21T07:47:10.871600+0000","last_became_active":"2024-09-21T07:35:27.864136+0000","last_became_peered":"2024-09-21T07:35:27.864136+0000","last_unstale":"2024-09-21T07:47:10.871600+0000","last_undegraded":"2024-09-21T07:47:10.871600+0000","last_fullsized":"2024-09-21T07:47:10.871600+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:20.162240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":275,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871257+0000","last_change":"2024-09-21T07:35:28.858784+0000","last_active":"2024-09-21T07:47:10.871257+0000","last_peered":"2024-09-21T07:47:10.871257+0000","last_clean":"2024-09-21T07:47:10.871257+0000","last_became_active":"2024-09-21T07:35:26.876890+0000","last_became_peered":"2024-09-21T07:35:26.876890+0000","last_unstale":"2024-09-21T07:47:10.871257+0000","last_undegraded":"2024-09-21T07:47:10.871257+0000","last_fullsized":"2024-09-21T07:47:10.871257+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:40.738327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064543499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":324,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638180+0000","last_change":"2024-09-21T07:34:24.258712+0000","last_active":"2024-09-21T07:47:10.638180+0000","last_peered":"2024-09-21T07:47:10.638180+0000","last_clean":"2024-09-21T07:47:10.638180+0000","last_became_active":"2024-09-21T07:34:23.274692+0000","last_became_peered":"2024-09-21T07:34:23.274692+0000","last_unstale":"2024-09-21T07:47:10.638180+0000","last_undegraded":"2024-09-21T07:47:10.638180+0000","last_fullsized":"2024-09-21T07:47:10.638180+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:43.279454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018051479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870518+0000","last_change":"2024-09-21T07:47:09.650215+0000","last_active":"2024-09-21T07:47:10.870518+0000","last_peered":"2024-09-21T07:47:10.870518+0000","last_clean":"2024-09-21T07:47:10.870518+0000","last_became_active":"2024-09-21T07:47:09.649552+0000","last_became_peered":"2024-09-21T07:47:09.649552+0000","last_unstale":"2024-09-21T07:47:10.870518+0000","last_undegraded":"2024-09-21T07:47:10.870518+0000","last_fullsized":"2024-09-21T07:47:10.870518+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:39.943614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":15,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870698+0000","last_change":"2024-09-21T07:47:08.626468+0000","last_active":"2024-09-21T07:47:10.870698+0000","last_peered":"2024-09-21T07:47:10.870698+0000","last_clean":"2024-09-21T07:47:10.870698+0000","last_became_active":"2024-09-21T07:47:08.626131+0000","last_became_peered":"2024-09-21T07:47:08.626131+0000","last_unstale":"2024-09-21T07:47:10.870698+0000","last_undegraded":"2024-09-21T07:47:10.870698+0000","last_fullsized":"2024-09-21T07:47:10.870698+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:18.852388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":263,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.637904+0000","last_change":"2024-09-21T07:35:27.878020+0000","last_active":"2024-09-21T07:47:10.637904+0000","last_peered":"2024-09-21T07:47:10.637904+0000","last_clean":"2024-09-21T07:47:10.637904+0000","last_became_active":"2024-09-21T07:35:27.877532+0000","last_became_peered":"2024-09-21T07:35:27.877532+0000","last_unstale":"2024-09-21T07:47:10.637904+0000","last_undegraded":"2024-09-21T07:47:10.637904+0000","last_fullsized":"2024-09-21T07:47:10.637904+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:19.756073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":268,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.872738+0000","last_change":"2024-09-21T07:35:28.861049+0000","last_active":"2024-09-21T07:47:10.872738+0000","last_peered":"2024-09-21T07:47:10.872738+0000","last_clean":"2024-09-21T07:47:10.872738+0000","last_became_active":"2024-09-21T07:35:26.876819+0000","last_became_peered":"2024-09-21T07:35:26.876819+0000","last_unstale":"2024-09-21T07:47:10.872738+0000","last_undegraded":"2024-09-21T07:47:10.872738+0000","last_fullsized":"2024-09-21T07:47:10.872738+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:19.633803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055780599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":318,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869280+0000","last_change":"2024-09-21T07:34:25.084849+0000","last_active":"2024-09-21T07:47:10.869280+0000","last_peered":"2024-09-21T07:47:10.869280+0000","last_clean":"2024-09-21T07:47:10.869280+0000","last_became_active":"2024-09-21T07:34:23.277713+0000","last_became_peered":"2024-09-21T07:34:23.277713+0000","last_unstale":"2024-09-21T07:47:10.869280+0000","last_undegraded":"2024-09-21T07:47:10.869280+0000","last_fullsized":"2024-09-21T07:47:10.869280+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:05.811810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059079499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.640785+0000","last_change":"2024-09-21T07:47:09.643720+0000","last_active":"2024-09-21T07:47:10.640785+0000","last_peered":"2024-09-21T07:47:10.640785+0000","last_clean":"2024-09-21T07:47:10.640785+0000","last_became_active":"2024-09-21T07:47:09.643421+0000","last_became_peered":"2024-09-21T07:47:09.643421+0000","last_unstale":"2024-09-21T07:47:10.640785+0000","last_undegraded":"2024-09-21T07:47:10.640785+0000","last_fullsized":"2024-09-21T07:47:10.640785+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:01:04.956620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":15,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638057+0000","last_change":"2024-09-21T07:47:08.628390+0000","last_active":"2024-09-21T07:47:10.638057+0000","last_peered":"2024-09-21T07:47:10.638057+0000","last_clean":"2024-09-21T07:47:10.638057+0000","last_became_active":"2024-09-21T07:47:08.628015+0000","last_became_peered":"2024-09-21T07:47:08.628015+0000","last_unstale":"2024-09-21T07:47:10.638057+0000","last_undegraded":"2024-09-21T07:47:10.638057+0000","last_fullsized":"2024-09-21T07:47:10.638057+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:15.112979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":262,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.873712+0000","last_change":"2024-09-21T07:35:27.884248+0000","last_active":"2024-09-21T07:47:10.873712+0000","last_peered":"2024-09-21T07:47:10.873712+0000","last_clean":"2024-09-21T07:47:10.873712+0000","last_became_active":"2024-09-21T07:35:27.883861+0000","last_became_peered":"2024-09-21T07:35:27.883861+0000","last_unstale":"2024-09-21T07:47:10.873712+0000","last_undegraded":"2024-09-21T07:47:10.873712+0000","last_fullsized":"2024-09-21T07:47:10.873712+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":269,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870468+0000","last_change":"2024-09-21T07:35:29.207135+0000","last_active":"2024-09-21T07:47:10.870468+0000","last_peered":"2024-09-21T07:47:10.870468+0000","last_clean":"2024-09-21T07:47:10.870468+0000","last_became_active":"2024-09-21T07:35:26.872787+0000","last_became_peered":"2024-09-21T07:35:26.872787+0000","last_unstale":"2024-09-21T07:47:10.870468+0000","last_undegraded":"2024-09-21T07:47:10.870468+0000","last_fullsized":"2024-09-21T07:47:10.870468+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:27.543217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011157509999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":328,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871681+0000","last_change":"2024-09-21T07:34:24.538610+0000","last_active":"2024-09-21T07:47:10.871681+0000","last_peered":"2024-09-21T07:47:10.871681+0000","last_clean":"2024-09-21T07:47:10.871681+0000","last_became_active":"2024-09-21T07:34:23.259700+0000","last_became_peered":"2024-09-21T07:34:23.259700+0000","last_unstale":"2024-09-21T07:47:10.871681+0000","last_undegraded":"2024-09-21T07:47:10.871681+0000","last_fullsized":"2024-09-21T07:47:10.871681+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:57.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070522399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.873170+0000","last_change":"2024-09-21T07:47:09.641729+0000","last_active":"2024-09-21T07:47:10.873170+0000","last_peered":"2024-09-21T07:47:10.873170+0000","last_clean":"2024-09-21T07:47:10.873170+0000","last_became_active":"2024-09-21T07:47:09.641130+0000","last_became_peered":"2024-09-21T07:47:09.641130+0000","last_unstale":"2024-09-21T07:47:10.873170+0000","last_undegraded":"2024-09-21T07:47:10.873170+0000","last_fullsized":"2024-09-21T07:47:10.873170+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:00:03.886850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"193'1","reported_seq":16,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638773+0000","last_change":"2024-09-21T07:47:08.644619+0000","last_active":"2024-09-21T07:47:10.638773+0000","last_peered":"2024-09-21T07:47:10.638773+0000","last_clean":"2024-09-21T07:47:10.638773+0000","last_became_active":"2024-09-21T07:47:08.642613+0000","last_became_peered":"2024-09-21T07:47:08.642613+0000","last_unstale":"2024-09-21T07:47:10.638773+0000","last_undegraded":"2024-09-21T07:47:10.638773+0000","last_fullsized":"2024-09-21T07:47:10.638773+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:02:30.516252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":269,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870288+0000","last_change":"2024-09-21T07:35:27.868024+0000","last_active":"2024-09-21T07:47:10.870288+0000","last_peered":"2024-09-21T07:47:10.870288+0000","last_clean":"2024-09-21T07:47:10.870288+0000","last_became_active":"2024-09-21T07:35:27.867339+0000","last_became_peered":"2024-09-21T07:35:27.867339+0000","last_unstale":"2024-09-21T07:47:10.870288+0000","last_undegraded":"2024-09-21T07:47:10.870288+0000","last_fullsized":"2024-09-21T07:47:10.870288+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":276,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870718+0000","last_change":"2024-09-21T07:35:28.858505+0000","last_active":"2024-09-21T07:47:10.870718+0000","last_peered":"2024-09-21T07:47:10.870718+0000","last_clean":"2024-09-21T07:47:10.870718+0000","last_became_active":"2024-09-21T07:35:26.877320+0000","last_became_peered":"2024-09-21T07:35:26.877320+0000","last_unstale":"2024-09-21T07:47:10.870718+0000","last_undegraded":"2024-09-21T07:47:10.870718+0000","last_fullsized":"2024-09-21T07:47:10.870718+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:53.672782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244119999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":315,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.639400+0000","last_change":"2024-09-21T07:34:24.597597+0000","last_active":"2024-09-21T07:47:10.639400+0000","last_peered":"2024-09-21T07:47:10.639400+0000","last_clean":"2024-09-21T07:47:10.639400+0000","last_became_active":"2024-09-21T07:34:23.635204+0000","last_became_peered":"2024-09-21T07:34:23.635204+0000","last_unstale":"2024-09-21T07:47:10.639400+0000","last_undegraded":"2024-09-21T07:47:10.639400+0000","last_fullsized":"2024-09-21T07:47:10.639400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:48.644789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081139199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869986+0000","last_change":"2024-09-21T07:47:09.635405+0000","last_active":"2024-09-21T07:47:10.869986+0000","last_peered":"2024-09-21T07:47:10.869986+0000","last_clean":"2024-09-21T07:47:10.869986+0000","last_became_active":"2024-09-21T07:47:09.634326+0000","last_became_peered":"2024-09-21T07:47:09.634326+0000","last_unstale":"2024-09-21T07:47:10.869986+0000","last_undegraded":"2024-09-21T07:47:10.869986+0000","last_fullsized":"2024-09-21T07:47:10.869986+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:46.468467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"193'2","reported_seq":17,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.874377+0000","last_change":"2024-09-21T07:47:08.633421+0000","last_active":"2024-09-21T07:47:10.874377+0000","last_peered":"2024-09-21T07:47:10.874377+0000","last_clean":"2024-09-21T07:47:10.874377+0000","last_became_active":"2024-09-21T07:47:08.633080+0000","last_became_peered":"2024-09-21T07:47:08.633080+0000","last_unstale":"2024-09-21T07:47:10.874377+0000","last_undegraded":"2024-09-21T07:47:10.874377+0000","last_fullsized":"2024-09-21T07:47:10.874377+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:19.537094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":261,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870241+0000","last_change":"2024-09-21T07:35:27.877503+0000","last_active":"2024-09-21T07:47:10.870241+0000","last_peered":"2024-09-21T07:47:10.870241+0000","last_clean":"2024-09-21T07:47:10.870241+0000","last_became_active":"2024-09-21T07:35:27.875775+0000","last_became_peered":"2024-09-21T07:35:27.875775+0000","last_unstale":"2024-09-21T07:47:10.870241+0000","last_undegraded":"2024-09-21T07:47:10.870241+0000","last_fullsized":"2024-09-21T07:47:10.870241+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:49.291023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":269,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.639729+0000","last_change":"2024-09-21T07:35:29.491412+0000","last_active":"2024-09-21T07:47:10.639729+0000","last_peered":"2024-09-21T07:47:10.639729+0000","last_clean":"2024-09-21T07:47:10.639729+0000","last_became_active":"2024-09-21T07:35:26.876404+0000","last_became_peered":"2024-09-21T07:35:26.876404+0000","last_unstale":"2024-09-21T07:47:10.639729+0000","last_undegraded":"2024-09-21T07:47:10.639729+0000","last_fullsized":"2024-09-21T07:47:10.639729+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063918000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":325,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.638838+0000","last_change":"2024-09-21T07:34:24.259026+0000","last_active":"2024-09-21T07:47:10.638838+0000","last_peered":"2024-09-21T07:47:10.638838+0000","last_clean":"2024-09-21T07:47:10.638838+0000","last_became_active":"2024-09-21T07:34:23.635456+0000","last_became_peered":"2024-09-21T07:34:23.635456+0000","last_unstale":"2024-09-21T07:47:10.638838+0000","last_undegraded":"2024-09-21T07:47:10.638838+0000","last_fullsized":"2024-09-21T07:47:10.638838+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:09.592752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018654419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870438+0000","last_change":"2024-09-21T07:47:09.635440+0000","last_active":"2024-09-21T07:47:10.870438+0000","last_peered":"2024-09-21T07:47:10.870438+0000","last_clean":"2024-09-21T07:47:10.870438+0000","last_became_active":"2024-09-21T07:47:09.635009+0000","last_became_peered":"2024-09-21T07:47:09.635009+0000","last_unstale":"2024-09-21T07:47:10.870438+0000","last_undegraded":"2024-09-21T07:47:10.870438+0000","last_fullsized":"2024-09-21T07:47:10.870438+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:53.955407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"193'1","reported_seq":16,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.872853+0000","last_change":"2024-09-21T07:47:08.644049+0000","last_active":"2024-09-21T07:47:10.872853+0000","last_peered":"2024-09-21T07:47:10.872853+0000","last_clean":"2024-09-21T07:47:10.872853+0000","last_became_active":"2024-09-21T07:47:08.643739+0000","last_became_peered":"2024-09-21T07:47:08.643739+0000","last_unstale":"2024-09-21T07:47:10.872853+0000","last_undegraded":"2024-09-21T07:47:10.872853+0000","last_fullsized":"2024-09-21T07:47:10.872853+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:36.097864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":261,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870867+0000","last_change":"2024-09-21T07:35:27.871984+0000","last_active":"2024-09-21T07:47:10.870867+0000","last_peered":"2024-09-21T07:47:10.870867+0000","last_clean":"2024-09-21T07:47:10.870867+0000","last_became_active":"2024-09-21T07:35:27.870336+0000","last_became_peered":"2024-09-21T07:35:27.870336+0000","last_unstale":"2024-09-21T07:47:10.870867+0000","last_undegraded":"2024-09-21T07:47:10.870867+0000","last_fullsized":"2024-09-21T07:47:10.870867+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:57.120479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":269,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.870906+0000","last_change":"2024-09-21T07:35:28.860351+0000","last_active":"2024-09-21T07:47:10.870906+0000","last_peered":"2024-09-21T07:47:10.870906+0000","last_clean":"2024-09-21T07:47:10.870906+0000","last_became_active":"2024-09-21T07:35:26.872046+0000","last_became_peered":"2024-09-21T07:35:26.872046+0000","last_unstale":"2024-09-21T07:47:10.870906+0000","last_undegraded":"2024-09-21T07:47:10.870906+0000","last_fullsized":"2024-09-21T07:47:10.870906+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:47.074327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098384399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":322,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.640432+0000","last_change":"2024-09-21T07:34:24.260300+0000","last_active":"2024-09-21T07:47:10.640432+0000","last_peered":"2024-09-21T07:47:10.640432+0000","last_clean":"2024-09-21T07:47:10.640432+0000","last_became_active":"2024-09-21T07:34:23.270128+0000","last_became_peered":"2024-09-21T07:34:23.270128+0000","last_unstale":"2024-09-21T07:47:10.640432+0000","last_undegraded":"2024-09-21T07:47:10.640432+0000","last_fullsized":"2024-09-21T07:47:10.640432+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:25.260931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00176651,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":13,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869988+0000","last_change":"2024-09-21T07:47:09.641302+0000","last_active":"2024-09-21T07:47:10.869988+0000","last_peered":"2024-09-21T07:47:10.869988+0000","last_clean":"2024-09-21T07:47:10.869988+0000","last_became_active":"2024-09-21T07:47:09.640989+0000","last_became_peered":"2024-09-21T07:47:09.640989+0000","last_unstale":"2024-09-21T07:47:10.869988+0000","last_undegraded":"2024-09-21T07:47:10.869988+0000","last_fullsized":"2024-09-21T07:47:10.869988+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:08.600233+0000","last_clean_scrub_stamp":"2024-09-21T07:47:08.600233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:54:27.135545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":15,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.872669+0000","last_change":"2024-09-21T07:47:08.641672+0000","last_active":"2024-09-21T07:47:10.872669+0000","last_peered":"2024-09-21T07:47:10.872669+0000","last_clean":"2024-09-21T07:47:10.872669+0000","last_became_active":"2024-09-21T07:47:08.641351+0000","last_became_peered":"2024-09-21T07:47:08.641351+0000","last_unstale":"2024-09-21T07:47:10.872669+0000","last_undegraded":"2024-09-21T07:47:10.872669+0000","last_fullsized":"2024-09-21T07:47:10.872669+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:07.593555+0000","last_clean_scrub_stamp":"2024-09-21T07:47:07.593555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:02:42.466658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":261,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.871333+0000","last_change":"2024-09-21T07:35:27.877861+0000","last_active":"2024-09-21T07:47:10.871333+0000","last_peered":"2024-09-21T07:47:10.871333+0000","last_clean":"2024-09-21T07:47:10.871333+0000","last_became_active":"2024-09-21T07:35:27.877434+0000","last_became_peered":"2024-09-21T07:35:27.877434+0000","last_unstale":"2024-09-21T07:47:10.871333+0000","last_undegraded":"2024-09-21T07:47:10.871333+0000","last_fullsized":"2024-09-21T07:47:10.871333+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":267,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.869461+0000","last_change":"2024-09-21T07:35:28.862954+0000","last_active":"2024-09-21T07:47:10.869461+0000","last_peered":"2024-09-21T07:47:10.869461+0000","last_clean":"2024-09-21T07:47:10.869461+0000","last_became_active":"2024-09-21T07:35:26.870613+0000","last_became_peered":"2024-09-21T07:35:26.870613+0000","last_unstale":"2024-09-21T07:47:10.869461+0000","last_undegraded":"2024-09-21T07:47:10.869461+0000","last_fullsized":"2024-09-21T07:47:10.869461+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:06.901043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059842100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":320,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-21T07:47:10.874114+0000","last_change":"2024-09-21T07:34:25.028904+0000","last_active":"2024-09-21T07:47:10.874114+0000","last_peered":"2024-09-21T07:47:10.874114+0000","last_clean":"2024-09-21T07:47:10.874114+0000","last_became_active":"2024-09-21T07:34:23.260942+0000","last_became_peered":"2024-09-21T07:34:23.260942+0000","last_unstale":"2024-09-21T07:47:10.874114+0000","last_undegraded":"2024-09-21T07:47:10.874114+0000","last_fullsized":"2024-09-21T07:47:10.874114+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:22.709328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060553899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509613,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66080,"kb_used_data":2956,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94305760,"statfs":{"total":96636764160,"available":96569098240,"internally_reserved":0,"allocated":3026944,"data_stored":1884957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509615,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51616,"kb_used_data":2376,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320224,"statfs":{"total":96636764160,"available":96583909376,"internally_reserved":0,"allocated":2433024,"data_stored":1294589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509613,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29416,"kb_used_data":2384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342424,"statfs":{"total":96636764160,"available":96606642176,"internally_reserved":0,"allocated":2441216,"data_stored":1294713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509614,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46048,"kb_used_data":2376,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94325792,"statfs":{"total":96636764160,"available":96589611008,"internally_reserved":0,"allocated":2433024,"data_stored":1294589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509615,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41900,"kb_used_data":2388,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329940,"statfs":{"total":96636764160,"available":96593858560,"internally_reserved":0,"allocated":2445312,"data_stored":1296201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509615,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52208,"kb_used_data":2964,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319632,"statfs":{"total":96636764160,"available":96583303168,"internally_reserved":0,"allocated":3035136,"data_stored":1885061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542318,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51636,"kb_used_data":2392,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320204,"statfs":{"total":96636764160,"available":96583888896,"internally_reserved":0,"allocated":2449408,"data_stored":1295353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542317,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73844,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297996,"statfs":{"total":96636764160,"available":96561147904,"internally_reserved":0,"allocated":2453504,"data_stored":1296805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:47:14.631 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T07:47:14.631 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T07:47:14.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.897+0000 7feec2800640 1 -- 172.21.15.39:0/886792018 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feebc072220 msgr2=0x7feebc0d0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:14.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.897+0000 7feec2800640 1 --2- 172.21.15.39:0/886792018 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feebc072220 0x7feebc0d0c50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7feeb4005fd0 tx=0x7feeb402aaa0 comp rx=0 tx=0).stop 2024-09-21T07:47:14.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.897+0000 7feec2800640 1 -- 172.21.15.39:0/886792018 shutdown_connections 2024-09-21T07:47:14.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.897+0000 7feec2800640 1 --2- 172.21.15.39:0/886792018 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feebc072220 0x7feebc0d0c50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:14.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.897+0000 7feec2800640 1 --2- 172.21.15.39:0/886792018 >> v2:172.21.15.148:3300/0 conn(0x7feebc0cd590 0x7feebc0cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:14.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.897+0000 7feec2800640 1 -- 172.21.15.39:0/886792018 >> 172.21.15.39:0/886792018 conn(0x7feebc0cefe0 msgr2=0x7feebc0cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:14.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.897+0000 7feec2800640 1 -- 172.21.15.39:0/886792018 shutdown_connections 2024-09-21T07:47:14.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.898+0000 7feec2800640 1 -- 172.21.15.39:0/886792018 wait complete. 2024-09-21T07:47:14.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.900+0000 7feec2800640 1 Processor -- start 2024-09-21T07:47:14.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.900+0000 7feec2800640 1 -- start start 2024-09-21T07:47:14.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.901+0000 7feec2800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feebc072220 0x7feebc0cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:14.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.901+0000 7feec2800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feebc0cd590 0x7feebc1612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:14.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.901+0000 7feec2800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feebc0d0130 0x7feebc1f91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:14.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.901+0000 7feec2800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7feebc0c5d70 con 0x7feebc0cd590 2024-09-21T07:47:14.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.901+0000 7feec2800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7feebc0c5bf0 con 0x7feebc0d0130 2024-09-21T07:47:14.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.901+0000 7feec2800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7feebc167230 con 0x7feebc072220 2024-09-21T07:47:14.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.901+0000 7feeb3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feebc0cd590 0x7feebc1612a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:14.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.902+0000 7feeb3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feebc0cd590 0x7feebc1612a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48346/0 (socket says 172.21.15.39:48346) 2024-09-21T07:47:14.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.902+0000 7feeb3e00640 1 -- 172.21.15.39:0/3200298346 learned_addr learned my addr 172.21.15.39:0/3200298346 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:14.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.902+0000 7feebbe00640 1 --2- 172.21.15.39:0/3200298346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feebc0d0130 0x7feebc1f91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:14.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.902+0000 7feebb400640 1 --2- 172.21.15.39:0/3200298346 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feebc072220 0x7feebc0cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:14.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.902+0000 7feeb3e00640 1 -- 172.21.15.39:0/3200298346 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feebc072220 msgr2=0x7feebc0cc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:14.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.902+0000 7feeb3e00640 1 --2- 172.21.15.39:0/3200298346 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feebc072220 0x7feebc0cc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:14.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.902+0000 7feeb3e00640 1 -- 172.21.15.39:0/3200298346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feebc0d0130 msgr2=0x7feebc1f91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:14.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.902+0000 7feeb3e00640 1 --2- 172.21.15.39:0/3200298346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feebc0d0130 0x7feebc1f91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:14.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.902+0000 7feeb3e00640 1 -- 172.21.15.39:0/3200298346 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feebc1f6710 con 0x7feebc0cd590 2024-09-21T07:47:14.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.902+0000 7feebb400640 1 --2- 172.21.15.39:0/3200298346 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feebc072220 0x7feebc0cc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:14.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.902+0000 7feebbe00640 1 --2- 172.21.15.39:0/3200298346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feebc0d0130 0x7feebc1f91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:14.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.902+0000 7feeb3e00640 1 --2- 172.21.15.39:0/3200298346 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feebc0cd590 0x7feebc1612a0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7feea400c3d0 tx=0x7feea400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:14.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.902+0000 7feeb3400640 1 -- 172.21.15.39:0/3200298346 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea4017070 con 0x7feebc0cd590 2024-09-21T07:47:14.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.903+0000 7feeb3400640 1 -- 172.21.15.39:0/3200298346 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feea40053c0 con 0x7feebc0cd590 2024-09-21T07:47:14.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.903+0000 7feec2800640 1 -- 172.21.15.39:0/3200298346 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feebc1f69f0 con 0x7feebc0cd590 2024-09-21T07:47:14.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.903+0000 7feeb3400640 1 -- 172.21.15.39:0/3200298346 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea4003850 con 0x7feebc0cd590 2024-09-21T07:47:14.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.903+0000 7feec2800640 1 -- 172.21.15.39:0/3200298346 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feebc0c7cd0 con 0x7feebc0cd590 2024-09-21T07:47:14.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.904+0000 7feec2800640 1 -- 172.21.15.39:0/3200298346 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feebc0cd9a0 con 0x7feebc0cd590 2024-09-21T07:47:14.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.905+0000 7feeb3400640 1 -- 172.21.15.39:0/3200298346 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7feea401a070 con 0x7feebc0cd590 2024-09-21T07:47:14.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.906+0000 7feeb3400640 1 --2- 172.21.15.39:0/3200298346 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fee98078b90 0x7fee9807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:14.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.906+0000 7feeb3400640 1 -- 172.21.15.39:0/3200298346 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 6932+0+0 (secure 0 0 0) 0x7feea409a660 con 0x7feebc0cd590 2024-09-21T07:47:14.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.907+0000 7feebb400640 1 --2- 172.21.15.39:0/3200298346 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fee98078b90 0x7fee9807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:14.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.907+0000 7feebb400640 1 --2- 172.21.15.39:0/3200298346 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fee98078b90 0x7fee9807b0a0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7feeac009b80 tx=0x7feeac005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:14.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:14.911+0000 7feeb3400640 1 -- 172.21.15.39:0/3200298346 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7feea4062540 con 0x7feebc0cd590 2024-09-21T07:47:15.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.179+0000 7feec2800640 1 -- 172.21.15.39:0/3200298346 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7feebc167010 con 0x7feebc0cd590 2024-09-21T07:47:15.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.187+0000 7feeb3400640 1 -- 172.21.15.39:0/3200298346 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) ==== 190+0+147 (secure 0 0 0) 0x7feea4061e40 con 0x7feebc0cd590 2024-09-21T07:47:15.189 INFO:teuthology.orchestra.run.smithi039.stdout:[client.0] 2024-09-21T07:47:15.189 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQB0du5mGfb1DBAA/aSMKR5or3xy53rB/iUjpA== 2024-09-21T07:47:15.189 INFO:teuthology.orchestra.run.smithi039.stdout: caps mds = "allow" 2024-09-21T07:47:15.189 INFO:teuthology.orchestra.run.smithi039.stdout: caps mon = "allow r" 2024-09-21T07:47:15.189 INFO:teuthology.orchestra.run.smithi039.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T07:47:15.189 INFO:teuthology.orchestra.run.smithi039.stderr:updated caps for client.0 2024-09-21T07:47:15.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.189+0000 7feec2800640 1 -- 172.21.15.39:0/3200298346 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fee98078b90 msgr2=0x7fee9807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:15.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.189+0000 7feec2800640 1 --2- 172.21.15.39:0/3200298346 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fee98078b90 0x7fee9807b0a0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7feeac009b80 tx=0x7feeac005f90 comp rx=0 tx=0).stop 2024-09-21T07:47:15.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.189+0000 7feec2800640 1 -- 172.21.15.39:0/3200298346 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feebc0cd590 msgr2=0x7feebc1612a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:15.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.189+0000 7feec2800640 1 --2- 172.21.15.39:0/3200298346 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feebc0cd590 0x7feebc1612a0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7feea400c3d0 tx=0x7feea400c890 comp rx=0 tx=0).stop 2024-09-21T07:47:15.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.189+0000 7feec2800640 1 -- 172.21.15.39:0/3200298346 shutdown_connections 2024-09-21T07:47:15.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.189+0000 7feec2800640 1 --2- 172.21.15.39:0/3200298346 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fee98078b90 0x7fee9807b0a0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:15.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.189+0000 7feec2800640 1 --2- 172.21.15.39:0/3200298346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feebc0d0130 0x7feebc1f91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:15.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.189+0000 7feec2800640 1 --2- 172.21.15.39:0/3200298346 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feebc0cd590 0x7feebc1612a0 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:15.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.189+0000 7feec2800640 1 --2- 172.21.15.39:0/3200298346 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feebc072220 0x7feebc0cc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:15.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.189+0000 7feec2800640 1 -- 172.21.15.39:0/3200298346 >> 172.21.15.39:0/3200298346 conn(0x7feebc0cefe0 msgr2=0x7feebc161fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:15.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.190+0000 7feec2800640 1 -- 172.21.15.39:0/3200298346 shutdown_connections 2024-09-21T07:47:15.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.190+0000 7feec2800640 1 -- 172.21.15.39:0/3200298346 wait complete. 2024-09-21T07:47:15.212 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:47:15.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.435+0000 7fdef1200640 1 -- 172.21.15.39:0/915488432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdeec073dd0 msgr2=0x7fdeec15d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:15.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.435+0000 7fdef1200640 1 --2- 172.21.15.39:0/915488432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdeec073dd0 0x7fdeec15d5b0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fdedc009830 tx=0x7fdedc02a480 comp rx=0 tx=0).stop 2024-09-21T07:47:15.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.435+0000 7fdef1200640 1 -- 172.21.15.39:0/915488432 shutdown_connections 2024-09-21T07:47:15.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.435+0000 7fdef1200640 1 --2- 172.21.15.39:0/915488432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdeec073dd0 0x7fdeec15d5b0 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:15.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.435+0000 7fdef1200640 1 -- 172.21.15.39:0/915488432 >> 172.21.15.39:0/915488432 conn(0x7fdeec06cfc0 msgr2=0x7fdeec06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:15.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.435+0000 7fdef1200640 1 -- 172.21.15.39:0/915488432 shutdown_connections 2024-09-21T07:47:15.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.435+0000 7fdef1200640 1 -- 172.21.15.39:0/915488432 wait complete. 2024-09-21T07:47:15.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.437+0000 7fdef1200640 1 Processor -- start 2024-09-21T07:47:15.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.437+0000 7fdef1200640 1 -- start start 2024-09-21T07:47:15.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.439+0000 7fdef1200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdeec073dd0 0x7fdeec137840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:15.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.439+0000 7fdef1200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdeec0d0630 0x7fdeec137d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:15.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.439+0000 7fdef1200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeec15daf0 0x7fdeec136980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:15.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.439+0000 7fdef1200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdeec167d20 con 0x7fdeec073dd0 2024-09-21T07:47:15.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.439+0000 7fdef1200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdeec167ba0 con 0x7fdeec15daf0 2024-09-21T07:47:15.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.439+0000 7fdef1200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fdeec167ea0 con 0x7fdeec0d0630 2024-09-21T07:47:15.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.439+0000 7fdee9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdeec073dd0 0x7fdeec137840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:15.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.439+0000 7fdee9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdeec073dd0 0x7fdeec137840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48374/0 (socket says 172.21.15.39:48374) 2024-09-21T07:47:15.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.439+0000 7fdee9e00640 1 -- 172.21.15.39:0/249054681 learned_addr learned my addr 172.21.15.39:0/249054681 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:15.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.439+0000 7fdee1400640 1 --2- 172.21.15.39:0/249054681 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdeec0d0630 0x7fdeec137d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:15.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.439+0000 7fdeea800640 1 --2- 172.21.15.39:0/249054681 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeec15daf0 0x7fdeec136980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:15.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.440+0000 7fdee9e00640 1 -- 172.21.15.39:0/249054681 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdeec0d0630 msgr2=0x7fdeec137d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:15.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.440+0000 7fdee9e00640 1 --2- 172.21.15.39:0/249054681 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdeec0d0630 0x7fdeec137d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:15.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.440+0000 7fdee9e00640 1 -- 172.21.15.39:0/249054681 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeec15daf0 msgr2=0x7fdeec136980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:15.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.440+0000 7fdee9e00640 1 --2- 172.21.15.39:0/249054681 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeec15daf0 0x7fdeec136980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:15.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.440+0000 7fdee9e00640 1 -- 172.21.15.39:0/249054681 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeec14c160 con 0x7fdeec073dd0 2024-09-21T07:47:15.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.440+0000 7fdee1400640 1 --2- 172.21.15.39:0/249054681 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdeec0d0630 0x7fdeec137d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:15.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.440+0000 7fdeea800640 1 --2- 172.21.15.39:0/249054681 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeec15daf0 0x7fdeec136980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:15.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.440+0000 7fdee9e00640 1 --2- 172.21.15.39:0/249054681 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdeec073dd0 0x7fdeec137840 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fdedc009950 tx=0x7fdedc033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:15.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.440+0000 7fdee2a00640 1 -- 172.21.15.39:0/249054681 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdedc031d60 con 0x7fdeec073dd0 2024-09-21T07:47:15.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.440+0000 7fdee2a00640 1 -- 172.21.15.39:0/249054681 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdedc033e10 con 0x7fdeec073dd0 2024-09-21T07:47:15.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.440+0000 7fdef1200640 1 -- 172.21.15.39:0/249054681 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeec1358e0 con 0x7fdeec073dd0 2024-09-21T07:47:15.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.440+0000 7fdee2a00640 1 -- 172.21.15.39:0/249054681 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdedc005260 con 0x7fdeec073dd0 2024-09-21T07:47:15.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.441+0000 7fdef1200640 1 -- 172.21.15.39:0/249054681 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeec135ea0 con 0x7fdeec073dd0 2024-09-21T07:47:15.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.441+0000 7fdef1200640 1 -- 172.21.15.39:0/249054681 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdeec14f160 con 0x7fdeec073dd0 2024-09-21T07:47:15.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.443+0000 7fdee2a00640 1 -- 172.21.15.39:0/249054681 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdedc005400 con 0x7fdeec073dd0 2024-09-21T07:47:15.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.444+0000 7fdee2a00640 1 --2- 172.21.15.39:0/249054681 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdec8078bc0 0x7fdec807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:15.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.444+0000 7fdee2a00640 1 -- 172.21.15.39:0/249054681 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 6932+0+0 (secure 0 0 0) 0x7fdedc0bc5e0 con 0x7fdeec073dd0 2024-09-21T07:47:15.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.444+0000 7fdee1400640 1 --2- 172.21.15.39:0/249054681 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdec8078bc0 0x7fdec807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:15.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.445+0000 7fdee1400640 1 --2- 172.21.15.39:0/249054681 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdec8078bc0 0x7fdec807b0d0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fdeec14c630 tx=0x7fded4009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:15.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.448+0000 7fdee2a00640 1 -- 172.21.15.39:0/249054681 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdeec14c160 con 0x7fdeec073dd0 2024-09-21T07:47:15.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.686+0000 7fdef1200640 1 -- 172.21.15.39:0/249054681 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdeec15d5b0 con 0x7fdeec073dd0 2024-09-21T07:47:15.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.686+0000 7fdee2a00640 1 -- 172.21.15.39:0/249054681 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 199 v199) ==== 95+0+5230 (secure 0 0 0) 0x7fdedc03f030 con 0x7fdeec073dd0 2024-09-21T07:47:15.688 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:47:15.688 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":199,"btime":"2024-09-21T07:47:12:642015+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7715,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3057062733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3057062733},{"type":"v1","addr":"172.21.15.148:6836","nonce":3057062733}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":194},{"gid":8689,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/8652512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":8652512},{"type":"v1","addr":"172.21.15.148:6837","nonce":8652512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":194},{"gid":9483,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/3199732683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3199732683},{"type":"v1","addr":"172.21.15.39:6833","nonce":3199732683}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":193}],"filesystems":[{"mdsmap":{"epoch":199,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:47:09.140224+0000","modified":"2024-09-21T07:47:12.153995+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9492},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9492":{"gid":9492,"name":"c","rank":0,"incarnation":195,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/3348864735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3348864735},{"type":"v1","addr":"172.21.15.39:6835","nonce":3348864735}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9492,"qdb_cluster":[9492]},"id":10}]} 2024-09-21T07:47:15.689 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 199 2024-09-21T07:47:15.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.689+0000 7fdef1200640 1 -- 172.21.15.39:0/249054681 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdec8078bc0 msgr2=0x7fdec807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:15.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.689+0000 7fdef1200640 1 --2- 172.21.15.39:0/249054681 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdec8078bc0 0x7fdec807b0d0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fdeec14c630 tx=0x7fded4009040 comp rx=0 tx=0).stop 2024-09-21T07:47:15.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.689+0000 7fdef1200640 1 -- 172.21.15.39:0/249054681 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdeec073dd0 msgr2=0x7fdeec137840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:15.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.689+0000 7fdef1200640 1 --2- 172.21.15.39:0/249054681 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdeec073dd0 0x7fdeec137840 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fdedc009950 tx=0x7fdedc033c20 comp rx=0 tx=0).stop 2024-09-21T07:47:15.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.689+0000 7fdef1200640 1 -- 172.21.15.39:0/249054681 shutdown_connections 2024-09-21T07:47:15.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.689+0000 7fdef1200640 1 --2- 172.21.15.39:0/249054681 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdec8078bc0 0x7fdec807b0d0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:15.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.689+0000 7fdef1200640 1 --2- 172.21.15.39:0/249054681 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdeec15daf0 0x7fdeec136980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:15.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.689+0000 7fdef1200640 1 --2- 172.21.15.39:0/249054681 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdeec0d0630 0x7fdeec137d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:15.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.689+0000 7fdef1200640 1 --2- 172.21.15.39:0/249054681 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdeec073dd0 0x7fdeec137840 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:15.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.689+0000 7fdef1200640 1 -- 172.21.15.39:0/249054681 >> 172.21.15.39:0/249054681 conn(0x7fdeec06cfc0 msgr2=0x7fdeec0b2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:15.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.689+0000 7fdef1200640 1 -- 172.21.15.39:0/249054681 shutdown_connections 2024-09-21T07:47:15.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:15.689+0000 7fdef1200640 1 -- 172.21.15.39:0/249054681 wait complete. 2024-09-21T07:47:16.116 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 199, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T07:47:09.140224+0000', 'modified': '2024-09-21T07:47:12.153995+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9492}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9492': {'gid': 9492, 'name': 'c', 'rank': 0, 'incarnation': 195, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.39:6835/3348864735', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6834', 'nonce': 3348864735}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 3348864735}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9492, 'qdb_cluster': [9492]} 2024-09-21T07:47:16.116 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T07:47:16.116 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T07:47:16.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.346+0000 7f5fbb800640 1 -- 172.21.15.39:0/1476040138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f9c005350 msgr2=0x7f5f9c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:16.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.346+0000 7f5fbb800640 1 --2- 172.21.15.39:0/1476040138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f9c005350 0x7f5f9c00b9c0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f5fac001d90 tx=0x7f5f9c005830 comp rx=0 tx=0).stop 2024-09-21T07:47:16.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.346+0000 7f5fbb800640 1 -- 172.21.15.39:0/1476040138 shutdown_connections 2024-09-21T07:47:16.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.346+0000 7f5fbb800640 1 --2- 172.21.15.39:0/1476040138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f9c005350 0x7f5f9c00b9c0 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:16.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.346+0000 7f5fbb800640 1 -- 172.21.15.39:0/1476040138 >> 172.21.15.39:0/1476040138 conn(0x7f5fb40b2460 msgr2=0x7f5fb406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:16.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.346+0000 7f5fbb800640 1 -- 172.21.15.39:0/1476040138 shutdown_connections 2024-09-21T07:47:16.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.346+0000 7f5fbb800640 1 -- 172.21.15.39:0/1476040138 wait complete. 2024-09-21T07:47:16.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.349+0000 7f5fbb800640 1 Processor -- start 2024-09-21T07:47:16.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.349+0000 7f5fbb800640 1 -- start start 2024-09-21T07:47:16.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.350+0000 7f5fbb800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f9c005350 0x7f5fb40c4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:16.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.350+0000 7f5fbb800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5fb41547c0 0x7f5fb4155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:16.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.350+0000 7f5fbb800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fb4156010 0x7f5fb41f6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:16.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.350+0000 7f5fbb800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb4156bb0 con 0x7f5fb4156010 2024-09-21T07:47:16.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.350+0000 7f5fbb800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb40cbbc0 con 0x7f5f9c005350 2024-09-21T07:47:16.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.350+0000 7f5fbb800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f5fb4156d30 con 0x7f5fb41547c0 2024-09-21T07:47:16.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.351+0000 7f5fb9800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fb4156010 0x7f5fb41f6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:16.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.351+0000 7f5fb9800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fb4156010 0x7f5fb41f6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48392/0 (socket says 172.21.15.39:48392) 2024-09-21T07:47:16.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.351+0000 7f5fb9800640 1 -- 172.21.15.39:0/3676946157 learned_addr learned my addr 172.21.15.39:0/3676946157 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:16.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.351+0000 7f5fb3e00640 1 --2- 172.21.15.39:0/3676946157 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5fb41547c0 0x7f5fb4155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:16.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.351+0000 7f5fb8e00640 1 --2- 172.21.15.39:0/3676946157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f9c005350 0x7f5fb40c4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:16.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.351+0000 7f5fb9800640 1 -- 172.21.15.39:0/3676946157 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5fb41547c0 msgr2=0x7f5fb4155ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:16.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.351+0000 7f5fb9800640 1 --2- 172.21.15.39:0/3676946157 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5fb41547c0 0x7f5fb4155ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:16.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.351+0000 7f5fb9800640 1 -- 172.21.15.39:0/3676946157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f9c005350 msgr2=0x7f5fb40c4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:16.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.351+0000 7f5fb9800640 1 --2- 172.21.15.39:0/3676946157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f9c005350 0x7f5fb40c4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:16.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.351+0000 7f5fb9800640 1 -- 172.21.15.39:0/3676946157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb4155030 con 0x7f5fb4156010 2024-09-21T07:47:16.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.351+0000 7f5fb8e00640 1 --2- 172.21.15.39:0/3676946157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f9c005350 0x7f5fb40c4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:16.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.351+0000 7f5fb9800640 1 --2- 172.21.15.39:0/3676946157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fb4156010 0x7f5fb41f6990 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f5fac02ac80 tx=0x7f5fac006d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:16.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.352+0000 7f5fb0c00640 1 -- 172.21.15.39:0/3676946157 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fac003d80 con 0x7f5fb4156010 2024-09-21T07:47:16.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.352+0000 7f5fb0c00640 1 -- 172.21.15.39:0/3676946157 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5fac005a90 con 0x7f5fb4156010 2024-09-21T07:47:16.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.352+0000 7f5fbb800640 1 -- 172.21.15.39:0/3676946157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fb41f7660 con 0x7f5fb4156010 2024-09-21T07:47:16.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.352+0000 7f5fb0c00640 1 -- 172.21.15.39:0/3676946157 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fac005ec0 con 0x7f5fb4156010 2024-09-21T07:47:16.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.352+0000 7f5fbb800640 1 -- 172.21.15.39:0/3676946157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fb40aed70 con 0x7f5fb4156010 2024-09-21T07:47:16.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.354+0000 7f5fb0c00640 1 -- 172.21.15.39:0/3676946157 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5fac006060 con 0x7f5fb4156010 2024-09-21T07:47:16.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.355+0000 7f5fb0c00640 1 --2- 172.21.15.39:0/3676946157 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5f94078b90 0x7f5f9407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:16.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.355+0000 7f5fb0c00640 1 -- 172.21.15.39:0/3676946157 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 6932+0+0 (secure 0 0 0) 0x7f5fac0bac80 con 0x7f5fb4156010 2024-09-21T07:47:16.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.356+0000 7f5fc29f2740 1 Processor -- start 2024-09-21T07:47:16.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.356+0000 7f5fc29f2740 1 -- start start 2024-09-21T07:47:16.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.356+0000 7f5fb8e00640 1 --2- 172.21.15.39:0/3676946157 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5f94078b90 0x7f5f9407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:16.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.357+0000 7f5fb8e00640 1 --2- 172.21.15.39:0/3676946157 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5f94078b90 0x7f5f9407b0a0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8009ae0 tx=0x7f5fa8005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:16.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.357+0000 7f5fc29f2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa1869c270 0x55fa18a6c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:16.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.357+0000 7f5fc29f2740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fa186733b0 0x55fa18a6ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:16.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.357+0000 7f5fc29f2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fa1867b590 0x55fa18a3cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:16.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.357+0000 7f5fc29f2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55fa189c6430 con 0x55fa1867b590 2024-09-21T07:47:16.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.357+0000 7f5fc29f2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55fa189b65d0 con 0x55fa1869c270 2024-09-21T07:47:16.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.357+0000 7f5fc29f2740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55fa189a5a00 con 0x55fa186733b0 2024-09-21T07:47:16.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.357+0000 7f5fb9800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fa1867b590 0x55fa18a3cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:16.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.357+0000 7f5fb9800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fa1867b590 0x55fa18a3cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48396/0 (socket says 172.21.15.39:48396) 2024-09-21T07:47:16.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.357+0000 7f5fb9800640 1 -- 172.21.15.39:0/3966168101 learned_addr learned my addr 172.21.15.39:0/3966168101 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:16.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.358+0000 7f5fb3e00640 1 --2- 172.21.15.39:0/3966168101 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fa186733b0 0x55fa18a6ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:16.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.358+0000 7f5fb8e00640 1 --2- 172.21.15.39:0/3966168101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa1869c270 0x55fa18a6c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:16.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.358+0000 7f5fb9800640 1 -- 172.21.15.39:0/3966168101 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fa186733b0 msgr2=0x55fa18a6ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:47:16.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.358+0000 7f5fb9800640 1 --2- 172.21.15.39:0/3966168101 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fa186733b0 0x55fa18a6ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:16.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.358+0000 7f5fb9800640 1 -- 172.21.15.39:0/3966168101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa1869c270 msgr2=0x55fa18a6c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:47:16.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.358+0000 7f5fb9800640 1 --2- 172.21.15.39:0/3966168101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa1869c270 0x55fa18a6c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:16.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.358+0000 7f5fb9800640 1 -- 172.21.15.39:0/3966168101 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa1896e520 con 0x55fa1867b590 2024-09-21T07:47:16.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.358+0000 7f5fb3e00640 1 --2- 172.21.15.39:0/3966168101 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fa186733b0 0x55fa18a6ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:16.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.358+0000 7f5fb8e00640 1 --2- 172.21.15.39:0/3966168101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa1869c270 0x55fa18a6c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:16.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.358+0000 7f5fb9800640 1 --2- 172.21.15.39:0/3966168101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fa1867b590 0x55fa18a3cf60 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f5fac0b8b80 tx=0x7f5fac081e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d0b7b33255f955 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:16.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.358+0000 7f5fa5600640 1 -- 172.21.15.39:0/3966168101 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fac043040 con 0x55fa1867b590 2024-09-21T07:47:16.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.358+0000 7f5fa5600640 1 -- 172.21.15.39:0/3966168101 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5fac086eb0 con 0x55fa1867b590 2024-09-21T07:47:16.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.358+0000 7f5fa5600640 1 -- 172.21.15.39:0/3966168101 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fac081920 con 0x55fa1867b590 2024-09-21T07:47:16.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.359+0000 7f5fc29f2740 1 -- 172.21.15.39:0/3966168101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fa1867b590 msgr2=0x55fa18a3cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:16.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.359+0000 7f5fc29f2740 1 --2- 172.21.15.39:0/3966168101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fa1867b590 0x55fa18a3cf60 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f5fac0b8b80 tx=0x7f5fac081e90 comp rx=0 tx=0).stop 2024-09-21T07:47:16.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.359+0000 7f5fc29f2740 1 -- 172.21.15.39:0/3966168101 shutdown_connections 2024-09-21T07:47:16.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.359+0000 7f5fc29f2740 1 --2- 172.21.15.39:0/3966168101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fa1867b590 0x55fa18a3cf60 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:16.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.359+0000 7f5fc29f2740 1 --2- 172.21.15.39:0/3966168101 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fa186733b0 0x55fa18a6ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:16.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.359+0000 7f5fc29f2740 1 --2- 172.21.15.39:0/3966168101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa1869c270 0x55fa18a6c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:16.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.359+0000 7f5fc29f2740 1 -- 172.21.15.39:0/3966168101 >> 172.21.15.39:0/3966168101 conn(0x55fa186903c0 msgr2=0x55fa18a42970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:16.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.359+0000 7f5fc29f2740 1 -- 172.21.15.39:0/3966168101 shutdown_connections 2024-09-21T07:47:16.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.359+0000 7f5fc29f2740 1 -- 172.21.15.39:0/3966168101 wait complete. 2024-09-21T07:47:16.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.360+0000 7f5fc29f2740 1 -- start start 2024-09-21T07:47:16.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.360+0000 7f5fc29f2740 1 Processor -- start 2024-09-21T07:47:16.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.362+0000 7f5fc29f2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa186733b0 0x55fa18a97600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:16.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.362+0000 7f5fc29f2740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fa1867b590 0x55fa18a62ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:16.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.362+0000 7f5fc29f2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fa1869c270 0x55fa18a67c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:16.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.362+0000 7f5fc29f2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55fa1894c330 con 0x55fa1869c270 2024-09-21T07:47:16.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.362+0000 7f5fc29f2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55fa189a9810 con 0x55fa186733b0 2024-09-21T07:47:16.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.362+0000 7f5fc29f2740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55fa18716620 con 0x55fa1867b590 2024-09-21T07:47:16.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.362+0000 7f5fc29f2740 10 client.0 _ll_register_callbacks cb 0x55fa18951420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:47:16.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.362+0000 7f5fb9800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fa1869c270 0x55fa18a67c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:16.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.362+0000 7f5fb9800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fa1869c270 0x55fa18a67c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48400/0 (socket says 172.21.15.39:48400) 2024-09-21T07:47:16.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.362+0000 7f5fb9800640 1 -- 172.21.15.39:0/1602200896 learned_addr learned my addr 172.21.15.39:0/1602200896 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:16.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.362+0000 7f5fb3e00640 1 --2- 172.21.15.39:0/1602200896 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fa1867b590 0x55fa18a62ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:16.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.362+0000 7f5fb8e00640 1 --2- 172.21.15.39:0/1602200896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa186733b0 0x55fa18a97600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:16.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.363+0000 7f5fb9800640 1 -- 172.21.15.39:0/1602200896 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fa1867b590 msgr2=0x55fa18a62ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:47:16.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.363+0000 7f5fb9800640 1 --2- 172.21.15.39:0/1602200896 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fa1867b590 0x55fa18a62ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:16.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.363+0000 7f5fb9800640 1 -- 172.21.15.39:0/1602200896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa186733b0 msgr2=0x55fa18a97600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:47:16.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.363+0000 7f5fb9800640 1 --2- 172.21.15.39:0/1602200896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa186733b0 0x55fa18a97600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:16.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.363+0000 7f5fb9800640 1 -- 172.21.15.39:0/1602200896 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa18707850 con 0x55fa1869c270 2024-09-21T07:47:16.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.363+0000 7f5fb9800640 1 --2- 172.21.15.39:0/1602200896 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fa1869c270 0x55fa18a67c90 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f5fac0a6040 tx=0x7f5fac081910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8161ecca027c20a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:16.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.363+0000 7f5f9a000640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:47:16.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.363+0000 7f5f9a000640 1 -- 172.21.15.39:0/1602200896 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fac043070 con 0x55fa1869c270 2024-09-21T07:47:16.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.363+0000 7f5f9a000640 1 -- 172.21.15.39:0/1602200896 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(194..194 src has 1..194) ==== 6932+0+0 (secure 0 0 0) 0x7f5fac07dfe0 con 0x55fa1869c270 2024-09-21T07:47:16.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.363+0000 7f5fc29f2740 1 -- 172.21.15.39:0/1602200896 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fac086eb0 con 0x55fa1869c270 2024-09-21T07:47:16.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.363+0000 7f5f9a000640 1 -- 172.21.15.39:0/1602200896 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=199) ==== 24+0+0 (secure 0 0 0) 0x7f5fac08bdf0 con 0x55fa1869c270 2024-09-21T07:47:16.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.364+0000 7f5fc29f2740 10 client.9600 fetch_fsmap learned FSMap version 199 2024-09-21T07:47:16.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.364+0000 7f5fc29f2740 1 -- 172.21.15.39:0/1602200896 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=199,monmap=2+}) -- 0x55fa1877e6b0 con 0x55fa1869c270 2024-09-21T07:47:16.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.364+0000 7f5f9a000640 1 -- 172.21.15.39:0/1602200896 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5fac086eb0 con 0x55fa1869c270 2024-09-21T07:47:16.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.364+0000 7f5f9a000640 1 -- 172.21.15.39:0/1602200896 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 199) ==== 3502+0+0 (secure 0 0 0) 0x7f5fac046060 con 0x55fa1869c270 2024-09-21T07:47:16.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.364+0000 7f5fc29f2740 10 client.9600 fetch_fsmap finished waiting for FSMap version 199 2024-09-21T07:47:16.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.364+0000 7f5fc29f2740 10 client.9600 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:47:16.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.364+0000 7f5fc29f2740 20 client.9600 populate_metadata read hostname 'smithi039' 2024-09-21T07:47:16.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.364+0000 7f5fc29f2740 1 --2- 172.21.15.39:0/1602200896 >> [v2:172.21.15.39:6834/3348864735,v1:172.21.15.39:6835/3348864735] conn(0x55fa1865ae40 0x55fa18a9a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:16.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.364+0000 7f5fc29f2740 4 client.9600 mds_command: new command op to 9492 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:47:16.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.364+0000 7f5fc29f2740 1 -- 172.21.15.39:0/1602200896 --> [v2:172.21.15.39:6834/3348864735,v1:172.21.15.39:6835/3348864735] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa189b8520 con 0x55fa1865ae40 2024-09-21T07:47:16.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.364+0000 7f5fb8e00640 1 --2- 172.21.15.39:0/1602200896 >> [v2:172.21.15.39:6834/3348864735,v1:172.21.15.39:6835/3348864735] conn(0x55fa1865ae40 0x55fa18a9a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:16.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.365+0000 7f5fb8e00640 1 --2- 172.21.15.39:0/1602200896 >> [v2:172.21.15.39:6834/3348864735,v1:172.21.15.39:6835/3348864735] conn(0x55fa1865ae40 0x55fa18a9a170 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1819f2933ea1f383 server_cookie=7b5f0204c780eb3d in_seq=0 out_seq=0 2024-09-21T07:47:16.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.365+0000 7f5f9a000640 10 client.9600 ms_handle_connect on v2:172.21.15.39:6834/3348864735 2024-09-21T07:47:16.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.366+0000 7f5f9a000640 1 -- 172.21.15.39:0/1602200896 <== mds.0 v2:172.21.15.39:6834/3348864735 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fa189b8520 con 0x55fa1865ae40 2024-09-21T07:47:16.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.366+0000 7f5f9a000640 10 client.9600 handle_command_reply: tid=0 2024-09-21T07:47:16.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.366+0000 7f5fc29f2740 1 client.9600 shutdown 2024-09-21T07:47:16.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.366+0000 7f5fc29f2740 1 -- 172.21.15.39:0/1602200896 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fa1869c270 msgr2=0x55fa18a67c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:16.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.366+0000 7f5fc29f2740 1 --2- 172.21.15.39:0/1602200896 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fa1869c270 0x55fa18a67c90 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f5fac0a6040 tx=0x7f5fac081910 comp rx=0 tx=0).stop 2024-09-21T07:47:16.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.366+0000 7f5fc29f2740 1 -- 172.21.15.39:0/1602200896 shutdown_connections 2024-09-21T07:47:16.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.366+0000 7f5fc29f2740 1 --2- 172.21.15.39:0/1602200896 >> [v2:172.21.15.39:6834/3348864735,v1:172.21.15.39:6835/3348864735] conn(0x55fa1865ae40 0x55fa18a9a170 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:16.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.366+0000 7f5fc29f2740 1 --2- 172.21.15.39:0/1602200896 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fa1869c270 0x55fa18a67c90 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:16.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.366+0000 7f5fc29f2740 1 --2- 172.21.15.39:0/1602200896 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fa1867b590 0x55fa18a62ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:16.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.366+0000 7f5fc29f2740 1 --2- 172.21.15.39:0/1602200896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa186733b0 0x55fa18a97600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:16.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.366+0000 7f5f9a000640 0 client.9600 ms_handle_reset on v2:172.21.15.39:6834/3348864735 2024-09-21T07:47:16.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.366+0000 7f5fc29f2740 1 -- 172.21.15.39:0/1602200896 >> 172.21.15.39:0/1602200896 conn(0x55fa186903c0 msgr2=0x55fa18a3bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:16.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.366+0000 7f5fc29f2740 1 -- 172.21.15.39:0/1602200896 shutdown_connections 2024-09-21T07:47:16.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.366+0000 7f5fc29f2740 1 -- 172.21.15.39:0/1602200896 wait complete. 2024-09-21T07:47:16.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.367+0000 7f5fc29f2740 20 client.9600 trim_cache size 0 max 16384 2024-09-21T07:47:16.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.386+0000 7f5fc29f2740 1 Processor -- start 2024-09-21T07:47:16.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.386+0000 7f5fc29f2740 1 -- start start 2024-09-21T07:47:16.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.387+0000 7f5fc29f2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa186733b0 0x55fa18a65b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:16.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.388+0000 7f5fc29f2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fa1867b590 0x55fa18a3bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:16.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.388+0000 7f5fc29f2740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fa1869c270 0x55fa18a82580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:16.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.388+0000 7f5fc29f2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb41ffb70 con 0x55fa1867b590 2024-09-21T07:47:16.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.388+0000 7f5fc29f2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb40cfab0 con 0x55fa186733b0 2024-09-21T07:47:16.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.388+0000 7f5fc29f2740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f5fb41ff630 con 0x55fa1869c270 2024-09-21T07:47:16.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.388+0000 7f5fb3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fa1867b590 0x55fa18a3bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:16.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.388+0000 7f5fb3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fa1867b590 0x55fa18a3bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48414/0 (socket says 172.21.15.39:48414) 2024-09-21T07:47:16.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.388+0000 7f5fb3e00640 1 -- 172.21.15.39:0/3719528187 learned_addr learned my addr 172.21.15.39:0/3719528187 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:16.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.388+0000 7f5fb8e00640 1 --2- 172.21.15.39:0/3719528187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa186733b0 0x55fa18a65b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:16.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.388+0000 7f5fb9800640 1 --2- 172.21.15.39:0/3719528187 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fa1869c270 0x55fa18a82580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:16.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.388+0000 7f5fb3e00640 1 -- 172.21.15.39:0/3719528187 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fa1869c270 msgr2=0x55fa18a82580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:47:16.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.388+0000 7f5fb3e00640 1 --2- 172.21.15.39:0/3719528187 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fa1869c270 0x55fa18a82580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:16.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.388+0000 7f5fb3e00640 1 -- 172.21.15.39:0/3719528187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa186733b0 msgr2=0x55fa18a65b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:47:16.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.388+0000 7f5fb3e00640 1 --2- 172.21.15.39:0/3719528187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa186733b0 0x55fa18a65b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:16.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.388+0000 7f5fb3e00640 1 -- 172.21.15.39:0/3719528187 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa18a87280 con 0x55fa1867b590 2024-09-21T07:47:16.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.389+0000 7f5fb3e00640 1 --2- 172.21.15.39:0/3719528187 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fa1867b590 0x55fa18a3bc60 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0005ad0 tx=0x7f5fa0001e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3443fae3e537904 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:16.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.389+0000 7f5fa4c00640 1 -- 172.21.15.39:0/3719528187 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0012be0 con 0x55fa1867b590 2024-09-21T07:47:16.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.389+0000 7f5fa4c00640 1 -- 172.21.15.39:0/3719528187 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5fa0012d80 con 0x55fa1867b590 2024-09-21T07:47:16.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.389+0000 7f5fa4c00640 1 -- 172.21.15.39:0/3719528187 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa001c640 con 0x55fa1867b590 2024-09-21T07:47:16.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.389+0000 7f5fc29f2740 1 -- 172.21.15.39:0/3719528187 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fa1867b590 msgr2=0x55fa18a3bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:16.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.390+0000 7f5fc29f2740 1 --2- 172.21.15.39:0/3719528187 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fa1867b590 0x55fa18a3bc60 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0005ad0 tx=0x7f5fa0001e20 comp rx=0 tx=0).stop 2024-09-21T07:47:16.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.390+0000 7f5fc29f2740 1 -- 172.21.15.39:0/3719528187 shutdown_connections 2024-09-21T07:47:16.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.390+0000 7f5fc29f2740 1 --2- 172.21.15.39:0/3719528187 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fa1869c270 0x55fa18a82580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:16.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.390+0000 7f5fc29f2740 1 --2- 172.21.15.39:0/3719528187 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fa1867b590 0x55fa18a3bc60 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:16.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.390+0000 7f5fc29f2740 1 --2- 172.21.15.39:0/3719528187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa186733b0 0x55fa18a65b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:16.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.390+0000 7f5fc29f2740 1 -- 172.21.15.39:0/3719528187 >> 172.21.15.39:0/3719528187 conn(0x55fa186903c0 msgr2=0x55fa18a88820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:16.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.390+0000 7f5fc29f2740 1 -- 172.21.15.39:0/3719528187 shutdown_connections 2024-09-21T07:47:16.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.390+0000 7f5fc29f2740 1 -- 172.21.15.39:0/3719528187 wait complete. 2024-09-21T07:47:16.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.391+0000 7f5fc29f2740 1 -- start start 2024-09-21T07:47:16.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.391+0000 7f5fc29f2740 1 Processor -- start 2024-09-21T07:47:16.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.392+0000 7f5fc29f2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa186733b0 0x55fa18a9f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:16.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.392+0000 7f5fc29f2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fa1867b590 0x55fa18b0b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:16.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.392+0000 7f5fc29f2740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fa1869c270 0x55fa18a97190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:16.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.392+0000 7f5fc29f2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55fa1894c330 con 0x55fa1867b590 2024-09-21T07:47:16.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.392+0000 7f5fc29f2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55fa189a9810 con 0x55fa186733b0 2024-09-21T07:47:16.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.392+0000 7f5fc29f2740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55fa18a66d30 con 0x55fa1869c270 2024-09-21T07:47:16.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.392+0000 7f5fc29f2740 10 client.0 _ll_register_callbacks cb 0x7f5fac0b8b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:47:16.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.393+0000 7f5fb3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fa1867b590 0x55fa18b0b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:16.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.393+0000 7f5fb3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fa1867b590 0x55fa18b0b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48416/0 (socket says 172.21.15.39:48416) 2024-09-21T07:47:16.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.393+0000 7f5fb8e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa186733b0 0x55fa18a9f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:16.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.393+0000 7f5fb3e00640 1 -- 172.21.15.39:0/582993509 learned_addr learned my addr 172.21.15.39:0/582993509 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:16.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.393+0000 7f5fb9800640 1 --2- 172.21.15.39:0/582993509 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fa1869c270 0x55fa18a97190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:16.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.393+0000 7f5fb3e00640 1 -- 172.21.15.39:0/582993509 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fa1869c270 msgr2=0x55fa18a97190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:47:16.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.393+0000 7f5fb3e00640 1 --2- 172.21.15.39:0/582993509 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fa1869c270 0x55fa18a97190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:16.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.393+0000 7f5fb3e00640 1 -- 172.21.15.39:0/582993509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa186733b0 msgr2=0x55fa18a9f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:47:16.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.393+0000 7f5fb3e00640 1 --2- 172.21.15.39:0/582993509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa186733b0 0x55fa18a9f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:16.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.393+0000 7f5fb3e00640 1 -- 172.21.15.39:0/582993509 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa18a43650 con 0x55fa1867b590 2024-09-21T07:47:16.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.393+0000 7f5fb3e00640 1 --2- 172.21.15.39:0/582993509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fa1867b590 0x55fa18b0b330 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f5fa00128d0 tx=0x7f5fa001d6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f539a7cf63a5992 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:16.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.393+0000 7f5f9a000640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:47:16.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.394+0000 7f5f9a000640 1 -- 172.21.15.39:0/582993509 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa00057a0 con 0x55fa1867b590 2024-09-21T07:47:16.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.394+0000 7f5f9a000640 1 -- 172.21.15.39:0/582993509 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(194..194 src has 1..194) ==== 6932+0+0 (secure 0 0 0) 0x7f5fa0021070 con 0x55fa1867b590 2024-09-21T07:47:16.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.394+0000 7f5fc29f2740 1 -- 172.21.15.39:0/582993509 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fa0012d80 con 0x55fa1867b590 2024-09-21T07:47:16.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.394+0000 7f5f9a000640 1 -- 172.21.15.39:0/582993509 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=199) ==== 24+0+0 (secure 0 0 0) 0x7f5fa0032050 con 0x55fa1867b590 2024-09-21T07:47:16.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.394+0000 7f5fc29f2740 10 client.9606 fetch_fsmap learned FSMap version 199 2024-09-21T07:47:16.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.394+0000 7f5fc29f2740 1 -- 172.21.15.39:0/582993509 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=199,monmap=2+}) -- 0x55fa18b0b870 con 0x55fa1867b590 2024-09-21T07:47:16.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.394+0000 7f5f9a000640 1 -- 172.21.15.39:0/582993509 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5fa0012d80 con 0x55fa1867b590 2024-09-21T07:47:16.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.394+0000 7f5f9a000640 1 -- 172.21.15.39:0/582993509 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 199) ==== 3502+0+0 (secure 0 0 0) 0x7f5fa0035020 con 0x55fa1867b590 2024-09-21T07:47:16.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.394+0000 7f5fc29f2740 10 client.9606 fetch_fsmap finished waiting for FSMap version 199 2024-09-21T07:47:16.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.394+0000 7f5fc29f2740 10 client.9606 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:47:16.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.394+0000 7f5fc29f2740 20 client.9606 populate_metadata read hostname 'smithi039' 2024-09-21T07:47:16.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.394+0000 7f5fc29f2740 1 --2- 172.21.15.39:0/582993509 >> [v2:172.21.15.39:6834/3348864735,v1:172.21.15.39:6835/3348864735] conn(0x55fa18a82c40 0x55fa18a86040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:16.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.394+0000 7f5fc29f2740 4 client.9606 mds_command: new command op to 9492 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:47:16.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.394+0000 7f5fc29f2740 1 -- 172.21.15.39:0/582993509 --> [v2:172.21.15.39:6834/3348864735,v1:172.21.15.39:6835/3348864735] -- command(tid 0: {"prefix": "status"}) -- 0x55fa189b3fd0 con 0x55fa18a82c40 2024-09-21T07:47:16.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.395+0000 7f5fb8e00640 1 --2- 172.21.15.39:0/582993509 >> [v2:172.21.15.39:6834/3348864735,v1:172.21.15.39:6835/3348864735] conn(0x55fa18a82c40 0x55fa18a86040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:16.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.395+0000 7f5fb8e00640 1 --2- 172.21.15.39:0/582993509 >> [v2:172.21.15.39:6834/3348864735,v1:172.21.15.39:6835/3348864735] conn(0x55fa18a82c40 0x55fa18a86040 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=519683171a9803ef server_cookie=897a8c4af339473e in_seq=0 out_seq=0 2024-09-21T07:47:16.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.395+0000 7f5f9a000640 10 client.9606 ms_handle_connect on v2:172.21.15.39:6834/3348864735 2024-09-21T07:47:16.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.395+0000 7f5f9a000640 1 -- 172.21.15.39:0/582993509 <== mds.0 v2:172.21.15.39:6834/3348864735 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x55fa189b3fd0 con 0x55fa18a82c40 2024-09-21T07:47:16.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.395+0000 7f5f9a000640 10 client.9606 handle_command_reply: tid=0 2024-09-21T07:47:16.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.395+0000 7f5fc29f2740 1 client.9606 shutdown 2024-09-21T07:47:16.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.396+0000 7f5fc29f2740 1 -- 172.21.15.39:0/582993509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fa1867b590 msgr2=0x55fa18b0b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:16.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.396+0000 7f5fc29f2740 1 --2- 172.21.15.39:0/582993509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fa1867b590 0x55fa18b0b330 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f5fa00128d0 tx=0x7f5fa001d6e0 comp rx=0 tx=0).stop 2024-09-21T07:47:16.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.396+0000 7f5fc29f2740 1 -- 172.21.15.39:0/582993509 shutdown_connections 2024-09-21T07:47:16.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.396+0000 7f5fc29f2740 1 --2- 172.21.15.39:0/582993509 >> [v2:172.21.15.39:6834/3348864735,v1:172.21.15.39:6835/3348864735] conn(0x55fa18a82c40 0x55fa18a86040 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:16.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.396+0000 7f5fc29f2740 1 --2- 172.21.15.39:0/582993509 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55fa1869c270 0x55fa18a97190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:16.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.396+0000 7f5fc29f2740 1 --2- 172.21.15.39:0/582993509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fa1867b590 0x55fa18b0b330 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:16.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.396+0000 7f5fc29f2740 1 --2- 172.21.15.39:0/582993509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fa186733b0 0x55fa18a9f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:16.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.396+0000 7f5f9a000640 0 client.9606 ms_handle_reset on v2:172.21.15.39:6834/3348864735 2024-09-21T07:47:16.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.396+0000 7f5fc29f2740 1 -- 172.21.15.39:0/582993509 >> 172.21.15.39:0/582993509 conn(0x55fa186903c0 msgr2=0x55fa18a62aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:16.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.396+0000 7f5fc29f2740 1 -- 172.21.15.39:0/582993509 shutdown_connections 2024-09-21T07:47:16.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.396+0000 7f5fc29f2740 1 -- 172.21.15.39:0/582993509 wait complete. 2024-09-21T07:47:16.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.396+0000 7f5fc29f2740 20 client.9606 trim_cache size 0 max 16384 2024-09-21T07:47:16.401 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:47:16.401 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:47:16.401 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-09-21T07:47:16.401 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 9492, 2024-09-21T07:47:16.402 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:47:16.402 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:47:16.402 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:47:16.402 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 6.7849400729999996, 2024-09-21T07:47:16.402 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 199, 2024-09-21T07:47:16.402 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 194, 2024-09-21T07:47:16.402 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 194, 2024-09-21T07:47:16.402 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 11.747475666 2024-09-21T07:47:16.402 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:47:16.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.397+0000 7f5fbb800640 1 -- 172.21.15.39:0/3676946157 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5f94078b90 msgr2=0x7f5f9407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:16.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.397+0000 7f5fbb800640 1 --2- 172.21.15.39:0/3676946157 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5f94078b90 0x7f5f9407b0a0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8009ae0 tx=0x7f5fa8005f90 comp rx=0 tx=0).stop 2024-09-21T07:47:16.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.397+0000 7f5fbb800640 1 -- 172.21.15.39:0/3676946157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fb4156010 msgr2=0x7f5fb41f6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:16.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.397+0000 7f5fbb800640 1 --2- 172.21.15.39:0/3676946157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fb4156010 0x7f5fb41f6990 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f5fac02ac80 tx=0x7f5fac006d20 comp rx=0 tx=0).stop 2024-09-21T07:47:16.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.397+0000 7f5fbb800640 1 -- 172.21.15.39:0/3676946157 shutdown_connections 2024-09-21T07:47:16.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.397+0000 7f5fbb800640 1 --2- 172.21.15.39:0/3676946157 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5f94078b90 0x7f5f9407b0a0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:16.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.397+0000 7f5fbb800640 1 --2- 172.21.15.39:0/3676946157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fb4156010 0x7f5fb41f6990 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:16.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.397+0000 7f5fbb800640 1 --2- 172.21.15.39:0/3676946157 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5fb41547c0 0x7f5fb4155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:16.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.397+0000 7f5fbb800640 1 --2- 172.21.15.39:0/3676946157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f9c005350 0x7f5fb40c4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:16.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.397+0000 7f5fbb800640 1 -- 172.21.15.39:0/3676946157 >> 172.21.15.39:0/3676946157 conn(0x7f5fb40b2460 msgr2=0x7f5fb415ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:16.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.397+0000 7f5fbb800640 1 -- 172.21.15.39:0/3676946157 shutdown_connections 2024-09-21T07:47:16.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.397+0000 7f5fbb800640 1 -- 172.21.15.39:0/3676946157 wait complete. 2024-09-21T07:47:16.423 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:47:16.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.652+0000 7f8696400640 1 -- 172.21.15.39:0/3766012870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8678005150 msgr2=0x7f867800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:16.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.652+0000 7f8696400640 1 --2- 172.21.15.39:0/3766012870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8678005150 0x7f867800b9c0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f869008e1d0 tx=0x7f8678005830 comp rx=0 tx=0).stop 2024-09-21T07:47:16.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.652+0000 7f8695a00640 1 -- 172.21.15.39:0/3766012870 reap_dead start 2024-09-21T07:47:16.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.652+0000 7f8696400640 1 -- 172.21.15.39:0/3766012870 shutdown_connections 2024-09-21T07:47:16.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.652+0000 7f8696400640 1 -- 172.21.15.39:0/3766012870 >> 172.21.15.39:0/3766012870 conn(0x7f869009d7b0 msgr2=0x7f86900d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:16.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.652+0000 7f8696400640 1 -- 172.21.15.39:0/3766012870 shutdown_connections 2024-09-21T07:47:16.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.652+0000 7f8696400640 1 -- 172.21.15.39:0/3766012870 wait complete. 2024-09-21T07:47:16.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.654+0000 7f8696400640 1 Processor -- start 2024-09-21T07:47:16.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.655+0000 7f8696400640 1 -- start start 2024-09-21T07:47:16.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.656+0000 7f8696400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8678005150 0x7f86900b3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:16.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.656+0000 7f8696400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f869014b160 0x7f86900cb400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:16.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.656+0000 7f8696400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86900cb940 0x7f86900b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:16.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.656+0000 7f8696400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f86900d1700 con 0x7f869014b160 2024-09-21T07:47:16.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.656+0000 7f8696400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f86900d1580 con 0x7f86900cb940 2024-09-21T07:47:16.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.656+0000 7f8696400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f86900d1ac0 con 0x7f8678005150 2024-09-21T07:47:16.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.656+0000 7f868fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f869014b160 0x7f86900cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:16.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.656+0000 7f868fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f869014b160 0x7f86900cb400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48438/0 (socket says 172.21.15.39:48438) 2024-09-21T07:47:16.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.656+0000 7f868fe00640 1 -- 172.21.15.39:0/3255115223 learned_addr learned my addr 172.21.15.39:0/3255115223 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:16.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.656+0000 7f8695000640 1 --2- 172.21.15.39:0/3255115223 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8678005150 0x7f86900b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:16.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.657+0000 7f8695a00640 1 --2- 172.21.15.39:0/3255115223 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86900cb940 0x7f86900b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:16.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.657+0000 7f868fe00640 1 -- 172.21.15.39:0/3255115223 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8678005150 msgr2=0x7f86900b3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:16.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.657+0000 7f868fe00640 1 --2- 172.21.15.39:0/3255115223 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8678005150 0x7f86900b3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:16.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.657+0000 7f868fe00640 1 -- 172.21.15.39:0/3255115223 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86900cb940 msgr2=0x7f86900b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:16.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.657+0000 7f868fe00640 1 --2- 172.21.15.39:0/3255115223 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86900cb940 0x7f86900b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:16.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.657+0000 7f868fe00640 1 -- 172.21.15.39:0/3255115223 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86900cbd50 con 0x7f869014b160 2024-09-21T07:47:16.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.657+0000 7f868fe00640 1 --2- 172.21.15.39:0/3255115223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f869014b160 0x7f86900cb400 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f867c011ae0 tx=0x7f867c009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:16.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.657+0000 7f868cc00640 1 -- 172.21.15.39:0/3255115223 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c01e070 con 0x7f869014b160 2024-09-21T07:47:16.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.657+0000 7f868cc00640 1 -- 172.21.15.39:0/3255115223 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f867c004460 con 0x7f869014b160 2024-09-21T07:47:16.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.657+0000 7f8696400640 1 -- 172.21.15.39:0/3255115223 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86901f8920 con 0x7f869014b160 2024-09-21T07:47:16.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.657+0000 7f868cc00640 1 -- 172.21.15.39:0/3255115223 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c00fc60 con 0x7f869014b160 2024-09-21T07:47:16.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.658+0000 7f8696400640 1 -- 172.21.15.39:0/3255115223 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86901f8ee0 con 0x7f869014b160 2024-09-21T07:47:16.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.659+0000 7f8696400640 1 -- 172.21.15.39:0/3255115223 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86901f95a0 con 0x7f869014b160 2024-09-21T07:47:16.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.660+0000 7f868cc00640 1 -- 172.21.15.39:0/3255115223 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f867c021070 con 0x7f869014b160 2024-09-21T07:47:16.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.662+0000 7f868cc00640 1 --2- 172.21.15.39:0/3255115223 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8670078b90 0x7f867007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:16.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.662+0000 7f868cc00640 1 -- 172.21.15.39:0/3255115223 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 6932+0+0 (secure 0 0 0) 0x7f867c09bda0 con 0x7f869014b160 2024-09-21T07:47:16.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.662+0000 7f8695000640 1 --2- 172.21.15.39:0/3255115223 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8670078b90 0x7f867007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:16.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.663+0000 7f8695000640 1 --2- 172.21.15.39:0/3255115223 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8670078b90 0x7f867007b0a0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f8684000c00 tx=0x7f8684006bc0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:16.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.670+0000 7f868cc00640 1 -- 172.21.15.39:0/3255115223 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f86901f95a0 con 0x7f869014b160 2024-09-21T07:47:16.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.909+0000 7f8696400640 1 -- 172.21.15.39:0/3255115223 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f86900d0e70 con 0x7f869014b160 2024-09-21T07:47:16.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.909+0000 7f868cc00640 1 -- 172.21.15.39:0/3255115223 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v199) ==== 53+0+73 (secure 0 0 0) 0x7f86900cbd50 con 0x7f869014b160 2024-09-21T07:47:16.911 INFO:teuthology.orchestra.run.smithi039.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:47:16.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.912+0000 7f8696400640 1 -- 172.21.15.39:0/3255115223 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8670078b90 msgr2=0x7f867007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:16.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.912+0000 7f8696400640 1 --2- 172.21.15.39:0/3255115223 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8670078b90 0x7f867007b0a0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f8684000c00 tx=0x7f8684006bc0 comp rx=0 tx=0).stop 2024-09-21T07:47:16.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.912+0000 7f8696400640 1 -- 172.21.15.39:0/3255115223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f869014b160 msgr2=0x7f86900cb400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:16.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.912+0000 7f8696400640 1 --2- 172.21.15.39:0/3255115223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f869014b160 0x7f86900cb400 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f867c011ae0 tx=0x7f867c009e40 comp rx=0 tx=0).stop 2024-09-21T07:47:16.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.912+0000 7f8696400640 1 -- 172.21.15.39:0/3255115223 shutdown_connections 2024-09-21T07:47:16.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.912+0000 7f8696400640 1 --2- 172.21.15.39:0/3255115223 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8670078b90 0x7f867007b0a0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:16.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.912+0000 7f8696400640 1 --2- 172.21.15.39:0/3255115223 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86900cb940 0x7f86900b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:16.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.912+0000 7f8696400640 1 --2- 172.21.15.39:0/3255115223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f869014b160 0x7f86900cb400 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:16.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.912+0000 7f8696400640 1 --2- 172.21.15.39:0/3255115223 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8678005150 0x7f86900b3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:16.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.912+0000 7f8696400640 1 -- 172.21.15.39:0/3255115223 >> 172.21.15.39:0/3255115223 conn(0x7f869009d7b0 msgr2=0x7f86900b5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:16.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.912+0000 7f8696400640 1 -- 172.21.15.39:0/3255115223 shutdown_connections 2024-09-21T07:47:16.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:16.912+0000 7f8696400640 1 -- 172.21.15.39:0/3255115223 wait complete. 2024-09-21T07:47:16.939 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:47:16.939 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:47:16.939 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi039.front.sepia.ceph.com 2024-09-21T07:47:16.939 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T07:47:16.939 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:47:16.939 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:47:16.939 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:47:16.940 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:47:16.940 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:47:16.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:16.940 DEBUG:teuthology.orchestra.run.smithi039:> ip addr 2024-09-21T07:47:16.976 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:47:16.976 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:47:16.976 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:47:16.976 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:47:16.976 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2024-09-21T07:47:16.976 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:47:16.976 INFO:teuthology.orchestra.run.smithi039.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:47:16.976 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:47:16.976 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f0 2024-09-21T07:47:16.977 INFO:teuthology.orchestra.run.smithi039.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:47:16.977 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c7 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:47:16.977 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f1 2024-09-21T07:47:16.977 INFO:teuthology.orchestra.run.smithi039.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:47:16.977 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:70 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:47:16.977 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f0 2024-09-21T07:47:16.977 INFO:teuthology.orchestra.run.smithi039.stdout: inet 172.21.15.39/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T07:47:16.977 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft 41882sec preferred_lft 41882sec 2024-09-21T07:47:16.977 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec4:7aff:febd:1370/64 scope link 2024-09-21T07:47:16.977 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:47:16.977 INFO:teuthology.orchestra.run.smithi039.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:47:16.977 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:71 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:47:16.977 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f1 2024-09-21T07:47:16.978 INFO:teuthology.orchestra.run.smithi039.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:47:16.978 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff 2024-09-21T07:47:16.978 INFO:teuthology.orchestra.run.smithi039.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:47:16.978 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:47:16.978 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::b079:ddff:fe0e:6333/64 scope link 2024-09-21T07:47:16.978 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:47:16.978 INFO:teuthology.orchestra.run.smithi039.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:47:16.978 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:47:16.978 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::2ca7:98ff:fe8a:d0ed/64 scope link 2024-09-21T07:47:16.978 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:47:16.978 INFO:teuthology.orchestra.run.smithi039.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:47:16.978 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether ee:10:53:01:0c:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:47:16.979 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec10:53ff:fe01:c44/64 scope link 2024-09-21T07:47:16.979 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:47:16.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:16.979 DEBUG:teuthology.orchestra.run.smithi039:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:47:17.069 INFO:teuthology.orchestra.run.smithi039.stdout:1 2024-09-21T07:47:17.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:17.073 DEBUG:teuthology.orchestra.run.smithi039:> route 2024-09-21T07:47:17.108 INFO:teuthology.orchestra.run.smithi039.stdout:Kernel IP routing table 2024-09-21T07:47:17.109 INFO:teuthology.orchestra.run.smithi039.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:47:17.109 INFO:teuthology.orchestra.run.smithi039.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T07:47:17.109 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T07:47:17.109 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:47:17.109 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:47:17.109 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-21T07:47:17.109 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T07:47:17.109 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T07:47:17.109 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T07:47:17.110 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-21T07:47:17.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:47:17.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:47:17.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:17.307 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-09-21T07:47:17.342 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:47:17.342 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T07:47:17.342 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:47:17.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:17.342 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-09-21T07:47:17.438 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:47:17.438 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T07:47:17.473 INFO:teuthology.orchestra.run.smithi039.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:47:17.474 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:47:17.474 DEBUG:teuthology.orchestra.run.smithi039:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:47:17.548 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-21T07:47:17.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:17.548 DEBUG:teuthology.orchestra.run.smithi039:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:47:17.749 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:47:17.749 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:47:17.785 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:47:17.785 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:47:17.785 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: resolved to: "172.21.15.39:3300,172.21.15.148:3300,172.21.15.148:3301" 2024-09-21T07:47:17.785 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: trying mount with new device syntax: 0@7ab9b04a-f88d-49aa-a493-e5c076637ffb.cephfs=/ 2024-09-21T07:47:17.785 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.39:3300/172.21.15.148:3300/172.21.15.148:3301" will pass to kernel 2024-09-21T07:47:17.786 INFO:teuthology.orchestra.run.smithi039.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T07:47:17.786 INFO:teuthology.orchestra.run.smithi039.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:47:17.786 INFO:teuthology.orchestra.run.smithi039.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:47:17.787 INFO:teuthology.orchestra.run.smithi039.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:47:17.787 INFO:teuthology.orchestra.run.smithi039.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:47:17.789 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:47:17.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:17.790 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:47:17.845 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:47:17.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:47:17.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:47:17.942 INFO:teuthology.orchestra.run.smithi039.stdout:client7823 2024-09-21T07:47:17.943 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:47:17.943 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:47:17.978 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get mds_max_snaps_per_dir 2024-09-21T07:47:18.262 INFO:teuthology.orchestra.run.smithi039.stdout:{"mds_max_snaps_per_dir":"100"} 2024-09-21T07:47:18.262 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_max_snaps_per_dir": "100" } 2024-09-21T07:47:18.263 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2024-09-21T07:47:18.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.491+0000 7f7df7200640 1 -- 172.21.15.39:0/1761247132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7df006f4b0 msgr2=0x7f7df015d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:18.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.492+0000 7f7df7200640 1 --2- 172.21.15.39:0/1761247132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7df006f4b0 0x7f7df015d440 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f7df00729c0 tx=0x7f7dec02aa30 comp rx=0 tx=0).stop 2024-09-21T07:47:18.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.492+0000 7f7df7200640 1 -- 172.21.15.39:0/1761247132 shutdown_connections 2024-09-21T07:47:18.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.492+0000 7f7df7200640 1 --2- 172.21.15.39:0/1761247132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7df006f4b0 0x7f7df015d440 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:18.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.492+0000 7f7df7200640 1 --2- 172.21.15.39:0/1761247132 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7df0160340 0x7f7df006ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:18.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.492+0000 7f7df7200640 1 --2- 172.21.15.39:0/1761247132 >> v2:172.21.15.148:3300/0 conn(0x7f7df01539f0 0x7f7df00b4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:18.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.492+0000 7f7df7200640 1 -- 172.21.15.39:0/1761247132 >> 172.21.15.39:0/1761247132 conn(0x7f7df006d660 msgr2=0x7f7df00b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:18.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.492+0000 7f7df7200640 1 -- 172.21.15.39:0/1761247132 shutdown_connections 2024-09-21T07:47:18.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.492+0000 7f7df7200640 1 -- 172.21.15.39:0/1761247132 wait complete. 2024-09-21T07:47:18.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.494+0000 7f7df7200640 1 Processor -- start 2024-09-21T07:47:18.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.494+0000 7f7df7200640 1 -- start start 2024-09-21T07:47:18.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.496+0000 7f7df7200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7df006f4b0 0x7f7df0157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:18.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.496+0000 7f7df7200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7df01539f0 0x7f7df01f3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:18.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.496+0000 7f7df7200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7df0160340 0x7f7df01f6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:18.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.496+0000 7f7df7200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7df00ad460 con 0x7f7df01539f0 2024-09-21T07:47:18.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.496+0000 7f7df7200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7df00ad2e0 con 0x7f7df006f4b0 2024-09-21T07:47:18.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.496+0000 7f7df7200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f7df00ad5e0 con 0x7f7df0160340 2024-09-21T07:47:18.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.496+0000 7f7df5400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7df01539f0 0x7f7df01f3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:18.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.496+0000 7f7df5400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7df01539f0 0x7f7df01f3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48454/0 (socket says 172.21.15.39:48454) 2024-09-21T07:47:18.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.496+0000 7f7df5400640 1 -- 172.21.15.39:0/704975233 learned_addr learned my addr 172.21.15.39:0/704975233 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:18.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.496+0000 7f7df5e00640 1 --2- 172.21.15.39:0/704975233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7df006f4b0 0x7f7df0157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:18.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.496+0000 7f7df6800640 1 --2- 172.21.15.39:0/704975233 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7df0160340 0x7f7df01f6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:18.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.496+0000 7f7df5400640 1 -- 172.21.15.39:0/704975233 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7df0160340 msgr2=0x7f7df01f6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:18.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.496+0000 7f7df5400640 1 --2- 172.21.15.39:0/704975233 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7df0160340 0x7f7df01f6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:18.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.496+0000 7f7df5400640 1 -- 172.21.15.39:0/704975233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7df006f4b0 msgr2=0x7f7df0157640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:18.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.496+0000 7f7df5400640 1 --2- 172.21.15.39:0/704975233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7df006f4b0 0x7f7df0157640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:18.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.496+0000 7f7df5400640 1 -- 172.21.15.39:0/704975233 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df00ced40 con 0x7f7df01539f0 2024-09-21T07:47:18.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.497+0000 7f7df5400640 1 --2- 172.21.15.39:0/704975233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7df01539f0 0x7f7df01f3c00 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f7de000e750 tx=0x7f7de000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:18.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.497+0000 7f7dea000640 1 -- 172.21.15.39:0/704975233 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de000ccc0 con 0x7f7df01539f0 2024-09-21T07:47:18.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.497+0000 7f7dea000640 1 -- 172.21.15.39:0/704975233 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7de000ce60 con 0x7f7df01539f0 2024-09-21T07:47:18.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.497+0000 7f7dea000640 1 -- 172.21.15.39:0/704975233 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de0005280 con 0x7f7df01539f0 2024-09-21T07:47:18.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.497+0000 7f7df7200640 1 -- 172.21.15.39:0/704975233 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df00cf020 con 0x7f7df01539f0 2024-09-21T07:47:18.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.497+0000 7f7df7200640 1 -- 172.21.15.39:0/704975233 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df00cf5e0 con 0x7f7df01539f0 2024-09-21T07:47:18.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.498+0000 7f7df7200640 1 -- 172.21.15.39:0/704975233 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7df00adb00 con 0x7f7df01539f0 2024-09-21T07:47:18.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.500+0000 7f7dea000640 1 -- 172.21.15.39:0/704975233 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7de0005420 con 0x7f7df01539f0 2024-09-21T07:47:18.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.501+0000 7f7dea000640 1 --2- 172.21.15.39:0/704975233 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7dd4078bc0 0x7f7dd407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:18.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.501+0000 7f7dea000640 1 -- 172.21.15.39:0/704975233 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 6932+0+0 (secure 0 0 0) 0x7f7de009bb00 con 0x7f7df01539f0 2024-09-21T07:47:18.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.501+0000 7f7df5e00640 1 --2- 172.21.15.39:0/704975233 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7dd4078bc0 0x7f7dd407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:18.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.502+0000 7f7df5e00640 1 --2- 172.21.15.39:0/704975233 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7dd4078bc0 0x7f7dd407b0d0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f7de4009ab0 tx=0x7f7de4005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:18.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.505+0000 7f7dea000640 1 -- 172.21.15.39:0/704975233 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7de000ee60 con 0x7f7df01539f0 2024-09-21T07:47:18.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:18.743+0000 7f7df7200640 1 -- 172.21.15.39:0/704975233 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f7df006df10 con 0x7f7df01539f0 2024-09-21T07:47:19.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.698+0000 7f7dea000640 1 -- 172.21.15.39:0/704975233 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v200) ==== 167+0+0 (secure 0 0 0) 0x7f7de009be20 con 0x7f7df01539f0 2024-09-21T07:47:19.700 INFO:teuthology.orchestra.run.smithi039.stderr:enabled new snapshots 2024-09-21T07:47:19.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.701+0000 7f7df7200640 1 -- 172.21.15.39:0/704975233 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7dd4078bc0 msgr2=0x7f7dd407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:19.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.701+0000 7f7df7200640 1 --2- 172.21.15.39:0/704975233 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7dd4078bc0 0x7f7dd407b0d0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f7de4009ab0 tx=0x7f7de4005f90 comp rx=0 tx=0).stop 2024-09-21T07:47:19.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.701+0000 7f7df7200640 1 -- 172.21.15.39:0/704975233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7df01539f0 msgr2=0x7f7df01f3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:19.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.701+0000 7f7df7200640 1 --2- 172.21.15.39:0/704975233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7df01539f0 0x7f7df01f3c00 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f7de000e750 tx=0x7f7de000ec10 comp rx=0 tx=0).stop 2024-09-21T07:47:19.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.701+0000 7f7df7200640 1 -- 172.21.15.39:0/704975233 shutdown_connections 2024-09-21T07:47:19.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.701+0000 7f7df7200640 1 --2- 172.21.15.39:0/704975233 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7dd4078bc0 0x7f7dd407b0d0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:19.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.701+0000 7f7df7200640 1 --2- 172.21.15.39:0/704975233 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7df0160340 0x7f7df01f6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:19.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.701+0000 7f7df7200640 1 --2- 172.21.15.39:0/704975233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7df01539f0 0x7f7df01f3c00 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:19.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.701+0000 7f7df7200640 1 --2- 172.21.15.39:0/704975233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7df006f4b0 0x7f7df0157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:19.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.701+0000 7f7df7200640 1 -- 172.21.15.39:0/704975233 >> 172.21.15.39:0/704975233 conn(0x7f7df006d660 msgr2=0x7f7df00d0210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:19.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.701+0000 7f7df7200640 1 -- 172.21.15.39:0/704975233 shutdown_connections 2024-09-21T07:47:19.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.701+0000 7f7df7200640 1 -- 172.21.15.39:0/704975233 wait complete. 2024-09-21T07:47:19.727 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 2 2024-09-21T07:47:19.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.959+0000 7fc756a00640 1 -- 172.21.15.39:0/482920018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7500cde20 msgr2=0x7fc75015ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:19.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.959+0000 7fc756a00640 1 --2- 172.21.15.39:0/482920018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7500cde20 0x7fc75015ccb0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7fc748005fd0 tx=0x7fc74802aa00 comp rx=0 tx=0).stop 2024-09-21T07:47:19.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.959+0000 7fc756a00640 1 -- 172.21.15.39:0/482920018 shutdown_connections 2024-09-21T07:47:19.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.959+0000 7fc756a00640 1 --2- 172.21.15.39:0/482920018 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc7500cd630 0x7fc750153870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:19.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.959+0000 7fc756a00640 1 --2- 172.21.15.39:0/482920018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7500cde20 0x7fc75015ccb0 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:19.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.959+0000 7fc756a00640 1 -- 172.21.15.39:0/482920018 >> 172.21.15.39:0/482920018 conn(0x7fc7500d0c70 msgr2=0x7fc7500d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:19.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.959+0000 7fc756a00640 1 -- 172.21.15.39:0/482920018 shutdown_connections 2024-09-21T07:47:19.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.959+0000 7fc756a00640 1 -- 172.21.15.39:0/482920018 wait complete. 2024-09-21T07:47:19.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.961+0000 7fc756a00640 1 Processor -- start 2024-09-21T07:47:19.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.962+0000 7fc756a00640 1 -- start start 2024-09-21T07:47:19.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.963+0000 7fc756a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7500b5ac0 0x7fc7501467a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:19.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.963+0000 7fc756a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7500cde20 0x7fc7500b1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:19.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.963+0000 7fc756a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc75006f4e0 0x7fc7500b1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:19.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.963+0000 7fc756a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc75015d580 con 0x7fc7500b5ac0 2024-09-21T07:47:19.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.963+0000 7fc756a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc75015d400 con 0x7fc7500cde20 2024-09-21T07:47:19.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.963+0000 7fc756a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc75015d700 con 0x7fc75006f4e0 2024-09-21T07:47:19.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.963+0000 7fc755600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7500b5ac0 0x7fc7501467a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:19.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.963+0000 7fc755600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7500b5ac0 0x7fc7501467a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48472/0 (socket says 172.21.15.39:48472) 2024-09-21T07:47:19.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.963+0000 7fc755600640 1 -- 172.21.15.39:0/2136474248 learned_addr learned my addr 172.21.15.39:0/2136474248 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:19.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.963+0000 7fc754c00640 1 --2- 172.21.15.39:0/2136474248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7500cde20 0x7fc7500b1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:19.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.963+0000 7fc756000640 1 --2- 172.21.15.39:0/2136474248 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc75006f4e0 0x7fc7500b1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:19.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.964+0000 7fc755600640 1 -- 172.21.15.39:0/2136474248 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc75006f4e0 msgr2=0x7fc7500b1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:19.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.964+0000 7fc755600640 1 --2- 172.21.15.39:0/2136474248 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc75006f4e0 0x7fc7500b1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:19.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.964+0000 7fc755600640 1 -- 172.21.15.39:0/2136474248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7500cde20 msgr2=0x7fc7500b1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:19.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.964+0000 7fc755600640 1 --2- 172.21.15.39:0/2136474248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7500cde20 0x7fc7500b1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:19.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.964+0000 7fc755600640 1 -- 172.21.15.39:0/2136474248 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7501475c0 con 0x7fc7500b5ac0 2024-09-21T07:47:19.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.964+0000 7fc755600640 1 --2- 172.21.15.39:0/2136474248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7500b5ac0 0x7fc7501467a0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7fc74400b3f0 tx=0x7fc74400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:19.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.964+0000 7fc74d600640 1 -- 172.21.15.39:0/2136474248 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc744015020 con 0x7fc7500b5ac0 2024-09-21T07:47:19.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.965+0000 7fc74d600640 1 -- 172.21.15.39:0/2136474248 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc744011920 con 0x7fc7500b5ac0 2024-09-21T07:47:19.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.965+0000 7fc74d600640 1 -- 172.21.15.39:0/2136474248 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc744011d70 con 0x7fc7500b5ac0 2024-09-21T07:47:19.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.965+0000 7fc756a00640 1 -- 172.21.15.39:0/2136474248 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7500af980 con 0x7fc7500b5ac0 2024-09-21T07:47:19.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.965+0000 7fc756a00640 1 -- 172.21.15.39:0/2136474248 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7500aff40 con 0x7fc7500b5ac0 2024-09-21T07:47:19.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.966+0000 7fc756a00640 1 -- 172.21.15.39:0/2136474248 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc75015dbb0 con 0x7fc7500b5ac0 2024-09-21T07:47:19.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.967+0000 7fc74d600640 1 -- 172.21.15.39:0/2136474248 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc744018020 con 0x7fc7500b5ac0 2024-09-21T07:47:19.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.968+0000 7fc74d600640 1 --2- 172.21.15.39:0/2136474248 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc730078bc0 0x7fc73007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:19.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.968+0000 7fc74d600640 1 -- 172.21.15.39:0/2136474248 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 6932+0+0 (secure 0 0 0) 0x7fc74409ad80 con 0x7fc7500b5ac0 2024-09-21T07:47:19.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.968+0000 7fc754c00640 1 --2- 172.21.15.39:0/2136474248 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc730078bc0 0x7fc73007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:19.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.969+0000 7fc754c00640 1 --2- 172.21.15.39:0/2136474248 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc730078bc0 0x7fc73007b0d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fc7501521b0 tx=0x7fc73c00b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:19.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:19.972+0000 7fc74d600640 1 -- 172.21.15.39:0/2136474248 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc74401f030 con 0x7fc7500b5ac0 2024-09-21T07:47:20.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.210+0000 7fc756a00640 1 -- 172.21.15.39:0/2136474248 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) -- 0x7fc750150970 con 0x7fc7500b5ac0 2024-09-21T07:47:20.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.708+0000 7fc74d600640 1 -- 172.21.15.39:0/2136474248 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v201) ==== 105+0+0 (secure 0 0 0) 0x7fc744062690 con 0x7fc7500b5ac0 2024-09-21T07:47:20.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.711+0000 7fc756a00640 1 -- 172.21.15.39:0/2136474248 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc730078bc0 msgr2=0x7fc73007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:20.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.711+0000 7fc756a00640 1 --2- 172.21.15.39:0/2136474248 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc730078bc0 0x7fc73007b0d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fc7501521b0 tx=0x7fc73c00b2e0 comp rx=0 tx=0).stop 2024-09-21T07:47:20.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.711+0000 7fc756a00640 1 -- 172.21.15.39:0/2136474248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7500b5ac0 msgr2=0x7fc7501467a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:20.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.711+0000 7fc756a00640 1 --2- 172.21.15.39:0/2136474248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7500b5ac0 0x7fc7501467a0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7fc74400b3f0 tx=0x7fc74400b8b0 comp rx=0 tx=0).stop 2024-09-21T07:47:20.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.711+0000 7fc756a00640 1 -- 172.21.15.39:0/2136474248 shutdown_connections 2024-09-21T07:47:20.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.711+0000 7fc756a00640 1 --2- 172.21.15.39:0/2136474248 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc730078bc0 0x7fc73007b0d0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:20.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.711+0000 7fc756a00640 1 --2- 172.21.15.39:0/2136474248 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc75006f4e0 0x7fc7500b1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:20.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.711+0000 7fc756a00640 1 --2- 172.21.15.39:0/2136474248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7500cde20 0x7fc7500b1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:20.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.711+0000 7fc756a00640 1 --2- 172.21.15.39:0/2136474248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7500b5ac0 0x7fc7501467a0 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:20.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.711+0000 7fc756a00640 1 -- 172.21.15.39:0/2136474248 >> 172.21.15.39:0/2136474248 conn(0x7fc7500d0c70 msgr2=0x7fc75006d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:20.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.711+0000 7fc756a00640 1 -- 172.21.15.39:0/2136474248 shutdown_connections 2024-09-21T07:47:20.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.711+0000 7fc756a00640 1 -- 172.21.15.39:0/2136474248 wait complete. 2024-09-21T07:47:20.741 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:47:20.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.965+0000 7f839c800640 1 -- 172.21.15.39:0/2707764956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83940ccc60 msgr2=0x7f83940cd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:20.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.965+0000 7f839c800640 1 --2- 172.21.15.39:0/2707764956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83940ccc60 0x7f83940cd070 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f8394071e60 tx=0x7f839002a980 comp rx=0 tx=0).stop 2024-09-21T07:47:20.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.965+0000 7f839c800640 1 -- 172.21.15.39:0/2707764956 shutdown_connections 2024-09-21T07:47:20.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.966+0000 7f839c800640 1 --2- 172.21.15.39:0/2707764956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83940ccc60 0x7f83940cd070 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:20.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.966+0000 7f839c800640 1 --2- 172.21.15.39:0/2707764956 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f839415d830 0x7f8394156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:20.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.966+0000 7f839c800640 1 --2- 172.21.15.39:0/2707764956 >> v2:172.21.15.148:3300/0 conn(0x7f83940b30a0 0x7f83940cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:20.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.966+0000 7f839c800640 1 -- 172.21.15.39:0/2707764956 >> 172.21.15.39:0/2707764956 conn(0x7f83940b2460 msgr2=0x7f839406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:20.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.966+0000 7f839c800640 1 -- 172.21.15.39:0/2707764956 shutdown_connections 2024-09-21T07:47:20.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.966+0000 7f839c800640 1 -- 172.21.15.39:0/2707764956 wait complete. 2024-09-21T07:47:20.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.968+0000 7f839c800640 1 Processor -- start 2024-09-21T07:47:20.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.968+0000 7f839c800640 1 -- start start 2024-09-21T07:47:20.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.969+0000 7f839c800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83940b30a0 0x7f83941f3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:20.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.969+0000 7f839c800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83940ccc60 0x7f83941f6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:20.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.970+0000 7f839c800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f839415d830 0x7f83941f7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:20.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.970+0000 7f839c800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f83940af200 con 0x7f839415d830 2024-09-21T07:47:20.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.970+0000 7f839c800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f83940af080 con 0x7f83940ccc60 2024-09-21T07:47:20.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.970+0000 7f839c800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f83940af380 con 0x7f83940b30a0 2024-09-21T07:47:20.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.970+0000 7f839a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f839415d830 0x7f83941f7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:20.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.970+0000 7f839a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f839415d830 0x7f83941f7500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48488/0 (socket says 172.21.15.39:48488) 2024-09-21T07:47:20.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.970+0000 7f839a800640 1 -- 172.21.15.39:0/3163604276 learned_addr learned my addr 172.21.15.39:0/3163604276 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:20.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.970+0000 7f8399400640 1 --2- 172.21.15.39:0/3163604276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83940ccc60 0x7f83941f6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:20.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.970+0000 7f8399e00640 1 --2- 172.21.15.39:0/3163604276 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83940b30a0 0x7f83941f3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:20.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.970+0000 7f839a800640 1 -- 172.21.15.39:0/3163604276 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83940b30a0 msgr2=0x7f83941f3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:20.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.970+0000 7f839a800640 1 --2- 172.21.15.39:0/3163604276 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83940b30a0 0x7f83941f3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:20.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.970+0000 7f839a800640 1 -- 172.21.15.39:0/3163604276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83940ccc60 msgr2=0x7f83941f6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:20.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.970+0000 7f839a800640 1 --2- 172.21.15.39:0/3163604276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83940ccc60 0x7f83941f6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:20.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.970+0000 7f839a800640 1 -- 172.21.15.39:0/3163604276 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83940c5850 con 0x7f839415d830 2024-09-21T07:47:20.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.971+0000 7f839a800640 1 --2- 172.21.15.39:0/3163604276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f839415d830 0x7f83941f7500 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f8390033040 tx=0x7f8390003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:20.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.971+0000 7f838e000640 1 -- 172.21.15.39:0/3163604276 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8390034040 con 0x7f839415d830 2024-09-21T07:47:20.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.971+0000 7f838e000640 1 -- 172.21.15.39:0/3163604276 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8390004540 con 0x7f839415d830 2024-09-21T07:47:20.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.971+0000 7f839c800640 1 -- 172.21.15.39:0/3163604276 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83940c5570 con 0x7f839415d830 2024-09-21T07:47:20.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.971+0000 7f838e000640 1 -- 172.21.15.39:0/3163604276 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8390045480 con 0x7f839415d830 2024-09-21T07:47:20.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.971+0000 7f839c800640 1 -- 172.21.15.39:0/3163604276 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83940c76f0 con 0x7f839415d830 2024-09-21T07:47:20.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.972+0000 7f839c800640 1 -- 172.21.15.39:0/3163604276 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83940c52e0 con 0x7f839415d830 2024-09-21T07:47:20.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.973+0000 7f838e000640 1 -- 172.21.15.39:0/3163604276 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8390037020 con 0x7f839415d830 2024-09-21T07:47:20.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.974+0000 7f838e000640 1 --2- 172.21.15.39:0/3163604276 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8378078bc0 0x7f837807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:20.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.974+0000 7f838e000640 1 -- 172.21.15.39:0/3163604276 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 6932+0+0 (secure 0 0 0) 0x7f83900bc9d0 con 0x7f839415d830 2024-09-21T07:47:20.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.975+0000 7f8399e00640 1 --2- 172.21.15.39:0/3163604276 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8378078bc0 0x7f837807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:20.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.976+0000 7f8399e00640 1 --2- 172.21.15.39:0/3163604276 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8378078bc0 0x7f837807b0d0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f8388005fd0 tx=0x7f838800b260 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:20.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:20.979+0000 7f838e000640 1 -- 172.21.15.39:0/3163604276 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f83900842f0 con 0x7f839415d830 2024-09-21T07:47:21.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:21.212+0000 7f839c800640 1 -- 172.21.15.39:0/3163604276 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83940b09f0 con 0x7f839415d830 2024-09-21T07:47:21.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:21.213+0000 7f838e000640 1 -- 172.21.15.39:0/3163604276 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 202 v202) ==== 95+0+5246 (secure 0 0 0) 0x7f8390088d20 con 0x7f839415d830 2024-09-21T07:47:21.215 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:47:21.215 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":202,"btime":"2024-09-21T07:47:20:716873+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7715,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3057062733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3057062733},{"type":"v1","addr":"172.21.15.148:6836","nonce":3057062733}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":194},{"gid":8689,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/8652512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":8652512},{"type":"v1","addr":"172.21.15.148:6837","nonce":8652512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":194}],"filesystems":[{"mdsmap":{"epoch":202,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:47:09.140224+0000","modified":"2024-09-21T07:47:20.716857+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":9492,"mds_1":9483},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9483":{"gid":9483,"name":"a","rank":1,"incarnation":202,"state":"up:creating","state_seq":1,"addr":"172.21.15.39:6833/3199732683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3199732683},{"type":"v1","addr":"172.21.15.39:6833","nonce":3199732683}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_9492":{"gid":9492,"name":"c","rank":0,"incarnation":195,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/3348864735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3348864735},{"type":"v1","addr":"172.21.15.39:6835","nonce":3348864735}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9492,"qdb_cluster":[9492]},"id":10}]} 2024-09-21T07:47:21.215 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 202 2024-09-21T07:47:21.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:21.215+0000 7f839c800640 1 -- 172.21.15.39:0/3163604276 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8378078bc0 msgr2=0x7f837807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:21.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:21.215+0000 7f839c800640 1 --2- 172.21.15.39:0/3163604276 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8378078bc0 0x7f837807b0d0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f8388005fd0 tx=0x7f838800b260 comp rx=0 tx=0).stop 2024-09-21T07:47:21.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:21.215+0000 7f839c800640 1 -- 172.21.15.39:0/3163604276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f839415d830 msgr2=0x7f83941f7500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:21.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:21.215+0000 7f839c800640 1 --2- 172.21.15.39:0/3163604276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f839415d830 0x7f83941f7500 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f8390033040 tx=0x7f8390003f10 comp rx=0 tx=0).stop 2024-09-21T07:47:21.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:21.215+0000 7f839c800640 1 -- 172.21.15.39:0/3163604276 shutdown_connections 2024-09-21T07:47:21.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:21.215+0000 7f839c800640 1 --2- 172.21.15.39:0/3163604276 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8378078bc0 0x7f837807b0d0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:21.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:21.215+0000 7f839c800640 1 --2- 172.21.15.39:0/3163604276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f839415d830 0x7f83941f7500 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:21.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:21.215+0000 7f839c800640 1 --2- 172.21.15.39:0/3163604276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83940ccc60 0x7f83941f6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:21.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:21.215+0000 7f839c800640 1 --2- 172.21.15.39:0/3163604276 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83940b30a0 0x7f83941f3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:21.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:21.215+0000 7f839c800640 1 -- 172.21.15.39:0/3163604276 >> 172.21.15.39:0/3163604276 conn(0x7f83940b2460 msgr2=0x7f839415eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:21.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:21.215+0000 7f839c800640 1 -- 172.21.15.39:0/3163604276 shutdown_connections 2024-09-21T07:47:21.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:21.215+0000 7f839c800640 1 -- 172.21.15.39:0/3163604276 wait complete. 2024-09-21T07:47:21.641 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 202, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:47:09.140224+0000', 'modified': '2024-09-21T07:47:20.716857+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 9492, 'mds_1': 9483}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9483': {'gid': 9483, 'name': 'a', 'rank': 1, 'incarnation': 202, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.39:6833/3199732683', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 3199732683}, {'type': 'v1', 'addr': '172.21.15.39:6833', 'nonce': 3199732683}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_9492': {'gid': 9492, 'name': 'c', 'rank': 0, 'incarnation': 195, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.39:6835/3348864735', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6834', 'nonce': 3348864735}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 3348864735}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9492, 'qdb_cluster': [9492]} 2024-09-21T07:47:21.641 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9483:up:creating 2024-09-21T07:47:22.642 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:47:22.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:22.878+0000 7f4e85600640 1 -- 172.21.15.39:0/818934487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e800b3d80 msgr2=0x7f4e800ce2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:22.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:22.878+0000 7f4e85600640 1 --2- 172.21.15.39:0/818934487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e800b3d80 0x7f4e800ce2b0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4e68009830 tx=0x7f4e6802a510 comp rx=0 tx=0).stop 2024-09-21T07:47:22.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:22.878+0000 7f4e85600640 1 -- 172.21.15.39:0/818934487 shutdown_connections 2024-09-21T07:47:22.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:22.878+0000 7f4e85600640 1 --2- 172.21.15.39:0/818934487 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4e8006ef90 0x7f4e8006f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:22.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:22.878+0000 7f4e85600640 1 --2- 172.21.15.39:0/818934487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e800b3d80 0x7f4e800ce2b0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:22.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:22.878+0000 7f4e85600640 1 -- 172.21.15.39:0/818934487 >> 172.21.15.39:0/818934487 conn(0x7f4e800b2460 msgr2=0x7f4e8006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:22.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:22.878+0000 7f4e85600640 1 -- 172.21.15.39:0/818934487 shutdown_connections 2024-09-21T07:47:22.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:22.879+0000 7f4e85600640 1 -- 172.21.15.39:0/818934487 wait complete. 2024-09-21T07:47:22.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:22.881+0000 7f4e85600640 1 Processor -- start 2024-09-21T07:47:22.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:22.881+0000 7f4e85600640 1 -- start start 2024-09-21T07:47:22.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:22.882+0000 7f4e85600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e8006ef90 0x7f4e80158d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:22.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:22.882+0000 7f4e85600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e800b4c40 0x7f4e800999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:22.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:22.882+0000 7f4e85600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4e800b09f0 0x7f4e800744a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:22.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:22.882+0000 7f4e85600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8006fa60 con 0x7f4e800b4c40 2024-09-21T07:47:22.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:22.882+0000 7f4e85600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8006f8e0 con 0x7f4e8006ef90 2024-09-21T07:47:22.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:22.882+0000 7f4e85600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f4e8006fbe0 con 0x7f4e800b09f0 2024-09-21T07:47:22.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:22.883+0000 7f4e7ea00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e8006ef90 0x7f4e80158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:22.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:22.883+0000 7f4e7ea00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e8006ef90 0x7f4e80158d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.39:46136/0 (socket says 172.21.15.39:46136) 2024-09-21T07:47:22.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:22.883+0000 7f4e7ea00640 1 -- 172.21.15.39:0/3869337520 learned_addr learned my addr 172.21.15.39:0/3869337520 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:22.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:22.883+0000 7f4e7f400640 1 --2- 172.21.15.39:0/3869337520 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4e800b09f0 0x7f4e800744a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:22.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:22.883+0000 7f4e77e00640 1 --2- 172.21.15.39:0/3869337520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e800b4c40 0x7f4e800999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:22.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:22.883+0000 7f4e7ea00640 1 -- 172.21.15.39:0/3869337520 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4e800b09f0 msgr2=0x7f4e800744a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:22.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:22.883+0000 7f4e7ea00640 1 --2- 172.21.15.39:0/3869337520 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4e800b09f0 0x7f4e800744a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:22.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:22.883+0000 7f4e7ea00640 1 -- 172.21.15.39:0/3869337520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e800b4c40 msgr2=0x7f4e800999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:22.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:22.883+0000 7f4e7ea00640 1 --2- 172.21.15.39:0/3869337520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e800b4c40 0x7f4e800999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:22.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:22.883+0000 7f4e7ea00640 1 -- 172.21.15.39:0/3869337520 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e80099ee0 con 0x7f4e8006ef90 2024-09-21T07:47:22.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:22.884+0000 7f4e7ea00640 1 --2- 172.21.15.39:0/3869337520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e8006ef90 0x7f4e80158d70 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f4e7000c3d0 tx=0x7f4e7000c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:22.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:22.884+0000 7f4e76a00640 1 -- 172.21.15.39:0/3869337520 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e70006890 con 0x7f4e8006ef90 2024-09-21T07:47:22.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:22.884+0000 7f4e76a00640 1 -- 172.21.15.39:0/3869337520 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e70006b20 con 0x7f4e8006ef90 2024-09-21T07:47:22.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:22.884+0000 7f4e85600640 1 -- 172.21.15.39:0/3869337520 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e80157ea0 con 0x7f4e8006ef90 2024-09-21T07:47:22.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:22.884+0000 7f4e76a00640 1 -- 172.21.15.39:0/3869337520 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e70007020 con 0x7f4e8006ef90 2024-09-21T07:47:22.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:22.884+0000 7f4e85600640 1 -- 172.21.15.39:0/3869337520 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e8015bfb0 con 0x7f4e8006ef90 2024-09-21T07:47:22.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:22.885+0000 7f4e85600640 1 -- 172.21.15.39:0/3869337520 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e801583c0 con 0x7f4e8006ef90 2024-09-21T07:47:22.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:22.886+0000 7f4e76a00640 1 -- 172.21.15.39:0/3869337520 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4e70019020 con 0x7f4e8006ef90 2024-09-21T07:47:22.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:22.887+0000 7f4e76a00640 1 --2- 172.21.15.39:0/3869337520 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4e5c078b90 0x7f4e5c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:22.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:22.887+0000 7f4e76a00640 1 -- 172.21.15.39:0/3869337520 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 6932+0+0 (secure 0 0 0) 0x7f4e7009a8e0 con 0x7f4e8006ef90 2024-09-21T07:47:22.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:22.888+0000 7f4e77e00640 1 --2- 172.21.15.39:0/3869337520 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4e5c078b90 0x7f4e5c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:22.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:22.889+0000 7f4e77e00640 1 --2- 172.21.15.39:0/3869337520 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4e5c078b90 0x7f4e5c07b0a0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f4e68034000 tx=0x7f4e68004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:22.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:22.893+0000 7f4e76a00640 1 -- 172.21.15.39:0/3869337520 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4e801583c0 con 0x7f4e8006ef90 2024-09-21T07:47:23.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.127+0000 7f4e85600640 1 -- 172.21.15.39:0/3869337520 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e8006f3a0 con 0x7f4e8006ef90 2024-09-21T07:47:23.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.129+0000 7f4e76a00640 1 -- 172.21.15.39:0/3869337520 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 203 v203) ==== 95+0+5249 (secure 0 0 0) 0x7f4e70062160 con 0x7f4e8006ef90 2024-09-21T07:47:23.131 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:47:23.131 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":203,"btime":"2024-09-21T07:47:21:720630+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7715,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3057062733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3057062733},{"type":"v1","addr":"172.21.15.148:6836","nonce":3057062733}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":194},{"gid":8689,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/8652512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":8652512},{"type":"v1","addr":"172.21.15.148:6837","nonce":8652512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":194}],"filesystems":[{"mdsmap":{"epoch":203,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:47:09.140224+0000","modified":"2024-09-21T07:47:21.720616+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":9492,"mds_1":9483},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9483":{"gid":9483,"name":"a","rank":1,"incarnation":202,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6833/3199732683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3199732683},{"type":"v1","addr":"172.21.15.39:6833","nonce":3199732683}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_9492":{"gid":9492,"name":"c","rank":0,"incarnation":195,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/3348864735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3348864735},{"type":"v1","addr":"172.21.15.39:6835","nonce":3348864735}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9492,"qdb_cluster":[9483,9492]},"id":10}]} 2024-09-21T07:47:23.131 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 203 2024-09-21T07:47:23.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.131+0000 7f4e85600640 1 -- 172.21.15.39:0/3869337520 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4e5c078b90 msgr2=0x7f4e5c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:23.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.131+0000 7f4e85600640 1 --2- 172.21.15.39:0/3869337520 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4e5c078b90 0x7f4e5c07b0a0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f4e68034000 tx=0x7f4e68004730 comp rx=0 tx=0).stop 2024-09-21T07:47:23.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.131+0000 7f4e85600640 1 -- 172.21.15.39:0/3869337520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e8006ef90 msgr2=0x7f4e80158d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:23.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.131+0000 7f4e85600640 1 --2- 172.21.15.39:0/3869337520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e8006ef90 0x7f4e80158d70 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f4e7000c3d0 tx=0x7f4e7000c890 comp rx=0 tx=0).stop 2024-09-21T07:47:23.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.131+0000 7f4e85600640 1 -- 172.21.15.39:0/3869337520 shutdown_connections 2024-09-21T07:47:23.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.131+0000 7f4e85600640 1 --2- 172.21.15.39:0/3869337520 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4e5c078b90 0x7f4e5c07b0a0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.131+0000 7f4e85600640 1 --2- 172.21.15.39:0/3869337520 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4e800b09f0 0x7f4e800744a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.131+0000 7f4e85600640 1 --2- 172.21.15.39:0/3869337520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e800b4c40 0x7f4e800999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.131+0000 7f4e85600640 1 --2- 172.21.15.39:0/3869337520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4e8006ef90 0x7f4e80158d70 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.131+0000 7f4e85600640 1 -- 172.21.15.39:0/3869337520 >> 172.21.15.39:0/3869337520 conn(0x7f4e800b2460 msgr2=0x7f4e800d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:23.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.131+0000 7f4e85600640 1 -- 172.21.15.39:0/3869337520 shutdown_connections 2024-09-21T07:47:23.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.131+0000 7f4e85600640 1 -- 172.21.15.39:0/3869337520 wait complete. 2024-09-21T07:47:23.561 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 203, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:47:09.140224+0000', 'modified': '2024-09-21T07:47:21.720616+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 9492, 'mds_1': 9483}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9483': {'gid': 9483, 'name': 'a', 'rank': 1, 'incarnation': 202, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.39:6833/3199732683', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 3199732683}, {'type': 'v1', 'addr': '172.21.15.39:6833', 'nonce': 3199732683}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_9492': {'gid': 9492, 'name': 'c', 'rank': 0, 'incarnation': 195, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.39:6835/3348864735', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6834', 'nonce': 3348864735}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 3348864735}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9492, 'qdb_cluster': [9483, 9492]} 2024-09-21T07:47:23.562 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2024-09-21T07:47:23.562 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T07:47:23.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.790+0000 7f9711c00640 1 -- 172.21.15.39:0/2373523810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f970c0d0d40 msgr2=0x7f970c0cf860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:23.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.790+0000 7f9711c00640 1 --2- 172.21.15.39:0/2373523810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f970c0d0d40 0x7f970c0cf860 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f970c0cff20 tx=0x7f970402a890 comp rx=0 tx=0).stop 2024-09-21T07:47:23.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.791+0000 7f9711c00640 1 -- 172.21.15.39:0/2373523810 shutdown_connections 2024-09-21T07:47:23.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.791+0000 7f9711c00640 1 --2- 172.21.15.39:0/2373523810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f970c0d0d40 0x7f970c0cf860 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.791+0000 7f9711c00640 1 -- 172.21.15.39:0/2373523810 >> 172.21.15.39:0/2373523810 conn(0x7f970c06cfc0 msgr2=0x7f970c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:23.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.791+0000 7f9711c00640 1 -- 172.21.15.39:0/2373523810 shutdown_connections 2024-09-21T07:47:23.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.791+0000 7f9711c00640 1 -- 172.21.15.39:0/2373523810 wait complete. 2024-09-21T07:47:23.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.793+0000 7f9711c00640 1 Processor -- start 2024-09-21T07:47:23.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.793+0000 7f9711c00640 1 -- start start 2024-09-21T07:47:23.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.794+0000 7f9711c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f970c073dd0 0x7f970c13acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:23.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.794+0000 7f9711c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f970c0cf450 0x7f970c0b1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:23.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.794+0000 7f9711c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f970c0b2cf0 0x7f970c0b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:23.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.794+0000 7f9711c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f970c167d20 con 0x7f970c0cf450 2024-09-21T07:47:23.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.794+0000 7f9711c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f970c167ba0 con 0x7f970c0b2cf0 2024-09-21T07:47:23.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.794+0000 7f9711c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f970c167ea0 con 0x7f970c073dd0 2024-09-21T07:47:23.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.795+0000 7f9703e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f970c0cf450 0x7f970c0b1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:23.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.795+0000 7f9703e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f970c0cf450 0x7f970c0b1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49650/0 (socket says 172.21.15.39:49650) 2024-09-21T07:47:23.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.795+0000 7f9703e00640 1 -- 172.21.15.39:0/3562677309 learned_addr learned my addr 172.21.15.39:0/3562677309 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:23.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.795+0000 7f970aa00640 1 --2- 172.21.15.39:0/3562677309 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f970c073dd0 0x7f970c13acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:23.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.795+0000 7f970b400640 1 --2- 172.21.15.39:0/3562677309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f970c0b2cf0 0x7f970c0b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:23.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.795+0000 7f9703e00640 1 -- 172.21.15.39:0/3562677309 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f970c073dd0 msgr2=0x7f970c13acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:23.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.795+0000 7f9703e00640 1 --2- 172.21.15.39:0/3562677309 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f970c073dd0 0x7f970c13acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.795+0000 7f9703e00640 1 -- 172.21.15.39:0/3562677309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f970c0b2cf0 msgr2=0x7f970c0b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:23.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.795+0000 7f9703e00640 1 --2- 172.21.15.39:0/3562677309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f970c0b2cf0 0x7f970c0b1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.795+0000 7f9703e00640 1 -- 172.21.15.39:0/3562677309 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f970c149980 con 0x7f970c0cf450 2024-09-21T07:47:23.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.796+0000 7f9703e00640 1 --2- 172.21.15.39:0/3562677309 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f970c0cf450 0x7f970c0b1610 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f96f400c3a0 tx=0x7f96f400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:23.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.796+0000 7f9702a00640 1 -- 172.21.15.39:0/3562677309 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f4017070 con 0x7f970c0cf450 2024-09-21T07:47:23.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.796+0000 7f9702a00640 1 -- 172.21.15.39:0/3562677309 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96f40053c0 con 0x7f970c0cf450 2024-09-21T07:47:23.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.796+0000 7f9711c00640 1 -- 172.21.15.39:0/3562677309 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f970c14d2e0 con 0x7f970c0cf450 2024-09-21T07:47:23.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.796+0000 7f9702a00640 1 -- 172.21.15.39:0/3562677309 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f4003850 con 0x7f970c0cf450 2024-09-21T07:47:23.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.796+0000 7f9711c00640 1 -- 172.21.15.39:0/3562677309 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f970c149bf0 con 0x7f970c0cf450 2024-09-21T07:47:23.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.798+0000 7f9702a00640 1 -- 172.21.15.39:0/3562677309 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f96f401a070 con 0x7f970c0cf450 2024-09-21T07:47:23.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.799+0000 7f9702a00640 1 --2- 172.21.15.39:0/3562677309 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f96e8078bc0 0x7f96e807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:23.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.800+0000 7f9702a00640 1 -- 172.21.15.39:0/3562677309 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 6932+0+0 (secure 0 0 0) 0x7f96f409a590 con 0x7f970c0cf450 2024-09-21T07:47:23.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.800+0000 7f9718c57740 1 Processor -- start 2024-09-21T07:47:23.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.800+0000 7f970aa00640 1 --2- 172.21.15.39:0/3562677309 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f96e8078bc0 0x7f96e807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:23.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.800+0000 7f9718c57740 1 -- start start 2024-09-21T07:47:23.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.801+0000 7f970aa00640 1 --2- 172.21.15.39:0/3562677309 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f96e8078bc0 0x7f96e807b0d0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f96fc000f60 tx=0x7f96fc009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:23.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.801+0000 7f9718c57740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56233e81a270 0x56233ebea970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:23.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.801+0000 7f9718c57740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56233e7f13b0 0x56233ebeaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:23.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.801+0000 7f9718c57740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233e7f9590 0x56233ebbaf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:23.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.802+0000 7f9718c57740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56233eb44430 con 0x56233e7f13b0 2024-09-21T07:47:23.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.802+0000 7f9718c57740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56233eb345d0 con 0x56233e7f9590 2024-09-21T07:47:23.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.802+0000 7f9718c57740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x56233eb23a00 con 0x56233e81a270 2024-09-21T07:47:23.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.802+0000 7f9703e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56233e7f13b0 0x56233ebeaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:23.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.802+0000 7f9703e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56233e7f13b0 0x56233ebeaeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49654/0 (socket says 172.21.15.39:49654) 2024-09-21T07:47:23.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.802+0000 7f9703e00640 1 -- 172.21.15.39:0/3412304742 learned_addr learned my addr 172.21.15.39:0/3412304742 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:23.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.802+0000 7f970aa00640 1 --2- 172.21.15.39:0/3412304742 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56233e81a270 0x56233ebea970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:23.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.802+0000 7f970b400640 1 --2- 172.21.15.39:0/3412304742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233e7f9590 0x56233ebbaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:23.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.802+0000 7f9703e00640 1 -- 172.21.15.39:0/3412304742 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56233e81a270 msgr2=0x56233ebea970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:47:23.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.802+0000 7f9703e00640 1 --2- 172.21.15.39:0/3412304742 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56233e81a270 0x56233ebea970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.802+0000 7f9703e00640 1 -- 172.21.15.39:0/3412304742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233e7f9590 msgr2=0x56233ebbaf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:47:23.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.802+0000 7f9703e00640 1 --2- 172.21.15.39:0/3412304742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233e7f9590 0x56233ebbaf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.802+0000 7f9703e00640 1 -- 172.21.15.39:0/3412304742 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56233eaec520 con 0x56233e7f13b0 2024-09-21T07:47:23.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.802+0000 7f9703e00640 1 --2- 172.21.15.39:0/3412304742 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56233e7f13b0 0x56233ebeaeb0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f96f4061b10 tx=0x7f96f4061580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5d8df752b4850b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:23.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.803+0000 7f96fb400640 1 -- 172.21.15.39:0/3412304742 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f4088180 con 0x56233e7f13b0 2024-09-21T07:47:23.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.803+0000 7f96fb400640 1 -- 172.21.15.39:0/3412304742 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96f4061e10 con 0x56233e7f13b0 2024-09-21T07:47:23.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.803+0000 7f96fb400640 1 -- 172.21.15.39:0/3412304742 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f4066840 con 0x56233e7f13b0 2024-09-21T07:47:23.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.803+0000 7f9718c57740 1 -- 172.21.15.39:0/3412304742 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56233e7f13b0 msgr2=0x56233ebeaeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:23.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.803+0000 7f9718c57740 1 --2- 172.21.15.39:0/3412304742 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56233e7f13b0 0x56233ebeaeb0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f96f4061b10 tx=0x7f96f4061580 comp rx=0 tx=0).stop 2024-09-21T07:47:23.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.804+0000 7f9718c57740 1 -- 172.21.15.39:0/3412304742 shutdown_connections 2024-09-21T07:47:23.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.804+0000 7f9718c57740 1 --2- 172.21.15.39:0/3412304742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233e7f9590 0x56233ebbaf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.804+0000 7f9718c57740 1 --2- 172.21.15.39:0/3412304742 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56233e7f13b0 0x56233ebeaeb0 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.804+0000 7f9718c57740 1 --2- 172.21.15.39:0/3412304742 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56233e81a270 0x56233ebea970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.804+0000 7f9718c57740 1 -- 172.21.15.39:0/3412304742 >> 172.21.15.39:0/3412304742 conn(0x56233e80e3c0 msgr2=0x56233ebc0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:23.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.804+0000 7f9718c57740 1 -- 172.21.15.39:0/3412304742 shutdown_connections 2024-09-21T07:47:23.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.804+0000 7f9718c57740 1 -- 172.21.15.39:0/3412304742 wait complete. 2024-09-21T07:47:23.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.805+0000 7f9718c57740 1 -- start start 2024-09-21T07:47:23.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.805+0000 7f9718c57740 1 Processor -- start 2024-09-21T07:47:23.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.806+0000 7f9718c57740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56233e7f13b0 0x56233ec060c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:23.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.806+0000 7f9718c57740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56233e7f9590 0x56233ebe10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:23.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.806+0000 7f9718c57740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233e81a270 0x56233ec148b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:23.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.806+0000 7f9718c57740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56233eaca330 con 0x56233e7f13b0 2024-09-21T07:47:23.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.806+0000 7f9718c57740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56233eb27810 con 0x56233e81a270 2024-09-21T07:47:23.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.806+0000 7f9718c57740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x56233e894620 con 0x56233e7f9590 2024-09-21T07:47:23.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.807+0000 7f9718c57740 10 client.0 _ll_register_callbacks cb 0x56233eacf420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:47:23.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.807+0000 7f970aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56233e7f13b0 0x56233ec060c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:23.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.807+0000 7f970aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56233e7f13b0 0x56233ec060c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49668/0 (socket says 172.21.15.39:49668) 2024-09-21T07:47:23.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.807+0000 7f970aa00640 1 -- 172.21.15.39:0/4212734033 learned_addr learned my addr 172.21.15.39:0/4212734033 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:23.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.807+0000 7f970aa00640 1 -- 172.21.15.39:0/4212734033 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56233e7f9590 msgr2=0x56233ebe10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:47:23.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.807+0000 7f970aa00640 1 --2- 172.21.15.39:0/4212734033 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56233e7f9590 0x56233ebe10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.807+0000 7f970aa00640 1 -- 172.21.15.39:0/4212734033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233e81a270 msgr2=0x56233ec148b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:47:23.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.807+0000 7f970aa00640 1 --2- 172.21.15.39:0/4212734033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233e81a270 0x56233ec148b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.807+0000 7f970aa00640 1 -- 172.21.15.39:0/4212734033 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56233e885850 con 0x56233e7f13b0 2024-09-21T07:47:23.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.807+0000 7f970aa00640 1 --2- 172.21.15.39:0/4212734033 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56233e7f13b0 0x56233ec060c0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f96fc014730 tx=0x7f96fc000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac52d1a031383633 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:23.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.807+0000 7f96efe00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:47:23.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.808+0000 7f96efe00640 1 -- 172.21.15.39:0/4212734033 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc0135c0 con 0x56233e7f13b0 2024-09-21T07:47:23.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.808+0000 7f96efe00640 1 -- 172.21.15.39:0/4212734033 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(194..194 src has 1..194) ==== 6932+0+0 (secure 0 0 0) 0x7f96fc013760 con 0x56233e7f13b0 2024-09-21T07:47:23.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.808+0000 7f9718c57740 1 -- 172.21.15.39:0/4212734033 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96f4061e10 con 0x56233e7f13b0 2024-09-21T07:47:23.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.808+0000 7f96efe00640 1 -- 172.21.15.39:0/4212734033 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=203) ==== 24+0+0 (secure 0 0 0) 0x7f96fc02e050 con 0x56233e7f13b0 2024-09-21T07:47:23.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.808+0000 7f9718c57740 10 client.9648 fetch_fsmap learned FSMap version 203 2024-09-21T07:47:23.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.808+0000 7f9718c57740 1 -- 172.21.15.39:0/4212734033 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=203,monmap=2+}) -- 0x56233e8fc6b0 con 0x56233e7f13b0 2024-09-21T07:47:23.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.808+0000 7f96efe00640 1 -- 172.21.15.39:0/4212734033 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96f4061e10 con 0x56233e7f13b0 2024-09-21T07:47:23.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.808+0000 7f96efe00640 1 -- 172.21.15.39:0/4212734033 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 203) ==== 3522+0+0 (secure 0 0 0) 0x7f96fc031020 con 0x56233e7f13b0 2024-09-21T07:47:23.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.808+0000 7f9718c57740 10 client.9648 fetch_fsmap finished waiting for FSMap version 203 2024-09-21T07:47:23.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.808+0000 7f9718c57740 10 client.9648 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:47:23.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.808+0000 7f9718c57740 20 client.9648 populate_metadata read hostname 'smithi039' 2024-09-21T07:47:23.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.808+0000 7f9718c57740 1 --2- 172.21.15.39:0/4212734033 >> [v2:172.21.15.39:6832/3199732683,v1:172.21.15.39:6833/3199732683] conn(0x56233e7d8e40 0x56233ec18110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:23.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.808+0000 7f9718c57740 4 client.9648 mds_command: new command op to 9483 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:47:23.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.808+0000 7f9718c57740 1 -- 172.21.15.39:0/4212734033 --> [v2:172.21.15.39:6832/3199732683,v1:172.21.15.39:6833/3199732683] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56233eb36520 con 0x56233e7d8e40 2024-09-21T07:47:23.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.809+0000 7f9703e00640 1 --2- 172.21.15.39:0/4212734033 >> [v2:172.21.15.39:6832/3199732683,v1:172.21.15.39:6833/3199732683] conn(0x56233e7d8e40 0x56233ec18110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:23.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.809+0000 7f9703e00640 1 --2- 172.21.15.39:0/4212734033 >> [v2:172.21.15.39:6832/3199732683,v1:172.21.15.39:6833/3199732683] conn(0x56233e7d8e40 0x56233ec18110 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e53ea477d2486e53 server_cookie=49e9d3ea0509520e in_seq=0 out_seq=0 2024-09-21T07:47:23.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.809+0000 7f96efe00640 10 client.9648 ms_handle_connect on v2:172.21.15.39:6832/3199732683 2024-09-21T07:47:23.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.810+0000 7f96efe00640 1 -- 172.21.15.39:0/4212734033 <== mds.1 v2:172.21.15.39:6832/3199732683 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56233eb36520 con 0x56233e7d8e40 2024-09-21T07:47:23.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.810+0000 7f96efe00640 10 client.9648 handle_command_reply: tid=0 2024-09-21T07:47:23.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.810+0000 7f9718c57740 1 client.9648 shutdown 2024-09-21T07:47:23.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.810+0000 7f9718c57740 1 -- 172.21.15.39:0/4212734033 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56233e7f13b0 msgr2=0x56233ec060c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:23.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.810+0000 7f9718c57740 1 --2- 172.21.15.39:0/4212734033 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56233e7f13b0 0x56233ec060c0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f96fc014730 tx=0x7f96fc000ed0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.810+0000 7f9718c57740 1 -- 172.21.15.39:0/4212734033 shutdown_connections 2024-09-21T07:47:23.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.810+0000 7f9718c57740 1 --2- 172.21.15.39:0/4212734033 >> [v2:172.21.15.39:6832/3199732683,v1:172.21.15.39:6833/3199732683] conn(0x56233e7d8e40 0x56233ec18110 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.810+0000 7f9718c57740 1 --2- 172.21.15.39:0/4212734033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233e81a270 0x56233ec148b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.810+0000 7f9718c57740 1 --2- 172.21.15.39:0/4212734033 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56233e7f9590 0x56233ebe10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.810+0000 7f9718c57740 1 --2- 172.21.15.39:0/4212734033 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56233e7f13b0 0x56233ec060c0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.810+0000 7f96efe00640 0 client.9648 ms_handle_reset on v2:172.21.15.39:6832/3199732683 2024-09-21T07:47:23.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.810+0000 7f9718c57740 1 -- 172.21.15.39:0/4212734033 >> 172.21.15.39:0/4212734033 conn(0x56233e80e3c0 msgr2=0x56233ec083d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:23.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.811+0000 7f9718c57740 1 -- 172.21.15.39:0/4212734033 shutdown_connections 2024-09-21T07:47:23.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.811+0000 7f9718c57740 1 -- 172.21.15.39:0/4212734033 wait complete. 2024-09-21T07:47:23.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.811+0000 7f9718c57740 20 client.9648 trim_cache size 0 max 16384 2024-09-21T07:47:23.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.830+0000 7f9718c57740 1 Processor -- start 2024-09-21T07:47:23.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.830+0000 7f9718c57740 1 -- start start 2024-09-21T07:47:23.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.832+0000 7f9718c57740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56233e7f13b0 0x56233ebe0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:23.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.832+0000 7f9718c57740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233e7f9590 0x56233ec289b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:23.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.832+0000 7f9718c57740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56233e81a270 0x56233ebe59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:23.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.832+0000 7f9718c57740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f970c2049b0 con 0x56233e7f13b0 2024-09-21T07:47:23.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.832+0000 7f9718c57740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f970c202ce0 con 0x56233e7f9590 2024-09-21T07:47:23.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.832+0000 7f9718c57740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f970c203330 con 0x56233e81a270 2024-09-21T07:47:23.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.832+0000 7f970aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56233e7f13b0 0x56233ebe0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:23.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.832+0000 7f970aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56233e7f13b0 0x56233ebe0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49674/0 (socket says 172.21.15.39:49674) 2024-09-21T07:47:23.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.832+0000 7f970aa00640 1 -- 172.21.15.39:0/247678768 learned_addr learned my addr 172.21.15.39:0/247678768 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:23.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.832+0000 7f970b400640 1 --2- 172.21.15.39:0/247678768 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56233e81a270 0x56233ebe59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:23.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.832+0000 7f9703e00640 1 --2- 172.21.15.39:0/247678768 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233e7f9590 0x56233ec289b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:23.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.832+0000 7f970aa00640 1 -- 172.21.15.39:0/247678768 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56233e81a270 msgr2=0x56233ebe59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:47:23.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.832+0000 7f970aa00640 1 --2- 172.21.15.39:0/247678768 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56233e81a270 0x56233ebe59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.832+0000 7f970aa00640 1 -- 172.21.15.39:0/247678768 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233e7f9590 msgr2=0x56233ec289b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:47:23.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.832+0000 7f970aa00640 1 --2- 172.21.15.39:0/247678768 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233e7f9590 0x56233ec289b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.832+0000 7f970aa00640 1 -- 172.21.15.39:0/247678768 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56233ebbf860 con 0x56233e7f13b0 2024-09-21T07:47:23.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.833+0000 7f970aa00640 1 --2- 172.21.15.39:0/247678768 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56233e7f13b0 0x56233ebe0e60 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f96fc001eb0 tx=0x7f96fc00ff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ccc86701bf4ddf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:23.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.833+0000 7f96faa00640 1 -- 172.21.15.39:0/247678768 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc036070 con 0x56233e7f13b0 2024-09-21T07:47:23.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.833+0000 7f96faa00640 1 -- 172.21.15.39:0/247678768 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96fc026070 con 0x56233e7f13b0 2024-09-21T07:47:23.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.833+0000 7f96faa00640 1 -- 172.21.15.39:0/247678768 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc021430 con 0x56233e7f13b0 2024-09-21T07:47:23.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.834+0000 7f9718c57740 1 -- 172.21.15.39:0/247678768 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56233e7f13b0 msgr2=0x56233ebe0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:23.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.834+0000 7f9718c57740 1 --2- 172.21.15.39:0/247678768 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56233e7f13b0 0x56233ebe0e60 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f96fc001eb0 tx=0x7f96fc00ff80 comp rx=0 tx=0).stop 2024-09-21T07:47:23.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.834+0000 7f9718c57740 1 -- 172.21.15.39:0/247678768 shutdown_connections 2024-09-21T07:47:23.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.834+0000 7f9718c57740 1 --2- 172.21.15.39:0/247678768 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56233e81a270 0x56233ebe59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.834+0000 7f9718c57740 1 --2- 172.21.15.39:0/247678768 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233e7f9590 0x56233ec289b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.834+0000 7f9718c57740 1 --2- 172.21.15.39:0/247678768 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56233e7f13b0 0x56233ebe0e60 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.834+0000 7f9718c57740 1 -- 172.21.15.39:0/247678768 >> 172.21.15.39:0/247678768 conn(0x56233e80e3c0 msgr2=0x56233ebeab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:23.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.834+0000 7f9718c57740 1 -- 172.21.15.39:0/247678768 shutdown_connections 2024-09-21T07:47:23.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.834+0000 7f9718c57740 1 -- 172.21.15.39:0/247678768 wait complete. 2024-09-21T07:47:23.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.835+0000 7f9718c57740 1 -- start start 2024-09-21T07:47:23.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.835+0000 7f9718c57740 1 Processor -- start 2024-09-21T07:47:23.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.836+0000 7f9718c57740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56233e7f13b0 0x56233ec7ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:23.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.836+0000 7f9718c57740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233e7f9590 0x56233ebe5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:23.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.836+0000 7f9718c57740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56233e81a270 0x56233ebe5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:23.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.836+0000 7f9718c57740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56233ec00cb0 con 0x56233e7f13b0 2024-09-21T07:47:23.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.836+0000 7f9718c57740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56233ebe1670 con 0x56233e7f9590 2024-09-21T07:47:23.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.836+0000 7f9718c57740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x56233ebe4b70 con 0x56233e81a270 2024-09-21T07:47:23.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.837+0000 7f9718c57740 10 client.0 _ll_register_callbacks cb 0x7f96fc00fe20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:47:23.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.837+0000 7f970aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56233e7f13b0 0x56233ec7ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:23.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.837+0000 7f970aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56233e7f13b0 0x56233ec7ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49676/0 (socket says 172.21.15.39:49676) 2024-09-21T07:47:23.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.837+0000 7f970aa00640 1 -- 172.21.15.39:0/2854378137 learned_addr learned my addr 172.21.15.39:0/2854378137 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:23.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.837+0000 7f970b400640 1 --2- 172.21.15.39:0/2854378137 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56233e81a270 0x56233ebe5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:23.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.837+0000 7f9703e00640 1 --2- 172.21.15.39:0/2854378137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233e7f9590 0x56233ebe5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:23.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.837+0000 7f970aa00640 1 -- 172.21.15.39:0/2854378137 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56233e81a270 msgr2=0x56233ebe5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:47:23.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.837+0000 7f970aa00640 1 --2- 172.21.15.39:0/2854378137 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56233e81a270 0x56233ebe5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.837+0000 7f970aa00640 1 -- 172.21.15.39:0/2854378137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233e7f9590 msgr2=0x56233ebe5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:47:23.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.837+0000 7f970aa00640 1 --2- 172.21.15.39:0/2854378137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233e7f9590 0x56233ebe5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.837+0000 7f970aa00640 1 -- 172.21.15.39:0/2854378137 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56233ebffde0 con 0x56233e7f13b0 2024-09-21T07:47:23.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.837+0000 7f9703e00640 1 --2- 172.21.15.39:0/2854378137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233e7f9590 0x56233ebe5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:23.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.837+0000 7f970b400640 1 --2- 172.21.15.39:0/2854378137 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56233e81a270 0x56233ebe5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:23.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.837+0000 7f970aa00640 1 --2- 172.21.15.39:0/2854378137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56233e7f13b0 0x56233ec7ecf0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f96fc008fd0 tx=0x7f96fc021b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c19983358487a50a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:23.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.837+0000 7f96efe00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:47:23.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.838+0000 7f96efe00640 1 -- 172.21.15.39:0/2854378137 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc041070 con 0x56233e7f13b0 2024-09-21T07:47:23.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.838+0000 7f96efe00640 1 -- 172.21.15.39:0/2854378137 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(194..194 src has 1..194) ==== 6932+0+0 (secure 0 0 0) 0x7f96fc0136b0 con 0x56233e7f13b0 2024-09-21T07:47:23.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.838+0000 7f9718c57740 1 -- 172.21.15.39:0/2854378137 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96fc026070 con 0x56233e7f13b0 2024-09-21T07:47:23.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.838+0000 7f96efe00640 1 -- 172.21.15.39:0/2854378137 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=203) ==== 24+0+0 (secure 0 0 0) 0x7f96fc04b050 con 0x56233e7f13b0 2024-09-21T07:47:23.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.838+0000 7f9718c57740 10 client.9654 fetch_fsmap learned FSMap version 203 2024-09-21T07:47:23.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.838+0000 7f9718c57740 1 -- 172.21.15.39:0/2854378137 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=203,monmap=2+}) -- 0x56233ec851b0 con 0x56233e7f13b0 2024-09-21T07:47:23.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.838+0000 7f96efe00640 1 -- 172.21.15.39:0/2854378137 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96fc026070 con 0x56233e7f13b0 2024-09-21T07:47:23.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.838+0000 7f96efe00640 1 -- 172.21.15.39:0/2854378137 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 203) ==== 3522+0+0 (secure 0 0 0) 0x7f96fc04e020 con 0x56233e7f13b0 2024-09-21T07:47:23.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.838+0000 7f9718c57740 10 client.9654 fetch_fsmap finished waiting for FSMap version 203 2024-09-21T07:47:23.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.838+0000 7f9718c57740 10 client.9654 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:47:23.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.838+0000 7f9718c57740 20 client.9654 populate_metadata read hostname 'smithi039' 2024-09-21T07:47:23.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.838+0000 7f9718c57740 1 --2- 172.21.15.39:0/2854378137 >> [v2:172.21.15.39:6832/3199732683,v1:172.21.15.39:6833/3199732683] conn(0x56233ec17f60 0x56233ec88e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:23.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.838+0000 7f9718c57740 4 client.9654 mds_command: new command op to 9483 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:47:23.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.838+0000 7f9718c57740 1 -- 172.21.15.39:0/2854378137 --> [v2:172.21.15.39:6832/3199732683,v1:172.21.15.39:6833/3199732683] -- command(tid 0: {"prefix": "status"}) -- 0x56233ec01b70 con 0x56233ec17f60 2024-09-21T07:47:23.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.839+0000 7f9703e00640 1 --2- 172.21.15.39:0/2854378137 >> [v2:172.21.15.39:6832/3199732683,v1:172.21.15.39:6833/3199732683] conn(0x56233ec17f60 0x56233ec88e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:23.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.839+0000 7f96efe00640 10 client.9654 ms_handle_connect on v2:172.21.15.39:6832/3199732683 2024-09-21T07:47:23.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.839+0000 7f9703e00640 1 --2- 172.21.15.39:0/2854378137 >> [v2:172.21.15.39:6832/3199732683,v1:172.21.15.39:6833/3199732683] conn(0x56233ec17f60 0x56233ec88e30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cb305a6d7a438d10 server_cookie=3724a2182f8a2072 in_seq=0 out_seq=0 2024-09-21T07:47:23.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.839+0000 7f96efe00640 1 -- 172.21.15.39:0/2854378137 <== mds.1 v2:172.21.15.39:6832/3199732683 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x56233ec01b70 con 0x56233ec17f60 2024-09-21T07:47:23.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.839+0000 7f96efe00640 10 client.9654 handle_command_reply: tid=0 2024-09-21T07:47:23.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.839+0000 7f9718c57740 1 client.9654 shutdown 2024-09-21T07:47:23.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.840+0000 7f9718c57740 1 -- 172.21.15.39:0/2854378137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56233e7f13b0 msgr2=0x56233ec7ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:23.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.840+0000 7f9718c57740 1 --2- 172.21.15.39:0/2854378137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56233e7f13b0 0x56233ec7ecf0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f96fc008fd0 tx=0x7f96fc021b70 comp rx=0 tx=0).stop 2024-09-21T07:47:23.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.840+0000 7f9718c57740 1 -- 172.21.15.39:0/2854378137 shutdown_connections 2024-09-21T07:47:23.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.840+0000 7f9718c57740 1 --2- 172.21.15.39:0/2854378137 >> [v2:172.21.15.39:6832/3199732683,v1:172.21.15.39:6833/3199732683] conn(0x56233ec17f60 0x56233ec88e30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.840+0000 7f9718c57740 1 --2- 172.21.15.39:0/2854378137 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56233e81a270 0x56233ebe5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.840+0000 7f9718c57740 1 --2- 172.21.15.39:0/2854378137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56233e7f9590 0x56233ebe5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.840+0000 7f9718c57740 1 --2- 172.21.15.39:0/2854378137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56233e7f13b0 0x56233ec7ecf0 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.840+0000 7f9718c57740 1 -- 172.21.15.39:0/2854378137 >> 172.21.15.39:0/2854378137 conn(0x56233e80e3c0 msgr2=0x56233ec289b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:23.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.840+0000 7f96efe00640 0 client.9654 ms_handle_reset on v2:172.21.15.39:6832/3199732683 2024-09-21T07:47:23.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.840+0000 7f9718c57740 1 -- 172.21.15.39:0/2854378137 shutdown_connections 2024-09-21T07:47:23.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.840+0000 7f9718c57740 1 -- 172.21.15.39:0/2854378137 wait complete. 2024-09-21T07:47:23.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.840+0000 7f9718c57740 20 client.9654 trim_cache size 0 max 16384 2024-09-21T07:47:23.847 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:47:23.847 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:47:23.847 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 1, 2024-09-21T07:47:23.847 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 9483, 2024-09-21T07:47:23.847 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:47:23.847 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:47:23.847 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:47:23.847 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 3.1194556480000002, 2024-09-21T07:47:23.847 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 203, 2024-09-21T07:47:23.847 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 194, 2024-09-21T07:47:23.848 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 194, 2024-09-21T07:47:23.848 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 19.203649247000001 2024-09-21T07:47:23.848 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:47:23.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.841+0000 7f9711c00640 1 -- 172.21.15.39:0/3562677309 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f96e8078bc0 msgr2=0x7f96e807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:23.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.841+0000 7f9711c00640 1 --2- 172.21.15.39:0/3562677309 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f96e8078bc0 0x7f96e807b0d0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f96fc000f60 tx=0x7f96fc009040 comp rx=0 tx=0).stop 2024-09-21T07:47:23.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.841+0000 7f9711c00640 1 -- 172.21.15.39:0/3562677309 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f970c0cf450 msgr2=0x7f970c0b1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:23.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.841+0000 7f9711c00640 1 --2- 172.21.15.39:0/3562677309 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f970c0cf450 0x7f970c0b1610 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f96f400c3a0 tx=0x7f96f400c860 comp rx=0 tx=0).stop 2024-09-21T07:47:23.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.841+0000 7f9711c00640 1 -- 172.21.15.39:0/3562677309 shutdown_connections 2024-09-21T07:47:23.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.841+0000 7f9711c00640 1 --2- 172.21.15.39:0/3562677309 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f96e8078bc0 0x7f96e807b0d0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.841+0000 7f9711c00640 1 --2- 172.21.15.39:0/3562677309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f970c0b2cf0 0x7f970c0b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.841+0000 7f9711c00640 1 --2- 172.21.15.39:0/3562677309 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f970c0cf450 0x7f970c0b1610 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.841+0000 7f9711c00640 1 --2- 172.21.15.39:0/3562677309 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f970c073dd0 0x7f970c13acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.841+0000 7f9711c00640 1 -- 172.21.15.39:0/3562677309 >> 172.21.15.39:0/3562677309 conn(0x7f970c06cfc0 msgr2=0x7f970c15d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:23.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.841+0000 7f9711c00640 1 -- 172.21.15.39:0/3562677309 shutdown_connections 2024-09-21T07:47:23.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:23.841+0000 7f9711c00640 1 -- 172.21.15.39:0/3562677309 wait complete. 2024-09-21T07:47:23.867 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T07:47:24.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.094+0000 7f380d200640 1 -- 172.21.15.39:0/1391386968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38080d1ac0 msgr2=0x7f38080b4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:24.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.094+0000 7f380d200640 1 --2- 172.21.15.39:0/1391386968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38080d1ac0 0x7f38080b4270 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f3800005fd0 tx=0x7f380002ab40 comp rx=0 tx=0).stop 2024-09-21T07:47:24.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.094+0000 7f380d200640 1 -- 172.21.15.39:0/1391386968 shutdown_connections 2024-09-21T07:47:24.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.094+0000 7f380d200640 1 --2- 172.21.15.39:0/1391386968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38080d1ac0 0x7f38080b4270 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.094+0000 7f380d200640 1 -- 172.21.15.39:0/1391386968 >> 172.21.15.39:0/1391386968 conn(0x7f380809d7b0 msgr2=0x7f38080d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:24.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.094+0000 7f380d200640 1 -- 172.21.15.39:0/1391386968 shutdown_connections 2024-09-21T07:47:24.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.094+0000 7f380d200640 1 -- 172.21.15.39:0/1391386968 wait complete. 2024-09-21T07:47:24.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.096+0000 7f380d200640 1 Processor -- start 2024-09-21T07:47:24.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.096+0000 7f380d200640 1 -- start start 2024-09-21T07:47:24.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.098+0000 7f380d200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38080b5a90 0x7f380815ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:24.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.098+0000 7f380d200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f38080d1ac0 0x7f38081f8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:24.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.098+0000 7f380d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38081f8db0 0x7f38081f5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:24.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.098+0000 7f380d200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f38080b47b0 con 0x7f38081f8db0 2024-09-21T07:47:24.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.098+0000 7f380d200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f38080d2280 con 0x7f38080b5a90 2024-09-21T07:47:24.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.098+0000 7f380d200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f38080b4930 con 0x7f38080d1ac0 2024-09-21T07:47:24.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.098+0000 7f3807e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38081f8db0 0x7f38081f5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:24.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.098+0000 7f3807e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38081f8db0 0x7f38081f5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49690/0 (socket says 172.21.15.39:49690) 2024-09-21T07:47:24.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.098+0000 7f3807e00640 1 -- 172.21.15.39:0/3252047421 learned_addr learned my addr 172.21.15.39:0/3252047421 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:24.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.098+0000 7f3806a00640 1 --2- 172.21.15.39:0/3252047421 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f38080d1ac0 0x7f38081f8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:24.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.098+0000 7f3807400640 1 --2- 172.21.15.39:0/3252047421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38080b5a90 0x7f380815ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:24.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.099+0000 7f3807e00640 1 -- 172.21.15.39:0/3252047421 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f38080d1ac0 msgr2=0x7f38081f8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:24.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.099+0000 7f3807e00640 1 --2- 172.21.15.39:0/3252047421 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f38080d1ac0 0x7f38081f8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.099+0000 7f3807e00640 1 -- 172.21.15.39:0/3252047421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38080b5a90 msgr2=0x7f380815ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:24.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.099+0000 7f3807e00640 1 --2- 172.21.15.39:0/3252047421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38080b5a90 0x7f380815ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.099+0000 7f3807e00640 1 -- 172.21.15.39:0/3252047421 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38081f91c0 con 0x7f38081f8db0 2024-09-21T07:47:24.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.099+0000 7f3807e00640 1 --2- 172.21.15.39:0/3252047421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38081f8db0 0x7f38081f5620 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f3800006010 tx=0x7f380002c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:24.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.099+0000 7f37ff400640 1 -- 172.21.15.39:0/3252047421 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3800044070 con 0x7f38081f8db0 2024-09-21T07:47:24.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.099+0000 7f37ff400640 1 -- 172.21.15.39:0/3252047421 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3800004250 con 0x7f38081f8db0 2024-09-21T07:47:24.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.099+0000 7f37ff400640 1 -- 172.21.15.39:0/3252047421 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38000046a0 con 0x7f38081f8db0 2024-09-21T07:47:24.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.099+0000 7f380d200640 1 -- 172.21.15.39:0/3252047421 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38080cd390 con 0x7f38081f8db0 2024-09-21T07:47:24.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.099+0000 7f380d200640 1 -- 172.21.15.39:0/3252047421 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38080cb970 con 0x7f38081f8db0 2024-09-21T07:47:24.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.102+0000 7f37ff400640 1 -- 172.21.15.39:0/3252047421 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3800036400 con 0x7f38081f8db0 2024-09-21T07:47:24.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.104+0000 7f37ff400640 1 --2- 172.21.15.39:0/3252047421 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f37e8078b90 0x7f37e807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:24.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.104+0000 7f37ff400640 1 -- 172.21.15.39:0/3252047421 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 6932+0+0 (secure 0 0 0) 0x7f38000bbad0 con 0x7f38081f8db0 2024-09-21T07:47:24.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.104+0000 7f3807400640 1 --2- 172.21.15.39:0/3252047421 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f37e8078b90 0x7f37e807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:24.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.104+0000 7f38159ff740 1 Processor -- start 2024-09-21T07:47:24.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.104+0000 7f38159ff740 1 -- start start 2024-09-21T07:47:24.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.105+0000 7f3807400640 1 --2- 172.21.15.39:0/3252047421 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f37e8078b90 0x7f37e807b0a0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f37f80016b0 tx=0x7f37f8009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:24.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.106+0000 7f38159ff740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ea88c40270 0x55ea89010b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:24.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.106+0000 7f38159ff740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ea88c173b0 0x55ea890110a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:24.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.106+0000 7f38159ff740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea88c1f590 0x55ea88fe1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:24.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.106+0000 7f38159ff740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ea88f6a430 con 0x55ea88c1f590 2024-09-21T07:47:24.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.106+0000 7f38159ff740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ea88f5a5d0 con 0x55ea88c173b0 2024-09-21T07:47:24.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.106+0000 7f38159ff740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55ea88f49a00 con 0x55ea88c40270 2024-09-21T07:47:24.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.106+0000 7f3807e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea88c1f590 0x55ea88fe1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:24.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.106+0000 7f3807e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea88c1f590 0x55ea88fe1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49692/0 (socket says 172.21.15.39:49692) 2024-09-21T07:47:24.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.106+0000 7f3806a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ea88c173b0 0x55ea890110a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:24.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.106+0000 7f3807e00640 1 -- 172.21.15.39:0/1315306215 learned_addr learned my addr 172.21.15.39:0/1315306215 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:24.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.106+0000 7f3807400640 1 --2- 172.21.15.39:0/1315306215 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ea88c40270 0x55ea89010b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:24.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.106+0000 7f3807e00640 1 -- 172.21.15.39:0/1315306215 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ea88c40270 msgr2=0x55ea89010b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:47:24.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.106+0000 7f3807e00640 1 --2- 172.21.15.39:0/1315306215 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ea88c40270 0x55ea89010b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.106+0000 7f3807e00640 1 -- 172.21.15.39:0/1315306215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ea88c173b0 msgr2=0x55ea890110a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:47:24.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.106+0000 7f3807e00640 1 --2- 172.21.15.39:0/1315306215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ea88c173b0 0x55ea890110a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.106+0000 7f3807e00640 1 -- 172.21.15.39:0/1315306215 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea88f12520 con 0x55ea88c1f590 2024-09-21T07:47:24.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.107+0000 7f3807e00640 1 --2- 172.21.15.39:0/1315306215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea88c1f590 0x55ea88fe1150 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f38000b9a60 tx=0x7f380007fbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6ee9dc07bb6ea92 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:24.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.107+0000 7f37f3e00640 1 -- 172.21.15.39:0/1315306215 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3800082f80 con 0x55ea88c1f590 2024-09-21T07:47:24.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.107+0000 7f37f3e00640 1 -- 172.21.15.39:0/1315306215 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3800083470 con 0x55ea88c1f590 2024-09-21T07:47:24.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.107+0000 7f37f3e00640 1 -- 172.21.15.39:0/1315306215 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3800004d80 con 0x55ea88c1f590 2024-09-21T07:47:24.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.108+0000 7f38159ff740 1 -- 172.21.15.39:0/1315306215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea88c1f590 msgr2=0x55ea88fe1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:24.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.108+0000 7f38159ff740 1 --2- 172.21.15.39:0/1315306215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea88c1f590 0x55ea88fe1150 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f38000b9a60 tx=0x7f380007fbf0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.108+0000 7f38159ff740 1 -- 172.21.15.39:0/1315306215 shutdown_connections 2024-09-21T07:47:24.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.108+0000 7f38159ff740 1 --2- 172.21.15.39:0/1315306215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea88c1f590 0x55ea88fe1150 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.108+0000 7f38159ff740 1 --2- 172.21.15.39:0/1315306215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ea88c173b0 0x55ea890110a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.108+0000 7f38159ff740 1 --2- 172.21.15.39:0/1315306215 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ea88c40270 0x55ea89010b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.108+0000 7f38159ff740 1 -- 172.21.15.39:0/1315306215 >> 172.21.15.39:0/1315306215 conn(0x55ea88c343c0 msgr2=0x55ea88fe6b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:24.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.108+0000 7f38159ff740 1 -- 172.21.15.39:0/1315306215 shutdown_connections 2024-09-21T07:47:24.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.108+0000 7f38159ff740 1 -- 172.21.15.39:0/1315306215 wait complete. 2024-09-21T07:47:24.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.109+0000 7f38159ff740 1 -- start start 2024-09-21T07:47:24.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.109+0000 7f38159ff740 1 Processor -- start 2024-09-21T07:47:24.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.111+0000 7f38159ff740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea88c173b0 0x55ea89009d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:24.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.111+0000 7f38159ff740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ea88c1f590 0x55ea89025f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:24.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.111+0000 7f38159ff740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ea88c40270 0x55ea890071e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:24.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.111+0000 7f38159ff740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ea88ef0330 con 0x55ea88c173b0 2024-09-21T07:47:24.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.111+0000 7f38159ff740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ea88f4d810 con 0x55ea88c40270 2024-09-21T07:47:24.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.111+0000 7f38159ff740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55ea88cba620 con 0x55ea88c1f590 2024-09-21T07:47:24.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.111+0000 7f38159ff740 10 client.0 _ll_register_callbacks cb 0x55ea88ef5420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:47:24.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.111+0000 7f3807400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea88c173b0 0x55ea89009d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:24.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.111+0000 7f3807400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea88c173b0 0x55ea89009d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49694/0 (socket says 172.21.15.39:49694) 2024-09-21T07:47:24.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.111+0000 7f3807400640 1 -- 172.21.15.39:0/2553217017 learned_addr learned my addr 172.21.15.39:0/2553217017 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:24.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.111+0000 7f3806a00640 1 --2- 172.21.15.39:0/2553217017 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ea88c1f590 0x55ea89025f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:24.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.111+0000 7f3807e00640 1 --2- 172.21.15.39:0/2553217017 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ea88c40270 0x55ea890071e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:24.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.111+0000 7f3807400640 1 -- 172.21.15.39:0/2553217017 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ea88c1f590 msgr2=0x55ea89025f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:47:24.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.111+0000 7f3807400640 1 --2- 172.21.15.39:0/2553217017 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ea88c1f590 0x55ea89025f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.111+0000 7f3807400640 1 -- 172.21.15.39:0/2553217017 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ea88c40270 msgr2=0x55ea890071e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:47:24.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.111+0000 7f3807400640 1 --2- 172.21.15.39:0/2553217017 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ea88c40270 0x55ea890071e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.111+0000 7f3807400640 1 -- 172.21.15.39:0/2553217017 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea88cab850 con 0x55ea88c173b0 2024-09-21T07:47:24.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.112+0000 7f3807400640 1 --2- 172.21.15.39:0/2553217017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea88c173b0 0x55ea89009d50 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f37f8014700 tx=0x7f37f8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc7a4da7b13e305 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:24.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.112+0000 7f37f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:47:24.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.112+0000 7f37f0c00640 1 -- 172.21.15.39:0/2553217017 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f8025020 con 0x55ea88c173b0 2024-09-21T07:47:24.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.112+0000 7f37f0c00640 1 -- 172.21.15.39:0/2553217017 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(194..194 src has 1..194) ==== 6932+0+0 (secure 0 0 0) 0x7f37f8021610 con 0x55ea88c173b0 2024-09-21T07:47:24.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.112+0000 7f38159ff740 1 -- 172.21.15.39:0/2553217017 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3800083470 con 0x55ea88c173b0 2024-09-21T07:47:24.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.112+0000 7f37f0c00640 1 -- 172.21.15.39:0/2553217017 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=203) ==== 24+0+0 (secure 0 0 0) 0x7f37f802f050 con 0x55ea88c173b0 2024-09-21T07:47:24.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.112+0000 7f38159ff740 10 client.9669 fetch_fsmap learned FSMap version 203 2024-09-21T07:47:24.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.112+0000 7f38159ff740 1 -- 172.21.15.39:0/2553217017 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=203,monmap=2+}) -- 0x55ea88d226b0 con 0x55ea88c173b0 2024-09-21T07:47:24.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.112+0000 7f37f0c00640 1 -- 172.21.15.39:0/2553217017 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3800083470 con 0x55ea88c173b0 2024-09-21T07:47:24.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.113+0000 7f37f0c00640 1 -- 172.21.15.39:0/2553217017 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 203) ==== 3522+0+0 (secure 0 0 0) 0x7f37f8032020 con 0x55ea88c173b0 2024-09-21T07:47:24.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.113+0000 7f38159ff740 10 client.9669 fetch_fsmap finished waiting for FSMap version 203 2024-09-21T07:47:24.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.113+0000 7f38159ff740 10 client.9669 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:47:24.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.113+0000 7f38159ff740 20 client.9669 populate_metadata read hostname 'smithi039' 2024-09-21T07:47:24.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.113+0000 7f38159ff740 1 --2- 172.21.15.39:0/2553217017 >> [v2:172.21.15.39:6834/3348864735,v1:172.21.15.39:6835/3348864735] conn(0x55ea88bfee40 0x55ea8903b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:24.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.113+0000 7f38159ff740 4 client.9669 mds_command: new command op to 9492 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:47:24.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.113+0000 7f38159ff740 1 -- 172.21.15.39:0/2553217017 --> [v2:172.21.15.39:6834/3348864735,v1:172.21.15.39:6835/3348864735] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea88f5c520 con 0x55ea88bfee40 2024-09-21T07:47:24.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.113+0000 7f3806a00640 1 --2- 172.21.15.39:0/2553217017 >> [v2:172.21.15.39:6834/3348864735,v1:172.21.15.39:6835/3348864735] conn(0x55ea88bfee40 0x55ea8903b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:24.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.113+0000 7f3806a00640 1 --2- 172.21.15.39:0/2553217017 >> [v2:172.21.15.39:6834/3348864735,v1:172.21.15.39:6835/3348864735] conn(0x55ea88bfee40 0x55ea8903b340 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61a4014526105958 server_cookie=60e0d32e8d0f828f in_seq=0 out_seq=0 2024-09-21T07:47:24.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.113+0000 7f37f0c00640 10 client.9669 ms_handle_connect on v2:172.21.15.39:6834/3348864735 2024-09-21T07:47:24.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.114+0000 7f37f0c00640 1 -- 172.21.15.39:0/2553217017 <== mds.0 v2:172.21.15.39:6834/3348864735 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ea88f5c520 con 0x55ea88bfee40 2024-09-21T07:47:24.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.114+0000 7f37f0c00640 10 client.9669 handle_command_reply: tid=0 2024-09-21T07:47:24.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.114+0000 7f38159ff740 1 client.9669 shutdown 2024-09-21T07:47:24.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.115+0000 7f38159ff740 1 -- 172.21.15.39:0/2553217017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea88c173b0 msgr2=0x55ea89009d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:24.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.115+0000 7f38159ff740 1 --2- 172.21.15.39:0/2553217017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea88c173b0 0x55ea89009d50 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f37f8014700 tx=0x7f37f8000ed0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.115+0000 7f38159ff740 1 -- 172.21.15.39:0/2553217017 shutdown_connections 2024-09-21T07:47:24.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.115+0000 7f38159ff740 1 --2- 172.21.15.39:0/2553217017 >> [v2:172.21.15.39:6834/3348864735,v1:172.21.15.39:6835/3348864735] conn(0x55ea88bfee40 0x55ea8903b340 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.115+0000 7f38159ff740 1 --2- 172.21.15.39:0/2553217017 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ea88c40270 0x55ea890071e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.115+0000 7f38159ff740 1 --2- 172.21.15.39:0/2553217017 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ea88c1f590 0x55ea89025f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.115+0000 7f38159ff740 1 --2- 172.21.15.39:0/2553217017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea88c173b0 0x55ea89009d50 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.115+0000 7f37f0c00640 0 client.9669 ms_handle_reset on v2:172.21.15.39:6834/3348864735 2024-09-21T07:47:24.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.115+0000 7f38159ff740 1 -- 172.21.15.39:0/2553217017 >> 172.21.15.39:0/2553217017 conn(0x55ea88c343c0 msgr2=0x55ea88fe25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:24.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.115+0000 7f38159ff740 1 -- 172.21.15.39:0/2553217017 shutdown_connections 2024-09-21T07:47:24.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.115+0000 7f38159ff740 1 -- 172.21.15.39:0/2553217017 wait complete. 2024-09-21T07:47:24.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.115+0000 7f38159ff740 20 client.9669 trim_cache size 0 max 16384 2024-09-21T07:47:24.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.134+0000 7f38159ff740 1 Processor -- start 2024-09-21T07:47:24.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.134+0000 7f38159ff740 1 -- start start 2024-09-21T07:47:24.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.136+0000 7f38159ff740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ea88c173b0 0x55ea89039f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:24.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.136+0000 7f38159ff740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea88c1f590 0x55ea88fdc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:24.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.136+0000 7f38159ff740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ea88c40270 0x55ea8900dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:24.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.136+0000 7f38159ff740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3808204870 con 0x55ea88c1f590 2024-09-21T07:47:24.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.136+0000 7f38159ff740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3808202b40 con 0x55ea88c173b0 2024-09-21T07:47:24.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.136+0000 7f38159ff740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f3808203190 con 0x55ea88c40270 2024-09-21T07:47:24.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.136+0000 7f3806a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea88c1f590 0x55ea88fdc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:24.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.136+0000 7f3806a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea88c1f590 0x55ea88fdc7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49704/0 (socket says 172.21.15.39:49704) 2024-09-21T07:47:24.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.136+0000 7f3806a00640 1 -- 172.21.15.39:0/3866307841 learned_addr learned my addr 172.21.15.39:0/3866307841 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:24.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.136+0000 7f3807400640 1 --2- 172.21.15.39:0/3866307841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ea88c173b0 0x55ea89039f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:24.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.136+0000 7f3807e00640 1 --2- 172.21.15.39:0/3866307841 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ea88c40270 0x55ea8900dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:24.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.136+0000 7f3806a00640 1 -- 172.21.15.39:0/3866307841 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ea88c40270 msgr2=0x55ea8900dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:47:24.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.137+0000 7f3806a00640 1 --2- 172.21.15.39:0/3866307841 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ea88c40270 0x55ea8900dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.137+0000 7f3806a00640 1 -- 172.21.15.39:0/3866307841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ea88c173b0 msgr2=0x55ea89039f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:47:24.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.137+0000 7f3806a00640 1 --2- 172.21.15.39:0/3866307841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ea88c173b0 0x55ea89039f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.137+0000 7f3806a00640 1 -- 172.21.15.39:0/3866307841 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea8902d2f0 con 0x55ea88c1f590 2024-09-21T07:47:24.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.137+0000 7f3806a00640 1 --2- 172.21.15.39:0/3866307841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea88c1f590 0x55ea88fdc7f0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f37f40054a0 tx=0x7f37f4009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2045fdb6a5105ea8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:24.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.137+0000 7f37f3400640 1 -- 172.21.15.39:0/3866307841 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f40166e0 con 0x55ea88c1f590 2024-09-21T07:47:24.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.137+0000 7f37f3400640 1 -- 172.21.15.39:0/3866307841 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f37f4016970 con 0x55ea88c1f590 2024-09-21T07:47:24.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.137+0000 7f37f3400640 1 -- 172.21.15.39:0/3866307841 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f4016dc0 con 0x55ea88c1f590 2024-09-21T07:47:24.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.138+0000 7f38159ff740 1 -- 172.21.15.39:0/3866307841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea88c1f590 msgr2=0x55ea88fdc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:24.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.138+0000 7f38159ff740 1 --2- 172.21.15.39:0/3866307841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea88c1f590 0x55ea88fdc7f0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f37f40054a0 tx=0x7f37f4009040 comp rx=0 tx=0).stop 2024-09-21T07:47:24.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.138+0000 7f38159ff740 1 -- 172.21.15.39:0/3866307841 shutdown_connections 2024-09-21T07:47:24.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.138+0000 7f38159ff740 1 --2- 172.21.15.39:0/3866307841 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ea88c40270 0x55ea8900dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.138+0000 7f38159ff740 1 --2- 172.21.15.39:0/3866307841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea88c1f590 0x55ea88fdc7f0 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.138+0000 7f38159ff740 1 --2- 172.21.15.39:0/3866307841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ea88c173b0 0x55ea89039f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.138+0000 7f38159ff740 1 -- 172.21.15.39:0/3866307841 >> 172.21.15.39:0/3866307841 conn(0x55ea88c343c0 msgr2=0x55ea88fe3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:24.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.138+0000 7f38159ff740 1 -- 172.21.15.39:0/3866307841 shutdown_connections 2024-09-21T07:47:24.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.138+0000 7f38159ff740 1 -- 172.21.15.39:0/3866307841 wait complete. 2024-09-21T07:47:24.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.139+0000 7f38159ff740 1 -- start start 2024-09-21T07:47:24.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.139+0000 7f38159ff740 1 Processor -- start 2024-09-21T07:47:24.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.141+0000 7f38159ff740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea88c173b0 0x55ea89007150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:24.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.141+0000 7f38159ff740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ea88c1f590 0x55ea890a2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:24.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.141+0000 7f38159ff740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ea88c40270 0x55ea8900dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:24.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.141+0000 7f38159ff740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ea8900daf0 con 0x55ea88c173b0 2024-09-21T07:47:24.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.141+0000 7f38159ff740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ea89026470 con 0x55ea88c40270 2024-09-21T07:47:24.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.141+0000 7f38159ff740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55ea8903e770 con 0x55ea88c1f590 2024-09-21T07:47:24.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.141+0000 7f38159ff740 10 client.0 _ll_register_callbacks cb 0x7f37f800fe50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:47:24.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.141+0000 7f3807400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea88c173b0 0x55ea89007150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:24.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.141+0000 7f3807400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea88c173b0 0x55ea89007150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49714/0 (socket says 172.21.15.39:49714) 2024-09-21T07:47:24.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.141+0000 7f3807400640 1 -- 172.21.15.39:0/4040634390 learned_addr learned my addr 172.21.15.39:0/4040634390 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:24.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.141+0000 7f3807e00640 1 --2- 172.21.15.39:0/4040634390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ea88c40270 0x55ea8900dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:24.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.141+0000 7f3806a00640 1 --2- 172.21.15.39:0/4040634390 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ea88c1f590 0x55ea890a2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:24.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.141+0000 7f3807400640 1 -- 172.21.15.39:0/4040634390 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ea88c1f590 msgr2=0x55ea890a2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:47:24.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.141+0000 7f3807400640 1 --2- 172.21.15.39:0/4040634390 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ea88c1f590 0x55ea890a2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.141+0000 7f3807400640 1 -- 172.21.15.39:0/4040634390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ea88c40270 msgr2=0x55ea8900dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:47:24.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.141+0000 7f3807400640 1 --2- 172.21.15.39:0/4040634390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ea88c40270 0x55ea8900dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.141+0000 7f3807400640 1 -- 172.21.15.39:0/4040634390 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea89008520 con 0x55ea88c173b0 2024-09-21T07:47:24.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.142+0000 7f3807400640 1 --2- 172.21.15.39:0/4040634390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea88c173b0 0x55ea89007150 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f37f802abc0 tx=0x7f37f802abf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cbee7c51fc97f24 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:24.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.142+0000 7f37f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:47:24.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.142+0000 7f37f0c00640 1 -- 172.21.15.39:0/4040634390 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f8025070 con 0x55ea88c173b0 2024-09-21T07:47:24.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.142+0000 7f37f0c00640 1 -- 172.21.15.39:0/4040634390 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(194..194 src has 1..194) ==== 6932+0+0 (secure 0 0 0) 0x7f37f8007030 con 0x55ea88c173b0 2024-09-21T07:47:24.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.142+0000 7f38159ff740 1 -- 172.21.15.39:0/4040634390 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37f4016970 con 0x55ea88c173b0 2024-09-21T07:47:24.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.142+0000 7f37f0c00640 1 -- 172.21.15.39:0/4040634390 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=203) ==== 24+0+0 (secure 0 0 0) 0x7f37f8041050 con 0x55ea88c173b0 2024-09-21T07:47:24.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.142+0000 7f38159ff740 10 client.9675 fetch_fsmap learned FSMap version 203 2024-09-21T07:47:24.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.142+0000 7f38159ff740 1 -- 172.21.15.39:0/4040634390 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=203,monmap=2+}) -- 0x55ea89008040 con 0x55ea88c173b0 2024-09-21T07:47:24.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.142+0000 7f37f0c00640 1 -- 172.21.15.39:0/4040634390 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f37f4016970 con 0x55ea88c173b0 2024-09-21T07:47:24.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.143+0000 7f37f0c00640 1 -- 172.21.15.39:0/4040634390 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 203) ==== 3522+0+0 (secure 0 0 0) 0x7f37f8044020 con 0x55ea88c173b0 2024-09-21T07:47:24.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.143+0000 7f38159ff740 10 client.9675 fetch_fsmap finished waiting for FSMap version 203 2024-09-21T07:47:24.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.143+0000 7f38159ff740 10 client.9675 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:47:24.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.143+0000 7f38159ff740 20 client.9675 populate_metadata read hostname 'smithi039' 2024-09-21T07:47:24.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.143+0000 7f38159ff740 1 --2- 172.21.15.39:0/4040634390 >> [v2:172.21.15.39:6834/3348864735,v1:172.21.15.39:6835/3348864735] conn(0x55ea8900ac70 0x55ea890aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:24.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.143+0000 7f38159ff740 4 client.9675 mds_command: new command op to 9492 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:47:24.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.143+0000 7f38159ff740 1 -- 172.21.15.39:0/4040634390 --> [v2:172.21.15.39:6834/3348864735,v1:172.21.15.39:6835/3348864735] -- command(tid 0: {"prefix": "status"}) -- 0x55ea88f57fd0 con 0x55ea8900ac70 2024-09-21T07:47:24.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.143+0000 7f3806a00640 1 --2- 172.21.15.39:0/4040634390 >> [v2:172.21.15.39:6834/3348864735,v1:172.21.15.39:6835/3348864735] conn(0x55ea8900ac70 0x55ea890aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:24.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.144+0000 7f37f0c00640 10 client.9675 ms_handle_connect on v2:172.21.15.39:6834/3348864735 2024-09-21T07:47:24.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.144+0000 7f3806a00640 1 --2- 172.21.15.39:0/4040634390 >> [v2:172.21.15.39:6834/3348864735,v1:172.21.15.39:6835/3348864735] conn(0x55ea8900ac70 0x55ea890aebe0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=685aa8778e6b8c54 server_cookie=db2932dd6ef6500 in_seq=0 out_seq=0 2024-09-21T07:47:24.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.144+0000 7f37f0c00640 1 -- 172.21.15.39:0/4040634390 <== mds.0 v2:172.21.15.39:6834/3348864735 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55ea88f57fd0 con 0x55ea8900ac70 2024-09-21T07:47:24.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.144+0000 7f37f0c00640 10 client.9675 handle_command_reply: tid=0 2024-09-21T07:47:24.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.144+0000 7f38159ff740 1 client.9675 shutdown 2024-09-21T07:47:24.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.144+0000 7f38159ff740 1 -- 172.21.15.39:0/4040634390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea88c173b0 msgr2=0x55ea89007150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:24.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.144+0000 7f38159ff740 1 --2- 172.21.15.39:0/4040634390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea88c173b0 0x55ea89007150 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f37f802abc0 tx=0x7f37f802abf0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.144+0000 7f38159ff740 1 -- 172.21.15.39:0/4040634390 shutdown_connections 2024-09-21T07:47:24.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.144+0000 7f38159ff740 1 --2- 172.21.15.39:0/4040634390 >> [v2:172.21.15.39:6834/3348864735,v1:172.21.15.39:6835/3348864735] conn(0x55ea8900ac70 0x55ea890aebe0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.144+0000 7f38159ff740 1 --2- 172.21.15.39:0/4040634390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ea88c40270 0x55ea8900dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.144+0000 7f38159ff740 1 --2- 172.21.15.39:0/4040634390 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ea88c1f590 0x55ea890a2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.144+0000 7f38159ff740 1 --2- 172.21.15.39:0/4040634390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ea88c173b0 0x55ea89007150 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.144+0000 7f37f0c00640 0 client.9675 ms_handle_reset on v2:172.21.15.39:6834/3348864735 2024-09-21T07:47:24.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.144+0000 7f38159ff740 1 -- 172.21.15.39:0/4040634390 >> 172.21.15.39:0/4040634390 conn(0x55ea88c343c0 msgr2=0x55ea88fe45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:24.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.144+0000 7f38159ff740 1 -- 172.21.15.39:0/4040634390 shutdown_connections 2024-09-21T07:47:24.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.144+0000 7f38159ff740 1 -- 172.21.15.39:0/4040634390 wait complete. 2024-09-21T07:47:24.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.144+0000 7f38159ff740 20 client.9675 trim_cache size 0 max 16384 2024-09-21T07:47:24.150 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:47:24.150 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:47:24.150 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-09-21T07:47:24.150 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 9492, 2024-09-21T07:47:24.150 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:47:24.150 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:47:24.150 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:47:24.150 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 14.533301936999999, 2024-09-21T07:47:24.150 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 203, 2024-09-21T07:47:24.150 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 194, 2024-09-21T07:47:24.150 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 194, 2024-09-21T07:47:24.150 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 19.495838717000002 2024-09-21T07:47:24.150 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:47:24.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.145+0000 7f380d200640 1 -- 172.21.15.39:0/3252047421 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f37e8078b90 msgr2=0x7f37e807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:24.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.145+0000 7f380d200640 1 --2- 172.21.15.39:0/3252047421 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f37e8078b90 0x7f37e807b0a0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f37f80016b0 tx=0x7f37f8009040 comp rx=0 tx=0).stop 2024-09-21T07:47:24.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.145+0000 7f380d200640 1 -- 172.21.15.39:0/3252047421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38081f8db0 msgr2=0x7f38081f5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:24.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.145+0000 7f380d200640 1 --2- 172.21.15.39:0/3252047421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38081f8db0 0x7f38081f5620 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f3800006010 tx=0x7f380002c8b0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.145+0000 7f380d200640 1 -- 172.21.15.39:0/3252047421 shutdown_connections 2024-09-21T07:47:24.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.145+0000 7f380d200640 1 --2- 172.21.15.39:0/3252047421 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f37e8078b90 0x7f37e807b0a0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.145+0000 7f380d200640 1 --2- 172.21.15.39:0/3252047421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38081f8db0 0x7f38081f5620 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.145+0000 7f380d200640 1 --2- 172.21.15.39:0/3252047421 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f38080d1ac0 0x7f38081f8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.145+0000 7f380d200640 1 --2- 172.21.15.39:0/3252047421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38080b5a90 0x7f380815ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.145+0000 7f380d200640 1 -- 172.21.15.39:0/3252047421 >> 172.21.15.39:0/3252047421 conn(0x7f380809d7b0 msgr2=0x7f38080b3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:24.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.145+0000 7f380d200640 1 -- 172.21.15.39:0/3252047421 shutdown_connections 2024-09-21T07:47:24.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24.145+0000 7f380d200640 1 -- 172.21.15.39:0/3252047421 wait complete. 2024-09-21T07:47:24.168 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'mkdir -p {d0,d1}/empty') 2024-09-21T07:47:24.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:47:24.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:47:24.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:24.265 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d0) 2024-09-21T07:47:24.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:24.304 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1) 2024-09-21T07:47:24.381 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-21T07:47:24.669 INFO:teuthology.orchestra.run.smithi039.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"9","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":2,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T07:47:24.669 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 2, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "9" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T07:47:24.669 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d0', 0), ('/d1', 1)] 2024-09-21T07:47:26.671 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-21T07:47:26.927 INFO:teuthology.orchestra.run.smithi039.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T07:47:26.928 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T07:47:26.928 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d0', 0), ('/d1', 1)] 2024-09-21T07:47:28.929 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-21T07:47:29.185 INFO:teuthology.orchestra.run.smithi039.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T07:47:29.186 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T07:47:29.186 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d0', 0), ('/d1', 1)] 2024-09-21T07:47:31.187 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-21T07:47:31.443 INFO:teuthology.orchestra.run.smithi039.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"14","version":"14","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d0","dirfrag":"0x10000000000","snapid_first":2,"projected_version":"3","version":"3","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1","dirfrag":"0x10000000002","snapid_first":2,"projected_version":"3","version":"3","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-09-21T07:47:31.444 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000000", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/d0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "3", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "3" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000002", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "3", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "3" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-21T07:47:31.444 INFO:tasks.cephfs.cephfs_test_case:[('/d0', 0), ('/d1', 1)] =?= [('/d0', 0), ('/d1', 1)] 2024-09-21T07:47:31.444 DEBUG:teuthology.orchestra.run.smithi039:> stdin-killer -- python3 -c ' 2024-09-21T07:47:31.445 DEBUG:teuthology.orchestra.run.smithi039:> import os 2024-09-21T07:47:31.445 DEBUG:teuthology.orchestra.run.smithi039:> open(os.path.join("/home/ubuntu/cephtest/mnt.0", "d0/file1"), '"'"'w'"'"').write("asdf") 2024-09-21T07:47:31.445 DEBUG:teuthology.orchestra.run.smithi039:> open(os.path.join("/home/ubuntu/cephtest/mnt.0", "d0/file2"), '"'"'w'"'"').write("asdf") 2024-09-21T07:47:31.445 DEBUG:teuthology.orchestra.run.smithi039:> ' 2024-09-21T07:47:31.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:47:31.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:31.565 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln d0/file1 d1/file1) 2024-09-21T07:47:32.747 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:47:32.752 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:47:34.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:34.668 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln d0/file2 d1/file2) 2024-09-21T07:47:34.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:34.717 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d1/.snap/s1) 2024-09-21T07:47:34.782 DEBUG:teuthology.orchestra.run.smithi039:> stdin-killer -- python3 -c ' 2024-09-21T07:47:34.782 DEBUG:teuthology.orchestra.run.smithi039:> import os 2024-09-21T07:47:34.783 DEBUG:teuthology.orchestra.run.smithi039:> open(os.path.join("/home/ubuntu/cephtest/mnt.0", "d0/file1"), '"'"'w'"'"').write("qwer") 2024-09-21T07:47:34.783 DEBUG:teuthology.orchestra.run.smithi039:> ' 2024-09-21T07:47:34.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:47:34.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:34.906 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec grep asdf d1/.snap/s1/file1) 2024-09-21T07:47:34.990 INFO:teuthology.orchestra.run.smithi039.stdout:asdf 2024-09-21T07:47:34.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:34.991 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f d0/file2) 2024-09-21T07:47:35.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:35.072 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec grep asdf d1/.snap/s1/file2) 2024-09-21T07:47:35.164 INFO:teuthology.orchestra.run.smithi039.stdout:asdf 2024-09-21T07:47:35.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:35.166 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f d1/file2) 2024-09-21T07:47:35.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:35.205 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec grep asdf d1/.snap/s1/file2) 2024-09-21T07:47:35.280 INFO:teuthology.orchestra.run.smithi039.stdout:asdf 2024-09-21T07:47:35.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:35.281 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir d1/.snap/s1) 2024-09-21T07:47:35.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:35.364 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -rf d0 d1) 2024-09-21T07:47:35.490 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:47:35.491 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:47:35.577 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:47:35.682 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:47:35.683 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:47:35.683 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:47:35.683 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:47:35.718 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:47:35.718 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:47:35.794 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:47:36.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.062+0000 7f8b38c00640 1 -- 172.21.15.39:0/1643238235 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b340b5a90 msgr2=0x7f8b340d1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:36.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.062+0000 7f8b38c00640 1 --2- 172.21.15.39:0/1643238235 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b340b5a90 0x7f8b340d1ac0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f8b340d5560 tx=0x7f8b2402a540 comp rx=0 tx=0).stop 2024-09-21T07:47:36.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.062+0000 7f8b38c00640 1 -- 172.21.15.39:0/1643238235 shutdown_connections 2024-09-21T07:47:36.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.062+0000 7f8b38c00640 1 --2- 172.21.15.39:0/1643238235 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b340cef70 0x7f8b340cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:36.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.062+0000 7f8b38c00640 1 --2- 172.21.15.39:0/1643238235 >> v2:172.21.15.148:3300/0 conn(0x7f8b340d0e70 0x7f8b340d1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:36.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.062+0000 7f8b38c00640 1 --2- 172.21.15.39:0/1643238235 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b340b5a90 0x7f8b340d1ac0 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:36.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.062+0000 7f8b38c00640 1 -- 172.21.15.39:0/1643238235 >> 172.21.15.39:0/1643238235 conn(0x7f8b3409d7b0 msgr2=0x7f8b340d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:36.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.062+0000 7f8b38c00640 1 -- 172.21.15.39:0/1643238235 shutdown_connections 2024-09-21T07:47:36.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.062+0000 7f8b38c00640 1 -- 172.21.15.39:0/1643238235 wait complete. 2024-09-21T07:47:36.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.064+0000 7f8b38c00640 1 Processor -- start 2024-09-21T07:47:36.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.064+0000 7f8b38c00640 1 -- start start 2024-09-21T07:47:36.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.066+0000 7f8b38c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b340b5a90 0x7f8b340cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:36.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.066+0000 7f8b38c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b340cef70 0x7f8b341588b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:36.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.066+0000 7f8b38c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b340d0e70 0x7f8b3415b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:36.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.066+0000 7f8b38c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8b340cf8c0 con 0x7f8b340cef70 2024-09-21T07:47:36.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.066+0000 7f8b38c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8b340d2240 con 0x7f8b340b5a90 2024-09-21T07:47:36.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.066+0000 7f8b38c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f8b340cfa40 con 0x7f8b340d0e70 2024-09-21T07:47:36.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.066+0000 7f8b2be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b340cef70 0x7f8b341588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:36.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.066+0000 7f8b33400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b340b5a90 0x7f8b340cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:36.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.066+0000 7f8b2be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b340cef70 0x7f8b341588b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34160/0 (socket says 172.21.15.39:34160) 2024-09-21T07:47:36.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.066+0000 7f8b2be00640 1 -- 172.21.15.39:0/43927497 learned_addr learned my addr 172.21.15.39:0/43927497 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:36.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.066+0000 7f8b33e00640 1 --2- 172.21.15.39:0/43927497 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b340d0e70 0x7f8b3415b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:36.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.066+0000 7f8b2be00640 1 -- 172.21.15.39:0/43927497 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b340d0e70 msgr2=0x7f8b3415b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:36.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.066+0000 7f8b2be00640 1 --2- 172.21.15.39:0/43927497 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b340d0e70 0x7f8b3415b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:36.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.066+0000 7f8b2be00640 1 -- 172.21.15.39:0/43927497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b340b5a90 msgr2=0x7f8b340cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:36.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.067+0000 7f8b2be00640 1 --2- 172.21.15.39:0/43927497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b340b5a90 0x7f8b340cdd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:36.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.067+0000 7f8b2be00640 1 -- 172.21.15.39:0/43927497 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b341f5480 con 0x7f8b340cef70 2024-09-21T07:47:36.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.067+0000 7f8b2be00640 1 --2- 172.21.15.39:0/43927497 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b340cef70 0x7f8b341588b0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c00e7c0 tx=0x7f8b1c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:36.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.067+0000 7f8b2b400640 1 -- 172.21.15.39:0/43927497 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c00ccc0 con 0x7f8b340cef70 2024-09-21T07:47:36.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.067+0000 7f8b2b400640 1 -- 172.21.15.39:0/43927497 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b1c010430 con 0x7f8b340cef70 2024-09-21T07:47:36.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.067+0000 7f8b2b400640 1 -- 172.21.15.39:0/43927497 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c010860 con 0x7f8b340cef70 2024-09-21T07:47:36.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.067+0000 7f8b38c00640 1 -- 172.21.15.39:0/43927497 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b341f5700 con 0x7f8b340cef70 2024-09-21T07:47:36.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.067+0000 7f8b38c00640 1 -- 172.21.15.39:0/43927497 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b341f5cc0 con 0x7f8b340cef70 2024-09-21T07:47:36.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.068+0000 7f8b38c00640 1 -- 172.21.15.39:0/43927497 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b340cf380 con 0x7f8b340cef70 2024-09-21T07:47:36.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.069+0000 7f8b2b400640 1 -- 172.21.15.39:0/43927497 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8b1c010a00 con 0x7f8b340cef70 2024-09-21T07:47:36.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.070+0000 7f8b2b400640 1 --2- 172.21.15.39:0/43927497 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8b10078b90 0x7f8b1007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:36.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.070+0000 7f8b2b400640 1 -- 172.21.15.39:0/43927497 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 6932+0+0 (secure 0 0 0) 0x7f8b1c09bb90 con 0x7f8b340cef70 2024-09-21T07:47:36.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.071+0000 7f8b33400640 1 --2- 172.21.15.39:0/43927497 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8b10078b90 0x7f8b1007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:36.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.072+0000 7f8b33400640 1 --2- 172.21.15.39:0/43927497 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8b10078b90 0x7f8b1007b0a0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f8b24034000 tx=0x7f8b24004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:36.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.075+0000 7f8b2b400640 1 -- 172.21.15.39:0/43927497 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8b341f5480 con 0x7f8b340cef70 2024-09-21T07:47:36.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.284+0000 7f8b38c00640 1 -- 172.21.15.39:0/43927497 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8b3406e7c0 con 0x7f8b340cef70 2024-09-21T07:47:36.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.284+0000 7f8b2b400640 1 -- 172.21.15.39:0/43927497 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v194) ==== 80+0+0 (secure 0 0 0) 0x7f8b1c063410 con 0x7f8b340cef70 2024-09-21T07:47:36.287 INFO:teuthology.orchestra.run.smithi039.stderr:listed 0 entries 2024-09-21T07:47:36.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.286+0000 7f8b38c00640 1 -- 172.21.15.39:0/43927497 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8b10078b90 msgr2=0x7f8b1007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:36.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.287+0000 7f8b38c00640 1 --2- 172.21.15.39:0/43927497 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8b10078b90 0x7f8b1007b0a0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f8b24034000 tx=0x7f8b24004730 comp rx=0 tx=0).stop 2024-09-21T07:47:36.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.287+0000 7f8b38c00640 1 -- 172.21.15.39:0/43927497 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b340cef70 msgr2=0x7f8b341588b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:36.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.287+0000 7f8b38c00640 1 --2- 172.21.15.39:0/43927497 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b340cef70 0x7f8b341588b0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c00e7c0 tx=0x7f8b1c00ec80 comp rx=0 tx=0).stop 2024-09-21T07:47:36.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.287+0000 7f8b38c00640 1 -- 172.21.15.39:0/43927497 shutdown_connections 2024-09-21T07:47:36.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.287+0000 7f8b38c00640 1 --2- 172.21.15.39:0/43927497 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8b10078b90 0x7f8b1007b0a0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:36.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.287+0000 7f8b38c00640 1 --2- 172.21.15.39:0/43927497 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8b340d0e70 0x7f8b3415b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:36.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.287+0000 7f8b38c00640 1 --2- 172.21.15.39:0/43927497 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b340cef70 0x7f8b341588b0 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:36.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.287+0000 7f8b38c00640 1 --2- 172.21.15.39:0/43927497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b340b5a90 0x7f8b340cdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:36.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.287+0000 7f8b38c00640 1 -- 172.21.15.39:0/43927497 >> 172.21.15.39:0/43927497 conn(0x7f8b3409d7b0 msgr2=0x7f8b3406dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:36.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.287+0000 7f8b38c00640 1 -- 172.21.15.39:0/43927497 shutdown_connections 2024-09-21T07:47:36.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.287+0000 7f8b38c00640 1 -- 172.21.15.39:0/43927497 wait complete. 2024-09-21T07:47:36.314 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T07:47:36.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:36.314 DEBUG:teuthology.orchestra.run.smithi039:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T07:47:36.417 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T07:47:36.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:36.417 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:47:36.465 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:47:36.465 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:47:36.465 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:47:36.504 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:47:36.504 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:47:36.505 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:47:36.505 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:47:36.541 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:47:36.541 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:47:36.541 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:47:36.541 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:47:36.580 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:47:36.580 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:47:36.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.809+0000 7f6dcde00640 1 -- 172.21.15.39:0/657653836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6dc80d1580 msgr2=0x7f6dc80d1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:36.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.809+0000 7f6dcde00640 1 --2- 172.21.15.39:0/657653836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6dc80d1580 0x7f6dc80d1ac0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f6db0009830 tx=0x7f6db002a540 comp rx=0 tx=0).stop 2024-09-21T07:47:36.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.809+0000 7f6dcde00640 1 -- 172.21.15.39:0/657653836 shutdown_connections 2024-09-21T07:47:36.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.809+0000 7f6dcde00640 1 --2- 172.21.15.39:0/657653836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6dc80d1580 0x7f6dc80d1ac0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:36.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.809+0000 7f6dcde00640 1 -- 172.21.15.39:0/657653836 >> 172.21.15.39:0/657653836 conn(0x7f6dc809d7b0 msgr2=0x7f6dc80d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:36.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.809+0000 7f6dcde00640 1 -- 172.21.15.39:0/657653836 shutdown_connections 2024-09-21T07:47:36.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.809+0000 7f6dcde00640 1 -- 172.21.15.39:0/657653836 wait complete. 2024-09-21T07:47:36.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.811+0000 7f6dcde00640 1 Processor -- start 2024-09-21T07:47:36.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.812+0000 7f6dcde00640 1 -- start start 2024-09-21T07:47:36.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.813+0000 7f6dcde00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6dc80b3b60 0x7f6dc81eabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:36.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.813+0000 7f6dcde00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6dc80b5a90 0x7f6dc81513c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:36.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.813+0000 7f6dcde00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6dc81543a0 0x7f6dc81f8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:36.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.813+0000 7f6dcde00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc80b4800 con 0x7f6dc80b5a90 2024-09-21T07:47:36.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.813+0000 7f6dcde00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc80b4680 con 0x7f6dc81543a0 2024-09-21T07:47:36.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.813+0000 7f6dcde00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f6dc80b4980 con 0x7f6dc80b3b60 2024-09-21T07:47:36.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.813+0000 7f6dbfe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6dc80b5a90 0x7f6dc81513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:36.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.813+0000 7f6dbfe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6dc80b5a90 0x7f6dc81513c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34164/0 (socket says 172.21.15.39:34164) 2024-09-21T07:47:36.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.813+0000 7f6dbfe00640 1 -- 172.21.15.39:0/4054393765 learned_addr learned my addr 172.21.15.39:0/4054393765 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:36.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.813+0000 7f6dcca00640 1 --2- 172.21.15.39:0/4054393765 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6dc80b3b60 0x7f6dc81eabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:36.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.813+0000 7f6dcd400640 1 --2- 172.21.15.39:0/4054393765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6dc81543a0 0x7f6dc81f8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:36.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.814+0000 7f6dbfe00640 1 -- 172.21.15.39:0/4054393765 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6dc80b3b60 msgr2=0x7f6dc81eabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:36.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.814+0000 7f6dbfe00640 1 --2- 172.21.15.39:0/4054393765 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6dc80b3b60 0x7f6dc81eabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:36.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.814+0000 7f6dbfe00640 1 -- 172.21.15.39:0/4054393765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6dc81543a0 msgr2=0x7f6dc81f8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:36.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.814+0000 7f6dbfe00640 1 --2- 172.21.15.39:0/4054393765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6dc81543a0 0x7f6dc81f8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:36.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.814+0000 7f6dbfe00640 1 -- 172.21.15.39:0/4054393765 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc81f9410 con 0x7f6dc80b5a90 2024-09-21T07:47:36.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.814+0000 7f6dbfe00640 1 --2- 172.21.15.39:0/4054393765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6dc80b5a90 0x7f6dc81513c0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f6db002ab30 tx=0x7f6db0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:36.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.814+0000 7f6dc5600640 1 -- 172.21.15.39:0/4054393765 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db0005c90 con 0x7f6dc80b5a90 2024-09-21T07:47:36.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.814+0000 7f6dc5600640 1 -- 172.21.15.39:0/4054393765 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6db0005020 con 0x7f6dc80b5a90 2024-09-21T07:47:36.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.814+0000 7f6dcde00640 1 -- 172.21.15.39:0/4054393765 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dc81f96f0 con 0x7f6dc80b5a90 2024-09-21T07:47:36.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.815+0000 7f6dc5600640 1 -- 172.21.15.39:0/4054393765 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db0005450 con 0x7f6dc80b5a90 2024-09-21T07:47:36.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.815+0000 7f6dcde00640 1 -- 172.21.15.39:0/4054393765 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dc8150bc0 con 0x7f6dc80b5a90 2024-09-21T07:47:36.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.816+0000 7f6dcde00640 1 -- 172.21.15.39:0/4054393765 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dc80d1ac0 con 0x7f6dc80b5a90 2024-09-21T07:47:36.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.817+0000 7f6dc5600640 1 -- 172.21.15.39:0/4054393765 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6db00317c0 con 0x7f6dc80b5a90 2024-09-21T07:47:36.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.818+0000 7f6dc5600640 1 --2- 172.21.15.39:0/4054393765 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6da4078b90 0x7f6da407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:36.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.818+0000 7f6dc5600640 1 -- 172.21.15.39:0/4054393765 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 6932+0+0 (secure 0 0 0) 0x7f6db00bc160 con 0x7f6dc80b5a90 2024-09-21T07:47:36.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.819+0000 7f6dcca00640 1 --2- 172.21.15.39:0/4054393765 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6da4078b90 0x7f6da407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:36.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.822+0000 7f6dcca00640 1 --2- 172.21.15.39:0/4054393765 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6da4078b90 0x7f6da407b0a0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f6db8007fd0 tx=0x7f6db8009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:36.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:36.823+0000 7f6dc5600640 1 -- 172.21.15.39:0/4054393765 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6db003f030 con 0x7f6dc80b5a90 2024-09-21T07:47:37.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.060+0000 7f6dcde00640 1 -- 172.21.15.39:0/4054393765 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6dc80d2e60 con 0x7f6dc80b5a90 2024-09-21T07:47:37.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.061+0000 7f6dc5600640 1 -- 172.21.15.39:0/4054393765 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 204 v204) ==== 95+0+5250 (secure 0 0 0) 0x7f6db0083a70 con 0x7f6dc80b5a90 2024-09-21T07:47:37.063 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:47:37.063 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":204,"btime":"2024-09-21T07:47:29:886149+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7715,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3057062733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3057062733},{"type":"v1","addr":"172.21.15.148:6836","nonce":3057062733}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":194},{"gid":8689,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/8652512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":8652512},{"type":"v1","addr":"172.21.15.148:6837","nonce":8652512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":194}],"filesystems":[{"mdsmap":{"epoch":204,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:47:09.140224+0000","modified":"2024-09-21T07:47:29.653225+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":9492,"mds_1":9483},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9483":{"gid":9483,"name":"a","rank":1,"incarnation":202,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6833/3199732683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3199732683},{"type":"v1","addr":"172.21.15.39:6833","nonce":3199732683}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_9492":{"gid":9492,"name":"c","rank":0,"incarnation":195,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/3348864735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3348864735},{"type":"v1","addr":"172.21.15.39:6835","nonce":3348864735}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9492,"qdb_cluster":[9492,9483]},"id":10}]} 2024-09-21T07:47:37.064 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 204 2024-09-21T07:47:37.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.063+0000 7f6dcde00640 1 -- 172.21.15.39:0/4054393765 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6da4078b90 msgr2=0x7f6da407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:37.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.063+0000 7f6dcde00640 1 --2- 172.21.15.39:0/4054393765 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6da4078b90 0x7f6da407b0a0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f6db8007fd0 tx=0x7f6db8009040 comp rx=0 tx=0).stop 2024-09-21T07:47:37.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.063+0000 7f6dcde00640 1 -- 172.21.15.39:0/4054393765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6dc80b5a90 msgr2=0x7f6dc81513c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:37.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.063+0000 7f6dcde00640 1 --2- 172.21.15.39:0/4054393765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6dc80b5a90 0x7f6dc81513c0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f6db002ab30 tx=0x7f6db0003c10 comp rx=0 tx=0).stop 2024-09-21T07:47:37.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.064+0000 7f6dcde00640 1 -- 172.21.15.39:0/4054393765 shutdown_connections 2024-09-21T07:47:37.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.064+0000 7f6dcde00640 1 --2- 172.21.15.39:0/4054393765 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6da4078b90 0x7f6da407b0a0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:37.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.064+0000 7f6dcde00640 1 --2- 172.21.15.39:0/4054393765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6dc81543a0 0x7f6dc81f8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:37.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.064+0000 7f6dcde00640 1 --2- 172.21.15.39:0/4054393765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6dc80b5a90 0x7f6dc81513c0 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:37.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.064+0000 7f6dcde00640 1 --2- 172.21.15.39:0/4054393765 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6dc80b3b60 0x7f6dc81eabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:37.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.064+0000 7f6dcde00640 1 -- 172.21.15.39:0/4054393765 >> 172.21.15.39:0/4054393765 conn(0x7f6dc809d7b0 msgr2=0x7f6dc80b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:37.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.064+0000 7f6dcde00640 1 -- 172.21.15.39:0/4054393765 shutdown_connections 2024-09-21T07:47:37.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.064+0000 7f6dcde00640 1 -- 172.21.15.39:0/4054393765 wait complete. 2024-09-21T07:47:37.090 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:47:37.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.314+0000 7f5693e00640 1 -- 172.21.15.39:0/904082510 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5694160250 msgr2=0x7f569415d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:37.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.314+0000 7f5693e00640 1 --2- 172.21.15.39:0/904082510 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5694160250 0x7f569415d570 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5684009830 tx=0x7f568402a510 comp rx=0 tx=0).stop 2024-09-21T07:47:37.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.314+0000 7f5693e00640 1 -- 172.21.15.39:0/904082510 shutdown_connections 2024-09-21T07:47:37.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.314+0000 7f5693e00640 1 --2- 172.21.15.39:0/904082510 >> v2:172.21.15.148:3300/0 conn(0x7f56940d0390 0x7f56940d07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:37.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.314+0000 7f5693e00640 1 --2- 172.21.15.39:0/904082510 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5694160250 0x7f569415d570 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:37.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.314+0000 7f5693e00640 1 -- 172.21.15.39:0/904082510 >> 172.21.15.39:0/904082510 conn(0x7f569406cfc0 msgr2=0x7f569406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:37.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.315+0000 7f5693e00640 1 -- 172.21.15.39:0/904082510 shutdown_connections 2024-09-21T07:47:37.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.315+0000 7f5693e00640 1 -- 172.21.15.39:0/904082510 wait complete. 2024-09-21T07:47:37.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.317+0000 7f5693e00640 1 Processor -- start 2024-09-21T07:47:37.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.317+0000 7f5693e00640 1 -- start start 2024-09-21T07:47:37.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.318+0000 7f5693e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56940d0390 0x7f56941e6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:37.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.318+0000 7f5693e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5694160250 0x7f56941f9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:37.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.318+0000 7f5693e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5694160660 0x7f56941f2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:37.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.318+0000 7f5693e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f56940ccdc0 con 0x7f5694160660 2024-09-21T07:47:37.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.318+0000 7f5693e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f56940ccc40 con 0x7f56940d0390 2024-09-21T07:47:37.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.318+0000 7f5693e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f56940ccf40 con 0x7f5694160250 2024-09-21T07:47:37.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.319+0000 7f5693400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5694160660 0x7f56941f2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:37.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.319+0000 7f5693400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5694160660 0x7f56941f2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34170/0 (socket says 172.21.15.39:34170) 2024-09-21T07:47:37.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.319+0000 7f5693400640 1 -- 172.21.15.39:0/4182838168 learned_addr learned my addr 172.21.15.39:0/4182838168 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:37.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.319+0000 7f5692a00640 1 --2- 172.21.15.39:0/4182838168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56940d0390 0x7f56941e6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:37.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.319+0000 7f568a000640 1 --2- 172.21.15.39:0/4182838168 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5694160250 0x7f56941f9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:37.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.319+0000 7f5693400640 1 -- 172.21.15.39:0/4182838168 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5694160250 msgr2=0x7f56941f9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:37.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.319+0000 7f5693400640 1 --2- 172.21.15.39:0/4182838168 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5694160250 0x7f56941f9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:37.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.319+0000 7f5693400640 1 -- 172.21.15.39:0/4182838168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56940d0390 msgr2=0x7f56941e6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:37.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.319+0000 7f5693400640 1 --2- 172.21.15.39:0/4182838168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56940d0390 0x7f56941e6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:37.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.319+0000 7f5693400640 1 -- 172.21.15.39:0/4182838168 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56941e65a0 con 0x7f5694160660 2024-09-21T07:47:37.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.320+0000 7f5693400640 1 --2- 172.21.15.39:0/4182838168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5694160660 0x7f56941f2bc0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f568c00efc0 tx=0x7f568c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:37.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.320+0000 7f568be00640 1 -- 172.21.15.39:0/4182838168 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568c01b070 con 0x7f5694160660 2024-09-21T07:47:37.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.320+0000 7f568be00640 1 -- 172.21.15.39:0/4182838168 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f568c003b80 con 0x7f5694160660 2024-09-21T07:47:37.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.320+0000 7f568be00640 1 -- 172.21.15.39:0/4182838168 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568c004030 con 0x7f5694160660 2024-09-21T07:47:37.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.320+0000 7f5693e00640 1 -- 172.21.15.39:0/4182838168 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56941f6a90 con 0x7f5694160660 2024-09-21T07:47:37.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.320+0000 7f5693e00640 1 -- 172.21.15.39:0/4182838168 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56940b0de0 con 0x7f5694160660 2024-09-21T07:47:37.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.321+0000 7f5693e00640 1 -- 172.21.15.39:0/4182838168 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56941f6800 con 0x7f5694160660 2024-09-21T07:47:37.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.322+0000 7f568be00640 1 -- 172.21.15.39:0/4182838168 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f568c01e070 con 0x7f5694160660 2024-09-21T07:47:37.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.323+0000 7f568be00640 1 --2- 172.21.15.39:0/4182838168 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5670078b90 0x7f567007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:37.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.323+0000 7f568be00640 1 -- 172.21.15.39:0/4182838168 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 6932+0+0 (secure 0 0 0) 0x7f568c09b960 con 0x7f5694160660 2024-09-21T07:47:37.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.324+0000 7f5692a00640 1 --2- 172.21.15.39:0/4182838168 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5670078b90 0x7f567007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:37.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.325+0000 7f5692a00640 1 --2- 172.21.15.39:0/4182838168 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5670078b90 0x7f567007b0a0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f5684034000 tx=0x7f5684004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:37.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.328+0000 7f568be00640 1 -- 172.21.15.39:0/4182838168 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f568c0631e0 con 0x7f5694160660 2024-09-21T07:47:37.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.562+0000 7f5693e00640 1 -- 172.21.15.39:0/4182838168 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f569406a3b0 con 0x7f5694160660 2024-09-21T07:47:37.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.563+0000 7f568be00640 1 -- 172.21.15.39:0/4182838168 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 204 v204) ==== 95+0+5250 (secure 0 0 0) 0x7f569406a3b0 con 0x7f5694160660 2024-09-21T07:47:37.565 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:47:37.565 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":204,"btime":"2024-09-21T07:47:29:886149+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7715,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3057062733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3057062733},{"type":"v1","addr":"172.21.15.148:6836","nonce":3057062733}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":194},{"gid":8689,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/8652512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":8652512},{"type":"v1","addr":"172.21.15.148:6837","nonce":8652512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":194}],"filesystems":[{"mdsmap":{"epoch":204,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:47:09.140224+0000","modified":"2024-09-21T07:47:29.653225+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":9492,"mds_1":9483},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9483":{"gid":9483,"name":"a","rank":1,"incarnation":202,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6833/3199732683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3199732683},{"type":"v1","addr":"172.21.15.39:6833","nonce":3199732683}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_9492":{"gid":9492,"name":"c","rank":0,"incarnation":195,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/3348864735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3348864735},{"type":"v1","addr":"172.21.15.39:6835","nonce":3348864735}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9492,"qdb_cluster":[9492,9483]},"id":10}]} 2024-09-21T07:47:37.566 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 204 2024-09-21T07:47:37.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.566+0000 7f5693e00640 1 -- 172.21.15.39:0/4182838168 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5670078b90 msgr2=0x7f567007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:37.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.566+0000 7f5693e00640 1 --2- 172.21.15.39:0/4182838168 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5670078b90 0x7f567007b0a0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f5684034000 tx=0x7f5684004730 comp rx=0 tx=0).stop 2024-09-21T07:47:37.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.566+0000 7f5693e00640 1 -- 172.21.15.39:0/4182838168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5694160660 msgr2=0x7f56941f2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:37.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.566+0000 7f5693e00640 1 --2- 172.21.15.39:0/4182838168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5694160660 0x7f56941f2bc0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f568c00efc0 tx=0x7f568c00c800 comp rx=0 tx=0).stop 2024-09-21T07:47:37.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.566+0000 7f5693e00640 1 -- 172.21.15.39:0/4182838168 shutdown_connections 2024-09-21T07:47:37.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.566+0000 7f5693e00640 1 --2- 172.21.15.39:0/4182838168 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5670078b90 0x7f567007b0a0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:37.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.566+0000 7f5693e00640 1 --2- 172.21.15.39:0/4182838168 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5694160660 0x7f56941f2bc0 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:37.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.566+0000 7f5693e00640 1 --2- 172.21.15.39:0/4182838168 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5694160250 0x7f56941f9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:37.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.566+0000 7f5693e00640 1 --2- 172.21.15.39:0/4182838168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56940d0390 0x7f56941e6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:37.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.566+0000 7f5693e00640 1 -- 172.21.15.39:0/4182838168 >> 172.21.15.39:0/4182838168 conn(0x7f569406cfc0 msgr2=0x7f5694073f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:37.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.566+0000 7f5693e00640 1 -- 172.21.15.39:0/4182838168 shutdown_connections 2024-09-21T07:47:37.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.566+0000 7f5693e00640 1 -- 172.21.15.39:0/4182838168 wait complete. 2024-09-21T07:47:37.591 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:47:37.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.815+0000 7f40a9a00640 1 -- 172.21.15.39:0/729345710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40a40ccc60 msgr2=0x7f40a40cd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:37.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.815+0000 7f40a9a00640 1 --2- 172.21.15.39:0/729345710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40a40ccc60 0x7f40a40cd070 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f40a4071e60 tx=0x7f409c02a820 comp rx=0 tx=0).stop 2024-09-21T07:47:37.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.815+0000 7f40a9a00640 1 -- 172.21.15.39:0/729345710 shutdown_connections 2024-09-21T07:47:37.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.815+0000 7f40a9a00640 1 --2- 172.21.15.39:0/729345710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40a40ccc60 0x7f40a40cd070 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:37.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.815+0000 7f40a9a00640 1 --2- 172.21.15.39:0/729345710 >> v2:172.21.15.148:3300/0 conn(0x7f40a40b30a0 0x7f40a40cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:37.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.815+0000 7f40a9a00640 1 -- 172.21.15.39:0/729345710 >> 172.21.15.39:0/729345710 conn(0x7f40a40b2460 msgr2=0x7f40a406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:37.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.815+0000 7f40a9a00640 1 -- 172.21.15.39:0/729345710 shutdown_connections 2024-09-21T07:47:37.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.815+0000 7f40a9a00640 1 -- 172.21.15.39:0/729345710 wait complete. 2024-09-21T07:47:37.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.817+0000 7f40a9a00640 1 Processor -- start 2024-09-21T07:47:37.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.817+0000 7f40a9a00640 1 -- start start 2024-09-21T07:47:37.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.819+0000 7f40a9a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40a40b30a0 0x7f40a41f34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:37.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.819+0000 7f40a9a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40a40ccc60 0x7f40a41f6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:37.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.819+0000 7f40a9a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f40a415d830 0x7f40a40c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:37.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.819+0000 7f40a9a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f40a40af410 con 0x7f40a40ccc60 2024-09-21T07:47:37.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.819+0000 7f40a9a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f40a40af290 con 0x7f40a40b30a0 2024-09-21T07:47:37.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.819+0000 7f40a9a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f40a40af590 con 0x7f40a415d830 2024-09-21T07:47:37.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.819+0000 7f409be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40a40ccc60 0x7f40a41f6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:37.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.819+0000 7f409be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40a40ccc60 0x7f40a41f6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34180/0 (socket says 172.21.15.39:34180) 2024-09-21T07:47:37.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.819+0000 7f409be00640 1 -- 172.21.15.39:0/3045322212 learned_addr learned my addr 172.21.15.39:0/3045322212 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:37.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.819+0000 7f40a2a00640 1 --2- 172.21.15.39:0/3045322212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40a40b30a0 0x7f40a41f34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:37.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.819+0000 7f40a3400640 1 --2- 172.21.15.39:0/3045322212 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f40a415d830 0x7f40a40c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:37.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.820+0000 7f409be00640 1 -- 172.21.15.39:0/3045322212 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f40a415d830 msgr2=0x7f40a40c77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:37.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.820+0000 7f409be00640 1 --2- 172.21.15.39:0/3045322212 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f40a415d830 0x7f40a40c77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:37.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.820+0000 7f409be00640 1 -- 172.21.15.39:0/3045322212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40a40b30a0 msgr2=0x7f40a41f34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:37.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.820+0000 7f409be00640 1 --2- 172.21.15.39:0/3045322212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40a40b30a0 0x7f40a41f34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:37.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.820+0000 7f409be00640 1 -- 172.21.15.39:0/3045322212 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40a40c5370 con 0x7f40a40ccc60 2024-09-21T07:47:37.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.820+0000 7f409be00640 1 --2- 172.21.15.39:0/3045322212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40a40ccc60 0x7f40a41f6010 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f408c00c3c0 tx=0x7f408c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:37.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.820+0000 7f409aa00640 1 -- 172.21.15.39:0/3045322212 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c017070 con 0x7f40a40ccc60 2024-09-21T07:47:37.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.820+0000 7f409aa00640 1 -- 172.21.15.39:0/3045322212 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f408c0035f0 con 0x7f40a40ccc60 2024-09-21T07:47:37.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.820+0000 7f409aa00640 1 -- 172.21.15.39:0/3045322212 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c003a80 con 0x7f40a40ccc60 2024-09-21T07:47:37.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.820+0000 7f40a9a00640 1 -- 172.21.15.39:0/3045322212 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40a41f6b40 con 0x7f40a40ccc60 2024-09-21T07:47:37.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.820+0000 7f40a9a00640 1 -- 172.21.15.39:0/3045322212 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40a40c5630 con 0x7f40a40ccc60 2024-09-21T07:47:37.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.821+0000 7f40a9a00640 1 -- 172.21.15.39:0/3045322212 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40a41f68b0 con 0x7f40a40ccc60 2024-09-21T07:47:37.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.823+0000 7f409aa00640 1 -- 172.21.15.39:0/3045322212 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f408c01a070 con 0x7f40a40ccc60 2024-09-21T07:47:37.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.824+0000 7f409aa00640 1 --2- 172.21.15.39:0/3045322212 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4080078bc0 0x7f408007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:37.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.824+0000 7f409aa00640 1 -- 172.21.15.39:0/3045322212 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 6932+0+0 (secure 0 0 0) 0x7f408c09a4f0 con 0x7f40a40ccc60 2024-09-21T07:47:37.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.824+0000 7f40a2a00640 1 --2- 172.21.15.39:0/3045322212 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4080078bc0 0x7f408007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:37.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.825+0000 7f40a2a00640 1 --2- 172.21.15.39:0/3045322212 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4080078bc0 0x7f408007b0d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f409400bb50 tx=0x7f409400b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:37.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:37.828+0000 7f409aa00640 1 -- 172.21.15.39:0/3045322212 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f40a40cd5b0 con 0x7f40a40ccc60 2024-09-21T07:47:38.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.028+0000 7f40a9a00640 1 -- 172.21.15.39:0/3045322212 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f40a40afab0 con 0x7f40a40ccc60 2024-09-21T07:47:38.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.030+0000 7f409aa00640 1 -- 172.21.15.39:0/3045322212 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v194) ==== 74+0+22348 (secure 0 0 0) 0x7f408c061d70 con 0x7f40a40ccc60 2024-09-21T07:47:38.032 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:47:38.033 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":194,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:47:10.626955+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:47:07.077866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"193","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-09-21T07:47:08.095979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"194","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]},{"pool":20,"snaps":[{"begin":80,"length":23},{"begin":181,"length":23}]},{"pool":22,"snaps":[{"begin":100,"length":1},{"begin":199,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:47:38.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.032+0000 7f40a9a00640 1 -- 172.21.15.39:0/3045322212 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4080078bc0 msgr2=0x7f408007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:38.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.032+0000 7f40a9a00640 1 --2- 172.21.15.39:0/3045322212 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4080078bc0 0x7f408007b0d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f409400bb50 tx=0x7f409400b2e0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.032+0000 7f40a9a00640 1 -- 172.21.15.39:0/3045322212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40a40ccc60 msgr2=0x7f40a41f6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:38.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.032+0000 7f40a9a00640 1 --2- 172.21.15.39:0/3045322212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40a40ccc60 0x7f40a41f6010 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f408c00c3c0 tx=0x7f408c00c880 comp rx=0 tx=0).stop 2024-09-21T07:47:38.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.032+0000 7f40a9a00640 1 -- 172.21.15.39:0/3045322212 shutdown_connections 2024-09-21T07:47:38.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.032+0000 7f40a9a00640 1 --2- 172.21.15.39:0/3045322212 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4080078bc0 0x7f408007b0d0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.032+0000 7f40a9a00640 1 --2- 172.21.15.39:0/3045322212 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f40a415d830 0x7f40a40c77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.032+0000 7f40a9a00640 1 --2- 172.21.15.39:0/3045322212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40a40ccc60 0x7f40a41f6010 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.032+0000 7f40a9a00640 1 --2- 172.21.15.39:0/3045322212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40a40b30a0 0x7f40a41f34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.032+0000 7f40a9a00640 1 -- 172.21.15.39:0/3045322212 >> 172.21.15.39:0/3045322212 conn(0x7f40a40b2460 msgr2=0x7f40a40ad060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:38.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.033+0000 7f40a9a00640 1 -- 172.21.15.39:0/3045322212 shutdown_connections 2024-09-21T07:47:38.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.033+0000 7f40a9a00640 1 -- 172.21.15.39:0/3045322212 wait complete. 2024-09-21T07:47:38.060 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T07:47:38.061 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:47:38.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.287+0000 7f73cb200640 1 -- 172.21.15.39:0/216595936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73c40b43f0 msgr2=0x7f73c415b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:38.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.287+0000 7f73cb200640 1 --2- 172.21.15.39:0/216595936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73c40b43f0 0x7f73c415b640 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f73b4005fd0 tx=0x7f73b402a5f0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.287+0000 7f73cb200640 1 -- 172.21.15.39:0/216595936 shutdown_connections 2024-09-21T07:47:38.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.287+0000 7f73cb200640 1 --2- 172.21.15.39:0/216595936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73c40b43f0 0x7f73c415b640 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.287+0000 7f73cb200640 1 -- 172.21.15.39:0/216595936 >> 172.21.15.39:0/216595936 conn(0x7f73c40d0c70 msgr2=0x7f73c40d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:38.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.287+0000 7f73cb200640 1 -- 172.21.15.39:0/216595936 shutdown_connections 2024-09-21T07:47:38.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.287+0000 7f73cb200640 1 -- 172.21.15.39:0/216595936 wait complete. 2024-09-21T07:47:38.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.289+0000 7f73cb200640 1 Processor -- start 2024-09-21T07:47:38.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.289+0000 7f73cb200640 1 -- start start 2024-09-21T07:47:38.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.291+0000 7f73cb200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f73c406f4e0 0x7f73c4152b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:38.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.291+0000 7f73cb200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73c40b43f0 0x7f73c40afec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:38.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.291+0000 7f73cb200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73c40d0400 0x7f73c414c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:38.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.291+0000 7f73cb200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f73c40b5580 con 0x7f73c40d0400 2024-09-21T07:47:38.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.291+0000 7f73cb200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f73c40d0260 con 0x7f73c40b43f0 2024-09-21T07:47:38.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.291+0000 7f73cb200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f73c40b5700 con 0x7f73c406f4e0 2024-09-21T07:47:38.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.291+0000 7f73ca800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73c40d0400 0x7f73c414c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:38.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.291+0000 7f73ca800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73c40d0400 0x7f73c414c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34204/0 (socket says 172.21.15.39:34204) 2024-09-21T07:47:38.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.291+0000 7f73ca800640 1 -- 172.21.15.39:0/2591941688 learned_addr learned my addr 172.21.15.39:0/2591941688 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:38.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.291+0000 7f73c9400640 1 --2- 172.21.15.39:0/2591941688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73c40b43f0 0x7f73c40afec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:38.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.291+0000 7f73c9e00640 1 --2- 172.21.15.39:0/2591941688 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f73c406f4e0 0x7f73c4152b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:38.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.291+0000 7f73ca800640 1 -- 172.21.15.39:0/2591941688 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f73c406f4e0 msgr2=0x7f73c4152b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:38.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.291+0000 7f73ca800640 1 --2- 172.21.15.39:0/2591941688 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f73c406f4e0 0x7f73c4152b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.291+0000 7f73ca800640 1 -- 172.21.15.39:0/2591941688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73c40b43f0 msgr2=0x7f73c40afec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:38.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.291+0000 7f73ca800640 1 --2- 172.21.15.39:0/2591941688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73c40b43f0 0x7f73c40afec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.291+0000 7f73ca800640 1 -- 172.21.15.39:0/2591941688 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73c41525d0 con 0x7f73c40d0400 2024-09-21T07:47:38.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.292+0000 7f73ca800640 1 --2- 172.21.15.39:0/2591941688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73c40d0400 0x7f73c414c450 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f73c000c760 tx=0x7f73c000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:38.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.292+0000 7f73be000640 1 -- 172.21.15.39:0/2591941688 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c0003da0 con 0x7f73c40d0400 2024-09-21T07:47:38.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.292+0000 7f73be000640 1 -- 172.21.15.39:0/2591941688 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f73c000ce10 con 0x7f73c40d0400 2024-09-21T07:47:38.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.292+0000 7f73be000640 1 -- 172.21.15.39:0/2591941688 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c00052d0 con 0x7f73c40d0400 2024-09-21T07:47:38.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.292+0000 7f73cb200640 1 -- 172.21.15.39:0/2591941688 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73c41522f0 con 0x7f73c40d0400 2024-09-21T07:47:38.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.292+0000 7f73cb200640 1 -- 172.21.15.39:0/2591941688 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73c40af6c0 con 0x7f73c40d0400 2024-09-21T07:47:38.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.293+0000 7f73cb200640 1 -- 172.21.15.39:0/2591941688 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73c40b5c20 con 0x7f73c40d0400 2024-09-21T07:47:38.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.294+0000 7f73be000640 1 -- 172.21.15.39:0/2591941688 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f73c0005470 con 0x7f73c40d0400 2024-09-21T07:47:38.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.295+0000 7f73be000640 1 --2- 172.21.15.39:0/2591941688 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f73a8078bc0 0x7f73a807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:38.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.295+0000 7f73be000640 1 -- 172.21.15.39:0/2591941688 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 6932+0+0 (secure 0 0 0) 0x7f73c009aff0 con 0x7f73c40d0400 2024-09-21T07:47:38.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.296+0000 7f73c9e00640 1 --2- 172.21.15.39:0/2591941688 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f73a8078bc0 0x7f73a807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:38.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.297+0000 7f73c9e00640 1 --2- 172.21.15.39:0/2591941688 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f73a8078bc0 0x7f73a807b0d0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f73b8000f60 tx=0x7f73b8009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:38.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.300+0000 7f73be000640 1 -- 172.21.15.39:0/2591941688 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f73c40b4800 con 0x7f73c40d0400 2024-09-21T07:47:38.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.534+0000 7f73cb200640 1 -- 172.21.15.39:0/2591941688 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73c4069fc0 con 0x7f73c40d0400 2024-09-21T07:47:38.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.535+0000 7f73be000640 1 -- 172.21.15.39:0/2591941688 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 204 v204) ==== 95+0+5250 (secure 0 0 0) 0x7f73c0062ec0 con 0x7f73c40d0400 2024-09-21T07:47:38.537 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:47:38.537 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":204,"btime":"2024-09-21T07:47:29:886149+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7715,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3057062733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3057062733},{"type":"v1","addr":"172.21.15.148:6836","nonce":3057062733}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":194},{"gid":8689,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/8652512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":8652512},{"type":"v1","addr":"172.21.15.148:6837","nonce":8652512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":194}],"filesystems":[{"mdsmap":{"epoch":204,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:47:09.140224+0000","modified":"2024-09-21T07:47:29.653225+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":9492,"mds_1":9483},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9483":{"gid":9483,"name":"a","rank":1,"incarnation":202,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6833/3199732683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3199732683},{"type":"v1","addr":"172.21.15.39:6833","nonce":3199732683}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_9492":{"gid":9492,"name":"c","rank":0,"incarnation":195,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/3348864735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3348864735},{"type":"v1","addr":"172.21.15.39:6835","nonce":3348864735}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9492,"qdb_cluster":[9492,9483]},"id":10}]} 2024-09-21T07:47:38.537 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 204 2024-09-21T07:47:38.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.537+0000 7f73cb200640 1 -- 172.21.15.39:0/2591941688 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f73a8078bc0 msgr2=0x7f73a807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:38.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.537+0000 7f73cb200640 1 --2- 172.21.15.39:0/2591941688 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f73a8078bc0 0x7f73a807b0d0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f73b8000f60 tx=0x7f73b8009040 comp rx=0 tx=0).stop 2024-09-21T07:47:38.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.537+0000 7f73cb200640 1 -- 172.21.15.39:0/2591941688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73c40d0400 msgr2=0x7f73c414c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:38.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.537+0000 7f73cb200640 1 --2- 172.21.15.39:0/2591941688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73c40d0400 0x7f73c414c450 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f73c000c760 tx=0x7f73c000cc20 comp rx=0 tx=0).stop 2024-09-21T07:47:38.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.537+0000 7f73cb200640 1 -- 172.21.15.39:0/2591941688 shutdown_connections 2024-09-21T07:47:38.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.537+0000 7f73cb200640 1 --2- 172.21.15.39:0/2591941688 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f73a8078bc0 0x7f73a807b0d0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.537+0000 7f73cb200640 1 --2- 172.21.15.39:0/2591941688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73c40d0400 0x7f73c414c450 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.537+0000 7f73cb200640 1 --2- 172.21.15.39:0/2591941688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73c40b43f0 0x7f73c40afec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.537+0000 7f73cb200640 1 --2- 172.21.15.39:0/2591941688 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f73c406f4e0 0x7f73c4152b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.537+0000 7f73cb200640 1 -- 172.21.15.39:0/2591941688 >> 172.21.15.39:0/2591941688 conn(0x7f73c40d0c70 msgr2=0x7f73c40b1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:38.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.537+0000 7f73cb200640 1 -- 172.21.15.39:0/2591941688 shutdown_connections 2024-09-21T07:47:38.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.538+0000 7f73cb200640 1 -- 172.21.15.39:0/2591941688 wait complete. 2024-09-21T07:47:38.566 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:47:38.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.790+0000 7fe0a3200640 1 -- 172.21.15.39:0/2975227603 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe0a40d0400 msgr2=0x7fe0a40b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:38.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.790+0000 7fe0a3200640 1 --2- 172.21.15.39:0/2975227603 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe0a40d0400 0x7fe0a40b43f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe094005fd0 tx=0x7fe09402a540 comp rx=0 tx=0).stop 2024-09-21T07:47:38.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.790+0000 7fe0a3200640 1 -- 172.21.15.39:0/2975227603 shutdown_connections 2024-09-21T07:47:38.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.790+0000 7fe0a3200640 1 --2- 172.21.15.39:0/2975227603 >> v2:172.21.15.148:3300/0 conn(0x7fe0a415b640 0x7fe0a40cfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.790+0000 7fe0a3200640 1 --2- 172.21.15.39:0/2975227603 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe0a40d0400 0x7fe0a40b43f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.790+0000 7fe0a3200640 1 -- 172.21.15.39:0/2975227603 >> 172.21.15.39:0/2975227603 conn(0x7fe0a40d0c70 msgr2=0x7fe0a40d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:38.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.790+0000 7fe0a3200640 1 -- 172.21.15.39:0/2975227603 shutdown_connections 2024-09-21T07:47:38.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.790+0000 7fe0a3200640 1 -- 172.21.15.39:0/2975227603 wait complete. 2024-09-21T07:47:38.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.793+0000 7fe0a3200640 1 Processor -- start 2024-09-21T07:47:38.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.793+0000 7fe0a3200640 1 -- start start 2024-09-21T07:47:38.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.794+0000 7fe0a3200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe0a40cff80 0x7fe0a413bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:38.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.794+0000 7fe0a3200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0a40d0400 0x7fe0a413eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:38.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.794+0000 7fe0a3200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0a415b640 0x7fe0a413c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:38.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.794+0000 7fe0a3200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a40b5370 con 0x7fe0a40d0400 2024-09-21T07:47:38.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.794+0000 7fe0a3200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a406fe00 con 0x7fe0a415b640 2024-09-21T07:47:38.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.794+0000 7fe0a3200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe0a40b54f0 con 0x7fe0a40cff80 2024-09-21T07:47:38.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.794+0000 7fe099400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0a40d0400 0x7fe0a413eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:38.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.795+0000 7fe099400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0a40d0400 0x7fe0a413eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34210/0 (socket says 172.21.15.39:34210) 2024-09-21T07:47:38.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.795+0000 7fe099400640 1 -- 172.21.15.39:0/2222086114 learned_addr learned my addr 172.21.15.39:0/2222086114 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:38.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.795+0000 7fe0a1e00640 1 --2- 172.21.15.39:0/2222086114 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe0a40cff80 0x7fe0a413bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:38.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.795+0000 7fe0a2800640 1 --2- 172.21.15.39:0/2222086114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0a415b640 0x7fe0a413c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:38.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.795+0000 7fe099400640 1 -- 172.21.15.39:0/2222086114 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe0a40cff80 msgr2=0x7fe0a413bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:38.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.795+0000 7fe099400640 1 --2- 172.21.15.39:0/2222086114 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe0a40cff80 0x7fe0a413bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.795+0000 7fe099400640 1 -- 172.21.15.39:0/2222086114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0a415b640 msgr2=0x7fe0a413c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:38.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.795+0000 7fe099400640 1 --2- 172.21.15.39:0/2222086114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0a415b640 0x7fe0a413c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.795+0000 7fe099400640 1 -- 172.21.15.39:0/2222086114 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0a41e3c70 con 0x7fe0a40d0400 2024-09-21T07:47:38.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.795+0000 7fe099400640 1 --2- 172.21.15.39:0/2222086114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0a40d0400 0x7fe0a413eef0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fe08c00ea20 tx=0x7fe08c00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:38.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.796+0000 7fe09aa00640 1 -- 172.21.15.39:0/2222086114 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08c00ce40 con 0x7fe0a40d0400 2024-09-21T07:47:38.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.796+0000 7fe09aa00640 1 -- 172.21.15.39:0/2222086114 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe08c004020 con 0x7fe0a40d0400 2024-09-21T07:47:38.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.796+0000 7fe09aa00640 1 -- 172.21.15.39:0/2222086114 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08c004470 con 0x7fe0a40d0400 2024-09-21T07:47:38.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.796+0000 7fe0a3200640 1 -- 172.21.15.39:0/2222086114 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0a413f430 con 0x7fe0a40d0400 2024-09-21T07:47:38.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.796+0000 7fe0a3200640 1 -- 172.21.15.39:0/2222086114 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0a41e5960 con 0x7fe0a40d0400 2024-09-21T07:47:38.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.797+0000 7fe0a3200640 1 -- 172.21.15.39:0/2222086114 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0a40b43f0 con 0x7fe0a40d0400 2024-09-21T07:47:38.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.798+0000 7fe09aa00640 1 -- 172.21.15.39:0/2222086114 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe08c0035f0 con 0x7fe0a40d0400 2024-09-21T07:47:38.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.799+0000 7fe09aa00640 1 --2- 172.21.15.39:0/2222086114 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe080078b90 0x7fe08007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:38.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.799+0000 7fe09aa00640 1 -- 172.21.15.39:0/2222086114 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 6932+0+0 (secure 0 0 0) 0x7fe08c09b9e0 con 0x7fe0a40d0400 2024-09-21T07:47:38.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.800+0000 7fe0a1e00640 1 --2- 172.21.15.39:0/2222086114 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe080078b90 0x7fe08007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:38.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.800+0000 7fe0a1e00640 1 --2- 172.21.15.39:0/2222086114 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe080078b90 0x7fe08007b0a0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe094033000 tx=0x7fe094004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:38.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:38.804+0000 7fe09aa00640 1 -- 172.21.15.39:0/2222086114 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe08c063380 con 0x7fe0a40d0400 2024-09-21T07:47:39.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.044+0000 7fe0a3200640 1 -- 172.21.15.39:0/2222086114 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0a40ccbe0 con 0x7fe0a40d0400 2024-09-21T07:47:39.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.044+0000 7fe09aa00640 1 -- 172.21.15.39:0/2222086114 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 204 v204) ==== 95+0+5250 (secure 0 0 0) 0x7fe08c067db0 con 0x7fe0a40d0400 2024-09-21T07:47:39.047 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:47:39.047 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":204,"btime":"2024-09-21T07:47:29:886149+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7715,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3057062733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3057062733},{"type":"v1","addr":"172.21.15.148:6836","nonce":3057062733}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":194},{"gid":8689,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/8652512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":8652512},{"type":"v1","addr":"172.21.15.148:6837","nonce":8652512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":194}],"filesystems":[{"mdsmap":{"epoch":204,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:47:09.140224+0000","modified":"2024-09-21T07:47:29.653225+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":9492,"mds_1":9483},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9483":{"gid":9483,"name":"a","rank":1,"incarnation":202,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6833/3199732683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3199732683},{"type":"v1","addr":"172.21.15.39:6833","nonce":3199732683}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_9492":{"gid":9492,"name":"c","rank":0,"incarnation":195,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/3348864735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3348864735},{"type":"v1","addr":"172.21.15.39:6835","nonce":3348864735}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9492,"qdb_cluster":[9492,9483]},"id":10}]} 2024-09-21T07:47:39.047 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 204 2024-09-21T07:47:39.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.047+0000 7fe0a3200640 1 -- 172.21.15.39:0/2222086114 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe080078b90 msgr2=0x7fe08007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:39.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.047+0000 7fe0a3200640 1 --2- 172.21.15.39:0/2222086114 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe080078b90 0x7fe08007b0a0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe094033000 tx=0x7fe094004730 comp rx=0 tx=0).stop 2024-09-21T07:47:39.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.047+0000 7fe0a3200640 1 -- 172.21.15.39:0/2222086114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0a40d0400 msgr2=0x7fe0a413eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:39.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.047+0000 7fe0a3200640 1 --2- 172.21.15.39:0/2222086114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0a40d0400 0x7fe0a413eef0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fe08c00ea20 tx=0x7fe08c00eee0 comp rx=0 tx=0).stop 2024-09-21T07:47:39.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.047+0000 7fe0a3200640 1 -- 172.21.15.39:0/2222086114 shutdown_connections 2024-09-21T07:47:39.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.047+0000 7fe0a3200640 1 --2- 172.21.15.39:0/2222086114 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe080078b90 0x7fe08007b0a0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:39.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.047+0000 7fe0a3200640 1 --2- 172.21.15.39:0/2222086114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0a415b640 0x7fe0a413c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:39.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.047+0000 7fe0a3200640 1 --2- 172.21.15.39:0/2222086114 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0a40d0400 0x7fe0a413eef0 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:39.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.047+0000 7fe0a3200640 1 --2- 172.21.15.39:0/2222086114 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe0a40cff80 0x7fe0a413bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:39.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.047+0000 7fe0a3200640 1 -- 172.21.15.39:0/2222086114 >> 172.21.15.39:0/2222086114 conn(0x7fe0a40d0c70 msgr2=0x7fe0a414f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:39.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.047+0000 7fe0a3200640 1 -- 172.21.15.39:0/2222086114 shutdown_connections 2024-09-21T07:47:39.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.047+0000 7fe0a3200640 1 -- 172.21.15.39:0/2222086114 wait complete. 2024-09-21T07:47:39.072 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:47:39.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.296+0000 7f8ce9000640 1 -- 172.21.15.39:0/2736345885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ce406ef90 msgr2=0x7f8ce406f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:39.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.296+0000 7f8ce9000640 1 --2- 172.21.15.39:0/2736345885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ce406ef90 0x7f8ce406f3a0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc005fd0 tx=0x7f8cdc02aa90 comp rx=0 tx=0).stop 2024-09-21T07:47:39.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.296+0000 7f8ce9000640 1 -- 172.21.15.39:0/2736345885 shutdown_connections 2024-09-21T07:47:39.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.296+0000 7f8ce9000640 1 --2- 172.21.15.39:0/2736345885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ce406ef90 0x7f8ce406f3a0 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:39.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.296+0000 7f8ce9000640 1 --2- 172.21.15.39:0/2736345885 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ce40b3d80 0x7f8ce40ce2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:39.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.296+0000 7f8ce9000640 1 -- 172.21.15.39:0/2736345885 >> 172.21.15.39:0/2736345885 conn(0x7f8ce40b2460 msgr2=0x7f8ce406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:39.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.296+0000 7f8ce9000640 1 -- 172.21.15.39:0/2736345885 shutdown_connections 2024-09-21T07:47:39.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.296+0000 7f8ce9000640 1 -- 172.21.15.39:0/2736345885 wait complete. 2024-09-21T07:47:39.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.298+0000 7f8ce9000640 1 Processor -- start 2024-09-21T07:47:39.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.298+0000 7f8ce9000640 1 -- start start 2024-09-21T07:47:39.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.300+0000 7f8ce9000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ce406ef90 0x7f8ce415aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:39.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.300+0000 7f8ce9000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ce40b4c40 0x7f8ce41f92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:39.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.300+0000 7f8ce9000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ce41f9830 0x7f8ce40ae3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:39.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.300+0000 7f8ce9000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce406fa60 con 0x7f8ce41f9830 2024-09-21T07:47:39.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.300+0000 7f8ce9000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce406f8e0 con 0x7f8ce406ef90 2024-09-21T07:47:39.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.300+0000 7f8ce9000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f8ce406fbe0 con 0x7f8ce40b4c40 2024-09-21T07:47:39.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.300+0000 7f8ce2800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ce41f9830 0x7f8ce40ae3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:39.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.300+0000 7f8ce2800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ce41f9830 0x7f8ce40ae3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34230/0 (socket says 172.21.15.39:34230) 2024-09-21T07:47:39.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.300+0000 7f8ce2800640 1 -- 172.21.15.39:0/1633848080 learned_addr learned my addr 172.21.15.39:0/1633848080 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:39.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.300+0000 7f8cdbe00640 1 --2- 172.21.15.39:0/1633848080 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ce40b4c40 0x7f8ce41f92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:39.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.300+0000 7f8ce1e00640 1 --2- 172.21.15.39:0/1633848080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ce406ef90 0x7f8ce415aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:39.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.301+0000 7f8ce2800640 1 -- 172.21.15.39:0/1633848080 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ce40b4c40 msgr2=0x7f8ce41f92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:39.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.301+0000 7f8ce2800640 1 --2- 172.21.15.39:0/1633848080 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ce40b4c40 0x7f8ce41f92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:39.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.301+0000 7f8ce2800640 1 -- 172.21.15.39:0/1633848080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ce406ef90 msgr2=0x7f8ce415aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:39.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.301+0000 7f8ce2800640 1 --2- 172.21.15.39:0/1633848080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ce406ef90 0x7f8ce415aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:39.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.301+0000 7f8ce2800640 1 -- 172.21.15.39:0/1633848080 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ce40c83e0 con 0x7f8ce41f9830 2024-09-21T07:47:39.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.301+0000 7f8cdbe00640 1 --2- 172.21.15.39:0/1633848080 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ce40b4c40 0x7f8ce41f92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:39.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.301+0000 7f8ce1e00640 1 --2- 172.21.15.39:0/1633848080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ce406ef90 0x7f8ce415aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:39.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.301+0000 7f8ce2800640 1 --2- 172.21.15.39:0/1633848080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ce41f9830 0x7f8ce40ae3a0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc009bf0 tx=0x7f8cdc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:39.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.301+0000 7f8cda000640 1 -- 172.21.15.39:0/1633848080 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc044070 con 0x7f8ce41f9830 2024-09-21T07:47:39.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.301+0000 7f8cda000640 1 -- 172.21.15.39:0/1633848080 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8cdc003e00 con 0x7f8ce41f9830 2024-09-21T07:47:39.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.301+0000 7f8cda000640 1 -- 172.21.15.39:0/1633848080 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc0044b0 con 0x7f8ce41f9830 2024-09-21T07:47:39.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.301+0000 7f8ce9000640 1 -- 172.21.15.39:0/1633848080 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ce40c86c0 con 0x7f8ce41f9830 2024-09-21T07:47:39.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.301+0000 7f8ce9000640 1 -- 172.21.15.39:0/1633848080 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ce40c8c80 con 0x7f8ce41f9830 2024-09-21T07:47:39.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.303+0000 7f8ce9000640 1 -- 172.21.15.39:0/1633848080 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ce406f3a0 con 0x7f8ce41f9830 2024-09-21T07:47:39.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.304+0000 7f8cda000640 1 -- 172.21.15.39:0/1633848080 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8cdc047070 con 0x7f8ce41f9830 2024-09-21T07:47:39.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.305+0000 7f8cda000640 1 --2- 172.21.15.39:0/1633848080 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8cc0078b90 0x7f8cc007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:39.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.305+0000 7f8cda000640 1 -- 172.21.15.39:0/1633848080 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 6932+0+0 (secure 0 0 0) 0x7f8cdc0bc1c0 con 0x7f8ce41f9830 2024-09-21T07:47:39.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.305+0000 7f8ce1e00640 1 --2- 172.21.15.39:0/1633848080 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8cc0078b90 0x7f8cc007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:39.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.306+0000 7f8ce1e00640 1 --2- 172.21.15.39:0/1633848080 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8cc0078b90 0x7f8cc007b0a0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4000f60 tx=0x7f8cd4009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:39.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.309+0000 7f8cda000640 1 -- 172.21.15.39:0/1633848080 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8cdc032030 con 0x7f8ce41f9830 2024-09-21T07:47:39.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.507+0000 7f8ce9000640 1 -- 172.21.15.39:0/1633848080 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8ce40b3130 con 0x7f8ce41f9830 2024-09-21T07:47:39.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.509+0000 7f8cda000640 1 -- 172.21.15.39:0/1633848080 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v194) ==== 74+0+22348 (secure 0 0 0) 0x7f8cdc0098b0 con 0x7f8ce41f9830 2024-09-21T07:47:39.512 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:47:39.512 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":194,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:47:10.626955+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:47:07.077866+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"193","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-09-21T07:47:08.095979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"194","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]},{"pool":20,"snaps":[{"begin":80,"length":23},{"begin":181,"length":23}]},{"pool":22,"snaps":[{"begin":100,"length":1},{"begin":199,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:47:39.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.512+0000 7f8ce9000640 1 -- 172.21.15.39:0/1633848080 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8cc0078b90 msgr2=0x7f8cc007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:39.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.512+0000 7f8ce9000640 1 --2- 172.21.15.39:0/1633848080 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8cc0078b90 0x7f8cc007b0a0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4000f60 tx=0x7f8cd4009040 comp rx=0 tx=0).stop 2024-09-21T07:47:39.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.512+0000 7f8ce9000640 1 -- 172.21.15.39:0/1633848080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ce41f9830 msgr2=0x7f8ce40ae3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:39.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.512+0000 7f8ce9000640 1 --2- 172.21.15.39:0/1633848080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ce41f9830 0x7f8ce40ae3a0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc009bf0 tx=0x7f8cdc003c10 comp rx=0 tx=0).stop 2024-09-21T07:47:39.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.512+0000 7f8ce9000640 1 -- 172.21.15.39:0/1633848080 shutdown_connections 2024-09-21T07:47:39.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.512+0000 7f8ce9000640 1 --2- 172.21.15.39:0/1633848080 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8cc0078b90 0x7f8cc007b0a0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:39.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.512+0000 7f8ce9000640 1 --2- 172.21.15.39:0/1633848080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ce41f9830 0x7f8ce40ae3a0 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:39.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.512+0000 7f8ce9000640 1 --2- 172.21.15.39:0/1633848080 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ce40b4c40 0x7f8ce41f92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:39.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.512+0000 7f8ce9000640 1 --2- 172.21.15.39:0/1633848080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ce406ef90 0x7f8ce415aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:39.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.512+0000 7f8ce9000640 1 -- 172.21.15.39:0/1633848080 >> 172.21.15.39:0/1633848080 conn(0x7f8ce40b2460 msgr2=0x7f8ce40d0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:39.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.512+0000 7f8ce9000640 1 -- 172.21.15.39:0/1633848080 shutdown_connections 2024-09-21T07:47:39.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.512+0000 7f8ce9000640 1 -- 172.21.15.39:0/1633848080 wait complete. 2024-09-21T07:47:39.537 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T07:47:39.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.764+0000 7f92a2e00640 1 -- 172.21.15.39:0/4139334229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f929c0d0e60 msgr2=0x7f929c0d1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:39.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.764+0000 7f92a2e00640 1 --2- 172.21.15.39:0/4139334229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f929c0d0e60 0x7f929c0d1270 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f929c0d2570 tx=0x7f929402aa10 comp rx=0 tx=0).stop 2024-09-21T07:47:39.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.765+0000 7f92a2e00640 1 -- 172.21.15.39:0/4139334229 shutdown_connections 2024-09-21T07:47:39.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.765+0000 7f92a2e00640 1 --2- 172.21.15.39:0/4139334229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f929c0d0e60 0x7f929c0d1270 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:39.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.765+0000 7f92a2e00640 1 --2- 172.21.15.39:0/4139334229 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f929c0b59e0 0x7f929c0b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:39.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.765+0000 7f92a2e00640 1 --2- 172.21.15.39:0/4139334229 >> v2:172.21.15.148:3300/0 conn(0x7f929c0d5060 0x7f929c0b96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:39.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.765+0000 7f92a2e00640 1 -- 172.21.15.39:0/4139334229 >> 172.21.15.39:0/4139334229 conn(0x7f929c09d7d0 msgr2=0x7f929c0d4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:39.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.765+0000 7f92a2e00640 1 -- 172.21.15.39:0/4139334229 shutdown_connections 2024-09-21T07:47:39.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.765+0000 7f92a2e00640 1 -- 172.21.15.39:0/4139334229 wait complete. 2024-09-21T07:47:39.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.767+0000 7f92a2e00640 1 Processor -- start 2024-09-21T07:47:39.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.767+0000 7f92a2e00640 1 -- start start 2024-09-21T07:47:39.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.769+0000 7f92a2e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f929c0b59e0 0x7f929c1f9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:39.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.769+0000 7f92a2e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f929c0d0e60 0x7f929c1f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:39.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.769+0000 7f92a2e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f929c0d5060 0x7f929c14d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:39.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.769+0000 7f92a2e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f929c0d2880 con 0x7f929c0d5060 2024-09-21T07:47:39.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.769+0000 7f92a2e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f929c0d0320 con 0x7f929c0b59e0 2024-09-21T07:47:39.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.769+0000 7f92a2e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f929c0d2a00 con 0x7f929c0d0e60 2024-09-21T07:47:39.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.769+0000 7f92a2400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f929c0d5060 0x7f929c14d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:39.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.769+0000 7f92a2400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f929c0d5060 0x7f929c14d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34256/0 (socket says 172.21.15.39:34256) 2024-09-21T07:47:39.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.769+0000 7f92a2400640 1 -- 172.21.15.39:0/3097518858 learned_addr learned my addr 172.21.15.39:0/3097518858 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:39.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.769+0000 7f92a1000640 1 --2- 172.21.15.39:0/3097518858 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f929c0d0e60 0x7f929c1f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:39.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.769+0000 7f92a1a00640 1 --2- 172.21.15.39:0/3097518858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f929c0b59e0 0x7f929c1f9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:39.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.769+0000 7f92a2400640 1 -- 172.21.15.39:0/3097518858 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f929c0d0e60 msgr2=0x7f929c1f5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:39.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.769+0000 7f92a2400640 1 --2- 172.21.15.39:0/3097518858 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f929c0d0e60 0x7f929c1f5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:39.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.769+0000 7f92a2400640 1 -- 172.21.15.39:0/3097518858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f929c0b59e0 msgr2=0x7f929c1f9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:39.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.769+0000 7f92a2400640 1 --2- 172.21.15.39:0/3097518858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f929c0b59e0 0x7f929c1f9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:39.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.769+0000 7f92a2400640 1 -- 172.21.15.39:0/3097518858 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f929c0a0410 con 0x7f929c0d5060 2024-09-21T07:47:39.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.769+0000 7f92a1a00640 1 --2- 172.21.15.39:0/3097518858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f929c0b59e0 0x7f929c1f9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:39.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.770+0000 7f92a2400640 1 --2- 172.21.15.39:0/3097518858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f929c0d5060 0x7f929c14d1f0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f9294005fd0 tx=0x7f9294003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:39.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.770+0000 7f9299600640 1 -- 172.21.15.39:0/3097518858 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9294033050 con 0x7f929c0d5060 2024-09-21T07:47:39.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.770+0000 7f9299600640 1 -- 172.21.15.39:0/3097518858 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f92940042d0 con 0x7f929c0d5060 2024-09-21T07:47:39.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.770+0000 7f9299600640 1 -- 172.21.15.39:0/3097518858 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9294004720 con 0x7f929c0d5060 2024-09-21T07:47:39.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.770+0000 7f92a2e00640 1 -- 172.21.15.39:0/3097518858 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f929c1f6420 con 0x7f929c0d5060 2024-09-21T07:47:39.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.770+0000 7f92a2e00640 1 -- 172.21.15.39:0/3097518858 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f929c1f69e0 con 0x7f929c0d5060 2024-09-21T07:47:39.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.771+0000 7f92a2e00640 1 -- 172.21.15.39:0/3097518858 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f929c1f6190 con 0x7f929c0d5060 2024-09-21T07:47:39.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.773+0000 7f9299600640 1 -- 172.21.15.39:0/3097518858 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9294036070 con 0x7f929c0d5060 2024-09-21T07:47:39.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.773+0000 7f9299600640 1 --2- 172.21.15.39:0/3097518858 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f927c078bc0 0x7f927c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:39.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.774+0000 7f9299600640 1 -- 172.21.15.39:0/3097518858 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 6932+0+0 (secure 0 0 0) 0x7f92940bc0a0 con 0x7f929c0d5060 2024-09-21T07:47:39.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.774+0000 7f92a1a00640 1 --2- 172.21.15.39:0/3097518858 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f927c078bc0 0x7f927c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:39.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.775+0000 7f92a1a00640 1 --2- 172.21.15.39:0/3097518858 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f927c078bc0 0x7f927c07b0d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f9290005e30 tx=0x7f9290005dc0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:39.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:39.778+0000 7f9299600640 1 -- 172.21.15.39:0/3097518858 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9294049020 con 0x7f929c0d5060 2024-09-21T07:47:40.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.028+0000 7f92a2e00640 1 -- 172.21.15.39:0/3097518858 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f929c0b3b60 con 0x7f929c0d5060 2024-09-21T07:47:40.031 INFO:tasks.ceph.mon.a.smithi039.stderr:2024-09-21T07:47:40.029+0000 7fabab600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T07:47:40.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.031+0000 7f9299600640 1 -- 172.21.15.39:0/3097518858 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v206) ==== 161+0+0 (secure 0 0 0) 0x7f9294084010 con 0x7f929c0d5060 2024-09-21T07:47:40.033 INFO:teuthology.orchestra.run.smithi039.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T07:47:40.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.034+0000 7f92a2e00640 1 -- 172.21.15.39:0/3097518858 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f927c078bc0 msgr2=0x7f927c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:40.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.034+0000 7f92a2e00640 1 --2- 172.21.15.39:0/3097518858 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f927c078bc0 0x7f927c07b0d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f9290005e30 tx=0x7f9290005dc0 comp rx=0 tx=0).stop 2024-09-21T07:47:40.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.034+0000 7f92a2e00640 1 -- 172.21.15.39:0/3097518858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f929c0d5060 msgr2=0x7f929c14d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:40.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.034+0000 7f92a2e00640 1 --2- 172.21.15.39:0/3097518858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f929c0d5060 0x7f929c14d1f0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f9294005fd0 tx=0x7f9294003f10 comp rx=0 tx=0).stop 2024-09-21T07:47:40.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.035+0000 7f92a2e00640 1 -- 172.21.15.39:0/3097518858 shutdown_connections 2024-09-21T07:47:40.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.035+0000 7f92a2e00640 1 --2- 172.21.15.39:0/3097518858 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f927c078bc0 0x7f927c07b0d0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:40.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.035+0000 7f92a2e00640 1 --2- 172.21.15.39:0/3097518858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f929c0d5060 0x7f929c14d1f0 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:40.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.035+0000 7f92a2e00640 1 --2- 172.21.15.39:0/3097518858 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f929c0d0e60 0x7f929c1f5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:40.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.035+0000 7f92a2e00640 1 --2- 172.21.15.39:0/3097518858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f929c0b59e0 0x7f929c1f9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:40.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.035+0000 7f92a2e00640 1 -- 172.21.15.39:0/3097518858 >> 172.21.15.39:0/3097518858 conn(0x7f929c09d7d0 msgr2=0x7f929c06e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:40.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.035+0000 7f92a2e00640 1 -- 172.21.15.39:0/3097518858 shutdown_connections 2024-09-21T07:47:40.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.035+0000 7f92a2e00640 1 -- 172.21.15.39:0/3097518858 wait complete. 2024-09-21T07:47:40.063 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T07:47:40.185 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-21T07:47:40.190 INFO:tasks.ceph.mds.c.smithi039.stdout:starting mds.c at 2024-09-21T07:47:40.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.305+0000 7f3d2fe00640 1 -- 172.21.15.39:0/3415516534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d28070960 msgr2=0x7f3d280cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:40.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.305+0000 7f3d2fe00640 1 --2- 172.21.15.39:0/3415516534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d28070960 0x7f3d280cdf80 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f3d28072d00 tx=0x7f3d1c02a560 comp rx=0 tx=0).stop 2024-09-21T07:47:40.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.305+0000 7f3d2fe00640 1 -- 172.21.15.39:0/3415516534 shutdown_connections 2024-09-21T07:47:40.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.305+0000 7f3d2fe00640 1 --2- 172.21.15.39:0/3415516534 >> v2:172.21.15.148:3300/0 conn(0x7f3d280cc9d0 0x7f3d280ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:40.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.305+0000 7f3d2fe00640 1 --2- 172.21.15.39:0/3415516534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d28070960 0x7f3d280cdf80 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:40.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.305+0000 7f3d2fe00640 1 -- 172.21.15.39:0/3415516534 >> 172.21.15.39:0/3415516534 conn(0x7f3d280b2460 msgr2=0x7f3d2806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:40.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.305+0000 7f3d2fe00640 1 -- 172.21.15.39:0/3415516534 shutdown_connections 2024-09-21T07:47:40.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.305+0000 7f3d2fe00640 1 -- 172.21.15.39:0/3415516534 wait complete. 2024-09-21T07:47:40.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.308+0000 7f3d2fe00640 1 Processor -- start 2024-09-21T07:47:40.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.308+0000 7f3d2fe00640 1 -- start start 2024-09-21T07:47:40.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.309+0000 7f3d2fe00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d28070960 0x7f3d2815ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:40.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.309+0000 7f3d2fe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d280cc9d0 0x7f3d2815cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:40.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.309+0000 7f3d2fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d2815ae20 0x7f3d2815d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:40.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.309+0000 7f3d2fe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2806d880 con 0x7f3d2815ae20 2024-09-21T07:47:40.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.309+0000 7f3d2fe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2806d700 con 0x7f3d280cc9d0 2024-09-21T07:47:40.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.309+0000 7f3d2fe00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f3d2806da00 con 0x7f3d28070960 2024-09-21T07:47:40.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.309+0000 7f3d2de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d2815ae20 0x7f3d2815d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:40.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.309+0000 7f3d2de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d2815ae20 0x7f3d2815d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34308/0 (socket says 172.21.15.39:34308) 2024-09-21T07:47:40.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.309+0000 7f3d2de00640 1 -- 172.21.15.39:0/4013039617 learned_addr learned my addr 172.21.15.39:0/4013039617 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:40.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.310+0000 7f3d2ca00640 1 --2- 172.21.15.39:0/4013039617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d280cc9d0 0x7f3d2815cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:40.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.310+0000 7f3d2d400640 1 --2- 172.21.15.39:0/4013039617 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d28070960 0x7f3d2815ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:40.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.310+0000 7f3d2de00640 1 -- 172.21.15.39:0/4013039617 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d28070960 msgr2=0x7f3d2815ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:40.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.310+0000 7f3d2de00640 1 --2- 172.21.15.39:0/4013039617 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d28070960 0x7f3d2815ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:40.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.310+0000 7f3d2de00640 1 -- 172.21.15.39:0/4013039617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d280cc9d0 msgr2=0x7f3d2815cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:40.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.310+0000 7f3d2de00640 1 --2- 172.21.15.39:0/4013039617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d280cc9d0 0x7f3d2815cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:40.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.310+0000 7f3d2de00640 1 -- 172.21.15.39:0/4013039617 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d281587c0 con 0x7f3d2815ae20 2024-09-21T07:47:40.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.310+0000 7f3d2de00640 1 --2- 172.21.15.39:0/4013039617 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d2815ae20 0x7f3d2815d8d0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f3d2000ddf0 tx=0x7f3d2000b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:40.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.310+0000 7f3d25600640 1 -- 172.21.15.39:0/4013039617 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d20003b70 con 0x7f3d2815ae20 2024-09-21T07:47:40.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.311+0000 7f3d25600640 1 -- 172.21.15.39:0/4013039617 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d20003e00 con 0x7f3d2815ae20 2024-09-21T07:47:40.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.311+0000 7f3d2fe00640 1 -- 172.21.15.39:0/4013039617 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d28158310 con 0x7f3d2815ae20 2024-09-21T07:47:40.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.311+0000 7f3d25600640 1 -- 172.21.15.39:0/4013039617 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d20004280 con 0x7f3d2815ae20 2024-09-21T07:47:40.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.311+0000 7f3d2fe00640 1 -- 172.21.15.39:0/4013039617 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d2806fa60 con 0x7f3d2815ae20 2024-09-21T07:47:40.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.312+0000 7f3d2fe00640 1 -- 172.21.15.39:0/4013039617 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d281585f0 con 0x7f3d2815ae20 2024-09-21T07:47:40.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.313+0000 7f3d25600640 1 -- 172.21.15.39:0/4013039617 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3d20004420 con 0x7f3d2815ae20 2024-09-21T07:47:40.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.314+0000 7f3d25600640 1 --2- 172.21.15.39:0/4013039617 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3d08078bc0 0x7f3d0807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:40.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.315+0000 7f3d25600640 1 -- 172.21.15.39:0/4013039617 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(196..196 src has 1..196) ==== 7132+0+0 (secure 0 0 0) 0x7f3d2009bc60 con 0x7f3d2815ae20 2024-09-21T07:47:40.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.315+0000 7f3d2d400640 1 --2- 172.21.15.39:0/4013039617 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3d08078bc0 0x7f3d0807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:40.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.316+0000 7f3d2d400640 1 --2- 172.21.15.39:0/4013039617 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3d08078bc0 0x7f3d0807b0d0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c033000 tx=0x7f3d1c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:40.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.319+0000 7f3d25600640 1 -- 172.21.15.39:0/4013039617 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3d200633a0 con 0x7f3d2815ae20 2024-09-21T07:47:40.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:40.559+0000 7f3d2fe00640 1 -- 172.21.15.39:0/4013039617 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f3d280cdf80 con 0x7f3d2815ae20 2024-09-21T07:47:41.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.033+0000 7f3d25600640 1 -- 172.21.15.39:0/4013039617 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v207) ==== 104+0+0 (secure 0 0 0) 0x7f3d20067dd0 con 0x7f3d2815ae20 2024-09-21T07:47:41.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.035+0000 7f3d2fe00640 1 -- 172.21.15.39:0/4013039617 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3d08078bc0 msgr2=0x7f3d0807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:41.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.035+0000 7f3d2fe00640 1 --2- 172.21.15.39:0/4013039617 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3d08078bc0 0x7f3d0807b0d0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c033000 tx=0x7f3d1c004730 comp rx=0 tx=0).stop 2024-09-21T07:47:41.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.035+0000 7f3d2fe00640 1 -- 172.21.15.39:0/4013039617 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d2815ae20 msgr2=0x7f3d2815d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:41.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.035+0000 7f3d2fe00640 1 --2- 172.21.15.39:0/4013039617 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d2815ae20 0x7f3d2815d8d0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f3d2000ddf0 tx=0x7f3d2000b730 comp rx=0 tx=0).stop 2024-09-21T07:47:41.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.035+0000 7f3d2fe00640 1 -- 172.21.15.39:0/4013039617 shutdown_connections 2024-09-21T07:47:41.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.035+0000 7f3d2fe00640 1 --2- 172.21.15.39:0/4013039617 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3d08078bc0 0x7f3d0807b0d0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:41.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.035+0000 7f3d2fe00640 1 --2- 172.21.15.39:0/4013039617 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3d2815ae20 0x7f3d2815d8d0 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:41.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.035+0000 7f3d2fe00640 1 --2- 172.21.15.39:0/4013039617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d280cc9d0 0x7f3d2815cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:41.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.035+0000 7f3d2fe00640 1 --2- 172.21.15.39:0/4013039617 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3d28070960 0x7f3d2815ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:41.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.035+0000 7f3d2fe00640 1 -- 172.21.15.39:0/4013039617 >> 172.21.15.39:0/4013039617 conn(0x7f3d280b2460 msgr2=0x7f3d28073d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:41.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.036+0000 7f3d2fe00640 1 -- 172.21.15.39:0/4013039617 shutdown_connections 2024-09-21T07:47:41.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.036+0000 7f3d2fe00640 1 -- 172.21.15.39:0/4013039617 wait complete. 2024-09-21T07:47:41.062 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T07:47:41.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.292+0000 7f4debe00640 1 -- 172.21.15.39:0/3105221857 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4dec0b5100 msgr2=0x7f4dec0b4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:41.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.292+0000 7f4debe00640 1 --2- 172.21.15.39:0/3105221857 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4dec0b5100 0x7f4dec0b4230 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc005fd0 tx=0x7f4ddc02a5d0 comp rx=0 tx=0).stop 2024-09-21T07:47:41.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.292+0000 7f4debe00640 1 -- 172.21.15.39:0/3105221857 shutdown_connections 2024-09-21T07:47:41.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.292+0000 7f4debe00640 1 --2- 172.21.15.39:0/3105221857 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4dec0b5100 0x7f4dec0b4230 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:41.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.292+0000 7f4debe00640 1 -- 172.21.15.39:0/3105221857 >> 172.21.15.39:0/3105221857 conn(0x7f4dec0b1e60 msgr2=0x7f4dec0b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:41.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.293+0000 7f4debe00640 1 -- 172.21.15.39:0/3105221857 shutdown_connections 2024-09-21T07:47:41.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.293+0000 7f4debe00640 1 -- 172.21.15.39:0/3105221857 wait complete. 2024-09-21T07:47:41.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.295+0000 7f4debe00640 1 Processor -- start 2024-09-21T07:47:41.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.295+0000 7f4debe00640 1 -- start start 2024-09-21T07:47:41.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.296+0000 7f4debe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4dec0acf60 0x7f4dec0c3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:41.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.296+0000 7f4debe00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4dec0b4bb0 0x7f4dec166990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:41.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.296+0000 7f4debe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4dec0b5100 0x7f4dec1f5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:41.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.296+0000 7f4debe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4dec0b48f0 con 0x7f4dec0b5100 2024-09-21T07:47:41.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.296+0000 7f4debe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4dec0b4770 con 0x7f4dec0acf60 2024-09-21T07:47:41.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.296+0000 7f4debe00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f4dec167f80 con 0x7f4dec0b4bb0 2024-09-21T07:47:41.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.297+0000 7f4deb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4dec0b5100 0x7f4dec1f5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:41.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.297+0000 7f4deb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4dec0b5100 0x7f4dec1f5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34340/0 (socket says 172.21.15.39:34340) 2024-09-21T07:47:41.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.297+0000 7f4deb400640 1 -- 172.21.15.39:0/883613421 learned_addr learned my addr 172.21.15.39:0/883613421 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:41.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.297+0000 7f4deaa00640 1 --2- 172.21.15.39:0/883613421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4dec0acf60 0x7f4dec0c3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:41.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.297+0000 7f4dea000640 1 --2- 172.21.15.39:0/883613421 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4dec0b4bb0 0x7f4dec166990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:41.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.297+0000 7f4deb400640 1 -- 172.21.15.39:0/883613421 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4dec0b4bb0 msgr2=0x7f4dec166990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:41.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.297+0000 7f4deb400640 1 --2- 172.21.15.39:0/883613421 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4dec0b4bb0 0x7f4dec166990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:41.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.297+0000 7f4deb400640 1 -- 172.21.15.39:0/883613421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4dec0acf60 msgr2=0x7f4dec0c3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:41.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.297+0000 7f4deb400640 1 --2- 172.21.15.39:0/883613421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4dec0acf60 0x7f4dec0c3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:41.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.297+0000 7f4deb400640 1 -- 172.21.15.39:0/883613421 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dec0c6b30 con 0x7f4dec0b5100 2024-09-21T07:47:41.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.298+0000 7f4deb400640 1 --2- 172.21.15.39:0/883613421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4dec0b5100 0x7f4dec1f5c80 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f4de400c6c0 tx=0x7f4de400cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:41.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.298+0000 7f4de2a00640 1 -- 172.21.15.39:0/883613421 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de4003b70 con 0x7f4dec0b5100 2024-09-21T07:47:41.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.298+0000 7f4de2a00640 1 -- 172.21.15.39:0/883613421 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4de4003d30 con 0x7f4dec0b5100 2024-09-21T07:47:41.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.298+0000 7f4de2a00640 1 -- 172.21.15.39:0/883613421 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de4005330 con 0x7f4dec0b5100 2024-09-21T07:47:41.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.298+0000 7f4debe00640 1 -- 172.21.15.39:0/883613421 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dec09ab10 con 0x7f4dec0b5100 2024-09-21T07:47:41.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.298+0000 7f4debe00640 1 -- 172.21.15.39:0/883613421 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dec0c65c0 con 0x7f4dec0b5100 2024-09-21T07:47:41.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.299+0000 7f4debe00640 1 -- 172.21.15.39:0/883613421 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dec0b4230 con 0x7f4dec0b5100 2024-09-21T07:47:41.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.301+0000 7f4de2a00640 1 -- 172.21.15.39:0/883613421 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4de40054d0 con 0x7f4dec0b5100 2024-09-21T07:47:41.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.301+0000 7f4de2a00640 1 --2- 172.21.15.39:0/883613421 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4dcc078b90 0x7f4dcc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:41.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.302+0000 7f4de2a00640 1 -- 172.21.15.39:0/883613421 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(196..196 src has 1..196) ==== 7132+0+0 (secure 0 0 0) 0x7f4de409af90 con 0x7f4dec0b5100 2024-09-21T07:47:41.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.302+0000 7f4deaa00640 1 --2- 172.21.15.39:0/883613421 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4dcc078b90 0x7f4dcc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:41.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.305+0000 7f4deaa00640 1 --2- 172.21.15.39:0/883613421 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4dcc078b90 0x7f4dcc07b0a0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc033000 tx=0x7f4ddc004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:41.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.306+0000 7f4de2a00640 1 -- 172.21.15.39:0/883613421 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4de4062eb0 con 0x7f4dec0b5100 2024-09-21T07:47:41.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:41.502+0000 7f4debe00640 1 -- 172.21.15.39:0/883613421 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4dec0aed20 con 0x7f4dec0b5100 2024-09-21T07:47:42.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.045+0000 7f4de2a00640 1 -- 172.21.15.39:0/883613421 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v197) ==== 181+0+0 (secure 0 0 0) 0x7f4de40627b0 con 0x7f4dec0b5100 2024-09-21T07:47:42.047 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' removed 2024-09-21T07:47:42.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.049+0000 7f4debe00640 1 -- 172.21.15.39:0/883613421 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4dcc078b90 msgr2=0x7f4dcc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:42.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.049+0000 7f4debe00640 1 --2- 172.21.15.39:0/883613421 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4dcc078b90 0x7f4dcc07b0a0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc033000 tx=0x7f4ddc004730 comp rx=0 tx=0).stop 2024-09-21T07:47:42.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.049+0000 7f4debe00640 1 -- 172.21.15.39:0/883613421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4dec0b5100 msgr2=0x7f4dec1f5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:42.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.049+0000 7f4debe00640 1 --2- 172.21.15.39:0/883613421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4dec0b5100 0x7f4dec1f5c80 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f4de400c6c0 tx=0x7f4de400cb80 comp rx=0 tx=0).stop 2024-09-21T07:47:42.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.049+0000 7f4debe00640 1 -- 172.21.15.39:0/883613421 shutdown_connections 2024-09-21T07:47:42.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.049+0000 7f4debe00640 1 --2- 172.21.15.39:0/883613421 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4dcc078b90 0x7f4dcc07b0a0 secure :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc033000 tx=0x7f4ddc004730 comp rx=0 tx=0).stop 2024-09-21T07:47:42.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.049+0000 7f4debe00640 1 --2- 172.21.15.39:0/883613421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4dec0b5100 0x7f4dec1f5c80 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:42.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.049+0000 7f4debe00640 1 --2- 172.21.15.39:0/883613421 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4dec0b4bb0 0x7f4dec166990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:42.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.049+0000 7f4debe00640 1 --2- 172.21.15.39:0/883613421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4dec0acf60 0x7f4dec0c3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:42.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.049+0000 7f4debe00640 1 -- 172.21.15.39:0/883613421 >> 172.21.15.39:0/883613421 conn(0x7f4dec0b1e60 msgr2=0x7f4dec0b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:42.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.051+0000 7f4debe00640 1 -- 172.21.15.39:0/883613421 shutdown_connections 2024-09-21T07:47:42.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.052+0000 7f4debe00640 1 -- 172.21.15.39:0/883613421 wait complete. 2024-09-21T07:47:42.079 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T07:47:42.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.314+0000 7f84ddc00640 1 -- 172.21.15.39:0/344549035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d80cba00 msgr2=0x7f84d80cccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:42.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.314+0000 7f84ddc00640 1 --2- 172.21.15.39:0/344549035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d80cba00 0x7f84d80cccf0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f84d80d1190 tx=0x7f84c402a4d0 comp rx=0 tx=0).stop 2024-09-21T07:47:42.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.314+0000 7f84ddc00640 1 -- 172.21.15.39:0/344549035 shutdown_connections 2024-09-21T07:47:42.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.314+0000 7f84ddc00640 1 --2- 172.21.15.39:0/344549035 >> v2:172.21.15.148:3300/0 conn(0x7f84d80cbe10 0x7f84d80cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:42.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.314+0000 7f84ddc00640 1 --2- 172.21.15.39:0/344549035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d80cba00 0x7f84d80cccf0 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:42.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.314+0000 7f84ddc00640 1 --2- 172.21.15.39:0/344549035 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84d815da30 0x7f84d815a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:42.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.314+0000 7f84ddc00640 1 -- 172.21.15.39:0/344549035 >> 172.21.15.39:0/344549035 conn(0x7f84d80b2650 msgr2=0x7f84d80d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:42.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.315+0000 7f84ddc00640 1 -- 172.21.15.39:0/344549035 shutdown_connections 2024-09-21T07:47:42.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.315+0000 7f84ddc00640 1 -- 172.21.15.39:0/344549035 wait complete. 2024-09-21T07:47:42.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.317+0000 7f84ddc00640 1 Processor -- start 2024-09-21T07:47:42.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.317+0000 7f84ddc00640 1 -- start start 2024-09-21T07:47:42.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.318+0000 7f84ddc00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84d80cba00 0x7f84d81521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:42.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.318+0000 7f84ddc00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84d80cbe10 0x7f84d80c4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:42.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.319+0000 7f84ddc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d815da30 0x7f84d8073e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:42.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.319+0000 7f84ddc00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f84d8160bf0 con 0x7f84d815da30 2024-09-21T07:47:42.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.319+0000 7f84ddc00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f84d8160a70 con 0x7f84d80cbe10 2024-09-21T07:47:42.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.319+0000 7f84ddc00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f84d8160d70 con 0x7f84d80cba00 2024-09-21T07:47:42.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.319+0000 7f84d7400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d815da30 0x7f84d8073e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:42.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.319+0000 7f84d7400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d815da30 0x7f84d8073e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37620/0 (socket says 172.21.15.39:37620) 2024-09-21T07:47:42.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.319+0000 7f84d7400640 1 -- 172.21.15.39:0/2200501309 learned_addr learned my addr 172.21.15.39:0/2200501309 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:42.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.319+0000 7f84d6a00640 1 --2- 172.21.15.39:0/2200501309 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84d80cba00 0x7f84d81521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:42.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.319+0000 7f84d6000640 1 --2- 172.21.15.39:0/2200501309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84d80cbe10 0x7f84d80c4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:42.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.319+0000 7f84d7400640 1 -- 172.21.15.39:0/2200501309 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84d80cba00 msgr2=0x7f84d81521b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:42.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.319+0000 7f84d7400640 1 --2- 172.21.15.39:0/2200501309 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84d80cba00 0x7f84d81521b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:42.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.319+0000 7f84d7400640 1 -- 172.21.15.39:0/2200501309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84d80cbe10 msgr2=0x7f84d80c4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:42.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.319+0000 7f84d7400640 1 --2- 172.21.15.39:0/2200501309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84d80cbe10 0x7f84d80c4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:42.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.319+0000 7f84d7400640 1 -- 172.21.15.39:0/2200501309 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d80c75e0 con 0x7f84d815da30 2024-09-21T07:47:42.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.320+0000 7f84d7400640 1 --2- 172.21.15.39:0/2200501309 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d815da30 0x7f84d8073e60 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f84d000e990 tx=0x7f84d000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:42.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.320+0000 7f84cea00640 1 -- 172.21.15.39:0/2200501309 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d0004d20 con 0x7f84d815da30 2024-09-21T07:47:42.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.320+0000 7f84cea00640 1 -- 172.21.15.39:0/2200501309 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f84d000caa0 con 0x7f84d815da30 2024-09-21T07:47:42.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.320+0000 7f84ddc00640 1 -- 172.21.15.39:0/2200501309 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84d80c78c0 con 0x7f84d815da30 2024-09-21T07:47:42.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.320+0000 7f84cea00640 1 -- 172.21.15.39:0/2200501309 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d0005020 con 0x7f84d815da30 2024-09-21T07:47:42.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.320+0000 7f84ddc00640 1 -- 172.21.15.39:0/2200501309 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84d80c7e80 con 0x7f84d815da30 2024-09-21T07:47:42.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.321+0000 7f84ddc00640 1 -- 172.21.15.39:0/2200501309 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84d815a440 con 0x7f84d815da30 2024-09-21T07:47:42.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.322+0000 7f84cea00640 1 -- 172.21.15.39:0/2200501309 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f84d00051c0 con 0x7f84d815da30 2024-09-21T07:47:42.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.323+0000 7f84cea00640 1 --2- 172.21.15.39:0/2200501309 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f84b8078b90 0x7f84b807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:42.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.323+0000 7f84cea00640 1 -- 172.21.15.39:0/2200501309 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(197..197 src has 1..197) ==== 6694+0+0 (secure 0 0 0) 0x7f84d009ba00 con 0x7f84d815da30 2024-09-21T07:47:42.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.324+0000 7f84d6a00640 1 --2- 172.21.15.39:0/2200501309 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f84b8078b90 0x7f84b807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:42.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.325+0000 7f84d6a00640 1 --2- 172.21.15.39:0/2200501309 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f84b8078b90 0x7f84b807b0a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f84c8005fd0 tx=0x7f84c8005850 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:42.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.328+0000 7f84cea00640 1 -- 172.21.15.39:0/2200501309 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f84d0016030 con 0x7f84d815da30 2024-09-21T07:47:42.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:42.524+0000 7f84ddc00640 1 -- 172.21.15.39:0/2200501309 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f84d80c7310 con 0x7f84d815da30 2024-09-21T07:47:43.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.054+0000 7f84cea00640 1 -- 172.21.15.39:0/2200501309 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v198) ==== 169+0+0 (secure 0 0 0) 0x7f84d0063a50 con 0x7f84d815da30 2024-09-21T07:47:43.056 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' removed 2024-09-21T07:47:43.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.058+0000 7f84ddc00640 1 -- 172.21.15.39:0/2200501309 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f84b8078b90 msgr2=0x7f84b807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:43.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.058+0000 7f84ddc00640 1 --2- 172.21.15.39:0/2200501309 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f84b8078b90 0x7f84b807b0a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f84c8005fd0 tx=0x7f84c8005850 comp rx=0 tx=0).stop 2024-09-21T07:47:43.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.058+0000 7f84ddc00640 1 -- 172.21.15.39:0/2200501309 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d815da30 msgr2=0x7f84d8073e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:43.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.058+0000 7f84ddc00640 1 --2- 172.21.15.39:0/2200501309 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d815da30 0x7f84d8073e60 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f84d000e990 tx=0x7f84d000ee50 comp rx=0 tx=0).stop 2024-09-21T07:47:43.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.059+0000 7f84ddc00640 1 -- 172.21.15.39:0/2200501309 shutdown_connections 2024-09-21T07:47:43.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.059+0000 7f84ddc00640 1 --2- 172.21.15.39:0/2200501309 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f84b8078b90 0x7f84b807b0a0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:43.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.059+0000 7f84ddc00640 1 --2- 172.21.15.39:0/2200501309 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d815da30 0x7f84d8073e60 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:43.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.059+0000 7f84ddc00640 1 --2- 172.21.15.39:0/2200501309 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84d80cbe10 0x7f84d80c4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:43.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.059+0000 7f84ddc00640 1 --2- 172.21.15.39:0/2200501309 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84d80cba00 0x7f84d81521b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:43.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.059+0000 7f84ddc00640 1 -- 172.21.15.39:0/2200501309 >> 172.21.15.39:0/2200501309 conn(0x7f84d80b2650 msgr2=0x7f84d8068f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:43.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.062+0000 7f84ddc00640 1 -- 172.21.15.39:0/2200501309 shutdown_connections 2024-09-21T07:47:43.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.062+0000 7f84ddc00640 1 -- 172.21.15.39:0/2200501309 wait complete. 2024-09-21T07:47:43.091 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T07:47:43.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.323+0000 7f5a48a00640 1 -- 172.21.15.39:0/1047183236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a28003a50 msgr2=0x7f5a28003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:43.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.323+0000 7f5a48a00640 1 --2- 172.21.15.39:0/1047183236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a28003a50 0x7f5a28003ec0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f5a34009b50 tx=0x7f5a28005830 comp rx=0 tx=0).stop 2024-09-21T07:47:43.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.323+0000 7f5a48a00640 1 -- 172.21.15.39:0/1047183236 shutdown_connections 2024-09-21T07:47:43.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.323+0000 7f5a48a00640 1 --2- 172.21.15.39:0/1047183236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a28003a50 0x7f5a28003ec0 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:43.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.323+0000 7f5a48a00640 1 -- 172.21.15.39:0/1047183236 >> 172.21.15.39:0/1047183236 conn(0x7f5a440b2460 msgr2=0x7f5a4406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:43.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.323+0000 7f5a48a00640 1 -- 172.21.15.39:0/1047183236 shutdown_connections 2024-09-21T07:47:43.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.323+0000 7f5a48a00640 1 -- 172.21.15.39:0/1047183236 wait complete. 2024-09-21T07:47:43.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.326+0000 7f5a48a00640 1 Processor -- start 2024-09-21T07:47:43.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.326+0000 7f5a48a00640 1 -- start start 2024-09-21T07:47:43.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.327+0000 7f5a48a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a28003a50 0x7f5a441f3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:43.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.327+0000 7f5a48a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5a441f6230 0x7f5a440ad6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:43.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.327+0000 7f5a48a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a441f6640 0x7f5a440af2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:43.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.327+0000 7f5a48a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5a44156bb0 con 0x7f5a441f6640 2024-09-21T07:47:43.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.327+0000 7f5a48a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5a440cbbc0 con 0x7f5a28003a50 2024-09-21T07:47:43.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.327+0000 7f5a48a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f5a44156d30 con 0x7f5a441f6230 2024-09-21T07:47:43.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.327+0000 7f5a42800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a441f6640 0x7f5a440af2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:43.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.327+0000 7f5a42800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a441f6640 0x7f5a440af2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37634/0 (socket says 172.21.15.39:37634) 2024-09-21T07:47:43.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.328+0000 7f5a42800640 1 -- 172.21.15.39:0/2508693731 learned_addr learned my addr 172.21.15.39:0/2508693731 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:43.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.328+0000 7f5a41e00640 1 --2- 172.21.15.39:0/2508693731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a28003a50 0x7f5a441f3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:43.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.328+0000 7f5a3be00640 1 --2- 172.21.15.39:0/2508693731 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5a441f6230 0x7f5a440ad6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:43.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.328+0000 7f5a42800640 1 -- 172.21.15.39:0/2508693731 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5a441f6230 msgr2=0x7f5a440ad6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:43.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.328+0000 7f5a42800640 1 --2- 172.21.15.39:0/2508693731 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5a441f6230 0x7f5a440ad6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:43.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.328+0000 7f5a42800640 1 -- 172.21.15.39:0/2508693731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a28003a50 msgr2=0x7f5a441f3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:43.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.328+0000 7f5a42800640 1 --2- 172.21.15.39:0/2508693731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a28003a50 0x7f5a441f3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:43.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.328+0000 7f5a42800640 1 -- 172.21.15.39:0/2508693731 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a44156390 con 0x7f5a441f6640 2024-09-21T07:47:43.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.328+0000 7f5a3be00640 1 --2- 172.21.15.39:0/2508693731 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5a441f6230 0x7f5a440ad6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:47:43.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.328+0000 7f5a41e00640 1 --2- 172.21.15.39:0/2508693731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a28003a50 0x7f5a441f3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:47:43.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.328+0000 7f5a42800640 1 --2- 172.21.15.39:0/2508693731 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a441f6640 0x7f5a440af2d0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f5a440b50c0 tx=0x7f5a3c0019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:43.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.328+0000 7f5a3a000640 1 -- 172.21.15.39:0/2508693731 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c011870 con 0x7f5a441f6640 2024-09-21T07:47:43.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.329+0000 7f5a3a000640 1 -- 172.21.15.39:0/2508693731 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a3c017070 con 0x7f5a441f6640 2024-09-21T07:47:43.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.329+0000 7f5a48a00640 1 -- 172.21.15.39:0/2508693731 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a440ae230 con 0x7f5a441f6640 2024-09-21T07:47:43.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.329+0000 7f5a3a000640 1 -- 172.21.15.39:0/2508693731 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c011dc0 con 0x7f5a441f6640 2024-09-21T07:47:43.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.329+0000 7f5a48a00640 1 -- 172.21.15.39:0/2508693731 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a440ae7f0 con 0x7f5a441f6640 2024-09-21T07:47:43.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.330+0000 7f5a48a00640 1 -- 172.21.15.39:0/2508693731 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a440cd070 con 0x7f5a441f6640 2024-09-21T07:47:43.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.331+0000 7f5a3a000640 1 -- 172.21.15.39:0/2508693731 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5a3c01c460 con 0x7f5a441f6640 2024-09-21T07:47:43.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.332+0000 7f5a3a000640 1 --2- 172.21.15.39:0/2508693731 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5a20078b90 0x7f5a2007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:43.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.333+0000 7f5a3a000640 1 -- 172.21.15.39:0/2508693731 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(198..198 src has 1..198) ==== 6280+0+0 (secure 0 0 0) 0x7f5a3c09b870 con 0x7f5a441f6640 2024-09-21T07:47:43.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.333+0000 7f5a41e00640 1 --2- 172.21.15.39:0/2508693731 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5a20078b90 0x7f5a2007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:43.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.334+0000 7f5a41e00640 1 --2- 172.21.15.39:0/2508693731 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5a20078b90 0x7f5a2007b0a0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f5a34035000 tx=0x7f5a340046c0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:43.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.337+0000 7f5a3a000640 1 -- 172.21.15.39:0/2508693731 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5a3c0639d0 con 0x7f5a441f6640 2024-09-21T07:47:43.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.533+0000 7f5a48a00640 1 -- 172.21.15.39:0/2508693731 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f5a440adfc0 con 0x7f5a441f6640 2024-09-21T07:47:43.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.538+0000 7f5a3a000640 1 -- 172.21.15.39:0/2508693731 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v50) ==== 71+0+0 (secure 0 0 0) 0x7f5a3c0632d0 con 0x7f5a441f6640 2024-09-21T07:47:43.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.540+0000 7f5a48a00640 1 -- 172.21.15.39:0/2508693731 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5a20078b90 msgr2=0x7f5a2007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:43.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.540+0000 7f5a48a00640 1 --2- 172.21.15.39:0/2508693731 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5a20078b90 0x7f5a2007b0a0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f5a34035000 tx=0x7f5a340046c0 comp rx=0 tx=0).stop 2024-09-21T07:47:43.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.540+0000 7f5a48a00640 1 -- 172.21.15.39:0/2508693731 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a441f6640 msgr2=0x7f5a440af2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:43.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.540+0000 7f5a48a00640 1 --2- 172.21.15.39:0/2508693731 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a441f6640 0x7f5a440af2d0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f5a440b50c0 tx=0x7f5a3c0019b0 comp rx=0 tx=0).stop 2024-09-21T07:47:43.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.540+0000 7f5a48a00640 1 -- 172.21.15.39:0/2508693731 shutdown_connections 2024-09-21T07:47:43.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.540+0000 7f5a48a00640 1 --2- 172.21.15.39:0/2508693731 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5a20078b90 0x7f5a2007b0a0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:43.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.540+0000 7f5a48a00640 1 --2- 172.21.15.39:0/2508693731 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a441f6640 0x7f5a440af2d0 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:43.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.540+0000 7f5a48a00640 1 --2- 172.21.15.39:0/2508693731 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5a441f6230 0x7f5a440ad6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:43.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.540+0000 7f5a48a00640 1 --2- 172.21.15.39:0/2508693731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a28003a50 0x7f5a441f3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:43.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.540+0000 7f5a48a00640 1 -- 172.21.15.39:0/2508693731 >> 172.21.15.39:0/2508693731 conn(0x7f5a440b2460 msgr2=0x7f5a440c9310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:43.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.540+0000 7f5a48a00640 1 -- 172.21.15.39:0/2508693731 shutdown_connections 2024-09-21T07:47:43.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.540+0000 7f5a48a00640 1 -- 172.21.15.39:0/2508693731 wait complete. 2024-09-21T07:47:43.563 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_multimds_hardlink' 2024-09-21T07:47:43.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.792+0000 7fe0a5000640 1 -- 172.21.15.39:0/3250837415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0a00d04a0 msgr2=0x7fe0a00d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:43.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.792+0000 7fe0a5000640 1 --2- 172.21.15.39:0/3250837415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0a00d04a0 0x7fe0a00d08b0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fe0a00d0fa0 tx=0x7fe09802aa10 comp rx=0 tx=0).stop 2024-09-21T07:47:43.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.792+0000 7fe0a5000640 1 -- 172.21.15.39:0/3250837415 shutdown_connections 2024-09-21T07:47:43.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.792+0000 7fe0a5000640 1 --2- 172.21.15.39:0/3250837415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0a00d04a0 0x7fe0a00d08b0 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:43.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.792+0000 7fe0a5000640 1 --2- 172.21.15.39:0/3250837415 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe0a015bb00 0x7fe0a015bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:43.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.792+0000 7fe0a5000640 1 --2- 172.21.15.39:0/3250837415 >> v2:172.21.15.148:3300/0 conn(0x7fe0a0152590 0x7fe0a00b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:43.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.792+0000 7fe0a5000640 1 -- 172.21.15.39:0/3250837415 >> 172.21.15.39:0/3250837415 conn(0x7fe0a006cfc0 msgr2=0x7fe0a006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:43.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.792+0000 7fe0a5000640 1 -- 172.21.15.39:0/3250837415 shutdown_connections 2024-09-21T07:47:43.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.792+0000 7fe0a5000640 1 -- 172.21.15.39:0/3250837415 wait complete. 2024-09-21T07:47:43.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.794+0000 7fe0a5000640 1 Processor -- start 2024-09-21T07:47:43.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.795+0000 7fe0a5000640 1 -- start start 2024-09-21T07:47:43.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.796+0000 7fe0a5000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0a00d04a0 0x7fe0a0164630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:43.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.796+0000 7fe0a5000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe0a0152590 0x7fe0a0164b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:43.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.796+0000 7fe0a5000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0a015bb00 0x7fe0a014ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:43.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.796+0000 7fe0a5000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a0072da0 con 0x7fe0a015bb00 2024-09-21T07:47:43.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.796+0000 7fe0a5000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a00b4bd0 con 0x7fe0a00d04a0 2024-09-21T07:47:43.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.796+0000 7fe0a5000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe0a0072f20 con 0x7fe0a0152590 2024-09-21T07:47:43.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.796+0000 7fe09fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0a015bb00 0x7fe0a014ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:43.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.796+0000 7fe09fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0a015bb00 0x7fe0a014ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37650/0 (socket says 172.21.15.39:37650) 2024-09-21T07:47:43.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.796+0000 7fe09fe00640 1 -- 172.21.15.39:0/1877241995 learned_addr learned my addr 172.21.15.39:0/1877241995 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:43.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.796+0000 7fe09f400640 1 --2- 172.21.15.39:0/1877241995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0a00d04a0 0x7fe0a0164630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:43.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.797+0000 7fe097e00640 1 --2- 172.21.15.39:0/1877241995 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe0a0152590 0x7fe0a0164b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:43.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.797+0000 7fe09fe00640 1 -- 172.21.15.39:0/1877241995 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe0a0152590 msgr2=0x7fe0a0164b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:43.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.797+0000 7fe09fe00640 1 --2- 172.21.15.39:0/1877241995 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe0a0152590 0x7fe0a0164b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:43.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.797+0000 7fe09fe00640 1 -- 172.21.15.39:0/1877241995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0a00d04a0 msgr2=0x7fe0a0164630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:43.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.797+0000 7fe09fe00640 1 --2- 172.21.15.39:0/1877241995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0a00d04a0 0x7fe0a0164630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:43.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.797+0000 7fe09fe00640 1 -- 172.21.15.39:0/1877241995 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0a014a8d0 con 0x7fe0a015bb00 2024-09-21T07:47:43.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.797+0000 7fe097e00640 1 --2- 172.21.15.39:0/1877241995 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe0a0152590 0x7fe0a0164b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:43.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.797+0000 7fe09fe00640 1 --2- 172.21.15.39:0/1877241995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0a015bb00 0x7fe0a014ad80 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7fe098005fd0 tx=0x7fe098003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:43.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.797+0000 7fe097400640 1 -- 172.21.15.39:0/1877241995 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe098033050 con 0x7fe0a015bb00 2024-09-21T07:47:43.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.797+0000 7fe097400640 1 -- 172.21.15.39:0/1877241995 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe098004450 con 0x7fe0a015bb00 2024-09-21T07:47:43.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.797+0000 7fe097400640 1 -- 172.21.15.39:0/1877241995 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe098004860 con 0x7fe0a015bb00 2024-09-21T07:47:43.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.797+0000 7fe0a5000640 1 -- 172.21.15.39:0/1877241995 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0a01f90b0 con 0x7fe0a015bb00 2024-09-21T07:47:43.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.798+0000 7fe0a5000640 1 -- 172.21.15.39:0/1877241995 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0a01f9670 con 0x7fe0a015bb00 2024-09-21T07:47:43.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.799+0000 7fe0a5000640 1 -- 172.21.15.39:0/1877241995 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0a00b3950 con 0x7fe0a015bb00 2024-09-21T07:47:43.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.800+0000 7fe097400640 1 -- 172.21.15.39:0/1877241995 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe098036070 con 0x7fe0a015bb00 2024-09-21T07:47:43.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.801+0000 7fe097400640 1 --2- 172.21.15.39:0/1877241995 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe07c078bc0 0x7fe07c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:43.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.801+0000 7fe097400640 1 -- 172.21.15.39:0/1877241995 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(198..198 src has 1..198) ==== 6280+0+0 (secure 0 0 0) 0x7fe0980bc450 con 0x7fe0a015bb00 2024-09-21T07:47:43.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.801+0000 7fe09f400640 1 --2- 172.21.15.39:0/1877241995 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe07c078bc0 0x7fe07c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:43.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.802+0000 7fe09f400640 1 --2- 172.21.15.39:0/1877241995 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe07c078bc0 0x7fe07c07b0d0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fe09000bb50 tx=0x7fe09000b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:43.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:43.805+0000 7fe097400640 1 -- 172.21.15.39:0/1877241995 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe0980845b0 con 0x7fe0a015bb00 2024-09-21T07:47:44.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:44.084+0000 7fe0a5000640 1 -- 172.21.15.39:0/1877241995 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_multimds_hardlink"]} v 0) -- 0x7fe0a01f8e00 con 0x7fe0a015bb00 2024-09-21T07:47:45.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.059+0000 7fe097400640 1 -- 172.21.15.39:0/1877241995 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_multimds_hardlink"]}]=0 v707) ==== 143+0+0 (secure 0 0 0) 0x7fe098083eb0 con 0x7fe0a015bb00 2024-09-21T07:47:45.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.062+0000 7fe0a5000640 1 -- 172.21.15.39:0/1877241995 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe07c078bc0 msgr2=0x7fe07c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:45.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.062+0000 7fe0a5000640 1 --2- 172.21.15.39:0/1877241995 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe07c078bc0 0x7fe07c07b0d0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fe09000bb50 tx=0x7fe09000b2e0 comp rx=0 tx=0).stop 2024-09-21T07:47:45.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.062+0000 7fe0a5000640 1 -- 172.21.15.39:0/1877241995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0a015bb00 msgr2=0x7fe0a014ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:45.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.062+0000 7fe0a5000640 1 --2- 172.21.15.39:0/1877241995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0a015bb00 0x7fe0a014ad80 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7fe098005fd0 tx=0x7fe098003f10 comp rx=0 tx=0).stop 2024-09-21T07:47:45.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.062+0000 7fe0a5000640 1 -- 172.21.15.39:0/1877241995 shutdown_connections 2024-09-21T07:47:45.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.062+0000 7fe0a5000640 1 --2- 172.21.15.39:0/1877241995 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe07c078bc0 0x7fe07c07b0d0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:45.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.062+0000 7fe0a5000640 1 --2- 172.21.15.39:0/1877241995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0a015bb00 0x7fe0a014ad80 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:45.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.062+0000 7fe0a5000640 1 --2- 172.21.15.39:0/1877241995 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe0a0152590 0x7fe0a0164b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:45.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.062+0000 7fe0a5000640 1 --2- 172.21.15.39:0/1877241995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0a00d04a0 0x7fe0a0164630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:45.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.062+0000 7fe0a5000640 1 -- 172.21.15.39:0/1877241995 >> 172.21.15.39:0/1877241995 conn(0x7fe0a006cfc0 msgr2=0x7fe0a01659e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:45.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.062+0000 7fe0a5000640 1 -- 172.21.15.39:0/1877241995 shutdown_connections 2024-09-21T07:47:45.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.062+0000 7fe0a5000640 1 -- 172.21.15.39:0/1877241995 wait complete. 2024-09-21T07:47:45.089 INFO:tasks.cephfs_test_runner:check if hardlink snapshot works in multimds setup ... ok 2024-09-21T07:47:45.089 INFO:tasks.cephfs_test_runner:Starting test: test_multimds_mksnap (tasks.cephfs.test_snapshots.TestSnapshots) check if snapshot takes effect across authority subtrees 2024-09-21T07:47:45.089 INFO:tasks.cephfs_test_runner:test_multimds_mksnap (tasks.cephfs.test_snapshots.TestSnapshots) 2024-09-21T07:47:45.089 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:47:45.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.319+0000 7f7895800640 1 -- 172.21.15.39:0/1535690796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7890153870 msgr2=0x7f78900cde20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:45.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.319+0000 7f7895800640 1 --2- 172.21.15.39:0/1535690796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7890153870 0x7f78900cde20 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f7888005fd0 tx=0x7f788802aa30 comp rx=0 tx=0).stop 2024-09-21T07:47:45.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.319+0000 7f7895800640 1 -- 172.21.15.39:0/1535690796 shutdown_connections 2024-09-21T07:47:45.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.319+0000 7f7895800640 1 --2- 172.21.15.39:0/1535690796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7890153870 0x7f78900cde20 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:45.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.319+0000 7f7895800640 1 -- 172.21.15.39:0/1535690796 >> 172.21.15.39:0/1535690796 conn(0x7f78900d0c70 msgr2=0x7f78900d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:45.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.319+0000 7f7895800640 1 -- 172.21.15.39:0/1535690796 shutdown_connections 2024-09-21T07:47:45.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.319+0000 7f7895800640 1 -- 172.21.15.39:0/1535690796 wait complete. 2024-09-21T07:47:45.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.321+0000 7f7895800640 1 Processor -- start 2024-09-21T07:47:45.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.322+0000 7f7895800640 1 -- start start 2024-09-21T07:47:45.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.323+0000 7f7895800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78900b5ac0 0x7f78901445f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:45.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.323+0000 7f7895800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7890153870 0x7f789014ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:45.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.323+0000 7f7895800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7890156000 0x7f7890156410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:45.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.323+0000 7f7895800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f789015d750 con 0x7f7890153870 2024-09-21T07:47:45.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.323+0000 7f7895800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f789015d5d0 con 0x7f7890156000 2024-09-21T07:47:45.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.323+0000 7f7895800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f789015d8d0 con 0x7f78900b5ac0 2024-09-21T07:47:45.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.323+0000 7f7887e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7890153870 0x7f789014ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:45.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.323+0000 7f7887e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7890153870 0x7f789014ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37664/0 (socket says 172.21.15.39:37664) 2024-09-21T07:47:45.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.323+0000 7f7887e00640 1 -- 172.21.15.39:0/1987497056 learned_addr learned my addr 172.21.15.39:0/1987497056 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:45.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.324+0000 7f7887e00640 1 -- 172.21.15.39:0/1987497056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78900b5ac0 msgr2=0x7f78901445f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:45.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.324+0000 7f788fe00640 1 --2- 172.21.15.39:0/1987497056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78900b5ac0 0x7f78901445f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:45.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.324+0000 7f7894e00640 1 --2- 172.21.15.39:0/1987497056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7890156000 0x7f7890156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:45.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.324+0000 7f7887e00640 1 --2- 172.21.15.39:0/1987497056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78900b5ac0 0x7f78901445f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:45.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.324+0000 7f7887e00640 1 -- 172.21.15.39:0/1987497056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7890156000 msgr2=0x7f7890156410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:45.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.324+0000 7f7887e00640 1 --2- 172.21.15.39:0/1987497056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7890156000 0x7f7890156410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:45.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.324+0000 7f7887e00640 1 -- 172.21.15.39:0/1987497056 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7890156950 con 0x7f7890153870 2024-09-21T07:47:45.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.324+0000 7f7894e00640 1 --2- 172.21.15.39:0/1987497056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7890156000 0x7f7890156410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:47:45.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.324+0000 7f788fe00640 1 --2- 172.21.15.39:0/1987497056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78900b5ac0 0x7f78901445f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:47:45.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.324+0000 7f7887e00640 1 --2- 172.21.15.39:0/1987497056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7890153870 0x7f789014ddd0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f787800c730 tx=0x7f787800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:45.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.324+0000 7f788cc00640 1 -- 172.21.15.39:0/1987497056 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7878017070 con 0x7f7890153870 2024-09-21T07:47:45.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.324+0000 7f788cc00640 1 -- 172.21.15.39:0/1987497056 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7878005380 con 0x7f7890153870 2024-09-21T07:47:45.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.324+0000 7f788cc00640 1 -- 172.21.15.39:0/1987497056 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7878003850 con 0x7f7890153870 2024-09-21T07:47:45.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.324+0000 7f7895800640 1 -- 172.21.15.39:0/1987497056 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7890154210 con 0x7f7890153870 2024-09-21T07:47:45.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.325+0000 7f7895800640 1 -- 172.21.15.39:0/1987497056 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78901547d0 con 0x7f7890153870 2024-09-21T07:47:45.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.325+0000 7f7895800640 1 -- 172.21.15.39:0/1987497056 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78900cde20 con 0x7f7890153870 2024-09-21T07:47:45.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.327+0000 7f788cc00640 1 -- 172.21.15.39:0/1987497056 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f787801a070 con 0x7f7890153870 2024-09-21T07:47:45.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.328+0000 7f788cc00640 1 --2- 172.21.15.39:0/1987497056 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f786c078b90 0x7f786c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:45.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.328+0000 7f788cc00640 1 -- 172.21.15.39:0/1987497056 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(198..198 src has 1..198) ==== 6280+0+0 (secure 0 0 0) 0x7f787809a3d0 con 0x7f7890153870 2024-09-21T07:47:45.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.328+0000 7f788fe00640 1 --2- 172.21.15.39:0/1987497056 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f786c078b90 0x7f786c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:45.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.329+0000 7f788fe00640 1 --2- 172.21.15.39:0/1987497056 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f786c078b90 0x7f786c07b0a0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f78800016a0 tx=0x7f7880009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:45.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.333+0000 7f788cc00640 1 -- 172.21.15.39:0/1987497056 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7890156950 con 0x7f7890153870 2024-09-21T07:47:45.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.527+0000 7f7895800640 1 -- 172.21.15.39:0/1987497056 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7890153fa0 con 0x7f7890153870 2024-09-21T07:47:45.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.529+0000 7f788cc00640 1 -- 172.21.15.39:0/1987497056 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v198) ==== 74+0+18894 (secure 0 0 0) 0x7f78780625c0 con 0x7f7890153870 2024-09-21T07:47:45.531 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:47:45.531 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":198,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:47:43.046374+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6835/3348864735":"2024-09-22T07:47:40.030263+0000","172.21.15.39:6834/3348864735":"2024-09-22T07:47:40.030263+0000","172.21.15.39:6833/3199732683":"2024-09-22T07:47:40.030243+0000","172.21.15.39:6832/3199732683":"2024-09-22T07:47:40.030243+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]},{"pool":20,"snaps":[{"begin":80,"length":23},{"begin":181,"length":23}]},{"pool":22,"snaps":[{"begin":100,"length":1},{"begin":199,"length":1}]},{"pool":24,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:47:45.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.531+0000 7f7895800640 1 -- 172.21.15.39:0/1987497056 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f786c078b90 msgr2=0x7f786c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:45.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.531+0000 7f7895800640 1 --2- 172.21.15.39:0/1987497056 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f786c078b90 0x7f786c07b0a0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f78800016a0 tx=0x7f7880009040 comp rx=0 tx=0).stop 2024-09-21T07:47:45.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.531+0000 7f7895800640 1 -- 172.21.15.39:0/1987497056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7890153870 msgr2=0x7f789014ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:45.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.531+0000 7f7895800640 1 --2- 172.21.15.39:0/1987497056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7890153870 0x7f789014ddd0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f787800c730 tx=0x7f787800cbf0 comp rx=0 tx=0).stop 2024-09-21T07:47:45.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.531+0000 7f7895800640 1 -- 172.21.15.39:0/1987497056 shutdown_connections 2024-09-21T07:47:45.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.531+0000 7f7895800640 1 --2- 172.21.15.39:0/1987497056 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f786c078b90 0x7f786c07b0a0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:45.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.531+0000 7f7895800640 1 --2- 172.21.15.39:0/1987497056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7890156000 0x7f7890156410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:45.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.531+0000 7f7895800640 1 --2- 172.21.15.39:0/1987497056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7890153870 0x7f789014ddd0 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:45.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.531+0000 7f7895800640 1 --2- 172.21.15.39:0/1987497056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78900b5ac0 0x7f78901445f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:45.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.532+0000 7f7895800640 1 -- 172.21.15.39:0/1987497056 >> 172.21.15.39:0/1987497056 conn(0x7f78900d0c70 msgr2=0x7f789006ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:45.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.532+0000 7f7895800640 1 -- 172.21.15.39:0/1987497056 shutdown_connections 2024-09-21T07:47:45.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.532+0000 7f7895800640 1 -- 172.21.15.39:0/1987497056 wait complete. 2024-09-21T07:47:45.557 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T07:33:01.569433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'snap-test-pool', 'create_time': '2024-09-21T07:34:22.032185+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 1, 'snap_epoch': 37, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'test_data_pool', 'create_time': '2024-09-21T07:35:25.604532+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 1, 'snap_epoch': 66, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_metadata_pool', 'create_time': '2024-09-21T07:35:26.327607+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T07:47:45.558 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T07:47:45.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.787+0000 7fa237e00640 1 -- 172.21.15.39:0/2562675927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2380743b0 msgr2=0x7fa23815b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:45.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.787+0000 7fa237e00640 1 --2- 172.21.15.39:0/2562675927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2380743b0 0x7fa23815b4f0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7fa220005fd0 tx=0x7fa22002a5d0 comp rx=0 tx=0).stop 2024-09-21T07:47:45.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.787+0000 7fa237e00640 1 -- 172.21.15.39:0/2562675927 shutdown_connections 2024-09-21T07:47:45.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.787+0000 7fa237e00640 1 --2- 172.21.15.39:0/2562675927 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa23806fee0 0x7fa2380702f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:45.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.787+0000 7fa237e00640 1 --2- 172.21.15.39:0/2562675927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2380743b0 0x7fa23815b4f0 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:45.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.787+0000 7fa237e00640 1 -- 172.21.15.39:0/2562675927 >> 172.21.15.39:0/2562675927 conn(0x7fa2380b6f80 msgr2=0x7fa23806def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:45.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.787+0000 7fa237e00640 1 -- 172.21.15.39:0/2562675927 shutdown_connections 2024-09-21T07:47:45.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.787+0000 7fa237e00640 1 -- 172.21.15.39:0/2562675927 wait complete. 2024-09-21T07:47:45.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.789+0000 7fa237e00640 1 Processor -- start 2024-09-21T07:47:45.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.790+0000 7fa237e00640 1 -- start start 2024-09-21T07:47:45.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.791+0000 7fa237e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa23806fee0 0x7fa238150e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:45.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.791+0000 7fa237e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2380743b0 0x7fa238153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:45.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.791+0000 7fa237e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2380b05d0 0x7fa238153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:45.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.791+0000 7fa237e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa2380b5090 con 0x7fa2380743b0 2024-09-21T07:47:45.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.791+0000 7fa237e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa2380b4f10 con 0x7fa2380b05d0 2024-09-21T07:47:45.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.791+0000 7fa237e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa2380b5210 con 0x7fa23806fee0 2024-09-21T07:47:45.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.791+0000 7fa22e000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2380743b0 0x7fa238153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:45.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.791+0000 7fa22e000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2380743b0 0x7fa238153ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37684/0 (socket says 172.21.15.39:37684) 2024-09-21T07:47:45.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.791+0000 7fa22e000640 1 -- 172.21.15.39:0/3081696133 learned_addr learned my addr 172.21.15.39:0/3081696133 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:45.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.791+0000 7fa236a00640 1 --2- 172.21.15.39:0/3081696133 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa23806fee0 0x7fa238150e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:45.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.791+0000 7fa237400640 1 --2- 172.21.15.39:0/3081696133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2380b05d0 0x7fa238153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:45.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.792+0000 7fa22e000640 1 -- 172.21.15.39:0/3081696133 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa23806fee0 msgr2=0x7fa238150e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:45.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.792+0000 7fa22e000640 1 --2- 172.21.15.39:0/3081696133 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa23806fee0 0x7fa238150e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:45.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.792+0000 7fa22e000640 1 -- 172.21.15.39:0/3081696133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2380b05d0 msgr2=0x7fa238153ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:45.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.792+0000 7fa22e000640 1 --2- 172.21.15.39:0/3081696133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2380b05d0 0x7fa238153ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:45.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.792+0000 7fa22e000640 1 -- 172.21.15.39:0/3081696133 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa238153060 con 0x7fa2380743b0 2024-09-21T07:47:45.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.792+0000 7fa22e000640 1 --2- 172.21.15.39:0/3081696133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2380743b0 0x7fa238153ab0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7fa220034040 tx=0x7fa2200035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:45.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.792+0000 7fa22f400640 1 -- 172.21.15.39:0/3081696133 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22003e030 con 0x7fa2380743b0 2024-09-21T07:47:45.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.793+0000 7fa22f400640 1 -- 172.21.15.39:0/3081696133 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa220003ae0 con 0x7fa2380743b0 2024-09-21T07:47:45.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.793+0000 7fa22f400640 1 -- 172.21.15.39:0/3081696133 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa220035040 con 0x7fa2380743b0 2024-09-21T07:47:45.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.793+0000 7fa237e00640 1 -- 172.21.15.39:0/3081696133 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2380b0e20 con 0x7fa2380743b0 2024-09-21T07:47:45.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.793+0000 7fa237e00640 1 -- 172.21.15.39:0/3081696133 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2381f5590 con 0x7fa2380743b0 2024-09-21T07:47:45.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.794+0000 7fa237e00640 1 -- 172.21.15.39:0/3081696133 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2380b5700 con 0x7fa2380743b0 2024-09-21T07:47:45.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.795+0000 7fa22f400640 1 -- 172.21.15.39:0/3081696133 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa220041030 con 0x7fa2380743b0 2024-09-21T07:47:45.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.796+0000 7fa22f400640 1 --2- 172.21.15.39:0/3081696133 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa214078b90 0x7fa21407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:45.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.796+0000 7fa22f400640 1 -- 172.21.15.39:0/3081696133 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(198..198 src has 1..198) ==== 6280+0+0 (secure 0 0 0) 0x7fa2200bb490 con 0x7fa2380743b0 2024-09-21T07:47:45.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.796+0000 7fa236a00640 1 --2- 172.21.15.39:0/3081696133 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa214078b90 0x7fa21407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:45.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.797+0000 7fa236a00640 1 --2- 172.21.15.39:0/3081696133 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa214078b90 0x7fa21407b0a0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fa228001720 tx=0x7fa228009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:45.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.801+0000 7fa22f400640 1 -- 172.21.15.39:0/3081696133 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa220083030 con 0x7fa2380743b0 2024-09-21T07:47:45.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.995+0000 7fa237e00640 1 -- 172.21.15.39:0/3081696133 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa238152df0 con 0x7fa2380743b0 2024-09-21T07:47:45.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.996+0000 7fa22f400640 1 -- 172.21.15.39:0/3081696133 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v198) ==== 93+0+10 (secure 0 0 0) 0x7fa220087a60 con 0x7fa2380743b0 2024-09-21T07:47:45.998 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-09-21T07:47:46.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.998+0000 7fa237e00640 1 -- 172.21.15.39:0/3081696133 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa214078b90 msgr2=0x7fa21407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:46.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.998+0000 7fa237e00640 1 --2- 172.21.15.39:0/3081696133 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa214078b90 0x7fa21407b0a0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fa228001720 tx=0x7fa228009040 comp rx=0 tx=0).stop 2024-09-21T07:47:46.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.998+0000 7fa237e00640 1 -- 172.21.15.39:0/3081696133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2380743b0 msgr2=0x7fa238153ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:46.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.998+0000 7fa237e00640 1 --2- 172.21.15.39:0/3081696133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2380743b0 0x7fa238153ab0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7fa220034040 tx=0x7fa2200035f0 comp rx=0 tx=0).stop 2024-09-21T07:47:46.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.998+0000 7fa237e00640 1 -- 172.21.15.39:0/3081696133 shutdown_connections 2024-09-21T07:47:46.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.998+0000 7fa237e00640 1 --2- 172.21.15.39:0/3081696133 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa214078b90 0x7fa21407b0a0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:46.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.998+0000 7fa237e00640 1 --2- 172.21.15.39:0/3081696133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2380b05d0 0x7fa238153ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:46.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.998+0000 7fa237e00640 1 --2- 172.21.15.39:0/3081696133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2380743b0 0x7fa238153ab0 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:46.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.998+0000 7fa237e00640 1 --2- 172.21.15.39:0/3081696133 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa23806fee0 0x7fa238150e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:46.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.998+0000 7fa237e00640 1 -- 172.21.15.39:0/3081696133 >> 172.21.15.39:0/3081696133 conn(0x7fa2380b6f80 msgr2=0x7fa238151640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:46.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.998+0000 7fa237e00640 1 -- 172.21.15.39:0/3081696133 shutdown_connections 2024-09-21T07:47:46.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:45.998+0000 7fa237e00640 1 -- 172.21.15.39:0/3081696133 wait complete. 2024-09-21T07:47:46.025 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2024-09-21T07:47:46.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.256+0000 7f727d400640 1 -- 172.21.15.39:0/1766265586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7278152490 msgr2=0x7f72780cd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:46.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.256+0000 7f727d400640 1 --2- 172.21.15.39:0/1766265586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7278152490 0x7f72780cd070 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f7278073430 tx=0x7f726002a560 comp rx=0 tx=0).stop 2024-09-21T07:47:46.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.256+0000 7f727d400640 1 -- 172.21.15.39:0/1766265586 shutdown_connections 2024-09-21T07:47:46.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.256+0000 7f727d400640 1 --2- 172.21.15.39:0/1766265586 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f72781528a0 0x7f727806ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:46.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.256+0000 7f727d400640 1 --2- 172.21.15.39:0/1766265586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7278152490 0x7f72780cd070 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:46.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.256+0000 7f727d400640 1 --2- 172.21.15.39:0/1766265586 >> v2:172.21.15.148:3300/0 conn(0x7f727815bbd0 0x7f72780b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:46.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.256+0000 7f727d400640 1 -- 172.21.15.39:0/1766265586 >> 172.21.15.39:0/1766265586 conn(0x7f7278073980 msgr2=0x7f7278073d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:46.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.256+0000 7f727d400640 1 -- 172.21.15.39:0/1766265586 shutdown_connections 2024-09-21T07:47:46.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.256+0000 7f727d400640 1 -- 172.21.15.39:0/1766265586 wait complete. 2024-09-21T07:47:46.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.258+0000 7f727d400640 1 Processor -- start 2024-09-21T07:47:46.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.258+0000 7f727d400640 1 -- start start 2024-09-21T07:47:46.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.260+0000 7f727d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7278152490 0x7f72780a9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:46.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.260+0000 7f727d400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72781528a0 0x7f72781f5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:46.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.260+0000 7f727d400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f727815bbd0 0x7f7278154cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:46.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.260+0000 7f727d400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7278070c60 con 0x7f7278152490 2024-09-21T07:47:46.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.260+0000 7f727d400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f727806e680 con 0x7f72781528a0 2024-09-21T07:47:46.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.260+0000 7f727d400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f7278070de0 con 0x7f727815bbd0 2024-09-21T07:47:46.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.260+0000 7f7276800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7278152490 0x7f72780a9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:46.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.260+0000 7f7276800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7278152490 0x7f72780a9fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37712/0 (socket says 172.21.15.39:37712) 2024-09-21T07:47:46.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.260+0000 7f7276800640 1 -- 172.21.15.39:0/1841358441 learned_addr learned my addr 172.21.15.39:0/1841358441 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:46.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.260+0000 7f726de00640 1 --2- 172.21.15.39:0/1841358441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72781528a0 0x7f72781f5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:46.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.260+0000 7f7277200640 1 --2- 172.21.15.39:0/1841358441 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f727815bbd0 0x7f7278154cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:46.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.260+0000 7f7276800640 1 -- 172.21.15.39:0/1841358441 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f727815bbd0 msgr2=0x7f7278154cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:46.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.261+0000 7f7276800640 1 --2- 172.21.15.39:0/1841358441 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f727815bbd0 0x7f7278154cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:46.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.261+0000 7f7276800640 1 -- 172.21.15.39:0/1841358441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72781528a0 msgr2=0x7f72781f5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:46.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.261+0000 7f7276800640 1 --2- 172.21.15.39:0/1841358441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72781528a0 0x7f72781f5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:46.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.261+0000 7f7276800640 1 -- 172.21.15.39:0/1841358441 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72781e4000 con 0x7f7278152490 2024-09-21T07:47:46.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.261+0000 7f7277200640 1 --2- 172.21.15.39:0/1841358441 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f727815bbd0 0x7f7278154cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:47:46.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.261+0000 7f7276800640 1 --2- 172.21.15.39:0/1841358441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7278152490 0x7f72780a9fc0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f726800d770 tx=0x7f726800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:46.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.261+0000 7f726f400640 1 -- 172.21.15.39:0/1841358441 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7268017070 con 0x7f7278152490 2024-09-21T07:47:46.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.261+0000 7f726f400640 1 -- 172.21.15.39:0/1841358441 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7268009d20 con 0x7f7278152490 2024-09-21T07:47:46.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.261+0000 7f727d400640 1 -- 172.21.15.39:0/1841358441 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72781e42e0 con 0x7f7278152490 2024-09-21T07:47:46.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.261+0000 7f726f400640 1 -- 172.21.15.39:0/1841358441 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7268004280 con 0x7f7278152490 2024-09-21T07:47:46.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.261+0000 7f727d400640 1 -- 172.21.15.39:0/1841358441 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72781e48a0 con 0x7f7278152490 2024-09-21T07:47:46.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.262+0000 7f727d400640 1 -- 172.21.15.39:0/1841358441 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f727806ddf0 con 0x7f7278152490 2024-09-21T07:47:46.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.264+0000 7f726f400640 1 -- 172.21.15.39:0/1841358441 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7268005020 con 0x7f7278152490 2024-09-21T07:47:46.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.264+0000 7f726f400640 1 --2- 172.21.15.39:0/1841358441 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7254078bc0 0x7f725407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:46.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.265+0000 7f726f400640 1 -- 172.21.15.39:0/1841358441 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(198..198 src has 1..198) ==== 6280+0+0 (secure 0 0 0) 0x7f726809b410 con 0x7f7278152490 2024-09-21T07:47:46.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.265+0000 7f726de00640 1 --2- 172.21.15.39:0/1841358441 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7254078bc0 0x7f725407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:46.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.266+0000 7f726de00640 1 --2- 172.21.15.39:0/1841358441 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7254078bc0 0x7f725407b0d0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f7260033000 tx=0x7f7260004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:46.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.269+0000 7f726f400640 1 -- 172.21.15.39:0/1841358441 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7268020020 con 0x7f7278152490 2024-09-21T07:47:46.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.468+0000 7f727d400640 1 -- 172.21.15.39:0/1841358441 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) -- 0x7f7278068fb0 con 0x7f7278152490 2024-09-21T07:47:46.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.468+0000 7f726f400640 1 -- 172.21.15.39:0/1841358441 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v198) ==== 103+0+11 (secure 0 0 0) 0x7f7268063570 con 0x7f7278152490 2024-09-21T07:47:46.469 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-21T07:47:46.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.471+0000 7f727d400640 1 -- 172.21.15.39:0/1841358441 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7254078bc0 msgr2=0x7f725407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:46.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.471+0000 7f727d400640 1 --2- 172.21.15.39:0/1841358441 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7254078bc0 0x7f725407b0d0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f7260033000 tx=0x7f7260004730 comp rx=0 tx=0).stop 2024-09-21T07:47:46.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.471+0000 7f727d400640 1 -- 172.21.15.39:0/1841358441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7278152490 msgr2=0x7f72780a9fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:46.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.471+0000 7f727d400640 1 --2- 172.21.15.39:0/1841358441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7278152490 0x7f72780a9fc0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f726800d770 tx=0x7f726800dc30 comp rx=0 tx=0).stop 2024-09-21T07:47:46.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.471+0000 7f727d400640 1 -- 172.21.15.39:0/1841358441 shutdown_connections 2024-09-21T07:47:46.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.471+0000 7f727d400640 1 --2- 172.21.15.39:0/1841358441 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7254078bc0 0x7f725407b0d0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:46.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.471+0000 7f727d400640 1 --2- 172.21.15.39:0/1841358441 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f727815bbd0 0x7f7278154cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:46.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.471+0000 7f727d400640 1 --2- 172.21.15.39:0/1841358441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72781528a0 0x7f72781f5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:46.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.471+0000 7f727d400640 1 --2- 172.21.15.39:0/1841358441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7278152490 0x7f72780a9fc0 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:46.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.471+0000 7f727d400640 1 -- 172.21.15.39:0/1841358441 >> 172.21.15.39:0/1841358441 conn(0x7f7278073980 msgr2=0x7f72780b2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:46.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.471+0000 7f727d400640 1 -- 172.21.15.39:0/1841358441 shutdown_connections 2024-09-21T07:47:46.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.471+0000 7f727d400640 1 -- 172.21.15.39:0/1841358441 wait complete. 2024-09-21T07:47:46.494 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2024-09-21T07:47:46.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.723+0000 7f5cb6e00640 1 -- 172.21.15.39:0/3188366279 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cb0073da0 msgr2=0x7f5cb00741b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:46.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.723+0000 7f5cb6e00640 1 --2- 172.21.15.39:0/3188366279 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cb0073da0 0x7f5cb00741b0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0074930 tx=0x7f5c9802c4e0 comp rx=0 tx=0).stop 2024-09-21T07:47:46.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.723+0000 7f5cb6e00640 1 -- 172.21.15.39:0/3188366279 shutdown_connections 2024-09-21T07:47:46.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.723+0000 7f5cb6e00640 1 --2- 172.21.15.39:0/3188366279 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cb0073da0 0x7f5cb00741b0 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:46.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.723+0000 7f5cb6e00640 1 --2- 172.21.15.39:0/3188366279 >> v2:172.21.15.148:3300/0 conn(0x7f5cb00718e0 0x7f5cb015fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:46.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.723+0000 7f5cb6e00640 1 -- 172.21.15.39:0/3188366279 >> 172.21.15.39:0/3188366279 conn(0x7f5cb00d1450 msgr2=0x7f5cb0073690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:46.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.723+0000 7f5cb6e00640 1 -- 172.21.15.39:0/3188366279 shutdown_connections 2024-09-21T07:47:46.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.723+0000 7f5cb6e00640 1 -- 172.21.15.39:0/3188366279 wait complete. 2024-09-21T07:47:46.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.725+0000 7f5cb6e00640 1 Processor -- start 2024-09-21T07:47:46.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.725+0000 7f5cb6e00640 1 -- start start 2024-09-21T07:47:46.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.727+0000 7f5cb6e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5cb006f900 0x7f5cb014de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:46.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.727+0000 7f5cb6e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cb00718e0 0x7f5cb00caee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:46.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.727+0000 7f5cb6e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cb0073da0 0x7f5cb01508c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:46.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.727+0000 7f5cb6e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb015c9f0 con 0x7f5cb00718e0 2024-09-21T07:47:46.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.727+0000 7f5cb6e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb015c870 con 0x7f5cb0073da0 2024-09-21T07:47:46.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.727+0000 7f5cb6e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f5cb0070420 con 0x7f5cb006f900 2024-09-21T07:47:46.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.727+0000 7f5ca7400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cb00718e0 0x7f5cb00caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:46.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.727+0000 7f5ca7400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cb00718e0 0x7f5cb00caee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37734/0 (socket says 172.21.15.39:37734) 2024-09-21T07:47:46.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.727+0000 7f5ca7400640 1 -- 172.21.15.39:0/2593979138 learned_addr learned my addr 172.21.15.39:0/2593979138 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:46.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.727+0000 7f5cb4e00640 1 --2- 172.21.15.39:0/2593979138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cb0073da0 0x7f5cb01508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:46.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.727+0000 7f5cafe00640 1 --2- 172.21.15.39:0/2593979138 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5cb006f900 0x7f5cb014de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:46.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.728+0000 7f5ca7400640 1 -- 172.21.15.39:0/2593979138 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5cb006f900 msgr2=0x7f5cb014de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:46.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.728+0000 7f5ca7400640 1 --2- 172.21.15.39:0/2593979138 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5cb006f900 0x7f5cb014de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:46.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.728+0000 7f5ca7400640 1 -- 172.21.15.39:0/2593979138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cb0073da0 msgr2=0x7f5cb01508c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:46.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.728+0000 7f5ca7400640 1 --2- 172.21.15.39:0/2593979138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cb0073da0 0x7f5cb01508c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:46.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.728+0000 7f5ca7400640 1 -- 172.21.15.39:0/2593979138 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb01f8e90 con 0x7f5cb00718e0 2024-09-21T07:47:46.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.728+0000 7f5ca7400640 1 --2- 172.21.15.39:0/2593979138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cb00718e0 0x7f5cb00caee0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f5c9800d860 tx=0x7f5c98005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:46.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.728+0000 7f5cacc00640 1 -- 172.21.15.39:0/2593979138 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c98004230 con 0x7f5cb00718e0 2024-09-21T07:47:46.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.728+0000 7f5cacc00640 1 -- 172.21.15.39:0/2593979138 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c9800d3b0 con 0x7f5cb00718e0 2024-09-21T07:47:46.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.728+0000 7f5cacc00640 1 -- 172.21.15.39:0/2593979138 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c980044e0 con 0x7f5cb00718e0 2024-09-21T07:47:46.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.728+0000 7f5cb6e00640 1 -- 172.21.15.39:0/2593979138 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cb01f9170 con 0x7f5cb00718e0 2024-09-21T07:47:46.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.728+0000 7f5cb6e00640 1 -- 172.21.15.39:0/2593979138 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cb01f9730 con 0x7f5cb00718e0 2024-09-21T07:47:46.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.729+0000 7f5cb6e00640 1 -- 172.21.15.39:0/2593979138 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cb01f9970 con 0x7f5cb00718e0 2024-09-21T07:47:46.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.731+0000 7f5cacc00640 1 -- 172.21.15.39:0/2593979138 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5c9800c050 con 0x7f5cb00718e0 2024-09-21T07:47:46.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.732+0000 7f5cacc00640 1 --2- 172.21.15.39:0/2593979138 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5c8c078b90 0x7f5c8c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:46.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.732+0000 7f5cacc00640 1 -- 172.21.15.39:0/2593979138 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(198..198 src has 1..198) ==== 6280+0+0 (secure 0 0 0) 0x7f5c980bba30 con 0x7f5cb00718e0 2024-09-21T07:47:46.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.732+0000 7f5cafe00640 1 --2- 172.21.15.39:0/2593979138 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5c8c078b90 0x7f5c8c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:46.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.733+0000 7f5cafe00640 1 --2- 172.21.15.39:0/2593979138 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5c8c078b90 0x7f5c8c07b0a0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f5cb015f060 tx=0x7f5ca0005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:46.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.736+0000 7f5cacc00640 1 -- 172.21.15.39:0/2593979138 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5cb01f9970 con 0x7f5cb00718e0 2024-09-21T07:47:46.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.933+0000 7f5cb6e00640 1 -- 172.21.15.39:0/2593979138 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) -- 0x7f5cb01f8c20 con 0x7f5cb00718e0 2024-09-21T07:47:46.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.934+0000 7f5cacc00640 1 -- 172.21.15.39:0/2593979138 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v198) ==== 103+0+11 (secure 0 0 0) 0x7f5c98041020 con 0x7f5cb00718e0 2024-09-21T07:47:46.935 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-21T07:47:46.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.936+0000 7f5cb6e00640 1 -- 172.21.15.39:0/2593979138 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5c8c078b90 msgr2=0x7f5c8c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:46.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.936+0000 7f5cb6e00640 1 --2- 172.21.15.39:0/2593979138 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5c8c078b90 0x7f5c8c07b0a0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f5cb015f060 tx=0x7f5ca0005f90 comp rx=0 tx=0).stop 2024-09-21T07:47:46.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.936+0000 7f5cb6e00640 1 -- 172.21.15.39:0/2593979138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cb00718e0 msgr2=0x7f5cb00caee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:46.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.936+0000 7f5cb6e00640 1 --2- 172.21.15.39:0/2593979138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cb00718e0 0x7f5cb00caee0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f5c9800d860 tx=0x7f5c98005570 comp rx=0 tx=0).stop 2024-09-21T07:47:46.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.936+0000 7f5cb6e00640 1 -- 172.21.15.39:0/2593979138 shutdown_connections 2024-09-21T07:47:46.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.936+0000 7f5cb6e00640 1 --2- 172.21.15.39:0/2593979138 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5c8c078b90 0x7f5c8c07b0a0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:46.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.936+0000 7f5cb6e00640 1 --2- 172.21.15.39:0/2593979138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cb0073da0 0x7f5cb01508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:46.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.936+0000 7f5cb6e00640 1 --2- 172.21.15.39:0/2593979138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cb00718e0 0x7f5cb00caee0 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:46.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.936+0000 7f5cb6e00640 1 --2- 172.21.15.39:0/2593979138 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5cb006f900 0x7f5cb014de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:46.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.936+0000 7f5cb6e00640 1 -- 172.21.15.39:0/2593979138 >> 172.21.15.39:0/2593979138 conn(0x7f5cb00d1450 msgr2=0x7f5cb00b07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:46.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.936+0000 7f5cb6e00640 1 -- 172.21.15.39:0/2593979138 shutdown_connections 2024-09-21T07:47:46.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:46.936+0000 7f5cb6e00640 1 -- 172.21.15.39:0/2593979138 wait complete. 2024-09-21T07:47:46.962 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2024-09-21T07:47:47.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.191+0000 7f5133e00640 1 -- 172.21.15.39:0/94721619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f512c0d06d0 msgr2=0x7f512c0d0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:47.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.191+0000 7f5133e00640 1 --2- 172.21.15.39:0/94721619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f512c0d06d0 0x7f512c0d0ae0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f512c15ef30 tx=0x7f512402a8b0 comp rx=0 tx=0).stop 2024-09-21T07:47:47.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.192+0000 7f5133e00640 1 -- 172.21.15.39:0/94721619 shutdown_connections 2024-09-21T07:47:47.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.192+0000 7f5133e00640 1 --2- 172.21.15.39:0/94721619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f512c0d06d0 0x7f512c0d0ae0 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:47.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.192+0000 7f5133e00640 1 --2- 172.21.15.39:0/94721619 >> v2:172.21.15.148:3300/0 conn(0x7f512c162520 0x7f512c162930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:47.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.192+0000 7f5133e00640 1 -- 172.21.15.39:0/94721619 >> 172.21.15.39:0/94721619 conn(0x7f512c06cfc0 msgr2=0x7f512c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:47.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.192+0000 7f5133e00640 1 -- 172.21.15.39:0/94721619 shutdown_connections 2024-09-21T07:47:47.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.192+0000 7f5133e00640 1 -- 172.21.15.39:0/94721619 wait complete. 2024-09-21T07:47:47.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.194+0000 7f5133e00640 1 Processor -- start 2024-09-21T07:47:47.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.194+0000 7f5133e00640 1 -- start start 2024-09-21T07:47:47.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.195+0000 7f5133e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f512c0d06d0 0x7f512c157b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:47.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.196+0000 7f5133e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f512c15fb20 0x7f512c14afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:47.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.196+0000 7f5133e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f512c162520 0x7f512c161180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:47.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.196+0000 7f5133e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f512c0702d0 con 0x7f512c15fb20 2024-09-21T07:47:47.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.196+0000 7f5133e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f512c070150 con 0x7f512c0d06d0 2024-09-21T07:47:47.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.196+0000 7f5133e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f512c070450 con 0x7f512c162520 2024-09-21T07:47:47.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.196+0000 7f5130a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f512c15fb20 0x7f512c14afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:47.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.196+0000 7f5130a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f512c15fb20 0x7f512c14afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37744/0 (socket says 172.21.15.39:37744) 2024-09-21T07:47:47.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.196+0000 7f5130a00640 1 -- 172.21.15.39:0/3184574164 learned_addr learned my addr 172.21.15.39:0/3184574164 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:47.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.196+0000 7f5131e00640 1 --2- 172.21.15.39:0/3184574164 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f512c162520 0x7f512c161180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:47.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.196+0000 7f5131400640 1 --2- 172.21.15.39:0/3184574164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f512c0d06d0 0x7f512c157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:47.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.196+0000 7f5130a00640 1 -- 172.21.15.39:0/3184574164 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f512c162520 msgr2=0x7f512c161180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:47.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.196+0000 7f5130a00640 1 --2- 172.21.15.39:0/3184574164 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f512c162520 0x7f512c161180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:47.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.196+0000 7f5130a00640 1 -- 172.21.15.39:0/3184574164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f512c0d06d0 msgr2=0x7f512c157b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:47.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.196+0000 7f5130a00640 1 --2- 172.21.15.39:0/3184574164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f512c0d06d0 0x7f512c157b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:47.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.196+0000 7f5130a00640 1 -- 172.21.15.39:0/3184574164 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f512c14b500 con 0x7f512c15fb20 2024-09-21T07:47:47.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.197+0000 7f5130a00640 1 --2- 172.21.15.39:0/3184574164 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f512c15fb20 0x7f512c14afc0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f511800d740 tx=0x7f511800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:47.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.197+0000 7f5129600640 1 -- 172.21.15.39:0/3184574164 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5118017070 con 0x7f512c15fb20 2024-09-21T07:47:47.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.197+0000 7f5129600640 1 -- 172.21.15.39:0/3184574164 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f511800de50 con 0x7f512c15fb20 2024-09-21T07:47:47.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.197+0000 7f5133e00640 1 -- 172.21.15.39:0/3184574164 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f512c158470 con 0x7f512c15fb20 2024-09-21T07:47:47.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.197+0000 7f5129600640 1 -- 172.21.15.39:0/3184574164 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5118004020 con 0x7f512c15fb20 2024-09-21T07:47:47.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.197+0000 7f5133e00640 1 -- 172.21.15.39:0/3184574164 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f512c1f5c60 con 0x7f512c15fb20 2024-09-21T07:47:47.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.198+0000 7f5133e00640 1 -- 172.21.15.39:0/3184574164 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f512c0709e0 con 0x7f512c15fb20 2024-09-21T07:47:47.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.200+0000 7f5129600640 1 -- 172.21.15.39:0/3184574164 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f51180041c0 con 0x7f512c15fb20 2024-09-21T07:47:47.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.200+0000 7f5129600640 1 --2- 172.21.15.39:0/3184574164 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f510c078b90 0x7f510c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:47.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.201+0000 7f5129600640 1 -- 172.21.15.39:0/3184574164 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(198..198 src has 1..198) ==== 6280+0+0 (secure 0 0 0) 0x7f511809b2c0 con 0x7f512c15fb20 2024-09-21T07:47:47.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.201+0000 7f5131400640 1 --2- 172.21.15.39:0/3184574164 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f510c078b90 0x7f510c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:47.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.202+0000 7f5131400640 1 --2- 172.21.15.39:0/3184574164 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f510c078b90 0x7f510c07b0a0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f5120007d60 tx=0x7f5120007cf0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:47.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.205+0000 7f5129600640 1 -- 172.21.15.39:0/3184574164 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5118020020 con 0x7f512c15fb20 2024-09-21T07:47:47.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.400+0000 7f5133e00640 1 -- 172.21.15.39:0/3184574164 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) -- 0x7f512c1581e0 con 0x7f512c15fb20 2024-09-21T07:47:47.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.401+0000 7f5129600640 1 -- 172.21.15.39:0/3184574164 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v198) ==== 107+0+11 (secure 0 0 0) 0x7f5118062e60 con 0x7f512c15fb20 2024-09-21T07:47:47.402 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-21T07:47:47.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.403+0000 7f5133e00640 1 -- 172.21.15.39:0/3184574164 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f510c078b90 msgr2=0x7f510c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:47.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.403+0000 7f5133e00640 1 --2- 172.21.15.39:0/3184574164 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f510c078b90 0x7f510c07b0a0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f5120007d60 tx=0x7f5120007cf0 comp rx=0 tx=0).stop 2024-09-21T07:47:47.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.403+0000 7f5133e00640 1 -- 172.21.15.39:0/3184574164 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f512c15fb20 msgr2=0x7f512c14afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:47.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.403+0000 7f5133e00640 1 --2- 172.21.15.39:0/3184574164 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f512c15fb20 0x7f512c14afc0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f511800d740 tx=0x7f511800dc00 comp rx=0 tx=0).stop 2024-09-21T07:47:47.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.403+0000 7f5133e00640 1 -- 172.21.15.39:0/3184574164 shutdown_connections 2024-09-21T07:47:47.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.403+0000 7f5133e00640 1 --2- 172.21.15.39:0/3184574164 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f510c078b90 0x7f510c07b0a0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:47.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.403+0000 7f5133e00640 1 --2- 172.21.15.39:0/3184574164 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f512c162520 0x7f512c161180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:47.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.403+0000 7f5133e00640 1 --2- 172.21.15.39:0/3184574164 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f512c15fb20 0x7f512c14afc0 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:47.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.403+0000 7f5133e00640 1 --2- 172.21.15.39:0/3184574164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f512c0d06d0 0x7f512c157b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:47.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.403+0000 7f5133e00640 1 -- 172.21.15.39:0/3184574164 >> 172.21.15.39:0/3184574164 conn(0x7f512c06cfc0 msgr2=0x7f512c160020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:47.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.403+0000 7f5133e00640 1 -- 172.21.15.39:0/3184574164 shutdown_connections 2024-09-21T07:47:47.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.403+0000 7f5133e00640 1 -- 172.21.15.39:0/3184574164 wait complete. 2024-09-21T07:47:47.426 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_multimds_mksnap' 2024-09-21T07:47:47.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.653+0000 7f55fe800640 1 -- 172.21.15.39:0/3034656234 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55f80d0630 msgr2=0x7f55f80d0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:47.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.653+0000 7f55fe800640 1 --2- 172.21.15.39:0/3034656234 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55f80d0630 0x7f55f80d0a40 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f55f8072900 tx=0x7f55f002aa30 comp rx=0 tx=0).stop 2024-09-21T07:47:47.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.653+0000 7f55fe800640 1 -- 172.21.15.39:0/3034656234 shutdown_connections 2024-09-21T07:47:47.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.653+0000 7f55fe800640 1 --2- 172.21.15.39:0/3034656234 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55f80d0630 0x7f55f80d0a40 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:47.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.653+0000 7f55fe800640 1 --2- 172.21.15.39:0/3034656234 >> v2:172.21.15.148:3300/0 conn(0x7f55f8073dd0 0x7f55f815d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:47.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.653+0000 7f55fe800640 1 -- 172.21.15.39:0/3034656234 >> 172.21.15.39:0/3034656234 conn(0x7f55f806cfc0 msgr2=0x7f55f806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:47.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.653+0000 7f55fe800640 1 -- 172.21.15.39:0/3034656234 shutdown_connections 2024-09-21T07:47:47.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.653+0000 7f55fe800640 1 -- 172.21.15.39:0/3034656234 wait complete. 2024-09-21T07:47:47.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.655+0000 7f55fe800640 1 Processor -- start 2024-09-21T07:47:47.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.655+0000 7f55fe800640 1 -- start start 2024-09-21T07:47:47.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.657+0000 7f55fe800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55f8073dd0 0x7f55f80a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:47.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.657+0000 7f55fe800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55f80d0630 0x7f55f80ae480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:47.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.657+0000 7f55fe800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f55f815daf0 0x7f55f81f6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:47.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.657+0000 7f55fe800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f55f80cd1b0 con 0x7f55f8073dd0 2024-09-21T07:47:47.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.657+0000 7f55fe800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f55f80cd030 con 0x7f55f80d0630 2024-09-21T07:47:47.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.657+0000 7f55fe800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f55f80cd330 con 0x7f55f815daf0 2024-09-21T07:47:47.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.657+0000 7f55f7400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55f8073dd0 0x7f55f80a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:47.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.657+0000 7f55f7400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55f8073dd0 0x7f55f80a94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37752/0 (socket says 172.21.15.39:37752) 2024-09-21T07:47:47.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.657+0000 7f55f7400640 1 -- 172.21.15.39:0/1771794336 learned_addr learned my addr 172.21.15.39:0/1771794336 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:47.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.657+0000 7f55f6a00640 1 --2- 172.21.15.39:0/1771794336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55f80d0630 0x7f55f80ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:47.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.657+0000 7f55f7e00640 1 --2- 172.21.15.39:0/1771794336 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f55f815daf0 0x7f55f81f6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:47.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.658+0000 7f55f7400640 1 -- 172.21.15.39:0/1771794336 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f55f815daf0 msgr2=0x7f55f81f6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:47.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.658+0000 7f55f7400640 1 --2- 172.21.15.39:0/1771794336 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f55f815daf0 0x7f55f81f6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:47.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.658+0000 7f55f7400640 1 -- 172.21.15.39:0/1771794336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55f80d0630 msgr2=0x7f55f80ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:47.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.658+0000 7f55f7400640 1 --2- 172.21.15.39:0/1771794336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55f80d0630 0x7f55f80ae480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:47.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.658+0000 7f55f7400640 1 -- 172.21.15.39:0/1771794336 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f80a9d40 con 0x7f55f8073dd0 2024-09-21T07:47:47.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.658+0000 7f55f6a00640 1 --2- 172.21.15.39:0/1771794336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55f80d0630 0x7f55f80ae480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:47.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.658+0000 7f55f7400640 1 --2- 172.21.15.39:0/1771794336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55f8073dd0 0x7f55f80a94d0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f55e800e790 tx=0x7f55e800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:47.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.658+0000 7f55ef400640 1 -- 172.21.15.39:0/1771794336 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e800eea0 con 0x7f55f8073dd0 2024-09-21T07:47:47.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.658+0000 7f55ef400640 1 -- 172.21.15.39:0/1771794336 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55e800ccc0 con 0x7f55f8073dd0 2024-09-21T07:47:47.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.658+0000 7f55fe800640 1 -- 172.21.15.39:0/1771794336 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55f8158fc0 con 0x7f55f8073dd0 2024-09-21T07:47:47.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.658+0000 7f55ef400640 1 -- 172.21.15.39:0/1771794336 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e8004280 con 0x7f55f8073dd0 2024-09-21T07:47:47.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.658+0000 7f55fe800640 1 -- 172.21.15.39:0/1771794336 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55f8159580 con 0x7f55f8073dd0 2024-09-21T07:47:47.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.659+0000 7f55fe800640 1 -- 172.21.15.39:0/1771794336 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55f80cd850 con 0x7f55f8073dd0 2024-09-21T07:47:47.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.661+0000 7f55ef400640 1 -- 172.21.15.39:0/1771794336 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f55e80035f0 con 0x7f55f8073dd0 2024-09-21T07:47:47.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.662+0000 7f55ef400640 1 --2- 172.21.15.39:0/1771794336 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f55d8078b90 0x7f55d807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:47.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.662+0000 7f55ef400640 1 -- 172.21.15.39:0/1771794336 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(198..198 src has 1..198) ==== 6280+0+0 (secure 0 0 0) 0x7f55e809b910 con 0x7f55f8073dd0 2024-09-21T07:47:47.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.662+0000 7f55f6a00640 1 --2- 172.21.15.39:0/1771794336 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f55d8078b90 0x7f55d807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:47.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.663+0000 7f55f6a00640 1 --2- 172.21.15.39:0/1771794336 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f55d8078b90 0x7f55d807b0a0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f55f80af670 tx=0x7f55e4009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:47.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.666+0000 7f55ef400640 1 -- 172.21.15.39:0/1771794336 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f55e8063a70 con 0x7f55f8073dd0 2024-09-21T07:47:47.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:47.939+0000 7f55fe800640 1 -- 172.21.15.39:0/1771794336 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_multimds_mksnap"]} v 0) -- 0x7f55f8153670 con 0x7f55f8073dd0 2024-09-21T07:47:48.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.082+0000 7f55ef400640 1 -- 172.21.15.39:0/1771794336 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_multimds_mksnap"]}]=0 v710) ==== 144+0+0 (secure 0 0 0) 0x7f55e8063370 con 0x7f55f8073dd0 2024-09-21T07:47:48.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.084+0000 7f55fe800640 1 -- 172.21.15.39:0/1771794336 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f55d8078b90 msgr2=0x7f55d807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:48.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.084+0000 7f55fe800640 1 --2- 172.21.15.39:0/1771794336 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f55d8078b90 0x7f55d807b0a0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f55f80af670 tx=0x7f55e4009040 comp rx=0 tx=0).stop 2024-09-21T07:47:48.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.084+0000 7f55fe800640 1 -- 172.21.15.39:0/1771794336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55f8073dd0 msgr2=0x7f55f80a94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:48.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.084+0000 7f55fe800640 1 --2- 172.21.15.39:0/1771794336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55f8073dd0 0x7f55f80a94d0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f55e800e790 tx=0x7f55e800ec50 comp rx=0 tx=0).stop 2024-09-21T07:47:48.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.085+0000 7f55fe800640 1 -- 172.21.15.39:0/1771794336 shutdown_connections 2024-09-21T07:47:48.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.085+0000 7f55fe800640 1 --2- 172.21.15.39:0/1771794336 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f55d8078b90 0x7f55d807b0a0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:48.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.085+0000 7f55fe800640 1 --2- 172.21.15.39:0/1771794336 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f55f815daf0 0x7f55f81f6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:48.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.085+0000 7f55fe800640 1 --2- 172.21.15.39:0/1771794336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55f80d0630 0x7f55f80ae480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:48.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.085+0000 7f55fe800640 1 --2- 172.21.15.39:0/1771794336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55f8073dd0 0x7f55f80a94d0 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:48.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.085+0000 7f55fe800640 1 -- 172.21.15.39:0/1771794336 >> 172.21.15.39:0/1771794336 conn(0x7f55f806cfc0 msgr2=0x7f55f80b09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:48.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.085+0000 7f55fe800640 1 -- 172.21.15.39:0/1771794336 shutdown_connections 2024-09-21T07:47:48.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.085+0000 7f55fe800640 1 -- 172.21.15.39:0/1771794336 wait complete. 2024-09-21T07:47:48.111 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T07:47:48.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.336+0000 7ff6b8400640 1 -- 172.21.15.39:0/2225602527 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6b00b3d80 msgr2=0x7ff6b00ce2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:48.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.336+0000 7ff6b8400640 1 --2- 172.21.15.39:0/2225602527 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6b00b3d80 0x7ff6b00ce2b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff69c005fd0 tx=0x7ff69c02a5d0 comp rx=0 tx=0).stop 2024-09-21T07:47:48.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.336+0000 7ff6b8400640 1 -- 172.21.15.39:0/2225602527 shutdown_connections 2024-09-21T07:47:48.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.336+0000 7ff6b8400640 1 --2- 172.21.15.39:0/2225602527 >> v2:172.21.15.148:3300/0 conn(0x7ff6b006ef90 0x7ff6b006f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:48.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.336+0000 7ff6b8400640 1 --2- 172.21.15.39:0/2225602527 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6b00b3d80 0x7ff6b00ce2b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:48.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.336+0000 7ff6b8400640 1 -- 172.21.15.39:0/2225602527 >> 172.21.15.39:0/2225602527 conn(0x7ff6b00b2460 msgr2=0x7ff6b006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:48.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.336+0000 7ff6b8400640 1 -- 172.21.15.39:0/2225602527 shutdown_connections 2024-09-21T07:47:48.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.336+0000 7ff6b8400640 1 -- 172.21.15.39:0/2225602527 wait complete. 2024-09-21T07:47:48.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.338+0000 7ff6b8400640 1 Processor -- start 2024-09-21T07:47:48.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.338+0000 7ff6b8400640 1 -- start start 2024-09-21T07:47:48.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.340+0000 7ff6b8400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6b006ef90 0x7ff6b014c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:48.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.340+0000 7ff6b8400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6b00b4c40 0x7ff6b01f8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:48.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.340+0000 7ff6b8400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6b0157380 0x7ff6b01f9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:48.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.340+0000 7ff6b8400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b006fa60 con 0x7ff6b006ef90 2024-09-21T07:47:48.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.340+0000 7ff6b8400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b006f8e0 con 0x7ff6b0157380 2024-09-21T07:47:48.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.340+0000 7ff6b8400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7ff6b006fbe0 con 0x7ff6b00b4c40 2024-09-21T07:47:48.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.340+0000 7ff6b5a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6b006ef90 0x7ff6b014c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:48.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.340+0000 7ff6b5a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6b006ef90 0x7ff6b014c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37758/0 (socket says 172.21.15.39:37758) 2024-09-21T07:47:48.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.340+0000 7ff6b5a00640 1 -- 172.21.15.39:0/620689458 learned_addr learned my addr 172.21.15.39:0/620689458 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:48.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.340+0000 7ff6b5000640 1 --2- 172.21.15.39:0/620689458 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6b00b4c40 0x7ff6b01f8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:48.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.340+0000 7ff6b6400640 1 --2- 172.21.15.39:0/620689458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6b0157380 0x7ff6b01f9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:48.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.341+0000 7ff6b5a00640 1 -- 172.21.15.39:0/620689458 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6b00b4c40 msgr2=0x7ff6b01f8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:48.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.341+0000 7ff6b5a00640 1 --2- 172.21.15.39:0/620689458 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6b00b4c40 0x7ff6b01f8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:48.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.341+0000 7ff6b5a00640 1 -- 172.21.15.39:0/620689458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6b0157380 msgr2=0x7ff6b01f9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:48.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.341+0000 7ff6b5a00640 1 --2- 172.21.15.39:0/620689458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6b0157380 0x7ff6b01f9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:48.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.341+0000 7ff6b5a00640 1 -- 172.21.15.39:0/620689458 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6b015c1f0 con 0x7ff6b006ef90 2024-09-21T07:47:48.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.341+0000 7ff6b5a00640 1 --2- 172.21.15.39:0/620689458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6b006ef90 0x7ff6b014c280 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7ff6a400c3a0 tx=0x7ff6a400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:48.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.341+0000 7ff6ad600640 1 -- 172.21.15.39:0/620689458 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a40052d0 con 0x7ff6b006ef90 2024-09-21T07:47:48.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.341+0000 7ff6ad600640 1 -- 172.21.15.39:0/620689458 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6a4007020 con 0x7ff6b006ef90 2024-09-21T07:47:48.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.341+0000 7ff6b8400640 1 -- 172.21.15.39:0/620689458 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6b015c4d0 con 0x7ff6b006ef90 2024-09-21T07:47:48.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.341+0000 7ff6ad600640 1 -- 172.21.15.39:0/620689458 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a4007470 con 0x7ff6b006ef90 2024-09-21T07:47:48.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.341+0000 7ff6b8400640 1 -- 172.21.15.39:0/620689458 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6b015ca90 con 0x7ff6b006ef90 2024-09-21T07:47:48.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.343+0000 7ff6b8400640 1 -- 172.21.15.39:0/620689458 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6b006f3a0 con 0x7ff6b006ef90 2024-09-21T07:47:48.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.344+0000 7ff6ad600640 1 -- 172.21.15.39:0/620689458 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff6a40067d0 con 0x7ff6b006ef90 2024-09-21T07:47:48.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.345+0000 7ff6ad600640 1 --2- 172.21.15.39:0/620689458 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff690078b90 0x7ff69007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:48.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.345+0000 7ff6ad600640 1 -- 172.21.15.39:0/620689458 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(198..198 src has 1..198) ==== 6280+0+0 (secure 0 0 0) 0x7ff6a409a360 con 0x7ff6b006ef90 2024-09-21T07:47:48.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.345+0000 7ff6b5000640 1 --2- 172.21.15.39:0/620689458 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff690078b90 0x7ff69007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:48.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.346+0000 7ff6b5000640 1 --2- 172.21.15.39:0/620689458 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff690078b90 0x7ff69007b0a0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7ff69c033000 tx=0x7ff69c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:48.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.349+0000 7ff6ad600640 1 -- 172.21.15.39:0/620689458 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff6a4061f90 con 0x7ff6b006ef90 2024-09-21T07:47:48.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.546+0000 7ff6b8400640 1 -- 172.21.15.39:0/620689458 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7ff6b00b3130 con 0x7ff6b006ef90 2024-09-21T07:47:48.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.546+0000 7ff6ad600640 1 -- 172.21.15.39:0/620689458 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v50) ==== 59+0+2294 (secure 0 0 0) 0x7ff6a40669c0 con 0x7ff6b006ef90 2024-09-21T07:47:48.547 INFO:teuthology.orchestra.run.smithi039.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T07:47:48.547 INFO:teuthology.orchestra.run.smithi039.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T07:47:48.547 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced client_mount_timeout 30 2024-09-21T07:47:48.547 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_client 20/20 2024-09-21T07:47:48.548 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_mgr 20/20 2024-09-21T07:47:48.548 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_ms 1/1 2024-09-21T07:47:48.548 INFO:teuthology.orchestra.run.smithi039.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T07:47:48.548 INFO:teuthology.orchestra.run.smithi039.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T07:47:48.548 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3421.881759 2024-09-21T07:47:48.548 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3520.252978 2024-09-21T07:47:48.548 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3511.183316 2024-09-21T07:47:48.548 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3829.631847 2024-09-21T07:47:48.548 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 5474.545872 2024-09-21T07:47:48.548 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4013.628559 2024-09-21T07:47:48.548 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4230.187288 2024-09-21T07:47:48.548 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4148.983107 2024-09-21T07:47:48.548 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_mds 20/20 2024-09-21T07:47:48.548 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T07:47:48.548 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_ms 2/2 2024-09-21T07:47:48.548 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_debug_frag true 2024-09-21T07:47:48.549 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_debug_scatterstat true 2024-09-21T07:47:48.549 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T07:47:48.549 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_verify_scatter true 2024-09-21T07:47:48.549 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T07:47:48.549 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T07:47:48.549 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T07:47:48.549 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced client_mount_timeout 600 2024-09-21T07:47:48.549 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced debug_client 20/20 2024-09-21T07:47:48.549 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced debug_ms 1/1 2024-09-21T07:47:48.549 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T07:47:48.549 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T07:47:48.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.549+0000 7ff6b8400640 1 -- 172.21.15.39:0/620689458 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff690078b90 msgr2=0x7ff69007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:48.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.549+0000 7ff6b8400640 1 --2- 172.21.15.39:0/620689458 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff690078b90 0x7ff69007b0a0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7ff69c033000 tx=0x7ff69c004730 comp rx=0 tx=0).stop 2024-09-21T07:47:48.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.549+0000 7ff6b8400640 1 -- 172.21.15.39:0/620689458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6b006ef90 msgr2=0x7ff6b014c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:48.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.549+0000 7ff6b8400640 1 --2- 172.21.15.39:0/620689458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6b006ef90 0x7ff6b014c280 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7ff6a400c3a0 tx=0x7ff6a400c860 comp rx=0 tx=0).stop 2024-09-21T07:47:48.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.549+0000 7ff6b8400640 1 -- 172.21.15.39:0/620689458 shutdown_connections 2024-09-21T07:47:48.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.549+0000 7ff6b8400640 1 --2- 172.21.15.39:0/620689458 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff690078b90 0x7ff69007b0a0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:48.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.549+0000 7ff6b8400640 1 --2- 172.21.15.39:0/620689458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6b0157380 0x7ff6b01f9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:48.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.549+0000 7ff6b8400640 1 --2- 172.21.15.39:0/620689458 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6b00b4c40 0x7ff6b01f8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:48.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.549+0000 7ff6b8400640 1 --2- 172.21.15.39:0/620689458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6b006ef90 0x7ff6b014c280 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:48.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.549+0000 7ff6b8400640 1 -- 172.21.15.39:0/620689458 >> 172.21.15.39:0/620689458 conn(0x7ff6b00b2460 msgr2=0x7ff6b00b1600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:48.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.549+0000 7ff6b8400640 1 -- 172.21.15.39:0/620689458 shutdown_connections 2024-09-21T07:47:48.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.549+0000 7ff6b8400640 1 -- 172.21.15.39:0/620689458 wait complete. 2024-09-21T07:47:48.572 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T07:47:48.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.802+0000 7fab06c00640 1 -- 172.21.15.39:0/740949225 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab000cb8a0 msgr2=0x7fab00156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:48.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.802+0000 7fab06c00640 1 --2- 172.21.15.39:0/740949225 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab000cb8a0 0x7fab00156bb0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7faae8005fd0 tx=0x7faae802a540 comp rx=0 tx=0).stop 2024-09-21T07:47:48.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.802+0000 7fab06c00640 1 -- 172.21.15.39:0/740949225 shutdown_connections 2024-09-21T07:47:48.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.802+0000 7fab06c00640 1 --2- 172.21.15.39:0/740949225 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab000ccc60 0x7fab000cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:48.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.802+0000 7fab06c00640 1 --2- 172.21.15.39:0/740949225 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab000cb8a0 0x7fab00156bb0 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:48.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.802+0000 7fab06c00640 1 -- 172.21.15.39:0/740949225 >> 172.21.15.39:0/740949225 conn(0x7fab000b2460 msgr2=0x7fab0006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:48.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.802+0000 7fab06c00640 1 -- 172.21.15.39:0/740949225 shutdown_connections 2024-09-21T07:47:48.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.802+0000 7fab06c00640 1 -- 172.21.15.39:0/740949225 wait complete. 2024-09-21T07:47:48.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.805+0000 7fab06c00640 1 Processor -- start 2024-09-21T07:47:48.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.805+0000 7fab06c00640 1 -- start start 2024-09-21T07:47:48.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.806+0000 7fab06c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab000b30a0 0x7fab001562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:48.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.806+0000 7fab06c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab000cb8a0 0x7fab000cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:48.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.806+0000 7fab06c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab000ccc60 0x7fab001e93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:48.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.806+0000 7fab06c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fab000af410 con 0x7fab000ccc60 2024-09-21T07:47:48.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.806+0000 7fab06c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fab000af290 con 0x7fab000b30a0 2024-09-21T07:47:48.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.806+0000 7fab06c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fab000af590 con 0x7fab000cb8a0 2024-09-21T07:47:48.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.806+0000 7fab04c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab000ccc60 0x7fab001e93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:48.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.807+0000 7faaffe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab000b30a0 0x7fab001562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:48.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.807+0000 7fab04c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab000ccc60 0x7fab001e93e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37784/0 (socket says 172.21.15.39:37784) 2024-09-21T07:47:48.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.807+0000 7fab04c00640 1 -- 172.21.15.39:0/726419458 learned_addr learned my addr 172.21.15.39:0/726419458 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:48.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.807+0000 7faaf7e00640 1 --2- 172.21.15.39:0/726419458 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab000cb8a0 0x7fab000cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:48.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.807+0000 7fab04c00640 1 -- 172.21.15.39:0/726419458 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab000cb8a0 msgr2=0x7fab000cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:48.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.807+0000 7fab04c00640 1 --2- 172.21.15.39:0/726419458 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab000cb8a0 0x7fab000cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:48.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.807+0000 7fab04c00640 1 -- 172.21.15.39:0/726419458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab000b30a0 msgr2=0x7fab001562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:48.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.807+0000 7fab04c00640 1 --2- 172.21.15.39:0/726419458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab000b30a0 0x7fab001562b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:48.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.807+0000 7fab04c00640 1 -- 172.21.15.39:0/726419458 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab000cfe40 con 0x7fab000ccc60 2024-09-21T07:47:48.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.807+0000 7fab04c00640 1 --2- 172.21.15.39:0/726419458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab000ccc60 0x7fab001e93e0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7faaf800eab0 tx=0x7faaf800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:48.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.808+0000 7faafcc00640 1 -- 172.21.15.39:0/726419458 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf800cea0 con 0x7fab000ccc60 2024-09-21T07:47:48.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.808+0000 7faafcc00640 1 -- 172.21.15.39:0/726419458 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faaf8004b20 con 0x7fab000ccc60 2024-09-21T07:47:48.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.808+0000 7faafcc00640 1 -- 172.21.15.39:0/726419458 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf8005020 con 0x7fab000ccc60 2024-09-21T07:47:48.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.808+0000 7fab06c00640 1 -- 172.21.15.39:0/726419458 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab001f96a0 con 0x7fab000ccc60 2024-09-21T07:47:48.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.808+0000 7fab06c00640 1 -- 172.21.15.39:0/726419458 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab000cf900 con 0x7fab000ccc60 2024-09-21T07:47:48.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.809+0000 7fab06c00640 1 -- 172.21.15.39:0/726419458 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab000b44c0 con 0x7fab000ccc60 2024-09-21T07:47:48.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.810+0000 7faafcc00640 1 -- 172.21.15.39:0/726419458 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7faaf80051c0 con 0x7fab000ccc60 2024-09-21T07:47:48.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.812+0000 7faafcc00640 1 --2- 172.21.15.39:0/726419458 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faadc078b90 0x7faadc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:48.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.812+0000 7faafcc00640 1 -- 172.21.15.39:0/726419458 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(198..198 src has 1..198) ==== 6280+0+0 (secure 0 0 0) 0x7faaf809baa0 con 0x7fab000ccc60 2024-09-21T07:47:48.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.812+0000 7faaffe00640 1 --2- 172.21.15.39:0/726419458 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faadc078b90 0x7faadc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:48.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.813+0000 7faaffe00640 1 --2- 172.21.15.39:0/726419458 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faadc078b90 0x7faadc07b0a0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7faaf0001720 tx=0x7faaf0009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:48.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:48.816+0000 7faafcc00640 1 -- 172.21.15.39:0/726419458 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fab000b44c0 con 0x7fab000ccc60 2024-09-21T07:47:49.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.013+0000 7fab06c00640 1 -- 172.21.15.39:0/726419458 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fab000cfbd0 con 0x7fab000ccc60 2024-09-21T07:47:49.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.018+0000 7faafcc00640 1 -- 172.21.15.39:0/726419458 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v51)=0 v51) ==== 122+0+0 (secure 0 0 0) 0x7faaf80636d0 con 0x7fab000ccc60 2024-09-21T07:47:49.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.021+0000 7fab06c00640 1 -- 172.21.15.39:0/726419458 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faadc078b90 msgr2=0x7faadc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:49.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.021+0000 7fab06c00640 1 --2- 172.21.15.39:0/726419458 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faadc078b90 0x7faadc07b0a0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7faaf0001720 tx=0x7faaf0009040 comp rx=0 tx=0).stop 2024-09-21T07:47:49.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.021+0000 7fab06c00640 1 -- 172.21.15.39:0/726419458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab000ccc60 msgr2=0x7fab001e93e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:49.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.021+0000 7fab06c00640 1 --2- 172.21.15.39:0/726419458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab000ccc60 0x7fab001e93e0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7faaf800eab0 tx=0x7faaf800ef70 comp rx=0 tx=0).stop 2024-09-21T07:47:49.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.021+0000 7fab06c00640 1 -- 172.21.15.39:0/726419458 shutdown_connections 2024-09-21T07:47:49.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.021+0000 7fab06c00640 1 --2- 172.21.15.39:0/726419458 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faadc078b90 0x7faadc07b0a0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:49.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.021+0000 7fab06c00640 1 --2- 172.21.15.39:0/726419458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab000ccc60 0x7fab001e93e0 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:49.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.021+0000 7fab06c00640 1 --2- 172.21.15.39:0/726419458 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fab000cb8a0 0x7fab000cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:49.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.021+0000 7fab06c00640 1 --2- 172.21.15.39:0/726419458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab000b30a0 0x7fab001562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:49.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.021+0000 7fab06c00640 1 -- 172.21.15.39:0/726419458 >> 172.21.15.39:0/726419458 conn(0x7fab000b2460 msgr2=0x7fab000b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:49.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.021+0000 7fab06c00640 1 -- 172.21.15.39:0/726419458 shutdown_connections 2024-09-21T07:47:49.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.021+0000 7fab06c00640 1 -- 172.21.15.39:0/726419458 wait complete. 2024-09-21T07:47:49.042 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:47:49.043 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:47:49.139 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:47:49.231 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:47:49.232 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:47:49.232 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:47:49.267 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:47:49.267 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:47:49.267 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:47:49.304 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:47:49.305 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:47:49.305 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:47:49.342 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:47:49.342 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:47:49.343 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:47:49.381 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:47:49.381 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T07:47:49.381 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:47:49.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.607+0000 7fc2efe00640 1 -- 172.21.15.39:0/4233957032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2e80b3da0 msgr2=0x7fc2e80b4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:49.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.607+0000 7fc2efe00640 1 --2- 172.21.15.39:0/4233957032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2e80b3da0 0x7fc2e80b4990 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0005fd0 tx=0x7fc2e002aa90 comp rx=0 tx=0).stop 2024-09-21T07:47:49.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.607+0000 7fc2efe00640 1 -- 172.21.15.39:0/4233957032 shutdown_connections 2024-09-21T07:47:49.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.607+0000 7fc2efe00640 1 --2- 172.21.15.39:0/4233957032 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc2e80b3990 0x7fc2e80b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:49.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.607+0000 7fc2efe00640 1 --2- 172.21.15.39:0/4233957032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2e80b3da0 0x7fc2e80b4990 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:49.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.607+0000 7fc2efe00640 1 -- 172.21.15.39:0/4233957032 >> 172.21.15.39:0/4233957032 conn(0x7fc2e80b2460 msgr2=0x7fc2e806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:49.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.607+0000 7fc2efe00640 1 -- 172.21.15.39:0/4233957032 shutdown_connections 2024-09-21T07:47:49.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.607+0000 7fc2efe00640 1 -- 172.21.15.39:0/4233957032 wait complete. 2024-09-21T07:47:49.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.609+0000 7fc2efe00640 1 Processor -- start 2024-09-21T07:47:49.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.610+0000 7fc2efe00640 1 -- start start 2024-09-21T07:47:49.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.611+0000 7fc2efe00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc2e80b3990 0x7fc2e814aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:49.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.611+0000 7fc2efe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2e80b3da0 0x7fc2e80ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:49.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.611+0000 7fc2efe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2e8162f10 0x7fc2e80b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:49.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.611+0000 7fc2efe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e80af200 con 0x7fc2e8162f10 2024-09-21T07:47:49.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.611+0000 7fc2efe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e80af080 con 0x7fc2e80b3da0 2024-09-21T07:47:49.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.611+0000 7fc2efe00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc2e80af380 con 0x7fc2e80b3990 2024-09-21T07:47:49.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.611+0000 7fc2ede00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2e8162f10 0x7fc2e80b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:49.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.611+0000 7fc2ede00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2e8162f10 0x7fc2e80b0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37810/0 (socket says 172.21.15.39:37810) 2024-09-21T07:47:49.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.611+0000 7fc2eca00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2e80b3da0 0x7fc2e80ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:49.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.611+0000 7fc2ede00640 1 -- 172.21.15.39:0/600101491 learned_addr learned my addr 172.21.15.39:0/600101491 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:49.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.612+0000 7fc2ede00640 1 -- 172.21.15.39:0/600101491 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc2e80b3990 msgr2=0x7fc2e814aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:49.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.612+0000 7fc2ede00640 1 --2- 172.21.15.39:0/600101491 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc2e80b3990 0x7fc2e814aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:49.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.612+0000 7fc2ede00640 1 -- 172.21.15.39:0/600101491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2e80b3da0 msgr2=0x7fc2e80ca5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:49.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.612+0000 7fc2ede00640 1 --2- 172.21.15.39:0/600101491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2e80b3da0 0x7fc2e80ca5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:49.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.612+0000 7fc2ede00640 1 -- 172.21.15.39:0/600101491 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2e814bab0 con 0x7fc2e8162f10 2024-09-21T07:47:49.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.612+0000 7fc2ede00640 1 --2- 172.21.15.39:0/600101491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2e8162f10 0x7fc2e80b0b00 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0009d90 tx=0x7fc2e0003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:49.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.612+0000 7fc2e5600640 1 -- 172.21.15.39:0/600101491 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e0044070 con 0x7fc2e8162f10 2024-09-21T07:47:49.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.612+0000 7fc2e5600640 1 -- 172.21.15.39:0/600101491 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2e0004040 con 0x7fc2e8162f10 2024-09-21T07:47:49.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.612+0000 7fc2e5600640 1 -- 172.21.15.39:0/600101491 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e0004430 con 0x7fc2e8162f10 2024-09-21T07:47:49.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.612+0000 7fc2efe00640 1 -- 172.21.15.39:0/600101491 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2e814bd90 con 0x7fc2e8162f10 2024-09-21T07:47:49.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.613+0000 7fc2efe00640 1 -- 172.21.15.39:0/600101491 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2e814c350 con 0x7fc2e8162f10 2024-09-21T07:47:49.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.613+0000 7fc2efe00640 1 -- 172.21.15.39:0/600101491 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2e80af8a0 con 0x7fc2e8162f10 2024-09-21T07:47:49.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.615+0000 7fc2e5600640 1 -- 172.21.15.39:0/600101491 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc2e0047070 con 0x7fc2e8162f10 2024-09-21T07:47:49.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.616+0000 7fc2e5600640 1 --2- 172.21.15.39:0/600101491 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc2c8078b90 0x7fc2c807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:49.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.617+0000 7fc2e5600640 1 -- 172.21.15.39:0/600101491 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(198..198 src has 1..198) ==== 6280+0+0 (secure 0 0 0) 0x7fc2e00bbeb0 con 0x7fc2e8162f10 2024-09-21T07:47:49.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.617+0000 7fc2ed400640 1 --2- 172.21.15.39:0/600101491 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc2c8078b90 0x7fc2c807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:49.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.617+0000 7fc2ed400640 1 --2- 172.21.15.39:0/600101491 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc2c8078b90 0x7fc2c807b0a0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc000f60 tx=0x7fc2dc0079b0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:49.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.621+0000 7fc2e5600640 1 -- 172.21.15.39:0/600101491 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc2e0032030 con 0x7fc2e8162f10 2024-09-21T07:47:49.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.856+0000 7fc2efe00640 1 -- 172.21.15.39:0/600101491 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2e80cdb70 con 0x7fc2e8162f10 2024-09-21T07:47:49.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.857+0000 7fc2e5600640 1 -- 172.21.15.39:0/600101491 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 207 v207) ==== 95+0+3913 (secure 0 0 0) 0x7fc2e0083940 con 0x7fc2e8162f10 2024-09-21T07:47:49.858 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:47:49.858 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":207,"btime":"2024-09-21T07:47:41:032274+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7715,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3057062733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3057062733},{"type":"v1","addr":"172.21.15.148:6836","nonce":3057062733}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":194},{"gid":8689,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/8652512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":8652512},{"type":"v1","addr":"172.21.15.148:6837","nonce":8652512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":194},{"gid":9735,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/2912162391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2912162391},{"type":"v1","addr":"172.21.15.39:6833","nonce":2912162391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":207},{"gid":9738,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4196287591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":4196287591},{"type":"v1","addr":"172.21.15.39:6835","nonce":4196287591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":207}],"filesystems":[]} 2024-09-21T07:47:49.858 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 207 2024-09-21T07:47:49.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.859+0000 7fc2efe00640 1 -- 172.21.15.39:0/600101491 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc2c8078b90 msgr2=0x7fc2c807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:49.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.859+0000 7fc2efe00640 1 --2- 172.21.15.39:0/600101491 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc2c8078b90 0x7fc2c807b0a0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc000f60 tx=0x7fc2dc0079b0 comp rx=0 tx=0).stop 2024-09-21T07:47:49.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.859+0000 7fc2efe00640 1 -- 172.21.15.39:0/600101491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2e8162f10 msgr2=0x7fc2e80b0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:49.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.859+0000 7fc2efe00640 1 --2- 172.21.15.39:0/600101491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2e8162f10 0x7fc2e80b0b00 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0009d90 tx=0x7fc2e0003ce0 comp rx=0 tx=0).stop 2024-09-21T07:47:49.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.859+0000 7fc2efe00640 1 -- 172.21.15.39:0/600101491 shutdown_connections 2024-09-21T07:47:49.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.859+0000 7fc2efe00640 1 --2- 172.21.15.39:0/600101491 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc2c8078b90 0x7fc2c807b0a0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:49.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.859+0000 7fc2efe00640 1 --2- 172.21.15.39:0/600101491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2e8162f10 0x7fc2e80b0b00 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:49.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.859+0000 7fc2efe00640 1 --2- 172.21.15.39:0/600101491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2e80b3da0 0x7fc2e80ca5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:49.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.859+0000 7fc2efe00640 1 --2- 172.21.15.39:0/600101491 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc2e80b3990 0x7fc2e814aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:49.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.859+0000 7fc2efe00640 1 -- 172.21.15.39:0/600101491 >> 172.21.15.39:0/600101491 conn(0x7fc2e80b2460 msgr2=0x7fc2e8160ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:49.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.860+0000 7fc2efe00640 1 -- 172.21.15.39:0/600101491 shutdown_connections 2024-09-21T07:47:49.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:49.860+0000 7fc2efe00640 1 -- 172.21.15.39:0/600101491 wait complete. 2024-09-21T07:47:49.882 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T07:47:49.882 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T07:47:49.882 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T07:47:49.882 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:47:49.882 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T07:47:49.882 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T07:47:49.883 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T07:47:49.883 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:47:49.883 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T07:47:49.883 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T07:47:49.883 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T07:47:49.884 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:47:49.884 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T07:47:49.884 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T07:47:49.884 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T07:47:49.884 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:47:49.908 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T07:47:49.908 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T07:47:49.909 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T07:47:49.909 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T07:47:49.909 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T07:47:49.909 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T07:47:49.910 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T07:47:49.911 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T07:47:49.952 INFO:tasks.ceph.mds.b:Started 2024-09-21T07:47:49.952 DEBUG:teuthology.parallel:result is None 2024-09-21T07:47:49.954 INFO:tasks.ceph.mds.a:Started 2024-09-21T07:47:49.954 DEBUG:teuthology.parallel:result is None 2024-09-21T07:47:49.955 INFO:tasks.ceph.mds.d:Started 2024-09-21T07:47:49.955 DEBUG:teuthology.parallel:result is None 2024-09-21T07:47:49.957 INFO:tasks.ceph.mds.c:Started 2024-09-21T07:47:49.957 DEBUG:teuthology.parallel:result is None 2024-09-21T07:47:49.957 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T07:47:50.151 INFO:tasks.ceph.mds.d.smithi148.stdout:starting mds.d at 2024-09-21T07:47:50.152 INFO:tasks.ceph.mds.b.smithi148.stdout:starting mds.b at 2024-09-21T07:47:50.153 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-21T07:47:50.160 INFO:tasks.ceph.mds.c.smithi039.stdout:starting mds.c at 2024-09-21T07:47:50.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.195+0000 7f03b5e00640 1 -- 172.21.15.39:0/2292339053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03b0162f10 msgr2=0x7f03b00b30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:50.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.195+0000 7f03b5e00640 1 --2- 172.21.15.39:0/2292339053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03b0162f10 0x7f03b00b30a0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f03a0009830 tx=0x7f03a002a510 comp rx=0 tx=0).stop 2024-09-21T07:47:50.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.196+0000 7f03b5e00640 1 -- 172.21.15.39:0/2292339053 shutdown_connections 2024-09-21T07:47:50.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.196+0000 7f03b5e00640 1 --2- 172.21.15.39:0/2292339053 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03b00b3990 0x7f03b00b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:50.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.196+0000 7f03b5e00640 1 --2- 172.21.15.39:0/2292339053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03b0162f10 0x7f03b00b30a0 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:50.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.196+0000 7f03b5e00640 1 -- 172.21.15.39:0/2292339053 >> 172.21.15.39:0/2292339053 conn(0x7f03b00b2460 msgr2=0x7f03b006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:50.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.196+0000 7f03b5e00640 1 -- 172.21.15.39:0/2292339053 shutdown_connections 2024-09-21T07:47:50.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.196+0000 7f03b5e00640 1 -- 172.21.15.39:0/2292339053 wait complete. 2024-09-21T07:47:50.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.198+0000 7f03b5e00640 1 Processor -- start 2024-09-21T07:47:50.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.198+0000 7f03b5e00640 1 -- start start 2024-09-21T07:47:50.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.200+0000 7f03b5e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03b00b3990 0x7f03b014b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:50.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.200+0000 7f03b5e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03b00b3da0 0x7f03b00b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:50.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.200+0000 7f03b5e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03b0162f10 0x7f03b00c3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:50.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.200+0000 7f03b5e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f03b00af910 con 0x7f03b00b3990 2024-09-21T07:47:50.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.200+0000 7f03b5e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f03b00af790 con 0x7f03b00b3da0 2024-09-21T07:47:50.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.200+0000 7f03b5e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f03b00afa90 con 0x7f03b0162f10 2024-09-21T07:47:50.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.200+0000 7f03aea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03b00b3990 0x7f03b014b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:50.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.200+0000 7f03aea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03b00b3990 0x7f03b014b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37846/0 (socket says 172.21.15.39:37846) 2024-09-21T07:47:50.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.200+0000 7f03aea00640 1 -- 172.21.15.39:0/2504543890 learned_addr learned my addr 172.21.15.39:0/2504543890 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:50.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.200+0000 7f03ae000640 1 --2- 172.21.15.39:0/2504543890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03b00b3da0 0x7f03b00b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:50.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.200+0000 7f03af400640 1 --2- 172.21.15.39:0/2504543890 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03b0162f10 0x7f03b00c3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:50.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.200+0000 7f03aea00640 1 -- 172.21.15.39:0/2504543890 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03b0162f10 msgr2=0x7f03b00c3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:50.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.200+0000 7f03aea00640 1 --2- 172.21.15.39:0/2504543890 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03b0162f10 0x7f03b00c3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:50.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.200+0000 7f03aea00640 1 -- 172.21.15.39:0/2504543890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03b00b3da0 msgr2=0x7f03b00b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:50.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.200+0000 7f03aea00640 1 --2- 172.21.15.39:0/2504543890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03b00b3da0 0x7f03b00b10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:50.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.201+0000 7f03aea00640 1 -- 172.21.15.39:0/2504543890 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b00c5c80 con 0x7f03b00b3990 2024-09-21T07:47:50.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.201+0000 7f03af400640 1 --2- 172.21.15.39:0/2504543890 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03b0162f10 0x7f03b00c3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:50.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.201+0000 7f03aea00640 1 --2- 172.21.15.39:0/2504543890 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03b00b3990 0x7f03b014b240 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f03a0009950 tx=0x7f03a0033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:50.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.201+0000 7f03a7400640 1 -- 172.21.15.39:0/2504543890 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a0005c90 con 0x7f03b00b3990 2024-09-21T07:47:50.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.201+0000 7f03a7400640 1 -- 172.21.15.39:0/2504543890 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f03a0031e50 con 0x7f03b00b3990 2024-09-21T07:47:50.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.201+0000 7f03a7400640 1 -- 172.21.15.39:0/2504543890 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a0005260 con 0x7f03b00b3990 2024-09-21T07:47:50.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.201+0000 7f03b5e00640 1 -- 172.21.15.39:0/2504543890 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03b014da40 con 0x7f03b00b3990 2024-09-21T07:47:50.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.201+0000 7f03b5e00640 1 -- 172.21.15.39:0/2504543890 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03b014e000 con 0x7f03b00b3990 2024-09-21T07:47:50.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.203+0000 7f03b5e00640 1 -- 172.21.15.39:0/2504543890 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03b00694e0 con 0x7f03b00b3990 2024-09-21T07:47:50.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.204+0000 7f03a7400640 1 -- 172.21.15.39:0/2504543890 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f03a0005400 con 0x7f03b00b3990 2024-09-21T07:47:50.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.205+0000 7f03a7400640 1 --2- 172.21.15.39:0/2504543890 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0390078b90 0x7f039007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:50.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.205+0000 7f03a7400640 1 -- 172.21.15.39:0/2504543890 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(198..198 src has 1..198) ==== 6280+0+0 (secure 0 0 0) 0x7f03a00bc0a0 con 0x7f03b00b3990 2024-09-21T07:47:50.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.205+0000 7f03ae000640 1 --2- 172.21.15.39:0/2504543890 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0390078b90 0x7f039007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:50.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.206+0000 7f03ae000640 1 --2- 172.21.15.39:0/2504543890 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0390078b90 0x7f039007b0a0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f03b006fb30 tx=0x7f039c00b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:50.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.209+0000 7f03a7400640 1 -- 172.21.15.39:0/2504543890 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f03b00694e0 con 0x7f03b00b3990 2024-09-21T07:47:50.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:50.417+0000 7f03b5e00640 1 -- 172.21.15.39:0/2504543890 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f03b00b4b00 con 0x7f03b00b3990 2024-09-21T07:47:51.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.124+0000 7f03a7400640 1 -- 172.21.15.39:0/2504543890 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v200) ==== 97+0+0 (secure 0 0 0) 0x7f03a003f030 con 0x7f03b00b3990 2024-09-21T07:47:51.125 INFO:teuthology.orchestra.run.smithi039.stderr: removed all blocklist entries 2024-09-21T07:47:51.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.128+0000 7f03b5e00640 1 -- 172.21.15.39:0/2504543890 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0390078b90 msgr2=0x7f039007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:51.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.128+0000 7f03b5e00640 1 --2- 172.21.15.39:0/2504543890 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0390078b90 0x7f039007b0a0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f03b006fb30 tx=0x7f039c00b2e0 comp rx=0 tx=0).stop 2024-09-21T07:47:51.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.128+0000 7f03b5e00640 1 -- 172.21.15.39:0/2504543890 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03b00b3990 msgr2=0x7f03b014b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:51.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.128+0000 7f03b5e00640 1 --2- 172.21.15.39:0/2504543890 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03b00b3990 0x7f03b014b240 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f03a0009950 tx=0x7f03a0033c20 comp rx=0 tx=0).stop 2024-09-21T07:47:51.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.128+0000 7f03b5e00640 1 -- 172.21.15.39:0/2504543890 shutdown_connections 2024-09-21T07:47:51.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.128+0000 7f03b5e00640 1 --2- 172.21.15.39:0/2504543890 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0390078b90 0x7f039007b0a0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:51.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.128+0000 7f03b5e00640 1 --2- 172.21.15.39:0/2504543890 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03b0162f10 0x7f03b00c3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:51.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.128+0000 7f03b5e00640 1 --2- 172.21.15.39:0/2504543890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03b00b3da0 0x7f03b00b10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:51.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.128+0000 7f03b5e00640 1 --2- 172.21.15.39:0/2504543890 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03b00b3990 0x7f03b014b240 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:51.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.128+0000 7f03b5e00640 1 -- 172.21.15.39:0/2504543890 >> 172.21.15.39:0/2504543890 conn(0x7f03b00b2460 msgr2=0x7f03b006ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:51.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.129+0000 7f03b5e00640 1 -- 172.21.15.39:0/2504543890 shutdown_connections 2024-09-21T07:47:51.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.129+0000 7f03b5e00640 1 -- 172.21.15.39:0/2504543890 wait complete. 2024-09-21T07:47:51.159 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T07:47:51.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.388+0000 7f1187200640 1 -- 172.21.15.39:0/2146140551 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f116c005250 msgr2=0x7f116c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:51.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.388+0000 7f1187200640 1 --2- 172.21.15.39:0/2146140551 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f116c005250 0x7f116c00b9c0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f118000c610 tx=0x7f116c005830 comp rx=0 tx=0).stop 2024-09-21T07:47:51.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.388+0000 7f1187200640 1 -- 172.21.15.39:0/2146140551 shutdown_connections 2024-09-21T07:47:51.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.388+0000 7f1187200640 1 --2- 172.21.15.39:0/2146140551 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f116c005250 0x7f116c00b9c0 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:51.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.388+0000 7f1187200640 1 -- 172.21.15.39:0/2146140551 >> 172.21.15.39:0/2146140551 conn(0x7f118806cfc0 msgr2=0x7f118806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:51.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.388+0000 7f1187200640 1 -- 172.21.15.39:0/2146140551 shutdown_connections 2024-09-21T07:47:51.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.388+0000 7f1187200640 1 -- 172.21.15.39:0/2146140551 wait complete. 2024-09-21T07:47:51.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.390+0000 7f1187200640 1 Processor -- start 2024-09-21T07:47:51.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.391+0000 7f1187200640 1 -- start start 2024-09-21T07:47:51.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.392+0000 7f1187200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f116c005250 0x7f11880afe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:51.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.392+0000 7f1187200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11880ae8a0 0x7f11880cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:51.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.392+0000 7f1187200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11880aecb0 0x7f11880cc800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:51.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.392+0000 7f1187200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f118815d900 con 0x7f11880ae8a0 2024-09-21T07:47:51.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.392+0000 7f1187200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f118815d780 con 0x7f116c005250 2024-09-21T07:47:51.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.392+0000 7f1187200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f118815da80 con 0x7f11880aecb0 2024-09-21T07:47:51.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.392+0000 7f117fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11880ae8a0 0x7f11880cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:51.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.392+0000 7f117fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11880ae8a0 0x7f11880cc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37854/0 (socket says 172.21.15.39:37854) 2024-09-21T07:47:51.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.392+0000 7f117fe00640 1 -- 172.21.15.39:0/2678696325 learned_addr learned my addr 172.21.15.39:0/2678696325 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:51.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.392+0000 7f1185e00640 1 --2- 172.21.15.39:0/2678696325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f116c005250 0x7f11880afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:51.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.393+0000 7f1186800640 1 --2- 172.21.15.39:0/2678696325 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11880aecb0 0x7f11880cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:51.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.393+0000 7f117fe00640 1 -- 172.21.15.39:0/2678696325 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11880aecb0 msgr2=0x7f11880cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:51.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.393+0000 7f117fe00640 1 --2- 172.21.15.39:0/2678696325 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11880aecb0 0x7f11880cc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:51.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.393+0000 7f117fe00640 1 -- 172.21.15.39:0/2678696325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f116c005250 msgr2=0x7f11880afe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:51.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.393+0000 7f117fe00640 1 --2- 172.21.15.39:0/2678696325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f116c005250 0x7f11880afe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:51.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.393+0000 7f117fe00640 1 -- 172.21.15.39:0/2678696325 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11880b08d0 con 0x7f11880ae8a0 2024-09-21T07:47:51.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.393+0000 7f1186800640 1 --2- 172.21.15.39:0/2678696325 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11880aecb0 0x7f11880cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:51.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.393+0000 7f117fe00640 1 --2- 172.21.15.39:0/2678696325 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11880ae8a0 0x7f11880cc2c0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f1170011ab0 tx=0x7f1170011eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:51.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.393+0000 7f117e000640 1 -- 172.21.15.39:0/2678696325 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117001e070 con 0x7f11880ae8a0 2024-09-21T07:47:51.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.393+0000 7f117e000640 1 -- 172.21.15.39:0/2678696325 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1170004460 con 0x7f11880ae8a0 2024-09-21T07:47:51.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.393+0000 7f1187200640 1 -- 172.21.15.39:0/2678696325 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11880b0bb0 con 0x7f11880ae8a0 2024-09-21T07:47:51.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.393+0000 7f117e000640 1 -- 172.21.15.39:0/2678696325 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117000fc00 con 0x7f11880ae8a0 2024-09-21T07:47:51.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.394+0000 7f1187200640 1 -- 172.21.15.39:0/2678696325 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11880b1170 con 0x7f11880ae8a0 2024-09-21T07:47:51.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.395+0000 7f1187200640 1 -- 172.21.15.39:0/2678696325 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f118815fca0 con 0x7f11880ae8a0 2024-09-21T07:47:51.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.396+0000 7f117e000640 1 -- 172.21.15.39:0/2678696325 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1170021070 con 0x7f11880ae8a0 2024-09-21T07:47:51.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.398+0000 7f117e000640 1 --2- 172.21.15.39:0/2678696325 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1164078b90 0x7f116407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:51.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.398+0000 7f117e000640 1 -- 172.21.15.39:0/2678696325 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(202..202 src has 1..202) ==== 6108+0+0 (secure 0 0 0) 0x7f117009b8e0 con 0x7f11880ae8a0 2024-09-21T07:47:51.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.398+0000 7f1185e00640 1 --2- 172.21.15.39:0/2678696325 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1164078b90 0x7f116407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:51.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.399+0000 7f1185e00640 1 --2- 172.21.15.39:0/2678696325 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1164078b90 0x7f116407b0a0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f1178006680 tx=0x7f1178006b70 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:51.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.402+0000 7f117e000640 1 -- 172.21.15.39:0/2678696325 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f118815fca0 con 0x7f11880ae8a0 2024-09-21T07:47:51.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.674+0000 7f1187200640 1 -- 172.21.15.39:0/2678696325 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f11880b0660 con 0x7f11880ae8a0 2024-09-21T07:47:51.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.674+0000 7f117e000640 1 -- 172.21.15.39:0/2678696325 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) ==== 80+0+5304 (secure 0 0 0) 0x7f11700634a0 con 0x7f11880ae8a0 2024-09-21T07:47:51.675 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:47:51.675 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:47:51.676 INFO:teuthology.orchestra.run.smithi039.stdout: "auth_dump": [ 2024-09-21T07:47:51.676 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:51.676 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.a", 2024-09-21T07:47:51.676 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5m+P6EERAABQqmlkrocSi1+hwc73Adrg==", 2024-09-21T07:47:51.676 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:47:51.676 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:47:51.676 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:47:51.676 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:47:51.676 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:47:51.676 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:51.676 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:51.676 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:51.676 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.b", 2024-09-21T07:47:51.676 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5m5/GeLRAAfE8fr5dbitQa89rNK9pZ2Q==", 2024-09-21T07:47:51.677 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:47:51.677 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:47:51.677 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:47:51.677 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:47:51.677 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:47:51.677 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:51.677 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:51.677 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:51.677 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.c", 2024-09-21T07:47:51.677 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5msbx6HxAALsF8ORvIoJpIAFZ5uxmOEw==", 2024-09-21T07:47:51.677 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:47:51.677 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:47:51.677 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:47:51.677 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:47:51.678 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:47:51.678 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:51.678 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:51.678 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:51.678 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.d", 2024-09-21T07:47:51.678 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mIrcBABAApKAB/7FkZ5glNTxlbkIsBw==", 2024-09-21T07:47:51.678 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:47:51.678 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:47:51.678 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:47:51.678 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:47:51.678 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:47:51.678 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:51.678 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:51.678 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:51.679 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.0", 2024-09-21T07:47:51.679 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB7du5msw81DhAAmVimqN4x9K3YpT0nbovppQ==", 2024-09-21T07:47:51.679 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:47:51.679 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:47:51.679 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:47:51.679 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:47:51.679 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:51.679 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:51.679 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:51.679 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.1", 2024-09-21T07:47:51.679 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB+du5mdnCIOhAAtVB4ThehQwb0zRriyKcaxA==", 2024-09-21T07:47:51.679 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:47:51.679 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:47:51.680 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:47:51.680 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:47:51.680 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:51.680 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:51.680 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:51.680 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.2", 2024-09-21T07:47:51.680 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCCdu5mTfWxDhAAadJCbPN5xBx/f9L0v2yTdg==", 2024-09-21T07:47:51.680 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:47:51.680 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:47:51.680 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:47:51.680 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:47:51.680 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:51.680 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:51.680 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:51.680 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.3", 2024-09-21T07:47:51.681 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCFdu5mL1E2HxAAyOhY97uUXEw5L1CcDeONVA==", 2024-09-21T07:47:51.681 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:47:51.681 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:47:51.681 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:47:51.681 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:47:51.681 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:51.681 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:51.681 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:51.681 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.4", 2024-09-21T07:47:51.681 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCLdu5m6PY/IhAAgvcTgm+6uWA7lIG4tcTIEg==", 2024-09-21T07:47:51.681 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:47:51.681 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:47:51.681 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:47:51.681 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:47:51.681 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:51.682 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:51.682 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:51.682 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.5", 2024-09-21T07:47:51.682 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCOdu5mE/wlMxAA3JBg4xH+N/PPC1Uv2ZVrYQ==", 2024-09-21T07:47:51.682 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:47:51.682 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:47:51.682 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:47:51.682 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:47:51.682 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:51.682 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:51.682 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:51.682 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.6", 2024-09-21T07:47:51.682 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCSdu5mC3TgBxAAwsArjeIdFcApY3/1OJffEg==", 2024-09-21T07:47:51.682 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:47:51.682 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:47:51.683 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:47:51.683 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:47:51.683 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:51.683 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:51.683 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:51.683 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.7", 2024-09-21T07:47:51.683 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCVdu5mSWHWFBAA9qPHWRdsvrB74inrp1lj4Q==", 2024-09-21T07:47:51.683 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:47:51.683 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:47:51.683 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:47:51.683 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:47:51.683 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:51.683 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:51.683 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:51.683 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.0", 2024-09-21T07:47:51.684 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mGfb1DBAA/aSMKR5or3xy53rB/iUjpA==", 2024-09-21T07:47:51.684 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:47:51.684 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:47:51.684 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow r", 2024-09-21T07:47:51.684 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T07:47:51.684 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:51.684 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:51.684 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:51.684 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.1", 2024-09-21T07:47:51.684 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mEkciIhAAid+bxJEr4P7vMnZ42PAy5w==", 2024-09-21T07:47:51.685 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:47:51.685 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:47:51.685 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow r", 2024-09-21T07:47:51.685 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow rw", 2024-09-21T07:47:51.685 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rwx" 2024-09-21T07:47:51.685 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:51.685 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:51.685 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:51.685 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.2", 2024-09-21T07:47:51.685 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mSu2oFxAAgjy2m/wWKh6xaoh+YDoU1Q==", 2024-09-21T07:47:51.685 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:47:51.685 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:47:51.685 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow r", 2024-09-21T07:47:51.685 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow rw", 2024-09-21T07:47:51.685 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rwx" 2024-09-21T07:47:51.686 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:51.686 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:51.686 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:51.686 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.3", 2024-09-21T07:47:51.686 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mCHy0LBAAZneF2jUWgfuyGehZFbOxDg==", 2024-09-21T07:47:51.686 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:47:51.686 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:47:51.686 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow r", 2024-09-21T07:47:51.686 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow rw", 2024-09-21T07:47:51.686 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rwx" 2024-09-21T07:47:51.686 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:51.686 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:51.686 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:51.686 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.admin", 2024-09-21T07:47:51.687 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBydu5m9LInERAAAlDpqb94f2Qb9VYJJwTBcA==", 2024-09-21T07:47:51.687 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:47:51.687 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow *", 2024-09-21T07:47:51.687 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:47:51.687 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:47:51.687 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:47:51.687 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:51.687 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:51.687 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:51.687 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mgr.x", 2024-09-21T07:47:51.687 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5mA2GRBhAAkyCKdgH6laHy8xNt080dXg==", 2024-09-21T07:47:51.687 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:47:51.687 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow *", 2024-09-21T07:47:51.687 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile mgr", 2024-09-21T07:47:51.687 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:47:51.688 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:51.688 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:51.688 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:51.688 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mgr.y", 2024-09-21T07:47:51.688 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBydu5mr+8aNxAA7TFYNms2WsXUfa4ITTJOHw==", 2024-09-21T07:47:51.688 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:47:51.688 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow *", 2024-09-21T07:47:51.688 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile mgr", 2024-09-21T07:47:51.688 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:47:51.688 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:51.688 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:51.688 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-21T07:47:51.688 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:47:51.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.677+0000 7f1187200640 1 -- 172.21.15.39:0/2678696325 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1164078b90 msgr2=0x7f116407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:51.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.677+0000 7f1187200640 1 --2- 172.21.15.39:0/2678696325 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1164078b90 0x7f116407b0a0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f1178006680 tx=0x7f1178006b70 comp rx=0 tx=0).stop 2024-09-21T07:47:51.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.677+0000 7f1187200640 1 -- 172.21.15.39:0/2678696325 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11880ae8a0 msgr2=0x7f11880cc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:51.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.677+0000 7f1187200640 1 --2- 172.21.15.39:0/2678696325 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11880ae8a0 0x7f11880cc2c0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f1170011ab0 tx=0x7f1170011eb0 comp rx=0 tx=0).stop 2024-09-21T07:47:51.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.677+0000 7f1187200640 1 -- 172.21.15.39:0/2678696325 shutdown_connections 2024-09-21T07:47:51.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.677+0000 7f1187200640 1 --2- 172.21.15.39:0/2678696325 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1164078b90 0x7f116407b0a0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:51.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.677+0000 7f1187200640 1 --2- 172.21.15.39:0/2678696325 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f11880aecb0 0x7f11880cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:51.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.677+0000 7f1187200640 1 --2- 172.21.15.39:0/2678696325 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11880ae8a0 0x7f11880cc2c0 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:51.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.677+0000 7f1187200640 1 --2- 172.21.15.39:0/2678696325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f116c005250 0x7f11880afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:51.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.677+0000 7f1187200640 1 -- 172.21.15.39:0/2678696325 >> 172.21.15.39:0/2678696325 conn(0x7f118806cfc0 msgr2=0x7f118815edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:51.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.677+0000 7f1187200640 1 -- 172.21.15.39:0/2678696325 shutdown_connections 2024-09-21T07:47:51.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.677+0000 7f1187200640 1 -- 172.21.15.39:0/2678696325 wait complete. 2024-09-21T07:47:51.700 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T07:47:51.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.929+0000 7fc499000640 1 -- 172.21.15.39:0/15930577 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4940cfc50 msgr2=0x7fc4940d0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:51.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.929+0000 7fc499000640 1 --2- 172.21.15.39:0/15930577 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4940cfc50 0x7fc4940d0390 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fc48c005fd0 tx=0x7fc48c02a8b0 comp rx=0 tx=0).stop 2024-09-21T07:47:51.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.929+0000 7fc499000640 1 -- 172.21.15.39:0/15930577 shutdown_connections 2024-09-21T07:47:51.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.929+0000 7fc499000640 1 --2- 172.21.15.39:0/15930577 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4940cfc50 0x7fc4940d0390 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:51.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.929+0000 7fc499000640 1 --2- 172.21.15.39:0/15930577 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc494160660 0x7fc4940cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:51.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.929+0000 7fc499000640 1 -- 172.21.15.39:0/15930577 >> 172.21.15.39:0/15930577 conn(0x7fc49406cfc0 msgr2=0x7fc49406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:51.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.929+0000 7fc499000640 1 -- 172.21.15.39:0/15930577 shutdown_connections 2024-09-21T07:47:51.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.929+0000 7fc499000640 1 -- 172.21.15.39:0/15930577 wait complete. 2024-09-21T07:47:51.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.931+0000 7fc499000640 1 Processor -- start 2024-09-21T07:47:51.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.932+0000 7fc499000640 1 -- start start 2024-09-21T07:47:51.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.933+0000 7fc499000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4940cfc50 0x7fc4941f9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:51.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.933+0000 7fc499000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc494160250 0x7fc4940b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:51.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.933+0000 7fc499000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc494160660 0x7fc49413e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:51.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.933+0000 7fc499000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc4940ccdc0 con 0x7fc494160250 2024-09-21T07:47:51.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.933+0000 7fc499000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc4940ccc40 con 0x7fc4940cfc50 2024-09-21T07:47:51.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.933+0000 7fc499000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc4940ccf40 con 0x7fc494160660 2024-09-21T07:47:51.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.933+0000 7fc48be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc494160250 0x7fc4940b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:51.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.933+0000 7fc48be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc494160250 0x7fc4940b4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37872/0 (socket says 172.21.15.39:37872) 2024-09-21T07:47:51.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.933+0000 7fc48be00640 1 -- 172.21.15.39:0/648686990 learned_addr learned my addr 172.21.15.39:0/648686990 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:51.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.934+0000 7fc493400640 1 --2- 172.21.15.39:0/648686990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4940cfc50 0x7fc4941f9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:51.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.934+0000 7fc493e00640 1 --2- 172.21.15.39:0/648686990 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc494160660 0x7fc49413e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:51.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.934+0000 7fc48be00640 1 -- 172.21.15.39:0/648686990 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc494160660 msgr2=0x7fc49413e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:51.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.934+0000 7fc48be00640 1 --2- 172.21.15.39:0/648686990 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc494160660 0x7fc49413e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:51.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.934+0000 7fc48be00640 1 -- 172.21.15.39:0/648686990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4940cfc50 msgr2=0x7fc4941f9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:51.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.934+0000 7fc48be00640 1 --2- 172.21.15.39:0/648686990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4940cfc50 0x7fc4941f9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:51.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.934+0000 7fc48be00640 1 -- 172.21.15.39:0/648686990 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4940aeab0 con 0x7fc494160250 2024-09-21T07:47:51.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.934+0000 7fc48be00640 1 --2- 172.21.15.39:0/648686990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc494160250 0x7fc4940b4ae0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7fc47c00d740 tx=0x7fc47c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:51.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.934+0000 7fc48b400640 1 -- 172.21.15.39:0/648686990 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47c017070 con 0x7fc494160250 2024-09-21T07:47:51.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.934+0000 7fc48b400640 1 -- 172.21.15.39:0/648686990 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc47c009d20 con 0x7fc494160250 2024-09-21T07:47:51.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.935+0000 7fc48b400640 1 -- 172.21.15.39:0/648686990 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47c0106c0 con 0x7fc494160250 2024-09-21T07:47:51.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.935+0000 7fc499000640 1 -- 172.21.15.39:0/648686990 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4940aed90 con 0x7fc494160250 2024-09-21T07:47:51.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.935+0000 7fc499000640 1 -- 172.21.15.39:0/648686990 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4940af350 con 0x7fc494160250 2024-09-21T07:47:51.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.936+0000 7fc499000640 1 -- 172.21.15.39:0/648686990 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4940cd3f0 con 0x7fc494160250 2024-09-21T07:47:51.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.937+0000 7fc48b400640 1 -- 172.21.15.39:0/648686990 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc47c01a070 con 0x7fc494160250 2024-09-21T07:47:51.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.938+0000 7fc48b400640 1 --2- 172.21.15.39:0/648686990 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc470078b90 0x7fc47007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:51.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.938+0000 7fc48b400640 1 -- 172.21.15.39:0/648686990 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(202..202 src has 1..202) ==== 6108+0+0 (secure 0 0 0) 0x7fc47c09b090 con 0x7fc494160250 2024-09-21T07:47:51.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.938+0000 7fc493400640 1 --2- 172.21.15.39:0/648686990 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc470078b90 0x7fc47007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:51.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.939+0000 7fc493400640 1 --2- 172.21.15.39:0/648686990 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc470078b90 0x7fc47007b0a0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fc484007fd0 tx=0x7fc484001370 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:51.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:51.943+0000 7fc48b400640 1 -- 172.21.15.39:0/648686990 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc47c062c50 con 0x7fc494160250 2024-09-21T07:47:52.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.146+0000 7fc499000640 1 -- 172.21.15.39:0/648686990 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fc494073190 con 0x7fc494160250 2024-09-21T07:47:52.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.146+0000 7fc48b400640 1 -- 172.21.15.39:0/648686990 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v202) ==== 84+0+287 (secure 0 0 0) 0x7fc47c067680 con 0x7fc494160250 2024-09-21T07:47:52.147 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:47:52.148 INFO:teuthology.orchestra.run.smithi039.stdout:[ 2024-09-21T07:47:52.148 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:52.148 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 1, 2024-09-21T07:47:52.148 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": ".mgr" 2024-09-21T07:47:52.148 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:52.148 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:52.148 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 6, 2024-09-21T07:47:52.149 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": "snap-test-pool" 2024-09-21T07:47:52.149 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:52.149 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:52.149 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 11, 2024-09-21T07:47:52.149 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": "test_data_pool" 2024-09-21T07:47:52.149 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:47:52.149 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:47:52.149 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 12, 2024-09-21T07:47:52.149 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": "test_metadata_pool" 2024-09-21T07:47:52.150 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:47:52.150 INFO:teuthology.orchestra.run.smithi039.stdout:] 2024-09-21T07:47:52.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.149+0000 7fc499000640 1 -- 172.21.15.39:0/648686990 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc470078b90 msgr2=0x7fc47007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:52.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.149+0000 7fc499000640 1 --2- 172.21.15.39:0/648686990 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc470078b90 0x7fc47007b0a0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fc484007fd0 tx=0x7fc484001370 comp rx=0 tx=0).stop 2024-09-21T07:47:52.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.149+0000 7fc499000640 1 -- 172.21.15.39:0/648686990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc494160250 msgr2=0x7fc4940b4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:52.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.149+0000 7fc499000640 1 --2- 172.21.15.39:0/648686990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc494160250 0x7fc4940b4ae0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7fc47c00d740 tx=0x7fc47c00dc00 comp rx=0 tx=0).stop 2024-09-21T07:47:52.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.149+0000 7fc499000640 1 -- 172.21.15.39:0/648686990 shutdown_connections 2024-09-21T07:47:52.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.149+0000 7fc499000640 1 --2- 172.21.15.39:0/648686990 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc470078b90 0x7fc47007b0a0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:52.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.149+0000 7fc499000640 1 --2- 172.21.15.39:0/648686990 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc494160660 0x7fc49413e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:52.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.149+0000 7fc499000640 1 --2- 172.21.15.39:0/648686990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc494160250 0x7fc4940b4ae0 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:52.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.149+0000 7fc499000640 1 --2- 172.21.15.39:0/648686990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4940cfc50 0x7fc4941f9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:52.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.149+0000 7fc499000640 1 -- 172.21.15.39:0/648686990 >> 172.21.15.39:0/648686990 conn(0x7fc49406cfc0 msgr2=0x7fc4940afa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:52.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.149+0000 7fc499000640 1 -- 172.21.15.39:0/648686990 shutdown_connections 2024-09-21T07:47:52.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.149+0000 7fc499000640 1 -- 172.21.15.39:0/648686990 wait complete. 2024-09-21T07:47:52.171 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T07:47:52.171 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T07:47:52.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.400+0000 7f93e3e00640 1 -- 172.21.15.39:0/3513823977 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93e40732b0 msgr2=0x7f93e40736c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:52.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.400+0000 7f93e3e00640 1 --2- 172.21.15.39:0/3513823977 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93e40732b0 0x7f93e40736c0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f93e40b2970 tx=0x7f93cc02a540 comp rx=0 tx=0).stop 2024-09-21T07:47:52.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.400+0000 7f93e3e00640 1 -- 172.21.15.39:0/3513823977 shutdown_connections 2024-09-21T07:47:52.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.400+0000 7f93e3e00640 1 --2- 172.21.15.39:0/3513823977 >> v2:172.21.15.148:3300/0 conn(0x7f93e4155060 0x7f93e4155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:52.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.400+0000 7f93e3e00640 1 --2- 172.21.15.39:0/3513823977 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93e40732b0 0x7f93e40736c0 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:52.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.400+0000 7f93e3e00640 1 -- 172.21.15.39:0/3513823977 >> 172.21.15.39:0/3513823977 conn(0x7f93e406d660 msgr2=0x7f93e40b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:52.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.400+0000 7f93e3e00640 1 -- 172.21.15.39:0/3513823977 shutdown_connections 2024-09-21T07:47:52.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.400+0000 7f93e3e00640 1 -- 172.21.15.39:0/3513823977 wait complete. 2024-09-21T07:47:52.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.402+0000 7f93e3e00640 1 Processor -- start 2024-09-21T07:47:52.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.403+0000 7f93e3e00640 1 -- start start 2024-09-21T07:47:52.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.404+0000 7f93e3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93e40732b0 0x7f93e40cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:52.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.404+0000 7f93e3e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93e40b4db0 0x7f93e4158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:52.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.404+0000 7f93e3e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f93e4155060 0x7f93e41465c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:52.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.404+0000 7f93e3e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f93e4155b30 con 0x7f93e40732b0 2024-09-21T07:47:52.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.404+0000 7f93e3e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f93e41559b0 con 0x7f93e40b4db0 2024-09-21T07:47:52.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.404+0000 7f93e3e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f93e4155cb0 con 0x7f93e4155060 2024-09-21T07:47:52.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.404+0000 7f93e2a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93e40732b0 0x7f93e40cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:52.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.404+0000 7f93e2a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93e40732b0 0x7f93e40cf6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53524/0 (socket says 172.21.15.39:53524) 2024-09-21T07:47:52.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.404+0000 7f93e2a00640 1 -- 172.21.15.39:0/4270229936 learned_addr learned my addr 172.21.15.39:0/4270229936 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:52.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.404+0000 7f93e3400640 1 --2- 172.21.15.39:0/4270229936 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f93e4155060 0x7f93e41465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:52.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.404+0000 7f93da000640 1 --2- 172.21.15.39:0/4270229936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93e40b4db0 0x7f93e4158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:52.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.405+0000 7f93e2a00640 1 -- 172.21.15.39:0/4270229936 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f93e4155060 msgr2=0x7f93e41465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:52.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.405+0000 7f93e2a00640 1 --2- 172.21.15.39:0/4270229936 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f93e4155060 0x7f93e41465c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:52.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.405+0000 7f93e2a00640 1 -- 172.21.15.39:0/4270229936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93e40b4db0 msgr2=0x7f93e4158590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:52.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.405+0000 7f93e2a00640 1 --2- 172.21.15.39:0/4270229936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93e40b4db0 0x7f93e4158590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:52.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.405+0000 7f93e2a00640 1 -- 172.21.15.39:0/4270229936 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e40c9700 con 0x7f93e40732b0 2024-09-21T07:47:52.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.405+0000 7f93da000640 1 --2- 172.21.15.39:0/4270229936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93e40b4db0 0x7f93e4158590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:52.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.405+0000 7f93e2a00640 1 --2- 172.21.15.39:0/4270229936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93e40732b0 0x7f93e40cf6f0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f93d400cbf0 tx=0x7f93d400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:52.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.405+0000 7f93dbe00640 1 -- 172.21.15.39:0/4270229936 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d4016020 con 0x7f93e40732b0 2024-09-21T07:47:52.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.405+0000 7f93dbe00640 1 -- 172.21.15.39:0/4270229936 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93d400ad90 con 0x7f93e40732b0 2024-09-21T07:47:52.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.405+0000 7f93dbe00640 1 -- 172.21.15.39:0/4270229936 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d4012680 con 0x7f93e40732b0 2024-09-21T07:47:52.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.405+0000 7f93e3e00640 1 -- 172.21.15.39:0/4270229936 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93e40c99e0 con 0x7f93e40732b0 2024-09-21T07:47:52.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.406+0000 7f93e3e00640 1 -- 172.21.15.39:0/4270229936 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93e4157d90 con 0x7f93e40732b0 2024-09-21T07:47:52.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.407+0000 7f93e3e00640 1 -- 172.21.15.39:0/4270229936 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93e40736c0 con 0x7f93e40732b0 2024-09-21T07:47:52.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.408+0000 7f93dbe00640 1 -- 172.21.15.39:0/4270229936 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f93d4019020 con 0x7f93e40732b0 2024-09-21T07:47:52.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.409+0000 7f93dbe00640 1 --2- 172.21.15.39:0/4270229936 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f93c0078b90 0x7f93c007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:52.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.409+0000 7f93dbe00640 1 -- 172.21.15.39:0/4270229936 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(202..202 src has 1..202) ==== 6108+0+0 (secure 0 0 0) 0x7f93d409b500 con 0x7f93e40732b0 2024-09-21T07:47:52.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.409+0000 7f93da000640 1 --2- 172.21.15.39:0/4270229936 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f93c0078b90 0x7f93c007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:52.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.410+0000 7f93da000640 1 --2- 172.21.15.39:0/4270229936 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f93c0078b90 0x7f93c007b0a0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f93cc034000 tx=0x7f93cc004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:52.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.413+0000 7f93dbe00640 1 -- 172.21.15.39:0/4270229936 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f93d4020020 con 0x7f93e40732b0 2024-09-21T07:47:52.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:52.610+0000 7f93e3e00640 1 -- 172.21.15.39:0/4270229936 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f93e409e240 con 0x7f93e40732b0 2024-09-21T07:47:53.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.150+0000 7f93dbe00640 1 -- 172.21.15.39:0/4270229936 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v203) ==== 138+0+0 (secure 0 0 0) 0x7f93e40b51c0 con 0x7f93e40732b0 2024-09-21T07:47:53.151 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' created 2024-09-21T07:47:53.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.154+0000 7f93e3e00640 1 -- 172.21.15.39:0/4270229936 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f93c0078b90 msgr2=0x7f93c007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:53.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.154+0000 7f93e3e00640 1 --2- 172.21.15.39:0/4270229936 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f93c0078b90 0x7f93c007b0a0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f93cc034000 tx=0x7f93cc004730 comp rx=0 tx=0).stop 2024-09-21T07:47:53.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.154+0000 7f93e3e00640 1 -- 172.21.15.39:0/4270229936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93e40732b0 msgr2=0x7f93e40cf6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:53.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.154+0000 7f93e3e00640 1 --2- 172.21.15.39:0/4270229936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93e40732b0 0x7f93e40cf6f0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f93d400cbf0 tx=0x7f93d400a060 comp rx=0 tx=0).stop 2024-09-21T07:47:53.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.154+0000 7f93e3e00640 1 -- 172.21.15.39:0/4270229936 shutdown_connections 2024-09-21T07:47:53.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.154+0000 7f93e3e00640 1 --2- 172.21.15.39:0/4270229936 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f93c0078b90 0x7f93c007b0a0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:53.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.154+0000 7f93e3e00640 1 --2- 172.21.15.39:0/4270229936 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f93e4155060 0x7f93e41465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:53.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.154+0000 7f93e3e00640 1 --2- 172.21.15.39:0/4270229936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f93e40b4db0 0x7f93e4158590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:53.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.154+0000 7f93e3e00640 1 --2- 172.21.15.39:0/4270229936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93e40732b0 0x7f93e40cf6f0 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:53.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.154+0000 7f93e3e00640 1 -- 172.21.15.39:0/4270229936 >> 172.21.15.39:0/4270229936 conn(0x7f93e406d660 msgr2=0x7f93e40d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:53.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.155+0000 7f93e3e00640 1 -- 172.21.15.39:0/4270229936 shutdown_connections 2024-09-21T07:47:53.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.156+0000 7f93e3e00640 1 -- 172.21.15.39:0/4270229936 wait complete. 2024-09-21T07:47:53.182 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T07:47:53.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.415+0000 7f4204600640 1 -- 172.21.15.39:0/2005522208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41fc073dd0 msgr2=0x7f41fc15d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:53.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.415+0000 7f4204600640 1 --2- 172.21.15.39:0/2005522208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41fc073dd0 0x7f41fc15d5b0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f41f0005fd0 tx=0x7f41f002a8b0 comp rx=0 tx=0).stop 2024-09-21T07:47:53.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.415+0000 7f4204600640 1 -- 172.21.15.39:0/2005522208 shutdown_connections 2024-09-21T07:47:53.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.415+0000 7f4204600640 1 --2- 172.21.15.39:0/2005522208 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41fc0d0630 0x7f41fc0d0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:53.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.415+0000 7f4204600640 1 --2- 172.21.15.39:0/2005522208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41fc073dd0 0x7f41fc15d5b0 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:53.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.415+0000 7f4204600640 1 -- 172.21.15.39:0/2005522208 >> 172.21.15.39:0/2005522208 conn(0x7f41fc06cfc0 msgr2=0x7f41fc06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:53.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.415+0000 7f4204600640 1 -- 172.21.15.39:0/2005522208 shutdown_connections 2024-09-21T07:47:53.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.415+0000 7f4204600640 1 -- 172.21.15.39:0/2005522208 wait complete. 2024-09-21T07:47:53.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.418+0000 7f4204600640 1 Processor -- start 2024-09-21T07:47:53.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.418+0000 7f4204600640 1 -- start start 2024-09-21T07:47:53.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.419+0000 7f4204600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41fc073dd0 0x7f41fc14ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:53.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.419+0000 7f4204600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41fc0d0630 0x7f41fc0b4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:53.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.419+0000 7f4204600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41fc15daf0 0x7f41fc0b4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:53.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.419+0000 7f4204600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc0cd230 con 0x7f41fc15daf0 2024-09-21T07:47:53.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.419+0000 7f4204600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc0cd0b0 con 0x7f41fc073dd0 2024-09-21T07:47:53.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.419+0000 7f4204600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f41fc0cd3b0 con 0x7f41fc0d0630 2024-09-21T07:47:53.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.419+0000 7f4202600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41fc15daf0 0x7f41fc0b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:53.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.420+0000 7f4202600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41fc15daf0 0x7f41fc0b4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53540/0 (socket says 172.21.15.39:53540) 2024-09-21T07:47:53.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.420+0000 7f4202600640 1 -- 172.21.15.39:0/1361341001 learned_addr learned my addr 172.21.15.39:0/1361341001 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:53.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.420+0000 7f4201200640 1 --2- 172.21.15.39:0/1361341001 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41fc0d0630 0x7f41fc0b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:53.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.420+0000 7f4201c00640 1 --2- 172.21.15.39:0/1361341001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41fc073dd0 0x7f41fc14ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:53.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.420+0000 7f4202600640 1 -- 172.21.15.39:0/1361341001 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41fc0d0630 msgr2=0x7f41fc0b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:53.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.420+0000 7f4202600640 1 --2- 172.21.15.39:0/1361341001 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41fc0d0630 0x7f41fc0b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:53.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.420+0000 7f4202600640 1 -- 172.21.15.39:0/1361341001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41fc073dd0 msgr2=0x7f41fc14ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:53.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.420+0000 7f4202600640 1 --2- 172.21.15.39:0/1361341001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41fc073dd0 0x7f41fc14ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:53.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.420+0000 7f4202600640 1 -- 172.21.15.39:0/1361341001 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41fc1e4f60 con 0x7f41fc15daf0 2024-09-21T07:47:53.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.420+0000 7f4202600640 1 --2- 172.21.15.39:0/1361341001 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41fc15daf0 0x7f41fc0b4860 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f41f400e740 tx=0x7f41f400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:53.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.421+0000 7f41f9600640 1 -- 172.21.15.39:0/1361341001 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f400ccc0 con 0x7f41fc15daf0 2024-09-21T07:47:53.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.421+0000 7f41f9600640 1 -- 172.21.15.39:0/1361341001 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41f400ce80 con 0x7f41fc15daf0 2024-09-21T07:47:53.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.421+0000 7f41f9600640 1 -- 172.21.15.39:0/1361341001 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f4005280 con 0x7f41fc15daf0 2024-09-21T07:47:53.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.421+0000 7f4204600640 1 -- 172.21.15.39:0/1361341001 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41fc1e51d0 con 0x7f41fc15daf0 2024-09-21T07:47:53.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.421+0000 7f4204600640 1 -- 172.21.15.39:0/1361341001 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41fc0b4da0 con 0x7f41fc15daf0 2024-09-21T07:47:53.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.422+0000 7f4204600640 1 -- 172.21.15.39:0/1361341001 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41fc151ce0 con 0x7f41fc15daf0 2024-09-21T07:47:53.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.423+0000 7f41f9600640 1 -- 172.21.15.39:0/1361341001 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f41f4005420 con 0x7f41fc15daf0 2024-09-21T07:47:53.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.424+0000 7f41f9600640 1 --2- 172.21.15.39:0/1361341001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f41dc078b90 0x7f41dc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:53.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.424+0000 7f41f9600640 1 -- 172.21.15.39:0/1361341001 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(203..203 src has 1..203) ==== 6478+0+0 (secure 0 0 0) 0x7f41f409b8e0 con 0x7f41fc15daf0 2024-09-21T07:47:53.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.425+0000 7f4201c00640 1 --2- 172.21.15.39:0/1361341001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f41dc078b90 0x7f41dc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:53.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.426+0000 7f4201c00640 1 --2- 172.21.15.39:0/1361341001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f41dc078b90 0x7f41dc07b0a0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f41f0034000 tx=0x7f41f0004ab0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:53.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.429+0000 7f41f9600640 1 -- 172.21.15.39:0/1361341001 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f41f4063330 con 0x7f41fc15daf0 2024-09-21T07:47:53.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:53.625+0000 7f4204600640 1 -- 172.21.15.39:0/1361341001 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f41fc0cd8d0 con 0x7f41fc15daf0 2024-09-21T07:47:54.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.159+0000 7f41f9600640 1 -- 172.21.15.39:0/1361341001 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v204) ==== 170+0+0 (secure 0 0 0) 0x7f41f4067d60 con 0x7f41fc15daf0 2024-09-21T07:47:54.160 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' created 2024-09-21T07:47:54.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.162+0000 7f4204600640 1 -- 172.21.15.39:0/1361341001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f41dc078b90 msgr2=0x7f41dc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:54.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.162+0000 7f4204600640 1 --2- 172.21.15.39:0/1361341001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f41dc078b90 0x7f41dc07b0a0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f41f0034000 tx=0x7f41f0004ab0 comp rx=0 tx=0).stop 2024-09-21T07:47:54.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.162+0000 7f4204600640 1 -- 172.21.15.39:0/1361341001 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41fc15daf0 msgr2=0x7f41fc0b4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:54.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.162+0000 7f4204600640 1 --2- 172.21.15.39:0/1361341001 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41fc15daf0 0x7f41fc0b4860 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f41f400e740 tx=0x7f41f400ec00 comp rx=0 tx=0).stop 2024-09-21T07:47:54.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.162+0000 7f4204600640 1 -- 172.21.15.39:0/1361341001 shutdown_connections 2024-09-21T07:47:54.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.162+0000 7f4204600640 1 --2- 172.21.15.39:0/1361341001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f41dc078b90 0x7f41dc07b0a0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:54.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.162+0000 7f4204600640 1 --2- 172.21.15.39:0/1361341001 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41fc15daf0 0x7f41fc0b4860 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:54.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.162+0000 7f4204600640 1 --2- 172.21.15.39:0/1361341001 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41fc0d0630 0x7f41fc0b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:54.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.162+0000 7f4204600640 1 --2- 172.21.15.39:0/1361341001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41fc073dd0 0x7f41fc14ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:54.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.162+0000 7f4204600640 1 -- 172.21.15.39:0/1361341001 >> 172.21.15.39:0/1361341001 conn(0x7f41fc06cfc0 msgr2=0x7f41fc0b0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:54.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.162+0000 7f4204600640 1 -- 172.21.15.39:0/1361341001 shutdown_connections 2024-09-21T07:47:54.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.162+0000 7f4204600640 1 -- 172.21.15.39:0/1361341001 wait complete. 2024-09-21T07:47:54.212 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T07:47:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.440+0000 7efde6a00640 1 -- 172.21.15.39:0/1703579887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efde006e610 msgr2=0x7efde0163170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.440+0000 7efde6a00640 1 --2- 172.21.15.39:0/1703579887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efde006e610 0x7efde0163170 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7efdc8005fd0 tx=0x7efdc802a5d0 comp rx=0 tx=0).stop 2024-09-21T07:47:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.441+0000 7efde6a00640 1 -- 172.21.15.39:0/1703579887 shutdown_connections 2024-09-21T07:47:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.441+0000 7efde6a00640 1 --2- 172.21.15.39:0/1703579887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efde006e610 0x7efde0163170 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:54.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.441+0000 7efde6a00640 1 --2- 172.21.15.39:0/1703579887 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efde00cfa50 0x7efde006e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:54.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.441+0000 7efde6a00640 1 -- 172.21.15.39:0/1703579887 >> 172.21.15.39:0/1703579887 conn(0x7efde00cef30 msgr2=0x7efde00cf340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:54.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.441+0000 7efde6a00640 1 -- 172.21.15.39:0/1703579887 shutdown_connections 2024-09-21T07:47:54.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.441+0000 7efde6a00640 1 -- 172.21.15.39:0/1703579887 wait complete. 2024-09-21T07:47:54.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.443+0000 7efde6a00640 1 Processor -- start 2024-09-21T07:47:54.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.443+0000 7efde6a00640 1 -- start start 2024-09-21T07:47:54.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.444+0000 7efde6a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efde006e610 0x7efde009c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:54.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.444+0000 7efde6a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efde00cfa50 0x7efde009ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:54.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.444+0000 7efde6a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efde01636b0 0x7efde01f8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:54.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.444+0000 7efde6a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efde00b2e80 con 0x7efde006e610 2024-09-21T07:47:54.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.444+0000 7efde6a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efde00b2d00 con 0x7efde01636b0 2024-09-21T07:47:54.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.444+0000 7efde6a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7efde0163ac0 con 0x7efde00cfa50 2024-09-21T07:47:54.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.445+0000 7efddfe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efde006e610 0x7efde009c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:54.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.445+0000 7efddfe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efde006e610 0x7efde009c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53546/0 (socket says 172.21.15.39:53546) 2024-09-21T07:47:54.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.445+0000 7efddfe00640 1 -- 172.21.15.39:0/1912125150 learned_addr learned my addr 172.21.15.39:0/1912125150 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:54.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.445+0000 7efdd7e00640 1 --2- 172.21.15.39:0/1912125150 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efde00cfa50 0x7efde009ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:54.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.445+0000 7efde4a00640 1 --2- 172.21.15.39:0/1912125150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efde01636b0 0x7efde01f8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:54.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.445+0000 7efddfe00640 1 -- 172.21.15.39:0/1912125150 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efde00cfa50 msgr2=0x7efde009ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:54.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.445+0000 7efddfe00640 1 --2- 172.21.15.39:0/1912125150 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efde00cfa50 0x7efde009ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:54.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.445+0000 7efddfe00640 1 -- 172.21.15.39:0/1912125150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efde01636b0 msgr2=0x7efde01f8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:54.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.445+0000 7efddfe00640 1 --2- 172.21.15.39:0/1912125150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efde01636b0 0x7efde01f8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:54.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.445+0000 7efddfe00640 1 -- 172.21.15.39:0/1912125150 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde00a0480 con 0x7efde006e610 2024-09-21T07:47:54.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.446+0000 7efddfe00640 1 --2- 172.21.15.39:0/1912125150 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efde006e610 0x7efde009c260 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7efdd000d720 tx=0x7efdd000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:54.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.446+0000 7efddcc00640 1 -- 172.21.15.39:0/1912125150 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd0017070 con 0x7efde006e610 2024-09-21T07:47:54.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.446+0000 7efddcc00640 1 -- 172.21.15.39:0/1912125150 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efdd0009b30 con 0x7efde006e610 2024-09-21T07:47:54.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.446+0000 7efddcc00640 1 -- 172.21.15.39:0/1912125150 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd0010460 con 0x7efde006e610 2024-09-21T07:47:54.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.446+0000 7efde6a00640 1 -- 172.21.15.39:0/1912125150 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efde009ca90 con 0x7efde006e610 2024-09-21T07:47:54.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.446+0000 7efde6a00640 1 -- 172.21.15.39:0/1912125150 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efde009d050 con 0x7efde006e610 2024-09-21T07:47:54.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.447+0000 7efde6a00640 1 -- 172.21.15.39:0/1912125150 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efde00d01b0 con 0x7efde006e610 2024-09-21T07:47:54.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.448+0000 7efddcc00640 1 -- 172.21.15.39:0/1912125150 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7efdd001a070 con 0x7efde006e610 2024-09-21T07:47:54.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.449+0000 7efddcc00640 1 --2- 172.21.15.39:0/1912125150 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efdbc078b90 0x7efdbc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:54.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.449+0000 7efddcc00640 1 -- 172.21.15.39:0/1912125150 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(204..204 src has 1..204) ==== 6860+0+0 (secure 0 0 0) 0x7efdd009b380 con 0x7efde006e610 2024-09-21T07:47:54.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.450+0000 7efdd7e00640 1 --2- 172.21.15.39:0/1912125150 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efdbc078b90 0x7efdbc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:54.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.451+0000 7efdd7e00640 1 --2- 172.21.15.39:0/1912125150 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efdbc078b90 0x7efdbc07b0a0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7efdc8033000 tx=0x7efdc8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:54.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.454+0000 7efddcc00640 1 -- 172.21.15.39:0/1912125150 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7efde00d01b0 con 0x7efde006e610 2024-09-21T07:47:54.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:54.690+0000 7efde6a00640 1 -- 172.21.15.39:0/1912125150 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7efde0163170 con 0x7efde006e610 2024-09-21T07:47:55.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.160+0000 7efddcc00640 1 -- 172.21.15.39:0/1912125150 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 25 and data pool 26 v210) ==== 174+0+0 (secure 0 0 0) 0x7efdd0062c50 con 0x7efde006e610 2024-09-21T07:47:55.163 INFO:teuthology.orchestra.run.smithi039.stderr:new fs with metadata pool 25 and data pool 26 2024-09-21T07:47:55.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.164+0000 7efde6a00640 1 -- 172.21.15.39:0/1912125150 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efdbc078b90 msgr2=0x7efdbc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:55.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.164+0000 7efde6a00640 1 --2- 172.21.15.39:0/1912125150 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efdbc078b90 0x7efdbc07b0a0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7efdc8033000 tx=0x7efdc8004730 comp rx=0 tx=0).stop 2024-09-21T07:47:55.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.164+0000 7efde6a00640 1 -- 172.21.15.39:0/1912125150 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efde006e610 msgr2=0x7efde009c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:55.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.164+0000 7efde6a00640 1 --2- 172.21.15.39:0/1912125150 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efde006e610 0x7efde009c260 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7efdd000d720 tx=0x7efdd000dbe0 comp rx=0 tx=0).stop 2024-09-21T07:47:55.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.169+0000 7efde6a00640 1 -- 172.21.15.39:0/1912125150 shutdown_connections 2024-09-21T07:47:55.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.169+0000 7efde6a00640 1 --2- 172.21.15.39:0/1912125150 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efdbc078b90 0x7efdbc07b0a0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:55.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.169+0000 7efde6a00640 1 --2- 172.21.15.39:0/1912125150 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efde01636b0 0x7efde01f8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:55.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.169+0000 7efde6a00640 1 --2- 172.21.15.39:0/1912125150 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efde00cfa50 0x7efde009ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:55.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.169+0000 7efde6a00640 1 --2- 172.21.15.39:0/1912125150 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efde006e610 0x7efde009c260 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:55.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.169+0000 7efde6a00640 1 -- 172.21.15.39:0/1912125150 >> 172.21.15.39:0/1912125150 conn(0x7efde00cef30 msgr2=0x7efde0162960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:55.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.169+0000 7efde6a00640 1 -- 172.21.15.39:0/1912125150 shutdown_connections 2024-09-21T07:47:55.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.170+0000 7efde6a00640 1 -- 172.21.15.39:0/1912125150 wait complete. 2024-09-21T07:47:55.224 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:47:55.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.448+0000 7f3498e00640 1 -- 172.21.15.39:0/1328862878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34940cd260 msgr2=0x7f34940cf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:55.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.448+0000 7f3498e00640 1 --2- 172.21.15.39:0/1328862878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34940cd260 0x7f34940cf450 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f34940707f0 tx=0x7f348002a530 comp rx=0 tx=0).stop 2024-09-21T07:47:55.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.448+0000 7f3498e00640 1 -- 172.21.15.39:0/1328862878 shutdown_connections 2024-09-21T07:47:55.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.448+0000 7f3498e00640 1 --2- 172.21.15.39:0/1328862878 >> v2:172.21.15.148:3300/0 conn(0x7f34940cf990 0x7f34940b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:55.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.448+0000 7f3498e00640 1 --2- 172.21.15.39:0/1328862878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34940cd260 0x7f34940cf450 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:55.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.448+0000 7f3498e00640 1 -- 172.21.15.39:0/1328862878 >> 172.21.15.39:0/1328862878 conn(0x7f34940b2460 msgr2=0x7f349406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:55.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.448+0000 7f3498e00640 1 -- 172.21.15.39:0/1328862878 shutdown_connections 2024-09-21T07:47:55.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.448+0000 7f3498e00640 1 -- 172.21.15.39:0/1328862878 wait complete. 2024-09-21T07:47:55.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.451+0000 7f3498e00640 1 Processor -- start 2024-09-21T07:47:55.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.451+0000 7f3498e00640 1 -- start start 2024-09-21T07:47:55.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.452+0000 7f3498e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34940af430 0x7f3494150c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:55.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.452+0000 7f3498e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34940cd260 0x7f34941f7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:55.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.452+0000 7f3498e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f34940cf990 0x7f34940c3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:55.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.452+0000 7f3498e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f34940b45f0 con 0x7f34940af430 2024-09-21T07:47:55.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.452+0000 7f3498e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3494071230 con 0x7f34940cd260 2024-09-21T07:47:55.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.452+0000 7f3498e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f34940b4770 con 0x7f34940cf990 2024-09-21T07:47:55.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.452+0000 7f3491e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34940af430 0x7f3494150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:55.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.452+0000 7f3491e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34940af430 0x7f3494150c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53558/0 (socket says 172.21.15.39:53558) 2024-09-21T07:47:55.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.452+0000 7f3491e00640 1 -- 172.21.15.39:0/3806523406 learned_addr learned my addr 172.21.15.39:0/3806523406 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:55.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.453+0000 7f3491400640 1 --2- 172.21.15.39:0/3806523406 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34940cd260 0x7f34941f7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:55.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.453+0000 7f3492800640 1 --2- 172.21.15.39:0/3806523406 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f34940cf990 0x7f34940c3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:55.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.453+0000 7f3491e00640 1 -- 172.21.15.39:0/3806523406 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f34940cf990 msgr2=0x7f34940c3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:55.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.453+0000 7f3491e00640 1 --2- 172.21.15.39:0/3806523406 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f34940cf990 0x7f34940c3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:55.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.453+0000 7f3491e00640 1 -- 172.21.15.39:0/3806523406 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34940cd260 msgr2=0x7f34941f7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:55.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.453+0000 7f3491e00640 1 --2- 172.21.15.39:0/3806523406 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34940cd260 0x7f34941f7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:55.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.453+0000 7f3491e00640 1 -- 172.21.15.39:0/3806523406 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34941511c0 con 0x7f34940af430 2024-09-21T07:47:55.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.453+0000 7f3491400640 1 --2- 172.21.15.39:0/3806523406 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34940cd260 0x7f34941f7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:55.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.453+0000 7f3491e00640 1 --2- 172.21.15.39:0/3806523406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34940af430 0x7f3494150c80 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f348400c3d0 tx=0x7f348400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:55.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.453+0000 7f348a000640 1 -- 172.21.15.39:0/3806523406 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3484011790 con 0x7f34940af430 2024-09-21T07:47:55.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.453+0000 7f348a000640 1 -- 172.21.15.39:0/3806523406 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3484017070 con 0x7f34940af430 2024-09-21T07:47:55.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.454+0000 7f348a000640 1 -- 172.21.15.39:0/3806523406 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3484011cc0 con 0x7f34940af430 2024-09-21T07:47:55.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.454+0000 7f3498e00640 1 -- 172.21.15.39:0/3806523406 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34941562d0 con 0x7f34940af430 2024-09-21T07:47:55.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.454+0000 7f3498e00640 1 -- 172.21.15.39:0/3806523406 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34941f6f50 con 0x7f34940af430 2024-09-21T07:47:55.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.455+0000 7f3498e00640 1 -- 172.21.15.39:0/3806523406 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34940cdc20 con 0x7f34940af430 2024-09-21T07:47:55.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.456+0000 7f348a000640 1 -- 172.21.15.39:0/3806523406 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f348401c460 con 0x7f34940af430 2024-09-21T07:47:55.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.457+0000 7f348a000640 1 --2- 172.21.15.39:0/3806523406 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3474078bc0 0x7f347407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:55.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.457+0000 7f348a000640 1 -- 172.21.15.39:0/3806523406 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(205..205 src has 1..205) ==== 6960+0+0 (secure 0 0 0) 0x7f348409b120 con 0x7f34940af430 2024-09-21T07:47:55.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.457+0000 7f3491400640 1 --2- 172.21.15.39:0/3806523406 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3474078bc0 0x7f347407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:55.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.458+0000 7f3491400640 1 --2- 172.21.15.39:0/3806523406 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3474078bc0 0x7f347407b0d0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f3480034000 tx=0x7f3480004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:55.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.462+0000 7f348a000640 1 -- 172.21.15.39:0/3806523406 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3484062f80 con 0x7f34940af430 2024-09-21T07:47:55.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.663+0000 7f3498e00640 1 -- 172.21.15.39:0/3806523406 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f34940cf450 con 0x7f34940af430 2024-09-21T07:47:55.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.665+0000 7f348a000640 1 -- 172.21.15.39:0/3806523406 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v205) ==== 74+0+22410 (secure 0 0 0) 0x7f3484062880 con 0x7f34940af430 2024-09-21T07:47:55.667 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:47:55.667 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":205,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:47:54.691880+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:47:52.623339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"205","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-09-21T07:47:53.638900+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"205","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]},{"pool":20,"snaps":[{"begin":80,"length":23},{"begin":181,"length":23}]},{"pool":22,"snaps":[{"begin":100,"length":1},{"begin":199,"length":1}]},{"pool":24,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:47:55.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.668+0000 7f3498e00640 1 -- 172.21.15.39:0/3806523406 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3474078bc0 msgr2=0x7f347407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:55.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.668+0000 7f3498e00640 1 --2- 172.21.15.39:0/3806523406 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3474078bc0 0x7f347407b0d0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f3480034000 tx=0x7f3480004730 comp rx=0 tx=0).stop 2024-09-21T07:47:55.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.668+0000 7f3498e00640 1 -- 172.21.15.39:0/3806523406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34940af430 msgr2=0x7f3494150c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:55.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.668+0000 7f3498e00640 1 --2- 172.21.15.39:0/3806523406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34940af430 0x7f3494150c80 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f348400c3d0 tx=0x7f348400c890 comp rx=0 tx=0).stop 2024-09-21T07:47:55.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.668+0000 7f3498e00640 1 -- 172.21.15.39:0/3806523406 shutdown_connections 2024-09-21T07:47:55.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.668+0000 7f3498e00640 1 --2- 172.21.15.39:0/3806523406 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3474078bc0 0x7f347407b0d0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:55.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.668+0000 7f3498e00640 1 --2- 172.21.15.39:0/3806523406 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f34940cf990 0x7f34940c3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:55.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.668+0000 7f3498e00640 1 --2- 172.21.15.39:0/3806523406 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34940cd260 0x7f34941f7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:55.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.668+0000 7f3498e00640 1 --2- 172.21.15.39:0/3806523406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34940af430 0x7f3494150c80 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:55.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.668+0000 7f3498e00640 1 -- 172.21.15.39:0/3806523406 >> 172.21.15.39:0/3806523406 conn(0x7f34940b2460 msgr2=0x7f349415a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:55.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.668+0000 7f3498e00640 1 -- 172.21.15.39:0/3806523406 shutdown_connections 2024-09-21T07:47:55.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.668+0000 7f3498e00640 1 -- 172.21.15.39:0/3806523406 wait complete. 2024-09-21T07:47:55.692 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:47:55.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.916+0000 7f2531e00640 1 -- 172.21.15.39:0/3972876712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f252c15daf0 msgr2=0x7f252c0cf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:55.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.916+0000 7f2531e00640 1 --2- 172.21.15.39:0/3972876712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f252c15daf0 0x7f252c0cf450 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f2514009830 tx=0x7f251402a510 comp rx=0 tx=0).stop 2024-09-21T07:47:55.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.916+0000 7f2531e00640 1 -- 172.21.15.39:0/3972876712 shutdown_connections 2024-09-21T07:47:55.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.916+0000 7f2531e00640 1 --2- 172.21.15.39:0/3972876712 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f252c15daf0 0x7f252c0cf450 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:55.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.916+0000 7f2531e00640 1 --2- 172.21.15.39:0/3972876712 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f252c073dd0 0x7f252c15d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:55.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.916+0000 7f2531e00640 1 -- 172.21.15.39:0/3972876712 >> 172.21.15.39:0/3972876712 conn(0x7f252c06cfc0 msgr2=0x7f252c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:55.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.917+0000 7f2531e00640 1 -- 172.21.15.39:0/3972876712 shutdown_connections 2024-09-21T07:47:55.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.917+0000 7f2531e00640 1 -- 172.21.15.39:0/3972876712 wait complete. 2024-09-21T07:47:55.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.919+0000 7f2531e00640 1 Processor -- start 2024-09-21T07:47:55.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.919+0000 7f2531e00640 1 -- start start 2024-09-21T07:47:55.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.920+0000 7f2531e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f252c073dd0 0x7f252c145760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:55.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.920+0000 7f2531e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f252c0cf990 0x7f252c1568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:55.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.921+0000 7f2531e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f252c15daf0 0x7f252c156e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:55.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.921+0000 7f2531e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f252c0cd1d0 con 0x7f252c15daf0 2024-09-21T07:47:55.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.921+0000 7f2531e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f252c0cd050 con 0x7f252c0cf990 2024-09-21T07:47:55.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.921+0000 7f2531e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f252c0cd350 con 0x7f252c073dd0 2024-09-21T07:47:55.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.921+0000 7f252b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f252c15daf0 0x7f252c156e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:55.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.921+0000 7f252b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f252c15daf0 0x7f252c156e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53588/0 (socket says 172.21.15.39:53588) 2024-09-21T07:47:55.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.921+0000 7f252b400640 1 -- 172.21.15.39:0/3699399174 learned_addr learned my addr 172.21.15.39:0/3699399174 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:55.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.921+0000 7f252aa00640 1 --2- 172.21.15.39:0/3699399174 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f252c073dd0 0x7f252c145760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:55.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.921+0000 7f2522000640 1 --2- 172.21.15.39:0/3699399174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f252c0cf990 0x7f252c1568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:55.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.921+0000 7f252b400640 1 -- 172.21.15.39:0/3699399174 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f252c073dd0 msgr2=0x7f252c145760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:55.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.921+0000 7f252b400640 1 --2- 172.21.15.39:0/3699399174 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f252c073dd0 0x7f252c145760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:55.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.921+0000 7f252b400640 1 -- 172.21.15.39:0/3699399174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f252c0cf990 msgr2=0x7f252c1568c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:55.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.921+0000 7f252b400640 1 --2- 172.21.15.39:0/3699399174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f252c0cf990 0x7f252c1568c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:55.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.921+0000 7f252b400640 1 -- 172.21.15.39:0/3699399174 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f252c145ca0 con 0x7f252c15daf0 2024-09-21T07:47:55.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.921+0000 7f2522000640 1 --2- 172.21.15.39:0/3699399174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f252c0cf990 0x7f252c1568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:55.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.922+0000 7f252b400640 1 --2- 172.21.15.39:0/3699399174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f252c15daf0 0x7f252c156e00 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f2524002230 tx=0x7f252400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:55.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.922+0000 7f2523e00640 1 -- 172.21.15.39:0/3699399174 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2524016020 con 0x7f252c15daf0 2024-09-21T07:47:55.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.922+0000 7f2523e00640 1 -- 172.21.15.39:0/3699399174 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f252400ad20 con 0x7f252c15daf0 2024-09-21T07:47:55.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.922+0000 7f2531e00640 1 -- 172.21.15.39:0/3699399174 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f252c0a9a00 con 0x7f252c15daf0 2024-09-21T07:47:55.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.922+0000 7f2523e00640 1 -- 172.21.15.39:0/3699399174 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2524012680 con 0x7f252c15daf0 2024-09-21T07:47:55.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.922+0000 7f2531e00640 1 -- 172.21.15.39:0/3699399174 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f252c0a9fc0 con 0x7f252c15daf0 2024-09-21T07:47:55.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.923+0000 7f2531e00640 1 -- 172.21.15.39:0/3699399174 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f252c0cd800 con 0x7f252c15daf0 2024-09-21T07:47:55.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.924+0000 7f2523e00640 1 -- 172.21.15.39:0/3699399174 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2524019020 con 0x7f252c15daf0 2024-09-21T07:47:55.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.925+0000 7f2523e00640 1 --2- 172.21.15.39:0/3699399174 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2508078b90 0x7f250807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:55.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.926+0000 7f2523e00640 1 -- 172.21.15.39:0/3699399174 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(205..205 src has 1..205) ==== 6960+0+0 (secure 0 0 0) 0x7f252409b490 con 0x7f252c15daf0 2024-09-21T07:47:55.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.926+0000 7f252aa00640 1 --2- 172.21.15.39:0/3699399174 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2508078b90 0x7f250807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:55.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.927+0000 7f252aa00640 1 --2- 172.21.15.39:0/3699399174 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2508078b90 0x7f250807b0a0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f252c0730e0 tx=0x7f251c005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:55.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:55.930+0000 7f2523e00640 1 -- 172.21.15.39:0/3699399174 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2524020030 con 0x7f252c15daf0 2024-09-21T07:47:56.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.124+0000 7f2531e00640 1 -- 172.21.15.39:0/3699399174 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f252c146020 con 0x7f252c15daf0 2024-09-21T07:47:56.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.126+0000 7f2523e00640 1 -- 172.21.15.39:0/3699399174 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v205) ==== 74+0+22410 (secure 0 0 0) 0x7f2524063370 con 0x7f252c15daf0 2024-09-21T07:47:56.128 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:47:56.128 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":205,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:47:54.691880+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:47:52.623339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"205","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-09-21T07:47:53.638900+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"205","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]},{"pool":20,"snaps":[{"begin":80,"length":23},{"begin":181,"length":23}]},{"pool":22,"snaps":[{"begin":100,"length":1},{"begin":199,"length":1}]},{"pool":24,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:47:56.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.128+0000 7f2531e00640 1 -- 172.21.15.39:0/3699399174 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2508078b90 msgr2=0x7f250807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:56.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.128+0000 7f2531e00640 1 --2- 172.21.15.39:0/3699399174 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2508078b90 0x7f250807b0a0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f252c0730e0 tx=0x7f251c005f90 comp rx=0 tx=0).stop 2024-09-21T07:47:56.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.128+0000 7f2531e00640 1 -- 172.21.15.39:0/3699399174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f252c15daf0 msgr2=0x7f252c156e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:56.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.128+0000 7f2531e00640 1 --2- 172.21.15.39:0/3699399174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f252c15daf0 0x7f252c156e00 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f2524002230 tx=0x7f252400cf70 comp rx=0 tx=0).stop 2024-09-21T07:47:56.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.128+0000 7f2531e00640 1 -- 172.21.15.39:0/3699399174 shutdown_connections 2024-09-21T07:47:56.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.128+0000 7f2531e00640 1 --2- 172.21.15.39:0/3699399174 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2508078b90 0x7f250807b0a0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:56.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.128+0000 7f2531e00640 1 --2- 172.21.15.39:0/3699399174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f252c15daf0 0x7f252c156e00 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:56.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.128+0000 7f2531e00640 1 --2- 172.21.15.39:0/3699399174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f252c0cf990 0x7f252c1568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:56.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.128+0000 7f2531e00640 1 --2- 172.21.15.39:0/3699399174 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f252c073dd0 0x7f252c145760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:56.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.128+0000 7f2531e00640 1 -- 172.21.15.39:0/3699399174 >> 172.21.15.39:0/3699399174 conn(0x7f252c06cfc0 msgr2=0x7f252c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:56.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.128+0000 7f2531e00640 1 -- 172.21.15.39:0/3699399174 shutdown_connections 2024-09-21T07:47:56.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.128+0000 7f2531e00640 1 -- 172.21.15.39:0/3699399174 wait complete. 2024-09-21T07:47:56.152 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T07:47:56.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.383+0000 7f5166a00640 1 -- 172.21.15.39:0/342475664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5148003a50 msgr2=0x7f5148003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:56.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.383+0000 7f5166a00640 1 --2- 172.21.15.39:0/342475664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5148003a50 0x7f5148003ec0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f5154009b50 tx=0x7f5148005830 comp rx=0 tx=0).stop 2024-09-21T07:47:56.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.383+0000 7f5166a00640 1 -- 172.21.15.39:0/342475664 shutdown_connections 2024-09-21T07:47:56.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.383+0000 7f5166a00640 1 --2- 172.21.15.39:0/342475664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5148003a50 0x7f5148003ec0 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:56.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.383+0000 7f5166a00640 1 -- 172.21.15.39:0/342475664 >> 172.21.15.39:0/342475664 conn(0x7f516006cfc0 msgr2=0x7f516006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:56.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.383+0000 7f5166a00640 1 -- 172.21.15.39:0/342475664 shutdown_connections 2024-09-21T07:47:56.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.383+0000 7f5166a00640 1 -- 172.21.15.39:0/342475664 wait complete. 2024-09-21T07:47:56.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.385+0000 7f5166a00640 1 Processor -- start 2024-09-21T07:47:56.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.385+0000 7f5166a00640 1 -- start start 2024-09-21T07:47:56.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.387+0000 7f5166a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5148003a50 0x7f51601f7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:56.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.387+0000 7f5166a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51600b2610 0x7f5160140a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:56.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.387+0000 7f5166a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f516015b370 0x7f516015b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:56.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.387+0000 7f5166a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f516015d900 con 0x7f516015b370 2024-09-21T07:47:56.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.387+0000 7f5166a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f516015d780 con 0x7f51600b2610 2024-09-21T07:47:56.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.387+0000 7f5166a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f516015da80 con 0x7f5148003a50 2024-09-21T07:47:56.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.387+0000 7f5166000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f516015b370 0x7f516015b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:56.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.387+0000 7f5166000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f516015b370 0x7f516015b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53602/0 (socket says 172.21.15.39:53602) 2024-09-21T07:47:56.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.387+0000 7f5166000640 1 -- 172.21.15.39:0/4095350122 learned_addr learned my addr 172.21.15.39:0/4095350122 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:56.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.387+0000 7f5165600640 1 --2- 172.21.15.39:0/4095350122 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5148003a50 0x7f51601f7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:56.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.387+0000 7f5164c00640 1 --2- 172.21.15.39:0/4095350122 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51600b2610 0x7f5160140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:56.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.387+0000 7f5166000640 1 -- 172.21.15.39:0/4095350122 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5148003a50 msgr2=0x7f51601f7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:56.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.387+0000 7f5166000640 1 --2- 172.21.15.39:0/4095350122 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5148003a50 0x7f51601f7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:56.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.387+0000 7f5166000640 1 -- 172.21.15.39:0/4095350122 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51600b2610 msgr2=0x7f5160140a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:56.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.387+0000 7f5166000640 1 --2- 172.21.15.39:0/4095350122 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51600b2610 0x7f5160140a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:56.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.387+0000 7f5166000640 1 -- 172.21.15.39:0/4095350122 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f516015bcc0 con 0x7f516015b370 2024-09-21T07:47:56.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.388+0000 7f5166000640 1 --2- 172.21.15.39:0/4095350122 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f516015b370 0x7f516015b780 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f5160072480 tx=0x7f51580019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:56.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.388+0000 7f515d600640 1 -- 172.21.15.39:0/4095350122 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5158015020 con 0x7f516015b370 2024-09-21T07:47:56.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.388+0000 7f515d600640 1 -- 172.21.15.39:0/4095350122 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5158011960 con 0x7f516015b370 2024-09-21T07:47:56.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.388+0000 7f5166a00640 1 -- 172.21.15.39:0/4095350122 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51600b2bb0 con 0x7f516015b370 2024-09-21T07:47:56.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.388+0000 7f515d600640 1 -- 172.21.15.39:0/4095350122 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5158011d70 con 0x7f516015b370 2024-09-21T07:47:56.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.388+0000 7f5166a00640 1 -- 172.21.15.39:0/4095350122 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51600b1b20 con 0x7f516015b370 2024-09-21T07:47:56.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.389+0000 7f5166a00640 1 -- 172.21.15.39:0/4095350122 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51601414b0 con 0x7f516015b370 2024-09-21T07:47:56.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.391+0000 7f515d600640 1 -- 172.21.15.39:0/4095350122 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5158018020 con 0x7f516015b370 2024-09-21T07:47:56.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.392+0000 7f515d600640 1 --2- 172.21.15.39:0/4095350122 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5140078b90 0x7f514007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:56.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.392+0000 7f515d600640 1 -- 172.21.15.39:0/4095350122 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(206..206 src has 1..206) ==== 6960+0+0 (secure 0 0 0) 0x7f515809c060 con 0x7f516015b370 2024-09-21T07:47:56.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.392+0000 7f5165600640 1 --2- 172.21.15.39:0/4095350122 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5140078b90 0x7f514007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:56.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.393+0000 7f5165600640 1 --2- 172.21.15.39:0/4095350122 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5140078b90 0x7f514007b0a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f5154035000 tx=0x7f51540046c0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:56.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.395+0000 7f515d600640 1 -- 172.21.15.39:0/4095350122 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f51601414b0 con 0x7f516015b370 2024-09-21T07:47:56.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:56.629+0000 7f5166a00640 1 -- 172.21.15.39:0/4095350122 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f51600cf920 con 0x7f516015b370 2024-09-21T07:47:57.125 INFO:tasks.ceph.mds.c.smithi039.stderr:2024-09-21T07:47:57.124+0000 7f44f1a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:47:57.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.211+0000 7f515d600640 1 -- 172.21.15.39:0/4095350122 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v213) ==== 118+0+0 (secure 0 0 0) 0x7f515801f020 con 0x7f516015b370 2024-09-21T07:47:57.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.213+0000 7f5166a00640 1 -- 172.21.15.39:0/4095350122 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5140078b90 msgr2=0x7f514007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:57.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.213+0000 7f5166a00640 1 --2- 172.21.15.39:0/4095350122 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5140078b90 0x7f514007b0a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f5154035000 tx=0x7f51540046c0 comp rx=0 tx=0).stop 2024-09-21T07:47:57.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.213+0000 7f5166a00640 1 -- 172.21.15.39:0/4095350122 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f516015b370 msgr2=0x7f516015b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:57.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.213+0000 7f5166a00640 1 --2- 172.21.15.39:0/4095350122 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f516015b370 0x7f516015b780 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f5160072480 tx=0x7f51580019b0 comp rx=0 tx=0).stop 2024-09-21T07:47:57.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.214+0000 7f5166a00640 1 -- 172.21.15.39:0/4095350122 shutdown_connections 2024-09-21T07:47:57.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.214+0000 7f5166a00640 1 --2- 172.21.15.39:0/4095350122 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5140078b90 0x7f514007b0a0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:57.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.214+0000 7f5166a00640 1 --2- 172.21.15.39:0/4095350122 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f516015b370 0x7f516015b780 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:57.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.214+0000 7f5166a00640 1 --2- 172.21.15.39:0/4095350122 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51600b2610 0x7f5160140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:57.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.214+0000 7f5166a00640 1 --2- 172.21.15.39:0/4095350122 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5148003a50 0x7f51601f7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:57.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.214+0000 7f5166a00640 1 -- 172.21.15.39:0/4095350122 >> 172.21.15.39:0/4095350122 conn(0x7f516006cfc0 msgr2=0x7f516015e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:57.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.214+0000 7f5166a00640 1 -- 172.21.15.39:0/4095350122 shutdown_connections 2024-09-21T07:47:57.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.214+0000 7f5166a00640 1 -- 172.21.15.39:0/4095350122 wait complete. 2024-09-21T07:47:57.241 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T07:47:57.241 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T07:47:57.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.471+0000 7f66d2600640 1 -- 172.21.15.39:0/976225043 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f66cc0d5060 msgr2=0x7f66cc0b96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:57.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.471+0000 7f66d2600640 1 --2- 172.21.15.39:0/976225043 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f66cc0d5060 0x7f66cc0b96a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f66bc0098e0 tx=0x7f66bc02a5d0 comp rx=0 tx=0).stop 2024-09-21T07:47:57.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.471+0000 7f66d2600640 1 -- 172.21.15.39:0/976225043 shutdown_connections 2024-09-21T07:47:57.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.471+0000 7f66d2600640 1 --2- 172.21.15.39:0/976225043 >> v2:172.21.15.148:3300/0 conn(0x7f66cc0d0e60 0x7f66cc0d1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:57.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.471+0000 7f66d2600640 1 --2- 172.21.15.39:0/976225043 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f66cc0d5060 0x7f66cc0b96a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:57.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.471+0000 7f66d2600640 1 -- 172.21.15.39:0/976225043 >> 172.21.15.39:0/976225043 conn(0x7f66cc09d7d0 msgr2=0x7f66cc0d4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:57.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.471+0000 7f66d2600640 1 -- 172.21.15.39:0/976225043 shutdown_connections 2024-09-21T07:47:57.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.471+0000 7f66d2600640 1 -- 172.21.15.39:0/976225043 wait complete. 2024-09-21T07:47:57.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.473+0000 7f66d2600640 1 Processor -- start 2024-09-21T07:47:57.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.473+0000 7f66d2600640 1 -- start start 2024-09-21T07:47:57.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.475+0000 7f66d2600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66cc0b59e0 0x7f66cc0cca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:57.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.475+0000 7f66d2600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f66cc0d0e60 0x7f66cc0ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:57.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.475+0000 7f66d2600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66cc0d5060 0x7f66cc0a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:57.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.475+0000 7f66d2600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f66cc0d2c10 con 0x7f66cc0d5060 2024-09-21T07:47:57.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.475+0000 7f66d2600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f66cc0d2a90 con 0x7f66cc0b59e0 2024-09-21T07:47:57.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.475+0000 7f66d2600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f66cc0d2d90 con 0x7f66cc0d0e60 2024-09-21T07:47:57.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.475+0000 7f66d1c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66cc0d5060 0x7f66cc0a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:57.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.475+0000 7f66d1c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66cc0d5060 0x7f66cc0a1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53616/0 (socket says 172.21.15.39:53616) 2024-09-21T07:47:57.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.475+0000 7f66d1c00640 1 -- 172.21.15.39:0/1373669104 learned_addr learned my addr 172.21.15.39:0/1373669104 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:57.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.475+0000 7f66d1200640 1 --2- 172.21.15.39:0/1373669104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66cc0b59e0 0x7f66cc0cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:57.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.475+0000 7f66c3e00640 1 --2- 172.21.15.39:0/1373669104 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f66cc0d0e60 0x7f66cc0ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:57.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.475+0000 7f66d1c00640 1 -- 172.21.15.39:0/1373669104 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f66cc0d0e60 msgr2=0x7f66cc0ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:57.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.475+0000 7f66d1c00640 1 --2- 172.21.15.39:0/1373669104 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f66cc0d0e60 0x7f66cc0ccf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:57.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.475+0000 7f66d1c00640 1 -- 172.21.15.39:0/1373669104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66cc0b59e0 msgr2=0x7f66cc0cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:57.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.475+0000 7f66d1c00640 1 --2- 172.21.15.39:0/1373669104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66cc0b59e0 0x7f66cc0cca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:57.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.475+0000 7f66d1c00640 1 -- 172.21.15.39:0/1373669104 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66cc0caa60 con 0x7f66cc0d5060 2024-09-21T07:47:57.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.476+0000 7f66d1c00640 1 --2- 172.21.15.39:0/1373669104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66cc0d5060 0x7f66cc0a1780 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f66c400ea40 tx=0x7f66c400ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:57.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.476+0000 7f66c9600640 1 -- 172.21.15.39:0/1373669104 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c400ce40 con 0x7f66cc0d5060 2024-09-21T07:47:57.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.476+0000 7f66c9600640 1 -- 172.21.15.39:0/1373669104 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66c4004c10 con 0x7f66cc0d5060 2024-09-21T07:47:57.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.476+0000 7f66c9600640 1 -- 172.21.15.39:0/1373669104 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c4005020 con 0x7f66cc0d5060 2024-09-21T07:47:57.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.476+0000 7f66d2600640 1 -- 172.21.15.39:0/1373669104 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66cc151bf0 con 0x7f66cc0d5060 2024-09-21T07:47:57.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.476+0000 7f66d2600640 1 -- 172.21.15.39:0/1373669104 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66cc1521b0 con 0x7f66cc0d5060 2024-09-21T07:47:57.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.477+0000 7f66d2600640 1 -- 172.21.15.39:0/1373669104 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66cc0ca7d0 con 0x7f66cc0d5060 2024-09-21T07:47:57.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.478+0000 7f66c9600640 1 -- 172.21.15.39:0/1373669104 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f66c40051c0 con 0x7f66cc0d5060 2024-09-21T07:47:57.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.479+0000 7f66c9600640 1 --2- 172.21.15.39:0/1373669104 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f66a8078b90 0x7f66a807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:57.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.480+0000 7f66c9600640 1 -- 172.21.15.39:0/1373669104 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(206..206 src has 1..206) ==== 6960+0+0 (secure 0 0 0) 0x7f66c409beb0 con 0x7f66cc0d5060 2024-09-21T07:47:57.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.480+0000 7f66d1200640 1 --2- 172.21.15.39:0/1373669104 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f66a8078b90 0x7f66a807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:57.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.481+0000 7f66d1200640 1 --2- 172.21.15.39:0/1373669104 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f66a8078b90 0x7f66a807b0a0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f66bc034000 tx=0x7f66bc004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:57.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.484+0000 7f66c9600640 1 -- 172.21.15.39:0/1373669104 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f66c4063790 con 0x7f66cc0d5060 2024-09-21T07:47:57.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:57.721+0000 7f66d2600640 1 -- 172.21.15.39:0/1373669104 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f66cc0d3240 con 0x7f66cc0d5060 2024-09-21T07:47:58.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.218+0000 7f66c9600640 1 -- 172.21.15.39:0/1373669104 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v214) ==== 122+0+0 (secure 0 0 0) 0x7f66c40681c0 con 0x7f66cc0d5060 2024-09-21T07:47:58.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.220+0000 7f66d2600640 1 -- 172.21.15.39:0/1373669104 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f66a8078b90 msgr2=0x7f66a807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:58.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.220+0000 7f66d2600640 1 --2- 172.21.15.39:0/1373669104 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f66a8078b90 0x7f66a807b0a0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f66bc034000 tx=0x7f66bc004730 comp rx=0 tx=0).stop 2024-09-21T07:47:58.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.220+0000 7f66d2600640 1 -- 172.21.15.39:0/1373669104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66cc0d5060 msgr2=0x7f66cc0a1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:58.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.220+0000 7f66d2600640 1 --2- 172.21.15.39:0/1373669104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66cc0d5060 0x7f66cc0a1780 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f66c400ea40 tx=0x7f66c400ef00 comp rx=0 tx=0).stop 2024-09-21T07:47:58.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.220+0000 7f66d2600640 1 -- 172.21.15.39:0/1373669104 shutdown_connections 2024-09-21T07:47:58.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.220+0000 7f66d2600640 1 --2- 172.21.15.39:0/1373669104 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f66a8078b90 0x7f66a807b0a0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:58.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.220+0000 7f66d2600640 1 --2- 172.21.15.39:0/1373669104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66cc0d5060 0x7f66cc0a1780 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:58.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.220+0000 7f66d2600640 1 --2- 172.21.15.39:0/1373669104 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f66cc0d0e60 0x7f66cc0ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:58.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.220+0000 7f66d2600640 1 --2- 172.21.15.39:0/1373669104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66cc0b59e0 0x7f66cc0cca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:58.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.220+0000 7f66d2600640 1 -- 172.21.15.39:0/1373669104 >> 172.21.15.39:0/1373669104 conn(0x7f66cc09d7d0 msgr2=0x7f66cc0b1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:58.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.221+0000 7f66d2600640 1 -- 172.21.15.39:0/1373669104 shutdown_connections 2024-09-21T07:47:58.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.221+0000 7f66d2600640 1 -- 172.21.15.39:0/1373669104 wait complete. 2024-09-21T07:47:58.250 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T07:47:58.250 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:47:58.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.474+0000 7efff7400640 1 -- 172.21.15.39:0/1552792285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efff00cb8a0 msgr2=0x7efff0156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:58.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.474+0000 7efff7400640 1 --2- 172.21.15.39:0/1552792285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efff00cb8a0 0x7efff0156bb0 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7effdc009830 tx=0x7effdc02a520 comp rx=0 tx=0).stop 2024-09-21T07:47:58.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.474+0000 7efff7400640 1 -- 172.21.15.39:0/1552792285 shutdown_connections 2024-09-21T07:47:58.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.474+0000 7efff7400640 1 --2- 172.21.15.39:0/1552792285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efff00cb8a0 0x7efff0156bb0 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:58.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.474+0000 7efff7400640 1 -- 172.21.15.39:0/1552792285 >> 172.21.15.39:0/1552792285 conn(0x7efff00b2460 msgr2=0x7efff006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:58.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.474+0000 7efff7400640 1 -- 172.21.15.39:0/1552792285 shutdown_connections 2024-09-21T07:47:58.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.474+0000 7efff7400640 1 -- 172.21.15.39:0/1552792285 wait complete. 2024-09-21T07:47:58.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.476+0000 7efff7400640 1 Processor -- start 2024-09-21T07:47:58.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.476+0000 7efff7400640 1 -- start start 2024-09-21T07:47:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.478+0000 7efff7400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efff00b30a0 0x7efff00c6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.478+0000 7efff7400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efff00cb8a0 0x7efff00d06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.478+0000 7efff7400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efff00ccc60 0x7efff00ce500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:58.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.478+0000 7efff7400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efff00af520 con 0x7efff00b30a0 2024-09-21T07:47:58.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.478+0000 7efff7400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efff00af3a0 con 0x7efff00cb8a0 2024-09-21T07:47:58.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.478+0000 7efff7400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7efff00af6a0 con 0x7efff00ccc60 2024-09-21T07:47:58.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.478+0000 7efff4a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efff00b30a0 0x7efff00c6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:58.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.478+0000 7efff4a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efff00b30a0 0x7efff00c6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53632/0 (socket says 172.21.15.39:53632) 2024-09-21T07:47:58.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.478+0000 7efff4a00640 1 -- 172.21.15.39:0/2739662858 learned_addr learned my addr 172.21.15.39:0/2739662858 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:58.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.478+0000 7effefe00640 1 --2- 172.21.15.39:0/2739662858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efff00cb8a0 0x7efff00d06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:58.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.478+0000 7efff5400640 1 --2- 172.21.15.39:0/2739662858 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efff00ccc60 0x7efff00ce500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:58.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.478+0000 7efff4a00640 1 -- 172.21.15.39:0/2739662858 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efff00ccc60 msgr2=0x7efff00ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:58.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.478+0000 7efff4a00640 1 --2- 172.21.15.39:0/2739662858 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efff00ccc60 0x7efff00ce500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:58.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.478+0000 7efff4a00640 1 -- 172.21.15.39:0/2739662858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efff00cb8a0 msgr2=0x7efff00d06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:58.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.478+0000 7efff4a00640 1 --2- 172.21.15.39:0/2739662858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efff00cb8a0 0x7efff00d06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:58.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.478+0000 7efff4a00640 1 -- 172.21.15.39:0/2739662858 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efff01f5670 con 0x7efff00b30a0 2024-09-21T07:47:58.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.479+0000 7efff4a00640 1 --2- 172.21.15.39:0/2739662858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efff00b30a0 0x7efff00c6230 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7effe400c400 tx=0x7effe400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:58.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.479+0000 7effecc00640 1 -- 172.21.15.39:0/2739662858 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe4011790 con 0x7efff00b30a0 2024-09-21T07:47:58.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.479+0000 7effecc00640 1 -- 172.21.15.39:0/2739662858 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7effe4017070 con 0x7efff00b30a0 2024-09-21T07:47:58.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.479+0000 7efff7400640 1 -- 172.21.15.39:0/2739662858 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efff00c7550 con 0x7efff00b30a0 2024-09-21T07:47:58.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.479+0000 7effecc00640 1 -- 172.21.15.39:0/2739662858 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe4011ce0 con 0x7efff00b30a0 2024-09-21T07:47:58.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.479+0000 7efff7400640 1 -- 172.21.15.39:0/2739662858 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efff00c5a30 con 0x7efff00b30a0 2024-09-21T07:47:58.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.480+0000 7efff7400640 1 -- 172.21.15.39:0/2739662858 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efff0156bb0 con 0x7efff00b30a0 2024-09-21T07:47:58.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.481+0000 7effecc00640 1 -- 172.21.15.39:0/2739662858 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7effe401c460 con 0x7efff00b30a0 2024-09-21T07:47:58.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.482+0000 7effecc00640 1 --2- 172.21.15.39:0/2739662858 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7effd0078b90 0x7effd007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:58.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.483+0000 7effecc00640 1 -- 172.21.15.39:0/2739662858 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(206..206 src has 1..206) ==== 6960+0+0 (secure 0 0 0) 0x7effe409b070 con 0x7efff00b30a0 2024-09-21T07:47:58.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.483+0000 7effefe00640 1 --2- 172.21.15.39:0/2739662858 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7effd0078b90 0x7effd007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:58.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.484+0000 7effefe00640 1 --2- 172.21.15.39:0/2739662858 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7effd0078b90 0x7effd007b0a0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7effdc034000 tx=0x7effdc004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:58.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.487+0000 7effecc00640 1 -- 172.21.15.39:0/2739662858 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7effe4062fe0 con 0x7efff00b30a0 2024-09-21T07:47:58.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.722+0000 7efff7400640 1 -- 172.21.15.39:0/2739662858 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efff00c7280 con 0x7efff00b30a0 2024-09-21T07:47:58.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.723+0000 7effecc00640 1 -- 172.21.15.39:0/2739662858 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 214 v214) ==== 95+0+5239 (secure 0 0 0) 0x7effe40628e0 con 0x7efff00b30a0 2024-09-21T07:47:58.725 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:47:58.725 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":214,"btime":"2024-09-21T07:47:58:212546+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7988,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1912631346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1912631346},{"type":"v1","addr":"172.21.15.148:6837","nonce":1912631346}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":209},{"gid":9043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2220628944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2220628944},{"type":"v1","addr":"172.21.15.148:6836","nonce":2220628944}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":208},{"gid":9049,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/1947479913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1947479913},{"type":"v1","addr":"172.21.15.39:6833","nonce":1947479913}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":214,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:47:54.691835+0000","modified":"2024-09-21T07:47:57.723294+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9852},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9852":{"gid":9852,"name":"c","rank":0,"incarnation":210,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/2356847598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2356847598},{"type":"v1","addr":"172.21.15.39:6835","nonce":2356847598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9852,"qdb_cluster":[9852]},"id":11}]} 2024-09-21T07:47:58.725 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 214 2024-09-21T07:47:58.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.725+0000 7efff7400640 1 -- 172.21.15.39:0/2739662858 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7effd0078b90 msgr2=0x7effd007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:58.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.725+0000 7efff7400640 1 --2- 172.21.15.39:0/2739662858 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7effd0078b90 0x7effd007b0a0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7effdc034000 tx=0x7effdc004730 comp rx=0 tx=0).stop 2024-09-21T07:47:58.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.725+0000 7efff7400640 1 -- 172.21.15.39:0/2739662858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efff00b30a0 msgr2=0x7efff00c6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:58.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.725+0000 7efff7400640 1 --2- 172.21.15.39:0/2739662858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efff00b30a0 0x7efff00c6230 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7effe400c400 tx=0x7effe400c8c0 comp rx=0 tx=0).stop 2024-09-21T07:47:58.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.725+0000 7efff7400640 1 -- 172.21.15.39:0/2739662858 shutdown_connections 2024-09-21T07:47:58.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.725+0000 7efff7400640 1 --2- 172.21.15.39:0/2739662858 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7effd0078b90 0x7effd007b0a0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:58.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.725+0000 7efff7400640 1 --2- 172.21.15.39:0/2739662858 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efff00ccc60 0x7efff00ce500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:58.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.725+0000 7efff7400640 1 --2- 172.21.15.39:0/2739662858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efff00cb8a0 0x7efff00d06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:58.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.725+0000 7efff7400640 1 --2- 172.21.15.39:0/2739662858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efff00b30a0 0x7efff00c6230 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:58.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.725+0000 7efff7400640 1 -- 172.21.15.39:0/2739662858 >> 172.21.15.39:0/2739662858 conn(0x7efff00b2460 msgr2=0x7efff015f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:58.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.725+0000 7efff7400640 1 -- 172.21.15.39:0/2739662858 shutdown_connections 2024-09-21T07:47:58.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.725+0000 7efff7400640 1 -- 172.21.15.39:0/2739662858 wait complete. 2024-09-21T07:47:58.749 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:47:58.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.973+0000 7f7c75800640 1 -- 172.21.15.39:0/2574530185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c700d2740 msgr2=0x7f7c700d0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:58.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.973+0000 7f7c75800640 1 --2- 172.21.15.39:0/2574530185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c700d2740 0x7f7c700d0470 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f7c700ccc40 tx=0x7f7c6802a9e0 comp rx=0 tx=0).stop 2024-09-21T07:47:58.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.973+0000 7f7c75800640 1 -- 172.21.15.39:0/2574530185 shutdown_connections 2024-09-21T07:47:58.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.973+0000 7f7c75800640 1 --2- 172.21.15.39:0/2574530185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c700d2740 0x7f7c700d0470 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:58.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.973+0000 7f7c75800640 1 --2- 172.21.15.39:0/2574530185 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c700ce690 0x7f7c700b2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:58.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.973+0000 7f7c75800640 1 -- 172.21.15.39:0/2574530185 >> 172.21.15.39:0/2574530185 conn(0x7f7c7009b5f0 msgr2=0x7f7c7008c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:58.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.974+0000 7f7c75800640 1 -- 172.21.15.39:0/2574530185 shutdown_connections 2024-09-21T07:47:58.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.974+0000 7f7c75800640 1 -- 172.21.15.39:0/2574530185 wait complete. 2024-09-21T07:47:58.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.976+0000 7f7c75800640 1 Processor -- start 2024-09-21T07:47:58.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.976+0000 7f7c75800640 1 -- start start 2024-09-21T07:47:58.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.977+0000 7f7c75800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c700ce690 0x7f7c70150670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:58.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.978+0000 7f7c75800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c700d0060 0x7f7c700affd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:58.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.978+0000 7f7c75800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c700d2740 0x7f7c700b0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:58.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.978+0000 7f7c75800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7c700cd3c0 con 0x7f7c700ce690 2024-09-21T07:47:58.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.978+0000 7f7c75800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7c700cd240 con 0x7f7c700d0060 2024-09-21T07:47:58.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.978+0000 7f7c75800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f7c700cd540 con 0x7f7c700d2740 2024-09-21T07:47:58.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.978+0000 7f7c6fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c700ce690 0x7f7c70150670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:58.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.978+0000 7f7c6fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c700ce690 0x7f7c70150670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53660/0 (socket says 172.21.15.39:53660) 2024-09-21T07:47:58.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.978+0000 7f7c6fe00640 1 -- 172.21.15.39:0/1869383736 learned_addr learned my addr 172.21.15.39:0/1869383736 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:58.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.978+0000 7f7c74e00640 1 --2- 172.21.15.39:0/1869383736 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c700d2740 0x7f7c700b0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:58.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.978+0000 7f7c6f400640 1 --2- 172.21.15.39:0/1869383736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c700d0060 0x7f7c700affd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:58.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.978+0000 7f7c6fe00640 1 -- 172.21.15.39:0/1869383736 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c700d2740 msgr2=0x7f7c700b0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:58.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.978+0000 7f7c6fe00640 1 --2- 172.21.15.39:0/1869383736 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c700d2740 0x7f7c700b0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:58.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.978+0000 7f7c6fe00640 1 -- 172.21.15.39:0/1869383736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c700d0060 msgr2=0x7f7c700affd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:58.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.978+0000 7f7c6fe00640 1 --2- 172.21.15.39:0/1869383736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c700d0060 0x7f7c700affd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:58.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.978+0000 7f7c6fe00640 1 -- 172.21.15.39:0/1869383736 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c70156980 con 0x7f7c700ce690 2024-09-21T07:47:58.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.979+0000 7f7c6fe00640 1 --2- 172.21.15.39:0/1869383736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c700ce690 0x7f7c70150670 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f7c6000c9a0 tx=0x7f7c6000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:58.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.979+0000 7f7c67e00640 1 -- 172.21.15.39:0/1869383736 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c60007bb0 con 0x7f7c700ce690 2024-09-21T07:47:58.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.979+0000 7f7c67e00640 1 -- 172.21.15.39:0/1869383736 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c60007e40 con 0x7f7c700ce690 2024-09-21T07:47:58.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.979+0000 7f7c67e00640 1 -- 172.21.15.39:0/1869383736 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c60005290 con 0x7f7c700ce690 2024-09-21T07:47:58.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.979+0000 7f7c75800640 1 -- 172.21.15.39:0/1869383736 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c70157960 con 0x7f7c700ce690 2024-09-21T07:47:58.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.979+0000 7f7c75800640 1 -- 172.21.15.39:0/1869383736 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c701f8d80 con 0x7f7c700ce690 2024-09-21T07:47:58.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.980+0000 7f7c75800640 1 -- 172.21.15.39:0/1869383736 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c700cda60 con 0x7f7c700ce690 2024-09-21T07:47:58.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.981+0000 7f7c67e00640 1 -- 172.21.15.39:0/1869383736 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7c60005430 con 0x7f7c700ce690 2024-09-21T07:47:58.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.983+0000 7f7c67e00640 1 --2- 172.21.15.39:0/1869383736 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7c50078bc0 0x7f7c5007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:58.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.983+0000 7f7c67e00640 1 -- 172.21.15.39:0/1869383736 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(206..206 src has 1..206) ==== 6960+0+0 (secure 0 0 0) 0x7f7c6009b6d0 con 0x7f7c700ce690 2024-09-21T07:47:58.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.983+0000 7f7c6f400640 1 --2- 172.21.15.39:0/1869383736 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7c50078bc0 0x7f7c5007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:58.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.984+0000 7f7c6f400640 1 --2- 172.21.15.39:0/1869383736 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7c50078bc0 0x7f7c5007b0d0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f7c7015fd10 tx=0x7f7c5c0090e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:58.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:58.988+0000 7f7c67e00640 1 -- 172.21.15.39:0/1869383736 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7c60014070 con 0x7f7c700ce690 2024-09-21T07:47:59.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.186+0000 7f7c75800640 1 -- 172.21.15.39:0/1869383736 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7c70150bb0 con 0x7f7c700ce690 2024-09-21T07:47:59.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.187+0000 7f7c67e00640 1 -- 172.21.15.39:0/1869383736 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v206) ==== 74+0+22397 (secure 0 0 0) 0x7f7c60062f60 con 0x7f7c700ce690 2024-09-21T07:47:59.189 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:47:59.190 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":206,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:47:56.161453+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:47:52.623339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"205","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-09-21T07:47:53.638900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"206","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]},{"pool":20,"snaps":[{"begin":80,"length":23},{"begin":181,"length":23}]},{"pool":22,"snaps":[{"begin":100,"length":1},{"begin":199,"length":1}]},{"pool":24,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:47:59.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.189+0000 7f7c75800640 1 -- 172.21.15.39:0/1869383736 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7c50078bc0 msgr2=0x7f7c5007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:59.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.189+0000 7f7c75800640 1 --2- 172.21.15.39:0/1869383736 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7c50078bc0 0x7f7c5007b0d0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f7c7015fd10 tx=0x7f7c5c0090e0 comp rx=0 tx=0).stop 2024-09-21T07:47:59.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.189+0000 7f7c75800640 1 -- 172.21.15.39:0/1869383736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c700ce690 msgr2=0x7f7c70150670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:59.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.189+0000 7f7c75800640 1 --2- 172.21.15.39:0/1869383736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c700ce690 0x7f7c70150670 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f7c6000c9a0 tx=0x7f7c6000ce60 comp rx=0 tx=0).stop 2024-09-21T07:47:59.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.190+0000 7f7c75800640 1 -- 172.21.15.39:0/1869383736 shutdown_connections 2024-09-21T07:47:59.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.190+0000 7f7c75800640 1 --2- 172.21.15.39:0/1869383736 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7c50078bc0 0x7f7c5007b0d0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:59.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.190+0000 7f7c75800640 1 --2- 172.21.15.39:0/1869383736 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7c700d2740 0x7f7c700b0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:59.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.190+0000 7f7c75800640 1 --2- 172.21.15.39:0/1869383736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c700d0060 0x7f7c700affd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:59.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.190+0000 7f7c75800640 1 --2- 172.21.15.39:0/1869383736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c700ce690 0x7f7c70150670 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:59.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.190+0000 7f7c75800640 1 -- 172.21.15.39:0/1869383736 >> 172.21.15.39:0/1869383736 conn(0x7f7c7009b5f0 msgr2=0x7f7c700b44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:59.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.190+0000 7f7c75800640 1 -- 172.21.15.39:0/1869383736 shutdown_connections 2024-09-21T07:47:59.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.190+0000 7f7c75800640 1 -- 172.21.15.39:0/1869383736 wait complete. 2024-09-21T07:47:59.218 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T07:47:59.218 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:47:59.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.444+0000 7f1759000640 1 -- 172.21.15.39:0/2675936194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17540b3d80 msgr2=0x7f17540ce2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:59.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.444+0000 7f1759000640 1 --2- 172.21.15.39:0/2675936194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17540b3d80 0x7f17540ce2b0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f17540726d0 tx=0x7f173c02a5f0 comp rx=0 tx=0).stop 2024-09-21T07:47:59.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.444+0000 7f1759000640 1 -- 172.21.15.39:0/2675936194 shutdown_connections 2024-09-21T07:47:59.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.444+0000 7f1759000640 1 --2- 172.21.15.39:0/2675936194 >> v2:172.21.15.148:3300/0 conn(0x7f175406ef90 0x7f175406f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:59.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.444+0000 7f1759000640 1 --2- 172.21.15.39:0/2675936194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17540b3d80 0x7f17540ce2b0 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:59.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.444+0000 7f1759000640 1 -- 172.21.15.39:0/2675936194 >> 172.21.15.39:0/2675936194 conn(0x7f17540b2460 msgr2=0x7f175406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:59.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.444+0000 7f1759000640 1 -- 172.21.15.39:0/2675936194 shutdown_connections 2024-09-21T07:47:59.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.444+0000 7f1759000640 1 -- 172.21.15.39:0/2675936194 wait complete. 2024-09-21T07:47:59.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.447+0000 7f1759000640 1 Processor -- start 2024-09-21T07:47:59.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.447+0000 7f1759000640 1 -- start start 2024-09-21T07:47:59.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.448+0000 7f1759000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f175406ef90 0x7f175415ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:59.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.448+0000 7f1759000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f17540b4c40 0x7f17540c9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:59.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.448+0000 7f1759000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17541f53e0 0x7f17540b0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:59.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.448+0000 7f1759000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f175406fa60 con 0x7f17541f53e0 2024-09-21T07:47:59.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.448+0000 7f1759000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f175406f8e0 con 0x7f175406ef90 2024-09-21T07:47:59.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.448+0000 7f1759000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f175406fbe0 con 0x7f17540b4c40 2024-09-21T07:47:59.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.449+0000 7f1752800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17541f53e0 0x7f17540b0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:59.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.449+0000 7f1752800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17541f53e0 0x7f17540b0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53672/0 (socket says 172.21.15.39:53672) 2024-09-21T07:47:59.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.449+0000 7f1752800640 1 -- 172.21.15.39:0/4189817450 learned_addr learned my addr 172.21.15.39:0/4189817450 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:59.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.449+0000 7f174be00640 1 --2- 172.21.15.39:0/4189817450 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f17540b4c40 0x7f17540c9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:59.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.449+0000 7f1751e00640 1 --2- 172.21.15.39:0/4189817450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f175406ef90 0x7f175415ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:59.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.449+0000 7f1752800640 1 -- 172.21.15.39:0/4189817450 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f17540b4c40 msgr2=0x7f17540c9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:59.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.449+0000 7f1752800640 1 --2- 172.21.15.39:0/4189817450 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f17540b4c40 0x7f17540c9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:59.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.449+0000 7f1752800640 1 -- 172.21.15.39:0/4189817450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f175406ef90 msgr2=0x7f175415ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:59.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.449+0000 7f1752800640 1 --2- 172.21.15.39:0/4189817450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f175406ef90 0x7f175415ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:59.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.449+0000 7f1752800640 1 -- 172.21.15.39:0/4189817450 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f175415e2e0 con 0x7f17541f53e0 2024-09-21T07:47:59.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.449+0000 7f1751e00640 1 --2- 172.21.15.39:0/4189817450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f175406ef90 0x7f175415ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.450+0000 7f1752800640 1 --2- 172.21.15.39:0/4189817450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17541f53e0 0x7f17540b0790 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f174c00d9e0 tx=0x7f174c00dea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.450+0000 7f174a000640 1 -- 172.21.15.39:0/4189817450 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174c017070 con 0x7f17541f53e0 2024-09-21T07:47:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.450+0000 7f174a000640 1 -- 172.21.15.39:0/4189817450 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f174c00be30 con 0x7f17541f53e0 2024-09-21T07:47:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.450+0000 7f174a000640 1 -- 172.21.15.39:0/4189817450 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174c003bb0 con 0x7f17541f53e0 2024-09-21T07:47:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.450+0000 7f1759000640 1 -- 172.21.15.39:0/4189817450 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f175415d760 con 0x7f17541f53e0 2024-09-21T07:47:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.450+0000 7f1759000640 1 -- 172.21.15.39:0/4189817450 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f175415dd20 con 0x7f17541f53e0 2024-09-21T07:47:59.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.451+0000 7f1759000640 1 -- 172.21.15.39:0/4189817450 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f175406f3a0 con 0x7f17541f53e0 2024-09-21T07:47:59.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.452+0000 7f174a000640 1 -- 172.21.15.39:0/4189817450 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f174c010460 con 0x7f17541f53e0 2024-09-21T07:47:59.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.453+0000 7f174a000640 1 --2- 172.21.15.39:0/4189817450 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1730078b90 0x7f173007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:59.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.453+0000 7f174a000640 1 -- 172.21.15.39:0/4189817450 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(206..206 src has 1..206) ==== 6960+0+0 (secure 0 0 0) 0x7f174c09b6b0 con 0x7f17541f53e0 2024-09-21T07:47:59.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.454+0000 7f1751e00640 1 --2- 172.21.15.39:0/4189817450 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1730078b90 0x7f173007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:59.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.455+0000 7f1751e00640 1 --2- 172.21.15.39:0/4189817450 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1730078b90 0x7f173007b0a0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f1744007fd0 tx=0x7f1744009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:59.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.458+0000 7f174a000640 1 -- 172.21.15.39:0/4189817450 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f174c062f40 con 0x7f17541f53e0 2024-09-21T07:47:59.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.652+0000 7f1759000640 1 -- 172.21.15.39:0/4189817450 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f175415d4f0 con 0x7f1730078b90 2024-09-21T07:47:59.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.661+0000 7f174a000640 1 -- 172.21.15.39:0/4189817450 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+495244 (secure 0 0 0) 0x7f175415d4f0 con 0x7f1730078b90 2024-09-21T07:47:59.670 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:47:59.673 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:47:59.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.667+0000 7f1759000640 1 -- 172.21.15.39:0/4189817450 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1730078b90 msgr2=0x7f173007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:59.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.667+0000 7f1759000640 1 --2- 172.21.15.39:0/4189817450 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1730078b90 0x7f173007b0a0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f1744007fd0 tx=0x7f1744009040 comp rx=0 tx=0).stop 2024-09-21T07:47:59.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.667+0000 7f1759000640 1 -- 172.21.15.39:0/4189817450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17541f53e0 msgr2=0x7f17540b0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:59.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.667+0000 7f1759000640 1 --2- 172.21.15.39:0/4189817450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17541f53e0 0x7f17540b0790 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f174c00d9e0 tx=0x7f174c00dea0 comp rx=0 tx=0).stop 2024-09-21T07:47:59.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.667+0000 7f1759000640 1 -- 172.21.15.39:0/4189817450 shutdown_connections 2024-09-21T07:47:59.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.667+0000 7f1759000640 1 --2- 172.21.15.39:0/4189817450 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1730078b90 0x7f173007b0a0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:59.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.667+0000 7f1759000640 1 --2- 172.21.15.39:0/4189817450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17541f53e0 0x7f17540b0790 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:59.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.667+0000 7f1759000640 1 --2- 172.21.15.39:0/4189817450 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f17540b4c40 0x7f17540c9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:59.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.667+0000 7f1759000640 1 --2- 172.21.15.39:0/4189817450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f175406ef90 0x7f175415ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:59.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.667+0000 7f1759000640 1 -- 172.21.15.39:0/4189817450 >> 172.21.15.39:0/4189817450 conn(0x7f17540b2460 msgr2=0x7f175415b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:59.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.667+0000 7f1759000640 1 -- 172.21.15.39:0/4189817450 shutdown_connections 2024-09-21T07:47:59.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.667+0000 7f1759000640 1 -- 172.21.15.39:0/4189817450 wait complete. 2024-09-21T07:47:59.691 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":649,"stamp":"2024-09-21T07:47:57.832631+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":405576,"kb_used_data":21388,"kb_used_omap":12,"kb_used_meta":383987,"kb_avail":754569144,"statfs":{"total":773094113280,"available":772678803456,"internally_reserved":0,"allocated":21901312,"data_stored":12185057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":393203248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.680873"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.170421+0000","last_change":"2024-09-21T07:47:55.201069+0000","last_active":"2024-09-21T07:47:56.170421+0000","last_peered":"2024-09-21T07:47:56.170421+0000","last_clean":"2024-09-21T07:47:56.170421+0000","last_became_active":"2024-09-21T07:47:55.200379+0000","last_became_peered":"2024-09-21T07:47:55.200379+0000","last_unstale":"2024-09-21T07:47:56.170421+0000","last_undegraded":"2024-09-21T07:47:56.170421+0000","last_fullsized":"2024-09-21T07:47:56.170421+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:05:25.718314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.170039+0000","last_change":"2024-09-21T07:47:55.186523+0000","last_active":"2024-09-21T07:47:56.170039+0000","last_peered":"2024-09-21T07:47:56.170039+0000","last_clean":"2024-09-21T07:47:56.170039+0000","last_became_active":"2024-09-21T07:47:55.186240+0000","last_became_peered":"2024-09-21T07:47:55.186240+0000","last_unstale":"2024-09-21T07:47:56.170039+0000","last_undegraded":"2024-09-21T07:47:56.170039+0000","last_fullsized":"2024-09-21T07:47:56.170039+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:12:04.414896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.187987+0000","last_change":"2024-09-21T07:47:55.182034+0000","last_active":"2024-09-21T07:47:55.187987+0000","last_peered":"2024-09-21T07:47:55.187987+0000","last_clean":"2024-09-21T07:47:55.187987+0000","last_became_active":"2024-09-21T07:47:55.181704+0000","last_became_peered":"2024-09-21T07:47:55.181704+0000","last_unstale":"2024-09-21T07:47:55.187987+0000","last_undegraded":"2024-09-21T07:47:55.187987+0000","last_fullsized":"2024-09-21T07:47:55.187987+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:08:23.323166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.207848+0000","last_change":"2024-09-21T07:47:55.202353+0000","last_active":"2024-09-21T07:47:55.207848+0000","last_peered":"2024-09-21T07:47:55.207848+0000","last_clean":"2024-09-21T07:47:55.207848+0000","last_became_active":"2024-09-21T07:47:55.202019+0000","last_became_peered":"2024-09-21T07:47:55.202019+0000","last_unstale":"2024-09-21T07:47:55.207848+0000","last_undegraded":"2024-09-21T07:47:55.207848+0000","last_fullsized":"2024-09-21T07:47:55.207848+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:03:18.039256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.180032+0000","last_change":"2024-09-21T07:47:55.174878+0000","last_active":"2024-09-21T07:47:55.180032+0000","last_peered":"2024-09-21T07:47:55.180032+0000","last_clean":"2024-09-21T07:47:55.180032+0000","last_became_active":"2024-09-21T07:47:55.174598+0000","last_became_peered":"2024-09-21T07:47:55.174598+0000","last_unstale":"2024-09-21T07:47:55.180032+0000","last_undegraded":"2024-09-21T07:47:55.180032+0000","last_fullsized":"2024-09-21T07:47:55.180032+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:36:56.278477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.206490+0000","last_change":"2024-09-21T07:47:55.204444+0000","last_active":"2024-09-21T07:47:55.206490+0000","last_peered":"2024-09-21T07:47:55.206490+0000","last_clean":"2024-09-21T07:47:55.206490+0000","last_became_active":"2024-09-21T07:47:55.204116+0000","last_became_peered":"2024-09-21T07:47:55.204116+0000","last_unstale":"2024-09-21T07:47:55.206490+0000","last_undegraded":"2024-09-21T07:47:55.206490+0000","last_fullsized":"2024-09-21T07:47:55.206490+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:00.304956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.171594+0000","last_change":"2024-09-21T07:47:55.191399+0000","last_active":"2024-09-21T07:47:56.171594+0000","last_peered":"2024-09-21T07:47:56.171594+0000","last_clean":"2024-09-21T07:47:56.171594+0000","last_became_active":"2024-09-21T07:47:55.190024+0000","last_became_peered":"2024-09-21T07:47:55.190024+0000","last_unstale":"2024-09-21T07:47:56.171594+0000","last_undegraded":"2024-09-21T07:47:56.171594+0000","last_fullsized":"2024-09-21T07:47:56.171594+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:57:10.017744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.202554+0000","last_change":"2024-09-21T07:47:55.198086+0000","last_active":"2024-09-21T07:47:55.202554+0000","last_peered":"2024-09-21T07:47:55.202554+0000","last_clean":"2024-09-21T07:47:55.202554+0000","last_became_active":"2024-09-21T07:47:55.196776+0000","last_became_peered":"2024-09-21T07:47:55.196776+0000","last_unstale":"2024-09-21T07:47:55.202554+0000","last_undegraded":"2024-09-21T07:47:55.202554+0000","last_fullsized":"2024-09-21T07:47:55.202554+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:42.509301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.170490+0000","last_change":"2024-09-21T07:47:55.190919+0000","last_active":"2024-09-21T07:47:56.170490+0000","last_peered":"2024-09-21T07:47:56.170490+0000","last_clean":"2024-09-21T07:47:56.170490+0000","last_became_active":"2024-09-21T07:47:55.190629+0000","last_became_peered":"2024-09-21T07:47:55.190629+0000","last_unstale":"2024-09-21T07:47:56.170490+0000","last_undegraded":"2024-09-21T07:47:56.170490+0000","last_fullsized":"2024-09-21T07:47:56.170490+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:43.304964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.195521+0000","last_change":"2024-09-21T07:47:55.192028+0000","last_active":"2024-09-21T07:47:55.195521+0000","last_peered":"2024-09-21T07:47:55.195521+0000","last_clean":"2024-09-21T07:47:55.195521+0000","last_became_active":"2024-09-21T07:47:55.191672+0000","last_became_peered":"2024-09-21T07:47:55.191672+0000","last_unstale":"2024-09-21T07:47:55.195521+0000","last_undegraded":"2024-09-21T07:47:55.195521+0000","last_fullsized":"2024-09-21T07:47:55.195521+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:16.143413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172466+0000","last_change":"2024-09-21T07:47:55.187516+0000","last_active":"2024-09-21T07:47:56.172466+0000","last_peered":"2024-09-21T07:47:56.172466+0000","last_clean":"2024-09-21T07:47:56.172466+0000","last_became_active":"2024-09-21T07:47:55.187198+0000","last_became_peered":"2024-09-21T07:47:55.187198+0000","last_unstale":"2024-09-21T07:47:56.172466+0000","last_undegraded":"2024-09-21T07:47:56.172466+0000","last_fullsized":"2024-09-21T07:47:56.172466+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:50.493155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.207863+0000","last_change":"2024-09-21T07:47:55.205897+0000","last_active":"2024-09-21T07:47:55.207863+0000","last_peered":"2024-09-21T07:47:55.207863+0000","last_clean":"2024-09-21T07:47:55.207863+0000","last_became_active":"2024-09-21T07:47:55.205278+0000","last_became_peered":"2024-09-21T07:47:55.205278+0000","last_unstale":"2024-09-21T07:47:55.207863+0000","last_undegraded":"2024-09-21T07:47:55.207863+0000","last_fullsized":"2024-09-21T07:47:55.207863+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:00:49.437527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.202680+0000","last_change":"2024-09-21T07:47:55.193427+0000","last_active":"2024-09-21T07:47:55.202680+0000","last_peered":"2024-09-21T07:47:55.202680+0000","last_clean":"2024-09-21T07:47:55.202680+0000","last_became_active":"2024-09-21T07:47:55.191920+0000","last_became_peered":"2024-09-21T07:47:55.191920+0000","last_unstale":"2024-09-21T07:47:55.202680+0000","last_undegraded":"2024-09-21T07:47:55.202680+0000","last_fullsized":"2024-09-21T07:47:55.202680+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:19.278614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.171559+0000","last_change":"2024-09-21T07:47:55.187111+0000","last_active":"2024-09-21T07:47:56.171559+0000","last_peered":"2024-09-21T07:47:56.171559+0000","last_clean":"2024-09-21T07:47:56.171559+0000","last_became_active":"2024-09-21T07:47:55.186688+0000","last_became_peered":"2024-09-21T07:47:55.186688+0000","last_unstale":"2024-09-21T07:47:56.171559+0000","last_undegraded":"2024-09-21T07:47:56.171559+0000","last_fullsized":"2024-09-21T07:47:56.171559+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:52.955721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.168585+0000","last_change":"2024-09-21T07:47:55.196437+0000","last_active":"2024-09-21T07:47:56.168585+0000","last_peered":"2024-09-21T07:47:56.168585+0000","last_clean":"2024-09-21T07:47:56.168585+0000","last_became_active":"2024-09-21T07:47:55.196127+0000","last_became_peered":"2024-09-21T07:47:55.196127+0000","last_unstale":"2024-09-21T07:47:56.168585+0000","last_undegraded":"2024-09-21T07:47:56.168585+0000","last_fullsized":"2024-09-21T07:47:56.168585+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:55:56.600312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.180838+0000","last_change":"2024-09-21T07:47:55.173391+0000","last_active":"2024-09-21T07:47:55.180838+0000","last_peered":"2024-09-21T07:47:55.180838+0000","last_clean":"2024-09-21T07:47:55.180838+0000","last_became_active":"2024-09-21T07:47:55.172004+0000","last_became_peered":"2024-09-21T07:47:55.172004+0000","last_unstale":"2024-09-21T07:47:55.180838+0000","last_undegraded":"2024-09-21T07:47:55.180838+0000","last_fullsized":"2024-09-21T07:47:55.180838+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:34:16.990305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.204759+0000","last_change":"2024-09-21T07:47:55.197091+0000","last_active":"2024-09-21T07:47:55.204759+0000","last_peered":"2024-09-21T07:47:55.204759+0000","last_clean":"2024-09-21T07:47:55.204759+0000","last_became_active":"2024-09-21T07:47:55.196737+0000","last_became_peered":"2024-09-21T07:47:55.196737+0000","last_unstale":"2024-09-21T07:47:55.204759+0000","last_undegraded":"2024-09-21T07:47:55.204759+0000","last_fullsized":"2024-09-21T07:47:55.204759+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:53.383651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.205907+0000","last_change":"2024-09-21T07:47:55.203810+0000","last_active":"2024-09-21T07:47:55.205907+0000","last_peered":"2024-09-21T07:47:55.205907+0000","last_clean":"2024-09-21T07:47:55.205907+0000","last_became_active":"2024-09-21T07:47:55.203482+0000","last_became_peered":"2024-09-21T07:47:55.203482+0000","last_unstale":"2024-09-21T07:47:55.205907+0000","last_undegraded":"2024-09-21T07:47:55.205907+0000","last_fullsized":"2024-09-21T07:47:55.205907+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:17.269487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.204475+0000","last_change":"2024-09-21T07:47:55.193855+0000","last_active":"2024-09-21T07:47:55.204475+0000","last_peered":"2024-09-21T07:47:55.204475+0000","last_clean":"2024-09-21T07:47:55.204475+0000","last_became_active":"2024-09-21T07:47:55.190548+0000","last_became_peered":"2024-09-21T07:47:55.190548+0000","last_unstale":"2024-09-21T07:47:55.204475+0000","last_undegraded":"2024-09-21T07:47:55.204475+0000","last_fullsized":"2024-09-21T07:47:55.204475+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:03.931741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.206110+0000","last_change":"2024-09-21T07:47:55.199657+0000","last_active":"2024-09-21T07:47:55.206110+0000","last_peered":"2024-09-21T07:47:55.206110+0000","last_clean":"2024-09-21T07:47:55.206110+0000","last_became_active":"2024-09-21T07:47:55.199360+0000","last_became_peered":"2024-09-21T07:47:55.199360+0000","last_unstale":"2024-09-21T07:47:55.206110+0000","last_undegraded":"2024-09-21T07:47:55.206110+0000","last_fullsized":"2024-09-21T07:47:55.206110+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:05.149895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.169245+0000","last_change":"2024-09-21T07:47:55.197437+0000","last_active":"2024-09-21T07:47:56.169245+0000","last_peered":"2024-09-21T07:47:56.169245+0000","last_clean":"2024-09-21T07:47:56.169245+0000","last_became_active":"2024-09-21T07:47:55.196996+0000","last_became_peered":"2024-09-21T07:47:55.196996+0000","last_unstale":"2024-09-21T07:47:56.169245+0000","last_undegraded":"2024-09-21T07:47:56.169245+0000","last_fullsized":"2024-09-21T07:47:56.169245+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:45:46.537703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.169098+0000","last_change":"2024-09-21T07:47:55.187489+0000","last_active":"2024-09-21T07:47:56.169098+0000","last_peered":"2024-09-21T07:47:56.169098+0000","last_clean":"2024-09-21T07:47:56.169098+0000","last_became_active":"2024-09-21T07:47:55.187178+0000","last_became_peered":"2024-09-21T07:47:55.187178+0000","last_unstale":"2024-09-21T07:47:56.169098+0000","last_undegraded":"2024-09-21T07:47:56.169098+0000","last_fullsized":"2024-09-21T07:47:56.169098+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:23:39.689508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.207407+0000","last_change":"2024-09-21T07:47:55.203335+0000","last_active":"2024-09-21T07:47:55.207407+0000","last_peered":"2024-09-21T07:47:55.207407+0000","last_clean":"2024-09-21T07:47:55.207407+0000","last_became_active":"2024-09-21T07:47:55.202467+0000","last_became_peered":"2024-09-21T07:47:55.202467+0000","last_unstale":"2024-09-21T07:47:55.207407+0000","last_undegraded":"2024-09-21T07:47:55.207407+0000","last_fullsized":"2024-09-21T07:47:55.207407+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:52.609270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.168695+0000","last_change":"2024-09-21T07:47:55.200289+0000","last_active":"2024-09-21T07:47:56.168695+0000","last_peered":"2024-09-21T07:47:56.168695+0000","last_clean":"2024-09-21T07:47:56.168695+0000","last_became_active":"2024-09-21T07:47:55.199903+0000","last_became_peered":"2024-09-21T07:47:55.199903+0000","last_unstale":"2024-09-21T07:47:56.168695+0000","last_undegraded":"2024-09-21T07:47:56.168695+0000","last_fullsized":"2024-09-21T07:47:56.168695+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:35:55.605781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172692+0000","last_change":"2024-09-21T07:47:55.174506+0000","last_active":"2024-09-21T07:47:56.172692+0000","last_peered":"2024-09-21T07:47:56.172692+0000","last_clean":"2024-09-21T07:47:56.172692+0000","last_became_active":"2024-09-21T07:47:55.174207+0000","last_became_peered":"2024-09-21T07:47:55.174207+0000","last_unstale":"2024-09-21T07:47:56.172692+0000","last_undegraded":"2024-09-21T07:47:56.172692+0000","last_fullsized":"2024-09-21T07:47:56.172692+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:42.250865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172548+0000","last_change":"2024-09-21T07:47:55.176732+0000","last_active":"2024-09-21T07:47:56.172548+0000","last_peered":"2024-09-21T07:47:56.172548+0000","last_clean":"2024-09-21T07:47:56.172548+0000","last_became_active":"2024-09-21T07:47:55.175848+0000","last_became_peered":"2024-09-21T07:47:55.175848+0000","last_unstale":"2024-09-21T07:47:56.172548+0000","last_undegraded":"2024-09-21T07:47:56.172548+0000","last_fullsized":"2024-09-21T07:47:56.172548+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:04:42.927874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.202768+0000","last_change":"2024-09-21T07:47:55.199515+0000","last_active":"2024-09-21T07:47:55.202768+0000","last_peered":"2024-09-21T07:47:55.202768+0000","last_clean":"2024-09-21T07:47:55.202768+0000","last_became_active":"2024-09-21T07:47:55.196699+0000","last_became_peered":"2024-09-21T07:47:55.196699+0000","last_unstale":"2024-09-21T07:47:55.202768+0000","last_undegraded":"2024-09-21T07:47:55.202768+0000","last_fullsized":"2024-09-21T07:47:55.202768+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:55.754358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.179419+0000","last_change":"2024-09-21T07:47:55.175818+0000","last_active":"2024-09-21T07:47:55.179419+0000","last_peered":"2024-09-21T07:47:55.179419+0000","last_clean":"2024-09-21T07:47:55.179419+0000","last_became_active":"2024-09-21T07:47:55.175507+0000","last_became_peered":"2024-09-21T07:47:55.175507+0000","last_unstale":"2024-09-21T07:47:55.179419+0000","last_undegraded":"2024-09-21T07:47:55.179419+0000","last_fullsized":"2024-09-21T07:47:55.179419+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:06.645509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172433+0000","last_change":"2024-09-21T07:47:55.198596+0000","last_active":"2024-09-21T07:47:56.172433+0000","last_peered":"2024-09-21T07:47:56.172433+0000","last_clean":"2024-09-21T07:47:56.172433+0000","last_became_active":"2024-09-21T07:47:55.198281+0000","last_became_peered":"2024-09-21T07:47:55.198281+0000","last_unstale":"2024-09-21T07:47:56.172433+0000","last_undegraded":"2024-09-21T07:47:56.172433+0000","last_fullsized":"2024-09-21T07:47:56.172433+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:34.670653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172092+0000","last_change":"2024-09-21T07:47:55.183389+0000","last_active":"2024-09-21T07:47:56.172092+0000","last_peered":"2024-09-21T07:47:56.172092+0000","last_clean":"2024-09-21T07:47:56.172092+0000","last_became_active":"2024-09-21T07:47:55.183064+0000","last_became_peered":"2024-09-21T07:47:55.183064+0000","last_unstale":"2024-09-21T07:47:56.172092+0000","last_undegraded":"2024-09-21T07:47:56.172092+0000","last_fullsized":"2024-09-21T07:47:56.172092+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:26:40.505785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.171548+0000","last_change":"2024-09-21T07:47:55.183428+0000","last_active":"2024-09-21T07:47:56.171548+0000","last_peered":"2024-09-21T07:47:56.171548+0000","last_clean":"2024-09-21T07:47:56.171548+0000","last_became_active":"2024-09-21T07:47:55.182138+0000","last_became_peered":"2024-09-21T07:47:55.182138+0000","last_unstale":"2024-09-21T07:47:56.171548+0000","last_undegraded":"2024-09-21T07:47:56.171548+0000","last_fullsized":"2024-09-21T07:47:56.171548+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:19.513238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.196763+0000","last_change":"2024-09-21T07:47:55.191988+0000","last_active":"2024-09-21T07:47:55.196763+0000","last_peered":"2024-09-21T07:47:55.196763+0000","last_clean":"2024-09-21T07:47:55.196763+0000","last_became_active":"2024-09-21T07:47:55.191656+0000","last_became_peered":"2024-09-21T07:47:55.191656+0000","last_unstale":"2024-09-21T07:47:55.196763+0000","last_undegraded":"2024-09-21T07:47:55.196763+0000","last_fullsized":"2024-09-21T07:47:55.196763+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:22.981316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.196614+0000","last_change":"2024-09-21T07:47:55.190051+0000","last_active":"2024-09-21T07:47:55.196614+0000","last_peered":"2024-09-21T07:47:55.196614+0000","last_clean":"2024-09-21T07:47:55.196614+0000","last_became_active":"2024-09-21T07:47:55.188973+0000","last_became_peered":"2024-09-21T07:47:55.188973+0000","last_unstale":"2024-09-21T07:47:55.196614+0000","last_undegraded":"2024-09-21T07:47:55.196614+0000","last_fullsized":"2024-09-21T07:47:55.196614+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:01.028459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":13,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.179136+0000","last_change":"2024-09-21T07:47:54.184109+0000","last_active":"2024-09-21T07:47:55.179136+0000","last_peered":"2024-09-21T07:47:55.179136+0000","last_clean":"2024-09-21T07:47:55.179136+0000","last_became_active":"2024-09-21T07:47:54.183814+0000","last_became_peered":"2024-09-21T07:47:54.183814+0000","last_unstale":"2024-09-21T07:47:55.179136+0000","last_undegraded":"2024-09-21T07:47:55.179136+0000","last_fullsized":"2024-09-21T07:47:55.179136+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:00.609458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":280,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.179351+0000","last_change":"2024-09-21T07:35:27.879605+0000","last_active":"2024-09-21T07:47:55.179351+0000","last_peered":"2024-09-21T07:47:55.179351+0000","last_clean":"2024-09-21T07:47:55.179351+0000","last_became_active":"2024-09-21T07:35:27.879274+0000","last_became_peered":"2024-09-21T07:35:27.879274+0000","last_unstale":"2024-09-21T07:47:55.179351+0000","last_undegraded":"2024-09-21T07:47:55.179351+0000","last_fullsized":"2024-09-21T07:47:55.179351+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:24.241636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":290,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.165507+0000","last_change":"2024-09-21T07:35:29.491336+0000","last_active":"2024-09-21T07:47:55.165507+0000","last_peered":"2024-09-21T07:47:55.165507+0000","last_clean":"2024-09-21T07:47:55.165507+0000","last_became_active":"2024-09-21T07:35:26.866484+0000","last_became_peered":"2024-09-21T07:35:26.866484+0000","last_unstale":"2024-09-21T07:47:55.165507+0000","last_undegraded":"2024-09-21T07:47:55.165507+0000","last_fullsized":"2024-09-21T07:47:55.165507+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:10.449665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076010100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":466,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.181807+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:47:55.181807+0000","last_peered":"2024-09-21T07:47:55.181807+0000","last_clean":"2024-09-21T07:47:55.181807+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:47:55.181807+0000","last_undegraded":"2024-09-21T07:47:55.181807+0000","last_fullsized":"2024-09-21T07:47:55.181807+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":353,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.173167+0000","last_change":"2024-09-21T07:34:24.539248+0000","last_active":"2024-09-21T07:47:56.173167+0000","last_peered":"2024-09-21T07:47:56.173167+0000","last_clean":"2024-09-21T07:47:56.173167+0000","last_became_active":"2024-09-21T07:34:23.273465+0000","last_became_peered":"2024-09-21T07:34:23.273465+0000","last_unstale":"2024-09-21T07:47:56.173167+0000","last_undegraded":"2024-09-21T07:47:56.173167+0000","last_fullsized":"2024-09-21T07:47:56.173167+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:55.517452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060627700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172828+0000","last_change":"2024-09-21T07:47:55.198057+0000","last_active":"2024-09-21T07:47:56.172828+0000","last_peered":"2024-09-21T07:47:56.172828+0000","last_clean":"2024-09-21T07:47:56.172828+0000","last_became_active":"2024-09-21T07:47:55.197705+0000","last_became_peered":"2024-09-21T07:47:55.197705+0000","last_unstale":"2024-09-21T07:47:56.172828+0000","last_undegraded":"2024-09-21T07:47:56.172828+0000","last_fullsized":"2024-09-21T07:47:56.172828+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:55.610512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"205'2","reported_seq":17,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172617+0000","last_change":"2024-09-21T07:47:54.177156+0000","last_active":"2024-09-21T07:47:56.172617+0000","last_peered":"2024-09-21T07:47:56.172617+0000","last_clean":"2024-09-21T07:47:56.172617+0000","last_became_active":"2024-09-21T07:47:54.176814+0000","last_became_peered":"2024-09-21T07:47:54.176814+0000","last_unstale":"2024-09-21T07:47:56.172617+0000","last_undegraded":"2024-09-21T07:47:56.172617+0000","last_fullsized":"2024-09-21T07:47:56.172617+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:26.953022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":283,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.164751+0000","last_change":"2024-09-21T07:35:27.864492+0000","last_active":"2024-09-21T07:47:55.164751+0000","last_peered":"2024-09-21T07:47:55.164751+0000","last_clean":"2024-09-21T07:47:55.164751+0000","last_became_active":"2024-09-21T07:35:27.864176+0000","last_became_peered":"2024-09-21T07:35:27.864176+0000","last_unstale":"2024-09-21T07:47:55.164751+0000","last_undegraded":"2024-09-21T07:47:55.164751+0000","last_fullsized":"2024-09-21T07:47:55.164751+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:05.153467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":287,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.177082+0000","last_change":"2024-09-21T07:35:29.493678+0000","last_active":"2024-09-21T07:47:55.177082+0000","last_peered":"2024-09-21T07:47:55.177082+0000","last_clean":"2024-09-21T07:47:55.177082+0000","last_became_active":"2024-09-21T07:35:26.870749+0000","last_became_peered":"2024-09-21T07:35:26.870749+0000","last_unstale":"2024-09-21T07:47:55.177082+0000","last_undegraded":"2024-09-21T07:47:55.177082+0000","last_fullsized":"2024-09-21T07:47:55.177082+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:55.367877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010159570000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":333,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.184364+0000","last_change":"2024-09-21T07:34:24.597274+0000","last_active":"2024-09-21T07:47:55.184364+0000","last_peered":"2024-09-21T07:47:55.184364+0000","last_clean":"2024-09-21T07:47:55.184364+0000","last_became_active":"2024-09-21T07:34:23.279560+0000","last_became_peered":"2024-09-21T07:34:23.279560+0000","last_unstale":"2024-09-21T07:47:55.184364+0000","last_undegraded":"2024-09-21T07:47:55.184364+0000","last_fullsized":"2024-09-21T07:47:55.184364+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:15.845265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059699799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.189575+0000","last_change":"2024-09-21T07:47:55.185072+0000","last_active":"2024-09-21T07:47:55.189575+0000","last_peered":"2024-09-21T07:47:55.189575+0000","last_clean":"2024-09-21T07:47:55.189575+0000","last_became_active":"2024-09-21T07:47:55.182751+0000","last_became_peered":"2024-09-21T07:47:55.182751+0000","last_unstale":"2024-09-21T07:47:55.189575+0000","last_undegraded":"2024-09-21T07:47:55.189575+0000","last_fullsized":"2024-09-21T07:47:55.189575+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:53:02.800636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"205'2","reported_seq":15,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.203505+0000","last_change":"2024-09-21T07:47:54.193048+0000","last_active":"2024-09-21T07:47:55.203505+0000","last_peered":"2024-09-21T07:47:55.203505+0000","last_clean":"2024-09-21T07:47:55.203505+0000","last_became_active":"2024-09-21T07:47:54.192522+0000","last_became_peered":"2024-09-21T07:47:54.192522+0000","last_unstale":"2024-09-21T07:47:55.203505+0000","last_undegraded":"2024-09-21T07:47:55.203505+0000","last_fullsized":"2024-09-21T07:47:55.203505+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:12.868785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":285,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.171964+0000","last_change":"2024-09-21T07:35:27.874387+0000","last_active":"2024-09-21T07:47:56.171964+0000","last_peered":"2024-09-21T07:47:56.171964+0000","last_clean":"2024-09-21T07:47:56.171964+0000","last_became_active":"2024-09-21T07:35:27.873277+0000","last_became_peered":"2024-09-21T07:35:27.873277+0000","last_unstale":"2024-09-21T07:47:56.171964+0000","last_undegraded":"2024-09-21T07:47:56.171964+0000","last_fullsized":"2024-09-21T07:47:56.171964+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:04.581463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":283,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.186438+0000","last_change":"2024-09-21T07:35:29.387094+0000","last_active":"2024-09-21T07:47:55.186438+0000","last_peered":"2024-09-21T07:47:55.186438+0000","last_clean":"2024-09-21T07:47:55.186438+0000","last_became_active":"2024-09-21T07:35:26.874757+0000","last_became_peered":"2024-09-21T07:35:26.874757+0000","last_unstale":"2024-09-21T07:47:55.186438+0000","last_undegraded":"2024-09-21T07:47:55.186438+0000","last_fullsized":"2024-09-21T07:47:55.186438+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:33.542750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061065700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":340,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.176872+0000","last_change":"2024-09-21T07:34:24.259906+0000","last_active":"2024-09-21T07:47:55.176872+0000","last_peered":"2024-09-21T07:47:55.176872+0000","last_clean":"2024-09-21T07:47:55.176872+0000","last_became_active":"2024-09-21T07:34:23.635282+0000","last_became_peered":"2024-09-21T07:34:23.635282+0000","last_unstale":"2024-09-21T07:47:55.176872+0000","last_undegraded":"2024-09-21T07:47:55.176872+0000","last_fullsized":"2024-09-21T07:47:55.176872+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:00.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001527664,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.205209+0000","last_change":"2024-09-21T07:47:55.198827+0000","last_active":"2024-09-21T07:47:55.205209+0000","last_peered":"2024-09-21T07:47:55.205209+0000","last_clean":"2024-09-21T07:47:55.205209+0000","last_became_active":"2024-09-21T07:47:55.198535+0000","last_became_peered":"2024-09-21T07:47:55.198535+0000","last_unstale":"2024-09-21T07:47:55.205209+0000","last_undegraded":"2024-09-21T07:47:55.205209+0000","last_fullsized":"2024-09-21T07:47:55.205209+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:31:14.691225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"205'4","reported_seq":17,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.209633+0000","last_change":"2024-09-21T07:47:54.193701+0000","last_active":"2024-09-21T07:47:55.209633+0000","last_peered":"2024-09-21T07:47:55.209633+0000","last_clean":"2024-09-21T07:47:55.209633+0000","last_became_active":"2024-09-21T07:47:54.193394+0000","last_became_peered":"2024-09-21T07:47:54.193394+0000","last_unstale":"2024-09-21T07:47:55.209633+0000","last_undegraded":"2024-09-21T07:47:55.209633+0000","last_fullsized":"2024-09-21T07:47:55.209633+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:26:38.936794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":285,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.171848+0000","last_change":"2024-09-21T07:35:27.873645+0000","last_active":"2024-09-21T07:47:56.171848+0000","last_peered":"2024-09-21T07:47:56.171848+0000","last_clean":"2024-09-21T07:47:56.171848+0000","last_became_active":"2024-09-21T07:35:27.873245+0000","last_became_peered":"2024-09-21T07:35:27.873245+0000","last_unstale":"2024-09-21T07:47:56.171848+0000","last_undegraded":"2024-09-21T07:47:56.171848+0000","last_fullsized":"2024-09-21T07:47:56.171848+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:55.090953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":292,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172335+0000","last_change":"2024-09-21T07:35:28.861744+0000","last_active":"2024-09-21T07:47:56.172335+0000","last_peered":"2024-09-21T07:47:56.172335+0000","last_clean":"2024-09-21T07:47:56.172335+0000","last_became_active":"2024-09-21T07:35:26.876462+0000","last_became_peered":"2024-09-21T07:35:26.876462+0000","last_unstale":"2024-09-21T07:47:56.172335+0000","last_undegraded":"2024-09-21T07:47:56.172335+0000","last_fullsized":"2024-09-21T07:47:56.172335+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:48.632568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012020920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":337,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.178862+0000","last_change":"2024-09-21T07:34:25.085454+0000","last_active":"2024-09-21T07:47:55.178862+0000","last_peered":"2024-09-21T07:47:55.178862+0000","last_clean":"2024-09-21T07:47:55.178862+0000","last_became_active":"2024-09-21T07:34:23.279205+0000","last_became_peered":"2024-09-21T07:34:23.279205+0000","last_unstale":"2024-09-21T07:47:55.178862+0000","last_undegraded":"2024-09-21T07:47:55.178862+0000","last_fullsized":"2024-09-21T07:47:55.178862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:54.231666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095320599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172338+0000","last_change":"2024-09-21T07:47:55.191049+0000","last_active":"2024-09-21T07:47:56.172338+0000","last_peered":"2024-09-21T07:47:56.172338+0000","last_clean":"2024-09-21T07:47:56.172338+0000","last_became_active":"2024-09-21T07:47:55.189300+0000","last_became_peered":"2024-09-21T07:47:55.189300+0000","last_unstale":"2024-09-21T07:47:56.172338+0000","last_undegraded":"2024-09-21T07:47:56.172338+0000","last_fullsized":"2024-09-21T07:47:56.172338+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:03.710044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"205'1","reported_seq":14,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.201531+0000","last_change":"2024-09-21T07:47:54.192849+0000","last_active":"2024-09-21T07:47:55.201531+0000","last_peered":"2024-09-21T07:47:55.201531+0000","last_clean":"2024-09-21T07:47:55.201531+0000","last_became_active":"2024-09-21T07:47:54.189254+0000","last_became_peered":"2024-09-21T07:47:54.189254+0000","last_unstale":"2024-09-21T07:47:55.201531+0000","last_undegraded":"2024-09-21T07:47:55.201531+0000","last_fullsized":"2024-09-21T07:47:55.201531+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:45.982647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":283,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.169829+0000","last_change":"2024-09-21T07:35:27.861373+0000","last_active":"2024-09-21T07:47:55.169829+0000","last_peered":"2024-09-21T07:47:55.169829+0000","last_clean":"2024-09-21T07:47:55.169829+0000","last_became_active":"2024-09-21T07:35:27.861016+0000","last_became_peered":"2024-09-21T07:35:27.861016+0000","last_unstale":"2024-09-21T07:47:55.169829+0000","last_undegraded":"2024-09-21T07:47:55.169829+0000","last_fullsized":"2024-09-21T07:47:55.169829+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:56.043763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":291,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.169748+0000","last_change":"2024-09-21T07:35:29.053677+0000","last_active":"2024-09-21T07:47:56.169748+0000","last_peered":"2024-09-21T07:47:56.169748+0000","last_clean":"2024-09-21T07:47:56.169748+0000","last_became_active":"2024-09-21T07:35:26.875496+0000","last_became_peered":"2024-09-21T07:35:26.875496+0000","last_unstale":"2024-09-21T07:47:56.169748+0000","last_undegraded":"2024-09-21T07:47:56.169748+0000","last_fullsized":"2024-09-21T07:47:56.169748+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:04.850784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065859000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":343,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.168810+0000","last_change":"2024-09-21T07:34:25.028521+0000","last_active":"2024-09-21T07:47:56.168810+0000","last_peered":"2024-09-21T07:47:56.168810+0000","last_clean":"2024-09-21T07:47:56.168810+0000","last_became_active":"2024-09-21T07:34:23.635467+0000","last_became_peered":"2024-09-21T07:34:23.635467+0000","last_unstale":"2024-09-21T07:47:56.168810+0000","last_undegraded":"2024-09-21T07:47:56.168810+0000","last_fullsized":"2024-09-21T07:47:56.168810+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:33.729123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000660558,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.169236+0000","last_change":"2024-09-21T07:47:55.198778+0000","last_active":"2024-09-21T07:47:56.169236+0000","last_peered":"2024-09-21T07:47:56.169236+0000","last_clean":"2024-09-21T07:47:56.169236+0000","last_became_active":"2024-09-21T07:47:55.198442+0000","last_became_peered":"2024-09-21T07:47:55.198442+0000","last_unstale":"2024-09-21T07:47:56.169236+0000","last_undegraded":"2024-09-21T07:47:56.169236+0000","last_fullsized":"2024-09-21T07:47:56.169236+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:28:10.861128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":13,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.184382+0000","last_change":"2024-09-21T07:47:54.187807+0000","last_active":"2024-09-21T07:47:55.184382+0000","last_peered":"2024-09-21T07:47:55.184382+0000","last_clean":"2024-09-21T07:47:55.184382+0000","last_became_active":"2024-09-21T07:47:54.183644+0000","last_became_peered":"2024-09-21T07:47:54.183644+0000","last_unstale":"2024-09-21T07:47:55.184382+0000","last_undegraded":"2024-09-21T07:47:55.184382+0000","last_fullsized":"2024-09-21T07:47:55.184382+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:26:52.301094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":279,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.190182+0000","last_change":"2024-09-21T07:35:27.871201+0000","last_active":"2024-09-21T07:47:55.190182+0000","last_peered":"2024-09-21T07:47:55.190182+0000","last_clean":"2024-09-21T07:47:55.190182+0000","last_became_active":"2024-09-21T07:35:27.870816+0000","last_became_peered":"2024-09-21T07:35:27.870816+0000","last_unstale":"2024-09-21T07:47:55.190182+0000","last_undegraded":"2024-09-21T07:47:55.190182+0000","last_fullsized":"2024-09-21T07:47:55.190182+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:04.787450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":290,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.170408+0000","last_change":"2024-09-21T07:35:29.491874+0000","last_active":"2024-09-21T07:47:55.170408+0000","last_peered":"2024-09-21T07:47:55.170408+0000","last_clean":"2024-09-21T07:47:55.170408+0000","last_became_active":"2024-09-21T07:35:26.866381+0000","last_became_peered":"2024-09-21T07:35:26.866381+0000","last_unstale":"2024-09-21T07:47:55.170408+0000","last_undegraded":"2024-09-21T07:47:55.170408+0000","last_fullsized":"2024-09-21T07:47:55.170408+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:36.404647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073235399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":345,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.166761+0000","last_change":"2024-09-21T07:34:24.260046+0000","last_active":"2024-09-21T07:47:55.166761+0000","last_peered":"2024-09-21T07:47:55.166761+0000","last_clean":"2024-09-21T07:47:55.166761+0000","last_became_active":"2024-09-21T07:34:23.273431+0000","last_became_peered":"2024-09-21T07:34:23.273431+0000","last_unstale":"2024-09-21T07:47:55.166761+0000","last_undegraded":"2024-09-21T07:47:55.166761+0000","last_fullsized":"2024-09-21T07:47:55.166761+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:18.876057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019803310000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.203648+0000","last_change":"2024-09-21T07:47:55.200282+0000","last_active":"2024-09-21T07:47:55.203648+0000","last_peered":"2024-09-21T07:47:55.203648+0000","last_clean":"2024-09-21T07:47:55.203648+0000","last_became_active":"2024-09-21T07:47:55.199405+0000","last_became_peered":"2024-09-21T07:47:55.199405+0000","last_unstale":"2024-09-21T07:47:55.203648+0000","last_undegraded":"2024-09-21T07:47:55.203648+0000","last_fullsized":"2024-09-21T07:47:55.203648+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:07.342878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"205'1","reported_seq":14,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.194254+0000","last_change":"2024-09-21T07:47:54.192931+0000","last_active":"2024-09-21T07:47:55.194254+0000","last_peered":"2024-09-21T07:47:55.194254+0000","last_clean":"2024-09-21T07:47:55.194254+0000","last_became_active":"2024-09-21T07:47:54.190292+0000","last_became_peered":"2024-09-21T07:47:54.190292+0000","last_unstale":"2024-09-21T07:47:55.194254+0000","last_undegraded":"2024-09-21T07:47:55.194254+0000","last_fullsized":"2024-09-21T07:47:55.194254+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:55.206956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":279,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.191076+0000","last_change":"2024-09-21T07:35:27.871736+0000","last_active":"2024-09-21T07:47:55.191076+0000","last_peered":"2024-09-21T07:47:55.191076+0000","last_clean":"2024-09-21T07:47:55.191076+0000","last_became_active":"2024-09-21T07:35:27.871232+0000","last_became_peered":"2024-09-21T07:35:27.871232+0000","last_unstale":"2024-09-21T07:47:55.191076+0000","last_undegraded":"2024-09-21T07:47:55.191076+0000","last_fullsized":"2024-09-21T07:47:55.191076+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:04.419301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":290,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.169552+0000","last_change":"2024-09-21T07:35:29.491246+0000","last_active":"2024-09-21T07:47:55.169552+0000","last_peered":"2024-09-21T07:47:55.169552+0000","last_clean":"2024-09-21T07:47:55.169552+0000","last_became_active":"2024-09-21T07:35:26.866783+0000","last_became_peered":"2024-09-21T07:35:26.866783+0000","last_unstale":"2024-09-21T07:47:55.169552+0000","last_undegraded":"2024-09-21T07:47:55.169552+0000","last_fullsized":"2024-09-21T07:47:55.169552+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:28.881914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098474099999999992,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":343,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.169624+0000","last_change":"2024-09-21T07:34:25.028640+0000","last_active":"2024-09-21T07:47:56.169624+0000","last_peered":"2024-09-21T07:47:56.169624+0000","last_clean":"2024-09-21T07:47:56.169624+0000","last_became_active":"2024-09-21T07:34:23.260950+0000","last_became_peered":"2024-09-21T07:34:23.260950+0000","last_unstale":"2024-09-21T07:47:56.169624+0000","last_undegraded":"2024-09-21T07:47:56.169624+0000","last_fullsized":"2024-09-21T07:47:56.169624+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:23:42.121398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055135099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.168768+0000","last_change":"2024-09-21T07:47:55.181700+0000","last_active":"2024-09-21T07:47:56.168768+0000","last_peered":"2024-09-21T07:47:56.168768+0000","last_clean":"2024-09-21T07:47:56.168768+0000","last_became_active":"2024-09-21T07:47:55.181342+0000","last_became_peered":"2024-09-21T07:47:55.181342+0000","last_unstale":"2024-09-21T07:47:56.168768+0000","last_undegraded":"2024-09-21T07:47:56.168768+0000","last_fullsized":"2024-09-21T07:47:56.168768+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:23.800598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"205'1","reported_seq":14,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.206721+0000","last_change":"2024-09-21T07:47:54.192086+0000","last_active":"2024-09-21T07:47:55.206721+0000","last_peered":"2024-09-21T07:47:55.206721+0000","last_clean":"2024-09-21T07:47:55.206721+0000","last_became_active":"2024-09-21T07:47:54.191756+0000","last_became_peered":"2024-09-21T07:47:54.191756+0000","last_unstale":"2024-09-21T07:47:55.206721+0000","last_undegraded":"2024-09-21T07:47:55.206721+0000","last_fullsized":"2024-09-21T07:47:55.206721+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:10:23.633614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":281,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.180363+0000","last_change":"2024-09-21T07:35:27.872027+0000","last_active":"2024-09-21T07:47:55.180363+0000","last_peered":"2024-09-21T07:47:55.180363+0000","last_clean":"2024-09-21T07:47:55.180363+0000","last_became_active":"2024-09-21T07:35:27.870426+0000","last_became_peered":"2024-09-21T07:35:27.870426+0000","last_unstale":"2024-09-21T07:47:55.180363+0000","last_undegraded":"2024-09-21T07:47:55.180363+0000","last_fullsized":"2024-09-21T07:47:55.180363+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":290,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.165413+0000","last_change":"2024-09-21T07:35:29.491008+0000","last_active":"2024-09-21T07:47:55.165413+0000","last_peered":"2024-09-21T07:47:55.165413+0000","last_clean":"2024-09-21T07:47:55.165413+0000","last_became_active":"2024-09-21T07:35:26.866750+0000","last_became_peered":"2024-09-21T07:35:26.866750+0000","last_unstale":"2024-09-21T07:47:55.165413+0000","last_undegraded":"2024-09-21T07:47:55.165413+0000","last_fullsized":"2024-09-21T07:47:55.165413+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:25.490406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083529700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":341,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.184483+0000","last_change":"2024-09-21T07:34:24.260665+0000","last_active":"2024-09-21T07:47:55.184483+0000","last_peered":"2024-09-21T07:47:55.184483+0000","last_clean":"2024-09-21T07:47:55.184483+0000","last_became_active":"2024-09-21T07:34:23.635909+0000","last_became_peered":"2024-09-21T07:34:23.635909+0000","last_unstale":"2024-09-21T07:47:55.184483+0000","last_undegraded":"2024-09-21T07:47:55.184483+0000","last_fullsized":"2024-09-21T07:47:55.184483+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:44.536885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.204904+0000","last_change":"2024-09-21T07:47:55.192084+0000","last_active":"2024-09-21T07:47:55.204904+0000","last_peered":"2024-09-21T07:47:55.204904+0000","last_clean":"2024-09-21T07:47:55.204904+0000","last_became_active":"2024-09-21T07:47:55.189708+0000","last_became_peered":"2024-09-21T07:47:55.189708+0000","last_unstale":"2024-09-21T07:47:55.204904+0000","last_undegraded":"2024-09-21T07:47:55.204904+0000","last_fullsized":"2024-09-21T07:47:55.204904+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:42:29.513220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"205'3","reported_seq":16,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.207936+0000","last_change":"2024-09-21T07:47:54.197045+0000","last_active":"2024-09-21T07:47:55.207936+0000","last_peered":"2024-09-21T07:47:55.207936+0000","last_clean":"2024-09-21T07:47:55.207936+0000","last_became_active":"2024-09-21T07:47:54.194131+0000","last_became_peered":"2024-09-21T07:47:54.194131+0000","last_unstale":"2024-09-21T07:47:55.207936+0000","last_undegraded":"2024-09-21T07:47:55.207936+0000","last_fullsized":"2024-09-21T07:47:55.207936+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:57.606493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":277,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.182347+0000","last_change":"2024-09-21T07:35:27.874772+0000","last_active":"2024-09-21T07:47:55.182347+0000","last_peered":"2024-09-21T07:47:55.182347+0000","last_clean":"2024-09-21T07:47:55.182347+0000","last_became_active":"2024-09-21T07:35:27.874434+0000","last_became_peered":"2024-09-21T07:35:27.874434+0000","last_unstale":"2024-09-21T07:47:55.182347+0000","last_undegraded":"2024-09-21T07:47:55.182347+0000","last_fullsized":"2024-09-21T07:47:55.182347+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:31.998755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":300,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.173166+0000","last_change":"2024-09-21T07:35:28.859153+0000","last_active":"2024-09-21T07:47:56.173166+0000","last_peered":"2024-09-21T07:47:56.173166+0000","last_clean":"2024-09-21T07:47:56.173166+0000","last_became_active":"2024-09-21T07:35:26.874428+0000","last_became_peered":"2024-09-21T07:35:26.874428+0000","last_unstale":"2024-09-21T07:47:56.173166+0000","last_undegraded":"2024-09-21T07:47:56.173166+0000","last_fullsized":"2024-09-21T07:47:56.173166+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:04.487621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048557299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":353,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172080+0000","last_change":"2024-09-21T07:34:24.538845+0000","last_active":"2024-09-21T07:47:56.172080+0000","last_peered":"2024-09-21T07:47:56.172080+0000","last_clean":"2024-09-21T07:47:56.172080+0000","last_became_active":"2024-09-21T07:34:23.636372+0000","last_became_peered":"2024-09-21T07:34:23.636372+0000","last_unstale":"2024-09-21T07:47:56.172080+0000","last_undegraded":"2024-09-21T07:47:56.172080+0000","last_fullsized":"2024-09-21T07:47:56.172080+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:29.393813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841451,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.169685+0000","last_change":"2024-09-21T07:47:55.185787+0000","last_active":"2024-09-21T07:47:56.169685+0000","last_peered":"2024-09-21T07:47:56.169685+0000","last_clean":"2024-09-21T07:47:56.169685+0000","last_became_active":"2024-09-21T07:47:55.185387+0000","last_became_peered":"2024-09-21T07:47:55.185387+0000","last_unstale":"2024-09-21T07:47:56.169685+0000","last_undegraded":"2024-09-21T07:47:56.169685+0000","last_fullsized":"2024-09-21T07:47:56.169685+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:43.317041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"205'6","reported_seq":19,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.208655+0000","last_change":"2024-09-21T07:47:54.179178+0000","last_active":"2024-09-21T07:47:55.208655+0000","last_peered":"2024-09-21T07:47:55.208655+0000","last_clean":"2024-09-21T07:47:55.208655+0000","last_became_active":"2024-09-21T07:47:54.178851+0000","last_became_peered":"2024-09-21T07:47:54.178851+0000","last_unstale":"2024-09-21T07:47:55.208655+0000","last_undegraded":"2024-09-21T07:47:55.208655+0000","last_fullsized":"2024-09-21T07:47:55.208655+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:13.663241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":282,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.178675+0000","last_change":"2024-09-21T07:35:27.877801+0000","last_active":"2024-09-21T07:47:55.178675+0000","last_peered":"2024-09-21T07:47:55.178675+0000","last_clean":"2024-09-21T07:47:55.178675+0000","last_became_active":"2024-09-21T07:35:27.877464+0000","last_became_peered":"2024-09-21T07:35:27.877464+0000","last_unstale":"2024-09-21T07:47:55.178675+0000","last_undegraded":"2024-09-21T07:47:55.178675+0000","last_fullsized":"2024-09-21T07:47:55.178675+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:08.814929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":286,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.191204+0000","last_change":"2024-09-21T07:35:28.863732+0000","last_active":"2024-09-21T07:47:55.191204+0000","last_peered":"2024-09-21T07:47:55.191204+0000","last_clean":"2024-09-21T07:47:55.191204+0000","last_became_active":"2024-09-21T07:35:26.877392+0000","last_became_peered":"2024-09-21T07:35:26.877392+0000","last_unstale":"2024-09-21T07:47:55.191204+0000","last_undegraded":"2024-09-21T07:47:55.191204+0000","last_fullsized":"2024-09-21T07:47:55.191204+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071425700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":345,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.170285+0000","last_change":"2024-09-21T07:34:24.259686+0000","last_active":"2024-09-21T07:47:55.170285+0000","last_peered":"2024-09-21T07:47:55.170285+0000","last_clean":"2024-09-21T07:47:55.170285+0000","last_became_active":"2024-09-21T07:34:23.274405+0000","last_became_peered":"2024-09-21T07:34:23.274405+0000","last_unstale":"2024-09-21T07:47:55.170285+0000","last_undegraded":"2024-09-21T07:47:55.170285+0000","last_fullsized":"2024-09-21T07:47:55.170285+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:26.304693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022009920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.207451+0000","last_change":"2024-09-21T07:47:55.204047+0000","last_active":"2024-09-21T07:47:55.207451+0000","last_peered":"2024-09-21T07:47:55.207451+0000","last_clean":"2024-09-21T07:47:55.207451+0000","last_became_active":"2024-09-21T07:47:55.202068+0000","last_became_peered":"2024-09-21T07:47:55.202068+0000","last_unstale":"2024-09-21T07:47:55.207451+0000","last_undegraded":"2024-09-21T07:47:55.207451+0000","last_fullsized":"2024-09-21T07:47:55.207451+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:23.515802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":13,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.166889+0000","last_change":"2024-09-21T07:47:54.188332+0000","last_active":"2024-09-21T07:47:55.166889+0000","last_peered":"2024-09-21T07:47:55.166889+0000","last_clean":"2024-09-21T07:47:55.166889+0000","last_became_active":"2024-09-21T07:47:54.187549+0000","last_became_peered":"2024-09-21T07:47:54.187549+0000","last_unstale":"2024-09-21T07:47:55.166889+0000","last_undegraded":"2024-09-21T07:47:55.166889+0000","last_fullsized":"2024-09-21T07:47:55.166889+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:39:16.832519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":276,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.184158+0000","last_change":"2024-09-21T07:35:27.867336+0000","last_active":"2024-09-21T07:47:55.184158+0000","last_peered":"2024-09-21T07:47:55.184158+0000","last_clean":"2024-09-21T07:47:55.184158+0000","last_became_active":"2024-09-21T07:35:27.867015+0000","last_became_peered":"2024-09-21T07:35:27.867015+0000","last_unstale":"2024-09-21T07:47:55.184158+0000","last_undegraded":"2024-09-21T07:47:55.184158+0000","last_fullsized":"2024-09-21T07:47:55.184158+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:59.214139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":288,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.179498+0000","last_change":"2024-09-21T07:35:29.493137+0000","last_active":"2024-09-21T07:47:55.179498+0000","last_peered":"2024-09-21T07:47:55.179498+0000","last_clean":"2024-09-21T07:47:55.179498+0000","last_became_active":"2024-09-21T07:35:26.872230+0000","last_became_peered":"2024-09-21T07:35:26.872230+0000","last_unstale":"2024-09-21T07:47:55.179498+0000","last_undegraded":"2024-09-21T07:47:55.179498+0000","last_fullsized":"2024-09-21T07:47:55.179498+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:24.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001197449,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":341,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.181691+0000","last_change":"2024-09-21T07:34:24.258754+0000","last_active":"2024-09-21T07:47:55.181691+0000","last_peered":"2024-09-21T07:47:55.181691+0000","last_clean":"2024-09-21T07:47:55.181691+0000","last_became_active":"2024-09-21T07:34:23.272657+0000","last_became_peered":"2024-09-21T07:34:23.272657+0000","last_unstale":"2024-09-21T07:47:55.181691+0000","last_undegraded":"2024-09-21T07:47:55.181691+0000","last_fullsized":"2024-09-21T07:47:55.181691+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:05.477099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067143200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.187781+0000","last_change":"2024-09-21T07:47:55.184820+0000","last_active":"2024-09-21T07:47:55.187781+0000","last_peered":"2024-09-21T07:47:55.187781+0000","last_clean":"2024-09-21T07:47:55.187781+0000","last_became_active":"2024-09-21T07:47:55.184576+0000","last_became_peered":"2024-09-21T07:47:55.184576+0000","last_unstale":"2024-09-21T07:47:55.187781+0000","last_undegraded":"2024-09-21T07:47:55.187781+0000","last_fullsized":"2024-09-21T07:47:55.187781+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:40.419277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":15,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.169940+0000","last_change":"2024-09-21T07:47:54.191233+0000","last_active":"2024-09-21T07:47:56.169940+0000","last_peered":"2024-09-21T07:47:56.169940+0000","last_clean":"2024-09-21T07:47:56.169940+0000","last_became_active":"2024-09-21T07:47:54.190931+0000","last_became_peered":"2024-09-21T07:47:54.190931+0000","last_unstale":"2024-09-21T07:47:56.169940+0000","last_undegraded":"2024-09-21T07:47:56.169940+0000","last_fullsized":"2024-09-21T07:47:56.169940+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:40:18.466133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":280,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.178960+0000","last_change":"2024-09-21T07:35:27.876021+0000","last_active":"2024-09-21T07:47:55.178960+0000","last_peered":"2024-09-21T07:47:55.178960+0000","last_clean":"2024-09-21T07:47:55.178960+0000","last_became_active":"2024-09-21T07:35:27.875724+0000","last_became_peered":"2024-09-21T07:35:27.875724+0000","last_unstale":"2024-09-21T07:47:55.178960+0000","last_undegraded":"2024-09-21T07:47:55.178960+0000","last_fullsized":"2024-09-21T07:47:55.178960+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:03.022812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":288,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.178489+0000","last_change":"2024-09-21T07:35:29.492570+0000","last_active":"2024-09-21T07:47:55.178489+0000","last_peered":"2024-09-21T07:47:55.178489+0000","last_clean":"2024-09-21T07:47:55.178489+0000","last_became_active":"2024-09-21T07:35:26.874316+0000","last_became_peered":"2024-09-21T07:35:26.874316+0000","last_unstale":"2024-09-21T07:47:55.178489+0000","last_undegraded":"2024-09-21T07:47:55.178489+0000","last_fullsized":"2024-09-21T07:47:55.178489+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:16.512323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011990760000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":338,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.178425+0000","last_change":"2024-09-21T07:34:25.085232+0000","last_active":"2024-09-21T07:47:55.178425+0000","last_peered":"2024-09-21T07:47:55.178425+0000","last_clean":"2024-09-21T07:47:55.178425+0000","last_became_active":"2024-09-21T07:34:23.277807+0000","last_became_peered":"2024-09-21T07:34:23.277807+0000","last_unstale":"2024-09-21T07:47:55.178425+0000","last_undegraded":"2024-09-21T07:47:55.178425+0000","last_fullsized":"2024-09-21T07:47:55.178425+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:36.362919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086619099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.202404+0000","last_change":"2024-09-21T07:47:55.193141+0000","last_active":"2024-09-21T07:47:55.202404+0000","last_peered":"2024-09-21T07:47:55.202404+0000","last_clean":"2024-09-21T07:47:55.202404+0000","last_became_active":"2024-09-21T07:47:55.192811+0000","last_became_peered":"2024-09-21T07:47:55.192811+0000","last_unstale":"2024-09-21T07:47:55.202404+0000","last_undegraded":"2024-09-21T07:47:55.202404+0000","last_fullsized":"2024-09-21T07:47:55.202404+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:59.015996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":13,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.178077+0000","last_change":"2024-09-21T07:47:54.182443+0000","last_active":"2024-09-21T07:47:55.178077+0000","last_peered":"2024-09-21T07:47:55.178077+0000","last_clean":"2024-09-21T07:47:55.178077+0000","last_became_active":"2024-09-21T07:47:54.182126+0000","last_became_peered":"2024-09-21T07:47:54.182126+0000","last_unstale":"2024-09-21T07:47:55.178077+0000","last_undegraded":"2024-09-21T07:47:55.178077+0000","last_fullsized":"2024-09-21T07:47:55.178077+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:29.670475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":282,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.179745+0000","last_change":"2024-09-21T07:35:27.872798+0000","last_active":"2024-09-21T07:47:55.179745+0000","last_peered":"2024-09-21T07:47:55.179745+0000","last_clean":"2024-09-21T07:47:55.179745+0000","last_became_active":"2024-09-21T07:35:27.872483+0000","last_became_peered":"2024-09-21T07:35:27.872483+0000","last_unstale":"2024-09-21T07:47:55.179745+0000","last_undegraded":"2024-09-21T07:47:55.179745+0000","last_fullsized":"2024-09-21T07:47:55.179745+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:11.933469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":289,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.178460+0000","last_change":"2024-09-21T07:35:29.206723+0000","last_active":"2024-09-21T07:47:55.178460+0000","last_peered":"2024-09-21T07:47:55.178460+0000","last_clean":"2024-09-21T07:47:55.178460+0000","last_became_active":"2024-09-21T07:35:26.872377+0000","last_became_peered":"2024-09-21T07:35:26.872377+0000","last_unstale":"2024-09-21T07:47:55.178460+0000","last_undegraded":"2024-09-21T07:47:55.178460+0000","last_fullsized":"2024-09-21T07:47:55.178460+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:29.086862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083077200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":333,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.188254+0000","last_change":"2024-09-21T07:34:24.597672+0000","last_active":"2024-09-21T07:47:55.188254+0000","last_peered":"2024-09-21T07:47:55.188254+0000","last_clean":"2024-09-21T07:47:55.188254+0000","last_became_active":"2024-09-21T07:34:23.267770+0000","last_became_peered":"2024-09-21T07:34:23.267770+0000","last_unstale":"2024-09-21T07:47:55.188254+0000","last_undegraded":"2024-09-21T07:47:55.188254+0000","last_fullsized":"2024-09-21T07:47:55.188254+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:51.629476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086378500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.204644+0000","last_change":"2024-09-21T07:47:55.200811+0000","last_active":"2024-09-21T07:47:55.204644+0000","last_peered":"2024-09-21T07:47:55.204644+0000","last_clean":"2024-09-21T07:47:55.204644+0000","last_became_active":"2024-09-21T07:47:55.199782+0000","last_became_peered":"2024-09-21T07:47:55.199782+0000","last_unstale":"2024-09-21T07:47:55.204644+0000","last_undegraded":"2024-09-21T07:47:55.204644+0000","last_fullsized":"2024-09-21T07:47:55.204644+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:02:42.373550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"205'8","reported_seq":23,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.171817+0000","last_change":"2024-09-21T07:47:54.182287+0000","last_active":"2024-09-21T07:47:56.171817+0000","last_peered":"2024-09-21T07:47:56.171817+0000","last_clean":"2024-09-21T07:47:56.171817+0000","last_became_active":"2024-09-21T07:47:54.181670+0000","last_became_peered":"2024-09-21T07:47:54.181670+0000","last_unstale":"2024-09-21T07:47:56.171817+0000","last_undegraded":"2024-09-21T07:47:56.171817+0000","last_fullsized":"2024-09-21T07:47:56.171817+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:56:15.576668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":282,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.179711+0000","last_change":"2024-09-21T07:35:27.870121+0000","last_active":"2024-09-21T07:47:55.179711+0000","last_peered":"2024-09-21T07:47:55.179711+0000","last_clean":"2024-09-21T07:47:55.179711+0000","last_became_active":"2024-09-21T07:35:27.869811+0000","last_became_peered":"2024-09-21T07:35:27.869811+0000","last_unstale":"2024-09-21T07:47:55.179711+0000","last_undegraded":"2024-09-21T07:47:55.179711+0000","last_fullsized":"2024-09-21T07:47:55.179711+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:31.778461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":293,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.171823+0000","last_change":"2024-09-21T07:35:28.860699+0000","last_active":"2024-09-21T07:47:56.171823+0000","last_peered":"2024-09-21T07:47:56.171823+0000","last_clean":"2024-09-21T07:47:56.171823+0000","last_became_active":"2024-09-21T07:35:26.876779+0000","last_became_peered":"2024-09-21T07:35:26.876779+0000","last_unstale":"2024-09-21T07:47:56.171823+0000","last_undegraded":"2024-09-21T07:47:56.171823+0000","last_fullsized":"2024-09-21T07:47:56.171823+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:28.286403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000913188,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":345,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172607+0000","last_change":"2024-09-21T07:34:24.260181+0000","last_active":"2024-09-21T07:47:56.172607+0000","last_peered":"2024-09-21T07:47:56.172607+0000","last_clean":"2024-09-21T07:47:56.172607+0000","last_became_active":"2024-09-21T07:34:23.275580+0000","last_became_peered":"2024-09-21T07:34:23.275580+0000","last_unstale":"2024-09-21T07:47:56.172607+0000","last_undegraded":"2024-09-21T07:47:56.172607+0000","last_fullsized":"2024-09-21T07:47:56.172607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:49.857622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014706980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.171633+0000","last_change":"2024-09-21T07:47:55.189436+0000","last_active":"2024-09-21T07:47:56.171633+0000","last_peered":"2024-09-21T07:47:56.171633+0000","last_clean":"2024-09-21T07:47:56.171633+0000","last_became_active":"2024-09-21T07:47:55.189097+0000","last_became_peered":"2024-09-21T07:47:55.189097+0000","last_unstale":"2024-09-21T07:47:56.171633+0000","last_undegraded":"2024-09-21T07:47:56.171633+0000","last_fullsized":"2024-09-21T07:47:56.171633+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:49:09.381698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"205'2","reported_seq":15,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.202614+0000","last_change":"2024-09-21T07:47:54.187442+0000","last_active":"2024-09-21T07:47:55.202614+0000","last_peered":"2024-09-21T07:47:55.202614+0000","last_clean":"2024-09-21T07:47:55.202614+0000","last_became_active":"2024-09-21T07:47:54.186896+0000","last_became_peered":"2024-09-21T07:47:54.186896+0000","last_unstale":"2024-09-21T07:47:55.202614+0000","last_undegraded":"2024-09-21T07:47:55.202614+0000","last_fullsized":"2024-09-21T07:47:55.202614+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:39:02.858914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":284,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.170517+0000","last_change":"2024-09-21T07:35:27.875393+0000","last_active":"2024-09-21T07:47:56.170517+0000","last_peered":"2024-09-21T07:47:56.170517+0000","last_clean":"2024-09-21T07:47:56.170517+0000","last_became_active":"2024-09-21T07:35:27.874671+0000","last_became_peered":"2024-09-21T07:35:27.874671+0000","last_unstale":"2024-09-21T07:47:56.170517+0000","last_undegraded":"2024-09-21T07:47:56.170517+0000","last_fullsized":"2024-09-21T07:47:56.170517+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":289,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.178797+0000","last_change":"2024-09-21T07:35:29.206296+0000","last_active":"2024-09-21T07:47:55.178797+0000","last_peered":"2024-09-21T07:47:55.178797+0000","last_clean":"2024-09-21T07:47:55.178797+0000","last_became_active":"2024-09-21T07:35:26.874575+0000","last_became_peered":"2024-09-21T07:35:26.874575+0000","last_unstale":"2024-09-21T07:47:55.178797+0000","last_undegraded":"2024-09-21T07:47:55.178797+0000","last_fullsized":"2024-09-21T07:47:55.178797+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:22.771154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061011599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":345,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.170685+0000","last_change":"2024-09-21T07:34:24.259112+0000","last_active":"2024-09-21T07:47:55.170685+0000","last_peered":"2024-09-21T07:47:55.170685+0000","last_clean":"2024-09-21T07:47:55.170685+0000","last_became_active":"2024-09-21T07:34:23.635444+0000","last_became_peered":"2024-09-21T07:34:23.635444+0000","last_unstale":"2024-09-21T07:47:55.170685+0000","last_undegraded":"2024-09-21T07:47:55.170685+0000","last_fullsized":"2024-09-21T07:47:55.170685+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:46.255385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001952636,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.195542+0000","last_change":"2024-09-21T07:47:55.192886+0000","last_active":"2024-09-21T07:47:55.195542+0000","last_peered":"2024-09-21T07:47:55.195542+0000","last_clean":"2024-09-21T07:47:55.195542+0000","last_became_active":"2024-09-21T07:47:55.192478+0000","last_became_peered":"2024-09-21T07:47:55.192478+0000","last_unstale":"2024-09-21T07:47:55.195542+0000","last_undegraded":"2024-09-21T07:47:55.195542+0000","last_fullsized":"2024-09-21T07:47:55.195542+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:45.995506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":15,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.170182+0000","last_change":"2024-09-21T07:47:54.186176+0000","last_active":"2024-09-21T07:47:56.170182+0000","last_peered":"2024-09-21T07:47:56.170182+0000","last_clean":"2024-09-21T07:47:56.170182+0000","last_became_active":"2024-09-21T07:47:54.185846+0000","last_became_peered":"2024-09-21T07:47:54.185846+0000","last_unstale":"2024-09-21T07:47:56.170182+0000","last_undegraded":"2024-09-21T07:47:56.170182+0000","last_fullsized":"2024-09-21T07:47:56.170182+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:48.885930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":284,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.170487+0000","last_change":"2024-09-21T07:35:27.878356+0000","last_active":"2024-09-21T07:47:56.170487+0000","last_peered":"2024-09-21T07:47:56.170487+0000","last_clean":"2024-09-21T07:47:56.170487+0000","last_became_active":"2024-09-21T07:35:27.877982+0000","last_became_peered":"2024-09-21T07:35:27.877982+0000","last_unstale":"2024-09-21T07:47:56.170487+0000","last_undegraded":"2024-09-21T07:47:56.170487+0000","last_fullsized":"2024-09-21T07:47:56.170487+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":292,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.168604+0000","last_change":"2024-09-21T07:35:29.053131+0000","last_active":"2024-09-21T07:47:56.168604+0000","last_peered":"2024-09-21T07:47:56.168604+0000","last_clean":"2024-09-21T07:47:56.168604+0000","last_became_active":"2024-09-21T07:35:26.870516+0000","last_became_peered":"2024-09-21T07:35:26.870516+0000","last_unstale":"2024-09-21T07:47:56.168604+0000","last_undegraded":"2024-09-21T07:47:56.168604+0000","last_fullsized":"2024-09-21T07:47:56.168604+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:10.924491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070705699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":353,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172815+0000","last_change":"2024-09-21T07:34:24.539152+0000","last_active":"2024-09-21T07:47:56.172815+0000","last_peered":"2024-09-21T07:47:56.172815+0000","last_clean":"2024-09-21T07:47:56.172815+0000","last_became_active":"2024-09-21T07:34:23.259573+0000","last_became_peered":"2024-09-21T07:34:23.259573+0000","last_unstale":"2024-09-21T07:47:56.172815+0000","last_undegraded":"2024-09-21T07:47:56.172815+0000","last_fullsized":"2024-09-21T07:47:56.172815+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:26.986900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058108400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.173046+0000","last_change":"2024-09-21T07:47:55.176590+0000","last_active":"2024-09-21T07:47:56.173046+0000","last_peered":"2024-09-21T07:47:56.173046+0000","last_clean":"2024-09-21T07:47:56.173046+0000","last_became_active":"2024-09-21T07:47:55.175383+0000","last_became_peered":"2024-09-21T07:47:55.175383+0000","last_unstale":"2024-09-21T07:47:56.173046+0000","last_undegraded":"2024-09-21T07:47:56.173046+0000","last_fullsized":"2024-09-21T07:47:56.173046+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:25.181429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"205'1","reported_seq":14,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.206122+0000","last_change":"2024-09-21T07:47:54.195938+0000","last_active":"2024-09-21T07:47:55.206122+0000","last_peered":"2024-09-21T07:47:55.206122+0000","last_clean":"2024-09-21T07:47:55.206122+0000","last_became_active":"2024-09-21T07:47:54.195639+0000","last_became_peered":"2024-09-21T07:47:54.195639+0000","last_unstale":"2024-09-21T07:47:55.206122+0000","last_undegraded":"2024-09-21T07:47:55.206122+0000","last_fullsized":"2024-09-21T07:47:55.206122+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:20:56.328017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":284,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.168389+0000","last_change":"2024-09-21T07:35:27.873726+0000","last_active":"2024-09-21T07:47:56.168389+0000","last_peered":"2024-09-21T07:47:56.168389+0000","last_clean":"2024-09-21T07:47:56.168389+0000","last_became_active":"2024-09-21T07:35:27.871421+0000","last_became_peered":"2024-09-21T07:35:27.871421+0000","last_unstale":"2024-09-21T07:47:56.168389+0000","last_undegraded":"2024-09-21T07:47:56.168389+0000","last_fullsized":"2024-09-21T07:47:56.168389+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:00.168915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":293,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172481+0000","last_change":"2024-09-21T07:35:28.861032+0000","last_active":"2024-09-21T07:47:56.172481+0000","last_peered":"2024-09-21T07:47:56.172481+0000","last_clean":"2024-09-21T07:47:56.172481+0000","last_became_active":"2024-09-21T07:35:26.873149+0000","last_became_peered":"2024-09-21T07:35:26.873149+0000","last_unstale":"2024-09-21T07:47:56.172481+0000","last_undegraded":"2024-09-21T07:47:56.172481+0000","last_fullsized":"2024-09-21T07:47:56.172481+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:59.435981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011988369999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":340,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.175754+0000","last_change":"2024-09-21T07:34:24.259902+0000","last_active":"2024-09-21T07:47:55.175754+0000","last_peered":"2024-09-21T07:47:55.175754+0000","last_clean":"2024-09-21T07:47:55.175754+0000","last_became_active":"2024-09-21T07:34:23.271902+0000","last_became_peered":"2024-09-21T07:34:23.271902+0000","last_unstale":"2024-09-21T07:47:55.175754+0000","last_undegraded":"2024-09-21T07:47:55.175754+0000","last_fullsized":"2024-09-21T07:47:55.175754+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:59.439525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001686358,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.169943+0000","last_change":"2024-09-21T07:47:55.198986+0000","last_active":"2024-09-21T07:47:56.169943+0000","last_peered":"2024-09-21T07:47:56.169943+0000","last_clean":"2024-09-21T07:47:56.169943+0000","last_became_active":"2024-09-21T07:47:55.195736+0000","last_became_peered":"2024-09-21T07:47:55.195736+0000","last_unstale":"2024-09-21T07:47:56.169943+0000","last_undegraded":"2024-09-21T07:47:56.169943+0000","last_fullsized":"2024-09-21T07:47:56.169943+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:43.281303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":15,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172405+0000","last_change":"2024-09-21T07:47:54.185310+0000","last_active":"2024-09-21T07:47:56.172405+0000","last_peered":"2024-09-21T07:47:56.172405+0000","last_clean":"2024-09-21T07:47:56.172405+0000","last_became_active":"2024-09-21T07:47:54.184966+0000","last_became_peered":"2024-09-21T07:47:54.184966+0000","last_unstale":"2024-09-21T07:47:56.172405+0000","last_undegraded":"2024-09-21T07:47:56.172405+0000","last_fullsized":"2024-09-21T07:47:56.172405+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:03:17.404002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":279,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.178151+0000","last_change":"2024-09-21T07:35:27.872637+0000","last_active":"2024-09-21T07:47:55.178151+0000","last_peered":"2024-09-21T07:47:55.178151+0000","last_clean":"2024-09-21T07:47:55.178151+0000","last_became_active":"2024-09-21T07:35:27.872149+0000","last_became_peered":"2024-09-21T07:35:27.872149+0000","last_unstale":"2024-09-21T07:47:55.178151+0000","last_undegraded":"2024-09-21T07:47:55.178151+0000","last_fullsized":"2024-09-21T07:47:55.178151+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:27.697872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":288,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.175994+0000","last_change":"2024-09-21T07:35:29.493362+0000","last_active":"2024-09-21T07:47:55.175994+0000","last_peered":"2024-09-21T07:47:55.175994+0000","last_clean":"2024-09-21T07:47:55.175994+0000","last_became_active":"2024-09-21T07:35:26.873018+0000","last_became_peered":"2024-09-21T07:35:26.873018+0000","last_unstale":"2024-09-21T07:47:55.175994+0000","last_undegraded":"2024-09-21T07:47:55.175994+0000","last_fullsized":"2024-09-21T07:47:55.175994+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:02.460557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012496510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":344,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.170071+0000","last_change":"2024-09-21T07:34:24.259662+0000","last_active":"2024-09-21T07:47:55.170071+0000","last_peered":"2024-09-21T07:47:55.170071+0000","last_clean":"2024-09-21T07:47:55.170071+0000","last_became_active":"2024-09-21T07:34:23.278812+0000","last_became_peered":"2024-09-21T07:34:23.278812+0000","last_unstale":"2024-09-21T07:47:55.170071+0000","last_undegraded":"2024-09-21T07:47:55.170071+0000","last_fullsized":"2024-09-21T07:47:55.170071+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:49.829974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356463,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172012+0000","last_change":"2024-09-21T07:47:55.190247+0000","last_active":"2024-09-21T07:47:56.172012+0000","last_peered":"2024-09-21T07:47:56.172012+0000","last_clean":"2024-09-21T07:47:56.172012+0000","last_became_active":"2024-09-21T07:47:55.189871+0000","last_became_peered":"2024-09-21T07:47:55.189871+0000","last_unstale":"2024-09-21T07:47:56.172012+0000","last_undegraded":"2024-09-21T07:47:56.172012+0000","last_fullsized":"2024-09-21T07:47:56.172012+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:59:45.198861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"205'1","reported_seq":16,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.173115+0000","last_change":"2024-09-21T07:47:54.180420+0000","last_active":"2024-09-21T07:47:56.173115+0000","last_peered":"2024-09-21T07:47:56.173115+0000","last_clean":"2024-09-21T07:47:56.173115+0000","last_became_active":"2024-09-21T07:47:54.180041+0000","last_became_peered":"2024-09-21T07:47:54.180041+0000","last_unstale":"2024-09-21T07:47:56.173115+0000","last_undegraded":"2024-09-21T07:47:56.173115+0000","last_fullsized":"2024-09-21T07:47:56.173115+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:17:42.163288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":275,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.184006+0000","last_change":"2024-09-21T07:35:27.868448+0000","last_active":"2024-09-21T07:47:55.184006+0000","last_peered":"2024-09-21T07:47:55.184006+0000","last_clean":"2024-09-21T07:47:55.184006+0000","last_became_active":"2024-09-21T07:35:27.868105+0000","last_became_peered":"2024-09-21T07:35:27.868105+0000","last_unstale":"2024-09-21T07:47:55.184006+0000","last_undegraded":"2024-09-21T07:47:55.184006+0000","last_fullsized":"2024-09-21T07:47:55.184006+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:41.468702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":287,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.179291+0000","last_change":"2024-09-21T07:35:29.492974+0000","last_active":"2024-09-21T07:47:55.179291+0000","last_peered":"2024-09-21T07:47:55.179291+0000","last_clean":"2024-09-21T07:47:55.179291+0000","last_became_active":"2024-09-21T07:35:26.872165+0000","last_became_peered":"2024-09-21T07:35:26.872165+0000","last_unstale":"2024-09-21T07:47:55.179291+0000","last_undegraded":"2024-09-21T07:47:55.179291+0000","last_fullsized":"2024-09-21T07:47:55.179291+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:59.912520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013795179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":337,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.178726+0000","last_change":"2024-09-21T07:34:25.084861+0000","last_active":"2024-09-21T07:47:55.178726+0000","last_peered":"2024-09-21T07:47:55.178726+0000","last_clean":"2024-09-21T07:47:55.178726+0000","last_became_active":"2024-09-21T07:34:23.277708+0000","last_became_peered":"2024-09-21T07:34:23.277708+0000","last_unstale":"2024-09-21T07:47:55.178726+0000","last_undegraded":"2024-09-21T07:47:55.178726+0000","last_fullsized":"2024-09-21T07:47:55.178726+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:40.816488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000597566,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.192904+0000","last_change":"2024-09-21T07:47:55.188872+0000","last_active":"2024-09-21T07:47:55.192904+0000","last_peered":"2024-09-21T07:47:55.192904+0000","last_clean":"2024-09-21T07:47:55.192904+0000","last_became_active":"2024-09-21T07:47:55.188550+0000","last_became_peered":"2024-09-21T07:47:55.188550+0000","last_unstale":"2024-09-21T07:47:55.192904+0000","last_undegraded":"2024-09-21T07:47:55.192904+0000","last_fullsized":"2024-09-21T07:47:55.192904+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:18.496098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"205'2","reported_seq":15,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.208654+0000","last_change":"2024-09-21T07:47:54.181899+0000","last_active":"2024-09-21T07:47:55.208654+0000","last_peered":"2024-09-21T07:47:55.208654+0000","last_clean":"2024-09-21T07:47:55.208654+0000","last_became_active":"2024-09-21T07:47:54.181590+0000","last_became_peered":"2024-09-21T07:47:54.181590+0000","last_unstale":"2024-09-21T07:47:55.208654+0000","last_undegraded":"2024-09-21T07:47:55.208654+0000","last_fullsized":"2024-09-21T07:47:55.208654+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:41.106267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":275,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.185294+0000","last_change":"2024-09-21T07:35:27.874640+0000","last_active":"2024-09-21T07:47:55.185294+0000","last_peered":"2024-09-21T07:47:55.185294+0000","last_clean":"2024-09-21T07:47:55.185294+0000","last_became_active":"2024-09-21T07:35:27.874265+0000","last_became_peered":"2024-09-21T07:35:27.874265+0000","last_unstale":"2024-09-21T07:47:55.185294+0000","last_undegraded":"2024-09-21T07:47:55.185294+0000","last_fullsized":"2024-09-21T07:47:55.185294+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:45.754654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":288,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.179985+0000","last_change":"2024-09-21T07:35:29.207392+0000","last_active":"2024-09-21T07:47:55.179985+0000","last_peered":"2024-09-21T07:47:55.179985+0000","last_clean":"2024-09-21T07:47:55.179985+0000","last_became_active":"2024-09-21T07:35:26.870917+0000","last_became_peered":"2024-09-21T07:35:26.870917+0000","last_unstale":"2024-09-21T07:47:55.179985+0000","last_undegraded":"2024-09-21T07:47:55.179985+0000","last_fullsized":"2024-09-21T07:47:55.179985+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:59.159937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059198499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":344,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.165558+0000","last_change":"2024-09-21T07:34:24.258709+0000","last_active":"2024-09-21T07:47:55.165558+0000","last_peered":"2024-09-21T07:47:55.165558+0000","last_clean":"2024-09-21T07:47:55.165558+0000","last_became_active":"2024-09-21T07:34:23.279042+0000","last_became_peered":"2024-09-21T07:34:23.279042+0000","last_unstale":"2024-09-21T07:47:55.165558+0000","last_undegraded":"2024-09-21T07:47:55.165558+0000","last_fullsized":"2024-09-21T07:47:55.165558+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:01.762117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019282100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.189457+0000","last_change":"2024-09-21T07:47:55.187859+0000","last_active":"2024-09-21T07:47:55.189457+0000","last_peered":"2024-09-21T07:47:55.189457+0000","last_clean":"2024-09-21T07:47:55.189457+0000","last_became_active":"2024-09-21T07:47:55.187572+0000","last_became_peered":"2024-09-21T07:47:55.187572+0000","last_unstale":"2024-09-21T07:47:55.189457+0000","last_undegraded":"2024-09-21T07:47:55.189457+0000","last_fullsized":"2024-09-21T07:47:55.189457+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:26.667066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":15,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.171707+0000","last_change":"2024-09-21T07:47:54.187585+0000","last_active":"2024-09-21T07:47:56.171707+0000","last_peered":"2024-09-21T07:47:56.171707+0000","last_clean":"2024-09-21T07:47:56.171707+0000","last_became_active":"2024-09-21T07:47:54.187063+0000","last_became_peered":"2024-09-21T07:47:54.187063+0000","last_unstale":"2024-09-21T07:47:56.171707+0000","last_undegraded":"2024-09-21T07:47:56.171707+0000","last_fullsized":"2024-09-21T07:47:56.171707+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:15.785391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":279,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.190091+0000","last_change":"2024-09-21T07:35:27.870260+0000","last_active":"2024-09-21T07:47:55.190091+0000","last_peered":"2024-09-21T07:47:55.190091+0000","last_clean":"2024-09-21T07:47:55.190091+0000","last_became_active":"2024-09-21T07:35:27.869448+0000","last_became_peered":"2024-09-21T07:35:27.869448+0000","last_unstale":"2024-09-21T07:47:55.190091+0000","last_undegraded":"2024-09-21T07:47:55.190091+0000","last_fullsized":"2024-09-21T07:47:55.190091+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:16.946790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":283,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.184959+0000","last_change":"2024-09-21T07:35:29.386367+0000","last_active":"2024-09-21T07:47:55.184959+0000","last_peered":"2024-09-21T07:47:55.184959+0000","last_clean":"2024-09-21T07:47:55.184959+0000","last_became_active":"2024-09-21T07:35:26.879238+0000","last_became_peered":"2024-09-21T07:35:26.879238+0000","last_unstale":"2024-09-21T07:47:55.184959+0000","last_undegraded":"2024-09-21T07:47:55.184959+0000","last_fullsized":"2024-09-21T07:47:55.184959+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:16.528260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092223100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":337,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.195355+0000","last_change":"2024-09-21T07:34:25.085635+0000","last_active":"2024-09-21T07:47:55.195355+0000","last_peered":"2024-09-21T07:47:55.195355+0000","last_clean":"2024-09-21T07:47:55.195355+0000","last_became_active":"2024-09-21T07:34:23.273566+0000","last_became_peered":"2024-09-21T07:34:23.273566+0000","last_unstale":"2024-09-21T07:47:55.195355+0000","last_undegraded":"2024-09-21T07:47:55.195355+0000","last_fullsized":"2024-09-21T07:47:55.195355+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:14.142272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079897300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.187707+0000","last_change":"2024-09-21T07:47:55.182577+0000","last_active":"2024-09-21T07:47:55.187707+0000","last_peered":"2024-09-21T07:47:55.187707+0000","last_clean":"2024-09-21T07:47:55.187707+0000","last_became_active":"2024-09-21T07:47:55.180283+0000","last_became_peered":"2024-09-21T07:47:55.180283+0000","last_unstale":"2024-09-21T07:47:55.187707+0000","last_undegraded":"2024-09-21T07:47:55.187707+0000","last_fullsized":"2024-09-21T07:47:55.187707+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:40.604618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":13,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.178451+0000","last_change":"2024-09-21T07:47:54.185311+0000","last_active":"2024-09-21T07:47:55.178451+0000","last_peered":"2024-09-21T07:47:55.178451+0000","last_clean":"2024-09-21T07:47:55.178451+0000","last_became_active":"2024-09-21T07:47:54.184612+0000","last_became_peered":"2024-09-21T07:47:54.184612+0000","last_unstale":"2024-09-21T07:47:55.178451+0000","last_undegraded":"2024-09-21T07:47:55.178451+0000","last_fullsized":"2024-09-21T07:47:55.178451+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:51:48.140187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":279,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.191876+0000","last_change":"2024-09-21T07:35:27.878728+0000","last_active":"2024-09-21T07:47:55.191876+0000","last_peered":"2024-09-21T07:47:55.191876+0000","last_clean":"2024-09-21T07:47:55.191876+0000","last_became_active":"2024-09-21T07:35:27.878335+0000","last_became_peered":"2024-09-21T07:35:27.878335+0000","last_unstale":"2024-09-21T07:47:55.191876+0000","last_undegraded":"2024-09-21T07:47:55.191876+0000","last_fullsized":"2024-09-21T07:47:55.191876+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:25:37.731771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":300,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.171927+0000","last_change":"2024-09-21T07:35:28.858418+0000","last_active":"2024-09-21T07:47:56.171927+0000","last_peered":"2024-09-21T07:47:56.171927+0000","last_clean":"2024-09-21T07:47:56.171927+0000","last_became_active":"2024-09-21T07:35:26.876944+0000","last_became_peered":"2024-09-21T07:35:26.876944+0000","last_unstale":"2024-09-21T07:47:56.171927+0000","last_undegraded":"2024-09-21T07:47:56.171927+0000","last_fullsized":"2024-09-21T07:47:56.171927+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:04.514304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090862799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":332,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.185850+0000","last_change":"2024-09-21T07:34:24.597926+0000","last_active":"2024-09-21T07:47:55.185850+0000","last_peered":"2024-09-21T07:47:55.185850+0000","last_clean":"2024-09-21T07:47:55.185850+0000","last_became_active":"2024-09-21T07:34:23.268926+0000","last_became_peered":"2024-09-21T07:34:23.268926+0000","last_unstale":"2024-09-21T07:47:55.185850+0000","last_undegraded":"2024-09-21T07:47:55.185850+0000","last_fullsized":"2024-09-21T07:47:55.185850+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:46.190752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101851,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.168546+0000","last_change":"2024-09-21T07:47:55.192828+0000","last_active":"2024-09-21T07:47:56.168546+0000","last_peered":"2024-09-21T07:47:56.168546+0000","last_clean":"2024-09-21T07:47:56.168546+0000","last_became_active":"2024-09-21T07:47:55.192534+0000","last_became_peered":"2024-09-21T07:47:55.192534+0000","last_unstale":"2024-09-21T07:47:56.168546+0000","last_undegraded":"2024-09-21T07:47:56.168546+0000","last_fullsized":"2024-09-21T07:47:56.168546+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:44:08.148357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"205'2","reported_seq":15,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.206365+0000","last_change":"2024-09-21T07:47:54.190402+0000","last_active":"2024-09-21T07:47:55.206365+0000","last_peered":"2024-09-21T07:47:55.206365+0000","last_clean":"2024-09-21T07:47:55.206365+0000","last_became_active":"2024-09-21T07:47:54.190057+0000","last_became_peered":"2024-09-21T07:47:54.190057+0000","last_unstale":"2024-09-21T07:47:55.206365+0000","last_undegraded":"2024-09-21T07:47:55.206365+0000","last_fullsized":"2024-09-21T07:47:55.206365+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:10.894204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":277,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.186287+0000","last_change":"2024-09-21T07:35:27.877426+0000","last_active":"2024-09-21T07:47:55.186287+0000","last_peered":"2024-09-21T07:47:55.186287+0000","last_clean":"2024-09-21T07:47:55.186287+0000","last_became_active":"2024-09-21T07:35:27.877050+0000","last_became_peered":"2024-09-21T07:35:27.877050+0000","last_unstale":"2024-09-21T07:47:55.186287+0000","last_undegraded":"2024-09-21T07:47:55.186287+0000","last_fullsized":"2024-09-21T07:47:55.186287+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:36.268334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":286,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.178267+0000","last_change":"2024-09-21T07:35:28.862273+0000","last_active":"2024-09-21T07:47:55.178267+0000","last_peered":"2024-09-21T07:47:55.178267+0000","last_clean":"2024-09-21T07:47:55.178267+0000","last_became_active":"2024-09-21T07:35:26.874011+0000","last_became_peered":"2024-09-21T07:35:26.874011+0000","last_unstale":"2024-09-21T07:47:55.178267+0000","last_undegraded":"2024-09-21T07:47:55.178267+0000","last_fullsized":"2024-09-21T07:47:55.178267+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:11.986950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071644799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":344,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172240+0000","last_change":"2024-09-21T07:34:24.259794+0000","last_active":"2024-09-21T07:47:56.172240+0000","last_peered":"2024-09-21T07:47:56.172240+0000","last_clean":"2024-09-21T07:47:56.172240+0000","last_became_active":"2024-09-21T07:34:23.265679+0000","last_became_peered":"2024-09-21T07:34:23.265679+0000","last_unstale":"2024-09-21T07:47:56.172240+0000","last_undegraded":"2024-09-21T07:47:56.172240+0000","last_fullsized":"2024-09-21T07:47:56.172240+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:02.575796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001298518,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.204175+0000","last_change":"2024-09-21T07:47:55.195673+0000","last_active":"2024-09-21T07:47:55.204175+0000","last_peered":"2024-09-21T07:47:55.204175+0000","last_clean":"2024-09-21T07:47:55.204175+0000","last_became_active":"2024-09-21T07:47:55.195332+0000","last_became_peered":"2024-09-21T07:47:55.195332+0000","last_unstale":"2024-09-21T07:47:55.204175+0000","last_undegraded":"2024-09-21T07:47:55.204175+0000","last_fullsized":"2024-09-21T07:47:55.204175+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:12.292785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":13,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.178931+0000","last_change":"2024-09-21T07:47:54.193515+0000","last_active":"2024-09-21T07:47:55.178931+0000","last_peered":"2024-09-21T07:47:55.178931+0000","last_clean":"2024-09-21T07:47:55.178931+0000","last_became_active":"2024-09-21T07:47:54.193198+0000","last_became_peered":"2024-09-21T07:47:54.193198+0000","last_unstale":"2024-09-21T07:47:55.178931+0000","last_undegraded":"2024-09-21T07:47:55.178931+0000","last_fullsized":"2024-09-21T07:47:55.178931+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:29:13.611283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":282,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.180198+0000","last_change":"2024-09-21T07:35:27.877725+0000","last_active":"2024-09-21T07:47:55.180198+0000","last_peered":"2024-09-21T07:47:55.180198+0000","last_clean":"2024-09-21T07:47:55.180198+0000","last_became_active":"2024-09-21T07:35:27.877398+0000","last_became_peered":"2024-09-21T07:35:27.877398+0000","last_unstale":"2024-09-21T07:47:55.180198+0000","last_undegraded":"2024-09-21T07:47:55.180198+0000","last_fullsized":"2024-09-21T07:47:55.180198+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:05.859762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":292,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.171363+0000","last_change":"2024-09-21T07:35:28.860612+0000","last_active":"2024-09-21T07:47:56.171363+0000","last_peered":"2024-09-21T07:47:56.171363+0000","last_clean":"2024-09-21T07:47:56.171363+0000","last_became_active":"2024-09-21T07:35:26.876562+0000","last_became_peered":"2024-09-21T07:35:26.876562+0000","last_unstale":"2024-09-21T07:47:56.171363+0000","last_undegraded":"2024-09-21T07:47:56.171363+0000","last_fullsized":"2024-09-21T07:47:56.171363+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:06.824331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012252470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":340,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.184348+0000","last_change":"2024-09-21T07:34:24.260330+0000","last_active":"2024-09-21T07:47:55.184348+0000","last_peered":"2024-09-21T07:47:55.184348+0000","last_clean":"2024-09-21T07:47:55.184348+0000","last_became_active":"2024-09-21T07:34:23.637210+0000","last_became_peered":"2024-09-21T07:34:23.637210+0000","last_unstale":"2024-09-21T07:47:55.184348+0000","last_undegraded":"2024-09-21T07:47:55.184348+0000","last_fullsized":"2024-09-21T07:47:55.184348+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.488451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001537093,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.204587+0000","last_change":"2024-09-21T07:47:55.196976+0000","last_active":"2024-09-21T07:47:55.204587+0000","last_peered":"2024-09-21T07:47:55.204587+0000","last_clean":"2024-09-21T07:47:55.204587+0000","last_became_active":"2024-09-21T07:47:55.196562+0000","last_became_peered":"2024-09-21T07:47:55.196562+0000","last_unstale":"2024-09-21T07:47:55.204587+0000","last_undegraded":"2024-09-21T07:47:55.204587+0000","last_fullsized":"2024-09-21T07:47:55.204587+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:26:38.838493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":13,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.167049+0000","last_change":"2024-09-21T07:47:54.191403+0000","last_active":"2024-09-21T07:47:55.167049+0000","last_peered":"2024-09-21T07:47:55.167049+0000","last_clean":"2024-09-21T07:47:55.167049+0000","last_became_active":"2024-09-21T07:47:54.191082+0000","last_became_peered":"2024-09-21T07:47:54.191082+0000","last_unstale":"2024-09-21T07:47:55.167049+0000","last_undegraded":"2024-09-21T07:47:55.167049+0000","last_fullsized":"2024-09-21T07:47:55.167049+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:47.109877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":279,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.178433+0000","last_change":"2024-09-21T07:35:27.874757+0000","last_active":"2024-09-21T07:47:55.178433+0000","last_peered":"2024-09-21T07:47:55.178433+0000","last_clean":"2024-09-21T07:47:55.178433+0000","last_became_active":"2024-09-21T07:35:27.874360+0000","last_became_peered":"2024-09-21T07:35:27.874360+0000","last_unstale":"2024-09-21T07:47:55.178433+0000","last_undegraded":"2024-09-21T07:47:55.178433+0000","last_fullsized":"2024-09-21T07:47:55.178433+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:17.646912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":287,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.176957+0000","last_change":"2024-09-21T07:35:29.493431+0000","last_active":"2024-09-21T07:47:55.176957+0000","last_peered":"2024-09-21T07:47:55.176957+0000","last_clean":"2024-09-21T07:47:55.176957+0000","last_became_active":"2024-09-21T07:35:26.873691+0000","last_became_peered":"2024-09-21T07:35:26.873691+0000","last_unstale":"2024-09-21T07:47:55.176957+0000","last_undegraded":"2024-09-21T07:47:55.176957+0000","last_fullsized":"2024-09-21T07:47:55.176957+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:51.705051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001162464,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":332,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.184255+0000","last_change":"2024-09-21T07:34:24.597286+0000","last_active":"2024-09-21T07:47:55.184255+0000","last_peered":"2024-09-21T07:47:55.184255+0000","last_clean":"2024-09-21T07:47:55.184255+0000","last_became_active":"2024-09-21T07:34:23.267133+0000","last_became_peered":"2024-09-21T07:34:23.267133+0000","last_unstale":"2024-09-21T07:47:55.184255+0000","last_undegraded":"2024-09-21T07:47:55.184255+0000","last_fullsized":"2024-09-21T07:47:55.184255+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:18.257010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172530+0000","last_change":"2024-09-21T07:47:55.175551+0000","last_active":"2024-09-21T07:47:56.172530+0000","last_peered":"2024-09-21T07:47:56.172530+0000","last_clean":"2024-09-21T07:47:56.172530+0000","last_became_active":"2024-09-21T07:47:55.175221+0000","last_became_peered":"2024-09-21T07:47:55.175221+0000","last_unstale":"2024-09-21T07:47:56.172530+0000","last_undegraded":"2024-09-21T07:47:56.172530+0000","last_fullsized":"2024-09-21T07:47:56.172530+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:34.371499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":13,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.170091+0000","last_change":"2024-09-21T07:47:54.194068+0000","last_active":"2024-09-21T07:47:55.170091+0000","last_peered":"2024-09-21T07:47:55.170091+0000","last_clean":"2024-09-21T07:47:55.170091+0000","last_became_active":"2024-09-21T07:47:54.193741+0000","last_became_peered":"2024-09-21T07:47:54.193741+0000","last_unstale":"2024-09-21T07:47:55.170091+0000","last_undegraded":"2024-09-21T07:47:55.170091+0000","last_fullsized":"2024-09-21T07:47:55.170091+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:34:42.075929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":293,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.171664+0000","last_change":"2024-09-21T07:35:27.864458+0000","last_active":"2024-09-21T07:47:56.171664+0000","last_peered":"2024-09-21T07:47:56.171664+0000","last_clean":"2024-09-21T07:47:56.171664+0000","last_became_active":"2024-09-21T07:35:27.864136+0000","last_became_peered":"2024-09-21T07:35:27.864136+0000","last_unstale":"2024-09-21T07:47:56.171664+0000","last_undegraded":"2024-09-21T07:47:56.171664+0000","last_fullsized":"2024-09-21T07:47:56.171664+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:20.162240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":300,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172911+0000","last_change":"2024-09-21T07:35:28.858784+0000","last_active":"2024-09-21T07:47:56.172911+0000","last_peered":"2024-09-21T07:47:56.172911+0000","last_clean":"2024-09-21T07:47:56.172911+0000","last_became_active":"2024-09-21T07:35:26.876890+0000","last_became_peered":"2024-09-21T07:35:26.876890+0000","last_unstale":"2024-09-21T07:47:56.172911+0000","last_undegraded":"2024-09-21T07:47:56.172911+0000","last_fullsized":"2024-09-21T07:47:56.172911+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:40.738327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064543499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":344,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.169957+0000","last_change":"2024-09-21T07:34:24.258712+0000","last_active":"2024-09-21T07:47:55.169957+0000","last_peered":"2024-09-21T07:47:55.169957+0000","last_clean":"2024-09-21T07:47:55.169957+0000","last_became_active":"2024-09-21T07:34:23.274692+0000","last_became_peered":"2024-09-21T07:34:23.274692+0000","last_unstale":"2024-09-21T07:47:55.169957+0000","last_undegraded":"2024-09-21T07:47:55.169957+0000","last_fullsized":"2024-09-21T07:47:55.169957+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:43.279454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018051479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172147+0000","last_change":"2024-09-21T07:47:55.173038+0000","last_active":"2024-09-21T07:47:56.172147+0000","last_peered":"2024-09-21T07:47:56.172147+0000","last_clean":"2024-09-21T07:47:56.172147+0000","last_became_active":"2024-09-21T07:47:55.172601+0000","last_became_peered":"2024-09-21T07:47:55.172601+0000","last_unstale":"2024-09-21T07:47:56.172147+0000","last_undegraded":"2024-09-21T07:47:56.172147+0000","last_fullsized":"2024-09-21T07:47:56.172147+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:24:08.769397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"205'2","reported_seq":15,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.208297+0000","last_change":"2024-09-21T07:47:54.190101+0000","last_active":"2024-09-21T07:47:55.208297+0000","last_peered":"2024-09-21T07:47:55.208297+0000","last_clean":"2024-09-21T07:47:55.208297+0000","last_became_active":"2024-09-21T07:47:54.189757+0000","last_became_peered":"2024-09-21T07:47:54.189757+0000","last_unstale":"2024-09-21T07:47:55.208297+0000","last_undegraded":"2024-09-21T07:47:55.208297+0000","last_fullsized":"2024-09-21T07:47:55.208297+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:47.948529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":283,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.166853+0000","last_change":"2024-09-21T07:35:27.878020+0000","last_active":"2024-09-21T07:47:55.166853+0000","last_peered":"2024-09-21T07:47:55.166853+0000","last_clean":"2024-09-21T07:47:55.166853+0000","last_became_active":"2024-09-21T07:35:27.877532+0000","last_became_peered":"2024-09-21T07:35:27.877532+0000","last_unstale":"2024-09-21T07:47:55.166853+0000","last_undegraded":"2024-09-21T07:47:55.166853+0000","last_fullsized":"2024-09-21T07:47:55.166853+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:19.756073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":292,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172277+0000","last_change":"2024-09-21T07:35:28.861049+0000","last_active":"2024-09-21T07:47:56.172277+0000","last_peered":"2024-09-21T07:47:56.172277+0000","last_clean":"2024-09-21T07:47:56.172277+0000","last_became_active":"2024-09-21T07:35:26.876819+0000","last_became_peered":"2024-09-21T07:35:26.876819+0000","last_unstale":"2024-09-21T07:47:56.172277+0000","last_undegraded":"2024-09-21T07:47:56.172277+0000","last_fullsized":"2024-09-21T07:47:56.172277+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:19.633803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055780599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":337,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.178544+0000","last_change":"2024-09-21T07:34:25.084849+0000","last_active":"2024-09-21T07:47:55.178544+0000","last_peered":"2024-09-21T07:47:55.178544+0000","last_clean":"2024-09-21T07:47:55.178544+0000","last_became_active":"2024-09-21T07:34:23.277713+0000","last_became_peered":"2024-09-21T07:34:23.277713+0000","last_unstale":"2024-09-21T07:47:55.178544+0000","last_undegraded":"2024-09-21T07:47:55.178544+0000","last_fullsized":"2024-09-21T07:47:55.178544+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:05.811810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059079499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.169971+0000","last_change":"2024-09-21T07:47:55.184966+0000","last_active":"2024-09-21T07:47:56.169971+0000","last_peered":"2024-09-21T07:47:56.169971+0000","last_clean":"2024-09-21T07:47:56.169971+0000","last_became_active":"2024-09-21T07:47:55.184645+0000","last_became_peered":"2024-09-21T07:47:55.184645+0000","last_unstale":"2024-09-21T07:47:56.169971+0000","last_undegraded":"2024-09-21T07:47:56.169971+0000","last_fullsized":"2024-09-21T07:47:56.169971+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:26:35.775546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"206'1","reported_seq":16,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:57.171463+0000","last_change":"2024-09-21T07:47:54.198999+0000","last_active":"2024-09-21T07:47:57.171463+0000","last_peered":"2024-09-21T07:47:57.171463+0000","last_clean":"2024-09-21T07:47:57.171463+0000","last_became_active":"2024-09-21T07:47:54.198632+0000","last_became_peered":"2024-09-21T07:47:54.198632+0000","last_unstale":"2024-09-21T07:47:57.171463+0000","last_undegraded":"2024-09-21T07:47:57.171463+0000","last_fullsized":"2024-09-21T07:47:57.171463+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:10.747530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":284,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.169730+0000","last_change":"2024-09-21T07:35:27.884248+0000","last_active":"2024-09-21T07:47:56.169730+0000","last_peered":"2024-09-21T07:47:56.169730+0000","last_clean":"2024-09-21T07:47:56.169730+0000","last_became_active":"2024-09-21T07:35:27.883861+0000","last_became_peered":"2024-09-21T07:35:27.883861+0000","last_unstale":"2024-09-21T07:47:56.169730+0000","last_undegraded":"2024-09-21T07:47:56.169730+0000","last_fullsized":"2024-09-21T07:47:56.169730+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":288,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.179827+0000","last_change":"2024-09-21T07:35:29.207135+0000","last_active":"2024-09-21T07:47:55.179827+0000","last_peered":"2024-09-21T07:47:55.179827+0000","last_clean":"2024-09-21T07:47:55.179827+0000","last_became_active":"2024-09-21T07:35:26.872787+0000","last_became_peered":"2024-09-21T07:35:26.872787+0000","last_unstale":"2024-09-21T07:47:55.179827+0000","last_undegraded":"2024-09-21T07:47:55.179827+0000","last_fullsized":"2024-09-21T07:47:55.179827+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:27.543217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011157509999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":352,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.171742+0000","last_change":"2024-09-21T07:34:24.538610+0000","last_active":"2024-09-21T07:47:56.171742+0000","last_peered":"2024-09-21T07:47:56.171742+0000","last_clean":"2024-09-21T07:47:56.171742+0000","last_became_active":"2024-09-21T07:34:23.259700+0000","last_became_peered":"2024-09-21T07:34:23.259700+0000","last_unstale":"2024-09-21T07:47:56.171742+0000","last_undegraded":"2024-09-21T07:47:56.171742+0000","last_fullsized":"2024-09-21T07:47:56.171742+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:57.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070522399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.169679+0000","last_change":"2024-09-21T07:47:55.202961+0000","last_active":"2024-09-21T07:47:56.169679+0000","last_peered":"2024-09-21T07:47:56.169679+0000","last_clean":"2024-09-21T07:47:56.169679+0000","last_became_active":"2024-09-21T07:47:55.202640+0000","last_became_peered":"2024-09-21T07:47:55.202640+0000","last_unstale":"2024-09-21T07:47:56.169679+0000","last_undegraded":"2024-09-21T07:47:56.169679+0000","last_fullsized":"2024-09-21T07:47:56.169679+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:41:07.382956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":15,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.171751+0000","last_change":"2024-09-21T07:47:54.191694+0000","last_active":"2024-09-21T07:47:56.171751+0000","last_peered":"2024-09-21T07:47:56.171751+0000","last_clean":"2024-09-21T07:47:56.171751+0000","last_became_active":"2024-09-21T07:47:54.191392+0000","last_became_peered":"2024-09-21T07:47:54.191392+0000","last_unstale":"2024-09-21T07:47:56.171751+0000","last_undegraded":"2024-09-21T07:47:56.171751+0000","last_fullsized":"2024-09-21T07:47:56.171751+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:39.417382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":293,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172466+0000","last_change":"2024-09-21T07:35:27.868024+0000","last_active":"2024-09-21T07:47:56.172466+0000","last_peered":"2024-09-21T07:47:56.172466+0000","last_clean":"2024-09-21T07:47:56.172466+0000","last_became_active":"2024-09-21T07:35:27.867339+0000","last_became_peered":"2024-09-21T07:35:27.867339+0000","last_unstale":"2024-09-21T07:47:56.172466+0000","last_undegraded":"2024-09-21T07:47:56.172466+0000","last_fullsized":"2024-09-21T07:47:56.172466+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":300,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172031+0000","last_change":"2024-09-21T07:35:28.858505+0000","last_active":"2024-09-21T07:47:56.172031+0000","last_peered":"2024-09-21T07:47:56.172031+0000","last_clean":"2024-09-21T07:47:56.172031+0000","last_became_active":"2024-09-21T07:35:26.877320+0000","last_became_peered":"2024-09-21T07:35:26.877320+0000","last_unstale":"2024-09-21T07:47:56.172031+0000","last_undegraded":"2024-09-21T07:47:56.172031+0000","last_fullsized":"2024-09-21T07:47:56.172031+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:53.672782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244119999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":333,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.188231+0000","last_change":"2024-09-21T07:34:24.597597+0000","last_active":"2024-09-21T07:47:55.188231+0000","last_peered":"2024-09-21T07:47:55.188231+0000","last_clean":"2024-09-21T07:47:55.188231+0000","last_became_active":"2024-09-21T07:34:23.635204+0000","last_became_peered":"2024-09-21T07:34:23.635204+0000","last_unstale":"2024-09-21T07:47:55.188231+0000","last_undegraded":"2024-09-21T07:47:55.188231+0000","last_fullsized":"2024-09-21T07:47:55.188231+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:48.644789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081139199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.205770+0000","last_change":"2024-09-21T07:47:55.202438+0000","last_active":"2024-09-21T07:47:55.205770+0000","last_peered":"2024-09-21T07:47:55.205770+0000","last_clean":"2024-09-21T07:47:55.205770+0000","last_became_active":"2024-09-21T07:47:55.201888+0000","last_became_peered":"2024-09-21T07:47:55.201888+0000","last_unstale":"2024-09-21T07:47:55.205770+0000","last_undegraded":"2024-09-21T07:47:55.205770+0000","last_fullsized":"2024-09-21T07:47:55.205770+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:59.420037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":13,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.178702+0000","last_change":"2024-09-21T07:47:54.193368+0000","last_active":"2024-09-21T07:47:55.178702+0000","last_peered":"2024-09-21T07:47:55.178702+0000","last_clean":"2024-09-21T07:47:55.178702+0000","last_became_active":"2024-09-21T07:47:54.192921+0000","last_became_peered":"2024-09-21T07:47:54.192921+0000","last_unstale":"2024-09-21T07:47:55.178702+0000","last_undegraded":"2024-09-21T07:47:55.178702+0000","last_fullsized":"2024-09-21T07:47:55.178702+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:05:07.952836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":281,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.179610+0000","last_change":"2024-09-21T07:35:27.877503+0000","last_active":"2024-09-21T07:47:55.179610+0000","last_peered":"2024-09-21T07:47:55.179610+0000","last_clean":"2024-09-21T07:47:55.179610+0000","last_became_active":"2024-09-21T07:35:27.875775+0000","last_became_peered":"2024-09-21T07:35:27.875775+0000","last_unstale":"2024-09-21T07:47:55.179610+0000","last_undegraded":"2024-09-21T07:47:55.179610+0000","last_fullsized":"2024-09-21T07:47:55.179610+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:49.291023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":287,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.175627+0000","last_change":"2024-09-21T07:35:29.491412+0000","last_active":"2024-09-21T07:47:55.175627+0000","last_peered":"2024-09-21T07:47:55.175627+0000","last_clean":"2024-09-21T07:47:55.175627+0000","last_became_active":"2024-09-21T07:35:26.876404+0000","last_became_peered":"2024-09-21T07:35:26.876404+0000","last_unstale":"2024-09-21T07:47:55.175627+0000","last_undegraded":"2024-09-21T07:47:55.175627+0000","last_fullsized":"2024-09-21T07:47:55.175627+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063918000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":345,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.166213+0000","last_change":"2024-09-21T07:34:24.259026+0000","last_active":"2024-09-21T07:47:55.166213+0000","last_peered":"2024-09-21T07:47:55.166213+0000","last_clean":"2024-09-21T07:47:55.166213+0000","last_became_active":"2024-09-21T07:34:23.635456+0000","last_became_peered":"2024-09-21T07:34:23.635456+0000","last_unstale":"2024-09-21T07:47:55.166213+0000","last_undegraded":"2024-09-21T07:47:55.166213+0000","last_fullsized":"2024-09-21T07:47:55.166213+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:09.592752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018654419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.196848+0000","last_change":"2024-09-21T07:47:55.191605+0000","last_active":"2024-09-21T07:47:55.196848+0000","last_peered":"2024-09-21T07:47:55.196848+0000","last_clean":"2024-09-21T07:47:55.196848+0000","last_became_active":"2024-09-21T07:47:55.188078+0000","last_became_peered":"2024-09-21T07:47:55.188078+0000","last_unstale":"2024-09-21T07:47:55.196848+0000","last_undegraded":"2024-09-21T07:47:55.196848+0000","last_fullsized":"2024-09-21T07:47:55.196848+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:41.382319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":13,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.176728+0000","last_change":"2024-09-21T07:47:54.190454+0000","last_active":"2024-09-21T07:47:55.176728+0000","last_peered":"2024-09-21T07:47:55.176728+0000","last_clean":"2024-09-21T07:47:55.176728+0000","last_became_active":"2024-09-21T07:47:54.189988+0000","last_became_peered":"2024-09-21T07:47:54.189988+0000","last_unstale":"2024-09-21T07:47:55.176728+0000","last_undegraded":"2024-09-21T07:47:55.176728+0000","last_fullsized":"2024-09-21T07:47:55.176728+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:06:43.125613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":281,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.179887+0000","last_change":"2024-09-21T07:35:27.871984+0000","last_active":"2024-09-21T07:47:55.179887+0000","last_peered":"2024-09-21T07:47:55.179887+0000","last_clean":"2024-09-21T07:47:55.179887+0000","last_became_active":"2024-09-21T07:35:27.870336+0000","last_became_peered":"2024-09-21T07:35:27.870336+0000","last_unstale":"2024-09-21T07:47:55.179887+0000","last_undegraded":"2024-09-21T07:47:55.179887+0000","last_fullsized":"2024-09-21T07:47:55.179887+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:57.120479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":292,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.171819+0000","last_change":"2024-09-21T07:35:28.860351+0000","last_active":"2024-09-21T07:47:56.171819+0000","last_peered":"2024-09-21T07:47:56.171819+0000","last_clean":"2024-09-21T07:47:56.171819+0000","last_became_active":"2024-09-21T07:35:26.872046+0000","last_became_peered":"2024-09-21T07:35:26.872046+0000","last_unstale":"2024-09-21T07:47:56.171819+0000","last_undegraded":"2024-09-21T07:47:56.171819+0000","last_fullsized":"2024-09-21T07:47:56.171819+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:47.074327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098384399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":340,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.182385+0000","last_change":"2024-09-21T07:34:24.260300+0000","last_active":"2024-09-21T07:47:55.182385+0000","last_peered":"2024-09-21T07:47:55.182385+0000","last_clean":"2024-09-21T07:47:55.182385+0000","last_became_active":"2024-09-21T07:34:23.270128+0000","last_became_peered":"2024-09-21T07:34:23.270128+0000","last_unstale":"2024-09-21T07:47:55.182385+0000","last_undegraded":"2024-09-21T07:47:55.182385+0000","last_fullsized":"2024-09-21T07:47:55.182385+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:25.260931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00176651,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.204913+0000","last_change":"2024-09-21T07:47:55.199347+0000","last_active":"2024-09-21T07:47:55.204913+0000","last_peered":"2024-09-21T07:47:55.204913+0000","last_clean":"2024-09-21T07:47:55.204913+0000","last_became_active":"2024-09-21T07:47:55.198641+0000","last_became_peered":"2024-09-21T07:47:55.198641+0000","last_unstale":"2024-09-21T07:47:55.204913+0000","last_undegraded":"2024-09-21T07:47:55.204913+0000","last_fullsized":"2024-09-21T07:47:55.204913+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:11:34.236907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":13,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.175545+0000","last_change":"2024-09-21T07:47:54.194255+0000","last_active":"2024-09-21T07:47:55.175545+0000","last_peered":"2024-09-21T07:47:55.175545+0000","last_clean":"2024-09-21T07:47:55.175545+0000","last_became_active":"2024-09-21T07:47:54.193776+0000","last_became_peered":"2024-09-21T07:47:54.193776+0000","last_unstale":"2024-09-21T07:47:55.175545+0000","last_undegraded":"2024-09-21T07:47:55.175545+0000","last_fullsized":"2024-09-21T07:47:55.175545+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:16:13.117235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":281,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.180262+0000","last_change":"2024-09-21T07:35:27.877861+0000","last_active":"2024-09-21T07:47:55.180262+0000","last_peered":"2024-09-21T07:47:55.180262+0000","last_clean":"2024-09-21T07:47:55.180262+0000","last_became_active":"2024-09-21T07:35:27.877434+0000","last_became_peered":"2024-09-21T07:35:27.877434+0000","last_unstale":"2024-09-21T07:47:55.180262+0000","last_undegraded":"2024-09-21T07:47:55.180262+0000","last_fullsized":"2024-09-21T07:47:55.180262+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":286,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.181321+0000","last_change":"2024-09-21T07:35:28.862954+0000","last_active":"2024-09-21T07:47:55.181321+0000","last_peered":"2024-09-21T07:47:55.181321+0000","last_clean":"2024-09-21T07:47:55.181321+0000","last_became_active":"2024-09-21T07:35:26.870613+0000","last_became_peered":"2024-09-21T07:35:26.870613+0000","last_unstale":"2024-09-21T07:47:55.181321+0000","last_undegraded":"2024-09-21T07:47:55.181321+0000","last_fullsized":"2024-09-21T07:47:55.181321+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:06.901043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059842100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":342,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.170048+0000","last_change":"2024-09-21T07:34:25.028904+0000","last_active":"2024-09-21T07:47:56.170048+0000","last_peered":"2024-09-21T07:47:56.170048+0000","last_clean":"2024-09-21T07:47:56.170048+0000","last_became_active":"2024-09-21T07:34:23.260942+0000","last_became_peered":"2024-09-21T07:34:23.260942+0000","last_unstale":"2024-09-21T07:47:56.170048+0000","last_undegraded":"2024-09-21T07:47:56.170048+0000","last_fullsized":"2024-09-21T07:47:56.170048+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:22.709328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060553899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509622,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66220,"kb_used_data":3096,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94305620,"statfs":{"total":96636764160,"available":96568954880,"internally_reserved":0,"allocated":3170304,"data_stored":1962474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509624,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51760,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320080,"statfs":{"total":96636764160,"available":96583761920,"internally_reserved":0,"allocated":2580480,"data_stored":1372176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509622,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33736,"kb_used_data":2544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338104,"statfs":{"total":96636764160,"available":96602218496,"internally_reserved":0,"allocated":2605056,"data_stored":1381291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509623,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29564,"kb_used_data":2532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342276,"statfs":{"total":96636764160,"available":96606490624,"internally_reserved":0,"allocated":2592768,"data_stored":1380239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509624,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46200,"kb_used_data":2528,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94325640,"statfs":{"total":96636764160,"available":96589455360,"internally_reserved":0,"allocated":2588672,"data_stored":1379657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509624,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52352,"kb_used_data":3112,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319488,"statfs":{"total":96636764160,"available":96583155712,"internally_reserved":0,"allocated":3186688,"data_stored":1963202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542327,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51772,"kb_used_data":2532,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320068,"statfs":{"total":96636764160,"available":96583749632,"internally_reserved":0,"allocated":2592768,"data_stored":1373752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542326,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73972,"kb_used_data":2524,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297868,"statfs":{"total":96636764160,"available":96561016832,"internally_reserved":0,"allocated":2584576,"data_stored":1372266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:47:59.699 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:47:59.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.958+0000 7f6081400640 1 -- 172.21.15.39:0/827182014 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f607c0b4f40 msgr2=0x7f607c151870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:59.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.958+0000 7f6081400640 1 --2- 172.21.15.39:0/827182014 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f607c0b4f40 0x7f607c151870 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f606c009830 tx=0x7f606c02a510 comp rx=0 tx=0).stop 2024-09-21T07:47:59.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.959+0000 7f6081400640 1 -- 172.21.15.39:0/827182014 shutdown_connections 2024-09-21T07:47:59.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.959+0000 7f6081400640 1 --2- 172.21.15.39:0/827182014 >> v2:172.21.15.148:3300/0 conn(0x7f607c15aeb0 0x7f607c15b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:59.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.959+0000 7f6081400640 1 --2- 172.21.15.39:0/827182014 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f607c0b4f40 0x7f607c151870 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:59.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.959+0000 7f6081400640 1 -- 172.21.15.39:0/827182014 >> 172.21.15.39:0/827182014 conn(0x7f607c06cfc0 msgr2=0x7f607c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:59.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.959+0000 7f6081400640 1 -- 172.21.15.39:0/827182014 shutdown_connections 2024-09-21T07:47:59.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.959+0000 7f6081400640 1 -- 172.21.15.39:0/827182014 wait complete. 2024-09-21T07:47:59.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.961+0000 7f6081400640 1 Processor -- start 2024-09-21T07:47:59.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.961+0000 7f6081400640 1 -- start start 2024-09-21T07:47:59.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.963+0000 7f6081400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607c0b4f40 0x7f607c1f8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:59.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.963+0000 7f6081400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f607c0d0600 0x7f607c1f5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:59.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.963+0000 7f6081400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f607c15aeb0 0x7f607c143820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:59.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.963+0000 7f6081400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f607c15e7e0 con 0x7f607c0d0600 2024-09-21T07:47:59.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.963+0000 7f6081400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f607c0710a0 con 0x7f607c0b4f40 2024-09-21T07:47:59.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.963+0000 7f6081400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f607c0cf550 con 0x7f607c15aeb0 2024-09-21T07:47:59.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.963+0000 7f6073e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f607c0d0600 0x7f607c1f5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:59.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.963+0000 7f6073e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f607c0d0600 0x7f607c1f5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53676/0 (socket says 172.21.15.39:53676) 2024-09-21T07:47:59.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.963+0000 7f6073e00640 1 -- 172.21.15.39:0/3535516721 learned_addr learned my addr 172.21.15.39:0/3535516721 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:47:59.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.963+0000 7f607be00640 1 --2- 172.21.15.39:0/3535516721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607c0b4f40 0x7f607c1f8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:59.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.963+0000 7f6080a00640 1 --2- 172.21.15.39:0/3535516721 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f607c15aeb0 0x7f607c143820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:59.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.964+0000 7f6073e00640 1 -- 172.21.15.39:0/3535516721 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f607c15aeb0 msgr2=0x7f607c143820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:59.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.964+0000 7f6073e00640 1 --2- 172.21.15.39:0/3535516721 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f607c15aeb0 0x7f607c143820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:59.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.964+0000 7f6073e00640 1 -- 172.21.15.39:0/3535516721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607c0b4f40 msgr2=0x7f607c1f8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:59.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.964+0000 7f6073e00640 1 --2- 172.21.15.39:0/3535516721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607c0b4f40 0x7f607c1f8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:59.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.964+0000 7f6073e00640 1 -- 172.21.15.39:0/3535516721 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f607c1f62f0 con 0x7f607c0d0600 2024-09-21T07:47:59.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.964+0000 7f6073e00640 1 --2- 172.21.15.39:0/3535516721 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f607c0d0600 0x7f607c1f5db0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f6064009800 tx=0x7f6064009cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:59.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.964+0000 7f6078c00640 1 -- 172.21.15.39:0/3535516721 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6064004230 con 0x7f607c0d0600 2024-09-21T07:47:59.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.964+0000 7f6078c00640 1 -- 172.21.15.39:0/3535516721 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60640044c0 con 0x7f607c0d0600 2024-09-21T07:47:59.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.964+0000 7f6081400640 1 -- 172.21.15.39:0/3535516721 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f607c143d60 con 0x7f607c0d0600 2024-09-21T07:47:59.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.964+0000 7f6078c00640 1 -- 172.21.15.39:0/3535516721 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606400e910 con 0x7f607c0d0600 2024-09-21T07:47:59.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.964+0000 7f6081400640 1 -- 172.21.15.39:0/3535516721 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f607c1f95c0 con 0x7f607c0d0600 2024-09-21T07:47:59.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.965+0000 7f6081400640 1 -- 172.21.15.39:0/3535516721 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f607c151870 con 0x7f607c0d0600 2024-09-21T07:47:59.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.967+0000 7f6078c00640 1 -- 172.21.15.39:0/3535516721 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f606400eab0 con 0x7f607c0d0600 2024-09-21T07:47:59.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.968+0000 7f6078c00640 1 --2- 172.21.15.39:0/3535516721 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6058078bc0 0x7f605807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:59.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.968+0000 7f6078c00640 1 -- 172.21.15.39:0/3535516721 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(206..206 src has 1..206) ==== 6960+0+0 (secure 0 0 0) 0x7f606409b3e0 con 0x7f607c0d0600 2024-09-21T07:47:59.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.968+0000 7f607be00640 1 --2- 172.21.15.39:0/3535516721 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6058078bc0 0x7f605807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:59.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.969+0000 7f607be00640 1 --2- 172.21.15.39:0/3535516721 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6058078bc0 0x7f605807b0d0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f606c034000 tx=0x7f606c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:59.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:47:59.972+0000 7f6078c00640 1 -- 172.21.15.39:0/3535516721 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6064022070 con 0x7f607c0d0600 2024-09-21T07:48:00.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.166+0000 7f6081400640 1 -- 172.21.15.39:0/3535516721 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f607c073ee0 con 0x7f6058078bc0 2024-09-21T07:48:00.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.175+0000 7f6078c00640 1 -- 172.21.15.39:0/3535516721 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+495244 (secure 0 0 0) 0x7f607c073ee0 con 0x7f6058078bc0 2024-09-21T07:48:00.183 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:48:00.186 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:48:00.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.181+0000 7f6081400640 1 -- 172.21.15.39:0/3535516721 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6058078bc0 msgr2=0x7f605807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:00.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.181+0000 7f6081400640 1 --2- 172.21.15.39:0/3535516721 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6058078bc0 0x7f605807b0d0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f606c034000 tx=0x7f606c004730 comp rx=0 tx=0).stop 2024-09-21T07:48:00.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.181+0000 7f6081400640 1 -- 172.21.15.39:0/3535516721 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f607c0d0600 msgr2=0x7f607c1f5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:00.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.181+0000 7f6081400640 1 --2- 172.21.15.39:0/3535516721 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f607c0d0600 0x7f607c1f5db0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f6064009800 tx=0x7f6064009cc0 comp rx=0 tx=0).stop 2024-09-21T07:48:00.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.181+0000 7f6081400640 1 -- 172.21.15.39:0/3535516721 shutdown_connections 2024-09-21T07:48:00.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.181+0000 7f6081400640 1 --2- 172.21.15.39:0/3535516721 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6058078bc0 0x7f605807b0d0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:00.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.181+0000 7f6081400640 1 --2- 172.21.15.39:0/3535516721 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f607c15aeb0 0x7f607c143820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:00.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.181+0000 7f6081400640 1 --2- 172.21.15.39:0/3535516721 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f607c0d0600 0x7f607c1f5db0 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:00.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.181+0000 7f6081400640 1 --2- 172.21.15.39:0/3535516721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607c0b4f40 0x7f607c1f8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:00.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.181+0000 7f6081400640 1 -- 172.21.15.39:0/3535516721 >> 172.21.15.39:0/3535516721 conn(0x7f607c06cfc0 msgr2=0x7f607c068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:00.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.181+0000 7f6081400640 1 -- 172.21.15.39:0/3535516721 shutdown_connections 2024-09-21T07:48:00.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.181+0000 7f6081400640 1 -- 172.21.15.39:0/3535516721 wait complete. 2024-09-21T07:48:00.206 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":650,"stamp":"2024-09-21T07:47:59.833525+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":405576,"kb_used_data":21388,"kb_used_omap":12,"kb_used_meta":383987,"kb_avail":754569144,"statfs":{"total":773094113280,"available":772678803456,"internally_reserved":0,"allocated":21901312,"data_stored":12185057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":393203248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001929"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.170421+0000","last_change":"2024-09-21T07:47:55.201069+0000","last_active":"2024-09-21T07:47:56.170421+0000","last_peered":"2024-09-21T07:47:56.170421+0000","last_clean":"2024-09-21T07:47:56.170421+0000","last_became_active":"2024-09-21T07:47:55.200379+0000","last_became_peered":"2024-09-21T07:47:55.200379+0000","last_unstale":"2024-09-21T07:47:56.170421+0000","last_undegraded":"2024-09-21T07:47:56.170421+0000","last_fullsized":"2024-09-21T07:47:56.170421+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:05:25.718314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.170039+0000","last_change":"2024-09-21T07:47:55.186523+0000","last_active":"2024-09-21T07:47:56.170039+0000","last_peered":"2024-09-21T07:47:56.170039+0000","last_clean":"2024-09-21T07:47:56.170039+0000","last_became_active":"2024-09-21T07:47:55.186240+0000","last_became_peered":"2024-09-21T07:47:55.186240+0000","last_unstale":"2024-09-21T07:47:56.170039+0000","last_undegraded":"2024-09-21T07:47:56.170039+0000","last_fullsized":"2024-09-21T07:47:56.170039+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:12:04.414896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.187987+0000","last_change":"2024-09-21T07:47:55.182034+0000","last_active":"2024-09-21T07:47:55.187987+0000","last_peered":"2024-09-21T07:47:55.187987+0000","last_clean":"2024-09-21T07:47:55.187987+0000","last_became_active":"2024-09-21T07:47:55.181704+0000","last_became_peered":"2024-09-21T07:47:55.181704+0000","last_unstale":"2024-09-21T07:47:55.187987+0000","last_undegraded":"2024-09-21T07:47:55.187987+0000","last_fullsized":"2024-09-21T07:47:55.187987+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:08:23.323166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.207848+0000","last_change":"2024-09-21T07:47:55.202353+0000","last_active":"2024-09-21T07:47:55.207848+0000","last_peered":"2024-09-21T07:47:55.207848+0000","last_clean":"2024-09-21T07:47:55.207848+0000","last_became_active":"2024-09-21T07:47:55.202019+0000","last_became_peered":"2024-09-21T07:47:55.202019+0000","last_unstale":"2024-09-21T07:47:55.207848+0000","last_undegraded":"2024-09-21T07:47:55.207848+0000","last_fullsized":"2024-09-21T07:47:55.207848+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:03:18.039256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.180032+0000","last_change":"2024-09-21T07:47:55.174878+0000","last_active":"2024-09-21T07:47:55.180032+0000","last_peered":"2024-09-21T07:47:55.180032+0000","last_clean":"2024-09-21T07:47:55.180032+0000","last_became_active":"2024-09-21T07:47:55.174598+0000","last_became_peered":"2024-09-21T07:47:55.174598+0000","last_unstale":"2024-09-21T07:47:55.180032+0000","last_undegraded":"2024-09-21T07:47:55.180032+0000","last_fullsized":"2024-09-21T07:47:55.180032+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:36:56.278477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.206490+0000","last_change":"2024-09-21T07:47:55.204444+0000","last_active":"2024-09-21T07:47:55.206490+0000","last_peered":"2024-09-21T07:47:55.206490+0000","last_clean":"2024-09-21T07:47:55.206490+0000","last_became_active":"2024-09-21T07:47:55.204116+0000","last_became_peered":"2024-09-21T07:47:55.204116+0000","last_unstale":"2024-09-21T07:47:55.206490+0000","last_undegraded":"2024-09-21T07:47:55.206490+0000","last_fullsized":"2024-09-21T07:47:55.206490+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:00.304956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.171594+0000","last_change":"2024-09-21T07:47:55.191399+0000","last_active":"2024-09-21T07:47:56.171594+0000","last_peered":"2024-09-21T07:47:56.171594+0000","last_clean":"2024-09-21T07:47:56.171594+0000","last_became_active":"2024-09-21T07:47:55.190024+0000","last_became_peered":"2024-09-21T07:47:55.190024+0000","last_unstale":"2024-09-21T07:47:56.171594+0000","last_undegraded":"2024-09-21T07:47:56.171594+0000","last_fullsized":"2024-09-21T07:47:56.171594+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:57:10.017744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.202554+0000","last_change":"2024-09-21T07:47:55.198086+0000","last_active":"2024-09-21T07:47:55.202554+0000","last_peered":"2024-09-21T07:47:55.202554+0000","last_clean":"2024-09-21T07:47:55.202554+0000","last_became_active":"2024-09-21T07:47:55.196776+0000","last_became_peered":"2024-09-21T07:47:55.196776+0000","last_unstale":"2024-09-21T07:47:55.202554+0000","last_undegraded":"2024-09-21T07:47:55.202554+0000","last_fullsized":"2024-09-21T07:47:55.202554+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:42.509301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.170490+0000","last_change":"2024-09-21T07:47:55.190919+0000","last_active":"2024-09-21T07:47:56.170490+0000","last_peered":"2024-09-21T07:47:56.170490+0000","last_clean":"2024-09-21T07:47:56.170490+0000","last_became_active":"2024-09-21T07:47:55.190629+0000","last_became_peered":"2024-09-21T07:47:55.190629+0000","last_unstale":"2024-09-21T07:47:56.170490+0000","last_undegraded":"2024-09-21T07:47:56.170490+0000","last_fullsized":"2024-09-21T07:47:56.170490+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:43.304964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.195521+0000","last_change":"2024-09-21T07:47:55.192028+0000","last_active":"2024-09-21T07:47:55.195521+0000","last_peered":"2024-09-21T07:47:55.195521+0000","last_clean":"2024-09-21T07:47:55.195521+0000","last_became_active":"2024-09-21T07:47:55.191672+0000","last_became_peered":"2024-09-21T07:47:55.191672+0000","last_unstale":"2024-09-21T07:47:55.195521+0000","last_undegraded":"2024-09-21T07:47:55.195521+0000","last_fullsized":"2024-09-21T07:47:55.195521+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:16.143413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172466+0000","last_change":"2024-09-21T07:47:55.187516+0000","last_active":"2024-09-21T07:47:56.172466+0000","last_peered":"2024-09-21T07:47:56.172466+0000","last_clean":"2024-09-21T07:47:56.172466+0000","last_became_active":"2024-09-21T07:47:55.187198+0000","last_became_peered":"2024-09-21T07:47:55.187198+0000","last_unstale":"2024-09-21T07:47:56.172466+0000","last_undegraded":"2024-09-21T07:47:56.172466+0000","last_fullsized":"2024-09-21T07:47:56.172466+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:50.493155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.207863+0000","last_change":"2024-09-21T07:47:55.205897+0000","last_active":"2024-09-21T07:47:55.207863+0000","last_peered":"2024-09-21T07:47:55.207863+0000","last_clean":"2024-09-21T07:47:55.207863+0000","last_became_active":"2024-09-21T07:47:55.205278+0000","last_became_peered":"2024-09-21T07:47:55.205278+0000","last_unstale":"2024-09-21T07:47:55.207863+0000","last_undegraded":"2024-09-21T07:47:55.207863+0000","last_fullsized":"2024-09-21T07:47:55.207863+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:00:49.437527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.202680+0000","last_change":"2024-09-21T07:47:55.193427+0000","last_active":"2024-09-21T07:47:55.202680+0000","last_peered":"2024-09-21T07:47:55.202680+0000","last_clean":"2024-09-21T07:47:55.202680+0000","last_became_active":"2024-09-21T07:47:55.191920+0000","last_became_peered":"2024-09-21T07:47:55.191920+0000","last_unstale":"2024-09-21T07:47:55.202680+0000","last_undegraded":"2024-09-21T07:47:55.202680+0000","last_fullsized":"2024-09-21T07:47:55.202680+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:19.278614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.171559+0000","last_change":"2024-09-21T07:47:55.187111+0000","last_active":"2024-09-21T07:47:56.171559+0000","last_peered":"2024-09-21T07:47:56.171559+0000","last_clean":"2024-09-21T07:47:56.171559+0000","last_became_active":"2024-09-21T07:47:55.186688+0000","last_became_peered":"2024-09-21T07:47:55.186688+0000","last_unstale":"2024-09-21T07:47:56.171559+0000","last_undegraded":"2024-09-21T07:47:56.171559+0000","last_fullsized":"2024-09-21T07:47:56.171559+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:52.955721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.168585+0000","last_change":"2024-09-21T07:47:55.196437+0000","last_active":"2024-09-21T07:47:56.168585+0000","last_peered":"2024-09-21T07:47:56.168585+0000","last_clean":"2024-09-21T07:47:56.168585+0000","last_became_active":"2024-09-21T07:47:55.196127+0000","last_became_peered":"2024-09-21T07:47:55.196127+0000","last_unstale":"2024-09-21T07:47:56.168585+0000","last_undegraded":"2024-09-21T07:47:56.168585+0000","last_fullsized":"2024-09-21T07:47:56.168585+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:55:56.600312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.180838+0000","last_change":"2024-09-21T07:47:55.173391+0000","last_active":"2024-09-21T07:47:55.180838+0000","last_peered":"2024-09-21T07:47:55.180838+0000","last_clean":"2024-09-21T07:47:55.180838+0000","last_became_active":"2024-09-21T07:47:55.172004+0000","last_became_peered":"2024-09-21T07:47:55.172004+0000","last_unstale":"2024-09-21T07:47:55.180838+0000","last_undegraded":"2024-09-21T07:47:55.180838+0000","last_fullsized":"2024-09-21T07:47:55.180838+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:34:16.990305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.204759+0000","last_change":"2024-09-21T07:47:55.197091+0000","last_active":"2024-09-21T07:47:55.204759+0000","last_peered":"2024-09-21T07:47:55.204759+0000","last_clean":"2024-09-21T07:47:55.204759+0000","last_became_active":"2024-09-21T07:47:55.196737+0000","last_became_peered":"2024-09-21T07:47:55.196737+0000","last_unstale":"2024-09-21T07:47:55.204759+0000","last_undegraded":"2024-09-21T07:47:55.204759+0000","last_fullsized":"2024-09-21T07:47:55.204759+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:53.383651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.205907+0000","last_change":"2024-09-21T07:47:55.203810+0000","last_active":"2024-09-21T07:47:55.205907+0000","last_peered":"2024-09-21T07:47:55.205907+0000","last_clean":"2024-09-21T07:47:55.205907+0000","last_became_active":"2024-09-21T07:47:55.203482+0000","last_became_peered":"2024-09-21T07:47:55.203482+0000","last_unstale":"2024-09-21T07:47:55.205907+0000","last_undegraded":"2024-09-21T07:47:55.205907+0000","last_fullsized":"2024-09-21T07:47:55.205907+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:17.269487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.204475+0000","last_change":"2024-09-21T07:47:55.193855+0000","last_active":"2024-09-21T07:47:55.204475+0000","last_peered":"2024-09-21T07:47:55.204475+0000","last_clean":"2024-09-21T07:47:55.204475+0000","last_became_active":"2024-09-21T07:47:55.190548+0000","last_became_peered":"2024-09-21T07:47:55.190548+0000","last_unstale":"2024-09-21T07:47:55.204475+0000","last_undegraded":"2024-09-21T07:47:55.204475+0000","last_fullsized":"2024-09-21T07:47:55.204475+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:03.931741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.206110+0000","last_change":"2024-09-21T07:47:55.199657+0000","last_active":"2024-09-21T07:47:55.206110+0000","last_peered":"2024-09-21T07:47:55.206110+0000","last_clean":"2024-09-21T07:47:55.206110+0000","last_became_active":"2024-09-21T07:47:55.199360+0000","last_became_peered":"2024-09-21T07:47:55.199360+0000","last_unstale":"2024-09-21T07:47:55.206110+0000","last_undegraded":"2024-09-21T07:47:55.206110+0000","last_fullsized":"2024-09-21T07:47:55.206110+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:05.149895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.169245+0000","last_change":"2024-09-21T07:47:55.197437+0000","last_active":"2024-09-21T07:47:56.169245+0000","last_peered":"2024-09-21T07:47:56.169245+0000","last_clean":"2024-09-21T07:47:56.169245+0000","last_became_active":"2024-09-21T07:47:55.196996+0000","last_became_peered":"2024-09-21T07:47:55.196996+0000","last_unstale":"2024-09-21T07:47:56.169245+0000","last_undegraded":"2024-09-21T07:47:56.169245+0000","last_fullsized":"2024-09-21T07:47:56.169245+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:45:46.537703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.169098+0000","last_change":"2024-09-21T07:47:55.187489+0000","last_active":"2024-09-21T07:47:56.169098+0000","last_peered":"2024-09-21T07:47:56.169098+0000","last_clean":"2024-09-21T07:47:56.169098+0000","last_became_active":"2024-09-21T07:47:55.187178+0000","last_became_peered":"2024-09-21T07:47:55.187178+0000","last_unstale":"2024-09-21T07:47:56.169098+0000","last_undegraded":"2024-09-21T07:47:56.169098+0000","last_fullsized":"2024-09-21T07:47:56.169098+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:23:39.689508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.207407+0000","last_change":"2024-09-21T07:47:55.203335+0000","last_active":"2024-09-21T07:47:55.207407+0000","last_peered":"2024-09-21T07:47:55.207407+0000","last_clean":"2024-09-21T07:47:55.207407+0000","last_became_active":"2024-09-21T07:47:55.202467+0000","last_became_peered":"2024-09-21T07:47:55.202467+0000","last_unstale":"2024-09-21T07:47:55.207407+0000","last_undegraded":"2024-09-21T07:47:55.207407+0000","last_fullsized":"2024-09-21T07:47:55.207407+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:52.609270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.168695+0000","last_change":"2024-09-21T07:47:55.200289+0000","last_active":"2024-09-21T07:47:56.168695+0000","last_peered":"2024-09-21T07:47:56.168695+0000","last_clean":"2024-09-21T07:47:56.168695+0000","last_became_active":"2024-09-21T07:47:55.199903+0000","last_became_peered":"2024-09-21T07:47:55.199903+0000","last_unstale":"2024-09-21T07:47:56.168695+0000","last_undegraded":"2024-09-21T07:47:56.168695+0000","last_fullsized":"2024-09-21T07:47:56.168695+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:35:55.605781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172692+0000","last_change":"2024-09-21T07:47:55.174506+0000","last_active":"2024-09-21T07:47:56.172692+0000","last_peered":"2024-09-21T07:47:56.172692+0000","last_clean":"2024-09-21T07:47:56.172692+0000","last_became_active":"2024-09-21T07:47:55.174207+0000","last_became_peered":"2024-09-21T07:47:55.174207+0000","last_unstale":"2024-09-21T07:47:56.172692+0000","last_undegraded":"2024-09-21T07:47:56.172692+0000","last_fullsized":"2024-09-21T07:47:56.172692+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:42.250865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172548+0000","last_change":"2024-09-21T07:47:55.176732+0000","last_active":"2024-09-21T07:47:56.172548+0000","last_peered":"2024-09-21T07:47:56.172548+0000","last_clean":"2024-09-21T07:47:56.172548+0000","last_became_active":"2024-09-21T07:47:55.175848+0000","last_became_peered":"2024-09-21T07:47:55.175848+0000","last_unstale":"2024-09-21T07:47:56.172548+0000","last_undegraded":"2024-09-21T07:47:56.172548+0000","last_fullsized":"2024-09-21T07:47:56.172548+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:04:42.927874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.202768+0000","last_change":"2024-09-21T07:47:55.199515+0000","last_active":"2024-09-21T07:47:55.202768+0000","last_peered":"2024-09-21T07:47:55.202768+0000","last_clean":"2024-09-21T07:47:55.202768+0000","last_became_active":"2024-09-21T07:47:55.196699+0000","last_became_peered":"2024-09-21T07:47:55.196699+0000","last_unstale":"2024-09-21T07:47:55.202768+0000","last_undegraded":"2024-09-21T07:47:55.202768+0000","last_fullsized":"2024-09-21T07:47:55.202768+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:55.754358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.179419+0000","last_change":"2024-09-21T07:47:55.175818+0000","last_active":"2024-09-21T07:47:55.179419+0000","last_peered":"2024-09-21T07:47:55.179419+0000","last_clean":"2024-09-21T07:47:55.179419+0000","last_became_active":"2024-09-21T07:47:55.175507+0000","last_became_peered":"2024-09-21T07:47:55.175507+0000","last_unstale":"2024-09-21T07:47:55.179419+0000","last_undegraded":"2024-09-21T07:47:55.179419+0000","last_fullsized":"2024-09-21T07:47:55.179419+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:06.645509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172433+0000","last_change":"2024-09-21T07:47:55.198596+0000","last_active":"2024-09-21T07:47:56.172433+0000","last_peered":"2024-09-21T07:47:56.172433+0000","last_clean":"2024-09-21T07:47:56.172433+0000","last_became_active":"2024-09-21T07:47:55.198281+0000","last_became_peered":"2024-09-21T07:47:55.198281+0000","last_unstale":"2024-09-21T07:47:56.172433+0000","last_undegraded":"2024-09-21T07:47:56.172433+0000","last_fullsized":"2024-09-21T07:47:56.172433+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:34.670653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172092+0000","last_change":"2024-09-21T07:47:55.183389+0000","last_active":"2024-09-21T07:47:56.172092+0000","last_peered":"2024-09-21T07:47:56.172092+0000","last_clean":"2024-09-21T07:47:56.172092+0000","last_became_active":"2024-09-21T07:47:55.183064+0000","last_became_peered":"2024-09-21T07:47:55.183064+0000","last_unstale":"2024-09-21T07:47:56.172092+0000","last_undegraded":"2024-09-21T07:47:56.172092+0000","last_fullsized":"2024-09-21T07:47:56.172092+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:26:40.505785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.171548+0000","last_change":"2024-09-21T07:47:55.183428+0000","last_active":"2024-09-21T07:47:56.171548+0000","last_peered":"2024-09-21T07:47:56.171548+0000","last_clean":"2024-09-21T07:47:56.171548+0000","last_became_active":"2024-09-21T07:47:55.182138+0000","last_became_peered":"2024-09-21T07:47:55.182138+0000","last_unstale":"2024-09-21T07:47:56.171548+0000","last_undegraded":"2024-09-21T07:47:56.171548+0000","last_fullsized":"2024-09-21T07:47:56.171548+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:19.513238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.196763+0000","last_change":"2024-09-21T07:47:55.191988+0000","last_active":"2024-09-21T07:47:55.196763+0000","last_peered":"2024-09-21T07:47:55.196763+0000","last_clean":"2024-09-21T07:47:55.196763+0000","last_became_active":"2024-09-21T07:47:55.191656+0000","last_became_peered":"2024-09-21T07:47:55.191656+0000","last_unstale":"2024-09-21T07:47:55.196763+0000","last_undegraded":"2024-09-21T07:47:55.196763+0000","last_fullsized":"2024-09-21T07:47:55.196763+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:22.981316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.196614+0000","last_change":"2024-09-21T07:47:55.190051+0000","last_active":"2024-09-21T07:47:55.196614+0000","last_peered":"2024-09-21T07:47:55.196614+0000","last_clean":"2024-09-21T07:47:55.196614+0000","last_became_active":"2024-09-21T07:47:55.188973+0000","last_became_peered":"2024-09-21T07:47:55.188973+0000","last_unstale":"2024-09-21T07:47:55.196614+0000","last_undegraded":"2024-09-21T07:47:55.196614+0000","last_fullsized":"2024-09-21T07:47:55.196614+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:01.028459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":13,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.179136+0000","last_change":"2024-09-21T07:47:54.184109+0000","last_active":"2024-09-21T07:47:55.179136+0000","last_peered":"2024-09-21T07:47:55.179136+0000","last_clean":"2024-09-21T07:47:55.179136+0000","last_became_active":"2024-09-21T07:47:54.183814+0000","last_became_peered":"2024-09-21T07:47:54.183814+0000","last_unstale":"2024-09-21T07:47:55.179136+0000","last_undegraded":"2024-09-21T07:47:55.179136+0000","last_fullsized":"2024-09-21T07:47:55.179136+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:00.609458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":280,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.179351+0000","last_change":"2024-09-21T07:35:27.879605+0000","last_active":"2024-09-21T07:47:55.179351+0000","last_peered":"2024-09-21T07:47:55.179351+0000","last_clean":"2024-09-21T07:47:55.179351+0000","last_became_active":"2024-09-21T07:35:27.879274+0000","last_became_peered":"2024-09-21T07:35:27.879274+0000","last_unstale":"2024-09-21T07:47:55.179351+0000","last_undegraded":"2024-09-21T07:47:55.179351+0000","last_fullsized":"2024-09-21T07:47:55.179351+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:24.241636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":290,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.165507+0000","last_change":"2024-09-21T07:35:29.491336+0000","last_active":"2024-09-21T07:47:55.165507+0000","last_peered":"2024-09-21T07:47:55.165507+0000","last_clean":"2024-09-21T07:47:55.165507+0000","last_became_active":"2024-09-21T07:35:26.866484+0000","last_became_peered":"2024-09-21T07:35:26.866484+0000","last_unstale":"2024-09-21T07:47:55.165507+0000","last_undegraded":"2024-09-21T07:47:55.165507+0000","last_fullsized":"2024-09-21T07:47:55.165507+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:10.449665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076010100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":466,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.181807+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:47:55.181807+0000","last_peered":"2024-09-21T07:47:55.181807+0000","last_clean":"2024-09-21T07:47:55.181807+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:47:55.181807+0000","last_undegraded":"2024-09-21T07:47:55.181807+0000","last_fullsized":"2024-09-21T07:47:55.181807+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":353,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.173167+0000","last_change":"2024-09-21T07:34:24.539248+0000","last_active":"2024-09-21T07:47:56.173167+0000","last_peered":"2024-09-21T07:47:56.173167+0000","last_clean":"2024-09-21T07:47:56.173167+0000","last_became_active":"2024-09-21T07:34:23.273465+0000","last_became_peered":"2024-09-21T07:34:23.273465+0000","last_unstale":"2024-09-21T07:47:56.173167+0000","last_undegraded":"2024-09-21T07:47:56.173167+0000","last_fullsized":"2024-09-21T07:47:56.173167+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:55.517452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060627700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172828+0000","last_change":"2024-09-21T07:47:55.198057+0000","last_active":"2024-09-21T07:47:56.172828+0000","last_peered":"2024-09-21T07:47:56.172828+0000","last_clean":"2024-09-21T07:47:56.172828+0000","last_became_active":"2024-09-21T07:47:55.197705+0000","last_became_peered":"2024-09-21T07:47:55.197705+0000","last_unstale":"2024-09-21T07:47:56.172828+0000","last_undegraded":"2024-09-21T07:47:56.172828+0000","last_fullsized":"2024-09-21T07:47:56.172828+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:55.610512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"205'2","reported_seq":17,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172617+0000","last_change":"2024-09-21T07:47:54.177156+0000","last_active":"2024-09-21T07:47:56.172617+0000","last_peered":"2024-09-21T07:47:56.172617+0000","last_clean":"2024-09-21T07:47:56.172617+0000","last_became_active":"2024-09-21T07:47:54.176814+0000","last_became_peered":"2024-09-21T07:47:54.176814+0000","last_unstale":"2024-09-21T07:47:56.172617+0000","last_undegraded":"2024-09-21T07:47:56.172617+0000","last_fullsized":"2024-09-21T07:47:56.172617+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:26.953022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":283,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.164751+0000","last_change":"2024-09-21T07:35:27.864492+0000","last_active":"2024-09-21T07:47:55.164751+0000","last_peered":"2024-09-21T07:47:55.164751+0000","last_clean":"2024-09-21T07:47:55.164751+0000","last_became_active":"2024-09-21T07:35:27.864176+0000","last_became_peered":"2024-09-21T07:35:27.864176+0000","last_unstale":"2024-09-21T07:47:55.164751+0000","last_undegraded":"2024-09-21T07:47:55.164751+0000","last_fullsized":"2024-09-21T07:47:55.164751+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:05.153467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":287,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.177082+0000","last_change":"2024-09-21T07:35:29.493678+0000","last_active":"2024-09-21T07:47:55.177082+0000","last_peered":"2024-09-21T07:47:55.177082+0000","last_clean":"2024-09-21T07:47:55.177082+0000","last_became_active":"2024-09-21T07:35:26.870749+0000","last_became_peered":"2024-09-21T07:35:26.870749+0000","last_unstale":"2024-09-21T07:47:55.177082+0000","last_undegraded":"2024-09-21T07:47:55.177082+0000","last_fullsized":"2024-09-21T07:47:55.177082+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:55.367877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010159570000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":333,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.184364+0000","last_change":"2024-09-21T07:34:24.597274+0000","last_active":"2024-09-21T07:47:55.184364+0000","last_peered":"2024-09-21T07:47:55.184364+0000","last_clean":"2024-09-21T07:47:55.184364+0000","last_became_active":"2024-09-21T07:34:23.279560+0000","last_became_peered":"2024-09-21T07:34:23.279560+0000","last_unstale":"2024-09-21T07:47:55.184364+0000","last_undegraded":"2024-09-21T07:47:55.184364+0000","last_fullsized":"2024-09-21T07:47:55.184364+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:15.845265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059699799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.189575+0000","last_change":"2024-09-21T07:47:55.185072+0000","last_active":"2024-09-21T07:47:55.189575+0000","last_peered":"2024-09-21T07:47:55.189575+0000","last_clean":"2024-09-21T07:47:55.189575+0000","last_became_active":"2024-09-21T07:47:55.182751+0000","last_became_peered":"2024-09-21T07:47:55.182751+0000","last_unstale":"2024-09-21T07:47:55.189575+0000","last_undegraded":"2024-09-21T07:47:55.189575+0000","last_fullsized":"2024-09-21T07:47:55.189575+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:53:02.800636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"205'2","reported_seq":15,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.203505+0000","last_change":"2024-09-21T07:47:54.193048+0000","last_active":"2024-09-21T07:47:55.203505+0000","last_peered":"2024-09-21T07:47:55.203505+0000","last_clean":"2024-09-21T07:47:55.203505+0000","last_became_active":"2024-09-21T07:47:54.192522+0000","last_became_peered":"2024-09-21T07:47:54.192522+0000","last_unstale":"2024-09-21T07:47:55.203505+0000","last_undegraded":"2024-09-21T07:47:55.203505+0000","last_fullsized":"2024-09-21T07:47:55.203505+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:12.868785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":285,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.171964+0000","last_change":"2024-09-21T07:35:27.874387+0000","last_active":"2024-09-21T07:47:56.171964+0000","last_peered":"2024-09-21T07:47:56.171964+0000","last_clean":"2024-09-21T07:47:56.171964+0000","last_became_active":"2024-09-21T07:35:27.873277+0000","last_became_peered":"2024-09-21T07:35:27.873277+0000","last_unstale":"2024-09-21T07:47:56.171964+0000","last_undegraded":"2024-09-21T07:47:56.171964+0000","last_fullsized":"2024-09-21T07:47:56.171964+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:04.581463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":283,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.186438+0000","last_change":"2024-09-21T07:35:29.387094+0000","last_active":"2024-09-21T07:47:55.186438+0000","last_peered":"2024-09-21T07:47:55.186438+0000","last_clean":"2024-09-21T07:47:55.186438+0000","last_became_active":"2024-09-21T07:35:26.874757+0000","last_became_peered":"2024-09-21T07:35:26.874757+0000","last_unstale":"2024-09-21T07:47:55.186438+0000","last_undegraded":"2024-09-21T07:47:55.186438+0000","last_fullsized":"2024-09-21T07:47:55.186438+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:33.542750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061065700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":340,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.176872+0000","last_change":"2024-09-21T07:34:24.259906+0000","last_active":"2024-09-21T07:47:55.176872+0000","last_peered":"2024-09-21T07:47:55.176872+0000","last_clean":"2024-09-21T07:47:55.176872+0000","last_became_active":"2024-09-21T07:34:23.635282+0000","last_became_peered":"2024-09-21T07:34:23.635282+0000","last_unstale":"2024-09-21T07:47:55.176872+0000","last_undegraded":"2024-09-21T07:47:55.176872+0000","last_fullsized":"2024-09-21T07:47:55.176872+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:00.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001527664,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.205209+0000","last_change":"2024-09-21T07:47:55.198827+0000","last_active":"2024-09-21T07:47:55.205209+0000","last_peered":"2024-09-21T07:47:55.205209+0000","last_clean":"2024-09-21T07:47:55.205209+0000","last_became_active":"2024-09-21T07:47:55.198535+0000","last_became_peered":"2024-09-21T07:47:55.198535+0000","last_unstale":"2024-09-21T07:47:55.205209+0000","last_undegraded":"2024-09-21T07:47:55.205209+0000","last_fullsized":"2024-09-21T07:47:55.205209+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:31:14.691225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"205'4","reported_seq":17,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.209633+0000","last_change":"2024-09-21T07:47:54.193701+0000","last_active":"2024-09-21T07:47:55.209633+0000","last_peered":"2024-09-21T07:47:55.209633+0000","last_clean":"2024-09-21T07:47:55.209633+0000","last_became_active":"2024-09-21T07:47:54.193394+0000","last_became_peered":"2024-09-21T07:47:54.193394+0000","last_unstale":"2024-09-21T07:47:55.209633+0000","last_undegraded":"2024-09-21T07:47:55.209633+0000","last_fullsized":"2024-09-21T07:47:55.209633+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:26:38.936794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":285,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.171848+0000","last_change":"2024-09-21T07:35:27.873645+0000","last_active":"2024-09-21T07:47:56.171848+0000","last_peered":"2024-09-21T07:47:56.171848+0000","last_clean":"2024-09-21T07:47:56.171848+0000","last_became_active":"2024-09-21T07:35:27.873245+0000","last_became_peered":"2024-09-21T07:35:27.873245+0000","last_unstale":"2024-09-21T07:47:56.171848+0000","last_undegraded":"2024-09-21T07:47:56.171848+0000","last_fullsized":"2024-09-21T07:47:56.171848+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:55.090953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":292,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172335+0000","last_change":"2024-09-21T07:35:28.861744+0000","last_active":"2024-09-21T07:47:56.172335+0000","last_peered":"2024-09-21T07:47:56.172335+0000","last_clean":"2024-09-21T07:47:56.172335+0000","last_became_active":"2024-09-21T07:35:26.876462+0000","last_became_peered":"2024-09-21T07:35:26.876462+0000","last_unstale":"2024-09-21T07:47:56.172335+0000","last_undegraded":"2024-09-21T07:47:56.172335+0000","last_fullsized":"2024-09-21T07:47:56.172335+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:48.632568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012020920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":337,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.178862+0000","last_change":"2024-09-21T07:34:25.085454+0000","last_active":"2024-09-21T07:47:55.178862+0000","last_peered":"2024-09-21T07:47:55.178862+0000","last_clean":"2024-09-21T07:47:55.178862+0000","last_became_active":"2024-09-21T07:34:23.279205+0000","last_became_peered":"2024-09-21T07:34:23.279205+0000","last_unstale":"2024-09-21T07:47:55.178862+0000","last_undegraded":"2024-09-21T07:47:55.178862+0000","last_fullsized":"2024-09-21T07:47:55.178862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:54.231666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095320599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172338+0000","last_change":"2024-09-21T07:47:55.191049+0000","last_active":"2024-09-21T07:47:56.172338+0000","last_peered":"2024-09-21T07:47:56.172338+0000","last_clean":"2024-09-21T07:47:56.172338+0000","last_became_active":"2024-09-21T07:47:55.189300+0000","last_became_peered":"2024-09-21T07:47:55.189300+0000","last_unstale":"2024-09-21T07:47:56.172338+0000","last_undegraded":"2024-09-21T07:47:56.172338+0000","last_fullsized":"2024-09-21T07:47:56.172338+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:03.710044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"205'1","reported_seq":14,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.201531+0000","last_change":"2024-09-21T07:47:54.192849+0000","last_active":"2024-09-21T07:47:55.201531+0000","last_peered":"2024-09-21T07:47:55.201531+0000","last_clean":"2024-09-21T07:47:55.201531+0000","last_became_active":"2024-09-21T07:47:54.189254+0000","last_became_peered":"2024-09-21T07:47:54.189254+0000","last_unstale":"2024-09-21T07:47:55.201531+0000","last_undegraded":"2024-09-21T07:47:55.201531+0000","last_fullsized":"2024-09-21T07:47:55.201531+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:45.982647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":283,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.169829+0000","last_change":"2024-09-21T07:35:27.861373+0000","last_active":"2024-09-21T07:47:55.169829+0000","last_peered":"2024-09-21T07:47:55.169829+0000","last_clean":"2024-09-21T07:47:55.169829+0000","last_became_active":"2024-09-21T07:35:27.861016+0000","last_became_peered":"2024-09-21T07:35:27.861016+0000","last_unstale":"2024-09-21T07:47:55.169829+0000","last_undegraded":"2024-09-21T07:47:55.169829+0000","last_fullsized":"2024-09-21T07:47:55.169829+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:56.043763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":291,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.169748+0000","last_change":"2024-09-21T07:35:29.053677+0000","last_active":"2024-09-21T07:47:56.169748+0000","last_peered":"2024-09-21T07:47:56.169748+0000","last_clean":"2024-09-21T07:47:56.169748+0000","last_became_active":"2024-09-21T07:35:26.875496+0000","last_became_peered":"2024-09-21T07:35:26.875496+0000","last_unstale":"2024-09-21T07:47:56.169748+0000","last_undegraded":"2024-09-21T07:47:56.169748+0000","last_fullsized":"2024-09-21T07:47:56.169748+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:04.850784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065859000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":343,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.168810+0000","last_change":"2024-09-21T07:34:25.028521+0000","last_active":"2024-09-21T07:47:56.168810+0000","last_peered":"2024-09-21T07:47:56.168810+0000","last_clean":"2024-09-21T07:47:56.168810+0000","last_became_active":"2024-09-21T07:34:23.635467+0000","last_became_peered":"2024-09-21T07:34:23.635467+0000","last_unstale":"2024-09-21T07:47:56.168810+0000","last_undegraded":"2024-09-21T07:47:56.168810+0000","last_fullsized":"2024-09-21T07:47:56.168810+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:33.729123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000660558,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.169236+0000","last_change":"2024-09-21T07:47:55.198778+0000","last_active":"2024-09-21T07:47:56.169236+0000","last_peered":"2024-09-21T07:47:56.169236+0000","last_clean":"2024-09-21T07:47:56.169236+0000","last_became_active":"2024-09-21T07:47:55.198442+0000","last_became_peered":"2024-09-21T07:47:55.198442+0000","last_unstale":"2024-09-21T07:47:56.169236+0000","last_undegraded":"2024-09-21T07:47:56.169236+0000","last_fullsized":"2024-09-21T07:47:56.169236+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:28:10.861128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":13,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.184382+0000","last_change":"2024-09-21T07:47:54.187807+0000","last_active":"2024-09-21T07:47:55.184382+0000","last_peered":"2024-09-21T07:47:55.184382+0000","last_clean":"2024-09-21T07:47:55.184382+0000","last_became_active":"2024-09-21T07:47:54.183644+0000","last_became_peered":"2024-09-21T07:47:54.183644+0000","last_unstale":"2024-09-21T07:47:55.184382+0000","last_undegraded":"2024-09-21T07:47:55.184382+0000","last_fullsized":"2024-09-21T07:47:55.184382+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:26:52.301094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":279,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.190182+0000","last_change":"2024-09-21T07:35:27.871201+0000","last_active":"2024-09-21T07:47:55.190182+0000","last_peered":"2024-09-21T07:47:55.190182+0000","last_clean":"2024-09-21T07:47:55.190182+0000","last_became_active":"2024-09-21T07:35:27.870816+0000","last_became_peered":"2024-09-21T07:35:27.870816+0000","last_unstale":"2024-09-21T07:47:55.190182+0000","last_undegraded":"2024-09-21T07:47:55.190182+0000","last_fullsized":"2024-09-21T07:47:55.190182+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:04.787450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":290,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.170408+0000","last_change":"2024-09-21T07:35:29.491874+0000","last_active":"2024-09-21T07:47:55.170408+0000","last_peered":"2024-09-21T07:47:55.170408+0000","last_clean":"2024-09-21T07:47:55.170408+0000","last_became_active":"2024-09-21T07:35:26.866381+0000","last_became_peered":"2024-09-21T07:35:26.866381+0000","last_unstale":"2024-09-21T07:47:55.170408+0000","last_undegraded":"2024-09-21T07:47:55.170408+0000","last_fullsized":"2024-09-21T07:47:55.170408+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:36.404647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073235399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":345,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.166761+0000","last_change":"2024-09-21T07:34:24.260046+0000","last_active":"2024-09-21T07:47:55.166761+0000","last_peered":"2024-09-21T07:47:55.166761+0000","last_clean":"2024-09-21T07:47:55.166761+0000","last_became_active":"2024-09-21T07:34:23.273431+0000","last_became_peered":"2024-09-21T07:34:23.273431+0000","last_unstale":"2024-09-21T07:47:55.166761+0000","last_undegraded":"2024-09-21T07:47:55.166761+0000","last_fullsized":"2024-09-21T07:47:55.166761+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:18.876057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019803310000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.203648+0000","last_change":"2024-09-21T07:47:55.200282+0000","last_active":"2024-09-21T07:47:55.203648+0000","last_peered":"2024-09-21T07:47:55.203648+0000","last_clean":"2024-09-21T07:47:55.203648+0000","last_became_active":"2024-09-21T07:47:55.199405+0000","last_became_peered":"2024-09-21T07:47:55.199405+0000","last_unstale":"2024-09-21T07:47:55.203648+0000","last_undegraded":"2024-09-21T07:47:55.203648+0000","last_fullsized":"2024-09-21T07:47:55.203648+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:07.342878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"205'1","reported_seq":14,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.194254+0000","last_change":"2024-09-21T07:47:54.192931+0000","last_active":"2024-09-21T07:47:55.194254+0000","last_peered":"2024-09-21T07:47:55.194254+0000","last_clean":"2024-09-21T07:47:55.194254+0000","last_became_active":"2024-09-21T07:47:54.190292+0000","last_became_peered":"2024-09-21T07:47:54.190292+0000","last_unstale":"2024-09-21T07:47:55.194254+0000","last_undegraded":"2024-09-21T07:47:55.194254+0000","last_fullsized":"2024-09-21T07:47:55.194254+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:55.206956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":279,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.191076+0000","last_change":"2024-09-21T07:35:27.871736+0000","last_active":"2024-09-21T07:47:55.191076+0000","last_peered":"2024-09-21T07:47:55.191076+0000","last_clean":"2024-09-21T07:47:55.191076+0000","last_became_active":"2024-09-21T07:35:27.871232+0000","last_became_peered":"2024-09-21T07:35:27.871232+0000","last_unstale":"2024-09-21T07:47:55.191076+0000","last_undegraded":"2024-09-21T07:47:55.191076+0000","last_fullsized":"2024-09-21T07:47:55.191076+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:04.419301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":290,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.169552+0000","last_change":"2024-09-21T07:35:29.491246+0000","last_active":"2024-09-21T07:47:55.169552+0000","last_peered":"2024-09-21T07:47:55.169552+0000","last_clean":"2024-09-21T07:47:55.169552+0000","last_became_active":"2024-09-21T07:35:26.866783+0000","last_became_peered":"2024-09-21T07:35:26.866783+0000","last_unstale":"2024-09-21T07:47:55.169552+0000","last_undegraded":"2024-09-21T07:47:55.169552+0000","last_fullsized":"2024-09-21T07:47:55.169552+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:28.881914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098474099999999992,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":343,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.169624+0000","last_change":"2024-09-21T07:34:25.028640+0000","last_active":"2024-09-21T07:47:56.169624+0000","last_peered":"2024-09-21T07:47:56.169624+0000","last_clean":"2024-09-21T07:47:56.169624+0000","last_became_active":"2024-09-21T07:34:23.260950+0000","last_became_peered":"2024-09-21T07:34:23.260950+0000","last_unstale":"2024-09-21T07:47:56.169624+0000","last_undegraded":"2024-09-21T07:47:56.169624+0000","last_fullsized":"2024-09-21T07:47:56.169624+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:23:42.121398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055135099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.168768+0000","last_change":"2024-09-21T07:47:55.181700+0000","last_active":"2024-09-21T07:47:56.168768+0000","last_peered":"2024-09-21T07:47:56.168768+0000","last_clean":"2024-09-21T07:47:56.168768+0000","last_became_active":"2024-09-21T07:47:55.181342+0000","last_became_peered":"2024-09-21T07:47:55.181342+0000","last_unstale":"2024-09-21T07:47:56.168768+0000","last_undegraded":"2024-09-21T07:47:56.168768+0000","last_fullsized":"2024-09-21T07:47:56.168768+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:23.800598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"205'1","reported_seq":14,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.206721+0000","last_change":"2024-09-21T07:47:54.192086+0000","last_active":"2024-09-21T07:47:55.206721+0000","last_peered":"2024-09-21T07:47:55.206721+0000","last_clean":"2024-09-21T07:47:55.206721+0000","last_became_active":"2024-09-21T07:47:54.191756+0000","last_became_peered":"2024-09-21T07:47:54.191756+0000","last_unstale":"2024-09-21T07:47:55.206721+0000","last_undegraded":"2024-09-21T07:47:55.206721+0000","last_fullsized":"2024-09-21T07:47:55.206721+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:10:23.633614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":281,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.180363+0000","last_change":"2024-09-21T07:35:27.872027+0000","last_active":"2024-09-21T07:47:55.180363+0000","last_peered":"2024-09-21T07:47:55.180363+0000","last_clean":"2024-09-21T07:47:55.180363+0000","last_became_active":"2024-09-21T07:35:27.870426+0000","last_became_peered":"2024-09-21T07:35:27.870426+0000","last_unstale":"2024-09-21T07:47:55.180363+0000","last_undegraded":"2024-09-21T07:47:55.180363+0000","last_fullsized":"2024-09-21T07:47:55.180363+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":290,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.165413+0000","last_change":"2024-09-21T07:35:29.491008+0000","last_active":"2024-09-21T07:47:55.165413+0000","last_peered":"2024-09-21T07:47:55.165413+0000","last_clean":"2024-09-21T07:47:55.165413+0000","last_became_active":"2024-09-21T07:35:26.866750+0000","last_became_peered":"2024-09-21T07:35:26.866750+0000","last_unstale":"2024-09-21T07:47:55.165413+0000","last_undegraded":"2024-09-21T07:47:55.165413+0000","last_fullsized":"2024-09-21T07:47:55.165413+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:25.490406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083529700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":341,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.184483+0000","last_change":"2024-09-21T07:34:24.260665+0000","last_active":"2024-09-21T07:47:55.184483+0000","last_peered":"2024-09-21T07:47:55.184483+0000","last_clean":"2024-09-21T07:47:55.184483+0000","last_became_active":"2024-09-21T07:34:23.635909+0000","last_became_peered":"2024-09-21T07:34:23.635909+0000","last_unstale":"2024-09-21T07:47:55.184483+0000","last_undegraded":"2024-09-21T07:47:55.184483+0000","last_fullsized":"2024-09-21T07:47:55.184483+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:44.536885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.204904+0000","last_change":"2024-09-21T07:47:55.192084+0000","last_active":"2024-09-21T07:47:55.204904+0000","last_peered":"2024-09-21T07:47:55.204904+0000","last_clean":"2024-09-21T07:47:55.204904+0000","last_became_active":"2024-09-21T07:47:55.189708+0000","last_became_peered":"2024-09-21T07:47:55.189708+0000","last_unstale":"2024-09-21T07:47:55.204904+0000","last_undegraded":"2024-09-21T07:47:55.204904+0000","last_fullsized":"2024-09-21T07:47:55.204904+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:42:29.513220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"205'3","reported_seq":16,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.207936+0000","last_change":"2024-09-21T07:47:54.197045+0000","last_active":"2024-09-21T07:47:55.207936+0000","last_peered":"2024-09-21T07:47:55.207936+0000","last_clean":"2024-09-21T07:47:55.207936+0000","last_became_active":"2024-09-21T07:47:54.194131+0000","last_became_peered":"2024-09-21T07:47:54.194131+0000","last_unstale":"2024-09-21T07:47:55.207936+0000","last_undegraded":"2024-09-21T07:47:55.207936+0000","last_fullsized":"2024-09-21T07:47:55.207936+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:57.606493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":277,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.182347+0000","last_change":"2024-09-21T07:35:27.874772+0000","last_active":"2024-09-21T07:47:55.182347+0000","last_peered":"2024-09-21T07:47:55.182347+0000","last_clean":"2024-09-21T07:47:55.182347+0000","last_became_active":"2024-09-21T07:35:27.874434+0000","last_became_peered":"2024-09-21T07:35:27.874434+0000","last_unstale":"2024-09-21T07:47:55.182347+0000","last_undegraded":"2024-09-21T07:47:55.182347+0000","last_fullsized":"2024-09-21T07:47:55.182347+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:31.998755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":300,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.173166+0000","last_change":"2024-09-21T07:35:28.859153+0000","last_active":"2024-09-21T07:47:56.173166+0000","last_peered":"2024-09-21T07:47:56.173166+0000","last_clean":"2024-09-21T07:47:56.173166+0000","last_became_active":"2024-09-21T07:35:26.874428+0000","last_became_peered":"2024-09-21T07:35:26.874428+0000","last_unstale":"2024-09-21T07:47:56.173166+0000","last_undegraded":"2024-09-21T07:47:56.173166+0000","last_fullsized":"2024-09-21T07:47:56.173166+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:04.487621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048557299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":353,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172080+0000","last_change":"2024-09-21T07:34:24.538845+0000","last_active":"2024-09-21T07:47:56.172080+0000","last_peered":"2024-09-21T07:47:56.172080+0000","last_clean":"2024-09-21T07:47:56.172080+0000","last_became_active":"2024-09-21T07:34:23.636372+0000","last_became_peered":"2024-09-21T07:34:23.636372+0000","last_unstale":"2024-09-21T07:47:56.172080+0000","last_undegraded":"2024-09-21T07:47:56.172080+0000","last_fullsized":"2024-09-21T07:47:56.172080+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:29.393813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841451,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.169685+0000","last_change":"2024-09-21T07:47:55.185787+0000","last_active":"2024-09-21T07:47:56.169685+0000","last_peered":"2024-09-21T07:47:56.169685+0000","last_clean":"2024-09-21T07:47:56.169685+0000","last_became_active":"2024-09-21T07:47:55.185387+0000","last_became_peered":"2024-09-21T07:47:55.185387+0000","last_unstale":"2024-09-21T07:47:56.169685+0000","last_undegraded":"2024-09-21T07:47:56.169685+0000","last_fullsized":"2024-09-21T07:47:56.169685+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:43.317041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"205'6","reported_seq":19,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.208655+0000","last_change":"2024-09-21T07:47:54.179178+0000","last_active":"2024-09-21T07:47:55.208655+0000","last_peered":"2024-09-21T07:47:55.208655+0000","last_clean":"2024-09-21T07:47:55.208655+0000","last_became_active":"2024-09-21T07:47:54.178851+0000","last_became_peered":"2024-09-21T07:47:54.178851+0000","last_unstale":"2024-09-21T07:47:55.208655+0000","last_undegraded":"2024-09-21T07:47:55.208655+0000","last_fullsized":"2024-09-21T07:47:55.208655+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:13.663241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":282,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.178675+0000","last_change":"2024-09-21T07:35:27.877801+0000","last_active":"2024-09-21T07:47:55.178675+0000","last_peered":"2024-09-21T07:47:55.178675+0000","last_clean":"2024-09-21T07:47:55.178675+0000","last_became_active":"2024-09-21T07:35:27.877464+0000","last_became_peered":"2024-09-21T07:35:27.877464+0000","last_unstale":"2024-09-21T07:47:55.178675+0000","last_undegraded":"2024-09-21T07:47:55.178675+0000","last_fullsized":"2024-09-21T07:47:55.178675+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:08.814929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":286,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.191204+0000","last_change":"2024-09-21T07:35:28.863732+0000","last_active":"2024-09-21T07:47:55.191204+0000","last_peered":"2024-09-21T07:47:55.191204+0000","last_clean":"2024-09-21T07:47:55.191204+0000","last_became_active":"2024-09-21T07:35:26.877392+0000","last_became_peered":"2024-09-21T07:35:26.877392+0000","last_unstale":"2024-09-21T07:47:55.191204+0000","last_undegraded":"2024-09-21T07:47:55.191204+0000","last_fullsized":"2024-09-21T07:47:55.191204+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071425700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":345,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.170285+0000","last_change":"2024-09-21T07:34:24.259686+0000","last_active":"2024-09-21T07:47:55.170285+0000","last_peered":"2024-09-21T07:47:55.170285+0000","last_clean":"2024-09-21T07:47:55.170285+0000","last_became_active":"2024-09-21T07:34:23.274405+0000","last_became_peered":"2024-09-21T07:34:23.274405+0000","last_unstale":"2024-09-21T07:47:55.170285+0000","last_undegraded":"2024-09-21T07:47:55.170285+0000","last_fullsized":"2024-09-21T07:47:55.170285+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:26.304693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022009920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.207451+0000","last_change":"2024-09-21T07:47:55.204047+0000","last_active":"2024-09-21T07:47:55.207451+0000","last_peered":"2024-09-21T07:47:55.207451+0000","last_clean":"2024-09-21T07:47:55.207451+0000","last_became_active":"2024-09-21T07:47:55.202068+0000","last_became_peered":"2024-09-21T07:47:55.202068+0000","last_unstale":"2024-09-21T07:47:55.207451+0000","last_undegraded":"2024-09-21T07:47:55.207451+0000","last_fullsized":"2024-09-21T07:47:55.207451+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:23.515802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":13,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.166889+0000","last_change":"2024-09-21T07:47:54.188332+0000","last_active":"2024-09-21T07:47:55.166889+0000","last_peered":"2024-09-21T07:47:55.166889+0000","last_clean":"2024-09-21T07:47:55.166889+0000","last_became_active":"2024-09-21T07:47:54.187549+0000","last_became_peered":"2024-09-21T07:47:54.187549+0000","last_unstale":"2024-09-21T07:47:55.166889+0000","last_undegraded":"2024-09-21T07:47:55.166889+0000","last_fullsized":"2024-09-21T07:47:55.166889+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:39:16.832519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":276,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.184158+0000","last_change":"2024-09-21T07:35:27.867336+0000","last_active":"2024-09-21T07:47:55.184158+0000","last_peered":"2024-09-21T07:47:55.184158+0000","last_clean":"2024-09-21T07:47:55.184158+0000","last_became_active":"2024-09-21T07:35:27.867015+0000","last_became_peered":"2024-09-21T07:35:27.867015+0000","last_unstale":"2024-09-21T07:47:55.184158+0000","last_undegraded":"2024-09-21T07:47:55.184158+0000","last_fullsized":"2024-09-21T07:47:55.184158+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:59.214139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":288,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.179498+0000","last_change":"2024-09-21T07:35:29.493137+0000","last_active":"2024-09-21T07:47:55.179498+0000","last_peered":"2024-09-21T07:47:55.179498+0000","last_clean":"2024-09-21T07:47:55.179498+0000","last_became_active":"2024-09-21T07:35:26.872230+0000","last_became_peered":"2024-09-21T07:35:26.872230+0000","last_unstale":"2024-09-21T07:47:55.179498+0000","last_undegraded":"2024-09-21T07:47:55.179498+0000","last_fullsized":"2024-09-21T07:47:55.179498+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:24.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001197449,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":341,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.181691+0000","last_change":"2024-09-21T07:34:24.258754+0000","last_active":"2024-09-21T07:47:55.181691+0000","last_peered":"2024-09-21T07:47:55.181691+0000","last_clean":"2024-09-21T07:47:55.181691+0000","last_became_active":"2024-09-21T07:34:23.272657+0000","last_became_peered":"2024-09-21T07:34:23.272657+0000","last_unstale":"2024-09-21T07:47:55.181691+0000","last_undegraded":"2024-09-21T07:47:55.181691+0000","last_fullsized":"2024-09-21T07:47:55.181691+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:05.477099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067143200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.187781+0000","last_change":"2024-09-21T07:47:55.184820+0000","last_active":"2024-09-21T07:47:55.187781+0000","last_peered":"2024-09-21T07:47:55.187781+0000","last_clean":"2024-09-21T07:47:55.187781+0000","last_became_active":"2024-09-21T07:47:55.184576+0000","last_became_peered":"2024-09-21T07:47:55.184576+0000","last_unstale":"2024-09-21T07:47:55.187781+0000","last_undegraded":"2024-09-21T07:47:55.187781+0000","last_fullsized":"2024-09-21T07:47:55.187781+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:40.419277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":15,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.169940+0000","last_change":"2024-09-21T07:47:54.191233+0000","last_active":"2024-09-21T07:47:56.169940+0000","last_peered":"2024-09-21T07:47:56.169940+0000","last_clean":"2024-09-21T07:47:56.169940+0000","last_became_active":"2024-09-21T07:47:54.190931+0000","last_became_peered":"2024-09-21T07:47:54.190931+0000","last_unstale":"2024-09-21T07:47:56.169940+0000","last_undegraded":"2024-09-21T07:47:56.169940+0000","last_fullsized":"2024-09-21T07:47:56.169940+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:40:18.466133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":280,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.178960+0000","last_change":"2024-09-21T07:35:27.876021+0000","last_active":"2024-09-21T07:47:55.178960+0000","last_peered":"2024-09-21T07:47:55.178960+0000","last_clean":"2024-09-21T07:47:55.178960+0000","last_became_active":"2024-09-21T07:35:27.875724+0000","last_became_peered":"2024-09-21T07:35:27.875724+0000","last_unstale":"2024-09-21T07:47:55.178960+0000","last_undegraded":"2024-09-21T07:47:55.178960+0000","last_fullsized":"2024-09-21T07:47:55.178960+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:03.022812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":288,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.178489+0000","last_change":"2024-09-21T07:35:29.492570+0000","last_active":"2024-09-21T07:47:55.178489+0000","last_peered":"2024-09-21T07:47:55.178489+0000","last_clean":"2024-09-21T07:47:55.178489+0000","last_became_active":"2024-09-21T07:35:26.874316+0000","last_became_peered":"2024-09-21T07:35:26.874316+0000","last_unstale":"2024-09-21T07:47:55.178489+0000","last_undegraded":"2024-09-21T07:47:55.178489+0000","last_fullsized":"2024-09-21T07:47:55.178489+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:16.512323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011990760000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":338,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.178425+0000","last_change":"2024-09-21T07:34:25.085232+0000","last_active":"2024-09-21T07:47:55.178425+0000","last_peered":"2024-09-21T07:47:55.178425+0000","last_clean":"2024-09-21T07:47:55.178425+0000","last_became_active":"2024-09-21T07:34:23.277807+0000","last_became_peered":"2024-09-21T07:34:23.277807+0000","last_unstale":"2024-09-21T07:47:55.178425+0000","last_undegraded":"2024-09-21T07:47:55.178425+0000","last_fullsized":"2024-09-21T07:47:55.178425+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:36.362919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086619099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.202404+0000","last_change":"2024-09-21T07:47:55.193141+0000","last_active":"2024-09-21T07:47:55.202404+0000","last_peered":"2024-09-21T07:47:55.202404+0000","last_clean":"2024-09-21T07:47:55.202404+0000","last_became_active":"2024-09-21T07:47:55.192811+0000","last_became_peered":"2024-09-21T07:47:55.192811+0000","last_unstale":"2024-09-21T07:47:55.202404+0000","last_undegraded":"2024-09-21T07:47:55.202404+0000","last_fullsized":"2024-09-21T07:47:55.202404+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:59.015996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":13,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.178077+0000","last_change":"2024-09-21T07:47:54.182443+0000","last_active":"2024-09-21T07:47:55.178077+0000","last_peered":"2024-09-21T07:47:55.178077+0000","last_clean":"2024-09-21T07:47:55.178077+0000","last_became_active":"2024-09-21T07:47:54.182126+0000","last_became_peered":"2024-09-21T07:47:54.182126+0000","last_unstale":"2024-09-21T07:47:55.178077+0000","last_undegraded":"2024-09-21T07:47:55.178077+0000","last_fullsized":"2024-09-21T07:47:55.178077+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:29.670475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":282,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.179745+0000","last_change":"2024-09-21T07:35:27.872798+0000","last_active":"2024-09-21T07:47:55.179745+0000","last_peered":"2024-09-21T07:47:55.179745+0000","last_clean":"2024-09-21T07:47:55.179745+0000","last_became_active":"2024-09-21T07:35:27.872483+0000","last_became_peered":"2024-09-21T07:35:27.872483+0000","last_unstale":"2024-09-21T07:47:55.179745+0000","last_undegraded":"2024-09-21T07:47:55.179745+0000","last_fullsized":"2024-09-21T07:47:55.179745+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:11.933469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":289,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.178460+0000","last_change":"2024-09-21T07:35:29.206723+0000","last_active":"2024-09-21T07:47:55.178460+0000","last_peered":"2024-09-21T07:47:55.178460+0000","last_clean":"2024-09-21T07:47:55.178460+0000","last_became_active":"2024-09-21T07:35:26.872377+0000","last_became_peered":"2024-09-21T07:35:26.872377+0000","last_unstale":"2024-09-21T07:47:55.178460+0000","last_undegraded":"2024-09-21T07:47:55.178460+0000","last_fullsized":"2024-09-21T07:47:55.178460+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:29.086862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083077200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":333,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.188254+0000","last_change":"2024-09-21T07:34:24.597672+0000","last_active":"2024-09-21T07:47:55.188254+0000","last_peered":"2024-09-21T07:47:55.188254+0000","last_clean":"2024-09-21T07:47:55.188254+0000","last_became_active":"2024-09-21T07:34:23.267770+0000","last_became_peered":"2024-09-21T07:34:23.267770+0000","last_unstale":"2024-09-21T07:47:55.188254+0000","last_undegraded":"2024-09-21T07:47:55.188254+0000","last_fullsized":"2024-09-21T07:47:55.188254+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:51.629476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086378500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.204644+0000","last_change":"2024-09-21T07:47:55.200811+0000","last_active":"2024-09-21T07:47:55.204644+0000","last_peered":"2024-09-21T07:47:55.204644+0000","last_clean":"2024-09-21T07:47:55.204644+0000","last_became_active":"2024-09-21T07:47:55.199782+0000","last_became_peered":"2024-09-21T07:47:55.199782+0000","last_unstale":"2024-09-21T07:47:55.204644+0000","last_undegraded":"2024-09-21T07:47:55.204644+0000","last_fullsized":"2024-09-21T07:47:55.204644+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:02:42.373550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"205'8","reported_seq":23,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.171817+0000","last_change":"2024-09-21T07:47:54.182287+0000","last_active":"2024-09-21T07:47:56.171817+0000","last_peered":"2024-09-21T07:47:56.171817+0000","last_clean":"2024-09-21T07:47:56.171817+0000","last_became_active":"2024-09-21T07:47:54.181670+0000","last_became_peered":"2024-09-21T07:47:54.181670+0000","last_unstale":"2024-09-21T07:47:56.171817+0000","last_undegraded":"2024-09-21T07:47:56.171817+0000","last_fullsized":"2024-09-21T07:47:56.171817+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:56:15.576668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":282,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.179711+0000","last_change":"2024-09-21T07:35:27.870121+0000","last_active":"2024-09-21T07:47:55.179711+0000","last_peered":"2024-09-21T07:47:55.179711+0000","last_clean":"2024-09-21T07:47:55.179711+0000","last_became_active":"2024-09-21T07:35:27.869811+0000","last_became_peered":"2024-09-21T07:35:27.869811+0000","last_unstale":"2024-09-21T07:47:55.179711+0000","last_undegraded":"2024-09-21T07:47:55.179711+0000","last_fullsized":"2024-09-21T07:47:55.179711+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:31.778461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":293,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.171823+0000","last_change":"2024-09-21T07:35:28.860699+0000","last_active":"2024-09-21T07:47:56.171823+0000","last_peered":"2024-09-21T07:47:56.171823+0000","last_clean":"2024-09-21T07:47:56.171823+0000","last_became_active":"2024-09-21T07:35:26.876779+0000","last_became_peered":"2024-09-21T07:35:26.876779+0000","last_unstale":"2024-09-21T07:47:56.171823+0000","last_undegraded":"2024-09-21T07:47:56.171823+0000","last_fullsized":"2024-09-21T07:47:56.171823+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:28.286403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000913188,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":345,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172607+0000","last_change":"2024-09-21T07:34:24.260181+0000","last_active":"2024-09-21T07:47:56.172607+0000","last_peered":"2024-09-21T07:47:56.172607+0000","last_clean":"2024-09-21T07:47:56.172607+0000","last_became_active":"2024-09-21T07:34:23.275580+0000","last_became_peered":"2024-09-21T07:34:23.275580+0000","last_unstale":"2024-09-21T07:47:56.172607+0000","last_undegraded":"2024-09-21T07:47:56.172607+0000","last_fullsized":"2024-09-21T07:47:56.172607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:49.857622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014706980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.171633+0000","last_change":"2024-09-21T07:47:55.189436+0000","last_active":"2024-09-21T07:47:56.171633+0000","last_peered":"2024-09-21T07:47:56.171633+0000","last_clean":"2024-09-21T07:47:56.171633+0000","last_became_active":"2024-09-21T07:47:55.189097+0000","last_became_peered":"2024-09-21T07:47:55.189097+0000","last_unstale":"2024-09-21T07:47:56.171633+0000","last_undegraded":"2024-09-21T07:47:56.171633+0000","last_fullsized":"2024-09-21T07:47:56.171633+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:49:09.381698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"205'2","reported_seq":15,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.202614+0000","last_change":"2024-09-21T07:47:54.187442+0000","last_active":"2024-09-21T07:47:55.202614+0000","last_peered":"2024-09-21T07:47:55.202614+0000","last_clean":"2024-09-21T07:47:55.202614+0000","last_became_active":"2024-09-21T07:47:54.186896+0000","last_became_peered":"2024-09-21T07:47:54.186896+0000","last_unstale":"2024-09-21T07:47:55.202614+0000","last_undegraded":"2024-09-21T07:47:55.202614+0000","last_fullsized":"2024-09-21T07:47:55.202614+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:39:02.858914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":284,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.170517+0000","last_change":"2024-09-21T07:35:27.875393+0000","last_active":"2024-09-21T07:47:56.170517+0000","last_peered":"2024-09-21T07:47:56.170517+0000","last_clean":"2024-09-21T07:47:56.170517+0000","last_became_active":"2024-09-21T07:35:27.874671+0000","last_became_peered":"2024-09-21T07:35:27.874671+0000","last_unstale":"2024-09-21T07:47:56.170517+0000","last_undegraded":"2024-09-21T07:47:56.170517+0000","last_fullsized":"2024-09-21T07:47:56.170517+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":289,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.178797+0000","last_change":"2024-09-21T07:35:29.206296+0000","last_active":"2024-09-21T07:47:55.178797+0000","last_peered":"2024-09-21T07:47:55.178797+0000","last_clean":"2024-09-21T07:47:55.178797+0000","last_became_active":"2024-09-21T07:35:26.874575+0000","last_became_peered":"2024-09-21T07:35:26.874575+0000","last_unstale":"2024-09-21T07:47:55.178797+0000","last_undegraded":"2024-09-21T07:47:55.178797+0000","last_fullsized":"2024-09-21T07:47:55.178797+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:22.771154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061011599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":345,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.170685+0000","last_change":"2024-09-21T07:34:24.259112+0000","last_active":"2024-09-21T07:47:55.170685+0000","last_peered":"2024-09-21T07:47:55.170685+0000","last_clean":"2024-09-21T07:47:55.170685+0000","last_became_active":"2024-09-21T07:34:23.635444+0000","last_became_peered":"2024-09-21T07:34:23.635444+0000","last_unstale":"2024-09-21T07:47:55.170685+0000","last_undegraded":"2024-09-21T07:47:55.170685+0000","last_fullsized":"2024-09-21T07:47:55.170685+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:46.255385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001952636,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.195542+0000","last_change":"2024-09-21T07:47:55.192886+0000","last_active":"2024-09-21T07:47:55.195542+0000","last_peered":"2024-09-21T07:47:55.195542+0000","last_clean":"2024-09-21T07:47:55.195542+0000","last_became_active":"2024-09-21T07:47:55.192478+0000","last_became_peered":"2024-09-21T07:47:55.192478+0000","last_unstale":"2024-09-21T07:47:55.195542+0000","last_undegraded":"2024-09-21T07:47:55.195542+0000","last_fullsized":"2024-09-21T07:47:55.195542+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:45.995506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":15,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.170182+0000","last_change":"2024-09-21T07:47:54.186176+0000","last_active":"2024-09-21T07:47:56.170182+0000","last_peered":"2024-09-21T07:47:56.170182+0000","last_clean":"2024-09-21T07:47:56.170182+0000","last_became_active":"2024-09-21T07:47:54.185846+0000","last_became_peered":"2024-09-21T07:47:54.185846+0000","last_unstale":"2024-09-21T07:47:56.170182+0000","last_undegraded":"2024-09-21T07:47:56.170182+0000","last_fullsized":"2024-09-21T07:47:56.170182+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:48.885930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":284,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.170487+0000","last_change":"2024-09-21T07:35:27.878356+0000","last_active":"2024-09-21T07:47:56.170487+0000","last_peered":"2024-09-21T07:47:56.170487+0000","last_clean":"2024-09-21T07:47:56.170487+0000","last_became_active":"2024-09-21T07:35:27.877982+0000","last_became_peered":"2024-09-21T07:35:27.877982+0000","last_unstale":"2024-09-21T07:47:56.170487+0000","last_undegraded":"2024-09-21T07:47:56.170487+0000","last_fullsized":"2024-09-21T07:47:56.170487+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":292,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.168604+0000","last_change":"2024-09-21T07:35:29.053131+0000","last_active":"2024-09-21T07:47:56.168604+0000","last_peered":"2024-09-21T07:47:56.168604+0000","last_clean":"2024-09-21T07:47:56.168604+0000","last_became_active":"2024-09-21T07:35:26.870516+0000","last_became_peered":"2024-09-21T07:35:26.870516+0000","last_unstale":"2024-09-21T07:47:56.168604+0000","last_undegraded":"2024-09-21T07:47:56.168604+0000","last_fullsized":"2024-09-21T07:47:56.168604+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:10.924491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070705699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":353,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172815+0000","last_change":"2024-09-21T07:34:24.539152+0000","last_active":"2024-09-21T07:47:56.172815+0000","last_peered":"2024-09-21T07:47:56.172815+0000","last_clean":"2024-09-21T07:47:56.172815+0000","last_became_active":"2024-09-21T07:34:23.259573+0000","last_became_peered":"2024-09-21T07:34:23.259573+0000","last_unstale":"2024-09-21T07:47:56.172815+0000","last_undegraded":"2024-09-21T07:47:56.172815+0000","last_fullsized":"2024-09-21T07:47:56.172815+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:26.986900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058108400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.173046+0000","last_change":"2024-09-21T07:47:55.176590+0000","last_active":"2024-09-21T07:47:56.173046+0000","last_peered":"2024-09-21T07:47:56.173046+0000","last_clean":"2024-09-21T07:47:56.173046+0000","last_became_active":"2024-09-21T07:47:55.175383+0000","last_became_peered":"2024-09-21T07:47:55.175383+0000","last_unstale":"2024-09-21T07:47:56.173046+0000","last_undegraded":"2024-09-21T07:47:56.173046+0000","last_fullsized":"2024-09-21T07:47:56.173046+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:25.181429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"205'1","reported_seq":14,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.206122+0000","last_change":"2024-09-21T07:47:54.195938+0000","last_active":"2024-09-21T07:47:55.206122+0000","last_peered":"2024-09-21T07:47:55.206122+0000","last_clean":"2024-09-21T07:47:55.206122+0000","last_became_active":"2024-09-21T07:47:54.195639+0000","last_became_peered":"2024-09-21T07:47:54.195639+0000","last_unstale":"2024-09-21T07:47:55.206122+0000","last_undegraded":"2024-09-21T07:47:55.206122+0000","last_fullsized":"2024-09-21T07:47:55.206122+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:20:56.328017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":284,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.168389+0000","last_change":"2024-09-21T07:35:27.873726+0000","last_active":"2024-09-21T07:47:56.168389+0000","last_peered":"2024-09-21T07:47:56.168389+0000","last_clean":"2024-09-21T07:47:56.168389+0000","last_became_active":"2024-09-21T07:35:27.871421+0000","last_became_peered":"2024-09-21T07:35:27.871421+0000","last_unstale":"2024-09-21T07:47:56.168389+0000","last_undegraded":"2024-09-21T07:47:56.168389+0000","last_fullsized":"2024-09-21T07:47:56.168389+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:00.168915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":293,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172481+0000","last_change":"2024-09-21T07:35:28.861032+0000","last_active":"2024-09-21T07:47:56.172481+0000","last_peered":"2024-09-21T07:47:56.172481+0000","last_clean":"2024-09-21T07:47:56.172481+0000","last_became_active":"2024-09-21T07:35:26.873149+0000","last_became_peered":"2024-09-21T07:35:26.873149+0000","last_unstale":"2024-09-21T07:47:56.172481+0000","last_undegraded":"2024-09-21T07:47:56.172481+0000","last_fullsized":"2024-09-21T07:47:56.172481+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:59.435981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011988369999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":340,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.175754+0000","last_change":"2024-09-21T07:34:24.259902+0000","last_active":"2024-09-21T07:47:55.175754+0000","last_peered":"2024-09-21T07:47:55.175754+0000","last_clean":"2024-09-21T07:47:55.175754+0000","last_became_active":"2024-09-21T07:34:23.271902+0000","last_became_peered":"2024-09-21T07:34:23.271902+0000","last_unstale":"2024-09-21T07:47:55.175754+0000","last_undegraded":"2024-09-21T07:47:55.175754+0000","last_fullsized":"2024-09-21T07:47:55.175754+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:59.439525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001686358,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.169943+0000","last_change":"2024-09-21T07:47:55.198986+0000","last_active":"2024-09-21T07:47:56.169943+0000","last_peered":"2024-09-21T07:47:56.169943+0000","last_clean":"2024-09-21T07:47:56.169943+0000","last_became_active":"2024-09-21T07:47:55.195736+0000","last_became_peered":"2024-09-21T07:47:55.195736+0000","last_unstale":"2024-09-21T07:47:56.169943+0000","last_undegraded":"2024-09-21T07:47:56.169943+0000","last_fullsized":"2024-09-21T07:47:56.169943+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:43.281303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":15,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172405+0000","last_change":"2024-09-21T07:47:54.185310+0000","last_active":"2024-09-21T07:47:56.172405+0000","last_peered":"2024-09-21T07:47:56.172405+0000","last_clean":"2024-09-21T07:47:56.172405+0000","last_became_active":"2024-09-21T07:47:54.184966+0000","last_became_peered":"2024-09-21T07:47:54.184966+0000","last_unstale":"2024-09-21T07:47:56.172405+0000","last_undegraded":"2024-09-21T07:47:56.172405+0000","last_fullsized":"2024-09-21T07:47:56.172405+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:03:17.404002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":279,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.178151+0000","last_change":"2024-09-21T07:35:27.872637+0000","last_active":"2024-09-21T07:47:55.178151+0000","last_peered":"2024-09-21T07:47:55.178151+0000","last_clean":"2024-09-21T07:47:55.178151+0000","last_became_active":"2024-09-21T07:35:27.872149+0000","last_became_peered":"2024-09-21T07:35:27.872149+0000","last_unstale":"2024-09-21T07:47:55.178151+0000","last_undegraded":"2024-09-21T07:47:55.178151+0000","last_fullsized":"2024-09-21T07:47:55.178151+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:27.697872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":288,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.175994+0000","last_change":"2024-09-21T07:35:29.493362+0000","last_active":"2024-09-21T07:47:55.175994+0000","last_peered":"2024-09-21T07:47:55.175994+0000","last_clean":"2024-09-21T07:47:55.175994+0000","last_became_active":"2024-09-21T07:35:26.873018+0000","last_became_peered":"2024-09-21T07:35:26.873018+0000","last_unstale":"2024-09-21T07:47:55.175994+0000","last_undegraded":"2024-09-21T07:47:55.175994+0000","last_fullsized":"2024-09-21T07:47:55.175994+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:02.460557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012496510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":344,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.170071+0000","last_change":"2024-09-21T07:34:24.259662+0000","last_active":"2024-09-21T07:47:55.170071+0000","last_peered":"2024-09-21T07:47:55.170071+0000","last_clean":"2024-09-21T07:47:55.170071+0000","last_became_active":"2024-09-21T07:34:23.278812+0000","last_became_peered":"2024-09-21T07:34:23.278812+0000","last_unstale":"2024-09-21T07:47:55.170071+0000","last_undegraded":"2024-09-21T07:47:55.170071+0000","last_fullsized":"2024-09-21T07:47:55.170071+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:49.829974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356463,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172012+0000","last_change":"2024-09-21T07:47:55.190247+0000","last_active":"2024-09-21T07:47:56.172012+0000","last_peered":"2024-09-21T07:47:56.172012+0000","last_clean":"2024-09-21T07:47:56.172012+0000","last_became_active":"2024-09-21T07:47:55.189871+0000","last_became_peered":"2024-09-21T07:47:55.189871+0000","last_unstale":"2024-09-21T07:47:56.172012+0000","last_undegraded":"2024-09-21T07:47:56.172012+0000","last_fullsized":"2024-09-21T07:47:56.172012+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:59:45.198861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"205'1","reported_seq":16,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.173115+0000","last_change":"2024-09-21T07:47:54.180420+0000","last_active":"2024-09-21T07:47:56.173115+0000","last_peered":"2024-09-21T07:47:56.173115+0000","last_clean":"2024-09-21T07:47:56.173115+0000","last_became_active":"2024-09-21T07:47:54.180041+0000","last_became_peered":"2024-09-21T07:47:54.180041+0000","last_unstale":"2024-09-21T07:47:56.173115+0000","last_undegraded":"2024-09-21T07:47:56.173115+0000","last_fullsized":"2024-09-21T07:47:56.173115+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:17:42.163288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":275,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.184006+0000","last_change":"2024-09-21T07:35:27.868448+0000","last_active":"2024-09-21T07:47:55.184006+0000","last_peered":"2024-09-21T07:47:55.184006+0000","last_clean":"2024-09-21T07:47:55.184006+0000","last_became_active":"2024-09-21T07:35:27.868105+0000","last_became_peered":"2024-09-21T07:35:27.868105+0000","last_unstale":"2024-09-21T07:47:55.184006+0000","last_undegraded":"2024-09-21T07:47:55.184006+0000","last_fullsized":"2024-09-21T07:47:55.184006+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:41.468702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":287,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.179291+0000","last_change":"2024-09-21T07:35:29.492974+0000","last_active":"2024-09-21T07:47:55.179291+0000","last_peered":"2024-09-21T07:47:55.179291+0000","last_clean":"2024-09-21T07:47:55.179291+0000","last_became_active":"2024-09-21T07:35:26.872165+0000","last_became_peered":"2024-09-21T07:35:26.872165+0000","last_unstale":"2024-09-21T07:47:55.179291+0000","last_undegraded":"2024-09-21T07:47:55.179291+0000","last_fullsized":"2024-09-21T07:47:55.179291+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:59.912520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013795179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":337,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.178726+0000","last_change":"2024-09-21T07:34:25.084861+0000","last_active":"2024-09-21T07:47:55.178726+0000","last_peered":"2024-09-21T07:47:55.178726+0000","last_clean":"2024-09-21T07:47:55.178726+0000","last_became_active":"2024-09-21T07:34:23.277708+0000","last_became_peered":"2024-09-21T07:34:23.277708+0000","last_unstale":"2024-09-21T07:47:55.178726+0000","last_undegraded":"2024-09-21T07:47:55.178726+0000","last_fullsized":"2024-09-21T07:47:55.178726+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:40.816488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000597566,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.192904+0000","last_change":"2024-09-21T07:47:55.188872+0000","last_active":"2024-09-21T07:47:55.192904+0000","last_peered":"2024-09-21T07:47:55.192904+0000","last_clean":"2024-09-21T07:47:55.192904+0000","last_became_active":"2024-09-21T07:47:55.188550+0000","last_became_peered":"2024-09-21T07:47:55.188550+0000","last_unstale":"2024-09-21T07:47:55.192904+0000","last_undegraded":"2024-09-21T07:47:55.192904+0000","last_fullsized":"2024-09-21T07:47:55.192904+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:18.496098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"205'2","reported_seq":15,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.208654+0000","last_change":"2024-09-21T07:47:54.181899+0000","last_active":"2024-09-21T07:47:55.208654+0000","last_peered":"2024-09-21T07:47:55.208654+0000","last_clean":"2024-09-21T07:47:55.208654+0000","last_became_active":"2024-09-21T07:47:54.181590+0000","last_became_peered":"2024-09-21T07:47:54.181590+0000","last_unstale":"2024-09-21T07:47:55.208654+0000","last_undegraded":"2024-09-21T07:47:55.208654+0000","last_fullsized":"2024-09-21T07:47:55.208654+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:41.106267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":275,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.185294+0000","last_change":"2024-09-21T07:35:27.874640+0000","last_active":"2024-09-21T07:47:55.185294+0000","last_peered":"2024-09-21T07:47:55.185294+0000","last_clean":"2024-09-21T07:47:55.185294+0000","last_became_active":"2024-09-21T07:35:27.874265+0000","last_became_peered":"2024-09-21T07:35:27.874265+0000","last_unstale":"2024-09-21T07:47:55.185294+0000","last_undegraded":"2024-09-21T07:47:55.185294+0000","last_fullsized":"2024-09-21T07:47:55.185294+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:45.754654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":288,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.179985+0000","last_change":"2024-09-21T07:35:29.207392+0000","last_active":"2024-09-21T07:47:55.179985+0000","last_peered":"2024-09-21T07:47:55.179985+0000","last_clean":"2024-09-21T07:47:55.179985+0000","last_became_active":"2024-09-21T07:35:26.870917+0000","last_became_peered":"2024-09-21T07:35:26.870917+0000","last_unstale":"2024-09-21T07:47:55.179985+0000","last_undegraded":"2024-09-21T07:47:55.179985+0000","last_fullsized":"2024-09-21T07:47:55.179985+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:59.159937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059198499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":344,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.165558+0000","last_change":"2024-09-21T07:34:24.258709+0000","last_active":"2024-09-21T07:47:55.165558+0000","last_peered":"2024-09-21T07:47:55.165558+0000","last_clean":"2024-09-21T07:47:55.165558+0000","last_became_active":"2024-09-21T07:34:23.279042+0000","last_became_peered":"2024-09-21T07:34:23.279042+0000","last_unstale":"2024-09-21T07:47:55.165558+0000","last_undegraded":"2024-09-21T07:47:55.165558+0000","last_fullsized":"2024-09-21T07:47:55.165558+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:01.762117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019282100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.189457+0000","last_change":"2024-09-21T07:47:55.187859+0000","last_active":"2024-09-21T07:47:55.189457+0000","last_peered":"2024-09-21T07:47:55.189457+0000","last_clean":"2024-09-21T07:47:55.189457+0000","last_became_active":"2024-09-21T07:47:55.187572+0000","last_became_peered":"2024-09-21T07:47:55.187572+0000","last_unstale":"2024-09-21T07:47:55.189457+0000","last_undegraded":"2024-09-21T07:47:55.189457+0000","last_fullsized":"2024-09-21T07:47:55.189457+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:26.667066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":15,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.171707+0000","last_change":"2024-09-21T07:47:54.187585+0000","last_active":"2024-09-21T07:47:56.171707+0000","last_peered":"2024-09-21T07:47:56.171707+0000","last_clean":"2024-09-21T07:47:56.171707+0000","last_became_active":"2024-09-21T07:47:54.187063+0000","last_became_peered":"2024-09-21T07:47:54.187063+0000","last_unstale":"2024-09-21T07:47:56.171707+0000","last_undegraded":"2024-09-21T07:47:56.171707+0000","last_fullsized":"2024-09-21T07:47:56.171707+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:15.785391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":279,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.190091+0000","last_change":"2024-09-21T07:35:27.870260+0000","last_active":"2024-09-21T07:47:55.190091+0000","last_peered":"2024-09-21T07:47:55.190091+0000","last_clean":"2024-09-21T07:47:55.190091+0000","last_became_active":"2024-09-21T07:35:27.869448+0000","last_became_peered":"2024-09-21T07:35:27.869448+0000","last_unstale":"2024-09-21T07:47:55.190091+0000","last_undegraded":"2024-09-21T07:47:55.190091+0000","last_fullsized":"2024-09-21T07:47:55.190091+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:16.946790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":283,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.184959+0000","last_change":"2024-09-21T07:35:29.386367+0000","last_active":"2024-09-21T07:47:55.184959+0000","last_peered":"2024-09-21T07:47:55.184959+0000","last_clean":"2024-09-21T07:47:55.184959+0000","last_became_active":"2024-09-21T07:35:26.879238+0000","last_became_peered":"2024-09-21T07:35:26.879238+0000","last_unstale":"2024-09-21T07:47:55.184959+0000","last_undegraded":"2024-09-21T07:47:55.184959+0000","last_fullsized":"2024-09-21T07:47:55.184959+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:16.528260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092223100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":337,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.195355+0000","last_change":"2024-09-21T07:34:25.085635+0000","last_active":"2024-09-21T07:47:55.195355+0000","last_peered":"2024-09-21T07:47:55.195355+0000","last_clean":"2024-09-21T07:47:55.195355+0000","last_became_active":"2024-09-21T07:34:23.273566+0000","last_became_peered":"2024-09-21T07:34:23.273566+0000","last_unstale":"2024-09-21T07:47:55.195355+0000","last_undegraded":"2024-09-21T07:47:55.195355+0000","last_fullsized":"2024-09-21T07:47:55.195355+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:14.142272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079897300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.187707+0000","last_change":"2024-09-21T07:47:55.182577+0000","last_active":"2024-09-21T07:47:55.187707+0000","last_peered":"2024-09-21T07:47:55.187707+0000","last_clean":"2024-09-21T07:47:55.187707+0000","last_became_active":"2024-09-21T07:47:55.180283+0000","last_became_peered":"2024-09-21T07:47:55.180283+0000","last_unstale":"2024-09-21T07:47:55.187707+0000","last_undegraded":"2024-09-21T07:47:55.187707+0000","last_fullsized":"2024-09-21T07:47:55.187707+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:40.604618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":13,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.178451+0000","last_change":"2024-09-21T07:47:54.185311+0000","last_active":"2024-09-21T07:47:55.178451+0000","last_peered":"2024-09-21T07:47:55.178451+0000","last_clean":"2024-09-21T07:47:55.178451+0000","last_became_active":"2024-09-21T07:47:54.184612+0000","last_became_peered":"2024-09-21T07:47:54.184612+0000","last_unstale":"2024-09-21T07:47:55.178451+0000","last_undegraded":"2024-09-21T07:47:55.178451+0000","last_fullsized":"2024-09-21T07:47:55.178451+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:51:48.140187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":279,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.191876+0000","last_change":"2024-09-21T07:35:27.878728+0000","last_active":"2024-09-21T07:47:55.191876+0000","last_peered":"2024-09-21T07:47:55.191876+0000","last_clean":"2024-09-21T07:47:55.191876+0000","last_became_active":"2024-09-21T07:35:27.878335+0000","last_became_peered":"2024-09-21T07:35:27.878335+0000","last_unstale":"2024-09-21T07:47:55.191876+0000","last_undegraded":"2024-09-21T07:47:55.191876+0000","last_fullsized":"2024-09-21T07:47:55.191876+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:25:37.731771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":300,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.171927+0000","last_change":"2024-09-21T07:35:28.858418+0000","last_active":"2024-09-21T07:47:56.171927+0000","last_peered":"2024-09-21T07:47:56.171927+0000","last_clean":"2024-09-21T07:47:56.171927+0000","last_became_active":"2024-09-21T07:35:26.876944+0000","last_became_peered":"2024-09-21T07:35:26.876944+0000","last_unstale":"2024-09-21T07:47:56.171927+0000","last_undegraded":"2024-09-21T07:47:56.171927+0000","last_fullsized":"2024-09-21T07:47:56.171927+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:04.514304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090862799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":332,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.185850+0000","last_change":"2024-09-21T07:34:24.597926+0000","last_active":"2024-09-21T07:47:55.185850+0000","last_peered":"2024-09-21T07:47:55.185850+0000","last_clean":"2024-09-21T07:47:55.185850+0000","last_became_active":"2024-09-21T07:34:23.268926+0000","last_became_peered":"2024-09-21T07:34:23.268926+0000","last_unstale":"2024-09-21T07:47:55.185850+0000","last_undegraded":"2024-09-21T07:47:55.185850+0000","last_fullsized":"2024-09-21T07:47:55.185850+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:46.190752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101851,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.168546+0000","last_change":"2024-09-21T07:47:55.192828+0000","last_active":"2024-09-21T07:47:56.168546+0000","last_peered":"2024-09-21T07:47:56.168546+0000","last_clean":"2024-09-21T07:47:56.168546+0000","last_became_active":"2024-09-21T07:47:55.192534+0000","last_became_peered":"2024-09-21T07:47:55.192534+0000","last_unstale":"2024-09-21T07:47:56.168546+0000","last_undegraded":"2024-09-21T07:47:56.168546+0000","last_fullsized":"2024-09-21T07:47:56.168546+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:44:08.148357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"205'2","reported_seq":15,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.206365+0000","last_change":"2024-09-21T07:47:54.190402+0000","last_active":"2024-09-21T07:47:55.206365+0000","last_peered":"2024-09-21T07:47:55.206365+0000","last_clean":"2024-09-21T07:47:55.206365+0000","last_became_active":"2024-09-21T07:47:54.190057+0000","last_became_peered":"2024-09-21T07:47:54.190057+0000","last_unstale":"2024-09-21T07:47:55.206365+0000","last_undegraded":"2024-09-21T07:47:55.206365+0000","last_fullsized":"2024-09-21T07:47:55.206365+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:10.894204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":277,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.186287+0000","last_change":"2024-09-21T07:35:27.877426+0000","last_active":"2024-09-21T07:47:55.186287+0000","last_peered":"2024-09-21T07:47:55.186287+0000","last_clean":"2024-09-21T07:47:55.186287+0000","last_became_active":"2024-09-21T07:35:27.877050+0000","last_became_peered":"2024-09-21T07:35:27.877050+0000","last_unstale":"2024-09-21T07:47:55.186287+0000","last_undegraded":"2024-09-21T07:47:55.186287+0000","last_fullsized":"2024-09-21T07:47:55.186287+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:36.268334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":286,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.178267+0000","last_change":"2024-09-21T07:35:28.862273+0000","last_active":"2024-09-21T07:47:55.178267+0000","last_peered":"2024-09-21T07:47:55.178267+0000","last_clean":"2024-09-21T07:47:55.178267+0000","last_became_active":"2024-09-21T07:35:26.874011+0000","last_became_peered":"2024-09-21T07:35:26.874011+0000","last_unstale":"2024-09-21T07:47:55.178267+0000","last_undegraded":"2024-09-21T07:47:55.178267+0000","last_fullsized":"2024-09-21T07:47:55.178267+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:11.986950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071644799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":344,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172240+0000","last_change":"2024-09-21T07:34:24.259794+0000","last_active":"2024-09-21T07:47:56.172240+0000","last_peered":"2024-09-21T07:47:56.172240+0000","last_clean":"2024-09-21T07:47:56.172240+0000","last_became_active":"2024-09-21T07:34:23.265679+0000","last_became_peered":"2024-09-21T07:34:23.265679+0000","last_unstale":"2024-09-21T07:47:56.172240+0000","last_undegraded":"2024-09-21T07:47:56.172240+0000","last_fullsized":"2024-09-21T07:47:56.172240+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:02.575796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001298518,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.204175+0000","last_change":"2024-09-21T07:47:55.195673+0000","last_active":"2024-09-21T07:47:55.204175+0000","last_peered":"2024-09-21T07:47:55.204175+0000","last_clean":"2024-09-21T07:47:55.204175+0000","last_became_active":"2024-09-21T07:47:55.195332+0000","last_became_peered":"2024-09-21T07:47:55.195332+0000","last_unstale":"2024-09-21T07:47:55.204175+0000","last_undegraded":"2024-09-21T07:47:55.204175+0000","last_fullsized":"2024-09-21T07:47:55.204175+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:12.292785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":13,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.178931+0000","last_change":"2024-09-21T07:47:54.193515+0000","last_active":"2024-09-21T07:47:55.178931+0000","last_peered":"2024-09-21T07:47:55.178931+0000","last_clean":"2024-09-21T07:47:55.178931+0000","last_became_active":"2024-09-21T07:47:54.193198+0000","last_became_peered":"2024-09-21T07:47:54.193198+0000","last_unstale":"2024-09-21T07:47:55.178931+0000","last_undegraded":"2024-09-21T07:47:55.178931+0000","last_fullsized":"2024-09-21T07:47:55.178931+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:29:13.611283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":282,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.180198+0000","last_change":"2024-09-21T07:35:27.877725+0000","last_active":"2024-09-21T07:47:55.180198+0000","last_peered":"2024-09-21T07:47:55.180198+0000","last_clean":"2024-09-21T07:47:55.180198+0000","last_became_active":"2024-09-21T07:35:27.877398+0000","last_became_peered":"2024-09-21T07:35:27.877398+0000","last_unstale":"2024-09-21T07:47:55.180198+0000","last_undegraded":"2024-09-21T07:47:55.180198+0000","last_fullsized":"2024-09-21T07:47:55.180198+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:05.859762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":292,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.171363+0000","last_change":"2024-09-21T07:35:28.860612+0000","last_active":"2024-09-21T07:47:56.171363+0000","last_peered":"2024-09-21T07:47:56.171363+0000","last_clean":"2024-09-21T07:47:56.171363+0000","last_became_active":"2024-09-21T07:35:26.876562+0000","last_became_peered":"2024-09-21T07:35:26.876562+0000","last_unstale":"2024-09-21T07:47:56.171363+0000","last_undegraded":"2024-09-21T07:47:56.171363+0000","last_fullsized":"2024-09-21T07:47:56.171363+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:06.824331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012252470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":340,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.184348+0000","last_change":"2024-09-21T07:34:24.260330+0000","last_active":"2024-09-21T07:47:55.184348+0000","last_peered":"2024-09-21T07:47:55.184348+0000","last_clean":"2024-09-21T07:47:55.184348+0000","last_became_active":"2024-09-21T07:34:23.637210+0000","last_became_peered":"2024-09-21T07:34:23.637210+0000","last_unstale":"2024-09-21T07:47:55.184348+0000","last_undegraded":"2024-09-21T07:47:55.184348+0000","last_fullsized":"2024-09-21T07:47:55.184348+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.488451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001537093,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.204587+0000","last_change":"2024-09-21T07:47:55.196976+0000","last_active":"2024-09-21T07:47:55.204587+0000","last_peered":"2024-09-21T07:47:55.204587+0000","last_clean":"2024-09-21T07:47:55.204587+0000","last_became_active":"2024-09-21T07:47:55.196562+0000","last_became_peered":"2024-09-21T07:47:55.196562+0000","last_unstale":"2024-09-21T07:47:55.204587+0000","last_undegraded":"2024-09-21T07:47:55.204587+0000","last_fullsized":"2024-09-21T07:47:55.204587+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:26:38.838493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":13,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.167049+0000","last_change":"2024-09-21T07:47:54.191403+0000","last_active":"2024-09-21T07:47:55.167049+0000","last_peered":"2024-09-21T07:47:55.167049+0000","last_clean":"2024-09-21T07:47:55.167049+0000","last_became_active":"2024-09-21T07:47:54.191082+0000","last_became_peered":"2024-09-21T07:47:54.191082+0000","last_unstale":"2024-09-21T07:47:55.167049+0000","last_undegraded":"2024-09-21T07:47:55.167049+0000","last_fullsized":"2024-09-21T07:47:55.167049+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:47.109877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":279,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.178433+0000","last_change":"2024-09-21T07:35:27.874757+0000","last_active":"2024-09-21T07:47:55.178433+0000","last_peered":"2024-09-21T07:47:55.178433+0000","last_clean":"2024-09-21T07:47:55.178433+0000","last_became_active":"2024-09-21T07:35:27.874360+0000","last_became_peered":"2024-09-21T07:35:27.874360+0000","last_unstale":"2024-09-21T07:47:55.178433+0000","last_undegraded":"2024-09-21T07:47:55.178433+0000","last_fullsized":"2024-09-21T07:47:55.178433+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:17.646912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":287,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.176957+0000","last_change":"2024-09-21T07:35:29.493431+0000","last_active":"2024-09-21T07:47:55.176957+0000","last_peered":"2024-09-21T07:47:55.176957+0000","last_clean":"2024-09-21T07:47:55.176957+0000","last_became_active":"2024-09-21T07:35:26.873691+0000","last_became_peered":"2024-09-21T07:35:26.873691+0000","last_unstale":"2024-09-21T07:47:55.176957+0000","last_undegraded":"2024-09-21T07:47:55.176957+0000","last_fullsized":"2024-09-21T07:47:55.176957+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:51.705051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001162464,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":332,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.184255+0000","last_change":"2024-09-21T07:34:24.597286+0000","last_active":"2024-09-21T07:47:55.184255+0000","last_peered":"2024-09-21T07:47:55.184255+0000","last_clean":"2024-09-21T07:47:55.184255+0000","last_became_active":"2024-09-21T07:34:23.267133+0000","last_became_peered":"2024-09-21T07:34:23.267133+0000","last_unstale":"2024-09-21T07:47:55.184255+0000","last_undegraded":"2024-09-21T07:47:55.184255+0000","last_fullsized":"2024-09-21T07:47:55.184255+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:18.257010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172530+0000","last_change":"2024-09-21T07:47:55.175551+0000","last_active":"2024-09-21T07:47:56.172530+0000","last_peered":"2024-09-21T07:47:56.172530+0000","last_clean":"2024-09-21T07:47:56.172530+0000","last_became_active":"2024-09-21T07:47:55.175221+0000","last_became_peered":"2024-09-21T07:47:55.175221+0000","last_unstale":"2024-09-21T07:47:56.172530+0000","last_undegraded":"2024-09-21T07:47:56.172530+0000","last_fullsized":"2024-09-21T07:47:56.172530+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:34.371499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":13,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.170091+0000","last_change":"2024-09-21T07:47:54.194068+0000","last_active":"2024-09-21T07:47:55.170091+0000","last_peered":"2024-09-21T07:47:55.170091+0000","last_clean":"2024-09-21T07:47:55.170091+0000","last_became_active":"2024-09-21T07:47:54.193741+0000","last_became_peered":"2024-09-21T07:47:54.193741+0000","last_unstale":"2024-09-21T07:47:55.170091+0000","last_undegraded":"2024-09-21T07:47:55.170091+0000","last_fullsized":"2024-09-21T07:47:55.170091+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:34:42.075929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":293,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.171664+0000","last_change":"2024-09-21T07:35:27.864458+0000","last_active":"2024-09-21T07:47:56.171664+0000","last_peered":"2024-09-21T07:47:56.171664+0000","last_clean":"2024-09-21T07:47:56.171664+0000","last_became_active":"2024-09-21T07:35:27.864136+0000","last_became_peered":"2024-09-21T07:35:27.864136+0000","last_unstale":"2024-09-21T07:47:56.171664+0000","last_undegraded":"2024-09-21T07:47:56.171664+0000","last_fullsized":"2024-09-21T07:47:56.171664+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:20.162240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":300,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172911+0000","last_change":"2024-09-21T07:35:28.858784+0000","last_active":"2024-09-21T07:47:56.172911+0000","last_peered":"2024-09-21T07:47:56.172911+0000","last_clean":"2024-09-21T07:47:56.172911+0000","last_became_active":"2024-09-21T07:35:26.876890+0000","last_became_peered":"2024-09-21T07:35:26.876890+0000","last_unstale":"2024-09-21T07:47:56.172911+0000","last_undegraded":"2024-09-21T07:47:56.172911+0000","last_fullsized":"2024-09-21T07:47:56.172911+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:40.738327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064543499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":344,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.169957+0000","last_change":"2024-09-21T07:34:24.258712+0000","last_active":"2024-09-21T07:47:55.169957+0000","last_peered":"2024-09-21T07:47:55.169957+0000","last_clean":"2024-09-21T07:47:55.169957+0000","last_became_active":"2024-09-21T07:34:23.274692+0000","last_became_peered":"2024-09-21T07:34:23.274692+0000","last_unstale":"2024-09-21T07:47:55.169957+0000","last_undegraded":"2024-09-21T07:47:55.169957+0000","last_fullsized":"2024-09-21T07:47:55.169957+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:43.279454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018051479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172147+0000","last_change":"2024-09-21T07:47:55.173038+0000","last_active":"2024-09-21T07:47:56.172147+0000","last_peered":"2024-09-21T07:47:56.172147+0000","last_clean":"2024-09-21T07:47:56.172147+0000","last_became_active":"2024-09-21T07:47:55.172601+0000","last_became_peered":"2024-09-21T07:47:55.172601+0000","last_unstale":"2024-09-21T07:47:56.172147+0000","last_undegraded":"2024-09-21T07:47:56.172147+0000","last_fullsized":"2024-09-21T07:47:56.172147+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:24:08.769397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"205'2","reported_seq":15,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.208297+0000","last_change":"2024-09-21T07:47:54.190101+0000","last_active":"2024-09-21T07:47:55.208297+0000","last_peered":"2024-09-21T07:47:55.208297+0000","last_clean":"2024-09-21T07:47:55.208297+0000","last_became_active":"2024-09-21T07:47:54.189757+0000","last_became_peered":"2024-09-21T07:47:54.189757+0000","last_unstale":"2024-09-21T07:47:55.208297+0000","last_undegraded":"2024-09-21T07:47:55.208297+0000","last_fullsized":"2024-09-21T07:47:55.208297+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:47.948529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":283,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.166853+0000","last_change":"2024-09-21T07:35:27.878020+0000","last_active":"2024-09-21T07:47:55.166853+0000","last_peered":"2024-09-21T07:47:55.166853+0000","last_clean":"2024-09-21T07:47:55.166853+0000","last_became_active":"2024-09-21T07:35:27.877532+0000","last_became_peered":"2024-09-21T07:35:27.877532+0000","last_unstale":"2024-09-21T07:47:55.166853+0000","last_undegraded":"2024-09-21T07:47:55.166853+0000","last_fullsized":"2024-09-21T07:47:55.166853+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:19.756073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":292,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172277+0000","last_change":"2024-09-21T07:35:28.861049+0000","last_active":"2024-09-21T07:47:56.172277+0000","last_peered":"2024-09-21T07:47:56.172277+0000","last_clean":"2024-09-21T07:47:56.172277+0000","last_became_active":"2024-09-21T07:35:26.876819+0000","last_became_peered":"2024-09-21T07:35:26.876819+0000","last_unstale":"2024-09-21T07:47:56.172277+0000","last_undegraded":"2024-09-21T07:47:56.172277+0000","last_fullsized":"2024-09-21T07:47:56.172277+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:19.633803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055780599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":337,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.178544+0000","last_change":"2024-09-21T07:34:25.084849+0000","last_active":"2024-09-21T07:47:55.178544+0000","last_peered":"2024-09-21T07:47:55.178544+0000","last_clean":"2024-09-21T07:47:55.178544+0000","last_became_active":"2024-09-21T07:34:23.277713+0000","last_became_peered":"2024-09-21T07:34:23.277713+0000","last_unstale":"2024-09-21T07:47:55.178544+0000","last_undegraded":"2024-09-21T07:47:55.178544+0000","last_fullsized":"2024-09-21T07:47:55.178544+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:05.811810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059079499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.169971+0000","last_change":"2024-09-21T07:47:55.184966+0000","last_active":"2024-09-21T07:47:56.169971+0000","last_peered":"2024-09-21T07:47:56.169971+0000","last_clean":"2024-09-21T07:47:56.169971+0000","last_became_active":"2024-09-21T07:47:55.184645+0000","last_became_peered":"2024-09-21T07:47:55.184645+0000","last_unstale":"2024-09-21T07:47:56.169971+0000","last_undegraded":"2024-09-21T07:47:56.169971+0000","last_fullsized":"2024-09-21T07:47:56.169971+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:26:35.775546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"206'1","reported_seq":16,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:57.171463+0000","last_change":"2024-09-21T07:47:54.198999+0000","last_active":"2024-09-21T07:47:57.171463+0000","last_peered":"2024-09-21T07:47:57.171463+0000","last_clean":"2024-09-21T07:47:57.171463+0000","last_became_active":"2024-09-21T07:47:54.198632+0000","last_became_peered":"2024-09-21T07:47:54.198632+0000","last_unstale":"2024-09-21T07:47:57.171463+0000","last_undegraded":"2024-09-21T07:47:57.171463+0000","last_fullsized":"2024-09-21T07:47:57.171463+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:10.747530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":284,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.169730+0000","last_change":"2024-09-21T07:35:27.884248+0000","last_active":"2024-09-21T07:47:56.169730+0000","last_peered":"2024-09-21T07:47:56.169730+0000","last_clean":"2024-09-21T07:47:56.169730+0000","last_became_active":"2024-09-21T07:35:27.883861+0000","last_became_peered":"2024-09-21T07:35:27.883861+0000","last_unstale":"2024-09-21T07:47:56.169730+0000","last_undegraded":"2024-09-21T07:47:56.169730+0000","last_fullsized":"2024-09-21T07:47:56.169730+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":288,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.179827+0000","last_change":"2024-09-21T07:35:29.207135+0000","last_active":"2024-09-21T07:47:55.179827+0000","last_peered":"2024-09-21T07:47:55.179827+0000","last_clean":"2024-09-21T07:47:55.179827+0000","last_became_active":"2024-09-21T07:35:26.872787+0000","last_became_peered":"2024-09-21T07:35:26.872787+0000","last_unstale":"2024-09-21T07:47:55.179827+0000","last_undegraded":"2024-09-21T07:47:55.179827+0000","last_fullsized":"2024-09-21T07:47:55.179827+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:27.543217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011157509999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":352,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.171742+0000","last_change":"2024-09-21T07:34:24.538610+0000","last_active":"2024-09-21T07:47:56.171742+0000","last_peered":"2024-09-21T07:47:56.171742+0000","last_clean":"2024-09-21T07:47:56.171742+0000","last_became_active":"2024-09-21T07:34:23.259700+0000","last_became_peered":"2024-09-21T07:34:23.259700+0000","last_unstale":"2024-09-21T07:47:56.171742+0000","last_undegraded":"2024-09-21T07:47:56.171742+0000","last_fullsized":"2024-09-21T07:47:56.171742+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:57.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070522399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.169679+0000","last_change":"2024-09-21T07:47:55.202961+0000","last_active":"2024-09-21T07:47:56.169679+0000","last_peered":"2024-09-21T07:47:56.169679+0000","last_clean":"2024-09-21T07:47:56.169679+0000","last_became_active":"2024-09-21T07:47:55.202640+0000","last_became_peered":"2024-09-21T07:47:55.202640+0000","last_unstale":"2024-09-21T07:47:56.169679+0000","last_undegraded":"2024-09-21T07:47:56.169679+0000","last_fullsized":"2024-09-21T07:47:56.169679+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:41:07.382956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":15,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.171751+0000","last_change":"2024-09-21T07:47:54.191694+0000","last_active":"2024-09-21T07:47:56.171751+0000","last_peered":"2024-09-21T07:47:56.171751+0000","last_clean":"2024-09-21T07:47:56.171751+0000","last_became_active":"2024-09-21T07:47:54.191392+0000","last_became_peered":"2024-09-21T07:47:54.191392+0000","last_unstale":"2024-09-21T07:47:56.171751+0000","last_undegraded":"2024-09-21T07:47:56.171751+0000","last_fullsized":"2024-09-21T07:47:56.171751+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:39.417382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":293,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172466+0000","last_change":"2024-09-21T07:35:27.868024+0000","last_active":"2024-09-21T07:47:56.172466+0000","last_peered":"2024-09-21T07:47:56.172466+0000","last_clean":"2024-09-21T07:47:56.172466+0000","last_became_active":"2024-09-21T07:35:27.867339+0000","last_became_peered":"2024-09-21T07:35:27.867339+0000","last_unstale":"2024-09-21T07:47:56.172466+0000","last_undegraded":"2024-09-21T07:47:56.172466+0000","last_fullsized":"2024-09-21T07:47:56.172466+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":300,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.172031+0000","last_change":"2024-09-21T07:35:28.858505+0000","last_active":"2024-09-21T07:47:56.172031+0000","last_peered":"2024-09-21T07:47:56.172031+0000","last_clean":"2024-09-21T07:47:56.172031+0000","last_became_active":"2024-09-21T07:35:26.877320+0000","last_became_peered":"2024-09-21T07:35:26.877320+0000","last_unstale":"2024-09-21T07:47:56.172031+0000","last_undegraded":"2024-09-21T07:47:56.172031+0000","last_fullsized":"2024-09-21T07:47:56.172031+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:53.672782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244119999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":333,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.188231+0000","last_change":"2024-09-21T07:34:24.597597+0000","last_active":"2024-09-21T07:47:55.188231+0000","last_peered":"2024-09-21T07:47:55.188231+0000","last_clean":"2024-09-21T07:47:55.188231+0000","last_became_active":"2024-09-21T07:34:23.635204+0000","last_became_peered":"2024-09-21T07:34:23.635204+0000","last_unstale":"2024-09-21T07:47:55.188231+0000","last_undegraded":"2024-09-21T07:47:55.188231+0000","last_fullsized":"2024-09-21T07:47:55.188231+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:48.644789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081139199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.205770+0000","last_change":"2024-09-21T07:47:55.202438+0000","last_active":"2024-09-21T07:47:55.205770+0000","last_peered":"2024-09-21T07:47:55.205770+0000","last_clean":"2024-09-21T07:47:55.205770+0000","last_became_active":"2024-09-21T07:47:55.201888+0000","last_became_peered":"2024-09-21T07:47:55.201888+0000","last_unstale":"2024-09-21T07:47:55.205770+0000","last_undegraded":"2024-09-21T07:47:55.205770+0000","last_fullsized":"2024-09-21T07:47:55.205770+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:59.420037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":13,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.178702+0000","last_change":"2024-09-21T07:47:54.193368+0000","last_active":"2024-09-21T07:47:55.178702+0000","last_peered":"2024-09-21T07:47:55.178702+0000","last_clean":"2024-09-21T07:47:55.178702+0000","last_became_active":"2024-09-21T07:47:54.192921+0000","last_became_peered":"2024-09-21T07:47:54.192921+0000","last_unstale":"2024-09-21T07:47:55.178702+0000","last_undegraded":"2024-09-21T07:47:55.178702+0000","last_fullsized":"2024-09-21T07:47:55.178702+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:05:07.952836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":281,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.179610+0000","last_change":"2024-09-21T07:35:27.877503+0000","last_active":"2024-09-21T07:47:55.179610+0000","last_peered":"2024-09-21T07:47:55.179610+0000","last_clean":"2024-09-21T07:47:55.179610+0000","last_became_active":"2024-09-21T07:35:27.875775+0000","last_became_peered":"2024-09-21T07:35:27.875775+0000","last_unstale":"2024-09-21T07:47:55.179610+0000","last_undegraded":"2024-09-21T07:47:55.179610+0000","last_fullsized":"2024-09-21T07:47:55.179610+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:49.291023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":287,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.175627+0000","last_change":"2024-09-21T07:35:29.491412+0000","last_active":"2024-09-21T07:47:55.175627+0000","last_peered":"2024-09-21T07:47:55.175627+0000","last_clean":"2024-09-21T07:47:55.175627+0000","last_became_active":"2024-09-21T07:35:26.876404+0000","last_became_peered":"2024-09-21T07:35:26.876404+0000","last_unstale":"2024-09-21T07:47:55.175627+0000","last_undegraded":"2024-09-21T07:47:55.175627+0000","last_fullsized":"2024-09-21T07:47:55.175627+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063918000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":345,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.166213+0000","last_change":"2024-09-21T07:34:24.259026+0000","last_active":"2024-09-21T07:47:55.166213+0000","last_peered":"2024-09-21T07:47:55.166213+0000","last_clean":"2024-09-21T07:47:55.166213+0000","last_became_active":"2024-09-21T07:34:23.635456+0000","last_became_peered":"2024-09-21T07:34:23.635456+0000","last_unstale":"2024-09-21T07:47:55.166213+0000","last_undegraded":"2024-09-21T07:47:55.166213+0000","last_fullsized":"2024-09-21T07:47:55.166213+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:09.592752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018654419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.196848+0000","last_change":"2024-09-21T07:47:55.191605+0000","last_active":"2024-09-21T07:47:55.196848+0000","last_peered":"2024-09-21T07:47:55.196848+0000","last_clean":"2024-09-21T07:47:55.196848+0000","last_became_active":"2024-09-21T07:47:55.188078+0000","last_became_peered":"2024-09-21T07:47:55.188078+0000","last_unstale":"2024-09-21T07:47:55.196848+0000","last_undegraded":"2024-09-21T07:47:55.196848+0000","last_fullsized":"2024-09-21T07:47:55.196848+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:41.382319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":13,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.176728+0000","last_change":"2024-09-21T07:47:54.190454+0000","last_active":"2024-09-21T07:47:55.176728+0000","last_peered":"2024-09-21T07:47:55.176728+0000","last_clean":"2024-09-21T07:47:55.176728+0000","last_became_active":"2024-09-21T07:47:54.189988+0000","last_became_peered":"2024-09-21T07:47:54.189988+0000","last_unstale":"2024-09-21T07:47:55.176728+0000","last_undegraded":"2024-09-21T07:47:55.176728+0000","last_fullsized":"2024-09-21T07:47:55.176728+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:06:43.125613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":281,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.179887+0000","last_change":"2024-09-21T07:35:27.871984+0000","last_active":"2024-09-21T07:47:55.179887+0000","last_peered":"2024-09-21T07:47:55.179887+0000","last_clean":"2024-09-21T07:47:55.179887+0000","last_became_active":"2024-09-21T07:35:27.870336+0000","last_became_peered":"2024-09-21T07:35:27.870336+0000","last_unstale":"2024-09-21T07:47:55.179887+0000","last_undegraded":"2024-09-21T07:47:55.179887+0000","last_fullsized":"2024-09-21T07:47:55.179887+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:57.120479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":292,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.171819+0000","last_change":"2024-09-21T07:35:28.860351+0000","last_active":"2024-09-21T07:47:56.171819+0000","last_peered":"2024-09-21T07:47:56.171819+0000","last_clean":"2024-09-21T07:47:56.171819+0000","last_became_active":"2024-09-21T07:35:26.872046+0000","last_became_peered":"2024-09-21T07:35:26.872046+0000","last_unstale":"2024-09-21T07:47:56.171819+0000","last_undegraded":"2024-09-21T07:47:56.171819+0000","last_fullsized":"2024-09-21T07:47:56.171819+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:47.074327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098384399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":340,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.182385+0000","last_change":"2024-09-21T07:34:24.260300+0000","last_active":"2024-09-21T07:47:55.182385+0000","last_peered":"2024-09-21T07:47:55.182385+0000","last_clean":"2024-09-21T07:47:55.182385+0000","last_became_active":"2024-09-21T07:34:23.270128+0000","last_became_peered":"2024-09-21T07:34:23.270128+0000","last_unstale":"2024-09-21T07:47:55.182385+0000","last_undegraded":"2024-09-21T07:47:55.182385+0000","last_fullsized":"2024-09-21T07:47:55.182385+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:25.260931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00176651,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.204913+0000","last_change":"2024-09-21T07:47:55.199347+0000","last_active":"2024-09-21T07:47:55.204913+0000","last_peered":"2024-09-21T07:47:55.204913+0000","last_clean":"2024-09-21T07:47:55.204913+0000","last_became_active":"2024-09-21T07:47:55.198641+0000","last_became_peered":"2024-09-21T07:47:55.198641+0000","last_unstale":"2024-09-21T07:47:55.204913+0000","last_undegraded":"2024-09-21T07:47:55.204913+0000","last_fullsized":"2024-09-21T07:47:55.204913+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:54.150910+0000","last_clean_scrub_stamp":"2024-09-21T07:47:54.150910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:11:34.236907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":13,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.175545+0000","last_change":"2024-09-21T07:47:54.194255+0000","last_active":"2024-09-21T07:47:55.175545+0000","last_peered":"2024-09-21T07:47:55.175545+0000","last_clean":"2024-09-21T07:47:55.175545+0000","last_became_active":"2024-09-21T07:47:54.193776+0000","last_became_peered":"2024-09-21T07:47:54.193776+0000","last_unstale":"2024-09-21T07:47:55.175545+0000","last_undegraded":"2024-09-21T07:47:55.175545+0000","last_fullsized":"2024-09-21T07:47:55.175545+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:53.147924+0000","last_clean_scrub_stamp":"2024-09-21T07:47:53.147924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:16:13.117235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":281,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.180262+0000","last_change":"2024-09-21T07:35:27.877861+0000","last_active":"2024-09-21T07:47:55.180262+0000","last_peered":"2024-09-21T07:47:55.180262+0000","last_clean":"2024-09-21T07:47:55.180262+0000","last_became_active":"2024-09-21T07:35:27.877434+0000","last_became_peered":"2024-09-21T07:35:27.877434+0000","last_unstale":"2024-09-21T07:47:55.180262+0000","last_undegraded":"2024-09-21T07:47:55.180262+0000","last_fullsized":"2024-09-21T07:47:55.180262+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":286,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-21T07:47:55.181321+0000","last_change":"2024-09-21T07:35:28.862954+0000","last_active":"2024-09-21T07:47:55.181321+0000","last_peered":"2024-09-21T07:47:55.181321+0000","last_clean":"2024-09-21T07:47:55.181321+0000","last_became_active":"2024-09-21T07:35:26.870613+0000","last_became_peered":"2024-09-21T07:35:26.870613+0000","last_unstale":"2024-09-21T07:47:55.181321+0000","last_undegraded":"2024-09-21T07:47:55.181321+0000","last_fullsized":"2024-09-21T07:47:55.181321+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:06.901043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059842100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":342,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-21T07:47:56.170048+0000","last_change":"2024-09-21T07:34:25.028904+0000","last_active":"2024-09-21T07:47:56.170048+0000","last_peered":"2024-09-21T07:47:56.170048+0000","last_clean":"2024-09-21T07:47:56.170048+0000","last_became_active":"2024-09-21T07:34:23.260942+0000","last_became_peered":"2024-09-21T07:34:23.260942+0000","last_unstale":"2024-09-21T07:47:56.170048+0000","last_undegraded":"2024-09-21T07:47:56.170048+0000","last_fullsized":"2024-09-21T07:47:56.170048+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:22.709328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060553899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509622,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66220,"kb_used_data":3096,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94305620,"statfs":{"total":96636764160,"available":96568954880,"internally_reserved":0,"allocated":3170304,"data_stored":1962474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509624,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51760,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320080,"statfs":{"total":96636764160,"available":96583761920,"internally_reserved":0,"allocated":2580480,"data_stored":1372176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509622,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33736,"kb_used_data":2544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338104,"statfs":{"total":96636764160,"available":96602218496,"internally_reserved":0,"allocated":2605056,"data_stored":1381291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509623,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29564,"kb_used_data":2532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342276,"statfs":{"total":96636764160,"available":96606490624,"internally_reserved":0,"allocated":2592768,"data_stored":1380239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509624,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46200,"kb_used_data":2528,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94325640,"statfs":{"total":96636764160,"available":96589455360,"internally_reserved":0,"allocated":2588672,"data_stored":1379657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509624,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52352,"kb_used_data":3112,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319488,"statfs":{"total":96636764160,"available":96583155712,"internally_reserved":0,"allocated":3186688,"data_stored":1963202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542327,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51772,"kb_used_data":2532,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320068,"statfs":{"total":96636764160,"available":96583749632,"internally_reserved":0,"allocated":2592768,"data_stored":1373752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542326,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73972,"kb_used_data":2524,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297868,"statfs":{"total":96636764160,"available":96561016832,"internally_reserved":0,"allocated":2584576,"data_stored":1372266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:48:00.213 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T07:48:00.213 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T07:48:00.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.477+0000 7f31ec000640 1 -- 172.21.15.39:0/2436211391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31e40cb8a0 msgr2=0x7f31e4156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:00.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.477+0000 7f31ec000640 1 --2- 172.21.15.39:0/2436211391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31e40cb8a0 0x7f31e4156bb0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f31d0009830 tx=0x7f31d002a480 comp rx=0 tx=0).stop 2024-09-21T07:48:00.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.477+0000 7f31ec000640 1 -- 172.21.15.39:0/2436211391 shutdown_connections 2024-09-21T07:48:00.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.477+0000 7f31ec000640 1 --2- 172.21.15.39:0/2436211391 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f31e40ccc60 0x7f31e40cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:00.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.477+0000 7f31ec000640 1 --2- 172.21.15.39:0/2436211391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31e40cb8a0 0x7f31e4156bb0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:00.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.477+0000 7f31ec000640 1 -- 172.21.15.39:0/2436211391 >> 172.21.15.39:0/2436211391 conn(0x7f31e40b2460 msgr2=0x7f31e406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:00.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.477+0000 7f31ec000640 1 -- 172.21.15.39:0/2436211391 shutdown_connections 2024-09-21T07:48:00.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.477+0000 7f31ec000640 1 -- 172.21.15.39:0/2436211391 wait complete. 2024-09-21T07:48:00.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.480+0000 7f31ec000640 1 Processor -- start 2024-09-21T07:48:00.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.480+0000 7f31ec000640 1 -- start start 2024-09-21T07:48:00.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.481+0000 7f31ec000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31e40b30a0 0x7f31e41f2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:00.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.481+0000 7f31ec000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31e40cb8a0 0x7f31e409ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:00.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.481+0000 7f31ec000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f31e40ccc60 0x7f31e409d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:00.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.481+0000 7f31ec000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f31e40af410 con 0x7f31e40cb8a0 2024-09-21T07:48:00.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.481+0000 7f31ec000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f31e40af290 con 0x7f31e40b30a0 2024-09-21T07:48:00.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.481+0000 7f31ec000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f31e40af590 con 0x7f31e40ccc60 2024-09-21T07:48:00.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.481+0000 7f31e8c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31e40cb8a0 0x7f31e409ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:00.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.481+0000 7f31e8c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31e40cb8a0 0x7f31e409ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53684/0 (socket says 172.21.15.39:53684) 2024-09-21T07:48:00.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.482+0000 7f31e8c00640 1 -- 172.21.15.39:0/1546417007 learned_addr learned my addr 172.21.15.39:0/1546417007 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:00.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.482+0000 7f31ea000640 1 --2- 172.21.15.39:0/1546417007 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f31e40ccc60 0x7f31e409d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:00.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.482+0000 7f31e9600640 1 --2- 172.21.15.39:0/1546417007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31e40b30a0 0x7f31e41f2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:00.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.482+0000 7f31e8c00640 1 -- 172.21.15.39:0/1546417007 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f31e40ccc60 msgr2=0x7f31e409d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:00.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.482+0000 7f31e8c00640 1 --2- 172.21.15.39:0/1546417007 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f31e40ccc60 0x7f31e409d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:00.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.482+0000 7f31e8c00640 1 -- 172.21.15.39:0/1546417007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31e40b30a0 msgr2=0x7f31e41f2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:00.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.482+0000 7f31e8c00640 1 --2- 172.21.15.39:0/1546417007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31e40b30a0 0x7f31e41f2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:00.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.482+0000 7f31e8c00640 1 -- 172.21.15.39:0/1546417007 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e409bd80 con 0x7f31e40cb8a0 2024-09-21T07:48:00.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.482+0000 7f31e9600640 1 --2- 172.21.15.39:0/1546417007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31e40b30a0 0x7f31e41f2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:48:00.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.482+0000 7f31e8c00640 1 --2- 172.21.15.39:0/1546417007 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31e40cb8a0 0x7f31e409ce20 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f31d0000c00 tx=0x7f31d0033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:00.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.483+0000 7f31e1600640 1 -- 172.21.15.39:0/1546417007 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d0005c90 con 0x7f31e40cb8a0 2024-09-21T07:48:00.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.483+0000 7f31e1600640 1 -- 172.21.15.39:0/1546417007 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f31d0031e50 con 0x7f31e40cb8a0 2024-09-21T07:48:00.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.483+0000 7f31e1600640 1 -- 172.21.15.39:0/1546417007 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d00332f0 con 0x7f31e40cb8a0 2024-09-21T07:48:00.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.483+0000 7f31ec000640 1 -- 172.21.15.39:0/1546417007 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31e409c060 con 0x7f31e40cb8a0 2024-09-21T07:48:00.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.483+0000 7f31ec000640 1 -- 172.21.15.39:0/1546417007 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31e409c620 con 0x7f31e40cb8a0 2024-09-21T07:48:00.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.484+0000 7f31ec000640 1 -- 172.21.15.39:0/1546417007 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31e40afae0 con 0x7f31e40cb8a0 2024-09-21T07:48:00.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.486+0000 7f31e1600640 1 -- 172.21.15.39:0/1546417007 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f31d0033490 con 0x7f31e40cb8a0 2024-09-21T07:48:00.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.487+0000 7f31e1600640 1 --2- 172.21.15.39:0/1546417007 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f31c4078b90 0x7f31c407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:00.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.487+0000 7f31e1600640 1 -- 172.21.15.39:0/1546417007 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(206..206 src has 1..206) ==== 6960+0+0 (secure 0 0 0) 0x7f31d00bca40 con 0x7f31e40cb8a0 2024-09-21T07:48:00.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.488+0000 7f31e9600640 1 --2- 172.21.15.39:0/1546417007 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f31c4078b90 0x7f31c407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:00.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.488+0000 7f31e9600640 1 --2- 172.21.15.39:0/1546417007 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f31c4078b90 0x7f31c407b0a0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f31d80016c0 tx=0x7f31d8009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:00.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.491+0000 7f31e1600640 1 -- 172.21.15.39:0/1546417007 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f31d0052080 con 0x7f31e40cb8a0 2024-09-21T07:48:00.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.760+0000 7f31ec000640 1 -- 172.21.15.39:0/1546417007 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f31e40b06c0 con 0x7f31e40cb8a0 2024-09-21T07:48:00.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.763+0000 7f31e1600640 1 -- 172.21.15.39:0/1546417007 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) ==== 190+0+147 (secure 0 0 0) 0x7f31d003f030 con 0x7f31e40cb8a0 2024-09-21T07:48:00.765 INFO:teuthology.orchestra.run.smithi039.stdout:[client.0] 2024-09-21T07:48:00.766 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQB0du5mGfb1DBAA/aSMKR5or3xy53rB/iUjpA== 2024-09-21T07:48:00.766 INFO:teuthology.orchestra.run.smithi039.stdout: caps mds = "allow" 2024-09-21T07:48:00.766 INFO:teuthology.orchestra.run.smithi039.stdout: caps mon = "allow r" 2024-09-21T07:48:00.766 INFO:teuthology.orchestra.run.smithi039.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T07:48:00.766 INFO:teuthology.orchestra.run.smithi039.stderr:updated caps for client.0 2024-09-21T07:48:00.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.766+0000 7f31ec000640 1 -- 172.21.15.39:0/1546417007 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f31c4078b90 msgr2=0x7f31c407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:00.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.766+0000 7f31ec000640 1 --2- 172.21.15.39:0/1546417007 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f31c4078b90 0x7f31c407b0a0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f31d80016c0 tx=0x7f31d8009040 comp rx=0 tx=0).stop 2024-09-21T07:48:00.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.766+0000 7f31ec000640 1 -- 172.21.15.39:0/1546417007 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31e40cb8a0 msgr2=0x7f31e409ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:00.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.766+0000 7f31ec000640 1 --2- 172.21.15.39:0/1546417007 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31e40cb8a0 0x7f31e409ce20 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f31d0000c00 tx=0x7f31d0033c20 comp rx=0 tx=0).stop 2024-09-21T07:48:00.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.766+0000 7f31ec000640 1 -- 172.21.15.39:0/1546417007 shutdown_connections 2024-09-21T07:48:00.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.766+0000 7f31ec000640 1 --2- 172.21.15.39:0/1546417007 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f31c4078b90 0x7f31c407b0a0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:00.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.766+0000 7f31ec000640 1 --2- 172.21.15.39:0/1546417007 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f31e40ccc60 0x7f31e409d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:00.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.766+0000 7f31ec000640 1 --2- 172.21.15.39:0/1546417007 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31e40cb8a0 0x7f31e409ce20 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:00.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.766+0000 7f31ec000640 1 --2- 172.21.15.39:0/1546417007 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31e40b30a0 0x7f31e41f2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:00.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.766+0000 7f31ec000640 1 -- 172.21.15.39:0/1546417007 >> 172.21.15.39:0/1546417007 conn(0x7f31e40b2460 msgr2=0x7f31e4074830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:00.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.766+0000 7f31ec000640 1 -- 172.21.15.39:0/1546417007 shutdown_connections 2024-09-21T07:48:00.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:00.766+0000 7f31ec000640 1 -- 172.21.15.39:0/1546417007 wait complete. 2024-09-21T07:48:00.790 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:48:01.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.015+0000 7f3f49000640 1 -- 172.21.15.39:0/3477790917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f440d1580 msgr2=0x7f3f440d1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:01.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.015+0000 7f3f49000640 1 --2- 172.21.15.39:0/3477790917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f440d1580 0x7f3f440d1ac0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c009830 tx=0x7f3f2c02a410 comp rx=0 tx=0).stop 2024-09-21T07:48:01.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.015+0000 7f3f49000640 1 -- 172.21.15.39:0/3477790917 shutdown_connections 2024-09-21T07:48:01.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.015+0000 7f3f49000640 1 --2- 172.21.15.39:0/3477790917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f440d1580 0x7f3f440d1ac0 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.015+0000 7f3f49000640 1 -- 172.21.15.39:0/3477790917 >> 172.21.15.39:0/3477790917 conn(0x7f3f4409d7b0 msgr2=0x7f3f440d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:01.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.015+0000 7f3f49000640 1 -- 172.21.15.39:0/3477790917 shutdown_connections 2024-09-21T07:48:01.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.015+0000 7f3f49000640 1 -- 172.21.15.39:0/3477790917 wait complete. 2024-09-21T07:48:01.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.017+0000 7f3f49000640 1 Processor -- start 2024-09-21T07:48:01.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.018+0000 7f3f49000640 1 -- start start 2024-09-21T07:48:01.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.019+0000 7f3f49000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f440b3b60 0x7f3f440b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:01.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.019+0000 7f3f49000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3f440b5a90 0x7f3f441f8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:01.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.019+0000 7f3f49000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f441f8d70 0x7f3f44148540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:01.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.019+0000 7f3f49000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3f440b4800 con 0x7f3f441f8d70 2024-09-21T07:48:01.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.019+0000 7f3f49000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3f440b4680 con 0x7f3f440b3b60 2024-09-21T07:48:01.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.019+0000 7f3f49000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f3f440b4980 con 0x7f3f440b5a90 2024-09-21T07:48:01.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.019+0000 7f3f43e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f441f8d70 0x7f3f44148540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:01.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.019+0000 7f3f43e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f441f8d70 0x7f3f44148540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53698/0 (socket says 172.21.15.39:53698) 2024-09-21T07:48:01.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.019+0000 7f3f43e00640 1 -- 172.21.15.39:0/2265781045 learned_addr learned my addr 172.21.15.39:0/2265781045 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:01.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.019+0000 7f3f43400640 1 --2- 172.21.15.39:0/2265781045 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f440b3b60 0x7f3f440b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:01.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.020+0000 7f3f3be00640 1 --2- 172.21.15.39:0/2265781045 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3f440b5a90 0x7f3f441f8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:01.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.020+0000 7f3f43e00640 1 -- 172.21.15.39:0/2265781045 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3f440b5a90 msgr2=0x7f3f441f8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:01.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.020+0000 7f3f43e00640 1 --2- 172.21.15.39:0/2265781045 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3f440b5a90 0x7f3f441f8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.020+0000 7f3f43e00640 1 -- 172.21.15.39:0/2265781045 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f440b3b60 msgr2=0x7f3f440b26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:01.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.020+0000 7f3f43e00640 1 --2- 172.21.15.39:0/2265781045 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f440b3b60 0x7f3f440b26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.020+0000 7f3f43e00640 1 -- 172.21.15.39:0/2265781045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f440cb8f0 con 0x7f3f441f8d70 2024-09-21T07:48:01.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.020+0000 7f3f43e00640 1 --2- 172.21.15.39:0/2265781045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f441f8d70 0x7f3f44148540 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00ba10 tx=0x7f3f3c00bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:01.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.020+0000 7f3f3b400640 1 -- 172.21.15.39:0/2265781045 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c019070 con 0x7f3f441f8d70 2024-09-21T07:48:01.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.020+0000 7f3f3b400640 1 -- 172.21.15.39:0/2265781045 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f3c00cc30 con 0x7f3f441f8d70 2024-09-21T07:48:01.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.020+0000 7f3f49000640 1 -- 172.21.15.39:0/2265781045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f440b1bc0 con 0x7f3f441f8d70 2024-09-21T07:48:01.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.020+0000 7f3f3b400640 1 -- 172.21.15.39:0/2265781045 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c00ce60 con 0x7f3f441f8d70 2024-09-21T07:48:01.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.021+0000 7f3f49000640 1 -- 172.21.15.39:0/2265781045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f44149010 con 0x7f3f441f8d70 2024-09-21T07:48:01.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.021+0000 7f3f49000640 1 -- 172.21.15.39:0/2265781045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f440cb660 con 0x7f3f441f8d70 2024-09-21T07:48:01.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.023+0000 7f3f3b400640 1 -- 172.21.15.39:0/2265781045 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3f3c01c070 con 0x7f3f441f8d70 2024-09-21T07:48:01.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.024+0000 7f3f3b400640 1 --2- 172.21.15.39:0/2265781045 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3f20078b90 0x7f3f2007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:01.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.024+0000 7f3f3b400640 1 -- 172.21.15.39:0/2265781045 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(206..206 src has 1..206) ==== 6960+0+0 (secure 0 0 0) 0x7f3f3c09ad60 con 0x7f3f441f8d70 2024-09-21T07:48:01.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.024+0000 7f3f43400640 1 --2- 172.21.15.39:0/2265781045 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3f20078b90 0x7f3f2007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:01.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.025+0000 7f3f43400640 1 --2- 172.21.15.39:0/2265781045 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3f20078b90 0x7f3f2007b0a0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f3f340018a0 tx=0x7f3f34009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:01.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.028+0000 7f3f3b400640 1 -- 172.21.15.39:0/2265781045 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3f3c062680 con 0x7f3f441f8d70 2024-09-21T07:48:01.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.268+0000 7f3f49000640 1 -- 172.21.15.39:0/2265781045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f440d1ac0 con 0x7f3f441f8d70 2024-09-21T07:48:01.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.269+0000 7f3f3b400640 1 -- 172.21.15.39:0/2265781045 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 214 v214) ==== 95+0+5239 (secure 0 0 0) 0x7f3f3c0670b0 con 0x7f3f441f8d70 2024-09-21T07:48:01.271 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:48:01.271 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":214,"btime":"2024-09-21T07:47:58:212546+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7988,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1912631346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1912631346},{"type":"v1","addr":"172.21.15.148:6837","nonce":1912631346}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":209},{"gid":9043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2220628944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2220628944},{"type":"v1","addr":"172.21.15.148:6836","nonce":2220628944}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":208},{"gid":9049,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/1947479913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1947479913},{"type":"v1","addr":"172.21.15.39:6833","nonce":1947479913}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":214,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:47:54.691835+0000","modified":"2024-09-21T07:47:57.723294+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9852},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9852":{"gid":9852,"name":"c","rank":0,"incarnation":210,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/2356847598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2356847598},{"type":"v1","addr":"172.21.15.39:6835","nonce":2356847598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9852,"qdb_cluster":[9852]},"id":11}]} 2024-09-21T07:48:01.271 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 214 2024-09-21T07:48:01.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.271+0000 7f3f49000640 1 -- 172.21.15.39:0/2265781045 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3f20078b90 msgr2=0x7f3f2007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:01.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.271+0000 7f3f49000640 1 --2- 172.21.15.39:0/2265781045 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3f20078b90 0x7f3f2007b0a0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f3f340018a0 tx=0x7f3f34009040 comp rx=0 tx=0).stop 2024-09-21T07:48:01.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.271+0000 7f3f49000640 1 -- 172.21.15.39:0/2265781045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f441f8d70 msgr2=0x7f3f44148540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:01.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.271+0000 7f3f49000640 1 --2- 172.21.15.39:0/2265781045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f441f8d70 0x7f3f44148540 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00ba10 tx=0x7f3f3c00bed0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.271+0000 7f3f49000640 1 -- 172.21.15.39:0/2265781045 shutdown_connections 2024-09-21T07:48:01.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.271+0000 7f3f49000640 1 --2- 172.21.15.39:0/2265781045 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3f20078b90 0x7f3f2007b0a0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.271+0000 7f3f49000640 1 --2- 172.21.15.39:0/2265781045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f441f8d70 0x7f3f44148540 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.271+0000 7f3f49000640 1 --2- 172.21.15.39:0/2265781045 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3f440b5a90 0x7f3f441f8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.271+0000 7f3f49000640 1 --2- 172.21.15.39:0/2265781045 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f440b3b60 0x7f3f440b26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.271+0000 7f3f49000640 1 -- 172.21.15.39:0/2265781045 >> 172.21.15.39:0/2265781045 conn(0x7f3f4409d7b0 msgr2=0x7f3f440b3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:01.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.271+0000 7f3f49000640 1 -- 172.21.15.39:0/2265781045 shutdown_connections 2024-09-21T07:48:01.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.271+0000 7f3f49000640 1 -- 172.21.15.39:0/2265781045 wait complete. 2024-09-21T07:48:01.697 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 214, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T07:47:54.691835+0000', 'modified': '2024-09-21T07:47:57.723294+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9852}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9852': {'gid': 9852, 'name': 'c', 'rank': 0, 'incarnation': 210, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.39:6835/2356847598', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6834', 'nonce': 2356847598}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 2356847598}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9852, 'qdb_cluster': [9852]} 2024-09-21T07:48:01.697 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T07:48:01.697 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T07:48:01.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.922+0000 7f4101200640 1 -- 172.21.15.39:0/4250792868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40fc0cf990 msgr2=0x7f40fc0d0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:01.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.922+0000 7f4101200640 1 --2- 172.21.15.39:0/4250792868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40fc0cf990 0x7f40fc0d0630 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f40fc0d1390 tx=0x7f40f402aa30 comp rx=0 tx=0).stop 2024-09-21T07:48:01.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.922+0000 7f4101200640 1 -- 172.21.15.39:0/4250792868 shutdown_connections 2024-09-21T07:48:01.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.922+0000 7f4101200640 1 --2- 172.21.15.39:0/4250792868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40fc0cf990 0x7f40fc0d0630 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.922+0000 7f4101200640 1 --2- 172.21.15.39:0/4250792868 >> v2:172.21.15.148:3300/0 conn(0x7f40fc15daf0 0x7f40fc0cf450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.922+0000 7f4101200640 1 --2- 172.21.15.39:0/4250792868 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f40fc073dd0 0x7f40fc15d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.922+0000 7f4101200640 1 -- 172.21.15.39:0/4250792868 >> 172.21.15.39:0/4250792868 conn(0x7f40fc06cfc0 msgr2=0x7f40fc06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:01.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.922+0000 7f4101200640 1 -- 172.21.15.39:0/4250792868 shutdown_connections 2024-09-21T07:48:01.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.922+0000 7f4101200640 1 -- 172.21.15.39:0/4250792868 wait complete. 2024-09-21T07:48:01.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.924+0000 7f4101200640 1 Processor -- start 2024-09-21T07:48:01.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.925+0000 7f4101200640 1 -- start start 2024-09-21T07:48:01.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.926+0000 7f4101200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40fc073dd0 0x7f40fc1544c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:01.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.926+0000 7f4101200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40fc0cf990 0x7f40fc13fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:01.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.926+0000 7f4101200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f40fc15daf0 0x7f40fc14c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:01.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.926+0000 7f4101200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f40fc160740 con 0x7f40fc0cf990 2024-09-21T07:48:01.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.926+0000 7f4101200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f40fc1605c0 con 0x7f40fc073dd0 2024-09-21T07:48:01.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.926+0000 7f4101200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f40fc1608c0 con 0x7f40fc15daf0 2024-09-21T07:48:01.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.926+0000 7f40f3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40fc0cf990 0x7f40fc13fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:01.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.926+0000 7f40f3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40fc0cf990 0x7f40fc13fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53714/0 (socket says 172.21.15.39:53714) 2024-09-21T07:48:01.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.926+0000 7f40f3e00640 1 -- 172.21.15.39:0/3311904015 learned_addr learned my addr 172.21.15.39:0/3311904015 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:01.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.927+0000 7f40f9e00640 1 --2- 172.21.15.39:0/3311904015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40fc073dd0 0x7f40fc1544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:01.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.927+0000 7f40fa800640 1 --2- 172.21.15.39:0/3311904015 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f40fc15daf0 0x7f40fc14c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:01.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.927+0000 7f40f3e00640 1 -- 172.21.15.39:0/3311904015 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f40fc15daf0 msgr2=0x7f40fc14c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:01.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.927+0000 7f40f3e00640 1 --2- 172.21.15.39:0/3311904015 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f40fc15daf0 0x7f40fc14c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.927+0000 7f40f3e00640 1 -- 172.21.15.39:0/3311904015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40fc073dd0 msgr2=0x7f40fc1544c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:01.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.927+0000 7f40f3e00640 1 --2- 172.21.15.39:0/3311904015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40fc073dd0 0x7f40fc1544c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.927+0000 7f40f3e00640 1 -- 172.21.15.39:0/3311904015 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40fc140350 con 0x7f40fc0cf990 2024-09-21T07:48:01.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.927+0000 7f40f3e00640 1 --2- 172.21.15.39:0/3311904015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40fc0cf990 0x7f40fc13fe10 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f40e400d750 tx=0x7f40e400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:01.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.928+0000 7f40f2000640 1 -- 172.21.15.39:0/3311904015 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e4017070 con 0x7f40fc0cf990 2024-09-21T07:48:01.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.928+0000 7f40f2000640 1 -- 172.21.15.39:0/3311904015 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40e4004310 con 0x7f40fc0cf990 2024-09-21T07:48:01.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.928+0000 7f4101200640 1 -- 172.21.15.39:0/3311904015 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40fc1f96b0 con 0x7f40fc0cf990 2024-09-21T07:48:01.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.928+0000 7f40f2000640 1 -- 172.21.15.39:0/3311904015 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e4009c30 con 0x7f40fc0cf990 2024-09-21T07:48:01.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.928+0000 7f4101200640 1 -- 172.21.15.39:0/3311904015 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40fc14d0d0 con 0x7f40fc0cf990 2024-09-21T07:48:01.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.930+0000 7f40f2000640 1 -- 172.21.15.39:0/3311904015 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f40e4005020 con 0x7f40fc0cf990 2024-09-21T07:48:01.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.931+0000 7f40f2000640 1 --2- 172.21.15.39:0/3311904015 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f40d8078b90 0x7f40d807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:01.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.931+0000 7f40f2000640 1 -- 172.21.15.39:0/3311904015 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(206..206 src has 1..206) ==== 6960+0+0 (secure 0 0 0) 0x7f40e409b470 con 0x7f40fc0cf990 2024-09-21T07:48:01.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.932+0000 7f41081e9740 1 Processor -- start 2024-09-21T07:48:01.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.932+0000 7f40f9e00640 1 --2- 172.21.15.39:0/3311904015 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f40d8078b90 0x7f40d807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:01.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.932+0000 7f41081e9740 1 -- start start 2024-09-21T07:48:01.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.933+0000 7f40f9e00640 1 --2- 172.21.15.39:0/3311904015 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f40d8078b90 0x7f40d807b0a0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f40fc0730e0 tx=0x7f40ec00b260 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:01.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.933+0000 7f41081e9740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da29ae9270 0x55da29eb9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:01.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.933+0000 7f41081e9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da29ac03b0 0x55da29eb9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:01.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.933+0000 7f41081e9740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55da29ac8590 0x55da29e89f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:01.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.933+0000 7f41081e9740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55da29e13430 con 0x55da29ae9270 2024-09-21T07:48:01.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.933+0000 7f41081e9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55da29e035d0 con 0x55da29ac03b0 2024-09-21T07:48:01.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.933+0000 7f41081e9740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55da29df2a00 con 0x55da29ac8590 2024-09-21T07:48:01.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.933+0000 7f40f9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da29ae9270 0x55da29eb9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:01.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.933+0000 7f40f9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da29ae9270 0x55da29eb9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53718/0 (socket says 172.21.15.39:53718) 2024-09-21T07:48:01.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.933+0000 7f40f9e00640 1 -- 172.21.15.39:0/1786285494 learned_addr learned my addr 172.21.15.39:0/1786285494 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:01.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.934+0000 7f40f3e00640 1 --2- 172.21.15.39:0/1786285494 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da29ac03b0 0x55da29eb9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:01.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.934+0000 7f40fa800640 1 --2- 172.21.15.39:0/1786285494 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55da29ac8590 0x55da29e89f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:01.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.934+0000 7f40f9e00640 1 -- 172.21.15.39:0/1786285494 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55da29ac8590 msgr2=0x55da29e89f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:48:01.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.934+0000 7f40f9e00640 1 --2- 172.21.15.39:0/1786285494 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55da29ac8590 0x55da29e89f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.934+0000 7f40f9e00640 1 -- 172.21.15.39:0/1786285494 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da29ac03b0 msgr2=0x55da29eb9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:48:01.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.934+0000 7f40f9e00640 1 --2- 172.21.15.39:0/1786285494 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da29ac03b0 0x55da29eb9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.934+0000 7f40f9e00640 1 -- 172.21.15.39:0/1786285494 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da29dbb520 con 0x55da29ae9270 2024-09-21T07:48:01.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.934+0000 7f40f9e00640 1 --2- 172.21.15.39:0/1786285494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da29ae9270 0x55da29eb9970 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f40ec00b5d0 tx=0x7f40ec013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11cbbe04e721d05a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:01.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.934+0000 7f40eaa00640 1 -- 172.21.15.39:0/1786285494 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec01e620 con 0x55da29ae9270 2024-09-21T07:48:01.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.934+0000 7f40eaa00640 1 -- 172.21.15.39:0/1786285494 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40ec011e50 con 0x55da29ae9270 2024-09-21T07:48:01.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.934+0000 7f40eaa00640 1 -- 172.21.15.39:0/1786285494 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec01eaf0 con 0x55da29ae9270 2024-09-21T07:48:01.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.935+0000 7f41081e9740 1 -- 172.21.15.39:0/1786285494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da29ae9270 msgr2=0x55da29eb9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:01.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.935+0000 7f41081e9740 1 --2- 172.21.15.39:0/1786285494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da29ae9270 0x55da29eb9970 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f40ec00b5d0 tx=0x7f40ec013000 comp rx=0 tx=0).stop 2024-09-21T07:48:01.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.935+0000 7f41081e9740 1 -- 172.21.15.39:0/1786285494 shutdown_connections 2024-09-21T07:48:01.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.935+0000 7f41081e9740 1 --2- 172.21.15.39:0/1786285494 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55da29ac8590 0x55da29e89f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.935+0000 7f41081e9740 1 --2- 172.21.15.39:0/1786285494 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da29ac03b0 0x55da29eb9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.935+0000 7f41081e9740 1 --2- 172.21.15.39:0/1786285494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da29ae9270 0x55da29eb9970 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.935+0000 7f41081e9740 1 -- 172.21.15.39:0/1786285494 >> 172.21.15.39:0/1786285494 conn(0x55da29add3c0 msgr2=0x55da29e8f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:01.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.935+0000 7f41081e9740 1 -- 172.21.15.39:0/1786285494 shutdown_connections 2024-09-21T07:48:01.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.935+0000 7f41081e9740 1 -- 172.21.15.39:0/1786285494 wait complete. 2024-09-21T07:48:01.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.936+0000 7f41081e9740 1 -- start start 2024-09-21T07:48:01.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.937+0000 7f41081e9740 1 Processor -- start 2024-09-21T07:48:01.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.938+0000 7f41081e9740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da29ac03b0 0x55da29eb00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:01.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.938+0000 7f41081e9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da29ac8590 0x55da29eb3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:01.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.938+0000 7f41081e9740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55da29ae9270 0x55da29eb3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:01.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.938+0000 7f41081e9740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55da29d99330 con 0x55da29ac03b0 2024-09-21T07:48:01.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.938+0000 7f41081e9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55da29df6810 con 0x55da29ac8590 2024-09-21T07:48:01.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.938+0000 7f41081e9740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55da29b63620 con 0x55da29ae9270 2024-09-21T07:48:01.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.938+0000 7f41081e9740 10 client.0 _ll_register_callbacks cb 0x55da29d9e420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:48:01.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.938+0000 7f40f9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da29ac03b0 0x55da29eb00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:01.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.938+0000 7f40f9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da29ac03b0 0x55da29eb00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53720/0 (socket says 172.21.15.39:53720) 2024-09-21T07:48:01.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.938+0000 7f40f9e00640 1 -- 172.21.15.39:0/3586578739 learned_addr learned my addr 172.21.15.39:0/3586578739 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:01.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.939+0000 7f40fa800640 1 --2- 172.21.15.39:0/3586578739 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55da29ae9270 0x55da29eb3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:01.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.939+0000 7f40f3e00640 1 --2- 172.21.15.39:0/3586578739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da29ac8590 0x55da29eb3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:01.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.939+0000 7f40f9e00640 1 -- 172.21.15.39:0/3586578739 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55da29ae9270 msgr2=0x55da29eb3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:48:01.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.939+0000 7f40f9e00640 1 --2- 172.21.15.39:0/3586578739 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55da29ae9270 0x55da29eb3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.939+0000 7f40f9e00640 1 -- 172.21.15.39:0/3586578739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da29ac8590 msgr2=0x55da29eb3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:48:01.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.939+0000 7f40f9e00640 1 --2- 172.21.15.39:0/3586578739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da29ac8590 0x55da29eb3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.939+0000 7f40f9e00640 1 -- 172.21.15.39:0/3586578739 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da29b54850 con 0x55da29ac03b0 2024-09-21T07:48:01.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.939+0000 7f40fa800640 1 --2- 172.21.15.39:0/3586578739 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55da29ae9270 0x55da29eb3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:48:01.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.939+0000 7f40f9e00640 1 --2- 172.21.15.39:0/3586578739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da29ac03b0 0x55da29eb00f0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f40ec011ca0 tx=0x7f40ec01e650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7903a8d15a73c56f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:01.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.939+0000 7f40df400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:48:01.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.939+0000 7f40df400640 1 -- 172.21.15.39:0/3586578739 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec034060 con 0x55da29ac03b0 2024-09-21T07:48:01.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.939+0000 7f40df400640 1 -- 172.21.15.39:0/3586578739 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(206..206 src has 1..206) ==== 6960+0+0 (secure 0 0 0) 0x7f40ec01ebc0 con 0x55da29ac03b0 2024-09-21T07:48:01.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.939+0000 7f41081e9740 1 -- 172.21.15.39:0/3586578739 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40ec011e50 con 0x55da29ac03b0 2024-09-21T07:48:01.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.940+0000 7f40df400640 1 -- 172.21.15.39:0/3586578739 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=214) ==== 24+0+0 (secure 0 0 0) 0x7f40ec03a050 con 0x55da29ac03b0 2024-09-21T07:48:01.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.940+0000 7f41081e9740 10 client.9975 fetch_fsmap learned FSMap version 214 2024-09-21T07:48:01.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.940+0000 7f41081e9740 1 -- 172.21.15.39:0/3586578739 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=214,monmap=2+}) -- 0x55da29bcb6b0 con 0x55da29ac03b0 2024-09-21T07:48:01.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.940+0000 7f40df400640 1 -- 172.21.15.39:0/3586578739 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40ec011e50 con 0x55da29ac03b0 2024-09-21T07:48:01.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.940+0000 7f40df400640 1 -- 172.21.15.39:0/3586578739 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 214) ==== 3502+0+0 (secure 0 0 0) 0x7f40ec03d020 con 0x55da29ac03b0 2024-09-21T07:48:01.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.940+0000 7f41081e9740 10 client.9975 fetch_fsmap finished waiting for FSMap version 214 2024-09-21T07:48:01.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.940+0000 7f41081e9740 10 client.9975 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:48:01.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.940+0000 7f41081e9740 20 client.9975 populate_metadata read hostname 'smithi039' 2024-09-21T07:48:01.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.940+0000 7f41081e9740 1 --2- 172.21.15.39:0/3586578739 >> [v2:172.21.15.39:6834/2356847598,v1:172.21.15.39:6835/2356847598] conn(0x55da29aa7e40 0x55da29ee3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:01.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.940+0000 7f41081e9740 4 client.9975 mds_command: new command op to 9852 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:48:01.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.940+0000 7f41081e9740 1 -- 172.21.15.39:0/3586578739 --> [v2:172.21.15.39:6834/2356847598,v1:172.21.15.39:6835/2356847598] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da29e05520 con 0x55da29aa7e40 2024-09-21T07:48:01.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.941+0000 7f40f3e00640 1 --2- 172.21.15.39:0/3586578739 >> [v2:172.21.15.39:6834/2356847598,v1:172.21.15.39:6835/2356847598] conn(0x55da29aa7e40 0x55da29ee3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:01.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.941+0000 7f40f3e00640 1 --2- 172.21.15.39:0/3586578739 >> [v2:172.21.15.39:6834/2356847598,v1:172.21.15.39:6835/2356847598] conn(0x55da29aa7e40 0x55da29ee3f70 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5189691de28416e8 server_cookie=c5dc3bd6e44b6af2 in_seq=0 out_seq=0 2024-09-21T07:48:01.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.941+0000 7f40df400640 10 client.9975 ms_handle_connect on v2:172.21.15.39:6834/2356847598 2024-09-21T07:48:01.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.942+0000 7f40df400640 1 -- 172.21.15.39:0/3586578739 <== mds.0 v2:172.21.15.39:6834/2356847598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55da29e05520 con 0x55da29aa7e40 2024-09-21T07:48:01.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.942+0000 7f40df400640 10 client.9975 handle_command_reply: tid=0 2024-09-21T07:48:01.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.942+0000 7f41081e9740 1 client.9975 shutdown 2024-09-21T07:48:01.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.942+0000 7f41081e9740 1 -- 172.21.15.39:0/3586578739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da29ac03b0 msgr2=0x55da29eb00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:01.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.942+0000 7f41081e9740 1 --2- 172.21.15.39:0/3586578739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da29ac03b0 0x55da29eb00f0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f40ec011ca0 tx=0x7f40ec01e650 comp rx=0 tx=0).stop 2024-09-21T07:48:01.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.942+0000 7f41081e9740 1 -- 172.21.15.39:0/3586578739 shutdown_connections 2024-09-21T07:48:01.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.942+0000 7f41081e9740 1 --2- 172.21.15.39:0/3586578739 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55da29ae9270 0x55da29eb3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.942+0000 7f41081e9740 1 --2- 172.21.15.39:0/3586578739 >> [v2:172.21.15.39:6834/2356847598,v1:172.21.15.39:6835/2356847598] conn(0x55da29aa7e40 0x55da29ee3f70 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.942+0000 7f41081e9740 1 --2- 172.21.15.39:0/3586578739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da29ac8590 0x55da29eb3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.942+0000 7f41081e9740 1 --2- 172.21.15.39:0/3586578739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da29ac03b0 0x55da29eb00f0 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.942+0000 7f40df400640 0 client.9975 ms_handle_reset on v2:172.21.15.39:6834/2356847598 2024-09-21T07:48:01.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.942+0000 7f41081e9740 1 -- 172.21.15.39:0/3586578739 >> 172.21.15.39:0/3586578739 conn(0x55da29add3c0 msgr2=0x55da29e8b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:01.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.942+0000 7f41081e9740 1 -- 172.21.15.39:0/3586578739 shutdown_connections 2024-09-21T07:48:01.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.942+0000 7f41081e9740 1 -- 172.21.15.39:0/3586578739 wait complete. 2024-09-21T07:48:01.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.943+0000 7f41081e9740 20 client.9975 trim_cache size 0 max 16384 2024-09-21T07:48:01.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.962+0000 7f41081e9740 1 Processor -- start 2024-09-21T07:48:01.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.962+0000 7f41081e9740 1 -- start start 2024-09-21T07:48:01.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.964+0000 7f41081e9740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55da29ac03b0 0x55da29ece1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:01.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.964+0000 7f41081e9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da29ac8590 0x55da29ece6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:01.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.964+0000 7f41081e9740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da29ae9270 0x55da29ed33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:01.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.964+0000 7f41081e9740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f40fc203a40 con 0x55da29ae9270 2024-09-21T07:48:01.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.964+0000 7f41081e9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f40fc156da0 con 0x55da29ac8590 2024-09-21T07:48:01.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.964+0000 7f41081e9740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f40fc2033f0 con 0x55da29ac03b0 2024-09-21T07:48:01.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.964+0000 7f40fa800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da29ae9270 0x55da29ed33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:01.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.964+0000 7f40fa800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da29ae9270 0x55da29ed33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53724/0 (socket says 172.21.15.39:53724) 2024-09-21T07:48:01.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.964+0000 7f40fa800640 1 -- 172.21.15.39:0/2418751758 learned_addr learned my addr 172.21.15.39:0/2418751758 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:01.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.964+0000 7f40f9e00640 1 --2- 172.21.15.39:0/2418751758 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55da29ac03b0 0x55da29ece1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:01.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.964+0000 7f40f3e00640 1 --2- 172.21.15.39:0/2418751758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da29ac8590 0x55da29ece6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:01.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.964+0000 7f40fa800640 1 -- 172.21.15.39:0/2418751758 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55da29ac03b0 msgr2=0x55da29ece1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:48:01.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.964+0000 7f40fa800640 1 --2- 172.21.15.39:0/2418751758 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55da29ac03b0 0x55da29ece1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.964+0000 7f40fa800640 1 -- 172.21.15.39:0/2418751758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da29ac8590 msgr2=0x55da29ece6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:48:01.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.964+0000 7f40fa800640 1 --2- 172.21.15.39:0/2418751758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da29ac8590 0x55da29ece6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.964+0000 7f40fa800640 1 -- 172.21.15.39:0/2418751758 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da29e8bcf0 con 0x55da29ae9270 2024-09-21T07:48:01.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.965+0000 7f40fa800640 1 --2- 172.21.15.39:0/2418751758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da29ae9270 0x55da29ed33b0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f40f4005fd0 tx=0x7f40f4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b68dd49f58ff4f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:01.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.965+0000 7f40ea000640 1 -- 172.21.15.39:0/2418751758 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f4033040 con 0x55da29ae9270 2024-09-21T07:48:01.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.965+0000 7f40ea000640 1 -- 172.21.15.39:0/2418751758 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40f402fcc0 con 0x55da29ae9270 2024-09-21T07:48:01.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.965+0000 7f40ea000640 1 -- 172.21.15.39:0/2418751758 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f4003bd0 con 0x55da29ae9270 2024-09-21T07:48:01.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.966+0000 7f41081e9740 1 -- 172.21.15.39:0/2418751758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da29ae9270 msgr2=0x55da29ed33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:01.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.966+0000 7f41081e9740 1 --2- 172.21.15.39:0/2418751758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da29ae9270 0x55da29ed33b0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f40f4005fd0 tx=0x7f40f4004ab0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.966+0000 7f41081e9740 1 -- 172.21.15.39:0/2418751758 shutdown_connections 2024-09-21T07:48:01.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.966+0000 7f41081e9740 1 --2- 172.21.15.39:0/2418751758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da29ae9270 0x55da29ed33b0 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.966+0000 7f41081e9740 1 --2- 172.21.15.39:0/2418751758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da29ac8590 0x55da29ece6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.966+0000 7f41081e9740 1 --2- 172.21.15.39:0/2418751758 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55da29ac03b0 0x55da29ece1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.966+0000 7f41081e9740 1 -- 172.21.15.39:0/2418751758 >> 172.21.15.39:0/2418751758 conn(0x55da29add3c0 msgr2=0x55da29ed5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:01.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.966+0000 7f41081e9740 1 -- 172.21.15.39:0/2418751758 shutdown_connections 2024-09-21T07:48:01.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.966+0000 7f41081e9740 1 -- 172.21.15.39:0/2418751758 wait complete. 2024-09-21T07:48:01.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.967+0000 7f41081e9740 1 -- start start 2024-09-21T07:48:01.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.967+0000 7f41081e9740 1 Processor -- start 2024-09-21T07:48:01.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.969+0000 7f41081e9740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55da29ac03b0 0x55da29f4e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:01.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.969+0000 7f41081e9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da29ac8590 0x55da29eb15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:01.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.969+0000 7f41081e9740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da29ae9270 0x55da29eb1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:01.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.969+0000 7f41081e9740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55da29ed0050 con 0x55da29ae9270 2024-09-21T07:48:01.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.969+0000 7f41081e9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55da29eb0690 con 0x55da29ac8590 2024-09-21T07:48:01.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.969+0000 7f41081e9740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55da29ee7940 con 0x55da29ac03b0 2024-09-21T07:48:01.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.969+0000 7f41081e9740 10 client.0 _ll_register_callbacks cb 0x7f40ec010bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:48:01.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.969+0000 7f40fa800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da29ae9270 0x55da29eb1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:01.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.969+0000 7f40fa800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da29ae9270 0x55da29eb1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53732/0 (socket says 172.21.15.39:53732) 2024-09-21T07:48:01.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.969+0000 7f40fa800640 1 -- 172.21.15.39:0/2370777091 learned_addr learned my addr 172.21.15.39:0/2370777091 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:01.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.969+0000 7f40f3e00640 1 --2- 172.21.15.39:0/2370777091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da29ac8590 0x55da29eb15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:01.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.969+0000 7f40f9e00640 1 --2- 172.21.15.39:0/2370777091 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55da29ac03b0 0x55da29f4e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:01.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.969+0000 7f40fa800640 1 -- 172.21.15.39:0/2370777091 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55da29ac03b0 msgr2=0x55da29f4e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:48:01.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.969+0000 7f40fa800640 1 --2- 172.21.15.39:0/2370777091 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55da29ac03b0 0x55da29f4e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.969+0000 7f40fa800640 1 -- 172.21.15.39:0/2370777091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da29ac8590 msgr2=0x55da29eb15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:48:01.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.969+0000 7f40fa800640 1 --2- 172.21.15.39:0/2370777091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da29ac8590 0x55da29eb15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.969+0000 7f40fa800640 1 -- 172.21.15.39:0/2370777091 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da29e90650 con 0x55da29ae9270 2024-09-21T07:48:01.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.969+0000 7f40f3e00640 1 --2- 172.21.15.39:0/2370777091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da29ac8590 0x55da29eb15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:48:01.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.970+0000 7f40fa800640 1 --2- 172.21.15.39:0/2370777091 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da29ae9270 0x55da29eb1b20 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f40f4040000 tx=0x7f40f40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=628c2932359a2bc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:01.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.970+0000 7f40df400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:48:01.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.970+0000 7f40df400640 1 -- 172.21.15.39:0/2370777091 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f4050060 con 0x55da29ae9270 2024-09-21T07:48:01.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.970+0000 7f40df400640 1 -- 172.21.15.39:0/2370777091 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(206..206 src has 1..206) ==== 6960+0+0 (secure 0 0 0) 0x7f40f402fac0 con 0x55da29ae9270 2024-09-21T07:48:01.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.970+0000 7f41081e9740 1 -- 172.21.15.39:0/2370777091 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40f402fcc0 con 0x55da29ae9270 2024-09-21T07:48:01.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.970+0000 7f40df400640 1 -- 172.21.15.39:0/2370777091 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=214) ==== 24+0+0 (secure 0 0 0) 0x7f40f4056050 con 0x55da29ae9270 2024-09-21T07:48:01.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.970+0000 7f41081e9740 10 client.9981 fetch_fsmap learned FSMap version 214 2024-09-21T07:48:01.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.970+0000 7f41081e9740 1 -- 172.21.15.39:0/2370777091 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=214,monmap=2+}) -- 0x55da29a9a970 con 0x55da29ae9270 2024-09-21T07:48:01.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.970+0000 7f40df400640 1 -- 172.21.15.39:0/2370777091 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40f402fcc0 con 0x55da29ae9270 2024-09-21T07:48:01.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.971+0000 7f40df400640 1 -- 172.21.15.39:0/2370777091 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 214) ==== 3502+0+0 (secure 0 0 0) 0x7f40f4059020 con 0x55da29ae9270 2024-09-21T07:48:01.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.971+0000 7f41081e9740 10 client.9981 fetch_fsmap finished waiting for FSMap version 214 2024-09-21T07:48:01.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.971+0000 7f41081e9740 10 client.9981 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:48:01.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.971+0000 7f41081e9740 20 client.9981 populate_metadata read hostname 'smithi039' 2024-09-21T07:48:01.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.971+0000 7f41081e9740 1 --2- 172.21.15.39:0/2370777091 >> [v2:172.21.15.39:6834/2356847598,v1:172.21.15.39:6835/2356847598] conn(0x55da29f54ab0 0x55da29f54ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:01.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.971+0000 7f41081e9740 4 client.9981 mds_command: new command op to 9852 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:48:01.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.971+0000 7f41081e9740 1 -- 172.21.15.39:0/2370777091 --> [v2:172.21.15.39:6834/2356847598,v1:172.21.15.39:6835/2356847598] -- command(tid 0: {"prefix": "status"}) -- 0x55da29e00fd0 con 0x55da29f54ab0 2024-09-21T07:48:01.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.971+0000 7f40f9e00640 1 --2- 172.21.15.39:0/2370777091 >> [v2:172.21.15.39:6834/2356847598,v1:172.21.15.39:6835/2356847598] conn(0x55da29f54ab0 0x55da29f54ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:01.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.971+0000 7f40f9e00640 1 --2- 172.21.15.39:0/2370777091 >> [v2:172.21.15.39:6834/2356847598,v1:172.21.15.39:6835/2356847598] conn(0x55da29f54ab0 0x55da29f54ec0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e817b58ab8c419bd server_cookie=1f4fd001648346b4 in_seq=0 out_seq=0 2024-09-21T07:48:01.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.971+0000 7f40df400640 10 client.9981 ms_handle_connect on v2:172.21.15.39:6834/2356847598 2024-09-21T07:48:01.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.972+0000 7f40df400640 1 -- 172.21.15.39:0/2370777091 <== mds.0 v2:172.21.15.39:6834/2356847598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x55da29e00fd0 con 0x55da29f54ab0 2024-09-21T07:48:01.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.972+0000 7f40df400640 10 client.9981 handle_command_reply: tid=0 2024-09-21T07:48:01.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.972+0000 7f41081e9740 1 client.9981 shutdown 2024-09-21T07:48:01.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.972+0000 7f41081e9740 1 -- 172.21.15.39:0/2370777091 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da29ae9270 msgr2=0x55da29eb1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:01.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.972+0000 7f41081e9740 1 --2- 172.21.15.39:0/2370777091 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da29ae9270 0x55da29eb1b20 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f40f4040000 tx=0x7f40f40042f0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.972+0000 7f41081e9740 1 -- 172.21.15.39:0/2370777091 shutdown_connections 2024-09-21T07:48:01.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.972+0000 7f41081e9740 1 --2- 172.21.15.39:0/2370777091 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da29ae9270 0x55da29eb1b20 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.972+0000 7f41081e9740 1 --2- 172.21.15.39:0/2370777091 >> [v2:172.21.15.39:6834/2356847598,v1:172.21.15.39:6835/2356847598] conn(0x55da29f54ab0 0x55da29f54ec0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.972+0000 7f41081e9740 1 --2- 172.21.15.39:0/2370777091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da29ac8590 0x55da29eb15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.972+0000 7f41081e9740 1 --2- 172.21.15.39:0/2370777091 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55da29ac03b0 0x55da29f4e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.972+0000 7f41081e9740 1 -- 172.21.15.39:0/2370777091 >> 172.21.15.39:0/2370777091 conn(0x55da29add3c0 msgr2=0x55da29eb3730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:01.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.972+0000 7f40df400640 0 client.9981 ms_handle_reset on v2:172.21.15.39:6834/2356847598 2024-09-21T07:48:01.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.972+0000 7f41081e9740 1 -- 172.21.15.39:0/2370777091 shutdown_connections 2024-09-21T07:48:01.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.972+0000 7f41081e9740 1 -- 172.21.15.39:0/2370777091 wait complete. 2024-09-21T07:48:01.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.972+0000 7f41081e9740 20 client.9981 trim_cache size 0 max 16384 2024-09-21T07:48:01.978 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:48:01.978 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:48:01.978 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-09-21T07:48:01.978 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 9852, 2024-09-21T07:48:01.978 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:48:01.978 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:48:01.979 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:48:01.979 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 6.8107196290000003, 2024-09-21T07:48:01.979 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 214, 2024-09-21T07:48:01.979 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 206, 2024-09-21T07:48:01.979 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 206, 2024-09-21T07:48:01.979 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 11.809166226 2024-09-21T07:48:01.979 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:48:01.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.973+0000 7f4101200640 1 -- 172.21.15.39:0/3311904015 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f40d8078b90 msgr2=0x7f40d807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:01.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.973+0000 7f4101200640 1 --2- 172.21.15.39:0/3311904015 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f40d8078b90 0x7f40d807b0a0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f40fc0730e0 tx=0x7f40ec00b260 comp rx=0 tx=0).stop 2024-09-21T07:48:01.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.973+0000 7f4101200640 1 -- 172.21.15.39:0/3311904015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40fc0cf990 msgr2=0x7f40fc13fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:01.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.973+0000 7f4101200640 1 --2- 172.21.15.39:0/3311904015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40fc0cf990 0x7f40fc13fe10 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f40e400d750 tx=0x7f40e400dc10 comp rx=0 tx=0).stop 2024-09-21T07:48:01.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.973+0000 7f4101200640 1 -- 172.21.15.39:0/3311904015 shutdown_connections 2024-09-21T07:48:01.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.973+0000 7f4101200640 1 --2- 172.21.15.39:0/3311904015 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f40d8078b90 0x7f40d807b0a0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.973+0000 7f4101200640 1 --2- 172.21.15.39:0/3311904015 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f40fc15daf0 0x7f40fc14c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.973+0000 7f4101200640 1 --2- 172.21.15.39:0/3311904015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40fc0cf990 0x7f40fc13fe10 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.973+0000 7f4101200640 1 --2- 172.21.15.39:0/3311904015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40fc073dd0 0x7f40fc1544c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:01.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.973+0000 7f4101200640 1 -- 172.21.15.39:0/3311904015 >> 172.21.15.39:0/3311904015 conn(0x7f40fc06cfc0 msgr2=0x7f40fc0b0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:01.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.973+0000 7f4101200640 1 -- 172.21.15.39:0/3311904015 shutdown_connections 2024-09-21T07:48:01.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:01.973+0000 7f4101200640 1 -- 172.21.15.39:0/3311904015 wait complete. 2024-09-21T07:48:01.999 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:48:02.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.226+0000 7f84d6e00640 1 -- 172.21.15.39:0/3432834043 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84d00b3990 msgr2=0x7f84d00b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:02.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.226+0000 7f84d6e00640 1 --2- 172.21.15.39:0/3432834043 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84d00b3990 0x7f84d00b4450 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f84b8005fd0 tx=0x7f84b802a5d0 comp rx=0 tx=0).stop 2024-09-21T07:48:02.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.226+0000 7f84d6e00640 1 -- 172.21.15.39:0/3432834043 shutdown_connections 2024-09-21T07:48:02.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.226+0000 7f84d6e00640 1 --2- 172.21.15.39:0/3432834043 >> v2:172.21.15.148:3300/0 conn(0x7f84d00b3da0 0x7f84d00b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:02.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.226+0000 7f84d6e00640 1 --2- 172.21.15.39:0/3432834043 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84d00b3990 0x7f84d00b4450 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:02.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.226+0000 7f84d6e00640 1 -- 172.21.15.39:0/3432834043 >> 172.21.15.39:0/3432834043 conn(0x7f84d00b2460 msgr2=0x7f84d006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:02.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.226+0000 7f84d6e00640 1 -- 172.21.15.39:0/3432834043 shutdown_connections 2024-09-21T07:48:02.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.226+0000 7f84d6e00640 1 -- 172.21.15.39:0/3432834043 wait complete. 2024-09-21T07:48:02.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.228+0000 7f84d6e00640 1 Processor -- start 2024-09-21T07:48:02.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.229+0000 7f84d6e00640 1 -- start start 2024-09-21T07:48:02.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.230+0000 7f84d6e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84d00b3990 0x7f84d01f9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:02.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.230+0000 7f84d6e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d00b3da0 0x7f84d01f6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:02.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.230+0000 7f84d6e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84d0162f10 0x7f84d015c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:02.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.230+0000 7f84d6e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f84d00af200 con 0x7f84d00b3da0 2024-09-21T07:48:02.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.230+0000 7f84d6e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f84d00af080 con 0x7f84d0162f10 2024-09-21T07:48:02.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.230+0000 7f84d6e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f84d00af380 con 0x7f84d00b3990 2024-09-21T07:48:02.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.230+0000 7f84c7400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d00b3da0 0x7f84d01f6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:02.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.231+0000 7f84c7400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d00b3da0 0x7f84d01f6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56248/0 (socket says 172.21.15.39:56248) 2024-09-21T07:48:02.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.231+0000 7f84cfe00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84d00b3990 0x7f84d01f9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:02.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.231+0000 7f84c7400640 1 -- 172.21.15.39:0/3798928031 learned_addr learned my addr 172.21.15.39:0/3798928031 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:02.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.231+0000 7f84d4e00640 1 --2- 172.21.15.39:0/3798928031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84d0162f10 0x7f84d015c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:02.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.231+0000 7f84c7400640 1 -- 172.21.15.39:0/3798928031 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84d00b3990 msgr2=0x7f84d01f9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:02.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.231+0000 7f84c7400640 1 --2- 172.21.15.39:0/3798928031 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84d00b3990 0x7f84d01f9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:02.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.231+0000 7f84c7400640 1 -- 172.21.15.39:0/3798928031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84d0162f10 msgr2=0x7f84d015c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:02.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.231+0000 7f84c7400640 1 --2- 172.21.15.39:0/3798928031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84d0162f10 0x7f84d015c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:02.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.231+0000 7f84c7400640 1 -- 172.21.15.39:0/3798928031 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d01f5cb0 con 0x7f84d00b3da0 2024-09-21T07:48:02.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.231+0000 7f84c7400640 1 --2- 172.21.15.39:0/3798928031 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d00b3da0 0x7f84d01f6750 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f84b8006010 tx=0x7f84b80035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:02.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.234+0000 7f84ccc00640 1 -- 172.21.15.39:0/3798928031 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b803e030 con 0x7f84d00b3da0 2024-09-21T07:48:02.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.234+0000 7f84ccc00640 1 -- 172.21.15.39:0/3798928031 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f84b8003ae0 con 0x7f84d00b3da0 2024-09-21T07:48:02.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.234+0000 7f84d6e00640 1 -- 172.21.15.39:0/3798928031 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84d015dd90 con 0x7f84d00b3da0 2024-09-21T07:48:02.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.234+0000 7f84ccc00640 1 -- 172.21.15.39:0/3798928031 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b8035040 con 0x7f84d00b3da0 2024-09-21T07:48:02.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.234+0000 7f84d6e00640 1 -- 172.21.15.39:0/3798928031 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84d01f5f70 con 0x7f84d00b3da0 2024-09-21T07:48:02.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.235+0000 7f84d6e00640 1 -- 172.21.15.39:0/3798928031 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84d0074ee0 con 0x7f84d00b3da0 2024-09-21T07:48:02.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.237+0000 7f84ccc00640 1 -- 172.21.15.39:0/3798928031 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f84b8041030 con 0x7f84d00b3da0 2024-09-21T07:48:02.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.238+0000 7f84ccc00640 1 --2- 172.21.15.39:0/3798928031 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f84ac078b90 0x7f84ac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:02.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.238+0000 7f84ccc00640 1 -- 172.21.15.39:0/3798928031 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(206..206 src has 1..206) ==== 6960+0+0 (secure 0 0 0) 0x7f84b80bb680 con 0x7f84d00b3da0 2024-09-21T07:48:02.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.238+0000 7f84cfe00640 1 --2- 172.21.15.39:0/3798928031 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f84ac078b90 0x7f84ac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:02.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.239+0000 7f84cfe00640 1 --2- 172.21.15.39:0/3798928031 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f84ac078b90 0x7f84ac07b0a0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f84d0075150 tx=0x7f84c0009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:02.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.244+0000 7f84ccc00640 1 -- 172.21.15.39:0/3798928031 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f84d0074ee0 con 0x7f84d00b3da0 2024-09-21T07:48:02.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.480+0000 7f84d6e00640 1 -- 172.21.15.39:0/3798928031 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f84d015db20 con 0x7f84d00b3da0 2024-09-21T07:48:02.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.480+0000 7f84ccc00640 1 -- 172.21.15.39:0/3798928031 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v214) ==== 53+0+73 (secure 0 0 0) 0x7f84b8082fa0 con 0x7f84d00b3da0 2024-09-21T07:48:02.482 INFO:teuthology.orchestra.run.smithi039.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:48:02.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.482+0000 7f84d6e00640 1 -- 172.21.15.39:0/3798928031 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f84ac078b90 msgr2=0x7f84ac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:02.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.482+0000 7f84d6e00640 1 --2- 172.21.15.39:0/3798928031 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f84ac078b90 0x7f84ac07b0a0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f84d0075150 tx=0x7f84c0009040 comp rx=0 tx=0).stop 2024-09-21T07:48:02.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.482+0000 7f84d6e00640 1 -- 172.21.15.39:0/3798928031 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d00b3da0 msgr2=0x7f84d01f6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:02.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.482+0000 7f84d6e00640 1 --2- 172.21.15.39:0/3798928031 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d00b3da0 0x7f84d01f6750 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f84b8006010 tx=0x7f84b80035f0 comp rx=0 tx=0).stop 2024-09-21T07:48:02.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.483+0000 7f84d6e00640 1 -- 172.21.15.39:0/3798928031 shutdown_connections 2024-09-21T07:48:02.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.483+0000 7f84d6e00640 1 --2- 172.21.15.39:0/3798928031 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f84ac078b90 0x7f84ac07b0a0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:02.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.483+0000 7f84d6e00640 1 --2- 172.21.15.39:0/3798928031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f84d0162f10 0x7f84d015c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:02.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.483+0000 7f84d6e00640 1 --2- 172.21.15.39:0/3798928031 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84d00b3da0 0x7f84d01f6750 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:02.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.483+0000 7f84d6e00640 1 --2- 172.21.15.39:0/3798928031 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84d00b3990 0x7f84d01f9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:02.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.483+0000 7f84d6e00640 1 -- 172.21.15.39:0/3798928031 >> 172.21.15.39:0/3798928031 conn(0x7f84d00b2460 msgr2=0x7f84d00ca4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:02.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.483+0000 7f84d6e00640 1 -- 172.21.15.39:0/3798928031 shutdown_connections 2024-09-21T07:48:02.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02.483+0000 7f84d6e00640 1 -- 172.21.15.39:0/3798928031 wait complete. 2024-09-21T07:48:02.506 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:48:02.506 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:48:02.506 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi039.front.sepia.ceph.com 2024-09-21T07:48:02.506 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T07:48:02.506 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:48:02.506 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:48:02.506 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:48:02.506 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:48:02.506 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:48:02.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:48:02.507 DEBUG:teuthology.orchestra.run.smithi039:> ip addr 2024-09-21T07:48:02.542 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:48:02.542 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:48:02.542 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:48:02.542 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:48:02.542 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2024-09-21T07:48:02.542 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:48:02.543 INFO:teuthology.orchestra.run.smithi039.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:48:02.543 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:48:02.543 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f0 2024-09-21T07:48:02.543 INFO:teuthology.orchestra.run.smithi039.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:48:02.543 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c7 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:48:02.543 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f1 2024-09-21T07:48:02.543 INFO:teuthology.orchestra.run.smithi039.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:48:02.543 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:70 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:48:02.543 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f0 2024-09-21T07:48:02.543 INFO:teuthology.orchestra.run.smithi039.stdout: inet 172.21.15.39/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T07:48:02.543 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft 41836sec preferred_lft 41836sec 2024-09-21T07:48:02.543 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec4:7aff:febd:1370/64 scope link 2024-09-21T07:48:02.543 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:48:02.543 INFO:teuthology.orchestra.run.smithi039.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:48:02.544 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:71 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:48:02.544 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f1 2024-09-21T07:48:02.544 INFO:teuthology.orchestra.run.smithi039.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:48:02.544 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff 2024-09-21T07:48:02.544 INFO:teuthology.orchestra.run.smithi039.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:48:02.544 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:48:02.544 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::b079:ddff:fe0e:6333/64 scope link 2024-09-21T07:48:02.544 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:48:02.544 INFO:teuthology.orchestra.run.smithi039.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:48:02.544 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:48:02.544 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::2ca7:98ff:fe8a:d0ed/64 scope link 2024-09-21T07:48:02.544 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:48:02.544 INFO:teuthology.orchestra.run.smithi039.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:48:02.544 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether ee:10:53:01:0c:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:48:02.545 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec10:53ff:fe01:c44/64 scope link 2024-09-21T07:48:02.545 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:48:02.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:48:02.545 DEBUG:teuthology.orchestra.run.smithi039:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:48:02.636 INFO:teuthology.orchestra.run.smithi039.stdout:1 2024-09-21T07:48:02.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:48:02.639 DEBUG:teuthology.orchestra.run.smithi039:> route 2024-09-21T07:48:02.675 INFO:teuthology.orchestra.run.smithi039.stdout:Kernel IP routing table 2024-09-21T07:48:02.675 INFO:teuthology.orchestra.run.smithi039.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:48:02.675 INFO:teuthology.orchestra.run.smithi039.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T07:48:02.675 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T07:48:02.675 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:48:02.675 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:48:02.675 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-21T07:48:02.676 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T07:48:02.676 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T07:48:02.676 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T07:48:02.676 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-21T07:48:02.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:48:02.847 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:48:02.852 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:48:02.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:48:02.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:48:02.875 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-09-21T07:48:02.950 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:48:02.950 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T07:48:02.950 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:48:02.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:48:02.950 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-09-21T07:48:03.046 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:48:03.046 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T07:48:03.080 INFO:teuthology.orchestra.run.smithi039.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:48:03.081 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:48:03.081 DEBUG:teuthology.orchestra.run.smithi039:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:48:03.155 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-21T07:48:03.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:48:03.156 DEBUG:teuthology.orchestra.run.smithi039:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:48:03.354 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:48:03.355 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:48:03.392 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:48:03.392 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:48:03.392 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: resolved to: "172.21.15.39:3300,172.21.15.148:3300,172.21.15.148:3301" 2024-09-21T07:48:03.392 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: trying mount with new device syntax: 0@7ab9b04a-f88d-49aa-a493-e5c076637ffb.cephfs=/ 2024-09-21T07:48:03.392 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.39:3300/172.21.15.148:3300/172.21.15.148:3301" will pass to kernel 2024-09-21T07:48:03.393 INFO:teuthology.orchestra.run.smithi039.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T07:48:03.393 INFO:teuthology.orchestra.run.smithi039.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:48:03.393 INFO:teuthology.orchestra.run.smithi039.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:48:03.393 INFO:teuthology.orchestra.run.smithi039.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:48:03.393 INFO:teuthology.orchestra.run.smithi039.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:48:03.396 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:48:03.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:48:03.396 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:48:03.453 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:48:03.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:48:03.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:48:03.552 INFO:teuthology.orchestra.run.smithi039.stdout:client9166 2024-09-21T07:48:03.552 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:48:03.552 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:48:03.587 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get mds_max_snaps_per_dir 2024-09-21T07:48:03.870 INFO:teuthology.orchestra.run.smithi039.stdout:{"mds_max_snaps_per_dir":"100"} 2024-09-21T07:48:03.870 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_max_snaps_per_dir": "100" } 2024-09-21T07:48:03.870 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2024-09-21T07:48:04.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:04.100+0000 7f2f58800640 1 -- 172.21.15.39:0/634539594 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f38005150 msgr2=0x7f2f3800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:04.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:04.100+0000 7f2f58800640 1 --2- 172.21.15.39:0/634539594 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f38005150 0x7f2f3800b9c0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c006fd0 tx=0x7f2f38005830 comp rx=0 tx=0).stop 2024-09-21T07:48:04.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:04.100+0000 7f2f56800640 1 -- 172.21.15.39:0/634539594 reap_dead start 2024-09-21T07:48:04.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:04.100+0000 7f2f58800640 1 -- 172.21.15.39:0/634539594 shutdown_connections 2024-09-21T07:48:04.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:04.100+0000 7f2f58800640 1 -- 172.21.15.39:0/634539594 >> 172.21.15.39:0/634539594 conn(0x7f2f500b3fc0 msgr2=0x7f2f500b43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:04.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:04.100+0000 7f2f58800640 1 -- 172.21.15.39:0/634539594 shutdown_connections 2024-09-21T07:48:04.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:04.100+0000 7f2f58800640 1 -- 172.21.15.39:0/634539594 wait complete. 2024-09-21T07:48:04.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:04.102+0000 7f2f58800640 1 Processor -- start 2024-09-21T07:48:04.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:04.102+0000 7f2f58800640 1 -- start start 2024-09-21T07:48:04.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:04.104+0000 7f2f58800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f38005150 0x7f2f5014c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:04.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:04.104+0000 7f2f58800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2f501590e0 0x7f2f501f8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:04.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:04.104+0000 7f2f58800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f501f92e0 0x7f2f501f96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:04.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:04.104+0000 7f2f58800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5009afb0 con 0x7f2f501f92e0 2024-09-21T07:48:04.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:04.104+0000 7f2f58800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2f500ce240 con 0x7f2f38005150 2024-09-21T07:48:04.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:04.104+0000 7f2f58800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f2f5009b130 con 0x7f2f501590e0 2024-09-21T07:48:04.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:04.104+0000 7f2f56800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f501f92e0 0x7f2f501f96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:04.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:04.104+0000 7f2f56800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f501f92e0 0x7f2f501f96f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56270/0 (socket says 172.21.15.39:56270) 2024-09-21T07:48:04.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:04.104+0000 7f2f56800640 1 -- 172.21.15.39:0/125020123 learned_addr learned my addr 172.21.15.39:0/125020123 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:04.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:04.104+0000 7f2f55e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f38005150 0x7f2f5014c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:04.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:04.104+0000 7f2f55400640 1 --2- 172.21.15.39:0/125020123 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2f501590e0 0x7f2f501f8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:04.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:04.105+0000 7f2f56800640 1 -- 172.21.15.39:0/125020123 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2f501590e0 msgr2=0x7f2f501f8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:04.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:04.105+0000 7f2f56800640 1 --2- 172.21.15.39:0/125020123 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2f501590e0 0x7f2f501f8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:04.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:04.105+0000 7f2f56800640 1 -- 172.21.15.39:0/125020123 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f38005150 msgr2=0x7f2f5014c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:04.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:04.105+0000 7f2f56800640 1 --2- 172.21.15.39:0/125020123 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f38005150 0x7f2f5014c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:04.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:04.105+0000 7f2f56800640 1 -- 172.21.15.39:0/125020123 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f5015bec0 con 0x7f2f501f92e0 2024-09-21T07:48:04.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:04.105+0000 7f2f56800640 1 --2- 172.21.15.39:0/125020123 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f501f92e0 0x7f2f501f96f0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c00c700 tx=0x7f2f4c004bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:04.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:04.105+0000 7f2f4a000640 1 -- 172.21.15.39:0/125020123 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c02b9e0 con 0x7f2f501f92e0 2024-09-21T07:48:04.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:04.105+0000 7f2f4a000640 1 -- 172.21.15.39:0/125020123 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f4c02bc70 con 0x7f2f501f92e0 2024-09-21T07:48:04.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:04.105+0000 7f2f58800640 1 -- 172.21.15.39:0/125020123 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f5015c1a0 con 0x7f2f501f92e0 2024-09-21T07:48:04.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:04.105+0000 7f2f4a000640 1 -- 172.21.15.39:0/125020123 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c006300 con 0x7f2f501f92e0 2024-09-21T07:48:04.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:04.105+0000 7f2f58800640 1 -- 172.21.15.39:0/125020123 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f5015c760 con 0x7f2f501f92e0 2024-09-21T07:48:04.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:04.106+0000 7f2f58800640 1 -- 172.21.15.39:0/125020123 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f5009aa70 con 0x7f2f501f92e0 2024-09-21T07:48:04.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:04.108+0000 7f2f4a000640 1 -- 172.21.15.39:0/125020123 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2f4c0064a0 con 0x7f2f501f92e0 2024-09-21T07:48:04.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:04.109+0000 7f2f4a000640 1 --2- 172.21.15.39:0/125020123 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2f34078b90 0x7f2f3407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:04.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:04.109+0000 7f2f4a000640 1 -- 172.21.15.39:0/125020123 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(206..206 src has 1..206) ==== 6960+0+0 (secure 0 0 0) 0x7f2f4c0bb210 con 0x7f2f501f92e0 2024-09-21T07:48:04.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:04.109+0000 7f2f55e00640 1 --2- 172.21.15.39:0/125020123 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2f34078b90 0x7f2f3407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:04.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:04.110+0000 7f2f55e00640 1 --2- 172.21.15.39:0/125020123 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2f34078b90 0x7f2f3407b0a0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f2f500749c0 tx=0x7f2f44005f00 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:04.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:04.113+0000 7f2f4a000640 1 -- 172.21.15.39:0/125020123 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2f4c082b20 con 0x7f2f501f92e0 2024-09-21T07:48:04.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:04.353+0000 7f2f58800640 1 -- 172.21.15.39:0/125020123 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f2f5015bc50 con 0x7f2f501f92e0 2024-09-21T07:48:05.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.246+0000 7f2f4a000640 1 -- 172.21.15.39:0/125020123 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v215) ==== 167+0+0 (secure 0 0 0) 0x7f2f4c087550 con 0x7f2f501f92e0 2024-09-21T07:48:05.249 INFO:teuthology.orchestra.run.smithi039.stderr:enabled new snapshots 2024-09-21T07:48:05.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.250+0000 7f2f58800640 1 -- 172.21.15.39:0/125020123 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2f34078b90 msgr2=0x7f2f3407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:05.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.250+0000 7f2f58800640 1 --2- 172.21.15.39:0/125020123 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2f34078b90 0x7f2f3407b0a0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f2f500749c0 tx=0x7f2f44005f00 comp rx=0 tx=0).stop 2024-09-21T07:48:05.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.250+0000 7f2f58800640 1 -- 172.21.15.39:0/125020123 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f501f92e0 msgr2=0x7f2f501f96f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:05.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.250+0000 7f2f58800640 1 --2- 172.21.15.39:0/125020123 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f501f92e0 0x7f2f501f96f0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c00c700 tx=0x7f2f4c004bb0 comp rx=0 tx=0).stop 2024-09-21T07:48:05.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.250+0000 7f2f58800640 1 -- 172.21.15.39:0/125020123 shutdown_connections 2024-09-21T07:48:05.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.250+0000 7f2f58800640 1 --2- 172.21.15.39:0/125020123 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2f34078b90 0x7f2f3407b0a0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:05.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.250+0000 7f2f58800640 1 --2- 172.21.15.39:0/125020123 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f501f92e0 0x7f2f501f96f0 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:05.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.250+0000 7f2f58800640 1 --2- 172.21.15.39:0/125020123 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2f501590e0 0x7f2f501f8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:05.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.250+0000 7f2f58800640 1 --2- 172.21.15.39:0/125020123 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f38005150 0x7f2f5014c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:05.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.250+0000 7f2f58800640 1 -- 172.21.15.39:0/125020123 >> 172.21.15.39:0/125020123 conn(0x7f2f500b3fc0 msgr2=0x7f2f5006e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:05.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.250+0000 7f2f58800640 1 -- 172.21.15.39:0/125020123 shutdown_connections 2024-09-21T07:48:05.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.250+0000 7f2f58800640 1 -- 172.21.15.39:0/125020123 wait complete. 2024-09-21T07:48:05.280 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 2 2024-09-21T07:48:05.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.510+0000 7fb39a600640 1 -- 172.21.15.39:0/818248763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3940ccc60 msgr2=0x7fb3940cd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:05.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.510+0000 7fb39a600640 1 --2- 172.21.15.39:0/818248763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3940ccc60 0x7fb3940cd070 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fb38c005fd0 tx=0x7fb38c02a820 comp rx=0 tx=0).stop 2024-09-21T07:48:05.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.510+0000 7fb39a600640 1 -- 172.21.15.39:0/818248763 shutdown_connections 2024-09-21T07:48:05.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.510+0000 7fb39a600640 1 --2- 172.21.15.39:0/818248763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3940ccc60 0x7fb3940cd070 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:05.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.510+0000 7fb39a600640 1 --2- 172.21.15.39:0/818248763 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb3940b30a0 0x7fb3940cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:05.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.510+0000 7fb39a600640 1 -- 172.21.15.39:0/818248763 >> 172.21.15.39:0/818248763 conn(0x7fb3940b2460 msgr2=0x7fb39406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:05.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.510+0000 7fb39a600640 1 -- 172.21.15.39:0/818248763 shutdown_connections 2024-09-21T07:48:05.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.510+0000 7fb39a600640 1 -- 172.21.15.39:0/818248763 wait complete. 2024-09-21T07:48:05.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.512+0000 7fb39a600640 1 Processor -- start 2024-09-21T07:48:05.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.513+0000 7fb39a600640 1 -- start start 2024-09-21T07:48:05.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.514+0000 7fb39a600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3940b30a0 0x7fb3941f78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:05.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.514+0000 7fb39a600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb3940ccc60 0x7fb3941557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:05.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.514+0000 7fb39a600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb39415d830 0x7fb3940b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:05.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.514+0000 7fb39a600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb3940af490 con 0x7fb39415d830 2024-09-21T07:48:05.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.514+0000 7fb39a600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb3940af310 con 0x7fb3940b30a0 2024-09-21T07:48:05.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.514+0000 7fb39a600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb3940af610 con 0x7fb3940ccc60 2024-09-21T07:48:05.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.514+0000 7fb393e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb39415d830 0x7fb3940b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:05.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.514+0000 7fb393e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb39415d830 0x7fb3940b0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56288/0 (socket says 172.21.15.39:56288) 2024-09-21T07:48:05.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.514+0000 7fb393e00640 1 -- 172.21.15.39:0/1912643090 learned_addr learned my addr 172.21.15.39:0/1912643090 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:05.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.515+0000 7fb393400640 1 --2- 172.21.15.39:0/1912643090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3940b30a0 0x7fb3941f78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:05.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.515+0000 7fb393e00640 1 -- 172.21.15.39:0/1912643090 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb3940ccc60 msgr2=0x7fb3941557f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:05.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.515+0000 7fb38be00640 1 --2- 172.21.15.39:0/1912643090 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb3940ccc60 0x7fb3941557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:05.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.515+0000 7fb393e00640 1 --2- 172.21.15.39:0/1912643090 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb3940ccc60 0x7fb3941557f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:05.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.515+0000 7fb393e00640 1 -- 172.21.15.39:0/1912643090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3940b30a0 msgr2=0x7fb3941f78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:05.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.515+0000 7fb393e00640 1 --2- 172.21.15.39:0/1912643090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3940b30a0 0x7fb3941f78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:05.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.515+0000 7fb393e00640 1 -- 172.21.15.39:0/1912643090 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3941f4ae0 con 0x7fb39415d830 2024-09-21T07:48:05.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.515+0000 7fb393400640 1 --2- 172.21.15.39:0/1912643090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3940b30a0 0x7fb3941f78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:48:05.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.515+0000 7fb38be00640 1 --2- 172.21.15.39:0/1912643090 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb3940ccc60 0x7fb3941557f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:48:05.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.515+0000 7fb393e00640 1 --2- 172.21.15.39:0/1912643090 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb39415d830 0x7fb3940b0b00 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fb38c006010 tx=0x7fb38c032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:05.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.515+0000 7fb38b400640 1 -- 172.21.15.39:0/1912643090 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38c037040 con 0x7fb39415d830 2024-09-21T07:48:05.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.515+0000 7fb38b400640 1 -- 172.21.15.39:0/1912643090 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb38c042070 con 0x7fb39415d830 2024-09-21T07:48:05.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.515+0000 7fb39a600640 1 -- 172.21.15.39:0/1912643090 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3941e7990 con 0x7fb39415d830 2024-09-21T07:48:05.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.516+0000 7fb38b400640 1 -- 172.21.15.39:0/1912643090 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38c0042a0 con 0x7fb39415d830 2024-09-21T07:48:05.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.516+0000 7fb39a600640 1 -- 172.21.15.39:0/1912643090 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3941e7f50 con 0x7fb39415d830 2024-09-21T07:48:05.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.517+0000 7fb39a600640 1 -- 172.21.15.39:0/1912643090 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3940cb8a0 con 0x7fb39415d830 2024-09-21T07:48:05.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.518+0000 7fb38b400640 1 -- 172.21.15.39:0/1912643090 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb38c004440 con 0x7fb39415d830 2024-09-21T07:48:05.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.519+0000 7fb38b400640 1 --2- 172.21.15.39:0/1912643090 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb370078b90 0x7fb37007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:05.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.519+0000 7fb38b400640 1 -- 172.21.15.39:0/1912643090 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(206..206 src has 1..206) ==== 6960+0+0 (secure 0 0 0) 0x7fb38c0bbfa0 con 0x7fb39415d830 2024-09-21T07:48:05.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.519+0000 7fb393400640 1 --2- 172.21.15.39:0/1912643090 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb370078b90 0x7fb37007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:05.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.520+0000 7fb393400640 1 --2- 172.21.15.39:0/1912643090 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb370078b90 0x7fb37007b0a0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fb384009ab0 tx=0x7fb384005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:05.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.523+0000 7fb38b400640 1 -- 172.21.15.39:0/1912643090 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb38c083f00 con 0x7fb39415d830 2024-09-21T07:48:05.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:05.761+0000 7fb39a600640 1 -- 172.21.15.39:0/1912643090 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) -- 0x7fb3940adf10 con 0x7fb39415d830 2024-09-21T07:48:06.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.258+0000 7fb38b400640 1 -- 172.21.15.39:0/1912643090 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v216) ==== 105+0+0 (secure 0 0 0) 0x7fb38c083800 con 0x7fb39415d830 2024-09-21T07:48:06.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.260+0000 7fb39a600640 1 -- 172.21.15.39:0/1912643090 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb370078b90 msgr2=0x7fb37007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:06.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.260+0000 7fb39a600640 1 --2- 172.21.15.39:0/1912643090 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb370078b90 0x7fb37007b0a0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fb384009ab0 tx=0x7fb384005f90 comp rx=0 tx=0).stop 2024-09-21T07:48:06.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.260+0000 7fb39a600640 1 -- 172.21.15.39:0/1912643090 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb39415d830 msgr2=0x7fb3940b0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:06.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.260+0000 7fb39a600640 1 --2- 172.21.15.39:0/1912643090 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb39415d830 0x7fb3940b0b00 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fb38c006010 tx=0x7fb38c032c90 comp rx=0 tx=0).stop 2024-09-21T07:48:06.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.261+0000 7fb39a600640 1 -- 172.21.15.39:0/1912643090 shutdown_connections 2024-09-21T07:48:06.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.261+0000 7fb39a600640 1 --2- 172.21.15.39:0/1912643090 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb370078b90 0x7fb37007b0a0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:06.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.261+0000 7fb39a600640 1 --2- 172.21.15.39:0/1912643090 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb39415d830 0x7fb3940b0b00 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:06.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.261+0000 7fb39a600640 1 --2- 172.21.15.39:0/1912643090 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb3940ccc60 0x7fb3941557f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:06.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.261+0000 7fb39a600640 1 --2- 172.21.15.39:0/1912643090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3940b30a0 0x7fb3941f78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:06.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.261+0000 7fb39a600640 1 -- 172.21.15.39:0/1912643090 >> 172.21.15.39:0/1912643090 conn(0x7fb3940b2460 msgr2=0x7fb3940ae7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:06.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.261+0000 7fb39a600640 1 -- 172.21.15.39:0/1912643090 shutdown_connections 2024-09-21T07:48:06.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.261+0000 7fb39a600640 1 -- 172.21.15.39:0/1912643090 wait complete. 2024-09-21T07:48:06.294 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:48:06.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.518+0000 7fde52e00640 1 -- 172.21.15.39:0/3323400588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde4c151b80 msgr2=0x7fde4c070570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:06.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.518+0000 7fde52e00640 1 --2- 172.21.15.39:0/3323400588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde4c151b80 0x7fde4c070570 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7fde4c0cce90 tx=0x7fde3402a4d0 comp rx=0 tx=0).stop 2024-09-21T07:48:06.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.518+0000 7fde52e00640 1 -- 172.21.15.39:0/3323400588 shutdown_connections 2024-09-21T07:48:06.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.518+0000 7fde52e00640 1 --2- 172.21.15.39:0/3323400588 >> v2:172.21.15.148:3300/0 conn(0x7fde4c070ab0 0x7fde4c0cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:06.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.518+0000 7fde52e00640 1 --2- 172.21.15.39:0/3323400588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde4c151b80 0x7fde4c070570 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:06.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.518+0000 7fde52e00640 1 --2- 172.21.15.39:0/3323400588 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde4c071220 0x7fde4c151640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:06.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.518+0000 7fde52e00640 1 -- 172.21.15.39:0/3323400588 >> 172.21.15.39:0/3323400588 conn(0x7fde4c06cfc0 msgr2=0x7fde4c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:06.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.519+0000 7fde52e00640 1 -- 172.21.15.39:0/3323400588 shutdown_connections 2024-09-21T07:48:06.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.519+0000 7fde52e00640 1 -- 172.21.15.39:0/3323400588 wait complete. 2024-09-21T07:48:06.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.521+0000 7fde52e00640 1 Processor -- start 2024-09-21T07:48:06.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.521+0000 7fde52e00640 1 -- start start 2024-09-21T07:48:06.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.522+0000 7fde52e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde4c070ab0 0x7fde4c14f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:06.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.522+0000 7fde52e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde4c071220 0x7fde4c1f6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:06.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.523+0000 7fde52e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde4c151b80 0x7fde4c0cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:06.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.523+0000 7fde52e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fde4c0d00c0 con 0x7fde4c070ab0 2024-09-21T07:48:06.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.523+0000 7fde52e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fde4c0cff40 con 0x7fde4c071220 2024-09-21T07:48:06.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.523+0000 7fde52e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fde4c0d0240 con 0x7fde4c151b80 2024-09-21T07:48:06.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.523+0000 7fde4be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde4c070ab0 0x7fde4c14f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:06.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.523+0000 7fde4be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde4c070ab0 0x7fde4c14f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56302/0 (socket says 172.21.15.39:56302) 2024-09-21T07:48:06.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.523+0000 7fde4be00640 1 -- 172.21.15.39:0/1647644669 learned_addr learned my addr 172.21.15.39:0/1647644669 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:06.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.523+0000 7fde43400640 1 --2- 172.21.15.39:0/1647644669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde4c071220 0x7fde4c1f6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:06.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.523+0000 7fde50e00640 1 --2- 172.21.15.39:0/1647644669 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde4c151b80 0x7fde4c0cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:06.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.523+0000 7fde4be00640 1 -- 172.21.15.39:0/1647644669 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde4c151b80 msgr2=0x7fde4c0cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:06.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.523+0000 7fde4be00640 1 --2- 172.21.15.39:0/1647644669 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde4c151b80 0x7fde4c0cd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:06.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.523+0000 7fde4be00640 1 -- 172.21.15.39:0/1647644669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde4c071220 msgr2=0x7fde4c1f6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:06.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.523+0000 7fde4be00640 1 --2- 172.21.15.39:0/1647644669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde4c071220 0x7fde4c1f6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:06.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.523+0000 7fde4be00640 1 -- 172.21.15.39:0/1647644669 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde4c1f66a0 con 0x7fde4c070ab0 2024-09-21T07:48:06.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.524+0000 7fde4be00640 1 --2- 172.21.15.39:0/1647644669 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde4c070ab0 0x7fde4c14f350 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fde3c00d750 tx=0x7fde3c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:06.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.524+0000 7fde48c00640 1 -- 172.21.15.39:0/1647644669 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c017070 con 0x7fde4c070ab0 2024-09-21T07:48:06.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.524+0000 7fde48c00640 1 -- 172.21.15.39:0/1647644669 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde3c004310 con 0x7fde4c070ab0 2024-09-21T07:48:06.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.524+0000 7fde52e00640 1 -- 172.21.15.39:0/1647644669 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde4c1e6f40 con 0x7fde4c070ab0 2024-09-21T07:48:06.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.524+0000 7fde48c00640 1 -- 172.21.15.39:0/1647644669 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c00de60 con 0x7fde4c070ab0 2024-09-21T07:48:06.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.524+0000 7fde52e00640 1 -- 172.21.15.39:0/1647644669 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde4c1e7500 con 0x7fde4c070ab0 2024-09-21T07:48:06.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.525+0000 7fde52e00640 1 -- 172.21.15.39:0/1647644669 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde4c0afc10 con 0x7fde4c070ab0 2024-09-21T07:48:06.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.527+0000 7fde48c00640 1 -- 172.21.15.39:0/1647644669 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fde3c009c30 con 0x7fde4c070ab0 2024-09-21T07:48:06.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.527+0000 7fde48c00640 1 --2- 172.21.15.39:0/1647644669 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fde28078b90 0x7fde2807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:06.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.528+0000 7fde48c00640 1 -- 172.21.15.39:0/1647644669 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(206..206 src has 1..206) ==== 6960+0+0 (secure 0 0 0) 0x7fde3c09b610 con 0x7fde4c070ab0 2024-09-21T07:48:06.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.528+0000 7fde43400640 1 --2- 172.21.15.39:0/1647644669 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fde28078b90 0x7fde2807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:06.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.529+0000 7fde43400640 1 --2- 172.21.15.39:0/1647644669 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fde28078b90 0x7fde2807b0a0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fde34033000 tx=0x7fde34004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:06.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.532+0000 7fde48c00640 1 -- 172.21.15.39:0/1647644669 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fde4c0afc10 con 0x7fde4c070ab0 2024-09-21T07:48:06.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.769+0000 7fde52e00640 1 -- 172.21.15.39:0/1647644669 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde4c151640 con 0x7fde4c070ab0 2024-09-21T07:48:06.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.770+0000 7fde48c00640 1 -- 172.21.15.39:0/1647644669 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 217 v217) ==== 95+0+5255 (secure 0 0 0) 0x7fde3c020020 con 0x7fde4c070ab0 2024-09-21T07:48:06.772 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:48:06.772 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":217,"btime":"2024-09-21T07:48:06:262910+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7988,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1912631346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1912631346},{"type":"v1","addr":"172.21.15.148:6837","nonce":1912631346}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":209},{"gid":9043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2220628944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2220628944},{"type":"v1","addr":"172.21.15.148:6836","nonce":2220628944}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":208}],"filesystems":[{"mdsmap":{"epoch":217,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:47:54.691835+0000","modified":"2024-09-21T07:48:06.262893+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":9852,"mds_1":9049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9049":{"gid":9049,"name":"a","rank":1,"incarnation":217,"state":"up:creating","state_seq":1,"addr":"172.21.15.39:6833/1947479913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1947479913},{"type":"v1","addr":"172.21.15.39:6833","nonce":1947479913}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_9852":{"gid":9852,"name":"c","rank":0,"incarnation":210,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/2356847598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2356847598},{"type":"v1","addr":"172.21.15.39:6835","nonce":2356847598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9852,"qdb_cluster":[9852]},"id":11}]} 2024-09-21T07:48:06.773 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 217 2024-09-21T07:48:06.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.773+0000 7fde52e00640 1 -- 172.21.15.39:0/1647644669 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fde28078b90 msgr2=0x7fde2807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:06.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.773+0000 7fde52e00640 1 --2- 172.21.15.39:0/1647644669 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fde28078b90 0x7fde2807b0a0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fde34033000 tx=0x7fde34004730 comp rx=0 tx=0).stop 2024-09-21T07:48:06.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.773+0000 7fde52e00640 1 -- 172.21.15.39:0/1647644669 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde4c070ab0 msgr2=0x7fde4c14f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:06.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.773+0000 7fde52e00640 1 --2- 172.21.15.39:0/1647644669 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde4c070ab0 0x7fde4c14f350 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fde3c00d750 tx=0x7fde3c00dc10 comp rx=0 tx=0).stop 2024-09-21T07:48:06.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.773+0000 7fde52e00640 1 -- 172.21.15.39:0/1647644669 shutdown_connections 2024-09-21T07:48:06.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.773+0000 7fde52e00640 1 --2- 172.21.15.39:0/1647644669 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fde28078b90 0x7fde2807b0a0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:06.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.773+0000 7fde52e00640 1 --2- 172.21.15.39:0/1647644669 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde4c151b80 0x7fde4c0cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:06.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.773+0000 7fde52e00640 1 --2- 172.21.15.39:0/1647644669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde4c071220 0x7fde4c1f6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:06.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.773+0000 7fde52e00640 1 --2- 172.21.15.39:0/1647644669 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde4c070ab0 0x7fde4c14f350 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:06.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.773+0000 7fde52e00640 1 -- 172.21.15.39:0/1647644669 >> 172.21.15.39:0/1647644669 conn(0x7fde4c06cfc0 msgr2=0x7fde4c15d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:06.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.773+0000 7fde52e00640 1 -- 172.21.15.39:0/1647644669 shutdown_connections 2024-09-21T07:48:06.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:06.773+0000 7fde52e00640 1 -- 172.21.15.39:0/1647644669 wait complete. 2024-09-21T07:48:07.200 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 217, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:47:54.691835+0000', 'modified': '2024-09-21T07:48:06.262893+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 9852, 'mds_1': 9049}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9049': {'gid': 9049, 'name': 'a', 'rank': 1, 'incarnation': 217, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.39:6833/1947479913', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 1947479913}, {'type': 'v1', 'addr': '172.21.15.39:6833', 'nonce': 1947479913}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_9852': {'gid': 9852, 'name': 'c', 'rank': 0, 'incarnation': 210, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.39:6835/2356847598', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6834', 'nonce': 2356847598}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 2356847598}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9852, 'qdb_cluster': [9852]} 2024-09-21T07:48:07.200 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9049:up:creating 2024-09-21T07:48:08.201 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:48:08.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.439+0000 7f45b7e00640 1 -- 172.21.15.39:0/181961573 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45b815aeb0 msgr2=0x7f45b815b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:08.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.439+0000 7f45b7e00640 1 --2- 172.21.15.39:0/181961573 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45b815aeb0 0x7f45b815b2c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f45a4009830 tx=0x7f45a402a510 comp rx=0 tx=0).stop 2024-09-21T07:48:08.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.439+0000 7f45b7e00640 1 -- 172.21.15.39:0/181961573 shutdown_connections 2024-09-21T07:48:08.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.439+0000 7f45b7e00640 1 --2- 172.21.15.39:0/181961573 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45b815aeb0 0x7f45b815b2c0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:08.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.439+0000 7f45b7e00640 1 --2- 172.21.15.39:0/181961573 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45b80b4f40 0x7f45b8151870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:08.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.439+0000 7f45b7e00640 1 -- 172.21.15.39:0/181961573 >> 172.21.15.39:0/181961573 conn(0x7f45b806cfc0 msgr2=0x7f45b806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:08.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.439+0000 7f45b7e00640 1 -- 172.21.15.39:0/181961573 shutdown_connections 2024-09-21T07:48:08.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.439+0000 7f45b7e00640 1 -- 172.21.15.39:0/181961573 wait complete. 2024-09-21T07:48:08.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.441+0000 7f45b7e00640 1 Processor -- start 2024-09-21T07:48:08.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.441+0000 7f45b7e00640 1 -- start start 2024-09-21T07:48:08.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.443+0000 7f45b7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45b80b4f40 0x7f45b8159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:08.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.443+0000 7f45b7e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45b80d0600 0x7f45b815a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:08.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.443+0000 7f45b7e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45b815aeb0 0x7f45b81f8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:08.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.443+0000 7f45b7e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f45b815e7e0 con 0x7f45b80b4f40 2024-09-21T07:48:08.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.443+0000 7f45b7e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f45b80710a0 con 0x7f45b80d0600 2024-09-21T07:48:08.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.443+0000 7f45b7e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f45b80cf550 con 0x7f45b815aeb0 2024-09-21T07:48:08.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.443+0000 7f45b6a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45b80b4f40 0x7f45b8159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:08.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.443+0000 7f45b6a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45b80b4f40 0x7f45b8159e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56304/0 (socket says 172.21.15.39:56304) 2024-09-21T07:48:08.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.443+0000 7f45b6a00640 1 -- 172.21.15.39:0/3183793579 learned_addr learned my addr 172.21.15.39:0/3183793579 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:08.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.443+0000 7f45b6000640 1 --2- 172.21.15.39:0/3183793579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45b80d0600 0x7f45b815a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:08.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.443+0000 7f45b7400640 1 --2- 172.21.15.39:0/3183793579 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45b815aeb0 0x7f45b81f8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:08.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.444+0000 7f45b6a00640 1 -- 172.21.15.39:0/3183793579 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45b815aeb0 msgr2=0x7f45b81f8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:08.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.444+0000 7f45b6a00640 1 --2- 172.21.15.39:0/3183793579 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45b815aeb0 0x7f45b81f8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:08.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.444+0000 7f45b6a00640 1 -- 172.21.15.39:0/3183793579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45b80d0600 msgr2=0x7f45b815a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:08.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.444+0000 7f45b6a00640 1 --2- 172.21.15.39:0/3183793579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45b80d0600 0x7f45b815a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:08.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.444+0000 7f45b6a00640 1 -- 172.21.15.39:0/3183793579 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b8158870 con 0x7f45b80b4f40 2024-09-21T07:48:08.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.444+0000 7f45b6a00640 1 --2- 172.21.15.39:0/3183793579 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45b80b4f40 0x7f45b8159e20 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f45a800e9f0 tx=0x7f45a800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:08.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.444+0000 7f45aea00640 1 -- 172.21.15.39:0/3183793579 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a800cd80 con 0x7f45b80b4f40 2024-09-21T07:48:08.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.444+0000 7f45aea00640 1 -- 172.21.15.39:0/3183793579 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45a8004020 con 0x7f45b80b4f40 2024-09-21T07:48:08.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.444+0000 7f45b7e00640 1 -- 172.21.15.39:0/3183793579 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45b8155670 con 0x7f45b80b4f40 2024-09-21T07:48:08.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.444+0000 7f45aea00640 1 -- 172.21.15.39:0/3183793579 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a8004470 con 0x7f45b80b4f40 2024-09-21T07:48:08.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.444+0000 7f45b7e00640 1 -- 172.21.15.39:0/3183793579 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45b8155c30 con 0x7f45b80b4f40 2024-09-21T07:48:08.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.445+0000 7f45b7e00640 1 -- 172.21.15.39:0/3183793579 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45b8151870 con 0x7f45b80b4f40 2024-09-21T07:48:08.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.447+0000 7f45aea00640 1 -- 172.21.15.39:0/3183793579 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f45a80035f0 con 0x7f45b80b4f40 2024-09-21T07:48:08.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.448+0000 7f45aea00640 1 --2- 172.21.15.39:0/3183793579 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4598078bc0 0x7f459807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:08.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.448+0000 7f45aea00640 1 -- 172.21.15.39:0/3183793579 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(206..206 src has 1..206) ==== 6960+0+0 (secure 0 0 0) 0x7f45a809be70 con 0x7f45b80b4f40 2024-09-21T07:48:08.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.448+0000 7f45b6000640 1 --2- 172.21.15.39:0/3183793579 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4598078bc0 0x7f459807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:08.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.449+0000 7f45b6000640 1 --2- 172.21.15.39:0/3183793579 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4598078bc0 0x7f459807b0d0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f45a4034000 tx=0x7f45a4004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:08.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.452+0000 7f45aea00640 1 -- 172.21.15.39:0/3183793579 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f45a8063cd0 con 0x7f45b80b4f40 2024-09-21T07:48:08.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.688+0000 7f45b7e00640 1 -- 172.21.15.39:0/3183793579 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45b806d710 con 0x7f45b80b4f40 2024-09-21T07:48:08.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.689+0000 7f45aea00640 1 -- 172.21.15.39:0/3183793579 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) ==== 95+0+5258 (secure 0 0 0) 0x7f45a80635d0 con 0x7f45b80b4f40 2024-09-21T07:48:08.691 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:48:08.691 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":218,"btime":"2024-09-21T07:48:07:264514+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7988,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1912631346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1912631346},{"type":"v1","addr":"172.21.15.148:6837","nonce":1912631346}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":209},{"gid":9043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2220628944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2220628944},{"type":"v1","addr":"172.21.15.148:6836","nonce":2220628944}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":208}],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:47:54.691835+0000","modified":"2024-09-21T07:48:07.264503+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":9852,"mds_1":9049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9049":{"gid":9049,"name":"a","rank":1,"incarnation":217,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6833/1947479913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1947479913},{"type":"v1","addr":"172.21.15.39:6833","nonce":1947479913}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_9852":{"gid":9852,"name":"c","rank":0,"incarnation":210,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/2356847598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2356847598},{"type":"v1","addr":"172.21.15.39:6835","nonce":2356847598}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9852,"qdb_cluster":[9049,9852]},"id":11}]} 2024-09-21T07:48:08.691 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 218 2024-09-21T07:48:08.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.691+0000 7f45b7e00640 1 -- 172.21.15.39:0/3183793579 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4598078bc0 msgr2=0x7f459807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:08.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.691+0000 7f45b7e00640 1 --2- 172.21.15.39:0/3183793579 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4598078bc0 0x7f459807b0d0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f45a4034000 tx=0x7f45a4004730 comp rx=0 tx=0).stop 2024-09-21T07:48:08.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.691+0000 7f45b7e00640 1 -- 172.21.15.39:0/3183793579 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45b80b4f40 msgr2=0x7f45b8159e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:08.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.691+0000 7f45b7e00640 1 --2- 172.21.15.39:0/3183793579 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45b80b4f40 0x7f45b8159e20 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f45a800e9f0 tx=0x7f45a800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:48:08.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.691+0000 7f45b7e00640 1 -- 172.21.15.39:0/3183793579 shutdown_connections 2024-09-21T07:48:08.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.692+0000 7f45b7e00640 1 --2- 172.21.15.39:0/3183793579 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4598078bc0 0x7f459807b0d0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:08.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.692+0000 7f45b7e00640 1 --2- 172.21.15.39:0/3183793579 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45b815aeb0 0x7f45b81f8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:08.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.692+0000 7f45b7e00640 1 --2- 172.21.15.39:0/3183793579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45b80d0600 0x7f45b815a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:08.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.692+0000 7f45b7e00640 1 --2- 172.21.15.39:0/3183793579 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45b80b4f40 0x7f45b8159e20 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:08.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.692+0000 7f45b7e00640 1 -- 172.21.15.39:0/3183793579 >> 172.21.15.39:0/3183793579 conn(0x7f45b806cfc0 msgr2=0x7f45b80ae2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:08.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.692+0000 7f45b7e00640 1 -- 172.21.15.39:0/3183793579 shutdown_connections 2024-09-21T07:48:08.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:08.692+0000 7f45b7e00640 1 -- 172.21.15.39:0/3183793579 wait complete. 2024-09-21T07:48:09.119 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:47:54.691835+0000', 'modified': '2024-09-21T07:48:07.264503+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 9852, 'mds_1': 9049}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9049': {'gid': 9049, 'name': 'a', 'rank': 1, 'incarnation': 217, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.39:6833/1947479913', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 1947479913}, {'type': 'v1', 'addr': '172.21.15.39:6833', 'nonce': 1947479913}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_9852': {'gid': 9852, 'name': 'c', 'rank': 0, 'incarnation': 210, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.39:6835/2356847598', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6834', 'nonce': 2356847598}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 2356847598}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9852, 'qdb_cluster': [9049, 9852]} 2024-09-21T07:48:09.119 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2024-09-21T07:48:09.119 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T07:48:09.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.345+0000 7faeaec00640 1 -- 172.21.15.39:0/2896772241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faea80ccc60 msgr2=0x7faea80cd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:09.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.345+0000 7faeaec00640 1 --2- 172.21.15.39:0/2896772241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faea80ccc60 0x7faea80cd070 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7faea8075160 tx=0x7faea002a820 comp rx=0 tx=0).stop 2024-09-21T07:48:09.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.345+0000 7faeaec00640 1 -- 172.21.15.39:0/2896772241 shutdown_connections 2024-09-21T07:48:09.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.345+0000 7faeaec00640 1 --2- 172.21.15.39:0/2896772241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faea80ccc60 0x7faea80cd070 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.345+0000 7faeaec00640 1 --2- 172.21.15.39:0/2896772241 >> v2:172.21.15.148:3300/0 conn(0x7faea80cb8a0 0x7faea8156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.345+0000 7faeaec00640 1 -- 172.21.15.39:0/2896772241 >> 172.21.15.39:0/2896772241 conn(0x7faea80b2460 msgr2=0x7faea806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:09.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.345+0000 7faeaec00640 1 -- 172.21.15.39:0/2896772241 shutdown_connections 2024-09-21T07:48:09.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.345+0000 7faeaec00640 1 -- 172.21.15.39:0/2896772241 wait complete. 2024-09-21T07:48:09.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.348+0000 7faeaec00640 1 Processor -- start 2024-09-21T07:48:09.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.348+0000 7faeaec00640 1 -- start start 2024-09-21T07:48:09.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.349+0000 7faeaec00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faea80b30a0 0x7faea81f8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:09.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.349+0000 7faeaec00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faea80cb8a0 0x7faea815b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:09.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.349+0000 7faeaec00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faea80ccc60 0x7faea815bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:09.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.349+0000 7faeaec00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7faea80af410 con 0x7faea80ccc60 2024-09-21T07:48:09.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.349+0000 7faeaec00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7faea80af290 con 0x7faea80b30a0 2024-09-21T07:48:09.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.349+0000 7faeaec00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7faea80af590 con 0x7faea80cb8a0 2024-09-21T07:48:09.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.350+0000 7faeacc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faea80ccc60 0x7faea815bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:09.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.350+0000 7faeacc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faea80ccc60 0x7faea815bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56310/0 (socket says 172.21.15.39:56310) 2024-09-21T07:48:09.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.350+0000 7faeacc00640 1 -- 172.21.15.39:0/2474660531 learned_addr learned my addr 172.21.15.39:0/2474660531 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:09.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.350+0000 7faea7e00640 1 --2- 172.21.15.39:0/2474660531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faea80b30a0 0x7faea81f8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:09.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.350+0000 7faea7400640 1 --2- 172.21.15.39:0/2474660531 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faea80cb8a0 0x7faea815b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:09.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.350+0000 7faeacc00640 1 -- 172.21.15.39:0/2474660531 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faea80cb8a0 msgr2=0x7faea815b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:09.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.350+0000 7faeacc00640 1 --2- 172.21.15.39:0/2474660531 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faea80cb8a0 0x7faea815b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.350+0000 7faeacc00640 1 -- 172.21.15.39:0/2474660531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faea80b30a0 msgr2=0x7faea81f8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:09.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.350+0000 7faeacc00640 1 --2- 172.21.15.39:0/2474660531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faea80b30a0 0x7faea81f8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.350+0000 7faeacc00640 1 -- 172.21.15.39:0/2474660531 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faea8074fa0 con 0x7faea80ccc60 2024-09-21T07:48:09.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.350+0000 7faea7e00640 1 --2- 172.21.15.39:0/2474660531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faea80b30a0 0x7faea81f8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:48:09.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.350+0000 7faea7400640 1 --2- 172.21.15.39:0/2474660531 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faea80cb8a0 0x7faea815b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:48:09.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.350+0000 7faeacc00640 1 --2- 172.21.15.39:0/2474660531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faea80ccc60 0x7faea815bce0 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7faea0009b30 tx=0x7faea0032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:09.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.351+0000 7fae9fe00640 1 -- 172.21.15.39:0/2474660531 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea0037040 con 0x7faea80ccc60 2024-09-21T07:48:09.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.351+0000 7fae9fe00640 1 -- 172.21.15.39:0/2474660531 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faea0042070 con 0x7faea80ccc60 2024-09-21T07:48:09.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.351+0000 7fae9fe00640 1 -- 172.21.15.39:0/2474660531 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea0004280 con 0x7faea80ccc60 2024-09-21T07:48:09.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.351+0000 7faeaec00640 1 -- 172.21.15.39:0/2474660531 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faea80c3e20 con 0x7faea80ccc60 2024-09-21T07:48:09.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.351+0000 7faeaec00640 1 -- 172.21.15.39:0/2474660531 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faea80c40c0 con 0x7faea80ccc60 2024-09-21T07:48:09.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.353+0000 7fae9fe00640 1 -- 172.21.15.39:0/2474660531 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7faea0004420 con 0x7faea80ccc60 2024-09-21T07:48:09.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.355+0000 7fae9fe00640 1 --2- 172.21.15.39:0/2474660531 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae88078bc0 0x7fae8807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:09.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.355+0000 7fae9fe00640 1 -- 172.21.15.39:0/2474660531 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(206..206 src has 1..206) ==== 6960+0+0 (secure 0 0 0) 0x7faea00bc240 con 0x7faea80ccc60 2024-09-21T07:48:09.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.355+0000 7faeb5dd0740 1 Processor -- start 2024-09-21T07:48:09.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.356+0000 7faea7e00640 1 --2- 172.21.15.39:0/2474660531 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae88078bc0 0x7fae8807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:09.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.356+0000 7faeb5dd0740 1 -- start start 2024-09-21T07:48:09.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.356+0000 7faea7e00640 1 --2- 172.21.15.39:0/2474660531 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae88078bc0 0x7fae8807b0d0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fae98000f60 tx=0x7fae98009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:09.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.357+0000 7faeb5dd0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620b7078270 0x5620b7448970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:09.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.357+0000 7faeb5dd0740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5620b704f3b0 0x5620b7448eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:09.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.357+0000 7faeb5dd0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620b7057590 0x5620b7418f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:09.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.357+0000 7faeb5dd0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5620b73a2430 con 0x5620b7078270 2024-09-21T07:48:09.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.357+0000 7faeb5dd0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5620b73925d0 con 0x5620b7057590 2024-09-21T07:48:09.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.357+0000 7faeb5dd0740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x5620b7381a00 con 0x5620b704f3b0 2024-09-21T07:48:09.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.357+0000 7faea7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620b7078270 0x5620b7448970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:09.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.357+0000 7faea7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620b7078270 0x5620b7448970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56312/0 (socket says 172.21.15.39:56312) 2024-09-21T07:48:09.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.357+0000 7faea7e00640 1 -- 172.21.15.39:0/2858070228 learned_addr learned my addr 172.21.15.39:0/2858070228 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:09.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.357+0000 7faeacc00640 1 --2- 172.21.15.39:0/2858070228 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620b7057590 0x5620b7418f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:09.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.357+0000 7faea7400640 1 --2- 172.21.15.39:0/2858070228 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5620b704f3b0 0x5620b7448eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:09.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.358+0000 7faea7e00640 1 -- 172.21.15.39:0/2858070228 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5620b704f3b0 msgr2=0x5620b7448eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:48:09.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.358+0000 7faea7e00640 1 --2- 172.21.15.39:0/2858070228 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5620b704f3b0 0x5620b7448eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.358+0000 7faea7e00640 1 -- 172.21.15.39:0/2858070228 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620b7057590 msgr2=0x5620b7418f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:48:09.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.358+0000 7faea7e00640 1 --2- 172.21.15.39:0/2858070228 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620b7057590 0x5620b7418f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.358+0000 7faea7e00640 1 -- 172.21.15.39:0/2858070228 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620b734a520 con 0x5620b7078270 2024-09-21T07:48:09.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.358+0000 7faea7e00640 1 --2- 172.21.15.39:0/2858070228 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620b7078270 0x5620b7448970 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7fae98005360 tx=0x7fae98010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa03c64633bb44d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:09.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.358+0000 7fae93e00640 1 -- 172.21.15.39:0/2858070228 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98003a20 con 0x5620b7078270 2024-09-21T07:48:09.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.358+0000 7fae93e00640 1 -- 172.21.15.39:0/2858070228 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae98003bc0 con 0x5620b7078270 2024-09-21T07:48:09.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.358+0000 7fae93e00640 1 -- 172.21.15.39:0/2858070228 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98020450 con 0x5620b7078270 2024-09-21T07:48:09.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.359+0000 7faeb5dd0740 1 -- 172.21.15.39:0/2858070228 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620b7078270 msgr2=0x5620b7448970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:09.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.359+0000 7faeb5dd0740 1 --2- 172.21.15.39:0/2858070228 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620b7078270 0x5620b7448970 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7fae98005360 tx=0x7fae98010000 comp rx=0 tx=0).stop 2024-09-21T07:48:09.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.359+0000 7faeb5dd0740 1 -- 172.21.15.39:0/2858070228 shutdown_connections 2024-09-21T07:48:09.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.359+0000 7faeb5dd0740 1 --2- 172.21.15.39:0/2858070228 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620b7057590 0x5620b7418f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.359+0000 7faeb5dd0740 1 --2- 172.21.15.39:0/2858070228 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5620b704f3b0 0x5620b7448eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.359+0000 7faeb5dd0740 1 --2- 172.21.15.39:0/2858070228 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620b7078270 0x5620b7448970 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.359+0000 7faeb5dd0740 1 -- 172.21.15.39:0/2858070228 >> 172.21.15.39:0/2858070228 conn(0x5620b706c3c0 msgr2=0x5620b741e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:09.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.359+0000 7faeb5dd0740 1 -- 172.21.15.39:0/2858070228 shutdown_connections 2024-09-21T07:48:09.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.359+0000 7faeb5dd0740 1 -- 172.21.15.39:0/2858070228 wait complete. 2024-09-21T07:48:09.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.360+0000 7faeb5dd0740 1 -- start start 2024-09-21T07:48:09.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.360+0000 7faeb5dd0740 1 Processor -- start 2024-09-21T07:48:09.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.362+0000 7faeb5dd0740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5620b704f3b0 0x5620b7443d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:09.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.362+0000 7faeb5dd0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620b7057590 0x5620b74457e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:09.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.362+0000 7faeb5dd0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620b7078270 0x5620b7433e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:09.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.362+0000 7faeb5dd0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5620b7328330 con 0x5620b7078270 2024-09-21T07:48:09.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.362+0000 7faeb5dd0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5620b7385810 con 0x5620b7057590 2024-09-21T07:48:09.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.362+0000 7faeb5dd0740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x5620b70f2620 con 0x5620b704f3b0 2024-09-21T07:48:09.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.362+0000 7faeb5dd0740 10 client.0 _ll_register_callbacks cb 0x5620b732d420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:48:09.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.362+0000 7faeacc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620b7078270 0x5620b7433e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:09.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.362+0000 7faeacc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620b7078270 0x5620b7433e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56326/0 (socket says 172.21.15.39:56326) 2024-09-21T07:48:09.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.362+0000 7faeacc00640 1 -- 172.21.15.39:0/4092024872 learned_addr learned my addr 172.21.15.39:0/4092024872 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:09.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.362+0000 7faea7400640 1 --2- 172.21.15.39:0/4092024872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620b7057590 0x5620b74457e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:09.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.362+0000 7faea7e00640 1 --2- 172.21.15.39:0/4092024872 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5620b704f3b0 0x5620b7443d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:09.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.363+0000 7faeacc00640 1 -- 172.21.15.39:0/4092024872 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5620b704f3b0 msgr2=0x5620b7443d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:48:09.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.363+0000 7faeacc00640 1 --2- 172.21.15.39:0/4092024872 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5620b704f3b0 0x5620b7443d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.363+0000 7faeacc00640 1 -- 172.21.15.39:0/4092024872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620b7057590 msgr2=0x5620b74457e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:48:09.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.363+0000 7faeacc00640 1 --2- 172.21.15.39:0/4092024872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620b7057590 0x5620b74457e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.363+0000 7faeacc00640 1 -- 172.21.15.39:0/4092024872 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620b70e3850 con 0x5620b7078270 2024-09-21T07:48:09.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.363+0000 7faeacc00640 1 --2- 172.21.15.39:0/4092024872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620b7078270 0x5620b7433e20 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7faea0083720 tx=0x7faea0083370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67b4accb6ee032a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:09.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.363+0000 7fae90c00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:48:09.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.363+0000 7fae90c00640 1 -- 172.21.15.39:0/4092024872 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea0083e80 con 0x5620b7078270 2024-09-21T07:48:09.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.363+0000 7fae90c00640 1 -- 172.21.15.39:0/4092024872 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(206..206 src has 1..206) ==== 6960+0+0 (secure 0 0 0) 0x7faea008cfb0 con 0x5620b7078270 2024-09-21T07:48:09.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.363+0000 7faeb5dd0740 1 -- 172.21.15.39:0/4092024872 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae98003bc0 con 0x5620b7078270 2024-09-21T07:48:09.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.364+0000 7fae90c00640 1 -- 172.21.15.39:0/4092024872 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=218) ==== 24+0+0 (secure 0 0 0) 0x7faea0083530 con 0x5620b7078270 2024-09-21T07:48:09.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.364+0000 7faeb5dd0740 10 client.10026 fetch_fsmap learned FSMap version 218 2024-09-21T07:48:09.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.364+0000 7faeb5dd0740 1 -- 172.21.15.39:0/4092024872 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=218,monmap=2+}) -- 0x5620b715a6b0 con 0x5620b7078270 2024-09-21T07:48:09.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.364+0000 7fae90c00640 1 -- 172.21.15.39:0/4092024872 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae98003bc0 con 0x5620b7078270 2024-09-21T07:48:09.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.364+0000 7fae90c00640 1 -- 172.21.15.39:0/4092024872 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 218) ==== 3522+0+0 (secure 0 0 0) 0x7faea003a030 con 0x5620b7078270 2024-09-21T07:48:09.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.364+0000 7faeb5dd0740 10 client.10026 fetch_fsmap finished waiting for FSMap version 218 2024-09-21T07:48:09.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.364+0000 7faeb5dd0740 10 client.10026 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:48:09.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.364+0000 7faeb5dd0740 20 client.10026 populate_metadata read hostname 'smithi039' 2024-09-21T07:48:09.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.364+0000 7faeb5dd0740 1 --2- 172.21.15.39:0/4092024872 >> [v2:172.21.15.39:6832/1947479913,v1:172.21.15.39:6833/1947479913] conn(0x5620b7036e40 0x5620b74731f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:09.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.364+0000 7faeb5dd0740 4 client.10026 mds_command: new command op to 9049 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:48:09.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.364+0000 7faeb5dd0740 1 -- 172.21.15.39:0/4092024872 --> [v2:172.21.15.39:6832/1947479913,v1:172.21.15.39:6833/1947479913] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620b7394520 con 0x5620b7036e40 2024-09-21T07:48:09.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.364+0000 7faea7e00640 1 --2- 172.21.15.39:0/4092024872 >> [v2:172.21.15.39:6832/1947479913,v1:172.21.15.39:6833/1947479913] conn(0x5620b7036e40 0x5620b74731f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:09.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.365+0000 7fae90c00640 10 client.10026 ms_handle_connect on v2:172.21.15.39:6832/1947479913 2024-09-21T07:48:09.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.365+0000 7faea7e00640 1 --2- 172.21.15.39:0/4092024872 >> [v2:172.21.15.39:6832/1947479913,v1:172.21.15.39:6833/1947479913] conn(0x5620b7036e40 0x5620b74731f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6b087ee349648ca6 server_cookie=4e74e59c7f4b90d in_seq=0 out_seq=0 2024-09-21T07:48:09.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.366+0000 7fae90c00640 1 -- 172.21.15.39:0/4092024872 <== mds.1 v2:172.21.15.39:6832/1947479913 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5620b7394520 con 0x5620b7036e40 2024-09-21T07:48:09.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.366+0000 7fae90c00640 10 client.10026 handle_command_reply: tid=0 2024-09-21T07:48:09.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.366+0000 7faeb5dd0740 1 client.10026 shutdown 2024-09-21T07:48:09.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.366+0000 7faeb5dd0740 1 -- 172.21.15.39:0/4092024872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620b7078270 msgr2=0x5620b7433e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:09.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.366+0000 7faeb5dd0740 1 --2- 172.21.15.39:0/4092024872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620b7078270 0x5620b7433e20 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7faea0083720 tx=0x7faea0083370 comp rx=0 tx=0).stop 2024-09-21T07:48:09.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.366+0000 7faeb5dd0740 1 -- 172.21.15.39:0/4092024872 shutdown_connections 2024-09-21T07:48:09.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.366+0000 7faeb5dd0740 1 --2- 172.21.15.39:0/4092024872 >> [v2:172.21.15.39:6832/1947479913,v1:172.21.15.39:6833/1947479913] conn(0x5620b7036e40 0x5620b74731f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.366+0000 7faeb5dd0740 1 --2- 172.21.15.39:0/4092024872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620b7078270 0x5620b7433e20 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.366+0000 7faeb5dd0740 1 --2- 172.21.15.39:0/4092024872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620b7057590 0x5620b74457e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.366+0000 7faeb5dd0740 1 --2- 172.21.15.39:0/4092024872 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5620b704f3b0 0x5620b7443d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.366+0000 7fae90c00640 0 client.10026 ms_handle_reset on v2:172.21.15.39:6832/1947479913 2024-09-21T07:48:09.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.366+0000 7faeb5dd0740 1 -- 172.21.15.39:0/4092024872 >> 172.21.15.39:0/4092024872 conn(0x5620b706c3c0 msgr2=0x5620b74665b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:09.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.366+0000 7faeb5dd0740 1 -- 172.21.15.39:0/4092024872 shutdown_connections 2024-09-21T07:48:09.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.366+0000 7faeb5dd0740 1 -- 172.21.15.39:0/4092024872 wait complete. 2024-09-21T07:48:09.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.366+0000 7faeb5dd0740 20 client.10026 trim_cache size 0 max 16384 2024-09-21T07:48:09.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.386+0000 7faeb5dd0740 1 Processor -- start 2024-09-21T07:48:09.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.386+0000 7faeb5dd0740 1 -- start start 2024-09-21T07:48:09.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.387+0000 7faeb5dd0740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5620b704f3b0 0x5620b7444570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:09.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.387+0000 7faeb5dd0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620b7057590 0x5620b7486f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:09.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.387+0000 7faeb5dd0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620b7078270 0x5620b74874a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:09.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.387+0000 7faeb5dd0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7faea8204b80 con 0x5620b7057590 2024-09-21T07:48:09.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.387+0000 7faeb5dd0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7faea8202ec0 con 0x5620b7078270 2024-09-21T07:48:09.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.387+0000 7faeb5dd0740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7faea82034e0 con 0x5620b704f3b0 2024-09-21T07:48:09.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.387+0000 7faea7400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620b7057590 0x5620b7486f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:09.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.388+0000 7faea7400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620b7057590 0x5620b7486f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56338/0 (socket says 172.21.15.39:56338) 2024-09-21T07:48:09.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.388+0000 7faea7400640 1 -- 172.21.15.39:0/3277040850 learned_addr learned my addr 172.21.15.39:0/3277040850 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:09.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.388+0000 7faea7e00640 1 --2- 172.21.15.39:0/3277040850 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5620b704f3b0 0x5620b7444570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:09.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.388+0000 7faeacc00640 1 --2- 172.21.15.39:0/3277040850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620b7078270 0x5620b74874a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:09.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.388+0000 7faea7400640 1 -- 172.21.15.39:0/3277040850 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5620b704f3b0 msgr2=0x5620b7444570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:48:09.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.388+0000 7faea7400640 1 --2- 172.21.15.39:0/3277040850 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5620b704f3b0 0x5620b7444570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.388+0000 7faea7400640 1 -- 172.21.15.39:0/3277040850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620b7078270 msgr2=0x5620b74874a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:48:09.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.388+0000 7faea7400640 1 --2- 172.21.15.39:0/3277040850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620b7078270 0x5620b74874a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.388+0000 7faea7400640 1 -- 172.21.15.39:0/3277040850 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620b7414830 con 0x5620b7057590 2024-09-21T07:48:09.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.388+0000 7faea7400640 1 --2- 172.21.15.39:0/3277040850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620b7057590 0x5620b7486f60 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7faea80ca6b0 tx=0x7fae94013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b7111c9eacd047 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:09.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.389+0000 7fae93400640 1 -- 172.21.15.39:0/3277040850 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9401d3f0 con 0x5620b7057590 2024-09-21T07:48:09.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.389+0000 7fae93400640 1 -- 172.21.15.39:0/3277040850 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae9401d680 con 0x5620b7057590 2024-09-21T07:48:09.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.389+0000 7fae93400640 1 -- 172.21.15.39:0/3277040850 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9401dab0 con 0x5620b7057590 2024-09-21T07:48:09.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.389+0000 7faeb5dd0740 1 -- 172.21.15.39:0/3277040850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620b7057590 msgr2=0x5620b7486f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:09.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.389+0000 7faeb5dd0740 1 --2- 172.21.15.39:0/3277040850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620b7057590 0x5620b7486f60 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7faea80ca6b0 tx=0x7fae94013830 comp rx=0 tx=0).stop 2024-09-21T07:48:09.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.389+0000 7faeb5dd0740 1 -- 172.21.15.39:0/3277040850 shutdown_connections 2024-09-21T07:48:09.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.389+0000 7faeb5dd0740 1 --2- 172.21.15.39:0/3277040850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620b7078270 0x5620b74874a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.389+0000 7faeb5dd0740 1 --2- 172.21.15.39:0/3277040850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620b7057590 0x5620b7486f60 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.389+0000 7faeb5dd0740 1 --2- 172.21.15.39:0/3277040850 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5620b704f3b0 0x5620b7444570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.389+0000 7faeb5dd0740 1 -- 172.21.15.39:0/3277040850 >> 172.21.15.39:0/3277040850 conn(0x5620b706c3c0 msgr2=0x5620b7448ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:09.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.389+0000 7faeb5dd0740 1 -- 172.21.15.39:0/3277040850 shutdown_connections 2024-09-21T07:48:09.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.389+0000 7faeb5dd0740 1 -- 172.21.15.39:0/3277040850 wait complete. 2024-09-21T07:48:09.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.390+0000 7faeb5dd0740 1 -- start start 2024-09-21T07:48:09.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.391+0000 7faeb5dd0740 1 Processor -- start 2024-09-21T07:48:09.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.392+0000 7faeb5dd0740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5620b704f3b0 0x5620b7433390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:09.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.392+0000 7faeb5dd0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620b7057590 0x5620b7440a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:09.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.392+0000 7faeb5dd0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620b7078270 0x5620b745f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:09.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.392+0000 7faeb5dd0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5620b7477590 con 0x5620b7078270 2024-09-21T07:48:09.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.392+0000 7faeb5dd0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5620b7328330 con 0x5620b7057590 2024-09-21T07:48:09.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.392+0000 7faeb5dd0740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x5620b74422c0 con 0x5620b704f3b0 2024-09-21T07:48:09.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.392+0000 7faeb5dd0740 10 client.0 _ll_register_callbacks cb 0x7faea00838c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:48:09.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.392+0000 7faeacc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620b7078270 0x5620b745f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:09.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.392+0000 7faeacc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620b7078270 0x5620b745f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56350/0 (socket says 172.21.15.39:56350) 2024-09-21T07:48:09.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.392+0000 7faeacc00640 1 -- 172.21.15.39:0/2145072056 learned_addr learned my addr 172.21.15.39:0/2145072056 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:09.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.393+0000 7faea7e00640 1 --2- 172.21.15.39:0/2145072056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5620b704f3b0 0x5620b7433390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:09.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.393+0000 7faea7400640 1 --2- 172.21.15.39:0/2145072056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620b7057590 0x5620b7440a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:09.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.393+0000 7faeacc00640 1 -- 172.21.15.39:0/2145072056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5620b704f3b0 msgr2=0x5620b7433390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:48:09.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.393+0000 7faeacc00640 1 --2- 172.21.15.39:0/2145072056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5620b704f3b0 0x5620b7433390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.393+0000 7faeacc00640 1 -- 172.21.15.39:0/2145072056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620b7057590 msgr2=0x5620b7440a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:48:09.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.393+0000 7faeacc00640 1 --2- 172.21.15.39:0/2145072056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620b7057590 0x5620b7440a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.393+0000 7faeacc00640 1 -- 172.21.15.39:0/2145072056 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620b7419a20 con 0x5620b7078270 2024-09-21T07:48:09.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.393+0000 7faeacc00640 1 --2- 172.21.15.39:0/2145072056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620b7078270 0x5620b745f370 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7faea003fff0 tx=0x7faea008d1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396c98262b406cd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:09.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.393+0000 7fae90c00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:48:09.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.393+0000 7fae90c00640 1 -- 172.21.15.39:0/2145072056 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea0083190 con 0x5620b7078270 2024-09-21T07:48:09.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.393+0000 7fae90c00640 1 -- 172.21.15.39:0/2145072056 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(206..206 src has 1..206) ==== 6960+0+0 (secure 0 0 0) 0x7faea0033bb0 con 0x5620b7078270 2024-09-21T07:48:09.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.393+0000 7faeb5dd0740 1 -- 172.21.15.39:0/2145072056 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae9401d680 con 0x5620b7078270 2024-09-21T07:48:09.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.394+0000 7fae90c00640 1 -- 172.21.15.39:0/2145072056 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=218) ==== 24+0+0 (secure 0 0 0) 0x7faea0093050 con 0x5620b7078270 2024-09-21T07:48:09.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.394+0000 7faeb5dd0740 10 client.10032 fetch_fsmap learned FSMap version 218 2024-09-21T07:48:09.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.394+0000 7faeb5dd0740 1 -- 172.21.15.39:0/2145072056 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=218,monmap=2+}) -- 0x5620b7476500 con 0x5620b7078270 2024-09-21T07:48:09.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.394+0000 7fae90c00640 1 -- 172.21.15.39:0/2145072056 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae9401d680 con 0x5620b7078270 2024-09-21T07:48:09.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.394+0000 7fae90c00640 1 -- 172.21.15.39:0/2145072056 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 218) ==== 3522+0+0 (secure 0 0 0) 0x7faea0096020 con 0x5620b7078270 2024-09-21T07:48:09.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.394+0000 7faeb5dd0740 10 client.10032 fetch_fsmap finished waiting for FSMap version 218 2024-09-21T07:48:09.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.394+0000 7faeb5dd0740 10 client.10032 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:48:09.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.394+0000 7faeb5dd0740 20 client.10032 populate_metadata read hostname 'smithi039' 2024-09-21T07:48:09.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.394+0000 7faeb5dd0740 1 --2- 172.21.15.39:0/2145072056 >> [v2:172.21.15.39:6832/1947479913,v1:172.21.15.39:6833/1947479913] conn(0x5620b7487410 0x5620b7465500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:09.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.394+0000 7faeb5dd0740 4 client.10032 mds_command: new command op to 9049 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:48:09.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.394+0000 7faeb5dd0740 1 -- 172.21.15.39:0/2145072056 --> [v2:172.21.15.39:6832/1947479913,v1:172.21.15.39:6833/1947479913] -- command(tid 0: {"prefix": "status"}) -- 0x5620b738ffd0 con 0x5620b7487410 2024-09-21T07:48:09.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.394+0000 7faea7e00640 1 --2- 172.21.15.39:0/2145072056 >> [v2:172.21.15.39:6832/1947479913,v1:172.21.15.39:6833/1947479913] conn(0x5620b7487410 0x5620b7465500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:09.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.395+0000 7fae90c00640 10 client.10032 ms_handle_connect on v2:172.21.15.39:6832/1947479913 2024-09-21T07:48:09.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.395+0000 7faea7e00640 1 --2- 172.21.15.39:0/2145072056 >> [v2:172.21.15.39:6832/1947479913,v1:172.21.15.39:6833/1947479913] conn(0x5620b7487410 0x5620b7465500 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=85832961b6192f2d server_cookie=96c0665a08000a31 in_seq=0 out_seq=0 2024-09-21T07:48:09.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.395+0000 7fae90c00640 1 -- 172.21.15.39:0/2145072056 <== mds.1 v2:172.21.15.39:6832/1947479913 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x5620b738ffd0 con 0x5620b7487410 2024-09-21T07:48:09.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.395+0000 7fae90c00640 10 client.10032 handle_command_reply: tid=0 2024-09-21T07:48:09.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.395+0000 7faeb5dd0740 1 client.10032 shutdown 2024-09-21T07:48:09.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.395+0000 7faeb5dd0740 1 -- 172.21.15.39:0/2145072056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620b7078270 msgr2=0x5620b745f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:09.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.395+0000 7faeb5dd0740 1 --2- 172.21.15.39:0/2145072056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620b7078270 0x5620b745f370 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7faea003fff0 tx=0x7faea008d1e0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.395+0000 7faeb5dd0740 1 -- 172.21.15.39:0/2145072056 shutdown_connections 2024-09-21T07:48:09.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.395+0000 7faeb5dd0740 1 --2- 172.21.15.39:0/2145072056 >> [v2:172.21.15.39:6832/1947479913,v1:172.21.15.39:6833/1947479913] conn(0x5620b7487410 0x5620b7465500 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.395+0000 7faeb5dd0740 1 --2- 172.21.15.39:0/2145072056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5620b7078270 0x5620b745f370 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.395+0000 7faeb5dd0740 1 --2- 172.21.15.39:0/2145072056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5620b7057590 0x5620b7440a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.395+0000 7faeb5dd0740 1 --2- 172.21.15.39:0/2145072056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5620b704f3b0 0x5620b7433390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.395+0000 7fae90c00640 0 client.10032 ms_handle_reset on v2:172.21.15.39:6832/1947479913 2024-09-21T07:48:09.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.395+0000 7faeb5dd0740 1 -- 172.21.15.39:0/2145072056 >> 172.21.15.39:0/2145072056 conn(0x5620b706c3c0 msgr2=0x5620b741af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:09.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.396+0000 7faeb5dd0740 1 -- 172.21.15.39:0/2145072056 shutdown_connections 2024-09-21T07:48:09.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.396+0000 7faeb5dd0740 1 -- 172.21.15.39:0/2145072056 wait complete. 2024-09-21T07:48:09.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.396+0000 7faeb5dd0740 20 client.10032 trim_cache size 0 max 16384 2024-09-21T07:48:09.401 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:48:09.401 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:48:09.402 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 1, 2024-09-21T07:48:09.402 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 9049, 2024-09-21T07:48:09.402 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:48:09.402 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:48:09.402 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:48:09.402 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 3.1307553260000001, 2024-09-21T07:48:09.402 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 218, 2024-09-21T07:48:09.402 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 206, 2024-09-21T07:48:09.402 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 206, 2024-09-21T07:48:09.402 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 19.238354658999999 2024-09-21T07:48:09.402 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:48:09.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.396+0000 7faeaec00640 1 -- 172.21.15.39:0/2474660531 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae88078bc0 msgr2=0x7fae8807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:09.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.396+0000 7faeaec00640 1 --2- 172.21.15.39:0/2474660531 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae88078bc0 0x7fae8807b0d0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fae98000f60 tx=0x7fae98009040 comp rx=0 tx=0).stop 2024-09-21T07:48:09.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.396+0000 7faeaec00640 1 -- 172.21.15.39:0/2474660531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faea80ccc60 msgr2=0x7faea815bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:09.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.396+0000 7faeaec00640 1 --2- 172.21.15.39:0/2474660531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faea80ccc60 0x7faea815bce0 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7faea0009b30 tx=0x7faea0032c90 comp rx=0 tx=0).stop 2024-09-21T07:48:09.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.396+0000 7faeaec00640 1 -- 172.21.15.39:0/2474660531 shutdown_connections 2024-09-21T07:48:09.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.396+0000 7faeaec00640 1 --2- 172.21.15.39:0/2474660531 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae88078bc0 0x7fae8807b0d0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.396+0000 7faeaec00640 1 --2- 172.21.15.39:0/2474660531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faea80ccc60 0x7faea815bce0 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.396+0000 7faeaec00640 1 --2- 172.21.15.39:0/2474660531 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faea80cb8a0 0x7faea815b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.396+0000 7faeaec00640 1 --2- 172.21.15.39:0/2474660531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faea80b30a0 0x7faea81f8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.396+0000 7faeaec00640 1 -- 172.21.15.39:0/2474660531 >> 172.21.15.39:0/2474660531 conn(0x7faea80b2460 msgr2=0x7faea80b3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:09.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.397+0000 7faeaec00640 1 -- 172.21.15.39:0/2474660531 shutdown_connections 2024-09-21T07:48:09.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.397+0000 7faeaec00640 1 -- 172.21.15.39:0/2474660531 wait complete. 2024-09-21T07:48:09.424 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T07:48:09.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.650+0000 7f4b0f000640 1 -- 172.21.15.39:0/1317845928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b080d2570 msgr2=0x7f4b080cce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:09.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.650+0000 7f4b0f000640 1 --2- 172.21.15.39:0/1317845928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b080d2570 0x7f4b080cce90 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f4b080b2c60 tx=0x7f4b0002aaa0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.650+0000 7f4b0f000640 1 -- 172.21.15.39:0/1317845928 shutdown_connections 2024-09-21T07:48:09.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.650+0000 7f4b0f000640 1 --2- 172.21.15.39:0/1317845928 >> v2:172.21.15.148:3300/0 conn(0x7f4b080b2f70 0x7f4b080d2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.650+0000 7f4b0f000640 1 --2- 172.21.15.39:0/1317845928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b080d2570 0x7f4b080cce90 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.650+0000 7f4b0f000640 1 -- 172.21.15.39:0/1317845928 >> 172.21.15.39:0/1317845928 conn(0x7f4b0809b5f0 msgr2=0x7f4b0808c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:09.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.650+0000 7f4b0f000640 1 -- 172.21.15.39:0/1317845928 shutdown_connections 2024-09-21T07:48:09.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.650+0000 7f4b0f000640 1 -- 172.21.15.39:0/1317845928 wait complete. 2024-09-21T07:48:09.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.652+0000 7f4b0f000640 1 Processor -- start 2024-09-21T07:48:09.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.652+0000 7f4b0f000640 1 -- start start 2024-09-21T07:48:09.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.654+0000 7f4b0f000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b080b2f70 0x7f4b0814efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:09.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.654+0000 7f4b0f000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b080ce690 0x7f4b080c7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:09.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.654+0000 7f4b0f000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b080d2570 0x7f4b081517d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:09.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.654+0000 7f4b0f000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4b080cd550 con 0x7f4b080b2f70 2024-09-21T07:48:09.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.654+0000 7f4b0f000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4b080cd3d0 con 0x7f4b080d2570 2024-09-21T07:48:09.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.654+0000 7f4b0f000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f4b080cd6d0 con 0x7f4b080ce690 2024-09-21T07:48:09.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.654+0000 7f4b0dc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b080b2f70 0x7f4b0814efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:09.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.654+0000 7f4b0dc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b080b2f70 0x7f4b0814efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56368/0 (socket says 172.21.15.39:56368) 2024-09-21T07:48:09.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.654+0000 7f4b0dc00640 1 -- 172.21.15.39:0/3786092270 learned_addr learned my addr 172.21.15.39:0/3786092270 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:09.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.654+0000 7f4b0e600640 1 --2- 172.21.15.39:0/3786092270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b080d2570 0x7f4b081517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:09.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.654+0000 7f4b0d200640 1 --2- 172.21.15.39:0/3786092270 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b080ce690 0x7f4b080c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:09.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.655+0000 7f4b0dc00640 1 -- 172.21.15.39:0/3786092270 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b080ce690 msgr2=0x7f4b080c7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:09.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.655+0000 7f4b0dc00640 1 --2- 172.21.15.39:0/3786092270 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b080ce690 0x7f4b080c7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.655+0000 7f4b0dc00640 1 -- 172.21.15.39:0/3786092270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b080d2570 msgr2=0x7f4b081517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:09.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.655+0000 7f4b0dc00640 1 --2- 172.21.15.39:0/3786092270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b080d2570 0x7f4b081517d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.655+0000 7f4b0dc00640 1 -- 172.21.15.39:0/3786092270 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b080c7d50 con 0x7f4b080b2f70 2024-09-21T07:48:09.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.655+0000 7f4b0dc00640 1 --2- 172.21.15.39:0/3786092270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b080b2f70 0x7f4b0814efe0 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f4afc00c390 tx=0x7f4afc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:09.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.655+0000 7f4b05600640 1 -- 172.21.15.39:0/3786092270 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4afc017070 con 0x7f4b080b2f70 2024-09-21T07:48:09.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.655+0000 7f4b05600640 1 -- 172.21.15.39:0/3786092270 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4afc0053c0 con 0x7f4b080b2f70 2024-09-21T07:48:09.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.655+0000 7f4b05600640 1 -- 172.21.15.39:0/3786092270 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4afc003880 con 0x7f4b080b2f70 2024-09-21T07:48:09.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.655+0000 7f4b0f000640 1 -- 172.21.15.39:0/3786092270 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b080c8260 con 0x7f4b080b2f70 2024-09-21T07:48:09.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.655+0000 7f4b0f000640 1 -- 172.21.15.39:0/3786092270 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b080c8820 con 0x7f4b080b2f70 2024-09-21T07:48:09.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.659+0000 7f4b05600640 1 -- 172.21.15.39:0/3786092270 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4afc01a070 con 0x7f4b080b2f70 2024-09-21T07:48:09.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.660+0000 7f4b05600640 1 --2- 172.21.15.39:0/3786092270 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4ae8078b90 0x7f4ae807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:09.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.660+0000 7f4b175c5740 1 Processor -- start 2024-09-21T07:48:09.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.660+0000 7f4b05600640 1 -- 172.21.15.39:0/3786092270 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(206..206 src has 1..206) ==== 6960+0+0 (secure 0 0 0) 0x7f4afc09a560 con 0x7f4b080b2f70 2024-09-21T07:48:09.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.660+0000 7f4b175c5740 1 -- start start 2024-09-21T07:48:09.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.661+0000 7f4b0d200640 1 --2- 172.21.15.39:0/3786092270 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4ae8078b90 0x7f4ae807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:09.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.662+0000 7f4b0d200640 1 --2- 172.21.15.39:0/3786092270 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4ae8078b90 0x7f4ae807b0a0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f4b080c8d80 tx=0x7f4af4005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:09.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.662+0000 7f4b175c5740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c99d6a7270 0x55c99da77b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:09.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.662+0000 7f4b175c5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c99d67e3b0 0x55c99da780a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:09.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.662+0000 7f4b175c5740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c99d686590 0x55c99da48150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:09.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.662+0000 7f4b175c5740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c99d9d1430 con 0x55c99d686590 2024-09-21T07:48:09.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.662+0000 7f4b175c5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c99d9c15d0 con 0x55c99d67e3b0 2024-09-21T07:48:09.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.662+0000 7f4b175c5740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55c99d9b0a00 con 0x55c99d6a7270 2024-09-21T07:48:09.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.662+0000 7f4b0e600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c99d686590 0x55c99da48150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:09.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.662+0000 7f4b0e600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c99d686590 0x55c99da48150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56384/0 (socket says 172.21.15.39:56384) 2024-09-21T07:48:09.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.662+0000 7f4b0e600640 1 -- 172.21.15.39:0/2868430607 learned_addr learned my addr 172.21.15.39:0/2868430607 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:09.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.662+0000 7f4b0dc00640 1 --2- 172.21.15.39:0/2868430607 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c99d6a7270 0x55c99da77b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:09.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.662+0000 7f4b0d200640 1 --2- 172.21.15.39:0/2868430607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c99d67e3b0 0x55c99da780a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:09.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.663+0000 7f4b0e600640 1 -- 172.21.15.39:0/2868430607 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c99d6a7270 msgr2=0x55c99da77b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:48:09.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.663+0000 7f4b0e600640 1 --2- 172.21.15.39:0/2868430607 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c99d6a7270 0x55c99da77b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.663+0000 7f4b0e600640 1 -- 172.21.15.39:0/2868430607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c99d67e3b0 msgr2=0x55c99da780a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:48:09.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.663+0000 7f4b0e600640 1 --2- 172.21.15.39:0/2868430607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c99d67e3b0 0x55c99da780a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.663+0000 7f4b0e600640 1 -- 172.21.15.39:0/2868430607 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c99d979520 con 0x55c99d686590 2024-09-21T07:48:09.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.663+0000 7f4b0d200640 1 --2- 172.21.15.39:0/2868430607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c99d67e3b0 0x55c99da780a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:48:09.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.663+0000 7f4b0e600640 1 --2- 172.21.15.39:0/2868430607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c99d686590 0x55c99da48150 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f4b0002b090 tx=0x7f4b00004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67294ba773a846fb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:09.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.663+0000 7f4afa000640 1 -- 172.21.15.39:0/2868430607 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b00045070 con 0x55c99d686590 2024-09-21T07:48:09.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.664+0000 7f4afa000640 1 -- 172.21.15.39:0/2868430607 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b0002ecf0 con 0x55c99d686590 2024-09-21T07:48:09.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.664+0000 7f4afa000640 1 -- 172.21.15.39:0/2868430607 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b00003bd0 con 0x55c99d686590 2024-09-21T07:48:09.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.664+0000 7f4b175c5740 1 -- 172.21.15.39:0/2868430607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c99d686590 msgr2=0x55c99da48150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:09.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.664+0000 7f4b175c5740 1 --2- 172.21.15.39:0/2868430607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c99d686590 0x55c99da48150 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f4b0002b090 tx=0x7f4b00004ab0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.664+0000 7f4b175c5740 1 -- 172.21.15.39:0/2868430607 shutdown_connections 2024-09-21T07:48:09.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.664+0000 7f4b175c5740 1 --2- 172.21.15.39:0/2868430607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c99d686590 0x55c99da48150 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.664+0000 7f4b175c5740 1 --2- 172.21.15.39:0/2868430607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c99d67e3b0 0x55c99da780a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.664+0000 7f4b175c5740 1 --2- 172.21.15.39:0/2868430607 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c99d6a7270 0x55c99da77b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.664+0000 7f4b175c5740 1 -- 172.21.15.39:0/2868430607 >> 172.21.15.39:0/2868430607 conn(0x55c99d69b3c0 msgr2=0x55c99da4db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:09.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.664+0000 7f4b175c5740 1 -- 172.21.15.39:0/2868430607 shutdown_connections 2024-09-21T07:48:09.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.664+0000 7f4b175c5740 1 -- 172.21.15.39:0/2868430607 wait complete. 2024-09-21T07:48:09.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.665+0000 7f4b175c5740 1 -- start start 2024-09-21T07:48:09.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.666+0000 7f4b175c5740 1 Processor -- start 2024-09-21T07:48:09.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.667+0000 7f4b175c5740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c99d67e3b0 0x55c99da712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:09.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.667+0000 7f4b175c5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c99d686590 0x55c99da8c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:09.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.667+0000 7f4b175c5740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c99d6a7270 0x55c99da62460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:09.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.667+0000 7f4b175c5740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c99d957330 con 0x55c99d67e3b0 2024-09-21T07:48:09.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.667+0000 7f4b175c5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c99d9b4810 con 0x55c99d686590 2024-09-21T07:48:09.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.667+0000 7f4b175c5740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55c99d721620 con 0x55c99d6a7270 2024-09-21T07:48:09.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.667+0000 7f4b175c5740 10 client.0 _ll_register_callbacks cb 0x55c99d95c420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:48:09.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.667+0000 7f4b0dc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c99d67e3b0 0x55c99da712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:09.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.667+0000 7f4b0dc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c99d67e3b0 0x55c99da712e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56398/0 (socket says 172.21.15.39:56398) 2024-09-21T07:48:09.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.668+0000 7f4b0dc00640 1 -- 172.21.15.39:0/3092970256 learned_addr learned my addr 172.21.15.39:0/3092970256 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:09.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.668+0000 7f4b0d200640 1 --2- 172.21.15.39:0/3092970256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c99d686590 0x55c99da8c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:09.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.668+0000 7f4b0e600640 1 --2- 172.21.15.39:0/3092970256 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c99d6a7270 0x55c99da62460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:09.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.668+0000 7f4b0dc00640 1 -- 172.21.15.39:0/3092970256 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c99d6a7270 msgr2=0x55c99da62460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:48:09.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.668+0000 7f4b0dc00640 1 --2- 172.21.15.39:0/3092970256 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c99d6a7270 0x55c99da62460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.668+0000 7f4b0dc00640 1 -- 172.21.15.39:0/3092970256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c99d686590 msgr2=0x55c99da8c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:48:09.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.668+0000 7f4b0dc00640 1 --2- 172.21.15.39:0/3092970256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c99d686590 0x55c99da8c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.668+0000 7f4b0dc00640 1 -- 172.21.15.39:0/3092970256 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c99d712850 con 0x55c99d67e3b0 2024-09-21T07:48:09.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.668+0000 7f4b0dc00640 1 --2- 172.21.15.39:0/3092970256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c99d67e3b0 0x55c99da712e0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f4afc00cd50 tx=0x7f4afc05dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece2ff54a249d10c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:09.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.668+0000 7f4aeea00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:48:09.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.668+0000 7f4aeea00640 1 -- 172.21.15.39:0/3092970256 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4afc065040 con 0x55c99d67e3b0 2024-09-21T07:48:09.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.668+0000 7f4aeea00640 1 -- 172.21.15.39:0/3092970256 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(206..206 src has 1..206) ==== 6960+0+0 (secure 0 0 0) 0x7f4afc09a310 con 0x55c99d67e3b0 2024-09-21T07:48:09.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.668+0000 7f4b175c5740 1 -- 172.21.15.39:0/3092970256 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b0002ecf0 con 0x55c99d67e3b0 2024-09-21T07:48:09.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.669+0000 7f4aeea00640 1 -- 172.21.15.39:0/3092970256 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=218) ==== 24+0+0 (secure 0 0 0) 0x7f4afc088180 con 0x55c99d67e3b0 2024-09-21T07:48:09.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.669+0000 7f4b175c5740 10 client.10044 fetch_fsmap learned FSMap version 218 2024-09-21T07:48:09.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.669+0000 7f4b175c5740 1 -- 172.21.15.39:0/3092970256 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=218,monmap=2+}) -- 0x55c99d7896b0 con 0x55c99d67e3b0 2024-09-21T07:48:09.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.669+0000 7f4aeea00640 1 -- 172.21.15.39:0/3092970256 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b0002ecf0 con 0x55c99d67e3b0 2024-09-21T07:48:09.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.669+0000 7f4aeea00640 1 -- 172.21.15.39:0/3092970256 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 218) ==== 3522+0+0 (secure 0 0 0) 0x7f4afc07e070 con 0x55c99d67e3b0 2024-09-21T07:48:09.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.669+0000 7f4b175c5740 10 client.10044 fetch_fsmap finished waiting for FSMap version 218 2024-09-21T07:48:09.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.669+0000 7f4b175c5740 10 client.10044 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:48:09.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.669+0000 7f4b175c5740 20 client.10044 populate_metadata read hostname 'smithi039' 2024-09-21T07:48:09.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.669+0000 7f4b175c5740 1 --2- 172.21.15.39:0/3092970256 >> [v2:172.21.15.39:6834/2356847598,v1:172.21.15.39:6835/2356847598] conn(0x55c99d665e40 0x55c99da636c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:09.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.669+0000 7f4b175c5740 4 client.10044 mds_command: new command op to 9852 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:48:09.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.669+0000 7f4b175c5740 1 -- 172.21.15.39:0/3092970256 --> [v2:172.21.15.39:6834/2356847598,v1:172.21.15.39:6835/2356847598] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c99d9c3520 con 0x55c99d665e40 2024-09-21T07:48:09.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.669+0000 7f4b0d200640 1 --2- 172.21.15.39:0/3092970256 >> [v2:172.21.15.39:6834/2356847598,v1:172.21.15.39:6835/2356847598] conn(0x55c99d665e40 0x55c99da636c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:09.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.670+0000 7f4aeea00640 10 client.10044 ms_handle_connect on v2:172.21.15.39:6834/2356847598 2024-09-21T07:48:09.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.670+0000 7f4b0d200640 1 --2- 172.21.15.39:0/3092970256 >> [v2:172.21.15.39:6834/2356847598,v1:172.21.15.39:6835/2356847598] conn(0x55c99d665e40 0x55c99da636c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3be772eac0ea8a0 server_cookie=e2dfe90a28fd35be in_seq=0 out_seq=0 2024-09-21T07:48:09.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.671+0000 7f4aeea00640 1 -- 172.21.15.39:0/3092970256 <== mds.0 v2:172.21.15.39:6834/2356847598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c99d9c3520 con 0x55c99d665e40 2024-09-21T07:48:09.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.671+0000 7f4aeea00640 10 client.10044 handle_command_reply: tid=0 2024-09-21T07:48:09.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.671+0000 7f4b175c5740 1 client.10044 shutdown 2024-09-21T07:48:09.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.671+0000 7f4b175c5740 1 -- 172.21.15.39:0/3092970256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c99d67e3b0 msgr2=0x55c99da712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:09.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.671+0000 7f4b175c5740 1 --2- 172.21.15.39:0/3092970256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c99d67e3b0 0x55c99da712e0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f4afc00cd50 tx=0x7f4afc05dd20 comp rx=0 tx=0).stop 2024-09-21T07:48:09.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.671+0000 7f4b175c5740 1 -- 172.21.15.39:0/3092970256 shutdown_connections 2024-09-21T07:48:09.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.671+0000 7f4b175c5740 1 --2- 172.21.15.39:0/3092970256 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c99d6a7270 0x55c99da62460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.671+0000 7f4b175c5740 1 --2- 172.21.15.39:0/3092970256 >> [v2:172.21.15.39:6834/2356847598,v1:172.21.15.39:6835/2356847598] conn(0x55c99d665e40 0x55c99da636c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.671+0000 7f4b175c5740 1 --2- 172.21.15.39:0/3092970256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c99d686590 0x55c99da8c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.671+0000 7f4b175c5740 1 --2- 172.21.15.39:0/3092970256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c99d67e3b0 0x55c99da712e0 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.671+0000 7f4b175c5740 1 -- 172.21.15.39:0/3092970256 >> 172.21.15.39:0/3092970256 conn(0x55c99d69b3c0 msgr2=0x55c99da46e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:09.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.671+0000 7f4aeea00640 0 client.10044 ms_handle_reset on v2:172.21.15.39:6834/2356847598 2024-09-21T07:48:09.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.671+0000 7f4b175c5740 1 -- 172.21.15.39:0/3092970256 shutdown_connections 2024-09-21T07:48:09.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.671+0000 7f4b175c5740 1 -- 172.21.15.39:0/3092970256 wait complete. 2024-09-21T07:48:09.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.671+0000 7f4b175c5740 20 client.10044 trim_cache size 0 max 16384 2024-09-21T07:48:09.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.691+0000 7f4b175c5740 1 Processor -- start 2024-09-21T07:48:09.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.691+0000 7f4b175c5740 1 -- start start 2024-09-21T07:48:09.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.692+0000 7f4b175c5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c99d67e3b0 0x55c99da6d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:09.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.692+0000 7f4b175c5740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c99d686590 0x55c99da77100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:09.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.692+0000 7f4b175c5740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c99d6a7270 0x55c99da77640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:09.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.692+0000 7f4b175c5740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4b08200580 con 0x55c99d686590 2024-09-21T07:48:09.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.692+0000 7f4b175c5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4b08151350 con 0x55c99d67e3b0 2024-09-21T07:48:09.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.692+0000 7f4b175c5740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f4b0806fef0 con 0x55c99d6a7270 2024-09-21T07:48:09.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.693+0000 7f4b0d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c99d686590 0x55c99da77100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:09.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.693+0000 7f4b0d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c99d686590 0x55c99da77100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56404/0 (socket says 172.21.15.39:56404) 2024-09-21T07:48:09.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.693+0000 7f4b0d200640 1 -- 172.21.15.39:0/477722861 learned_addr learned my addr 172.21.15.39:0/477722861 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:09.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.693+0000 7f4b0dc00640 1 --2- 172.21.15.39:0/477722861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c99d67e3b0 0x55c99da6d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:09.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.693+0000 7f4b0e600640 1 --2- 172.21.15.39:0/477722861 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c99d6a7270 0x55c99da77640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:09.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.693+0000 7f4b0d200640 1 -- 172.21.15.39:0/477722861 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c99d6a7270 msgr2=0x55c99da77640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:48:09.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.693+0000 7f4b0d200640 1 --2- 172.21.15.39:0/477722861 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c99d6a7270 0x55c99da77640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.693+0000 7f4b0d200640 1 -- 172.21.15.39:0/477722861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c99d67e3b0 msgr2=0x55c99da6d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:48:09.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.693+0000 7f4b0d200640 1 --2- 172.21.15.39:0/477722861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c99d67e3b0 0x55c99da6d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.693+0000 7f4b0d200640 1 -- 172.21.15.39:0/477722861 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c99da4e840 con 0x55c99d686590 2024-09-21T07:48:09.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.693+0000 7f4b0dc00640 1 --2- 172.21.15.39:0/477722861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c99d67e3b0 0x55c99da6d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:48:09.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.693+0000 7f4b0d200640 1 --2- 172.21.15.39:0/477722861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c99d686590 0x55c99da77100 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f4b0815dd50 tx=0x7f4af4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d2a52215643a439 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:09.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.694+0000 7f4af9600640 1 -- 172.21.15.39:0/477722861 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af4003770 con 0x55c99d686590 2024-09-21T07:48:09.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.694+0000 7f4af9600640 1 -- 172.21.15.39:0/477722861 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4af4003910 con 0x55c99d686590 2024-09-21T07:48:09.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.694+0000 7f4af9600640 1 -- 172.21.15.39:0/477722861 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af4003d20 con 0x55c99d686590 2024-09-21T07:48:09.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.694+0000 7f4b175c5740 1 -- 172.21.15.39:0/477722861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c99d686590 msgr2=0x55c99da77100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:09.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.694+0000 7f4b175c5740 1 --2- 172.21.15.39:0/477722861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c99d686590 0x55c99da77100 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f4b0815dd50 tx=0x7f4af4014000 comp rx=0 tx=0).stop 2024-09-21T07:48:09.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.694+0000 7f4b175c5740 1 -- 172.21.15.39:0/477722861 shutdown_connections 2024-09-21T07:48:09.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.694+0000 7f4b175c5740 1 --2- 172.21.15.39:0/477722861 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c99d6a7270 0x55c99da77640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.694+0000 7f4b175c5740 1 --2- 172.21.15.39:0/477722861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c99d686590 0x55c99da77100 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.694+0000 7f4b175c5740 1 --2- 172.21.15.39:0/477722861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c99d67e3b0 0x55c99da6d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.694+0000 7f4b175c5740 1 -- 172.21.15.39:0/477722861 >> 172.21.15.39:0/477722861 conn(0x55c99d69b3c0 msgr2=0x55c99da4b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:09.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.695+0000 7f4b175c5740 1 -- 172.21.15.39:0/477722861 shutdown_connections 2024-09-21T07:48:09.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.695+0000 7f4b175c5740 1 -- 172.21.15.39:0/477722861 wait complete. 2024-09-21T07:48:09.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.695+0000 7f4b175c5740 1 -- start start 2024-09-21T07:48:09.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.696+0000 7f4b175c5740 1 Processor -- start 2024-09-21T07:48:09.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.697+0000 7f4b175c5740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c99d67e3b0 0x55c99daff9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:09.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.697+0000 7f4b175c5740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c99d686590 0x55c99daa94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:09.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.697+0000 7f4b175c5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c99d6a7270 0x55c99daa9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:09.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.697+0000 7f4b175c5740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c99da91480 con 0x55c99d686590 2024-09-21T07:48:09.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.697+0000 7f4b175c5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c99da62e70 con 0x55c99d6a7270 2024-09-21T07:48:09.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.697+0000 7f4b175c5740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55c99da91aa0 con 0x55c99d67e3b0 2024-09-21T07:48:09.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.697+0000 7f4b175c5740 10 client.0 _ll_register_callbacks cb 0x7f4afc0618a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:48:09.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.697+0000 7f4b0d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c99d686590 0x55c99daa94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:09.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.697+0000 7f4b0d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c99d686590 0x55c99daa94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56420/0 (socket says 172.21.15.39:56420) 2024-09-21T07:48:09.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.697+0000 7f4b0d200640 1 -- 172.21.15.39:0/3709485015 learned_addr learned my addr 172.21.15.39:0/3709485015 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:09.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.698+0000 7f4b0dc00640 1 --2- 172.21.15.39:0/3709485015 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c99d67e3b0 0x55c99daff9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:09.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.698+0000 7f4b0e600640 1 --2- 172.21.15.39:0/3709485015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c99d6a7270 0x55c99daa9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:09.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.698+0000 7f4b0d200640 1 -- 172.21.15.39:0/3709485015 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c99d67e3b0 msgr2=0x55c99daff9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:48:09.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.698+0000 7f4b0d200640 1 --2- 172.21.15.39:0/3709485015 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c99d67e3b0 0x55c99daff9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.698+0000 7f4b0d200640 1 -- 172.21.15.39:0/3709485015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c99d6a7270 msgr2=0x55c99daa9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:48:09.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.698+0000 7f4b0d200640 1 --2- 172.21.15.39:0/3709485015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c99d6a7270 0x55c99daa9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.698+0000 7f4b0d200640 1 -- 172.21.15.39:0/3709485015 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c99dab8fd0 con 0x55c99d686590 2024-09-21T07:48:09.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.698+0000 7f4b0d200640 1 --2- 172.21.15.39:0/3709485015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c99d686590 0x55c99daa94d0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f4af4009240 tx=0x7f4af4003ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5335147036ced5f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:09.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.698+0000 7f4aeea00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:48:09.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.698+0000 7f4aeea00640 1 -- 172.21.15.39:0/3709485015 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af4035020 con 0x55c99d686590 2024-09-21T07:48:09.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.699+0000 7f4aeea00640 1 -- 172.21.15.39:0/3709485015 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(206..206 src has 1..206) ==== 6960+0+0 (secure 0 0 0) 0x7f4af4031470 con 0x55c99d686590 2024-09-21T07:48:09.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.699+0000 7f4b175c5740 1 -- 172.21.15.39:0/3709485015 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4af4003910 con 0x55c99d686590 2024-09-21T07:48:09.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.699+0000 7f4aeea00640 1 -- 172.21.15.39:0/3709485015 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=218) ==== 24+0+0 (secure 0 0 0) 0x7f4af4031670 con 0x55c99d686590 2024-09-21T07:48:09.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.699+0000 7f4b175c5740 10 client.10050 fetch_fsmap learned FSMap version 218 2024-09-21T07:48:09.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.699+0000 7f4b175c5740 1 -- 172.21.15.39:0/3709485015 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=218,monmap=2+}) -- 0x55c99da8c210 con 0x55c99d686590 2024-09-21T07:48:09.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.699+0000 7f4aeea00640 1 -- 172.21.15.39:0/3709485015 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4af4003910 con 0x55c99d686590 2024-09-21T07:48:09.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.699+0000 7f4aeea00640 1 -- 172.21.15.39:0/3709485015 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 218) ==== 3522+0+0 (secure 0 0 0) 0x7f4af4042080 con 0x55c99d686590 2024-09-21T07:48:09.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.699+0000 7f4b175c5740 10 client.10050 fetch_fsmap finished waiting for FSMap version 218 2024-09-21T07:48:09.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.699+0000 7f4b175c5740 10 client.10050 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:48:09.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.699+0000 7f4b175c5740 20 client.10050 populate_metadata read hostname 'smithi039' 2024-09-21T07:48:09.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.699+0000 7f4b175c5740 1 --2- 172.21.15.39:0/3709485015 >> [v2:172.21.15.39:6834/2356847598,v1:172.21.15.39:6835/2356847598] conn(0x55c99db1a500 0x55c99da8b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:09.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.699+0000 7f4b175c5740 4 client.10050 mds_command: new command op to 9852 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:48:09.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.699+0000 7f4b175c5740 1 -- 172.21.15.39:0/3709485015 --> [v2:172.21.15.39:6834/2356847598,v1:172.21.15.39:6835/2356847598] -- command(tid 0: {"prefix": "status"}) -- 0x55c99da4d1a0 con 0x55c99db1a500 2024-09-21T07:48:09.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.700+0000 7f4b0dc00640 1 --2- 172.21.15.39:0/3709485015 >> [v2:172.21.15.39:6834/2356847598,v1:172.21.15.39:6835/2356847598] conn(0x55c99db1a500 0x55c99da8b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:09.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.700+0000 7f4b0dc00640 1 --2- 172.21.15.39:0/3709485015 >> [v2:172.21.15.39:6834/2356847598,v1:172.21.15.39:6835/2356847598] conn(0x55c99db1a500 0x55c99da8b1d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66c479b1bf12b377 server_cookie=f82501222a4acc25 in_seq=0 out_seq=0 2024-09-21T07:48:09.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.700+0000 7f4aeea00640 10 client.10050 ms_handle_connect on v2:172.21.15.39:6834/2356847598 2024-09-21T07:48:09.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.700+0000 7f4aeea00640 1 -- 172.21.15.39:0/3709485015 <== mds.0 v2:172.21.15.39:6834/2356847598 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55c99da4d1a0 con 0x55c99db1a500 2024-09-21T07:48:09.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.700+0000 7f4aeea00640 10 client.10050 handle_command_reply: tid=0 2024-09-21T07:48:09.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.700+0000 7f4b175c5740 1 client.10050 shutdown 2024-09-21T07:48:09.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.701+0000 7f4b175c5740 1 -- 172.21.15.39:0/3709485015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c99d686590 msgr2=0x55c99daa94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:09.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.701+0000 7f4b175c5740 1 --2- 172.21.15.39:0/3709485015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c99d686590 0x55c99daa94d0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f4af4009240 tx=0x7f4af4003ac0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.701+0000 7f4b175c5740 1 -- 172.21.15.39:0/3709485015 shutdown_connections 2024-09-21T07:48:09.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.701+0000 7f4b175c5740 1 --2- 172.21.15.39:0/3709485015 >> [v2:172.21.15.39:6834/2356847598,v1:172.21.15.39:6835/2356847598] conn(0x55c99db1a500 0x55c99da8b1d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.701+0000 7f4b175c5740 1 --2- 172.21.15.39:0/3709485015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c99d6a7270 0x55c99daa9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.701+0000 7f4b175c5740 1 --2- 172.21.15.39:0/3709485015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c99d686590 0x55c99daa94d0 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.701+0000 7f4aeea00640 0 client.10050 ms_handle_reset on v2:172.21.15.39:6834/2356847598 2024-09-21T07:48:09.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.701+0000 7f4b175c5740 1 --2- 172.21.15.39:0/3709485015 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c99d67e3b0 0x55c99daff9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.701+0000 7f4b175c5740 1 -- 172.21.15.39:0/3709485015 >> 172.21.15.39:0/3709485015 conn(0x55c99d69b3c0 msgr2=0x55c99da62220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:09.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.701+0000 7f4b175c5740 1 -- 172.21.15.39:0/3709485015 shutdown_connections 2024-09-21T07:48:09.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.701+0000 7f4b175c5740 1 -- 172.21.15.39:0/3709485015 wait complete. 2024-09-21T07:48:09.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.701+0000 7f4b175c5740 20 client.10050 trim_cache size 0 max 16384 2024-09-21T07:48:09.706 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:48:09.706 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:48:09.706 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-09-21T07:48:09.706 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 9852, 2024-09-21T07:48:09.706 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:48:09.706 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:48:09.706 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:48:09.706 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 14.539360984, 2024-09-21T07:48:09.706 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 218, 2024-09-21T07:48:09.706 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 206, 2024-09-21T07:48:09.706 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 206, 2024-09-21T07:48:09.706 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 19.537808407 2024-09-21T07:48:09.707 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:48:09.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.702+0000 7f4b0f000640 1 -- 172.21.15.39:0/3786092270 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4ae8078b90 msgr2=0x7f4ae807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:09.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.702+0000 7f4b0f000640 1 --2- 172.21.15.39:0/3786092270 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4ae8078b90 0x7f4ae807b0a0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f4b080c8d80 tx=0x7f4af4005f90 comp rx=0 tx=0).stop 2024-09-21T07:48:09.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.702+0000 7f4b0f000640 1 -- 172.21.15.39:0/3786092270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b080b2f70 msgr2=0x7f4b0814efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:09.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.702+0000 7f4b0f000640 1 --2- 172.21.15.39:0/3786092270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b080b2f70 0x7f4b0814efe0 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f4afc00c390 tx=0x7f4afc00c850 comp rx=0 tx=0).stop 2024-09-21T07:48:09.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.702+0000 7f4b0f000640 1 -- 172.21.15.39:0/3786092270 shutdown_connections 2024-09-21T07:48:09.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.702+0000 7f4b0f000640 1 --2- 172.21.15.39:0/3786092270 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4ae8078b90 0x7f4ae807b0a0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.702+0000 7f4b0f000640 1 --2- 172.21.15.39:0/3786092270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b080d2570 0x7f4b081517d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.702+0000 7f4b0f000640 1 --2- 172.21.15.39:0/3786092270 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b080ce690 0x7f4b080c7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.702+0000 7f4b0f000640 1 --2- 172.21.15.39:0/3786092270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b080b2f70 0x7f4b0814efe0 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:09.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.702+0000 7f4b0f000640 1 -- 172.21.15.39:0/3786092270 >> 172.21.15.39:0/3786092270 conn(0x7f4b0809b5f0 msgr2=0x7f4b080b6010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:09.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.702+0000 7f4b0f000640 1 -- 172.21.15.39:0/3786092270 shutdown_connections 2024-09-21T07:48:09.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:09.702+0000 7f4b0f000640 1 -- 172.21.15.39:0/3786092270 wait complete. 2024-09-21T07:48:09.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:48:09.728 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d0/d1/empty) 2024-09-21T07:48:09.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:48:09.772 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d0) 2024-09-21T07:48:09.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:48:09.850 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d0/d1) 2024-09-21T07:48:09.926 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-21T07:48:10.213 INFO:teuthology.orchestra.run.smithi039.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T07:48:10.214 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T07:48:10.214 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds1", "dirfrag": "0x101", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-09-21T07:48:10.214 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-21T07:48:10.463 INFO:teuthology.orchestra.run.smithi039.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"9","version":"9","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T07:48:10.463 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "9", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "9" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T07:48:10.463 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "9", "version": "9", "committing_version": "1", "committed_version": "1", "is_rep": true, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "replicated": 0, "dirty": 1, "authpin": 0 }, "nref": 3 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds0", "dirfrag": "0x100", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "0", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-09-21T07:48:10.463 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d0', 0), ('/d0/d1', 1)] 2024-09-21T07:48:12.465 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-21T07:48:12.721 INFO:teuthology.orchestra.run.smithi039.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T07:48:12.721 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T07:48:12.721 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds1", "dirfrag": "0x101", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-09-21T07:48:12.722 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-21T07:48:12.967 INFO:teuthology.orchestra.run.smithi039.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"9","version":"9","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T07:48:12.968 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "9", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "9" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T07:48:12.968 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "9", "version": "9", "committing_version": "1", "committed_version": "1", "is_rep": true, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "replicated": 0, "dirty": 1, "authpin": 0 }, "nref": 3 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds0", "dirfrag": "0x100", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "0", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-09-21T07:48:12.968 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d0', 0), ('/d0/d1', 1)] 2024-09-21T07:48:14.969 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-21T07:48:15.231 INFO:teuthology.orchestra.run.smithi039.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T07:48:15.231 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T07:48:15.232 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds1", "dirfrag": "0x101", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-09-21T07:48:15.232 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-21T07:48:15.486 INFO:teuthology.orchestra.run.smithi039.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d0","dirfrag":"0x10000000000","snapid_first":2,"projected_version":"8","version":"8","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d0/d1","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"3","version":"3","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-09-21T07:48:15.487 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000000", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "8", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "8" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000001", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d0/d1", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "3", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "3" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-21T07:48:15.487 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "13", "version": "13", "committing_version": "1", "committed_version": "1", "is_rep": true, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "1": 1 } }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "replicated": 1, "dirty": 1, "authpin": 0 }, "nref": 4 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds0", "dirfrag": "0x100", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "0", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": 0, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/d0", "dirfrag": "0x10000000000", "snapid_first": 2, "projected_version": "8", "version": "8", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "1": 1 } }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "replicated": 1, "dirty": 1, "authpin": 0 }, "nref": 4 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/d0/d1", "dirfrag": "0x10000000001", "snapid_first": 2, "projected_version": "3", "version": "3", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } } ] 2024-09-21T07:48:15.487 INFO:tasks.cephfs.cephfs_test_case:[('/d0', 0)] =?= [('/d0', 0), ('/d0/d1', 1)] 2024-09-21T07:48:17.489 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-21T07:48:17.745 INFO:teuthology.orchestra.run.smithi039.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d0/d1","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"3","version":"3","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-09-21T07:48:17.746 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d0/d1", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "3", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "3" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T07:48:17.746 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds1", "dirfrag": "0x101", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } }, { "is_auth": false, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "0", "version": "13", "committing_version": "0", "committed_version": "0", "is_rep": true, "dir_auth": "0", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1 }, "nref": 2 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/d0/d1", "dirfrag": "0x10000000001", "snapid_first": 2, "projected_version": "3", "version": "3", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } } ] 2024-09-21T07:48:17.747 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-21T07:48:17.991 INFO:teuthology.orchestra.run.smithi039.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d0","dirfrag":"0x10000000000","snapid_first":2,"projected_version":"8","version":"8","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d0/d1","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"3","version":"3","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-09-21T07:48:17.991 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000000", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "8", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "8" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000001", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d0/d1", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "3", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "3" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-21T07:48:17.992 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "13", "version": "13", "committing_version": "1", "committed_version": "1", "is_rep": true, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "1": 1 } }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "replicated": 1, "dirty": 1, "authpin": 0 }, "nref": 4 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds0", "dirfrag": "0x100", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "0", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": 0, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/d0", "dirfrag": "0x10000000000", "snapid_first": 2, "projected_version": "8", "version": "8", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "1": 1 } }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "replicated": 1, "dirty": 1, "authpin": 0 }, "nref": 4 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/d0/d1", "dirfrag": "0x10000000001", "snapid_first": 2, "projected_version": "3", "version": "3", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } } ] 2024-09-21T07:48:17.992 INFO:tasks.cephfs.cephfs_test_case:[('/d0', 0), ('/d0/d1', 1)] =?= [('/d0', 0), ('/d0/d1', 1)] 2024-09-21T07:48:17.992 INFO:tasks.cephfs.mount:Writing 8388608 bytes to d0/d1/file_a 2024-09-21T07:48:17.992 DEBUG:teuthology.orchestra.run.smithi039:> stdin-killer -- python3 -c ' 2024-09-21T07:48:17.992 DEBUG:teuthology.orchestra.run.smithi039:> import zlib 2024-09-21T07:48:17.992 DEBUG:teuthology.orchestra.run.smithi039:> path = "/home/ubuntu/cephtest/mnt.0/d0/d1/file_a" 2024-09-21T07:48:17.992 DEBUG:teuthology.orchestra.run.smithi039:> with open(path, '"'"'w'"'"') as f: 2024-09-21T07:48:17.992 DEBUG:teuthology.orchestra.run.smithi039:> for i in range(0, 8388608): 2024-09-21T07:48:17.992 DEBUG:teuthology.orchestra.run.smithi039:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2024-09-21T07:48:17.993 DEBUG:teuthology.orchestra.run.smithi039:> f.write(chr(val)) 2024-09-21T07:48:17.993 DEBUG:teuthology.orchestra.run.smithi039:> ' 2024-09-21T07:48:22.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:48:22.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:48:22.628 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d0/.snap/s1) 2024-09-21T07:48:22.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:48:22.732 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f d0/d1/file_a) 2024-09-21T07:48:22.769 INFO:tasks.cephfs.mount:Validating 8388608 bytes from d0/.snap/s1/d1/file_a 2024-09-21T07:48:22.769 DEBUG:teuthology.orchestra.run.smithi039:> sudo stdin-killer --timeout 20 -- python3 -c ' 2024-09-21T07:48:22.769 DEBUG:teuthology.orchestra.run.smithi039:> import zlib 2024-09-21T07:48:22.770 DEBUG:teuthology.orchestra.run.smithi039:> path = "/home/ubuntu/cephtest/mnt.0/d0/.snap/s1/d1/file_a" 2024-09-21T07:48:22.770 DEBUG:teuthology.orchestra.run.smithi039:> with open(path, '"'"'r'"'"') as f: 2024-09-21T07:48:22.770 DEBUG:teuthology.orchestra.run.smithi039:> bytes = f.read() 2024-09-21T07:48:22.770 DEBUG:teuthology.orchestra.run.smithi039:> if len(bytes) != 8388608: 2024-09-21T07:48:22.770 DEBUG:teuthology.orchestra.run.smithi039:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2024-09-21T07:48:22.770 DEBUG:teuthology.orchestra.run.smithi039:> len(bytes), 8388608 2024-09-21T07:48:22.770 DEBUG:teuthology.orchestra.run.smithi039:> )) 2024-09-21T07:48:22.770 DEBUG:teuthology.orchestra.run.smithi039:> for i, b in enumerate(bytes): 2024-09-21T07:48:22.770 DEBUG:teuthology.orchestra.run.smithi039:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2024-09-21T07:48:22.770 DEBUG:teuthology.orchestra.run.smithi039:> if b != chr(val): 2024-09-21T07:48:22.770 DEBUG:teuthology.orchestra.run.smithi039:> raise RuntimeError("Bad data at offset {0}".format(i)) 2024-09-21T07:48:22.770 DEBUG:teuthology.orchestra.run.smithi039:> ' 2024-09-21T07:48:27.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:48:27.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:48:27.460 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir d0/.snap/s1) 2024-09-21T07:48:27.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:48:27.546 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -rf d0) 2024-09-21T07:48:27.653 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:48:27.653 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:48:27.747 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:48:27.849 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:48:27.849 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:48:27.849 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:48:27.849 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:48:27.885 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:48:27.885 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:48:27.960 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:48:28.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.228+0000 7f8f47200640 1 -- 172.21.15.39:0/467978435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f40160250 msgr2=0x7f8f4015d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:28.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.228+0000 7f8f47200640 1 --2- 172.21.15.39:0/467978435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f40160250 0x7f8f4015d570 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f8f34009830 tx=0x7f8f3402a510 comp rx=0 tx=0).stop 2024-09-21T07:48:28.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.229+0000 7f8f47200640 1 -- 172.21.15.39:0/467978435 shutdown_connections 2024-09-21T07:48:28.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.229+0000 7f8f47200640 1 --2- 172.21.15.39:0/467978435 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f400d0390 0x7f8f400d07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:28.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.229+0000 7f8f47200640 1 --2- 172.21.15.39:0/467978435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f40160250 0x7f8f4015d570 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:28.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.229+0000 7f8f47200640 1 -- 172.21.15.39:0/467978435 >> 172.21.15.39:0/467978435 conn(0x7f8f4006cfc0 msgr2=0x7f8f4006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:28.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.229+0000 7f8f47200640 1 -- 172.21.15.39:0/467978435 shutdown_connections 2024-09-21T07:48:28.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.229+0000 7f8f47200640 1 -- 172.21.15.39:0/467978435 wait complete. 2024-09-21T07:48:28.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.231+0000 7f8f47200640 1 Processor -- start 2024-09-21T07:48:28.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.231+0000 7f8f47200640 1 -- start start 2024-09-21T07:48:28.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.232+0000 7f8f47200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f400d0390 0x7f8f401f6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:28.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.232+0000 7f8f47200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f40160250 0x7f8f40148b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:28.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.232+0000 7f8f47200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f40160660 0x7f8f401e6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:28.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.232+0000 7f8f47200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8f400ccdc0 con 0x7f8f400d0390 2024-09-21T07:48:28.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.232+0000 7f8f47200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8f400ccc40 con 0x7f8f40160660 2024-09-21T07:48:28.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.232+0000 7f8f47200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f8f400ccf40 con 0x7f8f40160250 2024-09-21T07:48:28.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.233+0000 7f8f45e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f400d0390 0x7f8f401f6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:28.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.233+0000 7f8f45e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f400d0390 0x7f8f401f6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43002/0 (socket says 172.21.15.39:43002) 2024-09-21T07:48:28.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.233+0000 7f8f45e00640 1 -- 172.21.15.39:0/1293749798 learned_addr learned my addr 172.21.15.39:0/1293749798 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:28.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.233+0000 7f8f46800640 1 --2- 172.21.15.39:0/1293749798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f40160660 0x7f8f401e6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:28.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.233+0000 7f8f45400640 1 --2- 172.21.15.39:0/1293749798 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f40160250 0x7f8f40148b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:28.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.233+0000 7f8f45e00640 1 -- 172.21.15.39:0/1293749798 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f40160250 msgr2=0x7f8f40148b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:28.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.233+0000 7f8f45e00640 1 --2- 172.21.15.39:0/1293749798 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f40160250 0x7f8f40148b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:28.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.233+0000 7f8f45e00640 1 -- 172.21.15.39:0/1293749798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f40160660 msgr2=0x7f8f401e6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:28.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.233+0000 7f8f45e00640 1 --2- 172.21.15.39:0/1293749798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f40160660 0x7f8f401e6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:28.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.233+0000 7f8f45e00640 1 -- 172.21.15.39:0/1293749798 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f401f6c90 con 0x7f8f400d0390 2024-09-21T07:48:28.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.234+0000 7f8f45e00640 1 --2- 172.21.15.39:0/1293749798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f400d0390 0x7f8f401f6750 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f8f34009950 tx=0x7f8f34033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:28.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.234+0000 7f8f3a000640 1 -- 172.21.15.39:0/1293749798 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f34005c90 con 0x7f8f400d0390 2024-09-21T07:48:28.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.234+0000 7f8f3a000640 1 -- 172.21.15.39:0/1293749798 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f34031e50 con 0x7f8f400d0390 2024-09-21T07:48:28.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.234+0000 7f8f47200640 1 -- 172.21.15.39:0/1293749798 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f400ad070 con 0x7f8f400d0390 2024-09-21T07:48:28.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.234+0000 7f8f3a000640 1 -- 172.21.15.39:0/1293749798 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f340332f0 con 0x7f8f400d0390 2024-09-21T07:48:28.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.234+0000 7f8f47200640 1 -- 172.21.15.39:0/1293749798 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f400ad940 con 0x7f8f400d0390 2024-09-21T07:48:28.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.235+0000 7f8f47200640 1 -- 172.21.15.39:0/1293749798 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f400ad760 con 0x7f8f400d0390 2024-09-21T07:48:28.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.237+0000 7f8f3a000640 1 -- 172.21.15.39:0/1293749798 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8f34033490 con 0x7f8f400d0390 2024-09-21T07:48:28.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.238+0000 7f8f3a000640 1 --2- 172.21.15.39:0/1293749798 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8f24078b90 0x7f8f2407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:28.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.238+0000 7f8f3a000640 1 -- 172.21.15.39:0/1293749798 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(206..206 src has 1..206) ==== 6960+0+0 (secure 0 0 0) 0x7f8f340bcb00 con 0x7f8f400d0390 2024-09-21T07:48:28.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.238+0000 7f8f45400640 1 --2- 172.21.15.39:0/1293749798 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8f24078b90 0x7f8f2407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:28.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.239+0000 7f8f45400640 1 --2- 172.21.15.39:0/1293749798 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8f24078b90 0x7f8f2407b0a0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f8f30007d60 tx=0x7f8f30007cf0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:28.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.241+0000 7f8f3a000640 1 -- 172.21.15.39:0/1293749798 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8f400ad760 con 0x7f8f400d0390 2024-09-21T07:48:28.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.452+0000 7f8f47200640 1 -- 172.21.15.39:0/1293749798 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8f400cd460 con 0x7f8f400d0390 2024-09-21T07:48:28.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.452+0000 7f8f3a000640 1 -- 172.21.15.39:0/1293749798 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v206) ==== 80+0+0 (secure 0 0 0) 0x7f8f34051020 con 0x7f8f400d0390 2024-09-21T07:48:28.454 INFO:teuthology.orchestra.run.smithi039.stderr:listed 0 entries 2024-09-21T07:48:28.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.454+0000 7f8f47200640 1 -- 172.21.15.39:0/1293749798 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8f24078b90 msgr2=0x7f8f2407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:28.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.454+0000 7f8f47200640 1 --2- 172.21.15.39:0/1293749798 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8f24078b90 0x7f8f2407b0a0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f8f30007d60 tx=0x7f8f30007cf0 comp rx=0 tx=0).stop 2024-09-21T07:48:28.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.454+0000 7f8f47200640 1 -- 172.21.15.39:0/1293749798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f400d0390 msgr2=0x7f8f401f6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:28.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.454+0000 7f8f47200640 1 --2- 172.21.15.39:0/1293749798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f400d0390 0x7f8f401f6750 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f8f34009950 tx=0x7f8f34033c20 comp rx=0 tx=0).stop 2024-09-21T07:48:28.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.455+0000 7f8f47200640 1 -- 172.21.15.39:0/1293749798 shutdown_connections 2024-09-21T07:48:28.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.455+0000 7f8f47200640 1 --2- 172.21.15.39:0/1293749798 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8f24078b90 0x7f8f2407b0a0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:28.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.455+0000 7f8f47200640 1 --2- 172.21.15.39:0/1293749798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f40160660 0x7f8f401e6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:28.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.455+0000 7f8f47200640 1 --2- 172.21.15.39:0/1293749798 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8f40160250 0x7f8f40148b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:28.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.455+0000 7f8f47200640 1 --2- 172.21.15.39:0/1293749798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f400d0390 0x7f8f401f6750 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:28.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.455+0000 7f8f47200640 1 -- 172.21.15.39:0/1293749798 >> 172.21.15.39:0/1293749798 conn(0x7f8f4006cfc0 msgr2=0x7f8f400cd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:28.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.455+0000 7f8f47200640 1 -- 172.21.15.39:0/1293749798 shutdown_connections 2024-09-21T07:48:28.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.455+0000 7f8f47200640 1 -- 172.21.15.39:0/1293749798 wait complete. 2024-09-21T07:48:28.481 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T07:48:28.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:48:28.482 DEBUG:teuthology.orchestra.run.smithi039:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T07:48:28.549 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T07:48:28.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:48:28.550 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:48:28.586 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:48:28.586 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:48:28.586 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:48:28.626 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:48:28.626 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:48:28.626 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:48:28.627 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:48:28.662 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:48:28.662 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:48:28.662 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:48:28.662 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:48:28.705 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:48:28.705 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:48:28.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.929+0000 7f68ccc00640 1 -- 172.21.15.39:0/3875961701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68c80b30a0 msgr2=0x7f68c80cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:28.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.929+0000 7f68ccc00640 1 --2- 172.21.15.39:0/3875961701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68c80b30a0 0x7f68c80cb8a0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f68c80cbde0 tx=0x7f68b802a4b0 comp rx=0 tx=0).stop 2024-09-21T07:48:28.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.929+0000 7f68ccc00640 1 -- 172.21.15.39:0/3875961701 shutdown_connections 2024-09-21T07:48:28.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.929+0000 7f68ccc00640 1 --2- 172.21.15.39:0/3875961701 >> v2:172.21.15.148:3300/0 conn(0x7f68c80ccc60 0x7f68c80cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:28.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.929+0000 7f68ccc00640 1 --2- 172.21.15.39:0/3875961701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68c80b30a0 0x7f68c80cb8a0 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:28.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.929+0000 7f68ccc00640 1 -- 172.21.15.39:0/3875961701 >> 172.21.15.39:0/3875961701 conn(0x7f68c80b2460 msgr2=0x7f68c806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:28.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.930+0000 7f68ccc00640 1 -- 172.21.15.39:0/3875961701 shutdown_connections 2024-09-21T07:48:28.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.930+0000 7f68ccc00640 1 -- 172.21.15.39:0/3875961701 wait complete. 2024-09-21T07:48:28.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.932+0000 7f68ccc00640 1 Processor -- start 2024-09-21T07:48:28.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.932+0000 7f68ccc00640 1 -- start start 2024-09-21T07:48:28.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.933+0000 7f68ccc00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68c80b30a0 0x7f68c8153e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:28.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.933+0000 7f68ccc00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68c80ccc60 0x7f68c80994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:28.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.933+0000 7f68ccc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68c815d830 0x7f68c81f70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:28.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.933+0000 7f68ccc00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f68c80af410 con 0x7f68c815d830 2024-09-21T07:48:28.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.933+0000 7f68ccc00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f68c80af290 con 0x7f68c80ccc60 2024-09-21T07:48:28.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.933+0000 7f68ccc00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f68c80af590 con 0x7f68c80b30a0 2024-09-21T07:48:28.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.934+0000 7f68c6800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68c815d830 0x7f68c81f70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:28.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.934+0000 7f68c6800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68c815d830 0x7f68c81f70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43014/0 (socket says 172.21.15.39:43014) 2024-09-21T07:48:28.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.934+0000 7f68c6800640 1 -- 172.21.15.39:0/3458472576 learned_addr learned my addr 172.21.15.39:0/3458472576 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:28.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.934+0000 7f68c5e00640 1 --2- 172.21.15.39:0/3458472576 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68c80b30a0 0x7f68c8153e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:28.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.934+0000 7f68c5400640 1 --2- 172.21.15.39:0/3458472576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68c80ccc60 0x7f68c80994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:28.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.934+0000 7f68c6800640 1 -- 172.21.15.39:0/3458472576 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68c80b30a0 msgr2=0x7f68c8153e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:28.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.934+0000 7f68c6800640 1 --2- 172.21.15.39:0/3458472576 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68c80b30a0 0x7f68c8153e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:28.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.934+0000 7f68c6800640 1 -- 172.21.15.39:0/3458472576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68c80ccc60 msgr2=0x7f68c80994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:28.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.934+0000 7f68c6800640 1 --2- 172.21.15.39:0/3458472576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68c80ccc60 0x7f68c80994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:28.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.934+0000 7f68c6800640 1 -- 172.21.15.39:0/3458472576 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c8099a20 con 0x7f68c815d830 2024-09-21T07:48:28.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.934+0000 7f68c5400640 1 --2- 172.21.15.39:0/3458472576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68c80ccc60 0x7f68c80994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:48:28.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.934+0000 7f68c6800640 1 --2- 172.21.15.39:0/3458472576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68c815d830 0x7f68c81f70e0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f68c000e9c0 tx=0x7f68c000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:28.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.935+0000 7f68be000640 1 -- 172.21.15.39:0/3458472576 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c0004d20 con 0x7f68c815d830 2024-09-21T07:48:28.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.935+0000 7f68be000640 1 -- 172.21.15.39:0/3458472576 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68c000cb20 con 0x7f68c815d830 2024-09-21T07:48:28.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.935+0000 7f68be000640 1 -- 172.21.15.39:0/3458472576 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c0005020 con 0x7f68c815d830 2024-09-21T07:48:28.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.935+0000 7f68ccc00640 1 -- 172.21.15.39:0/3458472576 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68c8154490 con 0x7f68c815d830 2024-09-21T07:48:28.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.935+0000 7f68ccc00640 1 -- 172.21.15.39:0/3458472576 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68c8154a50 con 0x7f68c815d830 2024-09-21T07:48:28.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.936+0000 7f68ccc00640 1 -- 172.21.15.39:0/3458472576 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68c80ca680 con 0x7f68c815d830 2024-09-21T07:48:28.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.937+0000 7f68be000640 1 -- 172.21.15.39:0/3458472576 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f68c00051c0 con 0x7f68c815d830 2024-09-21T07:48:28.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.938+0000 7f68be000640 1 --2- 172.21.15.39:0/3458472576 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f68a8078b90 0x7f68a807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:28.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.938+0000 7f68be000640 1 -- 172.21.15.39:0/3458472576 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(206..206 src has 1..206) ==== 6960+0+0 (secure 0 0 0) 0x7f68c009bae0 con 0x7f68c815d830 2024-09-21T07:48:28.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.939+0000 7f68c5e00640 1 --2- 172.21.15.39:0/3458472576 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f68a8078b90 0x7f68a807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:28.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.939+0000 7f68c5e00640 1 --2- 172.21.15.39:0/3458472576 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f68a8078b90 0x7f68a807b0a0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f68b8034000 tx=0x7f68b8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:28.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:28.943+0000 7f68be000640 1 -- 172.21.15.39:0/3458472576 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f68c0016030 con 0x7f68c815d830 2024-09-21T07:48:29.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.176+0000 7f68ccc00640 1 -- 172.21.15.39:0/3458472576 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68c806ff70 con 0x7f68c815d830 2024-09-21T07:48:29.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.176+0000 7f68be000640 1 -- 172.21.15.39:0/3458472576 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 219 v219) ==== 95+0+5259 (secure 0 0 0) 0x7f68c0063a50 con 0x7f68c815d830 2024-09-21T07:48:29.179 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:48:29.179 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":219,"btime":"2024-09-21T07:48:15:307850+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7988,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1912631346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1912631346},{"type":"v1","addr":"172.21.15.148:6837","nonce":1912631346}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":209},{"gid":9043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2220628944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2220628944},{"type":"v1","addr":"172.21.15.148:6836","nonce":2220628944}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":208}],"filesystems":[{"mdsmap":{"epoch":219,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:47:54.691835+0000","modified":"2024-09-21T07:48:15.172238+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":9852,"mds_1":9049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9049":{"gid":9049,"name":"a","rank":1,"incarnation":217,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6833/1947479913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1947479913},{"type":"v1","addr":"172.21.15.39:6833","nonce":1947479913}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_9852":{"gid":9852,"name":"c","rank":0,"incarnation":210,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/2356847598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2356847598},{"type":"v1","addr":"172.21.15.39:6835","nonce":2356847598}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9852,"qdb_cluster":[9852,9049]},"id":11}]} 2024-09-21T07:48:29.179 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 219 2024-09-21T07:48:29.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.179+0000 7f68ccc00640 1 -- 172.21.15.39:0/3458472576 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f68a8078b90 msgr2=0x7f68a807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:29.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.179+0000 7f68ccc00640 1 --2- 172.21.15.39:0/3458472576 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f68a8078b90 0x7f68a807b0a0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f68b8034000 tx=0x7f68b8004730 comp rx=0 tx=0).stop 2024-09-21T07:48:29.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.179+0000 7f68ccc00640 1 -- 172.21.15.39:0/3458472576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68c815d830 msgr2=0x7f68c81f70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:29.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.179+0000 7f68ccc00640 1 --2- 172.21.15.39:0/3458472576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68c815d830 0x7f68c81f70e0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f68c000e9c0 tx=0x7f68c000ee80 comp rx=0 tx=0).stop 2024-09-21T07:48:29.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.179+0000 7f68ccc00640 1 -- 172.21.15.39:0/3458472576 shutdown_connections 2024-09-21T07:48:29.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.179+0000 7f68ccc00640 1 --2- 172.21.15.39:0/3458472576 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f68a8078b90 0x7f68a807b0a0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:29.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.179+0000 7f68ccc00640 1 --2- 172.21.15.39:0/3458472576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68c815d830 0x7f68c81f70e0 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:29.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.179+0000 7f68ccc00640 1 --2- 172.21.15.39:0/3458472576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f68c80ccc60 0x7f68c80994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:29.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.179+0000 7f68ccc00640 1 --2- 172.21.15.39:0/3458472576 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68c80b30a0 0x7f68c8153e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:29.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.179+0000 7f68ccc00640 1 -- 172.21.15.39:0/3458472576 >> 172.21.15.39:0/3458472576 conn(0x7f68c80b2460 msgr2=0x7f68c80b4910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:29.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.179+0000 7f68ccc00640 1 -- 172.21.15.39:0/3458472576 shutdown_connections 2024-09-21T07:48:29.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.179+0000 7f68ccc00640 1 -- 172.21.15.39:0/3458472576 wait complete. 2024-09-21T07:48:29.204 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:48:29.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.429+0000 7f6b70c00640 1 -- 172.21.15.39:0/574498689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b6c0cfa20 msgr2=0x7f6b6c0cd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:29.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.429+0000 7f6b70c00640 1 --2- 172.21.15.39:0/574498689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b6c0cfa20 0x7f6b6c0cd5a0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c072f70 tx=0x7f6b5c02a530 comp rx=0 tx=0).stop 2024-09-21T07:48:29.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.429+0000 7f6b70c00640 1 -- 172.21.15.39:0/574498689 shutdown_connections 2024-09-21T07:48:29.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.429+0000 7f6b70c00640 1 --2- 172.21.15.39:0/574498689 >> v2:172.21.15.148:3300/0 conn(0x7f6b6c0d0bc0 0x7f6b6c15d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:29.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.429+0000 7f6b70c00640 1 --2- 172.21.15.39:0/574498689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b6c0cfa20 0x7f6b6c0cd5a0 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:29.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.429+0000 7f6b70c00640 1 -- 172.21.15.39:0/574498689 >> 172.21.15.39:0/574498689 conn(0x7f6b6c06cfc0 msgr2=0x7f6b6c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:29.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.429+0000 7f6b70c00640 1 -- 172.21.15.39:0/574498689 shutdown_connections 2024-09-21T07:48:29.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.429+0000 7f6b70c00640 1 -- 172.21.15.39:0/574498689 wait complete. 2024-09-21T07:48:29.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.431+0000 7f6b70c00640 1 Processor -- start 2024-09-21T07:48:29.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.431+0000 7f6b70c00640 1 -- start start 2024-09-21T07:48:29.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.433+0000 7f6b70c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b6c0cfa20 0x7f6b6c152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:29.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.433+0000 7f6b70c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b6c0d0bc0 0x7f6b6c1546f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:29.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.433+0000 7f6b70c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b6c15cc00 0x7f6b6c1f5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:29.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.433+0000 7f6b70c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6c06ee70 con 0x7f6b6c0d0bc0 2024-09-21T07:48:29.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.433+0000 7f6b70c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6c06ecf0 con 0x7f6b6c15cc00 2024-09-21T07:48:29.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.433+0000 7f6b70c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f6b6c06eff0 con 0x7f6b6c0cfa20 2024-09-21T07:48:29.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.433+0000 7f6b62a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b6c0d0bc0 0x7f6b6c1546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:29.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.433+0000 7f6b62a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b6c0d0bc0 0x7f6b6c1546f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43032/0 (socket says 172.21.15.39:43032) 2024-09-21T07:48:29.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.433+0000 7f6b62a00640 1 -- 172.21.15.39:0/794372446 learned_addr learned my addr 172.21.15.39:0/794372446 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:29.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.433+0000 7f6b6b400640 1 --2- 172.21.15.39:0/794372446 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b6c0cfa20 0x7f6b6c152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:29.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.433+0000 7f6b6be00640 1 --2- 172.21.15.39:0/794372446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b6c15cc00 0x7f6b6c1f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:29.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.433+0000 7f6b62a00640 1 -- 172.21.15.39:0/794372446 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b6c0cfa20 msgr2=0x7f6b6c152cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:29.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.433+0000 7f6b62a00640 1 --2- 172.21.15.39:0/794372446 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b6c0cfa20 0x7f6b6c152cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:29.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.433+0000 7f6b62a00640 1 -- 172.21.15.39:0/794372446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b6c15cc00 msgr2=0x7f6b6c1f5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:29.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.433+0000 7f6b62a00640 1 --2- 172.21.15.39:0/794372446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b6c15cc00 0x7f6b6c1f5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:29.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.433+0000 7f6b62a00640 1 -- 172.21.15.39:0/794372446 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b6c0af560 con 0x7f6b6c0d0bc0 2024-09-21T07:48:29.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.434+0000 7f6b62a00640 1 --2- 172.21.15.39:0/794372446 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b6c0d0bc0 0x7f6b6c1546f0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f6b5400c3c0 tx=0x7f6b5400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:29.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.434+0000 7f6b63e00640 1 -- 172.21.15.39:0/794372446 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b54017070 con 0x7f6b6c0d0bc0 2024-09-21T07:48:29.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.434+0000 7f6b63e00640 1 -- 172.21.15.39:0/794372446 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b540035f0 con 0x7f6b6c0d0bc0 2024-09-21T07:48:29.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.434+0000 7f6b70c00640 1 -- 172.21.15.39:0/794372446 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b6c0af840 con 0x7f6b6c0d0bc0 2024-09-21T07:48:29.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.434+0000 7f6b63e00640 1 -- 172.21.15.39:0/794372446 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b54003a80 con 0x7f6b6c0d0bc0 2024-09-21T07:48:29.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.434+0000 7f6b70c00640 1 -- 172.21.15.39:0/794372446 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b6c154010 con 0x7f6b6c0d0bc0 2024-09-21T07:48:29.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.435+0000 7f6b70c00640 1 -- 172.21.15.39:0/794372446 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b6c06f480 con 0x7f6b6c0d0bc0 2024-09-21T07:48:29.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.436+0000 7f6b63e00640 1 -- 172.21.15.39:0/794372446 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6b5401a070 con 0x7f6b6c0d0bc0 2024-09-21T07:48:29.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.437+0000 7f6b63e00640 1 --2- 172.21.15.39:0/794372446 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6b48078bc0 0x7f6b4807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:29.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.438+0000 7f6b63e00640 1 -- 172.21.15.39:0/794372446 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(206..206 src has 1..206) ==== 6960+0+0 (secure 0 0 0) 0x7f6b5409a430 con 0x7f6b6c0d0bc0 2024-09-21T07:48:29.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.438+0000 7f6b6b400640 1 --2- 172.21.15.39:0/794372446 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6b48078bc0 0x7f6b4807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:29.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.439+0000 7f6b6b400640 1 --2- 172.21.15.39:0/794372446 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6b48078bc0 0x7f6b4807b0d0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c034000 tx=0x7f6b5c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:29.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.442+0000 7f6b63e00640 1 -- 172.21.15.39:0/794372446 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6b54061d50 con 0x7f6b6c0d0bc0 2024-09-21T07:48:29.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.678+0000 7f6b70c00640 1 -- 172.21.15.39:0/794372446 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b6c0b3610 con 0x7f6b6c0d0bc0 2024-09-21T07:48:29.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.678+0000 7f6b63e00640 1 -- 172.21.15.39:0/794372446 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 219 v219) ==== 95+0+5259 (secure 0 0 0) 0x7f6b54066780 con 0x7f6b6c0d0bc0 2024-09-21T07:48:29.679 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:48:29.680 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":219,"btime":"2024-09-21T07:48:15:307850+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7988,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1912631346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1912631346},{"type":"v1","addr":"172.21.15.148:6837","nonce":1912631346}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":209},{"gid":9043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2220628944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2220628944},{"type":"v1","addr":"172.21.15.148:6836","nonce":2220628944}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":208}],"filesystems":[{"mdsmap":{"epoch":219,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:47:54.691835+0000","modified":"2024-09-21T07:48:15.172238+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":9852,"mds_1":9049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9049":{"gid":9049,"name":"a","rank":1,"incarnation":217,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6833/1947479913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1947479913},{"type":"v1","addr":"172.21.15.39:6833","nonce":1947479913}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_9852":{"gid":9852,"name":"c","rank":0,"incarnation":210,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/2356847598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2356847598},{"type":"v1","addr":"172.21.15.39:6835","nonce":2356847598}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9852,"qdb_cluster":[9852,9049]},"id":11}]} 2024-09-21T07:48:29.680 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 219 2024-09-21T07:48:29.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.680+0000 7f6b70c00640 1 -- 172.21.15.39:0/794372446 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6b48078bc0 msgr2=0x7f6b4807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:29.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.680+0000 7f6b70c00640 1 --2- 172.21.15.39:0/794372446 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6b48078bc0 0x7f6b4807b0d0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c034000 tx=0x7f6b5c004730 comp rx=0 tx=0).stop 2024-09-21T07:48:29.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.680+0000 7f6b70c00640 1 -- 172.21.15.39:0/794372446 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b6c0d0bc0 msgr2=0x7f6b6c1546f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:29.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.680+0000 7f6b70c00640 1 --2- 172.21.15.39:0/794372446 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b6c0d0bc0 0x7f6b6c1546f0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f6b5400c3c0 tx=0x7f6b5400c880 comp rx=0 tx=0).stop 2024-09-21T07:48:29.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.680+0000 7f6b70c00640 1 -- 172.21.15.39:0/794372446 shutdown_connections 2024-09-21T07:48:29.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.680+0000 7f6b70c00640 1 --2- 172.21.15.39:0/794372446 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6b48078bc0 0x7f6b4807b0d0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:29.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.680+0000 7f6b70c00640 1 --2- 172.21.15.39:0/794372446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b6c15cc00 0x7f6b6c1f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:29.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.680+0000 7f6b70c00640 1 --2- 172.21.15.39:0/794372446 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b6c0d0bc0 0x7f6b6c1546f0 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:29.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.680+0000 7f6b70c00640 1 --2- 172.21.15.39:0/794372446 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b6c0cfa20 0x7f6b6c152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:29.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.680+0000 7f6b70c00640 1 -- 172.21.15.39:0/794372446 >> 172.21.15.39:0/794372446 conn(0x7f6b6c06cfc0 msgr2=0x7f6b6c0b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:29.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.680+0000 7f6b70c00640 1 -- 172.21.15.39:0/794372446 shutdown_connections 2024-09-21T07:48:29.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.680+0000 7f6b70c00640 1 -- 172.21.15.39:0/794372446 wait complete. 2024-09-21T07:48:29.709 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:48:29.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.933+0000 7fa0f8400640 1 -- 172.21.15.39:0/1060004733 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0f00b3d80 msgr2=0x7fa0f00ce2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:29.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.933+0000 7fa0f8400640 1 --2- 172.21.15.39:0/1060004733 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0f00b3d80 0x7fa0f00ce2b0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc009830 tx=0x7fa0dc02a510 comp rx=0 tx=0).stop 2024-09-21T07:48:29.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.933+0000 7fa0f8400640 1 -- 172.21.15.39:0/1060004733 shutdown_connections 2024-09-21T07:48:29.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.933+0000 7fa0f8400640 1 --2- 172.21.15.39:0/1060004733 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa0f006ef90 0x7fa0f006f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:29.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.933+0000 7fa0f8400640 1 --2- 172.21.15.39:0/1060004733 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0f00b3d80 0x7fa0f00ce2b0 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:29.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.933+0000 7fa0f8400640 1 -- 172.21.15.39:0/1060004733 >> 172.21.15.39:0/1060004733 conn(0x7fa0f00b2460 msgr2=0x7fa0f006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:29.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.933+0000 7fa0f8400640 1 -- 172.21.15.39:0/1060004733 shutdown_connections 2024-09-21T07:48:29.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.933+0000 7fa0f8400640 1 -- 172.21.15.39:0/1060004733 wait complete. 2024-09-21T07:48:29.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.935+0000 7fa0f8400640 1 Processor -- start 2024-09-21T07:48:29.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.936+0000 7fa0f8400640 1 -- start start 2024-09-21T07:48:29.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.937+0000 7fa0f8400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa0f006ef90 0x7fa0f01f2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:29.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.937+0000 7fa0f8400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0f00b4c40 0x7fa0f01f8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:29.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.937+0000 7fa0f8400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0f0074660 0x7fa0f01f9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:29.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.937+0000 7fa0f8400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f006fa60 con 0x7fa0f0074660 2024-09-21T07:48:29.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.937+0000 7fa0f8400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f006f8e0 con 0x7fa0f00b4c40 2024-09-21T07:48:29.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.937+0000 7fa0f8400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa0f006fbe0 con 0x7fa0f006ef90 2024-09-21T07:48:29.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.937+0000 7fa0f6400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0f0074660 0x7fa0f01f9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:29.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.937+0000 7fa0f6400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0f0074660 0x7fa0f01f9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43044/0 (socket says 172.21.15.39:43044) 2024-09-21T07:48:29.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.937+0000 7fa0f6400640 1 -- 172.21.15.39:0/2733946922 learned_addr learned my addr 172.21.15.39:0/2733946922 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:29.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.937+0000 7fa0f5a00640 1 --2- 172.21.15.39:0/2733946922 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa0f006ef90 0x7fa0f01f2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:29.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.937+0000 7fa0f5000640 1 --2- 172.21.15.39:0/2733946922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0f00b4c40 0x7fa0f01f8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:29.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.938+0000 7fa0f6400640 1 -- 172.21.15.39:0/2733946922 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa0f006ef90 msgr2=0x7fa0f01f2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:29.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.938+0000 7fa0f6400640 1 --2- 172.21.15.39:0/2733946922 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa0f006ef90 0x7fa0f01f2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:29.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.938+0000 7fa0f6400640 1 -- 172.21.15.39:0/2733946922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0f00b4c40 msgr2=0x7fa0f01f8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:29.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.938+0000 7fa0f6400640 1 --2- 172.21.15.39:0/2733946922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0f00b4c40 0x7fa0f01f8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:29.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.938+0000 7fa0f6400640 1 -- 172.21.15.39:0/2733946922 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0f01f9e10 con 0x7fa0f0074660 2024-09-21T07:48:29.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.938+0000 7fa0f6400640 1 --2- 172.21.15.39:0/2733946922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0f0074660 0x7fa0f01f9450 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7fa0e800e990 tx=0x7fa0e800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:29.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.938+0000 7fa0ed600640 1 -- 172.21.15.39:0/2733946922 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8004d20 con 0x7fa0f0074660 2024-09-21T07:48:29.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.939+0000 7fa0f8400640 1 -- 172.21.15.39:0/2733946922 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0f015a1d0 con 0x7fa0f0074660 2024-09-21T07:48:29.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.939+0000 7fa0ed600640 1 -- 172.21.15.39:0/2733946922 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa0e800cb00 con 0x7fa0f0074660 2024-09-21T07:48:29.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.939+0000 7fa0ed600640 1 -- 172.21.15.39:0/2733946922 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8005020 con 0x7fa0f0074660 2024-09-21T07:48:29.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.939+0000 7fa0f8400640 1 -- 172.21.15.39:0/2733946922 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0f01f9b30 con 0x7fa0f0074660 2024-09-21T07:48:29.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.940+0000 7fa0f8400640 1 -- 172.21.15.39:0/2733946922 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0f00d0910 con 0x7fa0f0074660 2024-09-21T07:48:29.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.941+0000 7fa0ed600640 1 -- 172.21.15.39:0/2733946922 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa0e80051c0 con 0x7fa0f0074660 2024-09-21T07:48:29.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.942+0000 7fa0ed600640 1 --2- 172.21.15.39:0/2733946922 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa0d0078b90 0x7fa0d007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:29.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.942+0000 7fa0ed600640 1 -- 172.21.15.39:0/2733946922 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(206..206 src has 1..206) ==== 6960+0+0 (secure 0 0 0) 0x7fa0e809bba0 con 0x7fa0f0074660 2024-09-21T07:48:29.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.943+0000 7fa0f5a00640 1 --2- 172.21.15.39:0/2733946922 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa0d0078b90 0x7fa0d007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:29.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.943+0000 7fa0f5a00640 1 --2- 172.21.15.39:0/2733946922 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa0d0078b90 0x7fa0d007b0a0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fa0f00ca850 tx=0x7fa0e4007eb0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:29.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:29.947+0000 7fa0ed600640 1 -- 172.21.15.39:0/2733946922 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa0e8016030 con 0x7fa0f0074660 2024-09-21T07:48:30.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.144+0000 7fa0f8400640 1 -- 172.21.15.39:0/2733946922 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa0f006f3a0 con 0x7fa0f0074660 2024-09-21T07:48:30.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.146+0000 7fa0ed600640 1 -- 172.21.15.39:0/2733946922 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v206) ==== 74+0+22397 (secure 0 0 0) 0x7fa0e8063440 con 0x7fa0f0074660 2024-09-21T07:48:30.148 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:48:30.148 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":206,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:47:56.161453+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:47:52.623339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"205","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-09-21T07:47:53.638900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"206","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]},{"pool":20,"snaps":[{"begin":80,"length":23},{"begin":181,"length":23}]},{"pool":22,"snaps":[{"begin":100,"length":1},{"begin":199,"length":1}]},{"pool":24,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:48:30.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.149+0000 7fa0f8400640 1 -- 172.21.15.39:0/2733946922 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa0d0078b90 msgr2=0x7fa0d007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:30.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.149+0000 7fa0f8400640 1 --2- 172.21.15.39:0/2733946922 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa0d0078b90 0x7fa0d007b0a0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fa0f00ca850 tx=0x7fa0e4007eb0 comp rx=0 tx=0).stop 2024-09-21T07:48:30.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.149+0000 7fa0f8400640 1 -- 172.21.15.39:0/2733946922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0f0074660 msgr2=0x7fa0f01f9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:30.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.149+0000 7fa0f8400640 1 --2- 172.21.15.39:0/2733946922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0f0074660 0x7fa0f01f9450 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7fa0e800e990 tx=0x7fa0e800ee50 comp rx=0 tx=0).stop 2024-09-21T07:48:30.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.149+0000 7fa0f8400640 1 -- 172.21.15.39:0/2733946922 shutdown_connections 2024-09-21T07:48:30.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.149+0000 7fa0f8400640 1 --2- 172.21.15.39:0/2733946922 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa0d0078b90 0x7fa0d007b0a0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:30.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.149+0000 7fa0f8400640 1 --2- 172.21.15.39:0/2733946922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0f0074660 0x7fa0f01f9450 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:30.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.149+0000 7fa0f8400640 1 --2- 172.21.15.39:0/2733946922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0f00b4c40 0x7fa0f01f8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:30.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.149+0000 7fa0f8400640 1 --2- 172.21.15.39:0/2733946922 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa0f006ef90 0x7fa0f01f2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:30.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.149+0000 7fa0f8400640 1 -- 172.21.15.39:0/2733946922 >> 172.21.15.39:0/2733946922 conn(0x7fa0f00b2460 msgr2=0x7fa0f00d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:30.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.149+0000 7fa0f8400640 1 -- 172.21.15.39:0/2733946922 shutdown_connections 2024-09-21T07:48:30.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.149+0000 7fa0f8400640 1 -- 172.21.15.39:0/2733946922 wait complete. 2024-09-21T07:48:30.177 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T07:48:30.178 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:48:30.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.416+0000 7fc62d200640 1 -- 172.21.15.39:0/2022981839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6280d08b0 msgr2=0x7fc6280d0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:30.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.416+0000 7fc62d200640 1 --2- 172.21.15.39:0/2022981839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6280d08b0 0x7fc6280d0cc0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7fc628075500 tx=0x7fc61402a5f0 comp rx=0 tx=0).stop 2024-09-21T07:48:30.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.416+0000 7fc62d200640 1 -- 172.21.15.39:0/2022981839 shutdown_connections 2024-09-21T07:48:30.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.416+0000 7fc62d200640 1 --2- 172.21.15.39:0/2022981839 >> v2:172.21.15.148:3300/0 conn(0x7fc6280d1200 0x7fc6280b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:30.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.416+0000 7fc62d200640 1 --2- 172.21.15.39:0/2022981839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6280d08b0 0x7fc6280d0cc0 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:30.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.416+0000 7fc62d200640 1 -- 172.21.15.39:0/2022981839 >> 172.21.15.39:0/2022981839 conn(0x7fc6280cef30 msgr2=0x7fc6280cf340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:30.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.416+0000 7fc62d200640 1 -- 172.21.15.39:0/2022981839 shutdown_connections 2024-09-21T07:48:30.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.416+0000 7fc62d200640 1 -- 172.21.15.39:0/2022981839 wait complete. 2024-09-21T07:48:30.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.419+0000 7fc62d200640 1 Processor -- start 2024-09-21T07:48:30.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.419+0000 7fc62d200640 1 -- start start 2024-09-21T07:48:30.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.420+0000 7fc62d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6280cfa50 0x7fc6281f8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:30.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.420+0000 7fc62d200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc6280d08b0 0x7fc6280b6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:30.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.420+0000 7fc62d200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6280d1200 0x7fc6280b6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:30.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.420+0000 7fc62d200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc62809bc00 con 0x7fc6280cfa50 2024-09-21T07:48:30.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.420+0000 7fc62d200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc62809ba80 con 0x7fc6280d1200 2024-09-21T07:48:30.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.420+0000 7fc62d200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc6280d1610 con 0x7fc6280d08b0 2024-09-21T07:48:30.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.421+0000 7fc625e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6280cfa50 0x7fc6281f8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:30.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.421+0000 7fc625e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6280cfa50 0x7fc6281f8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43060/0 (socket says 172.21.15.39:43060) 2024-09-21T07:48:30.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.421+0000 7fc625e00640 1 -- 172.21.15.39:0/3594503936 learned_addr learned my addr 172.21.15.39:0/3594503936 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:30.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.421+0000 7fc625400640 1 --2- 172.21.15.39:0/3594503936 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc6280d08b0 0x7fc6280b6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:30.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.421+0000 7fc626800640 1 --2- 172.21.15.39:0/3594503936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6280d1200 0x7fc6280b6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:30.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.421+0000 7fc625e00640 1 -- 172.21.15.39:0/3594503936 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc6280d08b0 msgr2=0x7fc6280b6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:30.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.421+0000 7fc625e00640 1 --2- 172.21.15.39:0/3594503936 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc6280d08b0 0x7fc6280b6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:30.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.421+0000 7fc625e00640 1 -- 172.21.15.39:0/3594503936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6280d1200 msgr2=0x7fc6280b6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:30.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.421+0000 7fc625e00640 1 --2- 172.21.15.39:0/3594503936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6280d1200 0x7fc6280b6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:30.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.421+0000 7fc625e00640 1 -- 172.21.15.39:0/3594503936 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6280b6f60 con 0x7fc6280cfa50 2024-09-21T07:48:30.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.422+0000 7fc625e00640 1 --2- 172.21.15.39:0/3594503936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6280cfa50 0x7fc6281f8780 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7fc61800c390 tx=0x7fc61800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:30.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.422+0000 7fc61e000640 1 -- 172.21.15.39:0/3594503936 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc618017070 con 0x7fc6280cfa50 2024-09-21T07:48:30.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.422+0000 7fc61e000640 1 -- 172.21.15.39:0/3594503936 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc6180053c0 con 0x7fc6280cfa50 2024-09-21T07:48:30.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.422+0000 7fc61e000640 1 -- 172.21.15.39:0/3594503936 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc618003880 con 0x7fc6280cfa50 2024-09-21T07:48:30.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.422+0000 7fc62d200640 1 -- 172.21.15.39:0/3594503936 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc62809c300 con 0x7fc6280cfa50 2024-09-21T07:48:30.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.422+0000 7fc62d200640 1 -- 172.21.15.39:0/3594503936 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc62809c880 con 0x7fc6280cfa50 2024-09-21T07:48:30.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.423+0000 7fc62d200640 1 -- 172.21.15.39:0/3594503936 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6280d0cc0 con 0x7fc6280cfa50 2024-09-21T07:48:30.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.424+0000 7fc61e000640 1 -- 172.21.15.39:0/3594503936 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc61801a070 con 0x7fc6280cfa50 2024-09-21T07:48:30.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.425+0000 7fc61e000640 1 --2- 172.21.15.39:0/3594503936 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc608078b90 0x7fc60807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:30.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.425+0000 7fc61e000640 1 -- 172.21.15.39:0/3594503936 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(207..207 src has 1..207) ==== 7016+0+0 (secure 0 0 0) 0x7fc61809a5e0 con 0x7fc6280cfa50 2024-09-21T07:48:30.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.426+0000 7fc625400640 1 --2- 172.21.15.39:0/3594503936 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc608078b90 0x7fc60807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:30.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.427+0000 7fc625400640 1 --2- 172.21.15.39:0/3594503936 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc608078b90 0x7fc60807b0a0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fc614033000 tx=0x7fc614004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:30.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.430+0000 7fc61e000640 1 -- 172.21.15.39:0/3594503936 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc62809c300 con 0x7fc6280cfa50 2024-09-21T07:48:30.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.667+0000 7fc62d200640 1 -- 172.21.15.39:0/3594503936 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc628069c30 con 0x7fc6280cfa50 2024-09-21T07:48:30.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.668+0000 7fc61e000640 1 -- 172.21.15.39:0/3594503936 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 220 v220) ==== 95+0+5260 (secure 0 0 0) 0x7fc618061ea0 con 0x7fc6280cfa50 2024-09-21T07:48:30.670 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:48:30.670 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":220,"btime":"2024-09-21T07:48:30:358899+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7988,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1912631346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1912631346},{"type":"v1","addr":"172.21.15.148:6837","nonce":1912631346}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":209},{"gid":9043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2220628944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2220628944},{"type":"v1","addr":"172.21.15.148:6836","nonce":2220628944}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":208}],"filesystems":[{"mdsmap":{"epoch":220,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:47:54.691835+0000","modified":"2024-09-21T07:48:30.162777+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":9852,"mds_1":9049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9049":{"gid":9049,"name":"a","rank":1,"incarnation":217,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6833/1947479913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1947479913},{"type":"v1","addr":"172.21.15.39:6833","nonce":1947479913}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_9852":{"gid":9852,"name":"c","rank":0,"incarnation":210,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/2356847598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2356847598},{"type":"v1","addr":"172.21.15.39:6835","nonce":2356847598}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9852,"qdb_cluster":[9049,9852]},"id":11}]} 2024-09-21T07:48:30.671 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 220 2024-09-21T07:48:30.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.670+0000 7fc62d200640 1 -- 172.21.15.39:0/3594503936 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc608078b90 msgr2=0x7fc60807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:30.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.670+0000 7fc62d200640 1 --2- 172.21.15.39:0/3594503936 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc608078b90 0x7fc60807b0a0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fc614033000 tx=0x7fc614004730 comp rx=0 tx=0).stop 2024-09-21T07:48:30.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.670+0000 7fc62d200640 1 -- 172.21.15.39:0/3594503936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6280cfa50 msgr2=0x7fc6281f8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:30.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.670+0000 7fc62d200640 1 --2- 172.21.15.39:0/3594503936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6280cfa50 0x7fc6281f8780 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7fc61800c390 tx=0x7fc61800c850 comp rx=0 tx=0).stop 2024-09-21T07:48:30.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.670+0000 7fc62d200640 1 -- 172.21.15.39:0/3594503936 shutdown_connections 2024-09-21T07:48:30.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.670+0000 7fc62d200640 1 --2- 172.21.15.39:0/3594503936 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc608078b90 0x7fc60807b0a0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:30.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.670+0000 7fc62d200640 1 --2- 172.21.15.39:0/3594503936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6280d1200 0x7fc6280b6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:30.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.670+0000 7fc62d200640 1 --2- 172.21.15.39:0/3594503936 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc6280d08b0 0x7fc6280b6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:30.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.670+0000 7fc62d200640 1 --2- 172.21.15.39:0/3594503936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6280cfa50 0x7fc6281f8780 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:30.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.670+0000 7fc62d200640 1 -- 172.21.15.39:0/3594503936 >> 172.21.15.39:0/3594503936 conn(0x7fc6280cef30 msgr2=0x7fc6281e6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:30.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.671+0000 7fc62d200640 1 -- 172.21.15.39:0/3594503936 shutdown_connections 2024-09-21T07:48:30.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.671+0000 7fc62d200640 1 -- 172.21.15.39:0/3594503936 wait complete. 2024-09-21T07:48:30.698 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:48:30.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.923+0000 7feb31c00640 1 -- 172.21.15.39:0/1530609362 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb2c0d0da0 msgr2=0x7feb2c162520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:30.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.923+0000 7feb31c00640 1 --2- 172.21.15.39:0/1530609362 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb2c0d0da0 0x7feb2c162520 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7feb14009860 tx=0x7feb1402a510 comp rx=0 tx=0).stop 2024-09-21T07:48:30.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.923+0000 7feb31c00640 1 -- 172.21.15.39:0/1530609362 shutdown_connections 2024-09-21T07:48:30.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.923+0000 7feb31c00640 1 --2- 172.21.15.39:0/1530609362 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb2c0d0da0 0x7feb2c162520 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:30.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.923+0000 7feb31c00640 1 -- 172.21.15.39:0/1530609362 >> 172.21.15.39:0/1530609362 conn(0x7feb2c06cfc0 msgr2=0x7feb2c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:30.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.924+0000 7feb31c00640 1 -- 172.21.15.39:0/1530609362 shutdown_connections 2024-09-21T07:48:30.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.924+0000 7feb31c00640 1 -- 172.21.15.39:0/1530609362 wait complete. 2024-09-21T07:48:30.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.926+0000 7feb31c00640 1 Processor -- start 2024-09-21T07:48:30.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.926+0000 7feb31c00640 1 -- start start 2024-09-21T07:48:30.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.927+0000 7feb31c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb2c155a90 0x7feb2c142bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:30.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.927+0000 7feb31c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb2c15fc20 0x7feb2c0ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:30.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.927+0000 7feb31c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb2c151c50 0x7feb2c0ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:30.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.927+0000 7feb31c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7feb2c070840 con 0x7feb2c15fc20 2024-09-21T07:48:30.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.927+0000 7feb31c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7feb2c0706c0 con 0x7feb2c155a90 2024-09-21T07:48:30.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.927+0000 7feb31c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7feb2c0709c0 con 0x7feb2c151c50 2024-09-21T07:48:30.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.928+0000 7feb22000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb2c15fc20 0x7feb2c0ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:30.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.928+0000 7feb22000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb2c15fc20 0x7feb2c0ae3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43062/0 (socket says 172.21.15.39:43062) 2024-09-21T07:48:30.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.928+0000 7feb22000640 1 -- 172.21.15.39:0/656861701 learned_addr learned my addr 172.21.15.39:0/656861701 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:30.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.928+0000 7feb2b400640 1 --2- 172.21.15.39:0/656861701 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb2c151c50 0x7feb2c0ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:30.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.928+0000 7feb2aa00640 1 --2- 172.21.15.39:0/656861701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb2c155a90 0x7feb2c142bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:30.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.928+0000 7feb22000640 1 -- 172.21.15.39:0/656861701 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb2c151c50 msgr2=0x7feb2c0ae930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:30.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.928+0000 7feb22000640 1 --2- 172.21.15.39:0/656861701 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb2c151c50 0x7feb2c0ae930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:30.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.928+0000 7feb22000640 1 -- 172.21.15.39:0/656861701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb2c155a90 msgr2=0x7feb2c142bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:30.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.928+0000 7feb22000640 1 --2- 172.21.15.39:0/656861701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb2c155a90 0x7feb2c142bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:30.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.928+0000 7feb22000640 1 -- 172.21.15.39:0/656861701 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb2c1432f0 con 0x7feb2c15fc20 2024-09-21T07:48:30.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.928+0000 7feb2b400640 1 --2- 172.21.15.39:0/656861701 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb2c151c50 0x7feb2c0ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:48:30.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.928+0000 7feb2aa00640 1 --2- 172.21.15.39:0/656861701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb2c155a90 0x7feb2c142bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:48:30.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.928+0000 7feb22000640 1 --2- 172.21.15.39:0/656861701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb2c15fc20 0x7feb2c0ae3f0 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7feb14009980 tx=0x7feb14033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:30.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.929+0000 7feb23400640 1 -- 172.21.15.39:0/656861701 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb14031b50 con 0x7feb2c15fc20 2024-09-21T07:48:30.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.929+0000 7feb23400640 1 -- 172.21.15.39:0/656861701 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb14031de0 con 0x7feb2c15fc20 2024-09-21T07:48:30.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.929+0000 7feb23400640 1 -- 172.21.15.39:0/656861701 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb14005280 con 0x7feb2c15fc20 2024-09-21T07:48:30.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.929+0000 7feb31c00640 1 -- 172.21.15.39:0/656861701 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb2c1435d0 con 0x7feb2c15fc20 2024-09-21T07:48:30.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.929+0000 7feb31c00640 1 -- 172.21.15.39:0/656861701 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb2c0aded0 con 0x7feb2c15fc20 2024-09-21T07:48:30.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.930+0000 7feb31c00640 1 -- 172.21.15.39:0/656861701 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb2c162520 con 0x7feb2c15fc20 2024-09-21T07:48:30.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.931+0000 7feb23400640 1 -- 172.21.15.39:0/656861701 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7feb14005420 con 0x7feb2c15fc20 2024-09-21T07:48:30.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.932+0000 7feb23400640 1 --2- 172.21.15.39:0/656861701 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feb08078b90 0x7feb0807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:30.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.932+0000 7feb23400640 1 -- 172.21.15.39:0/656861701 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(207..207 src has 1..207) ==== 7016+0+0 (secure 0 0 0) 0x7feb140bc6d0 con 0x7feb2c15fc20 2024-09-21T07:48:30.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.933+0000 7feb2aa00640 1 --2- 172.21.15.39:0/656861701 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feb08078b90 0x7feb0807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:30.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.933+0000 7feb2aa00640 1 --2- 172.21.15.39:0/656861701 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feb08078b90 0x7feb0807b0a0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7feb1c000f60 tx=0x7feb1c009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:30.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:30.937+0000 7feb23400640 1 -- 172.21.15.39:0/656861701 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7feb1403f030 con 0x7feb2c15fc20 2024-09-21T07:48:31.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.175+0000 7feb31c00640 1 -- 172.21.15.39:0/656861701 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb2c156340 con 0x7feb2c15fc20 2024-09-21T07:48:31.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.176+0000 7feb23400640 1 -- 172.21.15.39:0/656861701 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 220 v220) ==== 95+0+5260 (secure 0 0 0) 0x7feb140845e0 con 0x7feb2c15fc20 2024-09-21T07:48:31.178 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:48:31.178 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":220,"btime":"2024-09-21T07:48:30:358899+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7988,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1912631346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1912631346},{"type":"v1","addr":"172.21.15.148:6837","nonce":1912631346}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":209},{"gid":9043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2220628944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2220628944},{"type":"v1","addr":"172.21.15.148:6836","nonce":2220628944}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":208}],"filesystems":[{"mdsmap":{"epoch":220,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:47:54.691835+0000","modified":"2024-09-21T07:48:30.162777+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":9852,"mds_1":9049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9049":{"gid":9049,"name":"a","rank":1,"incarnation":217,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6833/1947479913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1947479913},{"type":"v1","addr":"172.21.15.39:6833","nonce":1947479913}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_9852":{"gid":9852,"name":"c","rank":0,"incarnation":210,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/2356847598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2356847598},{"type":"v1","addr":"172.21.15.39:6835","nonce":2356847598}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9852,"qdb_cluster":[9049,9852]},"id":11}]} 2024-09-21T07:48:31.178 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 220 2024-09-21T07:48:31.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.178+0000 7feb31c00640 1 -- 172.21.15.39:0/656861701 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feb08078b90 msgr2=0x7feb0807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:31.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.178+0000 7feb31c00640 1 --2- 172.21.15.39:0/656861701 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feb08078b90 0x7feb0807b0a0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7feb1c000f60 tx=0x7feb1c009040 comp rx=0 tx=0).stop 2024-09-21T07:48:31.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.178+0000 7feb31c00640 1 -- 172.21.15.39:0/656861701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb2c15fc20 msgr2=0x7feb2c0ae3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:31.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.178+0000 7feb31c00640 1 --2- 172.21.15.39:0/656861701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb2c15fc20 0x7feb2c0ae3f0 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7feb14009980 tx=0x7feb14033ba0 comp rx=0 tx=0).stop 2024-09-21T07:48:31.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.178+0000 7feb31c00640 1 -- 172.21.15.39:0/656861701 shutdown_connections 2024-09-21T07:48:31.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.178+0000 7feb31c00640 1 --2- 172.21.15.39:0/656861701 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feb08078b90 0x7feb0807b0a0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:31.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.178+0000 7feb31c00640 1 --2- 172.21.15.39:0/656861701 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb2c151c50 0x7feb2c0ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:31.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.178+0000 7feb31c00640 1 --2- 172.21.15.39:0/656861701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb2c15fc20 0x7feb2c0ae3f0 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:31.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.178+0000 7feb31c00640 1 --2- 172.21.15.39:0/656861701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb2c155a90 0x7feb2c142bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:31.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.178+0000 7feb31c00640 1 -- 172.21.15.39:0/656861701 >> 172.21.15.39:0/656861701 conn(0x7feb2c06cfc0 msgr2=0x7feb2c155130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:31.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.179+0000 7feb31c00640 1 -- 172.21.15.39:0/656861701 shutdown_connections 2024-09-21T07:48:31.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.179+0000 7feb31c00640 1 -- 172.21.15.39:0/656861701 wait complete. 2024-09-21T07:48:31.206 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:48:31.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.432+0000 7f5bec400640 1 -- 172.21.15.39:0/3944127895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5be40b30a0 msgr2=0x7f5be40cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:31.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.432+0000 7f5bec400640 1 --2- 172.21.15.39:0/3944127895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5be40b30a0 0x7f5be40cb8a0 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f5be4075160 tx=0x7f5bd802a4b0 comp rx=0 tx=0).stop 2024-09-21T07:48:31.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.432+0000 7f5bec400640 1 -- 172.21.15.39:0/3944127895 shutdown_connections 2024-09-21T07:48:31.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.432+0000 7f5bec400640 1 --2- 172.21.15.39:0/3944127895 >> v2:172.21.15.148:3300/0 conn(0x7f5be415d830 0x7f5be4156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:31.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.432+0000 7f5bec400640 1 --2- 172.21.15.39:0/3944127895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5be40b30a0 0x7f5be40cb8a0 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:31.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.432+0000 7f5bec400640 1 -- 172.21.15.39:0/3944127895 >> 172.21.15.39:0/3944127895 conn(0x7f5be40b2460 msgr2=0x7f5be406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:31.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.432+0000 7f5bec400640 1 -- 172.21.15.39:0/3944127895 shutdown_connections 2024-09-21T07:48:31.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.432+0000 7f5bec400640 1 -- 172.21.15.39:0/3944127895 wait complete. 2024-09-21T07:48:31.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.435+0000 7f5bec400640 1 Processor -- start 2024-09-21T07:48:31.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.435+0000 7f5bec400640 1 -- start start 2024-09-21T07:48:31.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.436+0000 7f5bec400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5be40b30a0 0x7f5be40cf560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:31.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.436+0000 7f5bec400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5be40ccc60 0x7f5be41f5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:31.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.436+0000 7f5bec400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5be415d830 0x7f5be415b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:31.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.436+0000 7f5bec400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5be40af410 con 0x7f5be40ccc60 2024-09-21T07:48:31.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.436+0000 7f5bec400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5be40af290 con 0x7f5be40b30a0 2024-09-21T07:48:31.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.436+0000 7f5bec400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f5be40af590 con 0x7f5be415d830 2024-09-21T07:48:31.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.436+0000 7f5be9000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5be40ccc60 0x7f5be41f5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:31.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.436+0000 7f5be9000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5be40ccc60 0x7f5be41f5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43084/0 (socket says 172.21.15.39:43084) 2024-09-21T07:48:31.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.437+0000 7f5be9000640 1 -- 172.21.15.39:0/3844442466 learned_addr learned my addr 172.21.15.39:0/3844442466 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:31.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.437+0000 7f5be9000640 1 -- 172.21.15.39:0/3844442466 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5be415d830 msgr2=0x7f5be415b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:31.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.437+0000 7f5be9000640 1 --2- 172.21.15.39:0/3844442466 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5be415d830 0x7f5be415b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:31.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.437+0000 7f5be9a00640 1 --2- 172.21.15.39:0/3844442466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5be40b30a0 0x7f5be40cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:31.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.437+0000 7f5be9000640 1 -- 172.21.15.39:0/3844442466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5be40b30a0 msgr2=0x7f5be40cf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:31.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.437+0000 7f5be9000640 1 --2- 172.21.15.39:0/3844442466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5be40b30a0 0x7f5be40cf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:31.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.437+0000 7f5be9000640 1 -- 172.21.15.39:0/3844442466 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be40cfaa0 con 0x7f5be40ccc60 2024-09-21T07:48:31.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.437+0000 7f5be9a00640 1 --2- 172.21.15.39:0/3844442466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5be40b30a0 0x7f5be40cf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:48:31.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.437+0000 7f5be9000640 1 --2- 172.21.15.39:0/3844442466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5be40ccc60 0x7f5be41f5300 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000d740 tx=0x7f5bd000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:31.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.437+0000 7f5be1600640 1 -- 172.21.15.39:0/3844442466 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0017070 con 0x7f5be40ccc60 2024-09-21T07:48:31.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.437+0000 7f5be1600640 1 -- 172.21.15.39:0/3844442466 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5bd0009d20 con 0x7f5be40ccc60 2024-09-21T07:48:31.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.437+0000 7f5be1600640 1 -- 172.21.15.39:0/3844442466 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd00106c0 con 0x7f5be40ccc60 2024-09-21T07:48:31.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.438+0000 7f5bec400640 1 -- 172.21.15.39:0/3844442466 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5be40ce0b0 con 0x7f5be40ccc60 2024-09-21T07:48:31.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.438+0000 7f5bec400640 1 -- 172.21.15.39:0/3844442466 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5be40ce670 con 0x7f5be40ccc60 2024-09-21T07:48:31.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.439+0000 7f5bec400640 1 -- 172.21.15.39:0/3844442466 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5be40cfd20 con 0x7f5be40ccc60 2024-09-21T07:48:31.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.440+0000 7f5be1600640 1 -- 172.21.15.39:0/3844442466 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5bd001a070 con 0x7f5be40ccc60 2024-09-21T07:48:31.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.441+0000 7f5be1600640 1 --2- 172.21.15.39:0/3844442466 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5bc4078b90 0x7f5bc407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:31.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.441+0000 7f5be1600640 1 -- 172.21.15.39:0/3844442466 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(207..207 src has 1..207) ==== 7016+0+0 (secure 0 0 0) 0x7f5bd009b3b0 con 0x7f5be40ccc60 2024-09-21T07:48:31.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.441+0000 7f5be9a00640 1 --2- 172.21.15.39:0/3844442466 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5bc4078b90 0x7f5bc407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:31.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.442+0000 7f5be9a00640 1 --2- 172.21.15.39:0/3844442466 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5bc4078b90 0x7f5bc407b0a0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8034000 tx=0x7f5bd8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:31.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.445+0000 7f5be1600640 1 -- 172.21.15.39:0/3844442466 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5bd0063350 con 0x7f5be40ccc60 2024-09-21T07:48:31.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.645+0000 7f5bec400640 1 -- 172.21.15.39:0/3844442466 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5be40afa40 con 0x7f5be40ccc60 2024-09-21T07:48:31.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.647+0000 7f5be1600640 1 -- 172.21.15.39:0/3844442466 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v207) ==== 74+0+22509 (secure 0 0 0) 0x7f5bd0062c50 con 0x7f5be40ccc60 2024-09-21T07:48:31.649 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:48:31.649 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":207,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:48:30.358147+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:47:52.623339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"205","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-09-21T07:47:53.638900+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"207","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":207,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]},{"pool":20,"snaps":[{"begin":80,"length":23},{"begin":181,"length":23}]},{"pool":22,"snaps":[{"begin":100,"length":1},{"begin":199,"length":1}]},{"pool":24,"snaps":[{"begin":2,"length":2}]},{"pool":26,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":26,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:48:31.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.649+0000 7f5bec400640 1 -- 172.21.15.39:0/3844442466 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5bc4078b90 msgr2=0x7f5bc407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:31.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.649+0000 7f5bec400640 1 --2- 172.21.15.39:0/3844442466 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5bc4078b90 0x7f5bc407b0a0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8034000 tx=0x7f5bd8004730 comp rx=0 tx=0).stop 2024-09-21T07:48:31.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.649+0000 7f5bec400640 1 -- 172.21.15.39:0/3844442466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5be40ccc60 msgr2=0x7f5be41f5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:31.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.649+0000 7f5bec400640 1 --2- 172.21.15.39:0/3844442466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5be40ccc60 0x7f5be41f5300 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000d740 tx=0x7f5bd000dc00 comp rx=0 tx=0).stop 2024-09-21T07:48:31.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.649+0000 7f5bec400640 1 -- 172.21.15.39:0/3844442466 shutdown_connections 2024-09-21T07:48:31.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.649+0000 7f5bec400640 1 --2- 172.21.15.39:0/3844442466 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5bc4078b90 0x7f5bc407b0a0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:31.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.649+0000 7f5bec400640 1 --2- 172.21.15.39:0/3844442466 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5be415d830 0x7f5be415b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:31.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.649+0000 7f5bec400640 1 --2- 172.21.15.39:0/3844442466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5be40ccc60 0x7f5be41f5300 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:31.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.649+0000 7f5bec400640 1 --2- 172.21.15.39:0/3844442466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5be40b30a0 0x7f5be40cf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:31.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.649+0000 7f5bec400640 1 -- 172.21.15.39:0/3844442466 >> 172.21.15.39:0/3844442466 conn(0x7f5be40b2460 msgr2=0x7f5be40adab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:31.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.649+0000 7f5bec400640 1 -- 172.21.15.39:0/3844442466 shutdown_connections 2024-09-21T07:48:31.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.649+0000 7f5bec400640 1 -- 172.21.15.39:0/3844442466 wait complete. 2024-09-21T07:48:31.674 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T07:48:31.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.902+0000 7f6c16400640 1 -- 172.21.15.39:0/2816338491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c10071020 msgr2=0x7f6c100cd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:31.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.902+0000 7f6c16400640 1 --2- 172.21.15.39:0/2816338491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c10071020 0x7f6c100cd260 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f6c0000a940 tx=0x7f6c0002b850 comp rx=0 tx=0).stop 2024-09-21T07:48:31.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.902+0000 7f6c16400640 1 -- 172.21.15.39:0/2816338491 shutdown_connections 2024-09-21T07:48:31.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.902+0000 7f6c16400640 1 --2- 172.21.15.39:0/2816338491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c10071020 0x7f6c100cd260 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:31.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.902+0000 7f6c16400640 1 -- 172.21.15.39:0/2816338491 >> 172.21.15.39:0/2816338491 conn(0x7f6c100b2460 msgr2=0x7f6c1006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:31.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.902+0000 7f6c16400640 1 -- 172.21.15.39:0/2816338491 shutdown_connections 2024-09-21T07:48:31.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.902+0000 7f6c16400640 1 -- 172.21.15.39:0/2816338491 wait complete. 2024-09-21T07:48:31.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.904+0000 7f6c16400640 1 Processor -- start 2024-09-21T07:48:31.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.904+0000 7f6c16400640 1 -- start start 2024-09-21T07:48:31.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.906+0000 7f6c16400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c10071020 0x7f6c100aed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:31.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.906+0000 7f6c16400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6c100af430 0x7f6c101f2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:31.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.906+0000 7f6c16400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c100cfb60 0x7f6c101e3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:31.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.906+0000 7f6c16400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6c100b47c0 con 0x7f6c100cfb60 2024-09-21T07:48:31.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.906+0000 7f6c16400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1006ffa0 con 0x7f6c10071020 2024-09-21T07:48:31.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.906+0000 7f6c16400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f6c100b4940 con 0x7f6c100af430 2024-09-21T07:48:31.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.906+0000 7f6c0fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c100cfb60 0x7f6c101e3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:31.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.906+0000 7f6c0fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c100cfb60 0x7f6c101e3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43092/0 (socket says 172.21.15.39:43092) 2024-09-21T07:48:31.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.906+0000 7f6c0f400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c10071020 0x7f6c100aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:31.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.906+0000 7f6c0fe00640 1 -- 172.21.15.39:0/1220305127 learned_addr learned my addr 172.21.15.39:0/1220305127 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:31.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.906+0000 7f6c0ea00640 1 --2- 172.21.15.39:0/1220305127 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6c100af430 0x7f6c101f2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:31.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.907+0000 7f6c0fe00640 1 -- 172.21.15.39:0/1220305127 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6c100af430 msgr2=0x7f6c101f2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:31.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.907+0000 7f6c0fe00640 1 --2- 172.21.15.39:0/1220305127 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6c100af430 0x7f6c101f2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:31.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.907+0000 7f6c0fe00640 1 -- 172.21.15.39:0/1220305127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c10071020 msgr2=0x7f6c100aed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:31.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.907+0000 7f6c0fe00640 1 --2- 172.21.15.39:0/1220305127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c10071020 0x7f6c100aed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:31.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.907+0000 7f6c0fe00640 1 -- 172.21.15.39:0/1220305127 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c10097be0 con 0x7f6c100cfb60 2024-09-21T07:48:31.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.907+0000 7f6c0fe00640 1 --2- 172.21.15.39:0/1220305127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c100cfb60 0x7f6c101e3ff0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f6c0800c6e0 tx=0x7f6c0800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:31.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.907+0000 7f6c07400640 1 -- 172.21.15.39:0/1220305127 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c08003b70 con 0x7f6c100cfb60 2024-09-21T07:48:31.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.907+0000 7f6c07400640 1 -- 172.21.15.39:0/1220305127 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c08003d10 con 0x7f6c100cfb60 2024-09-21T07:48:31.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.907+0000 7f6c07400640 1 -- 172.21.15.39:0/1220305127 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c08005330 con 0x7f6c100cfb60 2024-09-21T07:48:31.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.907+0000 7f6c16400640 1 -- 172.21.15.39:0/1220305127 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c101f7400 con 0x7f6c100cfb60 2024-09-21T07:48:31.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.907+0000 7f6c16400640 1 -- 172.21.15.39:0/1220305127 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c101f79c0 con 0x7f6c100cfb60 2024-09-21T07:48:31.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.909+0000 7f6c16400640 1 -- 172.21.15.39:0/1220305127 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c100b4df0 con 0x7f6c100cfb60 2024-09-21T07:48:31.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.910+0000 7f6c07400640 1 -- 172.21.15.39:0/1220305127 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6c080054d0 con 0x7f6c100cfb60 2024-09-21T07:48:31.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.911+0000 7f6c07400640 1 --2- 172.21.15.39:0/1220305127 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6bf0078b90 0x7f6bf007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:31.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.911+0000 7f6c07400640 1 -- 172.21.15.39:0/1220305127 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(207..207 src has 1..207) ==== 7016+0+0 (secure 0 0 0) 0x7f6c0809adf0 con 0x7f6c100cfb60 2024-09-21T07:48:31.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.911+0000 7f6c0f400640 1 --2- 172.21.15.39:0/1220305127 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6bf0078b90 0x7f6bf007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:31.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.912+0000 7f6c0f400640 1 --2- 172.21.15.39:0/1220305127 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6bf0078b90 0x7f6bf007b0a0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f6c00036000 tx=0x7f6c00004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:31.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:31.916+0000 7f6c07400640 1 -- 172.21.15.39:0/1220305127 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6c08062760 con 0x7f6c100cfb60 2024-09-21T07:48:32.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.153+0000 7f6c16400640 1 -- 172.21.15.39:0/1220305127 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f6c101f7190 con 0x7f6c100cfb60 2024-09-21T07:48:32.155 INFO:tasks.ceph.mon.a.smithi039.stderr:2024-09-21T07:48:32.154+0000 7fabab600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T07:48:32.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.156+0000 7f6c07400640 1 -- 172.21.15.39:0/1220305127 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v221) ==== 161+0+0 (secure 0 0 0) 0x7f6c08067190 con 0x7f6c100cfb60 2024-09-21T07:48:32.158 INFO:teuthology.orchestra.run.smithi039.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T07:48:32.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.160+0000 7f6c16400640 1 -- 172.21.15.39:0/1220305127 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6bf0078b90 msgr2=0x7f6bf007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:32.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.160+0000 7f6c16400640 1 --2- 172.21.15.39:0/1220305127 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6bf0078b90 0x7f6bf007b0a0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f6c00036000 tx=0x7f6c00004730 comp rx=0 tx=0).stop 2024-09-21T07:48:32.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.160+0000 7f6c16400640 1 -- 172.21.15.39:0/1220305127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c100cfb60 msgr2=0x7f6c101e3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:32.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.160+0000 7f6c16400640 1 --2- 172.21.15.39:0/1220305127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c100cfb60 0x7f6c101e3ff0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f6c0800c6e0 tx=0x7f6c0800cba0 comp rx=0 tx=0).stop 2024-09-21T07:48:32.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.160+0000 7f6c16400640 1 -- 172.21.15.39:0/1220305127 shutdown_connections 2024-09-21T07:48:32.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.160+0000 7f6c16400640 1 --2- 172.21.15.39:0/1220305127 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6bf0078b90 0x7f6bf007b0a0 secure :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f6c00036000 tx=0x7f6c00004730 comp rx=0 tx=0).stop 2024-09-21T07:48:32.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.160+0000 7f6c16400640 1 --2- 172.21.15.39:0/1220305127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c100cfb60 0x7f6c101e3ff0 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:32.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.160+0000 7f6c16400640 1 --2- 172.21.15.39:0/1220305127 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6c100af430 0x7f6c101f2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:32.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.160+0000 7f6c16400640 1 --2- 172.21.15.39:0/1220305127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c10071020 0x7f6c100aed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:32.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.160+0000 7f6c16400640 1 -- 172.21.15.39:0/1220305127 >> 172.21.15.39:0/1220305127 conn(0x7f6c100b2460 msgr2=0x7f6c1006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:32.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.161+0000 7f6c16400640 1 -- 172.21.15.39:0/1220305127 shutdown_connections 2024-09-21T07:48:32.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.161+0000 7f6c16400640 1 -- 172.21.15.39:0/1220305127 wait complete. 2024-09-21T07:48:32.173 INFO:tasks.ceph.mds.c.smithi039.stderr: -8935> 2024-09-21T07:47:57.124+0000 7f44f1a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:48:32.191 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T07:48:32.304 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-21T07:48:32.304 INFO:tasks.ceph.mds.c.smithi039.stdout:starting mds.c at 2024-09-21T07:48:32.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.433+0000 7fc74ec00640 1 -- 172.21.15.39:0/1319974647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc72c0039e0 msgr2=0x7fc72c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:32.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.433+0000 7fc74ec00640 1 --2- 172.21.15.39:0/1319974647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc72c0039e0 0x7fc72c003e10 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7fc73800c3c0 tx=0x7fc72c005830 comp rx=0 tx=0).stop 2024-09-21T07:48:32.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.433+0000 7fc74cc00640 1 -- 172.21.15.39:0/1319974647 reap_dead start 2024-09-21T07:48:32.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.433+0000 7fc74ec00640 1 -- 172.21.15.39:0/1319974647 shutdown_connections 2024-09-21T07:48:32.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.433+0000 7fc74ec00640 1 -- 172.21.15.39:0/1319974647 >> 172.21.15.39:0/1319974647 conn(0x7fc7480b2460 msgr2=0x7fc74806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:32.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.434+0000 7fc74ec00640 1 -- 172.21.15.39:0/1319974647 shutdown_connections 2024-09-21T07:48:32.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.434+0000 7fc74ec00640 1 -- 172.21.15.39:0/1319974647 wait complete. 2024-09-21T07:48:32.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.436+0000 7fc74ec00640 1 Processor -- start 2024-09-21T07:48:32.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.436+0000 7fc74ec00640 1 -- start start 2024-09-21T07:48:32.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.437+0000 7fc74ec00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc72c0039e0 0x7fc748154970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:32.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.437+0000 7fc74ec00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7481550b0 0x7fc7480c8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:32.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.437+0000 7fc74ec00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc7480c3b50 0x7fc74815f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:32.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.437+0000 7fc74ec00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc7480af080 con 0x7fc72c0039e0 2024-09-21T07:48:32.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.437+0000 7fc74ec00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc7480cd3c0 con 0x7fc7481550b0 2024-09-21T07:48:32.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.437+0000 7fc74ec00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc7480af200 con 0x7fc7480c3b50 2024-09-21T07:48:32.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.438+0000 7fc747e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc72c0039e0 0x7fc748154970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:32.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.438+0000 7fc747e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc72c0039e0 0x7fc748154970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60074/0 (socket says 172.21.15.39:60074) 2024-09-21T07:48:32.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.438+0000 7fc747e00640 1 -- 172.21.15.39:0/2599140649 learned_addr learned my addr 172.21.15.39:0/2599140649 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:32.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.438+0000 7fc747400640 1 --2- 172.21.15.39:0/2599140649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7481550b0 0x7fc7480c8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:32.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.438+0000 7fc74cc00640 1 --2- 172.21.15.39:0/2599140649 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc7480c3b50 0x7fc74815f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:32.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.438+0000 7fc747e00640 1 -- 172.21.15.39:0/2599140649 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc7480c3b50 msgr2=0x7fc74815f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:32.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.438+0000 7fc747e00640 1 --2- 172.21.15.39:0/2599140649 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc7480c3b50 0x7fc74815f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:32.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.438+0000 7fc747e00640 1 -- 172.21.15.39:0/2599140649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7481550b0 msgr2=0x7fc7480c8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:32.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.438+0000 7fc747e00640 1 --2- 172.21.15.39:0/2599140649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7481550b0 0x7fc7480c8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:32.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.438+0000 7fc747e00640 1 -- 172.21.15.39:0/2599140649 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc748154eb0 con 0x7fc72c0039e0 2024-09-21T07:48:32.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.438+0000 7fc74cc00640 1 --2- 172.21.15.39:0/2599140649 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc7480c3b50 0x7fc74815f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:48:32.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.438+0000 7fc747e00640 1 --2- 172.21.15.39:0/2599140649 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc72c0039e0 0x7fc748154970 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7fc73800c4b0 tx=0x7fc738006060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:32.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.439+0000 7fc73fe00640 1 -- 172.21.15.39:0/2599140649 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc738042070 con 0x7fc72c0039e0 2024-09-21T07:48:32.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.439+0000 7fc73fe00640 1 -- 172.21.15.39:0/2599140649 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7380035f0 con 0x7fc72c0039e0 2024-09-21T07:48:32.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.439+0000 7fc73fe00640 1 -- 172.21.15.39:0/2599140649 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc738039070 con 0x7fc72c0039e0 2024-09-21T07:48:32.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.439+0000 7fc74ec00640 1 -- 172.21.15.39:0/2599140649 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7480c9cc0 con 0x7fc72c0039e0 2024-09-21T07:48:32.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.439+0000 7fc74ec00640 1 -- 172.21.15.39:0/2599140649 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7480c85a0 con 0x7fc72c0039e0 2024-09-21T07:48:32.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.440+0000 7fc74ec00640 1 -- 172.21.15.39:0/2599140649 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7480af680 con 0x7fc72c0039e0 2024-09-21T07:48:32.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.441+0000 7fc73fe00640 1 -- 172.21.15.39:0/2599140649 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc73802b5b0 con 0x7fc72c0039e0 2024-09-21T07:48:32.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.442+0000 7fc73fe00640 1 --2- 172.21.15.39:0/2599140649 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc728078bc0 0x7fc72807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:32.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.442+0000 7fc73fe00640 1 -- 172.21.15.39:0/2599140649 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(208..208 src has 1..208) ==== 7160+0+0 (secure 0 0 0) 0x7fc7380bab00 con 0x7fc72c0039e0 2024-09-21T07:48:32.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.443+0000 7fc747400640 1 --2- 172.21.15.39:0/2599140649 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc728078bc0 0x7fc72807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:32.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.443+0000 7fc747400640 1 --2- 172.21.15.39:0/2599140649 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc728078bc0 0x7fc72807b0d0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fc7480511a0 tx=0x7fc734006be0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:32.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.447+0000 7fc73fe00640 1 -- 172.21.15.39:0/2599140649 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc748154eb0 con 0x7fc72c0039e0 2024-09-21T07:48:32.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:32.683+0000 7fc74ec00640 1 -- 172.21.15.39:0/2599140649 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc74815c960 con 0x7fc72c0039e0 2024-09-21T07:48:32.995 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:48:32.999 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:48:33.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.159+0000 7fc73fe00640 1 -- 172.21.15.39:0/2599140649 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v222) ==== 104+0+0 (secure 0 0 0) 0x7fc7480cd5b0 con 0x7fc72c0039e0 2024-09-21T07:48:33.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.161+0000 7fc74ec00640 1 -- 172.21.15.39:0/2599140649 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc728078bc0 msgr2=0x7fc72807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:33.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.161+0000 7fc74ec00640 1 --2- 172.21.15.39:0/2599140649 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc728078bc0 0x7fc72807b0d0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fc7480511a0 tx=0x7fc734006be0 comp rx=0 tx=0).stop 2024-09-21T07:48:33.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.161+0000 7fc74ec00640 1 -- 172.21.15.39:0/2599140649 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc72c0039e0 msgr2=0x7fc748154970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:33.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.161+0000 7fc74ec00640 1 --2- 172.21.15.39:0/2599140649 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc72c0039e0 0x7fc748154970 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7fc73800c4b0 tx=0x7fc738006060 comp rx=0 tx=0).stop 2024-09-21T07:48:33.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.161+0000 7fc74ec00640 1 -- 172.21.15.39:0/2599140649 shutdown_connections 2024-09-21T07:48:33.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.161+0000 7fc74ec00640 1 --2- 172.21.15.39:0/2599140649 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc728078bc0 0x7fc72807b0d0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:33.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.161+0000 7fc74ec00640 1 --2- 172.21.15.39:0/2599140649 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc7480c3b50 0x7fc74815f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:33.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.161+0000 7fc74ec00640 1 --2- 172.21.15.39:0/2599140649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc7481550b0 0x7fc7480c8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:33.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.161+0000 7fc74ec00640 1 --2- 172.21.15.39:0/2599140649 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc72c0039e0 0x7fc748154970 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:33.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.161+0000 7fc74ec00640 1 -- 172.21.15.39:0/2599140649 >> 172.21.15.39:0/2599140649 conn(0x7fc7480b2460 msgr2=0x7fc7480af890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:33.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.162+0000 7fc74ec00640 1 -- 172.21.15.39:0/2599140649 shutdown_connections 2024-09-21T07:48:33.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.162+0000 7fc74ec00640 1 -- 172.21.15.39:0/2599140649 wait complete. 2024-09-21T07:48:33.189 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T07:48:33.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.419+0000 7f71df800640 1 -- 172.21.15.39:0/3924547091 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71c0003c30 msgr2=0x7f71c0004080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:33.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.419+0000 7f71df800640 1 --2- 172.21.15.39:0/3924547091 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71c0003c30 0x7f71c0004080 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f71cc001830 tx=0x7f71c0005830 comp rx=0 tx=0).stop 2024-09-21T07:48:33.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.419+0000 7f71df800640 1 -- 172.21.15.39:0/3924547091 shutdown_connections 2024-09-21T07:48:33.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.419+0000 7f71df800640 1 --2- 172.21.15.39:0/3924547091 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71c0003c30 0x7f71c0004080 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:33.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.419+0000 7f71df800640 1 -- 172.21.15.39:0/3924547091 >> 172.21.15.39:0/3924547091 conn(0x7f71d80b2650 msgr2=0x7f71d80d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:33.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.420+0000 7f71df800640 1 -- 172.21.15.39:0/3924547091 shutdown_connections 2024-09-21T07:48:33.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.420+0000 7f71df800640 1 -- 172.21.15.39:0/3924547091 wait complete. 2024-09-21T07:48:33.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.422+0000 7f71df800640 1 Processor -- start 2024-09-21T07:48:33.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.422+0000 7f71df800640 1 -- start start 2024-09-21T07:48:33.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.423+0000 7f71df800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71c0003c30 0x7f71d81f6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:33.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.423+0000 7f71df800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f71d80cfa10 0x7f71d80ae1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:33.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.423+0000 7f71df800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71d80ae700 0x7f71d8073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:33.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.423+0000 7f71df800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f71d80cbb80 con 0x7f71d80ae700 2024-09-21T07:48:33.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.423+0000 7f71df800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f71d80cba00 con 0x7f71c0003c30 2024-09-21T07:48:33.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.423+0000 7f71df800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f71d80cbd00 con 0x7f71d80cfa10 2024-09-21T07:48:33.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.424+0000 7f71dd800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71d80ae700 0x7f71d8073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:33.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.424+0000 7f71dd800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71d80ae700 0x7f71d8073b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60088/0 (socket says 172.21.15.39:60088) 2024-09-21T07:48:33.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.424+0000 7f71dd800640 1 -- 172.21.15.39:0/3088860403 learned_addr learned my addr 172.21.15.39:0/3088860403 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:33.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.424+0000 7f71dce00640 1 --2- 172.21.15.39:0/3088860403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71c0003c30 0x7f71d81f6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:33.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.424+0000 7f71d7e00640 1 --2- 172.21.15.39:0/3088860403 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f71d80cfa10 0x7f71d80ae1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:33.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.424+0000 7f71dd800640 1 -- 172.21.15.39:0/3088860403 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f71d80cfa10 msgr2=0x7f71d80ae1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:33.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.424+0000 7f71dd800640 1 --2- 172.21.15.39:0/3088860403 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f71d80cfa10 0x7f71d80ae1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:33.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.424+0000 7f71dd800640 1 -- 172.21.15.39:0/3088860403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71c0003c30 msgr2=0x7f71d81f6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:33.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.424+0000 7f71dd800640 1 --2- 172.21.15.39:0/3088860403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71c0003c30 0x7f71d81f6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:33.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.424+0000 7f71dd800640 1 -- 172.21.15.39:0/3088860403 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d81f67d0 con 0x7f71d80ae700 2024-09-21T07:48:33.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.424+0000 7f71d7e00640 1 --2- 172.21.15.39:0/3088860403 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f71d80cfa10 0x7f71d80ae1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:48:33.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.425+0000 7f71dd800640 1 --2- 172.21.15.39:0/3088860403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71d80ae700 0x7f71d8073b40 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f71c00057e0 tx=0x7f71d0006710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:33.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.425+0000 7f71d4c00640 1 -- 172.21.15.39:0/3088860403 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d000e7b0 con 0x7f71d80ae700 2024-09-21T07:48:33.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.425+0000 7f71d4c00640 1 -- 172.21.15.39:0/3088860403 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71d000ea40 con 0x7f71d80ae700 2024-09-21T07:48:33.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.425+0000 7f71df800640 1 -- 172.21.15.39:0/3088860403 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71d80aeb10 con 0x7f71d80ae700 2024-09-21T07:48:33.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.425+0000 7f71d4c00640 1 -- 172.21.15.39:0/3088860403 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d000ee90 con 0x7f71d80ae700 2024-09-21T07:48:33.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.425+0000 7f71df800640 1 -- 172.21.15.39:0/3088860403 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71d8073060 con 0x7f71d80ae700 2024-09-21T07:48:33.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.426+0000 7f71df800640 1 -- 172.21.15.39:0/3088860403 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71d8160410 con 0x7f71d80ae700 2024-09-21T07:48:33.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.427+0000 7f71d4c00640 1 -- 172.21.15.39:0/3088860403 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f71d0019020 con 0x7f71d80ae700 2024-09-21T07:48:33.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.428+0000 7f71d4c00640 1 --2- 172.21.15.39:0/3088860403 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f71b8078d30 0x7f71b807b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:33.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.428+0000 7f71d4c00640 1 -- 172.21.15.39:0/3088860403 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(208..208 src has 1..208) ==== 7160+0+0 (secure 0 0 0) 0x7f71d009b360 con 0x7f71d80ae700 2024-09-21T07:48:33.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.429+0000 7f71dce00640 1 --2- 172.21.15.39:0/3088860403 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f71b8078d30 0x7f71b807b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:33.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.430+0000 7f71dce00640 1 --2- 172.21.15.39:0/3088860403 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f71b8078d30 0x7f71b807b240 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f71cc032000 tx=0x7f71cc001360 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:33.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.433+0000 7f71d4c00640 1 -- 172.21.15.39:0/3088860403 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f71d0020020 con 0x7f71d80ae700 2024-09-21T07:48:33.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:33.632+0000 7f71df800640 1 -- 172.21.15.39:0/3088860403 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f71d80cd880 con 0x7f71d80ae700 2024-09-21T07:48:34.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.392+0000 7f71d4c00640 1 -- 172.21.15.39:0/3088860403 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v209) ==== 181+0+0 (secure 0 0 0) 0x7f71d80aeb10 con 0x7f71d80ae700 2024-09-21T07:48:34.395 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' removed 2024-09-21T07:48:34.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.397+0000 7f71df800640 1 -- 172.21.15.39:0/3088860403 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f71b8078d30 msgr2=0x7f71b807b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:34.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.397+0000 7f71df800640 1 --2- 172.21.15.39:0/3088860403 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f71b8078d30 0x7f71b807b240 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f71cc032000 tx=0x7f71cc001360 comp rx=0 tx=0).stop 2024-09-21T07:48:34.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.397+0000 7f71df800640 1 -- 172.21.15.39:0/3088860403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71d80ae700 msgr2=0x7f71d8073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:34.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.397+0000 7f71df800640 1 --2- 172.21.15.39:0/3088860403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71d80ae700 0x7f71d8073b40 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f71c00057e0 tx=0x7f71d0006710 comp rx=0 tx=0).stop 2024-09-21T07:48:34.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.397+0000 7f71df800640 1 -- 172.21.15.39:0/3088860403 shutdown_connections 2024-09-21T07:48:34.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.397+0000 7f71df800640 1 --2- 172.21.15.39:0/3088860403 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f71b8078d30 0x7f71b807b240 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:34.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.398+0000 7f71df800640 1 --2- 172.21.15.39:0/3088860403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71d80ae700 0x7f71d8073b40 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:34.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.398+0000 7f71df800640 1 --2- 172.21.15.39:0/3088860403 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f71d80cfa10 0x7f71d80ae1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:34.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.398+0000 7f71df800640 1 --2- 172.21.15.39:0/3088860403 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71c0003c30 0x7f71d81f6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:34.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.398+0000 7f71df800640 1 -- 172.21.15.39:0/3088860403 >> 172.21.15.39:0/3088860403 conn(0x7f71d80b2650 msgr2=0x7f71d806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:34.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.398+0000 7f71df800640 1 -- 172.21.15.39:0/3088860403 shutdown_connections 2024-09-21T07:48:34.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.398+0000 7f71df800640 1 -- 172.21.15.39:0/3088860403 wait complete. 2024-09-21T07:48:34.428 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T07:48:34.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.661+0000 7fdff3200640 1 -- 172.21.15.39:0/1403373931 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdff415d440 msgr2=0x7fdff415d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:34.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.661+0000 7fdff3200640 1 --2- 172.21.15.39:0/1403373931 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdff415d440 0x7fdff415d850 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fdfec009be0 tx=0x7fdfec02a920 comp rx=0 tx=0).stop 2024-09-21T07:48:34.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.661+0000 7fdff3200640 1 -- 172.21.15.39:0/1403373931 shutdown_connections 2024-09-21T07:48:34.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.661+0000 7fdff3200640 1 --2- 172.21.15.39:0/1403373931 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdff415d440 0x7fdff415d850 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:34.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.661+0000 7fdff3200640 1 --2- 172.21.15.39:0/1403373931 >> v2:172.21.15.148:3300/0 conn(0x7fdff41539f0 0x7fdff40b4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:34.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.661+0000 7fdff3200640 1 -- 172.21.15.39:0/1403373931 >> 172.21.15.39:0/1403373931 conn(0x7fdff406d660 msgr2=0x7fdff40b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:34.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.661+0000 7fdff3200640 1 -- 172.21.15.39:0/1403373931 shutdown_connections 2024-09-21T07:48:34.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.661+0000 7fdff3200640 1 -- 172.21.15.39:0/1403373931 wait complete. 2024-09-21T07:48:34.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.663+0000 7fdff3200640 1 Processor -- start 2024-09-21T07:48:34.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.663+0000 7fdff3200640 1 -- start start 2024-09-21T07:48:34.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.665+0000 7fdff3200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdff41539f0 0x7fdff41f6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:34.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.665+0000 7fdff3200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdff415d440 0x7fdff41f9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:34.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.665+0000 7fdff3200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdff4160340 0x7fdff40cf810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:34.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.665+0000 7fdff3200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdff40ad670 con 0x7fdff4160340 2024-09-21T07:48:34.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.665+0000 7fdff3200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdff40ad4f0 con 0x7fdff415d440 2024-09-21T07:48:34.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.665+0000 7fdff3200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fdff40ad7f0 con 0x7fdff41539f0 2024-09-21T07:48:34.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.665+0000 7fdff2800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdff4160340 0x7fdff40cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:34.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.665+0000 7fdff2800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdff4160340 0x7fdff40cf810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60106/0 (socket says 172.21.15.39:60106) 2024-09-21T07:48:34.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.665+0000 7fdff2800640 1 -- 172.21.15.39:0/2060049929 learned_addr learned my addr 172.21.15.39:0/2060049929 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:34.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.665+0000 7fdff1e00640 1 --2- 172.21.15.39:0/2060049929 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdff41539f0 0x7fdff41f6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:34.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.665+0000 7fdfebe00640 1 --2- 172.21.15.39:0/2060049929 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdff415d440 0x7fdff41f9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:34.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.665+0000 7fdff2800640 1 -- 172.21.15.39:0/2060049929 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdff41539f0 msgr2=0x7fdff41f6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:34.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.665+0000 7fdff2800640 1 --2- 172.21.15.39:0/2060049929 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdff41539f0 0x7fdff41f6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:34.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.665+0000 7fdff2800640 1 -- 172.21.15.39:0/2060049929 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdff415d440 msgr2=0x7fdff41f9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:34.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.665+0000 7fdff2800640 1 --2- 172.21.15.39:0/2060049929 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdff415d440 0x7fdff41f9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:34.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.665+0000 7fdff2800640 1 -- 172.21.15.39:0/2060049929 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdff41ec950 con 0x7fdff4160340 2024-09-21T07:48:34.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.666+0000 7fdff1e00640 1 --2- 172.21.15.39:0/2060049929 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdff41539f0 0x7fdff41f6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:48:34.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.666+0000 7fdff2800640 1 --2- 172.21.15.39:0/2060049929 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdff4160340 0x7fdff40cf810 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fdfec009d00 tx=0x7fdfec032e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:34.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.666+0000 7fdfea000640 1 -- 172.21.15.39:0/2060049929 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfec03f030 con 0x7fdff4160340 2024-09-21T07:48:34.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.666+0000 7fdfea000640 1 -- 172.21.15.39:0/2060049929 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfec004250 con 0x7fdff4160340 2024-09-21T07:48:34.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.666+0000 7fdfea000640 1 -- 172.21.15.39:0/2060049929 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfec0046a0 con 0x7fdff4160340 2024-09-21T07:48:34.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.666+0000 7fdff3200640 1 -- 172.21.15.39:0/2060049929 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdff41f69a0 con 0x7fdff4160340 2024-09-21T07:48:34.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.666+0000 7fdff3200640 1 -- 172.21.15.39:0/2060049929 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdff41f5c80 con 0x7fdff4160340 2024-09-21T07:48:34.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.667+0000 7fdff3200640 1 -- 172.21.15.39:0/2060049929 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdff41f97a0 con 0x7fdff4160340 2024-09-21T07:48:34.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.669+0000 7fdfea000640 1 -- 172.21.15.39:0/2060049929 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdfec042030 con 0x7fdff4160340 2024-09-21T07:48:34.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.669+0000 7fdfea000640 1 --2- 172.21.15.39:0/2060049929 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdfd0078b90 0x7fdfd007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:34.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.670+0000 7fdfea000640 1 -- 172.21.15.39:0/2060049929 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(209..209 src has 1..209) ==== 6722+0+0 (secure 0 0 0) 0x7fdfec0bc2a0 con 0x7fdff4160340 2024-09-21T07:48:34.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.670+0000 7fdff1e00640 1 --2- 172.21.15.39:0/2060049929 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdfd0078b90 0x7fdfd007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:34.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.671+0000 7fdff1e00640 1 --2- 172.21.15.39:0/2060049929 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdfd0078b90 0x7fdfd007b0a0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4005fd0 tx=0x7fdfe400f360 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:34.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.674+0000 7fdfea000640 1 -- 172.21.15.39:0/2060049929 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdff41f97a0 con 0x7fdff4160340 2024-09-21T07:48:34.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:34.878+0000 7fdff3200640 1 -- 172.21.15.39:0/2060049929 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fdff40b4c40 con 0x7fdff4160340 2024-09-21T07:48:35.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.395+0000 7fdfea000640 1 -- 172.21.15.39:0/2060049929 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v210) ==== 169+0+0 (secure 0 0 0) 0x7fdfec084240 con 0x7fdff4160340 2024-09-21T07:48:35.397 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' removed 2024-09-21T07:48:35.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.397+0000 7fdff3200640 1 -- 172.21.15.39:0/2060049929 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdfd0078b90 msgr2=0x7fdfd007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:35.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.397+0000 7fdff3200640 1 --2- 172.21.15.39:0/2060049929 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdfd0078b90 0x7fdfd007b0a0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4005fd0 tx=0x7fdfe400f360 comp rx=0 tx=0).stop 2024-09-21T07:48:35.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.397+0000 7fdff3200640 1 -- 172.21.15.39:0/2060049929 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdff4160340 msgr2=0x7fdff40cf810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:35.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.397+0000 7fdff3200640 1 --2- 172.21.15.39:0/2060049929 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdff4160340 0x7fdff40cf810 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fdfec009d00 tx=0x7fdfec032e50 comp rx=0 tx=0).stop 2024-09-21T07:48:35.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.397+0000 7fdff3200640 1 -- 172.21.15.39:0/2060049929 shutdown_connections 2024-09-21T07:48:35.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.397+0000 7fdff3200640 1 --2- 172.21.15.39:0/2060049929 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdfd0078b90 0x7fdfd007b0a0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:35.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.397+0000 7fdff3200640 1 --2- 172.21.15.39:0/2060049929 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdff4160340 0x7fdff40cf810 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:35.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.397+0000 7fdff3200640 1 --2- 172.21.15.39:0/2060049929 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdff415d440 0x7fdff41f9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:35.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.397+0000 7fdff3200640 1 --2- 172.21.15.39:0/2060049929 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdff41539f0 0x7fdff41f6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:35.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.397+0000 7fdff3200640 1 -- 172.21.15.39:0/2060049929 >> 172.21.15.39:0/2060049929 conn(0x7fdff406d660 msgr2=0x7fdff415ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:35.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.397+0000 7fdff3200640 1 -- 172.21.15.39:0/2060049929 shutdown_connections 2024-09-21T07:48:35.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.398+0000 7fdff3200640 1 -- 172.21.15.39:0/2060049929 wait complete. 2024-09-21T07:48:35.426 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T07:48:35.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.655+0000 7f35f2000640 1 -- 172.21.15.39:0/1306132527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35ec0ccdb0 msgr2=0x7f35ec0cd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:35.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.655+0000 7f35f2000640 1 --2- 172.21.15.39:0/1306132527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35ec0ccdb0 0x7f35ec0cd1c0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f35ec08cb10 tx=0x7f35d802a560 comp rx=0 tx=0).stop 2024-09-21T07:48:35.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.655+0000 7f35f2000640 1 -- 172.21.15.39:0/1306132527 shutdown_connections 2024-09-21T07:48:35.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.655+0000 7f35f2000640 1 --2- 172.21.15.39:0/1306132527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35ec0ccdb0 0x7f35ec0cd1c0 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:35.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.655+0000 7f35f2000640 1 --2- 172.21.15.39:0/1306132527 >> v2:172.21.15.148:3300/0 conn(0x7f35ec0b50f0 0x7f35ec15fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:35.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.655+0000 7f35f2000640 1 -- 172.21.15.39:0/1306132527 >> 172.21.15.39:0/1306132527 conn(0x7f35ec09b5f0 msgr2=0x7f35ec08c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:35.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.655+0000 7f35f2000640 1 -- 172.21.15.39:0/1306132527 shutdown_connections 2024-09-21T07:48:35.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.655+0000 7f35f2000640 1 -- 172.21.15.39:0/1306132527 wait complete. 2024-09-21T07:48:35.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.657+0000 7f35f2000640 1 Processor -- start 2024-09-21T07:48:35.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.657+0000 7f35f2000640 1 -- start start 2024-09-21T07:48:35.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.659+0000 7f35f2000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35ec0b50f0 0x7f35ec14fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:35.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.659+0000 7f35f2000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35ec0ccdb0 0x7f35ec06f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:35.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.659+0000 7f35f2000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35ec0d1ec0 0x7f35ec1f8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:35.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.659+0000 7f35f2000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f35ec0b3040 con 0x7f35ec0d1ec0 2024-09-21T07:48:35.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.659+0000 7f35f2000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f35ec0b2ec0 con 0x7f35ec0b50f0 2024-09-21T07:48:35.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.659+0000 7f35f2000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f35ec0b31c0 con 0x7f35ec0ccdb0 2024-09-21T07:48:35.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.659+0000 7f35f1600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35ec0d1ec0 0x7f35ec1f8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:35.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.659+0000 7f35f1600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35ec0d1ec0 0x7f35ec1f8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60124/0 (socket says 172.21.15.39:60124) 2024-09-21T07:48:35.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.659+0000 7f35f1600640 1 -- 172.21.15.39:0/2967902491 learned_addr learned my addr 172.21.15.39:0/2967902491 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:35.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.659+0000 7f35ebe00640 1 --2- 172.21.15.39:0/2967902491 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35ec0ccdb0 0x7f35ec06f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:35.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.659+0000 7f35f0c00640 1 --2- 172.21.15.39:0/2967902491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35ec0b50f0 0x7f35ec14fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:35.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.659+0000 7f35f1600640 1 -- 172.21.15.39:0/2967902491 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35ec0ccdb0 msgr2=0x7f35ec06f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:35.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.659+0000 7f35f1600640 1 --2- 172.21.15.39:0/2967902491 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35ec0ccdb0 0x7f35ec06f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:35.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.659+0000 7f35f1600640 1 -- 172.21.15.39:0/2967902491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35ec0b50f0 msgr2=0x7f35ec14fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:35.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.659+0000 7f35f1600640 1 --2- 172.21.15.39:0/2967902491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35ec0b50f0 0x7f35ec14fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:35.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.660+0000 7f35f1600640 1 -- 172.21.15.39:0/2967902491 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35ec1f9900 con 0x7f35ec0d1ec0 2024-09-21T07:48:35.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.660+0000 7f35ebe00640 1 --2- 172.21.15.39:0/2967902491 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35ec0ccdb0 0x7f35ec06f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:48:35.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.660+0000 7f35f1600640 1 --2- 172.21.15.39:0/2967902491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35ec0d1ec0 0x7f35ec1f8c60 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f35e400c730 tx=0x7f35e400cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:35.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.660+0000 7f35e8c00640 1 -- 172.21.15.39:0/2967902491 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e4003b70 con 0x7f35ec0d1ec0 2024-09-21T07:48:35.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.660+0000 7f35e8c00640 1 -- 172.21.15.39:0/2967902491 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f35e4003d10 con 0x7f35ec0d1ec0 2024-09-21T07:48:35.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.660+0000 7f35f2000640 1 -- 172.21.15.39:0/2967902491 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35ec1502a0 con 0x7f35ec0d1ec0 2024-09-21T07:48:35.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.660+0000 7f35e8c00640 1 -- 172.21.15.39:0/2967902491 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e40052d0 con 0x7f35ec0d1ec0 2024-09-21T07:48:35.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.660+0000 7f35f2000640 1 -- 172.21.15.39:0/2967902491 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35ec06e5e0 con 0x7f35ec0d1ec0 2024-09-21T07:48:35.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.661+0000 7f35f2000640 1 -- 172.21.15.39:0/2967902491 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35ec15fb10 con 0x7f35ec0d1ec0 2024-09-21T07:48:35.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.663+0000 7f35e8c00640 1 -- 172.21.15.39:0/2967902491 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f35e4005470 con 0x7f35ec0d1ec0 2024-09-21T07:48:35.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.663+0000 7f35e8c00640 1 --2- 172.21.15.39:0/2967902491 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f35cc078b90 0x7f35cc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:35.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.664+0000 7f35e8c00640 1 -- 172.21.15.39:0/2967902491 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(210..210 src has 1..210) ==== 6308+0+0 (secure 0 0 0) 0x7f35e409ae60 con 0x7f35ec0d1ec0 2024-09-21T07:48:35.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.664+0000 7f35f0c00640 1 --2- 172.21.15.39:0/2967902491 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f35cc078b90 0x7f35cc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:35.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.665+0000 7f35f0c00640 1 --2- 172.21.15.39:0/2967902491 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f35cc078b90 0x7f35cc07b0a0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f35e0009b90 tx=0x7f35e0005f30 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:35.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.668+0000 7f35e8c00640 1 -- 172.21.15.39:0/2967902491 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f35e4062f20 con 0x7f35ec0d1ec0 2024-09-21T07:48:35.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.873+0000 7f35f2000640 1 -- 172.21.15.39:0/2967902491 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f35ec069f80 con 0x7f35ec0d1ec0 2024-09-21T07:48:35.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.877+0000 7f35e8c00640 1 -- 172.21.15.39:0/2967902491 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v52) ==== 71+0+0 (secure 0 0 0) 0x7f35e4062820 con 0x7f35ec0d1ec0 2024-09-21T07:48:35.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.880+0000 7f35f2000640 1 -- 172.21.15.39:0/2967902491 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f35cc078b90 msgr2=0x7f35cc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:35.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.880+0000 7f35f2000640 1 --2- 172.21.15.39:0/2967902491 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f35cc078b90 0x7f35cc07b0a0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f35e0009b90 tx=0x7f35e0005f30 comp rx=0 tx=0).stop 2024-09-21T07:48:35.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.880+0000 7f35f2000640 1 -- 172.21.15.39:0/2967902491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35ec0d1ec0 msgr2=0x7f35ec1f8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:35.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.880+0000 7f35f2000640 1 --2- 172.21.15.39:0/2967902491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35ec0d1ec0 0x7f35ec1f8c60 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f35e400c730 tx=0x7f35e400cb30 comp rx=0 tx=0).stop 2024-09-21T07:48:35.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.880+0000 7f35f2000640 1 -- 172.21.15.39:0/2967902491 shutdown_connections 2024-09-21T07:48:35.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.880+0000 7f35f2000640 1 --2- 172.21.15.39:0/2967902491 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f35cc078b90 0x7f35cc07b0a0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:35.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.880+0000 7f35f2000640 1 --2- 172.21.15.39:0/2967902491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35ec0d1ec0 0x7f35ec1f8c60 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:35.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.880+0000 7f35f2000640 1 --2- 172.21.15.39:0/2967902491 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35ec0ccdb0 0x7f35ec06f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:35.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.880+0000 7f35f2000640 1 --2- 172.21.15.39:0/2967902491 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35ec0b50f0 0x7f35ec14fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:35.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.880+0000 7f35f2000640 1 -- 172.21.15.39:0/2967902491 >> 172.21.15.39:0/2967902491 conn(0x7f35ec09b5f0 msgr2=0x7f35ec0b75e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:35.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.880+0000 7f35f2000640 1 -- 172.21.15.39:0/2967902491 shutdown_connections 2024-09-21T07:48:35.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:35.880+0000 7f35f2000640 1 -- 172.21.15.39:0/2967902491 wait complete. 2024-09-21T07:48:35.908 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_multimds_mksnap' 2024-09-21T07:48:36.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:36.141+0000 7f040be00640 1 -- 172.21.15.39:0/590914520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03f0004800 msgr2=0x7f03f0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:36.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:36.141+0000 7f040be00640 1 --2- 172.21.15.39:0/590914520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03f0004800 0x7f03f0004c10 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f03f800c310 tx=0x7f03f0005830 comp rx=0 tx=0).stop 2024-09-21T07:48:36.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:36.141+0000 7f040be00640 1 -- 172.21.15.39:0/590914520 shutdown_connections 2024-09-21T07:48:36.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:36.141+0000 7f040be00640 1 --2- 172.21.15.39:0/590914520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03f0004800 0x7f03f0004c10 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:36.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:36.141+0000 7f040be00640 1 -- 172.21.15.39:0/590914520 >> 172.21.15.39:0/590914520 conn(0x7f040c06cfc0 msgr2=0x7f040c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:36.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:36.141+0000 7f040be00640 1 -- 172.21.15.39:0/590914520 shutdown_connections 2024-09-21T07:48:36.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:36.141+0000 7f040be00640 1 -- 172.21.15.39:0/590914520 wait complete. 2024-09-21T07:48:36.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:36.143+0000 7f040be00640 1 Processor -- start 2024-09-21T07:48:36.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:36.143+0000 7f040be00640 1 -- start start 2024-09-21T07:48:36.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:36.145+0000 7f040be00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03f0004800 0x7f040c153c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:36.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:36.145+0000 7f040be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f040c1f4b30 0x7f040c1f4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:36.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:36.145+0000 7f040be00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f040c1f5480 0x7f040c1f8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:36.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:36.145+0000 7f040be00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f040c15d3a0 con 0x7f040c1f4b30 2024-09-21T07:48:36.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:36.145+0000 7f040be00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f040c15d220 con 0x7f040c1f5480 2024-09-21T07:48:36.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:36.145+0000 7f040be00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f040c0cd5a0 con 0x7f03f0004800 2024-09-21T07:48:36.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:36.145+0000 7f040a000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f040c1f4b30 0x7f040c1f4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:36.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:36.145+0000 7f040a000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f040c1f4b30 0x7f040c1f4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60146/0 (socket says 172.21.15.39:60146) 2024-09-21T07:48:36.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:36.145+0000 7f040a000640 1 -- 172.21.15.39:0/3689957127 learned_addr learned my addr 172.21.15.39:0/3689957127 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:36.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:36.145+0000 7f040aa00640 1 --2- 172.21.15.39:0/3689957127 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03f0004800 0x7f040c153c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:36.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:36.145+0000 7f040b400640 1 --2- 172.21.15.39:0/3689957127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f040c1f5480 0x7f040c1f8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:36.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:36.146+0000 7f040a000640 1 -- 172.21.15.39:0/3689957127 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03f0004800 msgr2=0x7f040c153c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:36.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:36.146+0000 7f040a000640 1 --2- 172.21.15.39:0/3689957127 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03f0004800 0x7f040c153c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:36.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:36.146+0000 7f040a000640 1 -- 172.21.15.39:0/3689957127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f040c1f5480 msgr2=0x7f040c1f8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:36.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:36.146+0000 7f040a000640 1 --2- 172.21.15.39:0/3689957127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f040c1f5480 0x7f040c1f8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:36.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:36.146+0000 7f040a000640 1 -- 172.21.15.39:0/3689957127 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f040c0b3ba0 con 0x7f040c1f4b30 2024-09-21T07:48:36.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:36.146+0000 7f040a000640 1 --2- 172.21.15.39:0/3689957127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f040c1f4b30 0x7f040c1f4f40 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f03f802b140 tx=0x7f03f8001d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:36.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:36.146+0000 7f0402a00640 1 -- 172.21.15.39:0/3689957127 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f8031040 con 0x7f040c1f4b30 2024-09-21T07:48:36.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:36.146+0000 7f0402a00640 1 -- 172.21.15.39:0/3689957127 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f03f80035f0 con 0x7f040c1f4b30 2024-09-21T07:48:36.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:36.146+0000 7f040be00640 1 -- 172.21.15.39:0/3689957127 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f040c0b40e0 con 0x7f040c1f4b30 2024-09-21T07:48:36.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:36.146+0000 7f0402a00640 1 -- 172.21.15.39:0/3689957127 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f8003a20 con 0x7f040c1f4b30 2024-09-21T07:48:36.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:36.146+0000 7f040be00640 1 -- 172.21.15.39:0/3689957127 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f040c0b46a0 con 0x7f040c1f4b30 2024-09-21T07:48:36.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:36.147+0000 7f040be00640 1 -- 172.21.15.39:0/3689957127 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f040c0b4dc0 con 0x7f040c1f4b30 2024-09-21T07:48:36.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:36.149+0000 7f0402a00640 1 -- 172.21.15.39:0/3689957127 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f03f8034070 con 0x7f040c1f4b30 2024-09-21T07:48:36.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:36.150+0000 7f0402a00640 1 --2- 172.21.15.39:0/3689957127 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f03ec078b90 0x7f03ec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:36.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:36.150+0000 7f0402a00640 1 -- 172.21.15.39:0/3689957127 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(210..210 src has 1..210) ==== 6308+0+0 (secure 0 0 0) 0x7f03f80baec0 con 0x7f040c1f4b30 2024-09-21T07:48:36.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:36.150+0000 7f040aa00640 1 --2- 172.21.15.39:0/3689957127 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f03ec078b90 0x7f03ec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:36.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:36.151+0000 7f040aa00640 1 --2- 172.21.15.39:0/3689957127 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f03ec078b90 0x7f03ec07b0a0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f03fc006680 tx=0x7f03fc006b70 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:36.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:36.154+0000 7f0402a00640 1 -- 172.21.15.39:0/3689957127 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f03f8083010 con 0x7f040c1f4b30 2024-09-21T07:48:36.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:36.426+0000 7f040be00640 1 -- 172.21.15.39:0/3689957127 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_multimds_mksnap"]} v 0) -- 0x7f040c0d04b0 con 0x7f040c1f4b30 2024-09-21T07:48:37.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.425+0000 7f0402a00640 1 -- 172.21.15.39:0/3689957127 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_multimds_mksnap"]}]=0 v749) ==== 141+0+0 (secure 0 0 0) 0x7f03f8082910 con 0x7f040c1f4b30 2024-09-21T07:48:37.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.429+0000 7f040be00640 1 -- 172.21.15.39:0/3689957127 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f03ec078b90 msgr2=0x7f03ec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:37.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.429+0000 7f040be00640 1 --2- 172.21.15.39:0/3689957127 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f03ec078b90 0x7f03ec07b0a0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f03fc006680 tx=0x7f03fc006b70 comp rx=0 tx=0).stop 2024-09-21T07:48:37.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.429+0000 7f040be00640 1 -- 172.21.15.39:0/3689957127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f040c1f4b30 msgr2=0x7f040c1f4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:37.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.429+0000 7f040be00640 1 --2- 172.21.15.39:0/3689957127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f040c1f4b30 0x7f040c1f4f40 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f03f802b140 tx=0x7f03f8001d20 comp rx=0 tx=0).stop 2024-09-21T07:48:37.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.429+0000 7f040be00640 1 -- 172.21.15.39:0/3689957127 shutdown_connections 2024-09-21T07:48:37.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.429+0000 7f040be00640 1 --2- 172.21.15.39:0/3689957127 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f03ec078b90 0x7f03ec07b0a0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:37.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.429+0000 7f040be00640 1 --2- 172.21.15.39:0/3689957127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f040c1f5480 0x7f040c1f8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:37.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.429+0000 7f040be00640 1 --2- 172.21.15.39:0/3689957127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f040c1f4b30 0x7f040c1f4f40 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:37.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.429+0000 7f040be00640 1 --2- 172.21.15.39:0/3689957127 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03f0004800 0x7f040c153c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:37.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.429+0000 7f040be00640 1 -- 172.21.15.39:0/3689957127 >> 172.21.15.39:0/3689957127 conn(0x7f040c06cfc0 msgr2=0x7f040c06f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:37.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.429+0000 7f040be00640 1 -- 172.21.15.39:0/3689957127 shutdown_connections 2024-09-21T07:48:37.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.429+0000 7f040be00640 1 -- 172.21.15.39:0/3689957127 wait complete. 2024-09-21T07:48:37.456 INFO:tasks.cephfs_test_runner:check if snapshot takes effect across authority subtrees ... ok 2024-09-21T07:48:37.456 INFO:tasks.cephfs_test_runner:Starting test: test_multimds_past_parents (tasks.cephfs.test_snapshots.TestSnapshots) check if past parents are properly recorded during across authority rename 2024-09-21T07:48:37.456 INFO:tasks.cephfs_test_runner:test_multimds_past_parents (tasks.cephfs.test_snapshots.TestSnapshots) 2024-09-21T07:48:37.456 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:48:37.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.686+0000 7f2638e00640 1 -- 172.21.15.39:0/2148591047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26340cb8a0 msgr2=0x7f2634156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:37.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.686+0000 7f2638e00640 1 --2- 172.21.15.39:0/2148591047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26340cb8a0 0x7f2634156bb0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f26340cb3e0 tx=0x7f262002a4d0 comp rx=0 tx=0).stop 2024-09-21T07:48:37.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.686+0000 7f2638e00640 1 -- 172.21.15.39:0/2148591047 shutdown_connections 2024-09-21T07:48:37.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.686+0000 7f2638e00640 1 --2- 172.21.15.39:0/2148591047 >> v2:172.21.15.148:3300/0 conn(0x7f26340ccc60 0x7f26340cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:37.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.686+0000 7f2638e00640 1 --2- 172.21.15.39:0/2148591047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26340cb8a0 0x7f2634156bb0 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:37.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.686+0000 7f2638e00640 1 -- 172.21.15.39:0/2148591047 >> 172.21.15.39:0/2148591047 conn(0x7f26340b2460 msgr2=0x7f263406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:37.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.686+0000 7f2638e00640 1 -- 172.21.15.39:0/2148591047 shutdown_connections 2024-09-21T07:48:37.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.687+0000 7f2638e00640 1 -- 172.21.15.39:0/2148591047 wait complete. 2024-09-21T07:48:37.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.689+0000 7f2638e00640 1 Processor -- start 2024-09-21T07:48:37.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.689+0000 7f2638e00640 1 -- start start 2024-09-21T07:48:37.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.690+0000 7f2638e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f26340b30a0 0x7f26341f6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:37.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.691+0000 7f2638e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26340cb8a0 0x7f26341f3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:37.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.691+0000 7f2638e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26340ccc60 0x7f26340cf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:37.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.691+0000 7f2638e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f26340af410 con 0x7f26340ccc60 2024-09-21T07:48:37.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.691+0000 7f2638e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f26340af290 con 0x7f26340cb8a0 2024-09-21T07:48:37.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.691+0000 7f2638e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f26340af590 con 0x7f26340b30a0 2024-09-21T07:48:37.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.691+0000 7f2632800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26340ccc60 0x7f26340cf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:37.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.691+0000 7f2632800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26340ccc60 0x7f26340cf4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60154/0 (socket says 172.21.15.39:60154) 2024-09-21T07:48:37.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.691+0000 7f2632800640 1 -- 172.21.15.39:0/95695146 learned_addr learned my addr 172.21.15.39:0/95695146 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:37.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.691+0000 7f2631e00640 1 --2- 172.21.15.39:0/95695146 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f26340b30a0 0x7f26341f6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:37.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.691+0000 7f2631400640 1 --2- 172.21.15.39:0/95695146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26340cb8a0 0x7f26341f3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:37.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.691+0000 7f2632800640 1 -- 172.21.15.39:0/95695146 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f26340b30a0 msgr2=0x7f26341f6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:37.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.691+0000 7f2632800640 1 --2- 172.21.15.39:0/95695146 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f26340b30a0 0x7f26341f6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:37.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.691+0000 7f2632800640 1 -- 172.21.15.39:0/95695146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26340cb8a0 msgr2=0x7f26341f3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:37.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.691+0000 7f2632800640 1 --2- 172.21.15.39:0/95695146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26340cb8a0 0x7f26341f3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:37.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.691+0000 7f2632800640 1 -- 172.21.15.39:0/95695146 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26340c7300 con 0x7f26340ccc60 2024-09-21T07:48:37.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.692+0000 7f2632800640 1 --2- 172.21.15.39:0/95695146 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26340ccc60 0x7f26340cf4e0 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f262c00e960 tx=0x7f262c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:37.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.692+0000 7f262a000640 1 -- 172.21.15.39:0/95695146 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c004d20 con 0x7f26340ccc60 2024-09-21T07:48:37.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.692+0000 7f262a000640 1 -- 172.21.15.39:0/95695146 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f262c00cb20 con 0x7f26340ccc60 2024-09-21T07:48:37.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.692+0000 7f2638e00640 1 -- 172.21.15.39:0/95695146 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26340c75c0 con 0x7f26340ccc60 2024-09-21T07:48:37.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.692+0000 7f262a000640 1 -- 172.21.15.39:0/95695146 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c005020 con 0x7f26340ccc60 2024-09-21T07:48:37.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.692+0000 7f2638e00640 1 -- 172.21.15.39:0/95695146 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26340c8fb0 con 0x7f26340ccc60 2024-09-21T07:48:37.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.693+0000 7f2638e00640 1 -- 172.21.15.39:0/95695146 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26340afab0 con 0x7f26340ccc60 2024-09-21T07:48:37.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.695+0000 7f262a000640 1 -- 172.21.15.39:0/95695146 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f262c0051c0 con 0x7f26340ccc60 2024-09-21T07:48:37.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.696+0000 7f262a000640 1 --2- 172.21.15.39:0/95695146 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2614078b90 0x7f261407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:37.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.696+0000 7f262a000640 1 -- 172.21.15.39:0/95695146 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(210..210 src has 1..210) ==== 6308+0+0 (secure 0 0 0) 0x7f262c09b800 con 0x7f26340ccc60 2024-09-21T07:48:37.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.697+0000 7f2631e00640 1 --2- 172.21.15.39:0/95695146 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2614078b90 0x7f261407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:37.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.697+0000 7f2631e00640 1 --2- 172.21.15.39:0/95695146 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2614078b90 0x7f261407b0a0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f2624006fd0 tx=0x7f2624009060 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:37.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.701+0000 7f262a000640 1 -- 172.21.15.39:0/95695146 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f262c016030 con 0x7f26340ccc60 2024-09-21T07:48:37.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.896+0000 7f2638e00640 1 -- 172.21.15.39:0/95695146 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f263406d2a0 con 0x7f26340ccc60 2024-09-21T07:48:37.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.898+0000 7f262a000640 1 -- 172.21.15.39:0/95695146 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v210) ==== 74+0+18939 (secure 0 0 0) 0x7f262c063380 con 0x7f26340ccc60 2024-09-21T07:48:37.899 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:48:37.899 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":210,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:48:35.393829+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6835/2356847598":"2024-09-22T07:48:32.155030+0000","172.21.15.39:6834/2356847598":"2024-09-22T07:48:32.155030+0000","172.21.15.39:6833/1947479913":"2024-09-22T07:48:32.155010+0000","172.21.15.39:6832/1947479913":"2024-09-22T07:48:32.155010+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]},{"pool":20,"snaps":[{"begin":80,"length":23},{"begin":181,"length":23}]},{"pool":22,"snaps":[{"begin":100,"length":1},{"begin":199,"length":1}]},{"pool":24,"snaps":[{"begin":2,"length":2}]},{"pool":26,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:48:37.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.900+0000 7f2638e00640 1 -- 172.21.15.39:0/95695146 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2614078b90 msgr2=0x7f261407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:37.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.900+0000 7f2638e00640 1 --2- 172.21.15.39:0/95695146 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2614078b90 0x7f261407b0a0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f2624006fd0 tx=0x7f2624009060 comp rx=0 tx=0).stop 2024-09-21T07:48:37.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.900+0000 7f2638e00640 1 -- 172.21.15.39:0/95695146 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26340ccc60 msgr2=0x7f26340cf4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:37.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.900+0000 7f2638e00640 1 --2- 172.21.15.39:0/95695146 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26340ccc60 0x7f26340cf4e0 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f262c00e960 tx=0x7f262c00ee20 comp rx=0 tx=0).stop 2024-09-21T07:48:37.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.900+0000 7f2638e00640 1 -- 172.21.15.39:0/95695146 shutdown_connections 2024-09-21T07:48:37.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.900+0000 7f2638e00640 1 --2- 172.21.15.39:0/95695146 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2614078b90 0x7f261407b0a0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:37.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.900+0000 7f2638e00640 1 --2- 172.21.15.39:0/95695146 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26340ccc60 0x7f26340cf4e0 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:37.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.900+0000 7f2638e00640 1 --2- 172.21.15.39:0/95695146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26340cb8a0 0x7f26341f3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:37.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.900+0000 7f2638e00640 1 --2- 172.21.15.39:0/95695146 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f26340b30a0 0x7f26341f6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:37.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.900+0000 7f2638e00640 1 -- 172.21.15.39:0/95695146 >> 172.21.15.39:0/95695146 conn(0x7f26340b2460 msgr2=0x7f26340b06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:37.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.900+0000 7f2638e00640 1 -- 172.21.15.39:0/95695146 shutdown_connections 2024-09-21T07:48:37.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:37.900+0000 7f2638e00640 1 -- 172.21.15.39:0/95695146 wait complete. 2024-09-21T07:48:37.924 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T07:33:01.569433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'snap-test-pool', 'create_time': '2024-09-21T07:34:22.032185+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 1, 'snap_epoch': 37, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'test_data_pool', 'create_time': '2024-09-21T07:35:25.604532+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 1, 'snap_epoch': 66, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_metadata_pool', 'create_time': '2024-09-21T07:35:26.327607+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T07:48:37.924 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T07:48:38.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.154+0000 7fa822200640 1 -- 172.21.15.39:0/1521592083 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa81c0b3f60 msgr2=0x7fa81c0cfe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:38.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.154+0000 7fa822200640 1 --2- 172.21.15.39:0/1521592083 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa81c0b3f60 0x7fa81c0cfe30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa810009850 tx=0x7fa81002a5a0 comp rx=0 tx=0).stop 2024-09-21T07:48:38.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.154+0000 7fa822200640 1 -- 172.21.15.39:0/1521592083 shutdown_connections 2024-09-21T07:48:38.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.154+0000 7fa822200640 1 --2- 172.21.15.39:0/1521592083 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa81c0b3f60 0x7fa81c0cfe30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:38.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.154+0000 7fa822200640 1 -- 172.21.15.39:0/1521592083 >> 172.21.15.39:0/1521592083 conn(0x7fa81c08db00 msgr2=0x7fa81c09d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:38.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.154+0000 7fa822200640 1 -- 172.21.15.39:0/1521592083 shutdown_connections 2024-09-21T07:48:38.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.154+0000 7fa822200640 1 -- 172.21.15.39:0/1521592083 wait complete. 2024-09-21T07:48:38.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.157+0000 7fa822200640 1 Processor -- start 2024-09-21T07:48:38.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.157+0000 7fa822200640 1 -- start start 2024-09-21T07:48:38.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.158+0000 7fa822200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa81c0b3f60 0x7fa81c1f2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:38.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.158+0000 7fa822200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa81c0d11a0 0x7fa81c0b83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:38.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.158+0000 7fa822200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa81c0a1d00 0x7fa81c0b88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:38.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.158+0000 7fa822200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa81c0b5040 con 0x7fa81c0d11a0 2024-09-21T07:48:38.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.158+0000 7fa822200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa81c0d1930 con 0x7fa81c0a1d00 2024-09-21T07:48:38.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.158+0000 7fa822200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa81c0b51c0 con 0x7fa81c0b3f60 2024-09-21T07:48:38.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.159+0000 7fa81be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa81c0d11a0 0x7fa81c0b83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:38.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.159+0000 7fa81be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa81c0d11a0 0x7fa81c0b83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60160/0 (socket says 172.21.15.39:60160) 2024-09-21T07:48:38.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.159+0000 7fa81be00640 1 -- 172.21.15.39:0/1996808426 learned_addr learned my addr 172.21.15.39:0/1996808426 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:38.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.159+0000 7fa820e00640 1 --2- 172.21.15.39:0/1996808426 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa81c0b3f60 0x7fa81c1f2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:38.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.159+0000 7fa821800640 1 --2- 172.21.15.39:0/1996808426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa81c0a1d00 0x7fa81c0b88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:38.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.159+0000 7fa81be00640 1 -- 172.21.15.39:0/1996808426 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa81c0b3f60 msgr2=0x7fa81c1f2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:38.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.159+0000 7fa81be00640 1 --2- 172.21.15.39:0/1996808426 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa81c0b3f60 0x7fa81c1f2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:38.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.159+0000 7fa81be00640 1 -- 172.21.15.39:0/1996808426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa81c0a1d00 msgr2=0x7fa81c0b88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:38.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.159+0000 7fa81be00640 1 --2- 172.21.15.39:0/1996808426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa81c0a1d00 0x7fa81c0b88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:38.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.159+0000 7fa81be00640 1 -- 172.21.15.39:0/1996808426 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa81c1f51f0 con 0x7fa81c0d11a0 2024-09-21T07:48:38.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.159+0000 7fa81be00640 1 --2- 172.21.15.39:0/1996808426 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa81c0d11a0 0x7fa81c0b83b0 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7fa80800c400 tx=0x7fa80800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:38.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.160+0000 7fa818c00640 1 -- 172.21.15.39:0/1996808426 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa808011790 con 0x7fa81c0d11a0 2024-09-21T07:48:38.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.160+0000 7fa822200640 1 -- 172.21.15.39:0/1996808426 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa81c1f54d0 con 0x7fa81c0d11a0 2024-09-21T07:48:38.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.160+0000 7fa818c00640 1 -- 172.21.15.39:0/1996808426 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa808017070 con 0x7fa81c0d11a0 2024-09-21T07:48:38.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.160+0000 7fa818c00640 1 -- 172.21.15.39:0/1996808426 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa808011d00 con 0x7fa81c0d11a0 2024-09-21T07:48:38.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.160+0000 7fa822200640 1 -- 172.21.15.39:0/1996808426 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa81c1f5a90 con 0x7fa81c0d11a0 2024-09-21T07:48:38.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.161+0000 7fa822200640 1 -- 172.21.15.39:0/1996808426 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa81c0cfe30 con 0x7fa81c0d11a0 2024-09-21T07:48:38.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.162+0000 7fa818c00640 1 -- 172.21.15.39:0/1996808426 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa80801c460 con 0x7fa81c0d11a0 2024-09-21T07:48:38.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.163+0000 7fa818c00640 1 --2- 172.21.15.39:0/1996808426 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa7fc078bc0 0x7fa7fc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:38.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.163+0000 7fa818c00640 1 -- 172.21.15.39:0/1996808426 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(210..210 src has 1..210) ==== 6308+0+0 (secure 0 0 0) 0x7fa80809aea0 con 0x7fa81c0d11a0 2024-09-21T07:48:38.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.164+0000 7fa820e00640 1 --2- 172.21.15.39:0/1996808426 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa7fc078bc0 0x7fa7fc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:38.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.165+0000 7fa820e00640 1 --2- 172.21.15.39:0/1996808426 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa7fc078bc0 0x7fa7fc07b0d0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fa810034000 tx=0x7fa810004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:38.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.168+0000 7fa818c00640 1 -- 172.21.15.39:0/1996808426 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa808062f60 con 0x7fa81c0d11a0 2024-09-21T07:48:38.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.363+0000 7fa822200640 1 -- 172.21.15.39:0/1996808426 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa81c06e140 con 0x7fa81c0d11a0 2024-09-21T07:48:38.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.363+0000 7fa818c00640 1 -- 172.21.15.39:0/1996808426 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v210) ==== 93+0+10 (secure 0 0 0) 0x7fa808062860 con 0x7fa81c0d11a0 2024-09-21T07:48:38.364 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-09-21T07:48:38.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.365+0000 7fa822200640 1 -- 172.21.15.39:0/1996808426 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa7fc078bc0 msgr2=0x7fa7fc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:38.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.365+0000 7fa822200640 1 --2- 172.21.15.39:0/1996808426 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa7fc078bc0 0x7fa7fc07b0d0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fa810034000 tx=0x7fa810004730 comp rx=0 tx=0).stop 2024-09-21T07:48:38.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.365+0000 7fa822200640 1 -- 172.21.15.39:0/1996808426 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa81c0d11a0 msgr2=0x7fa81c0b83b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:38.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.365+0000 7fa822200640 1 --2- 172.21.15.39:0/1996808426 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa81c0d11a0 0x7fa81c0b83b0 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7fa80800c400 tx=0x7fa80800c8c0 comp rx=0 tx=0).stop 2024-09-21T07:48:38.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.365+0000 7fa822200640 1 -- 172.21.15.39:0/1996808426 shutdown_connections 2024-09-21T07:48:38.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.365+0000 7fa822200640 1 --2- 172.21.15.39:0/1996808426 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa7fc078bc0 0x7fa7fc07b0d0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:38.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.365+0000 7fa822200640 1 --2- 172.21.15.39:0/1996808426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa81c0a1d00 0x7fa81c0b88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:38.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.365+0000 7fa822200640 1 --2- 172.21.15.39:0/1996808426 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa81c0d11a0 0x7fa81c0b83b0 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:38.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.365+0000 7fa822200640 1 --2- 172.21.15.39:0/1996808426 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa81c0b3f60 0x7fa81c1f2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:38.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.366+0000 7fa822200640 1 -- 172.21.15.39:0/1996808426 >> 172.21.15.39:0/1996808426 conn(0x7fa81c08db00 msgr2=0x7fa81c0d4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:38.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.366+0000 7fa822200640 1 -- 172.21.15.39:0/1996808426 shutdown_connections 2024-09-21T07:48:38.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.366+0000 7fa822200640 1 -- 172.21.15.39:0/1996808426 wait complete. 2024-09-21T07:48:38.391 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2024-09-21T07:48:38.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.620+0000 7f092a200640 1 -- 172.21.15.39:0/2206022701 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f092415d830 msgr2=0x7f0924156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:38.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.620+0000 7f092a200640 1 --2- 172.21.15.39:0/2206022701 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f092415d830 0x7f0924156bb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0910009830 tx=0x7f091002a480 comp rx=0 tx=0).stop 2024-09-21T07:48:38.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.620+0000 7f092a200640 1 -- 172.21.15.39:0/2206022701 shutdown_connections 2024-09-21T07:48:38.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.620+0000 7f092a200640 1 --2- 172.21.15.39:0/2206022701 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f092415d830 0x7f0924156bb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:38.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.620+0000 7f092a200640 1 --2- 172.21.15.39:0/2206022701 >> v2:172.21.15.148:3300/0 conn(0x7f09240b30a0 0x7f09240cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:38.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.620+0000 7f092a200640 1 -- 172.21.15.39:0/2206022701 >> 172.21.15.39:0/2206022701 conn(0x7f09240b2460 msgr2=0x7f092406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:38.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.620+0000 7f092a200640 1 -- 172.21.15.39:0/2206022701 shutdown_connections 2024-09-21T07:48:38.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.620+0000 7f092a200640 1 -- 172.21.15.39:0/2206022701 wait complete. 2024-09-21T07:48:38.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.622+0000 7f092a200640 1 Processor -- start 2024-09-21T07:48:38.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.623+0000 7f092a200640 1 -- start start 2024-09-21T07:48:38.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.624+0000 7f092a200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09240b30a0 0x7f0924155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:38.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.624+0000 7f092a200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09240ccc60 0x7f09240c7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:38.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.624+0000 7f092a200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f092415d830 0x7f092415bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:38.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.624+0000 7f092a200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f09240af450 con 0x7f09240ccc60 2024-09-21T07:48:38.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.624+0000 7f092a200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f09240af2d0 con 0x7f09240b30a0 2024-09-21T07:48:38.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.624+0000 7f092a200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f09240af5d0 con 0x7f092415d830 2024-09-21T07:48:38.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.624+0000 7f0922a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09240ccc60 0x7f09240c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:38.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.624+0000 7f0922a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09240ccc60 0x7f09240c7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60166/0 (socket says 172.21.15.39:60166) 2024-09-21T07:48:38.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.624+0000 7f0922a00640 1 -- 172.21.15.39:0/435100003 learned_addr learned my addr 172.21.15.39:0/435100003 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:38.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.625+0000 7f0923400640 1 --2- 172.21.15.39:0/435100003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09240b30a0 0x7f0924155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:38.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.625+0000 7f0923e00640 1 --2- 172.21.15.39:0/435100003 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f092415d830 0x7f092415bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:38.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.625+0000 7f0922a00640 1 -- 172.21.15.39:0/435100003 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f092415d830 msgr2=0x7f092415bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:38.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.625+0000 7f0922a00640 1 --2- 172.21.15.39:0/435100003 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f092415d830 0x7f092415bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:38.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.625+0000 7f0922a00640 1 -- 172.21.15.39:0/435100003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09240b30a0 msgr2=0x7f0924155dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:38.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.625+0000 7f0922a00640 1 --2- 172.21.15.39:0/435100003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09240b30a0 0x7f0924155dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:38.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.625+0000 7f0922a00640 1 -- 172.21.15.39:0/435100003 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09241f5ee0 con 0x7f09240ccc60 2024-09-21T07:48:38.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.625+0000 7f0922a00640 1 --2- 172.21.15.39:0/435100003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09240ccc60 0x7f09240c7530 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f0910000c00 tx=0x7f0910033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:38.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.626+0000 7f091b400640 1 -- 172.21.15.39:0/435100003 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0910031d60 con 0x7f09240ccc60 2024-09-21T07:48:38.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.626+0000 7f091b400640 1 -- 172.21.15.39:0/435100003 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0910033e10 con 0x7f09240ccc60 2024-09-21T07:48:38.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.626+0000 7f092a200640 1 -- 172.21.15.39:0/435100003 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09241f61a0 con 0x7f09240ccc60 2024-09-21T07:48:38.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.626+0000 7f091b400640 1 -- 172.21.15.39:0/435100003 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0910033310 con 0x7f09240ccc60 2024-09-21T07:48:38.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.626+0000 7f092a200640 1 -- 172.21.15.39:0/435100003 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09241f6760 con 0x7f09240ccc60 2024-09-21T07:48:38.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.627+0000 7f092a200640 1 -- 172.21.15.39:0/435100003 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09240c92a0 con 0x7f09240ccc60 2024-09-21T07:48:38.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.628+0000 7f091b400640 1 -- 172.21.15.39:0/435100003 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f09100334b0 con 0x7f09240ccc60 2024-09-21T07:48:38.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.629+0000 7f091b400640 1 --2- 172.21.15.39:0/435100003 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0904078b90 0x7f090407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:38.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.629+0000 7f091b400640 1 -- 172.21.15.39:0/435100003 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(210..210 src has 1..210) ==== 6308+0+0 (secure 0 0 0) 0x7f09100bc2f0 con 0x7f09240ccc60 2024-09-21T07:48:38.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.629+0000 7f0923400640 1 --2- 172.21.15.39:0/435100003 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0904078b90 0x7f090407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:38.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.630+0000 7f0923400640 1 --2- 172.21.15.39:0/435100003 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0904078b90 0x7f090407b0a0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f0914009530 tx=0x7f0914009260 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:38.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.634+0000 7f091b400640 1 -- 172.21.15.39:0/435100003 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f091003f030 con 0x7f09240ccc60 2024-09-21T07:48:38.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.830+0000 7f092a200640 1 -- 172.21.15.39:0/435100003 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) -- 0x7f09240afa30 con 0x7f09240ccc60 2024-09-21T07:48:38.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.830+0000 7f091b400640 1 -- 172.21.15.39:0/435100003 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v210) ==== 103+0+11 (secure 0 0 0) 0x7f0910083e70 con 0x7f09240ccc60 2024-09-21T07:48:38.831 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-21T07:48:38.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.832+0000 7f092a200640 1 -- 172.21.15.39:0/435100003 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0904078b90 msgr2=0x7f090407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:38.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.832+0000 7f092a200640 1 --2- 172.21.15.39:0/435100003 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0904078b90 0x7f090407b0a0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f0914009530 tx=0x7f0914009260 comp rx=0 tx=0).stop 2024-09-21T07:48:38.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.832+0000 7f092a200640 1 -- 172.21.15.39:0/435100003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09240ccc60 msgr2=0x7f09240c7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:38.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.832+0000 7f092a200640 1 --2- 172.21.15.39:0/435100003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09240ccc60 0x7f09240c7530 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f0910000c00 tx=0x7f0910033c20 comp rx=0 tx=0).stop 2024-09-21T07:48:38.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.833+0000 7f092a200640 1 -- 172.21.15.39:0/435100003 shutdown_connections 2024-09-21T07:48:38.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.833+0000 7f092a200640 1 --2- 172.21.15.39:0/435100003 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0904078b90 0x7f090407b0a0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:38.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.833+0000 7f092a200640 1 --2- 172.21.15.39:0/435100003 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f092415d830 0x7f092415bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:38.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.833+0000 7f092a200640 1 --2- 172.21.15.39:0/435100003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09240ccc60 0x7f09240c7530 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:38.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.833+0000 7f092a200640 1 --2- 172.21.15.39:0/435100003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09240b30a0 0x7f0924155dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:38.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.833+0000 7f092a200640 1 -- 172.21.15.39:0/435100003 >> 172.21.15.39:0/435100003 conn(0x7f09240b2460 msgr2=0x7f092406f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:38.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.833+0000 7f092a200640 1 -- 172.21.15.39:0/435100003 shutdown_connections 2024-09-21T07:48:38.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:38.833+0000 7f092a200640 1 -- 172.21.15.39:0/435100003 wait complete. 2024-09-21T07:48:38.856 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2024-09-21T07:48:39.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.086+0000 7ff6b3400640 1 -- 172.21.15.39:0/1322553392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6ac073dd0 msgr2=0x7ff6ac15d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:39.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.086+0000 7ff6b3400640 1 --2- 172.21.15.39:0/1322553392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6ac073dd0 0x7ff6ac15d5b0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0009830 tx=0x7ff6a002a510 comp rx=0 tx=0).stop 2024-09-21T07:48:39.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.086+0000 7ff6b3400640 1 -- 172.21.15.39:0/1322553392 shutdown_connections 2024-09-21T07:48:39.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.086+0000 7ff6b3400640 1 --2- 172.21.15.39:0/1322553392 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6ac15daf0 0x7ff6ac0cf450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:39.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.086+0000 7ff6b3400640 1 --2- 172.21.15.39:0/1322553392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6ac073dd0 0x7ff6ac15d5b0 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:39.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.086+0000 7ff6b3400640 1 -- 172.21.15.39:0/1322553392 >> 172.21.15.39:0/1322553392 conn(0x7ff6ac06cfc0 msgr2=0x7ff6ac06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:39.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.086+0000 7ff6b3400640 1 -- 172.21.15.39:0/1322553392 shutdown_connections 2024-09-21T07:48:39.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.086+0000 7ff6b3400640 1 -- 172.21.15.39:0/1322553392 wait complete. 2024-09-21T07:48:39.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.088+0000 7ff6b3400640 1 Processor -- start 2024-09-21T07:48:39.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.088+0000 7ff6b3400640 1 -- start start 2024-09-21T07:48:39.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.089+0000 7ff6b3400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6ac073dd0 0x7ff6ac1f63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:39.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.089+0000 7ff6b3400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6ac0cf990 0x7ff6ac158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:39.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.089+0000 7ff6b3400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6ac15daf0 0x7ff6ac148990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:39.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.089+0000 7ff6b3400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac0cd1d0 con 0x7ff6ac15daf0 2024-09-21T07:48:39.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.089+0000 7ff6b3400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ac0cd050 con 0x7ff6ac073dd0 2024-09-21T07:48:39.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.090+0000 7ff6b3400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7ff6ac0cd350 con 0x7ff6ac0cf990 2024-09-21T07:48:39.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.090+0000 7ff6b1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6ac15daf0 0x7ff6ac148990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:39.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.090+0000 7ff6b1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6ac15daf0 0x7ff6ac148990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60190/0 (socket says 172.21.15.39:60190) 2024-09-21T07:48:39.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.090+0000 7ff6b1400640 1 -- 172.21.15.39:0/2944379259 learned_addr learned my addr 172.21.15.39:0/2944379259 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:39.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.090+0000 7ff6abe00640 1 --2- 172.21.15.39:0/2944379259 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6ac0cf990 0x7ff6ac158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:39.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.090+0000 7ff6b0a00640 1 --2- 172.21.15.39:0/2944379259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6ac073dd0 0x7ff6ac1f63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:39.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.090+0000 7ff6b1400640 1 -- 172.21.15.39:0/2944379259 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6ac0cf990 msgr2=0x7ff6ac158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:39.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.090+0000 7ff6b1400640 1 --2- 172.21.15.39:0/2944379259 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6ac0cf990 0x7ff6ac158920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:39.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.090+0000 7ff6b1400640 1 -- 172.21.15.39:0/2944379259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6ac073dd0 msgr2=0x7ff6ac1f63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:39.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.090+0000 7ff6b1400640 1 --2- 172.21.15.39:0/2944379259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6ac073dd0 0x7ff6ac1f63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:39.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.090+0000 7ff6b1400640 1 -- 172.21.15.39:0/2944379259 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6ac1f6910 con 0x7ff6ac15daf0 2024-09-21T07:48:39.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.090+0000 7ff6b0a00640 1 --2- 172.21.15.39:0/2944379259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6ac073dd0 0x7ff6ac1f63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:48:39.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.091+0000 7ff6b1400640 1 --2- 172.21.15.39:0/2944379259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6ac15daf0 0x7ff6ac148990 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7ff6a400ca80 tx=0x7ff6a400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:39.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.091+0000 7ff6a8c00640 1 -- 172.21.15.39:0/2944379259 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a4016020 con 0x7ff6ac15daf0 2024-09-21T07:48:39.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.091+0000 7ff6a8c00640 1 -- 172.21.15.39:0/2944379259 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6a400ad20 con 0x7ff6ac15daf0 2024-09-21T07:48:39.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.091+0000 7ff6b3400640 1 -- 172.21.15.39:0/2944379259 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6ac15aab0 con 0x7ff6ac15daf0 2024-09-21T07:48:39.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.091+0000 7ff6a8c00640 1 -- 172.21.15.39:0/2944379259 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a4012680 con 0x7ff6ac15daf0 2024-09-21T07:48:39.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.091+0000 7ff6b3400640 1 -- 172.21.15.39:0/2944379259 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6ac15b070 con 0x7ff6ac15daf0 2024-09-21T07:48:39.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.092+0000 7ff6b3400640 1 -- 172.21.15.39:0/2944379259 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6ac0697d0 con 0x7ff6ac15daf0 2024-09-21T07:48:39.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.094+0000 7ff6a8c00640 1 -- 172.21.15.39:0/2944379259 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff6a4019020 con 0x7ff6ac15daf0 2024-09-21T07:48:39.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.094+0000 7ff6a8c00640 1 --2- 172.21.15.39:0/2944379259 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff68c078b90 0x7ff68c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:39.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.095+0000 7ff6a8c00640 1 -- 172.21.15.39:0/2944379259 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(210..210 src has 1..210) ==== 6308+0+0 (secure 0 0 0) 0x7ff6a409b270 con 0x7ff6ac15daf0 2024-09-21T07:48:39.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.095+0000 7ff6b0a00640 1 --2- 172.21.15.39:0/2944379259 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff68c078b90 0x7ff68c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:39.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.096+0000 7ff6b0a00640 1 --2- 172.21.15.39:0/2944379259 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff68c078b90 0x7ff68c07b0a0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0034000 tx=0x7ff6a0004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:39.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.099+0000 7ff6a8c00640 1 -- 172.21.15.39:0/2944379259 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff6ac0697d0 con 0x7ff6ac15daf0 2024-09-21T07:48:39.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.298+0000 7ff6b3400640 1 -- 172.21.15.39:0/2944379259 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) -- 0x7ff6ac0cd800 con 0x7ff6ac15daf0 2024-09-21T07:48:39.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.298+0000 7ff6a8c00640 1 -- 172.21.15.39:0/2944379259 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v210) ==== 103+0+11 (secure 0 0 0) 0x7ff6a4020030 con 0x7ff6ac15daf0 2024-09-21T07:48:39.299 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-21T07:48:39.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.301+0000 7ff6b3400640 1 -- 172.21.15.39:0/2944379259 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff68c078b90 msgr2=0x7ff68c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:39.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.301+0000 7ff6b3400640 1 --2- 172.21.15.39:0/2944379259 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff68c078b90 0x7ff68c07b0a0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0034000 tx=0x7ff6a0004730 comp rx=0 tx=0).stop 2024-09-21T07:48:39.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.301+0000 7ff6b3400640 1 -- 172.21.15.39:0/2944379259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6ac15daf0 msgr2=0x7ff6ac148990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:39.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.301+0000 7ff6b3400640 1 --2- 172.21.15.39:0/2944379259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6ac15daf0 0x7ff6ac148990 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7ff6a400ca80 tx=0x7ff6a400cf40 comp rx=0 tx=0).stop 2024-09-21T07:48:39.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.301+0000 7ff6b3400640 1 -- 172.21.15.39:0/2944379259 shutdown_connections 2024-09-21T07:48:39.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.301+0000 7ff6b3400640 1 --2- 172.21.15.39:0/2944379259 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff68c078b90 0x7ff68c07b0a0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:39.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.301+0000 7ff6b3400640 1 --2- 172.21.15.39:0/2944379259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6ac15daf0 0x7ff6ac148990 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:39.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.301+0000 7ff6b3400640 1 --2- 172.21.15.39:0/2944379259 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff6ac0cf990 0x7ff6ac158920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:39.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.301+0000 7ff6b3400640 1 --2- 172.21.15.39:0/2944379259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6ac073dd0 0x7ff6ac1f63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:39.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.301+0000 7ff6b3400640 1 -- 172.21.15.39:0/2944379259 >> 172.21.15.39:0/2944379259 conn(0x7ff6ac06cfc0 msgr2=0x7ff6ac06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:39.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.301+0000 7ff6b3400640 1 -- 172.21.15.39:0/2944379259 shutdown_connections 2024-09-21T07:48:39.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.301+0000 7ff6b3400640 1 -- 172.21.15.39:0/2944379259 wait complete. 2024-09-21T07:48:39.328 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2024-09-21T07:48:39.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.558+0000 7f2cbf200640 1 -- 172.21.15.39:0/3683234126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c9c005350 msgr2=0x7f2c9c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:39.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.558+0000 7f2cbf200640 1 --2- 172.21.15.39:0/3683234126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c9c005350 0x7f2c9c00b9c0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0009ed0 tx=0x7f2c9c005830 comp rx=0 tx=0).stop 2024-09-21T07:48:39.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.559+0000 7f2cbf200640 1 -- 172.21.15.39:0/3683234126 shutdown_connections 2024-09-21T07:48:39.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.559+0000 7f2cbf200640 1 --2- 172.21.15.39:0/3683234126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c9c005350 0x7f2c9c00b9c0 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:39.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.559+0000 7f2cbf200640 1 -- 172.21.15.39:0/3683234126 >> 172.21.15.39:0/3683234126 conn(0x7f2cb806cfc0 msgr2=0x7f2cb806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:39.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.559+0000 7f2cbf200640 1 -- 172.21.15.39:0/3683234126 shutdown_connections 2024-09-21T07:48:39.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.559+0000 7f2cbf200640 1 -- 172.21.15.39:0/3683234126 wait complete. 2024-09-21T07:48:39.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.561+0000 7f2cbf200640 1 Processor -- start 2024-09-21T07:48:39.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.561+0000 7f2cbf200640 1 -- start start 2024-09-21T07:48:39.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.562+0000 7f2cbf200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c9c005350 0x7f2cb80b5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:39.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.563+0000 7f2cbf200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2cb81f9b10 0x7f2cb8156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:39.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.563+0000 7f2cbf200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cb81502d0 0x7f2cb81506e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:39.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.563+0000 7f2cbf200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb815da50 con 0x7f2c9c005350 2024-09-21T07:48:39.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.563+0000 7f2cbf200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb815d8d0 con 0x7f2cb81502d0 2024-09-21T07:48:39.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.563+0000 7f2cbf200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f2cb815dbd0 con 0x7f2cb81f9b10 2024-09-21T07:48:39.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.563+0000 7f2cb7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c9c005350 0x7f2cb80b5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:39.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.563+0000 7f2cb7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c9c005350 0x7f2cb80b5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60210/0 (socket says 172.21.15.39:60210) 2024-09-21T07:48:39.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.563+0000 7f2cb7e00640 1 -- 172.21.15.39:0/872493524 learned_addr learned my addr 172.21.15.39:0/872493524 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:39.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.563+0000 7f2cb7400640 1 --2- 172.21.15.39:0/872493524 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2cb81f9b10 0x7f2cb8156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:39.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.563+0000 7f2cbd200640 1 --2- 172.21.15.39:0/872493524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cb81502d0 0x7f2cb81506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:39.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.563+0000 7f2cb7e00640 1 -- 172.21.15.39:0/872493524 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2cb81f9b10 msgr2=0x7f2cb8156ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:39.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.563+0000 7f2cb7e00640 1 --2- 172.21.15.39:0/872493524 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2cb81f9b10 0x7f2cb8156ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:39.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.563+0000 7f2cb7e00640 1 -- 172.21.15.39:0/872493524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cb81502d0 msgr2=0x7f2cb81506e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:39.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.563+0000 7f2cb7e00640 1 --2- 172.21.15.39:0/872493524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cb81502d0 0x7f2cb81506e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:39.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.563+0000 7f2cb7e00640 1 -- 172.21.15.39:0/872493524 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb8150c20 con 0x7f2c9c005350 2024-09-21T07:48:39.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.564+0000 7f2cb7e00640 1 --2- 172.21.15.39:0/872493524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c9c005350 0x7f2cb80b5180 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f2ca800e5a0 tx=0x7f2ca800ea60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:39.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.564+0000 7f2cafe00640 1 -- 172.21.15.39:0/872493524 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8015020 con 0x7f2c9c005350 2024-09-21T07:48:39.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.564+0000 7f2cafe00640 1 -- 172.21.15.39:0/872493524 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ca8010980 con 0x7f2c9c005350 2024-09-21T07:48:39.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.564+0000 7f2cafe00640 1 -- 172.21.15.39:0/872493524 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8010d50 con 0x7f2c9c005350 2024-09-21T07:48:39.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.564+0000 7f2cbf200640 1 -- 172.21.15.39:0/872493524 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cb815c010 con 0x7f2c9c005350 2024-09-21T07:48:39.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.564+0000 7f2cbf200640 1 -- 172.21.15.39:0/872493524 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cb8157010 con 0x7f2c9c005350 2024-09-21T07:48:39.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.565+0000 7f2cbf200640 1 -- 172.21.15.39:0/872493524 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cb81f8e80 con 0x7f2c9c005350 2024-09-21T07:48:39.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.566+0000 7f2cafe00640 1 -- 172.21.15.39:0/872493524 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2ca8018020 con 0x7f2c9c005350 2024-09-21T07:48:39.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.567+0000 7f2cafe00640 1 --2- 172.21.15.39:0/872493524 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2c98078bc0 0x7f2c9807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:39.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.567+0000 7f2cafe00640 1 -- 172.21.15.39:0/872493524 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(210..210 src has 1..210) ==== 6308+0+0 (secure 0 0 0) 0x7f2ca809b150 con 0x7f2c9c005350 2024-09-21T07:48:39.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.568+0000 7f2cb7400640 1 --2- 172.21.15.39:0/872493524 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2c98078bc0 0x7f2c9807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:39.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.569+0000 7f2cb7400640 1 --2- 172.21.15.39:0/872493524 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2c98078bc0 0x7f2c9807b0d0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4000c00 tx=0x7f2ca4006b70 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:39.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.572+0000 7f2cafe00640 1 -- 172.21.15.39:0/872493524 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2ca801f030 con 0x7f2c9c005350 2024-09-21T07:48:39.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.766+0000 7f2cbf200640 1 -- 172.21.15.39:0/872493524 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) -- 0x7f2cb80d0840 con 0x7f2c9c005350 2024-09-21T07:48:39.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.766+0000 7f2cafe00640 1 -- 172.21.15.39:0/872493524 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v210) ==== 107+0+11 (secure 0 0 0) 0x7f2cb8150c20 con 0x7f2c9c005350 2024-09-21T07:48:39.768 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-21T07:48:39.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.769+0000 7f2cbf200640 1 -- 172.21.15.39:0/872493524 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2c98078bc0 msgr2=0x7f2c9807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:39.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.769+0000 7f2cbf200640 1 --2- 172.21.15.39:0/872493524 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2c98078bc0 0x7f2c9807b0d0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4000c00 tx=0x7f2ca4006b70 comp rx=0 tx=0).stop 2024-09-21T07:48:39.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.769+0000 7f2cbf200640 1 -- 172.21.15.39:0/872493524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c9c005350 msgr2=0x7f2cb80b5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:39.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.769+0000 7f2cbf200640 1 --2- 172.21.15.39:0/872493524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c9c005350 0x7f2cb80b5180 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f2ca800e5a0 tx=0x7f2ca800ea60 comp rx=0 tx=0).stop 2024-09-21T07:48:39.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.769+0000 7f2cbf200640 1 -- 172.21.15.39:0/872493524 shutdown_connections 2024-09-21T07:48:39.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.769+0000 7f2cbf200640 1 --2- 172.21.15.39:0/872493524 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2c98078bc0 0x7f2c9807b0d0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:39.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.769+0000 7f2cbf200640 1 --2- 172.21.15.39:0/872493524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2cb81502d0 0x7f2cb81506e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:39.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.769+0000 7f2cbf200640 1 --2- 172.21.15.39:0/872493524 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2cb81f9b10 0x7f2cb8156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:39.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.769+0000 7f2cbf200640 1 --2- 172.21.15.39:0/872493524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2c9c005350 0x7f2cb80b5180 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:39.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.769+0000 7f2cbf200640 1 -- 172.21.15.39:0/872493524 >> 172.21.15.39:0/872493524 conn(0x7f2cb806cfc0 msgr2=0x7f2cb8151110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:39.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.769+0000 7f2cbf200640 1 -- 172.21.15.39:0/872493524 shutdown_connections 2024-09-21T07:48:39.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:39.769+0000 7f2cbf200640 1 -- 172.21.15.39:0/872493524 wait complete. 2024-09-21T07:48:39.797 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_multimds_past_parents' 2024-09-21T07:48:40.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.024+0000 7f39bec00640 1 -- 172.21.15.39:0/978851515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39b80d0e60 msgr2=0x7f39b80d1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:40.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.024+0000 7f39bec00640 1 --2- 172.21.15.39:0/978851515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39b80d0e60 0x7f39b80d1270 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f39b80d2570 tx=0x7f39b002aa00 comp rx=0 tx=0).stop 2024-09-21T07:48:40.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.024+0000 7f39bec00640 1 -- 172.21.15.39:0/978851515 shutdown_connections 2024-09-21T07:48:40.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.024+0000 7f39bec00640 1 --2- 172.21.15.39:0/978851515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39b80d0e60 0x7f39b80d1270 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:40.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.024+0000 7f39bec00640 1 --2- 172.21.15.39:0/978851515 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39b80b59e0 0x7f39b80b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:40.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.024+0000 7f39bec00640 1 --2- 172.21.15.39:0/978851515 >> v2:172.21.15.148:3300/0 conn(0x7f39b80d5060 0x7f39b80b96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:40.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.024+0000 7f39bec00640 1 -- 172.21.15.39:0/978851515 >> 172.21.15.39:0/978851515 conn(0x7f39b809d7d0 msgr2=0x7f39b80d4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:40.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.024+0000 7f39bec00640 1 -- 172.21.15.39:0/978851515 shutdown_connections 2024-09-21T07:48:40.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.024+0000 7f39bec00640 1 -- 172.21.15.39:0/978851515 wait complete. 2024-09-21T07:48:40.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.027+0000 7f39bec00640 1 Processor -- start 2024-09-21T07:48:40.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.027+0000 7f39bec00640 1 -- start start 2024-09-21T07:48:40.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.028+0000 7f39bec00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39b80b59e0 0x7f39b8160510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:40.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.028+0000 7f39bec00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39b80d0e60 0x7f39b815c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:40.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.028+0000 7f39bec00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39b80d5060 0x7f39b81f90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:40.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.028+0000 7f39bec00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f39b80d2880 con 0x7f39b80d5060 2024-09-21T07:48:40.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.028+0000 7f39bec00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f39b80d0320 con 0x7f39b80d0e60 2024-09-21T07:48:40.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.028+0000 7f39bec00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f39b80d2a00 con 0x7f39b80b59e0 2024-09-21T07:48:40.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.028+0000 7f39be200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39b80d5060 0x7f39b81f90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:40.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.028+0000 7f39be200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39b80d5060 0x7f39b81f90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60226/0 (socket says 172.21.15.39:60226) 2024-09-21T07:48:40.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.028+0000 7f39be200640 1 -- 172.21.15.39:0/1753674197 learned_addr learned my addr 172.21.15.39:0/1753674197 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:40.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.029+0000 7f39bce00640 1 --2- 172.21.15.39:0/1753674197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39b80d0e60 0x7f39b815c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:40.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.029+0000 7f39bd800640 1 --2- 172.21.15.39:0/1753674197 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39b80b59e0 0x7f39b8160510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:40.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.029+0000 7f39be200640 1 -- 172.21.15.39:0/1753674197 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39b80b59e0 msgr2=0x7f39b8160510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:40.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.029+0000 7f39be200640 1 --2- 172.21.15.39:0/1753674197 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39b80b59e0 0x7f39b8160510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:40.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.029+0000 7f39be200640 1 -- 172.21.15.39:0/1753674197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39b80d0e60 msgr2=0x7f39b815c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:40.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.029+0000 7f39be200640 1 --2- 172.21.15.39:0/1753674197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39b80d0e60 0x7f39b815c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:40.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.029+0000 7f39be200640 1 -- 172.21.15.39:0/1753674197 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39b81f9e50 con 0x7f39b80d5060 2024-09-21T07:48:40.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.029+0000 7f39be200640 1 --2- 172.21.15.39:0/1753674197 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39b80d5060 0x7f39b81f90e0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f39b0005fd0 tx=0x7f39b0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:40.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.030+0000 7f39b5600640 1 -- 172.21.15.39:0/1753674197 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b0033050 con 0x7f39b80d5060 2024-09-21T07:48:40.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.030+0000 7f39b5600640 1 -- 172.21.15.39:0/1753674197 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39b0004450 con 0x7f39b80d5060 2024-09-21T07:48:40.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.030+0000 7f39bec00640 1 -- 172.21.15.39:0/1753674197 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39b815f750 con 0x7f39b80d5060 2024-09-21T07:48:40.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.030+0000 7f39b5600640 1 -- 172.21.15.39:0/1753674197 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b0038450 con 0x7f39b80d5060 2024-09-21T07:48:40.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.030+0000 7f39bec00640 1 -- 172.21.15.39:0/1753674197 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39b815fd10 con 0x7f39b80d5060 2024-09-21T07:48:40.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.031+0000 7f39bec00640 1 -- 172.21.15.39:0/1753674197 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39b80b3b60 con 0x7f39b80d5060 2024-09-21T07:48:40.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.032+0000 7f39b5600640 1 -- 172.21.15.39:0/1753674197 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f39b0036070 con 0x7f39b80d5060 2024-09-21T07:48:40.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.033+0000 7f39b5600640 1 --2- 172.21.15.39:0/1753674197 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3998078b90 0x7f399807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:40.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.033+0000 7f39b5600640 1 -- 172.21.15.39:0/1753674197 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(210..210 src has 1..210) ==== 6308+0+0 (secure 0 0 0) 0x7f39b00bc2a0 con 0x7f39b80d5060 2024-09-21T07:48:40.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.033+0000 7f39bd800640 1 --2- 172.21.15.39:0/1753674197 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3998078b90 0x7f399807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:40.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.034+0000 7f39bd800640 1 --2- 172.21.15.39:0/1753674197 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3998078b90 0x7f399807b0a0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f39ac00bab0 tx=0x7f39ac00b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:40.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.037+0000 7f39b5600640 1 -- 172.21.15.39:0/1753674197 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f39b0083e20 con 0x7f39b80d5060 2024-09-21T07:48:40.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.310+0000 7f39bec00640 1 -- 172.21.15.39:0/1753674197 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_multimds_past_parents"]} v 0) -- 0x7f39b80d07c0 con 0x7f39b80d5060 2024-09-21T07:48:40.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.445+0000 7f39b5600640 1 -- 172.21.15.39:0/1753674197 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_multimds_past_parents"]}]=0 v752) ==== 150+0+0 (secure 0 0 0) 0x7f39b0088850 con 0x7f39b80d5060 2024-09-21T07:48:40.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.448+0000 7f39bec00640 1 -- 172.21.15.39:0/1753674197 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3998078b90 msgr2=0x7f399807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:40.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.448+0000 7f39bec00640 1 --2- 172.21.15.39:0/1753674197 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3998078b90 0x7f399807b0a0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f39ac00bab0 tx=0x7f39ac00b2e0 comp rx=0 tx=0).stop 2024-09-21T07:48:40.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.448+0000 7f39bec00640 1 -- 172.21.15.39:0/1753674197 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39b80d5060 msgr2=0x7f39b81f90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:40.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.448+0000 7f39bec00640 1 --2- 172.21.15.39:0/1753674197 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39b80d5060 0x7f39b81f90e0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f39b0005fd0 tx=0x7f39b0003f10 comp rx=0 tx=0).stop 2024-09-21T07:48:40.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.448+0000 7f39bec00640 1 -- 172.21.15.39:0/1753674197 shutdown_connections 2024-09-21T07:48:40.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.448+0000 7f39bec00640 1 --2- 172.21.15.39:0/1753674197 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3998078b90 0x7f399807b0a0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:40.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.448+0000 7f39bec00640 1 --2- 172.21.15.39:0/1753674197 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39b80d5060 0x7f39b81f90e0 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:40.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.448+0000 7f39bec00640 1 --2- 172.21.15.39:0/1753674197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39b80d0e60 0x7f39b815c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:40.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.448+0000 7f39bec00640 1 --2- 172.21.15.39:0/1753674197 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f39b80b59e0 0x7f39b8160510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:40.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.448+0000 7f39bec00640 1 -- 172.21.15.39:0/1753674197 >> 172.21.15.39:0/1753674197 conn(0x7f39b809d7d0 msgr2=0x7f39b80b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:40.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.448+0000 7f39bec00640 1 -- 172.21.15.39:0/1753674197 shutdown_connections 2024-09-21T07:48:40.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.448+0000 7f39bec00640 1 -- 172.21.15.39:0/1753674197 wait complete. 2024-09-21T07:48:40.472 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T07:48:40.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.699+0000 7fbc96200640 1 -- 172.21.15.39:0/656870650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc900cb8a0 msgr2=0x7fbc90156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:40.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.699+0000 7fbc96200640 1 --2- 172.21.15.39:0/656870650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc900cb8a0 0x7fbc90156bb0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fbc78005fd0 tx=0x7fbc7802a4a0 comp rx=0 tx=0).stop 2024-09-21T07:48:40.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.699+0000 7fbc96200640 1 -- 172.21.15.39:0/656870650 shutdown_connections 2024-09-21T07:48:40.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.699+0000 7fbc96200640 1 --2- 172.21.15.39:0/656870650 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc900ccc60 0x7fbc900cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:40.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.699+0000 7fbc96200640 1 --2- 172.21.15.39:0/656870650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc900cb8a0 0x7fbc90156bb0 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:40.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.699+0000 7fbc96200640 1 -- 172.21.15.39:0/656870650 >> 172.21.15.39:0/656870650 conn(0x7fbc900b2460 msgr2=0x7fbc9006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:40.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.699+0000 7fbc96200640 1 -- 172.21.15.39:0/656870650 shutdown_connections 2024-09-21T07:48:40.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.699+0000 7fbc96200640 1 -- 172.21.15.39:0/656870650 wait complete. 2024-09-21T07:48:40.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.702+0000 7fbc96200640 1 Processor -- start 2024-09-21T07:48:40.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.702+0000 7fbc96200640 1 -- start start 2024-09-21T07:48:40.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.703+0000 7fbc96200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc900b30a0 0x7fbc9009dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:40.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.703+0000 7fbc96200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc900cb8a0 0x7fbc900cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:40.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.703+0000 7fbc96200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc900ccc60 0x7fbc901e93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:40.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.703+0000 7fbc96200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbc900af410 con 0x7fbc900ccc60 2024-09-21T07:48:40.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.703+0000 7fbc96200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbc900af290 con 0x7fbc900cb8a0 2024-09-21T07:48:40.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.703+0000 7fbc96200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fbc900af590 con 0x7fbc900b30a0 2024-09-21T07:48:40.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.703+0000 7fbc8fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc900ccc60 0x7fbc901e93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:40.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.703+0000 7fbc8fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc900ccc60 0x7fbc901e93e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60246/0 (socket says 172.21.15.39:60246) 2024-09-21T07:48:40.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.703+0000 7fbc8fe00640 1 -- 172.21.15.39:0/3339844727 learned_addr learned my addr 172.21.15.39:0/3339844727 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:40.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.704+0000 7fbc86a00640 1 --2- 172.21.15.39:0/3339844727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc900cb8a0 0x7fbc900cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:40.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.704+0000 7fbc8f400640 1 --2- 172.21.15.39:0/3339844727 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc900b30a0 0x7fbc9009dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:40.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.704+0000 7fbc8fe00640 1 -- 172.21.15.39:0/3339844727 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc900b30a0 msgr2=0x7fbc9009dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:40.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.704+0000 7fbc8fe00640 1 --2- 172.21.15.39:0/3339844727 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc900b30a0 0x7fbc9009dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:40.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.704+0000 7fbc8fe00640 1 -- 172.21.15.39:0/3339844727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc900cb8a0 msgr2=0x7fbc900cecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:40.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.704+0000 7fbc8fe00640 1 --2- 172.21.15.39:0/3339844727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc900cb8a0 0x7fbc900cecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:40.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.704+0000 7fbc8fe00640 1 -- 172.21.15.39:0/3339844727 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc901f9d50 con 0x7fbc900ccc60 2024-09-21T07:48:40.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.704+0000 7fbc8fe00640 1 --2- 172.21.15.39:0/3339844727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc900ccc60 0x7fbc901e93e0 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fbc8800cb00 tx=0x7fbc8800a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:40.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.705+0000 7fbc87e00640 1 -- 172.21.15.39:0/3339844727 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8800f450 con 0x7fbc900ccc60 2024-09-21T07:48:40.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.705+0000 7fbc87e00640 1 -- 172.21.15.39:0/3339844727 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc8800f700 con 0x7fbc900ccc60 2024-09-21T07:48:40.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.705+0000 7fbc87e00640 1 -- 172.21.15.39:0/3339844727 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8800fb50 con 0x7fbc900ccc60 2024-09-21T07:48:40.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.705+0000 7fbc96200640 1 -- 172.21.15.39:0/3339844727 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc901fa030 con 0x7fbc900ccc60 2024-09-21T07:48:40.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.705+0000 7fbc96200640 1 -- 172.21.15.39:0/3339844727 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc901fa5f0 con 0x7fbc900ccc60 2024-09-21T07:48:40.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.706+0000 7fbc96200640 1 -- 172.21.15.39:0/3339844727 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc900afae0 con 0x7fbc900ccc60 2024-09-21T07:48:40.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.707+0000 7fbc87e00640 1 -- 172.21.15.39:0/3339844727 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbc8800fcf0 con 0x7fbc900ccc60 2024-09-21T07:48:40.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.708+0000 7fbc87e00640 1 --2- 172.21.15.39:0/3339844727 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbc6c078b90 0x7fbc6c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:40.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.708+0000 7fbc87e00640 1 -- 172.21.15.39:0/3339844727 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(210..210 src has 1..210) ==== 6308+0+0 (secure 0 0 0) 0x7fbc8809bb60 con 0x7fbc900ccc60 2024-09-21T07:48:40.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.708+0000 7fbc8f400640 1 --2- 172.21.15.39:0/3339844727 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbc6c078b90 0x7fbc6c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:40.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.709+0000 7fbc8f400640 1 --2- 172.21.15.39:0/3339844727 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbc6c078b90 0x7fbc6c07b0a0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fbc80006fd0 tx=0x7fbc80008040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:40.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.713+0000 7fbc87e00640 1 -- 172.21.15.39:0/3339844727 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbc88014030 con 0x7fbc900ccc60 2024-09-21T07:48:40.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.910+0000 7fbc96200640 1 -- 172.21.15.39:0/3339844727 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fbc901f9ae0 con 0x7fbc900ccc60 2024-09-21T07:48:40.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.910+0000 7fbc87e00640 1 -- 172.21.15.39:0/3339844727 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v52) ==== 59+0+2294 (secure 0 0 0) 0x7fbc8809b070 con 0x7fbc900ccc60 2024-09-21T07:48:40.911 INFO:teuthology.orchestra.run.smithi039.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T07:48:40.911 INFO:teuthology.orchestra.run.smithi039.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T07:48:40.911 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced client_mount_timeout 30 2024-09-21T07:48:40.912 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_client 20/20 2024-09-21T07:48:40.912 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_mgr 20/20 2024-09-21T07:48:40.912 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_ms 1/1 2024-09-21T07:48:40.912 INFO:teuthology.orchestra.run.smithi039.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T07:48:40.912 INFO:teuthology.orchestra.run.smithi039.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T07:48:40.912 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3421.881759 2024-09-21T07:48:40.912 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3520.252978 2024-09-21T07:48:40.912 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3511.183316 2024-09-21T07:48:40.913 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3829.631847 2024-09-21T07:48:40.913 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 5474.545872 2024-09-21T07:48:40.913 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4013.628559 2024-09-21T07:48:40.913 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4230.187288 2024-09-21T07:48:40.913 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4148.983107 2024-09-21T07:48:40.913 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_mds 20/20 2024-09-21T07:48:40.913 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T07:48:40.913 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_ms 2/2 2024-09-21T07:48:40.913 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_debug_frag true 2024-09-21T07:48:40.913 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_debug_scatterstat true 2024-09-21T07:48:40.913 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T07:48:40.913 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_verify_scatter true 2024-09-21T07:48:40.913 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T07:48:40.913 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T07:48:40.913 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T07:48:40.914 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced client_mount_timeout 600 2024-09-21T07:48:40.914 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced debug_client 20/20 2024-09-21T07:48:40.914 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced debug_ms 1/1 2024-09-21T07:48:40.914 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T07:48:40.914 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T07:48:40.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.912+0000 7fbc96200640 1 -- 172.21.15.39:0/3339844727 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbc6c078b90 msgr2=0x7fbc6c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:40.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.912+0000 7fbc96200640 1 --2- 172.21.15.39:0/3339844727 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbc6c078b90 0x7fbc6c07b0a0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fbc80006fd0 tx=0x7fbc80008040 comp rx=0 tx=0).stop 2024-09-21T07:48:40.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.912+0000 7fbc96200640 1 -- 172.21.15.39:0/3339844727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc900ccc60 msgr2=0x7fbc901e93e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:40.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.912+0000 7fbc96200640 1 --2- 172.21.15.39:0/3339844727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc900ccc60 0x7fbc901e93e0 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fbc8800cb00 tx=0x7fbc8800a250 comp rx=0 tx=0).stop 2024-09-21T07:48:40.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.913+0000 7fbc96200640 1 -- 172.21.15.39:0/3339844727 shutdown_connections 2024-09-21T07:48:40.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.913+0000 7fbc96200640 1 --2- 172.21.15.39:0/3339844727 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbc6c078b90 0x7fbc6c07b0a0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:40.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.913+0000 7fbc96200640 1 --2- 172.21.15.39:0/3339844727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc900ccc60 0x7fbc901e93e0 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:40.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.913+0000 7fbc96200640 1 --2- 172.21.15.39:0/3339844727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc900cb8a0 0x7fbc900cecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:40.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.913+0000 7fbc96200640 1 --2- 172.21.15.39:0/3339844727 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbc900b30a0 0x7fbc9009dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:40.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.913+0000 7fbc96200640 1 -- 172.21.15.39:0/3339844727 >> 172.21.15.39:0/3339844727 conn(0x7fbc900b2460 msgr2=0x7fbc900b14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:40.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.913+0000 7fbc96200640 1 -- 172.21.15.39:0/3339844727 shutdown_connections 2024-09-21T07:48:40.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:40.913+0000 7fbc96200640 1 -- 172.21.15.39:0/3339844727 wait complete. 2024-09-21T07:48:40.939 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T07:48:41.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.168+0000 7f803be00640 1 -- 172.21.15.39:0/1478602504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f803c15ccd0 msgr2=0x7f803c0ccf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:41.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.168+0000 7f803be00640 1 --2- 172.21.15.39:0/1478602504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f803c15ccd0 0x7f803c0ccf90 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f8024005fd0 tx=0x7f802402a530 comp rx=0 tx=0).stop 2024-09-21T07:48:41.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.168+0000 7f803be00640 1 -- 172.21.15.39:0/1478602504 shutdown_connections 2024-09-21T07:48:41.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.168+0000 7f803be00640 1 --2- 172.21.15.39:0/1478602504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f803c15ccd0 0x7f803c0ccf90 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:41.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.168+0000 7f803be00640 1 --2- 172.21.15.39:0/1478602504 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f803c1539b0 0x7f803c0ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:41.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.168+0000 7f803be00640 1 -- 172.21.15.39:0/1478602504 >> 172.21.15.39:0/1478602504 conn(0x7f803c0d0cb0 msgr2=0x7f803c0d10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:41.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.168+0000 7f803be00640 1 -- 172.21.15.39:0/1478602504 shutdown_connections 2024-09-21T07:48:41.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.168+0000 7f803be00640 1 -- 172.21.15.39:0/1478602504 wait complete. 2024-09-21T07:48:41.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.170+0000 7f803be00640 1 Processor -- start 2024-09-21T07:48:41.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.170+0000 7f803be00640 1 -- start start 2024-09-21T07:48:41.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.172+0000 7f803be00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f803c0cd4d0 0x7f803c0afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:41.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.172+0000 7f803be00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f803c1539b0 0x7f803c157390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:41.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.172+0000 7f803be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f803c15ccd0 0x7f803c1f9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:41.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.172+0000 7f803be00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f803c1609d0 con 0x7f803c15ccd0 2024-09-21T07:48:41.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.172+0000 7f803be00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f803c0b5630 con 0x7f803c1539b0 2024-09-21T07:48:41.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.172+0000 7f803be00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f803c160b50 con 0x7f803c0cd4d0 2024-09-21T07:48:41.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.172+0000 7f803b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f803c15ccd0 0x7f803c1f9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:41.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.172+0000 7f803b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f803c15ccd0 0x7f803c1f9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60252/0 (socket says 172.21.15.39:60252) 2024-09-21T07:48:41.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.172+0000 7f803b400640 1 -- 172.21.15.39:0/4127661704 learned_addr learned my addr 172.21.15.39:0/4127661704 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:41.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.172+0000 7f803aa00640 1 --2- 172.21.15.39:0/4127661704 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f803c0cd4d0 0x7f803c0afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:41.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.172+0000 7f8033e00640 1 --2- 172.21.15.39:0/4127661704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f803c1539b0 0x7f803c157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:41.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.173+0000 7f803b400640 1 -- 172.21.15.39:0/4127661704 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f803c0cd4d0 msgr2=0x7f803c0afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:41.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.173+0000 7f803b400640 1 --2- 172.21.15.39:0/4127661704 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f803c0cd4d0 0x7f803c0afdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:41.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.173+0000 7f803b400640 1 -- 172.21.15.39:0/4127661704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f803c1539b0 msgr2=0x7f803c157390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:41.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.173+0000 7f803b400640 1 --2- 172.21.15.39:0/4127661704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f803c1539b0 0x7f803c157390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:41.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.173+0000 7f803b400640 1 -- 172.21.15.39:0/4127661704 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f803c1599e0 con 0x7f803c15ccd0 2024-09-21T07:48:41.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.173+0000 7f803b400640 1 --2- 172.21.15.39:0/4127661704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f803c15ccd0 0x7f803c1f9440 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f803400ba10 tx=0x7f803400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:41.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.173+0000 7f8033400640 1 -- 172.21.15.39:0/4127661704 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8034019070 con 0x7f803c15ccd0 2024-09-21T07:48:41.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.173+0000 7f8033400640 1 -- 172.21.15.39:0/4127661704 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f803400cc30 con 0x7f803c15ccd0 2024-09-21T07:48:41.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.173+0000 7f803be00640 1 -- 172.21.15.39:0/4127661704 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f803c159ee0 con 0x7f803c15ccd0 2024-09-21T07:48:41.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.173+0000 7f8033400640 1 -- 172.21.15.39:0/4127661704 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80340146a0 con 0x7f803c15ccd0 2024-09-21T07:48:41.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.173+0000 7f803be00640 1 -- 172.21.15.39:0/4127661704 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f803c06f680 con 0x7f803c15ccd0 2024-09-21T07:48:41.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.175+0000 7f803be00640 1 -- 172.21.15.39:0/4127661704 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f803c15a400 con 0x7f803c15ccd0 2024-09-21T07:48:41.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.176+0000 7f8033400640 1 -- 172.21.15.39:0/4127661704 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f803401c070 con 0x7f803c15ccd0 2024-09-21T07:48:41.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.177+0000 7f8033400640 1 --2- 172.21.15.39:0/4127661704 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8018078b90 0x7f801807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:41.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.177+0000 7f8033400640 1 -- 172.21.15.39:0/4127661704 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(210..210 src has 1..210) ==== 6308+0+0 (secure 0 0 0) 0x7f803409ab00 con 0x7f803c15ccd0 2024-09-21T07:48:41.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.177+0000 7f803aa00640 1 --2- 172.21.15.39:0/4127661704 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8018078b90 0x7f801807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:41.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.178+0000 7f803aa00640 1 --2- 172.21.15.39:0/4127661704 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8018078b90 0x7f801807b0a0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f802c005fd0 tx=0x7f802c00a250 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:41.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.181+0000 7f8033400640 1 -- 172.21.15.39:0/4127661704 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f803c15a400 con 0x7f803c15ccd0 2024-09-21T07:48:41.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.378+0000 7f803be00640 1 -- 172.21.15.39:0/4127661704 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f803c159c70 con 0x7f803c15ccd0 2024-09-21T07:48:41.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.383+0000 7f8033400640 1 -- 172.21.15.39:0/4127661704 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v53)=0 v53) ==== 122+0+0 (secure 0 0 0) 0x7f803400ce20 con 0x7f803c15ccd0 2024-09-21T07:48:41.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.386+0000 7f803be00640 1 -- 172.21.15.39:0/4127661704 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8018078b90 msgr2=0x7f801807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:41.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.386+0000 7f803be00640 1 --2- 172.21.15.39:0/4127661704 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8018078b90 0x7f801807b0a0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f802c005fd0 tx=0x7f802c00a250 comp rx=0 tx=0).stop 2024-09-21T07:48:41.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.386+0000 7f803be00640 1 -- 172.21.15.39:0/4127661704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f803c15ccd0 msgr2=0x7f803c1f9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:41.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.386+0000 7f803be00640 1 --2- 172.21.15.39:0/4127661704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f803c15ccd0 0x7f803c1f9440 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f803400ba10 tx=0x7f803400bed0 comp rx=0 tx=0).stop 2024-09-21T07:48:41.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.386+0000 7f803be00640 1 -- 172.21.15.39:0/4127661704 shutdown_connections 2024-09-21T07:48:41.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.386+0000 7f803be00640 1 --2- 172.21.15.39:0/4127661704 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8018078b90 0x7f801807b0a0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:41.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.386+0000 7f803be00640 1 --2- 172.21.15.39:0/4127661704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f803c15ccd0 0x7f803c1f9440 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:41.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.386+0000 7f803be00640 1 --2- 172.21.15.39:0/4127661704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f803c1539b0 0x7f803c157390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:41.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.386+0000 7f803be00640 1 --2- 172.21.15.39:0/4127661704 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f803c0cd4d0 0x7f803c0afdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:41.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.386+0000 7f803be00640 1 -- 172.21.15.39:0/4127661704 >> 172.21.15.39:0/4127661704 conn(0x7f803c0d0cb0 msgr2=0x7f803c0b06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:41.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.386+0000 7f803be00640 1 -- 172.21.15.39:0/4127661704 shutdown_connections 2024-09-21T07:48:41.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.386+0000 7f803be00640 1 -- 172.21.15.39:0/4127661704 wait complete. 2024-09-21T07:48:41.409 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:48:41.409 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:48:41.494 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:48:41.595 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:48:41.596 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:48:41.596 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:48:41.631 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:48:41.632 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:48:41.632 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:48:41.670 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:48:41.670 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:48:41.670 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:48:41.707 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:48:41.707 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:48:41.707 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:48:41.747 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:48:41.747 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T07:48:41.747 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:48:41.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.971+0000 7fda31800640 1 -- 172.21.15.39:0/2520070327 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda2c0cfa20 msgr2=0x7fda2c0cd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:41.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.971+0000 7fda31800640 1 --2- 172.21.15.39:0/2520070327 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda2c0cfa20 0x7fda2c0cd5a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fda1c005fd0 tx=0x7fda1c02a5d0 comp rx=0 tx=0).stop 2024-09-21T07:48:41.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.971+0000 7fda31800640 1 -- 172.21.15.39:0/2520070327 shutdown_connections 2024-09-21T07:48:41.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.971+0000 7fda31800640 1 --2- 172.21.15.39:0/2520070327 >> v2:172.21.15.148:3300/0 conn(0x7fda2c15cc00 0x7fda2c15d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:41.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.971+0000 7fda31800640 1 --2- 172.21.15.39:0/2520070327 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda2c0cfa20 0x7fda2c0cd5a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:41.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.971+0000 7fda31800640 1 -- 172.21.15.39:0/2520070327 >> 172.21.15.39:0/2520070327 conn(0x7fda2c06cfc0 msgr2=0x7fda2c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:41.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.971+0000 7fda31800640 1 -- 172.21.15.39:0/2520070327 shutdown_connections 2024-09-21T07:48:41.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.971+0000 7fda31800640 1 -- 172.21.15.39:0/2520070327 wait complete. 2024-09-21T07:48:41.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.973+0000 7fda31800640 1 Processor -- start 2024-09-21T07:48:41.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.973+0000 7fda31800640 1 -- start start 2024-09-21T07:48:41.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.975+0000 7fda31800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda2c0cfa20 0x7fda2c0c50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:41.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.975+0000 7fda31800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda2c0d04b0 0x7fda2c1386e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:41.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.975+0000 7fda31800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda2c15cc00 0x7fda2c1f6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:41.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.975+0000 7fda31800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c06ee70 con 0x7fda2c15cc00 2024-09-21T07:48:41.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.975+0000 7fda31800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c06ecf0 con 0x7fda2c0cfa20 2024-09-21T07:48:41.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.975+0000 7fda31800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fda2c06eff0 con 0x7fda2c0d04b0 2024-09-21T07:48:41.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.975+0000 7fda30e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda2c15cc00 0x7fda2c1f6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:41.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.975+0000 7fda30e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda2c15cc00 0x7fda2c1f6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51212/0 (socket says 172.21.15.39:51212) 2024-09-21T07:48:41.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.975+0000 7fda30e00640 1 -- 172.21.15.39:0/1784396831 learned_addr learned my addr 172.21.15.39:0/1784396831 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:41.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.975+0000 7fda2be00640 1 --2- 172.21.15.39:0/1784396831 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda2c0cfa20 0x7fda2c0c50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:41.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.975+0000 7fda2b400640 1 --2- 172.21.15.39:0/1784396831 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda2c0d04b0 0x7fda2c1386e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:41.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.975+0000 7fda30e00640 1 -- 172.21.15.39:0/1784396831 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda2c0d04b0 msgr2=0x7fda2c1386e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:41.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.975+0000 7fda30e00640 1 --2- 172.21.15.39:0/1784396831 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda2c0d04b0 0x7fda2c1386e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:41.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.975+0000 7fda30e00640 1 -- 172.21.15.39:0/1784396831 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda2c0cfa20 msgr2=0x7fda2c0c50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:41.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.975+0000 7fda30e00640 1 --2- 172.21.15.39:0/1784396831 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda2c0cfa20 0x7fda2c0c50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:41.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.975+0000 7fda30e00640 1 -- 172.21.15.39:0/1784396831 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda2c1f4310 con 0x7fda2c15cc00 2024-09-21T07:48:41.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.976+0000 7fda30e00640 1 --2- 172.21.15.39:0/1784396831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda2c15cc00 0x7fda2c1f6950 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7fda2400c700 tx=0x7fda2400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:41.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.976+0000 7fda23e00640 1 -- 172.21.15.39:0/1784396831 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda24003b70 con 0x7fda2c15cc00 2024-09-21T07:48:41.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.976+0000 7fda23e00640 1 -- 172.21.15.39:0/1784396831 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda24003d10 con 0x7fda2c15cc00 2024-09-21T07:48:41.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.976+0000 7fda23e00640 1 -- 172.21.15.39:0/1784396831 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2400aab0 con 0x7fda2c15cc00 2024-09-21T07:48:41.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.976+0000 7fda31800640 1 -- 172.21.15.39:0/1784396831 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda2c0aa650 con 0x7fda2c15cc00 2024-09-21T07:48:41.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.976+0000 7fda31800640 1 -- 172.21.15.39:0/1784396831 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda2c0aa8f0 con 0x7fda2c15cc00 2024-09-21T07:48:41.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.977+0000 7fda31800640 1 -- 172.21.15.39:0/1784396831 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda2c0af3c0 con 0x7fda2c15cc00 2024-09-21T07:48:41.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.979+0000 7fda23e00640 1 -- 172.21.15.39:0/1784396831 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fda2400ac50 con 0x7fda2c15cc00 2024-09-21T07:48:41.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.980+0000 7fda23e00640 1 --2- 172.21.15.39:0/1784396831 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fda0c078b90 0x7fda0c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:41.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.980+0000 7fda23e00640 1 -- 172.21.15.39:0/1784396831 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(210..210 src has 1..210) ==== 6308+0+0 (secure 0 0 0) 0x7fda2409acb0 con 0x7fda2c15cc00 2024-09-21T07:48:41.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.980+0000 7fda2be00640 1 --2- 172.21.15.39:0/1784396831 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fda0c078b90 0x7fda0c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:41.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.981+0000 7fda2be00640 1 --2- 172.21.15.39:0/1784396831 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fda0c078b90 0x7fda0c07b0a0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fda1c033000 tx=0x7fda1c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:41.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:41.985+0000 7fda23e00640 1 -- 172.21.15.39:0/1784396831 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fda24062840 con 0x7fda2c15cc00 2024-09-21T07:48:42.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.220+0000 7fda31800640 1 -- 172.21.15.39:0/1784396831 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda2c06f410 con 0x7fda2c15cc00 2024-09-21T07:48:42.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.221+0000 7fda23e00640 1 -- 172.21.15.39:0/1784396831 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 222 v222) ==== 95+0+3918 (secure 0 0 0) 0x7fda24067270 con 0x7fda2c15cc00 2024-09-21T07:48:42.222 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:48:42.222 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":222,"btime":"2024-09-21T07:48:33:157635+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7988,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1912631346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1912631346},{"type":"v1","addr":"172.21.15.148:6837","nonce":1912631346}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":209},{"gid":9043,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/2220628944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2220628944},{"type":"v1","addr":"172.21.15.148:6836","nonce":2220628944}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":208},{"gid":10119,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/938413776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":938413776},{"type":"v1","addr":"172.21.15.39:6834","nonce":938413776}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":222},{"gid":10122,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/511472809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":511472809},{"type":"v1","addr":"172.21.15.39:6835","nonce":511472809}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":222}],"filesystems":[]} 2024-09-21T07:48:42.222 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 222 2024-09-21T07:48:42.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.223+0000 7fda31800640 1 -- 172.21.15.39:0/1784396831 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fda0c078b90 msgr2=0x7fda0c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:42.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.223+0000 7fda31800640 1 --2- 172.21.15.39:0/1784396831 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fda0c078b90 0x7fda0c07b0a0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fda1c033000 tx=0x7fda1c004730 comp rx=0 tx=0).stop 2024-09-21T07:48:42.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.223+0000 7fda31800640 1 -- 172.21.15.39:0/1784396831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda2c15cc00 msgr2=0x7fda2c1f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:42.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.223+0000 7fda31800640 1 --2- 172.21.15.39:0/1784396831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda2c15cc00 0x7fda2c1f6950 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7fda2400c700 tx=0x7fda2400cbc0 comp rx=0 tx=0).stop 2024-09-21T07:48:42.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.223+0000 7fda31800640 1 -- 172.21.15.39:0/1784396831 shutdown_connections 2024-09-21T07:48:42.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.223+0000 7fda31800640 1 --2- 172.21.15.39:0/1784396831 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fda0c078b90 0x7fda0c07b0a0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:42.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.223+0000 7fda31800640 1 --2- 172.21.15.39:0/1784396831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda2c15cc00 0x7fda2c1f6950 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:42.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.223+0000 7fda31800640 1 --2- 172.21.15.39:0/1784396831 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda2c0d04b0 0x7fda2c1386e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:42.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.223+0000 7fda31800640 1 --2- 172.21.15.39:0/1784396831 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda2c0cfa20 0x7fda2c0c50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:42.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.223+0000 7fda31800640 1 -- 172.21.15.39:0/1784396831 >> 172.21.15.39:0/1784396831 conn(0x7fda2c06cfc0 msgr2=0x7fda2c157640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:42.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.223+0000 7fda31800640 1 -- 172.21.15.39:0/1784396831 shutdown_connections 2024-09-21T07:48:42.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.223+0000 7fda31800640 1 -- 172.21.15.39:0/1784396831 wait complete. 2024-09-21T07:48:42.248 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T07:48:42.248 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T07:48:42.248 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T07:48:42.248 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:48:42.248 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T07:48:42.249 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T07:48:42.249 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T07:48:42.249 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:48:42.249 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T07:48:42.249 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T07:48:42.249 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T07:48:42.249 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:48:42.249 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T07:48:42.250 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T07:48:42.250 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T07:48:42.250 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:48:42.270 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T07:48:42.271 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T07:48:42.274 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T07:48:42.275 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T07:48:42.275 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T07:48:42.275 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T07:48:42.276 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T07:48:42.276 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T07:48:42.318 INFO:tasks.ceph.mds.a:Started 2024-09-21T07:48:42.318 DEBUG:teuthology.parallel:result is None 2024-09-21T07:48:42.320 INFO:tasks.ceph.mds.b:Started 2024-09-21T07:48:42.320 DEBUG:teuthology.parallel:result is None 2024-09-21T07:48:42.321 INFO:tasks.ceph.mds.c:Started 2024-09-21T07:48:42.321 DEBUG:teuthology.parallel:result is None 2024-09-21T07:48:42.323 INFO:tasks.ceph.mds.d:Started 2024-09-21T07:48:42.323 DEBUG:teuthology.parallel:result is None 2024-09-21T07:48:42.323 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T07:48:42.518 INFO:tasks.ceph.mds.c.smithi039.stdout:starting mds.c at 2024-09-21T07:48:42.518 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-21T07:48:42.522 INFO:tasks.ceph.mds.b.smithi148.stdout:starting mds.b at 2024-09-21T07:48:42.522 INFO:tasks.ceph.mds.d.smithi148.stdout:starting mds.d at 2024-09-21T07:48:42.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.564+0000 7fed9ca00640 1 -- 172.21.15.39:0/4176981522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed7c004800 msgr2=0x7fed7c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:42.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.564+0000 7fed9ca00640 1 --2- 172.21.15.39:0/4176981522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed7c004800 0x7fed7c004c10 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7fed80009b50 tx=0x7fed7c005830 comp rx=0 tx=0).stop 2024-09-21T07:48:42.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.564+0000 7fed96800640 1 -- 172.21.15.39:0/4176981522 reap_dead start 2024-09-21T07:48:42.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.564+0000 7fed9ca00640 1 -- 172.21.15.39:0/4176981522 shutdown_connections 2024-09-21T07:48:42.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.564+0000 7fed9ca00640 1 -- 172.21.15.39:0/4176981522 >> 172.21.15.39:0/4176981522 conn(0x7fed9806cfc0 msgr2=0x7fed9806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:42.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.564+0000 7fed9ca00640 1 -- 172.21.15.39:0/4176981522 shutdown_connections 2024-09-21T07:48:42.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.564+0000 7fed9ca00640 1 -- 172.21.15.39:0/4176981522 wait complete. 2024-09-21T07:48:42.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.567+0000 7fed9ca00640 1 Processor -- start 2024-09-21T07:48:42.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.567+0000 7fed9ca00640 1 -- start start 2024-09-21T07:48:42.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.568+0000 7fed9ca00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fed7c004800 0x7fed980a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:42.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.568+0000 7fed9ca00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed981f6370 0x7fed981f6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:42.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.568+0000 7fed9ca00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed981f6780 0x7fed981f7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:42.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.568+0000 7fed9ca00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fed98070120 con 0x7fed981f6780 2024-09-21T07:48:42.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.568+0000 7fed9ca00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fed980d0630 con 0x7fed981f6370 2024-09-21T07:48:42.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.568+0000 7fed9ca00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fed980d07b0 con 0x7fed7c004800 2024-09-21T07:48:42.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.568+0000 7fed96800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed981f6780 0x7fed981f7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:42.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.568+0000 7fed95e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fed7c004800 0x7fed980a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:42.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.568+0000 7fed96800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed981f6780 0x7fed981f7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51244/0 (socket says 172.21.15.39:51244) 2024-09-21T07:48:42.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.568+0000 7fed8fe00640 1 --2- 172.21.15.39:0/1382603921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed981f6370 0x7fed981f6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:42.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.568+0000 7fed96800640 1 -- 172.21.15.39:0/1382603921 learned_addr learned my addr 172.21.15.39:0/1382603921 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:42.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.569+0000 7fed96800640 1 -- 172.21.15.39:0/1382603921 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fed7c004800 msgr2=0x7fed980a94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:42.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.569+0000 7fed96800640 1 --2- 172.21.15.39:0/1382603921 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fed7c004800 0x7fed980a94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:42.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.569+0000 7fed96800640 1 -- 172.21.15.39:0/1382603921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed981f6370 msgr2=0x7fed981f6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:42.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.569+0000 7fed96800640 1 --2- 172.21.15.39:0/1382603921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed981f6370 0x7fed981f6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:42.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.569+0000 7fed96800640 1 -- 172.21.15.39:0/1382603921 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed981f48f0 con 0x7fed981f6780 2024-09-21T07:48:42.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.569+0000 7fed96800640 1 --2- 172.21.15.39:0/1382603921 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed981f6780 0x7fed981f7260 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7fed980cdbf0 tx=0x7fed900019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:42.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.569+0000 7fed8e000640 1 -- 172.21.15.39:0/1382603921 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed90011870 con 0x7fed981f6780 2024-09-21T07:48:42.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.569+0000 7fed8e000640 1 -- 172.21.15.39:0/1382603921 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed90017070 con 0x7fed981f6780 2024-09-21T07:48:42.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.569+0000 7fed8e000640 1 -- 172.21.15.39:0/1382603921 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed90011dc0 con 0x7fed981f6780 2024-09-21T07:48:42.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.569+0000 7fed9ca00640 1 -- 172.21.15.39:0/1382603921 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed980afb30 con 0x7fed981f6780 2024-09-21T07:48:42.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.570+0000 7fed9ca00640 1 -- 172.21.15.39:0/1382603921 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed981e5430 con 0x7fed981f6780 2024-09-21T07:48:42.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.571+0000 7fed9ca00640 1 -- 172.21.15.39:0/1382603921 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed980af8a0 con 0x7fed981f6780 2024-09-21T07:48:42.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.572+0000 7fed8e000640 1 -- 172.21.15.39:0/1382603921 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fed9001c460 con 0x7fed981f6780 2024-09-21T07:48:42.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.573+0000 7fed8e000640 1 --2- 172.21.15.39:0/1382603921 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fed74078bc0 0x7fed7407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:42.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.574+0000 7fed8e000640 1 -- 172.21.15.39:0/1382603921 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(214..214 src has 1..214) ==== 6308+0+0 (secure 0 0 0) 0x7fed9009b610 con 0x7fed981f6780 2024-09-21T07:48:42.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.574+0000 7fed95e00640 1 --2- 172.21.15.39:0/1382603921 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fed74078bc0 0x7fed7407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:42.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.575+0000 7fed95e00640 1 --2- 172.21.15.39:0/1382603921 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fed74078bc0 0x7fed7407b0d0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fed88000fb0 tx=0x7fed88001310 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:42.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.578+0000 7fed8e000640 1 -- 172.21.15.39:0/1382603921 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fed90063190 con 0x7fed981f6780 2024-09-21T07:48:42.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:42.791+0000 7fed9ca00640 1 -- 172.21.15.39:0/1382603921 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fed980b4cf0 con 0x7fed981f6780 2024-09-21T07:48:43.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.559+0000 7fed8e000640 1 -- 172.21.15.39:0/1382603921 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v215) ==== 97+0+0 (secure 0 0 0) 0x7fed90067bc0 con 0x7fed981f6780 2024-09-21T07:48:43.561 INFO:teuthology.orchestra.run.smithi039.stderr: removed all blocklist entries 2024-09-21T07:48:43.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.563+0000 7fed9ca00640 1 -- 172.21.15.39:0/1382603921 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fed74078bc0 msgr2=0x7fed7407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:43.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.563+0000 7fed9ca00640 1 --2- 172.21.15.39:0/1382603921 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fed74078bc0 0x7fed7407b0d0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fed88000fb0 tx=0x7fed88001310 comp rx=0 tx=0).stop 2024-09-21T07:48:43.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.563+0000 7fed9ca00640 1 -- 172.21.15.39:0/1382603921 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed981f6780 msgr2=0x7fed981f7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:43.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.563+0000 7fed9ca00640 1 --2- 172.21.15.39:0/1382603921 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed981f6780 0x7fed981f7260 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7fed980cdbf0 tx=0x7fed900019b0 comp rx=0 tx=0).stop 2024-09-21T07:48:43.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.563+0000 7fed9ca00640 1 -- 172.21.15.39:0/1382603921 shutdown_connections 2024-09-21T07:48:43.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.563+0000 7fed9ca00640 1 --2- 172.21.15.39:0/1382603921 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fed74078bc0 0x7fed7407b0d0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:43.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.563+0000 7fed9ca00640 1 --2- 172.21.15.39:0/1382603921 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed981f6780 0x7fed981f7260 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:43.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.563+0000 7fed9ca00640 1 --2- 172.21.15.39:0/1382603921 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed981f6370 0x7fed981f6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:43.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.563+0000 7fed9ca00640 1 --2- 172.21.15.39:0/1382603921 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fed7c004800 0x7fed980a94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:43.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.563+0000 7fed9ca00640 1 -- 172.21.15.39:0/1382603921 >> 172.21.15.39:0/1382603921 conn(0x7fed9806cfc0 msgr2=0x7fed980cf560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:43.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.564+0000 7fed9ca00640 1 -- 172.21.15.39:0/1382603921 shutdown_connections 2024-09-21T07:48:43.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.564+0000 7fed9ca00640 1 -- 172.21.15.39:0/1382603921 wait complete. 2024-09-21T07:48:43.589 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T07:48:43.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.819+0000 7fe9c2e00640 1 -- 172.21.15.39:0/3656488702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9bc0d0e60 msgr2=0x7fe9bc0d1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:43.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.819+0000 7fe9c2e00640 1 --2- 172.21.15.39:0/3656488702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9bc0d0e60 0x7fe9bc0d1270 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc0ced20 tx=0x7fe9b402a9d0 comp rx=0 tx=0).stop 2024-09-21T07:48:43.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.819+0000 7fe9c2e00640 1 -- 172.21.15.39:0/3656488702 shutdown_connections 2024-09-21T07:48:43.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.819+0000 7fe9c2e00640 1 --2- 172.21.15.39:0/3656488702 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9bc0b96a0 0x7fe9bc0b59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:43.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.819+0000 7fe9c2e00640 1 --2- 172.21.15.39:0/3656488702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9bc0d0e60 0x7fe9bc0d1270 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:43.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.819+0000 7fe9c2e00640 1 -- 172.21.15.39:0/3656488702 >> 172.21.15.39:0/3656488702 conn(0x7fe9bc09d7d0 msgr2=0x7fe9bc0d4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:43.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.819+0000 7fe9c2e00640 1 -- 172.21.15.39:0/3656488702 shutdown_connections 2024-09-21T07:48:43.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.819+0000 7fe9c2e00640 1 -- 172.21.15.39:0/3656488702 wait complete. 2024-09-21T07:48:43.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.821+0000 7fe9c2e00640 1 Processor -- start 2024-09-21T07:48:43.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.821+0000 7fe9c2e00640 1 -- start start 2024-09-21T07:48:43.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.823+0000 7fe9c2e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9bc0b96a0 0x7fe9bc0a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:43.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.823+0000 7fe9c2e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9bc0d0e60 0x7fe9bc15b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:43.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.823+0000 7fe9c2e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9bc0d5060 0x7fe9bc15b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:43.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.823+0000 7fe9c2e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe9bc0d2c10 con 0x7fe9bc0b96a0 2024-09-21T07:48:43.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.823+0000 7fe9c2e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe9bc0d2a90 con 0x7fe9bc0d0e60 2024-09-21T07:48:43.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.823+0000 7fe9c2e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe9bc0d2d90 con 0x7fe9bc0d5060 2024-09-21T07:48:43.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.823+0000 7fe9c1a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9bc0b96a0 0x7fe9bc0a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:43.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.823+0000 7fe9c1a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9bc0b96a0 0x7fe9bc0a1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51254/0 (socket says 172.21.15.39:51254) 2024-09-21T07:48:43.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.823+0000 7fe9c1a00640 1 -- 172.21.15.39:0/1476445863 learned_addr learned my addr 172.21.15.39:0/1476445863 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:43.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.823+0000 7fe9c2400640 1 --2- 172.21.15.39:0/1476445863 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9bc0d5060 0x7fe9bc15b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:43.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.823+0000 7fe9c1000640 1 --2- 172.21.15.39:0/1476445863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9bc0d0e60 0x7fe9bc15b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:43.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.824+0000 7fe9c1a00640 1 -- 172.21.15.39:0/1476445863 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9bc0d5060 msgr2=0x7fe9bc15b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:43.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.824+0000 7fe9c1a00640 1 --2- 172.21.15.39:0/1476445863 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9bc0d5060 0x7fe9bc15b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:43.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.824+0000 7fe9c1a00640 1 -- 172.21.15.39:0/1476445863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9bc0d0e60 msgr2=0x7fe9bc15b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:43.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.824+0000 7fe9c1a00640 1 --2- 172.21.15.39:0/1476445863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9bc0d0e60 0x7fe9bc15b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:43.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.824+0000 7fe9c1a00640 1 -- 172.21.15.39:0/1476445863 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9bc15be60 con 0x7fe9bc0b96a0 2024-09-21T07:48:43.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.824+0000 7fe9c1000640 1 --2- 172.21.15.39:0/1476445863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9bc0d0e60 0x7fe9bc15b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:48:43.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.824+0000 7fe9c1a00640 1 --2- 172.21.15.39:0/1476445863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9bc0b96a0 0x7fe9bc0a1a60 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7fe9b000b3e0 tx=0x7fe9b000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:43.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.824+0000 7fe9b9600640 1 -- 172.21.15.39:0/1476445863 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0015020 con 0x7fe9bc0b96a0 2024-09-21T07:48:43.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.824+0000 7fe9b9600640 1 -- 172.21.15.39:0/1476445863 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe9b0011940 con 0x7fe9bc0b96a0 2024-09-21T07:48:43.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.824+0000 7fe9c2e00640 1 -- 172.21.15.39:0/1476445863 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9bc14e600 con 0x7fe9bc0b96a0 2024-09-21T07:48:43.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.824+0000 7fe9b9600640 1 -- 172.21.15.39:0/1476445863 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0011dd0 con 0x7fe9bc0b96a0 2024-09-21T07:48:43.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.824+0000 7fe9c2e00640 1 -- 172.21.15.39:0/1476445863 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9bc14ebc0 con 0x7fe9bc0b96a0 2024-09-21T07:48:43.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.826+0000 7fe9c2e00640 1 -- 172.21.15.39:0/1476445863 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9bc14e420 con 0x7fe9bc0b96a0 2024-09-21T07:48:43.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.826+0000 7fe9b9600640 1 -- 172.21.15.39:0/1476445863 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe9b0018020 con 0x7fe9bc0b96a0 2024-09-21T07:48:43.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.828+0000 7fe9b9600640 1 --2- 172.21.15.39:0/1476445863 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe99c078bc0 0x7fe99c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:43.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.828+0000 7fe9b9600640 1 -- 172.21.15.39:0/1476445863 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(215..215 src has 1..215) ==== 6136+0+0 (secure 0 0 0) 0x7fe9b009ad40 con 0x7fe9bc0b96a0 2024-09-21T07:48:43.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.829+0000 7fe9c1000640 1 --2- 172.21.15.39:0/1476445863 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe99c078bc0 0x7fe99c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:43.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.829+0000 7fe9c1000640 1 --2- 172.21.15.39:0/1476445863 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe99c078bc0 0x7fe99c07b0d0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8005fd0 tx=0x7fe9a8005ea0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:43.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:43.833+0000 7fe9b9600640 1 -- 172.21.15.39:0/1476445863 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe9bc14e420 con 0x7fe9bc0b96a0 2024-09-21T07:48:44.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.107+0000 7fe9c2e00640 1 -- 172.21.15.39:0/1476445863 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fe9bc0d3240 con 0x7fe9bc0b96a0 2024-09-21T07:48:44.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.107+0000 7fe9b9600640 1 -- 172.21.15.39:0/1476445863 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) ==== 80+0+5304 (secure 0 0 0) 0x7fe9b0062eb0 con 0x7fe9bc0b96a0 2024-09-21T07:48:44.108 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:48:44.108 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:48:44.109 INFO:teuthology.orchestra.run.smithi039.stdout: "auth_dump": [ 2024-09-21T07:48:44.109 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:48:44.109 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.a", 2024-09-21T07:48:44.109 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5m+P6EERAABQqmlkrocSi1+hwc73Adrg==", 2024-09-21T07:48:44.109 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:48:44.109 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:48:44.109 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:48:44.109 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:48:44.109 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:48:44.109 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:48:44.109 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:48:44.109 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:48:44.109 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.b", 2024-09-21T07:48:44.109 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5m5/GeLRAAfE8fr5dbitQa89rNK9pZ2Q==", 2024-09-21T07:48:44.110 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:48:44.110 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:48:44.110 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:48:44.110 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:48:44.110 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:48:44.110 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:48:44.110 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:48:44.110 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:48:44.110 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.c", 2024-09-21T07:48:44.110 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5msbx6HxAALsF8ORvIoJpIAFZ5uxmOEw==", 2024-09-21T07:48:44.110 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:48:44.110 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:48:44.111 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:48:44.111 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:48:44.111 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:48:44.111 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:48:44.111 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:48:44.111 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:48:44.111 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.d", 2024-09-21T07:48:44.111 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mIrcBABAApKAB/7FkZ5glNTxlbkIsBw==", 2024-09-21T07:48:44.111 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:48:44.112 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:48:44.112 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:48:44.112 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:48:44.112 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:48:44.112 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:48:44.112 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:48:44.112 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:48:44.112 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.0", 2024-09-21T07:48:44.112 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB7du5msw81DhAAmVimqN4x9K3YpT0nbovppQ==", 2024-09-21T07:48:44.112 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:48:44.112 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:48:44.112 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:48:44.112 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:48:44.112 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:48:44.112 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:48:44.113 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:48:44.113 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.1", 2024-09-21T07:48:44.113 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB+du5mdnCIOhAAtVB4ThehQwb0zRriyKcaxA==", 2024-09-21T07:48:44.113 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:48:44.113 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:48:44.113 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:48:44.113 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:48:44.113 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:48:44.113 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:48:44.113 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:48:44.113 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.2", 2024-09-21T07:48:44.113 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCCdu5mTfWxDhAAadJCbPN5xBx/f9L0v2yTdg==", 2024-09-21T07:48:44.113 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:48:44.113 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:48:44.114 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:48:44.114 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:48:44.114 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:48:44.114 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:48:44.114 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:48:44.114 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.3", 2024-09-21T07:48:44.114 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCFdu5mL1E2HxAAyOhY97uUXEw5L1CcDeONVA==", 2024-09-21T07:48:44.114 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:48:44.114 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:48:44.114 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:48:44.114 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:48:44.114 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:48:44.114 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:48:44.115 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:48:44.115 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.4", 2024-09-21T07:48:44.115 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCLdu5m6PY/IhAAgvcTgm+6uWA7lIG4tcTIEg==", 2024-09-21T07:48:44.115 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:48:44.115 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:48:44.115 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:48:44.115 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:48:44.115 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:48:44.115 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:48:44.115 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:48:44.115 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.5", 2024-09-21T07:48:44.115 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCOdu5mE/wlMxAA3JBg4xH+N/PPC1Uv2ZVrYQ==", 2024-09-21T07:48:44.115 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:48:44.116 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:48:44.116 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:48:44.116 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:48:44.116 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:48:44.116 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:48:44.116 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:48:44.116 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.6", 2024-09-21T07:48:44.116 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCSdu5mC3TgBxAAwsArjeIdFcApY3/1OJffEg==", 2024-09-21T07:48:44.116 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:48:44.116 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:48:44.116 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:48:44.116 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:48:44.117 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:48:44.117 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:48:44.117 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:48:44.117 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.7", 2024-09-21T07:48:44.117 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCVdu5mSWHWFBAA9qPHWRdsvrB74inrp1lj4Q==", 2024-09-21T07:48:44.117 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:48:44.117 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:48:44.117 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:48:44.117 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:48:44.117 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:48:44.117 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:48:44.117 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:48:44.117 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.0", 2024-09-21T07:48:44.117 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mGfb1DBAA/aSMKR5or3xy53rB/iUjpA==", 2024-09-21T07:48:44.117 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:48:44.117 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:48:44.118 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow r", 2024-09-21T07:48:44.118 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T07:48:44.118 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:48:44.118 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:48:44.118 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:48:44.118 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.1", 2024-09-21T07:48:44.118 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mEkciIhAAid+bxJEr4P7vMnZ42PAy5w==", 2024-09-21T07:48:44.118 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:48:44.118 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:48:44.118 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow r", 2024-09-21T07:48:44.118 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow rw", 2024-09-21T07:48:44.118 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rwx" 2024-09-21T07:48:44.118 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:48:44.118 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:48:44.119 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:48:44.119 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.2", 2024-09-21T07:48:44.119 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mSu2oFxAAgjy2m/wWKh6xaoh+YDoU1Q==", 2024-09-21T07:48:44.119 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:48:44.119 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:48:44.119 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow r", 2024-09-21T07:48:44.119 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow rw", 2024-09-21T07:48:44.119 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rwx" 2024-09-21T07:48:44.119 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:48:44.119 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:48:44.119 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:48:44.119 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.3", 2024-09-21T07:48:44.119 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mCHy0LBAAZneF2jUWgfuyGehZFbOxDg==", 2024-09-21T07:48:44.120 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:48:44.120 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:48:44.120 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow r", 2024-09-21T07:48:44.120 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow rw", 2024-09-21T07:48:44.120 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rwx" 2024-09-21T07:48:44.120 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:48:44.120 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:48:44.120 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:48:44.120 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.admin", 2024-09-21T07:48:44.120 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBydu5m9LInERAAAlDpqb94f2Qb9VYJJwTBcA==", 2024-09-21T07:48:44.120 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:48:44.120 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow *", 2024-09-21T07:48:44.120 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:48:44.120 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:48:44.120 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:48:44.121 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:48:44.121 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:48:44.121 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:48:44.121 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mgr.x", 2024-09-21T07:48:44.121 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5mA2GRBhAAkyCKdgH6laHy8xNt080dXg==", 2024-09-21T07:48:44.121 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:48:44.121 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow *", 2024-09-21T07:48:44.121 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile mgr", 2024-09-21T07:48:44.121 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:48:44.121 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:48:44.121 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:48:44.121 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:48:44.121 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mgr.y", 2024-09-21T07:48:44.121 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBydu5mr+8aNxAA7TFYNms2WsXUfa4ITTJOHw==", 2024-09-21T07:48:44.121 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:48:44.122 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow *", 2024-09-21T07:48:44.122 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile mgr", 2024-09-21T07:48:44.122 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:48:44.122 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:48:44.122 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:48:44.122 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-21T07:48:44.122 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:48:44.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.110+0000 7fe9c2e00640 1 -- 172.21.15.39:0/1476445863 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe99c078bc0 msgr2=0x7fe99c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:44.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.110+0000 7fe9c2e00640 1 --2- 172.21.15.39:0/1476445863 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe99c078bc0 0x7fe99c07b0d0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8005fd0 tx=0x7fe9a8005ea0 comp rx=0 tx=0).stop 2024-09-21T07:48:44.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.110+0000 7fe9c2e00640 1 -- 172.21.15.39:0/1476445863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9bc0b96a0 msgr2=0x7fe9bc0a1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:44.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.110+0000 7fe9c2e00640 1 --2- 172.21.15.39:0/1476445863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9bc0b96a0 0x7fe9bc0a1a60 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7fe9b000b3e0 tx=0x7fe9b000b8a0 comp rx=0 tx=0).stop 2024-09-21T07:48:44.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.110+0000 7fe9c2e00640 1 -- 172.21.15.39:0/1476445863 shutdown_connections 2024-09-21T07:48:44.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.110+0000 7fe9c2e00640 1 --2- 172.21.15.39:0/1476445863 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe99c078bc0 0x7fe99c07b0d0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:44.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.110+0000 7fe9c2e00640 1 --2- 172.21.15.39:0/1476445863 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe9bc0d5060 0x7fe9bc15b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:44.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.110+0000 7fe9c2e00640 1 --2- 172.21.15.39:0/1476445863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe9bc0d0e60 0x7fe9bc15b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:44.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.110+0000 7fe9c2e00640 1 --2- 172.21.15.39:0/1476445863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9bc0b96a0 0x7fe9bc0a1a60 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:44.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.110+0000 7fe9c2e00640 1 -- 172.21.15.39:0/1476445863 >> 172.21.15.39:0/1476445863 conn(0x7fe9bc09d7d0 msgr2=0x7fe9bc06f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:44.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.110+0000 7fe9c2e00640 1 -- 172.21.15.39:0/1476445863 shutdown_connections 2024-09-21T07:48:44.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.110+0000 7fe9c2e00640 1 -- 172.21.15.39:0/1476445863 wait complete. 2024-09-21T07:48:44.137 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T07:48:44.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.364+0000 7f92b6800640 1 -- 172.21.15.39:0/753049206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9294005150 msgr2=0x7f929400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:44.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.364+0000 7f92b6800640 1 --2- 172.21.15.39:0/753049206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9294005150 0x7f929400b9c0 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f92a800c610 tx=0x7f9294005830 comp rx=0 tx=0).stop 2024-09-21T07:48:44.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.364+0000 7f92afe00640 1 -- 172.21.15.39:0/753049206 reap_dead start 2024-09-21T07:48:44.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.364+0000 7f92b6800640 1 -- 172.21.15.39:0/753049206 shutdown_connections 2024-09-21T07:48:44.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.364+0000 7f92b6800640 1 -- 172.21.15.39:0/753049206 >> 172.21.15.39:0/753049206 conn(0x7f92b00b2460 msgr2=0x7f92b006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:44.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.364+0000 7f92b6800640 1 -- 172.21.15.39:0/753049206 shutdown_connections 2024-09-21T07:48:44.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.364+0000 7f92b6800640 1 -- 172.21.15.39:0/753049206 wait complete. 2024-09-21T07:48:44.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.366+0000 7f92b6800640 1 Processor -- start 2024-09-21T07:48:44.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.366+0000 7f92b6800640 1 -- start start 2024-09-21T07:48:44.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.368+0000 7f92b6800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9294005150 0x7f92b0150350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:44.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.368+0000 7f92b6800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92b00d0b50 0x7f92b00cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:44.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.368+0000 7f92b6800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92b0156020 0x7f92b00c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:44.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.368+0000 7f92b6800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f92b00cbc30 con 0x7f92b0156020 2024-09-21T07:48:44.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.368+0000 7f92b6800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f92b00cbab0 con 0x7f9294005150 2024-09-21T07:48:44.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.368+0000 7f92b6800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f92b00af080 con 0x7f92b00d0b50 2024-09-21T07:48:44.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.368+0000 7f92afe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92b0156020 0x7f92b00c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:44.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.368+0000 7f92afe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92b0156020 0x7f92b00c9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51270/0 (socket says 172.21.15.39:51270) 2024-09-21T07:48:44.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.368+0000 7f92afe00640 1 -- 172.21.15.39:0/322804039 learned_addr learned my addr 172.21.15.39:0/322804039 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:44.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.368+0000 7f92af400640 1 --2- 172.21.15.39:0/322804039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9294005150 0x7f92b0150350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:44.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.368+0000 7f92a7e00640 1 --2- 172.21.15.39:0/322804039 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92b00d0b50 0x7f92b00cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:44.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.369+0000 7f92afe00640 1 -- 172.21.15.39:0/322804039 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92b00d0b50 msgr2=0x7f92b00cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:44.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.369+0000 7f92afe00640 1 --2- 172.21.15.39:0/322804039 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92b00d0b50 0x7f92b00cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:44.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.369+0000 7f92afe00640 1 -- 172.21.15.39:0/322804039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9294005150 msgr2=0x7f92b0150350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:44.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.369+0000 7f92afe00640 1 --2- 172.21.15.39:0/322804039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9294005150 0x7f92b0150350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:44.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.369+0000 7f92afe00640 1 -- 172.21.15.39:0/322804039 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b00ce4c0 con 0x7f92b0156020 2024-09-21T07:48:44.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.369+0000 7f92afe00640 1 --2- 172.21.15.39:0/322804039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92b0156020 0x7f92b00c9aa0 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f92a802b450 tx=0x7f92a8001aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:44.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.369+0000 7f92a7400640 1 -- 172.21.15.39:0/322804039 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a8039070 con 0x7f92b0156020 2024-09-21T07:48:44.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.369+0000 7f92a7400640 1 -- 172.21.15.39:0/322804039 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f92a8006590 con 0x7f92b0156020 2024-09-21T07:48:44.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.369+0000 7f92a7400640 1 -- 172.21.15.39:0/322804039 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a8002030 con 0x7f92b0156020 2024-09-21T07:48:44.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.369+0000 7f92b6800640 1 -- 172.21.15.39:0/322804039 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92b00c75e0 con 0x7f92b0156020 2024-09-21T07:48:44.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.369+0000 7f92b6800640 1 -- 172.21.15.39:0/322804039 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92b00c92a0 con 0x7f92b0156020 2024-09-21T07:48:44.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.371+0000 7f92b6800640 1 -- 172.21.15.39:0/322804039 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92b00c8db0 con 0x7f92b0156020 2024-09-21T07:48:44.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.372+0000 7f92a7400640 1 -- 172.21.15.39:0/322804039 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f92a803c020 con 0x7f92b0156020 2024-09-21T07:48:44.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.373+0000 7f92a7400640 1 --2- 172.21.15.39:0/322804039 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f928c078b90 0x7f928c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:44.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.373+0000 7f92a7400640 1 -- 172.21.15.39:0/322804039 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(215..215 src has 1..215) ==== 6136+0+0 (secure 0 0 0) 0x7f92a80ba290 con 0x7f92b0156020 2024-09-21T07:48:44.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.373+0000 7f92af400640 1 --2- 172.21.15.39:0/322804039 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f928c078b90 0x7f928c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:44.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.374+0000 7f92af400640 1 --2- 172.21.15.39:0/322804039 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f928c078b90 0x7f928c07b0a0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f92b0071e60 tx=0x7f92a0005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:44.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.378+0000 7f92a7400640 1 -- 172.21.15.39:0/322804039 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f92a8045030 con 0x7f92b0156020 2024-09-21T07:48:44.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.575+0000 7f92b6800640 1 -- 172.21.15.39:0/322804039 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f92b0156bb0 con 0x7f92b0156020 2024-09-21T07:48:44.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.576+0000 7f92a7400640 1 -- 172.21.15.39:0/322804039 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v215) ==== 84+0+287 (secure 0 0 0) 0x7f92a8082780 con 0x7f92b0156020 2024-09-21T07:48:44.577 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:48:44.577 INFO:teuthology.orchestra.run.smithi039.stdout:[ 2024-09-21T07:48:44.577 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:48:44.577 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 1, 2024-09-21T07:48:44.577 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": ".mgr" 2024-09-21T07:48:44.577 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:48:44.577 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:48:44.577 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 6, 2024-09-21T07:48:44.577 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": "snap-test-pool" 2024-09-21T07:48:44.577 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:48:44.577 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:48:44.578 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 11, 2024-09-21T07:48:44.578 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": "test_data_pool" 2024-09-21T07:48:44.578 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:48:44.578 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:48:44.578 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 12, 2024-09-21T07:48:44.578 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": "test_metadata_pool" 2024-09-21T07:48:44.578 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:48:44.578 INFO:teuthology.orchestra.run.smithi039.stdout:] 2024-09-21T07:48:44.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.579+0000 7f92b6800640 1 -- 172.21.15.39:0/322804039 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f928c078b90 msgr2=0x7f928c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:44.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.579+0000 7f92b6800640 1 --2- 172.21.15.39:0/322804039 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f928c078b90 0x7f928c07b0a0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f92b0071e60 tx=0x7f92a0005f90 comp rx=0 tx=0).stop 2024-09-21T07:48:44.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.579+0000 7f92b6800640 1 -- 172.21.15.39:0/322804039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92b0156020 msgr2=0x7f92b00c9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:44.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.579+0000 7f92b6800640 1 --2- 172.21.15.39:0/322804039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92b0156020 0x7f92b00c9aa0 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f92a802b450 tx=0x7f92a8001aa0 comp rx=0 tx=0).stop 2024-09-21T07:48:44.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.579+0000 7f92b6800640 1 -- 172.21.15.39:0/322804039 shutdown_connections 2024-09-21T07:48:44.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.579+0000 7f92b6800640 1 --2- 172.21.15.39:0/322804039 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f928c078b90 0x7f928c07b0a0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:44.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.579+0000 7f92b6800640 1 --2- 172.21.15.39:0/322804039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92b0156020 0x7f92b00c9aa0 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:44.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.579+0000 7f92b6800640 1 --2- 172.21.15.39:0/322804039 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f92b00d0b50 0x7f92b00cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:44.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.579+0000 7f92b6800640 1 --2- 172.21.15.39:0/322804039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9294005150 0x7f92b0150350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:44.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.579+0000 7f92b6800640 1 -- 172.21.15.39:0/322804039 >> 172.21.15.39:0/322804039 conn(0x7f92b00b2460 msgr2=0x7f92b015f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:44.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.579+0000 7f92b6800640 1 -- 172.21.15.39:0/322804039 shutdown_connections 2024-09-21T07:48:44.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.579+0000 7f92b6800640 1 -- 172.21.15.39:0/322804039 wait complete. 2024-09-21T07:48:44.601 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T07:48:44.601 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T07:48:44.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.833+0000 7f14f5e00640 1 -- 172.21.15.39:0/364430311 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14d4005280 msgr2=0x7f14d400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:44.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.833+0000 7f14f5e00640 1 --2- 172.21.15.39:0/364430311 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14d4005280 0x7f14d400b9c0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f14e8001f30 tx=0x7f14d4005830 comp rx=0 tx=0).stop 2024-09-21T07:48:44.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.833+0000 7f14f5e00640 1 -- 172.21.15.39:0/364430311 shutdown_connections 2024-09-21T07:48:44.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.833+0000 7f14f5e00640 1 --2- 172.21.15.39:0/364430311 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14d4005280 0x7f14d400b9c0 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:44.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.833+0000 7f14f5e00640 1 -- 172.21.15.39:0/364430311 >> 172.21.15.39:0/364430311 conn(0x7f14f00b2640 msgr2=0x7f14f00d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:44.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.833+0000 7f14f5e00640 1 -- 172.21.15.39:0/364430311 shutdown_connections 2024-09-21T07:48:44.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.833+0000 7f14f5e00640 1 -- 172.21.15.39:0/364430311 wait complete. 2024-09-21T07:48:44.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.835+0000 7f14f5e00640 1 Processor -- start 2024-09-21T07:48:44.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.835+0000 7f14f5e00640 1 -- start start 2024-09-21T07:48:44.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.837+0000 7f14f5e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14d4005280 0x7f14f00ae120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:44.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.837+0000 7f14f5e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14f0099c50 0x7f14f00ae660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:44.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.837+0000 7f14f5e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f14f015b500 0x7f14f01f7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:44.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.837+0000 7f14f5e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f14f00cd670 con 0x7f14d4005280 2024-09-21T07:48:44.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.837+0000 7f14f5e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f14f00cd4f0 con 0x7f14f0099c50 2024-09-21T07:48:44.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.837+0000 7f14f5e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f14f006f020 con 0x7f14f015b500 2024-09-21T07:48:44.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.837+0000 7f14eea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14d4005280 0x7f14f00ae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:44.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.837+0000 7f14eea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14d4005280 0x7f14f00ae120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51292/0 (socket says 172.21.15.39:51292) 2024-09-21T07:48:44.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.837+0000 7f14eea00640 1 -- 172.21.15.39:0/1584675380 learned_addr learned my addr 172.21.15.39:0/1584675380 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:44.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.837+0000 7f14e7e00640 1 --2- 172.21.15.39:0/1584675380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14f0099c50 0x7f14f00ae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:44.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.837+0000 7f14ef400640 1 --2- 172.21.15.39:0/1584675380 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f14f015b500 0x7f14f01f7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:44.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.838+0000 7f14eea00640 1 -- 172.21.15.39:0/1584675380 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f14f015b500 msgr2=0x7f14f01f7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:44.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.838+0000 7f14eea00640 1 --2- 172.21.15.39:0/1584675380 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f14f015b500 0x7f14f01f7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:44.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.838+0000 7f14eea00640 1 -- 172.21.15.39:0/1584675380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14f0099c50 msgr2=0x7f14f00ae660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:44.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.838+0000 7f14eea00640 1 --2- 172.21.15.39:0/1584675380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14f0099c50 0x7f14f00ae660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:44.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.838+0000 7f14eea00640 1 -- 172.21.15.39:0/1584675380 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f00ccd00 con 0x7f14d4005280 2024-09-21T07:48:44.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.838+0000 7f14eea00640 1 --2- 172.21.15.39:0/1584675380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14d4005280 0x7f14f00ae120 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f14e0011ae0 tx=0x7f14e0009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:44.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.838+0000 7f14e7400640 1 -- 172.21.15.39:0/1584675380 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e001e070 con 0x7f14d4005280 2024-09-21T07:48:44.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.838+0000 7f14e7400640 1 -- 172.21.15.39:0/1584675380 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14e0004480 con 0x7f14d4005280 2024-09-21T07:48:44.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.838+0000 7f14e7400640 1 -- 172.21.15.39:0/1584675380 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e000fc60 con 0x7f14d4005280 2024-09-21T07:48:44.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.838+0000 7f14f5e00640 1 -- 172.21.15.39:0/1584675380 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14f0155b00 con 0x7f14d4005280 2024-09-21T07:48:44.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.838+0000 7f14f5e00640 1 -- 172.21.15.39:0/1584675380 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14f015f490 con 0x7f14d4005280 2024-09-21T07:48:44.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.839+0000 7f14f5e00640 1 -- 172.21.15.39:0/1584675380 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14f015cc30 con 0x7f14d4005280 2024-09-21T07:48:44.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.841+0000 7f14e7400640 1 -- 172.21.15.39:0/1584675380 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f14e0021070 con 0x7f14d4005280 2024-09-21T07:48:44.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.842+0000 7f14e7400640 1 --2- 172.21.15.39:0/1584675380 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f14cc078b90 0x7f14cc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:44.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.842+0000 7f14e7400640 1 -- 172.21.15.39:0/1584675380 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(215..215 src has 1..215) ==== 6136+0+0 (secure 0 0 0) 0x7f14e009b850 con 0x7f14d4005280 2024-09-21T07:48:44.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.842+0000 7f14e7e00640 1 --2- 172.21.15.39:0/1584675380 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f14cc078b90 0x7f14cc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:44.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.843+0000 7f14e7e00640 1 --2- 172.21.15.39:0/1584675380 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f14cc078b90 0x7f14cc07b0a0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f14f00af300 tx=0x7f14d8000ed0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:44.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:44.846+0000 7f14e7400640 1 -- 172.21.15.39:0/1584675380 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f14e00635b0 con 0x7f14d4005280 2024-09-21T07:48:45.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.046+0000 7f14f5e00640 1 -- 172.21.15.39:0/1584675380 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f14f00753d0 con 0x7f14d4005280 2024-09-21T07:48:45.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.575+0000 7f14e7400640 1 -- 172.21.15.39:0/1584675380 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v216) ==== 138+0+0 (secure 0 0 0) 0x7f14e0067fe0 con 0x7f14d4005280 2024-09-21T07:48:45.576 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' created 2024-09-21T07:48:45.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.579+0000 7f14f5e00640 1 -- 172.21.15.39:0/1584675380 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f14cc078b90 msgr2=0x7f14cc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:45.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.579+0000 7f14f5e00640 1 --2- 172.21.15.39:0/1584675380 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f14cc078b90 0x7f14cc07b0a0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f14f00af300 tx=0x7f14d8000ed0 comp rx=0 tx=0).stop 2024-09-21T07:48:45.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.579+0000 7f14f5e00640 1 -- 172.21.15.39:0/1584675380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14d4005280 msgr2=0x7f14f00ae120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:45.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.579+0000 7f14f5e00640 1 --2- 172.21.15.39:0/1584675380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14d4005280 0x7f14f00ae120 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f14e0011ae0 tx=0x7f14e0009e40 comp rx=0 tx=0).stop 2024-09-21T07:48:45.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.579+0000 7f14f5e00640 1 -- 172.21.15.39:0/1584675380 shutdown_connections 2024-09-21T07:48:45.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.579+0000 7f14f5e00640 1 --2- 172.21.15.39:0/1584675380 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f14cc078b90 0x7f14cc07b0a0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:45.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.579+0000 7f14f5e00640 1 --2- 172.21.15.39:0/1584675380 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f14f015b500 0x7f14f01f7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:45.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.579+0000 7f14f5e00640 1 --2- 172.21.15.39:0/1584675380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14f0099c50 0x7f14f00ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:45.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.579+0000 7f14f5e00640 1 --2- 172.21.15.39:0/1584675380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14d4005280 0x7f14f00ae120 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:45.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.579+0000 7f14f5e00640 1 -- 172.21.15.39:0/1584675380 >> 172.21.15.39:0/1584675380 conn(0x7f14f00b2640 msgr2=0x7f14f015dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:45.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.580+0000 7f14f5e00640 1 -- 172.21.15.39:0/1584675380 shutdown_connections 2024-09-21T07:48:45.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.580+0000 7f14f5e00640 1 -- 172.21.15.39:0/1584675380 wait complete. 2024-09-21T07:48:45.608 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T07:48:45.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.843+0000 7fe022a00640 1 -- 172.21.15.39:0/4118125683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe01c0cfb60 msgr2=0x7fe01c1674a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:45.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.843+0000 7fe022a00640 1 --2- 172.21.15.39:0/4118125683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe01c0cfb60 0x7fe01c1674a0 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7fe014005fd0 tx=0x7fe01402a820 comp rx=0 tx=0).stop 2024-09-21T07:48:45.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.844+0000 7fe022a00640 1 -- 172.21.15.39:0/4118125683 shutdown_connections 2024-09-21T07:48:45.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.844+0000 7fe022a00640 1 --2- 172.21.15.39:0/4118125683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe01c0cfb60 0x7fe01c1674a0 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:45.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.844+0000 7fe022a00640 1 -- 172.21.15.39:0/4118125683 >> 172.21.15.39:0/4118125683 conn(0x7fe01c06cfc0 msgr2=0x7fe01c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:45.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.844+0000 7fe022a00640 1 -- 172.21.15.39:0/4118125683 shutdown_connections 2024-09-21T07:48:45.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.844+0000 7fe022a00640 1 -- 172.21.15.39:0/4118125683 wait complete. 2024-09-21T07:48:45.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.846+0000 7fe022a00640 1 Processor -- start 2024-09-21T07:48:45.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.846+0000 7fe022a00640 1 -- start start 2024-09-21T07:48:45.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.847+0000 7fe022a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe01c070c30 0x7fe01c1f5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:45.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.848+0000 7fe022a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe01c0cfb60 0x7fe01c14d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:45.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.848+0000 7fe022a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe01c1f8ac0 0x7fe01c15b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:45.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.848+0000 7fe022a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe01c167d70 con 0x7fe01c070c30 2024-09-21T07:48:45.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.848+0000 7fe022a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe01c167bf0 con 0x7fe01c0cfb60 2024-09-21T07:48:45.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.848+0000 7fe022a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe01c167ef0 con 0x7fe01c1f8ac0 2024-09-21T07:48:45.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.848+0000 7fe01be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe01c070c30 0x7fe01c1f5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:45.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.848+0000 7fe01be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe01c070c30 0x7fe01c1f5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51312/0 (socket says 172.21.15.39:51312) 2024-09-21T07:48:45.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.848+0000 7fe01be00640 1 -- 172.21.15.39:0/592792034 learned_addr learned my addr 172.21.15.39:0/592792034 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:45.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.848+0000 7fe01b400640 1 --2- 172.21.15.39:0/592792034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe01c0cfb60 0x7fe01c14d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:45.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.848+0000 7fe020a00640 1 --2- 172.21.15.39:0/592792034 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe01c1f8ac0 0x7fe01c15b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:45.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.848+0000 7fe01be00640 1 -- 172.21.15.39:0/592792034 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe01c1f8ac0 msgr2=0x7fe01c15b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:45.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.848+0000 7fe01be00640 1 --2- 172.21.15.39:0/592792034 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe01c1f8ac0 0x7fe01c15b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:45.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.848+0000 7fe01be00640 1 -- 172.21.15.39:0/592792034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe01c0cfb60 msgr2=0x7fe01c14d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:45.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.848+0000 7fe01be00640 1 --2- 172.21.15.39:0/592792034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe01c0cfb60 0x7fe01c14d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:45.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.848+0000 7fe01be00640 1 -- 172.21.15.39:0/592792034 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe01c1f9230 con 0x7fe01c070c30 2024-09-21T07:48:45.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.848+0000 7fe020a00640 1 --2- 172.21.15.39:0/592792034 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe01c1f8ac0 0x7fe01c15b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:48:45.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.849+0000 7fe01be00640 1 --2- 172.21.15.39:0/592792034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe01c070c30 0x7fe01c1f5510 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7fe00c00c3a0 tx=0x7fe00c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:45.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.849+0000 7fe013e00640 1 -- 172.21.15.39:0/592792034 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00c011790 con 0x7fe01c070c30 2024-09-21T07:48:45.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.849+0000 7fe013e00640 1 -- 172.21.15.39:0/592792034 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe00c017070 con 0x7fe01c070c30 2024-09-21T07:48:45.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.849+0000 7fe013e00640 1 -- 172.21.15.39:0/592792034 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00c011cc0 con 0x7fe01c070c30 2024-09-21T07:48:45.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.849+0000 7fe022a00640 1 -- 172.21.15.39:0/592792034 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe01c1f9510 con 0x7fe01c070c30 2024-09-21T07:48:45.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.849+0000 7fe022a00640 1 -- 172.21.15.39:0/592792034 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe01c14c170 con 0x7fe01c070c30 2024-09-21T07:48:45.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.850+0000 7fe022a00640 1 -- 172.21.15.39:0/592792034 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe01c0cf450 con 0x7fe01c070c30 2024-09-21T07:48:45.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.852+0000 7fe013e00640 1 -- 172.21.15.39:0/592792034 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe00c01c460 con 0x7fe01c070c30 2024-09-21T07:48:45.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.853+0000 7fe013e00640 1 --2- 172.21.15.39:0/592792034 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdffc078bc0 0x7fdffc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:45.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.853+0000 7fe013e00640 1 -- 172.21.15.39:0/592792034 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(216..216 src has 1..216) ==== 6506+0+0 (secure 0 0 0) 0x7fe00c09b910 con 0x7fe01c070c30 2024-09-21T07:48:45.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.853+0000 7fe01b400640 1 --2- 172.21.15.39:0/592792034 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdffc078bc0 0x7fdffc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:45.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.854+0000 7fe01b400640 1 --2- 172.21.15.39:0/592792034 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdffc078bc0 0x7fdffc07b0d0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe008006fd0 tx=0x7fe008008040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:45.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:45.857+0000 7fe013e00640 1 -- 172.21.15.39:0/592792034 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe00c063a20 con 0x7fe01c070c30 2024-09-21T07:48:46.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.054+0000 7fe022a00640 1 -- 172.21.15.39:0/592792034 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe01c1503b0 con 0x7fe01c070c30 2024-09-21T07:48:46.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.581+0000 7fe013e00640 1 -- 172.21.15.39:0/592792034 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v217) ==== 170+0+0 (secure 0 0 0) 0x7fe00c063320 con 0x7fe01c070c30 2024-09-21T07:48:46.584 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' created 2024-09-21T07:48:46.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.585+0000 7fe022a00640 1 -- 172.21.15.39:0/592792034 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdffc078bc0 msgr2=0x7fdffc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:46.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.585+0000 7fe022a00640 1 --2- 172.21.15.39:0/592792034 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdffc078bc0 0x7fdffc07b0d0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe008006fd0 tx=0x7fe008008040 comp rx=0 tx=0).stop 2024-09-21T07:48:46.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.585+0000 7fe022a00640 1 -- 172.21.15.39:0/592792034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe01c070c30 msgr2=0x7fe01c1f5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:46.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.585+0000 7fe022a00640 1 --2- 172.21.15.39:0/592792034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe01c070c30 0x7fe01c1f5510 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7fe00c00c3a0 tx=0x7fe00c00c860 comp rx=0 tx=0).stop 2024-09-21T07:48:46.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.585+0000 7fe022a00640 1 -- 172.21.15.39:0/592792034 shutdown_connections 2024-09-21T07:48:46.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.585+0000 7fe022a00640 1 --2- 172.21.15.39:0/592792034 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdffc078bc0 0x7fdffc07b0d0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:46.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.585+0000 7fe022a00640 1 --2- 172.21.15.39:0/592792034 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe01c1f8ac0 0x7fe01c15b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:46.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.585+0000 7fe022a00640 1 --2- 172.21.15.39:0/592792034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe01c0cfb60 0x7fe01c14d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:46.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.585+0000 7fe022a00640 1 --2- 172.21.15.39:0/592792034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe01c070c30 0x7fe01c1f5510 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:46.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.585+0000 7fe022a00640 1 -- 172.21.15.39:0/592792034 >> 172.21.15.39:0/592792034 conn(0x7fe01c06cfc0 msgr2=0x7fe01c14f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:46.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.585+0000 7fe022a00640 1 -- 172.21.15.39:0/592792034 shutdown_connections 2024-09-21T07:48:46.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.585+0000 7fe022a00640 1 -- 172.21.15.39:0/592792034 wait complete. 2024-09-21T07:48:46.628 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T07:48:46.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.855+0000 7fafc1c00640 1 -- 172.21.15.39:0/12106432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafbc0d0630 msgr2=0x7fafbc0d0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:46.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.855+0000 7fafc1c00640 1 --2- 172.21.15.39:0/12106432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafbc0d0630 0x7fafbc0d0a40 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7fafb4005fd0 tx=0x7fafb402aa90 comp rx=0 tx=0).stop 2024-09-21T07:48:46.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.856+0000 7fafc1c00640 1 -- 172.21.15.39:0/12106432 shutdown_connections 2024-09-21T07:48:46.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.856+0000 7fafc1c00640 1 --2- 172.21.15.39:0/12106432 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fafbc0cf450 0x7fafbc0cf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:46.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.856+0000 7fafc1c00640 1 --2- 172.21.15.39:0/12106432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafbc0d0630 0x7fafbc0d0a40 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:46.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.856+0000 7fafc1c00640 1 -- 172.21.15.39:0/12106432 >> 172.21.15.39:0/12106432 conn(0x7fafbc06cfc0 msgr2=0x7fafbc06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:46.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.856+0000 7fafc1c00640 1 -- 172.21.15.39:0/12106432 shutdown_connections 2024-09-21T07:48:46.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.856+0000 7fafc1c00640 1 -- 172.21.15.39:0/12106432 wait complete. 2024-09-21T07:48:46.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.858+0000 7fafc1c00640 1 Processor -- start 2024-09-21T07:48:46.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.858+0000 7fafc1c00640 1 -- start start 2024-09-21T07:48:46.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.859+0000 7fafc1c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafbc073dd0 0x7fafbc154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:46.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.859+0000 7fafc1c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fafbc0cf450 0x7fafbc1e92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:46.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.859+0000 7fafc1c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafbc0d0630 0x7fafbc1e9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:46.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.860+0000 7fafc1c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc0cd230 con 0x7fafbc0d0630 2024-09-21T07:48:46.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.860+0000 7fafc1c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc0cd0b0 con 0x7fafbc073dd0 2024-09-21T07:48:46.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.860+0000 7fafc1c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fafbc0cd3b0 con 0x7fafbc0cf450 2024-09-21T07:48:46.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.860+0000 7fafbb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafbc0d0630 0x7fafbc1e9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:46.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.860+0000 7fafbb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafbc0d0630 0x7fafbc1e9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51338/0 (socket says 172.21.15.39:51338) 2024-09-21T07:48:46.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.860+0000 7fafbb400640 1 -- 172.21.15.39:0/1875865089 learned_addr learned my addr 172.21.15.39:0/1875865089 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:46.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.860+0000 7fafb3e00640 1 --2- 172.21.15.39:0/1875865089 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fafbc0cf450 0x7fafbc1e92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:46.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.860+0000 7fafbaa00640 1 --2- 172.21.15.39:0/1875865089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafbc073dd0 0x7fafbc154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:46.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.860+0000 7fafbb400640 1 -- 172.21.15.39:0/1875865089 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fafbc0cf450 msgr2=0x7fafbc1e92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:46.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.860+0000 7fafbb400640 1 --2- 172.21.15.39:0/1875865089 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fafbc0cf450 0x7fafbc1e92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:46.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.860+0000 7fafbb400640 1 -- 172.21.15.39:0/1875865089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafbc073dd0 msgr2=0x7fafbc154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:46.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.860+0000 7fafbb400640 1 --2- 172.21.15.39:0/1875865089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafbc073dd0 0x7fafbc154050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:46.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.860+0000 7fafbb400640 1 -- 172.21.15.39:0/1875865089 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafbc0b34c0 con 0x7fafbc0d0630 2024-09-21T07:48:46.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.861+0000 7fafbb400640 1 --2- 172.21.15.39:0/1875865089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafbc0d0630 0x7fafbc1e9810 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7fafb4006010 tx=0x7fafb4003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:46.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.861+0000 7fafb2a00640 1 -- 172.21.15.39:0/1875865089 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4044070 con 0x7fafbc0d0630 2024-09-21T07:48:46.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.861+0000 7fafb2a00640 1 -- 172.21.15.39:0/1875865089 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fafb4004040 con 0x7fafbc0d0630 2024-09-21T07:48:46.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.861+0000 7fafb2a00640 1 -- 172.21.15.39:0/1875865089 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4004430 con 0x7fafbc0d0630 2024-09-21T07:48:46.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.861+0000 7fafc1c00640 1 -- 172.21.15.39:0/1875865089 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafbc1f9670 con 0x7fafbc0d0630 2024-09-21T07:48:46.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.861+0000 7fafc1c00640 1 -- 172.21.15.39:0/1875865089 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafbc1f9c30 con 0x7fafbc0d0630 2024-09-21T07:48:46.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.862+0000 7fafc1c00640 1 -- 172.21.15.39:0/1875865089 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafbc1f93e0 con 0x7fafbc0d0630 2024-09-21T07:48:46.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.864+0000 7fafb2a00640 1 -- 172.21.15.39:0/1875865089 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fafb4047070 con 0x7fafbc0d0630 2024-09-21T07:48:46.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.865+0000 7fafb2a00640 1 --2- 172.21.15.39:0/1875865089 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faf98078b90 0x7faf9807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:46.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.865+0000 7fafb2a00640 1 -- 172.21.15.39:0/1875865089 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(217..217 src has 1..217) ==== 6888+0+0 (secure 0 0 0) 0x7fafb40bc530 con 0x7fafbc0d0630 2024-09-21T07:48:46.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.866+0000 7fafbaa00640 1 --2- 172.21.15.39:0/1875865089 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faf98078b90 0x7faf9807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:46.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.866+0000 7fafbaa00640 1 --2- 172.21.15.39:0/1875865089 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faf98078b90 0x7faf9807b0a0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fafbc0730e0 tx=0x7fafac009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:46.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:46.869+0000 7fafb2a00640 1 -- 172.21.15.39:0/1875865089 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fafb404e020 con 0x7fafbc0d0630 2024-09-21T07:48:47.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.104+0000 7fafc1c00640 1 -- 172.21.15.39:0/1875865089 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fafbc0cd860 con 0x7fafbc0d0630 2024-09-21T07:48:47.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.584+0000 7fafb2a00640 1 -- 172.21.15.39:0/1875865089 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 27 and data pool 28 v225) ==== 174+0+0 (secure 0 0 0) 0x7fafb4032030 con 0x7fafbc0d0630 2024-09-21T07:48:47.586 INFO:teuthology.orchestra.run.smithi039.stderr:new fs with metadata pool 27 and data pool 28 2024-09-21T07:48:47.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.588+0000 7fafc1c00640 1 -- 172.21.15.39:0/1875865089 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faf98078b90 msgr2=0x7faf9807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:47.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.588+0000 7fafc1c00640 1 --2- 172.21.15.39:0/1875865089 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faf98078b90 0x7faf9807b0a0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fafbc0730e0 tx=0x7fafac009040 comp rx=0 tx=0).stop 2024-09-21T07:48:47.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.588+0000 7fafc1c00640 1 -- 172.21.15.39:0/1875865089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafbc0d0630 msgr2=0x7fafbc1e9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:47.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.588+0000 7fafc1c00640 1 --2- 172.21.15.39:0/1875865089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafbc0d0630 0x7fafbc1e9810 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7fafb4006010 tx=0x7fafb4003ce0 comp rx=0 tx=0).stop 2024-09-21T07:48:47.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.589+0000 7fafc1c00640 1 -- 172.21.15.39:0/1875865089 shutdown_connections 2024-09-21T07:48:47.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.589+0000 7fafc1c00640 1 --2- 172.21.15.39:0/1875865089 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faf98078b90 0x7faf9807b0a0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:47.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.589+0000 7fafc1c00640 1 --2- 172.21.15.39:0/1875865089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafbc0d0630 0x7fafbc1e9810 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:47.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.589+0000 7fafc1c00640 1 --2- 172.21.15.39:0/1875865089 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fafbc0cf450 0x7fafbc1e92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:47.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.589+0000 7fafc1c00640 1 --2- 172.21.15.39:0/1875865089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafbc073dd0 0x7fafbc154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:47.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.589+0000 7fafc1c00640 1 -- 172.21.15.39:0/1875865089 >> 172.21.15.39:0/1875865089 conn(0x7fafbc06cfc0 msgr2=0x7fafbc15c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:47.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.592+0000 7fafc1c00640 1 -- 172.21.15.39:0/1875865089 shutdown_connections 2024-09-21T07:48:47.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.593+0000 7fafc1c00640 1 -- 172.21.15.39:0/1875865089 wait complete. 2024-09-21T07:48:47.642 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:48:47.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.869+0000 7f34d0a00640 1 -- 172.21.15.39:0/806950612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34b0004d90 msgr2=0x7f34b0005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:47.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.869+0000 7f34d0a00640 1 --2- 172.21.15.39:0/806950612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34b0004d90 0x7f34b0005220 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f34c400c610 tx=0x7f34b0005830 comp rx=0 tx=0).stop 2024-09-21T07:48:47.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.869+0000 7f34ca800640 1 -- 172.21.15.39:0/806950612 reap_dead start 2024-09-21T07:48:47.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.869+0000 7f34d0a00640 1 -- 172.21.15.39:0/806950612 shutdown_connections 2024-09-21T07:48:47.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.869+0000 7f34d0a00640 1 -- 172.21.15.39:0/806950612 >> 172.21.15.39:0/806950612 conn(0x7f34cc06cfc0 msgr2=0x7f34cc06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:47.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.869+0000 7f34d0a00640 1 -- 172.21.15.39:0/806950612 shutdown_connections 2024-09-21T07:48:47.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.869+0000 7f34d0a00640 1 -- 172.21.15.39:0/806950612 wait complete. 2024-09-21T07:48:47.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.871+0000 7f34d0a00640 1 Processor -- start 2024-09-21T07:48:47.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.871+0000 7f34d0a00640 1 -- start start 2024-09-21T07:48:47.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.873+0000 7f34d0a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f34b0004d90 0x7f34cc151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:47.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.873+0000 7f34d0a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34cc15ad10 0x7f34cc157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:47.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.873+0000 7f34d0a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34cc157f80 0x7f34cc1f69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:47.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.873+0000 7f34d0a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc15c690 con 0x7f34cc157f80 2024-09-21T07:48:47.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.873+0000 7f34d0a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc0cff40 con 0x7f34cc15ad10 2024-09-21T07:48:47.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.873+0000 7f34d0a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f34cc15c810 con 0x7f34b0004d90 2024-09-21T07:48:47.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.873+0000 7f34ca800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34cc157f80 0x7f34cc1f69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:47.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.873+0000 7f34ca800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34cc157f80 0x7f34cc1f69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51366/0 (socket says 172.21.15.39:51366) 2024-09-21T07:48:47.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.873+0000 7f34ca800640 1 -- 172.21.15.39:0/1855599179 learned_addr learned my addr 172.21.15.39:0/1855599179 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:47.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.873+0000 7f34c9e00640 1 --2- 172.21.15.39:0/1855599179 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f34b0004d90 0x7f34cc151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:47.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.873+0000 7f34c3e00640 1 --2- 172.21.15.39:0/1855599179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34cc15ad10 0x7f34cc157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:47.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.873+0000 7f34ca800640 1 -- 172.21.15.39:0/1855599179 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f34b0004d90 msgr2=0x7f34cc151a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:47.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.873+0000 7f34ca800640 1 --2- 172.21.15.39:0/1855599179 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f34b0004d90 0x7f34cc151a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:47.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.873+0000 7f34ca800640 1 -- 172.21.15.39:0/1855599179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34cc15ad10 msgr2=0x7f34cc157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:47.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.873+0000 7f34ca800640 1 --2- 172.21.15.39:0/1855599179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34cc15ad10 0x7f34cc157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:47.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.873+0000 7f34ca800640 1 -- 172.21.15.39:0/1855599179 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34cc0a94d0 con 0x7f34cc157f80 2024-09-21T07:48:47.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.874+0000 7f34ca800640 1 --2- 172.21.15.39:0/1855599179 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34cc157f80 0x7f34cc1f69f0 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f34c400c730 tx=0x7f34c4001aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:47.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.874+0000 7f34c2000640 1 -- 172.21.15.39:0/1855599179 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c4039070 con 0x7f34cc157f80 2024-09-21T07:48:47.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.874+0000 7f34c2000640 1 -- 172.21.15.39:0/1855599179 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34c4006680 con 0x7f34cc157f80 2024-09-21T07:48:47.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.874+0000 7f34c2000640 1 -- 172.21.15.39:0/1855599179 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c4001fb0 con 0x7f34cc157f80 2024-09-21T07:48:47.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.874+0000 7f34d0a00640 1 -- 172.21.15.39:0/1855599179 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34cc0a9770 con 0x7f34cc157f80 2024-09-21T07:48:47.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.874+0000 7f34d0a00640 1 -- 172.21.15.39:0/1855599179 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34cc0a9d30 con 0x7f34cc157f80 2024-09-21T07:48:47.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.875+0000 7f34d0a00640 1 -- 172.21.15.39:0/1855599179 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34cc0b4bc0 con 0x7f34cc157f80 2024-09-21T07:48:47.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.877+0000 7f34c2000640 1 -- 172.21.15.39:0/1855599179 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f34c403c020 con 0x7f34cc157f80 2024-09-21T07:48:47.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.878+0000 7f34c2000640 1 --2- 172.21.15.39:0/1855599179 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f34a8078b90 0x7f34a807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:47.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.878+0000 7f34c2000640 1 -- 172.21.15.39:0/1855599179 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(218..218 src has 1..218) ==== 6988+0+0 (secure 0 0 0) 0x7f34c40ba680 con 0x7f34cc157f80 2024-09-21T07:48:47.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.878+0000 7f34c9e00640 1 --2- 172.21.15.39:0/1855599179 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f34a8078b90 0x7f34a807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:47.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.879+0000 7f34c9e00640 1 --2- 172.21.15.39:0/1855599179 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f34a8078b90 0x7f34a807b0a0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f34bc009460 tx=0x7f34bc009530 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:47.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:47.883+0000 7f34c2000640 1 -- 172.21.15.39:0/1855599179 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f34c4045070 con 0x7f34cc157f80 2024-09-21T07:48:48.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.084+0000 7f34d0a00640 1 -- 172.21.15.39:0/1855599179 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f34cc06d6b0 con 0x7f34cc157f80 2024-09-21T07:48:48.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.086+0000 7f34c2000640 1 -- 172.21.15.39:0/1855599179 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v218) ==== 74+0+22519 (secure 0 0 0) 0x7f34c4081fe0 con 0x7f34cc157f80 2024-09-21T07:48:48.088 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:48:48.088 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":218,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:48:47.106516+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:48:45.059810+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"218","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-09-21T07:48:46.068380+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"218","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]},{"pool":20,"snaps":[{"begin":80,"length":23},{"begin":181,"length":23}]},{"pool":22,"snaps":[{"begin":100,"length":1},{"begin":199,"length":1}]},{"pool":24,"snaps":[{"begin":2,"length":2}]},{"pool":26,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:48:48.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.088+0000 7f34d0a00640 1 -- 172.21.15.39:0/1855599179 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f34a8078b90 msgr2=0x7f34a807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:48.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.088+0000 7f34d0a00640 1 --2- 172.21.15.39:0/1855599179 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f34a8078b90 0x7f34a807b0a0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f34bc009460 tx=0x7f34bc009530 comp rx=0 tx=0).stop 2024-09-21T07:48:48.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.088+0000 7f34d0a00640 1 -- 172.21.15.39:0/1855599179 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34cc157f80 msgr2=0x7f34cc1f69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:48.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.088+0000 7f34d0a00640 1 --2- 172.21.15.39:0/1855599179 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34cc157f80 0x7f34cc1f69f0 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f34c400c730 tx=0x7f34c4001aa0 comp rx=0 tx=0).stop 2024-09-21T07:48:48.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.089+0000 7f34d0a00640 1 -- 172.21.15.39:0/1855599179 shutdown_connections 2024-09-21T07:48:48.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.089+0000 7f34d0a00640 1 --2- 172.21.15.39:0/1855599179 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f34a8078b90 0x7f34a807b0a0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:48.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.089+0000 7f34d0a00640 1 --2- 172.21.15.39:0/1855599179 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34cc157f80 0x7f34cc1f69f0 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:48.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.089+0000 7f34d0a00640 1 --2- 172.21.15.39:0/1855599179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34cc15ad10 0x7f34cc157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:48.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.089+0000 7f34d0a00640 1 --2- 172.21.15.39:0/1855599179 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f34b0004d90 0x7f34cc151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:48.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.089+0000 7f34d0a00640 1 -- 172.21.15.39:0/1855599179 >> 172.21.15.39:0/1855599179 conn(0x7f34cc06cfc0 msgr2=0x7f34cc154c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:48.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.089+0000 7f34d0a00640 1 -- 172.21.15.39:0/1855599179 shutdown_connections 2024-09-21T07:48:48.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.089+0000 7f34d0a00640 1 -- 172.21.15.39:0/1855599179 wait complete. 2024-09-21T07:48:48.116 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:48:48.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.340+0000 7f7f3a200640 1 -- 172.21.15.39:0/2865331853 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f340d04a0 msgr2=0x7f7f340d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:48.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.340+0000 7f7f3a200640 1 --2- 172.21.15.39:0/2865331853 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f340d04a0 0x7f7f340d08b0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f7f340cc430 tx=0x7f7f2002a5f0 comp rx=0 tx=0).stop 2024-09-21T07:48:48.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.340+0000 7f7f3a200640 1 -- 172.21.15.39:0/2865331853 shutdown_connections 2024-09-21T07:48:48.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.340+0000 7f7f3a200640 1 --2- 172.21.15.39:0/2865331853 >> v2:172.21.15.148:3300/0 conn(0x7f7f3406ee90 0x7f7f3406f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:48.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.340+0000 7f7f3a200640 1 --2- 172.21.15.39:0/2865331853 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f340d04a0 0x7f7f340d08b0 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:48.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.340+0000 7f7f3a200640 1 -- 172.21.15.39:0/2865331853 >> 172.21.15.39:0/2865331853 conn(0x7f7f3406cfc0 msgr2=0x7f7f3406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:48.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.340+0000 7f7f3a200640 1 -- 172.21.15.39:0/2865331853 shutdown_connections 2024-09-21T07:48:48.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.340+0000 7f7f3a200640 1 -- 172.21.15.39:0/2865331853 wait complete. 2024-09-21T07:48:48.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.342+0000 7f7f3a200640 1 Processor -- start 2024-09-21T07:48:48.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.342+0000 7f7f3a200640 1 -- start start 2024-09-21T07:48:48.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.344+0000 7f7f3a200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f3406ee90 0x7f7f341414d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:48.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.344+0000 7f7f3a200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f340d04a0 0x7f7f34159880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:48.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.344+0000 7f7f3a200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7f341519d0 0x7f7f34159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:48.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.344+0000 7f7f3a200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7f340b3940 con 0x7f7f3406ee90 2024-09-21T07:48:48.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.344+0000 7f7f3a200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7f340b3ac0 con 0x7f7f340d04a0 2024-09-21T07:48:48.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.344+0000 7f7f3a200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f7f340b3c40 con 0x7f7f341519d0 2024-09-21T07:48:48.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.344+0000 7f7f38e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f3406ee90 0x7f7f341414d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:48.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.344+0000 7f7f38e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f3406ee90 0x7f7f341414d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51392/0 (socket says 172.21.15.39:51392) 2024-09-21T07:48:48.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.344+0000 7f7f38e00640 1 -- 172.21.15.39:0/3436818234 learned_addr learned my addr 172.21.15.39:0/3436818234 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:48.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.344+0000 7f7f33e00640 1 --2- 172.21.15.39:0/3436818234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f340d04a0 0x7f7f34159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:48.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.344+0000 7f7f39800640 1 --2- 172.21.15.39:0/3436818234 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7f341519d0 0x7f7f34159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:48.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.345+0000 7f7f38e00640 1 -- 172.21.15.39:0/3436818234 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7f341519d0 msgr2=0x7f7f34159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:48.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.345+0000 7f7f38e00640 1 --2- 172.21.15.39:0/3436818234 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7f341519d0 0x7f7f34159dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:48.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.345+0000 7f7f38e00640 1 -- 172.21.15.39:0/3436818234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f340d04a0 msgr2=0x7f7f34159880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:48.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.345+0000 7f7f38e00640 1 --2- 172.21.15.39:0/3436818234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f340d04a0 0x7f7f34159880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:48.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.345+0000 7f7f38e00640 1 -- 172.21.15.39:0/3436818234 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f34150c80 con 0x7f7f3406ee90 2024-09-21T07:48:48.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.345+0000 7f7f38e00640 1 --2- 172.21.15.39:0/3436818234 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f3406ee90 0x7f7f341414d0 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7f7f2800c3a0 tx=0x7f7f2800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:48.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.345+0000 7f7f30c00640 1 -- 172.21.15.39:0/3436818234 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f28017070 con 0x7f7f3406ee90 2024-09-21T07:48:48.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.345+0000 7f7f30c00640 1 -- 172.21.15.39:0/3436818234 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f280053c0 con 0x7f7f3406ee90 2024-09-21T07:48:48.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.345+0000 7f7f3a200640 1 -- 172.21.15.39:0/3436818234 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f3414ed40 con 0x7f7f3406ee90 2024-09-21T07:48:48.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.345+0000 7f7f30c00640 1 -- 172.21.15.39:0/3436818234 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f28003850 con 0x7f7f3406ee90 2024-09-21T07:48:48.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.345+0000 7f7f3a200640 1 -- 172.21.15.39:0/3436818234 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f3414f300 con 0x7f7f3406ee90 2024-09-21T07:48:48.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.346+0000 7f7f3a200640 1 -- 172.21.15.39:0/3436818234 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f3406f2a0 con 0x7f7f3406ee90 2024-09-21T07:48:48.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.348+0000 7f7f30c00640 1 -- 172.21.15.39:0/3436818234 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7f2801a070 con 0x7f7f3406ee90 2024-09-21T07:48:48.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.349+0000 7f7f30c00640 1 --2- 172.21.15.39:0/3436818234 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7f14078b90 0x7f7f1407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:48.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.349+0000 7f7f30c00640 1 -- 172.21.15.39:0/3436818234 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(218..218 src has 1..218) ==== 6988+0+0 (secure 0 0 0) 0x7f7f2809a440 con 0x7f7f3406ee90 2024-09-21T07:48:48.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.349+0000 7f7f33e00640 1 --2- 172.21.15.39:0/3436818234 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7f14078b90 0x7f7f1407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:48.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.350+0000 7f7f33e00640 1 --2- 172.21.15.39:0/3436818234 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7f14078b90 0x7f7f1407b0a0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f7f20033000 tx=0x7f7f20004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:48.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.354+0000 7f7f30c00640 1 -- 172.21.15.39:0/3436818234 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7f340d0df0 con 0x7f7f3406ee90 2024-09-21T07:48:48.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.550+0000 7f7f3a200640 1 -- 172.21.15.39:0/3436818234 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7f3414b310 con 0x7f7f3406ee90 2024-09-21T07:48:48.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.552+0000 7f7f30c00640 1 -- 172.21.15.39:0/3436818234 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v218) ==== 74+0+22519 (secure 0 0 0) 0x7f7f28061e50 con 0x7f7f3406ee90 2024-09-21T07:48:48.554 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:48:48.554 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":218,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:48:47.106516+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:48:45.059810+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"218","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-09-21T07:48:46.068380+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"218","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]},{"pool":20,"snaps":[{"begin":80,"length":23},{"begin":181,"length":23}]},{"pool":22,"snaps":[{"begin":100,"length":1},{"begin":199,"length":1}]},{"pool":24,"snaps":[{"begin":2,"length":2}]},{"pool":26,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:48:48.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.554+0000 7f7f3a200640 1 -- 172.21.15.39:0/3436818234 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7f14078b90 msgr2=0x7f7f1407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:48.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.554+0000 7f7f3a200640 1 --2- 172.21.15.39:0/3436818234 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7f14078b90 0x7f7f1407b0a0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f7f20033000 tx=0x7f7f20004730 comp rx=0 tx=0).stop 2024-09-21T07:48:48.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.554+0000 7f7f3a200640 1 -- 172.21.15.39:0/3436818234 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f3406ee90 msgr2=0x7f7f341414d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:48.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.554+0000 7f7f3a200640 1 --2- 172.21.15.39:0/3436818234 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f3406ee90 0x7f7f341414d0 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7f7f2800c3a0 tx=0x7f7f2800c860 comp rx=0 tx=0).stop 2024-09-21T07:48:48.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.554+0000 7f7f3a200640 1 -- 172.21.15.39:0/3436818234 shutdown_connections 2024-09-21T07:48:48.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.554+0000 7f7f3a200640 1 --2- 172.21.15.39:0/3436818234 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7f14078b90 0x7f7f1407b0a0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:48.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.554+0000 7f7f3a200640 1 --2- 172.21.15.39:0/3436818234 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7f341519d0 0x7f7f34159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:48.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.554+0000 7f7f3a200640 1 --2- 172.21.15.39:0/3436818234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f340d04a0 0x7f7f34159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:48.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.554+0000 7f7f3a200640 1 --2- 172.21.15.39:0/3436818234 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f3406ee90 0x7f7f341414d0 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:48.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.554+0000 7f7f3a200640 1 -- 172.21.15.39:0/3436818234 >> 172.21.15.39:0/3436818234 conn(0x7f7f3406cfc0 msgr2=0x7f7f3414e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:48.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.554+0000 7f7f3a200640 1 -- 172.21.15.39:0/3436818234 shutdown_connections 2024-09-21T07:48:48.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.554+0000 7f7f3a200640 1 -- 172.21.15.39:0/3436818234 wait complete. 2024-09-21T07:48:48.581 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T07:48:48.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.811+0000 7f223b000640 1 -- 172.21.15.39:0/438635093 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2234160660 msgr2=0x7f22340cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:48.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.811+0000 7f223b000640 1 --2- 172.21.15.39:0/438635093 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2234160660 0x7f22340cf710 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f2220005fd0 tx=0x7f222002a580 comp rx=0 tx=0).stop 2024-09-21T07:48:48.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.812+0000 7f223b000640 1 -- 172.21.15.39:0/438635093 shutdown_connections 2024-09-21T07:48:48.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.812+0000 7f223b000640 1 --2- 172.21.15.39:0/438635093 >> v2:172.21.15.148:3300/0 conn(0x7f22340cfc50 0x7f22340d0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:48.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.812+0000 7f223b000640 1 --2- 172.21.15.39:0/438635093 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2234160660 0x7f22340cf710 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:48.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.812+0000 7f223b000640 1 -- 172.21.15.39:0/438635093 >> 172.21.15.39:0/438635093 conn(0x7f223406cfc0 msgr2=0x7f223406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:48.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.812+0000 7f223b000640 1 -- 172.21.15.39:0/438635093 shutdown_connections 2024-09-21T07:48:48.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.812+0000 7f223b000640 1 -- 172.21.15.39:0/438635093 wait complete. 2024-09-21T07:48:48.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.814+0000 7f223b000640 1 Processor -- start 2024-09-21T07:48:48.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.814+0000 7f223b000640 1 -- start start 2024-09-21T07:48:48.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.816+0000 7f223b000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22340cfc50 0x7f2234157320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:48.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.816+0000 7f223b000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2234160250 0x7f223415b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:48.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.816+0000 7f223b000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2234160660 0x7f223415bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:48.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.816+0000 7f223b000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f22340ccdc0 con 0x7f2234160660 2024-09-21T07:48:48.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.816+0000 7f223b000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f22340ccc40 con 0x7f22340cfc50 2024-09-21T07:48:48.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.816+0000 7f223b000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f22340ccf40 con 0x7f2234160250 2024-09-21T07:48:48.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.816+0000 7f223a600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2234160660 0x7f223415bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:48.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.816+0000 7f223a600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2234160660 0x7f223415bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51406/0 (socket says 172.21.15.39:51406) 2024-09-21T07:48:48.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.816+0000 7f223a600640 1 -- 172.21.15.39:0/2660048118 learned_addr learned my addr 172.21.15.39:0/2660048118 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:48.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.816+0000 7f2239200640 1 --2- 172.21.15.39:0/2660048118 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2234160250 0x7f223415b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:48.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.816+0000 7f2239c00640 1 --2- 172.21.15.39:0/2660048118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22340cfc50 0x7f2234157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:48.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.816+0000 7f223a600640 1 -- 172.21.15.39:0/2660048118 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2234160250 msgr2=0x7f223415b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:48.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.816+0000 7f223a600640 1 --2- 172.21.15.39:0/2660048118 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2234160250 0x7f223415b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:48.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.816+0000 7f223a600640 1 -- 172.21.15.39:0/2660048118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22340cfc50 msgr2=0x7f2234157320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:48.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.816+0000 7f223a600640 1 --2- 172.21.15.39:0/2660048118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22340cfc50 0x7f2234157320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:48.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.816+0000 7f223a600640 1 -- 172.21.15.39:0/2660048118 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2234157860 con 0x7f2234160660 2024-09-21T07:48:48.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.817+0000 7f223a600640 1 --2- 172.21.15.39:0/2660048118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2234160660 0x7f223415bf20 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f222c00cad0 tx=0x7f222c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:48.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.817+0000 7f2231600640 1 -- 172.21.15.39:0/2660048118 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222c00adc0 con 0x7f2234160660 2024-09-21T07:48:48.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.817+0000 7f2231600640 1 -- 172.21.15.39:0/2660048118 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f222c00f450 con 0x7f2234160660 2024-09-21T07:48:48.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.817+0000 7f223b000640 1 -- 172.21.15.39:0/2660048118 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2234157ce0 con 0x7f2234160660 2024-09-21T07:48:48.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.817+0000 7f2231600640 1 -- 172.21.15.39:0/2660048118 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222c00f900 con 0x7f2234160660 2024-09-21T07:48:48.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.817+0000 7f223b000640 1 -- 172.21.15.39:0/2660048118 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22341f60d0 con 0x7f2234160660 2024-09-21T07:48:48.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.818+0000 7f223b000640 1 -- 172.21.15.39:0/2660048118 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2234157a30 con 0x7f2234160660 2024-09-21T07:48:48.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.819+0000 7f2231600640 1 -- 172.21.15.39:0/2660048118 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f222c00faa0 con 0x7f2234160660 2024-09-21T07:48:48.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.820+0000 7f2231600640 1 --2- 172.21.15.39:0/2660048118 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2214078b90 0x7f221407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:48.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.820+0000 7f2231600640 1 -- 172.21.15.39:0/2660048118 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(219..219 src has 1..219) ==== 6988+0+0 (secure 0 0 0) 0x7f222c09bcc0 con 0x7f2234160660 2024-09-21T07:48:48.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.821+0000 7f2239c00640 1 --2- 172.21.15.39:0/2660048118 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2214078b90 0x7f221407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:48.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.822+0000 7f2239c00640 1 --2- 172.21.15.39:0/2660048118 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2214078b90 0x7f221407b0a0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f2228006fd0 tx=0x7f2228008040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:48.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:48.825+0000 7f2231600640 1 -- 172.21.15.39:0/2660048118 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f222c014030 con 0x7f2234160660 2024-09-21T07:48:49.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.064+0000 7f223b000640 1 -- 172.21.15.39:0/2660048118 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f22340cd460 con 0x7f2234160660 2024-09-21T07:48:49.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.613+0000 7f2231600640 1 -- 172.21.15.39:0/2660048118 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v228) ==== 118+0+0 (secure 0 0 0) 0x7f222c063640 con 0x7f2234160660 2024-09-21T07:48:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.616+0000 7f223b000640 1 -- 172.21.15.39:0/2660048118 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2214078b90 msgr2=0x7f221407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.616+0000 7f223b000640 1 --2- 172.21.15.39:0/2660048118 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2214078b90 0x7f221407b0a0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f2228006fd0 tx=0x7f2228008040 comp rx=0 tx=0).stop 2024-09-21T07:48:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.616+0000 7f223b000640 1 -- 172.21.15.39:0/2660048118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2234160660 msgr2=0x7f223415bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.616+0000 7f223b000640 1 --2- 172.21.15.39:0/2660048118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2234160660 0x7f223415bf20 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f222c00cad0 tx=0x7f222c00cf90 comp rx=0 tx=0).stop 2024-09-21T07:48:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.616+0000 7f223b000640 1 -- 172.21.15.39:0/2660048118 shutdown_connections 2024-09-21T07:48:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.616+0000 7f223b000640 1 --2- 172.21.15.39:0/2660048118 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2214078b90 0x7f221407b0a0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.616+0000 7f223b000640 1 --2- 172.21.15.39:0/2660048118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2234160660 0x7f223415bf20 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.616+0000 7f223b000640 1 --2- 172.21.15.39:0/2660048118 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2234160250 0x7f223415b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.616+0000 7f223b000640 1 --2- 172.21.15.39:0/2660048118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f22340cfc50 0x7f2234157320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.616+0000 7f223b000640 1 -- 172.21.15.39:0/2660048118 >> 172.21.15.39:0/2660048118 conn(0x7f223406cfc0 msgr2=0x7f223406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.617+0000 7f223b000640 1 -- 172.21.15.39:0/2660048118 shutdown_connections 2024-09-21T07:48:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.617+0000 7f223b000640 1 -- 172.21.15.39:0/2660048118 wait complete. 2024-09-21T07:48:49.642 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T07:48:49.642 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T07:48:49.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.872+0000 7f91d2e00640 1 -- 172.21.15.39:0/304127015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91cc073dd0 msgr2=0x7f91cc15d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:49.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.872+0000 7f91d2e00640 1 --2- 172.21.15.39:0/304127015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91cc073dd0 0x7f91cc15d5b0 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f91cc073500 tx=0x7f91bc02a540 comp rx=0 tx=0).stop 2024-09-21T07:48:49.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.872+0000 7f91d2e00640 1 -- 172.21.15.39:0/304127015 shutdown_connections 2024-09-21T07:48:49.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.872+0000 7f91d2e00640 1 --2- 172.21.15.39:0/304127015 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91cc0cf990 0x7f91cc0d0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:49.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.872+0000 7f91d2e00640 1 --2- 172.21.15.39:0/304127015 >> v2:172.21.15.148:3300/0 conn(0x7f91cc15daf0 0x7f91cc0cf450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:49.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.872+0000 7f91d2e00640 1 --2- 172.21.15.39:0/304127015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91cc073dd0 0x7f91cc15d5b0 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:49.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.872+0000 7f91d2e00640 1 -- 172.21.15.39:0/304127015 >> 172.21.15.39:0/304127015 conn(0x7f91cc06cfc0 msgr2=0x7f91cc06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:49.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.872+0000 7f91d2e00640 1 -- 172.21.15.39:0/304127015 shutdown_connections 2024-09-21T07:48:49.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.872+0000 7f91d2e00640 1 -- 172.21.15.39:0/304127015 wait complete. 2024-09-21T07:48:49.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.874+0000 7f91d2e00640 1 Processor -- start 2024-09-21T07:48:49.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.875+0000 7f91d2e00640 1 -- start start 2024-09-21T07:48:49.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.876+0000 7f91d2e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91cc073dd0 0x7f91cc0c5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:49.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.876+0000 7f91d2e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91cc0cf990 0x7f91cc0aed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:49.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.876+0000 7f91d2e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91cc15daf0 0x7f91cc1f8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:49.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.876+0000 7f91d2e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f91cc160740 con 0x7f91cc073dd0 2024-09-21T07:48:49.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.876+0000 7f91d2e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f91cc1605c0 con 0x7f91cc0cf990 2024-09-21T07:48:49.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.876+0000 7f91d2e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f91cc1608c0 con 0x7f91cc15daf0 2024-09-21T07:48:49.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.876+0000 7f91cbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91cc073dd0 0x7f91cc0c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:49.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.876+0000 7f91cbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91cc073dd0 0x7f91cc0c5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51436/0 (socket says 172.21.15.39:51436) 2024-09-21T07:48:49.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.876+0000 7f91cbe00640 1 -- 172.21.15.39:0/2740805423 learned_addr learned my addr 172.21.15.39:0/2740805423 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:49.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.876+0000 7f91cb400640 1 --2- 172.21.15.39:0/2740805423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91cc0cf990 0x7f91cc0aed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:49.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.877+0000 7f91d0e00640 1 --2- 172.21.15.39:0/2740805423 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91cc15daf0 0x7f91cc1f8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:49.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.877+0000 7f91cbe00640 1 -- 172.21.15.39:0/2740805423 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91cc15daf0 msgr2=0x7f91cc1f8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:49.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.877+0000 7f91cbe00640 1 --2- 172.21.15.39:0/2740805423 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91cc15daf0 0x7f91cc1f8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:49.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.877+0000 7f91cbe00640 1 -- 172.21.15.39:0/2740805423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91cc0cf990 msgr2=0x7f91cc0aed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:49.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.877+0000 7f91cbe00640 1 --2- 172.21.15.39:0/2740805423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91cc0cf990 0x7f91cc0aed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:49.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.877+0000 7f91cbe00640 1 -- 172.21.15.39:0/2740805423 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91cc0aaf30 con 0x7f91cc073dd0 2024-09-21T07:48:49.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.877+0000 7f91d0e00640 1 --2- 172.21.15.39:0/2740805423 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91cc15daf0 0x7f91cc1f8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:48:49.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.877+0000 7f91cbe00640 1 --2- 172.21.15.39:0/2740805423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91cc073dd0 0x7f91cc0c5330 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f91bc034040 tx=0x7f91bc003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:49.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.878+0000 7f91c3e00640 1 -- 172.21.15.39:0/2740805423 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91bc03e030 con 0x7f91cc073dd0 2024-09-21T07:48:49.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.878+0000 7f91c3e00640 1 -- 172.21.15.39:0/2740805423 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91bc003e00 con 0x7f91cc073dd0 2024-09-21T07:48:49.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.878+0000 7f91d2e00640 1 -- 172.21.15.39:0/2740805423 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91cc0ab430 con 0x7f91cc073dd0 2024-09-21T07:48:49.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.878+0000 7f91c3e00640 1 -- 172.21.15.39:0/2740805423 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91bc035040 con 0x7f91cc073dd0 2024-09-21T07:48:49.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.878+0000 7f91d2e00640 1 -- 172.21.15.39:0/2740805423 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91cc0ab9f0 con 0x7f91cc073dd0 2024-09-21T07:48:49.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.879+0000 7f91d2e00640 1 -- 172.21.15.39:0/2740805423 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91cc0d0630 con 0x7f91cc073dd0 2024-09-21T07:48:49.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.880+0000 7f91c3e00640 1 -- 172.21.15.39:0/2740805423 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f91bc041030 con 0x7f91cc073dd0 2024-09-21T07:48:49.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.881+0000 7f91c3e00640 1 --2- 172.21.15.39:0/2740805423 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f91ac078bc0 0x7f91ac07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:49.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.881+0000 7f91c3e00640 1 -- 172.21.15.39:0/2740805423 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(219..219 src has 1..219) ==== 6988+0+0 (secure 0 0 0) 0x7f91bc0bb8d0 con 0x7f91cc073dd0 2024-09-21T07:48:49.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.882+0000 7f91cb400640 1 --2- 172.21.15.39:0/2740805423 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f91ac078bc0 0x7f91ac07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:49.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.882+0000 7f91cb400640 1 --2- 172.21.15.39:0/2740805423 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f91ac078bc0 0x7f91ac07b0d0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f91cc1f8b90 tx=0x7f91b800b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:49.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:49.885+0000 7f91c3e00640 1 -- 172.21.15.39:0/2740805423 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f91bc083260 con 0x7f91cc073dd0 2024-09-21T07:48:50.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.122+0000 7f91d2e00640 1 -- 172.21.15.39:0/2740805423 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f91cc0b3660 con 0x7f91cc073dd0 2024-09-21T07:48:50.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.621+0000 7f91c3e00640 1 -- 172.21.15.39:0/2740805423 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v229) ==== 122+0+0 (secure 0 0 0) 0x7f91bc088e20 con 0x7f91cc073dd0 2024-09-21T07:48:50.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.624+0000 7f91d2e00640 1 -- 172.21.15.39:0/2740805423 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f91ac078bc0 msgr2=0x7f91ac07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:50.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.624+0000 7f91d2e00640 1 --2- 172.21.15.39:0/2740805423 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f91ac078bc0 0x7f91ac07b0d0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f91cc1f8b90 tx=0x7f91b800b2e0 comp rx=0 tx=0).stop 2024-09-21T07:48:50.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.624+0000 7f91d2e00640 1 -- 172.21.15.39:0/2740805423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91cc073dd0 msgr2=0x7f91cc0c5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:50.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.624+0000 7f91d2e00640 1 --2- 172.21.15.39:0/2740805423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91cc073dd0 0x7f91cc0c5330 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f91bc034040 tx=0x7f91bc003910 comp rx=0 tx=0).stop 2024-09-21T07:48:50.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.624+0000 7f91d2e00640 1 -- 172.21.15.39:0/2740805423 shutdown_connections 2024-09-21T07:48:50.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.624+0000 7f91d2e00640 1 --2- 172.21.15.39:0/2740805423 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f91ac078bc0 0x7f91ac07b0d0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:50.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.624+0000 7f91d2e00640 1 --2- 172.21.15.39:0/2740805423 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f91cc15daf0 0x7f91cc1f8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:50.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.624+0000 7f91d2e00640 1 --2- 172.21.15.39:0/2740805423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f91cc0cf990 0x7f91cc0aed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:50.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.624+0000 7f91d2e00640 1 --2- 172.21.15.39:0/2740805423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91cc073dd0 0x7f91cc0c5330 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:50.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.624+0000 7f91d2e00640 1 -- 172.21.15.39:0/2740805423 >> 172.21.15.39:0/2740805423 conn(0x7f91cc06cfc0 msgr2=0x7f91cc156500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:50.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.624+0000 7f91d2e00640 1 -- 172.21.15.39:0/2740805423 shutdown_connections 2024-09-21T07:48:50.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.624+0000 7f91d2e00640 1 -- 172.21.15.39:0/2740805423 wait complete. 2024-09-21T07:48:50.652 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T07:48:50.652 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:48:50.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.877+0000 7f466be00640 1 -- 172.21.15.39:0/1340106980 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f466c074c10 msgr2=0x7f466c0cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:50.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.877+0000 7f466be00640 1 --2- 172.21.15.39:0/1340106980 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f466c074c10 0x7f466c0cfb40 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f465c009830 tx=0x7f465c02a510 comp rx=0 tx=0).stop 2024-09-21T07:48:50.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.877+0000 7f466be00640 1 -- 172.21.15.39:0/1340106980 shutdown_connections 2024-09-21T07:48:50.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.877+0000 7f466be00640 1 --2- 172.21.15.39:0/1340106980 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f466c06ecf0 0x7f466c160bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:50.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.877+0000 7f466be00640 1 --2- 172.21.15.39:0/1340106980 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f466c074c10 0x7f466c0cfb40 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:50.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.877+0000 7f466be00640 1 -- 172.21.15.39:0/1340106980 >> 172.21.15.39:0/1340106980 conn(0x7f466c06cfc0 msgr2=0x7f466c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:50.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.877+0000 7f466be00640 1 -- 172.21.15.39:0/1340106980 shutdown_connections 2024-09-21T07:48:50.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.877+0000 7f466be00640 1 -- 172.21.15.39:0/1340106980 wait complete. 2024-09-21T07:48:50.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.880+0000 7f466be00640 1 Processor -- start 2024-09-21T07:48:50.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.880+0000 7f466be00640 1 -- start start 2024-09-21T07:48:50.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.881+0000 7f466be00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f466c06ecf0 0x7f466c1f8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:50.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.881+0000 7f466be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f466c074c10 0x7f466c0cacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:50.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.881+0000 7f466be00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f466c0d0550 0x7f466c0cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:50.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.881+0000 7f466be00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f466c15d0a0 con 0x7f466c074c10 2024-09-21T07:48:50.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.881+0000 7f466be00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f466c15cf20 con 0x7f466c06ecf0 2024-09-21T07:48:50.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.881+0000 7f466be00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f466c15d220 con 0x7f466c0d0550 2024-09-21T07:48:50.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.881+0000 7f4663e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f466c074c10 0x7f466c0cacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:50.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.881+0000 7f4663e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f466c074c10 0x7f466c0cacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51458/0 (socket says 172.21.15.39:51458) 2024-09-21T07:48:50.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.881+0000 7f4663e00640 1 -- 172.21.15.39:0/138798561 learned_addr learned my addr 172.21.15.39:0/138798561 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:50.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.882+0000 7f466aa00640 1 --2- 172.21.15.39:0/138798561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f466c06ecf0 0x7f466c1f8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:50.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.882+0000 7f466b400640 1 --2- 172.21.15.39:0/138798561 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f466c0d0550 0x7f466c0cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:50.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.882+0000 7f4663e00640 1 -- 172.21.15.39:0/138798561 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f466c0d0550 msgr2=0x7f466c0cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:50.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.882+0000 7f4663e00640 1 --2- 172.21.15.39:0/138798561 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f466c0d0550 0x7f466c0cb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:50.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.882+0000 7f4663e00640 1 -- 172.21.15.39:0/138798561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f466c06ecf0 msgr2=0x7f466c1f8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:50.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.882+0000 7f4663e00640 1 --2- 172.21.15.39:0/138798561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f466c06ecf0 0x7f466c1f8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:50.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.882+0000 7f4663e00640 1 -- 172.21.15.39:0/138798561 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f466c1e95f0 con 0x7f466c074c10 2024-09-21T07:48:50.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.882+0000 7f466aa00640 1 --2- 172.21.15.39:0/138798561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f466c06ecf0 0x7f466c1f8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:48:50.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.882+0000 7f466b400640 1 --2- 172.21.15.39:0/138798561 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f466c0d0550 0x7f466c0cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:48:50.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.882+0000 7f4663e00640 1 --2- 172.21.15.39:0/138798561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f466c074c10 0x7f466c0cacb0 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f465400d770 tx=0x7f465400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:50.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.882+0000 7f4662a00640 1 -- 172.21.15.39:0/138798561 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4654017070 con 0x7f466c074c10 2024-09-21T07:48:50.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.882+0000 7f4662a00640 1 -- 172.21.15.39:0/138798561 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4654009d20 con 0x7f466c074c10 2024-09-21T07:48:50.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.882+0000 7f466be00640 1 -- 172.21.15.39:0/138798561 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f466c0cba00 con 0x7f466c074c10 2024-09-21T07:48:50.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.882+0000 7f4662a00640 1 -- 172.21.15.39:0/138798561 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46540106c0 con 0x7f466c074c10 2024-09-21T07:48:50.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.883+0000 7f466be00640 1 -- 172.21.15.39:0/138798561 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f466c1e9b90 con 0x7f466c074c10 2024-09-21T07:48:50.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.884+0000 7f466be00640 1 -- 172.21.15.39:0/138798561 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f466c15d6d0 con 0x7f466c074c10 2024-09-21T07:48:50.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.885+0000 7f4662a00640 1 -- 172.21.15.39:0/138798561 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f465401a070 con 0x7f466c074c10 2024-09-21T07:48:50.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.886+0000 7f4662a00640 1 --2- 172.21.15.39:0/138798561 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4648078b90 0x7f464807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:50.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.886+0000 7f4662a00640 1 -- 172.21.15.39:0/138798561 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(219..219 src has 1..219) ==== 6988+0+0 (secure 0 0 0) 0x7f465409b3b0 con 0x7f466c074c10 2024-09-21T07:48:50.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.886+0000 7f466aa00640 1 --2- 172.21.15.39:0/138798561 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4648078b90 0x7f464807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:50.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.887+0000 7f466aa00640 1 --2- 172.21.15.39:0/138798561 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4648078b90 0x7f464807b0a0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f465c034000 tx=0x7f465c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:50.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:50.890+0000 7f4662a00640 1 -- 172.21.15.39:0/138798561 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f46540632f0 con 0x7f466c074c10 2024-09-21T07:48:51.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.126+0000 7f466be00640 1 -- 172.21.15.39:0/138798561 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f466c0cb790 con 0x7f466c074c10 2024-09-21T07:48:51.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.126+0000 7f4662a00640 1 -- 172.21.15.39:0/138798561 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 229 v229) ==== 95+0+5241 (secure 0 0 0) 0x7f4654062bf0 con 0x7f466c074c10 2024-09-21T07:48:51.128 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:48:51.128 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":229,"btime":"2024-09-21T07:48:50:615411+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8249,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/550771731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":550771731},{"type":"v1","addr":"172.21.15.148:6837","nonce":550771731}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":224},{"gid":9385,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1307677219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":1307677219},{"type":"v1","addr":"172.21.15.39:6835","nonce":1307677219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":224},{"gid":9391,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1256253563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1256253563},{"type":"v1","addr":"172.21.15.148:6836","nonce":1256253563}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":224}],"filesystems":[{"mdsmap":{"epoch":229,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:48:47.106474+0000","modified":"2024-09-21T07:48:50.124278+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10224},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10224":{"gid":10224,"name":"c","rank":0,"incarnation":225,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/2837221709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2837221709},{"type":"v1","addr":"172.21.15.39:6834","nonce":2837221709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10224,"qdb_cluster":[10224]},"id":12}]} 2024-09-21T07:48:51.129 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 229 2024-09-21T07:48:51.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.128+0000 7f466be00640 1 -- 172.21.15.39:0/138798561 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4648078b90 msgr2=0x7f464807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:51.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.128+0000 7f466be00640 1 --2- 172.21.15.39:0/138798561 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4648078b90 0x7f464807b0a0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f465c034000 tx=0x7f465c004730 comp rx=0 tx=0).stop 2024-09-21T07:48:51.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.128+0000 7f466be00640 1 -- 172.21.15.39:0/138798561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f466c074c10 msgr2=0x7f466c0cacb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:51.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.128+0000 7f466be00640 1 --2- 172.21.15.39:0/138798561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f466c074c10 0x7f466c0cacb0 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f465400d770 tx=0x7f465400dc30 comp rx=0 tx=0).stop 2024-09-21T07:48:51.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.129+0000 7f466be00640 1 -- 172.21.15.39:0/138798561 shutdown_connections 2024-09-21T07:48:51.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.129+0000 7f466be00640 1 --2- 172.21.15.39:0/138798561 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4648078b90 0x7f464807b0a0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:51.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.129+0000 7f466be00640 1 --2- 172.21.15.39:0/138798561 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f466c0d0550 0x7f466c0cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:51.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.129+0000 7f466be00640 1 --2- 172.21.15.39:0/138798561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f466c074c10 0x7f466c0cacb0 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:51.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.129+0000 7f466be00640 1 --2- 172.21.15.39:0/138798561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f466c06ecf0 0x7f466c1f8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:51.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.129+0000 7f466be00640 1 -- 172.21.15.39:0/138798561 >> 172.21.15.39:0/138798561 conn(0x7f466c06cfc0 msgr2=0x7f466c1561d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:51.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.129+0000 7f466be00640 1 -- 172.21.15.39:0/138798561 shutdown_connections 2024-09-21T07:48:51.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.129+0000 7f466be00640 1 -- 172.21.15.39:0/138798561 wait complete. 2024-09-21T07:48:51.154 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:48:51.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.378+0000 7f5d9dc00640 1 -- 172.21.15.39:0/133769935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d980cfa20 msgr2=0x7f5d980cd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:51.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.378+0000 7f5d9dc00640 1 --2- 172.21.15.39:0/133769935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d980cfa20 0x7f5d980cd5a0 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f5d88009830 tx=0x7f5d8802a510 comp rx=0 tx=0).stop 2024-09-21T07:48:51.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.378+0000 7f5d9dc00640 1 -- 172.21.15.39:0/133769935 shutdown_connections 2024-09-21T07:48:51.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.378+0000 7f5d9dc00640 1 --2- 172.21.15.39:0/133769935 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5d9815cc00 0x7f5d9815d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:51.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.378+0000 7f5d9dc00640 1 --2- 172.21.15.39:0/133769935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d980cfa20 0x7f5d980cd5a0 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:51.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.378+0000 7f5d9dc00640 1 -- 172.21.15.39:0/133769935 >> 172.21.15.39:0/133769935 conn(0x7f5d9806cfc0 msgr2=0x7f5d9806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:51.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.378+0000 7f5d9dc00640 1 -- 172.21.15.39:0/133769935 shutdown_connections 2024-09-21T07:48:51.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.378+0000 7f5d9dc00640 1 -- 172.21.15.39:0/133769935 wait complete. 2024-09-21T07:48:51.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.380+0000 7f5d9dc00640 1 Processor -- start 2024-09-21T07:48:51.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.380+0000 7f5d9dc00640 1 -- start start 2024-09-21T07:48:51.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.382+0000 7f5d9dc00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5d980cfa20 0x7f5d9815b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:51.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.382+0000 7f5d9dc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d980d04b0 0x7f5d981f3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:51.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.382+0000 7f5d9dc00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d9815cc00 0x7f5d98139370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:51.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.382+0000 7f5d9dc00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9806ee70 con 0x7f5d980d04b0 2024-09-21T07:48:51.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.382+0000 7f5d9dc00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9806ecf0 con 0x7f5d9815cc00 2024-09-21T07:48:51.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.382+0000 7f5d9dc00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f5d9806eff0 con 0x7f5d980cfa20 2024-09-21T07:48:51.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.382+0000 7f5d8fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d980d04b0 0x7f5d981f3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:51.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.382+0000 7f5d8fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d980d04b0 0x7f5d981f3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51472/0 (socket says 172.21.15.39:51472) 2024-09-21T07:48:51.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.382+0000 7f5d8fe00640 1 -- 172.21.15.39:0/2406156075 learned_addr learned my addr 172.21.15.39:0/2406156075 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:51.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.382+0000 7f5d9d200640 1 --2- 172.21.15.39:0/2406156075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d9815cc00 0x7f5d98139370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:51.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.382+0000 7f5d97e00640 1 --2- 172.21.15.39:0/2406156075 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5d980cfa20 0x7f5d9815b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:51.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.383+0000 7f5d8fe00640 1 -- 172.21.15.39:0/2406156075 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5d980cfa20 msgr2=0x7f5d9815b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:51.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.383+0000 7f5d8fe00640 1 --2- 172.21.15.39:0/2406156075 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5d980cfa20 0x7f5d9815b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:51.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.383+0000 7f5d8fe00640 1 -- 172.21.15.39:0/2406156075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d9815cc00 msgr2=0x7f5d98139370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:51.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.383+0000 7f5d8fe00640 1 --2- 172.21.15.39:0/2406156075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d9815cc00 0x7f5d98139370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:51.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.383+0000 7f5d8fe00640 1 -- 172.21.15.39:0/2406156075 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d9815a860 con 0x7f5d980d04b0 2024-09-21T07:48:51.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.383+0000 7f5d9d200640 1 --2- 172.21.15.39:0/2406156075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d9815cc00 0x7f5d98139370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:48:51.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.383+0000 7f5d8fe00640 1 --2- 172.21.15.39:0/2406156075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d980d04b0 0x7f5d981f3170 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f5d8000d770 tx=0x7f5d8000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:51.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.383+0000 7f5d94c00640 1 -- 172.21.15.39:0/2406156075 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d80017070 con 0x7f5d980d04b0 2024-09-21T07:48:51.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.383+0000 7f5d94c00640 1 -- 172.21.15.39:0/2406156075 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d80009d20 con 0x7f5d980d04b0 2024-09-21T07:48:51.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.383+0000 7f5d9dc00640 1 -- 172.21.15.39:0/2406156075 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d9815ab40 con 0x7f5d980d04b0 2024-09-21T07:48:51.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.383+0000 7f5d94c00640 1 -- 172.21.15.39:0/2406156075 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d80004280 con 0x7f5d980d04b0 2024-09-21T07:48:51.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.383+0000 7f5d9dc00640 1 -- 172.21.15.39:0/2406156075 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d9815b100 con 0x7f5d980d04b0 2024-09-21T07:48:51.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.384+0000 7f5d9dc00640 1 -- 172.21.15.39:0/2406156075 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d9815a5d0 con 0x7f5d980d04b0 2024-09-21T07:48:51.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.386+0000 7f5d94c00640 1 -- 172.21.15.39:0/2406156075 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5d80005020 con 0x7f5d980d04b0 2024-09-21T07:48:51.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.387+0000 7f5d94c00640 1 --2- 172.21.15.39:0/2406156075 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5d74078bc0 0x7f5d7407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:51.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.387+0000 7f5d94c00640 1 -- 172.21.15.39:0/2406156075 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(219..219 src has 1..219) ==== 6988+0+0 (secure 0 0 0) 0x7f5d8009b670 con 0x7f5d980d04b0 2024-09-21T07:48:51.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.387+0000 7f5d97e00640 1 --2- 172.21.15.39:0/2406156075 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5d74078bc0 0x7f5d7407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:51.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.388+0000 7f5d97e00640 1 --2- 172.21.15.39:0/2406156075 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5d74078bc0 0x7f5d7407b0d0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f5d88034000 tx=0x7f5d88004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:51.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.391+0000 7f5d94c00640 1 -- 172.21.15.39:0/2406156075 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5d80020020 con 0x7f5d980d04b0 2024-09-21T07:48:51.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.590+0000 7f5d9dc00640 1 -- 172.21.15.39:0/2406156075 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5d9806f410 con 0x7f5d980d04b0 2024-09-21T07:48:51.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.591+0000 7f5d94c00640 1 -- 172.21.15.39:0/2406156075 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v219) ==== 74+0+22506 (secure 0 0 0) 0x7f5d800635a0 con 0x7f5d980d04b0 2024-09-21T07:48:51.593 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:48:51.594 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":219,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:48:48.596106+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:48:45.059810+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"218","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-09-21T07:48:46.068380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"219","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]},{"pool":20,"snaps":[{"begin":80,"length":23},{"begin":181,"length":23}]},{"pool":22,"snaps":[{"begin":100,"length":1},{"begin":199,"length":1}]},{"pool":24,"snaps":[{"begin":2,"length":2}]},{"pool":26,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:48:51.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.594+0000 7f5d9dc00640 1 -- 172.21.15.39:0/2406156075 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5d74078bc0 msgr2=0x7f5d7407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:51.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.594+0000 7f5d9dc00640 1 --2- 172.21.15.39:0/2406156075 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5d74078bc0 0x7f5d7407b0d0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f5d88034000 tx=0x7f5d88004730 comp rx=0 tx=0).stop 2024-09-21T07:48:51.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.594+0000 7f5d9dc00640 1 -- 172.21.15.39:0/2406156075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d980d04b0 msgr2=0x7f5d981f3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:51.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.594+0000 7f5d9dc00640 1 --2- 172.21.15.39:0/2406156075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d980d04b0 0x7f5d981f3170 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f5d8000d770 tx=0x7f5d8000dc30 comp rx=0 tx=0).stop 2024-09-21T07:48:51.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.594+0000 7f5d9dc00640 1 -- 172.21.15.39:0/2406156075 shutdown_connections 2024-09-21T07:48:51.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.594+0000 7f5d9dc00640 1 --2- 172.21.15.39:0/2406156075 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5d74078bc0 0x7f5d7407b0d0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:51.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.594+0000 7f5d9dc00640 1 --2- 172.21.15.39:0/2406156075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d9815cc00 0x7f5d98139370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:51.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.594+0000 7f5d9dc00640 1 --2- 172.21.15.39:0/2406156075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d980d04b0 0x7f5d981f3170 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:51.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.594+0000 7f5d9dc00640 1 --2- 172.21.15.39:0/2406156075 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5d980cfa20 0x7f5d9815b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:51.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.594+0000 7f5d9dc00640 1 -- 172.21.15.39:0/2406156075 >> 172.21.15.39:0/2406156075 conn(0x7f5d9806cfc0 msgr2=0x7f5d980b1600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:51.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.594+0000 7f5d9dc00640 1 -- 172.21.15.39:0/2406156075 shutdown_connections 2024-09-21T07:48:51.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.594+0000 7f5d9dc00640 1 -- 172.21.15.39:0/2406156075 wait complete. 2024-09-21T07:48:51.619 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T07:48:51.619 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:48:51.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.846+0000 7f8ea0800640 1 -- 172.21.15.39:0/2177688088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e98073dd0 msgr2=0x7f8e9815d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:51.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.846+0000 7f8ea0800640 1 --2- 172.21.15.39:0/2177688088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e98073dd0 0x7f8e9815d5b0 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f8e980cef90 tx=0x7f8e8c02a4a0 comp rx=0 tx=0).stop 2024-09-21T07:48:51.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.846+0000 7f8ea0800640 1 -- 172.21.15.39:0/2177688088 shutdown_connections 2024-09-21T07:48:51.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.846+0000 7f8ea0800640 1 --2- 172.21.15.39:0/2177688088 >> v2:172.21.15.148:3300/0 conn(0x7f8e9815daf0 0x7f8e980cf450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:51.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.846+0000 7f8ea0800640 1 --2- 172.21.15.39:0/2177688088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e98073dd0 0x7f8e9815d5b0 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:51.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.846+0000 7f8ea0800640 1 -- 172.21.15.39:0/2177688088 >> 172.21.15.39:0/2177688088 conn(0x7f8e9806cfc0 msgr2=0x7f8e9806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:51.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.846+0000 7f8ea0800640 1 -- 172.21.15.39:0/2177688088 shutdown_connections 2024-09-21T07:48:51.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.846+0000 7f8ea0800640 1 -- 172.21.15.39:0/2177688088 wait complete. 2024-09-21T07:48:51.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.849+0000 7f8ea0800640 1 Processor -- start 2024-09-21T07:48:51.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.849+0000 7f8ea0800640 1 -- start start 2024-09-21T07:48:51.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.850+0000 7f8ea0800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e98073dd0 0x7f8e9814de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:51.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.850+0000 7f8ea0800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e980cf990 0x7f8e981e92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:51.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.850+0000 7f8ea0800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e9815daf0 0x7f8e981e9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:51.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.850+0000 7f8ea0800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8e980cd1d0 con 0x7f8e98073dd0 2024-09-21T07:48:51.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.850+0000 7f8ea0800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8e980cd050 con 0x7f8e9815daf0 2024-09-21T07:48:51.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.850+0000 7f8ea0800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f8e980cd350 con 0x7f8e980cf990 2024-09-21T07:48:51.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.851+0000 7f8e9de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e98073dd0 0x7f8e9814de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:51.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.851+0000 7f8e9de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e98073dd0 0x7f8e9814de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51490/0 (socket says 172.21.15.39:51490) 2024-09-21T07:48:51.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.851+0000 7f8e9de00640 1 -- 172.21.15.39:0/1707359798 learned_addr learned my addr 172.21.15.39:0/1707359798 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:51.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.851+0000 7f8e9de00640 1 -- 172.21.15.39:0/1707359798 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e980cf990 msgr2=0x7f8e981e92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:51.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.851+0000 7f8e9d400640 1 --2- 172.21.15.39:0/1707359798 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e980cf990 0x7f8e981e92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:51.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.851+0000 7f8e9e800640 1 --2- 172.21.15.39:0/1707359798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e9815daf0 0x7f8e981e9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:51.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.851+0000 7f8e9de00640 1 --2- 172.21.15.39:0/1707359798 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e980cf990 0x7f8e981e92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:51.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.851+0000 7f8e9de00640 1 -- 172.21.15.39:0/1707359798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e9815daf0 msgr2=0x7f8e981e9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:51.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.851+0000 7f8e9de00640 1 --2- 172.21.15.39:0/1707359798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e9815daf0 0x7f8e981e9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:51.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.851+0000 7f8e9de00640 1 -- 172.21.15.39:0/1707359798 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e98153920 con 0x7f8e98073dd0 2024-09-21T07:48:51.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.851+0000 7f8e9e800640 1 --2- 172.21.15.39:0/1707359798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e9815daf0 0x7f8e981e9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:48:51.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.851+0000 7f8e9d400640 1 --2- 172.21.15.39:0/1707359798 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e980cf990 0x7f8e981e92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:48:51.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.851+0000 7f8e9de00640 1 --2- 172.21.15.39:0/1707359798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e98073dd0 0x7f8e9814de40 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c02aa90 tx=0x7f8e8c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:51.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.852+0000 7f8e92000640 1 -- 172.21.15.39:0/1707359798 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c005c90 con 0x7f8e98073dd0 2024-09-21T07:48:51.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.852+0000 7f8e92000640 1 -- 172.21.15.39:0/1707359798 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e8c031e50 con 0x7f8e98073dd0 2024-09-21T07:48:51.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.852+0000 7f8ea0800640 1 -- 172.21.15.39:0/1707359798 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e98153c00 con 0x7f8e98073dd0 2024-09-21T07:48:51.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.852+0000 7f8e92000640 1 -- 172.21.15.39:0/1707359798 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c005240 con 0x7f8e98073dd0 2024-09-21T07:48:51.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.852+0000 7f8ea0800640 1 -- 172.21.15.39:0/1707359798 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e981541c0 con 0x7f8e98073dd0 2024-09-21T07:48:51.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.853+0000 7f8ea0800640 1 -- 172.21.15.39:0/1707359798 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e980cd800 con 0x7f8e98073dd0 2024-09-21T07:48:51.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.854+0000 7f8e92000640 1 -- 172.21.15.39:0/1707359798 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8e8c0053e0 con 0x7f8e98073dd0 2024-09-21T07:48:51.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.855+0000 7f8e92000640 1 --2- 172.21.15.39:0/1707359798 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8e7c078bc0 0x7f8e7c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:51.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.855+0000 7f8e92000640 1 -- 172.21.15.39:0/1707359798 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(219..219 src has 1..219) ==== 6988+0+0 (secure 0 0 0) 0x7f8e8c0bc310 con 0x7f8e98073dd0 2024-09-21T07:48:51.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.856+0000 7f8e9d400640 1 --2- 172.21.15.39:0/1707359798 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8e7c078bc0 0x7f8e7c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:51.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.857+0000 7f8e9d400640 1 --2- 172.21.15.39:0/1707359798 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8e7c078bc0 0x7f8e7c07b0d0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f8e980b2ee0 tx=0x7f8e88005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:51.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:51.860+0000 7f8e92000640 1 -- 172.21.15.39:0/1707359798 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8e8c03f030 con 0x7f8e98073dd0 2024-09-21T07:48:52.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.053+0000 7f8ea0800640 1 -- 172.21.15.39:0/1707359798 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8e9806e110 con 0x7f8e7c078bc0 2024-09-21T07:48:52.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.062+0000 7f8e92000640 1 -- 172.21.15.39:0/1707359798 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+493904 (secure 0 0 0) 0x7f8e9806e110 con 0x7f8e7c078bc0 2024-09-21T07:48:52.070 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:48:52.074 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:48:52.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.068+0000 7f8ea0800640 1 -- 172.21.15.39:0/1707359798 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8e7c078bc0 msgr2=0x7f8e7c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:52.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.068+0000 7f8ea0800640 1 --2- 172.21.15.39:0/1707359798 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8e7c078bc0 0x7f8e7c07b0d0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f8e980b2ee0 tx=0x7f8e88005f90 comp rx=0 tx=0).stop 2024-09-21T07:48:52.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.068+0000 7f8ea0800640 1 -- 172.21.15.39:0/1707359798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e98073dd0 msgr2=0x7f8e9814de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:52.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.068+0000 7f8ea0800640 1 --2- 172.21.15.39:0/1707359798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e98073dd0 0x7f8e9814de40 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c02aa90 tx=0x7f8e8c033c20 comp rx=0 tx=0).stop 2024-09-21T07:48:52.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.068+0000 7f8ea0800640 1 -- 172.21.15.39:0/1707359798 shutdown_connections 2024-09-21T07:48:52.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.068+0000 7f8ea0800640 1 --2- 172.21.15.39:0/1707359798 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8e7c078bc0 0x7f8e7c07b0d0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:52.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.068+0000 7f8ea0800640 1 --2- 172.21.15.39:0/1707359798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e9815daf0 0x7f8e981e9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:52.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.068+0000 7f8ea0800640 1 --2- 172.21.15.39:0/1707359798 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8e980cf990 0x7f8e981e92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:52.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.068+0000 7f8ea0800640 1 --2- 172.21.15.39:0/1707359798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e98073dd0 0x7f8e9814de40 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:52.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.068+0000 7f8ea0800640 1 -- 172.21.15.39:0/1707359798 >> 172.21.15.39:0/1707359798 conn(0x7f8e9806cfc0 msgr2=0x7f8e98156ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:52.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.069+0000 7f8ea0800640 1 -- 172.21.15.39:0/1707359798 shutdown_connections 2024-09-21T07:48:52.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.069+0000 7f8ea0800640 1 -- 172.21.15.39:0/1707359798 wait complete. 2024-09-21T07:48:52.092 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":689,"stamp":"2024-09-21T07:48:51.852637+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591144,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":135,"num_write_kb":1638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":284,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":408160,"kb_used_data":22568,"kb_used_omap":12,"kb_used_meta":385395,"kb_avail":754566560,"statfs":{"total":773094113280,"available":772676157440,"internally_reserved":0,"allocated":23109632,"data_stored":12862950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":394645040},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":776,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001975"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.590080+0000","last_change":"2024-09-21T07:48:46.590080+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.590080+0000","last_undegraded":"2024-09-21T07:48:46.590080+0000","last_fullsized":"2024-09-21T07:48:46.590080+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.610021+0000","last_change":"2024-09-21T07:48:47.608680+0000","last_active":"2024-09-21T07:48:48.610021+0000","last_peered":"2024-09-21T07:48:48.610021+0000","last_clean":"2024-09-21T07:48:48.610021+0000","last_became_active":"2024-09-21T07:48:47.606772+0000","last_became_peered":"2024-09-21T07:48:47.606772+0000","last_unstale":"2024-09-21T07:48:48.610021+0000","last_undegraded":"2024-09-21T07:48:48.610021+0000","last_fullsized":"2024-09-21T07:48:48.610021+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:13.951566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.626629+0000","last_change":"2024-09-21T07:48:47.622897+0000","last_active":"2024-09-21T07:48:47.626629+0000","last_peered":"2024-09-21T07:48:47.626629+0000","last_clean":"2024-09-21T07:48:47.626629+0000","last_became_active":"2024-09-21T07:48:47.622568+0000","last_became_peered":"2024-09-21T07:48:47.622568+0000","last_unstale":"2024-09-21T07:48:47.626629+0000","last_undegraded":"2024-09-21T07:48:47.626629+0000","last_fullsized":"2024-09-21T07:48:47.626629+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:53:43.260855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.600167+0000","last_change":"2024-09-21T07:48:46.600167+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.600167+0000","last_undegraded":"2024-09-21T07:48:46.600167+0000","last_fullsized":"2024-09-21T07:48:46.600167+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.606103+0000","last_change":"2024-09-21T07:48:47.602747+0000","last_active":"2024-09-21T07:48:47.606103+0000","last_peered":"2024-09-21T07:48:47.606103+0000","last_clean":"2024-09-21T07:48:47.606103+0000","last_became_active":"2024-09-21T07:48:47.602453+0000","last_became_peered":"2024-09-21T07:48:47.602453+0000","last_unstale":"2024-09-21T07:48:47.606103+0000","last_undegraded":"2024-09-21T07:48:47.606103+0000","last_fullsized":"2024-09-21T07:48:47.606103+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:29.795876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.597012+0000","last_change":"2024-09-21T07:48:46.597012+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.597012+0000","last_undegraded":"2024-09-21T07:48:46.597012+0000","last_fullsized":"2024-09-21T07:48:46.597012+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.604038+0000","last_change":"2024-09-21T07:48:47.602027+0000","last_active":"2024-09-21T07:48:47.604038+0000","last_peered":"2024-09-21T07:48:47.604038+0000","last_clean":"2024-09-21T07:48:47.604038+0000","last_became_active":"2024-09-21T07:48:47.601703+0000","last_became_peered":"2024-09-21T07:48:47.601703+0000","last_unstale":"2024-09-21T07:48:47.604038+0000","last_undegraded":"2024-09-21T07:48:47.604038+0000","last_fullsized":"2024-09-21T07:48:47.604038+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:39.401545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.632462+0000","last_change":"2024-09-21T07:48:47.627224+0000","last_active":"2024-09-21T07:48:47.632462+0000","last_peered":"2024-09-21T07:48:47.632462+0000","last_clean":"2024-09-21T07:48:47.632462+0000","last_became_active":"2024-09-21T07:48:47.626767+0000","last_became_peered":"2024-09-21T07:48:47.626767+0000","last_unstale":"2024-09-21T07:48:47.632462+0000","last_undegraded":"2024-09-21T07:48:47.632462+0000","last_fullsized":"2024-09-21T07:48:47.632462+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:44:57.315947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.623586+0000","last_change":"2024-09-21T07:48:47.617745+0000","last_active":"2024-09-21T07:48:47.623586+0000","last_peered":"2024-09-21T07:48:47.623586+0000","last_clean":"2024-09-21T07:48:47.623586+0000","last_became_active":"2024-09-21T07:48:47.614030+0000","last_became_peered":"2024-09-21T07:48:47.614030+0000","last_unstale":"2024-09-21T07:48:47.623586+0000","last_undegraded":"2024-09-21T07:48:47.623586+0000","last_fullsized":"2024-09-21T07:48:47.623586+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:44:50.001056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.631705+0000","last_change":"2024-09-21T07:48:47.621852+0000","last_active":"2024-09-21T07:48:47.631705+0000","last_peered":"2024-09-21T07:48:47.631705+0000","last_clean":"2024-09-21T07:48:47.631705+0000","last_became_active":"2024-09-21T07:48:47.617439+0000","last_became_peered":"2024-09-21T07:48:47.617439+0000","last_unstale":"2024-09-21T07:48:47.631705+0000","last_undegraded":"2024-09-21T07:48:47.631705+0000","last_fullsized":"2024-09-21T07:48:47.631705+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:17:23.080659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.632943+0000","last_change":"2024-09-21T07:48:47.632098+0000","last_active":"2024-09-21T07:48:47.632943+0000","last_peered":"2024-09-21T07:48:47.632943+0000","last_clean":"2024-09-21T07:48:47.632943+0000","last_became_active":"2024-09-21T07:48:47.631802+0000","last_became_peered":"2024-09-21T07:48:47.631802+0000","last_unstale":"2024-09-21T07:48:47.632943+0000","last_undegraded":"2024-09-21T07:48:47.632943+0000","last_fullsized":"2024-09-21T07:48:47.632943+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:21.715374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.621185+0000","last_change":"2024-09-21T07:48:47.617229+0000","last_active":"2024-09-21T07:48:47.621185+0000","last_peered":"2024-09-21T07:48:47.621185+0000","last_clean":"2024-09-21T07:48:47.621185+0000","last_became_active":"2024-09-21T07:48:47.616863+0000","last_became_peered":"2024-09-21T07:48:47.616863+0000","last_unstale":"2024-09-21T07:48:47.621185+0000","last_undegraded":"2024-09-21T07:48:47.621185+0000","last_fullsized":"2024-09-21T07:48:47.621185+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:51.375781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.599782+0000","last_change":"2024-09-21T07:48:46.599782+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.599782+0000","last_undegraded":"2024-09-21T07:48:46.599782+0000","last_fullsized":"2024-09-21T07:48:46.599782+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.632509+0000","last_change":"2024-09-21T07:48:47.623337+0000","last_active":"2024-09-21T07:48:47.632509+0000","last_peered":"2024-09-21T07:48:47.632509+0000","last_clean":"2024-09-21T07:48:47.632509+0000","last_became_active":"2024-09-21T07:48:47.620228+0000","last_became_peered":"2024-09-21T07:48:47.620228+0000","last_unstale":"2024-09-21T07:48:47.632509+0000","last_undegraded":"2024-09-21T07:48:47.632509+0000","last_fullsized":"2024-09-21T07:48:47.632509+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:31.930727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051089+0000","last_change":"2024-09-21T07:48:47.627205+0000","last_active":"2024-09-21T07:48:50.051089+0000","last_peered":"2024-09-21T07:48:50.051089+0000","last_clean":"2024-09-21T07:48:50.051089+0000","last_became_active":"2024-09-21T07:48:47.626871+0000","last_became_peered":"2024-09-21T07:48:47.626871+0000","last_unstale":"2024-09-21T07:48:50.051089+0000","last_undegraded":"2024-09-21T07:48:50.051089+0000","last_fullsized":"2024-09-21T07:48:50.051089+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:57.118108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.629854+0000","last_change":"2024-09-21T07:48:47.627220+0000","last_active":"2024-09-21T07:48:47.629854+0000","last_peered":"2024-09-21T07:48:47.629854+0000","last_clean":"2024-09-21T07:48:47.629854+0000","last_became_active":"2024-09-21T07:48:47.626875+0000","last_became_peered":"2024-09-21T07:48:47.626875+0000","last_unstale":"2024-09-21T07:48:47.629854+0000","last_undegraded":"2024-09-21T07:48:47.629854+0000","last_fullsized":"2024-09-21T07:48:47.629854+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:11.801442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.628849+0000","last_change":"2024-09-21T07:48:47.625920+0000","last_active":"2024-09-21T07:48:47.628849+0000","last_peered":"2024-09-21T07:48:47.628849+0000","last_clean":"2024-09-21T07:48:47.628849+0000","last_became_active":"2024-09-21T07:48:47.625605+0000","last_became_peered":"2024-09-21T07:48:47.625605+0000","last_unstale":"2024-09-21T07:48:47.628849+0000","last_undegraded":"2024-09-21T07:48:47.628849+0000","last_fullsized":"2024-09-21T07:48:47.628849+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:17:04.946971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.629605+0000","last_change":"2024-09-21T07:48:47.628044+0000","last_active":"2024-09-21T07:48:47.629605+0000","last_peered":"2024-09-21T07:48:47.629605+0000","last_clean":"2024-09-21T07:48:47.629605+0000","last_became_active":"2024-09-21T07:48:47.626515+0000","last_became_peered":"2024-09-21T07:48:47.626515+0000","last_unstale":"2024-09-21T07:48:47.629605+0000","last_undegraded":"2024-09-21T07:48:47.629605+0000","last_fullsized":"2024-09-21T07:48:47.629605+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:40.234280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.632098+0000","last_change":"2024-09-21T07:48:47.623935+0000","last_active":"2024-09-21T07:48:47.632098+0000","last_peered":"2024-09-21T07:48:47.632098+0000","last_clean":"2024-09-21T07:48:47.632098+0000","last_became_active":"2024-09-21T07:48:47.623233+0000","last_became_peered":"2024-09-21T07:48:47.623233+0000","last_unstale":"2024-09-21T07:48:47.632098+0000","last_undegraded":"2024-09-21T07:48:47.632098+0000","last_fullsized":"2024-09-21T07:48:47.632098+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:29.332356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.630373+0000","last_change":"2024-09-21T07:48:47.617959+0000","last_active":"2024-09-21T07:48:47.630373+0000","last_peered":"2024-09-21T07:48:47.630373+0000","last_clean":"2024-09-21T07:48:47.630373+0000","last_became_active":"2024-09-21T07:48:47.617595+0000","last_became_peered":"2024-09-21T07:48:47.617595+0000","last_unstale":"2024-09-21T07:48:47.630373+0000","last_undegraded":"2024-09-21T07:48:47.630373+0000","last_fullsized":"2024-09-21T07:48:47.630373+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:16.164291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.631855+0000","last_change":"2024-09-21T07:48:47.621502+0000","last_active":"2024-09-21T07:48:47.631855+0000","last_peered":"2024-09-21T07:48:47.631855+0000","last_clean":"2024-09-21T07:48:47.631855+0000","last_became_active":"2024-09-21T07:48:47.616971+0000","last_became_peered":"2024-09-21T07:48:47.616971+0000","last_unstale":"2024-09-21T07:48:47.631855+0000","last_undegraded":"2024-09-21T07:48:47.631855+0000","last_fullsized":"2024-09-21T07:48:47.631855+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:59.711618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.631967+0000","last_change":"2024-09-21T07:48:47.622917+0000","last_active":"2024-09-21T07:48:47.631967+0000","last_peered":"2024-09-21T07:48:47.631967+0000","last_clean":"2024-09-21T07:48:47.631967+0000","last_became_active":"2024-09-21T07:48:47.619537+0000","last_became_peered":"2024-09-21T07:48:47.619537+0000","last_unstale":"2024-09-21T07:48:47.631967+0000","last_undegraded":"2024-09-21T07:48:47.631967+0000","last_fullsized":"2024-09-21T07:48:47.631967+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:14:17.772422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.610863+0000","last_change":"2024-09-21T07:48:47.598763+0000","last_active":"2024-09-21T07:48:48.610863+0000","last_peered":"2024-09-21T07:48:48.610863+0000","last_clean":"2024-09-21T07:48:48.610863+0000","last_became_active":"2024-09-21T07:48:47.598446+0000","last_became_peered":"2024-09-21T07:48:47.598446+0000","last_unstale":"2024-09-21T07:48:48.610863+0000","last_undegraded":"2024-09-21T07:48:48.610863+0000","last_fullsized":"2024-09-21T07:48:48.610863+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:48.912495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.607435+0000","last_change":"2024-09-21T07:48:47.596907+0000","last_active":"2024-09-21T07:48:47.607435+0000","last_peered":"2024-09-21T07:48:47.607435+0000","last_clean":"2024-09-21T07:48:47.607435+0000","last_became_active":"2024-09-21T07:48:47.596568+0000","last_became_peered":"2024-09-21T07:48:47.596568+0000","last_unstale":"2024-09-21T07:48:47.607435+0000","last_undegraded":"2024-09-21T07:48:47.607435+0000","last_fullsized":"2024-09-21T07:48:47.607435+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:47.548875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.623612+0000","last_change":"2024-09-21T07:48:47.620026+0000","last_active":"2024-09-21T07:48:47.623612+0000","last_peered":"2024-09-21T07:48:47.623612+0000","last_clean":"2024-09-21T07:48:47.623612+0000","last_became_active":"2024-09-21T07:48:47.619713+0000","last_became_peered":"2024-09-21T07:48:47.619713+0000","last_unstale":"2024-09-21T07:48:47.623612+0000","last_undegraded":"2024-09-21T07:48:47.623612+0000","last_fullsized":"2024-09-21T07:48:47.623612+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:15.278604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.592481+0000","last_change":"2024-09-21T07:48:46.592481+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.592481+0000","last_undegraded":"2024-09-21T07:48:46.592481+0000","last_fullsized":"2024-09-21T07:48:46.592481+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.627562+0000","last_change":"2024-09-21T07:48:47.625275+0000","last_active":"2024-09-21T07:48:47.627562+0000","last_peered":"2024-09-21T07:48:47.627562+0000","last_clean":"2024-09-21T07:48:47.627562+0000","last_became_active":"2024-09-21T07:48:47.624985+0000","last_became_peered":"2024-09-21T07:48:47.624985+0000","last_unstale":"2024-09-21T07:48:47.627562+0000","last_undegraded":"2024-09-21T07:48:47.627562+0000","last_fullsized":"2024-09-21T07:48:47.627562+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:22.189614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.612236+0000","last_change":"2024-09-21T07:48:47.608774+0000","last_active":"2024-09-21T07:48:47.612236+0000","last_peered":"2024-09-21T07:48:47.612236+0000","last_clean":"2024-09-21T07:48:47.612236+0000","last_became_active":"2024-09-21T07:48:47.608457+0000","last_became_peered":"2024-09-21T07:48:47.608457+0000","last_unstale":"2024-09-21T07:48:47.612236+0000","last_undegraded":"2024-09-21T07:48:47.612236+0000","last_fullsized":"2024-09-21T07:48:47.612236+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:25.633315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.613015+0000","last_change":"2024-09-21T07:48:47.605730+0000","last_active":"2024-09-21T07:48:47.613015+0000","last_peered":"2024-09-21T07:48:47.613015+0000","last_clean":"2024-09-21T07:48:47.613015+0000","last_became_active":"2024-09-21T07:48:47.605433+0000","last_became_peered":"2024-09-21T07:48:47.605433+0000","last_unstale":"2024-09-21T07:48:47.613015+0000","last_undegraded":"2024-09-21T07:48:47.613015+0000","last_fullsized":"2024-09-21T07:48:47.613015+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:15.258575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051126+0000","last_change":"2024-09-21T07:48:47.604557+0000","last_active":"2024-09-21T07:48:50.051126+0000","last_peered":"2024-09-21T07:48:50.051126+0000","last_clean":"2024-09-21T07:48:50.051126+0000","last_became_active":"2024-09-21T07:48:47.604196+0000","last_became_peered":"2024-09-21T07:48:47.604196+0000","last_unstale":"2024-09-21T07:48:50.051126+0000","last_undegraded":"2024-09-21T07:48:50.051126+0000","last_fullsized":"2024-09-21T07:48:50.051126+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:56.323906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.595955+0000","last_change":"2024-09-21T07:48:46.595955+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.595955+0000","last_undegraded":"2024-09-21T07:48:46.595955+0000","last_fullsized":"2024-09-21T07:48:46.595955+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.595810+0000","last_change":"2024-09-21T07:48:46.595810+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.595810+0000","last_undegraded":"2024-09-21T07:48:46.595810+0000","last_fullsized":"2024-09-21T07:48:46.595810+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.591446+0000","last_change":"2024-09-21T07:48:46.591446+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.591446+0000","last_undegraded":"2024-09-21T07:48:46.591446+0000","last_fullsized":"2024-09-21T07:48:46.591446+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1a","version":"218'1","reported_seq":14,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.622300+0000","last_change":"2024-09-21T07:48:46.603075+0000","last_active":"2024-09-21T07:48:47.622300+0000","last_peered":"2024-09-21T07:48:47.622300+0000","last_clean":"2024-09-21T07:48:47.622300+0000","last_became_active":"2024-09-21T07:48:46.602882+0000","last_became_peered":"2024-09-21T07:48:46.602882+0000","last_unstale":"2024-09-21T07:48:47.622300+0000","last_undegraded":"2024-09-21T07:48:47.622300+0000","last_fullsized":"2024-09-21T07:48:47.622300+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:49.025973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":301,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.602282+0000","last_change":"2024-09-21T07:35:27.879605+0000","last_active":"2024-09-21T07:48:46.602282+0000","last_peered":"2024-09-21T07:48:46.602282+0000","last_clean":"2024-09-21T07:48:46.602282+0000","last_became_active":"2024-09-21T07:35:27.879274+0000","last_became_peered":"2024-09-21T07:35:27.879274+0000","last_unstale":"2024-09-21T07:48:46.602282+0000","last_undegraded":"2024-09-21T07:48:46.602282+0000","last_fullsized":"2024-09-21T07:48:46.602282+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:24.241636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":316,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609470+0000","last_change":"2024-09-21T07:35:29.491336+0000","last_active":"2024-09-21T07:48:48.609470+0000","last_peered":"2024-09-21T07:48:48.609470+0000","last_clean":"2024-09-21T07:48:48.609470+0000","last_became_active":"2024-09-21T07:35:26.866484+0000","last_became_peered":"2024-09-21T07:35:26.866484+0000","last_unstale":"2024-09-21T07:48:48.609470+0000","last_undegraded":"2024-09-21T07:48:48.609470+0000","last_fullsized":"2024-09-21T07:48:48.609470+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:10.449665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076010100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":487,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.596417+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:48:46.596417+0000","last_peered":"2024-09-21T07:48:46.596417+0000","last_clean":"2024-09-21T07:48:46.596417+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:48:46.596417+0000","last_undegraded":"2024-09-21T07:48:46.596417+0000","last_fullsized":"2024-09-21T07:48:46.596417+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":375,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.590586+0000","last_change":"2024-09-21T07:34:24.539248+0000","last_active":"2024-09-21T07:48:46.590586+0000","last_peered":"2024-09-21T07:48:46.590586+0000","last_clean":"2024-09-21T07:48:46.590586+0000","last_became_active":"2024-09-21T07:34:23.273465+0000","last_became_peered":"2024-09-21T07:34:23.273465+0000","last_unstale":"2024-09-21T07:48:46.590586+0000","last_undegraded":"2024-09-21T07:48:46.590586+0000","last_fullsized":"2024-09-21T07:48:46.590586+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:55.517452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060627700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608458+0000","last_change":"2024-09-21T07:48:47.612024+0000","last_active":"2024-09-21T07:48:48.608458+0000","last_peered":"2024-09-21T07:48:48.608458+0000","last_clean":"2024-09-21T07:48:48.608458+0000","last_became_active":"2024-09-21T07:48:47.601212+0000","last_became_peered":"2024-09-21T07:48:47.601212+0000","last_unstale":"2024-09-21T07:48:48.608458+0000","last_undegraded":"2024-09-21T07:48:48.608458+0000","last_fullsized":"2024-09-21T07:48:48.608458+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:20.715873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"218'4","reported_seq":19,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051794+0000","last_change":"2024-09-21T07:48:46.613621+0000","last_active":"2024-09-21T07:48:50.051794+0000","last_peered":"2024-09-21T07:48:50.051794+0000","last_clean":"2024-09-21T07:48:50.051794+0000","last_became_active":"2024-09-21T07:48:46.613266+0000","last_became_peered":"2024-09-21T07:48:46.613266+0000","last_unstale":"2024-09-21T07:48:50.051794+0000","last_undegraded":"2024-09-21T07:48:50.051794+0000","last_fullsized":"2024-09-21T07:48:50.051794+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:40.639085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":309,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608414+0000","last_change":"2024-09-21T07:35:27.864492+0000","last_active":"2024-09-21T07:48:48.608414+0000","last_peered":"2024-09-21T07:48:48.608414+0000","last_clean":"2024-09-21T07:48:48.608414+0000","last_became_active":"2024-09-21T07:35:27.864176+0000","last_became_peered":"2024-09-21T07:35:27.864176+0000","last_unstale":"2024-09-21T07:48:48.608414+0000","last_undegraded":"2024-09-21T07:48:48.608414+0000","last_fullsized":"2024-09-21T07:48:48.608414+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:05.153467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":308,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.602895+0000","last_change":"2024-09-21T07:35:29.493678+0000","last_active":"2024-09-21T07:48:46.602895+0000","last_peered":"2024-09-21T07:48:46.602895+0000","last_clean":"2024-09-21T07:48:46.602895+0000","last_became_active":"2024-09-21T07:35:26.870749+0000","last_became_peered":"2024-09-21T07:35:26.870749+0000","last_unstale":"2024-09-21T07:48:46.602895+0000","last_undegraded":"2024-09-21T07:48:46.602895+0000","last_fullsized":"2024-09-21T07:48:46.602895+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:55.367877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010159570000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":359,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051441+0000","last_change":"2024-09-21T07:34:24.597274+0000","last_active":"2024-09-21T07:48:50.051441+0000","last_peered":"2024-09-21T07:48:50.051441+0000","last_clean":"2024-09-21T07:48:50.051441+0000","last_became_active":"2024-09-21T07:34:23.279560+0000","last_became_peered":"2024-09-21T07:34:23.279560+0000","last_unstale":"2024-09-21T07:48:50.051441+0000","last_undegraded":"2024-09-21T07:48:50.051441+0000","last_fullsized":"2024-09-21T07:48:50.051441+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:15.845265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059699799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.588781+0000","last_change":"2024-09-21T07:48:46.588781+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.588781+0000","last_undegraded":"2024-09-21T07:48:46.588781+0000","last_fullsized":"2024-09-21T07:48:46.588781+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.613288+0000","last_change":"2024-09-21T07:48:46.610499+0000","last_active":"2024-09-21T07:48:46.613288+0000","last_peered":"2024-09-21T07:48:46.613288+0000","last_clean":"2024-09-21T07:48:46.613288+0000","last_became_active":"2024-09-21T07:48:46.610137+0000","last_became_peered":"2024-09-21T07:48:46.610137+0000","last_unstale":"2024-09-21T07:48:46.613288+0000","last_undegraded":"2024-09-21T07:48:46.613288+0000","last_fullsized":"2024-09-21T07:48:46.613288+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:29:58.572619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":308,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.592520+0000","last_change":"2024-09-21T07:35:27.874387+0000","last_active":"2024-09-21T07:48:47.592520+0000","last_peered":"2024-09-21T07:48:47.592520+0000","last_clean":"2024-09-21T07:48:47.592520+0000","last_became_active":"2024-09-21T07:35:27.873277+0000","last_became_peered":"2024-09-21T07:35:27.873277+0000","last_unstale":"2024-09-21T07:48:47.592520+0000","last_undegraded":"2024-09-21T07:48:47.592520+0000","last_fullsized":"2024-09-21T07:48:47.592520+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:04.581463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":309,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051894+0000","last_change":"2024-09-21T07:35:29.387094+0000","last_active":"2024-09-21T07:48:50.051894+0000","last_peered":"2024-09-21T07:48:50.051894+0000","last_clean":"2024-09-21T07:48:50.051894+0000","last_became_active":"2024-09-21T07:35:26.874757+0000","last_became_peered":"2024-09-21T07:35:26.874757+0000","last_unstale":"2024-09-21T07:48:50.051894+0000","last_undegraded":"2024-09-21T07:48:50.051894+0000","last_fullsized":"2024-09-21T07:48:50.051894+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:33.542750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061065700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":361,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.604051+0000","last_change":"2024-09-21T07:34:24.259906+0000","last_active":"2024-09-21T07:48:46.604051+0000","last_peered":"2024-09-21T07:48:46.604051+0000","last_clean":"2024-09-21T07:48:46.604051+0000","last_became_active":"2024-09-21T07:34:23.635282+0000","last_became_peered":"2024-09-21T07:34:23.635282+0000","last_unstale":"2024-09-21T07:48:46.604051+0000","last_undegraded":"2024-09-21T07:48:46.604051+0000","last_fullsized":"2024-09-21T07:48:46.604051+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:00.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001527664,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.632254+0000","last_change":"2024-09-21T07:48:47.622852+0000","last_active":"2024-09-21T07:48:47.632254+0000","last_peered":"2024-09-21T07:48:47.632254+0000","last_clean":"2024-09-21T07:48:47.632254+0000","last_became_active":"2024-09-21T07:48:47.619895+0000","last_became_peered":"2024-09-21T07:48:47.619895+0000","last_unstale":"2024-09-21T07:48:47.632254+0000","last_undegraded":"2024-09-21T07:48:47.632254+0000","last_fullsized":"2024-09-21T07:48:47.632254+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:56:05.060203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"218'2","reported_seq":17,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609737+0000","last_change":"2024-09-21T07:48:46.606441+0000","last_active":"2024-09-21T07:48:48.609737+0000","last_peered":"2024-09-21T07:48:48.609737+0000","last_clean":"2024-09-21T07:48:48.609737+0000","last_became_active":"2024-09-21T07:48:46.604712+0000","last_became_peered":"2024-09-21T07:48:46.604712+0000","last_unstale":"2024-09-21T07:48:48.609737+0000","last_undegraded":"2024-09-21T07:48:48.609737+0000","last_fullsized":"2024-09-21T07:48:48.609737+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:22.700415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":308,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.593732+0000","last_change":"2024-09-21T07:35:27.873645+0000","last_active":"2024-09-21T07:48:47.593732+0000","last_peered":"2024-09-21T07:48:47.593732+0000","last_clean":"2024-09-21T07:48:47.593732+0000","last_became_active":"2024-09-21T07:35:27.873245+0000","last_became_peered":"2024-09-21T07:35:27.873245+0000","last_unstale":"2024-09-21T07:48:47.593732+0000","last_undegraded":"2024-09-21T07:48:47.593732+0000","last_fullsized":"2024-09-21T07:48:47.593732+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:55.090953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":314,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.592916+0000","last_change":"2024-09-21T07:35:28.861744+0000","last_active":"2024-09-21T07:48:47.592916+0000","last_peered":"2024-09-21T07:48:47.592916+0000","last_clean":"2024-09-21T07:48:47.592916+0000","last_became_active":"2024-09-21T07:35:26.876462+0000","last_became_peered":"2024-09-21T07:35:26.876462+0000","last_unstale":"2024-09-21T07:48:47.592916+0000","last_undegraded":"2024-09-21T07:48:47.592916+0000","last_fullsized":"2024-09-21T07:48:47.592916+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:48.632568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012020920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":363,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.594772+0000","last_change":"2024-09-21T07:34:25.085454+0000","last_active":"2024-09-21T07:48:47.594772+0000","last_peered":"2024-09-21T07:48:47.594772+0000","last_clean":"2024-09-21T07:48:47.594772+0000","last_became_active":"2024-09-21T07:34:23.279205+0000","last_became_peered":"2024-09-21T07:34:23.279205+0000","last_unstale":"2024-09-21T07:48:47.594772+0000","last_undegraded":"2024-09-21T07:48:47.594772+0000","last_fullsized":"2024-09-21T07:48:47.594772+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:54.231666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095320599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.614070+0000","last_change":"2024-09-21T07:48:47.610088+0000","last_active":"2024-09-21T07:48:47.614070+0000","last_peered":"2024-09-21T07:48:47.614070+0000","last_clean":"2024-09-21T07:48:47.614070+0000","last_became_active":"2024-09-21T07:48:47.607791+0000","last_became_peered":"2024-09-21T07:48:47.607791+0000","last_unstale":"2024-09-21T07:48:47.614070+0000","last_undegraded":"2024-09-21T07:48:47.614070+0000","last_fullsized":"2024-09-21T07:48:47.614070+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:03.037375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"218'2","reported_seq":15,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.615890+0000","last_change":"2024-09-21T07:48:46.602844+0000","last_active":"2024-09-21T07:48:47.615890+0000","last_peered":"2024-09-21T07:48:47.615890+0000","last_clean":"2024-09-21T07:48:47.615890+0000","last_became_active":"2024-09-21T07:48:46.602544+0000","last_became_peered":"2024-09-21T07:48:46.602544+0000","last_unstale":"2024-09-21T07:48:47.615890+0000","last_undegraded":"2024-09-21T07:48:47.615890+0000","last_fullsized":"2024-09-21T07:48:47.615890+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:36:36.719598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":308,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609553+0000","last_change":"2024-09-21T07:35:27.861373+0000","last_active":"2024-09-21T07:48:48.609553+0000","last_peered":"2024-09-21T07:48:48.609553+0000","last_clean":"2024-09-21T07:48:48.609553+0000","last_became_active":"2024-09-21T07:35:27.861016+0000","last_became_peered":"2024-09-21T07:35:27.861016+0000","last_unstale":"2024-09-21T07:48:48.609553+0000","last_undegraded":"2024-09-21T07:48:48.609553+0000","last_fullsized":"2024-09-21T07:48:48.609553+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:56.043763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":313,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.599014+0000","last_change":"2024-09-21T07:35:29.053677+0000","last_active":"2024-09-21T07:48:47.599014+0000","last_peered":"2024-09-21T07:48:47.599014+0000","last_clean":"2024-09-21T07:48:47.599014+0000","last_became_active":"2024-09-21T07:35:26.875496+0000","last_became_peered":"2024-09-21T07:35:26.875496+0000","last_unstale":"2024-09-21T07:48:47.599014+0000","last_undegraded":"2024-09-21T07:48:47.599014+0000","last_fullsized":"2024-09-21T07:48:47.599014+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:04.850784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065859000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":365,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.607231+0000","last_change":"2024-09-21T07:34:25.028521+0000","last_active":"2024-09-21T07:48:47.607231+0000","last_peered":"2024-09-21T07:48:47.607231+0000","last_clean":"2024-09-21T07:48:47.607231+0000","last_became_active":"2024-09-21T07:34:23.635467+0000","last_became_peered":"2024-09-21T07:34:23.635467+0000","last_unstale":"2024-09-21T07:48:47.607231+0000","last_undegraded":"2024-09-21T07:48:47.607231+0000","last_fullsized":"2024-09-21T07:48:47.607231+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:33.729123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000660558,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051543+0000","last_change":"2024-09-21T07:48:47.616627+0000","last_active":"2024-09-21T07:48:50.051543+0000","last_peered":"2024-09-21T07:48:50.051543+0000","last_clean":"2024-09-21T07:48:50.051543+0000","last_became_active":"2024-09-21T07:48:47.616246+0000","last_became_peered":"2024-09-21T07:48:47.616246+0000","last_unstale":"2024-09-21T07:48:50.051543+0000","last_undegraded":"2024-09-21T07:48:50.051543+0000","last_fullsized":"2024-09-21T07:48:50.051543+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:07.746423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.615861+0000","last_change":"2024-09-21T07:48:46.614962+0000","last_active":"2024-09-21T07:48:46.615861+0000","last_peered":"2024-09-21T07:48:46.615861+0000","last_clean":"2024-09-21T07:48:46.615861+0000","last_became_active":"2024-09-21T07:48:46.614616+0000","last_became_peered":"2024-09-21T07:48:46.614616+0000","last_unstale":"2024-09-21T07:48:46.615861+0000","last_undegraded":"2024-09-21T07:48:46.615861+0000","last_fullsized":"2024-09-21T07:48:46.615861+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:41:33.005009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":304,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.599104+0000","last_change":"2024-09-21T07:35:27.871201+0000","last_active":"2024-09-21T07:48:47.599104+0000","last_peered":"2024-09-21T07:48:47.599104+0000","last_clean":"2024-09-21T07:48:47.599104+0000","last_became_active":"2024-09-21T07:35:27.870816+0000","last_became_peered":"2024-09-21T07:35:27.870816+0000","last_unstale":"2024-09-21T07:48:47.599104+0000","last_undegraded":"2024-09-21T07:48:47.599104+0000","last_fullsized":"2024-09-21T07:48:47.599104+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:04.787450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":315,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.610027+0000","last_change":"2024-09-21T07:35:29.491874+0000","last_active":"2024-09-21T07:48:48.610027+0000","last_peered":"2024-09-21T07:48:48.610027+0000","last_clean":"2024-09-21T07:48:48.610027+0000","last_became_active":"2024-09-21T07:35:26.866381+0000","last_became_peered":"2024-09-21T07:35:26.866381+0000","last_unstale":"2024-09-21T07:48:48.610027+0000","last_undegraded":"2024-09-21T07:48:48.610027+0000","last_fullsized":"2024-09-21T07:48:48.610027+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:36.404647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073235399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":370,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.610052+0000","last_change":"2024-09-21T07:34:24.260046+0000","last_active":"2024-09-21T07:48:48.610052+0000","last_peered":"2024-09-21T07:48:48.610052+0000","last_clean":"2024-09-21T07:48:48.610052+0000","last_became_active":"2024-09-21T07:34:23.273431+0000","last_became_peered":"2024-09-21T07:34:23.273431+0000","last_unstale":"2024-09-21T07:48:48.610052+0000","last_undegraded":"2024-09-21T07:48:48.610052+0000","last_fullsized":"2024-09-21T07:48:48.610052+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:18.876057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019803310000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051791+0000","last_change":"2024-09-21T07:48:47.605350+0000","last_active":"2024-09-21T07:48:50.051791+0000","last_peered":"2024-09-21T07:48:50.051791+0000","last_clean":"2024-09-21T07:48:50.051791+0000","last_became_active":"2024-09-21T07:48:47.605046+0000","last_became_peered":"2024-09-21T07:48:47.605046+0000","last_unstale":"2024-09-21T07:48:50.051791+0000","last_undegraded":"2024-09-21T07:48:50.051791+0000","last_fullsized":"2024-09-21T07:48:50.051791+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:07.906313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.611056+0000","last_change":"2024-09-21T07:48:46.607052+0000","last_active":"2024-09-21T07:48:46.611056+0000","last_peered":"2024-09-21T07:48:46.611056+0000","last_clean":"2024-09-21T07:48:46.611056+0000","last_became_active":"2024-09-21T07:48:46.606707+0000","last_became_peered":"2024-09-21T07:48:46.606707+0000","last_unstale":"2024-09-21T07:48:46.611056+0000","last_undegraded":"2024-09-21T07:48:46.611056+0000","last_fullsized":"2024-09-21T07:48:46.611056+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:29:39.188621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":304,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.603501+0000","last_change":"2024-09-21T07:35:27.871736+0000","last_active":"2024-09-21T07:48:47.603501+0000","last_peered":"2024-09-21T07:48:47.603501+0000","last_clean":"2024-09-21T07:48:47.603501+0000","last_became_active":"2024-09-21T07:35:27.871232+0000","last_became_peered":"2024-09-21T07:35:27.871232+0000","last_unstale":"2024-09-21T07:48:47.603501+0000","last_undegraded":"2024-09-21T07:48:47.603501+0000","last_fullsized":"2024-09-21T07:48:47.603501+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:04.419301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":316,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608800+0000","last_change":"2024-09-21T07:35:29.491246+0000","last_active":"2024-09-21T07:48:48.608800+0000","last_peered":"2024-09-21T07:48:48.608800+0000","last_clean":"2024-09-21T07:48:48.608800+0000","last_became_active":"2024-09-21T07:35:26.866783+0000","last_became_peered":"2024-09-21T07:35:26.866783+0000","last_unstale":"2024-09-21T07:48:48.608800+0000","last_undegraded":"2024-09-21T07:48:48.608800+0000","last_fullsized":"2024-09-21T07:48:48.608800+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:28.881914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098474099999999992,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":365,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.599139+0000","last_change":"2024-09-21T07:34:25.028640+0000","last_active":"2024-09-21T07:48:47.599139+0000","last_peered":"2024-09-21T07:48:47.599139+0000","last_clean":"2024-09-21T07:48:47.599139+0000","last_became_active":"2024-09-21T07:34:23.260950+0000","last_became_peered":"2024-09-21T07:34:23.260950+0000","last_unstale":"2024-09-21T07:48:47.599139+0000","last_undegraded":"2024-09-21T07:48:47.599139+0000","last_fullsized":"2024-09-21T07:48:47.599139+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:23:42.121398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055135099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.623528+0000","last_change":"2024-09-21T07:48:47.614858+0000","last_active":"2024-09-21T07:48:47.623528+0000","last_peered":"2024-09-21T07:48:47.623528+0000","last_clean":"2024-09-21T07:48:47.623528+0000","last_became_active":"2024-09-21T07:48:47.614484+0000","last_became_peered":"2024-09-21T07:48:47.614484+0000","last_unstale":"2024-09-21T07:48:47.623528+0000","last_undegraded":"2024-09-21T07:48:47.623528+0000","last_fullsized":"2024-09-21T07:48:47.623528+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:24.141015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":13,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.603705+0000","last_change":"2024-09-21T07:48:46.607151+0000","last_active":"2024-09-21T07:48:47.603705+0000","last_peered":"2024-09-21T07:48:47.603705+0000","last_clean":"2024-09-21T07:48:47.603705+0000","last_became_active":"2024-09-21T07:48:46.606839+0000","last_became_peered":"2024-09-21T07:48:46.606839+0000","last_unstale":"2024-09-21T07:48:47.603705+0000","last_undegraded":"2024-09-21T07:48:47.603705+0000","last_fullsized":"2024-09-21T07:48:47.603705+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:34.780363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":305,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.609983+0000","last_change":"2024-09-21T07:35:27.872027+0000","last_active":"2024-09-21T07:48:47.609983+0000","last_peered":"2024-09-21T07:48:47.609983+0000","last_clean":"2024-09-21T07:48:47.609983+0000","last_became_active":"2024-09-21T07:35:27.870426+0000","last_became_peered":"2024-09-21T07:35:27.870426+0000","last_unstale":"2024-09-21T07:48:47.609983+0000","last_undegraded":"2024-09-21T07:48:47.609983+0000","last_fullsized":"2024-09-21T07:48:47.609983+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":316,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608519+0000","last_change":"2024-09-21T07:35:29.491008+0000","last_active":"2024-09-21T07:48:48.608519+0000","last_peered":"2024-09-21T07:48:48.608519+0000","last_clean":"2024-09-21T07:48:48.608519+0000","last_became_active":"2024-09-21T07:35:26.866750+0000","last_became_peered":"2024-09-21T07:35:26.866750+0000","last_unstale":"2024-09-21T07:48:48.608519+0000","last_undegraded":"2024-09-21T07:48:48.608519+0000","last_fullsized":"2024-09-21T07:48:48.608519+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:25.490406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083529700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":362,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.602630+0000","last_change":"2024-09-21T07:34:24.260665+0000","last_active":"2024-09-21T07:48:46.602630+0000","last_peered":"2024-09-21T07:48:46.602630+0000","last_clean":"2024-09-21T07:48:46.602630+0000","last_became_active":"2024-09-21T07:34:23.635909+0000","last_became_peered":"2024-09-21T07:34:23.635909+0000","last_unstale":"2024-09-21T07:48:46.602630+0000","last_undegraded":"2024-09-21T07:48:46.602630+0000","last_fullsized":"2024-09-21T07:48:46.602630+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:44.536885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051072+0000","last_change":"2024-09-21T07:48:47.615147+0000","last_active":"2024-09-21T07:48:50.051072+0000","last_peered":"2024-09-21T07:48:50.051072+0000","last_clean":"2024-09-21T07:48:50.051072+0000","last_became_active":"2024-09-21T07:48:47.614792+0000","last_became_peered":"2024-09-21T07:48:47.614792+0000","last_unstale":"2024-09-21T07:48:50.051072+0000","last_undegraded":"2024-09-21T07:48:50.051072+0000","last_fullsized":"2024-09-21T07:48:50.051072+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:23.043769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":13,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.593919+0000","last_change":"2024-09-21T07:48:46.614465+0000","last_active":"2024-09-21T07:48:47.593919+0000","last_peered":"2024-09-21T07:48:47.593919+0000","last_clean":"2024-09-21T07:48:47.593919+0000","last_became_active":"2024-09-21T07:48:46.614155+0000","last_became_peered":"2024-09-21T07:48:46.614155+0000","last_unstale":"2024-09-21T07:48:47.593919+0000","last_undegraded":"2024-09-21T07:48:47.593919+0000","last_fullsized":"2024-09-21T07:48:47.593919+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:23:53.595692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":303,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051021+0000","last_change":"2024-09-21T07:35:27.874772+0000","last_active":"2024-09-21T07:48:50.051021+0000","last_peered":"2024-09-21T07:48:50.051021+0000","last_clean":"2024-09-21T07:48:50.051021+0000","last_became_active":"2024-09-21T07:35:27.874434+0000","last_became_peered":"2024-09-21T07:35:27.874434+0000","last_unstale":"2024-09-21T07:48:50.051021+0000","last_undegraded":"2024-09-21T07:48:50.051021+0000","last_fullsized":"2024-09-21T07:48:50.051021+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:31.998755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":323,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.590456+0000","last_change":"2024-09-21T07:35:28.859153+0000","last_active":"2024-09-21T07:48:46.590456+0000","last_peered":"2024-09-21T07:48:46.590456+0000","last_clean":"2024-09-21T07:48:46.590456+0000","last_became_active":"2024-09-21T07:35:26.874428+0000","last_became_peered":"2024-09-21T07:35:26.874428+0000","last_unstale":"2024-09-21T07:48:46.590456+0000","last_undegraded":"2024-09-21T07:48:46.590456+0000","last_fullsized":"2024-09-21T07:48:46.590456+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:04.487621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048557299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":375,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.591699+0000","last_change":"2024-09-21T07:34:24.538845+0000","last_active":"2024-09-21T07:48:46.591699+0000","last_peered":"2024-09-21T07:48:46.591699+0000","last_clean":"2024-09-21T07:48:46.591699+0000","last_became_active":"2024-09-21T07:34:23.636372+0000","last_became_peered":"2024-09-21T07:34:23.636372+0000","last_unstale":"2024-09-21T07:48:46.591699+0000","last_undegraded":"2024-09-21T07:48:46.591699+0000","last_fullsized":"2024-09-21T07:48:46.591699+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:29.393813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841451,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.624317+0000","last_change":"2024-09-21T07:48:47.617117+0000","last_active":"2024-09-21T07:48:47.624317+0000","last_peered":"2024-09-21T07:48:47.624317+0000","last_clean":"2024-09-21T07:48:47.624317+0000","last_became_active":"2024-09-21T07:48:47.616813+0000","last_became_peered":"2024-09-21T07:48:47.616813+0000","last_unstale":"2024-09-21T07:48:47.624317+0000","last_undegraded":"2024-09-21T07:48:47.624317+0000","last_fullsized":"2024-09-21T07:48:47.624317+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:50.252324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.612150+0000","last_change":"2024-09-21T07:48:46.610433+0000","last_active":"2024-09-21T07:48:46.612150+0000","last_peered":"2024-09-21T07:48:46.612150+0000","last_clean":"2024-09-21T07:48:46.612150+0000","last_became_active":"2024-09-21T07:48:46.610123+0000","last_became_peered":"2024-09-21T07:48:46.610123+0000","last_unstale":"2024-09-21T07:48:46.612150+0000","last_undegraded":"2024-09-21T07:48:46.612150+0000","last_fullsized":"2024-09-21T07:48:46.612150+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:59.556565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":306,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.607367+0000","last_change":"2024-09-21T07:35:27.877801+0000","last_active":"2024-09-21T07:48:47.607367+0000","last_peered":"2024-09-21T07:48:47.607367+0000","last_clean":"2024-09-21T07:48:47.607367+0000","last_became_active":"2024-09-21T07:35:27.877464+0000","last_became_peered":"2024-09-21T07:35:27.877464+0000","last_unstale":"2024-09-21T07:48:47.607367+0000","last_undegraded":"2024-09-21T07:48:47.607367+0000","last_fullsized":"2024-09-21T07:48:47.607367+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:08.814929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":312,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.603961+0000","last_change":"2024-09-21T07:35:28.863732+0000","last_active":"2024-09-21T07:48:47.603961+0000","last_peered":"2024-09-21T07:48:47.603961+0000","last_clean":"2024-09-21T07:48:47.603961+0000","last_became_active":"2024-09-21T07:35:26.877392+0000","last_became_peered":"2024-09-21T07:35:26.877392+0000","last_unstale":"2024-09-21T07:48:47.603961+0000","last_undegraded":"2024-09-21T07:48:47.603961+0000","last_fullsized":"2024-09-21T07:48:47.603961+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071425700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":370,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.610101+0000","last_change":"2024-09-21T07:34:24.259686+0000","last_active":"2024-09-21T07:48:48.610101+0000","last_peered":"2024-09-21T07:48:48.610101+0000","last_clean":"2024-09-21T07:48:48.610101+0000","last_became_active":"2024-09-21T07:34:23.274405+0000","last_became_peered":"2024-09-21T07:34:23.274405+0000","last_unstale":"2024-09-21T07:48:48.610101+0000","last_undegraded":"2024-09-21T07:48:48.610101+0000","last_fullsized":"2024-09-21T07:48:48.610101+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:26.304693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022009920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.623488+0000","last_change":"2024-09-21T07:48:47.616194+0000","last_active":"2024-09-21T07:48:47.623488+0000","last_peered":"2024-09-21T07:48:47.623488+0000","last_clean":"2024-09-21T07:48:47.623488+0000","last_became_active":"2024-09-21T07:48:47.615876+0000","last_became_peered":"2024-09-21T07:48:47.615876+0000","last_unstale":"2024-09-21T07:48:47.623488+0000","last_undegraded":"2024-09-21T07:48:47.623488+0000","last_fullsized":"2024-09-21T07:48:47.623488+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:07.420806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"218'3","reported_seq":16,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.620054+0000","last_change":"2024-09-21T07:48:46.605473+0000","last_active":"2024-09-21T07:48:47.620054+0000","last_peered":"2024-09-21T07:48:47.620054+0000","last_clean":"2024-09-21T07:48:47.620054+0000","last_became_active":"2024-09-21T07:48:46.604548+0000","last_became_peered":"2024-09-21T07:48:46.604548+0000","last_unstale":"2024-09-21T07:48:47.620054+0000","last_undegraded":"2024-09-21T07:48:47.620054+0000","last_fullsized":"2024-09-21T07:48:47.620054+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:05.902786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":303,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051307+0000","last_change":"2024-09-21T07:35:27.867336+0000","last_active":"2024-09-21T07:48:50.051307+0000","last_peered":"2024-09-21T07:48:50.051307+0000","last_clean":"2024-09-21T07:48:50.051307+0000","last_became_active":"2024-09-21T07:35:27.867015+0000","last_became_peered":"2024-09-21T07:35:27.867015+0000","last_unstale":"2024-09-21T07:48:50.051307+0000","last_undegraded":"2024-09-21T07:48:50.051307+0000","last_fullsized":"2024-09-21T07:48:50.051307+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:59.214139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":309,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.602390+0000","last_change":"2024-09-21T07:35:29.493137+0000","last_active":"2024-09-21T07:48:46.602390+0000","last_peered":"2024-09-21T07:48:46.602390+0000","last_clean":"2024-09-21T07:48:46.602390+0000","last_became_active":"2024-09-21T07:35:26.872230+0000","last_became_peered":"2024-09-21T07:35:26.872230+0000","last_unstale":"2024-09-21T07:48:46.602390+0000","last_undegraded":"2024-09-21T07:48:46.602390+0000","last_fullsized":"2024-09-21T07:48:46.602390+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:24.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001197449,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":362,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.596253+0000","last_change":"2024-09-21T07:34:24.258754+0000","last_active":"2024-09-21T07:48:46.596253+0000","last_peered":"2024-09-21T07:48:46.596253+0000","last_clean":"2024-09-21T07:48:46.596253+0000","last_became_active":"2024-09-21T07:34:23.272657+0000","last_became_peered":"2024-09-21T07:34:23.272657+0000","last_unstale":"2024-09-21T07:48:46.596253+0000","last_undegraded":"2024-09-21T07:48:46.596253+0000","last_fullsized":"2024-09-21T07:48:46.596253+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:05.477099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067143200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.630257+0000","last_change":"2024-09-21T07:48:47.629237+0000","last_active":"2024-09-21T07:48:47.630257+0000","last_peered":"2024-09-21T07:48:47.630257+0000","last_clean":"2024-09-21T07:48:47.630257+0000","last_became_active":"2024-09-21T07:48:47.628900+0000","last_became_peered":"2024-09-21T07:48:47.628900+0000","last_unstale":"2024-09-21T07:48:47.630257+0000","last_undegraded":"2024-09-21T07:48:47.630257+0000","last_fullsized":"2024-09-21T07:48:47.630257+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:19.087720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.615052+0000","last_change":"2024-09-21T07:48:46.612660+0000","last_active":"2024-09-21T07:48:46.615052+0000","last_peered":"2024-09-21T07:48:46.615052+0000","last_clean":"2024-09-21T07:48:46.615052+0000","last_became_active":"2024-09-21T07:48:46.612362+0000","last_became_peered":"2024-09-21T07:48:46.612362+0000","last_unstale":"2024-09-21T07:48:46.615052+0000","last_undegraded":"2024-09-21T07:48:46.615052+0000","last_fullsized":"2024-09-21T07:48:46.615052+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:10.600575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":305,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.597247+0000","last_change":"2024-09-21T07:35:27.876021+0000","last_active":"2024-09-21T07:48:47.597247+0000","last_peered":"2024-09-21T07:48:47.597247+0000","last_clean":"2024-09-21T07:48:47.597247+0000","last_became_active":"2024-09-21T07:35:27.875724+0000","last_became_peered":"2024-09-21T07:35:27.875724+0000","last_unstale":"2024-09-21T07:48:47.597247+0000","last_undegraded":"2024-09-21T07:48:47.597247+0000","last_fullsized":"2024-09-21T07:48:47.597247+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:03.022812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":309,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.600781+0000","last_change":"2024-09-21T07:35:29.492570+0000","last_active":"2024-09-21T07:48:46.600781+0000","last_peered":"2024-09-21T07:48:46.600781+0000","last_clean":"2024-09-21T07:48:46.600781+0000","last_became_active":"2024-09-21T07:35:26.874316+0000","last_became_peered":"2024-09-21T07:35:26.874316+0000","last_unstale":"2024-09-21T07:48:46.600781+0000","last_undegraded":"2024-09-21T07:48:46.600781+0000","last_fullsized":"2024-09-21T07:48:46.600781+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:16.512323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011990760000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":363,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.596170+0000","last_change":"2024-09-21T07:34:25.085232+0000","last_active":"2024-09-21T07:48:47.596170+0000","last_peered":"2024-09-21T07:48:47.596170+0000","last_clean":"2024-09-21T07:48:47.596170+0000","last_became_active":"2024-09-21T07:34:23.277807+0000","last_became_peered":"2024-09-21T07:34:23.277807+0000","last_unstale":"2024-09-21T07:48:47.596170+0000","last_undegraded":"2024-09-21T07:48:47.596170+0000","last_fullsized":"2024-09-21T07:48:47.596170+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:36.362919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086619099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609167+0000","last_change":"2024-09-21T07:48:47.607795+0000","last_active":"2024-09-21T07:48:48.609167+0000","last_peered":"2024-09-21T07:48:48.609167+0000","last_clean":"2024-09-21T07:48:48.609167+0000","last_became_active":"2024-09-21T07:48:47.607511+0000","last_became_peered":"2024-09-21T07:48:47.607511+0000","last_unstale":"2024-09-21T07:48:48.609167+0000","last_undegraded":"2024-09-21T07:48:48.609167+0000","last_fullsized":"2024-09-21T07:48:48.609167+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:27.353767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"218'2","reported_seq":17,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609872+0000","last_change":"2024-09-21T07:48:46.609344+0000","last_active":"2024-09-21T07:48:48.609872+0000","last_peered":"2024-09-21T07:48:48.609872+0000","last_clean":"2024-09-21T07:48:48.609872+0000","last_became_active":"2024-09-21T07:48:46.606541+0000","last_became_peered":"2024-09-21T07:48:46.606541+0000","last_unstale":"2024-09-21T07:48:48.609872+0000","last_undegraded":"2024-09-21T07:48:48.609872+0000","last_fullsized":"2024-09-21T07:48:48.609872+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:36.274172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":306,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.606343+0000","last_change":"2024-09-21T07:35:27.872798+0000","last_active":"2024-09-21T07:48:47.606343+0000","last_peered":"2024-09-21T07:48:47.606343+0000","last_clean":"2024-09-21T07:48:47.606343+0000","last_became_active":"2024-09-21T07:35:27.872483+0000","last_became_peered":"2024-09-21T07:35:27.872483+0000","last_unstale":"2024-09-21T07:48:47.606343+0000","last_undegraded":"2024-09-21T07:48:47.606343+0000","last_fullsized":"2024-09-21T07:48:47.606343+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:11.933469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":313,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.605746+0000","last_change":"2024-09-21T07:35:29.206723+0000","last_active":"2024-09-21T07:48:47.605746+0000","last_peered":"2024-09-21T07:48:47.605746+0000","last_clean":"2024-09-21T07:48:47.605746+0000","last_became_active":"2024-09-21T07:35:26.872377+0000","last_became_peered":"2024-09-21T07:35:26.872377+0000","last_unstale":"2024-09-21T07:48:47.605746+0000","last_undegraded":"2024-09-21T07:48:47.605746+0000","last_fullsized":"2024-09-21T07:48:47.605746+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:29.086862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083077200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":359,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051990+0000","last_change":"2024-09-21T07:34:24.597672+0000","last_active":"2024-09-21T07:48:50.051990+0000","last_peered":"2024-09-21T07:48:50.051990+0000","last_clean":"2024-09-21T07:48:50.051990+0000","last_became_active":"2024-09-21T07:34:23.267770+0000","last_became_peered":"2024-09-21T07:34:23.267770+0000","last_unstale":"2024-09-21T07:48:50.051990+0000","last_undegraded":"2024-09-21T07:48:50.051990+0000","last_fullsized":"2024-09-21T07:48:50.051990+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:51.629476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086378500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.624444+0000","last_change":"2024-09-21T07:48:47.618502+0000","last_active":"2024-09-21T07:48:47.624444+0000","last_peered":"2024-09-21T07:48:47.624444+0000","last_clean":"2024-09-21T07:48:47.624444+0000","last_became_active":"2024-09-21T07:48:47.618213+0000","last_became_peered":"2024-09-21T07:48:47.618213+0000","last_unstale":"2024-09-21T07:48:47.624444+0000","last_undegraded":"2024-09-21T07:48:47.624444+0000","last_fullsized":"2024-09-21T07:48:47.624444+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:19.164379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":13,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.599459+0000","last_change":"2024-09-21T07:48:46.602566+0000","last_active":"2024-09-21T07:48:47.599459+0000","last_peered":"2024-09-21T07:48:47.599459+0000","last_clean":"2024-09-21T07:48:47.599459+0000","last_became_active":"2024-09-21T07:48:46.602254+0000","last_became_peered":"2024-09-21T07:48:46.602254+0000","last_unstale":"2024-09-21T07:48:47.599459+0000","last_undegraded":"2024-09-21T07:48:47.599459+0000","last_fullsized":"2024-09-21T07:48:47.599459+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:09.449030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":306,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.606118+0000","last_change":"2024-09-21T07:35:27.870121+0000","last_active":"2024-09-21T07:48:47.606118+0000","last_peered":"2024-09-21T07:48:47.606118+0000","last_clean":"2024-09-21T07:48:47.606118+0000","last_became_active":"2024-09-21T07:35:27.869811+0000","last_became_peered":"2024-09-21T07:35:27.869811+0000","last_unstale":"2024-09-21T07:48:47.606118+0000","last_undegraded":"2024-09-21T07:48:47.606118+0000","last_fullsized":"2024-09-21T07:48:47.606118+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:31.778461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":315,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.595449+0000","last_change":"2024-09-21T07:35:28.860699+0000","last_active":"2024-09-21T07:48:47.595449+0000","last_peered":"2024-09-21T07:48:47.595449+0000","last_clean":"2024-09-21T07:48:47.595449+0000","last_became_active":"2024-09-21T07:35:26.876779+0000","last_became_peered":"2024-09-21T07:35:26.876779+0000","last_unstale":"2024-09-21T07:48:47.595449+0000","last_undegraded":"2024-09-21T07:48:47.595449+0000","last_fullsized":"2024-09-21T07:48:47.595449+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:28.286403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000913188,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":367,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.594177+0000","last_change":"2024-09-21T07:34:24.260181+0000","last_active":"2024-09-21T07:48:47.594177+0000","last_peered":"2024-09-21T07:48:47.594177+0000","last_clean":"2024-09-21T07:48:47.594177+0000","last_became_active":"2024-09-21T07:34:23.275580+0000","last_became_peered":"2024-09-21T07:34:23.275580+0000","last_unstale":"2024-09-21T07:48:47.594177+0000","last_undegraded":"2024-09-21T07:48:47.594177+0000","last_fullsized":"2024-09-21T07:48:47.594177+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:49.857622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014706980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.629579+0000","last_change":"2024-09-21T07:48:47.627712+0000","last_active":"2024-09-21T07:48:47.629579+0000","last_peered":"2024-09-21T07:48:47.629579+0000","last_clean":"2024-09-21T07:48:47.629579+0000","last_became_active":"2024-09-21T07:48:47.623777+0000","last_became_peered":"2024-09-21T07:48:47.623777+0000","last_unstale":"2024-09-21T07:48:47.629579+0000","last_undegraded":"2024-09-21T07:48:47.629579+0000","last_fullsized":"2024-09-21T07:48:47.629579+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:42:35.733593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":13,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.596000+0000","last_change":"2024-09-21T07:48:46.602604+0000","last_active":"2024-09-21T07:48:47.596000+0000","last_peered":"2024-09-21T07:48:47.596000+0000","last_clean":"2024-09-21T07:48:47.596000+0000","last_became_active":"2024-09-21T07:48:46.602252+0000","last_became_peered":"2024-09-21T07:48:46.602252+0000","last_unstale":"2024-09-21T07:48:47.596000+0000","last_undegraded":"2024-09-21T07:48:47.596000+0000","last_fullsized":"2024-09-21T07:48:47.596000+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:02:41.387196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":307,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.600169+0000","last_change":"2024-09-21T07:35:27.875393+0000","last_active":"2024-09-21T07:48:47.600169+0000","last_peered":"2024-09-21T07:48:47.600169+0000","last_clean":"2024-09-21T07:48:47.600169+0000","last_became_active":"2024-09-21T07:35:27.874671+0000","last_became_peered":"2024-09-21T07:35:27.874671+0000","last_unstale":"2024-09-21T07:48:47.600169+0000","last_undegraded":"2024-09-21T07:48:47.600169+0000","last_fullsized":"2024-09-21T07:48:47.600169+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":313,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.608651+0000","last_change":"2024-09-21T07:35:29.206296+0000","last_active":"2024-09-21T07:48:47.608651+0000","last_peered":"2024-09-21T07:48:47.608651+0000","last_clean":"2024-09-21T07:48:47.608651+0000","last_became_active":"2024-09-21T07:35:26.874575+0000","last_became_peered":"2024-09-21T07:35:26.874575+0000","last_unstale":"2024-09-21T07:48:47.608651+0000","last_undegraded":"2024-09-21T07:48:47.608651+0000","last_fullsized":"2024-09-21T07:48:47.608651+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:22.771154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061011599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":370,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609427+0000","last_change":"2024-09-21T07:34:24.259112+0000","last_active":"2024-09-21T07:48:48.609427+0000","last_peered":"2024-09-21T07:48:48.609427+0000","last_clean":"2024-09-21T07:48:48.609427+0000","last_became_active":"2024-09-21T07:34:23.635444+0000","last_became_peered":"2024-09-21T07:34:23.635444+0000","last_unstale":"2024-09-21T07:48:48.609427+0000","last_undegraded":"2024-09-21T07:48:48.609427+0000","last_fullsized":"2024-09-21T07:48:48.609427+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:46.255385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001952636,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.596453+0000","last_change":"2024-09-21T07:48:46.596453+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.596453+0000","last_undegraded":"2024-09-21T07:48:46.596453+0000","last_fullsized":"2024-09-21T07:48:46.596453+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.10","version":"218'8","reported_seq":21,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.633173+0000","last_change":"2024-09-21T07:48:46.607918+0000","last_active":"2024-09-21T07:48:47.633173+0000","last_peered":"2024-09-21T07:48:47.633173+0000","last_clean":"2024-09-21T07:48:47.633173+0000","last_became_active":"2024-09-21T07:48:46.607600+0000","last_became_peered":"2024-09-21T07:48:46.607600+0000","last_unstale":"2024-09-21T07:48:47.633173+0000","last_undegraded":"2024-09-21T07:48:47.633173+0000","last_fullsized":"2024-09-21T07:48:47.633173+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:55:20.508968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":307,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.605147+0000","last_change":"2024-09-21T07:35:27.878356+0000","last_active":"2024-09-21T07:48:47.605147+0000","last_peered":"2024-09-21T07:48:47.605147+0000","last_clean":"2024-09-21T07:48:47.605147+0000","last_became_active":"2024-09-21T07:35:27.877982+0000","last_became_peered":"2024-09-21T07:35:27.877982+0000","last_unstale":"2024-09-21T07:48:47.605147+0000","last_undegraded":"2024-09-21T07:48:47.605147+0000","last_fullsized":"2024-09-21T07:48:47.605147+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":314,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.603882+0000","last_change":"2024-09-21T07:35:29.053131+0000","last_active":"2024-09-21T07:48:47.603882+0000","last_peered":"2024-09-21T07:48:47.603882+0000","last_clean":"2024-09-21T07:48:47.603882+0000","last_became_active":"2024-09-21T07:35:26.870516+0000","last_became_peered":"2024-09-21T07:35:26.870516+0000","last_unstale":"2024-09-21T07:48:47.603882+0000","last_undegraded":"2024-09-21T07:48:47.603882+0000","last_fullsized":"2024-09-21T07:48:47.603882+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:10.924491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070705699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":375,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.591878+0000","last_change":"2024-09-21T07:34:24.539152+0000","last_active":"2024-09-21T07:48:46.591878+0000","last_peered":"2024-09-21T07:48:46.591878+0000","last_clean":"2024-09-21T07:48:46.591878+0000","last_became_active":"2024-09-21T07:34:23.259573+0000","last_became_peered":"2024-09-21T07:34:23.259573+0000","last_unstale":"2024-09-21T07:48:46.591878+0000","last_undegraded":"2024-09-21T07:48:46.591878+0000","last_fullsized":"2024-09-21T07:48:46.591878+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:26.986900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058108400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.632435+0000","last_change":"2024-09-21T07:48:47.627105+0000","last_active":"2024-09-21T07:48:47.632435+0000","last_peered":"2024-09-21T07:48:47.632435+0000","last_clean":"2024-09-21T07:48:47.632435+0000","last_became_active":"2024-09-21T07:48:47.626559+0000","last_became_peered":"2024-09-21T07:48:47.626559+0000","last_unstale":"2024-09-21T07:48:47.632435+0000","last_undegraded":"2024-09-21T07:48:47.632435+0000","last_fullsized":"2024-09-21T07:48:47.632435+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:45.810340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.611508+0000","last_change":"2024-09-21T07:48:46.607214+0000","last_active":"2024-09-21T07:48:46.611508+0000","last_peered":"2024-09-21T07:48:46.611508+0000","last_clean":"2024-09-21T07:48:46.611508+0000","last_became_active":"2024-09-21T07:48:46.606843+0000","last_became_peered":"2024-09-21T07:48:46.606843+0000","last_unstale":"2024-09-21T07:48:46.611508+0000","last_undegraded":"2024-09-21T07:48:46.611508+0000","last_fullsized":"2024-09-21T07:48:46.611508+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:37.232512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":306,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.603776+0000","last_change":"2024-09-21T07:35:27.873726+0000","last_active":"2024-09-21T07:48:47.603776+0000","last_peered":"2024-09-21T07:48:47.603776+0000","last_clean":"2024-09-21T07:48:47.603776+0000","last_became_active":"2024-09-21T07:35:27.871421+0000","last_became_peered":"2024-09-21T07:35:27.871421+0000","last_unstale":"2024-09-21T07:48:47.603776+0000","last_undegraded":"2024-09-21T07:48:47.603776+0000","last_fullsized":"2024-09-21T07:48:47.603776+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:00.168915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":315,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.593992+0000","last_change":"2024-09-21T07:35:28.861032+0000","last_active":"2024-09-21T07:48:47.593992+0000","last_peered":"2024-09-21T07:48:47.593992+0000","last_clean":"2024-09-21T07:48:47.593992+0000","last_became_active":"2024-09-21T07:35:26.873149+0000","last_became_peered":"2024-09-21T07:35:26.873149+0000","last_unstale":"2024-09-21T07:48:47.593992+0000","last_undegraded":"2024-09-21T07:48:47.593992+0000","last_fullsized":"2024-09-21T07:48:47.593992+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:59.435981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011988369999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":362,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.597392+0000","last_change":"2024-09-21T07:34:24.259902+0000","last_active":"2024-09-21T07:48:46.597392+0000","last_peered":"2024-09-21T07:48:46.597392+0000","last_clean":"2024-09-21T07:48:46.597392+0000","last_became_active":"2024-09-21T07:34:23.271902+0000","last_became_peered":"2024-09-21T07:34:23.271902+0000","last_unstale":"2024-09-21T07:48:46.597392+0000","last_undegraded":"2024-09-21T07:48:46.597392+0000","last_fullsized":"2024-09-21T07:48:46.597392+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:59.439525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001686358,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.589360+0000","last_change":"2024-09-21T07:48:46.589360+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.589360+0000","last_undegraded":"2024-09-21T07:48:46.589360+0000","last_fullsized":"2024-09-21T07:48:46.589360+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.608012+0000","last_change":"2024-09-21T07:48:46.606382+0000","last_active":"2024-09-21T07:48:46.608012+0000","last_peered":"2024-09-21T07:48:46.608012+0000","last_clean":"2024-09-21T07:48:46.608012+0000","last_became_active":"2024-09-21T07:48:46.606002+0000","last_became_peered":"2024-09-21T07:48:46.606002+0000","last_unstale":"2024-09-21T07:48:46.608012+0000","last_undegraded":"2024-09-21T07:48:46.608012+0000","last_fullsized":"2024-09-21T07:48:46.608012+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:03:59.840933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":304,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.595132+0000","last_change":"2024-09-21T07:35:27.872637+0000","last_active":"2024-09-21T07:48:47.595132+0000","last_peered":"2024-09-21T07:48:47.595132+0000","last_clean":"2024-09-21T07:48:47.595132+0000","last_became_active":"2024-09-21T07:35:27.872149+0000","last_became_peered":"2024-09-21T07:35:27.872149+0000","last_unstale":"2024-09-21T07:48:47.595132+0000","last_undegraded":"2024-09-21T07:48:47.595132+0000","last_fullsized":"2024-09-21T07:48:47.595132+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:27.697872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":309,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.602775+0000","last_change":"2024-09-21T07:35:29.493362+0000","last_active":"2024-09-21T07:48:46.602775+0000","last_peered":"2024-09-21T07:48:46.602775+0000","last_clean":"2024-09-21T07:48:46.602775+0000","last_became_active":"2024-09-21T07:35:26.873018+0000","last_became_peered":"2024-09-21T07:35:26.873018+0000","last_unstale":"2024-09-21T07:48:46.602775+0000","last_undegraded":"2024-09-21T07:48:46.602775+0000","last_fullsized":"2024-09-21T07:48:46.602775+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:02.460557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012496510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":370,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609996+0000","last_change":"2024-09-21T07:34:24.259662+0000","last_active":"2024-09-21T07:48:48.609996+0000","last_peered":"2024-09-21T07:48:48.609996+0000","last_clean":"2024-09-21T07:48:48.609996+0000","last_became_active":"2024-09-21T07:34:23.278812+0000","last_became_peered":"2024-09-21T07:34:23.278812+0000","last_unstale":"2024-09-21T07:48:48.609996+0000","last_undegraded":"2024-09-21T07:48:48.609996+0000","last_fullsized":"2024-09-21T07:48:48.609996+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:49.829974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356463,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.591252+0000","last_change":"2024-09-21T07:48:46.591252+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.591252+0000","last_undegraded":"2024-09-21T07:48:46.591252+0000","last_fullsized":"2024-09-21T07:48:46.591252+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"218'1","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609860+0000","last_change":"2024-09-21T07:48:46.605998+0000","last_active":"2024-09-21T07:48:48.609860+0000","last_peered":"2024-09-21T07:48:48.609860+0000","last_clean":"2024-09-21T07:48:48.609860+0000","last_became_active":"2024-09-21T07:48:46.605714+0000","last_became_peered":"2024-09-21T07:48:46.605714+0000","last_unstale":"2024-09-21T07:48:48.609860+0000","last_undegraded":"2024-09-21T07:48:48.609860+0000","last_fullsized":"2024-09-21T07:48:48.609860+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:51.672371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":302,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051131+0000","last_change":"2024-09-21T07:35:27.868448+0000","last_active":"2024-09-21T07:48:50.051131+0000","last_peered":"2024-09-21T07:48:50.051131+0000","last_clean":"2024-09-21T07:48:50.051131+0000","last_became_active":"2024-09-21T07:35:27.868105+0000","last_became_peered":"2024-09-21T07:35:27.868105+0000","last_unstale":"2024-09-21T07:48:50.051131+0000","last_undegraded":"2024-09-21T07:48:50.051131+0000","last_fullsized":"2024-09-21T07:48:50.051131+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:41.468702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":308,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.602167+0000","last_change":"2024-09-21T07:35:29.492974+0000","last_active":"2024-09-21T07:48:46.602167+0000","last_peered":"2024-09-21T07:48:46.602167+0000","last_clean":"2024-09-21T07:48:46.602167+0000","last_became_active":"2024-09-21T07:35:26.872165+0000","last_became_peered":"2024-09-21T07:35:26.872165+0000","last_unstale":"2024-09-21T07:48:46.602167+0000","last_undegraded":"2024-09-21T07:48:46.602167+0000","last_fullsized":"2024-09-21T07:48:46.602167+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:59.912520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013795179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":363,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.594739+0000","last_change":"2024-09-21T07:34:25.084861+0000","last_active":"2024-09-21T07:48:47.594739+0000","last_peered":"2024-09-21T07:48:47.594739+0000","last_clean":"2024-09-21T07:48:47.594739+0000","last_became_active":"2024-09-21T07:34:23.277708+0000","last_became_peered":"2024-09-21T07:34:23.277708+0000","last_unstale":"2024-09-21T07:48:47.594739+0000","last_undegraded":"2024-09-21T07:48:47.594739+0000","last_fullsized":"2024-09-21T07:48:47.594739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:40.816488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000597566,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.632205+0000","last_change":"2024-09-21T07:48:47.621555+0000","last_active":"2024-09-21T07:48:47.632205+0000","last_peered":"2024-09-21T07:48:47.632205+0000","last_clean":"2024-09-21T07:48:47.632205+0000","last_became_active":"2024-09-21T07:48:47.620805+0000","last_became_peered":"2024-09-21T07:48:47.620805+0000","last_unstale":"2024-09-21T07:48:47.632205+0000","last_undegraded":"2024-09-21T07:48:47.632205+0000","last_fullsized":"2024-09-21T07:48:47.632205+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:20.833555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":13,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.592490+0000","last_change":"2024-09-21T07:48:46.611660+0000","last_active":"2024-09-21T07:48:47.592490+0000","last_peered":"2024-09-21T07:48:47.592490+0000","last_clean":"2024-09-21T07:48:47.592490+0000","last_became_active":"2024-09-21T07:48:46.610843+0000","last_became_peered":"2024-09-21T07:48:46.610843+0000","last_unstale":"2024-09-21T07:48:47.592490+0000","last_undegraded":"2024-09-21T07:48:47.592490+0000","last_fullsized":"2024-09-21T07:48:47.592490+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:43:38.733833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":302,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051512+0000","last_change":"2024-09-21T07:35:27.874640+0000","last_active":"2024-09-21T07:48:50.051512+0000","last_peered":"2024-09-21T07:48:50.051512+0000","last_clean":"2024-09-21T07:48:50.051512+0000","last_became_active":"2024-09-21T07:35:27.874265+0000","last_became_peered":"2024-09-21T07:35:27.874265+0000","last_unstale":"2024-09-21T07:48:50.051512+0000","last_undegraded":"2024-09-21T07:48:50.051512+0000","last_fullsized":"2024-09-21T07:48:50.051512+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:45.754654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":312,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.606600+0000","last_change":"2024-09-21T07:35:29.207392+0000","last_active":"2024-09-21T07:48:47.606600+0000","last_peered":"2024-09-21T07:48:47.606600+0000","last_clean":"2024-09-21T07:48:47.606600+0000","last_became_active":"2024-09-21T07:35:26.870917+0000","last_became_peered":"2024-09-21T07:35:26.870917+0000","last_unstale":"2024-09-21T07:48:47.606600+0000","last_undegraded":"2024-09-21T07:48:47.606600+0000","last_fullsized":"2024-09-21T07:48:47.606600+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:59.159937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059198499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":370,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608972+0000","last_change":"2024-09-21T07:34:24.258709+0000","last_active":"2024-09-21T07:48:48.608972+0000","last_peered":"2024-09-21T07:48:48.608972+0000","last_clean":"2024-09-21T07:48:48.608972+0000","last_became_active":"2024-09-21T07:34:23.279042+0000","last_became_peered":"2024-09-21T07:34:23.279042+0000","last_unstale":"2024-09-21T07:48:48.608972+0000","last_undegraded":"2024-09-21T07:48:48.608972+0000","last_fullsized":"2024-09-21T07:48:48.608972+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:01.762117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019282100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608922+0000","last_change":"2024-09-21T07:48:47.607091+0000","last_active":"2024-09-21T07:48:48.608922+0000","last_peered":"2024-09-21T07:48:48.608922+0000","last_clean":"2024-09-21T07:48:48.608922+0000","last_became_active":"2024-09-21T07:48:47.606739+0000","last_became_peered":"2024-09-21T07:48:47.606739+0000","last_unstale":"2024-09-21T07:48:48.608922+0000","last_undegraded":"2024-09-21T07:48:48.608922+0000","last_fullsized":"2024-09-21T07:48:48.608922+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:42:23.765051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"218'2","reported_seq":15,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.628600+0000","last_change":"2024-09-21T07:48:46.612384+0000","last_active":"2024-09-21T07:48:47.628600+0000","last_peered":"2024-09-21T07:48:47.628600+0000","last_clean":"2024-09-21T07:48:47.628600+0000","last_became_active":"2024-09-21T07:48:46.610836+0000","last_became_peered":"2024-09-21T07:48:46.610836+0000","last_unstale":"2024-09-21T07:48:47.628600+0000","last_undegraded":"2024-09-21T07:48:47.628600+0000","last_fullsized":"2024-09-21T07:48:47.628600+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:04.310354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":304,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.598997+0000","last_change":"2024-09-21T07:35:27.870260+0000","last_active":"2024-09-21T07:48:47.598997+0000","last_peered":"2024-09-21T07:48:47.598997+0000","last_clean":"2024-09-21T07:48:47.598997+0000","last_became_active":"2024-09-21T07:35:27.869448+0000","last_became_peered":"2024-09-21T07:35:27.869448+0000","last_unstale":"2024-09-21T07:48:47.598997+0000","last_undegraded":"2024-09-21T07:48:47.598997+0000","last_fullsized":"2024-09-21T07:48:47.598997+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:16.946790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":309,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051488+0000","last_change":"2024-09-21T07:35:29.386367+0000","last_active":"2024-09-21T07:48:50.051488+0000","last_peered":"2024-09-21T07:48:50.051488+0000","last_clean":"2024-09-21T07:48:50.051488+0000","last_became_active":"2024-09-21T07:35:26.879238+0000","last_became_peered":"2024-09-21T07:35:26.879238+0000","last_unstale":"2024-09-21T07:48:50.051488+0000","last_undegraded":"2024-09-21T07:48:50.051488+0000","last_fullsized":"2024-09-21T07:48:50.051488+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:16.528260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092223100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":362,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.603686+0000","last_change":"2024-09-21T07:34:25.085635+0000","last_active":"2024-09-21T07:48:47.603686+0000","last_peered":"2024-09-21T07:48:47.603686+0000","last_clean":"2024-09-21T07:48:47.603686+0000","last_became_active":"2024-09-21T07:34:23.273566+0000","last_became_peered":"2024-09-21T07:34:23.273566+0000","last_unstale":"2024-09-21T07:48:47.603686+0000","last_undegraded":"2024-09-21T07:48:47.603686+0000","last_fullsized":"2024-09-21T07:48:47.603686+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:14.142272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079897300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.598103+0000","last_change":"2024-09-21T07:48:46.598103+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.598103+0000","last_undegraded":"2024-09-21T07:48:46.598103+0000","last_fullsized":"2024-09-21T07:48:46.598103+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609104+0000","last_change":"2024-09-21T07:48:46.610951+0000","last_active":"2024-09-21T07:48:48.609104+0000","last_peered":"2024-09-21T07:48:48.609104+0000","last_clean":"2024-09-21T07:48:48.609104+0000","last_became_active":"2024-09-21T07:48:46.610658+0000","last_became_peered":"2024-09-21T07:48:46.610658+0000","last_unstale":"2024-09-21T07:48:48.609104+0000","last_undegraded":"2024-09-21T07:48:48.609104+0000","last_fullsized":"2024-09-21T07:48:48.609104+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:15:20.903496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":304,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.596777+0000","last_change":"2024-09-21T07:35:27.878728+0000","last_active":"2024-09-21T07:48:47.596777+0000","last_peered":"2024-09-21T07:48:47.596777+0000","last_clean":"2024-09-21T07:48:47.596777+0000","last_became_active":"2024-09-21T07:35:27.878335+0000","last_became_peered":"2024-09-21T07:35:27.878335+0000","last_unstale":"2024-09-21T07:48:47.596777+0000","last_undegraded":"2024-09-21T07:48:47.596777+0000","last_fullsized":"2024-09-21T07:48:47.596777+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:25:37.731771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":322,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.591500+0000","last_change":"2024-09-21T07:35:28.858418+0000","last_active":"2024-09-21T07:48:46.591500+0000","last_peered":"2024-09-21T07:48:46.591500+0000","last_clean":"2024-09-21T07:48:46.591500+0000","last_became_active":"2024-09-21T07:35:26.876944+0000","last_became_peered":"2024-09-21T07:35:26.876944+0000","last_unstale":"2024-09-21T07:48:46.591500+0000","last_undegraded":"2024-09-21T07:48:46.591500+0000","last_fullsized":"2024-09-21T07:48:46.591500+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:04.514304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090862799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":358,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051805+0000","last_change":"2024-09-21T07:34:24.597926+0000","last_active":"2024-09-21T07:48:50.051805+0000","last_peered":"2024-09-21T07:48:50.051805+0000","last_clean":"2024-09-21T07:48:50.051805+0000","last_became_active":"2024-09-21T07:34:23.268926+0000","last_became_peered":"2024-09-21T07:34:23.268926+0000","last_unstale":"2024-09-21T07:48:50.051805+0000","last_undegraded":"2024-09-21T07:48:50.051805+0000","last_fullsized":"2024-09-21T07:48:50.051805+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:46.190752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101851,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.629415+0000","last_change":"2024-09-21T07:48:47.624905+0000","last_active":"2024-09-21T07:48:47.629415+0000","last_peered":"2024-09-21T07:48:47.629415+0000","last_clean":"2024-09-21T07:48:47.629415+0000","last_became_active":"2024-09-21T07:48:47.621782+0000","last_became_peered":"2024-09-21T07:48:47.621782+0000","last_unstale":"2024-09-21T07:48:47.629415+0000","last_undegraded":"2024-09-21T07:48:47.629415+0000","last_fullsized":"2024-09-21T07:48:47.629415+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:35.524796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051148+0000","last_change":"2024-09-21T07:48:46.613457+0000","last_active":"2024-09-21T07:48:50.051148+0000","last_peered":"2024-09-21T07:48:50.051148+0000","last_clean":"2024-09-21T07:48:50.051148+0000","last_became_active":"2024-09-21T07:48:46.612037+0000","last_became_peered":"2024-09-21T07:48:46.612037+0000","last_unstale":"2024-09-21T07:48:50.051148+0000","last_undegraded":"2024-09-21T07:48:50.051148+0000","last_fullsized":"2024-09-21T07:48:50.051148+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:06.200769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":303,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051801+0000","last_change":"2024-09-21T07:35:27.877426+0000","last_active":"2024-09-21T07:48:50.051801+0000","last_peered":"2024-09-21T07:48:50.051801+0000","last_clean":"2024-09-21T07:48:50.051801+0000","last_became_active":"2024-09-21T07:35:27.877050+0000","last_became_peered":"2024-09-21T07:35:27.877050+0000","last_unstale":"2024-09-21T07:48:50.051801+0000","last_undegraded":"2024-09-21T07:48:50.051801+0000","last_fullsized":"2024-09-21T07:48:50.051801+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:36.268334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":311,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.595671+0000","last_change":"2024-09-21T07:35:28.862273+0000","last_active":"2024-09-21T07:48:47.595671+0000","last_peered":"2024-09-21T07:48:47.595671+0000","last_clean":"2024-09-21T07:48:47.595671+0000","last_became_active":"2024-09-21T07:35:26.874011+0000","last_became_peered":"2024-09-21T07:35:26.874011+0000","last_unstale":"2024-09-21T07:48:47.595671+0000","last_undegraded":"2024-09-21T07:48:47.595671+0000","last_fullsized":"2024-09-21T07:48:47.595671+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:11.986950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071644799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":367,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.592690+0000","last_change":"2024-09-21T07:34:24.259794+0000","last_active":"2024-09-21T07:48:47.592690+0000","last_peered":"2024-09-21T07:48:47.592690+0000","last_clean":"2024-09-21T07:48:47.592690+0000","last_became_active":"2024-09-21T07:34:23.265679+0000","last_became_peered":"2024-09-21T07:34:23.265679+0000","last_unstale":"2024-09-21T07:48:47.592690+0000","last_undegraded":"2024-09-21T07:48:47.592690+0000","last_fullsized":"2024-09-21T07:48:47.592690+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:02.575796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001298518,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.630328+0000","last_change":"2024-09-21T07:48:47.618294+0000","last_active":"2024-09-21T07:48:47.630328+0000","last_peered":"2024-09-21T07:48:47.630328+0000","last_clean":"2024-09-21T07:48:47.630328+0000","last_became_active":"2024-09-21T07:48:47.617975+0000","last_became_peered":"2024-09-21T07:48:47.617975+0000","last_unstale":"2024-09-21T07:48:47.630328+0000","last_undegraded":"2024-09-21T07:48:47.630328+0000","last_fullsized":"2024-09-21T07:48:47.630328+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:42.345168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.610511+0000","last_change":"2024-09-21T07:48:46.608424+0000","last_active":"2024-09-21T07:48:46.610511+0000","last_peered":"2024-09-21T07:48:46.610511+0000","last_clean":"2024-09-21T07:48:46.610511+0000","last_became_active":"2024-09-21T07:48:46.607778+0000","last_became_peered":"2024-09-21T07:48:46.607778+0000","last_unstale":"2024-09-21T07:48:46.610511+0000","last_undegraded":"2024-09-21T07:48:46.610511+0000","last_fullsized":"2024-09-21T07:48:46.610511+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:59.561123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":306,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.609776+0000","last_change":"2024-09-21T07:35:27.877725+0000","last_active":"2024-09-21T07:48:47.609776+0000","last_peered":"2024-09-21T07:48:47.609776+0000","last_clean":"2024-09-21T07:48:47.609776+0000","last_became_active":"2024-09-21T07:35:27.877398+0000","last_became_peered":"2024-09-21T07:35:27.877398+0000","last_unstale":"2024-09-21T07:48:47.609776+0000","last_undegraded":"2024-09-21T07:48:47.609776+0000","last_fullsized":"2024-09-21T07:48:47.609776+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:05.859762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":314,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.593198+0000","last_change":"2024-09-21T07:35:28.860612+0000","last_active":"2024-09-21T07:48:47.593198+0000","last_peered":"2024-09-21T07:48:47.593198+0000","last_clean":"2024-09-21T07:48:47.593198+0000","last_became_active":"2024-09-21T07:35:26.876562+0000","last_became_peered":"2024-09-21T07:35:26.876562+0000","last_unstale":"2024-09-21T07:48:47.593198+0000","last_undegraded":"2024-09-21T07:48:47.593198+0000","last_fullsized":"2024-09-21T07:48:47.593198+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:06.824331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012252470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":361,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.602500+0000","last_change":"2024-09-21T07:34:24.260330+0000","last_active":"2024-09-21T07:48:46.602500+0000","last_peered":"2024-09-21T07:48:46.602500+0000","last_clean":"2024-09-21T07:48:46.602500+0000","last_became_active":"2024-09-21T07:34:23.637210+0000","last_became_peered":"2024-09-21T07:34:23.637210+0000","last_unstale":"2024-09-21T07:48:46.602500+0000","last_undegraded":"2024-09-21T07:48:46.602500+0000","last_fullsized":"2024-09-21T07:48:46.602500+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.488451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001537093,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.632407+0000","last_change":"2024-09-21T07:48:47.626890+0000","last_active":"2024-09-21T07:48:47.632407+0000","last_peered":"2024-09-21T07:48:47.632407+0000","last_clean":"2024-09-21T07:48:47.632407+0000","last_became_active":"2024-09-21T07:48:47.626088+0000","last_became_peered":"2024-09-21T07:48:47.626088+0000","last_unstale":"2024-09-21T07:48:47.632407+0000","last_undegraded":"2024-09-21T07:48:47.632407+0000","last_fullsized":"2024-09-21T07:48:47.632407+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:56:05.110865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.7","version":"218'2","reported_seq":15,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.625284+0000","last_change":"2024-09-21T07:48:46.607721+0000","last_active":"2024-09-21T07:48:47.625284+0000","last_peered":"2024-09-21T07:48:47.625284+0000","last_clean":"2024-09-21T07:48:47.625284+0000","last_became_active":"2024-09-21T07:48:46.607413+0000","last_became_peered":"2024-09-21T07:48:46.607413+0000","last_unstale":"2024-09-21T07:48:47.625284+0000","last_undegraded":"2024-09-21T07:48:47.625284+0000","last_fullsized":"2024-09-21T07:48:47.625284+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:47.152954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":304,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.594537+0000","last_change":"2024-09-21T07:35:27.874757+0000","last_active":"2024-09-21T07:48:47.594537+0000","last_peered":"2024-09-21T07:48:47.594537+0000","last_clean":"2024-09-21T07:48:47.594537+0000","last_became_active":"2024-09-21T07:35:27.874360+0000","last_became_peered":"2024-09-21T07:35:27.874360+0000","last_unstale":"2024-09-21T07:48:47.594537+0000","last_undegraded":"2024-09-21T07:48:47.594537+0000","last_fullsized":"2024-09-21T07:48:47.594537+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:17.646912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":308,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.602783+0000","last_change":"2024-09-21T07:35:29.493431+0000","last_active":"2024-09-21T07:48:46.602783+0000","last_peered":"2024-09-21T07:48:46.602783+0000","last_clean":"2024-09-21T07:48:46.602783+0000","last_became_active":"2024-09-21T07:35:26.873691+0000","last_became_peered":"2024-09-21T07:35:26.873691+0000","last_unstale":"2024-09-21T07:48:46.602783+0000","last_undegraded":"2024-09-21T07:48:46.602783+0000","last_fullsized":"2024-09-21T07:48:46.602783+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:51.705051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001162464,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":358,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051405+0000","last_change":"2024-09-21T07:34:24.597286+0000","last_active":"2024-09-21T07:48:50.051405+0000","last_peered":"2024-09-21T07:48:50.051405+0000","last_clean":"2024-09-21T07:48:50.051405+0000","last_became_active":"2024-09-21T07:34:23.267133+0000","last_became_peered":"2024-09-21T07:34:23.267133+0000","last_unstale":"2024-09-21T07:48:50.051405+0000","last_undegraded":"2024-09-21T07:48:50.051405+0000","last_fullsized":"2024-09-21T07:48:50.051405+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:18.257010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.627524+0000","last_change":"2024-09-21T07:48:47.624322+0000","last_active":"2024-09-21T07:48:47.627524+0000","last_peered":"2024-09-21T07:48:47.627524+0000","last_clean":"2024-09-21T07:48:47.627524+0000","last_became_active":"2024-09-21T07:48:47.623920+0000","last_became_peered":"2024-09-21T07:48:47.623920+0000","last_unstale":"2024-09-21T07:48:47.627524+0000","last_undegraded":"2024-09-21T07:48:47.627524+0000","last_fullsized":"2024-09-21T07:48:47.627524+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:31.490205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":13,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.605003+0000","last_change":"2024-09-21T07:48:46.599263+0000","last_active":"2024-09-21T07:48:47.605003+0000","last_peered":"2024-09-21T07:48:47.605003+0000","last_clean":"2024-09-21T07:48:47.605003+0000","last_became_active":"2024-09-21T07:48:46.597628+0000","last_became_peered":"2024-09-21T07:48:46.597628+0000","last_unstale":"2024-09-21T07:48:47.605003+0000","last_undegraded":"2024-09-21T07:48:47.605003+0000","last_fullsized":"2024-09-21T07:48:47.605003+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:38.541554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":315,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.593982+0000","last_change":"2024-09-21T07:35:27.864458+0000","last_active":"2024-09-21T07:48:46.593982+0000","last_peered":"2024-09-21T07:48:46.593982+0000","last_clean":"2024-09-21T07:48:46.593982+0000","last_became_active":"2024-09-21T07:35:27.864136+0000","last_became_peered":"2024-09-21T07:35:27.864136+0000","last_unstale":"2024-09-21T07:48:46.593982+0000","last_undegraded":"2024-09-21T07:48:46.593982+0000","last_fullsized":"2024-09-21T07:48:46.593982+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:20.162240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":322,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.594391+0000","last_change":"2024-09-21T07:35:28.858784+0000","last_active":"2024-09-21T07:48:46.594391+0000","last_peered":"2024-09-21T07:48:46.594391+0000","last_clean":"2024-09-21T07:48:46.594391+0000","last_became_active":"2024-09-21T07:35:26.876890+0000","last_became_peered":"2024-09-21T07:35:26.876890+0000","last_unstale":"2024-09-21T07:48:46.594391+0000","last_undegraded":"2024-09-21T07:48:46.594391+0000","last_fullsized":"2024-09-21T07:48:46.594391+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:40.738327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064543499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":369,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609757+0000","last_change":"2024-09-21T07:34:24.258712+0000","last_active":"2024-09-21T07:48:48.609757+0000","last_peered":"2024-09-21T07:48:48.609757+0000","last_clean":"2024-09-21T07:48:48.609757+0000","last_became_active":"2024-09-21T07:34:23.274692+0000","last_became_peered":"2024-09-21T07:34:23.274692+0000","last_unstale":"2024-09-21T07:48:48.609757+0000","last_undegraded":"2024-09-21T07:48:48.609757+0000","last_fullsized":"2024-09-21T07:48:48.609757+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:43.279454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018051479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.623521+0000","last_change":"2024-09-21T07:48:47.617896+0000","last_active":"2024-09-21T07:48:47.623521+0000","last_peered":"2024-09-21T07:48:47.623521+0000","last_clean":"2024-09-21T07:48:47.623521+0000","last_became_active":"2024-09-21T07:48:47.617588+0000","last_became_peered":"2024-09-21T07:48:47.617588+0000","last_unstale":"2024-09-21T07:48:47.623521+0000","last_undegraded":"2024-09-21T07:48:47.623521+0000","last_fullsized":"2024-09-21T07:48:47.623521+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:46.539795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051487+0000","last_change":"2024-09-21T07:48:46.609456+0000","last_active":"2024-09-21T07:48:50.051487+0000","last_peered":"2024-09-21T07:48:50.051487+0000","last_clean":"2024-09-21T07:48:50.051487+0000","last_became_active":"2024-09-21T07:48:46.609142+0000","last_became_peered":"2024-09-21T07:48:46.609142+0000","last_unstale":"2024-09-21T07:48:50.051487+0000","last_undegraded":"2024-09-21T07:48:50.051487+0000","last_fullsized":"2024-09-21T07:48:50.051487+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:21.813300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":308,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609370+0000","last_change":"2024-09-21T07:35:27.878020+0000","last_active":"2024-09-21T07:48:48.609370+0000","last_peered":"2024-09-21T07:48:48.609370+0000","last_clean":"2024-09-21T07:48:48.609370+0000","last_became_active":"2024-09-21T07:35:27.877532+0000","last_became_peered":"2024-09-21T07:35:27.877532+0000","last_unstale":"2024-09-21T07:48:48.609370+0000","last_undegraded":"2024-09-21T07:48:48.609370+0000","last_fullsized":"2024-09-21T07:48:48.609370+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:19.756073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":314,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.592718+0000","last_change":"2024-09-21T07:35:28.861049+0000","last_active":"2024-09-21T07:48:47.592718+0000","last_peered":"2024-09-21T07:48:47.592718+0000","last_clean":"2024-09-21T07:48:47.592718+0000","last_became_active":"2024-09-21T07:35:26.876819+0000","last_became_peered":"2024-09-21T07:35:26.876819+0000","last_unstale":"2024-09-21T07:48:47.592718+0000","last_undegraded":"2024-09-21T07:48:47.592718+0000","last_fullsized":"2024-09-21T07:48:47.592718+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:19.633803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055780599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":362,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.594656+0000","last_change":"2024-09-21T07:34:25.084849+0000","last_active":"2024-09-21T07:48:47.594656+0000","last_peered":"2024-09-21T07:48:47.594656+0000","last_clean":"2024-09-21T07:48:47.594656+0000","last_became_active":"2024-09-21T07:34:23.277713+0000","last_became_peered":"2024-09-21T07:34:23.277713+0000","last_unstale":"2024-09-21T07:48:47.594656+0000","last_undegraded":"2024-09-21T07:48:47.594656+0000","last_fullsized":"2024-09-21T07:48:47.594656+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:05.811810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059079499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.590874+0000","last_change":"2024-09-21T07:48:46.590874+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.590874+0000","last_undegraded":"2024-09-21T07:48:46.590874+0000","last_fullsized":"2024-09-21T07:48:46.590874+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.610468+0000","last_change":"2024-09-21T07:48:46.607396+0000","last_active":"2024-09-21T07:48:46.610468+0000","last_peered":"2024-09-21T07:48:46.610468+0000","last_clean":"2024-09-21T07:48:46.610468+0000","last_became_active":"2024-09-21T07:48:46.607082+0000","last_became_peered":"2024-09-21T07:48:46.607082+0000","last_unstale":"2024-09-21T07:48:46.610468+0000","last_undegraded":"2024-09-21T07:48:46.610468+0000","last_fullsized":"2024-09-21T07:48:46.610468+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:20.897148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":306,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.601794+0000","last_change":"2024-09-21T07:35:27.884248+0000","last_active":"2024-09-21T07:48:47.601794+0000","last_peered":"2024-09-21T07:48:47.601794+0000","last_clean":"2024-09-21T07:48:47.601794+0000","last_became_active":"2024-09-21T07:35:27.883861+0000","last_became_peered":"2024-09-21T07:35:27.883861+0000","last_unstale":"2024-09-21T07:48:47.601794+0000","last_undegraded":"2024-09-21T07:48:47.601794+0000","last_fullsized":"2024-09-21T07:48:47.601794+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":312,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.606467+0000","last_change":"2024-09-21T07:35:29.207135+0000","last_active":"2024-09-21T07:48:47.606467+0000","last_peered":"2024-09-21T07:48:47.606467+0000","last_clean":"2024-09-21T07:48:47.606467+0000","last_became_active":"2024-09-21T07:35:26.872787+0000","last_became_peered":"2024-09-21T07:35:26.872787+0000","last_unstale":"2024-09-21T07:48:47.606467+0000","last_undegraded":"2024-09-21T07:48:47.606467+0000","last_fullsized":"2024-09-21T07:48:47.606467+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:27.543217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011157509999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":374,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.594088+0000","last_change":"2024-09-21T07:34:24.538610+0000","last_active":"2024-09-21T07:48:46.594088+0000","last_peered":"2024-09-21T07:48:46.594088+0000","last_clean":"2024-09-21T07:48:46.594088+0000","last_became_active":"2024-09-21T07:34:23.259700+0000","last_became_peered":"2024-09-21T07:34:23.259700+0000","last_unstale":"2024-09-21T07:48:46.594088+0000","last_undegraded":"2024-09-21T07:48:46.594088+0000","last_fullsized":"2024-09-21T07:48:46.594088+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:57.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070522399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.632186+0000","last_change":"2024-09-21T07:48:47.623345+0000","last_active":"2024-09-21T07:48:47.632186+0000","last_peered":"2024-09-21T07:48:47.632186+0000","last_clean":"2024-09-21T07:48:47.632186+0000","last_became_active":"2024-09-21T07:48:47.622998+0000","last_became_peered":"2024-09-21T07:48:47.622998+0000","last_unstale":"2024-09-21T07:48:47.632186+0000","last_undegraded":"2024-09-21T07:48:47.632186+0000","last_fullsized":"2024-09-21T07:48:47.632186+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:26:23.537534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":13,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.605671+0000","last_change":"2024-09-21T07:48:46.599294+0000","last_active":"2024-09-21T07:48:47.605671+0000","last_peered":"2024-09-21T07:48:47.605671+0000","last_clean":"2024-09-21T07:48:47.605671+0000","last_became_active":"2024-09-21T07:48:46.598633+0000","last_became_peered":"2024-09-21T07:48:46.598633+0000","last_unstale":"2024-09-21T07:48:47.605671+0000","last_undegraded":"2024-09-21T07:48:47.605671+0000","last_fullsized":"2024-09-21T07:48:47.605671+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:34.227316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":315,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.592285+0000","last_change":"2024-09-21T07:35:27.868024+0000","last_active":"2024-09-21T07:48:46.592285+0000","last_peered":"2024-09-21T07:48:46.592285+0000","last_clean":"2024-09-21T07:48:46.592285+0000","last_became_active":"2024-09-21T07:35:27.867339+0000","last_became_peered":"2024-09-21T07:35:27.867339+0000","last_unstale":"2024-09-21T07:48:46.592285+0000","last_undegraded":"2024-09-21T07:48:46.592285+0000","last_fullsized":"2024-09-21T07:48:46.592285+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":322,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.591127+0000","last_change":"2024-09-21T07:35:28.858505+0000","last_active":"2024-09-21T07:48:46.591127+0000","last_peered":"2024-09-21T07:48:46.591127+0000","last_clean":"2024-09-21T07:48:46.591127+0000","last_became_active":"2024-09-21T07:35:26.877320+0000","last_became_peered":"2024-09-21T07:35:26.877320+0000","last_unstale":"2024-09-21T07:48:46.591127+0000","last_undegraded":"2024-09-21T07:48:46.591127+0000","last_fullsized":"2024-09-21T07:48:46.591127+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:53.672782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244119999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":360,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.052005+0000","last_change":"2024-09-21T07:34:24.597597+0000","last_active":"2024-09-21T07:48:50.052005+0000","last_peered":"2024-09-21T07:48:50.052005+0000","last_clean":"2024-09-21T07:48:50.052005+0000","last_became_active":"2024-09-21T07:34:23.635204+0000","last_became_peered":"2024-09-21T07:34:23.635204+0000","last_unstale":"2024-09-21T07:48:50.052005+0000","last_undegraded":"2024-09-21T07:48:50.052005+0000","last_fullsized":"2024-09-21T07:48:50.052005+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:48.644789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081139199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.597041+0000","last_change":"2024-09-21T07:48:46.597041+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.597041+0000","last_undegraded":"2024-09-21T07:48:46.597041+0000","last_fullsized":"2024-09-21T07:48:46.597041+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":13,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.605822+0000","last_change":"2024-09-21T07:48:46.602945+0000","last_active":"2024-09-21T07:48:47.605822+0000","last_peered":"2024-09-21T07:48:47.605822+0000","last_clean":"2024-09-21T07:48:47.605822+0000","last_became_active":"2024-09-21T07:48:46.602486+0000","last_became_peered":"2024-09-21T07:48:46.602486+0000","last_unstale":"2024-09-21T07:48:47.605822+0000","last_undegraded":"2024-09-21T07:48:47.605822+0000","last_fullsized":"2024-09-21T07:48:47.605822+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:51:05.478321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":305,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.606002+0000","last_change":"2024-09-21T07:35:27.877503+0000","last_active":"2024-09-21T07:48:47.606002+0000","last_peered":"2024-09-21T07:48:47.606002+0000","last_clean":"2024-09-21T07:48:47.606002+0000","last_became_active":"2024-09-21T07:35:27.875775+0000","last_became_peered":"2024-09-21T07:35:27.875775+0000","last_unstale":"2024-09-21T07:48:47.606002+0000","last_undegraded":"2024-09-21T07:48:47.606002+0000","last_fullsized":"2024-09-21T07:48:47.606002+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:49.291023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":308,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.597282+0000","last_change":"2024-09-21T07:35:29.491412+0000","last_active":"2024-09-21T07:48:46.597282+0000","last_peered":"2024-09-21T07:48:46.597282+0000","last_clean":"2024-09-21T07:48:46.597282+0000","last_became_active":"2024-09-21T07:35:26.876404+0000","last_became_peered":"2024-09-21T07:35:26.876404+0000","last_unstale":"2024-09-21T07:48:46.597282+0000","last_undegraded":"2024-09-21T07:48:46.597282+0000","last_fullsized":"2024-09-21T07:48:46.597282+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063918000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":370,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.610303+0000","last_change":"2024-09-21T07:34:24.259026+0000","last_active":"2024-09-21T07:48:48.610303+0000","last_peered":"2024-09-21T07:48:48.610303+0000","last_clean":"2024-09-21T07:48:48.610303+0000","last_became_active":"2024-09-21T07:34:23.635456+0000","last_became_peered":"2024-09-21T07:34:23.635456+0000","last_unstale":"2024-09-21T07:48:48.610303+0000","last_undegraded":"2024-09-21T07:48:48.610303+0000","last_fullsized":"2024-09-21T07:48:48.610303+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:09.592752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018654419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.598035+0000","last_change":"2024-09-21T07:48:46.598035+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.598035+0000","last_undegraded":"2024-09-21T07:48:46.598035+0000","last_fullsized":"2024-09-21T07:48:46.598035+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.613990+0000","last_change":"2024-09-21T07:48:46.612048+0000","last_active":"2024-09-21T07:48:46.613990+0000","last_peered":"2024-09-21T07:48:46.613990+0000","last_clean":"2024-09-21T07:48:46.613990+0000","last_became_active":"2024-09-21T07:48:46.611755+0000","last_became_peered":"2024-09-21T07:48:46.611755+0000","last_unstale":"2024-09-21T07:48:46.613990+0000","last_undegraded":"2024-09-21T07:48:46.613990+0000","last_fullsized":"2024-09-21T07:48:46.613990+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:12:05.809707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":305,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.605145+0000","last_change":"2024-09-21T07:35:27.871984+0000","last_active":"2024-09-21T07:48:47.605145+0000","last_peered":"2024-09-21T07:48:47.605145+0000","last_clean":"2024-09-21T07:48:47.605145+0000","last_became_active":"2024-09-21T07:35:27.870336+0000","last_became_peered":"2024-09-21T07:35:27.870336+0000","last_unstale":"2024-09-21T07:48:47.605145+0000","last_undegraded":"2024-09-21T07:48:47.605145+0000","last_fullsized":"2024-09-21T07:48:47.605145+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:57.120479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":314,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.592628+0000","last_change":"2024-09-21T07:35:28.860351+0000","last_active":"2024-09-21T07:48:47.592628+0000","last_peered":"2024-09-21T07:48:47.592628+0000","last_clean":"2024-09-21T07:48:47.592628+0000","last_became_active":"2024-09-21T07:35:26.872046+0000","last_became_peered":"2024-09-21T07:35:26.872046+0000","last_unstale":"2024-09-21T07:48:47.592628+0000","last_undegraded":"2024-09-21T07:48:47.592628+0000","last_fullsized":"2024-09-21T07:48:47.592628+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:47.074327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098384399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":361,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.600950+0000","last_change":"2024-09-21T07:34:24.260300+0000","last_active":"2024-09-21T07:48:46.600950+0000","last_peered":"2024-09-21T07:48:46.600950+0000","last_clean":"2024-09-21T07:48:46.600950+0000","last_became_active":"2024-09-21T07:34:23.270128+0000","last_became_peered":"2024-09-21T07:34:23.270128+0000","last_unstale":"2024-09-21T07:48:46.600950+0000","last_undegraded":"2024-09-21T07:48:46.600950+0000","last_fullsized":"2024-09-21T07:48:46.600950+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:25.260931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00176651,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.628380+0000","last_change":"2024-09-21T07:48:47.626058+0000","last_active":"2024-09-21T07:48:47.628380+0000","last_peered":"2024-09-21T07:48:47.628380+0000","last_clean":"2024-09-21T07:48:47.628380+0000","last_became_active":"2024-09-21T07:48:47.625779+0000","last_became_peered":"2024-09-21T07:48:47.625779+0000","last_unstale":"2024-09-21T07:48:47.628380+0000","last_undegraded":"2024-09-21T07:48:47.628380+0000","last_fullsized":"2024-09-21T07:48:47.628380+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:03.169164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":13,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.593455+0000","last_change":"2024-09-21T07:48:46.607858+0000","last_active":"2024-09-21T07:48:47.593455+0000","last_peered":"2024-09-21T07:48:47.593455+0000","last_clean":"2024-09-21T07:48:47.593455+0000","last_became_active":"2024-09-21T07:48:46.607521+0000","last_became_peered":"2024-09-21T07:48:46.607521+0000","last_unstale":"2024-09-21T07:48:47.593455+0000","last_undegraded":"2024-09-21T07:48:47.593455+0000","last_fullsized":"2024-09-21T07:48:47.593455+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:18.298760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":305,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.609882+0000","last_change":"2024-09-21T07:35:27.877861+0000","last_active":"2024-09-21T07:48:47.609882+0000","last_peered":"2024-09-21T07:48:47.609882+0000","last_clean":"2024-09-21T07:48:47.609882+0000","last_became_active":"2024-09-21T07:35:27.877434+0000","last_became_peered":"2024-09-21T07:35:27.877434+0000","last_unstale":"2024-09-21T07:48:47.609882+0000","last_undegraded":"2024-09-21T07:48:47.609882+0000","last_fullsized":"2024-09-21T07:48:47.609882+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":311,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.597277+0000","last_change":"2024-09-21T07:35:28.862954+0000","last_active":"2024-09-21T07:48:47.597277+0000","last_peered":"2024-09-21T07:48:47.597277+0000","last_clean":"2024-09-21T07:48:47.597277+0000","last_became_active":"2024-09-21T07:35:26.870613+0000","last_became_peered":"2024-09-21T07:35:26.870613+0000","last_unstale":"2024-09-21T07:48:47.597277+0000","last_undegraded":"2024-09-21T07:48:47.597277+0000","last_fullsized":"2024-09-21T07:48:47.597277+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:06.901043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059842100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":364,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.613383+0000","last_change":"2024-09-21T07:34:25.028904+0000","last_active":"2024-09-21T07:48:47.613383+0000","last_peered":"2024-09-21T07:48:47.613383+0000","last_clean":"2024-09-21T07:48:47.613383+0000","last_became_active":"2024-09-21T07:34:23.260942+0000","last_became_peered":"2024-09-21T07:34:23.260942+0000","last_unstale":"2024-09-21T07:48:47.613383+0000","last_undegraded":"2024-09-21T07:48:47.613383+0000","last_fullsized":"2024-09-21T07:48:47.613383+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:22.709328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060553899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":776,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27,"ondisk_log_size":27,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509632,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66372,"kb_used_data":3248,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94305468,"statfs":{"total":96636764160,"available":96568799232,"internally_reserved":0,"allocated":3325952,"data_stored":2046648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509634,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51908,"kb_used_data":2668,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319932,"statfs":{"total":96636764160,"available":96583610368,"internally_reserved":0,"allocated":2732032,"data_stored":1456280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509632,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33860,"kb_used_data":2668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337980,"statfs":{"total":96636764160,"available":96602091520,"internally_reserved":0,"allocated":2732032,"data_stored":1456280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509633,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29696,"kb_used_data":2668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342144,"statfs":{"total":96636764160,"available":96606355456,"internally_reserved":0,"allocated":2732032,"data_stored":1456280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509634,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47748,"kb_used_data":2668,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324092,"statfs":{"total":96636764160,"available":96587870208,"internally_reserved":0,"allocated":2732032,"data_stored":1456280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509634,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52492,"kb_used_data":3248,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319348,"statfs":{"total":96636764160,"available":96583012352,"internally_reserved":0,"allocated":3325952,"data_stored":2046648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542338,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51936,"kb_used_data":2696,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319904,"statfs":{"total":96636764160,"available":96583581696,"internally_reserved":0,"allocated":2760704,"data_stored":1472495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542337,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74148,"kb_used_data":2704,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297692,"statfs":{"total":96636764160,"available":96560836608,"internally_reserved":0,"allocated":2768896,"data_stored":1472039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:48:52.100 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:48:52.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.355+0000 7fa8bda00640 1 -- 172.21.15.39:0/949606130 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8b806eab0 msgr2=0x7fa8b806eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:52.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.355+0000 7fa8bda00640 1 --2- 172.21.15.39:0/949606130 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8b806eab0 0x7fa8b806eec0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0005fd0 tx=0x7fa8b002aa90 comp rx=0 tx=0).stop 2024-09-21T07:48:52.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.356+0000 7fa8bda00640 1 -- 172.21.15.39:0/949606130 shutdown_connections 2024-09-21T07:48:52.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.356+0000 7fa8bda00640 1 --2- 172.21.15.39:0/949606130 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8b80d2940 0x7fa8b806e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:52.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.356+0000 7fa8bda00640 1 --2- 172.21.15.39:0/949606130 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8b806eab0 0x7fa8b806eec0 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:52.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.356+0000 7fa8bda00640 1 -- 172.21.15.39:0/949606130 >> 172.21.15.39:0/949606130 conn(0x7fa8b809b5f0 msgr2=0x7fa8b808c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:52.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.356+0000 7fa8bda00640 1 -- 172.21.15.39:0/949606130 shutdown_connections 2024-09-21T07:48:52.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.356+0000 7fa8bda00640 1 -- 172.21.15.39:0/949606130 wait complete. 2024-09-21T07:48:52.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.358+0000 7fa8bda00640 1 Processor -- start 2024-09-21T07:48:52.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.358+0000 7fa8bda00640 1 -- start start 2024-09-21T07:48:52.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.359+0000 7fa8bda00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8b806eab0 0x7fa8b80cbd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:52.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.359+0000 7fa8bda00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8b80d0410 0x7fa8b815f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:52.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.359+0000 7fa8bda00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8b809e740 0x7fa8b809eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:52.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.359+0000 7fa8bda00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b80cd3a0 con 0x7fa8b806eab0 2024-09-21T07:48:52.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.359+0000 7fa8bda00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b80cd220 con 0x7fa8b80d0410 2024-09-21T07:48:52.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.359+0000 7fa8bda00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa8b80cd520 con 0x7fa8b809e740 2024-09-21T07:48:52.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.360+0000 7fa8b7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8b806eab0 0x7fa8b80cbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:52.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.360+0000 7fa8b7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8b806eab0 0x7fa8b80cbd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48814/0 (socket says 172.21.15.39:48814) 2024-09-21T07:48:52.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.360+0000 7fa8b7e00640 1 -- 172.21.15.39:0/83756160 learned_addr learned my addr 172.21.15.39:0/83756160 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:52.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.360+0000 7fa8b7e00640 1 -- 172.21.15.39:0/83756160 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8b809e740 msgr2=0x7fa8b809eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:52.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.360+0000 7fa8bd000640 1 --2- 172.21.15.39:0/83756160 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8b809e740 0x7fa8b809eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:52.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.360+0000 7fa8b7e00640 1 --2- 172.21.15.39:0/83756160 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8b809e740 0x7fa8b809eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:52.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.360+0000 7fa8b7e00640 1 -- 172.21.15.39:0/83756160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8b80d0410 msgr2=0x7fa8b815f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:52.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.360+0000 7fa8b7400640 1 --2- 172.21.15.39:0/83756160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8b80d0410 0x7fa8b815f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:52.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.360+0000 7fa8b7e00640 1 --2- 172.21.15.39:0/83756160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8b80d0410 0x7fa8b815f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:52.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.360+0000 7fa8b7e00640 1 -- 172.21.15.39:0/83756160 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8b809f090 con 0x7fa8b806eab0 2024-09-21T07:48:52.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.360+0000 7fa8b7400640 1 --2- 172.21.15.39:0/83756160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8b80d0410 0x7fa8b815f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:48:52.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.361+0000 7fa8b7e00640 1 --2- 172.21.15.39:0/83756160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8b806eab0 0x7fa8b80cbd70 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7fa8a800c3c0 tx=0x7fa8a800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:52.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.361+0000 7fa8afe00640 1 -- 172.21.15.39:0/83756160 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a8017070 con 0x7fa8b806eab0 2024-09-21T07:48:52.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.361+0000 7fa8afe00640 1 -- 172.21.15.39:0/83756160 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8a8003640 con 0x7fa8b806eab0 2024-09-21T07:48:52.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.361+0000 7fa8bda00640 1 -- 172.21.15.39:0/83756160 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8b81f5a00 con 0x7fa8b806eab0 2024-09-21T07:48:52.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.361+0000 7fa8afe00640 1 -- 172.21.15.39:0/83756160 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a80054e0 con 0x7fa8b806eab0 2024-09-21T07:48:52.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.361+0000 7fa8bda00640 1 -- 172.21.15.39:0/83756160 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8b81f5fc0 con 0x7fa8b806eab0 2024-09-21T07:48:52.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.362+0000 7fa8bda00640 1 -- 172.21.15.39:0/83756160 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8b808c8c0 con 0x7fa8b806eab0 2024-09-21T07:48:52.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.363+0000 7fa8afe00640 1 -- 172.21.15.39:0/83756160 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa8a801a070 con 0x7fa8b806eab0 2024-09-21T07:48:52.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.364+0000 7fa8afe00640 1 --2- 172.21.15.39:0/83756160 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa898078bc0 0x7fa89807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:52.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.364+0000 7fa8afe00640 1 -- 172.21.15.39:0/83756160 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(219..219 src has 1..219) ==== 6988+0+0 (secure 0 0 0) 0x7fa8a809a4d0 con 0x7fa8b806eab0 2024-09-21T07:48:52.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.365+0000 7fa8b7400640 1 --2- 172.21.15.39:0/83756160 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa898078bc0 0x7fa89807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:52.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.366+0000 7fa8b7400640 1 --2- 172.21.15.39:0/83756160 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa898078bc0 0x7fa89807b0d0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fa8b815fe00 tx=0x7fa8a4005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:52.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.369+0000 7fa8afe00640 1 -- 172.21.15.39:0/83756160 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa8b809f090 con 0x7fa8b806eab0 2024-09-21T07:48:52.554 INFO:tasks.ceph.mds.c.smithi039.stderr:2024-09-21T07:48:52.552+0000 7fe9b7000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:48:52.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.561+0000 7fa8bda00640 1 -- 172.21.15.39:0/83756160 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa8b81f6610 con 0x7fa898078bc0 2024-09-21T07:48:52.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.571+0000 7fa8afe00640 1 -- 172.21.15.39:0/83756160 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+493904 (secure 0 0 0) 0x7fa8b81f6610 con 0x7fa898078bc0 2024-09-21T07:48:52.578 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:48:52.581 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:48:52.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.576+0000 7fa8bda00640 1 -- 172.21.15.39:0/83756160 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa898078bc0 msgr2=0x7fa89807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:52.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.576+0000 7fa8bda00640 1 --2- 172.21.15.39:0/83756160 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa898078bc0 0x7fa89807b0d0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fa8b815fe00 tx=0x7fa8a4005f90 comp rx=0 tx=0).stop 2024-09-21T07:48:52.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.576+0000 7fa8bda00640 1 -- 172.21.15.39:0/83756160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8b806eab0 msgr2=0x7fa8b80cbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:52.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.577+0000 7fa8bda00640 1 --2- 172.21.15.39:0/83756160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8b806eab0 0x7fa8b80cbd70 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7fa8a800c3c0 tx=0x7fa8a800c880 comp rx=0 tx=0).stop 2024-09-21T07:48:52.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.577+0000 7fa8bda00640 1 -- 172.21.15.39:0/83756160 shutdown_connections 2024-09-21T07:48:52.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.577+0000 7fa8bda00640 1 --2- 172.21.15.39:0/83756160 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa898078bc0 0x7fa89807b0d0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:52.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.577+0000 7fa8bda00640 1 --2- 172.21.15.39:0/83756160 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8b809e740 0x7fa8b809eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:52.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.577+0000 7fa8bda00640 1 --2- 172.21.15.39:0/83756160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8b80d0410 0x7fa8b815f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:52.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.577+0000 7fa8bda00640 1 --2- 172.21.15.39:0/83756160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8b806eab0 0x7fa8b80cbd70 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:52.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.577+0000 7fa8bda00640 1 -- 172.21.15.39:0/83756160 >> 172.21.15.39:0/83756160 conn(0x7fa8b809b5f0 msgr2=0x7fa8b81600c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:52.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.577+0000 7fa8bda00640 1 -- 172.21.15.39:0/83756160 shutdown_connections 2024-09-21T07:48:52.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.577+0000 7fa8bda00640 1 -- 172.21.15.39:0/83756160 wait complete. 2024-09-21T07:48:52.603 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":689,"stamp":"2024-09-21T07:48:51.852637+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591144,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":135,"num_write_kb":1638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":284,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":408160,"kb_used_data":22568,"kb_used_omap":12,"kb_used_meta":385395,"kb_avail":754566560,"statfs":{"total":773094113280,"available":772676157440,"internally_reserved":0,"allocated":23109632,"data_stored":12862950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":394645040},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":776,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001975"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.590080+0000","last_change":"2024-09-21T07:48:46.590080+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.590080+0000","last_undegraded":"2024-09-21T07:48:46.590080+0000","last_fullsized":"2024-09-21T07:48:46.590080+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.610021+0000","last_change":"2024-09-21T07:48:47.608680+0000","last_active":"2024-09-21T07:48:48.610021+0000","last_peered":"2024-09-21T07:48:48.610021+0000","last_clean":"2024-09-21T07:48:48.610021+0000","last_became_active":"2024-09-21T07:48:47.606772+0000","last_became_peered":"2024-09-21T07:48:47.606772+0000","last_unstale":"2024-09-21T07:48:48.610021+0000","last_undegraded":"2024-09-21T07:48:48.610021+0000","last_fullsized":"2024-09-21T07:48:48.610021+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:13.951566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.626629+0000","last_change":"2024-09-21T07:48:47.622897+0000","last_active":"2024-09-21T07:48:47.626629+0000","last_peered":"2024-09-21T07:48:47.626629+0000","last_clean":"2024-09-21T07:48:47.626629+0000","last_became_active":"2024-09-21T07:48:47.622568+0000","last_became_peered":"2024-09-21T07:48:47.622568+0000","last_unstale":"2024-09-21T07:48:47.626629+0000","last_undegraded":"2024-09-21T07:48:47.626629+0000","last_fullsized":"2024-09-21T07:48:47.626629+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:53:43.260855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.600167+0000","last_change":"2024-09-21T07:48:46.600167+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.600167+0000","last_undegraded":"2024-09-21T07:48:46.600167+0000","last_fullsized":"2024-09-21T07:48:46.600167+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.606103+0000","last_change":"2024-09-21T07:48:47.602747+0000","last_active":"2024-09-21T07:48:47.606103+0000","last_peered":"2024-09-21T07:48:47.606103+0000","last_clean":"2024-09-21T07:48:47.606103+0000","last_became_active":"2024-09-21T07:48:47.602453+0000","last_became_peered":"2024-09-21T07:48:47.602453+0000","last_unstale":"2024-09-21T07:48:47.606103+0000","last_undegraded":"2024-09-21T07:48:47.606103+0000","last_fullsized":"2024-09-21T07:48:47.606103+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:29.795876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.597012+0000","last_change":"2024-09-21T07:48:46.597012+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.597012+0000","last_undegraded":"2024-09-21T07:48:46.597012+0000","last_fullsized":"2024-09-21T07:48:46.597012+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.604038+0000","last_change":"2024-09-21T07:48:47.602027+0000","last_active":"2024-09-21T07:48:47.604038+0000","last_peered":"2024-09-21T07:48:47.604038+0000","last_clean":"2024-09-21T07:48:47.604038+0000","last_became_active":"2024-09-21T07:48:47.601703+0000","last_became_peered":"2024-09-21T07:48:47.601703+0000","last_unstale":"2024-09-21T07:48:47.604038+0000","last_undegraded":"2024-09-21T07:48:47.604038+0000","last_fullsized":"2024-09-21T07:48:47.604038+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:39.401545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.632462+0000","last_change":"2024-09-21T07:48:47.627224+0000","last_active":"2024-09-21T07:48:47.632462+0000","last_peered":"2024-09-21T07:48:47.632462+0000","last_clean":"2024-09-21T07:48:47.632462+0000","last_became_active":"2024-09-21T07:48:47.626767+0000","last_became_peered":"2024-09-21T07:48:47.626767+0000","last_unstale":"2024-09-21T07:48:47.632462+0000","last_undegraded":"2024-09-21T07:48:47.632462+0000","last_fullsized":"2024-09-21T07:48:47.632462+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:44:57.315947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.623586+0000","last_change":"2024-09-21T07:48:47.617745+0000","last_active":"2024-09-21T07:48:47.623586+0000","last_peered":"2024-09-21T07:48:47.623586+0000","last_clean":"2024-09-21T07:48:47.623586+0000","last_became_active":"2024-09-21T07:48:47.614030+0000","last_became_peered":"2024-09-21T07:48:47.614030+0000","last_unstale":"2024-09-21T07:48:47.623586+0000","last_undegraded":"2024-09-21T07:48:47.623586+0000","last_fullsized":"2024-09-21T07:48:47.623586+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:44:50.001056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.631705+0000","last_change":"2024-09-21T07:48:47.621852+0000","last_active":"2024-09-21T07:48:47.631705+0000","last_peered":"2024-09-21T07:48:47.631705+0000","last_clean":"2024-09-21T07:48:47.631705+0000","last_became_active":"2024-09-21T07:48:47.617439+0000","last_became_peered":"2024-09-21T07:48:47.617439+0000","last_unstale":"2024-09-21T07:48:47.631705+0000","last_undegraded":"2024-09-21T07:48:47.631705+0000","last_fullsized":"2024-09-21T07:48:47.631705+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:17:23.080659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.632943+0000","last_change":"2024-09-21T07:48:47.632098+0000","last_active":"2024-09-21T07:48:47.632943+0000","last_peered":"2024-09-21T07:48:47.632943+0000","last_clean":"2024-09-21T07:48:47.632943+0000","last_became_active":"2024-09-21T07:48:47.631802+0000","last_became_peered":"2024-09-21T07:48:47.631802+0000","last_unstale":"2024-09-21T07:48:47.632943+0000","last_undegraded":"2024-09-21T07:48:47.632943+0000","last_fullsized":"2024-09-21T07:48:47.632943+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:21.715374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.621185+0000","last_change":"2024-09-21T07:48:47.617229+0000","last_active":"2024-09-21T07:48:47.621185+0000","last_peered":"2024-09-21T07:48:47.621185+0000","last_clean":"2024-09-21T07:48:47.621185+0000","last_became_active":"2024-09-21T07:48:47.616863+0000","last_became_peered":"2024-09-21T07:48:47.616863+0000","last_unstale":"2024-09-21T07:48:47.621185+0000","last_undegraded":"2024-09-21T07:48:47.621185+0000","last_fullsized":"2024-09-21T07:48:47.621185+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:51.375781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.599782+0000","last_change":"2024-09-21T07:48:46.599782+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.599782+0000","last_undegraded":"2024-09-21T07:48:46.599782+0000","last_fullsized":"2024-09-21T07:48:46.599782+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.632509+0000","last_change":"2024-09-21T07:48:47.623337+0000","last_active":"2024-09-21T07:48:47.632509+0000","last_peered":"2024-09-21T07:48:47.632509+0000","last_clean":"2024-09-21T07:48:47.632509+0000","last_became_active":"2024-09-21T07:48:47.620228+0000","last_became_peered":"2024-09-21T07:48:47.620228+0000","last_unstale":"2024-09-21T07:48:47.632509+0000","last_undegraded":"2024-09-21T07:48:47.632509+0000","last_fullsized":"2024-09-21T07:48:47.632509+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:31.930727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051089+0000","last_change":"2024-09-21T07:48:47.627205+0000","last_active":"2024-09-21T07:48:50.051089+0000","last_peered":"2024-09-21T07:48:50.051089+0000","last_clean":"2024-09-21T07:48:50.051089+0000","last_became_active":"2024-09-21T07:48:47.626871+0000","last_became_peered":"2024-09-21T07:48:47.626871+0000","last_unstale":"2024-09-21T07:48:50.051089+0000","last_undegraded":"2024-09-21T07:48:50.051089+0000","last_fullsized":"2024-09-21T07:48:50.051089+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:57.118108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.629854+0000","last_change":"2024-09-21T07:48:47.627220+0000","last_active":"2024-09-21T07:48:47.629854+0000","last_peered":"2024-09-21T07:48:47.629854+0000","last_clean":"2024-09-21T07:48:47.629854+0000","last_became_active":"2024-09-21T07:48:47.626875+0000","last_became_peered":"2024-09-21T07:48:47.626875+0000","last_unstale":"2024-09-21T07:48:47.629854+0000","last_undegraded":"2024-09-21T07:48:47.629854+0000","last_fullsized":"2024-09-21T07:48:47.629854+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:11.801442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.628849+0000","last_change":"2024-09-21T07:48:47.625920+0000","last_active":"2024-09-21T07:48:47.628849+0000","last_peered":"2024-09-21T07:48:47.628849+0000","last_clean":"2024-09-21T07:48:47.628849+0000","last_became_active":"2024-09-21T07:48:47.625605+0000","last_became_peered":"2024-09-21T07:48:47.625605+0000","last_unstale":"2024-09-21T07:48:47.628849+0000","last_undegraded":"2024-09-21T07:48:47.628849+0000","last_fullsized":"2024-09-21T07:48:47.628849+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:17:04.946971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.629605+0000","last_change":"2024-09-21T07:48:47.628044+0000","last_active":"2024-09-21T07:48:47.629605+0000","last_peered":"2024-09-21T07:48:47.629605+0000","last_clean":"2024-09-21T07:48:47.629605+0000","last_became_active":"2024-09-21T07:48:47.626515+0000","last_became_peered":"2024-09-21T07:48:47.626515+0000","last_unstale":"2024-09-21T07:48:47.629605+0000","last_undegraded":"2024-09-21T07:48:47.629605+0000","last_fullsized":"2024-09-21T07:48:47.629605+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:40.234280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.632098+0000","last_change":"2024-09-21T07:48:47.623935+0000","last_active":"2024-09-21T07:48:47.632098+0000","last_peered":"2024-09-21T07:48:47.632098+0000","last_clean":"2024-09-21T07:48:47.632098+0000","last_became_active":"2024-09-21T07:48:47.623233+0000","last_became_peered":"2024-09-21T07:48:47.623233+0000","last_unstale":"2024-09-21T07:48:47.632098+0000","last_undegraded":"2024-09-21T07:48:47.632098+0000","last_fullsized":"2024-09-21T07:48:47.632098+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:29.332356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.630373+0000","last_change":"2024-09-21T07:48:47.617959+0000","last_active":"2024-09-21T07:48:47.630373+0000","last_peered":"2024-09-21T07:48:47.630373+0000","last_clean":"2024-09-21T07:48:47.630373+0000","last_became_active":"2024-09-21T07:48:47.617595+0000","last_became_peered":"2024-09-21T07:48:47.617595+0000","last_unstale":"2024-09-21T07:48:47.630373+0000","last_undegraded":"2024-09-21T07:48:47.630373+0000","last_fullsized":"2024-09-21T07:48:47.630373+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:16.164291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.631855+0000","last_change":"2024-09-21T07:48:47.621502+0000","last_active":"2024-09-21T07:48:47.631855+0000","last_peered":"2024-09-21T07:48:47.631855+0000","last_clean":"2024-09-21T07:48:47.631855+0000","last_became_active":"2024-09-21T07:48:47.616971+0000","last_became_peered":"2024-09-21T07:48:47.616971+0000","last_unstale":"2024-09-21T07:48:47.631855+0000","last_undegraded":"2024-09-21T07:48:47.631855+0000","last_fullsized":"2024-09-21T07:48:47.631855+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:59.711618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.631967+0000","last_change":"2024-09-21T07:48:47.622917+0000","last_active":"2024-09-21T07:48:47.631967+0000","last_peered":"2024-09-21T07:48:47.631967+0000","last_clean":"2024-09-21T07:48:47.631967+0000","last_became_active":"2024-09-21T07:48:47.619537+0000","last_became_peered":"2024-09-21T07:48:47.619537+0000","last_unstale":"2024-09-21T07:48:47.631967+0000","last_undegraded":"2024-09-21T07:48:47.631967+0000","last_fullsized":"2024-09-21T07:48:47.631967+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:14:17.772422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.610863+0000","last_change":"2024-09-21T07:48:47.598763+0000","last_active":"2024-09-21T07:48:48.610863+0000","last_peered":"2024-09-21T07:48:48.610863+0000","last_clean":"2024-09-21T07:48:48.610863+0000","last_became_active":"2024-09-21T07:48:47.598446+0000","last_became_peered":"2024-09-21T07:48:47.598446+0000","last_unstale":"2024-09-21T07:48:48.610863+0000","last_undegraded":"2024-09-21T07:48:48.610863+0000","last_fullsized":"2024-09-21T07:48:48.610863+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:48.912495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.607435+0000","last_change":"2024-09-21T07:48:47.596907+0000","last_active":"2024-09-21T07:48:47.607435+0000","last_peered":"2024-09-21T07:48:47.607435+0000","last_clean":"2024-09-21T07:48:47.607435+0000","last_became_active":"2024-09-21T07:48:47.596568+0000","last_became_peered":"2024-09-21T07:48:47.596568+0000","last_unstale":"2024-09-21T07:48:47.607435+0000","last_undegraded":"2024-09-21T07:48:47.607435+0000","last_fullsized":"2024-09-21T07:48:47.607435+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:47.548875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.623612+0000","last_change":"2024-09-21T07:48:47.620026+0000","last_active":"2024-09-21T07:48:47.623612+0000","last_peered":"2024-09-21T07:48:47.623612+0000","last_clean":"2024-09-21T07:48:47.623612+0000","last_became_active":"2024-09-21T07:48:47.619713+0000","last_became_peered":"2024-09-21T07:48:47.619713+0000","last_unstale":"2024-09-21T07:48:47.623612+0000","last_undegraded":"2024-09-21T07:48:47.623612+0000","last_fullsized":"2024-09-21T07:48:47.623612+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:15.278604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.592481+0000","last_change":"2024-09-21T07:48:46.592481+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.592481+0000","last_undegraded":"2024-09-21T07:48:46.592481+0000","last_fullsized":"2024-09-21T07:48:46.592481+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.627562+0000","last_change":"2024-09-21T07:48:47.625275+0000","last_active":"2024-09-21T07:48:47.627562+0000","last_peered":"2024-09-21T07:48:47.627562+0000","last_clean":"2024-09-21T07:48:47.627562+0000","last_became_active":"2024-09-21T07:48:47.624985+0000","last_became_peered":"2024-09-21T07:48:47.624985+0000","last_unstale":"2024-09-21T07:48:47.627562+0000","last_undegraded":"2024-09-21T07:48:47.627562+0000","last_fullsized":"2024-09-21T07:48:47.627562+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:22.189614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.612236+0000","last_change":"2024-09-21T07:48:47.608774+0000","last_active":"2024-09-21T07:48:47.612236+0000","last_peered":"2024-09-21T07:48:47.612236+0000","last_clean":"2024-09-21T07:48:47.612236+0000","last_became_active":"2024-09-21T07:48:47.608457+0000","last_became_peered":"2024-09-21T07:48:47.608457+0000","last_unstale":"2024-09-21T07:48:47.612236+0000","last_undegraded":"2024-09-21T07:48:47.612236+0000","last_fullsized":"2024-09-21T07:48:47.612236+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:25.633315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.613015+0000","last_change":"2024-09-21T07:48:47.605730+0000","last_active":"2024-09-21T07:48:47.613015+0000","last_peered":"2024-09-21T07:48:47.613015+0000","last_clean":"2024-09-21T07:48:47.613015+0000","last_became_active":"2024-09-21T07:48:47.605433+0000","last_became_peered":"2024-09-21T07:48:47.605433+0000","last_unstale":"2024-09-21T07:48:47.613015+0000","last_undegraded":"2024-09-21T07:48:47.613015+0000","last_fullsized":"2024-09-21T07:48:47.613015+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:15.258575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051126+0000","last_change":"2024-09-21T07:48:47.604557+0000","last_active":"2024-09-21T07:48:50.051126+0000","last_peered":"2024-09-21T07:48:50.051126+0000","last_clean":"2024-09-21T07:48:50.051126+0000","last_became_active":"2024-09-21T07:48:47.604196+0000","last_became_peered":"2024-09-21T07:48:47.604196+0000","last_unstale":"2024-09-21T07:48:50.051126+0000","last_undegraded":"2024-09-21T07:48:50.051126+0000","last_fullsized":"2024-09-21T07:48:50.051126+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:56.323906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.595955+0000","last_change":"2024-09-21T07:48:46.595955+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.595955+0000","last_undegraded":"2024-09-21T07:48:46.595955+0000","last_fullsized":"2024-09-21T07:48:46.595955+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.595810+0000","last_change":"2024-09-21T07:48:46.595810+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.595810+0000","last_undegraded":"2024-09-21T07:48:46.595810+0000","last_fullsized":"2024-09-21T07:48:46.595810+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.591446+0000","last_change":"2024-09-21T07:48:46.591446+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.591446+0000","last_undegraded":"2024-09-21T07:48:46.591446+0000","last_fullsized":"2024-09-21T07:48:46.591446+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1a","version":"218'1","reported_seq":14,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.622300+0000","last_change":"2024-09-21T07:48:46.603075+0000","last_active":"2024-09-21T07:48:47.622300+0000","last_peered":"2024-09-21T07:48:47.622300+0000","last_clean":"2024-09-21T07:48:47.622300+0000","last_became_active":"2024-09-21T07:48:46.602882+0000","last_became_peered":"2024-09-21T07:48:46.602882+0000","last_unstale":"2024-09-21T07:48:47.622300+0000","last_undegraded":"2024-09-21T07:48:47.622300+0000","last_fullsized":"2024-09-21T07:48:47.622300+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:49.025973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":301,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.602282+0000","last_change":"2024-09-21T07:35:27.879605+0000","last_active":"2024-09-21T07:48:46.602282+0000","last_peered":"2024-09-21T07:48:46.602282+0000","last_clean":"2024-09-21T07:48:46.602282+0000","last_became_active":"2024-09-21T07:35:27.879274+0000","last_became_peered":"2024-09-21T07:35:27.879274+0000","last_unstale":"2024-09-21T07:48:46.602282+0000","last_undegraded":"2024-09-21T07:48:46.602282+0000","last_fullsized":"2024-09-21T07:48:46.602282+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:24.241636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":316,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609470+0000","last_change":"2024-09-21T07:35:29.491336+0000","last_active":"2024-09-21T07:48:48.609470+0000","last_peered":"2024-09-21T07:48:48.609470+0000","last_clean":"2024-09-21T07:48:48.609470+0000","last_became_active":"2024-09-21T07:35:26.866484+0000","last_became_peered":"2024-09-21T07:35:26.866484+0000","last_unstale":"2024-09-21T07:48:48.609470+0000","last_undegraded":"2024-09-21T07:48:48.609470+0000","last_fullsized":"2024-09-21T07:48:48.609470+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:10.449665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076010100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":487,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.596417+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:48:46.596417+0000","last_peered":"2024-09-21T07:48:46.596417+0000","last_clean":"2024-09-21T07:48:46.596417+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:48:46.596417+0000","last_undegraded":"2024-09-21T07:48:46.596417+0000","last_fullsized":"2024-09-21T07:48:46.596417+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":375,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.590586+0000","last_change":"2024-09-21T07:34:24.539248+0000","last_active":"2024-09-21T07:48:46.590586+0000","last_peered":"2024-09-21T07:48:46.590586+0000","last_clean":"2024-09-21T07:48:46.590586+0000","last_became_active":"2024-09-21T07:34:23.273465+0000","last_became_peered":"2024-09-21T07:34:23.273465+0000","last_unstale":"2024-09-21T07:48:46.590586+0000","last_undegraded":"2024-09-21T07:48:46.590586+0000","last_fullsized":"2024-09-21T07:48:46.590586+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:55.517452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060627700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608458+0000","last_change":"2024-09-21T07:48:47.612024+0000","last_active":"2024-09-21T07:48:48.608458+0000","last_peered":"2024-09-21T07:48:48.608458+0000","last_clean":"2024-09-21T07:48:48.608458+0000","last_became_active":"2024-09-21T07:48:47.601212+0000","last_became_peered":"2024-09-21T07:48:47.601212+0000","last_unstale":"2024-09-21T07:48:48.608458+0000","last_undegraded":"2024-09-21T07:48:48.608458+0000","last_fullsized":"2024-09-21T07:48:48.608458+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:20.715873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"218'4","reported_seq":19,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051794+0000","last_change":"2024-09-21T07:48:46.613621+0000","last_active":"2024-09-21T07:48:50.051794+0000","last_peered":"2024-09-21T07:48:50.051794+0000","last_clean":"2024-09-21T07:48:50.051794+0000","last_became_active":"2024-09-21T07:48:46.613266+0000","last_became_peered":"2024-09-21T07:48:46.613266+0000","last_unstale":"2024-09-21T07:48:50.051794+0000","last_undegraded":"2024-09-21T07:48:50.051794+0000","last_fullsized":"2024-09-21T07:48:50.051794+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:40.639085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":309,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608414+0000","last_change":"2024-09-21T07:35:27.864492+0000","last_active":"2024-09-21T07:48:48.608414+0000","last_peered":"2024-09-21T07:48:48.608414+0000","last_clean":"2024-09-21T07:48:48.608414+0000","last_became_active":"2024-09-21T07:35:27.864176+0000","last_became_peered":"2024-09-21T07:35:27.864176+0000","last_unstale":"2024-09-21T07:48:48.608414+0000","last_undegraded":"2024-09-21T07:48:48.608414+0000","last_fullsized":"2024-09-21T07:48:48.608414+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:05.153467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":308,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.602895+0000","last_change":"2024-09-21T07:35:29.493678+0000","last_active":"2024-09-21T07:48:46.602895+0000","last_peered":"2024-09-21T07:48:46.602895+0000","last_clean":"2024-09-21T07:48:46.602895+0000","last_became_active":"2024-09-21T07:35:26.870749+0000","last_became_peered":"2024-09-21T07:35:26.870749+0000","last_unstale":"2024-09-21T07:48:46.602895+0000","last_undegraded":"2024-09-21T07:48:46.602895+0000","last_fullsized":"2024-09-21T07:48:46.602895+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:55.367877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010159570000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":359,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051441+0000","last_change":"2024-09-21T07:34:24.597274+0000","last_active":"2024-09-21T07:48:50.051441+0000","last_peered":"2024-09-21T07:48:50.051441+0000","last_clean":"2024-09-21T07:48:50.051441+0000","last_became_active":"2024-09-21T07:34:23.279560+0000","last_became_peered":"2024-09-21T07:34:23.279560+0000","last_unstale":"2024-09-21T07:48:50.051441+0000","last_undegraded":"2024-09-21T07:48:50.051441+0000","last_fullsized":"2024-09-21T07:48:50.051441+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:15.845265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059699799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.588781+0000","last_change":"2024-09-21T07:48:46.588781+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.588781+0000","last_undegraded":"2024-09-21T07:48:46.588781+0000","last_fullsized":"2024-09-21T07:48:46.588781+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.613288+0000","last_change":"2024-09-21T07:48:46.610499+0000","last_active":"2024-09-21T07:48:46.613288+0000","last_peered":"2024-09-21T07:48:46.613288+0000","last_clean":"2024-09-21T07:48:46.613288+0000","last_became_active":"2024-09-21T07:48:46.610137+0000","last_became_peered":"2024-09-21T07:48:46.610137+0000","last_unstale":"2024-09-21T07:48:46.613288+0000","last_undegraded":"2024-09-21T07:48:46.613288+0000","last_fullsized":"2024-09-21T07:48:46.613288+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:29:58.572619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":308,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.592520+0000","last_change":"2024-09-21T07:35:27.874387+0000","last_active":"2024-09-21T07:48:47.592520+0000","last_peered":"2024-09-21T07:48:47.592520+0000","last_clean":"2024-09-21T07:48:47.592520+0000","last_became_active":"2024-09-21T07:35:27.873277+0000","last_became_peered":"2024-09-21T07:35:27.873277+0000","last_unstale":"2024-09-21T07:48:47.592520+0000","last_undegraded":"2024-09-21T07:48:47.592520+0000","last_fullsized":"2024-09-21T07:48:47.592520+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:04.581463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":309,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051894+0000","last_change":"2024-09-21T07:35:29.387094+0000","last_active":"2024-09-21T07:48:50.051894+0000","last_peered":"2024-09-21T07:48:50.051894+0000","last_clean":"2024-09-21T07:48:50.051894+0000","last_became_active":"2024-09-21T07:35:26.874757+0000","last_became_peered":"2024-09-21T07:35:26.874757+0000","last_unstale":"2024-09-21T07:48:50.051894+0000","last_undegraded":"2024-09-21T07:48:50.051894+0000","last_fullsized":"2024-09-21T07:48:50.051894+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:33.542750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061065700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":361,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.604051+0000","last_change":"2024-09-21T07:34:24.259906+0000","last_active":"2024-09-21T07:48:46.604051+0000","last_peered":"2024-09-21T07:48:46.604051+0000","last_clean":"2024-09-21T07:48:46.604051+0000","last_became_active":"2024-09-21T07:34:23.635282+0000","last_became_peered":"2024-09-21T07:34:23.635282+0000","last_unstale":"2024-09-21T07:48:46.604051+0000","last_undegraded":"2024-09-21T07:48:46.604051+0000","last_fullsized":"2024-09-21T07:48:46.604051+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:00.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001527664,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.632254+0000","last_change":"2024-09-21T07:48:47.622852+0000","last_active":"2024-09-21T07:48:47.632254+0000","last_peered":"2024-09-21T07:48:47.632254+0000","last_clean":"2024-09-21T07:48:47.632254+0000","last_became_active":"2024-09-21T07:48:47.619895+0000","last_became_peered":"2024-09-21T07:48:47.619895+0000","last_unstale":"2024-09-21T07:48:47.632254+0000","last_undegraded":"2024-09-21T07:48:47.632254+0000","last_fullsized":"2024-09-21T07:48:47.632254+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:56:05.060203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"218'2","reported_seq":17,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609737+0000","last_change":"2024-09-21T07:48:46.606441+0000","last_active":"2024-09-21T07:48:48.609737+0000","last_peered":"2024-09-21T07:48:48.609737+0000","last_clean":"2024-09-21T07:48:48.609737+0000","last_became_active":"2024-09-21T07:48:46.604712+0000","last_became_peered":"2024-09-21T07:48:46.604712+0000","last_unstale":"2024-09-21T07:48:48.609737+0000","last_undegraded":"2024-09-21T07:48:48.609737+0000","last_fullsized":"2024-09-21T07:48:48.609737+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:22.700415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":308,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.593732+0000","last_change":"2024-09-21T07:35:27.873645+0000","last_active":"2024-09-21T07:48:47.593732+0000","last_peered":"2024-09-21T07:48:47.593732+0000","last_clean":"2024-09-21T07:48:47.593732+0000","last_became_active":"2024-09-21T07:35:27.873245+0000","last_became_peered":"2024-09-21T07:35:27.873245+0000","last_unstale":"2024-09-21T07:48:47.593732+0000","last_undegraded":"2024-09-21T07:48:47.593732+0000","last_fullsized":"2024-09-21T07:48:47.593732+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:55.090953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":314,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.592916+0000","last_change":"2024-09-21T07:35:28.861744+0000","last_active":"2024-09-21T07:48:47.592916+0000","last_peered":"2024-09-21T07:48:47.592916+0000","last_clean":"2024-09-21T07:48:47.592916+0000","last_became_active":"2024-09-21T07:35:26.876462+0000","last_became_peered":"2024-09-21T07:35:26.876462+0000","last_unstale":"2024-09-21T07:48:47.592916+0000","last_undegraded":"2024-09-21T07:48:47.592916+0000","last_fullsized":"2024-09-21T07:48:47.592916+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:48.632568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012020920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":363,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.594772+0000","last_change":"2024-09-21T07:34:25.085454+0000","last_active":"2024-09-21T07:48:47.594772+0000","last_peered":"2024-09-21T07:48:47.594772+0000","last_clean":"2024-09-21T07:48:47.594772+0000","last_became_active":"2024-09-21T07:34:23.279205+0000","last_became_peered":"2024-09-21T07:34:23.279205+0000","last_unstale":"2024-09-21T07:48:47.594772+0000","last_undegraded":"2024-09-21T07:48:47.594772+0000","last_fullsized":"2024-09-21T07:48:47.594772+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:54.231666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095320599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.614070+0000","last_change":"2024-09-21T07:48:47.610088+0000","last_active":"2024-09-21T07:48:47.614070+0000","last_peered":"2024-09-21T07:48:47.614070+0000","last_clean":"2024-09-21T07:48:47.614070+0000","last_became_active":"2024-09-21T07:48:47.607791+0000","last_became_peered":"2024-09-21T07:48:47.607791+0000","last_unstale":"2024-09-21T07:48:47.614070+0000","last_undegraded":"2024-09-21T07:48:47.614070+0000","last_fullsized":"2024-09-21T07:48:47.614070+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:03.037375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"218'2","reported_seq":15,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.615890+0000","last_change":"2024-09-21T07:48:46.602844+0000","last_active":"2024-09-21T07:48:47.615890+0000","last_peered":"2024-09-21T07:48:47.615890+0000","last_clean":"2024-09-21T07:48:47.615890+0000","last_became_active":"2024-09-21T07:48:46.602544+0000","last_became_peered":"2024-09-21T07:48:46.602544+0000","last_unstale":"2024-09-21T07:48:47.615890+0000","last_undegraded":"2024-09-21T07:48:47.615890+0000","last_fullsized":"2024-09-21T07:48:47.615890+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:36:36.719598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":308,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609553+0000","last_change":"2024-09-21T07:35:27.861373+0000","last_active":"2024-09-21T07:48:48.609553+0000","last_peered":"2024-09-21T07:48:48.609553+0000","last_clean":"2024-09-21T07:48:48.609553+0000","last_became_active":"2024-09-21T07:35:27.861016+0000","last_became_peered":"2024-09-21T07:35:27.861016+0000","last_unstale":"2024-09-21T07:48:48.609553+0000","last_undegraded":"2024-09-21T07:48:48.609553+0000","last_fullsized":"2024-09-21T07:48:48.609553+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:56.043763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":313,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.599014+0000","last_change":"2024-09-21T07:35:29.053677+0000","last_active":"2024-09-21T07:48:47.599014+0000","last_peered":"2024-09-21T07:48:47.599014+0000","last_clean":"2024-09-21T07:48:47.599014+0000","last_became_active":"2024-09-21T07:35:26.875496+0000","last_became_peered":"2024-09-21T07:35:26.875496+0000","last_unstale":"2024-09-21T07:48:47.599014+0000","last_undegraded":"2024-09-21T07:48:47.599014+0000","last_fullsized":"2024-09-21T07:48:47.599014+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:04.850784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065859000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":365,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.607231+0000","last_change":"2024-09-21T07:34:25.028521+0000","last_active":"2024-09-21T07:48:47.607231+0000","last_peered":"2024-09-21T07:48:47.607231+0000","last_clean":"2024-09-21T07:48:47.607231+0000","last_became_active":"2024-09-21T07:34:23.635467+0000","last_became_peered":"2024-09-21T07:34:23.635467+0000","last_unstale":"2024-09-21T07:48:47.607231+0000","last_undegraded":"2024-09-21T07:48:47.607231+0000","last_fullsized":"2024-09-21T07:48:47.607231+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:33.729123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000660558,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051543+0000","last_change":"2024-09-21T07:48:47.616627+0000","last_active":"2024-09-21T07:48:50.051543+0000","last_peered":"2024-09-21T07:48:50.051543+0000","last_clean":"2024-09-21T07:48:50.051543+0000","last_became_active":"2024-09-21T07:48:47.616246+0000","last_became_peered":"2024-09-21T07:48:47.616246+0000","last_unstale":"2024-09-21T07:48:50.051543+0000","last_undegraded":"2024-09-21T07:48:50.051543+0000","last_fullsized":"2024-09-21T07:48:50.051543+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:07.746423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.615861+0000","last_change":"2024-09-21T07:48:46.614962+0000","last_active":"2024-09-21T07:48:46.615861+0000","last_peered":"2024-09-21T07:48:46.615861+0000","last_clean":"2024-09-21T07:48:46.615861+0000","last_became_active":"2024-09-21T07:48:46.614616+0000","last_became_peered":"2024-09-21T07:48:46.614616+0000","last_unstale":"2024-09-21T07:48:46.615861+0000","last_undegraded":"2024-09-21T07:48:46.615861+0000","last_fullsized":"2024-09-21T07:48:46.615861+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:41:33.005009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":304,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.599104+0000","last_change":"2024-09-21T07:35:27.871201+0000","last_active":"2024-09-21T07:48:47.599104+0000","last_peered":"2024-09-21T07:48:47.599104+0000","last_clean":"2024-09-21T07:48:47.599104+0000","last_became_active":"2024-09-21T07:35:27.870816+0000","last_became_peered":"2024-09-21T07:35:27.870816+0000","last_unstale":"2024-09-21T07:48:47.599104+0000","last_undegraded":"2024-09-21T07:48:47.599104+0000","last_fullsized":"2024-09-21T07:48:47.599104+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:04.787450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":315,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.610027+0000","last_change":"2024-09-21T07:35:29.491874+0000","last_active":"2024-09-21T07:48:48.610027+0000","last_peered":"2024-09-21T07:48:48.610027+0000","last_clean":"2024-09-21T07:48:48.610027+0000","last_became_active":"2024-09-21T07:35:26.866381+0000","last_became_peered":"2024-09-21T07:35:26.866381+0000","last_unstale":"2024-09-21T07:48:48.610027+0000","last_undegraded":"2024-09-21T07:48:48.610027+0000","last_fullsized":"2024-09-21T07:48:48.610027+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:36.404647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073235399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":370,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.610052+0000","last_change":"2024-09-21T07:34:24.260046+0000","last_active":"2024-09-21T07:48:48.610052+0000","last_peered":"2024-09-21T07:48:48.610052+0000","last_clean":"2024-09-21T07:48:48.610052+0000","last_became_active":"2024-09-21T07:34:23.273431+0000","last_became_peered":"2024-09-21T07:34:23.273431+0000","last_unstale":"2024-09-21T07:48:48.610052+0000","last_undegraded":"2024-09-21T07:48:48.610052+0000","last_fullsized":"2024-09-21T07:48:48.610052+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:18.876057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019803310000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051791+0000","last_change":"2024-09-21T07:48:47.605350+0000","last_active":"2024-09-21T07:48:50.051791+0000","last_peered":"2024-09-21T07:48:50.051791+0000","last_clean":"2024-09-21T07:48:50.051791+0000","last_became_active":"2024-09-21T07:48:47.605046+0000","last_became_peered":"2024-09-21T07:48:47.605046+0000","last_unstale":"2024-09-21T07:48:50.051791+0000","last_undegraded":"2024-09-21T07:48:50.051791+0000","last_fullsized":"2024-09-21T07:48:50.051791+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:07.906313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.611056+0000","last_change":"2024-09-21T07:48:46.607052+0000","last_active":"2024-09-21T07:48:46.611056+0000","last_peered":"2024-09-21T07:48:46.611056+0000","last_clean":"2024-09-21T07:48:46.611056+0000","last_became_active":"2024-09-21T07:48:46.606707+0000","last_became_peered":"2024-09-21T07:48:46.606707+0000","last_unstale":"2024-09-21T07:48:46.611056+0000","last_undegraded":"2024-09-21T07:48:46.611056+0000","last_fullsized":"2024-09-21T07:48:46.611056+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:29:39.188621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":304,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.603501+0000","last_change":"2024-09-21T07:35:27.871736+0000","last_active":"2024-09-21T07:48:47.603501+0000","last_peered":"2024-09-21T07:48:47.603501+0000","last_clean":"2024-09-21T07:48:47.603501+0000","last_became_active":"2024-09-21T07:35:27.871232+0000","last_became_peered":"2024-09-21T07:35:27.871232+0000","last_unstale":"2024-09-21T07:48:47.603501+0000","last_undegraded":"2024-09-21T07:48:47.603501+0000","last_fullsized":"2024-09-21T07:48:47.603501+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:04.419301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":316,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608800+0000","last_change":"2024-09-21T07:35:29.491246+0000","last_active":"2024-09-21T07:48:48.608800+0000","last_peered":"2024-09-21T07:48:48.608800+0000","last_clean":"2024-09-21T07:48:48.608800+0000","last_became_active":"2024-09-21T07:35:26.866783+0000","last_became_peered":"2024-09-21T07:35:26.866783+0000","last_unstale":"2024-09-21T07:48:48.608800+0000","last_undegraded":"2024-09-21T07:48:48.608800+0000","last_fullsized":"2024-09-21T07:48:48.608800+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:28.881914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098474099999999992,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":365,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.599139+0000","last_change":"2024-09-21T07:34:25.028640+0000","last_active":"2024-09-21T07:48:47.599139+0000","last_peered":"2024-09-21T07:48:47.599139+0000","last_clean":"2024-09-21T07:48:47.599139+0000","last_became_active":"2024-09-21T07:34:23.260950+0000","last_became_peered":"2024-09-21T07:34:23.260950+0000","last_unstale":"2024-09-21T07:48:47.599139+0000","last_undegraded":"2024-09-21T07:48:47.599139+0000","last_fullsized":"2024-09-21T07:48:47.599139+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:23:42.121398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055135099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.623528+0000","last_change":"2024-09-21T07:48:47.614858+0000","last_active":"2024-09-21T07:48:47.623528+0000","last_peered":"2024-09-21T07:48:47.623528+0000","last_clean":"2024-09-21T07:48:47.623528+0000","last_became_active":"2024-09-21T07:48:47.614484+0000","last_became_peered":"2024-09-21T07:48:47.614484+0000","last_unstale":"2024-09-21T07:48:47.623528+0000","last_undegraded":"2024-09-21T07:48:47.623528+0000","last_fullsized":"2024-09-21T07:48:47.623528+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:24.141015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":13,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.603705+0000","last_change":"2024-09-21T07:48:46.607151+0000","last_active":"2024-09-21T07:48:47.603705+0000","last_peered":"2024-09-21T07:48:47.603705+0000","last_clean":"2024-09-21T07:48:47.603705+0000","last_became_active":"2024-09-21T07:48:46.606839+0000","last_became_peered":"2024-09-21T07:48:46.606839+0000","last_unstale":"2024-09-21T07:48:47.603705+0000","last_undegraded":"2024-09-21T07:48:47.603705+0000","last_fullsized":"2024-09-21T07:48:47.603705+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:34.780363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":305,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.609983+0000","last_change":"2024-09-21T07:35:27.872027+0000","last_active":"2024-09-21T07:48:47.609983+0000","last_peered":"2024-09-21T07:48:47.609983+0000","last_clean":"2024-09-21T07:48:47.609983+0000","last_became_active":"2024-09-21T07:35:27.870426+0000","last_became_peered":"2024-09-21T07:35:27.870426+0000","last_unstale":"2024-09-21T07:48:47.609983+0000","last_undegraded":"2024-09-21T07:48:47.609983+0000","last_fullsized":"2024-09-21T07:48:47.609983+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":316,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608519+0000","last_change":"2024-09-21T07:35:29.491008+0000","last_active":"2024-09-21T07:48:48.608519+0000","last_peered":"2024-09-21T07:48:48.608519+0000","last_clean":"2024-09-21T07:48:48.608519+0000","last_became_active":"2024-09-21T07:35:26.866750+0000","last_became_peered":"2024-09-21T07:35:26.866750+0000","last_unstale":"2024-09-21T07:48:48.608519+0000","last_undegraded":"2024-09-21T07:48:48.608519+0000","last_fullsized":"2024-09-21T07:48:48.608519+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:25.490406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083529700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":362,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.602630+0000","last_change":"2024-09-21T07:34:24.260665+0000","last_active":"2024-09-21T07:48:46.602630+0000","last_peered":"2024-09-21T07:48:46.602630+0000","last_clean":"2024-09-21T07:48:46.602630+0000","last_became_active":"2024-09-21T07:34:23.635909+0000","last_became_peered":"2024-09-21T07:34:23.635909+0000","last_unstale":"2024-09-21T07:48:46.602630+0000","last_undegraded":"2024-09-21T07:48:46.602630+0000","last_fullsized":"2024-09-21T07:48:46.602630+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:44.536885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051072+0000","last_change":"2024-09-21T07:48:47.615147+0000","last_active":"2024-09-21T07:48:50.051072+0000","last_peered":"2024-09-21T07:48:50.051072+0000","last_clean":"2024-09-21T07:48:50.051072+0000","last_became_active":"2024-09-21T07:48:47.614792+0000","last_became_peered":"2024-09-21T07:48:47.614792+0000","last_unstale":"2024-09-21T07:48:50.051072+0000","last_undegraded":"2024-09-21T07:48:50.051072+0000","last_fullsized":"2024-09-21T07:48:50.051072+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:23.043769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":13,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.593919+0000","last_change":"2024-09-21T07:48:46.614465+0000","last_active":"2024-09-21T07:48:47.593919+0000","last_peered":"2024-09-21T07:48:47.593919+0000","last_clean":"2024-09-21T07:48:47.593919+0000","last_became_active":"2024-09-21T07:48:46.614155+0000","last_became_peered":"2024-09-21T07:48:46.614155+0000","last_unstale":"2024-09-21T07:48:47.593919+0000","last_undegraded":"2024-09-21T07:48:47.593919+0000","last_fullsized":"2024-09-21T07:48:47.593919+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:23:53.595692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":303,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051021+0000","last_change":"2024-09-21T07:35:27.874772+0000","last_active":"2024-09-21T07:48:50.051021+0000","last_peered":"2024-09-21T07:48:50.051021+0000","last_clean":"2024-09-21T07:48:50.051021+0000","last_became_active":"2024-09-21T07:35:27.874434+0000","last_became_peered":"2024-09-21T07:35:27.874434+0000","last_unstale":"2024-09-21T07:48:50.051021+0000","last_undegraded":"2024-09-21T07:48:50.051021+0000","last_fullsized":"2024-09-21T07:48:50.051021+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:31.998755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":323,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.590456+0000","last_change":"2024-09-21T07:35:28.859153+0000","last_active":"2024-09-21T07:48:46.590456+0000","last_peered":"2024-09-21T07:48:46.590456+0000","last_clean":"2024-09-21T07:48:46.590456+0000","last_became_active":"2024-09-21T07:35:26.874428+0000","last_became_peered":"2024-09-21T07:35:26.874428+0000","last_unstale":"2024-09-21T07:48:46.590456+0000","last_undegraded":"2024-09-21T07:48:46.590456+0000","last_fullsized":"2024-09-21T07:48:46.590456+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:04.487621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048557299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":375,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.591699+0000","last_change":"2024-09-21T07:34:24.538845+0000","last_active":"2024-09-21T07:48:46.591699+0000","last_peered":"2024-09-21T07:48:46.591699+0000","last_clean":"2024-09-21T07:48:46.591699+0000","last_became_active":"2024-09-21T07:34:23.636372+0000","last_became_peered":"2024-09-21T07:34:23.636372+0000","last_unstale":"2024-09-21T07:48:46.591699+0000","last_undegraded":"2024-09-21T07:48:46.591699+0000","last_fullsized":"2024-09-21T07:48:46.591699+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:29.393813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841451,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.624317+0000","last_change":"2024-09-21T07:48:47.617117+0000","last_active":"2024-09-21T07:48:47.624317+0000","last_peered":"2024-09-21T07:48:47.624317+0000","last_clean":"2024-09-21T07:48:47.624317+0000","last_became_active":"2024-09-21T07:48:47.616813+0000","last_became_peered":"2024-09-21T07:48:47.616813+0000","last_unstale":"2024-09-21T07:48:47.624317+0000","last_undegraded":"2024-09-21T07:48:47.624317+0000","last_fullsized":"2024-09-21T07:48:47.624317+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:50.252324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.612150+0000","last_change":"2024-09-21T07:48:46.610433+0000","last_active":"2024-09-21T07:48:46.612150+0000","last_peered":"2024-09-21T07:48:46.612150+0000","last_clean":"2024-09-21T07:48:46.612150+0000","last_became_active":"2024-09-21T07:48:46.610123+0000","last_became_peered":"2024-09-21T07:48:46.610123+0000","last_unstale":"2024-09-21T07:48:46.612150+0000","last_undegraded":"2024-09-21T07:48:46.612150+0000","last_fullsized":"2024-09-21T07:48:46.612150+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:59.556565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":306,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.607367+0000","last_change":"2024-09-21T07:35:27.877801+0000","last_active":"2024-09-21T07:48:47.607367+0000","last_peered":"2024-09-21T07:48:47.607367+0000","last_clean":"2024-09-21T07:48:47.607367+0000","last_became_active":"2024-09-21T07:35:27.877464+0000","last_became_peered":"2024-09-21T07:35:27.877464+0000","last_unstale":"2024-09-21T07:48:47.607367+0000","last_undegraded":"2024-09-21T07:48:47.607367+0000","last_fullsized":"2024-09-21T07:48:47.607367+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:08.814929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":312,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.603961+0000","last_change":"2024-09-21T07:35:28.863732+0000","last_active":"2024-09-21T07:48:47.603961+0000","last_peered":"2024-09-21T07:48:47.603961+0000","last_clean":"2024-09-21T07:48:47.603961+0000","last_became_active":"2024-09-21T07:35:26.877392+0000","last_became_peered":"2024-09-21T07:35:26.877392+0000","last_unstale":"2024-09-21T07:48:47.603961+0000","last_undegraded":"2024-09-21T07:48:47.603961+0000","last_fullsized":"2024-09-21T07:48:47.603961+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071425700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":370,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.610101+0000","last_change":"2024-09-21T07:34:24.259686+0000","last_active":"2024-09-21T07:48:48.610101+0000","last_peered":"2024-09-21T07:48:48.610101+0000","last_clean":"2024-09-21T07:48:48.610101+0000","last_became_active":"2024-09-21T07:34:23.274405+0000","last_became_peered":"2024-09-21T07:34:23.274405+0000","last_unstale":"2024-09-21T07:48:48.610101+0000","last_undegraded":"2024-09-21T07:48:48.610101+0000","last_fullsized":"2024-09-21T07:48:48.610101+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:26.304693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022009920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.623488+0000","last_change":"2024-09-21T07:48:47.616194+0000","last_active":"2024-09-21T07:48:47.623488+0000","last_peered":"2024-09-21T07:48:47.623488+0000","last_clean":"2024-09-21T07:48:47.623488+0000","last_became_active":"2024-09-21T07:48:47.615876+0000","last_became_peered":"2024-09-21T07:48:47.615876+0000","last_unstale":"2024-09-21T07:48:47.623488+0000","last_undegraded":"2024-09-21T07:48:47.623488+0000","last_fullsized":"2024-09-21T07:48:47.623488+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:07.420806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"218'3","reported_seq":16,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.620054+0000","last_change":"2024-09-21T07:48:46.605473+0000","last_active":"2024-09-21T07:48:47.620054+0000","last_peered":"2024-09-21T07:48:47.620054+0000","last_clean":"2024-09-21T07:48:47.620054+0000","last_became_active":"2024-09-21T07:48:46.604548+0000","last_became_peered":"2024-09-21T07:48:46.604548+0000","last_unstale":"2024-09-21T07:48:47.620054+0000","last_undegraded":"2024-09-21T07:48:47.620054+0000","last_fullsized":"2024-09-21T07:48:47.620054+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:05.902786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":303,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051307+0000","last_change":"2024-09-21T07:35:27.867336+0000","last_active":"2024-09-21T07:48:50.051307+0000","last_peered":"2024-09-21T07:48:50.051307+0000","last_clean":"2024-09-21T07:48:50.051307+0000","last_became_active":"2024-09-21T07:35:27.867015+0000","last_became_peered":"2024-09-21T07:35:27.867015+0000","last_unstale":"2024-09-21T07:48:50.051307+0000","last_undegraded":"2024-09-21T07:48:50.051307+0000","last_fullsized":"2024-09-21T07:48:50.051307+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:59.214139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":309,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.602390+0000","last_change":"2024-09-21T07:35:29.493137+0000","last_active":"2024-09-21T07:48:46.602390+0000","last_peered":"2024-09-21T07:48:46.602390+0000","last_clean":"2024-09-21T07:48:46.602390+0000","last_became_active":"2024-09-21T07:35:26.872230+0000","last_became_peered":"2024-09-21T07:35:26.872230+0000","last_unstale":"2024-09-21T07:48:46.602390+0000","last_undegraded":"2024-09-21T07:48:46.602390+0000","last_fullsized":"2024-09-21T07:48:46.602390+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:24.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001197449,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":362,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.596253+0000","last_change":"2024-09-21T07:34:24.258754+0000","last_active":"2024-09-21T07:48:46.596253+0000","last_peered":"2024-09-21T07:48:46.596253+0000","last_clean":"2024-09-21T07:48:46.596253+0000","last_became_active":"2024-09-21T07:34:23.272657+0000","last_became_peered":"2024-09-21T07:34:23.272657+0000","last_unstale":"2024-09-21T07:48:46.596253+0000","last_undegraded":"2024-09-21T07:48:46.596253+0000","last_fullsized":"2024-09-21T07:48:46.596253+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:05.477099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067143200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.630257+0000","last_change":"2024-09-21T07:48:47.629237+0000","last_active":"2024-09-21T07:48:47.630257+0000","last_peered":"2024-09-21T07:48:47.630257+0000","last_clean":"2024-09-21T07:48:47.630257+0000","last_became_active":"2024-09-21T07:48:47.628900+0000","last_became_peered":"2024-09-21T07:48:47.628900+0000","last_unstale":"2024-09-21T07:48:47.630257+0000","last_undegraded":"2024-09-21T07:48:47.630257+0000","last_fullsized":"2024-09-21T07:48:47.630257+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:19.087720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.615052+0000","last_change":"2024-09-21T07:48:46.612660+0000","last_active":"2024-09-21T07:48:46.615052+0000","last_peered":"2024-09-21T07:48:46.615052+0000","last_clean":"2024-09-21T07:48:46.615052+0000","last_became_active":"2024-09-21T07:48:46.612362+0000","last_became_peered":"2024-09-21T07:48:46.612362+0000","last_unstale":"2024-09-21T07:48:46.615052+0000","last_undegraded":"2024-09-21T07:48:46.615052+0000","last_fullsized":"2024-09-21T07:48:46.615052+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:10.600575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":305,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.597247+0000","last_change":"2024-09-21T07:35:27.876021+0000","last_active":"2024-09-21T07:48:47.597247+0000","last_peered":"2024-09-21T07:48:47.597247+0000","last_clean":"2024-09-21T07:48:47.597247+0000","last_became_active":"2024-09-21T07:35:27.875724+0000","last_became_peered":"2024-09-21T07:35:27.875724+0000","last_unstale":"2024-09-21T07:48:47.597247+0000","last_undegraded":"2024-09-21T07:48:47.597247+0000","last_fullsized":"2024-09-21T07:48:47.597247+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:03.022812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":309,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.600781+0000","last_change":"2024-09-21T07:35:29.492570+0000","last_active":"2024-09-21T07:48:46.600781+0000","last_peered":"2024-09-21T07:48:46.600781+0000","last_clean":"2024-09-21T07:48:46.600781+0000","last_became_active":"2024-09-21T07:35:26.874316+0000","last_became_peered":"2024-09-21T07:35:26.874316+0000","last_unstale":"2024-09-21T07:48:46.600781+0000","last_undegraded":"2024-09-21T07:48:46.600781+0000","last_fullsized":"2024-09-21T07:48:46.600781+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:16.512323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011990760000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":363,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.596170+0000","last_change":"2024-09-21T07:34:25.085232+0000","last_active":"2024-09-21T07:48:47.596170+0000","last_peered":"2024-09-21T07:48:47.596170+0000","last_clean":"2024-09-21T07:48:47.596170+0000","last_became_active":"2024-09-21T07:34:23.277807+0000","last_became_peered":"2024-09-21T07:34:23.277807+0000","last_unstale":"2024-09-21T07:48:47.596170+0000","last_undegraded":"2024-09-21T07:48:47.596170+0000","last_fullsized":"2024-09-21T07:48:47.596170+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:36.362919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086619099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609167+0000","last_change":"2024-09-21T07:48:47.607795+0000","last_active":"2024-09-21T07:48:48.609167+0000","last_peered":"2024-09-21T07:48:48.609167+0000","last_clean":"2024-09-21T07:48:48.609167+0000","last_became_active":"2024-09-21T07:48:47.607511+0000","last_became_peered":"2024-09-21T07:48:47.607511+0000","last_unstale":"2024-09-21T07:48:48.609167+0000","last_undegraded":"2024-09-21T07:48:48.609167+0000","last_fullsized":"2024-09-21T07:48:48.609167+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:27.353767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"218'2","reported_seq":17,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609872+0000","last_change":"2024-09-21T07:48:46.609344+0000","last_active":"2024-09-21T07:48:48.609872+0000","last_peered":"2024-09-21T07:48:48.609872+0000","last_clean":"2024-09-21T07:48:48.609872+0000","last_became_active":"2024-09-21T07:48:46.606541+0000","last_became_peered":"2024-09-21T07:48:46.606541+0000","last_unstale":"2024-09-21T07:48:48.609872+0000","last_undegraded":"2024-09-21T07:48:48.609872+0000","last_fullsized":"2024-09-21T07:48:48.609872+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:36.274172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":306,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.606343+0000","last_change":"2024-09-21T07:35:27.872798+0000","last_active":"2024-09-21T07:48:47.606343+0000","last_peered":"2024-09-21T07:48:47.606343+0000","last_clean":"2024-09-21T07:48:47.606343+0000","last_became_active":"2024-09-21T07:35:27.872483+0000","last_became_peered":"2024-09-21T07:35:27.872483+0000","last_unstale":"2024-09-21T07:48:47.606343+0000","last_undegraded":"2024-09-21T07:48:47.606343+0000","last_fullsized":"2024-09-21T07:48:47.606343+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:11.933469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":313,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.605746+0000","last_change":"2024-09-21T07:35:29.206723+0000","last_active":"2024-09-21T07:48:47.605746+0000","last_peered":"2024-09-21T07:48:47.605746+0000","last_clean":"2024-09-21T07:48:47.605746+0000","last_became_active":"2024-09-21T07:35:26.872377+0000","last_became_peered":"2024-09-21T07:35:26.872377+0000","last_unstale":"2024-09-21T07:48:47.605746+0000","last_undegraded":"2024-09-21T07:48:47.605746+0000","last_fullsized":"2024-09-21T07:48:47.605746+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:29.086862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083077200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":359,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051990+0000","last_change":"2024-09-21T07:34:24.597672+0000","last_active":"2024-09-21T07:48:50.051990+0000","last_peered":"2024-09-21T07:48:50.051990+0000","last_clean":"2024-09-21T07:48:50.051990+0000","last_became_active":"2024-09-21T07:34:23.267770+0000","last_became_peered":"2024-09-21T07:34:23.267770+0000","last_unstale":"2024-09-21T07:48:50.051990+0000","last_undegraded":"2024-09-21T07:48:50.051990+0000","last_fullsized":"2024-09-21T07:48:50.051990+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:51.629476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086378500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.624444+0000","last_change":"2024-09-21T07:48:47.618502+0000","last_active":"2024-09-21T07:48:47.624444+0000","last_peered":"2024-09-21T07:48:47.624444+0000","last_clean":"2024-09-21T07:48:47.624444+0000","last_became_active":"2024-09-21T07:48:47.618213+0000","last_became_peered":"2024-09-21T07:48:47.618213+0000","last_unstale":"2024-09-21T07:48:47.624444+0000","last_undegraded":"2024-09-21T07:48:47.624444+0000","last_fullsized":"2024-09-21T07:48:47.624444+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:19.164379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":13,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.599459+0000","last_change":"2024-09-21T07:48:46.602566+0000","last_active":"2024-09-21T07:48:47.599459+0000","last_peered":"2024-09-21T07:48:47.599459+0000","last_clean":"2024-09-21T07:48:47.599459+0000","last_became_active":"2024-09-21T07:48:46.602254+0000","last_became_peered":"2024-09-21T07:48:46.602254+0000","last_unstale":"2024-09-21T07:48:47.599459+0000","last_undegraded":"2024-09-21T07:48:47.599459+0000","last_fullsized":"2024-09-21T07:48:47.599459+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:09.449030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":306,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.606118+0000","last_change":"2024-09-21T07:35:27.870121+0000","last_active":"2024-09-21T07:48:47.606118+0000","last_peered":"2024-09-21T07:48:47.606118+0000","last_clean":"2024-09-21T07:48:47.606118+0000","last_became_active":"2024-09-21T07:35:27.869811+0000","last_became_peered":"2024-09-21T07:35:27.869811+0000","last_unstale":"2024-09-21T07:48:47.606118+0000","last_undegraded":"2024-09-21T07:48:47.606118+0000","last_fullsized":"2024-09-21T07:48:47.606118+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:31.778461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":315,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.595449+0000","last_change":"2024-09-21T07:35:28.860699+0000","last_active":"2024-09-21T07:48:47.595449+0000","last_peered":"2024-09-21T07:48:47.595449+0000","last_clean":"2024-09-21T07:48:47.595449+0000","last_became_active":"2024-09-21T07:35:26.876779+0000","last_became_peered":"2024-09-21T07:35:26.876779+0000","last_unstale":"2024-09-21T07:48:47.595449+0000","last_undegraded":"2024-09-21T07:48:47.595449+0000","last_fullsized":"2024-09-21T07:48:47.595449+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:28.286403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000913188,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":367,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.594177+0000","last_change":"2024-09-21T07:34:24.260181+0000","last_active":"2024-09-21T07:48:47.594177+0000","last_peered":"2024-09-21T07:48:47.594177+0000","last_clean":"2024-09-21T07:48:47.594177+0000","last_became_active":"2024-09-21T07:34:23.275580+0000","last_became_peered":"2024-09-21T07:34:23.275580+0000","last_unstale":"2024-09-21T07:48:47.594177+0000","last_undegraded":"2024-09-21T07:48:47.594177+0000","last_fullsized":"2024-09-21T07:48:47.594177+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:49.857622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014706980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.629579+0000","last_change":"2024-09-21T07:48:47.627712+0000","last_active":"2024-09-21T07:48:47.629579+0000","last_peered":"2024-09-21T07:48:47.629579+0000","last_clean":"2024-09-21T07:48:47.629579+0000","last_became_active":"2024-09-21T07:48:47.623777+0000","last_became_peered":"2024-09-21T07:48:47.623777+0000","last_unstale":"2024-09-21T07:48:47.629579+0000","last_undegraded":"2024-09-21T07:48:47.629579+0000","last_fullsized":"2024-09-21T07:48:47.629579+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:42:35.733593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":13,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.596000+0000","last_change":"2024-09-21T07:48:46.602604+0000","last_active":"2024-09-21T07:48:47.596000+0000","last_peered":"2024-09-21T07:48:47.596000+0000","last_clean":"2024-09-21T07:48:47.596000+0000","last_became_active":"2024-09-21T07:48:46.602252+0000","last_became_peered":"2024-09-21T07:48:46.602252+0000","last_unstale":"2024-09-21T07:48:47.596000+0000","last_undegraded":"2024-09-21T07:48:47.596000+0000","last_fullsized":"2024-09-21T07:48:47.596000+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:02:41.387196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":307,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.600169+0000","last_change":"2024-09-21T07:35:27.875393+0000","last_active":"2024-09-21T07:48:47.600169+0000","last_peered":"2024-09-21T07:48:47.600169+0000","last_clean":"2024-09-21T07:48:47.600169+0000","last_became_active":"2024-09-21T07:35:27.874671+0000","last_became_peered":"2024-09-21T07:35:27.874671+0000","last_unstale":"2024-09-21T07:48:47.600169+0000","last_undegraded":"2024-09-21T07:48:47.600169+0000","last_fullsized":"2024-09-21T07:48:47.600169+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":313,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.608651+0000","last_change":"2024-09-21T07:35:29.206296+0000","last_active":"2024-09-21T07:48:47.608651+0000","last_peered":"2024-09-21T07:48:47.608651+0000","last_clean":"2024-09-21T07:48:47.608651+0000","last_became_active":"2024-09-21T07:35:26.874575+0000","last_became_peered":"2024-09-21T07:35:26.874575+0000","last_unstale":"2024-09-21T07:48:47.608651+0000","last_undegraded":"2024-09-21T07:48:47.608651+0000","last_fullsized":"2024-09-21T07:48:47.608651+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:22.771154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061011599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":370,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609427+0000","last_change":"2024-09-21T07:34:24.259112+0000","last_active":"2024-09-21T07:48:48.609427+0000","last_peered":"2024-09-21T07:48:48.609427+0000","last_clean":"2024-09-21T07:48:48.609427+0000","last_became_active":"2024-09-21T07:34:23.635444+0000","last_became_peered":"2024-09-21T07:34:23.635444+0000","last_unstale":"2024-09-21T07:48:48.609427+0000","last_undegraded":"2024-09-21T07:48:48.609427+0000","last_fullsized":"2024-09-21T07:48:48.609427+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:46.255385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001952636,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.596453+0000","last_change":"2024-09-21T07:48:46.596453+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.596453+0000","last_undegraded":"2024-09-21T07:48:46.596453+0000","last_fullsized":"2024-09-21T07:48:46.596453+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.10","version":"218'8","reported_seq":21,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.633173+0000","last_change":"2024-09-21T07:48:46.607918+0000","last_active":"2024-09-21T07:48:47.633173+0000","last_peered":"2024-09-21T07:48:47.633173+0000","last_clean":"2024-09-21T07:48:47.633173+0000","last_became_active":"2024-09-21T07:48:46.607600+0000","last_became_peered":"2024-09-21T07:48:46.607600+0000","last_unstale":"2024-09-21T07:48:47.633173+0000","last_undegraded":"2024-09-21T07:48:47.633173+0000","last_fullsized":"2024-09-21T07:48:47.633173+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:55:20.508968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":307,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.605147+0000","last_change":"2024-09-21T07:35:27.878356+0000","last_active":"2024-09-21T07:48:47.605147+0000","last_peered":"2024-09-21T07:48:47.605147+0000","last_clean":"2024-09-21T07:48:47.605147+0000","last_became_active":"2024-09-21T07:35:27.877982+0000","last_became_peered":"2024-09-21T07:35:27.877982+0000","last_unstale":"2024-09-21T07:48:47.605147+0000","last_undegraded":"2024-09-21T07:48:47.605147+0000","last_fullsized":"2024-09-21T07:48:47.605147+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":314,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.603882+0000","last_change":"2024-09-21T07:35:29.053131+0000","last_active":"2024-09-21T07:48:47.603882+0000","last_peered":"2024-09-21T07:48:47.603882+0000","last_clean":"2024-09-21T07:48:47.603882+0000","last_became_active":"2024-09-21T07:35:26.870516+0000","last_became_peered":"2024-09-21T07:35:26.870516+0000","last_unstale":"2024-09-21T07:48:47.603882+0000","last_undegraded":"2024-09-21T07:48:47.603882+0000","last_fullsized":"2024-09-21T07:48:47.603882+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:10.924491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070705699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":375,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.591878+0000","last_change":"2024-09-21T07:34:24.539152+0000","last_active":"2024-09-21T07:48:46.591878+0000","last_peered":"2024-09-21T07:48:46.591878+0000","last_clean":"2024-09-21T07:48:46.591878+0000","last_became_active":"2024-09-21T07:34:23.259573+0000","last_became_peered":"2024-09-21T07:34:23.259573+0000","last_unstale":"2024-09-21T07:48:46.591878+0000","last_undegraded":"2024-09-21T07:48:46.591878+0000","last_fullsized":"2024-09-21T07:48:46.591878+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:26.986900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058108400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.632435+0000","last_change":"2024-09-21T07:48:47.627105+0000","last_active":"2024-09-21T07:48:47.632435+0000","last_peered":"2024-09-21T07:48:47.632435+0000","last_clean":"2024-09-21T07:48:47.632435+0000","last_became_active":"2024-09-21T07:48:47.626559+0000","last_became_peered":"2024-09-21T07:48:47.626559+0000","last_unstale":"2024-09-21T07:48:47.632435+0000","last_undegraded":"2024-09-21T07:48:47.632435+0000","last_fullsized":"2024-09-21T07:48:47.632435+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:45.810340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.611508+0000","last_change":"2024-09-21T07:48:46.607214+0000","last_active":"2024-09-21T07:48:46.611508+0000","last_peered":"2024-09-21T07:48:46.611508+0000","last_clean":"2024-09-21T07:48:46.611508+0000","last_became_active":"2024-09-21T07:48:46.606843+0000","last_became_peered":"2024-09-21T07:48:46.606843+0000","last_unstale":"2024-09-21T07:48:46.611508+0000","last_undegraded":"2024-09-21T07:48:46.611508+0000","last_fullsized":"2024-09-21T07:48:46.611508+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:37.232512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":306,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.603776+0000","last_change":"2024-09-21T07:35:27.873726+0000","last_active":"2024-09-21T07:48:47.603776+0000","last_peered":"2024-09-21T07:48:47.603776+0000","last_clean":"2024-09-21T07:48:47.603776+0000","last_became_active":"2024-09-21T07:35:27.871421+0000","last_became_peered":"2024-09-21T07:35:27.871421+0000","last_unstale":"2024-09-21T07:48:47.603776+0000","last_undegraded":"2024-09-21T07:48:47.603776+0000","last_fullsized":"2024-09-21T07:48:47.603776+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:00.168915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":315,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.593992+0000","last_change":"2024-09-21T07:35:28.861032+0000","last_active":"2024-09-21T07:48:47.593992+0000","last_peered":"2024-09-21T07:48:47.593992+0000","last_clean":"2024-09-21T07:48:47.593992+0000","last_became_active":"2024-09-21T07:35:26.873149+0000","last_became_peered":"2024-09-21T07:35:26.873149+0000","last_unstale":"2024-09-21T07:48:47.593992+0000","last_undegraded":"2024-09-21T07:48:47.593992+0000","last_fullsized":"2024-09-21T07:48:47.593992+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:59.435981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011988369999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":362,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.597392+0000","last_change":"2024-09-21T07:34:24.259902+0000","last_active":"2024-09-21T07:48:46.597392+0000","last_peered":"2024-09-21T07:48:46.597392+0000","last_clean":"2024-09-21T07:48:46.597392+0000","last_became_active":"2024-09-21T07:34:23.271902+0000","last_became_peered":"2024-09-21T07:34:23.271902+0000","last_unstale":"2024-09-21T07:48:46.597392+0000","last_undegraded":"2024-09-21T07:48:46.597392+0000","last_fullsized":"2024-09-21T07:48:46.597392+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:59.439525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001686358,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.589360+0000","last_change":"2024-09-21T07:48:46.589360+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.589360+0000","last_undegraded":"2024-09-21T07:48:46.589360+0000","last_fullsized":"2024-09-21T07:48:46.589360+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.608012+0000","last_change":"2024-09-21T07:48:46.606382+0000","last_active":"2024-09-21T07:48:46.608012+0000","last_peered":"2024-09-21T07:48:46.608012+0000","last_clean":"2024-09-21T07:48:46.608012+0000","last_became_active":"2024-09-21T07:48:46.606002+0000","last_became_peered":"2024-09-21T07:48:46.606002+0000","last_unstale":"2024-09-21T07:48:46.608012+0000","last_undegraded":"2024-09-21T07:48:46.608012+0000","last_fullsized":"2024-09-21T07:48:46.608012+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:03:59.840933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":304,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.595132+0000","last_change":"2024-09-21T07:35:27.872637+0000","last_active":"2024-09-21T07:48:47.595132+0000","last_peered":"2024-09-21T07:48:47.595132+0000","last_clean":"2024-09-21T07:48:47.595132+0000","last_became_active":"2024-09-21T07:35:27.872149+0000","last_became_peered":"2024-09-21T07:35:27.872149+0000","last_unstale":"2024-09-21T07:48:47.595132+0000","last_undegraded":"2024-09-21T07:48:47.595132+0000","last_fullsized":"2024-09-21T07:48:47.595132+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:27.697872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":309,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.602775+0000","last_change":"2024-09-21T07:35:29.493362+0000","last_active":"2024-09-21T07:48:46.602775+0000","last_peered":"2024-09-21T07:48:46.602775+0000","last_clean":"2024-09-21T07:48:46.602775+0000","last_became_active":"2024-09-21T07:35:26.873018+0000","last_became_peered":"2024-09-21T07:35:26.873018+0000","last_unstale":"2024-09-21T07:48:46.602775+0000","last_undegraded":"2024-09-21T07:48:46.602775+0000","last_fullsized":"2024-09-21T07:48:46.602775+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:02.460557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012496510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":370,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609996+0000","last_change":"2024-09-21T07:34:24.259662+0000","last_active":"2024-09-21T07:48:48.609996+0000","last_peered":"2024-09-21T07:48:48.609996+0000","last_clean":"2024-09-21T07:48:48.609996+0000","last_became_active":"2024-09-21T07:34:23.278812+0000","last_became_peered":"2024-09-21T07:34:23.278812+0000","last_unstale":"2024-09-21T07:48:48.609996+0000","last_undegraded":"2024-09-21T07:48:48.609996+0000","last_fullsized":"2024-09-21T07:48:48.609996+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:49.829974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356463,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.591252+0000","last_change":"2024-09-21T07:48:46.591252+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.591252+0000","last_undegraded":"2024-09-21T07:48:46.591252+0000","last_fullsized":"2024-09-21T07:48:46.591252+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"218'1","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609860+0000","last_change":"2024-09-21T07:48:46.605998+0000","last_active":"2024-09-21T07:48:48.609860+0000","last_peered":"2024-09-21T07:48:48.609860+0000","last_clean":"2024-09-21T07:48:48.609860+0000","last_became_active":"2024-09-21T07:48:46.605714+0000","last_became_peered":"2024-09-21T07:48:46.605714+0000","last_unstale":"2024-09-21T07:48:48.609860+0000","last_undegraded":"2024-09-21T07:48:48.609860+0000","last_fullsized":"2024-09-21T07:48:48.609860+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:51.672371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":302,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051131+0000","last_change":"2024-09-21T07:35:27.868448+0000","last_active":"2024-09-21T07:48:50.051131+0000","last_peered":"2024-09-21T07:48:50.051131+0000","last_clean":"2024-09-21T07:48:50.051131+0000","last_became_active":"2024-09-21T07:35:27.868105+0000","last_became_peered":"2024-09-21T07:35:27.868105+0000","last_unstale":"2024-09-21T07:48:50.051131+0000","last_undegraded":"2024-09-21T07:48:50.051131+0000","last_fullsized":"2024-09-21T07:48:50.051131+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:41.468702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":308,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.602167+0000","last_change":"2024-09-21T07:35:29.492974+0000","last_active":"2024-09-21T07:48:46.602167+0000","last_peered":"2024-09-21T07:48:46.602167+0000","last_clean":"2024-09-21T07:48:46.602167+0000","last_became_active":"2024-09-21T07:35:26.872165+0000","last_became_peered":"2024-09-21T07:35:26.872165+0000","last_unstale":"2024-09-21T07:48:46.602167+0000","last_undegraded":"2024-09-21T07:48:46.602167+0000","last_fullsized":"2024-09-21T07:48:46.602167+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:59.912520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013795179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":363,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.594739+0000","last_change":"2024-09-21T07:34:25.084861+0000","last_active":"2024-09-21T07:48:47.594739+0000","last_peered":"2024-09-21T07:48:47.594739+0000","last_clean":"2024-09-21T07:48:47.594739+0000","last_became_active":"2024-09-21T07:34:23.277708+0000","last_became_peered":"2024-09-21T07:34:23.277708+0000","last_unstale":"2024-09-21T07:48:47.594739+0000","last_undegraded":"2024-09-21T07:48:47.594739+0000","last_fullsized":"2024-09-21T07:48:47.594739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:40.816488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000597566,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.632205+0000","last_change":"2024-09-21T07:48:47.621555+0000","last_active":"2024-09-21T07:48:47.632205+0000","last_peered":"2024-09-21T07:48:47.632205+0000","last_clean":"2024-09-21T07:48:47.632205+0000","last_became_active":"2024-09-21T07:48:47.620805+0000","last_became_peered":"2024-09-21T07:48:47.620805+0000","last_unstale":"2024-09-21T07:48:47.632205+0000","last_undegraded":"2024-09-21T07:48:47.632205+0000","last_fullsized":"2024-09-21T07:48:47.632205+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:20.833555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":13,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.592490+0000","last_change":"2024-09-21T07:48:46.611660+0000","last_active":"2024-09-21T07:48:47.592490+0000","last_peered":"2024-09-21T07:48:47.592490+0000","last_clean":"2024-09-21T07:48:47.592490+0000","last_became_active":"2024-09-21T07:48:46.610843+0000","last_became_peered":"2024-09-21T07:48:46.610843+0000","last_unstale":"2024-09-21T07:48:47.592490+0000","last_undegraded":"2024-09-21T07:48:47.592490+0000","last_fullsized":"2024-09-21T07:48:47.592490+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:43:38.733833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":302,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051512+0000","last_change":"2024-09-21T07:35:27.874640+0000","last_active":"2024-09-21T07:48:50.051512+0000","last_peered":"2024-09-21T07:48:50.051512+0000","last_clean":"2024-09-21T07:48:50.051512+0000","last_became_active":"2024-09-21T07:35:27.874265+0000","last_became_peered":"2024-09-21T07:35:27.874265+0000","last_unstale":"2024-09-21T07:48:50.051512+0000","last_undegraded":"2024-09-21T07:48:50.051512+0000","last_fullsized":"2024-09-21T07:48:50.051512+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:45.754654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":312,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.606600+0000","last_change":"2024-09-21T07:35:29.207392+0000","last_active":"2024-09-21T07:48:47.606600+0000","last_peered":"2024-09-21T07:48:47.606600+0000","last_clean":"2024-09-21T07:48:47.606600+0000","last_became_active":"2024-09-21T07:35:26.870917+0000","last_became_peered":"2024-09-21T07:35:26.870917+0000","last_unstale":"2024-09-21T07:48:47.606600+0000","last_undegraded":"2024-09-21T07:48:47.606600+0000","last_fullsized":"2024-09-21T07:48:47.606600+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:59.159937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059198499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":370,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608972+0000","last_change":"2024-09-21T07:34:24.258709+0000","last_active":"2024-09-21T07:48:48.608972+0000","last_peered":"2024-09-21T07:48:48.608972+0000","last_clean":"2024-09-21T07:48:48.608972+0000","last_became_active":"2024-09-21T07:34:23.279042+0000","last_became_peered":"2024-09-21T07:34:23.279042+0000","last_unstale":"2024-09-21T07:48:48.608972+0000","last_undegraded":"2024-09-21T07:48:48.608972+0000","last_fullsized":"2024-09-21T07:48:48.608972+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:01.762117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019282100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608922+0000","last_change":"2024-09-21T07:48:47.607091+0000","last_active":"2024-09-21T07:48:48.608922+0000","last_peered":"2024-09-21T07:48:48.608922+0000","last_clean":"2024-09-21T07:48:48.608922+0000","last_became_active":"2024-09-21T07:48:47.606739+0000","last_became_peered":"2024-09-21T07:48:47.606739+0000","last_unstale":"2024-09-21T07:48:48.608922+0000","last_undegraded":"2024-09-21T07:48:48.608922+0000","last_fullsized":"2024-09-21T07:48:48.608922+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:42:23.765051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"218'2","reported_seq":15,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.628600+0000","last_change":"2024-09-21T07:48:46.612384+0000","last_active":"2024-09-21T07:48:47.628600+0000","last_peered":"2024-09-21T07:48:47.628600+0000","last_clean":"2024-09-21T07:48:47.628600+0000","last_became_active":"2024-09-21T07:48:46.610836+0000","last_became_peered":"2024-09-21T07:48:46.610836+0000","last_unstale":"2024-09-21T07:48:47.628600+0000","last_undegraded":"2024-09-21T07:48:47.628600+0000","last_fullsized":"2024-09-21T07:48:47.628600+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:04.310354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":304,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.598997+0000","last_change":"2024-09-21T07:35:27.870260+0000","last_active":"2024-09-21T07:48:47.598997+0000","last_peered":"2024-09-21T07:48:47.598997+0000","last_clean":"2024-09-21T07:48:47.598997+0000","last_became_active":"2024-09-21T07:35:27.869448+0000","last_became_peered":"2024-09-21T07:35:27.869448+0000","last_unstale":"2024-09-21T07:48:47.598997+0000","last_undegraded":"2024-09-21T07:48:47.598997+0000","last_fullsized":"2024-09-21T07:48:47.598997+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:16.946790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":309,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051488+0000","last_change":"2024-09-21T07:35:29.386367+0000","last_active":"2024-09-21T07:48:50.051488+0000","last_peered":"2024-09-21T07:48:50.051488+0000","last_clean":"2024-09-21T07:48:50.051488+0000","last_became_active":"2024-09-21T07:35:26.879238+0000","last_became_peered":"2024-09-21T07:35:26.879238+0000","last_unstale":"2024-09-21T07:48:50.051488+0000","last_undegraded":"2024-09-21T07:48:50.051488+0000","last_fullsized":"2024-09-21T07:48:50.051488+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:16.528260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092223100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":362,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.603686+0000","last_change":"2024-09-21T07:34:25.085635+0000","last_active":"2024-09-21T07:48:47.603686+0000","last_peered":"2024-09-21T07:48:47.603686+0000","last_clean":"2024-09-21T07:48:47.603686+0000","last_became_active":"2024-09-21T07:34:23.273566+0000","last_became_peered":"2024-09-21T07:34:23.273566+0000","last_unstale":"2024-09-21T07:48:47.603686+0000","last_undegraded":"2024-09-21T07:48:47.603686+0000","last_fullsized":"2024-09-21T07:48:47.603686+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:14.142272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079897300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.598103+0000","last_change":"2024-09-21T07:48:46.598103+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.598103+0000","last_undegraded":"2024-09-21T07:48:46.598103+0000","last_fullsized":"2024-09-21T07:48:46.598103+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609104+0000","last_change":"2024-09-21T07:48:46.610951+0000","last_active":"2024-09-21T07:48:48.609104+0000","last_peered":"2024-09-21T07:48:48.609104+0000","last_clean":"2024-09-21T07:48:48.609104+0000","last_became_active":"2024-09-21T07:48:46.610658+0000","last_became_peered":"2024-09-21T07:48:46.610658+0000","last_unstale":"2024-09-21T07:48:48.609104+0000","last_undegraded":"2024-09-21T07:48:48.609104+0000","last_fullsized":"2024-09-21T07:48:48.609104+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:15:20.903496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":304,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.596777+0000","last_change":"2024-09-21T07:35:27.878728+0000","last_active":"2024-09-21T07:48:47.596777+0000","last_peered":"2024-09-21T07:48:47.596777+0000","last_clean":"2024-09-21T07:48:47.596777+0000","last_became_active":"2024-09-21T07:35:27.878335+0000","last_became_peered":"2024-09-21T07:35:27.878335+0000","last_unstale":"2024-09-21T07:48:47.596777+0000","last_undegraded":"2024-09-21T07:48:47.596777+0000","last_fullsized":"2024-09-21T07:48:47.596777+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:25:37.731771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":322,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.591500+0000","last_change":"2024-09-21T07:35:28.858418+0000","last_active":"2024-09-21T07:48:46.591500+0000","last_peered":"2024-09-21T07:48:46.591500+0000","last_clean":"2024-09-21T07:48:46.591500+0000","last_became_active":"2024-09-21T07:35:26.876944+0000","last_became_peered":"2024-09-21T07:35:26.876944+0000","last_unstale":"2024-09-21T07:48:46.591500+0000","last_undegraded":"2024-09-21T07:48:46.591500+0000","last_fullsized":"2024-09-21T07:48:46.591500+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:04.514304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090862799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":358,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051805+0000","last_change":"2024-09-21T07:34:24.597926+0000","last_active":"2024-09-21T07:48:50.051805+0000","last_peered":"2024-09-21T07:48:50.051805+0000","last_clean":"2024-09-21T07:48:50.051805+0000","last_became_active":"2024-09-21T07:34:23.268926+0000","last_became_peered":"2024-09-21T07:34:23.268926+0000","last_unstale":"2024-09-21T07:48:50.051805+0000","last_undegraded":"2024-09-21T07:48:50.051805+0000","last_fullsized":"2024-09-21T07:48:50.051805+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:46.190752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101851,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.629415+0000","last_change":"2024-09-21T07:48:47.624905+0000","last_active":"2024-09-21T07:48:47.629415+0000","last_peered":"2024-09-21T07:48:47.629415+0000","last_clean":"2024-09-21T07:48:47.629415+0000","last_became_active":"2024-09-21T07:48:47.621782+0000","last_became_peered":"2024-09-21T07:48:47.621782+0000","last_unstale":"2024-09-21T07:48:47.629415+0000","last_undegraded":"2024-09-21T07:48:47.629415+0000","last_fullsized":"2024-09-21T07:48:47.629415+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:35.524796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051148+0000","last_change":"2024-09-21T07:48:46.613457+0000","last_active":"2024-09-21T07:48:50.051148+0000","last_peered":"2024-09-21T07:48:50.051148+0000","last_clean":"2024-09-21T07:48:50.051148+0000","last_became_active":"2024-09-21T07:48:46.612037+0000","last_became_peered":"2024-09-21T07:48:46.612037+0000","last_unstale":"2024-09-21T07:48:50.051148+0000","last_undegraded":"2024-09-21T07:48:50.051148+0000","last_fullsized":"2024-09-21T07:48:50.051148+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:06.200769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":303,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051801+0000","last_change":"2024-09-21T07:35:27.877426+0000","last_active":"2024-09-21T07:48:50.051801+0000","last_peered":"2024-09-21T07:48:50.051801+0000","last_clean":"2024-09-21T07:48:50.051801+0000","last_became_active":"2024-09-21T07:35:27.877050+0000","last_became_peered":"2024-09-21T07:35:27.877050+0000","last_unstale":"2024-09-21T07:48:50.051801+0000","last_undegraded":"2024-09-21T07:48:50.051801+0000","last_fullsized":"2024-09-21T07:48:50.051801+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:36.268334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":311,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.595671+0000","last_change":"2024-09-21T07:35:28.862273+0000","last_active":"2024-09-21T07:48:47.595671+0000","last_peered":"2024-09-21T07:48:47.595671+0000","last_clean":"2024-09-21T07:48:47.595671+0000","last_became_active":"2024-09-21T07:35:26.874011+0000","last_became_peered":"2024-09-21T07:35:26.874011+0000","last_unstale":"2024-09-21T07:48:47.595671+0000","last_undegraded":"2024-09-21T07:48:47.595671+0000","last_fullsized":"2024-09-21T07:48:47.595671+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:11.986950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071644799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":367,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.592690+0000","last_change":"2024-09-21T07:34:24.259794+0000","last_active":"2024-09-21T07:48:47.592690+0000","last_peered":"2024-09-21T07:48:47.592690+0000","last_clean":"2024-09-21T07:48:47.592690+0000","last_became_active":"2024-09-21T07:34:23.265679+0000","last_became_peered":"2024-09-21T07:34:23.265679+0000","last_unstale":"2024-09-21T07:48:47.592690+0000","last_undegraded":"2024-09-21T07:48:47.592690+0000","last_fullsized":"2024-09-21T07:48:47.592690+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:02.575796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001298518,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.630328+0000","last_change":"2024-09-21T07:48:47.618294+0000","last_active":"2024-09-21T07:48:47.630328+0000","last_peered":"2024-09-21T07:48:47.630328+0000","last_clean":"2024-09-21T07:48:47.630328+0000","last_became_active":"2024-09-21T07:48:47.617975+0000","last_became_peered":"2024-09-21T07:48:47.617975+0000","last_unstale":"2024-09-21T07:48:47.630328+0000","last_undegraded":"2024-09-21T07:48:47.630328+0000","last_fullsized":"2024-09-21T07:48:47.630328+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:42.345168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.610511+0000","last_change":"2024-09-21T07:48:46.608424+0000","last_active":"2024-09-21T07:48:46.610511+0000","last_peered":"2024-09-21T07:48:46.610511+0000","last_clean":"2024-09-21T07:48:46.610511+0000","last_became_active":"2024-09-21T07:48:46.607778+0000","last_became_peered":"2024-09-21T07:48:46.607778+0000","last_unstale":"2024-09-21T07:48:46.610511+0000","last_undegraded":"2024-09-21T07:48:46.610511+0000","last_fullsized":"2024-09-21T07:48:46.610511+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:59.561123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":306,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.609776+0000","last_change":"2024-09-21T07:35:27.877725+0000","last_active":"2024-09-21T07:48:47.609776+0000","last_peered":"2024-09-21T07:48:47.609776+0000","last_clean":"2024-09-21T07:48:47.609776+0000","last_became_active":"2024-09-21T07:35:27.877398+0000","last_became_peered":"2024-09-21T07:35:27.877398+0000","last_unstale":"2024-09-21T07:48:47.609776+0000","last_undegraded":"2024-09-21T07:48:47.609776+0000","last_fullsized":"2024-09-21T07:48:47.609776+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:05.859762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":314,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.593198+0000","last_change":"2024-09-21T07:35:28.860612+0000","last_active":"2024-09-21T07:48:47.593198+0000","last_peered":"2024-09-21T07:48:47.593198+0000","last_clean":"2024-09-21T07:48:47.593198+0000","last_became_active":"2024-09-21T07:35:26.876562+0000","last_became_peered":"2024-09-21T07:35:26.876562+0000","last_unstale":"2024-09-21T07:48:47.593198+0000","last_undegraded":"2024-09-21T07:48:47.593198+0000","last_fullsized":"2024-09-21T07:48:47.593198+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:06.824331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012252470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":361,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.602500+0000","last_change":"2024-09-21T07:34:24.260330+0000","last_active":"2024-09-21T07:48:46.602500+0000","last_peered":"2024-09-21T07:48:46.602500+0000","last_clean":"2024-09-21T07:48:46.602500+0000","last_became_active":"2024-09-21T07:34:23.637210+0000","last_became_peered":"2024-09-21T07:34:23.637210+0000","last_unstale":"2024-09-21T07:48:46.602500+0000","last_undegraded":"2024-09-21T07:48:46.602500+0000","last_fullsized":"2024-09-21T07:48:46.602500+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.488451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001537093,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.632407+0000","last_change":"2024-09-21T07:48:47.626890+0000","last_active":"2024-09-21T07:48:47.632407+0000","last_peered":"2024-09-21T07:48:47.632407+0000","last_clean":"2024-09-21T07:48:47.632407+0000","last_became_active":"2024-09-21T07:48:47.626088+0000","last_became_peered":"2024-09-21T07:48:47.626088+0000","last_unstale":"2024-09-21T07:48:47.632407+0000","last_undegraded":"2024-09-21T07:48:47.632407+0000","last_fullsized":"2024-09-21T07:48:47.632407+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:56:05.110865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.7","version":"218'2","reported_seq":15,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.625284+0000","last_change":"2024-09-21T07:48:46.607721+0000","last_active":"2024-09-21T07:48:47.625284+0000","last_peered":"2024-09-21T07:48:47.625284+0000","last_clean":"2024-09-21T07:48:47.625284+0000","last_became_active":"2024-09-21T07:48:46.607413+0000","last_became_peered":"2024-09-21T07:48:46.607413+0000","last_unstale":"2024-09-21T07:48:47.625284+0000","last_undegraded":"2024-09-21T07:48:47.625284+0000","last_fullsized":"2024-09-21T07:48:47.625284+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:47.152954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":304,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.594537+0000","last_change":"2024-09-21T07:35:27.874757+0000","last_active":"2024-09-21T07:48:47.594537+0000","last_peered":"2024-09-21T07:48:47.594537+0000","last_clean":"2024-09-21T07:48:47.594537+0000","last_became_active":"2024-09-21T07:35:27.874360+0000","last_became_peered":"2024-09-21T07:35:27.874360+0000","last_unstale":"2024-09-21T07:48:47.594537+0000","last_undegraded":"2024-09-21T07:48:47.594537+0000","last_fullsized":"2024-09-21T07:48:47.594537+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:17.646912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":308,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.602783+0000","last_change":"2024-09-21T07:35:29.493431+0000","last_active":"2024-09-21T07:48:46.602783+0000","last_peered":"2024-09-21T07:48:46.602783+0000","last_clean":"2024-09-21T07:48:46.602783+0000","last_became_active":"2024-09-21T07:35:26.873691+0000","last_became_peered":"2024-09-21T07:35:26.873691+0000","last_unstale":"2024-09-21T07:48:46.602783+0000","last_undegraded":"2024-09-21T07:48:46.602783+0000","last_fullsized":"2024-09-21T07:48:46.602783+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:51.705051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001162464,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":358,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051405+0000","last_change":"2024-09-21T07:34:24.597286+0000","last_active":"2024-09-21T07:48:50.051405+0000","last_peered":"2024-09-21T07:48:50.051405+0000","last_clean":"2024-09-21T07:48:50.051405+0000","last_became_active":"2024-09-21T07:34:23.267133+0000","last_became_peered":"2024-09-21T07:34:23.267133+0000","last_unstale":"2024-09-21T07:48:50.051405+0000","last_undegraded":"2024-09-21T07:48:50.051405+0000","last_fullsized":"2024-09-21T07:48:50.051405+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:18.257010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.627524+0000","last_change":"2024-09-21T07:48:47.624322+0000","last_active":"2024-09-21T07:48:47.627524+0000","last_peered":"2024-09-21T07:48:47.627524+0000","last_clean":"2024-09-21T07:48:47.627524+0000","last_became_active":"2024-09-21T07:48:47.623920+0000","last_became_peered":"2024-09-21T07:48:47.623920+0000","last_unstale":"2024-09-21T07:48:47.627524+0000","last_undegraded":"2024-09-21T07:48:47.627524+0000","last_fullsized":"2024-09-21T07:48:47.627524+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:31.490205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":13,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.605003+0000","last_change":"2024-09-21T07:48:46.599263+0000","last_active":"2024-09-21T07:48:47.605003+0000","last_peered":"2024-09-21T07:48:47.605003+0000","last_clean":"2024-09-21T07:48:47.605003+0000","last_became_active":"2024-09-21T07:48:46.597628+0000","last_became_peered":"2024-09-21T07:48:46.597628+0000","last_unstale":"2024-09-21T07:48:47.605003+0000","last_undegraded":"2024-09-21T07:48:47.605003+0000","last_fullsized":"2024-09-21T07:48:47.605003+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:38.541554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":315,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.593982+0000","last_change":"2024-09-21T07:35:27.864458+0000","last_active":"2024-09-21T07:48:46.593982+0000","last_peered":"2024-09-21T07:48:46.593982+0000","last_clean":"2024-09-21T07:48:46.593982+0000","last_became_active":"2024-09-21T07:35:27.864136+0000","last_became_peered":"2024-09-21T07:35:27.864136+0000","last_unstale":"2024-09-21T07:48:46.593982+0000","last_undegraded":"2024-09-21T07:48:46.593982+0000","last_fullsized":"2024-09-21T07:48:46.593982+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:20.162240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":322,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.594391+0000","last_change":"2024-09-21T07:35:28.858784+0000","last_active":"2024-09-21T07:48:46.594391+0000","last_peered":"2024-09-21T07:48:46.594391+0000","last_clean":"2024-09-21T07:48:46.594391+0000","last_became_active":"2024-09-21T07:35:26.876890+0000","last_became_peered":"2024-09-21T07:35:26.876890+0000","last_unstale":"2024-09-21T07:48:46.594391+0000","last_undegraded":"2024-09-21T07:48:46.594391+0000","last_fullsized":"2024-09-21T07:48:46.594391+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:40.738327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064543499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":369,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609757+0000","last_change":"2024-09-21T07:34:24.258712+0000","last_active":"2024-09-21T07:48:48.609757+0000","last_peered":"2024-09-21T07:48:48.609757+0000","last_clean":"2024-09-21T07:48:48.609757+0000","last_became_active":"2024-09-21T07:34:23.274692+0000","last_became_peered":"2024-09-21T07:34:23.274692+0000","last_unstale":"2024-09-21T07:48:48.609757+0000","last_undegraded":"2024-09-21T07:48:48.609757+0000","last_fullsized":"2024-09-21T07:48:48.609757+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:43.279454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018051479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.623521+0000","last_change":"2024-09-21T07:48:47.617896+0000","last_active":"2024-09-21T07:48:47.623521+0000","last_peered":"2024-09-21T07:48:47.623521+0000","last_clean":"2024-09-21T07:48:47.623521+0000","last_became_active":"2024-09-21T07:48:47.617588+0000","last_became_peered":"2024-09-21T07:48:47.617588+0000","last_unstale":"2024-09-21T07:48:47.623521+0000","last_undegraded":"2024-09-21T07:48:47.623521+0000","last_fullsized":"2024-09-21T07:48:47.623521+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:46.539795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051487+0000","last_change":"2024-09-21T07:48:46.609456+0000","last_active":"2024-09-21T07:48:50.051487+0000","last_peered":"2024-09-21T07:48:50.051487+0000","last_clean":"2024-09-21T07:48:50.051487+0000","last_became_active":"2024-09-21T07:48:46.609142+0000","last_became_peered":"2024-09-21T07:48:46.609142+0000","last_unstale":"2024-09-21T07:48:50.051487+0000","last_undegraded":"2024-09-21T07:48:50.051487+0000","last_fullsized":"2024-09-21T07:48:50.051487+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:21.813300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":308,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609370+0000","last_change":"2024-09-21T07:35:27.878020+0000","last_active":"2024-09-21T07:48:48.609370+0000","last_peered":"2024-09-21T07:48:48.609370+0000","last_clean":"2024-09-21T07:48:48.609370+0000","last_became_active":"2024-09-21T07:35:27.877532+0000","last_became_peered":"2024-09-21T07:35:27.877532+0000","last_unstale":"2024-09-21T07:48:48.609370+0000","last_undegraded":"2024-09-21T07:48:48.609370+0000","last_fullsized":"2024-09-21T07:48:48.609370+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:19.756073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":314,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.592718+0000","last_change":"2024-09-21T07:35:28.861049+0000","last_active":"2024-09-21T07:48:47.592718+0000","last_peered":"2024-09-21T07:48:47.592718+0000","last_clean":"2024-09-21T07:48:47.592718+0000","last_became_active":"2024-09-21T07:35:26.876819+0000","last_became_peered":"2024-09-21T07:35:26.876819+0000","last_unstale":"2024-09-21T07:48:47.592718+0000","last_undegraded":"2024-09-21T07:48:47.592718+0000","last_fullsized":"2024-09-21T07:48:47.592718+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:19.633803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055780599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":362,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.594656+0000","last_change":"2024-09-21T07:34:25.084849+0000","last_active":"2024-09-21T07:48:47.594656+0000","last_peered":"2024-09-21T07:48:47.594656+0000","last_clean":"2024-09-21T07:48:47.594656+0000","last_became_active":"2024-09-21T07:34:23.277713+0000","last_became_peered":"2024-09-21T07:34:23.277713+0000","last_unstale":"2024-09-21T07:48:47.594656+0000","last_undegraded":"2024-09-21T07:48:47.594656+0000","last_fullsized":"2024-09-21T07:48:47.594656+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:05.811810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059079499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.590874+0000","last_change":"2024-09-21T07:48:46.590874+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.590874+0000","last_undegraded":"2024-09-21T07:48:46.590874+0000","last_fullsized":"2024-09-21T07:48:46.590874+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.610468+0000","last_change":"2024-09-21T07:48:46.607396+0000","last_active":"2024-09-21T07:48:46.610468+0000","last_peered":"2024-09-21T07:48:46.610468+0000","last_clean":"2024-09-21T07:48:46.610468+0000","last_became_active":"2024-09-21T07:48:46.607082+0000","last_became_peered":"2024-09-21T07:48:46.607082+0000","last_unstale":"2024-09-21T07:48:46.610468+0000","last_undegraded":"2024-09-21T07:48:46.610468+0000","last_fullsized":"2024-09-21T07:48:46.610468+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:20.897148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":306,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.601794+0000","last_change":"2024-09-21T07:35:27.884248+0000","last_active":"2024-09-21T07:48:47.601794+0000","last_peered":"2024-09-21T07:48:47.601794+0000","last_clean":"2024-09-21T07:48:47.601794+0000","last_became_active":"2024-09-21T07:35:27.883861+0000","last_became_peered":"2024-09-21T07:35:27.883861+0000","last_unstale":"2024-09-21T07:48:47.601794+0000","last_undegraded":"2024-09-21T07:48:47.601794+0000","last_fullsized":"2024-09-21T07:48:47.601794+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":312,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.606467+0000","last_change":"2024-09-21T07:35:29.207135+0000","last_active":"2024-09-21T07:48:47.606467+0000","last_peered":"2024-09-21T07:48:47.606467+0000","last_clean":"2024-09-21T07:48:47.606467+0000","last_became_active":"2024-09-21T07:35:26.872787+0000","last_became_peered":"2024-09-21T07:35:26.872787+0000","last_unstale":"2024-09-21T07:48:47.606467+0000","last_undegraded":"2024-09-21T07:48:47.606467+0000","last_fullsized":"2024-09-21T07:48:47.606467+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:27.543217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011157509999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":374,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.594088+0000","last_change":"2024-09-21T07:34:24.538610+0000","last_active":"2024-09-21T07:48:46.594088+0000","last_peered":"2024-09-21T07:48:46.594088+0000","last_clean":"2024-09-21T07:48:46.594088+0000","last_became_active":"2024-09-21T07:34:23.259700+0000","last_became_peered":"2024-09-21T07:34:23.259700+0000","last_unstale":"2024-09-21T07:48:46.594088+0000","last_undegraded":"2024-09-21T07:48:46.594088+0000","last_fullsized":"2024-09-21T07:48:46.594088+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:57.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070522399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.632186+0000","last_change":"2024-09-21T07:48:47.623345+0000","last_active":"2024-09-21T07:48:47.632186+0000","last_peered":"2024-09-21T07:48:47.632186+0000","last_clean":"2024-09-21T07:48:47.632186+0000","last_became_active":"2024-09-21T07:48:47.622998+0000","last_became_peered":"2024-09-21T07:48:47.622998+0000","last_unstale":"2024-09-21T07:48:47.632186+0000","last_undegraded":"2024-09-21T07:48:47.632186+0000","last_fullsized":"2024-09-21T07:48:47.632186+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:26:23.537534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":13,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.605671+0000","last_change":"2024-09-21T07:48:46.599294+0000","last_active":"2024-09-21T07:48:47.605671+0000","last_peered":"2024-09-21T07:48:47.605671+0000","last_clean":"2024-09-21T07:48:47.605671+0000","last_became_active":"2024-09-21T07:48:46.598633+0000","last_became_peered":"2024-09-21T07:48:46.598633+0000","last_unstale":"2024-09-21T07:48:47.605671+0000","last_undegraded":"2024-09-21T07:48:47.605671+0000","last_fullsized":"2024-09-21T07:48:47.605671+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:34.227316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":315,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.592285+0000","last_change":"2024-09-21T07:35:27.868024+0000","last_active":"2024-09-21T07:48:46.592285+0000","last_peered":"2024-09-21T07:48:46.592285+0000","last_clean":"2024-09-21T07:48:46.592285+0000","last_became_active":"2024-09-21T07:35:27.867339+0000","last_became_peered":"2024-09-21T07:35:27.867339+0000","last_unstale":"2024-09-21T07:48:46.592285+0000","last_undegraded":"2024-09-21T07:48:46.592285+0000","last_fullsized":"2024-09-21T07:48:46.592285+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":322,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.591127+0000","last_change":"2024-09-21T07:35:28.858505+0000","last_active":"2024-09-21T07:48:46.591127+0000","last_peered":"2024-09-21T07:48:46.591127+0000","last_clean":"2024-09-21T07:48:46.591127+0000","last_became_active":"2024-09-21T07:35:26.877320+0000","last_became_peered":"2024-09-21T07:35:26.877320+0000","last_unstale":"2024-09-21T07:48:46.591127+0000","last_undegraded":"2024-09-21T07:48:46.591127+0000","last_fullsized":"2024-09-21T07:48:46.591127+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:53.672782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244119999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":360,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.052005+0000","last_change":"2024-09-21T07:34:24.597597+0000","last_active":"2024-09-21T07:48:50.052005+0000","last_peered":"2024-09-21T07:48:50.052005+0000","last_clean":"2024-09-21T07:48:50.052005+0000","last_became_active":"2024-09-21T07:34:23.635204+0000","last_became_peered":"2024-09-21T07:34:23.635204+0000","last_unstale":"2024-09-21T07:48:50.052005+0000","last_undegraded":"2024-09-21T07:48:50.052005+0000","last_fullsized":"2024-09-21T07:48:50.052005+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:48.644789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081139199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.597041+0000","last_change":"2024-09-21T07:48:46.597041+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.597041+0000","last_undegraded":"2024-09-21T07:48:46.597041+0000","last_fullsized":"2024-09-21T07:48:46.597041+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":13,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.605822+0000","last_change":"2024-09-21T07:48:46.602945+0000","last_active":"2024-09-21T07:48:47.605822+0000","last_peered":"2024-09-21T07:48:47.605822+0000","last_clean":"2024-09-21T07:48:47.605822+0000","last_became_active":"2024-09-21T07:48:46.602486+0000","last_became_peered":"2024-09-21T07:48:46.602486+0000","last_unstale":"2024-09-21T07:48:47.605822+0000","last_undegraded":"2024-09-21T07:48:47.605822+0000","last_fullsized":"2024-09-21T07:48:47.605822+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:51:05.478321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":305,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.606002+0000","last_change":"2024-09-21T07:35:27.877503+0000","last_active":"2024-09-21T07:48:47.606002+0000","last_peered":"2024-09-21T07:48:47.606002+0000","last_clean":"2024-09-21T07:48:47.606002+0000","last_became_active":"2024-09-21T07:35:27.875775+0000","last_became_peered":"2024-09-21T07:35:27.875775+0000","last_unstale":"2024-09-21T07:48:47.606002+0000","last_undegraded":"2024-09-21T07:48:47.606002+0000","last_fullsized":"2024-09-21T07:48:47.606002+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:49.291023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":308,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.597282+0000","last_change":"2024-09-21T07:35:29.491412+0000","last_active":"2024-09-21T07:48:46.597282+0000","last_peered":"2024-09-21T07:48:46.597282+0000","last_clean":"2024-09-21T07:48:46.597282+0000","last_became_active":"2024-09-21T07:35:26.876404+0000","last_became_peered":"2024-09-21T07:35:26.876404+0000","last_unstale":"2024-09-21T07:48:46.597282+0000","last_undegraded":"2024-09-21T07:48:46.597282+0000","last_fullsized":"2024-09-21T07:48:46.597282+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063918000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":370,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.610303+0000","last_change":"2024-09-21T07:34:24.259026+0000","last_active":"2024-09-21T07:48:48.610303+0000","last_peered":"2024-09-21T07:48:48.610303+0000","last_clean":"2024-09-21T07:48:48.610303+0000","last_became_active":"2024-09-21T07:34:23.635456+0000","last_became_peered":"2024-09-21T07:34:23.635456+0000","last_unstale":"2024-09-21T07:48:48.610303+0000","last_undegraded":"2024-09-21T07:48:48.610303+0000","last_fullsized":"2024-09-21T07:48:48.610303+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:09.592752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018654419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.598035+0000","last_change":"2024-09-21T07:48:46.598035+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.598035+0000","last_undegraded":"2024-09-21T07:48:46.598035+0000","last_fullsized":"2024-09-21T07:48:46.598035+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.613990+0000","last_change":"2024-09-21T07:48:46.612048+0000","last_active":"2024-09-21T07:48:46.613990+0000","last_peered":"2024-09-21T07:48:46.613990+0000","last_clean":"2024-09-21T07:48:46.613990+0000","last_became_active":"2024-09-21T07:48:46.611755+0000","last_became_peered":"2024-09-21T07:48:46.611755+0000","last_unstale":"2024-09-21T07:48:46.613990+0000","last_undegraded":"2024-09-21T07:48:46.613990+0000","last_fullsized":"2024-09-21T07:48:46.613990+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:12:05.809707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":305,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.605145+0000","last_change":"2024-09-21T07:35:27.871984+0000","last_active":"2024-09-21T07:48:47.605145+0000","last_peered":"2024-09-21T07:48:47.605145+0000","last_clean":"2024-09-21T07:48:47.605145+0000","last_became_active":"2024-09-21T07:35:27.870336+0000","last_became_peered":"2024-09-21T07:35:27.870336+0000","last_unstale":"2024-09-21T07:48:47.605145+0000","last_undegraded":"2024-09-21T07:48:47.605145+0000","last_fullsized":"2024-09-21T07:48:47.605145+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:57.120479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":314,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.592628+0000","last_change":"2024-09-21T07:35:28.860351+0000","last_active":"2024-09-21T07:48:47.592628+0000","last_peered":"2024-09-21T07:48:47.592628+0000","last_clean":"2024-09-21T07:48:47.592628+0000","last_became_active":"2024-09-21T07:35:26.872046+0000","last_became_peered":"2024-09-21T07:35:26.872046+0000","last_unstale":"2024-09-21T07:48:47.592628+0000","last_undegraded":"2024-09-21T07:48:47.592628+0000","last_fullsized":"2024-09-21T07:48:47.592628+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:47.074327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098384399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":361,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.600950+0000","last_change":"2024-09-21T07:34:24.260300+0000","last_active":"2024-09-21T07:48:46.600950+0000","last_peered":"2024-09-21T07:48:46.600950+0000","last_clean":"2024-09-21T07:48:46.600950+0000","last_became_active":"2024-09-21T07:34:23.270128+0000","last_became_peered":"2024-09-21T07:34:23.270128+0000","last_unstale":"2024-09-21T07:48:46.600950+0000","last_undegraded":"2024-09-21T07:48:46.600950+0000","last_fullsized":"2024-09-21T07:48:46.600950+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:25.260931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00176651,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.628380+0000","last_change":"2024-09-21T07:48:47.626058+0000","last_active":"2024-09-21T07:48:47.628380+0000","last_peered":"2024-09-21T07:48:47.628380+0000","last_clean":"2024-09-21T07:48:47.628380+0000","last_became_active":"2024-09-21T07:48:47.625779+0000","last_became_peered":"2024-09-21T07:48:47.625779+0000","last_unstale":"2024-09-21T07:48:47.628380+0000","last_undegraded":"2024-09-21T07:48:47.628380+0000","last_fullsized":"2024-09-21T07:48:47.628380+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:03.169164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":13,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.593455+0000","last_change":"2024-09-21T07:48:46.607858+0000","last_active":"2024-09-21T07:48:47.593455+0000","last_peered":"2024-09-21T07:48:47.593455+0000","last_clean":"2024-09-21T07:48:47.593455+0000","last_became_active":"2024-09-21T07:48:46.607521+0000","last_became_peered":"2024-09-21T07:48:46.607521+0000","last_unstale":"2024-09-21T07:48:47.593455+0000","last_undegraded":"2024-09-21T07:48:47.593455+0000","last_fullsized":"2024-09-21T07:48:47.593455+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:18.298760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":305,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.609882+0000","last_change":"2024-09-21T07:35:27.877861+0000","last_active":"2024-09-21T07:48:47.609882+0000","last_peered":"2024-09-21T07:48:47.609882+0000","last_clean":"2024-09-21T07:48:47.609882+0000","last_became_active":"2024-09-21T07:35:27.877434+0000","last_became_peered":"2024-09-21T07:35:27.877434+0000","last_unstale":"2024-09-21T07:48:47.609882+0000","last_undegraded":"2024-09-21T07:48:47.609882+0000","last_fullsized":"2024-09-21T07:48:47.609882+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":311,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.597277+0000","last_change":"2024-09-21T07:35:28.862954+0000","last_active":"2024-09-21T07:48:47.597277+0000","last_peered":"2024-09-21T07:48:47.597277+0000","last_clean":"2024-09-21T07:48:47.597277+0000","last_became_active":"2024-09-21T07:35:26.870613+0000","last_became_peered":"2024-09-21T07:35:26.870613+0000","last_unstale":"2024-09-21T07:48:47.597277+0000","last_undegraded":"2024-09-21T07:48:47.597277+0000","last_fullsized":"2024-09-21T07:48:47.597277+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:06.901043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059842100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":364,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.613383+0000","last_change":"2024-09-21T07:34:25.028904+0000","last_active":"2024-09-21T07:48:47.613383+0000","last_peered":"2024-09-21T07:48:47.613383+0000","last_clean":"2024-09-21T07:48:47.613383+0000","last_became_active":"2024-09-21T07:34:23.260942+0000","last_became_peered":"2024-09-21T07:34:23.260942+0000","last_unstale":"2024-09-21T07:48:47.613383+0000","last_undegraded":"2024-09-21T07:48:47.613383+0000","last_fullsized":"2024-09-21T07:48:47.613383+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:22.709328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060553899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":776,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27,"ondisk_log_size":27,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509632,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66372,"kb_used_data":3248,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94305468,"statfs":{"total":96636764160,"available":96568799232,"internally_reserved":0,"allocated":3325952,"data_stored":2046648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509634,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51908,"kb_used_data":2668,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319932,"statfs":{"total":96636764160,"available":96583610368,"internally_reserved":0,"allocated":2732032,"data_stored":1456280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509632,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33860,"kb_used_data":2668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337980,"statfs":{"total":96636764160,"available":96602091520,"internally_reserved":0,"allocated":2732032,"data_stored":1456280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509633,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29696,"kb_used_data":2668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342144,"statfs":{"total":96636764160,"available":96606355456,"internally_reserved":0,"allocated":2732032,"data_stored":1456280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509634,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47748,"kb_used_data":2668,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324092,"statfs":{"total":96636764160,"available":96587870208,"internally_reserved":0,"allocated":2732032,"data_stored":1456280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509634,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52492,"kb_used_data":3248,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319348,"statfs":{"total":96636764160,"available":96583012352,"internally_reserved":0,"allocated":3325952,"data_stored":2046648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542338,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51936,"kb_used_data":2696,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319904,"statfs":{"total":96636764160,"available":96583581696,"internally_reserved":0,"allocated":2760704,"data_stored":1472495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542337,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74148,"kb_used_data":2704,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297692,"statfs":{"total":96636764160,"available":96560836608,"internally_reserved":0,"allocated":2768896,"data_stored":1472039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:48:52.610 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:48:52.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.868+0000 7fd146000640 1 -- 172.21.15.39:0/1755076660 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1400b3670 msgr2=0x7fd1400b3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:52.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.868+0000 7fd146000640 1 --2- 172.21.15.39:0/1755076660 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1400b3670 0x7fd1400b3a80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd128005fd0 tx=0x7fd12802a5d0 comp rx=0 tx=0).stop 2024-09-21T07:48:52.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.868+0000 7fd146000640 1 -- 172.21.15.39:0/1755076660 shutdown_connections 2024-09-21T07:48:52.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.868+0000 7fd146000640 1 --2- 172.21.15.39:0/1755076660 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1400b3670 0x7fd1400b3a80 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:52.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.868+0000 7fd146000640 1 --2- 172.21.15.39:0/1755076660 >> v2:172.21.15.148:3300/0 conn(0x7fd1400b4c40 0x7fd1400ce2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:52.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.868+0000 7fd146000640 1 -- 172.21.15.39:0/1755076660 >> 172.21.15.39:0/1755076660 conn(0x7fd1400b2460 msgr2=0x7fd14006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:52.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.868+0000 7fd146000640 1 -- 172.21.15.39:0/1755076660 shutdown_connections 2024-09-21T07:48:52.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.869+0000 7fd146000640 1 -- 172.21.15.39:0/1755076660 wait complete. 2024-09-21T07:48:52.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.871+0000 7fd146000640 1 Processor -- start 2024-09-21T07:48:52.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.871+0000 7fd146000640 1 -- start start 2024-09-21T07:48:52.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.872+0000 7fd146000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1400b3670 0x7fd1400c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:52.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.872+0000 7fd146000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1400b4c40 0x7fd1401f7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:52.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.872+0000 7fd146000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1400ca8a0 0x7fd1401f7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:52.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.872+0000 7fd146000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd140070c20 con 0x7fd1400b4c40 2024-09-21T07:48:52.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.872+0000 7fd146000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd140070da0 con 0x7fd1400b3670 2024-09-21T07:48:52.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.872+0000 7fd146000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd1400cb3c0 con 0x7fd1400ca8a0 2024-09-21T07:48:52.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.873+0000 7fd137e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1400b4c40 0x7fd1401f7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:52.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.873+0000 7fd137e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1400b4c40 0x7fd1401f7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48824/0 (socket says 172.21.15.39:48824) 2024-09-21T07:48:52.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.873+0000 7fd137e00640 1 -- 172.21.15.39:0/3532765536 learned_addr learned my addr 172.21.15.39:0/3532765536 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:52.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.873+0000 7fd13f400640 1 --2- 172.21.15.39:0/3532765536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1400b3670 0x7fd1400c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:52.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.873+0000 7fd13fe00640 1 --2- 172.21.15.39:0/3532765536 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1400ca8a0 0x7fd1401f7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:52.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.873+0000 7fd137e00640 1 -- 172.21.15.39:0/3532765536 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1400ca8a0 msgr2=0x7fd1401f7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:52.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.873+0000 7fd137e00640 1 --2- 172.21.15.39:0/3532765536 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1400ca8a0 0x7fd1401f7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:52.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.873+0000 7fd137e00640 1 -- 172.21.15.39:0/3532765536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1400b3670 msgr2=0x7fd1400c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:52.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.873+0000 7fd137e00640 1 --2- 172.21.15.39:0/3532765536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1400b3670 0x7fd1400c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:52.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.873+0000 7fd137e00640 1 -- 172.21.15.39:0/3532765536 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd14009afc0 con 0x7fd1400b4c40 2024-09-21T07:48:52.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.873+0000 7fd13fe00640 1 --2- 172.21.15.39:0/3532765536 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1400ca8a0 0x7fd1401f7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:48:52.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.873+0000 7fd137e00640 1 --2- 172.21.15.39:0/3532765536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1400b4c40 0x7fd1401f7590 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7fd128034040 tx=0x7fd1280035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:52.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.874+0000 7fd137400640 1 -- 172.21.15.39:0/3532765536 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12803e030 con 0x7fd1400b4c40 2024-09-21T07:48:52.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.874+0000 7fd137400640 1 -- 172.21.15.39:0/3532765536 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd128003ae0 con 0x7fd1400b4c40 2024-09-21T07:48:52.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.874+0000 7fd146000640 1 -- 172.21.15.39:0/3532765536 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd14009b2a0 con 0x7fd1400b4c40 2024-09-21T07:48:52.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.874+0000 7fd137400640 1 -- 172.21.15.39:0/3532765536 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd128035040 con 0x7fd1400b4c40 2024-09-21T07:48:52.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.874+0000 7fd146000640 1 -- 172.21.15.39:0/3532765536 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd14009b860 con 0x7fd1400b4c40 2024-09-21T07:48:52.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.875+0000 7fd146000640 1 -- 172.21.15.39:0/3532765536 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1400b3a80 con 0x7fd1400b4c40 2024-09-21T07:48:52.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.876+0000 7fd137400640 1 -- 172.21.15.39:0/3532765536 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd128041030 con 0x7fd1400b4c40 2024-09-21T07:48:52.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.877+0000 7fd137400640 1 --2- 172.21.15.39:0/3532765536 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd11c078b90 0x7fd11c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:52.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.877+0000 7fd137400640 1 -- 172.21.15.39:0/3532765536 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(219..219 src has 1..219) ==== 6988+0+0 (secure 0 0 0) 0x7fd1280bb960 con 0x7fd1400b4c40 2024-09-21T07:48:52.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.877+0000 7fd13f400640 1 --2- 172.21.15.39:0/3532765536 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd11c078b90 0x7fd11c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:52.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.878+0000 7fd13f400640 1 --2- 172.21.15.39:0/3532765536 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd11c078b90 0x7fd11c07b0a0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fd1300095f0 tx=0x7fd130005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:52.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:52.882+0000 7fd137400640 1 -- 172.21.15.39:0/3532765536 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd1280832e0 con 0x7fd1400b4c40 2024-09-21T07:48:53.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.077+0000 7fd146000640 1 -- 172.21.15.39:0/3532765536 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd1400cc090 con 0x7fd11c078b90 2024-09-21T07:48:53.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.086+0000 7fd137400640 1 -- 172.21.15.39:0/3532765536 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+493904 (secure 0 0 0) 0x7fd13000e460 con 0x7fd11c078b90 2024-09-21T07:48:53.093 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:48:53.097 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:48:53.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.091+0000 7fd146000640 1 -- 172.21.15.39:0/3532765536 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd11c078b90 msgr2=0x7fd11c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:53.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.091+0000 7fd146000640 1 --2- 172.21.15.39:0/3532765536 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd11c078b90 0x7fd11c07b0a0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fd1300095f0 tx=0x7fd130005f90 comp rx=0 tx=0).stop 2024-09-21T07:48:53.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.091+0000 7fd146000640 1 -- 172.21.15.39:0/3532765536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1400b4c40 msgr2=0x7fd1401f7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:53.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.091+0000 7fd146000640 1 --2- 172.21.15.39:0/3532765536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1400b4c40 0x7fd1401f7590 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7fd128034040 tx=0x7fd1280035f0 comp rx=0 tx=0).stop 2024-09-21T07:48:53.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.091+0000 7fd146000640 1 -- 172.21.15.39:0/3532765536 shutdown_connections 2024-09-21T07:48:53.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.091+0000 7fd146000640 1 --2- 172.21.15.39:0/3532765536 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd11c078b90 0x7fd11c07b0a0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:53.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.091+0000 7fd146000640 1 --2- 172.21.15.39:0/3532765536 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd1400ca8a0 0x7fd1401f7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:53.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.091+0000 7fd146000640 1 --2- 172.21.15.39:0/3532765536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1400b4c40 0x7fd1401f7590 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:53.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.091+0000 7fd146000640 1 --2- 172.21.15.39:0/3532765536 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd1400b3670 0x7fd1400c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:53.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.091+0000 7fd146000640 1 -- 172.21.15.39:0/3532765536 >> 172.21.15.39:0/3532765536 conn(0x7fd1400b2460 msgr2=0x7fd140074d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:53.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.092+0000 7fd146000640 1 -- 172.21.15.39:0/3532765536 shutdown_connections 2024-09-21T07:48:53.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.092+0000 7fd146000640 1 -- 172.21.15.39:0/3532765536 wait complete. 2024-09-21T07:48:53.120 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":689,"stamp":"2024-09-21T07:48:51.852637+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591144,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":135,"num_write_kb":1638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":284,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":408160,"kb_used_data":22568,"kb_used_omap":12,"kb_used_meta":385395,"kb_avail":754566560,"statfs":{"total":773094113280,"available":772676157440,"internally_reserved":0,"allocated":23109632,"data_stored":12862950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":394645040},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":776,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001975"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.590080+0000","last_change":"2024-09-21T07:48:46.590080+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.590080+0000","last_undegraded":"2024-09-21T07:48:46.590080+0000","last_fullsized":"2024-09-21T07:48:46.590080+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.610021+0000","last_change":"2024-09-21T07:48:47.608680+0000","last_active":"2024-09-21T07:48:48.610021+0000","last_peered":"2024-09-21T07:48:48.610021+0000","last_clean":"2024-09-21T07:48:48.610021+0000","last_became_active":"2024-09-21T07:48:47.606772+0000","last_became_peered":"2024-09-21T07:48:47.606772+0000","last_unstale":"2024-09-21T07:48:48.610021+0000","last_undegraded":"2024-09-21T07:48:48.610021+0000","last_fullsized":"2024-09-21T07:48:48.610021+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:13.951566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.626629+0000","last_change":"2024-09-21T07:48:47.622897+0000","last_active":"2024-09-21T07:48:47.626629+0000","last_peered":"2024-09-21T07:48:47.626629+0000","last_clean":"2024-09-21T07:48:47.626629+0000","last_became_active":"2024-09-21T07:48:47.622568+0000","last_became_peered":"2024-09-21T07:48:47.622568+0000","last_unstale":"2024-09-21T07:48:47.626629+0000","last_undegraded":"2024-09-21T07:48:47.626629+0000","last_fullsized":"2024-09-21T07:48:47.626629+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:53:43.260855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.600167+0000","last_change":"2024-09-21T07:48:46.600167+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.600167+0000","last_undegraded":"2024-09-21T07:48:46.600167+0000","last_fullsized":"2024-09-21T07:48:46.600167+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.606103+0000","last_change":"2024-09-21T07:48:47.602747+0000","last_active":"2024-09-21T07:48:47.606103+0000","last_peered":"2024-09-21T07:48:47.606103+0000","last_clean":"2024-09-21T07:48:47.606103+0000","last_became_active":"2024-09-21T07:48:47.602453+0000","last_became_peered":"2024-09-21T07:48:47.602453+0000","last_unstale":"2024-09-21T07:48:47.606103+0000","last_undegraded":"2024-09-21T07:48:47.606103+0000","last_fullsized":"2024-09-21T07:48:47.606103+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:29.795876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.597012+0000","last_change":"2024-09-21T07:48:46.597012+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.597012+0000","last_undegraded":"2024-09-21T07:48:46.597012+0000","last_fullsized":"2024-09-21T07:48:46.597012+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.604038+0000","last_change":"2024-09-21T07:48:47.602027+0000","last_active":"2024-09-21T07:48:47.604038+0000","last_peered":"2024-09-21T07:48:47.604038+0000","last_clean":"2024-09-21T07:48:47.604038+0000","last_became_active":"2024-09-21T07:48:47.601703+0000","last_became_peered":"2024-09-21T07:48:47.601703+0000","last_unstale":"2024-09-21T07:48:47.604038+0000","last_undegraded":"2024-09-21T07:48:47.604038+0000","last_fullsized":"2024-09-21T07:48:47.604038+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:39.401545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.632462+0000","last_change":"2024-09-21T07:48:47.627224+0000","last_active":"2024-09-21T07:48:47.632462+0000","last_peered":"2024-09-21T07:48:47.632462+0000","last_clean":"2024-09-21T07:48:47.632462+0000","last_became_active":"2024-09-21T07:48:47.626767+0000","last_became_peered":"2024-09-21T07:48:47.626767+0000","last_unstale":"2024-09-21T07:48:47.632462+0000","last_undegraded":"2024-09-21T07:48:47.632462+0000","last_fullsized":"2024-09-21T07:48:47.632462+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:44:57.315947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.623586+0000","last_change":"2024-09-21T07:48:47.617745+0000","last_active":"2024-09-21T07:48:47.623586+0000","last_peered":"2024-09-21T07:48:47.623586+0000","last_clean":"2024-09-21T07:48:47.623586+0000","last_became_active":"2024-09-21T07:48:47.614030+0000","last_became_peered":"2024-09-21T07:48:47.614030+0000","last_unstale":"2024-09-21T07:48:47.623586+0000","last_undegraded":"2024-09-21T07:48:47.623586+0000","last_fullsized":"2024-09-21T07:48:47.623586+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:44:50.001056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.631705+0000","last_change":"2024-09-21T07:48:47.621852+0000","last_active":"2024-09-21T07:48:47.631705+0000","last_peered":"2024-09-21T07:48:47.631705+0000","last_clean":"2024-09-21T07:48:47.631705+0000","last_became_active":"2024-09-21T07:48:47.617439+0000","last_became_peered":"2024-09-21T07:48:47.617439+0000","last_unstale":"2024-09-21T07:48:47.631705+0000","last_undegraded":"2024-09-21T07:48:47.631705+0000","last_fullsized":"2024-09-21T07:48:47.631705+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:17:23.080659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.632943+0000","last_change":"2024-09-21T07:48:47.632098+0000","last_active":"2024-09-21T07:48:47.632943+0000","last_peered":"2024-09-21T07:48:47.632943+0000","last_clean":"2024-09-21T07:48:47.632943+0000","last_became_active":"2024-09-21T07:48:47.631802+0000","last_became_peered":"2024-09-21T07:48:47.631802+0000","last_unstale":"2024-09-21T07:48:47.632943+0000","last_undegraded":"2024-09-21T07:48:47.632943+0000","last_fullsized":"2024-09-21T07:48:47.632943+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:21.715374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.621185+0000","last_change":"2024-09-21T07:48:47.617229+0000","last_active":"2024-09-21T07:48:47.621185+0000","last_peered":"2024-09-21T07:48:47.621185+0000","last_clean":"2024-09-21T07:48:47.621185+0000","last_became_active":"2024-09-21T07:48:47.616863+0000","last_became_peered":"2024-09-21T07:48:47.616863+0000","last_unstale":"2024-09-21T07:48:47.621185+0000","last_undegraded":"2024-09-21T07:48:47.621185+0000","last_fullsized":"2024-09-21T07:48:47.621185+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:51.375781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.599782+0000","last_change":"2024-09-21T07:48:46.599782+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.599782+0000","last_undegraded":"2024-09-21T07:48:46.599782+0000","last_fullsized":"2024-09-21T07:48:46.599782+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.632509+0000","last_change":"2024-09-21T07:48:47.623337+0000","last_active":"2024-09-21T07:48:47.632509+0000","last_peered":"2024-09-21T07:48:47.632509+0000","last_clean":"2024-09-21T07:48:47.632509+0000","last_became_active":"2024-09-21T07:48:47.620228+0000","last_became_peered":"2024-09-21T07:48:47.620228+0000","last_unstale":"2024-09-21T07:48:47.632509+0000","last_undegraded":"2024-09-21T07:48:47.632509+0000","last_fullsized":"2024-09-21T07:48:47.632509+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:31.930727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051089+0000","last_change":"2024-09-21T07:48:47.627205+0000","last_active":"2024-09-21T07:48:50.051089+0000","last_peered":"2024-09-21T07:48:50.051089+0000","last_clean":"2024-09-21T07:48:50.051089+0000","last_became_active":"2024-09-21T07:48:47.626871+0000","last_became_peered":"2024-09-21T07:48:47.626871+0000","last_unstale":"2024-09-21T07:48:50.051089+0000","last_undegraded":"2024-09-21T07:48:50.051089+0000","last_fullsized":"2024-09-21T07:48:50.051089+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:57.118108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.629854+0000","last_change":"2024-09-21T07:48:47.627220+0000","last_active":"2024-09-21T07:48:47.629854+0000","last_peered":"2024-09-21T07:48:47.629854+0000","last_clean":"2024-09-21T07:48:47.629854+0000","last_became_active":"2024-09-21T07:48:47.626875+0000","last_became_peered":"2024-09-21T07:48:47.626875+0000","last_unstale":"2024-09-21T07:48:47.629854+0000","last_undegraded":"2024-09-21T07:48:47.629854+0000","last_fullsized":"2024-09-21T07:48:47.629854+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:11.801442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.628849+0000","last_change":"2024-09-21T07:48:47.625920+0000","last_active":"2024-09-21T07:48:47.628849+0000","last_peered":"2024-09-21T07:48:47.628849+0000","last_clean":"2024-09-21T07:48:47.628849+0000","last_became_active":"2024-09-21T07:48:47.625605+0000","last_became_peered":"2024-09-21T07:48:47.625605+0000","last_unstale":"2024-09-21T07:48:47.628849+0000","last_undegraded":"2024-09-21T07:48:47.628849+0000","last_fullsized":"2024-09-21T07:48:47.628849+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:17:04.946971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.629605+0000","last_change":"2024-09-21T07:48:47.628044+0000","last_active":"2024-09-21T07:48:47.629605+0000","last_peered":"2024-09-21T07:48:47.629605+0000","last_clean":"2024-09-21T07:48:47.629605+0000","last_became_active":"2024-09-21T07:48:47.626515+0000","last_became_peered":"2024-09-21T07:48:47.626515+0000","last_unstale":"2024-09-21T07:48:47.629605+0000","last_undegraded":"2024-09-21T07:48:47.629605+0000","last_fullsized":"2024-09-21T07:48:47.629605+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:40.234280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.632098+0000","last_change":"2024-09-21T07:48:47.623935+0000","last_active":"2024-09-21T07:48:47.632098+0000","last_peered":"2024-09-21T07:48:47.632098+0000","last_clean":"2024-09-21T07:48:47.632098+0000","last_became_active":"2024-09-21T07:48:47.623233+0000","last_became_peered":"2024-09-21T07:48:47.623233+0000","last_unstale":"2024-09-21T07:48:47.632098+0000","last_undegraded":"2024-09-21T07:48:47.632098+0000","last_fullsized":"2024-09-21T07:48:47.632098+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:29.332356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.630373+0000","last_change":"2024-09-21T07:48:47.617959+0000","last_active":"2024-09-21T07:48:47.630373+0000","last_peered":"2024-09-21T07:48:47.630373+0000","last_clean":"2024-09-21T07:48:47.630373+0000","last_became_active":"2024-09-21T07:48:47.617595+0000","last_became_peered":"2024-09-21T07:48:47.617595+0000","last_unstale":"2024-09-21T07:48:47.630373+0000","last_undegraded":"2024-09-21T07:48:47.630373+0000","last_fullsized":"2024-09-21T07:48:47.630373+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:16.164291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.631855+0000","last_change":"2024-09-21T07:48:47.621502+0000","last_active":"2024-09-21T07:48:47.631855+0000","last_peered":"2024-09-21T07:48:47.631855+0000","last_clean":"2024-09-21T07:48:47.631855+0000","last_became_active":"2024-09-21T07:48:47.616971+0000","last_became_peered":"2024-09-21T07:48:47.616971+0000","last_unstale":"2024-09-21T07:48:47.631855+0000","last_undegraded":"2024-09-21T07:48:47.631855+0000","last_fullsized":"2024-09-21T07:48:47.631855+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:59.711618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.631967+0000","last_change":"2024-09-21T07:48:47.622917+0000","last_active":"2024-09-21T07:48:47.631967+0000","last_peered":"2024-09-21T07:48:47.631967+0000","last_clean":"2024-09-21T07:48:47.631967+0000","last_became_active":"2024-09-21T07:48:47.619537+0000","last_became_peered":"2024-09-21T07:48:47.619537+0000","last_unstale":"2024-09-21T07:48:47.631967+0000","last_undegraded":"2024-09-21T07:48:47.631967+0000","last_fullsized":"2024-09-21T07:48:47.631967+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:14:17.772422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.610863+0000","last_change":"2024-09-21T07:48:47.598763+0000","last_active":"2024-09-21T07:48:48.610863+0000","last_peered":"2024-09-21T07:48:48.610863+0000","last_clean":"2024-09-21T07:48:48.610863+0000","last_became_active":"2024-09-21T07:48:47.598446+0000","last_became_peered":"2024-09-21T07:48:47.598446+0000","last_unstale":"2024-09-21T07:48:48.610863+0000","last_undegraded":"2024-09-21T07:48:48.610863+0000","last_fullsized":"2024-09-21T07:48:48.610863+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:48.912495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.607435+0000","last_change":"2024-09-21T07:48:47.596907+0000","last_active":"2024-09-21T07:48:47.607435+0000","last_peered":"2024-09-21T07:48:47.607435+0000","last_clean":"2024-09-21T07:48:47.607435+0000","last_became_active":"2024-09-21T07:48:47.596568+0000","last_became_peered":"2024-09-21T07:48:47.596568+0000","last_unstale":"2024-09-21T07:48:47.607435+0000","last_undegraded":"2024-09-21T07:48:47.607435+0000","last_fullsized":"2024-09-21T07:48:47.607435+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:47.548875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.623612+0000","last_change":"2024-09-21T07:48:47.620026+0000","last_active":"2024-09-21T07:48:47.623612+0000","last_peered":"2024-09-21T07:48:47.623612+0000","last_clean":"2024-09-21T07:48:47.623612+0000","last_became_active":"2024-09-21T07:48:47.619713+0000","last_became_peered":"2024-09-21T07:48:47.619713+0000","last_unstale":"2024-09-21T07:48:47.623612+0000","last_undegraded":"2024-09-21T07:48:47.623612+0000","last_fullsized":"2024-09-21T07:48:47.623612+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:15.278604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.592481+0000","last_change":"2024-09-21T07:48:46.592481+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.592481+0000","last_undegraded":"2024-09-21T07:48:46.592481+0000","last_fullsized":"2024-09-21T07:48:46.592481+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.627562+0000","last_change":"2024-09-21T07:48:47.625275+0000","last_active":"2024-09-21T07:48:47.627562+0000","last_peered":"2024-09-21T07:48:47.627562+0000","last_clean":"2024-09-21T07:48:47.627562+0000","last_became_active":"2024-09-21T07:48:47.624985+0000","last_became_peered":"2024-09-21T07:48:47.624985+0000","last_unstale":"2024-09-21T07:48:47.627562+0000","last_undegraded":"2024-09-21T07:48:47.627562+0000","last_fullsized":"2024-09-21T07:48:47.627562+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:22.189614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.612236+0000","last_change":"2024-09-21T07:48:47.608774+0000","last_active":"2024-09-21T07:48:47.612236+0000","last_peered":"2024-09-21T07:48:47.612236+0000","last_clean":"2024-09-21T07:48:47.612236+0000","last_became_active":"2024-09-21T07:48:47.608457+0000","last_became_peered":"2024-09-21T07:48:47.608457+0000","last_unstale":"2024-09-21T07:48:47.612236+0000","last_undegraded":"2024-09-21T07:48:47.612236+0000","last_fullsized":"2024-09-21T07:48:47.612236+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:25.633315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.613015+0000","last_change":"2024-09-21T07:48:47.605730+0000","last_active":"2024-09-21T07:48:47.613015+0000","last_peered":"2024-09-21T07:48:47.613015+0000","last_clean":"2024-09-21T07:48:47.613015+0000","last_became_active":"2024-09-21T07:48:47.605433+0000","last_became_peered":"2024-09-21T07:48:47.605433+0000","last_unstale":"2024-09-21T07:48:47.613015+0000","last_undegraded":"2024-09-21T07:48:47.613015+0000","last_fullsized":"2024-09-21T07:48:47.613015+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:15.258575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051126+0000","last_change":"2024-09-21T07:48:47.604557+0000","last_active":"2024-09-21T07:48:50.051126+0000","last_peered":"2024-09-21T07:48:50.051126+0000","last_clean":"2024-09-21T07:48:50.051126+0000","last_became_active":"2024-09-21T07:48:47.604196+0000","last_became_peered":"2024-09-21T07:48:47.604196+0000","last_unstale":"2024-09-21T07:48:50.051126+0000","last_undegraded":"2024-09-21T07:48:50.051126+0000","last_fullsized":"2024-09-21T07:48:50.051126+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:56.323906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.595955+0000","last_change":"2024-09-21T07:48:46.595955+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.595955+0000","last_undegraded":"2024-09-21T07:48:46.595955+0000","last_fullsized":"2024-09-21T07:48:46.595955+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.595810+0000","last_change":"2024-09-21T07:48:46.595810+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.595810+0000","last_undegraded":"2024-09-21T07:48:46.595810+0000","last_fullsized":"2024-09-21T07:48:46.595810+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.591446+0000","last_change":"2024-09-21T07:48:46.591446+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.591446+0000","last_undegraded":"2024-09-21T07:48:46.591446+0000","last_fullsized":"2024-09-21T07:48:46.591446+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1a","version":"218'1","reported_seq":14,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.622300+0000","last_change":"2024-09-21T07:48:46.603075+0000","last_active":"2024-09-21T07:48:47.622300+0000","last_peered":"2024-09-21T07:48:47.622300+0000","last_clean":"2024-09-21T07:48:47.622300+0000","last_became_active":"2024-09-21T07:48:46.602882+0000","last_became_peered":"2024-09-21T07:48:46.602882+0000","last_unstale":"2024-09-21T07:48:47.622300+0000","last_undegraded":"2024-09-21T07:48:47.622300+0000","last_fullsized":"2024-09-21T07:48:47.622300+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:49.025973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":301,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.602282+0000","last_change":"2024-09-21T07:35:27.879605+0000","last_active":"2024-09-21T07:48:46.602282+0000","last_peered":"2024-09-21T07:48:46.602282+0000","last_clean":"2024-09-21T07:48:46.602282+0000","last_became_active":"2024-09-21T07:35:27.879274+0000","last_became_peered":"2024-09-21T07:35:27.879274+0000","last_unstale":"2024-09-21T07:48:46.602282+0000","last_undegraded":"2024-09-21T07:48:46.602282+0000","last_fullsized":"2024-09-21T07:48:46.602282+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:24.241636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":316,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609470+0000","last_change":"2024-09-21T07:35:29.491336+0000","last_active":"2024-09-21T07:48:48.609470+0000","last_peered":"2024-09-21T07:48:48.609470+0000","last_clean":"2024-09-21T07:48:48.609470+0000","last_became_active":"2024-09-21T07:35:26.866484+0000","last_became_peered":"2024-09-21T07:35:26.866484+0000","last_unstale":"2024-09-21T07:48:48.609470+0000","last_undegraded":"2024-09-21T07:48:48.609470+0000","last_fullsized":"2024-09-21T07:48:48.609470+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:10.449665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076010100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":487,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.596417+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:48:46.596417+0000","last_peered":"2024-09-21T07:48:46.596417+0000","last_clean":"2024-09-21T07:48:46.596417+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:48:46.596417+0000","last_undegraded":"2024-09-21T07:48:46.596417+0000","last_fullsized":"2024-09-21T07:48:46.596417+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":375,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.590586+0000","last_change":"2024-09-21T07:34:24.539248+0000","last_active":"2024-09-21T07:48:46.590586+0000","last_peered":"2024-09-21T07:48:46.590586+0000","last_clean":"2024-09-21T07:48:46.590586+0000","last_became_active":"2024-09-21T07:34:23.273465+0000","last_became_peered":"2024-09-21T07:34:23.273465+0000","last_unstale":"2024-09-21T07:48:46.590586+0000","last_undegraded":"2024-09-21T07:48:46.590586+0000","last_fullsized":"2024-09-21T07:48:46.590586+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:55.517452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060627700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608458+0000","last_change":"2024-09-21T07:48:47.612024+0000","last_active":"2024-09-21T07:48:48.608458+0000","last_peered":"2024-09-21T07:48:48.608458+0000","last_clean":"2024-09-21T07:48:48.608458+0000","last_became_active":"2024-09-21T07:48:47.601212+0000","last_became_peered":"2024-09-21T07:48:47.601212+0000","last_unstale":"2024-09-21T07:48:48.608458+0000","last_undegraded":"2024-09-21T07:48:48.608458+0000","last_fullsized":"2024-09-21T07:48:48.608458+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:20.715873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"218'4","reported_seq":19,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051794+0000","last_change":"2024-09-21T07:48:46.613621+0000","last_active":"2024-09-21T07:48:50.051794+0000","last_peered":"2024-09-21T07:48:50.051794+0000","last_clean":"2024-09-21T07:48:50.051794+0000","last_became_active":"2024-09-21T07:48:46.613266+0000","last_became_peered":"2024-09-21T07:48:46.613266+0000","last_unstale":"2024-09-21T07:48:50.051794+0000","last_undegraded":"2024-09-21T07:48:50.051794+0000","last_fullsized":"2024-09-21T07:48:50.051794+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:40.639085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":309,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608414+0000","last_change":"2024-09-21T07:35:27.864492+0000","last_active":"2024-09-21T07:48:48.608414+0000","last_peered":"2024-09-21T07:48:48.608414+0000","last_clean":"2024-09-21T07:48:48.608414+0000","last_became_active":"2024-09-21T07:35:27.864176+0000","last_became_peered":"2024-09-21T07:35:27.864176+0000","last_unstale":"2024-09-21T07:48:48.608414+0000","last_undegraded":"2024-09-21T07:48:48.608414+0000","last_fullsized":"2024-09-21T07:48:48.608414+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:05.153467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":308,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.602895+0000","last_change":"2024-09-21T07:35:29.493678+0000","last_active":"2024-09-21T07:48:46.602895+0000","last_peered":"2024-09-21T07:48:46.602895+0000","last_clean":"2024-09-21T07:48:46.602895+0000","last_became_active":"2024-09-21T07:35:26.870749+0000","last_became_peered":"2024-09-21T07:35:26.870749+0000","last_unstale":"2024-09-21T07:48:46.602895+0000","last_undegraded":"2024-09-21T07:48:46.602895+0000","last_fullsized":"2024-09-21T07:48:46.602895+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:55.367877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010159570000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":359,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051441+0000","last_change":"2024-09-21T07:34:24.597274+0000","last_active":"2024-09-21T07:48:50.051441+0000","last_peered":"2024-09-21T07:48:50.051441+0000","last_clean":"2024-09-21T07:48:50.051441+0000","last_became_active":"2024-09-21T07:34:23.279560+0000","last_became_peered":"2024-09-21T07:34:23.279560+0000","last_unstale":"2024-09-21T07:48:50.051441+0000","last_undegraded":"2024-09-21T07:48:50.051441+0000","last_fullsized":"2024-09-21T07:48:50.051441+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:15.845265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059699799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.588781+0000","last_change":"2024-09-21T07:48:46.588781+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.588781+0000","last_undegraded":"2024-09-21T07:48:46.588781+0000","last_fullsized":"2024-09-21T07:48:46.588781+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.613288+0000","last_change":"2024-09-21T07:48:46.610499+0000","last_active":"2024-09-21T07:48:46.613288+0000","last_peered":"2024-09-21T07:48:46.613288+0000","last_clean":"2024-09-21T07:48:46.613288+0000","last_became_active":"2024-09-21T07:48:46.610137+0000","last_became_peered":"2024-09-21T07:48:46.610137+0000","last_unstale":"2024-09-21T07:48:46.613288+0000","last_undegraded":"2024-09-21T07:48:46.613288+0000","last_fullsized":"2024-09-21T07:48:46.613288+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:29:58.572619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":308,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.592520+0000","last_change":"2024-09-21T07:35:27.874387+0000","last_active":"2024-09-21T07:48:47.592520+0000","last_peered":"2024-09-21T07:48:47.592520+0000","last_clean":"2024-09-21T07:48:47.592520+0000","last_became_active":"2024-09-21T07:35:27.873277+0000","last_became_peered":"2024-09-21T07:35:27.873277+0000","last_unstale":"2024-09-21T07:48:47.592520+0000","last_undegraded":"2024-09-21T07:48:47.592520+0000","last_fullsized":"2024-09-21T07:48:47.592520+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:04.581463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":309,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051894+0000","last_change":"2024-09-21T07:35:29.387094+0000","last_active":"2024-09-21T07:48:50.051894+0000","last_peered":"2024-09-21T07:48:50.051894+0000","last_clean":"2024-09-21T07:48:50.051894+0000","last_became_active":"2024-09-21T07:35:26.874757+0000","last_became_peered":"2024-09-21T07:35:26.874757+0000","last_unstale":"2024-09-21T07:48:50.051894+0000","last_undegraded":"2024-09-21T07:48:50.051894+0000","last_fullsized":"2024-09-21T07:48:50.051894+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:33.542750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061065700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":361,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.604051+0000","last_change":"2024-09-21T07:34:24.259906+0000","last_active":"2024-09-21T07:48:46.604051+0000","last_peered":"2024-09-21T07:48:46.604051+0000","last_clean":"2024-09-21T07:48:46.604051+0000","last_became_active":"2024-09-21T07:34:23.635282+0000","last_became_peered":"2024-09-21T07:34:23.635282+0000","last_unstale":"2024-09-21T07:48:46.604051+0000","last_undegraded":"2024-09-21T07:48:46.604051+0000","last_fullsized":"2024-09-21T07:48:46.604051+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:00.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001527664,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.632254+0000","last_change":"2024-09-21T07:48:47.622852+0000","last_active":"2024-09-21T07:48:47.632254+0000","last_peered":"2024-09-21T07:48:47.632254+0000","last_clean":"2024-09-21T07:48:47.632254+0000","last_became_active":"2024-09-21T07:48:47.619895+0000","last_became_peered":"2024-09-21T07:48:47.619895+0000","last_unstale":"2024-09-21T07:48:47.632254+0000","last_undegraded":"2024-09-21T07:48:47.632254+0000","last_fullsized":"2024-09-21T07:48:47.632254+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:56:05.060203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"218'2","reported_seq":17,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609737+0000","last_change":"2024-09-21T07:48:46.606441+0000","last_active":"2024-09-21T07:48:48.609737+0000","last_peered":"2024-09-21T07:48:48.609737+0000","last_clean":"2024-09-21T07:48:48.609737+0000","last_became_active":"2024-09-21T07:48:46.604712+0000","last_became_peered":"2024-09-21T07:48:46.604712+0000","last_unstale":"2024-09-21T07:48:48.609737+0000","last_undegraded":"2024-09-21T07:48:48.609737+0000","last_fullsized":"2024-09-21T07:48:48.609737+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:22.700415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":308,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.593732+0000","last_change":"2024-09-21T07:35:27.873645+0000","last_active":"2024-09-21T07:48:47.593732+0000","last_peered":"2024-09-21T07:48:47.593732+0000","last_clean":"2024-09-21T07:48:47.593732+0000","last_became_active":"2024-09-21T07:35:27.873245+0000","last_became_peered":"2024-09-21T07:35:27.873245+0000","last_unstale":"2024-09-21T07:48:47.593732+0000","last_undegraded":"2024-09-21T07:48:47.593732+0000","last_fullsized":"2024-09-21T07:48:47.593732+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:55.090953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":314,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.592916+0000","last_change":"2024-09-21T07:35:28.861744+0000","last_active":"2024-09-21T07:48:47.592916+0000","last_peered":"2024-09-21T07:48:47.592916+0000","last_clean":"2024-09-21T07:48:47.592916+0000","last_became_active":"2024-09-21T07:35:26.876462+0000","last_became_peered":"2024-09-21T07:35:26.876462+0000","last_unstale":"2024-09-21T07:48:47.592916+0000","last_undegraded":"2024-09-21T07:48:47.592916+0000","last_fullsized":"2024-09-21T07:48:47.592916+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:48.632568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012020920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":363,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.594772+0000","last_change":"2024-09-21T07:34:25.085454+0000","last_active":"2024-09-21T07:48:47.594772+0000","last_peered":"2024-09-21T07:48:47.594772+0000","last_clean":"2024-09-21T07:48:47.594772+0000","last_became_active":"2024-09-21T07:34:23.279205+0000","last_became_peered":"2024-09-21T07:34:23.279205+0000","last_unstale":"2024-09-21T07:48:47.594772+0000","last_undegraded":"2024-09-21T07:48:47.594772+0000","last_fullsized":"2024-09-21T07:48:47.594772+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:54.231666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095320599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.614070+0000","last_change":"2024-09-21T07:48:47.610088+0000","last_active":"2024-09-21T07:48:47.614070+0000","last_peered":"2024-09-21T07:48:47.614070+0000","last_clean":"2024-09-21T07:48:47.614070+0000","last_became_active":"2024-09-21T07:48:47.607791+0000","last_became_peered":"2024-09-21T07:48:47.607791+0000","last_unstale":"2024-09-21T07:48:47.614070+0000","last_undegraded":"2024-09-21T07:48:47.614070+0000","last_fullsized":"2024-09-21T07:48:47.614070+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:03.037375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"218'2","reported_seq":15,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.615890+0000","last_change":"2024-09-21T07:48:46.602844+0000","last_active":"2024-09-21T07:48:47.615890+0000","last_peered":"2024-09-21T07:48:47.615890+0000","last_clean":"2024-09-21T07:48:47.615890+0000","last_became_active":"2024-09-21T07:48:46.602544+0000","last_became_peered":"2024-09-21T07:48:46.602544+0000","last_unstale":"2024-09-21T07:48:47.615890+0000","last_undegraded":"2024-09-21T07:48:47.615890+0000","last_fullsized":"2024-09-21T07:48:47.615890+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:36:36.719598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":308,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609553+0000","last_change":"2024-09-21T07:35:27.861373+0000","last_active":"2024-09-21T07:48:48.609553+0000","last_peered":"2024-09-21T07:48:48.609553+0000","last_clean":"2024-09-21T07:48:48.609553+0000","last_became_active":"2024-09-21T07:35:27.861016+0000","last_became_peered":"2024-09-21T07:35:27.861016+0000","last_unstale":"2024-09-21T07:48:48.609553+0000","last_undegraded":"2024-09-21T07:48:48.609553+0000","last_fullsized":"2024-09-21T07:48:48.609553+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:56.043763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":313,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.599014+0000","last_change":"2024-09-21T07:35:29.053677+0000","last_active":"2024-09-21T07:48:47.599014+0000","last_peered":"2024-09-21T07:48:47.599014+0000","last_clean":"2024-09-21T07:48:47.599014+0000","last_became_active":"2024-09-21T07:35:26.875496+0000","last_became_peered":"2024-09-21T07:35:26.875496+0000","last_unstale":"2024-09-21T07:48:47.599014+0000","last_undegraded":"2024-09-21T07:48:47.599014+0000","last_fullsized":"2024-09-21T07:48:47.599014+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:04.850784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065859000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":365,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.607231+0000","last_change":"2024-09-21T07:34:25.028521+0000","last_active":"2024-09-21T07:48:47.607231+0000","last_peered":"2024-09-21T07:48:47.607231+0000","last_clean":"2024-09-21T07:48:47.607231+0000","last_became_active":"2024-09-21T07:34:23.635467+0000","last_became_peered":"2024-09-21T07:34:23.635467+0000","last_unstale":"2024-09-21T07:48:47.607231+0000","last_undegraded":"2024-09-21T07:48:47.607231+0000","last_fullsized":"2024-09-21T07:48:47.607231+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:33.729123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000660558,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051543+0000","last_change":"2024-09-21T07:48:47.616627+0000","last_active":"2024-09-21T07:48:50.051543+0000","last_peered":"2024-09-21T07:48:50.051543+0000","last_clean":"2024-09-21T07:48:50.051543+0000","last_became_active":"2024-09-21T07:48:47.616246+0000","last_became_peered":"2024-09-21T07:48:47.616246+0000","last_unstale":"2024-09-21T07:48:50.051543+0000","last_undegraded":"2024-09-21T07:48:50.051543+0000","last_fullsized":"2024-09-21T07:48:50.051543+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:07.746423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.615861+0000","last_change":"2024-09-21T07:48:46.614962+0000","last_active":"2024-09-21T07:48:46.615861+0000","last_peered":"2024-09-21T07:48:46.615861+0000","last_clean":"2024-09-21T07:48:46.615861+0000","last_became_active":"2024-09-21T07:48:46.614616+0000","last_became_peered":"2024-09-21T07:48:46.614616+0000","last_unstale":"2024-09-21T07:48:46.615861+0000","last_undegraded":"2024-09-21T07:48:46.615861+0000","last_fullsized":"2024-09-21T07:48:46.615861+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:41:33.005009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":304,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.599104+0000","last_change":"2024-09-21T07:35:27.871201+0000","last_active":"2024-09-21T07:48:47.599104+0000","last_peered":"2024-09-21T07:48:47.599104+0000","last_clean":"2024-09-21T07:48:47.599104+0000","last_became_active":"2024-09-21T07:35:27.870816+0000","last_became_peered":"2024-09-21T07:35:27.870816+0000","last_unstale":"2024-09-21T07:48:47.599104+0000","last_undegraded":"2024-09-21T07:48:47.599104+0000","last_fullsized":"2024-09-21T07:48:47.599104+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:04.787450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":315,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.610027+0000","last_change":"2024-09-21T07:35:29.491874+0000","last_active":"2024-09-21T07:48:48.610027+0000","last_peered":"2024-09-21T07:48:48.610027+0000","last_clean":"2024-09-21T07:48:48.610027+0000","last_became_active":"2024-09-21T07:35:26.866381+0000","last_became_peered":"2024-09-21T07:35:26.866381+0000","last_unstale":"2024-09-21T07:48:48.610027+0000","last_undegraded":"2024-09-21T07:48:48.610027+0000","last_fullsized":"2024-09-21T07:48:48.610027+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:36.404647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073235399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":370,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.610052+0000","last_change":"2024-09-21T07:34:24.260046+0000","last_active":"2024-09-21T07:48:48.610052+0000","last_peered":"2024-09-21T07:48:48.610052+0000","last_clean":"2024-09-21T07:48:48.610052+0000","last_became_active":"2024-09-21T07:34:23.273431+0000","last_became_peered":"2024-09-21T07:34:23.273431+0000","last_unstale":"2024-09-21T07:48:48.610052+0000","last_undegraded":"2024-09-21T07:48:48.610052+0000","last_fullsized":"2024-09-21T07:48:48.610052+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:18.876057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019803310000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051791+0000","last_change":"2024-09-21T07:48:47.605350+0000","last_active":"2024-09-21T07:48:50.051791+0000","last_peered":"2024-09-21T07:48:50.051791+0000","last_clean":"2024-09-21T07:48:50.051791+0000","last_became_active":"2024-09-21T07:48:47.605046+0000","last_became_peered":"2024-09-21T07:48:47.605046+0000","last_unstale":"2024-09-21T07:48:50.051791+0000","last_undegraded":"2024-09-21T07:48:50.051791+0000","last_fullsized":"2024-09-21T07:48:50.051791+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:07.906313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.611056+0000","last_change":"2024-09-21T07:48:46.607052+0000","last_active":"2024-09-21T07:48:46.611056+0000","last_peered":"2024-09-21T07:48:46.611056+0000","last_clean":"2024-09-21T07:48:46.611056+0000","last_became_active":"2024-09-21T07:48:46.606707+0000","last_became_peered":"2024-09-21T07:48:46.606707+0000","last_unstale":"2024-09-21T07:48:46.611056+0000","last_undegraded":"2024-09-21T07:48:46.611056+0000","last_fullsized":"2024-09-21T07:48:46.611056+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:29:39.188621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":304,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.603501+0000","last_change":"2024-09-21T07:35:27.871736+0000","last_active":"2024-09-21T07:48:47.603501+0000","last_peered":"2024-09-21T07:48:47.603501+0000","last_clean":"2024-09-21T07:48:47.603501+0000","last_became_active":"2024-09-21T07:35:27.871232+0000","last_became_peered":"2024-09-21T07:35:27.871232+0000","last_unstale":"2024-09-21T07:48:47.603501+0000","last_undegraded":"2024-09-21T07:48:47.603501+0000","last_fullsized":"2024-09-21T07:48:47.603501+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:04.419301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":316,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608800+0000","last_change":"2024-09-21T07:35:29.491246+0000","last_active":"2024-09-21T07:48:48.608800+0000","last_peered":"2024-09-21T07:48:48.608800+0000","last_clean":"2024-09-21T07:48:48.608800+0000","last_became_active":"2024-09-21T07:35:26.866783+0000","last_became_peered":"2024-09-21T07:35:26.866783+0000","last_unstale":"2024-09-21T07:48:48.608800+0000","last_undegraded":"2024-09-21T07:48:48.608800+0000","last_fullsized":"2024-09-21T07:48:48.608800+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:28.881914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098474099999999992,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":365,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.599139+0000","last_change":"2024-09-21T07:34:25.028640+0000","last_active":"2024-09-21T07:48:47.599139+0000","last_peered":"2024-09-21T07:48:47.599139+0000","last_clean":"2024-09-21T07:48:47.599139+0000","last_became_active":"2024-09-21T07:34:23.260950+0000","last_became_peered":"2024-09-21T07:34:23.260950+0000","last_unstale":"2024-09-21T07:48:47.599139+0000","last_undegraded":"2024-09-21T07:48:47.599139+0000","last_fullsized":"2024-09-21T07:48:47.599139+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:23:42.121398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055135099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.623528+0000","last_change":"2024-09-21T07:48:47.614858+0000","last_active":"2024-09-21T07:48:47.623528+0000","last_peered":"2024-09-21T07:48:47.623528+0000","last_clean":"2024-09-21T07:48:47.623528+0000","last_became_active":"2024-09-21T07:48:47.614484+0000","last_became_peered":"2024-09-21T07:48:47.614484+0000","last_unstale":"2024-09-21T07:48:47.623528+0000","last_undegraded":"2024-09-21T07:48:47.623528+0000","last_fullsized":"2024-09-21T07:48:47.623528+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:24.141015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":13,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.603705+0000","last_change":"2024-09-21T07:48:46.607151+0000","last_active":"2024-09-21T07:48:47.603705+0000","last_peered":"2024-09-21T07:48:47.603705+0000","last_clean":"2024-09-21T07:48:47.603705+0000","last_became_active":"2024-09-21T07:48:46.606839+0000","last_became_peered":"2024-09-21T07:48:46.606839+0000","last_unstale":"2024-09-21T07:48:47.603705+0000","last_undegraded":"2024-09-21T07:48:47.603705+0000","last_fullsized":"2024-09-21T07:48:47.603705+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:34.780363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":305,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.609983+0000","last_change":"2024-09-21T07:35:27.872027+0000","last_active":"2024-09-21T07:48:47.609983+0000","last_peered":"2024-09-21T07:48:47.609983+0000","last_clean":"2024-09-21T07:48:47.609983+0000","last_became_active":"2024-09-21T07:35:27.870426+0000","last_became_peered":"2024-09-21T07:35:27.870426+0000","last_unstale":"2024-09-21T07:48:47.609983+0000","last_undegraded":"2024-09-21T07:48:47.609983+0000","last_fullsized":"2024-09-21T07:48:47.609983+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":316,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608519+0000","last_change":"2024-09-21T07:35:29.491008+0000","last_active":"2024-09-21T07:48:48.608519+0000","last_peered":"2024-09-21T07:48:48.608519+0000","last_clean":"2024-09-21T07:48:48.608519+0000","last_became_active":"2024-09-21T07:35:26.866750+0000","last_became_peered":"2024-09-21T07:35:26.866750+0000","last_unstale":"2024-09-21T07:48:48.608519+0000","last_undegraded":"2024-09-21T07:48:48.608519+0000","last_fullsized":"2024-09-21T07:48:48.608519+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:25.490406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083529700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":362,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.602630+0000","last_change":"2024-09-21T07:34:24.260665+0000","last_active":"2024-09-21T07:48:46.602630+0000","last_peered":"2024-09-21T07:48:46.602630+0000","last_clean":"2024-09-21T07:48:46.602630+0000","last_became_active":"2024-09-21T07:34:23.635909+0000","last_became_peered":"2024-09-21T07:34:23.635909+0000","last_unstale":"2024-09-21T07:48:46.602630+0000","last_undegraded":"2024-09-21T07:48:46.602630+0000","last_fullsized":"2024-09-21T07:48:46.602630+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:44.536885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051072+0000","last_change":"2024-09-21T07:48:47.615147+0000","last_active":"2024-09-21T07:48:50.051072+0000","last_peered":"2024-09-21T07:48:50.051072+0000","last_clean":"2024-09-21T07:48:50.051072+0000","last_became_active":"2024-09-21T07:48:47.614792+0000","last_became_peered":"2024-09-21T07:48:47.614792+0000","last_unstale":"2024-09-21T07:48:50.051072+0000","last_undegraded":"2024-09-21T07:48:50.051072+0000","last_fullsized":"2024-09-21T07:48:50.051072+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:23.043769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":13,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.593919+0000","last_change":"2024-09-21T07:48:46.614465+0000","last_active":"2024-09-21T07:48:47.593919+0000","last_peered":"2024-09-21T07:48:47.593919+0000","last_clean":"2024-09-21T07:48:47.593919+0000","last_became_active":"2024-09-21T07:48:46.614155+0000","last_became_peered":"2024-09-21T07:48:46.614155+0000","last_unstale":"2024-09-21T07:48:47.593919+0000","last_undegraded":"2024-09-21T07:48:47.593919+0000","last_fullsized":"2024-09-21T07:48:47.593919+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:23:53.595692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":303,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051021+0000","last_change":"2024-09-21T07:35:27.874772+0000","last_active":"2024-09-21T07:48:50.051021+0000","last_peered":"2024-09-21T07:48:50.051021+0000","last_clean":"2024-09-21T07:48:50.051021+0000","last_became_active":"2024-09-21T07:35:27.874434+0000","last_became_peered":"2024-09-21T07:35:27.874434+0000","last_unstale":"2024-09-21T07:48:50.051021+0000","last_undegraded":"2024-09-21T07:48:50.051021+0000","last_fullsized":"2024-09-21T07:48:50.051021+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:31.998755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":323,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.590456+0000","last_change":"2024-09-21T07:35:28.859153+0000","last_active":"2024-09-21T07:48:46.590456+0000","last_peered":"2024-09-21T07:48:46.590456+0000","last_clean":"2024-09-21T07:48:46.590456+0000","last_became_active":"2024-09-21T07:35:26.874428+0000","last_became_peered":"2024-09-21T07:35:26.874428+0000","last_unstale":"2024-09-21T07:48:46.590456+0000","last_undegraded":"2024-09-21T07:48:46.590456+0000","last_fullsized":"2024-09-21T07:48:46.590456+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:04.487621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048557299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":375,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.591699+0000","last_change":"2024-09-21T07:34:24.538845+0000","last_active":"2024-09-21T07:48:46.591699+0000","last_peered":"2024-09-21T07:48:46.591699+0000","last_clean":"2024-09-21T07:48:46.591699+0000","last_became_active":"2024-09-21T07:34:23.636372+0000","last_became_peered":"2024-09-21T07:34:23.636372+0000","last_unstale":"2024-09-21T07:48:46.591699+0000","last_undegraded":"2024-09-21T07:48:46.591699+0000","last_fullsized":"2024-09-21T07:48:46.591699+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:29.393813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841451,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.624317+0000","last_change":"2024-09-21T07:48:47.617117+0000","last_active":"2024-09-21T07:48:47.624317+0000","last_peered":"2024-09-21T07:48:47.624317+0000","last_clean":"2024-09-21T07:48:47.624317+0000","last_became_active":"2024-09-21T07:48:47.616813+0000","last_became_peered":"2024-09-21T07:48:47.616813+0000","last_unstale":"2024-09-21T07:48:47.624317+0000","last_undegraded":"2024-09-21T07:48:47.624317+0000","last_fullsized":"2024-09-21T07:48:47.624317+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:50.252324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.612150+0000","last_change":"2024-09-21T07:48:46.610433+0000","last_active":"2024-09-21T07:48:46.612150+0000","last_peered":"2024-09-21T07:48:46.612150+0000","last_clean":"2024-09-21T07:48:46.612150+0000","last_became_active":"2024-09-21T07:48:46.610123+0000","last_became_peered":"2024-09-21T07:48:46.610123+0000","last_unstale":"2024-09-21T07:48:46.612150+0000","last_undegraded":"2024-09-21T07:48:46.612150+0000","last_fullsized":"2024-09-21T07:48:46.612150+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:59.556565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":306,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.607367+0000","last_change":"2024-09-21T07:35:27.877801+0000","last_active":"2024-09-21T07:48:47.607367+0000","last_peered":"2024-09-21T07:48:47.607367+0000","last_clean":"2024-09-21T07:48:47.607367+0000","last_became_active":"2024-09-21T07:35:27.877464+0000","last_became_peered":"2024-09-21T07:35:27.877464+0000","last_unstale":"2024-09-21T07:48:47.607367+0000","last_undegraded":"2024-09-21T07:48:47.607367+0000","last_fullsized":"2024-09-21T07:48:47.607367+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:08.814929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":312,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.603961+0000","last_change":"2024-09-21T07:35:28.863732+0000","last_active":"2024-09-21T07:48:47.603961+0000","last_peered":"2024-09-21T07:48:47.603961+0000","last_clean":"2024-09-21T07:48:47.603961+0000","last_became_active":"2024-09-21T07:35:26.877392+0000","last_became_peered":"2024-09-21T07:35:26.877392+0000","last_unstale":"2024-09-21T07:48:47.603961+0000","last_undegraded":"2024-09-21T07:48:47.603961+0000","last_fullsized":"2024-09-21T07:48:47.603961+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071425700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":370,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.610101+0000","last_change":"2024-09-21T07:34:24.259686+0000","last_active":"2024-09-21T07:48:48.610101+0000","last_peered":"2024-09-21T07:48:48.610101+0000","last_clean":"2024-09-21T07:48:48.610101+0000","last_became_active":"2024-09-21T07:34:23.274405+0000","last_became_peered":"2024-09-21T07:34:23.274405+0000","last_unstale":"2024-09-21T07:48:48.610101+0000","last_undegraded":"2024-09-21T07:48:48.610101+0000","last_fullsized":"2024-09-21T07:48:48.610101+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:26.304693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022009920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.623488+0000","last_change":"2024-09-21T07:48:47.616194+0000","last_active":"2024-09-21T07:48:47.623488+0000","last_peered":"2024-09-21T07:48:47.623488+0000","last_clean":"2024-09-21T07:48:47.623488+0000","last_became_active":"2024-09-21T07:48:47.615876+0000","last_became_peered":"2024-09-21T07:48:47.615876+0000","last_unstale":"2024-09-21T07:48:47.623488+0000","last_undegraded":"2024-09-21T07:48:47.623488+0000","last_fullsized":"2024-09-21T07:48:47.623488+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:07.420806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"218'3","reported_seq":16,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.620054+0000","last_change":"2024-09-21T07:48:46.605473+0000","last_active":"2024-09-21T07:48:47.620054+0000","last_peered":"2024-09-21T07:48:47.620054+0000","last_clean":"2024-09-21T07:48:47.620054+0000","last_became_active":"2024-09-21T07:48:46.604548+0000","last_became_peered":"2024-09-21T07:48:46.604548+0000","last_unstale":"2024-09-21T07:48:47.620054+0000","last_undegraded":"2024-09-21T07:48:47.620054+0000","last_fullsized":"2024-09-21T07:48:47.620054+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:05.902786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":303,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051307+0000","last_change":"2024-09-21T07:35:27.867336+0000","last_active":"2024-09-21T07:48:50.051307+0000","last_peered":"2024-09-21T07:48:50.051307+0000","last_clean":"2024-09-21T07:48:50.051307+0000","last_became_active":"2024-09-21T07:35:27.867015+0000","last_became_peered":"2024-09-21T07:35:27.867015+0000","last_unstale":"2024-09-21T07:48:50.051307+0000","last_undegraded":"2024-09-21T07:48:50.051307+0000","last_fullsized":"2024-09-21T07:48:50.051307+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:59.214139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":309,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.602390+0000","last_change":"2024-09-21T07:35:29.493137+0000","last_active":"2024-09-21T07:48:46.602390+0000","last_peered":"2024-09-21T07:48:46.602390+0000","last_clean":"2024-09-21T07:48:46.602390+0000","last_became_active":"2024-09-21T07:35:26.872230+0000","last_became_peered":"2024-09-21T07:35:26.872230+0000","last_unstale":"2024-09-21T07:48:46.602390+0000","last_undegraded":"2024-09-21T07:48:46.602390+0000","last_fullsized":"2024-09-21T07:48:46.602390+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:24.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001197449,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":362,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.596253+0000","last_change":"2024-09-21T07:34:24.258754+0000","last_active":"2024-09-21T07:48:46.596253+0000","last_peered":"2024-09-21T07:48:46.596253+0000","last_clean":"2024-09-21T07:48:46.596253+0000","last_became_active":"2024-09-21T07:34:23.272657+0000","last_became_peered":"2024-09-21T07:34:23.272657+0000","last_unstale":"2024-09-21T07:48:46.596253+0000","last_undegraded":"2024-09-21T07:48:46.596253+0000","last_fullsized":"2024-09-21T07:48:46.596253+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:05.477099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067143200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.630257+0000","last_change":"2024-09-21T07:48:47.629237+0000","last_active":"2024-09-21T07:48:47.630257+0000","last_peered":"2024-09-21T07:48:47.630257+0000","last_clean":"2024-09-21T07:48:47.630257+0000","last_became_active":"2024-09-21T07:48:47.628900+0000","last_became_peered":"2024-09-21T07:48:47.628900+0000","last_unstale":"2024-09-21T07:48:47.630257+0000","last_undegraded":"2024-09-21T07:48:47.630257+0000","last_fullsized":"2024-09-21T07:48:47.630257+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:19.087720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.615052+0000","last_change":"2024-09-21T07:48:46.612660+0000","last_active":"2024-09-21T07:48:46.615052+0000","last_peered":"2024-09-21T07:48:46.615052+0000","last_clean":"2024-09-21T07:48:46.615052+0000","last_became_active":"2024-09-21T07:48:46.612362+0000","last_became_peered":"2024-09-21T07:48:46.612362+0000","last_unstale":"2024-09-21T07:48:46.615052+0000","last_undegraded":"2024-09-21T07:48:46.615052+0000","last_fullsized":"2024-09-21T07:48:46.615052+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:10.600575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":305,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.597247+0000","last_change":"2024-09-21T07:35:27.876021+0000","last_active":"2024-09-21T07:48:47.597247+0000","last_peered":"2024-09-21T07:48:47.597247+0000","last_clean":"2024-09-21T07:48:47.597247+0000","last_became_active":"2024-09-21T07:35:27.875724+0000","last_became_peered":"2024-09-21T07:35:27.875724+0000","last_unstale":"2024-09-21T07:48:47.597247+0000","last_undegraded":"2024-09-21T07:48:47.597247+0000","last_fullsized":"2024-09-21T07:48:47.597247+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:03.022812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":309,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.600781+0000","last_change":"2024-09-21T07:35:29.492570+0000","last_active":"2024-09-21T07:48:46.600781+0000","last_peered":"2024-09-21T07:48:46.600781+0000","last_clean":"2024-09-21T07:48:46.600781+0000","last_became_active":"2024-09-21T07:35:26.874316+0000","last_became_peered":"2024-09-21T07:35:26.874316+0000","last_unstale":"2024-09-21T07:48:46.600781+0000","last_undegraded":"2024-09-21T07:48:46.600781+0000","last_fullsized":"2024-09-21T07:48:46.600781+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:16.512323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011990760000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":363,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.596170+0000","last_change":"2024-09-21T07:34:25.085232+0000","last_active":"2024-09-21T07:48:47.596170+0000","last_peered":"2024-09-21T07:48:47.596170+0000","last_clean":"2024-09-21T07:48:47.596170+0000","last_became_active":"2024-09-21T07:34:23.277807+0000","last_became_peered":"2024-09-21T07:34:23.277807+0000","last_unstale":"2024-09-21T07:48:47.596170+0000","last_undegraded":"2024-09-21T07:48:47.596170+0000","last_fullsized":"2024-09-21T07:48:47.596170+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:36.362919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086619099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609167+0000","last_change":"2024-09-21T07:48:47.607795+0000","last_active":"2024-09-21T07:48:48.609167+0000","last_peered":"2024-09-21T07:48:48.609167+0000","last_clean":"2024-09-21T07:48:48.609167+0000","last_became_active":"2024-09-21T07:48:47.607511+0000","last_became_peered":"2024-09-21T07:48:47.607511+0000","last_unstale":"2024-09-21T07:48:48.609167+0000","last_undegraded":"2024-09-21T07:48:48.609167+0000","last_fullsized":"2024-09-21T07:48:48.609167+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:27.353767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"218'2","reported_seq":17,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609872+0000","last_change":"2024-09-21T07:48:46.609344+0000","last_active":"2024-09-21T07:48:48.609872+0000","last_peered":"2024-09-21T07:48:48.609872+0000","last_clean":"2024-09-21T07:48:48.609872+0000","last_became_active":"2024-09-21T07:48:46.606541+0000","last_became_peered":"2024-09-21T07:48:46.606541+0000","last_unstale":"2024-09-21T07:48:48.609872+0000","last_undegraded":"2024-09-21T07:48:48.609872+0000","last_fullsized":"2024-09-21T07:48:48.609872+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:36.274172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":306,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.606343+0000","last_change":"2024-09-21T07:35:27.872798+0000","last_active":"2024-09-21T07:48:47.606343+0000","last_peered":"2024-09-21T07:48:47.606343+0000","last_clean":"2024-09-21T07:48:47.606343+0000","last_became_active":"2024-09-21T07:35:27.872483+0000","last_became_peered":"2024-09-21T07:35:27.872483+0000","last_unstale":"2024-09-21T07:48:47.606343+0000","last_undegraded":"2024-09-21T07:48:47.606343+0000","last_fullsized":"2024-09-21T07:48:47.606343+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:11.933469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":313,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.605746+0000","last_change":"2024-09-21T07:35:29.206723+0000","last_active":"2024-09-21T07:48:47.605746+0000","last_peered":"2024-09-21T07:48:47.605746+0000","last_clean":"2024-09-21T07:48:47.605746+0000","last_became_active":"2024-09-21T07:35:26.872377+0000","last_became_peered":"2024-09-21T07:35:26.872377+0000","last_unstale":"2024-09-21T07:48:47.605746+0000","last_undegraded":"2024-09-21T07:48:47.605746+0000","last_fullsized":"2024-09-21T07:48:47.605746+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:29.086862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083077200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":359,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051990+0000","last_change":"2024-09-21T07:34:24.597672+0000","last_active":"2024-09-21T07:48:50.051990+0000","last_peered":"2024-09-21T07:48:50.051990+0000","last_clean":"2024-09-21T07:48:50.051990+0000","last_became_active":"2024-09-21T07:34:23.267770+0000","last_became_peered":"2024-09-21T07:34:23.267770+0000","last_unstale":"2024-09-21T07:48:50.051990+0000","last_undegraded":"2024-09-21T07:48:50.051990+0000","last_fullsized":"2024-09-21T07:48:50.051990+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:51.629476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086378500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.624444+0000","last_change":"2024-09-21T07:48:47.618502+0000","last_active":"2024-09-21T07:48:47.624444+0000","last_peered":"2024-09-21T07:48:47.624444+0000","last_clean":"2024-09-21T07:48:47.624444+0000","last_became_active":"2024-09-21T07:48:47.618213+0000","last_became_peered":"2024-09-21T07:48:47.618213+0000","last_unstale":"2024-09-21T07:48:47.624444+0000","last_undegraded":"2024-09-21T07:48:47.624444+0000","last_fullsized":"2024-09-21T07:48:47.624444+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:19.164379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":13,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.599459+0000","last_change":"2024-09-21T07:48:46.602566+0000","last_active":"2024-09-21T07:48:47.599459+0000","last_peered":"2024-09-21T07:48:47.599459+0000","last_clean":"2024-09-21T07:48:47.599459+0000","last_became_active":"2024-09-21T07:48:46.602254+0000","last_became_peered":"2024-09-21T07:48:46.602254+0000","last_unstale":"2024-09-21T07:48:47.599459+0000","last_undegraded":"2024-09-21T07:48:47.599459+0000","last_fullsized":"2024-09-21T07:48:47.599459+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:09.449030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":306,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.606118+0000","last_change":"2024-09-21T07:35:27.870121+0000","last_active":"2024-09-21T07:48:47.606118+0000","last_peered":"2024-09-21T07:48:47.606118+0000","last_clean":"2024-09-21T07:48:47.606118+0000","last_became_active":"2024-09-21T07:35:27.869811+0000","last_became_peered":"2024-09-21T07:35:27.869811+0000","last_unstale":"2024-09-21T07:48:47.606118+0000","last_undegraded":"2024-09-21T07:48:47.606118+0000","last_fullsized":"2024-09-21T07:48:47.606118+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:31.778461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":315,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.595449+0000","last_change":"2024-09-21T07:35:28.860699+0000","last_active":"2024-09-21T07:48:47.595449+0000","last_peered":"2024-09-21T07:48:47.595449+0000","last_clean":"2024-09-21T07:48:47.595449+0000","last_became_active":"2024-09-21T07:35:26.876779+0000","last_became_peered":"2024-09-21T07:35:26.876779+0000","last_unstale":"2024-09-21T07:48:47.595449+0000","last_undegraded":"2024-09-21T07:48:47.595449+0000","last_fullsized":"2024-09-21T07:48:47.595449+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:28.286403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000913188,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":367,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.594177+0000","last_change":"2024-09-21T07:34:24.260181+0000","last_active":"2024-09-21T07:48:47.594177+0000","last_peered":"2024-09-21T07:48:47.594177+0000","last_clean":"2024-09-21T07:48:47.594177+0000","last_became_active":"2024-09-21T07:34:23.275580+0000","last_became_peered":"2024-09-21T07:34:23.275580+0000","last_unstale":"2024-09-21T07:48:47.594177+0000","last_undegraded":"2024-09-21T07:48:47.594177+0000","last_fullsized":"2024-09-21T07:48:47.594177+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:49.857622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014706980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.629579+0000","last_change":"2024-09-21T07:48:47.627712+0000","last_active":"2024-09-21T07:48:47.629579+0000","last_peered":"2024-09-21T07:48:47.629579+0000","last_clean":"2024-09-21T07:48:47.629579+0000","last_became_active":"2024-09-21T07:48:47.623777+0000","last_became_peered":"2024-09-21T07:48:47.623777+0000","last_unstale":"2024-09-21T07:48:47.629579+0000","last_undegraded":"2024-09-21T07:48:47.629579+0000","last_fullsized":"2024-09-21T07:48:47.629579+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:42:35.733593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":13,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.596000+0000","last_change":"2024-09-21T07:48:46.602604+0000","last_active":"2024-09-21T07:48:47.596000+0000","last_peered":"2024-09-21T07:48:47.596000+0000","last_clean":"2024-09-21T07:48:47.596000+0000","last_became_active":"2024-09-21T07:48:46.602252+0000","last_became_peered":"2024-09-21T07:48:46.602252+0000","last_unstale":"2024-09-21T07:48:47.596000+0000","last_undegraded":"2024-09-21T07:48:47.596000+0000","last_fullsized":"2024-09-21T07:48:47.596000+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:02:41.387196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":307,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.600169+0000","last_change":"2024-09-21T07:35:27.875393+0000","last_active":"2024-09-21T07:48:47.600169+0000","last_peered":"2024-09-21T07:48:47.600169+0000","last_clean":"2024-09-21T07:48:47.600169+0000","last_became_active":"2024-09-21T07:35:27.874671+0000","last_became_peered":"2024-09-21T07:35:27.874671+0000","last_unstale":"2024-09-21T07:48:47.600169+0000","last_undegraded":"2024-09-21T07:48:47.600169+0000","last_fullsized":"2024-09-21T07:48:47.600169+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":313,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.608651+0000","last_change":"2024-09-21T07:35:29.206296+0000","last_active":"2024-09-21T07:48:47.608651+0000","last_peered":"2024-09-21T07:48:47.608651+0000","last_clean":"2024-09-21T07:48:47.608651+0000","last_became_active":"2024-09-21T07:35:26.874575+0000","last_became_peered":"2024-09-21T07:35:26.874575+0000","last_unstale":"2024-09-21T07:48:47.608651+0000","last_undegraded":"2024-09-21T07:48:47.608651+0000","last_fullsized":"2024-09-21T07:48:47.608651+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:22.771154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061011599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":370,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609427+0000","last_change":"2024-09-21T07:34:24.259112+0000","last_active":"2024-09-21T07:48:48.609427+0000","last_peered":"2024-09-21T07:48:48.609427+0000","last_clean":"2024-09-21T07:48:48.609427+0000","last_became_active":"2024-09-21T07:34:23.635444+0000","last_became_peered":"2024-09-21T07:34:23.635444+0000","last_unstale":"2024-09-21T07:48:48.609427+0000","last_undegraded":"2024-09-21T07:48:48.609427+0000","last_fullsized":"2024-09-21T07:48:48.609427+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:46.255385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001952636,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.596453+0000","last_change":"2024-09-21T07:48:46.596453+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.596453+0000","last_undegraded":"2024-09-21T07:48:46.596453+0000","last_fullsized":"2024-09-21T07:48:46.596453+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.10","version":"218'8","reported_seq":21,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.633173+0000","last_change":"2024-09-21T07:48:46.607918+0000","last_active":"2024-09-21T07:48:47.633173+0000","last_peered":"2024-09-21T07:48:47.633173+0000","last_clean":"2024-09-21T07:48:47.633173+0000","last_became_active":"2024-09-21T07:48:46.607600+0000","last_became_peered":"2024-09-21T07:48:46.607600+0000","last_unstale":"2024-09-21T07:48:47.633173+0000","last_undegraded":"2024-09-21T07:48:47.633173+0000","last_fullsized":"2024-09-21T07:48:47.633173+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:55:20.508968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":307,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.605147+0000","last_change":"2024-09-21T07:35:27.878356+0000","last_active":"2024-09-21T07:48:47.605147+0000","last_peered":"2024-09-21T07:48:47.605147+0000","last_clean":"2024-09-21T07:48:47.605147+0000","last_became_active":"2024-09-21T07:35:27.877982+0000","last_became_peered":"2024-09-21T07:35:27.877982+0000","last_unstale":"2024-09-21T07:48:47.605147+0000","last_undegraded":"2024-09-21T07:48:47.605147+0000","last_fullsized":"2024-09-21T07:48:47.605147+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":314,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.603882+0000","last_change":"2024-09-21T07:35:29.053131+0000","last_active":"2024-09-21T07:48:47.603882+0000","last_peered":"2024-09-21T07:48:47.603882+0000","last_clean":"2024-09-21T07:48:47.603882+0000","last_became_active":"2024-09-21T07:35:26.870516+0000","last_became_peered":"2024-09-21T07:35:26.870516+0000","last_unstale":"2024-09-21T07:48:47.603882+0000","last_undegraded":"2024-09-21T07:48:47.603882+0000","last_fullsized":"2024-09-21T07:48:47.603882+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:10.924491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070705699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":375,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.591878+0000","last_change":"2024-09-21T07:34:24.539152+0000","last_active":"2024-09-21T07:48:46.591878+0000","last_peered":"2024-09-21T07:48:46.591878+0000","last_clean":"2024-09-21T07:48:46.591878+0000","last_became_active":"2024-09-21T07:34:23.259573+0000","last_became_peered":"2024-09-21T07:34:23.259573+0000","last_unstale":"2024-09-21T07:48:46.591878+0000","last_undegraded":"2024-09-21T07:48:46.591878+0000","last_fullsized":"2024-09-21T07:48:46.591878+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:26.986900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058108400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.632435+0000","last_change":"2024-09-21T07:48:47.627105+0000","last_active":"2024-09-21T07:48:47.632435+0000","last_peered":"2024-09-21T07:48:47.632435+0000","last_clean":"2024-09-21T07:48:47.632435+0000","last_became_active":"2024-09-21T07:48:47.626559+0000","last_became_peered":"2024-09-21T07:48:47.626559+0000","last_unstale":"2024-09-21T07:48:47.632435+0000","last_undegraded":"2024-09-21T07:48:47.632435+0000","last_fullsized":"2024-09-21T07:48:47.632435+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:45.810340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.611508+0000","last_change":"2024-09-21T07:48:46.607214+0000","last_active":"2024-09-21T07:48:46.611508+0000","last_peered":"2024-09-21T07:48:46.611508+0000","last_clean":"2024-09-21T07:48:46.611508+0000","last_became_active":"2024-09-21T07:48:46.606843+0000","last_became_peered":"2024-09-21T07:48:46.606843+0000","last_unstale":"2024-09-21T07:48:46.611508+0000","last_undegraded":"2024-09-21T07:48:46.611508+0000","last_fullsized":"2024-09-21T07:48:46.611508+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:37.232512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":306,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.603776+0000","last_change":"2024-09-21T07:35:27.873726+0000","last_active":"2024-09-21T07:48:47.603776+0000","last_peered":"2024-09-21T07:48:47.603776+0000","last_clean":"2024-09-21T07:48:47.603776+0000","last_became_active":"2024-09-21T07:35:27.871421+0000","last_became_peered":"2024-09-21T07:35:27.871421+0000","last_unstale":"2024-09-21T07:48:47.603776+0000","last_undegraded":"2024-09-21T07:48:47.603776+0000","last_fullsized":"2024-09-21T07:48:47.603776+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:00.168915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":315,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.593992+0000","last_change":"2024-09-21T07:35:28.861032+0000","last_active":"2024-09-21T07:48:47.593992+0000","last_peered":"2024-09-21T07:48:47.593992+0000","last_clean":"2024-09-21T07:48:47.593992+0000","last_became_active":"2024-09-21T07:35:26.873149+0000","last_became_peered":"2024-09-21T07:35:26.873149+0000","last_unstale":"2024-09-21T07:48:47.593992+0000","last_undegraded":"2024-09-21T07:48:47.593992+0000","last_fullsized":"2024-09-21T07:48:47.593992+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:59.435981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011988369999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":362,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.597392+0000","last_change":"2024-09-21T07:34:24.259902+0000","last_active":"2024-09-21T07:48:46.597392+0000","last_peered":"2024-09-21T07:48:46.597392+0000","last_clean":"2024-09-21T07:48:46.597392+0000","last_became_active":"2024-09-21T07:34:23.271902+0000","last_became_peered":"2024-09-21T07:34:23.271902+0000","last_unstale":"2024-09-21T07:48:46.597392+0000","last_undegraded":"2024-09-21T07:48:46.597392+0000","last_fullsized":"2024-09-21T07:48:46.597392+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:59.439525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001686358,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.589360+0000","last_change":"2024-09-21T07:48:46.589360+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.589360+0000","last_undegraded":"2024-09-21T07:48:46.589360+0000","last_fullsized":"2024-09-21T07:48:46.589360+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.608012+0000","last_change":"2024-09-21T07:48:46.606382+0000","last_active":"2024-09-21T07:48:46.608012+0000","last_peered":"2024-09-21T07:48:46.608012+0000","last_clean":"2024-09-21T07:48:46.608012+0000","last_became_active":"2024-09-21T07:48:46.606002+0000","last_became_peered":"2024-09-21T07:48:46.606002+0000","last_unstale":"2024-09-21T07:48:46.608012+0000","last_undegraded":"2024-09-21T07:48:46.608012+0000","last_fullsized":"2024-09-21T07:48:46.608012+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:03:59.840933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":304,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.595132+0000","last_change":"2024-09-21T07:35:27.872637+0000","last_active":"2024-09-21T07:48:47.595132+0000","last_peered":"2024-09-21T07:48:47.595132+0000","last_clean":"2024-09-21T07:48:47.595132+0000","last_became_active":"2024-09-21T07:35:27.872149+0000","last_became_peered":"2024-09-21T07:35:27.872149+0000","last_unstale":"2024-09-21T07:48:47.595132+0000","last_undegraded":"2024-09-21T07:48:47.595132+0000","last_fullsized":"2024-09-21T07:48:47.595132+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:27.697872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":309,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.602775+0000","last_change":"2024-09-21T07:35:29.493362+0000","last_active":"2024-09-21T07:48:46.602775+0000","last_peered":"2024-09-21T07:48:46.602775+0000","last_clean":"2024-09-21T07:48:46.602775+0000","last_became_active":"2024-09-21T07:35:26.873018+0000","last_became_peered":"2024-09-21T07:35:26.873018+0000","last_unstale":"2024-09-21T07:48:46.602775+0000","last_undegraded":"2024-09-21T07:48:46.602775+0000","last_fullsized":"2024-09-21T07:48:46.602775+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:02.460557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012496510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":370,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609996+0000","last_change":"2024-09-21T07:34:24.259662+0000","last_active":"2024-09-21T07:48:48.609996+0000","last_peered":"2024-09-21T07:48:48.609996+0000","last_clean":"2024-09-21T07:48:48.609996+0000","last_became_active":"2024-09-21T07:34:23.278812+0000","last_became_peered":"2024-09-21T07:34:23.278812+0000","last_unstale":"2024-09-21T07:48:48.609996+0000","last_undegraded":"2024-09-21T07:48:48.609996+0000","last_fullsized":"2024-09-21T07:48:48.609996+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:49.829974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356463,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.591252+0000","last_change":"2024-09-21T07:48:46.591252+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.591252+0000","last_undegraded":"2024-09-21T07:48:46.591252+0000","last_fullsized":"2024-09-21T07:48:46.591252+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"218'1","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609860+0000","last_change":"2024-09-21T07:48:46.605998+0000","last_active":"2024-09-21T07:48:48.609860+0000","last_peered":"2024-09-21T07:48:48.609860+0000","last_clean":"2024-09-21T07:48:48.609860+0000","last_became_active":"2024-09-21T07:48:46.605714+0000","last_became_peered":"2024-09-21T07:48:46.605714+0000","last_unstale":"2024-09-21T07:48:48.609860+0000","last_undegraded":"2024-09-21T07:48:48.609860+0000","last_fullsized":"2024-09-21T07:48:48.609860+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:51.672371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":302,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051131+0000","last_change":"2024-09-21T07:35:27.868448+0000","last_active":"2024-09-21T07:48:50.051131+0000","last_peered":"2024-09-21T07:48:50.051131+0000","last_clean":"2024-09-21T07:48:50.051131+0000","last_became_active":"2024-09-21T07:35:27.868105+0000","last_became_peered":"2024-09-21T07:35:27.868105+0000","last_unstale":"2024-09-21T07:48:50.051131+0000","last_undegraded":"2024-09-21T07:48:50.051131+0000","last_fullsized":"2024-09-21T07:48:50.051131+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:41.468702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":308,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.602167+0000","last_change":"2024-09-21T07:35:29.492974+0000","last_active":"2024-09-21T07:48:46.602167+0000","last_peered":"2024-09-21T07:48:46.602167+0000","last_clean":"2024-09-21T07:48:46.602167+0000","last_became_active":"2024-09-21T07:35:26.872165+0000","last_became_peered":"2024-09-21T07:35:26.872165+0000","last_unstale":"2024-09-21T07:48:46.602167+0000","last_undegraded":"2024-09-21T07:48:46.602167+0000","last_fullsized":"2024-09-21T07:48:46.602167+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:59.912520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013795179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":363,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.594739+0000","last_change":"2024-09-21T07:34:25.084861+0000","last_active":"2024-09-21T07:48:47.594739+0000","last_peered":"2024-09-21T07:48:47.594739+0000","last_clean":"2024-09-21T07:48:47.594739+0000","last_became_active":"2024-09-21T07:34:23.277708+0000","last_became_peered":"2024-09-21T07:34:23.277708+0000","last_unstale":"2024-09-21T07:48:47.594739+0000","last_undegraded":"2024-09-21T07:48:47.594739+0000","last_fullsized":"2024-09-21T07:48:47.594739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:40.816488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000597566,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.632205+0000","last_change":"2024-09-21T07:48:47.621555+0000","last_active":"2024-09-21T07:48:47.632205+0000","last_peered":"2024-09-21T07:48:47.632205+0000","last_clean":"2024-09-21T07:48:47.632205+0000","last_became_active":"2024-09-21T07:48:47.620805+0000","last_became_peered":"2024-09-21T07:48:47.620805+0000","last_unstale":"2024-09-21T07:48:47.632205+0000","last_undegraded":"2024-09-21T07:48:47.632205+0000","last_fullsized":"2024-09-21T07:48:47.632205+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:20.833555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":13,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.592490+0000","last_change":"2024-09-21T07:48:46.611660+0000","last_active":"2024-09-21T07:48:47.592490+0000","last_peered":"2024-09-21T07:48:47.592490+0000","last_clean":"2024-09-21T07:48:47.592490+0000","last_became_active":"2024-09-21T07:48:46.610843+0000","last_became_peered":"2024-09-21T07:48:46.610843+0000","last_unstale":"2024-09-21T07:48:47.592490+0000","last_undegraded":"2024-09-21T07:48:47.592490+0000","last_fullsized":"2024-09-21T07:48:47.592490+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:43:38.733833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":302,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051512+0000","last_change":"2024-09-21T07:35:27.874640+0000","last_active":"2024-09-21T07:48:50.051512+0000","last_peered":"2024-09-21T07:48:50.051512+0000","last_clean":"2024-09-21T07:48:50.051512+0000","last_became_active":"2024-09-21T07:35:27.874265+0000","last_became_peered":"2024-09-21T07:35:27.874265+0000","last_unstale":"2024-09-21T07:48:50.051512+0000","last_undegraded":"2024-09-21T07:48:50.051512+0000","last_fullsized":"2024-09-21T07:48:50.051512+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:45.754654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":312,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.606600+0000","last_change":"2024-09-21T07:35:29.207392+0000","last_active":"2024-09-21T07:48:47.606600+0000","last_peered":"2024-09-21T07:48:47.606600+0000","last_clean":"2024-09-21T07:48:47.606600+0000","last_became_active":"2024-09-21T07:35:26.870917+0000","last_became_peered":"2024-09-21T07:35:26.870917+0000","last_unstale":"2024-09-21T07:48:47.606600+0000","last_undegraded":"2024-09-21T07:48:47.606600+0000","last_fullsized":"2024-09-21T07:48:47.606600+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:59.159937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059198499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":370,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608972+0000","last_change":"2024-09-21T07:34:24.258709+0000","last_active":"2024-09-21T07:48:48.608972+0000","last_peered":"2024-09-21T07:48:48.608972+0000","last_clean":"2024-09-21T07:48:48.608972+0000","last_became_active":"2024-09-21T07:34:23.279042+0000","last_became_peered":"2024-09-21T07:34:23.279042+0000","last_unstale":"2024-09-21T07:48:48.608972+0000","last_undegraded":"2024-09-21T07:48:48.608972+0000","last_fullsized":"2024-09-21T07:48:48.608972+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:01.762117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019282100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608922+0000","last_change":"2024-09-21T07:48:47.607091+0000","last_active":"2024-09-21T07:48:48.608922+0000","last_peered":"2024-09-21T07:48:48.608922+0000","last_clean":"2024-09-21T07:48:48.608922+0000","last_became_active":"2024-09-21T07:48:47.606739+0000","last_became_peered":"2024-09-21T07:48:47.606739+0000","last_unstale":"2024-09-21T07:48:48.608922+0000","last_undegraded":"2024-09-21T07:48:48.608922+0000","last_fullsized":"2024-09-21T07:48:48.608922+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:42:23.765051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"218'2","reported_seq":15,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.628600+0000","last_change":"2024-09-21T07:48:46.612384+0000","last_active":"2024-09-21T07:48:47.628600+0000","last_peered":"2024-09-21T07:48:47.628600+0000","last_clean":"2024-09-21T07:48:47.628600+0000","last_became_active":"2024-09-21T07:48:46.610836+0000","last_became_peered":"2024-09-21T07:48:46.610836+0000","last_unstale":"2024-09-21T07:48:47.628600+0000","last_undegraded":"2024-09-21T07:48:47.628600+0000","last_fullsized":"2024-09-21T07:48:47.628600+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:04.310354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":304,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.598997+0000","last_change":"2024-09-21T07:35:27.870260+0000","last_active":"2024-09-21T07:48:47.598997+0000","last_peered":"2024-09-21T07:48:47.598997+0000","last_clean":"2024-09-21T07:48:47.598997+0000","last_became_active":"2024-09-21T07:35:27.869448+0000","last_became_peered":"2024-09-21T07:35:27.869448+0000","last_unstale":"2024-09-21T07:48:47.598997+0000","last_undegraded":"2024-09-21T07:48:47.598997+0000","last_fullsized":"2024-09-21T07:48:47.598997+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:16.946790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":309,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051488+0000","last_change":"2024-09-21T07:35:29.386367+0000","last_active":"2024-09-21T07:48:50.051488+0000","last_peered":"2024-09-21T07:48:50.051488+0000","last_clean":"2024-09-21T07:48:50.051488+0000","last_became_active":"2024-09-21T07:35:26.879238+0000","last_became_peered":"2024-09-21T07:35:26.879238+0000","last_unstale":"2024-09-21T07:48:50.051488+0000","last_undegraded":"2024-09-21T07:48:50.051488+0000","last_fullsized":"2024-09-21T07:48:50.051488+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:16.528260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092223100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":362,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.603686+0000","last_change":"2024-09-21T07:34:25.085635+0000","last_active":"2024-09-21T07:48:47.603686+0000","last_peered":"2024-09-21T07:48:47.603686+0000","last_clean":"2024-09-21T07:48:47.603686+0000","last_became_active":"2024-09-21T07:34:23.273566+0000","last_became_peered":"2024-09-21T07:34:23.273566+0000","last_unstale":"2024-09-21T07:48:47.603686+0000","last_undegraded":"2024-09-21T07:48:47.603686+0000","last_fullsized":"2024-09-21T07:48:47.603686+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:14.142272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079897300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.598103+0000","last_change":"2024-09-21T07:48:46.598103+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.598103+0000","last_undegraded":"2024-09-21T07:48:46.598103+0000","last_fullsized":"2024-09-21T07:48:46.598103+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609104+0000","last_change":"2024-09-21T07:48:46.610951+0000","last_active":"2024-09-21T07:48:48.609104+0000","last_peered":"2024-09-21T07:48:48.609104+0000","last_clean":"2024-09-21T07:48:48.609104+0000","last_became_active":"2024-09-21T07:48:46.610658+0000","last_became_peered":"2024-09-21T07:48:46.610658+0000","last_unstale":"2024-09-21T07:48:48.609104+0000","last_undegraded":"2024-09-21T07:48:48.609104+0000","last_fullsized":"2024-09-21T07:48:48.609104+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:15:20.903496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":304,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.596777+0000","last_change":"2024-09-21T07:35:27.878728+0000","last_active":"2024-09-21T07:48:47.596777+0000","last_peered":"2024-09-21T07:48:47.596777+0000","last_clean":"2024-09-21T07:48:47.596777+0000","last_became_active":"2024-09-21T07:35:27.878335+0000","last_became_peered":"2024-09-21T07:35:27.878335+0000","last_unstale":"2024-09-21T07:48:47.596777+0000","last_undegraded":"2024-09-21T07:48:47.596777+0000","last_fullsized":"2024-09-21T07:48:47.596777+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:25:37.731771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":322,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.591500+0000","last_change":"2024-09-21T07:35:28.858418+0000","last_active":"2024-09-21T07:48:46.591500+0000","last_peered":"2024-09-21T07:48:46.591500+0000","last_clean":"2024-09-21T07:48:46.591500+0000","last_became_active":"2024-09-21T07:35:26.876944+0000","last_became_peered":"2024-09-21T07:35:26.876944+0000","last_unstale":"2024-09-21T07:48:46.591500+0000","last_undegraded":"2024-09-21T07:48:46.591500+0000","last_fullsized":"2024-09-21T07:48:46.591500+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:04.514304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090862799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":358,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051805+0000","last_change":"2024-09-21T07:34:24.597926+0000","last_active":"2024-09-21T07:48:50.051805+0000","last_peered":"2024-09-21T07:48:50.051805+0000","last_clean":"2024-09-21T07:48:50.051805+0000","last_became_active":"2024-09-21T07:34:23.268926+0000","last_became_peered":"2024-09-21T07:34:23.268926+0000","last_unstale":"2024-09-21T07:48:50.051805+0000","last_undegraded":"2024-09-21T07:48:50.051805+0000","last_fullsized":"2024-09-21T07:48:50.051805+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:46.190752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101851,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.629415+0000","last_change":"2024-09-21T07:48:47.624905+0000","last_active":"2024-09-21T07:48:47.629415+0000","last_peered":"2024-09-21T07:48:47.629415+0000","last_clean":"2024-09-21T07:48:47.629415+0000","last_became_active":"2024-09-21T07:48:47.621782+0000","last_became_peered":"2024-09-21T07:48:47.621782+0000","last_unstale":"2024-09-21T07:48:47.629415+0000","last_undegraded":"2024-09-21T07:48:47.629415+0000","last_fullsized":"2024-09-21T07:48:47.629415+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:35.524796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051148+0000","last_change":"2024-09-21T07:48:46.613457+0000","last_active":"2024-09-21T07:48:50.051148+0000","last_peered":"2024-09-21T07:48:50.051148+0000","last_clean":"2024-09-21T07:48:50.051148+0000","last_became_active":"2024-09-21T07:48:46.612037+0000","last_became_peered":"2024-09-21T07:48:46.612037+0000","last_unstale":"2024-09-21T07:48:50.051148+0000","last_undegraded":"2024-09-21T07:48:50.051148+0000","last_fullsized":"2024-09-21T07:48:50.051148+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:06.200769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":303,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051801+0000","last_change":"2024-09-21T07:35:27.877426+0000","last_active":"2024-09-21T07:48:50.051801+0000","last_peered":"2024-09-21T07:48:50.051801+0000","last_clean":"2024-09-21T07:48:50.051801+0000","last_became_active":"2024-09-21T07:35:27.877050+0000","last_became_peered":"2024-09-21T07:35:27.877050+0000","last_unstale":"2024-09-21T07:48:50.051801+0000","last_undegraded":"2024-09-21T07:48:50.051801+0000","last_fullsized":"2024-09-21T07:48:50.051801+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:36.268334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":311,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.595671+0000","last_change":"2024-09-21T07:35:28.862273+0000","last_active":"2024-09-21T07:48:47.595671+0000","last_peered":"2024-09-21T07:48:47.595671+0000","last_clean":"2024-09-21T07:48:47.595671+0000","last_became_active":"2024-09-21T07:35:26.874011+0000","last_became_peered":"2024-09-21T07:35:26.874011+0000","last_unstale":"2024-09-21T07:48:47.595671+0000","last_undegraded":"2024-09-21T07:48:47.595671+0000","last_fullsized":"2024-09-21T07:48:47.595671+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:11.986950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071644799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":367,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.592690+0000","last_change":"2024-09-21T07:34:24.259794+0000","last_active":"2024-09-21T07:48:47.592690+0000","last_peered":"2024-09-21T07:48:47.592690+0000","last_clean":"2024-09-21T07:48:47.592690+0000","last_became_active":"2024-09-21T07:34:23.265679+0000","last_became_peered":"2024-09-21T07:34:23.265679+0000","last_unstale":"2024-09-21T07:48:47.592690+0000","last_undegraded":"2024-09-21T07:48:47.592690+0000","last_fullsized":"2024-09-21T07:48:47.592690+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:02.575796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001298518,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.630328+0000","last_change":"2024-09-21T07:48:47.618294+0000","last_active":"2024-09-21T07:48:47.630328+0000","last_peered":"2024-09-21T07:48:47.630328+0000","last_clean":"2024-09-21T07:48:47.630328+0000","last_became_active":"2024-09-21T07:48:47.617975+0000","last_became_peered":"2024-09-21T07:48:47.617975+0000","last_unstale":"2024-09-21T07:48:47.630328+0000","last_undegraded":"2024-09-21T07:48:47.630328+0000","last_fullsized":"2024-09-21T07:48:47.630328+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:42.345168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.610511+0000","last_change":"2024-09-21T07:48:46.608424+0000","last_active":"2024-09-21T07:48:46.610511+0000","last_peered":"2024-09-21T07:48:46.610511+0000","last_clean":"2024-09-21T07:48:46.610511+0000","last_became_active":"2024-09-21T07:48:46.607778+0000","last_became_peered":"2024-09-21T07:48:46.607778+0000","last_unstale":"2024-09-21T07:48:46.610511+0000","last_undegraded":"2024-09-21T07:48:46.610511+0000","last_fullsized":"2024-09-21T07:48:46.610511+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:59.561123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":306,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.609776+0000","last_change":"2024-09-21T07:35:27.877725+0000","last_active":"2024-09-21T07:48:47.609776+0000","last_peered":"2024-09-21T07:48:47.609776+0000","last_clean":"2024-09-21T07:48:47.609776+0000","last_became_active":"2024-09-21T07:35:27.877398+0000","last_became_peered":"2024-09-21T07:35:27.877398+0000","last_unstale":"2024-09-21T07:48:47.609776+0000","last_undegraded":"2024-09-21T07:48:47.609776+0000","last_fullsized":"2024-09-21T07:48:47.609776+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:05.859762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":314,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.593198+0000","last_change":"2024-09-21T07:35:28.860612+0000","last_active":"2024-09-21T07:48:47.593198+0000","last_peered":"2024-09-21T07:48:47.593198+0000","last_clean":"2024-09-21T07:48:47.593198+0000","last_became_active":"2024-09-21T07:35:26.876562+0000","last_became_peered":"2024-09-21T07:35:26.876562+0000","last_unstale":"2024-09-21T07:48:47.593198+0000","last_undegraded":"2024-09-21T07:48:47.593198+0000","last_fullsized":"2024-09-21T07:48:47.593198+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:06.824331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012252470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":361,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.602500+0000","last_change":"2024-09-21T07:34:24.260330+0000","last_active":"2024-09-21T07:48:46.602500+0000","last_peered":"2024-09-21T07:48:46.602500+0000","last_clean":"2024-09-21T07:48:46.602500+0000","last_became_active":"2024-09-21T07:34:23.637210+0000","last_became_peered":"2024-09-21T07:34:23.637210+0000","last_unstale":"2024-09-21T07:48:46.602500+0000","last_undegraded":"2024-09-21T07:48:46.602500+0000","last_fullsized":"2024-09-21T07:48:46.602500+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.488451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001537093,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.632407+0000","last_change":"2024-09-21T07:48:47.626890+0000","last_active":"2024-09-21T07:48:47.632407+0000","last_peered":"2024-09-21T07:48:47.632407+0000","last_clean":"2024-09-21T07:48:47.632407+0000","last_became_active":"2024-09-21T07:48:47.626088+0000","last_became_peered":"2024-09-21T07:48:47.626088+0000","last_unstale":"2024-09-21T07:48:47.632407+0000","last_undegraded":"2024-09-21T07:48:47.632407+0000","last_fullsized":"2024-09-21T07:48:47.632407+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:56:05.110865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.7","version":"218'2","reported_seq":15,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.625284+0000","last_change":"2024-09-21T07:48:46.607721+0000","last_active":"2024-09-21T07:48:47.625284+0000","last_peered":"2024-09-21T07:48:47.625284+0000","last_clean":"2024-09-21T07:48:47.625284+0000","last_became_active":"2024-09-21T07:48:46.607413+0000","last_became_peered":"2024-09-21T07:48:46.607413+0000","last_unstale":"2024-09-21T07:48:47.625284+0000","last_undegraded":"2024-09-21T07:48:47.625284+0000","last_fullsized":"2024-09-21T07:48:47.625284+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:47.152954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":304,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.594537+0000","last_change":"2024-09-21T07:35:27.874757+0000","last_active":"2024-09-21T07:48:47.594537+0000","last_peered":"2024-09-21T07:48:47.594537+0000","last_clean":"2024-09-21T07:48:47.594537+0000","last_became_active":"2024-09-21T07:35:27.874360+0000","last_became_peered":"2024-09-21T07:35:27.874360+0000","last_unstale":"2024-09-21T07:48:47.594537+0000","last_undegraded":"2024-09-21T07:48:47.594537+0000","last_fullsized":"2024-09-21T07:48:47.594537+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:17.646912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":308,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.602783+0000","last_change":"2024-09-21T07:35:29.493431+0000","last_active":"2024-09-21T07:48:46.602783+0000","last_peered":"2024-09-21T07:48:46.602783+0000","last_clean":"2024-09-21T07:48:46.602783+0000","last_became_active":"2024-09-21T07:35:26.873691+0000","last_became_peered":"2024-09-21T07:35:26.873691+0000","last_unstale":"2024-09-21T07:48:46.602783+0000","last_undegraded":"2024-09-21T07:48:46.602783+0000","last_fullsized":"2024-09-21T07:48:46.602783+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:51.705051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001162464,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":358,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051405+0000","last_change":"2024-09-21T07:34:24.597286+0000","last_active":"2024-09-21T07:48:50.051405+0000","last_peered":"2024-09-21T07:48:50.051405+0000","last_clean":"2024-09-21T07:48:50.051405+0000","last_became_active":"2024-09-21T07:34:23.267133+0000","last_became_peered":"2024-09-21T07:34:23.267133+0000","last_unstale":"2024-09-21T07:48:50.051405+0000","last_undegraded":"2024-09-21T07:48:50.051405+0000","last_fullsized":"2024-09-21T07:48:50.051405+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:18.257010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.627524+0000","last_change":"2024-09-21T07:48:47.624322+0000","last_active":"2024-09-21T07:48:47.627524+0000","last_peered":"2024-09-21T07:48:47.627524+0000","last_clean":"2024-09-21T07:48:47.627524+0000","last_became_active":"2024-09-21T07:48:47.623920+0000","last_became_peered":"2024-09-21T07:48:47.623920+0000","last_unstale":"2024-09-21T07:48:47.627524+0000","last_undegraded":"2024-09-21T07:48:47.627524+0000","last_fullsized":"2024-09-21T07:48:47.627524+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:31.490205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":13,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.605003+0000","last_change":"2024-09-21T07:48:46.599263+0000","last_active":"2024-09-21T07:48:47.605003+0000","last_peered":"2024-09-21T07:48:47.605003+0000","last_clean":"2024-09-21T07:48:47.605003+0000","last_became_active":"2024-09-21T07:48:46.597628+0000","last_became_peered":"2024-09-21T07:48:46.597628+0000","last_unstale":"2024-09-21T07:48:47.605003+0000","last_undegraded":"2024-09-21T07:48:47.605003+0000","last_fullsized":"2024-09-21T07:48:47.605003+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:38.541554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":315,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.593982+0000","last_change":"2024-09-21T07:35:27.864458+0000","last_active":"2024-09-21T07:48:46.593982+0000","last_peered":"2024-09-21T07:48:46.593982+0000","last_clean":"2024-09-21T07:48:46.593982+0000","last_became_active":"2024-09-21T07:35:27.864136+0000","last_became_peered":"2024-09-21T07:35:27.864136+0000","last_unstale":"2024-09-21T07:48:46.593982+0000","last_undegraded":"2024-09-21T07:48:46.593982+0000","last_fullsized":"2024-09-21T07:48:46.593982+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:20.162240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":322,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.594391+0000","last_change":"2024-09-21T07:35:28.858784+0000","last_active":"2024-09-21T07:48:46.594391+0000","last_peered":"2024-09-21T07:48:46.594391+0000","last_clean":"2024-09-21T07:48:46.594391+0000","last_became_active":"2024-09-21T07:35:26.876890+0000","last_became_peered":"2024-09-21T07:35:26.876890+0000","last_unstale":"2024-09-21T07:48:46.594391+0000","last_undegraded":"2024-09-21T07:48:46.594391+0000","last_fullsized":"2024-09-21T07:48:46.594391+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:40.738327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064543499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":369,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609757+0000","last_change":"2024-09-21T07:34:24.258712+0000","last_active":"2024-09-21T07:48:48.609757+0000","last_peered":"2024-09-21T07:48:48.609757+0000","last_clean":"2024-09-21T07:48:48.609757+0000","last_became_active":"2024-09-21T07:34:23.274692+0000","last_became_peered":"2024-09-21T07:34:23.274692+0000","last_unstale":"2024-09-21T07:48:48.609757+0000","last_undegraded":"2024-09-21T07:48:48.609757+0000","last_fullsized":"2024-09-21T07:48:48.609757+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:43.279454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018051479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.623521+0000","last_change":"2024-09-21T07:48:47.617896+0000","last_active":"2024-09-21T07:48:47.623521+0000","last_peered":"2024-09-21T07:48:47.623521+0000","last_clean":"2024-09-21T07:48:47.623521+0000","last_became_active":"2024-09-21T07:48:47.617588+0000","last_became_peered":"2024-09-21T07:48:47.617588+0000","last_unstale":"2024-09-21T07:48:47.623521+0000","last_undegraded":"2024-09-21T07:48:47.623521+0000","last_fullsized":"2024-09-21T07:48:47.623521+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:46.539795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051487+0000","last_change":"2024-09-21T07:48:46.609456+0000","last_active":"2024-09-21T07:48:50.051487+0000","last_peered":"2024-09-21T07:48:50.051487+0000","last_clean":"2024-09-21T07:48:50.051487+0000","last_became_active":"2024-09-21T07:48:46.609142+0000","last_became_peered":"2024-09-21T07:48:46.609142+0000","last_unstale":"2024-09-21T07:48:50.051487+0000","last_undegraded":"2024-09-21T07:48:50.051487+0000","last_fullsized":"2024-09-21T07:48:50.051487+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:21.813300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":308,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609370+0000","last_change":"2024-09-21T07:35:27.878020+0000","last_active":"2024-09-21T07:48:48.609370+0000","last_peered":"2024-09-21T07:48:48.609370+0000","last_clean":"2024-09-21T07:48:48.609370+0000","last_became_active":"2024-09-21T07:35:27.877532+0000","last_became_peered":"2024-09-21T07:35:27.877532+0000","last_unstale":"2024-09-21T07:48:48.609370+0000","last_undegraded":"2024-09-21T07:48:48.609370+0000","last_fullsized":"2024-09-21T07:48:48.609370+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:19.756073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":314,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.592718+0000","last_change":"2024-09-21T07:35:28.861049+0000","last_active":"2024-09-21T07:48:47.592718+0000","last_peered":"2024-09-21T07:48:47.592718+0000","last_clean":"2024-09-21T07:48:47.592718+0000","last_became_active":"2024-09-21T07:35:26.876819+0000","last_became_peered":"2024-09-21T07:35:26.876819+0000","last_unstale":"2024-09-21T07:48:47.592718+0000","last_undegraded":"2024-09-21T07:48:47.592718+0000","last_fullsized":"2024-09-21T07:48:47.592718+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:19.633803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055780599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":362,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.594656+0000","last_change":"2024-09-21T07:34:25.084849+0000","last_active":"2024-09-21T07:48:47.594656+0000","last_peered":"2024-09-21T07:48:47.594656+0000","last_clean":"2024-09-21T07:48:47.594656+0000","last_became_active":"2024-09-21T07:34:23.277713+0000","last_became_peered":"2024-09-21T07:34:23.277713+0000","last_unstale":"2024-09-21T07:48:47.594656+0000","last_undegraded":"2024-09-21T07:48:47.594656+0000","last_fullsized":"2024-09-21T07:48:47.594656+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:05.811810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059079499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.590874+0000","last_change":"2024-09-21T07:48:46.590874+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.590874+0000","last_undegraded":"2024-09-21T07:48:46.590874+0000","last_fullsized":"2024-09-21T07:48:46.590874+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.610468+0000","last_change":"2024-09-21T07:48:46.607396+0000","last_active":"2024-09-21T07:48:46.610468+0000","last_peered":"2024-09-21T07:48:46.610468+0000","last_clean":"2024-09-21T07:48:46.610468+0000","last_became_active":"2024-09-21T07:48:46.607082+0000","last_became_peered":"2024-09-21T07:48:46.607082+0000","last_unstale":"2024-09-21T07:48:46.610468+0000","last_undegraded":"2024-09-21T07:48:46.610468+0000","last_fullsized":"2024-09-21T07:48:46.610468+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:20.897148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":306,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.601794+0000","last_change":"2024-09-21T07:35:27.884248+0000","last_active":"2024-09-21T07:48:47.601794+0000","last_peered":"2024-09-21T07:48:47.601794+0000","last_clean":"2024-09-21T07:48:47.601794+0000","last_became_active":"2024-09-21T07:35:27.883861+0000","last_became_peered":"2024-09-21T07:35:27.883861+0000","last_unstale":"2024-09-21T07:48:47.601794+0000","last_undegraded":"2024-09-21T07:48:47.601794+0000","last_fullsized":"2024-09-21T07:48:47.601794+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":312,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.606467+0000","last_change":"2024-09-21T07:35:29.207135+0000","last_active":"2024-09-21T07:48:47.606467+0000","last_peered":"2024-09-21T07:48:47.606467+0000","last_clean":"2024-09-21T07:48:47.606467+0000","last_became_active":"2024-09-21T07:35:26.872787+0000","last_became_peered":"2024-09-21T07:35:26.872787+0000","last_unstale":"2024-09-21T07:48:47.606467+0000","last_undegraded":"2024-09-21T07:48:47.606467+0000","last_fullsized":"2024-09-21T07:48:47.606467+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:27.543217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011157509999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":374,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.594088+0000","last_change":"2024-09-21T07:34:24.538610+0000","last_active":"2024-09-21T07:48:46.594088+0000","last_peered":"2024-09-21T07:48:46.594088+0000","last_clean":"2024-09-21T07:48:46.594088+0000","last_became_active":"2024-09-21T07:34:23.259700+0000","last_became_peered":"2024-09-21T07:34:23.259700+0000","last_unstale":"2024-09-21T07:48:46.594088+0000","last_undegraded":"2024-09-21T07:48:46.594088+0000","last_fullsized":"2024-09-21T07:48:46.594088+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:57.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070522399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.632186+0000","last_change":"2024-09-21T07:48:47.623345+0000","last_active":"2024-09-21T07:48:47.632186+0000","last_peered":"2024-09-21T07:48:47.632186+0000","last_clean":"2024-09-21T07:48:47.632186+0000","last_became_active":"2024-09-21T07:48:47.622998+0000","last_became_peered":"2024-09-21T07:48:47.622998+0000","last_unstale":"2024-09-21T07:48:47.632186+0000","last_undegraded":"2024-09-21T07:48:47.632186+0000","last_fullsized":"2024-09-21T07:48:47.632186+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:26:23.537534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":13,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.605671+0000","last_change":"2024-09-21T07:48:46.599294+0000","last_active":"2024-09-21T07:48:47.605671+0000","last_peered":"2024-09-21T07:48:47.605671+0000","last_clean":"2024-09-21T07:48:47.605671+0000","last_became_active":"2024-09-21T07:48:46.598633+0000","last_became_peered":"2024-09-21T07:48:46.598633+0000","last_unstale":"2024-09-21T07:48:47.605671+0000","last_undegraded":"2024-09-21T07:48:47.605671+0000","last_fullsized":"2024-09-21T07:48:47.605671+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:34.227316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":315,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.592285+0000","last_change":"2024-09-21T07:35:27.868024+0000","last_active":"2024-09-21T07:48:46.592285+0000","last_peered":"2024-09-21T07:48:46.592285+0000","last_clean":"2024-09-21T07:48:46.592285+0000","last_became_active":"2024-09-21T07:35:27.867339+0000","last_became_peered":"2024-09-21T07:35:27.867339+0000","last_unstale":"2024-09-21T07:48:46.592285+0000","last_undegraded":"2024-09-21T07:48:46.592285+0000","last_fullsized":"2024-09-21T07:48:46.592285+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":322,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.591127+0000","last_change":"2024-09-21T07:35:28.858505+0000","last_active":"2024-09-21T07:48:46.591127+0000","last_peered":"2024-09-21T07:48:46.591127+0000","last_clean":"2024-09-21T07:48:46.591127+0000","last_became_active":"2024-09-21T07:35:26.877320+0000","last_became_peered":"2024-09-21T07:35:26.877320+0000","last_unstale":"2024-09-21T07:48:46.591127+0000","last_undegraded":"2024-09-21T07:48:46.591127+0000","last_fullsized":"2024-09-21T07:48:46.591127+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:53.672782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244119999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":360,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.052005+0000","last_change":"2024-09-21T07:34:24.597597+0000","last_active":"2024-09-21T07:48:50.052005+0000","last_peered":"2024-09-21T07:48:50.052005+0000","last_clean":"2024-09-21T07:48:50.052005+0000","last_became_active":"2024-09-21T07:34:23.635204+0000","last_became_peered":"2024-09-21T07:34:23.635204+0000","last_unstale":"2024-09-21T07:48:50.052005+0000","last_undegraded":"2024-09-21T07:48:50.052005+0000","last_fullsized":"2024-09-21T07:48:50.052005+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:48.644789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081139199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.597041+0000","last_change":"2024-09-21T07:48:46.597041+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.597041+0000","last_undegraded":"2024-09-21T07:48:46.597041+0000","last_fullsized":"2024-09-21T07:48:46.597041+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":13,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.605822+0000","last_change":"2024-09-21T07:48:46.602945+0000","last_active":"2024-09-21T07:48:47.605822+0000","last_peered":"2024-09-21T07:48:47.605822+0000","last_clean":"2024-09-21T07:48:47.605822+0000","last_became_active":"2024-09-21T07:48:46.602486+0000","last_became_peered":"2024-09-21T07:48:46.602486+0000","last_unstale":"2024-09-21T07:48:47.605822+0000","last_undegraded":"2024-09-21T07:48:47.605822+0000","last_fullsized":"2024-09-21T07:48:47.605822+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:51:05.478321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":305,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.606002+0000","last_change":"2024-09-21T07:35:27.877503+0000","last_active":"2024-09-21T07:48:47.606002+0000","last_peered":"2024-09-21T07:48:47.606002+0000","last_clean":"2024-09-21T07:48:47.606002+0000","last_became_active":"2024-09-21T07:35:27.875775+0000","last_became_peered":"2024-09-21T07:35:27.875775+0000","last_unstale":"2024-09-21T07:48:47.606002+0000","last_undegraded":"2024-09-21T07:48:47.606002+0000","last_fullsized":"2024-09-21T07:48:47.606002+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:49.291023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":308,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.597282+0000","last_change":"2024-09-21T07:35:29.491412+0000","last_active":"2024-09-21T07:48:46.597282+0000","last_peered":"2024-09-21T07:48:46.597282+0000","last_clean":"2024-09-21T07:48:46.597282+0000","last_became_active":"2024-09-21T07:35:26.876404+0000","last_became_peered":"2024-09-21T07:35:26.876404+0000","last_unstale":"2024-09-21T07:48:46.597282+0000","last_undegraded":"2024-09-21T07:48:46.597282+0000","last_fullsized":"2024-09-21T07:48:46.597282+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063918000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":370,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.610303+0000","last_change":"2024-09-21T07:34:24.259026+0000","last_active":"2024-09-21T07:48:48.610303+0000","last_peered":"2024-09-21T07:48:48.610303+0000","last_clean":"2024-09-21T07:48:48.610303+0000","last_became_active":"2024-09-21T07:34:23.635456+0000","last_became_peered":"2024-09-21T07:34:23.635456+0000","last_unstale":"2024-09-21T07:48:48.610303+0000","last_undegraded":"2024-09-21T07:48:48.610303+0000","last_fullsized":"2024-09-21T07:48:48.610303+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:09.592752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018654419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":1,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-21T07:48:46.598035+0000","last_change":"2024-09-21T07:48:46.598035+0000","last_active":"2024-09-21T07:48:46.575553+0000","last_peered":"2024-09-21T07:48:46.575553+0000","last_clean":"2024-09-21T07:48:46.575553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:48:46.598035+0000","last_undegraded":"2024-09-21T07:48:46.598035+0000","last_fullsized":"2024-09-21T07:48:46.598035+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":10,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.613990+0000","last_change":"2024-09-21T07:48:46.612048+0000","last_active":"2024-09-21T07:48:46.613990+0000","last_peered":"2024-09-21T07:48:46.613990+0000","last_clean":"2024-09-21T07:48:46.613990+0000","last_became_active":"2024-09-21T07:48:46.611755+0000","last_became_peered":"2024-09-21T07:48:46.611755+0000","last_unstale":"2024-09-21T07:48:46.613990+0000","last_undegraded":"2024-09-21T07:48:46.613990+0000","last_fullsized":"2024-09-21T07:48:46.613990+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:12:05.809707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":305,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.605145+0000","last_change":"2024-09-21T07:35:27.871984+0000","last_active":"2024-09-21T07:48:47.605145+0000","last_peered":"2024-09-21T07:48:47.605145+0000","last_clean":"2024-09-21T07:48:47.605145+0000","last_became_active":"2024-09-21T07:35:27.870336+0000","last_became_peered":"2024-09-21T07:35:27.870336+0000","last_unstale":"2024-09-21T07:48:47.605145+0000","last_undegraded":"2024-09-21T07:48:47.605145+0000","last_fullsized":"2024-09-21T07:48:47.605145+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:57.120479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":314,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.592628+0000","last_change":"2024-09-21T07:35:28.860351+0000","last_active":"2024-09-21T07:48:47.592628+0000","last_peered":"2024-09-21T07:48:47.592628+0000","last_clean":"2024-09-21T07:48:47.592628+0000","last_became_active":"2024-09-21T07:35:26.872046+0000","last_became_peered":"2024-09-21T07:35:26.872046+0000","last_unstale":"2024-09-21T07:48:47.592628+0000","last_undegraded":"2024-09-21T07:48:47.592628+0000","last_fullsized":"2024-09-21T07:48:47.592628+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:47.074327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098384399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":361,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-21T07:48:46.600950+0000","last_change":"2024-09-21T07:34:24.260300+0000","last_active":"2024-09-21T07:48:46.600950+0000","last_peered":"2024-09-21T07:48:46.600950+0000","last_clean":"2024-09-21T07:48:46.600950+0000","last_became_active":"2024-09-21T07:34:23.270128+0000","last_became_peered":"2024-09-21T07:34:23.270128+0000","last_unstale":"2024-09-21T07:48:46.600950+0000","last_undegraded":"2024-09-21T07:48:46.600950+0000","last_fullsized":"2024-09-21T07:48:46.600950+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:25.260931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00176651,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":10,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.628380+0000","last_change":"2024-09-21T07:48:47.626058+0000","last_active":"2024-09-21T07:48:47.628380+0000","last_peered":"2024-09-21T07:48:47.628380+0000","last_clean":"2024-09-21T07:48:47.628380+0000","last_became_active":"2024-09-21T07:48:47.625779+0000","last_became_peered":"2024-09-21T07:48:47.625779+0000","last_unstale":"2024-09-21T07:48:47.628380+0000","last_undegraded":"2024-09-21T07:48:47.628380+0000","last_fullsized":"2024-09-21T07:48:47.628380+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:03.169164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":13,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.593455+0000","last_change":"2024-09-21T07:48:46.607858+0000","last_active":"2024-09-21T07:48:47.593455+0000","last_peered":"2024-09-21T07:48:47.593455+0000","last_clean":"2024-09-21T07:48:47.593455+0000","last_became_active":"2024-09-21T07:48:46.607521+0000","last_became_peered":"2024-09-21T07:48:46.607521+0000","last_unstale":"2024-09-21T07:48:47.593455+0000","last_undegraded":"2024-09-21T07:48:47.593455+0000","last_fullsized":"2024-09-21T07:48:47.593455+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:18.298760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":305,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.609882+0000","last_change":"2024-09-21T07:35:27.877861+0000","last_active":"2024-09-21T07:48:47.609882+0000","last_peered":"2024-09-21T07:48:47.609882+0000","last_clean":"2024-09-21T07:48:47.609882+0000","last_became_active":"2024-09-21T07:35:27.877434+0000","last_became_peered":"2024-09-21T07:35:27.877434+0000","last_unstale":"2024-09-21T07:48:47.609882+0000","last_undegraded":"2024-09-21T07:48:47.609882+0000","last_fullsized":"2024-09-21T07:48:47.609882+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":311,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.597277+0000","last_change":"2024-09-21T07:35:28.862954+0000","last_active":"2024-09-21T07:48:47.597277+0000","last_peered":"2024-09-21T07:48:47.597277+0000","last_clean":"2024-09-21T07:48:47.597277+0000","last_became_active":"2024-09-21T07:35:26.870613+0000","last_became_peered":"2024-09-21T07:35:26.870613+0000","last_unstale":"2024-09-21T07:48:47.597277+0000","last_undegraded":"2024-09-21T07:48:47.597277+0000","last_fullsized":"2024-09-21T07:48:47.597277+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:06.901043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059842100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":364,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-21T07:48:47.613383+0000","last_change":"2024-09-21T07:34:25.028904+0000","last_active":"2024-09-21T07:48:47.613383+0000","last_peered":"2024-09-21T07:48:47.613383+0000","last_clean":"2024-09-21T07:48:47.613383+0000","last_became_active":"2024-09-21T07:34:23.260942+0000","last_became_peered":"2024-09-21T07:34:23.260942+0000","last_unstale":"2024-09-21T07:48:47.613383+0000","last_undegraded":"2024-09-21T07:48:47.613383+0000","last_fullsized":"2024-09-21T07:48:47.613383+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:22.709328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060553899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":776,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27,"ondisk_log_size":27,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509632,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66372,"kb_used_data":3248,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94305468,"statfs":{"total":96636764160,"available":96568799232,"internally_reserved":0,"allocated":3325952,"data_stored":2046648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509634,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51908,"kb_used_data":2668,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319932,"statfs":{"total":96636764160,"available":96583610368,"internally_reserved":0,"allocated":2732032,"data_stored":1456280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509632,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33860,"kb_used_data":2668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337980,"statfs":{"total":96636764160,"available":96602091520,"internally_reserved":0,"allocated":2732032,"data_stored":1456280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509633,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29696,"kb_used_data":2668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342144,"statfs":{"total":96636764160,"available":96606355456,"internally_reserved":0,"allocated":2732032,"data_stored":1456280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509634,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47748,"kb_used_data":2668,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324092,"statfs":{"total":96636764160,"available":96587870208,"internally_reserved":0,"allocated":2732032,"data_stored":1456280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509634,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52492,"kb_used_data":3248,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319348,"statfs":{"total":96636764160,"available":96583012352,"internally_reserved":0,"allocated":3325952,"data_stored":2046648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542338,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51936,"kb_used_data":2696,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319904,"statfs":{"total":96636764160,"available":96583581696,"internally_reserved":0,"allocated":2760704,"data_stored":1472495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542337,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74148,"kb_used_data":2704,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297692,"statfs":{"total":96636764160,"available":96560836608,"internally_reserved":0,"allocated":2768896,"data_stored":1472039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:48:53.127 INFO:tasks.ceph.ceph_manager.ceph:PG 28.3f is not active+clean 2024-09-21T07:48:53.127 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '28.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 217, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:48:46.590080+0000', 'last_change': '2024-09-21T07:48:46.590080+0000', 'last_active': '2024-09-21T07:48:46.575553+0000', 'last_peered': '2024-09-21T07:48:46.575553+0000', 'last_clean': '2024-09-21T07:48:46.575553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:48:46.590080+0000', 'last_undegraded': '2024-09-21T07:48:46.590080+0000', 'last_fullsized': '2024-09-21T07:48:46.590080+0000', 'mapping_epoch': 217, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 217, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'last_clean_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T07:48:53.127 INFO:tasks.ceph.ceph_manager.ceph:PG 28.3c is not active+clean 2024-09-21T07:48:53.127 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '28.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 217, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:48:46.600167+0000', 'last_change': '2024-09-21T07:48:46.600167+0000', 'last_active': '2024-09-21T07:48:46.575553+0000', 'last_peered': '2024-09-21T07:48:46.575553+0000', 'last_clean': '2024-09-21T07:48:46.575553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:48:46.600167+0000', 'last_undegraded': '2024-09-21T07:48:46.600167+0000', 'last_fullsized': '2024-09-21T07:48:46.600167+0000', 'mapping_epoch': 217, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 217, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'last_clean_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T07:48:53.128 INFO:tasks.ceph.ceph_manager.ceph:PG 28.3a is not active+clean 2024-09-21T07:48:53.128 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '28.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 217, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:48:46.597012+0000', 'last_change': '2024-09-21T07:48:46.597012+0000', 'last_active': '2024-09-21T07:48:46.575553+0000', 'last_peered': '2024-09-21T07:48:46.575553+0000', 'last_clean': '2024-09-21T07:48:46.575553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:48:46.597012+0000', 'last_undegraded': '2024-09-21T07:48:46.597012+0000', 'last_fullsized': '2024-09-21T07:48:46.597012+0000', 'mapping_epoch': 217, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 217, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'last_clean_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T07:48:53.129 INFO:tasks.ceph.ceph_manager.ceph:PG 28.33 is not active+clean 2024-09-21T07:48:53.129 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '28.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 217, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:48:46.599782+0000', 'last_change': '2024-09-21T07:48:46.599782+0000', 'last_active': '2024-09-21T07:48:46.575553+0000', 'last_peered': '2024-09-21T07:48:46.575553+0000', 'last_clean': '2024-09-21T07:48:46.575553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:48:46.599782+0000', 'last_undegraded': '2024-09-21T07:48:46.599782+0000', 'last_fullsized': '2024-09-21T07:48:46.599782+0000', 'mapping_epoch': 217, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 217, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'last_clean_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T07:48:53.130 INFO:tasks.ceph.ceph_manager.ceph:PG 28.26 is not active+clean 2024-09-21T07:48:53.130 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '28.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 217, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:48:46.592481+0000', 'last_change': '2024-09-21T07:48:46.592481+0000', 'last_active': '2024-09-21T07:48:46.575553+0000', 'last_peered': '2024-09-21T07:48:46.575553+0000', 'last_clean': '2024-09-21T07:48:46.575553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:48:46.592481+0000', 'last_undegraded': '2024-09-21T07:48:46.592481+0000', 'last_fullsized': '2024-09-21T07:48:46.592481+0000', 'mapping_epoch': 217, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 217, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'last_clean_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T07:48:53.130 INFO:tasks.ceph.ceph_manager.ceph:PG 28.21 is not active+clean 2024-09-21T07:48:53.130 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '28.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 217, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:48:46.595955+0000', 'last_change': '2024-09-21T07:48:46.595955+0000', 'last_active': '2024-09-21T07:48:46.575553+0000', 'last_peered': '2024-09-21T07:48:46.575553+0000', 'last_clean': '2024-09-21T07:48:46.575553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:48:46.595955+0000', 'last_undegraded': '2024-09-21T07:48:46.595955+0000', 'last_fullsized': '2024-09-21T07:48:46.595955+0000', 'mapping_epoch': 217, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 217, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'last_clean_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T07:48:53.130 INFO:tasks.ceph.ceph_manager.ceph:PG 28.20 is not active+clean 2024-09-21T07:48:53.130 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '28.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 217, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:48:46.595810+0000', 'last_change': '2024-09-21T07:48:46.595810+0000', 'last_active': '2024-09-21T07:48:46.575553+0000', 'last_peered': '2024-09-21T07:48:46.575553+0000', 'last_clean': '2024-09-21T07:48:46.575553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:48:46.595810+0000', 'last_undegraded': '2024-09-21T07:48:46.595810+0000', 'last_fullsized': '2024-09-21T07:48:46.595810+0000', 'mapping_epoch': 217, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 217, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'last_clean_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T07:48:53.130 INFO:tasks.ceph.ceph_manager.ceph:PG 28.1d is not active+clean 2024-09-21T07:48:53.130 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '28.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 217, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:48:46.591446+0000', 'last_change': '2024-09-21T07:48:46.591446+0000', 'last_active': '2024-09-21T07:48:46.575553+0000', 'last_peered': '2024-09-21T07:48:46.575553+0000', 'last_clean': '2024-09-21T07:48:46.575553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:48:46.591446+0000', 'last_undegraded': '2024-09-21T07:48:46.591446+0000', 'last_fullsized': '2024-09-21T07:48:46.591446+0000', 'mapping_epoch': 217, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 217, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'last_clean_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T07:48:53.130 INFO:tasks.ceph.ceph_manager.ceph:PG 28.19 is not active+clean 2024-09-21T07:48:53.130 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '28.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 217, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:48:46.588781+0000', 'last_change': '2024-09-21T07:48:46.588781+0000', 'last_active': '2024-09-21T07:48:46.575553+0000', 'last_peered': '2024-09-21T07:48:46.575553+0000', 'last_clean': '2024-09-21T07:48:46.575553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:48:46.588781+0000', 'last_undegraded': '2024-09-21T07:48:46.588781+0000', 'last_fullsized': '2024-09-21T07:48:46.588781+0000', 'mapping_epoch': 217, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 217, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'last_clean_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T07:48:53.131 INFO:tasks.ceph.ceph_manager.ceph:PG 28.17 is not active+clean 2024-09-21T07:48:53.131 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '28.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 217, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:48:46.596453+0000', 'last_change': '2024-09-21T07:48:46.596453+0000', 'last_active': '2024-09-21T07:48:46.575553+0000', 'last_peered': '2024-09-21T07:48:46.575553+0000', 'last_clean': '2024-09-21T07:48:46.575553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:48:46.596453+0000', 'last_undegraded': '2024-09-21T07:48:46.596453+0000', 'last_fullsized': '2024-09-21T07:48:46.596453+0000', 'mapping_epoch': 217, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 217, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'last_clean_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T07:48:53.131 INFO:tasks.ceph.ceph_manager.ceph:PG 28.9 is not active+clean 2024-09-21T07:48:53.131 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '28.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 217, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:48:46.589360+0000', 'last_change': '2024-09-21T07:48:46.589360+0000', 'last_active': '2024-09-21T07:48:46.575553+0000', 'last_peered': '2024-09-21T07:48:46.575553+0000', 'last_clean': '2024-09-21T07:48:46.575553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:48:46.589360+0000', 'last_undegraded': '2024-09-21T07:48:46.589360+0000', 'last_fullsized': '2024-09-21T07:48:46.589360+0000', 'mapping_epoch': 217, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 217, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'last_clean_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T07:48:53.131 INFO:tasks.ceph.ceph_manager.ceph:PG 28.a is not active+clean 2024-09-21T07:48:53.131 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '28.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 217, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:48:46.591252+0000', 'last_change': '2024-09-21T07:48:46.591252+0000', 'last_active': '2024-09-21T07:48:46.575553+0000', 'last_peered': '2024-09-21T07:48:46.575553+0000', 'last_clean': '2024-09-21T07:48:46.575553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:48:46.591252+0000', 'last_undegraded': '2024-09-21T07:48:46.591252+0000', 'last_fullsized': '2024-09-21T07:48:46.591252+0000', 'mapping_epoch': 217, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 217, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'last_clean_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T07:48:53.131 INFO:tasks.ceph.ceph_manager.ceph:PG 28.d is not active+clean 2024-09-21T07:48:53.131 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '28.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 217, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:48:46.598103+0000', 'last_change': '2024-09-21T07:48:46.598103+0000', 'last_active': '2024-09-21T07:48:46.575553+0000', 'last_peered': '2024-09-21T07:48:46.575553+0000', 'last_clean': '2024-09-21T07:48:46.575553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:48:46.598103+0000', 'last_undegraded': '2024-09-21T07:48:46.598103+0000', 'last_fullsized': '2024-09-21T07:48:46.598103+0000', 'mapping_epoch': 217, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 217, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'last_clean_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T07:48:53.131 INFO:tasks.ceph.ceph_manager.ceph:PG 28.3 is not active+clean 2024-09-21T07:48:53.131 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '28.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 217, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:48:46.590874+0000', 'last_change': '2024-09-21T07:48:46.590874+0000', 'last_active': '2024-09-21T07:48:46.575553+0000', 'last_peered': '2024-09-21T07:48:46.575553+0000', 'last_clean': '2024-09-21T07:48:46.575553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:48:46.590874+0000', 'last_undegraded': '2024-09-21T07:48:46.590874+0000', 'last_fullsized': '2024-09-21T07:48:46.590874+0000', 'mapping_epoch': 217, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 217, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'last_clean_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T07:48:53.131 INFO:tasks.ceph.ceph_manager.ceph:PG 28.5 is not active+clean 2024-09-21T07:48:53.132 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '28.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 217, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:48:46.597041+0000', 'last_change': '2024-09-21T07:48:46.597041+0000', 'last_active': '2024-09-21T07:48:46.575553+0000', 'last_peered': '2024-09-21T07:48:46.575553+0000', 'last_clean': '2024-09-21T07:48:46.575553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:48:46.597041+0000', 'last_undegraded': '2024-09-21T07:48:46.597041+0000', 'last_fullsized': '2024-09-21T07:48:46.597041+0000', 'mapping_epoch': 217, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 217, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'last_clean_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T07:48:53.132 INFO:tasks.ceph.ceph_manager.ceph:PG 28.6 is not active+clean 2024-09-21T07:48:53.132 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '28.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 217, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:48:46.598035+0000', 'last_change': '2024-09-21T07:48:46.598035+0000', 'last_active': '2024-09-21T07:48:46.575553+0000', 'last_peered': '2024-09-21T07:48:46.575553+0000', 'last_clean': '2024-09-21T07:48:46.575553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:48:46.598035+0000', 'last_undegraded': '2024-09-21T07:48:46.598035+0000', 'last_fullsized': '2024-09-21T07:48:46.598035+0000', 'mapping_epoch': 217, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 217, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'last_clean_scrub_stamp': '2024-09-21T07:48:46.575553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T07:48:53.133 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T07:48:53.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.385+0000 7f12d5200640 1 -- 172.21.15.39:0/3373138656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12d015bb00 msgr2=0x7f12d015bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:53.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.385+0000 7f12d5200640 1 --2- 172.21.15.39:0/3373138656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12d015bb00 0x7f12d015bf10 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f12b8009830 tx=0x7f12b802a490 comp rx=0 tx=0).stop 2024-09-21T07:48:53.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.385+0000 7f12d5200640 1 -- 172.21.15.39:0/3373138656 shutdown_connections 2024-09-21T07:48:53.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.385+0000 7f12d5200640 1 --2- 172.21.15.39:0/3373138656 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f12d00d04a0 0x7f12d00d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:53.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.385+0000 7f12d5200640 1 --2- 172.21.15.39:0/3373138656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12d015bb00 0x7f12d015bf10 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:53.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.385+0000 7f12d5200640 1 -- 172.21.15.39:0/3373138656 >> 172.21.15.39:0/3373138656 conn(0x7f12d006cfc0 msgr2=0x7f12d006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:53.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.385+0000 7f12d5200640 1 -- 172.21.15.39:0/3373138656 shutdown_connections 2024-09-21T07:48:53.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.385+0000 7f12d5200640 1 -- 172.21.15.39:0/3373138656 wait complete. 2024-09-21T07:48:53.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.387+0000 7f12d5200640 1 Processor -- start 2024-09-21T07:48:53.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.387+0000 7f12d5200640 1 -- start start 2024-09-21T07:48:53.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.389+0000 7f12d5200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f12d00d04a0 0x7f12d01f3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:53.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.389+0000 7f12d5200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12d0152590 0x7f12d0158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:53.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.389+0000 7f12d5200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12d015bb00 0x7f12d01f46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:53.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.389+0000 7f12d5200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f12d0073130 con 0x7f12d0152590 2024-09-21T07:48:53.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.389+0000 7f12d5200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f12d0072fb0 con 0x7f12d015bb00 2024-09-21T07:48:53.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.389+0000 7f12d5200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f12d00732b0 con 0x7f12d00d04a0 2024-09-21T07:48:53.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.389+0000 7f12c7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12d0152590 0x7f12d0158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:53.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.389+0000 7f12c7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12d0152590 0x7f12d0158c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48834/0 (socket says 172.21.15.39:48834) 2024-09-21T07:48:53.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.389+0000 7f12c7e00640 1 -- 172.21.15.39:0/1217656929 learned_addr learned my addr 172.21.15.39:0/1217656929 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:53.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.389+0000 7f12cf400640 1 --2- 172.21.15.39:0/1217656929 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f12d00d04a0 0x7f12d01f3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:53.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.389+0000 7f12cfe00640 1 --2- 172.21.15.39:0/1217656929 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12d015bb00 0x7f12d01f46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:53.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.390+0000 7f12c7e00640 1 -- 172.21.15.39:0/1217656929 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f12d00d04a0 msgr2=0x7f12d01f3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:53.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.390+0000 7f12c7e00640 1 --2- 172.21.15.39:0/1217656929 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f12d00d04a0 0x7f12d01f3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:53.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.390+0000 7f12c7e00640 1 -- 172.21.15.39:0/1217656929 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12d015bb00 msgr2=0x7f12d01f46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:53.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.390+0000 7f12c7e00640 1 --2- 172.21.15.39:0/1217656929 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12d015bb00 0x7f12d01f46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:53.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.390+0000 7f12c7e00640 1 -- 172.21.15.39:0/1217656929 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12d01f4490 con 0x7f12d0152590 2024-09-21T07:48:53.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.390+0000 7f12c7e00640 1 --2- 172.21.15.39:0/1217656929 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12d0152590 0x7f12d0158c90 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f12b8009950 tx=0x7f12b8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:53.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.390+0000 7f12c7400640 1 -- 172.21.15.39:0/1217656929 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b8005c90 con 0x7f12d0152590 2024-09-21T07:48:53.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.390+0000 7f12c7400640 1 -- 172.21.15.39:0/1217656929 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12b8005020 con 0x7f12d0152590 2024-09-21T07:48:53.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.390+0000 7f12d5200640 1 -- 172.21.15.39:0/1217656929 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12d0154720 con 0x7f12d0152590 2024-09-21T07:48:53.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.390+0000 7f12c7400640 1 -- 172.21.15.39:0/1217656929 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b8005450 con 0x7f12d0152590 2024-09-21T07:48:53.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.390+0000 7f12d5200640 1 -- 172.21.15.39:0/1217656929 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12d0154ce0 con 0x7f12d0152590 2024-09-21T07:48:53.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.392+0000 7f12d5200640 1 -- 172.21.15.39:0/1217656929 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12d0070fb0 con 0x7f12d0152590 2024-09-21T07:48:53.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.393+0000 7f12c7400640 1 -- 172.21.15.39:0/1217656929 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f12b80317c0 con 0x7f12d0152590 2024-09-21T07:48:53.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.394+0000 7f12c7400640 1 --2- 172.21.15.39:0/1217656929 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f12ac078b90 0x7f12ac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:53.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.394+0000 7f12c7400640 1 -- 172.21.15.39:0/1217656929 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(219..219 src has 1..219) ==== 6988+0+0 (secure 0 0 0) 0x7f12b80bc310 con 0x7f12d0152590 2024-09-21T07:48:53.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.394+0000 7f12cf400640 1 --2- 172.21.15.39:0/1217656929 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f12ac078b90 0x7f12ac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:53.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.395+0000 7f12cf400640 1 --2- 172.21.15.39:0/1217656929 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f12ac078b90 0x7f12ac07b0a0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f12c0007960 tx=0x7f12c0001d50 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:53.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.398+0000 7f12c7400640 1 -- 172.21.15.39:0/1217656929 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f12d0070fb0 con 0x7f12d0152590 2024-09-21T07:48:53.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.668+0000 7f12d5200640 1 -- 172.21.15.39:0/1217656929 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f12d01594d0 con 0x7f12d0152590 2024-09-21T07:48:53.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.669+0000 7f12c7400640 1 -- 172.21.15.39:0/1217656929 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1569 (secure 0 0 0) 0x7f12d01f4490 con 0x7f12d0152590 2024-09-21T07:48:53.671 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:48:53.671 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":967,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":219,"num_osds":8,"num_up_osds":8,"osd_up_since":1726903982,"num_in_osds":8,"osd_in_since":1726903972,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":177},{"state_name":"creating+peering","count":16}],"num_pgs":193,"num_pools":6,"num_objects":17,"data_bytes":591144,"bytes_used":417955840,"bytes_avail":772676157440,"bytes_total":773094113280,"inactive_pgs_ratio":0.082901552319526672,"write_bytes_sec":1706,"read_op_per_sec":0,"write_op_per_sec":5},"fsmap":{"epoch":229,"btime":"2024-09-21T07:48:50:615411+0000","id":12,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":12,"rank":0,"name":"c","status":"up:active","gid":10224}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":17,"modified":"2024-09-21T07:46:11.797663+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-21T07:48:53.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.672+0000 7f12d5200640 1 -- 172.21.15.39:0/1217656929 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f12ac078b90 msgr2=0x7f12ac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:53.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.672+0000 7f12d5200640 1 --2- 172.21.15.39:0/1217656929 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f12ac078b90 0x7f12ac07b0a0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f12c0007960 tx=0x7f12c0001d50 comp rx=0 tx=0).stop 2024-09-21T07:48:53.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.672+0000 7f12d5200640 1 -- 172.21.15.39:0/1217656929 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12d0152590 msgr2=0x7f12d0158c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:53.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.672+0000 7f12d5200640 1 --2- 172.21.15.39:0/1217656929 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12d0152590 0x7f12d0158c90 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f12b8009950 tx=0x7f12b8003c10 comp rx=0 tx=0).stop 2024-09-21T07:48:53.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.672+0000 7f12d5200640 1 -- 172.21.15.39:0/1217656929 shutdown_connections 2024-09-21T07:48:53.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.672+0000 7f12d5200640 1 --2- 172.21.15.39:0/1217656929 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f12ac078b90 0x7f12ac07b0a0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:53.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.672+0000 7f12d5200640 1 --2- 172.21.15.39:0/1217656929 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12d015bb00 0x7f12d01f46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:53.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.672+0000 7f12d5200640 1 --2- 172.21.15.39:0/1217656929 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12d0152590 0x7f12d0158c90 unknown :-1 s=CLOSED pgs=2106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:53.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.672+0000 7f12d5200640 1 --2- 172.21.15.39:0/1217656929 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f12d00d04a0 0x7f12d01f3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:53.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.672+0000 7f12d5200640 1 -- 172.21.15.39:0/1217656929 >> 172.21.15.39:0/1217656929 conn(0x7f12d006cfc0 msgr2=0x7f12d0151aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:53.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.672+0000 7f12d5200640 1 -- 172.21.15.39:0/1217656929 shutdown_connections 2024-09-21T07:48:53.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.672+0000 7f12d5200640 1 -- 172.21.15.39:0/1217656929 wait complete. 2024-09-21T07:48:53.696 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-21T07:48:53.697 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:48:53.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.922+0000 7f15ba600640 1 -- 172.21.15.39:0/3693395644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15b4160250 msgr2=0x7f15b415d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:53.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.922+0000 7f15ba600640 1 --2- 172.21.15.39:0/3693395644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15b4160250 0x7f15b415d570 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f15a8009860 tx=0x7f15a802a5f0 comp rx=0 tx=0).stop 2024-09-21T07:48:53.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.922+0000 7f15ba600640 1 -- 172.21.15.39:0/3693395644 shutdown_connections 2024-09-21T07:48:53.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.922+0000 7f15ba600640 1 --2- 172.21.15.39:0/3693395644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15b4160250 0x7f15b415d570 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:53.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.922+0000 7f15ba600640 1 -- 172.21.15.39:0/3693395644 >> 172.21.15.39:0/3693395644 conn(0x7f15b406cfc0 msgr2=0x7f15b406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:53.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.922+0000 7f15ba600640 1 -- 172.21.15.39:0/3693395644 shutdown_connections 2024-09-21T07:48:53.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.922+0000 7f15ba600640 1 -- 172.21.15.39:0/3693395644 wait complete. 2024-09-21T07:48:53.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.925+0000 7f15ba600640 1 Processor -- start 2024-09-21T07:48:53.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.925+0000 7f15ba600640 1 -- start start 2024-09-21T07:48:53.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.926+0000 7f15ba600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15b40d0390 0x7f15b40b23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:53.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.926+0000 7f15ba600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15b4160250 0x7f15b415b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:53.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.926+0000 7f15ba600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15b4160660 0x7f15b415bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:53.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.926+0000 7f15ba600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f15b40ccf00 con 0x7f15b4160250 2024-09-21T07:48:53.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.926+0000 7f15ba600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f15b40ccd80 con 0x7f15b40d0390 2024-09-21T07:48:53.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.926+0000 7f15ba600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f15b40cd080 con 0x7f15b4160660 2024-09-21T07:48:53.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.927+0000 7f15b3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15b4160250 0x7f15b415b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:53.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.927+0000 7f15b3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15b4160250 0x7f15b415b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48856/0 (socket says 172.21.15.39:48856) 2024-09-21T07:48:53.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.927+0000 7f15b3e00640 1 -- 172.21.15.39:0/524571004 learned_addr learned my addr 172.21.15.39:0/524571004 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:53.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.927+0000 7f15b9200640 1 --2- 172.21.15.39:0/524571004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15b40d0390 0x7f15b40b23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:53.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.927+0000 7f15b9c00640 1 --2- 172.21.15.39:0/524571004 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15b4160660 0x7f15b415bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:53.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.927+0000 7f15b3e00640 1 -- 172.21.15.39:0/524571004 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15b4160660 msgr2=0x7f15b415bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:53.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.927+0000 7f15b3e00640 1 --2- 172.21.15.39:0/524571004 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15b4160660 0x7f15b415bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:53.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.927+0000 7f15b3e00640 1 -- 172.21.15.39:0/524571004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15b40d0390 msgr2=0x7f15b40b23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:53.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.927+0000 7f15b3e00640 1 --2- 172.21.15.39:0/524571004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15b40d0390 0x7f15b40b23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:53.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.927+0000 7f15b3e00640 1 -- 172.21.15.39:0/524571004 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b4157d30 con 0x7f15b4160250 2024-09-21T07:48:53.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.928+0000 7f15b3e00640 1 --2- 172.21.15.39:0/524571004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15b4160250 0x7f15b415b8b0 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f15a000c400 tx=0x7f15a000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:53.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.928+0000 7f15b0c00640 1 -- 172.21.15.39:0/524571004 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a0017070 con 0x7f15b4160250 2024-09-21T07:48:53.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.928+0000 7f15b0c00640 1 -- 172.21.15.39:0/524571004 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15a00053c0 con 0x7f15b4160250 2024-09-21T07:48:53.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.928+0000 7f15ba600640 1 -- 172.21.15.39:0/524571004 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15b4158010 con 0x7f15b4160250 2024-09-21T07:48:53.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.928+0000 7f15b0c00640 1 -- 172.21.15.39:0/524571004 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a0003850 con 0x7f15b4160250 2024-09-21T07:48:53.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.928+0000 7f15ba600640 1 -- 172.21.15.39:0/524571004 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15b415a450 con 0x7f15b4160250 2024-09-21T07:48:53.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.929+0000 7f15ba600640 1 -- 172.21.15.39:0/524571004 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15b40ce970 con 0x7f15b4160250 2024-09-21T07:48:53.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.930+0000 7f15b0c00640 1 -- 172.21.15.39:0/524571004 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f15a001a070 con 0x7f15b4160250 2024-09-21T07:48:53.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.931+0000 7f15b0c00640 1 --2- 172.21.15.39:0/524571004 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1594078b90 0x7f159407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:53.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.931+0000 7f15b0c00640 1 -- 172.21.15.39:0/524571004 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(219..219 src has 1..219) ==== 6988+0+0 (secure 0 0 0) 0x7f15a009a4b0 con 0x7f15b4160250 2024-09-21T07:48:53.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.932+0000 7f15b9200640 1 --2- 172.21.15.39:0/524571004 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1594078b90 0x7f159407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:53.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.932+0000 7f15b9200640 1 --2- 172.21.15.39:0/524571004 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1594078b90 0x7f159407b0a0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f15a8034000 tx=0x7f15a8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:53.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:53.936+0000 7f15b0c00640 1 -- 172.21.15.39:0/524571004 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f15b40ce970 con 0x7f15b4160250 2024-09-21T07:48:54.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:54.132+0000 7f15ba600640 1 -- 172.21.15.39:0/524571004 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f15b4157740 con 0x7f1594078b90 2024-09-21T07:48:54.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:54.141+0000 7f15b0c00640 1 -- 172.21.15.39:0/524571004 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+495211 (secure 0 0 0) 0x7f15a8004460 con 0x7f1594078b90 2024-09-21T07:48:54.148 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:48:54.151 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:48:54.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:54.146+0000 7f15ba600640 1 -- 172.21.15.39:0/524571004 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1594078b90 msgr2=0x7f159407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:54.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:54.146+0000 7f15ba600640 1 --2- 172.21.15.39:0/524571004 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1594078b90 0x7f159407b0a0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f15a8034000 tx=0x7f15a8004730 comp rx=0 tx=0).stop 2024-09-21T07:48:54.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:54.146+0000 7f15ba600640 1 -- 172.21.15.39:0/524571004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15b4160250 msgr2=0x7f15b415b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:54.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:54.146+0000 7f15ba600640 1 --2- 172.21.15.39:0/524571004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15b4160250 0x7f15b415b8b0 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f15a000c400 tx=0x7f15a000c8c0 comp rx=0 tx=0).stop 2024-09-21T07:48:54.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:54.146+0000 7f15ba600640 1 -- 172.21.15.39:0/524571004 shutdown_connections 2024-09-21T07:48:54.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:54.146+0000 7f15ba600640 1 --2- 172.21.15.39:0/524571004 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1594078b90 0x7f159407b0a0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:54.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:54.146+0000 7f15ba600640 1 --2- 172.21.15.39:0/524571004 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15b4160660 0x7f15b415bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:54.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:54.146+0000 7f15ba600640 1 --2- 172.21.15.39:0/524571004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15b4160250 0x7f15b415b8b0 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:54.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:54.146+0000 7f15ba600640 1 --2- 172.21.15.39:0/524571004 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15b40d0390 0x7f15b40b23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:54.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:54.146+0000 7f15ba600640 1 -- 172.21.15.39:0/524571004 >> 172.21.15.39:0/524571004 conn(0x7f15b406cfc0 msgr2=0x7f15b415de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:54.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:54.146+0000 7f15ba600640 1 -- 172.21.15.39:0/524571004 shutdown_connections 2024-09-21T07:48:54.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:54.146+0000 7f15ba600640 1 -- 172.21.15.39:0/524571004 wait complete. 2024-09-21T07:48:54.174 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":690,"stamp":"2024-09-21T07:48:53.853330+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":379,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":408348,"kb_used_data":22760,"kb_used_omap":12,"kb_used_meta":385395,"kb_avail":754566372,"statfs":{"total":773094113280,"available":772675964928,"internally_reserved":0,"allocated":23306240,"data_stored":12960860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":394645040},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.268559"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.551157+0000","last_change":"2024-09-21T07:48:47.620778+0000","last_active":"2024-09-21T07:48:49.551157+0000","last_peered":"2024-09-21T07:48:49.551157+0000","last_clean":"2024-09-21T07:48:49.551157+0000","last_became_active":"2024-09-21T07:48:47.620167+0000","last_became_peered":"2024-09-21T07:48:47.620167+0000","last_unstale":"2024-09-21T07:48:49.551157+0000","last_undegraded":"2024-09-21T07:48:49.551157+0000","last_fullsized":"2024-09-21T07:48:49.551157+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:17:04.946971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.610021+0000","last_change":"2024-09-21T07:48:47.608680+0000","last_active":"2024-09-21T07:48:48.610021+0000","last_peered":"2024-09-21T07:48:48.610021+0000","last_clean":"2024-09-21T07:48:48.610021+0000","last_became_active":"2024-09-21T07:48:47.606772+0000","last_became_peered":"2024-09-21T07:48:47.606772+0000","last_unstale":"2024-09-21T07:48:48.610021+0000","last_undegraded":"2024-09-21T07:48:48.610021+0000","last_fullsized":"2024-09-21T07:48:48.610021+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:13.951566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.602360+0000","last_change":"2024-09-21T07:48:47.622897+0000","last_active":"2024-09-21T07:48:49.602360+0000","last_peered":"2024-09-21T07:48:49.602360+0000","last_clean":"2024-09-21T07:48:49.602360+0000","last_became_active":"2024-09-21T07:48:47.622568+0000","last_became_peered":"2024-09-21T07:48:47.622568+0000","last_unstale":"2024-09-21T07:48:49.602360+0000","last_undegraded":"2024-09-21T07:48:49.602360+0000","last_fullsized":"2024-09-21T07:48:49.602360+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:20.786012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608060+0000","last_change":"2024-09-21T07:48:47.624850+0000","last_active":"2024-09-21T07:48:48.608060+0000","last_peered":"2024-09-21T07:48:48.608060+0000","last_clean":"2024-09-21T07:48:48.608060+0000","last_became_active":"2024-09-21T07:48:47.624508+0000","last_became_peered":"2024-09-21T07:48:47.624508+0000","last_unstale":"2024-09-21T07:48:48.608060+0000","last_undegraded":"2024-09-21T07:48:48.608060+0000","last_fullsized":"2024-09-21T07:48:48.608060+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:48.164441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609274+0000","last_change":"2024-09-21T07:48:47.602747+0000","last_active":"2024-09-21T07:48:48.609274+0000","last_peered":"2024-09-21T07:48:48.609274+0000","last_clean":"2024-09-21T07:48:48.609274+0000","last_became_active":"2024-09-21T07:48:47.602453+0000","last_became_peered":"2024-09-21T07:48:47.602453+0000","last_unstale":"2024-09-21T07:48:48.609274+0000","last_undegraded":"2024-09-21T07:48:48.609274+0000","last_fullsized":"2024-09-21T07:48:48.609274+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:20:13.838574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608103+0000","last_change":"2024-09-21T07:48:47.620304+0000","last_active":"2024-09-21T07:48:48.608103+0000","last_peered":"2024-09-21T07:48:48.608103+0000","last_clean":"2024-09-21T07:48:48.608103+0000","last_became_active":"2024-09-21T07:48:47.619974+0000","last_became_peered":"2024-09-21T07:48:47.619974+0000","last_unstale":"2024-09-21T07:48:48.608103+0000","last_undegraded":"2024-09-21T07:48:48.608103+0000","last_fullsized":"2024-09-21T07:48:48.608103+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:36:37.731085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609064+0000","last_change":"2024-09-21T07:48:47.602027+0000","last_active":"2024-09-21T07:48:48.609064+0000","last_peered":"2024-09-21T07:48:48.609064+0000","last_clean":"2024-09-21T07:48:48.609064+0000","last_became_active":"2024-09-21T07:48:47.601703+0000","last_became_peered":"2024-09-21T07:48:47.601703+0000","last_unstale":"2024-09-21T07:48:48.609064+0000","last_undegraded":"2024-09-21T07:48:48.609064+0000","last_fullsized":"2024-09-21T07:48:48.609064+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:37:16.197847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.550179+0000","last_change":"2024-09-21T07:48:47.627224+0000","last_active":"2024-09-21T07:48:49.550179+0000","last_peered":"2024-09-21T07:48:49.550179+0000","last_clean":"2024-09-21T07:48:49.550179+0000","last_became_active":"2024-09-21T07:48:47.626767+0000","last_became_peered":"2024-09-21T07:48:47.626767+0000","last_unstale":"2024-09-21T07:48:49.550179+0000","last_undegraded":"2024-09-21T07:48:49.550179+0000","last_fullsized":"2024-09-21T07:48:49.550179+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:42:45.011736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.050815+0000","last_change":"2024-09-21T07:48:47.617745+0000","last_active":"2024-09-21T07:48:50.050815+0000","last_peered":"2024-09-21T07:48:50.050815+0000","last_clean":"2024-09-21T07:48:50.050815+0000","last_became_active":"2024-09-21T07:48:47.614030+0000","last_became_peered":"2024-09-21T07:48:47.614030+0000","last_unstale":"2024-09-21T07:48:50.050815+0000","last_undegraded":"2024-09-21T07:48:50.050815+0000","last_fullsized":"2024-09-21T07:48:50.050815+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:33:56.311006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.552644+0000","last_change":"2024-09-21T07:48:47.621852+0000","last_active":"2024-09-21T07:48:49.552644+0000","last_peered":"2024-09-21T07:48:49.552644+0000","last_clean":"2024-09-21T07:48:49.552644+0000","last_became_active":"2024-09-21T07:48:47.617439+0000","last_became_peered":"2024-09-21T07:48:47.617439+0000","last_unstale":"2024-09-21T07:48:49.552644+0000","last_undegraded":"2024-09-21T07:48:49.552644+0000","last_fullsized":"2024-09-21T07:48:49.552644+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:44.083185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.602400+0000","last_change":"2024-09-21T07:48:47.632098+0000","last_active":"2024-09-21T07:48:49.602400+0000","last_peered":"2024-09-21T07:48:49.602400+0000","last_clean":"2024-09-21T07:48:49.602400+0000","last_became_active":"2024-09-21T07:48:47.631802+0000","last_became_peered":"2024-09-21T07:48:47.631802+0000","last_unstale":"2024-09-21T07:48:49.602400+0000","last_undegraded":"2024-09-21T07:48:49.602400+0000","last_fullsized":"2024-09-21T07:48:49.602400+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:36.943336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609791+0000","last_change":"2024-09-21T07:48:47.617229+0000","last_active":"2024-09-21T07:48:48.609791+0000","last_peered":"2024-09-21T07:48:48.609791+0000","last_clean":"2024-09-21T07:48:48.609791+0000","last_became_active":"2024-09-21T07:48:47.616863+0000","last_became_peered":"2024-09-21T07:48:47.616863+0000","last_unstale":"2024-09-21T07:48:48.609791+0000","last_undegraded":"2024-09-21T07:48:48.609791+0000","last_fullsized":"2024-09-21T07:48:48.609791+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:42.345168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608262+0000","last_change":"2024-09-21T07:48:47.618884+0000","last_active":"2024-09-21T07:48:48.608262+0000","last_peered":"2024-09-21T07:48:48.608262+0000","last_clean":"2024-09-21T07:48:48.608262+0000","last_became_active":"2024-09-21T07:48:47.618553+0000","last_became_peered":"2024-09-21T07:48:47.618553+0000","last_unstale":"2024-09-21T07:48:48.608262+0000","last_undegraded":"2024-09-21T07:48:48.608262+0000","last_fullsized":"2024-09-21T07:48:48.608262+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:51:06.489808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.551008+0000","last_change":"2024-09-21T07:48:47.623337+0000","last_active":"2024-09-21T07:48:49.551008+0000","last_peered":"2024-09-21T07:48:49.551008+0000","last_clean":"2024-09-21T07:48:49.551008+0000","last_became_active":"2024-09-21T07:48:47.620228+0000","last_became_peered":"2024-09-21T07:48:47.620228+0000","last_unstale":"2024-09-21T07:48:49.551008+0000","last_undegraded":"2024-09-21T07:48:49.551008+0000","last_fullsized":"2024-09-21T07:48:49.551008+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:43.841113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051089+0000","last_change":"2024-09-21T07:48:47.627205+0000","last_active":"2024-09-21T07:48:50.051089+0000","last_peered":"2024-09-21T07:48:50.051089+0000","last_clean":"2024-09-21T07:48:50.051089+0000","last_became_active":"2024-09-21T07:48:47.626871+0000","last_became_peered":"2024-09-21T07:48:47.626871+0000","last_unstale":"2024-09-21T07:48:50.051089+0000","last_undegraded":"2024-09-21T07:48:50.051089+0000","last_fullsized":"2024-09-21T07:48:50.051089+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:57.118108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.601783+0000","last_change":"2024-09-21T07:48:47.627220+0000","last_active":"2024-09-21T07:48:49.601783+0000","last_peered":"2024-09-21T07:48:49.601783+0000","last_clean":"2024-09-21T07:48:49.601783+0000","last_became_active":"2024-09-21T07:48:47.626875+0000","last_became_peered":"2024-09-21T07:48:47.626875+0000","last_unstale":"2024-09-21T07:48:49.601783+0000","last_undegraded":"2024-09-21T07:48:49.601783+0000","last_fullsized":"2024-09-21T07:48:49.601783+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:09:11.971175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051104+0000","last_change":"2024-09-21T07:48:47.625920+0000","last_active":"2024-09-21T07:48:50.051104+0000","last_peered":"2024-09-21T07:48:50.051104+0000","last_clean":"2024-09-21T07:48:50.051104+0000","last_became_active":"2024-09-21T07:48:47.625605+0000","last_became_peered":"2024-09-21T07:48:47.625605+0000","last_unstale":"2024-09-21T07:48:50.051104+0000","last_undegraded":"2024-09-21T07:48:50.051104+0000","last_fullsized":"2024-09-21T07:48:50.051104+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:38.962346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.050945+0000","last_change":"2024-09-21T07:48:47.628044+0000","last_active":"2024-09-21T07:48:50.050945+0000","last_peered":"2024-09-21T07:48:50.050945+0000","last_clean":"2024-09-21T07:48:50.050945+0000","last_became_active":"2024-09-21T07:48:47.626515+0000","last_became_peered":"2024-09-21T07:48:47.626515+0000","last_unstale":"2024-09-21T07:48:50.050945+0000","last_undegraded":"2024-09-21T07:48:50.050945+0000","last_fullsized":"2024-09-21T07:48:50.050945+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:03.431109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.552606+0000","last_change":"2024-09-21T07:48:47.623935+0000","last_active":"2024-09-21T07:48:49.552606+0000","last_peered":"2024-09-21T07:48:49.552606+0000","last_clean":"2024-09-21T07:48:49.552606+0000","last_became_active":"2024-09-21T07:48:47.623233+0000","last_became_peered":"2024-09-21T07:48:47.623233+0000","last_unstale":"2024-09-21T07:48:49.552606+0000","last_undegraded":"2024-09-21T07:48:49.552606+0000","last_fullsized":"2024-09-21T07:48:49.552606+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:23.647378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.550816+0000","last_change":"2024-09-21T07:48:47.617959+0000","last_active":"2024-09-21T07:48:49.550816+0000","last_peered":"2024-09-21T07:48:49.550816+0000","last_clean":"2024-09-21T07:48:49.550816+0000","last_became_active":"2024-09-21T07:48:47.617595+0000","last_became_peered":"2024-09-21T07:48:47.617595+0000","last_unstale":"2024-09-21T07:48:49.550816+0000","last_undegraded":"2024-09-21T07:48:49.550816+0000","last_fullsized":"2024-09-21T07:48:49.550816+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:08:09.285660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.551440+0000","last_change":"2024-09-21T07:48:47.621502+0000","last_active":"2024-09-21T07:48:49.551440+0000","last_peered":"2024-09-21T07:48:49.551440+0000","last_clean":"2024-09-21T07:48:49.551440+0000","last_became_active":"2024-09-21T07:48:47.616971+0000","last_became_peered":"2024-09-21T07:48:47.616971+0000","last_unstale":"2024-09-21T07:48:49.551440+0000","last_undegraded":"2024-09-21T07:48:49.551440+0000","last_fullsized":"2024-09-21T07:48:49.551440+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:15.121823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.551985+0000","last_change":"2024-09-21T07:48:47.622917+0000","last_active":"2024-09-21T07:48:49.551985+0000","last_peered":"2024-09-21T07:48:49.551985+0000","last_clean":"2024-09-21T07:48:49.551985+0000","last_became_active":"2024-09-21T07:48:47.619537+0000","last_became_peered":"2024-09-21T07:48:47.619537+0000","last_unstale":"2024-09-21T07:48:49.551985+0000","last_undegraded":"2024-09-21T07:48:49.551985+0000","last_fullsized":"2024-09-21T07:48:49.551985+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:45.507548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.610863+0000","last_change":"2024-09-21T07:48:47.598763+0000","last_active":"2024-09-21T07:48:48.610863+0000","last_peered":"2024-09-21T07:48:48.610863+0000","last_clean":"2024-09-21T07:48:48.610863+0000","last_became_active":"2024-09-21T07:48:47.598446+0000","last_became_peered":"2024-09-21T07:48:47.598446+0000","last_unstale":"2024-09-21T07:48:48.610863+0000","last_undegraded":"2024-09-21T07:48:48.610863+0000","last_fullsized":"2024-09-21T07:48:48.610863+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:48.912495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608756+0000","last_change":"2024-09-21T07:48:47.596907+0000","last_active":"2024-09-21T07:48:48.608756+0000","last_peered":"2024-09-21T07:48:48.608756+0000","last_clean":"2024-09-21T07:48:48.608756+0000","last_became_active":"2024-09-21T07:48:47.596568+0000","last_became_peered":"2024-09-21T07:48:47.596568+0000","last_unstale":"2024-09-21T07:48:48.608756+0000","last_undegraded":"2024-09-21T07:48:48.608756+0000","last_fullsized":"2024-09-21T07:48:48.608756+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:33:45.187090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051256+0000","last_change":"2024-09-21T07:48:47.620026+0000","last_active":"2024-09-21T07:48:50.051256+0000","last_peered":"2024-09-21T07:48:50.051256+0000","last_clean":"2024-09-21T07:48:50.051256+0000","last_became_active":"2024-09-21T07:48:47.619713+0000","last_became_peered":"2024-09-21T07:48:47.619713+0000","last_unstale":"2024-09-21T07:48:50.051256+0000","last_undegraded":"2024-09-21T07:48:50.051256+0000","last_fullsized":"2024-09-21T07:48:50.051256+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:43:53.702721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.551438+0000","last_change":"2024-09-21T07:48:47.629000+0000","last_active":"2024-09-21T07:48:49.551438+0000","last_peered":"2024-09-21T07:48:49.551438+0000","last_clean":"2024-09-21T07:48:49.551438+0000","last_became_active":"2024-09-21T07:48:47.628658+0000","last_became_peered":"2024-09-21T07:48:47.628658+0000","last_unstale":"2024-09-21T07:48:49.551438+0000","last_undegraded":"2024-09-21T07:48:49.551438+0000","last_fullsized":"2024-09-21T07:48:49.551438+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:12.875555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.602633+0000","last_change":"2024-09-21T07:48:47.625275+0000","last_active":"2024-09-21T07:48:49.602633+0000","last_peered":"2024-09-21T07:48:49.602633+0000","last_clean":"2024-09-21T07:48:49.602633+0000","last_became_active":"2024-09-21T07:48:47.624985+0000","last_became_peered":"2024-09-21T07:48:47.624985+0000","last_unstale":"2024-09-21T07:48:49.602633+0000","last_undegraded":"2024-09-21T07:48:49.602633+0000","last_fullsized":"2024-09-21T07:48:49.602633+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:04.926778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609989+0000","last_change":"2024-09-21T07:48:47.608774+0000","last_active":"2024-09-21T07:48:48.609989+0000","last_peered":"2024-09-21T07:48:48.609989+0000","last_clean":"2024-09-21T07:48:48.609989+0000","last_became_active":"2024-09-21T07:48:47.608457+0000","last_became_peered":"2024-09-21T07:48:47.608457+0000","last_unstale":"2024-09-21T07:48:48.609989+0000","last_undegraded":"2024-09-21T07:48:48.609989+0000","last_fullsized":"2024-09-21T07:48:48.609989+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:16.164291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.050811+0000","last_change":"2024-09-21T07:48:47.605730+0000","last_active":"2024-09-21T07:48:50.050811+0000","last_peered":"2024-09-21T07:48:50.050811+0000","last_clean":"2024-09-21T07:48:50.050811+0000","last_became_active":"2024-09-21T07:48:47.605433+0000","last_became_peered":"2024-09-21T07:48:47.605433+0000","last_unstale":"2024-09-21T07:48:50.050811+0000","last_undegraded":"2024-09-21T07:48:50.050811+0000","last_fullsized":"2024-09-21T07:48:50.050811+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:59.823864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051126+0000","last_change":"2024-09-21T07:48:47.604557+0000","last_active":"2024-09-21T07:48:50.051126+0000","last_peered":"2024-09-21T07:48:50.051126+0000","last_clean":"2024-09-21T07:48:50.051126+0000","last_became_active":"2024-09-21T07:48:47.604196+0000","last_became_peered":"2024-09-21T07:48:47.604196+0000","last_unstale":"2024-09-21T07:48:50.051126+0000","last_undegraded":"2024-09-21T07:48:50.051126+0000","last_fullsized":"2024-09-21T07:48:50.051126+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:56.323906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.607674+0000","last_change":"2024-09-21T07:48:47.621972+0000","last_active":"2024-09-21T07:48:48.607674+0000","last_peered":"2024-09-21T07:48:48.607674+0000","last_clean":"2024-09-21T07:48:48.607674+0000","last_became_active":"2024-09-21T07:48:47.621514+0000","last_became_peered":"2024-09-21T07:48:47.621514+0000","last_unstale":"2024-09-21T07:48:48.607674+0000","last_undegraded":"2024-09-21T07:48:48.607674+0000","last_fullsized":"2024-09-21T07:48:48.607674+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:06.914273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.607610+0000","last_change":"2024-09-21T07:48:47.608860+0000","last_active":"2024-09-21T07:48:48.607610+0000","last_peered":"2024-09-21T07:48:48.607610+0000","last_clean":"2024-09-21T07:48:48.607610+0000","last_became_active":"2024-09-21T07:48:47.608522+0000","last_became_peered":"2024-09-21T07:48:47.608522+0000","last_unstale":"2024-09-21T07:48:48.607610+0000","last_undegraded":"2024-09-21T07:48:48.607610+0000","last_fullsized":"2024-09-21T07:48:48.607610+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:02:42.398684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.551693+0000","last_change":"2024-09-21T07:48:47.629782+0000","last_active":"2024-09-21T07:48:49.551693+0000","last_peered":"2024-09-21T07:48:49.551693+0000","last_clean":"2024-09-21T07:48:49.551693+0000","last_became_active":"2024-09-21T07:48:47.628902+0000","last_became_peered":"2024-09-21T07:48:47.628902+0000","last_unstale":"2024-09-21T07:48:49.551693+0000","last_undegraded":"2024-09-21T07:48:49.551693+0000","last_fullsized":"2024-09-21T07:48:49.551693+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:56:31.556703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1a","version":"218'1","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.050809+0000","last_change":"2024-09-21T07:48:46.603075+0000","last_active":"2024-09-21T07:48:50.050809+0000","last_peered":"2024-09-21T07:48:50.050809+0000","last_clean":"2024-09-21T07:48:50.050809+0000","last_became_active":"2024-09-21T07:48:46.602882+0000","last_became_peered":"2024-09-21T07:48:46.602882+0000","last_unstale":"2024-09-21T07:48:50.050809+0000","last_undegraded":"2024-09-21T07:48:50.050809+0000","last_fullsized":"2024-09-21T07:48:50.050809+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:49.025973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":305,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608869+0000","last_change":"2024-09-21T07:35:27.879605+0000","last_active":"2024-09-21T07:48:48.608869+0000","last_peered":"2024-09-21T07:48:48.608869+0000","last_clean":"2024-09-21T07:48:48.608869+0000","last_became_active":"2024-09-21T07:35:27.879274+0000","last_became_peered":"2024-09-21T07:35:27.879274+0000","last_unstale":"2024-09-21T07:48:48.608869+0000","last_undegraded":"2024-09-21T07:48:48.608869+0000","last_fullsized":"2024-09-21T07:48:48.608869+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:24.241636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":316,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609470+0000","last_change":"2024-09-21T07:35:29.491336+0000","last_active":"2024-09-21T07:48:48.609470+0000","last_peered":"2024-09-21T07:48:48.609470+0000","last_clean":"2024-09-21T07:48:48.609470+0000","last_became_active":"2024-09-21T07:35:26.866484+0000","last_became_peered":"2024-09-21T07:35:26.866484+0000","last_unstale":"2024-09-21T07:48:48.609470+0000","last_undegraded":"2024-09-21T07:48:48.609470+0000","last_fullsized":"2024-09-21T07:48:48.609470+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:10.449665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076010100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":491,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.607805+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:48:48.607805+0000","last_peered":"2024-09-21T07:48:48.607805+0000","last_clean":"2024-09-21T07:48:48.607805+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:48:48.607805+0000","last_undegraded":"2024-09-21T07:48:48.607805+0000","last_fullsized":"2024-09-21T07:48:48.607805+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":379,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.551485+0000","last_change":"2024-09-21T07:34:24.539248+0000","last_active":"2024-09-21T07:48:49.551485+0000","last_peered":"2024-09-21T07:48:49.551485+0000","last_clean":"2024-09-21T07:48:49.551485+0000","last_became_active":"2024-09-21T07:34:23.273465+0000","last_became_peered":"2024-09-21T07:34:23.273465+0000","last_unstale":"2024-09-21T07:48:49.551485+0000","last_undegraded":"2024-09-21T07:48:49.551485+0000","last_fullsized":"2024-09-21T07:48:49.551485+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:55.517452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060627700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608458+0000","last_change":"2024-09-21T07:48:47.612024+0000","last_active":"2024-09-21T07:48:48.608458+0000","last_peered":"2024-09-21T07:48:48.608458+0000","last_clean":"2024-09-21T07:48:48.608458+0000","last_became_active":"2024-09-21T07:48:47.601212+0000","last_became_peered":"2024-09-21T07:48:47.601212+0000","last_unstale":"2024-09-21T07:48:48.608458+0000","last_undegraded":"2024-09-21T07:48:48.608458+0000","last_fullsized":"2024-09-21T07:48:48.608458+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:20.715873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"218'4","reported_seq":19,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051794+0000","last_change":"2024-09-21T07:48:46.613621+0000","last_active":"2024-09-21T07:48:50.051794+0000","last_peered":"2024-09-21T07:48:50.051794+0000","last_clean":"2024-09-21T07:48:50.051794+0000","last_became_active":"2024-09-21T07:48:46.613266+0000","last_became_peered":"2024-09-21T07:48:46.613266+0000","last_unstale":"2024-09-21T07:48:50.051794+0000","last_undegraded":"2024-09-21T07:48:50.051794+0000","last_fullsized":"2024-09-21T07:48:50.051794+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:40.639085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":309,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608414+0000","last_change":"2024-09-21T07:35:27.864492+0000","last_active":"2024-09-21T07:48:48.608414+0000","last_peered":"2024-09-21T07:48:48.608414+0000","last_clean":"2024-09-21T07:48:48.608414+0000","last_became_active":"2024-09-21T07:35:27.864176+0000","last_became_peered":"2024-09-21T07:35:27.864176+0000","last_unstale":"2024-09-21T07:48:48.608414+0000","last_undegraded":"2024-09-21T07:48:48.608414+0000","last_fullsized":"2024-09-21T07:48:48.608414+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:05.153467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":312,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608726+0000","last_change":"2024-09-21T07:35:29.493678+0000","last_active":"2024-09-21T07:48:48.608726+0000","last_peered":"2024-09-21T07:48:48.608726+0000","last_clean":"2024-09-21T07:48:48.608726+0000","last_became_active":"2024-09-21T07:35:26.870749+0000","last_became_peered":"2024-09-21T07:35:26.870749+0000","last_unstale":"2024-09-21T07:48:48.608726+0000","last_undegraded":"2024-09-21T07:48:48.608726+0000","last_fullsized":"2024-09-21T07:48:48.608726+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:55.367877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010159570000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":359,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051441+0000","last_change":"2024-09-21T07:34:24.597274+0000","last_active":"2024-09-21T07:48:50.051441+0000","last_peered":"2024-09-21T07:48:50.051441+0000","last_clean":"2024-09-21T07:48:50.051441+0000","last_became_active":"2024-09-21T07:34:23.279560+0000","last_became_peered":"2024-09-21T07:34:23.279560+0000","last_unstale":"2024-09-21T07:48:50.051441+0000","last_undegraded":"2024-09-21T07:48:50.051441+0000","last_fullsized":"2024-09-21T07:48:50.051441+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:15.845265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059699799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.550483+0000","last_change":"2024-09-21T07:48:47.605601+0000","last_active":"2024-09-21T07:48:49.550483+0000","last_peered":"2024-09-21T07:48:49.550483+0000","last_clean":"2024-09-21T07:48:49.550483+0000","last_became_active":"2024-09-21T07:48:47.603036+0000","last_became_peered":"2024-09-21T07:48:47.603036+0000","last_unstale":"2024-09-21T07:48:49.550483+0000","last_undegraded":"2024-09-21T07:48:49.550483+0000","last_fullsized":"2024-09-21T07:48:49.550483+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:02:57.337644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"218'1","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.550771+0000","last_change":"2024-09-21T07:48:46.610499+0000","last_active":"2024-09-21T07:48:49.550771+0000","last_peered":"2024-09-21T07:48:49.550771+0000","last_clean":"2024-09-21T07:48:49.550771+0000","last_became_active":"2024-09-21T07:48:46.610137+0000","last_became_peered":"2024-09-21T07:48:46.610137+0000","last_unstale":"2024-09-21T07:48:49.550771+0000","last_undegraded":"2024-09-21T07:48:49.550771+0000","last_fullsized":"2024-09-21T07:48:49.550771+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:19.623404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":310,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609240+0000","last_change":"2024-09-21T07:35:27.874387+0000","last_active":"2024-09-21T07:48:48.609240+0000","last_peered":"2024-09-21T07:48:48.609240+0000","last_clean":"2024-09-21T07:48:48.609240+0000","last_became_active":"2024-09-21T07:35:27.873277+0000","last_became_peered":"2024-09-21T07:35:27.873277+0000","last_unstale":"2024-09-21T07:48:48.609240+0000","last_undegraded":"2024-09-21T07:48:48.609240+0000","last_fullsized":"2024-09-21T07:48:48.609240+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:04.581463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":309,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051894+0000","last_change":"2024-09-21T07:35:29.387094+0000","last_active":"2024-09-21T07:48:50.051894+0000","last_peered":"2024-09-21T07:48:50.051894+0000","last_clean":"2024-09-21T07:48:50.051894+0000","last_became_active":"2024-09-21T07:35:26.874757+0000","last_became_peered":"2024-09-21T07:35:26.874757+0000","last_unstale":"2024-09-21T07:48:50.051894+0000","last_undegraded":"2024-09-21T07:48:50.051894+0000","last_fullsized":"2024-09-21T07:48:50.051894+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:33.542750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061065700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":365,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608865+0000","last_change":"2024-09-21T07:34:24.259906+0000","last_active":"2024-09-21T07:48:48.608865+0000","last_peered":"2024-09-21T07:48:48.608865+0000","last_clean":"2024-09-21T07:48:48.608865+0000","last_became_active":"2024-09-21T07:34:23.635282+0000","last_became_peered":"2024-09-21T07:34:23.635282+0000","last_unstale":"2024-09-21T07:48:48.608865+0000","last_undegraded":"2024-09-21T07:48:48.608865+0000","last_fullsized":"2024-09-21T07:48:48.608865+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:00.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001527664,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.551699+0000","last_change":"2024-09-21T07:48:47.622852+0000","last_active":"2024-09-21T07:48:49.551699+0000","last_peered":"2024-09-21T07:48:49.551699+0000","last_clean":"2024-09-21T07:48:49.551699+0000","last_became_active":"2024-09-21T07:48:47.619895+0000","last_became_peered":"2024-09-21T07:48:47.619895+0000","last_unstale":"2024-09-21T07:48:49.551699+0000","last_undegraded":"2024-09-21T07:48:49.551699+0000","last_fullsized":"2024-09-21T07:48:49.551699+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:53.333545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"218'2","reported_seq":17,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609737+0000","last_change":"2024-09-21T07:48:46.606441+0000","last_active":"2024-09-21T07:48:48.609737+0000","last_peered":"2024-09-21T07:48:48.609737+0000","last_clean":"2024-09-21T07:48:48.609737+0000","last_became_active":"2024-09-21T07:48:46.604712+0000","last_became_peered":"2024-09-21T07:48:46.604712+0000","last_unstale":"2024-09-21T07:48:48.609737+0000","last_undegraded":"2024-09-21T07:48:48.609737+0000","last_fullsized":"2024-09-21T07:48:48.609737+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:22.700415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":310,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609146+0000","last_change":"2024-09-21T07:35:27.873645+0000","last_active":"2024-09-21T07:48:48.609146+0000","last_peered":"2024-09-21T07:48:48.609146+0000","last_clean":"2024-09-21T07:48:48.609146+0000","last_became_active":"2024-09-21T07:35:27.873245+0000","last_became_peered":"2024-09-21T07:35:27.873245+0000","last_unstale":"2024-09-21T07:48:48.609146+0000","last_undegraded":"2024-09-21T07:48:48.609146+0000","last_fullsized":"2024-09-21T07:48:48.609146+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:55.090953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":316,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.610962+0000","last_change":"2024-09-21T07:35:28.861744+0000","last_active":"2024-09-21T07:48:48.610962+0000","last_peered":"2024-09-21T07:48:48.610962+0000","last_clean":"2024-09-21T07:48:48.610962+0000","last_became_active":"2024-09-21T07:35:26.876462+0000","last_became_peered":"2024-09-21T07:35:26.876462+0000","last_unstale":"2024-09-21T07:48:48.610962+0000","last_undegraded":"2024-09-21T07:48:48.610962+0000","last_fullsized":"2024-09-21T07:48:48.610962+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:48.632568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012020920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":365,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.050526+0000","last_change":"2024-09-21T07:34:25.085454+0000","last_active":"2024-09-21T07:48:50.050526+0000","last_peered":"2024-09-21T07:48:50.050526+0000","last_clean":"2024-09-21T07:48:50.050526+0000","last_became_active":"2024-09-21T07:34:23.279205+0000","last_became_peered":"2024-09-21T07:34:23.279205+0000","last_unstale":"2024-09-21T07:48:50.050526+0000","last_undegraded":"2024-09-21T07:48:50.050526+0000","last_fullsized":"2024-09-21T07:48:50.050526+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:54.231666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095320599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609766+0000","last_change":"2024-09-21T07:48:47.610088+0000","last_active":"2024-09-21T07:48:48.609766+0000","last_peered":"2024-09-21T07:48:48.609766+0000","last_clean":"2024-09-21T07:48:48.609766+0000","last_became_active":"2024-09-21T07:48:47.607791+0000","last_became_peered":"2024-09-21T07:48:47.607791+0000","last_unstale":"2024-09-21T07:48:48.609766+0000","last_undegraded":"2024-09-21T07:48:48.609766+0000","last_fullsized":"2024-09-21T07:48:48.609766+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:35.238803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"218'2","reported_seq":17,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.050817+0000","last_change":"2024-09-21T07:48:46.602844+0000","last_active":"2024-09-21T07:48:50.050817+0000","last_peered":"2024-09-21T07:48:50.050817+0000","last_clean":"2024-09-21T07:48:50.050817+0000","last_became_active":"2024-09-21T07:48:46.602544+0000","last_became_peered":"2024-09-21T07:48:46.602544+0000","last_unstale":"2024-09-21T07:48:50.050817+0000","last_undegraded":"2024-09-21T07:48:50.050817+0000","last_fullsized":"2024-09-21T07:48:50.050817+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:36:36.719598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":308,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609553+0000","last_change":"2024-09-21T07:35:27.861373+0000","last_active":"2024-09-21T07:48:48.609553+0000","last_peered":"2024-09-21T07:48:48.609553+0000","last_clean":"2024-09-21T07:48:48.609553+0000","last_became_active":"2024-09-21T07:35:27.861016+0000","last_became_peered":"2024-09-21T07:35:27.861016+0000","last_unstale":"2024-09-21T07:48:48.609553+0000","last_undegraded":"2024-09-21T07:48:48.609553+0000","last_fullsized":"2024-09-21T07:48:48.609553+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:56.043763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":315,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.602233+0000","last_change":"2024-09-21T07:35:29.053677+0000","last_active":"2024-09-21T07:48:49.602233+0000","last_peered":"2024-09-21T07:48:49.602233+0000","last_clean":"2024-09-21T07:48:49.602233+0000","last_became_active":"2024-09-21T07:35:26.875496+0000","last_became_peered":"2024-09-21T07:35:26.875496+0000","last_unstale":"2024-09-21T07:48:49.602233+0000","last_undegraded":"2024-09-21T07:48:49.602233+0000","last_fullsized":"2024-09-21T07:48:49.602233+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:04.850784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065859000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":367,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.602122+0000","last_change":"2024-09-21T07:34:25.028521+0000","last_active":"2024-09-21T07:48:49.602122+0000","last_peered":"2024-09-21T07:48:49.602122+0000","last_clean":"2024-09-21T07:48:49.602122+0000","last_became_active":"2024-09-21T07:34:23.635467+0000","last_became_peered":"2024-09-21T07:34:23.635467+0000","last_unstale":"2024-09-21T07:48:49.602122+0000","last_undegraded":"2024-09-21T07:48:49.602122+0000","last_fullsized":"2024-09-21T07:48:49.602122+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:33.729123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000660558,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051543+0000","last_change":"2024-09-21T07:48:47.616627+0000","last_active":"2024-09-21T07:48:50.051543+0000","last_peered":"2024-09-21T07:48:50.051543+0000","last_clean":"2024-09-21T07:48:50.051543+0000","last_became_active":"2024-09-21T07:48:47.616246+0000","last_became_peered":"2024-09-21T07:48:47.616246+0000","last_unstale":"2024-09-21T07:48:50.051543+0000","last_undegraded":"2024-09-21T07:48:50.051543+0000","last_fullsized":"2024-09-21T07:48:50.051543+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:07.746423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"218'1","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608686+0000","last_change":"2024-09-21T07:48:46.614962+0000","last_active":"2024-09-21T07:48:48.608686+0000","last_peered":"2024-09-21T07:48:48.608686+0000","last_clean":"2024-09-21T07:48:48.608686+0000","last_became_active":"2024-09-21T07:48:46.614616+0000","last_became_peered":"2024-09-21T07:48:46.614616+0000","last_unstale":"2024-09-21T07:48:48.608686+0000","last_undegraded":"2024-09-21T07:48:48.608686+0000","last_fullsized":"2024-09-21T07:48:48.608686+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:56:58.381332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":306,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051042+0000","last_change":"2024-09-21T07:35:27.871201+0000","last_active":"2024-09-21T07:48:50.051042+0000","last_peered":"2024-09-21T07:48:50.051042+0000","last_clean":"2024-09-21T07:48:50.051042+0000","last_became_active":"2024-09-21T07:35:27.870816+0000","last_became_peered":"2024-09-21T07:35:27.870816+0000","last_unstale":"2024-09-21T07:48:50.051042+0000","last_undegraded":"2024-09-21T07:48:50.051042+0000","last_fullsized":"2024-09-21T07:48:50.051042+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:04.787450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":315,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.610027+0000","last_change":"2024-09-21T07:35:29.491874+0000","last_active":"2024-09-21T07:48:48.610027+0000","last_peered":"2024-09-21T07:48:48.610027+0000","last_clean":"2024-09-21T07:48:48.610027+0000","last_became_active":"2024-09-21T07:35:26.866381+0000","last_became_peered":"2024-09-21T07:35:26.866381+0000","last_unstale":"2024-09-21T07:48:48.610027+0000","last_undegraded":"2024-09-21T07:48:48.610027+0000","last_fullsized":"2024-09-21T07:48:48.610027+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:36.404647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073235399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":370,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.610052+0000","last_change":"2024-09-21T07:34:24.260046+0000","last_active":"2024-09-21T07:48:48.610052+0000","last_peered":"2024-09-21T07:48:48.610052+0000","last_clean":"2024-09-21T07:48:48.610052+0000","last_became_active":"2024-09-21T07:34:23.273431+0000","last_became_peered":"2024-09-21T07:34:23.273431+0000","last_unstale":"2024-09-21T07:48:48.610052+0000","last_undegraded":"2024-09-21T07:48:48.610052+0000","last_fullsized":"2024-09-21T07:48:48.610052+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:18.876057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019803310000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051791+0000","last_change":"2024-09-21T07:48:47.605350+0000","last_active":"2024-09-21T07:48:50.051791+0000","last_peered":"2024-09-21T07:48:50.051791+0000","last_clean":"2024-09-21T07:48:50.051791+0000","last_became_active":"2024-09-21T07:48:47.605046+0000","last_became_peered":"2024-09-21T07:48:47.605046+0000","last_unstale":"2024-09-21T07:48:50.051791+0000","last_undegraded":"2024-09-21T07:48:50.051791+0000","last_fullsized":"2024-09-21T07:48:50.051791+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:07.906313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.607657+0000","last_change":"2024-09-21T07:48:46.607052+0000","last_active":"2024-09-21T07:48:48.607657+0000","last_peered":"2024-09-21T07:48:48.607657+0000","last_clean":"2024-09-21T07:48:48.607657+0000","last_became_active":"2024-09-21T07:48:46.606707+0000","last_became_peered":"2024-09-21T07:48:46.606707+0000","last_unstale":"2024-09-21T07:48:48.607657+0000","last_undegraded":"2024-09-21T07:48:48.607657+0000","last_fullsized":"2024-09-21T07:48:48.607657+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:20.963352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":306,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051347+0000","last_change":"2024-09-21T07:35:27.871736+0000","last_active":"2024-09-21T07:48:50.051347+0000","last_peered":"2024-09-21T07:48:50.051347+0000","last_clean":"2024-09-21T07:48:50.051347+0000","last_became_active":"2024-09-21T07:35:27.871232+0000","last_became_peered":"2024-09-21T07:35:27.871232+0000","last_unstale":"2024-09-21T07:48:50.051347+0000","last_undegraded":"2024-09-21T07:48:50.051347+0000","last_fullsized":"2024-09-21T07:48:50.051347+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:04.419301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":316,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608800+0000","last_change":"2024-09-21T07:35:29.491246+0000","last_active":"2024-09-21T07:48:48.608800+0000","last_peered":"2024-09-21T07:48:48.608800+0000","last_clean":"2024-09-21T07:48:48.608800+0000","last_became_active":"2024-09-21T07:35:26.866783+0000","last_became_peered":"2024-09-21T07:35:26.866783+0000","last_unstale":"2024-09-21T07:48:48.608800+0000","last_undegraded":"2024-09-21T07:48:48.608800+0000","last_fullsized":"2024-09-21T07:48:48.608800+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:28.881914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098474099999999992,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":367,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.602292+0000","last_change":"2024-09-21T07:34:25.028640+0000","last_active":"2024-09-21T07:48:49.602292+0000","last_peered":"2024-09-21T07:48:49.602292+0000","last_clean":"2024-09-21T07:48:49.602292+0000","last_became_active":"2024-09-21T07:34:23.260950+0000","last_became_peered":"2024-09-21T07:34:23.260950+0000","last_unstale":"2024-09-21T07:48:49.602292+0000","last_undegraded":"2024-09-21T07:48:49.602292+0000","last_fullsized":"2024-09-21T07:48:49.602292+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:23:42.121398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055135099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051414+0000","last_change":"2024-09-21T07:48:47.614858+0000","last_active":"2024-09-21T07:48:50.051414+0000","last_peered":"2024-09-21T07:48:50.051414+0000","last_clean":"2024-09-21T07:48:50.051414+0000","last_became_active":"2024-09-21T07:48:47.614484+0000","last_became_peered":"2024-09-21T07:48:47.614484+0000","last_unstale":"2024-09-21T07:48:50.051414+0000","last_undegraded":"2024-09-21T07:48:50.051414+0000","last_fullsized":"2024-09-21T07:48:50.051414+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:04:35.741683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.601774+0000","last_change":"2024-09-21T07:48:46.607151+0000","last_active":"2024-09-21T07:48:49.601774+0000","last_peered":"2024-09-21T07:48:49.601774+0000","last_clean":"2024-09-21T07:48:49.601774+0000","last_became_active":"2024-09-21T07:48:46.606839+0000","last_became_peered":"2024-09-21T07:48:46.606839+0000","last_unstale":"2024-09-21T07:48:49.601774+0000","last_undegraded":"2024-09-21T07:48:49.601774+0000","last_fullsized":"2024-09-21T07:48:49.601774+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:34.780363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":307,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.553085+0000","last_change":"2024-09-21T07:35:27.872027+0000","last_active":"2024-09-21T07:48:49.553085+0000","last_peered":"2024-09-21T07:48:49.553085+0000","last_clean":"2024-09-21T07:48:49.553085+0000","last_became_active":"2024-09-21T07:35:27.870426+0000","last_became_peered":"2024-09-21T07:35:27.870426+0000","last_unstale":"2024-09-21T07:48:49.553085+0000","last_undegraded":"2024-09-21T07:48:49.553085+0000","last_fullsized":"2024-09-21T07:48:49.553085+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":316,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608519+0000","last_change":"2024-09-21T07:35:29.491008+0000","last_active":"2024-09-21T07:48:48.608519+0000","last_peered":"2024-09-21T07:48:48.608519+0000","last_clean":"2024-09-21T07:48:48.608519+0000","last_became_active":"2024-09-21T07:35:26.866750+0000","last_became_peered":"2024-09-21T07:35:26.866750+0000","last_unstale":"2024-09-21T07:48:48.608519+0000","last_undegraded":"2024-09-21T07:48:48.608519+0000","last_fullsized":"2024-09-21T07:48:48.608519+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:25.490406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083529700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":366,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609266+0000","last_change":"2024-09-21T07:34:24.260665+0000","last_active":"2024-09-21T07:48:48.609266+0000","last_peered":"2024-09-21T07:48:48.609266+0000","last_clean":"2024-09-21T07:48:48.609266+0000","last_became_active":"2024-09-21T07:34:23.635909+0000","last_became_peered":"2024-09-21T07:34:23.635909+0000","last_unstale":"2024-09-21T07:48:48.609266+0000","last_undegraded":"2024-09-21T07:48:48.609266+0000","last_fullsized":"2024-09-21T07:48:48.609266+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:44.536885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051072+0000","last_change":"2024-09-21T07:48:47.615147+0000","last_active":"2024-09-21T07:48:50.051072+0000","last_peered":"2024-09-21T07:48:50.051072+0000","last_clean":"2024-09-21T07:48:50.051072+0000","last_became_active":"2024-09-21T07:48:47.614792+0000","last_became_peered":"2024-09-21T07:48:47.614792+0000","last_unstale":"2024-09-21T07:48:50.051072+0000","last_undegraded":"2024-09-21T07:48:50.051072+0000","last_fullsized":"2024-09-21T07:48:50.051072+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:23.043769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608722+0000","last_change":"2024-09-21T07:48:46.614465+0000","last_active":"2024-09-21T07:48:48.608722+0000","last_peered":"2024-09-21T07:48:48.608722+0000","last_clean":"2024-09-21T07:48:48.608722+0000","last_became_active":"2024-09-21T07:48:46.614155+0000","last_became_peered":"2024-09-21T07:48:46.614155+0000","last_unstale":"2024-09-21T07:48:48.608722+0000","last_undegraded":"2024-09-21T07:48:48.608722+0000","last_fullsized":"2024-09-21T07:48:48.608722+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:23:53.595692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":303,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051021+0000","last_change":"2024-09-21T07:35:27.874772+0000","last_active":"2024-09-21T07:48:50.051021+0000","last_peered":"2024-09-21T07:48:50.051021+0000","last_clean":"2024-09-21T07:48:50.051021+0000","last_became_active":"2024-09-21T07:35:27.874434+0000","last_became_peered":"2024-09-21T07:35:27.874434+0000","last_unstale":"2024-09-21T07:48:50.051021+0000","last_undegraded":"2024-09-21T07:48:50.051021+0000","last_fullsized":"2024-09-21T07:48:50.051021+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:31.998755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":327,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.551358+0000","last_change":"2024-09-21T07:35:28.859153+0000","last_active":"2024-09-21T07:48:49.551358+0000","last_peered":"2024-09-21T07:48:49.551358+0000","last_clean":"2024-09-21T07:48:49.551358+0000","last_became_active":"2024-09-21T07:35:26.874428+0000","last_became_peered":"2024-09-21T07:35:26.874428+0000","last_unstale":"2024-09-21T07:48:49.551358+0000","last_undegraded":"2024-09-21T07:48:49.551358+0000","last_fullsized":"2024-09-21T07:48:49.551358+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:04.487621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048557299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":379,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.551111+0000","last_change":"2024-09-21T07:34:24.538845+0000","last_active":"2024-09-21T07:48:49.551111+0000","last_peered":"2024-09-21T07:48:49.551111+0000","last_clean":"2024-09-21T07:48:49.551111+0000","last_became_active":"2024-09-21T07:34:23.636372+0000","last_became_peered":"2024-09-21T07:34:23.636372+0000","last_unstale":"2024-09-21T07:48:49.551111+0000","last_undegraded":"2024-09-21T07:48:49.551111+0000","last_fullsized":"2024-09-21T07:48:49.551111+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:29.393813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841451,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.601892+0000","last_change":"2024-09-21T07:48:47.617117+0000","last_active":"2024-09-21T07:48:49.601892+0000","last_peered":"2024-09-21T07:48:49.601892+0000","last_clean":"2024-09-21T07:48:49.601892+0000","last_became_active":"2024-09-21T07:48:47.616813+0000","last_became_peered":"2024-09-21T07:48:47.616813+0000","last_unstale":"2024-09-21T07:48:49.601892+0000","last_undegraded":"2024-09-21T07:48:49.601892+0000","last_fullsized":"2024-09-21T07:48:49.601892+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:25.693091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608080+0000","last_change":"2024-09-21T07:48:46.610433+0000","last_active":"2024-09-21T07:48:48.608080+0000","last_peered":"2024-09-21T07:48:48.608080+0000","last_clean":"2024-09-21T07:48:48.608080+0000","last_became_active":"2024-09-21T07:48:46.610123+0000","last_became_peered":"2024-09-21T07:48:46.610123+0000","last_unstale":"2024-09-21T07:48:48.608080+0000","last_undegraded":"2024-09-21T07:48:48.608080+0000","last_fullsized":"2024-09-21T07:48:48.608080+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:19:23.729259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":308,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.551193+0000","last_change":"2024-09-21T07:35:27.877801+0000","last_active":"2024-09-21T07:48:49.551193+0000","last_peered":"2024-09-21T07:48:49.551193+0000","last_clean":"2024-09-21T07:48:49.551193+0000","last_became_active":"2024-09-21T07:35:27.877464+0000","last_became_peered":"2024-09-21T07:35:27.877464+0000","last_unstale":"2024-09-21T07:48:49.551193+0000","last_undegraded":"2024-09-21T07:48:49.551193+0000","last_fullsized":"2024-09-21T07:48:49.551193+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:08.814929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":314,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051401+0000","last_change":"2024-09-21T07:35:28.863732+0000","last_active":"2024-09-21T07:48:50.051401+0000","last_peered":"2024-09-21T07:48:50.051401+0000","last_clean":"2024-09-21T07:48:50.051401+0000","last_became_active":"2024-09-21T07:35:26.877392+0000","last_became_peered":"2024-09-21T07:35:26.877392+0000","last_unstale":"2024-09-21T07:48:50.051401+0000","last_undegraded":"2024-09-21T07:48:50.051401+0000","last_fullsized":"2024-09-21T07:48:50.051401+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071425700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":370,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.610101+0000","last_change":"2024-09-21T07:34:24.259686+0000","last_active":"2024-09-21T07:48:48.610101+0000","last_peered":"2024-09-21T07:48:48.610101+0000","last_clean":"2024-09-21T07:48:48.610101+0000","last_became_active":"2024-09-21T07:34:23.274405+0000","last_became_peered":"2024-09-21T07:34:23.274405+0000","last_unstale":"2024-09-21T07:48:48.610101+0000","last_undegraded":"2024-09-21T07:48:48.610101+0000","last_fullsized":"2024-09-21T07:48:48.610101+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:26.304693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022009920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.050593+0000","last_change":"2024-09-21T07:48:47.616194+0000","last_active":"2024-09-21T07:48:50.050593+0000","last_peered":"2024-09-21T07:48:50.050593+0000","last_clean":"2024-09-21T07:48:50.050593+0000","last_became_active":"2024-09-21T07:48:47.615876+0000","last_became_peered":"2024-09-21T07:48:47.615876+0000","last_unstale":"2024-09-21T07:48:50.050593+0000","last_undegraded":"2024-09-21T07:48:50.050593+0000","last_fullsized":"2024-09-21T07:48:50.050593+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:56.874912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"218'3","reported_seq":18,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.050860+0000","last_change":"2024-09-21T07:48:46.605473+0000","last_active":"2024-09-21T07:48:50.050860+0000","last_peered":"2024-09-21T07:48:50.050860+0000","last_clean":"2024-09-21T07:48:50.050860+0000","last_became_active":"2024-09-21T07:48:46.604548+0000","last_became_peered":"2024-09-21T07:48:46.604548+0000","last_unstale":"2024-09-21T07:48:50.050860+0000","last_undegraded":"2024-09-21T07:48:50.050860+0000","last_fullsized":"2024-09-21T07:48:50.050860+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:05.902786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":303,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051307+0000","last_change":"2024-09-21T07:35:27.867336+0000","last_active":"2024-09-21T07:48:50.051307+0000","last_peered":"2024-09-21T07:48:50.051307+0000","last_clean":"2024-09-21T07:48:50.051307+0000","last_became_active":"2024-09-21T07:35:27.867015+0000","last_became_peered":"2024-09-21T07:35:27.867015+0000","last_unstale":"2024-09-21T07:48:50.051307+0000","last_undegraded":"2024-09-21T07:48:50.051307+0000","last_fullsized":"2024-09-21T07:48:50.051307+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:59.214139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":313,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608985+0000","last_change":"2024-09-21T07:35:29.493137+0000","last_active":"2024-09-21T07:48:48.608985+0000","last_peered":"2024-09-21T07:48:48.608985+0000","last_clean":"2024-09-21T07:48:48.608985+0000","last_became_active":"2024-09-21T07:35:26.872230+0000","last_became_peered":"2024-09-21T07:35:26.872230+0000","last_unstale":"2024-09-21T07:48:48.608985+0000","last_undegraded":"2024-09-21T07:48:48.608985+0000","last_fullsized":"2024-09-21T07:48:48.608985+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:24.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001197449,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":366,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.607680+0000","last_change":"2024-09-21T07:34:24.258754+0000","last_active":"2024-09-21T07:48:48.607680+0000","last_peered":"2024-09-21T07:48:48.607680+0000","last_clean":"2024-09-21T07:48:48.607680+0000","last_became_active":"2024-09-21T07:34:23.272657+0000","last_became_peered":"2024-09-21T07:34:23.272657+0000","last_unstale":"2024-09-21T07:48:48.607680+0000","last_undegraded":"2024-09-21T07:48:48.607680+0000","last_fullsized":"2024-09-21T07:48:48.607680+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:05.477099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067143200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609516+0000","last_change":"2024-09-21T07:48:47.629237+0000","last_active":"2024-09-21T07:48:48.609516+0000","last_peered":"2024-09-21T07:48:48.609516+0000","last_clean":"2024-09-21T07:48:48.609516+0000","last_became_active":"2024-09-21T07:48:47.628900+0000","last_became_peered":"2024-09-21T07:48:47.628900+0000","last_unstale":"2024-09-21T07:48:48.609516+0000","last_undegraded":"2024-09-21T07:48:48.609516+0000","last_fullsized":"2024-09-21T07:48:48.609516+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:39.553041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"218'6","reported_seq":21,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608023+0000","last_change":"2024-09-21T07:48:46.612660+0000","last_active":"2024-09-21T07:48:48.608023+0000","last_peered":"2024-09-21T07:48:48.608023+0000","last_clean":"2024-09-21T07:48:48.608023+0000","last_became_active":"2024-09-21T07:48:46.612362+0000","last_became_peered":"2024-09-21T07:48:46.612362+0000","last_unstale":"2024-09-21T07:48:48.608023+0000","last_undegraded":"2024-09-21T07:48:48.608023+0000","last_fullsized":"2024-09-21T07:48:48.608023+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:06.201994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":307,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.050951+0000","last_change":"2024-09-21T07:35:27.876021+0000","last_active":"2024-09-21T07:48:50.050951+0000","last_peered":"2024-09-21T07:48:50.050951+0000","last_clean":"2024-09-21T07:48:50.050951+0000","last_became_active":"2024-09-21T07:35:27.875724+0000","last_became_peered":"2024-09-21T07:35:27.875724+0000","last_unstale":"2024-09-21T07:48:50.050951+0000","last_undegraded":"2024-09-21T07:48:50.050951+0000","last_fullsized":"2024-09-21T07:48:50.050951+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:03.022812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":313,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608151+0000","last_change":"2024-09-21T07:35:29.492570+0000","last_active":"2024-09-21T07:48:48.608151+0000","last_peered":"2024-09-21T07:48:48.608151+0000","last_clean":"2024-09-21T07:48:48.608151+0000","last_became_active":"2024-09-21T07:35:26.874316+0000","last_became_peered":"2024-09-21T07:35:26.874316+0000","last_unstale":"2024-09-21T07:48:48.608151+0000","last_undegraded":"2024-09-21T07:48:48.608151+0000","last_fullsized":"2024-09-21T07:48:48.608151+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:16.512323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011990760000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":365,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.050595+0000","last_change":"2024-09-21T07:34:25.085232+0000","last_active":"2024-09-21T07:48:50.050595+0000","last_peered":"2024-09-21T07:48:50.050595+0000","last_clean":"2024-09-21T07:48:50.050595+0000","last_became_active":"2024-09-21T07:34:23.277807+0000","last_became_peered":"2024-09-21T07:34:23.277807+0000","last_unstale":"2024-09-21T07:48:50.050595+0000","last_undegraded":"2024-09-21T07:48:50.050595+0000","last_fullsized":"2024-09-21T07:48:50.050595+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:36.362919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086619099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609167+0000","last_change":"2024-09-21T07:48:47.607795+0000","last_active":"2024-09-21T07:48:48.609167+0000","last_peered":"2024-09-21T07:48:48.609167+0000","last_clean":"2024-09-21T07:48:48.609167+0000","last_became_active":"2024-09-21T07:48:47.607511+0000","last_became_peered":"2024-09-21T07:48:47.607511+0000","last_unstale":"2024-09-21T07:48:48.609167+0000","last_undegraded":"2024-09-21T07:48:48.609167+0000","last_fullsized":"2024-09-21T07:48:48.609167+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:27.353767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"218'2","reported_seq":17,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609872+0000","last_change":"2024-09-21T07:48:46.609344+0000","last_active":"2024-09-21T07:48:48.609872+0000","last_peered":"2024-09-21T07:48:48.609872+0000","last_clean":"2024-09-21T07:48:48.609872+0000","last_became_active":"2024-09-21T07:48:46.606541+0000","last_became_peered":"2024-09-21T07:48:46.606541+0000","last_unstale":"2024-09-21T07:48:48.609872+0000","last_undegraded":"2024-09-21T07:48:48.609872+0000","last_fullsized":"2024-09-21T07:48:48.609872+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:36.274172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":308,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.552023+0000","last_change":"2024-09-21T07:35:27.872798+0000","last_active":"2024-09-21T07:48:49.552023+0000","last_peered":"2024-09-21T07:48:49.552023+0000","last_clean":"2024-09-21T07:48:49.552023+0000","last_became_active":"2024-09-21T07:35:27.872483+0000","last_became_peered":"2024-09-21T07:35:27.872483+0000","last_unstale":"2024-09-21T07:48:49.552023+0000","last_undegraded":"2024-09-21T07:48:49.552023+0000","last_fullsized":"2024-09-21T07:48:49.552023+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:11.933469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":315,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.552061+0000","last_change":"2024-09-21T07:35:29.206723+0000","last_active":"2024-09-21T07:48:49.552061+0000","last_peered":"2024-09-21T07:48:49.552061+0000","last_clean":"2024-09-21T07:48:49.552061+0000","last_became_active":"2024-09-21T07:35:26.872377+0000","last_became_peered":"2024-09-21T07:35:26.872377+0000","last_unstale":"2024-09-21T07:48:49.552061+0000","last_undegraded":"2024-09-21T07:48:49.552061+0000","last_fullsized":"2024-09-21T07:48:49.552061+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:29.086862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083077200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":359,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051990+0000","last_change":"2024-09-21T07:34:24.597672+0000","last_active":"2024-09-21T07:48:50.051990+0000","last_peered":"2024-09-21T07:48:50.051990+0000","last_clean":"2024-09-21T07:48:50.051990+0000","last_became_active":"2024-09-21T07:34:23.267770+0000","last_became_peered":"2024-09-21T07:34:23.267770+0000","last_unstale":"2024-09-21T07:48:50.051990+0000","last_undegraded":"2024-09-21T07:48:50.051990+0000","last_fullsized":"2024-09-21T07:48:50.051990+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:51.629476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086378500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.602650+0000","last_change":"2024-09-21T07:48:47.618502+0000","last_active":"2024-09-21T07:48:49.602650+0000","last_peered":"2024-09-21T07:48:49.602650+0000","last_clean":"2024-09-21T07:48:49.602650+0000","last_became_active":"2024-09-21T07:48:47.618213+0000","last_became_peered":"2024-09-21T07:48:47.618213+0000","last_unstale":"2024-09-21T07:48:49.602650+0000","last_undegraded":"2024-09-21T07:48:49.602650+0000","last_fullsized":"2024-09-21T07:48:49.602650+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:02.405575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.601927+0000","last_change":"2024-09-21T07:48:46.602566+0000","last_active":"2024-09-21T07:48:49.601927+0000","last_peered":"2024-09-21T07:48:49.601927+0000","last_clean":"2024-09-21T07:48:49.601927+0000","last_became_active":"2024-09-21T07:48:46.602254+0000","last_became_peered":"2024-09-21T07:48:46.602254+0000","last_unstale":"2024-09-21T07:48:49.601927+0000","last_undegraded":"2024-09-21T07:48:49.601927+0000","last_fullsized":"2024-09-21T07:48:49.601927+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:09.449030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":308,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.552715+0000","last_change":"2024-09-21T07:35:27.870121+0000","last_active":"2024-09-21T07:48:49.552715+0000","last_peered":"2024-09-21T07:48:49.552715+0000","last_clean":"2024-09-21T07:48:49.552715+0000","last_became_active":"2024-09-21T07:35:27.869811+0000","last_became_peered":"2024-09-21T07:35:27.869811+0000","last_unstale":"2024-09-21T07:48:49.552715+0000","last_undegraded":"2024-09-21T07:48:49.552715+0000","last_fullsized":"2024-09-21T07:48:49.552715+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:31.778461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":317,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608956+0000","last_change":"2024-09-21T07:35:28.860699+0000","last_active":"2024-09-21T07:48:48.608956+0000","last_peered":"2024-09-21T07:48:48.608956+0000","last_clean":"2024-09-21T07:48:48.608956+0000","last_became_active":"2024-09-21T07:35:26.876779+0000","last_became_peered":"2024-09-21T07:35:26.876779+0000","last_unstale":"2024-09-21T07:48:48.608956+0000","last_undegraded":"2024-09-21T07:48:48.608956+0000","last_fullsized":"2024-09-21T07:48:48.608956+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:28.286403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000913188,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":369,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.611431+0000","last_change":"2024-09-21T07:34:24.260181+0000","last_active":"2024-09-21T07:48:48.611431+0000","last_peered":"2024-09-21T07:48:48.611431+0000","last_clean":"2024-09-21T07:48:48.611431+0000","last_became_active":"2024-09-21T07:34:23.275580+0000","last_became_peered":"2024-09-21T07:34:23.275580+0000","last_unstale":"2024-09-21T07:48:48.611431+0000","last_undegraded":"2024-09-21T07:48:48.611431+0000","last_fullsized":"2024-09-21T07:48:48.611431+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:49.857622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014706980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051494+0000","last_change":"2024-09-21T07:48:47.627712+0000","last_active":"2024-09-21T07:48:50.051494+0000","last_peered":"2024-09-21T07:48:50.051494+0000","last_clean":"2024-09-21T07:48:50.051494+0000","last_became_active":"2024-09-21T07:48:47.623777+0000","last_became_peered":"2024-09-21T07:48:47.623777+0000","last_unstale":"2024-09-21T07:48:50.051494+0000","last_undegraded":"2024-09-21T07:48:50.051494+0000","last_fullsized":"2024-09-21T07:48:50.051494+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:42:59.497138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.050486+0000","last_change":"2024-09-21T07:48:46.602604+0000","last_active":"2024-09-21T07:48:50.050486+0000","last_peered":"2024-09-21T07:48:50.050486+0000","last_clean":"2024-09-21T07:48:50.050486+0000","last_became_active":"2024-09-21T07:48:46.602252+0000","last_became_peered":"2024-09-21T07:48:46.602252+0000","last_unstale":"2024-09-21T07:48:50.050486+0000","last_undegraded":"2024-09-21T07:48:50.050486+0000","last_fullsized":"2024-09-21T07:48:50.050486+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:02:41.387196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":309,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.602448+0000","last_change":"2024-09-21T07:35:27.875393+0000","last_active":"2024-09-21T07:48:49.602448+0000","last_peered":"2024-09-21T07:48:49.602448+0000","last_clean":"2024-09-21T07:48:49.602448+0000","last_became_active":"2024-09-21T07:35:27.874671+0000","last_became_peered":"2024-09-21T07:35:27.874671+0000","last_unstale":"2024-09-21T07:48:49.602448+0000","last_undegraded":"2024-09-21T07:48:49.602448+0000","last_fullsized":"2024-09-21T07:48:49.602448+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":315,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.551396+0000","last_change":"2024-09-21T07:35:29.206296+0000","last_active":"2024-09-21T07:48:49.551396+0000","last_peered":"2024-09-21T07:48:49.551396+0000","last_clean":"2024-09-21T07:48:49.551396+0000","last_became_active":"2024-09-21T07:35:26.874575+0000","last_became_peered":"2024-09-21T07:35:26.874575+0000","last_unstale":"2024-09-21T07:48:49.551396+0000","last_undegraded":"2024-09-21T07:48:49.551396+0000","last_fullsized":"2024-09-21T07:48:49.551396+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:22.771154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061011599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":370,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609427+0000","last_change":"2024-09-21T07:34:24.259112+0000","last_active":"2024-09-21T07:48:48.609427+0000","last_peered":"2024-09-21T07:48:48.609427+0000","last_clean":"2024-09-21T07:48:48.609427+0000","last_became_active":"2024-09-21T07:34:23.635444+0000","last_became_peered":"2024-09-21T07:34:23.635444+0000","last_unstale":"2024-09-21T07:48:48.609427+0000","last_undegraded":"2024-09-21T07:48:48.609427+0000","last_fullsized":"2024-09-21T07:48:48.609427+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:46.255385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001952636,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608515+0000","last_change":"2024-09-21T07:48:47.625691+0000","last_active":"2024-09-21T07:48:48.608515+0000","last_peered":"2024-09-21T07:48:48.608515+0000","last_clean":"2024-09-21T07:48:48.608515+0000","last_became_active":"2024-09-21T07:48:47.625169+0000","last_became_peered":"2024-09-21T07:48:47.625169+0000","last_unstale":"2024-09-21T07:48:48.608515+0000","last_undegraded":"2024-09-21T07:48:48.608515+0000","last_fullsized":"2024-09-21T07:48:48.608515+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:50.037460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.10","version":"218'8","reported_seq":23,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608107+0000","last_change":"2024-09-21T07:48:46.607918+0000","last_active":"2024-09-21T07:48:48.608107+0000","last_peered":"2024-09-21T07:48:48.608107+0000","last_clean":"2024-09-21T07:48:48.608107+0000","last_became_active":"2024-09-21T07:48:46.607600+0000","last_became_peered":"2024-09-21T07:48:46.607600+0000","last_unstale":"2024-09-21T07:48:48.608107+0000","last_undegraded":"2024-09-21T07:48:48.608107+0000","last_fullsized":"2024-09-21T07:48:48.608107+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:55:20.508968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":309,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.602702+0000","last_change":"2024-09-21T07:35:27.878356+0000","last_active":"2024-09-21T07:48:49.602702+0000","last_peered":"2024-09-21T07:48:49.602702+0000","last_clean":"2024-09-21T07:48:49.602702+0000","last_became_active":"2024-09-21T07:35:27.877982+0000","last_became_peered":"2024-09-21T07:35:27.877982+0000","last_unstale":"2024-09-21T07:48:49.602702+0000","last_undegraded":"2024-09-21T07:48:49.602702+0000","last_fullsized":"2024-09-21T07:48:49.602702+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":316,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.601909+0000","last_change":"2024-09-21T07:35:29.053131+0000","last_active":"2024-09-21T07:48:49.601909+0000","last_peered":"2024-09-21T07:48:49.601909+0000","last_clean":"2024-09-21T07:48:49.601909+0000","last_became_active":"2024-09-21T07:35:26.870516+0000","last_became_peered":"2024-09-21T07:35:26.870516+0000","last_unstale":"2024-09-21T07:48:49.601909+0000","last_undegraded":"2024-09-21T07:48:49.601909+0000","last_fullsized":"2024-09-21T07:48:49.601909+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:10.924491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070705699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":379,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.552030+0000","last_change":"2024-09-21T07:34:24.539152+0000","last_active":"2024-09-21T07:48:49.552030+0000","last_peered":"2024-09-21T07:48:49.552030+0000","last_clean":"2024-09-21T07:48:49.552030+0000","last_became_active":"2024-09-21T07:34:23.259573+0000","last_became_peered":"2024-09-21T07:34:23.259573+0000","last_unstale":"2024-09-21T07:48:49.552030+0000","last_undegraded":"2024-09-21T07:48:49.552030+0000","last_fullsized":"2024-09-21T07:48:49.552030+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:26.986900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058108400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.550320+0000","last_change":"2024-09-21T07:48:47.627105+0000","last_active":"2024-09-21T07:48:49.550320+0000","last_peered":"2024-09-21T07:48:49.550320+0000","last_clean":"2024-09-21T07:48:49.550320+0000","last_became_active":"2024-09-21T07:48:47.626559+0000","last_became_peered":"2024-09-21T07:48:47.626559+0000","last_unstale":"2024-09-21T07:48:49.550320+0000","last_undegraded":"2024-09-21T07:48:49.550320+0000","last_fullsized":"2024-09-21T07:48:49.550320+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:28:36.631991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.f","version":"218'1","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.551148+0000","last_change":"2024-09-21T07:48:46.607214+0000","last_active":"2024-09-21T07:48:49.551148+0000","last_peered":"2024-09-21T07:48:49.551148+0000","last_clean":"2024-09-21T07:48:49.551148+0000","last_became_active":"2024-09-21T07:48:46.606843+0000","last_became_peered":"2024-09-21T07:48:46.606843+0000","last_unstale":"2024-09-21T07:48:49.551148+0000","last_undegraded":"2024-09-21T07:48:49.551148+0000","last_fullsized":"2024-09-21T07:48:49.551148+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:47:20.124577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":308,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.601849+0000","last_change":"2024-09-21T07:35:27.873726+0000","last_active":"2024-09-21T07:48:49.601849+0000","last_peered":"2024-09-21T07:48:49.601849+0000","last_clean":"2024-09-21T07:48:49.601849+0000","last_became_active":"2024-09-21T07:35:27.871421+0000","last_became_peered":"2024-09-21T07:35:27.871421+0000","last_unstale":"2024-09-21T07:48:49.601849+0000","last_undegraded":"2024-09-21T07:48:49.601849+0000","last_fullsized":"2024-09-21T07:48:49.601849+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:00.168915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":317,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608845+0000","last_change":"2024-09-21T07:35:28.861032+0000","last_active":"2024-09-21T07:48:48.608845+0000","last_peered":"2024-09-21T07:48:48.608845+0000","last_clean":"2024-09-21T07:48:48.608845+0000","last_became_active":"2024-09-21T07:35:26.873149+0000","last_became_peered":"2024-09-21T07:35:26.873149+0000","last_unstale":"2024-09-21T07:48:48.608845+0000","last_undegraded":"2024-09-21T07:48:48.608845+0000","last_fullsized":"2024-09-21T07:48:48.608845+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:59.435981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011988369999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":366,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608427+0000","last_change":"2024-09-21T07:34:24.259902+0000","last_active":"2024-09-21T07:48:48.608427+0000","last_peered":"2024-09-21T07:48:48.608427+0000","last_clean":"2024-09-21T07:48:48.608427+0000","last_became_active":"2024-09-21T07:34:23.271902+0000","last_became_peered":"2024-09-21T07:34:23.271902+0000","last_unstale":"2024-09-21T07:48:48.608427+0000","last_undegraded":"2024-09-21T07:48:48.608427+0000","last_fullsized":"2024-09-21T07:48:48.608427+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:59.439525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001686358,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.550216+0000","last_change":"2024-09-21T07:48:47.618224+0000","last_active":"2024-09-21T07:48:49.550216+0000","last_peered":"2024-09-21T07:48:49.550216+0000","last_clean":"2024-09-21T07:48:49.550216+0000","last_became_active":"2024-09-21T07:48:47.617876+0000","last_became_peered":"2024-09-21T07:48:47.617876+0000","last_unstale":"2024-09-21T07:48:49.550216+0000","last_undegraded":"2024-09-21T07:48:49.550216+0000","last_fullsized":"2024-09-21T07:48:49.550216+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:46.265092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"218'2","reported_seq":17,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.550920+0000","last_change":"2024-09-21T07:48:46.606382+0000","last_active":"2024-09-21T07:48:49.550920+0000","last_peered":"2024-09-21T07:48:49.550920+0000","last_clean":"2024-09-21T07:48:49.550920+0000","last_became_active":"2024-09-21T07:48:46.606002+0000","last_became_peered":"2024-09-21T07:48:46.606002+0000","last_unstale":"2024-09-21T07:48:49.550920+0000","last_undegraded":"2024-09-21T07:48:49.550920+0000","last_fullsized":"2024-09-21T07:48:49.550920+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:48:18.461842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":306,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.050380+0000","last_change":"2024-09-21T07:35:27.872637+0000","last_active":"2024-09-21T07:48:50.050380+0000","last_peered":"2024-09-21T07:48:50.050380+0000","last_clean":"2024-09-21T07:48:50.050380+0000","last_became_active":"2024-09-21T07:35:27.872149+0000","last_became_peered":"2024-09-21T07:35:27.872149+0000","last_unstale":"2024-09-21T07:48:50.050380+0000","last_undegraded":"2024-09-21T07:48:50.050380+0000","last_fullsized":"2024-09-21T07:48:50.050380+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:27.697872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":313,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609094+0000","last_change":"2024-09-21T07:35:29.493362+0000","last_active":"2024-09-21T07:48:48.609094+0000","last_peered":"2024-09-21T07:48:48.609094+0000","last_clean":"2024-09-21T07:48:48.609094+0000","last_became_active":"2024-09-21T07:35:26.873018+0000","last_became_peered":"2024-09-21T07:35:26.873018+0000","last_unstale":"2024-09-21T07:48:48.609094+0000","last_undegraded":"2024-09-21T07:48:48.609094+0000","last_fullsized":"2024-09-21T07:48:48.609094+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:02.460557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012496510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":370,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609996+0000","last_change":"2024-09-21T07:34:24.259662+0000","last_active":"2024-09-21T07:48:48.609996+0000","last_peered":"2024-09-21T07:48:48.609996+0000","last_clean":"2024-09-21T07:48:48.609996+0000","last_became_active":"2024-09-21T07:34:23.278812+0000","last_became_peered":"2024-09-21T07:34:23.278812+0000","last_unstale":"2024-09-21T07:48:48.609996+0000","last_undegraded":"2024-09-21T07:48:48.609996+0000","last_fullsized":"2024-09-21T07:48:48.609996+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:49.829974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356463,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.550374+0000","last_change":"2024-09-21T07:48:47.598977+0000","last_active":"2024-09-21T07:48:49.550374+0000","last_peered":"2024-09-21T07:48:49.550374+0000","last_clean":"2024-09-21T07:48:49.550374+0000","last_became_active":"2024-09-21T07:48:47.598620+0000","last_became_peered":"2024-09-21T07:48:47.598620+0000","last_unstale":"2024-09-21T07:48:49.550374+0000","last_undegraded":"2024-09-21T07:48:49.550374+0000","last_fullsized":"2024-09-21T07:48:49.550374+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:07.336524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"218'1","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609860+0000","last_change":"2024-09-21T07:48:46.605998+0000","last_active":"2024-09-21T07:48:48.609860+0000","last_peered":"2024-09-21T07:48:48.609860+0000","last_clean":"2024-09-21T07:48:48.609860+0000","last_became_active":"2024-09-21T07:48:46.605714+0000","last_became_peered":"2024-09-21T07:48:46.605714+0000","last_unstale":"2024-09-21T07:48:48.609860+0000","last_undegraded":"2024-09-21T07:48:48.609860+0000","last_fullsized":"2024-09-21T07:48:48.609860+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:51.672371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":302,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051131+0000","last_change":"2024-09-21T07:35:27.868448+0000","last_active":"2024-09-21T07:48:50.051131+0000","last_peered":"2024-09-21T07:48:50.051131+0000","last_clean":"2024-09-21T07:48:50.051131+0000","last_became_active":"2024-09-21T07:35:27.868105+0000","last_became_peered":"2024-09-21T07:35:27.868105+0000","last_unstale":"2024-09-21T07:48:50.051131+0000","last_undegraded":"2024-09-21T07:48:50.051131+0000","last_fullsized":"2024-09-21T07:48:50.051131+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:41.468702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":312,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608736+0000","last_change":"2024-09-21T07:35:29.492974+0000","last_active":"2024-09-21T07:48:48.608736+0000","last_peered":"2024-09-21T07:48:48.608736+0000","last_clean":"2024-09-21T07:48:48.608736+0000","last_became_active":"2024-09-21T07:35:26.872165+0000","last_became_peered":"2024-09-21T07:35:26.872165+0000","last_unstale":"2024-09-21T07:48:48.608736+0000","last_undegraded":"2024-09-21T07:48:48.608736+0000","last_fullsized":"2024-09-21T07:48:48.608736+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:59.912520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013795179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":365,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.050414+0000","last_change":"2024-09-21T07:34:25.084861+0000","last_active":"2024-09-21T07:48:50.050414+0000","last_peered":"2024-09-21T07:48:50.050414+0000","last_clean":"2024-09-21T07:48:50.050414+0000","last_became_active":"2024-09-21T07:34:23.277708+0000","last_became_peered":"2024-09-21T07:34:23.277708+0000","last_unstale":"2024-09-21T07:48:50.050414+0000","last_undegraded":"2024-09-21T07:48:50.050414+0000","last_fullsized":"2024-09-21T07:48:50.050414+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:40.816488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000597566,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.551729+0000","last_change":"2024-09-21T07:48:47.621555+0000","last_active":"2024-09-21T07:48:49.551729+0000","last_peered":"2024-09-21T07:48:49.551729+0000","last_clean":"2024-09-21T07:48:49.551729+0000","last_became_active":"2024-09-21T07:48:47.620805+0000","last_became_peered":"2024-09-21T07:48:47.620805+0000","last_unstale":"2024-09-21T07:48:49.551729+0000","last_undegraded":"2024-09-21T07:48:49.551729+0000","last_fullsized":"2024-09-21T07:48:49.551729+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:34.601718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.610125+0000","last_change":"2024-09-21T07:48:46.611660+0000","last_active":"2024-09-21T07:48:48.610125+0000","last_peered":"2024-09-21T07:48:48.610125+0000","last_clean":"2024-09-21T07:48:48.610125+0000","last_became_active":"2024-09-21T07:48:46.610843+0000","last_became_peered":"2024-09-21T07:48:46.610843+0000","last_unstale":"2024-09-21T07:48:48.610125+0000","last_undegraded":"2024-09-21T07:48:48.610125+0000","last_fullsized":"2024-09-21T07:48:48.610125+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:43:38.733833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":302,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051512+0000","last_change":"2024-09-21T07:35:27.874640+0000","last_active":"2024-09-21T07:48:50.051512+0000","last_peered":"2024-09-21T07:48:50.051512+0000","last_clean":"2024-09-21T07:48:50.051512+0000","last_became_active":"2024-09-21T07:35:27.874265+0000","last_became_peered":"2024-09-21T07:35:27.874265+0000","last_unstale":"2024-09-21T07:48:50.051512+0000","last_undegraded":"2024-09-21T07:48:50.051512+0000","last_fullsized":"2024-09-21T07:48:50.051512+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:45.754654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":314,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.552233+0000","last_change":"2024-09-21T07:35:29.207392+0000","last_active":"2024-09-21T07:48:49.552233+0000","last_peered":"2024-09-21T07:48:49.552233+0000","last_clean":"2024-09-21T07:48:49.552233+0000","last_became_active":"2024-09-21T07:35:26.870917+0000","last_became_peered":"2024-09-21T07:35:26.870917+0000","last_unstale":"2024-09-21T07:48:49.552233+0000","last_undegraded":"2024-09-21T07:48:49.552233+0000","last_fullsized":"2024-09-21T07:48:49.552233+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:59.159937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059198499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":370,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608972+0000","last_change":"2024-09-21T07:34:24.258709+0000","last_active":"2024-09-21T07:48:48.608972+0000","last_peered":"2024-09-21T07:48:48.608972+0000","last_clean":"2024-09-21T07:48:48.608972+0000","last_became_active":"2024-09-21T07:34:23.279042+0000","last_became_peered":"2024-09-21T07:34:23.279042+0000","last_unstale":"2024-09-21T07:48:48.608972+0000","last_undegraded":"2024-09-21T07:48:48.608972+0000","last_fullsized":"2024-09-21T07:48:48.608972+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:01.762117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019282100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608922+0000","last_change":"2024-09-21T07:48:47.607091+0000","last_active":"2024-09-21T07:48:48.608922+0000","last_peered":"2024-09-21T07:48:48.608922+0000","last_clean":"2024-09-21T07:48:48.608922+0000","last_became_active":"2024-09-21T07:48:47.606739+0000","last_became_peered":"2024-09-21T07:48:47.606739+0000","last_unstale":"2024-09-21T07:48:48.608922+0000","last_undegraded":"2024-09-21T07:48:48.608922+0000","last_fullsized":"2024-09-21T07:48:48.608922+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:42:23.765051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"218'2","reported_seq":17,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609559+0000","last_change":"2024-09-21T07:48:46.612384+0000","last_active":"2024-09-21T07:48:48.609559+0000","last_peered":"2024-09-21T07:48:48.609559+0000","last_clean":"2024-09-21T07:48:48.609559+0000","last_became_active":"2024-09-21T07:48:46.610836+0000","last_became_peered":"2024-09-21T07:48:46.610836+0000","last_unstale":"2024-09-21T07:48:48.609559+0000","last_undegraded":"2024-09-21T07:48:48.609559+0000","last_fullsized":"2024-09-21T07:48:48.609559+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:04.310354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":306,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.050941+0000","last_change":"2024-09-21T07:35:27.870260+0000","last_active":"2024-09-21T07:48:50.050941+0000","last_peered":"2024-09-21T07:48:50.050941+0000","last_clean":"2024-09-21T07:48:50.050941+0000","last_became_active":"2024-09-21T07:35:27.869448+0000","last_became_peered":"2024-09-21T07:35:27.869448+0000","last_unstale":"2024-09-21T07:48:50.050941+0000","last_undegraded":"2024-09-21T07:48:50.050941+0000","last_fullsized":"2024-09-21T07:48:50.050941+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:16.946790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":309,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051488+0000","last_change":"2024-09-21T07:35:29.386367+0000","last_active":"2024-09-21T07:48:50.051488+0000","last_peered":"2024-09-21T07:48:50.051488+0000","last_clean":"2024-09-21T07:48:50.051488+0000","last_became_active":"2024-09-21T07:35:26.879238+0000","last_became_peered":"2024-09-21T07:35:26.879238+0000","last_unstale":"2024-09-21T07:48:50.051488+0000","last_undegraded":"2024-09-21T07:48:50.051488+0000","last_fullsized":"2024-09-21T07:48:50.051488+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:16.528260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092223100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":365,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051594+0000","last_change":"2024-09-21T07:34:25.085635+0000","last_active":"2024-09-21T07:48:50.051594+0000","last_peered":"2024-09-21T07:48:50.051594+0000","last_clean":"2024-09-21T07:48:50.051594+0000","last_became_active":"2024-09-21T07:34:23.273566+0000","last_became_peered":"2024-09-21T07:34:23.273566+0000","last_unstale":"2024-09-21T07:48:50.051594+0000","last_undegraded":"2024-09-21T07:48:50.051594+0000","last_fullsized":"2024-09-21T07:48:50.051594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:14.142272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079897300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608418+0000","last_change":"2024-09-21T07:48:47.625742+0000","last_active":"2024-09-21T07:48:48.608418+0000","last_peered":"2024-09-21T07:48:48.608418+0000","last_clean":"2024-09-21T07:48:48.608418+0000","last_became_active":"2024-09-21T07:48:47.625290+0000","last_became_peered":"2024-09-21T07:48:47.625290+0000","last_unstale":"2024-09-21T07:48:48.608418+0000","last_undegraded":"2024-09-21T07:48:48.608418+0000","last_fullsized":"2024-09-21T07:48:48.608418+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:15.258575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609104+0000","last_change":"2024-09-21T07:48:46.610951+0000","last_active":"2024-09-21T07:48:48.609104+0000","last_peered":"2024-09-21T07:48:48.609104+0000","last_clean":"2024-09-21T07:48:48.609104+0000","last_became_active":"2024-09-21T07:48:46.610658+0000","last_became_peered":"2024-09-21T07:48:46.610658+0000","last_unstale":"2024-09-21T07:48:48.609104+0000","last_undegraded":"2024-09-21T07:48:48.609104+0000","last_fullsized":"2024-09-21T07:48:48.609104+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:15:20.903496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":306,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051065+0000","last_change":"2024-09-21T07:35:27.878728+0000","last_active":"2024-09-21T07:48:50.051065+0000","last_peered":"2024-09-21T07:48:50.051065+0000","last_clean":"2024-09-21T07:48:50.051065+0000","last_became_active":"2024-09-21T07:35:27.878335+0000","last_became_peered":"2024-09-21T07:35:27.878335+0000","last_unstale":"2024-09-21T07:48:50.051065+0000","last_undegraded":"2024-09-21T07:48:50.051065+0000","last_fullsized":"2024-09-21T07:48:50.051065+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:25:37.731771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":326,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.550843+0000","last_change":"2024-09-21T07:35:28.858418+0000","last_active":"2024-09-21T07:48:49.550843+0000","last_peered":"2024-09-21T07:48:49.550843+0000","last_clean":"2024-09-21T07:48:49.550843+0000","last_became_active":"2024-09-21T07:35:26.876944+0000","last_became_peered":"2024-09-21T07:35:26.876944+0000","last_unstale":"2024-09-21T07:48:49.550843+0000","last_undegraded":"2024-09-21T07:48:49.550843+0000","last_fullsized":"2024-09-21T07:48:49.550843+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:04.514304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090862799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":358,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051805+0000","last_change":"2024-09-21T07:34:24.597926+0000","last_active":"2024-09-21T07:48:50.051805+0000","last_peered":"2024-09-21T07:48:50.051805+0000","last_clean":"2024-09-21T07:48:50.051805+0000","last_became_active":"2024-09-21T07:34:23.268926+0000","last_became_peered":"2024-09-21T07:34:23.268926+0000","last_unstale":"2024-09-21T07:48:50.051805+0000","last_undegraded":"2024-09-21T07:48:50.051805+0000","last_fullsized":"2024-09-21T07:48:50.051805+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:46.190752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101851,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051213+0000","last_change":"2024-09-21T07:48:47.624905+0000","last_active":"2024-09-21T07:48:50.051213+0000","last_peered":"2024-09-21T07:48:50.051213+0000","last_clean":"2024-09-21T07:48:50.051213+0000","last_became_active":"2024-09-21T07:48:47.621782+0000","last_became_peered":"2024-09-21T07:48:47.621782+0000","last_unstale":"2024-09-21T07:48:50.051213+0000","last_undegraded":"2024-09-21T07:48:50.051213+0000","last_fullsized":"2024-09-21T07:48:50.051213+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:18.457620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051148+0000","last_change":"2024-09-21T07:48:46.613457+0000","last_active":"2024-09-21T07:48:50.051148+0000","last_peered":"2024-09-21T07:48:50.051148+0000","last_clean":"2024-09-21T07:48:50.051148+0000","last_became_active":"2024-09-21T07:48:46.612037+0000","last_became_peered":"2024-09-21T07:48:46.612037+0000","last_unstale":"2024-09-21T07:48:50.051148+0000","last_undegraded":"2024-09-21T07:48:50.051148+0000","last_fullsized":"2024-09-21T07:48:50.051148+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:06.200769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":303,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051801+0000","last_change":"2024-09-21T07:35:27.877426+0000","last_active":"2024-09-21T07:48:50.051801+0000","last_peered":"2024-09-21T07:48:50.051801+0000","last_clean":"2024-09-21T07:48:50.051801+0000","last_became_active":"2024-09-21T07:35:27.877050+0000","last_became_peered":"2024-09-21T07:35:27.877050+0000","last_unstale":"2024-09-21T07:48:50.051801+0000","last_undegraded":"2024-09-21T07:48:50.051801+0000","last_fullsized":"2024-09-21T07:48:50.051801+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:36.268334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":313,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.050387+0000","last_change":"2024-09-21T07:35:28.862273+0000","last_active":"2024-09-21T07:48:50.050387+0000","last_peered":"2024-09-21T07:48:50.050387+0000","last_clean":"2024-09-21T07:48:50.050387+0000","last_became_active":"2024-09-21T07:35:26.874011+0000","last_became_peered":"2024-09-21T07:35:26.874011+0000","last_unstale":"2024-09-21T07:48:50.050387+0000","last_undegraded":"2024-09-21T07:48:50.050387+0000","last_fullsized":"2024-09-21T07:48:50.050387+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:11.986950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071644799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":369,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.610345+0000","last_change":"2024-09-21T07:34:24.259794+0000","last_active":"2024-09-21T07:48:48.610345+0000","last_peered":"2024-09-21T07:48:48.610345+0000","last_clean":"2024-09-21T07:48:48.610345+0000","last_became_active":"2024-09-21T07:34:23.265679+0000","last_became_peered":"2024-09-21T07:34:23.265679+0000","last_unstale":"2024-09-21T07:48:48.610345+0000","last_undegraded":"2024-09-21T07:48:48.610345+0000","last_fullsized":"2024-09-21T07:48:48.610345+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:02.575796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001298518,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.552939+0000","last_change":"2024-09-21T07:48:47.618294+0000","last_active":"2024-09-21T07:48:49.552939+0000","last_peered":"2024-09-21T07:48:49.552939+0000","last_clean":"2024-09-21T07:48:49.552939+0000","last_became_active":"2024-09-21T07:48:47.617975+0000","last_became_peered":"2024-09-21T07:48:47.617975+0000","last_unstale":"2024-09-21T07:48:49.552939+0000","last_undegraded":"2024-09-21T07:48:49.552939+0000","last_fullsized":"2024-09-21T07:48:49.552939+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:28:21.189669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.550020+0000","last_change":"2024-09-21T07:48:46.608424+0000","last_active":"2024-09-21T07:48:49.550020+0000","last_peered":"2024-09-21T07:48:49.550020+0000","last_clean":"2024-09-21T07:48:49.550020+0000","last_became_active":"2024-09-21T07:48:46.607778+0000","last_became_peered":"2024-09-21T07:48:46.607778+0000","last_unstale":"2024-09-21T07:48:49.550020+0000","last_undegraded":"2024-09-21T07:48:49.550020+0000","last_fullsized":"2024-09-21T07:48:49.550020+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:14.577558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":308,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.552899+0000","last_change":"2024-09-21T07:35:27.877725+0000","last_active":"2024-09-21T07:48:49.552899+0000","last_peered":"2024-09-21T07:48:49.552899+0000","last_clean":"2024-09-21T07:48:49.552899+0000","last_became_active":"2024-09-21T07:35:27.877398+0000","last_became_peered":"2024-09-21T07:35:27.877398+0000","last_unstale":"2024-09-21T07:48:49.552899+0000","last_undegraded":"2024-09-21T07:48:49.552899+0000","last_fullsized":"2024-09-21T07:48:49.552899+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:05.859762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":316,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608538+0000","last_change":"2024-09-21T07:35:28.860612+0000","last_active":"2024-09-21T07:48:48.608538+0000","last_peered":"2024-09-21T07:48:48.608538+0000","last_clean":"2024-09-21T07:48:48.608538+0000","last_became_active":"2024-09-21T07:35:26.876562+0000","last_became_peered":"2024-09-21T07:35:26.876562+0000","last_unstale":"2024-09-21T07:48:48.608538+0000","last_undegraded":"2024-09-21T07:48:48.608538+0000","last_fullsized":"2024-09-21T07:48:48.608538+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:06.824331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012252470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":366,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609177+0000","last_change":"2024-09-21T07:34:24.260330+0000","last_active":"2024-09-21T07:48:48.609177+0000","last_peered":"2024-09-21T07:48:48.609177+0000","last_clean":"2024-09-21T07:48:48.609177+0000","last_became_active":"2024-09-21T07:34:23.637210+0000","last_became_peered":"2024-09-21T07:34:23.637210+0000","last_unstale":"2024-09-21T07:48:48.609177+0000","last_undegraded":"2024-09-21T07:48:48.609177+0000","last_fullsized":"2024-09-21T07:48:48.609177+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.488451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001537093,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.550464+0000","last_change":"2024-09-21T07:48:47.626890+0000","last_active":"2024-09-21T07:48:49.550464+0000","last_peered":"2024-09-21T07:48:49.550464+0000","last_clean":"2024-09-21T07:48:49.550464+0000","last_became_active":"2024-09-21T07:48:47.626088+0000","last_became_peered":"2024-09-21T07:48:47.626088+0000","last_unstale":"2024-09-21T07:48:49.550464+0000","last_undegraded":"2024-09-21T07:48:49.550464+0000","last_fullsized":"2024-09-21T07:48:49.550464+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:06:43.660952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.7","version":"218'2","reported_seq":17,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051435+0000","last_change":"2024-09-21T07:48:46.607721+0000","last_active":"2024-09-21T07:48:50.051435+0000","last_peered":"2024-09-21T07:48:50.051435+0000","last_clean":"2024-09-21T07:48:50.051435+0000","last_became_active":"2024-09-21T07:48:46.607413+0000","last_became_peered":"2024-09-21T07:48:46.607413+0000","last_unstale":"2024-09-21T07:48:50.051435+0000","last_undegraded":"2024-09-21T07:48:50.051435+0000","last_fullsized":"2024-09-21T07:48:50.051435+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:47.152954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":306,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.050294+0000","last_change":"2024-09-21T07:35:27.874757+0000","last_active":"2024-09-21T07:48:50.050294+0000","last_peered":"2024-09-21T07:48:50.050294+0000","last_clean":"2024-09-21T07:48:50.050294+0000","last_became_active":"2024-09-21T07:35:27.874360+0000","last_became_peered":"2024-09-21T07:35:27.874360+0000","last_unstale":"2024-09-21T07:48:50.050294+0000","last_undegraded":"2024-09-21T07:48:50.050294+0000","last_fullsized":"2024-09-21T07:48:50.050294+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:17.646912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":312,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608687+0000","last_change":"2024-09-21T07:35:29.493431+0000","last_active":"2024-09-21T07:48:48.608687+0000","last_peered":"2024-09-21T07:48:48.608687+0000","last_clean":"2024-09-21T07:48:48.608687+0000","last_became_active":"2024-09-21T07:35:26.873691+0000","last_became_peered":"2024-09-21T07:35:26.873691+0000","last_unstale":"2024-09-21T07:48:48.608687+0000","last_undegraded":"2024-09-21T07:48:48.608687+0000","last_fullsized":"2024-09-21T07:48:48.608687+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:51.705051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001162464,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":358,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051405+0000","last_change":"2024-09-21T07:34:24.597286+0000","last_active":"2024-09-21T07:48:50.051405+0000","last_peered":"2024-09-21T07:48:50.051405+0000","last_clean":"2024-09-21T07:48:50.051405+0000","last_became_active":"2024-09-21T07:34:23.267133+0000","last_became_peered":"2024-09-21T07:34:23.267133+0000","last_unstale":"2024-09-21T07:48:50.051405+0000","last_undegraded":"2024-09-21T07:48:50.051405+0000","last_fullsized":"2024-09-21T07:48:50.051405+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:18.257010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.601891+0000","last_change":"2024-09-21T07:48:47.624322+0000","last_active":"2024-09-21T07:48:49.601891+0000","last_peered":"2024-09-21T07:48:49.601891+0000","last_clean":"2024-09-21T07:48:49.601891+0000","last_became_active":"2024-09-21T07:48:47.623920+0000","last_became_peered":"2024-09-21T07:48:47.623920+0000","last_unstale":"2024-09-21T07:48:49.601891+0000","last_undegraded":"2024-09-21T07:48:49.601891+0000","last_fullsized":"2024-09-21T07:48:49.601891+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:08.454622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"219'1","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.552713+0000","last_change":"2024-09-21T07:48:46.599263+0000","last_active":"2024-09-21T07:48:49.552713+0000","last_peered":"2024-09-21T07:48:49.552713+0000","last_clean":"2024-09-21T07:48:49.552713+0000","last_became_active":"2024-09-21T07:48:46.597628+0000","last_became_peered":"2024-09-21T07:48:46.597628+0000","last_unstale":"2024-09-21T07:48:49.552713+0000","last_undegraded":"2024-09-21T07:48:49.552713+0000","last_fullsized":"2024-09-21T07:48:49.552713+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:38.541554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":319,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.550320+0000","last_change":"2024-09-21T07:35:27.864458+0000","last_active":"2024-09-21T07:48:49.550320+0000","last_peered":"2024-09-21T07:48:49.550320+0000","last_clean":"2024-09-21T07:48:49.550320+0000","last_became_active":"2024-09-21T07:35:27.864136+0000","last_became_peered":"2024-09-21T07:35:27.864136+0000","last_unstale":"2024-09-21T07:48:49.550320+0000","last_undegraded":"2024-09-21T07:48:49.550320+0000","last_fullsized":"2024-09-21T07:48:49.550320+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:20.162240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":326,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.551403+0000","last_change":"2024-09-21T07:35:28.858784+0000","last_active":"2024-09-21T07:48:49.551403+0000","last_peered":"2024-09-21T07:48:49.551403+0000","last_clean":"2024-09-21T07:48:49.551403+0000","last_became_active":"2024-09-21T07:35:26.876890+0000","last_became_peered":"2024-09-21T07:35:26.876890+0000","last_unstale":"2024-09-21T07:48:49.551403+0000","last_undegraded":"2024-09-21T07:48:49.551403+0000","last_fullsized":"2024-09-21T07:48:49.551403+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:40.738327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064543499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":369,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609757+0000","last_change":"2024-09-21T07:34:24.258712+0000","last_active":"2024-09-21T07:48:48.609757+0000","last_peered":"2024-09-21T07:48:48.609757+0000","last_clean":"2024-09-21T07:48:48.609757+0000","last_became_active":"2024-09-21T07:34:23.274692+0000","last_became_peered":"2024-09-21T07:34:23.274692+0000","last_unstale":"2024-09-21T07:48:48.609757+0000","last_undegraded":"2024-09-21T07:48:48.609757+0000","last_fullsized":"2024-09-21T07:48:48.609757+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:43.279454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018051479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.050557+0000","last_change":"2024-09-21T07:48:47.617896+0000","last_active":"2024-09-21T07:48:50.050557+0000","last_peered":"2024-09-21T07:48:50.050557+0000","last_clean":"2024-09-21T07:48:50.050557+0000","last_became_active":"2024-09-21T07:48:47.617588+0000","last_became_peered":"2024-09-21T07:48:47.617588+0000","last_unstale":"2024-09-21T07:48:50.050557+0000","last_undegraded":"2024-09-21T07:48:50.050557+0000","last_fullsized":"2024-09-21T07:48:50.050557+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:47.436066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051487+0000","last_change":"2024-09-21T07:48:46.609456+0000","last_active":"2024-09-21T07:48:50.051487+0000","last_peered":"2024-09-21T07:48:50.051487+0000","last_clean":"2024-09-21T07:48:50.051487+0000","last_became_active":"2024-09-21T07:48:46.609142+0000","last_became_peered":"2024-09-21T07:48:46.609142+0000","last_unstale":"2024-09-21T07:48:50.051487+0000","last_undegraded":"2024-09-21T07:48:50.051487+0000","last_fullsized":"2024-09-21T07:48:50.051487+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:21.813300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":308,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609370+0000","last_change":"2024-09-21T07:35:27.878020+0000","last_active":"2024-09-21T07:48:48.609370+0000","last_peered":"2024-09-21T07:48:48.609370+0000","last_clean":"2024-09-21T07:48:48.609370+0000","last_became_active":"2024-09-21T07:35:27.877532+0000","last_became_peered":"2024-09-21T07:35:27.877532+0000","last_unstale":"2024-09-21T07:48:48.609370+0000","last_undegraded":"2024-09-21T07:48:48.609370+0000","last_fullsized":"2024-09-21T07:48:48.609370+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:19.756073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":316,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.610786+0000","last_change":"2024-09-21T07:35:28.861049+0000","last_active":"2024-09-21T07:48:48.610786+0000","last_peered":"2024-09-21T07:48:48.610786+0000","last_clean":"2024-09-21T07:48:48.610786+0000","last_became_active":"2024-09-21T07:35:26.876819+0000","last_became_peered":"2024-09-21T07:35:26.876819+0000","last_unstale":"2024-09-21T07:48:48.610786+0000","last_undegraded":"2024-09-21T07:48:48.610786+0000","last_fullsized":"2024-09-21T07:48:48.610786+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:19.633803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055780599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":365,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.050459+0000","last_change":"2024-09-21T07:34:25.084849+0000","last_active":"2024-09-21T07:48:50.050459+0000","last_peered":"2024-09-21T07:48:50.050459+0000","last_clean":"2024-09-21T07:48:50.050459+0000","last_became_active":"2024-09-21T07:34:23.277713+0000","last_became_peered":"2024-09-21T07:34:23.277713+0000","last_unstale":"2024-09-21T07:48:50.050459+0000","last_undegraded":"2024-09-21T07:48:50.050459+0000","last_fullsized":"2024-09-21T07:48:50.050459+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:05.811810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059079499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.550788+0000","last_change":"2024-09-21T07:48:47.611938+0000","last_active":"2024-09-21T07:48:49.550788+0000","last_peered":"2024-09-21T07:48:49.550788+0000","last_clean":"2024-09-21T07:48:49.550788+0000","last_became_active":"2024-09-21T07:48:47.611580+0000","last_became_peered":"2024-09-21T07:48:47.611580+0000","last_unstale":"2024-09-21T07:48:49.550788+0000","last_undegraded":"2024-09-21T07:48:49.550788+0000","last_fullsized":"2024-09-21T07:48:49.550788+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:12:50.764399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.551355+0000","last_change":"2024-09-21T07:48:46.607396+0000","last_active":"2024-09-21T07:48:49.551355+0000","last_peered":"2024-09-21T07:48:49.551355+0000","last_clean":"2024-09-21T07:48:49.551355+0000","last_became_active":"2024-09-21T07:48:46.607082+0000","last_became_peered":"2024-09-21T07:48:46.607082+0000","last_unstale":"2024-09-21T07:48:49.551355+0000","last_undegraded":"2024-09-21T07:48:49.551355+0000","last_fullsized":"2024-09-21T07:48:49.551355+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:53:23.751238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":308,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.602217+0000","last_change":"2024-09-21T07:35:27.884248+0000","last_active":"2024-09-21T07:48:49.602217+0000","last_peered":"2024-09-21T07:48:49.602217+0000","last_clean":"2024-09-21T07:48:49.602217+0000","last_became_active":"2024-09-21T07:35:27.883861+0000","last_became_peered":"2024-09-21T07:35:27.883861+0000","last_unstale":"2024-09-21T07:48:49.602217+0000","last_undegraded":"2024-09-21T07:48:49.602217+0000","last_fullsized":"2024-09-21T07:48:49.602217+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":314,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.552147+0000","last_change":"2024-09-21T07:35:29.207135+0000","last_active":"2024-09-21T07:48:49.552147+0000","last_peered":"2024-09-21T07:48:49.552147+0000","last_clean":"2024-09-21T07:48:49.552147+0000","last_became_active":"2024-09-21T07:35:26.872787+0000","last_became_peered":"2024-09-21T07:35:26.872787+0000","last_unstale":"2024-09-21T07:48:49.552147+0000","last_undegraded":"2024-09-21T07:48:49.552147+0000","last_fullsized":"2024-09-21T07:48:49.552147+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:27.543217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011157509999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":378,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.550475+0000","last_change":"2024-09-21T07:34:24.538610+0000","last_active":"2024-09-21T07:48:49.550475+0000","last_peered":"2024-09-21T07:48:49.550475+0000","last_clean":"2024-09-21T07:48:49.550475+0000","last_became_active":"2024-09-21T07:34:23.259700+0000","last_became_peered":"2024-09-21T07:34:23.259700+0000","last_unstale":"2024-09-21T07:48:49.550475+0000","last_undegraded":"2024-09-21T07:48:49.550475+0000","last_fullsized":"2024-09-21T07:48:49.550475+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:57.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070522399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.551089+0000","last_change":"2024-09-21T07:48:47.623345+0000","last_active":"2024-09-21T07:48:49.551089+0000","last_peered":"2024-09-21T07:48:49.551089+0000","last_clean":"2024-09-21T07:48:49.551089+0000","last_became_active":"2024-09-21T07:48:47.622998+0000","last_became_peered":"2024-09-21T07:48:47.622998+0000","last_unstale":"2024-09-21T07:48:49.551089+0000","last_undegraded":"2024-09-21T07:48:49.551089+0000","last_fullsized":"2024-09-21T07:48:49.551089+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:55:53.150258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.551941+0000","last_change":"2024-09-21T07:48:46.599294+0000","last_active":"2024-09-21T07:48:49.551941+0000","last_peered":"2024-09-21T07:48:49.551941+0000","last_clean":"2024-09-21T07:48:49.551941+0000","last_became_active":"2024-09-21T07:48:46.598633+0000","last_became_peered":"2024-09-21T07:48:46.598633+0000","last_unstale":"2024-09-21T07:48:49.551941+0000","last_undegraded":"2024-09-21T07:48:49.551941+0000","last_fullsized":"2024-09-21T07:48:49.551941+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:34.227316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":319,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.551422+0000","last_change":"2024-09-21T07:35:27.868024+0000","last_active":"2024-09-21T07:48:49.551422+0000","last_peered":"2024-09-21T07:48:49.551422+0000","last_clean":"2024-09-21T07:48:49.551422+0000","last_became_active":"2024-09-21T07:35:27.867339+0000","last_became_peered":"2024-09-21T07:35:27.867339+0000","last_unstale":"2024-09-21T07:48:49.551422+0000","last_undegraded":"2024-09-21T07:48:49.551422+0000","last_fullsized":"2024-09-21T07:48:49.551422+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":326,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.550780+0000","last_change":"2024-09-21T07:35:28.858505+0000","last_active":"2024-09-21T07:48:49.550780+0000","last_peered":"2024-09-21T07:48:49.550780+0000","last_clean":"2024-09-21T07:48:49.550780+0000","last_became_active":"2024-09-21T07:35:26.877320+0000","last_became_peered":"2024-09-21T07:35:26.877320+0000","last_unstale":"2024-09-21T07:48:49.550780+0000","last_undegraded":"2024-09-21T07:48:49.550780+0000","last_fullsized":"2024-09-21T07:48:49.550780+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:53.672782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244119999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":360,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.052005+0000","last_change":"2024-09-21T07:34:24.597597+0000","last_active":"2024-09-21T07:48:50.052005+0000","last_peered":"2024-09-21T07:48:50.052005+0000","last_clean":"2024-09-21T07:48:50.052005+0000","last_became_active":"2024-09-21T07:34:23.635204+0000","last_became_peered":"2024-09-21T07:34:23.635204+0000","last_unstale":"2024-09-21T07:48:50.052005+0000","last_undegraded":"2024-09-21T07:48:50.052005+0000","last_fullsized":"2024-09-21T07:48:50.052005+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:48.644789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081139199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609468+0000","last_change":"2024-09-21T07:48:47.624826+0000","last_active":"2024-09-21T07:48:48.609468+0000","last_peered":"2024-09-21T07:48:48.609468+0000","last_clean":"2024-09-21T07:48:48.609468+0000","last_became_active":"2024-09-21T07:48:47.624259+0000","last_became_peered":"2024-09-21T07:48:47.624259+0000","last_unstale":"2024-09-21T07:48:48.609468+0000","last_undegraded":"2024-09-21T07:48:48.609468+0000","last_fullsized":"2024-09-21T07:48:48.609468+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:07.420806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051183+0000","last_change":"2024-09-21T07:48:46.602945+0000","last_active":"2024-09-21T07:48:50.051183+0000","last_peered":"2024-09-21T07:48:50.051183+0000","last_clean":"2024-09-21T07:48:50.051183+0000","last_became_active":"2024-09-21T07:48:46.602486+0000","last_became_peered":"2024-09-21T07:48:46.602486+0000","last_unstale":"2024-09-21T07:48:50.051183+0000","last_undegraded":"2024-09-21T07:48:50.051183+0000","last_fullsized":"2024-09-21T07:48:50.051183+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:51:05.478321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":307,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.552544+0000","last_change":"2024-09-21T07:35:27.877503+0000","last_active":"2024-09-21T07:48:49.552544+0000","last_peered":"2024-09-21T07:48:49.552544+0000","last_clean":"2024-09-21T07:48:49.552544+0000","last_became_active":"2024-09-21T07:35:27.875775+0000","last_became_peered":"2024-09-21T07:35:27.875775+0000","last_unstale":"2024-09-21T07:48:49.552544+0000","last_undegraded":"2024-09-21T07:48:49.552544+0000","last_fullsized":"2024-09-21T07:48:49.552544+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:49.291023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":312,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608298+0000","last_change":"2024-09-21T07:35:29.491412+0000","last_active":"2024-09-21T07:48:48.608298+0000","last_peered":"2024-09-21T07:48:48.608298+0000","last_clean":"2024-09-21T07:48:48.608298+0000","last_became_active":"2024-09-21T07:35:26.876404+0000","last_became_peered":"2024-09-21T07:35:26.876404+0000","last_unstale":"2024-09-21T07:48:48.608298+0000","last_undegraded":"2024-09-21T07:48:48.608298+0000","last_fullsized":"2024-09-21T07:48:48.608298+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063918000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":370,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.610303+0000","last_change":"2024-09-21T07:34:24.259026+0000","last_active":"2024-09-21T07:48:48.610303+0000","last_peered":"2024-09-21T07:48:48.610303+0000","last_clean":"2024-09-21T07:48:48.610303+0000","last_became_active":"2024-09-21T07:34:23.635456+0000","last_became_peered":"2024-09-21T07:34:23.635456+0000","last_unstale":"2024-09-21T07:48:48.610303+0000","last_undegraded":"2024-09-21T07:48:48.610303+0000","last_fullsized":"2024-09-21T07:48:48.610303+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:09.592752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018654419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609222+0000","last_change":"2024-09-21T07:48:47.624848+0000","last_active":"2024-09-21T07:48:48.609222+0000","last_peered":"2024-09-21T07:48:48.609222+0000","last_clean":"2024-09-21T07:48:48.609222+0000","last_became_active":"2024-09-21T07:48:47.624301+0000","last_became_peered":"2024-09-21T07:48:47.624301+0000","last_unstale":"2024-09-21T07:48:48.609222+0000","last_undegraded":"2024-09-21T07:48:48.609222+0000","last_fullsized":"2024-09-21T07:48:48.609222+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:24.141015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.550123+0000","last_change":"2024-09-21T07:48:46.612048+0000","last_active":"2024-09-21T07:48:49.550123+0000","last_peered":"2024-09-21T07:48:49.550123+0000","last_clean":"2024-09-21T07:48:49.550123+0000","last_became_active":"2024-09-21T07:48:46.611755+0000","last_became_peered":"2024-09-21T07:48:46.611755+0000","last_unstale":"2024-09-21T07:48:49.550123+0000","last_undegraded":"2024-09-21T07:48:49.550123+0000","last_fullsized":"2024-09-21T07:48:49.550123+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:16.350687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":307,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.552864+0000","last_change":"2024-09-21T07:35:27.871984+0000","last_active":"2024-09-21T07:48:49.552864+0000","last_peered":"2024-09-21T07:48:49.552864+0000","last_clean":"2024-09-21T07:48:49.552864+0000","last_became_active":"2024-09-21T07:35:27.870336+0000","last_became_peered":"2024-09-21T07:35:27.870336+0000","last_unstale":"2024-09-21T07:48:49.552864+0000","last_undegraded":"2024-09-21T07:48:49.552864+0000","last_fullsized":"2024-09-21T07:48:49.552864+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:57.120479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":316,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609408+0000","last_change":"2024-09-21T07:35:28.860351+0000","last_active":"2024-09-21T07:48:48.609408+0000","last_peered":"2024-09-21T07:48:48.609408+0000","last_clean":"2024-09-21T07:48:48.609408+0000","last_became_active":"2024-09-21T07:35:26.872046+0000","last_became_peered":"2024-09-21T07:35:26.872046+0000","last_unstale":"2024-09-21T07:48:48.609408+0000","last_undegraded":"2024-09-21T07:48:48.609408+0000","last_fullsized":"2024-09-21T07:48:48.609408+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:47.074327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098384399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":365,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609455+0000","last_change":"2024-09-21T07:34:24.260300+0000","last_active":"2024-09-21T07:48:48.609455+0000","last_peered":"2024-09-21T07:48:48.609455+0000","last_clean":"2024-09-21T07:48:48.609455+0000","last_became_active":"2024-09-21T07:34:23.270128+0000","last_became_peered":"2024-09-21T07:34:23.270128+0000","last_unstale":"2024-09-21T07:48:48.609455+0000","last_undegraded":"2024-09-21T07:48:48.609455+0000","last_fullsized":"2024-09-21T07:48:48.609455+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:25.260931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00176651,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.602582+0000","last_change":"2024-09-21T07:48:47.626058+0000","last_active":"2024-09-21T07:48:49.602582+0000","last_peered":"2024-09-21T07:48:49.602582+0000","last_clean":"2024-09-21T07:48:49.602582+0000","last_became_active":"2024-09-21T07:48:47.625779+0000","last_became_peered":"2024-09-21T07:48:47.625779+0000","last_unstale":"2024-09-21T07:48:49.602582+0000","last_undegraded":"2024-09-21T07:48:49.602582+0000","last_fullsized":"2024-09-21T07:48:49.602582+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:47.074247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.610266+0000","last_change":"2024-09-21T07:48:46.607858+0000","last_active":"2024-09-21T07:48:48.610266+0000","last_peered":"2024-09-21T07:48:48.610266+0000","last_clean":"2024-09-21T07:48:48.610266+0000","last_became_active":"2024-09-21T07:48:46.607521+0000","last_became_peered":"2024-09-21T07:48:46.607521+0000","last_unstale":"2024-09-21T07:48:48.610266+0000","last_undegraded":"2024-09-21T07:48:48.610266+0000","last_fullsized":"2024-09-21T07:48:48.610266+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:18.298760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":307,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.553043+0000","last_change":"2024-09-21T07:35:27.877861+0000","last_active":"2024-09-21T07:48:49.553043+0000","last_peered":"2024-09-21T07:48:49.553043+0000","last_clean":"2024-09-21T07:48:49.553043+0000","last_became_active":"2024-09-21T07:35:27.877434+0000","last_became_peered":"2024-09-21T07:35:27.877434+0000","last_unstale":"2024-09-21T07:48:49.553043+0000","last_undegraded":"2024-09-21T07:48:49.553043+0000","last_fullsized":"2024-09-21T07:48:49.553043+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":313,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.050648+0000","last_change":"2024-09-21T07:35:28.862954+0000","last_active":"2024-09-21T07:48:50.050648+0000","last_peered":"2024-09-21T07:48:50.050648+0000","last_clean":"2024-09-21T07:48:50.050648+0000","last_became_active":"2024-09-21T07:35:26.870613+0000","last_became_peered":"2024-09-21T07:35:26.870613+0000","last_unstale":"2024-09-21T07:48:50.050648+0000","last_undegraded":"2024-09-21T07:48:50.050648+0000","last_fullsized":"2024-09-21T07:48:50.050648+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:06.901043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059842100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":366,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.602888+0000","last_change":"2024-09-21T07:34:25.028904+0000","last_active":"2024-09-21T07:48:49.602888+0000","last_peered":"2024-09-21T07:48:49.602888+0000","last_clean":"2024-09-21T07:48:49.602888+0000","last_became_active":"2024-09-21T07:34:23.260942+0000","last_became_peered":"2024-09-21T07:34:23.260942+0000","last_unstale":"2024-09-21T07:48:49.602888+0000","last_undegraded":"2024-09-21T07:48:49.602888+0000","last_fullsized":"2024-09-21T07:48:49.602888+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:22.709328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060553899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509633,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66420,"kb_used_data":3288,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94305420,"statfs":{"total":96636764160,"available":96568750080,"internally_reserved":0,"allocated":3366912,"data_stored":2063963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509635,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51944,"kb_used_data":2708,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319896,"statfs":{"total":96636764160,"available":96583573504,"internally_reserved":0,"allocated":2772992,"data_stored":1474107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509633,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33888,"kb_used_data":2700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337952,"statfs":{"total":96636764160,"available":96602062848,"internally_reserved":0,"allocated":2764800,"data_stored":1471969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509634,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29732,"kb_used_data":2696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342108,"statfs":{"total":96636764160,"available":96606318592,"internally_reserved":0,"allocated":2760704,"data_stored":1472003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509635,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47768,"kb_used_data":2692,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324072,"statfs":{"total":96636764160,"available":96587849728,"internally_reserved":0,"allocated":2756608,"data_stored":1471913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509635,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52512,"kb_used_data":3276,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319328,"statfs":{"total":96636764160,"available":96582991872,"internally_reserved":0,"allocated":3354624,"data_stored":2062371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542338,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51936,"kb_used_data":2696,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319904,"statfs":{"total":96636764160,"available":96583581696,"internally_reserved":0,"allocated":2760704,"data_stored":1472495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542337,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74148,"kb_used_data":2704,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297692,"statfs":{"total":96636764160,"available":96560836608,"internally_reserved":0,"allocated":2768896,"data_stored":1472039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:48:57.182 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:48:57.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.416+0000 7fa039400640 1 -- 172.21.15.39:0/3328002994 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0340ccc60 msgr2=0x7fa0340cd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:57.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.416+0000 7fa039400640 1 --2- 172.21.15.39:0/3328002994 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0340ccc60 0x7fa0340cd070 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7fa02c005fd0 tx=0x7fa02c02aa00 comp rx=0 tx=0).stop 2024-09-21T07:48:57.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.416+0000 7fa039400640 1 -- 172.21.15.39:0/3328002994 shutdown_connections 2024-09-21T07:48:57.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.416+0000 7fa039400640 1 --2- 172.21.15.39:0/3328002994 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa0340cb8a0 0x7fa034156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:57.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.416+0000 7fa039400640 1 --2- 172.21.15.39:0/3328002994 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0340ccc60 0x7fa0340cd070 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:57.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.416+0000 7fa039400640 1 -- 172.21.15.39:0/3328002994 >> 172.21.15.39:0/3328002994 conn(0x7fa0340b2460 msgr2=0x7fa03406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:57.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.416+0000 7fa039400640 1 -- 172.21.15.39:0/3328002994 shutdown_connections 2024-09-21T07:48:57.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.416+0000 7fa039400640 1 -- 172.21.15.39:0/3328002994 wait complete. 2024-09-21T07:48:57.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.419+0000 7fa039400640 1 Processor -- start 2024-09-21T07:48:57.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.419+0000 7fa039400640 1 -- start start 2024-09-21T07:48:57.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.420+0000 7fa039400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0340b30a0 0x7fa034155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:57.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.420+0000 7fa039400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0340cb8a0 0x7fa0340c6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:57.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.420+0000 7fa039400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa0340ccc60 0x7fa0341e93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:57.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.420+0000 7fa039400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa0340af410 con 0x7fa0340b30a0 2024-09-21T07:48:57.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.420+0000 7fa039400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa0340af290 con 0x7fa0340cb8a0 2024-09-21T07:48:57.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.420+0000 7fa039400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa0340af590 con 0x7fa0340ccc60 2024-09-21T07:48:57.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.420+0000 7fa032800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0340b30a0 0x7fa034155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:57.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.421+0000 7fa032800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0340b30a0 0x7fa034155dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48872/0 (socket says 172.21.15.39:48872) 2024-09-21T07:48:57.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.421+0000 7fa032800640 1 -- 172.21.15.39:0/3185657385 learned_addr learned my addr 172.21.15.39:0/3185657385 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:57.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.421+0000 7fa031e00640 1 --2- 172.21.15.39:0/3185657385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0340cb8a0 0x7fa0340c6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:57.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.421+0000 7fa033200640 1 --2- 172.21.15.39:0/3185657385 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa0340ccc60 0x7fa0341e93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:57.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.421+0000 7fa032800640 1 -- 172.21.15.39:0/3185657385 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa0340ccc60 msgr2=0x7fa0341e93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:57.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.421+0000 7fa032800640 1 --2- 172.21.15.39:0/3185657385 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa0340ccc60 0x7fa0341e93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:57.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.421+0000 7fa032800640 1 -- 172.21.15.39:0/3185657385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0340cb8a0 msgr2=0x7fa0340c6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:57.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.421+0000 7fa032800640 1 --2- 172.21.15.39:0/3185657385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0340cb8a0 0x7fa0340c6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:57.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.421+0000 7fa032800640 1 -- 172.21.15.39:0/3185657385 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0340c5f90 con 0x7fa0340b30a0 2024-09-21T07:48:57.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.421+0000 7fa032800640 1 --2- 172.21.15.39:0/3185657385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0340b30a0 0x7fa034155dd0 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7fa02400c3a0 tx=0x7fa02400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:57.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.422+0000 7fa02aa00640 1 -- 172.21.15.39:0/3185657385 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa024017070 con 0x7fa0340b30a0 2024-09-21T07:48:57.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.422+0000 7fa02aa00640 1 -- 172.21.15.39:0/3185657385 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa0240053c0 con 0x7fa0340b30a0 2024-09-21T07:48:57.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.422+0000 7fa02aa00640 1 -- 172.21.15.39:0/3185657385 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa024003850 con 0x7fa0340b30a0 2024-09-21T07:48:57.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.422+0000 7fa039400640 1 -- 172.21.15.39:0/3185657385 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0340c6270 con 0x7fa0340b30a0 2024-09-21T07:48:57.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.422+0000 7fa039400640 1 -- 172.21.15.39:0/3185657385 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0340ce420 con 0x7fa0340b30a0 2024-09-21T07:48:57.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.423+0000 7fa039400640 1 -- 172.21.15.39:0/3185657385 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0340c6490 con 0x7fa0340b30a0 2024-09-21T07:48:57.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.424+0000 7fa02aa00640 1 -- 172.21.15.39:0/3185657385 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa02401a070 con 0x7fa0340b30a0 2024-09-21T07:48:57.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.425+0000 7fa02aa00640 1 --2- 172.21.15.39:0/3185657385 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa014078bc0 0x7fa01407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:57.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.425+0000 7fa02aa00640 1 -- 172.21.15.39:0/3185657385 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(221..221 src has 1..221) ==== 7075+0+0 (secure 0 0 0) 0x7fa02409a650 con 0x7fa0340b30a0 2024-09-21T07:48:57.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.425+0000 7fa031e00640 1 --2- 172.21.15.39:0/3185657385 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa014078bc0 0x7fa01407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:57.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.426+0000 7fa031e00640 1 --2- 172.21.15.39:0/3185657385 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa014078bc0 0x7fa01407b0d0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fa020005fd0 tx=0x7fa020005850 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:57.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.430+0000 7fa02aa00640 1 -- 172.21.15.39:0/3185657385 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa0340c6490 con 0x7fa0340b30a0 2024-09-21T07:48:57.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.623+0000 7fa039400640 1 -- 172.21.15.39:0/3185657385 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa0340cdf80 con 0x7fa014078bc0 2024-09-21T07:48:57.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.632+0000 7fa02aa00640 1 -- 172.21.15.39:0/3185657385 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+495210 (secure 0 0 0) 0x7fa02000fad0 con 0x7fa014078bc0 2024-09-21T07:48:57.639 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:48:57.642 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:48:57.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.637+0000 7fa039400640 1 -- 172.21.15.39:0/3185657385 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa014078bc0 msgr2=0x7fa01407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:57.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.637+0000 7fa039400640 1 --2- 172.21.15.39:0/3185657385 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa014078bc0 0x7fa01407b0d0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fa020005fd0 tx=0x7fa020005850 comp rx=0 tx=0).stop 2024-09-21T07:48:57.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.637+0000 7fa039400640 1 -- 172.21.15.39:0/3185657385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0340b30a0 msgr2=0x7fa034155dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:57.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.637+0000 7fa039400640 1 --2- 172.21.15.39:0/3185657385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0340b30a0 0x7fa034155dd0 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7fa02400c3a0 tx=0x7fa02400c860 comp rx=0 tx=0).stop 2024-09-21T07:48:57.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.637+0000 7fa039400640 1 -- 172.21.15.39:0/3185657385 shutdown_connections 2024-09-21T07:48:57.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.637+0000 7fa039400640 1 --2- 172.21.15.39:0/3185657385 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa014078bc0 0x7fa01407b0d0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:57.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.637+0000 7fa039400640 1 --2- 172.21.15.39:0/3185657385 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa0340ccc60 0x7fa0341e93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:57.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.637+0000 7fa039400640 1 --2- 172.21.15.39:0/3185657385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0340cb8a0 0x7fa0340c6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:57.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.637+0000 7fa039400640 1 --2- 172.21.15.39:0/3185657385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0340b30a0 0x7fa034155dd0 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:57.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.637+0000 7fa039400640 1 -- 172.21.15.39:0/3185657385 >> 172.21.15.39:0/3185657385 conn(0x7fa0340b2460 msgr2=0x7fa0340b11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:57.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.638+0000 7fa039400640 1 -- 172.21.15.39:0/3185657385 shutdown_connections 2024-09-21T07:48:57.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.638+0000 7fa039400640 1 -- 172.21.15.39:0/3185657385 wait complete. 2024-09-21T07:48:57.663 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":693,"stamp":"2024-09-21T07:48:55.854302+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":379,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":408348,"kb_used_data":22760,"kb_used_omap":12,"kb_used_meta":385395,"kb_avail":754566372,"statfs":{"total":773094113280,"available":772675964928,"internally_reserved":0,"allocated":23306240,"data_stored":12960860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":394645040},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2320,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.249000"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.551157+0000","last_change":"2024-09-21T07:48:47.620778+0000","last_active":"2024-09-21T07:48:49.551157+0000","last_peered":"2024-09-21T07:48:49.551157+0000","last_clean":"2024-09-21T07:48:49.551157+0000","last_became_active":"2024-09-21T07:48:47.620167+0000","last_became_peered":"2024-09-21T07:48:47.620167+0000","last_unstale":"2024-09-21T07:48:49.551157+0000","last_undegraded":"2024-09-21T07:48:49.551157+0000","last_fullsized":"2024-09-21T07:48:49.551157+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:17:04.946971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.610021+0000","last_change":"2024-09-21T07:48:47.608680+0000","last_active":"2024-09-21T07:48:48.610021+0000","last_peered":"2024-09-21T07:48:48.610021+0000","last_clean":"2024-09-21T07:48:48.610021+0000","last_became_active":"2024-09-21T07:48:47.606772+0000","last_became_peered":"2024-09-21T07:48:47.606772+0000","last_unstale":"2024-09-21T07:48:48.610021+0000","last_undegraded":"2024-09-21T07:48:48.610021+0000","last_fullsized":"2024-09-21T07:48:48.610021+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:13.951566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.602360+0000","last_change":"2024-09-21T07:48:47.622897+0000","last_active":"2024-09-21T07:48:49.602360+0000","last_peered":"2024-09-21T07:48:49.602360+0000","last_clean":"2024-09-21T07:48:49.602360+0000","last_became_active":"2024-09-21T07:48:47.622568+0000","last_became_peered":"2024-09-21T07:48:47.622568+0000","last_unstale":"2024-09-21T07:48:49.602360+0000","last_undegraded":"2024-09-21T07:48:49.602360+0000","last_fullsized":"2024-09-21T07:48:49.602360+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:20.786012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608060+0000","last_change":"2024-09-21T07:48:47.624850+0000","last_active":"2024-09-21T07:48:48.608060+0000","last_peered":"2024-09-21T07:48:48.608060+0000","last_clean":"2024-09-21T07:48:48.608060+0000","last_became_active":"2024-09-21T07:48:47.624508+0000","last_became_peered":"2024-09-21T07:48:47.624508+0000","last_unstale":"2024-09-21T07:48:48.608060+0000","last_undegraded":"2024-09-21T07:48:48.608060+0000","last_fullsized":"2024-09-21T07:48:48.608060+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:48.164441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609274+0000","last_change":"2024-09-21T07:48:47.602747+0000","last_active":"2024-09-21T07:48:48.609274+0000","last_peered":"2024-09-21T07:48:48.609274+0000","last_clean":"2024-09-21T07:48:48.609274+0000","last_became_active":"2024-09-21T07:48:47.602453+0000","last_became_peered":"2024-09-21T07:48:47.602453+0000","last_unstale":"2024-09-21T07:48:48.609274+0000","last_undegraded":"2024-09-21T07:48:48.609274+0000","last_fullsized":"2024-09-21T07:48:48.609274+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:20:13.838574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608103+0000","last_change":"2024-09-21T07:48:47.620304+0000","last_active":"2024-09-21T07:48:48.608103+0000","last_peered":"2024-09-21T07:48:48.608103+0000","last_clean":"2024-09-21T07:48:48.608103+0000","last_became_active":"2024-09-21T07:48:47.619974+0000","last_became_peered":"2024-09-21T07:48:47.619974+0000","last_unstale":"2024-09-21T07:48:48.608103+0000","last_undegraded":"2024-09-21T07:48:48.608103+0000","last_fullsized":"2024-09-21T07:48:48.608103+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:36:37.731085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609064+0000","last_change":"2024-09-21T07:48:47.602027+0000","last_active":"2024-09-21T07:48:48.609064+0000","last_peered":"2024-09-21T07:48:48.609064+0000","last_clean":"2024-09-21T07:48:48.609064+0000","last_became_active":"2024-09-21T07:48:47.601703+0000","last_became_peered":"2024-09-21T07:48:47.601703+0000","last_unstale":"2024-09-21T07:48:48.609064+0000","last_undegraded":"2024-09-21T07:48:48.609064+0000","last_fullsized":"2024-09-21T07:48:48.609064+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:37:16.197847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.550179+0000","last_change":"2024-09-21T07:48:47.627224+0000","last_active":"2024-09-21T07:48:49.550179+0000","last_peered":"2024-09-21T07:48:49.550179+0000","last_clean":"2024-09-21T07:48:49.550179+0000","last_became_active":"2024-09-21T07:48:47.626767+0000","last_became_peered":"2024-09-21T07:48:47.626767+0000","last_unstale":"2024-09-21T07:48:49.550179+0000","last_undegraded":"2024-09-21T07:48:49.550179+0000","last_fullsized":"2024-09-21T07:48:49.550179+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:42:45.011736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.050815+0000","last_change":"2024-09-21T07:48:47.617745+0000","last_active":"2024-09-21T07:48:50.050815+0000","last_peered":"2024-09-21T07:48:50.050815+0000","last_clean":"2024-09-21T07:48:50.050815+0000","last_became_active":"2024-09-21T07:48:47.614030+0000","last_became_peered":"2024-09-21T07:48:47.614030+0000","last_unstale":"2024-09-21T07:48:50.050815+0000","last_undegraded":"2024-09-21T07:48:50.050815+0000","last_fullsized":"2024-09-21T07:48:50.050815+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:33:56.311006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.552644+0000","last_change":"2024-09-21T07:48:47.621852+0000","last_active":"2024-09-21T07:48:49.552644+0000","last_peered":"2024-09-21T07:48:49.552644+0000","last_clean":"2024-09-21T07:48:49.552644+0000","last_became_active":"2024-09-21T07:48:47.617439+0000","last_became_peered":"2024-09-21T07:48:47.617439+0000","last_unstale":"2024-09-21T07:48:49.552644+0000","last_undegraded":"2024-09-21T07:48:49.552644+0000","last_fullsized":"2024-09-21T07:48:49.552644+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:44.083185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.602400+0000","last_change":"2024-09-21T07:48:47.632098+0000","last_active":"2024-09-21T07:48:49.602400+0000","last_peered":"2024-09-21T07:48:49.602400+0000","last_clean":"2024-09-21T07:48:49.602400+0000","last_became_active":"2024-09-21T07:48:47.631802+0000","last_became_peered":"2024-09-21T07:48:47.631802+0000","last_unstale":"2024-09-21T07:48:49.602400+0000","last_undegraded":"2024-09-21T07:48:49.602400+0000","last_fullsized":"2024-09-21T07:48:49.602400+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:36.943336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609791+0000","last_change":"2024-09-21T07:48:47.617229+0000","last_active":"2024-09-21T07:48:48.609791+0000","last_peered":"2024-09-21T07:48:48.609791+0000","last_clean":"2024-09-21T07:48:48.609791+0000","last_became_active":"2024-09-21T07:48:47.616863+0000","last_became_peered":"2024-09-21T07:48:47.616863+0000","last_unstale":"2024-09-21T07:48:48.609791+0000","last_undegraded":"2024-09-21T07:48:48.609791+0000","last_fullsized":"2024-09-21T07:48:48.609791+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:42.345168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608262+0000","last_change":"2024-09-21T07:48:47.618884+0000","last_active":"2024-09-21T07:48:48.608262+0000","last_peered":"2024-09-21T07:48:48.608262+0000","last_clean":"2024-09-21T07:48:48.608262+0000","last_became_active":"2024-09-21T07:48:47.618553+0000","last_became_peered":"2024-09-21T07:48:47.618553+0000","last_unstale":"2024-09-21T07:48:48.608262+0000","last_undegraded":"2024-09-21T07:48:48.608262+0000","last_fullsized":"2024-09-21T07:48:48.608262+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:51:06.489808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.551008+0000","last_change":"2024-09-21T07:48:47.623337+0000","last_active":"2024-09-21T07:48:49.551008+0000","last_peered":"2024-09-21T07:48:49.551008+0000","last_clean":"2024-09-21T07:48:49.551008+0000","last_became_active":"2024-09-21T07:48:47.620228+0000","last_became_peered":"2024-09-21T07:48:47.620228+0000","last_unstale":"2024-09-21T07:48:49.551008+0000","last_undegraded":"2024-09-21T07:48:49.551008+0000","last_fullsized":"2024-09-21T07:48:49.551008+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:43.841113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051089+0000","last_change":"2024-09-21T07:48:47.627205+0000","last_active":"2024-09-21T07:48:50.051089+0000","last_peered":"2024-09-21T07:48:50.051089+0000","last_clean":"2024-09-21T07:48:50.051089+0000","last_became_active":"2024-09-21T07:48:47.626871+0000","last_became_peered":"2024-09-21T07:48:47.626871+0000","last_unstale":"2024-09-21T07:48:50.051089+0000","last_undegraded":"2024-09-21T07:48:50.051089+0000","last_fullsized":"2024-09-21T07:48:50.051089+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:57.118108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.601783+0000","last_change":"2024-09-21T07:48:47.627220+0000","last_active":"2024-09-21T07:48:49.601783+0000","last_peered":"2024-09-21T07:48:49.601783+0000","last_clean":"2024-09-21T07:48:49.601783+0000","last_became_active":"2024-09-21T07:48:47.626875+0000","last_became_peered":"2024-09-21T07:48:47.626875+0000","last_unstale":"2024-09-21T07:48:49.601783+0000","last_undegraded":"2024-09-21T07:48:49.601783+0000","last_fullsized":"2024-09-21T07:48:49.601783+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:09:11.971175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051104+0000","last_change":"2024-09-21T07:48:47.625920+0000","last_active":"2024-09-21T07:48:50.051104+0000","last_peered":"2024-09-21T07:48:50.051104+0000","last_clean":"2024-09-21T07:48:50.051104+0000","last_became_active":"2024-09-21T07:48:47.625605+0000","last_became_peered":"2024-09-21T07:48:47.625605+0000","last_unstale":"2024-09-21T07:48:50.051104+0000","last_undegraded":"2024-09-21T07:48:50.051104+0000","last_fullsized":"2024-09-21T07:48:50.051104+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:38.962346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.050945+0000","last_change":"2024-09-21T07:48:47.628044+0000","last_active":"2024-09-21T07:48:50.050945+0000","last_peered":"2024-09-21T07:48:50.050945+0000","last_clean":"2024-09-21T07:48:50.050945+0000","last_became_active":"2024-09-21T07:48:47.626515+0000","last_became_peered":"2024-09-21T07:48:47.626515+0000","last_unstale":"2024-09-21T07:48:50.050945+0000","last_undegraded":"2024-09-21T07:48:50.050945+0000","last_fullsized":"2024-09-21T07:48:50.050945+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:03.431109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.552606+0000","last_change":"2024-09-21T07:48:47.623935+0000","last_active":"2024-09-21T07:48:49.552606+0000","last_peered":"2024-09-21T07:48:49.552606+0000","last_clean":"2024-09-21T07:48:49.552606+0000","last_became_active":"2024-09-21T07:48:47.623233+0000","last_became_peered":"2024-09-21T07:48:47.623233+0000","last_unstale":"2024-09-21T07:48:49.552606+0000","last_undegraded":"2024-09-21T07:48:49.552606+0000","last_fullsized":"2024-09-21T07:48:49.552606+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:23.647378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.550816+0000","last_change":"2024-09-21T07:48:47.617959+0000","last_active":"2024-09-21T07:48:49.550816+0000","last_peered":"2024-09-21T07:48:49.550816+0000","last_clean":"2024-09-21T07:48:49.550816+0000","last_became_active":"2024-09-21T07:48:47.617595+0000","last_became_peered":"2024-09-21T07:48:47.617595+0000","last_unstale":"2024-09-21T07:48:49.550816+0000","last_undegraded":"2024-09-21T07:48:49.550816+0000","last_fullsized":"2024-09-21T07:48:49.550816+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:08:09.285660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.551440+0000","last_change":"2024-09-21T07:48:47.621502+0000","last_active":"2024-09-21T07:48:49.551440+0000","last_peered":"2024-09-21T07:48:49.551440+0000","last_clean":"2024-09-21T07:48:49.551440+0000","last_became_active":"2024-09-21T07:48:47.616971+0000","last_became_peered":"2024-09-21T07:48:47.616971+0000","last_unstale":"2024-09-21T07:48:49.551440+0000","last_undegraded":"2024-09-21T07:48:49.551440+0000","last_fullsized":"2024-09-21T07:48:49.551440+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:15.121823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.551985+0000","last_change":"2024-09-21T07:48:47.622917+0000","last_active":"2024-09-21T07:48:49.551985+0000","last_peered":"2024-09-21T07:48:49.551985+0000","last_clean":"2024-09-21T07:48:49.551985+0000","last_became_active":"2024-09-21T07:48:47.619537+0000","last_became_peered":"2024-09-21T07:48:47.619537+0000","last_unstale":"2024-09-21T07:48:49.551985+0000","last_undegraded":"2024-09-21T07:48:49.551985+0000","last_fullsized":"2024-09-21T07:48:49.551985+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:45.507548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.610863+0000","last_change":"2024-09-21T07:48:47.598763+0000","last_active":"2024-09-21T07:48:48.610863+0000","last_peered":"2024-09-21T07:48:48.610863+0000","last_clean":"2024-09-21T07:48:48.610863+0000","last_became_active":"2024-09-21T07:48:47.598446+0000","last_became_peered":"2024-09-21T07:48:47.598446+0000","last_unstale":"2024-09-21T07:48:48.610863+0000","last_undegraded":"2024-09-21T07:48:48.610863+0000","last_fullsized":"2024-09-21T07:48:48.610863+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:48.912495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608756+0000","last_change":"2024-09-21T07:48:47.596907+0000","last_active":"2024-09-21T07:48:48.608756+0000","last_peered":"2024-09-21T07:48:48.608756+0000","last_clean":"2024-09-21T07:48:48.608756+0000","last_became_active":"2024-09-21T07:48:47.596568+0000","last_became_peered":"2024-09-21T07:48:47.596568+0000","last_unstale":"2024-09-21T07:48:48.608756+0000","last_undegraded":"2024-09-21T07:48:48.608756+0000","last_fullsized":"2024-09-21T07:48:48.608756+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:33:45.187090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051256+0000","last_change":"2024-09-21T07:48:47.620026+0000","last_active":"2024-09-21T07:48:50.051256+0000","last_peered":"2024-09-21T07:48:50.051256+0000","last_clean":"2024-09-21T07:48:50.051256+0000","last_became_active":"2024-09-21T07:48:47.619713+0000","last_became_peered":"2024-09-21T07:48:47.619713+0000","last_unstale":"2024-09-21T07:48:50.051256+0000","last_undegraded":"2024-09-21T07:48:50.051256+0000","last_fullsized":"2024-09-21T07:48:50.051256+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:43:53.702721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.551438+0000","last_change":"2024-09-21T07:48:47.629000+0000","last_active":"2024-09-21T07:48:49.551438+0000","last_peered":"2024-09-21T07:48:49.551438+0000","last_clean":"2024-09-21T07:48:49.551438+0000","last_became_active":"2024-09-21T07:48:47.628658+0000","last_became_peered":"2024-09-21T07:48:47.628658+0000","last_unstale":"2024-09-21T07:48:49.551438+0000","last_undegraded":"2024-09-21T07:48:49.551438+0000","last_fullsized":"2024-09-21T07:48:49.551438+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:12.875555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.602633+0000","last_change":"2024-09-21T07:48:47.625275+0000","last_active":"2024-09-21T07:48:49.602633+0000","last_peered":"2024-09-21T07:48:49.602633+0000","last_clean":"2024-09-21T07:48:49.602633+0000","last_became_active":"2024-09-21T07:48:47.624985+0000","last_became_peered":"2024-09-21T07:48:47.624985+0000","last_unstale":"2024-09-21T07:48:49.602633+0000","last_undegraded":"2024-09-21T07:48:49.602633+0000","last_fullsized":"2024-09-21T07:48:49.602633+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:04.926778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609989+0000","last_change":"2024-09-21T07:48:47.608774+0000","last_active":"2024-09-21T07:48:48.609989+0000","last_peered":"2024-09-21T07:48:48.609989+0000","last_clean":"2024-09-21T07:48:48.609989+0000","last_became_active":"2024-09-21T07:48:47.608457+0000","last_became_peered":"2024-09-21T07:48:47.608457+0000","last_unstale":"2024-09-21T07:48:48.609989+0000","last_undegraded":"2024-09-21T07:48:48.609989+0000","last_fullsized":"2024-09-21T07:48:48.609989+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:16.164291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.050811+0000","last_change":"2024-09-21T07:48:47.605730+0000","last_active":"2024-09-21T07:48:50.050811+0000","last_peered":"2024-09-21T07:48:50.050811+0000","last_clean":"2024-09-21T07:48:50.050811+0000","last_became_active":"2024-09-21T07:48:47.605433+0000","last_became_peered":"2024-09-21T07:48:47.605433+0000","last_unstale":"2024-09-21T07:48:50.050811+0000","last_undegraded":"2024-09-21T07:48:50.050811+0000","last_fullsized":"2024-09-21T07:48:50.050811+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:59.823864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051126+0000","last_change":"2024-09-21T07:48:47.604557+0000","last_active":"2024-09-21T07:48:50.051126+0000","last_peered":"2024-09-21T07:48:50.051126+0000","last_clean":"2024-09-21T07:48:50.051126+0000","last_became_active":"2024-09-21T07:48:47.604196+0000","last_became_peered":"2024-09-21T07:48:47.604196+0000","last_unstale":"2024-09-21T07:48:50.051126+0000","last_undegraded":"2024-09-21T07:48:50.051126+0000","last_fullsized":"2024-09-21T07:48:50.051126+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:56.323906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.607674+0000","last_change":"2024-09-21T07:48:47.621972+0000","last_active":"2024-09-21T07:48:48.607674+0000","last_peered":"2024-09-21T07:48:48.607674+0000","last_clean":"2024-09-21T07:48:48.607674+0000","last_became_active":"2024-09-21T07:48:47.621514+0000","last_became_peered":"2024-09-21T07:48:47.621514+0000","last_unstale":"2024-09-21T07:48:48.607674+0000","last_undegraded":"2024-09-21T07:48:48.607674+0000","last_fullsized":"2024-09-21T07:48:48.607674+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:06.914273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.607610+0000","last_change":"2024-09-21T07:48:47.608860+0000","last_active":"2024-09-21T07:48:48.607610+0000","last_peered":"2024-09-21T07:48:48.607610+0000","last_clean":"2024-09-21T07:48:48.607610+0000","last_became_active":"2024-09-21T07:48:47.608522+0000","last_became_peered":"2024-09-21T07:48:47.608522+0000","last_unstale":"2024-09-21T07:48:48.607610+0000","last_undegraded":"2024-09-21T07:48:48.607610+0000","last_fullsized":"2024-09-21T07:48:48.607610+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:02:42.398684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.551693+0000","last_change":"2024-09-21T07:48:47.629782+0000","last_active":"2024-09-21T07:48:49.551693+0000","last_peered":"2024-09-21T07:48:49.551693+0000","last_clean":"2024-09-21T07:48:49.551693+0000","last_became_active":"2024-09-21T07:48:47.628902+0000","last_became_peered":"2024-09-21T07:48:47.628902+0000","last_unstale":"2024-09-21T07:48:49.551693+0000","last_undegraded":"2024-09-21T07:48:49.551693+0000","last_fullsized":"2024-09-21T07:48:49.551693+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:56:31.556703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1a","version":"218'1","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.050809+0000","last_change":"2024-09-21T07:48:46.603075+0000","last_active":"2024-09-21T07:48:50.050809+0000","last_peered":"2024-09-21T07:48:50.050809+0000","last_clean":"2024-09-21T07:48:50.050809+0000","last_became_active":"2024-09-21T07:48:46.602882+0000","last_became_peered":"2024-09-21T07:48:46.602882+0000","last_unstale":"2024-09-21T07:48:50.050809+0000","last_undegraded":"2024-09-21T07:48:50.050809+0000","last_fullsized":"2024-09-21T07:48:50.050809+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:49.025973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":305,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608869+0000","last_change":"2024-09-21T07:35:27.879605+0000","last_active":"2024-09-21T07:48:48.608869+0000","last_peered":"2024-09-21T07:48:48.608869+0000","last_clean":"2024-09-21T07:48:48.608869+0000","last_became_active":"2024-09-21T07:35:27.879274+0000","last_became_peered":"2024-09-21T07:35:27.879274+0000","last_unstale":"2024-09-21T07:48:48.608869+0000","last_undegraded":"2024-09-21T07:48:48.608869+0000","last_fullsized":"2024-09-21T07:48:48.608869+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:24.241636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":316,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609470+0000","last_change":"2024-09-21T07:35:29.491336+0000","last_active":"2024-09-21T07:48:48.609470+0000","last_peered":"2024-09-21T07:48:48.609470+0000","last_clean":"2024-09-21T07:48:48.609470+0000","last_became_active":"2024-09-21T07:35:26.866484+0000","last_became_peered":"2024-09-21T07:35:26.866484+0000","last_unstale":"2024-09-21T07:48:48.609470+0000","last_undegraded":"2024-09-21T07:48:48.609470+0000","last_fullsized":"2024-09-21T07:48:48.609470+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:10.449665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076010100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":491,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.607805+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:48:48.607805+0000","last_peered":"2024-09-21T07:48:48.607805+0000","last_clean":"2024-09-21T07:48:48.607805+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:48:48.607805+0000","last_undegraded":"2024-09-21T07:48:48.607805+0000","last_fullsized":"2024-09-21T07:48:48.607805+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":379,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.551485+0000","last_change":"2024-09-21T07:34:24.539248+0000","last_active":"2024-09-21T07:48:49.551485+0000","last_peered":"2024-09-21T07:48:49.551485+0000","last_clean":"2024-09-21T07:48:49.551485+0000","last_became_active":"2024-09-21T07:34:23.273465+0000","last_became_peered":"2024-09-21T07:34:23.273465+0000","last_unstale":"2024-09-21T07:48:49.551485+0000","last_undegraded":"2024-09-21T07:48:49.551485+0000","last_fullsized":"2024-09-21T07:48:49.551485+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:55.517452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060627700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608458+0000","last_change":"2024-09-21T07:48:47.612024+0000","last_active":"2024-09-21T07:48:48.608458+0000","last_peered":"2024-09-21T07:48:48.608458+0000","last_clean":"2024-09-21T07:48:48.608458+0000","last_became_active":"2024-09-21T07:48:47.601212+0000","last_became_peered":"2024-09-21T07:48:47.601212+0000","last_unstale":"2024-09-21T07:48:48.608458+0000","last_undegraded":"2024-09-21T07:48:48.608458+0000","last_fullsized":"2024-09-21T07:48:48.608458+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:20.715873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"218'4","reported_seq":19,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051794+0000","last_change":"2024-09-21T07:48:46.613621+0000","last_active":"2024-09-21T07:48:50.051794+0000","last_peered":"2024-09-21T07:48:50.051794+0000","last_clean":"2024-09-21T07:48:50.051794+0000","last_became_active":"2024-09-21T07:48:46.613266+0000","last_became_peered":"2024-09-21T07:48:46.613266+0000","last_unstale":"2024-09-21T07:48:50.051794+0000","last_undegraded":"2024-09-21T07:48:50.051794+0000","last_fullsized":"2024-09-21T07:48:50.051794+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:40.639085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":309,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608414+0000","last_change":"2024-09-21T07:35:27.864492+0000","last_active":"2024-09-21T07:48:48.608414+0000","last_peered":"2024-09-21T07:48:48.608414+0000","last_clean":"2024-09-21T07:48:48.608414+0000","last_became_active":"2024-09-21T07:35:27.864176+0000","last_became_peered":"2024-09-21T07:35:27.864176+0000","last_unstale":"2024-09-21T07:48:48.608414+0000","last_undegraded":"2024-09-21T07:48:48.608414+0000","last_fullsized":"2024-09-21T07:48:48.608414+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:05.153467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":312,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608726+0000","last_change":"2024-09-21T07:35:29.493678+0000","last_active":"2024-09-21T07:48:48.608726+0000","last_peered":"2024-09-21T07:48:48.608726+0000","last_clean":"2024-09-21T07:48:48.608726+0000","last_became_active":"2024-09-21T07:35:26.870749+0000","last_became_peered":"2024-09-21T07:35:26.870749+0000","last_unstale":"2024-09-21T07:48:48.608726+0000","last_undegraded":"2024-09-21T07:48:48.608726+0000","last_fullsized":"2024-09-21T07:48:48.608726+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:55.367877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010159570000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":359,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051441+0000","last_change":"2024-09-21T07:34:24.597274+0000","last_active":"2024-09-21T07:48:50.051441+0000","last_peered":"2024-09-21T07:48:50.051441+0000","last_clean":"2024-09-21T07:48:50.051441+0000","last_became_active":"2024-09-21T07:34:23.279560+0000","last_became_peered":"2024-09-21T07:34:23.279560+0000","last_unstale":"2024-09-21T07:48:50.051441+0000","last_undegraded":"2024-09-21T07:48:50.051441+0000","last_fullsized":"2024-09-21T07:48:50.051441+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:15.845265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059699799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.550483+0000","last_change":"2024-09-21T07:48:47.605601+0000","last_active":"2024-09-21T07:48:49.550483+0000","last_peered":"2024-09-21T07:48:49.550483+0000","last_clean":"2024-09-21T07:48:49.550483+0000","last_became_active":"2024-09-21T07:48:47.603036+0000","last_became_peered":"2024-09-21T07:48:47.603036+0000","last_unstale":"2024-09-21T07:48:49.550483+0000","last_undegraded":"2024-09-21T07:48:49.550483+0000","last_fullsized":"2024-09-21T07:48:49.550483+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:02:57.337644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"218'1","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.550771+0000","last_change":"2024-09-21T07:48:46.610499+0000","last_active":"2024-09-21T07:48:49.550771+0000","last_peered":"2024-09-21T07:48:49.550771+0000","last_clean":"2024-09-21T07:48:49.550771+0000","last_became_active":"2024-09-21T07:48:46.610137+0000","last_became_peered":"2024-09-21T07:48:46.610137+0000","last_unstale":"2024-09-21T07:48:49.550771+0000","last_undegraded":"2024-09-21T07:48:49.550771+0000","last_fullsized":"2024-09-21T07:48:49.550771+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:19.623404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":310,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609240+0000","last_change":"2024-09-21T07:35:27.874387+0000","last_active":"2024-09-21T07:48:48.609240+0000","last_peered":"2024-09-21T07:48:48.609240+0000","last_clean":"2024-09-21T07:48:48.609240+0000","last_became_active":"2024-09-21T07:35:27.873277+0000","last_became_peered":"2024-09-21T07:35:27.873277+0000","last_unstale":"2024-09-21T07:48:48.609240+0000","last_undegraded":"2024-09-21T07:48:48.609240+0000","last_fullsized":"2024-09-21T07:48:48.609240+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:04.581463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":309,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051894+0000","last_change":"2024-09-21T07:35:29.387094+0000","last_active":"2024-09-21T07:48:50.051894+0000","last_peered":"2024-09-21T07:48:50.051894+0000","last_clean":"2024-09-21T07:48:50.051894+0000","last_became_active":"2024-09-21T07:35:26.874757+0000","last_became_peered":"2024-09-21T07:35:26.874757+0000","last_unstale":"2024-09-21T07:48:50.051894+0000","last_undegraded":"2024-09-21T07:48:50.051894+0000","last_fullsized":"2024-09-21T07:48:50.051894+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:33.542750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061065700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":365,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608865+0000","last_change":"2024-09-21T07:34:24.259906+0000","last_active":"2024-09-21T07:48:48.608865+0000","last_peered":"2024-09-21T07:48:48.608865+0000","last_clean":"2024-09-21T07:48:48.608865+0000","last_became_active":"2024-09-21T07:34:23.635282+0000","last_became_peered":"2024-09-21T07:34:23.635282+0000","last_unstale":"2024-09-21T07:48:48.608865+0000","last_undegraded":"2024-09-21T07:48:48.608865+0000","last_fullsized":"2024-09-21T07:48:48.608865+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:00.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001527664,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.551699+0000","last_change":"2024-09-21T07:48:47.622852+0000","last_active":"2024-09-21T07:48:49.551699+0000","last_peered":"2024-09-21T07:48:49.551699+0000","last_clean":"2024-09-21T07:48:49.551699+0000","last_became_active":"2024-09-21T07:48:47.619895+0000","last_became_peered":"2024-09-21T07:48:47.619895+0000","last_unstale":"2024-09-21T07:48:49.551699+0000","last_undegraded":"2024-09-21T07:48:49.551699+0000","last_fullsized":"2024-09-21T07:48:49.551699+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:53.333545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"218'2","reported_seq":17,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609737+0000","last_change":"2024-09-21T07:48:46.606441+0000","last_active":"2024-09-21T07:48:48.609737+0000","last_peered":"2024-09-21T07:48:48.609737+0000","last_clean":"2024-09-21T07:48:48.609737+0000","last_became_active":"2024-09-21T07:48:46.604712+0000","last_became_peered":"2024-09-21T07:48:46.604712+0000","last_unstale":"2024-09-21T07:48:48.609737+0000","last_undegraded":"2024-09-21T07:48:48.609737+0000","last_fullsized":"2024-09-21T07:48:48.609737+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:22.700415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":310,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609146+0000","last_change":"2024-09-21T07:35:27.873645+0000","last_active":"2024-09-21T07:48:48.609146+0000","last_peered":"2024-09-21T07:48:48.609146+0000","last_clean":"2024-09-21T07:48:48.609146+0000","last_became_active":"2024-09-21T07:35:27.873245+0000","last_became_peered":"2024-09-21T07:35:27.873245+0000","last_unstale":"2024-09-21T07:48:48.609146+0000","last_undegraded":"2024-09-21T07:48:48.609146+0000","last_fullsized":"2024-09-21T07:48:48.609146+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:55.090953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":316,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.610962+0000","last_change":"2024-09-21T07:35:28.861744+0000","last_active":"2024-09-21T07:48:48.610962+0000","last_peered":"2024-09-21T07:48:48.610962+0000","last_clean":"2024-09-21T07:48:48.610962+0000","last_became_active":"2024-09-21T07:35:26.876462+0000","last_became_peered":"2024-09-21T07:35:26.876462+0000","last_unstale":"2024-09-21T07:48:48.610962+0000","last_undegraded":"2024-09-21T07:48:48.610962+0000","last_fullsized":"2024-09-21T07:48:48.610962+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:48.632568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012020920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":365,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.050526+0000","last_change":"2024-09-21T07:34:25.085454+0000","last_active":"2024-09-21T07:48:50.050526+0000","last_peered":"2024-09-21T07:48:50.050526+0000","last_clean":"2024-09-21T07:48:50.050526+0000","last_became_active":"2024-09-21T07:34:23.279205+0000","last_became_peered":"2024-09-21T07:34:23.279205+0000","last_unstale":"2024-09-21T07:48:50.050526+0000","last_undegraded":"2024-09-21T07:48:50.050526+0000","last_fullsized":"2024-09-21T07:48:50.050526+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:54.231666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095320599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609766+0000","last_change":"2024-09-21T07:48:47.610088+0000","last_active":"2024-09-21T07:48:48.609766+0000","last_peered":"2024-09-21T07:48:48.609766+0000","last_clean":"2024-09-21T07:48:48.609766+0000","last_became_active":"2024-09-21T07:48:47.607791+0000","last_became_peered":"2024-09-21T07:48:47.607791+0000","last_unstale":"2024-09-21T07:48:48.609766+0000","last_undegraded":"2024-09-21T07:48:48.609766+0000","last_fullsized":"2024-09-21T07:48:48.609766+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:35.238803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"218'2","reported_seq":17,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.050817+0000","last_change":"2024-09-21T07:48:46.602844+0000","last_active":"2024-09-21T07:48:50.050817+0000","last_peered":"2024-09-21T07:48:50.050817+0000","last_clean":"2024-09-21T07:48:50.050817+0000","last_became_active":"2024-09-21T07:48:46.602544+0000","last_became_peered":"2024-09-21T07:48:46.602544+0000","last_unstale":"2024-09-21T07:48:50.050817+0000","last_undegraded":"2024-09-21T07:48:50.050817+0000","last_fullsized":"2024-09-21T07:48:50.050817+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:36:36.719598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":308,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609553+0000","last_change":"2024-09-21T07:35:27.861373+0000","last_active":"2024-09-21T07:48:48.609553+0000","last_peered":"2024-09-21T07:48:48.609553+0000","last_clean":"2024-09-21T07:48:48.609553+0000","last_became_active":"2024-09-21T07:35:27.861016+0000","last_became_peered":"2024-09-21T07:35:27.861016+0000","last_unstale":"2024-09-21T07:48:48.609553+0000","last_undegraded":"2024-09-21T07:48:48.609553+0000","last_fullsized":"2024-09-21T07:48:48.609553+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:56.043763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":315,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.602233+0000","last_change":"2024-09-21T07:35:29.053677+0000","last_active":"2024-09-21T07:48:49.602233+0000","last_peered":"2024-09-21T07:48:49.602233+0000","last_clean":"2024-09-21T07:48:49.602233+0000","last_became_active":"2024-09-21T07:35:26.875496+0000","last_became_peered":"2024-09-21T07:35:26.875496+0000","last_unstale":"2024-09-21T07:48:49.602233+0000","last_undegraded":"2024-09-21T07:48:49.602233+0000","last_fullsized":"2024-09-21T07:48:49.602233+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:04.850784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065859000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":367,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.602122+0000","last_change":"2024-09-21T07:34:25.028521+0000","last_active":"2024-09-21T07:48:49.602122+0000","last_peered":"2024-09-21T07:48:49.602122+0000","last_clean":"2024-09-21T07:48:49.602122+0000","last_became_active":"2024-09-21T07:34:23.635467+0000","last_became_peered":"2024-09-21T07:34:23.635467+0000","last_unstale":"2024-09-21T07:48:49.602122+0000","last_undegraded":"2024-09-21T07:48:49.602122+0000","last_fullsized":"2024-09-21T07:48:49.602122+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:33.729123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000660558,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051543+0000","last_change":"2024-09-21T07:48:47.616627+0000","last_active":"2024-09-21T07:48:50.051543+0000","last_peered":"2024-09-21T07:48:50.051543+0000","last_clean":"2024-09-21T07:48:50.051543+0000","last_became_active":"2024-09-21T07:48:47.616246+0000","last_became_peered":"2024-09-21T07:48:47.616246+0000","last_unstale":"2024-09-21T07:48:50.051543+0000","last_undegraded":"2024-09-21T07:48:50.051543+0000","last_fullsized":"2024-09-21T07:48:50.051543+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:07.746423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"218'1","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608686+0000","last_change":"2024-09-21T07:48:46.614962+0000","last_active":"2024-09-21T07:48:48.608686+0000","last_peered":"2024-09-21T07:48:48.608686+0000","last_clean":"2024-09-21T07:48:48.608686+0000","last_became_active":"2024-09-21T07:48:46.614616+0000","last_became_peered":"2024-09-21T07:48:46.614616+0000","last_unstale":"2024-09-21T07:48:48.608686+0000","last_undegraded":"2024-09-21T07:48:48.608686+0000","last_fullsized":"2024-09-21T07:48:48.608686+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:56:58.381332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":306,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051042+0000","last_change":"2024-09-21T07:35:27.871201+0000","last_active":"2024-09-21T07:48:50.051042+0000","last_peered":"2024-09-21T07:48:50.051042+0000","last_clean":"2024-09-21T07:48:50.051042+0000","last_became_active":"2024-09-21T07:35:27.870816+0000","last_became_peered":"2024-09-21T07:35:27.870816+0000","last_unstale":"2024-09-21T07:48:50.051042+0000","last_undegraded":"2024-09-21T07:48:50.051042+0000","last_fullsized":"2024-09-21T07:48:50.051042+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:04.787450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":315,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.610027+0000","last_change":"2024-09-21T07:35:29.491874+0000","last_active":"2024-09-21T07:48:48.610027+0000","last_peered":"2024-09-21T07:48:48.610027+0000","last_clean":"2024-09-21T07:48:48.610027+0000","last_became_active":"2024-09-21T07:35:26.866381+0000","last_became_peered":"2024-09-21T07:35:26.866381+0000","last_unstale":"2024-09-21T07:48:48.610027+0000","last_undegraded":"2024-09-21T07:48:48.610027+0000","last_fullsized":"2024-09-21T07:48:48.610027+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:36.404647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073235399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":370,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.610052+0000","last_change":"2024-09-21T07:34:24.260046+0000","last_active":"2024-09-21T07:48:48.610052+0000","last_peered":"2024-09-21T07:48:48.610052+0000","last_clean":"2024-09-21T07:48:48.610052+0000","last_became_active":"2024-09-21T07:34:23.273431+0000","last_became_peered":"2024-09-21T07:34:23.273431+0000","last_unstale":"2024-09-21T07:48:48.610052+0000","last_undegraded":"2024-09-21T07:48:48.610052+0000","last_fullsized":"2024-09-21T07:48:48.610052+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:18.876057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019803310000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051791+0000","last_change":"2024-09-21T07:48:47.605350+0000","last_active":"2024-09-21T07:48:50.051791+0000","last_peered":"2024-09-21T07:48:50.051791+0000","last_clean":"2024-09-21T07:48:50.051791+0000","last_became_active":"2024-09-21T07:48:47.605046+0000","last_became_peered":"2024-09-21T07:48:47.605046+0000","last_unstale":"2024-09-21T07:48:50.051791+0000","last_undegraded":"2024-09-21T07:48:50.051791+0000","last_fullsized":"2024-09-21T07:48:50.051791+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:07.906313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.607657+0000","last_change":"2024-09-21T07:48:46.607052+0000","last_active":"2024-09-21T07:48:48.607657+0000","last_peered":"2024-09-21T07:48:48.607657+0000","last_clean":"2024-09-21T07:48:48.607657+0000","last_became_active":"2024-09-21T07:48:46.606707+0000","last_became_peered":"2024-09-21T07:48:46.606707+0000","last_unstale":"2024-09-21T07:48:48.607657+0000","last_undegraded":"2024-09-21T07:48:48.607657+0000","last_fullsized":"2024-09-21T07:48:48.607657+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:20.963352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":306,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051347+0000","last_change":"2024-09-21T07:35:27.871736+0000","last_active":"2024-09-21T07:48:50.051347+0000","last_peered":"2024-09-21T07:48:50.051347+0000","last_clean":"2024-09-21T07:48:50.051347+0000","last_became_active":"2024-09-21T07:35:27.871232+0000","last_became_peered":"2024-09-21T07:35:27.871232+0000","last_unstale":"2024-09-21T07:48:50.051347+0000","last_undegraded":"2024-09-21T07:48:50.051347+0000","last_fullsized":"2024-09-21T07:48:50.051347+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:04.419301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":316,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608800+0000","last_change":"2024-09-21T07:35:29.491246+0000","last_active":"2024-09-21T07:48:48.608800+0000","last_peered":"2024-09-21T07:48:48.608800+0000","last_clean":"2024-09-21T07:48:48.608800+0000","last_became_active":"2024-09-21T07:35:26.866783+0000","last_became_peered":"2024-09-21T07:35:26.866783+0000","last_unstale":"2024-09-21T07:48:48.608800+0000","last_undegraded":"2024-09-21T07:48:48.608800+0000","last_fullsized":"2024-09-21T07:48:48.608800+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:28.881914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098474099999999992,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":367,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.602292+0000","last_change":"2024-09-21T07:34:25.028640+0000","last_active":"2024-09-21T07:48:49.602292+0000","last_peered":"2024-09-21T07:48:49.602292+0000","last_clean":"2024-09-21T07:48:49.602292+0000","last_became_active":"2024-09-21T07:34:23.260950+0000","last_became_peered":"2024-09-21T07:34:23.260950+0000","last_unstale":"2024-09-21T07:48:49.602292+0000","last_undegraded":"2024-09-21T07:48:49.602292+0000","last_fullsized":"2024-09-21T07:48:49.602292+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:23:42.121398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055135099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051414+0000","last_change":"2024-09-21T07:48:47.614858+0000","last_active":"2024-09-21T07:48:50.051414+0000","last_peered":"2024-09-21T07:48:50.051414+0000","last_clean":"2024-09-21T07:48:50.051414+0000","last_became_active":"2024-09-21T07:48:47.614484+0000","last_became_peered":"2024-09-21T07:48:47.614484+0000","last_unstale":"2024-09-21T07:48:50.051414+0000","last_undegraded":"2024-09-21T07:48:50.051414+0000","last_fullsized":"2024-09-21T07:48:50.051414+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:04:35.741683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.601774+0000","last_change":"2024-09-21T07:48:46.607151+0000","last_active":"2024-09-21T07:48:49.601774+0000","last_peered":"2024-09-21T07:48:49.601774+0000","last_clean":"2024-09-21T07:48:49.601774+0000","last_became_active":"2024-09-21T07:48:46.606839+0000","last_became_peered":"2024-09-21T07:48:46.606839+0000","last_unstale":"2024-09-21T07:48:49.601774+0000","last_undegraded":"2024-09-21T07:48:49.601774+0000","last_fullsized":"2024-09-21T07:48:49.601774+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:34.780363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":307,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.553085+0000","last_change":"2024-09-21T07:35:27.872027+0000","last_active":"2024-09-21T07:48:49.553085+0000","last_peered":"2024-09-21T07:48:49.553085+0000","last_clean":"2024-09-21T07:48:49.553085+0000","last_became_active":"2024-09-21T07:35:27.870426+0000","last_became_peered":"2024-09-21T07:35:27.870426+0000","last_unstale":"2024-09-21T07:48:49.553085+0000","last_undegraded":"2024-09-21T07:48:49.553085+0000","last_fullsized":"2024-09-21T07:48:49.553085+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":316,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608519+0000","last_change":"2024-09-21T07:35:29.491008+0000","last_active":"2024-09-21T07:48:48.608519+0000","last_peered":"2024-09-21T07:48:48.608519+0000","last_clean":"2024-09-21T07:48:48.608519+0000","last_became_active":"2024-09-21T07:35:26.866750+0000","last_became_peered":"2024-09-21T07:35:26.866750+0000","last_unstale":"2024-09-21T07:48:48.608519+0000","last_undegraded":"2024-09-21T07:48:48.608519+0000","last_fullsized":"2024-09-21T07:48:48.608519+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:25.490406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083529700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":366,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609266+0000","last_change":"2024-09-21T07:34:24.260665+0000","last_active":"2024-09-21T07:48:48.609266+0000","last_peered":"2024-09-21T07:48:48.609266+0000","last_clean":"2024-09-21T07:48:48.609266+0000","last_became_active":"2024-09-21T07:34:23.635909+0000","last_became_peered":"2024-09-21T07:34:23.635909+0000","last_unstale":"2024-09-21T07:48:48.609266+0000","last_undegraded":"2024-09-21T07:48:48.609266+0000","last_fullsized":"2024-09-21T07:48:48.609266+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:44.536885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051072+0000","last_change":"2024-09-21T07:48:47.615147+0000","last_active":"2024-09-21T07:48:50.051072+0000","last_peered":"2024-09-21T07:48:50.051072+0000","last_clean":"2024-09-21T07:48:50.051072+0000","last_became_active":"2024-09-21T07:48:47.614792+0000","last_became_peered":"2024-09-21T07:48:47.614792+0000","last_unstale":"2024-09-21T07:48:50.051072+0000","last_undegraded":"2024-09-21T07:48:50.051072+0000","last_fullsized":"2024-09-21T07:48:50.051072+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:23.043769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608722+0000","last_change":"2024-09-21T07:48:46.614465+0000","last_active":"2024-09-21T07:48:48.608722+0000","last_peered":"2024-09-21T07:48:48.608722+0000","last_clean":"2024-09-21T07:48:48.608722+0000","last_became_active":"2024-09-21T07:48:46.614155+0000","last_became_peered":"2024-09-21T07:48:46.614155+0000","last_unstale":"2024-09-21T07:48:48.608722+0000","last_undegraded":"2024-09-21T07:48:48.608722+0000","last_fullsized":"2024-09-21T07:48:48.608722+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:23:53.595692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":303,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051021+0000","last_change":"2024-09-21T07:35:27.874772+0000","last_active":"2024-09-21T07:48:50.051021+0000","last_peered":"2024-09-21T07:48:50.051021+0000","last_clean":"2024-09-21T07:48:50.051021+0000","last_became_active":"2024-09-21T07:35:27.874434+0000","last_became_peered":"2024-09-21T07:35:27.874434+0000","last_unstale":"2024-09-21T07:48:50.051021+0000","last_undegraded":"2024-09-21T07:48:50.051021+0000","last_fullsized":"2024-09-21T07:48:50.051021+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:31.998755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":327,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.551358+0000","last_change":"2024-09-21T07:35:28.859153+0000","last_active":"2024-09-21T07:48:49.551358+0000","last_peered":"2024-09-21T07:48:49.551358+0000","last_clean":"2024-09-21T07:48:49.551358+0000","last_became_active":"2024-09-21T07:35:26.874428+0000","last_became_peered":"2024-09-21T07:35:26.874428+0000","last_unstale":"2024-09-21T07:48:49.551358+0000","last_undegraded":"2024-09-21T07:48:49.551358+0000","last_fullsized":"2024-09-21T07:48:49.551358+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:04.487621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048557299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":379,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.551111+0000","last_change":"2024-09-21T07:34:24.538845+0000","last_active":"2024-09-21T07:48:49.551111+0000","last_peered":"2024-09-21T07:48:49.551111+0000","last_clean":"2024-09-21T07:48:49.551111+0000","last_became_active":"2024-09-21T07:34:23.636372+0000","last_became_peered":"2024-09-21T07:34:23.636372+0000","last_unstale":"2024-09-21T07:48:49.551111+0000","last_undegraded":"2024-09-21T07:48:49.551111+0000","last_fullsized":"2024-09-21T07:48:49.551111+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:29.393813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841451,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.601892+0000","last_change":"2024-09-21T07:48:47.617117+0000","last_active":"2024-09-21T07:48:49.601892+0000","last_peered":"2024-09-21T07:48:49.601892+0000","last_clean":"2024-09-21T07:48:49.601892+0000","last_became_active":"2024-09-21T07:48:47.616813+0000","last_became_peered":"2024-09-21T07:48:47.616813+0000","last_unstale":"2024-09-21T07:48:49.601892+0000","last_undegraded":"2024-09-21T07:48:49.601892+0000","last_fullsized":"2024-09-21T07:48:49.601892+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:25.693091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608080+0000","last_change":"2024-09-21T07:48:46.610433+0000","last_active":"2024-09-21T07:48:48.608080+0000","last_peered":"2024-09-21T07:48:48.608080+0000","last_clean":"2024-09-21T07:48:48.608080+0000","last_became_active":"2024-09-21T07:48:46.610123+0000","last_became_peered":"2024-09-21T07:48:46.610123+0000","last_unstale":"2024-09-21T07:48:48.608080+0000","last_undegraded":"2024-09-21T07:48:48.608080+0000","last_fullsized":"2024-09-21T07:48:48.608080+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:19:23.729259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":308,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.551193+0000","last_change":"2024-09-21T07:35:27.877801+0000","last_active":"2024-09-21T07:48:49.551193+0000","last_peered":"2024-09-21T07:48:49.551193+0000","last_clean":"2024-09-21T07:48:49.551193+0000","last_became_active":"2024-09-21T07:35:27.877464+0000","last_became_peered":"2024-09-21T07:35:27.877464+0000","last_unstale":"2024-09-21T07:48:49.551193+0000","last_undegraded":"2024-09-21T07:48:49.551193+0000","last_fullsized":"2024-09-21T07:48:49.551193+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:08.814929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":314,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051401+0000","last_change":"2024-09-21T07:35:28.863732+0000","last_active":"2024-09-21T07:48:50.051401+0000","last_peered":"2024-09-21T07:48:50.051401+0000","last_clean":"2024-09-21T07:48:50.051401+0000","last_became_active":"2024-09-21T07:35:26.877392+0000","last_became_peered":"2024-09-21T07:35:26.877392+0000","last_unstale":"2024-09-21T07:48:50.051401+0000","last_undegraded":"2024-09-21T07:48:50.051401+0000","last_fullsized":"2024-09-21T07:48:50.051401+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071425700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":370,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.610101+0000","last_change":"2024-09-21T07:34:24.259686+0000","last_active":"2024-09-21T07:48:48.610101+0000","last_peered":"2024-09-21T07:48:48.610101+0000","last_clean":"2024-09-21T07:48:48.610101+0000","last_became_active":"2024-09-21T07:34:23.274405+0000","last_became_peered":"2024-09-21T07:34:23.274405+0000","last_unstale":"2024-09-21T07:48:48.610101+0000","last_undegraded":"2024-09-21T07:48:48.610101+0000","last_fullsized":"2024-09-21T07:48:48.610101+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:26.304693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022009920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.050593+0000","last_change":"2024-09-21T07:48:47.616194+0000","last_active":"2024-09-21T07:48:50.050593+0000","last_peered":"2024-09-21T07:48:50.050593+0000","last_clean":"2024-09-21T07:48:50.050593+0000","last_became_active":"2024-09-21T07:48:47.615876+0000","last_became_peered":"2024-09-21T07:48:47.615876+0000","last_unstale":"2024-09-21T07:48:50.050593+0000","last_undegraded":"2024-09-21T07:48:50.050593+0000","last_fullsized":"2024-09-21T07:48:50.050593+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:56.874912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"218'3","reported_seq":18,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.050860+0000","last_change":"2024-09-21T07:48:46.605473+0000","last_active":"2024-09-21T07:48:50.050860+0000","last_peered":"2024-09-21T07:48:50.050860+0000","last_clean":"2024-09-21T07:48:50.050860+0000","last_became_active":"2024-09-21T07:48:46.604548+0000","last_became_peered":"2024-09-21T07:48:46.604548+0000","last_unstale":"2024-09-21T07:48:50.050860+0000","last_undegraded":"2024-09-21T07:48:50.050860+0000","last_fullsized":"2024-09-21T07:48:50.050860+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:05.902786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":303,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051307+0000","last_change":"2024-09-21T07:35:27.867336+0000","last_active":"2024-09-21T07:48:50.051307+0000","last_peered":"2024-09-21T07:48:50.051307+0000","last_clean":"2024-09-21T07:48:50.051307+0000","last_became_active":"2024-09-21T07:35:27.867015+0000","last_became_peered":"2024-09-21T07:35:27.867015+0000","last_unstale":"2024-09-21T07:48:50.051307+0000","last_undegraded":"2024-09-21T07:48:50.051307+0000","last_fullsized":"2024-09-21T07:48:50.051307+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:59.214139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":313,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608985+0000","last_change":"2024-09-21T07:35:29.493137+0000","last_active":"2024-09-21T07:48:48.608985+0000","last_peered":"2024-09-21T07:48:48.608985+0000","last_clean":"2024-09-21T07:48:48.608985+0000","last_became_active":"2024-09-21T07:35:26.872230+0000","last_became_peered":"2024-09-21T07:35:26.872230+0000","last_unstale":"2024-09-21T07:48:48.608985+0000","last_undegraded":"2024-09-21T07:48:48.608985+0000","last_fullsized":"2024-09-21T07:48:48.608985+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:24.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001197449,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":366,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.607680+0000","last_change":"2024-09-21T07:34:24.258754+0000","last_active":"2024-09-21T07:48:48.607680+0000","last_peered":"2024-09-21T07:48:48.607680+0000","last_clean":"2024-09-21T07:48:48.607680+0000","last_became_active":"2024-09-21T07:34:23.272657+0000","last_became_peered":"2024-09-21T07:34:23.272657+0000","last_unstale":"2024-09-21T07:48:48.607680+0000","last_undegraded":"2024-09-21T07:48:48.607680+0000","last_fullsized":"2024-09-21T07:48:48.607680+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:05.477099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067143200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609516+0000","last_change":"2024-09-21T07:48:47.629237+0000","last_active":"2024-09-21T07:48:48.609516+0000","last_peered":"2024-09-21T07:48:48.609516+0000","last_clean":"2024-09-21T07:48:48.609516+0000","last_became_active":"2024-09-21T07:48:47.628900+0000","last_became_peered":"2024-09-21T07:48:47.628900+0000","last_unstale":"2024-09-21T07:48:48.609516+0000","last_undegraded":"2024-09-21T07:48:48.609516+0000","last_fullsized":"2024-09-21T07:48:48.609516+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:39.553041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"218'6","reported_seq":21,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608023+0000","last_change":"2024-09-21T07:48:46.612660+0000","last_active":"2024-09-21T07:48:48.608023+0000","last_peered":"2024-09-21T07:48:48.608023+0000","last_clean":"2024-09-21T07:48:48.608023+0000","last_became_active":"2024-09-21T07:48:46.612362+0000","last_became_peered":"2024-09-21T07:48:46.612362+0000","last_unstale":"2024-09-21T07:48:48.608023+0000","last_undegraded":"2024-09-21T07:48:48.608023+0000","last_fullsized":"2024-09-21T07:48:48.608023+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:06.201994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":307,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.050951+0000","last_change":"2024-09-21T07:35:27.876021+0000","last_active":"2024-09-21T07:48:50.050951+0000","last_peered":"2024-09-21T07:48:50.050951+0000","last_clean":"2024-09-21T07:48:50.050951+0000","last_became_active":"2024-09-21T07:35:27.875724+0000","last_became_peered":"2024-09-21T07:35:27.875724+0000","last_unstale":"2024-09-21T07:48:50.050951+0000","last_undegraded":"2024-09-21T07:48:50.050951+0000","last_fullsized":"2024-09-21T07:48:50.050951+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:03.022812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":313,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608151+0000","last_change":"2024-09-21T07:35:29.492570+0000","last_active":"2024-09-21T07:48:48.608151+0000","last_peered":"2024-09-21T07:48:48.608151+0000","last_clean":"2024-09-21T07:48:48.608151+0000","last_became_active":"2024-09-21T07:35:26.874316+0000","last_became_peered":"2024-09-21T07:35:26.874316+0000","last_unstale":"2024-09-21T07:48:48.608151+0000","last_undegraded":"2024-09-21T07:48:48.608151+0000","last_fullsized":"2024-09-21T07:48:48.608151+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:16.512323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011990760000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":365,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.050595+0000","last_change":"2024-09-21T07:34:25.085232+0000","last_active":"2024-09-21T07:48:50.050595+0000","last_peered":"2024-09-21T07:48:50.050595+0000","last_clean":"2024-09-21T07:48:50.050595+0000","last_became_active":"2024-09-21T07:34:23.277807+0000","last_became_peered":"2024-09-21T07:34:23.277807+0000","last_unstale":"2024-09-21T07:48:50.050595+0000","last_undegraded":"2024-09-21T07:48:50.050595+0000","last_fullsized":"2024-09-21T07:48:50.050595+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:36.362919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086619099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609167+0000","last_change":"2024-09-21T07:48:47.607795+0000","last_active":"2024-09-21T07:48:48.609167+0000","last_peered":"2024-09-21T07:48:48.609167+0000","last_clean":"2024-09-21T07:48:48.609167+0000","last_became_active":"2024-09-21T07:48:47.607511+0000","last_became_peered":"2024-09-21T07:48:47.607511+0000","last_unstale":"2024-09-21T07:48:48.609167+0000","last_undegraded":"2024-09-21T07:48:48.609167+0000","last_fullsized":"2024-09-21T07:48:48.609167+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:27.353767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"218'2","reported_seq":17,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609872+0000","last_change":"2024-09-21T07:48:46.609344+0000","last_active":"2024-09-21T07:48:48.609872+0000","last_peered":"2024-09-21T07:48:48.609872+0000","last_clean":"2024-09-21T07:48:48.609872+0000","last_became_active":"2024-09-21T07:48:46.606541+0000","last_became_peered":"2024-09-21T07:48:46.606541+0000","last_unstale":"2024-09-21T07:48:48.609872+0000","last_undegraded":"2024-09-21T07:48:48.609872+0000","last_fullsized":"2024-09-21T07:48:48.609872+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:36.274172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":308,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.552023+0000","last_change":"2024-09-21T07:35:27.872798+0000","last_active":"2024-09-21T07:48:49.552023+0000","last_peered":"2024-09-21T07:48:49.552023+0000","last_clean":"2024-09-21T07:48:49.552023+0000","last_became_active":"2024-09-21T07:35:27.872483+0000","last_became_peered":"2024-09-21T07:35:27.872483+0000","last_unstale":"2024-09-21T07:48:49.552023+0000","last_undegraded":"2024-09-21T07:48:49.552023+0000","last_fullsized":"2024-09-21T07:48:49.552023+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:11.933469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":315,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.552061+0000","last_change":"2024-09-21T07:35:29.206723+0000","last_active":"2024-09-21T07:48:49.552061+0000","last_peered":"2024-09-21T07:48:49.552061+0000","last_clean":"2024-09-21T07:48:49.552061+0000","last_became_active":"2024-09-21T07:35:26.872377+0000","last_became_peered":"2024-09-21T07:35:26.872377+0000","last_unstale":"2024-09-21T07:48:49.552061+0000","last_undegraded":"2024-09-21T07:48:49.552061+0000","last_fullsized":"2024-09-21T07:48:49.552061+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:29.086862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083077200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":359,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051990+0000","last_change":"2024-09-21T07:34:24.597672+0000","last_active":"2024-09-21T07:48:50.051990+0000","last_peered":"2024-09-21T07:48:50.051990+0000","last_clean":"2024-09-21T07:48:50.051990+0000","last_became_active":"2024-09-21T07:34:23.267770+0000","last_became_peered":"2024-09-21T07:34:23.267770+0000","last_unstale":"2024-09-21T07:48:50.051990+0000","last_undegraded":"2024-09-21T07:48:50.051990+0000","last_fullsized":"2024-09-21T07:48:50.051990+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:51.629476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086378500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.602650+0000","last_change":"2024-09-21T07:48:47.618502+0000","last_active":"2024-09-21T07:48:49.602650+0000","last_peered":"2024-09-21T07:48:49.602650+0000","last_clean":"2024-09-21T07:48:49.602650+0000","last_became_active":"2024-09-21T07:48:47.618213+0000","last_became_peered":"2024-09-21T07:48:47.618213+0000","last_unstale":"2024-09-21T07:48:49.602650+0000","last_undegraded":"2024-09-21T07:48:49.602650+0000","last_fullsized":"2024-09-21T07:48:49.602650+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:02.405575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.601927+0000","last_change":"2024-09-21T07:48:46.602566+0000","last_active":"2024-09-21T07:48:49.601927+0000","last_peered":"2024-09-21T07:48:49.601927+0000","last_clean":"2024-09-21T07:48:49.601927+0000","last_became_active":"2024-09-21T07:48:46.602254+0000","last_became_peered":"2024-09-21T07:48:46.602254+0000","last_unstale":"2024-09-21T07:48:49.601927+0000","last_undegraded":"2024-09-21T07:48:49.601927+0000","last_fullsized":"2024-09-21T07:48:49.601927+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:09.449030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":308,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.552715+0000","last_change":"2024-09-21T07:35:27.870121+0000","last_active":"2024-09-21T07:48:49.552715+0000","last_peered":"2024-09-21T07:48:49.552715+0000","last_clean":"2024-09-21T07:48:49.552715+0000","last_became_active":"2024-09-21T07:35:27.869811+0000","last_became_peered":"2024-09-21T07:35:27.869811+0000","last_unstale":"2024-09-21T07:48:49.552715+0000","last_undegraded":"2024-09-21T07:48:49.552715+0000","last_fullsized":"2024-09-21T07:48:49.552715+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:31.778461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":317,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608956+0000","last_change":"2024-09-21T07:35:28.860699+0000","last_active":"2024-09-21T07:48:48.608956+0000","last_peered":"2024-09-21T07:48:48.608956+0000","last_clean":"2024-09-21T07:48:48.608956+0000","last_became_active":"2024-09-21T07:35:26.876779+0000","last_became_peered":"2024-09-21T07:35:26.876779+0000","last_unstale":"2024-09-21T07:48:48.608956+0000","last_undegraded":"2024-09-21T07:48:48.608956+0000","last_fullsized":"2024-09-21T07:48:48.608956+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:28.286403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000913188,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":369,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.611431+0000","last_change":"2024-09-21T07:34:24.260181+0000","last_active":"2024-09-21T07:48:48.611431+0000","last_peered":"2024-09-21T07:48:48.611431+0000","last_clean":"2024-09-21T07:48:48.611431+0000","last_became_active":"2024-09-21T07:34:23.275580+0000","last_became_peered":"2024-09-21T07:34:23.275580+0000","last_unstale":"2024-09-21T07:48:48.611431+0000","last_undegraded":"2024-09-21T07:48:48.611431+0000","last_fullsized":"2024-09-21T07:48:48.611431+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:49.857622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014706980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051494+0000","last_change":"2024-09-21T07:48:47.627712+0000","last_active":"2024-09-21T07:48:50.051494+0000","last_peered":"2024-09-21T07:48:50.051494+0000","last_clean":"2024-09-21T07:48:50.051494+0000","last_became_active":"2024-09-21T07:48:47.623777+0000","last_became_peered":"2024-09-21T07:48:47.623777+0000","last_unstale":"2024-09-21T07:48:50.051494+0000","last_undegraded":"2024-09-21T07:48:50.051494+0000","last_fullsized":"2024-09-21T07:48:50.051494+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:42:59.497138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.050486+0000","last_change":"2024-09-21T07:48:46.602604+0000","last_active":"2024-09-21T07:48:50.050486+0000","last_peered":"2024-09-21T07:48:50.050486+0000","last_clean":"2024-09-21T07:48:50.050486+0000","last_became_active":"2024-09-21T07:48:46.602252+0000","last_became_peered":"2024-09-21T07:48:46.602252+0000","last_unstale":"2024-09-21T07:48:50.050486+0000","last_undegraded":"2024-09-21T07:48:50.050486+0000","last_fullsized":"2024-09-21T07:48:50.050486+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:02:41.387196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":309,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.602448+0000","last_change":"2024-09-21T07:35:27.875393+0000","last_active":"2024-09-21T07:48:49.602448+0000","last_peered":"2024-09-21T07:48:49.602448+0000","last_clean":"2024-09-21T07:48:49.602448+0000","last_became_active":"2024-09-21T07:35:27.874671+0000","last_became_peered":"2024-09-21T07:35:27.874671+0000","last_unstale":"2024-09-21T07:48:49.602448+0000","last_undegraded":"2024-09-21T07:48:49.602448+0000","last_fullsized":"2024-09-21T07:48:49.602448+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":315,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.551396+0000","last_change":"2024-09-21T07:35:29.206296+0000","last_active":"2024-09-21T07:48:49.551396+0000","last_peered":"2024-09-21T07:48:49.551396+0000","last_clean":"2024-09-21T07:48:49.551396+0000","last_became_active":"2024-09-21T07:35:26.874575+0000","last_became_peered":"2024-09-21T07:35:26.874575+0000","last_unstale":"2024-09-21T07:48:49.551396+0000","last_undegraded":"2024-09-21T07:48:49.551396+0000","last_fullsized":"2024-09-21T07:48:49.551396+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:22.771154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061011599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":370,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609427+0000","last_change":"2024-09-21T07:34:24.259112+0000","last_active":"2024-09-21T07:48:48.609427+0000","last_peered":"2024-09-21T07:48:48.609427+0000","last_clean":"2024-09-21T07:48:48.609427+0000","last_became_active":"2024-09-21T07:34:23.635444+0000","last_became_peered":"2024-09-21T07:34:23.635444+0000","last_unstale":"2024-09-21T07:48:48.609427+0000","last_undegraded":"2024-09-21T07:48:48.609427+0000","last_fullsized":"2024-09-21T07:48:48.609427+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:46.255385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001952636,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608515+0000","last_change":"2024-09-21T07:48:47.625691+0000","last_active":"2024-09-21T07:48:48.608515+0000","last_peered":"2024-09-21T07:48:48.608515+0000","last_clean":"2024-09-21T07:48:48.608515+0000","last_became_active":"2024-09-21T07:48:47.625169+0000","last_became_peered":"2024-09-21T07:48:47.625169+0000","last_unstale":"2024-09-21T07:48:48.608515+0000","last_undegraded":"2024-09-21T07:48:48.608515+0000","last_fullsized":"2024-09-21T07:48:48.608515+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:50.037460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.10","version":"218'8","reported_seq":23,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608107+0000","last_change":"2024-09-21T07:48:46.607918+0000","last_active":"2024-09-21T07:48:48.608107+0000","last_peered":"2024-09-21T07:48:48.608107+0000","last_clean":"2024-09-21T07:48:48.608107+0000","last_became_active":"2024-09-21T07:48:46.607600+0000","last_became_peered":"2024-09-21T07:48:46.607600+0000","last_unstale":"2024-09-21T07:48:48.608107+0000","last_undegraded":"2024-09-21T07:48:48.608107+0000","last_fullsized":"2024-09-21T07:48:48.608107+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:55:20.508968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":309,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.602702+0000","last_change":"2024-09-21T07:35:27.878356+0000","last_active":"2024-09-21T07:48:49.602702+0000","last_peered":"2024-09-21T07:48:49.602702+0000","last_clean":"2024-09-21T07:48:49.602702+0000","last_became_active":"2024-09-21T07:35:27.877982+0000","last_became_peered":"2024-09-21T07:35:27.877982+0000","last_unstale":"2024-09-21T07:48:49.602702+0000","last_undegraded":"2024-09-21T07:48:49.602702+0000","last_fullsized":"2024-09-21T07:48:49.602702+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":316,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.601909+0000","last_change":"2024-09-21T07:35:29.053131+0000","last_active":"2024-09-21T07:48:49.601909+0000","last_peered":"2024-09-21T07:48:49.601909+0000","last_clean":"2024-09-21T07:48:49.601909+0000","last_became_active":"2024-09-21T07:35:26.870516+0000","last_became_peered":"2024-09-21T07:35:26.870516+0000","last_unstale":"2024-09-21T07:48:49.601909+0000","last_undegraded":"2024-09-21T07:48:49.601909+0000","last_fullsized":"2024-09-21T07:48:49.601909+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:10.924491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070705699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":379,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.552030+0000","last_change":"2024-09-21T07:34:24.539152+0000","last_active":"2024-09-21T07:48:49.552030+0000","last_peered":"2024-09-21T07:48:49.552030+0000","last_clean":"2024-09-21T07:48:49.552030+0000","last_became_active":"2024-09-21T07:34:23.259573+0000","last_became_peered":"2024-09-21T07:34:23.259573+0000","last_unstale":"2024-09-21T07:48:49.552030+0000","last_undegraded":"2024-09-21T07:48:49.552030+0000","last_fullsized":"2024-09-21T07:48:49.552030+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:26.986900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058108400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.550320+0000","last_change":"2024-09-21T07:48:47.627105+0000","last_active":"2024-09-21T07:48:49.550320+0000","last_peered":"2024-09-21T07:48:49.550320+0000","last_clean":"2024-09-21T07:48:49.550320+0000","last_became_active":"2024-09-21T07:48:47.626559+0000","last_became_peered":"2024-09-21T07:48:47.626559+0000","last_unstale":"2024-09-21T07:48:49.550320+0000","last_undegraded":"2024-09-21T07:48:49.550320+0000","last_fullsized":"2024-09-21T07:48:49.550320+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:28:36.631991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.f","version":"218'1","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.551148+0000","last_change":"2024-09-21T07:48:46.607214+0000","last_active":"2024-09-21T07:48:49.551148+0000","last_peered":"2024-09-21T07:48:49.551148+0000","last_clean":"2024-09-21T07:48:49.551148+0000","last_became_active":"2024-09-21T07:48:46.606843+0000","last_became_peered":"2024-09-21T07:48:46.606843+0000","last_unstale":"2024-09-21T07:48:49.551148+0000","last_undegraded":"2024-09-21T07:48:49.551148+0000","last_fullsized":"2024-09-21T07:48:49.551148+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:47:20.124577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":308,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.601849+0000","last_change":"2024-09-21T07:35:27.873726+0000","last_active":"2024-09-21T07:48:49.601849+0000","last_peered":"2024-09-21T07:48:49.601849+0000","last_clean":"2024-09-21T07:48:49.601849+0000","last_became_active":"2024-09-21T07:35:27.871421+0000","last_became_peered":"2024-09-21T07:35:27.871421+0000","last_unstale":"2024-09-21T07:48:49.601849+0000","last_undegraded":"2024-09-21T07:48:49.601849+0000","last_fullsized":"2024-09-21T07:48:49.601849+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:00.168915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":317,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608845+0000","last_change":"2024-09-21T07:35:28.861032+0000","last_active":"2024-09-21T07:48:48.608845+0000","last_peered":"2024-09-21T07:48:48.608845+0000","last_clean":"2024-09-21T07:48:48.608845+0000","last_became_active":"2024-09-21T07:35:26.873149+0000","last_became_peered":"2024-09-21T07:35:26.873149+0000","last_unstale":"2024-09-21T07:48:48.608845+0000","last_undegraded":"2024-09-21T07:48:48.608845+0000","last_fullsized":"2024-09-21T07:48:48.608845+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:59.435981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011988369999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":366,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608427+0000","last_change":"2024-09-21T07:34:24.259902+0000","last_active":"2024-09-21T07:48:48.608427+0000","last_peered":"2024-09-21T07:48:48.608427+0000","last_clean":"2024-09-21T07:48:48.608427+0000","last_became_active":"2024-09-21T07:34:23.271902+0000","last_became_peered":"2024-09-21T07:34:23.271902+0000","last_unstale":"2024-09-21T07:48:48.608427+0000","last_undegraded":"2024-09-21T07:48:48.608427+0000","last_fullsized":"2024-09-21T07:48:48.608427+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:59.439525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001686358,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.550216+0000","last_change":"2024-09-21T07:48:47.618224+0000","last_active":"2024-09-21T07:48:49.550216+0000","last_peered":"2024-09-21T07:48:49.550216+0000","last_clean":"2024-09-21T07:48:49.550216+0000","last_became_active":"2024-09-21T07:48:47.617876+0000","last_became_peered":"2024-09-21T07:48:47.617876+0000","last_unstale":"2024-09-21T07:48:49.550216+0000","last_undegraded":"2024-09-21T07:48:49.550216+0000","last_fullsized":"2024-09-21T07:48:49.550216+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:46.265092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"218'2","reported_seq":17,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.550920+0000","last_change":"2024-09-21T07:48:46.606382+0000","last_active":"2024-09-21T07:48:49.550920+0000","last_peered":"2024-09-21T07:48:49.550920+0000","last_clean":"2024-09-21T07:48:49.550920+0000","last_became_active":"2024-09-21T07:48:46.606002+0000","last_became_peered":"2024-09-21T07:48:46.606002+0000","last_unstale":"2024-09-21T07:48:49.550920+0000","last_undegraded":"2024-09-21T07:48:49.550920+0000","last_fullsized":"2024-09-21T07:48:49.550920+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:48:18.461842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":306,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.050380+0000","last_change":"2024-09-21T07:35:27.872637+0000","last_active":"2024-09-21T07:48:50.050380+0000","last_peered":"2024-09-21T07:48:50.050380+0000","last_clean":"2024-09-21T07:48:50.050380+0000","last_became_active":"2024-09-21T07:35:27.872149+0000","last_became_peered":"2024-09-21T07:35:27.872149+0000","last_unstale":"2024-09-21T07:48:50.050380+0000","last_undegraded":"2024-09-21T07:48:50.050380+0000","last_fullsized":"2024-09-21T07:48:50.050380+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:27.697872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":313,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609094+0000","last_change":"2024-09-21T07:35:29.493362+0000","last_active":"2024-09-21T07:48:48.609094+0000","last_peered":"2024-09-21T07:48:48.609094+0000","last_clean":"2024-09-21T07:48:48.609094+0000","last_became_active":"2024-09-21T07:35:26.873018+0000","last_became_peered":"2024-09-21T07:35:26.873018+0000","last_unstale":"2024-09-21T07:48:48.609094+0000","last_undegraded":"2024-09-21T07:48:48.609094+0000","last_fullsized":"2024-09-21T07:48:48.609094+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:02.460557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012496510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":370,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609996+0000","last_change":"2024-09-21T07:34:24.259662+0000","last_active":"2024-09-21T07:48:48.609996+0000","last_peered":"2024-09-21T07:48:48.609996+0000","last_clean":"2024-09-21T07:48:48.609996+0000","last_became_active":"2024-09-21T07:34:23.278812+0000","last_became_peered":"2024-09-21T07:34:23.278812+0000","last_unstale":"2024-09-21T07:48:48.609996+0000","last_undegraded":"2024-09-21T07:48:48.609996+0000","last_fullsized":"2024-09-21T07:48:48.609996+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:49.829974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356463,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.550374+0000","last_change":"2024-09-21T07:48:47.598977+0000","last_active":"2024-09-21T07:48:49.550374+0000","last_peered":"2024-09-21T07:48:49.550374+0000","last_clean":"2024-09-21T07:48:49.550374+0000","last_became_active":"2024-09-21T07:48:47.598620+0000","last_became_peered":"2024-09-21T07:48:47.598620+0000","last_unstale":"2024-09-21T07:48:49.550374+0000","last_undegraded":"2024-09-21T07:48:49.550374+0000","last_fullsized":"2024-09-21T07:48:49.550374+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:07.336524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"218'1","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609860+0000","last_change":"2024-09-21T07:48:46.605998+0000","last_active":"2024-09-21T07:48:48.609860+0000","last_peered":"2024-09-21T07:48:48.609860+0000","last_clean":"2024-09-21T07:48:48.609860+0000","last_became_active":"2024-09-21T07:48:46.605714+0000","last_became_peered":"2024-09-21T07:48:46.605714+0000","last_unstale":"2024-09-21T07:48:48.609860+0000","last_undegraded":"2024-09-21T07:48:48.609860+0000","last_fullsized":"2024-09-21T07:48:48.609860+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:51.672371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":302,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051131+0000","last_change":"2024-09-21T07:35:27.868448+0000","last_active":"2024-09-21T07:48:50.051131+0000","last_peered":"2024-09-21T07:48:50.051131+0000","last_clean":"2024-09-21T07:48:50.051131+0000","last_became_active":"2024-09-21T07:35:27.868105+0000","last_became_peered":"2024-09-21T07:35:27.868105+0000","last_unstale":"2024-09-21T07:48:50.051131+0000","last_undegraded":"2024-09-21T07:48:50.051131+0000","last_fullsized":"2024-09-21T07:48:50.051131+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:41.468702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":312,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608736+0000","last_change":"2024-09-21T07:35:29.492974+0000","last_active":"2024-09-21T07:48:48.608736+0000","last_peered":"2024-09-21T07:48:48.608736+0000","last_clean":"2024-09-21T07:48:48.608736+0000","last_became_active":"2024-09-21T07:35:26.872165+0000","last_became_peered":"2024-09-21T07:35:26.872165+0000","last_unstale":"2024-09-21T07:48:48.608736+0000","last_undegraded":"2024-09-21T07:48:48.608736+0000","last_fullsized":"2024-09-21T07:48:48.608736+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:59.912520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013795179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":365,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.050414+0000","last_change":"2024-09-21T07:34:25.084861+0000","last_active":"2024-09-21T07:48:50.050414+0000","last_peered":"2024-09-21T07:48:50.050414+0000","last_clean":"2024-09-21T07:48:50.050414+0000","last_became_active":"2024-09-21T07:34:23.277708+0000","last_became_peered":"2024-09-21T07:34:23.277708+0000","last_unstale":"2024-09-21T07:48:50.050414+0000","last_undegraded":"2024-09-21T07:48:50.050414+0000","last_fullsized":"2024-09-21T07:48:50.050414+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:40.816488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000597566,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.551729+0000","last_change":"2024-09-21T07:48:47.621555+0000","last_active":"2024-09-21T07:48:49.551729+0000","last_peered":"2024-09-21T07:48:49.551729+0000","last_clean":"2024-09-21T07:48:49.551729+0000","last_became_active":"2024-09-21T07:48:47.620805+0000","last_became_peered":"2024-09-21T07:48:47.620805+0000","last_unstale":"2024-09-21T07:48:49.551729+0000","last_undegraded":"2024-09-21T07:48:49.551729+0000","last_fullsized":"2024-09-21T07:48:49.551729+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:34.601718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.610125+0000","last_change":"2024-09-21T07:48:46.611660+0000","last_active":"2024-09-21T07:48:48.610125+0000","last_peered":"2024-09-21T07:48:48.610125+0000","last_clean":"2024-09-21T07:48:48.610125+0000","last_became_active":"2024-09-21T07:48:46.610843+0000","last_became_peered":"2024-09-21T07:48:46.610843+0000","last_unstale":"2024-09-21T07:48:48.610125+0000","last_undegraded":"2024-09-21T07:48:48.610125+0000","last_fullsized":"2024-09-21T07:48:48.610125+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:43:38.733833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":302,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051512+0000","last_change":"2024-09-21T07:35:27.874640+0000","last_active":"2024-09-21T07:48:50.051512+0000","last_peered":"2024-09-21T07:48:50.051512+0000","last_clean":"2024-09-21T07:48:50.051512+0000","last_became_active":"2024-09-21T07:35:27.874265+0000","last_became_peered":"2024-09-21T07:35:27.874265+0000","last_unstale":"2024-09-21T07:48:50.051512+0000","last_undegraded":"2024-09-21T07:48:50.051512+0000","last_fullsized":"2024-09-21T07:48:50.051512+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:45.754654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":314,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.552233+0000","last_change":"2024-09-21T07:35:29.207392+0000","last_active":"2024-09-21T07:48:49.552233+0000","last_peered":"2024-09-21T07:48:49.552233+0000","last_clean":"2024-09-21T07:48:49.552233+0000","last_became_active":"2024-09-21T07:35:26.870917+0000","last_became_peered":"2024-09-21T07:35:26.870917+0000","last_unstale":"2024-09-21T07:48:49.552233+0000","last_undegraded":"2024-09-21T07:48:49.552233+0000","last_fullsized":"2024-09-21T07:48:49.552233+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:59.159937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059198499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":370,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608972+0000","last_change":"2024-09-21T07:34:24.258709+0000","last_active":"2024-09-21T07:48:48.608972+0000","last_peered":"2024-09-21T07:48:48.608972+0000","last_clean":"2024-09-21T07:48:48.608972+0000","last_became_active":"2024-09-21T07:34:23.279042+0000","last_became_peered":"2024-09-21T07:34:23.279042+0000","last_unstale":"2024-09-21T07:48:48.608972+0000","last_undegraded":"2024-09-21T07:48:48.608972+0000","last_fullsized":"2024-09-21T07:48:48.608972+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:01.762117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019282100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608922+0000","last_change":"2024-09-21T07:48:47.607091+0000","last_active":"2024-09-21T07:48:48.608922+0000","last_peered":"2024-09-21T07:48:48.608922+0000","last_clean":"2024-09-21T07:48:48.608922+0000","last_became_active":"2024-09-21T07:48:47.606739+0000","last_became_peered":"2024-09-21T07:48:47.606739+0000","last_unstale":"2024-09-21T07:48:48.608922+0000","last_undegraded":"2024-09-21T07:48:48.608922+0000","last_fullsized":"2024-09-21T07:48:48.608922+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:42:23.765051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"218'2","reported_seq":17,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609559+0000","last_change":"2024-09-21T07:48:46.612384+0000","last_active":"2024-09-21T07:48:48.609559+0000","last_peered":"2024-09-21T07:48:48.609559+0000","last_clean":"2024-09-21T07:48:48.609559+0000","last_became_active":"2024-09-21T07:48:46.610836+0000","last_became_peered":"2024-09-21T07:48:46.610836+0000","last_unstale":"2024-09-21T07:48:48.609559+0000","last_undegraded":"2024-09-21T07:48:48.609559+0000","last_fullsized":"2024-09-21T07:48:48.609559+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:04.310354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":306,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.050941+0000","last_change":"2024-09-21T07:35:27.870260+0000","last_active":"2024-09-21T07:48:50.050941+0000","last_peered":"2024-09-21T07:48:50.050941+0000","last_clean":"2024-09-21T07:48:50.050941+0000","last_became_active":"2024-09-21T07:35:27.869448+0000","last_became_peered":"2024-09-21T07:35:27.869448+0000","last_unstale":"2024-09-21T07:48:50.050941+0000","last_undegraded":"2024-09-21T07:48:50.050941+0000","last_fullsized":"2024-09-21T07:48:50.050941+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:16.946790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":309,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051488+0000","last_change":"2024-09-21T07:35:29.386367+0000","last_active":"2024-09-21T07:48:50.051488+0000","last_peered":"2024-09-21T07:48:50.051488+0000","last_clean":"2024-09-21T07:48:50.051488+0000","last_became_active":"2024-09-21T07:35:26.879238+0000","last_became_peered":"2024-09-21T07:35:26.879238+0000","last_unstale":"2024-09-21T07:48:50.051488+0000","last_undegraded":"2024-09-21T07:48:50.051488+0000","last_fullsized":"2024-09-21T07:48:50.051488+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:16.528260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092223100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":365,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051594+0000","last_change":"2024-09-21T07:34:25.085635+0000","last_active":"2024-09-21T07:48:50.051594+0000","last_peered":"2024-09-21T07:48:50.051594+0000","last_clean":"2024-09-21T07:48:50.051594+0000","last_became_active":"2024-09-21T07:34:23.273566+0000","last_became_peered":"2024-09-21T07:34:23.273566+0000","last_unstale":"2024-09-21T07:48:50.051594+0000","last_undegraded":"2024-09-21T07:48:50.051594+0000","last_fullsized":"2024-09-21T07:48:50.051594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:14.142272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079897300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608418+0000","last_change":"2024-09-21T07:48:47.625742+0000","last_active":"2024-09-21T07:48:48.608418+0000","last_peered":"2024-09-21T07:48:48.608418+0000","last_clean":"2024-09-21T07:48:48.608418+0000","last_became_active":"2024-09-21T07:48:47.625290+0000","last_became_peered":"2024-09-21T07:48:47.625290+0000","last_unstale":"2024-09-21T07:48:48.608418+0000","last_undegraded":"2024-09-21T07:48:48.608418+0000","last_fullsized":"2024-09-21T07:48:48.608418+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:15.258575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609104+0000","last_change":"2024-09-21T07:48:46.610951+0000","last_active":"2024-09-21T07:48:48.609104+0000","last_peered":"2024-09-21T07:48:48.609104+0000","last_clean":"2024-09-21T07:48:48.609104+0000","last_became_active":"2024-09-21T07:48:46.610658+0000","last_became_peered":"2024-09-21T07:48:46.610658+0000","last_unstale":"2024-09-21T07:48:48.609104+0000","last_undegraded":"2024-09-21T07:48:48.609104+0000","last_fullsized":"2024-09-21T07:48:48.609104+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:15:20.903496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":306,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051065+0000","last_change":"2024-09-21T07:35:27.878728+0000","last_active":"2024-09-21T07:48:50.051065+0000","last_peered":"2024-09-21T07:48:50.051065+0000","last_clean":"2024-09-21T07:48:50.051065+0000","last_became_active":"2024-09-21T07:35:27.878335+0000","last_became_peered":"2024-09-21T07:35:27.878335+0000","last_unstale":"2024-09-21T07:48:50.051065+0000","last_undegraded":"2024-09-21T07:48:50.051065+0000","last_fullsized":"2024-09-21T07:48:50.051065+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:25:37.731771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":326,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.550843+0000","last_change":"2024-09-21T07:35:28.858418+0000","last_active":"2024-09-21T07:48:49.550843+0000","last_peered":"2024-09-21T07:48:49.550843+0000","last_clean":"2024-09-21T07:48:49.550843+0000","last_became_active":"2024-09-21T07:35:26.876944+0000","last_became_peered":"2024-09-21T07:35:26.876944+0000","last_unstale":"2024-09-21T07:48:49.550843+0000","last_undegraded":"2024-09-21T07:48:49.550843+0000","last_fullsized":"2024-09-21T07:48:49.550843+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:04.514304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090862799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":358,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051805+0000","last_change":"2024-09-21T07:34:24.597926+0000","last_active":"2024-09-21T07:48:50.051805+0000","last_peered":"2024-09-21T07:48:50.051805+0000","last_clean":"2024-09-21T07:48:50.051805+0000","last_became_active":"2024-09-21T07:34:23.268926+0000","last_became_peered":"2024-09-21T07:34:23.268926+0000","last_unstale":"2024-09-21T07:48:50.051805+0000","last_undegraded":"2024-09-21T07:48:50.051805+0000","last_fullsized":"2024-09-21T07:48:50.051805+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:46.190752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101851,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051213+0000","last_change":"2024-09-21T07:48:47.624905+0000","last_active":"2024-09-21T07:48:50.051213+0000","last_peered":"2024-09-21T07:48:50.051213+0000","last_clean":"2024-09-21T07:48:50.051213+0000","last_became_active":"2024-09-21T07:48:47.621782+0000","last_became_peered":"2024-09-21T07:48:47.621782+0000","last_unstale":"2024-09-21T07:48:50.051213+0000","last_undegraded":"2024-09-21T07:48:50.051213+0000","last_fullsized":"2024-09-21T07:48:50.051213+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:18.457620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051148+0000","last_change":"2024-09-21T07:48:46.613457+0000","last_active":"2024-09-21T07:48:50.051148+0000","last_peered":"2024-09-21T07:48:50.051148+0000","last_clean":"2024-09-21T07:48:50.051148+0000","last_became_active":"2024-09-21T07:48:46.612037+0000","last_became_peered":"2024-09-21T07:48:46.612037+0000","last_unstale":"2024-09-21T07:48:50.051148+0000","last_undegraded":"2024-09-21T07:48:50.051148+0000","last_fullsized":"2024-09-21T07:48:50.051148+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:06.200769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":303,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051801+0000","last_change":"2024-09-21T07:35:27.877426+0000","last_active":"2024-09-21T07:48:50.051801+0000","last_peered":"2024-09-21T07:48:50.051801+0000","last_clean":"2024-09-21T07:48:50.051801+0000","last_became_active":"2024-09-21T07:35:27.877050+0000","last_became_peered":"2024-09-21T07:35:27.877050+0000","last_unstale":"2024-09-21T07:48:50.051801+0000","last_undegraded":"2024-09-21T07:48:50.051801+0000","last_fullsized":"2024-09-21T07:48:50.051801+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:36.268334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":313,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.050387+0000","last_change":"2024-09-21T07:35:28.862273+0000","last_active":"2024-09-21T07:48:50.050387+0000","last_peered":"2024-09-21T07:48:50.050387+0000","last_clean":"2024-09-21T07:48:50.050387+0000","last_became_active":"2024-09-21T07:35:26.874011+0000","last_became_peered":"2024-09-21T07:35:26.874011+0000","last_unstale":"2024-09-21T07:48:50.050387+0000","last_undegraded":"2024-09-21T07:48:50.050387+0000","last_fullsized":"2024-09-21T07:48:50.050387+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:11.986950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071644799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":369,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.610345+0000","last_change":"2024-09-21T07:34:24.259794+0000","last_active":"2024-09-21T07:48:48.610345+0000","last_peered":"2024-09-21T07:48:48.610345+0000","last_clean":"2024-09-21T07:48:48.610345+0000","last_became_active":"2024-09-21T07:34:23.265679+0000","last_became_peered":"2024-09-21T07:34:23.265679+0000","last_unstale":"2024-09-21T07:48:48.610345+0000","last_undegraded":"2024-09-21T07:48:48.610345+0000","last_fullsized":"2024-09-21T07:48:48.610345+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:02.575796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001298518,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.552939+0000","last_change":"2024-09-21T07:48:47.618294+0000","last_active":"2024-09-21T07:48:49.552939+0000","last_peered":"2024-09-21T07:48:49.552939+0000","last_clean":"2024-09-21T07:48:49.552939+0000","last_became_active":"2024-09-21T07:48:47.617975+0000","last_became_peered":"2024-09-21T07:48:47.617975+0000","last_unstale":"2024-09-21T07:48:49.552939+0000","last_undegraded":"2024-09-21T07:48:49.552939+0000","last_fullsized":"2024-09-21T07:48:49.552939+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:28:21.189669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.550020+0000","last_change":"2024-09-21T07:48:46.608424+0000","last_active":"2024-09-21T07:48:49.550020+0000","last_peered":"2024-09-21T07:48:49.550020+0000","last_clean":"2024-09-21T07:48:49.550020+0000","last_became_active":"2024-09-21T07:48:46.607778+0000","last_became_peered":"2024-09-21T07:48:46.607778+0000","last_unstale":"2024-09-21T07:48:49.550020+0000","last_undegraded":"2024-09-21T07:48:49.550020+0000","last_fullsized":"2024-09-21T07:48:49.550020+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:14.577558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":308,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.552899+0000","last_change":"2024-09-21T07:35:27.877725+0000","last_active":"2024-09-21T07:48:49.552899+0000","last_peered":"2024-09-21T07:48:49.552899+0000","last_clean":"2024-09-21T07:48:49.552899+0000","last_became_active":"2024-09-21T07:35:27.877398+0000","last_became_peered":"2024-09-21T07:35:27.877398+0000","last_unstale":"2024-09-21T07:48:49.552899+0000","last_undegraded":"2024-09-21T07:48:49.552899+0000","last_fullsized":"2024-09-21T07:48:49.552899+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:05.859762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":316,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608538+0000","last_change":"2024-09-21T07:35:28.860612+0000","last_active":"2024-09-21T07:48:48.608538+0000","last_peered":"2024-09-21T07:48:48.608538+0000","last_clean":"2024-09-21T07:48:48.608538+0000","last_became_active":"2024-09-21T07:35:26.876562+0000","last_became_peered":"2024-09-21T07:35:26.876562+0000","last_unstale":"2024-09-21T07:48:48.608538+0000","last_undegraded":"2024-09-21T07:48:48.608538+0000","last_fullsized":"2024-09-21T07:48:48.608538+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:06.824331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012252470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":366,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609177+0000","last_change":"2024-09-21T07:34:24.260330+0000","last_active":"2024-09-21T07:48:48.609177+0000","last_peered":"2024-09-21T07:48:48.609177+0000","last_clean":"2024-09-21T07:48:48.609177+0000","last_became_active":"2024-09-21T07:34:23.637210+0000","last_became_peered":"2024-09-21T07:34:23.637210+0000","last_unstale":"2024-09-21T07:48:48.609177+0000","last_undegraded":"2024-09-21T07:48:48.609177+0000","last_fullsized":"2024-09-21T07:48:48.609177+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.488451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001537093,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.550464+0000","last_change":"2024-09-21T07:48:47.626890+0000","last_active":"2024-09-21T07:48:49.550464+0000","last_peered":"2024-09-21T07:48:49.550464+0000","last_clean":"2024-09-21T07:48:49.550464+0000","last_became_active":"2024-09-21T07:48:47.626088+0000","last_became_peered":"2024-09-21T07:48:47.626088+0000","last_unstale":"2024-09-21T07:48:49.550464+0000","last_undegraded":"2024-09-21T07:48:49.550464+0000","last_fullsized":"2024-09-21T07:48:49.550464+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:06:43.660952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.7","version":"218'2","reported_seq":17,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051435+0000","last_change":"2024-09-21T07:48:46.607721+0000","last_active":"2024-09-21T07:48:50.051435+0000","last_peered":"2024-09-21T07:48:50.051435+0000","last_clean":"2024-09-21T07:48:50.051435+0000","last_became_active":"2024-09-21T07:48:46.607413+0000","last_became_peered":"2024-09-21T07:48:46.607413+0000","last_unstale":"2024-09-21T07:48:50.051435+0000","last_undegraded":"2024-09-21T07:48:50.051435+0000","last_fullsized":"2024-09-21T07:48:50.051435+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:47.152954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":306,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.050294+0000","last_change":"2024-09-21T07:35:27.874757+0000","last_active":"2024-09-21T07:48:50.050294+0000","last_peered":"2024-09-21T07:48:50.050294+0000","last_clean":"2024-09-21T07:48:50.050294+0000","last_became_active":"2024-09-21T07:35:27.874360+0000","last_became_peered":"2024-09-21T07:35:27.874360+0000","last_unstale":"2024-09-21T07:48:50.050294+0000","last_undegraded":"2024-09-21T07:48:50.050294+0000","last_fullsized":"2024-09-21T07:48:50.050294+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:17.646912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":312,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608687+0000","last_change":"2024-09-21T07:35:29.493431+0000","last_active":"2024-09-21T07:48:48.608687+0000","last_peered":"2024-09-21T07:48:48.608687+0000","last_clean":"2024-09-21T07:48:48.608687+0000","last_became_active":"2024-09-21T07:35:26.873691+0000","last_became_peered":"2024-09-21T07:35:26.873691+0000","last_unstale":"2024-09-21T07:48:48.608687+0000","last_undegraded":"2024-09-21T07:48:48.608687+0000","last_fullsized":"2024-09-21T07:48:48.608687+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:51.705051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001162464,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":358,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051405+0000","last_change":"2024-09-21T07:34:24.597286+0000","last_active":"2024-09-21T07:48:50.051405+0000","last_peered":"2024-09-21T07:48:50.051405+0000","last_clean":"2024-09-21T07:48:50.051405+0000","last_became_active":"2024-09-21T07:34:23.267133+0000","last_became_peered":"2024-09-21T07:34:23.267133+0000","last_unstale":"2024-09-21T07:48:50.051405+0000","last_undegraded":"2024-09-21T07:48:50.051405+0000","last_fullsized":"2024-09-21T07:48:50.051405+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:18.257010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.601891+0000","last_change":"2024-09-21T07:48:47.624322+0000","last_active":"2024-09-21T07:48:49.601891+0000","last_peered":"2024-09-21T07:48:49.601891+0000","last_clean":"2024-09-21T07:48:49.601891+0000","last_became_active":"2024-09-21T07:48:47.623920+0000","last_became_peered":"2024-09-21T07:48:47.623920+0000","last_unstale":"2024-09-21T07:48:49.601891+0000","last_undegraded":"2024-09-21T07:48:49.601891+0000","last_fullsized":"2024-09-21T07:48:49.601891+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:08.454622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"219'1","reported_seq":16,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.552713+0000","last_change":"2024-09-21T07:48:46.599263+0000","last_active":"2024-09-21T07:48:49.552713+0000","last_peered":"2024-09-21T07:48:49.552713+0000","last_clean":"2024-09-21T07:48:49.552713+0000","last_became_active":"2024-09-21T07:48:46.597628+0000","last_became_peered":"2024-09-21T07:48:46.597628+0000","last_unstale":"2024-09-21T07:48:49.552713+0000","last_undegraded":"2024-09-21T07:48:49.552713+0000","last_fullsized":"2024-09-21T07:48:49.552713+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:38.541554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":319,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.550320+0000","last_change":"2024-09-21T07:35:27.864458+0000","last_active":"2024-09-21T07:48:49.550320+0000","last_peered":"2024-09-21T07:48:49.550320+0000","last_clean":"2024-09-21T07:48:49.550320+0000","last_became_active":"2024-09-21T07:35:27.864136+0000","last_became_peered":"2024-09-21T07:35:27.864136+0000","last_unstale":"2024-09-21T07:48:49.550320+0000","last_undegraded":"2024-09-21T07:48:49.550320+0000","last_fullsized":"2024-09-21T07:48:49.550320+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:20.162240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":326,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.551403+0000","last_change":"2024-09-21T07:35:28.858784+0000","last_active":"2024-09-21T07:48:49.551403+0000","last_peered":"2024-09-21T07:48:49.551403+0000","last_clean":"2024-09-21T07:48:49.551403+0000","last_became_active":"2024-09-21T07:35:26.876890+0000","last_became_peered":"2024-09-21T07:35:26.876890+0000","last_unstale":"2024-09-21T07:48:49.551403+0000","last_undegraded":"2024-09-21T07:48:49.551403+0000","last_fullsized":"2024-09-21T07:48:49.551403+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:40.738327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064543499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":369,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609757+0000","last_change":"2024-09-21T07:34:24.258712+0000","last_active":"2024-09-21T07:48:48.609757+0000","last_peered":"2024-09-21T07:48:48.609757+0000","last_clean":"2024-09-21T07:48:48.609757+0000","last_became_active":"2024-09-21T07:34:23.274692+0000","last_became_peered":"2024-09-21T07:34:23.274692+0000","last_unstale":"2024-09-21T07:48:48.609757+0000","last_undegraded":"2024-09-21T07:48:48.609757+0000","last_fullsized":"2024-09-21T07:48:48.609757+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:43.279454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018051479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.050557+0000","last_change":"2024-09-21T07:48:47.617896+0000","last_active":"2024-09-21T07:48:50.050557+0000","last_peered":"2024-09-21T07:48:50.050557+0000","last_clean":"2024-09-21T07:48:50.050557+0000","last_became_active":"2024-09-21T07:48:47.617588+0000","last_became_peered":"2024-09-21T07:48:47.617588+0000","last_unstale":"2024-09-21T07:48:50.050557+0000","last_undegraded":"2024-09-21T07:48:50.050557+0000","last_fullsized":"2024-09-21T07:48:50.050557+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:47.436066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051487+0000","last_change":"2024-09-21T07:48:46.609456+0000","last_active":"2024-09-21T07:48:50.051487+0000","last_peered":"2024-09-21T07:48:50.051487+0000","last_clean":"2024-09-21T07:48:50.051487+0000","last_became_active":"2024-09-21T07:48:46.609142+0000","last_became_peered":"2024-09-21T07:48:46.609142+0000","last_unstale":"2024-09-21T07:48:50.051487+0000","last_undegraded":"2024-09-21T07:48:50.051487+0000","last_fullsized":"2024-09-21T07:48:50.051487+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:21.813300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":308,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609370+0000","last_change":"2024-09-21T07:35:27.878020+0000","last_active":"2024-09-21T07:48:48.609370+0000","last_peered":"2024-09-21T07:48:48.609370+0000","last_clean":"2024-09-21T07:48:48.609370+0000","last_became_active":"2024-09-21T07:35:27.877532+0000","last_became_peered":"2024-09-21T07:35:27.877532+0000","last_unstale":"2024-09-21T07:48:48.609370+0000","last_undegraded":"2024-09-21T07:48:48.609370+0000","last_fullsized":"2024-09-21T07:48:48.609370+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:19.756073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":316,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.610786+0000","last_change":"2024-09-21T07:35:28.861049+0000","last_active":"2024-09-21T07:48:48.610786+0000","last_peered":"2024-09-21T07:48:48.610786+0000","last_clean":"2024-09-21T07:48:48.610786+0000","last_became_active":"2024-09-21T07:35:26.876819+0000","last_became_peered":"2024-09-21T07:35:26.876819+0000","last_unstale":"2024-09-21T07:48:48.610786+0000","last_undegraded":"2024-09-21T07:48:48.610786+0000","last_fullsized":"2024-09-21T07:48:48.610786+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:19.633803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055780599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":365,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.050459+0000","last_change":"2024-09-21T07:34:25.084849+0000","last_active":"2024-09-21T07:48:50.050459+0000","last_peered":"2024-09-21T07:48:50.050459+0000","last_clean":"2024-09-21T07:48:50.050459+0000","last_became_active":"2024-09-21T07:34:23.277713+0000","last_became_peered":"2024-09-21T07:34:23.277713+0000","last_unstale":"2024-09-21T07:48:50.050459+0000","last_undegraded":"2024-09-21T07:48:50.050459+0000","last_fullsized":"2024-09-21T07:48:50.050459+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:05.811810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059079499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.550788+0000","last_change":"2024-09-21T07:48:47.611938+0000","last_active":"2024-09-21T07:48:49.550788+0000","last_peered":"2024-09-21T07:48:49.550788+0000","last_clean":"2024-09-21T07:48:49.550788+0000","last_became_active":"2024-09-21T07:48:47.611580+0000","last_became_peered":"2024-09-21T07:48:47.611580+0000","last_unstale":"2024-09-21T07:48:49.550788+0000","last_undegraded":"2024-09-21T07:48:49.550788+0000","last_fullsized":"2024-09-21T07:48:49.550788+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:12:50.764399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.551355+0000","last_change":"2024-09-21T07:48:46.607396+0000","last_active":"2024-09-21T07:48:49.551355+0000","last_peered":"2024-09-21T07:48:49.551355+0000","last_clean":"2024-09-21T07:48:49.551355+0000","last_became_active":"2024-09-21T07:48:46.607082+0000","last_became_peered":"2024-09-21T07:48:46.607082+0000","last_unstale":"2024-09-21T07:48:49.551355+0000","last_undegraded":"2024-09-21T07:48:49.551355+0000","last_fullsized":"2024-09-21T07:48:49.551355+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:53:23.751238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":308,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.602217+0000","last_change":"2024-09-21T07:35:27.884248+0000","last_active":"2024-09-21T07:48:49.602217+0000","last_peered":"2024-09-21T07:48:49.602217+0000","last_clean":"2024-09-21T07:48:49.602217+0000","last_became_active":"2024-09-21T07:35:27.883861+0000","last_became_peered":"2024-09-21T07:35:27.883861+0000","last_unstale":"2024-09-21T07:48:49.602217+0000","last_undegraded":"2024-09-21T07:48:49.602217+0000","last_fullsized":"2024-09-21T07:48:49.602217+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":314,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.552147+0000","last_change":"2024-09-21T07:35:29.207135+0000","last_active":"2024-09-21T07:48:49.552147+0000","last_peered":"2024-09-21T07:48:49.552147+0000","last_clean":"2024-09-21T07:48:49.552147+0000","last_became_active":"2024-09-21T07:35:26.872787+0000","last_became_peered":"2024-09-21T07:35:26.872787+0000","last_unstale":"2024-09-21T07:48:49.552147+0000","last_undegraded":"2024-09-21T07:48:49.552147+0000","last_fullsized":"2024-09-21T07:48:49.552147+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:27.543217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011157509999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":378,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.550475+0000","last_change":"2024-09-21T07:34:24.538610+0000","last_active":"2024-09-21T07:48:49.550475+0000","last_peered":"2024-09-21T07:48:49.550475+0000","last_clean":"2024-09-21T07:48:49.550475+0000","last_became_active":"2024-09-21T07:34:23.259700+0000","last_became_peered":"2024-09-21T07:34:23.259700+0000","last_unstale":"2024-09-21T07:48:49.550475+0000","last_undegraded":"2024-09-21T07:48:49.550475+0000","last_fullsized":"2024-09-21T07:48:49.550475+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:57.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070522399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.551089+0000","last_change":"2024-09-21T07:48:47.623345+0000","last_active":"2024-09-21T07:48:49.551089+0000","last_peered":"2024-09-21T07:48:49.551089+0000","last_clean":"2024-09-21T07:48:49.551089+0000","last_became_active":"2024-09-21T07:48:47.622998+0000","last_became_peered":"2024-09-21T07:48:47.622998+0000","last_unstale":"2024-09-21T07:48:49.551089+0000","last_undegraded":"2024-09-21T07:48:49.551089+0000","last_fullsized":"2024-09-21T07:48:49.551089+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:55:53.150258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.551941+0000","last_change":"2024-09-21T07:48:46.599294+0000","last_active":"2024-09-21T07:48:49.551941+0000","last_peered":"2024-09-21T07:48:49.551941+0000","last_clean":"2024-09-21T07:48:49.551941+0000","last_became_active":"2024-09-21T07:48:46.598633+0000","last_became_peered":"2024-09-21T07:48:46.598633+0000","last_unstale":"2024-09-21T07:48:49.551941+0000","last_undegraded":"2024-09-21T07:48:49.551941+0000","last_fullsized":"2024-09-21T07:48:49.551941+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:34.227316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":319,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.551422+0000","last_change":"2024-09-21T07:35:27.868024+0000","last_active":"2024-09-21T07:48:49.551422+0000","last_peered":"2024-09-21T07:48:49.551422+0000","last_clean":"2024-09-21T07:48:49.551422+0000","last_became_active":"2024-09-21T07:35:27.867339+0000","last_became_peered":"2024-09-21T07:35:27.867339+0000","last_unstale":"2024-09-21T07:48:49.551422+0000","last_undegraded":"2024-09-21T07:48:49.551422+0000","last_fullsized":"2024-09-21T07:48:49.551422+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":326,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.550780+0000","last_change":"2024-09-21T07:35:28.858505+0000","last_active":"2024-09-21T07:48:49.550780+0000","last_peered":"2024-09-21T07:48:49.550780+0000","last_clean":"2024-09-21T07:48:49.550780+0000","last_became_active":"2024-09-21T07:35:26.877320+0000","last_became_peered":"2024-09-21T07:35:26.877320+0000","last_unstale":"2024-09-21T07:48:49.550780+0000","last_undegraded":"2024-09-21T07:48:49.550780+0000","last_fullsized":"2024-09-21T07:48:49.550780+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:53.672782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244119999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":360,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.052005+0000","last_change":"2024-09-21T07:34:24.597597+0000","last_active":"2024-09-21T07:48:50.052005+0000","last_peered":"2024-09-21T07:48:50.052005+0000","last_clean":"2024-09-21T07:48:50.052005+0000","last_became_active":"2024-09-21T07:34:23.635204+0000","last_became_peered":"2024-09-21T07:34:23.635204+0000","last_unstale":"2024-09-21T07:48:50.052005+0000","last_undegraded":"2024-09-21T07:48:50.052005+0000","last_fullsized":"2024-09-21T07:48:50.052005+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:48.644789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081139199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609468+0000","last_change":"2024-09-21T07:48:47.624826+0000","last_active":"2024-09-21T07:48:48.609468+0000","last_peered":"2024-09-21T07:48:48.609468+0000","last_clean":"2024-09-21T07:48:48.609468+0000","last_became_active":"2024-09-21T07:48:47.624259+0000","last_became_peered":"2024-09-21T07:48:47.624259+0000","last_unstale":"2024-09-21T07:48:48.609468+0000","last_undegraded":"2024-09-21T07:48:48.609468+0000","last_fullsized":"2024-09-21T07:48:48.609468+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:07.420806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.051183+0000","last_change":"2024-09-21T07:48:46.602945+0000","last_active":"2024-09-21T07:48:50.051183+0000","last_peered":"2024-09-21T07:48:50.051183+0000","last_clean":"2024-09-21T07:48:50.051183+0000","last_became_active":"2024-09-21T07:48:46.602486+0000","last_became_peered":"2024-09-21T07:48:46.602486+0000","last_unstale":"2024-09-21T07:48:50.051183+0000","last_undegraded":"2024-09-21T07:48:50.051183+0000","last_fullsized":"2024-09-21T07:48:50.051183+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:51:05.478321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":307,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.552544+0000","last_change":"2024-09-21T07:35:27.877503+0000","last_active":"2024-09-21T07:48:49.552544+0000","last_peered":"2024-09-21T07:48:49.552544+0000","last_clean":"2024-09-21T07:48:49.552544+0000","last_became_active":"2024-09-21T07:35:27.875775+0000","last_became_peered":"2024-09-21T07:35:27.875775+0000","last_unstale":"2024-09-21T07:48:49.552544+0000","last_undegraded":"2024-09-21T07:48:49.552544+0000","last_fullsized":"2024-09-21T07:48:49.552544+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:49.291023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":312,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.608298+0000","last_change":"2024-09-21T07:35:29.491412+0000","last_active":"2024-09-21T07:48:48.608298+0000","last_peered":"2024-09-21T07:48:48.608298+0000","last_clean":"2024-09-21T07:48:48.608298+0000","last_became_active":"2024-09-21T07:35:26.876404+0000","last_became_peered":"2024-09-21T07:35:26.876404+0000","last_unstale":"2024-09-21T07:48:48.608298+0000","last_undegraded":"2024-09-21T07:48:48.608298+0000","last_fullsized":"2024-09-21T07:48:48.608298+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063918000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":370,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.610303+0000","last_change":"2024-09-21T07:34:24.259026+0000","last_active":"2024-09-21T07:48:48.610303+0000","last_peered":"2024-09-21T07:48:48.610303+0000","last_clean":"2024-09-21T07:48:48.610303+0000","last_became_active":"2024-09-21T07:34:23.635456+0000","last_became_peered":"2024-09-21T07:34:23.635456+0000","last_unstale":"2024-09-21T07:48:48.610303+0000","last_undegraded":"2024-09-21T07:48:48.610303+0000","last_fullsized":"2024-09-21T07:48:48.610303+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:09.592752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018654419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609222+0000","last_change":"2024-09-21T07:48:47.624848+0000","last_active":"2024-09-21T07:48:48.609222+0000","last_peered":"2024-09-21T07:48:48.609222+0000","last_clean":"2024-09-21T07:48:48.609222+0000","last_became_active":"2024-09-21T07:48:47.624301+0000","last_became_peered":"2024-09-21T07:48:47.624301+0000","last_unstale":"2024-09-21T07:48:48.609222+0000","last_undegraded":"2024-09-21T07:48:48.609222+0000","last_fullsized":"2024-09-21T07:48:48.609222+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:24.141015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.550123+0000","last_change":"2024-09-21T07:48:46.612048+0000","last_active":"2024-09-21T07:48:49.550123+0000","last_peered":"2024-09-21T07:48:49.550123+0000","last_clean":"2024-09-21T07:48:49.550123+0000","last_became_active":"2024-09-21T07:48:46.611755+0000","last_became_peered":"2024-09-21T07:48:46.611755+0000","last_unstale":"2024-09-21T07:48:49.550123+0000","last_undegraded":"2024-09-21T07:48:49.550123+0000","last_fullsized":"2024-09-21T07:48:49.550123+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:16.350687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":307,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.552864+0000","last_change":"2024-09-21T07:35:27.871984+0000","last_active":"2024-09-21T07:48:49.552864+0000","last_peered":"2024-09-21T07:48:49.552864+0000","last_clean":"2024-09-21T07:48:49.552864+0000","last_became_active":"2024-09-21T07:35:27.870336+0000","last_became_peered":"2024-09-21T07:35:27.870336+0000","last_unstale":"2024-09-21T07:48:49.552864+0000","last_undegraded":"2024-09-21T07:48:49.552864+0000","last_fullsized":"2024-09-21T07:48:49.552864+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:57.120479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":316,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609408+0000","last_change":"2024-09-21T07:35:28.860351+0000","last_active":"2024-09-21T07:48:48.609408+0000","last_peered":"2024-09-21T07:48:48.609408+0000","last_clean":"2024-09-21T07:48:48.609408+0000","last_became_active":"2024-09-21T07:35:26.872046+0000","last_became_peered":"2024-09-21T07:35:26.872046+0000","last_unstale":"2024-09-21T07:48:48.609408+0000","last_undegraded":"2024-09-21T07:48:48.609408+0000","last_fullsized":"2024-09-21T07:48:48.609408+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:47.074327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098384399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":365,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.609455+0000","last_change":"2024-09-21T07:34:24.260300+0000","last_active":"2024-09-21T07:48:48.609455+0000","last_peered":"2024-09-21T07:48:48.609455+0000","last_clean":"2024-09-21T07:48:48.609455+0000","last_became_active":"2024-09-21T07:34:23.270128+0000","last_became_peered":"2024-09-21T07:34:23.270128+0000","last_unstale":"2024-09-21T07:48:48.609455+0000","last_undegraded":"2024-09-21T07:48:48.609455+0000","last_fullsized":"2024-09-21T07:48:48.609455+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:25.260931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00176651,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":13,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.602582+0000","last_change":"2024-09-21T07:48:47.626058+0000","last_active":"2024-09-21T07:48:49.602582+0000","last_peered":"2024-09-21T07:48:49.602582+0000","last_clean":"2024-09-21T07:48:49.602582+0000","last_became_active":"2024-09-21T07:48:47.625779+0000","last_became_peered":"2024-09-21T07:48:47.625779+0000","last_unstale":"2024-09-21T07:48:49.602582+0000","last_undegraded":"2024-09-21T07:48:49.602582+0000","last_fullsized":"2024-09-21T07:48:49.602582+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":217,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:46.575553+0000","last_clean_scrub_stamp":"2024-09-21T07:48:46.575553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:47.074247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":15,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:48.610266+0000","last_change":"2024-09-21T07:48:46.607858+0000","last_active":"2024-09-21T07:48:48.610266+0000","last_peered":"2024-09-21T07:48:48.610266+0000","last_clean":"2024-09-21T07:48:48.610266+0000","last_became_active":"2024-09-21T07:48:46.607521+0000","last_became_peered":"2024-09-21T07:48:46.607521+0000","last_unstale":"2024-09-21T07:48:48.610266+0000","last_undegraded":"2024-09-21T07:48:48.610266+0000","last_fullsized":"2024-09-21T07:48:48.610266+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":216,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:48:45.564066+0000","last_clean_scrub_stamp":"2024-09-21T07:48:45.564066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:18.298760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":307,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.553043+0000","last_change":"2024-09-21T07:35:27.877861+0000","last_active":"2024-09-21T07:48:49.553043+0000","last_peered":"2024-09-21T07:48:49.553043+0000","last_clean":"2024-09-21T07:48:49.553043+0000","last_became_active":"2024-09-21T07:35:27.877434+0000","last_became_peered":"2024-09-21T07:35:27.877434+0000","last_unstale":"2024-09-21T07:48:49.553043+0000","last_undegraded":"2024-09-21T07:48:49.553043+0000","last_fullsized":"2024-09-21T07:48:49.553043+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":313,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:50.050648+0000","last_change":"2024-09-21T07:35:28.862954+0000","last_active":"2024-09-21T07:48:50.050648+0000","last_peered":"2024-09-21T07:48:50.050648+0000","last_clean":"2024-09-21T07:48:50.050648+0000","last_became_active":"2024-09-21T07:35:26.870613+0000","last_became_peered":"2024-09-21T07:35:26.870613+0000","last_unstale":"2024-09-21T07:48:50.050648+0000","last_undegraded":"2024-09-21T07:48:50.050648+0000","last_fullsized":"2024-09-21T07:48:50.050648+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:06.901043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059842100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":366,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-21T07:48:49.602888+0000","last_change":"2024-09-21T07:34:25.028904+0000","last_active":"2024-09-21T07:48:49.602888+0000","last_peered":"2024-09-21T07:48:49.602888+0000","last_clean":"2024-09-21T07:48:49.602888+0000","last_became_active":"2024-09-21T07:34:23.260942+0000","last_became_peered":"2024-09-21T07:34:23.260942+0000","last_unstale":"2024-09-21T07:48:49.602888+0000","last_undegraded":"2024-09-21T07:48:49.602888+0000","last_fullsized":"2024-09-21T07:48:49.602888+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:22.709328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060553899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509633,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66420,"kb_used_data":3288,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94305420,"statfs":{"total":96636764160,"available":96568750080,"internally_reserved":0,"allocated":3366912,"data_stored":2063963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509635,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51944,"kb_used_data":2708,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319896,"statfs":{"total":96636764160,"available":96583573504,"internally_reserved":0,"allocated":2772992,"data_stored":1474107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509633,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33888,"kb_used_data":2700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337952,"statfs":{"total":96636764160,"available":96602062848,"internally_reserved":0,"allocated":2764800,"data_stored":1471969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509634,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29732,"kb_used_data":2696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342108,"statfs":{"total":96636764160,"available":96606318592,"internally_reserved":0,"allocated":2760704,"data_stored":1472003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509635,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47768,"kb_used_data":2692,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324072,"statfs":{"total":96636764160,"available":96587849728,"internally_reserved":0,"allocated":2756608,"data_stored":1471913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509635,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52512,"kb_used_data":3276,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319328,"statfs":{"total":96636764160,"available":96582991872,"internally_reserved":0,"allocated":3354624,"data_stored":2062371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542338,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51936,"kb_used_data":2696,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319904,"statfs":{"total":96636764160,"available":96583581696,"internally_reserved":0,"allocated":2760704,"data_stored":1472495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542337,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74148,"kb_used_data":2704,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297692,"statfs":{"total":96636764160,"available":96560836608,"internally_reserved":0,"allocated":2768896,"data_stored":1472039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:48:57.670 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T07:48:57.670 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T07:48:57.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.934+0000 7fb411600640 1 -- 172.21.15.39:0/2799816378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb40c0d0da0 msgr2=0x7fb40c0cf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:57.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.934+0000 7fb411600640 1 --2- 172.21.15.39:0/2799816378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb40c0d0da0 0x7fb40c0cf450 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4009830 tx=0x7fb3f402a530 comp rx=0 tx=0).stop 2024-09-21T07:48:57.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.935+0000 7fb411600640 1 -- 172.21.15.39:0/2799816378 shutdown_connections 2024-09-21T07:48:57.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.935+0000 7fb411600640 1 --2- 172.21.15.39:0/2799816378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb40c0d0da0 0x7fb40c0cf450 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:57.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.935+0000 7fb411600640 1 -- 172.21.15.39:0/2799816378 >> 172.21.15.39:0/2799816378 conn(0x7fb40c06cfc0 msgr2=0x7fb40c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:57.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.935+0000 7fb411600640 1 -- 172.21.15.39:0/2799816378 shutdown_connections 2024-09-21T07:48:57.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.935+0000 7fb411600640 1 -- 172.21.15.39:0/2799816378 wait complete. 2024-09-21T07:48:57.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.937+0000 7fb411600640 1 Processor -- start 2024-09-21T07:48:57.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.937+0000 7fb411600640 1 -- start start 2024-09-21T07:48:57.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.939+0000 7fb411600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb40c070c30 0x7fb40c1e5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:57.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.939+0000 7fb411600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb40c0cf990 0x7fb40c1eb590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:57.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.939+0000 7fb411600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb40c1f9760 0x7fb40c151a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:57.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.939+0000 7fb411600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c073bd0 con 0x7fb40c0cf990 2024-09-21T07:48:57.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.939+0000 7fb411600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c073a50 con 0x7fb40c070c30 2024-09-21T07:48:57.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.939+0000 7fb411600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb40c073d50 con 0x7fb40c1f9760 2024-09-21T07:48:57.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.939+0000 7fb403e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb40c0cf990 0x7fb40c1eb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:57.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.939+0000 7fb403e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb40c0cf990 0x7fb40c1eb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48890/0 (socket says 172.21.15.39:48890) 2024-09-21T07:48:57.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.939+0000 7fb403e00640 1 -- 172.21.15.39:0/731815965 learned_addr learned my addr 172.21.15.39:0/731815965 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:57.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.939+0000 7fb40aa00640 1 --2- 172.21.15.39:0/731815965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb40c070c30 0x7fb40c1e5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:57.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.939+0000 7fb40b400640 1 --2- 172.21.15.39:0/731815965 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb40c1f9760 0x7fb40c151a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:57.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.939+0000 7fb403e00640 1 -- 172.21.15.39:0/731815965 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb40c1f9760 msgr2=0x7fb40c151a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:57.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.939+0000 7fb403e00640 1 --2- 172.21.15.39:0/731815965 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb40c1f9760 0x7fb40c151a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:57.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.939+0000 7fb403e00640 1 -- 172.21.15.39:0/731815965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb40c070c30 msgr2=0x7fb40c1e5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:57.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.939+0000 7fb403e00640 1 --2- 172.21.15.39:0/731815965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb40c070c30 0x7fb40c1e5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:57.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.939+0000 7fb403e00640 1 -- 172.21.15.39:0/731815965 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb40c1ec480 con 0x7fb40c0cf990 2024-09-21T07:48:57.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.940+0000 7fb403e00640 1 --2- 172.21.15.39:0/731815965 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb40c0cf990 0x7fb40c1eb590 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4009950 tx=0x7fb3f4033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:57.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.940+0000 7fb402a00640 1 -- 172.21.15.39:0/731815965 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4031b60 con 0x7fb40c0cf990 2024-09-21T07:48:57.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.940+0000 7fb402a00640 1 -- 172.21.15.39:0/731815965 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb3f4031df0 con 0x7fb40c0cf990 2024-09-21T07:48:57.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.940+0000 7fb402a00640 1 -- 172.21.15.39:0/731815965 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4005280 con 0x7fb40c0cf990 2024-09-21T07:48:57.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.940+0000 7fb411600640 1 -- 172.21.15.39:0/731815965 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb40c1ec1a0 con 0x7fb40c0cf990 2024-09-21T07:48:57.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.940+0000 7fb411600640 1 -- 172.21.15.39:0/731815965 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb40c1f8f60 con 0x7fb40c0cf990 2024-09-21T07:48:57.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.941+0000 7fb411600640 1 -- 172.21.15.39:0/731815965 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb40c1ebf10 con 0x7fb40c0cf990 2024-09-21T07:48:57.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.943+0000 7fb402a00640 1 -- 172.21.15.39:0/731815965 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb3f4005420 con 0x7fb40c0cf990 2024-09-21T07:48:57.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.943+0000 7fb402a00640 1 --2- 172.21.15.39:0/731815965 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb3e8078b90 0x7fb3e807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:57.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.944+0000 7fb402a00640 1 -- 172.21.15.39:0/731815965 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(221..221 src has 1..221) ==== 7075+0+0 (secure 0 0 0) 0x7fb3f40bc480 con 0x7fb40c0cf990 2024-09-21T07:48:57.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.944+0000 7fb40aa00640 1 --2- 172.21.15.39:0/731815965 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb3e8078b90 0x7fb3e807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:57.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.945+0000 7fb40aa00640 1 --2- 172.21.15.39:0/731815965 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb3e8078b90 0x7fb3e807b0a0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc000f60 tx=0x7fb3fc009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:57.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:57.948+0000 7fb402a00640 1 -- 172.21.15.39:0/731815965 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb3f403f030 con 0x7fb40c0cf990 2024-09-21T07:48:58.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.216+0000 7fb411600640 1 -- 172.21.15.39:0/731815965 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fb40c0cf450 con 0x7fb40c0cf990 2024-09-21T07:48:58.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.220+0000 7fb402a00640 1 -- 172.21.15.39:0/731815965 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) ==== 190+0+147 (secure 0 0 0) 0x7fb3f4031770 con 0x7fb40c0cf990 2024-09-21T07:48:58.222 INFO:teuthology.orchestra.run.smithi039.stdout:[client.0] 2024-09-21T07:48:58.222 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQB0du5mGfb1DBAA/aSMKR5or3xy53rB/iUjpA== 2024-09-21T07:48:58.222 INFO:teuthology.orchestra.run.smithi039.stdout: caps mds = "allow" 2024-09-21T07:48:58.222 INFO:teuthology.orchestra.run.smithi039.stdout: caps mon = "allow r" 2024-09-21T07:48:58.222 INFO:teuthology.orchestra.run.smithi039.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T07:48:58.222 INFO:teuthology.orchestra.run.smithi039.stderr:updated caps for client.0 2024-09-21T07:48:58.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.222+0000 7fb411600640 1 -- 172.21.15.39:0/731815965 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb3e8078b90 msgr2=0x7fb3e807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:58.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.222+0000 7fb411600640 1 --2- 172.21.15.39:0/731815965 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb3e8078b90 0x7fb3e807b0a0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc000f60 tx=0x7fb3fc009040 comp rx=0 tx=0).stop 2024-09-21T07:48:58.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.222+0000 7fb411600640 1 -- 172.21.15.39:0/731815965 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb40c0cf990 msgr2=0x7fb40c1eb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:58.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.222+0000 7fb411600640 1 --2- 172.21.15.39:0/731815965 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb40c0cf990 0x7fb40c1eb590 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4009950 tx=0x7fb3f4033c10 comp rx=0 tx=0).stop 2024-09-21T07:48:58.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.222+0000 7fb411600640 1 -- 172.21.15.39:0/731815965 shutdown_connections 2024-09-21T07:48:58.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.222+0000 7fb411600640 1 --2- 172.21.15.39:0/731815965 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb3e8078b90 0x7fb3e807b0a0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:58.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.222+0000 7fb411600640 1 --2- 172.21.15.39:0/731815965 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb40c1f9760 0x7fb40c151a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:58.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.222+0000 7fb411600640 1 --2- 172.21.15.39:0/731815965 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb40c0cf990 0x7fb40c1eb590 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:58.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.222+0000 7fb411600640 1 --2- 172.21.15.39:0/731815965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb40c070c30 0x7fb40c1e5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:58.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.222+0000 7fb411600640 1 -- 172.21.15.39:0/731815965 >> 172.21.15.39:0/731815965 conn(0x7fb40c06cfc0 msgr2=0x7fb40c1eae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:58.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.222+0000 7fb411600640 1 -- 172.21.15.39:0/731815965 shutdown_connections 2024-09-21T07:48:58.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.222+0000 7fb411600640 1 -- 172.21.15.39:0/731815965 wait complete. 2024-09-21T07:48:58.248 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:48:58.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.472+0000 7fc1a6800640 1 -- 172.21.15.39:0/1942853816 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1a00cf990 msgr2=0x7fc1a00b40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:58.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.472+0000 7fc1a6800640 1 --2- 172.21.15.39:0/1942853816 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1a00cf990 0x7fc1a00b40b0 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7fc1a00cdd60 tx=0x7fc19802a8b0 comp rx=0 tx=0).stop 2024-09-21T07:48:58.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.472+0000 7fc1a6800640 1 -- 172.21.15.39:0/1942853816 shutdown_connections 2024-09-21T07:48:58.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.472+0000 7fc1a6800640 1 --2- 172.21.15.39:0/1942853816 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1a00cf990 0x7fc1a00b40b0 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:58.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.472+0000 7fc1a6800640 1 --2- 172.21.15.39:0/1942853816 >> v2:172.21.15.148:3300/0 conn(0x7fc1a00cd260 0x7fc1a00cf450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:58.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.472+0000 7fc1a6800640 1 -- 172.21.15.39:0/1942853816 >> 172.21.15.39:0/1942853816 conn(0x7fc1a00b2460 msgr2=0x7fc1a006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:58.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.472+0000 7fc1a6800640 1 -- 172.21.15.39:0/1942853816 shutdown_connections 2024-09-21T07:48:58.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.473+0000 7fc1a6800640 1 -- 172.21.15.39:0/1942853816 wait complete. 2024-09-21T07:48:58.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.475+0000 7fc1a6800640 1 Processor -- start 2024-09-21T07:48:58.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.475+0000 7fc1a6800640 1 -- start start 2024-09-21T07:48:58.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.476+0000 7fc1a6800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1a00af430 0x7fc1a0099ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:58.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.476+0000 7fc1a6800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1a00cd260 0x7fc1a01593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:58.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.476+0000 7fc1a6800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc1a00cf990 0x7fc1a01598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:58.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.476+0000 7fc1a6800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a00b45f0 con 0x7fc1a00af430 2024-09-21T07:48:58.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.476+0000 7fc1a6800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a0071230 con 0x7fc1a00cd260 2024-09-21T07:48:58.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.476+0000 7fc1a6800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc1a00b4770 con 0x7fc1a00cf990 2024-09-21T07:48:58.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.477+0000 7fc19f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1a00af430 0x7fc1a0099ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.477+0000 7fc19f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1a00af430 0x7fc1a0099ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48904/0 (socket says 172.21.15.39:48904) 2024-09-21T07:48:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.477+0000 7fc19f400640 1 -- 172.21.15.39:0/702855401 learned_addr learned my addr 172.21.15.39:0/702855401 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.477+0000 7fc19ea00640 1 --2- 172.21.15.39:0/702855401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1a00cd260 0x7fc1a01593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.477+0000 7fc19fe00640 1 --2- 172.21.15.39:0/702855401 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc1a00cf990 0x7fc1a01598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.477+0000 7fc19f400640 1 -- 172.21.15.39:0/702855401 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc1a00cf990 msgr2=0x7fc1a01598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.477+0000 7fc19f400640 1 --2- 172.21.15.39:0/702855401 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc1a00cf990 0x7fc1a01598f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.477+0000 7fc19f400640 1 -- 172.21.15.39:0/702855401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1a00cd260 msgr2=0x7fc1a01593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.477+0000 7fc19f400640 1 --2- 172.21.15.39:0/702855401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1a00cd260 0x7fc1a01593b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.477+0000 7fc19f400640 1 -- 172.21.15.39:0/702855401 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a00c74a0 con 0x7fc1a00af430 2024-09-21T07:48:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.477+0000 7fc19fe00640 1 --2- 172.21.15.39:0/702855401 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc1a00cf990 0x7fc1a01598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:48:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.477+0000 7fc19f400640 1 --2- 172.21.15.39:0/702855401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1a00af430 0x7fc1a0099ac0 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7fc19000c3c0 tx=0x7fc19000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:58.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.478+0000 7fc197400640 1 -- 172.21.15.39:0/702855401 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc190017070 con 0x7fc1a00af430 2024-09-21T07:48:58.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.478+0000 7fc197400640 1 -- 172.21.15.39:0/702855401 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1900035f0 con 0x7fc1a00af430 2024-09-21T07:48:58.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.478+0000 7fc197400640 1 -- 172.21.15.39:0/702855401 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc190003a80 con 0x7fc1a00af430 2024-09-21T07:48:58.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.478+0000 7fc1a6800640 1 -- 172.21.15.39:0/702855401 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1a00c7780 con 0x7fc1a00af430 2024-09-21T07:48:58.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.478+0000 7fc1a6800640 1 -- 172.21.15.39:0/702855401 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1a00afed0 con 0x7fc1a00af430 2024-09-21T07:48:58.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.479+0000 7fc1a6800640 1 -- 172.21.15.39:0/702855401 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1a00b0130 con 0x7fc1a00af430 2024-09-21T07:48:58.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.480+0000 7fc197400640 1 -- 172.21.15.39:0/702855401 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc19001a070 con 0x7fc1a00af430 2024-09-21T07:48:58.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.481+0000 7fc197400640 1 --2- 172.21.15.39:0/702855401 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc180078b90 0x7fc18007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:58.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.481+0000 7fc197400640 1 -- 172.21.15.39:0/702855401 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(221..221 src has 1..221) ==== 7075+0+0 (secure 0 0 0) 0x7fc19009a550 con 0x7fc1a00af430 2024-09-21T07:48:58.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.482+0000 7fc19ea00640 1 --2- 172.21.15.39:0/702855401 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc180078b90 0x7fc18007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:58.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.483+0000 7fc19ea00640 1 --2- 172.21.15.39:0/702855401 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc180078b90 0x7fc18007b0a0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fc1a015a730 tx=0x7fc18c00b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:58.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.486+0000 7fc197400640 1 -- 172.21.15.39:0/702855401 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc190061dd0 con 0x7fc1a00af430 2024-09-21T07:48:58.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.720+0000 7fc1a6800640 1 -- 172.21.15.39:0/702855401 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1a00cf450 con 0x7fc1a00af430 2024-09-21T07:48:58.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.721+0000 7fc197400640 1 -- 172.21.15.39:0/702855401 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 229 v229) ==== 95+0+5241 (secure 0 0 0) 0x7fc190066800 con 0x7fc1a00af430 2024-09-21T07:48:58.723 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:48:58.723 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":229,"btime":"2024-09-21T07:48:50:615411+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8249,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/550771731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":550771731},{"type":"v1","addr":"172.21.15.148:6837","nonce":550771731}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":224},{"gid":9385,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1307677219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":1307677219},{"type":"v1","addr":"172.21.15.39:6835","nonce":1307677219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":224},{"gid":9391,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1256253563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1256253563},{"type":"v1","addr":"172.21.15.148:6836","nonce":1256253563}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":224}],"filesystems":[{"mdsmap":{"epoch":229,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:48:47.106474+0000","modified":"2024-09-21T07:48:50.124278+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10224},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10224":{"gid":10224,"name":"c","rank":0,"incarnation":225,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/2837221709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2837221709},{"type":"v1","addr":"172.21.15.39:6834","nonce":2837221709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10224,"qdb_cluster":[10224]},"id":12}]} 2024-09-21T07:48:58.723 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 229 2024-09-21T07:48:58.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.723+0000 7fc1a6800640 1 -- 172.21.15.39:0/702855401 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc180078b90 msgr2=0x7fc18007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:58.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.723+0000 7fc1a6800640 1 --2- 172.21.15.39:0/702855401 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc180078b90 0x7fc18007b0a0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fc1a015a730 tx=0x7fc18c00b2e0 comp rx=0 tx=0).stop 2024-09-21T07:48:58.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.723+0000 7fc1a6800640 1 -- 172.21.15.39:0/702855401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1a00af430 msgr2=0x7fc1a0099ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:58.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.723+0000 7fc1a6800640 1 --2- 172.21.15.39:0/702855401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1a00af430 0x7fc1a0099ac0 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7fc19000c3c0 tx=0x7fc19000c880 comp rx=0 tx=0).stop 2024-09-21T07:48:58.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.723+0000 7fc1a6800640 1 -- 172.21.15.39:0/702855401 shutdown_connections 2024-09-21T07:48:58.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.723+0000 7fc1a6800640 1 --2- 172.21.15.39:0/702855401 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc180078b90 0x7fc18007b0a0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:58.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.723+0000 7fc1a6800640 1 --2- 172.21.15.39:0/702855401 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc1a00cf990 0x7fc1a01598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:58.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.723+0000 7fc1a6800640 1 --2- 172.21.15.39:0/702855401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc1a00cd260 0x7fc1a01593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:58.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.723+0000 7fc1a6800640 1 --2- 172.21.15.39:0/702855401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1a00af430 0x7fc1a0099ac0 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:58.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.723+0000 7fc1a6800640 1 -- 172.21.15.39:0/702855401 >> 172.21.15.39:0/702855401 conn(0x7fc1a00b2460 msgr2=0x7fc1a006ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:58.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.723+0000 7fc1a6800640 1 -- 172.21.15.39:0/702855401 shutdown_connections 2024-09-21T07:48:58.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:58.723+0000 7fc1a6800640 1 -- 172.21.15.39:0/702855401 wait complete. 2024-09-21T07:48:59.151 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 229, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T07:48:47.106474+0000', 'modified': '2024-09-21T07:48:50.124278+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10224}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10224': {'gid': 10224, 'name': 'c', 'rank': 0, 'incarnation': 225, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.39:6834/2837221709', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 2837221709}, {'type': 'v1', 'addr': '172.21.15.39:6834', 'nonce': 2837221709}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10224, 'qdb_cluster': [10224]} 2024-09-21T07:48:59.151 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T07:48:59.151 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T07:48:59.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.379+0000 7f4805800640 1 -- 172.21.15.39:0/1835314950 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47e4003fd0 msgr2=0x7f47e4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:59.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.379+0000 7f4805800640 1 --2- 172.21.15.39:0/1835314950 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47e4003fd0 0x7f47e4004800 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f4800075160 tx=0x7f47e4005830 comp rx=0 tx=0).stop 2024-09-21T07:48:59.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.379+0000 7f47ff400640 1 -- 172.21.15.39:0/1835314950 reap_dead start 2024-09-21T07:48:59.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.379+0000 7f4805800640 1 -- 172.21.15.39:0/1835314950 shutdown_connections 2024-09-21T07:48:59.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.379+0000 7f4805800640 1 -- 172.21.15.39:0/1835314950 >> 172.21.15.39:0/1835314950 conn(0x7f48000b2460 msgr2=0x7f480006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:59.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.379+0000 7f4805800640 1 -- 172.21.15.39:0/1835314950 shutdown_connections 2024-09-21T07:48:59.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.379+0000 7f4805800640 1 -- 172.21.15.39:0/1835314950 wait complete. 2024-09-21T07:48:59.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.382+0000 7f4805800640 1 Processor -- start 2024-09-21T07:48:59.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.382+0000 7f4805800640 1 -- start start 2024-09-21T07:48:59.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.383+0000 7f4805800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47e4003fd0 0x7f48000c6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:59.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.383+0000 7f4805800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4800155e30 0x7f48001f6f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:59.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.383+0000 7f4805800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48001f7470 0x7f48001f4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:59.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.383+0000 7f4805800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f48000cbc30 con 0x7f47e4003fd0 2024-09-21T07:48:59.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.383+0000 7f4805800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f48000cbab0 con 0x7f4800155e30 2024-09-21T07:48:59.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.383+0000 7f4805800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f48000af080 con 0x7f48001f7470 2024-09-21T07:48:59.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.383+0000 7f47fea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47e4003fd0 0x7f48000c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:59.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.383+0000 7f47fea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47e4003fd0 0x7f48000c6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48924/0 (socket says 172.21.15.39:48924) 2024-09-21T07:48:59.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.383+0000 7f47fea00640 1 -- 172.21.15.39:0/1316194144 learned_addr learned my addr 172.21.15.39:0/1316194144 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:59.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.384+0000 7f47f7e00640 1 --2- 172.21.15.39:0/1316194144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4800155e30 0x7f48001f6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:59.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.384+0000 7f47ff400640 1 --2- 172.21.15.39:0/1316194144 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48001f7470 0x7f48001f4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:59.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.384+0000 7f47fea00640 1 -- 172.21.15.39:0/1316194144 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48001f7470 msgr2=0x7f48001f4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:59.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.384+0000 7f47fea00640 1 --2- 172.21.15.39:0/1316194144 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48001f7470 0x7f48001f4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:59.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.384+0000 7f47fea00640 1 -- 172.21.15.39:0/1316194144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4800155e30 msgr2=0x7f48001f6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:59.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.384+0000 7f47fea00640 1 --2- 172.21.15.39:0/1316194144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4800155e30 0x7f48001f6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:59.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.384+0000 7f47fea00640 1 -- 172.21.15.39:0/1316194144 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48000c7eb0 con 0x7f47e4003fd0 2024-09-21T07:48:59.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.384+0000 7f47fea00640 1 --2- 172.21.15.39:0/1316194144 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47e4003fd0 0x7f48000c6330 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f47f0011ae0 tx=0x7f47f0009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:59.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.384+0000 7f47f6a00640 1 -- 172.21.15.39:0/1316194144 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f001e070 con 0x7f47e4003fd0 2024-09-21T07:48:59.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.384+0000 7f47f6a00640 1 -- 172.21.15.39:0/1316194144 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47f000fe70 con 0x7f47e4003fd0 2024-09-21T07:48:59.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.384+0000 7f47f6a00640 1 -- 172.21.15.39:0/1316194144 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f0004290 con 0x7f47e4003fd0 2024-09-21T07:48:59.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.384+0000 7f4805800640 1 -- 172.21.15.39:0/1316194144 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48000c8190 con 0x7f47e4003fd0 2024-09-21T07:48:59.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.385+0000 7f4805800640 1 -- 172.21.15.39:0/1316194144 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48001f5a00 con 0x7f47e4003fd0 2024-09-21T07:48:59.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.387+0000 7f47f6a00640 1 -- 172.21.15.39:0/1316194144 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f47f0021070 con 0x7f47e4003fd0 2024-09-21T07:48:59.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.388+0000 7f47f6a00640 1 --2- 172.21.15.39:0/1316194144 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f47dc078b90 0x7f47dc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:59.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.388+0000 7f47f6a00640 1 -- 172.21.15.39:0/1316194144 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(221..221 src has 1..221) ==== 7075+0+0 (secure 0 0 0) 0x7f47f009baf0 con 0x7f47e4003fd0 2024-09-21T07:48:59.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.388+0000 7f47f7e00640 1 --2- 172.21.15.39:0/1316194144 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f47dc078b90 0x7f47dc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:59.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.388+0000 7f480c8d8740 1 Processor -- start 2024-09-21T07:48:59.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.389+0000 7f480c8d8740 1 -- start start 2024-09-21T07:48:59.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.389+0000 7f47f7e00640 1 --2- 172.21.15.39:0/1316194144 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f47dc078b90 0x7f47dc07b0a0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f47e8035000 tx=0x7f47e80046c0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:59.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.390+0000 7f480c8d8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55699567d270 0x556995a68a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:59.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.390+0000 7f480c8d8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5569956543b0 0x556995a1df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:59.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.390+0000 7f480c8d8740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55699565c590 0x556995a1e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:59.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.390+0000 7f480c8d8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5569959a7430 con 0x5569956543b0 2024-09-21T07:48:59.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.390+0000 7f480c8d8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5569959975d0 con 0x55699567d270 2024-09-21T07:48:59.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.390+0000 7f480c8d8740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x556995986a00 con 0x55699565c590 2024-09-21T07:48:59.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.390+0000 7f47f7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5569956543b0 0x556995a1df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:59.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.390+0000 7f47f7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5569956543b0 0x556995a1df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48926/0 (socket says 172.21.15.39:48926) 2024-09-21T07:48:59.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.390+0000 7f47f7e00640 1 -- 172.21.15.39:0/2603779948 learned_addr learned my addr 172.21.15.39:0/2603779948 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:59.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.390+0000 7f47fea00640 1 --2- 172.21.15.39:0/2603779948 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55699567d270 0x556995a68a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:59.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.391+0000 7f47ff400640 1 --2- 172.21.15.39:0/2603779948 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55699565c590 0x556995a1e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:59.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.391+0000 7f47f7e00640 1 -- 172.21.15.39:0/2603779948 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55699565c590 msgr2=0x556995a1e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:48:59.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.391+0000 7f47f7e00640 1 --2- 172.21.15.39:0/2603779948 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55699565c590 0x556995a1e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:59.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.391+0000 7f47f7e00640 1 -- 172.21.15.39:0/2603779948 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55699567d270 msgr2=0x556995a68a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:48:59.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.391+0000 7f47f7e00640 1 --2- 172.21.15.39:0/2603779948 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55699567d270 0x556995a68a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:59.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.391+0000 7f47f7e00640 1 -- 172.21.15.39:0/2603779948 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55699594f520 con 0x5569956543b0 2024-09-21T07:48:59.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.391+0000 7f47f7e00640 1 --2- 172.21.15.39:0/2603779948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5569956543b0 0x556995a1df60 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f47e8009dc0 tx=0x7f47e803c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4ab1761666ebf42 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:59.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.391+0000 7f47ef400640 1 -- 172.21.15.39:0/2603779948 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e8038040 con 0x5569956543b0 2024-09-21T07:48:59.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.391+0000 7f47ef400640 1 -- 172.21.15.39:0/2603779948 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47e8045040 con 0x5569956543b0 2024-09-21T07:48:59.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.391+0000 7f47ef400640 1 -- 172.21.15.39:0/2603779948 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e8005550 con 0x5569956543b0 2024-09-21T07:48:59.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.392+0000 7f480c8d8740 1 -- 172.21.15.39:0/2603779948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5569956543b0 msgr2=0x556995a1df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:59.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.392+0000 7f480c8d8740 1 --2- 172.21.15.39:0/2603779948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5569956543b0 0x556995a1df60 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f47e8009dc0 tx=0x7f47e803c000 comp rx=0 tx=0).stop 2024-09-21T07:48:59.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.392+0000 7f480c8d8740 1 -- 172.21.15.39:0/2603779948 shutdown_connections 2024-09-21T07:48:59.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.392+0000 7f480c8d8740 1 --2- 172.21.15.39:0/2603779948 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55699565c590 0x556995a1e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:59.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.392+0000 7f480c8d8740 1 --2- 172.21.15.39:0/2603779948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5569956543b0 0x556995a1df60 unknown :-1 s=CLOSED pgs=2121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:59.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.392+0000 7f480c8d8740 1 --2- 172.21.15.39:0/2603779948 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55699567d270 0x556995a68a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:59.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.392+0000 7f480c8d8740 1 -- 172.21.15.39:0/2603779948 >> 172.21.15.39:0/2603779948 conn(0x5569956713c0 msgr2=0x556995a23970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:59.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.392+0000 7f480c8d8740 1 -- 172.21.15.39:0/2603779948 shutdown_connections 2024-09-21T07:48:59.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.392+0000 7f480c8d8740 1 -- 172.21.15.39:0/2603779948 wait complete. 2024-09-21T07:48:59.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.393+0000 7f480c8d8740 1 -- start start 2024-09-21T07:48:59.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.394+0000 7f480c8d8740 1 Processor -- start 2024-09-21T07:48:59.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.395+0000 7f480c8d8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5569956543b0 0x556995a793a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:59.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.395+0000 7f480c8d8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55699565c590 0x556995a656a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:59.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.395+0000 7f480c8d8740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55699567d270 0x556995a7f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:59.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.395+0000 7f480c8d8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55699592d330 con 0x55699565c590 2024-09-21T07:48:59.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.395+0000 7f480c8d8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55699598a810 con 0x5569956543b0 2024-09-21T07:48:59.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.395+0000 7f480c8d8740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x5569956f7620 con 0x55699567d270 2024-09-21T07:48:59.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.395+0000 7f480c8d8740 10 client.0 _ll_register_callbacks cb 0x556995932420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:48:59.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.395+0000 7f47f7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55699565c590 0x556995a656a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:59.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.395+0000 7f47f7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55699565c590 0x556995a656a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48928/0 (socket says 172.21.15.39:48928) 2024-09-21T07:48:59.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.395+0000 7f47f7e00640 1 -- 172.21.15.39:0/4246549370 learned_addr learned my addr 172.21.15.39:0/4246549370 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:59.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.395+0000 7f47fea00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5569956543b0 0x556995a793a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:59.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.396+0000 7f47ff400640 1 --2- 172.21.15.39:0/4246549370 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55699567d270 0x556995a7f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:59.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.396+0000 7f47f7e00640 1 -- 172.21.15.39:0/4246549370 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55699567d270 msgr2=0x556995a7f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:48:59.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.396+0000 7f47f7e00640 1 --2- 172.21.15.39:0/4246549370 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55699567d270 0x556995a7f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:59.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.396+0000 7f47f7e00640 1 -- 172.21.15.39:0/4246549370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5569956543b0 msgr2=0x556995a793a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:48:59.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.396+0000 7f47f7e00640 1 --2- 172.21.15.39:0/4246549370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5569956543b0 0x556995a793a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:59.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.396+0000 7f47f7e00640 1 -- 172.21.15.39:0/4246549370 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569956e8850 con 0x55699565c590 2024-09-21T07:48:59.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.396+0000 7f47ff400640 1 --2- 172.21.15.39:0/4246549370 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55699567d270 0x556995a7f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:48:59.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.396+0000 7f47f7e00640 1 --2- 172.21.15.39:0/4246549370 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55699565c590 0x556995a656a0 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f47e804b040 tx=0x7f47e80056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f70b18f316fd5ec8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:59.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.396+0000 7f47e3e00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:48:59.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.397+0000 7f47e3e00640 1 -- 172.21.15.39:0/4246549370 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e8038040 con 0x55699565c590 2024-09-21T07:48:59.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.397+0000 7f480c8d8740 1 -- 172.21.15.39:0/4246549370 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47e8045040 con 0x55699565c590 2024-09-21T07:48:59.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.397+0000 7f47e3e00640 1 -- 172.21.15.39:0/4246549370 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(221..221 src has 1..221) ==== 7075+0+0 (secure 0 0 0) 0x7f47e8003c00 con 0x55699565c590 2024-09-21T07:48:59.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.397+0000 7f47e3e00640 1 -- 172.21.15.39:0/4246549370 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=229) ==== 24+0+0 (secure 0 0 0) 0x7f47e805e050 con 0x55699565c590 2024-09-21T07:48:59.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.397+0000 7f480c8d8740 10 client.10383 fetch_fsmap learned FSMap version 229 2024-09-21T07:48:59.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.397+0000 7f480c8d8740 1 -- 172.21.15.39:0/4246549370 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=229,monmap=2+}) -- 0x55699575f6b0 con 0x55699565c590 2024-09-21T07:48:59.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.397+0000 7f47e3e00640 1 -- 172.21.15.39:0/4246549370 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47e8045040 con 0x55699565c590 2024-09-21T07:48:59.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.397+0000 7f47e3e00640 1 -- 172.21.15.39:0/4246549370 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 229) ==== 3502+0+0 (secure 0 0 0) 0x7f47e8061020 con 0x55699565c590 2024-09-21T07:48:59.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.397+0000 7f480c8d8740 10 client.10383 fetch_fsmap finished waiting for FSMap version 229 2024-09-21T07:48:59.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.397+0000 7f480c8d8740 10 client.10383 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:48:59.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.397+0000 7f480c8d8740 20 client.10383 populate_metadata read hostname 'smithi039' 2024-09-21T07:48:59.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.397+0000 7f480c8d8740 1 --2- 172.21.15.39:0/4246549370 >> [v2:172.21.15.39:6832/2837221709,v1:172.21.15.39:6834/2837221709] conn(0x55699563be40 0x556995a7c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:59.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.397+0000 7f480c8d8740 4 client.10383 mds_command: new command op to 10224 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:48:59.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.397+0000 7f480c8d8740 1 -- 172.21.15.39:0/4246549370 --> [v2:172.21.15.39:6832/2837221709,v1:172.21.15.39:6834/2837221709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556995999520 con 0x55699563be40 2024-09-21T07:48:59.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.398+0000 7f47fea00640 1 --2- 172.21.15.39:0/4246549370 >> [v2:172.21.15.39:6832/2837221709,v1:172.21.15.39:6834/2837221709] conn(0x55699563be40 0x556995a7c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:59.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.398+0000 7f47fea00640 1 --2- 172.21.15.39:0/4246549370 >> [v2:172.21.15.39:6832/2837221709,v1:172.21.15.39:6834/2837221709] conn(0x55699563be40 0x556995a7c030 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=925243b1e46e0916 server_cookie=842f499e685974cd in_seq=0 out_seq=0 2024-09-21T07:48:59.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.398+0000 7f47e3e00640 10 client.10383 ms_handle_connect on v2:172.21.15.39:6832/2837221709 2024-09-21T07:48:59.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.399+0000 7f47e3e00640 1 -- 172.21.15.39:0/4246549370 <== mds.0 v2:172.21.15.39:6832/2837221709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556995999520 con 0x55699563be40 2024-09-21T07:48:59.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.399+0000 7f47e3e00640 10 client.10383 handle_command_reply: tid=0 2024-09-21T07:48:59.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.399+0000 7f480c8d8740 1 client.10383 shutdown 2024-09-21T07:48:59.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.399+0000 7f480c8d8740 1 -- 172.21.15.39:0/4246549370 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55699565c590 msgr2=0x556995a656a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:59.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.399+0000 7f480c8d8740 1 --2- 172.21.15.39:0/4246549370 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55699565c590 0x556995a656a0 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f47e804b040 tx=0x7f47e80056f0 comp rx=0 tx=0).stop 2024-09-21T07:48:59.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.399+0000 7f480c8d8740 1 -- 172.21.15.39:0/4246549370 shutdown_connections 2024-09-21T07:48:59.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.399+0000 7f480c8d8740 1 --2- 172.21.15.39:0/4246549370 >> [v2:172.21.15.39:6832/2837221709,v1:172.21.15.39:6834/2837221709] conn(0x55699563be40 0x556995a7c030 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:59.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.399+0000 7f480c8d8740 1 --2- 172.21.15.39:0/4246549370 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55699567d270 0x556995a7f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:59.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.399+0000 7f480c8d8740 1 --2- 172.21.15.39:0/4246549370 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55699565c590 0x556995a656a0 unknown :-1 s=CLOSED pgs=2122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:59.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.399+0000 7f480c8d8740 1 --2- 172.21.15.39:0/4246549370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5569956543b0 0x556995a793a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:59.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.399+0000 7f47e3e00640 0 client.10383 ms_handle_reset on v2:172.21.15.39:6832/2837221709 2024-09-21T07:48:59.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.399+0000 7f480c8d8740 1 -- 172.21.15.39:0/4246549370 >> 172.21.15.39:0/4246549370 conn(0x5569956713c0 msgr2=0x556995a4a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:59.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.400+0000 7f480c8d8740 1 -- 172.21.15.39:0/4246549370 shutdown_connections 2024-09-21T07:48:59.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.400+0000 7f480c8d8740 1 -- 172.21.15.39:0/4246549370 wait complete. 2024-09-21T07:48:59.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.400+0000 7f480c8d8740 20 client.10383 trim_cache size 0 max 16384 2024-09-21T07:48:59.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.419+0000 7f480c8d8740 1 Processor -- start 2024-09-21T07:48:59.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.419+0000 7f480c8d8740 1 -- start start 2024-09-21T07:48:59.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.421+0000 7f480c8d8740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5569956543b0 0x556995a4f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:59.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.421+0000 7f480c8d8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55699565c590 0x556995ae5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:59.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.421+0000 7f480c8d8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55699567d270 0x556995a7af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:59.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.421+0000 7f480c8d8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f48001fff20 con 0x55699567d270 2024-09-21T07:48:59.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.421+0000 7f480c8d8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f48000cffb0 con 0x55699565c590 2024-09-21T07:48:59.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.421+0000 7f480c8d8740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f48001f4650 con 0x5569956543b0 2024-09-21T07:48:59.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.421+0000 7f47ff400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55699567d270 0x556995a7af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:59.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.421+0000 7f47ff400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55699567d270 0x556995a7af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48938/0 (socket says 172.21.15.39:48938) 2024-09-21T07:48:59.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.421+0000 7f47ff400640 1 -- 172.21.15.39:0/1292539349 learned_addr learned my addr 172.21.15.39:0/1292539349 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:59.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.421+0000 7f47fea00640 1 --2- 172.21.15.39:0/1292539349 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5569956543b0 0x556995a4f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:59.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.421+0000 7f47f7e00640 1 --2- 172.21.15.39:0/1292539349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55699565c590 0x556995ae5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:59.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.421+0000 7f47ff400640 1 -- 172.21.15.39:0/1292539349 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5569956543b0 msgr2=0x556995a4f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:48:59.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.421+0000 7f47ff400640 1 --2- 172.21.15.39:0/1292539349 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5569956543b0 0x556995a4f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:59.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.421+0000 7f47ff400640 1 -- 172.21.15.39:0/1292539349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55699565c590 msgr2=0x556995ae5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:48:59.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.421+0000 7f47ff400640 1 --2- 172.21.15.39:0/1292539349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55699565c590 0x556995ae5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:59.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.422+0000 7f47ff400640 1 -- 172.21.15.39:0/1292539349 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556995a24260 con 0x55699567d270 2024-09-21T07:48:59.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.422+0000 7f47fea00640 1 --2- 172.21.15.39:0/1292539349 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5569956543b0 0x556995a4f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:48:59.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.422+0000 7f47ff400640 1 --2- 172.21.15.39:0/1292539349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55699567d270 0x556995a7af90 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f48000cdd10 tx=0x7f47f8006b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38eef503bc749cdb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:59.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.422+0000 7f47eea00640 1 -- 172.21.15.39:0/1292539349 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f8006570 con 0x55699567d270 2024-09-21T07:48:59.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.422+0000 7f47eea00640 1 -- 172.21.15.39:0/1292539349 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47f8006730 con 0x55699567d270 2024-09-21T07:48:59.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.422+0000 7f47eea00640 1 -- 172.21.15.39:0/1292539349 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f8006980 con 0x55699567d270 2024-09-21T07:48:59.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.423+0000 7f480c8d8740 1 -- 172.21.15.39:0/1292539349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55699567d270 msgr2=0x556995a7af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:59.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.423+0000 7f480c8d8740 1 --2- 172.21.15.39:0/1292539349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55699567d270 0x556995a7af90 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f48000cdd10 tx=0x7f47f8006b20 comp rx=0 tx=0).stop 2024-09-21T07:48:59.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.423+0000 7f480c8d8740 1 -- 172.21.15.39:0/1292539349 shutdown_connections 2024-09-21T07:48:59.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.423+0000 7f480c8d8740 1 --2- 172.21.15.39:0/1292539349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55699567d270 0x556995a7af90 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:59.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.423+0000 7f480c8d8740 1 --2- 172.21.15.39:0/1292539349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55699565c590 0x556995ae5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:59.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.423+0000 7f480c8d8740 1 --2- 172.21.15.39:0/1292539349 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5569956543b0 0x556995a4f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:59.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.423+0000 7f480c8d8740 1 -- 172.21.15.39:0/1292539349 >> 172.21.15.39:0/1292539349 conn(0x5569956713c0 msgr2=0x556995a1f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:59.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.423+0000 7f480c8d8740 1 -- 172.21.15.39:0/1292539349 shutdown_connections 2024-09-21T07:48:59.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.423+0000 7f480c8d8740 1 -- 172.21.15.39:0/1292539349 wait complete. 2024-09-21T07:48:59.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.424+0000 7f480c8d8740 1 -- start start 2024-09-21T07:48:59.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.424+0000 7f480c8d8740 1 Processor -- start 2024-09-21T07:48:59.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.426+0000 7f480c8d8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5569956543b0 0x556995ae24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:59.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.426+0000 7f480c8d8740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55699565c590 0x556995a7ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:59.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.426+0000 7f480c8d8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55699567d270 0x556995a7efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:59.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.426+0000 7f480c8d8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556995a6b2e0 con 0x55699567d270 2024-09-21T07:48:59.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.426+0000 7f480c8d8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55699592d330 con 0x5569956543b0 2024-09-21T07:48:59.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.426+0000 7f480c8d8740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x556995a62d40 con 0x55699565c590 2024-09-21T07:48:59.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.426+0000 7f480c8d8740 10 client.0 _ll_register_callbacks cb 0x7f47f005fc30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:48:59.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.426+0000 7f47ff400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55699567d270 0x556995a7efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:59.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.426+0000 7f47ff400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55699567d270 0x556995a7efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48954/0 (socket says 172.21.15.39:48954) 2024-09-21T07:48:59.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.426+0000 7f47ff400640 1 -- 172.21.15.39:0/3162979038 learned_addr learned my addr 172.21.15.39:0/3162979038 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:59.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.426+0000 7f47f7e00640 1 --2- 172.21.15.39:0/3162979038 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55699565c590 0x556995a7ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:59.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.426+0000 7f47fea00640 1 --2- 172.21.15.39:0/3162979038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5569956543b0 0x556995ae24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:59.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.426+0000 7f47ff400640 1 -- 172.21.15.39:0/3162979038 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55699565c590 msgr2=0x556995a7ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:48:59.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.426+0000 7f47ff400640 1 --2- 172.21.15.39:0/3162979038 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55699565c590 0x556995a7ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:59.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.426+0000 7f47ff400640 1 -- 172.21.15.39:0/3162979038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5569956543b0 msgr2=0x556995ae24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:48:59.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.427+0000 7f47ff400640 1 --2- 172.21.15.39:0/3162979038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5569956543b0 0x556995ae24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:59.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.427+0000 7f47ff400640 1 -- 172.21.15.39:0/3162979038 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556995a81730 con 0x55699567d270 2024-09-21T07:48:59.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.427+0000 7f47ff400640 1 --2- 172.21.15.39:0/3162979038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55699567d270 0x556995a7efd0 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f4800072ec0 tx=0x7f47f8012b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4deb000246f05d19 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:59.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.427+0000 7f47e3e00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:48:59.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.427+0000 7f47e3e00640 1 -- 172.21.15.39:0/3162979038 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f802a060 con 0x55699567d270 2024-09-21T07:48:59.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.427+0000 7f47e3e00640 1 -- 172.21.15.39:0/3162979038 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(221..221 src has 1..221) ==== 7075+0+0 (secure 0 0 0) 0x7f47f8003e00 con 0x55699567d270 2024-09-21T07:48:59.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.427+0000 7f480c8d8740 1 -- 172.21.15.39:0/3162979038 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47f8006730 con 0x55699567d270 2024-09-21T07:48:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.427+0000 7f47e3e00640 1 -- 172.21.15.39:0/3162979038 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=229) ==== 24+0+0 (secure 0 0 0) 0x7f47f8030050 con 0x55699567d270 2024-09-21T07:48:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.427+0000 7f480c8d8740 10 client.10389 fetch_fsmap learned FSMap version 229 2024-09-21T07:48:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.427+0000 7f480c8d8740 1 -- 172.21.15.39:0/3162979038 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=229,monmap=2+}) -- 0x556995ae31d0 con 0x55699567d270 2024-09-21T07:48:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.428+0000 7f47e3e00640 1 -- 172.21.15.39:0/3162979038 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47f8006730 con 0x55699567d270 2024-09-21T07:48:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.428+0000 7f47e3e00640 1 -- 172.21.15.39:0/3162979038 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 229) ==== 3502+0+0 (secure 0 0 0) 0x7f47f8033020 con 0x55699567d270 2024-09-21T07:48:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.428+0000 7f480c8d8740 10 client.10389 fetch_fsmap finished waiting for FSMap version 229 2024-09-21T07:48:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.428+0000 7f480c8d8740 10 client.10389 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:48:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.428+0000 7f480c8d8740 20 client.10389 populate_metadata read hostname 'smithi039' 2024-09-21T07:48:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.428+0000 7f480c8d8740 1 --2- 172.21.15.39:0/3162979038 >> [v2:172.21.15.39:6832/2837221709,v1:172.21.15.39:6834/2837221709] conn(0x556995a8ead0 0x556995a8eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.428+0000 7f480c8d8740 4 client.10389 mds_command: new command op to 10224 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:48:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.428+0000 7f480c8d8740 1 -- 172.21.15.39:0/3162979038 --> [v2:172.21.15.39:6832/2837221709,v1:172.21.15.39:6834/2837221709] -- command(tid 0: {"prefix": "status"}) -- 0x556995a45ab0 con 0x556995a8ead0 2024-09-21T07:48:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.428+0000 7f47fea00640 1 --2- 172.21.15.39:0/3162979038 >> [v2:172.21.15.39:6832/2837221709,v1:172.21.15.39:6834/2837221709] conn(0x556995a8ead0 0x556995a8eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:59.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.429+0000 7f47fea00640 1 --2- 172.21.15.39:0/3162979038 >> [v2:172.21.15.39:6832/2837221709,v1:172.21.15.39:6834/2837221709] conn(0x556995a8ead0 0x556995a8eee0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e38498ca9e9e16a server_cookie=a7e19134f484d1b7 in_seq=0 out_seq=0 2024-09-21T07:48:59.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.429+0000 7f47e3e00640 10 client.10389 ms_handle_connect on v2:172.21.15.39:6832/2837221709 2024-09-21T07:48:59.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.429+0000 7f47e3e00640 1 -- 172.21.15.39:0/3162979038 <== mds.0 v2:172.21.15.39:6832/2837221709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+329 (crc 0 0 0) 0x556995a45ab0 con 0x556995a8ead0 2024-09-21T07:48:59.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.429+0000 7f47e3e00640 10 client.10389 handle_command_reply: tid=0 2024-09-21T07:48:59.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.429+0000 7f480c8d8740 1 client.10389 shutdown 2024-09-21T07:48:59.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.429+0000 7f480c8d8740 1 -- 172.21.15.39:0/3162979038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55699567d270 msgr2=0x556995a7efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:59.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.429+0000 7f480c8d8740 1 --2- 172.21.15.39:0/3162979038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55699567d270 0x556995a7efd0 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f4800072ec0 tx=0x7f47f8012b90 comp rx=0 tx=0).stop 2024-09-21T07:48:59.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.429+0000 7f480c8d8740 1 -- 172.21.15.39:0/3162979038 shutdown_connections 2024-09-21T07:48:59.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.429+0000 7f480c8d8740 1 --2- 172.21.15.39:0/3162979038 >> [v2:172.21.15.39:6832/2837221709,v1:172.21.15.39:6834/2837221709] conn(0x556995a8ead0 0x556995a8eee0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:59.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.429+0000 7f480c8d8740 1 --2- 172.21.15.39:0/3162979038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55699567d270 0x556995a7efd0 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:59.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.429+0000 7f480c8d8740 1 --2- 172.21.15.39:0/3162979038 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55699565c590 0x556995a7ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:59.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.429+0000 7f480c8d8740 1 --2- 172.21.15.39:0/3162979038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5569956543b0 0x556995ae24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:59.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.429+0000 7f47e3e00640 0 client.10389 ms_handle_reset on v2:172.21.15.39:6832/2837221709 2024-09-21T07:48:59.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.429+0000 7f480c8d8740 1 -- 172.21.15.39:0/3162979038 >> 172.21.15.39:0/3162979038 conn(0x5569956713c0 msgr2=0x556995a7af90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:59.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.429+0000 7f480c8d8740 1 -- 172.21.15.39:0/3162979038 shutdown_connections 2024-09-21T07:48:59.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.429+0000 7f480c8d8740 1 -- 172.21.15.39:0/3162979038 wait complete. 2024-09-21T07:48:59.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.430+0000 7f480c8d8740 20 client.10389 trim_cache size 0 max 16384 2024-09-21T07:48:59.435 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:48:59.435 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:48:59.435 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-09-21T07:48:59.435 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 10224, 2024-09-21T07:48:59.435 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:48:59.435 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:48:59.435 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:48:59.435 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 11.844103204, 2024-09-21T07:48:59.436 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 229, 2024-09-21T07:48:59.436 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 221, 2024-09-21T07:48:59.436 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 219, 2024-09-21T07:48:59.436 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 16.908461733999999 2024-09-21T07:48:59.436 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:48:59.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.430+0000 7f4805800640 1 -- 172.21.15.39:0/1316194144 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f47dc078b90 msgr2=0x7f47dc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.430+0000 7f4805800640 1 --2- 172.21.15.39:0/1316194144 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f47dc078b90 0x7f47dc07b0a0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f47e8035000 tx=0x7f47e80046c0 comp rx=0 tx=0).stop 2024-09-21T07:48:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.430+0000 7f4805800640 1 -- 172.21.15.39:0/1316194144 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47e4003fd0 msgr2=0x7f48000c6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.430+0000 7f4805800640 1 --2- 172.21.15.39:0/1316194144 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47e4003fd0 0x7f48000c6330 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f47f0011ae0 tx=0x7f47f0009e90 comp rx=0 tx=0).stop 2024-09-21T07:48:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.430+0000 7f4805800640 1 -- 172.21.15.39:0/1316194144 shutdown_connections 2024-09-21T07:48:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.430+0000 7f4805800640 1 --2- 172.21.15.39:0/1316194144 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f47dc078b90 0x7f47dc07b0a0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.430+0000 7f4805800640 1 --2- 172.21.15.39:0/1316194144 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f48001f7470 0x7f48001f4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.430+0000 7f4805800640 1 --2- 172.21.15.39:0/1316194144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4800155e30 0x7f48001f6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.430+0000 7f4805800640 1 --2- 172.21.15.39:0/1316194144 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47e4003fd0 0x7f48000c6330 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.430+0000 7f4805800640 1 -- 172.21.15.39:0/1316194144 >> 172.21.15.39:0/1316194144 conn(0x7f48000b2460 msgr2=0x7f48000d0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.430+0000 7f4805800640 1 -- 172.21.15.39:0/1316194144 shutdown_connections 2024-09-21T07:48:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.430+0000 7f4805800640 1 -- 172.21.15.39:0/1316194144 wait complete. 2024-09-21T07:48:59.454 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:48:59.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.680+0000 7f06b2000640 1 -- 172.21.15.39:0/2412459595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06ac15bce0 msgr2=0x7f06ac0d0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:59.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.680+0000 7f06b2000640 1 --2- 172.21.15.39:0/2412459595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06ac15bce0 0x7f06ac0d0540 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f06a0009830 tx=0x7f06a002a5c0 comp rx=0 tx=0).stop 2024-09-21T07:48:59.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.680+0000 7f06b2000640 1 -- 172.21.15.39:0/2412459595 shutdown_connections 2024-09-21T07:48:59.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.680+0000 7f06b2000640 1 --2- 172.21.15.39:0/2412459595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06ac15bce0 0x7f06ac0d0540 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:59.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.680+0000 7f06b2000640 1 -- 172.21.15.39:0/2412459595 >> 172.21.15.39:0/2412459595 conn(0x7f06ac0d0cb0 msgr2=0x7f06ac0d10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:59.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.680+0000 7f06b2000640 1 -- 172.21.15.39:0/2412459595 shutdown_connections 2024-09-21T07:48:59.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.680+0000 7f06b2000640 1 -- 172.21.15.39:0/2412459595 wait complete. 2024-09-21T07:48:59.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.682+0000 7f06b2000640 1 Processor -- start 2024-09-21T07:48:59.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.682+0000 7f06b2000640 1 -- start start 2024-09-21T07:48:59.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.684+0000 7f06b2000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06ac15bce0 0x7f06ac14f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:59.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.684+0000 7f06b2000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06ac15f2d0 0x7f06ac15e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:59.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.684+0000 7f06b2000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f06ac15e5f0 0x7f06ac14a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:59.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.684+0000 7f06b2000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f06ac070450 con 0x7f06ac15bce0 2024-09-21T07:48:59.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.684+0000 7f06b2000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f06ac0702d0 con 0x7f06ac15f2d0 2024-09-21T07:48:59.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.684+0000 7f06b2000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f06ac0705d0 con 0x7f06ac15e5f0 2024-09-21T07:48:59.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.684+0000 7f06b0c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06ac15bce0 0x7f06ac14f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:59.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.684+0000 7f06b0c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06ac15bce0 0x7f06ac14f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48976/0 (socket says 172.21.15.39:48976) 2024-09-21T07:48:59.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.684+0000 7f06b0c00640 1 -- 172.21.15.39:0/110088016 learned_addr learned my addr 172.21.15.39:0/110088016 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:48:59.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.684+0000 7f06b1600640 1 --2- 172.21.15.39:0/110088016 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f06ac15e5f0 0x7f06ac14a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:59.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.684+0000 7f06abe00640 1 --2- 172.21.15.39:0/110088016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06ac15f2d0 0x7f06ac15e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:59.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.684+0000 7f06b0c00640 1 -- 172.21.15.39:0/110088016 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f06ac15e5f0 msgr2=0x7f06ac14a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:59.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.684+0000 7f06b0c00640 1 --2- 172.21.15.39:0/110088016 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f06ac15e5f0 0x7f06ac14a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:59.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.684+0000 7f06b0c00640 1 -- 172.21.15.39:0/110088016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06ac15f2d0 msgr2=0x7f06ac15e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:59.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.684+0000 7f06b0c00640 1 --2- 172.21.15.39:0/110088016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06ac15f2d0 0x7f06ac15e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:59.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.684+0000 7f06b0c00640 1 -- 172.21.15.39:0/110088016 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06ac1f9910 con 0x7f06ac15bce0 2024-09-21T07:48:59.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.685+0000 7f06b0c00640 1 --2- 172.21.15.39:0/110088016 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06ac15bce0 0x7f06ac14f850 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f06a0000c00 tx=0x7f06a0033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:59.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.685+0000 7f06a8c00640 1 -- 172.21.15.39:0/110088016 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a0031d60 con 0x7f06ac15bce0 2024-09-21T07:48:59.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.685+0000 7f06a8c00640 1 -- 172.21.15.39:0/110088016 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06a00330d0 con 0x7f06ac15bce0 2024-09-21T07:48:59.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.685+0000 7f06b2000640 1 -- 172.21.15.39:0/110088016 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06ac153040 con 0x7f06ac15bce0 2024-09-21T07:48:59.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.685+0000 7f06a8c00640 1 -- 172.21.15.39:0/110088016 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a0033520 con 0x7f06ac15bce0 2024-09-21T07:48:59.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.685+0000 7f06b2000640 1 -- 172.21.15.39:0/110088016 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06ac150380 con 0x7f06ac15bce0 2024-09-21T07:48:59.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.686+0000 7f06b2000640 1 -- 172.21.15.39:0/110088016 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06ac070a80 con 0x7f06ac15bce0 2024-09-21T07:48:59.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.688+0000 7f06a8c00640 1 -- 172.21.15.39:0/110088016 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f06a00035f0 con 0x7f06ac15bce0 2024-09-21T07:48:59.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.689+0000 7f06a8c00640 1 --2- 172.21.15.39:0/110088016 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f068c078bc0 0x7f068c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:59.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.689+0000 7f06a8c00640 1 -- 172.21.15.39:0/110088016 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(221..221 src has 1..221) ==== 7075+0+0 (secure 0 0 0) 0x7f06a00bc740 con 0x7f06ac15bce0 2024-09-21T07:48:59.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.689+0000 7f06abe00640 1 --2- 172.21.15.39:0/110088016 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f068c078bc0 0x7f068c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:59.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.690+0000 7f06abe00640 1 --2- 172.21.15.39:0/110088016 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f068c078bc0 0x7f068c07b0d0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f06ac1551d0 tx=0x7f0698007eb0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:59.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.693+0000 7f06a8c00640 1 -- 172.21.15.39:0/110088016 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f06a003f030 con 0x7f06ac15bce0 2024-09-21T07:48:59.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.931+0000 7f06b2000640 1 -- 172.21.15.39:0/110088016 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f06ac154d90 con 0x7f06ac15bce0 2024-09-21T07:48:59.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.932+0000 7f06a8c00640 1 -- 172.21.15.39:0/110088016 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v229) ==== 53+0+73 (secure 0 0 0) 0x7f06a0083fc0 con 0x7f06ac15bce0 2024-09-21T07:48:59.934 INFO:teuthology.orchestra.run.smithi039.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:48:59.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.934+0000 7f06b2000640 1 -- 172.21.15.39:0/110088016 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f068c078bc0 msgr2=0x7f068c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:59.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.934+0000 7f06b2000640 1 --2- 172.21.15.39:0/110088016 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f068c078bc0 0x7f068c07b0d0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f06ac1551d0 tx=0x7f0698007eb0 comp rx=0 tx=0).stop 2024-09-21T07:48:59.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.934+0000 7f06b2000640 1 -- 172.21.15.39:0/110088016 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06ac15bce0 msgr2=0x7f06ac14f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:59.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.934+0000 7f06b2000640 1 --2- 172.21.15.39:0/110088016 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06ac15bce0 0x7f06ac14f850 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f06a0000c00 tx=0x7f06a0033d60 comp rx=0 tx=0).stop 2024-09-21T07:48:59.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.934+0000 7f06b2000640 1 -- 172.21.15.39:0/110088016 shutdown_connections 2024-09-21T07:48:59.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.934+0000 7f06b2000640 1 --2- 172.21.15.39:0/110088016 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f068c078bc0 0x7f068c07b0d0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:59.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.934+0000 7f06b2000640 1 --2- 172.21.15.39:0/110088016 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f06ac15e5f0 0x7f06ac14a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:59.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.934+0000 7f06b2000640 1 --2- 172.21.15.39:0/110088016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06ac15f2d0 0x7f06ac15e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:59.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.934+0000 7f06b2000640 1 --2- 172.21.15.39:0/110088016 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06ac15bce0 0x7f06ac14f850 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:59.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.934+0000 7f06b2000640 1 -- 172.21.15.39:0/110088016 >> 172.21.15.39:0/110088016 conn(0x7f06ac0d0cb0 msgr2=0x7f06ac069120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:59.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.934+0000 7f06b2000640 1 -- 172.21.15.39:0/110088016 shutdown_connections 2024-09-21T07:48:59.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:48:59.934+0000 7f06b2000640 1 -- 172.21.15.39:0/110088016 wait complete. 2024-09-21T07:48:59.958 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:48:59.958 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:48:59.958 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi039.front.sepia.ceph.com 2024-09-21T07:48:59.958 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T07:48:59.958 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:48:59.958 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:48:59.959 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:48:59.959 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:48:59.959 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:48:59.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:48:59.959 DEBUG:teuthology.orchestra.run.smithi039:> ip addr 2024-09-21T07:48:59.995 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:48:59.995 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:48:59.995 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:48:59.995 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:48:59.995 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2024-09-21T07:48:59.995 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:48:59.995 INFO:teuthology.orchestra.run.smithi039.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:48:59.996 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:48:59.996 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f0 2024-09-21T07:48:59.996 INFO:teuthology.orchestra.run.smithi039.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:48:59.996 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c7 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:48:59.996 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f1 2024-09-21T07:48:59.996 INFO:teuthology.orchestra.run.smithi039.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:48:59.996 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:70 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:48:59.996 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f0 2024-09-21T07:48:59.996 INFO:teuthology.orchestra.run.smithi039.stdout: inet 172.21.15.39/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T07:48:59.996 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft 41779sec preferred_lft 41779sec 2024-09-21T07:48:59.996 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec4:7aff:febd:1370/64 scope link 2024-09-21T07:48:59.996 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:48:59.996 INFO:teuthology.orchestra.run.smithi039.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:48:59.996 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:71 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:48:59.997 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f1 2024-09-21T07:48:59.997 INFO:teuthology.orchestra.run.smithi039.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:48:59.997 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff 2024-09-21T07:48:59.997 INFO:teuthology.orchestra.run.smithi039.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:48:59.997 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:48:59.997 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::b079:ddff:fe0e:6333/64 scope link 2024-09-21T07:48:59.997 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:48:59.997 INFO:teuthology.orchestra.run.smithi039.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:48:59.997 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:48:59.997 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::2ca7:98ff:fe8a:d0ed/64 scope link 2024-09-21T07:48:59.997 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:48:59.997 INFO:teuthology.orchestra.run.smithi039.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:48:59.997 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether ee:10:53:01:0c:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:48:59.997 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec10:53ff:fe01:c44/64 scope link 2024-09-21T07:48:59.998 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:48:59.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:48:59.998 DEBUG:teuthology.orchestra.run.smithi039:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:49:00.089 INFO:teuthology.orchestra.run.smithi039.stdout:1 2024-09-21T07:49:00.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:00.093 DEBUG:teuthology.orchestra.run.smithi039:> route 2024-09-21T07:49:00.129 INFO:teuthology.orchestra.run.smithi039.stdout:Kernel IP routing table 2024-09-21T07:49:00.129 INFO:teuthology.orchestra.run.smithi039.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:49:00.129 INFO:teuthology.orchestra.run.smithi039.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T07:49:00.129 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T07:49:00.129 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:49:00.129 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:49:00.129 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-21T07:49:00.130 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T07:49:00.130 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T07:49:00.130 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T07:49:00.130 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-21T07:49:00.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:49:00.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:49:00.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:00.327 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-09-21T07:49:00.363 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:49:00.363 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T07:49:00.363 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:49:00.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:00.363 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-09-21T07:49:00.459 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:49:00.459 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T07:49:00.494 INFO:teuthology.orchestra.run.smithi039.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:49:00.494 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:49:00.494 DEBUG:teuthology.orchestra.run.smithi039:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:49:00.569 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-21T07:49:00.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:00.570 DEBUG:teuthology.orchestra.run.smithi039:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:49:00.768 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:49:00.768 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:49:00.805 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:49:00.805 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:49:00.805 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: resolved to: "172.21.15.39:3300,172.21.15.148:3300,172.21.15.148:3301" 2024-09-21T07:49:00.805 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: trying mount with new device syntax: 0@7ab9b04a-f88d-49aa-a493-e5c076637ffb.cephfs=/ 2024-09-21T07:49:00.805 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.39:3300/172.21.15.148:3300/172.21.15.148:3301" will pass to kernel 2024-09-21T07:49:00.806 INFO:teuthology.orchestra.run.smithi039.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T07:49:00.807 INFO:teuthology.orchestra.run.smithi039.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:49:00.807 INFO:teuthology.orchestra.run.smithi039.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:49:00.807 INFO:teuthology.orchestra.run.smithi039.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:49:00.807 INFO:teuthology.orchestra.run.smithi039.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:49:00.810 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:49:00.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:00.810 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:49:00.866 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:49:00.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:49:00.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:49:00.964 INFO:teuthology.orchestra.run.smithi039.stdout:client9517 2024-09-21T07:49:00.964 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:49:00.964 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:49:01.000 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get mds_max_snaps_per_dir 2024-09-21T07:49:01.283 INFO:teuthology.orchestra.run.smithi039.stdout:{"mds_max_snaps_per_dir":"100"} 2024-09-21T07:49:01.283 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_max_snaps_per_dir": "100" } 2024-09-21T07:49:01.283 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2024-09-21T07:49:01.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:01.513+0000 7fa6bf800640 1 -- 172.21.15.39:0/736517191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6b80cd400 msgr2=0x7fa6b81677a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:01.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:01.513+0000 7fa6bf800640 1 --2- 172.21.15.39:0/736517191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6b80cd400 0x7fa6b81677a0 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0005fd0 tx=0x7fa6b002a820 comp rx=0 tx=0).stop 2024-09-21T07:49:01.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:01.513+0000 7fa6bf800640 1 -- 172.21.15.39:0/736517191 shutdown_connections 2024-09-21T07:49:01.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:01.513+0000 7fa6bf800640 1 --2- 172.21.15.39:0/736517191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6b80cd400 0x7fa6b81677a0 unknown :-1 s=CLOSED pgs=2129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:01.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:01.513+0000 7fa6bf800640 1 -- 172.21.15.39:0/736517191 >> 172.21.15.39:0/736517191 conn(0x7fa6b80b2650 msgr2=0x7fa6b80d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:01.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:01.513+0000 7fa6bf800640 1 -- 172.21.15.39:0/736517191 shutdown_connections 2024-09-21T07:49:01.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:01.513+0000 7fa6bf800640 1 -- 172.21.15.39:0/736517191 wait complete. 2024-09-21T07:49:01.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:01.516+0000 7fa6bf800640 1 Processor -- start 2024-09-21T07:49:01.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:01.516+0000 7fa6bf800640 1 -- start start 2024-09-21T07:49:01.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:01.517+0000 7fa6bf800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6b815a440 0x7fa6b8099a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:01.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:01.517+0000 7fa6bf800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6b815da30 0x7fa6b80ad170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:01.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:01.517+0000 7fa6bf800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6b80cf6c0 0x7fa6b8070180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:01.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:01.517+0000 7fa6bf800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b8168070 con 0x7fa6b815da30 2024-09-21T07:49:01.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:01.517+0000 7fa6bf800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b8167ef0 con 0x7fa6b815a440 2024-09-21T07:49:01.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:01.517+0000 7fa6bf800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa6b81681f0 con 0x7fa6b80cf6c0 2024-09-21T07:49:01.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:01.517+0000 7fa6afe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6b815da30 0x7fa6b80ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:01.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:01.518+0000 7fa6afe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6b815da30 0x7fa6b80ad170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48982/0 (socket says 172.21.15.39:48982) 2024-09-21T07:49:01.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:01.518+0000 7fa6afe00640 1 -- 172.21.15.39:0/1780825118 learned_addr learned my addr 172.21.15.39:0/1780825118 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:01.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:01.518+0000 7fa6bce00640 1 --2- 172.21.15.39:0/1780825118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6b815a440 0x7fa6b8099a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:01.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:01.518+0000 7fa6bd800640 1 --2- 172.21.15.39:0/1780825118 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6b80cf6c0 0x7fa6b8070180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:01.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:01.518+0000 7fa6afe00640 1 -- 172.21.15.39:0/1780825118 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6b80cf6c0 msgr2=0x7fa6b8070180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:01.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:01.518+0000 7fa6afe00640 1 --2- 172.21.15.39:0/1780825118 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6b80cf6c0 0x7fa6b8070180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:01.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:01.518+0000 7fa6afe00640 1 -- 172.21.15.39:0/1780825118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6b815a440 msgr2=0x7fa6b8099a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:01.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:01.518+0000 7fa6afe00640 1 --2- 172.21.15.39:0/1780825118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6b815a440 0x7fa6b8099a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:01.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:01.518+0000 7fa6afe00640 1 -- 172.21.15.39:0/1780825118 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b80ad6b0 con 0x7fa6b815da30 2024-09-21T07:49:01.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:01.518+0000 7fa6afe00640 1 --2- 172.21.15.39:0/1780825118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6b815da30 0x7fa6b80ad170 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7fa6a000b3f0 tx=0x7fa6a000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:01.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:01.518+0000 7fa6b5600640 1 -- 172.21.15.39:0/1780825118 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0015020 con 0x7fa6b815da30 2024-09-21T07:49:01.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:01.519+0000 7fa6b5600640 1 -- 172.21.15.39:0/1780825118 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6a0011920 con 0x7fa6b815da30 2024-09-21T07:49:01.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:01.519+0000 7fa6bf800640 1 -- 172.21.15.39:0/1780825118 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6b80706c0 con 0x7fa6b815da30 2024-09-21T07:49:01.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:01.519+0000 7fa6b5600640 1 -- 172.21.15.39:0/1780825118 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0011d70 con 0x7fa6b815da30 2024-09-21T07:49:01.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:01.519+0000 7fa6bf800640 1 -- 172.21.15.39:0/1780825118 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6b815b660 con 0x7fa6b815da30 2024-09-21T07:49:01.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:01.520+0000 7fa6bf800640 1 -- 172.21.15.39:0/1780825118 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6b815b450 con 0x7fa6b815da30 2024-09-21T07:49:01.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:01.521+0000 7fa6b5600640 1 -- 172.21.15.39:0/1780825118 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa6a0018020 con 0x7fa6b815da30 2024-09-21T07:49:01.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:01.522+0000 7fa6b5600640 1 --2- 172.21.15.39:0/1780825118 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa694078bc0 0x7fa69407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:01.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:01.522+0000 7fa6b5600640 1 -- 172.21.15.39:0/1780825118 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(221..221 src has 1..221) ==== 7075+0+0 (secure 0 0 0) 0x7fa6a009add0 con 0x7fa6b815da30 2024-09-21T07:49:01.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:01.522+0000 7fa6bce00640 1 --2- 172.21.15.39:0/1780825118 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa694078bc0 0x7fa69407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:01.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:01.523+0000 7fa6bce00640 1 --2- 172.21.15.39:0/1780825118 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa694078bc0 0x7fa69407b0d0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8000f60 tx=0x7fa6a8009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:01.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:01.526+0000 7fa6b5600640 1 -- 172.21.15.39:0/1780825118 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa6a001f030 con 0x7fa6b815da30 2024-09-21T07:49:01.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:01.762+0000 7fa6bf800640 1 -- 172.21.15.39:0/1780825118 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fa6b80cba00 con 0x7fa6b815da30 2024-09-21T07:49:02.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.694+0000 7fa6b5600640 1 -- 172.21.15.39:0/1780825118 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v230) ==== 167+0+0 (secure 0 0 0) 0x7fa6a0062650 con 0x7fa6b815da30 2024-09-21T07:49:02.697 INFO:teuthology.orchestra.run.smithi039.stderr:enabled new snapshots 2024-09-21T07:49:02.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.697+0000 7fa6bf800640 1 -- 172.21.15.39:0/1780825118 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa694078bc0 msgr2=0x7fa69407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:02.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.697+0000 7fa6bf800640 1 --2- 172.21.15.39:0/1780825118 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa694078bc0 0x7fa69407b0d0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8000f60 tx=0x7fa6a8009040 comp rx=0 tx=0).stop 2024-09-21T07:49:02.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.697+0000 7fa6bf800640 1 -- 172.21.15.39:0/1780825118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6b815da30 msgr2=0x7fa6b80ad170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:02.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.697+0000 7fa6bf800640 1 --2- 172.21.15.39:0/1780825118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6b815da30 0x7fa6b80ad170 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7fa6a000b3f0 tx=0x7fa6a000b8b0 comp rx=0 tx=0).stop 2024-09-21T07:49:02.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.698+0000 7fa6bf800640 1 -- 172.21.15.39:0/1780825118 shutdown_connections 2024-09-21T07:49:02.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.698+0000 7fa6bf800640 1 --2- 172.21.15.39:0/1780825118 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa694078bc0 0x7fa69407b0d0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:02.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.698+0000 7fa6bf800640 1 --2- 172.21.15.39:0/1780825118 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa6b80cf6c0 0x7fa6b8070180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:02.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.698+0000 7fa6bf800640 1 --2- 172.21.15.39:0/1780825118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6b815da30 0x7fa6b80ad170 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:02.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.698+0000 7fa6bf800640 1 --2- 172.21.15.39:0/1780825118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa6b815a440 0x7fa6b8099a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:02.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.698+0000 7fa6bf800640 1 -- 172.21.15.39:0/1780825118 >> 172.21.15.39:0/1780825118 conn(0x7fa6b80b2650 msgr2=0x7fa6b80b0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:02.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.698+0000 7fa6bf800640 1 -- 172.21.15.39:0/1780825118 shutdown_connections 2024-09-21T07:49:02.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.698+0000 7fa6bf800640 1 -- 172.21.15.39:0/1780825118 wait complete. 2024-09-21T07:49:02.726 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 2 2024-09-21T07:49:02.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.960+0000 7fa823e00640 1 -- 172.21.15.39:0/239948333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8240d25a0 msgr2=0x7fa8240d29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:02.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.960+0000 7fa823e00640 1 --2- 172.21.15.39:0/239948333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8240d25a0 0x7fa8240d29b0 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7fa8240d44b0 tx=0x7fa81c02a8d0 comp rx=0 tx=0).stop 2024-09-21T07:49:02.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.960+0000 7fa823e00640 1 -- 172.21.15.39:0/239948333 shutdown_connections 2024-09-21T07:49:02.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.960+0000 7fa823e00640 1 --2- 172.21.15.39:0/239948333 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8240b5f60 0x7fa8240b6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:02.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.960+0000 7fa823e00640 1 --2- 172.21.15.39:0/239948333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8240d25a0 0x7fa8240d29b0 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:02.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.960+0000 7fa823e00640 1 -- 172.21.15.39:0/239948333 >> 172.21.15.39:0/239948333 conn(0x7fa82409d7b0 msgr2=0x7fa8240d44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:02.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.960+0000 7fa823e00640 1 -- 172.21.15.39:0/239948333 shutdown_connections 2024-09-21T07:49:02.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.960+0000 7fa823e00640 1 -- 172.21.15.39:0/239948333 wait complete. 2024-09-21T07:49:02.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.962+0000 7fa823e00640 1 Processor -- start 2024-09-21T07:49:02.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.962+0000 7fa823e00640 1 -- start start 2024-09-21T07:49:02.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.964+0000 7fa823e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8240b5f60 0x7fa8241f6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:02.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.964+0000 7fa823e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8240b6c90 0x7fa8241f8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:02.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.964+0000 7fa823e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8240d25a0 0x7fa82415e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:02.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.964+0000 7fa823e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa8240b9e60 con 0x7fa8240d25a0 2024-09-21T07:49:02.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.964+0000 7fa823e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa8240b9a40 con 0x7fa8240b5f60 2024-09-21T07:49:02.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.964+0000 7fa823e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa8240b9fe0 con 0x7fa8240b6c90 2024-09-21T07:49:02.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.964+0000 7fa823400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8240d25a0 0x7fa82415e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:02.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.964+0000 7fa823400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8240d25a0 0x7fa82415e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36000/0 (socket says 172.21.15.39:36000) 2024-09-21T07:49:02.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.964+0000 7fa823400640 1 -- 172.21.15.39:0/4054774720 learned_addr learned my addr 172.21.15.39:0/4054774720 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:02.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.964+0000 7fa81be00640 1 --2- 172.21.15.39:0/4054774720 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8240b6c90 0x7fa8241f8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:02.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.964+0000 7fa822a00640 1 --2- 172.21.15.39:0/4054774720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8240b5f60 0x7fa8241f6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:02.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.965+0000 7fa823400640 1 -- 172.21.15.39:0/4054774720 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8240b6c90 msgr2=0x7fa8241f8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:02.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.965+0000 7fa823400640 1 --2- 172.21.15.39:0/4054774720 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8240b6c90 0x7fa8241f8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:02.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.965+0000 7fa823400640 1 -- 172.21.15.39:0/4054774720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8240b5f60 msgr2=0x7fa8241f6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:02.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.965+0000 7fa823400640 1 --2- 172.21.15.39:0/4054774720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8240b5f60 0x7fa8241f6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:02.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.965+0000 7fa823400640 1 -- 172.21.15.39:0/4054774720 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8241f93c0 con 0x7fa8240d25a0 2024-09-21T07:49:02.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.965+0000 7fa823400640 1 --2- 172.21.15.39:0/4054774720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8240d25a0 0x7fa82415e860 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7fa81c02aec0 tx=0x7fa81c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:02.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.965+0000 7fa81aa00640 1 -- 172.21.15.39:0/4054774720 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c004240 con 0x7fa8240d25a0 2024-09-21T07:49:02.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.965+0000 7fa81aa00640 1 -- 172.21.15.39:0/4054774720 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa81c0043e0 con 0x7fa8240d25a0 2024-09-21T07:49:02.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.965+0000 7fa823e00640 1 -- 172.21.15.39:0/4054774720 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8241f98d0 con 0x7fa8240d25a0 2024-09-21T07:49:02.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.965+0000 7fa81aa00640 1 -- 172.21.15.39:0/4054774720 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c004830 con 0x7fa8240d25a0 2024-09-21T07:49:02.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.965+0000 7fa823e00640 1 -- 172.21.15.39:0/4054774720 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8240cb730 con 0x7fa8240d25a0 2024-09-21T07:49:02.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.966+0000 7fa823e00640 1 -- 172.21.15.39:0/4054774720 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8241f9620 con 0x7fa8240d25a0 2024-09-21T07:49:02.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.968+0000 7fa81aa00640 1 -- 172.21.15.39:0/4054774720 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa81c031460 con 0x7fa8240d25a0 2024-09-21T07:49:02.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.968+0000 7fa81aa00640 1 --2- 172.21.15.39:0/4054774720 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa800078b90 0x7fa80007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:02.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.969+0000 7fa81aa00640 1 -- 172.21.15.39:0/4054774720 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(221..221 src has 1..221) ==== 7075+0+0 (secure 0 0 0) 0x7fa81c0bcd00 con 0x7fa8240d25a0 2024-09-21T07:49:02.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.969+0000 7fa822a00640 1 --2- 172.21.15.39:0/4054774720 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa800078b90 0x7fa80007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:02.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.970+0000 7fa822a00640 1 --2- 172.21.15.39:0/4054774720 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa800078b90 0x7fa80007b0a0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fa814006fd0 tx=0x7fa814009060 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:02.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:02.973+0000 7fa81aa00640 1 -- 172.21.15.39:0/4054774720 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa8240cfef0 con 0x7fa8240d25a0 2024-09-21T07:49:03.099 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:49:03.103 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:49:03.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:03.211+0000 7fa823e00640 1 -- 172.21.15.39:0/4054774720 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) -- 0x7fa8240b9680 con 0x7fa8240d25a0 2024-09-21T07:49:03.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:03.697+0000 7fa81aa00640 1 -- 172.21.15.39:0/4054774720 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v231) ==== 105+0+0 (secure 0 0 0) 0x7fa81c0844f0 con 0x7fa8240d25a0 2024-09-21T07:49:03.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:03.700+0000 7fa823e00640 1 -- 172.21.15.39:0/4054774720 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa800078b90 msgr2=0x7fa80007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:03.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:03.700+0000 7fa823e00640 1 --2- 172.21.15.39:0/4054774720 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa800078b90 0x7fa80007b0a0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fa814006fd0 tx=0x7fa814009060 comp rx=0 tx=0).stop 2024-09-21T07:49:03.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:03.700+0000 7fa823e00640 1 -- 172.21.15.39:0/4054774720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8240d25a0 msgr2=0x7fa82415e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:03.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:03.700+0000 7fa823e00640 1 --2- 172.21.15.39:0/4054774720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8240d25a0 0x7fa82415e860 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7fa81c02aec0 tx=0x7fa81c003c20 comp rx=0 tx=0).stop 2024-09-21T07:49:03.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:03.700+0000 7fa823e00640 1 -- 172.21.15.39:0/4054774720 shutdown_connections 2024-09-21T07:49:03.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:03.700+0000 7fa823e00640 1 --2- 172.21.15.39:0/4054774720 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa800078b90 0x7fa80007b0a0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:03.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:03.700+0000 7fa823e00640 1 --2- 172.21.15.39:0/4054774720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8240d25a0 0x7fa82415e860 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:03.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:03.700+0000 7fa823e00640 1 --2- 172.21.15.39:0/4054774720 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8240b6c90 0x7fa8241f8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:03.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:03.700+0000 7fa823e00640 1 --2- 172.21.15.39:0/4054774720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8240b5f60 0x7fa8241f6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:03.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:03.700+0000 7fa823e00640 1 -- 172.21.15.39:0/4054774720 >> 172.21.15.39:0/4054774720 conn(0x7fa82409d7b0 msgr2=0x7fa8240b2690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:03.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:03.700+0000 7fa823e00640 1 -- 172.21.15.39:0/4054774720 shutdown_connections 2024-09-21T07:49:03.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:03.700+0000 7fa823e00640 1 -- 172.21.15.39:0/4054774720 wait complete. 2024-09-21T07:49:03.733 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:49:04.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.001+0000 7f15d3a00640 1 -- 172.21.15.39:0/4198127705 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15cc070c80 msgr2=0x7f15cc151a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:04.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.001+0000 7f15d3a00640 1 --2- 172.21.15.39:0/4198127705 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15cc070c80 0x7f15cc151a50 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f15b8009370 tx=0x7f15b802a580 comp rx=0 tx=0).stop 2024-09-21T07:49:04.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.001+0000 7f15d3a00640 1 -- 172.21.15.39:0/4198127705 shutdown_connections 2024-09-21T07:49:04.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.001+0000 7f15d3a00640 1 --2- 172.21.15.39:0/4198127705 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15cc0cf750 0x7f15cc0cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:04.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.001+0000 7f15d3a00640 1 --2- 172.21.15.39:0/4198127705 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15cc070c80 0x7f15cc151a50 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:04.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.001+0000 7f15d3a00640 1 -- 172.21.15.39:0/4198127705 >> 172.21.15.39:0/4198127705 conn(0x7f15cc06cfc0 msgr2=0x7f15cc06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:04.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.001+0000 7f15d3a00640 1 -- 172.21.15.39:0/4198127705 shutdown_connections 2024-09-21T07:49:04.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.001+0000 7f15d3a00640 1 -- 172.21.15.39:0/4198127705 wait complete. 2024-09-21T07:49:04.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.003+0000 7f15d3a00640 1 Processor -- start 2024-09-21T07:49:04.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.003+0000 7f15d3a00640 1 -- start start 2024-09-21T07:49:04.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.005+0000 7f15d3a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15cc0cf750 0x7f15cc13cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:04.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.005+0000 7f15d3a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15cc151640 0x7f15cc153ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:04.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.005+0000 7f15d3a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15cc152620 0x7f15cc1408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:04.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.005+0000 7f15d3a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc15eb60 con 0x7f15cc152620 2024-09-21T07:49:04.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.005+0000 7f15d3a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc15e9e0 con 0x7f15cc0cf750 2024-09-21T07:49:04.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.005+0000 7f15d3a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f15cc15ece0 con 0x7f15cc151640 2024-09-21T07:49:04.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.005+0000 7f15d1a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15cc152620 0x7f15cc1408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:04.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.005+0000 7f15d1a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15cc152620 0x7f15cc1408e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36012/0 (socket says 172.21.15.39:36012) 2024-09-21T07:49:04.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.005+0000 7f15d1a00640 1 -- 172.21.15.39:0/279170989 learned_addr learned my addr 172.21.15.39:0/279170989 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:04.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.005+0000 7f15d1000640 1 --2- 172.21.15.39:0/279170989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15cc0cf750 0x7f15cc13cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:04.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.005+0000 7f15cbe00640 1 --2- 172.21.15.39:0/279170989 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15cc151640 0x7f15cc153ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:04.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.006+0000 7f15d1a00640 1 -- 172.21.15.39:0/279170989 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15cc151640 msgr2=0x7f15cc153ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:04.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.006+0000 7f15d1a00640 1 --2- 172.21.15.39:0/279170989 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15cc151640 0x7f15cc153ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:04.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.006+0000 7f15d1a00640 1 -- 172.21.15.39:0/279170989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15cc0cf750 msgr2=0x7f15cc13cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:04.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.006+0000 7f15d1a00640 1 --2- 172.21.15.39:0/279170989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15cc0cf750 0x7f15cc13cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:04.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.006+0000 7f15d1a00640 1 -- 172.21.15.39:0/279170989 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15cc13d390 con 0x7f15cc152620 2024-09-21T07:49:04.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.006+0000 7f15d1000640 1 --2- 172.21.15.39:0/279170989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15cc0cf750 0x7f15cc13cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:04.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.006+0000 7f15cbe00640 1 --2- 172.21.15.39:0/279170989 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15cc151640 0x7f15cc153ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:04.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.006+0000 7f15d1a00640 1 --2- 172.21.15.39:0/279170989 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15cc152620 0x7f15cc1408e0 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f15c400afd0 tx=0x7f15c400c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:04.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.006+0000 7f15c8c00640 1 -- 172.21.15.39:0/279170989 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c4004d20 con 0x7f15cc152620 2024-09-21T07:49:04.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.006+0000 7f15c8c00640 1 -- 172.21.15.39:0/279170989 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15c400b570 con 0x7f15cc152620 2024-09-21T07:49:04.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.006+0000 7f15d3a00640 1 -- 172.21.15.39:0/279170989 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15cc13d670 con 0x7f15cc152620 2024-09-21T07:49:04.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.006+0000 7f15c8c00640 1 -- 172.21.15.39:0/279170989 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c400b980 con 0x7f15cc152620 2024-09-21T07:49:04.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.006+0000 7f15d3a00640 1 -- 172.21.15.39:0/279170989 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15cc1f6d50 con 0x7f15cc152620 2024-09-21T07:49:04.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.008+0000 7f15d3a00640 1 -- 172.21.15.39:0/279170989 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15cc13d100 con 0x7f15cc152620 2024-09-21T07:49:04.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.009+0000 7f15c8c00640 1 -- 172.21.15.39:0/279170989 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f15c400bb20 con 0x7f15cc152620 2024-09-21T07:49:04.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.010+0000 7f15c8c00640 1 --2- 172.21.15.39:0/279170989 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f15ac078b90 0x7f15ac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:04.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.010+0000 7f15c8c00640 1 -- 172.21.15.39:0/279170989 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(221..221 src has 1..221) ==== 7075+0+0 (secure 0 0 0) 0x7f15c409bf00 con 0x7f15cc152620 2024-09-21T07:49:04.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.011+0000 7f15d1000640 1 --2- 172.21.15.39:0/279170989 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f15ac078b90 0x7f15ac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:04.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.011+0000 7f15d1000640 1 --2- 172.21.15.39:0/279170989 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f15ac078b90 0x7f15ac07b0a0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f15c0007fd0 tx=0x7f15c0009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:04.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.015+0000 7f15c8c00640 1 -- 172.21.15.39:0/279170989 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f15c4063e70 con 0x7f15cc152620 2024-09-21T07:49:04.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.251+0000 7f15d3a00640 1 -- 172.21.15.39:0/279170989 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15cc15f200 con 0x7f15cc152620 2024-09-21T07:49:04.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.251+0000 7f15c8c00640 1 -- 172.21.15.39:0/279170989 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 232 v232) ==== 95+0+5257 (secure 0 0 0) 0x7f15c4063770 con 0x7f15cc152620 2024-09-21T07:49:04.253 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:49:04.253 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":232,"btime":"2024-09-21T07:49:03:701737+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8249,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/550771731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":550771731},{"type":"v1","addr":"172.21.15.148:6837","nonce":550771731}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":224},{"gid":9385,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1307677219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":1307677219},{"type":"v1","addr":"172.21.15.39:6835","nonce":1307677219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":224}],"filesystems":[{"mdsmap":{"epoch":232,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:48:47.106474+0000","modified":"2024-09-21T07:49:03.701712+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":10224,"mds_1":9391},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9391":{"gid":9391,"name":"b","rank":1,"incarnation":232,"state":"up:creating","state_seq":1,"addr":"172.21.15.148:6836/1256253563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1256253563},{"type":"v1","addr":"172.21.15.148:6836","nonce":1256253563}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10224":{"gid":10224,"name":"c","rank":0,"incarnation":225,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/2837221709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2837221709},{"type":"v1","addr":"172.21.15.39:6834","nonce":2837221709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10224,"qdb_cluster":[10224]},"id":12}]} 2024-09-21T07:49:04.254 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 232 2024-09-21T07:49:04.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.254+0000 7f15d3a00640 1 -- 172.21.15.39:0/279170989 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f15ac078b90 msgr2=0x7f15ac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:04.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.254+0000 7f15d3a00640 1 --2- 172.21.15.39:0/279170989 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f15ac078b90 0x7f15ac07b0a0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f15c0007fd0 tx=0x7f15c0009040 comp rx=0 tx=0).stop 2024-09-21T07:49:04.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.254+0000 7f15d3a00640 1 -- 172.21.15.39:0/279170989 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15cc152620 msgr2=0x7f15cc1408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:04.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.254+0000 7f15d3a00640 1 --2- 172.21.15.39:0/279170989 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15cc152620 0x7f15cc1408e0 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f15c400afd0 tx=0x7f15c400c7a0 comp rx=0 tx=0).stop 2024-09-21T07:49:04.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.254+0000 7f15d3a00640 1 -- 172.21.15.39:0/279170989 shutdown_connections 2024-09-21T07:49:04.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.254+0000 7f15d3a00640 1 --2- 172.21.15.39:0/279170989 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f15ac078b90 0x7f15ac07b0a0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:04.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.254+0000 7f15d3a00640 1 --2- 172.21.15.39:0/279170989 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15cc152620 0x7f15cc1408e0 unknown :-1 s=CLOSED pgs=2134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:04.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.254+0000 7f15d3a00640 1 --2- 172.21.15.39:0/279170989 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f15cc151640 0x7f15cc153ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:04.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.254+0000 7f15d3a00640 1 --2- 172.21.15.39:0/279170989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15cc0cf750 0x7f15cc13cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:04.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.254+0000 7f15d3a00640 1 -- 172.21.15.39:0/279170989 >> 172.21.15.39:0/279170989 conn(0x7f15cc06cfc0 msgr2=0x7f15cc14c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:04.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.254+0000 7f15d3a00640 1 -- 172.21.15.39:0/279170989 shutdown_connections 2024-09-21T07:49:04.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:04.254+0000 7f15d3a00640 1 -- 172.21.15.39:0/279170989 wait complete. 2024-09-21T07:49:04.682 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 232, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:48:47.106474+0000', 'modified': '2024-09-21T07:49:03.701712+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 10224, 'mds_1': 9391}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9391': {'gid': 9391, 'name': 'b', 'rank': 1, 'incarnation': 232, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.148:6836/1256253563', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 1256253563}, {'type': 'v1', 'addr': '172.21.15.148:6836', 'nonce': 1256253563}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10224': {'gid': 10224, 'name': 'c', 'rank': 0, 'incarnation': 225, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.39:6834/2837221709', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 2837221709}, {'type': 'v1', 'addr': '172.21.15.39:6834', 'nonce': 2837221709}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10224, 'qdb_cluster': [10224]} 2024-09-21T07:49:04.682 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9391:up:creating 2024-09-21T07:49:05.682 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:49:05.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:05.920+0000 7f58db200640 1 -- 172.21.15.39:0/94947223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58d40b3b60 msgr2=0x7f58d40b3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:05.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:05.920+0000 7f58db200640 1 --2- 172.21.15.39:0/94947223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58d40b3b60 0x7f58d40b3f70 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f58d408c880 tx=0x7f58d002a8b0 comp rx=0 tx=0).stop 2024-09-21T07:49:05.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:05.920+0000 7f58db200640 1 -- 172.21.15.39:0/94947223 shutdown_connections 2024-09-21T07:49:05.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:05.920+0000 7f58db200640 1 --2- 172.21.15.39:0/94947223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58d40b3b60 0x7f58d40b3f70 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:05.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:05.920+0000 7f58db200640 1 --2- 172.21.15.39:0/94947223 >> v2:172.21.15.148:3300/0 conn(0x7f58d40b5a90 0x7f58d40d1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:05.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:05.920+0000 7f58db200640 1 -- 172.21.15.39:0/94947223 >> 172.21.15.39:0/94947223 conn(0x7f58d409d7b0 msgr2=0x7f58d40d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:05.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:05.920+0000 7f58db200640 1 -- 172.21.15.39:0/94947223 shutdown_connections 2024-09-21T07:49:05.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:05.920+0000 7f58db200640 1 -- 172.21.15.39:0/94947223 wait complete. 2024-09-21T07:49:05.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:05.922+0000 7f58db200640 1 Processor -- start 2024-09-21T07:49:05.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:05.922+0000 7f58db200640 1 -- start start 2024-09-21T07:49:05.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:05.924+0000 7f58db200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f58d40b3b60 0x7f58d40c9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:05.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:05.924+0000 7f58db200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58d40b5a90 0x7f58d41f9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:05.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:05.924+0000 7f58db200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58d40d0e70 0x7f58d41527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:05.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:05.924+0000 7f58db200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f58d40b4630 con 0x7f58d40d0e70 2024-09-21T07:49:05.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:05.924+0000 7f58db200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f58d40b44b0 con 0x7f58d40b5a90 2024-09-21T07:49:05.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:05.924+0000 7f58db200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f58d40b47b0 con 0x7f58d40b3b60 2024-09-21T07:49:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:05.924+0000 7f58da800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58d40d0e70 0x7f58d41527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:05.924+0000 7f58da800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58d40d0e70 0x7f58d41527e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36026/0 (socket says 172.21.15.39:36026) 2024-09-21T07:49:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:05.924+0000 7f58da800640 1 -- 172.21.15.39:0/2680944490 learned_addr learned my addr 172.21.15.39:0/2680944490 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:05.924+0000 7f58d9400640 1 --2- 172.21.15.39:0/2680944490 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58d40b5a90 0x7f58d41f9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:05.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:05.924+0000 7f58d9e00640 1 --2- 172.21.15.39:0/2680944490 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f58d40b3b60 0x7f58d40c9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:05.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:05.925+0000 7f58da800640 1 -- 172.21.15.39:0/2680944490 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f58d40b3b60 msgr2=0x7f58d40c9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:05.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:05.925+0000 7f58da800640 1 --2- 172.21.15.39:0/2680944490 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f58d40b3b60 0x7f58d40c9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:05.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:05.925+0000 7f58da800640 1 -- 172.21.15.39:0/2680944490 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58d40b5a90 msgr2=0x7f58d41f9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:05.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:05.925+0000 7f58da800640 1 --2- 172.21.15.39:0/2680944490 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58d40b5a90 0x7f58d41f9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:05.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:05.925+0000 7f58da800640 1 -- 172.21.15.39:0/2680944490 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d415df40 con 0x7f58d40d0e70 2024-09-21T07:49:05.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:05.925+0000 7f58d9e00640 1 --2- 172.21.15.39:0/2680944490 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f58d40b3b60 0x7f58d40c9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:05.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:05.925+0000 7f58da800640 1 --2- 172.21.15.39:0/2680944490 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58d40d0e70 0x7f58d41527e0 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f58d0009ca0 tx=0x7f58d0033f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:05.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:05.925+0000 7f58ce000640 1 -- 172.21.15.39:0/2680944490 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d0031990 con 0x7f58d40d0e70 2024-09-21T07:49:05.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:05.925+0000 7f58ce000640 1 -- 172.21.15.39:0/2680944490 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58d0031c20 con 0x7f58d40d0e70 2024-09-21T07:49:05.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:05.925+0000 7f58ce000640 1 -- 172.21.15.39:0/2680944490 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d00330d0 con 0x7f58d40d0e70 2024-09-21T07:49:05.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:05.925+0000 7f58db200640 1 -- 172.21.15.39:0/2680944490 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58d41f9e40 con 0x7f58d40d0e70 2024-09-21T07:49:05.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:05.926+0000 7f58db200640 1 -- 172.21.15.39:0/2680944490 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58d41f9100 con 0x7f58d40d0e70 2024-09-21T07:49:05.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:05.927+0000 7f58db200640 1 -- 172.21.15.39:0/2680944490 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58d40b4c60 con 0x7f58d40d0e70 2024-09-21T07:49:05.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:05.928+0000 7f58ce000640 1 -- 172.21.15.39:0/2680944490 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f58d0033270 con 0x7f58d40d0e70 2024-09-21T07:49:05.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:05.929+0000 7f58ce000640 1 --2- 172.21.15.39:0/2680944490 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f58b8078b90 0x7f58b807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:05.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:05.929+0000 7f58ce000640 1 -- 172.21.15.39:0/2680944490 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(221..221 src has 1..221) ==== 7075+0+0 (secure 0 0 0) 0x7f58d00bc6e0 con 0x7f58d40d0e70 2024-09-21T07:49:05.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:05.929+0000 7f58d9e00640 1 --2- 172.21.15.39:0/2680944490 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f58b8078b90 0x7f58b807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:05.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:05.930+0000 7f58d9e00640 1 --2- 172.21.15.39:0/2680944490 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f58b8078b90 0x7f58b807b0a0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f58c8005fd0 tx=0x7f58c8005ee0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:05.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:05.933+0000 7f58ce000640 1 -- 172.21.15.39:0/2680944490 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f58d003f030 con 0x7f58d40d0e70 2024-09-21T07:49:06.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.170+0000 7f58db200640 1 -- 172.21.15.39:0/2680944490 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58d415dcd0 con 0x7f58d40d0e70 2024-09-21T07:49:06.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.171+0000 7f58ce000640 1 -- 172.21.15.39:0/2680944490 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 233 v233) ==== 95+0+5260 (secure 0 0 0) 0x7f58d0084640 con 0x7f58d40d0e70 2024-09-21T07:49:06.173 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:49:06.173 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":233,"btime":"2024-09-21T07:49:04:704269+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8249,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/550771731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":550771731},{"type":"v1","addr":"172.21.15.148:6837","nonce":550771731}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":224},{"gid":9385,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1307677219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":1307677219},{"type":"v1","addr":"172.21.15.39:6835","nonce":1307677219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":224}],"filesystems":[{"mdsmap":{"epoch":233,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:48:47.106474+0000","modified":"2024-09-21T07:49:04.704262+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":10224,"mds_1":9391},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9391":{"gid":9391,"name":"b","rank":1,"incarnation":232,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6836/1256253563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1256253563},{"type":"v1","addr":"172.21.15.148:6836","nonce":1256253563}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10224":{"gid":10224,"name":"c","rank":0,"incarnation":225,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/2837221709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2837221709},{"type":"v1","addr":"172.21.15.39:6834","nonce":2837221709}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10224,"qdb_cluster":[9391,10224]},"id":12}]} 2024-09-21T07:49:06.173 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 233 2024-09-21T07:49:06.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.174+0000 7f58db200640 1 -- 172.21.15.39:0/2680944490 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f58b8078b90 msgr2=0x7f58b807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:06.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.174+0000 7f58db200640 1 --2- 172.21.15.39:0/2680944490 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f58b8078b90 0x7f58b807b0a0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f58c8005fd0 tx=0x7f58c8005ee0 comp rx=0 tx=0).stop 2024-09-21T07:49:06.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.174+0000 7f58db200640 1 -- 172.21.15.39:0/2680944490 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58d40d0e70 msgr2=0x7f58d41527e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:06.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.174+0000 7f58db200640 1 --2- 172.21.15.39:0/2680944490 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58d40d0e70 0x7f58d41527e0 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f58d0009ca0 tx=0x7f58d0033f70 comp rx=0 tx=0).stop 2024-09-21T07:49:06.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.174+0000 7f58db200640 1 -- 172.21.15.39:0/2680944490 shutdown_connections 2024-09-21T07:49:06.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.174+0000 7f58db200640 1 --2- 172.21.15.39:0/2680944490 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f58b8078b90 0x7f58b807b0a0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:06.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.174+0000 7f58db200640 1 --2- 172.21.15.39:0/2680944490 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58d40d0e70 0x7f58d41527e0 unknown :-1 s=CLOSED pgs=2137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:06.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.174+0000 7f58db200640 1 --2- 172.21.15.39:0/2680944490 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58d40b5a90 0x7f58d41f9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:06.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.174+0000 7f58db200640 1 --2- 172.21.15.39:0/2680944490 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f58d40b3b60 0x7f58d40c9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:06.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.174+0000 7f58db200640 1 -- 172.21.15.39:0/2680944490 >> 172.21.15.39:0/2680944490 conn(0x7f58d409d7b0 msgr2=0x7f58d406f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:06.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.174+0000 7f58db200640 1 -- 172.21.15.39:0/2680944490 shutdown_connections 2024-09-21T07:49:06.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.174+0000 7f58db200640 1 -- 172.21.15.39:0/2680944490 wait complete. 2024-09-21T07:49:06.602 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 233, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:48:47.106474+0000', 'modified': '2024-09-21T07:49:04.704262+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 10224, 'mds_1': 9391}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9391': {'gid': 9391, 'name': 'b', 'rank': 1, 'incarnation': 232, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.148:6836/1256253563', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 1256253563}, {'type': 'v1', 'addr': '172.21.15.148:6836', 'nonce': 1256253563}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10224': {'gid': 10224, 'name': 'c', 'rank': 0, 'incarnation': 225, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.39:6834/2837221709', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 2837221709}, {'type': 'v1', 'addr': '172.21.15.39:6834', 'nonce': 2837221709}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10224, 'qdb_cluster': [9391, 10224]} 2024-09-21T07:49:06.603 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2024-09-21T07:49:06.603 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-21T07:49:06.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.831+0000 7f82c1600640 1 -- 172.21.15.39:0/3386701011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82a0005210 msgr2=0x7f82a000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:06.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.831+0000 7f82c1600640 1 --2- 172.21.15.39:0/3386701011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82a0005210 0x7f82a000b9c0 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7f82b400c640 tx=0x7f82a0005830 comp rx=0 tx=0).stop 2024-09-21T07:49:06.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.831+0000 7f82bb400640 1 -- 172.21.15.39:0/3386701011 reap_dead start 2024-09-21T07:49:06.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.831+0000 7f82c1600640 1 -- 172.21.15.39:0/3386701011 shutdown_connections 2024-09-21T07:49:06.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.831+0000 7f82c1600640 1 -- 172.21.15.39:0/3386701011 >> 172.21.15.39:0/3386701011 conn(0x7f82bc06cfc0 msgr2=0x7f82bc06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:06.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.831+0000 7f82c1600640 1 -- 172.21.15.39:0/3386701011 shutdown_connections 2024-09-21T07:49:06.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.831+0000 7f82c1600640 1 -- 172.21.15.39:0/3386701011 wait complete. 2024-09-21T07:49:06.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.833+0000 7f82c1600640 1 Processor -- start 2024-09-21T07:49:06.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.833+0000 7f82c1600640 1 -- start start 2024-09-21T07:49:06.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.835+0000 7f82c1600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82a0005210 0x7f82bc1573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:06.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.835+0000 7f82c1600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82bc1f6e10 0x7f82bc1f7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:06.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.835+0000 7f82c1600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82bc152d60 0x7f82bc153170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:06.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.835+0000 7f82c1600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f82bc161890 con 0x7f82bc152d60 2024-09-21T07:49:06.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.835+0000 7f82c1600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f82bc161710 con 0x7f82a0005210 2024-09-21T07:49:06.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.835+0000 7f82c1600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f82bc161a10 con 0x7f82bc1f6e10 2024-09-21T07:49:06.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.835+0000 7f82bb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82bc152d60 0x7f82bc153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:06.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.835+0000 7f82bb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82bc152d60 0x7f82bc153170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36044/0 (socket says 172.21.15.39:36044) 2024-09-21T07:49:06.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.835+0000 7f82bb400640 1 -- 172.21.15.39:0/1084940134 learned_addr learned my addr 172.21.15.39:0/1084940134 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:06.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.835+0000 7f82baa00640 1 --2- 172.21.15.39:0/1084940134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82a0005210 0x7f82bc1573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:06.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.835+0000 7f82ba000640 1 --2- 172.21.15.39:0/1084940134 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82bc1f6e10 0x7f82bc1f7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:06.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.835+0000 7f82bb400640 1 -- 172.21.15.39:0/1084940134 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82bc1f6e10 msgr2=0x7f82bc1f7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:06.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.835+0000 7f82bb400640 1 --2- 172.21.15.39:0/1084940134 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82bc1f6e10 0x7f82bc1f7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:06.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.835+0000 7f82bb400640 1 -- 172.21.15.39:0/1084940134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82a0005210 msgr2=0x7f82bc1573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:06.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.835+0000 7f82bb400640 1 --2- 172.21.15.39:0/1084940134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82a0005210 0x7f82bc1573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:06.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.835+0000 7f82bb400640 1 -- 172.21.15.39:0/1084940134 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82bc1f7760 con 0x7f82bc152d60 2024-09-21T07:49:06.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.836+0000 7f82bb400640 1 --2- 172.21.15.39:0/1084940134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82bc152d60 0x7f82bc153170 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f82b400c730 tx=0x7f82b4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:06.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.836+0000 7f82b2a00640 1 -- 172.21.15.39:0/1084940134 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b4039070 con 0x7f82bc152d60 2024-09-21T07:49:06.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.836+0000 7f82b2a00640 1 -- 172.21.15.39:0/1084940134 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82b402ba40 con 0x7f82bc152d60 2024-09-21T07:49:06.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.836+0000 7f82b2a00640 1 -- 172.21.15.39:0/1084940134 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b402be50 con 0x7f82bc152d60 2024-09-21T07:49:06.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.836+0000 7f82c1600640 1 -- 172.21.15.39:0/1084940134 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82bc1536b0 con 0x7f82bc152d60 2024-09-21T07:49:06.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.836+0000 7f82c1600640 1 -- 172.21.15.39:0/1084940134 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82bc14f020 con 0x7f82bc152d60 2024-09-21T07:49:06.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.838+0000 7f82b2a00640 1 -- 172.21.15.39:0/1084940134 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f82b4034680 con 0x7f82bc152d60 2024-09-21T07:49:06.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.839+0000 7f82b2a00640 1 --2- 172.21.15.39:0/1084940134 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f829c078d30 0x7f829c07b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:06.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.839+0000 7f82b2a00640 1 -- 172.21.15.39:0/1084940134 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(221..221 src has 1..221) ==== 7075+0+0 (secure 0 0 0) 0x7f82b40bb4f0 con 0x7f82bc152d60 2024-09-21T07:49:06.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.840+0000 7f82baa00640 1 --2- 172.21.15.39:0/1084940134 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f829c078d30 0x7f829c07b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:06.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.840+0000 7f82c883c740 1 Processor -- start 2024-09-21T07:49:06.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.840+0000 7f82c883c740 1 -- start start 2024-09-21T07:49:06.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.841+0000 7f82baa00640 1 --2- 172.21.15.39:0/1084940134 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f829c078d30 0x7f829c07b240 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f82bc072900 tx=0x7f82ac005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:06.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.842+0000 7f82c883c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55968aea1270 0x55968b271970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:06.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.842+0000 7f82c883c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55968ae783b0 0x55968b271eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:06.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.842+0000 7f82c883c740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55968ae80590 0x55968b241f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:06.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.842+0000 7f82c883c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55968b1cb430 con 0x55968ae783b0 2024-09-21T07:49:06.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.842+0000 7f82c883c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55968b1bb5d0 con 0x55968aea1270 2024-09-21T07:49:06.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.842+0000 7f82c883c740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55968b1aaa00 con 0x55968ae80590 2024-09-21T07:49:06.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.842+0000 7f82ba000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55968ae783b0 0x55968b271eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:06.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.842+0000 7f82ba000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55968ae783b0 0x55968b271eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36054/0 (socket says 172.21.15.39:36054) 2024-09-21T07:49:06.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.842+0000 7f82ba000640 1 -- 172.21.15.39:0/3654271913 learned_addr learned my addr 172.21.15.39:0/3654271913 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:06.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.842+0000 7f82bb400640 1 --2- 172.21.15.39:0/3654271913 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55968ae80590 0x55968b241f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:06.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.842+0000 7f82baa00640 1 --2- 172.21.15.39:0/3654271913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55968aea1270 0x55968b271970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:06.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.842+0000 7f82ba000640 1 -- 172.21.15.39:0/3654271913 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55968ae80590 msgr2=0x55968b241f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:06.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.842+0000 7f82ba000640 1 --2- 172.21.15.39:0/3654271913 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55968ae80590 0x55968b241f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:06.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.842+0000 7f82ba000640 1 -- 172.21.15.39:0/3654271913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55968aea1270 msgr2=0x55968b271970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:06.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.842+0000 7f82ba000640 1 --2- 172.21.15.39:0/3654271913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55968aea1270 0x55968b271970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:06.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.842+0000 7f82ba000640 1 -- 172.21.15.39:0/3654271913 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55968b173520 con 0x55968ae783b0 2024-09-21T07:49:06.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.843+0000 7f82ba000640 1 --2- 172.21.15.39:0/3654271913 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55968ae783b0 0x55968b271eb0 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f82a8006520 tx=0x7f82a8006610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32eb6d1730e1052d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:06.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.843+0000 7f82a7000640 1 -- 172.21.15.39:0/3654271913 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a8016070 con 0x55968ae783b0 2024-09-21T07:49:06.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.843+0000 7f82a7000640 1 -- 172.21.15.39:0/3654271913 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82a80018c0 con 0x55968ae783b0 2024-09-21T07:49:06.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.843+0000 7f82a7000640 1 -- 172.21.15.39:0/3654271913 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a8001d10 con 0x55968ae783b0 2024-09-21T07:49:06.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.844+0000 7f82c883c740 1 -- 172.21.15.39:0/3654271913 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55968ae783b0 msgr2=0x55968b271eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:06.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.844+0000 7f82c883c740 1 --2- 172.21.15.39:0/3654271913 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55968ae783b0 0x55968b271eb0 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f82a8006520 tx=0x7f82a8006610 comp rx=0 tx=0).stop 2024-09-21T07:49:06.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.844+0000 7f82c883c740 1 -- 172.21.15.39:0/3654271913 shutdown_connections 2024-09-21T07:49:06.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.844+0000 7f82c883c740 1 --2- 172.21.15.39:0/3654271913 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55968ae80590 0x55968b241f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:06.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.844+0000 7f82c883c740 1 --2- 172.21.15.39:0/3654271913 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55968ae783b0 0x55968b271eb0 unknown :-1 s=CLOSED pgs=2141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:06.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.844+0000 7f82c883c740 1 --2- 172.21.15.39:0/3654271913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55968aea1270 0x55968b271970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:06.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.844+0000 7f82c883c740 1 -- 172.21.15.39:0/3654271913 >> 172.21.15.39:0/3654271913 conn(0x55968ae953c0 msgr2=0x55968b247970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:06.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.844+0000 7f82c883c740 1 -- 172.21.15.39:0/3654271913 shutdown_connections 2024-09-21T07:49:06.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.844+0000 7f82c883c740 1 -- 172.21.15.39:0/3654271913 wait complete. 2024-09-21T07:49:06.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.845+0000 7f82c883c740 1 -- start start 2024-09-21T07:49:06.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.845+0000 7f82c883c740 1 Processor -- start 2024-09-21T07:49:06.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.847+0000 7f82c883c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55968ae783b0 0x55968b2556d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:06.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.847+0000 7f82c883c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55968ae80590 0x55968b255c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:06.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.847+0000 7f82c883c740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55968aea1270 0x55968b25b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:06.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.847+0000 7f82c883c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55968b151330 con 0x55968ae783b0 2024-09-21T07:49:06.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.847+0000 7f82c883c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55968b1ae810 con 0x55968ae80590 2024-09-21T07:49:06.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.847+0000 7f82c883c740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55968af1b620 con 0x55968aea1270 2024-09-21T07:49:06.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.847+0000 7f82c883c740 10 client.0 _ll_register_callbacks cb 0x55968b156420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:49:06.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.847+0000 7f82baa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55968ae783b0 0x55968b2556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:06.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.847+0000 7f82baa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55968ae783b0 0x55968b2556d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36064/0 (socket says 172.21.15.39:36064) 2024-09-21T07:49:06.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.847+0000 7f82baa00640 1 -- 172.21.15.39:0/2551648970 learned_addr learned my addr 172.21.15.39:0/2551648970 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:06.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.847+0000 7f82ba000640 1 --2- 172.21.15.39:0/2551648970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55968ae80590 0x55968b255c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:06.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.847+0000 7f82bb400640 1 --2- 172.21.15.39:0/2551648970 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55968aea1270 0x55968b25b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:06.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.847+0000 7f82baa00640 1 -- 172.21.15.39:0/2551648970 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55968aea1270 msgr2=0x55968b25b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:06.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.847+0000 7f82baa00640 1 --2- 172.21.15.39:0/2551648970 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55968aea1270 0x55968b25b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:06.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.847+0000 7f82baa00640 1 -- 172.21.15.39:0/2551648970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55968ae80590 msgr2=0x55968b255c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:06.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.847+0000 7f82baa00640 1 --2- 172.21.15.39:0/2551648970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55968ae80590 0x55968b255c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:06.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.847+0000 7f82baa00640 1 -- 172.21.15.39:0/2551648970 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55968af0c850 con 0x55968ae783b0 2024-09-21T07:49:06.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.847+0000 7f82ba000640 1 --2- 172.21.15.39:0/2551648970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55968ae80590 0x55968b255c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:06.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.847+0000 7f82bb400640 1 --2- 172.21.15.39:0/2551648970 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55968aea1270 0x55968b25b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:06.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.848+0000 7f82baa00640 1 --2- 172.21.15.39:0/2551648970 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55968ae783b0 0x55968b2556d0 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f82ac01fa00 tx=0x7f82ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e2d3b109e4e8f96 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:06.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.848+0000 7f829b400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:49:06.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.848+0000 7f829b400640 1 -- 172.21.15.39:0/2551648970 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ac004050 con 0x55968ae783b0 2024-09-21T07:49:06.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.848+0000 7f829b400640 1 -- 172.21.15.39:0/2551648970 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(221..221 src has 1..221) ==== 7075+0+0 (secure 0 0 0) 0x7f82ac0041f0 con 0x55968ae783b0 2024-09-21T07:49:06.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.848+0000 7f82c883c740 1 -- 172.21.15.39:0/2551648970 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82a80018c0 con 0x55968ae783b0 2024-09-21T07:49:06.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.848+0000 7f829b400640 1 -- 172.21.15.39:0/2551648970 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=233) ==== 24+0+0 (secure 0 0 0) 0x7f82ac0043f0 con 0x55968ae783b0 2024-09-21T07:49:06.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.848+0000 7f82c883c740 10 client.10437 fetch_fsmap learned FSMap version 233 2024-09-21T07:49:06.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.848+0000 7f82c883c740 1 -- 172.21.15.39:0/2551648970 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=233,monmap=2+}) -- 0x55968af836b0 con 0x55968ae783b0 2024-09-21T07:49:06.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.848+0000 7f829b400640 1 -- 172.21.15.39:0/2551648970 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82a80018c0 con 0x55968ae783b0 2024-09-21T07:49:06.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.849+0000 7f829b400640 1 -- 172.21.15.39:0/2551648970 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 233) ==== 3522+0+0 (secure 0 0 0) 0x7f82ac036080 con 0x55968ae783b0 2024-09-21T07:49:06.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.849+0000 7f82c883c740 10 client.10437 fetch_fsmap finished waiting for FSMap version 233 2024-09-21T07:49:06.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.849+0000 7f82c883c740 10 client.10437 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T07:49:06.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.849+0000 7f82c883c740 20 client.10437 populate_metadata read hostname 'smithi039' 2024-09-21T07:49:06.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.849+0000 7f82c883c740 1 --2- 172.21.15.39:0/2551648970 >> [v2:172.21.15.148:6834/1256253563,v1:172.21.15.148:6836/1256253563] conn(0x55968ae5fe40 0x55968b29efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:06.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.849+0000 7f82c883c740 4 client.10437 mds_command: new command op to 9391 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:49:06.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.849+0000 7f82c883c740 1 -- 172.21.15.39:0/2551648970 --> [v2:172.21.15.148:6834/1256253563,v1:172.21.15.148:6836/1256253563] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55968b1bd520 con 0x55968ae5fe40 2024-09-21T07:49:06.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.849+0000 7f82ba000640 1 --2- 172.21.15.39:0/2551648970 >> [v2:172.21.15.148:6834/1256253563,v1:172.21.15.148:6836/1256253563] conn(0x55968ae5fe40 0x55968b29efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:06.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.850+0000 7f82ba000640 1 --2- 172.21.15.39:0/2551648970 >> [v2:172.21.15.148:6834/1256253563,v1:172.21.15.148:6836/1256253563] conn(0x55968ae5fe40 0x55968b29efe0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=43426742ec0a2c4f server_cookie=621fb0c894a1605a in_seq=0 out_seq=0 2024-09-21T07:49:06.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.850+0000 7f829b400640 10 client.10437 ms_handle_connect on v2:172.21.15.148:6834/1256253563 2024-09-21T07:49:06.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.851+0000 7f829b400640 1 -- 172.21.15.39:0/2551648970 <== mds.1 v2:172.21.15.148:6834/1256253563 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55968b1bd520 con 0x55968ae5fe40 2024-09-21T07:49:06.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.851+0000 7f829b400640 10 client.10437 handle_command_reply: tid=0 2024-09-21T07:49:06.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.851+0000 7f82c883c740 1 client.10437 shutdown 2024-09-21T07:49:06.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.852+0000 7f82c883c740 1 -- 172.21.15.39:0/2551648970 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55968ae783b0 msgr2=0x55968b2556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:06.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.852+0000 7f82c883c740 1 --2- 172.21.15.39:0/2551648970 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55968ae783b0 0x55968b2556d0 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f82ac01fa00 tx=0x7f82ac0046c0 comp rx=0 tx=0).stop 2024-09-21T07:49:06.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.852+0000 7f82c883c740 1 -- 172.21.15.39:0/2551648970 shutdown_connections 2024-09-21T07:49:06.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.852+0000 7f82c883c740 1 --2- 172.21.15.39:0/2551648970 >> [v2:172.21.15.148:6834/1256253563,v1:172.21.15.148:6836/1256253563] conn(0x55968ae5fe40 0x55968b29efe0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:06.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.852+0000 7f82c883c740 1 --2- 172.21.15.39:0/2551648970 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55968aea1270 0x55968b25b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:06.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.852+0000 7f82c883c740 1 --2- 172.21.15.39:0/2551648970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55968ae80590 0x55968b255c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:06.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.852+0000 7f82c883c740 1 --2- 172.21.15.39:0/2551648970 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55968ae783b0 0x55968b2556d0 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:06.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.852+0000 7f82c883c740 1 -- 172.21.15.39:0/2551648970 >> 172.21.15.39:0/2551648970 conn(0x55968ae953c0 msgr2=0x55968b28cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:06.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.852+0000 7f829b400640 0 client.10437 ms_handle_reset on v2:172.21.15.148:6834/1256253563 2024-09-21T07:49:06.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.852+0000 7f82c883c740 1 -- 172.21.15.39:0/2551648970 shutdown_connections 2024-09-21T07:49:06.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.852+0000 7f82c883c740 1 -- 172.21.15.39:0/2551648970 wait complete. 2024-09-21T07:49:06.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.852+0000 7f82c883c740 20 client.10437 trim_cache size 0 max 16384 2024-09-21T07:49:06.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.871+0000 7f82c883c740 1 Processor -- start 2024-09-21T07:49:06.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.871+0000 7f82c883c740 1 -- start start 2024-09-21T07:49:06.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.873+0000 7f82c883c740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55968ae783b0 0x55968b2af9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:06.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.873+0000 7f82c883c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55968ae80590 0x55968b255670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:06.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.873+0000 7f82c883c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55968aea1270 0x55968b255bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:06.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.873+0000 7f82c883c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f82bc203060 con 0x55968aea1270 2024-09-21T07:49:06.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.873+0000 7f82c883c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f82bc2013d0 con 0x55968ae80590 2024-09-21T07:49:06.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.873+0000 7f82c883c740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f82bc201950 con 0x55968ae783b0 2024-09-21T07:49:06.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.873+0000 7f82bb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55968aea1270 0x55968b255bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:06.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.873+0000 7f82bb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55968aea1270 0x55968b255bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36066/0 (socket says 172.21.15.39:36066) 2024-09-21T07:49:06.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.873+0000 7f82bb400640 1 -- 172.21.15.39:0/383168011 learned_addr learned my addr 172.21.15.39:0/383168011 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:06.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.873+0000 7f82baa00640 1 --2- 172.21.15.39:0/383168011 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55968ae783b0 0x55968b2af9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:06.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.873+0000 7f82ba000640 1 --2- 172.21.15.39:0/383168011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55968ae80590 0x55968b255670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:06.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.873+0000 7f82bb400640 1 -- 172.21.15.39:0/383168011 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55968ae783b0 msgr2=0x55968b2af9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:06.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.873+0000 7f82bb400640 1 --2- 172.21.15.39:0/383168011 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55968ae783b0 0x55968b2af9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:06.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.873+0000 7f82bb400640 1 -- 172.21.15.39:0/383168011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55968ae80590 msgr2=0x55968b255670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:06.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.873+0000 7f82bb400640 1 --2- 172.21.15.39:0/383168011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55968ae80590 0x55968b255670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:06.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.873+0000 7f82bb400640 1 -- 172.21.15.39:0/383168011 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55968b286b30 con 0x55968aea1270 2024-09-21T07:49:06.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.874+0000 7f82bb400640 1 --2- 172.21.15.39:0/383168011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55968aea1270 0x55968b255bb0 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f82b40b94c0 tx=0x7f82b40012e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d707aab114a9e85 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:06.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.874+0000 7f82a6600640 1 -- 172.21.15.39:0/383168011 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b4044040 con 0x55968aea1270 2024-09-21T07:49:06.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.874+0000 7f82a6600640 1 -- 172.21.15.39:0/383168011 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82b402f040 con 0x55968aea1270 2024-09-21T07:49:06.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.874+0000 7f82a6600640 1 -- 172.21.15.39:0/383168011 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b4083520 con 0x55968aea1270 2024-09-21T07:49:06.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.875+0000 7f82c883c740 1 -- 172.21.15.39:0/383168011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55968aea1270 msgr2=0x55968b255bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:06.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.875+0000 7f82c883c740 1 --2- 172.21.15.39:0/383168011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55968aea1270 0x55968b255bb0 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f82b40b94c0 tx=0x7f82b40012e0 comp rx=0 tx=0).stop 2024-09-21T07:49:06.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.875+0000 7f82c883c740 1 -- 172.21.15.39:0/383168011 shutdown_connections 2024-09-21T07:49:06.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.875+0000 7f82c883c740 1 --2- 172.21.15.39:0/383168011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55968aea1270 0x55968b255bb0 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:06.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.875+0000 7f82c883c740 1 --2- 172.21.15.39:0/383168011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55968ae80590 0x55968b255670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:06.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.875+0000 7f82c883c740 1 --2- 172.21.15.39:0/383168011 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55968ae783b0 0x55968b2af9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:06.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.875+0000 7f82c883c740 1 -- 172.21.15.39:0/383168011 >> 172.21.15.39:0/383168011 conn(0x55968ae953c0 msgr2=0x55968b28d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:06.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.875+0000 7f82c883c740 1 -- 172.21.15.39:0/383168011 shutdown_connections 2024-09-21T07:49:06.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.875+0000 7f82c883c740 1 -- 172.21.15.39:0/383168011 wait complete. 2024-09-21T07:49:06.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.876+0000 7f82c883c740 1 -- start start 2024-09-21T07:49:06.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.876+0000 7f82c883c740 1 Processor -- start 2024-09-21T07:49:06.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.878+0000 7f82c883c740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55968ae783b0 0x55968b309b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:06.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.878+0000 7f82c883c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55968ae80590 0x55968b2715d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:06.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.878+0000 7f82c883c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55968aea1270 0x55968b271b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:06.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.878+0000 7f82c883c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55968b25c680 con 0x55968ae80590 2024-09-21T07:49:06.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.878+0000 7f82c883c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55968b26a4a0 con 0x55968aea1270 2024-09-21T07:49:06.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.878+0000 7f82c883c740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55968b256720 con 0x55968ae783b0 2024-09-21T07:49:06.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.878+0000 7f82c883c740 10 client.0 _ll_register_callbacks cb 0x7f82ac01f890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:49:06.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.878+0000 7f82ba000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55968ae80590 0x55968b2715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:06.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.878+0000 7f82ba000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55968ae80590 0x55968b2715d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36070/0 (socket says 172.21.15.39:36070) 2024-09-21T07:49:06.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.878+0000 7f82ba000640 1 -- 172.21.15.39:0/1397753552 learned_addr learned my addr 172.21.15.39:0/1397753552 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:06.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.878+0000 7f82baa00640 1 --2- 172.21.15.39:0/1397753552 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55968ae783b0 0x55968b309b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:06.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.878+0000 7f82bb400640 1 --2- 172.21.15.39:0/1397753552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55968aea1270 0x55968b271b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:06.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.878+0000 7f82ba000640 1 -- 172.21.15.39:0/1397753552 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55968ae783b0 msgr2=0x55968b309b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:06.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.878+0000 7f82ba000640 1 --2- 172.21.15.39:0/1397753552 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55968ae783b0 0x55968b309b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:06.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.878+0000 7f82ba000640 1 -- 172.21.15.39:0/1397753552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55968aea1270 msgr2=0x55968b271b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:06.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.878+0000 7f82ba000640 1 --2- 172.21.15.39:0/1397753552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55968aea1270 0x55968b271b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:06.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.878+0000 7f82ba000640 1 -- 172.21.15.39:0/1397753552 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55968b267f30 con 0x55968ae80590 2024-09-21T07:49:06.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.879+0000 7f82ba000640 1 --2- 172.21.15.39:0/1397753552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55968ae80590 0x55968b2715d0 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f82a8004e60 tx=0x7f82a8006cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e12f50764df11c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:06.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.879+0000 7f829b400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:49:06.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.879+0000 7f829b400640 1 -- 172.21.15.39:0/1397753552 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a801f070 con 0x55968ae80590 2024-09-21T07:49:06.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.879+0000 7f82c883c740 1 -- 172.21.15.39:0/1397753552 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82b402f040 con 0x55968ae80590 2024-09-21T07:49:06.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.879+0000 7f829b400640 1 -- 172.21.15.39:0/1397753552 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(221..221 src has 1..221) ==== 7075+0+0 (secure 0 0 0) 0x7f82a8028450 con 0x55968ae80590 2024-09-21T07:49:06.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.879+0000 7f829b400640 1 -- 172.21.15.39:0/1397753552 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=233) ==== 24+0+0 (secure 0 0 0) 0x7f82a8035020 con 0x55968ae80590 2024-09-21T07:49:06.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.879+0000 7f82c883c740 10 client.10443 fetch_fsmap learned FSMap version 233 2024-09-21T07:49:06.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.879+0000 7f82c883c740 1 -- 172.21.15.39:0/1397753552 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=233,monmap=2+}) -- 0x55968b248650 con 0x55968ae80590 2024-09-21T07:49:06.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.879+0000 7f829b400640 1 -- 172.21.15.39:0/1397753552 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82b402f040 con 0x55968ae80590 2024-09-21T07:49:06.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.879+0000 7f829b400640 1 -- 172.21.15.39:0/1397753552 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 233) ==== 3522+0+0 (secure 0 0 0) 0x7f82a8028650 con 0x55968ae80590 2024-09-21T07:49:06.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.880+0000 7f82c883c740 10 client.10443 fetch_fsmap finished waiting for FSMap version 233 2024-09-21T07:49:06.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.880+0000 7f82c883c740 10 client.10443 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T07:49:06.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.880+0000 7f82c883c740 20 client.10443 populate_metadata read hostname 'smithi039' 2024-09-21T07:49:06.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.880+0000 7f82c883c740 1 --2- 172.21.15.39:0/1397753552 >> [v2:172.21.15.148:6834/1256253563,v1:172.21.15.148:6836/1256253563] conn(0x55968b2ae310 0x55968b2ae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:06.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.880+0000 7f82c883c740 4 client.10443 mds_command: new command op to 9391 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:49:06.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.880+0000 7f82c883c740 1 -- 172.21.15.39:0/1397753552 --> [v2:172.21.15.148:6834/1256253563,v1:172.21.15.148:6836/1256253563] -- command(tid 0: {"prefix": "status"}) -- 0x55968b1b8fd0 con 0x55968b2ae310 2024-09-21T07:49:06.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.880+0000 7f82baa00640 1 --2- 172.21.15.39:0/1397753552 >> [v2:172.21.15.148:6834/1256253563,v1:172.21.15.148:6836/1256253563] conn(0x55968b2ae310 0x55968b2ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:06.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.881+0000 7f82baa00640 1 --2- 172.21.15.39:0/1397753552 >> [v2:172.21.15.148:6834/1256253563,v1:172.21.15.148:6836/1256253563] conn(0x55968b2ae310 0x55968b2ae720 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9e7a6c0674b00f5 server_cookie=42e6d278f46e0d33 in_seq=0 out_seq=0 2024-09-21T07:49:06.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.881+0000 7f829b400640 10 client.10443 ms_handle_connect on v2:172.21.15.148:6834/1256253563 2024-09-21T07:49:06.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.881+0000 7f829b400640 1 -- 172.21.15.39:0/1397753552 <== mds.1 v2:172.21.15.148:6834/1256253563 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55968b1b8fd0 con 0x55968b2ae310 2024-09-21T07:49:06.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.881+0000 7f829b400640 10 client.10443 handle_command_reply: tid=0 2024-09-21T07:49:06.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.881+0000 7f82c883c740 1 client.10443 shutdown 2024-09-21T07:49:06.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.881+0000 7f82c883c740 1 -- 172.21.15.39:0/1397753552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55968ae80590 msgr2=0x55968b2715d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:06.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.881+0000 7f82c883c740 1 --2- 172.21.15.39:0/1397753552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55968ae80590 0x55968b2715d0 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f82a8004e60 tx=0x7f82a8006cc0 comp rx=0 tx=0).stop 2024-09-21T07:49:06.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.881+0000 7f82c883c740 1 -- 172.21.15.39:0/1397753552 shutdown_connections 2024-09-21T07:49:06.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.881+0000 7f82c883c740 1 --2- 172.21.15.39:0/1397753552 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55968aea1270 0x55968b271b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:06.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.881+0000 7f82c883c740 1 --2- 172.21.15.39:0/1397753552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55968ae80590 0x55968b2715d0 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:06.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.881+0000 7f82c883c740 1 --2- 172.21.15.39:0/1397753552 >> [v2:172.21.15.148:6834/1256253563,v1:172.21.15.148:6836/1256253563] conn(0x55968b2ae310 0x55968b2ae720 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:06.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.881+0000 7f82c883c740 1 --2- 172.21.15.39:0/1397753552 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55968ae783b0 0x55968b309b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:06.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.881+0000 7f82c883c740 1 -- 172.21.15.39:0/1397753552 >> 172.21.15.39:0/1397753552 conn(0x55968ae953c0 msgr2=0x55968b243080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:06.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.881+0000 7f829b400640 0 client.10443 ms_handle_reset on v2:172.21.15.148:6834/1256253563 2024-09-21T07:49:06.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.882+0000 7f82c883c740 1 -- 172.21.15.39:0/1397753552 shutdown_connections 2024-09-21T07:49:06.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.882+0000 7f82c883c740 1 -- 172.21.15.39:0/1397753552 wait complete. 2024-09-21T07:49:06.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.882+0000 7f82c883c740 20 client.10443 trim_cache size 0 max 16384 2024-09-21T07:49:06.887 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:49:06.887 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:49:06.887 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 1, 2024-09-21T07:49:06.887 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 9391, 2024-09-21T07:49:06.887 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:49:06.887 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:49:06.887 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:49:06.887 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 3.1775506359999999, 2024-09-21T07:49:06.887 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 233, 2024-09-21T07:49:06.888 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 221, 2024-09-21T07:49:06.888 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 221, 2024-09-21T07:49:06.888 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 24.357352807000002 2024-09-21T07:49:06.888 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:49:06.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.882+0000 7f82c1600640 1 -- 172.21.15.39:0/1084940134 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f829c078d30 msgr2=0x7f829c07b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:06.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.882+0000 7f82c1600640 1 --2- 172.21.15.39:0/1084940134 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f829c078d30 0x7f829c07b240 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f82bc072900 tx=0x7f82ac005f90 comp rx=0 tx=0).stop 2024-09-21T07:49:06.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.882+0000 7f82c1600640 1 -- 172.21.15.39:0/1084940134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82bc152d60 msgr2=0x7f82bc153170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:06.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.882+0000 7f82c1600640 1 --2- 172.21.15.39:0/1084940134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82bc152d60 0x7f82bc153170 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f82b400c730 tx=0x7f82b4003ba0 comp rx=0 tx=0).stop 2024-09-21T07:49:06.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.882+0000 7f82c1600640 1 -- 172.21.15.39:0/1084940134 shutdown_connections 2024-09-21T07:49:06.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.882+0000 7f82c1600640 1 --2- 172.21.15.39:0/1084940134 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f829c078d30 0x7f829c07b240 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:06.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.882+0000 7f82c1600640 1 --2- 172.21.15.39:0/1084940134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82bc152d60 0x7f82bc153170 unknown :-1 s=CLOSED pgs=2140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:06.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.882+0000 7f82c1600640 1 --2- 172.21.15.39:0/1084940134 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82bc1f6e10 0x7f82bc1f7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:06.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.882+0000 7f82c1600640 1 --2- 172.21.15.39:0/1084940134 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82a0005210 0x7f82bc1573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:06.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.882+0000 7f82c1600640 1 -- 172.21.15.39:0/1084940134 >> 172.21.15.39:0/1084940134 conn(0x7f82bc06cfc0 msgr2=0x7f82bc160900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:06.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.883+0000 7f82c1600640 1 -- 172.21.15.39:0/1084940134 shutdown_connections 2024-09-21T07:49:06.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:06.883+0000 7f82c1600640 1 -- 172.21.15.39:0/1084940134 wait complete. 2024-09-21T07:49:06.906 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T07:49:07.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.132+0000 7fa34aa00640 1 -- 172.21.15.39:0/228244113 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa3440ccc60 msgr2=0x7fa3440cd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:07.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.132+0000 7fa34aa00640 1 --2- 172.21.15.39:0/228244113 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa3440ccc60 0x7fa3440cd070 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa33c005fd0 tx=0x7fa33c02a820 comp rx=0 tx=0).stop 2024-09-21T07:49:07.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.132+0000 7fa34aa00640 1 -- 172.21.15.39:0/228244113 shutdown_connections 2024-09-21T07:49:07.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.132+0000 7fa34aa00640 1 --2- 172.21.15.39:0/228244113 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa3440ccc60 0x7fa3440cd070 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:07.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.132+0000 7fa34aa00640 1 --2- 172.21.15.39:0/228244113 >> v2:172.21.15.148:3300/0 conn(0x7fa3440b30a0 0x7fa3440cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:07.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.132+0000 7fa34aa00640 1 -- 172.21.15.39:0/228244113 >> 172.21.15.39:0/228244113 conn(0x7fa3440b2460 msgr2=0x7fa34406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:07.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.132+0000 7fa34aa00640 1 -- 172.21.15.39:0/228244113 shutdown_connections 2024-09-21T07:49:07.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.133+0000 7fa34aa00640 1 -- 172.21.15.39:0/228244113 wait complete. 2024-09-21T07:49:07.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.135+0000 7fa34aa00640 1 Processor -- start 2024-09-21T07:49:07.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.135+0000 7fa34aa00640 1 -- start start 2024-09-21T07:49:07.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.136+0000 7fa34aa00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa3440b30a0 0x7fa3440cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:07.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.136+0000 7fa34aa00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3440ccc60 0x7fa3441f4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:07.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.136+0000 7fa34aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa34415d830 0x7fa3440c6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:07.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.136+0000 7fa34aa00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa3440af410 con 0x7fa34415d830 2024-09-21T07:49:07.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.136+0000 7fa34aa00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa3440af290 con 0x7fa3440ccc60 2024-09-21T07:49:07.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.136+0000 7fa34aa00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa3440af590 con 0x7fa3440b30a0 2024-09-21T07:49:07.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.136+0000 7fa348a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa34415d830 0x7fa3440c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:07.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.137+0000 7fa348a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa34415d830 0x7fa3440c6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36072/0 (socket says 172.21.15.39:36072) 2024-09-21T07:49:07.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.137+0000 7fa348a00640 1 -- 172.21.15.39:0/1593511634 learned_addr learned my addr 172.21.15.39:0/1593511634 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:07.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.137+0000 7fa343e00640 1 --2- 172.21.15.39:0/1593511634 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa3440b30a0 0x7fa3440cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:07.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.137+0000 7fa33b400640 1 --2- 172.21.15.39:0/1593511634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3440ccc60 0x7fa3441f4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:07.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.137+0000 7fa348a00640 1 -- 172.21.15.39:0/1593511634 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa3440b30a0 msgr2=0x7fa3440cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:07.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.137+0000 7fa348a00640 1 --2- 172.21.15.39:0/1593511634 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa3440b30a0 0x7fa3440cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:07.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.137+0000 7fa348a00640 1 -- 172.21.15.39:0/1593511634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3440ccc60 msgr2=0x7fa3441f4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:07.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.137+0000 7fa348a00640 1 --2- 172.21.15.39:0/1593511634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3440ccc60 0x7fa3441f4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:07.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.137+0000 7fa348a00640 1 -- 172.21.15.39:0/1593511634 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3440b4b80 con 0x7fa34415d830 2024-09-21T07:49:07.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.137+0000 7fa343e00640 1 --2- 172.21.15.39:0/1593511634 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa3440b30a0 0x7fa3440cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:49:07.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.137+0000 7fa33b400640 1 --2- 172.21.15.39:0/1593511634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3440ccc60 0x7fa3441f4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:49:07.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.137+0000 7fa348a00640 1 --2- 172.21.15.39:0/1593511634 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa34415d830 0x7fa3440c6870 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7fa33c035040 tx=0x7fa33c032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:07.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.138+0000 7fa340c00640 1 -- 172.21.15.39:0/1593511634 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c037040 con 0x7fa34415d830 2024-09-21T07:49:07.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.138+0000 7fa340c00640 1 -- 172.21.15.39:0/1593511634 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa33c042070 con 0x7fa34415d830 2024-09-21T07:49:07.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.138+0000 7fa340c00640 1 -- 172.21.15.39:0/1593511634 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c003e60 con 0x7fa34415d830 2024-09-21T07:49:07.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.138+0000 7fa34aa00640 1 -- 172.21.15.39:0/1593511634 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3440c9c10 con 0x7fa34415d830 2024-09-21T07:49:07.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.138+0000 7fa34aa00640 1 -- 172.21.15.39:0/1593511634 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3440c4200 con 0x7fa34415d830 2024-09-21T07:49:07.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.140+0000 7fa340c00640 1 -- 172.21.15.39:0/1593511634 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa33c004260 con 0x7fa34415d830 2024-09-21T07:49:07.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.141+0000 7fa340c00640 1 --2- 172.21.15.39:0/1593511634 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa320078b90 0x7fa32007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:07.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.141+0000 7fa340c00640 1 -- 172.21.15.39:0/1593511634 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(221..221 src has 1..221) ==== 7075+0+0 (secure 0 0 0) 0x7fa33c0bbfb0 con 0x7fa34415d830 2024-09-21T07:49:07.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.142+0000 7fa351bcb740 1 Processor -- start 2024-09-21T07:49:07.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.142+0000 7fa343e00640 1 --2- 172.21.15.39:0/1593511634 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa320078b90 0x7fa32007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:07.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.142+0000 7fa351bcb740 1 -- start start 2024-09-21T07:49:07.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.143+0000 7fa343e00640 1 --2- 172.21.15.39:0/1593511634 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa320078b90 0x7fa32007b0a0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fa334009ab0 tx=0x7fa334005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:07.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.144+0000 7fa351bcb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572b9938270 0x5572b9d08970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:07.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.144+0000 7fa351bcb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572b990f3b0 0x5572b9d08eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:07.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.144+0000 7fa351bcb740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5572b9917590 0x5572b9cd8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:07.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.144+0000 7fa351bcb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5572b9c62430 con 0x5572b990f3b0 2024-09-21T07:49:07.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.144+0000 7fa351bcb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5572b9c525d0 con 0x5572b9938270 2024-09-21T07:49:07.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.144+0000 7fa351bcb740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x5572b9c41a00 con 0x5572b9917590 2024-09-21T07:49:07.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.144+0000 7fa33b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572b990f3b0 0x5572b9d08eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:07.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.144+0000 7fa33b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572b990f3b0 0x5572b9d08eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36088/0 (socket says 172.21.15.39:36088) 2024-09-21T07:49:07.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.144+0000 7fa33b400640 1 -- 172.21.15.39:0/738824104 learned_addr learned my addr 172.21.15.39:0/738824104 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:07.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.144+0000 7fa348a00640 1 --2- 172.21.15.39:0/738824104 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5572b9917590 0x5572b9cd8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:07.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.144+0000 7fa343e00640 1 --2- 172.21.15.39:0/738824104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572b9938270 0x5572b9d08970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:07.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.144+0000 7fa33b400640 1 -- 172.21.15.39:0/738824104 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5572b9917590 msgr2=0x5572b9cd8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:07.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.144+0000 7fa33b400640 1 --2- 172.21.15.39:0/738824104 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5572b9917590 0x5572b9cd8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:07.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.144+0000 7fa33b400640 1 -- 172.21.15.39:0/738824104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572b9938270 msgr2=0x5572b9d08970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:07.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.144+0000 7fa33b400640 1 --2- 172.21.15.39:0/738824104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572b9938270 0x5572b9d08970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:07.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.144+0000 7fa33b400640 1 -- 172.21.15.39:0/738824104 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572b9c0a520 con 0x5572b990f3b0 2024-09-21T07:49:07.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.144+0000 7fa343e00640 1 --2- 172.21.15.39:0/738824104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572b9938270 0x5572b9d08970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:07.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.145+0000 7fa33b400640 1 --2- 172.21.15.39:0/738824104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572b990f3b0 0x5572b9d08eb0 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7fa32c001700 tx=0x7fa32c001bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de322927e5aaecf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:07.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.145+0000 7fa333e00640 1 -- 172.21.15.39:0/738824104 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32c016030 con 0x5572b990f3b0 2024-09-21T07:49:07.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.145+0000 7fa333e00640 1 -- 172.21.15.39:0/738824104 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa32c00d940 con 0x5572b990f3b0 2024-09-21T07:49:07.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.145+0000 7fa333e00640 1 -- 172.21.15.39:0/738824104 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32c00dd10 con 0x5572b990f3b0 2024-09-21T07:49:07.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.146+0000 7fa351bcb740 1 -- 172.21.15.39:0/738824104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572b990f3b0 msgr2=0x5572b9d08eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:07.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.146+0000 7fa351bcb740 1 --2- 172.21.15.39:0/738824104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572b990f3b0 0x5572b9d08eb0 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7fa32c001700 tx=0x7fa32c001bc0 comp rx=0 tx=0).stop 2024-09-21T07:49:07.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.146+0000 7fa351bcb740 1 -- 172.21.15.39:0/738824104 shutdown_connections 2024-09-21T07:49:07.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.146+0000 7fa351bcb740 1 --2- 172.21.15.39:0/738824104 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5572b9917590 0x5572b9cd8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:07.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.146+0000 7fa351bcb740 1 --2- 172.21.15.39:0/738824104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572b990f3b0 0x5572b9d08eb0 unknown :-1 s=CLOSED pgs=2147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:07.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.146+0000 7fa351bcb740 1 --2- 172.21.15.39:0/738824104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572b9938270 0x5572b9d08970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:07.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.146+0000 7fa351bcb740 1 -- 172.21.15.39:0/738824104 >> 172.21.15.39:0/738824104 conn(0x5572b992c3c0 msgr2=0x5572b9cde970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:07.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.146+0000 7fa351bcb740 1 -- 172.21.15.39:0/738824104 shutdown_connections 2024-09-21T07:49:07.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.146+0000 7fa351bcb740 1 -- 172.21.15.39:0/738824104 wait complete. 2024-09-21T07:49:07.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.147+0000 7fa351bcb740 1 -- start start 2024-09-21T07:49:07.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.147+0000 7fa351bcb740 1 Processor -- start 2024-09-21T07:49:07.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.149+0000 7fa351bcb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572b990f3b0 0x5572b9d36a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:07.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.149+0000 7fa351bcb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572b9917590 0x5572b9d05960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:07.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.149+0000 7fa351bcb740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5572b9938270 0x5572b9cd9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:07.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.149+0000 7fa351bcb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5572b9be8330 con 0x5572b990f3b0 2024-09-21T07:49:07.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.149+0000 7fa351bcb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5572b9c45810 con 0x5572b9917590 2024-09-21T07:49:07.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.149+0000 7fa351bcb740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x5572b99b2620 con 0x5572b9938270 2024-09-21T07:49:07.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.149+0000 7fa351bcb740 10 client.0 _ll_register_callbacks cb 0x5572b9bed420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:49:07.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.149+0000 7fa343e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572b990f3b0 0x5572b9d36a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:07.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.149+0000 7fa343e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572b990f3b0 0x5572b9d36a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36100/0 (socket says 172.21.15.39:36100) 2024-09-21T07:49:07.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.149+0000 7fa343e00640 1 -- 172.21.15.39:0/3992471646 learned_addr learned my addr 172.21.15.39:0/3992471646 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:07.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.149+0000 7fa33b400640 1 --2- 172.21.15.39:0/3992471646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572b9917590 0x5572b9d05960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:07.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.149+0000 7fa348a00640 1 --2- 172.21.15.39:0/3992471646 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5572b9938270 0x5572b9cd9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:07.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.149+0000 7fa343e00640 1 -- 172.21.15.39:0/3992471646 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5572b9938270 msgr2=0x5572b9cd9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:07.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.149+0000 7fa343e00640 1 --2- 172.21.15.39:0/3992471646 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5572b9938270 0x5572b9cd9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:07.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.149+0000 7fa343e00640 1 -- 172.21.15.39:0/3992471646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572b9917590 msgr2=0x5572b9d05960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:07.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.149+0000 7fa343e00640 1 --2- 172.21.15.39:0/3992471646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572b9917590 0x5572b9d05960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:07.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.149+0000 7fa343e00640 1 -- 172.21.15.39:0/3992471646 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572b99a3850 con 0x5572b990f3b0 2024-09-21T07:49:07.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.150+0000 7fa343e00640 1 --2- 172.21.15.39:0/3992471646 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572b990f3b0 0x5572b9d36a10 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7fa334011010 tx=0x7fa3340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fde8d8198bfd684 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:07.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.150+0000 7fa330c00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:49:07.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.150+0000 7fa330c00640 1 -- 172.21.15.39:0/3992471646 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa334020840 con 0x5572b990f3b0 2024-09-21T07:49:07.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.150+0000 7fa330c00640 1 -- 172.21.15.39:0/3992471646 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(221..221 src has 1..221) ==== 7075+0+0 (secure 0 0 0) 0x7fa3340209e0 con 0x5572b990f3b0 2024-09-21T07:49:07.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.150+0000 7fa351bcb740 1 -- 172.21.15.39:0/3992471646 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa32c00d940 con 0x5572b990f3b0 2024-09-21T07:49:07.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.150+0000 7fa330c00640 1 -- 172.21.15.39:0/3992471646 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=233) ==== 24+0+0 (secure 0 0 0) 0x7fa334030050 con 0x5572b990f3b0 2024-09-21T07:49:07.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.150+0000 7fa351bcb740 10 client.10455 fetch_fsmap learned FSMap version 233 2024-09-21T07:49:07.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.150+0000 7fa351bcb740 1 -- 172.21.15.39:0/3992471646 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=233,monmap=2+}) -- 0x5572b9a1a6b0 con 0x5572b990f3b0 2024-09-21T07:49:07.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.151+0000 7fa330c00640 1 -- 172.21.15.39:0/3992471646 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa32c00d940 con 0x5572b990f3b0 2024-09-21T07:49:07.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.151+0000 7fa330c00640 1 -- 172.21.15.39:0/3992471646 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 233) ==== 3522+0+0 (secure 0 0 0) 0x7fa33400f040 con 0x5572b990f3b0 2024-09-21T07:49:07.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.151+0000 7fa351bcb740 10 client.10455 fetch_fsmap finished waiting for FSMap version 233 2024-09-21T07:49:07.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.151+0000 7fa351bcb740 10 client.10455 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:49:07.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.151+0000 7fa351bcb740 20 client.10455 populate_metadata read hostname 'smithi039' 2024-09-21T07:49:07.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.151+0000 7fa351bcb740 1 --2- 172.21.15.39:0/3992471646 >> [v2:172.21.15.39:6832/2837221709,v1:172.21.15.39:6834/2837221709] conn(0x5572b98f6e40 0x5572b9d34180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:07.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.151+0000 7fa351bcb740 4 client.10455 mds_command: new command op to 10224 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:49:07.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.151+0000 7fa351bcb740 1 -- 172.21.15.39:0/3992471646 --> [v2:172.21.15.39:6832/2837221709,v1:172.21.15.39:6834/2837221709] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572b9c54520 con 0x5572b98f6e40 2024-09-21T07:49:07.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.151+0000 7fa33b400640 1 --2- 172.21.15.39:0/3992471646 >> [v2:172.21.15.39:6832/2837221709,v1:172.21.15.39:6834/2837221709] conn(0x5572b98f6e40 0x5572b9d34180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:07.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.152+0000 7fa33b400640 1 --2- 172.21.15.39:0/3992471646 >> [v2:172.21.15.39:6832/2837221709,v1:172.21.15.39:6834/2837221709] conn(0x5572b98f6e40 0x5572b9d34180 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec4f7554f8062e1a server_cookie=77e5a8832d9fc3be in_seq=0 out_seq=0 2024-09-21T07:49:07.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.152+0000 7fa330c00640 10 client.10455 ms_handle_connect on v2:172.21.15.39:6832/2837221709 2024-09-21T07:49:07.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.152+0000 7fa330c00640 1 -- 172.21.15.39:0/3992471646 <== mds.0 v2:172.21.15.39:6832/2837221709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5572b9c54520 con 0x5572b98f6e40 2024-09-21T07:49:07.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.152+0000 7fa330c00640 10 client.10455 handle_command_reply: tid=0 2024-09-21T07:49:07.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.152+0000 7fa351bcb740 1 client.10455 shutdown 2024-09-21T07:49:07.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.153+0000 7fa351bcb740 1 -- 172.21.15.39:0/3992471646 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572b990f3b0 msgr2=0x5572b9d36a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:07.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.153+0000 7fa351bcb740 1 --2- 172.21.15.39:0/3992471646 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572b990f3b0 0x5572b9d36a10 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7fa334011010 tx=0x7fa3340046c0 comp rx=0 tx=0).stop 2024-09-21T07:49:07.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.153+0000 7fa351bcb740 1 -- 172.21.15.39:0/3992471646 shutdown_connections 2024-09-21T07:49:07.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.153+0000 7fa351bcb740 1 --2- 172.21.15.39:0/3992471646 >> [v2:172.21.15.39:6832/2837221709,v1:172.21.15.39:6834/2837221709] conn(0x5572b98f6e40 0x5572b9d34180 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:07.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.153+0000 7fa351bcb740 1 --2- 172.21.15.39:0/3992471646 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5572b9938270 0x5572b9cd9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:07.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.153+0000 7fa351bcb740 1 --2- 172.21.15.39:0/3992471646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572b9917590 0x5572b9d05960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:07.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.153+0000 7fa351bcb740 1 --2- 172.21.15.39:0/3992471646 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572b990f3b0 0x5572b9d36a10 unknown :-1 s=CLOSED pgs=2148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:07.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.153+0000 7fa330c00640 0 client.10455 ms_handle_reset on v2:172.21.15.39:6832/2837221709 2024-09-21T07:49:07.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.153+0000 7fa351bcb740 1 -- 172.21.15.39:0/3992471646 >> 172.21.15.39:0/3992471646 conn(0x5572b992c3c0 msgr2=0x5572b9d23fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:07.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.153+0000 7fa351bcb740 1 -- 172.21.15.39:0/3992471646 shutdown_connections 2024-09-21T07:49:07.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.153+0000 7fa351bcb740 1 -- 172.21.15.39:0/3992471646 wait complete. 2024-09-21T07:49:07.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.153+0000 7fa351bcb740 20 client.10455 trim_cache size 0 max 16384 2024-09-21T07:49:07.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.172+0000 7fa351bcb740 1 Processor -- start 2024-09-21T07:49:07.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.173+0000 7fa351bcb740 1 -- start start 2024-09-21T07:49:07.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.174+0000 7fa351bcb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572b990f3b0 0x5572b9d35f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:07.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.174+0000 7fa351bcb740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5572b9917590 0x5572b9d2fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:07.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.174+0000 7fa351bcb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572b9938270 0x5572b9d2ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:07.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.174+0000 7fa351bcb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa344204cf0 con 0x5572b9938270 2024-09-21T07:49:07.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.174+0000 7fa351bcb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa344202f60 con 0x5572b990f3b0 2024-09-21T07:49:07.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.174+0000 7fa351bcb740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa3442035b0 con 0x5572b9917590 2024-09-21T07:49:07.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.174+0000 7fa348a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572b9938270 0x5572b9d2ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:07.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.174+0000 7fa348a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572b9938270 0x5572b9d2ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36106/0 (socket says 172.21.15.39:36106) 2024-09-21T07:49:07.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.174+0000 7fa348a00640 1 -- 172.21.15.39:0/2402403057 learned_addr learned my addr 172.21.15.39:0/2402403057 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:07.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.174+0000 7fa33b400640 1 --2- 172.21.15.39:0/2402403057 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5572b9917590 0x5572b9d2fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:07.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.175+0000 7fa343e00640 1 --2- 172.21.15.39:0/2402403057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572b990f3b0 0x5572b9d35f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:07.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.175+0000 7fa348a00640 1 -- 172.21.15.39:0/2402403057 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5572b9917590 msgr2=0x5572b9d2fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:07.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.175+0000 7fa348a00640 1 --2- 172.21.15.39:0/2402403057 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5572b9917590 0x5572b9d2fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:07.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.175+0000 7fa348a00640 1 -- 172.21.15.39:0/2402403057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572b990f3b0 msgr2=0x5572b9d35f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:07.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.175+0000 7fa348a00640 1 --2- 172.21.15.39:0/2402403057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572b990f3b0 0x5572b9d35f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:07.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.175+0000 7fa348a00640 1 -- 172.21.15.39:0/2402403057 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572b9cdf650 con 0x5572b9938270 2024-09-21T07:49:07.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.175+0000 7fa348a00640 1 --2- 172.21.15.39:0/2402403057 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572b9938270 0x5572b9d2ff70 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7fa33c080a80 tx=0x7fa33c02c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fda213ed4113b89 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:07.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.175+0000 7fa333400640 1 -- 172.21.15.39:0/2402403057 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c083f10 con 0x5572b9938270 2024-09-21T07:49:07.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.175+0000 7fa333400640 1 -- 172.21.15.39:0/2402403057 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa33c083810 con 0x5572b9938270 2024-09-21T07:49:07.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.175+0000 7fa333400640 1 -- 172.21.15.39:0/2402403057 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c08cf70 con 0x5572b9938270 2024-09-21T07:49:07.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.176+0000 7fa351bcb740 1 -- 172.21.15.39:0/2402403057 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572b9938270 msgr2=0x5572b9d2ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:07.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.176+0000 7fa351bcb740 1 --2- 172.21.15.39:0/2402403057 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572b9938270 0x5572b9d2ff70 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7fa33c080a80 tx=0x7fa33c02c040 comp rx=0 tx=0).stop 2024-09-21T07:49:07.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.176+0000 7fa351bcb740 1 -- 172.21.15.39:0/2402403057 shutdown_connections 2024-09-21T07:49:07.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.176+0000 7fa351bcb740 1 --2- 172.21.15.39:0/2402403057 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572b9938270 0x5572b9d2ff70 unknown :-1 s=CLOSED pgs=2149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:07.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.176+0000 7fa351bcb740 1 --2- 172.21.15.39:0/2402403057 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5572b9917590 0x5572b9d2fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:07.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.176+0000 7fa351bcb740 1 --2- 172.21.15.39:0/2402403057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572b990f3b0 0x5572b9d35f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:07.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.176+0000 7fa351bcb740 1 -- 172.21.15.39:0/2402403057 >> 172.21.15.39:0/2402403057 conn(0x5572b992c3c0 msgr2=0x5572b9d1dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:07.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.176+0000 7fa351bcb740 1 -- 172.21.15.39:0/2402403057 shutdown_connections 2024-09-21T07:49:07.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.176+0000 7fa351bcb740 1 -- 172.21.15.39:0/2402403057 wait complete. 2024-09-21T07:49:07.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.177+0000 7fa351bcb740 1 -- start start 2024-09-21T07:49:07.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.177+0000 7fa351bcb740 1 Processor -- start 2024-09-21T07:49:07.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.179+0000 7fa351bcb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572b990f3b0 0x5572b9cd7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:07.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.179+0000 7fa351bcb740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5572b9917590 0x5572b9cde9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:07.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.179+0000 7fa351bcb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572b9938270 0x5572b9da3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:07.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.179+0000 7fa351bcb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5572b9d35180 con 0x5572b990f3b0 2024-09-21T07:49:07.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.179+0000 7fa351bcb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5572b9d338a0 con 0x5572b9938270 2024-09-21T07:49:07.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.179+0000 7fa351bcb740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x5572b9d01f50 con 0x5572b9917590 2024-09-21T07:49:07.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.179+0000 7fa351bcb740 10 client.0 _ll_register_callbacks cb 0x7fa334010c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:49:07.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.179+0000 7fa343e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572b990f3b0 0x5572b9cd7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:07.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.179+0000 7fa343e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572b990f3b0 0x5572b9cd7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36112/0 (socket says 172.21.15.39:36112) 2024-09-21T07:49:07.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.179+0000 7fa343e00640 1 -- 172.21.15.39:0/3516677390 learned_addr learned my addr 172.21.15.39:0/3516677390 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:07.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.179+0000 7fa348a00640 1 --2- 172.21.15.39:0/3516677390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572b9938270 0x5572b9da3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:07.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.179+0000 7fa343e00640 1 -- 172.21.15.39:0/3516677390 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5572b9917590 msgr2=0x5572b9cde9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:07.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.179+0000 7fa33b400640 1 --2- 172.21.15.39:0/3516677390 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5572b9917590 0x5572b9cde9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:07.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.180+0000 7fa343e00640 1 --2- 172.21.15.39:0/3516677390 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5572b9917590 0x5572b9cde9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:07.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.180+0000 7fa343e00640 1 -- 172.21.15.39:0/3516677390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572b9938270 msgr2=0x5572b9da3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:07.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.180+0000 7fa343e00640 1 --2- 172.21.15.39:0/3516677390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572b9938270 0x5572b9da3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:07.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.180+0000 7fa343e00640 1 -- 172.21.15.39:0/3516677390 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572b9cd81a0 con 0x5572b990f3b0 2024-09-21T07:49:07.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.180+0000 7fa348a00640 1 --2- 172.21.15.39:0/3516677390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572b9938270 0x5572b9da3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:49:07.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.180+0000 7fa33b400640 1 --2- 172.21.15.39:0/3516677390 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5572b9917590 0x5572b9cde9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:49:07.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.180+0000 7fa343e00640 1 --2- 172.21.15.39:0/3516677390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572b990f3b0 0x5572b9cd7c60 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7fa334011010 tx=0x7fa3340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8076c85c6adf4de8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:07.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.180+0000 7fa330c00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:49:07.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.180+0000 7fa330c00640 1 -- 172.21.15.39:0/3516677390 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa334020a50 con 0x5572b990f3b0 2024-09-21T07:49:07.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.180+0000 7fa330c00640 1 -- 172.21.15.39:0/3516677390 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(221..221 src has 1..221) ==== 7075+0+0 (secure 0 0 0) 0x7fa334005020 con 0x5572b990f3b0 2024-09-21T07:49:07.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.180+0000 7fa351bcb740 1 -- 172.21.15.39:0/3516677390 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa33c083810 con 0x5572b990f3b0 2024-09-21T07:49:07.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.180+0000 7fa330c00640 1 -- 172.21.15.39:0/3516677390 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=233) ==== 24+0+0 (secure 0 0 0) 0x7fa334042050 con 0x5572b990f3b0 2024-09-21T07:49:07.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.180+0000 7fa351bcb740 10 client.10461 fetch_fsmap learned FSMap version 233 2024-09-21T07:49:07.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.180+0000 7fa351bcb740 1 -- 172.21.15.39:0/3516677390 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=233,monmap=2+}) -- 0x5572b9da6bb0 con 0x5572b990f3b0 2024-09-21T07:49:07.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.181+0000 7fa330c00640 1 -- 172.21.15.39:0/3516677390 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa33c083810 con 0x5572b990f3b0 2024-09-21T07:49:07.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.181+0000 7fa330c00640 1 -- 172.21.15.39:0/3516677390 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 233) ==== 3522+0+0 (secure 0 0 0) 0x7fa334045020 con 0x5572b990f3b0 2024-09-21T07:49:07.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.181+0000 7fa351bcb740 10 client.10461 fetch_fsmap finished waiting for FSMap version 233 2024-09-21T07:49:07.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.181+0000 7fa351bcb740 10 client.10461 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:49:07.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.181+0000 7fa351bcb740 20 client.10461 populate_metadata read hostname 'smithi039' 2024-09-21T07:49:07.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.181+0000 7fa351bcb740 1 --2- 172.21.15.39:0/3516677390 >> [v2:172.21.15.39:6832/2837221709,v1:172.21.15.39:6834/2837221709] conn(0x5572b9d39020 0x5572b9d33fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:07.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.181+0000 7fa351bcb740 4 client.10461 mds_command: new command op to 10224 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:49:07.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.181+0000 7fa351bcb740 1 -- 172.21.15.39:0/3516677390 --> [v2:172.21.15.39:6832/2837221709,v1:172.21.15.39:6834/2837221709] -- command(tid 0: {"prefix": "status"}) -- 0x5572b9c4ffd0 con 0x5572b9d39020 2024-09-21T07:49:07.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.181+0000 7fa33b400640 1 --2- 172.21.15.39:0/3516677390 >> [v2:172.21.15.39:6832/2837221709,v1:172.21.15.39:6834/2837221709] conn(0x5572b9d39020 0x5572b9d33fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:07.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.182+0000 7fa33b400640 1 --2- 172.21.15.39:0/3516677390 >> [v2:172.21.15.39:6832/2837221709,v1:172.21.15.39:6834/2837221709] conn(0x5572b9d39020 0x5572b9d33fb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7873786d68029bf3 server_cookie=e79df15b462d1505 in_seq=0 out_seq=0 2024-09-21T07:49:07.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.182+0000 7fa330c00640 10 client.10461 ms_handle_connect on v2:172.21.15.39:6832/2837221709 2024-09-21T07:49:07.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.182+0000 7fa330c00640 1 -- 172.21.15.39:0/3516677390 <== mds.0 v2:172.21.15.39:6832/2837221709 1 ==== command_reply(tid 0: 0 ) ==== 8+0+329 (crc 0 0 0) 0x5572b9c4ffd0 con 0x5572b9d39020 2024-09-21T07:49:07.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.182+0000 7fa330c00640 10 client.10461 handle_command_reply: tid=0 2024-09-21T07:49:07.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.182+0000 7fa351bcb740 1 client.10461 shutdown 2024-09-21T07:49:07.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.182+0000 7fa351bcb740 1 -- 172.21.15.39:0/3516677390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572b990f3b0 msgr2=0x5572b9cd7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:07.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.182+0000 7fa351bcb740 1 --2- 172.21.15.39:0/3516677390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572b990f3b0 0x5572b9cd7c60 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7fa334011010 tx=0x7fa3340042f0 comp rx=0 tx=0).stop 2024-09-21T07:49:07.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.182+0000 7fa351bcb740 1 -- 172.21.15.39:0/3516677390 shutdown_connections 2024-09-21T07:49:07.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.182+0000 7fa351bcb740 1 --2- 172.21.15.39:0/3516677390 >> [v2:172.21.15.39:6832/2837221709,v1:172.21.15.39:6834/2837221709] conn(0x5572b9d39020 0x5572b9d33fb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:07.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.182+0000 7fa351bcb740 1 --2- 172.21.15.39:0/3516677390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5572b9938270 0x5572b9da3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:07.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.182+0000 7fa351bcb740 1 --2- 172.21.15.39:0/3516677390 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5572b9917590 0x5572b9cde9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:07.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.182+0000 7fa351bcb740 1 --2- 172.21.15.39:0/3516677390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5572b990f3b0 0x5572b9cd7c60 unknown :-1 s=CLOSED pgs=2150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:07.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.182+0000 7fa330c00640 0 client.10461 ms_handle_reset on v2:172.21.15.39:6832/2837221709 2024-09-21T07:49:07.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.182+0000 7fa351bcb740 1 -- 172.21.15.39:0/3516677390 >> 172.21.15.39:0/3516677390 conn(0x5572b992c3c0 msgr2=0x5572b9d1dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:07.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.182+0000 7fa351bcb740 1 -- 172.21.15.39:0/3516677390 shutdown_connections 2024-09-21T07:49:07.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.182+0000 7fa351bcb740 1 -- 172.21.15.39:0/3516677390 wait complete. 2024-09-21T07:49:07.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.183+0000 7fa351bcb740 20 client.10461 trim_cache size 0 max 16384 2024-09-21T07:49:07.188 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:49:07.188 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:49:07.188 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-09-21T07:49:07.188 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 10224, 2024-09-21T07:49:07.188 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:49:07.188 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:49:07.188 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:49:07.188 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 19.597069767000001, 2024-09-21T07:49:07.188 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 233, 2024-09-21T07:49:07.188 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 221, 2024-09-21T07:49:07.188 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 221, 2024-09-21T07:49:07.189 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 24.661428419 2024-09-21T07:49:07.189 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:49:07.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.183+0000 7fa34aa00640 1 -- 172.21.15.39:0/1593511634 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa320078b90 msgr2=0x7fa32007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:07.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.183+0000 7fa34aa00640 1 --2- 172.21.15.39:0/1593511634 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa320078b90 0x7fa32007b0a0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fa334009ab0 tx=0x7fa334005f90 comp rx=0 tx=0).stop 2024-09-21T07:49:07.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.183+0000 7fa34aa00640 1 -- 172.21.15.39:0/1593511634 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa34415d830 msgr2=0x7fa3440c6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:07.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.183+0000 7fa34aa00640 1 --2- 172.21.15.39:0/1593511634 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa34415d830 0x7fa3440c6870 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7fa33c035040 tx=0x7fa33c032c60 comp rx=0 tx=0).stop 2024-09-21T07:49:07.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.183+0000 7fa34aa00640 1 -- 172.21.15.39:0/1593511634 shutdown_connections 2024-09-21T07:49:07.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.183+0000 7fa34aa00640 1 --2- 172.21.15.39:0/1593511634 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa320078b90 0x7fa32007b0a0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:07.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.183+0000 7fa34aa00640 1 --2- 172.21.15.39:0/1593511634 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa34415d830 0x7fa3440c6870 unknown :-1 s=CLOSED pgs=2146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:07.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.183+0000 7fa34aa00640 1 --2- 172.21.15.39:0/1593511634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3440ccc60 0x7fa3441f4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:07.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.183+0000 7fa34aa00640 1 --2- 172.21.15.39:0/1593511634 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa3440b30a0 0x7fa3440cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:07.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.183+0000 7fa34aa00640 1 -- 172.21.15.39:0/1593511634 >> 172.21.15.39:0/1593511634 conn(0x7fa3440b2460 msgr2=0x7fa34406f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:07.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.183+0000 7fa34aa00640 1 -- 172.21.15.39:0/1593511634 shutdown_connections 2024-09-21T07:49:07.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07.183+0000 7fa34aa00640 1 -- 172.21.15.39:0/1593511634 wait complete. 2024-09-21T07:49:07.209 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'mkdir -p {d0,d1}/empty') 2024-09-21T07:49:07.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:49:07.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:49:07.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:07.307 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d0) 2024-09-21T07:49:07.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:07.347 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1) 2024-09-21T07:49:07.422 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-21T07:49:07.554 INFO:tasks.ceph.mds.c.smithi039.stderr:2024-09-21T07:49:07.553+0000 7fe9b7000640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:49:07.708 INFO:teuthology.orchestra.run.smithi039.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T07:49:07.709 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T07:49:07.709 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d0', 0), ('/d1', 1)] 2024-09-21T07:49:09.710 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-21T07:49:09.965 INFO:teuthology.orchestra.run.smithi039.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T07:49:09.965 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T07:49:09.965 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d0', 0), ('/d1', 1)] 2024-09-21T07:49:11.966 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-21T07:49:12.222 INFO:teuthology.orchestra.run.smithi039.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T07:49:12.223 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T07:49:12.223 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d0', 0), ('/d1', 1)] 2024-09-21T07:49:14.224 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-21T07:49:14.480 INFO:teuthology.orchestra.run.smithi039.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"14","version":"14","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d0","dirfrag":"0x10000000000","snapid_first":2,"projected_version":"3","version":"3","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1","dirfrag":"0x10000000002","snapid_first":2,"projected_version":"3","version":"3","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-09-21T07:49:14.481 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000000", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/d0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "3", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "3" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000002", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "3", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "3" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-21T07:49:14.481 INFO:tasks.cephfs.cephfs_test_case:[('/d0', 0), ('/d1', 1)] =?= [('/d0', 0), ('/d1', 1)] 2024-09-21T07:49:14.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:14.481 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d0/d3) 2024-09-21T07:49:14.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:14.518 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d0/.snap/s1) 2024-09-21T07:49:14.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:14.626 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls d0/d3/.snap) 2024-09-21T07:49:14.662 INFO:teuthology.orchestra.run.smithi039.stdout:_s1_1099511627776 2024-09-21T07:49:14.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:14.663 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mv d0/d3 d1/d3) 2024-09-21T07:49:14.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:14.758 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls d1/d3/.snap) 2024-09-21T07:49:14.793 INFO:teuthology.orchestra.run.smithi039.stdout:_s1_1099511627776 2024-09-21T07:49:14.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:14.794 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir d0/.snap/s1) 2024-09-21T07:49:14.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:14.878 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls d1/d3/.snap) 2024-09-21T07:49:14.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:14.956 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -rf d0 d1) 2024-09-21T07:49:15.079 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:49:15.080 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:49:15.162 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:49:15.255 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:49:15.256 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:49:15.256 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:49:15.256 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:49:15.292 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:49:15.292 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:49:15.367 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:49:15.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.634+0000 7f4d2b200640 1 -- 172.21.15.39:0/3056082311 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d2c0b59e0 msgr2=0x7f4d2c0b3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:15.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.634+0000 7f4d2b200640 1 --2- 172.21.15.39:0/3056082311 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d2c0b59e0 0x7f4d2c0b3b60 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c0d2570 tx=0x7f4d1802a540 comp rx=0 tx=0).stop 2024-09-21T07:49:15.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.634+0000 7f4d2b200640 1 -- 172.21.15.39:0/3056082311 shutdown_connections 2024-09-21T07:49:15.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.634+0000 7f4d2b200640 1 --2- 172.21.15.39:0/3056082311 >> v2:172.21.15.148:3300/0 conn(0x7f4d2c0d0e60 0x7f4d2c0d1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:15.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.634+0000 7f4d2b200640 1 --2- 172.21.15.39:0/3056082311 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d2c0b59e0 0x7f4d2c0b3b60 unknown :-1 s=CLOSED pgs=2151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:15.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.634+0000 7f4d2b200640 1 --2- 172.21.15.39:0/3056082311 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d2c0d5060 0x7f4d2c0b96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:15.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.634+0000 7f4d2b200640 1 -- 172.21.15.39:0/3056082311 >> 172.21.15.39:0/3056082311 conn(0x7f4d2c09d7d0 msgr2=0x7f4d2c0d4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:15.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.634+0000 7f4d2b200640 1 -- 172.21.15.39:0/3056082311 shutdown_connections 2024-09-21T07:49:15.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.635+0000 7f4d2b200640 1 -- 172.21.15.39:0/3056082311 wait complete. 2024-09-21T07:49:15.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.637+0000 7f4d2b200640 1 Processor -- start 2024-09-21T07:49:15.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.637+0000 7f4d2b200640 1 -- start start 2024-09-21T07:49:15.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.638+0000 7f4d2b200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d2c0b59e0 0x7f4d2c0c7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:15.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.638+0000 7f4d2b200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d2c0d0e60 0x7f4d2c155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:15.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.638+0000 7f4d2b200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d2c0d5060 0x7f4d2c06ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:15.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.638+0000 7f4d2b200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c0d2880 con 0x7f4d2c0d5060 2024-09-21T07:49:15.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.638+0000 7f4d2b200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c0d0320 con 0x7f4d2c0d0e60 2024-09-21T07:49:15.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.638+0000 7f4d2b200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f4d2c0d2a00 con 0x7f4d2c0b59e0 2024-09-21T07:49:15.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.639+0000 7f4d29e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d2c0b59e0 0x7f4d2c0c7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:15.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.639+0000 7f4d2a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d2c0d5060 0x7f4d2c06ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:15.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.639+0000 7f4d29400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d2c0d0e60 0x7f4d2c155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:15.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.639+0000 7f4d29e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d2c0b59e0 0x7f4d2c0c7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.39:47512/0 (socket says 172.21.15.39:47512) 2024-09-21T07:49:15.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.639+0000 7f4d29e00640 1 -- 172.21.15.39:0/4130935862 learned_addr learned my addr 172.21.15.39:0/4130935862 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:15.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.639+0000 7f4d29400640 1 -- 172.21.15.39:0/4130935862 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d2c0b59e0 msgr2=0x7f4d2c0c7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:15.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.639+0000 7f4d29400640 1 --2- 172.21.15.39:0/4130935862 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d2c0b59e0 0x7f4d2c0c7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:15.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.639+0000 7f4d29400640 1 -- 172.21.15.39:0/4130935862 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d2c0d5060 msgr2=0x7f4d2c06ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:15.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.639+0000 7f4d29400640 1 --2- 172.21.15.39:0/4130935862 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d2c0d5060 0x7f4d2c06ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:15.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.639+0000 7f4d29400640 1 -- 172.21.15.39:0/4130935862 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d2c155e50 con 0x7f4d2c0d0e60 2024-09-21T07:49:15.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.639+0000 7f4d2a800640 1 --2- 172.21.15.39:0/4130935862 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d2c0d5060 0x7f4d2c06ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:49:15.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.639+0000 7f4d29e00640 1 --2- 172.21.15.39:0/4130935862 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d2c0b59e0 0x7f4d2c0c7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:49:15.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.640+0000 7f4d29400640 1 --2- 172.21.15.39:0/4130935862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d2c0d0e60 0x7f4d2c155910 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f4d18031000 tx=0x7f4d180038a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:15.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.640+0000 7f4d22000640 1 -- 172.21.15.39:0/4130935862 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1803e030 con 0x7f4d2c0d0e60 2024-09-21T07:49:15.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.640+0000 7f4d22000640 1 -- 172.21.15.39:0/4130935862 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4d18003d90 con 0x7f4d2c0d0e60 2024-09-21T07:49:15.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.640+0000 7f4d22000640 1 -- 172.21.15.39:0/4130935862 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d18035040 con 0x7f4d2c0d0e60 2024-09-21T07:49:15.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.640+0000 7f4d2b200640 1 -- 172.21.15.39:0/4130935862 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d2c0c85e0 con 0x7f4d2c0d0e60 2024-09-21T07:49:15.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.640+0000 7f4d2b200640 1 -- 172.21.15.39:0/4130935862 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d2c0c8ba0 con 0x7f4d2c0d0e60 2024-09-21T07:49:15.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.641+0000 7f4d2b200640 1 -- 172.21.15.39:0/4130935862 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d2c0c8350 con 0x7f4d2c0d0e60 2024-09-21T07:49:15.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.643+0000 7f4d22000640 1 -- 172.21.15.39:0/4130935862 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4d18041030 con 0x7f4d2c0d0e60 2024-09-21T07:49:15.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.644+0000 7f4d22000640 1 --2- 172.21.15.39:0/4130935862 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4d0c078b90 0x7f4d0c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:15.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.644+0000 7f4d22000640 1 -- 172.21.15.39:0/4130935862 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(221..221 src has 1..221) ==== 7075+0+0 (secure 0 0 0) 0x7f4d180bc070 con 0x7f4d2c0d0e60 2024-09-21T07:49:15.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.644+0000 7f4d29e00640 1 --2- 172.21.15.39:0/4130935862 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4d0c078b90 0x7f4d0c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:15.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.645+0000 7f4d29e00640 1 --2- 172.21.15.39:0/4130935862 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4d0c078b90 0x7f4d0c07b0a0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c005860 tx=0x7f4d1c00b390 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:15.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.648+0000 7f4d22000640 1 -- 172.21.15.39:0/4130935862 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4d180838e0 con 0x7f4d2c0d0e60 2024-09-21T07:49:15.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.856+0000 7f4d2b200640 1 -- 172.21.15.39:0/4130935862 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4d2c0b3b60 con 0x7f4d2c0d0e60 2024-09-21T07:49:15.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.857+0000 7f4d22000640 1 -- 172.21.15.39:0/4130935862 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v221) ==== 80+0+0 (secure 0 0 0) 0x7f4d18088310 con 0x7f4d2c0d0e60 2024-09-21T07:49:15.860 INFO:teuthology.orchestra.run.smithi039.stderr:listed 0 entries 2024-09-21T07:49:15.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.860+0000 7f4d2b200640 1 -- 172.21.15.39:0/4130935862 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4d0c078b90 msgr2=0x7f4d0c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:15.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.860+0000 7f4d2b200640 1 --2- 172.21.15.39:0/4130935862 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4d0c078b90 0x7f4d0c07b0a0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c005860 tx=0x7f4d1c00b390 comp rx=0 tx=0).stop 2024-09-21T07:49:15.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.860+0000 7f4d2b200640 1 -- 172.21.15.39:0/4130935862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d2c0d0e60 msgr2=0x7f4d2c155910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:15.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.860+0000 7f4d2b200640 1 --2- 172.21.15.39:0/4130935862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d2c0d0e60 0x7f4d2c155910 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f4d18031000 tx=0x7f4d180038a0 comp rx=0 tx=0).stop 2024-09-21T07:49:15.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.860+0000 7f4d2b200640 1 -- 172.21.15.39:0/4130935862 shutdown_connections 2024-09-21T07:49:15.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.860+0000 7f4d2b200640 1 --2- 172.21.15.39:0/4130935862 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4d0c078b90 0x7f4d0c07b0a0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:15.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.860+0000 7f4d2b200640 1 --2- 172.21.15.39:0/4130935862 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d2c0d5060 0x7f4d2c06ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:15.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.860+0000 7f4d2b200640 1 --2- 172.21.15.39:0/4130935862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d2c0d0e60 0x7f4d2c155910 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:15.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.860+0000 7f4d2b200640 1 --2- 172.21.15.39:0/4130935862 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4d2c0b59e0 0x7f4d2c0c7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:15.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.860+0000 7f4d2b200640 1 -- 172.21.15.39:0/4130935862 >> 172.21.15.39:0/4130935862 conn(0x7f4d2c09d7d0 msgr2=0x7f4d2c0cd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:15.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.860+0000 7f4d2b200640 1 -- 172.21.15.39:0/4130935862 shutdown_connections 2024-09-21T07:49:15.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:15.860+0000 7f4d2b200640 1 -- 172.21.15.39:0/4130935862 wait complete. 2024-09-21T07:49:15.883 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T07:49:15.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:15.883 DEBUG:teuthology.orchestra.run.smithi039:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T07:49:15.948 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T07:49:15.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:15.948 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:49:15.986 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:49:15.986 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:49:15.987 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:49:16.027 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:49:16.027 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:49:16.027 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:49:16.027 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:49:16.062 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:49:16.062 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:49:16.062 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:49:16.062 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:49:16.105 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:49:16.105 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:49:16.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.331+0000 7f1235a00640 1 -- 172.21.15.39:0/2914284704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1214004d40 msgr2=0x7f1214005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:16.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.331+0000 7f1235a00640 1 --2- 172.21.15.39:0/2914284704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1214004d40 0x7f1214005150 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f122800c610 tx=0x7f1214005830 comp rx=0 tx=0).stop 2024-09-21T07:49:16.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.331+0000 7f1235000640 1 -- 172.21.15.39:0/2914284704 reap_dead start 2024-09-21T07:49:16.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.331+0000 7f1235a00640 1 -- 172.21.15.39:0/2914284704 shutdown_connections 2024-09-21T07:49:16.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.331+0000 7f1235a00640 1 -- 172.21.15.39:0/2914284704 >> 172.21.15.39:0/2914284704 conn(0x7f123006cfc0 msgr2=0x7f123006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:16.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.331+0000 7f1235a00640 1 -- 172.21.15.39:0/2914284704 shutdown_connections 2024-09-21T07:49:16.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.331+0000 7f1235a00640 1 -- 172.21.15.39:0/2914284704 wait complete. 2024-09-21T07:49:16.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.333+0000 7f1235a00640 1 Processor -- start 2024-09-21T07:49:16.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.333+0000 7f1235a00640 1 -- start start 2024-09-21T07:49:16.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.335+0000 7f1235a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1214004d40 0x7f1230145cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:16.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.335+0000 7f1235a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1230150fd0 0x7f12300c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:16.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.335+0000 7f1235a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f12300c5850 0x7f12300c5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:16.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.335+0000 7f1235a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f12300b3d90 con 0x7f1230150fd0 2024-09-21T07:49:16.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.335+0000 7f1235a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f12300b3c10 con 0x7f1214004d40 2024-09-21T07:49:16.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.335+0000 7f1235a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f12300b3f10 con 0x7f12300c5850 2024-09-21T07:49:16.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.335+0000 7f1227e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1230150fd0 0x7f12300c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:16.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.335+0000 7f1227e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1230150fd0 0x7f12300c9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59410/0 (socket says 172.21.15.39:59410) 2024-09-21T07:49:16.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.335+0000 7f1227e00640 1 -- 172.21.15.39:0/1399198030 learned_addr learned my addr 172.21.15.39:0/1399198030 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:16.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.335+0000 7f122fe00640 1 --2- 172.21.15.39:0/1399198030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1214004d40 0x7f1230145cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:16.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.336+0000 7f1227e00640 1 -- 172.21.15.39:0/1399198030 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f12300c5850 msgr2=0x7f12300c5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:16.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.336+0000 7f1235000640 1 --2- 172.21.15.39:0/1399198030 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f12300c5850 0x7f12300c5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:16.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.336+0000 7f1227e00640 1 --2- 172.21.15.39:0/1399198030 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f12300c5850 0x7f12300c5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:16.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.336+0000 7f1227e00640 1 -- 172.21.15.39:0/1399198030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1214004d40 msgr2=0x7f1230145cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:16.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.336+0000 7f1227e00640 1 --2- 172.21.15.39:0/1399198030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1214004d40 0x7f1230145cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:16.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.336+0000 7f1227e00640 1 -- 172.21.15.39:0/1399198030 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12300ca840 con 0x7f1230150fd0 2024-09-21T07:49:16.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.336+0000 7f1235000640 1 --2- 172.21.15.39:0/1399198030 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f12300c5850 0x7f12300c5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:49:16.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.336+0000 7f1227e00640 1 --2- 172.21.15.39:0/1399198030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1230150fd0 0x7f12300c9ed0 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f1218011ab0 tx=0x7f1218011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:16.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.336+0000 7f122cc00640 1 -- 172.21.15.39:0/1399198030 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1218012450 con 0x7f1230150fd0 2024-09-21T07:49:16.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.336+0000 7f122cc00640 1 -- 172.21.15.39:0/1399198030 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1218016070 con 0x7f1230150fd0 2024-09-21T07:49:16.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.336+0000 7f1235a00640 1 -- 172.21.15.39:0/1399198030 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12300cab20 con 0x7f1230150fd0 2024-09-21T07:49:16.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.336+0000 7f122cc00640 1 -- 172.21.15.39:0/1399198030 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1218012980 con 0x7f1230150fd0 2024-09-21T07:49:16.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.336+0000 7f1235a00640 1 -- 172.21.15.39:0/1399198030 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12300cb0e0 con 0x7f1230150fd0 2024-09-21T07:49:16.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.337+0000 7f1235a00640 1 -- 172.21.15.39:0/1399198030 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12300ca5b0 con 0x7f1230150fd0 2024-09-21T07:49:16.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.339+0000 7f122cc00640 1 -- 172.21.15.39:0/1399198030 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1218018030 con 0x7f1230150fd0 2024-09-21T07:49:16.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.340+0000 7f122cc00640 1 --2- 172.21.15.39:0/1399198030 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f120c078bc0 0x7f120c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:16.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.340+0000 7f122cc00640 1 -- 172.21.15.39:0/1399198030 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(221..221 src has 1..221) ==== 7075+0+0 (secure 0 0 0) 0x7f121809bdf0 con 0x7f1230150fd0 2024-09-21T07:49:16.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.341+0000 7f122fe00640 1 --2- 172.21.15.39:0/1399198030 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f120c078bc0 0x7f120c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:16.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.341+0000 7f122fe00640 1 --2- 172.21.15.39:0/1399198030 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f120c078bc0 0x7f120c07b0d0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f1220009b10 tx=0x7f1220005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:16.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.345+0000 7f122cc00640 1 -- 172.21.15.39:0/1399198030 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1218063c40 con 0x7f1230150fd0 2024-09-21T07:49:16.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.580+0000 7f1235a00640 1 -- 172.21.15.39:0/1399198030 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f123015bf10 con 0x7f1230150fd0 2024-09-21T07:49:16.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.581+0000 7f122cc00640 1 -- 172.21.15.39:0/1399198030 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 234 v234) ==== 95+0+5261 (secure 0 0 0) 0x7f121806a020 con 0x7f1230150fd0 2024-09-21T07:49:16.583 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:49:16.583 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":234,"btime":"2024-09-21T07:49:12:931490+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8249,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/550771731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":550771731},{"type":"v1","addr":"172.21.15.148:6837","nonce":550771731}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":224},{"gid":9385,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1307677219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":1307677219},{"type":"v1","addr":"172.21.15.39:6835","nonce":1307677219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":224}],"filesystems":[{"mdsmap":{"epoch":234,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:48:47.106474+0000","modified":"2024-09-21T07:49:12.526299+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":10224,"mds_1":9391},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9391":{"gid":9391,"name":"b","rank":1,"incarnation":232,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6836/1256253563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1256253563},{"type":"v1","addr":"172.21.15.148:6836","nonce":1256253563}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10224":{"gid":10224,"name":"c","rank":0,"incarnation":225,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/2837221709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2837221709},{"type":"v1","addr":"172.21.15.39:6834","nonce":2837221709}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10224,"qdb_cluster":[10224,9391]},"id":12}]} 2024-09-21T07:49:16.583 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 234 2024-09-21T07:49:16.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.583+0000 7f1235a00640 1 -- 172.21.15.39:0/1399198030 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f120c078bc0 msgr2=0x7f120c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:16.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.583+0000 7f1235a00640 1 --2- 172.21.15.39:0/1399198030 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f120c078bc0 0x7f120c07b0d0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f1220009b10 tx=0x7f1220005f90 comp rx=0 tx=0).stop 2024-09-21T07:49:16.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.583+0000 7f1235a00640 1 -- 172.21.15.39:0/1399198030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1230150fd0 msgr2=0x7f12300c9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:16.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.583+0000 7f1235a00640 1 --2- 172.21.15.39:0/1399198030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1230150fd0 0x7f12300c9ed0 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f1218011ab0 tx=0x7f1218011f70 comp rx=0 tx=0).stop 2024-09-21T07:49:16.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.584+0000 7f1235a00640 1 -- 172.21.15.39:0/1399198030 shutdown_connections 2024-09-21T07:49:16.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.584+0000 7f1235a00640 1 --2- 172.21.15.39:0/1399198030 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f120c078bc0 0x7f120c07b0d0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:16.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.584+0000 7f1235a00640 1 --2- 172.21.15.39:0/1399198030 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f12300c5850 0x7f12300c5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:16.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.584+0000 7f1235a00640 1 --2- 172.21.15.39:0/1399198030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1230150fd0 0x7f12300c9ed0 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:16.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.584+0000 7f1235a00640 1 --2- 172.21.15.39:0/1399198030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1214004d40 0x7f1230145cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:16.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.584+0000 7f1235a00640 1 -- 172.21.15.39:0/1399198030 >> 172.21.15.39:0/1399198030 conn(0x7f123006cfc0 msgr2=0x7f1230151b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:16.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.584+0000 7f1235a00640 1 -- 172.21.15.39:0/1399198030 shutdown_connections 2024-09-21T07:49:16.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.584+0000 7f1235a00640 1 -- 172.21.15.39:0/1399198030 wait complete. 2024-09-21T07:49:16.612 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:49:16.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.836+0000 7f3509200640 1 -- 172.21.15.39:0/3158018464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35040b30a0 msgr2=0x7f35040cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:16.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.836+0000 7f3509200640 1 --2- 172.21.15.39:0/3158018464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35040b30a0 0x7f35040cb8a0 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f34f4005fd0 tx=0x7f34f402a4a0 comp rx=0 tx=0).stop 2024-09-21T07:49:16.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.836+0000 7f3509200640 1 -- 172.21.15.39:0/3158018464 shutdown_connections 2024-09-21T07:49:16.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.836+0000 7f3509200640 1 --2- 172.21.15.39:0/3158018464 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f350415d830 0x7f3504156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:16.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.836+0000 7f3509200640 1 --2- 172.21.15.39:0/3158018464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35040b30a0 0x7f35040cb8a0 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:16.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.836+0000 7f3509200640 1 -- 172.21.15.39:0/3158018464 >> 172.21.15.39:0/3158018464 conn(0x7f35040b2460 msgr2=0x7f350406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:16.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.836+0000 7f3509200640 1 -- 172.21.15.39:0/3158018464 shutdown_connections 2024-09-21T07:49:16.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.836+0000 7f3509200640 1 -- 172.21.15.39:0/3158018464 wait complete. 2024-09-21T07:49:16.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.838+0000 7f3509200640 1 Processor -- start 2024-09-21T07:49:16.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.838+0000 7f3509200640 1 -- start start 2024-09-21T07:49:16.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.840+0000 7f3509200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35040b30a0 0x7f35040cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:16.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.840+0000 7f3509200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35040ccc60 0x7f35041556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:16.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.840+0000 7f3509200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f350415d830 0x7f35041e93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:16.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.840+0000 7f3509200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f35040af450 con 0x7f35040b30a0 2024-09-21T07:49:16.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.840+0000 7f3509200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f35040af2d0 con 0x7f35040ccc60 2024-09-21T07:49:16.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.840+0000 7f3509200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f35040af5d0 con 0x7f350415d830 2024-09-21T07:49:16.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.840+0000 7f3501e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35040b30a0 0x7f35040cfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:16.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.840+0000 7f3501e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35040b30a0 0x7f35040cfc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59438/0 (socket says 172.21.15.39:59438) 2024-09-21T07:49:16.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.840+0000 7f3501e00640 1 -- 172.21.15.39:0/2798516203 learned_addr learned my addr 172.21.15.39:0/2798516203 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:16.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.840+0000 7f3502800640 1 --2- 172.21.15.39:0/2798516203 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f350415d830 0x7f35041e93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:16.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.840+0000 7f34f9400640 1 --2- 172.21.15.39:0/2798516203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35040ccc60 0x7f35041556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:16.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.841+0000 7f3501e00640 1 -- 172.21.15.39:0/2798516203 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f350415d830 msgr2=0x7f35041e93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:16.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.841+0000 7f3501e00640 1 --2- 172.21.15.39:0/2798516203 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f350415d830 0x7f35041e93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:16.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.841+0000 7f3501e00640 1 -- 172.21.15.39:0/2798516203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35040ccc60 msgr2=0x7f35041556d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:16.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.841+0000 7f3501e00640 1 --2- 172.21.15.39:0/2798516203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35040ccc60 0x7f35041556d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:16.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.841+0000 7f3501e00640 1 -- 172.21.15.39:0/2798516203 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35040cf7f0 con 0x7f35040b30a0 2024-09-21T07:49:16.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.841+0000 7f3501e00640 1 --2- 172.21.15.39:0/2798516203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35040b30a0 0x7f35040cfc20 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7f34f4034040 tx=0x7f34f4003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:16.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.841+0000 7f34faa00640 1 -- 172.21.15.39:0/2798516203 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f403e030 con 0x7f35040b30a0 2024-09-21T07:49:16.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.841+0000 7f3509200640 1 -- 172.21.15.39:0/2798516203 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35041f5a80 con 0x7f35040b30a0 2024-09-21T07:49:16.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.841+0000 7f34faa00640 1 -- 172.21.15.39:0/2798516203 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34f40047a0 con 0x7f35040b30a0 2024-09-21T07:49:16.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.841+0000 7f34faa00640 1 -- 172.21.15.39:0/2798516203 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f4035040 con 0x7f35040b30a0 2024-09-21T07:49:16.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.841+0000 7f3509200640 1 -- 172.21.15.39:0/2798516203 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35041f5790 con 0x7f35040b30a0 2024-09-21T07:49:16.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.842+0000 7f3509200640 1 -- 172.21.15.39:0/2798516203 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35040afab0 con 0x7f35040b30a0 2024-09-21T07:49:16.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.844+0000 7f34faa00640 1 -- 172.21.15.39:0/2798516203 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f34f4041030 con 0x7f35040b30a0 2024-09-21T07:49:16.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.845+0000 7f34faa00640 1 --2- 172.21.15.39:0/2798516203 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f34e0078bc0 0x7f34e007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:16.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.845+0000 7f34faa00640 1 -- 172.21.15.39:0/2798516203 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(221..221 src has 1..221) ==== 7075+0+0 (secure 0 0 0) 0x7f34f40bba80 con 0x7f35040b30a0 2024-09-21T07:49:16.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.845+0000 7f34f9400640 1 --2- 172.21.15.39:0/2798516203 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f34e0078bc0 0x7f34e007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:16.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.846+0000 7f34f9400640 1 --2- 172.21.15.39:0/2798516203 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f34e0078bc0 0x7f34e007b0d0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f350409e360 tx=0x7f34ec009060 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:16.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:16.849+0000 7f34faa00640 1 -- 172.21.15.39:0/2798516203 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f34f40bde20 con 0x7f35040b30a0 2024-09-21T07:49:17.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.083+0000 7f3509200640 1 -- 172.21.15.39:0/2798516203 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f350409e3b0 con 0x7f35040b30a0 2024-09-21T07:49:17.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.083+0000 7f34faa00640 1 -- 172.21.15.39:0/2798516203 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 234 v234) ==== 95+0+5261 (secure 0 0 0) 0x7f34f4083310 con 0x7f35040b30a0 2024-09-21T07:49:17.085 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:49:17.085 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":234,"btime":"2024-09-21T07:49:12:931490+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8249,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/550771731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":550771731},{"type":"v1","addr":"172.21.15.148:6837","nonce":550771731}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":224},{"gid":9385,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1307677219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":1307677219},{"type":"v1","addr":"172.21.15.39:6835","nonce":1307677219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":224}],"filesystems":[{"mdsmap":{"epoch":234,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:48:47.106474+0000","modified":"2024-09-21T07:49:12.526299+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":10224,"mds_1":9391},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9391":{"gid":9391,"name":"b","rank":1,"incarnation":232,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6836/1256253563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1256253563},{"type":"v1","addr":"172.21.15.148:6836","nonce":1256253563}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10224":{"gid":10224,"name":"c","rank":0,"incarnation":225,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/2837221709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2837221709},{"type":"v1","addr":"172.21.15.39:6834","nonce":2837221709}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10224,"qdb_cluster":[10224,9391]},"id":12}]} 2024-09-21T07:49:17.086 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 234 2024-09-21T07:49:17.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.086+0000 7f3509200640 1 -- 172.21.15.39:0/2798516203 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f34e0078bc0 msgr2=0x7f34e007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:17.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.086+0000 7f3509200640 1 --2- 172.21.15.39:0/2798516203 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f34e0078bc0 0x7f34e007b0d0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f350409e360 tx=0x7f34ec009060 comp rx=0 tx=0).stop 2024-09-21T07:49:17.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.086+0000 7f3509200640 1 -- 172.21.15.39:0/2798516203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35040b30a0 msgr2=0x7f35040cfc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:17.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.086+0000 7f3509200640 1 --2- 172.21.15.39:0/2798516203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35040b30a0 0x7f35040cfc20 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7f34f4034040 tx=0x7f34f4003970 comp rx=0 tx=0).stop 2024-09-21T07:49:17.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.086+0000 7f3509200640 1 -- 172.21.15.39:0/2798516203 shutdown_connections 2024-09-21T07:49:17.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.086+0000 7f3509200640 1 --2- 172.21.15.39:0/2798516203 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f34e0078bc0 0x7f34e007b0d0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:17.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.086+0000 7f3509200640 1 --2- 172.21.15.39:0/2798516203 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f350415d830 0x7f35041e93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:17.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.086+0000 7f3509200640 1 --2- 172.21.15.39:0/2798516203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35040ccc60 0x7f35041556d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:17.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.086+0000 7f3509200640 1 --2- 172.21.15.39:0/2798516203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35040b30a0 0x7f35040cfc20 unknown :-1 s=CLOSED pgs=2156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:17.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.086+0000 7f3509200640 1 -- 172.21.15.39:0/2798516203 >> 172.21.15.39:0/2798516203 conn(0x7f35040b2460 msgr2=0x7f35040b34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:17.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.086+0000 7f3509200640 1 -- 172.21.15.39:0/2798516203 shutdown_connections 2024-09-21T07:49:17.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.086+0000 7f3509200640 1 -- 172.21.15.39:0/2798516203 wait complete. 2024-09-21T07:49:17.113 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:49:17.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.339+0000 7f0002000640 1 -- 172.21.15.39:0/382772781 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efffc1556d0 msgr2=0x7efffc0d04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:17.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.339+0000 7f0002000640 1 --2- 172.21.15.39:0/382772781 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efffc1556d0 0x7efffc0d04b0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7efff0005fd0 tx=0x7efff002a4b0 comp rx=0 tx=0).stop 2024-09-21T07:49:17.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.339+0000 7f0002000640 1 -- 172.21.15.39:0/382772781 shutdown_connections 2024-09-21T07:49:17.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.339+0000 7f0002000640 1 --2- 172.21.15.39:0/382772781 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efffc155ae0 0x7efffc0d09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:17.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.339+0000 7f0002000640 1 --2- 172.21.15.39:0/382772781 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efffc1556d0 0x7efffc0d04b0 unknown :-1 s=CLOSED pgs=2157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:17.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.339+0000 7f0002000640 1 -- 172.21.15.39:0/382772781 >> 172.21.15.39:0/382772781 conn(0x7efffc06cfc0 msgr2=0x7efffc06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:17.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.339+0000 7f0002000640 1 -- 172.21.15.39:0/382772781 shutdown_connections 2024-09-21T07:49:17.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.339+0000 7f0002000640 1 -- 172.21.15.39:0/382772781 wait complete. 2024-09-21T07:49:17.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.341+0000 7f0002000640 1 Processor -- start 2024-09-21T07:49:17.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.341+0000 7f0002000640 1 -- start start 2024-09-21T07:49:17.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.343+0000 7f0002000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efffc1556d0 0x7efffc140b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:17.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.343+0000 7f0002000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efffc155ae0 0x7efffc15a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:17.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.343+0000 7f0002000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efffc162420 0x7efffc15a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:17.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.343+0000 7f0002000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efffc0cf7e0 con 0x7efffc155ae0 2024-09-21T07:49:17.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.343+0000 7f0002000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efffc0cf660 con 0x7efffc162420 2024-09-21T07:49:17.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.343+0000 7f0002000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7efffc0cf960 con 0x7efffc1556d0 2024-09-21T07:49:17.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.343+0000 7efffbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efffc155ae0 0x7efffc15a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:17.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.343+0000 7efffbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efffc155ae0 0x7efffc15a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59458/0 (socket says 172.21.15.39:59458) 2024-09-21T07:49:17.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.343+0000 7efffbe00640 1 -- 172.21.15.39:0/4254942410 learned_addr learned my addr 172.21.15.39:0/4254942410 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:17.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.343+0000 7f0001600640 1 --2- 172.21.15.39:0/4254942410 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efffc162420 0x7efffc15a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:17.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.343+0000 7f0000c00640 1 --2- 172.21.15.39:0/4254942410 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efffc1556d0 0x7efffc140b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:17.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.343+0000 7efffbe00640 1 -- 172.21.15.39:0/4254942410 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efffc1556d0 msgr2=0x7efffc140b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:17.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.343+0000 7efffbe00640 1 --2- 172.21.15.39:0/4254942410 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efffc1556d0 0x7efffc140b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:17.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.343+0000 7efffbe00640 1 -- 172.21.15.39:0/4254942410 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efffc162420 msgr2=0x7efffc15a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:17.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.343+0000 7efffbe00640 1 --2- 172.21.15.39:0/4254942410 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efffc162420 0x7efffc15a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:17.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.343+0000 7efffbe00640 1 -- 172.21.15.39:0/4254942410 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efffc150580 con 0x7efffc155ae0 2024-09-21T07:49:17.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.343+0000 7f0000c00640 1 --2- 172.21.15.39:0/4254942410 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efffc1556d0 0x7efffc140b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:17.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.344+0000 7efffbe00640 1 --2- 172.21.15.39:0/4254942410 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efffc155ae0 0x7efffc15a210 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7effe800d740 tx=0x7effe800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:17.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.344+0000 7efff8c00640 1 -- 172.21.15.39:0/4254942410 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe8017070 con 0x7efffc155ae0 2024-09-21T07:49:17.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.344+0000 7efff8c00640 1 -- 172.21.15.39:0/4254942410 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7effe8009d20 con 0x7efffc155ae0 2024-09-21T07:49:17.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.344+0000 7efff8c00640 1 -- 172.21.15.39:0/4254942410 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe80106c0 con 0x7efffc155ae0 2024-09-21T07:49:17.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.344+0000 7f0002000640 1 -- 172.21.15.39:0/4254942410 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efffc14ffc0 con 0x7efffc155ae0 2024-09-21T07:49:17.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.344+0000 7f0002000640 1 -- 172.21.15.39:0/4254942410 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efffc159a10 con 0x7efffc155ae0 2024-09-21T07:49:17.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.345+0000 7f0002000640 1 -- 172.21.15.39:0/4254942410 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efffc14fd30 con 0x7efffc155ae0 2024-09-21T07:49:17.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.347+0000 7efff8c00640 1 -- 172.21.15.39:0/4254942410 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7effe801a070 con 0x7efffc155ae0 2024-09-21T07:49:17.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.347+0000 7efff8c00640 1 --2- 172.21.15.39:0/4254942410 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7effdc078b90 0x7effdc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:17.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.348+0000 7efff8c00640 1 -- 172.21.15.39:0/4254942410 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(221..221 src has 1..221) ==== 7075+0+0 (secure 0 0 0) 0x7effe809b460 con 0x7efffc155ae0 2024-09-21T07:49:17.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.348+0000 7f0000c00640 1 --2- 172.21.15.39:0/4254942410 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7effdc078b90 0x7effdc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:17.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.349+0000 7f0000c00640 1 --2- 172.21.15.39:0/4254942410 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7effdc078b90 0x7effdc07b0a0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7efff0033000 tx=0x7efff0004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:17.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.352+0000 7efff8c00640 1 -- 172.21.15.39:0/4254942410 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7effe8062c50 con 0x7efffc155ae0 2024-09-21T07:49:17.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.551+0000 7f0002000640 1 -- 172.21.15.39:0/4254942410 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efffc0cfe80 con 0x7efffc155ae0 2024-09-21T07:49:17.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.552+0000 7efff8c00640 1 -- 172.21.15.39:0/4254942410 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v221) ==== 74+0+22650 (secure 0 0 0) 0x7effe8067680 con 0x7efffc155ae0 2024-09-21T07:49:17.555 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:49:17.555 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":221,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:48:55.640995+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:48:45.059810+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"218","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-09-21T07:48:46.068380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"219","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":220,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":220,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"28.9","mappings":[{"from":6,"to":1}]},{"pgid":"28.f","mappings":[{"from":6,"to":4}]},{"pgid":"28.32","mappings":[{"from":6,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]},{"pool":20,"snaps":[{"begin":80,"length":23},{"begin":181,"length":23}]},{"pool":22,"snaps":[{"begin":100,"length":1},{"begin":199,"length":1}]},{"pool":24,"snaps":[{"begin":2,"length":2}]},{"pool":26,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:49:17.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.555+0000 7f0002000640 1 -- 172.21.15.39:0/4254942410 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7effdc078b90 msgr2=0x7effdc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:17.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.555+0000 7f0002000640 1 --2- 172.21.15.39:0/4254942410 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7effdc078b90 0x7effdc07b0a0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7efff0033000 tx=0x7efff0004730 comp rx=0 tx=0).stop 2024-09-21T07:49:17.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.555+0000 7f0002000640 1 -- 172.21.15.39:0/4254942410 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efffc155ae0 msgr2=0x7efffc15a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:17.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.555+0000 7f0002000640 1 --2- 172.21.15.39:0/4254942410 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efffc155ae0 0x7efffc15a210 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7effe800d740 tx=0x7effe800dc00 comp rx=0 tx=0).stop 2024-09-21T07:49:17.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.555+0000 7f0002000640 1 -- 172.21.15.39:0/4254942410 shutdown_connections 2024-09-21T07:49:17.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.555+0000 7f0002000640 1 --2- 172.21.15.39:0/4254942410 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7effdc078b90 0x7effdc07b0a0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:17.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.555+0000 7f0002000640 1 --2- 172.21.15.39:0/4254942410 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efffc162420 0x7efffc15a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:17.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.555+0000 7f0002000640 1 --2- 172.21.15.39:0/4254942410 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efffc155ae0 0x7efffc15a210 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:17.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.555+0000 7f0002000640 1 --2- 172.21.15.39:0/4254942410 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efffc1556d0 0x7efffc140b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:17.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.555+0000 7f0002000640 1 -- 172.21.15.39:0/4254942410 >> 172.21.15.39:0/4254942410 conn(0x7efffc06cfc0 msgr2=0x7efffc15d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:17.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.555+0000 7f0002000640 1 -- 172.21.15.39:0/4254942410 shutdown_connections 2024-09-21T07:49:17.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.555+0000 7f0002000640 1 -- 172.21.15.39:0/4254942410 wait complete. 2024-09-21T07:49:17.579 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T07:49:17.579 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:49:17.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.804+0000 7f9ab3200640 1 -- 172.21.15.39:0/450145696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9aac1539b0 msgr2=0x7f9aac0ce070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:17.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.804+0000 7f9ab3200640 1 --2- 172.21.15.39:0/450145696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9aac1539b0 0x7f9aac0ce070 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0005fd0 tx=0x7f9aa002a5d0 comp rx=0 tx=0).stop 2024-09-21T07:49:17.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.804+0000 7f9ab3200640 1 -- 172.21.15.39:0/450145696 shutdown_connections 2024-09-21T07:49:17.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.804+0000 7f9ab3200640 1 --2- 172.21.15.39:0/450145696 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9aac0cd6a0 0x7f9aac0b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:17.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.804+0000 7f9ab3200640 1 --2- 172.21.15.39:0/450145696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9aac1539b0 0x7f9aac0ce070 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:17.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.804+0000 7f9ab3200640 1 -- 172.21.15.39:0/450145696 >> 172.21.15.39:0/450145696 conn(0x7f9aac0d0cb0 msgr2=0x7f9aac0d10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:17.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.804+0000 7f9ab3200640 1 -- 172.21.15.39:0/450145696 shutdown_connections 2024-09-21T07:49:17.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.805+0000 7f9ab3200640 1 -- 172.21.15.39:0/450145696 wait complete. 2024-09-21T07:49:17.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.807+0000 7f9ab3200640 1 Processor -- start 2024-09-21T07:49:17.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.807+0000 7f9ab3200640 1 -- start start 2024-09-21T07:49:17.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.808+0000 7f9ab3200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9aac1539b0 0x7f9aac143400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:17.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.808+0000 7f9ab3200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9aac15ccd0 0x7f9aac157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:17.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.808+0000 7f9ab3200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9aac1f3590 0x7f9aac1582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:17.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.808+0000 7f9ab3200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9aac0b4e50 con 0x7f9aac1f3590 2024-09-21T07:49:17.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.808+0000 7f9ab3200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9aac0b4cd0 con 0x7f9aac15ccd0 2024-09-21T07:49:17.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.808+0000 7f9ab3200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f9aac0b4fd0 con 0x7f9aac1539b0 2024-09-21T07:49:17.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.808+0000 7f9ab2800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9aac1f3590 0x7f9aac1582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:17.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.809+0000 7f9ab2800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9aac1f3590 0x7f9aac1582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59482/0 (socket says 172.21.15.39:59482) 2024-09-21T07:49:17.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.809+0000 7f9ab2800640 1 -- 172.21.15.39:0/2468697469 learned_addr learned my addr 172.21.15.39:0/2468697469 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:17.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.809+0000 7f9ab1e00640 1 --2- 172.21.15.39:0/2468697469 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9aac1539b0 0x7f9aac143400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:17.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.809+0000 7f9ab2800640 1 -- 172.21.15.39:0/2468697469 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9aac1539b0 msgr2=0x7f9aac143400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:17.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.809+0000 7f9ab2800640 1 --2- 172.21.15.39:0/2468697469 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9aac1539b0 0x7f9aac143400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:17.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.809+0000 7f9ab2800640 1 -- 172.21.15.39:0/2468697469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9aac15ccd0 msgr2=0x7f9aac157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:17.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.809+0000 7f9ab2800640 1 --2- 172.21.15.39:0/2468697469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9aac15ccd0 0x7f9aac157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:17.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.809+0000 7f9ab2800640 1 -- 172.21.15.39:0/2468697469 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aac159f10 con 0x7f9aac1f3590 2024-09-21T07:49:17.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.809+0000 7f9ab2800640 1 --2- 172.21.15.39:0/2468697469 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9aac1f3590 0x7f9aac1582e0 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f9aa800eab0 tx=0x7f9aa800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:17.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.809+0000 7f9aa6000640 1 -- 172.21.15.39:0/2468697469 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa800cea0 con 0x7f9aac1f3590 2024-09-21T07:49:17.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.810+0000 7f9aa6000640 1 -- 172.21.15.39:0/2468697469 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9aa8004b20 con 0x7f9aac1f3590 2024-09-21T07:49:17.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.810+0000 7f9ab3200640 1 -- 172.21.15.39:0/2468697469 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aac15a1f0 con 0x7f9aac1f3590 2024-09-21T07:49:17.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.810+0000 7f9aa6000640 1 -- 172.21.15.39:0/2468697469 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa8005020 con 0x7f9aac1f3590 2024-09-21T07:49:17.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.810+0000 7f9ab3200640 1 -- 172.21.15.39:0/2468697469 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aac143940 con 0x7f9aac1f3590 2024-09-21T07:49:17.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.811+0000 7f9ab3200640 1 -- 172.21.15.39:0/2468697469 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9aac0b5480 con 0x7f9aac1f3590 2024-09-21T07:49:17.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.812+0000 7f9aa6000640 1 -- 172.21.15.39:0/2468697469 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9aa80051c0 con 0x7f9aac1f3590 2024-09-21T07:49:17.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.813+0000 7f9aa6000640 1 --2- 172.21.15.39:0/2468697469 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9a90078bc0 0x7f9a9007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:17.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.813+0000 7f9aa6000640 1 -- 172.21.15.39:0/2468697469 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(221..221 src has 1..221) ==== 7075+0+0 (secure 0 0 0) 0x7f9aa809bfa0 con 0x7f9aac1f3590 2024-09-21T07:49:17.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.813+0000 7f9ab1e00640 1 --2- 172.21.15.39:0/2468697469 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9a90078bc0 0x7f9a9007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:17.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.814+0000 7f9ab1e00640 1 --2- 172.21.15.39:0/2468697469 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9a90078bc0 0x7f9a9007b0d0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0033000 tx=0x7f9aa0004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:17.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:17.817+0000 7f9aa6000640 1 -- 172.21.15.39:0/2468697469 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9aa8063e80 con 0x7f9aac1f3590 2024-09-21T07:49:18.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.059+0000 7f9ab3200640 1 -- 172.21.15.39:0/2468697469 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9aac159ca0 con 0x7f9aac1f3590 2024-09-21T07:49:18.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.060+0000 7f9aa6000640 1 -- 172.21.15.39:0/2468697469 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 235 v235) ==== 95+0+5262 (secure 0 0 0) 0x7f9aa8063780 con 0x7f9aac1f3590 2024-09-21T07:49:18.062 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:49:18.062 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":235,"btime":"2024-09-21T07:49:17:967945+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8249,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/550771731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":550771731},{"type":"v1","addr":"172.21.15.148:6837","nonce":550771731}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":224},{"gid":9385,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1307677219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":1307677219},{"type":"v1","addr":"172.21.15.39:6835","nonce":1307677219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":224}],"filesystems":[{"mdsmap":{"epoch":235,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:48:47.106474+0000","modified":"2024-09-21T07:49:17.530204+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":10224,"mds_1":9391},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9391":{"gid":9391,"name":"b","rank":1,"incarnation":232,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6836/1256253563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1256253563},{"type":"v1","addr":"172.21.15.148:6836","nonce":1256253563}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10224":{"gid":10224,"name":"c","rank":0,"incarnation":225,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/2837221709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2837221709},{"type":"v1","addr":"172.21.15.39:6834","nonce":2837221709}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10224,"qdb_cluster":[9391,10224]},"id":12}]} 2024-09-21T07:49:18.062 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 235 2024-09-21T07:49:18.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.062+0000 7f9ab3200640 1 -- 172.21.15.39:0/2468697469 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9a90078bc0 msgr2=0x7f9a9007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:18.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.062+0000 7f9ab3200640 1 --2- 172.21.15.39:0/2468697469 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9a90078bc0 0x7f9a9007b0d0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0033000 tx=0x7f9aa0004730 comp rx=0 tx=0).stop 2024-09-21T07:49:18.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.062+0000 7f9ab3200640 1 -- 172.21.15.39:0/2468697469 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9aac1f3590 msgr2=0x7f9aac1582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:18.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.062+0000 7f9ab3200640 1 --2- 172.21.15.39:0/2468697469 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9aac1f3590 0x7f9aac1582e0 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f9aa800eab0 tx=0x7f9aa800ef70 comp rx=0 tx=0).stop 2024-09-21T07:49:18.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.063+0000 7f9ab3200640 1 -- 172.21.15.39:0/2468697469 shutdown_connections 2024-09-21T07:49:18.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.063+0000 7f9ab3200640 1 --2- 172.21.15.39:0/2468697469 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9a90078bc0 0x7f9a9007b0d0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:18.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.063+0000 7f9ab3200640 1 --2- 172.21.15.39:0/2468697469 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9aac1f3590 0x7f9aac1582e0 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:18.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.063+0000 7f9ab3200640 1 --2- 172.21.15.39:0/2468697469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9aac15ccd0 0x7f9aac157da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:18.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.063+0000 7f9ab3200640 1 --2- 172.21.15.39:0/2468697469 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9aac1539b0 0x7f9aac143400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:18.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.063+0000 7f9ab3200640 1 -- 172.21.15.39:0/2468697469 >> 172.21.15.39:0/2468697469 conn(0x7f9aac0d0cb0 msgr2=0x7f9aac0cdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:18.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.063+0000 7f9ab3200640 1 -- 172.21.15.39:0/2468697469 shutdown_connections 2024-09-21T07:49:18.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.063+0000 7f9ab3200640 1 -- 172.21.15.39:0/2468697469 wait complete. 2024-09-21T07:49:18.087 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:49:18.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.319+0000 7f7b17200640 1 -- 172.21.15.39:0/460865047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7af8004800 msgr2=0x7f7af8004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:18.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.319+0000 7f7b17200640 1 --2- 172.21.15.39:0/460865047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7af8004800 0x7f7af8004c10 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f7b00009d20 tx=0x7f7af8005830 comp rx=0 tx=0).stop 2024-09-21T07:49:18.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.319+0000 7f7b16800640 1 -- 172.21.15.39:0/460865047 reap_dead start 2024-09-21T07:49:18.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.319+0000 7f7b17200640 1 -- 172.21.15.39:0/460865047 shutdown_connections 2024-09-21T07:49:18.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.319+0000 7f7b17200640 1 -- 172.21.15.39:0/460865047 >> 172.21.15.39:0/460865047 conn(0x7f7b1006cfc0 msgr2=0x7f7b1006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:18.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.319+0000 7f7b17200640 1 -- 172.21.15.39:0/460865047 shutdown_connections 2024-09-21T07:49:18.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.319+0000 7f7b17200640 1 -- 172.21.15.39:0/460865047 wait complete. 2024-09-21T07:49:18.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.321+0000 7f7b17200640 1 Processor -- start 2024-09-21T07:49:18.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.321+0000 7f7b17200640 1 -- start start 2024-09-21T07:49:18.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.323+0000 7f7b17200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7af8004800 0x7f7b100c4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:18.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.323+0000 7f7b17200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b1014bb40 0x7f7b1013ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:18.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.323+0000 7f7b17200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b1014c4c0 0x7f7b1014c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:18.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.323+0000 7f7b17200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1006f630 con 0x7f7af8004800 2024-09-21T07:49:18.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.323+0000 7f7b17200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1006f4b0 con 0x7f7b1014c4c0 2024-09-21T07:49:18.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.323+0000 7f7b17200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f7b1006f7b0 con 0x7f7b1014bb40 2024-09-21T07:49:18.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.323+0000 7f7b15e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7af8004800 0x7f7b100c4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:18.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.323+0000 7f7b15e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7af8004800 0x7f7b100c4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59490/0 (socket says 172.21.15.39:59490) 2024-09-21T07:49:18.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.323+0000 7f7b15e00640 1 -- 172.21.15.39:0/1226768965 learned_addr learned my addr 172.21.15.39:0/1226768965 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:18.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.323+0000 7f7b15e00640 1 -- 172.21.15.39:0/1226768965 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b1014bb40 msgr2=0x7f7b1013ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:18.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.323+0000 7f7b15e00640 1 --2- 172.21.15.39:0/1226768965 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b1014bb40 0x7f7b1013ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:18.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.323+0000 7f7b15e00640 1 -- 172.21.15.39:0/1226768965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b1014c4c0 msgr2=0x7f7b1014c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:18.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.323+0000 7f7b15e00640 1 --2- 172.21.15.39:0/1226768965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b1014c4c0 0x7f7b1014c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:18.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.323+0000 7f7b15e00640 1 -- 172.21.15.39:0/1226768965 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b101f9ec0 con 0x7f7af8004800 2024-09-21T07:49:18.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.324+0000 7f7b15e00640 1 --2- 172.21.15.39:0/1226768965 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7af8004800 0x7f7b100c4fb0 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f7b0400e570 tx=0x7f7b0400e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:18.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.324+0000 7f7b0a000640 1 -- 172.21.15.39:0/1226768965 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b04015070 con 0x7f7af8004800 2024-09-21T07:49:18.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.324+0000 7f7b0a000640 1 -- 172.21.15.39:0/1226768965 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b04010980 con 0x7f7af8004800 2024-09-21T07:49:18.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.324+0000 7f7b0a000640 1 -- 172.21.15.39:0/1226768965 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b04010d30 con 0x7f7af8004800 2024-09-21T07:49:18.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.324+0000 7f7b17200640 1 -- 172.21.15.39:0/1226768965 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b1014ce10 con 0x7f7af8004800 2024-09-21T07:49:18.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.324+0000 7f7b17200640 1 -- 172.21.15.39:0/1226768965 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b101f9620 con 0x7f7af8004800 2024-09-21T07:49:18.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.325+0000 7f7b17200640 1 -- 172.21.15.39:0/1226768965 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b101f5a00 con 0x7f7af8004800 2024-09-21T07:49:18.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.326+0000 7f7b0a000640 1 -- 172.21.15.39:0/1226768965 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7b04018070 con 0x7f7af8004800 2024-09-21T07:49:18.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.327+0000 7f7b0a000640 1 --2- 172.21.15.39:0/1226768965 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7af4078bc0 0x7f7af407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:18.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.327+0000 7f7b0a000640 1 -- 172.21.15.39:0/1226768965 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(222..222 src has 1..222) ==== 7131+0+0 (secure 0 0 0) 0x7f7b0409b510 con 0x7f7af8004800 2024-09-21T07:49:18.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.328+0000 7f7b15400640 1 --2- 172.21.15.39:0/1226768965 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7af4078bc0 0x7f7af407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:18.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.329+0000 7f7b15400640 1 --2- 172.21.15.39:0/1226768965 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7af4078bc0 0x7f7af407b0d0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f7b00009d20 tx=0x7f7b00004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:18.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.331+0000 7f7b0a000640 1 -- 172.21.15.39:0/1226768965 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7b1014ce10 con 0x7f7af8004800 2024-09-21T07:49:18.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.566+0000 7f7b17200640 1 -- 172.21.15.39:0/1226768965 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b100d08b0 con 0x7f7af8004800 2024-09-21T07:49:18.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.566+0000 7f7b0a000640 1 -- 172.21.15.39:0/1226768965 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 235 v235) ==== 95+0+5262 (secure 0 0 0) 0x7f7b100d0df0 con 0x7f7af8004800 2024-09-21T07:49:18.568 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:49:18.568 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":235,"btime":"2024-09-21T07:49:17:967945+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8249,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/550771731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":550771731},{"type":"v1","addr":"172.21.15.148:6837","nonce":550771731}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":224},{"gid":9385,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1307677219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":1307677219},{"type":"v1","addr":"172.21.15.39:6835","nonce":1307677219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":224}],"filesystems":[{"mdsmap":{"epoch":235,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:48:47.106474+0000","modified":"2024-09-21T07:49:17.530204+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":10224,"mds_1":9391},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9391":{"gid":9391,"name":"b","rank":1,"incarnation":232,"state":"up:active","state_seq":7,"addr":"172.21.15.148:6836/1256253563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1256253563},{"type":"v1","addr":"172.21.15.148:6836","nonce":1256253563}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10224":{"gid":10224,"name":"c","rank":0,"incarnation":225,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/2837221709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2837221709},{"type":"v1","addr":"172.21.15.39:6834","nonce":2837221709}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10224,"qdb_cluster":[9391,10224]},"id":12}]} 2024-09-21T07:49:18.568 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 235 2024-09-21T07:49:18.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.568+0000 7f7b17200640 1 -- 172.21.15.39:0/1226768965 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7af4078bc0 msgr2=0x7f7af407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:18.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.568+0000 7f7b17200640 1 --2- 172.21.15.39:0/1226768965 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7af4078bc0 0x7f7af407b0d0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f7b00009d20 tx=0x7f7b00004730 comp rx=0 tx=0).stop 2024-09-21T07:49:18.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.568+0000 7f7b17200640 1 -- 172.21.15.39:0/1226768965 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7af8004800 msgr2=0x7f7b100c4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:18.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.568+0000 7f7b17200640 1 --2- 172.21.15.39:0/1226768965 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7af8004800 0x7f7b100c4fb0 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f7b0400e570 tx=0x7f7b0400e970 comp rx=0 tx=0).stop 2024-09-21T07:49:18.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.569+0000 7f7b17200640 1 -- 172.21.15.39:0/1226768965 shutdown_connections 2024-09-21T07:49:18.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.569+0000 7f7b17200640 1 --2- 172.21.15.39:0/1226768965 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7af4078bc0 0x7f7af407b0d0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:18.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.569+0000 7f7b17200640 1 --2- 172.21.15.39:0/1226768965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b1014c4c0 0x7f7b1014c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:18.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.569+0000 7f7b17200640 1 --2- 172.21.15.39:0/1226768965 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b1014bb40 0x7f7b1013ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:18.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.569+0000 7f7b17200640 1 --2- 172.21.15.39:0/1226768965 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7af8004800 0x7f7b100c4fb0 unknown :-1 s=CLOSED pgs=2163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:18.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.569+0000 7f7b17200640 1 -- 172.21.15.39:0/1226768965 >> 172.21.15.39:0/1226768965 conn(0x7f7b1006cfc0 msgr2=0x7f7b100ced80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:18.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.569+0000 7f7b17200640 1 -- 172.21.15.39:0/1226768965 shutdown_connections 2024-09-21T07:49:18.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.569+0000 7f7b17200640 1 -- 172.21.15.39:0/1226768965 wait complete. 2024-09-21T07:49:18.593 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:49:18.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.821+0000 7f3441800640 1 -- 172.21.15.39:0/4157315611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f343c15cc00 msgr2=0x7f343c15d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:18.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.821+0000 7f3441800640 1 --2- 172.21.15.39:0/4157315611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f343c15cc00 0x7f343c15d010 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f343c0d0170 tx=0x7f342402a530 comp rx=0 tx=0).stop 2024-09-21T07:49:18.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.821+0000 7f3441800640 1 -- 172.21.15.39:0/4157315611 shutdown_connections 2024-09-21T07:49:18.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.821+0000 7f3441800640 1 --2- 172.21.15.39:0/4157315611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f343c15cc00 0x7f343c15d010 unknown :-1 s=CLOSED pgs=2165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:18.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.821+0000 7f3441800640 1 --2- 172.21.15.39:0/4157315611 >> v2:172.21.15.148:3300/0 conn(0x7f343c0cfa20 0x7f343c0cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:18.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.821+0000 7f3441800640 1 -- 172.21.15.39:0/4157315611 >> 172.21.15.39:0/4157315611 conn(0x7f343c06cfc0 msgr2=0x7f343c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:18.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.821+0000 7f3441800640 1 -- 172.21.15.39:0/4157315611 shutdown_connections 2024-09-21T07:49:18.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.821+0000 7f3441800640 1 -- 172.21.15.39:0/4157315611 wait complete. 2024-09-21T07:49:18.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.823+0000 7f3441800640 1 Processor -- start 2024-09-21T07:49:18.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.823+0000 7f3441800640 1 -- start start 2024-09-21T07:49:18.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.825+0000 7f3441800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f343c0cfa20 0x7f343c1f3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:18.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.825+0000 7f3441800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f343c0d04b0 0x7f343c0ae810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:18.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.825+0000 7f3441800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f343c15cc00 0x7f343c13e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:18.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.825+0000 7f3441800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f343c06ee70 con 0x7f343c0cfa20 2024-09-21T07:49:18.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.825+0000 7f3441800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f343c06ecf0 con 0x7f343c0d04b0 2024-09-21T07:49:18.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.825+0000 7f3441800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f343c06eff0 con 0x7f343c15cc00 2024-09-21T07:49:18.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.825+0000 7f343be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f343c0cfa20 0x7f343c1f3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:18.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.825+0000 7f343be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f343c0cfa20 0x7f343c1f3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59522/0 (socket says 172.21.15.39:59522) 2024-09-21T07:49:18.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.825+0000 7f343be00640 1 -- 172.21.15.39:0/2967858040 learned_addr learned my addr 172.21.15.39:0/2967858040 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:18.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.825+0000 7f3433e00640 1 --2- 172.21.15.39:0/2967858040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f343c0d04b0 0x7f343c0ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:18.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.825+0000 7f3440e00640 1 --2- 172.21.15.39:0/2967858040 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f343c15cc00 0x7f343c13e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:18.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.826+0000 7f343be00640 1 -- 172.21.15.39:0/2967858040 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f343c15cc00 msgr2=0x7f343c13e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:18.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.826+0000 7f343be00640 1 --2- 172.21.15.39:0/2967858040 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f343c15cc00 0x7f343c13e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:18.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.826+0000 7f343be00640 1 -- 172.21.15.39:0/2967858040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f343c0d04b0 msgr2=0x7f343c0ae810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:18.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.826+0000 7f343be00640 1 --2- 172.21.15.39:0/2967858040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f343c0d04b0 0x7f343c0ae810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:18.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.826+0000 7f343be00640 1 -- 172.21.15.39:0/2967858040 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f343c0ace80 con 0x7f343c0cfa20 2024-09-21T07:49:18.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.826+0000 7f3440e00640 1 --2- 172.21.15.39:0/2967858040 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f343c15cc00 0x7f343c13e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:18.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.826+0000 7f343be00640 1 --2- 172.21.15.39:0/2967858040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f343c0cfa20 0x7f343c1f3f60 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f342c00d740 tx=0x7f342c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:18.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.826+0000 7f3438c00640 1 -- 172.21.15.39:0/2967858040 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c017070 con 0x7f343c0cfa20 2024-09-21T07:49:18.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.826+0000 7f3438c00640 1 -- 172.21.15.39:0/2967858040 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f342c009d20 con 0x7f343c0cfa20 2024-09-21T07:49:18.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.826+0000 7f3438c00640 1 -- 172.21.15.39:0/2967858040 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c0106c0 con 0x7f343c0cfa20 2024-09-21T07:49:18.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.826+0000 7f3441800640 1 -- 172.21.15.39:0/2967858040 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f343c15a9e0 con 0x7f343c0cfa20 2024-09-21T07:49:18.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.826+0000 7f3441800640 1 -- 172.21.15.39:0/2967858040 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f343c15afa0 con 0x7f343c0cfa20 2024-09-21T07:49:18.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.828+0000 7f3441800640 1 -- 172.21.15.39:0/2967858040 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f343c06f480 con 0x7f343c0cfa20 2024-09-21T07:49:18.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.829+0000 7f3438c00640 1 -- 172.21.15.39:0/2967858040 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f342c01a070 con 0x7f343c0cfa20 2024-09-21T07:49:18.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.830+0000 7f3438c00640 1 --2- 172.21.15.39:0/2967858040 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3418078bc0 0x7f341807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:18.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.830+0000 7f3438c00640 1 -- 172.21.15.39:0/2967858040 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(222..222 src has 1..222) ==== 7131+0+0 (secure 0 0 0) 0x7f342c09b370 con 0x7f343c0cfa20 2024-09-21T07:49:18.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.831+0000 7f3433e00640 1 --2- 172.21.15.39:0/2967858040 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3418078bc0 0x7f341807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:18.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.832+0000 7f3433e00640 1 --2- 172.21.15.39:0/2967858040 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3418078bc0 0x7f341807b0d0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f3424034000 tx=0x7f3424004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:18.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:18.835+0000 7f3438c00640 1 -- 172.21.15.39:0/2967858040 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f342c062bd0 con 0x7f343c0cfa20 2024-09-21T07:49:19.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.038+0000 7f3441800640 1 -- 172.21.15.39:0/2967858040 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f343c0cc980 con 0x7f343c0cfa20 2024-09-21T07:49:19.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.039+0000 7f3438c00640 1 -- 172.21.15.39:0/2967858040 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v222) ==== 74+0+22762 (secure 0 0 0) 0x7f342c067600 con 0x7f343c0cfa20 2024-09-21T07:49:19.041 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:49:19.042 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":222,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:49:17.968021+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:48:45.059810+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"218","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-09-21T07:48:46.068380+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"222","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":222,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":220,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":220,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"28.9","mappings":[{"from":6,"to":1}]},{"pgid":"28.f","mappings":[{"from":6,"to":4}]},{"pgid":"28.32","mappings":[{"from":6,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]},{"pool":20,"snaps":[{"begin":80,"length":23},{"begin":181,"length":23}]},{"pool":22,"snaps":[{"begin":100,"length":1},{"begin":199,"length":1}]},{"pool":24,"snaps":[{"begin":2,"length":2}]},{"pool":26,"snaps":[{"begin":2,"length":2}]},{"pool":28,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":28,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:49:19.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.042+0000 7f3441800640 1 -- 172.21.15.39:0/2967858040 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3418078bc0 msgr2=0x7f341807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:19.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.042+0000 7f3441800640 1 --2- 172.21.15.39:0/2967858040 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3418078bc0 0x7f341807b0d0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f3424034000 tx=0x7f3424004730 comp rx=0 tx=0).stop 2024-09-21T07:49:19.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.042+0000 7f3441800640 1 -- 172.21.15.39:0/2967858040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f343c0cfa20 msgr2=0x7f343c1f3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:19.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.042+0000 7f3441800640 1 --2- 172.21.15.39:0/2967858040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f343c0cfa20 0x7f343c1f3f60 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f342c00d740 tx=0x7f342c00dc00 comp rx=0 tx=0).stop 2024-09-21T07:49:19.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.042+0000 7f3441800640 1 -- 172.21.15.39:0/2967858040 shutdown_connections 2024-09-21T07:49:19.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.042+0000 7f3441800640 1 --2- 172.21.15.39:0/2967858040 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3418078bc0 0x7f341807b0d0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:19.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.042+0000 7f3441800640 1 --2- 172.21.15.39:0/2967858040 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f343c15cc00 0x7f343c13e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:19.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.042+0000 7f3441800640 1 --2- 172.21.15.39:0/2967858040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f343c0d04b0 0x7f343c0ae810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:19.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.042+0000 7f3441800640 1 --2- 172.21.15.39:0/2967858040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f343c0cfa20 0x7f343c1f3f60 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:19.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.042+0000 7f3441800640 1 -- 172.21.15.39:0/2967858040 >> 172.21.15.39:0/2967858040 conn(0x7f343c06cfc0 msgr2=0x7f343c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:19.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.042+0000 7f3441800640 1 -- 172.21.15.39:0/2967858040 shutdown_connections 2024-09-21T07:49:19.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.042+0000 7f3441800640 1 -- 172.21.15.39:0/2967858040 wait complete. 2024-09-21T07:49:19.071 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T07:49:19.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.297+0000 7f989b800640 1 -- 172.21.15.39:0/2422086613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98940b4b60 msgr2=0x7f98940af080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:19.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.297+0000 7f989b800640 1 --2- 172.21.15.39:0/2422086613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98940b4b60 0x7f98940af080 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f988c005fd0 tx=0x7f988c02a8b0 comp rx=0 tx=0).stop 2024-09-21T07:49:19.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.297+0000 7f989b800640 1 -- 172.21.15.39:0/2422086613 shutdown_connections 2024-09-21T07:49:19.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.297+0000 7f989b800640 1 --2- 172.21.15.39:0/2422086613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98940b4b60 0x7f98940af080 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:19.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.297+0000 7f989b800640 1 -- 172.21.15.39:0/2422086613 >> 172.21.15.39:0/2422086613 conn(0x7f98940b2460 msgr2=0x7f989406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:19.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.297+0000 7f989b800640 1 -- 172.21.15.39:0/2422086613 shutdown_connections 2024-09-21T07:49:19.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.297+0000 7f989b800640 1 -- 172.21.15.39:0/2422086613 wait complete. 2024-09-21T07:49:19.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.300+0000 7f989b800640 1 Processor -- start 2024-09-21T07:49:19.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.300+0000 7f989b800640 1 -- start start 2024-09-21T07:49:19.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.301+0000 7f989b800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f98940b3990 0x7f989409db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:19.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.301+0000 7f989b800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9894162f10 0x7f98940aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:19.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.301+0000 7f989b800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98940af000 0x7f98940b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:19.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.301+0000 7f989b800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f98940b3fb0 con 0x7f9894162f10 2024-09-21T07:49:19.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.301+0000 7f989b800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f98940af5c0 con 0x7f98940af000 2024-09-21T07:49:19.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.301+0000 7f989b800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f989415adb0 con 0x7f98940b3990 2024-09-21T07:49:19.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.302+0000 7f9893e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9894162f10 0x7f98940aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:19.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.302+0000 7f9893e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9894162f10 0x7f98940aeac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59542/0 (socket says 172.21.15.39:59542) 2024-09-21T07:49:19.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.302+0000 7f9893e00640 1 -- 172.21.15.39:0/571363632 learned_addr learned my addr 172.21.15.39:0/571363632 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:19.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.302+0000 7f9898e00640 1 --2- 172.21.15.39:0/571363632 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f98940b3990 0x7f989409db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:19.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.302+0000 7f9899800640 1 --2- 172.21.15.39:0/571363632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98940af000 0x7f98940b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:19.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.302+0000 7f9893e00640 1 -- 172.21.15.39:0/571363632 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f98940b3990 msgr2=0x7f989409db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:19.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.302+0000 7f9893e00640 1 --2- 172.21.15.39:0/571363632 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f98940b3990 0x7f989409db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:19.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.302+0000 7f9893e00640 1 -- 172.21.15.39:0/571363632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98940af000 msgr2=0x7f98940b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:19.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.302+0000 7f9893e00640 1 --2- 172.21.15.39:0/571363632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98940af000 0x7f98940b0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:19.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.302+0000 7f9893e00640 1 -- 172.21.15.39:0/571363632 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98940adbf0 con 0x7f9894162f10 2024-09-21T07:49:19.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.303+0000 7f9893e00640 1 --2- 172.21.15.39:0/571363632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9894162f10 0x7f98940aeac0 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f988000c3c0 tx=0x7f988000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:19.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.303+0000 7f9890c00640 1 -- 172.21.15.39:0/571363632 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9880017070 con 0x7f9894162f10 2024-09-21T07:49:19.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.303+0000 7f9890c00640 1 -- 172.21.15.39:0/571363632 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98800035f0 con 0x7f9894162f10 2024-09-21T07:49:19.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.303+0000 7f989b800640 1 -- 172.21.15.39:0/571363632 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98940aded0 con 0x7f9894162f10 2024-09-21T07:49:19.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.303+0000 7f9890c00640 1 -- 172.21.15.39:0/571363632 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9880003a40 con 0x7f9894162f10 2024-09-21T07:49:19.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.303+0000 7f989b800640 1 -- 172.21.15.39:0/571363632 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98940ae490 con 0x7f9894162f10 2024-09-21T07:49:19.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.304+0000 7f989b800640 1 -- 172.21.15.39:0/571363632 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98940694e0 con 0x7f9894162f10 2024-09-21T07:49:19.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.306+0000 7f9890c00640 1 -- 172.21.15.39:0/571363632 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f988001a070 con 0x7f9894162f10 2024-09-21T07:49:19.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.306+0000 7f9890c00640 1 --2- 172.21.15.39:0/571363632 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9874078bc0 0x7f987407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:19.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.307+0000 7f9890c00640 1 -- 172.21.15.39:0/571363632 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(222..222 src has 1..222) ==== 7131+0+0 (secure 0 0 0) 0x7f988009a3e0 con 0x7f9894162f10 2024-09-21T07:49:19.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.307+0000 7f9898e00640 1 --2- 172.21.15.39:0/571363632 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9874078bc0 0x7f987407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:19.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.308+0000 7f9898e00640 1 --2- 172.21.15.39:0/571363632 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9874078bc0 0x7f987407b0d0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f9888000f60 tx=0x7f9888009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:19.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.311+0000 7f9890c00640 1 -- 172.21.15.39:0/571363632 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f98940694e0 con 0x7f9894162f10 2024-09-21T07:49:19.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.546+0000 7f989b800640 1 -- 172.21.15.39:0/571363632 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f98940afab0 con 0x7f9894162f10 2024-09-21T07:49:19.548 INFO:tasks.ceph.mon.a.smithi039.stderr:2024-09-21T07:49:19.547+0000 7fabab600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T07:49:19.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.549+0000 7f9890c00640 1 -- 172.21.15.39:0/571363632 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v236) ==== 161+0+0 (secure 0 0 0) 0x7f9880061c30 con 0x7f9894162f10 2024-09-21T07:49:19.551 INFO:teuthology.orchestra.run.smithi039.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T07:49:19.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.552+0000 7f989b800640 1 -- 172.21.15.39:0/571363632 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9874078bc0 msgr2=0x7f987407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:19.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.552+0000 7f989b800640 1 --2- 172.21.15.39:0/571363632 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9874078bc0 0x7f987407b0d0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f9888000f60 tx=0x7f9888009040 comp rx=0 tx=0).stop 2024-09-21T07:49:19.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.552+0000 7f989b800640 1 -- 172.21.15.39:0/571363632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9894162f10 msgr2=0x7f98940aeac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:19.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.552+0000 7f989b800640 1 --2- 172.21.15.39:0/571363632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9894162f10 0x7f98940aeac0 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f988000c3c0 tx=0x7f988000c880 comp rx=0 tx=0).stop 2024-09-21T07:49:19.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.552+0000 7f989b800640 1 -- 172.21.15.39:0/571363632 shutdown_connections 2024-09-21T07:49:19.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.552+0000 7f989b800640 1 --2- 172.21.15.39:0/571363632 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9874078bc0 0x7f987407b0d0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:19.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.552+0000 7f989b800640 1 --2- 172.21.15.39:0/571363632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f98940af000 0x7f98940b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:19.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.552+0000 7f989b800640 1 --2- 172.21.15.39:0/571363632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9894162f10 0x7f98940aeac0 unknown :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:19.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.552+0000 7f989b800640 1 --2- 172.21.15.39:0/571363632 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f98940b3990 0x7f989409db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:19.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.552+0000 7f989b800640 1 -- 172.21.15.39:0/571363632 >> 172.21.15.39:0/571363632 conn(0x7f98940b2460 msgr2=0x7f98940b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:19.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.552+0000 7f989b800640 1 -- 172.21.15.39:0/571363632 shutdown_connections 2024-09-21T07:49:19.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.552+0000 7f989b800640 1 -- 172.21.15.39:0/571363632 wait complete. 2024-09-21T07:49:19.569 INFO:tasks.ceph.mds.c.smithi039.stderr: -6741> 2024-09-21T07:49:07.553+0000 7fe9b7000640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:49:19.579 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T07:49:19.692 INFO:tasks.ceph.mds.b.smithi148.stdout:starting mds.b at 2024-09-21T07:49:19.698 INFO:tasks.ceph.mds.c.smithi039.stdout:starting mds.c at 2024-09-21T07:49:19.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.813+0000 7fedc9600640 1 -- 172.21.15.39:0/1005214280 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fedc4160250 msgr2=0x7fedc415d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:19.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.813+0000 7fedc9600640 1 --2- 172.21.15.39:0/1005214280 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fedc4160250 0x7fedc415d570 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fedb4009830 tx=0x7fedb402a510 comp rx=0 tx=0).stop 2024-09-21T07:49:19.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.813+0000 7fedc9600640 1 -- 172.21.15.39:0/1005214280 shutdown_connections 2024-09-21T07:49:19.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.813+0000 7fedc9600640 1 --2- 172.21.15.39:0/1005214280 >> v2:172.21.15.148:3300/0 conn(0x7fedc40d0390 0x7fedc40d07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:19.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.813+0000 7fedc9600640 1 --2- 172.21.15.39:0/1005214280 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fedc4160250 0x7fedc415d570 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:19.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.813+0000 7fedc9600640 1 -- 172.21.15.39:0/1005214280 >> 172.21.15.39:0/1005214280 conn(0x7fedc406cfc0 msgr2=0x7fedc406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:19.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.813+0000 7fedc9600640 1 -- 172.21.15.39:0/1005214280 shutdown_connections 2024-09-21T07:49:19.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.813+0000 7fedc9600640 1 -- 172.21.15.39:0/1005214280 wait complete. 2024-09-21T07:49:19.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.815+0000 7fedc9600640 1 Processor -- start 2024-09-21T07:49:19.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.816+0000 7fedc9600640 1 -- start start 2024-09-21T07:49:19.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.817+0000 7fedc9600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fedc40d0390 0x7fedc413e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:19.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.817+0000 7fedc9600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fedc4160250 0x7fedc413b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:19.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.817+0000 7fedc9600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedc4160660 0x7fedc415f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:19.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.817+0000 7fedc9600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fedc40ccdc0 con 0x7fedc4160250 2024-09-21T07:49:19.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.817+0000 7fedc9600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fedc40ccc40 con 0x7fedc4160660 2024-09-21T07:49:19.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.817+0000 7fedc9600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fedc40ccf40 con 0x7fedc40d0390 2024-09-21T07:49:19.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.817+0000 7fedbb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fedc4160250 0x7fedc413b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:19.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.817+0000 7fedbb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fedc4160250 0x7fedc413b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59558/0 (socket says 172.21.15.39:59558) 2024-09-21T07:49:19.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.817+0000 7fedbb400640 1 -- 172.21.15.39:0/3441964891 learned_addr learned my addr 172.21.15.39:0/3441964891 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:19.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.817+0000 7fedc3e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fedc40d0390 0x7fedc413e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:19.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.817+0000 7fedc8c00640 1 --2- 172.21.15.39:0/3441964891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedc4160660 0x7fedc415f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:19.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.818+0000 7fedbb400640 1 -- 172.21.15.39:0/3441964891 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fedc40d0390 msgr2=0x7fedc413e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:19.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.818+0000 7fedbb400640 1 --2- 172.21.15.39:0/3441964891 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fedc40d0390 0x7fedc413e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:19.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.818+0000 7fedbb400640 1 -- 172.21.15.39:0/3441964891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedc4160660 msgr2=0x7fedc415f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:19.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.818+0000 7fedbb400640 1 --2- 172.21.15.39:0/3441964891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedc4160660 0x7fedc415f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:19.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.818+0000 7fedbb400640 1 -- 172.21.15.39:0/3441964891 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc415a060 con 0x7fedc4160250 2024-09-21T07:49:19.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.818+0000 7fedc3e00640 1 --2- 172.21.15.39:0/3441964891 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fedc40d0390 0x7fedc413e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:49:19.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.818+0000 7fedc8c00640 1 --2- 172.21.15.39:0/3441964891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedc4160660 0x7fedc415f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:49:19.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.818+0000 7fedbb400640 1 --2- 172.21.15.39:0/3441964891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fedc4160250 0x7fedc413b050 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7fedac00c760 tx=0x7fedac00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:19.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.818+0000 7fedc0c00640 1 -- 172.21.15.39:0/3441964891 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedac016020 con 0x7fedc4160250 2024-09-21T07:49:19.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.818+0000 7fedc0c00640 1 -- 172.21.15.39:0/3441964891 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fedac00a950 con 0x7fedc4160250 2024-09-21T07:49:19.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.818+0000 7fedc9600640 1 -- 172.21.15.39:0/3441964891 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedc41592e0 con 0x7fedc4160250 2024-09-21T07:49:19.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.818+0000 7fedc0c00640 1 -- 172.21.15.39:0/3441964891 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedac00ada0 con 0x7fedc4160250 2024-09-21T07:49:19.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.819+0000 7fedc9600640 1 -- 172.21.15.39:0/3441964891 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedc4159580 con 0x7fedc4160250 2024-09-21T07:49:19.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.820+0000 7fedc9600640 1 -- 172.21.15.39:0/3441964891 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedc40cd3f0 con 0x7fedc4160250 2024-09-21T07:49:19.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.821+0000 7fedc0c00640 1 -- 172.21.15.39:0/3441964891 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fedac019020 con 0x7fedc4160250 2024-09-21T07:49:19.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.822+0000 7fedc0c00640 1 --2- 172.21.15.39:0/3441964891 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feda0078bc0 0x7feda007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:19.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.822+0000 7fedc0c00640 1 -- 172.21.15.39:0/3441964891 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(223..223 src has 1..223) ==== 7275+0+0 (secure 0 0 0) 0x7fedac09b140 con 0x7fedc4160250 2024-09-21T07:49:19.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.822+0000 7fedc3e00640 1 --2- 172.21.15.39:0/3441964891 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feda0078bc0 0x7feda007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:19.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.823+0000 7fedc3e00640 1 --2- 172.21.15.39:0/3441964891 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feda0078bc0 0x7feda007b0d0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fedb4034000 tx=0x7fedb4004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:19.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:19.826+0000 7fedc0c00640 1 -- 172.21.15.39:0/3441964891 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fedac020020 con 0x7fedc4160250 2024-09-21T07:49:20.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.070+0000 7fedc9600640 1 -- 172.21.15.39:0/3441964891 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fedc4073190 con 0x7fedc4160250 2024-09-21T07:49:20.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.557+0000 7fedc0c00640 1 -- 172.21.15.39:0/3441964891 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v237) ==== 104+0+0 (secure 0 0 0) 0x7fedc40d0ce0 con 0x7fedc4160250 2024-09-21T07:49:20.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.559+0000 7fedc9600640 1 -- 172.21.15.39:0/3441964891 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feda0078bc0 msgr2=0x7feda007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:20.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.559+0000 7fedc9600640 1 --2- 172.21.15.39:0/3441964891 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feda0078bc0 0x7feda007b0d0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fedb4034000 tx=0x7fedb4004730 comp rx=0 tx=0).stop 2024-09-21T07:49:20.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.559+0000 7fedc9600640 1 -- 172.21.15.39:0/3441964891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fedc4160250 msgr2=0x7fedc413b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:20.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.559+0000 7fedc9600640 1 --2- 172.21.15.39:0/3441964891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fedc4160250 0x7fedc413b050 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7fedac00c760 tx=0x7fedac00cc20 comp rx=0 tx=0).stop 2024-09-21T07:49:20.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.560+0000 7fedc9600640 1 -- 172.21.15.39:0/3441964891 shutdown_connections 2024-09-21T07:49:20.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.560+0000 7fedc9600640 1 --2- 172.21.15.39:0/3441964891 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feda0078bc0 0x7feda007b0d0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:20.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.560+0000 7fedc9600640 1 --2- 172.21.15.39:0/3441964891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fedc4160660 0x7fedc415f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:20.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.560+0000 7fedc9600640 1 --2- 172.21.15.39:0/3441964891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fedc4160250 0x7fedc413b050 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:20.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.560+0000 7fedc9600640 1 --2- 172.21.15.39:0/3441964891 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fedc40d0390 0x7fedc413e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:20.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.560+0000 7fedc9600640 1 -- 172.21.15.39:0/3441964891 >> 172.21.15.39:0/3441964891 conn(0x7fedc406cfc0 msgr2=0x7fedc415e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:20.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.560+0000 7fedc9600640 1 -- 172.21.15.39:0/3441964891 shutdown_connections 2024-09-21T07:49:20.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.560+0000 7fedc9600640 1 -- 172.21.15.39:0/3441964891 wait complete. 2024-09-21T07:49:20.585 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T07:49:20.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.815+0000 7f4dc6a00640 1 -- 172.21.15.39:0/3730820419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4dc00b35f0 msgr2=0x7f4dc00b3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:20.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.815+0000 7f4dc6a00640 1 --2- 172.21.15.39:0/3730820419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4dc00b35f0 0x7f4dc00b3a00 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7f4da8005fd0 tx=0x7f4da802a660 comp rx=0 tx=0).stop 2024-09-21T07:49:20.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.815+0000 7f4dc6a00640 1 -- 172.21.15.39:0/3730820419 shutdown_connections 2024-09-21T07:49:20.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.815+0000 7f4dc6a00640 1 --2- 172.21.15.39:0/3730820419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4dc00b35f0 0x7f4dc00b3a00 unknown :-1 s=CLOSED pgs=2173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:20.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.815+0000 7f4dc6a00640 1 --2- 172.21.15.39:0/3730820419 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4dc00cc190 0x7f4dc00cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:20.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.815+0000 7f4dc6a00640 1 -- 172.21.15.39:0/3730820419 >> 172.21.15.39:0/3730820419 conn(0x7f4dc00b2640 msgr2=0x7f4dc00d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:20.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.815+0000 7f4dc6a00640 1 -- 172.21.15.39:0/3730820419 shutdown_connections 2024-09-21T07:49:20.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.815+0000 7f4dc6a00640 1 -- 172.21.15.39:0/3730820419 wait complete. 2024-09-21T07:49:20.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.817+0000 7f4dc6a00640 1 Processor -- start 2024-09-21T07:49:20.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.817+0000 7f4dc6a00640 1 -- start start 2024-09-21T07:49:20.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.819+0000 7f4dc6a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4dc00b35f0 0x7f4dc015c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:20.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.819+0000 7f4dc6a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4dc00ca950 0x7f4dc00ae490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:20.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.819+0000 7f4dc6a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4dc00cc190 0x7f4dc00c9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:20.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.819+0000 7f4dc6a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc00cb5f0 con 0x7f4dc00ca950 2024-09-21T07:49:20.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.819+0000 7f4dc6a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc00cb470 con 0x7f4dc00cc190 2024-09-21T07:49:20.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.819+0000 7f4dc6a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f4dc00cb770 con 0x7f4dc00b35f0 2024-09-21T07:49:20.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.819+0000 7f4db7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4dc00ca950 0x7f4dc00ae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:20.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.819+0000 7f4db7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4dc00ca950 0x7f4dc00ae490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59566/0 (socket says 172.21.15.39:59566) 2024-09-21T07:49:20.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.819+0000 7f4db7e00640 1 -- 172.21.15.39:0/3710599590 learned_addr learned my addr 172.21.15.39:0/3710599590 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:20.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.819+0000 7f4dbfe00640 1 --2- 172.21.15.39:0/3710599590 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4dc00b35f0 0x7f4dc015c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:20.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.819+0000 7f4dc4a00640 1 --2- 172.21.15.39:0/3710599590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4dc00cc190 0x7f4dc00c9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:20.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.819+0000 7f4db7e00640 1 -- 172.21.15.39:0/3710599590 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4dc00b35f0 msgr2=0x7f4dc015c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:20.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.819+0000 7f4db7e00640 1 --2- 172.21.15.39:0/3710599590 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4dc00b35f0 0x7f4dc015c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:20.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.819+0000 7f4db7e00640 1 -- 172.21.15.39:0/3710599590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4dc00cc190 msgr2=0x7f4dc00c9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:20.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.819+0000 7f4db7e00640 1 --2- 172.21.15.39:0/3710599590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4dc00cc190 0x7f4dc00c9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:20.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.819+0000 7f4db7e00640 1 -- 172.21.15.39:0/3710599590 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dc01f7620 con 0x7f4dc00ca950 2024-09-21T07:49:20.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.819+0000 7f4dc4a00640 1 --2- 172.21.15.39:0/3710599590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4dc00cc190 0x7f4dc00c9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:49:20.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.820+0000 7f4db7e00640 1 --2- 172.21.15.39:0/3710599590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4dc00ca950 0x7f4dc00ae490 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f4da8034040 tx=0x7f4da80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:20.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.820+0000 7f4dbcc00640 1 -- 172.21.15.39:0/3710599590 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da803e030 con 0x7f4dc00ca950 2024-09-21T07:49:20.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.820+0000 7f4dbcc00640 1 -- 172.21.15.39:0/3710599590 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4da8003d90 con 0x7f4dc00ca950 2024-09-21T07:49:20.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.820+0000 7f4dc6a00640 1 -- 172.21.15.39:0/3710599590 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dc00af0c0 con 0x7f4dc00ca950 2024-09-21T07:49:20.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.820+0000 7f4dbcc00640 1 -- 172.21.15.39:0/3710599590 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da8035040 con 0x7f4dc00ca950 2024-09-21T07:49:20.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.820+0000 7f4dc6a00640 1 -- 172.21.15.39:0/3710599590 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dc01f6b40 con 0x7f4dc00ca950 2024-09-21T07:49:20.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.821+0000 7f4dc6a00640 1 -- 172.21.15.39:0/3710599590 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dc0050f60 con 0x7f4dc00ca950 2024-09-21T07:49:20.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.823+0000 7f4dbcc00640 1 -- 172.21.15.39:0/3710599590 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4da8041030 con 0x7f4dc00ca950 2024-09-21T07:49:20.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.823+0000 7f4dbcc00640 1 --2- 172.21.15.39:0/3710599590 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4d9c078b90 0x7f4d9c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:20.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.824+0000 7f4dbcc00640 1 -- 172.21.15.39:0/3710599590 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(223..223 src has 1..223) ==== 7275+0+0 (secure 0 0 0) 0x7f4da80bbfb0 con 0x7f4dc00ca950 2024-09-21T07:49:20.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.824+0000 7f4dbfe00640 1 --2- 172.21.15.39:0/3710599590 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4d9c078b90 0x7f4d9c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:20.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.825+0000 7f4dbfe00640 1 --2- 172.21.15.39:0/3710599590 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4d9c078b90 0x7f4d9c07b0a0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f4db0005fd0 tx=0x7f4db0005850 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:20.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:20.828+0000 7f4dbcc00640 1 -- 172.21.15.39:0/3710599590 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4dc0050f60 con 0x7f4dc00ca950 2024-09-21T07:49:21.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:21.026+0000 7f4dc6a00640 1 -- 172.21.15.39:0/3710599590 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4dc0070bf0 con 0x7f4dc00ca950 2024-09-21T07:49:21.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:21.991+0000 7f4dbcc00640 1 -- 172.21.15.39:0/3710599590 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v224) ==== 181+0+0 (secure 0 0 0) 0x7f4dc0070bf0 con 0x7f4dc00ca950 2024-09-21T07:49:21.993 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' removed 2024-09-21T07:49:21.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:21.994+0000 7f4dc6a00640 1 -- 172.21.15.39:0/3710599590 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4d9c078b90 msgr2=0x7f4d9c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:21.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:21.994+0000 7f4dc6a00640 1 --2- 172.21.15.39:0/3710599590 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4d9c078b90 0x7f4d9c07b0a0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f4db0005fd0 tx=0x7f4db0005850 comp rx=0 tx=0).stop 2024-09-21T07:49:21.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:21.994+0000 7f4dc6a00640 1 -- 172.21.15.39:0/3710599590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4dc00ca950 msgr2=0x7f4dc00ae490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:21.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:21.994+0000 7f4dc6a00640 1 --2- 172.21.15.39:0/3710599590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4dc00ca950 0x7f4dc00ae490 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f4da8034040 tx=0x7f4da80038a0 comp rx=0 tx=0).stop 2024-09-21T07:49:21.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:21.994+0000 7f4dc6a00640 1 -- 172.21.15.39:0/3710599590 shutdown_connections 2024-09-21T07:49:21.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:21.994+0000 7f4dc6a00640 1 --2- 172.21.15.39:0/3710599590 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4d9c078b90 0x7f4d9c07b0a0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:21.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:21.994+0000 7f4dc6a00640 1 --2- 172.21.15.39:0/3710599590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4dc00cc190 0x7f4dc00c9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:21.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:21.994+0000 7f4dc6a00640 1 --2- 172.21.15.39:0/3710599590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4dc00ca950 0x7f4dc00ae490 unknown :-1 s=CLOSED pgs=2174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:21.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:21.994+0000 7f4dc6a00640 1 --2- 172.21.15.39:0/3710599590 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4dc00b35f0 0x7f4dc015c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:21.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:21.994+0000 7f4dc6a00640 1 -- 172.21.15.39:0/3710599590 >> 172.21.15.39:0/3710599590 conn(0x7f4dc00b2640 msgr2=0x7f4dc015df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:21.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:21.994+0000 7f4dc6a00640 1 -- 172.21.15.39:0/3710599590 shutdown_connections 2024-09-21T07:49:21.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:21.994+0000 7f4dc6a00640 1 -- 172.21.15.39:0/3710599590 wait complete. 2024-09-21T07:49:22.028 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T07:49:22.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.260+0000 7feace000640 1 -- 172.21.15.39:0/3198620669 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feac806f020 msgr2=0x7feac80cb920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:22.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.260+0000 7feace000640 1 --2- 172.21.15.39:0/3198620669 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feac806f020 0x7feac80cb920 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7feac8072790 tx=0x7feab402a5f0 comp rx=0 tx=0).stop 2024-09-21T07:49:22.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.261+0000 7feace000640 1 -- 172.21.15.39:0/3198620669 shutdown_connections 2024-09-21T07:49:22.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.261+0000 7feace000640 1 --2- 172.21.15.39:0/3198620669 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feac806f020 0x7feac80cb920 unknown :-1 s=CLOSED pgs=2175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:22.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.261+0000 7feace000640 1 --2- 172.21.15.39:0/3198620669 >> v2:172.21.15.148:3300/0 conn(0x7feac80cc460 0x7feac8070a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:22.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.261+0000 7feace000640 1 -- 172.21.15.39:0/3198620669 >> 172.21.15.39:0/3198620669 conn(0x7feac80b2640 msgr2=0x7feac80d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:22.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.261+0000 7feace000640 1 -- 172.21.15.39:0/3198620669 shutdown_connections 2024-09-21T07:49:22.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.261+0000 7feace000640 1 -- 172.21.15.39:0/3198620669 wait complete. 2024-09-21T07:49:22.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.263+0000 7feace000640 1 Processor -- start 2024-09-21T07:49:22.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.263+0000 7feace000640 1 -- start start 2024-09-21T07:49:22.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.265+0000 7feace000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feac806f020 0x7feac815d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:22.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.265+0000 7feace000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feac80cbe60 0x7feac8155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:22.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.265+0000 7feace000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feac80cc460 0x7feac81f6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:22.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.265+0000 7feace000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7feac80af980 con 0x7feac80cc460 2024-09-21T07:49:22.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.265+0000 7feace000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7feac80af800 con 0x7feac80cbe60 2024-09-21T07:49:22.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.265+0000 7feace000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7feac80ccc20 con 0x7feac806f020 2024-09-21T07:49:22.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.265+0000 7feac7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feac80cc460 0x7feac81f6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:22.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.265+0000 7feac7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feac80cc460 0x7feac81f6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35690/0 (socket says 172.21.15.39:35690) 2024-09-21T07:49:22.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.265+0000 7feac7e00640 1 -- 172.21.15.39:0/337190245 learned_addr learned my addr 172.21.15.39:0/337190245 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:22.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.265+0000 7feac7400640 1 --2- 172.21.15.39:0/337190245 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feac806f020 0x7feac815d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:22.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.265+0000 7feac6a00640 1 --2- 172.21.15.39:0/337190245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feac80cbe60 0x7feac8155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:22.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.266+0000 7feac7e00640 1 -- 172.21.15.39:0/337190245 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feac806f020 msgr2=0x7feac815d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:22.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.266+0000 7feac7e00640 1 --2- 172.21.15.39:0/337190245 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feac806f020 0x7feac815d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:22.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.266+0000 7feac7e00640 1 -- 172.21.15.39:0/337190245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feac80cbe60 msgr2=0x7feac8155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:22.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.266+0000 7feac7e00640 1 --2- 172.21.15.39:0/337190245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feac80cbe60 0x7feac8155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:22.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.266+0000 7feac7e00640 1 -- 172.21.15.39:0/337190245 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac81f3330 con 0x7feac80cc460 2024-09-21T07:49:22.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.266+0000 7feac7e00640 1 --2- 172.21.15.39:0/337190245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feac80cc460 0x7feac81f6080 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7feac000c6b0 tx=0x7feac000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:22.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.266+0000 7feabf400640 1 -- 172.21.15.39:0/337190245 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac0003b70 con 0x7feac80cc460 2024-09-21T07:49:22.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.266+0000 7feabf400640 1 -- 172.21.15.39:0/337190245 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feac0003d30 con 0x7feac80cc460 2024-09-21T07:49:22.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.266+0000 7feabf400640 1 -- 172.21.15.39:0/337190245 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac0005330 con 0x7feac80cc460 2024-09-21T07:49:22.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.266+0000 7feace000640 1 -- 172.21.15.39:0/337190245 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feac81f58e0 con 0x7feac80cc460 2024-09-21T07:49:22.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.266+0000 7feace000640 1 -- 172.21.15.39:0/337190245 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feac815b220 con 0x7feac80cc460 2024-09-21T07:49:22.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.267+0000 7feace000640 1 -- 172.21.15.39:0/337190245 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feac80cb920 con 0x7feac80cc460 2024-09-21T07:49:22.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.269+0000 7feabf400640 1 -- 172.21.15.39:0/337190245 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7feac00054d0 con 0x7feac80cc460 2024-09-21T07:49:22.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.270+0000 7feabf400640 1 --2- 172.21.15.39:0/337190245 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feaa8078b90 0x7feaa807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:22.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.270+0000 7feabf400640 1 -- 172.21.15.39:0/337190245 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(224..224 src has 1..224) ==== 6837+0+0 (secure 0 0 0) 0x7feac009ae80 con 0x7feac80cc460 2024-09-21T07:49:22.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.271+0000 7feac7400640 1 --2- 172.21.15.39:0/337190245 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feaa8078b90 0x7feaa807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:22.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.272+0000 7feac7400640 1 --2- 172.21.15.39:0/337190245 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feaa8078b90 0x7feaa807b0a0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7feab8004630 tx=0x7feab800b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:22.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.275+0000 7feabf400640 1 -- 172.21.15.39:0/337190245 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7feac81f58e0 con 0x7feac80cc460 2024-09-21T07:49:22.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.479+0000 7feace000640 1 -- 172.21.15.39:0/337190245 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7feac8073940 con 0x7feac80cc460 2024-09-21T07:49:22.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.993+0000 7feabf400640 1 -- 172.21.15.39:0/337190245 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v225) ==== 169+0+0 (secure 0 0 0) 0x7feac00627f0 con 0x7feac80cc460 2024-09-21T07:49:22.995 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' removed 2024-09-21T07:49:22.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.996+0000 7feace000640 1 -- 172.21.15.39:0/337190245 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feaa8078b90 msgr2=0x7feaa807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:22.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.996+0000 7feace000640 1 --2- 172.21.15.39:0/337190245 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feaa8078b90 0x7feaa807b0a0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7feab8004630 tx=0x7feab800b2e0 comp rx=0 tx=0).stop 2024-09-21T07:49:22.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.996+0000 7feace000640 1 -- 172.21.15.39:0/337190245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feac80cc460 msgr2=0x7feac81f6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:22.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.996+0000 7feace000640 1 --2- 172.21.15.39:0/337190245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feac80cc460 0x7feac81f6080 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7feac000c6b0 tx=0x7feac000cb70 comp rx=0 tx=0).stop 2024-09-21T07:49:22.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.996+0000 7feace000640 1 -- 172.21.15.39:0/337190245 shutdown_connections 2024-09-21T07:49:22.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.996+0000 7feace000640 1 --2- 172.21.15.39:0/337190245 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feaa8078b90 0x7feaa807b0a0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:22.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.996+0000 7feace000640 1 --2- 172.21.15.39:0/337190245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feac80cc460 0x7feac81f6080 unknown :-1 s=CLOSED pgs=2176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:22.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.996+0000 7feace000640 1 --2- 172.21.15.39:0/337190245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feac80cbe60 0x7feac8155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:22.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.996+0000 7feace000640 1 --2- 172.21.15.39:0/337190245 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feac806f020 0x7feac815d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:22.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.996+0000 7feace000640 1 -- 172.21.15.39:0/337190245 >> 172.21.15.39:0/337190245 conn(0x7feac80b2640 msgr2=0x7feac8073150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:22.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.997+0000 7feace000640 1 -- 172.21.15.39:0/337190245 shutdown_connections 2024-09-21T07:49:22.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:22.997+0000 7feace000640 1 -- 172.21.15.39:0/337190245 wait complete. 2024-09-21T07:49:23.029 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T07:49:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.256+0000 7f764f200640 1 -- 172.21.15.39:0/892779684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76480ccc60 msgr2=0x7f76480cd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.256+0000 7f764f200640 1 --2- 172.21.15.39:0/892779684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76480ccc60 0x7f76480cd070 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f7648071190 tx=0x7f764002a9a0 comp rx=0 tx=0).stop 2024-09-21T07:49:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.256+0000 7f764f200640 1 -- 172.21.15.39:0/892779684 shutdown_connections 2024-09-21T07:49:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.256+0000 7f764f200640 1 --2- 172.21.15.39:0/892779684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76480ccc60 0x7f76480cd070 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.256+0000 7f764f200640 1 --2- 172.21.15.39:0/892779684 >> v2:172.21.15.148:3300/0 conn(0x7f764815d830 0x7f7648156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.256+0000 7f764f200640 1 --2- 172.21.15.39:0/892779684 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f76480b30a0 0x7f76480cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.256+0000 7f764f200640 1 -- 172.21.15.39:0/892779684 >> 172.21.15.39:0/892779684 conn(0x7f76480b2460 msgr2=0x7f764806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.257+0000 7f764f200640 1 -- 172.21.15.39:0/892779684 shutdown_connections 2024-09-21T07:49:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.257+0000 7f764f200640 1 -- 172.21.15.39:0/892779684 wait complete. 2024-09-21T07:49:23.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.259+0000 7f764f200640 1 Processor -- start 2024-09-21T07:49:23.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.259+0000 7f764f200640 1 -- start start 2024-09-21T07:49:23.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.260+0000 7f764f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76480b30a0 0x7f76480c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:23.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.260+0000 7f764f200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76480ccc60 0x7f76480c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:23.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.260+0000 7f764f200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f764815d830 0x7f76480c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:23.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.260+0000 7f764f200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f76480af200 con 0x7f76480b30a0 2024-09-21T07:49:23.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.260+0000 7f764f200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f76480af080 con 0x7f76480ccc60 2024-09-21T07:49:23.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.260+0000 7f764f200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f76480af380 con 0x7f764815d830 2024-09-21T07:49:23.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.261+0000 7f7647e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76480b30a0 0x7f76480c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:23.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.261+0000 7f7647e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76480b30a0 0x7f76480c9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35714/0 (socket says 172.21.15.39:35714) 2024-09-21T07:49:23.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.261+0000 7f7647e00640 1 -- 172.21.15.39:0/1711232959 learned_addr learned my addr 172.21.15.39:0/1711232959 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:23.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.261+0000 7f7647400640 1 --2- 172.21.15.39:0/1711232959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76480ccc60 0x7f76480c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:23.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.261+0000 7f764d200640 1 --2- 172.21.15.39:0/1711232959 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f764815d830 0x7f76480c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:23.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.261+0000 7f7647e00640 1 -- 172.21.15.39:0/1711232959 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f764815d830 msgr2=0x7f76480c7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:23.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.261+0000 7f7647e00640 1 --2- 172.21.15.39:0/1711232959 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f764815d830 0x7f76480c7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:23.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.261+0000 7f7647e00640 1 -- 172.21.15.39:0/1711232959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76480ccc60 msgr2=0x7f76480c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:23.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.261+0000 7f7647e00640 1 --2- 172.21.15.39:0/1711232959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76480ccc60 0x7f76480c4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:23.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.261+0000 7f7647e00640 1 -- 172.21.15.39:0/1711232959 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76481f9530 con 0x7f76480b30a0 2024-09-21T07:49:23.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.261+0000 7f764d200640 1 --2- 172.21.15.39:0/1711232959 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f764815d830 0x7f76480c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:23.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.262+0000 7f7647e00640 1 --2- 172.21.15.39:0/1711232959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76480b30a0 0x7f76480c9f70 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f763800d660 tx=0x7f763800db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:23.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.262+0000 7f763fe00640 1 -- 172.21.15.39:0/1711232959 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7638017070 con 0x7f76480b30a0 2024-09-21T07:49:23.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.262+0000 7f763fe00640 1 -- 172.21.15.39:0/1711232959 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7638005310 con 0x7f76480b30a0 2024-09-21T07:49:23.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.262+0000 7f763fe00640 1 -- 172.21.15.39:0/1711232959 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7638009820 con 0x7f76480b30a0 2024-09-21T07:49:23.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.262+0000 7f764f200640 1 -- 172.21.15.39:0/1711232959 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76481f9810 con 0x7f76480b30a0 2024-09-21T07:49:23.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.262+0000 7f764f200640 1 -- 172.21.15.39:0/1711232959 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76481f9dd0 con 0x7f76480b30a0 2024-09-21T07:49:23.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.263+0000 7f764f200640 1 -- 172.21.15.39:0/1711232959 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7648156bb0 con 0x7f76480b30a0 2024-09-21T07:49:23.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.264+0000 7f763fe00640 1 -- 172.21.15.39:0/1711232959 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f76380099c0 con 0x7f76480b30a0 2024-09-21T07:49:23.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.266+0000 7f763fe00640 1 --2- 172.21.15.39:0/1711232959 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7628078b90 0x7f762807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:23.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.266+0000 7f763fe00640 1 -- 172.21.15.39:0/1711232959 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(225..225 src has 1..225) ==== 6336+0+0 (secure 0 0 0) 0x7f763809afa0 con 0x7f76480b30a0 2024-09-21T07:49:23.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.266+0000 7f7647400640 1 --2- 172.21.15.39:0/1711232959 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7628078b90 0x7f762807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:23.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.267+0000 7f7647400640 1 --2- 172.21.15.39:0/1711232959 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7628078b90 0x7f762807b0a0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f76480c4a80 tx=0x7f763400b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:23.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.271+0000 7f763fe00640 1 -- 172.21.15.39:0/1711232959 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7648156bb0 con 0x7f76480b30a0 2024-09-21T07:49:23.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.468+0000 7f764f200640 1 -- 172.21.15.39:0/1711232959 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f76480b09f0 con 0x7f76480b30a0 2024-09-21T07:49:23.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.473+0000 7f763fe00640 1 -- 172.21.15.39:0/1711232959 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v54) ==== 71+0+0 (secure 0 0 0) 0x7f7638062b20 con 0x7f76480b30a0 2024-09-21T07:49:23.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.475+0000 7f764f200640 1 -- 172.21.15.39:0/1711232959 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7628078b90 msgr2=0x7f762807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:23.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.475+0000 7f764f200640 1 --2- 172.21.15.39:0/1711232959 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7628078b90 0x7f762807b0a0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f76480c4a80 tx=0x7f763400b2e0 comp rx=0 tx=0).stop 2024-09-21T07:49:23.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.475+0000 7f764f200640 1 -- 172.21.15.39:0/1711232959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76480b30a0 msgr2=0x7f76480c9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:23.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.475+0000 7f764f200640 1 --2- 172.21.15.39:0/1711232959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76480b30a0 0x7f76480c9f70 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f763800d660 tx=0x7f763800db20 comp rx=0 tx=0).stop 2024-09-21T07:49:23.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.475+0000 7f764f200640 1 -- 172.21.15.39:0/1711232959 shutdown_connections 2024-09-21T07:49:23.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.475+0000 7f764f200640 1 --2- 172.21.15.39:0/1711232959 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7628078b90 0x7f762807b0a0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:23.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.475+0000 7f764f200640 1 --2- 172.21.15.39:0/1711232959 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f764815d830 0x7f76480c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:23.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.475+0000 7f764f200640 1 --2- 172.21.15.39:0/1711232959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f76480ccc60 0x7f76480c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:23.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.475+0000 7f764f200640 1 --2- 172.21.15.39:0/1711232959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76480b30a0 0x7f76480c9f70 unknown :-1 s=CLOSED pgs=2178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:23.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.475+0000 7f764f200640 1 -- 172.21.15.39:0/1711232959 >> 172.21.15.39:0/1711232959 conn(0x7f76480b2460 msgr2=0x7f76480ad970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:23.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.476+0000 7f764f200640 1 -- 172.21.15.39:0/1711232959 shutdown_connections 2024-09-21T07:49:23.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.476+0000 7f764f200640 1 -- 172.21.15.39:0/1711232959 wait complete. 2024-09-21T07:49:23.501 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_multimds_past_parents' 2024-09-21T07:49:23.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.732+0000 7ff42fe00640 1 -- 172.21.15.39:0/1904411577 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff43006fe10 msgr2=0x7ff43015ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:23.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.732+0000 7ff42fe00640 1 --2- 172.21.15.39:0/1904411577 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff43006fe10 0x7ff43015ee40 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7ff430074af0 tx=0x7ff42802aa30 comp rx=0 tx=0).stop 2024-09-21T07:49:23.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.732+0000 7ff42fe00640 1 -- 172.21.15.39:0/1904411577 shutdown_connections 2024-09-21T07:49:23.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.732+0000 7ff42fe00640 1 --2- 172.21.15.39:0/1904411577 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff43006fe10 0x7ff43015ee40 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:23.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.732+0000 7ff42fe00640 1 --2- 172.21.15.39:0/1904411577 >> v2:172.21.15.148:3300/0 conn(0x7ff4301629a0 0x7ff43006f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:23.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.732+0000 7ff42fe00640 1 --2- 172.21.15.39:0/1904411577 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4300b4cc0 0x7ff430162460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:23.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.732+0000 7ff42fe00640 1 -- 172.21.15.39:0/1904411577 >> 172.21.15.39:0/1904411577 conn(0x7ff4300d0c70 msgr2=0x7ff4300d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:23.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.732+0000 7ff42fe00640 1 -- 172.21.15.39:0/1904411577 shutdown_connections 2024-09-21T07:49:23.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.732+0000 7ff42fe00640 1 -- 172.21.15.39:0/1904411577 wait complete. 2024-09-21T07:49:23.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.734+0000 7ff42fe00640 1 Processor -- start 2024-09-21T07:49:23.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.735+0000 7ff42fe00640 1 -- start start 2024-09-21T07:49:23.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.736+0000 7ff42fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff43006fe10 0x7ff4301f8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:23.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.736+0000 7ff42fe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4300b4cc0 0x7ff430145c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:23.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.736+0000 7ff42fe00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4301629a0 0x7ff4301f5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:23.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.736+0000 7ff42fe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff43015f900 con 0x7ff43006fe10 2024-09-21T07:49:23.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.736+0000 7ff42fe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff43015f780 con 0x7ff4300b4cc0 2024-09-21T07:49:23.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.736+0000 7ff42fe00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7ff43015fa80 con 0x7ff4301629a0 2024-09-21T07:49:23.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.736+0000 7ff42ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff43006fe10 0x7ff4301f8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:23.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.736+0000 7ff42ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff43006fe10 0x7ff4301f8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35732/0 (socket says 172.21.15.39:35732) 2024-09-21T07:49:23.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.736+0000 7ff42ea00640 1 -- 172.21.15.39:0/3693297541 learned_addr learned my addr 172.21.15.39:0/3693297541 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:23.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.736+0000 7ff426000640 1 --2- 172.21.15.39:0/3693297541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4300b4cc0 0x7ff430145c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:23.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.736+0000 7ff42f400640 1 --2- 172.21.15.39:0/3693297541 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4301629a0 0x7ff4301f5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:23.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.737+0000 7ff42ea00640 1 -- 172.21.15.39:0/3693297541 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4301629a0 msgr2=0x7ff4301f5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:23.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.737+0000 7ff42ea00640 1 --2- 172.21.15.39:0/3693297541 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4301629a0 0x7ff4301f5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:23.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.737+0000 7ff42ea00640 1 -- 172.21.15.39:0/3693297541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4300b4cc0 msgr2=0x7ff430145c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:23.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.737+0000 7ff42ea00640 1 --2- 172.21.15.39:0/3693297541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4300b4cc0 0x7ff430145c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:23.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.737+0000 7ff42ea00640 1 -- 172.21.15.39:0/3693297541 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4301f9890 con 0x7ff43006fe10 2024-09-21T07:49:23.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.737+0000 7ff42ea00640 1 --2- 172.21.15.39:0/3693297541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff43006fe10 0x7ff4301f8fb0 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7ff42000d650 tx=0x7ff42000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:23.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.737+0000 7ff427400640 1 -- 172.21.15.39:0/3693297541 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420017070 con 0x7ff43006fe10 2024-09-21T07:49:23.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.737+0000 7ff427400640 1 -- 172.21.15.39:0/3693297541 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff420004310 con 0x7ff43006fe10 2024-09-21T07:49:23.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.737+0000 7ff42fe00640 1 -- 172.21.15.39:0/3693297541 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4301f9b70 con 0x7ff43006fe10 2024-09-21T07:49:23.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.737+0000 7ff427400640 1 -- 172.21.15.39:0/3693297541 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420005280 con 0x7ff43006fe10 2024-09-21T07:49:23.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.737+0000 7ff42fe00640 1 -- 172.21.15.39:0/3693297541 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4300c5950 con 0x7ff43006fe10 2024-09-21T07:49:23.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.738+0000 7ff42fe00640 1 -- 172.21.15.39:0/3693297541 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4300b5270 con 0x7ff43006fe10 2024-09-21T07:49:23.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.740+0000 7ff427400640 1 -- 172.21.15.39:0/3693297541 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff420005420 con 0x7ff43006fe10 2024-09-21T07:49:23.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.741+0000 7ff427400640 1 --2- 172.21.15.39:0/3693297541 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff40c078b90 0x7ff40c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:23.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.741+0000 7ff427400640 1 -- 172.21.15.39:0/3693297541 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(225..225 src has 1..225) ==== 6336+0+0 (secure 0 0 0) 0x7ff42009b450 con 0x7ff43006fe10 2024-09-21T07:49:23.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.741+0000 7ff426000640 1 --2- 172.21.15.39:0/3693297541 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff40c078b90 0x7ff40c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:23.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.742+0000 7ff426000640 1 --2- 172.21.15.39:0/3693297541 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff40c078b90 0x7ff40c07b0a0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7ff418005fd0 tx=0x7ff418005850 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:23.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:23.745+0000 7ff427400640 1 -- 172.21.15.39:0/3693297541 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff4300b5270 con 0x7ff43006fe10 2024-09-21T07:49:24.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:24.017+0000 7ff42fe00640 1 -- 172.21.15.39:0/3693297541 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_multimds_past_parents"]} v 0) -- 0x7ff430162460 con 0x7ff43006fe10 2024-09-21T07:49:25.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.008+0000 7ff427400640 1 -- 172.21.15.39:0/3693297541 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_multimds_past_parents"]}]=0 v792) ==== 147+0+0 (secure 0 0 0) 0x7ff420020020 con 0x7ff43006fe10 2024-09-21T07:49:25.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.011+0000 7ff42fe00640 1 -- 172.21.15.39:0/3693297541 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff40c078b90 msgr2=0x7ff40c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:25.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.011+0000 7ff42fe00640 1 --2- 172.21.15.39:0/3693297541 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff40c078b90 0x7ff40c07b0a0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7ff418005fd0 tx=0x7ff418005850 comp rx=0 tx=0).stop 2024-09-21T07:49:25.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.011+0000 7ff42fe00640 1 -- 172.21.15.39:0/3693297541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff43006fe10 msgr2=0x7ff4301f8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:25.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.011+0000 7ff42fe00640 1 --2- 172.21.15.39:0/3693297541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff43006fe10 0x7ff4301f8fb0 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7ff42000d650 tx=0x7ff42000db10 comp rx=0 tx=0).stop 2024-09-21T07:49:25.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.011+0000 7ff42fe00640 1 -- 172.21.15.39:0/3693297541 shutdown_connections 2024-09-21T07:49:25.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.011+0000 7ff42fe00640 1 --2- 172.21.15.39:0/3693297541 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff40c078b90 0x7ff40c07b0a0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:25.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.011+0000 7ff42fe00640 1 --2- 172.21.15.39:0/3693297541 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff4301629a0 0x7ff4301f5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:25.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.011+0000 7ff42fe00640 1 --2- 172.21.15.39:0/3693297541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff4300b4cc0 0x7ff430145c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:25.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.011+0000 7ff42fe00640 1 --2- 172.21.15.39:0/3693297541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff43006fe10 0x7ff4301f8fb0 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:25.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.011+0000 7ff42fe00640 1 -- 172.21.15.39:0/3693297541 >> 172.21.15.39:0/3693297541 conn(0x7ff4300d0c70 msgr2=0x7ff4300afcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:25.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.011+0000 7ff42fe00640 1 -- 172.21.15.39:0/3693297541 shutdown_connections 2024-09-21T07:49:25.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.011+0000 7ff42fe00640 1 -- 172.21.15.39:0/3693297541 wait complete. 2024-09-21T07:49:25.043 INFO:tasks.cephfs_test_runner:check if past parents are properly recorded during across authority rename ... ok 2024-09-21T07:49:25.044 INFO:tasks.cephfs_test_runner:Starting test: test_snapclient_cache (tasks.cephfs.test_snapshots.TestSnapshots) check if snapclient cache gets synced properly 2024-09-21T07:49:25.044 INFO:tasks.cephfs_test_runner:test_snapclient_cache (tasks.cephfs.test_snapshots.TestSnapshots) 2024-09-21T07:49:25.044 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:49:25.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.273+0000 7efd2ac00640 1 -- 172.21.15.39:0/2242157095 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efd24160660 msgr2=0x7efd240cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:25.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.273+0000 7efd2ac00640 1 --2- 172.21.15.39:0/2242157095 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efd24160660 0x7efd240cf710 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efd10005fd0 tx=0x7efd1002a5d0 comp rx=0 tx=0).stop 2024-09-21T07:49:25.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.274+0000 7efd2ac00640 1 -- 172.21.15.39:0/2242157095 shutdown_connections 2024-09-21T07:49:25.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.274+0000 7efd2ac00640 1 --2- 172.21.15.39:0/2242157095 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efd24160660 0x7efd240cf710 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:25.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.274+0000 7efd2ac00640 1 --2- 172.21.15.39:0/2242157095 >> v2:172.21.15.148:3300/0 conn(0x7efd24160250 0x7efd2415d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:25.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.274+0000 7efd2ac00640 1 -- 172.21.15.39:0/2242157095 >> 172.21.15.39:0/2242157095 conn(0x7efd2406cfc0 msgr2=0x7efd2406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:25.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.274+0000 7efd2ac00640 1 -- 172.21.15.39:0/2242157095 shutdown_connections 2024-09-21T07:49:25.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.274+0000 7efd2ac00640 1 -- 172.21.15.39:0/2242157095 wait complete. 2024-09-21T07:49:25.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.276+0000 7efd2ac00640 1 Processor -- start 2024-09-21T07:49:25.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.276+0000 7efd2ac00640 1 -- start start 2024-09-21T07:49:25.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.277+0000 7efd2ac00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd240cfc50 0x7efd241407e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:25.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.277+0000 7efd2ac00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd24160250 0x7efd2413fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:25.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.278+0000 7efd2ac00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efd24160660 0x7efd24141eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:25.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.278+0000 7efd2ac00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efd240ccdc0 con 0x7efd240cfc50 2024-09-21T07:49:25.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.278+0000 7efd2ac00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efd240ccc40 con 0x7efd24160250 2024-09-21T07:49:25.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.278+0000 7efd2ac00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7efd240ccf40 con 0x7efd24160660 2024-09-21T07:49:25.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.278+0000 7efd29800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd240cfc50 0x7efd241407e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:25.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.278+0000 7efd29800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd240cfc50 0x7efd241407e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35738/0 (socket says 172.21.15.39:35738) 2024-09-21T07:49:25.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.278+0000 7efd29800640 1 -- 172.21.15.39:0/1863363364 learned_addr learned my addr 172.21.15.39:0/1863363364 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:25.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.278+0000 7efd28e00640 1 --2- 172.21.15.39:0/1863363364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd24160250 0x7efd2413fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:25.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.278+0000 7efd2a200640 1 --2- 172.21.15.39:0/1863363364 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efd24160660 0x7efd24141eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:25.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.278+0000 7efd29800640 1 -- 172.21.15.39:0/1863363364 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efd24160660 msgr2=0x7efd24141eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:25.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.278+0000 7efd29800640 1 --2- 172.21.15.39:0/1863363364 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efd24160660 0x7efd24141eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:25.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.278+0000 7efd29800640 1 -- 172.21.15.39:0/1863363364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd24160250 msgr2=0x7efd2413fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:25.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.278+0000 7efd29800640 1 --2- 172.21.15.39:0/1863363364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd24160250 0x7efd2413fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:25.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.278+0000 7efd29800640 1 -- 172.21.15.39:0/1863363364 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd24141b80 con 0x7efd240cfc50 2024-09-21T07:49:25.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.279+0000 7efd29800640 1 --2- 172.21.15.39:0/1863363364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd240cfc50 0x7efd241407e0 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7efd1800e790 tx=0x7efd1800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:25.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.279+0000 7efd21600640 1 -- 172.21.15.39:0/1863363364 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1800eea0 con 0x7efd240cfc50 2024-09-21T07:49:25.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.279+0000 7efd21600640 1 -- 172.21.15.39:0/1863363364 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd1800cdb0 con 0x7efd240cfc50 2024-09-21T07:49:25.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.279+0000 7efd2ac00640 1 -- 172.21.15.39:0/1863363364 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd241412e0 con 0x7efd240cfc50 2024-09-21T07:49:25.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.279+0000 7efd21600640 1 -- 172.21.15.39:0/1863363364 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd18010690 con 0x7efd240cfc50 2024-09-21T07:49:25.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.279+0000 7efd2ac00640 1 -- 172.21.15.39:0/1863363364 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd241418a0 con 0x7efd240cfc50 2024-09-21T07:49:25.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.280+0000 7efd2ac00640 1 -- 172.21.15.39:0/1863363364 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd240696c0 con 0x7efd240cfc50 2024-09-21T07:49:25.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.282+0000 7efd21600640 1 -- 172.21.15.39:0/1863363364 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7efd18010830 con 0x7efd240cfc50 2024-09-21T07:49:25.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.282+0000 7efd21600640 1 --2- 172.21.15.39:0/1863363364 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efd04078bc0 0x7efd0407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:25.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.283+0000 7efd21600640 1 -- 172.21.15.39:0/1863363364 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(225..225 src has 1..225) ==== 6336+0+0 (secure 0 0 0) 0x7efd1809b8f0 con 0x7efd240cfc50 2024-09-21T07:49:25.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.284+0000 7efd28e00640 1 --2- 172.21.15.39:0/1863363364 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efd04078bc0 0x7efd0407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:25.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.285+0000 7efd28e00640 1 --2- 172.21.15.39:0/1863363364 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efd04078bc0 0x7efd0407b0d0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7efd10033000 tx=0x7efd10004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:25.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.287+0000 7efd21600640 1 -- 172.21.15.39:0/1863363364 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7efd240696c0 con 0x7efd240cfc50 2024-09-21T07:49:25.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.482+0000 7efd2ac00640 1 -- 172.21.15.39:0/1863363364 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efd240cd3f0 con 0x7efd240cfc50 2024-09-21T07:49:25.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.484+0000 7efd21600640 1 -- 172.21.15.39:0/1863363364 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v225) ==== 74+0+18986 (secure 0 0 0) 0x7efd18063480 con 0x7efd240cfc50 2024-09-21T07:49:25.486 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:49:25.486 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":225,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:49:22.992476+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":220,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":220,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:6836/1256253563":"2024-09-22T07:49:19.548115+0000","172.21.15.148:6834/1256253563":"2024-09-22T07:49:19.548115+0000","172.21.15.39:6834/2837221709":"2024-09-22T07:49:19.548133+0000","172.21.15.39:6832/2837221709":"2024-09-22T07:49:19.548133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]},{"pool":20,"snaps":[{"begin":80,"length":23},{"begin":181,"length":23}]},{"pool":22,"snaps":[{"begin":100,"length":1},{"begin":199,"length":1}]},{"pool":24,"snaps":[{"begin":2,"length":2}]},{"pool":26,"snaps":[{"begin":2,"length":2}]},{"pool":28,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:49:25.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.486+0000 7efd2ac00640 1 -- 172.21.15.39:0/1863363364 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efd04078bc0 msgr2=0x7efd0407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:25.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.486+0000 7efd2ac00640 1 --2- 172.21.15.39:0/1863363364 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efd04078bc0 0x7efd0407b0d0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7efd10033000 tx=0x7efd10004730 comp rx=0 tx=0).stop 2024-09-21T07:49:25.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.486+0000 7efd2ac00640 1 -- 172.21.15.39:0/1863363364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd240cfc50 msgr2=0x7efd241407e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:25.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.486+0000 7efd2ac00640 1 --2- 172.21.15.39:0/1863363364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd240cfc50 0x7efd241407e0 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7efd1800e790 tx=0x7efd1800ec50 comp rx=0 tx=0).stop 2024-09-21T07:49:25.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.486+0000 7efd2ac00640 1 -- 172.21.15.39:0/1863363364 shutdown_connections 2024-09-21T07:49:25.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.486+0000 7efd2ac00640 1 --2- 172.21.15.39:0/1863363364 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efd04078bc0 0x7efd0407b0d0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:25.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.486+0000 7efd2ac00640 1 --2- 172.21.15.39:0/1863363364 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efd24160660 0x7efd24141eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:25.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.486+0000 7efd2ac00640 1 --2- 172.21.15.39:0/1863363364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd24160250 0x7efd2413fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:25.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.486+0000 7efd2ac00640 1 --2- 172.21.15.39:0/1863363364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd240cfc50 0x7efd241407e0 unknown :-1 s=CLOSED pgs=2181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:25.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.486+0000 7efd2ac00640 1 -- 172.21.15.39:0/1863363364 >> 172.21.15.39:0/1863363364 conn(0x7efd2406cfc0 msgr2=0x7efd24073f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:25.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.486+0000 7efd2ac00640 1 -- 172.21.15.39:0/1863363364 shutdown_connections 2024-09-21T07:49:25.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.486+0000 7efd2ac00640 1 -- 172.21.15.39:0/1863363364 wait complete. 2024-09-21T07:49:25.510 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T07:33:01.569433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'snap-test-pool', 'create_time': '2024-09-21T07:34:22.032185+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 1, 'snap_epoch': 37, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'test_data_pool', 'create_time': '2024-09-21T07:35:25.604532+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 1, 'snap_epoch': 66, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_metadata_pool', 'create_time': '2024-09-21T07:35:26.327607+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T07:49:25.511 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T07:49:25.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.737+0000 7f2007e00640 1 -- 172.21.15.39:0/2636959407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2008160f10 msgr2=0x7f20080709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:25.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.737+0000 7f2007e00640 1 --2- 172.21.15.39:0/2636959407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2008160f10 0x7f20080709c0 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f200815dba0 tx=0x7f1ff802a5c0 comp rx=0 tx=0).stop 2024-09-21T07:49:25.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.737+0000 7f2007e00640 1 -- 172.21.15.39:0/2636959407 shutdown_connections 2024-09-21T07:49:25.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.737+0000 7f2007e00640 1 --2- 172.21.15.39:0/2636959407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2008160f10 0x7f20080709c0 unknown :-1 s=CLOSED pgs=2183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:25.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.737+0000 7f2007e00640 1 -- 172.21.15.39:0/2636959407 >> 172.21.15.39:0/2636959407 conn(0x7f200806def0 msgr2=0x7f200806e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:25.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.737+0000 7f2007e00640 1 -- 172.21.15.39:0/2636959407 shutdown_connections 2024-09-21T07:49:25.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.737+0000 7f2007e00640 1 -- 172.21.15.39:0/2636959407 wait complete. 2024-09-21T07:49:25.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.740+0000 7f2007e00640 1 Processor -- start 2024-09-21T07:49:25.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.740+0000 7f2007e00640 1 -- start start 2024-09-21T07:49:25.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.741+0000 7f2007e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20080d1d30 0x7f2008134910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:25.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.741+0000 7f2007e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2008160f10 0x7f200815b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:25.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.741+0000 7f2007e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f200813b920 0x7f200815bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:25.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.741+0000 7f2007e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f20080ce6c0 con 0x7f2008160f10 2024-09-21T07:49:25.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.741+0000 7f2007e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f20080ce540 con 0x7f200813b920 2024-09-21T07:49:25.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.741+0000 7f2007e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f20080ce840 con 0x7f20080d1d30 2024-09-21T07:49:25.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.741+0000 7f1ffe000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2008160f10 0x7f200815b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:25.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.742+0000 7f1ffe000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2008160f10 0x7f200815b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35750/0 (socket says 172.21.15.39:35750) 2024-09-21T07:49:25.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.742+0000 7f1ffe000640 1 -- 172.21.15.39:0/1686645553 learned_addr learned my addr 172.21.15.39:0/1686645553 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:25.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.742+0000 7f2007400640 1 --2- 172.21.15.39:0/1686645553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f200813b920 0x7f200815bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:25.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.742+0000 7f2006a00640 1 --2- 172.21.15.39:0/1686645553 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20080d1d30 0x7f2008134910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:25.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.742+0000 7f1ffe000640 1 -- 172.21.15.39:0/1686645553 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20080d1d30 msgr2=0x7f2008134910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:25.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.742+0000 7f1ffe000640 1 --2- 172.21.15.39:0/1686645553 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20080d1d30 0x7f2008134910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:25.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.742+0000 7f1ffe000640 1 -- 172.21.15.39:0/1686645553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f200813b920 msgr2=0x7f200815bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:25.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.742+0000 7f1ffe000640 1 --2- 172.21.15.39:0/1686645553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f200813b920 0x7f200815bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:25.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.742+0000 7f1ffe000640 1 -- 172.21.15.39:0/1686645553 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f200813f8c0 con 0x7f2008160f10 2024-09-21T07:49:25.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.742+0000 7f1ffe000640 1 --2- 172.21.15.39:0/1686645553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2008160f10 0x7f200815b740 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0001eb0 tx=0x7f1ff000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:25.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.743+0000 7f1fff400640 1 -- 172.21.15.39:0/1686645553 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0011790 con 0x7f2008160f10 2024-09-21T07:49:25.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.743+0000 7f1fff400640 1 -- 172.21.15.39:0/1686645553 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ff0017070 con 0x7f2008160f10 2024-09-21T07:49:25.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.743+0000 7f2007e00640 1 -- 172.21.15.39:0/1686645553 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f200813c0c0 con 0x7f2008160f10 2024-09-21T07:49:25.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.743+0000 7f1fff400640 1 -- 172.21.15.39:0/1686645553 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0011d00 con 0x7f2008160f10 2024-09-21T07:49:25.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.743+0000 7f2007e00640 1 -- 172.21.15.39:0/1686645553 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20080b5d70 con 0x7f2008160f10 2024-09-21T07:49:25.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.744+0000 7f2007e00640 1 -- 172.21.15.39:0/1686645553 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20080b0aa0 con 0x7f2008160f10 2024-09-21T07:49:25.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.745+0000 7f1fff400640 1 -- 172.21.15.39:0/1686645553 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1ff001c460 con 0x7f2008160f10 2024-09-21T07:49:25.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.746+0000 7f1fff400640 1 --2- 172.21.15.39:0/1686645553 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1fe4078b90 0x7f1fe407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:25.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.746+0000 7f1fff400640 1 -- 172.21.15.39:0/1686645553 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(225..225 src has 1..225) ==== 6336+0+0 (secure 0 0 0) 0x7f1ff009ab90 con 0x7f2008160f10 2024-09-21T07:49:25.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.747+0000 7f2006a00640 1 --2- 172.21.15.39:0/1686645553 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1fe4078b90 0x7f1fe407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:25.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.747+0000 7f2006a00640 1 --2- 172.21.15.39:0/1686645553 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1fe4078b90 0x7f1fe407b0a0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8034000 tx=0x7f1ff8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:25.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.751+0000 7f1fff400640 1 -- 172.21.15.39:0/1686645553 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1ff00627a0 con 0x7f2008160f10 2024-09-21T07:49:25.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.951+0000 7f2007e00640 1 -- 172.21.15.39:0/1686645553 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f20080b3490 con 0x7f2008160f10 2024-09-21T07:49:25.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.952+0000 7f1fff400640 1 -- 172.21.15.39:0/1686645553 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v225) ==== 93+0+10 (secure 0 0 0) 0x7f1ff00671d0 con 0x7f2008160f10 2024-09-21T07:49:25.954 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-09-21T07:49:25.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.954+0000 7f2007e00640 1 -- 172.21.15.39:0/1686645553 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1fe4078b90 msgr2=0x7f1fe407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:25.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.954+0000 7f2007e00640 1 --2- 172.21.15.39:0/1686645553 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1fe4078b90 0x7f1fe407b0a0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8034000 tx=0x7f1ff8004730 comp rx=0 tx=0).stop 2024-09-21T07:49:25.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.954+0000 7f2007e00640 1 -- 172.21.15.39:0/1686645553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2008160f10 msgr2=0x7f200815b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:25.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.954+0000 7f2007e00640 1 --2- 172.21.15.39:0/1686645553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2008160f10 0x7f200815b740 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0001eb0 tx=0x7f1ff000c820 comp rx=0 tx=0).stop 2024-09-21T07:49:25.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.954+0000 7f2007e00640 1 -- 172.21.15.39:0/1686645553 shutdown_connections 2024-09-21T07:49:25.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.954+0000 7f2007e00640 1 --2- 172.21.15.39:0/1686645553 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1fe4078b90 0x7f1fe407b0a0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:25.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.954+0000 7f2007e00640 1 --2- 172.21.15.39:0/1686645553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f200813b920 0x7f200815bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:25.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.954+0000 7f2007e00640 1 --2- 172.21.15.39:0/1686645553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2008160f10 0x7f200815b740 unknown :-1 s=CLOSED pgs=2184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:25.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.954+0000 7f2007e00640 1 --2- 172.21.15.39:0/1686645553 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20080d1d30 0x7f2008134910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:25.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.954+0000 7f2007e00640 1 -- 172.21.15.39:0/1686645553 >> 172.21.15.39:0/1686645553 conn(0x7f200806def0 msgr2=0x7f200806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:25.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.954+0000 7f2007e00640 1 -- 172.21.15.39:0/1686645553 shutdown_connections 2024-09-21T07:49:25.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:25.954+0000 7f2007e00640 1 -- 172.21.15.39:0/1686645553 wait complete. 2024-09-21T07:49:25.978 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2024-09-21T07:49:26.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.207+0000 7fc07f200640 1 -- 172.21.15.39:0/411817337 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc080160250 msgr2=0x7fc08015d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:26.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.207+0000 7fc07f200640 1 --2- 172.21.15.39:0/411817337 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc080160250 0x7fc08015d570 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7fc070005fd0 tx=0x7fc07002a580 comp rx=0 tx=0).stop 2024-09-21T07:49:26.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.207+0000 7fc07f200640 1 -- 172.21.15.39:0/411817337 shutdown_connections 2024-09-21T07:49:26.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.207+0000 7fc07f200640 1 --2- 172.21.15.39:0/411817337 >> v2:172.21.15.148:3300/0 conn(0x7fc0800d0390 0x7fc0800d07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:26.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.207+0000 7fc07f200640 1 --2- 172.21.15.39:0/411817337 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc080160250 0x7fc08015d570 unknown :-1 s=CLOSED pgs=2185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:26.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.207+0000 7fc07f200640 1 -- 172.21.15.39:0/411817337 >> 172.21.15.39:0/411817337 conn(0x7fc08006cfc0 msgr2=0x7fc08006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:26.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.207+0000 7fc07f200640 1 -- 172.21.15.39:0/411817337 shutdown_connections 2024-09-21T07:49:26.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.207+0000 7fc07f200640 1 -- 172.21.15.39:0/411817337 wait complete. 2024-09-21T07:49:26.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.209+0000 7fc07f200640 1 Processor -- start 2024-09-21T07:49:26.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.210+0000 7fc07f200640 1 -- start start 2024-09-21T07:49:26.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.211+0000 7fc07f200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0800d0390 0x7fc0800aef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:26.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.211+0000 7fc07f200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc080160250 0x7fc0800b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:26.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.211+0000 7fc07f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc080160660 0x7fc08014b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:26.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.211+0000 7fc07f200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc0800ccdc0 con 0x7fc080160660 2024-09-21T07:49:26.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.211+0000 7fc07f200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc0800ccc40 con 0x7fc0800d0390 2024-09-21T07:49:26.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.211+0000 7fc07f200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc0800ccf40 con 0x7fc080160250 2024-09-21T07:49:26.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.211+0000 7fc07e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc080160660 0x7fc08014b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:26.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.211+0000 7fc07e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc080160660 0x7fc08014b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35768/0 (socket says 172.21.15.39:35768) 2024-09-21T07:49:26.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.211+0000 7fc07e800640 1 -- 172.21.15.39:0/2262014020 learned_addr learned my addr 172.21.15.39:0/2262014020 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:26.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.212+0000 7fc07de00640 1 --2- 172.21.15.39:0/2262014020 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0800d0390 0x7fc0800aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:26.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.212+0000 7fc077e00640 1 --2- 172.21.15.39:0/2262014020 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc080160250 0x7fc0800b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:26.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.212+0000 7fc07e800640 1 -- 172.21.15.39:0/2262014020 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc080160250 msgr2=0x7fc0800b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:26.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.212+0000 7fc07e800640 1 --2- 172.21.15.39:0/2262014020 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc080160250 0x7fc0800b12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:26.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.212+0000 7fc07e800640 1 -- 172.21.15.39:0/2262014020 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0800d0390 msgr2=0x7fc0800aef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:26.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.212+0000 7fc07e800640 1 --2- 172.21.15.39:0/2262014020 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0800d0390 0x7fc0800aef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:26.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.212+0000 7fc07e800640 1 -- 172.21.15.39:0/2262014020 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0801f9350 con 0x7fc080160660 2024-09-21T07:49:26.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.212+0000 7fc07de00640 1 --2- 172.21.15.39:0/2262014020 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0800d0390 0x7fc0800aef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:26.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.212+0000 7fc07e800640 1 --2- 172.21.15.39:0/2262014020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc080160660 0x7fc08014b820 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7fc07800cad0 tx=0x7fc07800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:26.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.212+0000 7fc076000640 1 -- 172.21.15.39:0/2262014020 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07800ae20 con 0x7fc080160660 2024-09-21T07:49:26.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.212+0000 7fc076000640 1 -- 172.21.15.39:0/2262014020 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc07800f450 con 0x7fc080160660 2024-09-21T07:49:26.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.212+0000 7fc076000640 1 -- 172.21.15.39:0/2262014020 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07800f8c0 con 0x7fc080160660 2024-09-21T07:49:26.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.212+0000 7fc07f200640 1 -- 172.21.15.39:0/2262014020 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0801f9630 con 0x7fc080160660 2024-09-21T07:49:26.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.213+0000 7fc07f200640 1 -- 172.21.15.39:0/2262014020 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0801f9bf0 con 0x7fc080160660 2024-09-21T07:49:26.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.214+0000 7fc07f200640 1 -- 172.21.15.39:0/2262014020 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0800cd460 con 0x7fc080160660 2024-09-21T07:49:26.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.215+0000 7fc076000640 1 -- 172.21.15.39:0/2262014020 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc07800fa60 con 0x7fc080160660 2024-09-21T07:49:26.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.216+0000 7fc076000640 1 --2- 172.21.15.39:0/2262014020 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc05c078b90 0x7fc05c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:26.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.216+0000 7fc076000640 1 -- 172.21.15.39:0/2262014020 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(225..225 src has 1..225) ==== 6336+0+0 (secure 0 0 0) 0x7fc07809baf0 con 0x7fc080160660 2024-09-21T07:49:26.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.216+0000 7fc07de00640 1 --2- 172.21.15.39:0/2262014020 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc05c078b90 0x7fc05c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:26.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.217+0000 7fc07de00640 1 --2- 172.21.15.39:0/2262014020 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc05c078b90 0x7fc05c07b0a0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fc070033000 tx=0x7fc070004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:26.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.220+0000 7fc076000640 1 -- 172.21.15.39:0/2262014020 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc078014030 con 0x7fc080160660 2024-09-21T07:49:26.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.417+0000 7fc07f200640 1 -- 172.21.15.39:0/2262014020 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) -- 0x7fc0801f90e0 con 0x7fc080160660 2024-09-21T07:49:26.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.418+0000 7fc076000640 1 -- 172.21.15.39:0/2262014020 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v225) ==== 103+0+11 (secure 0 0 0) 0x7fc078063cd0 con 0x7fc080160660 2024-09-21T07:49:26.420 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-21T07:49:26.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.420+0000 7fc07f200640 1 -- 172.21.15.39:0/2262014020 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc05c078b90 msgr2=0x7fc05c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:26.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.420+0000 7fc07f200640 1 --2- 172.21.15.39:0/2262014020 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc05c078b90 0x7fc05c07b0a0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fc070033000 tx=0x7fc070004730 comp rx=0 tx=0).stop 2024-09-21T07:49:26.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.420+0000 7fc07f200640 1 -- 172.21.15.39:0/2262014020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc080160660 msgr2=0x7fc08014b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:26.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.420+0000 7fc07f200640 1 --2- 172.21.15.39:0/2262014020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc080160660 0x7fc08014b820 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7fc07800cad0 tx=0x7fc07800cf90 comp rx=0 tx=0).stop 2024-09-21T07:49:26.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.420+0000 7fc07f200640 1 -- 172.21.15.39:0/2262014020 shutdown_connections 2024-09-21T07:49:26.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.420+0000 7fc07f200640 1 --2- 172.21.15.39:0/2262014020 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc05c078b90 0x7fc05c07b0a0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:26.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.420+0000 7fc07f200640 1 --2- 172.21.15.39:0/2262014020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc080160660 0x7fc08014b820 unknown :-1 s=CLOSED pgs=2186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:26.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.420+0000 7fc07f200640 1 --2- 172.21.15.39:0/2262014020 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc080160250 0x7fc0800b12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:26.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.420+0000 7fc07f200640 1 --2- 172.21.15.39:0/2262014020 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0800d0390 0x7fc0800aef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:26.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.420+0000 7fc07f200640 1 -- 172.21.15.39:0/2262014020 >> 172.21.15.39:0/2262014020 conn(0x7fc08006cfc0 msgr2=0x7fc08015e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:26.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.420+0000 7fc07f200640 1 -- 172.21.15.39:0/2262014020 shutdown_connections 2024-09-21T07:49:26.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.421+0000 7fc07f200640 1 -- 172.21.15.39:0/2262014020 wait complete. 2024-09-21T07:49:26.448 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2024-09-21T07:49:26.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.678+0000 7faee7e00640 1 -- 172.21.15.39:0/4190212299 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faee80d0400 msgr2=0x7faee80b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:26.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.678+0000 7faee7e00640 1 --2- 172.21.15.39:0/4190212299 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faee80d0400 0x7faee80b43f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7faed8009830 tx=0x7faed802a490 comp rx=0 tx=0).stop 2024-09-21T07:49:26.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.678+0000 7faee7e00640 1 -- 172.21.15.39:0/4190212299 shutdown_connections 2024-09-21T07:49:26.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.678+0000 7faee7e00640 1 --2- 172.21.15.39:0/4190212299 >> v2:172.21.15.148:3300/0 conn(0x7faee806f4e0 0x7faee806f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:26.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.678+0000 7faee7e00640 1 --2- 172.21.15.39:0/4190212299 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faee80d0400 0x7faee80b43f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:26.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.678+0000 7faee7e00640 1 -- 172.21.15.39:0/4190212299 >> 172.21.15.39:0/4190212299 conn(0x7faee80d0c70 msgr2=0x7faee80d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:26.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.678+0000 7faee7e00640 1 -- 172.21.15.39:0/4190212299 shutdown_connections 2024-09-21T07:49:26.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.678+0000 7faee7e00640 1 -- 172.21.15.39:0/4190212299 wait complete. 2024-09-21T07:49:26.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.680+0000 7faee7e00640 1 Processor -- start 2024-09-21T07:49:26.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.680+0000 7faee7e00640 1 -- start start 2024-09-21T07:49:26.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.682+0000 7faee7e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faee806f4e0 0x7faee81f8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:26.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.682+0000 7faee7e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faee80d0400 0x7faee8070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:26.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.682+0000 7faee7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faee815b640 0x7faee8070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:26.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.682+0000 7faee7e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7faee80b54f0 con 0x7faee815b640 2024-09-21T07:49:26.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.682+0000 7faee7e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7faee80b5370 con 0x7faee806f4e0 2024-09-21T07:49:26.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.682+0000 7faee7e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7faee80b5670 con 0x7faee80d0400 2024-09-21T07:49:26.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.682+0000 7faee7400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faee815b640 0x7faee8070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:26.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.682+0000 7faee7400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faee815b640 0x7faee8070550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35780/0 (socket says 172.21.15.39:35780) 2024-09-21T07:49:26.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.682+0000 7faee7400640 1 -- 172.21.15.39:0/443763177 learned_addr learned my addr 172.21.15.39:0/443763177 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:26.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.682+0000 7faedfe00640 1 --2- 172.21.15.39:0/443763177 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faee80d0400 0x7faee8070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:26.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.682+0000 7faee6a00640 1 --2- 172.21.15.39:0/443763177 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faee806f4e0 0x7faee81f8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:26.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.683+0000 7faee7400640 1 -- 172.21.15.39:0/443763177 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faee80d0400 msgr2=0x7faee8070010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:26.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.683+0000 7faee7400640 1 --2- 172.21.15.39:0/443763177 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faee80d0400 0x7faee8070010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:26.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.683+0000 7faee7400640 1 -- 172.21.15.39:0/443763177 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faee806f4e0 msgr2=0x7faee81f8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:26.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.683+0000 7faee7400640 1 --2- 172.21.15.39:0/443763177 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faee806f4e0 0x7faee81f8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:26.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.683+0000 7faee7400640 1 -- 172.21.15.39:0/443763177 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee81e98c0 con 0x7faee815b640 2024-09-21T07:49:26.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.683+0000 7faee7400640 1 --2- 172.21.15.39:0/443763177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faee815b640 0x7faee8070550 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7faee000ea40 tx=0x7faee000ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:26.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.683+0000 7faedea00640 1 -- 172.21.15.39:0/443763177 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee000ce40 con 0x7faee815b640 2024-09-21T07:49:26.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.683+0000 7faedea00640 1 -- 172.21.15.39:0/443763177 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faee0004c10 con 0x7faee815b640 2024-09-21T07:49:26.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.683+0000 7faee7e00640 1 -- 172.21.15.39:0/443763177 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faee8159e10 con 0x7faee815b640 2024-09-21T07:49:26.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.683+0000 7faedea00640 1 -- 172.21.15.39:0/443763177 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee0005020 con 0x7faee815b640 2024-09-21T07:49:26.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.683+0000 7faee7e00640 1 -- 172.21.15.39:0/443763177 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faee815a3d0 con 0x7faee815b640 2024-09-21T07:49:26.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.684+0000 7faee7e00640 1 -- 172.21.15.39:0/443763177 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faee8159b80 con 0x7faee815b640 2024-09-21T07:49:26.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.686+0000 7faedea00640 1 -- 172.21.15.39:0/443763177 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7faee00051c0 con 0x7faee815b640 2024-09-21T07:49:26.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.687+0000 7faedea00640 1 --2- 172.21.15.39:0/443763177 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faec4078bc0 0x7faec407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:26.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.687+0000 7faedea00640 1 -- 172.21.15.39:0/443763177 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(225..225 src has 1..225) ==== 6336+0+0 (secure 0 0 0) 0x7faee009bce0 con 0x7faee815b640 2024-09-21T07:49:26.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.687+0000 7faee6a00640 1 --2- 172.21.15.39:0/443763177 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faec4078bc0 0x7faec407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:26.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.688+0000 7faee6a00640 1 --2- 172.21.15.39:0/443763177 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faec4078bc0 0x7faec407b0d0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7faed8034000 tx=0x7faed8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:26.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.691+0000 7faedea00640 1 -- 172.21.15.39:0/443763177 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7faee0063870 con 0x7faee815b640 2024-09-21T07:49:26.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.888+0000 7faee7e00640 1 -- 172.21.15.39:0/443763177 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) -- 0x7faee81e78e0 con 0x7faee815b640 2024-09-21T07:49:26.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.888+0000 7faedea00640 1 -- 172.21.15.39:0/443763177 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v225) ==== 103+0+11 (secure 0 0 0) 0x7faee81e78e0 con 0x7faee815b640 2024-09-21T07:49:26.890 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-21T07:49:26.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.890+0000 7faee7e00640 1 -- 172.21.15.39:0/443763177 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faec4078bc0 msgr2=0x7faec407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:26.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.891+0000 7faee7e00640 1 --2- 172.21.15.39:0/443763177 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faec4078bc0 0x7faec407b0d0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7faed8034000 tx=0x7faed8004730 comp rx=0 tx=0).stop 2024-09-21T07:49:26.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.891+0000 7faee7e00640 1 -- 172.21.15.39:0/443763177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faee815b640 msgr2=0x7faee8070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:26.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.891+0000 7faee7e00640 1 --2- 172.21.15.39:0/443763177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faee815b640 0x7faee8070550 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7faee000ea40 tx=0x7faee000ef00 comp rx=0 tx=0).stop 2024-09-21T07:49:26.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.891+0000 7faee7e00640 1 -- 172.21.15.39:0/443763177 shutdown_connections 2024-09-21T07:49:26.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.891+0000 7faee7e00640 1 --2- 172.21.15.39:0/443763177 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faec4078bc0 0x7faec407b0d0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:26.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.891+0000 7faee7e00640 1 --2- 172.21.15.39:0/443763177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faee815b640 0x7faee8070550 unknown :-1 s=CLOSED pgs=2188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:26.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.891+0000 7faee7e00640 1 --2- 172.21.15.39:0/443763177 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faee80d0400 0x7faee8070010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:26.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.891+0000 7faee7e00640 1 --2- 172.21.15.39:0/443763177 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faee806f4e0 0x7faee81f8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:26.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.891+0000 7faee7e00640 1 -- 172.21.15.39:0/443763177 >> 172.21.15.39:0/443763177 conn(0x7faee80d0c70 msgr2=0x7faee81ea670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:26.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.891+0000 7faee7e00640 1 -- 172.21.15.39:0/443763177 shutdown_connections 2024-09-21T07:49:26.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:26.891+0000 7faee7e00640 1 -- 172.21.15.39:0/443763177 wait complete. 2024-09-21T07:49:26.916 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2024-09-21T07:49:27.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.145+0000 7f52d6a00640 1 -- 172.21.15.39:0/902168395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52d0160250 msgr2=0x7f52d015d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:27.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.145+0000 7f52d6a00640 1 --2- 172.21.15.39:0/902168395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52d0160250 0x7f52d015d570 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7f52c4005fd0 tx=0x7f52c402a5d0 comp rx=0 tx=0).stop 2024-09-21T07:49:27.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.145+0000 7f52d6a00640 1 -- 172.21.15.39:0/902168395 shutdown_connections 2024-09-21T07:49:27.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.145+0000 7f52d6a00640 1 --2- 172.21.15.39:0/902168395 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52d0160660 0x7f52d00cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:27.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.145+0000 7f52d6a00640 1 --2- 172.21.15.39:0/902168395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52d0160250 0x7f52d015d570 unknown :-1 s=CLOSED pgs=2189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:27.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.145+0000 7f52d6a00640 1 -- 172.21.15.39:0/902168395 >> 172.21.15.39:0/902168395 conn(0x7f52d006cfc0 msgr2=0x7f52d006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:27.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.145+0000 7f52d6a00640 1 -- 172.21.15.39:0/902168395 shutdown_connections 2024-09-21T07:49:27.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.145+0000 7f52d6a00640 1 -- 172.21.15.39:0/902168395 wait complete. 2024-09-21T07:49:27.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.147+0000 7f52d6a00640 1 Processor -- start 2024-09-21T07:49:27.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.147+0000 7f52d6a00640 1 -- start start 2024-09-21T07:49:27.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.149+0000 7f52d6a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52d00cfc50 0x7f52d0155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:27.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.149+0000 7f52d6a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52d0160250 0x7f52d00b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:27.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.149+0000 7f52d6a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52d0160660 0x7f52d00b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:27.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.149+0000 7f52d6a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f52d00cce00 con 0x7f52d0160660 2024-09-21T07:49:27.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.149+0000 7f52d6a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f52d00ccc80 con 0x7f52d00cfc50 2024-09-21T07:49:27.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.149+0000 7f52d6a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f52d00ccf80 con 0x7f52d0160250 2024-09-21T07:49:27.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.149+0000 7f52d6000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52d0160660 0x7f52d00b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:27.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.149+0000 7f52d6000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52d0160660 0x7f52d00b2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35798/0 (socket says 172.21.15.39:35798) 2024-09-21T07:49:27.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.149+0000 7f52d6000640 1 -- 172.21.15.39:0/2673594160 learned_addr learned my addr 172.21.15.39:0/2673594160 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:27.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.149+0000 7f52d5600640 1 --2- 172.21.15.39:0/2673594160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52d00cfc50 0x7f52d0155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:27.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.149+0000 7f52d4c00640 1 --2- 172.21.15.39:0/2673594160 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52d0160250 0x7f52d00b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:27.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.150+0000 7f52d6000640 1 -- 172.21.15.39:0/2673594160 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52d0160250 msgr2=0x7f52d00b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:27.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.150+0000 7f52d6000640 1 --2- 172.21.15.39:0/2673594160 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52d0160250 0x7f52d00b1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:27.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.150+0000 7f52d6000640 1 -- 172.21.15.39:0/2673594160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52d00cfc50 msgr2=0x7f52d0155d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:27.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.150+0000 7f52d6000640 1 --2- 172.21.15.39:0/2673594160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52d00cfc50 0x7f52d0155d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:27.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.150+0000 7f52d6000640 1 -- 172.21.15.39:0/2673594160 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d01592b0 con 0x7f52d0160660 2024-09-21T07:49:27.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.150+0000 7f52d6000640 1 --2- 172.21.15.39:0/2673594160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52d0160660 0x7f52d00b2120 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f52c800c6b0 tx=0x7f52c800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:27.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.150+0000 7f52cd600640 1 -- 172.21.15.39:0/2673594160 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c8003b70 con 0x7f52d0160660 2024-09-21T07:49:27.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.150+0000 7f52cd600640 1 -- 172.21.15.39:0/2673594160 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52c8003d10 con 0x7f52d0160660 2024-09-21T07:49:27.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.150+0000 7f52d6a00640 1 -- 172.21.15.39:0/2673594160 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52d0159590 con 0x7f52d0160660 2024-09-21T07:49:27.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.151+0000 7f52cd600640 1 -- 172.21.15.39:0/2673594160 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c8005330 con 0x7f52d0160660 2024-09-21T07:49:27.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.151+0000 7f52d6a00640 1 -- 172.21.15.39:0/2673594160 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52d0159b50 con 0x7f52d0160660 2024-09-21T07:49:27.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.152+0000 7f52d6a00640 1 -- 172.21.15.39:0/2673594160 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52d00cd430 con 0x7f52d0160660 2024-09-21T07:49:27.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.153+0000 7f52cd600640 1 -- 172.21.15.39:0/2673594160 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f52c80054d0 con 0x7f52d0160660 2024-09-21T07:49:27.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.154+0000 7f52cd600640 1 --2- 172.21.15.39:0/2673594160 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f52b0078b90 0x7f52b007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:27.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.154+0000 7f52cd600640 1 -- 172.21.15.39:0/2673594160 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(225..225 src has 1..225) ==== 6336+0+0 (secure 0 0 0) 0x7f52c809ad50 con 0x7f52d0160660 2024-09-21T07:49:27.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.155+0000 7f52d5600640 1 --2- 172.21.15.39:0/2673594160 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f52b0078b90 0x7f52b007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:27.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.156+0000 7f52d5600640 1 --2- 172.21.15.39:0/2673594160 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f52b0078b90 0x7f52b007b0a0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f52c4033000 tx=0x7f52c4004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:27.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.159+0000 7f52cd600640 1 -- 172.21.15.39:0/2673594160 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f52c8062eb0 con 0x7f52d0160660 2024-09-21T07:49:27.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.355+0000 7f52d6a00640 1 -- 172.21.15.39:0/2673594160 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) -- 0x7f52d0073190 con 0x7f52d0160660 2024-09-21T07:49:27.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.355+0000 7f52cd600640 1 -- 172.21.15.39:0/2673594160 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v225) ==== 107+0+11 (secure 0 0 0) 0x7f52c80627b0 con 0x7f52d0160660 2024-09-21T07:49:27.357 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-21T07:49:27.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.358+0000 7f52d6a00640 1 -- 172.21.15.39:0/2673594160 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f52b0078b90 msgr2=0x7f52b007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:27.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.358+0000 7f52d6a00640 1 --2- 172.21.15.39:0/2673594160 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f52b0078b90 0x7f52b007b0a0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f52c4033000 tx=0x7f52c4004730 comp rx=0 tx=0).stop 2024-09-21T07:49:27.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.358+0000 7f52d6a00640 1 -- 172.21.15.39:0/2673594160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52d0160660 msgr2=0x7f52d00b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:27.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.358+0000 7f52d6a00640 1 --2- 172.21.15.39:0/2673594160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52d0160660 0x7f52d00b2120 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f52c800c6b0 tx=0x7f52c800cb70 comp rx=0 tx=0).stop 2024-09-21T07:49:27.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.358+0000 7f52d6a00640 1 -- 172.21.15.39:0/2673594160 shutdown_connections 2024-09-21T07:49:27.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.358+0000 7f52d6a00640 1 --2- 172.21.15.39:0/2673594160 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f52b0078b90 0x7f52b007b0a0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:27.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.358+0000 7f52d6a00640 1 --2- 172.21.15.39:0/2673594160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52d0160660 0x7f52d00b2120 unknown :-1 s=CLOSED pgs=2190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:27.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.358+0000 7f52d6a00640 1 --2- 172.21.15.39:0/2673594160 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52d0160250 0x7f52d00b1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:27.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.358+0000 7f52d6a00640 1 --2- 172.21.15.39:0/2673594160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52d00cfc50 0x7f52d0155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:27.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.358+0000 7f52d6a00640 1 -- 172.21.15.39:0/2673594160 >> 172.21.15.39:0/2673594160 conn(0x7f52d006cfc0 msgr2=0x7f52d006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:27.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.358+0000 7f52d6a00640 1 -- 172.21.15.39:0/2673594160 shutdown_connections 2024-09-21T07:49:27.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.358+0000 7f52d6a00640 1 -- 172.21.15.39:0/2673594160 wait complete. 2024-09-21T07:49:27.384 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_snapclient_cache' 2024-09-21T07:49:27.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.608+0000 7fcc39600640 1 -- 172.21.15.39:0/3699173062 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc340ce2b0 msgr2=0x7fcc3406f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:27.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.608+0000 7fcc39600640 1 --2- 172.21.15.39:0/3699173062 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc340ce2b0 0x7fcc3406f6a0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c005fd0 tx=0x7fcc2c02a8b0 comp rx=0 tx=0).stop 2024-09-21T07:49:27.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.609+0000 7fcc39600640 1 -- 172.21.15.39:0/3699173062 shutdown_connections 2024-09-21T07:49:27.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.609+0000 7fcc39600640 1 --2- 172.21.15.39:0/3699173062 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc340ce2b0 0x7fcc3406f6a0 unknown :-1 s=CLOSED pgs=2192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:27.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.609+0000 7fcc39600640 1 -- 172.21.15.39:0/3699173062 >> 172.21.15.39:0/3699173062 conn(0x7fcc340b2460 msgr2=0x7fcc3406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:27.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.609+0000 7fcc39600640 1 -- 172.21.15.39:0/3699173062 shutdown_connections 2024-09-21T07:49:27.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.609+0000 7fcc39600640 1 -- 172.21.15.39:0/3699173062 wait complete. 2024-09-21T07:49:27.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.611+0000 7fcc39600640 1 Processor -- start 2024-09-21T07:49:27.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.611+0000 7fcc39600640 1 -- start start 2024-09-21T07:49:27.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.612+0000 7fcc39600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcc340b4c40 0x7fcc34155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:27.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.612+0000 7fcc39600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc340ce2b0 0x7fcc3415c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:27.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.613+0000 7fcc39600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc34155700 0x7fcc3415c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:27.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.613+0000 7fcc39600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcc34070da0 con 0x7fcc340ce2b0 2024-09-21T07:49:27.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.613+0000 7fcc39600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcc34070c20 con 0x7fcc34155700 2024-09-21T07:49:27.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.613+0000 7fcc39600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fcc3406fbe0 con 0x7fcc340b4c40 2024-09-21T07:49:27.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.613+0000 7fcc32000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc340ce2b0 0x7fcc3415c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:27.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.613+0000 7fcc32000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc340ce2b0 0x7fcc3415c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35810/0 (socket says 172.21.15.39:35810) 2024-09-21T07:49:27.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.613+0000 7fcc32000640 1 -- 172.21.15.39:0/741475286 learned_addr learned my addr 172.21.15.39:0/741475286 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:27.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.613+0000 7fcc33400640 1 --2- 172.21.15.39:0/741475286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc34155700 0x7fcc3415c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:27.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.613+0000 7fcc32a00640 1 --2- 172.21.15.39:0/741475286 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcc340b4c40 0x7fcc34155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:27.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.613+0000 7fcc32000640 1 -- 172.21.15.39:0/741475286 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcc340b4c40 msgr2=0x7fcc34155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:27.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.613+0000 7fcc32000640 1 --2- 172.21.15.39:0/741475286 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcc340b4c40 0x7fcc34155070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:27.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.613+0000 7fcc32000640 1 -- 172.21.15.39:0/741475286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc34155700 msgr2=0x7fcc3415c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:27.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.613+0000 7fcc32000640 1 --2- 172.21.15.39:0/741475286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc34155700 0x7fcc3415c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:27.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.613+0000 7fcc32000640 1 -- 172.21.15.39:0/741475286 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc34070700 con 0x7fcc340ce2b0 2024-09-21T07:49:27.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.614+0000 7fcc32000640 1 --2- 172.21.15.39:0/741475286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc340ce2b0 0x7fcc3415c310 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7fcc2000c3c0 tx=0x7fcc2000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:27.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.614+0000 7fcc2aa00640 1 -- 172.21.15.39:0/741475286 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc20017070 con 0x7fcc340ce2b0 2024-09-21T07:49:27.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.614+0000 7fcc2aa00640 1 -- 172.21.15.39:0/741475286 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc200035f0 con 0x7fcc340ce2b0 2024-09-21T07:49:27.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.614+0000 7fcc2aa00640 1 -- 172.21.15.39:0/741475286 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc20003a40 con 0x7fcc340ce2b0 2024-09-21T07:49:27.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.614+0000 7fcc39600640 1 -- 172.21.15.39:0/741475286 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc341e6960 con 0x7fcc340ce2b0 2024-09-21T07:49:27.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.614+0000 7fcc39600640 1 -- 172.21.15.39:0/741475286 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc341e6ee0 con 0x7fcc340ce2b0 2024-09-21T07:49:27.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.615+0000 7fcc39600640 1 -- 172.21.15.39:0/741475286 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc340b3130 con 0x7fcc340ce2b0 2024-09-21T07:49:27.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.617+0000 7fcc2aa00640 1 -- 172.21.15.39:0/741475286 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcc2001a070 con 0x7fcc340ce2b0 2024-09-21T07:49:27.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.618+0000 7fcc2aa00640 1 --2- 172.21.15.39:0/741475286 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcc14078bc0 0x7fcc1407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:27.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.618+0000 7fcc2aa00640 1 -- 172.21.15.39:0/741475286 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(225..225 src has 1..225) ==== 6336+0+0 (secure 0 0 0) 0x7fcc2009a220 con 0x7fcc340ce2b0 2024-09-21T07:49:27.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.618+0000 7fcc32a00640 1 --2- 172.21.15.39:0/741475286 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcc14078bc0 0x7fcc1407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:27.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.619+0000 7fcc32a00640 1 --2- 172.21.15.39:0/741475286 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcc14078bc0 0x7fcc1407b0d0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fcc24006fd0 tx=0x7fcc24009060 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:27.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.622+0000 7fcc2aa00640 1 -- 172.21.15.39:0/741475286 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcc340ce6c0 con 0x7fcc340ce2b0 2024-09-21T07:49:27.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:27.896+0000 7fcc39600640 1 -- 172.21.15.39:0/741475286 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_snapclient_cache"]} v 0) -- 0x7fcc340b4330 con 0x7fcc340ce2b0 2024-09-21T07:49:28.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.033+0000 7fcc2aa00640 1 -- 172.21.15.39:0/741475286 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_snapclient_cache"]}]=0 v795) ==== 145+0+0 (secure 0 0 0) 0x7fcc20061db0 con 0x7fcc340ce2b0 2024-09-21T07:49:28.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.035+0000 7fcc39600640 1 -- 172.21.15.39:0/741475286 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcc14078bc0 msgr2=0x7fcc1407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:28.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.035+0000 7fcc39600640 1 --2- 172.21.15.39:0/741475286 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcc14078bc0 0x7fcc1407b0d0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fcc24006fd0 tx=0x7fcc24009060 comp rx=0 tx=0).stop 2024-09-21T07:49:28.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.035+0000 7fcc39600640 1 -- 172.21.15.39:0/741475286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc340ce2b0 msgr2=0x7fcc3415c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:28.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.035+0000 7fcc39600640 1 --2- 172.21.15.39:0/741475286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc340ce2b0 0x7fcc3415c310 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7fcc2000c3c0 tx=0x7fcc2000c880 comp rx=0 tx=0).stop 2024-09-21T07:49:28.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.035+0000 7fcc39600640 1 -- 172.21.15.39:0/741475286 shutdown_connections 2024-09-21T07:49:28.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.035+0000 7fcc39600640 1 --2- 172.21.15.39:0/741475286 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcc14078bc0 0x7fcc1407b0d0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:28.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.035+0000 7fcc39600640 1 --2- 172.21.15.39:0/741475286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc34155700 0x7fcc3415c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:28.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.035+0000 7fcc39600640 1 --2- 172.21.15.39:0/741475286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc340ce2b0 0x7fcc3415c310 unknown :-1 s=CLOSED pgs=2193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:28.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.035+0000 7fcc39600640 1 --2- 172.21.15.39:0/741475286 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcc340b4c40 0x7fcc34155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:28.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.035+0000 7fcc39600640 1 -- 172.21.15.39:0/741475286 >> 172.21.15.39:0/741475286 conn(0x7fcc340b2460 msgr2=0x7fcc340b0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:28.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.035+0000 7fcc39600640 1 -- 172.21.15.39:0/741475286 shutdown_connections 2024-09-21T07:49:28.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.035+0000 7fcc39600640 1 -- 172.21.15.39:0/741475286 wait complete. 2024-09-21T07:49:28.063 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T07:49:28.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.288+0000 7fa46b000640 1 -- 172.21.15.39:0/1048462250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4640b5a90 msgr2=0x7fa4640d1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:28.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.288+0000 7fa46b000640 1 --2- 172.21.15.39:0/1048462250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4640b5a90 0x7fa4640d1ac0 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7fa458009820 tx=0x7fa45802a510 comp rx=0 tx=0).stop 2024-09-21T07:49:28.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.288+0000 7fa46b000640 1 -- 172.21.15.39:0/1048462250 shutdown_connections 2024-09-21T07:49:28.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.288+0000 7fa46b000640 1 --2- 172.21.15.39:0/1048462250 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4640b3b60 0x7fa4640b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:28.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.288+0000 7fa46b000640 1 --2- 172.21.15.39:0/1048462250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4640b5a90 0x7fa4640d1ac0 unknown :-1 s=CLOSED pgs=2195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:28.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.288+0000 7fa46b000640 1 -- 172.21.15.39:0/1048462250 >> 172.21.15.39:0/1048462250 conn(0x7fa46409d7b0 msgr2=0x7fa4640d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:28.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.288+0000 7fa46b000640 1 -- 172.21.15.39:0/1048462250 shutdown_connections 2024-09-21T07:49:28.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.288+0000 7fa46b000640 1 -- 172.21.15.39:0/1048462250 wait complete. 2024-09-21T07:49:28.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.290+0000 7fa46b000640 1 Processor -- start 2024-09-21T07:49:28.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.290+0000 7fa46b000640 1 -- start start 2024-09-21T07:49:28.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.292+0000 7fa46b000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4640b3b60 0x7fa4641f5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:28.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.292+0000 7fa46b000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4640b5a90 0x7fa4640c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:28.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.292+0000 7fa46b000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4640d0e70 0x7fa4640c84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:28.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.292+0000 7fa46b000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa4640b4630 con 0x7fa4640b3b60 2024-09-21T07:49:28.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.292+0000 7fa46b000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa4640b44b0 con 0x7fa4640d0e70 2024-09-21T07:49:28.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.292+0000 7fa46b000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa4640b47b0 con 0x7fa4640b5a90 2024-09-21T07:49:28.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.292+0000 7fa469c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4640b3b60 0x7fa4641f5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:28.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.292+0000 7fa469c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4640b3b60 0x7fa4641f5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35828/0 (socket says 172.21.15.39:35828) 2024-09-21T07:49:28.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.292+0000 7fa469c00640 1 -- 172.21.15.39:0/3526393576 learned_addr learned my addr 172.21.15.39:0/3526393576 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:28.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.293+0000 7fa46a600640 1 --2- 172.21.15.39:0/3526393576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4640d0e70 0x7fa4640c84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:28.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.293+0000 7fa469200640 1 --2- 172.21.15.39:0/3526393576 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4640b5a90 0x7fa4640c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:28.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.293+0000 7fa469c00640 1 -- 172.21.15.39:0/3526393576 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4640b5a90 msgr2=0x7fa4640c7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:28.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.293+0000 7fa469c00640 1 --2- 172.21.15.39:0/3526393576 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4640b5a90 0x7fa4640c7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:28.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.293+0000 7fa469c00640 1 -- 172.21.15.39:0/3526393576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4640d0e70 msgr2=0x7fa4640c84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:28.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.293+0000 7fa469c00640 1 --2- 172.21.15.39:0/3526393576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4640d0e70 0x7fa4640c84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:28.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.293+0000 7fa469c00640 1 -- 172.21.15.39:0/3526393576 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4641f5a20 con 0x7fa4640b3b60 2024-09-21T07:49:28.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.293+0000 7fa46a600640 1 --2- 172.21.15.39:0/3526393576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4640d0e70 0x7fa4640c84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:49:28.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.293+0000 7fa469200640 1 --2- 172.21.15.39:0/3526393576 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4640b5a90 0x7fa4640c7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:49:28.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.293+0000 7fa469c00640 1 --2- 172.21.15.39:0/3526393576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4640b3b60 0x7fa4641f5160 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7fa45802ab00 tx=0x7fa458004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:28.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.293+0000 7fa461600640 1 -- 172.21.15.39:0/3526393576 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45803f070 con 0x7fa4640b3b60 2024-09-21T07:49:28.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.293+0000 7fa461600640 1 -- 172.21.15.39:0/3526393576 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa458004e50 con 0x7fa4640b3b60 2024-09-21T07:49:28.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.293+0000 7fa46b000640 1 -- 172.21.15.39:0/3526393576 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4641f5d00 con 0x7fa4640b3b60 2024-09-21T07:49:28.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.293+0000 7fa461600640 1 -- 172.21.15.39:0/3526393576 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa458005c30 con 0x7fa4640b3b60 2024-09-21T07:49:28.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.293+0000 7fa46b000640 1 -- 172.21.15.39:0/3526393576 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4640b16a0 con 0x7fa4640b3b60 2024-09-21T07:49:28.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.294+0000 7fa46b000640 1 -- 172.21.15.39:0/3526393576 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4640b4c60 con 0x7fa4640b3b60 2024-09-21T07:49:28.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.296+0000 7fa461600640 1 -- 172.21.15.39:0/3526393576 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa458042070 con 0x7fa4640b3b60 2024-09-21T07:49:28.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.297+0000 7fa461600640 1 --2- 172.21.15.39:0/3526393576 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa444078bc0 0x7fa44407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:28.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.297+0000 7fa461600640 1 -- 172.21.15.39:0/3526393576 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(225..225 src has 1..225) ==== 6336+0+0 (secure 0 0 0) 0x7fa4580bc300 con 0x7fa4640b3b60 2024-09-21T07:49:28.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.298+0000 7fa469200640 1 --2- 172.21.15.39:0/3526393576 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa444078bc0 0x7fa44407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:28.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.298+0000 7fa469200640 1 --2- 172.21.15.39:0/3526393576 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa444078bc0 0x7fa44407b0d0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fa46406e010 tx=0x7fa450009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:28.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.302+0000 7fa461600640 1 -- 172.21.15.39:0/3526393576 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa458084450 con 0x7fa4640b3b60 2024-09-21T07:49:28.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.497+0000 7fa46b000640 1 -- 172.21.15.39:0/3526393576 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fa4640b80f0 con 0x7fa4640b3b60 2024-09-21T07:49:28.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.498+0000 7fa461600640 1 -- 172.21.15.39:0/3526393576 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v54) ==== 59+0+2294 (secure 0 0 0) 0x7fa458083d50 con 0x7fa4640b3b60 2024-09-21T07:49:28.500 INFO:teuthology.orchestra.run.smithi039.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T07:49:28.500 INFO:teuthology.orchestra.run.smithi039.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T07:49:28.500 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced client_mount_timeout 30 2024-09-21T07:49:28.500 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_client 20/20 2024-09-21T07:49:28.500 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_mgr 20/20 2024-09-21T07:49:28.500 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_ms 1/1 2024-09-21T07:49:28.500 INFO:teuthology.orchestra.run.smithi039.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T07:49:28.500 INFO:teuthology.orchestra.run.smithi039.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T07:49:28.500 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3421.881759 2024-09-21T07:49:28.500 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3520.252978 2024-09-21T07:49:28.501 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3511.183316 2024-09-21T07:49:28.501 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3829.631847 2024-09-21T07:49:28.501 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 5474.545872 2024-09-21T07:49:28.501 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4013.628559 2024-09-21T07:49:28.501 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4230.187288 2024-09-21T07:49:28.501 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4148.983107 2024-09-21T07:49:28.501 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_mds 20/20 2024-09-21T07:49:28.501 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T07:49:28.501 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_ms 2/2 2024-09-21T07:49:28.501 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_debug_frag true 2024-09-21T07:49:28.501 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_debug_scatterstat true 2024-09-21T07:49:28.501 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T07:49:28.501 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_verify_scatter true 2024-09-21T07:49:28.501 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T07:49:28.502 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T07:49:28.502 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T07:49:28.502 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced client_mount_timeout 600 2024-09-21T07:49:28.502 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced debug_client 20/20 2024-09-21T07:49:28.502 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced debug_ms 1/1 2024-09-21T07:49:28.502 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T07:49:28.502 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T07:49:28.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.501+0000 7fa46b000640 1 -- 172.21.15.39:0/3526393576 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa444078bc0 msgr2=0x7fa44407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:28.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.501+0000 7fa46b000640 1 --2- 172.21.15.39:0/3526393576 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa444078bc0 0x7fa44407b0d0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fa46406e010 tx=0x7fa450009040 comp rx=0 tx=0).stop 2024-09-21T07:49:28.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.501+0000 7fa46b000640 1 -- 172.21.15.39:0/3526393576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4640b3b60 msgr2=0x7fa4641f5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:28.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.501+0000 7fa46b000640 1 --2- 172.21.15.39:0/3526393576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4640b3b60 0x7fa4641f5160 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7fa45802ab00 tx=0x7fa458004a50 comp rx=0 tx=0).stop 2024-09-21T07:49:28.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.501+0000 7fa46b000640 1 -- 172.21.15.39:0/3526393576 shutdown_connections 2024-09-21T07:49:28.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.501+0000 7fa46b000640 1 --2- 172.21.15.39:0/3526393576 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa444078bc0 0x7fa44407b0d0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:28.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.501+0000 7fa46b000640 1 --2- 172.21.15.39:0/3526393576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4640d0e70 0x7fa4640c84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:28.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.501+0000 7fa46b000640 1 --2- 172.21.15.39:0/3526393576 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4640b5a90 0x7fa4640c7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:28.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.501+0000 7fa46b000640 1 --2- 172.21.15.39:0/3526393576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4640b3b60 0x7fa4641f5160 unknown :-1 s=CLOSED pgs=2196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:28.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.501+0000 7fa46b000640 1 -- 172.21.15.39:0/3526393576 >> 172.21.15.39:0/3526393576 conn(0x7fa46409d7b0 msgr2=0x7fa46406e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:28.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.501+0000 7fa46b000640 1 -- 172.21.15.39:0/3526393576 shutdown_connections 2024-09-21T07:49:28.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.501+0000 7fa46b000640 1 -- 172.21.15.39:0/3526393576 wait complete. 2024-09-21T07:49:28.525 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T07:49:28.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.754+0000 7fecf7000640 1 -- 172.21.15.39:0/3811390325 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fecf00d0060 msgr2=0x7fecf00d0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:28.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.754+0000 7fecf7000640 1 --2- 172.21.15.39:0/3811390325 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fecf00d0060 0x7fecf00d0470 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fecdc009830 tx=0x7fecdc02a4a0 comp rx=0 tx=0).stop 2024-09-21T07:49:28.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.754+0000 7fecf7000640 1 -- 172.21.15.39:0/3811390325 shutdown_connections 2024-09-21T07:49:28.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.754+0000 7fecf7000640 1 --2- 172.21.15.39:0/3811390325 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fecf00d0060 0x7fecf00d0470 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:28.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.754+0000 7fecf7000640 1 --2- 172.21.15.39:0/3811390325 >> v2:172.21.15.148:3300/0 conn(0x7fecf00ce690 0x7fecf00b2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:28.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.754+0000 7fecf7000640 1 -- 172.21.15.39:0/3811390325 >> 172.21.15.39:0/3811390325 conn(0x7fecf009b5f0 msgr2=0x7fecf008c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:28.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.754+0000 7fecf7000640 1 -- 172.21.15.39:0/3811390325 shutdown_connections 2024-09-21T07:49:28.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.754+0000 7fecf7000640 1 -- 172.21.15.39:0/3811390325 wait complete. 2024-09-21T07:49:28.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.756+0000 7fecf7000640 1 Processor -- start 2024-09-21T07:49:28.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.756+0000 7fecf7000640 1 -- start start 2024-09-21T07:49:28.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.758+0000 7fecf7000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecf00ce690 0x7fecf01f6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:28.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.758+0000 7fecf7000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fecf00d0060 0x7fecf0070f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:28.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.758+0000 7fecf7000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecf00d2030 0x7fecf0071470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:28.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.758+0000 7fecf7000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fecf00cd3c0 con 0x7fecf00ce690 2024-09-21T07:49:28.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.758+0000 7fecf7000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fecf00cd240 con 0x7fecf00d2030 2024-09-21T07:49:28.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.758+0000 7fecf7000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fecf00cd540 con 0x7fecf00d0060 2024-09-21T07:49:28.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.758+0000 7fecf5c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecf00ce690 0x7fecf01f6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:28.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.758+0000 7fecf5c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecf00ce690 0x7fecf01f6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35836/0 (socket says 172.21.15.39:35836) 2024-09-21T07:49:28.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.758+0000 7fecf5c00640 1 -- 172.21.15.39:0/2573925739 learned_addr learned my addr 172.21.15.39:0/2573925739 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:28.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.758+0000 7fecf5200640 1 --2- 172.21.15.39:0/2573925739 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fecf00d0060 0x7fecf0070f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:28.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.758+0000 7fecf6600640 1 --2- 172.21.15.39:0/2573925739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecf00d2030 0x7fecf0071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:28.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.759+0000 7fecf5c00640 1 -- 172.21.15.39:0/2573925739 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fecf00d0060 msgr2=0x7fecf0070f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:28.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.759+0000 7fecf5c00640 1 --2- 172.21.15.39:0/2573925739 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fecf00d0060 0x7fecf0070f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:28.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.759+0000 7fecf5c00640 1 -- 172.21.15.39:0/2573925739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecf00d2030 msgr2=0x7fecf0071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:28.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.759+0000 7fecf5c00640 1 --2- 172.21.15.39:0/2573925739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecf00d2030 0x7fecf0071470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:28.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.759+0000 7fecf5c00640 1 -- 172.21.15.39:0/2573925739 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf01f7520 con 0x7fecf00ce690 2024-09-21T07:49:28.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.759+0000 7fecf6600640 1 --2- 172.21.15.39:0/2573925739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecf00d2030 0x7fecf0071470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:28.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.759+0000 7fecf5c00640 1 --2- 172.21.15.39:0/2573925739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecf00ce690 0x7fecf01f6fe0 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7fece400e7c0 tx=0x7fece400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:28.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.759+0000 7feced600640 1 -- 172.21.15.39:0/2573925739 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece400ccc0 con 0x7fecf00ce690 2024-09-21T07:49:28.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.759+0000 7feced600640 1 -- 172.21.15.39:0/2573925739 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fece4010430 con 0x7fecf00ce690 2024-09-21T07:49:28.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.759+0000 7fecf7000640 1 -- 172.21.15.39:0/2573925739 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecf01f3360 con 0x7fecf00ce690 2024-09-21T07:49:28.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.759+0000 7feced600640 1 -- 172.21.15.39:0/2573925739 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece4010840 con 0x7fecf00ce690 2024-09-21T07:49:28.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.760+0000 7fecf7000640 1 -- 172.21.15.39:0/2573925739 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecf01f3920 con 0x7fecf00ce690 2024-09-21T07:49:28.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.761+0000 7fecf7000640 1 -- 172.21.15.39:0/2573925739 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecf00cda60 con 0x7fecf00ce690 2024-09-21T07:49:28.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.762+0000 7feced600640 1 -- 172.21.15.39:0/2573925739 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fece40109e0 con 0x7fecf00ce690 2024-09-21T07:49:28.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.763+0000 7feced600640 1 --2- 172.21.15.39:0/2573925739 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fecd0078b90 0x7fecd007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:28.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.763+0000 7feced600640 1 -- 172.21.15.39:0/2573925739 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(225..225 src has 1..225) ==== 6336+0+0 (secure 0 0 0) 0x7fece409b980 con 0x7fecf00ce690 2024-09-21T07:49:28.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.763+0000 7fecf5200640 1 --2- 172.21.15.39:0/2573925739 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fecd0078b90 0x7fecd007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:28.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.764+0000 7fecf5200640 1 --2- 172.21.15.39:0/2573925739 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fecd0078b90 0x7fecd007b0a0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fecdc034000 tx=0x7fecdc004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:28.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.768+0000 7feced600640 1 -- 172.21.15.39:0/2573925739 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fecf01f7520 con 0x7fecf00ce690 2024-09-21T07:49:28.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.962+0000 7fecf7000640 1 -- 172.21.15.39:0/2573925739 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fecf0072a00 con 0x7fecf00ce690 2024-09-21T07:49:28.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.966+0000 7feced600640 1 -- 172.21.15.39:0/2573925739 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v55)=0 v55) ==== 122+0+0 (secure 0 0 0) 0x7fece4063520 con 0x7fecf00ce690 2024-09-21T07:49:28.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.969+0000 7fecf7000640 1 -- 172.21.15.39:0/2573925739 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fecd0078b90 msgr2=0x7fecd007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:28.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.969+0000 7fecf7000640 1 --2- 172.21.15.39:0/2573925739 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fecd0078b90 0x7fecd007b0a0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fecdc034000 tx=0x7fecdc004730 comp rx=0 tx=0).stop 2024-09-21T07:49:28.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.969+0000 7fecf7000640 1 -- 172.21.15.39:0/2573925739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecf00ce690 msgr2=0x7fecf01f6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:28.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.969+0000 7fecf7000640 1 --2- 172.21.15.39:0/2573925739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecf00ce690 0x7fecf01f6fe0 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7fece400e7c0 tx=0x7fece400ec80 comp rx=0 tx=0).stop 2024-09-21T07:49:28.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.969+0000 7fecf7000640 1 -- 172.21.15.39:0/2573925739 shutdown_connections 2024-09-21T07:49:28.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.969+0000 7fecf7000640 1 --2- 172.21.15.39:0/2573925739 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fecd0078b90 0x7fecd007b0a0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:28.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.969+0000 7fecf7000640 1 --2- 172.21.15.39:0/2573925739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecf00d2030 0x7fecf0071470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:28.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.969+0000 7fecf7000640 1 --2- 172.21.15.39:0/2573925739 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fecf00d0060 0x7fecf0070f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:28.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.969+0000 7fecf7000640 1 --2- 172.21.15.39:0/2573925739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fecf00ce690 0x7fecf01f6fe0 unknown :-1 s=CLOSED pgs=2198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:28.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.969+0000 7fecf7000640 1 -- 172.21.15.39:0/2573925739 >> 172.21.15.39:0/2573925739 conn(0x7fecf009b5f0 msgr2=0x7fecf015f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:28.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.969+0000 7fecf7000640 1 -- 172.21.15.39:0/2573925739 shutdown_connections 2024-09-21T07:49:28.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:28.969+0000 7fecf7000640 1 -- 172.21.15.39:0/2573925739 wait complete. 2024-09-21T07:49:28.996 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:49:28.996 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:49:29.083 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:49:29.178 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:49:29.178 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:49:29.178 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:49:29.213 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:49:29.214 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:49:29.214 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:49:29.251 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:49:29.252 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:49:29.252 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:49:29.289 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:49:29.289 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:49:29.289 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:49:29.328 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:49:29.328 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T07:49:29.328 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:49:29.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.553+0000 7f97e1600640 1 -- 172.21.15.39:0/341738212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97dc0d0e60 msgr2=0x7f97dc0d1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:29.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.553+0000 7f97e1600640 1 --2- 172.21.15.39:0/341738212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97dc0d0e60 0x7f97dc0d1270 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f97dc0d1d10 tx=0x7f97d402a8b0 comp rx=0 tx=0).stop 2024-09-21T07:49:29.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.553+0000 7f97e1600640 1 -- 172.21.15.39:0/341738212 shutdown_connections 2024-09-21T07:49:29.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.553+0000 7f97e1600640 1 --2- 172.21.15.39:0/341738212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97dc0d0e60 0x7f97dc0d1270 unknown :-1 s=CLOSED pgs=2200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:29.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.553+0000 7f97e1600640 1 --2- 172.21.15.39:0/341738212 >> v2:172.21.15.148:3300/0 conn(0x7f97dc0b96a0 0x7f97dc0b59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:29.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.553+0000 7f97e1600640 1 -- 172.21.15.39:0/341738212 >> 172.21.15.39:0/341738212 conn(0x7f97dc09d7d0 msgr2=0x7f97dc0d4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:29.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.553+0000 7f97e1600640 1 -- 172.21.15.39:0/341738212 shutdown_connections 2024-09-21T07:49:29.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.553+0000 7f97e1600640 1 -- 172.21.15.39:0/341738212 wait complete. 2024-09-21T07:49:29.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.555+0000 7f97e1600640 1 Processor -- start 2024-09-21T07:49:29.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.555+0000 7f97e1600640 1 -- start start 2024-09-21T07:49:29.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.557+0000 7f97e1600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97dc0b96a0 0x7f97dc1f5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:29.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.557+0000 7f97e1600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97dc0d0e60 0x7f97dc1f5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:29.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.557+0000 7f97e1600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f97dc0d5060 0x7f97dc1f8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:29.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.557+0000 7f97e1600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc0d2a00 con 0x7f97dc0b96a0 2024-09-21T07:49:29.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.557+0000 7f97e1600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc0d2880 con 0x7f97dc0d0e60 2024-09-21T07:49:29.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.557+0000 7f97e1600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f97dc0d2b80 con 0x7f97dc0d5060 2024-09-21T07:49:29.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.557+0000 7f97dbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97dc0b96a0 0x7f97dc1f5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:29.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.557+0000 7f97dbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97dc0b96a0 0x7f97dc1f5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35846/0 (socket says 172.21.15.39:35846) 2024-09-21T07:49:29.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.557+0000 7f97dbe00640 1 -- 172.21.15.39:0/2161631193 learned_addr learned my addr 172.21.15.39:0/2161631193 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:29.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.557+0000 7f97d3e00640 1 --2- 172.21.15.39:0/2161631193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97dc0d0e60 0x7f97dc1f5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:29.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.557+0000 7f97e0c00640 1 --2- 172.21.15.39:0/2161631193 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f97dc0d5060 0x7f97dc1f8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:29.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.557+0000 7f97dbe00640 1 -- 172.21.15.39:0/2161631193 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f97dc0d5060 msgr2=0x7f97dc1f8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:29.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.557+0000 7f97dbe00640 1 --2- 172.21.15.39:0/2161631193 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f97dc0d5060 0x7f97dc1f8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:29.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.557+0000 7f97dbe00640 1 -- 172.21.15.39:0/2161631193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97dc0d0e60 msgr2=0x7f97dc1f5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:29.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.557+0000 7f97dbe00640 1 --2- 172.21.15.39:0/2161631193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97dc0d0e60 0x7f97dc1f5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:29.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.557+0000 7f97dbe00640 1 -- 172.21.15.39:0/2161631193 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97dc146760 con 0x7f97dc0b96a0 2024-09-21T07:49:29.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.557+0000 7f97d3e00640 1 --2- 172.21.15.39:0/2161631193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97dc0d0e60 0x7f97dc1f5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:29.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.558+0000 7f97dbe00640 1 --2- 172.21.15.39:0/2161631193 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97dc0b96a0 0x7f97dc1f5960 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7f97cc00c410 tx=0x7f97cc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:29.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.558+0000 7f97d8c00640 1 -- 172.21.15.39:0/2161631193 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc017070 con 0x7f97dc0b96a0 2024-09-21T07:49:29.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.558+0000 7f97d8c00640 1 -- 172.21.15.39:0/2161631193 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f97cc0035f0 con 0x7f97dc0b96a0 2024-09-21T07:49:29.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.558+0000 7f97e1600640 1 -- 172.21.15.39:0/2161631193 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97dc146a40 con 0x7f97dc0b96a0 2024-09-21T07:49:29.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.558+0000 7f97d8c00640 1 -- 172.21.15.39:0/2161631193 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc003a40 con 0x7f97dc0b96a0 2024-09-21T07:49:29.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.558+0000 7f97e1600640 1 -- 172.21.15.39:0/2161631193 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97dc147000 con 0x7f97dc0b96a0 2024-09-21T07:49:29.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.559+0000 7f97e1600640 1 -- 172.21.15.39:0/2161631193 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97dc0d3050 con 0x7f97dc0b96a0 2024-09-21T07:49:29.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.561+0000 7f97d8c00640 1 -- 172.21.15.39:0/2161631193 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f97cc01a070 con 0x7f97dc0b96a0 2024-09-21T07:49:29.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.561+0000 7f97d8c00640 1 --2- 172.21.15.39:0/2161631193 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f97b8078b90 0x7f97b807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:29.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.562+0000 7f97d8c00640 1 -- 172.21.15.39:0/2161631193 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(225..225 src has 1..225) ==== 6336+0+0 (secure 0 0 0) 0x7f97cc09a240 con 0x7f97dc0b96a0 2024-09-21T07:49:29.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.562+0000 7f97d3e00640 1 --2- 172.21.15.39:0/2161631193 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f97b8078b90 0x7f97b807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:29.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.563+0000 7f97d3e00640 1 --2- 172.21.15.39:0/2161631193 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f97b8078b90 0x7f97b807b0a0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f97dc15b8c0 tx=0x7f97c4005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:29.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.566+0000 7f97d8c00640 1 -- 172.21.15.39:0/2161631193 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f97cc062420 con 0x7f97dc0b96a0 2024-09-21T07:49:29.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.803+0000 7f97e1600640 1 -- 172.21.15.39:0/2161631193 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97dc159640 con 0x7f97dc0b96a0 2024-09-21T07:49:29.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.804+0000 7f97d8c00640 1 -- 172.21.15.39:0/2161631193 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 237 v237) ==== 95+0+3920 (secure 0 0 0) 0x7f97cc061d20 con 0x7f97dc0b96a0 2024-09-21T07:49:29.805 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:49:29.805 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":237,"btime":"2024-09-21T07:49:20:551375+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8249,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/550771731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":550771731},{"type":"v1","addr":"172.21.15.148:6837","nonce":550771731}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":224},{"gid":9385,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1307677219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":1307677219},{"type":"v1","addr":"172.21.15.39:6835","nonce":1307677219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":224},{"gid":9616,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3153321617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3153321617},{"type":"v1","addr":"172.21.15.148:6836","nonce":3153321617}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":237},{"gid":10521,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2427004065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2427004065},{"type":"v1","addr":"172.21.15.39:6834","nonce":2427004065}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":237}],"filesystems":[]} 2024-09-21T07:49:29.806 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 237 2024-09-21T07:49:29.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.806+0000 7f97e1600640 1 -- 172.21.15.39:0/2161631193 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f97b8078b90 msgr2=0x7f97b807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:29.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.806+0000 7f97e1600640 1 --2- 172.21.15.39:0/2161631193 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f97b8078b90 0x7f97b807b0a0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f97dc15b8c0 tx=0x7f97c4005f90 comp rx=0 tx=0).stop 2024-09-21T07:49:29.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.806+0000 7f97e1600640 1 -- 172.21.15.39:0/2161631193 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97dc0b96a0 msgr2=0x7f97dc1f5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:29.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.806+0000 7f97e1600640 1 --2- 172.21.15.39:0/2161631193 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97dc0b96a0 0x7f97dc1f5960 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7f97cc00c410 tx=0x7f97cc00c8d0 comp rx=0 tx=0).stop 2024-09-21T07:49:29.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.806+0000 7f97e1600640 1 -- 172.21.15.39:0/2161631193 shutdown_connections 2024-09-21T07:49:29.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.806+0000 7f97e1600640 1 --2- 172.21.15.39:0/2161631193 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f97b8078b90 0x7f97b807b0a0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:29.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.806+0000 7f97e1600640 1 --2- 172.21.15.39:0/2161631193 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f97dc0d5060 0x7f97dc1f8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:29.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.806+0000 7f97e1600640 1 --2- 172.21.15.39:0/2161631193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97dc0d0e60 0x7f97dc1f5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:29.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.806+0000 7f97e1600640 1 --2- 172.21.15.39:0/2161631193 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97dc0b96a0 0x7f97dc1f5960 unknown :-1 s=CLOSED pgs=2201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:29.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.807+0000 7f97e1600640 1 -- 172.21.15.39:0/2161631193 >> 172.21.15.39:0/2161631193 conn(0x7f97dc09d7d0 msgr2=0x7f97dc0c9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:29.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.807+0000 7f97e1600640 1 -- 172.21.15.39:0/2161631193 shutdown_connections 2024-09-21T07:49:29.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:29.807+0000 7f97e1600640 1 -- 172.21.15.39:0/2161631193 wait complete. 2024-09-21T07:49:29.834 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T07:49:29.834 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T07:49:29.835 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T07:49:29.835 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:49:29.835 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T07:49:29.835 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T07:49:29.835 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T07:49:29.835 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:49:29.835 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T07:49:29.836 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T07:49:29.836 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T07:49:29.836 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:49:29.836 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T07:49:29.836 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T07:49:29.836 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T07:49:29.836 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:49:29.859 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T07:49:29.860 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T07:49:29.860 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T07:49:29.860 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T07:49:29.861 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T07:49:29.862 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T07:49:29.862 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T07:49:29.862 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T07:49:29.904 INFO:tasks.ceph.mds.c:Started 2024-09-21T07:49:29.904 DEBUG:teuthology.parallel:result is None 2024-09-21T07:49:29.906 INFO:tasks.ceph.mds.a:Started 2024-09-21T07:49:29.906 INFO:tasks.ceph.mds.b:Started 2024-09-21T07:49:29.907 DEBUG:teuthology.parallel:result is None 2024-09-21T07:49:29.907 DEBUG:teuthology.parallel:result is None 2024-09-21T07:49:29.909 INFO:tasks.ceph.mds.d:Started 2024-09-21T07:49:29.909 DEBUG:teuthology.parallel:result is None 2024-09-21T07:49:29.909 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T07:49:30.107 INFO:tasks.ceph.mds.d.smithi148.stdout:starting mds.d at 2024-09-21T07:49:30.108 INFO:tasks.ceph.mds.b.smithi148.stdout:starting mds.b at 2024-09-21T07:49:30.108 INFO:tasks.ceph.mds.c.smithi039.stdout:starting mds.c at 2024-09-21T07:49:30.108 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-21T07:49:30.109 INFO:tasks.ceph.mds.d.smithi148.stderr:2024-09-21T07:49:30.107+0000 7fcd68600640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-09-21T07:49:30.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.143+0000 7f61fbe00640 1 -- 172.21.15.39:0/3536547685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61fc073f90 msgr2=0x7f61fc0d04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:30.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.143+0000 7f61fbe00640 1 --2- 172.21.15.39:0/3536547685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61fc073f90 0x7f61fc0d04b0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f61e8005fd0 tx=0x7f61e802a540 comp rx=0 tx=0).stop 2024-09-21T07:49:30.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.143+0000 7f61fbe00640 1 -- 172.21.15.39:0/3536547685 shutdown_connections 2024-09-21T07:49:30.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.143+0000 7f61fbe00640 1 --2- 172.21.15.39:0/3536547685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61fc073f90 0x7f61fc0d04b0 unknown :-1 s=CLOSED pgs=2207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:30.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.143+0000 7f61fbe00640 1 --2- 172.21.15.39:0/3536547685 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f61fc0b3b50 0x7f61fc15c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:30.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.143+0000 7f61fbe00640 1 -- 172.21.15.39:0/3536547685 >> 172.21.15.39:0/3536547685 conn(0x7f61fc06cfc0 msgr2=0x7f61fc06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:30.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.143+0000 7f61fbe00640 1 -- 172.21.15.39:0/3536547685 shutdown_connections 2024-09-21T07:49:30.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.143+0000 7f61fbe00640 1 -- 172.21.15.39:0/3536547685 wait complete. 2024-09-21T07:49:30.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.145+0000 7f61fbe00640 1 Processor -- start 2024-09-21T07:49:30.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.146+0000 7f61fbe00640 1 -- start start 2024-09-21T07:49:30.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.147+0000 7f61fbe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61fc073f90 0x7f61fc1f3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:30.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.147+0000 7f61fbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61fc0743a0 0x7f61fc1f6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:30.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.147+0000 7f61fbe00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f61fc0b3b50 0x7f61fc0c6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:30.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.147+0000 7f61fbe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f61fc155d30 con 0x7f61fc0743a0 2024-09-21T07:49:30.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.147+0000 7f61fbe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f61fc155bb0 con 0x7f61fc073f90 2024-09-21T07:49:30.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.147+0000 7f61fbe00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f61fc155eb0 con 0x7f61fc0b3b50 2024-09-21T07:49:30.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.147+0000 7f61fa000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61fc0743a0 0x7f61fc1f6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:30.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.147+0000 7f61fa000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61fc0743a0 0x7f61fc1f6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35910/0 (socket says 172.21.15.39:35910) 2024-09-21T07:49:30.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.147+0000 7f61fa000640 1 -- 172.21.15.39:0/3588155988 learned_addr learned my addr 172.21.15.39:0/3588155988 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:30.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.148+0000 7f61faa00640 1 --2- 172.21.15.39:0/3588155988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61fc073f90 0x7f61fc1f3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:30.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.148+0000 7f61fb400640 1 --2- 172.21.15.39:0/3588155988 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f61fc0b3b50 0x7f61fc0c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:30.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.148+0000 7f61fa000640 1 -- 172.21.15.39:0/3588155988 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f61fc0b3b50 msgr2=0x7f61fc0c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:30.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.148+0000 7f61fa000640 1 --2- 172.21.15.39:0/3588155988 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f61fc0b3b50 0x7f61fc0c6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:30.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.148+0000 7f61fa000640 1 -- 172.21.15.39:0/3588155988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61fc073f90 msgr2=0x7f61fc1f3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:30.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.148+0000 7f61fa000640 1 --2- 172.21.15.39:0/3588155988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61fc073f90 0x7f61fc1f3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:30.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.148+0000 7f61fa000640 1 -- 172.21.15.39:0/3588155988 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61fc15ea60 con 0x7f61fc0743a0 2024-09-21T07:49:30.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.148+0000 7f61fb400640 1 --2- 172.21.15.39:0/3588155988 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f61fc0b3b50 0x7f61fc0c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:49:30.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.148+0000 7f61faa00640 1 --2- 172.21.15.39:0/3588155988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61fc073f90 0x7f61fc1f3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:49:30.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.148+0000 7f61fa000640 1 --2- 172.21.15.39:0/3588155988 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61fc0743a0 0x7f61fc1f6260 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f61e8006010 tx=0x7f61e80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:30.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.148+0000 7f61f2a00640 1 -- 172.21.15.39:0/3588155988 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e803e030 con 0x7f61fc0743a0 2024-09-21T07:49:30.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.148+0000 7f61f2a00640 1 -- 172.21.15.39:0/3588155988 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61e8003d90 con 0x7f61fc0743a0 2024-09-21T07:49:30.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.148+0000 7f61f2a00640 1 -- 172.21.15.39:0/3588155988 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e8035040 con 0x7f61fc0743a0 2024-09-21T07:49:30.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.149+0000 7f61fbe00640 1 -- 172.21.15.39:0/3588155988 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61fc0c5590 con 0x7f61fc0743a0 2024-09-21T07:49:30.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.149+0000 7f61fbe00640 1 -- 172.21.15.39:0/3588155988 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61fc0c5b50 con 0x7f61fc0743a0 2024-09-21T07:49:30.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.150+0000 7f61fbe00640 1 -- 172.21.15.39:0/3588155988 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61fc0d0b60 con 0x7f61fc0743a0 2024-09-21T07:49:30.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.151+0000 7f61f2a00640 1 -- 172.21.15.39:0/3588155988 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f61e8041030 con 0x7f61fc0743a0 2024-09-21T07:49:30.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.152+0000 7f61f2a00640 1 --2- 172.21.15.39:0/3588155988 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f61dc078b90 0x7f61dc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:30.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.152+0000 7f61f2a00640 1 -- 172.21.15.39:0/3588155988 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(225..225 src has 1..225) ==== 6336+0+0 (secure 0 0 0) 0x7f61e80bbb10 con 0x7f61fc0743a0 2024-09-21T07:49:30.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.153+0000 7f61faa00640 1 --2- 172.21.15.39:0/3588155988 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f61dc078b90 0x7f61dc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:30.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.153+0000 7f61faa00640 1 --2- 172.21.15.39:0/3588155988 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f61dc078b90 0x7f61dc07b0a0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f61ec009ab0 tx=0x7f61ec005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:30.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.156+0000 7f61f2a00640 1 -- 172.21.15.39:0/3588155988 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f61e8083d70 con 0x7f61fc0743a0 2024-09-21T07:49:30.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:30.363+0000 7f61fbe00640 1 -- 172.21.15.39:0/3588155988 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f61fc0b4780 con 0x7f61fc0743a0 2024-09-21T07:49:31.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.072+0000 7f61f2a00640 1 -- 172.21.15.39:0/3588155988 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v227) ==== 97+0+0 (secure 0 0 0) 0x7f61e8083670 con 0x7f61fc0743a0 2024-09-21T07:49:31.073 INFO:teuthology.orchestra.run.smithi039.stderr: removed all blocklist entries 2024-09-21T07:49:31.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.075+0000 7f61fbe00640 1 -- 172.21.15.39:0/3588155988 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f61dc078b90 msgr2=0x7f61dc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:31.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.075+0000 7f61fbe00640 1 --2- 172.21.15.39:0/3588155988 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f61dc078b90 0x7f61dc07b0a0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f61ec009ab0 tx=0x7f61ec005f90 comp rx=0 tx=0).stop 2024-09-21T07:49:31.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.075+0000 7f61fbe00640 1 -- 172.21.15.39:0/3588155988 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61fc0743a0 msgr2=0x7f61fc1f6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:31.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.075+0000 7f61fbe00640 1 --2- 172.21.15.39:0/3588155988 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61fc0743a0 0x7f61fc1f6260 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f61e8006010 tx=0x7f61e80038a0 comp rx=0 tx=0).stop 2024-09-21T07:49:31.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.075+0000 7f61fbe00640 1 -- 172.21.15.39:0/3588155988 shutdown_connections 2024-09-21T07:49:31.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.075+0000 7f61fbe00640 1 --2- 172.21.15.39:0/3588155988 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f61dc078b90 0x7f61dc07b0a0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:31.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.075+0000 7f61fbe00640 1 --2- 172.21.15.39:0/3588155988 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f61fc0b3b50 0x7f61fc0c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:31.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.075+0000 7f61fbe00640 1 --2- 172.21.15.39:0/3588155988 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61fc0743a0 0x7f61fc1f6260 unknown :-1 s=CLOSED pgs=2208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:31.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.075+0000 7f61fbe00640 1 --2- 172.21.15.39:0/3588155988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f61fc073f90 0x7f61fc1f3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:31.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.075+0000 7f61fbe00640 1 -- 172.21.15.39:0/3588155988 >> 172.21.15.39:0/3588155988 conn(0x7f61fc06cfc0 msgr2=0x7f61fc06f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:31.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.075+0000 7f61fbe00640 1 -- 172.21.15.39:0/3588155988 shutdown_connections 2024-09-21T07:49:31.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.075+0000 7f61fbe00640 1 -- 172.21.15.39:0/3588155988 wait complete. 2024-09-21T07:49:31.106 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T07:49:31.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.333+0000 7fbb1de00640 1 -- 172.21.15.39:0/1621185652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb1815fc20 msgr2=0x7fbb180d06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:31.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.333+0000 7fbb1de00640 1 --2- 172.21.15.39:0/1621185652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb1815fc20 0x7fbb180d06b0 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7fbb08009830 tx=0x7fbb0802a510 comp rx=0 tx=0).stop 2024-09-21T07:49:31.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.333+0000 7fbb1de00640 1 -- 172.21.15.39:0/1621185652 shutdown_connections 2024-09-21T07:49:31.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.333+0000 7fbb1de00640 1 --2- 172.21.15.39:0/1621185652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb1815fc20 0x7fbb180d06b0 unknown :-1 s=CLOSED pgs=2210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:31.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.333+0000 7fbb1de00640 1 -- 172.21.15.39:0/1621185652 >> 172.21.15.39:0/1621185652 conn(0x7fbb1806cfc0 msgr2=0x7fbb1806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:31.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.333+0000 7fbb1de00640 1 -- 172.21.15.39:0/1621185652 shutdown_connections 2024-09-21T07:49:31.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.333+0000 7fbb1de00640 1 -- 172.21.15.39:0/1621185652 wait complete. 2024-09-21T07:49:31.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.335+0000 7fbb1de00640 1 Processor -- start 2024-09-21T07:49:31.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.336+0000 7fbb1de00640 1 -- start start 2024-09-21T07:49:31.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.337+0000 7fbb1de00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb18155a90 0x7fbb181f9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:31.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.337+0000 7fbb1de00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbb1815fc20 0x7fbb1814afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:31.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.337+0000 7fbb1de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb18162c00 0x7fbb1814d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:31.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.337+0000 7fbb1de00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbb18070840 con 0x7fbb18162c00 2024-09-21T07:49:31.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.337+0000 7fbb1de00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbb180706c0 con 0x7fbb18155a90 2024-09-21T07:49:31.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.337+0000 7fbb1de00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fbb180709c0 con 0x7fbb1815fc20 2024-09-21T07:49:31.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.337+0000 7fbb17400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb18162c00 0x7fbb1814d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:31.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.337+0000 7fbb17400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb18162c00 0x7fbb1814d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35932/0 (socket says 172.21.15.39:35932) 2024-09-21T07:49:31.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.337+0000 7fbb17400640 1 -- 172.21.15.39:0/3400207895 learned_addr learned my addr 172.21.15.39:0/3400207895 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:31.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.338+0000 7fbb16a00640 1 --2- 172.21.15.39:0/3400207895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb18155a90 0x7fbb181f9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:31.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.338+0000 7fbb0e000640 1 --2- 172.21.15.39:0/3400207895 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbb1815fc20 0x7fbb1814afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:31.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.338+0000 7fbb17400640 1 -- 172.21.15.39:0/3400207895 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbb1815fc20 msgr2=0x7fbb1814afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:31.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.338+0000 7fbb17400640 1 --2- 172.21.15.39:0/3400207895 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbb1815fc20 0x7fbb1814afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:31.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.338+0000 7fbb17400640 1 -- 172.21.15.39:0/3400207895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb18155a90 msgr2=0x7fbb181f9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:31.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.338+0000 7fbb17400640 1 --2- 172.21.15.39:0/3400207895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb18155a90 0x7fbb181f9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:31.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.338+0000 7fbb17400640 1 -- 172.21.15.39:0/3400207895 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb181f9f80 con 0x7fbb18162c00 2024-09-21T07:49:31.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.338+0000 7fbb17400640 1 --2- 172.21.15.39:0/3400207895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb18162c00 0x7fbb1814d7b0 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7fbb1000ca80 tx=0x7fbb1000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:31.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.338+0000 7fbb0fe00640 1 -- 172.21.15.39:0/3400207895 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb10016020 con 0x7fbb18162c00 2024-09-21T07:49:31.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.339+0000 7fbb0fe00640 1 -- 172.21.15.39:0/3400207895 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb1000ad20 con 0x7fbb18162c00 2024-09-21T07:49:31.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.339+0000 7fbb0fe00640 1 -- 172.21.15.39:0/3400207895 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb10012680 con 0x7fbb18162c00 2024-09-21T07:49:31.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.339+0000 7fbb1de00640 1 -- 172.21.15.39:0/3400207895 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb180ae140 con 0x7fbb18162c00 2024-09-21T07:49:31.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.339+0000 7fbb1de00640 1 -- 172.21.15.39:0/3400207895 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb181f6ce0 con 0x7fbb18162c00 2024-09-21T07:49:31.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.340+0000 7fbb1de00640 1 -- 172.21.15.39:0/3400207895 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb180d06b0 con 0x7fbb18162c00 2024-09-21T07:49:31.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.341+0000 7fbb0fe00640 1 -- 172.21.15.39:0/3400207895 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbb10019020 con 0x7fbb18162c00 2024-09-21T07:49:31.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.342+0000 7fbb0fe00640 1 --2- 172.21.15.39:0/3400207895 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbaf4078b90 0x7fbaf407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:31.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.342+0000 7fbb0fe00640 1 -- 172.21.15.39:0/3400207895 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(229..229 src has 1..229) ==== 6164+0+0 (secure 0 0 0) 0x7fbb1009b390 con 0x7fbb18162c00 2024-09-21T07:49:31.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.342+0000 7fbb16a00640 1 --2- 172.21.15.39:0/3400207895 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbaf4078b90 0x7fbaf407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:31.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.343+0000 7fbb16a00640 1 --2- 172.21.15.39:0/3400207895 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbaf4078b90 0x7fbaf407b0a0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fbb08034000 tx=0x7fbb08004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:31.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.346+0000 7fbb0fe00640 1 -- 172.21.15.39:0/3400207895 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbb10020020 con 0x7fbb18162c00 2024-09-21T07:49:31.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.614+0000 7fbb1de00640 1 -- 172.21.15.39:0/3400207895 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fbb18070fd0 con 0x7fbb18162c00 2024-09-21T07:49:31.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.614+0000 7fbb0fe00640 1 -- 172.21.15.39:0/3400207895 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v23) ==== 80+0+5304 (secure 0 0 0) 0x7fbb10062fa0 con 0x7fbb18162c00 2024-09-21T07:49:31.617 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:49:31.617 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:49:31.617 INFO:teuthology.orchestra.run.smithi039.stdout: "auth_dump": [ 2024-09-21T07:49:31.618 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:49:31.618 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.a", 2024-09-21T07:49:31.618 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5m+P6EERAABQqmlkrocSi1+hwc73Adrg==", 2024-09-21T07:49:31.618 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:49:31.618 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:49:31.618 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:49:31.618 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:49:31.618 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:49:31.618 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:49:31.618 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:49:31.619 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:49:31.619 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.b", 2024-09-21T07:49:31.619 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5m5/GeLRAAfE8fr5dbitQa89rNK9pZ2Q==", 2024-09-21T07:49:31.619 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:49:31.619 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:49:31.619 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:49:31.619 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:49:31.619 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:49:31.619 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:49:31.619 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:49:31.619 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:49:31.619 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.c", 2024-09-21T07:49:31.620 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5msbx6HxAALsF8ORvIoJpIAFZ5uxmOEw==", 2024-09-21T07:49:31.620 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:49:31.620 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:49:31.620 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:49:31.620 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:49:31.620 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:49:31.620 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:49:31.620 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:49:31.620 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:49:31.620 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.d", 2024-09-21T07:49:31.620 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mIrcBABAApKAB/7FkZ5glNTxlbkIsBw==", 2024-09-21T07:49:31.620 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:49:31.620 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:49:31.621 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:49:31.621 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:49:31.621 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:49:31.621 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:49:31.621 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:49:31.621 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:49:31.621 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.0", 2024-09-21T07:49:31.621 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB7du5msw81DhAAmVimqN4x9K3YpT0nbovppQ==", 2024-09-21T07:49:31.621 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:49:31.621 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:49:31.621 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:49:31.621 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:49:31.621 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:49:31.622 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:49:31.622 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:49:31.622 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.1", 2024-09-21T07:49:31.622 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB+du5mdnCIOhAAtVB4ThehQwb0zRriyKcaxA==", 2024-09-21T07:49:31.622 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:49:31.622 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:49:31.622 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:49:31.622 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:49:31.622 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:49:31.622 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:49:31.623 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:49:31.623 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.2", 2024-09-21T07:49:31.623 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCCdu5mTfWxDhAAadJCbPN5xBx/f9L0v2yTdg==", 2024-09-21T07:49:31.623 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:49:31.623 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:49:31.623 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:49:31.623 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:49:31.623 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:49:31.623 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:49:31.623 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:49:31.624 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.3", 2024-09-21T07:49:31.624 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCFdu5mL1E2HxAAyOhY97uUXEw5L1CcDeONVA==", 2024-09-21T07:49:31.624 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:49:31.624 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:49:31.624 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:49:31.624 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:49:31.624 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:49:31.624 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:49:31.624 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:49:31.624 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.4", 2024-09-21T07:49:31.624 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCLdu5m6PY/IhAAgvcTgm+6uWA7lIG4tcTIEg==", 2024-09-21T07:49:31.625 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:49:31.625 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:49:31.625 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:49:31.625 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:49:31.625 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:49:31.625 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:49:31.625 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:49:31.625 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.5", 2024-09-21T07:49:31.625 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCOdu5mE/wlMxAA3JBg4xH+N/PPC1Uv2ZVrYQ==", 2024-09-21T07:49:31.625 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:49:31.625 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:49:31.625 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:49:31.625 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:49:31.626 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:49:31.626 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:49:31.626 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:49:31.626 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.6", 2024-09-21T07:49:31.626 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCSdu5mC3TgBxAAwsArjeIdFcApY3/1OJffEg==", 2024-09-21T07:49:31.626 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:49:31.626 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:49:31.626 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:49:31.626 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:49:31.626 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:49:31.626 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:49:31.627 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:49:31.627 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.7", 2024-09-21T07:49:31.627 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCVdu5mSWHWFBAA9qPHWRdsvrB74inrp1lj4Q==", 2024-09-21T07:49:31.627 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:49:31.627 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:49:31.627 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:49:31.627 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:49:31.627 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:49:31.627 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:49:31.627 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:49:31.627 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.0", 2024-09-21T07:49:31.627 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mGfb1DBAA/aSMKR5or3xy53rB/iUjpA==", 2024-09-21T07:49:31.627 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:49:31.627 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:49:31.628 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow r", 2024-09-21T07:49:31.628 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T07:49:31.628 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:49:31.628 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:49:31.628 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:49:31.628 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.1", 2024-09-21T07:49:31.628 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mEkciIhAAid+bxJEr4P7vMnZ42PAy5w==", 2024-09-21T07:49:31.628 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:49:31.628 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:49:31.628 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow r", 2024-09-21T07:49:31.628 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow rw", 2024-09-21T07:49:31.628 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rwx" 2024-09-21T07:49:31.628 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:49:31.629 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:49:31.629 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:49:31.629 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.2", 2024-09-21T07:49:31.629 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mSu2oFxAAgjy2m/wWKh6xaoh+YDoU1Q==", 2024-09-21T07:49:31.629 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:49:31.629 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:49:31.629 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow r", 2024-09-21T07:49:31.629 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow rw", 2024-09-21T07:49:31.629 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rwx" 2024-09-21T07:49:31.629 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:49:31.629 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:49:31.629 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:49:31.630 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.3", 2024-09-21T07:49:31.630 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mCHy0LBAAZneF2jUWgfuyGehZFbOxDg==", 2024-09-21T07:49:31.630 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:49:31.630 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:49:31.630 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow r", 2024-09-21T07:49:31.630 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow rw", 2024-09-21T07:49:31.630 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rwx" 2024-09-21T07:49:31.630 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:49:31.630 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:49:31.630 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:49:31.630 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.admin", 2024-09-21T07:49:31.630 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBydu5m9LInERAAAlDpqb94f2Qb9VYJJwTBcA==", 2024-09-21T07:49:31.631 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:49:31.631 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow *", 2024-09-21T07:49:31.631 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:49:31.631 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:49:31.631 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:49:31.631 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:49:31.631 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:49:31.631 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:49:31.631 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mgr.x", 2024-09-21T07:49:31.631 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5mA2GRBhAAkyCKdgH6laHy8xNt080dXg==", 2024-09-21T07:49:31.632 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:49:31.632 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow *", 2024-09-21T07:49:31.632 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile mgr", 2024-09-21T07:49:31.632 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:49:31.632 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:49:31.632 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:49:31.632 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:49:31.633 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mgr.y", 2024-09-21T07:49:31.633 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBydu5mr+8aNxAA7TFYNms2WsXUfa4ITTJOHw==", 2024-09-21T07:49:31.633 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:49:31.633 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow *", 2024-09-21T07:49:31.633 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile mgr", 2024-09-21T07:49:31.633 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:49:31.633 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:49:31.633 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:49:31.633 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-21T07:49:31.633 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:49:31.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.617+0000 7fbb1de00640 1 -- 172.21.15.39:0/3400207895 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbaf4078b90 msgr2=0x7fbaf407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:31.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.617+0000 7fbb1de00640 1 --2- 172.21.15.39:0/3400207895 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbaf4078b90 0x7fbaf407b0a0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fbb08034000 tx=0x7fbb08004730 comp rx=0 tx=0).stop 2024-09-21T07:49:31.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.617+0000 7fbb1de00640 1 -- 172.21.15.39:0/3400207895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb18162c00 msgr2=0x7fbb1814d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:31.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.617+0000 7fbb1de00640 1 --2- 172.21.15.39:0/3400207895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb18162c00 0x7fbb1814d7b0 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7fbb1000ca80 tx=0x7fbb1000cf40 comp rx=0 tx=0).stop 2024-09-21T07:49:31.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.617+0000 7fbb1de00640 1 -- 172.21.15.39:0/3400207895 shutdown_connections 2024-09-21T07:49:31.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.617+0000 7fbb1de00640 1 --2- 172.21.15.39:0/3400207895 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbaf4078b90 0x7fbaf407b0a0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:31.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.617+0000 7fbb1de00640 1 --2- 172.21.15.39:0/3400207895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb18162c00 0x7fbb1814d7b0 unknown :-1 s=CLOSED pgs=2211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:31.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.617+0000 7fbb1de00640 1 --2- 172.21.15.39:0/3400207895 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbb1815fc20 0x7fbb1814afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:31.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.617+0000 7fbb1de00640 1 --2- 172.21.15.39:0/3400207895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb18155a90 0x7fbb181f9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:31.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.617+0000 7fbb1de00640 1 -- 172.21.15.39:0/3400207895 >> 172.21.15.39:0/3400207895 conn(0x7fbb1806cfc0 msgr2=0x7fbb18162040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:31.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.617+0000 7fbb1de00640 1 -- 172.21.15.39:0/3400207895 shutdown_connections 2024-09-21T07:49:31.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.617+0000 7fbb1de00640 1 -- 172.21.15.39:0/3400207895 wait complete. 2024-09-21T07:49:31.640 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T07:49:31.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.878+0000 7fed82600640 1 -- 172.21.15.39:0/425664849 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed7c0d2d20 msgr2=0x7fed7c0b4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:31.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.878+0000 7fed82600640 1 --2- 172.21.15.39:0/425664849 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed7c0d2d20 0x7fed7c0b4520 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0ba330 tx=0x7fed7402aa10 comp rx=0 tx=0).stop 2024-09-21T07:49:31.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.878+0000 7fed82600640 1 -- 172.21.15.39:0/425664849 shutdown_connections 2024-09-21T07:49:31.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.878+0000 7fed82600640 1 --2- 172.21.15.39:0/425664849 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed7c0d2d20 0x7fed7c0b4520 unknown :-1 s=CLOSED pgs=2212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:31.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.878+0000 7fed82600640 1 --2- 172.21.15.39:0/425664849 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fed7c0b5450 0x7fed7c0d27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:31.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.878+0000 7fed82600640 1 --2- 172.21.15.39:0/425664849 >> v2:172.21.15.148:3300/0 conn(0x7fed7c0b97e0 0x7fed7c0b6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:31.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.878+0000 7fed82600640 1 -- 172.21.15.39:0/425664849 >> 172.21.15.39:0/425664849 conn(0x7fed7c09d7d0 msgr2=0x7fed7c0d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:31.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.878+0000 7fed82600640 1 -- 172.21.15.39:0/425664849 shutdown_connections 2024-09-21T07:49:31.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.878+0000 7fed82600640 1 -- 172.21.15.39:0/425664849 wait complete. 2024-09-21T07:49:31.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.880+0000 7fed82600640 1 Processor -- start 2024-09-21T07:49:31.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.880+0000 7fed82600640 1 -- start start 2024-09-21T07:49:31.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.882+0000 7fed82600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed7c0b5450 0x7fed7c151a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:31.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.882+0000 7fed82600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed7c0b97e0 0x7fed7c1f8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:31.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.882+0000 7fed82600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fed7c0d2d20 0x7fed7c1f8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:31.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.882+0000 7fed82600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c162a50 con 0x7fed7c0b5450 2024-09-21T07:49:31.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.882+0000 7fed82600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c0b4fd0 con 0x7fed7c0b97e0 2024-09-21T07:49:31.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.882+0000 7fed82600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fed7c162bd0 con 0x7fed7c0d2d20 2024-09-21T07:49:31.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.882+0000 7fed81200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed7c0b5450 0x7fed7c151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:31.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.882+0000 7fed81200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed7c0b5450 0x7fed7c151a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35956/0 (socket says 172.21.15.39:35956) 2024-09-21T07:49:31.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.882+0000 7fed81200640 1 -- 172.21.15.39:0/284393260 learned_addr learned my addr 172.21.15.39:0/284393260 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:31.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.882+0000 7fed73e00640 1 --2- 172.21.15.39:0/284393260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed7c0b97e0 0x7fed7c1f8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:31.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.882+0000 7fed81c00640 1 --2- 172.21.15.39:0/284393260 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fed7c0d2d20 0x7fed7c1f8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:31.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.882+0000 7fed81200640 1 -- 172.21.15.39:0/284393260 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fed7c0d2d20 msgr2=0x7fed7c1f8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:31.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.882+0000 7fed81200640 1 --2- 172.21.15.39:0/284393260 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fed7c0d2d20 0x7fed7c1f8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:31.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.882+0000 7fed81200640 1 -- 172.21.15.39:0/284393260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed7c0b97e0 msgr2=0x7fed7c1f8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:31.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.882+0000 7fed81200640 1 --2- 172.21.15.39:0/284393260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed7c0b97e0 0x7fed7c1f8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:31.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.882+0000 7fed81200640 1 -- 172.21.15.39:0/284393260 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed7c15f580 con 0x7fed7c0b5450 2024-09-21T07:49:31.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.883+0000 7fed81200640 1 --2- 172.21.15.39:0/284393260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed7c0b5450 0x7fed7c151a70 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00d770 tx=0x7fed6c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:31.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.883+0000 7fed79600640 1 -- 172.21.15.39:0/284393260 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c017070 con 0x7fed7c0b5450 2024-09-21T07:49:31.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.883+0000 7fed79600640 1 -- 172.21.15.39:0/284393260 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed6c009d20 con 0x7fed7c0b5450 2024-09-21T07:49:31.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.883+0000 7fed79600640 1 -- 172.21.15.39:0/284393260 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c0106c0 con 0x7fed7c0b5450 2024-09-21T07:49:31.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.883+0000 7fed82600640 1 -- 172.21.15.39:0/284393260 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed7c1616f0 con 0x7fed7c0b5450 2024-09-21T07:49:31.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.883+0000 7fed82600640 1 -- 172.21.15.39:0/284393260 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed7c161cb0 con 0x7fed7c0b5450 2024-09-21T07:49:31.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.884+0000 7fed82600640 1 -- 172.21.15.39:0/284393260 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed7c0d52e0 con 0x7fed7c0b5450 2024-09-21T07:49:31.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.886+0000 7fed79600640 1 -- 172.21.15.39:0/284393260 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fed6c01a070 con 0x7fed7c0b5450 2024-09-21T07:49:31.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.887+0000 7fed79600640 1 --2- 172.21.15.39:0/284393260 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fed58078bc0 0x7fed5807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:31.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.887+0000 7fed79600640 1 -- 172.21.15.39:0/284393260 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(229..229 src has 1..229) ==== 6164+0+0 (secure 0 0 0) 0x7fed6c09af10 con 0x7fed7c0b5450 2024-09-21T07:49:31.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.887+0000 7fed73e00640 1 --2- 172.21.15.39:0/284393260 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fed58078bc0 0x7fed5807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:31.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.888+0000 7fed73e00640 1 --2- 172.21.15.39:0/284393260 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fed58078bc0 0x7fed5807b0d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fed7c15f230 tx=0x7fed6400b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:31.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:31.892+0000 7fed79600640 1 -- 172.21.15.39:0/284393260 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fed7c0d52e0 con 0x7fed7c0b5450 2024-09-21T07:49:32.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.090+0000 7fed82600640 1 -- 172.21.15.39:0/284393260 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fed7c0d27e0 con 0x7fed7c0b5450 2024-09-21T07:49:32.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.091+0000 7fed79600640 1 -- 172.21.15.39:0/284393260 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v229) ==== 84+0+287 (secure 0 0 0) 0x7fed6c062b30 con 0x7fed7c0b5450 2024-09-21T07:49:32.092 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:49:32.092 INFO:teuthology.orchestra.run.smithi039.stdout:[ 2024-09-21T07:49:32.092 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:49:32.092 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 1, 2024-09-21T07:49:32.092 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": ".mgr" 2024-09-21T07:49:32.092 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:49:32.092 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:49:32.092 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 6, 2024-09-21T07:49:32.092 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": "snap-test-pool" 2024-09-21T07:49:32.092 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:49:32.092 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:49:32.092 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 11, 2024-09-21T07:49:32.093 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": "test_data_pool" 2024-09-21T07:49:32.093 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:49:32.093 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:49:32.093 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 12, 2024-09-21T07:49:32.093 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": "test_metadata_pool" 2024-09-21T07:49:32.093 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:49:32.093 INFO:teuthology.orchestra.run.smithi039.stdout:] 2024-09-21T07:49:32.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.093+0000 7fed82600640 1 -- 172.21.15.39:0/284393260 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fed58078bc0 msgr2=0x7fed5807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:32.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.093+0000 7fed82600640 1 --2- 172.21.15.39:0/284393260 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fed58078bc0 0x7fed5807b0d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fed7c15f230 tx=0x7fed6400b2e0 comp rx=0 tx=0).stop 2024-09-21T07:49:32.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.094+0000 7fed82600640 1 -- 172.21.15.39:0/284393260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed7c0b5450 msgr2=0x7fed7c151a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:32.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.094+0000 7fed82600640 1 --2- 172.21.15.39:0/284393260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed7c0b5450 0x7fed7c151a70 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00d770 tx=0x7fed6c00dc30 comp rx=0 tx=0).stop 2024-09-21T07:49:32.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.094+0000 7fed82600640 1 -- 172.21.15.39:0/284393260 shutdown_connections 2024-09-21T07:49:32.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.094+0000 7fed82600640 1 --2- 172.21.15.39:0/284393260 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fed58078bc0 0x7fed5807b0d0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:32.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.094+0000 7fed82600640 1 --2- 172.21.15.39:0/284393260 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fed7c0d2d20 0x7fed7c1f8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:32.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.094+0000 7fed82600640 1 --2- 172.21.15.39:0/284393260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed7c0b97e0 0x7fed7c1f8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:32.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.094+0000 7fed82600640 1 --2- 172.21.15.39:0/284393260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed7c0b5450 0x7fed7c151a70 unknown :-1 s=CLOSED pgs=2213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:32.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.094+0000 7fed82600640 1 -- 172.21.15.39:0/284393260 >> 172.21.15.39:0/284393260 conn(0x7fed7c09d7d0 msgr2=0x7fed7c0ceb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:32.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.094+0000 7fed82600640 1 -- 172.21.15.39:0/284393260 shutdown_connections 2024-09-21T07:49:32.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.094+0000 7fed82600640 1 -- 172.21.15.39:0/284393260 wait complete. 2024-09-21T07:49:32.119 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T07:49:32.120 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T07:49:32.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.348+0000 7f23ed800640 1 -- 172.21.15.39:0/3833309920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23e815d440 msgr2=0x7f23e815d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:32.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.348+0000 7f23ed800640 1 --2- 172.21.15.39:0/3833309920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23e815d440 0x7f23e815d850 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7f23e0005fd0 tx=0x7f23e002aa90 comp rx=0 tx=0).stop 2024-09-21T07:49:32.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.348+0000 7f23ed800640 1 -- 172.21.15.39:0/3833309920 shutdown_connections 2024-09-21T07:49:32.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.348+0000 7f23ed800640 1 --2- 172.21.15.39:0/3833309920 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f23e80b4c40 0x7f23e8160340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:32.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.348+0000 7f23ed800640 1 --2- 172.21.15.39:0/3833309920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23e815d440 0x7f23e815d850 unknown :-1 s=CLOSED pgs=2214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:32.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.348+0000 7f23ed800640 1 -- 172.21.15.39:0/3833309920 >> 172.21.15.39:0/3833309920 conn(0x7f23e806d660 msgr2=0x7f23e80b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:32.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.348+0000 7f23ed800640 1 -- 172.21.15.39:0/3833309920 shutdown_connections 2024-09-21T07:49:32.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.348+0000 7f23ed800640 1 -- 172.21.15.39:0/3833309920 wait complete. 2024-09-21T07:49:32.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.350+0000 7f23ed800640 1 Processor -- start 2024-09-21T07:49:32.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.350+0000 7f23ed800640 1 -- start start 2024-09-21T07:49:32.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.352+0000 7f23ed800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23e80b4c40 0x7f23e81e6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:32.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.352+0000 7f23ed800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23e81539f0 0x7f23e809d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:32.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.352+0000 7f23ed800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f23e815d440 0x7f23e80b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:32.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.352+0000 7f23ed800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f23e80ad670 con 0x7f23e80b4c40 2024-09-21T07:49:32.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.352+0000 7f23ed800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f23e80ad4f0 con 0x7f23e81539f0 2024-09-21T07:49:32.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.352+0000 7f23ed800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f23e80ad7f0 con 0x7f23e815d440 2024-09-21T07:49:32.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.352+0000 7f23e7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23e80b4c40 0x7f23e81e6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:32.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.352+0000 7f23e7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23e80b4c40 0x7f23e81e6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41558/0 (socket says 172.21.15.39:41558) 2024-09-21T07:49:32.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.352+0000 7f23e7e00640 1 -- 172.21.15.39:0/231398190 learned_addr learned my addr 172.21.15.39:0/231398190 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:32.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.352+0000 7f23dfe00640 1 --2- 172.21.15.39:0/231398190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23e81539f0 0x7f23e809d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:32.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.352+0000 7f23ece00640 1 --2- 172.21.15.39:0/231398190 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f23e815d440 0x7f23e80b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:32.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.353+0000 7f23e7e00640 1 -- 172.21.15.39:0/231398190 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f23e815d440 msgr2=0x7f23e80b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:32.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.353+0000 7f23e7e00640 1 --2- 172.21.15.39:0/231398190 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f23e815d440 0x7f23e80b0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:32.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.353+0000 7f23e7e00640 1 -- 172.21.15.39:0/231398190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23e81539f0 msgr2=0x7f23e809d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:32.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.353+0000 7f23e7e00640 1 --2- 172.21.15.39:0/231398190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23e81539f0 0x7f23e809d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:32.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.353+0000 7f23e7e00640 1 -- 172.21.15.39:0/231398190 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23e809df20 con 0x7f23e80b4c40 2024-09-21T07:49:32.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.353+0000 7f23e7e00640 1 --2- 172.21.15.39:0/231398190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23e80b4c40 0x7f23e81e6b40 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7f23d800c3a0 tx=0x7f23d800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:32.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.353+0000 7f23e4c00640 1 -- 172.21.15.39:0/231398190 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d8017070 con 0x7f23e80b4c40 2024-09-21T07:49:32.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.353+0000 7f23e4c00640 1 -- 172.21.15.39:0/231398190 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23d80053c0 con 0x7f23e80b4c40 2024-09-21T07:49:32.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.353+0000 7f23e4c00640 1 -- 172.21.15.39:0/231398190 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d8003850 con 0x7f23e80b4c40 2024-09-21T07:49:32.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.353+0000 7f23ed800640 1 -- 172.21.15.39:0/231398190 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23e81f9390 con 0x7f23e80b4c40 2024-09-21T07:49:32.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.354+0000 7f23ed800640 1 -- 172.21.15.39:0/231398190 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23e81f9950 con 0x7f23e80b4c40 2024-09-21T07:49:32.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.354+0000 7f23ed800640 1 -- 172.21.15.39:0/231398190 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23e8160340 con 0x7f23e80b4c40 2024-09-21T07:49:32.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.356+0000 7f23e4c00640 1 -- 172.21.15.39:0/231398190 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f23d801a070 con 0x7f23e80b4c40 2024-09-21T07:49:32.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.357+0000 7f23e4c00640 1 --2- 172.21.15.39:0/231398190 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f23c4078bc0 0x7f23c407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:32.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.357+0000 7f23e4c00640 1 -- 172.21.15.39:0/231398190 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(229..229 src has 1..229) ==== 6164+0+0 (secure 0 0 0) 0x7f23d809a2a0 con 0x7f23e80b4c40 2024-09-21T07:49:32.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.358+0000 7f23dfe00640 1 --2- 172.21.15.39:0/231398190 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f23c4078bc0 0x7f23c407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:32.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.359+0000 7f23dfe00640 1 --2- 172.21.15.39:0/231398190 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f23c4078bc0 0x7f23c407b0d0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f23e8150700 tx=0x7f23d000b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:32.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.362+0000 7f23e4c00640 1 -- 172.21.15.39:0/231398190 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f23d8061e20 con 0x7f23e80b4c40 2024-09-21T07:49:32.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:32.556+0000 7f23ed800640 1 -- 172.21.15.39:0/231398190 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f23e8160080 con 0x7f23e80b4c40 2024-09-21T07:49:33.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.098+0000 7f23e4c00640 1 -- 172.21.15.39:0/231398190 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v230) ==== 138+0+0 (secure 0 0 0) 0x7f23d8066850 con 0x7f23e80b4c40 2024-09-21T07:49:33.099 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' created 2024-09-21T07:49:33.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.100+0000 7f23ed800640 1 -- 172.21.15.39:0/231398190 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f23c4078bc0 msgr2=0x7f23c407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:33.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.100+0000 7f23ed800640 1 --2- 172.21.15.39:0/231398190 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f23c4078bc0 0x7f23c407b0d0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f23e8150700 tx=0x7f23d000b2e0 comp rx=0 tx=0).stop 2024-09-21T07:49:33.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.100+0000 7f23ed800640 1 -- 172.21.15.39:0/231398190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23e80b4c40 msgr2=0x7f23e81e6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:33.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.100+0000 7f23ed800640 1 --2- 172.21.15.39:0/231398190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23e80b4c40 0x7f23e81e6b40 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7f23d800c3a0 tx=0x7f23d800c860 comp rx=0 tx=0).stop 2024-09-21T07:49:33.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.100+0000 7f23ed800640 1 -- 172.21.15.39:0/231398190 shutdown_connections 2024-09-21T07:49:33.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.100+0000 7f23ed800640 1 --2- 172.21.15.39:0/231398190 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f23c4078bc0 0x7f23c407b0d0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:33.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.100+0000 7f23ed800640 1 --2- 172.21.15.39:0/231398190 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f23e815d440 0x7f23e80b0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:33.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.100+0000 7f23ed800640 1 --2- 172.21.15.39:0/231398190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23e81539f0 0x7f23e809d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:33.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.101+0000 7f23ed800640 1 --2- 172.21.15.39:0/231398190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23e80b4c40 0x7f23e81e6b40 unknown :-1 s=CLOSED pgs=2215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:33.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.101+0000 7f23ed800640 1 -- 172.21.15.39:0/231398190 >> 172.21.15.39:0/231398190 conn(0x7f23e806d660 msgr2=0x7f23e80700e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:33.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.101+0000 7f23ed800640 1 -- 172.21.15.39:0/231398190 shutdown_connections 2024-09-21T07:49:33.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.101+0000 7f23ed800640 1 -- 172.21.15.39:0/231398190 wait complete. 2024-09-21T07:49:33.130 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T07:49:33.197 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:49:33.201 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:49:33.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.366+0000 7ff089c00640 1 -- 172.21.15.39:0/1139053390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff084160660 msgr2=0x7ff0840cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:33.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.366+0000 7ff089c00640 1 --2- 172.21.15.39:0/1139053390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff084160660 0x7ff0840cf710 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7ff06c009830 tx=0x7ff06c02a530 comp rx=0 tx=0).stop 2024-09-21T07:49:33.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.367+0000 7ff089c00640 1 -- 172.21.15.39:0/1139053390 shutdown_connections 2024-09-21T07:49:33.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.367+0000 7ff089c00640 1 --2- 172.21.15.39:0/1139053390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff084160660 0x7ff0840cf710 unknown :-1 s=CLOSED pgs=2217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:33.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.367+0000 7ff089c00640 1 --2- 172.21.15.39:0/1139053390 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff084160250 0x7ff08415d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:33.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.367+0000 7ff089c00640 1 -- 172.21.15.39:0/1139053390 >> 172.21.15.39:0/1139053390 conn(0x7ff08406cfc0 msgr2=0x7ff08406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:33.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.367+0000 7ff089c00640 1 -- 172.21.15.39:0/1139053390 shutdown_connections 2024-09-21T07:49:33.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.367+0000 7ff089c00640 1 -- 172.21.15.39:0/1139053390 wait complete. 2024-09-21T07:49:33.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.369+0000 7ff089c00640 1 Processor -- start 2024-09-21T07:49:33.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.369+0000 7ff089c00640 1 -- start start 2024-09-21T07:49:33.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.371+0000 7ff089c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff0840cfc50 0x7ff0840b1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:33.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.371+0000 7ff089c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff084160250 0x7ff084158450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:33.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.371+0000 7ff089c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff084160660 0x7ff08413ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:33.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.371+0000 7ff089c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff0840cce00 con 0x7ff084160660 2024-09-21T07:49:33.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.371+0000 7ff089c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff0840ccc80 con 0x7ff084160250 2024-09-21T07:49:33.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.371+0000 7ff089c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7ff0840ccf80 con 0x7ff0840cfc50 2024-09-21T07:49:33.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.371+0000 7ff089200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff084160660 0x7ff08413ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:33.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.371+0000 7ff089200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff084160660 0x7ff08413ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41590/0 (socket says 172.21.15.39:41590) 2024-09-21T07:49:33.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.371+0000 7ff089200640 1 -- 172.21.15.39:0/2512170030 learned_addr learned my addr 172.21.15.39:0/2512170030 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:33.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.371+0000 7ff07b400640 1 --2- 172.21.15.39:0/2512170030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff084160250 0x7ff084158450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:33.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.371+0000 7ff083e00640 1 --2- 172.21.15.39:0/2512170030 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff0840cfc50 0x7ff0840b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:33.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.371+0000 7ff089200640 1 -- 172.21.15.39:0/2512170030 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff0840cfc50 msgr2=0x7ff0840b1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:33.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.371+0000 7ff089200640 1 --2- 172.21.15.39:0/2512170030 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff0840cfc50 0x7ff0840b1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:33.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.371+0000 7ff089200640 1 -- 172.21.15.39:0/2512170030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff084160250 msgr2=0x7ff084158450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:33.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.371+0000 7ff089200640 1 --2- 172.21.15.39:0/2512170030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff084160250 0x7ff084158450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:33.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.371+0000 7ff089200640 1 -- 172.21.15.39:0/2512170030 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0840afff0 con 0x7ff084160660 2024-09-21T07:49:33.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.372+0000 7ff089200640 1 --2- 172.21.15.39:0/2512170030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff084160660 0x7ff08413ba30 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7ff07c00c730 tx=0x7ff07c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:33.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.372+0000 7ff080c00640 1 -- 172.21.15.39:0/2512170030 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c017070 con 0x7ff084160660 2024-09-21T07:49:33.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.372+0000 7ff080c00640 1 -- 172.21.15.39:0/2512170030 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff07c003c60 con 0x7ff084160660 2024-09-21T07:49:33.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.372+0000 7ff080c00640 1 -- 172.21.15.39:0/2512170030 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c0052d0 con 0x7ff084160660 2024-09-21T07:49:33.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.372+0000 7ff089c00640 1 -- 172.21.15.39:0/2512170030 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0840b02d0 con 0x7ff084160660 2024-09-21T07:49:33.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.372+0000 7ff089c00640 1 -- 172.21.15.39:0/2512170030 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0840b0890 con 0x7ff084160660 2024-09-21T07:49:33.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.373+0000 7ff089c00640 1 -- 172.21.15.39:0/2512170030 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0840afd60 con 0x7ff084160660 2024-09-21T07:49:33.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.375+0000 7ff080c00640 1 -- 172.21.15.39:0/2512170030 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff07c01a070 con 0x7ff084160660 2024-09-21T07:49:33.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.376+0000 7ff080c00640 1 --2- 172.21.15.39:0/2512170030 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff060078bc0 0x7ff06007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:33.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.376+0000 7ff080c00640 1 -- 172.21.15.39:0/2512170030 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(230..230 src has 1..230) ==== 6534+0+0 (secure 0 0 0) 0x7ff07c09a840 con 0x7ff084160660 2024-09-21T07:49:33.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.376+0000 7ff083e00640 1 --2- 172.21.15.39:0/2512170030 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff060078bc0 0x7ff06007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:33.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.377+0000 7ff083e00640 1 --2- 172.21.15.39:0/2512170030 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff060078bc0 0x7ff06007b0d0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7ff07400bb50 tx=0x7ff07400b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:33.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.381+0000 7ff080c00640 1 -- 172.21.15.39:0/2512170030 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff07c062250 con 0x7ff084160660 2024-09-21T07:49:33.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:33.579+0000 7ff089c00640 1 -- 172.21.15.39:0/2512170030 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ff0840cd430 con 0x7ff084160660 2024-09-21T07:49:34.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.110+0000 7ff080c00640 1 -- 172.21.15.39:0/2512170030 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v231) ==== 170+0+0 (secure 0 0 0) 0x7ff07c066c80 con 0x7ff084160660 2024-09-21T07:49:34.111 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' created 2024-09-21T07:49:34.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.113+0000 7ff089c00640 1 -- 172.21.15.39:0/2512170030 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff060078bc0 msgr2=0x7ff06007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:34.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.113+0000 7ff089c00640 1 --2- 172.21.15.39:0/2512170030 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff060078bc0 0x7ff06007b0d0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7ff07400bb50 tx=0x7ff07400b2e0 comp rx=0 tx=0).stop 2024-09-21T07:49:34.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.113+0000 7ff089c00640 1 -- 172.21.15.39:0/2512170030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff084160660 msgr2=0x7ff08413ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:34.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.113+0000 7ff089c00640 1 --2- 172.21.15.39:0/2512170030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff084160660 0x7ff08413ba30 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7ff07c00c730 tx=0x7ff07c00cbf0 comp rx=0 tx=0).stop 2024-09-21T07:49:34.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.113+0000 7ff089c00640 1 -- 172.21.15.39:0/2512170030 shutdown_connections 2024-09-21T07:49:34.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.113+0000 7ff089c00640 1 --2- 172.21.15.39:0/2512170030 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff060078bc0 0x7ff06007b0d0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:34.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.113+0000 7ff089c00640 1 --2- 172.21.15.39:0/2512170030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff084160660 0x7ff08413ba30 unknown :-1 s=CLOSED pgs=2218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:34.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.113+0000 7ff089c00640 1 --2- 172.21.15.39:0/2512170030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff084160250 0x7ff084158450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:34.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.113+0000 7ff089c00640 1 --2- 172.21.15.39:0/2512170030 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff0840cfc50 0x7ff0840b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:34.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.113+0000 7ff089c00640 1 -- 172.21.15.39:0/2512170030 >> 172.21.15.39:0/2512170030 conn(0x7ff08406cfc0 msgr2=0x7ff0840735b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:34.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.113+0000 7ff089c00640 1 -- 172.21.15.39:0/2512170030 shutdown_connections 2024-09-21T07:49:34.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.113+0000 7ff089c00640 1 -- 172.21.15.39:0/2512170030 wait complete. 2024-09-21T07:49:34.154 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T07:49:34.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.386+0000 7fae08600640 1 -- 172.21.15.39:0/1390853819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae0015d830 msgr2=0x7fae00156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:34.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.386+0000 7fae08600640 1 --2- 172.21.15.39:0/1390853819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae0015d830 0x7fae00156bb0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fadec005fd0 tx=0x7fadec02a540 comp rx=0 tx=0).stop 2024-09-21T07:49:34.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.386+0000 7fae08600640 1 -- 172.21.15.39:0/1390853819 shutdown_connections 2024-09-21T07:49:34.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.386+0000 7fae08600640 1 --2- 172.21.15.39:0/1390853819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae0015d830 0x7fae00156bb0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:34.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.386+0000 7fae08600640 1 --2- 172.21.15.39:0/1390853819 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae000b30a0 0x7fae000cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:34.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.386+0000 7fae08600640 1 -- 172.21.15.39:0/1390853819 >> 172.21.15.39:0/1390853819 conn(0x7fae000b2460 msgr2=0x7fae0006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:34.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.386+0000 7fae08600640 1 -- 172.21.15.39:0/1390853819 shutdown_connections 2024-09-21T07:49:34.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.386+0000 7fae08600640 1 -- 172.21.15.39:0/1390853819 wait complete. 2024-09-21T07:49:34.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.388+0000 7fae08600640 1 Processor -- start 2024-09-21T07:49:34.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.388+0000 7fae08600640 1 -- start start 2024-09-21T07:49:34.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.390+0000 7fae08600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae000b30a0 0x7fae001f5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:34.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.390+0000 7fae08600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae000ccc60 0x7fae000b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:34.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.390+0000 7fae08600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae0015d830 0x7fae000b11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:34.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.390+0000 7fae08600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fae000af450 con 0x7fae000b30a0 2024-09-21T07:49:34.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.390+0000 7fae08600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fae000af2d0 con 0x7fae0015d830 2024-09-21T07:49:34.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.390+0000 7fae08600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fae000af5d0 con 0x7fae000ccc60 2024-09-21T07:49:34.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.390+0000 7fae05c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae000b30a0 0x7fae001f5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:34.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.390+0000 7fae05c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae000b30a0 0x7fae001f5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41594/0 (socket says 172.21.15.39:41594) 2024-09-21T07:49:34.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.390+0000 7fae05c00640 1 -- 172.21.15.39:0/1546250258 learned_addr learned my addr 172.21.15.39:0/1546250258 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:34.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.390+0000 7fae05200640 1 --2- 172.21.15.39:0/1546250258 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae000ccc60 0x7fae000b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:34.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.390+0000 7fae06600640 1 --2- 172.21.15.39:0/1546250258 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae0015d830 0x7fae000b11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:34.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.391+0000 7fae05c00640 1 -- 172.21.15.39:0/1546250258 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae000ccc60 msgr2=0x7fae000b0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:34.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.391+0000 7fae05c00640 1 --2- 172.21.15.39:0/1546250258 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae000ccc60 0x7fae000b0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:34.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.391+0000 7fae05c00640 1 -- 172.21.15.39:0/1546250258 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae0015d830 msgr2=0x7fae000b11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:34.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.391+0000 7fae05c00640 1 --2- 172.21.15.39:0/1546250258 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae0015d830 0x7fae000b11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:34.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.391+0000 7fae05c00640 1 -- 172.21.15.39:0/1546250258 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae000ca1f0 con 0x7fae000b30a0 2024-09-21T07:49:34.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.391+0000 7fae05c00640 1 --2- 172.21.15.39:0/1546250258 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae000b30a0 0x7fae001f5350 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7fadf400e790 tx=0x7fadf400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:34.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.391+0000 7fadfd600640 1 -- 172.21.15.39:0/1546250258 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf400eea0 con 0x7fae000b30a0 2024-09-21T07:49:34.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.392+0000 7fae08600640 1 -- 172.21.15.39:0/1546250258 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae00073140 con 0x7fae000b30a0 2024-09-21T07:49:34.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.392+0000 7fadfd600640 1 -- 172.21.15.39:0/1546250258 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fadf400cdb0 con 0x7fae000b30a0 2024-09-21T07:49:34.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.392+0000 7fadfd600640 1 -- 172.21.15.39:0/1546250258 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf4010690 con 0x7fae000b30a0 2024-09-21T07:49:34.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.392+0000 7fae08600640 1 -- 172.21.15.39:0/1546250258 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae000c6c30 con 0x7fae000b30a0 2024-09-21T07:49:34.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.393+0000 7fae08600640 1 -- 172.21.15.39:0/1546250258 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae000afa30 con 0x7fae000b30a0 2024-09-21T07:49:34.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.394+0000 7fadfd600640 1 -- 172.21.15.39:0/1546250258 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fadf4010830 con 0x7fae000b30a0 2024-09-21T07:49:34.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.395+0000 7fadfd600640 1 --2- 172.21.15.39:0/1546250258 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fade0078b90 0x7fade007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:34.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.395+0000 7fadfd600640 1 -- 172.21.15.39:0/1546250258 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(231..231 src has 1..231) ==== 6916+0+0 (secure 0 0 0) 0x7fadf409c410 con 0x7fae000b30a0 2024-09-21T07:49:34.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.396+0000 7fae05200640 1 --2- 172.21.15.39:0/1546250258 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fade0078b90 0x7fade007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:34.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.397+0000 7fae05200640 1 --2- 172.21.15.39:0/1546250258 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fade0078b90 0x7fade007b0a0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fadec033000 tx=0x7fadec004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:34.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.399+0000 7fadfd600640 1 -- 172.21.15.39:0/1546250258 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fadf4063d40 con 0x7fae000b30a0 2024-09-21T07:49:34.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:34.636+0000 7fae08600640 1 -- 172.21.15.39:0/1546250258 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fae0015f350 con 0x7fae000b30a0 2024-09-21T07:49:35.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.112+0000 7fadfd600640 1 -- 172.21.15.39:0/1546250258 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 29 and data pool 30 v240) ==== 174+0+0 (secure 0 0 0) 0x7fadf4068770 con 0x7fae000b30a0 2024-09-21T07:49:35.113 INFO:teuthology.orchestra.run.smithi039.stderr:new fs with metadata pool 29 and data pool 30 2024-09-21T07:49:35.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.116+0000 7fae08600640 1 -- 172.21.15.39:0/1546250258 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fade0078b90 msgr2=0x7fade007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:35.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.116+0000 7fae08600640 1 --2- 172.21.15.39:0/1546250258 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fade0078b90 0x7fade007b0a0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fadec033000 tx=0x7fadec004730 comp rx=0 tx=0).stop 2024-09-21T07:49:35.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.116+0000 7fae08600640 1 -- 172.21.15.39:0/1546250258 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae000b30a0 msgr2=0x7fae001f5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:35.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.116+0000 7fae08600640 1 --2- 172.21.15.39:0/1546250258 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae000b30a0 0x7fae001f5350 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7fadf400e790 tx=0x7fadf400ec50 comp rx=0 tx=0).stop 2024-09-21T07:49:35.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.116+0000 7fae08600640 1 -- 172.21.15.39:0/1546250258 shutdown_connections 2024-09-21T07:49:35.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.116+0000 7fae08600640 1 --2- 172.21.15.39:0/1546250258 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fade0078b90 0x7fade007b0a0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:35.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.116+0000 7fae08600640 1 --2- 172.21.15.39:0/1546250258 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae0015d830 0x7fae000b11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:35.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.116+0000 7fae08600640 1 --2- 172.21.15.39:0/1546250258 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae000ccc60 0x7fae000b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:35.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.116+0000 7fae08600640 1 --2- 172.21.15.39:0/1546250258 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae000b30a0 0x7fae001f5350 unknown :-1 s=CLOSED pgs=2219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:35.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.116+0000 7fae08600640 1 -- 172.21.15.39:0/1546250258 >> 172.21.15.39:0/1546250258 conn(0x7fae000b2460 msgr2=0x7fae000ad760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:35.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.116+0000 7fae08600640 1 -- 172.21.15.39:0/1546250258 shutdown_connections 2024-09-21T07:49:35.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.117+0000 7fae08600640 1 -- 172.21.15.39:0/1546250258 wait complete. 2024-09-21T07:49:35.166 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:49:35.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.395+0000 7f4c90e00640 1 -- 172.21.15.39:0/2437039450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c8c15cc00 msgr2=0x7f4c8c15d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:35.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.395+0000 7f4c90e00640 1 --2- 172.21.15.39:0/2437039450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c8c15cc00 0x7f4c8c15d010 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c0d0170 tx=0x7f4c7402a560 comp rx=0 tx=0).stop 2024-09-21T07:49:35.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.395+0000 7f4c90e00640 1 -- 172.21.15.39:0/2437039450 shutdown_connections 2024-09-21T07:49:35.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.395+0000 7f4c90e00640 1 --2- 172.21.15.39:0/2437039450 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4c8c0d04b0 0x7f4c8c0d08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:35.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.395+0000 7f4c90e00640 1 --2- 172.21.15.39:0/2437039450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c8c15cc00 0x7f4c8c15d010 unknown :-1 s=CLOSED pgs=2220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:35.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.395+0000 7f4c90e00640 1 --2- 172.21.15.39:0/2437039450 >> v2:172.21.15.148:3300/0 conn(0x7f4c8c0cfa20 0x7f4c8c0cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:35.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.395+0000 7f4c90e00640 1 -- 172.21.15.39:0/2437039450 >> 172.21.15.39:0/2437039450 conn(0x7f4c8c06cfc0 msgr2=0x7f4c8c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:35.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.395+0000 7f4c90e00640 1 -- 172.21.15.39:0/2437039450 shutdown_connections 2024-09-21T07:49:35.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.395+0000 7f4c90e00640 1 -- 172.21.15.39:0/2437039450 wait complete. 2024-09-21T07:49:35.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.397+0000 7f4c90e00640 1 Processor -- start 2024-09-21T07:49:35.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.398+0000 7f4c90e00640 1 -- start start 2024-09-21T07:49:35.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.399+0000 7f4c90e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c8c0cfa20 0x7f4c8c155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:35.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.399+0000 7f4c90e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4c8c0d04b0 0x7f4c8c0b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:35.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.399+0000 7f4c90e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c8c15cc00 0x7f4c8c0b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:35.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.399+0000 7f4c90e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c06ecf0 con 0x7f4c8c0cfa20 2024-09-21T07:49:35.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.399+0000 7f4c90e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c074f60 con 0x7f4c8c15cc00 2024-09-21T07:49:35.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.399+0000 7f4c90e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f4c8c06ee70 con 0x7f4c8c0d04b0 2024-09-21T07:49:35.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.399+0000 7f4c8b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c8c0cfa20 0x7f4c8c155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:35.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.399+0000 7f4c8b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c8c0cfa20 0x7f4c8c155d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41616/0 (socket says 172.21.15.39:41616) 2024-09-21T07:49:35.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.399+0000 7f4c8b400640 1 -- 172.21.15.39:0/2809007243 learned_addr learned my addr 172.21.15.39:0/2809007243 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:35.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.399+0000 7f4c83e00640 1 --2- 172.21.15.39:0/2809007243 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4c8c0d04b0 0x7f4c8c0b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:35.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.400+0000 7f4c8be00640 1 --2- 172.21.15.39:0/2809007243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c8c15cc00 0x7f4c8c0b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:35.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.400+0000 7f4c8b400640 1 -- 172.21.15.39:0/2809007243 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4c8c0d04b0 msgr2=0x7f4c8c0b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:35.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.400+0000 7f4c8b400640 1 --2- 172.21.15.39:0/2809007243 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4c8c0d04b0 0x7f4c8c0b10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:35.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.400+0000 7f4c8b400640 1 -- 172.21.15.39:0/2809007243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c8c15cc00 msgr2=0x7f4c8c0b15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:35.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.400+0000 7f4c8b400640 1 --2- 172.21.15.39:0/2809007243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c8c15cc00 0x7f4c8c0b15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:35.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.400+0000 7f4c8b400640 1 -- 172.21.15.39:0/2809007243 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c8c1542e0 con 0x7f4c8c0cfa20 2024-09-21T07:49:35.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.400+0000 7f4c8b400640 1 --2- 172.21.15.39:0/2809007243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c8c0cfa20 0x7f4c8c155d80 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c00d770 tx=0x7f4c7c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:35.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.400+0000 7f4c83400640 1 -- 172.21.15.39:0/2809007243 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c017070 con 0x7f4c8c0cfa20 2024-09-21T07:49:35.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.400+0000 7f4c83400640 1 -- 172.21.15.39:0/2809007243 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c7c009d20 con 0x7f4c8c0cfa20 2024-09-21T07:49:35.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.400+0000 7f4c90e00640 1 -- 172.21.15.39:0/2809007243 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c8c156cb0 con 0x7f4c8c0cfa20 2024-09-21T07:49:35.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.400+0000 7f4c83400640 1 -- 172.21.15.39:0/2809007243 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c0106c0 con 0x7f4c8c0cfa20 2024-09-21T07:49:35.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.400+0000 7f4c90e00640 1 -- 172.21.15.39:0/2809007243 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c8c1566f0 con 0x7f4c8c0cfa20 2024-09-21T07:49:35.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.401+0000 7f4c90e00640 1 -- 172.21.15.39:0/2809007243 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c8c153820 con 0x7f4c8c0cfa20 2024-09-21T07:49:35.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.403+0000 7f4c83400640 1 -- 172.21.15.39:0/2809007243 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4c7c01a070 con 0x7f4c8c0cfa20 2024-09-21T07:49:35.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.404+0000 7f4c83400640 1 --2- 172.21.15.39:0/2809007243 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4c68078b90 0x7f4c6807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:35.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.404+0000 7f4c83400640 1 -- 172.21.15.39:0/2809007243 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(232..232 src has 1..232) ==== 7016+0+0 (secure 0 0 0) 0x7f4c7c09b3b0 con 0x7f4c8c0cfa20 2024-09-21T07:49:35.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.404+0000 7f4c83e00640 1 --2- 172.21.15.39:0/2809007243 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4c68078b90 0x7f4c6807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:35.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.405+0000 7f4c83e00640 1 --2- 172.21.15.39:0/2809007243 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4c68078b90 0x7f4c6807b0a0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f4c74033000 tx=0x7f4c74004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:35.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.408+0000 7f4c83400640 1 -- 172.21.15.39:0/2809007243 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4c8c153820 con 0x7f4c8c0cfa20 2024-09-21T07:49:35.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.604+0000 7f4c90e00640 1 -- 172.21.15.39:0/2809007243 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4c8c0d08c0 con 0x7f4c8c0cfa20 2024-09-21T07:49:35.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.606+0000 7f4c83400640 1 -- 172.21.15.39:0/2809007243 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v232) ==== 74+0+22496 (secure 0 0 0) 0x7f4c7c062c80 con 0x7f4c8c0cfa20 2024-09-21T07:49:35.607 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:49:35.607 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":232,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:49:34.636661+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:49:32.570124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"232","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-09-21T07:49:33.592246+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"232","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]},{"pool":20,"snaps":[{"begin":80,"length":23},{"begin":181,"length":23}]},{"pool":22,"snaps":[{"begin":100,"length":1},{"begin":199,"length":1}]},{"pool":24,"snaps":[{"begin":2,"length":2}]},{"pool":26,"snaps":[{"begin":2,"length":2}]},{"pool":28,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:49:35.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.608+0000 7f4c90e00640 1 -- 172.21.15.39:0/2809007243 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4c68078b90 msgr2=0x7f4c6807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:35.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.608+0000 7f4c90e00640 1 --2- 172.21.15.39:0/2809007243 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4c68078b90 0x7f4c6807b0a0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f4c74033000 tx=0x7f4c74004730 comp rx=0 tx=0).stop 2024-09-21T07:49:35.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.608+0000 7f4c90e00640 1 -- 172.21.15.39:0/2809007243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c8c0cfa20 msgr2=0x7f4c8c155d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:35.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.608+0000 7f4c90e00640 1 --2- 172.21.15.39:0/2809007243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c8c0cfa20 0x7f4c8c155d80 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c00d770 tx=0x7f4c7c00dc30 comp rx=0 tx=0).stop 2024-09-21T07:49:35.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.608+0000 7f4c90e00640 1 -- 172.21.15.39:0/2809007243 shutdown_connections 2024-09-21T07:49:35.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.608+0000 7f4c90e00640 1 --2- 172.21.15.39:0/2809007243 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4c68078b90 0x7f4c6807b0a0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:35.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.608+0000 7f4c90e00640 1 --2- 172.21.15.39:0/2809007243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c8c15cc00 0x7f4c8c0b15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:35.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.608+0000 7f4c90e00640 1 --2- 172.21.15.39:0/2809007243 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4c8c0d04b0 0x7f4c8c0b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:35.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.608+0000 7f4c90e00640 1 --2- 172.21.15.39:0/2809007243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c8c0cfa20 0x7f4c8c155d80 unknown :-1 s=CLOSED pgs=2221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:35.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.608+0000 7f4c90e00640 1 -- 172.21.15.39:0/2809007243 >> 172.21.15.39:0/2809007243 conn(0x7f4c8c06cfc0 msgr2=0x7f4c8c15b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:35.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.608+0000 7f4c90e00640 1 -- 172.21.15.39:0/2809007243 shutdown_connections 2024-09-21T07:49:35.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.609+0000 7f4c90e00640 1 -- 172.21.15.39:0/2809007243 wait complete. 2024-09-21T07:49:35.634 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:49:35.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.861+0000 7f538fe00640 1 -- 172.21.15.39:0/1632271678 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53880ccc60 msgr2=0x7f53880cd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:35.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.861+0000 7f538fe00640 1 --2- 172.21.15.39:0/1632271678 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53880ccc60 0x7f53880cd070 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f5380005fd0 tx=0x7f538002a970 comp rx=0 tx=0).stop 2024-09-21T07:49:35.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.861+0000 7f538fe00640 1 -- 172.21.15.39:0/1632271678 shutdown_connections 2024-09-21T07:49:35.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.861+0000 7f538fe00640 1 --2- 172.21.15.39:0/1632271678 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f53880cb8a0 0x7f5388156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:35.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.861+0000 7f538fe00640 1 --2- 172.21.15.39:0/1632271678 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53880ccc60 0x7f53880cd070 unknown :-1 s=CLOSED pgs=2222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:35.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.861+0000 7f538fe00640 1 -- 172.21.15.39:0/1632271678 >> 172.21.15.39:0/1632271678 conn(0x7f53880b2460 msgr2=0x7f538806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:35.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.861+0000 7f538fe00640 1 -- 172.21.15.39:0/1632271678 shutdown_connections 2024-09-21T07:49:35.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.861+0000 7f538fe00640 1 -- 172.21.15.39:0/1632271678 wait complete. 2024-09-21T07:49:35.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.864+0000 7f538fe00640 1 Processor -- start 2024-09-21T07:49:35.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.864+0000 7f538fe00640 1 -- start start 2024-09-21T07:49:35.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.865+0000 7f538fe00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f53880b30a0 0x7f53880ca300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:35.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.865+0000 7f538fe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53880cb8a0 0x7f53881f52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:35.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.865+0000 7f538fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53880ccc60 0x7f53880734b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:35.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.865+0000 7f538fe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f53880af410 con 0x7f53880ccc60 2024-09-21T07:49:35.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.865+0000 7f538fe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f53880af290 con 0x7f53880cb8a0 2024-09-21T07:49:35.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.865+0000 7f538fe00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f53880af590 con 0x7f53880b30a0 2024-09-21T07:49:35.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.865+0000 7f538de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53880ccc60 0x7f53880734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:35.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.865+0000 7f538de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53880ccc60 0x7f53880734b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41644/0 (socket says 172.21.15.39:41644) 2024-09-21T07:49:35.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.866+0000 7f538de00640 1 -- 172.21.15.39:0/1404039599 learned_addr learned my addr 172.21.15.39:0/1404039599 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:35.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.866+0000 7f538ca00640 1 --2- 172.21.15.39:0/1404039599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53880cb8a0 0x7f53881f52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:35.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.866+0000 7f538d400640 1 --2- 172.21.15.39:0/1404039599 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f53880b30a0 0x7f53880ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:35.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.866+0000 7f538de00640 1 -- 172.21.15.39:0/1404039599 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f53880b30a0 msgr2=0x7f53880ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:35.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.866+0000 7f538de00640 1 --2- 172.21.15.39:0/1404039599 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f53880b30a0 0x7f53880ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:35.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.866+0000 7f538de00640 1 -- 172.21.15.39:0/1404039599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53880cb8a0 msgr2=0x7f53881f52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:35.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.866+0000 7f538de00640 1 --2- 172.21.15.39:0/1404039599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53880cb8a0 0x7f53881f52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:35.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.866+0000 7f538de00640 1 -- 172.21.15.39:0/1404039599 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53880c75e0 con 0x7f53880ccc60 2024-09-21T07:49:35.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.866+0000 7f538de00640 1 --2- 172.21.15.39:0/1404039599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53880ccc60 0x7f53880734b0 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f5380033040 tx=0x7f5380003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:35.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.867+0000 7f5385600640 1 -- 172.21.15.39:0/1404039599 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5380034040 con 0x7f53880ccc60 2024-09-21T07:49:35.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.867+0000 7f5385600640 1 -- 172.21.15.39:0/1404039599 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5380004540 con 0x7f53880ccc60 2024-09-21T07:49:35.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.867+0000 7f5385600640 1 -- 172.21.15.39:0/1404039599 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5380045480 con 0x7f53880ccc60 2024-09-21T07:49:35.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.867+0000 7f538fe00640 1 -- 172.21.15.39:0/1404039599 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53881f5c10 con 0x7f53880ccc60 2024-09-21T07:49:35.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.867+0000 7f538fe00640 1 -- 172.21.15.39:0/1404039599 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53881f6190 con 0x7f53880ccc60 2024-09-21T07:49:35.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.868+0000 7f538fe00640 1 -- 172.21.15.39:0/1404039599 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53880afae0 con 0x7f53880ccc60 2024-09-21T07:49:35.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.869+0000 7f5385600640 1 -- 172.21.15.39:0/1404039599 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5380037020 con 0x7f53880ccc60 2024-09-21T07:49:35.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.870+0000 7f5385600640 1 --2- 172.21.15.39:0/1404039599 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5368078b90 0x7f536807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:35.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.870+0000 7f5385600640 1 -- 172.21.15.39:0/1404039599 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(232..232 src has 1..232) ==== 7016+0+0 (secure 0 0 0) 0x7f53800bcb00 con 0x7f53880ccc60 2024-09-21T07:49:35.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.871+0000 7f538d400640 1 --2- 172.21.15.39:0/1404039599 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5368078b90 0x7f536807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:35.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.871+0000 7f538d400640 1 --2- 172.21.15.39:0/1404039599 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5368078b90 0x7f536807b0a0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f537c006fd0 tx=0x7f537c009060 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:35.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:35.875+0000 7f5385600640 1 -- 172.21.15.39:0/1404039599 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f53880cd5b0 con 0x7f53880ccc60 2024-09-21T07:49:36.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.077+0000 7f538fe00640 1 -- 172.21.15.39:0/1404039599 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f538806d030 con 0x7f53880ccc60 2024-09-21T07:49:36.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.078+0000 7f5385600640 1 -- 172.21.15.39:0/1404039599 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v232) ==== 74+0+22496 (secure 0 0 0) 0x7f5380084330 con 0x7f53880ccc60 2024-09-21T07:49:36.080 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:49:36.080 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":232,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:49:34.636661+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:49:32.570124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"232","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-09-21T07:49:33.592246+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"232","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]},{"pool":20,"snaps":[{"begin":80,"length":23},{"begin":181,"length":23}]},{"pool":22,"snaps":[{"begin":100,"length":1},{"begin":199,"length":1}]},{"pool":24,"snaps":[{"begin":2,"length":2}]},{"pool":26,"snaps":[{"begin":2,"length":2}]},{"pool":28,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:49:36.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.081+0000 7f538fe00640 1 -- 172.21.15.39:0/1404039599 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5368078b90 msgr2=0x7f536807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:36.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.081+0000 7f538fe00640 1 --2- 172.21.15.39:0/1404039599 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5368078b90 0x7f536807b0a0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f537c006fd0 tx=0x7f537c009060 comp rx=0 tx=0).stop 2024-09-21T07:49:36.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.081+0000 7f538fe00640 1 -- 172.21.15.39:0/1404039599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53880ccc60 msgr2=0x7f53880734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:36.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.081+0000 7f538fe00640 1 --2- 172.21.15.39:0/1404039599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53880ccc60 0x7f53880734b0 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f5380033040 tx=0x7f5380003f10 comp rx=0 tx=0).stop 2024-09-21T07:49:36.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.081+0000 7f538fe00640 1 -- 172.21.15.39:0/1404039599 shutdown_connections 2024-09-21T07:49:36.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.081+0000 7f538fe00640 1 --2- 172.21.15.39:0/1404039599 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5368078b90 0x7f536807b0a0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:36.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.081+0000 7f538fe00640 1 --2- 172.21.15.39:0/1404039599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53880ccc60 0x7f53880734b0 unknown :-1 s=CLOSED pgs=2223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:36.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.081+0000 7f538fe00640 1 --2- 172.21.15.39:0/1404039599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53880cb8a0 0x7f53881f52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:36.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.081+0000 7f538fe00640 1 --2- 172.21.15.39:0/1404039599 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f53880b30a0 0x7f53880ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:36.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.081+0000 7f538fe00640 1 -- 172.21.15.39:0/1404039599 >> 172.21.15.39:0/1404039599 conn(0x7f53880b2460 msgr2=0x7f53880747e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:36.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.081+0000 7f538fe00640 1 -- 172.21.15.39:0/1404039599 shutdown_connections 2024-09-21T07:49:36.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.081+0000 7f538fe00640 1 -- 172.21.15.39:0/1404039599 wait complete. 2024-09-21T07:49:36.105 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T07:49:36.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.338+0000 7fcdf1200640 1 -- 172.21.15.39:0/2339960291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcdec070bc0 msgr2=0x7fcdec0d06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:36.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.338+0000 7fcdf1200640 1 --2- 172.21.15.39:0/2339960291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcdec070bc0 0x7fcdec0d06b0 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7fcddc005fd0 tx=0x7fcddc02a530 comp rx=0 tx=0).stop 2024-09-21T07:49:36.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.338+0000 7fcdf1200640 1 -- 172.21.15.39:0/2339960291 shutdown_connections 2024-09-21T07:49:36.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.338+0000 7fcdf1200640 1 --2- 172.21.15.39:0/2339960291 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcdec0d0bf0 0x7fcdec0cf450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:36.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.338+0000 7fcdf1200640 1 --2- 172.21.15.39:0/2339960291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcdec070bc0 0x7fcdec0d06b0 unknown :-1 s=CLOSED pgs=2224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:36.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.338+0000 7fcdf1200640 1 -- 172.21.15.39:0/2339960291 >> 172.21.15.39:0/2339960291 conn(0x7fcdec06cfc0 msgr2=0x7fcdec06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:36.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.338+0000 7fcdf1200640 1 -- 172.21.15.39:0/2339960291 shutdown_connections 2024-09-21T07:49:36.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.338+0000 7fcdf1200640 1 -- 172.21.15.39:0/2339960291 wait complete. 2024-09-21T07:49:36.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.340+0000 7fcdf1200640 1 Processor -- start 2024-09-21T07:49:36.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.340+0000 7fcdf1200640 1 -- start start 2024-09-21T07:49:36.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.342+0000 7fcdf1200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcdec070bc0 0x7fcdec13a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:36.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.342+0000 7fcdf1200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdec0cf990 0x7fcdec1e5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:36.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.342+0000 7fcdf1200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcdec0d0bf0 0x7fcdec1e6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:36.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.342+0000 7fcdf1200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec0cffb0 con 0x7fcdec0d0bf0 2024-09-21T07:49:36.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.342+0000 7fcdf1200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec160a30 con 0x7fcdec0cf990 2024-09-21T07:49:36.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.342+0000 7fcdf1200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fcdec14d440 con 0x7fcdec070bc0 2024-09-21T07:49:36.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.342+0000 7fcdea800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcdec0d0bf0 0x7fcdec1e6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:36.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.342+0000 7fcdea800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcdec0d0bf0 0x7fcdec1e6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41666/0 (socket says 172.21.15.39:41666) 2024-09-21T07:49:36.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.342+0000 7fcdea800640 1 -- 172.21.15.39:0/2728504419 learned_addr learned my addr 172.21.15.39:0/2728504419 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:36.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.342+0000 7fcde9e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcdec070bc0 0x7fcdec13a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:36.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.342+0000 7fcde1400640 1 --2- 172.21.15.39:0/2728504419 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdec0cf990 0x7fcdec1e5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:36.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.342+0000 7fcdea800640 1 -- 172.21.15.39:0/2728504419 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcdec070bc0 msgr2=0x7fcdec13a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:36.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.342+0000 7fcdea800640 1 --2- 172.21.15.39:0/2728504419 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcdec070bc0 0x7fcdec13a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:36.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.342+0000 7fcdea800640 1 -- 172.21.15.39:0/2728504419 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdec0cf990 msgr2=0x7fcdec1e5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:36.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.342+0000 7fcdea800640 1 --2- 172.21.15.39:0/2728504419 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdec0cf990 0x7fcdec1e5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:36.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.343+0000 7fcdea800640 1 -- 172.21.15.39:0/2728504419 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdec14f070 con 0x7fcdec0d0bf0 2024-09-21T07:49:36.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.343+0000 7fcdea800640 1 --2- 172.21.15.39:0/2728504419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcdec0d0bf0 0x7fcdec1e6410 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7fcde400b990 tx=0x7fcde400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:36.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.343+0000 7fcde2a00640 1 -- 172.21.15.39:0/2728504419 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde4019070 con 0x7fcdec0d0bf0 2024-09-21T07:49:36.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.343+0000 7fcde2a00640 1 -- 172.21.15.39:0/2728504419 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcde400cc90 con 0x7fcdec0d0bf0 2024-09-21T07:49:36.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.343+0000 7fcde2a00640 1 -- 172.21.15.39:0/2728504419 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde40146a0 con 0x7fcdec0d0bf0 2024-09-21T07:49:36.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.343+0000 7fcdf1200640 1 -- 172.21.15.39:0/2728504419 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdec14f310 con 0x7fcdec0d0bf0 2024-09-21T07:49:36.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.343+0000 7fcdf1200640 1 -- 172.21.15.39:0/2728504419 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdec14f8d0 con 0x7fcdec0d0bf0 2024-09-21T07:49:36.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.344+0000 7fcdf1200640 1 -- 172.21.15.39:0/2728504419 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdec160fb0 con 0x7fcdec0d0bf0 2024-09-21T07:49:36.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.346+0000 7fcde2a00640 1 -- 172.21.15.39:0/2728504419 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcde401c070 con 0x7fcdec0d0bf0 2024-09-21T07:49:36.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.347+0000 7fcde2a00640 1 --2- 172.21.15.39:0/2728504419 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcdc8078b90 0x7fcdc807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:36.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.347+0000 7fcde2a00640 1 -- 172.21.15.39:0/2728504419 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(233..233 src has 1..233) ==== 7016+0+0 (secure 0 0 0) 0x7fcde409afe0 con 0x7fcdec0d0bf0 2024-09-21T07:49:36.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.347+0000 7fcde9e00640 1 --2- 172.21.15.39:0/2728504419 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcdc8078b90 0x7fcdc807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:36.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.348+0000 7fcde9e00640 1 --2- 172.21.15.39:0/2728504419 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcdc8078b90 0x7fcdc807b0a0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fcddc033000 tx=0x7fcddc004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:36.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.351+0000 7fcde2a00640 1 -- 172.21.15.39:0/2728504419 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcde4062870 con 0x7fcdec0d0bf0 2024-09-21T07:49:36.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:36.587+0000 7fcdf1200640 1 -- 172.21.15.39:0/2728504419 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fcdec069320 con 0x7fcdec0d0bf0 2024-09-21T07:49:37.079 INFO:tasks.ceph.mds.c.smithi039.stderr:2024-09-21T07:49:37.078+0000 7f7ede400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:49:37.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.150+0000 7fcde2a00640 1 -- 172.21.15.39:0/2728504419 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v243) ==== 118+0+0 (secure 0 0 0) 0x7fcde40672a0 con 0x7fcdec0d0bf0 2024-09-21T07:49:37.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.153+0000 7fcdf1200640 1 -- 172.21.15.39:0/2728504419 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcdc8078b90 msgr2=0x7fcdc807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:37.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.153+0000 7fcdf1200640 1 --2- 172.21.15.39:0/2728504419 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcdc8078b90 0x7fcdc807b0a0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fcddc033000 tx=0x7fcddc004730 comp rx=0 tx=0).stop 2024-09-21T07:49:37.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.153+0000 7fcdf1200640 1 -- 172.21.15.39:0/2728504419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcdec0d0bf0 msgr2=0x7fcdec1e6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:37.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.153+0000 7fcdf1200640 1 --2- 172.21.15.39:0/2728504419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcdec0d0bf0 0x7fcdec1e6410 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7fcde400b990 tx=0x7fcde400be50 comp rx=0 tx=0).stop 2024-09-21T07:49:37.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.153+0000 7fcdf1200640 1 -- 172.21.15.39:0/2728504419 shutdown_connections 2024-09-21T07:49:37.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.153+0000 7fcdf1200640 1 --2- 172.21.15.39:0/2728504419 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcdc8078b90 0x7fcdc807b0a0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:37.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.153+0000 7fcdf1200640 1 --2- 172.21.15.39:0/2728504419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcdec0d0bf0 0x7fcdec1e6410 unknown :-1 s=CLOSED pgs=2225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:37.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.153+0000 7fcdf1200640 1 --2- 172.21.15.39:0/2728504419 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdec0cf990 0x7fcdec1e5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:37.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.153+0000 7fcdf1200640 1 --2- 172.21.15.39:0/2728504419 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcdec070bc0 0x7fcdec13a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:37.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.153+0000 7fcdf1200640 1 -- 172.21.15.39:0/2728504419 >> 172.21.15.39:0/2728504419 conn(0x7fcdec06cfc0 msgr2=0x7fcdec06fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:37.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.153+0000 7fcdf1200640 1 -- 172.21.15.39:0/2728504419 shutdown_connections 2024-09-21T07:49:37.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.153+0000 7fcdf1200640 1 -- 172.21.15.39:0/2728504419 wait complete. 2024-09-21T07:49:37.179 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T07:49:37.179 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T07:49:37.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.409+0000 7ff30a200640 1 -- 172.21.15.39:0/779575692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3040d0390 msgr2=0x7ff3040d07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:37.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.409+0000 7ff30a200640 1 --2- 172.21.15.39:0/779575692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3040d0390 0x7ff3040d07a0 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc005fd0 tx=0x7ff2fc02a900 comp rx=0 tx=0).stop 2024-09-21T07:49:37.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.409+0000 7ff30a200640 1 -- 172.21.15.39:0/779575692 shutdown_connections 2024-09-21T07:49:37.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.409+0000 7ff30a200640 1 --2- 172.21.15.39:0/779575692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3040d0390 0x7ff3040d07a0 unknown :-1 s=CLOSED pgs=2227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:37.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.409+0000 7ff30a200640 1 --2- 172.21.15.39:0/779575692 >> v2:172.21.15.148:3300/0 conn(0x7ff304160250 0x7ff30415d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:37.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.409+0000 7ff30a200640 1 -- 172.21.15.39:0/779575692 >> 172.21.15.39:0/779575692 conn(0x7ff30406cfc0 msgr2=0x7ff30406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:37.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.409+0000 7ff30a200640 1 -- 172.21.15.39:0/779575692 shutdown_connections 2024-09-21T07:49:37.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.409+0000 7ff30a200640 1 -- 172.21.15.39:0/779575692 wait complete. 2024-09-21T07:49:37.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.411+0000 7ff30a200640 1 Processor -- start 2024-09-21T07:49:37.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.411+0000 7ff30a200640 1 -- start start 2024-09-21T07:49:37.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.413+0000 7ff30a200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3040d0390 0x7ff30415c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:37.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.413+0000 7ff30a200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff304160250 0x7ff3041f10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:37.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.413+0000 7ff30a200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff304160660 0x7ff304151170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:37.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.413+0000 7ff30a200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff3040ccdc0 con 0x7ff3040d0390 2024-09-21T07:49:37.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.413+0000 7ff30a200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff3040ccc40 con 0x7ff304160250 2024-09-21T07:49:37.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.413+0000 7ff30a200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7ff3040ccf40 con 0x7ff304160660 2024-09-21T07:49:37.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.413+0000 7ff308e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3040d0390 0x7ff30415c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:37.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.413+0000 7ff308e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3040d0390 0x7ff30415c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41692/0 (socket says 172.21.15.39:41692) 2024-09-21T07:49:37.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.413+0000 7ff308e00640 1 -- 172.21.15.39:0/3044842652 learned_addr learned my addr 172.21.15.39:0/3044842652 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:37.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.413+0000 7ff308e00640 1 -- 172.21.15.39:0/3044842652 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff304160660 msgr2=0x7ff304151170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:37.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.413+0000 7ff308e00640 1 --2- 172.21.15.39:0/3044842652 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff304160660 0x7ff304151170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:37.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.413+0000 7ff308e00640 1 -- 172.21.15.39:0/3044842652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff304160250 msgr2=0x7ff3041f10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:37.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.413+0000 7ff303e00640 1 --2- 172.21.15.39:0/3044842652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff304160250 0x7ff3041f10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:37.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.413+0000 7ff308e00640 1 --2- 172.21.15.39:0/3044842652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff304160250 0x7ff3041f10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:37.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.413+0000 7ff308e00640 1 -- 172.21.15.39:0/3044842652 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3041f3d00 con 0x7ff3040d0390 2024-09-21T07:49:37.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.413+0000 7ff303e00640 1 --2- 172.21.15.39:0/3044842652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff304160250 0x7ff3041f10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:49:37.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.414+0000 7ff308e00640 1 --2- 172.21.15.39:0/3044842652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3040d0390 0x7ff30415c220 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7ff2f800e790 tx=0x7ff2f800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:37.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.414+0000 7ff300c00640 1 -- 172.21.15.39:0/3044842652 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f800eea0 con 0x7ff3040d0390 2024-09-21T07:49:37.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.414+0000 7ff300c00640 1 -- 172.21.15.39:0/3044842652 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2f800cdb0 con 0x7ff3040d0390 2024-09-21T07:49:37.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.414+0000 7ff300c00640 1 -- 172.21.15.39:0/3044842652 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f8010690 con 0x7ff3040d0390 2024-09-21T07:49:37.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.414+0000 7ff30a200640 1 -- 172.21.15.39:0/3044842652 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff30415c760 con 0x7ff3040d0390 2024-09-21T07:49:37.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.414+0000 7ff30a200640 1 -- 172.21.15.39:0/3044842652 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3041f42a0 con 0x7ff3040d0390 2024-09-21T07:49:37.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.415+0000 7ff30a200640 1 -- 172.21.15.39:0/3044842652 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3040696c0 con 0x7ff3040d0390 2024-09-21T07:49:37.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.416+0000 7ff300c00640 1 -- 172.21.15.39:0/3044842652 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff2f8010830 con 0x7ff3040d0390 2024-09-21T07:49:37.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.417+0000 7ff300c00640 1 --2- 172.21.15.39:0/3044842652 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff2e4078b90 0x7ff2e407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:37.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.418+0000 7ff300c00640 1 -- 172.21.15.39:0/3044842652 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(233..233 src has 1..233) ==== 7016+0+0 (secure 0 0 0) 0x7ff2f809bcb0 con 0x7ff3040d0390 2024-09-21T07:49:37.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.418+0000 7ff303e00640 1 --2- 172.21.15.39:0/3044842652 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff2e4078b90 0x7ff2e407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:37.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.419+0000 7ff303e00640 1 --2- 172.21.15.39:0/3044842652 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff2e4078b90 0x7ff2e407b0a0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7ff2f00013f0 tx=0x7ff2f0009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:37.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.422+0000 7ff300c00640 1 -- 172.21.15.39:0/3044842652 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff3040696c0 con 0x7ff3040d0390 2024-09-21T07:49:37.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:37.655+0000 7ff30a200640 1 -- 172.21.15.39:0/3044842652 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7ff30406a3b0 con 0x7ff3040d0390 2024-09-21T07:49:38.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.158+0000 7ff300c00640 1 -- 172.21.15.39:0/3044842652 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v244) ==== 122+0+0 (secure 0 0 0) 0x7ff30406a3b0 con 0x7ff3040d0390 2024-09-21T07:49:38.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.160+0000 7ff30a200640 1 -- 172.21.15.39:0/3044842652 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff2e4078b90 msgr2=0x7ff2e407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:38.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.160+0000 7ff30a200640 1 --2- 172.21.15.39:0/3044842652 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff2e4078b90 0x7ff2e407b0a0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7ff2f00013f0 tx=0x7ff2f0009040 comp rx=0 tx=0).stop 2024-09-21T07:49:38.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.160+0000 7ff30a200640 1 -- 172.21.15.39:0/3044842652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3040d0390 msgr2=0x7ff30415c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:38.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.160+0000 7ff30a200640 1 --2- 172.21.15.39:0/3044842652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3040d0390 0x7ff30415c220 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7ff2f800e790 tx=0x7ff2f800ec50 comp rx=0 tx=0).stop 2024-09-21T07:49:38.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.160+0000 7ff30a200640 1 -- 172.21.15.39:0/3044842652 shutdown_connections 2024-09-21T07:49:38.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.160+0000 7ff30a200640 1 --2- 172.21.15.39:0/3044842652 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff2e4078b90 0x7ff2e407b0a0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:38.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.160+0000 7ff30a200640 1 --2- 172.21.15.39:0/3044842652 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff304160660 0x7ff304151170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:38.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.160+0000 7ff30a200640 1 --2- 172.21.15.39:0/3044842652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff304160250 0x7ff3041f10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:38.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.160+0000 7ff30a200640 1 --2- 172.21.15.39:0/3044842652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3040d0390 0x7ff30415c220 unknown :-1 s=CLOSED pgs=2228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:38.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.160+0000 7ff30a200640 1 -- 172.21.15.39:0/3044842652 >> 172.21.15.39:0/3044842652 conn(0x7ff30406cfc0 msgr2=0x7ff304155350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:38.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.161+0000 7ff30a200640 1 -- 172.21.15.39:0/3044842652 shutdown_connections 2024-09-21T07:49:38.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.161+0000 7ff30a200640 1 -- 172.21.15.39:0/3044842652 wait complete. 2024-09-21T07:49:38.185 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T07:49:38.185 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:49:38.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.410+0000 7f9c6d600640 1 -- 172.21.15.39:0/136836611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c680ccc60 msgr2=0x7f9c680cd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:38.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.410+0000 7f9c6d600640 1 --2- 172.21.15.39:0/136836611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c680ccc60 0x7f9c680cd070 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f9c68075160 tx=0x7f9c6002aa30 comp rx=0 tx=0).stop 2024-09-21T07:49:38.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.410+0000 7f9c6d600640 1 -- 172.21.15.39:0/136836611 shutdown_connections 2024-09-21T07:49:38.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.410+0000 7f9c6d600640 1 --2- 172.21.15.39:0/136836611 >> v2:172.21.15.148:3300/0 conn(0x7f9c680cb8a0 0x7f9c68156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:38.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.410+0000 7f9c6d600640 1 --2- 172.21.15.39:0/136836611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c680ccc60 0x7f9c680cd070 unknown :-1 s=CLOSED pgs=2229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:38.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.410+0000 7f9c6d600640 1 -- 172.21.15.39:0/136836611 >> 172.21.15.39:0/136836611 conn(0x7f9c680b2460 msgr2=0x7f9c6806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:38.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.410+0000 7f9c6d600640 1 -- 172.21.15.39:0/136836611 shutdown_connections 2024-09-21T07:49:38.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.411+0000 7f9c6d600640 1 -- 172.21.15.39:0/136836611 wait complete. 2024-09-21T07:49:38.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.413+0000 7f9c6d600640 1 Processor -- start 2024-09-21T07:49:38.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.413+0000 7f9c6d600640 1 -- start start 2024-09-21T07:49:38.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.414+0000 7f9c6d600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c680b30a0 0x7f9c681f6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:38.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.414+0000 7f9c6d600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9c680cb8a0 0x7f9c681e93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:38.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.414+0000 7f9c6d600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c680ccc60 0x7f9c680cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:38.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.414+0000 7f9c6d600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9c680af410 con 0x7f9c680ccc60 2024-09-21T07:49:38.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.414+0000 7f9c6d600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9c680af290 con 0x7f9c680b30a0 2024-09-21T07:49:38.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.414+0000 7f9c6d600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f9c680af590 con 0x7f9c680cb8a0 2024-09-21T07:49:38.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.415+0000 7f9c67400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c680ccc60 0x7f9c680cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:38.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.415+0000 7f9c67400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c680ccc60 0x7f9c680cdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41712/0 (socket says 172.21.15.39:41712) 2024-09-21T07:49:38.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.415+0000 7f9c67400640 1 -- 172.21.15.39:0/2777425558 learned_addr learned my addr 172.21.15.39:0/2777425558 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:38.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.415+0000 7f9c66000640 1 --2- 172.21.15.39:0/2777425558 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9c680cb8a0 0x7f9c681e93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:38.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.415+0000 7f9c66a00640 1 --2- 172.21.15.39:0/2777425558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c680b30a0 0x7f9c681f6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:38.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.415+0000 7f9c67400640 1 -- 172.21.15.39:0/2777425558 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9c680cb8a0 msgr2=0x7f9c681e93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:38.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.415+0000 7f9c67400640 1 --2- 172.21.15.39:0/2777425558 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9c680cb8a0 0x7f9c681e93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:38.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.415+0000 7f9c67400640 1 -- 172.21.15.39:0/2777425558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c680b30a0 msgr2=0x7f9c681f6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:38.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.415+0000 7f9c67400640 1 --2- 172.21.15.39:0/2777425558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c680b30a0 0x7f9c681f6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:38.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.415+0000 7f9c67400640 1 -- 172.21.15.39:0/2777425558 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c680c6970 con 0x7f9c680ccc60 2024-09-21T07:49:38.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.415+0000 7f9c66000640 1 --2- 172.21.15.39:0/2777425558 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9c680cb8a0 0x7f9c681e93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:49:38.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.415+0000 7f9c66a00640 1 --2- 172.21.15.39:0/2777425558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c680b30a0 0x7f9c681f6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:49:38.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.415+0000 7f9c67400640 1 --2- 172.21.15.39:0/2777425558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c680ccc60 0x7f9c680cdf80 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7f9c6002b020 tx=0x7f9c60003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:38.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.416+0000 7f9c5ea00640 1 -- 172.21.15.39:0/2777425558 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c60033050 con 0x7f9c680ccc60 2024-09-21T07:49:38.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.416+0000 7f9c5ea00640 1 -- 172.21.15.39:0/2777425558 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c60004450 con 0x7f9c680ccc60 2024-09-21T07:49:38.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.416+0000 7f9c5ea00640 1 -- 172.21.15.39:0/2777425558 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c60004840 con 0x7f9c680ccc60 2024-09-21T07:49:38.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.416+0000 7f9c6d600640 1 -- 172.21.15.39:0/2777425558 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c680c6c50 con 0x7f9c680ccc60 2024-09-21T07:49:38.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.416+0000 7f9c6d600640 1 -- 172.21.15.39:0/2777425558 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c680ce9e0 con 0x7f9c680ccc60 2024-09-21T07:49:38.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.417+0000 7f9c6d600640 1 -- 172.21.15.39:0/2777425558 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c680c66e0 con 0x7f9c680ccc60 2024-09-21T07:49:38.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.418+0000 7f9c5ea00640 1 -- 172.21.15.39:0/2777425558 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9c60036070 con 0x7f9c680ccc60 2024-09-21T07:49:38.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.419+0000 7f9c5ea00640 1 --2- 172.21.15.39:0/2777425558 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9c48078b90 0x7f9c4807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:38.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.419+0000 7f9c5ea00640 1 -- 172.21.15.39:0/2777425558 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(233..233 src has 1..233) ==== 7016+0+0 (secure 0 0 0) 0x7f9c600bc810 con 0x7f9c680ccc60 2024-09-21T07:49:38.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.419+0000 7f9c66a00640 1 --2- 172.21.15.39:0/2777425558 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9c48078b90 0x7f9c4807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:38.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.420+0000 7f9c66a00640 1 --2- 172.21.15.39:0/2777425558 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9c48078b90 0x7f9c4807b0a0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f9c580016a0 tx=0x7f9c58009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:38.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.424+0000 7f9c5ea00640 1 -- 172.21.15.39:0/2777425558 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9c60084720 con 0x7f9c680ccc60 2024-09-21T07:49:38.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.659+0000 7f9c6d600640 1 -- 172.21.15.39:0/2777425558 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c680cec40 con 0x7f9c680ccc60 2024-09-21T07:49:38.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.660+0000 7f9c5ea00640 1 -- 172.21.15.39:0/2777425558 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 244 v244) ==== 95+0+5245 (secure 0 0 0) 0x7f9c60084020 con 0x7f9c680ccc60 2024-09-21T07:49:38.662 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:49:38.662 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":244,"btime":"2024-09-21T07:49:38:150708+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8531,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1438851059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1438851059},{"type":"v1","addr":"172.21.15.148:6837","nonce":1438851059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":239},{"gid":9709,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3497801035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3497801035},{"type":"v1","addr":"172.21.15.148:6836","nonce":3497801035}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":239},{"gid":10632,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/4143807732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4143807732},{"type":"v1","addr":"172.21.15.39:6834","nonce":4143807732}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":238}],"filesystems":[{"mdsmap":{"epoch":244,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:49:37.657329+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10635},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10635":{"gid":10635,"name":"c","rank":0,"incarnation":240,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/2824493631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2824493631},{"type":"v1","addr":"172.21.15.39:6835","nonce":2824493631}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10635,"qdb_cluster":[10635]},"id":13}]} 2024-09-21T07:49:38.663 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 244 2024-09-21T07:49:38.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.663+0000 7f9c6d600640 1 -- 172.21.15.39:0/2777425558 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9c48078b90 msgr2=0x7f9c4807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:38.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.663+0000 7f9c6d600640 1 --2- 172.21.15.39:0/2777425558 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9c48078b90 0x7f9c4807b0a0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f9c580016a0 tx=0x7f9c58009040 comp rx=0 tx=0).stop 2024-09-21T07:49:38.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.663+0000 7f9c6d600640 1 -- 172.21.15.39:0/2777425558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c680ccc60 msgr2=0x7f9c680cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:38.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.663+0000 7f9c6d600640 1 --2- 172.21.15.39:0/2777425558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c680ccc60 0x7f9c680cdf80 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7f9c6002b020 tx=0x7f9c60003f10 comp rx=0 tx=0).stop 2024-09-21T07:49:38.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.663+0000 7f9c6d600640 1 -- 172.21.15.39:0/2777425558 shutdown_connections 2024-09-21T07:49:38.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.663+0000 7f9c6d600640 1 --2- 172.21.15.39:0/2777425558 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9c48078b90 0x7f9c4807b0a0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:38.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.663+0000 7f9c6d600640 1 --2- 172.21.15.39:0/2777425558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c680ccc60 0x7f9c680cdf80 unknown :-1 s=CLOSED pgs=2230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:38.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.663+0000 7f9c6d600640 1 --2- 172.21.15.39:0/2777425558 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9c680cb8a0 0x7f9c681e93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:38.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.663+0000 7f9c6d600640 1 --2- 172.21.15.39:0/2777425558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c680b30a0 0x7f9c681f6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:38.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.663+0000 7f9c6d600640 1 -- 172.21.15.39:0/2777425558 >> 172.21.15.39:0/2777425558 conn(0x7f9c680b2460 msgr2=0x7f9c680b34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:38.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.663+0000 7f9c6d600640 1 -- 172.21.15.39:0/2777425558 shutdown_connections 2024-09-21T07:49:38.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.663+0000 7f9c6d600640 1 -- 172.21.15.39:0/2777425558 wait complete. 2024-09-21T07:49:38.690 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:49:38.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.916+0000 7f5384a00640 1 -- 172.21.15.39:0/2460821245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53800d0390 msgr2=0x7f53800d07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:38.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.916+0000 7f5384a00640 1 --2- 172.21.15.39:0/2460821245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53800d0390 0x7f53800d07a0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f5378005fd0 tx=0x7f537802a8b0 comp rx=0 tx=0).stop 2024-09-21T07:49:38.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.916+0000 7f5384a00640 1 -- 172.21.15.39:0/2460821245 shutdown_connections 2024-09-21T07:49:38.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.916+0000 7f5384a00640 1 --2- 172.21.15.39:0/2460821245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53800d0390 0x7f53800d07a0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:38.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.916+0000 7f5384a00640 1 --2- 172.21.15.39:0/2460821245 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5380160250 0x7f538015d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:38.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.916+0000 7f5384a00640 1 -- 172.21.15.39:0/2460821245 >> 172.21.15.39:0/2460821245 conn(0x7f538006cfc0 msgr2=0x7f538006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:38.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.916+0000 7f5384a00640 1 -- 172.21.15.39:0/2460821245 shutdown_connections 2024-09-21T07:49:38.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.916+0000 7f5384a00640 1 -- 172.21.15.39:0/2460821245 wait complete. 2024-09-21T07:49:38.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.918+0000 7f5384a00640 1 Processor -- start 2024-09-21T07:49:38.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.919+0000 7f5384a00640 1 -- start start 2024-09-21T07:49:38.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.920+0000 7f5384a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53800d0390 0x7f5380148fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:38.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.920+0000 7f5384a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5380160250 0x7f53801f91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:38.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.920+0000 7f5384a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5380160660 0x7f53801f9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:38.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.920+0000 7f5384a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f53800ccdc0 con 0x7f53800d0390 2024-09-21T07:49:38.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.920+0000 7f5384a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f53800ccc40 con 0x7f5380160660 2024-09-21T07:49:38.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.920+0000 7f5384a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f53800ccf40 con 0x7f5380160250 2024-09-21T07:49:38.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.920+0000 7f537f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53800d0390 0x7f5380148fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:38.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.920+0000 7f537f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53800d0390 0x7f5380148fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41718/0 (socket says 172.21.15.39:41718) 2024-09-21T07:49:38.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.920+0000 7f537f400640 1 -- 172.21.15.39:0/10916776 learned_addr learned my addr 172.21.15.39:0/10916776 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:38.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.921+0000 7f537f400640 1 -- 172.21.15.39:0/10916776 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5380160250 msgr2=0x7f53801f91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:38.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.921+0000 7f5377e00640 1 --2- 172.21.15.39:0/10916776 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5380160250 0x7f53801f91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:38.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.921+0000 7f537fe00640 1 --2- 172.21.15.39:0/10916776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5380160660 0x7f53801f9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:38.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.921+0000 7f537f400640 1 --2- 172.21.15.39:0/10916776 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5380160250 0x7f53801f91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:38.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.921+0000 7f537f400640 1 -- 172.21.15.39:0/10916776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5380160660 msgr2=0x7f53801f9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:38.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.921+0000 7f537f400640 1 --2- 172.21.15.39:0/10916776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5380160660 0x7f53801f9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:38.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.921+0000 7f537f400640 1 -- 172.21.15.39:0/10916776 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53801f6300 con 0x7f53800d0390 2024-09-21T07:49:38.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.921+0000 7f537fe00640 1 --2- 172.21.15.39:0/10916776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5380160660 0x7f53801f9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:49:38.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.921+0000 7f5377e00640 1 --2- 172.21.15.39:0/10916776 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5380160250 0x7f53801f91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:49:38.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.921+0000 7f537f400640 1 --2- 172.21.15.39:0/10916776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53800d0390 0x7f5380148fa0 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f537000e7c0 tx=0x7f537000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:38.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.921+0000 7f5377400640 1 -- 172.21.15.39:0/10916776 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537000ccc0 con 0x7f53800d0390 2024-09-21T07:49:38.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.921+0000 7f5377400640 1 -- 172.21.15.39:0/10916776 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5370010430 con 0x7f53800d0390 2024-09-21T07:49:38.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.922+0000 7f5377400640 1 -- 172.21.15.39:0/10916776 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5370010860 con 0x7f53800d0390 2024-09-21T07:49:38.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.922+0000 7f5384a00640 1 -- 172.21.15.39:0/10916776 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53801f65e0 con 0x7f53800d0390 2024-09-21T07:49:38.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.922+0000 7f5384a00640 1 -- 172.21.15.39:0/10916776 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53801f6ba0 con 0x7f53800d0390 2024-09-21T07:49:38.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.923+0000 7f5384a00640 1 -- 172.21.15.39:0/10916776 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53800cd3f0 con 0x7f53800d0390 2024-09-21T07:49:38.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.924+0000 7f5377400640 1 -- 172.21.15.39:0/10916776 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5370010a00 con 0x7f53800d0390 2024-09-21T07:49:38.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.925+0000 7f5377400640 1 --2- 172.21.15.39:0/10916776 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f535c078b90 0x7f535c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:38.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.925+0000 7f5377400640 1 -- 172.21.15.39:0/10916776 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(233..233 src has 1..233) ==== 7016+0+0 (secure 0 0 0) 0x7f537009bc80 con 0x7f53800d0390 2024-09-21T07:49:38.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.926+0000 7f5377e00640 1 --2- 172.21.15.39:0/10916776 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f535c078b90 0x7f535c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:38.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.926+0000 7f5377e00640 1 --2- 172.21.15.39:0/10916776 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f535c078b90 0x7f535c07b0a0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f53680013f0 tx=0x7f5368009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:38.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:38.930+0000 7f5377400640 1 -- 172.21.15.39:0/10916776 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5370063b90 con 0x7f53800d0390 2024-09-21T07:49:39.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.128+0000 7f5384a00640 1 -- 172.21.15.39:0/10916776 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f53800b1090 con 0x7f53800d0390 2024-09-21T07:49:39.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.129+0000 7f5377400640 1 -- 172.21.15.39:0/10916776 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v233) ==== 74+0+22483 (secure 0 0 0) 0x7f5370063490 con 0x7f53800d0390 2024-09-21T07:49:39.132 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:49:39.132 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":233,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:49:36.131586+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:49:32.570124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"232","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-09-21T07:49:33.592246+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]},{"pool":20,"snaps":[{"begin":80,"length":23},{"begin":181,"length":23}]},{"pool":22,"snaps":[{"begin":100,"length":1},{"begin":199,"length":1}]},{"pool":24,"snaps":[{"begin":2,"length":2}]},{"pool":26,"snaps":[{"begin":2,"length":2}]},{"pool":28,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:49:39.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.132+0000 7f5384a00640 1 -- 172.21.15.39:0/10916776 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f535c078b90 msgr2=0x7f535c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:39.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.132+0000 7f5384a00640 1 --2- 172.21.15.39:0/10916776 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f535c078b90 0x7f535c07b0a0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f53680013f0 tx=0x7f5368009040 comp rx=0 tx=0).stop 2024-09-21T07:49:39.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.132+0000 7f5384a00640 1 -- 172.21.15.39:0/10916776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53800d0390 msgr2=0x7f5380148fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:39.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.132+0000 7f5384a00640 1 --2- 172.21.15.39:0/10916776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53800d0390 0x7f5380148fa0 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f537000e7c0 tx=0x7f537000ec80 comp rx=0 tx=0).stop 2024-09-21T07:49:39.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.132+0000 7f5384a00640 1 -- 172.21.15.39:0/10916776 shutdown_connections 2024-09-21T07:49:39.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.132+0000 7f5384a00640 1 --2- 172.21.15.39:0/10916776 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f535c078b90 0x7f535c07b0a0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:39.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.132+0000 7f5384a00640 1 --2- 172.21.15.39:0/10916776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5380160660 0x7f53801f9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:39.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.132+0000 7f5384a00640 1 --2- 172.21.15.39:0/10916776 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5380160250 0x7f53801f91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:39.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.132+0000 7f5384a00640 1 --2- 172.21.15.39:0/10916776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53800d0390 0x7f5380148fa0 unknown :-1 s=CLOSED pgs=2232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:39.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.132+0000 7f5384a00640 1 -- 172.21.15.39:0/10916776 >> 172.21.15.39:0/10916776 conn(0x7f538006cfc0 msgr2=0x7f538006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:39.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.132+0000 7f5384a00640 1 -- 172.21.15.39:0/10916776 shutdown_connections 2024-09-21T07:49:39.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.132+0000 7f5384a00640 1 -- 172.21.15.39:0/10916776 wait complete. 2024-09-21T07:49:39.158 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T07:49:39.158 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:49:39.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.384+0000 7f1921a00640 1 -- 172.21.15.39:0/3329980361 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f191c0d0bc0 msgr2=0x7f191c1554a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:39.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.384+0000 7f1921a00640 1 --2- 172.21.15.39:0/3329980361 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f191c0d0bc0 0x7f191c1554a0 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f1914005fd0 tx=0x7f191402aa50 comp rx=0 tx=0).stop 2024-09-21T07:49:39.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.384+0000 7f1921a00640 1 -- 172.21.15.39:0/3329980361 shutdown_connections 2024-09-21T07:49:39.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.384+0000 7f1921a00640 1 --2- 172.21.15.39:0/3329980361 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f191c0d0bc0 0x7f191c1554a0 unknown :-1 s=CLOSED pgs=2234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:39.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.384+0000 7f1921a00640 1 --2- 172.21.15.39:0/3329980361 >> v2:172.21.15.148:3300/0 conn(0x7f191c0b3b50 0x7f191c15c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:39.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.384+0000 7f1921a00640 1 -- 172.21.15.39:0/3329980361 >> 172.21.15.39:0/3329980361 conn(0x7f191c06cfc0 msgr2=0x7f191c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:39.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.384+0000 7f1921a00640 1 -- 172.21.15.39:0/3329980361 shutdown_connections 2024-09-21T07:49:39.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.384+0000 7f1921a00640 1 -- 172.21.15.39:0/3329980361 wait complete. 2024-09-21T07:49:39.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.386+0000 7f1921a00640 1 Processor -- start 2024-09-21T07:49:39.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.386+0000 7f1921a00640 1 -- start start 2024-09-21T07:49:39.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.388+0000 7f1921a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f191c073f90 0x7f191c14abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:39.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.388+0000 7f1921a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f191c0b3b50 0x7f191c0ca510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:39.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.388+0000 7f1921a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f191c0d0bc0 0x7f191c0caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:39.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.388+0000 7f1921a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f191c074520 con 0x7f191c0b3b50 2024-09-21T07:49:39.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.388+0000 7f1921a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f191c0743a0 con 0x7f191c073f90 2024-09-21T07:49:39.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.388+0000 7f1921a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f191c0746a0 con 0x7f191c0d0bc0 2024-09-21T07:49:39.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.388+0000 7f1913e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f191c0b3b50 0x7f191c0ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:39.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.388+0000 7f1913e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f191c0b3b50 0x7f191c0ca510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41736/0 (socket says 172.21.15.39:41736) 2024-09-21T07:49:39.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.388+0000 7f1913e00640 1 -- 172.21.15.39:0/140289559 learned_addr learned my addr 172.21.15.39:0/140289559 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:39.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.388+0000 7f191be00640 1 --2- 172.21.15.39:0/140289559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f191c073f90 0x7f191c14abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:39.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.388+0000 7f1921000640 1 --2- 172.21.15.39:0/140289559 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f191c0d0bc0 0x7f191c0caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:39.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.388+0000 7f1913e00640 1 -- 172.21.15.39:0/140289559 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f191c0d0bc0 msgr2=0x7f191c0caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:39.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.388+0000 7f1913e00640 1 --2- 172.21.15.39:0/140289559 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f191c0d0bc0 0x7f191c0caa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:39.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.388+0000 7f1913e00640 1 -- 172.21.15.39:0/140289559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f191c073f90 msgr2=0x7f191c14abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:39.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.388+0000 7f1913e00640 1 --2- 172.21.15.39:0/140289559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f191c073f90 0x7f191c14abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:39.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.388+0000 7f1913e00640 1 -- 172.21.15.39:0/140289559 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f191c0a9570 con 0x7f191c0b3b50 2024-09-21T07:49:39.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.388+0000 7f191be00640 1 --2- 172.21.15.39:0/140289559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f191c073f90 0x7f191c14abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:39.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.389+0000 7f1913e00640 1 --2- 172.21.15.39:0/140289559 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f191c0b3b50 0x7f191c0ca510 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f190400c370 tx=0x7f190400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:39.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.389+0000 7f1918c00640 1 -- 172.21.15.39:0/140289559 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1904017070 con 0x7f191c0b3b50 2024-09-21T07:49:39.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.389+0000 7f1918c00640 1 -- 172.21.15.39:0/140289559 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19040053c0 con 0x7f191c0b3b50 2024-09-21T07:49:39.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.389+0000 7f1918c00640 1 -- 172.21.15.39:0/140289559 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1904003850 con 0x7f191c0b3b50 2024-09-21T07:49:39.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.389+0000 7f1921a00640 1 -- 172.21.15.39:0/140289559 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f191c156d10 con 0x7f191c0b3b50 2024-09-21T07:49:39.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.389+0000 7f1921a00640 1 -- 172.21.15.39:0/140289559 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f191c157350 con 0x7f191c0b3b50 2024-09-21T07:49:39.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.390+0000 7f1921a00640 1 -- 172.21.15.39:0/140289559 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f191c156a80 con 0x7f191c0b3b50 2024-09-21T07:49:39.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.391+0000 7f1918c00640 1 -- 172.21.15.39:0/140289559 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f190401a070 con 0x7f191c0b3b50 2024-09-21T07:49:39.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.392+0000 7f1918c00640 1 --2- 172.21.15.39:0/140289559 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f18f8078bc0 0x7f18f807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:39.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.392+0000 7f1918c00640 1 -- 172.21.15.39:0/140289559 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(233..233 src has 1..233) ==== 7016+0+0 (secure 0 0 0) 0x7f190409a570 con 0x7f191c0b3b50 2024-09-21T07:49:39.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.393+0000 7f191be00640 1 --2- 172.21.15.39:0/140289559 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f18f8078bc0 0x7f18f807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:39.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.394+0000 7f191be00640 1 --2- 172.21.15.39:0/140289559 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f18f8078bc0 0x7f18f807b0d0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f190c005fd0 tx=0x7f190c00f360 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:39.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.397+0000 7f1918c00640 1 -- 172.21.15.39:0/140289559 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1904062510 con 0x7f191c0b3b50 2024-09-21T07:49:39.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.588+0000 7f1921a00640 1 -- 172.21.15.39:0/140289559 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f191c15c510 con 0x7f18f8078bc0 2024-09-21T07:49:39.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.599+0000 7f1918c00640 1 -- 172.21.15.39:0/140289559 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+495242 (secure 0 0 0) 0x7f191c15c510 con 0x7f18f8078bc0 2024-09-21T07:49:39.607 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:49:39.611 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:49:39.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.605+0000 7f1921a00640 1 -- 172.21.15.39:0/140289559 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f18f8078bc0 msgr2=0x7f18f807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:39.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.605+0000 7f1921a00640 1 --2- 172.21.15.39:0/140289559 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f18f8078bc0 0x7f18f807b0d0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f190c005fd0 tx=0x7f190c00f360 comp rx=0 tx=0).stop 2024-09-21T07:49:39.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.605+0000 7f1921a00640 1 -- 172.21.15.39:0/140289559 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f191c0b3b50 msgr2=0x7f191c0ca510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:39.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.605+0000 7f1921a00640 1 --2- 172.21.15.39:0/140289559 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f191c0b3b50 0x7f191c0ca510 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f190400c370 tx=0x7f190400c830 comp rx=0 tx=0).stop 2024-09-21T07:49:39.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.605+0000 7f1921a00640 1 -- 172.21.15.39:0/140289559 shutdown_connections 2024-09-21T07:49:39.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.605+0000 7f1921a00640 1 --2- 172.21.15.39:0/140289559 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f18f8078bc0 0x7f18f807b0d0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:39.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.605+0000 7f1921a00640 1 --2- 172.21.15.39:0/140289559 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f191c0d0bc0 0x7f191c0caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:39.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.605+0000 7f1921a00640 1 --2- 172.21.15.39:0/140289559 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f191c0b3b50 0x7f191c0ca510 unknown :-1 s=CLOSED pgs=2235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:39.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.605+0000 7f1921a00640 1 --2- 172.21.15.39:0/140289559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f191c073f90 0x7f191c14abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:39.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.605+0000 7f1921a00640 1 -- 172.21.15.39:0/140289559 >> 172.21.15.39:0/140289559 conn(0x7f191c06cfc0 msgr2=0x7f191c15d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:39.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.605+0000 7f1921a00640 1 -- 172.21.15.39:0/140289559 shutdown_connections 2024-09-21T07:49:39.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.605+0000 7f1921a00640 1 -- 172.21.15.39:0/140289559 wait complete. 2024-09-21T07:49:39.633 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":726,"stamp":"2024-09-21T07:49:37.868176+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":314,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":427768,"kb_used_data":24108,"kb_used_omap":12,"kb_used_meta":403443,"kb_avail":754546952,"statfs":{"total":773094113280,"available":772656078848,"internally_reserved":0,"allocated":24686592,"data_stored":13728465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":413126192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.768103"},"pg_stats":[{"pgid":"30.3f","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.266480+0000","last_change":"2024-09-21T07:49:35.154451+0000","last_active":"2024-09-21T07:49:37.266480+0000","last_peered":"2024-09-21T07:49:37.266480+0000","last_clean":"2024-09-21T07:49:37.266480+0000","last_became_active":"2024-09-21T07:49:35.151722+0000","last_became_peered":"2024-09-21T07:49:35.151722+0000","last_unstale":"2024-09-21T07:49:37.266480+0000","last_undegraded":"2024-09-21T07:49:37.266480+0000","last_fullsized":"2024-09-21T07:49:37.266480+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:42:52.483212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.965850+0000","last_change":"2024-09-21T07:49:35.152094+0000","last_active":"2024-09-21T07:49:36.965850+0000","last_peered":"2024-09-21T07:49:36.965850+0000","last_clean":"2024-09-21T07:49:36.965850+0000","last_became_active":"2024-09-21T07:49:35.151391+0000","last_became_peered":"2024-09-21T07:49:35.151391+0000","last_unstale":"2024-09-21T07:49:36.965850+0000","last_undegraded":"2024-09-21T07:49:36.965850+0000","last_fullsized":"2024-09-21T07:49:36.965850+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:57.983738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.144861+0000","last_change":"2024-09-21T07:49:35.149328+0000","last_active":"2024-09-21T07:49:36.144861+0000","last_peered":"2024-09-21T07:49:36.144861+0000","last_clean":"2024-09-21T07:49:36.144861+0000","last_became_active":"2024-09-21T07:49:35.148981+0000","last_became_peered":"2024-09-21T07:49:35.148981+0000","last_unstale":"2024-09-21T07:49:36.144861+0000","last_undegraded":"2024-09-21T07:49:36.144861+0000","last_fullsized":"2024-09-21T07:49:36.144861+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:11:04.993657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":10,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-21T07:49:35.155342+0000","last_change":"2024-09-21T07:49:35.151794+0000","last_active":"2024-09-21T07:49:35.155342+0000","last_peered":"2024-09-21T07:49:35.155342+0000","last_clean":"2024-09-21T07:49:35.155342+0000","last_became_active":"2024-09-21T07:49:35.151483+0000","last_became_peered":"2024-09-21T07:49:35.151483+0000","last_unstale":"2024-09-21T07:49:35.155342+0000","last_undegraded":"2024-09-21T07:49:35.155342+0000","last_fullsized":"2024-09-21T07:49:35.155342+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:36:19.455400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264918+0000","last_change":"2024-09-21T07:49:35.154570+0000","last_active":"2024-09-21T07:49:37.264918+0000","last_peered":"2024-09-21T07:49:37.264918+0000","last_clean":"2024-09-21T07:49:37.264918+0000","last_became_active":"2024-09-21T07:49:35.154248+0000","last_became_peered":"2024-09-21T07:49:35.154248+0000","last_unstale":"2024-09-21T07:49:37.264918+0000","last_undegraded":"2024-09-21T07:49:37.264918+0000","last_fullsized":"2024-09-21T07:49:37.264918+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:05:34.913847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.142917+0000","last_change":"2024-09-21T07:49:35.147635+0000","last_active":"2024-09-21T07:49:36.142917+0000","last_peered":"2024-09-21T07:49:36.142917+0000","last_clean":"2024-09-21T07:49:36.142917+0000","last_became_active":"2024-09-21T07:49:35.144743+0000","last_became_peered":"2024-09-21T07:49:35.144743+0000","last_unstale":"2024-09-21T07:49:36.142917+0000","last_undegraded":"2024-09-21T07:49:36.142917+0000","last_fullsized":"2024-09-21T07:49:36.142917+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:20.077714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143386+0000","last_change":"2024-09-21T07:49:35.129921+0000","last_active":"2024-09-21T07:49:36.143386+0000","last_peered":"2024-09-21T07:49:36.143386+0000","last_clean":"2024-09-21T07:49:36.143386+0000","last_became_active":"2024-09-21T07:49:35.129617+0000","last_became_peered":"2024-09-21T07:49:35.129617+0000","last_unstale":"2024-09-21T07:49:36.143386+0000","last_undegraded":"2024-09-21T07:49:36.143386+0000","last_fullsized":"2024-09-21T07:49:36.143386+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:55:59.193165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.141880+0000","last_change":"2024-09-21T07:49:35.138164+0000","last_active":"2024-09-21T07:49:36.141880+0000","last_peered":"2024-09-21T07:49:36.141880+0000","last_clean":"2024-09-21T07:49:36.141880+0000","last_became_active":"2024-09-21T07:49:35.137816+0000","last_became_peered":"2024-09-21T07:49:35.137816+0000","last_unstale":"2024-09-21T07:49:36.141880+0000","last_undegraded":"2024-09-21T07:49:36.141880+0000","last_fullsized":"2024-09-21T07:49:36.141880+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:10.230183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.145058+0000","last_change":"2024-09-21T07:49:35.152205+0000","last_active":"2024-09-21T07:49:36.145058+0000","last_peered":"2024-09-21T07:49:36.145058+0000","last_clean":"2024-09-21T07:49:36.145058+0000","last_became_active":"2024-09-21T07:49:35.151874+0000","last_became_peered":"2024-09-21T07:49:35.151874+0000","last_unstale":"2024-09-21T07:49:36.145058+0000","last_undegraded":"2024-09-21T07:49:36.145058+0000","last_fullsized":"2024-09-21T07:49:36.145058+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:02:34.917415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.144774+0000","last_change":"2024-09-21T07:49:35.144935+0000","last_active":"2024-09-21T07:49:36.144774+0000","last_peered":"2024-09-21T07:49:36.144774+0000","last_clean":"2024-09-21T07:49:36.144774+0000","last_became_active":"2024-09-21T07:49:35.144599+0000","last_became_peered":"2024-09-21T07:49:35.144599+0000","last_unstale":"2024-09-21T07:49:36.144774+0000","last_undegraded":"2024-09-21T07:49:36.144774+0000","last_fullsized":"2024-09-21T07:49:36.144774+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:17.706464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":10,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-21T07:49:35.156936+0000","last_change":"2024-09-21T07:49:35.153968+0000","last_active":"2024-09-21T07:49:35.156936+0000","last_peered":"2024-09-21T07:49:35.156936+0000","last_clean":"2024-09-21T07:49:35.156936+0000","last_became_active":"2024-09-21T07:49:35.153673+0000","last_became_peered":"2024-09-21T07:49:35.153673+0000","last_unstale":"2024-09-21T07:49:35.156936+0000","last_undegraded":"2024-09-21T07:49:35.156936+0000","last_fullsized":"2024-09-21T07:49:35.156936+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:12:38.863678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.265842+0000","last_change":"2024-09-21T07:49:35.149138+0000","last_active":"2024-09-21T07:49:37.265842+0000","last_peered":"2024-09-21T07:49:37.265842+0000","last_clean":"2024-09-21T07:49:37.265842+0000","last_became_active":"2024-09-21T07:49:35.148841+0000","last_became_peered":"2024-09-21T07:49:35.148841+0000","last_unstale":"2024-09-21T07:49:37.265842+0000","last_undegraded":"2024-09-21T07:49:37.265842+0000","last_fullsized":"2024-09-21T07:49:37.265842+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:37:25.254306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.142374+0000","last_change":"2024-09-21T07:49:35.144811+0000","last_active":"2024-09-21T07:49:36.142374+0000","last_peered":"2024-09-21T07:49:36.142374+0000","last_clean":"2024-09-21T07:49:36.142374+0000","last_became_active":"2024-09-21T07:49:35.144483+0000","last_became_peered":"2024-09-21T07:49:35.144483+0000","last_unstale":"2024-09-21T07:49:36.142374+0000","last_undegraded":"2024-09-21T07:49:36.142374+0000","last_fullsized":"2024-09-21T07:49:36.142374+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:11:34.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.266218+0000","last_change":"2024-09-21T07:49:35.137745+0000","last_active":"2024-09-21T07:49:37.266218+0000","last_peered":"2024-09-21T07:49:37.266218+0000","last_clean":"2024-09-21T07:49:37.266218+0000","last_became_active":"2024-09-21T07:49:35.134993+0000","last_became_peered":"2024-09-21T07:49:35.134993+0000","last_unstale":"2024-09-21T07:49:37.266218+0000","last_undegraded":"2024-09-21T07:49:37.266218+0000","last_fullsized":"2024-09-21T07:49:37.266218+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:02.781796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.121588+0000","last_change":"2024-09-21T07:49:35.144236+0000","last_active":"2024-09-21T07:49:37.121588+0000","last_peered":"2024-09-21T07:49:37.121588+0000","last_clean":"2024-09-21T07:49:37.121588+0000","last_became_active":"2024-09-21T07:49:35.143875+0000","last_became_peered":"2024-09-21T07:49:35.143875+0000","last_unstale":"2024-09-21T07:49:37.121588+0000","last_undegraded":"2024-09-21T07:49:37.121588+0000","last_fullsized":"2024-09-21T07:49:37.121588+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:33.174799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.141957+0000","last_change":"2024-09-21T07:49:35.139659+0000","last_active":"2024-09-21T07:49:36.141957+0000","last_peered":"2024-09-21T07:49:36.141957+0000","last_clean":"2024-09-21T07:49:36.141957+0000","last_became_active":"2024-09-21T07:49:35.139354+0000","last_became_peered":"2024-09-21T07:49:35.139354+0000","last_unstale":"2024-09-21T07:49:36.141957+0000","last_undegraded":"2024-09-21T07:49:36.141957+0000","last_fullsized":"2024-09-21T07:49:36.141957+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:45:34.545114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.266910+0000","last_change":"2024-09-21T07:49:35.146980+0000","last_active":"2024-09-21T07:49:37.266910+0000","last_peered":"2024-09-21T07:49:37.266910+0000","last_clean":"2024-09-21T07:49:37.266910+0000","last_became_active":"2024-09-21T07:49:35.142121+0000","last_became_peered":"2024-09-21T07:49:35.142121+0000","last_unstale":"2024-09-21T07:49:37.266910+0000","last_undegraded":"2024-09-21T07:49:37.266910+0000","last_fullsized":"2024-09-21T07:49:37.266910+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:41:11.664236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.144990+0000","last_change":"2024-09-21T07:49:35.146061+0000","last_active":"2024-09-21T07:49:36.144990+0000","last_peered":"2024-09-21T07:49:36.144990+0000","last_clean":"2024-09-21T07:49:36.144990+0000","last_became_active":"2024-09-21T07:49:35.141899+0000","last_became_peered":"2024-09-21T07:49:35.141899+0000","last_unstale":"2024-09-21T07:49:36.144990+0000","last_undegraded":"2024-09-21T07:49:36.144990+0000","last_fullsized":"2024-09-21T07:49:36.144990+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:41:40.905635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264337+0000","last_change":"2024-09-21T07:49:35.147825+0000","last_active":"2024-09-21T07:49:37.264337+0000","last_peered":"2024-09-21T07:49:37.264337+0000","last_clean":"2024-09-21T07:49:37.264337+0000","last_became_active":"2024-09-21T07:49:35.147451+0000","last_became_peered":"2024-09-21T07:49:35.147451+0000","last_unstale":"2024-09-21T07:49:37.264337+0000","last_undegraded":"2024-09-21T07:49:37.264337+0000","last_fullsized":"2024-09-21T07:49:37.264337+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:26.985337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.142623+0000","last_change":"2024-09-21T07:49:35.153370+0000","last_active":"2024-09-21T07:49:36.142623+0000","last_peered":"2024-09-21T07:49:36.142623+0000","last_clean":"2024-09-21T07:49:36.142623+0000","last_became_active":"2024-09-21T07:49:35.152865+0000","last_became_peered":"2024-09-21T07:49:35.152865+0000","last_unstale":"2024-09-21T07:49:36.142623+0000","last_undegraded":"2024-09-21T07:49:36.142623+0000","last_fullsized":"2024-09-21T07:49:36.142623+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:42:31.444496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.144022+0000","last_change":"2024-09-21T07:49:35.139492+0000","last_active":"2024-09-21T07:49:36.144022+0000","last_peered":"2024-09-21T07:49:36.144022+0000","last_clean":"2024-09-21T07:49:36.144022+0000","last_became_active":"2024-09-21T07:49:35.138366+0000","last_became_peered":"2024-09-21T07:49:35.138366+0000","last_unstale":"2024-09-21T07:49:36.144022+0000","last_undegraded":"2024-09-21T07:49:36.144022+0000","last_fullsized":"2024-09-21T07:49:36.144022+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:55.251141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.121925+0000","last_change":"2024-09-21T07:49:35.150421+0000","last_active":"2024-09-21T07:49:37.121925+0000","last_peered":"2024-09-21T07:49:37.121925+0000","last_clean":"2024-09-21T07:49:37.121925+0000","last_became_active":"2024-09-21T07:49:35.149998+0000","last_became_peered":"2024-09-21T07:49:35.149998+0000","last_unstale":"2024-09-21T07:49:37.121925+0000","last_undegraded":"2024-09-21T07:49:37.121925+0000","last_fullsized":"2024-09-21T07:49:37.121925+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:03:06.700393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.142875+0000","last_change":"2024-09-21T07:49:35.147014+0000","last_active":"2024-09-21T07:49:36.142875+0000","last_peered":"2024-09-21T07:49:36.142875+0000","last_clean":"2024-09-21T07:49:36.142875+0000","last_became_active":"2024-09-21T07:49:35.146672+0000","last_became_peered":"2024-09-21T07:49:35.146672+0000","last_unstale":"2024-09-21T07:49:36.142875+0000","last_undegraded":"2024-09-21T07:49:36.142875+0000","last_fullsized":"2024-09-21T07:49:36.142875+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:32.400747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.263900+0000","last_change":"2024-09-21T07:49:35.128590+0000","last_active":"2024-09-21T07:49:37.263900+0000","last_peered":"2024-09-21T07:49:37.263900+0000","last_clean":"2024-09-21T07:49:37.263900+0000","last_became_active":"2024-09-21T07:49:35.128230+0000","last_became_peered":"2024-09-21T07:49:35.128230+0000","last_unstale":"2024-09-21T07:49:37.263900+0000","last_undegraded":"2024-09-21T07:49:37.263900+0000","last_fullsized":"2024-09-21T07:49:37.263900+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:39.205266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.265758+0000","last_change":"2024-09-21T07:49:35.133443+0000","last_active":"2024-09-21T07:49:37.265758+0000","last_peered":"2024-09-21T07:49:37.265758+0000","last_clean":"2024-09-21T07:49:37.265758+0000","last_became_active":"2024-09-21T07:49:35.133150+0000","last_became_peered":"2024-09-21T07:49:35.133150+0000","last_unstale":"2024-09-21T07:49:37.265758+0000","last_undegraded":"2024-09-21T07:49:37.265758+0000","last_fullsized":"2024-09-21T07:49:37.265758+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:45.555083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.145232+0000","last_change":"2024-09-21T07:49:35.137175+0000","last_active":"2024-09-21T07:49:36.145232+0000","last_peered":"2024-09-21T07:49:36.145232+0000","last_clean":"2024-09-21T07:49:36.145232+0000","last_became_active":"2024-09-21T07:49:35.136514+0000","last_became_peered":"2024-09-21T07:49:35.136514+0000","last_unstale":"2024-09-21T07:49:36.145232+0000","last_undegraded":"2024-09-21T07:49:36.145232+0000","last_fullsized":"2024-09-21T07:49:36.145232+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:34:45.491476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":10,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-21T07:49:35.157988+0000","last_change":"2024-09-21T07:49:35.156104+0000","last_active":"2024-09-21T07:49:35.157988+0000","last_peered":"2024-09-21T07:49:35.157988+0000","last_clean":"2024-09-21T07:49:35.157988+0000","last_became_active":"2024-09-21T07:49:35.155751+0000","last_became_peered":"2024-09-21T07:49:35.155751+0000","last_unstale":"2024-09-21T07:49:35.157988+0000","last_undegraded":"2024-09-21T07:49:35.157988+0000","last_fullsized":"2024-09-21T07:49:35.157988+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:22:16.087564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.266132+0000","last_change":"2024-09-21T07:49:35.147755+0000","last_active":"2024-09-21T07:49:37.266132+0000","last_peered":"2024-09-21T07:49:37.266132+0000","last_clean":"2024-09-21T07:49:37.266132+0000","last_became_active":"2024-09-21T07:49:35.142034+0000","last_became_peered":"2024-09-21T07:49:35.142034+0000","last_unstale":"2024-09-21T07:49:37.266132+0000","last_undegraded":"2024-09-21T07:49:37.266132+0000","last_fullsized":"2024-09-21T07:49:37.266132+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:51:54.013029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.965559+0000","last_change":"2024-09-21T07:49:35.147889+0000","last_active":"2024-09-21T07:49:36.965559+0000","last_peered":"2024-09-21T07:49:36.965559+0000","last_clean":"2024-09-21T07:49:36.965559+0000","last_became_active":"2024-09-21T07:49:35.147609+0000","last_became_peered":"2024-09-21T07:49:35.147609+0000","last_unstale":"2024-09-21T07:49:36.965559+0000","last_undegraded":"2024-09-21T07:49:36.965559+0000","last_fullsized":"2024-09-21T07:49:36.965559+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:54:52.012917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.142201+0000","last_change":"2024-09-21T07:49:35.148893+0000","last_active":"2024-09-21T07:49:36.142201+0000","last_peered":"2024-09-21T07:49:36.142201+0000","last_clean":"2024-09-21T07:49:36.142201+0000","last_became_active":"2024-09-21T07:49:35.148580+0000","last_became_peered":"2024-09-21T07:49:35.148580+0000","last_unstale":"2024-09-21T07:49:36.142201+0000","last_undegraded":"2024-09-21T07:49:36.142201+0000","last_fullsized":"2024-09-21T07:49:36.142201+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:34:13.594958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.142894+0000","last_change":"2024-09-21T07:49:35.133699+0000","last_active":"2024-09-21T07:49:36.142894+0000","last_peered":"2024-09-21T07:49:36.142894+0000","last_clean":"2024-09-21T07:49:36.142894+0000","last_became_active":"2024-09-21T07:49:35.133323+0000","last_became_peered":"2024-09-21T07:49:35.133323+0000","last_unstale":"2024-09-21T07:49:36.142894+0000","last_undegraded":"2024-09-21T07:49:36.142894+0000","last_fullsized":"2024-09-21T07:49:36.142894+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:14:57.553709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.142157+0000","last_change":"2024-09-21T07:49:35.147226+0000","last_active":"2024-09-21T07:49:36.142157+0000","last_peered":"2024-09-21T07:49:36.142157+0000","last_clean":"2024-09-21T07:49:36.142157+0000","last_became_active":"2024-09-21T07:49:35.146764+0000","last_became_peered":"2024-09-21T07:49:35.146764+0000","last_unstale":"2024-09-21T07:49:36.142157+0000","last_undegraded":"2024-09-21T07:49:36.142157+0000","last_fullsized":"2024-09-21T07:49:36.142157+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:37:01.120540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1f","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.146053+0000","last_change":"2024-09-21T07:49:35.137241+0000","last_active":"2024-09-21T07:49:36.146053+0000","last_peered":"2024-09-21T07:49:36.146053+0000","last_clean":"2024-09-21T07:49:36.146053+0000","last_became_active":"2024-09-21T07:49:35.136928+0000","last_became_peered":"2024-09-21T07:49:35.136928+0000","last_unstale":"2024-09-21T07:49:36.146053+0000","last_undegraded":"2024-09-21T07:49:36.146053+0000","last_fullsized":"2024-09-21T07:49:36.146053+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:37.194496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1c","version":"232'2","reported_seq":17,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264774+0000","last_change":"2024-09-21T07:49:34.134258+0000","last_active":"2024-09-21T07:49:37.264774+0000","last_peered":"2024-09-21T07:49:37.264774+0000","last_clean":"2024-09-21T07:49:37.264774+0000","last_became_active":"2024-09-21T07:49:34.133961+0000","last_became_peered":"2024-09-21T07:49:34.133961+0000","last_unstale":"2024-09-21T07:49:37.264774+0000","last_undegraded":"2024-09-21T07:49:37.264774+0000","last_fullsized":"2024-09-21T07:49:37.264774+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:31.820492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":331,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264969+0000","last_change":"2024-09-21T07:35:27.879605+0000","last_active":"2024-09-21T07:49:37.264969+0000","last_peered":"2024-09-21T07:49:37.264969+0000","last_clean":"2024-09-21T07:49:37.264969+0000","last_became_active":"2024-09-21T07:35:27.879274+0000","last_became_peered":"2024-09-21T07:35:27.879274+0000","last_unstale":"2024-09-21T07:49:37.264969+0000","last_undegraded":"2024-09-21T07:49:37.264969+0000","last_fullsized":"2024-09-21T07:49:37.264969+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:24.241636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":341,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.144539+0000","last_change":"2024-09-21T07:35:29.491336+0000","last_active":"2024-09-21T07:49:36.144539+0000","last_peered":"2024-09-21T07:49:36.144539+0000","last_clean":"2024-09-21T07:49:36.144539+0000","last_became_active":"2024-09-21T07:35:26.866484+0000","last_became_peered":"2024-09-21T07:35:26.866484+0000","last_unstale":"2024-09-21T07:49:36.144539+0000","last_undegraded":"2024-09-21T07:49:36.144539+0000","last_fullsized":"2024-09-21T07:49:36.144539+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:10.449665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076010100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":517,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264014+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:49:37.264014+0000","last_peered":"2024-09-21T07:49:37.264014+0000","last_clean":"2024-09-21T07:49:37.264014+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:49:37.264014+0000","last_undegraded":"2024-09-21T07:49:37.264014+0000","last_fullsized":"2024-09-21T07:49:37.264014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":408,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143247+0000","last_change":"2024-09-21T07:34:24.539248+0000","last_active":"2024-09-21T07:49:36.143247+0000","last_peered":"2024-09-21T07:49:36.143247+0000","last_clean":"2024-09-21T07:49:36.143247+0000","last_became_active":"2024-09-21T07:34:23.273465+0000","last_became_peered":"2024-09-21T07:34:23.273465+0000","last_unstale":"2024-09-21T07:49:36.143247+0000","last_undegraded":"2024-09-21T07:49:36.143247+0000","last_fullsized":"2024-09-21T07:49:36.143247+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:55.517452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060627700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.122454+0000","last_change":"2024-09-21T07:49:35.150255+0000","last_active":"2024-09-21T07:49:37.122454+0000","last_peered":"2024-09-21T07:49:37.122454+0000","last_clean":"2024-09-21T07:49:37.122454+0000","last_became_active":"2024-09-21T07:49:35.149897+0000","last_became_peered":"2024-09-21T07:49:35.149897+0000","last_unstale":"2024-09-21T07:49:37.122454+0000","last_undegraded":"2024-09-21T07:49:37.122454+0000","last_fullsized":"2024-09-21T07:49:37.122454+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:38:38.777657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":13,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-21T07:49:35.127794+0000","last_change":"2024-09-21T07:49:34.144688+0000","last_active":"2024-09-21T07:49:35.127794+0000","last_peered":"2024-09-21T07:49:35.127794+0000","last_clean":"2024-09-21T07:49:35.127794+0000","last_became_active":"2024-09-21T07:49:34.144328+0000","last_became_peered":"2024-09-21T07:49:34.144328+0000","last_unstale":"2024-09-21T07:49:35.127794+0000","last_undegraded":"2024-09-21T07:49:35.127794+0000","last_fullsized":"2024-09-21T07:49:35.127794+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:57.881990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":334,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143539+0000","last_change":"2024-09-21T07:35:27.864492+0000","last_active":"2024-09-21T07:49:36.143539+0000","last_peered":"2024-09-21T07:49:36.143539+0000","last_clean":"2024-09-21T07:49:36.143539+0000","last_became_active":"2024-09-21T07:35:27.864176+0000","last_became_peered":"2024-09-21T07:35:27.864176+0000","last_unstale":"2024-09-21T07:49:36.143539+0000","last_undegraded":"2024-09-21T07:49:36.143539+0000","last_fullsized":"2024-09-21T07:49:36.143539+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:05.153467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":338,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.266427+0000","last_change":"2024-09-21T07:35:29.493678+0000","last_active":"2024-09-21T07:49:37.266427+0000","last_peered":"2024-09-21T07:49:37.266427+0000","last_clean":"2024-09-21T07:49:37.266427+0000","last_became_active":"2024-09-21T07:35:26.870749+0000","last_became_peered":"2024-09-21T07:35:26.870749+0000","last_unstale":"2024-09-21T07:49:37.266427+0000","last_undegraded":"2024-09-21T07:49:37.266427+0000","last_fullsized":"2024-09-21T07:49:37.266427+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:55.367877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010159570000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":384,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-21T07:49:35.132428+0000","last_change":"2024-09-21T07:34:24.597274+0000","last_active":"2024-09-21T07:49:35.132428+0000","last_peered":"2024-09-21T07:49:35.132428+0000","last_clean":"2024-09-21T07:49:35.132428+0000","last_became_active":"2024-09-21T07:34:23.279560+0000","last_became_peered":"2024-09-21T07:34:23.279560+0000","last_unstale":"2024-09-21T07:49:35.132428+0000","last_undegraded":"2024-09-21T07:49:35.132428+0000","last_fullsized":"2024-09-21T07:49:35.132428+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:15.845265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059699799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143252+0000","last_change":"2024-09-21T07:49:35.145803+0000","last_active":"2024-09-21T07:49:36.143252+0000","last_peered":"2024-09-21T07:49:36.143252+0000","last_clean":"2024-09-21T07:49:36.143252+0000","last_became_active":"2024-09-21T07:49:35.145504+0000","last_became_peered":"2024-09-21T07:49:35.145504+0000","last_unstale":"2024-09-21T07:49:36.143252+0000","last_undegraded":"2024-09-21T07:49:36.143252+0000","last_fullsized":"2024-09-21T07:49:36.143252+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:27:42.102361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143087+0000","last_change":"2024-09-21T07:49:34.131135+0000","last_active":"2024-09-21T07:49:36.143087+0000","last_peered":"2024-09-21T07:49:36.143087+0000","last_clean":"2024-09-21T07:49:36.143087+0000","last_became_active":"2024-09-21T07:49:34.130782+0000","last_became_peered":"2024-09-21T07:49:34.130782+0000","last_unstale":"2024-09-21T07:49:36.143087+0000","last_undegraded":"2024-09-21T07:49:36.143087+0000","last_fullsized":"2024-09-21T07:49:36.143087+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:01.932485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":335,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.142947+0000","last_change":"2024-09-21T07:35:27.874387+0000","last_active":"2024-09-21T07:49:36.142947+0000","last_peered":"2024-09-21T07:49:36.142947+0000","last_clean":"2024-09-21T07:49:36.142947+0000","last_became_active":"2024-09-21T07:35:27.873277+0000","last_became_peered":"2024-09-21T07:35:27.873277+0000","last_unstale":"2024-09-21T07:49:36.142947+0000","last_undegraded":"2024-09-21T07:49:36.142947+0000","last_fullsized":"2024-09-21T07:49:36.142947+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:04.581463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":334,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-21T07:49:35.138161+0000","last_change":"2024-09-21T07:35:29.387094+0000","last_active":"2024-09-21T07:49:35.138161+0000","last_peered":"2024-09-21T07:49:35.138161+0000","last_clean":"2024-09-21T07:49:35.138161+0000","last_became_active":"2024-09-21T07:35:26.874757+0000","last_became_peered":"2024-09-21T07:35:26.874757+0000","last_unstale":"2024-09-21T07:49:35.138161+0000","last_undegraded":"2024-09-21T07:49:35.138161+0000","last_fullsized":"2024-09-21T07:49:35.138161+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:33.542750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061065700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":391,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.265481+0000","last_change":"2024-09-21T07:34:24.259906+0000","last_active":"2024-09-21T07:49:37.265481+0000","last_peered":"2024-09-21T07:49:37.265481+0000","last_clean":"2024-09-21T07:49:37.265481+0000","last_became_active":"2024-09-21T07:34:23.635282+0000","last_became_peered":"2024-09-21T07:34:23.635282+0000","last_unstale":"2024-09-21T07:49:37.265481+0000","last_undegraded":"2024-09-21T07:49:37.265481+0000","last_fullsized":"2024-09-21T07:49:37.265481+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:00.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001527664,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143968+0000","last_change":"2024-09-21T07:49:35.144096+0000","last_active":"2024-09-21T07:49:36.143968+0000","last_peered":"2024-09-21T07:49:36.143968+0000","last_clean":"2024-09-21T07:49:36.143968+0000","last_became_active":"2024-09-21T07:49:35.143812+0000","last_became_peered":"2024-09-21T07:49:35.143812+0000","last_unstale":"2024-09-21T07:49:36.143968+0000","last_undegraded":"2024-09-21T07:49:36.143968+0000","last_fullsized":"2024-09-21T07:49:36.143968+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:46.238631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1b","version":"232'1","reported_seq":16,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.121724+0000","last_change":"2024-09-21T07:49:34.127268+0000","last_active":"2024-09-21T07:49:37.121724+0000","last_peered":"2024-09-21T07:49:37.121724+0000","last_clean":"2024-09-21T07:49:37.121724+0000","last_became_active":"2024-09-21T07:49:34.126215+0000","last_became_peered":"2024-09-21T07:49:34.126215+0000","last_unstale":"2024-09-21T07:49:37.121724+0000","last_undegraded":"2024-09-21T07:49:37.121724+0000","last_fullsized":"2024-09-21T07:49:37.121724+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:52.720478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":335,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143512+0000","last_change":"2024-09-21T07:35:27.873645+0000","last_active":"2024-09-21T07:49:36.143512+0000","last_peered":"2024-09-21T07:49:36.143512+0000","last_clean":"2024-09-21T07:49:36.143512+0000","last_became_active":"2024-09-21T07:35:27.873245+0000","last_became_peered":"2024-09-21T07:35:27.873245+0000","last_unstale":"2024-09-21T07:49:36.143512+0000","last_undegraded":"2024-09-21T07:49:36.143512+0000","last_fullsized":"2024-09-21T07:49:36.143512+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:55.090953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":342,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.144579+0000","last_change":"2024-09-21T07:35:28.861744+0000","last_active":"2024-09-21T07:49:36.144579+0000","last_peered":"2024-09-21T07:49:36.144579+0000","last_clean":"2024-09-21T07:49:36.144579+0000","last_became_active":"2024-09-21T07:35:26.876462+0000","last_became_peered":"2024-09-21T07:35:26.876462+0000","last_unstale":"2024-09-21T07:49:36.144579+0000","last_undegraded":"2024-09-21T07:49:36.144579+0000","last_fullsized":"2024-09-21T07:49:36.144579+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:48.632568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012020920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":391,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.965152+0000","last_change":"2024-09-21T07:34:25.085454+0000","last_active":"2024-09-21T07:49:36.965152+0000","last_peered":"2024-09-21T07:49:36.965152+0000","last_clean":"2024-09-21T07:49:36.965152+0000","last_became_active":"2024-09-21T07:34:23.279205+0000","last_became_peered":"2024-09-21T07:34:23.279205+0000","last_unstale":"2024-09-21T07:49:36.965152+0000","last_undegraded":"2024-09-21T07:49:36.965152+0000","last_fullsized":"2024-09-21T07:49:36.965152+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:54.231666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095320599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.266315+0000","last_change":"2024-09-21T07:49:35.146014+0000","last_active":"2024-09-21T07:49:37.266315+0000","last_peered":"2024-09-21T07:49:37.266315+0000","last_clean":"2024-09-21T07:49:37.266315+0000","last_became_active":"2024-09-21T07:49:35.141715+0000","last_became_peered":"2024-09-21T07:49:35.141715+0000","last_unstale":"2024-09-21T07:49:37.266315+0000","last_undegraded":"2024-09-21T07:49:37.266315+0000","last_fullsized":"2024-09-21T07:49:37.266315+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:37.560681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1a","version":"232'1","reported_seq":16,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264487+0000","last_change":"2024-09-21T07:49:34.139919+0000","last_active":"2024-09-21T07:49:37.264487+0000","last_peered":"2024-09-21T07:49:37.264487+0000","last_clean":"2024-09-21T07:49:37.264487+0000","last_became_active":"2024-09-21T07:49:34.139183+0000","last_became_peered":"2024-09-21T07:49:34.139183+0000","last_unstale":"2024-09-21T07:49:37.264487+0000","last_undegraded":"2024-09-21T07:49:37.264487+0000","last_fullsized":"2024-09-21T07:49:37.264487+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:57.776256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":334,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.144088+0000","last_change":"2024-09-21T07:35:27.861373+0000","last_active":"2024-09-21T07:49:36.144088+0000","last_peered":"2024-09-21T07:49:36.144088+0000","last_clean":"2024-09-21T07:49:36.144088+0000","last_became_active":"2024-09-21T07:35:27.861016+0000","last_became_peered":"2024-09-21T07:35:27.861016+0000","last_unstale":"2024-09-21T07:49:36.144088+0000","last_undegraded":"2024-09-21T07:49:36.144088+0000","last_fullsized":"2024-09-21T07:49:36.144088+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:56.043763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":343,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.121761+0000","last_change":"2024-09-21T07:35:29.053677+0000","last_active":"2024-09-21T07:49:37.121761+0000","last_peered":"2024-09-21T07:49:37.121761+0000","last_clean":"2024-09-21T07:49:37.121761+0000","last_became_active":"2024-09-21T07:35:26.875496+0000","last_became_peered":"2024-09-21T07:35:26.875496+0000","last_unstale":"2024-09-21T07:49:37.121761+0000","last_undegraded":"2024-09-21T07:49:37.121761+0000","last_fullsized":"2024-09-21T07:49:37.121761+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:04.850784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065859000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":395,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.121605+0000","last_change":"2024-09-21T07:34:25.028521+0000","last_active":"2024-09-21T07:49:37.121605+0000","last_peered":"2024-09-21T07:49:37.121605+0000","last_clean":"2024-09-21T07:49:37.121605+0000","last_became_active":"2024-09-21T07:34:23.635467+0000","last_became_peered":"2024-09-21T07:34:23.635467+0000","last_unstale":"2024-09-21T07:49:37.121605+0000","last_undegraded":"2024-09-21T07:49:37.121605+0000","last_fullsized":"2024-09-21T07:49:37.121605+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:33.729123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000660558,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.267007+0000","last_change":"2024-09-21T07:49:35.135388+0000","last_active":"2024-09-21T07:49:37.267007+0000","last_peered":"2024-09-21T07:49:37.267007+0000","last_clean":"2024-09-21T07:49:37.267007+0000","last_became_active":"2024-09-21T07:49:35.134074+0000","last_became_peered":"2024-09-21T07:49:35.134074+0000","last_unstale":"2024-09-21T07:49:37.267007+0000","last_undegraded":"2024-09-21T07:49:37.267007+0000","last_fullsized":"2024-09-21T07:49:37.267007+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:54.944026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1d","version":"232'2","reported_seq":17,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143045+0000","last_change":"2024-09-21T07:49:34.136420+0000","last_active":"2024-09-21T07:49:36.143045+0000","last_peered":"2024-09-21T07:49:36.143045+0000","last_clean":"2024-09-21T07:49:36.143045+0000","last_became_active":"2024-09-21T07:49:34.136041+0000","last_became_peered":"2024-09-21T07:49:34.136041+0000","last_unstale":"2024-09-21T07:49:36.143045+0000","last_undegraded":"2024-09-21T07:49:36.143045+0000","last_fullsized":"2024-09-21T07:49:36.143045+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:03:21.509136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":333,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.965862+0000","last_change":"2024-09-21T07:35:27.871201+0000","last_active":"2024-09-21T07:49:36.965862+0000","last_peered":"2024-09-21T07:49:36.965862+0000","last_clean":"2024-09-21T07:49:36.965862+0000","last_became_active":"2024-09-21T07:35:27.870816+0000","last_became_peered":"2024-09-21T07:35:27.870816+0000","last_unstale":"2024-09-21T07:49:36.965862+0000","last_undegraded":"2024-09-21T07:49:36.965862+0000","last_fullsized":"2024-09-21T07:49:36.965862+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:04.787450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":341,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.144343+0000","last_change":"2024-09-21T07:35:29.491874+0000","last_active":"2024-09-21T07:49:36.144343+0000","last_peered":"2024-09-21T07:49:36.144343+0000","last_clean":"2024-09-21T07:49:36.144343+0000","last_became_active":"2024-09-21T07:35:26.866381+0000","last_became_peered":"2024-09-21T07:35:26.866381+0000","last_unstale":"2024-09-21T07:49:36.144343+0000","last_undegraded":"2024-09-21T07:49:36.144343+0000","last_fullsized":"2024-09-21T07:49:36.144343+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:36.404647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073235399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":396,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.146133+0000","last_change":"2024-09-21T07:34:24.260046+0000","last_active":"2024-09-21T07:49:36.146133+0000","last_peered":"2024-09-21T07:49:36.146133+0000","last_clean":"2024-09-21T07:49:36.146133+0000","last_became_active":"2024-09-21T07:34:23.273431+0000","last_became_peered":"2024-09-21T07:34:23.273431+0000","last_unstale":"2024-09-21T07:49:36.146133+0000","last_undegraded":"2024-09-21T07:49:36.146133+0000","last_fullsized":"2024-09-21T07:49:36.146133+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:18.876057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019803310000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.144496+0000","last_change":"2024-09-21T07:49:35.146516+0000","last_active":"2024-09-21T07:49:36.144496+0000","last_peered":"2024-09-21T07:49:36.144496+0000","last_clean":"2024-09-21T07:49:36.144496+0000","last_became_active":"2024-09-21T07:49:35.143810+0000","last_became_peered":"2024-09-21T07:49:35.143810+0000","last_unstale":"2024-09-21T07:49:36.144496+0000","last_undegraded":"2024-09-21T07:49:36.144496+0000","last_fullsized":"2024-09-21T07:49:36.144496+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:44.416224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"232'4","reported_seq":19,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.145222+0000","last_change":"2024-09-21T07:49:34.141934+0000","last_active":"2024-09-21T07:49:36.145222+0000","last_peered":"2024-09-21T07:49:36.145222+0000","last_clean":"2024-09-21T07:49:36.145222+0000","last_became_active":"2024-09-21T07:49:34.141588+0000","last_became_peered":"2024-09-21T07:49:34.141588+0000","last_unstale":"2024-09-21T07:49:36.145222+0000","last_undegraded":"2024-09-21T07:49:36.145222+0000","last_fullsized":"2024-09-21T07:49:36.145222+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:21.118757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":333,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.965897+0000","last_change":"2024-09-21T07:35:27.871736+0000","last_active":"2024-09-21T07:49:36.965897+0000","last_peered":"2024-09-21T07:49:36.965897+0000","last_clean":"2024-09-21T07:49:36.965897+0000","last_became_active":"2024-09-21T07:35:27.871232+0000","last_became_peered":"2024-09-21T07:35:27.871232+0000","last_unstale":"2024-09-21T07:49:36.965897+0000","last_undegraded":"2024-09-21T07:49:36.965897+0000","last_fullsized":"2024-09-21T07:49:36.965897+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:04.419301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":341,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143647+0000","last_change":"2024-09-21T07:35:29.491246+0000","last_active":"2024-09-21T07:49:36.143647+0000","last_peered":"2024-09-21T07:49:36.143647+0000","last_clean":"2024-09-21T07:49:36.143647+0000","last_became_active":"2024-09-21T07:35:26.866783+0000","last_became_peered":"2024-09-21T07:35:26.866783+0000","last_unstale":"2024-09-21T07:49:36.143647+0000","last_undegraded":"2024-09-21T07:49:36.143647+0000","last_fullsized":"2024-09-21T07:49:36.143647+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:28.881914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098474099999999992,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":395,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.122019+0000","last_change":"2024-09-21T07:34:25.028640+0000","last_active":"2024-09-21T07:49:37.122019+0000","last_peered":"2024-09-21T07:49:37.122019+0000","last_clean":"2024-09-21T07:49:37.122019+0000","last_became_active":"2024-09-21T07:34:23.260950+0000","last_became_peered":"2024-09-21T07:34:23.260950+0000","last_unstale":"2024-09-21T07:49:37.122019+0000","last_undegraded":"2024-09-21T07:49:37.122019+0000","last_fullsized":"2024-09-21T07:49:37.122019+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:23:42.121398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055135099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.146123+0000","last_change":"2024-09-21T07:49:35.154197+0000","last_active":"2024-09-21T07:49:36.146123+0000","last_peered":"2024-09-21T07:49:36.146123+0000","last_clean":"2024-09-21T07:49:36.146123+0000","last_became_active":"2024-09-21T07:49:35.153887+0000","last_became_peered":"2024-09-21T07:49:35.153887+0000","last_unstale":"2024-09-21T07:49:36.146123+0000","last_undegraded":"2024-09-21T07:49:36.146123+0000","last_fullsized":"2024-09-21T07:49:36.146123+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:29:46.083612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"232'1","reported_seq":16,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.266738+0000","last_change":"2024-09-21T07:49:34.142279+0000","last_active":"2024-09-21T07:49:37.266738+0000","last_peered":"2024-09-21T07:49:37.266738+0000","last_clean":"2024-09-21T07:49:37.266738+0000","last_became_active":"2024-09-21T07:49:34.141943+0000","last_became_peered":"2024-09-21T07:49:34.141943+0000","last_unstale":"2024-09-21T07:49:37.266738+0000","last_undegraded":"2024-09-21T07:49:37.266738+0000","last_fullsized":"2024-09-21T07:49:37.266738+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:58.929915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":335,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.267311+0000","last_change":"2024-09-21T07:35:27.872027+0000","last_active":"2024-09-21T07:49:37.267311+0000","last_peered":"2024-09-21T07:49:37.267311+0000","last_clean":"2024-09-21T07:49:37.267311+0000","last_became_active":"2024-09-21T07:35:27.870426+0000","last_became_peered":"2024-09-21T07:35:27.870426+0000","last_unstale":"2024-09-21T07:49:37.267311+0000","last_undegraded":"2024-09-21T07:49:37.267311+0000","last_fullsized":"2024-09-21T07:49:37.267311+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":341,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143638+0000","last_change":"2024-09-21T07:35:29.491008+0000","last_active":"2024-09-21T07:49:36.143638+0000","last_peered":"2024-09-21T07:49:36.143638+0000","last_clean":"2024-09-21T07:49:36.143638+0000","last_became_active":"2024-09-21T07:35:26.866750+0000","last_became_peered":"2024-09-21T07:35:26.866750+0000","last_unstale":"2024-09-21T07:49:36.143638+0000","last_undegraded":"2024-09-21T07:49:36.143638+0000","last_fullsized":"2024-09-21T07:49:36.143638+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:25.490406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083529700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":392,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.265359+0000","last_change":"2024-09-21T07:34:24.260665+0000","last_active":"2024-09-21T07:49:37.265359+0000","last_peered":"2024-09-21T07:49:37.265359+0000","last_clean":"2024-09-21T07:49:37.265359+0000","last_became_active":"2024-09-21T07:34:23.635909+0000","last_became_peered":"2024-09-21T07:34:23.635909+0000","last_unstale":"2024-09-21T07:49:37.265359+0000","last_undegraded":"2024-09-21T07:49:37.265359+0000","last_fullsized":"2024-09-21T07:49:37.265359+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:44.536885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.144098+0000","last_change":"2024-09-21T07:49:35.135791+0000","last_active":"2024-09-21T07:49:36.144098+0000","last_peered":"2024-09-21T07:49:36.144098+0000","last_clean":"2024-09-21T07:49:36.144098+0000","last_became_active":"2024-09-21T07:49:35.135406+0000","last_became_peered":"2024-09-21T07:49:35.135406+0000","last_unstale":"2024-09-21T07:49:36.144098+0000","last_undegraded":"2024-09-21T07:49:36.144098+0000","last_fullsized":"2024-09-21T07:49:36.144098+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:43.306960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.141754+0000","last_change":"2024-09-21T07:49:34.138954+0000","last_active":"2024-09-21T07:49:36.141754+0000","last_peered":"2024-09-21T07:49:36.141754+0000","last_clean":"2024-09-21T07:49:36.141754+0000","last_became_active":"2024-09-21T07:49:34.138326+0000","last_became_peered":"2024-09-21T07:49:34.138326+0000","last_unstale":"2024-09-21T07:49:36.141754+0000","last_undegraded":"2024-09-21T07:49:36.141754+0000","last_fullsized":"2024-09-21T07:49:36.141754+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:51.125470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":328,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-21T07:49:35.127469+0000","last_change":"2024-09-21T07:35:27.874772+0000","last_active":"2024-09-21T07:49:35.127469+0000","last_peered":"2024-09-21T07:49:35.127469+0000","last_clean":"2024-09-21T07:49:35.127469+0000","last_became_active":"2024-09-21T07:35:27.874434+0000","last_became_peered":"2024-09-21T07:35:27.874434+0000","last_unstale":"2024-09-21T07:49:35.127469+0000","last_undegraded":"2024-09-21T07:49:35.127469+0000","last_fullsized":"2024-09-21T07:49:35.127469+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:31.998755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":355,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143099+0000","last_change":"2024-09-21T07:35:28.859153+0000","last_active":"2024-09-21T07:49:36.143099+0000","last_peered":"2024-09-21T07:49:36.143099+0000","last_clean":"2024-09-21T07:49:36.143099+0000","last_became_active":"2024-09-21T07:35:26.874428+0000","last_became_peered":"2024-09-21T07:35:26.874428+0000","last_unstale":"2024-09-21T07:49:36.143099+0000","last_undegraded":"2024-09-21T07:49:36.143099+0000","last_fullsized":"2024-09-21T07:49:36.143099+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:04.487621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048557299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":408,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.142842+0000","last_change":"2024-09-21T07:34:24.538845+0000","last_active":"2024-09-21T07:49:36.142842+0000","last_peered":"2024-09-21T07:49:36.142842+0000","last_clean":"2024-09-21T07:49:36.142842+0000","last_became_active":"2024-09-21T07:34:23.636372+0000","last_became_peered":"2024-09-21T07:34:23.636372+0000","last_unstale":"2024-09-21T07:49:36.142842+0000","last_undegraded":"2024-09-21T07:49:36.142842+0000","last_fullsized":"2024-09-21T07:49:36.142842+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:29.393813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841451,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.122083+0000","last_change":"2024-09-21T07:49:35.152975+0000","last_active":"2024-09-21T07:49:37.122083+0000","last_peered":"2024-09-21T07:49:37.122083+0000","last_clean":"2024-09-21T07:49:37.122083+0000","last_became_active":"2024-09-21T07:49:35.152637+0000","last_became_peered":"2024-09-21T07:49:35.152637+0000","last_unstale":"2024-09-21T07:49:37.122083+0000","last_undegraded":"2024-09-21T07:49:37.122083+0000","last_fullsized":"2024-09-21T07:49:37.122083+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:51:17.106760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"232'8","reported_seq":23,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264358+0000","last_change":"2024-09-21T07:49:34.142446+0000","last_active":"2024-09-21T07:49:37.264358+0000","last_peered":"2024-09-21T07:49:37.264358+0000","last_clean":"2024-09-21T07:49:37.264358+0000","last_became_active":"2024-09-21T07:49:34.142139+0000","last_became_peered":"2024-09-21T07:49:34.142139+0000","last_unstale":"2024-09-21T07:49:37.264358+0000","last_undegraded":"2024-09-21T07:49:37.264358+0000","last_fullsized":"2024-09-21T07:49:37.264358+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:03.943177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":335,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264453+0000","last_change":"2024-09-21T07:35:27.877801+0000","last_active":"2024-09-21T07:49:37.264453+0000","last_peered":"2024-09-21T07:49:37.264453+0000","last_clean":"2024-09-21T07:49:37.264453+0000","last_became_active":"2024-09-21T07:35:27.877464+0000","last_became_peered":"2024-09-21T07:35:27.877464+0000","last_unstale":"2024-09-21T07:49:37.264453+0000","last_undegraded":"2024-09-21T07:49:37.264453+0000","last_fullsized":"2024-09-21T07:49:37.264453+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:08.814929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":340,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.965925+0000","last_change":"2024-09-21T07:35:28.863732+0000","last_active":"2024-09-21T07:49:36.965925+0000","last_peered":"2024-09-21T07:49:36.965925+0000","last_clean":"2024-09-21T07:49:36.965925+0000","last_became_active":"2024-09-21T07:35:26.877392+0000","last_became_peered":"2024-09-21T07:35:26.877392+0000","last_unstale":"2024-09-21T07:49:36.965925+0000","last_undegraded":"2024-09-21T07:49:36.965925+0000","last_fullsized":"2024-09-21T07:49:36.965925+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071425700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":395,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.144446+0000","last_change":"2024-09-21T07:34:24.259686+0000","last_active":"2024-09-21T07:49:36.144446+0000","last_peered":"2024-09-21T07:49:36.144446+0000","last_clean":"2024-09-21T07:49:36.144446+0000","last_became_active":"2024-09-21T07:34:23.274405+0000","last_became_peered":"2024-09-21T07:34:23.274405+0000","last_unstale":"2024-09-21T07:49:36.144446+0000","last_undegraded":"2024-09-21T07:49:36.144446+0000","last_fullsized":"2024-09-21T07:49:36.144446+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:26.304693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022009920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.965518+0000","last_change":"2024-09-21T07:49:35.140522+0000","last_active":"2024-09-21T07:49:36.965518+0000","last_peered":"2024-09-21T07:49:36.965518+0000","last_clean":"2024-09-21T07:49:36.965518+0000","last_became_active":"2024-09-21T07:49:35.139108+0000","last_became_peered":"2024-09-21T07:49:35.139108+0000","last_unstale":"2024-09-21T07:49:36.965518+0000","last_undegraded":"2024-09-21T07:49:36.965518+0000","last_fullsized":"2024-09-21T07:49:36.965518+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:10.025992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"232'2","reported_seq":17,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.265254+0000","last_change":"2024-09-21T07:49:34.135311+0000","last_active":"2024-09-21T07:49:37.265254+0000","last_peered":"2024-09-21T07:49:37.265254+0000","last_clean":"2024-09-21T07:49:37.265254+0000","last_became_active":"2024-09-21T07:49:34.134991+0000","last_became_peered":"2024-09-21T07:49:34.134991+0000","last_unstale":"2024-09-21T07:49:37.265254+0000","last_undegraded":"2024-09-21T07:49:37.265254+0000","last_fullsized":"2024-09-21T07:49:37.265254+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:19.601267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":327,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-21T07:49:35.132219+0000","last_change":"2024-09-21T07:35:27.867336+0000","last_active":"2024-09-21T07:49:35.132219+0000","last_peered":"2024-09-21T07:49:35.132219+0000","last_clean":"2024-09-21T07:49:35.132219+0000","last_became_active":"2024-09-21T07:35:27.867015+0000","last_became_peered":"2024-09-21T07:35:27.867015+0000","last_unstale":"2024-09-21T07:49:35.132219+0000","last_undegraded":"2024-09-21T07:49:35.132219+0000","last_fullsized":"2024-09-21T07:49:35.132219+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:59.214139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":338,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.265064+0000","last_change":"2024-09-21T07:35:29.493137+0000","last_active":"2024-09-21T07:49:37.265064+0000","last_peered":"2024-09-21T07:49:37.265064+0000","last_clean":"2024-09-21T07:49:37.265064+0000","last_became_active":"2024-09-21T07:35:26.872230+0000","last_became_peered":"2024-09-21T07:35:26.872230+0000","last_unstale":"2024-09-21T07:49:37.265064+0000","last_undegraded":"2024-09-21T07:49:37.265064+0000","last_fullsized":"2024-09-21T07:49:37.265064+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:24.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001197449,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":392,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.263897+0000","last_change":"2024-09-21T07:34:24.258754+0000","last_active":"2024-09-21T07:49:37.263897+0000","last_peered":"2024-09-21T07:49:37.263897+0000","last_clean":"2024-09-21T07:49:37.263897+0000","last_became_active":"2024-09-21T07:34:23.272657+0000","last_became_peered":"2024-09-21T07:34:23.272657+0000","last_unstale":"2024-09-21T07:49:37.263897+0000","last_undegraded":"2024-09-21T07:49:37.263897+0000","last_fullsized":"2024-09-21T07:49:37.263897+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:05.477099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067143200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264799+0000","last_change":"2024-09-21T07:49:35.146953+0000","last_active":"2024-09-21T07:49:37.264799+0000","last_peered":"2024-09-21T07:49:37.264799+0000","last_clean":"2024-09-21T07:49:37.264799+0000","last_became_active":"2024-09-21T07:49:35.146634+0000","last_became_peered":"2024-09-21T07:49:35.146634+0000","last_unstale":"2024-09-21T07:49:37.264799+0000","last_undegraded":"2024-09-21T07:49:37.264799+0000","last_fullsized":"2024-09-21T07:49:37.264799+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:35.687662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.965653+0000","last_change":"2024-09-21T07:49:34.133721+0000","last_active":"2024-09-21T07:49:36.965653+0000","last_peered":"2024-09-21T07:49:36.965653+0000","last_clean":"2024-09-21T07:49:36.965653+0000","last_became_active":"2024-09-21T07:49:34.133423+0000","last_became_peered":"2024-09-21T07:49:34.133423+0000","last_unstale":"2024-09-21T07:49:36.965653+0000","last_undegraded":"2024-09-21T07:49:36.965653+0000","last_fullsized":"2024-09-21T07:49:36.965653+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:36.825122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":333,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.965431+0000","last_change":"2024-09-21T07:35:27.876021+0000","last_active":"2024-09-21T07:49:36.965431+0000","last_peered":"2024-09-21T07:49:36.965431+0000","last_clean":"2024-09-21T07:49:36.965431+0000","last_became_active":"2024-09-21T07:35:27.875724+0000","last_became_peered":"2024-09-21T07:35:27.875724+0000","last_unstale":"2024-09-21T07:49:36.965431+0000","last_undegraded":"2024-09-21T07:49:36.965431+0000","last_fullsized":"2024-09-21T07:49:36.965431+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:03.022812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":338,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264673+0000","last_change":"2024-09-21T07:35:29.492570+0000","last_active":"2024-09-21T07:49:37.264673+0000","last_peered":"2024-09-21T07:49:37.264673+0000","last_clean":"2024-09-21T07:49:37.264673+0000","last_became_active":"2024-09-21T07:35:26.874316+0000","last_became_peered":"2024-09-21T07:35:26.874316+0000","last_unstale":"2024-09-21T07:49:37.264673+0000","last_undegraded":"2024-09-21T07:49:37.264673+0000","last_fullsized":"2024-09-21T07:49:37.264673+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:16.512323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011990760000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":392,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.965657+0000","last_change":"2024-09-21T07:34:25.085232+0000","last_active":"2024-09-21T07:49:36.965657+0000","last_peered":"2024-09-21T07:49:36.965657+0000","last_clean":"2024-09-21T07:49:36.965657+0000","last_became_active":"2024-09-21T07:34:23.277807+0000","last_became_peered":"2024-09-21T07:34:23.277807+0000","last_unstale":"2024-09-21T07:49:36.965657+0000","last_undegraded":"2024-09-21T07:49:36.965657+0000","last_fullsized":"2024-09-21T07:49:36.965657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:36.362919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086619099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.145452+0000","last_change":"2024-09-21T07:49:35.130292+0000","last_active":"2024-09-21T07:49:36.145452+0000","last_peered":"2024-09-21T07:49:36.145452+0000","last_clean":"2024-09-21T07:49:36.145452+0000","last_became_active":"2024-09-21T07:49:35.129962+0000","last_became_peered":"2024-09-21T07:49:35.129962+0000","last_unstale":"2024-09-21T07:49:36.145452+0000","last_undegraded":"2024-09-21T07:49:36.145452+0000","last_fullsized":"2024-09-21T07:49:36.145452+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:39.715572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"232'3","reported_seq":18,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264495+0000","last_change":"2024-09-21T07:49:34.132809+0000","last_active":"2024-09-21T07:49:37.264495+0000","last_peered":"2024-09-21T07:49:37.264495+0000","last_clean":"2024-09-21T07:49:37.264495+0000","last_became_active":"2024-09-21T07:49:34.132298+0000","last_became_peered":"2024-09-21T07:49:34.132298+0000","last_unstale":"2024-09-21T07:49:37.264495+0000","last_undegraded":"2024-09-21T07:49:37.264495+0000","last_fullsized":"2024-09-21T07:49:37.264495+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:18:11.773683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":335,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.266464+0000","last_change":"2024-09-21T07:35:27.872798+0000","last_active":"2024-09-21T07:49:37.266464+0000","last_peered":"2024-09-21T07:49:37.266464+0000","last_clean":"2024-09-21T07:49:37.266464+0000","last_became_active":"2024-09-21T07:35:27.872483+0000","last_became_peered":"2024-09-21T07:35:27.872483+0000","last_unstale":"2024-09-21T07:49:37.266464+0000","last_undegraded":"2024-09-21T07:49:37.266464+0000","last_fullsized":"2024-09-21T07:49:37.266464+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:11.933469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":342,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.265481+0000","last_change":"2024-09-21T07:35:29.206723+0000","last_active":"2024-09-21T07:49:37.265481+0000","last_peered":"2024-09-21T07:49:37.265481+0000","last_clean":"2024-09-21T07:49:37.265481+0000","last_became_active":"2024-09-21T07:35:26.872377+0000","last_became_peered":"2024-09-21T07:35:26.872377+0000","last_unstale":"2024-09-21T07:49:37.265481+0000","last_undegraded":"2024-09-21T07:49:37.265481+0000","last_fullsized":"2024-09-21T07:49:37.265481+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:29.086862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083077200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":383,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-21T07:49:35.139703+0000","last_change":"2024-09-21T07:34:24.597672+0000","last_active":"2024-09-21T07:49:35.139703+0000","last_peered":"2024-09-21T07:49:35.139703+0000","last_clean":"2024-09-21T07:49:35.139703+0000","last_became_active":"2024-09-21T07:34:23.267770+0000","last_became_peered":"2024-09-21T07:34:23.267770+0000","last_unstale":"2024-09-21T07:49:35.139703+0000","last_undegraded":"2024-09-21T07:49:35.139703+0000","last_fullsized":"2024-09-21T07:49:35.139703+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:51.629476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086378500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":10,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-21T07:49:35.149976+0000","last_change":"2024-09-21T07:49:35.143803+0000","last_active":"2024-09-21T07:49:35.149976+0000","last_peered":"2024-09-21T07:49:35.149976+0000","last_clean":"2024-09-21T07:49:35.149976+0000","last_became_active":"2024-09-21T07:49:35.143462+0000","last_became_peered":"2024-09-21T07:49:35.143462+0000","last_unstale":"2024-09-21T07:49:35.149976+0000","last_undegraded":"2024-09-21T07:49:35.149976+0000","last_fullsized":"2024-09-21T07:49:35.149976+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:14.249616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.14","version":"232'6","reported_seq":21,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.142425+0000","last_change":"2024-09-21T07:49:34.139691+0000","last_active":"2024-09-21T07:49:36.142425+0000","last_peered":"2024-09-21T07:49:36.142425+0000","last_clean":"2024-09-21T07:49:36.142425+0000","last_became_active":"2024-09-21T07:49:34.139228+0000","last_became_peered":"2024-09-21T07:49:34.139228+0000","last_unstale":"2024-09-21T07:49:36.142425+0000","last_undegraded":"2024-09-21T07:49:36.142425+0000","last_fullsized":"2024-09-21T07:49:36.142425+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:50:51.158824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":335,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.266157+0000","last_change":"2024-09-21T07:35:27.870121+0000","last_active":"2024-09-21T07:49:37.266157+0000","last_peered":"2024-09-21T07:49:37.266157+0000","last_clean":"2024-09-21T07:49:37.266157+0000","last_became_active":"2024-09-21T07:35:27.869811+0000","last_became_peered":"2024-09-21T07:35:27.869811+0000","last_unstale":"2024-09-21T07:49:37.266157+0000","last_undegraded":"2024-09-21T07:49:37.266157+0000","last_fullsized":"2024-09-21T07:49:37.266157+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:31.778461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":342,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143415+0000","last_change":"2024-09-21T07:35:28.860699+0000","last_active":"2024-09-21T07:49:36.143415+0000","last_peered":"2024-09-21T07:49:36.143415+0000","last_clean":"2024-09-21T07:49:36.143415+0000","last_became_active":"2024-09-21T07:35:26.876779+0000","last_became_peered":"2024-09-21T07:35:26.876779+0000","last_unstale":"2024-09-21T07:49:36.143415+0000","last_undegraded":"2024-09-21T07:49:36.143415+0000","last_fullsized":"2024-09-21T07:49:36.143415+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:28.286403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000913188,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":394,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.145914+0000","last_change":"2024-09-21T07:34:24.260181+0000","last_active":"2024-09-21T07:49:36.145914+0000","last_peered":"2024-09-21T07:49:36.145914+0000","last_clean":"2024-09-21T07:49:36.145914+0000","last_became_active":"2024-09-21T07:34:23.275580+0000","last_became_peered":"2024-09-21T07:34:23.275580+0000","last_unstale":"2024-09-21T07:49:36.145914+0000","last_undegraded":"2024-09-21T07:49:36.145914+0000","last_fullsized":"2024-09-21T07:49:36.145914+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:49.857622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014706980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.122197+0000","last_change":"2024-09-21T07:49:35.146412+0000","last_active":"2024-09-21T07:49:37.122197+0000","last_peered":"2024-09-21T07:49:37.122197+0000","last_clean":"2024-09-21T07:49:37.122197+0000","last_became_active":"2024-09-21T07:49:35.146118+0000","last_became_peered":"2024-09-21T07:49:35.146118+0000","last_unstale":"2024-09-21T07:49:37.122197+0000","last_undegraded":"2024-09-21T07:49:37.122197+0000","last_fullsized":"2024-09-21T07:49:37.122197+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:40.422511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.265996+0000","last_change":"2024-09-21T07:49:34.132216+0000","last_active":"2024-09-21T07:49:37.265996+0000","last_peered":"2024-09-21T07:49:37.265996+0000","last_clean":"2024-09-21T07:49:37.265996+0000","last_became_active":"2024-09-21T07:49:34.131867+0000","last_became_peered":"2024-09-21T07:49:34.131867+0000","last_unstale":"2024-09-21T07:49:37.265996+0000","last_undegraded":"2024-09-21T07:49:37.265996+0000","last_fullsized":"2024-09-21T07:49:37.265996+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:03:21.509136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":336,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.122033+0000","last_change":"2024-09-21T07:35:27.875393+0000","last_active":"2024-09-21T07:49:37.122033+0000","last_peered":"2024-09-21T07:49:37.122033+0000","last_clean":"2024-09-21T07:49:37.122033+0000","last_became_active":"2024-09-21T07:35:27.874671+0000","last_became_peered":"2024-09-21T07:35:27.874671+0000","last_unstale":"2024-09-21T07:49:37.122033+0000","last_undegraded":"2024-09-21T07:49:37.122033+0000","last_fullsized":"2024-09-21T07:49:37.122033+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":342,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264970+0000","last_change":"2024-09-21T07:35:29.206296+0000","last_active":"2024-09-21T07:49:37.264970+0000","last_peered":"2024-09-21T07:49:37.264970+0000","last_clean":"2024-09-21T07:49:37.264970+0000","last_became_active":"2024-09-21T07:35:26.874575+0000","last_became_peered":"2024-09-21T07:35:26.874575+0000","last_unstale":"2024-09-21T07:49:37.264970+0000","last_undegraded":"2024-09-21T07:49:37.264970+0000","last_fullsized":"2024-09-21T07:49:37.264970+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:22.771154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061011599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":395,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.144697+0000","last_change":"2024-09-21T07:34:24.259112+0000","last_active":"2024-09-21T07:49:36.144697+0000","last_peered":"2024-09-21T07:49:36.144697+0000","last_clean":"2024-09-21T07:49:36.144697+0000","last_became_active":"2024-09-21T07:34:23.635444+0000","last_became_peered":"2024-09-21T07:34:23.635444+0000","last_unstale":"2024-09-21T07:49:36.144697+0000","last_undegraded":"2024-09-21T07:49:36.144697+0000","last_fullsized":"2024-09-21T07:49:36.144697+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:46.255385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001952636,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.122538+0000","last_change":"2024-09-21T07:49:35.149455+0000","last_active":"2024-09-21T07:49:37.122538+0000","last_peered":"2024-09-21T07:49:37.122538+0000","last_clean":"2024-09-21T07:49:37.122538+0000","last_became_active":"2024-09-21T07:49:35.149005+0000","last_became_peered":"2024-09-21T07:49:35.149005+0000","last_unstale":"2024-09-21T07:49:37.122538+0000","last_undegraded":"2024-09-21T07:49:37.122538+0000","last_fullsized":"2024-09-21T07:49:37.122538+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:07.998627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":13,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-21T07:49:35.138561+0000","last_change":"2024-09-21T07:49:34.144249+0000","last_active":"2024-09-21T07:49:35.138561+0000","last_peered":"2024-09-21T07:49:35.138561+0000","last_clean":"2024-09-21T07:49:35.138561+0000","last_became_active":"2024-09-21T07:49:34.143806+0000","last_became_peered":"2024-09-21T07:49:34.143806+0000","last_unstale":"2024-09-21T07:49:35.138561+0000","last_undegraded":"2024-09-21T07:49:35.138561+0000","last_fullsized":"2024-09-21T07:49:35.138561+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:51:12.362241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":336,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.122233+0000","last_change":"2024-09-21T07:35:27.878356+0000","last_active":"2024-09-21T07:49:37.122233+0000","last_peered":"2024-09-21T07:49:37.122233+0000","last_clean":"2024-09-21T07:49:37.122233+0000","last_became_active":"2024-09-21T07:35:27.877982+0000","last_became_peered":"2024-09-21T07:35:27.877982+0000","last_unstale":"2024-09-21T07:49:37.122233+0000","last_undegraded":"2024-09-21T07:49:37.122233+0000","last_fullsized":"2024-09-21T07:49:37.122233+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":344,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.121317+0000","last_change":"2024-09-21T07:35:29.053131+0000","last_active":"2024-09-21T07:49:37.121317+0000","last_peered":"2024-09-21T07:49:37.121317+0000","last_clean":"2024-09-21T07:49:37.121317+0000","last_became_active":"2024-09-21T07:35:26.870516+0000","last_became_peered":"2024-09-21T07:35:26.870516+0000","last_unstale":"2024-09-21T07:49:37.121317+0000","last_undegraded":"2024-09-21T07:49:37.121317+0000","last_fullsized":"2024-09-21T07:49:37.121317+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:10.924491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070705699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":407,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143191+0000","last_change":"2024-09-21T07:34:24.539152+0000","last_active":"2024-09-21T07:49:36.143191+0000","last_peered":"2024-09-21T07:49:36.143191+0000","last_clean":"2024-09-21T07:49:36.143191+0000","last_became_active":"2024-09-21T07:34:23.259573+0000","last_became_peered":"2024-09-21T07:34:23.259573+0000","last_unstale":"2024-09-21T07:49:36.143191+0000","last_undegraded":"2024-09-21T07:49:36.143191+0000","last_fullsized":"2024-09-21T07:49:36.143191+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:26.986900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058108400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.965393+0000","last_change":"2024-09-21T07:49:35.144737+0000","last_active":"2024-09-21T07:49:36.965393+0000","last_peered":"2024-09-21T07:49:36.965393+0000","last_clean":"2024-09-21T07:49:36.965393+0000","last_became_active":"2024-09-21T07:49:35.144412+0000","last_became_peered":"2024-09-21T07:49:35.144412+0000","last_unstale":"2024-09-21T07:49:36.965393+0000","last_undegraded":"2024-09-21T07:49:36.965393+0000","last_fullsized":"2024-09-21T07:49:36.965393+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:30.582221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.121530+0000","last_change":"2024-09-21T07:49:34.131998+0000","last_active":"2024-09-21T07:49:37.121530+0000","last_peered":"2024-09-21T07:49:37.121530+0000","last_clean":"2024-09-21T07:49:37.121530+0000","last_became_active":"2024-09-21T07:49:34.131674+0000","last_became_peered":"2024-09-21T07:49:34.131674+0000","last_unstale":"2024-09-21T07:49:37.121530+0000","last_undegraded":"2024-09-21T07:49:37.121530+0000","last_fullsized":"2024-09-21T07:49:37.121530+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:04.502261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":335,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.121267+0000","last_change":"2024-09-21T07:35:27.873726+0000","last_active":"2024-09-21T07:49:37.121267+0000","last_peered":"2024-09-21T07:49:37.121267+0000","last_clean":"2024-09-21T07:49:37.121267+0000","last_became_active":"2024-09-21T07:35:27.871421+0000","last_became_peered":"2024-09-21T07:35:27.871421+0000","last_unstale":"2024-09-21T07:49:37.121267+0000","last_undegraded":"2024-09-21T07:49:37.121267+0000","last_fullsized":"2024-09-21T07:49:37.121267+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:00.168915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":342,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.145722+0000","last_change":"2024-09-21T07:35:28.861032+0000","last_active":"2024-09-21T07:49:36.145722+0000","last_peered":"2024-09-21T07:49:36.145722+0000","last_clean":"2024-09-21T07:49:36.145722+0000","last_became_active":"2024-09-21T07:35:26.873149+0000","last_became_peered":"2024-09-21T07:35:26.873149+0000","last_unstale":"2024-09-21T07:49:36.145722+0000","last_undegraded":"2024-09-21T07:49:36.145722+0000","last_fullsized":"2024-09-21T07:49:36.145722+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:59.435981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011988369999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":391,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264585+0000","last_change":"2024-09-21T07:34:24.259902+0000","last_active":"2024-09-21T07:49:37.264585+0000","last_peered":"2024-09-21T07:49:37.264585+0000","last_clean":"2024-09-21T07:49:37.264585+0000","last_became_active":"2024-09-21T07:34:23.271902+0000","last_became_peered":"2024-09-21T07:34:23.271902+0000","last_unstale":"2024-09-21T07:49:37.264585+0000","last_undegraded":"2024-09-21T07:49:37.264585+0000","last_fullsized":"2024-09-21T07:49:37.264585+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:59.439525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001686358,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.144373+0000","last_change":"2024-09-21T07:49:35.138262+0000","last_active":"2024-09-21T07:49:36.144373+0000","last_peered":"2024-09-21T07:49:36.144373+0000","last_clean":"2024-09-21T07:49:36.144373+0000","last_became_active":"2024-09-21T07:49:35.136582+0000","last_became_peered":"2024-09-21T07:49:35.136582+0000","last_unstale":"2024-09-21T07:49:36.144373+0000","last_undegraded":"2024-09-21T07:49:36.144373+0000","last_fullsized":"2024-09-21T07:49:36.144373+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:45.528921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143616+0000","last_change":"2024-09-21T07:49:34.140307+0000","last_active":"2024-09-21T07:49:36.143616+0000","last_peered":"2024-09-21T07:49:36.143616+0000","last_clean":"2024-09-21T07:49:36.143616+0000","last_became_active":"2024-09-21T07:49:34.139959+0000","last_became_peered":"2024-09-21T07:49:34.139959+0000","last_unstale":"2024-09-21T07:49:36.143616+0000","last_undegraded":"2024-09-21T07:49:36.143616+0000","last_fullsized":"2024-09-21T07:49:36.143616+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:02:01.638863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":332,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.965076+0000","last_change":"2024-09-21T07:35:27.872637+0000","last_active":"2024-09-21T07:49:36.965076+0000","last_peered":"2024-09-21T07:49:36.965076+0000","last_clean":"2024-09-21T07:49:36.965076+0000","last_became_active":"2024-09-21T07:35:27.872149+0000","last_became_peered":"2024-09-21T07:35:27.872149+0000","last_unstale":"2024-09-21T07:49:36.965076+0000","last_undegraded":"2024-09-21T07:49:36.965076+0000","last_fullsized":"2024-09-21T07:49:36.965076+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:27.697872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":339,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264748+0000","last_change":"2024-09-21T07:35:29.493362+0000","last_active":"2024-09-21T07:49:37.264748+0000","last_peered":"2024-09-21T07:49:37.264748+0000","last_clean":"2024-09-21T07:49:37.264748+0000","last_became_active":"2024-09-21T07:35:26.873018+0000","last_became_peered":"2024-09-21T07:35:26.873018+0000","last_unstale":"2024-09-21T07:49:37.264748+0000","last_undegraded":"2024-09-21T07:49:37.264748+0000","last_fullsized":"2024-09-21T07:49:37.264748+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:02.460557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012496510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":395,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.144241+0000","last_change":"2024-09-21T07:34:24.259662+0000","last_active":"2024-09-21T07:49:36.144241+0000","last_peered":"2024-09-21T07:49:36.144241+0000","last_clean":"2024-09-21T07:49:36.144241+0000","last_became_active":"2024-09-21T07:34:23.278812+0000","last_became_peered":"2024-09-21T07:34:23.278812+0000","last_unstale":"2024-09-21T07:49:36.144241+0000","last_undegraded":"2024-09-21T07:49:36.144241+0000","last_fullsized":"2024-09-21T07:49:36.144241+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:49.829974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356463,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264200+0000","last_change":"2024-09-21T07:49:35.143792+0000","last_active":"2024-09-21T07:49:37.264200+0000","last_peered":"2024-09-21T07:49:37.264200+0000","last_clean":"2024-09-21T07:49:37.264200+0000","last_became_active":"2024-09-21T07:49:35.143413+0000","last_became_peered":"2024-09-21T07:49:35.143413+0000","last_unstale":"2024-09-21T07:49:37.264200+0000","last_undegraded":"2024-09-21T07:49:37.264200+0000","last_fullsized":"2024-09-21T07:49:37.264200+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:03:29.921904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"232'2","reported_seq":17,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.265298+0000","last_change":"2024-09-21T07:49:34.134681+0000","last_active":"2024-09-21T07:49:37.265298+0000","last_peered":"2024-09-21T07:49:37.265298+0000","last_clean":"2024-09-21T07:49:37.265298+0000","last_became_active":"2024-09-21T07:49:34.134299+0000","last_became_peered":"2024-09-21T07:49:34.134299+0000","last_unstale":"2024-09-21T07:49:37.265298+0000","last_undegraded":"2024-09-21T07:49:37.265298+0000","last_fullsized":"2024-09-21T07:49:37.265298+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:34.178661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":326,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-21T07:49:35.132115+0000","last_change":"2024-09-21T07:35:27.868448+0000","last_active":"2024-09-21T07:49:35.132115+0000","last_peered":"2024-09-21T07:49:35.132115+0000","last_clean":"2024-09-21T07:49:35.132115+0000","last_became_active":"2024-09-21T07:35:27.868105+0000","last_became_peered":"2024-09-21T07:35:27.868105+0000","last_unstale":"2024-09-21T07:49:35.132115+0000","last_undegraded":"2024-09-21T07:49:35.132115+0000","last_fullsized":"2024-09-21T07:49:35.132115+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:41.468702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":337,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264584+0000","last_change":"2024-09-21T07:35:29.492974+0000","last_active":"2024-09-21T07:49:37.264584+0000","last_peered":"2024-09-21T07:49:37.264584+0000","last_clean":"2024-09-21T07:49:37.264584+0000","last_became_active":"2024-09-21T07:35:26.872165+0000","last_became_peered":"2024-09-21T07:35:26.872165+0000","last_unstale":"2024-09-21T07:49:37.264584+0000","last_undegraded":"2024-09-21T07:49:37.264584+0000","last_fullsized":"2024-09-21T07:49:37.264584+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:59.912520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013795179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":391,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.965141+0000","last_change":"2024-09-21T07:34:25.084861+0000","last_active":"2024-09-21T07:49:36.965141+0000","last_peered":"2024-09-21T07:49:36.965141+0000","last_clean":"2024-09-21T07:49:36.965141+0000","last_became_active":"2024-09-21T07:34:23.277708+0000","last_became_peered":"2024-09-21T07:34:23.277708+0000","last_unstale":"2024-09-21T07:49:36.965141+0000","last_undegraded":"2024-09-21T07:49:36.965141+0000","last_fullsized":"2024-09-21T07:49:36.965141+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:40.816488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000597566,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264247+0000","last_change":"2024-09-21T07:49:35.144350+0000","last_active":"2024-09-21T07:49:37.264247+0000","last_peered":"2024-09-21T07:49:37.264247+0000","last_clean":"2024-09-21T07:49:37.264247+0000","last_became_active":"2024-09-21T07:49:35.141888+0000","last_became_peered":"2024-09-21T07:49:35.141888+0000","last_unstale":"2024-09-21T07:49:37.264247+0000","last_undegraded":"2024-09-21T07:49:37.264247+0000","last_fullsized":"2024-09-21T07:49:37.264247+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:46:36.677708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.121702+0000","last_change":"2024-09-21T07:49:34.136417+0000","last_active":"2024-09-21T07:49:37.121702+0000","last_peered":"2024-09-21T07:49:37.121702+0000","last_clean":"2024-09-21T07:49:37.121702+0000","last_became_active":"2024-09-21T07:49:34.132982+0000","last_became_peered":"2024-09-21T07:49:34.132982+0000","last_unstale":"2024-09-21T07:49:37.121702+0000","last_undegraded":"2024-09-21T07:49:37.121702+0000","last_fullsized":"2024-09-21T07:49:37.121702+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:46.034969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":326,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-21T07:49:35.133820+0000","last_change":"2024-09-21T07:35:27.874640+0000","last_active":"2024-09-21T07:49:35.133820+0000","last_peered":"2024-09-21T07:49:35.133820+0000","last_clean":"2024-09-21T07:49:35.133820+0000","last_became_active":"2024-09-21T07:35:27.874265+0000","last_became_peered":"2024-09-21T07:35:27.874265+0000","last_unstale":"2024-09-21T07:49:35.133820+0000","last_undegraded":"2024-09-21T07:49:35.133820+0000","last_fullsized":"2024-09-21T07:49:35.133820+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:45.754654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":341,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.266678+0000","last_change":"2024-09-21T07:35:29.207392+0000","last_active":"2024-09-21T07:49:37.266678+0000","last_peered":"2024-09-21T07:49:37.266678+0000","last_clean":"2024-09-21T07:49:37.266678+0000","last_became_active":"2024-09-21T07:35:26.870917+0000","last_became_peered":"2024-09-21T07:35:26.870917+0000","last_unstale":"2024-09-21T07:49:37.266678+0000","last_undegraded":"2024-09-21T07:49:37.266678+0000","last_fullsized":"2024-09-21T07:49:37.266678+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:59.159937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059198499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":395,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143926+0000","last_change":"2024-09-21T07:34:24.258709+0000","last_active":"2024-09-21T07:49:36.143926+0000","last_peered":"2024-09-21T07:49:36.143926+0000","last_clean":"2024-09-21T07:49:36.143926+0000","last_became_active":"2024-09-21T07:34:23.279042+0000","last_became_peered":"2024-09-21T07:34:23.279042+0000","last_unstale":"2024-09-21T07:49:36.143926+0000","last_undegraded":"2024-09-21T07:49:36.143926+0000","last_fullsized":"2024-09-21T07:49:36.143926+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:01.762117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019282100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.145517+0000","last_change":"2024-09-21T07:49:35.154518+0000","last_active":"2024-09-21T07:49:36.145517+0000","last_peered":"2024-09-21T07:49:36.145517+0000","last_clean":"2024-09-21T07:49:36.145517+0000","last_became_active":"2024-09-21T07:49:35.154227+0000","last_became_peered":"2024-09-21T07:49:35.154227+0000","last_unstale":"2024-09-21T07:49:36.145517+0000","last_undegraded":"2024-09-21T07:49:36.145517+0000","last_fullsized":"2024-09-21T07:49:36.145517+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:44.338565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"232'1","reported_seq":16,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.266431+0000","last_change":"2024-09-21T07:49:34.136167+0000","last_active":"2024-09-21T07:49:37.266431+0000","last_peered":"2024-09-21T07:49:37.266431+0000","last_clean":"2024-09-21T07:49:37.266431+0000","last_became_active":"2024-09-21T07:49:34.133552+0000","last_became_peered":"2024-09-21T07:49:34.133552+0000","last_unstale":"2024-09-21T07:49:37.266431+0000","last_undegraded":"2024-09-21T07:49:37.266431+0000","last_fullsized":"2024-09-21T07:49:37.266431+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:20.047656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":332,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.965457+0000","last_change":"2024-09-21T07:35:27.870260+0000","last_active":"2024-09-21T07:49:36.965457+0000","last_peered":"2024-09-21T07:49:36.965457+0000","last_clean":"2024-09-21T07:49:36.965457+0000","last_became_active":"2024-09-21T07:35:27.869448+0000","last_became_peered":"2024-09-21T07:35:27.869448+0000","last_unstale":"2024-09-21T07:49:36.965457+0000","last_undegraded":"2024-09-21T07:49:36.965457+0000","last_fullsized":"2024-09-21T07:49:36.965457+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:16.946790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":333,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-21T07:49:35.132407+0000","last_change":"2024-09-21T07:35:29.386367+0000","last_active":"2024-09-21T07:49:35.132407+0000","last_peered":"2024-09-21T07:49:35.132407+0000","last_clean":"2024-09-21T07:49:35.132407+0000","last_became_active":"2024-09-21T07:35:26.879238+0000","last_became_peered":"2024-09-21T07:35:26.879238+0000","last_unstale":"2024-09-21T07:49:35.132407+0000","last_undegraded":"2024-09-21T07:49:35.132407+0000","last_fullsized":"2024-09-21T07:49:35.132407+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:16.528260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092223100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":391,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.966024+0000","last_change":"2024-09-21T07:34:25.085635+0000","last_active":"2024-09-21T07:49:36.966024+0000","last_peered":"2024-09-21T07:49:36.966024+0000","last_clean":"2024-09-21T07:49:36.966024+0000","last_became_active":"2024-09-21T07:34:23.273566+0000","last_became_peered":"2024-09-21T07:34:23.273566+0000","last_unstale":"2024-09-21T07:49:36.966024+0000","last_undegraded":"2024-09-21T07:49:36.966024+0000","last_fullsized":"2024-09-21T07:49:36.966024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:14.142272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079897300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.267075+0000","last_change":"2024-09-21T07:49:35.154606+0000","last_active":"2024-09-21T07:49:37.267075+0000","last_peered":"2024-09-21T07:49:37.267075+0000","last_clean":"2024-09-21T07:49:37.267075+0000","last_became_active":"2024-09-21T07:49:35.154308+0000","last_became_peered":"2024-09-21T07:49:35.154308+0000","last_unstale":"2024-09-21T07:49:37.267075+0000","last_undegraded":"2024-09-21T07:49:37.267075+0000","last_fullsized":"2024-09-21T07:49:37.267075+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:45:37.524277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.265993+0000","last_change":"2024-09-21T07:49:34.136959+0000","last_active":"2024-09-21T07:49:37.265993+0000","last_peered":"2024-09-21T07:49:37.265993+0000","last_clean":"2024-09-21T07:49:37.265993+0000","last_became_active":"2024-09-21T07:49:34.133732+0000","last_became_peered":"2024-09-21T07:49:34.133732+0000","last_unstale":"2024-09-21T07:49:37.265993+0000","last_undegraded":"2024-09-21T07:49:37.265993+0000","last_fullsized":"2024-09-21T07:49:37.265993+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:46:08.491878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":332,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.965897+0000","last_change":"2024-09-21T07:35:27.878728+0000","last_active":"2024-09-21T07:49:36.965897+0000","last_peered":"2024-09-21T07:49:36.965897+0000","last_clean":"2024-09-21T07:49:36.965897+0000","last_became_active":"2024-09-21T07:35:27.878335+0000","last_became_peered":"2024-09-21T07:35:27.878335+0000","last_unstale":"2024-09-21T07:49:36.965897+0000","last_undegraded":"2024-09-21T07:49:36.965897+0000","last_fullsized":"2024-09-21T07:49:36.965897+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:25:37.731771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":354,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.142459+0000","last_change":"2024-09-21T07:35:28.858418+0000","last_active":"2024-09-21T07:49:36.142459+0000","last_peered":"2024-09-21T07:49:36.142459+0000","last_clean":"2024-09-21T07:49:36.142459+0000","last_became_active":"2024-09-21T07:35:26.876944+0000","last_became_peered":"2024-09-21T07:35:26.876944+0000","last_unstale":"2024-09-21T07:49:36.142459+0000","last_undegraded":"2024-09-21T07:49:36.142459+0000","last_fullsized":"2024-09-21T07:49:36.142459+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:04.514304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090862799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":383,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-21T07:49:35.137893+0000","last_change":"2024-09-21T07:34:24.597926+0000","last_active":"2024-09-21T07:49:35.137893+0000","last_peered":"2024-09-21T07:49:35.137893+0000","last_clean":"2024-09-21T07:49:35.137893+0000","last_became_active":"2024-09-21T07:34:23.268926+0000","last_became_peered":"2024-09-21T07:34:23.268926+0000","last_unstale":"2024-09-21T07:49:35.137893+0000","last_undegraded":"2024-09-21T07:49:35.137893+0000","last_fullsized":"2024-09-21T07:49:35.137893+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:46.190752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101851,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.966109+0000","last_change":"2024-09-21T07:49:35.161820+0000","last_active":"2024-09-21T07:49:36.966109+0000","last_peered":"2024-09-21T07:49:36.966109+0000","last_clean":"2024-09-21T07:49:36.966109+0000","last_became_active":"2024-09-21T07:49:35.161521+0000","last_became_peered":"2024-09-21T07:49:35.161521+0000","last_unstale":"2024-09-21T07:49:36.966109+0000","last_undegraded":"2024-09-21T07:49:36.966109+0000","last_fullsized":"2024-09-21T07:49:36.966109+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:23.315071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"232'1","reported_seq":16,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.145703+0000","last_change":"2024-09-21T07:49:34.136928+0000","last_active":"2024-09-21T07:49:36.145703+0000","last_peered":"2024-09-21T07:49:36.145703+0000","last_clean":"2024-09-21T07:49:36.145703+0000","last_became_active":"2024-09-21T07:49:34.136627+0000","last_became_peered":"2024-09-21T07:49:34.136627+0000","last_unstale":"2024-09-21T07:49:36.145703+0000","last_undegraded":"2024-09-21T07:49:36.145703+0000","last_fullsized":"2024-09-21T07:49:36.145703+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:13.445455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":328,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-21T07:49:35.138115+0000","last_change":"2024-09-21T07:35:27.877426+0000","last_active":"2024-09-21T07:49:35.138115+0000","last_peered":"2024-09-21T07:49:35.138115+0000","last_clean":"2024-09-21T07:49:35.138115+0000","last_became_active":"2024-09-21T07:35:27.877050+0000","last_became_peered":"2024-09-21T07:35:27.877050+0000","last_unstale":"2024-09-21T07:49:35.138115+0000","last_undegraded":"2024-09-21T07:49:35.138115+0000","last_fullsized":"2024-09-21T07:49:35.138115+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:36.268334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":339,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.965170+0000","last_change":"2024-09-21T07:35:28.862273+0000","last_active":"2024-09-21T07:49:36.965170+0000","last_peered":"2024-09-21T07:49:36.965170+0000","last_clean":"2024-09-21T07:49:36.965170+0000","last_became_active":"2024-09-21T07:35:26.874011+0000","last_became_peered":"2024-09-21T07:35:26.874011+0000","last_unstale":"2024-09-21T07:49:36.965170+0000","last_undegraded":"2024-09-21T07:49:36.965170+0000","last_fullsized":"2024-09-21T07:49:36.965170+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:11.986950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071644799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":394,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.144344+0000","last_change":"2024-09-21T07:34:24.259794+0000","last_active":"2024-09-21T07:49:36.144344+0000","last_peered":"2024-09-21T07:49:36.144344+0000","last_clean":"2024-09-21T07:49:36.144344+0000","last_became_active":"2024-09-21T07:34:23.265679+0000","last_became_peered":"2024-09-21T07:34:23.265679+0000","last_unstale":"2024-09-21T07:49:36.144344+0000","last_undegraded":"2024-09-21T07:49:36.144344+0000","last_fullsized":"2024-09-21T07:49:36.144344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:02.575796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001298518,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.265782+0000","last_change":"2024-09-21T07:49:35.140846+0000","last_active":"2024-09-21T07:49:37.265782+0000","last_peered":"2024-09-21T07:49:37.265782+0000","last_clean":"2024-09-21T07:49:37.265782+0000","last_became_active":"2024-09-21T07:49:35.140507+0000","last_became_peered":"2024-09-21T07:49:35.140507+0000","last_unstale":"2024-09-21T07:49:37.265782+0000","last_undegraded":"2024-09-21T07:49:37.265782+0000","last_fullsized":"2024-09-21T07:49:37.265782+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:02.011482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.e","version":"232'2","reported_seq":17,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.144890+0000","last_change":"2024-09-21T07:49:34.139443+0000","last_active":"2024-09-21T07:49:36.144890+0000","last_peered":"2024-09-21T07:49:36.144890+0000","last_clean":"2024-09-21T07:49:36.144890+0000","last_became_active":"2024-09-21T07:49:34.139011+0000","last_became_peered":"2024-09-21T07:49:34.139011+0000","last_unstale":"2024-09-21T07:49:36.144890+0000","last_undegraded":"2024-09-21T07:49:36.144890+0000","last_fullsized":"2024-09-21T07:49:36.144890+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:39.850584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":335,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.267189+0000","last_change":"2024-09-21T07:35:27.877725+0000","last_active":"2024-09-21T07:49:37.267189+0000","last_peered":"2024-09-21T07:49:37.267189+0000","last_clean":"2024-09-21T07:49:37.267189+0000","last_became_active":"2024-09-21T07:35:27.877398+0000","last_became_peered":"2024-09-21T07:35:27.877398+0000","last_unstale":"2024-09-21T07:49:37.267189+0000","last_undegraded":"2024-09-21T07:49:37.267189+0000","last_fullsized":"2024-09-21T07:49:37.267189+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:05.859762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":341,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143689+0000","last_change":"2024-09-21T07:35:28.860612+0000","last_active":"2024-09-21T07:49:36.143689+0000","last_peered":"2024-09-21T07:49:36.143689+0000","last_clean":"2024-09-21T07:49:36.143689+0000","last_became_active":"2024-09-21T07:35:26.876562+0000","last_became_peered":"2024-09-21T07:35:26.876562+0000","last_unstale":"2024-09-21T07:49:36.143689+0000","last_undegraded":"2024-09-21T07:49:36.143689+0000","last_fullsized":"2024-09-21T07:49:36.143689+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:06.824331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012252470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":391,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.265245+0000","last_change":"2024-09-21T07:34:24.260330+0000","last_active":"2024-09-21T07:49:37.265245+0000","last_peered":"2024-09-21T07:49:37.265245+0000","last_clean":"2024-09-21T07:49:37.265245+0000","last_became_active":"2024-09-21T07:34:23.637210+0000","last_became_peered":"2024-09-21T07:34:23.637210+0000","last_unstale":"2024-09-21T07:49:37.265245+0000","last_undegraded":"2024-09-21T07:49:37.265245+0000","last_fullsized":"2024-09-21T07:49:37.265245+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.488451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001537093,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143221+0000","last_change":"2024-09-21T07:49:35.141820+0000","last_active":"2024-09-21T07:49:36.143221+0000","last_peered":"2024-09-21T07:49:36.143221+0000","last_clean":"2024-09-21T07:49:36.143221+0000","last_became_active":"2024-09-21T07:49:35.140201+0000","last_became_peered":"2024-09-21T07:49:35.140201+0000","last_unstale":"2024-09-21T07:49:36.143221+0000","last_undegraded":"2024-09-21T07:49:36.143221+0000","last_fullsized":"2024-09-21T07:49:36.143221+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:55:12.992135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264088+0000","last_change":"2024-09-21T07:49:34.128778+0000","last_active":"2024-09-21T07:49:37.264088+0000","last_peered":"2024-09-21T07:49:37.264088+0000","last_clean":"2024-09-21T07:49:37.264088+0000","last_became_active":"2024-09-21T07:49:34.128445+0000","last_became_peered":"2024-09-21T07:49:34.128445+0000","last_unstale":"2024-09-21T07:49:37.264088+0000","last_undegraded":"2024-09-21T07:49:37.264088+0000","last_fullsized":"2024-09-21T07:49:37.264088+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:22.246646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":333,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.964989+0000","last_change":"2024-09-21T07:35:27.874757+0000","last_active":"2024-09-21T07:49:36.964989+0000","last_peered":"2024-09-21T07:49:36.964989+0000","last_clean":"2024-09-21T07:49:36.964989+0000","last_became_active":"2024-09-21T07:35:27.874360+0000","last_became_peered":"2024-09-21T07:35:27.874360+0000","last_unstale":"2024-09-21T07:49:36.964989+0000","last_undegraded":"2024-09-21T07:49:36.964989+0000","last_fullsized":"2024-09-21T07:49:36.964989+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:17.646912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":337,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.266231+0000","last_change":"2024-09-21T07:35:29.493431+0000","last_active":"2024-09-21T07:49:37.266231+0000","last_peered":"2024-09-21T07:49:37.266231+0000","last_clean":"2024-09-21T07:49:37.266231+0000","last_became_active":"2024-09-21T07:35:26.873691+0000","last_became_peered":"2024-09-21T07:35:26.873691+0000","last_unstale":"2024-09-21T07:49:37.266231+0000","last_undegraded":"2024-09-21T07:49:37.266231+0000","last_fullsized":"2024-09-21T07:49:37.266231+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:51.705051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001162464,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":383,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-21T07:49:35.132319+0000","last_change":"2024-09-21T07:34:24.597286+0000","last_active":"2024-09-21T07:49:35.132319+0000","last_peered":"2024-09-21T07:49:35.132319+0000","last_clean":"2024-09-21T07:49:35.132319+0000","last_became_active":"2024-09-21T07:34:23.267133+0000","last_became_peered":"2024-09-21T07:34:23.267133+0000","last_unstale":"2024-09-21T07:49:35.132319+0000","last_undegraded":"2024-09-21T07:49:35.132319+0000","last_fullsized":"2024-09-21T07:49:35.132319+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:18.257010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264440+0000","last_change":"2024-09-21T07:49:35.152801+0000","last_active":"2024-09-21T07:49:37.264440+0000","last_peered":"2024-09-21T07:49:37.264440+0000","last_clean":"2024-09-21T07:49:37.264440+0000","last_became_active":"2024-09-21T07:49:35.151957+0000","last_became_peered":"2024-09-21T07:49:35.151957+0000","last_unstale":"2024-09-21T07:49:37.264440+0000","last_undegraded":"2024-09-21T07:49:37.264440+0000","last_fullsized":"2024-09-21T07:49:37.264440+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:54.437494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.265794+0000","last_change":"2024-09-21T07:49:34.139221+0000","last_active":"2024-09-21T07:49:37.265794+0000","last_peered":"2024-09-21T07:49:37.265794+0000","last_clean":"2024-09-21T07:49:37.265794+0000","last_became_active":"2024-09-21T07:49:34.136048+0000","last_became_peered":"2024-09-21T07:49:34.136048+0000","last_unstale":"2024-09-21T07:49:37.265794+0000","last_undegraded":"2024-09-21T07:49:37.265794+0000","last_fullsized":"2024-09-21T07:49:37.265794+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:39:02.927347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":348,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.141843+0000","last_change":"2024-09-21T07:35:27.864458+0000","last_active":"2024-09-21T07:49:36.141843+0000","last_peered":"2024-09-21T07:49:36.141843+0000","last_clean":"2024-09-21T07:49:36.141843+0000","last_became_active":"2024-09-21T07:35:27.864136+0000","last_became_peered":"2024-09-21T07:35:27.864136+0000","last_unstale":"2024-09-21T07:49:36.141843+0000","last_undegraded":"2024-09-21T07:49:36.141843+0000","last_fullsized":"2024-09-21T07:49:36.141843+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:20.162240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":354,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143019+0000","last_change":"2024-09-21T07:35:28.858784+0000","last_active":"2024-09-21T07:49:36.143019+0000","last_peered":"2024-09-21T07:49:36.143019+0000","last_clean":"2024-09-21T07:49:36.143019+0000","last_became_active":"2024-09-21T07:35:26.876890+0000","last_became_peered":"2024-09-21T07:35:26.876890+0000","last_unstale":"2024-09-21T07:49:36.143019+0000","last_undegraded":"2024-09-21T07:49:36.143019+0000","last_fullsized":"2024-09-21T07:49:36.143019+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:40.738327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064543499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":395,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.144123+0000","last_change":"2024-09-21T07:34:24.258712+0000","last_active":"2024-09-21T07:49:36.144123+0000","last_peered":"2024-09-21T07:49:36.144123+0000","last_clean":"2024-09-21T07:49:36.144123+0000","last_became_active":"2024-09-21T07:34:23.274692+0000","last_became_peered":"2024-09-21T07:34:23.274692+0000","last_unstale":"2024-09-21T07:49:36.144123+0000","last_undegraded":"2024-09-21T07:49:36.144123+0000","last_fullsized":"2024-09-21T07:49:36.144123+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:43.279454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018051479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143469+0000","last_change":"2024-09-21T07:49:35.142235+0000","last_active":"2024-09-21T07:49:36.143469+0000","last_peered":"2024-09-21T07:49:36.143469+0000","last_clean":"2024-09-21T07:49:36.143469+0000","last_became_active":"2024-09-21T07:49:35.141898+0000","last_became_peered":"2024-09-21T07:49:35.141898+0000","last_unstale":"2024-09-21T07:49:36.143469+0000","last_undegraded":"2024-09-21T07:49:36.143469+0000","last_fullsized":"2024-09-21T07:49:36.143469+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:24:32.544254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.265401+0000","last_change":"2024-09-21T07:49:34.141564+0000","last_active":"2024-09-21T07:49:37.265401+0000","last_peered":"2024-09-21T07:49:37.265401+0000","last_clean":"2024-09-21T07:49:37.265401+0000","last_became_active":"2024-09-21T07:49:34.141237+0000","last_became_peered":"2024-09-21T07:49:34.141237+0000","last_unstale":"2024-09-21T07:49:37.265401+0000","last_undegraded":"2024-09-21T07:49:37.265401+0000","last_fullsized":"2024-09-21T07:49:37.265401+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:02.587670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":334,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.144178+0000","last_change":"2024-09-21T07:35:27.878020+0000","last_active":"2024-09-21T07:49:36.144178+0000","last_peered":"2024-09-21T07:49:36.144178+0000","last_clean":"2024-09-21T07:49:36.144178+0000","last_became_active":"2024-09-21T07:35:27.877532+0000","last_became_peered":"2024-09-21T07:35:27.877532+0000","last_unstale":"2024-09-21T07:49:36.144178+0000","last_undegraded":"2024-09-21T07:49:36.144178+0000","last_fullsized":"2024-09-21T07:49:36.144178+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:19.756073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":341,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.144488+0000","last_change":"2024-09-21T07:35:28.861049+0000","last_active":"2024-09-21T07:49:36.144488+0000","last_peered":"2024-09-21T07:49:36.144488+0000","last_clean":"2024-09-21T07:49:36.144488+0000","last_became_active":"2024-09-21T07:35:26.876819+0000","last_became_peered":"2024-09-21T07:35:26.876819+0000","last_unstale":"2024-09-21T07:49:36.144488+0000","last_undegraded":"2024-09-21T07:49:36.144488+0000","last_fullsized":"2024-09-21T07:49:36.144488+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:19.633803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055780599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":391,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.965002+0000","last_change":"2024-09-21T07:34:25.084849+0000","last_active":"2024-09-21T07:49:36.965002+0000","last_peered":"2024-09-21T07:49:36.965002+0000","last_clean":"2024-09-21T07:49:36.965002+0000","last_became_active":"2024-09-21T07:34:23.277713+0000","last_became_peered":"2024-09-21T07:34:23.277713+0000","last_unstale":"2024-09-21T07:49:36.965002+0000","last_undegraded":"2024-09-21T07:49:36.965002+0000","last_fullsized":"2024-09-21T07:49:36.965002+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:05.811810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059079499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143127+0000","last_change":"2024-09-21T07:49:35.137270+0000","last_active":"2024-09-21T07:49:36.143127+0000","last_peered":"2024-09-21T07:49:36.143127+0000","last_clean":"2024-09-21T07:49:36.143127+0000","last_became_active":"2024-09-21T07:49:35.136943+0000","last_became_peered":"2024-09-21T07:49:35.136943+0000","last_unstale":"2024-09-21T07:49:36.143127+0000","last_undegraded":"2024-09-21T07:49:36.143127+0000","last_fullsized":"2024-09-21T07:49:36.143127+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:00.103271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":13,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-21T07:49:35.132039+0000","last_change":"2024-09-21T07:49:34.137760+0000","last_active":"2024-09-21T07:49:35.132039+0000","last_peered":"2024-09-21T07:49:35.132039+0000","last_clean":"2024-09-21T07:49:35.132039+0000","last_became_active":"2024-09-21T07:49:34.137446+0000","last_became_peered":"2024-09-21T07:49:34.137446+0000","last_unstale":"2024-09-21T07:49:35.132039+0000","last_undegraded":"2024-09-21T07:49:35.132039+0000","last_fullsized":"2024-09-21T07:49:35.132039+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:03:04.362781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":336,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.121733+0000","last_change":"2024-09-21T07:35:27.884248+0000","last_active":"2024-09-21T07:49:37.121733+0000","last_peered":"2024-09-21T07:49:37.121733+0000","last_clean":"2024-09-21T07:49:37.121733+0000","last_became_active":"2024-09-21T07:35:27.883861+0000","last_became_peered":"2024-09-21T07:35:27.883861+0000","last_unstale":"2024-09-21T07:49:37.121733+0000","last_undegraded":"2024-09-21T07:49:37.121733+0000","last_fullsized":"2024-09-21T07:49:37.121733+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":342,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.266599+0000","last_change":"2024-09-21T07:35:29.207135+0000","last_active":"2024-09-21T07:49:37.266599+0000","last_peered":"2024-09-21T07:49:37.266599+0000","last_clean":"2024-09-21T07:49:37.266599+0000","last_became_active":"2024-09-21T07:35:26.872787+0000","last_became_peered":"2024-09-21T07:35:26.872787+0000","last_unstale":"2024-09-21T07:49:37.266599+0000","last_undegraded":"2024-09-21T07:49:37.266599+0000","last_fullsized":"2024-09-21T07:49:37.266599+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:27.543217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011157509999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":407,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.141961+0000","last_change":"2024-09-21T07:34:24.538610+0000","last_active":"2024-09-21T07:49:36.141961+0000","last_peered":"2024-09-21T07:49:36.141961+0000","last_clean":"2024-09-21T07:49:36.141961+0000","last_became_active":"2024-09-21T07:34:23.259700+0000","last_became_peered":"2024-09-21T07:34:23.259700+0000","last_unstale":"2024-09-21T07:49:36.141961+0000","last_undegraded":"2024-09-21T07:49:36.141961+0000","last_fullsized":"2024-09-21T07:49:36.141961+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:57.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070522399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143150+0000","last_change":"2024-09-21T07:49:35.149612+0000","last_active":"2024-09-21T07:49:36.143150+0000","last_peered":"2024-09-21T07:49:36.143150+0000","last_clean":"2024-09-21T07:49:36.143150+0000","last_became_active":"2024-09-21T07:49:35.148902+0000","last_became_peered":"2024-09-21T07:49:35.148902+0000","last_unstale":"2024-09-21T07:49:36.143150+0000","last_undegraded":"2024-09-21T07:49:36.143150+0000","last_fullsized":"2024-09-21T07:49:36.143150+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:56.054648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.122482+0000","last_change":"2024-09-21T07:49:34.131724+0000","last_active":"2024-09-21T07:49:37.122482+0000","last_peered":"2024-09-21T07:49:37.122482+0000","last_clean":"2024-09-21T07:49:37.122482+0000","last_became_active":"2024-09-21T07:49:34.129470+0000","last_became_peered":"2024-09-21T07:49:34.129470+0000","last_unstale":"2024-09-21T07:49:37.122482+0000","last_undegraded":"2024-09-21T07:49:37.122482+0000","last_fullsized":"2024-09-21T07:49:37.122482+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:04.866175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":348,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.142487+0000","last_change":"2024-09-21T07:35:27.868024+0000","last_active":"2024-09-21T07:49:36.142487+0000","last_peered":"2024-09-21T07:49:36.142487+0000","last_clean":"2024-09-21T07:49:36.142487+0000","last_became_active":"2024-09-21T07:35:27.867339+0000","last_became_peered":"2024-09-21T07:35:27.867339+0000","last_unstale":"2024-09-21T07:49:36.142487+0000","last_undegraded":"2024-09-21T07:49:36.142487+0000","last_fullsized":"2024-09-21T07:49:36.142487+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":355,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.142229+0000","last_change":"2024-09-21T07:35:28.858505+0000","last_active":"2024-09-21T07:49:36.142229+0000","last_peered":"2024-09-21T07:49:36.142229+0000","last_clean":"2024-09-21T07:49:36.142229+0000","last_became_active":"2024-09-21T07:35:26.877320+0000","last_became_peered":"2024-09-21T07:35:26.877320+0000","last_unstale":"2024-09-21T07:49:36.142229+0000","last_undegraded":"2024-09-21T07:49:36.142229+0000","last_fullsized":"2024-09-21T07:49:36.142229+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:53.672782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244119999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":384,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-21T07:49:35.138627+0000","last_change":"2024-09-21T07:34:24.597597+0000","last_active":"2024-09-21T07:49:35.138627+0000","last_peered":"2024-09-21T07:49:35.138627+0000","last_clean":"2024-09-21T07:49:35.138627+0000","last_became_active":"2024-09-21T07:34:23.635204+0000","last_became_peered":"2024-09-21T07:34:23.635204+0000","last_unstale":"2024-09-21T07:49:35.138627+0000","last_undegraded":"2024-09-21T07:49:35.138627+0000","last_fullsized":"2024-09-21T07:49:35.138627+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:48.644789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081139199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.145021+0000","last_change":"2024-09-21T07:49:35.133692+0000","last_active":"2024-09-21T07:49:36.145021+0000","last_peered":"2024-09-21T07:49:36.145021+0000","last_clean":"2024-09-21T07:49:36.145021+0000","last_became_active":"2024-09-21T07:49:35.133363+0000","last_became_peered":"2024-09-21T07:49:35.133363+0000","last_unstale":"2024-09-21T07:49:36.145021+0000","last_undegraded":"2024-09-21T07:49:36.145021+0000","last_fullsized":"2024-09-21T07:49:36.145021+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:25.982449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.266130+0000","last_change":"2024-09-21T07:49:34.136563+0000","last_active":"2024-09-21T07:49:37.266130+0000","last_peered":"2024-09-21T07:49:37.266130+0000","last_clean":"2024-09-21T07:49:37.266130+0000","last_became_active":"2024-09-21T07:49:34.134763+0000","last_became_peered":"2024-09-21T07:49:34.134763+0000","last_unstale":"2024-09-21T07:49:37.266130+0000","last_undegraded":"2024-09-21T07:49:37.266130+0000","last_fullsized":"2024-09-21T07:49:37.266130+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:53.730520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":334,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.266034+0000","last_change":"2024-09-21T07:35:27.877503+0000","last_active":"2024-09-21T07:49:37.266034+0000","last_peered":"2024-09-21T07:49:37.266034+0000","last_clean":"2024-09-21T07:49:37.266034+0000","last_became_active":"2024-09-21T07:35:27.875775+0000","last_became_peered":"2024-09-21T07:35:27.875775+0000","last_unstale":"2024-09-21T07:49:37.266034+0000","last_undegraded":"2024-09-21T07:49:37.266034+0000","last_fullsized":"2024-09-21T07:49:37.266034+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:49.291023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":338,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264210+0000","last_change":"2024-09-21T07:35:29.491412+0000","last_active":"2024-09-21T07:49:37.264210+0000","last_peered":"2024-09-21T07:49:37.264210+0000","last_clean":"2024-09-21T07:49:37.264210+0000","last_became_active":"2024-09-21T07:35:26.876404+0000","last_became_peered":"2024-09-21T07:35:26.876404+0000","last_unstale":"2024-09-21T07:49:37.264210+0000","last_undegraded":"2024-09-21T07:49:37.264210+0000","last_fullsized":"2024-09-21T07:49:37.264210+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063918000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":396,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.145769+0000","last_change":"2024-09-21T07:34:24.259026+0000","last_active":"2024-09-21T07:49:36.145769+0000","last_peered":"2024-09-21T07:49:36.145769+0000","last_clean":"2024-09-21T07:49:36.145769+0000","last_became_active":"2024-09-21T07:34:23.635456+0000","last_became_peered":"2024-09-21T07:34:23.635456+0000","last_unstale":"2024-09-21T07:49:36.145769+0000","last_undegraded":"2024-09-21T07:49:36.145769+0000","last_fullsized":"2024-09-21T07:49:36.145769+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:09.592752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018654419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.965787+0000","last_change":"2024-09-21T07:49:35.140067+0000","last_active":"2024-09-21T07:49:36.965787+0000","last_peered":"2024-09-21T07:49:36.965787+0000","last_clean":"2024-09-21T07:49:36.965787+0000","last_became_active":"2024-09-21T07:49:35.139746+0000","last_became_peered":"2024-09-21T07:49:35.139746+0000","last_unstale":"2024-09-21T07:49:36.965787+0000","last_undegraded":"2024-09-21T07:49:36.965787+0000","last_fullsized":"2024-09-21T07:49:36.965787+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:55:09.405295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.7","version":"232'2","reported_seq":17,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.267032+0000","last_change":"2024-09-21T07:49:34.140408+0000","last_active":"2024-09-21T07:49:37.267032+0000","last_peered":"2024-09-21T07:49:37.267032+0000","last_clean":"2024-09-21T07:49:37.267032+0000","last_became_active":"2024-09-21T07:49:34.139400+0000","last_became_peered":"2024-09-21T07:49:34.139400+0000","last_unstale":"2024-09-21T07:49:37.267032+0000","last_undegraded":"2024-09-21T07:49:37.267032+0000","last_fullsized":"2024-09-21T07:49:37.267032+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:11.257785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":334,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.266849+0000","last_change":"2024-09-21T07:35:27.871984+0000","last_active":"2024-09-21T07:49:37.266849+0000","last_peered":"2024-09-21T07:49:37.266849+0000","last_clean":"2024-09-21T07:49:37.266849+0000","last_became_active":"2024-09-21T07:35:27.870336+0000","last_became_peered":"2024-09-21T07:35:27.870336+0000","last_unstale":"2024-09-21T07:49:37.266849+0000","last_undegraded":"2024-09-21T07:49:37.266849+0000","last_fullsized":"2024-09-21T07:49:37.266849+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:57.120479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":342,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.142952+0000","last_change":"2024-09-21T07:35:28.860351+0000","last_active":"2024-09-21T07:49:36.142952+0000","last_peered":"2024-09-21T07:49:36.142952+0000","last_clean":"2024-09-21T07:49:36.142952+0000","last_became_active":"2024-09-21T07:35:26.872046+0000","last_became_peered":"2024-09-21T07:35:26.872046+0000","last_unstale":"2024-09-21T07:49:36.142952+0000","last_undegraded":"2024-09-21T07:49:36.142952+0000","last_fullsized":"2024-09-21T07:49:36.142952+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:47.074327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098384399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":391,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264885+0000","last_change":"2024-09-21T07:34:24.260300+0000","last_active":"2024-09-21T07:49:37.264885+0000","last_peered":"2024-09-21T07:49:37.264885+0000","last_clean":"2024-09-21T07:49:37.264885+0000","last_became_active":"2024-09-21T07:34:23.270128+0000","last_became_peered":"2024-09-21T07:34:23.270128+0000","last_unstale":"2024-09-21T07:49:37.264885+0000","last_undegraded":"2024-09-21T07:49:37.264885+0000","last_fullsized":"2024-09-21T07:49:37.264885+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:25.260931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00176651,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264970+0000","last_change":"2024-09-21T07:49:35.148351+0000","last_active":"2024-09-21T07:49:37.264970+0000","last_peered":"2024-09-21T07:49:37.264970+0000","last_clean":"2024-09-21T07:49:37.264970+0000","last_became_active":"2024-09-21T07:49:35.145379+0000","last_became_peered":"2024-09-21T07:49:35.145379+0000","last_unstale":"2024-09-21T07:49:37.264970+0000","last_undegraded":"2024-09-21T07:49:37.264970+0000","last_fullsized":"2024-09-21T07:49:37.264970+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:33.272994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.6","version":"233'1","reported_seq":16,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.124252+0000","last_change":"2024-09-21T07:49:34.134878+0000","last_active":"2024-09-21T07:49:37.124252+0000","last_peered":"2024-09-21T07:49:37.124252+0000","last_clean":"2024-09-21T07:49:37.124252+0000","last_became_active":"2024-09-21T07:49:34.134565+0000","last_became_peered":"2024-09-21T07:49:34.134565+0000","last_unstale":"2024-09-21T07:49:37.124252+0000","last_undegraded":"2024-09-21T07:49:37.124252+0000","last_fullsized":"2024-09-21T07:49:37.124252+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:01:32.536536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":334,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.267255+0000","last_change":"2024-09-21T07:35:27.877861+0000","last_active":"2024-09-21T07:49:37.267255+0000","last_peered":"2024-09-21T07:49:37.267255+0000","last_clean":"2024-09-21T07:49:37.267255+0000","last_became_active":"2024-09-21T07:35:27.877434+0000","last_became_peered":"2024-09-21T07:35:27.877434+0000","last_unstale":"2024-09-21T07:49:37.267255+0000","last_undegraded":"2024-09-21T07:49:37.267255+0000","last_fullsized":"2024-09-21T07:49:37.267255+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":340,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.965300+0000","last_change":"2024-09-21T07:35:28.862954+0000","last_active":"2024-09-21T07:49:36.965300+0000","last_peered":"2024-09-21T07:49:36.965300+0000","last_clean":"2024-09-21T07:49:36.965300+0000","last_became_active":"2024-09-21T07:35:26.870613+0000","last_became_peered":"2024-09-21T07:35:26.870613+0000","last_unstale":"2024-09-21T07:49:36.965300+0000","last_undegraded":"2024-09-21T07:49:36.965300+0000","last_fullsized":"2024-09-21T07:49:36.965300+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:06.901043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059842100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":394,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.122297+0000","last_change":"2024-09-21T07:34:25.028904+0000","last_active":"2024-09-21T07:49:37.122297+0000","last_peered":"2024-09-21T07:49:37.122297+0000","last_clean":"2024-09-21T07:49:37.122297+0000","last_became_active":"2024-09-21T07:34:23.260942+0000","last_became_peered":"2024-09-21T07:34:23.260942+0000","last_unstale":"2024-09-21T07:49:37.122297+0000","last_undegraded":"2024-09-21T07:49:37.122297+0000","last_fullsized":"2024-09-21T07:49:37.122297+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:22.709328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060553899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509642,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66572,"kb_used_data":3444,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94305268,"statfs":{"total":96636764160,"available":96568594432,"internally_reserved":0,"allocated":3526656,"data_stored":2156051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509644,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70168,"kb_used_data":2880,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301672,"statfs":{"total":96636764160,"available":96564912128,"internally_reserved":0,"allocated":2949120,"data_stored":1566749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509642,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34080,"kb_used_data":2884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337760,"statfs":{"total":96636764160,"available":96601866240,"internally_reserved":0,"allocated":2953216,"data_stored":1574300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509643,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29912,"kb_used_data":2876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341928,"statfs":{"total":96636764160,"available":96606134272,"internally_reserved":0,"allocated":2945024,"data_stored":1573234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509644,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47944,"kb_used_data":2868,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323896,"statfs":{"total":96636764160,"available":96587669504,"internally_reserved":0,"allocated":2936832,"data_stored":1572618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509644,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52680,"kb_used_data":3436,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319160,"statfs":{"total":96636764160,"available":96582819840,"internally_reserved":0,"allocated":3518464,"data_stored":2155379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542347,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52104,"kb_used_data":2860,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319736,"statfs":{"total":96636764160,"available":96583409664,"internally_reserved":0,"allocated":2928640,"data_stored":1565101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542346,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74308,"kb_used_data":2860,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297532,"statfs":{"total":96636764160,"available":96560672768,"internally_reserved":0,"allocated":2928640,"data_stored":1565033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:49:39.641 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:49:39.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.899+0000 7f8c53a00640 1 -- 172.21.15.39:0/2282216995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c30003a50 msgr2=0x7f8c30003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:39.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.899+0000 7f8c53a00640 1 --2- 172.21.15.39:0/2282216995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c30003a50 0x7f8c30003ec0 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0016c0 tx=0x7f8c30005830 comp rx=0 tx=0).stop 2024-09-21T07:49:39.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.899+0000 7f8c53a00640 1 -- 172.21.15.39:0/2282216995 shutdown_connections 2024-09-21T07:49:39.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.899+0000 7f8c53a00640 1 --2- 172.21.15.39:0/2282216995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c30003a50 0x7f8c30003ec0 unknown :-1 s=CLOSED pgs=2237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:39.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.899+0000 7f8c53a00640 1 -- 172.21.15.39:0/2282216995 >> 172.21.15.39:0/2282216995 conn(0x7f8c4c06cfc0 msgr2=0x7f8c4c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:39.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.899+0000 7f8c53a00640 1 -- 172.21.15.39:0/2282216995 shutdown_connections 2024-09-21T07:49:39.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.899+0000 7f8c53a00640 1 -- 172.21.15.39:0/2282216995 wait complete. 2024-09-21T07:49:39.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.902+0000 7f8c53a00640 1 Processor -- start 2024-09-21T07:49:39.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.902+0000 7f8c53a00640 1 -- start start 2024-09-21T07:49:39.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.903+0000 7f8c53a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8c30003a50 0x7f8c4c0b1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:39.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.903+0000 7f8c53a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c4c0b1590 0x7f8c4c15b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:39.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.903+0000 7f8c53a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c4c15b8a0 0x7f8c4c1f5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:39.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.903+0000 7f8c53a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4c155bf0 con 0x7f8c4c15b8a0 2024-09-21T07:49:39.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.903+0000 7f8c53a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4c155a70 con 0x7f8c4c0b1590 2024-09-21T07:49:39.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.903+0000 7f8c53a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f8c4c155d70 con 0x7f8c30003a50 2024-09-21T07:49:39.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.903+0000 7f8c51a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c4c15b8a0 0x7f8c4c1f5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:39.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.903+0000 7f8c51a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c4c15b8a0 0x7f8c4c1f5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41760/0 (socket says 172.21.15.39:41760) 2024-09-21T07:49:39.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.903+0000 7f8c51a00640 1 -- 172.21.15.39:0/2734992314 learned_addr learned my addr 172.21.15.39:0/2734992314 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:39.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.904+0000 7f8c43e00640 1 --2- 172.21.15.39:0/2734992314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c4c0b1590 0x7f8c4c15b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:39.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.904+0000 7f8c51000640 1 --2- 172.21.15.39:0/2734992314 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8c30003a50 0x7f8c4c0b1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:39.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.904+0000 7f8c51a00640 1 -- 172.21.15.39:0/2734992314 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8c30003a50 msgr2=0x7f8c4c0b1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:39.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.904+0000 7f8c51a00640 1 --2- 172.21.15.39:0/2734992314 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8c30003a50 0x7f8c4c0b1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:39.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.904+0000 7f8c51a00640 1 -- 172.21.15.39:0/2734992314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c4c0b1590 msgr2=0x7f8c4c15b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:39.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.904+0000 7f8c51a00640 1 --2- 172.21.15.39:0/2734992314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c4c0b1590 0x7f8c4c15b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:39.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.904+0000 7f8c51a00640 1 -- 172.21.15.39:0/2734992314 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c4c1f64b0 con 0x7f8c4c15b8a0 2024-09-21T07:49:39.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.904+0000 7f8c51a00640 1 --2- 172.21.15.39:0/2734992314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c4c15b8a0 0x7f8c4c1f5380 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f8c44001330 tx=0x7f8c4400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:39.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.904+0000 7f8c49600640 1 -- 172.21.15.39:0/2734992314 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4400ca10 con 0x7f8c4c15b8a0 2024-09-21T07:49:39.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.905+0000 7f8c49600640 1 -- 172.21.15.39:0/2734992314 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c44017070 con 0x7f8c4c15b8a0 2024-09-21T07:49:39.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.905+0000 7f8c49600640 1 -- 172.21.15.39:0/2734992314 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c44011460 con 0x7f8c4c15b8a0 2024-09-21T07:49:39.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.905+0000 7f8c53a00640 1 -- 172.21.15.39:0/2734992314 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c4c0b1d80 con 0x7f8c4c15b8a0 2024-09-21T07:49:39.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.905+0000 7f8c53a00640 1 -- 172.21.15.39:0/2734992314 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c4c15ab60 con 0x7f8c4c15b8a0 2024-09-21T07:49:39.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.906+0000 7f8c53a00640 1 -- 172.21.15.39:0/2734992314 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c4c0d0cf0 con 0x7f8c4c15b8a0 2024-09-21T07:49:39.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.907+0000 7f8c49600640 1 -- 172.21.15.39:0/2734992314 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8c44011600 con 0x7f8c4c15b8a0 2024-09-21T07:49:39.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.908+0000 7f8c49600640 1 --2- 172.21.15.39:0/2734992314 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8c28078bc0 0x7f8c2807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:39.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.908+0000 7f8c49600640 1 -- 172.21.15.39:0/2734992314 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(233..233 src has 1..233) ==== 7016+0+0 (secure 0 0 0) 0x7f8c4409b600 con 0x7f8c4c15b8a0 2024-09-21T07:49:39.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.908+0000 7f8c51000640 1 --2- 172.21.15.39:0/2734992314 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8c28078bc0 0x7f8c2807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:39.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.909+0000 7f8c51000640 1 --2- 172.21.15.39:0/2734992314 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8c28078bc0 0x7f8c2807b0d0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c032000 tx=0x7f8c3c02c040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:39.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:39.913+0000 7f8c49600640 1 -- 172.21.15.39:0/2734992314 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8c44062ec0 con 0x7f8c4c15b8a0 2024-09-21T07:49:40.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.106+0000 7f8c53a00640 1 -- 172.21.15.39:0/2734992314 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8c4c071450 con 0x7f8c28078bc0 2024-09-21T07:49:40.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.116+0000 7f8c49600640 1 -- 172.21.15.39:0/2734992314 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+495242 (secure 0 0 0) 0x7f8c4c071450 con 0x7f8c28078bc0 2024-09-21T07:49:40.123 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:49:40.127 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:49:40.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.121+0000 7f8c53a00640 1 -- 172.21.15.39:0/2734992314 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8c28078bc0 msgr2=0x7f8c2807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:40.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.121+0000 7f8c53a00640 1 --2- 172.21.15.39:0/2734992314 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8c28078bc0 0x7f8c2807b0d0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c032000 tx=0x7f8c3c02c040 comp rx=0 tx=0).stop 2024-09-21T07:49:40.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.121+0000 7f8c53a00640 1 -- 172.21.15.39:0/2734992314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c4c15b8a0 msgr2=0x7f8c4c1f5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:40.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.121+0000 7f8c53a00640 1 --2- 172.21.15.39:0/2734992314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c4c15b8a0 0x7f8c4c1f5380 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f8c44001330 tx=0x7f8c4400eb80 comp rx=0 tx=0).stop 2024-09-21T07:49:40.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.121+0000 7f8c53a00640 1 -- 172.21.15.39:0/2734992314 shutdown_connections 2024-09-21T07:49:40.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.121+0000 7f8c53a00640 1 --2- 172.21.15.39:0/2734992314 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8c28078bc0 0x7f8c2807b0d0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:40.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.121+0000 7f8c53a00640 1 --2- 172.21.15.39:0/2734992314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c4c15b8a0 0x7f8c4c1f5380 unknown :-1 s=CLOSED pgs=2238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:40.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.121+0000 7f8c53a00640 1 --2- 172.21.15.39:0/2734992314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c4c0b1590 0x7f8c4c15b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:40.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.121+0000 7f8c53a00640 1 --2- 172.21.15.39:0/2734992314 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8c30003a50 0x7f8c4c0b1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:40.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.121+0000 7f8c53a00640 1 -- 172.21.15.39:0/2734992314 >> 172.21.15.39:0/2734992314 conn(0x7f8c4c06cfc0 msgr2=0x7f8c4c0b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:40.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.121+0000 7f8c53a00640 1 -- 172.21.15.39:0/2734992314 shutdown_connections 2024-09-21T07:49:40.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.121+0000 7f8c53a00640 1 -- 172.21.15.39:0/2734992314 wait complete. 2024-09-21T07:49:40.145 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":727,"stamp":"2024-09-21T07:49:39.869005+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":314,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":427768,"kb_used_data":24108,"kb_used_omap":12,"kb_used_meta":403443,"kb_avail":754546952,"statfs":{"total":773094113280,"available":772656078848,"internally_reserved":0,"allocated":24686592,"data_stored":13728465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":413126192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001744"},"pg_stats":[{"pgid":"30.3f","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.266480+0000","last_change":"2024-09-21T07:49:35.154451+0000","last_active":"2024-09-21T07:49:37.266480+0000","last_peered":"2024-09-21T07:49:37.266480+0000","last_clean":"2024-09-21T07:49:37.266480+0000","last_became_active":"2024-09-21T07:49:35.151722+0000","last_became_peered":"2024-09-21T07:49:35.151722+0000","last_unstale":"2024-09-21T07:49:37.266480+0000","last_undegraded":"2024-09-21T07:49:37.266480+0000","last_fullsized":"2024-09-21T07:49:37.266480+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:42:52.483212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.965850+0000","last_change":"2024-09-21T07:49:35.152094+0000","last_active":"2024-09-21T07:49:36.965850+0000","last_peered":"2024-09-21T07:49:36.965850+0000","last_clean":"2024-09-21T07:49:36.965850+0000","last_became_active":"2024-09-21T07:49:35.151391+0000","last_became_peered":"2024-09-21T07:49:35.151391+0000","last_unstale":"2024-09-21T07:49:36.965850+0000","last_undegraded":"2024-09-21T07:49:36.965850+0000","last_fullsized":"2024-09-21T07:49:36.965850+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:57.983738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.144861+0000","last_change":"2024-09-21T07:49:35.149328+0000","last_active":"2024-09-21T07:49:36.144861+0000","last_peered":"2024-09-21T07:49:36.144861+0000","last_clean":"2024-09-21T07:49:36.144861+0000","last_became_active":"2024-09-21T07:49:35.148981+0000","last_became_peered":"2024-09-21T07:49:35.148981+0000","last_unstale":"2024-09-21T07:49:36.144861+0000","last_undegraded":"2024-09-21T07:49:36.144861+0000","last_fullsized":"2024-09-21T07:49:36.144861+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:11:04.993657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":10,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-21T07:49:35.155342+0000","last_change":"2024-09-21T07:49:35.151794+0000","last_active":"2024-09-21T07:49:35.155342+0000","last_peered":"2024-09-21T07:49:35.155342+0000","last_clean":"2024-09-21T07:49:35.155342+0000","last_became_active":"2024-09-21T07:49:35.151483+0000","last_became_peered":"2024-09-21T07:49:35.151483+0000","last_unstale":"2024-09-21T07:49:35.155342+0000","last_undegraded":"2024-09-21T07:49:35.155342+0000","last_fullsized":"2024-09-21T07:49:35.155342+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:36:19.455400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264918+0000","last_change":"2024-09-21T07:49:35.154570+0000","last_active":"2024-09-21T07:49:37.264918+0000","last_peered":"2024-09-21T07:49:37.264918+0000","last_clean":"2024-09-21T07:49:37.264918+0000","last_became_active":"2024-09-21T07:49:35.154248+0000","last_became_peered":"2024-09-21T07:49:35.154248+0000","last_unstale":"2024-09-21T07:49:37.264918+0000","last_undegraded":"2024-09-21T07:49:37.264918+0000","last_fullsized":"2024-09-21T07:49:37.264918+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:05:34.913847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.142917+0000","last_change":"2024-09-21T07:49:35.147635+0000","last_active":"2024-09-21T07:49:36.142917+0000","last_peered":"2024-09-21T07:49:36.142917+0000","last_clean":"2024-09-21T07:49:36.142917+0000","last_became_active":"2024-09-21T07:49:35.144743+0000","last_became_peered":"2024-09-21T07:49:35.144743+0000","last_unstale":"2024-09-21T07:49:36.142917+0000","last_undegraded":"2024-09-21T07:49:36.142917+0000","last_fullsized":"2024-09-21T07:49:36.142917+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:20.077714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143386+0000","last_change":"2024-09-21T07:49:35.129921+0000","last_active":"2024-09-21T07:49:36.143386+0000","last_peered":"2024-09-21T07:49:36.143386+0000","last_clean":"2024-09-21T07:49:36.143386+0000","last_became_active":"2024-09-21T07:49:35.129617+0000","last_became_peered":"2024-09-21T07:49:35.129617+0000","last_unstale":"2024-09-21T07:49:36.143386+0000","last_undegraded":"2024-09-21T07:49:36.143386+0000","last_fullsized":"2024-09-21T07:49:36.143386+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:55:59.193165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.141880+0000","last_change":"2024-09-21T07:49:35.138164+0000","last_active":"2024-09-21T07:49:36.141880+0000","last_peered":"2024-09-21T07:49:36.141880+0000","last_clean":"2024-09-21T07:49:36.141880+0000","last_became_active":"2024-09-21T07:49:35.137816+0000","last_became_peered":"2024-09-21T07:49:35.137816+0000","last_unstale":"2024-09-21T07:49:36.141880+0000","last_undegraded":"2024-09-21T07:49:36.141880+0000","last_fullsized":"2024-09-21T07:49:36.141880+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:10.230183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.145058+0000","last_change":"2024-09-21T07:49:35.152205+0000","last_active":"2024-09-21T07:49:36.145058+0000","last_peered":"2024-09-21T07:49:36.145058+0000","last_clean":"2024-09-21T07:49:36.145058+0000","last_became_active":"2024-09-21T07:49:35.151874+0000","last_became_peered":"2024-09-21T07:49:35.151874+0000","last_unstale":"2024-09-21T07:49:36.145058+0000","last_undegraded":"2024-09-21T07:49:36.145058+0000","last_fullsized":"2024-09-21T07:49:36.145058+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:02:34.917415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.144774+0000","last_change":"2024-09-21T07:49:35.144935+0000","last_active":"2024-09-21T07:49:36.144774+0000","last_peered":"2024-09-21T07:49:36.144774+0000","last_clean":"2024-09-21T07:49:36.144774+0000","last_became_active":"2024-09-21T07:49:35.144599+0000","last_became_peered":"2024-09-21T07:49:35.144599+0000","last_unstale":"2024-09-21T07:49:36.144774+0000","last_undegraded":"2024-09-21T07:49:36.144774+0000","last_fullsized":"2024-09-21T07:49:36.144774+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:17.706464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":10,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-21T07:49:35.156936+0000","last_change":"2024-09-21T07:49:35.153968+0000","last_active":"2024-09-21T07:49:35.156936+0000","last_peered":"2024-09-21T07:49:35.156936+0000","last_clean":"2024-09-21T07:49:35.156936+0000","last_became_active":"2024-09-21T07:49:35.153673+0000","last_became_peered":"2024-09-21T07:49:35.153673+0000","last_unstale":"2024-09-21T07:49:35.156936+0000","last_undegraded":"2024-09-21T07:49:35.156936+0000","last_fullsized":"2024-09-21T07:49:35.156936+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:12:38.863678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.265842+0000","last_change":"2024-09-21T07:49:35.149138+0000","last_active":"2024-09-21T07:49:37.265842+0000","last_peered":"2024-09-21T07:49:37.265842+0000","last_clean":"2024-09-21T07:49:37.265842+0000","last_became_active":"2024-09-21T07:49:35.148841+0000","last_became_peered":"2024-09-21T07:49:35.148841+0000","last_unstale":"2024-09-21T07:49:37.265842+0000","last_undegraded":"2024-09-21T07:49:37.265842+0000","last_fullsized":"2024-09-21T07:49:37.265842+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:37:25.254306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.142374+0000","last_change":"2024-09-21T07:49:35.144811+0000","last_active":"2024-09-21T07:49:36.142374+0000","last_peered":"2024-09-21T07:49:36.142374+0000","last_clean":"2024-09-21T07:49:36.142374+0000","last_became_active":"2024-09-21T07:49:35.144483+0000","last_became_peered":"2024-09-21T07:49:35.144483+0000","last_unstale":"2024-09-21T07:49:36.142374+0000","last_undegraded":"2024-09-21T07:49:36.142374+0000","last_fullsized":"2024-09-21T07:49:36.142374+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:11:34.450315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.266218+0000","last_change":"2024-09-21T07:49:35.137745+0000","last_active":"2024-09-21T07:49:37.266218+0000","last_peered":"2024-09-21T07:49:37.266218+0000","last_clean":"2024-09-21T07:49:37.266218+0000","last_became_active":"2024-09-21T07:49:35.134993+0000","last_became_peered":"2024-09-21T07:49:35.134993+0000","last_unstale":"2024-09-21T07:49:37.266218+0000","last_undegraded":"2024-09-21T07:49:37.266218+0000","last_fullsized":"2024-09-21T07:49:37.266218+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:02.781796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.121588+0000","last_change":"2024-09-21T07:49:35.144236+0000","last_active":"2024-09-21T07:49:37.121588+0000","last_peered":"2024-09-21T07:49:37.121588+0000","last_clean":"2024-09-21T07:49:37.121588+0000","last_became_active":"2024-09-21T07:49:35.143875+0000","last_became_peered":"2024-09-21T07:49:35.143875+0000","last_unstale":"2024-09-21T07:49:37.121588+0000","last_undegraded":"2024-09-21T07:49:37.121588+0000","last_fullsized":"2024-09-21T07:49:37.121588+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:33.174799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.141957+0000","last_change":"2024-09-21T07:49:35.139659+0000","last_active":"2024-09-21T07:49:36.141957+0000","last_peered":"2024-09-21T07:49:36.141957+0000","last_clean":"2024-09-21T07:49:36.141957+0000","last_became_active":"2024-09-21T07:49:35.139354+0000","last_became_peered":"2024-09-21T07:49:35.139354+0000","last_unstale":"2024-09-21T07:49:36.141957+0000","last_undegraded":"2024-09-21T07:49:36.141957+0000","last_fullsized":"2024-09-21T07:49:36.141957+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:45:34.545114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.266910+0000","last_change":"2024-09-21T07:49:35.146980+0000","last_active":"2024-09-21T07:49:37.266910+0000","last_peered":"2024-09-21T07:49:37.266910+0000","last_clean":"2024-09-21T07:49:37.266910+0000","last_became_active":"2024-09-21T07:49:35.142121+0000","last_became_peered":"2024-09-21T07:49:35.142121+0000","last_unstale":"2024-09-21T07:49:37.266910+0000","last_undegraded":"2024-09-21T07:49:37.266910+0000","last_fullsized":"2024-09-21T07:49:37.266910+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:41:11.664236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.144990+0000","last_change":"2024-09-21T07:49:35.146061+0000","last_active":"2024-09-21T07:49:36.144990+0000","last_peered":"2024-09-21T07:49:36.144990+0000","last_clean":"2024-09-21T07:49:36.144990+0000","last_became_active":"2024-09-21T07:49:35.141899+0000","last_became_peered":"2024-09-21T07:49:35.141899+0000","last_unstale":"2024-09-21T07:49:36.144990+0000","last_undegraded":"2024-09-21T07:49:36.144990+0000","last_fullsized":"2024-09-21T07:49:36.144990+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:41:40.905635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264337+0000","last_change":"2024-09-21T07:49:35.147825+0000","last_active":"2024-09-21T07:49:37.264337+0000","last_peered":"2024-09-21T07:49:37.264337+0000","last_clean":"2024-09-21T07:49:37.264337+0000","last_became_active":"2024-09-21T07:49:35.147451+0000","last_became_peered":"2024-09-21T07:49:35.147451+0000","last_unstale":"2024-09-21T07:49:37.264337+0000","last_undegraded":"2024-09-21T07:49:37.264337+0000","last_fullsized":"2024-09-21T07:49:37.264337+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:26.985337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.142623+0000","last_change":"2024-09-21T07:49:35.153370+0000","last_active":"2024-09-21T07:49:36.142623+0000","last_peered":"2024-09-21T07:49:36.142623+0000","last_clean":"2024-09-21T07:49:36.142623+0000","last_became_active":"2024-09-21T07:49:35.152865+0000","last_became_peered":"2024-09-21T07:49:35.152865+0000","last_unstale":"2024-09-21T07:49:36.142623+0000","last_undegraded":"2024-09-21T07:49:36.142623+0000","last_fullsized":"2024-09-21T07:49:36.142623+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:42:31.444496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.144022+0000","last_change":"2024-09-21T07:49:35.139492+0000","last_active":"2024-09-21T07:49:36.144022+0000","last_peered":"2024-09-21T07:49:36.144022+0000","last_clean":"2024-09-21T07:49:36.144022+0000","last_became_active":"2024-09-21T07:49:35.138366+0000","last_became_peered":"2024-09-21T07:49:35.138366+0000","last_unstale":"2024-09-21T07:49:36.144022+0000","last_undegraded":"2024-09-21T07:49:36.144022+0000","last_fullsized":"2024-09-21T07:49:36.144022+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:55.251141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.121925+0000","last_change":"2024-09-21T07:49:35.150421+0000","last_active":"2024-09-21T07:49:37.121925+0000","last_peered":"2024-09-21T07:49:37.121925+0000","last_clean":"2024-09-21T07:49:37.121925+0000","last_became_active":"2024-09-21T07:49:35.149998+0000","last_became_peered":"2024-09-21T07:49:35.149998+0000","last_unstale":"2024-09-21T07:49:37.121925+0000","last_undegraded":"2024-09-21T07:49:37.121925+0000","last_fullsized":"2024-09-21T07:49:37.121925+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:03:06.700393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.142875+0000","last_change":"2024-09-21T07:49:35.147014+0000","last_active":"2024-09-21T07:49:36.142875+0000","last_peered":"2024-09-21T07:49:36.142875+0000","last_clean":"2024-09-21T07:49:36.142875+0000","last_became_active":"2024-09-21T07:49:35.146672+0000","last_became_peered":"2024-09-21T07:49:35.146672+0000","last_unstale":"2024-09-21T07:49:36.142875+0000","last_undegraded":"2024-09-21T07:49:36.142875+0000","last_fullsized":"2024-09-21T07:49:36.142875+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:32.400747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.263900+0000","last_change":"2024-09-21T07:49:35.128590+0000","last_active":"2024-09-21T07:49:37.263900+0000","last_peered":"2024-09-21T07:49:37.263900+0000","last_clean":"2024-09-21T07:49:37.263900+0000","last_became_active":"2024-09-21T07:49:35.128230+0000","last_became_peered":"2024-09-21T07:49:35.128230+0000","last_unstale":"2024-09-21T07:49:37.263900+0000","last_undegraded":"2024-09-21T07:49:37.263900+0000","last_fullsized":"2024-09-21T07:49:37.263900+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:39.205266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.265758+0000","last_change":"2024-09-21T07:49:35.133443+0000","last_active":"2024-09-21T07:49:37.265758+0000","last_peered":"2024-09-21T07:49:37.265758+0000","last_clean":"2024-09-21T07:49:37.265758+0000","last_became_active":"2024-09-21T07:49:35.133150+0000","last_became_peered":"2024-09-21T07:49:35.133150+0000","last_unstale":"2024-09-21T07:49:37.265758+0000","last_undegraded":"2024-09-21T07:49:37.265758+0000","last_fullsized":"2024-09-21T07:49:37.265758+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:45.555083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.145232+0000","last_change":"2024-09-21T07:49:35.137175+0000","last_active":"2024-09-21T07:49:36.145232+0000","last_peered":"2024-09-21T07:49:36.145232+0000","last_clean":"2024-09-21T07:49:36.145232+0000","last_became_active":"2024-09-21T07:49:35.136514+0000","last_became_peered":"2024-09-21T07:49:35.136514+0000","last_unstale":"2024-09-21T07:49:36.145232+0000","last_undegraded":"2024-09-21T07:49:36.145232+0000","last_fullsized":"2024-09-21T07:49:36.145232+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:34:45.491476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":10,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-21T07:49:35.157988+0000","last_change":"2024-09-21T07:49:35.156104+0000","last_active":"2024-09-21T07:49:35.157988+0000","last_peered":"2024-09-21T07:49:35.157988+0000","last_clean":"2024-09-21T07:49:35.157988+0000","last_became_active":"2024-09-21T07:49:35.155751+0000","last_became_peered":"2024-09-21T07:49:35.155751+0000","last_unstale":"2024-09-21T07:49:35.157988+0000","last_undegraded":"2024-09-21T07:49:35.157988+0000","last_fullsized":"2024-09-21T07:49:35.157988+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:22:16.087564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.266132+0000","last_change":"2024-09-21T07:49:35.147755+0000","last_active":"2024-09-21T07:49:37.266132+0000","last_peered":"2024-09-21T07:49:37.266132+0000","last_clean":"2024-09-21T07:49:37.266132+0000","last_became_active":"2024-09-21T07:49:35.142034+0000","last_became_peered":"2024-09-21T07:49:35.142034+0000","last_unstale":"2024-09-21T07:49:37.266132+0000","last_undegraded":"2024-09-21T07:49:37.266132+0000","last_fullsized":"2024-09-21T07:49:37.266132+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:51:54.013029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.965559+0000","last_change":"2024-09-21T07:49:35.147889+0000","last_active":"2024-09-21T07:49:36.965559+0000","last_peered":"2024-09-21T07:49:36.965559+0000","last_clean":"2024-09-21T07:49:36.965559+0000","last_became_active":"2024-09-21T07:49:35.147609+0000","last_became_peered":"2024-09-21T07:49:35.147609+0000","last_unstale":"2024-09-21T07:49:36.965559+0000","last_undegraded":"2024-09-21T07:49:36.965559+0000","last_fullsized":"2024-09-21T07:49:36.965559+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:54:52.012917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.142201+0000","last_change":"2024-09-21T07:49:35.148893+0000","last_active":"2024-09-21T07:49:36.142201+0000","last_peered":"2024-09-21T07:49:36.142201+0000","last_clean":"2024-09-21T07:49:36.142201+0000","last_became_active":"2024-09-21T07:49:35.148580+0000","last_became_peered":"2024-09-21T07:49:35.148580+0000","last_unstale":"2024-09-21T07:49:36.142201+0000","last_undegraded":"2024-09-21T07:49:36.142201+0000","last_fullsized":"2024-09-21T07:49:36.142201+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:34:13.594958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.142894+0000","last_change":"2024-09-21T07:49:35.133699+0000","last_active":"2024-09-21T07:49:36.142894+0000","last_peered":"2024-09-21T07:49:36.142894+0000","last_clean":"2024-09-21T07:49:36.142894+0000","last_became_active":"2024-09-21T07:49:35.133323+0000","last_became_peered":"2024-09-21T07:49:35.133323+0000","last_unstale":"2024-09-21T07:49:36.142894+0000","last_undegraded":"2024-09-21T07:49:36.142894+0000","last_fullsized":"2024-09-21T07:49:36.142894+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:14:57.553709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.142157+0000","last_change":"2024-09-21T07:49:35.147226+0000","last_active":"2024-09-21T07:49:36.142157+0000","last_peered":"2024-09-21T07:49:36.142157+0000","last_clean":"2024-09-21T07:49:36.142157+0000","last_became_active":"2024-09-21T07:49:35.146764+0000","last_became_peered":"2024-09-21T07:49:35.146764+0000","last_unstale":"2024-09-21T07:49:36.142157+0000","last_undegraded":"2024-09-21T07:49:36.142157+0000","last_fullsized":"2024-09-21T07:49:36.142157+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:37:01.120540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1f","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.146053+0000","last_change":"2024-09-21T07:49:35.137241+0000","last_active":"2024-09-21T07:49:36.146053+0000","last_peered":"2024-09-21T07:49:36.146053+0000","last_clean":"2024-09-21T07:49:36.146053+0000","last_became_active":"2024-09-21T07:49:35.136928+0000","last_became_peered":"2024-09-21T07:49:35.136928+0000","last_unstale":"2024-09-21T07:49:36.146053+0000","last_undegraded":"2024-09-21T07:49:36.146053+0000","last_fullsized":"2024-09-21T07:49:36.146053+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:37.194496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1c","version":"232'2","reported_seq":17,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264774+0000","last_change":"2024-09-21T07:49:34.134258+0000","last_active":"2024-09-21T07:49:37.264774+0000","last_peered":"2024-09-21T07:49:37.264774+0000","last_clean":"2024-09-21T07:49:37.264774+0000","last_became_active":"2024-09-21T07:49:34.133961+0000","last_became_peered":"2024-09-21T07:49:34.133961+0000","last_unstale":"2024-09-21T07:49:37.264774+0000","last_undegraded":"2024-09-21T07:49:37.264774+0000","last_fullsized":"2024-09-21T07:49:37.264774+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:31.820492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":331,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264969+0000","last_change":"2024-09-21T07:35:27.879605+0000","last_active":"2024-09-21T07:49:37.264969+0000","last_peered":"2024-09-21T07:49:37.264969+0000","last_clean":"2024-09-21T07:49:37.264969+0000","last_became_active":"2024-09-21T07:35:27.879274+0000","last_became_peered":"2024-09-21T07:35:27.879274+0000","last_unstale":"2024-09-21T07:49:37.264969+0000","last_undegraded":"2024-09-21T07:49:37.264969+0000","last_fullsized":"2024-09-21T07:49:37.264969+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:24.241636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":341,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.144539+0000","last_change":"2024-09-21T07:35:29.491336+0000","last_active":"2024-09-21T07:49:36.144539+0000","last_peered":"2024-09-21T07:49:36.144539+0000","last_clean":"2024-09-21T07:49:36.144539+0000","last_became_active":"2024-09-21T07:35:26.866484+0000","last_became_peered":"2024-09-21T07:35:26.866484+0000","last_unstale":"2024-09-21T07:49:36.144539+0000","last_undegraded":"2024-09-21T07:49:36.144539+0000","last_fullsized":"2024-09-21T07:49:36.144539+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:10.449665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076010100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":517,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264014+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:49:37.264014+0000","last_peered":"2024-09-21T07:49:37.264014+0000","last_clean":"2024-09-21T07:49:37.264014+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:49:37.264014+0000","last_undegraded":"2024-09-21T07:49:37.264014+0000","last_fullsized":"2024-09-21T07:49:37.264014+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":408,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143247+0000","last_change":"2024-09-21T07:34:24.539248+0000","last_active":"2024-09-21T07:49:36.143247+0000","last_peered":"2024-09-21T07:49:36.143247+0000","last_clean":"2024-09-21T07:49:36.143247+0000","last_became_active":"2024-09-21T07:34:23.273465+0000","last_became_peered":"2024-09-21T07:34:23.273465+0000","last_unstale":"2024-09-21T07:49:36.143247+0000","last_undegraded":"2024-09-21T07:49:36.143247+0000","last_fullsized":"2024-09-21T07:49:36.143247+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:55.517452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060627700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.122454+0000","last_change":"2024-09-21T07:49:35.150255+0000","last_active":"2024-09-21T07:49:37.122454+0000","last_peered":"2024-09-21T07:49:37.122454+0000","last_clean":"2024-09-21T07:49:37.122454+0000","last_became_active":"2024-09-21T07:49:35.149897+0000","last_became_peered":"2024-09-21T07:49:35.149897+0000","last_unstale":"2024-09-21T07:49:37.122454+0000","last_undegraded":"2024-09-21T07:49:37.122454+0000","last_fullsized":"2024-09-21T07:49:37.122454+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:38:38.777657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":13,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-21T07:49:35.127794+0000","last_change":"2024-09-21T07:49:34.144688+0000","last_active":"2024-09-21T07:49:35.127794+0000","last_peered":"2024-09-21T07:49:35.127794+0000","last_clean":"2024-09-21T07:49:35.127794+0000","last_became_active":"2024-09-21T07:49:34.144328+0000","last_became_peered":"2024-09-21T07:49:34.144328+0000","last_unstale":"2024-09-21T07:49:35.127794+0000","last_undegraded":"2024-09-21T07:49:35.127794+0000","last_fullsized":"2024-09-21T07:49:35.127794+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:57.881990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":334,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143539+0000","last_change":"2024-09-21T07:35:27.864492+0000","last_active":"2024-09-21T07:49:36.143539+0000","last_peered":"2024-09-21T07:49:36.143539+0000","last_clean":"2024-09-21T07:49:36.143539+0000","last_became_active":"2024-09-21T07:35:27.864176+0000","last_became_peered":"2024-09-21T07:35:27.864176+0000","last_unstale":"2024-09-21T07:49:36.143539+0000","last_undegraded":"2024-09-21T07:49:36.143539+0000","last_fullsized":"2024-09-21T07:49:36.143539+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:05.153467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":338,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.266427+0000","last_change":"2024-09-21T07:35:29.493678+0000","last_active":"2024-09-21T07:49:37.266427+0000","last_peered":"2024-09-21T07:49:37.266427+0000","last_clean":"2024-09-21T07:49:37.266427+0000","last_became_active":"2024-09-21T07:35:26.870749+0000","last_became_peered":"2024-09-21T07:35:26.870749+0000","last_unstale":"2024-09-21T07:49:37.266427+0000","last_undegraded":"2024-09-21T07:49:37.266427+0000","last_fullsized":"2024-09-21T07:49:37.266427+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:55.367877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010159570000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":384,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-21T07:49:35.132428+0000","last_change":"2024-09-21T07:34:24.597274+0000","last_active":"2024-09-21T07:49:35.132428+0000","last_peered":"2024-09-21T07:49:35.132428+0000","last_clean":"2024-09-21T07:49:35.132428+0000","last_became_active":"2024-09-21T07:34:23.279560+0000","last_became_peered":"2024-09-21T07:34:23.279560+0000","last_unstale":"2024-09-21T07:49:35.132428+0000","last_undegraded":"2024-09-21T07:49:35.132428+0000","last_fullsized":"2024-09-21T07:49:35.132428+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:15.845265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059699799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143252+0000","last_change":"2024-09-21T07:49:35.145803+0000","last_active":"2024-09-21T07:49:36.143252+0000","last_peered":"2024-09-21T07:49:36.143252+0000","last_clean":"2024-09-21T07:49:36.143252+0000","last_became_active":"2024-09-21T07:49:35.145504+0000","last_became_peered":"2024-09-21T07:49:35.145504+0000","last_unstale":"2024-09-21T07:49:36.143252+0000","last_undegraded":"2024-09-21T07:49:36.143252+0000","last_fullsized":"2024-09-21T07:49:36.143252+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:27:42.102361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143087+0000","last_change":"2024-09-21T07:49:34.131135+0000","last_active":"2024-09-21T07:49:36.143087+0000","last_peered":"2024-09-21T07:49:36.143087+0000","last_clean":"2024-09-21T07:49:36.143087+0000","last_became_active":"2024-09-21T07:49:34.130782+0000","last_became_peered":"2024-09-21T07:49:34.130782+0000","last_unstale":"2024-09-21T07:49:36.143087+0000","last_undegraded":"2024-09-21T07:49:36.143087+0000","last_fullsized":"2024-09-21T07:49:36.143087+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:01.932485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":335,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.142947+0000","last_change":"2024-09-21T07:35:27.874387+0000","last_active":"2024-09-21T07:49:36.142947+0000","last_peered":"2024-09-21T07:49:36.142947+0000","last_clean":"2024-09-21T07:49:36.142947+0000","last_became_active":"2024-09-21T07:35:27.873277+0000","last_became_peered":"2024-09-21T07:35:27.873277+0000","last_unstale":"2024-09-21T07:49:36.142947+0000","last_undegraded":"2024-09-21T07:49:36.142947+0000","last_fullsized":"2024-09-21T07:49:36.142947+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:04.581463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":334,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-21T07:49:35.138161+0000","last_change":"2024-09-21T07:35:29.387094+0000","last_active":"2024-09-21T07:49:35.138161+0000","last_peered":"2024-09-21T07:49:35.138161+0000","last_clean":"2024-09-21T07:49:35.138161+0000","last_became_active":"2024-09-21T07:35:26.874757+0000","last_became_peered":"2024-09-21T07:35:26.874757+0000","last_unstale":"2024-09-21T07:49:35.138161+0000","last_undegraded":"2024-09-21T07:49:35.138161+0000","last_fullsized":"2024-09-21T07:49:35.138161+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:33.542750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061065700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":391,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.265481+0000","last_change":"2024-09-21T07:34:24.259906+0000","last_active":"2024-09-21T07:49:37.265481+0000","last_peered":"2024-09-21T07:49:37.265481+0000","last_clean":"2024-09-21T07:49:37.265481+0000","last_became_active":"2024-09-21T07:34:23.635282+0000","last_became_peered":"2024-09-21T07:34:23.635282+0000","last_unstale":"2024-09-21T07:49:37.265481+0000","last_undegraded":"2024-09-21T07:49:37.265481+0000","last_fullsized":"2024-09-21T07:49:37.265481+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:00.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001527664,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143968+0000","last_change":"2024-09-21T07:49:35.144096+0000","last_active":"2024-09-21T07:49:36.143968+0000","last_peered":"2024-09-21T07:49:36.143968+0000","last_clean":"2024-09-21T07:49:36.143968+0000","last_became_active":"2024-09-21T07:49:35.143812+0000","last_became_peered":"2024-09-21T07:49:35.143812+0000","last_unstale":"2024-09-21T07:49:36.143968+0000","last_undegraded":"2024-09-21T07:49:36.143968+0000","last_fullsized":"2024-09-21T07:49:36.143968+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:46.238631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1b","version":"232'1","reported_seq":16,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.121724+0000","last_change":"2024-09-21T07:49:34.127268+0000","last_active":"2024-09-21T07:49:37.121724+0000","last_peered":"2024-09-21T07:49:37.121724+0000","last_clean":"2024-09-21T07:49:37.121724+0000","last_became_active":"2024-09-21T07:49:34.126215+0000","last_became_peered":"2024-09-21T07:49:34.126215+0000","last_unstale":"2024-09-21T07:49:37.121724+0000","last_undegraded":"2024-09-21T07:49:37.121724+0000","last_fullsized":"2024-09-21T07:49:37.121724+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:52.720478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":335,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143512+0000","last_change":"2024-09-21T07:35:27.873645+0000","last_active":"2024-09-21T07:49:36.143512+0000","last_peered":"2024-09-21T07:49:36.143512+0000","last_clean":"2024-09-21T07:49:36.143512+0000","last_became_active":"2024-09-21T07:35:27.873245+0000","last_became_peered":"2024-09-21T07:35:27.873245+0000","last_unstale":"2024-09-21T07:49:36.143512+0000","last_undegraded":"2024-09-21T07:49:36.143512+0000","last_fullsized":"2024-09-21T07:49:36.143512+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:55.090953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":342,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.144579+0000","last_change":"2024-09-21T07:35:28.861744+0000","last_active":"2024-09-21T07:49:36.144579+0000","last_peered":"2024-09-21T07:49:36.144579+0000","last_clean":"2024-09-21T07:49:36.144579+0000","last_became_active":"2024-09-21T07:35:26.876462+0000","last_became_peered":"2024-09-21T07:35:26.876462+0000","last_unstale":"2024-09-21T07:49:36.144579+0000","last_undegraded":"2024-09-21T07:49:36.144579+0000","last_fullsized":"2024-09-21T07:49:36.144579+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:48.632568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012020920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":391,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.965152+0000","last_change":"2024-09-21T07:34:25.085454+0000","last_active":"2024-09-21T07:49:36.965152+0000","last_peered":"2024-09-21T07:49:36.965152+0000","last_clean":"2024-09-21T07:49:36.965152+0000","last_became_active":"2024-09-21T07:34:23.279205+0000","last_became_peered":"2024-09-21T07:34:23.279205+0000","last_unstale":"2024-09-21T07:49:36.965152+0000","last_undegraded":"2024-09-21T07:49:36.965152+0000","last_fullsized":"2024-09-21T07:49:36.965152+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:54.231666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095320599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.266315+0000","last_change":"2024-09-21T07:49:35.146014+0000","last_active":"2024-09-21T07:49:37.266315+0000","last_peered":"2024-09-21T07:49:37.266315+0000","last_clean":"2024-09-21T07:49:37.266315+0000","last_became_active":"2024-09-21T07:49:35.141715+0000","last_became_peered":"2024-09-21T07:49:35.141715+0000","last_unstale":"2024-09-21T07:49:37.266315+0000","last_undegraded":"2024-09-21T07:49:37.266315+0000","last_fullsized":"2024-09-21T07:49:37.266315+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:37.560681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1a","version":"232'1","reported_seq":16,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264487+0000","last_change":"2024-09-21T07:49:34.139919+0000","last_active":"2024-09-21T07:49:37.264487+0000","last_peered":"2024-09-21T07:49:37.264487+0000","last_clean":"2024-09-21T07:49:37.264487+0000","last_became_active":"2024-09-21T07:49:34.139183+0000","last_became_peered":"2024-09-21T07:49:34.139183+0000","last_unstale":"2024-09-21T07:49:37.264487+0000","last_undegraded":"2024-09-21T07:49:37.264487+0000","last_fullsized":"2024-09-21T07:49:37.264487+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:57.776256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":334,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.144088+0000","last_change":"2024-09-21T07:35:27.861373+0000","last_active":"2024-09-21T07:49:36.144088+0000","last_peered":"2024-09-21T07:49:36.144088+0000","last_clean":"2024-09-21T07:49:36.144088+0000","last_became_active":"2024-09-21T07:35:27.861016+0000","last_became_peered":"2024-09-21T07:35:27.861016+0000","last_unstale":"2024-09-21T07:49:36.144088+0000","last_undegraded":"2024-09-21T07:49:36.144088+0000","last_fullsized":"2024-09-21T07:49:36.144088+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:56.043763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":343,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.121761+0000","last_change":"2024-09-21T07:35:29.053677+0000","last_active":"2024-09-21T07:49:37.121761+0000","last_peered":"2024-09-21T07:49:37.121761+0000","last_clean":"2024-09-21T07:49:37.121761+0000","last_became_active":"2024-09-21T07:35:26.875496+0000","last_became_peered":"2024-09-21T07:35:26.875496+0000","last_unstale":"2024-09-21T07:49:37.121761+0000","last_undegraded":"2024-09-21T07:49:37.121761+0000","last_fullsized":"2024-09-21T07:49:37.121761+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:04.850784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065859000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":395,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.121605+0000","last_change":"2024-09-21T07:34:25.028521+0000","last_active":"2024-09-21T07:49:37.121605+0000","last_peered":"2024-09-21T07:49:37.121605+0000","last_clean":"2024-09-21T07:49:37.121605+0000","last_became_active":"2024-09-21T07:34:23.635467+0000","last_became_peered":"2024-09-21T07:34:23.635467+0000","last_unstale":"2024-09-21T07:49:37.121605+0000","last_undegraded":"2024-09-21T07:49:37.121605+0000","last_fullsized":"2024-09-21T07:49:37.121605+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:33.729123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000660558,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.267007+0000","last_change":"2024-09-21T07:49:35.135388+0000","last_active":"2024-09-21T07:49:37.267007+0000","last_peered":"2024-09-21T07:49:37.267007+0000","last_clean":"2024-09-21T07:49:37.267007+0000","last_became_active":"2024-09-21T07:49:35.134074+0000","last_became_peered":"2024-09-21T07:49:35.134074+0000","last_unstale":"2024-09-21T07:49:37.267007+0000","last_undegraded":"2024-09-21T07:49:37.267007+0000","last_fullsized":"2024-09-21T07:49:37.267007+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:54.944026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1d","version":"232'2","reported_seq":17,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143045+0000","last_change":"2024-09-21T07:49:34.136420+0000","last_active":"2024-09-21T07:49:36.143045+0000","last_peered":"2024-09-21T07:49:36.143045+0000","last_clean":"2024-09-21T07:49:36.143045+0000","last_became_active":"2024-09-21T07:49:34.136041+0000","last_became_peered":"2024-09-21T07:49:34.136041+0000","last_unstale":"2024-09-21T07:49:36.143045+0000","last_undegraded":"2024-09-21T07:49:36.143045+0000","last_fullsized":"2024-09-21T07:49:36.143045+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:03:21.509136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":333,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.965862+0000","last_change":"2024-09-21T07:35:27.871201+0000","last_active":"2024-09-21T07:49:36.965862+0000","last_peered":"2024-09-21T07:49:36.965862+0000","last_clean":"2024-09-21T07:49:36.965862+0000","last_became_active":"2024-09-21T07:35:27.870816+0000","last_became_peered":"2024-09-21T07:35:27.870816+0000","last_unstale":"2024-09-21T07:49:36.965862+0000","last_undegraded":"2024-09-21T07:49:36.965862+0000","last_fullsized":"2024-09-21T07:49:36.965862+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:04.787450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":341,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.144343+0000","last_change":"2024-09-21T07:35:29.491874+0000","last_active":"2024-09-21T07:49:36.144343+0000","last_peered":"2024-09-21T07:49:36.144343+0000","last_clean":"2024-09-21T07:49:36.144343+0000","last_became_active":"2024-09-21T07:35:26.866381+0000","last_became_peered":"2024-09-21T07:35:26.866381+0000","last_unstale":"2024-09-21T07:49:36.144343+0000","last_undegraded":"2024-09-21T07:49:36.144343+0000","last_fullsized":"2024-09-21T07:49:36.144343+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:36.404647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073235399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":396,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.146133+0000","last_change":"2024-09-21T07:34:24.260046+0000","last_active":"2024-09-21T07:49:36.146133+0000","last_peered":"2024-09-21T07:49:36.146133+0000","last_clean":"2024-09-21T07:49:36.146133+0000","last_became_active":"2024-09-21T07:34:23.273431+0000","last_became_peered":"2024-09-21T07:34:23.273431+0000","last_unstale":"2024-09-21T07:49:36.146133+0000","last_undegraded":"2024-09-21T07:49:36.146133+0000","last_fullsized":"2024-09-21T07:49:36.146133+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:18.876057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019803310000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.144496+0000","last_change":"2024-09-21T07:49:35.146516+0000","last_active":"2024-09-21T07:49:36.144496+0000","last_peered":"2024-09-21T07:49:36.144496+0000","last_clean":"2024-09-21T07:49:36.144496+0000","last_became_active":"2024-09-21T07:49:35.143810+0000","last_became_peered":"2024-09-21T07:49:35.143810+0000","last_unstale":"2024-09-21T07:49:36.144496+0000","last_undegraded":"2024-09-21T07:49:36.144496+0000","last_fullsized":"2024-09-21T07:49:36.144496+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:44.416224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"232'4","reported_seq":19,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.145222+0000","last_change":"2024-09-21T07:49:34.141934+0000","last_active":"2024-09-21T07:49:36.145222+0000","last_peered":"2024-09-21T07:49:36.145222+0000","last_clean":"2024-09-21T07:49:36.145222+0000","last_became_active":"2024-09-21T07:49:34.141588+0000","last_became_peered":"2024-09-21T07:49:34.141588+0000","last_unstale":"2024-09-21T07:49:36.145222+0000","last_undegraded":"2024-09-21T07:49:36.145222+0000","last_fullsized":"2024-09-21T07:49:36.145222+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:21.118757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":333,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.965897+0000","last_change":"2024-09-21T07:35:27.871736+0000","last_active":"2024-09-21T07:49:36.965897+0000","last_peered":"2024-09-21T07:49:36.965897+0000","last_clean":"2024-09-21T07:49:36.965897+0000","last_became_active":"2024-09-21T07:35:27.871232+0000","last_became_peered":"2024-09-21T07:35:27.871232+0000","last_unstale":"2024-09-21T07:49:36.965897+0000","last_undegraded":"2024-09-21T07:49:36.965897+0000","last_fullsized":"2024-09-21T07:49:36.965897+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:04.419301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":341,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143647+0000","last_change":"2024-09-21T07:35:29.491246+0000","last_active":"2024-09-21T07:49:36.143647+0000","last_peered":"2024-09-21T07:49:36.143647+0000","last_clean":"2024-09-21T07:49:36.143647+0000","last_became_active":"2024-09-21T07:35:26.866783+0000","last_became_peered":"2024-09-21T07:35:26.866783+0000","last_unstale":"2024-09-21T07:49:36.143647+0000","last_undegraded":"2024-09-21T07:49:36.143647+0000","last_fullsized":"2024-09-21T07:49:36.143647+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:28.881914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098474099999999992,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":395,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.122019+0000","last_change":"2024-09-21T07:34:25.028640+0000","last_active":"2024-09-21T07:49:37.122019+0000","last_peered":"2024-09-21T07:49:37.122019+0000","last_clean":"2024-09-21T07:49:37.122019+0000","last_became_active":"2024-09-21T07:34:23.260950+0000","last_became_peered":"2024-09-21T07:34:23.260950+0000","last_unstale":"2024-09-21T07:49:37.122019+0000","last_undegraded":"2024-09-21T07:49:37.122019+0000","last_fullsized":"2024-09-21T07:49:37.122019+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:23:42.121398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055135099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.146123+0000","last_change":"2024-09-21T07:49:35.154197+0000","last_active":"2024-09-21T07:49:36.146123+0000","last_peered":"2024-09-21T07:49:36.146123+0000","last_clean":"2024-09-21T07:49:36.146123+0000","last_became_active":"2024-09-21T07:49:35.153887+0000","last_became_peered":"2024-09-21T07:49:35.153887+0000","last_unstale":"2024-09-21T07:49:36.146123+0000","last_undegraded":"2024-09-21T07:49:36.146123+0000","last_fullsized":"2024-09-21T07:49:36.146123+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:29:46.083612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"232'1","reported_seq":16,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.266738+0000","last_change":"2024-09-21T07:49:34.142279+0000","last_active":"2024-09-21T07:49:37.266738+0000","last_peered":"2024-09-21T07:49:37.266738+0000","last_clean":"2024-09-21T07:49:37.266738+0000","last_became_active":"2024-09-21T07:49:34.141943+0000","last_became_peered":"2024-09-21T07:49:34.141943+0000","last_unstale":"2024-09-21T07:49:37.266738+0000","last_undegraded":"2024-09-21T07:49:37.266738+0000","last_fullsized":"2024-09-21T07:49:37.266738+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:58.929915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":335,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.267311+0000","last_change":"2024-09-21T07:35:27.872027+0000","last_active":"2024-09-21T07:49:37.267311+0000","last_peered":"2024-09-21T07:49:37.267311+0000","last_clean":"2024-09-21T07:49:37.267311+0000","last_became_active":"2024-09-21T07:35:27.870426+0000","last_became_peered":"2024-09-21T07:35:27.870426+0000","last_unstale":"2024-09-21T07:49:37.267311+0000","last_undegraded":"2024-09-21T07:49:37.267311+0000","last_fullsized":"2024-09-21T07:49:37.267311+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":341,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143638+0000","last_change":"2024-09-21T07:35:29.491008+0000","last_active":"2024-09-21T07:49:36.143638+0000","last_peered":"2024-09-21T07:49:36.143638+0000","last_clean":"2024-09-21T07:49:36.143638+0000","last_became_active":"2024-09-21T07:35:26.866750+0000","last_became_peered":"2024-09-21T07:35:26.866750+0000","last_unstale":"2024-09-21T07:49:36.143638+0000","last_undegraded":"2024-09-21T07:49:36.143638+0000","last_fullsized":"2024-09-21T07:49:36.143638+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:25.490406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083529700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":392,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.265359+0000","last_change":"2024-09-21T07:34:24.260665+0000","last_active":"2024-09-21T07:49:37.265359+0000","last_peered":"2024-09-21T07:49:37.265359+0000","last_clean":"2024-09-21T07:49:37.265359+0000","last_became_active":"2024-09-21T07:34:23.635909+0000","last_became_peered":"2024-09-21T07:34:23.635909+0000","last_unstale":"2024-09-21T07:49:37.265359+0000","last_undegraded":"2024-09-21T07:49:37.265359+0000","last_fullsized":"2024-09-21T07:49:37.265359+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:44.536885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.144098+0000","last_change":"2024-09-21T07:49:35.135791+0000","last_active":"2024-09-21T07:49:36.144098+0000","last_peered":"2024-09-21T07:49:36.144098+0000","last_clean":"2024-09-21T07:49:36.144098+0000","last_became_active":"2024-09-21T07:49:35.135406+0000","last_became_peered":"2024-09-21T07:49:35.135406+0000","last_unstale":"2024-09-21T07:49:36.144098+0000","last_undegraded":"2024-09-21T07:49:36.144098+0000","last_fullsized":"2024-09-21T07:49:36.144098+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:43.306960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.141754+0000","last_change":"2024-09-21T07:49:34.138954+0000","last_active":"2024-09-21T07:49:36.141754+0000","last_peered":"2024-09-21T07:49:36.141754+0000","last_clean":"2024-09-21T07:49:36.141754+0000","last_became_active":"2024-09-21T07:49:34.138326+0000","last_became_peered":"2024-09-21T07:49:34.138326+0000","last_unstale":"2024-09-21T07:49:36.141754+0000","last_undegraded":"2024-09-21T07:49:36.141754+0000","last_fullsized":"2024-09-21T07:49:36.141754+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:51.125470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":328,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-21T07:49:35.127469+0000","last_change":"2024-09-21T07:35:27.874772+0000","last_active":"2024-09-21T07:49:35.127469+0000","last_peered":"2024-09-21T07:49:35.127469+0000","last_clean":"2024-09-21T07:49:35.127469+0000","last_became_active":"2024-09-21T07:35:27.874434+0000","last_became_peered":"2024-09-21T07:35:27.874434+0000","last_unstale":"2024-09-21T07:49:35.127469+0000","last_undegraded":"2024-09-21T07:49:35.127469+0000","last_fullsized":"2024-09-21T07:49:35.127469+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:31.998755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":355,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143099+0000","last_change":"2024-09-21T07:35:28.859153+0000","last_active":"2024-09-21T07:49:36.143099+0000","last_peered":"2024-09-21T07:49:36.143099+0000","last_clean":"2024-09-21T07:49:36.143099+0000","last_became_active":"2024-09-21T07:35:26.874428+0000","last_became_peered":"2024-09-21T07:35:26.874428+0000","last_unstale":"2024-09-21T07:49:36.143099+0000","last_undegraded":"2024-09-21T07:49:36.143099+0000","last_fullsized":"2024-09-21T07:49:36.143099+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:04.487621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048557299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":408,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.142842+0000","last_change":"2024-09-21T07:34:24.538845+0000","last_active":"2024-09-21T07:49:36.142842+0000","last_peered":"2024-09-21T07:49:36.142842+0000","last_clean":"2024-09-21T07:49:36.142842+0000","last_became_active":"2024-09-21T07:34:23.636372+0000","last_became_peered":"2024-09-21T07:34:23.636372+0000","last_unstale":"2024-09-21T07:49:36.142842+0000","last_undegraded":"2024-09-21T07:49:36.142842+0000","last_fullsized":"2024-09-21T07:49:36.142842+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:29.393813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841451,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.122083+0000","last_change":"2024-09-21T07:49:35.152975+0000","last_active":"2024-09-21T07:49:37.122083+0000","last_peered":"2024-09-21T07:49:37.122083+0000","last_clean":"2024-09-21T07:49:37.122083+0000","last_became_active":"2024-09-21T07:49:35.152637+0000","last_became_peered":"2024-09-21T07:49:35.152637+0000","last_unstale":"2024-09-21T07:49:37.122083+0000","last_undegraded":"2024-09-21T07:49:37.122083+0000","last_fullsized":"2024-09-21T07:49:37.122083+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:51:17.106760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"232'8","reported_seq":23,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264358+0000","last_change":"2024-09-21T07:49:34.142446+0000","last_active":"2024-09-21T07:49:37.264358+0000","last_peered":"2024-09-21T07:49:37.264358+0000","last_clean":"2024-09-21T07:49:37.264358+0000","last_became_active":"2024-09-21T07:49:34.142139+0000","last_became_peered":"2024-09-21T07:49:34.142139+0000","last_unstale":"2024-09-21T07:49:37.264358+0000","last_undegraded":"2024-09-21T07:49:37.264358+0000","last_fullsized":"2024-09-21T07:49:37.264358+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:03.943177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":335,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264453+0000","last_change":"2024-09-21T07:35:27.877801+0000","last_active":"2024-09-21T07:49:37.264453+0000","last_peered":"2024-09-21T07:49:37.264453+0000","last_clean":"2024-09-21T07:49:37.264453+0000","last_became_active":"2024-09-21T07:35:27.877464+0000","last_became_peered":"2024-09-21T07:35:27.877464+0000","last_unstale":"2024-09-21T07:49:37.264453+0000","last_undegraded":"2024-09-21T07:49:37.264453+0000","last_fullsized":"2024-09-21T07:49:37.264453+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:08.814929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":340,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.965925+0000","last_change":"2024-09-21T07:35:28.863732+0000","last_active":"2024-09-21T07:49:36.965925+0000","last_peered":"2024-09-21T07:49:36.965925+0000","last_clean":"2024-09-21T07:49:36.965925+0000","last_became_active":"2024-09-21T07:35:26.877392+0000","last_became_peered":"2024-09-21T07:35:26.877392+0000","last_unstale":"2024-09-21T07:49:36.965925+0000","last_undegraded":"2024-09-21T07:49:36.965925+0000","last_fullsized":"2024-09-21T07:49:36.965925+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071425700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":395,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.144446+0000","last_change":"2024-09-21T07:34:24.259686+0000","last_active":"2024-09-21T07:49:36.144446+0000","last_peered":"2024-09-21T07:49:36.144446+0000","last_clean":"2024-09-21T07:49:36.144446+0000","last_became_active":"2024-09-21T07:34:23.274405+0000","last_became_peered":"2024-09-21T07:34:23.274405+0000","last_unstale":"2024-09-21T07:49:36.144446+0000","last_undegraded":"2024-09-21T07:49:36.144446+0000","last_fullsized":"2024-09-21T07:49:36.144446+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:26.304693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022009920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.965518+0000","last_change":"2024-09-21T07:49:35.140522+0000","last_active":"2024-09-21T07:49:36.965518+0000","last_peered":"2024-09-21T07:49:36.965518+0000","last_clean":"2024-09-21T07:49:36.965518+0000","last_became_active":"2024-09-21T07:49:35.139108+0000","last_became_peered":"2024-09-21T07:49:35.139108+0000","last_unstale":"2024-09-21T07:49:36.965518+0000","last_undegraded":"2024-09-21T07:49:36.965518+0000","last_fullsized":"2024-09-21T07:49:36.965518+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:10.025992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"232'2","reported_seq":17,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.265254+0000","last_change":"2024-09-21T07:49:34.135311+0000","last_active":"2024-09-21T07:49:37.265254+0000","last_peered":"2024-09-21T07:49:37.265254+0000","last_clean":"2024-09-21T07:49:37.265254+0000","last_became_active":"2024-09-21T07:49:34.134991+0000","last_became_peered":"2024-09-21T07:49:34.134991+0000","last_unstale":"2024-09-21T07:49:37.265254+0000","last_undegraded":"2024-09-21T07:49:37.265254+0000","last_fullsized":"2024-09-21T07:49:37.265254+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:19.601267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":327,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-21T07:49:35.132219+0000","last_change":"2024-09-21T07:35:27.867336+0000","last_active":"2024-09-21T07:49:35.132219+0000","last_peered":"2024-09-21T07:49:35.132219+0000","last_clean":"2024-09-21T07:49:35.132219+0000","last_became_active":"2024-09-21T07:35:27.867015+0000","last_became_peered":"2024-09-21T07:35:27.867015+0000","last_unstale":"2024-09-21T07:49:35.132219+0000","last_undegraded":"2024-09-21T07:49:35.132219+0000","last_fullsized":"2024-09-21T07:49:35.132219+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:59.214139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":338,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.265064+0000","last_change":"2024-09-21T07:35:29.493137+0000","last_active":"2024-09-21T07:49:37.265064+0000","last_peered":"2024-09-21T07:49:37.265064+0000","last_clean":"2024-09-21T07:49:37.265064+0000","last_became_active":"2024-09-21T07:35:26.872230+0000","last_became_peered":"2024-09-21T07:35:26.872230+0000","last_unstale":"2024-09-21T07:49:37.265064+0000","last_undegraded":"2024-09-21T07:49:37.265064+0000","last_fullsized":"2024-09-21T07:49:37.265064+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:24.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001197449,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":392,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.263897+0000","last_change":"2024-09-21T07:34:24.258754+0000","last_active":"2024-09-21T07:49:37.263897+0000","last_peered":"2024-09-21T07:49:37.263897+0000","last_clean":"2024-09-21T07:49:37.263897+0000","last_became_active":"2024-09-21T07:34:23.272657+0000","last_became_peered":"2024-09-21T07:34:23.272657+0000","last_unstale":"2024-09-21T07:49:37.263897+0000","last_undegraded":"2024-09-21T07:49:37.263897+0000","last_fullsized":"2024-09-21T07:49:37.263897+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:05.477099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067143200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264799+0000","last_change":"2024-09-21T07:49:35.146953+0000","last_active":"2024-09-21T07:49:37.264799+0000","last_peered":"2024-09-21T07:49:37.264799+0000","last_clean":"2024-09-21T07:49:37.264799+0000","last_became_active":"2024-09-21T07:49:35.146634+0000","last_became_peered":"2024-09-21T07:49:35.146634+0000","last_unstale":"2024-09-21T07:49:37.264799+0000","last_undegraded":"2024-09-21T07:49:37.264799+0000","last_fullsized":"2024-09-21T07:49:37.264799+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:35.687662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.965653+0000","last_change":"2024-09-21T07:49:34.133721+0000","last_active":"2024-09-21T07:49:36.965653+0000","last_peered":"2024-09-21T07:49:36.965653+0000","last_clean":"2024-09-21T07:49:36.965653+0000","last_became_active":"2024-09-21T07:49:34.133423+0000","last_became_peered":"2024-09-21T07:49:34.133423+0000","last_unstale":"2024-09-21T07:49:36.965653+0000","last_undegraded":"2024-09-21T07:49:36.965653+0000","last_fullsized":"2024-09-21T07:49:36.965653+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:36.825122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":333,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.965431+0000","last_change":"2024-09-21T07:35:27.876021+0000","last_active":"2024-09-21T07:49:36.965431+0000","last_peered":"2024-09-21T07:49:36.965431+0000","last_clean":"2024-09-21T07:49:36.965431+0000","last_became_active":"2024-09-21T07:35:27.875724+0000","last_became_peered":"2024-09-21T07:35:27.875724+0000","last_unstale":"2024-09-21T07:49:36.965431+0000","last_undegraded":"2024-09-21T07:49:36.965431+0000","last_fullsized":"2024-09-21T07:49:36.965431+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:03.022812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":338,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264673+0000","last_change":"2024-09-21T07:35:29.492570+0000","last_active":"2024-09-21T07:49:37.264673+0000","last_peered":"2024-09-21T07:49:37.264673+0000","last_clean":"2024-09-21T07:49:37.264673+0000","last_became_active":"2024-09-21T07:35:26.874316+0000","last_became_peered":"2024-09-21T07:35:26.874316+0000","last_unstale":"2024-09-21T07:49:37.264673+0000","last_undegraded":"2024-09-21T07:49:37.264673+0000","last_fullsized":"2024-09-21T07:49:37.264673+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:16.512323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011990760000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":392,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.965657+0000","last_change":"2024-09-21T07:34:25.085232+0000","last_active":"2024-09-21T07:49:36.965657+0000","last_peered":"2024-09-21T07:49:36.965657+0000","last_clean":"2024-09-21T07:49:36.965657+0000","last_became_active":"2024-09-21T07:34:23.277807+0000","last_became_peered":"2024-09-21T07:34:23.277807+0000","last_unstale":"2024-09-21T07:49:36.965657+0000","last_undegraded":"2024-09-21T07:49:36.965657+0000","last_fullsized":"2024-09-21T07:49:36.965657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:36.362919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086619099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.145452+0000","last_change":"2024-09-21T07:49:35.130292+0000","last_active":"2024-09-21T07:49:36.145452+0000","last_peered":"2024-09-21T07:49:36.145452+0000","last_clean":"2024-09-21T07:49:36.145452+0000","last_became_active":"2024-09-21T07:49:35.129962+0000","last_became_peered":"2024-09-21T07:49:35.129962+0000","last_unstale":"2024-09-21T07:49:36.145452+0000","last_undegraded":"2024-09-21T07:49:36.145452+0000","last_fullsized":"2024-09-21T07:49:36.145452+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:39.715572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"232'3","reported_seq":18,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264495+0000","last_change":"2024-09-21T07:49:34.132809+0000","last_active":"2024-09-21T07:49:37.264495+0000","last_peered":"2024-09-21T07:49:37.264495+0000","last_clean":"2024-09-21T07:49:37.264495+0000","last_became_active":"2024-09-21T07:49:34.132298+0000","last_became_peered":"2024-09-21T07:49:34.132298+0000","last_unstale":"2024-09-21T07:49:37.264495+0000","last_undegraded":"2024-09-21T07:49:37.264495+0000","last_fullsized":"2024-09-21T07:49:37.264495+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:18:11.773683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":335,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.266464+0000","last_change":"2024-09-21T07:35:27.872798+0000","last_active":"2024-09-21T07:49:37.266464+0000","last_peered":"2024-09-21T07:49:37.266464+0000","last_clean":"2024-09-21T07:49:37.266464+0000","last_became_active":"2024-09-21T07:35:27.872483+0000","last_became_peered":"2024-09-21T07:35:27.872483+0000","last_unstale":"2024-09-21T07:49:37.266464+0000","last_undegraded":"2024-09-21T07:49:37.266464+0000","last_fullsized":"2024-09-21T07:49:37.266464+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:11.933469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":342,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.265481+0000","last_change":"2024-09-21T07:35:29.206723+0000","last_active":"2024-09-21T07:49:37.265481+0000","last_peered":"2024-09-21T07:49:37.265481+0000","last_clean":"2024-09-21T07:49:37.265481+0000","last_became_active":"2024-09-21T07:35:26.872377+0000","last_became_peered":"2024-09-21T07:35:26.872377+0000","last_unstale":"2024-09-21T07:49:37.265481+0000","last_undegraded":"2024-09-21T07:49:37.265481+0000","last_fullsized":"2024-09-21T07:49:37.265481+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:29.086862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083077200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":383,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-21T07:49:35.139703+0000","last_change":"2024-09-21T07:34:24.597672+0000","last_active":"2024-09-21T07:49:35.139703+0000","last_peered":"2024-09-21T07:49:35.139703+0000","last_clean":"2024-09-21T07:49:35.139703+0000","last_became_active":"2024-09-21T07:34:23.267770+0000","last_became_peered":"2024-09-21T07:34:23.267770+0000","last_unstale":"2024-09-21T07:49:35.139703+0000","last_undegraded":"2024-09-21T07:49:35.139703+0000","last_fullsized":"2024-09-21T07:49:35.139703+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:51.629476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086378500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":10,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-21T07:49:35.149976+0000","last_change":"2024-09-21T07:49:35.143803+0000","last_active":"2024-09-21T07:49:35.149976+0000","last_peered":"2024-09-21T07:49:35.149976+0000","last_clean":"2024-09-21T07:49:35.149976+0000","last_became_active":"2024-09-21T07:49:35.143462+0000","last_became_peered":"2024-09-21T07:49:35.143462+0000","last_unstale":"2024-09-21T07:49:35.149976+0000","last_undegraded":"2024-09-21T07:49:35.149976+0000","last_fullsized":"2024-09-21T07:49:35.149976+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:14.249616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.14","version":"232'6","reported_seq":21,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.142425+0000","last_change":"2024-09-21T07:49:34.139691+0000","last_active":"2024-09-21T07:49:36.142425+0000","last_peered":"2024-09-21T07:49:36.142425+0000","last_clean":"2024-09-21T07:49:36.142425+0000","last_became_active":"2024-09-21T07:49:34.139228+0000","last_became_peered":"2024-09-21T07:49:34.139228+0000","last_unstale":"2024-09-21T07:49:36.142425+0000","last_undegraded":"2024-09-21T07:49:36.142425+0000","last_fullsized":"2024-09-21T07:49:36.142425+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:50:51.158824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":335,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.266157+0000","last_change":"2024-09-21T07:35:27.870121+0000","last_active":"2024-09-21T07:49:37.266157+0000","last_peered":"2024-09-21T07:49:37.266157+0000","last_clean":"2024-09-21T07:49:37.266157+0000","last_became_active":"2024-09-21T07:35:27.869811+0000","last_became_peered":"2024-09-21T07:35:27.869811+0000","last_unstale":"2024-09-21T07:49:37.266157+0000","last_undegraded":"2024-09-21T07:49:37.266157+0000","last_fullsized":"2024-09-21T07:49:37.266157+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:31.778461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":342,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143415+0000","last_change":"2024-09-21T07:35:28.860699+0000","last_active":"2024-09-21T07:49:36.143415+0000","last_peered":"2024-09-21T07:49:36.143415+0000","last_clean":"2024-09-21T07:49:36.143415+0000","last_became_active":"2024-09-21T07:35:26.876779+0000","last_became_peered":"2024-09-21T07:35:26.876779+0000","last_unstale":"2024-09-21T07:49:36.143415+0000","last_undegraded":"2024-09-21T07:49:36.143415+0000","last_fullsized":"2024-09-21T07:49:36.143415+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:28.286403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000913188,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":394,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.145914+0000","last_change":"2024-09-21T07:34:24.260181+0000","last_active":"2024-09-21T07:49:36.145914+0000","last_peered":"2024-09-21T07:49:36.145914+0000","last_clean":"2024-09-21T07:49:36.145914+0000","last_became_active":"2024-09-21T07:34:23.275580+0000","last_became_peered":"2024-09-21T07:34:23.275580+0000","last_unstale":"2024-09-21T07:49:36.145914+0000","last_undegraded":"2024-09-21T07:49:36.145914+0000","last_fullsized":"2024-09-21T07:49:36.145914+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:49.857622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014706980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.122197+0000","last_change":"2024-09-21T07:49:35.146412+0000","last_active":"2024-09-21T07:49:37.122197+0000","last_peered":"2024-09-21T07:49:37.122197+0000","last_clean":"2024-09-21T07:49:37.122197+0000","last_became_active":"2024-09-21T07:49:35.146118+0000","last_became_peered":"2024-09-21T07:49:35.146118+0000","last_unstale":"2024-09-21T07:49:37.122197+0000","last_undegraded":"2024-09-21T07:49:37.122197+0000","last_fullsized":"2024-09-21T07:49:37.122197+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:40.422511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.265996+0000","last_change":"2024-09-21T07:49:34.132216+0000","last_active":"2024-09-21T07:49:37.265996+0000","last_peered":"2024-09-21T07:49:37.265996+0000","last_clean":"2024-09-21T07:49:37.265996+0000","last_became_active":"2024-09-21T07:49:34.131867+0000","last_became_peered":"2024-09-21T07:49:34.131867+0000","last_unstale":"2024-09-21T07:49:37.265996+0000","last_undegraded":"2024-09-21T07:49:37.265996+0000","last_fullsized":"2024-09-21T07:49:37.265996+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:03:21.509136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":336,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.122033+0000","last_change":"2024-09-21T07:35:27.875393+0000","last_active":"2024-09-21T07:49:37.122033+0000","last_peered":"2024-09-21T07:49:37.122033+0000","last_clean":"2024-09-21T07:49:37.122033+0000","last_became_active":"2024-09-21T07:35:27.874671+0000","last_became_peered":"2024-09-21T07:35:27.874671+0000","last_unstale":"2024-09-21T07:49:37.122033+0000","last_undegraded":"2024-09-21T07:49:37.122033+0000","last_fullsized":"2024-09-21T07:49:37.122033+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":342,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264970+0000","last_change":"2024-09-21T07:35:29.206296+0000","last_active":"2024-09-21T07:49:37.264970+0000","last_peered":"2024-09-21T07:49:37.264970+0000","last_clean":"2024-09-21T07:49:37.264970+0000","last_became_active":"2024-09-21T07:35:26.874575+0000","last_became_peered":"2024-09-21T07:35:26.874575+0000","last_unstale":"2024-09-21T07:49:37.264970+0000","last_undegraded":"2024-09-21T07:49:37.264970+0000","last_fullsized":"2024-09-21T07:49:37.264970+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:22.771154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061011599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":395,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.144697+0000","last_change":"2024-09-21T07:34:24.259112+0000","last_active":"2024-09-21T07:49:36.144697+0000","last_peered":"2024-09-21T07:49:36.144697+0000","last_clean":"2024-09-21T07:49:36.144697+0000","last_became_active":"2024-09-21T07:34:23.635444+0000","last_became_peered":"2024-09-21T07:34:23.635444+0000","last_unstale":"2024-09-21T07:49:36.144697+0000","last_undegraded":"2024-09-21T07:49:36.144697+0000","last_fullsized":"2024-09-21T07:49:36.144697+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:46.255385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001952636,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.122538+0000","last_change":"2024-09-21T07:49:35.149455+0000","last_active":"2024-09-21T07:49:37.122538+0000","last_peered":"2024-09-21T07:49:37.122538+0000","last_clean":"2024-09-21T07:49:37.122538+0000","last_became_active":"2024-09-21T07:49:35.149005+0000","last_became_peered":"2024-09-21T07:49:35.149005+0000","last_unstale":"2024-09-21T07:49:37.122538+0000","last_undegraded":"2024-09-21T07:49:37.122538+0000","last_fullsized":"2024-09-21T07:49:37.122538+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:07.998627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":13,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-21T07:49:35.138561+0000","last_change":"2024-09-21T07:49:34.144249+0000","last_active":"2024-09-21T07:49:35.138561+0000","last_peered":"2024-09-21T07:49:35.138561+0000","last_clean":"2024-09-21T07:49:35.138561+0000","last_became_active":"2024-09-21T07:49:34.143806+0000","last_became_peered":"2024-09-21T07:49:34.143806+0000","last_unstale":"2024-09-21T07:49:35.138561+0000","last_undegraded":"2024-09-21T07:49:35.138561+0000","last_fullsized":"2024-09-21T07:49:35.138561+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:51:12.362241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":336,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.122233+0000","last_change":"2024-09-21T07:35:27.878356+0000","last_active":"2024-09-21T07:49:37.122233+0000","last_peered":"2024-09-21T07:49:37.122233+0000","last_clean":"2024-09-21T07:49:37.122233+0000","last_became_active":"2024-09-21T07:35:27.877982+0000","last_became_peered":"2024-09-21T07:35:27.877982+0000","last_unstale":"2024-09-21T07:49:37.122233+0000","last_undegraded":"2024-09-21T07:49:37.122233+0000","last_fullsized":"2024-09-21T07:49:37.122233+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":344,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.121317+0000","last_change":"2024-09-21T07:35:29.053131+0000","last_active":"2024-09-21T07:49:37.121317+0000","last_peered":"2024-09-21T07:49:37.121317+0000","last_clean":"2024-09-21T07:49:37.121317+0000","last_became_active":"2024-09-21T07:35:26.870516+0000","last_became_peered":"2024-09-21T07:35:26.870516+0000","last_unstale":"2024-09-21T07:49:37.121317+0000","last_undegraded":"2024-09-21T07:49:37.121317+0000","last_fullsized":"2024-09-21T07:49:37.121317+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:10.924491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070705699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":407,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143191+0000","last_change":"2024-09-21T07:34:24.539152+0000","last_active":"2024-09-21T07:49:36.143191+0000","last_peered":"2024-09-21T07:49:36.143191+0000","last_clean":"2024-09-21T07:49:36.143191+0000","last_became_active":"2024-09-21T07:34:23.259573+0000","last_became_peered":"2024-09-21T07:34:23.259573+0000","last_unstale":"2024-09-21T07:49:36.143191+0000","last_undegraded":"2024-09-21T07:49:36.143191+0000","last_fullsized":"2024-09-21T07:49:36.143191+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:26.986900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058108400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.965393+0000","last_change":"2024-09-21T07:49:35.144737+0000","last_active":"2024-09-21T07:49:36.965393+0000","last_peered":"2024-09-21T07:49:36.965393+0000","last_clean":"2024-09-21T07:49:36.965393+0000","last_became_active":"2024-09-21T07:49:35.144412+0000","last_became_peered":"2024-09-21T07:49:35.144412+0000","last_unstale":"2024-09-21T07:49:36.965393+0000","last_undegraded":"2024-09-21T07:49:36.965393+0000","last_fullsized":"2024-09-21T07:49:36.965393+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:30.582221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.121530+0000","last_change":"2024-09-21T07:49:34.131998+0000","last_active":"2024-09-21T07:49:37.121530+0000","last_peered":"2024-09-21T07:49:37.121530+0000","last_clean":"2024-09-21T07:49:37.121530+0000","last_became_active":"2024-09-21T07:49:34.131674+0000","last_became_peered":"2024-09-21T07:49:34.131674+0000","last_unstale":"2024-09-21T07:49:37.121530+0000","last_undegraded":"2024-09-21T07:49:37.121530+0000","last_fullsized":"2024-09-21T07:49:37.121530+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:04.502261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":335,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.121267+0000","last_change":"2024-09-21T07:35:27.873726+0000","last_active":"2024-09-21T07:49:37.121267+0000","last_peered":"2024-09-21T07:49:37.121267+0000","last_clean":"2024-09-21T07:49:37.121267+0000","last_became_active":"2024-09-21T07:35:27.871421+0000","last_became_peered":"2024-09-21T07:35:27.871421+0000","last_unstale":"2024-09-21T07:49:37.121267+0000","last_undegraded":"2024-09-21T07:49:37.121267+0000","last_fullsized":"2024-09-21T07:49:37.121267+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:00.168915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":342,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.145722+0000","last_change":"2024-09-21T07:35:28.861032+0000","last_active":"2024-09-21T07:49:36.145722+0000","last_peered":"2024-09-21T07:49:36.145722+0000","last_clean":"2024-09-21T07:49:36.145722+0000","last_became_active":"2024-09-21T07:35:26.873149+0000","last_became_peered":"2024-09-21T07:35:26.873149+0000","last_unstale":"2024-09-21T07:49:36.145722+0000","last_undegraded":"2024-09-21T07:49:36.145722+0000","last_fullsized":"2024-09-21T07:49:36.145722+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:59.435981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011988369999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":391,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264585+0000","last_change":"2024-09-21T07:34:24.259902+0000","last_active":"2024-09-21T07:49:37.264585+0000","last_peered":"2024-09-21T07:49:37.264585+0000","last_clean":"2024-09-21T07:49:37.264585+0000","last_became_active":"2024-09-21T07:34:23.271902+0000","last_became_peered":"2024-09-21T07:34:23.271902+0000","last_unstale":"2024-09-21T07:49:37.264585+0000","last_undegraded":"2024-09-21T07:49:37.264585+0000","last_fullsized":"2024-09-21T07:49:37.264585+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:59.439525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001686358,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.144373+0000","last_change":"2024-09-21T07:49:35.138262+0000","last_active":"2024-09-21T07:49:36.144373+0000","last_peered":"2024-09-21T07:49:36.144373+0000","last_clean":"2024-09-21T07:49:36.144373+0000","last_became_active":"2024-09-21T07:49:35.136582+0000","last_became_peered":"2024-09-21T07:49:35.136582+0000","last_unstale":"2024-09-21T07:49:36.144373+0000","last_undegraded":"2024-09-21T07:49:36.144373+0000","last_fullsized":"2024-09-21T07:49:36.144373+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:45.528921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143616+0000","last_change":"2024-09-21T07:49:34.140307+0000","last_active":"2024-09-21T07:49:36.143616+0000","last_peered":"2024-09-21T07:49:36.143616+0000","last_clean":"2024-09-21T07:49:36.143616+0000","last_became_active":"2024-09-21T07:49:34.139959+0000","last_became_peered":"2024-09-21T07:49:34.139959+0000","last_unstale":"2024-09-21T07:49:36.143616+0000","last_undegraded":"2024-09-21T07:49:36.143616+0000","last_fullsized":"2024-09-21T07:49:36.143616+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:02:01.638863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":332,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.965076+0000","last_change":"2024-09-21T07:35:27.872637+0000","last_active":"2024-09-21T07:49:36.965076+0000","last_peered":"2024-09-21T07:49:36.965076+0000","last_clean":"2024-09-21T07:49:36.965076+0000","last_became_active":"2024-09-21T07:35:27.872149+0000","last_became_peered":"2024-09-21T07:35:27.872149+0000","last_unstale":"2024-09-21T07:49:36.965076+0000","last_undegraded":"2024-09-21T07:49:36.965076+0000","last_fullsized":"2024-09-21T07:49:36.965076+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:27.697872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":339,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264748+0000","last_change":"2024-09-21T07:35:29.493362+0000","last_active":"2024-09-21T07:49:37.264748+0000","last_peered":"2024-09-21T07:49:37.264748+0000","last_clean":"2024-09-21T07:49:37.264748+0000","last_became_active":"2024-09-21T07:35:26.873018+0000","last_became_peered":"2024-09-21T07:35:26.873018+0000","last_unstale":"2024-09-21T07:49:37.264748+0000","last_undegraded":"2024-09-21T07:49:37.264748+0000","last_fullsized":"2024-09-21T07:49:37.264748+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:02.460557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012496510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":395,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.144241+0000","last_change":"2024-09-21T07:34:24.259662+0000","last_active":"2024-09-21T07:49:36.144241+0000","last_peered":"2024-09-21T07:49:36.144241+0000","last_clean":"2024-09-21T07:49:36.144241+0000","last_became_active":"2024-09-21T07:34:23.278812+0000","last_became_peered":"2024-09-21T07:34:23.278812+0000","last_unstale":"2024-09-21T07:49:36.144241+0000","last_undegraded":"2024-09-21T07:49:36.144241+0000","last_fullsized":"2024-09-21T07:49:36.144241+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:49.829974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356463,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264200+0000","last_change":"2024-09-21T07:49:35.143792+0000","last_active":"2024-09-21T07:49:37.264200+0000","last_peered":"2024-09-21T07:49:37.264200+0000","last_clean":"2024-09-21T07:49:37.264200+0000","last_became_active":"2024-09-21T07:49:35.143413+0000","last_became_peered":"2024-09-21T07:49:35.143413+0000","last_unstale":"2024-09-21T07:49:37.264200+0000","last_undegraded":"2024-09-21T07:49:37.264200+0000","last_fullsized":"2024-09-21T07:49:37.264200+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:03:29.921904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"232'2","reported_seq":17,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.265298+0000","last_change":"2024-09-21T07:49:34.134681+0000","last_active":"2024-09-21T07:49:37.265298+0000","last_peered":"2024-09-21T07:49:37.265298+0000","last_clean":"2024-09-21T07:49:37.265298+0000","last_became_active":"2024-09-21T07:49:34.134299+0000","last_became_peered":"2024-09-21T07:49:34.134299+0000","last_unstale":"2024-09-21T07:49:37.265298+0000","last_undegraded":"2024-09-21T07:49:37.265298+0000","last_fullsized":"2024-09-21T07:49:37.265298+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:34.178661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":326,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-21T07:49:35.132115+0000","last_change":"2024-09-21T07:35:27.868448+0000","last_active":"2024-09-21T07:49:35.132115+0000","last_peered":"2024-09-21T07:49:35.132115+0000","last_clean":"2024-09-21T07:49:35.132115+0000","last_became_active":"2024-09-21T07:35:27.868105+0000","last_became_peered":"2024-09-21T07:35:27.868105+0000","last_unstale":"2024-09-21T07:49:35.132115+0000","last_undegraded":"2024-09-21T07:49:35.132115+0000","last_fullsized":"2024-09-21T07:49:35.132115+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:41.468702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":337,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264584+0000","last_change":"2024-09-21T07:35:29.492974+0000","last_active":"2024-09-21T07:49:37.264584+0000","last_peered":"2024-09-21T07:49:37.264584+0000","last_clean":"2024-09-21T07:49:37.264584+0000","last_became_active":"2024-09-21T07:35:26.872165+0000","last_became_peered":"2024-09-21T07:35:26.872165+0000","last_unstale":"2024-09-21T07:49:37.264584+0000","last_undegraded":"2024-09-21T07:49:37.264584+0000","last_fullsized":"2024-09-21T07:49:37.264584+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:59.912520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013795179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":391,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.965141+0000","last_change":"2024-09-21T07:34:25.084861+0000","last_active":"2024-09-21T07:49:36.965141+0000","last_peered":"2024-09-21T07:49:36.965141+0000","last_clean":"2024-09-21T07:49:36.965141+0000","last_became_active":"2024-09-21T07:34:23.277708+0000","last_became_peered":"2024-09-21T07:34:23.277708+0000","last_unstale":"2024-09-21T07:49:36.965141+0000","last_undegraded":"2024-09-21T07:49:36.965141+0000","last_fullsized":"2024-09-21T07:49:36.965141+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:40.816488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000597566,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264247+0000","last_change":"2024-09-21T07:49:35.144350+0000","last_active":"2024-09-21T07:49:37.264247+0000","last_peered":"2024-09-21T07:49:37.264247+0000","last_clean":"2024-09-21T07:49:37.264247+0000","last_became_active":"2024-09-21T07:49:35.141888+0000","last_became_peered":"2024-09-21T07:49:35.141888+0000","last_unstale":"2024-09-21T07:49:37.264247+0000","last_undegraded":"2024-09-21T07:49:37.264247+0000","last_fullsized":"2024-09-21T07:49:37.264247+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:46:36.677708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.121702+0000","last_change":"2024-09-21T07:49:34.136417+0000","last_active":"2024-09-21T07:49:37.121702+0000","last_peered":"2024-09-21T07:49:37.121702+0000","last_clean":"2024-09-21T07:49:37.121702+0000","last_became_active":"2024-09-21T07:49:34.132982+0000","last_became_peered":"2024-09-21T07:49:34.132982+0000","last_unstale":"2024-09-21T07:49:37.121702+0000","last_undegraded":"2024-09-21T07:49:37.121702+0000","last_fullsized":"2024-09-21T07:49:37.121702+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:46.034969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":326,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-21T07:49:35.133820+0000","last_change":"2024-09-21T07:35:27.874640+0000","last_active":"2024-09-21T07:49:35.133820+0000","last_peered":"2024-09-21T07:49:35.133820+0000","last_clean":"2024-09-21T07:49:35.133820+0000","last_became_active":"2024-09-21T07:35:27.874265+0000","last_became_peered":"2024-09-21T07:35:27.874265+0000","last_unstale":"2024-09-21T07:49:35.133820+0000","last_undegraded":"2024-09-21T07:49:35.133820+0000","last_fullsized":"2024-09-21T07:49:35.133820+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:45.754654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":341,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.266678+0000","last_change":"2024-09-21T07:35:29.207392+0000","last_active":"2024-09-21T07:49:37.266678+0000","last_peered":"2024-09-21T07:49:37.266678+0000","last_clean":"2024-09-21T07:49:37.266678+0000","last_became_active":"2024-09-21T07:35:26.870917+0000","last_became_peered":"2024-09-21T07:35:26.870917+0000","last_unstale":"2024-09-21T07:49:37.266678+0000","last_undegraded":"2024-09-21T07:49:37.266678+0000","last_fullsized":"2024-09-21T07:49:37.266678+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:59.159937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059198499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":395,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143926+0000","last_change":"2024-09-21T07:34:24.258709+0000","last_active":"2024-09-21T07:49:36.143926+0000","last_peered":"2024-09-21T07:49:36.143926+0000","last_clean":"2024-09-21T07:49:36.143926+0000","last_became_active":"2024-09-21T07:34:23.279042+0000","last_became_peered":"2024-09-21T07:34:23.279042+0000","last_unstale":"2024-09-21T07:49:36.143926+0000","last_undegraded":"2024-09-21T07:49:36.143926+0000","last_fullsized":"2024-09-21T07:49:36.143926+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:01.762117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019282100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.145517+0000","last_change":"2024-09-21T07:49:35.154518+0000","last_active":"2024-09-21T07:49:36.145517+0000","last_peered":"2024-09-21T07:49:36.145517+0000","last_clean":"2024-09-21T07:49:36.145517+0000","last_became_active":"2024-09-21T07:49:35.154227+0000","last_became_peered":"2024-09-21T07:49:35.154227+0000","last_unstale":"2024-09-21T07:49:36.145517+0000","last_undegraded":"2024-09-21T07:49:36.145517+0000","last_fullsized":"2024-09-21T07:49:36.145517+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:44.338565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"232'1","reported_seq":16,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.266431+0000","last_change":"2024-09-21T07:49:34.136167+0000","last_active":"2024-09-21T07:49:37.266431+0000","last_peered":"2024-09-21T07:49:37.266431+0000","last_clean":"2024-09-21T07:49:37.266431+0000","last_became_active":"2024-09-21T07:49:34.133552+0000","last_became_peered":"2024-09-21T07:49:34.133552+0000","last_unstale":"2024-09-21T07:49:37.266431+0000","last_undegraded":"2024-09-21T07:49:37.266431+0000","last_fullsized":"2024-09-21T07:49:37.266431+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:20.047656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":332,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.965457+0000","last_change":"2024-09-21T07:35:27.870260+0000","last_active":"2024-09-21T07:49:36.965457+0000","last_peered":"2024-09-21T07:49:36.965457+0000","last_clean":"2024-09-21T07:49:36.965457+0000","last_became_active":"2024-09-21T07:35:27.869448+0000","last_became_peered":"2024-09-21T07:35:27.869448+0000","last_unstale":"2024-09-21T07:49:36.965457+0000","last_undegraded":"2024-09-21T07:49:36.965457+0000","last_fullsized":"2024-09-21T07:49:36.965457+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:16.946790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":333,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-21T07:49:35.132407+0000","last_change":"2024-09-21T07:35:29.386367+0000","last_active":"2024-09-21T07:49:35.132407+0000","last_peered":"2024-09-21T07:49:35.132407+0000","last_clean":"2024-09-21T07:49:35.132407+0000","last_became_active":"2024-09-21T07:35:26.879238+0000","last_became_peered":"2024-09-21T07:35:26.879238+0000","last_unstale":"2024-09-21T07:49:35.132407+0000","last_undegraded":"2024-09-21T07:49:35.132407+0000","last_fullsized":"2024-09-21T07:49:35.132407+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:16.528260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092223100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":391,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.966024+0000","last_change":"2024-09-21T07:34:25.085635+0000","last_active":"2024-09-21T07:49:36.966024+0000","last_peered":"2024-09-21T07:49:36.966024+0000","last_clean":"2024-09-21T07:49:36.966024+0000","last_became_active":"2024-09-21T07:34:23.273566+0000","last_became_peered":"2024-09-21T07:34:23.273566+0000","last_unstale":"2024-09-21T07:49:36.966024+0000","last_undegraded":"2024-09-21T07:49:36.966024+0000","last_fullsized":"2024-09-21T07:49:36.966024+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:14.142272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079897300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.267075+0000","last_change":"2024-09-21T07:49:35.154606+0000","last_active":"2024-09-21T07:49:37.267075+0000","last_peered":"2024-09-21T07:49:37.267075+0000","last_clean":"2024-09-21T07:49:37.267075+0000","last_became_active":"2024-09-21T07:49:35.154308+0000","last_became_peered":"2024-09-21T07:49:35.154308+0000","last_unstale":"2024-09-21T07:49:37.267075+0000","last_undegraded":"2024-09-21T07:49:37.267075+0000","last_fullsized":"2024-09-21T07:49:37.267075+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:45:37.524277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.265993+0000","last_change":"2024-09-21T07:49:34.136959+0000","last_active":"2024-09-21T07:49:37.265993+0000","last_peered":"2024-09-21T07:49:37.265993+0000","last_clean":"2024-09-21T07:49:37.265993+0000","last_became_active":"2024-09-21T07:49:34.133732+0000","last_became_peered":"2024-09-21T07:49:34.133732+0000","last_unstale":"2024-09-21T07:49:37.265993+0000","last_undegraded":"2024-09-21T07:49:37.265993+0000","last_fullsized":"2024-09-21T07:49:37.265993+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:46:08.491878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":332,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.965897+0000","last_change":"2024-09-21T07:35:27.878728+0000","last_active":"2024-09-21T07:49:36.965897+0000","last_peered":"2024-09-21T07:49:36.965897+0000","last_clean":"2024-09-21T07:49:36.965897+0000","last_became_active":"2024-09-21T07:35:27.878335+0000","last_became_peered":"2024-09-21T07:35:27.878335+0000","last_unstale":"2024-09-21T07:49:36.965897+0000","last_undegraded":"2024-09-21T07:49:36.965897+0000","last_fullsized":"2024-09-21T07:49:36.965897+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:25:37.731771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":354,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.142459+0000","last_change":"2024-09-21T07:35:28.858418+0000","last_active":"2024-09-21T07:49:36.142459+0000","last_peered":"2024-09-21T07:49:36.142459+0000","last_clean":"2024-09-21T07:49:36.142459+0000","last_became_active":"2024-09-21T07:35:26.876944+0000","last_became_peered":"2024-09-21T07:35:26.876944+0000","last_unstale":"2024-09-21T07:49:36.142459+0000","last_undegraded":"2024-09-21T07:49:36.142459+0000","last_fullsized":"2024-09-21T07:49:36.142459+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:04.514304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090862799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":383,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-21T07:49:35.137893+0000","last_change":"2024-09-21T07:34:24.597926+0000","last_active":"2024-09-21T07:49:35.137893+0000","last_peered":"2024-09-21T07:49:35.137893+0000","last_clean":"2024-09-21T07:49:35.137893+0000","last_became_active":"2024-09-21T07:34:23.268926+0000","last_became_peered":"2024-09-21T07:34:23.268926+0000","last_unstale":"2024-09-21T07:49:35.137893+0000","last_undegraded":"2024-09-21T07:49:35.137893+0000","last_fullsized":"2024-09-21T07:49:35.137893+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:46.190752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101851,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.966109+0000","last_change":"2024-09-21T07:49:35.161820+0000","last_active":"2024-09-21T07:49:36.966109+0000","last_peered":"2024-09-21T07:49:36.966109+0000","last_clean":"2024-09-21T07:49:36.966109+0000","last_became_active":"2024-09-21T07:49:35.161521+0000","last_became_peered":"2024-09-21T07:49:35.161521+0000","last_unstale":"2024-09-21T07:49:36.966109+0000","last_undegraded":"2024-09-21T07:49:36.966109+0000","last_fullsized":"2024-09-21T07:49:36.966109+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:23.315071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"232'1","reported_seq":16,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.145703+0000","last_change":"2024-09-21T07:49:34.136928+0000","last_active":"2024-09-21T07:49:36.145703+0000","last_peered":"2024-09-21T07:49:36.145703+0000","last_clean":"2024-09-21T07:49:36.145703+0000","last_became_active":"2024-09-21T07:49:34.136627+0000","last_became_peered":"2024-09-21T07:49:34.136627+0000","last_unstale":"2024-09-21T07:49:36.145703+0000","last_undegraded":"2024-09-21T07:49:36.145703+0000","last_fullsized":"2024-09-21T07:49:36.145703+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:13.445455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":328,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-21T07:49:35.138115+0000","last_change":"2024-09-21T07:35:27.877426+0000","last_active":"2024-09-21T07:49:35.138115+0000","last_peered":"2024-09-21T07:49:35.138115+0000","last_clean":"2024-09-21T07:49:35.138115+0000","last_became_active":"2024-09-21T07:35:27.877050+0000","last_became_peered":"2024-09-21T07:35:27.877050+0000","last_unstale":"2024-09-21T07:49:35.138115+0000","last_undegraded":"2024-09-21T07:49:35.138115+0000","last_fullsized":"2024-09-21T07:49:35.138115+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:36.268334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":339,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.965170+0000","last_change":"2024-09-21T07:35:28.862273+0000","last_active":"2024-09-21T07:49:36.965170+0000","last_peered":"2024-09-21T07:49:36.965170+0000","last_clean":"2024-09-21T07:49:36.965170+0000","last_became_active":"2024-09-21T07:35:26.874011+0000","last_became_peered":"2024-09-21T07:35:26.874011+0000","last_unstale":"2024-09-21T07:49:36.965170+0000","last_undegraded":"2024-09-21T07:49:36.965170+0000","last_fullsized":"2024-09-21T07:49:36.965170+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:11.986950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071644799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":394,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.144344+0000","last_change":"2024-09-21T07:34:24.259794+0000","last_active":"2024-09-21T07:49:36.144344+0000","last_peered":"2024-09-21T07:49:36.144344+0000","last_clean":"2024-09-21T07:49:36.144344+0000","last_became_active":"2024-09-21T07:34:23.265679+0000","last_became_peered":"2024-09-21T07:34:23.265679+0000","last_unstale":"2024-09-21T07:49:36.144344+0000","last_undegraded":"2024-09-21T07:49:36.144344+0000","last_fullsized":"2024-09-21T07:49:36.144344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:02.575796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001298518,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.265782+0000","last_change":"2024-09-21T07:49:35.140846+0000","last_active":"2024-09-21T07:49:37.265782+0000","last_peered":"2024-09-21T07:49:37.265782+0000","last_clean":"2024-09-21T07:49:37.265782+0000","last_became_active":"2024-09-21T07:49:35.140507+0000","last_became_peered":"2024-09-21T07:49:35.140507+0000","last_unstale":"2024-09-21T07:49:37.265782+0000","last_undegraded":"2024-09-21T07:49:37.265782+0000","last_fullsized":"2024-09-21T07:49:37.265782+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:02.011482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.e","version":"232'2","reported_seq":17,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.144890+0000","last_change":"2024-09-21T07:49:34.139443+0000","last_active":"2024-09-21T07:49:36.144890+0000","last_peered":"2024-09-21T07:49:36.144890+0000","last_clean":"2024-09-21T07:49:36.144890+0000","last_became_active":"2024-09-21T07:49:34.139011+0000","last_became_peered":"2024-09-21T07:49:34.139011+0000","last_unstale":"2024-09-21T07:49:36.144890+0000","last_undegraded":"2024-09-21T07:49:36.144890+0000","last_fullsized":"2024-09-21T07:49:36.144890+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:39.850584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":335,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.267189+0000","last_change":"2024-09-21T07:35:27.877725+0000","last_active":"2024-09-21T07:49:37.267189+0000","last_peered":"2024-09-21T07:49:37.267189+0000","last_clean":"2024-09-21T07:49:37.267189+0000","last_became_active":"2024-09-21T07:35:27.877398+0000","last_became_peered":"2024-09-21T07:35:27.877398+0000","last_unstale":"2024-09-21T07:49:37.267189+0000","last_undegraded":"2024-09-21T07:49:37.267189+0000","last_fullsized":"2024-09-21T07:49:37.267189+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:05.859762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":341,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143689+0000","last_change":"2024-09-21T07:35:28.860612+0000","last_active":"2024-09-21T07:49:36.143689+0000","last_peered":"2024-09-21T07:49:36.143689+0000","last_clean":"2024-09-21T07:49:36.143689+0000","last_became_active":"2024-09-21T07:35:26.876562+0000","last_became_peered":"2024-09-21T07:35:26.876562+0000","last_unstale":"2024-09-21T07:49:36.143689+0000","last_undegraded":"2024-09-21T07:49:36.143689+0000","last_fullsized":"2024-09-21T07:49:36.143689+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:06.824331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012252470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":391,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.265245+0000","last_change":"2024-09-21T07:34:24.260330+0000","last_active":"2024-09-21T07:49:37.265245+0000","last_peered":"2024-09-21T07:49:37.265245+0000","last_clean":"2024-09-21T07:49:37.265245+0000","last_became_active":"2024-09-21T07:34:23.637210+0000","last_became_peered":"2024-09-21T07:34:23.637210+0000","last_unstale":"2024-09-21T07:49:37.265245+0000","last_undegraded":"2024-09-21T07:49:37.265245+0000","last_fullsized":"2024-09-21T07:49:37.265245+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.488451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001537093,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143221+0000","last_change":"2024-09-21T07:49:35.141820+0000","last_active":"2024-09-21T07:49:36.143221+0000","last_peered":"2024-09-21T07:49:36.143221+0000","last_clean":"2024-09-21T07:49:36.143221+0000","last_became_active":"2024-09-21T07:49:35.140201+0000","last_became_peered":"2024-09-21T07:49:35.140201+0000","last_unstale":"2024-09-21T07:49:36.143221+0000","last_undegraded":"2024-09-21T07:49:36.143221+0000","last_fullsized":"2024-09-21T07:49:36.143221+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:55:12.992135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264088+0000","last_change":"2024-09-21T07:49:34.128778+0000","last_active":"2024-09-21T07:49:37.264088+0000","last_peered":"2024-09-21T07:49:37.264088+0000","last_clean":"2024-09-21T07:49:37.264088+0000","last_became_active":"2024-09-21T07:49:34.128445+0000","last_became_peered":"2024-09-21T07:49:34.128445+0000","last_unstale":"2024-09-21T07:49:37.264088+0000","last_undegraded":"2024-09-21T07:49:37.264088+0000","last_fullsized":"2024-09-21T07:49:37.264088+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:22.246646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":333,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.964989+0000","last_change":"2024-09-21T07:35:27.874757+0000","last_active":"2024-09-21T07:49:36.964989+0000","last_peered":"2024-09-21T07:49:36.964989+0000","last_clean":"2024-09-21T07:49:36.964989+0000","last_became_active":"2024-09-21T07:35:27.874360+0000","last_became_peered":"2024-09-21T07:35:27.874360+0000","last_unstale":"2024-09-21T07:49:36.964989+0000","last_undegraded":"2024-09-21T07:49:36.964989+0000","last_fullsized":"2024-09-21T07:49:36.964989+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:17.646912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":337,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.266231+0000","last_change":"2024-09-21T07:35:29.493431+0000","last_active":"2024-09-21T07:49:37.266231+0000","last_peered":"2024-09-21T07:49:37.266231+0000","last_clean":"2024-09-21T07:49:37.266231+0000","last_became_active":"2024-09-21T07:35:26.873691+0000","last_became_peered":"2024-09-21T07:35:26.873691+0000","last_unstale":"2024-09-21T07:49:37.266231+0000","last_undegraded":"2024-09-21T07:49:37.266231+0000","last_fullsized":"2024-09-21T07:49:37.266231+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:51.705051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001162464,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":383,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-21T07:49:35.132319+0000","last_change":"2024-09-21T07:34:24.597286+0000","last_active":"2024-09-21T07:49:35.132319+0000","last_peered":"2024-09-21T07:49:35.132319+0000","last_clean":"2024-09-21T07:49:35.132319+0000","last_became_active":"2024-09-21T07:34:23.267133+0000","last_became_peered":"2024-09-21T07:34:23.267133+0000","last_unstale":"2024-09-21T07:49:35.132319+0000","last_undegraded":"2024-09-21T07:49:35.132319+0000","last_fullsized":"2024-09-21T07:49:35.132319+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:18.257010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264440+0000","last_change":"2024-09-21T07:49:35.152801+0000","last_active":"2024-09-21T07:49:37.264440+0000","last_peered":"2024-09-21T07:49:37.264440+0000","last_clean":"2024-09-21T07:49:37.264440+0000","last_became_active":"2024-09-21T07:49:35.151957+0000","last_became_peered":"2024-09-21T07:49:35.151957+0000","last_unstale":"2024-09-21T07:49:37.264440+0000","last_undegraded":"2024-09-21T07:49:37.264440+0000","last_fullsized":"2024-09-21T07:49:37.264440+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:54.437494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.265794+0000","last_change":"2024-09-21T07:49:34.139221+0000","last_active":"2024-09-21T07:49:37.265794+0000","last_peered":"2024-09-21T07:49:37.265794+0000","last_clean":"2024-09-21T07:49:37.265794+0000","last_became_active":"2024-09-21T07:49:34.136048+0000","last_became_peered":"2024-09-21T07:49:34.136048+0000","last_unstale":"2024-09-21T07:49:37.265794+0000","last_undegraded":"2024-09-21T07:49:37.265794+0000","last_fullsized":"2024-09-21T07:49:37.265794+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:39:02.927347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":348,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.141843+0000","last_change":"2024-09-21T07:35:27.864458+0000","last_active":"2024-09-21T07:49:36.141843+0000","last_peered":"2024-09-21T07:49:36.141843+0000","last_clean":"2024-09-21T07:49:36.141843+0000","last_became_active":"2024-09-21T07:35:27.864136+0000","last_became_peered":"2024-09-21T07:35:27.864136+0000","last_unstale":"2024-09-21T07:49:36.141843+0000","last_undegraded":"2024-09-21T07:49:36.141843+0000","last_fullsized":"2024-09-21T07:49:36.141843+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:20.162240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":354,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143019+0000","last_change":"2024-09-21T07:35:28.858784+0000","last_active":"2024-09-21T07:49:36.143019+0000","last_peered":"2024-09-21T07:49:36.143019+0000","last_clean":"2024-09-21T07:49:36.143019+0000","last_became_active":"2024-09-21T07:35:26.876890+0000","last_became_peered":"2024-09-21T07:35:26.876890+0000","last_unstale":"2024-09-21T07:49:36.143019+0000","last_undegraded":"2024-09-21T07:49:36.143019+0000","last_fullsized":"2024-09-21T07:49:36.143019+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:40.738327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064543499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":395,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.144123+0000","last_change":"2024-09-21T07:34:24.258712+0000","last_active":"2024-09-21T07:49:36.144123+0000","last_peered":"2024-09-21T07:49:36.144123+0000","last_clean":"2024-09-21T07:49:36.144123+0000","last_became_active":"2024-09-21T07:34:23.274692+0000","last_became_peered":"2024-09-21T07:34:23.274692+0000","last_unstale":"2024-09-21T07:49:36.144123+0000","last_undegraded":"2024-09-21T07:49:36.144123+0000","last_fullsized":"2024-09-21T07:49:36.144123+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:43.279454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018051479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143469+0000","last_change":"2024-09-21T07:49:35.142235+0000","last_active":"2024-09-21T07:49:36.143469+0000","last_peered":"2024-09-21T07:49:36.143469+0000","last_clean":"2024-09-21T07:49:36.143469+0000","last_became_active":"2024-09-21T07:49:35.141898+0000","last_became_peered":"2024-09-21T07:49:35.141898+0000","last_unstale":"2024-09-21T07:49:36.143469+0000","last_undegraded":"2024-09-21T07:49:36.143469+0000","last_fullsized":"2024-09-21T07:49:36.143469+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:24:32.544254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.265401+0000","last_change":"2024-09-21T07:49:34.141564+0000","last_active":"2024-09-21T07:49:37.265401+0000","last_peered":"2024-09-21T07:49:37.265401+0000","last_clean":"2024-09-21T07:49:37.265401+0000","last_became_active":"2024-09-21T07:49:34.141237+0000","last_became_peered":"2024-09-21T07:49:34.141237+0000","last_unstale":"2024-09-21T07:49:37.265401+0000","last_undegraded":"2024-09-21T07:49:37.265401+0000","last_fullsized":"2024-09-21T07:49:37.265401+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:02.587670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":334,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.144178+0000","last_change":"2024-09-21T07:35:27.878020+0000","last_active":"2024-09-21T07:49:36.144178+0000","last_peered":"2024-09-21T07:49:36.144178+0000","last_clean":"2024-09-21T07:49:36.144178+0000","last_became_active":"2024-09-21T07:35:27.877532+0000","last_became_peered":"2024-09-21T07:35:27.877532+0000","last_unstale":"2024-09-21T07:49:36.144178+0000","last_undegraded":"2024-09-21T07:49:36.144178+0000","last_fullsized":"2024-09-21T07:49:36.144178+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:19.756073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":341,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.144488+0000","last_change":"2024-09-21T07:35:28.861049+0000","last_active":"2024-09-21T07:49:36.144488+0000","last_peered":"2024-09-21T07:49:36.144488+0000","last_clean":"2024-09-21T07:49:36.144488+0000","last_became_active":"2024-09-21T07:35:26.876819+0000","last_became_peered":"2024-09-21T07:35:26.876819+0000","last_unstale":"2024-09-21T07:49:36.144488+0000","last_undegraded":"2024-09-21T07:49:36.144488+0000","last_fullsized":"2024-09-21T07:49:36.144488+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:19.633803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055780599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":391,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.965002+0000","last_change":"2024-09-21T07:34:25.084849+0000","last_active":"2024-09-21T07:49:36.965002+0000","last_peered":"2024-09-21T07:49:36.965002+0000","last_clean":"2024-09-21T07:49:36.965002+0000","last_became_active":"2024-09-21T07:34:23.277713+0000","last_became_peered":"2024-09-21T07:34:23.277713+0000","last_unstale":"2024-09-21T07:49:36.965002+0000","last_undegraded":"2024-09-21T07:49:36.965002+0000","last_fullsized":"2024-09-21T07:49:36.965002+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:05.811810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059079499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143127+0000","last_change":"2024-09-21T07:49:35.137270+0000","last_active":"2024-09-21T07:49:36.143127+0000","last_peered":"2024-09-21T07:49:36.143127+0000","last_clean":"2024-09-21T07:49:36.143127+0000","last_became_active":"2024-09-21T07:49:35.136943+0000","last_became_peered":"2024-09-21T07:49:35.136943+0000","last_unstale":"2024-09-21T07:49:36.143127+0000","last_undegraded":"2024-09-21T07:49:36.143127+0000","last_fullsized":"2024-09-21T07:49:36.143127+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:00.103271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":13,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-21T07:49:35.132039+0000","last_change":"2024-09-21T07:49:34.137760+0000","last_active":"2024-09-21T07:49:35.132039+0000","last_peered":"2024-09-21T07:49:35.132039+0000","last_clean":"2024-09-21T07:49:35.132039+0000","last_became_active":"2024-09-21T07:49:34.137446+0000","last_became_peered":"2024-09-21T07:49:34.137446+0000","last_unstale":"2024-09-21T07:49:35.132039+0000","last_undegraded":"2024-09-21T07:49:35.132039+0000","last_fullsized":"2024-09-21T07:49:35.132039+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:03:04.362781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":336,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.121733+0000","last_change":"2024-09-21T07:35:27.884248+0000","last_active":"2024-09-21T07:49:37.121733+0000","last_peered":"2024-09-21T07:49:37.121733+0000","last_clean":"2024-09-21T07:49:37.121733+0000","last_became_active":"2024-09-21T07:35:27.883861+0000","last_became_peered":"2024-09-21T07:35:27.883861+0000","last_unstale":"2024-09-21T07:49:37.121733+0000","last_undegraded":"2024-09-21T07:49:37.121733+0000","last_fullsized":"2024-09-21T07:49:37.121733+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":342,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.266599+0000","last_change":"2024-09-21T07:35:29.207135+0000","last_active":"2024-09-21T07:49:37.266599+0000","last_peered":"2024-09-21T07:49:37.266599+0000","last_clean":"2024-09-21T07:49:37.266599+0000","last_became_active":"2024-09-21T07:35:26.872787+0000","last_became_peered":"2024-09-21T07:35:26.872787+0000","last_unstale":"2024-09-21T07:49:37.266599+0000","last_undegraded":"2024-09-21T07:49:37.266599+0000","last_fullsized":"2024-09-21T07:49:37.266599+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:27.543217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011157509999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":407,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.141961+0000","last_change":"2024-09-21T07:34:24.538610+0000","last_active":"2024-09-21T07:49:36.141961+0000","last_peered":"2024-09-21T07:49:36.141961+0000","last_clean":"2024-09-21T07:49:36.141961+0000","last_became_active":"2024-09-21T07:34:23.259700+0000","last_became_peered":"2024-09-21T07:34:23.259700+0000","last_unstale":"2024-09-21T07:49:36.141961+0000","last_undegraded":"2024-09-21T07:49:36.141961+0000","last_fullsized":"2024-09-21T07:49:36.141961+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:57.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070522399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.143150+0000","last_change":"2024-09-21T07:49:35.149612+0000","last_active":"2024-09-21T07:49:36.143150+0000","last_peered":"2024-09-21T07:49:36.143150+0000","last_clean":"2024-09-21T07:49:36.143150+0000","last_became_active":"2024-09-21T07:49:35.148902+0000","last_became_peered":"2024-09-21T07:49:35.148902+0000","last_unstale":"2024-09-21T07:49:36.143150+0000","last_undegraded":"2024-09-21T07:49:36.143150+0000","last_fullsized":"2024-09-21T07:49:36.143150+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:56.054648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.122482+0000","last_change":"2024-09-21T07:49:34.131724+0000","last_active":"2024-09-21T07:49:37.122482+0000","last_peered":"2024-09-21T07:49:37.122482+0000","last_clean":"2024-09-21T07:49:37.122482+0000","last_became_active":"2024-09-21T07:49:34.129470+0000","last_became_peered":"2024-09-21T07:49:34.129470+0000","last_unstale":"2024-09-21T07:49:37.122482+0000","last_undegraded":"2024-09-21T07:49:37.122482+0000","last_fullsized":"2024-09-21T07:49:37.122482+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:04.866175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":348,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.142487+0000","last_change":"2024-09-21T07:35:27.868024+0000","last_active":"2024-09-21T07:49:36.142487+0000","last_peered":"2024-09-21T07:49:36.142487+0000","last_clean":"2024-09-21T07:49:36.142487+0000","last_became_active":"2024-09-21T07:35:27.867339+0000","last_became_peered":"2024-09-21T07:35:27.867339+0000","last_unstale":"2024-09-21T07:49:36.142487+0000","last_undegraded":"2024-09-21T07:49:36.142487+0000","last_fullsized":"2024-09-21T07:49:36.142487+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":355,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.142229+0000","last_change":"2024-09-21T07:35:28.858505+0000","last_active":"2024-09-21T07:49:36.142229+0000","last_peered":"2024-09-21T07:49:36.142229+0000","last_clean":"2024-09-21T07:49:36.142229+0000","last_became_active":"2024-09-21T07:35:26.877320+0000","last_became_peered":"2024-09-21T07:35:26.877320+0000","last_unstale":"2024-09-21T07:49:36.142229+0000","last_undegraded":"2024-09-21T07:49:36.142229+0000","last_fullsized":"2024-09-21T07:49:36.142229+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:53.672782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244119999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":384,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-21T07:49:35.138627+0000","last_change":"2024-09-21T07:34:24.597597+0000","last_active":"2024-09-21T07:49:35.138627+0000","last_peered":"2024-09-21T07:49:35.138627+0000","last_clean":"2024-09-21T07:49:35.138627+0000","last_became_active":"2024-09-21T07:34:23.635204+0000","last_became_peered":"2024-09-21T07:34:23.635204+0000","last_unstale":"2024-09-21T07:49:35.138627+0000","last_undegraded":"2024-09-21T07:49:35.138627+0000","last_fullsized":"2024-09-21T07:49:35.138627+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:48.644789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081139199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.145021+0000","last_change":"2024-09-21T07:49:35.133692+0000","last_active":"2024-09-21T07:49:36.145021+0000","last_peered":"2024-09-21T07:49:36.145021+0000","last_clean":"2024-09-21T07:49:36.145021+0000","last_became_active":"2024-09-21T07:49:35.133363+0000","last_became_peered":"2024-09-21T07:49:35.133363+0000","last_unstale":"2024-09-21T07:49:36.145021+0000","last_undegraded":"2024-09-21T07:49:36.145021+0000","last_fullsized":"2024-09-21T07:49:36.145021+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:25.982449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.266130+0000","last_change":"2024-09-21T07:49:34.136563+0000","last_active":"2024-09-21T07:49:37.266130+0000","last_peered":"2024-09-21T07:49:37.266130+0000","last_clean":"2024-09-21T07:49:37.266130+0000","last_became_active":"2024-09-21T07:49:34.134763+0000","last_became_peered":"2024-09-21T07:49:34.134763+0000","last_unstale":"2024-09-21T07:49:37.266130+0000","last_undegraded":"2024-09-21T07:49:37.266130+0000","last_fullsized":"2024-09-21T07:49:37.266130+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:53.730520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":334,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.266034+0000","last_change":"2024-09-21T07:35:27.877503+0000","last_active":"2024-09-21T07:49:37.266034+0000","last_peered":"2024-09-21T07:49:37.266034+0000","last_clean":"2024-09-21T07:49:37.266034+0000","last_became_active":"2024-09-21T07:35:27.875775+0000","last_became_peered":"2024-09-21T07:35:27.875775+0000","last_unstale":"2024-09-21T07:49:37.266034+0000","last_undegraded":"2024-09-21T07:49:37.266034+0000","last_fullsized":"2024-09-21T07:49:37.266034+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:49.291023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":338,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264210+0000","last_change":"2024-09-21T07:35:29.491412+0000","last_active":"2024-09-21T07:49:37.264210+0000","last_peered":"2024-09-21T07:49:37.264210+0000","last_clean":"2024-09-21T07:49:37.264210+0000","last_became_active":"2024-09-21T07:35:26.876404+0000","last_became_peered":"2024-09-21T07:35:26.876404+0000","last_unstale":"2024-09-21T07:49:37.264210+0000","last_undegraded":"2024-09-21T07:49:37.264210+0000","last_fullsized":"2024-09-21T07:49:37.264210+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063918000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":396,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.145769+0000","last_change":"2024-09-21T07:34:24.259026+0000","last_active":"2024-09-21T07:49:36.145769+0000","last_peered":"2024-09-21T07:49:36.145769+0000","last_clean":"2024-09-21T07:49:36.145769+0000","last_became_active":"2024-09-21T07:34:23.635456+0000","last_became_peered":"2024-09-21T07:34:23.635456+0000","last_unstale":"2024-09-21T07:49:36.145769+0000","last_undegraded":"2024-09-21T07:49:36.145769+0000","last_fullsized":"2024-09-21T07:49:36.145769+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:09.592752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018654419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.965787+0000","last_change":"2024-09-21T07:49:35.140067+0000","last_active":"2024-09-21T07:49:36.965787+0000","last_peered":"2024-09-21T07:49:36.965787+0000","last_clean":"2024-09-21T07:49:36.965787+0000","last_became_active":"2024-09-21T07:49:35.139746+0000","last_became_peered":"2024-09-21T07:49:35.139746+0000","last_unstale":"2024-09-21T07:49:36.965787+0000","last_undegraded":"2024-09-21T07:49:36.965787+0000","last_fullsized":"2024-09-21T07:49:36.965787+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:55:09.405295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.7","version":"232'2","reported_seq":17,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.267032+0000","last_change":"2024-09-21T07:49:34.140408+0000","last_active":"2024-09-21T07:49:37.267032+0000","last_peered":"2024-09-21T07:49:37.267032+0000","last_clean":"2024-09-21T07:49:37.267032+0000","last_became_active":"2024-09-21T07:49:34.139400+0000","last_became_peered":"2024-09-21T07:49:34.139400+0000","last_unstale":"2024-09-21T07:49:37.267032+0000","last_undegraded":"2024-09-21T07:49:37.267032+0000","last_fullsized":"2024-09-21T07:49:37.267032+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:11.257785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":334,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.266849+0000","last_change":"2024-09-21T07:35:27.871984+0000","last_active":"2024-09-21T07:49:37.266849+0000","last_peered":"2024-09-21T07:49:37.266849+0000","last_clean":"2024-09-21T07:49:37.266849+0000","last_became_active":"2024-09-21T07:35:27.870336+0000","last_became_peered":"2024-09-21T07:35:27.870336+0000","last_unstale":"2024-09-21T07:49:37.266849+0000","last_undegraded":"2024-09-21T07:49:37.266849+0000","last_fullsized":"2024-09-21T07:49:37.266849+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:57.120479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":342,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.142952+0000","last_change":"2024-09-21T07:35:28.860351+0000","last_active":"2024-09-21T07:49:36.142952+0000","last_peered":"2024-09-21T07:49:36.142952+0000","last_clean":"2024-09-21T07:49:36.142952+0000","last_became_active":"2024-09-21T07:35:26.872046+0000","last_became_peered":"2024-09-21T07:35:26.872046+0000","last_unstale":"2024-09-21T07:49:36.142952+0000","last_undegraded":"2024-09-21T07:49:36.142952+0000","last_fullsized":"2024-09-21T07:49:36.142952+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:47.074327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098384399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":391,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264885+0000","last_change":"2024-09-21T07:34:24.260300+0000","last_active":"2024-09-21T07:49:37.264885+0000","last_peered":"2024-09-21T07:49:37.264885+0000","last_clean":"2024-09-21T07:49:37.264885+0000","last_became_active":"2024-09-21T07:34:23.270128+0000","last_became_peered":"2024-09-21T07:34:23.270128+0000","last_unstale":"2024-09-21T07:49:37.264885+0000","last_undegraded":"2024-09-21T07:49:37.264885+0000","last_fullsized":"2024-09-21T07:49:37.264885+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:25.260931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00176651,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.264970+0000","last_change":"2024-09-21T07:49:35.148351+0000","last_active":"2024-09-21T07:49:37.264970+0000","last_peered":"2024-09-21T07:49:37.264970+0000","last_clean":"2024-09-21T07:49:37.264970+0000","last_became_active":"2024-09-21T07:49:35.145379+0000","last_became_peered":"2024-09-21T07:49:35.145379+0000","last_unstale":"2024-09-21T07:49:37.264970+0000","last_undegraded":"2024-09-21T07:49:37.264970+0000","last_fullsized":"2024-09-21T07:49:37.264970+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:34.098774+0000","last_clean_scrub_stamp":"2024-09-21T07:49:34.098774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:33.272994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.6","version":"233'1","reported_seq":16,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.124252+0000","last_change":"2024-09-21T07:49:34.134878+0000","last_active":"2024-09-21T07:49:37.124252+0000","last_peered":"2024-09-21T07:49:37.124252+0000","last_clean":"2024-09-21T07:49:37.124252+0000","last_became_active":"2024-09-21T07:49:34.134565+0000","last_became_peered":"2024-09-21T07:49:34.134565+0000","last_unstale":"2024-09-21T07:49:37.124252+0000","last_undegraded":"2024-09-21T07:49:37.124252+0000","last_fullsized":"2024-09-21T07:49:37.124252+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:33.092592+0000","last_clean_scrub_stamp":"2024-09-21T07:49:33.092592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:01:32.536536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":334,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.267255+0000","last_change":"2024-09-21T07:35:27.877861+0000","last_active":"2024-09-21T07:49:37.267255+0000","last_peered":"2024-09-21T07:49:37.267255+0000","last_clean":"2024-09-21T07:49:37.267255+0000","last_became_active":"2024-09-21T07:35:27.877434+0000","last_became_peered":"2024-09-21T07:35:27.877434+0000","last_unstale":"2024-09-21T07:49:37.267255+0000","last_undegraded":"2024-09-21T07:49:37.267255+0000","last_fullsized":"2024-09-21T07:49:37.267255+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":340,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:36.965300+0000","last_change":"2024-09-21T07:35:28.862954+0000","last_active":"2024-09-21T07:49:36.965300+0000","last_peered":"2024-09-21T07:49:36.965300+0000","last_clean":"2024-09-21T07:49:36.965300+0000","last_became_active":"2024-09-21T07:35:26.870613+0000","last_became_peered":"2024-09-21T07:35:26.870613+0000","last_unstale":"2024-09-21T07:49:36.965300+0000","last_undegraded":"2024-09-21T07:49:36.965300+0000","last_fullsized":"2024-09-21T07:49:36.965300+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:06.901043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059842100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":394,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-21T07:49:37.122297+0000","last_change":"2024-09-21T07:34:25.028904+0000","last_active":"2024-09-21T07:49:37.122297+0000","last_peered":"2024-09-21T07:49:37.122297+0000","last_clean":"2024-09-21T07:49:37.122297+0000","last_became_active":"2024-09-21T07:34:23.260942+0000","last_became_peered":"2024-09-21T07:34:23.260942+0000","last_unstale":"2024-09-21T07:49:37.122297+0000","last_undegraded":"2024-09-21T07:49:37.122297+0000","last_fullsized":"2024-09-21T07:49:37.122297+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:22.709328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060553899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509642,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66572,"kb_used_data":3444,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94305268,"statfs":{"total":96636764160,"available":96568594432,"internally_reserved":0,"allocated":3526656,"data_stored":2156051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509644,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70168,"kb_used_data":2880,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301672,"statfs":{"total":96636764160,"available":96564912128,"internally_reserved":0,"allocated":2949120,"data_stored":1566749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509642,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34080,"kb_used_data":2884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337760,"statfs":{"total":96636764160,"available":96601866240,"internally_reserved":0,"allocated":2953216,"data_stored":1574300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509643,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29912,"kb_used_data":2876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341928,"statfs":{"total":96636764160,"available":96606134272,"internally_reserved":0,"allocated":2945024,"data_stored":1573234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509644,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47944,"kb_used_data":2868,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323896,"statfs":{"total":96636764160,"available":96587669504,"internally_reserved":0,"allocated":2936832,"data_stored":1572618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509644,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52680,"kb_used_data":3436,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319160,"statfs":{"total":96636764160,"available":96582819840,"internally_reserved":0,"allocated":3518464,"data_stored":2155379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542347,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52104,"kb_used_data":2860,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319736,"statfs":{"total":96636764160,"available":96583409664,"internally_reserved":0,"allocated":2928640,"data_stored":1565101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542346,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74308,"kb_used_data":2860,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297532,"statfs":{"total":96636764160,"available":96560672768,"internally_reserved":0,"allocated":2928640,"data_stored":1565033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:49:40.154 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T07:49:40.154 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T07:49:40.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.418+0000 7f8463e00640 1 -- 172.21.15.39:0/3261701135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84640d0400 msgr2=0x7f84640b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:40.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.418+0000 7f8463e00640 1 --2- 172.21.15.39:0/3261701135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84640d0400 0x7f84640b43f0 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7f8454009830 tx=0x7f845402a480 comp rx=0 tx=0).stop 2024-09-21T07:49:40.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.418+0000 7f8463e00640 1 -- 172.21.15.39:0/3261701135 shutdown_connections 2024-09-21T07:49:40.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.418+0000 7f8463e00640 1 --2- 172.21.15.39:0/3261701135 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f846406f4e0 0x7f846406f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:40.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.418+0000 7f8463e00640 1 --2- 172.21.15.39:0/3261701135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84640d0400 0x7f84640b43f0 unknown :-1 s=CLOSED pgs=2240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:40.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.418+0000 7f8463e00640 1 -- 172.21.15.39:0/3261701135 >> 172.21.15.39:0/3261701135 conn(0x7f84640d0c70 msgr2=0x7f84640d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:40.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.419+0000 7f8463e00640 1 -- 172.21.15.39:0/3261701135 shutdown_connections 2024-09-21T07:49:40.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.419+0000 7f8463e00640 1 -- 172.21.15.39:0/3261701135 wait complete. 2024-09-21T07:49:40.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.421+0000 7f8463e00640 1 Processor -- start 2024-09-21T07:49:40.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.421+0000 7f8463e00640 1 -- start start 2024-09-21T07:49:40.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.422+0000 7f8463e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f846406f4e0 0x7f846415a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:40.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.422+0000 7f8463e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84640d0400 0x7f846414b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:40.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.422+0000 7f8463e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f846415b640 0x7f8464154820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:40.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.422+0000 7f8463e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f84640b54f0 con 0x7f846415b640 2024-09-21T07:49:40.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.422+0000 7f8463e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f84640b5370 con 0x7f846406f4e0 2024-09-21T07:49:40.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.422+0000 7f8463e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f84640b5670 con 0x7f84640d0400 2024-09-21T07:49:40.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.423+0000 7f8463400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f846415b640 0x7f8464154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:40.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.423+0000 7f8463400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f846415b640 0x7f8464154820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41766/0 (socket says 172.21.15.39:41766) 2024-09-21T07:49:40.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.423+0000 7f8463400640 1 -- 172.21.15.39:0/3707294036 learned_addr learned my addr 172.21.15.39:0/3707294036 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:40.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.423+0000 7f8462a00640 1 --2- 172.21.15.39:0/3707294036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f846406f4e0 0x7f846415a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:40.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.423+0000 7f845be00640 1 --2- 172.21.15.39:0/3707294036 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84640d0400 0x7f846414b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:40.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.423+0000 7f8463400640 1 -- 172.21.15.39:0/3707294036 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84640d0400 msgr2=0x7f846414b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:40.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.423+0000 7f8463400640 1 --2- 172.21.15.39:0/3707294036 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84640d0400 0x7f846414b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:40.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.423+0000 7f8463400640 1 -- 172.21.15.39:0/3707294036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f846406f4e0 msgr2=0x7f846415a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:40.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.423+0000 7f8463400640 1 --2- 172.21.15.39:0/3707294036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f846406f4e0 0x7f846415a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:40.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.423+0000 7f8463400640 1 -- 172.21.15.39:0/3707294036 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f846415acb0 con 0x7f846415b640 2024-09-21T07:49:40.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.423+0000 7f845be00640 1 --2- 172.21.15.39:0/3707294036 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84640d0400 0x7f846414b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:49:40.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.423+0000 7f8463400640 1 --2- 172.21.15.39:0/3707294036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f846415b640 0x7f8464154820 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f845c00e990 tx=0x7f845c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:40.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.424+0000 7f845aa00640 1 -- 172.21.15.39:0/3707294036 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c004d20 con 0x7f846415b640 2024-09-21T07:49:40.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.424+0000 7f845aa00640 1 -- 172.21.15.39:0/3707294036 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f845c00caa0 con 0x7f846415b640 2024-09-21T07:49:40.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.424+0000 7f8463e00640 1 -- 172.21.15.39:0/3707294036 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f846415a890 con 0x7f846415b640 2024-09-21T07:49:40.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.424+0000 7f845aa00640 1 -- 172.21.15.39:0/3707294036 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c005020 con 0x7f846415b640 2024-09-21T07:49:40.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.424+0000 7f8463e00640 1 -- 172.21.15.39:0/3707294036 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f846414ba10 con 0x7f846415b640 2024-09-21T07:49:40.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.425+0000 7f8463e00640 1 -- 172.21.15.39:0/3707294036 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84640b5b40 con 0x7f846415b640 2024-09-21T07:49:40.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.426+0000 7f845aa00640 1 -- 172.21.15.39:0/3707294036 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f845c0051c0 con 0x7f846415b640 2024-09-21T07:49:40.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.427+0000 7f845aa00640 1 --2- 172.21.15.39:0/3707294036 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8440078bc0 0x7f844007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:40.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.427+0000 7f845aa00640 1 -- 172.21.15.39:0/3707294036 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(233..233 src has 1..233) ==== 7016+0+0 (secure 0 0 0) 0x7f845c09bbf0 con 0x7f846415b640 2024-09-21T07:49:40.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.427+0000 7f8462a00640 1 --2- 172.21.15.39:0/3707294036 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8440078bc0 0x7f844007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:40.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.428+0000 7f8462a00640 1 --2- 172.21.15.39:0/3707294036 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8440078bc0 0x7f844007b0d0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f8454034000 tx=0x7f8454004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:40.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.431+0000 7f845aa00640 1 -- 172.21.15.39:0/3707294036 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f845c016030 con 0x7f846415b640 2024-09-21T07:49:40.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.701+0000 7f8463e00640 1 -- 172.21.15.39:0/3707294036 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f846406a020 con 0x7f846415b640 2024-09-21T07:49:40.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.705+0000 7f845aa00640 1 -- 172.21.15.39:0/3707294036 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v24) ==== 190+0+147 (secure 0 0 0) 0x7f845c063a70 con 0x7f846415b640 2024-09-21T07:49:40.707 INFO:teuthology.orchestra.run.smithi039.stdout:[client.0] 2024-09-21T07:49:40.707 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQB0du5mGfb1DBAA/aSMKR5or3xy53rB/iUjpA== 2024-09-21T07:49:40.707 INFO:teuthology.orchestra.run.smithi039.stdout: caps mds = "allow" 2024-09-21T07:49:40.707 INFO:teuthology.orchestra.run.smithi039.stdout: caps mon = "allow r" 2024-09-21T07:49:40.707 INFO:teuthology.orchestra.run.smithi039.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T07:49:40.707 INFO:teuthology.orchestra.run.smithi039.stderr:updated caps for client.0 2024-09-21T07:49:40.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.707+0000 7f8463e00640 1 -- 172.21.15.39:0/3707294036 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8440078bc0 msgr2=0x7f844007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:40.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.707+0000 7f8463e00640 1 --2- 172.21.15.39:0/3707294036 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8440078bc0 0x7f844007b0d0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f8454034000 tx=0x7f8454004730 comp rx=0 tx=0).stop 2024-09-21T07:49:40.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.707+0000 7f8463e00640 1 -- 172.21.15.39:0/3707294036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f846415b640 msgr2=0x7f8464154820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:40.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.707+0000 7f8463e00640 1 --2- 172.21.15.39:0/3707294036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f846415b640 0x7f8464154820 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f845c00e990 tx=0x7f845c00ee50 comp rx=0 tx=0).stop 2024-09-21T07:49:40.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.707+0000 7f8463e00640 1 -- 172.21.15.39:0/3707294036 shutdown_connections 2024-09-21T07:49:40.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.707+0000 7f8463e00640 1 --2- 172.21.15.39:0/3707294036 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8440078bc0 0x7f844007b0d0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:40.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.707+0000 7f8463e00640 1 --2- 172.21.15.39:0/3707294036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f846415b640 0x7f8464154820 unknown :-1 s=CLOSED pgs=2241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:40.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.707+0000 7f8463e00640 1 --2- 172.21.15.39:0/3707294036 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f84640d0400 0x7f846414b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:40.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.707+0000 7f8463e00640 1 --2- 172.21.15.39:0/3707294036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f846406f4e0 0x7f846415a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:40.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.707+0000 7f8463e00640 1 -- 172.21.15.39:0/3707294036 >> 172.21.15.39:0/3707294036 conn(0x7f84640d0c70 msgr2=0x7f8464073640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:40.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.708+0000 7f8463e00640 1 -- 172.21.15.39:0/3707294036 shutdown_connections 2024-09-21T07:49:40.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.708+0000 7f8463e00640 1 -- 172.21.15.39:0/3707294036 wait complete. 2024-09-21T07:49:40.731 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:49:40.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.957+0000 7fce4cc00640 1 -- 172.21.15.39:0/3342822191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce2c003fd0 msgr2=0x7fce2c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:40.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.957+0000 7fce4cc00640 1 --2- 172.21.15.39:0/3342822191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce2c003fd0 0x7fce2c004800 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7fce30009b50 tx=0x7fce2c005830 comp rx=0 tx=0).stop 2024-09-21T07:49:40.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.957+0000 7fce47e00640 1 -- 172.21.15.39:0/3342822191 reap_dead start 2024-09-21T07:49:40.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.957+0000 7fce4cc00640 1 -- 172.21.15.39:0/3342822191 shutdown_connections 2024-09-21T07:49:40.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.957+0000 7fce4cc00640 1 -- 172.21.15.39:0/3342822191 >> 172.21.15.39:0/3342822191 conn(0x7fce4806cfc0 msgr2=0x7fce4806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:40.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.957+0000 7fce4cc00640 1 -- 172.21.15.39:0/3342822191 shutdown_connections 2024-09-21T07:49:40.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.957+0000 7fce4cc00640 1 -- 172.21.15.39:0/3342822191 wait complete. 2024-09-21T07:49:40.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.960+0000 7fce4cc00640 1 Processor -- start 2024-09-21T07:49:40.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.960+0000 7fce4cc00640 1 -- start start 2024-09-21T07:49:40.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.961+0000 7fce4cc00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fce2c003fd0 0x7fce48144a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:40.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.961+0000 7fce4cc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce481f5ac0 0x7fce480af080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:40.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.961+0000 7fce4cc00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce480b3140 0x7fce480b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:40.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.961+0000 7fce4cc00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fce4815da40 con 0x7fce481f5ac0 2024-09-21T07:49:40.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.961+0000 7fce4cc00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fce4815d8c0 con 0x7fce480b3140 2024-09-21T07:49:40.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.961+0000 7fce4cc00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fce4815dbc0 con 0x7fce2c003fd0 2024-09-21T07:49:40.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.961+0000 7fce3fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce481f5ac0 0x7fce480af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:40.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.961+0000 7fce3fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce481f5ac0 0x7fce480af080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41786/0 (socket says 172.21.15.39:41786) 2024-09-21T07:49:40.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.961+0000 7fce3fe00640 1 -- 172.21.15.39:0/2443041757 learned_addr learned my addr 172.21.15.39:0/2443041757 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:40.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.962+0000 7fce47400640 1 --2- 172.21.15.39:0/2443041757 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fce2c003fd0 0x7fce48144a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:40.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.962+0000 7fce47e00640 1 --2- 172.21.15.39:0/2443041757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce480b3140 0x7fce480b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:40.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.962+0000 7fce3fe00640 1 -- 172.21.15.39:0/2443041757 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fce2c003fd0 msgr2=0x7fce48144a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:40.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.962+0000 7fce3fe00640 1 --2- 172.21.15.39:0/2443041757 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fce2c003fd0 0x7fce48144a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:40.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.962+0000 7fce3fe00640 1 -- 172.21.15.39:0/2443041757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce480b3140 msgr2=0x7fce480b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:40.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.962+0000 7fce3fe00640 1 --2- 172.21.15.39:0/2443041757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce480b3140 0x7fce480b3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:40.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.962+0000 7fce3fe00640 1 -- 172.21.15.39:0/2443041757 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce480adfe0 con 0x7fce481f5ac0 2024-09-21T07:49:40.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.962+0000 7fce47400640 1 --2- 172.21.15.39:0/2443041757 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fce2c003fd0 0x7fce48144a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:49:40.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.962+0000 7fce47e00640 1 --2- 172.21.15.39:0/2443041757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce480b3140 0x7fce480b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:49:40.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.962+0000 7fce3fe00640 1 --2- 172.21.15.39:0/2443041757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce481f5ac0 0x7fce480af080 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7fce30009a70 tx=0x7fce30033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:40.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.962+0000 7fce3f400640 1 -- 172.21.15.39:0/2443041757 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce30038040 con 0x7fce481f5ac0 2024-09-21T07:49:40.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.962+0000 7fce3f400640 1 -- 172.21.15.39:0/2443041757 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce30041070 con 0x7fce481f5ac0 2024-09-21T07:49:40.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.962+0000 7fce4cc00640 1 -- 172.21.15.39:0/2443041757 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce480ae2c0 con 0x7fce481f5ac0 2024-09-21T07:49:40.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.963+0000 7fce3f400640 1 -- 172.21.15.39:0/2443041757 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce30033d70 con 0x7fce481f5ac0 2024-09-21T07:49:40.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.963+0000 7fce4cc00640 1 -- 172.21.15.39:0/2443041757 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce480ae880 con 0x7fce481f5ac0 2024-09-21T07:49:40.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.964+0000 7fce4cc00640 1 -- 172.21.15.39:0/2443041757 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce480cf710 con 0x7fce481f5ac0 2024-09-21T07:49:40.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.965+0000 7fce3f400640 1 -- 172.21.15.39:0/2443041757 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fce30046460 con 0x7fce481f5ac0 2024-09-21T07:49:40.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.966+0000 7fce3f400640 1 --2- 172.21.15.39:0/2443041757 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fce24078b90 0x7fce2407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:40.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.966+0000 7fce3f400640 1 -- 172.21.15.39:0/2443041757 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(233..233 src has 1..233) ==== 7016+0+0 (secure 0 0 0) 0x7fce300bc240 con 0x7fce481f5ac0 2024-09-21T07:49:40.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.967+0000 7fce47400640 1 --2- 172.21.15.39:0/2443041757 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fce24078b90 0x7fce2407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:40.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.968+0000 7fce47400640 1 --2- 172.21.15.39:0/2443041757 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fce24078b90 0x7fce2407b0a0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fce38005eb0 tx=0x7fce38005850 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:40.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:40.969+0000 7fce3f400640 1 -- 172.21.15.39:0/2443041757 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fce30084210 con 0x7fce481f5ac0 2024-09-21T07:49:41.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.202+0000 7fce4cc00640 1 -- 172.21.15.39:0/2443041757 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce48073190 con 0x7fce481f5ac0 2024-09-21T07:49:41.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.203+0000 7fce3f400640 1 -- 172.21.15.39:0/2443041757 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 244 v244) ==== 95+0+5245 (secure 0 0 0) 0x7fce30083b10 con 0x7fce481f5ac0 2024-09-21T07:49:41.205 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:49:41.205 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":244,"btime":"2024-09-21T07:49:38:150708+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8531,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1438851059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1438851059},{"type":"v1","addr":"172.21.15.148:6837","nonce":1438851059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":239},{"gid":9709,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3497801035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3497801035},{"type":"v1","addr":"172.21.15.148:6836","nonce":3497801035}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":239},{"gid":10632,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/4143807732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4143807732},{"type":"v1","addr":"172.21.15.39:6834","nonce":4143807732}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":238}],"filesystems":[{"mdsmap":{"epoch":244,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:49:37.657329+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10635},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10635":{"gid":10635,"name":"c","rank":0,"incarnation":240,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/2824493631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2824493631},{"type":"v1","addr":"172.21.15.39:6835","nonce":2824493631}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10635,"qdb_cluster":[10635]},"id":13}]} 2024-09-21T07:49:41.205 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 244 2024-09-21T07:49:41.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.205+0000 7fce4cc00640 1 -- 172.21.15.39:0/2443041757 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fce24078b90 msgr2=0x7fce2407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:41.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.205+0000 7fce4cc00640 1 --2- 172.21.15.39:0/2443041757 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fce24078b90 0x7fce2407b0a0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fce38005eb0 tx=0x7fce38005850 comp rx=0 tx=0).stop 2024-09-21T07:49:41.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.205+0000 7fce4cc00640 1 -- 172.21.15.39:0/2443041757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce481f5ac0 msgr2=0x7fce480af080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:41.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.205+0000 7fce4cc00640 1 --2- 172.21.15.39:0/2443041757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce481f5ac0 0x7fce480af080 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7fce30009a70 tx=0x7fce30033670 comp rx=0 tx=0).stop 2024-09-21T07:49:41.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.205+0000 7fce4cc00640 1 -- 172.21.15.39:0/2443041757 shutdown_connections 2024-09-21T07:49:41.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.205+0000 7fce4cc00640 1 --2- 172.21.15.39:0/2443041757 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fce24078b90 0x7fce2407b0a0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.205+0000 7fce4cc00640 1 --2- 172.21.15.39:0/2443041757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce480b3140 0x7fce480b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.205+0000 7fce4cc00640 1 --2- 172.21.15.39:0/2443041757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce481f5ac0 0x7fce480af080 unknown :-1 s=CLOSED pgs=2244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.205+0000 7fce4cc00640 1 --2- 172.21.15.39:0/2443041757 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fce2c003fd0 0x7fce48144a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.205+0000 7fce4cc00640 1 -- 172.21.15.39:0/2443041757 >> 172.21.15.39:0/2443041757 conn(0x7fce4806cfc0 msgr2=0x7fce4815ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:41.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.205+0000 7fce4cc00640 1 -- 172.21.15.39:0/2443041757 shutdown_connections 2024-09-21T07:49:41.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.205+0000 7fce4cc00640 1 -- 172.21.15.39:0/2443041757 wait complete. 2024-09-21T07:49:41.630 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 244, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:49:37.657329+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10635}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10635': {'gid': 10635, 'name': 'c', 'rank': 0, 'incarnation': 240, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.39:6835/2824493631', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 2824493631}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 2824493631}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10635, 'qdb_cluster': [10635]} 2024-09-21T07:49:41.630 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T07:49:41.630 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T07:49:41.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.858+0000 7fd986e00640 1 -- 172.21.15.39:0/3632138832 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9800cb8a0 msgr2=0x7fd980156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:41.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.858+0000 7fd986e00640 1 --2- 172.21.15.39:0/3632138832 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9800cb8a0 0x7fd980156bb0 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7fd9800cb3e0 tx=0x7fd96c02a4d0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.858+0000 7fd986e00640 1 -- 172.21.15.39:0/3632138832 shutdown_connections 2024-09-21T07:49:41.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.858+0000 7fd986e00640 1 --2- 172.21.15.39:0/3632138832 >> v2:172.21.15.148:3300/0 conn(0x7fd9800ccc60 0x7fd9800cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.858+0000 7fd986e00640 1 --2- 172.21.15.39:0/3632138832 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9800cb8a0 0x7fd980156bb0 unknown :-1 s=CLOSED pgs=2245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.858+0000 7fd986e00640 1 -- 172.21.15.39:0/3632138832 >> 172.21.15.39:0/3632138832 conn(0x7fd9800b2460 msgr2=0x7fd98006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:41.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.858+0000 7fd986e00640 1 -- 172.21.15.39:0/3632138832 shutdown_connections 2024-09-21T07:49:41.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.858+0000 7fd986e00640 1 -- 172.21.15.39:0/3632138832 wait complete. 2024-09-21T07:49:41.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.860+0000 7fd986e00640 1 Processor -- start 2024-09-21T07:49:41.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.861+0000 7fd986e00640 1 -- start start 2024-09-21T07:49:41.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.862+0000 7fd986e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9800b30a0 0x7fd9801f5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:41.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.862+0000 7fd986e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9800cb8a0 0x7fd9800c5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:41.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.862+0000 7fd986e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd9800ccc60 0x7fd9800c59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:41.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.862+0000 7fd986e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd9800af410 con 0x7fd9800b30a0 2024-09-21T07:49:41.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.862+0000 7fd986e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd9800af290 con 0x7fd9800cb8a0 2024-09-21T07:49:41.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.862+0000 7fd986e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd9800af590 con 0x7fd9800ccc60 2024-09-21T07:49:41.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.862+0000 7fd97fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9800b30a0 0x7fd9801f5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:41.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.862+0000 7fd97fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9800b30a0 0x7fd9801f5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41804/0 (socket says 172.21.15.39:41804) 2024-09-21T07:49:41.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.862+0000 7fd97fe00640 1 -- 172.21.15.39:0/3353646625 learned_addr learned my addr 172.21.15.39:0/3353646625 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:41.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.862+0000 7fd97f400640 1 --2- 172.21.15.39:0/3353646625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9800cb8a0 0x7fd9800c5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:41.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.863+0000 7fd984e00640 1 --2- 172.21.15.39:0/3353646625 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd9800ccc60 0x7fd9800c59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:41.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.863+0000 7fd97fe00640 1 -- 172.21.15.39:0/3353646625 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd9800ccc60 msgr2=0x7fd9800c59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:41.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.863+0000 7fd97fe00640 1 --2- 172.21.15.39:0/3353646625 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd9800ccc60 0x7fd9800c59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.863+0000 7fd97fe00640 1 -- 172.21.15.39:0/3353646625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9800cb8a0 msgr2=0x7fd9800c5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:41.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.863+0000 7fd97fe00640 1 --2- 172.21.15.39:0/3353646625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9800cb8a0 0x7fd9800c5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.863+0000 7fd97fe00640 1 -- 172.21.15.39:0/3353646625 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9801f5cb0 con 0x7fd9800b30a0 2024-09-21T07:49:41.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.863+0000 7fd97f400640 1 --2- 172.21.15.39:0/3353646625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9800cb8a0 0x7fd9800c5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:41.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.863+0000 7fd97fe00640 1 --2- 172.21.15.39:0/3353646625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9800b30a0 0x7fd9801f5770 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7fd97000b410 tx=0x7fd97000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:41.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.863+0000 7fd977e00640 1 -- 172.21.15.39:0/3353646625 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd970015020 con 0x7fd9800b30a0 2024-09-21T07:49:41.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.863+0000 7fd977e00640 1 -- 172.21.15.39:0/3353646625 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd970011920 con 0x7fd9800b30a0 2024-09-21T07:49:41.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.863+0000 7fd977e00640 1 -- 172.21.15.39:0/3353646625 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd970011d70 con 0x7fd9800b30a0 2024-09-21T07:49:41.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.863+0000 7fd986e00640 1 -- 172.21.15.39:0/3353646625 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9801f9f60 con 0x7fd9800b30a0 2024-09-21T07:49:41.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.864+0000 7fd986e00640 1 -- 172.21.15.39:0/3353646625 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9800cdf80 con 0x7fd9800b30a0 2024-09-21T07:49:41.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.866+0000 7fd977e00640 1 -- 172.21.15.39:0/3353646625 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd970018020 con 0x7fd9800b30a0 2024-09-21T07:49:41.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.867+0000 7fd977e00640 1 --2- 172.21.15.39:0/3353646625 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd960078bc0 0x7fd96007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:41.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.868+0000 7fd977e00640 1 -- 172.21.15.39:0/3353646625 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(233..233 src has 1..233) ==== 7016+0+0 (secure 0 0 0) 0x7fd97009b260 con 0x7fd9800b30a0 2024-09-21T07:49:41.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.868+0000 7fd97f400640 1 --2- 172.21.15.39:0/3353646625 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd960078bc0 0x7fd96007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:41.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.868+0000 7fd98e01c740 1 Processor -- start 2024-09-21T07:49:41.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.868+0000 7fd98e01c740 1 -- start start 2024-09-21T07:49:41.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.869+0000 7fd97f400640 1 --2- 172.21.15.39:0/3353646625 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd960078bc0 0x7fd96007b0d0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fd96c033000 tx=0x7fd96c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:41.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.870+0000 7fd98e01c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d4928b9270 0x55d492c89970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:41.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.870+0000 7fd98e01c740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d4928903b0 0x55d492c89eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:41.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.870+0000 7fd98e01c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d492898590 0x55d492c59f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:41.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.870+0000 7fd98e01c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d492be3430 con 0x55d492898590 2024-09-21T07:49:41.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.870+0000 7fd98e01c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d492bd35d0 con 0x55d4928b9270 2024-09-21T07:49:41.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.870+0000 7fd98e01c740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55d492bc2a00 con 0x55d4928903b0 2024-09-21T07:49:41.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.870+0000 7fd984e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d492898590 0x55d492c59f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:41.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.870+0000 7fd984e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d492898590 0x55d492c59f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41818/0 (socket says 172.21.15.39:41818) 2024-09-21T07:49:41.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.870+0000 7fd984e00640 1 -- 172.21.15.39:0/718924866 learned_addr learned my addr 172.21.15.39:0/718924866 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:41.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.870+0000 7fd97f400640 1 --2- 172.21.15.39:0/718924866 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d4928903b0 0x55d492c89eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:41.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.870+0000 7fd97fe00640 1 --2- 172.21.15.39:0/718924866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d4928b9270 0x55d492c89970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:41.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.870+0000 7fd984e00640 1 -- 172.21.15.39:0/718924866 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d4928903b0 msgr2=0x55d492c89eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:41.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.870+0000 7fd984e00640 1 --2- 172.21.15.39:0/718924866 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d4928903b0 0x55d492c89eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.870+0000 7fd984e00640 1 -- 172.21.15.39:0/718924866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d4928b9270 msgr2=0x55d492c89970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:41.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.870+0000 7fd984e00640 1 --2- 172.21.15.39:0/718924866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d4928b9270 0x55d492c89970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.870+0000 7fd984e00640 1 -- 172.21.15.39:0/718924866 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d492b8b520 con 0x55d492898590 2024-09-21T07:49:41.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.871+0000 7fd984e00640 1 --2- 172.21.15.39:0/718924866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d492898590 0x55d492c59f60 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7fd978009ef0 tx=0x7fd97800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4f3f061c288bb32 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:41.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.871+0000 7fd974c00640 1 -- 172.21.15.39:0/718924866 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97801a070 con 0x55d492898590 2024-09-21T07:49:41.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.871+0000 7fd974c00640 1 -- 172.21.15.39:0/718924866 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd978015410 con 0x55d492898590 2024-09-21T07:49:41.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.871+0000 7fd974c00640 1 -- 172.21.15.39:0/718924866 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9780157c0 con 0x55d492898590 2024-09-21T07:49:41.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.872+0000 7fd98e01c740 1 -- 172.21.15.39:0/718924866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d492898590 msgr2=0x55d492c59f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:41.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.872+0000 7fd98e01c740 1 --2- 172.21.15.39:0/718924866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d492898590 0x55d492c59f60 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7fd978009ef0 tx=0x7fd97800c7f0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.872+0000 7fd98e01c740 1 -- 172.21.15.39:0/718924866 shutdown_connections 2024-09-21T07:49:41.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.872+0000 7fd98e01c740 1 --2- 172.21.15.39:0/718924866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d492898590 0x55d492c59f60 unknown :-1 s=CLOSED pgs=2247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.872+0000 7fd98e01c740 1 --2- 172.21.15.39:0/718924866 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d4928903b0 0x55d492c89eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.872+0000 7fd98e01c740 1 --2- 172.21.15.39:0/718924866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d4928b9270 0x55d492c89970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.872+0000 7fd98e01c740 1 -- 172.21.15.39:0/718924866 >> 172.21.15.39:0/718924866 conn(0x55d4928ad3c0 msgr2=0x55d492c5f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:41.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.872+0000 7fd98e01c740 1 -- 172.21.15.39:0/718924866 shutdown_connections 2024-09-21T07:49:41.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.872+0000 7fd98e01c740 1 -- 172.21.15.39:0/718924866 wait complete. 2024-09-21T07:49:41.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.873+0000 7fd98e01c740 1 -- start start 2024-09-21T07:49:41.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.873+0000 7fd98e01c740 1 Processor -- start 2024-09-21T07:49:41.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.875+0000 7fd98e01c740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d4928903b0 0x55d492cb6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:41.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.875+0000 7fd98e01c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d492898590 0x55d492c9e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:41.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.875+0000 7fd98e01c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d4928b9270 0x55d492c9ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:41.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.875+0000 7fd98e01c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d492b69330 con 0x55d492898590 2024-09-21T07:49:41.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.875+0000 7fd98e01c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d492bc6810 con 0x55d4928b9270 2024-09-21T07:49:41.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.875+0000 7fd98e01c740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55d492933620 con 0x55d4928903b0 2024-09-21T07:49:41.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.875+0000 7fd98e01c740 10 client.0 _ll_register_callbacks cb 0x55d492b6e420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:49:41.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.875+0000 7fd97f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d492898590 0x55d492c9e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:41.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.875+0000 7fd97f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d492898590 0x55d492c9e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41822/0 (socket says 172.21.15.39:41822) 2024-09-21T07:49:41.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.875+0000 7fd97f400640 1 -- 172.21.15.39:0/2689392070 learned_addr learned my addr 172.21.15.39:0/2689392070 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:41.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.875+0000 7fd984e00640 1 --2- 172.21.15.39:0/2689392070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d4928b9270 0x55d492c9ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:41.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.875+0000 7fd97fe00640 1 --2- 172.21.15.39:0/2689392070 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d4928903b0 0x55d492cb6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:41.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.875+0000 7fd97f400640 1 -- 172.21.15.39:0/2689392070 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d4928903b0 msgr2=0x55d492cb6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:41.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.875+0000 7fd97f400640 1 --2- 172.21.15.39:0/2689392070 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d4928903b0 0x55d492cb6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.875+0000 7fd97f400640 1 -- 172.21.15.39:0/2689392070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d4928b9270 msgr2=0x55d492c9ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:41.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.875+0000 7fd97f400640 1 --2- 172.21.15.39:0/2689392070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d4928b9270 0x55d492c9ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.875+0000 7fd97f400640 1 -- 172.21.15.39:0/2689392070 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d492924850 con 0x55d492898590 2024-09-21T07:49:41.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.875+0000 7fd97fe00640 1 --2- 172.21.15.39:0/2689392070 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d4928903b0 0x55d492cb6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:41.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.876+0000 7fd97f400640 1 --2- 172.21.15.39:0/2689392070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d492898590 0x55d492c9e680 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0099c0 tx=0x7fd96c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c15ba2dd48d781b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:41.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.876+0000 7fd969600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:49:41.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.876+0000 7fd969600640 1 -- 172.21.15.39:0/2689392070 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96c04d020 con 0x55d492898590 2024-09-21T07:49:41.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.876+0000 7fd969600640 1 -- 172.21.15.39:0/2689392070 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(233..233 src has 1..233) ==== 7016+0+0 (secure 0 0 0) 0x7fd96c0049a0 con 0x55d492898590 2024-09-21T07:49:41.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.876+0000 7fd98e01c740 1 -- 172.21.15.39:0/2689392070 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd978015410 con 0x55d492898590 2024-09-21T07:49:41.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.876+0000 7fd969600640 1 -- 172.21.15.39:0/2689392070 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=244) ==== 24+0+0 (secure 0 0 0) 0x7fd96c057050 con 0x55d492898590 2024-09-21T07:49:41.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.876+0000 7fd98e01c740 10 client.10761 fetch_fsmap learned FSMap version 244 2024-09-21T07:49:41.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.876+0000 7fd98e01c740 1 -- 172.21.15.39:0/2689392070 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=244,monmap=2+}) -- 0x55d49299b6b0 con 0x55d492898590 2024-09-21T07:49:41.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.877+0000 7fd969600640 1 -- 172.21.15.39:0/2689392070 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd978015410 con 0x55d492898590 2024-09-21T07:49:41.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.877+0000 7fd969600640 1 -- 172.21.15.39:0/2689392070 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 244) ==== 3502+0+0 (secure 0 0 0) 0x7fd96c05a020 con 0x55d492898590 2024-09-21T07:49:41.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.877+0000 7fd98e01c740 10 client.10761 fetch_fsmap finished waiting for FSMap version 244 2024-09-21T07:49:41.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.877+0000 7fd98e01c740 10 client.10761 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:49:41.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.877+0000 7fd98e01c740 20 client.10761 populate_metadata read hostname 'smithi039' 2024-09-21T07:49:41.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.877+0000 7fd98e01c740 1 --2- 172.21.15.39:0/2689392070 >> [v2:172.21.15.39:6833/2824493631,v1:172.21.15.39:6835/2824493631] conn(0x55d492877e40 0x55d492c853b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:41.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.877+0000 7fd98e01c740 4 client.10761 mds_command: new command op to 10635 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:49:41.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.877+0000 7fd98e01c740 1 -- 172.21.15.39:0/2689392070 --> [v2:172.21.15.39:6833/2824493631,v1:172.21.15.39:6835/2824493631] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d492bd5520 con 0x55d492877e40 2024-09-21T07:49:41.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.877+0000 7fd97fe00640 1 --2- 172.21.15.39:0/2689392070 >> [v2:172.21.15.39:6833/2824493631,v1:172.21.15.39:6835/2824493631] conn(0x55d492877e40 0x55d492c853b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:41.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.878+0000 7fd97fe00640 1 --2- 172.21.15.39:0/2689392070 >> [v2:172.21.15.39:6833/2824493631,v1:172.21.15.39:6835/2824493631] conn(0x55d492877e40 0x55d492c853b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a126a808dd2b9e72 server_cookie=48ede54c511b7d5f in_seq=0 out_seq=0 2024-09-21T07:49:41.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.878+0000 7fd969600640 10 client.10761 ms_handle_connect on v2:172.21.15.39:6833/2824493631 2024-09-21T07:49:41.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.879+0000 7fd969600640 1 -- 172.21.15.39:0/2689392070 <== mds.0 v2:172.21.15.39:6833/2824493631 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d492bd5520 con 0x55d492877e40 2024-09-21T07:49:41.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.879+0000 7fd969600640 10 client.10761 handle_command_reply: tid=0 2024-09-21T07:49:41.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.879+0000 7fd98e01c740 1 client.10761 shutdown 2024-09-21T07:49:41.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.879+0000 7fd98e01c740 1 -- 172.21.15.39:0/2689392070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d492898590 msgr2=0x55d492c9e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:41.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.879+0000 7fd98e01c740 1 --2- 172.21.15.39:0/2689392070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d492898590 0x55d492c9e680 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0099c0 tx=0x7fd96c009260 comp rx=0 tx=0).stop 2024-09-21T07:49:41.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.879+0000 7fd98e01c740 1 -- 172.21.15.39:0/2689392070 shutdown_connections 2024-09-21T07:49:41.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.879+0000 7fd98e01c740 1 --2- 172.21.15.39:0/2689392070 >> [v2:172.21.15.39:6833/2824493631,v1:172.21.15.39:6835/2824493631] conn(0x55d492877e40 0x55d492c853b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.879+0000 7fd98e01c740 1 --2- 172.21.15.39:0/2689392070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d4928b9270 0x55d492c9ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.879+0000 7fd98e01c740 1 --2- 172.21.15.39:0/2689392070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d492898590 0x55d492c9e680 unknown :-1 s=CLOSED pgs=2248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.879+0000 7fd98e01c740 1 --2- 172.21.15.39:0/2689392070 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d4928903b0 0x55d492cb6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.879+0000 7fd969600640 0 client.10761 ms_handle_reset on v2:172.21.15.39:6833/2824493631 2024-09-21T07:49:41.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.879+0000 7fd98e01c740 1 -- 172.21.15.39:0/2689392070 >> 172.21.15.39:0/2689392070 conn(0x55d4928ad3c0 msgr2=0x55d492c5b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:41.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.879+0000 7fd98e01c740 1 -- 172.21.15.39:0/2689392070 shutdown_connections 2024-09-21T07:49:41.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.879+0000 7fd98e01c740 1 -- 172.21.15.39:0/2689392070 wait complete. 2024-09-21T07:49:41.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.879+0000 7fd98e01c740 20 client.10761 trim_cache size 0 max 16384 2024-09-21T07:49:41.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.899+0000 7fd98e01c740 1 Processor -- start 2024-09-21T07:49:41.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.899+0000 7fd98e01c740 1 -- start start 2024-09-21T07:49:41.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.900+0000 7fd98e01c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4928903b0 0x55d492cc79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:41.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.900+0000 7fd98e01c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d492898590 0x55d492c887d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:41.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.900+0000 7fd98e01c740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d4928b9270 0x55d492c88d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:41.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.900+0000 7fd98e01c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd980203ed0 con 0x55d4928903b0 2024-09-21T07:49:41.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.900+0000 7fd98e01c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd9802032e0 con 0x55d492898590 2024-09-21T07:49:41.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.900+0000 7fd98e01c740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd980203880 con 0x55d4928b9270 2024-09-21T07:49:41.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.900+0000 7fd97fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4928903b0 0x55d492cc79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:41.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.901+0000 7fd97fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4928903b0 0x55d492cc79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41830/0 (socket says 172.21.15.39:41830) 2024-09-21T07:49:41.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.901+0000 7fd97fe00640 1 -- 172.21.15.39:0/3159876597 learned_addr learned my addr 172.21.15.39:0/3159876597 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:41.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.901+0000 7fd97f400640 1 --2- 172.21.15.39:0/3159876597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d492898590 0x55d492c887d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:41.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.901+0000 7fd984e00640 1 --2- 172.21.15.39:0/3159876597 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d4928b9270 0x55d492c88d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:41.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.901+0000 7fd97fe00640 1 -- 172.21.15.39:0/3159876597 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d4928b9270 msgr2=0x55d492c88d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:41.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.901+0000 7fd97fe00640 1 --2- 172.21.15.39:0/3159876597 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d4928b9270 0x55d492c88d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.901+0000 7fd97fe00640 1 -- 172.21.15.39:0/3159876597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d492898590 msgr2=0x55d492c887d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:41.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.901+0000 7fd97fe00640 1 --2- 172.21.15.39:0/3159876597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d492898590 0x55d492c887d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.901+0000 7fd97fe00640 1 -- 172.21.15.39:0/3159876597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d492c60650 con 0x55d4928903b0 2024-09-21T07:49:41.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.901+0000 7fd984e00640 1 --2- 172.21.15.39:0/3159876597 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d4928b9270 0x55d492c88d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:41.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.901+0000 7fd97fe00640 1 --2- 172.21.15.39:0/3159876597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4928903b0 0x55d492cc79c0 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7fd97000bdd0 tx=0x7fd97001c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e204c7fdc7fe6c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:41.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.901+0000 7fd96be00640 1 -- 172.21.15.39:0/3159876597 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9700674c0 con 0x55d4928903b0 2024-09-21T07:49:41.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.901+0000 7fd96be00640 1 -- 172.21.15.39:0/3159876597 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd97006c400 con 0x55d4928903b0 2024-09-21T07:49:41.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.901+0000 7fd96be00640 1 -- 172.21.15.39:0/3159876597 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97005ec50 con 0x55d4928903b0 2024-09-21T07:49:41.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.902+0000 7fd98e01c740 1 -- 172.21.15.39:0/3159876597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4928903b0 msgr2=0x55d492cc79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:41.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.902+0000 7fd98e01c740 1 --2- 172.21.15.39:0/3159876597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4928903b0 0x55d492cc79c0 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7fd97000bdd0 tx=0x7fd97001c040 comp rx=0 tx=0).stop 2024-09-21T07:49:41.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.902+0000 7fd98e01c740 1 -- 172.21.15.39:0/3159876597 shutdown_connections 2024-09-21T07:49:41.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.902+0000 7fd98e01c740 1 --2- 172.21.15.39:0/3159876597 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d4928b9270 0x55d492c88d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.902+0000 7fd98e01c740 1 --2- 172.21.15.39:0/3159876597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d492898590 0x55d492c887d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.902+0000 7fd98e01c740 1 --2- 172.21.15.39:0/3159876597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4928903b0 0x55d492cc79c0 unknown :-1 s=CLOSED pgs=2249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.902+0000 7fd98e01c740 1 -- 172.21.15.39:0/3159876597 >> 172.21.15.39:0/3159876597 conn(0x55d4928ad3c0 msgr2=0x55d492c5f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:41.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.902+0000 7fd98e01c740 1 -- 172.21.15.39:0/3159876597 shutdown_connections 2024-09-21T07:49:41.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.902+0000 7fd98e01c740 1 -- 172.21.15.39:0/3159876597 wait complete. 2024-09-21T07:49:41.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.903+0000 7fd98e01c740 1 -- start start 2024-09-21T07:49:41.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.903+0000 7fd98e01c740 1 Processor -- start 2024-09-21T07:49:41.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.905+0000 7fd98e01c740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d4928903b0 0x55d492d217f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:41.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.905+0000 7fd98e01c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d492898590 0x55d492c7ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:41.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.905+0000 7fd98e01c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4928b9270 0x55d492c7ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:41.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.905+0000 7fd98e01c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d492c82b60 con 0x55d4928b9270 2024-09-21T07:49:41.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.905+0000 7fd98e01c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d492c84c80 con 0x55d492898590 2024-09-21T07:49:41.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.905+0000 7fd98e01c740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55d492c831b0 con 0x55d4928903b0 2024-09-21T07:49:41.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.905+0000 7fd98e01c740 10 client.0 _ll_register_callbacks cb 0x7fd96c0042a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:49:41.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.905+0000 7fd984e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4928b9270 0x55d492c7ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:41.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.905+0000 7fd984e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4928b9270 0x55d492c7ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41840/0 (socket says 172.21.15.39:41840) 2024-09-21T07:49:41.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.905+0000 7fd984e00640 1 -- 172.21.15.39:0/3996620874 learned_addr learned my addr 172.21.15.39:0/3996620874 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:41.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.905+0000 7fd97f400640 1 --2- 172.21.15.39:0/3996620874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d492898590 0x55d492c7ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:41.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.905+0000 7fd97fe00640 1 --2- 172.21.15.39:0/3996620874 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d4928903b0 0x55d492d217f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:41.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.906+0000 7fd984e00640 1 -- 172.21.15.39:0/3996620874 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d4928903b0 msgr2=0x55d492d217f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:41.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.906+0000 7fd984e00640 1 --2- 172.21.15.39:0/3996620874 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d4928903b0 0x55d492d217f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.906+0000 7fd984e00640 1 -- 172.21.15.39:0/3996620874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d492898590 msgr2=0x55d492c7ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:41.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.906+0000 7fd984e00640 1 --2- 172.21.15.39:0/3996620874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d492898590 0x55d492c7ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.906+0000 7fd984e00640 1 -- 172.21.15.39:0/3996620874 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d492cb7310 con 0x55d4928b9270 2024-09-21T07:49:41.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.906+0000 7fd984e00640 1 --2- 172.21.15.39:0/3996620874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4928b9270 0x55d492c7ef60 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7fd978014000 tx=0x7fd978015810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da6abf95c2e893e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:41.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.906+0000 7fd969600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:49:41.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.906+0000 7fd969600640 1 -- 172.21.15.39:0/3996620874 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97801a040 con 0x55d4928b9270 2024-09-21T07:49:41.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.906+0000 7fd969600640 1 -- 172.21.15.39:0/3996620874 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(233..233 src has 1..233) ==== 7016+0+0 (secure 0 0 0) 0x7fd978009390 con 0x55d4928b9270 2024-09-21T07:49:41.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.906+0000 7fd98e01c740 1 -- 172.21.15.39:0/3996620874 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd97006c400 con 0x55d4928b9270 2024-09-21T07:49:41.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.907+0000 7fd969600640 1 -- 172.21.15.39:0/3996620874 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=244) ==== 24+0+0 (secure 0 0 0) 0x7fd978015980 con 0x55d4928b9270 2024-09-21T07:49:41.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.907+0000 7fd98e01c740 10 client.10767 fetch_fsmap learned FSMap version 244 2024-09-21T07:49:41.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.907+0000 7fd98e01c740 1 -- 172.21.15.39:0/3996620874 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=244,monmap=2+}) -- 0x55d492c5fdd0 con 0x55d4928b9270 2024-09-21T07:49:41.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.907+0000 7fd969600640 1 -- 172.21.15.39:0/3996620874 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd97006c400 con 0x55d4928b9270 2024-09-21T07:49:41.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.907+0000 7fd969600640 1 -- 172.21.15.39:0/3996620874 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 244) ==== 3502+0+0 (secure 0 0 0) 0x7fd978015b20 con 0x55d4928b9270 2024-09-21T07:49:41.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.907+0000 7fd98e01c740 10 client.10767 fetch_fsmap finished waiting for FSMap version 244 2024-09-21T07:49:41.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.907+0000 7fd98e01c740 10 client.10767 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:49:41.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.907+0000 7fd98e01c740 20 client.10767 populate_metadata read hostname 'smithi039' 2024-09-21T07:49:41.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.907+0000 7fd98e01c740 1 --2- 172.21.15.39:0/3996620874 >> [v2:172.21.15.39:6833/2824493631,v1:172.21.15.39:6835/2824493631] conn(0x55d492d28230 0x55d492cb57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:41.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.907+0000 7fd98e01c740 4 client.10767 mds_command: new command op to 10635 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:49:41.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.907+0000 7fd98e01c740 1 -- 172.21.15.39:0/3996620874 --> [v2:172.21.15.39:6833/2824493631,v1:172.21.15.39:6835/2824493631] -- command(tid 0: {"prefix": "status"}) -- 0x55d492bd0fd0 con 0x55d492d28230 2024-09-21T07:49:41.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.907+0000 7fd97fe00640 1 --2- 172.21.15.39:0/3996620874 >> [v2:172.21.15.39:6833/2824493631,v1:172.21.15.39:6835/2824493631] conn(0x55d492d28230 0x55d492cb57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:41.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.908+0000 7fd97fe00640 1 --2- 172.21.15.39:0/3996620874 >> [v2:172.21.15.39:6833/2824493631,v1:172.21.15.39:6835/2824493631] conn(0x55d492d28230 0x55d492cb57c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72bb4f3883b08dc3 server_cookie=c994d9ab706d0d56 in_seq=0 out_seq=0 2024-09-21T07:49:41.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.908+0000 7fd969600640 10 client.10767 ms_handle_connect on v2:172.21.15.39:6833/2824493631 2024-09-21T07:49:41.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.908+0000 7fd969600640 1 -- 172.21.15.39:0/3996620874 <== mds.0 v2:172.21.15.39:6833/2824493631 1 ==== command_reply(tid 0: 0 ) ==== 8+0+335 (crc 0 0 0) 0x55d492bd0fd0 con 0x55d492d28230 2024-09-21T07:49:41.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.908+0000 7fd969600640 10 client.10767 handle_command_reply: tid=0 2024-09-21T07:49:41.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.908+0000 7fd98e01c740 1 client.10767 shutdown 2024-09-21T07:49:41.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.908+0000 7fd98e01c740 1 -- 172.21.15.39:0/3996620874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4928b9270 msgr2=0x55d492c7ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:41.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.908+0000 7fd98e01c740 1 --2- 172.21.15.39:0/3996620874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4928b9270 0x55d492c7ef60 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7fd978014000 tx=0x7fd978015810 comp rx=0 tx=0).stop 2024-09-21T07:49:41.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.908+0000 7fd98e01c740 1 -- 172.21.15.39:0/3996620874 shutdown_connections 2024-09-21T07:49:41.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.908+0000 7fd98e01c740 1 --2- 172.21.15.39:0/3996620874 >> [v2:172.21.15.39:6833/2824493631,v1:172.21.15.39:6835/2824493631] conn(0x55d492d28230 0x55d492cb57c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.908+0000 7fd98e01c740 1 --2- 172.21.15.39:0/3996620874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4928b9270 0x55d492c7ef60 unknown :-1 s=CLOSED pgs=2250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.908+0000 7fd98e01c740 1 --2- 172.21.15.39:0/3996620874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d492898590 0x55d492c7ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.908+0000 7fd98e01c740 1 --2- 172.21.15.39:0/3996620874 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d4928903b0 0x55d492d217f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.908+0000 7fd969600640 0 client.10767 ms_handle_reset on v2:172.21.15.39:6833/2824493631 2024-09-21T07:49:41.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.908+0000 7fd98e01c740 1 -- 172.21.15.39:0/3996620874 >> 172.21.15.39:0/3996620874 conn(0x55d4928ad3c0 msgr2=0x55d492c55560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:41.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.909+0000 7fd98e01c740 1 -- 172.21.15.39:0/3996620874 shutdown_connections 2024-09-21T07:49:41.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.909+0000 7fd98e01c740 1 -- 172.21.15.39:0/3996620874 wait complete. 2024-09-21T07:49:41.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.909+0000 7fd98e01c740 20 client.10767 trim_cache size 0 max 16384 2024-09-21T07:49:41.914 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:49:41.914 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:49:41.914 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-09-21T07:49:41.914 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 10635, 2024-09-21T07:49:41.914 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:49:41.914 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:49:41.914 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:49:41.914 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 6.7958814370000002, 2024-09-21T07:49:41.914 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 244, 2024-09-21T07:49:41.915 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 233, 2024-09-21T07:49:41.915 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 233, 2024-09-21T07:49:41.915 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 11.798644721000001 2024-09-21T07:49:41.915 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:49:41.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.909+0000 7fd986e00640 1 -- 172.21.15.39:0/3353646625 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd960078bc0 msgr2=0x7fd96007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:41.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.909+0000 7fd986e00640 1 --2- 172.21.15.39:0/3353646625 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd960078bc0 0x7fd96007b0d0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fd96c033000 tx=0x7fd96c004730 comp rx=0 tx=0).stop 2024-09-21T07:49:41.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.909+0000 7fd986e00640 1 -- 172.21.15.39:0/3353646625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9800b30a0 msgr2=0x7fd9801f5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:41.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.909+0000 7fd986e00640 1 --2- 172.21.15.39:0/3353646625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9800b30a0 0x7fd9801f5770 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7fd97000b410 tx=0x7fd97000b8d0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.909+0000 7fd986e00640 1 -- 172.21.15.39:0/3353646625 shutdown_connections 2024-09-21T07:49:41.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.909+0000 7fd986e00640 1 --2- 172.21.15.39:0/3353646625 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd960078bc0 0x7fd96007b0d0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.909+0000 7fd986e00640 1 --2- 172.21.15.39:0/3353646625 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd9800ccc60 0x7fd9800c59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.909+0000 7fd986e00640 1 --2- 172.21.15.39:0/3353646625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd9800cb8a0 0x7fd9800c5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.909+0000 7fd986e00640 1 --2- 172.21.15.39:0/3353646625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9800b30a0 0x7fd9801f5770 unknown :-1 s=CLOSED pgs=2246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.909+0000 7fd986e00640 1 -- 172.21.15.39:0/3353646625 >> 172.21.15.39:0/3353646625 conn(0x7fd9800b2460 msgr2=0x7fd9800b34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:41.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.910+0000 7fd986e00640 1 -- 172.21.15.39:0/3353646625 shutdown_connections 2024-09-21T07:49:41.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:41.910+0000 7fd986e00640 1 -- 172.21.15.39:0/3353646625 wait complete. 2024-09-21T07:49:41.936 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:49:42.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.161+0000 7f9e32200640 1 -- 172.21.15.39:0/4159244363 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e2c0d0630 msgr2=0x7f9e2c0d0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:42.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.161+0000 7f9e32200640 1 --2- 172.21.15.39:0/4159244363 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e2c0d0630 0x7f9e2c0d0a40 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f9e24005fd0 tx=0x7f9e2402aa90 comp rx=0 tx=0).stop 2024-09-21T07:49:42.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.161+0000 7f9e32200640 1 -- 172.21.15.39:0/4159244363 shutdown_connections 2024-09-21T07:49:42.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.161+0000 7f9e32200640 1 --2- 172.21.15.39:0/4159244363 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e2c0d0630 0x7f9e2c0d0a40 unknown :-1 s=CLOSED pgs=2251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.161+0000 7f9e32200640 1 --2- 172.21.15.39:0/4159244363 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9e2c073dd0 0x7f9e2c15d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.161+0000 7f9e32200640 1 -- 172.21.15.39:0/4159244363 >> 172.21.15.39:0/4159244363 conn(0x7f9e2c06cfc0 msgr2=0x7f9e2c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:42.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.161+0000 7f9e32200640 1 -- 172.21.15.39:0/4159244363 shutdown_connections 2024-09-21T07:49:42.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.161+0000 7f9e32200640 1 -- 172.21.15.39:0/4159244363 wait complete. 2024-09-21T07:49:42.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.164+0000 7f9e32200640 1 Processor -- start 2024-09-21T07:49:42.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.164+0000 7f9e32200640 1 -- start start 2024-09-21T07:49:42.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.165+0000 7f9e32200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9e2c073dd0 0x7f9e2c1f57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:42.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.165+0000 7f9e32200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e2c0d0630 0x7f9e2c0b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:42.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.165+0000 7f9e32200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e2c15daf0 0x7f9e2c152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:42.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.165+0000 7f9e32200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2c0cd1a0 con 0x7f9e2c0d0630 2024-09-21T07:49:42.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.165+0000 7f9e32200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2c0cd020 con 0x7f9e2c15daf0 2024-09-21T07:49:42.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.165+0000 7f9e32200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f9e2c0cd320 con 0x7f9e2c073dd0 2024-09-21T07:49:42.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.165+0000 7f9e22a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e2c0d0630 0x7f9e2c0b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:42.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.165+0000 7f9e22a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e2c0d0630 0x7f9e2c0b3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43622/0 (socket says 172.21.15.39:43622) 2024-09-21T07:49:42.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.165+0000 7f9e22a00640 1 -- 172.21.15.39:0/4086243877 learned_addr learned my addr 172.21.15.39:0/4086243877 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:42.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.166+0000 7f9e2b400640 1 --2- 172.21.15.39:0/4086243877 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9e2c073dd0 0x7f9e2c1f57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:42.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.166+0000 7f9e2be00640 1 --2- 172.21.15.39:0/4086243877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e2c15daf0 0x7f9e2c152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:42.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.166+0000 7f9e22a00640 1 -- 172.21.15.39:0/4086243877 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9e2c073dd0 msgr2=0x7f9e2c1f57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:42.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.166+0000 7f9e22a00640 1 --2- 172.21.15.39:0/4086243877 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9e2c073dd0 0x7f9e2c1f57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.166+0000 7f9e22a00640 1 -- 172.21.15.39:0/4086243877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e2c15daf0 msgr2=0x7f9e2c152880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:42.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.166+0000 7f9e22a00640 1 --2- 172.21.15.39:0/4086243877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e2c15daf0 0x7f9e2c152880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.166+0000 7f9e22a00640 1 -- 172.21.15.39:0/4086243877 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e2c1f5d10 con 0x7f9e2c0d0630 2024-09-21T07:49:42.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.166+0000 7f9e22a00640 1 --2- 172.21.15.39:0/4086243877 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e2c0d0630 0x7f9e2c0b3e60 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f9e1400c390 tx=0x7f9e1400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:42.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.166+0000 7f9e23e00640 1 -- 172.21.15.39:0/4086243877 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e14017070 con 0x7f9e2c0d0630 2024-09-21T07:49:42.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.166+0000 7f9e23e00640 1 -- 172.21.15.39:0/4086243877 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e140053e0 con 0x7f9e2c0d0630 2024-09-21T07:49:42.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.167+0000 7f9e23e00640 1 -- 172.21.15.39:0/4086243877 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e14003850 con 0x7f9e2c0d0630 2024-09-21T07:49:42.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.167+0000 7f9e32200640 1 -- 172.21.15.39:0/4086243877 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e2c0b43a0 con 0x7f9e2c0d0630 2024-09-21T07:49:42.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.167+0000 7f9e32200640 1 -- 172.21.15.39:0/4086243877 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e2c1f61e0 con 0x7f9e2c0d0630 2024-09-21T07:49:42.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.168+0000 7f9e32200640 1 -- 172.21.15.39:0/4086243877 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e2c1f5f50 con 0x7f9e2c0d0630 2024-09-21T07:49:42.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.169+0000 7f9e23e00640 1 -- 172.21.15.39:0/4086243877 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9e1401a070 con 0x7f9e2c0d0630 2024-09-21T07:49:42.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.170+0000 7f9e23e00640 1 --2- 172.21.15.39:0/4086243877 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9e08078b90 0x7f9e0807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:42.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.170+0000 7f9e23e00640 1 -- 172.21.15.39:0/4086243877 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(233..233 src has 1..233) ==== 7016+0+0 (secure 0 0 0) 0x7f9e1409a6a0 con 0x7f9e2c0d0630 2024-09-21T07:49:42.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.171+0000 7f9e2b400640 1 --2- 172.21.15.39:0/4086243877 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9e08078b90 0x7f9e0807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:42.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.173+0000 7f9e2b400640 1 --2- 172.21.15.39:0/4086243877 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9e08078b90 0x7f9e0807b0a0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c0730e0 tx=0x7f9e1c005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:42.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.175+0000 7f9e23e00640 1 -- 172.21.15.39:0/4086243877 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9e2c0d0f80 con 0x7f9e2c0d0630 2024-09-21T07:49:42.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.409+0000 7f9e32200640 1 -- 172.21.15.39:0/4086243877 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9e2c0cd840 con 0x7f9e2c0d0630 2024-09-21T07:49:42.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.410+0000 7f9e23e00640 1 -- 172.21.15.39:0/4086243877 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v244) ==== 53+0+73 (secure 0 0 0) 0x7f9e14061ed0 con 0x7f9e2c0d0630 2024-09-21T07:49:42.412 INFO:teuthology.orchestra.run.smithi039.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:49:42.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.412+0000 7f9e32200640 1 -- 172.21.15.39:0/4086243877 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9e08078b90 msgr2=0x7f9e0807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:42.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.412+0000 7f9e32200640 1 --2- 172.21.15.39:0/4086243877 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9e08078b90 0x7f9e0807b0a0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c0730e0 tx=0x7f9e1c005f90 comp rx=0 tx=0).stop 2024-09-21T07:49:42.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.412+0000 7f9e32200640 1 -- 172.21.15.39:0/4086243877 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e2c0d0630 msgr2=0x7f9e2c0b3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:42.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.412+0000 7f9e32200640 1 --2- 172.21.15.39:0/4086243877 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e2c0d0630 0x7f9e2c0b3e60 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f9e1400c390 tx=0x7f9e1400c850 comp rx=0 tx=0).stop 2024-09-21T07:49:42.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.412+0000 7f9e32200640 1 -- 172.21.15.39:0/4086243877 shutdown_connections 2024-09-21T07:49:42.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.412+0000 7f9e32200640 1 --2- 172.21.15.39:0/4086243877 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9e08078b90 0x7f9e0807b0a0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.412+0000 7f9e32200640 1 --2- 172.21.15.39:0/4086243877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e2c15daf0 0x7f9e2c152880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.412+0000 7f9e32200640 1 --2- 172.21.15.39:0/4086243877 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e2c0d0630 0x7f9e2c0b3e60 unknown :-1 s=CLOSED pgs=2252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.412+0000 7f9e32200640 1 --2- 172.21.15.39:0/4086243877 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9e2c073dd0 0x7f9e2c1f57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.412+0000 7f9e32200640 1 -- 172.21.15.39:0/4086243877 >> 172.21.15.39:0/4086243877 conn(0x7f9e2c06cfc0 msgr2=0x7f9e2c0b1490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:42.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.412+0000 7f9e32200640 1 -- 172.21.15.39:0/4086243877 shutdown_connections 2024-09-21T07:49:42.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42.412+0000 7f9e32200640 1 -- 172.21.15.39:0/4086243877 wait complete. 2024-09-21T07:49:42.436 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:49:42.436 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:49:42.436 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi039.front.sepia.ceph.com 2024-09-21T07:49:42.436 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T07:49:42.436 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:49:42.436 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:49:42.436 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:49:42.436 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:49:42.436 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:49:42.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:42.437 DEBUG:teuthology.orchestra.run.smithi039:> ip addr 2024-09-21T07:49:42.472 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:49:42.473 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:49:42.473 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:49:42.473 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:49:42.473 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2024-09-21T07:49:42.473 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:49:42.473 INFO:teuthology.orchestra.run.smithi039.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:49:42.473 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:49:42.473 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f0 2024-09-21T07:49:42.473 INFO:teuthology.orchestra.run.smithi039.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:49:42.473 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c7 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:49:42.473 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f1 2024-09-21T07:49:42.473 INFO:teuthology.orchestra.run.smithi039.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:49:42.474 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:70 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:49:42.474 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f0 2024-09-21T07:49:42.474 INFO:teuthology.orchestra.run.smithi039.stdout: inet 172.21.15.39/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T07:49:42.474 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft 41737sec preferred_lft 41737sec 2024-09-21T07:49:42.474 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec4:7aff:febd:1370/64 scope link 2024-09-21T07:49:42.474 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:49:42.474 INFO:teuthology.orchestra.run.smithi039.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:49:42.474 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:71 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:49:42.474 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f1 2024-09-21T07:49:42.474 INFO:teuthology.orchestra.run.smithi039.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:49:42.474 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff 2024-09-21T07:49:42.474 INFO:teuthology.orchestra.run.smithi039.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:49:42.474 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:49:42.475 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::b079:ddff:fe0e:6333/64 scope link 2024-09-21T07:49:42.475 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:49:42.475 INFO:teuthology.orchestra.run.smithi039.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:49:42.475 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:49:42.475 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::2ca7:98ff:fe8a:d0ed/64 scope link 2024-09-21T07:49:42.475 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:49:42.475 INFO:teuthology.orchestra.run.smithi039.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:49:42.475 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether ee:10:53:01:0c:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:49:42.475 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec10:53ff:fe01:c44/64 scope link 2024-09-21T07:49:42.475 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:49:42.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:42.476 DEBUG:teuthology.orchestra.run.smithi039:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:49:42.566 INFO:teuthology.orchestra.run.smithi039.stdout:1 2024-09-21T07:49:42.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:42.569 DEBUG:teuthology.orchestra.run.smithi039:> route 2024-09-21T07:49:42.605 INFO:teuthology.orchestra.run.smithi039.stdout:Kernel IP routing table 2024-09-21T07:49:42.606 INFO:teuthology.orchestra.run.smithi039.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:49:42.606 INFO:teuthology.orchestra.run.smithi039.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T07:49:42.606 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T07:49:42.606 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:49:42.606 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:49:42.606 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-21T07:49:42.606 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T07:49:42.607 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T07:49:42.607 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T07:49:42.607 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-21T07:49:42.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:49:42.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:49:42.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:42.804 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-09-21T07:49:42.839 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:49:42.839 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T07:49:42.839 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:49:42.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:42.840 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-09-21T07:49:42.937 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:49:42.937 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T07:49:42.971 INFO:teuthology.orchestra.run.smithi039.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:49:42.972 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:49:42.972 DEBUG:teuthology.orchestra.run.smithi039:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:49:43.046 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-21T07:49:43.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:43.046 DEBUG:teuthology.orchestra.run.smithi039:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:49:43.248 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:49:43.249 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:49:43.283 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:49:43.283 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:49:43.283 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: resolved to: "172.21.15.39:3300,172.21.15.148:3300,172.21.15.148:3301" 2024-09-21T07:49:43.283 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: trying mount with new device syntax: 0@7ab9b04a-f88d-49aa-a493-e5c076637ffb.cephfs=/ 2024-09-21T07:49:43.283 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.39:3300/172.21.15.148:3300/172.21.15.148:3301" will pass to kernel 2024-09-21T07:49:43.285 INFO:teuthology.orchestra.run.smithi039.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T07:49:43.285 INFO:teuthology.orchestra.run.smithi039.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:49:43.285 INFO:teuthology.orchestra.run.smithi039.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:49:43.285 INFO:teuthology.orchestra.run.smithi039.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:49:43.285 INFO:teuthology.orchestra.run.smithi039.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:49:43.288 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:49:43.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:43.288 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:49:43.345 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:49:43.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:49:43.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:49:43.442 INFO:teuthology.orchestra.run.smithi039.stdout:client8615 2024-09-21T07:49:43.443 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:49:43.443 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:49:43.478 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get mds_max_snaps_per_dir 2024-09-21T07:49:43.761 INFO:teuthology.orchestra.run.smithi039.stdout:{"mds_max_snaps_per_dir":"100"} 2024-09-21T07:49:43.762 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_max_snaps_per_dir": "100" } 2024-09-21T07:49:43.762 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2024-09-21T07:49:43.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:43.990+0000 7fa075600640 1 -- 172.21.15.39:0/2525040189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa070155060 msgr2=0x7fa070155470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:43.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:43.990+0000 7fa075600640 1 --2- 172.21.15.39:0/2525040189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa070155060 0x7fa070155470 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7fa0700afd10 tx=0x7fa06802aac0 comp rx=0 tx=0).stop 2024-09-21T07:49:43.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:43.990+0000 7fa075600640 1 -- 172.21.15.39:0/2525040189 shutdown_connections 2024-09-21T07:49:43.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:43.990+0000 7fa075600640 1 --2- 172.21.15.39:0/2525040189 >> v2:172.21.15.148:3300/0 conn(0x7fa0700732b0 0x7fa0700736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:43.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:43.990+0000 7fa075600640 1 --2- 172.21.15.39:0/2525040189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa070155060 0x7fa070155470 unknown :-1 s=CLOSED pgs=2253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:43.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:43.990+0000 7fa075600640 1 -- 172.21.15.39:0/2525040189 >> 172.21.15.39:0/2525040189 conn(0x7fa07006d660 msgr2=0x7fa0700b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:43.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:43.990+0000 7fa075600640 1 -- 172.21.15.39:0/2525040189 shutdown_connections 2024-09-21T07:49:43.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:43.990+0000 7fa075600640 1 -- 172.21.15.39:0/2525040189 wait complete. 2024-09-21T07:49:43.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:43.993+0000 7fa075600640 1 Processor -- start 2024-09-21T07:49:43.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:43.993+0000 7fa075600640 1 -- start start 2024-09-21T07:49:43.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:43.994+0000 7fa075600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa0700732b0 0x7fa070141a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:43.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:43.994+0000 7fa075600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0700b4db0 0x7fa0701e8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:43.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:43.994+0000 7fa075600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa070155060 0x7fa07015db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:43.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:43.994+0000 7fa075600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa070155b30 con 0x7fa0700b4db0 2024-09-21T07:49:43.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:43.994+0000 7fa075600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa0701559b0 con 0x7fa070155060 2024-09-21T07:49:43.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:43.994+0000 7fa075600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa070155cb0 con 0x7fa0700732b0 2024-09-21T07:49:43.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:43.995+0000 7fa067400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0700b4db0 0x7fa0701e8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:43.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:43.995+0000 7fa067400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0700b4db0 0x7fa0701e8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43630/0 (socket says 172.21.15.39:43630) 2024-09-21T07:49:43.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:43.995+0000 7fa067400640 1 -- 172.21.15.39:0/255252021 learned_addr learned my addr 172.21.15.39:0/255252021 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:43.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:43.995+0000 7fa06fe00640 1 --2- 172.21.15.39:0/255252021 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa0700732b0 0x7fa070141a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:43.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:43.995+0000 7fa067400640 1 -- 172.21.15.39:0/255252021 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa0700732b0 msgr2=0x7fa070141a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:43.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:43.995+0000 7fa067400640 1 --2- 172.21.15.39:0/255252021 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa0700732b0 0x7fa070141a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:43.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:43.995+0000 7fa067400640 1 -- 172.21.15.39:0/255252021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa070155060 msgr2=0x7fa07015db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:43.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:43.995+0000 7fa067400640 1 --2- 172.21.15.39:0/255252021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa070155060 0x7fa07015db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:43.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:43.995+0000 7fa067400640 1 -- 172.21.15.39:0/255252021 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa070142660 con 0x7fa0700b4db0 2024-09-21T07:49:43.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:43.995+0000 7fa067400640 1 --2- 172.21.15.39:0/255252021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0700b4db0 0x7fa0701e8a00 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7fa05800d700 tx=0x7fa05800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:43.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:43.996+0000 7fa06cc00640 1 -- 172.21.15.39:0/255252021 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa058017070 con 0x7fa0700b4db0 2024-09-21T07:49:43.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:43.996+0000 7fa06cc00640 1 -- 172.21.15.39:0/255252021 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa058004310 con 0x7fa0700b4db0 2024-09-21T07:49:43.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:43.996+0000 7fa06cc00640 1 -- 172.21.15.39:0/255252021 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa058009a80 con 0x7fa0700b4db0 2024-09-21T07:49:43.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:43.996+0000 7fa075600640 1 -- 172.21.15.39:0/255252021 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa070142940 con 0x7fa0700b4db0 2024-09-21T07:49:43.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:43.996+0000 7fa075600640 1 -- 172.21.15.39:0/255252021 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa07013fd40 con 0x7fa0700b4db0 2024-09-21T07:49:43.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:43.997+0000 7fa075600640 1 -- 172.21.15.39:0/255252021 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0701423d0 con 0x7fa0700b4db0 2024-09-21T07:49:44.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:43.998+0000 7fa06cc00640 1 -- 172.21.15.39:0/255252021 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa058009c20 con 0x7fa0700b4db0 2024-09-21T07:49:44.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:43.999+0000 7fa06cc00640 1 --2- 172.21.15.39:0/255252021 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa04c078b90 0x7fa04c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:44.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:43.999+0000 7fa06cc00640 1 -- 172.21.15.39:0/255252021 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(233..233 src has 1..233) ==== 7016+0+0 (secure 0 0 0) 0x7fa05809b370 con 0x7fa0700b4db0 2024-09-21T07:49:44.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:43.999+0000 7fa06fe00640 1 --2- 172.21.15.39:0/255252021 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa04c078b90 0x7fa04c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:44.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:44.000+0000 7fa06fe00640 1 --2- 172.21.15.39:0/255252021 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa04c078b90 0x7fa04c07b0a0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fa0600015e0 tx=0x7fa060009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:44.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:44.004+0000 7fa06cc00640 1 -- 172.21.15.39:0/255252021 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa0580631f0 con 0x7fa0700b4db0 2024-09-21T07:49:44.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:44.238+0000 7fa075600640 1 -- 172.21.15.39:0/255252021 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fa0700736c0 con 0x7fa0700b4db0 2024-09-21T07:49:45.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.197+0000 7fa06cc00640 1 -- 172.21.15.39:0/255252021 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v245) ==== 167+0+0 (secure 0 0 0) 0x7fa058062af0 con 0x7fa0700b4db0 2024-09-21T07:49:45.199 INFO:teuthology.orchestra.run.smithi039.stderr:enabled new snapshots 2024-09-21T07:49:45.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.199+0000 7fa075600640 1 -- 172.21.15.39:0/255252021 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa04c078b90 msgr2=0x7fa04c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:45.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.199+0000 7fa075600640 1 --2- 172.21.15.39:0/255252021 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa04c078b90 0x7fa04c07b0a0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fa0600015e0 tx=0x7fa060009040 comp rx=0 tx=0).stop 2024-09-21T07:49:45.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.199+0000 7fa075600640 1 -- 172.21.15.39:0/255252021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0700b4db0 msgr2=0x7fa0701e8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:45.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.199+0000 7fa075600640 1 --2- 172.21.15.39:0/255252021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0700b4db0 0x7fa0701e8a00 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7fa05800d700 tx=0x7fa05800dbc0 comp rx=0 tx=0).stop 2024-09-21T07:49:45.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.200+0000 7fa075600640 1 -- 172.21.15.39:0/255252021 shutdown_connections 2024-09-21T07:49:45.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.200+0000 7fa075600640 1 --2- 172.21.15.39:0/255252021 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa04c078b90 0x7fa04c07b0a0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:45.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.200+0000 7fa075600640 1 --2- 172.21.15.39:0/255252021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa070155060 0x7fa07015db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:45.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.200+0000 7fa075600640 1 --2- 172.21.15.39:0/255252021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0700b4db0 0x7fa0701e8a00 unknown :-1 s=CLOSED pgs=2254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:45.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.200+0000 7fa075600640 1 --2- 172.21.15.39:0/255252021 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa0700732b0 0x7fa070141a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:45.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.200+0000 7fa075600640 1 -- 172.21.15.39:0/255252021 >> 172.21.15.39:0/255252021 conn(0x7fa07006d660 msgr2=0x7fa0700d09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:45.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.200+0000 7fa075600640 1 -- 172.21.15.39:0/255252021 shutdown_connections 2024-09-21T07:49:45.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.200+0000 7fa075600640 1 -- 172.21.15.39:0/255252021 wait complete. 2024-09-21T07:49:45.227 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-09-21T07:49:45.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.457+0000 7fd004c00640 1 -- 172.21.15.39:0/901206378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd00015d830 msgr2=0x7fd000156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:45.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.457+0000 7fd004c00640 1 --2- 172.21.15.39:0/901206378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd00015d830 0x7fd000156bb0 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8005fd0 tx=0x7fcfe802a4a0 comp rx=0 tx=0).stop 2024-09-21T07:49:45.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.457+0000 7fd004c00640 1 -- 172.21.15.39:0/901206378 shutdown_connections 2024-09-21T07:49:45.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.457+0000 7fd004c00640 1 --2- 172.21.15.39:0/901206378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd00015d830 0x7fd000156bb0 unknown :-1 s=CLOSED pgs=2255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:45.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.457+0000 7fd004c00640 1 --2- 172.21.15.39:0/901206378 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd0000b30a0 0x7fd0000cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:45.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.457+0000 7fd004c00640 1 -- 172.21.15.39:0/901206378 >> 172.21.15.39:0/901206378 conn(0x7fd0000b2460 msgr2=0x7fd00006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:45.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.457+0000 7fd004c00640 1 -- 172.21.15.39:0/901206378 shutdown_connections 2024-09-21T07:49:45.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.458+0000 7fd004c00640 1 -- 172.21.15.39:0/901206378 wait complete. 2024-09-21T07:49:45.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.460+0000 7fd004c00640 1 Processor -- start 2024-09-21T07:49:45.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.460+0000 7fd004c00640 1 -- start start 2024-09-21T07:49:45.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.461+0000 7fd004c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd0000b30a0 0x7fd0000c6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:45.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.461+0000 7fd004c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0000ccc60 0x7fd0000c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:45.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.461+0000 7fd004c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd00015d830 0x7fd0001f90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:45.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.461+0000 7fd004c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd0000af450 con 0x7fd00015d830 2024-09-21T07:49:45.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.461+0000 7fd004c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd0000af2d0 con 0x7fd0000ccc60 2024-09-21T07:49:45.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.461+0000 7fd004c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd0000af5d0 con 0x7fd0000b30a0 2024-09-21T07:49:45.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.462+0000 7fcffe800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd00015d830 0x7fd0001f90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:45.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.462+0000 7fcffe800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd00015d830 0x7fd0001f90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43652/0 (socket says 172.21.15.39:43652) 2024-09-21T07:49:45.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.462+0000 7fcffe800640 1 -- 172.21.15.39:0/218395364 learned_addr learned my addr 172.21.15.39:0/218395364 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:45.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.462+0000 7fcffde00640 1 --2- 172.21.15.39:0/218395364 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd0000b30a0 0x7fd0000c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:45.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.462+0000 7fcff7e00640 1 --2- 172.21.15.39:0/218395364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0000ccc60 0x7fd0000c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:45.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.462+0000 7fcffe800640 1 -- 172.21.15.39:0/218395364 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd0000b30a0 msgr2=0x7fd0000c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:45.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.462+0000 7fcffe800640 1 --2- 172.21.15.39:0/218395364 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd0000b30a0 0x7fd0000c6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:45.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.462+0000 7fcffe800640 1 -- 172.21.15.39:0/218395364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0000ccc60 msgr2=0x7fd0000c8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:45.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.462+0000 7fcffe800640 1 --2- 172.21.15.39:0/218395364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0000ccc60 0x7fd0000c8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:45.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.462+0000 7fcffe800640 1 -- 172.21.15.39:0/218395364 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0000c9710 con 0x7fd00015d830 2024-09-21T07:49:45.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.462+0000 7fcff7e00640 1 --2- 172.21.15.39:0/218395364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0000ccc60 0x7fd0000c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:49:45.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.463+0000 7fcffe800640 1 --2- 172.21.15.39:0/218395364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd00015d830 0x7fd0001f90c0 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7fcff800ba10 tx=0x7fcff800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:45.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.463+0000 7fcff6000640 1 -- 172.21.15.39:0/218395364 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff8019070 con 0x7fd00015d830 2024-09-21T07:49:45.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.463+0000 7fcff6000640 1 -- 172.21.15.39:0/218395364 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcff800cc30 con 0x7fd00015d830 2024-09-21T07:49:45.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.463+0000 7fcff6000640 1 -- 172.21.15.39:0/218395364 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff80146a0 con 0x7fd00015d830 2024-09-21T07:49:45.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.463+0000 7fd004c00640 1 -- 172.21.15.39:0/218395364 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0000c99f0 con 0x7fd00015d830 2024-09-21T07:49:45.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.463+0000 7fd004c00640 1 -- 172.21.15.39:0/218395364 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0000c9fb0 con 0x7fd00015d830 2024-09-21T07:49:45.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.464+0000 7fd004c00640 1 -- 172.21.15.39:0/218395364 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0000afa30 con 0x7fd00015d830 2024-09-21T07:49:45.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.465+0000 7fcff6000640 1 -- 172.21.15.39:0/218395364 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcff801c070 con 0x7fd00015d830 2024-09-21T07:49:45.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.466+0000 7fcff6000640 1 --2- 172.21.15.39:0/218395364 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcfdc078b90 0x7fcfdc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:45.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.467+0000 7fcff6000640 1 -- 172.21.15.39:0/218395364 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(233..233 src has 1..233) ==== 7016+0+0 (secure 0 0 0) 0x7fcff809ade0 con 0x7fd00015d830 2024-09-21T07:49:45.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.467+0000 7fcffde00640 1 --2- 172.21.15.39:0/218395364 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcfdc078b90 0x7fcfdc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:45.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.468+0000 7fcffde00640 1 --2- 172.21.15.39:0/218395364 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcfdc078b90 0x7fcfdc07b0a0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fcff0005d70 tx=0x7fcff0005d00 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:45.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.471+0000 7fcff6000640 1 -- 172.21.15.39:0/218395364 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcff800ce20 con 0x7fd00015d830 2024-09-21T07:49:45.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:45.710+0000 7fd004c00640 1 -- 172.21.15.39:0/218395364 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7fd0000ce050 con 0x7fd00015d830 2024-09-21T07:49:46.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.207+0000 7fcff6000640 1 -- 172.21.15.39:0/218395364 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v246) ==== 105+0+0 (secure 0 0 0) 0x7fcff8062d90 con 0x7fd00015d830 2024-09-21T07:49:46.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.210+0000 7fd004c00640 1 -- 172.21.15.39:0/218395364 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcfdc078b90 msgr2=0x7fcfdc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:46.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.210+0000 7fd004c00640 1 --2- 172.21.15.39:0/218395364 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcfdc078b90 0x7fcfdc07b0a0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fcff0005d70 tx=0x7fcff0005d00 comp rx=0 tx=0).stop 2024-09-21T07:49:46.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.210+0000 7fd004c00640 1 -- 172.21.15.39:0/218395364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd00015d830 msgr2=0x7fd0001f90c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:46.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.210+0000 7fd004c00640 1 --2- 172.21.15.39:0/218395364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd00015d830 0x7fd0001f90c0 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7fcff800ba10 tx=0x7fcff800bed0 comp rx=0 tx=0).stop 2024-09-21T07:49:46.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.210+0000 7fd004c00640 1 -- 172.21.15.39:0/218395364 shutdown_connections 2024-09-21T07:49:46.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.210+0000 7fd004c00640 1 --2- 172.21.15.39:0/218395364 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcfdc078b90 0x7fcfdc07b0a0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:46.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.210+0000 7fd004c00640 1 --2- 172.21.15.39:0/218395364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd00015d830 0x7fd0001f90c0 unknown :-1 s=CLOSED pgs=2256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:46.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.210+0000 7fd004c00640 1 --2- 172.21.15.39:0/218395364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0000ccc60 0x7fd0000c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:46.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.210+0000 7fd004c00640 1 --2- 172.21.15.39:0/218395364 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd0000b30a0 0x7fd0000c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:46.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.210+0000 7fd004c00640 1 -- 172.21.15.39:0/218395364 >> 172.21.15.39:0/218395364 conn(0x7fd0000b2460 msgr2=0x7fd0000ade20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:46.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.210+0000 7fd004c00640 1 -- 172.21.15.39:0/218395364 shutdown_connections 2024-09-21T07:49:46.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.210+0000 7fd004c00640 1 -- 172.21.15.39:0/218395364 wait complete. 2024-09-21T07:49:46.242 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:49:46.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.469+0000 7f1e79200640 1 -- 172.21.15.39:0/3634972547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e58004800 msgr2=0x7f1e58004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:46.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.469+0000 7f1e79200640 1 --2- 172.21.15.39:0/3634972547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e58004800 0x7f1e58004c10 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c00c310 tx=0x7f1e58005830 comp rx=0 tx=0).stop 2024-09-21T07:49:46.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.469+0000 7f1e79200640 1 -- 172.21.15.39:0/3634972547 shutdown_connections 2024-09-21T07:49:46.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.469+0000 7f1e79200640 1 --2- 172.21.15.39:0/3634972547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e58004800 0x7f1e58004c10 unknown :-1 s=CLOSED pgs=2258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:46.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.469+0000 7f1e79200640 1 -- 172.21.15.39:0/3634972547 >> 172.21.15.39:0/3634972547 conn(0x7f1e7406cfc0 msgr2=0x7f1e7406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:46.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.469+0000 7f1e79200640 1 -- 172.21.15.39:0/3634972547 shutdown_connections 2024-09-21T07:49:46.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.469+0000 7f1e79200640 1 -- 172.21.15.39:0/3634972547 wait complete. 2024-09-21T07:49:46.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.471+0000 7f1e79200640 1 Processor -- start 2024-09-21T07:49:46.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.472+0000 7f1e79200640 1 -- start start 2024-09-21T07:49:46.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.473+0000 7f1e79200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1e58004800 0x7f1e74149500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:46.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.473+0000 7f1e79200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e7414ffb0 0x7f1e74152390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:46.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.473+0000 7f1e79200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e740b2870 0x7f1e74149b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:46.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.473+0000 7f1e79200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1e740d05b0 con 0x7f1e740b2870 2024-09-21T07:49:46.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.473+0000 7f1e79200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1e740d0430 con 0x7f1e7414ffb0 2024-09-21T07:49:46.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.473+0000 7f1e79200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f1e740d0730 con 0x7f1e58004800 2024-09-21T07:49:46.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.473+0000 7f1e73e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e740b2870 0x7f1e74149b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:46.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.473+0000 7f1e73e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e740b2870 0x7f1e74149b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43682/0 (socket says 172.21.15.39:43682) 2024-09-21T07:49:46.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.473+0000 7f1e73e00640 1 -- 172.21.15.39:0/551470171 learned_addr learned my addr 172.21.15.39:0/551470171 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:46.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.474+0000 7f1e73e00640 1 -- 172.21.15.39:0/551470171 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1e58004800 msgr2=0x7f1e74149500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:46.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.474+0000 7f1e73e00640 1 --2- 172.21.15.39:0/551470171 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1e58004800 0x7f1e74149500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:46.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.474+0000 7f1e73e00640 1 -- 172.21.15.39:0/551470171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e7414ffb0 msgr2=0x7f1e74152390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:46.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.474+0000 7f1e73e00640 1 --2- 172.21.15.39:0/551470171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e7414ffb0 0x7f1e74152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:46.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.474+0000 7f1e73e00640 1 -- 172.21.15.39:0/551470171 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e741528d0 con 0x7f1e740b2870 2024-09-21T07:49:46.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.474+0000 7f1e73e00640 1 --2- 172.21.15.39:0/551470171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e740b2870 0x7f1e74149b90 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c005fd0 tx=0x7f1e6c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:46.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.474+0000 7f1e6be00640 1 -- 172.21.15.39:0/551470171 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c003d80 con 0x7f1e740b2870 2024-09-21T07:49:46.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.474+0000 7f1e6be00640 1 -- 172.21.15.39:0/551470171 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e6c004b20 con 0x7f1e740b2870 2024-09-21T07:49:46.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.474+0000 7f1e6be00640 1 -- 172.21.15.39:0/551470171 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c004030 con 0x7f1e740b2870 2024-09-21T07:49:46.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.474+0000 7f1e79200640 1 -- 172.21.15.39:0/551470171 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e740aef00 con 0x7f1e740b2870 2024-09-21T07:49:46.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.475+0000 7f1e79200640 1 -- 172.21.15.39:0/551470171 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e74151b90 con 0x7f1e740b2870 2024-09-21T07:49:46.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.476+0000 7f1e79200640 1 -- 172.21.15.39:0/551470171 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e740d0c20 con 0x7f1e740b2870 2024-09-21T07:49:46.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.477+0000 7f1e6be00640 1 -- 172.21.15.39:0/551470171 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1e6c0041d0 con 0x7f1e740b2870 2024-09-21T07:49:46.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.478+0000 7f1e6be00640 1 --2- 172.21.15.39:0/551470171 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1e50078bc0 0x7f1e5007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:46.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.478+0000 7f1e6be00640 1 -- 172.21.15.39:0/551470171 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(233..233 src has 1..233) ==== 7016+0+0 (secure 0 0 0) 0x7f1e6c09c160 con 0x7f1e740b2870 2024-09-21T07:49:46.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.478+0000 7f1e73400640 1 --2- 172.21.15.39:0/551470171 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1e50078bc0 0x7f1e5007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:46.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.479+0000 7f1e73400640 1 --2- 172.21.15.39:0/551470171 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1e50078bc0 0x7f1e5007b0d0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f1e74052910 tx=0x7f1e64006a60 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:46.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.482+0000 7f1e6be00640 1 -- 172.21.15.39:0/551470171 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1e6c064100 con 0x7f1e740b2870 2024-09-21T07:49:46.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.718+0000 7f1e79200640 1 -- 172.21.15.39:0/551470171 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e7406d680 con 0x7f1e740b2870 2024-09-21T07:49:46.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.718+0000 7f1e6be00640 1 -- 172.21.15.39:0/551470171 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 247 v247) ==== 95+0+5263 (secure 0 0 0) 0x7f1e6c063a00 con 0x7f1e740b2870 2024-09-21T07:49:46.720 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:49:46.721 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":247,"btime":"2024-09-21T07:49:46:215892+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8531,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1438851059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1438851059},{"type":"v1","addr":"172.21.15.148:6837","nonce":1438851059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":239},{"gid":9709,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3497801035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3497801035},{"type":"v1","addr":"172.21.15.148:6836","nonce":3497801035}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":239}],"filesystems":[{"mdsmap":{"epoch":247,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:49:46.215876+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1],"up":{"mds_0":10635,"mds_1":10632},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10632":{"gid":10632,"name":"a","rank":1,"incarnation":247,"state":"up:creating","state_seq":1,"addr":"172.21.15.39:6834/4143807732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4143807732},{"type":"v1","addr":"172.21.15.39:6834","nonce":4143807732}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10635":{"gid":10635,"name":"c","rank":0,"incarnation":240,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/2824493631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2824493631},{"type":"v1","addr":"172.21.15.39:6835","nonce":2824493631}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10635,"qdb_cluster":[10635]},"id":13}]} 2024-09-21T07:49:46.722 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 247 2024-09-21T07:49:46.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.721+0000 7f1e79200640 1 -- 172.21.15.39:0/551470171 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1e50078bc0 msgr2=0x7f1e5007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:46.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.721+0000 7f1e79200640 1 --2- 172.21.15.39:0/551470171 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1e50078bc0 0x7f1e5007b0d0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f1e74052910 tx=0x7f1e64006a60 comp rx=0 tx=0).stop 2024-09-21T07:49:46.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.721+0000 7f1e79200640 1 -- 172.21.15.39:0/551470171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e740b2870 msgr2=0x7f1e74149b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:46.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.721+0000 7f1e79200640 1 --2- 172.21.15.39:0/551470171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e740b2870 0x7f1e74149b90 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c005fd0 tx=0x7f1e6c00f790 comp rx=0 tx=0).stop 2024-09-21T07:49:46.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.721+0000 7f1e79200640 1 -- 172.21.15.39:0/551470171 shutdown_connections 2024-09-21T07:49:46.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.721+0000 7f1e79200640 1 --2- 172.21.15.39:0/551470171 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1e50078bc0 0x7f1e5007b0d0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:46.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.721+0000 7f1e79200640 1 --2- 172.21.15.39:0/551470171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e740b2870 0x7f1e74149b90 unknown :-1 s=CLOSED pgs=2259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:46.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.721+0000 7f1e79200640 1 --2- 172.21.15.39:0/551470171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e7414ffb0 0x7f1e74152390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:46.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.721+0000 7f1e79200640 1 --2- 172.21.15.39:0/551470171 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1e58004800 0x7f1e74149500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:46.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.721+0000 7f1e79200640 1 -- 172.21.15.39:0/551470171 >> 172.21.15.39:0/551470171 conn(0x7f1e7406cfc0 msgr2=0x7f1e7415ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:46.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.721+0000 7f1e79200640 1 -- 172.21.15.39:0/551470171 shutdown_connections 2024-09-21T07:49:46.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:46.721+0000 7f1e79200640 1 -- 172.21.15.39:0/551470171 wait complete. 2024-09-21T07:49:47.150 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 247, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:49:46.215876+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 10635, 'mds_1': 10632}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10632': {'gid': 10632, 'name': 'a', 'rank': 1, 'incarnation': 247, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.39:6834/4143807732', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 4143807732}, {'type': 'v1', 'addr': '172.21.15.39:6834', 'nonce': 4143807732}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10635': {'gid': 10635, 'name': 'c', 'rank': 0, 'incarnation': 240, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.39:6835/2824493631', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 2824493631}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 2824493631}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10635, 'qdb_cluster': [10635]} 2024-09-21T07:49:47.150 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10632:up:creating 2024-09-21T07:49:48.151 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:49:48.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.390+0000 7f550be00640 1 -- 172.21.15.39:0/1914256854 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f550c0b5af0 msgr2=0x7f550c0d0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:48.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.390+0000 7f550be00640 1 --2- 172.21.15.39:0/1914256854 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f550c0b5af0 0x7f550c0d0540 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7f54f4005fd0 tx=0x7f54f402a670 comp rx=0 tx=0).stop 2024-09-21T07:49:48.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.390+0000 7f550be00640 1 -- 172.21.15.39:0/1914256854 shutdown_connections 2024-09-21T07:49:48.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.390+0000 7f550be00640 1 --2- 172.21.15.39:0/1914256854 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f550c0b5af0 0x7f550c0d0540 unknown :-1 s=CLOSED pgs=2261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:48.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.390+0000 7f550be00640 1 -- 172.21.15.39:0/1914256854 >> 172.21.15.39:0/1914256854 conn(0x7f550c0d0cb0 msgr2=0x7f550c0d10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:48.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.391+0000 7f550be00640 1 -- 172.21.15.39:0/1914256854 shutdown_connections 2024-09-21T07:49:48.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.391+0000 7f550be00640 1 -- 172.21.15.39:0/1914256854 wait complete. 2024-09-21T07:49:48.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.393+0000 7f550be00640 1 Processor -- start 2024-09-21T07:49:48.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.393+0000 7f550be00640 1 -- start start 2024-09-21T07:49:48.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.394+0000 7f550be00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f550c15bce0 0x7f550c156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:48.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.394+0000 7f550be00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f550c15f2d0 0x7f550c1f3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:48.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.394+0000 7f550be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f550c15e000 0x7f550c15e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:48.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.394+0000 7f550be00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f550c070410 con 0x7f550c15e000 2024-09-21T07:49:48.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.394+0000 7f550be00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f550c070290 con 0x7f550c15bce0 2024-09-21T07:49:48.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.394+0000 7f550be00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f550c070590 con 0x7f550c15f2d0 2024-09-21T07:49:48.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.395+0000 7f550b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f550c15e000 0x7f550c15e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:48.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.395+0000 7f550aa00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f550c15bce0 0x7f550c156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:48.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.395+0000 7f550b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f550c15e000 0x7f550c15e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43712/0 (socket says 172.21.15.39:43712) 2024-09-21T07:49:48.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.395+0000 7f550b400640 1 -- 172.21.15.39:0/3000225243 learned_addr learned my addr 172.21.15.39:0/3000225243 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:48.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.395+0000 7f550aa00640 1 --2- 172.21.15.39:0/3000225243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f550c15bce0 0x7f550c156de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.39:42468/0 (socket says 172.21.15.39:42468) 2024-09-21T07:49:48.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.395+0000 7f5502000640 1 --2- 172.21.15.39:0/3000225243 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f550c15f2d0 0x7f550c1f3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:48.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.395+0000 7f550b400640 1 -- 172.21.15.39:0/3000225243 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f550c15f2d0 msgr2=0x7f550c1f3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:48.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.395+0000 7f550b400640 1 --2- 172.21.15.39:0/3000225243 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f550c15f2d0 0x7f550c1f3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:48.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.395+0000 7f550b400640 1 -- 172.21.15.39:0/3000225243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f550c15bce0 msgr2=0x7f550c156de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:48.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.395+0000 7f550b400640 1 --2- 172.21.15.39:0/3000225243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f550c15bce0 0x7f550c156de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:48.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.395+0000 7f550b400640 1 -- 172.21.15.39:0/3000225243 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f550c15e950 con 0x7f550c15e000 2024-09-21T07:49:48.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.395+0000 7f550b400640 1 --2- 172.21.15.39:0/3000225243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f550c15e000 0x7f550c15e410 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7f550400c760 tx=0x7f550400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:48.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.396+0000 7f5503400640 1 -- 172.21.15.39:0/3000225243 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5504003b90 con 0x7f550c15e000 2024-09-21T07:49:48.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.396+0000 7f5503400640 1 -- 172.21.15.39:0/3000225243 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5504003d50 con 0x7f550c15e000 2024-09-21T07:49:48.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.396+0000 7f550be00640 1 -- 172.21.15.39:0/3000225243 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f550c14d190 con 0x7f550c15e000 2024-09-21T07:49:48.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.396+0000 7f5503400640 1 -- 172.21.15.39:0/3000225243 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550400aab0 con 0x7f550c15e000 2024-09-21T07:49:48.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.396+0000 7f550be00640 1 -- 172.21.15.39:0/3000225243 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f550c14d750 con 0x7f550c15e000 2024-09-21T07:49:48.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.397+0000 7f550be00640 1 -- 172.21.15.39:0/3000225243 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f550c157610 con 0x7f550c15e000 2024-09-21T07:49:48.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.398+0000 7f5503400640 1 -- 172.21.15.39:0/3000225243 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f550400ac50 con 0x7f550c15e000 2024-09-21T07:49:48.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.399+0000 7f5503400640 1 --2- 172.21.15.39:0/3000225243 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f54e8078b90 0x7f54e807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:48.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.399+0000 7f5503400640 1 -- 172.21.15.39:0/3000225243 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(233..233 src has 1..233) ==== 7016+0+0 (secure 0 0 0) 0x7f550409aec0 con 0x7f550c15e000 2024-09-21T07:49:48.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.400+0000 7f550aa00640 1 --2- 172.21.15.39:0/3000225243 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f54e8078b90 0x7f54e807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:48.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.401+0000 7f550aa00640 1 --2- 172.21.15.39:0/3000225243 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f54e8078b90 0x7f54e807b0a0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f54fc000f60 tx=0x7f54fc009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:48.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.404+0000 7f5503400640 1 -- 172.21.15.39:0/3000225243 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f550c15e950 con 0x7f550c15e000 2024-09-21T07:49:48.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.637+0000 7f550be00640 1 -- 172.21.15.39:0/3000225243 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f550c070a40 con 0x7f550c15e000 2024-09-21T07:49:48.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.638+0000 7f5503400640 1 -- 172.21.15.39:0/3000225243 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 250 v250) ==== 95+0+5286 (secure 0 0 0) 0x7f5504062780 con 0x7f550c15e000 2024-09-21T07:49:48.640 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:49:48.640 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":250,"btime":"2024-09-21T07:49:48:234824+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8531,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1438851059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1438851059},{"type":"v1","addr":"172.21.15.148:6837","nonce":1438851059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":239}],"filesystems":[{"mdsmap":{"epoch":250,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:49:48.234812+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":10635,"mds_1":10632,"mds_2":9709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9709":{"gid":9709,"name":"b","rank":2,"incarnation":249,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3497801035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3497801035},{"type":"v1","addr":"172.21.15.148:6836","nonce":3497801035}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10632":{"gid":10632,"name":"a","rank":1,"incarnation":247,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6834/4143807732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4143807732},{"type":"v1","addr":"172.21.15.39:6834","nonce":4143807732}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10635":{"gid":10635,"name":"c","rank":0,"incarnation":240,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/2824493631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2824493631},{"type":"v1","addr":"172.21.15.39:6835","nonce":2824493631}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10635,"qdb_cluster":[9709,10632,10635]},"id":13}]} 2024-09-21T07:49:48.640 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 250 2024-09-21T07:49:48.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.640+0000 7f550be00640 1 -- 172.21.15.39:0/3000225243 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f54e8078b90 msgr2=0x7f54e807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:48.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.640+0000 7f550be00640 1 --2- 172.21.15.39:0/3000225243 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f54e8078b90 0x7f54e807b0a0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f54fc000f60 tx=0x7f54fc009040 comp rx=0 tx=0).stop 2024-09-21T07:49:48.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.640+0000 7f550be00640 1 -- 172.21.15.39:0/3000225243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f550c15e000 msgr2=0x7f550c15e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:48.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.640+0000 7f550be00640 1 --2- 172.21.15.39:0/3000225243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f550c15e000 0x7f550c15e410 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7f550400c760 tx=0x7f550400cc20 comp rx=0 tx=0).stop 2024-09-21T07:49:48.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.640+0000 7f550be00640 1 -- 172.21.15.39:0/3000225243 shutdown_connections 2024-09-21T07:49:48.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.640+0000 7f550be00640 1 --2- 172.21.15.39:0/3000225243 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f54e8078b90 0x7f54e807b0a0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:48.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.640+0000 7f550be00640 1 --2- 172.21.15.39:0/3000225243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f550c15e000 0x7f550c15e410 unknown :-1 s=CLOSED pgs=2262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:48.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.640+0000 7f550be00640 1 --2- 172.21.15.39:0/3000225243 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f550c15f2d0 0x7f550c1f3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:48.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.640+0000 7f550be00640 1 --2- 172.21.15.39:0/3000225243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f550c15bce0 0x7f550c156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:48.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.640+0000 7f550be00640 1 -- 172.21.15.39:0/3000225243 >> 172.21.15.39:0/3000225243 conn(0x7f550c0d0cb0 msgr2=0x7f550c14c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:48.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.640+0000 7f550be00640 1 -- 172.21.15.39:0/3000225243 shutdown_connections 2024-09-21T07:49:48.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:48.640+0000 7f550be00640 1 -- 172.21.15.39:0/3000225243 wait complete. 2024-09-21T07:49:49.066 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 250, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:49:48.234812+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 10635, 'mds_1': 10632, 'mds_2': 9709}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9709': {'gid': 9709, 'name': 'b', 'rank': 2, 'incarnation': 249, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.148:6836/3497801035', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 3497801035}, {'type': 'v1', 'addr': '172.21.15.148:6836', 'nonce': 3497801035}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10632': {'gid': 10632, 'name': 'a', 'rank': 1, 'incarnation': 247, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.39:6834/4143807732', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 4143807732}, {'type': 'v1', 'addr': '172.21.15.39:6834', 'nonce': 4143807732}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10635': {'gid': 10635, 'name': 'c', 'rank': 0, 'incarnation': 240, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.39:6835/2824493631', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 2824493631}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 2824493631}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10635, 'qdb_cluster': [9709, 10632, 10635]} 2024-09-21T07:49:49.067 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-21T07:49:49.067 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-21T07:49:49.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.295+0000 7fbf41a00640 1 -- 172.21.15.39:0/2331534991 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf3c151640 msgr2=0x7fbf3c151a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:49.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.295+0000 7fbf41a00640 1 --2- 172.21.15.39:0/2331534991 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf3c151640 0x7fbf3c151a50 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c005fd0 tx=0x7fbf2c02b170 comp rx=0 tx=0).stop 2024-09-21T07:49:49.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.295+0000 7fbf41a00640 1 -- 172.21.15.39:0/2331534991 shutdown_connections 2024-09-21T07:49:49.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.295+0000 7fbf41a00640 1 --2- 172.21.15.39:0/2331534991 >> v2:172.21.15.148:3300/0 conn(0x7fbf3c070570 0x7fbf3c070980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.295+0000 7fbf41a00640 1 --2- 172.21.15.39:0/2331534991 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf3c151640 0x7fbf3c151a50 unknown :-1 s=CLOSED pgs=2263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.295+0000 7fbf41a00640 1 -- 172.21.15.39:0/2331534991 >> 172.21.15.39:0/2331534991 conn(0x7fbf3c06cfc0 msgr2=0x7fbf3c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:49.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.295+0000 7fbf41a00640 1 -- 172.21.15.39:0/2331534991 shutdown_connections 2024-09-21T07:49:49.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.295+0000 7fbf41a00640 1 -- 172.21.15.39:0/2331534991 wait complete. 2024-09-21T07:49:49.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.298+0000 7fbf41a00640 1 Processor -- start 2024-09-21T07:49:49.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.298+0000 7fbf41a00640 1 -- start start 2024-09-21T07:49:49.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.299+0000 7fbf41a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf3c070570 0x7fbf3c15d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.299+0000 7fbf41a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbf3c0cf750 0x7fbf3c1f98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.299+0000 7fbf41a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf3c151640 0x7fbf3c147e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.299+0000 7fbf41a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c15eb60 con 0x7fbf3c151640 2024-09-21T07:49:49.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.299+0000 7fbf41a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c15e9e0 con 0x7fbf3c070570 2024-09-21T07:49:49.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.299+0000 7fbf41a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fbf3c15ece0 con 0x7fbf3c0cf750 2024-09-21T07:49:49.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.300+0000 7fbf3b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf3c151640 0x7fbf3c147e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.300+0000 7fbf3b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf3c151640 0x7fbf3c147e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43730/0 (socket says 172.21.15.39:43730) 2024-09-21T07:49:49.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.300+0000 7fbf3b400640 1 -- 172.21.15.39:0/3805105569 learned_addr learned my addr 172.21.15.39:0/3805105569 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:49.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.300+0000 7fbf3aa00640 1 --2- 172.21.15.39:0/3805105569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf3c070570 0x7fbf3c15d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.300+0000 7fbf33e00640 1 --2- 172.21.15.39:0/3805105569 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbf3c0cf750 0x7fbf3c1f98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.300+0000 7fbf3b400640 1 -- 172.21.15.39:0/3805105569 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbf3c0cf750 msgr2=0x7fbf3c1f98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:49.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.300+0000 7fbf3b400640 1 --2- 172.21.15.39:0/3805105569 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbf3c0cf750 0x7fbf3c1f98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.300+0000 7fbf3b400640 1 -- 172.21.15.39:0/3805105569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf3c070570 msgr2=0x7fbf3c15d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:49.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.300+0000 7fbf3b400640 1 --2- 172.21.15.39:0/3805105569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf3c070570 0x7fbf3c15d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.300+0000 7fbf3b400640 1 -- 172.21.15.39:0/3805105569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf3c148340 con 0x7fbf3c151640 2024-09-21T07:49:49.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.300+0000 7fbf3b400640 1 --2- 172.21.15.39:0/3805105569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf3c151640 0x7fbf3c147e00 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7fbf3400b9e0 tx=0x7fbf3400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:49.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.301+0000 7fbf32a00640 1 -- 172.21.15.39:0/3805105569 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf34003b70 con 0x7fbf3c151640 2024-09-21T07:49:49.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.301+0000 7fbf32a00640 1 -- 172.21.15.39:0/3805105569 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf34003e00 con 0x7fbf3c151640 2024-09-21T07:49:49.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.301+0000 7fbf32a00640 1 -- 172.21.15.39:0/3805105569 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3400cb80 con 0x7fbf3c151640 2024-09-21T07:49:49.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.301+0000 7fbf41a00640 1 -- 172.21.15.39:0/3805105569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf3c15db30 con 0x7fbf3c151640 2024-09-21T07:49:49.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.301+0000 7fbf41a00640 1 -- 172.21.15.39:0/3805105569 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf3c1f90e0 con 0x7fbf3c151640 2024-09-21T07:49:49.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.303+0000 7fbf32a00640 1 -- 172.21.15.39:0/3805105569 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbf34004000 con 0x7fbf3c151640 2024-09-21T07:49:49.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.305+0000 7fbf32a00640 1 --2- 172.21.15.39:0/3805105569 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbf18078b90 0x7fbf1807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.305+0000 7fbf32a00640 1 -- 172.21.15.39:0/3805105569 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(233..233 src has 1..233) ==== 7016+0+0 (secure 0 0 0) 0x7fbf3409afd0 con 0x7fbf3c151640 2024-09-21T07:49:49.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.305+0000 7fbf3aa00640 1 --2- 172.21.15.39:0/3805105569 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbf18078b90 0x7fbf1807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.306+0000 7fbf48a62740 1 Processor -- start 2024-09-21T07:49:49.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.306+0000 7fbf48a62740 1 -- start start 2024-09-21T07:49:49.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.306+0000 7fbf3aa00640 1 --2- 172.21.15.39:0/3805105569 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbf18078b90 0x7fbf1807b0a0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c033000 tx=0x7fbf2c035040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:49.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.307+0000 7fbf48a62740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c7429e270 0x557c7466e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.307+0000 7fbf48a62740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c742753b0 0x557c7466eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.307+0000 7fbf48a62740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557c7427d590 0x557c7463ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.307+0000 7fbf48a62740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557c745c8430 con 0x557c7429e270 2024-09-21T07:49:49.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.307+0000 7fbf48a62740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557c745b85d0 con 0x557c742753b0 2024-09-21T07:49:49.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.307+0000 7fbf48a62740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x557c745a7a00 con 0x557c7427d590 2024-09-21T07:49:49.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.308+0000 7fbf3aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c7429e270 0x557c7466e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.308+0000 7fbf3aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c7429e270 0x557c7466e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43746/0 (socket says 172.21.15.39:43746) 2024-09-21T07:49:49.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.308+0000 7fbf3aa00640 1 -- 172.21.15.39:0/345569762 learned_addr learned my addr 172.21.15.39:0/345569762 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:49.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.308+0000 7fbf3b400640 1 --2- 172.21.15.39:0/345569762 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557c7427d590 0x557c7463ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.308+0000 7fbf33e00640 1 --2- 172.21.15.39:0/345569762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c742753b0 0x557c7466eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.308+0000 7fbf3aa00640 1 -- 172.21.15.39:0/345569762 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557c7427d590 msgr2=0x557c7463ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:49.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.308+0000 7fbf3aa00640 1 --2- 172.21.15.39:0/345569762 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557c7427d590 0x557c7463ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.308+0000 7fbf3aa00640 1 -- 172.21.15.39:0/345569762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c742753b0 msgr2=0x557c7466eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:49.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.308+0000 7fbf3aa00640 1 --2- 172.21.15.39:0/345569762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c742753b0 0x557c7466eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.308+0000 7fbf3aa00640 1 -- 172.21.15.39:0/345569762 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c74570520 con 0x557c7429e270 2024-09-21T07:49:49.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.308+0000 7fbf3aa00640 1 --2- 172.21.15.39:0/345569762 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c7429e270 0x557c7466e970 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c0075a0 tx=0x7fbf2c02b170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a1bdf5a6fbc43 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:49.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.309+0000 7fbf2b400640 1 -- 172.21.15.39:0/345569762 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c038040 con 0x557c7429e270 2024-09-21T07:49:49.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.309+0000 7fbf2b400640 1 -- 172.21.15.39:0/345569762 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf2c007b90 con 0x557c7429e270 2024-09-21T07:49:49.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.309+0000 7fbf2b400640 1 -- 172.21.15.39:0/345569762 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c004040 con 0x557c7429e270 2024-09-21T07:49:49.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.309+0000 7fbf48a62740 1 -- 172.21.15.39:0/345569762 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c7429e270 msgr2=0x557c7466e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:49.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.309+0000 7fbf48a62740 1 --2- 172.21.15.39:0/345569762 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c7429e270 0x557c7466e970 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c0075a0 tx=0x7fbf2c02b170 comp rx=0 tx=0).stop 2024-09-21T07:49:49.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.310+0000 7fbf48a62740 1 -- 172.21.15.39:0/345569762 shutdown_connections 2024-09-21T07:49:49.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.310+0000 7fbf48a62740 1 --2- 172.21.15.39:0/345569762 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557c7427d590 0x557c7463ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.310+0000 7fbf48a62740 1 --2- 172.21.15.39:0/345569762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c742753b0 0x557c7466eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.310+0000 7fbf48a62740 1 --2- 172.21.15.39:0/345569762 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c7429e270 0x557c7466e970 unknown :-1 s=CLOSED pgs=2265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.310+0000 7fbf48a62740 1 -- 172.21.15.39:0/345569762 >> 172.21.15.39:0/345569762 conn(0x557c742923c0 msgr2=0x557c74644970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:49.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.310+0000 7fbf48a62740 1 -- 172.21.15.39:0/345569762 shutdown_connections 2024-09-21T07:49:49.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.310+0000 7fbf48a62740 1 -- 172.21.15.39:0/345569762 wait complete. 2024-09-21T07:49:49.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.311+0000 7fbf48a62740 1 -- start start 2024-09-21T07:49:49.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.311+0000 7fbf48a62740 1 Processor -- start 2024-09-21T07:49:49.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.312+0000 7fbf48a62740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c742753b0 0x557c74689fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.312+0000 7fbf48a62740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557c7427d590 0x557c74664e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.313+0000 7fbf48a62740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c7429e270 0x557c74667240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.313+0000 7fbf48a62740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557c7454e330 con 0x557c742753b0 2024-09-21T07:49:49.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.313+0000 7fbf48a62740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557c745ab810 con 0x557c7429e270 2024-09-21T07:49:49.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.313+0000 7fbf48a62740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x557c74318620 con 0x557c7427d590 2024-09-21T07:49:49.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.313+0000 7fbf48a62740 10 client.0 _ll_register_callbacks cb 0x557c74553420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:49:49.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.313+0000 7fbf3aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c742753b0 0x557c74689fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.313+0000 7fbf3aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c742753b0 0x557c74689fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43762/0 (socket says 172.21.15.39:43762) 2024-09-21T07:49:49.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.313+0000 7fbf3aa00640 1 -- 172.21.15.39:0/3963525339 learned_addr learned my addr 172.21.15.39:0/3963525339 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:49.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.313+0000 7fbf33e00640 1 --2- 172.21.15.39:0/3963525339 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557c7427d590 0x557c74664e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.313+0000 7fbf3b400640 1 --2- 172.21.15.39:0/3963525339 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c7429e270 0x557c74667240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.313+0000 7fbf3aa00640 1 -- 172.21.15.39:0/3963525339 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557c7427d590 msgr2=0x557c74664e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:49.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.313+0000 7fbf3aa00640 1 --2- 172.21.15.39:0/3963525339 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557c7427d590 0x557c74664e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.313+0000 7fbf3aa00640 1 -- 172.21.15.39:0/3963525339 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c7429e270 msgr2=0x557c74667240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:49.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.313+0000 7fbf3aa00640 1 --2- 172.21.15.39:0/3963525339 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c7429e270 0x557c74667240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.313+0000 7fbf3aa00640 1 -- 172.21.15.39:0/3963525339 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c74309850 con 0x557c742753b0 2024-09-21T07:49:49.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.313+0000 7fbf33e00640 1 --2- 172.21.15.39:0/3963525339 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557c7427d590 0x557c74664e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:49.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.314+0000 7fbf3aa00640 1 --2- 172.21.15.39:0/3963525339 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c742753b0 0x557c74689fb0 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c006010 tx=0x7fbf2c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31261574b1fecf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:49.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.314+0000 7fbf1fe00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:49:49.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.314+0000 7fbf1fe00640 1 -- 172.21.15.39:0/3963525339 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c038040 con 0x557c742753b0 2024-09-21T07:49:49.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.314+0000 7fbf1fe00640 1 -- 172.21.15.39:0/3963525339 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(233..233 src has 1..233) ==== 7016+0+0 (secure 0 0 0) 0x7fbf2c003a40 con 0x557c742753b0 2024-09-21T07:49:49.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.314+0000 7fbf48a62740 1 -- 172.21.15.39:0/3963525339 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf2c007b90 con 0x557c742753b0 2024-09-21T07:49:49.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.314+0000 7fbf1fe00640 1 -- 172.21.15.39:0/3963525339 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=250) ==== 24+0+0 (secure 0 0 0) 0x7fbf2c003c80 con 0x557c742753b0 2024-09-21T07:49:49.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.314+0000 7fbf48a62740 10 client.10812 fetch_fsmap learned FSMap version 250 2024-09-21T07:49:49.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.314+0000 7fbf48a62740 1 -- 172.21.15.39:0/3963525339 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=250,monmap=2+}) -- 0x557c743806b0 con 0x557c742753b0 2024-09-21T07:49:49.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.314+0000 7fbf1fe00640 1 -- 172.21.15.39:0/3963525339 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf2c007b90 con 0x557c742753b0 2024-09-21T07:49:49.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.314+0000 7fbf1fe00640 1 -- 172.21.15.39:0/3963525339 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 250) ==== 3542+0+0 (secure 0 0 0) 0x7fbf2c03b030 con 0x557c742753b0 2024-09-21T07:49:49.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.315+0000 7fbf48a62740 10 client.10812 fetch_fsmap finished waiting for FSMap version 250 2024-09-21T07:49:49.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.315+0000 7fbf48a62740 10 client.10812 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T07:49:49.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.315+0000 7fbf48a62740 20 client.10812 populate_metadata read hostname 'smithi039' 2024-09-21T07:49:49.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.315+0000 7fbf48a62740 1 --2- 172.21.15.39:0/3963525339 >> [v2:172.21.15.148:6834/3497801035,v1:172.21.15.148:6836/3497801035] conn(0x557c7425ce40 0x557c74698ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.315+0000 7fbf48a62740 4 client.10812 mds_command: new command op to 9709 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:49:49.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.315+0000 7fbf48a62740 1 -- 172.21.15.39:0/3963525339 --> [v2:172.21.15.148:6834/3497801035,v1:172.21.15.148:6836/3497801035] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c745ba520 con 0x557c7425ce40 2024-09-21T07:49:49.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.315+0000 7fbf33e00640 1 --2- 172.21.15.39:0/3963525339 >> [v2:172.21.15.148:6834/3497801035,v1:172.21.15.148:6836/3497801035] conn(0x557c7425ce40 0x557c74698ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.316+0000 7fbf33e00640 1 --2- 172.21.15.39:0/3963525339 >> [v2:172.21.15.148:6834/3497801035,v1:172.21.15.148:6836/3497801035] conn(0x557c7425ce40 0x557c74698ac0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6179a58b6ce1a768 server_cookie=f12da7a79400f671 in_seq=0 out_seq=0 2024-09-21T07:49:49.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.316+0000 7fbf1fe00640 10 client.10812 ms_handle_connect on v2:172.21.15.148:6834/3497801035 2024-09-21T07:49:49.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.317+0000 7fbf1fe00640 1 -- 172.21.15.39:0/3963525339 <== mds.2 v2:172.21.15.148:6834/3497801035 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557c745ba520 con 0x557c7425ce40 2024-09-21T07:49:49.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.317+0000 7fbf1fe00640 10 client.10812 handle_command_reply: tid=0 2024-09-21T07:49:49.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.317+0000 7fbf48a62740 1 client.10812 shutdown 2024-09-21T07:49:49.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.317+0000 7fbf48a62740 1 -- 172.21.15.39:0/3963525339 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c742753b0 msgr2=0x557c74689fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:49.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.317+0000 7fbf48a62740 1 --2- 172.21.15.39:0/3963525339 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c742753b0 0x557c74689fb0 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c006010 tx=0x7fbf2c004600 comp rx=0 tx=0).stop 2024-09-21T07:49:49.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.317+0000 7fbf48a62740 1 -- 172.21.15.39:0/3963525339 shutdown_connections 2024-09-21T07:49:49.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.317+0000 7fbf48a62740 1 --2- 172.21.15.39:0/3963525339 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c7429e270 0x557c74667240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.317+0000 7fbf48a62740 1 --2- 172.21.15.39:0/3963525339 >> [v2:172.21.15.148:6834/3497801035,v1:172.21.15.148:6836/3497801035] conn(0x557c7425ce40 0x557c74698ac0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.317+0000 7fbf48a62740 1 --2- 172.21.15.39:0/3963525339 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557c7427d590 0x557c74664e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.317+0000 7fbf48a62740 1 --2- 172.21.15.39:0/3963525339 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c742753b0 0x557c74689fb0 unknown :-1 s=CLOSED pgs=2266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.317+0000 7fbf48a62740 1 -- 172.21.15.39:0/3963525339 >> 172.21.15.39:0/3963525339 conn(0x557c742923c0 msgr2=0x557c7467fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:49.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.317+0000 7fbf1fe00640 0 client.10812 ms_handle_reset on v2:172.21.15.148:6834/3497801035 2024-09-21T07:49:49.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.317+0000 7fbf48a62740 1 -- 172.21.15.39:0/3963525339 shutdown_connections 2024-09-21T07:49:49.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.317+0000 7fbf48a62740 1 -- 172.21.15.39:0/3963525339 wait complete. 2024-09-21T07:49:49.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.317+0000 7fbf48a62740 20 client.10812 trim_cache size 0 max 16384 2024-09-21T07:49:49.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.337+0000 7fbf48a62740 1 Processor -- start 2024-09-21T07:49:49.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.337+0000 7fbf48a62740 1 -- start start 2024-09-21T07:49:49.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.338+0000 7fbf48a62740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557c742753b0 0x557c74664e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.338+0000 7fbf48a62740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c7427d590 0x557c7463a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.338+0000 7fbf48a62740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c7429e270 0x557c747031c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.338+0000 7fbf48a62740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c203f10 con 0x557c7427d590 2024-09-21T07:49:49.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.338+0000 7fbf48a62740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c2045e0 con 0x557c7429e270 2024-09-21T07:49:49.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.338+0000 7fbf48a62740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fbf3c202f80 con 0x557c742753b0 2024-09-21T07:49:49.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.338+0000 7fbf33e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c7427d590 0x557c7463a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.338+0000 7fbf33e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c7427d590 0x557c7463a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43766/0 (socket says 172.21.15.39:43766) 2024-09-21T07:49:49.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.338+0000 7fbf33e00640 1 -- 172.21.15.39:0/3155068944 learned_addr learned my addr 172.21.15.39:0/3155068944 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:49.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.338+0000 7fbf3aa00640 1 --2- 172.21.15.39:0/3155068944 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557c742753b0 0x557c74664e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.339+0000 7fbf3b400640 1 --2- 172.21.15.39:0/3155068944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c7429e270 0x557c747031c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.339+0000 7fbf33e00640 1 -- 172.21.15.39:0/3155068944 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557c742753b0 msgr2=0x557c74664e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:49.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.339+0000 7fbf33e00640 1 --2- 172.21.15.39:0/3155068944 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557c742753b0 0x557c74664e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.339+0000 7fbf33e00640 1 -- 172.21.15.39:0/3155068944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c7429e270 msgr2=0x557c747031c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:49.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.339+0000 7fbf33e00640 1 --2- 172.21.15.39:0/3155068944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c7429e270 0x557c747031c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.339+0000 7fbf33e00640 1 -- 172.21.15.39:0/3155068944 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c74645650 con 0x557c7427d590 2024-09-21T07:49:49.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.339+0000 7fbf3aa00640 1 --2- 172.21.15.39:0/3155068944 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557c742753b0 0x557c74664e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:49:49.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.339+0000 7fbf33e00640 1 --2- 172.21.15.39:0/3155068944 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c7427d590 0x557c7463a450 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c1514b0 tx=0x7fbf24008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeacaa0ad8f08f23 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:49.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.339+0000 7fbf2aa00640 1 -- 172.21.15.39:0/3155068944 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf24017b60 con 0x557c7427d590 2024-09-21T07:49:49.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.340+0000 7fbf2aa00640 1 -- 172.21.15.39:0/3155068944 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf24017d00 con 0x557c7427d590 2024-09-21T07:49:49.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.340+0000 7fbf2aa00640 1 -- 172.21.15.39:0/3155068944 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf240255e0 con 0x557c7427d590 2024-09-21T07:49:49.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.340+0000 7fbf48a62740 1 -- 172.21.15.39:0/3155068944 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c7427d590 msgr2=0x557c7463a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:49.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.340+0000 7fbf48a62740 1 --2- 172.21.15.39:0/3155068944 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c7427d590 0x557c7463a450 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c1514b0 tx=0x7fbf24008040 comp rx=0 tx=0).stop 2024-09-21T07:49:49.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.340+0000 7fbf48a62740 1 -- 172.21.15.39:0/3155068944 shutdown_connections 2024-09-21T07:49:49.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.340+0000 7fbf48a62740 1 --2- 172.21.15.39:0/3155068944 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c7429e270 0x557c747031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.340+0000 7fbf48a62740 1 --2- 172.21.15.39:0/3155068944 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c7427d590 0x557c7463a450 unknown :-1 s=CLOSED pgs=2267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.340+0000 7fbf48a62740 1 --2- 172.21.15.39:0/3155068944 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557c742753b0 0x557c74664e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.340+0000 7fbf48a62740 1 -- 172.21.15.39:0/3155068944 >> 172.21.15.39:0/3155068944 conn(0x557c742923c0 msgr2=0x557c74689770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:49.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.340+0000 7fbf48a62740 1 -- 172.21.15.39:0/3155068944 shutdown_connections 2024-09-21T07:49:49.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.340+0000 7fbf48a62740 1 -- 172.21.15.39:0/3155068944 wait complete. 2024-09-21T07:49:49.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.341+0000 7fbf48a62740 1 -- start start 2024-09-21T07:49:49.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.341+0000 7fbf48a62740 1 Processor -- start 2024-09-21T07:49:49.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.343+0000 7fbf48a62740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c742753b0 0x557c746ffc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.343+0000 7fbf48a62740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c7427d590 0x557c7469b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.343+0000 7fbf48a62740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557c7429e270 0x557c7468cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.343+0000 7fbf48a62740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557c74667d40 con 0x557c742753b0 2024-09-21T07:49:49.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.343+0000 7fbf48a62740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557c74665370 con 0x557c7427d590 2024-09-21T07:49:49.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.343+0000 7fbf48a62740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x557c7469c3c0 con 0x557c7429e270 2024-09-21T07:49:49.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.343+0000 7fbf48a62740 10 client.0 _ll_register_callbacks cb 0x7fbf2c004420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:49:49.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.343+0000 7fbf3aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c742753b0 0x557c746ffc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.343+0000 7fbf3aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c742753b0 0x557c746ffc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43774/0 (socket says 172.21.15.39:43774) 2024-09-21T07:49:49.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.343+0000 7fbf3aa00640 1 -- 172.21.15.39:0/4152848265 learned_addr learned my addr 172.21.15.39:0/4152848265 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:49.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.344+0000 7fbf3aa00640 1 -- 172.21.15.39:0/4152848265 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557c7429e270 msgr2=0x557c7468cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:49.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.344+0000 7fbf3aa00640 1 --2- 172.21.15.39:0/4152848265 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557c7429e270 0x557c7468cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.344+0000 7fbf3aa00640 1 -- 172.21.15.39:0/4152848265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c7427d590 msgr2=0x557c7469b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:49.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.344+0000 7fbf3aa00640 1 --2- 172.21.15.39:0/4152848265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c7427d590 0x557c7469b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.344+0000 7fbf3aa00640 1 -- 172.21.15.39:0/4152848265 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c7470ce10 con 0x557c742753b0 2024-09-21T07:49:49.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.344+0000 7fbf3aa00640 1 --2- 172.21.15.39:0/4152848265 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c742753b0 0x557c746ffc20 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c006010 tx=0x7fbf2c005650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad339cc97a66fae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:49.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.344+0000 7fbf1fe00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:49:49.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.344+0000 7fbf1fe00640 1 -- 172.21.15.39:0/4152848265 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c043060 con 0x557c742753b0 2024-09-21T07:49:49.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.344+0000 7fbf1fe00640 1 -- 172.21.15.39:0/4152848265 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(233..233 src has 1..233) ==== 7016+0+0 (secure 0 0 0) 0x7fbf2c003a40 con 0x557c742753b0 2024-09-21T07:49:49.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.344+0000 7fbf48a62740 1 -- 172.21.15.39:0/4152848265 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf24017d00 con 0x557c742753b0 2024-09-21T07:49:49.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.344+0000 7fbf1fe00640 1 -- 172.21.15.39:0/4152848265 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=250) ==== 24+0+0 (secure 0 0 0) 0x7fbf2c064050 con 0x557c742753b0 2024-09-21T07:49:49.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.344+0000 7fbf48a62740 10 client.10818 fetch_fsmap learned FSMap version 250 2024-09-21T07:49:49.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.344+0000 7fbf48a62740 1 -- 172.21.15.39:0/4152848265 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=250,monmap=2+}) -- 0x557c746ac9c0 con 0x557c742753b0 2024-09-21T07:49:49.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.344+0000 7fbf1fe00640 1 -- 172.21.15.39:0/4152848265 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf24017d00 con 0x557c742753b0 2024-09-21T07:49:49.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.345+0000 7fbf1fe00640 1 -- 172.21.15.39:0/4152848265 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 250) ==== 3542+0+0 (secure 0 0 0) 0x7fbf2c067020 con 0x557c742753b0 2024-09-21T07:49:49.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.345+0000 7fbf48a62740 10 client.10818 fetch_fsmap finished waiting for FSMap version 250 2024-09-21T07:49:49.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.345+0000 7fbf48a62740 10 client.10818 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T07:49:49.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.345+0000 7fbf48a62740 20 client.10818 populate_metadata read hostname 'smithi039' 2024-09-21T07:49:49.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.345+0000 7fbf48a62740 1 --2- 172.21.15.39:0/4152848265 >> [v2:172.21.15.148:6834/3497801035,v1:172.21.15.148:6836/3497801035] conn(0x557c74666790 0x557c746ab940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.345+0000 7fbf48a62740 4 client.10818 mds_command: new command op to 9709 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:49:49.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.345+0000 7fbf48a62740 1 -- 172.21.15.39:0/4152848265 --> [v2:172.21.15.148:6834/3497801035,v1:172.21.15.148:6836/3497801035] -- command(tid 0: {"prefix": "status"}) -- 0x557c745b5fd0 con 0x557c74666790 2024-09-21T07:49:49.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.345+0000 7fbf33e00640 1 --2- 172.21.15.39:0/4152848265 >> [v2:172.21.15.148:6834/3497801035,v1:172.21.15.148:6836/3497801035] conn(0x557c74666790 0x557c746ab940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.346+0000 7fbf33e00640 1 --2- 172.21.15.39:0/4152848265 >> [v2:172.21.15.148:6834/3497801035,v1:172.21.15.148:6836/3497801035] conn(0x557c74666790 0x557c746ab940 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=88c2baadadd3167a server_cookie=a8a6bf28140773bf in_seq=0 out_seq=0 2024-09-21T07:49:49.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.346+0000 7fbf1fe00640 10 client.10818 ms_handle_connect on v2:172.21.15.148:6834/3497801035 2024-09-21T07:49:49.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.346+0000 7fbf1fe00640 1 -- 172.21.15.39:0/4152848265 <== mds.2 v2:172.21.15.148:6834/3497801035 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x557c745b5fd0 con 0x557c74666790 2024-09-21T07:49:49.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.346+0000 7fbf1fe00640 10 client.10818 handle_command_reply: tid=0 2024-09-21T07:49:49.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.346+0000 7fbf48a62740 1 client.10818 shutdown 2024-09-21T07:49:49.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.346+0000 7fbf48a62740 1 -- 172.21.15.39:0/4152848265 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c742753b0 msgr2=0x557c746ffc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:49.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.346+0000 7fbf48a62740 1 --2- 172.21.15.39:0/4152848265 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c742753b0 0x557c746ffc20 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c006010 tx=0x7fbf2c005650 comp rx=0 tx=0).stop 2024-09-21T07:49:49.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.346+0000 7fbf48a62740 1 -- 172.21.15.39:0/4152848265 shutdown_connections 2024-09-21T07:49:49.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.346+0000 7fbf48a62740 1 --2- 172.21.15.39:0/4152848265 >> [v2:172.21.15.148:6834/3497801035,v1:172.21.15.148:6836/3497801035] conn(0x557c74666790 0x557c746ab940 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.346+0000 7fbf48a62740 1 --2- 172.21.15.39:0/4152848265 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557c7429e270 0x557c7468cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.346+0000 7fbf48a62740 1 --2- 172.21.15.39:0/4152848265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c7427d590 0x557c7469b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.346+0000 7fbf48a62740 1 --2- 172.21.15.39:0/4152848265 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557c742753b0 0x557c746ffc20 unknown :-1 s=CLOSED pgs=2268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.346+0000 7fbf1fe00640 0 client.10818 ms_handle_reset on v2:172.21.15.148:6834/3497801035 2024-09-21T07:49:49.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.346+0000 7fbf48a62740 1 -- 172.21.15.39:0/4152848265 >> 172.21.15.39:0/4152848265 conn(0x557c742923c0 msgr2=0x557c7463a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:49.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.347+0000 7fbf48a62740 1 -- 172.21.15.39:0/4152848265 shutdown_connections 2024-09-21T07:49:49.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.347+0000 7fbf48a62740 1 -- 172.21.15.39:0/4152848265 wait complete. 2024-09-21T07:49:49.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.347+0000 7fbf48a62740 20 client.10818 trim_cache size 0 max 16384 2024-09-21T07:49:49.351 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:49:49.351 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:49:49.351 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 2, 2024-09-21T07:49:49.351 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 9709, 2024-09-21T07:49:49.351 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:49:49.352 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:49:49.352 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:49:49.352 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 2.111164644, 2024-09-21T07:49:49.352 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 250, 2024-09-21T07:49:49.352 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 233, 2024-09-21T07:49:49.352 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 233, 2024-09-21T07:49:49.352 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 19.235871175 2024-09-21T07:49:49.352 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:49:49.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.347+0000 7fbf41a00640 1 -- 172.21.15.39:0/3805105569 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbf18078b90 msgr2=0x7fbf1807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:49.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.347+0000 7fbf41a00640 1 --2- 172.21.15.39:0/3805105569 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbf18078b90 0x7fbf1807b0a0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c033000 tx=0x7fbf2c035040 comp rx=0 tx=0).stop 2024-09-21T07:49:49.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.347+0000 7fbf41a00640 1 -- 172.21.15.39:0/3805105569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf3c151640 msgr2=0x7fbf3c147e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:49.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.347+0000 7fbf41a00640 1 --2- 172.21.15.39:0/3805105569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf3c151640 0x7fbf3c147e00 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7fbf3400b9e0 tx=0x7fbf3400bea0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.347+0000 7fbf41a00640 1 -- 172.21.15.39:0/3805105569 shutdown_connections 2024-09-21T07:49:49.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.347+0000 7fbf41a00640 1 --2- 172.21.15.39:0/3805105569 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbf18078b90 0x7fbf1807b0a0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.347+0000 7fbf41a00640 1 --2- 172.21.15.39:0/3805105569 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf3c151640 0x7fbf3c147e00 unknown :-1 s=CLOSED pgs=2264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.347+0000 7fbf41a00640 1 --2- 172.21.15.39:0/3805105569 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbf3c0cf750 0x7fbf3c1f98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.347+0000 7fbf41a00640 1 --2- 172.21.15.39:0/3805105569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf3c070570 0x7fbf3c15d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.347+0000 7fbf41a00640 1 -- 172.21.15.39:0/3805105569 >> 172.21.15.39:0/3805105569 conn(0x7fbf3c06cfc0 msgr2=0x7fbf3c0cfb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:49.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.347+0000 7fbf41a00640 1 -- 172.21.15.39:0/3805105569 shutdown_connections 2024-09-21T07:49:49.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.348+0000 7fbf41a00640 1 -- 172.21.15.39:0/3805105569 wait complete. 2024-09-21T07:49:49.370 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T07:49:49.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.595+0000 7f6b8f000640 1 -- 172.21.15.39:0/1500983300 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b88160110 msgr2=0x7f6b880ccb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:49.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.595+0000 7f6b8f000640 1 --2- 172.21.15.39:0/1500983300 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b88160110 0x7f6b880ccb60 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f6b88075250 tx=0x7f6b7c02a4d0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.596+0000 7f6b8f000640 1 -- 172.21.15.39:0/1500983300 shutdown_connections 2024-09-21T07:49:49.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.596+0000 7f6b8f000640 1 --2- 172.21.15.39:0/1500983300 >> v2:172.21.15.148:3300/0 conn(0x7f6b880d0430 0x7f6b880d0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.596+0000 7f6b8f000640 1 --2- 172.21.15.39:0/1500983300 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b88160110 0x7f6b880ccb60 unknown :-1 s=CLOSED pgs=2269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.596+0000 7f6b8f000640 1 -- 172.21.15.39:0/1500983300 >> 172.21.15.39:0/1500983300 conn(0x7f6b8806cfc0 msgr2=0x7f6b8806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:49.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.596+0000 7f6b8f000640 1 -- 172.21.15.39:0/1500983300 shutdown_connections 2024-09-21T07:49:49.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.596+0000 7f6b8f000640 1 -- 172.21.15.39:0/1500983300 wait complete. 2024-09-21T07:49:49.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.598+0000 7f6b8f000640 1 Processor -- start 2024-09-21T07:49:49.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.598+0000 7f6b8f000640 1 -- start start 2024-09-21T07:49:49.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.599+0000 7f6b8f000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b880cf5e0 0x7f6b881429e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.599+0000 7f6b8f000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b880d0430 0x7f6b8806ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.599+0000 7f6b8f000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b88160110 0x7f6b8806f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.599+0000 7f6b8f000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8815d440 con 0x7f6b880d0430 2024-09-21T07:49:49.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.599+0000 7f6b8f000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8815d2c0 con 0x7f6b880cf5e0 2024-09-21T07:49:49.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.599+0000 7f6b8f000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f6b8815d5c0 con 0x7f6b88160110 2024-09-21T07:49:49.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.600+0000 7f6b8d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b880d0430 0x7f6b8806ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.600+0000 7f6b8d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b880d0430 0x7f6b8806ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43806/0 (socket says 172.21.15.39:43806) 2024-09-21T07:49:49.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.600+0000 7f6b8d200640 1 -- 172.21.15.39:0/3113445418 learned_addr learned my addr 172.21.15.39:0/3113445418 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:49.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.600+0000 7f6b8e600640 1 --2- 172.21.15.39:0/3113445418 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b88160110 0x7f6b8806f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.600+0000 7f6b8d200640 1 -- 172.21.15.39:0/3113445418 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b88160110 msgr2=0x7f6b8806f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:49.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.600+0000 7f6b8d200640 1 --2- 172.21.15.39:0/3113445418 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b88160110 0x7f6b8806f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.600+0000 7f6b8d200640 1 -- 172.21.15.39:0/3113445418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b880cf5e0 msgr2=0x7f6b881429e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:49.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.600+0000 7f6b8d200640 1 --2- 172.21.15.39:0/3113445418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b880cf5e0 0x7f6b881429e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.600+0000 7f6b8d200640 1 -- 172.21.15.39:0/3113445418 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b88158e60 con 0x7f6b880d0430 2024-09-21T07:49:49.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.600+0000 7f6b8e600640 1 --2- 172.21.15.39:0/3113445418 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b88160110 0x7f6b8806f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:49:49.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.600+0000 7f6b8d200640 1 --2- 172.21.15.39:0/3113445418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b880d0430 0x7f6b8806ecd0 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7f6b7400c970 tx=0x7f6b7400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:49.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.601+0000 7f6b85600640 1 -- 172.21.15.39:0/3113445418 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b74007bb0 con 0x7f6b880d0430 2024-09-21T07:49:49.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.601+0000 7f6b85600640 1 -- 172.21.15.39:0/3113445418 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b74007e40 con 0x7f6b880d0430 2024-09-21T07:49:49.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.601+0000 7f6b85600640 1 -- 172.21.15.39:0/3113445418 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7400f6b0 con 0x7f6b880d0430 2024-09-21T07:49:49.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.601+0000 7f6b8f000640 1 -- 172.21.15.39:0/3113445418 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b881f6bc0 con 0x7f6b880d0430 2024-09-21T07:49:49.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.601+0000 7f6b8f000640 1 -- 172.21.15.39:0/3113445418 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b881f7180 con 0x7f6b880d0430 2024-09-21T07:49:49.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.603+0000 7f6b85600640 1 -- 172.21.15.39:0/3113445418 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6b7400f850 con 0x7f6b880d0430 2024-09-21T07:49:49.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.604+0000 7f6b85600640 1 --2- 172.21.15.39:0/3113445418 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6b68078b90 0x7f6b6807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.604+0000 7f6b85600640 1 -- 172.21.15.39:0/3113445418 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(233..233 src has 1..233) ==== 7016+0+0 (secure 0 0 0) 0x7f6b7409bab0 con 0x7f6b880d0430 2024-09-21T07:49:49.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.605+0000 7f6b976c6740 1 Processor -- start 2024-09-21T07:49:49.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.605+0000 7f6b976c6740 1 -- start start 2024-09-21T07:49:49.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.605+0000 7f6b8dc00640 1 --2- 172.21.15.39:0/3113445418 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6b68078b90 0x7f6b6807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.606+0000 7f6b8dc00640 1 --2- 172.21.15.39:0/3113445418 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6b68078b90 0x7f6b6807b0a0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c033000 tx=0x7f6b7c0046c0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:49.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.606+0000 7f6b976c6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603d7a82270 0x5603d7e52b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.606+0000 7f6b976c6740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603d7a593b0 0x5603d7e530a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.606+0000 7f6b976c6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603d7a61590 0x5603d7e23150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.606+0000 7f6b976c6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5603d7dac430 con 0x5603d7a82270 2024-09-21T07:49:49.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.607+0000 7f6b976c6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5603d7d9c5d0 con 0x5603d7a61590 2024-09-21T07:49:49.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.607+0000 7f6b976c6740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x5603d7d8ba00 con 0x5603d7a593b0 2024-09-21T07:49:49.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.607+0000 7f6b8dc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603d7a82270 0x5603d7e52b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.607+0000 7f6b8dc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603d7a82270 0x5603d7e52b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43820/0 (socket says 172.21.15.39:43820) 2024-09-21T07:49:49.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.607+0000 7f6b8dc00640 1 -- 172.21.15.39:0/3466130935 learned_addr learned my addr 172.21.15.39:0/3466130935 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:49.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.607+0000 7f6b8e600640 1 --2- 172.21.15.39:0/3466130935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603d7a61590 0x5603d7e23150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.607+0000 7f6b8d200640 1 --2- 172.21.15.39:0/3466130935 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603d7a593b0 0x5603d7e530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.607+0000 7f6b8dc00640 1 -- 172.21.15.39:0/3466130935 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603d7a593b0 msgr2=0x5603d7e530a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:49.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.607+0000 7f6b8dc00640 1 --2- 172.21.15.39:0/3466130935 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603d7a593b0 0x5603d7e530a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.607+0000 7f6b8dc00640 1 -- 172.21.15.39:0/3466130935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603d7a61590 msgr2=0x5603d7e23150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:49.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.607+0000 7f6b8dc00640 1 --2- 172.21.15.39:0/3466130935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603d7a61590 0x5603d7e23150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.607+0000 7f6b8dc00640 1 -- 172.21.15.39:0/3466130935 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603d7d54520 con 0x5603d7a82270 2024-09-21T07:49:49.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.607+0000 7f6b8e600640 1 --2- 172.21.15.39:0/3466130935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603d7a61590 0x5603d7e23150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:49:49.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.607+0000 7f6b8d200640 1 --2- 172.21.15.39:0/3466130935 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603d7a593b0 0x5603d7e530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:49:49.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.608+0000 7f6b8dc00640 1 --2- 172.21.15.39:0/3466130935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603d7a82270 0x5603d7e52b60 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c02aac0 tx=0x7f6b7c03a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=517f89e8d271728c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:49.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.608+0000 7f6b7a000640 1 -- 172.21.15.39:0/3466130935 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c035040 con 0x5603d7a82270 2024-09-21T07:49:49.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.608+0000 7f6b7a000640 1 -- 172.21.15.39:0/3466130935 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b7c04b070 con 0x5603d7a82270 2024-09-21T07:49:49.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.608+0000 7f6b7a000640 1 -- 172.21.15.39:0/3466130935 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c003c40 con 0x5603d7a82270 2024-09-21T07:49:49.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.609+0000 7f6b976c6740 1 -- 172.21.15.39:0/3466130935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603d7a82270 msgr2=0x5603d7e52b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:49.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.609+0000 7f6b976c6740 1 --2- 172.21.15.39:0/3466130935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603d7a82270 0x5603d7e52b60 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c02aac0 tx=0x7f6b7c03a000 comp rx=0 tx=0).stop 2024-09-21T07:49:49.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.609+0000 7f6b976c6740 1 -- 172.21.15.39:0/3466130935 shutdown_connections 2024-09-21T07:49:49.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.609+0000 7f6b976c6740 1 --2- 172.21.15.39:0/3466130935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603d7a61590 0x5603d7e23150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.609+0000 7f6b976c6740 1 --2- 172.21.15.39:0/3466130935 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603d7a593b0 0x5603d7e530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.609+0000 7f6b976c6740 1 --2- 172.21.15.39:0/3466130935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603d7a82270 0x5603d7e52b60 unknown :-1 s=CLOSED pgs=2271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.609+0000 7f6b976c6740 1 -- 172.21.15.39:0/3466130935 >> 172.21.15.39:0/3466130935 conn(0x5603d7a763c0 msgr2=0x5603d7e28b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:49.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.609+0000 7f6b976c6740 1 -- 172.21.15.39:0/3466130935 shutdown_connections 2024-09-21T07:49:49.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.609+0000 7f6b976c6740 1 -- 172.21.15.39:0/3466130935 wait complete. 2024-09-21T07:49:49.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.610+0000 7f6b976c6740 1 -- start start 2024-09-21T07:49:49.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.610+0000 7f6b976c6740 1 Processor -- start 2024-09-21T07:49:49.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.612+0000 7f6b976c6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603d7a593b0 0x5603d7e24030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.612+0000 7f6b976c6740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603d7a61590 0x5603d7e4c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.612+0000 7f6b976c6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603d7a82270 0x5603d7e489e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.612+0000 7f6b976c6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5603d7d32330 con 0x5603d7a82270 2024-09-21T07:49:49.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.612+0000 7f6b976c6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5603d7d8f810 con 0x5603d7a593b0 2024-09-21T07:49:49.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.612+0000 7f6b976c6740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x5603d7afc620 con 0x5603d7a61590 2024-09-21T07:49:49.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.612+0000 7f6b976c6740 10 client.0 _ll_register_callbacks cb 0x5603d7d37420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:49:49.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.612+0000 7f6b8e600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603d7a82270 0x5603d7e489e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.612+0000 7f6b8e600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603d7a82270 0x5603d7e489e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43826/0 (socket says 172.21.15.39:43826) 2024-09-21T07:49:49.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.612+0000 7f6b8e600640 1 -- 172.21.15.39:0/1238597581 learned_addr learned my addr 172.21.15.39:0/1238597581 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:49.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.612+0000 7f6b8d200640 1 --2- 172.21.15.39:0/1238597581 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603d7a61590 0x5603d7e4c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.612+0000 7f6b8dc00640 1 --2- 172.21.15.39:0/1238597581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603d7a593b0 0x5603d7e24030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.612+0000 7f6b8e600640 1 -- 172.21.15.39:0/1238597581 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603d7a61590 msgr2=0x5603d7e4c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:49.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.612+0000 7f6b8e600640 1 --2- 172.21.15.39:0/1238597581 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603d7a61590 0x5603d7e4c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.613+0000 7f6b8e600640 1 -- 172.21.15.39:0/1238597581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603d7a593b0 msgr2=0x5603d7e24030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:49.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.613+0000 7f6b8e600640 1 --2- 172.21.15.39:0/1238597581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603d7a593b0 0x5603d7e24030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.613+0000 7f6b8e600640 1 -- 172.21.15.39:0/1238597581 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603d7aed850 con 0x5603d7a82270 2024-09-21T07:49:49.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.613+0000 7f6b8e600640 1 --2- 172.21.15.39:0/1238597581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603d7a82270 0x5603d7e489e0 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7f6b8000d940 tx=0x7f6b8000de00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6cf17d47111fd58 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:49.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.613+0000 7f6b6ea00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:49:49.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.613+0000 7f6b6ea00640 1 -- 172.21.15.39:0/1238597581 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8000bd40 con 0x5603d7a82270 2024-09-21T07:49:49.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.613+0000 7f6b6ea00640 1 -- 172.21.15.39:0/1238597581 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(233..233 src has 1..233) ==== 7016+0+0 (secure 0 0 0) 0x7f6b80017420 con 0x5603d7a82270 2024-09-21T07:49:49.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.613+0000 7f6b976c6740 1 -- 172.21.15.39:0/1238597581 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b7c04b070 con 0x5603d7a82270 2024-09-21T07:49:49.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.613+0000 7f6b6ea00640 1 -- 172.21.15.39:0/1238597581 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=250) ==== 24+0+0 (secure 0 0 0) 0x7f6b80021050 con 0x5603d7a82270 2024-09-21T07:49:49.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.613+0000 7f6b976c6740 10 client.10830 fetch_fsmap learned FSMap version 250 2024-09-21T07:49:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.613+0000 7f6b976c6740 1 -- 172.21.15.39:0/1238597581 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=250,monmap=2+}) -- 0x5603d7b646b0 con 0x5603d7a82270 2024-09-21T07:49:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.614+0000 7f6b6ea00640 1 -- 172.21.15.39:0/1238597581 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b7c04b070 con 0x5603d7a82270 2024-09-21T07:49:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.614+0000 7f6b6ea00640 1 -- 172.21.15.39:0/1238597581 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 250) ==== 3542+0+0 (secure 0 0 0) 0x7f6b80024020 con 0x5603d7a82270 2024-09-21T07:49:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.614+0000 7f6b976c6740 10 client.10830 fetch_fsmap finished waiting for FSMap version 250 2024-09-21T07:49:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.614+0000 7f6b976c6740 10 client.10830 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:49:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.614+0000 7f6b976c6740 20 client.10830 populate_metadata read hostname 'smithi039' 2024-09-21T07:49:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.614+0000 7f6b976c6740 1 --2- 172.21.15.39:0/1238597581 >> [v2:172.21.15.39:6832/4143807732,v1:172.21.15.39:6834/4143807732] conn(0x5603d7a40e40 0x5603d7e65ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.614+0000 7f6b976c6740 4 client.10830 mds_command: new command op to 10632 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:49:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.614+0000 7f6b976c6740 1 -- 172.21.15.39:0/1238597581 --> [v2:172.21.15.39:6832/4143807732,v1:172.21.15.39:6834/4143807732] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603d7d9e520 con 0x5603d7a40e40 2024-09-21T07:49:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.614+0000 7f6b8dc00640 1 --2- 172.21.15.39:0/1238597581 >> [v2:172.21.15.39:6832/4143807732,v1:172.21.15.39:6834/4143807732] conn(0x5603d7a40e40 0x5603d7e65ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.615+0000 7f6b6ea00640 10 client.10830 ms_handle_connect on v2:172.21.15.39:6832/4143807732 2024-09-21T07:49:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.615+0000 7f6b8dc00640 1 --2- 172.21.15.39:0/1238597581 >> [v2:172.21.15.39:6832/4143807732,v1:172.21.15.39:6834/4143807732] conn(0x5603d7a40e40 0x5603d7e65ba0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=870210a5151f532 server_cookie=c8bf65b9f864f42c in_seq=0 out_seq=0 2024-09-21T07:49:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.615+0000 7f6b6ea00640 1 -- 172.21.15.39:0/1238597581 <== mds.1 v2:172.21.15.39:6832/4143807732 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5603d7d9e520 con 0x5603d7a40e40 2024-09-21T07:49:49.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.615+0000 7f6b6ea00640 10 client.10830 handle_command_reply: tid=0 2024-09-21T07:49:49.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.616+0000 7f6b976c6740 1 client.10830 shutdown 2024-09-21T07:49:49.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.616+0000 7f6b976c6740 1 -- 172.21.15.39:0/1238597581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603d7a82270 msgr2=0x5603d7e489e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:49.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.616+0000 7f6b976c6740 1 --2- 172.21.15.39:0/1238597581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603d7a82270 0x5603d7e489e0 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7f6b8000d940 tx=0x7f6b8000de00 comp rx=0 tx=0).stop 2024-09-21T07:49:49.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.616+0000 7f6b976c6740 1 -- 172.21.15.39:0/1238597581 shutdown_connections 2024-09-21T07:49:49.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.616+0000 7f6b976c6740 1 --2- 172.21.15.39:0/1238597581 >> [v2:172.21.15.39:6832/4143807732,v1:172.21.15.39:6834/4143807732] conn(0x5603d7a40e40 0x5603d7e65ba0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.616+0000 7f6b976c6740 1 --2- 172.21.15.39:0/1238597581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603d7a82270 0x5603d7e489e0 unknown :-1 s=CLOSED pgs=2272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.616+0000 7f6b976c6740 1 --2- 172.21.15.39:0/1238597581 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603d7a61590 0x5603d7e4c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.616+0000 7f6b976c6740 1 --2- 172.21.15.39:0/1238597581 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603d7a593b0 0x5603d7e24030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.616+0000 7f6b976c6740 1 -- 172.21.15.39:0/1238597581 >> 172.21.15.39:0/1238597581 conn(0x5603d7a763c0 msgr2=0x5603d7e70630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:49.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.616+0000 7f6b6ea00640 0 client.10830 ms_handle_reset on v2:172.21.15.39:6832/4143807732 2024-09-21T07:49:49.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.616+0000 7f6b976c6740 1 -- 172.21.15.39:0/1238597581 shutdown_connections 2024-09-21T07:49:49.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.616+0000 7f6b976c6740 1 -- 172.21.15.39:0/1238597581 wait complete. 2024-09-21T07:49:49.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.616+0000 7f6b976c6740 20 client.10830 trim_cache size 0 max 16384 2024-09-21T07:49:49.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.635+0000 7f6b976c6740 1 Processor -- start 2024-09-21T07:49:49.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.635+0000 7f6b976c6740 1 -- start start 2024-09-21T07:49:49.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.637+0000 7f6b976c6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603d7a593b0 0x5603d7e7cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.637+0000 7f6b976c6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603d7a61590 0x5603d7e51680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.637+0000 7f6b976c6740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603d7a82270 0x5603d7e90730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.637+0000 7f6b976c6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6b88202870 con 0x5603d7a61590 2024-09-21T07:49:49.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.637+0000 7f6b976c6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6b88201ad0 con 0x5603d7a593b0 2024-09-21T07:49:49.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.637+0000 7f6b976c6740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f6b882021a0 con 0x5603d7a82270 2024-09-21T07:49:49.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.637+0000 7f6b8d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603d7a61590 0x5603d7e51680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.637+0000 7f6b8d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603d7a61590 0x5603d7e51680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43830/0 (socket says 172.21.15.39:43830) 2024-09-21T07:49:49.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.637+0000 7f6b8d200640 1 -- 172.21.15.39:0/4187239256 learned_addr learned my addr 172.21.15.39:0/4187239256 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:49.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.637+0000 7f6b8d200640 1 -- 172.21.15.39:0/4187239256 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603d7a82270 msgr2=0x5603d7e90730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:49.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.637+0000 7f6b8d200640 1 --2- 172.21.15.39:0/4187239256 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603d7a82270 0x5603d7e90730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.637+0000 7f6b8d200640 1 -- 172.21.15.39:0/4187239256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603d7a593b0 msgr2=0x5603d7e7cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:49.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.637+0000 7f6b8d200640 1 --2- 172.21.15.39:0/4187239256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603d7a593b0 0x5603d7e7cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.637+0000 7f6b8d200640 1 -- 172.21.15.39:0/4187239256 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603d7e7ff20 con 0x5603d7a61590 2024-09-21T07:49:49.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.638+0000 7f6b8d200640 1 --2- 172.21.15.39:0/4187239256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603d7a61590 0x5603d7e51680 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7f6b740631c0 tx=0x7f6b74063930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5dfa947064a11dd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:49.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.638+0000 7f6b79600640 1 -- 172.21.15.39:0/4187239256 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7401d070 con 0x5603d7a61590 2024-09-21T07:49:49.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.638+0000 7f6b79600640 1 -- 172.21.15.39:0/4187239256 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b74089030 con 0x5603d7a61590 2024-09-21T07:49:49.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.638+0000 7f6b79600640 1 -- 172.21.15.39:0/4187239256 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7406ca20 con 0x5603d7a61590 2024-09-21T07:49:49.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.639+0000 7f6b976c6740 1 -- 172.21.15.39:0/4187239256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603d7a61590 msgr2=0x5603d7e51680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:49.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.639+0000 7f6b976c6740 1 --2- 172.21.15.39:0/4187239256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603d7a61590 0x5603d7e51680 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7f6b740631c0 tx=0x7f6b74063930 comp rx=0 tx=0).stop 2024-09-21T07:49:49.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.639+0000 7f6b976c6740 1 -- 172.21.15.39:0/4187239256 shutdown_connections 2024-09-21T07:49:49.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.639+0000 7f6b976c6740 1 --2- 172.21.15.39:0/4187239256 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603d7a82270 0x5603d7e90730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.639+0000 7f6b976c6740 1 --2- 172.21.15.39:0/4187239256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603d7a61590 0x5603d7e51680 unknown :-1 s=CLOSED pgs=2273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.639+0000 7f6b976c6740 1 --2- 172.21.15.39:0/4187239256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603d7a593b0 0x5603d7e7cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.639+0000 7f6b976c6740 1 -- 172.21.15.39:0/4187239256 >> 172.21.15.39:0/4187239256 conn(0x5603d7a763c0 msgr2=0x5603d7e1e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:49.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.639+0000 7f6b976c6740 1 -- 172.21.15.39:0/4187239256 shutdown_connections 2024-09-21T07:49:49.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.639+0000 7f6b976c6740 1 -- 172.21.15.39:0/4187239256 wait complete. 2024-09-21T07:49:49.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.640+0000 7f6b976c6740 1 -- start start 2024-09-21T07:49:49.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.640+0000 7f6b976c6740 1 Processor -- start 2024-09-21T07:49:49.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.641+0000 7f6b976c6740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603d7a593b0 0x5603d7e65d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.641+0000 7f6b976c6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603d7a61590 0x5603d7e66250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.641+0000 7f6b976c6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603d7a82270 0x5603d7eed300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.641+0000 7f6b976c6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5603d7e80120 con 0x5603d7a61590 2024-09-21T07:49:49.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.641+0000 7f6b976c6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6b88200b50 con 0x5603d7a82270 2024-09-21T07:49:49.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.641+0000 7f6b976c6740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x5603d7e80710 con 0x5603d7a593b0 2024-09-21T07:49:49.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.641+0000 7f6b976c6740 10 client.0 _ll_register_callbacks cb 0x7f6b8000d540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:49:49.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.642+0000 7f6b8d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603d7a61590 0x5603d7e66250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.642+0000 7f6b8d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603d7a61590 0x5603d7e66250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43838/0 (socket says 172.21.15.39:43838) 2024-09-21T07:49:49.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.642+0000 7f6b8d200640 1 -- 172.21.15.39:0/561748232 learned_addr learned my addr 172.21.15.39:0/561748232 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:49.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.642+0000 7f6b8dc00640 1 --2- 172.21.15.39:0/561748232 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603d7a593b0 0x5603d7e65d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.642+0000 7f6b8e600640 1 --2- 172.21.15.39:0/561748232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603d7a82270 0x5603d7eed300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.642+0000 7f6b8d200640 1 -- 172.21.15.39:0/561748232 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603d7a593b0 msgr2=0x5603d7e65d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:49.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.642+0000 7f6b8d200640 1 --2- 172.21.15.39:0/561748232 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603d7a593b0 0x5603d7e65d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.642+0000 7f6b8d200640 1 -- 172.21.15.39:0/561748232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603d7a82270 msgr2=0x5603d7eed300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:49.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.642+0000 7f6b8d200640 1 --2- 172.21.15.39:0/561748232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603d7a82270 0x5603d7eed300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.642+0000 7f6b8d200640 1 -- 172.21.15.39:0/561748232 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603d7e911c0 con 0x5603d7a61590 2024-09-21T07:49:49.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.642+0000 7f6b8e600640 1 --2- 172.21.15.39:0/561748232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603d7a82270 0x5603d7eed300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:49:49.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.642+0000 7f6b8d200640 1 --2- 172.21.15.39:0/561748232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603d7a61590 0x5603d7e66250 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f6b740631c0 tx=0x7f6b7404ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc3665777da8ed6a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:49.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.642+0000 7f6b6ea00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:49:49.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.643+0000 7f6b6ea00640 1 -- 172.21.15.39:0/561748232 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b74082060 con 0x5603d7a61590 2024-09-21T07:49:49.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.643+0000 7f6b6ea00640 1 -- 172.21.15.39:0/561748232 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(233..233 src has 1..233) ==== 7016+0+0 (secure 0 0 0) 0x7f6b7400fd50 con 0x5603d7a61590 2024-09-21T07:49:49.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.643+0000 7f6b976c6740 1 -- 172.21.15.39:0/561748232 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b74089030 con 0x5603d7a61590 2024-09-21T07:49:49.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.643+0000 7f6b6ea00640 1 -- 172.21.15.39:0/561748232 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=250) ==== 24+0+0 (secure 0 0 0) 0x7f6b7401d070 con 0x5603d7a61590 2024-09-21T07:49:49.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.643+0000 7f6b976c6740 10 client.10836 fetch_fsmap learned FSMap version 250 2024-09-21T07:49:49.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.643+0000 7f6b976c6740 1 -- 172.21.15.39:0/561748232 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=250,monmap=2+}) -- 0x5603d7e4c630 con 0x5603d7a61590 2024-09-21T07:49:49.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.643+0000 7f6b6ea00640 1 -- 172.21.15.39:0/561748232 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b74089030 con 0x5603d7a61590 2024-09-21T07:49:49.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.643+0000 7f6b6ea00640 1 -- 172.21.15.39:0/561748232 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 250) ==== 3542+0+0 (secure 0 0 0) 0x7f6b74062490 con 0x5603d7a61590 2024-09-21T07:49:49.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.643+0000 7f6b976c6740 10 client.10836 fetch_fsmap finished waiting for FSMap version 250 2024-09-21T07:49:49.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.643+0000 7f6b976c6740 10 client.10836 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:49:49.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.643+0000 7f6b976c6740 20 client.10836 populate_metadata read hostname 'smithi039' 2024-09-21T07:49:49.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.643+0000 7f6b976c6740 1 --2- 172.21.15.39:0/561748232 >> [v2:172.21.15.39:6832/4143807732,v1:172.21.15.39:6834/4143807732] conn(0x5603d7e87140 0x5603d7e87550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.643+0000 7f6b976c6740 4 client.10836 mds_command: new command op to 10632 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:49:49.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.643+0000 7f6b976c6740 1 -- 172.21.15.39:0/561748232 --> [v2:172.21.15.39:6832/4143807732,v1:172.21.15.39:6834/4143807732] -- command(tid 0: {"prefix": "status"}) -- 0x5603d7d99fd0 con 0x5603d7e87140 2024-09-21T07:49:49.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.644+0000 7f6b8dc00640 1 --2- 172.21.15.39:0/561748232 >> [v2:172.21.15.39:6832/4143807732,v1:172.21.15.39:6834/4143807732] conn(0x5603d7e87140 0x5603d7e87550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.644+0000 7f6b6ea00640 10 client.10836 ms_handle_connect on v2:172.21.15.39:6832/4143807732 2024-09-21T07:49:49.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.644+0000 7f6b8dc00640 1 --2- 172.21.15.39:0/561748232 >> [v2:172.21.15.39:6832/4143807732,v1:172.21.15.39:6834/4143807732] conn(0x5603d7e87140 0x5603d7e87550 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9220c45e01f73e9b server_cookie=f5d22fe91d49e2a3 in_seq=0 out_seq=0 2024-09-21T07:49:49.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.644+0000 7f6b6ea00640 1 -- 172.21.15.39:0/561748232 <== mds.1 v2:172.21.15.39:6832/4143807732 1 ==== command_reply(tid 0: 0 ) ==== 8+0+335 (crc 0 0 0) 0x5603d7d99fd0 con 0x5603d7e87140 2024-09-21T07:49:49.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.644+0000 7f6b6ea00640 10 client.10836 handle_command_reply: tid=0 2024-09-21T07:49:49.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.644+0000 7f6b976c6740 1 client.10836 shutdown 2024-09-21T07:49:49.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.645+0000 7f6b976c6740 1 -- 172.21.15.39:0/561748232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603d7a61590 msgr2=0x5603d7e66250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:49.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.645+0000 7f6b976c6740 1 --2- 172.21.15.39:0/561748232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603d7a61590 0x5603d7e66250 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f6b740631c0 tx=0x7f6b7404ada0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.645+0000 7f6b976c6740 1 -- 172.21.15.39:0/561748232 shutdown_connections 2024-09-21T07:49:49.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.645+0000 7f6b976c6740 1 --2- 172.21.15.39:0/561748232 >> [v2:172.21.15.39:6832/4143807732,v1:172.21.15.39:6834/4143807732] conn(0x5603d7e87140 0x5603d7e87550 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.645+0000 7f6b976c6740 1 --2- 172.21.15.39:0/561748232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603d7a82270 0x5603d7eed300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.645+0000 7f6b976c6740 1 --2- 172.21.15.39:0/561748232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603d7a61590 0x5603d7e66250 unknown :-1 s=CLOSED pgs=2274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.645+0000 7f6b6ea00640 0 client.10836 ms_handle_reset on v2:172.21.15.39:6832/4143807732 2024-09-21T07:49:49.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.645+0000 7f6b976c6740 1 --2- 172.21.15.39:0/561748232 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603d7a593b0 0x5603d7e65d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.645+0000 7f6b976c6740 1 -- 172.21.15.39:0/561748232 >> 172.21.15.39:0/561748232 conn(0x5603d7a763c0 msgr2=0x5603d7e28b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:49.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.645+0000 7f6b976c6740 1 -- 172.21.15.39:0/561748232 shutdown_connections 2024-09-21T07:49:49.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.645+0000 7f6b976c6740 1 -- 172.21.15.39:0/561748232 wait complete. 2024-09-21T07:49:49.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.645+0000 7f6b976c6740 20 client.10836 trim_cache size 0 max 16384 2024-09-21T07:49:49.650 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:49:49.650 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:49:49.651 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 1, 2024-09-21T07:49:49.651 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 10632, 2024-09-21T07:49:49.651 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:49:49.651 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:49:49.651 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:49:49.651 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 3.4236767690000001, 2024-09-21T07:49:49.651 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 250, 2024-09-21T07:49:49.651 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 233, 2024-09-21T07:49:49.651 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 233, 2024-09-21T07:49:49.651 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 19.535658049999999 2024-09-21T07:49:49.651 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:49:49.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.646+0000 7f6b8f000640 1 -- 172.21.15.39:0/3113445418 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6b68078b90 msgr2=0x7f6b6807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:49.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.646+0000 7f6b8f000640 1 --2- 172.21.15.39:0/3113445418 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6b68078b90 0x7f6b6807b0a0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c033000 tx=0x7f6b7c0046c0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.646+0000 7f6b8f000640 1 -- 172.21.15.39:0/3113445418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b880d0430 msgr2=0x7f6b8806ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:49.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.646+0000 7f6b8f000640 1 --2- 172.21.15.39:0/3113445418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b880d0430 0x7f6b8806ecd0 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7f6b7400c970 tx=0x7f6b7400ce30 comp rx=0 tx=0).stop 2024-09-21T07:49:49.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.646+0000 7f6b8f000640 1 -- 172.21.15.39:0/3113445418 shutdown_connections 2024-09-21T07:49:49.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.646+0000 7f6b8f000640 1 --2- 172.21.15.39:0/3113445418 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6b68078b90 0x7f6b6807b0a0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.646+0000 7f6b8f000640 1 --2- 172.21.15.39:0/3113445418 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6b88160110 0x7f6b8806f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.646+0000 7f6b8f000640 1 --2- 172.21.15.39:0/3113445418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b880d0430 0x7f6b8806ecd0 unknown :-1 s=CLOSED pgs=2270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.646+0000 7f6b8f000640 1 --2- 172.21.15.39:0/3113445418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b880cf5e0 0x7f6b881429e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.646+0000 7f6b8f000640 1 -- 172.21.15.39:0/3113445418 >> 172.21.15.39:0/3113445418 conn(0x7f6b8806cfc0 msgr2=0x7f6b8815da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:49.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.646+0000 7f6b8f000640 1 -- 172.21.15.39:0/3113445418 shutdown_connections 2024-09-21T07:49:49.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.646+0000 7f6b8f000640 1 -- 172.21.15.39:0/3113445418 wait complete. 2024-09-21T07:49:49.669 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T07:49:49.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.895+0000 7feb7ec00640 1 -- 172.21.15.39:0/154597191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb5c004800 msgr2=0x7feb5c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:49.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.895+0000 7feb7ec00640 1 --2- 172.21.15.39:0/154597191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb5c004800 0x7feb5c004c10 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7feb780b5a80 tx=0x7feb5c005830 comp rx=0 tx=0).stop 2024-09-21T07:49:49.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.895+0000 7feb7cc00640 1 -- 172.21.15.39:0/154597191 reap_dead start 2024-09-21T07:49:49.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.896+0000 7feb7ec00640 1 -- 172.21.15.39:0/154597191 shutdown_connections 2024-09-21T07:49:49.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.896+0000 7feb7ec00640 1 -- 172.21.15.39:0/154597191 >> 172.21.15.39:0/154597191 conn(0x7feb7806cfc0 msgr2=0x7feb7806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:49.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.896+0000 7feb7ec00640 1 -- 172.21.15.39:0/154597191 shutdown_connections 2024-09-21T07:49:49.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.896+0000 7feb7ec00640 1 -- 172.21.15.39:0/154597191 wait complete. 2024-09-21T07:49:49.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.898+0000 7feb7ec00640 1 Processor -- start 2024-09-21T07:49:49.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.898+0000 7feb7ec00640 1 -- start start 2024-09-21T07:49:49.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.900+0000 7feb7ec00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb5c004800 0x7feb78156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.900+0000 7feb7ec00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb78156650 0x7feb780b1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.900+0000 7feb7ec00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb780b1c90 0x7feb7814fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.900+0000 7feb7ec00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7feb781656a0 con 0x7feb5c004800 2024-09-21T07:49:49.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.900+0000 7feb7ec00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7feb7815d830 con 0x7feb78156650 2024-09-21T07:49:49.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.900+0000 7feb7ec00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7feb78165820 con 0x7feb780b1c90 2024-09-21T07:49:49.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.900+0000 7feb77e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb5c004800 0x7feb78156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.900+0000 7feb77e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb5c004800 0x7feb78156110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43854/0 (socket says 172.21.15.39:43854) 2024-09-21T07:49:49.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.900+0000 7feb77e00640 1 -- 172.21.15.39:0/3779461427 learned_addr learned my addr 172.21.15.39:0/3779461427 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:49.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.900+0000 7feb6f400640 1 --2- 172.21.15.39:0/3779461427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb78156650 0x7feb780b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.900+0000 7feb7cc00640 1 --2- 172.21.15.39:0/3779461427 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb780b1c90 0x7feb7814fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.900+0000 7feb77e00640 1 -- 172.21.15.39:0/3779461427 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb780b1c90 msgr2=0x7feb7814fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:49.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.900+0000 7feb77e00640 1 --2- 172.21.15.39:0/3779461427 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb780b1c90 0x7feb7814fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.900+0000 7feb77e00640 1 -- 172.21.15.39:0/3779461427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb78156650 msgr2=0x7feb780b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:49.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.900+0000 7feb77e00640 1 --2- 172.21.15.39:0/3779461427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb78156650 0x7feb780b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.900+0000 7feb77e00640 1 -- 172.21.15.39:0/3779461427 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb781f6d00 con 0x7feb5c004800 2024-09-21T07:49:49.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.900+0000 7feb7cc00640 1 --2- 172.21.15.39:0/3779461427 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb780b1c90 0x7feb7814fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:49:49.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.900+0000 7feb6f400640 1 --2- 172.21.15.39:0/3779461427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb78156650 0x7feb780b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:49:49.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.901+0000 7feb77e00640 1 --2- 172.21.15.39:0/3779461427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb5c004800 0x7feb78156110 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7feb68004cc0 tx=0x7feb68001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:49.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.901+0000 7feb74c00640 1 -- 172.21.15.39:0/3779461427 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb680107d0 con 0x7feb5c004800 2024-09-21T07:49:49.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.901+0000 7feb74c00640 1 -- 172.21.15.39:0/3779461427 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb68016070 con 0x7feb5c004800 2024-09-21T07:49:49.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.901+0000 7feb7ec00640 1 -- 172.21.15.39:0/3779461427 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb781f6fe0 con 0x7feb5c004800 2024-09-21T07:49:49.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.901+0000 7feb74c00640 1 -- 172.21.15.39:0/3779461427 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb68010ce0 con 0x7feb5c004800 2024-09-21T07:49:49.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.901+0000 7feb7ec00640 1 -- 172.21.15.39:0/3779461427 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb781f75a0 con 0x7feb5c004800 2024-09-21T07:49:49.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.904+0000 7feb74c00640 1 -- 172.21.15.39:0/3779461427 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7feb6801b460 con 0x7feb5c004800 2024-09-21T07:49:49.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.904+0000 7feb74c00640 1 --2- 172.21.15.39:0/3779461427 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feb54078b90 0x7feb5407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.905+0000 7feb74c00640 1 -- 172.21.15.39:0/3779461427 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(233..233 src has 1..233) ==== 7016+0+0 (secure 0 0 0) 0x7feb6809bb70 con 0x7feb5c004800 2024-09-21T07:49:49.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.905+0000 7feb6f400640 1 --2- 172.21.15.39:0/3779461427 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feb54078b90 0x7feb5407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.905+0000 7feb85c53740 1 Processor -- start 2024-09-21T07:49:49.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.905+0000 7feb85c53740 1 -- start start 2024-09-21T07:49:49.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.906+0000 7feb6f400640 1 --2- 172.21.15.39:0/3779461427 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feb54078b90 0x7feb5407b0a0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7feb60033000 tx=0x7feb6002d000 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:49.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.907+0000 7feb85c53740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b4c3f2270 0x557b4c7dda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.907+0000 7feb85c53740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557b4c3c93b0 0x557b4c792f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.907+0000 7feb85c53740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b4c3d1590 0x557b4c7934a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.907+0000 7feb85c53740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557b4c71c430 con 0x557b4c3d1590 2024-09-21T07:49:49.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.907+0000 7feb85c53740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557b4c70c5d0 con 0x557b4c3f2270 2024-09-21T07:49:49.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.907+0000 7feb85c53740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x557b4c6fba00 con 0x557b4c3c93b0 2024-09-21T07:49:49.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.907+0000 7feb7cc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b4c3d1590 0x557b4c7934a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.907+0000 7feb7cc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b4c3d1590 0x557b4c7934a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43864/0 (socket says 172.21.15.39:43864) 2024-09-21T07:49:49.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.907+0000 7feb7cc00640 1 -- 172.21.15.39:0/4013607651 learned_addr learned my addr 172.21.15.39:0/4013607651 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:49.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.907+0000 7feb6f400640 1 --2- 172.21.15.39:0/4013607651 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557b4c3c93b0 0x557b4c792f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.907+0000 7feb77e00640 1 --2- 172.21.15.39:0/4013607651 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b4c3f2270 0x557b4c7dda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.907+0000 7feb7cc00640 1 -- 172.21.15.39:0/4013607651 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557b4c3c93b0 msgr2=0x557b4c792f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:49.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.907+0000 7feb7cc00640 1 --2- 172.21.15.39:0/4013607651 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557b4c3c93b0 0x557b4c792f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.907+0000 7feb7cc00640 1 -- 172.21.15.39:0/4013607651 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b4c3f2270 msgr2=0x557b4c7dda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:49.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.907+0000 7feb7cc00640 1 --2- 172.21.15.39:0/4013607651 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b4c3f2270 0x557b4c7dda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.907+0000 7feb7cc00640 1 -- 172.21.15.39:0/4013607651 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b4c6c4520 con 0x557b4c3d1590 2024-09-21T07:49:49.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.907+0000 7feb6f400640 1 --2- 172.21.15.39:0/4013607651 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557b4c3c93b0 0x557b4c792f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:49.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.908+0000 7feb7cc00640 1 --2- 172.21.15.39:0/4013607651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b4c3d1590 0x557b4c7934a0 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7feb780cdd80 tx=0x7feb70012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=637090e5d8b10115 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:49.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.908+0000 7feb67e00640 1 -- 172.21.15.39:0/4013607651 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb70003b70 con 0x557b4c3d1590 2024-09-21T07:49:49.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.908+0000 7feb67e00640 1 -- 172.21.15.39:0/4013607651 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb70003e00 con 0x557b4c3d1590 2024-09-21T07:49:49.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.908+0000 7feb67e00640 1 -- 172.21.15.39:0/4013607651 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb70004290 con 0x557b4c3d1590 2024-09-21T07:49:49.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.909+0000 7feb85c53740 1 -- 172.21.15.39:0/4013607651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b4c3d1590 msgr2=0x557b4c7934a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:49.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.909+0000 7feb85c53740 1 --2- 172.21.15.39:0/4013607651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b4c3d1590 0x557b4c7934a0 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7feb780cdd80 tx=0x7feb70012820 comp rx=0 tx=0).stop 2024-09-21T07:49:49.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.909+0000 7feb85c53740 1 -- 172.21.15.39:0/4013607651 shutdown_connections 2024-09-21T07:49:49.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.909+0000 7feb85c53740 1 --2- 172.21.15.39:0/4013607651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b4c3d1590 0x557b4c7934a0 unknown :-1 s=CLOSED pgs=2278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.909+0000 7feb85c53740 1 --2- 172.21.15.39:0/4013607651 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557b4c3c93b0 0x557b4c792f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.909+0000 7feb85c53740 1 --2- 172.21.15.39:0/4013607651 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b4c3f2270 0x557b4c7dda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.909+0000 7feb85c53740 1 -- 172.21.15.39:0/4013607651 >> 172.21.15.39:0/4013607651 conn(0x557b4c3e63c0 msgr2=0x557b4c798970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:49.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.909+0000 7feb85c53740 1 -- 172.21.15.39:0/4013607651 shutdown_connections 2024-09-21T07:49:49.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.909+0000 7feb85c53740 1 -- 172.21.15.39:0/4013607651 wait complete. 2024-09-21T07:49:49.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.910+0000 7feb85c53740 1 -- start start 2024-09-21T07:49:49.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.910+0000 7feb85c53740 1 Processor -- start 2024-09-21T07:49:49.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.912+0000 7feb85c53740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b4c3c93b0 0x557b4c7d6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.912+0000 7feb85c53740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b4c3d1590 0x557b4c7d9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.912+0000 7feb85c53740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557b4c3f2270 0x557b4c7d9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.912+0000 7feb85c53740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557b4c6a2330 con 0x557b4c3d1590 2024-09-21T07:49:49.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.912+0000 7feb85c53740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557b4c6ff810 con 0x557b4c3c93b0 2024-09-21T07:49:49.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.912+0000 7feb85c53740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x557b4c46c620 con 0x557b4c3f2270 2024-09-21T07:49:49.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.912+0000 7feb85c53740 10 client.0 _ll_register_callbacks cb 0x557b4c6a7420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:49:49.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.912+0000 7feb6f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b4c3d1590 0x557b4c7d9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.912+0000 7feb6f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b4c3d1590 0x557b4c7d9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43872/0 (socket says 172.21.15.39:43872) 2024-09-21T07:49:49.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.912+0000 7feb6f400640 1 -- 172.21.15.39:0/2118808156 learned_addr learned my addr 172.21.15.39:0/2118808156 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:49.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.913+0000 7feb6f400640 1 -- 172.21.15.39:0/2118808156 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557b4c3f2270 msgr2=0x557b4c7d9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:49.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.913+0000 7feb6f400640 1 --2- 172.21.15.39:0/2118808156 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557b4c3f2270 0x557b4c7d9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.913+0000 7feb6f400640 1 -- 172.21.15.39:0/2118808156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b4c3c93b0 msgr2=0x557b4c7d6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:49.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.913+0000 7feb6f400640 1 --2- 172.21.15.39:0/2118808156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b4c3c93b0 0x557b4c7d6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.913+0000 7feb6f400640 1 -- 172.21.15.39:0/2118808156 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b4c45d850 con 0x557b4c3d1590 2024-09-21T07:49:49.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.913+0000 7feb6f400640 1 --2- 172.21.15.39:0/2118808156 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b4c3d1590 0x557b4c7d9a90 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7feb6000c320 tx=0x7feb60006720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74bd661029fb709d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:49.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.913+0000 7feb64c00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:49:49.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.913+0000 7feb64c00640 1 -- 172.21.15.39:0/2118808156 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb60003a00 con 0x557b4c3d1590 2024-09-21T07:49:49.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.913+0000 7feb64c00640 1 -- 172.21.15.39:0/2118808156 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(233..233 src has 1..233) ==== 7016+0+0 (secure 0 0 0) 0x7feb60003ba0 con 0x557b4c3d1590 2024-09-21T07:49:49.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.913+0000 7feb85c53740 1 -- 172.21.15.39:0/2118808156 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb70003e00 con 0x557b4c3d1590 2024-09-21T07:49:49.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.913+0000 7feb64c00640 1 -- 172.21.15.39:0/2118808156 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=250) ==== 24+0+0 (secure 0 0 0) 0x7feb60003da0 con 0x557b4c3d1590 2024-09-21T07:49:49.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.913+0000 7feb85c53740 10 client.10848 fetch_fsmap learned FSMap version 250 2024-09-21T07:49:49.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.913+0000 7feb85c53740 1 -- 172.21.15.39:0/2118808156 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=250,monmap=2+}) -- 0x557b4c4d46b0 con 0x557b4c3d1590 2024-09-21T07:49:49.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.914+0000 7feb64c00640 1 -- 172.21.15.39:0/2118808156 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb70003e00 con 0x557b4c3d1590 2024-09-21T07:49:49.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.914+0000 7feb64c00640 1 -- 172.21.15.39:0/2118808156 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 250) ==== 3542+0+0 (secure 0 0 0) 0x7feb60039530 con 0x557b4c3d1590 2024-09-21T07:49:49.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.914+0000 7feb85c53740 10 client.10848 fetch_fsmap finished waiting for FSMap version 250 2024-09-21T07:49:49.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.914+0000 7feb85c53740 10 client.10848 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:49:49.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.914+0000 7feb85c53740 20 client.10848 populate_metadata read hostname 'smithi039' 2024-09-21T07:49:49.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.914+0000 7feb85c53740 1 --2- 172.21.15.39:0/2118808156 >> [v2:172.21.15.39:6833/2824493631,v1:172.21.15.39:6835/2824493631] conn(0x557b4c3b0e40 0x557b4c7f4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.914+0000 7feb85c53740 4 client.10848 mds_command: new command op to 10635 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:49:49.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.914+0000 7feb85c53740 1 -- 172.21.15.39:0/2118808156 --> [v2:172.21.15.39:6833/2824493631,v1:172.21.15.39:6835/2824493631] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b4c70e520 con 0x557b4c3b0e40 2024-09-21T07:49:49.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.914+0000 7feb77e00640 1 --2- 172.21.15.39:0/2118808156 >> [v2:172.21.15.39:6833/2824493631,v1:172.21.15.39:6835/2824493631] conn(0x557b4c3b0e40 0x557b4c7f4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.915+0000 7feb64c00640 10 client.10848 ms_handle_connect on v2:172.21.15.39:6833/2824493631 2024-09-21T07:49:49.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.915+0000 7feb77e00640 1 --2- 172.21.15.39:0/2118808156 >> [v2:172.21.15.39:6833/2824493631,v1:172.21.15.39:6835/2824493631] conn(0x557b4c3b0e40 0x557b4c7f4730 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e07e454c01f9a461 server_cookie=4c1267e9e8801c45 in_seq=0 out_seq=0 2024-09-21T07:49:49.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.916+0000 7feb64c00640 1 -- 172.21.15.39:0/2118808156 <== mds.0 v2:172.21.15.39:6833/2824493631 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557b4c70e520 con 0x557b4c3b0e40 2024-09-21T07:49:49.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.916+0000 7feb64c00640 10 client.10848 handle_command_reply: tid=0 2024-09-21T07:49:49.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.916+0000 7feb85c53740 1 client.10848 shutdown 2024-09-21T07:49:49.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.916+0000 7feb85c53740 1 -- 172.21.15.39:0/2118808156 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b4c3d1590 msgr2=0x557b4c7d9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:49.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.916+0000 7feb85c53740 1 --2- 172.21.15.39:0/2118808156 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b4c3d1590 0x557b4c7d9a90 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7feb6000c320 tx=0x7feb60006720 comp rx=0 tx=0).stop 2024-09-21T07:49:49.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.916+0000 7feb85c53740 1 -- 172.21.15.39:0/2118808156 shutdown_connections 2024-09-21T07:49:49.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.916+0000 7feb85c53740 1 --2- 172.21.15.39:0/2118808156 >> [v2:172.21.15.39:6833/2824493631,v1:172.21.15.39:6835/2824493631] conn(0x557b4c3b0e40 0x557b4c7f4730 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.916+0000 7feb85c53740 1 --2- 172.21.15.39:0/2118808156 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557b4c3f2270 0x557b4c7d9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.916+0000 7feb85c53740 1 --2- 172.21.15.39:0/2118808156 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b4c3d1590 0x557b4c7d9a90 unknown :-1 s=CLOSED pgs=2279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.916+0000 7feb85c53740 1 --2- 172.21.15.39:0/2118808156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b4c3c93b0 0x557b4c7d6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.916+0000 7feb64c00640 0 client.10848 ms_handle_reset on v2:172.21.15.39:6833/2824493631 2024-09-21T07:49:49.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.916+0000 7feb85c53740 1 -- 172.21.15.39:0/2118808156 >> 172.21.15.39:0/2118808156 conn(0x557b4c3e63c0 msgr2=0x557b4c7bf580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:49.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.916+0000 7feb85c53740 1 -- 172.21.15.39:0/2118808156 shutdown_connections 2024-09-21T07:49:49.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.916+0000 7feb85c53740 1 -- 172.21.15.39:0/2118808156 wait complete. 2024-09-21T07:49:49.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.916+0000 7feb85c53740 20 client.10848 trim_cache size 0 max 16384 2024-09-21T07:49:49.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.936+0000 7feb85c53740 1 Processor -- start 2024-09-21T07:49:49.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.936+0000 7feb85c53740 1 -- start start 2024-09-21T07:49:49.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.937+0000 7feb85c53740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b4c3c93b0 0x557b4c798910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.937+0000 7feb85c53740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557b4c3d1590 0x557b4c799770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.937+0000 7feb85c53740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b4c3f2270 0x557b4c7d6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.937+0000 7feb85c53740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7feb78200010 con 0x557b4c3c93b0 2024-09-21T07:49:49.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.937+0000 7feb85c53740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7feb782006e0 con 0x557b4c3f2270 2024-09-21T07:49:49.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.937+0000 7feb85c53740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7feb780b4ff0 con 0x557b4c3d1590 2024-09-21T07:49:49.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.937+0000 7feb77e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b4c3c93b0 0x557b4c798910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.938+0000 7feb77e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b4c3c93b0 0x557b4c798910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43878/0 (socket says 172.21.15.39:43878) 2024-09-21T07:49:49.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.938+0000 7feb77e00640 1 -- 172.21.15.39:0/203166928 learned_addr learned my addr 172.21.15.39:0/203166928 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:49.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.938+0000 7feb7cc00640 1 --2- 172.21.15.39:0/203166928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b4c3f2270 0x557b4c7d6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.938+0000 7feb6f400640 1 --2- 172.21.15.39:0/203166928 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557b4c3d1590 0x557b4c799770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.938+0000 7feb77e00640 1 -- 172.21.15.39:0/203166928 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557b4c3d1590 msgr2=0x557b4c799770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:49.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.938+0000 7feb77e00640 1 --2- 172.21.15.39:0/203166928 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557b4c3d1590 0x557b4c799770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.938+0000 7feb77e00640 1 -- 172.21.15.39:0/203166928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b4c3f2270 msgr2=0x557b4c7d6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:49.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.938+0000 7feb77e00640 1 --2- 172.21.15.39:0/203166928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b4c3f2270 0x557b4c7d6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.938+0000 7feb77e00640 1 -- 172.21.15.39:0/203166928 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b4c3a3970 con 0x557b4c3c93b0 2024-09-21T07:49:49.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.938+0000 7feb7cc00640 1 --2- 172.21.15.39:0/203166928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b4c3f2270 0x557b4c7d6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:49.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.938+0000 7feb77e00640 1 --2- 172.21.15.39:0/203166928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b4c3c93b0 0x557b4c798910 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7feb68001eb0 tx=0x7feb68010e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4be3371f350eb21 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:49.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.938+0000 7feb67400640 1 -- 172.21.15.39:0/203166928 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb680632f0 con 0x557b4c3c93b0 2024-09-21T07:49:49.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.938+0000 7feb67400640 1 -- 172.21.15.39:0/203166928 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb68067dd0 con 0x557b4c3c93b0 2024-09-21T07:49:49.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.938+0000 7feb67400640 1 -- 172.21.15.39:0/203166928 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6806cd10 con 0x557b4c3c93b0 2024-09-21T07:49:49.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.939+0000 7feb85c53740 1 -- 172.21.15.39:0/203166928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b4c3c93b0 msgr2=0x557b4c798910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:49.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.939+0000 7feb85c53740 1 --2- 172.21.15.39:0/203166928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b4c3c93b0 0x557b4c798910 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7feb68001eb0 tx=0x7feb68010e80 comp rx=0 tx=0).stop 2024-09-21T07:49:49.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.939+0000 7feb85c53740 1 -- 172.21.15.39:0/203166928 shutdown_connections 2024-09-21T07:49:49.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.939+0000 7feb85c53740 1 --2- 172.21.15.39:0/203166928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b4c3f2270 0x557b4c7d6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.939+0000 7feb85c53740 1 --2- 172.21.15.39:0/203166928 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557b4c3d1590 0x557b4c799770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.939+0000 7feb85c53740 1 --2- 172.21.15.39:0/203166928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b4c3c93b0 0x557b4c798910 unknown :-1 s=CLOSED pgs=2280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.939+0000 7feb85c53740 1 -- 172.21.15.39:0/203166928 >> 172.21.15.39:0/203166928 conn(0x557b4c3e63c0 msgr2=0x557b4c7bf480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:49.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.939+0000 7feb85c53740 1 -- 172.21.15.39:0/203166928 shutdown_connections 2024-09-21T07:49:49.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.939+0000 7feb85c53740 1 -- 172.21.15.39:0/203166928 wait complete. 2024-09-21T07:49:49.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.940+0000 7feb85c53740 1 -- start start 2024-09-21T07:49:49.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.940+0000 7feb85c53740 1 Processor -- start 2024-09-21T07:49:49.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.942+0000 7feb85c53740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b4c3c93b0 0x557b4c847790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.942+0000 7feb85c53740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557b4c3d1590 0x557b4c7f6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.942+0000 7feb85c53740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b4c3f2270 0x557b4c7d3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.942+0000 7feb85c53740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557b4c7f1b70 con 0x557b4c3c93b0 2024-09-21T07:49:49.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.942+0000 7feb85c53740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557b4c6a2330 con 0x557b4c3f2270 2024-09-21T07:49:49.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.942+0000 7feb85c53740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x557b4c7d4780 con 0x557b4c3d1590 2024-09-21T07:49:49.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.942+0000 7feb85c53740 10 client.0 _ll_register_callbacks cb 0x7feb5c00c060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:49:49.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.942+0000 7feb77e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b4c3c93b0 0x557b4c847790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.942+0000 7feb77e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b4c3c93b0 0x557b4c847790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43892/0 (socket says 172.21.15.39:43892) 2024-09-21T07:49:49.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.942+0000 7feb77e00640 1 -- 172.21.15.39:0/623558861 learned_addr learned my addr 172.21.15.39:0/623558861 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:49:49.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.943+0000 7feb77e00640 1 -- 172.21.15.39:0/623558861 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557b4c3d1590 msgr2=0x557b4c7f6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:49.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.943+0000 7feb77e00640 1 --2- 172.21.15.39:0/623558861 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557b4c3d1590 0x557b4c7f6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.943+0000 7feb77e00640 1 -- 172.21.15.39:0/623558861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b4c3f2270 msgr2=0x557b4c7d3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:49.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.943+0000 7feb77e00640 1 --2- 172.21.15.39:0/623558861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b4c3f2270 0x557b4c7d3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.943+0000 7feb77e00640 1 -- 172.21.15.39:0/623558861 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b4c847cd0 con 0x557b4c3c93b0 2024-09-21T07:49:49.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.943+0000 7feb77e00640 1 --2- 172.21.15.39:0/623558861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b4c3c93b0 0x557b4c847790 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7feb68063ba0 tx=0x7feb6806cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed411dea67256ac9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:49.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.943+0000 7feb64c00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:49:49.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.943+0000 7feb64c00640 1 -- 172.21.15.39:0/623558861 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6804e040 con 0x557b4c3c93b0 2024-09-21T07:49:49.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.943+0000 7feb64c00640 1 -- 172.21.15.39:0/623558861 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(233..233 src has 1..233) ==== 7016+0+0 (secure 0 0 0) 0x7feb68062550 con 0x557b4c3c93b0 2024-09-21T07:49:49.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.943+0000 7feb85c53740 1 -- 172.21.15.39:0/623558861 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb68067dd0 con 0x557b4c3c93b0 2024-09-21T07:49:49.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.943+0000 7feb64c00640 1 -- 172.21.15.39:0/623558861 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=250) ==== 24+0+0 (secure 0 0 0) 0x7feb68062750 con 0x557b4c3c93b0 2024-09-21T07:49:49.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.943+0000 7feb85c53740 10 client.10854 fetch_fsmap learned FSMap version 250 2024-09-21T07:49:49.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.943+0000 7feb85c53740 1 -- 172.21.15.39:0/623558861 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=250,monmap=2+}) -- 0x557b4c865af0 con 0x557b4c3c93b0 2024-09-21T07:49:49.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.944+0000 7feb64c00640 1 -- 172.21.15.39:0/623558861 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb68067dd0 con 0x557b4c3c93b0 2024-09-21T07:49:49.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.944+0000 7feb64c00640 1 -- 172.21.15.39:0/623558861 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 250) ==== 3542+0+0 (secure 0 0 0) 0x7feb68085af0 con 0x557b4c3c93b0 2024-09-21T07:49:49.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.944+0000 7feb85c53740 10 client.10854 fetch_fsmap finished waiting for FSMap version 250 2024-09-21T07:49:49.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.944+0000 7feb85c53740 10 client.10854 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:49:49.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.944+0000 7feb85c53740 20 client.10854 populate_metadata read hostname 'smithi039' 2024-09-21T07:49:49.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.944+0000 7feb85c53740 1 --2- 172.21.15.39:0/623558861 >> [v2:172.21.15.39:6833/2824493631,v1:172.21.15.39:6835/2824493631] conn(0x557b4c862130 0x557b4c862540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:49.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.944+0000 7feb85c53740 4 client.10854 mds_command: new command op to 10635 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:49:49.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.944+0000 7feb85c53740 1 -- 172.21.15.39:0/623558861 --> [v2:172.21.15.39:6833/2824493631,v1:172.21.15.39:6835/2824493631] -- command(tid 0: {"prefix": "status"}) -- 0x557b4c709fd0 con 0x557b4c862130 2024-09-21T07:49:49.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.944+0000 7feb6f400640 1 --2- 172.21.15.39:0/623558861 >> [v2:172.21.15.39:6833/2824493631,v1:172.21.15.39:6835/2824493631] conn(0x557b4c862130 0x557b4c862540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:49.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.945+0000 7feb6f400640 1 --2- 172.21.15.39:0/623558861 >> [v2:172.21.15.39:6833/2824493631,v1:172.21.15.39:6835/2824493631] conn(0x557b4c862130 0x557b4c862540 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b48f04a3e0ba09ce server_cookie=dc9b9e00e6d9a211 in_seq=0 out_seq=0 2024-09-21T07:49:49.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.945+0000 7feb64c00640 10 client.10854 ms_handle_connect on v2:172.21.15.39:6833/2824493631 2024-09-21T07:49:49.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.945+0000 7feb64c00640 1 -- 172.21.15.39:0/623558861 <== mds.0 v2:172.21.15.39:6833/2824493631 1 ==== command_reply(tid 0: 0 ) ==== 8+0+323 (crc 0 0 0) 0x557b4c709fd0 con 0x557b4c862130 2024-09-21T07:49:49.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.945+0000 7feb64c00640 10 client.10854 handle_command_reply: tid=0 2024-09-21T07:49:49.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.945+0000 7feb85c53740 1 client.10854 shutdown 2024-09-21T07:49:49.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.945+0000 7feb85c53740 1 -- 172.21.15.39:0/623558861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b4c3c93b0 msgr2=0x557b4c847790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:49.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.945+0000 7feb85c53740 1 --2- 172.21.15.39:0/623558861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b4c3c93b0 0x557b4c847790 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7feb68063ba0 tx=0x7feb6806cd10 comp rx=0 tx=0).stop 2024-09-21T07:49:49.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.945+0000 7feb85c53740 1 -- 172.21.15.39:0/623558861 shutdown_connections 2024-09-21T07:49:49.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.945+0000 7feb85c53740 1 --2- 172.21.15.39:0/623558861 >> [v2:172.21.15.39:6833/2824493631,v1:172.21.15.39:6835/2824493631] conn(0x557b4c862130 0x557b4c862540 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.945+0000 7feb85c53740 1 --2- 172.21.15.39:0/623558861 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557b4c3f2270 0x557b4c7d3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.945+0000 7feb85c53740 1 --2- 172.21.15.39:0/623558861 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x557b4c3d1590 0x557b4c7f6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.945+0000 7feb85c53740 1 --2- 172.21.15.39:0/623558861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b4c3c93b0 0x557b4c847790 unknown :-1 s=CLOSED pgs=2281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.945+0000 7feb64c00640 0 client.10854 ms_handle_reset on v2:172.21.15.39:6833/2824493631 2024-09-21T07:49:49.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.945+0000 7feb85c53740 1 -- 172.21.15.39:0/623558861 >> 172.21.15.39:0/623558861 conn(0x557b4c3e63c0 msgr2=0x557b4c7bf480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:49.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.945+0000 7feb85c53740 1 -- 172.21.15.39:0/623558861 shutdown_connections 2024-09-21T07:49:49.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.945+0000 7feb85c53740 1 -- 172.21.15.39:0/623558861 wait complete. 2024-09-21T07:49:49.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.945+0000 7feb85c53740 20 client.10854 trim_cache size 0 max 16384 2024-09-21T07:49:49.951 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:49:49.951 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:49:49.951 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-09-21T07:49:49.951 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 10635, 2024-09-21T07:49:49.951 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:49:49.951 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:49:49.951 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:49:49.951 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 14.832533491, 2024-09-21T07:49:49.951 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 250, 2024-09-21T07:49:49.951 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 233, 2024-09-21T07:49:49.951 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 233, 2024-09-21T07:49:49.952 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 19.835297851 2024-09-21T07:49:49.952 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:49:49.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.946+0000 7feb7ec00640 1 -- 172.21.15.39:0/3779461427 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feb54078b90 msgr2=0x7feb5407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:49.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.946+0000 7feb7ec00640 1 --2- 172.21.15.39:0/3779461427 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feb54078b90 0x7feb5407b0a0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7feb60033000 tx=0x7feb6002d000 comp rx=0 tx=0).stop 2024-09-21T07:49:49.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.946+0000 7feb7ec00640 1 -- 172.21.15.39:0/3779461427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb5c004800 msgr2=0x7feb78156110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:49.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.946+0000 7feb7ec00640 1 --2- 172.21.15.39:0/3779461427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb5c004800 0x7feb78156110 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7feb68004cc0 tx=0x7feb68001800 comp rx=0 tx=0).stop 2024-09-21T07:49:49.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.946+0000 7feb7ec00640 1 -- 172.21.15.39:0/3779461427 shutdown_connections 2024-09-21T07:49:49.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.946+0000 7feb7ec00640 1 --2- 172.21.15.39:0/3779461427 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feb54078b90 0x7feb5407b0a0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.946+0000 7feb7ec00640 1 --2- 172.21.15.39:0/3779461427 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb780b1c90 0x7feb7814fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.946+0000 7feb7ec00640 1 --2- 172.21.15.39:0/3779461427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb78156650 0x7feb780b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.946+0000 7feb7ec00640 1 --2- 172.21.15.39:0/3779461427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb5c004800 0x7feb78156110 unknown :-1 s=CLOSED pgs=2277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:49.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.946+0000 7feb7ec00640 1 -- 172.21.15.39:0/3779461427 >> 172.21.15.39:0/3779461427 conn(0x7feb7806cfc0 msgr2=0x7feb78157ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:49.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.946+0000 7feb7ec00640 1 -- 172.21.15.39:0/3779461427 shutdown_connections 2024-09-21T07:49:49.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:49:49.946+0000 7feb7ec00640 1 -- 172.21.15.39:0/3779461427 wait complete. 2024-09-21T07:49:49.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:49.973 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d0/d1/dir) 2024-09-21T07:49:50.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:50.016 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d0/d2/dir) 2024-09-21T07:49:50.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:50.096 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d0) 2024-09-21T07:49:50.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:50.173 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d0/d1) 2024-09-21T07:49:50.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:50.249 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 2 d0/d2) 2024-09-21T07:49:50.325 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-09-21T07:49:50.573 INFO:teuthology.orchestra.run.smithi148.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T07:49:50.574 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T07:49:50.574 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds2", "dirfrag": "0x102", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "2", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-09-21T07:49:50.574 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-21T07:49:50.819 INFO:teuthology.orchestra.run.smithi039.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T07:49:50.820 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T07:49:50.820 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds1", "dirfrag": "0x101", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-09-21T07:49:50.820 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-21T07:49:51.066 INFO:teuthology.orchestra.run.smithi039.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":3,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T07:49:51.067 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 3, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T07:49:51.067 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "13", "version": "7", "committing_version": "1", "committed_version": "1", "is_rep": true, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 3, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "replicated": 0, "dirty": 1, "authpin": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds0", "dirfrag": "0x100", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "0", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-09-21T07:49:51.067 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d0', 0), ('/d0/d1', 1), ('/d0/d2', 2)] 2024-09-21T07:49:53.068 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-09-21T07:49:53.319 INFO:teuthology.orchestra.run.smithi148.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T07:49:53.320 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T07:49:53.320 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds2", "dirfrag": "0x102", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "2", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-09-21T07:49:53.320 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-21T07:49:53.577 INFO:teuthology.orchestra.run.smithi039.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T07:49:53.577 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T07:49:53.578 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds1", "dirfrag": "0x101", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-09-21T07:49:53.578 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-21T07:49:53.823 INFO:teuthology.orchestra.run.smithi039.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":3,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T07:49:53.824 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 3, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T07:49:53.824 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "13", "version": "7", "committing_version": "1", "committed_version": "1", "is_rep": true, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 3, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "replicated": 0, "dirty": 1, "authpin": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds0", "dirfrag": "0x100", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "0", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-09-21T07:49:53.824 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d0', 0), ('/d0/d1', 1), ('/d0/d2', 2)] 2024-09-21T07:49:55.825 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-09-21T07:49:56.071 INFO:teuthology.orchestra.run.smithi148.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T07:49:56.072 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T07:49:56.072 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds2", "dirfrag": "0x102", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "2", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-09-21T07:49:56.072 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-21T07:49:56.327 INFO:teuthology.orchestra.run.smithi039.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T07:49:56.328 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T07:49:56.328 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds1", "dirfrag": "0x101", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-09-21T07:49:56.328 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-21T07:49:56.571 INFO:teuthology.orchestra.run.smithi039.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"16","version":"16","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T07:49:56.571 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "16", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "16" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T07:49:56.572 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "16", "version": "16", "committing_version": "1", "committed_version": "1", "is_rep": true, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "replicated": 0, "dirty": 1, "authpin": 0 }, "nref": 3 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds0", "dirfrag": "0x100", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "0", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-09-21T07:49:56.572 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d0', 0), ('/d0/d1', 1), ('/d0/d2', 2)] 2024-09-21T07:49:58.573 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-09-21T07:49:58.823 INFO:teuthology.orchestra.run.smithi148.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T07:49:58.824 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T07:49:58.824 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds2", "dirfrag": "0x102", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "2", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-09-21T07:49:58.824 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-21T07:49:59.081 INFO:teuthology.orchestra.run.smithi039.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T07:49:59.082 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T07:49:59.082 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds1", "dirfrag": "0x101", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-09-21T07:49:59.082 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-21T07:49:59.330 INFO:teuthology.orchestra.run.smithi039.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"16","version":"16","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-21T07:49:59.330 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "16", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "16" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-21T07:49:59.331 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "16", "version": "16", "committing_version": "1", "committed_version": "1", "is_rep": true, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "replicated": 0, "dirty": 1, "authpin": 0 }, "nref": 3 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds0", "dirfrag": "0x100", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "0", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-09-21T07:49:59.331 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d0', 0), ('/d0/d1', 1), ('/d0/d2', 2)] 2024-09-21T07:50:01.332 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-09-21T07:50:01.581 INFO:teuthology.orchestra.run.smithi148.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d0/d2","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"3","version":"3","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"16","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2024-09-21T07:50:01.582 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d0/d2", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "3", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "3" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "16" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-21T07:50:01.582 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds2", "dirfrag": "0x102", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "2", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } }, { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": 2, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/d0/d2", "dirfrag": "0x10000000003", "snapid_first": 2, "projected_version": "3", "version": "3", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "2", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": false, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "0", "version": "16", "committing_version": "0", "committed_version": "0", "is_rep": true, "dir_auth": "0", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1 }, "nref": 2 } } ] 2024-09-21T07:50:01.582 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-21T07:50:01.842 INFO:teuthology.orchestra.run.smithi039.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d0/d1","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"3","version":"3","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"16","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}}] 2024-09-21T07:50:01.843 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d0/d1", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "3", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "3" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "16" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-21T07:50:01.843 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds1", "dirfrag": "0x101", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/d0/d1", "dirfrag": "0x10000000001", "snapid_first": 2, "projected_version": "3", "version": "3", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": false, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "0", "version": "16", "committing_version": "0", "committed_version": "0", "is_rep": true, "dir_auth": "0", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1 }, "nref": 2 } } ] 2024-09-21T07:50:01.843 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-21T07:50:02.091 INFO:teuthology.orchestra.run.smithi039.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"16","version":"16","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1,"2":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d0/d2","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"3","version":"3","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d0","dirfrag":"0x10000000000","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1,"2":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d0/d1","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"3","version":"3","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-09-21T07:50:02.092 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1, "2": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "16", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "16" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d0/d2", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "3", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "3" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000000", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000001", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d0/d1", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "3", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "3" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-21T07:50:02.093 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "16", "version": "16", "committing_version": "1", "committed_version": "1", "is_rep": true, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "1": 1, "2": 1 } }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "replicated": 1, "dirty": 1, "authpin": 0 }, "nref": 4 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds0", "dirfrag": "0x100", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "0", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } }, { "is_auth": false, "auth_first": 2, "auth_second": -2, "export_pin": 2, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/d0/d2", "dirfrag": "0x10000000003", "snapid_first": 2, "projected_version": "3", "version": "3", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "2", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": 0, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/d0", "dirfrag": "0x10000000000", "snapid_first": 2, "projected_version": "14", "version": "14", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "1": 1, "2": 1 } }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "replicated": 1, "dirty": 1, "authpin": 0 }, "nref": 4 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/d0/d1", "dirfrag": "0x10000000001", "snapid_first": 2, "projected_version": "3", "version": "3", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } } ] 2024-09-21T07:50:02.093 INFO:tasks.cephfs.cephfs_test_case:[('/d0', 0), ('/d0/d1', 1), ('/d0/d2', 2)] =?= [('/d0', 0), ('/d0/d1', 1), ('/d0/d2', 2)] 2024-09-21T07:50:02.093 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:02.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.319+0000 7ff847c00640 1 -- 172.21.15.39:0/195358677 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff8400cb8a0 msgr2=0x7ff840156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:02.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.319+0000 7ff847c00640 1 --2- 172.21.15.39:0/195358677 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff8400cb8a0 0x7ff840156bb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff82c009830 tx=0x7ff82c02a480 comp rx=0 tx=0).stop 2024-09-21T07:50:02.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.319+0000 7ff847c00640 1 -- 172.21.15.39:0/195358677 shutdown_connections 2024-09-21T07:50:02.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.319+0000 7ff847c00640 1 --2- 172.21.15.39:0/195358677 >> v2:172.21.15.148:3300/0 conn(0x7ff8400ccc60 0x7ff8400cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:02.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.319+0000 7ff847c00640 1 --2- 172.21.15.39:0/195358677 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff8400cb8a0 0x7ff840156bb0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:02.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.319+0000 7ff847c00640 1 -- 172.21.15.39:0/195358677 >> 172.21.15.39:0/195358677 conn(0x7ff8400b2460 msgr2=0x7ff84006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:02.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.319+0000 7ff847c00640 1 -- 172.21.15.39:0/195358677 shutdown_connections 2024-09-21T07:50:02.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.319+0000 7ff847c00640 1 -- 172.21.15.39:0/195358677 wait complete. 2024-09-21T07:50:02.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.321+0000 7ff847c00640 1 Processor -- start 2024-09-21T07:50:02.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.321+0000 7ff847c00640 1 -- start start 2024-09-21T07:50:02.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.323+0000 7ff847c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff8400b30a0 0x7ff840155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:02.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.323+0000 7ff847c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8400cb8a0 0x7ff8400ce920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:02.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.323+0000 7ff847c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8400ccc60 0x7ff8400c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:02.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.323+0000 7ff847c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff8400af410 con 0x7ff8400cb8a0 2024-09-21T07:50:02.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.323+0000 7ff847c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff8400af290 con 0x7ff8400ccc60 2024-09-21T07:50:02.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.323+0000 7ff847c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7ff8400af590 con 0x7ff8400b30a0 2024-09-21T07:50:02.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.323+0000 7ff83fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8400cb8a0 0x7ff8400ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:02.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.323+0000 7ff83fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8400cb8a0 0x7ff8400ce920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39532/0 (socket says 172.21.15.39:39532) 2024-09-21T07:50:02.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.323+0000 7ff83fe00640 1 -- 172.21.15.39:0/3220593937 learned_addr learned my addr 172.21.15.39:0/3220593937 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:02.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.323+0000 7ff845200640 1 --2- 172.21.15.39:0/3220593937 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff8400b30a0 0x7ff840155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:02.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.323+0000 7ff845c00640 1 --2- 172.21.15.39:0/3220593937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8400ccc60 0x7ff8400c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:02.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.324+0000 7ff83fe00640 1 -- 172.21.15.39:0/3220593937 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff8400b30a0 msgr2=0x7ff840155f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:02.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.324+0000 7ff83fe00640 1 --2- 172.21.15.39:0/3220593937 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff8400b30a0 0x7ff840155f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:02.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.324+0000 7ff83fe00640 1 -- 172.21.15.39:0/3220593937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8400ccc60 msgr2=0x7ff8400c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:02.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.324+0000 7ff83fe00640 1 --2- 172.21.15.39:0/3220593937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8400ccc60 0x7ff8400c9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:02.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.324+0000 7ff83fe00640 1 -- 172.21.15.39:0/3220593937 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8400cee60 con 0x7ff8400cb8a0 2024-09-21T07:50:02.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.324+0000 7ff845c00640 1 --2- 172.21.15.39:0/3220593937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8400ccc60 0x7ff8400c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:50:02.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.324+0000 7ff83fe00640 1 --2- 172.21.15.39:0/3220593937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8400cb8a0 0x7ff8400ce920 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7ff82c009950 tx=0x7ff82c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:02.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.324+0000 7ff83cc00640 1 -- 172.21.15.39:0/3220593937 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c005c90 con 0x7ff8400cb8a0 2024-09-21T07:50:02.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.324+0000 7ff83cc00640 1 -- 172.21.15.39:0/3220593937 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff82c031e50 con 0x7ff8400cb8a0 2024-09-21T07:50:02.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.324+0000 7ff847c00640 1 -- 172.21.15.39:0/3220593937 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8400cf810 con 0x7ff8400cb8a0 2024-09-21T07:50:02.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.324+0000 7ff83cc00640 1 -- 172.21.15.39:0/3220593937 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c033310 con 0x7ff8400cb8a0 2024-09-21T07:50:02.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.325+0000 7ff847c00640 1 -- 172.21.15.39:0/3220593937 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8400ce120 con 0x7ff8400cb8a0 2024-09-21T07:50:02.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.326+0000 7ff847c00640 1 -- 172.21.15.39:0/3220593937 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8400cf580 con 0x7ff8400cb8a0 2024-09-21T07:50:02.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.327+0000 7ff83cc00640 1 -- 172.21.15.39:0/3220593937 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff82c0334b0 con 0x7ff8400cb8a0 2024-09-21T07:50:02.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.328+0000 7ff83cc00640 1 --2- 172.21.15.39:0/3220593937 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff820078b90 0x7ff82007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:02.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.328+0000 7ff83cc00640 1 -- 172.21.15.39:0/3220593937 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(235..235 src has 1..235) ==== 7132+0+0 (secure 0 0 0) 0x7ff82c0bc580 con 0x7ff8400cb8a0 2024-09-21T07:50:02.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.328+0000 7ff845200640 1 --2- 172.21.15.39:0/3220593937 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff820078b90 0x7ff82007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:02.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.329+0000 7ff845200640 1 --2- 172.21.15.39:0/3220593937 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff820078b90 0x7ff82007b0a0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7ff834007d60 tx=0x7ff834007cf0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:02.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.333+0000 7ff83cc00640 1 -- 172.21.15.39:0/3220593937 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff82c03f030 con 0x7ff8400cb8a0 2024-09-21T07:50:02.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.571+0000 7ff847c00640 1 -- 172.21.15.39:0/3220593937 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8400afae0 con 0x7ff8400cb8a0 2024-09-21T07:50:02.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.571+0000 7ff83cc00640 1 -- 172.21.15.39:0/3220593937 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 251 v251) ==== 95+0+5289 (secure 0 0 0) 0x7ff82c084390 con 0x7ff8400cb8a0 2024-09-21T07:50:02.574 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:50:02.574 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":251,"btime":"2024-09-21T07:50:00:589303+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8531,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1438851059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1438851059},{"type":"v1","addr":"172.21.15.148:6837","nonce":1438851059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":239}],"filesystems":[{"mdsmap":{"epoch":251,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:50:00.121724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":10635,"mds_1":10632,"mds_2":9709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9709":{"gid":9709,"name":"b","rank":2,"incarnation":249,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3497801035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3497801035},{"type":"v1","addr":"172.21.15.148:6836","nonce":3497801035}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10632":{"gid":10632,"name":"a","rank":1,"incarnation":247,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6834/4143807732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4143807732},{"type":"v1","addr":"172.21.15.39:6834","nonce":4143807732}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10635":{"gid":10635,"name":"c","rank":0,"incarnation":240,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/2824493631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2824493631},{"type":"v1","addr":"172.21.15.39:6835","nonce":2824493631}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10635,"qdb_cluster":[10635,10632,9709]},"id":13}]} 2024-09-21T07:50:02.574 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 251 2024-09-21T07:50:02.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.574+0000 7ff847c00640 1 -- 172.21.15.39:0/3220593937 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff820078b90 msgr2=0x7ff82007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:02.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.574+0000 7ff847c00640 1 --2- 172.21.15.39:0/3220593937 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff820078b90 0x7ff82007b0a0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7ff834007d60 tx=0x7ff834007cf0 comp rx=0 tx=0).stop 2024-09-21T07:50:02.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.574+0000 7ff847c00640 1 -- 172.21.15.39:0/3220593937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8400cb8a0 msgr2=0x7ff8400ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:02.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.574+0000 7ff847c00640 1 --2- 172.21.15.39:0/3220593937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8400cb8a0 0x7ff8400ce920 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7ff82c009950 tx=0x7ff82c033c20 comp rx=0 tx=0).stop 2024-09-21T07:50:02.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.574+0000 7ff847c00640 1 -- 172.21.15.39:0/3220593937 shutdown_connections 2024-09-21T07:50:02.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.574+0000 7ff847c00640 1 --2- 172.21.15.39:0/3220593937 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff820078b90 0x7ff82007b0a0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:02.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.574+0000 7ff847c00640 1 --2- 172.21.15.39:0/3220593937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8400ccc60 0x7ff8400c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:02.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.574+0000 7ff847c00640 1 --2- 172.21.15.39:0/3220593937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8400cb8a0 0x7ff8400ce920 unknown :-1 s=CLOSED pgs=2282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:02.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.574+0000 7ff847c00640 1 --2- 172.21.15.39:0/3220593937 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff8400b30a0 0x7ff840155f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:02.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.574+0000 7ff847c00640 1 -- 172.21.15.39:0/3220593937 >> 172.21.15.39:0/3220593937 conn(0x7ff8400b2460 msgr2=0x7ff8400ad060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:02.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.574+0000 7ff847c00640 1 -- 172.21.15.39:0/3220593937 shutdown_connections 2024-09-21T07:50:02.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:02.574+0000 7ff847c00640 1 -- 172.21.15.39:0/3220593937 wait complete. 2024-09-21T07:50:02.603 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2024-09-21T07:50:02.847 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_snap":1,"last_created":1,"last_destroyed":1,"pending_noop":[],"snaps":[],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2024-09-21T07:50:02.848 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 1, "last_destroyed": 1, "last_snap": 1, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [] } 2024-09-21T07:50:02.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:50:02.848 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d0/d1/dir/.snap/s1 d0/d1/dir/.snap/s2) 2024-09-21T07:50:02.936 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:03.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.162+0000 7f03b2e00640 1 -- 172.21.15.39:0/2320282237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03ac0b59e0 msgr2=0x7f03ac0b3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:03.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.162+0000 7f03b2e00640 1 --2- 172.21.15.39:0/2320282237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03ac0b59e0 0x7f03ac0b3b60 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f03ac0d2570 tx=0x7f039802a540 comp rx=0 tx=0).stop 2024-09-21T07:50:03.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.162+0000 7f03b2e00640 1 -- 172.21.15.39:0/2320282237 shutdown_connections 2024-09-21T07:50:03.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.162+0000 7f03b2e00640 1 --2- 172.21.15.39:0/2320282237 >> v2:172.21.15.148:3300/0 conn(0x7f03ac0d0e60 0x7f03ac0d1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:03.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.162+0000 7f03b2e00640 1 --2- 172.21.15.39:0/2320282237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03ac0b59e0 0x7f03ac0b3b60 unknown :-1 s=CLOSED pgs=2283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:03.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.162+0000 7f03b2e00640 1 --2- 172.21.15.39:0/2320282237 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03ac0d5060 0x7f03ac0b96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:03.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.162+0000 7f03b2e00640 1 -- 172.21.15.39:0/2320282237 >> 172.21.15.39:0/2320282237 conn(0x7f03ac09d7d0 msgr2=0x7f03ac0d4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:03.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.162+0000 7f03b2e00640 1 -- 172.21.15.39:0/2320282237 shutdown_connections 2024-09-21T07:50:03.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.162+0000 7f03b2e00640 1 -- 172.21.15.39:0/2320282237 wait complete. 2024-09-21T07:50:03.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.164+0000 7f03b2e00640 1 Processor -- start 2024-09-21T07:50:03.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.165+0000 7f03b2e00640 1 -- start start 2024-09-21T07:50:03.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.166+0000 7f03b2e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03ac0b59e0 0x7f03ac14d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:03.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.166+0000 7f03b2e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03ac0d0e60 0x7f03ac0ca440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:03.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.166+0000 7f03b2e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03ac0d5060 0x7f03ac0b42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:03.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.166+0000 7f03b2e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac0d2880 con 0x7f03ac0d0e60 2024-09-21T07:50:03.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.166+0000 7f03b2e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac0d0380 con 0x7f03ac0d5060 2024-09-21T07:50:03.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.166+0000 7f03b2e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f03ac0d2a00 con 0x7f03ac0b59e0 2024-09-21T07:50:03.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.166+0000 7f03b1000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03ac0d0e60 0x7f03ac0ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:03.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.166+0000 7f03b1000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03ac0d0e60 0x7f03ac0ca440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39554/0 (socket says 172.21.15.39:39554) 2024-09-21T07:50:03.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.166+0000 7f03b1000640 1 -- 172.21.15.39:0/4161216891 learned_addr learned my addr 172.21.15.39:0/4161216891 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:03.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.166+0000 7f03b1a00640 1 --2- 172.21.15.39:0/4161216891 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03ac0b59e0 0x7f03ac14d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:03.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.167+0000 7f03b2400640 1 --2- 172.21.15.39:0/4161216891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03ac0d5060 0x7f03ac0b42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:03.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.167+0000 7f03b1000640 1 -- 172.21.15.39:0/4161216891 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03ac0b59e0 msgr2=0x7f03ac14d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:03.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.167+0000 7f03b1000640 1 --2- 172.21.15.39:0/4161216891 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03ac0b59e0 0x7f03ac14d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:03.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.167+0000 7f03b1000640 1 -- 172.21.15.39:0/4161216891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03ac0d5060 msgr2=0x7f03ac0b42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:03.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.167+0000 7f03b1000640 1 --2- 172.21.15.39:0/4161216891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03ac0d5060 0x7f03ac0b42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:03.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.167+0000 7f03b1000640 1 -- 172.21.15.39:0/4161216891 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03ac15e5a0 con 0x7f03ac0d0e60 2024-09-21T07:50:03.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.167+0000 7f03b1000640 1 --2- 172.21.15.39:0/4161216891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03ac0d0e60 0x7f03ac0ca440 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f0398034040 tx=0x7f0398003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:03.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.167+0000 7f03a9600640 1 -- 172.21.15.39:0/4161216891 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039803e030 con 0x7f03ac0d0e60 2024-09-21T07:50:03.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.168+0000 7f03a9600640 1 -- 172.21.15.39:0/4161216891 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0398003e00 con 0x7f03ac0d0e60 2024-09-21T07:50:03.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.168+0000 7f03a9600640 1 -- 172.21.15.39:0/4161216891 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0398035040 con 0x7f03ac0d0e60 2024-09-21T07:50:03.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.168+0000 7f03b2e00640 1 -- 172.21.15.39:0/4161216891 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03ac15e880 con 0x7f03ac0d0e60 2024-09-21T07:50:03.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.168+0000 7f03b2e00640 1 -- 172.21.15.39:0/4161216891 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03ac15ee40 con 0x7f03ac0d0e60 2024-09-21T07:50:03.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.169+0000 7f03b2e00640 1 -- 172.21.15.39:0/4161216891 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03ac0d1d40 con 0x7f03ac0d0e60 2024-09-21T07:50:03.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.170+0000 7f03a9600640 1 -- 172.21.15.39:0/4161216891 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0398041030 con 0x7f03ac0d0e60 2024-09-21T07:50:03.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.171+0000 7f03a9600640 1 --2- 172.21.15.39:0/4161216891 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f038c078b90 0x7f038c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:03.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.171+0000 7f03a9600640 1 -- 172.21.15.39:0/4161216891 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(235..235 src has 1..235) ==== 7132+0+0 (secure 0 0 0) 0x7f03980bbae0 con 0x7f03ac0d0e60 2024-09-21T07:50:03.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.171+0000 7f03b1a00640 1 --2- 172.21.15.39:0/4161216891 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f038c078b90 0x7f038c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:03.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.172+0000 7f03b1a00640 1 --2- 172.21.15.39:0/4161216891 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f038c078b90 0x7f038c07b0a0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f03a000bab0 tx=0x7f03a000b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:03.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.175+0000 7f03a9600640 1 -- 172.21.15.39:0/4161216891 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0398083330 con 0x7f03ac0d0e60 2024-09-21T07:50:03.311 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:50:03.315 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:50:03.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.410+0000 7f03b2e00640 1 -- 172.21.15.39:0/4161216891 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03ac0d14a0 con 0x7f03ac0d0e60 2024-09-21T07:50:03.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.411+0000 7f03a9600640 1 -- 172.21.15.39:0/4161216891 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 251 v251) ==== 95+0+5289 (secure 0 0 0) 0x7f0398087d60 con 0x7f03ac0d0e60 2024-09-21T07:50:03.413 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:50:03.413 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":251,"btime":"2024-09-21T07:50:00:589303+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8531,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1438851059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1438851059},{"type":"v1","addr":"172.21.15.148:6837","nonce":1438851059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":239}],"filesystems":[{"mdsmap":{"epoch":251,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:50:00.121724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":10635,"mds_1":10632,"mds_2":9709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9709":{"gid":9709,"name":"b","rank":2,"incarnation":249,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3497801035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3497801035},{"type":"v1","addr":"172.21.15.148:6836","nonce":3497801035}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10632":{"gid":10632,"name":"a","rank":1,"incarnation":247,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6834/4143807732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4143807732},{"type":"v1","addr":"172.21.15.39:6834","nonce":4143807732}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10635":{"gid":10635,"name":"c","rank":0,"incarnation":240,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/2824493631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2824493631},{"type":"v1","addr":"172.21.15.39:6835","nonce":2824493631}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10635,"qdb_cluster":[10635,10632,9709]},"id":13}]} 2024-09-21T07:50:03.414 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 251 2024-09-21T07:50:03.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.413+0000 7f03b2e00640 1 -- 172.21.15.39:0/4161216891 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f038c078b90 msgr2=0x7f038c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:03.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.413+0000 7f03b2e00640 1 --2- 172.21.15.39:0/4161216891 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f038c078b90 0x7f038c07b0a0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f03a000bab0 tx=0x7f03a000b2e0 comp rx=0 tx=0).stop 2024-09-21T07:50:03.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.413+0000 7f03b2e00640 1 -- 172.21.15.39:0/4161216891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03ac0d0e60 msgr2=0x7f03ac0ca440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:03.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.413+0000 7f03b2e00640 1 --2- 172.21.15.39:0/4161216891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03ac0d0e60 0x7f03ac0ca440 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f0398034040 tx=0x7f0398003910 comp rx=0 tx=0).stop 2024-09-21T07:50:03.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.413+0000 7f03b2e00640 1 -- 172.21.15.39:0/4161216891 shutdown_connections 2024-09-21T07:50:03.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.413+0000 7f03b2e00640 1 --2- 172.21.15.39:0/4161216891 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f038c078b90 0x7f038c07b0a0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:03.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.413+0000 7f03b2e00640 1 --2- 172.21.15.39:0/4161216891 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03ac0d5060 0x7f03ac0b42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:03.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.413+0000 7f03b2e00640 1 --2- 172.21.15.39:0/4161216891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03ac0d0e60 0x7f03ac0ca440 unknown :-1 s=CLOSED pgs=2284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:03.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.413+0000 7f03b2e00640 1 --2- 172.21.15.39:0/4161216891 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03ac0b59e0 0x7f03ac14d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:03.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.413+0000 7f03b2e00640 1 -- 172.21.15.39:0/4161216891 >> 172.21.15.39:0/4161216891 conn(0x7f03ac09d7d0 msgr2=0x7f03ac160d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:03.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.414+0000 7f03b2e00640 1 -- 172.21.15.39:0/4161216891 shutdown_connections 2024-09-21T07:50:03.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:03.414+0000 7f03b2e00640 1 -- 172.21.15.39:0/4161216891 wait complete. 2024-09-21T07:50:03.441 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2024-09-21T07:50:03.726 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_snap":3,"last_created":2,"last_destroyed":1,"pending_noop":[],"snaps":[{"snapid":2,"ino":1099511627778,"stamp":"2024-09-21T07:50:02.883862+0000","name":"s1","metadata":{}}],"need_to_purge":{},"pending_update":[{"version":4,"snapinfo":{"snapid":3,"ino":1099511627778,"stamp":"2024-09-21T07:50:02.909971+0000","name":"s2","metadata":{}}}],"pending_destroy":[]} 2024-09-21T07:50:03.727 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 2, "last_destroyed": 1, "last_snap": 3, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [ { "snapInfinityo": { "ino": 1099511627778, "metadata": {}, "name": "s2", "snapid": 3, "stamp": "2024-09-21T07:50:02.909971+0000" }, "version": 4 } ], "snaps": [ { "ino": 1099511627778, "metadata": {}, "name": "s1", "snapid": 2, "stamp": "2024-09-21T07:50:02.883862+0000" } ] } 2024-09-21T07:50:03.727 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-09-21T07:50:08.728 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:08.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:08.965+0000 7f0ca8800640 1 -- 172.21.15.39:0/52149343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c88003a50 msgr2=0x7f0c88003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:08.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:08.965+0000 7f0ca8800640 1 --2- 172.21.15.39:0/52149343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c88003a50 0x7f0c88003ec0 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f0c94009b50 tx=0x7f0c88005830 comp rx=0 tx=0).stop 2024-09-21T07:50:08.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:08.965+0000 7f0ca8800640 1 -- 172.21.15.39:0/52149343 shutdown_connections 2024-09-21T07:50:08.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:08.965+0000 7f0ca8800640 1 --2- 172.21.15.39:0/52149343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c88003a50 0x7f0c88003ec0 unknown :-1 s=CLOSED pgs=2286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:08.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:08.965+0000 7f0ca8800640 1 -- 172.21.15.39:0/52149343 >> 172.21.15.39:0/52149343 conn(0x7f0ca00b2460 msgr2=0x7f0ca006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:08.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:08.965+0000 7f0ca8800640 1 -- 172.21.15.39:0/52149343 shutdown_connections 2024-09-21T07:50:08.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:08.965+0000 7f0ca8800640 1 -- 172.21.15.39:0/52149343 wait complete. 2024-09-21T07:50:08.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:08.967+0000 7f0ca8800640 1 Processor -- start 2024-09-21T07:50:08.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:08.968+0000 7f0ca8800640 1 -- start start 2024-09-21T07:50:08.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:08.969+0000 7f0ca8800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c88003a50 0x7f0ca01559c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:08.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:08.969+0000 7f0ca8800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ca00d08a0 0x7f0ca00ad060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:08.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:08.969+0000 7f0ca8800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ca00ad5a0 0x7f0ca00af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:08.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:08.969+0000 7f0ca8800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca0156bb0 con 0x7f0ca00d08a0 2024-09-21T07:50:08.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:08.969+0000 7f0ca8800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca00cbbc0 con 0x7f0ca00ad5a0 2024-09-21T07:50:08.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:08.969+0000 7f0ca8800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f0ca0156d30 con 0x7f0c88003a50 2024-09-21T07:50:08.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:08.969+0000 7f0ca5400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ca00d08a0 0x7f0ca00ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:08.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:08.969+0000 7f0ca5400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ca00d08a0 0x7f0ca00ad060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39572/0 (socket says 172.21.15.39:39572) 2024-09-21T07:50:08.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:08.969+0000 7f0ca5400640 1 -- 172.21.15.39:0/3734760665 learned_addr learned my addr 172.21.15.39:0/3734760665 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:08.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:08.969+0000 7f0ca5e00640 1 --2- 172.21.15.39:0/3734760665 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c88003a50 0x7f0ca01559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:08.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:08.969+0000 7f0ca6800640 1 --2- 172.21.15.39:0/3734760665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ca00ad5a0 0x7f0ca00af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:08.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:08.970+0000 7f0ca5400640 1 -- 172.21.15.39:0/3734760665 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c88003a50 msgr2=0x7f0ca01559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:08.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:08.970+0000 7f0ca5400640 1 --2- 172.21.15.39:0/3734760665 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c88003a50 0x7f0ca01559c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:08.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:08.970+0000 7f0ca5400640 1 -- 172.21.15.39:0/3734760665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ca00ad5a0 msgr2=0x7f0ca00af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:08.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:08.970+0000 7f0ca5400640 1 --2- 172.21.15.39:0/3734760665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ca00ad5a0 0x7f0ca00af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:08.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:08.970+0000 7f0ca5400640 1 -- 172.21.15.39:0/3734760665 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca0155f00 con 0x7f0ca00d08a0 2024-09-21T07:50:08.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:08.970+0000 7f0ca5400640 1 --2- 172.21.15.39:0/3734760665 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ca00d08a0 0x7f0ca00ad060 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7f0c9000e570 tx=0x7f0c9000ea30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:08.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:08.970+0000 7f0c9a000640 1 -- 172.21.15.39:0/3734760665 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c90015070 con 0x7f0ca00d08a0 2024-09-21T07:50:08.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:08.970+0000 7f0c9a000640 1 -- 172.21.15.39:0/3734760665 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c9000ca00 con 0x7f0ca00d08a0 2024-09-21T07:50:08.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:08.970+0000 7f0ca8800640 1 -- 172.21.15.39:0/3734760665 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ca0156540 con 0x7f0ca00d08a0 2024-09-21T07:50:08.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:08.970+0000 7f0c9a000640 1 -- 172.21.15.39:0/3734760665 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9000ce50 con 0x7f0ca00d08a0 2024-09-21T07:50:08.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:08.970+0000 7f0ca8800640 1 -- 172.21.15.39:0/3734760665 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ca01f3ff0 con 0x7f0ca00d08a0 2024-09-21T07:50:08.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:08.972+0000 7f0ca8800640 1 -- 172.21.15.39:0/3734760665 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ca00cd070 con 0x7f0ca00d08a0 2024-09-21T07:50:08.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:08.973+0000 7f0c9a000640 1 -- 172.21.15.39:0/3734760665 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0c90018070 con 0x7f0ca00d08a0 2024-09-21T07:50:08.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:08.975+0000 7f0c9a000640 1 --2- 172.21.15.39:0/3734760665 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0c84078d30 0x7f0c8407b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:08.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:08.975+0000 7f0c9a000640 1 -- 172.21.15.39:0/3734760665 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(235..235 src has 1..235) ==== 7132+0+0 (secure 0 0 0) 0x7f0c9009b6e0 con 0x7f0ca00d08a0 2024-09-21T07:50:08.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:08.976+0000 7f0ca5e00640 1 --2- 172.21.15.39:0/3734760665 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0c84078d30 0x7f0c8407b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:08.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:08.976+0000 7f0ca5e00640 1 --2- 172.21.15.39:0/3734760665 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0c84078d30 0x7f0c8407b240 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f0c94035000 tx=0x7f0c940046c0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:08.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:08.980+0000 7f0c9a000640 1 -- 172.21.15.39:0/3734760665 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0c90062f30 con 0x7f0ca00d08a0 2024-09-21T07:50:09.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.216+0000 7f0ca8800640 1 -- 172.21.15.39:0/3734760665 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ca015e470 con 0x7f0ca00d08a0 2024-09-21T07:50:09.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.216+0000 7f0c9a000640 1 -- 172.21.15.39:0/3734760665 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 251 v251) ==== 95+0+5289 (secure 0 0 0) 0x7f0c90067960 con 0x7f0ca00d08a0 2024-09-21T07:50:09.219 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:50:09.219 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":251,"btime":"2024-09-21T07:50:00:589303+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8531,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1438851059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1438851059},{"type":"v1","addr":"172.21.15.148:6837","nonce":1438851059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":239}],"filesystems":[{"mdsmap":{"epoch":251,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:50:00.121724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":10635,"mds_1":10632,"mds_2":9709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9709":{"gid":9709,"name":"b","rank":2,"incarnation":249,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3497801035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3497801035},{"type":"v1","addr":"172.21.15.148:6836","nonce":3497801035}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10632":{"gid":10632,"name":"a","rank":1,"incarnation":247,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6834/4143807732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4143807732},{"type":"v1","addr":"172.21.15.39:6834","nonce":4143807732}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10635":{"gid":10635,"name":"c","rank":0,"incarnation":240,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/2824493631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2824493631},{"type":"v1","addr":"172.21.15.39:6835","nonce":2824493631}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10635,"qdb_cluster":[10635,10632,9709]},"id":13}]} 2024-09-21T07:50:09.219 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 251 2024-09-21T07:50:09.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.219+0000 7f0ca8800640 1 -- 172.21.15.39:0/3734760665 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0c84078d30 msgr2=0x7f0c8407b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:09.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.219+0000 7f0ca8800640 1 --2- 172.21.15.39:0/3734760665 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0c84078d30 0x7f0c8407b240 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f0c94035000 tx=0x7f0c940046c0 comp rx=0 tx=0).stop 2024-09-21T07:50:09.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.219+0000 7f0ca8800640 1 -- 172.21.15.39:0/3734760665 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ca00d08a0 msgr2=0x7f0ca00ad060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:09.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.219+0000 7f0ca8800640 1 --2- 172.21.15.39:0/3734760665 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ca00d08a0 0x7f0ca00ad060 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7f0c9000e570 tx=0x7f0c9000ea30 comp rx=0 tx=0).stop 2024-09-21T07:50:09.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.219+0000 7f0ca8800640 1 -- 172.21.15.39:0/3734760665 shutdown_connections 2024-09-21T07:50:09.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.219+0000 7f0ca8800640 1 --2- 172.21.15.39:0/3734760665 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0c84078d30 0x7f0c8407b240 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:09.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.219+0000 7f0ca8800640 1 --2- 172.21.15.39:0/3734760665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ca00ad5a0 0x7f0ca00af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:09.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.219+0000 7f0ca8800640 1 --2- 172.21.15.39:0/3734760665 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ca00d08a0 0x7f0ca00ad060 unknown :-1 s=CLOSED pgs=2287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:09.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.219+0000 7f0ca8800640 1 --2- 172.21.15.39:0/3734760665 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0c88003a50 0x7f0ca01559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:09.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.219+0000 7f0ca8800640 1 -- 172.21.15.39:0/3734760665 >> 172.21.15.39:0/3734760665 conn(0x7f0ca00b2460 msgr2=0x7f0ca00d0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:09.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.219+0000 7f0ca8800640 1 -- 172.21.15.39:0/3734760665 shutdown_connections 2024-09-21T07:50:09.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.219+0000 7f0ca8800640 1 -- 172.21.15.39:0/3734760665 wait complete. 2024-09-21T07:50:09.243 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2024-09-21T07:50:09.487 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_snap":3,"last_created":3,"last_destroyed":1,"pending_noop":[],"snaps":[{"snapid":2,"ino":1099511627778,"stamp":"2024-09-21T07:50:02.883862+0000","name":"s1","metadata":{}},{"snapid":3,"ino":1099511627778,"stamp":"2024-09-21T07:50:02.909971+0000","name":"s2","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2024-09-21T07:50:09.487 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 3, "last_destroyed": 1, "last_snap": 3, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627778, "metadata": {}, "name": "s1", "snapid": 2, "stamp": "2024-09-21T07:50:02.883862+0000" }, { "ino": 1099511627778, "metadata": {}, "name": "s2", "snapid": 3, "stamp": "2024-09-21T07:50:02.909971+0000" } ] } 2024-09-21T07:50:09.488 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-21T07:50:09.488 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:09.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.714+0000 7f05dd000640 1 -- 172.21.15.39:0/1668301096 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05bc006430 msgr2=0x7f05bc00caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:09.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.714+0000 7f05dd000640 1 --2- 172.21.15.39:0/1668301096 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05bc006430 0x7f05bc00caf0 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f05d0009ed0 tx=0x7f05bc006940 comp rx=0 tx=0).stop 2024-09-21T07:50:09.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.715+0000 7f05dd000640 1 -- 172.21.15.39:0/1668301096 shutdown_connections 2024-09-21T07:50:09.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.715+0000 7f05dd000640 1 --2- 172.21.15.39:0/1668301096 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05bc006430 0x7f05bc00caf0 unknown :-1 s=CLOSED pgs=2289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:09.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.715+0000 7f05dd000640 1 -- 172.21.15.39:0/1668301096 >> 172.21.15.39:0/1668301096 conn(0x7f05d806cfc0 msgr2=0x7f05d806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:09.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.715+0000 7f05dd000640 1 -- 172.21.15.39:0/1668301096 shutdown_connections 2024-09-21T07:50:09.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.715+0000 7f05dd000640 1 -- 172.21.15.39:0/1668301096 wait complete. 2024-09-21T07:50:09.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.717+0000 7f05dd000640 1 Processor -- start 2024-09-21T07:50:09.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.717+0000 7f05dd000640 1 -- start start 2024-09-21T07:50:09.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.718+0000 7f05dd000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05bc006430 0x7f05d8137940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:09.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.718+0000 7f05dd000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f05d80b0e20 0x7f05d815d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:09.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.718+0000 7f05dd000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05d80aff30 0x7f05d815de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:09.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.718+0000 7f05dd000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f05d8151640 con 0x7f05d80aff30 2024-09-21T07:50:09.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.718+0000 7f05dd000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f05d80d0270 con 0x7f05bc006430 2024-09-21T07:50:09.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.718+0000 7f05dd000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f05d81517c0 con 0x7f05d80b0e20 2024-09-21T07:50:09.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.719+0000 7f05d6800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05d80aff30 0x7f05d815de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:09.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.719+0000 7f05d6800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05d80aff30 0x7f05d815de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39592/0 (socket says 172.21.15.39:39592) 2024-09-21T07:50:09.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.719+0000 7f05d6800640 1 -- 172.21.15.39:0/3972304701 learned_addr learned my addr 172.21.15.39:0/3972304701 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:09.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.719+0000 7f05d5e00640 1 --2- 172.21.15.39:0/3972304701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05bc006430 0x7f05d8137940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:09.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.719+0000 7f05d5400640 1 --2- 172.21.15.39:0/3972304701 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f05d80b0e20 0x7f05d815d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:09.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.719+0000 7f05d6800640 1 -- 172.21.15.39:0/3972304701 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f05d80b0e20 msgr2=0x7f05d815d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:09.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.719+0000 7f05d6800640 1 --2- 172.21.15.39:0/3972304701 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f05d80b0e20 0x7f05d815d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:09.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.719+0000 7f05d6800640 1 -- 172.21.15.39:0/3972304701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05bc006430 msgr2=0x7f05d8137940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:09.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.719+0000 7f05d6800640 1 --2- 172.21.15.39:0/3972304701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05bc006430 0x7f05d8137940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:09.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.719+0000 7f05d6800640 1 -- 172.21.15.39:0/3972304701 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05d8138330 con 0x7f05d80aff30 2024-09-21T07:50:09.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.720+0000 7f05d6800640 1 --2- 172.21.15.39:0/3972304701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05d80aff30 0x7f05d815de00 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7f05d0009e20 tx=0x7f05d0034ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:09.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.720+0000 7f05ce000640 1 -- 172.21.15.39:0/3972304701 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d0009200 con 0x7f05d80aff30 2024-09-21T07:50:09.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.720+0000 7f05ce000640 1 -- 172.21.15.39:0/3972304701 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05d0040070 con 0x7f05d80aff30 2024-09-21T07:50:09.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.720+0000 7f05ce000640 1 -- 172.21.15.39:0/3972304701 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d0009ad0 con 0x7f05d80aff30 2024-09-21T07:50:09.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.720+0000 7f05dd000640 1 -- 172.21.15.39:0/3972304701 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05d80b1550 con 0x7f05d80aff30 2024-09-21T07:50:09.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.720+0000 7f05dd000640 1 -- 172.21.15.39:0/3972304701 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05d8138880 con 0x7f05d80aff30 2024-09-21T07:50:09.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.721+0000 7f05dd000640 1 -- 172.21.15.39:0/3972304701 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05d80b12c0 con 0x7f05d80aff30 2024-09-21T07:50:09.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.723+0000 7f05ce000640 1 -- 172.21.15.39:0/3972304701 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f05d0032850 con 0x7f05d80aff30 2024-09-21T07:50:09.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.724+0000 7f05ce000640 1 --2- 172.21.15.39:0/3972304701 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f05b8078b90 0x7f05b807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:09.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.724+0000 7f05ce000640 1 -- 172.21.15.39:0/3972304701 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(235..235 src has 1..235) ==== 7132+0+0 (secure 0 0 0) 0x7f05d00bc320 con 0x7f05d80aff30 2024-09-21T07:50:09.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.724+0000 7f05d5e00640 1 --2- 172.21.15.39:0/3972304701 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f05b8078b90 0x7f05b807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:09.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.725+0000 7f05d5e00640 1 --2- 172.21.15.39:0/3972304701 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f05b8078b90 0x7f05b807b0a0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f05c8001850 tx=0x7f05c8006720 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:09.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.728+0000 7f05ce000640 1 -- 172.21.15.39:0/3972304701 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f05d0084130 con 0x7f05d80aff30 2024-09-21T07:50:09.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.967+0000 7f05dd000640 1 -- 172.21.15.39:0/3972304701 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05d8070570 con 0x7f05d80aff30 2024-09-21T07:50:09.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.968+0000 7f05ce000640 1 -- 172.21.15.39:0/3972304701 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 251 v251) ==== 95+0+5289 (secure 0 0 0) 0x7f05d00bc0e0 con 0x7f05d80aff30 2024-09-21T07:50:09.970 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:50:09.970 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":251,"btime":"2024-09-21T07:50:00:589303+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8531,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1438851059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1438851059},{"type":"v1","addr":"172.21.15.148:6837","nonce":1438851059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":239}],"filesystems":[{"mdsmap":{"epoch":251,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:50:00.121724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":10635,"mds_1":10632,"mds_2":9709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9709":{"gid":9709,"name":"b","rank":2,"incarnation":249,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3497801035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3497801035},{"type":"v1","addr":"172.21.15.148:6836","nonce":3497801035}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10632":{"gid":10632,"name":"a","rank":1,"incarnation":247,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6834/4143807732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4143807732},{"type":"v1","addr":"172.21.15.39:6834","nonce":4143807732}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10635":{"gid":10635,"name":"c","rank":0,"incarnation":240,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/2824493631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2824493631},{"type":"v1","addr":"172.21.15.39:6835","nonce":2824493631}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10635,"qdb_cluster":[10635,10632,9709]},"id":13}]} 2024-09-21T07:50:09.971 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 251 2024-09-21T07:50:09.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.971+0000 7f05dd000640 1 -- 172.21.15.39:0/3972304701 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f05b8078b90 msgr2=0x7f05b807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:09.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.971+0000 7f05dd000640 1 --2- 172.21.15.39:0/3972304701 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f05b8078b90 0x7f05b807b0a0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f05c8001850 tx=0x7f05c8006720 comp rx=0 tx=0).stop 2024-09-21T07:50:09.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.971+0000 7f05dd000640 1 -- 172.21.15.39:0/3972304701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05d80aff30 msgr2=0x7f05d815de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:09.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.971+0000 7f05dd000640 1 --2- 172.21.15.39:0/3972304701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05d80aff30 0x7f05d815de00 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7f05d0009e20 tx=0x7f05d0034ca0 comp rx=0 tx=0).stop 2024-09-21T07:50:09.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.971+0000 7f05dd000640 1 -- 172.21.15.39:0/3972304701 shutdown_connections 2024-09-21T07:50:09.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.971+0000 7f05dd000640 1 --2- 172.21.15.39:0/3972304701 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f05b8078b90 0x7f05b807b0a0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:09.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.971+0000 7f05dd000640 1 --2- 172.21.15.39:0/3972304701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05d80aff30 0x7f05d815de00 unknown :-1 s=CLOSED pgs=2290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:09.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.971+0000 7f05dd000640 1 --2- 172.21.15.39:0/3972304701 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f05d80b0e20 0x7f05d815d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:09.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.971+0000 7f05dd000640 1 --2- 172.21.15.39:0/3972304701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05bc006430 0x7f05d8137940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:09.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.971+0000 7f05dd000640 1 -- 172.21.15.39:0/3972304701 >> 172.21.15.39:0/3972304701 conn(0x7f05d806cfc0 msgr2=0x7f05d80b33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:09.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.971+0000 7f05dd000640 1 -- 172.21.15.39:0/3972304701 shutdown_connections 2024-09-21T07:50:09.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:09.971+0000 7f05dd000640 1 -- 172.21.15.39:0/3972304701 wait complete. 2024-09-21T07:50:09.998 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2024-09-21T07:50:10.243 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_snap":3,"last_created":3,"last_destroyed":1,"pending_noop":[],"snaps":[{"snapid":2,"ino":1099511627778,"stamp":"2024-09-21T07:50:02.883862+0000","name":"s1","metadata":{}},{"snapid":3,"ino":1099511627778,"stamp":"2024-09-21T07:50:02.909971+0000","name":"s2","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2024-09-21T07:50:10.243 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 3, "last_destroyed": 1, "last_snap": 3, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627778, "metadata": {}, "name": "s1", "snapid": 2, "stamp": "2024-09-21T07:50:02.883862+0000" }, { "ino": 1099511627778, "metadata": {}, "name": "s2", "snapid": 3, "stamp": "2024-09-21T07:50:02.909971+0000" } ] } 2024-09-21T07:50:10.244 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:10.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.468+0000 7fdcdf200640 1 -- 172.21.15.39:0/973821681 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdce015d4f0 msgr2=0x7fdce006cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:10.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.468+0000 7fdcdf200640 1 --2- 172.21.15.39:0/973821681 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdce015d4f0 0x7fdce006cde0 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7fdccc005fd0 tx=0x7fdccc02a540 comp rx=0 tx=0).stop 2024-09-21T07:50:10.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.468+0000 7fdcdf200640 1 -- 172.21.15.39:0/973821681 shutdown_connections 2024-09-21T07:50:10.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.468+0000 7fdcdf200640 1 --2- 172.21.15.39:0/973821681 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdce015d4f0 0x7fdce006cde0 unknown :-1 s=CLOSED pgs=2291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:10.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.468+0000 7fdcdf200640 1 --2- 172.21.15.39:0/973821681 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdce00d01b0 0x7fdce00cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:10.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.468+0000 7fdcdf200640 1 -- 172.21.15.39:0/973821681 >> 172.21.15.39:0/973821681 conn(0x7fdce00b6f80 msgr2=0x7fdce006def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:10.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.468+0000 7fdcdf200640 1 -- 172.21.15.39:0/973821681 shutdown_connections 2024-09-21T07:50:10.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.468+0000 7fdcdf200640 1 -- 172.21.15.39:0/973821681 wait complete. 2024-09-21T07:50:10.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.470+0000 7fdcdf200640 1 Processor -- start 2024-09-21T07:50:10.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.470+0000 7fdcdf200640 1 -- start start 2024-09-21T07:50:10.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.472+0000 7fdcdf200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdce00d01b0 0x7fdce0152930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:10.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.472+0000 7fdcdf200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdce015d4f0 0x7fdce0158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:10.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.472+0000 7fdcdf200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdce015d900 0x7fdce01594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:10.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.472+0000 7fdcdf200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdce0160920 con 0x7fdce015d4f0 2024-09-21T07:50:10.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.472+0000 7fdcdf200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdce006d6d0 con 0x7fdce015d900 2024-09-21T07:50:10.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.472+0000 7fdcdf200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fdce0160aa0 con 0x7fdce00d01b0 2024-09-21T07:50:10.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.472+0000 7fdcdd400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdce015d4f0 0x7fdce0158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:10.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.472+0000 7fdcdd400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdce015d4f0 0x7fdce0158f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39614/0 (socket says 172.21.15.39:39614) 2024-09-21T07:50:10.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.472+0000 7fdcdd400640 1 -- 172.21.15.39:0/3122525743 learned_addr learned my addr 172.21.15.39:0/3122525743 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:10.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.472+0000 7fdcde800640 1 --2- 172.21.15.39:0/3122525743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdce015d900 0x7fdce01594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:10.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.472+0000 7fdcdde00640 1 --2- 172.21.15.39:0/3122525743 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdce00d01b0 0x7fdce0152930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:10.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.473+0000 7fdcdd400640 1 -- 172.21.15.39:0/3122525743 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdce00d01b0 msgr2=0x7fdce0152930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:10.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.473+0000 7fdcdd400640 1 --2- 172.21.15.39:0/3122525743 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdce00d01b0 0x7fdce0152930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:10.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.473+0000 7fdcdd400640 1 -- 172.21.15.39:0/3122525743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdce015d900 msgr2=0x7fdce01594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:10.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.473+0000 7fdcdd400640 1 --2- 172.21.15.39:0/3122525743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdce015d900 0x7fdce01594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:10.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.473+0000 7fdcdd400640 1 -- 172.21.15.39:0/3122525743 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdce0159dc0 con 0x7fdce015d4f0 2024-09-21T07:50:10.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.473+0000 7fdcdd400640 1 --2- 172.21.15.39:0/3122525743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdce015d4f0 0x7fdce0158f60 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7fdccc006010 tx=0x7fdccc0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:10.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.473+0000 7fdcd6000640 1 -- 172.21.15.39:0/3122525743 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdccc03e030 con 0x7fdce015d4f0 2024-09-21T07:50:10.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.473+0000 7fdcd6000640 1 -- 172.21.15.39:0/3122525743 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdccc003d90 con 0x7fdce015d4f0 2024-09-21T07:50:10.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.473+0000 7fdcdf200640 1 -- 172.21.15.39:0/3122525743 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdce01538a0 con 0x7fdce015d4f0 2024-09-21T07:50:10.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.473+0000 7fdcd6000640 1 -- 172.21.15.39:0/3122525743 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdccc035040 con 0x7fdce015d4f0 2024-09-21T07:50:10.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.473+0000 7fdcdf200640 1 -- 172.21.15.39:0/3122525743 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdce00b1dd0 con 0x7fdce015d4f0 2024-09-21T07:50:10.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.474+0000 7fdcdf200640 1 -- 172.21.15.39:0/3122525743 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdce00cda00 con 0x7fdce015d4f0 2024-09-21T07:50:10.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.476+0000 7fdcd6000640 1 -- 172.21.15.39:0/3122525743 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdccc041030 con 0x7fdce015d4f0 2024-09-21T07:50:10.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.477+0000 7fdcd6000640 1 --2- 172.21.15.39:0/3122525743 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdcc0078bc0 0x7fdcc007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:10.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.477+0000 7fdcd6000640 1 -- 172.21.15.39:0/3122525743 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(235..235 src has 1..235) ==== 7132+0+0 (secure 0 0 0) 0x7fdccc0bbaf0 con 0x7fdce015d4f0 2024-09-21T07:50:10.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.478+0000 7fdcdde00640 1 --2- 172.21.15.39:0/3122525743 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdcc0078bc0 0x7fdcc007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:10.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.478+0000 7fdcdde00640 1 --2- 172.21.15.39:0/3122525743 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdcc0078bc0 0x7fdcc007b0d0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0004630 tx=0x7fdcd000b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:10.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.481+0000 7fdcd6000640 1 -- 172.21.15.39:0/3122525743 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdce0159dc0 con 0x7fdce015d4f0 2024-09-21T07:50:10.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.718+0000 7fdcdf200640 1 -- 172.21.15.39:0/3122525743 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdce00704f0 con 0x7fdce015d4f0 2024-09-21T07:50:10.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.718+0000 7fdcd6000640 1 -- 172.21.15.39:0/3122525743 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 251 v251) ==== 95+0+5289 (secure 0 0 0) 0x7fdccc0832b0 con 0x7fdce015d4f0 2024-09-21T07:50:10.720 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:50:10.721 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":251,"btime":"2024-09-21T07:50:00:589303+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8531,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1438851059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1438851059},{"type":"v1","addr":"172.21.15.148:6837","nonce":1438851059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":239}],"filesystems":[{"mdsmap":{"epoch":251,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:50:00.121724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":10635,"mds_1":10632,"mds_2":9709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9709":{"gid":9709,"name":"b","rank":2,"incarnation":249,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3497801035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3497801035},{"type":"v1","addr":"172.21.15.148:6836","nonce":3497801035}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10632":{"gid":10632,"name":"a","rank":1,"incarnation":247,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6834/4143807732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4143807732},{"type":"v1","addr":"172.21.15.39:6834","nonce":4143807732}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10635":{"gid":10635,"name":"c","rank":0,"incarnation":240,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/2824493631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2824493631},{"type":"v1","addr":"172.21.15.39:6835","nonce":2824493631}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10635,"qdb_cluster":[10635,10632,9709]},"id":13}]} 2024-09-21T07:50:10.721 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 251 2024-09-21T07:50:10.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.721+0000 7fdcdf200640 1 -- 172.21.15.39:0/3122525743 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdcc0078bc0 msgr2=0x7fdcc007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:10.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.721+0000 7fdcdf200640 1 --2- 172.21.15.39:0/3122525743 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdcc0078bc0 0x7fdcc007b0d0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0004630 tx=0x7fdcd000b2e0 comp rx=0 tx=0).stop 2024-09-21T07:50:10.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.721+0000 7fdcdf200640 1 -- 172.21.15.39:0/3122525743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdce015d4f0 msgr2=0x7fdce0158f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:10.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.721+0000 7fdcdf200640 1 --2- 172.21.15.39:0/3122525743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdce015d4f0 0x7fdce0158f60 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7fdccc006010 tx=0x7fdccc0038a0 comp rx=0 tx=0).stop 2024-09-21T07:50:10.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.721+0000 7fdcdf200640 1 -- 172.21.15.39:0/3122525743 shutdown_connections 2024-09-21T07:50:10.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.721+0000 7fdcdf200640 1 --2- 172.21.15.39:0/3122525743 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdcc0078bc0 0x7fdcc007b0d0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:10.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.721+0000 7fdcdf200640 1 --2- 172.21.15.39:0/3122525743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdce015d900 0x7fdce01594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:10.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.721+0000 7fdcdf200640 1 --2- 172.21.15.39:0/3122525743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdce015d4f0 0x7fdce0158f60 unknown :-1 s=CLOSED pgs=2292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:10.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.721+0000 7fdcdf200640 1 --2- 172.21.15.39:0/3122525743 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdce00d01b0 0x7fdce0152930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:10.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.721+0000 7fdcdf200640 1 -- 172.21.15.39:0/3122525743 >> 172.21.15.39:0/3122525743 conn(0x7fdce00b6f80 msgr2=0x7fdce00b5d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:10.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.721+0000 7fdcdf200640 1 -- 172.21.15.39:0/3122525743 shutdown_connections 2024-09-21T07:50:10.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:10.721+0000 7fdcdf200640 1 -- 172.21.15.39:0/3122525743 wait complete. 2024-09-21T07:50:10.749 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2024-09-21T07:50:10.993 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_snap":3,"last_created":3,"last_destroyed":1,"pending_noop":[],"snaps":[{"snapid":2,"ino":1099511627778,"stamp":"2024-09-21T07:50:02.883862+0000","name":"s1","metadata":{}},{"snapid":3,"ino":1099511627778,"stamp":"2024-09-21T07:50:02.909971+0000","name":"s2","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2024-09-21T07:50:10.994 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 3, "last_destroyed": 1, "last_snap": 3, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627778, "metadata": {}, "name": "s1", "snapid": 2, "stamp": "2024-09-21T07:50:02.883862+0000" }, { "ino": 1099511627778, "metadata": {}, "name": "s2", "snapid": 3, "stamp": "2024-09-21T07:50:02.909971+0000" } ] } 2024-09-21T07:50:10.994 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:11.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.218+0000 7fba2e600640 1 -- 172.21.15.39:0/1818342132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba28160250 msgr2=0x7fba2815d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:11.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.218+0000 7fba2e600640 1 --2- 172.21.15.39:0/1818342132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba28160250 0x7fba2815d570 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7fba1c005fd0 tx=0x7fba1c02a5d0 comp rx=0 tx=0).stop 2024-09-21T07:50:11.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.218+0000 7fba2e600640 1 -- 172.21.15.39:0/1818342132 shutdown_connections 2024-09-21T07:50:11.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.218+0000 7fba2e600640 1 --2- 172.21.15.39:0/1818342132 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fba280d0390 0x7fba280d07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:11.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.218+0000 7fba2e600640 1 --2- 172.21.15.39:0/1818342132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba28160250 0x7fba2815d570 unknown :-1 s=CLOSED pgs=2293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:11.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.218+0000 7fba2e600640 1 -- 172.21.15.39:0/1818342132 >> 172.21.15.39:0/1818342132 conn(0x7fba2806cfc0 msgr2=0x7fba2806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:11.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.218+0000 7fba2e600640 1 -- 172.21.15.39:0/1818342132 shutdown_connections 2024-09-21T07:50:11.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.218+0000 7fba2e600640 1 -- 172.21.15.39:0/1818342132 wait complete. 2024-09-21T07:50:11.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.220+0000 7fba2e600640 1 Processor -- start 2024-09-21T07:50:11.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.220+0000 7fba2e600640 1 -- start start 2024-09-21T07:50:11.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.222+0000 7fba2e600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba280d0390 0x7fba281f55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:11.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.222+0000 7fba2e600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fba28160250 0x7fba28137d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:11.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.222+0000 7fba2e600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba28160660 0x7fba28138270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:11.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.222+0000 7fba2e600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fba280ccdc0 con 0x7fba280d0390 2024-09-21T07:50:11.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.222+0000 7fba2e600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fba280ccc40 con 0x7fba28160660 2024-09-21T07:50:11.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.222+0000 7fba2e600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fba280ccf40 con 0x7fba28160250 2024-09-21T07:50:11.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.222+0000 7fba2d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba280d0390 0x7fba281f55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:11.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.222+0000 7fba2d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba280d0390 0x7fba281f55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39634/0 (socket says 172.21.15.39:39634) 2024-09-21T07:50:11.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.222+0000 7fba2d200640 1 -- 172.21.15.39:0/4293792619 learned_addr learned my addr 172.21.15.39:0/4293792619 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:11.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.222+0000 7fba2dc00640 1 --2- 172.21.15.39:0/4293792619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba28160660 0x7fba28138270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:11.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.222+0000 7fba27e00640 1 --2- 172.21.15.39:0/4293792619 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fba28160250 0x7fba28137d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:11.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.222+0000 7fba2d200640 1 -- 172.21.15.39:0/4293792619 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fba28160250 msgr2=0x7fba28137d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:11.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.222+0000 7fba2d200640 1 --2- 172.21.15.39:0/4293792619 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fba28160250 0x7fba28137d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:11.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.223+0000 7fba2d200640 1 -- 172.21.15.39:0/4293792619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba28160660 msgr2=0x7fba28138270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:11.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.223+0000 7fba2d200640 1 --2- 172.21.15.39:0/4293792619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba28160660 0x7fba28138270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:11.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.223+0000 7fba2d200640 1 -- 172.21.15.39:0/4293792619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba28145090 con 0x7fba280d0390 2024-09-21T07:50:11.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.223+0000 7fba2dc00640 1 --2- 172.21.15.39:0/4293792619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba28160660 0x7fba28138270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:50:11.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.223+0000 7fba2d200640 1 --2- 172.21.15.39:0/4293792619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba280d0390 0x7fba281f55b0 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7fba1c034040 tx=0x7fba1c0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:11.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.223+0000 7fba24c00640 1 -- 172.21.15.39:0/4293792619 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c03e030 con 0x7fba280d0390 2024-09-21T07:50:11.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.223+0000 7fba24c00640 1 -- 172.21.15.39:0/4293792619 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba1c003ae0 con 0x7fba280d0390 2024-09-21T07:50:11.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.223+0000 7fba2e600640 1 -- 172.21.15.39:0/4293792619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba280abc30 con 0x7fba280d0390 2024-09-21T07:50:11.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.223+0000 7fba24c00640 1 -- 172.21.15.39:0/4293792619 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c035040 con 0x7fba280d0390 2024-09-21T07:50:11.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.223+0000 7fba2e600640 1 -- 172.21.15.39:0/4293792619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba281360b0 con 0x7fba280d0390 2024-09-21T07:50:11.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.224+0000 7fba2e600640 1 -- 172.21.15.39:0/4293792619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba280abe00 con 0x7fba280d0390 2024-09-21T07:50:11.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.226+0000 7fba24c00640 1 -- 172.21.15.39:0/4293792619 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fba1c041030 con 0x7fba280d0390 2024-09-21T07:50:11.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.226+0000 7fba24c00640 1 --2- 172.21.15.39:0/4293792619 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fba08078b90 0x7fba0807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:11.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.227+0000 7fba24c00640 1 -- 172.21.15.39:0/4293792619 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(235..235 src has 1..235) ==== 7132+0+0 (secure 0 0 0) 0x7fba1c0bb890 con 0x7fba280d0390 2024-09-21T07:50:11.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.227+0000 7fba27e00640 1 --2- 172.21.15.39:0/4293792619 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fba08078b90 0x7fba0807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:11.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.228+0000 7fba27e00640 1 --2- 172.21.15.39:0/4293792619 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fba08078b90 0x7fba0807b0a0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fba28138de0 tx=0x7fba14009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:11.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.231+0000 7fba24c00640 1 -- 172.21.15.39:0/4293792619 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fba28145090 con 0x7fba280d0390 2024-09-21T07:50:11.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.467+0000 7fba2e600640 1 -- 172.21.15.39:0/4293792619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba280cd460 con 0x7fba280d0390 2024-09-21T07:50:11.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.467+0000 7fba24c00640 1 -- 172.21.15.39:0/4293792619 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 251 v251) ==== 95+0+5289 (secure 0 0 0) 0x7fba1c083730 con 0x7fba280d0390 2024-09-21T07:50:11.469 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:50:11.469 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":251,"btime":"2024-09-21T07:50:00:589303+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8531,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1438851059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1438851059},{"type":"v1","addr":"172.21.15.148:6837","nonce":1438851059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":239}],"filesystems":[{"mdsmap":{"epoch":251,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:50:00.121724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":10635,"mds_1":10632,"mds_2":9709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9709":{"gid":9709,"name":"b","rank":2,"incarnation":249,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3497801035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3497801035},{"type":"v1","addr":"172.21.15.148:6836","nonce":3497801035}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10632":{"gid":10632,"name":"a","rank":1,"incarnation":247,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6834/4143807732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4143807732},{"type":"v1","addr":"172.21.15.39:6834","nonce":4143807732}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10635":{"gid":10635,"name":"c","rank":0,"incarnation":240,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/2824493631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2824493631},{"type":"v1","addr":"172.21.15.39:6835","nonce":2824493631}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10635,"qdb_cluster":[10635,10632,9709]},"id":13}]} 2024-09-21T07:50:11.470 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 251 2024-09-21T07:50:11.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.469+0000 7fba2e600640 1 -- 172.21.15.39:0/4293792619 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fba08078b90 msgr2=0x7fba0807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:11.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.469+0000 7fba2e600640 1 --2- 172.21.15.39:0/4293792619 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fba08078b90 0x7fba0807b0a0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fba28138de0 tx=0x7fba14009040 comp rx=0 tx=0).stop 2024-09-21T07:50:11.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.469+0000 7fba2e600640 1 -- 172.21.15.39:0/4293792619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba280d0390 msgr2=0x7fba281f55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:11.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.469+0000 7fba2e600640 1 --2- 172.21.15.39:0/4293792619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba280d0390 0x7fba281f55b0 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7fba1c034040 tx=0x7fba1c0035f0 comp rx=0 tx=0).stop 2024-09-21T07:50:11.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.470+0000 7fba2e600640 1 -- 172.21.15.39:0/4293792619 shutdown_connections 2024-09-21T07:50:11.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.470+0000 7fba2e600640 1 --2- 172.21.15.39:0/4293792619 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fba08078b90 0x7fba0807b0a0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:11.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.470+0000 7fba2e600640 1 --2- 172.21.15.39:0/4293792619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba28160660 0x7fba28138270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:11.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.470+0000 7fba2e600640 1 --2- 172.21.15.39:0/4293792619 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fba28160250 0x7fba28137d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:11.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.470+0000 7fba2e600640 1 --2- 172.21.15.39:0/4293792619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba280d0390 0x7fba281f55b0 unknown :-1 s=CLOSED pgs=2294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:11.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.470+0000 7fba2e600640 1 -- 172.21.15.39:0/4293792619 >> 172.21.15.39:0/4293792619 conn(0x7fba2806cfc0 msgr2=0x7fba280735b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:11.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.470+0000 7fba2e600640 1 -- 172.21.15.39:0/4293792619 shutdown_connections 2024-09-21T07:50:11.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.470+0000 7fba2e600640 1 -- 172.21.15.39:0/4293792619 wait complete. 2024-09-21T07:50:11.494 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps 2024-09-21T07:50:11.740 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_created":3,"last_destroyed":1,"snaps":[{"snapid":2,"ino":1099511627778,"stamp":"2024-09-21T07:50:02.883862+0000","name":"s1","metadata":{}},{"snapid":3,"ino":1099511627778,"stamp":"2024-09-21T07:50:02.909971+0000","name":"s2","metadata":{}}]} 2024-09-21T07:50:11.740 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 3, "last_destroyed": 1, "snaps": [ { "ino": 1099511627778, "metadata": {}, "name": "s1", "snapid": 2, "stamp": "2024-09-21T07:50:02.883862+0000" }, { "ino": 1099511627778, "metadata": {}, "name": "s2", "snapid": 3, "stamp": "2024-09-21T07:50:02.909971+0000" } ] } 2024-09-21T07:50:11.740 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:11.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.967+0000 7f462f200640 1 -- 172.21.15.39:0/495730304 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4614004800 msgr2=0x7f4614004c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:11.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.967+0000 7f462f200640 1 --2- 172.21.15.39:0/495730304 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4614004800 0x7f4614004c30 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f4618001800 tx=0x7f4614005830 comp rx=0 tx=0).stop 2024-09-21T07:50:11.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.967+0000 7f462f200640 1 -- 172.21.15.39:0/495730304 shutdown_connections 2024-09-21T07:50:11.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.967+0000 7f462f200640 1 --2- 172.21.15.39:0/495730304 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4614004800 0x7f4614004c30 unknown :-1 s=CLOSED pgs=2296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:11.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.967+0000 7f462f200640 1 -- 172.21.15.39:0/495730304 >> 172.21.15.39:0/495730304 conn(0x7f463006cfc0 msgr2=0x7f463006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:11.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.967+0000 7f462f200640 1 -- 172.21.15.39:0/495730304 shutdown_connections 2024-09-21T07:50:11.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.967+0000 7f462f200640 1 -- 172.21.15.39:0/495730304 wait complete. 2024-09-21T07:50:11.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.970+0000 7f462f200640 1 Processor -- start 2024-09-21T07:50:11.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.970+0000 7f462f200640 1 -- start start 2024-09-21T07:50:11.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.971+0000 7f462f200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4614004800 0x7f463014ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:11.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.971+0000 7f462f200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f463013f260 0x7f4630151790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:11.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.971+0000 7f462f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46301f89c0 0x7f46301f8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:11.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.971+0000 7f462f200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f46300b3950 con 0x7f46301f89c0 2024-09-21T07:50:11.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.971+0000 7f462f200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f463015c1d0 con 0x7f463013f260 2024-09-21T07:50:11.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.971+0000 7f462f200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f46300b3ad0 con 0x7f4614004800 2024-09-21T07:50:11.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.972+0000 7f462e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46301f89c0 0x7f46301f8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:11.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.972+0000 7f462e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46301f89c0 0x7f46301f8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39646/0 (socket says 172.21.15.39:39646) 2024-09-21T07:50:11.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.972+0000 7f462e800640 1 -- 172.21.15.39:0/687530749 learned_addr learned my addr 172.21.15.39:0/687530749 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:11.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.972+0000 7f4625400640 1 --2- 172.21.15.39:0/687530749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f463013f260 0x7f4630151790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:11.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.972+0000 7f462de00640 1 --2- 172.21.15.39:0/687530749 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4614004800 0x7f463014ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:11.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.972+0000 7f462e800640 1 -- 172.21.15.39:0/687530749 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4614004800 msgr2=0x7f463014ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:11.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.972+0000 7f462e800640 1 --2- 172.21.15.39:0/687530749 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4614004800 0x7f463014ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:11.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.972+0000 7f462e800640 1 -- 172.21.15.39:0/687530749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f463013f260 msgr2=0x7f4630151790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:11.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.972+0000 7f462e800640 1 --2- 172.21.15.39:0/687530749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f463013f260 0x7f4630151790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:11.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.972+0000 7f462e800640 1 -- 172.21.15.39:0/687530749 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46301506f0 con 0x7f46301f89c0 2024-09-21T07:50:11.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.972+0000 7f462e800640 1 --2- 172.21.15.39:0/687530749 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46301f89c0 0x7f46301f8dd0 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f46300b4500 tx=0x7f462800ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:11.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.973+0000 7f4626a00640 1 -- 172.21.15.39:0/687530749 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4628010950 con 0x7f46301f89c0 2024-09-21T07:50:11.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.973+0000 7f4626a00640 1 -- 172.21.15.39:0/687530749 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4628018070 con 0x7f46301f89c0 2024-09-21T07:50:11.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.973+0000 7f462f200640 1 -- 172.21.15.39:0/687530749 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46301509d0 con 0x7f46301f89c0 2024-09-21T07:50:11.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.973+0000 7f4626a00640 1 -- 172.21.15.39:0/687530749 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4628011460 con 0x7f46301f89c0 2024-09-21T07:50:11.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.973+0000 7f462f200640 1 -- 172.21.15.39:0/687530749 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4630150f90 con 0x7f46301f89c0 2024-09-21T07:50:11.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.974+0000 7f462f200640 1 -- 172.21.15.39:0/687530749 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46300d04a0 con 0x7f46301f89c0 2024-09-21T07:50:11.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.975+0000 7f4626a00640 1 -- 172.21.15.39:0/687530749 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4628011600 con 0x7f46301f89c0 2024-09-21T07:50:11.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.976+0000 7f4626a00640 1 --2- 172.21.15.39:0/687530749 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f460c078bc0 0x7f460c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:11.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.976+0000 7f4626a00640 1 -- 172.21.15.39:0/687530749 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(235..235 src has 1..235) ==== 7132+0+0 (secure 0 0 0) 0x7f462809b4e0 con 0x7f46301f89c0 2024-09-21T07:50:11.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.977+0000 7f462de00640 1 --2- 172.21.15.39:0/687530749 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f460c078bc0 0x7f460c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:11.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.977+0000 7f462de00640 1 --2- 172.21.15.39:0/687530749 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f460c078bc0 0x7f460c07b0d0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f4620009c30 tx=0x7f4620005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:11.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:11.981+0000 7f4626a00640 1 -- 172.21.15.39:0/687530749 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4628063380 con 0x7f46301f89c0 2024-09-21T07:50:12.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.219+0000 7f462f200640 1 -- 172.21.15.39:0/687530749 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46300ceba0 con 0x7f46301f89c0 2024-09-21T07:50:12.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.220+0000 7f4626a00640 1 -- 172.21.15.39:0/687530749 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 251 v251) ==== 95+0+5289 (secure 0 0 0) 0x7f4628062c80 con 0x7f46301f89c0 2024-09-21T07:50:12.222 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:50:12.222 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":251,"btime":"2024-09-21T07:50:00:589303+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8531,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1438851059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1438851059},{"type":"v1","addr":"172.21.15.148:6837","nonce":1438851059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":239}],"filesystems":[{"mdsmap":{"epoch":251,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:50:00.121724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":10635,"mds_1":10632,"mds_2":9709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9709":{"gid":9709,"name":"b","rank":2,"incarnation":249,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3497801035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3497801035},{"type":"v1","addr":"172.21.15.148:6836","nonce":3497801035}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10632":{"gid":10632,"name":"a","rank":1,"incarnation":247,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6834/4143807732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4143807732},{"type":"v1","addr":"172.21.15.39:6834","nonce":4143807732}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10635":{"gid":10635,"name":"c","rank":0,"incarnation":240,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/2824493631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2824493631},{"type":"v1","addr":"172.21.15.39:6835","nonce":2824493631}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10635,"qdb_cluster":[10635,10632,9709]},"id":13}]} 2024-09-21T07:50:12.222 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 251 2024-09-21T07:50:12.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.222+0000 7f462f200640 1 -- 172.21.15.39:0/687530749 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f460c078bc0 msgr2=0x7f460c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:12.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.222+0000 7f462f200640 1 --2- 172.21.15.39:0/687530749 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f460c078bc0 0x7f460c07b0d0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f4620009c30 tx=0x7f4620005f90 comp rx=0 tx=0).stop 2024-09-21T07:50:12.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.222+0000 7f462f200640 1 -- 172.21.15.39:0/687530749 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46301f89c0 msgr2=0x7f46301f8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:12.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.222+0000 7f462f200640 1 --2- 172.21.15.39:0/687530749 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46301f89c0 0x7f46301f8dd0 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f46300b4500 tx=0x7f462800ea20 comp rx=0 tx=0).stop 2024-09-21T07:50:12.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.222+0000 7f462f200640 1 -- 172.21.15.39:0/687530749 shutdown_connections 2024-09-21T07:50:12.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.222+0000 7f462f200640 1 --2- 172.21.15.39:0/687530749 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f460c078bc0 0x7f460c07b0d0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:12.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.222+0000 7f462f200640 1 --2- 172.21.15.39:0/687530749 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46301f89c0 0x7f46301f8dd0 unknown :-1 s=CLOSED pgs=2297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:12.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.222+0000 7f462f200640 1 --2- 172.21.15.39:0/687530749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f463013f260 0x7f4630151790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:12.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.222+0000 7f462f200640 1 --2- 172.21.15.39:0/687530749 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4614004800 0x7f463014ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:12.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.222+0000 7f462f200640 1 -- 172.21.15.39:0/687530749 >> 172.21.15.39:0/687530749 conn(0x7f463006cfc0 msgr2=0x7f46300746d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:12.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.222+0000 7f462f200640 1 -- 172.21.15.39:0/687530749 shutdown_connections 2024-09-21T07:50:12.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.222+0000 7f462f200640 1 -- 172.21.15.39:0/687530749 wait complete. 2024-09-21T07:50:12.246 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump snaps 2024-09-21T07:50:12.490 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_created":3,"last_destroyed":1,"snaps":[{"snapid":2,"ino":1099511627778,"stamp":"2024-09-21T07:50:02.883862+0000","name":"s1","metadata":{}},{"snapid":3,"ino":1099511627778,"stamp":"2024-09-21T07:50:02.909971+0000","name":"s2","metadata":{}}]} 2024-09-21T07:50:12.490 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 3, "last_destroyed": 1, "snaps": [ { "ino": 1099511627778, "metadata": {}, "name": "s1", "snapid": 2, "stamp": "2024-09-21T07:50:02.883862+0000" }, { "ino": 1099511627778, "metadata": {}, "name": "s2", "snapid": 3, "stamp": "2024-09-21T07:50:02.909971+0000" } ] } 2024-09-21T07:50:12.490 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:12.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.720+0000 7f91a3600640 1 -- 172.21.15.39:0/2413479687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f919c0d0630 msgr2=0x7f919c0d0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:12.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.720+0000 7f91a3600640 1 --2- 172.21.15.39:0/2413479687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f919c0d0630 0x7f919c0d0a40 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7f9194005fd0 tx=0x7f919402a8b0 comp rx=0 tx=0).stop 2024-09-21T07:50:12.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.720+0000 7f91a3600640 1 -- 172.21.15.39:0/2413479687 shutdown_connections 2024-09-21T07:50:12.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.720+0000 7f91a3600640 1 --2- 172.21.15.39:0/2413479687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f919c0d0630 0x7f919c0d0a40 unknown :-1 s=CLOSED pgs=2299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:12.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.720+0000 7f91a3600640 1 --2- 172.21.15.39:0/2413479687 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f919c073dd0 0x7f919c15d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:12.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.720+0000 7f91a3600640 1 -- 172.21.15.39:0/2413479687 >> 172.21.15.39:0/2413479687 conn(0x7f919c06cfc0 msgr2=0x7f919c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:12.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.720+0000 7f91a3600640 1 -- 172.21.15.39:0/2413479687 shutdown_connections 2024-09-21T07:50:12.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.720+0000 7f91a3600640 1 -- 172.21.15.39:0/2413479687 wait complete. 2024-09-21T07:50:12.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.722+0000 7f91a3600640 1 Processor -- start 2024-09-21T07:50:12.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.722+0000 7f91a3600640 1 -- start start 2024-09-21T07:50:12.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.724+0000 7f91a3600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f919c073dd0 0x7f919c0ada20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:12.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.724+0000 7f91a3600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f919c0d0630 0x7f919c0ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:12.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.724+0000 7f91a3600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f919c15daf0 0x7f919c0c9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:12.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.724+0000 7f91a3600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f919c0cd1b0 con 0x7f919c0d0630 2024-09-21T07:50:12.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.724+0000 7f91a3600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f919c0cd030 con 0x7f919c073dd0 2024-09-21T07:50:12.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.724+0000 7f91a3600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f919c0cd330 con 0x7f919c15daf0 2024-09-21T07:50:12.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.724+0000 7f919be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f919c0d0630 0x7f919c0ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:12.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.724+0000 7f919be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f919c0d0630 0x7f919c0ca490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45192/0 (socket says 172.21.15.39:45192) 2024-09-21T07:50:12.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.724+0000 7f919be00640 1 -- 172.21.15.39:0/370920071 learned_addr learned my addr 172.21.15.39:0/370920071 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:12.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.725+0000 7f919be00640 1 -- 172.21.15.39:0/370920071 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f919c15daf0 msgr2=0x7f919c0c9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:12.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.725+0000 7f91a1600640 1 --2- 172.21.15.39:0/370920071 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f919c15daf0 0x7f919c0c9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:12.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.725+0000 7f91a0c00640 1 --2- 172.21.15.39:0/370920071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f919c073dd0 0x7f919c0ada20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:12.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.725+0000 7f919be00640 1 --2- 172.21.15.39:0/370920071 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f919c15daf0 0x7f919c0c9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:12.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.725+0000 7f919be00640 1 -- 172.21.15.39:0/370920071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f919c073dd0 msgr2=0x7f919c0ada20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:12.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.725+0000 7f919be00640 1 --2- 172.21.15.39:0/370920071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f919c073dd0 0x7f919c0ada20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:12.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.725+0000 7f919be00640 1 -- 172.21.15.39:0/370920071 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f919c0b3710 con 0x7f919c0d0630 2024-09-21T07:50:12.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.725+0000 7f91a0c00640 1 --2- 172.21.15.39:0/370920071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f919c073dd0 0x7f919c0ada20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:50:12.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.725+0000 7f91a1600640 1 --2- 172.21.15.39:0/370920071 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f919c15daf0 0x7f919c0c9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:50:12.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.725+0000 7f919be00640 1 --2- 172.21.15.39:0/370920071 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f919c0d0630 0x7f919c0ca490 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7f918800c340 tx=0x7f918800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:12.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.725+0000 7f9198c00640 1 -- 172.21.15.39:0/370920071 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188006890 con 0x7f919c0d0630 2024-09-21T07:50:12.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.725+0000 7f9198c00640 1 -- 172.21.15.39:0/370920071 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9188006b40 con 0x7f919c0d0630 2024-09-21T07:50:12.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.725+0000 7f9198c00640 1 -- 172.21.15.39:0/370920071 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188007020 con 0x7f919c0d0630 2024-09-21T07:50:12.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.725+0000 7f91a3600640 1 -- 172.21.15.39:0/370920071 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f919c0b39f0 con 0x7f919c0d0630 2024-09-21T07:50:12.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.725+0000 7f91a3600640 1 -- 172.21.15.39:0/370920071 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f919c0adf60 con 0x7f919c0d0630 2024-09-21T07:50:12.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.727+0000 7f91a3600640 1 -- 172.21.15.39:0/370920071 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f919c0cd850 con 0x7f919c0d0630 2024-09-21T07:50:12.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.728+0000 7f9198c00640 1 -- 172.21.15.39:0/370920071 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9188019020 con 0x7f919c0d0630 2024-09-21T07:50:12.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.729+0000 7f9198c00640 1 --2- 172.21.15.39:0/370920071 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f917c078b90 0x7f917c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:12.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.729+0000 7f9198c00640 1 -- 172.21.15.39:0/370920071 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(235..235 src has 1..235) ==== 7132+0+0 (secure 0 0 0) 0x7f918809a970 con 0x7f919c0d0630 2024-09-21T07:50:12.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.730+0000 7f91a0c00640 1 --2- 172.21.15.39:0/370920071 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f917c078b90 0x7f917c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:12.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.731+0000 7f91a0c00640 1 --2- 172.21.15.39:0/370920071 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f917c078b90 0x7f917c07b0a0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f919c0730e0 tx=0x7f9190005dd0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:12.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.734+0000 7f9198c00640 1 -- 172.21.15.39:0/370920071 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f919c0d0f80 con 0x7f919c0d0630 2024-09-21T07:50:12.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.971+0000 7f91a3600640 1 -- 172.21.15.39:0/370920071 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f919c071150 con 0x7f919c0d0630 2024-09-21T07:50:12.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.971+0000 7f9198c00640 1 -- 172.21.15.39:0/370920071 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 251 v251) ==== 95+0+5289 (secure 0 0 0) 0x7f9188067e20 con 0x7f919c0d0630 2024-09-21T07:50:12.973 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:50:12.974 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":251,"btime":"2024-09-21T07:50:00:589303+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8531,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1438851059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1438851059},{"type":"v1","addr":"172.21.15.148:6837","nonce":1438851059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":239}],"filesystems":[{"mdsmap":{"epoch":251,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:50:00.121724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":10635,"mds_1":10632,"mds_2":9709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9709":{"gid":9709,"name":"b","rank":2,"incarnation":249,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3497801035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3497801035},{"type":"v1","addr":"172.21.15.148:6836","nonce":3497801035}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10632":{"gid":10632,"name":"a","rank":1,"incarnation":247,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6834/4143807732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4143807732},{"type":"v1","addr":"172.21.15.39:6834","nonce":4143807732}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10635":{"gid":10635,"name":"c","rank":0,"incarnation":240,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/2824493631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2824493631},{"type":"v1","addr":"172.21.15.39:6835","nonce":2824493631}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10635,"qdb_cluster":[10635,10632,9709]},"id":13}]} 2024-09-21T07:50:12.974 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 251 2024-09-21T07:50:12.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.974+0000 7f91a3600640 1 -- 172.21.15.39:0/370920071 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f917c078b90 msgr2=0x7f917c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:12.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.974+0000 7f91a3600640 1 --2- 172.21.15.39:0/370920071 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f917c078b90 0x7f917c07b0a0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f919c0730e0 tx=0x7f9190005dd0 comp rx=0 tx=0).stop 2024-09-21T07:50:12.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.974+0000 7f91a3600640 1 -- 172.21.15.39:0/370920071 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f919c0d0630 msgr2=0x7f919c0ca490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:12.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.974+0000 7f91a3600640 1 --2- 172.21.15.39:0/370920071 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f919c0d0630 0x7f919c0ca490 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7f918800c340 tx=0x7f918800c800 comp rx=0 tx=0).stop 2024-09-21T07:50:12.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.974+0000 7f91a3600640 1 -- 172.21.15.39:0/370920071 shutdown_connections 2024-09-21T07:50:12.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.974+0000 7f91a3600640 1 --2- 172.21.15.39:0/370920071 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f917c078b90 0x7f917c07b0a0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:12.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.974+0000 7f91a3600640 1 --2- 172.21.15.39:0/370920071 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f919c15daf0 0x7f919c0c9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:12.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.974+0000 7f91a3600640 1 --2- 172.21.15.39:0/370920071 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f919c0d0630 0x7f919c0ca490 unknown :-1 s=CLOSED pgs=2300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:12.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.974+0000 7f91a3600640 1 --2- 172.21.15.39:0/370920071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f919c073dd0 0x7f919c0ada20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:12.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.974+0000 7f91a3600640 1 -- 172.21.15.39:0/370920071 >> 172.21.15.39:0/370920071 conn(0x7f919c06cfc0 msgr2=0x7f919c0b1790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:12.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.974+0000 7f91a3600640 1 -- 172.21.15.39:0/370920071 shutdown_connections 2024-09-21T07:50:12.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:12.974+0000 7f91a3600640 1 -- 172.21.15.39:0/370920071 wait complete. 2024-09-21T07:50:13.001 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json dump snaps 2024-09-21T07:50:13.249 INFO:teuthology.orchestra.run.smithi148.stdout:{"last_created":3,"last_destroyed":1,"snaps":[{"snapid":2,"ino":1099511627778,"stamp":"2024-09-21T07:50:02.883862+0000","name":"s1","metadata":{}},{"snapid":3,"ino":1099511627778,"stamp":"2024-09-21T07:50:02.909971+0000","name":"s2","metadata":{}}]} 2024-09-21T07:50:13.249 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 3, "last_destroyed": 1, "snaps": [ { "ino": 1099511627778, "metadata": {}, "name": "s1", "snapid": 2, "stamp": "2024-09-21T07:50:02.883862+0000" }, { "ino": 1099511627778, "metadata": {}, "name": "s2", "snapid": 3, "stamp": "2024-09-21T07:50:02.909971+0000" } ] } 2024-09-21T07:50:13.249 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:13.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.475+0000 7f5150200640 1 -- 172.21.15.39:0/1947551294 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51480cb8a0 msgr2=0x7f5148156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:13.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.475+0000 7f5150200640 1 --2- 172.21.15.39:0/1947551294 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51480cb8a0 0x7f5148156bb0 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f5134009830 tx=0x7f513402a530 comp rx=0 tx=0).stop 2024-09-21T07:50:13.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.475+0000 7f5150200640 1 -- 172.21.15.39:0/1947551294 shutdown_connections 2024-09-21T07:50:13.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.475+0000 7f5150200640 1 --2- 172.21.15.39:0/1947551294 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51480cb8a0 0x7f5148156bb0 unknown :-1 s=CLOSED pgs=2302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:13.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.475+0000 7f5150200640 1 -- 172.21.15.39:0/1947551294 >> 172.21.15.39:0/1947551294 conn(0x7f51480b2460 msgr2=0x7f514806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:13.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.475+0000 7f5150200640 1 -- 172.21.15.39:0/1947551294 shutdown_connections 2024-09-21T07:50:13.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.475+0000 7f5150200640 1 -- 172.21.15.39:0/1947551294 wait complete. 2024-09-21T07:50:13.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.478+0000 7f5150200640 1 Processor -- start 2024-09-21T07:50:13.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.478+0000 7f5150200640 1 -- start start 2024-09-21T07:50:13.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.479+0000 7f5150200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51480b30a0 0x7f51480ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:13.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.479+0000 7f5150200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51480cb8a0 0x7f51480c6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:13.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.479+0000 7f5150200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51480ccc60 0x7f514815b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:13.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.479+0000 7f5150200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f51480af520 con 0x7f51480cb8a0 2024-09-21T07:50:13.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.479+0000 7f5150200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f51480af3a0 con 0x7f51480ccc60 2024-09-21T07:50:13.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.479+0000 7f5150200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f51480af6a0 con 0x7f51480b30a0 2024-09-21T07:50:13.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.479+0000 7f514ce00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51480cb8a0 0x7f51480c6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:13.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.479+0000 7f514ce00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51480cb8a0 0x7f51480c6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45214/0 (socket says 172.21.15.39:45214) 2024-09-21T07:50:13.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.479+0000 7f514ce00640 1 -- 172.21.15.39:0/1513947262 learned_addr learned my addr 172.21.15.39:0/1513947262 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:13.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.480+0000 7f514d800640 1 --2- 172.21.15.39:0/1513947262 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51480b30a0 0x7f51480ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:13.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.480+0000 7f514e200640 1 --2- 172.21.15.39:0/1513947262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51480ccc60 0x7f514815b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:13.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.480+0000 7f514ce00640 1 -- 172.21.15.39:0/1513947262 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51480b30a0 msgr2=0x7f51480ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:13.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.480+0000 7f514ce00640 1 --2- 172.21.15.39:0/1513947262 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51480b30a0 0x7f51480ceae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:13.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.480+0000 7f514ce00640 1 -- 172.21.15.39:0/1513947262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51480ccc60 msgr2=0x7f514815b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:13.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.480+0000 7f514ce00640 1 --2- 172.21.15.39:0/1513947262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51480ccc60 0x7f514815b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:13.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.480+0000 7f514ce00640 1 -- 172.21.15.39:0/1513947262 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51480cf020 con 0x7f51480cb8a0 2024-09-21T07:50:13.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.480+0000 7f514e200640 1 --2- 172.21.15.39:0/1513947262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51480ccc60 0x7f514815b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:50:13.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.480+0000 7f514ce00640 1 --2- 172.21.15.39:0/1513947262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51480cb8a0 0x7f51480c6570 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f513402ab20 tx=0x7f5134033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:13.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.480+0000 7f5145600640 1 -- 172.21.15.39:0/1513947262 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5134031b60 con 0x7f51480cb8a0 2024-09-21T07:50:13.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.480+0000 7f5145600640 1 -- 172.21.15.39:0/1513947262 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5134031df0 con 0x7f51480cb8a0 2024-09-21T07:50:13.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.480+0000 7f5145600640 1 -- 172.21.15.39:0/1513947262 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5134005280 con 0x7f51480cb8a0 2024-09-21T07:50:13.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.481+0000 7f5150200640 1 -- 172.21.15.39:0/1513947262 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51481f6180 con 0x7f51480cb8a0 2024-09-21T07:50:13.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.481+0000 7f5150200640 1 -- 172.21.15.39:0/1513947262 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51481f6740 con 0x7f51480cb8a0 2024-09-21T07:50:13.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.482+0000 7f5150200640 1 -- 172.21.15.39:0/1513947262 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51481f5ef0 con 0x7f51480cb8a0 2024-09-21T07:50:13.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.483+0000 7f5145600640 1 -- 172.21.15.39:0/1513947262 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5134005420 con 0x7f51480cb8a0 2024-09-21T07:50:13.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.485+0000 7f5145600640 1 --2- 172.21.15.39:0/1513947262 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5128078bc0 0x7f512807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:13.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.485+0000 7f5145600640 1 -- 172.21.15.39:0/1513947262 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(235..235 src has 1..235) ==== 7132+0+0 (secure 0 0 0) 0x7f51340bc480 con 0x7f51480cb8a0 2024-09-21T07:50:13.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.485+0000 7f514d800640 1 --2- 172.21.15.39:0/1513947262 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5128078bc0 0x7f512807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:13.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.486+0000 7f514d800640 1 --2- 172.21.15.39:0/1513947262 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5128078bc0 0x7f512807b0d0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f513c000f60 tx=0x7f513c009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:13.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.489+0000 7f5145600640 1 -- 172.21.15.39:0/1513947262 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f513403f030 con 0x7f51480cb8a0 2024-09-21T07:50:13.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.725+0000 7f5150200640 1 -- 172.21.15.39:0/1513947262 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5148156bb0 con 0x7f51480cb8a0 2024-09-21T07:50:13.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.725+0000 7f5145600640 1 -- 172.21.15.39:0/1513947262 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 251 v251) ==== 95+0+5289 (secure 0 0 0) 0x7f5134031770 con 0x7f51480cb8a0 2024-09-21T07:50:13.728 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:50:13.728 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":251,"btime":"2024-09-21T07:50:00:589303+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8531,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1438851059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1438851059},{"type":"v1","addr":"172.21.15.148:6837","nonce":1438851059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":239}],"filesystems":[{"mdsmap":{"epoch":251,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:50:00.121724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":10635,"mds_1":10632,"mds_2":9709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9709":{"gid":9709,"name":"b","rank":2,"incarnation":249,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3497801035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3497801035},{"type":"v1","addr":"172.21.15.148:6836","nonce":3497801035}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10632":{"gid":10632,"name":"a","rank":1,"incarnation":247,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6834/4143807732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4143807732},{"type":"v1","addr":"172.21.15.39:6834","nonce":4143807732}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10635":{"gid":10635,"name":"c","rank":0,"incarnation":240,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/2824493631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2824493631},{"type":"v1","addr":"172.21.15.39:6835","nonce":2824493631}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10635,"qdb_cluster":[10635,10632,9709]},"id":13}]} 2024-09-21T07:50:13.728 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 251 2024-09-21T07:50:13.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.728+0000 7f5150200640 1 -- 172.21.15.39:0/1513947262 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5128078bc0 msgr2=0x7f512807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:13.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.728+0000 7f5150200640 1 --2- 172.21.15.39:0/1513947262 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5128078bc0 0x7f512807b0d0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f513c000f60 tx=0x7f513c009040 comp rx=0 tx=0).stop 2024-09-21T07:50:13.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.728+0000 7f5150200640 1 -- 172.21.15.39:0/1513947262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51480cb8a0 msgr2=0x7f51480c6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:13.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.728+0000 7f5150200640 1 --2- 172.21.15.39:0/1513947262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51480cb8a0 0x7f51480c6570 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f513402ab20 tx=0x7f5134033c10 comp rx=0 tx=0).stop 2024-09-21T07:50:13.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.728+0000 7f5150200640 1 -- 172.21.15.39:0/1513947262 shutdown_connections 2024-09-21T07:50:13.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.728+0000 7f5150200640 1 --2- 172.21.15.39:0/1513947262 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5128078bc0 0x7f512807b0d0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:13.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.728+0000 7f5150200640 1 --2- 172.21.15.39:0/1513947262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51480ccc60 0x7f514815b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:13.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.728+0000 7f5150200640 1 --2- 172.21.15.39:0/1513947262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51480cb8a0 0x7f51480c6570 unknown :-1 s=CLOSED pgs=2303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:13.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.728+0000 7f5150200640 1 --2- 172.21.15.39:0/1513947262 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f51480b30a0 0x7f51480ceae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:13.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.728+0000 7f5150200640 1 -- 172.21.15.39:0/1513947262 >> 172.21.15.39:0/1513947262 conn(0x7f51480b2460 msgr2=0x7f5148073560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:13.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.728+0000 7f5150200640 1 -- 172.21.15.39:0/1513947262 shutdown_connections 2024-09-21T07:50:13.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:13.728+0000 7f5150200640 1 -- 172.21.15.39:0/1513947262 wait complete. 2024-09-21T07:50:13.753 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2024-09-21T07:50:13.996 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_snap":3,"last_created":3,"last_destroyed":1,"pending_noop":[],"snaps":[{"snapid":2,"ino":1099511627778,"stamp":"2024-09-21T07:50:02.883862+0000","name":"s1","metadata":{}},{"snapid":3,"ino":1099511627778,"stamp":"2024-09-21T07:50:02.909971+0000","name":"s2","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2024-09-21T07:50:13.997 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 3, "last_destroyed": 1, "last_snap": 3, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627778, "metadata": {}, "name": "s1", "snapid": 2, "stamp": "2024-09-21T07:50:02.883862+0000" }, { "ino": 1099511627778, "metadata": {}, "name": "s2", "snapid": 3, "stamp": "2024-09-21T07:50:02.909971+0000" } ] } 2024-09-21T07:50:13.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:50:13.997 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir d0/d1/dir/.snap/s1) 2024-09-21T07:50:14.043 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:14.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.270+0000 7f42b7000640 1 -- 172.21.15.39:0/3669957038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42b00b43f0 msgr2=0x7f42b015b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:14.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.270+0000 7f42b7000640 1 --2- 172.21.15.39:0/3669957038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42b00b43f0 0x7f42b015b640 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7f42b0072850 tx=0x7f429c02a560 comp rx=0 tx=0).stop 2024-09-21T07:50:14.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.270+0000 7f42b7000640 1 -- 172.21.15.39:0/3669957038 shutdown_connections 2024-09-21T07:50:14.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.270+0000 7f42b7000640 1 --2- 172.21.15.39:0/3669957038 >> v2:172.21.15.148:3300/0 conn(0x7f42b006f4e0 0x7f42b006f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:14.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.270+0000 7f42b7000640 1 --2- 172.21.15.39:0/3669957038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42b00b43f0 0x7f42b015b640 unknown :-1 s=CLOSED pgs=2305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:14.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.270+0000 7f42b7000640 1 -- 172.21.15.39:0/3669957038 >> 172.21.15.39:0/3669957038 conn(0x7f42b00d0c70 msgr2=0x7f42b00d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:14.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.270+0000 7f42b7000640 1 -- 172.21.15.39:0/3669957038 shutdown_connections 2024-09-21T07:50:14.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.270+0000 7f42b7000640 1 -- 172.21.15.39:0/3669957038 wait complete. 2024-09-21T07:50:14.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.273+0000 7f42b7000640 1 Processor -- start 2024-09-21T07:50:14.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.273+0000 7f42b7000640 1 -- start start 2024-09-21T07:50:14.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.274+0000 7f42b7000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42b006f4e0 0x7f42b014dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:14.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.275+0000 7f42b7000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42b00b43f0 0x7f42b01f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:14.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.275+0000 7f42b7000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42b00d0400 0x7f42b0157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:14.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.275+0000 7f42b7000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f42b00b54f0 con 0x7f42b006f4e0 2024-09-21T07:50:14.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.275+0000 7f42b7000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f42b00b5370 con 0x7f42b00b43f0 2024-09-21T07:50:14.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.275+0000 7f42b7000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f42b00b5670 con 0x7f42b00d0400 2024-09-21T07:50:14.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.275+0000 7f42b5c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42b006f4e0 0x7f42b014dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:14.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.275+0000 7f42b5c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42b006f4e0 0x7f42b014dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45224/0 (socket says 172.21.15.39:45224) 2024-09-21T07:50:14.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.275+0000 7f42b5c00640 1 -- 172.21.15.39:0/350899884 learned_addr learned my addr 172.21.15.39:0/350899884 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:14.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.275+0000 7f42b5c00640 1 -- 172.21.15.39:0/350899884 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42b00d0400 msgr2=0x7f42b0157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:14.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.275+0000 7f42b5c00640 1 --2- 172.21.15.39:0/350899884 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42b00d0400 0x7f42b0157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:14.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.275+0000 7f42b5c00640 1 -- 172.21.15.39:0/350899884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42b00b43f0 msgr2=0x7f42b01f51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:14.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.275+0000 7f42b5c00640 1 --2- 172.21.15.39:0/350899884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42b00b43f0 0x7f42b01f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:14.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.275+0000 7f42b5c00640 1 -- 172.21.15.39:0/350899884 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42b01f5b50 con 0x7f42b006f4e0 2024-09-21T07:50:14.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.275+0000 7f42b5c00640 1 --2- 172.21.15.39:0/350899884 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42b006f4e0 0x7f42b014dd30 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7f42a400c3f0 tx=0x7f42a400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:14.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.276+0000 7f42ad600640 1 -- 172.21.15.39:0/350899884 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a4017070 con 0x7f42b006f4e0 2024-09-21T07:50:14.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.276+0000 7f42ad600640 1 -- 172.21.15.39:0/350899884 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f42a40035f0 con 0x7f42b006f4e0 2024-09-21T07:50:14.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.276+0000 7f42ad600640 1 -- 172.21.15.39:0/350899884 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a4003a40 con 0x7f42b006f4e0 2024-09-21T07:50:14.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.276+0000 7f42b7000640 1 -- 172.21.15.39:0/350899884 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42b01f5df0 con 0x7f42b006f4e0 2024-09-21T07:50:14.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.276+0000 7f42b7000640 1 -- 172.21.15.39:0/350899884 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42b01f63b0 con 0x7f42b006f4e0 2024-09-21T07:50:14.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.277+0000 7f42b7000640 1 -- 172.21.15.39:0/350899884 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42b00b5b90 con 0x7f42b006f4e0 2024-09-21T07:50:14.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.278+0000 7f42ad600640 1 -- 172.21.15.39:0/350899884 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f42a401a070 con 0x7f42b006f4e0 2024-09-21T07:50:14.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.280+0000 7f42ad600640 1 --2- 172.21.15.39:0/350899884 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4290078bc0 0x7f429007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:14.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.280+0000 7f42ad600640 1 -- 172.21.15.39:0/350899884 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(235..235 src has 1..235) ==== 7132+0+0 (secure 0 0 0) 0x7f42a409a6c0 con 0x7f42b006f4e0 2024-09-21T07:50:14.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.280+0000 7f42b5200640 1 --2- 172.21.15.39:0/350899884 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4290078bc0 0x7f429007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:14.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.281+0000 7f42b5200640 1 --2- 172.21.15.39:0/350899884 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4290078bc0 0x7f429007b0d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f429c033000 tx=0x7f429c0046c0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:14.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.284+0000 7f42ad600640 1 -- 172.21.15.39:0/350899884 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f42b00b4800 con 0x7f42b006f4e0 2024-09-21T07:50:14.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.519+0000 7f42b7000640 1 -- 172.21.15.39:0/350899884 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f42b00b1270 con 0x7f42b006f4e0 2024-09-21T07:50:14.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.519+0000 7f42ad600640 1 -- 172.21.15.39:0/350899884 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 251 v251) ==== 95+0+5289 (secure 0 0 0) 0x7f42a40624d0 con 0x7f42b006f4e0 2024-09-21T07:50:14.521 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:50:14.521 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":251,"btime":"2024-09-21T07:50:00:589303+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8531,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1438851059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1438851059},{"type":"v1","addr":"172.21.15.148:6837","nonce":1438851059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":239}],"filesystems":[{"mdsmap":{"epoch":251,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:50:00.121724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":10635,"mds_1":10632,"mds_2":9709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9709":{"gid":9709,"name":"b","rank":2,"incarnation":249,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3497801035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3497801035},{"type":"v1","addr":"172.21.15.148:6836","nonce":3497801035}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10632":{"gid":10632,"name":"a","rank":1,"incarnation":247,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6834/4143807732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4143807732},{"type":"v1","addr":"172.21.15.39:6834","nonce":4143807732}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10635":{"gid":10635,"name":"c","rank":0,"incarnation":240,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/2824493631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2824493631},{"type":"v1","addr":"172.21.15.39:6835","nonce":2824493631}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10635,"qdb_cluster":[10635,10632,9709]},"id":13}]} 2024-09-21T07:50:14.522 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 251 2024-09-21T07:50:14.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.521+0000 7f42b7000640 1 -- 172.21.15.39:0/350899884 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4290078bc0 msgr2=0x7f429007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:14.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.521+0000 7f42b7000640 1 --2- 172.21.15.39:0/350899884 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4290078bc0 0x7f429007b0d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f429c033000 tx=0x7f429c0046c0 comp rx=0 tx=0).stop 2024-09-21T07:50:14.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.521+0000 7f42b7000640 1 -- 172.21.15.39:0/350899884 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42b006f4e0 msgr2=0x7f42b014dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:14.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.521+0000 7f42b7000640 1 --2- 172.21.15.39:0/350899884 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42b006f4e0 0x7f42b014dd30 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7f42a400c3f0 tx=0x7f42a400c8b0 comp rx=0 tx=0).stop 2024-09-21T07:50:14.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.522+0000 7f42b7000640 1 -- 172.21.15.39:0/350899884 shutdown_connections 2024-09-21T07:50:14.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.522+0000 7f42b7000640 1 --2- 172.21.15.39:0/350899884 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4290078bc0 0x7f429007b0d0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:14.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.522+0000 7f42b7000640 1 --2- 172.21.15.39:0/350899884 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f42b00d0400 0x7f42b0157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:14.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.522+0000 7f42b7000640 1 --2- 172.21.15.39:0/350899884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42b00b43f0 0x7f42b01f51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:14.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.522+0000 7f42b7000640 1 --2- 172.21.15.39:0/350899884 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42b006f4e0 0x7f42b014dd30 unknown :-1 s=CLOSED pgs=2306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:14.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.522+0000 7f42b7000640 1 -- 172.21.15.39:0/350899884 >> 172.21.15.39:0/350899884 conn(0x7f42b00d0c70 msgr2=0x7f42b00b1790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:14.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.522+0000 7f42b7000640 1 -- 172.21.15.39:0/350899884 shutdown_connections 2024-09-21T07:50:14.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:14.522+0000 7f42b7000640 1 -- 172.21.15.39:0/350899884 wait complete. 2024-09-21T07:50:14.545 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2024-09-21T07:50:14.788 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_snap":4,"last_created":3,"last_destroyed":1,"pending_noop":[],"snaps":[{"snapid":2,"ino":1099511627778,"stamp":"2024-09-21T07:50:02.883862+0000","name":"s1","metadata":{}},{"snapid":3,"ino":1099511627778,"stamp":"2024-09-21T07:50:02.909971+0000","name":"s2","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[{"version":6,"removed_snap":2,"seq":4}]} 2024-09-21T07:50:14.788 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 3, "last_destroyed": 1, "last_snap": 4, "need_to_purge": {}, "pending_destroy": [ { "removed_snap": 2, "seq": 4, "version": 6 } ], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627778, "metadata": {}, "name": "s1", "snapid": 2, "stamp": "2024-09-21T07:50:02.883862+0000" }, { "ino": 1099511627778, "metadata": {}, "name": "s2", "snapid": 3, "stamp": "2024-09-21T07:50:02.909971+0000" } ] } 2024-09-21T07:50:14.788 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-09-21T07:50:19.790 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:20.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.025+0000 7f2803a00640 1 -- 172.21.15.39:0/2737467034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27fc0b5240 msgr2=0x7f27fc15c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:20.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.025+0000 7f2803a00640 1 --2- 172.21.15.39:0/2737467034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27fc0b5240 0x7f27fc15c690 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f27f0005fd0 tx=0x7f27f002a540 comp rx=0 tx=0).stop 2024-09-21T07:50:20.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.026+0000 7f2803a00640 1 -- 172.21.15.39:0/2737467034 shutdown_connections 2024-09-21T07:50:20.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.026+0000 7f2803a00640 1 --2- 172.21.15.39:0/2737467034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27fc0b5240 0x7f27fc15c690 unknown :-1 s=CLOSED pgs=2308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:20.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.026+0000 7f2803a00640 1 -- 172.21.15.39:0/2737467034 >> 172.21.15.39:0/2737467034 conn(0x7f27fc06cfc0 msgr2=0x7f27fc06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:20.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.026+0000 7f2803a00640 1 -- 172.21.15.39:0/2737467034 shutdown_connections 2024-09-21T07:50:20.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.026+0000 7f2803a00640 1 -- 172.21.15.39:0/2737467034 wait complete. 2024-09-21T07:50:20.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.028+0000 7f2803a00640 1 Processor -- start 2024-09-21T07:50:20.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.028+0000 7f2803a00640 1 -- start start 2024-09-21T07:50:20.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.030+0000 7f2803a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27fc0b5240 0x7f27fc1f9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:20.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.030+0000 7f2803a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27fc0cf920 0x7f27fc1f59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:20.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.030+0000 7f2803a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27fc1489c0 0x7f27fc1f6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:20.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.030+0000 7f2803a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f27fc0b4ae0 con 0x7f27fc1489c0 2024-09-21T07:50:20.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.030+0000 7f2803a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f27fc0cff40 con 0x7f27fc0b5240 2024-09-21T07:50:20.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.030+0000 7f2803a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f27fc0b4c60 con 0x7f27fc0cf920 2024-09-21T07:50:20.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.030+0000 7f2801a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27fc1489c0 0x7f27fc1f6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:20.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.030+0000 7f2801a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27fc1489c0 0x7f27fc1f6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45242/0 (socket says 172.21.15.39:45242) 2024-09-21T07:50:20.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.030+0000 7f2801a00640 1 -- 172.21.15.39:0/3533421214 learned_addr learned my addr 172.21.15.39:0/3533421214 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:20.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.030+0000 7f2801000640 1 --2- 172.21.15.39:0/3533421214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27fc0b5240 0x7f27fc1f9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:20.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.030+0000 7f2801a00640 1 -- 172.21.15.39:0/3533421214 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27fc0cf920 msgr2=0x7f27fc1f59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:20.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.030+0000 7f27fbe00640 1 --2- 172.21.15.39:0/3533421214 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27fc0cf920 0x7f27fc1f59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:20.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.031+0000 7f2801a00640 1 --2- 172.21.15.39:0/3533421214 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27fc0cf920 0x7f27fc1f59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:20.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.031+0000 7f2801a00640 1 -- 172.21.15.39:0/3533421214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27fc0b5240 msgr2=0x7f27fc1f9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:20.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.031+0000 7f2801a00640 1 --2- 172.21.15.39:0/3533421214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27fc0b5240 0x7f27fc1f9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:20.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.031+0000 7f2801a00640 1 -- 172.21.15.39:0/3533421214 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27fc1f5f00 con 0x7f27fc1489c0 2024-09-21T07:50:20.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.031+0000 7f27fbe00640 1 --2- 172.21.15.39:0/3533421214 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27fc0cf920 0x7f27fc1f59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:50:20.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.031+0000 7f2801000640 1 --2- 172.21.15.39:0/3533421214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27fc0b5240 0x7f27fc1f9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:50:20.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.031+0000 7f2801a00640 1 --2- 172.21.15.39:0/3533421214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27fc1489c0 0x7f27fc1f6b20 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7f27f400c700 tx=0x7f27f400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:20.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.031+0000 7f27f8c00640 1 -- 172.21.15.39:0/3533421214 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f4003b70 con 0x7f27fc1489c0 2024-09-21T07:50:20.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.031+0000 7f27f8c00640 1 -- 172.21.15.39:0/3533421214 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f27f4003d10 con 0x7f27fc1489c0 2024-09-21T07:50:20.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.031+0000 7f27f8c00640 1 -- 172.21.15.39:0/3533421214 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f400aab0 con 0x7f27fc1489c0 2024-09-21T07:50:20.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.031+0000 7f2803a00640 1 -- 172.21.15.39:0/3533421214 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27fc1f63f0 con 0x7f27fc1489c0 2024-09-21T07:50:20.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.031+0000 7f2803a00640 1 -- 172.21.15.39:0/3533421214 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27fc1f9180 con 0x7f27fc1489c0 2024-09-21T07:50:20.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.032+0000 7f2803a00640 1 -- 172.21.15.39:0/3533421214 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27fc1f6160 con 0x7f27fc1489c0 2024-09-21T07:50:20.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.034+0000 7f27f8c00640 1 -- 172.21.15.39:0/3533421214 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f27f400ac50 con 0x7f27fc1489c0 2024-09-21T07:50:20.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.035+0000 7f27f8c00640 1 --2- 172.21.15.39:0/3533421214 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f27dc078b90 0x7f27dc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:20.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.035+0000 7f27f8c00640 1 -- 172.21.15.39:0/3533421214 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(235..235 src has 1..235) ==== 7132+0+0 (secure 0 0 0) 0x7f27f409b150 con 0x7f27fc1489c0 2024-09-21T07:50:20.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.036+0000 7f2801000640 1 --2- 172.21.15.39:0/3533421214 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f27dc078b90 0x7f27dc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:20.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.036+0000 7f2801000640 1 --2- 172.21.15.39:0/3533421214 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f27dc078b90 0x7f27dc07b0a0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f27f0033000 tx=0x7f27f0004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:20.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.041+0000 7f27f8c00640 1 -- 172.21.15.39:0/3533421214 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f27f4062fa0 con 0x7f27fc1489c0 2024-09-21T07:50:20.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.277+0000 7f2803a00640 1 -- 172.21.15.39:0/3533421214 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27fc15c690 con 0x7f27fc1489c0 2024-09-21T07:50:20.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.278+0000 7f27f8c00640 1 -- 172.21.15.39:0/3533421214 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 251 v251) ==== 95+0+5289 (secure 0 0 0) 0x7f27f40628a0 con 0x7f27fc1489c0 2024-09-21T07:50:20.280 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:50:20.280 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":251,"btime":"2024-09-21T07:50:00:589303+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8531,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1438851059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1438851059},{"type":"v1","addr":"172.21.15.148:6837","nonce":1438851059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":239}],"filesystems":[{"mdsmap":{"epoch":251,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:50:00.121724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":10635,"mds_1":10632,"mds_2":9709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9709":{"gid":9709,"name":"b","rank":2,"incarnation":249,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3497801035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3497801035},{"type":"v1","addr":"172.21.15.148:6836","nonce":3497801035}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10632":{"gid":10632,"name":"a","rank":1,"incarnation":247,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6834/4143807732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4143807732},{"type":"v1","addr":"172.21.15.39:6834","nonce":4143807732}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10635":{"gid":10635,"name":"c","rank":0,"incarnation":240,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/2824493631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2824493631},{"type":"v1","addr":"172.21.15.39:6835","nonce":2824493631}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10635,"qdb_cluster":[10635,10632,9709]},"id":13}]} 2024-09-21T07:50:20.280 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 251 2024-09-21T07:50:20.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.281+0000 7f2803a00640 1 -- 172.21.15.39:0/3533421214 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f27dc078b90 msgr2=0x7f27dc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:20.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.281+0000 7f2803a00640 1 --2- 172.21.15.39:0/3533421214 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f27dc078b90 0x7f27dc07b0a0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f27f0033000 tx=0x7f27f0004730 comp rx=0 tx=0).stop 2024-09-21T07:50:20.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.281+0000 7f2803a00640 1 -- 172.21.15.39:0/3533421214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27fc1489c0 msgr2=0x7f27fc1f6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:20.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.281+0000 7f2803a00640 1 --2- 172.21.15.39:0/3533421214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27fc1489c0 0x7f27fc1f6b20 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7f27f400c700 tx=0x7f27f400cbc0 comp rx=0 tx=0).stop 2024-09-21T07:50:20.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.281+0000 7f2803a00640 1 -- 172.21.15.39:0/3533421214 shutdown_connections 2024-09-21T07:50:20.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.281+0000 7f2803a00640 1 --2- 172.21.15.39:0/3533421214 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f27dc078b90 0x7f27dc07b0a0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:20.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.281+0000 7f2803a00640 1 --2- 172.21.15.39:0/3533421214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27fc1489c0 0x7f27fc1f6b20 unknown :-1 s=CLOSED pgs=2309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:20.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.281+0000 7f2803a00640 1 --2- 172.21.15.39:0/3533421214 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f27fc0cf920 0x7f27fc1f59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:20.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.281+0000 7f2803a00640 1 --2- 172.21.15.39:0/3533421214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27fc0b5240 0x7f27fc1f9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:20.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.281+0000 7f2803a00640 1 -- 172.21.15.39:0/3533421214 >> 172.21.15.39:0/3533421214 conn(0x7f27fc06cfc0 msgr2=0x7f27fc15b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:20.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.281+0000 7f2803a00640 1 -- 172.21.15.39:0/3533421214 shutdown_connections 2024-09-21T07:50:20.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.281+0000 7f2803a00640 1 -- 172.21.15.39:0/3533421214 wait complete. 2024-09-21T07:50:20.307 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2024-09-21T07:50:20.555 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_snap":4,"last_created":3,"last_destroyed":4,"pending_noop":[],"snaps":[{"snapid":3,"ino":1099511627778,"stamp":"2024-09-21T07:50:02.909971+0000","name":"s2","metadata":{}}],"need_to_purge":{"30":[2,4]},"pending_update":[],"pending_destroy":[]} 2024-09-21T07:50:20.556 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 3, "last_destroyed": 4, "last_snap": 4, "need_to_purge": { "30": [ 2, 4 ] }, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627778, "metadata": {}, "name": "s2", "snapid": 3, "stamp": "2024-09-21T07:50:02.909971+0000" } ] } 2024-09-21T07:50:20.556 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-21T07:50:20.556 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:20.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.788+0000 7fc2e4600640 1 -- 172.21.15.39:0/3556226756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2c4003c10 msgr2=0x7fc2c4004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:20.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.788+0000 7fc2e4600640 1 --2- 172.21.15.39:0/3556226756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2c4003c10 0x7fc2c4004060 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0001830 tx=0x7fc2c4005830 comp rx=0 tx=0).stop 2024-09-21T07:50:20.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.788+0000 7fc2e2600640 1 -- 172.21.15.39:0/3556226756 reap_dead start 2024-09-21T07:50:20.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.788+0000 7fc2e4600640 1 -- 172.21.15.39:0/3556226756 shutdown_connections 2024-09-21T07:50:20.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.788+0000 7fc2e4600640 1 -- 172.21.15.39:0/3556226756 >> 172.21.15.39:0/3556226756 conn(0x7fc2dc06cfc0 msgr2=0x7fc2dc06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:20.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.788+0000 7fc2e4600640 1 -- 172.21.15.39:0/3556226756 shutdown_connections 2024-09-21T07:50:20.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.789+0000 7fc2e4600640 1 -- 172.21.15.39:0/3556226756 wait complete. 2024-09-21T07:50:20.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.791+0000 7fc2e4600640 1 Processor -- start 2024-09-21T07:50:20.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.791+0000 7fc2e4600640 1 -- start start 2024-09-21T07:50:20.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.792+0000 7fc2e4600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2c4003c10 0x7fc2dc147e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:20.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.792+0000 7fc2e4600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2dc14df00 0x7fc2dc156410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:20.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.792+0000 7fc2e4600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc2dc158130 0x7fc2dc0b4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:20.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.792+0000 7fc2e4600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc2dc0cd710 con 0x7fc2dc14df00 2024-09-21T07:50:20.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.792+0000 7fc2e4600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc2dc0cd590 con 0x7fc2c4003c10 2024-09-21T07:50:20.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.792+0000 7fc2e4600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc2dc0cd890 con 0x7fc2dc158130 2024-09-21T07:50:20.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.792+0000 7fc2e1200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2dc14df00 0x7fc2dc156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:20.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.792+0000 7fc2e1200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2dc14df00 0x7fc2dc156410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45264/0 (socket says 172.21.15.39:45264) 2024-09-21T07:50:20.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.792+0000 7fc2e1200640 1 -- 172.21.15.39:0/888740024 learned_addr learned my addr 172.21.15.39:0/888740024 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:20.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.792+0000 7fc2e1c00640 1 --2- 172.21.15.39:0/888740024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2c4003c10 0x7fc2dc147e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:20.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.792+0000 7fc2e2600640 1 --2- 172.21.15.39:0/888740024 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc2dc158130 0x7fc2dc0b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:20.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.792+0000 7fc2e1200640 1 -- 172.21.15.39:0/888740024 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc2dc158130 msgr2=0x7fc2dc0b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:20.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.792+0000 7fc2e1200640 1 --2- 172.21.15.39:0/888740024 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc2dc158130 0x7fc2dc0b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:20.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.792+0000 7fc2e1200640 1 -- 172.21.15.39:0/888740024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2c4003c10 msgr2=0x7fc2dc147e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:20.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.792+0000 7fc2e1200640 1 --2- 172.21.15.39:0/888740024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2c4003c10 0x7fc2dc147e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:20.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.792+0000 7fc2e1200640 1 -- 172.21.15.39:0/888740024 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2dc155370 con 0x7fc2dc14df00 2024-09-21T07:50:20.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.792+0000 7fc2e1200640 1 --2- 172.21.15.39:0/888740024 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2dc14df00 0x7fc2dc156410 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8011860 tx=0x7fc2c8011d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:20.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.794+0000 7fc2d9600640 1 -- 172.21.15.39:0/888740024 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c800f940 con 0x7fc2dc14df00 2024-09-21T07:50:20.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.794+0000 7fc2d9600640 1 -- 172.21.15.39:0/888740024 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2c800fae0 con 0x7fc2dc14df00 2024-09-21T07:50:20.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.794+0000 7fc2d9600640 1 -- 172.21.15.39:0/888740024 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c80035f0 con 0x7fc2dc14df00 2024-09-21T07:50:20.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.794+0000 7fc2e4600640 1 -- 172.21.15.39:0/888740024 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2dc155650 con 0x7fc2dc14df00 2024-09-21T07:50:20.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.794+0000 7fc2e4600640 1 -- 172.21.15.39:0/888740024 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2dc155c10 con 0x7fc2dc14df00 2024-09-21T07:50:20.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.795+0000 7fc2e4600640 1 -- 172.21.15.39:0/888740024 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2dc0cde20 con 0x7fc2dc14df00 2024-09-21T07:50:20.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.796+0000 7fc2d9600640 1 -- 172.21.15.39:0/888740024 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc2c8003790 con 0x7fc2dc14df00 2024-09-21T07:50:20.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.798+0000 7fc2d9600640 1 --2- 172.21.15.39:0/888740024 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc2bc078b90 0x7fc2bc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:20.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.798+0000 7fc2d9600640 1 -- 172.21.15.39:0/888740024 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(236..236 src has 1..236) ==== 7220+0+0 (secure 0 0 0) 0x7fc2c809c3d0 con 0x7fc2dc14df00 2024-09-21T07:50:20.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.798+0000 7fc2e1c00640 1 --2- 172.21.15.39:0/888740024 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc2bc078b90 0x7fc2bc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:20.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.799+0000 7fc2e1c00640 1 --2- 172.21.15.39:0/888740024 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc2bc078b90 0x7fc2bc07b0a0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0032000 tx=0x7fc2d0001360 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:20.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:20.801+0000 7fc2d9600640 1 -- 172.21.15.39:0/888740024 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc2c80642a0 con 0x7fc2dc14df00 2024-09-21T07:50:21.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.041+0000 7fc2e4600640 1 -- 172.21.15.39:0/888740024 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2dc06d550 con 0x7fc2dc14df00 2024-09-21T07:50:21.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.041+0000 7fc2d9600640 1 -- 172.21.15.39:0/888740024 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 251 v251) ==== 95+0+5289 (secure 0 0 0) 0x7fc2c8063ba0 con 0x7fc2dc14df00 2024-09-21T07:50:21.043 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:50:21.043 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":251,"btime":"2024-09-21T07:50:00:589303+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8531,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1438851059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1438851059},{"type":"v1","addr":"172.21.15.148:6837","nonce":1438851059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":239}],"filesystems":[{"mdsmap":{"epoch":251,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:50:00.121724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":10635,"mds_1":10632,"mds_2":9709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9709":{"gid":9709,"name":"b","rank":2,"incarnation":249,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3497801035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3497801035},{"type":"v1","addr":"172.21.15.148:6836","nonce":3497801035}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10632":{"gid":10632,"name":"a","rank":1,"incarnation":247,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6834/4143807732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4143807732},{"type":"v1","addr":"172.21.15.39:6834","nonce":4143807732}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10635":{"gid":10635,"name":"c","rank":0,"incarnation":240,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/2824493631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2824493631},{"type":"v1","addr":"172.21.15.39:6835","nonce":2824493631}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10635,"qdb_cluster":[10635,10632,9709]},"id":13}]} 2024-09-21T07:50:21.044 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 251 2024-09-21T07:50:21.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.044+0000 7fc2e4600640 1 -- 172.21.15.39:0/888740024 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc2bc078b90 msgr2=0x7fc2bc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:21.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.044+0000 7fc2e4600640 1 --2- 172.21.15.39:0/888740024 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc2bc078b90 0x7fc2bc07b0a0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0032000 tx=0x7fc2d0001360 comp rx=0 tx=0).stop 2024-09-21T07:50:21.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.044+0000 7fc2e4600640 1 -- 172.21.15.39:0/888740024 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2dc14df00 msgr2=0x7fc2dc156410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:21.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.044+0000 7fc2e4600640 1 --2- 172.21.15.39:0/888740024 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2dc14df00 0x7fc2dc156410 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8011860 tx=0x7fc2c8011d20 comp rx=0 tx=0).stop 2024-09-21T07:50:21.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.044+0000 7fc2e4600640 1 -- 172.21.15.39:0/888740024 shutdown_connections 2024-09-21T07:50:21.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.044+0000 7fc2e4600640 1 --2- 172.21.15.39:0/888740024 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc2bc078b90 0x7fc2bc07b0a0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:21.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.044+0000 7fc2e4600640 1 --2- 172.21.15.39:0/888740024 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc2dc158130 0x7fc2dc0b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:21.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.044+0000 7fc2e4600640 1 --2- 172.21.15.39:0/888740024 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2dc14df00 0x7fc2dc156410 unknown :-1 s=CLOSED pgs=2312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:21.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.044+0000 7fc2e4600640 1 --2- 172.21.15.39:0/888740024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc2c4003c10 0x7fc2dc147e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:21.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.044+0000 7fc2e4600640 1 -- 172.21.15.39:0/888740024 >> 172.21.15.39:0/888740024 conn(0x7fc2dc06cfc0 msgr2=0x7fc2dc15f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:21.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.045+0000 7fc2e4600640 1 -- 172.21.15.39:0/888740024 shutdown_connections 2024-09-21T07:50:21.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.045+0000 7fc2e4600640 1 -- 172.21.15.39:0/888740024 wait complete. 2024-09-21T07:50:21.071 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2024-09-21T07:50:21.319 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_snap":4,"last_created":3,"last_destroyed":4,"pending_noop":[],"snaps":[{"snapid":3,"ino":1099511627778,"stamp":"2024-09-21T07:50:02.909971+0000","name":"s2","metadata":{}}],"need_to_purge":{"30":[2,4]},"pending_update":[],"pending_destroy":[]} 2024-09-21T07:50:21.320 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 3, "last_destroyed": 4, "last_snap": 4, "need_to_purge": { "30": [ 2, 4 ] }, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627778, "metadata": {}, "name": "s2", "snapid": 3, "stamp": "2024-09-21T07:50:02.909971+0000" } ] } 2024-09-21T07:50:21.320 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:21.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.547+0000 7f83d8a00640 1 -- 172.21.15.39:0/1863608450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83d40d1590 msgr2=0x7f83d40cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:21.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.547+0000 7f83d8a00640 1 --2- 172.21.15.39:0/1863608450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83d40d1590 0x7f83d40cf280 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7f83d40d38a0 tx=0x7f83bc02a5f0 comp rx=0 tx=0).stop 2024-09-21T07:50:21.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.547+0000 7f83d8a00640 1 -- 172.21.15.39:0/1863608450 shutdown_connections 2024-09-21T07:50:21.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.547+0000 7f83d8a00640 1 --2- 172.21.15.39:0/1863608450 >> v2:172.21.15.148:3300/0 conn(0x7f83d40d19a0 0x7f83d40cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:21.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.547+0000 7f83d8a00640 1 --2- 172.21.15.39:0/1863608450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83d40d1590 0x7f83d40cf280 unknown :-1 s=CLOSED pgs=2313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:21.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.547+0000 7f83d8a00640 1 -- 172.21.15.39:0/1863608450 >> 172.21.15.39:0/1863608450 conn(0x7f83d408c000 msgr2=0x7f83d40d46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:21.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.547+0000 7f83d8a00640 1 -- 172.21.15.39:0/1863608450 shutdown_connections 2024-09-21T07:50:21.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.547+0000 7f83d8a00640 1 -- 172.21.15.39:0/1863608450 wait complete. 2024-09-21T07:50:21.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.549+0000 7f83d8a00640 1 Processor -- start 2024-09-21T07:50:21.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.550+0000 7f83d8a00640 1 -- start start 2024-09-21T07:50:21.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.551+0000 7f83d8a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83d40d1590 0x7f83d40b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:21.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.551+0000 7f83d8a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83d40d19a0 0x7f83d415f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:21.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.551+0000 7f83d8a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83d40d4dc0 0x7f83d415f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:21.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.551+0000 7f83d8a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f83d40d0090 con 0x7f83d40d4dc0 2024-09-21T07:50:21.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.551+0000 7f83d8a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f83d40cff10 con 0x7f83d40d1590 2024-09-21T07:50:21.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.551+0000 7f83d8a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f83d40d0210 con 0x7f83d40d19a0 2024-09-21T07:50:21.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.551+0000 7f83d2800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83d40d4dc0 0x7f83d415f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:21.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.551+0000 7f83d2800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83d40d4dc0 0x7f83d415f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45278/0 (socket says 172.21.15.39:45278) 2024-09-21T07:50:21.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.551+0000 7f83d2800640 1 -- 172.21.15.39:0/2288644995 learned_addr learned my addr 172.21.15.39:0/2288644995 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:21.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.551+0000 7f83d1e00640 1 --2- 172.21.15.39:0/2288644995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83d40d1590 0x7f83d40b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:21.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.552+0000 7f83c9400640 1 --2- 172.21.15.39:0/2288644995 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83d40d19a0 0x7f83d415f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:21.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.552+0000 7f83d2800640 1 -- 172.21.15.39:0/2288644995 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83d40d19a0 msgr2=0x7f83d415f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:21.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.552+0000 7f83d2800640 1 --2- 172.21.15.39:0/2288644995 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83d40d19a0 0x7f83d415f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:21.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.552+0000 7f83d2800640 1 -- 172.21.15.39:0/2288644995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83d40d1590 msgr2=0x7f83d40b4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:21.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.552+0000 7f83d2800640 1 --2- 172.21.15.39:0/2288644995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83d40d1590 0x7f83d40b4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:21.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.552+0000 7f83d2800640 1 -- 172.21.15.39:0/2288644995 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d4156930 con 0x7f83d40d4dc0 2024-09-21T07:50:21.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.552+0000 7f83d2800640 1 --2- 172.21.15.39:0/2288644995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83d40d4dc0 0x7f83d415f710 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00ea80 tx=0x7f83cc00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:21.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.552+0000 7f83caa00640 1 -- 172.21.15.39:0/2288644995 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc004b20 con 0x7f83d40d4dc0 2024-09-21T07:50:21.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.552+0000 7f83caa00640 1 -- 172.21.15.39:0/2288644995 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f83cc004ce0 con 0x7f83d40d4dc0 2024-09-21T07:50:21.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.552+0000 7f83d8a00640 1 -- 172.21.15.39:0/2288644995 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83d4167b50 con 0x7f83d40d4dc0 2024-09-21T07:50:21.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.553+0000 7f83caa00640 1 -- 172.21.15.39:0/2288644995 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc005280 con 0x7f83d40d4dc0 2024-09-21T07:50:21.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.553+0000 7f83d8a00640 1 -- 172.21.15.39:0/2288644995 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83d415b1b0 con 0x7f83d40d4dc0 2024-09-21T07:50:21.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.554+0000 7f83d8a00640 1 -- 172.21.15.39:0/2288644995 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83d415b410 con 0x7f83d40d4dc0 2024-09-21T07:50:21.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.555+0000 7f83caa00640 1 -- 172.21.15.39:0/2288644995 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f83cc005420 con 0x7f83d40d4dc0 2024-09-21T07:50:21.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.556+0000 7f83caa00640 1 --2- 172.21.15.39:0/2288644995 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f83b0078b90 0x7f83b007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:21.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.556+0000 7f83caa00640 1 -- 172.21.15.39:0/2288644995 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(236..236 src has 1..236) ==== 7220+0+0 (secure 0 0 0) 0x7f83cc09be30 con 0x7f83d40d4dc0 2024-09-21T07:50:21.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.557+0000 7f83d1e00640 1 --2- 172.21.15.39:0/2288644995 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f83b0078b90 0x7f83b007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:21.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.557+0000 7f83d1e00640 1 --2- 172.21.15.39:0/2288644995 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f83b0078b90 0x7f83b007b0a0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f83c4001720 tx=0x7f83c4009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:21.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.561+0000 7f83caa00640 1 -- 172.21.15.39:0/2288644995 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f83cc063da0 con 0x7f83d40d4dc0 2024-09-21T07:50:21.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.799+0000 7f83d8a00640 1 -- 172.21.15.39:0/2288644995 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83d4156e50 con 0x7f83d40d4dc0 2024-09-21T07:50:21.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.800+0000 7f83caa00640 1 -- 172.21.15.39:0/2288644995 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 251 v251) ==== 95+0+5289 (secure 0 0 0) 0x7f83cc0636a0 con 0x7f83d40d4dc0 2024-09-21T07:50:21.801 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:50:21.801 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":251,"btime":"2024-09-21T07:50:00:589303+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8531,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1438851059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1438851059},{"type":"v1","addr":"172.21.15.148:6837","nonce":1438851059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":239}],"filesystems":[{"mdsmap":{"epoch":251,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:50:00.121724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":10635,"mds_1":10632,"mds_2":9709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9709":{"gid":9709,"name":"b","rank":2,"incarnation":249,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3497801035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3497801035},{"type":"v1","addr":"172.21.15.148:6836","nonce":3497801035}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10632":{"gid":10632,"name":"a","rank":1,"incarnation":247,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6834/4143807732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4143807732},{"type":"v1","addr":"172.21.15.39:6834","nonce":4143807732}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10635":{"gid":10635,"name":"c","rank":0,"incarnation":240,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/2824493631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2824493631},{"type":"v1","addr":"172.21.15.39:6835","nonce":2824493631}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10635,"qdb_cluster":[10635,10632,9709]},"id":13}]} 2024-09-21T07:50:21.801 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 251 2024-09-21T07:50:21.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.802+0000 7f83d8a00640 1 -- 172.21.15.39:0/2288644995 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f83b0078b90 msgr2=0x7f83b007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:21.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.802+0000 7f83d8a00640 1 --2- 172.21.15.39:0/2288644995 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f83b0078b90 0x7f83b007b0a0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f83c4001720 tx=0x7f83c4009040 comp rx=0 tx=0).stop 2024-09-21T07:50:21.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.802+0000 7f83d8a00640 1 -- 172.21.15.39:0/2288644995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83d40d4dc0 msgr2=0x7f83d415f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:21.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.802+0000 7f83d8a00640 1 --2- 172.21.15.39:0/2288644995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83d40d4dc0 0x7f83d415f710 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00ea80 tx=0x7f83cc00ef40 comp rx=0 tx=0).stop 2024-09-21T07:50:21.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.802+0000 7f83d8a00640 1 -- 172.21.15.39:0/2288644995 shutdown_connections 2024-09-21T07:50:21.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.802+0000 7f83d8a00640 1 --2- 172.21.15.39:0/2288644995 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f83b0078b90 0x7f83b007b0a0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:21.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.802+0000 7f83d8a00640 1 --2- 172.21.15.39:0/2288644995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83d40d4dc0 0x7f83d415f710 unknown :-1 s=CLOSED pgs=2314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:21.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.802+0000 7f83d8a00640 1 --2- 172.21.15.39:0/2288644995 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f83d40d19a0 0x7f83d415f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:21.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.802+0000 7f83d8a00640 1 --2- 172.21.15.39:0/2288644995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83d40d1590 0x7f83d40b4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:21.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.802+0000 7f83d8a00640 1 -- 172.21.15.39:0/2288644995 >> 172.21.15.39:0/2288644995 conn(0x7f83d408c000 msgr2=0x7f83d40b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:21.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.803+0000 7f83d8a00640 1 -- 172.21.15.39:0/2288644995 shutdown_connections 2024-09-21T07:50:21.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:21.803+0000 7f83d8a00640 1 -- 172.21.15.39:0/2288644995 wait complete. 2024-09-21T07:50:21.825 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps --server 2024-09-21T07:50:22.068 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_snap":4,"last_created":3,"last_destroyed":4,"pending_noop":[],"snaps":[{"snapid":3,"ino":1099511627778,"stamp":"2024-09-21T07:50:02.909971+0000","name":"s2","metadata":{}}],"need_to_purge":{"30":[2,4]},"pending_update":[],"pending_destroy":[]} 2024-09-21T07:50:22.069 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 3, "last_destroyed": 4, "last_snap": 4, "need_to_purge": { "30": [ 2, 4 ] }, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627778, "metadata": {}, "name": "s2", "snapid": 3, "stamp": "2024-09-21T07:50:02.909971+0000" } ] } 2024-09-21T07:50:22.069 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:22.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.297+0000 7f1cc0a00640 1 -- 172.21.15.39:0/355725544 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1cbc073dd0 msgr2=0x7f1cbc15d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:22.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.297+0000 7f1cc0a00640 1 --2- 172.21.15.39:0/355725544 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1cbc073dd0 0x7f1cbc15d5b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1cac005fd0 tx=0x7f1cac02a5d0 comp rx=0 tx=0).stop 2024-09-21T07:50:22.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.297+0000 7f1cc0a00640 1 -- 172.21.15.39:0/355725544 shutdown_connections 2024-09-21T07:50:22.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.297+0000 7f1cc0a00640 1 --2- 172.21.15.39:0/355725544 >> v2:172.21.15.148:3300/0 conn(0x7f1cbc15daf0 0x7f1cbc0cf450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:22.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.297+0000 7f1cc0a00640 1 --2- 172.21.15.39:0/355725544 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1cbc073dd0 0x7f1cbc15d5b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:22.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.297+0000 7f1cc0a00640 1 -- 172.21.15.39:0/355725544 >> 172.21.15.39:0/355725544 conn(0x7f1cbc06cfc0 msgr2=0x7f1cbc06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:22.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.297+0000 7f1cc0a00640 1 -- 172.21.15.39:0/355725544 shutdown_connections 2024-09-21T07:50:22.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.297+0000 7f1cc0a00640 1 -- 172.21.15.39:0/355725544 wait complete. 2024-09-21T07:50:22.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.299+0000 7f1cc0a00640 1 Processor -- start 2024-09-21T07:50:22.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.299+0000 7f1cc0a00640 1 -- start start 2024-09-21T07:50:22.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.300+0000 7f1cc0a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cbc073dd0 0x7f1cbc1f3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:22.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.300+0000 7f1cc0a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1cbc0cf990 0x7f1cbc1f4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:22.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.300+0000 7f1cc0a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cbc15daf0 0x7f1cbc141bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:22.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.301+0000 7f1cc0a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc0cd1d0 con 0x7f1cbc15daf0 2024-09-21T07:50:22.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.301+0000 7f1cc0a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc0cd050 con 0x7f1cbc073dd0 2024-09-21T07:50:22.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.301+0000 7f1cc0a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f1cbc0cd350 con 0x7f1cbc0cf990 2024-09-21T07:50:22.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.301+0000 7f1cba800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cbc15daf0 0x7f1cbc141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:22.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.301+0000 7f1cba800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cbc15daf0 0x7f1cbc141bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58316/0 (socket says 172.21.15.39:58316) 2024-09-21T07:50:22.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.301+0000 7f1cba800640 1 -- 172.21.15.39:0/3288910190 learned_addr learned my addr 172.21.15.39:0/3288910190 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:22.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.301+0000 7f1cb3e00640 1 --2- 172.21.15.39:0/3288910190 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1cbc0cf990 0x7f1cbc1f4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:22.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.301+0000 7f1cb9e00640 1 --2- 172.21.15.39:0/3288910190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cbc073dd0 0x7f1cbc1f3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:22.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.301+0000 7f1cba800640 1 -- 172.21.15.39:0/3288910190 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1cbc0cf990 msgr2=0x7f1cbc1f4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:22.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.301+0000 7f1cba800640 1 --2- 172.21.15.39:0/3288910190 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1cbc0cf990 0x7f1cbc1f4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:22.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.301+0000 7f1cba800640 1 -- 172.21.15.39:0/3288910190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cbc073dd0 msgr2=0x7f1cbc1f3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:22.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.301+0000 7f1cba800640 1 --2- 172.21.15.39:0/3288910190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cbc073dd0 0x7f1cbc1f3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:22.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.301+0000 7f1cba800640 1 -- 172.21.15.39:0/3288910190 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cbc150eb0 con 0x7f1cbc15daf0 2024-09-21T07:50:22.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.301+0000 7f1cb9e00640 1 --2- 172.21.15.39:0/3288910190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cbc073dd0 0x7f1cbc1f3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:50:22.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.302+0000 7f1cba800640 1 --2- 172.21.15.39:0/3288910190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cbc15daf0 0x7f1cbc141bf0 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400c760 tx=0x7f1cb400cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:22.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.302+0000 7f1cb2000640 1 -- 172.21.15.39:0/3288910190 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4016020 con 0x7f1cbc15daf0 2024-09-21T07:50:22.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.302+0000 7f1cb2000640 1 -- 172.21.15.39:0/3288910190 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1cb4003990 con 0x7f1cbc15daf0 2024-09-21T07:50:22.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.302+0000 7f1cb2000640 1 -- 172.21.15.39:0/3288910190 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4006e60 con 0x7f1cbc15daf0 2024-09-21T07:50:22.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.302+0000 7f1cc0a00640 1 -- 172.21.15.39:0/3288910190 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cbc151190 con 0x7f1cbc15daf0 2024-09-21T07:50:22.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.302+0000 7f1cc0a00640 1 -- 172.21.15.39:0/3288910190 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cbc151750 con 0x7f1cbc15daf0 2024-09-21T07:50:22.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.303+0000 7f1cc0a00640 1 -- 172.21.15.39:0/3288910190 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cbc1f4900 con 0x7f1cbc15daf0 2024-09-21T07:50:22.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.304+0000 7f1cb2000640 1 -- 172.21.15.39:0/3288910190 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1cb40050d0 con 0x7f1cbc15daf0 2024-09-21T07:50:22.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.305+0000 7f1cb2000640 1 --2- 172.21.15.39:0/3288910190 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1c98078b90 0x7f1c9807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:22.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.306+0000 7f1cb2000640 1 -- 172.21.15.39:0/3288910190 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(236..236 src has 1..236) ==== 7220+0+0 (secure 0 0 0) 0x7f1cb409ae60 con 0x7f1cbc15daf0 2024-09-21T07:50:22.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.306+0000 7f1cb9e00640 1 --2- 172.21.15.39:0/3288910190 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1c98078b90 0x7f1c9807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:22.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.307+0000 7f1cb9e00640 1 --2- 172.21.15.39:0/3288910190 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1c98078b90 0x7f1c9807b0a0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f1cac033000 tx=0x7f1cac004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:22.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.310+0000 7f1cb2000640 1 -- 172.21.15.39:0/3288910190 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1cb4020020 con 0x7f1cbc15daf0 2024-09-21T07:50:22.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.546+0000 7f1cc0a00640 1 -- 172.21.15.39:0/3288910190 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1cbc0cd800 con 0x7f1cbc15daf0 2024-09-21T07:50:22.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.547+0000 7f1cb2000640 1 -- 172.21.15.39:0/3288910190 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 251 v251) ==== 95+0+5289 (secure 0 0 0) 0x7f1cb4062d30 con 0x7f1cbc15daf0 2024-09-21T07:50:22.548 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:50:22.548 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":251,"btime":"2024-09-21T07:50:00:589303+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8531,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1438851059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1438851059},{"type":"v1","addr":"172.21.15.148:6837","nonce":1438851059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":239}],"filesystems":[{"mdsmap":{"epoch":251,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:50:00.121724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":10635,"mds_1":10632,"mds_2":9709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9709":{"gid":9709,"name":"b","rank":2,"incarnation":249,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3497801035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3497801035},{"type":"v1","addr":"172.21.15.148:6836","nonce":3497801035}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10632":{"gid":10632,"name":"a","rank":1,"incarnation":247,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6834/4143807732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4143807732},{"type":"v1","addr":"172.21.15.39:6834","nonce":4143807732}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10635":{"gid":10635,"name":"c","rank":0,"incarnation":240,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/2824493631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2824493631},{"type":"v1","addr":"172.21.15.39:6835","nonce":2824493631}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10635,"qdb_cluster":[10635,10632,9709]},"id":13}]} 2024-09-21T07:50:22.548 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 251 2024-09-21T07:50:22.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.549+0000 7f1cc0a00640 1 -- 172.21.15.39:0/3288910190 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1c98078b90 msgr2=0x7f1c9807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:22.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.549+0000 7f1cc0a00640 1 --2- 172.21.15.39:0/3288910190 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1c98078b90 0x7f1c9807b0a0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f1cac033000 tx=0x7f1cac004730 comp rx=0 tx=0).stop 2024-09-21T07:50:22.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.549+0000 7f1cc0a00640 1 -- 172.21.15.39:0/3288910190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cbc15daf0 msgr2=0x7f1cbc141bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:22.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.549+0000 7f1cc0a00640 1 --2- 172.21.15.39:0/3288910190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cbc15daf0 0x7f1cbc141bf0 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400c760 tx=0x7f1cb400cb60 comp rx=0 tx=0).stop 2024-09-21T07:50:22.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.549+0000 7f1cc0a00640 1 -- 172.21.15.39:0/3288910190 shutdown_connections 2024-09-21T07:50:22.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.549+0000 7f1cc0a00640 1 --2- 172.21.15.39:0/3288910190 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1c98078b90 0x7f1c9807b0a0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:22.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.549+0000 7f1cc0a00640 1 --2- 172.21.15.39:0/3288910190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cbc15daf0 0x7f1cbc141bf0 unknown :-1 s=CLOSED pgs=2315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:22.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.549+0000 7f1cc0a00640 1 --2- 172.21.15.39:0/3288910190 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1cbc0cf990 0x7f1cbc1f4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:22.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.549+0000 7f1cc0a00640 1 --2- 172.21.15.39:0/3288910190 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cbc073dd0 0x7f1cbc1f3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:22.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.549+0000 7f1cc0a00640 1 -- 172.21.15.39:0/3288910190 >> 172.21.15.39:0/3288910190 conn(0x7f1cbc06cfc0 msgr2=0x7f1cbc0b0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:22.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.549+0000 7f1cc0a00640 1 -- 172.21.15.39:0/3288910190 shutdown_connections 2024-09-21T07:50:22.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:22.549+0000 7f1cc0a00640 1 -- 172.21.15.39:0/3288910190 wait complete. 2024-09-21T07:50:22.576 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps 2024-09-21T07:50:22.821 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_created":3,"last_destroyed":4,"snaps":[{"snapid":3,"ino":1099511627778,"stamp":"2024-09-21T07:50:02.909971+0000","name":"s2","metadata":{}}]} 2024-09-21T07:50:22.822 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 3, "last_destroyed": 4, "snaps": [ { "ino": 1099511627778, "metadata": {}, "name": "s2", "snapid": 3, "stamp": "2024-09-21T07:50:02.909971+0000" } ] } 2024-09-21T07:50:22.822 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:23.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.053+0000 7f239b200640 1 -- 172.21.15.39:0/1424979891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f237c003fd0 msgr2=0x7f237c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:23.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.053+0000 7f239b200640 1 --2- 172.21.15.39:0/1424979891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f237c003fd0 0x7f237c004800 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7f238400c4a0 tx=0x7f237c005830 comp rx=0 tx=0).stop 2024-09-21T07:50:23.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.053+0000 7f239a800640 1 -- 172.21.15.39:0/1424979891 reap_dead start 2024-09-21T07:50:23.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.053+0000 7f239b200640 1 -- 172.21.15.39:0/1424979891 shutdown_connections 2024-09-21T07:50:23.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.053+0000 7f239b200640 1 -- 172.21.15.39:0/1424979891 >> 172.21.15.39:0/1424979891 conn(0x7f239406cfc0 msgr2=0x7f239406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:23.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.053+0000 7f239b200640 1 -- 172.21.15.39:0/1424979891 shutdown_connections 2024-09-21T07:50:23.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.053+0000 7f239b200640 1 -- 172.21.15.39:0/1424979891 wait complete. 2024-09-21T07:50:23.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.055+0000 7f239b200640 1 Processor -- start 2024-09-21T07:50:23.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.055+0000 7f239b200640 1 -- start start 2024-09-21T07:50:23.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.057+0000 7f239b200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f237c003fd0 0x7f23941f6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:23.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.057+0000 7f239b200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2394142df0 0x7f23941406c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:23.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.057+0000 7f239b200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f239413b650 0x7f239413e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:23.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.057+0000 7f239b200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f239415da40 con 0x7f237c003fd0 2024-09-21T07:50:23.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.057+0000 7f239b200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f239415d8c0 con 0x7f239413b650 2024-09-21T07:50:23.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.057+0000 7f239b200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f239415dbc0 con 0x7f2394142df0 2024-09-21T07:50:23.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.057+0000 7f2399e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f237c003fd0 0x7f23941f6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:23.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.057+0000 7f2399e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f237c003fd0 0x7f23941f6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58342/0 (socket says 172.21.15.39:58342) 2024-09-21T07:50:23.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.057+0000 7f2399e00640 1 -- 172.21.15.39:0/1895093585 learned_addr learned my addr 172.21.15.39:0/1895093585 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:23.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.057+0000 7f2399400640 1 --2- 172.21.15.39:0/1895093585 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2394142df0 0x7f23941406c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:23.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.057+0000 7f239a800640 1 --2- 172.21.15.39:0/1895093585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f239413b650 0x7f239413e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:23.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.057+0000 7f2399e00640 1 -- 172.21.15.39:0/1895093585 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2394142df0 msgr2=0x7f23941406c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:23.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.058+0000 7f2399e00640 1 --2- 172.21.15.39:0/1895093585 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2394142df0 0x7f23941406c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:23.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.058+0000 7f2399e00640 1 -- 172.21.15.39:0/1895093585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f239413b650 msgr2=0x7f239413e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:23.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.058+0000 7f2399e00640 1 --2- 172.21.15.39:0/1895093585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f239413b650 0x7f239413e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:23.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.058+0000 7f2399e00640 1 -- 172.21.15.39:0/1895093585 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23941f7840 con 0x7f237c003fd0 2024-09-21T07:50:23.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.058+0000 7f2399e00640 1 --2- 172.21.15.39:0/1895093585 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f237c003fd0 0x7f23941f6ef0 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7f2388011b10 tx=0x7f2388009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:23.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.058+0000 7f238e000640 1 -- 172.21.15.39:0/1895093585 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238801e070 con 0x7f237c003fd0 2024-09-21T07:50:23.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.058+0000 7f239b200640 1 -- 172.21.15.39:0/1895093585 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23941f7b20 con 0x7f237c003fd0 2024-09-21T07:50:23.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.058+0000 7f238e000640 1 -- 172.21.15.39:0/1895093585 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2388004050 con 0x7f237c003fd0 2024-09-21T07:50:23.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.058+0000 7f238e000640 1 -- 172.21.15.39:0/1895093585 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238800fea0 con 0x7f237c003fd0 2024-09-21T07:50:23.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.058+0000 7f239b200640 1 -- 172.21.15.39:0/1895093585 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2394140c00 con 0x7f237c003fd0 2024-09-21T07:50:23.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.059+0000 7f239b200640 1 -- 172.21.15.39:0/1895093585 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23940d07a0 con 0x7f237c003fd0 2024-09-21T07:50:23.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.061+0000 7f238e000640 1 -- 172.21.15.39:0/1895093585 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2388021070 con 0x7f237c003fd0 2024-09-21T07:50:23.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.062+0000 7f238e000640 1 --2- 172.21.15.39:0/1895093585 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2378078b90 0x7f237807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:23.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.062+0000 7f238e000640 1 -- 172.21.15.39:0/1895093585 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(236..236 src has 1..236) ==== 7220+0+0 (secure 0 0 0) 0x7f238809be10 con 0x7f237c003fd0 2024-09-21T07:50:23.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.062+0000 7f2399400640 1 --2- 172.21.15.39:0/1895093585 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2378078b90 0x7f237807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:23.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.063+0000 7f2399400640 1 --2- 172.21.15.39:0/1895093585 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2378078b90 0x7f237807b0a0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f2384033000 tx=0x7f2394161000 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:23.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.066+0000 7f238e000640 1 -- 172.21.15.39:0/1895093585 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f238809f030 con 0x7f237c003fd0 2024-09-21T07:50:23.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.303+0000 7f239b200640 1 -- 172.21.15.39:0/1895093585 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2394073190 con 0x7f237c003fd0 2024-09-21T07:50:23.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.304+0000 7f238e000640 1 -- 172.21.15.39:0/1895093585 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 251 v251) ==== 95+0+5289 (secure 0 0 0) 0x7f23880044a0 con 0x7f237c003fd0 2024-09-21T07:50:23.305 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:50:23.305 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":251,"btime":"2024-09-21T07:50:00:589303+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8531,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1438851059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1438851059},{"type":"v1","addr":"172.21.15.148:6837","nonce":1438851059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":239}],"filesystems":[{"mdsmap":{"epoch":251,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:50:00.121724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":10635,"mds_1":10632,"mds_2":9709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9709":{"gid":9709,"name":"b","rank":2,"incarnation":249,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3497801035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3497801035},{"type":"v1","addr":"172.21.15.148:6836","nonce":3497801035}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10632":{"gid":10632,"name":"a","rank":1,"incarnation":247,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6834/4143807732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4143807732},{"type":"v1","addr":"172.21.15.39:6834","nonce":4143807732}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10635":{"gid":10635,"name":"c","rank":0,"incarnation":240,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/2824493631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2824493631},{"type":"v1","addr":"172.21.15.39:6835","nonce":2824493631}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10635,"qdb_cluster":[10635,10632,9709]},"id":13}]} 2024-09-21T07:50:23.305 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 251 2024-09-21T07:50:23.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.306+0000 7f239b200640 1 -- 172.21.15.39:0/1895093585 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2378078b90 msgr2=0x7f237807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:23.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.307+0000 7f239b200640 1 --2- 172.21.15.39:0/1895093585 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2378078b90 0x7f237807b0a0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f2384033000 tx=0x7f2394161000 comp rx=0 tx=0).stop 2024-09-21T07:50:23.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.307+0000 7f239b200640 1 -- 172.21.15.39:0/1895093585 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f237c003fd0 msgr2=0x7f23941f6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:23.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.307+0000 7f239b200640 1 --2- 172.21.15.39:0/1895093585 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f237c003fd0 0x7f23941f6ef0 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7f2388011b10 tx=0x7f2388009e90 comp rx=0 tx=0).stop 2024-09-21T07:50:23.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.307+0000 7f239b200640 1 -- 172.21.15.39:0/1895093585 shutdown_connections 2024-09-21T07:50:23.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.307+0000 7f239b200640 1 --2- 172.21.15.39:0/1895093585 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2378078b90 0x7f237807b0a0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:23.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.307+0000 7f239b200640 1 --2- 172.21.15.39:0/1895093585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f239413b650 0x7f239413e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:23.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.307+0000 7f239b200640 1 --2- 172.21.15.39:0/1895093585 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2394142df0 0x7f23941406c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:23.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.307+0000 7f239b200640 1 --2- 172.21.15.39:0/1895093585 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f237c003fd0 0x7f23941f6ef0 unknown :-1 s=CLOSED pgs=2318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:23.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.307+0000 7f239b200640 1 -- 172.21.15.39:0/1895093585 >> 172.21.15.39:0/1895093585 conn(0x7f239406cfc0 msgr2=0x7f239415eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:23.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.307+0000 7f239b200640 1 -- 172.21.15.39:0/1895093585 shutdown_connections 2024-09-21T07:50:23.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.307+0000 7f239b200640 1 -- 172.21.15.39:0/1895093585 wait complete. 2024-09-21T07:50:23.332 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump snaps 2024-09-21T07:50:23.576 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_created":3,"last_destroyed":4,"snaps":[{"snapid":3,"ino":1099511627778,"stamp":"2024-09-21T07:50:02.909971+0000","name":"s2","metadata":{}}]} 2024-09-21T07:50:23.577 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 3, "last_destroyed": 4, "snaps": [ { "ino": 1099511627778, "metadata": {}, "name": "s2", "snapid": 3, "stamp": "2024-09-21T07:50:02.909971+0000" } ] } 2024-09-21T07:50:23.577 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:23.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.801+0000 7fda66400640 1 -- 172.21.15.39:0/2431056440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda60162f10 msgr2=0x7fda600b30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:23.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.801+0000 7fda66400640 1 --2- 172.21.15.39:0/2431056440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda60162f10 0x7fda600b30a0 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7fda600b2e40 tx=0x7fda5002a5f0 comp rx=0 tx=0).stop 2024-09-21T07:50:23.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.801+0000 7fda66400640 1 -- 172.21.15.39:0/2431056440 shutdown_connections 2024-09-21T07:50:23.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.801+0000 7fda66400640 1 --2- 172.21.15.39:0/2431056440 >> v2:172.21.15.148:3300/0 conn(0x7fda600b3990 0x7fda600b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:23.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.801+0000 7fda66400640 1 --2- 172.21.15.39:0/2431056440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda60162f10 0x7fda600b30a0 unknown :-1 s=CLOSED pgs=2319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:23.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.801+0000 7fda66400640 1 -- 172.21.15.39:0/2431056440 >> 172.21.15.39:0/2431056440 conn(0x7fda600b2460 msgr2=0x7fda6006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:23.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.801+0000 7fda66400640 1 -- 172.21.15.39:0/2431056440 shutdown_connections 2024-09-21T07:50:23.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.801+0000 7fda66400640 1 -- 172.21.15.39:0/2431056440 wait complete. 2024-09-21T07:50:23.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.803+0000 7fda66400640 1 Processor -- start 2024-09-21T07:50:23.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.804+0000 7fda66400640 1 -- start start 2024-09-21T07:50:23.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.805+0000 7fda66400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda600b3990 0x7fda601f9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:23.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.805+0000 7fda66400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda600b3da0 0x7fda601501e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:23.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.805+0000 7fda66400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda60162f10 0x7fda600cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:23.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.805+0000 7fda66400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fda600af910 con 0x7fda600b3da0 2024-09-21T07:50:23.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.805+0000 7fda66400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fda600af790 con 0x7fda600b3990 2024-09-21T07:50:23.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.805+0000 7fda66400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fda600afa90 con 0x7fda60162f10 2024-09-21T07:50:23.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.805+0000 7fda57e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda600b3da0 0x7fda601501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:23.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.805+0000 7fda57e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda600b3da0 0x7fda601501e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58374/0 (socket says 172.21.15.39:58374) 2024-09-21T07:50:23.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.805+0000 7fda57e00640 1 -- 172.21.15.39:0/2810738103 learned_addr learned my addr 172.21.15.39:0/2810738103 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:23.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.805+0000 7fda5f400640 1 --2- 172.21.15.39:0/2810738103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda600b3990 0x7fda601f9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:23.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.805+0000 7fda5fe00640 1 --2- 172.21.15.39:0/2810738103 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda60162f10 0x7fda600cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:23.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.806+0000 7fda57e00640 1 -- 172.21.15.39:0/2810738103 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda60162f10 msgr2=0x7fda600cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:23.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.806+0000 7fda57e00640 1 --2- 172.21.15.39:0/2810738103 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda60162f10 0x7fda600cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:23.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.806+0000 7fda57e00640 1 -- 172.21.15.39:0/2810738103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda600b3990 msgr2=0x7fda601f9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:23.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.806+0000 7fda57e00640 1 --2- 172.21.15.39:0/2810738103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda600b3990 0x7fda601f9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:23.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.806+0000 7fda57e00640 1 -- 172.21.15.39:0/2810738103 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda60150c60 con 0x7fda600b3da0 2024-09-21T07:50:23.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.806+0000 7fda57e00640 1 --2- 172.21.15.39:0/2810738103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda600b3da0 0x7fda601501e0 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7fda4800d6a0 tx=0x7fda4800db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.806+0000 7fda57400640 1 -- 172.21.15.39:0/2810738103 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda48017070 con 0x7fda600b3da0 2024-09-21T07:50:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.806+0000 7fda57400640 1 -- 172.21.15.39:0/2810738103 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda48009b10 con 0x7fda600b3da0 2024-09-21T07:50:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.806+0000 7fda66400640 1 -- 172.21.15.39:0/2810738103 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda60150f40 con 0x7fda600b3da0 2024-09-21T07:50:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.806+0000 7fda57400640 1 -- 172.21.15.39:0/2810738103 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda48010460 con 0x7fda600b3da0 2024-09-21T07:50:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.806+0000 7fda66400640 1 -- 172.21.15.39:0/2810738103 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda601f9210 con 0x7fda600b3da0 2024-09-21T07:50:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.808+0000 7fda66400640 1 -- 172.21.15.39:0/2810738103 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda600b4c10 con 0x7fda600b3da0 2024-09-21T07:50:23.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.809+0000 7fda57400640 1 -- 172.21.15.39:0/2810738103 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fda4801a070 con 0x7fda600b3da0 2024-09-21T07:50:23.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.810+0000 7fda57400640 1 --2- 172.21.15.39:0/2810738103 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fda3c078b90 0x7fda3c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:23.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.810+0000 7fda57400640 1 -- 172.21.15.39:0/2810738103 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(236..236 src has 1..236) ==== 7220+0+0 (secure 0 0 0) 0x7fda4809b340 con 0x7fda600b3da0 2024-09-21T07:50:23.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.810+0000 7fda5f400640 1 --2- 172.21.15.39:0/2810738103 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fda3c078b90 0x7fda3c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:23.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.811+0000 7fda5f400640 1 --2- 172.21.15.39:0/2810738103 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fda3c078b90 0x7fda3c07b0a0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fda50033000 tx=0x7fda50004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:23.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:23.815+0000 7fda57400640 1 -- 172.21.15.39:0/2810738103 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fda48062bd0 con 0x7fda600b3da0 2024-09-21T07:50:24.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.049+0000 7fda66400640 1 -- 172.21.15.39:0/2810738103 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda601509f0 con 0x7fda600b3da0 2024-09-21T07:50:24.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.050+0000 7fda57400640 1 -- 172.21.15.39:0/2810738103 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 251 v251) ==== 95+0+5289 (secure 0 0 0) 0x7fda48067600 con 0x7fda600b3da0 2024-09-21T07:50:24.051 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:50:24.051 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":251,"btime":"2024-09-21T07:50:00:589303+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8531,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1438851059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1438851059},{"type":"v1","addr":"172.21.15.148:6837","nonce":1438851059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":239}],"filesystems":[{"mdsmap":{"epoch":251,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:50:00.121724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":10635,"mds_1":10632,"mds_2":9709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9709":{"gid":9709,"name":"b","rank":2,"incarnation":249,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3497801035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3497801035},{"type":"v1","addr":"172.21.15.148:6836","nonce":3497801035}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10632":{"gid":10632,"name":"a","rank":1,"incarnation":247,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6834/4143807732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4143807732},{"type":"v1","addr":"172.21.15.39:6834","nonce":4143807732}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10635":{"gid":10635,"name":"c","rank":0,"incarnation":240,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/2824493631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2824493631},{"type":"v1","addr":"172.21.15.39:6835","nonce":2824493631}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10635,"qdb_cluster":[10635,10632,9709]},"id":13}]} 2024-09-21T07:50:24.051 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 251 2024-09-21T07:50:24.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.052+0000 7fda66400640 1 -- 172.21.15.39:0/2810738103 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fda3c078b90 msgr2=0x7fda3c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:24.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.052+0000 7fda66400640 1 --2- 172.21.15.39:0/2810738103 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fda3c078b90 0x7fda3c07b0a0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fda50033000 tx=0x7fda50004730 comp rx=0 tx=0).stop 2024-09-21T07:50:24.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.052+0000 7fda66400640 1 -- 172.21.15.39:0/2810738103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda600b3da0 msgr2=0x7fda601501e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:24.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.052+0000 7fda66400640 1 --2- 172.21.15.39:0/2810738103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda600b3da0 0x7fda601501e0 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7fda4800d6a0 tx=0x7fda4800db60 comp rx=0 tx=0).stop 2024-09-21T07:50:24.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.052+0000 7fda66400640 1 -- 172.21.15.39:0/2810738103 shutdown_connections 2024-09-21T07:50:24.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.052+0000 7fda66400640 1 --2- 172.21.15.39:0/2810738103 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fda3c078b90 0x7fda3c07b0a0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:24.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.052+0000 7fda66400640 1 --2- 172.21.15.39:0/2810738103 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fda60162f10 0x7fda600cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:24.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.052+0000 7fda66400640 1 --2- 172.21.15.39:0/2810738103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda600b3da0 0x7fda601501e0 unknown :-1 s=CLOSED pgs=2320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:24.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.052+0000 7fda66400640 1 --2- 172.21.15.39:0/2810738103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fda600b3990 0x7fda601f9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:24.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.052+0000 7fda66400640 1 -- 172.21.15.39:0/2810738103 >> 172.21.15.39:0/2810738103 conn(0x7fda600b2460 msgr2=0x7fda600d01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:24.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.052+0000 7fda66400640 1 -- 172.21.15.39:0/2810738103 shutdown_connections 2024-09-21T07:50:24.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.052+0000 7fda66400640 1 -- 172.21.15.39:0/2810738103 wait complete. 2024-09-21T07:50:24.079 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json dump snaps 2024-09-21T07:50:24.327 INFO:teuthology.orchestra.run.smithi148.stdout:{"last_created":3,"last_destroyed":4,"snaps":[{"snapid":3,"ino":1099511627778,"stamp":"2024-09-21T07:50:02.909971+0000","name":"s2","metadata":{}}]} 2024-09-21T07:50:24.328 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 3, "last_destroyed": 4, "snaps": [ { "ino": 1099511627778, "metadata": {}, "name": "s2", "snapid": 3, "stamp": "2024-09-21T07:50:02.909971+0000" } ] } 2024-09-21T07:50:24.328 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 13:2 2024-09-21T07:50:24.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.556+0000 7f5bc1a00640 1 -- 172.21.15.39:0/1791301602 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bbc151640 msgr2=0x7f5bbc151a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:24.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.556+0000 7f5bc1a00640 1 --2- 172.21.15.39:0/1791301602 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bbc151640 0x7f5bbc151a50 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f5bac005fd0 tx=0x7f5bac02b7b0 comp rx=0 tx=0).stop 2024-09-21T07:50:24.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.556+0000 7f5bc1a00640 1 -- 172.21.15.39:0/1791301602 shutdown_connections 2024-09-21T07:50:24.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.556+0000 7f5bc1a00640 1 --2- 172.21.15.39:0/1791301602 >> v2:172.21.15.148:3300/0 conn(0x7f5bbc070570 0x7f5bbc070980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:24.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.556+0000 7f5bc1a00640 1 --2- 172.21.15.39:0/1791301602 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bbc151640 0x7f5bbc151a50 unknown :-1 s=CLOSED pgs=2321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:24.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.556+0000 7f5bc1a00640 1 -- 172.21.15.39:0/1791301602 >> 172.21.15.39:0/1791301602 conn(0x7f5bbc06cfc0 msgr2=0x7f5bbc06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:24.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.556+0000 7f5bc1a00640 1 -- 172.21.15.39:0/1791301602 shutdown_connections 2024-09-21T07:50:24.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.556+0000 7f5bc1a00640 1 -- 172.21.15.39:0/1791301602 wait complete. 2024-09-21T07:50:24.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.558+0000 7f5bc1a00640 1 Processor -- start 2024-09-21T07:50:24.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.558+0000 7f5bc1a00640 1 -- start start 2024-09-21T07:50:24.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.560+0000 7f5bc1a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bbc070570 0x7f5bbc1f6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:24.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.560+0000 7f5bc1a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bbc0cf750 0x7f5bbc0b40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:24.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.560+0000 7f5bc1a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5bbc151640 0x7f5bbc159a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:24.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.560+0000 7f5bc1a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5bbc15eb60 con 0x7f5bbc0cf750 2024-09-21T07:50:24.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.560+0000 7f5bc1a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5bbc15e9e0 con 0x7f5bbc070570 2024-09-21T07:50:24.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.560+0000 7f5bc1a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f5bbc15ece0 con 0x7f5bbc151640 2024-09-21T07:50:24.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.560+0000 7f5bb3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bbc0cf750 0x7f5bbc0b40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:24.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.560+0000 7f5bb3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bbc0cf750 0x7f5bbc0b40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58388/0 (socket says 172.21.15.39:58388) 2024-09-21T07:50:24.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.560+0000 7f5bb3e00640 1 -- 172.21.15.39:0/3423211874 learned_addr learned my addr 172.21.15.39:0/3423211874 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:24.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.560+0000 7f5bbaa00640 1 --2- 172.21.15.39:0/3423211874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bbc070570 0x7f5bbc1f6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:24.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.560+0000 7f5bbb400640 1 --2- 172.21.15.39:0/3423211874 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5bbc151640 0x7f5bbc159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:24.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.561+0000 7f5bb3e00640 1 -- 172.21.15.39:0/3423211874 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5bbc151640 msgr2=0x7f5bbc159a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:24.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.561+0000 7f5bb3e00640 1 --2- 172.21.15.39:0/3423211874 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5bbc151640 0x7f5bbc159a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:24.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.561+0000 7f5bb3e00640 1 -- 172.21.15.39:0/3423211874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bbc070570 msgr2=0x7f5bbc1f6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:24.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.561+0000 7f5bb3e00640 1 --2- 172.21.15.39:0/3423211874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bbc070570 0x7f5bbc1f6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:24.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.561+0000 7f5bb3e00640 1 -- 172.21.15.39:0/3423211874 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bbc1f56e0 con 0x7f5bbc0cf750 2024-09-21T07:50:24.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.561+0000 7f5bb3e00640 1 --2- 172.21.15.39:0/3423211874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bbc0cf750 0x7f5bbc0b40c0 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f5ba400d6d0 tx=0x7f5ba400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:24.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.561+0000 7f5bb2a00640 1 -- 172.21.15.39:0/3423211874 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba4017070 con 0x7f5bbc0cf750 2024-09-21T07:50:24.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.561+0000 7f5bb2a00640 1 -- 172.21.15.39:0/3423211874 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ba4005330 con 0x7f5bbc0cf750 2024-09-21T07:50:24.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.561+0000 7f5bb2a00640 1 -- 172.21.15.39:0/3423211874 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba4009820 con 0x7f5bbc0cf750 2024-09-21T07:50:24.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.561+0000 7f5bc1a00640 1 -- 172.21.15.39:0/3423211874 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bbc147a60 con 0x7f5bbc0cf750 2024-09-21T07:50:24.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.561+0000 7f5bc1a00640 1 -- 172.21.15.39:0/3423211874 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bbc148020 con 0x7f5bbc0cf750 2024-09-21T07:50:24.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.562+0000 7f5bc1a00640 1 -- 172.21.15.39:0/3423211874 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bbc15f200 con 0x7f5bbc0cf750 2024-09-21T07:50:24.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.564+0000 7f5bb2a00640 1 -- 172.21.15.39:0/3423211874 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5ba4009bd0 con 0x7f5bbc0cf750 2024-09-21T07:50:24.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.565+0000 7f5bb2a00640 1 --2- 172.21.15.39:0/3423211874 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5b98078b90 0x7f5b9807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:24.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.565+0000 7f5bb2a00640 1 -- 172.21.15.39:0/3423211874 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(236..236 src has 1..236) ==== 7220+0+0 (secure 0 0 0) 0x7f5ba409b620 con 0x7f5bbc0cf750 2024-09-21T07:50:24.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.565+0000 7f5bbaa00640 1 --2- 172.21.15.39:0/3423211874 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5b98078b90 0x7f5b9807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:24.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.566+0000 7f5bbaa00640 1 --2- 172.21.15.39:0/3423211874 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5b98078b90 0x7f5b9807b0a0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f5bac033000 tx=0x7f5bac004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:24.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.569+0000 7f5bb2a00640 1 -- 172.21.15.39:0/3423211874 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5ba4062d80 con 0x7f5bbc0cf750 2024-09-21T07:50:24.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.849+0000 7f5bc1a00640 1 -- 172.21.15.39:0/3423211874 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "13:2"} v 0) -- 0x7f5bbc0d0a80 con 0x7f5bbc0cf750 2024-09-21T07:50:24.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.852+0000 7f5bb2a00640 1 -- 172.21.15.39:0/3423211874 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "13:2"}]=0 failed mds gid 9709 v252) ==== 98+0+0 (secure 0 0 0) 0x7f5ba40677b0 con 0x7f5bbc0cf750 2024-09-21T07:50:24.853 INFO:teuthology.orchestra.run.smithi039.stderr:failed mds gid 9709 2024-09-21T07:50:24.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.855+0000 7f5bc1a00640 1 -- 172.21.15.39:0/3423211874 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5b98078b90 msgr2=0x7f5b9807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:24.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.855+0000 7f5bc1a00640 1 --2- 172.21.15.39:0/3423211874 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5b98078b90 0x7f5b9807b0a0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f5bac033000 tx=0x7f5bac004730 comp rx=0 tx=0).stop 2024-09-21T07:50:24.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.855+0000 7f5bc1a00640 1 -- 172.21.15.39:0/3423211874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bbc0cf750 msgr2=0x7f5bbc0b40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:24.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.855+0000 7f5bc1a00640 1 --2- 172.21.15.39:0/3423211874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bbc0cf750 0x7f5bbc0b40c0 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f5ba400d6d0 tx=0x7f5ba400db90 comp rx=0 tx=0).stop 2024-09-21T07:50:24.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.855+0000 7f5bc1a00640 1 -- 172.21.15.39:0/3423211874 shutdown_connections 2024-09-21T07:50:24.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.855+0000 7f5bc1a00640 1 --2- 172.21.15.39:0/3423211874 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5b98078b90 0x7f5b9807b0a0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:24.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.855+0000 7f5bc1a00640 1 --2- 172.21.15.39:0/3423211874 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5bbc151640 0x7f5bbc159a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:24.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.855+0000 7f5bc1a00640 1 --2- 172.21.15.39:0/3423211874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bbc0cf750 0x7f5bbc0b40c0 unknown :-1 s=CLOSED pgs=2322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:24.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.855+0000 7f5bc1a00640 1 --2- 172.21.15.39:0/3423211874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bbc070570 0x7f5bbc1f6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:24.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.855+0000 7f5bc1a00640 1 -- 172.21.15.39:0/3423211874 >> 172.21.15.39:0/3423211874 conn(0x7f5bbc06cfc0 msgr2=0x7f5bbc06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:24.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.855+0000 7f5bc1a00640 1 -- 172.21.15.39:0/3423211874 shutdown_connections 2024-09-21T07:50:24.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:24.855+0000 7f5bc1a00640 1 -- 172.21.15.39:0/3423211874 wait complete. 2024-09-21T07:50:24.881 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:24.985 INFO:tasks.ceph.mds.b.smithi148.stdout:starting mds.b at 2024-09-21T07:50:25.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.108+0000 7f1a49e00640 1 -- 172.21.15.39:0/1990555951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a440b3990 msgr2=0x7f1a440b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:25.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.108+0000 7f1a49e00640 1 --2- 172.21.15.39:0/1990555951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a440b3990 0x7f1a440b4450 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7f1a440729e0 tx=0x7f1a2c02a560 comp rx=0 tx=0).stop 2024-09-21T07:50:25.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.108+0000 7f1a49e00640 1 -- 172.21.15.39:0/1990555951 shutdown_connections 2024-09-21T07:50:25.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.108+0000 7f1a49e00640 1 --2- 172.21.15.39:0/1990555951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a440b3990 0x7f1a440b4450 unknown :-1 s=CLOSED pgs=2323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.108+0000 7f1a49e00640 1 --2- 172.21.15.39:0/1990555951 >> v2:172.21.15.148:3300/0 conn(0x7f1a44162f10 0x7f1a440b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.108+0000 7f1a49e00640 1 -- 172.21.15.39:0/1990555951 >> 172.21.15.39:0/1990555951 conn(0x7f1a440b2460 msgr2=0x7f1a4406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:25.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.108+0000 7f1a49e00640 1 -- 172.21.15.39:0/1990555951 shutdown_connections 2024-09-21T07:50:25.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.108+0000 7f1a49e00640 1 -- 172.21.15.39:0/1990555951 wait complete. 2024-09-21T07:50:25.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.110+0000 7f1a49e00640 1 Processor -- start 2024-09-21T07:50:25.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.111+0000 7f1a49e00640 1 -- start start 2024-09-21T07:50:25.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.112+0000 7f1a49e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a440b3990 0x7f1a440b0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:25.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.112+0000 7f1a49e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1a440b3da0 0x7f1a4415ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:25.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.112+0000 7f1a49e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a44162f10 0x7f1a4415beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:25.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.112+0000 7f1a49e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1a440af910 con 0x7f1a440b3990 2024-09-21T07:50:25.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.112+0000 7f1a49e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1a440af790 con 0x7f1a44162f10 2024-09-21T07:50:25.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.112+0000 7f1a49e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f1a440afa90 con 0x7f1a440b3da0 2024-09-21T07:50:25.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.112+0000 7f1a42a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a440b3990 0x7f1a440b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:25.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.112+0000 7f1a42a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a440b3990 0x7f1a440b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58408/0 (socket says 172.21.15.39:58408) 2024-09-21T07:50:25.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.112+0000 7f1a42a00640 1 -- 172.21.15.39:0/2462181393 learned_addr learned my addr 172.21.15.39:0/2462181393 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:25.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.112+0000 7f1a3a000640 1 --2- 172.21.15.39:0/2462181393 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1a440b3da0 0x7f1a4415ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:25.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.112+0000 7f1a43400640 1 --2- 172.21.15.39:0/2462181393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a44162f10 0x7f1a4415beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:25.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.113+0000 7f1a42a00640 1 -- 172.21.15.39:0/2462181393 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1a440b3da0 msgr2=0x7f1a4415ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:25.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.113+0000 7f1a42a00640 1 --2- 172.21.15.39:0/2462181393 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1a440b3da0 0x7f1a4415ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.113+0000 7f1a42a00640 1 -- 172.21.15.39:0/2462181393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a44162f10 msgr2=0x7f1a4415beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:25.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.113+0000 7f1a42a00640 1 --2- 172.21.15.39:0/2462181393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a44162f10 0x7f1a4415beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.113+0000 7f1a42a00640 1 -- 172.21.15.39:0/2462181393 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a440ad9e0 con 0x7f1a440b3990 2024-09-21T07:50:25.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.113+0000 7f1a3a000640 1 --2- 172.21.15.39:0/2462181393 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1a440b3da0 0x7f1a4415ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:50:25.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.113+0000 7f1a42a00640 1 --2- 172.21.15.39:0/2462181393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a440b3990 0x7f1a440b0870 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7f1a3400d780 tx=0x7f1a3400dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:25.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.113+0000 7f1a3be00640 1 -- 172.21.15.39:0/2462181393 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a34017070 con 0x7f1a440b3990 2024-09-21T07:50:25.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.113+0000 7f1a3be00640 1 -- 172.21.15.39:0/2462181393 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a34009b10 con 0x7f1a440b3990 2024-09-21T07:50:25.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.113+0000 7f1a3be00640 1 -- 172.21.15.39:0/2462181393 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a34010460 con 0x7f1a440b3990 2024-09-21T07:50:25.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.113+0000 7f1a49e00640 1 -- 172.21.15.39:0/2462181393 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a4415b5b0 con 0x7f1a440b3990 2024-09-21T07:50:25.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.113+0000 7f1a49e00640 1 -- 172.21.15.39:0/2462181393 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a440afd90 con 0x7f1a440b3990 2024-09-21T07:50:25.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.115+0000 7f1a49e00640 1 -- 172.21.15.39:0/2462181393 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a440b4450 con 0x7f1a440b3990 2024-09-21T07:50:25.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.116+0000 7f1a3be00640 1 -- 172.21.15.39:0/2462181393 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1a3401a070 con 0x7f1a440b3990 2024-09-21T07:50:25.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.118+0000 7f1a3be00640 1 --2- 172.21.15.39:0/2462181393 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1a20078bc0 0x7f1a2007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:25.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.118+0000 7f1a3be00640 1 -- 172.21.15.39:0/2462181393 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(237..237 src has 1..237) ==== 7262+0+0 (secure 0 0 0) 0x7f1a3409b5b0 con 0x7f1a440b3990 2024-09-21T07:50:25.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.118+0000 7f1a3a000640 1 --2- 172.21.15.39:0/2462181393 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1a20078bc0 0x7f1a2007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:25.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.119+0000 7f1a3a000640 1 --2- 172.21.15.39:0/2462181393 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1a20078bc0 0x7f1a2007b0d0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c033000 tx=0x7f1a2c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:25.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.123+0000 7f1a3be00640 1 -- 172.21.15.39:0/2462181393 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1a34063340 con 0x7f1a440b3990 2024-09-21T07:50:25.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.359+0000 7f1a49e00640 1 -- 172.21.15.39:0/2462181393 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a440cdb70 con 0x7f1a440b3990 2024-09-21T07:50:25.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.360+0000 7f1a3be00640 1 -- 172.21.15.39:0/2462181393 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 253 v253) ==== 95+0+4458 (secure 0 0 0) 0x7f1a34062c40 con 0x7f1a440b3990 2024-09-21T07:50:25.361 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:50:25.361 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":253,"btime":"2024-09-21T07:50:24:860443+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":253,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:50:24.860427+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":237,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":10635,"mds_1":10632,"mds_2":8531},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8531":{"gid":8531,"name":"d","rank":2,"incarnation":253,"state":"up:replay","state_seq":1,"addr":"172.21.15.148:6837/1438851059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1438851059},{"type":"v1","addr":"172.21.15.148:6837","nonce":1438851059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10632":{"gid":10632,"name":"a","rank":1,"incarnation":247,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6834/4143807732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4143807732},{"type":"v1","addr":"172.21.15.39:6834","nonce":4143807732}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10635":{"gid":10635,"name":"c","rank":0,"incarnation":240,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/2824493631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2824493631},{"type":"v1","addr":"172.21.15.39:6835","nonce":2824493631}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10635,"qdb_cluster":[10635,10632]},"id":13}]} 2024-09-21T07:50:25.361 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 253 2024-09-21T07:50:25.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.362+0000 7f1a49e00640 1 -- 172.21.15.39:0/2462181393 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1a20078bc0 msgr2=0x7f1a2007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:25.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.362+0000 7f1a49e00640 1 --2- 172.21.15.39:0/2462181393 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1a20078bc0 0x7f1a2007b0d0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c033000 tx=0x7f1a2c004730 comp rx=0 tx=0).stop 2024-09-21T07:50:25.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.362+0000 7f1a49e00640 1 -- 172.21.15.39:0/2462181393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a440b3990 msgr2=0x7f1a440b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:25.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.362+0000 7f1a49e00640 1 --2- 172.21.15.39:0/2462181393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a440b3990 0x7f1a440b0870 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7f1a3400d780 tx=0x7f1a3400dc40 comp rx=0 tx=0).stop 2024-09-21T07:50:25.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.362+0000 7f1a49e00640 1 -- 172.21.15.39:0/2462181393 shutdown_connections 2024-09-21T07:50:25.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.362+0000 7f1a49e00640 1 --2- 172.21.15.39:0/2462181393 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1a20078bc0 0x7f1a2007b0d0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.362+0000 7f1a49e00640 1 --2- 172.21.15.39:0/2462181393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1a44162f10 0x7f1a4415beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.362+0000 7f1a49e00640 1 --2- 172.21.15.39:0/2462181393 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1a440b3da0 0x7f1a4415ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.362+0000 7f1a49e00640 1 --2- 172.21.15.39:0/2462181393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a440b3990 0x7f1a440b0870 unknown :-1 s=CLOSED pgs=2324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.362+0000 7f1a49e00640 1 -- 172.21.15.39:0/2462181393 >> 172.21.15.39:0/2462181393 conn(0x7f1a440b2460 msgr2=0x7f1a4406f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:25.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.362+0000 7f1a49e00640 1 -- 172.21.15.39:0/2462181393 shutdown_connections 2024-09-21T07:50:25.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:25.362+0000 7f1a49e00640 1 -- 172.21.15.39:0/2462181393 wait complete. 2024-09-21T07:50:25.791 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 253, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:50:24.860427+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 237, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 10635, 'mds_1': 10632, 'mds_2': 8531}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8531': {'gid': 8531, 'name': 'd', 'rank': 2, 'incarnation': 253, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.148:6837/1438851059', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6835', 'nonce': 1438851059}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 1438851059}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10632': {'gid': 10632, 'name': 'a', 'rank': 1, 'incarnation': 247, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.39:6834/4143807732', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 4143807732}, {'type': 'v1', 'addr': '172.21.15.39:6834', 'nonce': 4143807732}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10635': {'gid': 10635, 'name': 'c', 'rank': 0, 'incarnation': 240, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.39:6835/2824493631', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 2824493631}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 2824493631}]}, 'join_fscid': -1, 'export_targets': [1, 2], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10635, 'qdb_cluster': [10635, 10632]} 2024-09-21T07:50:25.791 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8531:up:replay 2024-09-21T07:50:26.793 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:27.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.034+0000 7fb060400640 1 -- 172.21.15.39:0/4156815614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0580d06d0 msgr2=0x7fb0580d0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:27.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.034+0000 7fb060400640 1 --2- 172.21.15.39:0/4156815614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0580d06d0 0x7fb0580d0ae0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb050009be0 tx=0x7fb05002a8c0 comp rx=0 tx=0).stop 2024-09-21T07:50:27.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.034+0000 7fb060400640 1 -- 172.21.15.39:0/4156815614 shutdown_connections 2024-09-21T07:50:27.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.034+0000 7fb060400640 1 --2- 172.21.15.39:0/4156815614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0580d06d0 0x7fb0580d0ae0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:27.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.034+0000 7fb060400640 1 --2- 172.21.15.39:0/4156815614 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb058162520 0x7fb058162930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:27.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.034+0000 7fb060400640 1 -- 172.21.15.39:0/4156815614 >> 172.21.15.39:0/4156815614 conn(0x7fb05806cfc0 msgr2=0x7fb05806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:27.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.034+0000 7fb060400640 1 -- 172.21.15.39:0/4156815614 shutdown_connections 2024-09-21T07:50:27.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.034+0000 7fb060400640 1 -- 172.21.15.39:0/4156815614 wait complete. 2024-09-21T07:50:27.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.036+0000 7fb060400640 1 Processor -- start 2024-09-21T07:50:27.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.036+0000 7fb060400640 1 -- start start 2024-09-21T07:50:27.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.038+0000 7fb060400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb0580d06d0 0x7fb058158be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:27.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.038+0000 7fb060400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb05815fb20 0x7fb0580b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:27.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.038+0000 7fb060400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb058162520 0x7fb058156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:27.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.038+0000 7fb060400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb058070320 con 0x7fb058162520 2024-09-21T07:50:27.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.038+0000 7fb060400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb0580701a0 con 0x7fb05815fb20 2024-09-21T07:50:27.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.038+0000 7fb060400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb0580704a0 con 0x7fb0580d06d0 2024-09-21T07:50:27.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.038+0000 7fb05e400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb058162520 0x7fb058156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:27.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.038+0000 7fb05e400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb058162520 0x7fb058156bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58420/0 (socket says 172.21.15.39:58420) 2024-09-21T07:50:27.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.038+0000 7fb05e400640 1 -- 172.21.15.39:0/69959909 learned_addr learned my addr 172.21.15.39:0/69959909 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:27.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.038+0000 7fb05da00640 1 --2- 172.21.15.39:0/69959909 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb0580d06d0 0x7fb058158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:27.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.038+0000 7fb05d000640 1 --2- 172.21.15.39:0/69959909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb05815fb20 0x7fb0580b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:27.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.039+0000 7fb05e400640 1 -- 172.21.15.39:0/69959909 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb0580d06d0 msgr2=0x7fb058158be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:27.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.039+0000 7fb05e400640 1 --2- 172.21.15.39:0/69959909 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb0580d06d0 0x7fb058158be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:27.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.039+0000 7fb05e400640 1 -- 172.21.15.39:0/69959909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb05815fb20 msgr2=0x7fb0580b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:27.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.039+0000 7fb05e400640 1 --2- 172.21.15.39:0/69959909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb05815fb20 0x7fb0580b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:27.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.039+0000 7fb05e400640 1 -- 172.21.15.39:0/69959909 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0580ae3b0 con 0x7fb058162520 2024-09-21T07:50:27.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.039+0000 7fb05e400640 1 --2- 172.21.15.39:0/69959909 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb058162520 0x7fb058156bd0 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7fb050009be0 tx=0x7fb050003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:27.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.039+0000 7fb055600640 1 -- 172.21.15.39:0/69959909 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb050031b60 con 0x7fb058162520 2024-09-21T07:50:27.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.039+0000 7fb055600640 1 -- 172.21.15.39:0/69959909 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb050031df0 con 0x7fb058162520 2024-09-21T07:50:27.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.039+0000 7fb060400640 1 -- 172.21.15.39:0/69959909 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0580ae690 con 0x7fb058162520 2024-09-21T07:50:27.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.039+0000 7fb055600640 1 -- 172.21.15.39:0/69959909 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0500332d0 con 0x7fb058162520 2024-09-21T07:50:27.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.040+0000 7fb060400640 1 -- 172.21.15.39:0/69959909 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0580b2ee0 con 0x7fb058162520 2024-09-21T07:50:27.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.041+0000 7fb060400640 1 -- 172.21.15.39:0/69959909 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb058070a30 con 0x7fb058162520 2024-09-21T07:50:27.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.042+0000 7fb055600640 1 -- 172.21.15.39:0/69959909 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb050033470 con 0x7fb058162520 2024-09-21T07:50:27.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.043+0000 7fb055600640 1 --2- 172.21.15.39:0/69959909 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb038078b90 0x7fb03807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:27.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.043+0000 7fb055600640 1 -- 172.21.15.39:0/69959909 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(238..238 src has 1..238) ==== 7262+0+0 (secure 0 0 0) 0x7fb0500bc780 con 0x7fb058162520 2024-09-21T07:50:27.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.043+0000 7fb05da00640 1 --2- 172.21.15.39:0/69959909 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb038078b90 0x7fb03807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:27.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.044+0000 7fb05da00640 1 --2- 172.21.15.39:0/69959909 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb038078b90 0x7fb03807b0a0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fb04c001720 tx=0x7fb04c009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:27.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.047+0000 7fb055600640 1 -- 172.21.15.39:0/69959909 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb05003f030 con 0x7fb058162520 2024-09-21T07:50:27.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.283+0000 7fb060400640 1 -- 172.21.15.39:0/69959909 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb05815c850 con 0x7fb058162520 2024-09-21T07:50:27.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.283+0000 7fb055600640 1 -- 172.21.15.39:0/69959909 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 255 v255) ==== 95+0+5290 (secure 0 0 0) 0x7fb05002e050 con 0x7fb058162520 2024-09-21T07:50:27.284 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:50:27.284 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":255,"btime":"2024-09-21T07:50:26:869056+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9967,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":254}],"filesystems":[{"mdsmap":{"epoch":255,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:50:25.873602+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":237,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":10635,"mds_1":10632,"mds_2":8531},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8531":{"gid":8531,"name":"d","rank":2,"incarnation":253,"state":"up:reconnect","state_seq":16,"addr":"172.21.15.148:6837/1438851059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1438851059},{"type":"v1","addr":"172.21.15.148:6837","nonce":1438851059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10632":{"gid":10632,"name":"a","rank":1,"incarnation":247,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6834/4143807732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4143807732},{"type":"v1","addr":"172.21.15.39:6834","nonce":4143807732}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10635":{"gid":10635,"name":"c","rank":0,"incarnation":240,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/2824493631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2824493631},{"type":"v1","addr":"172.21.15.39:6835","nonce":2824493631}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10635,"qdb_cluster":[10635,10632]},"id":13}]} 2024-09-21T07:50:27.284 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 255 2024-09-21T07:50:27.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.285+0000 7fb060400640 1 -- 172.21.15.39:0/69959909 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb038078b90 msgr2=0x7fb03807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:27.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.285+0000 7fb060400640 1 --2- 172.21.15.39:0/69959909 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb038078b90 0x7fb03807b0a0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fb04c001720 tx=0x7fb04c009040 comp rx=0 tx=0).stop 2024-09-21T07:50:27.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.285+0000 7fb060400640 1 -- 172.21.15.39:0/69959909 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb058162520 msgr2=0x7fb058156bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:27.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.285+0000 7fb060400640 1 --2- 172.21.15.39:0/69959909 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb058162520 0x7fb058156bd0 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7fb050009be0 tx=0x7fb050003ba0 comp rx=0 tx=0).stop 2024-09-21T07:50:27.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.285+0000 7fb060400640 1 -- 172.21.15.39:0/69959909 shutdown_connections 2024-09-21T07:50:27.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.285+0000 7fb060400640 1 --2- 172.21.15.39:0/69959909 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb038078b90 0x7fb03807b0a0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:27.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.285+0000 7fb060400640 1 --2- 172.21.15.39:0/69959909 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb058162520 0x7fb058156bd0 unknown :-1 s=CLOSED pgs=2325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:27.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.285+0000 7fb060400640 1 --2- 172.21.15.39:0/69959909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb05815fb20 0x7fb0580b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:27.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.286+0000 7fb060400640 1 --2- 172.21.15.39:0/69959909 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb0580d06d0 0x7fb058158be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:27.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.286+0000 7fb060400640 1 -- 172.21.15.39:0/69959909 >> 172.21.15.39:0/69959909 conn(0x7fb05806cfc0 msgr2=0x7fb058160fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:27.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.286+0000 7fb060400640 1 -- 172.21.15.39:0/69959909 shutdown_connections 2024-09-21T07:50:27.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:27.286+0000 7fb060400640 1 -- 172.21.15.39:0/69959909 wait complete. 2024-09-21T07:50:27.710 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 255, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:50:25.873602+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 237, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 10635, 'mds_1': 10632, 'mds_2': 8531}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8531': {'gid': 8531, 'name': 'd', 'rank': 2, 'incarnation': 253, 'state': 'up:reconnect', 'state_seq': 16, 'addr': '172.21.15.148:6837/1438851059', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6835', 'nonce': 1438851059}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 1438851059}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10632': {'gid': 10632, 'name': 'a', 'rank': 1, 'incarnation': 247, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.39:6834/4143807732', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 4143807732}, {'type': 'v1', 'addr': '172.21.15.39:6834', 'nonce': 4143807732}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10635': {'gid': 10635, 'name': 'c', 'rank': 0, 'incarnation': 240, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.39:6835/2824493631', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 2824493631}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 2824493631}]}, 'join_fscid': -1, 'export_targets': [1, 2], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10635, 'qdb_cluster': [10635, 10632]} 2024-09-21T07:50:27.711 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8531:up:reconnect 2024-09-21T07:50:28.712 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:28.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:28.954+0000 7fd62da00640 1 -- 172.21.15.39:0/3139492341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd628160660 msgr2=0x7fd6280cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:28.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:28.954+0000 7fd62da00640 1 --2- 172.21.15.39:0/3139492341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd628160660 0x7fd6280cf710 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7fd610005fd0 tx=0x7fd61002a5d0 comp rx=0 tx=0).stop 2024-09-21T07:50:28.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:28.954+0000 7fd62da00640 1 -- 172.21.15.39:0/3139492341 shutdown_connections 2024-09-21T07:50:28.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:28.954+0000 7fd62da00640 1 --2- 172.21.15.39:0/3139492341 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd6280cfc50 0x7fd6280d0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:28.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:28.954+0000 7fd62da00640 1 --2- 172.21.15.39:0/3139492341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd628160660 0x7fd6280cf710 unknown :-1 s=CLOSED pgs=2326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:28.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:28.954+0000 7fd62da00640 1 -- 172.21.15.39:0/3139492341 >> 172.21.15.39:0/3139492341 conn(0x7fd62806cfc0 msgr2=0x7fd62806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:28.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:28.954+0000 7fd62da00640 1 -- 172.21.15.39:0/3139492341 shutdown_connections 2024-09-21T07:50:28.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:28.954+0000 7fd62da00640 1 -- 172.21.15.39:0/3139492341 wait complete. 2024-09-21T07:50:28.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:28.956+0000 7fd62da00640 1 Processor -- start 2024-09-21T07:50:28.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:28.956+0000 7fd62da00640 1 -- start start 2024-09-21T07:50:28.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:28.958+0000 7fd62da00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6280cfc50 0x7fd628148be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:28.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:28.958+0000 7fd62da00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd628160250 0x7fd6281f7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:28.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:28.958+0000 7fd62da00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd628160660 0x7fd6281f7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:28.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:28.958+0000 7fd62da00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd6280ccdc0 con 0x7fd6280cfc50 2024-09-21T07:50:28.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:28.958+0000 7fd62da00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd6280ccc40 con 0x7fd628160250 2024-09-21T07:50:28.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:28.958+0000 7fd62da00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd6280ccf40 con 0x7fd628160660 2024-09-21T07:50:28.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:28.958+0000 7fd627e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6280cfc50 0x7fd628148be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:28.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:28.958+0000 7fd627e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6280cfc50 0x7fd628148be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58448/0 (socket says 172.21.15.39:58448) 2024-09-21T07:50:28.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:28.958+0000 7fd627e00640 1 -- 172.21.15.39:0/3255723286 learned_addr learned my addr 172.21.15.39:0/3255723286 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:28.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:28.958+0000 7fd62d000640 1 --2- 172.21.15.39:0/3255723286 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd628160660 0x7fd6281f7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:28.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:28.958+0000 7fd61f400640 1 --2- 172.21.15.39:0/3255723286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd628160250 0x7fd6281f7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:28.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:28.959+0000 7fd627e00640 1 -- 172.21.15.39:0/3255723286 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd628160660 msgr2=0x7fd6281f7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:28.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:28.959+0000 7fd627e00640 1 --2- 172.21.15.39:0/3255723286 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd628160660 0x7fd6281f7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:28.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:28.959+0000 7fd627e00640 1 -- 172.21.15.39:0/3255723286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd628160250 msgr2=0x7fd6281f7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:28.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:28.959+0000 7fd627e00640 1 --2- 172.21.15.39:0/3255723286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd628160250 0x7fd6281f7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:28.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:28.959+0000 7fd627e00640 1 -- 172.21.15.39:0/3255723286 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd628149120 con 0x7fd6280cfc50 2024-09-21T07:50:28.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:28.959+0000 7fd627e00640 1 --2- 172.21.15.39:0/3255723286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6280cfc50 0x7fd628148be0 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7fd61800c3a0 tx=0x7fd61800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:28.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:28.959+0000 7fd624c00640 1 -- 172.21.15.39:0/3255723286 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd618017070 con 0x7fd6280cfc50 2024-09-21T07:50:28.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:28.959+0000 7fd624c00640 1 -- 172.21.15.39:0/3255723286 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6180053c0 con 0x7fd6280cfc50 2024-09-21T07:50:28.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:28.959+0000 7fd624c00640 1 -- 172.21.15.39:0/3255723286 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd618003850 con 0x7fd6280cfc50 2024-09-21T07:50:28.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:28.959+0000 7fd62da00640 1 -- 172.21.15.39:0/3255723286 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6280aafd0 con 0x7fd6280cfc50 2024-09-21T07:50:28.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:28.959+0000 7fd62da00640 1 -- 172.21.15.39:0/3255723286 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6280ab590 con 0x7fd6280cfc50 2024-09-21T07:50:28.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:28.961+0000 7fd62da00640 1 -- 172.21.15.39:0/3255723286 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6280cd3f0 con 0x7fd6280cfc50 2024-09-21T07:50:28.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:28.962+0000 7fd624c00640 1 -- 172.21.15.39:0/3255723286 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd61801a070 con 0x7fd6280cfc50 2024-09-21T07:50:28.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:28.963+0000 7fd624c00640 1 --2- 172.21.15.39:0/3255723286 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd604078b90 0x7fd60407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:28.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:28.963+0000 7fd624c00640 1 -- 172.21.15.39:0/3255723286 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(238..238 src has 1..238) ==== 7262+0+0 (secure 0 0 0) 0x7fd61809a5c0 con 0x7fd6280cfc50 2024-09-21T07:50:28.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:28.963+0000 7fd61f400640 1 --2- 172.21.15.39:0/3255723286 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd604078b90 0x7fd60407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:28.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:28.964+0000 7fd61f400640 1 --2- 172.21.15.39:0/3255723286 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd604078b90 0x7fd60407b0a0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fd610033000 tx=0x7fd610004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:28.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:28.968+0000 7fd624c00640 1 -- 172.21.15.39:0/3255723286 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd618061e20 con 0x7fd6280cfc50 2024-09-21T07:50:29.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.204+0000 7fd62da00640 1 -- 172.21.15.39:0/3255723286 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd628073da0 con 0x7fd6280cfc50 2024-09-21T07:50:29.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.204+0000 7fd624c00640 1 -- 172.21.15.39:0/3255723286 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 257 v257) ==== 95+0+5292 (secure 0 0 0) 0x7fd618066850 con 0x7fd6280cfc50 2024-09-21T07:50:29.206 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:50:29.206 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":257,"btime":"2024-09-21T07:50:28:886412+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9967,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":254}],"filesystems":[{"mdsmap":{"epoch":257,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:50:28.886406+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":237,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":10635,"mds_1":10632,"mds_2":8531},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8531":{"gid":8531,"name":"d","rank":2,"incarnation":253,"state":"up:active","state_seq":18,"addr":"172.21.15.148:6837/1438851059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1438851059},{"type":"v1","addr":"172.21.15.148:6837","nonce":1438851059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10632":{"gid":10632,"name":"a","rank":1,"incarnation":247,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6834/4143807732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4143807732},{"type":"v1","addr":"172.21.15.39:6834","nonce":4143807732}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10635":{"gid":10635,"name":"c","rank":0,"incarnation":240,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/2824493631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2824493631},{"type":"v1","addr":"172.21.15.39:6835","nonce":2824493631}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10635,"qdb_cluster":[8531,10632,10635]},"id":13}]} 2024-09-21T07:50:29.207 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 257 2024-09-21T07:50:29.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.207+0000 7fd62da00640 1 -- 172.21.15.39:0/3255723286 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd604078b90 msgr2=0x7fd60407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:29.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.207+0000 7fd62da00640 1 --2- 172.21.15.39:0/3255723286 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd604078b90 0x7fd60407b0a0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fd610033000 tx=0x7fd610004730 comp rx=0 tx=0).stop 2024-09-21T07:50:29.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.207+0000 7fd62da00640 1 -- 172.21.15.39:0/3255723286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6280cfc50 msgr2=0x7fd628148be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:29.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.207+0000 7fd62da00640 1 --2- 172.21.15.39:0/3255723286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6280cfc50 0x7fd628148be0 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7fd61800c3a0 tx=0x7fd61800c860 comp rx=0 tx=0).stop 2024-09-21T07:50:29.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.207+0000 7fd62da00640 1 -- 172.21.15.39:0/3255723286 shutdown_connections 2024-09-21T07:50:29.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.207+0000 7fd62da00640 1 --2- 172.21.15.39:0/3255723286 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd604078b90 0x7fd60407b0a0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.207+0000 7fd62da00640 1 --2- 172.21.15.39:0/3255723286 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd628160660 0x7fd6281f7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.207+0000 7fd62da00640 1 --2- 172.21.15.39:0/3255723286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd628160250 0x7fd6281f7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.207+0000 7fd62da00640 1 --2- 172.21.15.39:0/3255723286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6280cfc50 0x7fd628148be0 unknown :-1 s=CLOSED pgs=2327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.207+0000 7fd62da00640 1 -- 172.21.15.39:0/3255723286 >> 172.21.15.39:0/3255723286 conn(0x7fd62806cfc0 msgr2=0x7fd6280735b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:29.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.207+0000 7fd62da00640 1 -- 172.21.15.39:0/3255723286 shutdown_connections 2024-09-21T07:50:29.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.207+0000 7fd62da00640 1 -- 172.21.15.39:0/3255723286 wait complete. 2024-09-21T07:50:29.634 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 257, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:50:28.886406+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 237, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 10635, 'mds_1': 10632, 'mds_2': 8531}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8531': {'gid': 8531, 'name': 'd', 'rank': 2, 'incarnation': 253, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.148:6837/1438851059', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6835', 'nonce': 1438851059}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 1438851059}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10632': {'gid': 10632, 'name': 'a', 'rank': 1, 'incarnation': 247, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.39:6834/4143807732', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 4143807732}, {'type': 'v1', 'addr': '172.21.15.39:6834', 'nonce': 4143807732}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10635': {'gid': 10635, 'name': 'c', 'rank': 0, 'incarnation': 240, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.39:6835/2824493631', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 2824493631}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 2824493631}]}, 'join_fscid': -1, 'export_targets': [1, 2], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10635, 'qdb_cluster': [8531, 10632, 10635]} 2024-09-21T07:50:29.635 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-21T07:50:29.635 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-21T07:50:29.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.862+0000 7f2368000640 1 -- 172.21.15.39:0/1432510453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23600cb8a0 msgr2=0x7f2360156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:29.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.862+0000 7f2368000640 1 --2- 172.21.15.39:0/1432510453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23600cb8a0 0x7f2360156bb0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f234c009830 tx=0x7f234c02a520 comp rx=0 tx=0).stop 2024-09-21T07:50:29.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.862+0000 7f2368000640 1 -- 172.21.15.39:0/1432510453 shutdown_connections 2024-09-21T07:50:29.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.862+0000 7f2368000640 1 --2- 172.21.15.39:0/1432510453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23600cb8a0 0x7f2360156bb0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.862+0000 7f2368000640 1 -- 172.21.15.39:0/1432510453 >> 172.21.15.39:0/1432510453 conn(0x7f23600b2460 msgr2=0x7f236006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:29.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.862+0000 7f2368000640 1 -- 172.21.15.39:0/1432510453 shutdown_connections 2024-09-21T07:50:29.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.863+0000 7f2368000640 1 -- 172.21.15.39:0/1432510453 wait complete. 2024-09-21T07:50:29.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.865+0000 7f2368000640 1 Processor -- start 2024-09-21T07:50:29.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.865+0000 7f2368000640 1 -- start start 2024-09-21T07:50:29.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.866+0000 7f2368000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f23600b30a0 0x7f23601f5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:29.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.866+0000 7f2368000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23600cb8a0 0x7f23600ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:29.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.866+0000 7f2368000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23600ccc60 0x7f23600ce830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:29.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.866+0000 7f2368000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f23600af520 con 0x7f23600cb8a0 2024-09-21T07:50:29.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.866+0000 7f2368000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f23600af3a0 con 0x7f23600ccc60 2024-09-21T07:50:29.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.866+0000 7f2368000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f23600af6a0 con 0x7f23600b30a0 2024-09-21T07:50:29.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.867+0000 7f2364c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23600cb8a0 0x7f23600ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:29.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.867+0000 7f2364c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23600cb8a0 0x7f23600ce2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58460/0 (socket says 172.21.15.39:58460) 2024-09-21T07:50:29.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.867+0000 7f2364c00640 1 -- 172.21.15.39:0/3883388089 learned_addr learned my addr 172.21.15.39:0/3883388089 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:29.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.867+0000 7f2365600640 1 --2- 172.21.15.39:0/3883388089 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f23600b30a0 0x7f23601f5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:29.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.867+0000 7f2366000640 1 --2- 172.21.15.39:0/3883388089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23600ccc60 0x7f23600ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:29.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.867+0000 7f2364c00640 1 -- 172.21.15.39:0/3883388089 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f23600b30a0 msgr2=0x7f23601f5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:29.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.867+0000 7f2364c00640 1 --2- 172.21.15.39:0/3883388089 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f23600b30a0 0x7f23601f5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.867+0000 7f2364c00640 1 -- 172.21.15.39:0/3883388089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23600ccc60 msgr2=0x7f23600ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:29.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.867+0000 7f2364c00640 1 --2- 172.21.15.39:0/3883388089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23600ccc60 0x7f23600ce830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.867+0000 7f2364c00640 1 -- 172.21.15.39:0/3883388089 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23600c5470 con 0x7f23600cb8a0 2024-09-21T07:50:29.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.867+0000 7f2364c00640 1 --2- 172.21.15.39:0/3883388089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23600cb8a0 0x7f23600ce2f0 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7f234c009950 tx=0x7f234c033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:29.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.868+0000 7f235d600640 1 -- 172.21.15.39:0/3883388089 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234c031b50 con 0x7f23600cb8a0 2024-09-21T07:50:29.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.868+0000 7f235d600640 1 -- 172.21.15.39:0/3883388089 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f234c031de0 con 0x7f23600cb8a0 2024-09-21T07:50:29.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.868+0000 7f2368000640 1 -- 172.21.15.39:0/3883388089 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23600c5af0 con 0x7f23600cb8a0 2024-09-21T07:50:29.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.868+0000 7f235d600640 1 -- 172.21.15.39:0/3883388089 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234c005280 con 0x7f23600cb8a0 2024-09-21T07:50:29.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.868+0000 7f2368000640 1 -- 172.21.15.39:0/3883388089 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23600c60b0 con 0x7f23600cb8a0 2024-09-21T07:50:29.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.870+0000 7f235d600640 1 -- 172.21.15.39:0/3883388089 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f234c005420 con 0x7f23600cb8a0 2024-09-21T07:50:29.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.871+0000 7f235d600640 1 --2- 172.21.15.39:0/3883388089 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2340078bc0 0x7f234007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:29.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.871+0000 7f235d600640 1 -- 172.21.15.39:0/3883388089 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(238..238 src has 1..238) ==== 7262+0+0 (secure 0 0 0) 0x7f234c0bc470 con 0x7f23600cb8a0 2024-09-21T07:50:29.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.872+0000 7f236f02e740 1 Processor -- start 2024-09-21T07:50:29.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.872+0000 7f2365600640 1 --2- 172.21.15.39:0/3883388089 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2340078bc0 0x7f234007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:29.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.872+0000 7f236f02e740 1 -- start start 2024-09-21T07:50:29.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.873+0000 7f2365600640 1 --2- 172.21.15.39:0/3883388089 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2340078bc0 0x7f234007b0d0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f2354000f60 tx=0x7f2354009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:29.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.873+0000 7f236f02e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c9ea3270 0x55b6ca273970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:29.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.873+0000 7f236f02e740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b6c9e7a3b0 0x55b6ca273eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:29.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.873+0000 7f236f02e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c9e82590 0x55b6ca243f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:29.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.873+0000 7f236f02e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b6ca1cd430 con 0x55b6c9ea3270 2024-09-21T07:50:29.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.873+0000 7f236f02e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b6ca1bd5d0 con 0x55b6c9e82590 2024-09-21T07:50:29.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.873+0000 7f236f02e740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55b6ca1aca00 con 0x55b6c9e7a3b0 2024-09-21T07:50:29.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.874+0000 7f2365600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c9ea3270 0x55b6ca273970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:29.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.874+0000 7f2365600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c9ea3270 0x55b6ca273970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58476/0 (socket says 172.21.15.39:58476) 2024-09-21T07:50:29.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.874+0000 7f2365600640 1 -- 172.21.15.39:0/1670112542 learned_addr learned my addr 172.21.15.39:0/1670112542 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:29.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.874+0000 7f2366000640 1 --2- 172.21.15.39:0/1670112542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c9e82590 0x55b6ca243f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:29.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.874+0000 7f2364c00640 1 --2- 172.21.15.39:0/1670112542 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b6c9e7a3b0 0x55b6ca273eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:29.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.874+0000 7f2365600640 1 -- 172.21.15.39:0/1670112542 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b6c9e7a3b0 msgr2=0x55b6ca273eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:29.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.874+0000 7f2365600640 1 --2- 172.21.15.39:0/1670112542 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b6c9e7a3b0 0x55b6ca273eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.874+0000 7f2365600640 1 -- 172.21.15.39:0/1670112542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c9e82590 msgr2=0x55b6ca243f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:29.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.874+0000 7f2365600640 1 --2- 172.21.15.39:0/1670112542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c9e82590 0x55b6ca243f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.874+0000 7f2365600640 1 -- 172.21.15.39:0/1670112542 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6ca175520 con 0x55b6c9ea3270 2024-09-21T07:50:29.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.874+0000 7f2365600640 1 --2- 172.21.15.39:0/1670112542 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c9ea3270 0x55b6ca273970 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7f2354001e90 tx=0x7f2354011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3178ebdcada6b948 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:29.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.875+0000 7f2351c00640 1 -- 172.21.15.39:0/1670112542 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23540035f0 con 0x55b6c9ea3270 2024-09-21T07:50:29.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.875+0000 7f2351c00640 1 -- 172.21.15.39:0/1670112542 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2354003880 con 0x55b6c9ea3270 2024-09-21T07:50:29.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.875+0000 7f2351c00640 1 -- 172.21.15.39:0/1670112542 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2354003c70 con 0x55b6c9ea3270 2024-09-21T07:50:29.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.875+0000 7f236f02e740 1 -- 172.21.15.39:0/1670112542 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c9ea3270 msgr2=0x55b6ca273970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:29.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.875+0000 7f236f02e740 1 --2- 172.21.15.39:0/1670112542 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c9ea3270 0x55b6ca273970 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7f2354001e90 tx=0x7f2354011000 comp rx=0 tx=0).stop 2024-09-21T07:50:29.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.876+0000 7f236f02e740 1 -- 172.21.15.39:0/1670112542 shutdown_connections 2024-09-21T07:50:29.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.876+0000 7f236f02e740 1 --2- 172.21.15.39:0/1670112542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c9e82590 0x55b6ca243f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.876+0000 7f236f02e740 1 --2- 172.21.15.39:0/1670112542 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b6c9e7a3b0 0x55b6ca273eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.876+0000 7f236f02e740 1 --2- 172.21.15.39:0/1670112542 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c9ea3270 0x55b6ca273970 unknown :-1 s=CLOSED pgs=2330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.876+0000 7f236f02e740 1 -- 172.21.15.39:0/1670112542 >> 172.21.15.39:0/1670112542 conn(0x55b6c9e973c0 msgr2=0x55b6ca249970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:29.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.876+0000 7f236f02e740 1 -- 172.21.15.39:0/1670112542 shutdown_connections 2024-09-21T07:50:29.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.876+0000 7f236f02e740 1 -- 172.21.15.39:0/1670112542 wait complete. 2024-09-21T07:50:29.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.877+0000 7f236f02e740 1 -- start start 2024-09-21T07:50:29.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.877+0000 7f236f02e740 1 Processor -- start 2024-09-21T07:50:29.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.878+0000 7f236f02e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c9e7a3b0 0x55b6ca288d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:29.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.878+0000 7f236f02e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c9e82590 0x55b6ca289280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:29.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.878+0000 7f236f02e740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b6c9ea3270 0x55b6ca2a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:29.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.878+0000 7f236f02e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b6ca153330 con 0x55b6c9e7a3b0 2024-09-21T07:50:29.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.878+0000 7f236f02e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b6ca1b0810 con 0x55b6c9e82590 2024-09-21T07:50:29.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.878+0000 7f236f02e740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55b6c9f1d620 con 0x55b6c9ea3270 2024-09-21T07:50:29.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.878+0000 7f236f02e740 10 client.0 _ll_register_callbacks cb 0x55b6ca158420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:50:29.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.879+0000 7f2365600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c9e7a3b0 0x55b6ca288d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:29.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.879+0000 7f2365600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c9e7a3b0 0x55b6ca288d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58492/0 (socket says 172.21.15.39:58492) 2024-09-21T07:50:29.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.879+0000 7f2365600640 1 -- 172.21.15.39:0/3959579188 learned_addr learned my addr 172.21.15.39:0/3959579188 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:29.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.879+0000 7f2364c00640 1 --2- 172.21.15.39:0/3959579188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c9e82590 0x55b6ca289280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:29.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.879+0000 7f2366000640 1 --2- 172.21.15.39:0/3959579188 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b6c9ea3270 0x55b6ca2a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:29.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.879+0000 7f2365600640 1 -- 172.21.15.39:0/3959579188 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b6c9ea3270 msgr2=0x55b6ca2a0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:29.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.879+0000 7f2365600640 1 --2- 172.21.15.39:0/3959579188 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b6c9ea3270 0x55b6ca2a0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.879+0000 7f2365600640 1 -- 172.21.15.39:0/3959579188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c9e82590 msgr2=0x55b6ca289280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:29.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.879+0000 7f2365600640 1 --2- 172.21.15.39:0/3959579188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c9e82590 0x55b6ca289280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.879+0000 7f2365600640 1 -- 172.21.15.39:0/3959579188 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6c9f0e850 con 0x55b6c9e7a3b0 2024-09-21T07:50:29.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.879+0000 7f2365600640 1 --2- 172.21.15.39:0/3959579188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c9e7a3b0 0x55b6ca288d40 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7f235400e530 tx=0x7f2354003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab58fdd991a26a52 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:29.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.879+0000 7f2346000640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:50:29.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.880+0000 7f2346000640 1 -- 172.21.15.39:0/3959579188 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23540035f0 con 0x55b6c9e7a3b0 2024-09-21T07:50:29.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.880+0000 7f2346000640 1 -- 172.21.15.39:0/3959579188 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(238..238 src has 1..238) ==== 7262+0+0 (secure 0 0 0) 0x7f2354021860 con 0x55b6c9e7a3b0 2024-09-21T07:50:29.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.880+0000 7f236f02e740 1 -- 172.21.15.39:0/3959579188 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2354003880 con 0x55b6c9e7a3b0 2024-09-21T07:50:29.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.880+0000 7f2346000640 1 -- 172.21.15.39:0/3959579188 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=257) ==== 24+0+0 (secure 0 0 0) 0x7f2354021a60 con 0x55b6c9e7a3b0 2024-09-21T07:50:29.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.880+0000 7f236f02e740 10 client.10992 fetch_fsmap learned FSMap version 257 2024-09-21T07:50:29.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.880+0000 7f236f02e740 1 -- 172.21.15.39:0/3959579188 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=257,monmap=2+}) -- 0x55b6c9f856b0 con 0x55b6c9e7a3b0 2024-09-21T07:50:29.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.880+0000 7f2346000640 1 -- 172.21.15.39:0/3959579188 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2354003880 con 0x55b6c9e7a3b0 2024-09-21T07:50:29.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.880+0000 7f2346000640 1 -- 172.21.15.39:0/3959579188 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 257) ==== 3550+0+0 (secure 0 0 0) 0x7f2354036080 con 0x55b6c9e7a3b0 2024-09-21T07:50:29.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.880+0000 7f236f02e740 10 client.10992 fetch_fsmap finished waiting for FSMap version 257 2024-09-21T07:50:29.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.880+0000 7f236f02e740 10 client.10992 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T07:50:29.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.880+0000 7f236f02e740 20 client.10992 populate_metadata read hostname 'smithi039' 2024-09-21T07:50:29.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.880+0000 7f236f02e740 1 --2- 172.21.15.39:0/3959579188 >> [v2:172.21.15.148:6835/1438851059,v1:172.21.15.148:6837/1438851059] conn(0x55b6c9e61e40 0x55b6ca2ab0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:29.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.880+0000 7f236f02e740 4 client.10992 mds_command: new command op to 8531 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:50:29.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.880+0000 7f236f02e740 1 -- 172.21.15.39:0/3959579188 --> [v2:172.21.15.148:6835/1438851059,v1:172.21.15.148:6837/1438851059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6ca1bf520 con 0x55b6c9e61e40 2024-09-21T07:50:29.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.881+0000 7f2364c00640 1 --2- 172.21.15.39:0/3959579188 >> [v2:172.21.15.148:6835/1438851059,v1:172.21.15.148:6837/1438851059] conn(0x55b6c9e61e40 0x55b6ca2ab0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:29.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.882+0000 7f2364c00640 1 --2- 172.21.15.39:0/3959579188 >> [v2:172.21.15.148:6835/1438851059,v1:172.21.15.148:6837/1438851059] conn(0x55b6c9e61e40 0x55b6ca2ab0f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=73fb7ce8379ed3ec server_cookie=207f8d1daa1587a9 in_seq=0 out_seq=0 2024-09-21T07:50:29.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.882+0000 7f2346000640 10 client.10992 ms_handle_connect on v2:172.21.15.148:6835/1438851059 2024-09-21T07:50:29.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.883+0000 7f2346000640 1 -- 172.21.15.39:0/3959579188 <== mds.2 v2:172.21.15.148:6835/1438851059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b6ca1bf520 con 0x55b6c9e61e40 2024-09-21T07:50:29.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.883+0000 7f2346000640 10 client.10992 handle_command_reply: tid=0 2024-09-21T07:50:29.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.883+0000 7f236f02e740 1 client.10992 shutdown 2024-09-21T07:50:29.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.883+0000 7f236f02e740 1 -- 172.21.15.39:0/3959579188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c9e7a3b0 msgr2=0x55b6ca288d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:29.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.883+0000 7f236f02e740 1 --2- 172.21.15.39:0/3959579188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c9e7a3b0 0x55b6ca288d40 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7f235400e530 tx=0x7f2354003ce0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.883+0000 7f236f02e740 1 -- 172.21.15.39:0/3959579188 shutdown_connections 2024-09-21T07:50:29.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.883+0000 7f236f02e740 1 --2- 172.21.15.39:0/3959579188 >> [v2:172.21.15.148:6835/1438851059,v1:172.21.15.148:6837/1438851059] conn(0x55b6c9e61e40 0x55b6ca2ab0f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.883+0000 7f236f02e740 1 --2- 172.21.15.39:0/3959579188 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b6c9ea3270 0x55b6ca2a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.883+0000 7f236f02e740 1 --2- 172.21.15.39:0/3959579188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c9e82590 0x55b6ca289280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.883+0000 7f236f02e740 1 --2- 172.21.15.39:0/3959579188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c9e7a3b0 0x55b6ca288d40 unknown :-1 s=CLOSED pgs=2331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.883+0000 7f236f02e740 1 -- 172.21.15.39:0/3959579188 >> 172.21.15.39:0/3959579188 conn(0x55b6c9e973c0 msgr2=0x55b6ca26aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:29.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.883+0000 7f2346000640 0 client.10992 ms_handle_reset on v2:172.21.15.148:6835/1438851059 2024-09-21T07:50:29.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.883+0000 7f236f02e740 1 -- 172.21.15.39:0/3959579188 shutdown_connections 2024-09-21T07:50:29.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.883+0000 7f236f02e740 1 -- 172.21.15.39:0/3959579188 wait complete. 2024-09-21T07:50:29.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.884+0000 7f236f02e740 20 client.10992 trim_cache size 0 max 16384 2024-09-21T07:50:29.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.903+0000 7f236f02e740 1 Processor -- start 2024-09-21T07:50:29.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.903+0000 7f236f02e740 1 -- start start 2024-09-21T07:50:29.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.905+0000 7f236f02e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c9e7a3b0 0x55b6ca290840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:29.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.905+0000 7f236f02e740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b6c9e82590 0x55b6ca28f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:29.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.905+0000 7f236f02e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c9ea3270 0x55b6ca28fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:29.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.905+0000 7f236f02e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2360204300 con 0x55b6c9e7a3b0 2024-09-21T07:50:29.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.905+0000 7f236f02e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f23602049d0 con 0x55b6c9ea3270 2024-09-21T07:50:29.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.905+0000 7f236f02e740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f23602032e0 con 0x55b6c9e82590 2024-09-21T07:50:29.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.905+0000 7f2365600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c9e7a3b0 0x55b6ca290840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:29.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.905+0000 7f2365600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c9e7a3b0 0x55b6ca290840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58504/0 (socket says 172.21.15.39:58504) 2024-09-21T07:50:29.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.905+0000 7f2365600640 1 -- 172.21.15.39:0/178384507 learned_addr learned my addr 172.21.15.39:0/178384507 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:29.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.905+0000 7f2366000640 1 --2- 172.21.15.39:0/178384507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c9ea3270 0x55b6ca28fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:29.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.905+0000 7f2364c00640 1 --2- 172.21.15.39:0/178384507 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b6c9e82590 0x55b6ca28f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:29.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.906+0000 7f2366000640 1 -- 172.21.15.39:0/178384507 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b6c9e82590 msgr2=0x55b6ca28f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:29.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.906+0000 7f2366000640 1 --2- 172.21.15.39:0/178384507 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b6c9e82590 0x55b6ca28f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.906+0000 7f2366000640 1 -- 172.21.15.39:0/178384507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c9e7a3b0 msgr2=0x55b6ca290840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:29.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.906+0000 7f2366000640 1 --2- 172.21.15.39:0/178384507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c9e7a3b0 0x55b6ca290840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.906+0000 7f2366000640 1 -- 172.21.15.39:0/178384507 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6ca26f190 con 0x55b6c9ea3270 2024-09-21T07:50:29.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.906+0000 7f2364c00640 1 --2- 172.21.15.39:0/178384507 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b6c9e82590 0x55b6ca28f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:50:29.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.906+0000 7f2366000640 1 --2- 172.21.15.39:0/178384507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c9ea3270 0x55b6ca28fe70 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f23580012e0 tx=0x7f23580136a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81a40d4aec3b478e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:29.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.906+0000 7f2351200640 1 -- 172.21.15.39:0/178384507 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235801a030 con 0x55b6c9ea3270 2024-09-21T07:50:29.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.907+0000 7f2351200640 1 -- 172.21.15.39:0/178384507 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2358016450 con 0x55b6c9ea3270 2024-09-21T07:50:29.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.907+0000 7f2351200640 1 -- 172.21.15.39:0/178384507 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2358016680 con 0x55b6c9ea3270 2024-09-21T07:50:29.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.907+0000 7f236f02e740 1 -- 172.21.15.39:0/178384507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c9ea3270 msgr2=0x55b6ca28fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:29.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.907+0000 7f236f02e740 1 --2- 172.21.15.39:0/178384507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c9ea3270 0x55b6ca28fe70 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f23580012e0 tx=0x7f23580136a0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.907+0000 7f236f02e740 1 -- 172.21.15.39:0/178384507 shutdown_connections 2024-09-21T07:50:29.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.907+0000 7f236f02e740 1 --2- 172.21.15.39:0/178384507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c9ea3270 0x55b6ca28fe70 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.907+0000 7f236f02e740 1 --2- 172.21.15.39:0/178384507 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b6c9e82590 0x55b6ca28f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.907+0000 7f236f02e740 1 --2- 172.21.15.39:0/178384507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c9e7a3b0 0x55b6ca290840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.907+0000 7f236f02e740 1 -- 172.21.15.39:0/178384507 >> 172.21.15.39:0/178384507 conn(0x55b6c9e973c0 msgr2=0x55b6ca244cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:29.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.908+0000 7f236f02e740 1 -- 172.21.15.39:0/178384507 shutdown_connections 2024-09-21T07:50:29.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.908+0000 7f236f02e740 1 -- 172.21.15.39:0/178384507 wait complete. 2024-09-21T07:50:29.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.908+0000 7f236f02e740 1 -- start start 2024-09-21T07:50:29.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.909+0000 7f236f02e740 1 Processor -- start 2024-09-21T07:50:29.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.910+0000 7f236f02e740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b6c9e7a3b0 0x55b6ca2a23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:29.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.910+0000 7f236f02e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c9e82590 0x55b6ca304bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:29.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.910+0000 7f236f02e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c9ea3270 0x55b6ca290840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:29.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.910+0000 7f236f02e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b6ca153330 con 0x55b6c9ea3270 2024-09-21T07:50:29.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.910+0000 7f236f02e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b6ca26a610 con 0x55b6c9e82590 2024-09-21T07:50:29.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.910+0000 7f236f02e740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55b6ca25e0c0 con 0x55b6c9e7a3b0 2024-09-21T07:50:29.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.910+0000 7f236f02e740 10 client.0 _ll_register_callbacks cb 0x7f2354001d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:50:29.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.910+0000 7f2366000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c9ea3270 0x55b6ca290840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:29.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.911+0000 7f2366000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c9ea3270 0x55b6ca290840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58520/0 (socket says 172.21.15.39:58520) 2024-09-21T07:50:29.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.911+0000 7f2366000640 1 -- 172.21.15.39:0/2856763010 learned_addr learned my addr 172.21.15.39:0/2856763010 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:29.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.911+0000 7f2365600640 1 --2- 172.21.15.39:0/2856763010 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b6c9e7a3b0 0x55b6ca2a23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:29.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.911+0000 7f2364c00640 1 --2- 172.21.15.39:0/2856763010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c9e82590 0x55b6ca304bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:29.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.911+0000 7f2366000640 1 -- 172.21.15.39:0/2856763010 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b6c9e7a3b0 msgr2=0x55b6ca2a23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:29.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.911+0000 7f2366000640 1 --2- 172.21.15.39:0/2856763010 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b6c9e7a3b0 0x55b6ca2a23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.911+0000 7f2366000640 1 -- 172.21.15.39:0/2856763010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c9e82590 msgr2=0x55b6ca304bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:29.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.911+0000 7f2366000640 1 --2- 172.21.15.39:0/2856763010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c9e82590 0x55b6ca304bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.911+0000 7f2366000640 1 -- 172.21.15.39:0/2856763010 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6ca289850 con 0x55b6c9ea3270 2024-09-21T07:50:29.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.911+0000 7f2366000640 1 --2- 172.21.15.39:0/2856763010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c9ea3270 0x55b6ca290840 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f2358018040 tx=0x7f2358016c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb5a51abe010a95 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:29.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.911+0000 7f2346000640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:50:29.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.911+0000 7f2346000640 1 -- 172.21.15.39:0/2856763010 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2358016e60 con 0x55b6c9ea3270 2024-09-21T07:50:29.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.912+0000 7f2346000640 1 -- 172.21.15.39:0/2856763010 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(238..238 src has 1..238) ==== 7262+0+0 (secure 0 0 0) 0x7f235801f820 con 0x55b6c9ea3270 2024-09-21T07:50:29.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.912+0000 7f236f02e740 1 -- 172.21.15.39:0/2856763010 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2358016450 con 0x55b6c9ea3270 2024-09-21T07:50:29.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.912+0000 7f2346000640 1 -- 172.21.15.39:0/2856763010 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=257) ==== 24+0+0 (secure 0 0 0) 0x7f235802f050 con 0x55b6c9ea3270 2024-09-21T07:50:29.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.912+0000 7f236f02e740 10 client.10998 fetch_fsmap learned FSMap version 257 2024-09-21T07:50:29.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.912+0000 7f236f02e740 1 -- 172.21.15.39:0/2856763010 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=257,monmap=2+}) -- 0x55b6ca26d8c0 con 0x55b6c9ea3270 2024-09-21T07:50:29.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.912+0000 7f2346000640 1 -- 172.21.15.39:0/2856763010 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2358016450 con 0x55b6c9ea3270 2024-09-21T07:50:29.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.912+0000 7f2346000640 1 -- 172.21.15.39:0/2856763010 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 257) ==== 3550+0+0 (secure 0 0 0) 0x7f2358032020 con 0x55b6c9ea3270 2024-09-21T07:50:29.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.912+0000 7f236f02e740 10 client.10998 fetch_fsmap finished waiting for FSMap version 257 2024-09-21T07:50:29.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.912+0000 7f236f02e740 10 client.10998 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T07:50:29.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.912+0000 7f236f02e740 20 client.10998 populate_metadata read hostname 'smithi039' 2024-09-21T07:50:29.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.912+0000 7f236f02e740 1 --2- 172.21.15.39:0/2856763010 >> [v2:172.21.15.148:6835/1438851059,v1:172.21.15.148:6837/1438851059] conn(0x55b6ca268e00 0x55b6ca269210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:29.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.912+0000 7f236f02e740 4 client.10998 mds_command: new command op to 8531 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:50:29.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.912+0000 7f236f02e740 1 -- 172.21.15.39:0/2856763010 --> [v2:172.21.15.148:6835/1438851059,v1:172.21.15.148:6837/1438851059] -- command(tid 0: {"prefix": "status"}) -- 0x55b6ca1bafd0 con 0x55b6ca268e00 2024-09-21T07:50:29.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.913+0000 7f2365600640 1 --2- 172.21.15.39:0/2856763010 >> [v2:172.21.15.148:6835/1438851059,v1:172.21.15.148:6837/1438851059] conn(0x55b6ca268e00 0x55b6ca269210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:29.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.913+0000 7f2365600640 1 --2- 172.21.15.39:0/2856763010 >> [v2:172.21.15.148:6835/1438851059,v1:172.21.15.148:6837/1438851059] conn(0x55b6ca268e00 0x55b6ca269210 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f86a98985ae9f688 server_cookie=a84ae8ea7ada0d1a in_seq=0 out_seq=0 2024-09-21T07:50:29.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.913+0000 7f2346000640 10 client.10998 ms_handle_connect on v2:172.21.15.148:6835/1438851059 2024-09-21T07:50:29.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.914+0000 7f2346000640 1 -- 172.21.15.39:0/2856763010 <== mds.2 v2:172.21.15.148:6835/1438851059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55b6ca1bafd0 con 0x55b6ca268e00 2024-09-21T07:50:29.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.914+0000 7f2346000640 10 client.10998 handle_command_reply: tid=0 2024-09-21T07:50:29.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.914+0000 7f236f02e740 1 client.10998 shutdown 2024-09-21T07:50:29.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.914+0000 7f236f02e740 1 -- 172.21.15.39:0/2856763010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c9ea3270 msgr2=0x55b6ca290840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:29.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.914+0000 7f236f02e740 1 --2- 172.21.15.39:0/2856763010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c9ea3270 0x55b6ca290840 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f2358018040 tx=0x7f2358016c70 comp rx=0 tx=0).stop 2024-09-21T07:50:29.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.914+0000 7f236f02e740 1 -- 172.21.15.39:0/2856763010 shutdown_connections 2024-09-21T07:50:29.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.914+0000 7f236f02e740 1 --2- 172.21.15.39:0/2856763010 >> [v2:172.21.15.148:6835/1438851059,v1:172.21.15.148:6837/1438851059] conn(0x55b6ca268e00 0x55b6ca269210 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.914+0000 7f236f02e740 1 --2- 172.21.15.39:0/2856763010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6c9ea3270 0x55b6ca290840 unknown :-1 s=CLOSED pgs=2332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.914+0000 7f236f02e740 1 --2- 172.21.15.39:0/2856763010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b6c9e82590 0x55b6ca304bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.914+0000 7f236f02e740 1 --2- 172.21.15.39:0/2856763010 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55b6c9e7a3b0 0x55b6ca2a23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.914+0000 7f2346000640 0 client.10998 ms_handle_reset on v2:172.21.15.148:6835/1438851059 2024-09-21T07:50:29.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.914+0000 7f236f02e740 1 -- 172.21.15.39:0/2856763010 >> 172.21.15.39:0/2856763010 conn(0x55b6c9e973c0 msgr2=0x55b6ca26b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:29.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.914+0000 7f236f02e740 1 -- 172.21.15.39:0/2856763010 shutdown_connections 2024-09-21T07:50:29.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.914+0000 7f236f02e740 1 -- 172.21.15.39:0/2856763010 wait complete. 2024-09-21T07:50:29.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.914+0000 7f236f02e740 20 client.10998 trim_cache size 0 max 16384 2024-09-21T07:50:29.919 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:50:29.919 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:50:29.919 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 2, 2024-09-21T07:50:29.919 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 8531, 2024-09-21T07:50:29.919 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:50:29.919 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:50:29.919 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:50:29.920 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 5.0474555270000003, 2024-09-21T07:50:29.920 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 257, 2024-09-21T07:50:29.920 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 238, 2024-09-21T07:50:29.920 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 238, 2024-09-21T07:50:29.920 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 59.803399923000001 2024-09-21T07:50:29.920 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:50:29.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.915+0000 7f2368000640 1 -- 172.21.15.39:0/3883388089 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2340078bc0 msgr2=0x7f234007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:29.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.915+0000 7f2368000640 1 --2- 172.21.15.39:0/3883388089 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2340078bc0 0x7f234007b0d0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f2354000f60 tx=0x7f2354009040 comp rx=0 tx=0).stop 2024-09-21T07:50:29.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.915+0000 7f2368000640 1 -- 172.21.15.39:0/3883388089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23600cb8a0 msgr2=0x7f23600ce2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:29.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.915+0000 7f2368000640 1 --2- 172.21.15.39:0/3883388089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23600cb8a0 0x7f23600ce2f0 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7f234c009950 tx=0x7f234c033ba0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.915+0000 7f2368000640 1 -- 172.21.15.39:0/3883388089 shutdown_connections 2024-09-21T07:50:29.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.915+0000 7f2368000640 1 --2- 172.21.15.39:0/3883388089 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2340078bc0 0x7f234007b0d0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.915+0000 7f2368000640 1 --2- 172.21.15.39:0/3883388089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23600ccc60 0x7f23600ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.915+0000 7f2368000640 1 --2- 172.21.15.39:0/3883388089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23600cb8a0 0x7f23600ce2f0 unknown :-1 s=CLOSED pgs=2329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.915+0000 7f2368000640 1 --2- 172.21.15.39:0/3883388089 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f23600b30a0 0x7f23601f5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.915+0000 7f2368000640 1 -- 172.21.15.39:0/3883388089 >> 172.21.15.39:0/3883388089 conn(0x7f23600b2460 msgr2=0x7f23600ae330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:29.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.915+0000 7f2368000640 1 -- 172.21.15.39:0/3883388089 shutdown_connections 2024-09-21T07:50:29.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:29.915+0000 7f2368000640 1 -- 172.21.15.39:0/3883388089 wait complete. 2024-09-21T07:50:29.938 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T07:50:30.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.165+0000 7fea46400640 1 -- 172.21.15.39:0/1295163561 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea400b3990 msgr2=0x7fea400b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:30.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.165+0000 7fea46400640 1 --2- 172.21.15.39:0/1295163561 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea400b3990 0x7fea400b4450 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fea28009860 tx=0x7fea2802a5a0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.165+0000 7fea46400640 1 -- 172.21.15.39:0/1295163561 shutdown_connections 2024-09-21T07:50:30.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.165+0000 7fea46400640 1 --2- 172.21.15.39:0/1295163561 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea400b3990 0x7fea400b4450 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.165+0000 7fea46400640 1 -- 172.21.15.39:0/1295163561 >> 172.21.15.39:0/1295163561 conn(0x7fea400b2460 msgr2=0x7fea4006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:30.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.165+0000 7fea46400640 1 -- 172.21.15.39:0/1295163561 shutdown_connections 2024-09-21T07:50:30.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.165+0000 7fea46400640 1 -- 172.21.15.39:0/1295163561 wait complete. 2024-09-21T07:50:30.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.167+0000 7fea46400640 1 Processor -- start 2024-09-21T07:50:30.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.168+0000 7fea46400640 1 -- start start 2024-09-21T07:50:30.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.169+0000 7fea46400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea400b3990 0x7fea4014eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.169+0000 7fea46400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea400b3da0 0x7fea400b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.169+0000 7fea46400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea40162f10 0x7fea400b1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.169+0000 7fea46400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fea400afb20 con 0x7fea400b3da0 2024-09-21T07:50:30.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.169+0000 7fea46400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fea400af9a0 con 0x7fea400b3990 2024-09-21T07:50:30.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.169+0000 7fea46400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fea40070dc0 con 0x7fea40162f10 2024-09-21T07:50:30.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.169+0000 7fea36a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea400b3da0 0x7fea400b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.169+0000 7fea36a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea400b3da0 0x7fea400b0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58526/0 (socket says 172.21.15.39:58526) 2024-09-21T07:50:30.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.169+0000 7fea36a00640 1 -- 172.21.15.39:0/533903845 learned_addr learned my addr 172.21.15.39:0/533903845 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:30.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.170+0000 7fea3f400640 1 --2- 172.21.15.39:0/533903845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea400b3990 0x7fea4014eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.170+0000 7fea3fe00640 1 --2- 172.21.15.39:0/533903845 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea40162f10 0x7fea400b1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.170+0000 7fea36a00640 1 -- 172.21.15.39:0/533903845 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea40162f10 msgr2=0x7fea400b1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:30.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.170+0000 7fea36a00640 1 --2- 172.21.15.39:0/533903845 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea40162f10 0x7fea400b1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.170+0000 7fea36a00640 1 -- 172.21.15.39:0/533903845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea400b3990 msgr2=0x7fea4014eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:30.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.170+0000 7fea36a00640 1 --2- 172.21.15.39:0/533903845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea400b3990 0x7fea4014eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.170+0000 7fea36a00640 1 -- 172.21.15.39:0/533903845 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea400aebf0 con 0x7fea400b3da0 2024-09-21T07:50:30.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.170+0000 7fea36a00640 1 --2- 172.21.15.39:0/533903845 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea400b3da0 0x7fea400b0dd0 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7fea28009980 tx=0x7fea28033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:30.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.170+0000 7fea37e00640 1 -- 172.21.15.39:0/533903845 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea28031b50 con 0x7fea400b3da0 2024-09-21T07:50:30.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.171+0000 7fea37e00640 1 -- 172.21.15.39:0/533903845 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea28031de0 con 0x7fea400b3da0 2024-09-21T07:50:30.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.171+0000 7fea46400640 1 -- 172.21.15.39:0/533903845 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea400aeed0 con 0x7fea400b3da0 2024-09-21T07:50:30.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.171+0000 7fea37e00640 1 -- 172.21.15.39:0/533903845 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea28033330 con 0x7fea400b3da0 2024-09-21T07:50:30.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.171+0000 7fea46400640 1 -- 172.21.15.39:0/533903845 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea400c6be0 con 0x7fea400b3da0 2024-09-21T07:50:30.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.173+0000 7fea37e00640 1 -- 172.21.15.39:0/533903845 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fea280334d0 con 0x7fea400b3da0 2024-09-21T07:50:30.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.175+0000 7fea37e00640 1 --2- 172.21.15.39:0/533903845 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fea1c078b90 0x7fea1c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.175+0000 7fea37e00640 1 -- 172.21.15.39:0/533903845 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(238..238 src has 1..238) ==== 7262+0+0 (secure 0 0 0) 0x7fea280bc5a0 con 0x7fea400b3da0 2024-09-21T07:50:30.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.175+0000 7fea3f400640 1 --2- 172.21.15.39:0/533903845 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fea1c078b90 0x7fea1c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.176+0000 7fea4d3ed740 1 Processor -- start 2024-09-21T07:50:30.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.176+0000 7fea4d3ed740 1 -- start start 2024-09-21T07:50:30.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.176+0000 7fea3f400640 1 --2- 172.21.15.39:0/533903845 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fea1c078b90 0x7fea1c07b0a0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fea30001120 tx=0x7fea30009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:30.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.177+0000 7fea4d3ed740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb634af270 0x55cb6387f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.177+0000 7fea4d3ed740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cb634863b0 0x55cb6387feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.177+0000 7fea4d3ed740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb6348e590 0x55cb6384ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.177+0000 7fea4d3ed740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55cb637d9430 con 0x55cb634af270 2024-09-21T07:50:30.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.177+0000 7fea4d3ed740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55cb637c95d0 con 0x55cb6348e590 2024-09-21T07:50:30.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.177+0000 7fea4d3ed740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55cb637b8a00 con 0x55cb634863b0 2024-09-21T07:50:30.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.177+0000 7fea3f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb634af270 0x55cb6387f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.177+0000 7fea3f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb634af270 0x55cb6387f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58534/0 (socket says 172.21.15.39:58534) 2024-09-21T07:50:30.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.177+0000 7fea3f400640 1 -- 172.21.15.39:0/278052247 learned_addr learned my addr 172.21.15.39:0/278052247 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:30.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.178+0000 7fea3fe00640 1 --2- 172.21.15.39:0/278052247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb6348e590 0x55cb6384ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.178+0000 7fea36a00640 1 --2- 172.21.15.39:0/278052247 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cb634863b0 0x55cb6387feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.178+0000 7fea3f400640 1 -- 172.21.15.39:0/278052247 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cb634863b0 msgr2=0x55cb6387feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:30.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.178+0000 7fea3f400640 1 --2- 172.21.15.39:0/278052247 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cb634863b0 0x55cb6387feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.178+0000 7fea3f400640 1 -- 172.21.15.39:0/278052247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb6348e590 msgr2=0x55cb6384ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:30.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.178+0000 7fea3f400640 1 --2- 172.21.15.39:0/278052247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb6348e590 0x55cb6384ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.178+0000 7fea3f400640 1 -- 172.21.15.39:0/278052247 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb63781520 con 0x55cb634af270 2024-09-21T07:50:30.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.178+0000 7fea3f400640 1 --2- 172.21.15.39:0/278052247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb634af270 0x55cb6387f970 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7fea3000f890 tx=0x7fea30012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=882d0131e4a145ae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:30.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.179+0000 7fea2fe00640 1 -- 172.21.15.39:0/278052247 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea30025070 con 0x55cb634af270 2024-09-21T07:50:30.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.179+0000 7fea2fe00640 1 -- 172.21.15.39:0/278052247 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea3001b050 con 0x55cb634af270 2024-09-21T07:50:30.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.179+0000 7fea2fe00640 1 -- 172.21.15.39:0/278052247 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea30014cc0 con 0x55cb634af270 2024-09-21T07:50:30.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.179+0000 7fea4d3ed740 1 -- 172.21.15.39:0/278052247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb634af270 msgr2=0x55cb6387f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:30.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.179+0000 7fea4d3ed740 1 --2- 172.21.15.39:0/278052247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb634af270 0x55cb6387f970 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7fea3000f890 tx=0x7fea30012000 comp rx=0 tx=0).stop 2024-09-21T07:50:30.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.179+0000 7fea4d3ed740 1 -- 172.21.15.39:0/278052247 shutdown_connections 2024-09-21T07:50:30.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.179+0000 7fea4d3ed740 1 --2- 172.21.15.39:0/278052247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb6348e590 0x55cb6384ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.179+0000 7fea4d3ed740 1 --2- 172.21.15.39:0/278052247 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cb634863b0 0x55cb6387feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.179+0000 7fea4d3ed740 1 --2- 172.21.15.39:0/278052247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb634af270 0x55cb6387f970 unknown :-1 s=CLOSED pgs=2335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.179+0000 7fea4d3ed740 1 -- 172.21.15.39:0/278052247 >> 172.21.15.39:0/278052247 conn(0x55cb634a33c0 msgr2=0x55cb63855970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:30.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.180+0000 7fea4d3ed740 1 -- 172.21.15.39:0/278052247 shutdown_connections 2024-09-21T07:50:30.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.180+0000 7fea4d3ed740 1 -- 172.21.15.39:0/278052247 wait complete. 2024-09-21T07:50:30.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.181+0000 7fea4d3ed740 1 -- start start 2024-09-21T07:50:30.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.181+0000 7fea4d3ed740 1 Processor -- start 2024-09-21T07:50:30.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.182+0000 7fea4d3ed740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cb634863b0 0x55cb638927f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.182+0000 7fea4d3ed740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb6348e590 0x55cb63879d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.182+0000 7fea4d3ed740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb634af270 0x55cb6387c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.182+0000 7fea4d3ed740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55cb6375f330 con 0x55cb634af270 2024-09-21T07:50:30.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.182+0000 7fea4d3ed740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55cb637bc810 con 0x55cb6348e590 2024-09-21T07:50:30.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.182+0000 7fea4d3ed740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55cb63529620 con 0x55cb634863b0 2024-09-21T07:50:30.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.183+0000 7fea4d3ed740 10 client.0 _ll_register_callbacks cb 0x55cb63764420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:50:30.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.183+0000 7fea3fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb634af270 0x55cb6387c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.183+0000 7fea3fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb634af270 0x55cb6387c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58548/0 (socket says 172.21.15.39:58548) 2024-09-21T07:50:30.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.183+0000 7fea3fe00640 1 -- 172.21.15.39:0/3991353435 learned_addr learned my addr 172.21.15.39:0/3991353435 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:30.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.183+0000 7fea36a00640 1 --2- 172.21.15.39:0/3991353435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb6348e590 0x55cb63879d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.183+0000 7fea3f400640 1 --2- 172.21.15.39:0/3991353435 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cb634863b0 0x55cb638927f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.183+0000 7fea3fe00640 1 -- 172.21.15.39:0/3991353435 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cb634863b0 msgr2=0x55cb638927f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:30.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.183+0000 7fea3fe00640 1 --2- 172.21.15.39:0/3991353435 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cb634863b0 0x55cb638927f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.183+0000 7fea3fe00640 1 -- 172.21.15.39:0/3991353435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb6348e590 msgr2=0x55cb63879d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:30.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.183+0000 7fea3fe00640 1 --2- 172.21.15.39:0/3991353435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb6348e590 0x55cb63879d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.183+0000 7fea3fe00640 1 -- 172.21.15.39:0/3991353435 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb6351a850 con 0x55cb634af270 2024-09-21T07:50:30.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.184+0000 7fea3fe00640 1 --2- 172.21.15.39:0/3991353435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb634af270 0x55cb6387c4f0 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7fea38012d50 tx=0x7fea38010610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f8b09c3446ab6e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:30.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.184+0000 7fea2cc00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:50:30.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.184+0000 7fea2cc00640 1 -- 172.21.15.39:0/3991353435 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3801a020 con 0x55cb634af270 2024-09-21T07:50:30.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.184+0000 7fea2cc00640 1 -- 172.21.15.39:0/3991353435 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(238..238 src has 1..238) ==== 7262+0+0 (secure 0 0 0) 0x7fea38010c20 con 0x55cb634af270 2024-09-21T07:50:30.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.184+0000 7fea4d3ed740 1 -- 172.21.15.39:0/3991353435 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea3001b050 con 0x55cb634af270 2024-09-21T07:50:30.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.184+0000 7fea2cc00640 1 -- 172.21.15.39:0/3991353435 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=257) ==== 24+0+0 (secure 0 0 0) 0x7fea38010e60 con 0x55cb634af270 2024-09-21T07:50:30.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.184+0000 7fea4d3ed740 10 client.11010 fetch_fsmap learned FSMap version 257 2024-09-21T07:50:30.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.184+0000 7fea4d3ed740 1 -- 172.21.15.39:0/3991353435 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=257,monmap=2+}) -- 0x55cb635916b0 con 0x55cb634af270 2024-09-21T07:50:30.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.184+0000 7fea2cc00640 1 -- 172.21.15.39:0/3991353435 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea3001b050 con 0x55cb634af270 2024-09-21T07:50:30.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.184+0000 7fea2cc00640 1 -- 172.21.15.39:0/3991353435 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 257) ==== 3550+0+0 (secure 0 0 0) 0x7fea38015440 con 0x55cb634af270 2024-09-21T07:50:30.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.185+0000 7fea4d3ed740 10 client.11010 fetch_fsmap finished waiting for FSMap version 257 2024-09-21T07:50:30.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.185+0000 7fea4d3ed740 10 client.11010 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:50:30.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.185+0000 7fea4d3ed740 20 client.11010 populate_metadata read hostname 'smithi039' 2024-09-21T07:50:30.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.185+0000 7fea4d3ed740 1 --2- 172.21.15.39:0/3991353435 >> [v2:172.21.15.39:6832/4143807732,v1:172.21.15.39:6834/4143807732] conn(0x55cb6346de40 0x55cb63865430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.185+0000 7fea4d3ed740 4 client.11010 mds_command: new command op to 10632 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:50:30.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.185+0000 7fea4d3ed740 1 -- 172.21.15.39:0/3991353435 --> [v2:172.21.15.39:6832/4143807732,v1:172.21.15.39:6834/4143807732] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb637cb520 con 0x55cb6346de40 2024-09-21T07:50:30.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.185+0000 7fea3f400640 1 --2- 172.21.15.39:0/3991353435 >> [v2:172.21.15.39:6832/4143807732,v1:172.21.15.39:6834/4143807732] conn(0x55cb6346de40 0x55cb63865430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.185+0000 7fea2cc00640 10 client.11010 ms_handle_connect on v2:172.21.15.39:6832/4143807732 2024-09-21T07:50:30.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.185+0000 7fea3f400640 1 --2- 172.21.15.39:0/3991353435 >> [v2:172.21.15.39:6832/4143807732,v1:172.21.15.39:6834/4143807732] conn(0x55cb6346de40 0x55cb63865430 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3542a3ebff25e026 server_cookie=b470afcf528cc6c2 in_seq=0 out_seq=0 2024-09-21T07:50:30.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.186+0000 7fea2cc00640 1 -- 172.21.15.39:0/3991353435 <== mds.1 v2:172.21.15.39:6832/4143807732 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cb637cb520 con 0x55cb6346de40 2024-09-21T07:50:30.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.186+0000 7fea2cc00640 10 client.11010 handle_command_reply: tid=0 2024-09-21T07:50:30.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.186+0000 7fea4d3ed740 1 client.11010 shutdown 2024-09-21T07:50:30.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.186+0000 7fea4d3ed740 1 -- 172.21.15.39:0/3991353435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb634af270 msgr2=0x55cb6387c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:30.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.186+0000 7fea4d3ed740 1 --2- 172.21.15.39:0/3991353435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb634af270 0x55cb6387c4f0 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7fea38012d50 tx=0x7fea38010610 comp rx=0 tx=0).stop 2024-09-21T07:50:30.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.187+0000 7fea4d3ed740 1 -- 172.21.15.39:0/3991353435 shutdown_connections 2024-09-21T07:50:30.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.187+0000 7fea4d3ed740 1 --2- 172.21.15.39:0/3991353435 >> [v2:172.21.15.39:6832/4143807732,v1:172.21.15.39:6834/4143807732] conn(0x55cb6346de40 0x55cb63865430 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.187+0000 7fea4d3ed740 1 --2- 172.21.15.39:0/3991353435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb634af270 0x55cb6387c4f0 unknown :-1 s=CLOSED pgs=2336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.187+0000 7fea4d3ed740 1 --2- 172.21.15.39:0/3991353435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb6348e590 0x55cb63879d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.187+0000 7fea4d3ed740 1 --2- 172.21.15.39:0/3991353435 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cb634863b0 0x55cb638927f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.187+0000 7fea2cc00640 0 client.11010 ms_handle_reset on v2:172.21.15.39:6832/4143807732 2024-09-21T07:50:30.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.187+0000 7fea4d3ed740 1 -- 172.21.15.39:0/3991353435 >> 172.21.15.39:0/3991353435 conn(0x55cb634a33c0 msgr2=0x55cb63891010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:30.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.187+0000 7fea4d3ed740 1 -- 172.21.15.39:0/3991353435 shutdown_connections 2024-09-21T07:50:30.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.187+0000 7fea4d3ed740 1 -- 172.21.15.39:0/3991353435 wait complete. 2024-09-21T07:50:30.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.187+0000 7fea4d3ed740 20 client.11010 trim_cache size 0 max 16384 2024-09-21T07:50:30.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.206+0000 7fea4d3ed740 1 Processor -- start 2024-09-21T07:50:30.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.206+0000 7fea4d3ed740 1 -- start start 2024-09-21T07:50:30.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.207+0000 7fea4d3ed740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cb634863b0 0x55cb6387afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.208+0000 7fea4d3ed740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb6348e590 0x55cb638bde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.208+0000 7fea4d3ed740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb634af270 0x55cb638be3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.208+0000 7fea4d3ed740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fea402043a0 con 0x55cb634af270 2024-09-21T07:50:30.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.208+0000 7fea4d3ed740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fea40204a70 con 0x55cb6348e590 2024-09-21T07:50:30.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.208+0000 7fea4d3ed740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fea40161650 con 0x55cb634863b0 2024-09-21T07:50:30.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.208+0000 7fea3fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb634af270 0x55cb638be3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.208+0000 7fea3fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb634af270 0x55cb638be3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58562/0 (socket says 172.21.15.39:58562) 2024-09-21T07:50:30.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.208+0000 7fea3fe00640 1 -- 172.21.15.39:0/4108883069 learned_addr learned my addr 172.21.15.39:0/4108883069 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:30.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.208+0000 7fea3fe00640 1 -- 172.21.15.39:0/4108883069 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cb634863b0 msgr2=0x55cb6387afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:30.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.208+0000 7fea3f400640 1 --2- 172.21.15.39:0/4108883069 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cb634863b0 0x55cb6387afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.208+0000 7fea36a00640 1 --2- 172.21.15.39:0/4108883069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb6348e590 0x55cb638bde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.208+0000 7fea3fe00640 1 --2- 172.21.15.39:0/4108883069 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cb634863b0 0x55cb6387afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.208+0000 7fea3fe00640 1 -- 172.21.15.39:0/4108883069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb6348e590 msgr2=0x55cb638bde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:30.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.208+0000 7fea3fe00640 1 --2- 172.21.15.39:0/4108883069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb6348e590 0x55cb638bde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.208+0000 7fea3fe00640 1 -- 172.21.15.39:0/4108883069 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb63893790 con 0x55cb634af270 2024-09-21T07:50:30.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.208+0000 7fea3f400640 1 --2- 172.21.15.39:0/4108883069 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cb634863b0 0x55cb6387afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:50:30.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.208+0000 7fea36a00640 1 --2- 172.21.15.39:0/4108883069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb6348e590 0x55cb638bde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:50:30.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.209+0000 7fea3fe00640 1 --2- 172.21.15.39:0/4108883069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb634af270 0x55cb638be3c0 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7fea380126e0 tx=0x7fea380101d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9377e108f8d10a69 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:30.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.209+0000 7fea2f400640 1 -- 172.21.15.39:0/4108883069 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3801a070 con 0x55cb634af270 2024-09-21T07:50:30.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.209+0000 7fea2f400640 1 -- 172.21.15.39:0/4108883069 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea3800b070 con 0x55cb634af270 2024-09-21T07:50:30.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.209+0000 7fea2f400640 1 -- 172.21.15.39:0/4108883069 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38015990 con 0x55cb634af270 2024-09-21T07:50:30.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.210+0000 7fea4d3ed740 1 -- 172.21.15.39:0/4108883069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb634af270 msgr2=0x55cb638be3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:30.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.210+0000 7fea4d3ed740 1 --2- 172.21.15.39:0/4108883069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb634af270 0x55cb638be3c0 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7fea380126e0 tx=0x7fea380101d0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.210+0000 7fea4d3ed740 1 -- 172.21.15.39:0/4108883069 shutdown_connections 2024-09-21T07:50:30.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.210+0000 7fea4d3ed740 1 --2- 172.21.15.39:0/4108883069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb634af270 0x55cb638be3c0 unknown :-1 s=CLOSED pgs=2337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.210+0000 7fea4d3ed740 1 --2- 172.21.15.39:0/4108883069 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb6348e590 0x55cb638bde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.210+0000 7fea4d3ed740 1 --2- 172.21.15.39:0/4108883069 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cb634863b0 0x55cb6387afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.210+0000 7fea4d3ed740 1 -- 172.21.15.39:0/4108883069 >> 172.21.15.39:0/4108883069 conn(0x55cb634a33c0 msgr2=0x55cb63850f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:30.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.210+0000 7fea4d3ed740 1 -- 172.21.15.39:0/4108883069 shutdown_connections 2024-09-21T07:50:30.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.210+0000 7fea4d3ed740 1 -- 172.21.15.39:0/4108883069 wait complete. 2024-09-21T07:50:30.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.211+0000 7fea4d3ed740 1 -- start start 2024-09-21T07:50:30.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.211+0000 7fea4d3ed740 1 Processor -- start 2024-09-21T07:50:30.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.212+0000 7fea4d3ed740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb634863b0 0x55cb63914970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.212+0000 7fea4d3ed740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cb6348e590 0x55cb63865400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.212+0000 7fea4d3ed740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb634af270 0x55cb63865940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.212+0000 7fea4d3ed740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55cb63893d50 con 0x55cb634863b0 2024-09-21T07:50:30.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.212+0000 7fea4d3ed740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55cb6387a2b0 con 0x55cb634af270 2024-09-21T07:50:30.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.212+0000 7fea4d3ed740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55cb638677c0 con 0x55cb6348e590 2024-09-21T07:50:30.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.212+0000 7fea4d3ed740 10 client.0 _ll_register_callbacks cb 0x7fea380129b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:50:30.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.213+0000 7fea3f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb634863b0 0x55cb63914970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.213+0000 7fea3f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb634863b0 0x55cb63914970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58570/0 (socket says 172.21.15.39:58570) 2024-09-21T07:50:30.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.213+0000 7fea3f400640 1 -- 172.21.15.39:0/2557740342 learned_addr learned my addr 172.21.15.39:0/2557740342 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:30.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.213+0000 7fea3fe00640 1 --2- 172.21.15.39:0/2557740342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb634af270 0x55cb63865940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.213+0000 7fea36a00640 1 --2- 172.21.15.39:0/2557740342 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cb6348e590 0x55cb63865400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.213+0000 7fea3f400640 1 -- 172.21.15.39:0/2557740342 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cb6348e590 msgr2=0x55cb63865400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:30.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.213+0000 7fea3f400640 1 --2- 172.21.15.39:0/2557740342 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cb6348e590 0x55cb63865400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.213+0000 7fea3f400640 1 -- 172.21.15.39:0/2557740342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb634af270 msgr2=0x55cb63865940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:30.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.213+0000 7fea3f400640 1 --2- 172.21.15.39:0/2557740342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb634af270 0x55cb63865940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.213+0000 7fea3f400640 1 -- 172.21.15.39:0/2557740342 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb63892ac0 con 0x55cb634863b0 2024-09-21T07:50:30.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.213+0000 7fea36a00640 1 --2- 172.21.15.39:0/2557740342 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cb6348e590 0x55cb63865400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:50:30.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.213+0000 7fea3fe00640 1 --2- 172.21.15.39:0/2557740342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb634af270 0x55cb63865940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:50:30.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.213+0000 7fea3f400640 1 --2- 172.21.15.39:0/2557740342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb634863b0 0x55cb63914970 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7fea3000f860 tx=0x7fea30000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aadf51272b1726bc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:30.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.213+0000 7fea2cc00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:50:30.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.213+0000 7fea2cc00640 1 -- 172.21.15.39:0/2557740342 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3002f070 con 0x55cb634863b0 2024-09-21T07:50:30.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.214+0000 7fea2cc00640 1 -- 172.21.15.39:0/2557740342 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(238..238 src has 1..238) ==== 7262+0+0 (secure 0 0 0) 0x7fea3002b650 con 0x55cb634863b0 2024-09-21T07:50:30.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.214+0000 7fea4d3ed740 1 -- 172.21.15.39:0/2557740342 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea3800b070 con 0x55cb634863b0 2024-09-21T07:50:30.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.214+0000 7fea2cc00640 1 -- 172.21.15.39:0/2557740342 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=257) ==== 24+0+0 (secure 0 0 0) 0x7fea30025040 con 0x55cb634863b0 2024-09-21T07:50:30.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.214+0000 7fea4d3ed740 10 client.11016 fetch_fsmap learned FSMap version 257 2024-09-21T07:50:30.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.214+0000 7fea4d3ed740 1 -- 172.21.15.39:0/2557740342 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=257,monmap=2+}) -- 0x55cb6387d510 con 0x55cb634863b0 2024-09-21T07:50:30.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.214+0000 7fea2cc00640 1 -- 172.21.15.39:0/2557740342 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea3800b070 con 0x55cb634863b0 2024-09-21T07:50:30.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.214+0000 7fea2cc00640 1 -- 172.21.15.39:0/2557740342 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 257) ==== 3550+0+0 (secure 0 0 0) 0x7fea3002b870 con 0x55cb634863b0 2024-09-21T07:50:30.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.214+0000 7fea4d3ed740 10 client.11016 fetch_fsmap finished waiting for FSMap version 257 2024-09-21T07:50:30.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.214+0000 7fea4d3ed740 10 client.11016 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:50:30.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.214+0000 7fea4d3ed740 20 client.11016 populate_metadata read hostname 'smithi039' 2024-09-21T07:50:30.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.214+0000 7fea4d3ed740 1 --2- 172.21.15.39:0/2557740342 >> [v2:172.21.15.39:6832/4143807732,v1:172.21.15.39:6834/4143807732] conn(0x55cb6391b380 0x55cb6391dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.214+0000 7fea4d3ed740 4 client.11016 mds_command: new command op to 10632 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:50:30.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.214+0000 7fea4d3ed740 1 -- 172.21.15.39:0/2557740342 --> [v2:172.21.15.39:6832/4143807732,v1:172.21.15.39:6834/4143807732] -- command(tid 0: {"prefix": "status"}) -- 0x55cb637c6fd0 con 0x55cb6391b380 2024-09-21T07:50:30.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.215+0000 7fea36a00640 1 --2- 172.21.15.39:0/2557740342 >> [v2:172.21.15.39:6832/4143807732,v1:172.21.15.39:6834/4143807732] conn(0x55cb6391b380 0x55cb6391dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.215+0000 7fea36a00640 1 --2- 172.21.15.39:0/2557740342 >> [v2:172.21.15.39:6832/4143807732,v1:172.21.15.39:6834/4143807732] conn(0x55cb6391b380 0x55cb6391dfa0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=827bfb65a97ded3a server_cookie=f22f14e49950c21b in_seq=0 out_seq=0 2024-09-21T07:50:30.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.215+0000 7fea2cc00640 10 client.11016 ms_handle_connect on v2:172.21.15.39:6832/4143807732 2024-09-21T07:50:30.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.215+0000 7fea2cc00640 1 -- 172.21.15.39:0/2557740342 <== mds.1 v2:172.21.15.39:6832/4143807732 1 ==== command_reply(tid 0: 0 ) ==== 8+0+335 (crc 0 0 0) 0x55cb637c6fd0 con 0x55cb6391b380 2024-09-21T07:50:30.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.215+0000 7fea2cc00640 10 client.11016 handle_command_reply: tid=0 2024-09-21T07:50:30.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.215+0000 7fea4d3ed740 1 client.11016 shutdown 2024-09-21T07:50:30.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.215+0000 7fea4d3ed740 1 -- 172.21.15.39:0/2557740342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb634863b0 msgr2=0x55cb63914970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:30.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.215+0000 7fea4d3ed740 1 --2- 172.21.15.39:0/2557740342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb634863b0 0x55cb63914970 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7fea3000f860 tx=0x7fea30000ed0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.216+0000 7fea4d3ed740 1 -- 172.21.15.39:0/2557740342 shutdown_connections 2024-09-21T07:50:30.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.216+0000 7fea4d3ed740 1 --2- 172.21.15.39:0/2557740342 >> [v2:172.21.15.39:6832/4143807732,v1:172.21.15.39:6834/4143807732] conn(0x55cb6391b380 0x55cb6391dfa0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.216+0000 7fea4d3ed740 1 --2- 172.21.15.39:0/2557740342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cb634af270 0x55cb63865940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.216+0000 7fea4d3ed740 1 --2- 172.21.15.39:0/2557740342 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55cb6348e590 0x55cb63865400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.216+0000 7fea2cc00640 0 client.11016 ms_handle_reset on v2:172.21.15.39:6832/4143807732 2024-09-21T07:50:30.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.216+0000 7fea4d3ed740 1 --2- 172.21.15.39:0/2557740342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cb634863b0 0x55cb63914970 unknown :-1 s=CLOSED pgs=2338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.216+0000 7fea4d3ed740 1 -- 172.21.15.39:0/2557740342 >> 172.21.15.39:0/2557740342 conn(0x55cb634a33c0 msgr2=0x55cb6387afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:30.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.216+0000 7fea4d3ed740 1 -- 172.21.15.39:0/2557740342 shutdown_connections 2024-09-21T07:50:30.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.216+0000 7fea4d3ed740 1 -- 172.21.15.39:0/2557740342 wait complete. 2024-09-21T07:50:30.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.216+0000 7fea4d3ed740 20 client.11016 trim_cache size 0 max 16384 2024-09-21T07:50:30.221 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:50:30.221 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:50:30.221 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 1, 2024-09-21T07:50:30.221 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 10632, 2024-09-21T07:50:30.221 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:50:30.221 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:50:30.221 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:50:30.221 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 43.994607602999999, 2024-09-21T07:50:30.221 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 257, 2024-09-21T07:50:30.221 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 238, 2024-09-21T07:50:30.221 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 238, 2024-09-21T07:50:30.221 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 60.106589321000001 2024-09-21T07:50:30.221 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:50:30.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.216+0000 7fea46400640 1 -- 172.21.15.39:0/533903845 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fea1c078b90 msgr2=0x7fea1c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:30.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.216+0000 7fea46400640 1 --2- 172.21.15.39:0/533903845 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fea1c078b90 0x7fea1c07b0a0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fea30001120 tx=0x7fea30009040 comp rx=0 tx=0).stop 2024-09-21T07:50:30.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.216+0000 7fea46400640 1 -- 172.21.15.39:0/533903845 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea400b3da0 msgr2=0x7fea400b0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:30.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.216+0000 7fea46400640 1 --2- 172.21.15.39:0/533903845 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea400b3da0 0x7fea400b0dd0 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7fea28009980 tx=0x7fea28033ba0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.217+0000 7fea46400640 1 -- 172.21.15.39:0/533903845 shutdown_connections 2024-09-21T07:50:30.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.217+0000 7fea46400640 1 --2- 172.21.15.39:0/533903845 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fea1c078b90 0x7fea1c07b0a0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.217+0000 7fea46400640 1 --2- 172.21.15.39:0/533903845 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea40162f10 0x7fea400b1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.217+0000 7fea46400640 1 --2- 172.21.15.39:0/533903845 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea400b3da0 0x7fea400b0dd0 unknown :-1 s=CLOSED pgs=2334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.217+0000 7fea46400640 1 --2- 172.21.15.39:0/533903845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea400b3990 0x7fea4014eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.217+0000 7fea46400640 1 -- 172.21.15.39:0/533903845 >> 172.21.15.39:0/533903845 conn(0x7fea400b2460 msgr2=0x7fea4006ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:30.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.217+0000 7fea46400640 1 -- 172.21.15.39:0/533903845 shutdown_connections 2024-09-21T07:50:30.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.217+0000 7fea46400640 1 -- 172.21.15.39:0/533903845 wait complete. 2024-09-21T07:50:30.240 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T07:50:30.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.465+0000 7fa30ec00640 1 -- 172.21.15.39:0/2328060172 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3080b3990 msgr2=0x7fa3080b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:30.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.465+0000 7fa30ec00640 1 --2- 172.21.15.39:0/2328060172 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3080b3990 0x7fa3080b4450 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7fa30815f4e0 tx=0x7fa2f002a540 comp rx=0 tx=0).stop 2024-09-21T07:50:30.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.465+0000 7fa30ec00640 1 -- 172.21.15.39:0/2328060172 shutdown_connections 2024-09-21T07:50:30.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.465+0000 7fa30ec00640 1 --2- 172.21.15.39:0/2328060172 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa3080b3da0 0x7fa3080b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.465+0000 7fa30ec00640 1 --2- 172.21.15.39:0/2328060172 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3080b3990 0x7fa3080b4450 unknown :-1 s=CLOSED pgs=2339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.465+0000 7fa30ec00640 1 --2- 172.21.15.39:0/2328060172 >> v2:172.21.15.148:3300/0 conn(0x7fa308162f10 0x7fa3080b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.465+0000 7fa30ec00640 1 -- 172.21.15.39:0/2328060172 >> 172.21.15.39:0/2328060172 conn(0x7fa3080b2460 msgr2=0x7fa30806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:30.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.465+0000 7fa30ec00640 1 -- 172.21.15.39:0/2328060172 shutdown_connections 2024-09-21T07:50:30.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.465+0000 7fa30ec00640 1 -- 172.21.15.39:0/2328060172 wait complete. 2024-09-21T07:50:30.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.467+0000 7fa30ec00640 1 Processor -- start 2024-09-21T07:50:30.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.467+0000 7fa30ec00640 1 -- start start 2024-09-21T07:50:30.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.469+0000 7fa30ec00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa3080b3990 0x7fa3081e8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.469+0000 7fa30ec00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3080b3da0 0x7fa3080caa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.469+0000 7fa30ec00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa308162f10 0x7fa3080736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.469+0000 7fa30ec00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa3080af200 con 0x7fa308162f10 2024-09-21T07:50:30.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.469+0000 7fa30ec00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa3080af080 con 0x7fa3080b3da0 2024-09-21T07:50:30.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.469+0000 7fa30ec00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa3080af380 con 0x7fa3080b3990 2024-09-21T07:50:30.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.469+0000 7fa30cc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa308162f10 0x7fa3080736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.469+0000 7fa2ffe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3080b3da0 0x7fa3080caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.469+0000 7fa30cc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa308162f10 0x7fa3080736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58584/0 (socket says 172.21.15.39:58584) 2024-09-21T07:50:30.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.469+0000 7fa30cc00640 1 -- 172.21.15.39:0/1366591702 learned_addr learned my addr 172.21.15.39:0/1366591702 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:30.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.469+0000 7fa307e00640 1 --2- 172.21.15.39:0/1366591702 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa3080b3990 0x7fa3081e8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.470+0000 7fa30cc00640 1 -- 172.21.15.39:0/1366591702 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa3080b3990 msgr2=0x7fa3081e8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:30.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.470+0000 7fa30cc00640 1 --2- 172.21.15.39:0/1366591702 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa3080b3990 0x7fa3081e8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.470+0000 7fa30cc00640 1 -- 172.21.15.39:0/1366591702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3080b3da0 msgr2=0x7fa3080caa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:30.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.470+0000 7fa30cc00640 1 --2- 172.21.15.39:0/1366591702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3080b3da0 0x7fa3080caa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.470+0000 7fa30cc00640 1 -- 172.21.15.39:0/1366591702 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa30814b2f0 con 0x7fa308162f10 2024-09-21T07:50:30.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.470+0000 7fa30cc00640 1 --2- 172.21.15.39:0/1366591702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa308162f10 0x7fa3080736e0 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7fa300005f40 tx=0x7fa30000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:30.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.470+0000 7fa304c00640 1 -- 172.21.15.39:0/1366591702 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30000cd80 con 0x7fa308162f10 2024-09-21T07:50:30.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.470+0000 7fa304c00640 1 -- 172.21.15.39:0/1366591702 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa300004b20 con 0x7fa308162f10 2024-09-21T07:50:30.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.470+0000 7fa304c00640 1 -- 172.21.15.39:0/1366591702 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa300005020 con 0x7fa308162f10 2024-09-21T07:50:30.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.470+0000 7fa30ec00640 1 -- 172.21.15.39:0/1366591702 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa30814b5d0 con 0x7fa308162f10 2024-09-21T07:50:30.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.470+0000 7fa30ec00640 1 -- 172.21.15.39:0/1366591702 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa30814bb90 con 0x7fa308162f10 2024-09-21T07:50:30.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.473+0000 7fa304c00640 1 -- 172.21.15.39:0/1366591702 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa3000051c0 con 0x7fa308162f10 2024-09-21T07:50:30.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.474+0000 7fa304c00640 1 --2- 172.21.15.39:0/1366591702 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa2e4078b90 0x7fa2e407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.474+0000 7fa304c00640 1 -- 172.21.15.39:0/1366591702 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(238..238 src has 1..238) ==== 7262+0+0 (secure 0 0 0) 0x7fa30009bea0 con 0x7fa308162f10 2024-09-21T07:50:30.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.474+0000 7fa315dbd740 1 Processor -- start 2024-09-21T07:50:30.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.474+0000 7fa307e00640 1 --2- 172.21.15.39:0/1366591702 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa2e4078b90 0x7fa2e407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.474+0000 7fa315dbd740 1 -- start start 2024-09-21T07:50:30.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.475+0000 7fa307e00640 1 --2- 172.21.15.39:0/1366591702 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa2e4078b90 0x7fa2e407b0a0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fa308075150 tx=0x7fa2f8005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:30.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.476+0000 7fa315dbd740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55675650b270 0x5567568db970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.476+0000 7fa315dbd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567564e23b0 0x5567568dbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.476+0000 7fa315dbd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567564ea590 0x5567568abf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.476+0000 7fa315dbd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556756835430 con 0x5567564ea590 2024-09-21T07:50:30.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.476+0000 7fa315dbd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5567568255d0 con 0x5567564e23b0 2024-09-21T07:50:30.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.476+0000 7fa315dbd740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x556756814a00 con 0x55675650b270 2024-09-21T07:50:30.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.476+0000 7fa30cc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567564ea590 0x5567568abf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.476+0000 7fa30cc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567564ea590 0x5567568abf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58594/0 (socket says 172.21.15.39:58594) 2024-09-21T07:50:30.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.476+0000 7fa30cc00640 1 -- 172.21.15.39:0/3583463441 learned_addr learned my addr 172.21.15.39:0/3583463441 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:30.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.476+0000 7fa307e00640 1 --2- 172.21.15.39:0/3583463441 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55675650b270 0x5567568db970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.476+0000 7fa2ffe00640 1 --2- 172.21.15.39:0/3583463441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567564e23b0 0x5567568dbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.477+0000 7fa30cc00640 1 -- 172.21.15.39:0/3583463441 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55675650b270 msgr2=0x5567568db970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:30.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.477+0000 7fa30cc00640 1 --2- 172.21.15.39:0/3583463441 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55675650b270 0x5567568db970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.477+0000 7fa30cc00640 1 -- 172.21.15.39:0/3583463441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567564e23b0 msgr2=0x5567568dbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:30.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.477+0000 7fa30cc00640 1 --2- 172.21.15.39:0/3583463441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567564e23b0 0x5567568dbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.477+0000 7fa30cc00640 1 -- 172.21.15.39:0/3583463441 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567567dd520 con 0x5567564ea590 2024-09-21T07:50:30.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.477+0000 7fa2ffe00640 1 --2- 172.21.15.39:0/3583463441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567564e23b0 0x5567568dbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:50:30.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.477+0000 7fa30cc00640 1 --2- 172.21.15.39:0/3583463441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567564ea590 0x5567568abf60 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7fa3000632e0 tx=0x7fa300062ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4329e3ead9be2cc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:30.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.477+0000 7fa2f7e00640 1 -- 172.21.15.39:0/3583463441 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3000635e0 con 0x5567564ea590 2024-09-21T07:50:30.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.477+0000 7fa2f7e00640 1 -- 172.21.15.39:0/3583463441 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa300089070 con 0x5567564ea590 2024-09-21T07:50:30.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.477+0000 7fa2f7e00640 1 -- 172.21.15.39:0/3583463441 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30006ccc0 con 0x5567564ea590 2024-09-21T07:50:30.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.478+0000 7fa315dbd740 1 -- 172.21.15.39:0/3583463441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567564ea590 msgr2=0x5567568abf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:30.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.478+0000 7fa315dbd740 1 --2- 172.21.15.39:0/3583463441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567564ea590 0x5567568abf60 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7fa3000632e0 tx=0x7fa300062ff0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.478+0000 7fa315dbd740 1 -- 172.21.15.39:0/3583463441 shutdown_connections 2024-09-21T07:50:30.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.478+0000 7fa315dbd740 1 --2- 172.21.15.39:0/3583463441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567564ea590 0x5567568abf60 unknown :-1 s=CLOSED pgs=2341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.478+0000 7fa315dbd740 1 --2- 172.21.15.39:0/3583463441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567564e23b0 0x5567568dbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.478+0000 7fa315dbd740 1 --2- 172.21.15.39:0/3583463441 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55675650b270 0x5567568db970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.478+0000 7fa315dbd740 1 -- 172.21.15.39:0/3583463441 >> 172.21.15.39:0/3583463441 conn(0x5567564ff3c0 msgr2=0x5567568b1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:30.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.478+0000 7fa315dbd740 1 -- 172.21.15.39:0/3583463441 shutdown_connections 2024-09-21T07:50:30.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.478+0000 7fa315dbd740 1 -- 172.21.15.39:0/3583463441 wait complete. 2024-09-21T07:50:30.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.479+0000 7fa315dbd740 1 -- start start 2024-09-21T07:50:30.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.479+0000 7fa315dbd740 1 Processor -- start 2024-09-21T07:50:30.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.481+0000 7fa315dbd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567564e23b0 0x5567569061f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.481+0000 7fa315dbd740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5567564ea590 0x5567568d6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.481+0000 7fa315dbd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55675650b270 0x5567568f0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.481+0000 7fa315dbd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5567567bb330 con 0x55675650b270 2024-09-21T07:50:30.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.481+0000 7fa315dbd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556756818810 con 0x5567564e23b0 2024-09-21T07:50:30.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.481+0000 7fa315dbd740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x556756585620 con 0x5567564ea590 2024-09-21T07:50:30.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.481+0000 7fa315dbd740 10 client.0 _ll_register_callbacks cb 0x5567567c0420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:50:30.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.481+0000 7fa30cc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55675650b270 0x5567568f0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.481+0000 7fa30cc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55675650b270 0x5567568f0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58598/0 (socket says 172.21.15.39:58598) 2024-09-21T07:50:30.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.481+0000 7fa30cc00640 1 -- 172.21.15.39:0/2361598920 learned_addr learned my addr 172.21.15.39:0/2361598920 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:30.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.481+0000 7fa307e00640 1 --2- 172.21.15.39:0/2361598920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567564e23b0 0x5567569061f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.481+0000 7fa2ffe00640 1 --2- 172.21.15.39:0/2361598920 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5567564ea590 0x5567568d6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.482+0000 7fa30cc00640 1 -- 172.21.15.39:0/2361598920 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5567564ea590 msgr2=0x5567568d6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:30.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.482+0000 7fa30cc00640 1 --2- 172.21.15.39:0/2361598920 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5567564ea590 0x5567568d6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.482+0000 7fa30cc00640 1 -- 172.21.15.39:0/2361598920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567564e23b0 msgr2=0x5567569061f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:30.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.482+0000 7fa30cc00640 1 --2- 172.21.15.39:0/2361598920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567564e23b0 0x5567569061f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.482+0000 7fa30cc00640 1 -- 172.21.15.39:0/2361598920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556756576850 con 0x55675650b270 2024-09-21T07:50:30.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.482+0000 7fa30cc00640 1 --2- 172.21.15.39:0/2361598920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55675650b270 0x5567568f0f30 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7fa300063570 tx=0x7fa300062af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a911385daed4de23 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:30.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.482+0000 7fa2f4c00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:50:30.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.482+0000 7fa2f4c00640 1 -- 172.21.15.39:0/2361598920 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa300068010 con 0x55675650b270 2024-09-21T07:50:30.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.482+0000 7fa2f4c00640 1 -- 172.21.15.39:0/2361598920 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(238..238 src has 1..238) ==== 7262+0+0 (secure 0 0 0) 0x7fa3000056c0 con 0x55675650b270 2024-09-21T07:50:30.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.482+0000 7fa315dbd740 1 -- 172.21.15.39:0/2361598920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa300089070 con 0x55675650b270 2024-09-21T07:50:30.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.482+0000 7fa2f4c00640 1 -- 172.21.15.39:0/2361598920 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=257) ==== 24+0+0 (secure 0 0 0) 0x7fa30006cf50 con 0x55675650b270 2024-09-21T07:50:30.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.482+0000 7fa315dbd740 10 client.11028 fetch_fsmap learned FSMap version 257 2024-09-21T07:50:30.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.482+0000 7fa315dbd740 1 -- 172.21.15.39:0/2361598920 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=257,monmap=2+}) -- 0x5567565ed6b0 con 0x55675650b270 2024-09-21T07:50:30.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.483+0000 7fa2f4c00640 1 -- 172.21.15.39:0/2361598920 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa300089070 con 0x55675650b270 2024-09-21T07:50:30.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.483+0000 7fa2f4c00640 1 -- 172.21.15.39:0/2361598920 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 257) ==== 3550+0+0 (secure 0 0 0) 0x7fa30005ef30 con 0x55675650b270 2024-09-21T07:50:30.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.483+0000 7fa315dbd740 10 client.11028 fetch_fsmap finished waiting for FSMap version 257 2024-09-21T07:50:30.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.483+0000 7fa315dbd740 10 client.11028 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:50:30.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.483+0000 7fa315dbd740 20 client.11028 populate_metadata read hostname 'smithi039' 2024-09-21T07:50:30.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.483+0000 7fa315dbd740 1 --2- 172.21.15.39:0/2361598920 >> [v2:172.21.15.39:6833/2824493631,v1:172.21.15.39:6835/2824493631] conn(0x5567564c9e40 0x5567568d72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.483+0000 7fa315dbd740 4 client.11028 mds_command: new command op to 10635 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:50:30.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.483+0000 7fa315dbd740 1 -- 172.21.15.39:0/2361598920 --> [v2:172.21.15.39:6833/2824493631,v1:172.21.15.39:6835/2824493631] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556756827520 con 0x5567564c9e40 2024-09-21T07:50:30.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.483+0000 7fa307e00640 1 --2- 172.21.15.39:0/2361598920 >> [v2:172.21.15.39:6833/2824493631,v1:172.21.15.39:6835/2824493631] conn(0x5567564c9e40 0x5567568d72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.484+0000 7fa307e00640 1 --2- 172.21.15.39:0/2361598920 >> [v2:172.21.15.39:6833/2824493631,v1:172.21.15.39:6835/2824493631] conn(0x5567564c9e40 0x5567568d72c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d6078469b6fdbc8 server_cookie=2b1041fdc4787b3f in_seq=0 out_seq=0 2024-09-21T07:50:30.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.484+0000 7fa2f4c00640 10 client.11028 ms_handle_connect on v2:172.21.15.39:6833/2824493631 2024-09-21T07:50:30.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.484+0000 7fa2f4c00640 1 -- 172.21.15.39:0/2361598920 <== mds.0 v2:172.21.15.39:6833/2824493631 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556756827520 con 0x5567564c9e40 2024-09-21T07:50:30.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.484+0000 7fa2f4c00640 10 client.11028 handle_command_reply: tid=0 2024-09-21T07:50:30.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.485+0000 7fa315dbd740 1 client.11028 shutdown 2024-09-21T07:50:30.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.485+0000 7fa315dbd740 1 -- 172.21.15.39:0/2361598920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55675650b270 msgr2=0x5567568f0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:30.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.485+0000 7fa315dbd740 1 --2- 172.21.15.39:0/2361598920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55675650b270 0x5567568f0f30 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7fa300063570 tx=0x7fa300062af0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.485+0000 7fa315dbd740 1 -- 172.21.15.39:0/2361598920 shutdown_connections 2024-09-21T07:50:30.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.485+0000 7fa315dbd740 1 --2- 172.21.15.39:0/2361598920 >> [v2:172.21.15.39:6833/2824493631,v1:172.21.15.39:6835/2824493631] conn(0x5567564c9e40 0x5567568d72c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.485+0000 7fa315dbd740 1 --2- 172.21.15.39:0/2361598920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55675650b270 0x5567568f0f30 unknown :-1 s=CLOSED pgs=2342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.485+0000 7fa2f4c00640 0 client.11028 ms_handle_reset on v2:172.21.15.39:6833/2824493631 2024-09-21T07:50:30.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.485+0000 7fa315dbd740 1 --2- 172.21.15.39:0/2361598920 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5567564ea590 0x5567568d6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.485+0000 7fa315dbd740 1 --2- 172.21.15.39:0/2361598920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567564e23b0 0x5567569061f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.485+0000 7fa315dbd740 1 -- 172.21.15.39:0/2361598920 >> 172.21.15.39:0/2361598920 conn(0x5567564ff3c0 msgr2=0x5567568f77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:30.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.485+0000 7fa315dbd740 1 -- 172.21.15.39:0/2361598920 shutdown_connections 2024-09-21T07:50:30.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.485+0000 7fa315dbd740 1 -- 172.21.15.39:0/2361598920 wait complete. 2024-09-21T07:50:30.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.485+0000 7fa315dbd740 20 client.11028 trim_cache size 0 max 16384 2024-09-21T07:50:30.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.504+0000 7fa315dbd740 1 Processor -- start 2024-09-21T07:50:30.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.505+0000 7fa315dbd740 1 -- start start 2024-09-21T07:50:30.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.506+0000 7fa315dbd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567564e23b0 0x5567568b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.506+0000 7fa315dbd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567564ea590 0x5567568f6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.506+0000 7fa315dbd740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55675650b270 0x5567568f65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.506+0000 7fa315dbd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa308204360 con 0x5567564e23b0 2024-09-21T07:50:30.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.506+0000 7fa315dbd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa308204a30 con 0x5567564ea590 2024-09-21T07:50:30.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.506+0000 7fa315dbd740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa30814dc30 con 0x55675650b270 2024-09-21T07:50:30.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.506+0000 7fa307e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567564e23b0 0x5567568b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.506+0000 7fa307e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567564e23b0 0x5567568b19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58602/0 (socket says 172.21.15.39:58602) 2024-09-21T07:50:30.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.506+0000 7fa307e00640 1 -- 172.21.15.39:0/1138990875 learned_addr learned my addr 172.21.15.39:0/1138990875 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:30.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.506+0000 7fa2ffe00640 1 --2- 172.21.15.39:0/1138990875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567564ea590 0x5567568f6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.506+0000 7fa30cc00640 1 --2- 172.21.15.39:0/1138990875 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55675650b270 0x5567568f65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.507+0000 7fa307e00640 1 -- 172.21.15.39:0/1138990875 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55675650b270 msgr2=0x5567568f65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:30.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.507+0000 7fa307e00640 1 --2- 172.21.15.39:0/1138990875 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55675650b270 0x5567568f65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.507+0000 7fa307e00640 1 -- 172.21.15.39:0/1138990875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567564ea590 msgr2=0x5567568f6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:30.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.507+0000 7fa307e00640 1 --2- 172.21.15.39:0/1138990875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567564ea590 0x5567568f6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.507+0000 7fa307e00640 1 -- 172.21.15.39:0/1138990875 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567568f2ce0 con 0x5567564e23b0 2024-09-21T07:50:30.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.507+0000 7fa307e00640 1 --2- 172.21.15.39:0/1138990875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567564e23b0 0x5567568b19e0 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x5567568d12f0 tx=0x7fa2f8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c04839ea8aa306b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:30.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.507+0000 7fa2f7400640 1 -- 172.21.15.39:0/1138990875 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f8004220 con 0x5567564e23b0 2024-09-21T07:50:30.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.507+0000 7fa2f7400640 1 -- 172.21.15.39:0/1138990875 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa2f80043c0 con 0x5567564e23b0 2024-09-21T07:50:30.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.507+0000 7fa2f7400640 1 -- 172.21.15.39:0/1138990875 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f8003830 con 0x5567564e23b0 2024-09-21T07:50:30.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.508+0000 7fa315dbd740 1 -- 172.21.15.39:0/1138990875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567564e23b0 msgr2=0x5567568b19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:30.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.508+0000 7fa315dbd740 1 --2- 172.21.15.39:0/1138990875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567564e23b0 0x5567568b19e0 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x5567568d12f0 tx=0x7fa2f8014000 comp rx=0 tx=0).stop 2024-09-21T07:50:30.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.508+0000 7fa315dbd740 1 -- 172.21.15.39:0/1138990875 shutdown_connections 2024-09-21T07:50:30.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.508+0000 7fa315dbd740 1 --2- 172.21.15.39:0/1138990875 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55675650b270 0x5567568f65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.508+0000 7fa315dbd740 1 --2- 172.21.15.39:0/1138990875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567564ea590 0x5567568f6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.508+0000 7fa315dbd740 1 --2- 172.21.15.39:0/1138990875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567564e23b0 0x5567568b19e0 unknown :-1 s=CLOSED pgs=2343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.508+0000 7fa315dbd740 1 -- 172.21.15.39:0/1138990875 >> 172.21.15.39:0/1138990875 conn(0x5567564ff3c0 msgr2=0x5567569061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:30.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.508+0000 7fa315dbd740 1 -- 172.21.15.39:0/1138990875 shutdown_connections 2024-09-21T07:50:30.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.508+0000 7fa315dbd740 1 -- 172.21.15.39:0/1138990875 wait complete. 2024-09-21T07:50:30.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.509+0000 7fa315dbd740 1 -- start start 2024-09-21T07:50:30.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.509+0000 7fa315dbd740 1 Processor -- start 2024-09-21T07:50:30.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.511+0000 7fa315dbd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567564e23b0 0x5567568f6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.511+0000 7fa315dbd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567564ea590 0x5567568f66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.511+0000 7fa315dbd740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55675650b270 0x556756976590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.511+0000 7fa315dbd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556756818810 con 0x5567564ea590 2024-09-21T07:50:30.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.511+0000 7fa315dbd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5567567bb330 con 0x5567564e23b0 2024-09-21T07:50:30.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.511+0000 7fa315dbd740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x5567568d9190 con 0x55675650b270 2024-09-21T07:50:30.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.511+0000 7fa315dbd740 10 client.0 _ll_register_callbacks cb 0x7fa30006cdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:50:30.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.511+0000 7fa2ffe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567564ea590 0x5567568f66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.511+0000 7fa2ffe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567564ea590 0x5567568f66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58614/0 (socket says 172.21.15.39:58614) 2024-09-21T07:50:30.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.511+0000 7fa2ffe00640 1 -- 172.21.15.39:0/3324188474 learned_addr learned my addr 172.21.15.39:0/3324188474 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:30.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.511+0000 7fa307e00640 1 --2- 172.21.15.39:0/3324188474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567564e23b0 0x5567568f6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.511+0000 7fa30cc00640 1 --2- 172.21.15.39:0/3324188474 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55675650b270 0x556756976590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.511+0000 7fa2ffe00640 1 -- 172.21.15.39:0/3324188474 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55675650b270 msgr2=0x556756976590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:30.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.511+0000 7fa2ffe00640 1 --2- 172.21.15.39:0/3324188474 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55675650b270 0x556756976590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.511+0000 7fa2ffe00640 1 -- 172.21.15.39:0/3324188474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567564e23b0 msgr2=0x5567568f6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:30.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.511+0000 7fa2ffe00640 1 --2- 172.21.15.39:0/3324188474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567564e23b0 0x5567568f6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.511+0000 7fa2ffe00640 1 -- 172.21.15.39:0/3324188474 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567568ac950 con 0x5567564ea590 2024-09-21T07:50:30.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.512+0000 7fa2ffe00640 1 --2- 172.21.15.39:0/3324188474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567564ea590 0x5567568f66a0 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0009230 tx=0x7fa2f0005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca942c3661780b8a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:30.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.512+0000 7fa2f4c00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:50:30.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.512+0000 7fa2f4c00640 1 -- 172.21.15.39:0/3324188474 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0035040 con 0x5567564ea590 2024-09-21T07:50:30.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.512+0000 7fa2f4c00640 1 -- 172.21.15.39:0/3324188474 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(238..238 src has 1..238) ==== 7262+0+0 (secure 0 0 0) 0x7fa2f00042d0 con 0x5567564ea590 2024-09-21T07:50:30.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.512+0000 7fa315dbd740 1 -- 172.21.15.39:0/3324188474 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2f80043c0 con 0x5567564ea590 2024-09-21T07:50:30.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.512+0000 7fa2f4c00640 1 -- 172.21.15.39:0/3324188474 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=257) ==== 24+0+0 (secure 0 0 0) 0x7fa2f00044d0 con 0x5567564ea590 2024-09-21T07:50:30.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.512+0000 7fa315dbd740 10 client.11034 fetch_fsmap learned FSMap version 257 2024-09-21T07:50:30.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.512+0000 7fa315dbd740 1 -- 172.21.15.39:0/3324188474 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=257,monmap=2+}) -- 0x5567568ac470 con 0x5567564ea590 2024-09-21T07:50:30.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.513+0000 7fa2f4c00640 1 -- 172.21.15.39:0/3324188474 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa2f80043c0 con 0x5567564ea590 2024-09-21T07:50:30.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.513+0000 7fa2f4c00640 1 -- 172.21.15.39:0/3324188474 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 257) ==== 3550+0+0 (secure 0 0 0) 0x7fa2f00035f0 con 0x5567564ea590 2024-09-21T07:50:30.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.513+0000 7fa315dbd740 10 client.11034 fetch_fsmap finished waiting for FSMap version 257 2024-09-21T07:50:30.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.513+0000 7fa315dbd740 10 client.11034 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:50:30.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.513+0000 7fa315dbd740 20 client.11034 populate_metadata read hostname 'smithi039' 2024-09-21T07:50:30.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.513+0000 7fa315dbd740 1 --2- 172.21.15.39:0/3324188474 >> [v2:172.21.15.39:6833/2824493631,v1:172.21.15.39:6835/2824493631] conn(0x556756976ef0 0x5567568cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.513+0000 7fa315dbd740 4 client.11034 mds_command: new command op to 10635 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:50:30.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.513+0000 7fa315dbd740 1 -- 172.21.15.39:0/3324188474 --> [v2:172.21.15.39:6833/2824493631,v1:172.21.15.39:6835/2824493631] -- command(tid 0: {"prefix": "status"}) -- 0x556756822fd0 con 0x556756976ef0 2024-09-21T07:50:30.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.513+0000 7fa307e00640 1 --2- 172.21.15.39:0/3324188474 >> [v2:172.21.15.39:6833/2824493631,v1:172.21.15.39:6835/2824493631] conn(0x556756976ef0 0x5567568cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.514+0000 7fa307e00640 1 --2- 172.21.15.39:0/3324188474 >> [v2:172.21.15.39:6833/2824493631,v1:172.21.15.39:6835/2824493631] conn(0x556756976ef0 0x5567568cf6f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c3b9ef7ab99966 server_cookie=6a966b9bc88d6306 in_seq=0 out_seq=0 2024-09-21T07:50:30.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.514+0000 7fa2f4c00640 10 client.11034 ms_handle_connect on v2:172.21.15.39:6833/2824493631 2024-09-21T07:50:30.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.514+0000 7fa2f4c00640 1 -- 172.21.15.39:0/3324188474 <== mds.0 v2:172.21.15.39:6833/2824493631 1 ==== command_reply(tid 0: 0 ) ==== 8+0+335 (crc 0 0 0) 0x556756822fd0 con 0x556756976ef0 2024-09-21T07:50:30.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.514+0000 7fa2f4c00640 10 client.11034 handle_command_reply: tid=0 2024-09-21T07:50:30.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.514+0000 7fa315dbd740 1 client.11034 shutdown 2024-09-21T07:50:30.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.514+0000 7fa315dbd740 1 -- 172.21.15.39:0/3324188474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567564ea590 msgr2=0x5567568f66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:30.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.514+0000 7fa315dbd740 1 --2- 172.21.15.39:0/3324188474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567564ea590 0x5567568f66a0 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0009230 tx=0x7fa2f0005750 comp rx=0 tx=0).stop 2024-09-21T07:50:30.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.514+0000 7fa315dbd740 1 -- 172.21.15.39:0/3324188474 shutdown_connections 2024-09-21T07:50:30.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.514+0000 7fa315dbd740 1 --2- 172.21.15.39:0/3324188474 >> [v2:172.21.15.39:6833/2824493631,v1:172.21.15.39:6835/2824493631] conn(0x556756976ef0 0x5567568cf6f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.514+0000 7fa315dbd740 1 --2- 172.21.15.39:0/3324188474 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55675650b270 0x556756976590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.514+0000 7fa315dbd740 1 --2- 172.21.15.39:0/3324188474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567564ea590 0x5567568f66a0 unknown :-1 s=CLOSED pgs=2344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.514+0000 7fa315dbd740 1 --2- 172.21.15.39:0/3324188474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567564e23b0 0x5567568f6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.514+0000 7fa2f4c00640 0 client.11034 ms_handle_reset on v2:172.21.15.39:6833/2824493631 2024-09-21T07:50:30.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.514+0000 7fa315dbd740 1 -- 172.21.15.39:0/3324188474 >> 172.21.15.39:0/3324188474 conn(0x5567564ff3c0 msgr2=0x5567568f0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:30.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.514+0000 7fa315dbd740 1 -- 172.21.15.39:0/3324188474 shutdown_connections 2024-09-21T07:50:30.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.514+0000 7fa315dbd740 1 -- 172.21.15.39:0/3324188474 wait complete. 2024-09-21T07:50:30.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.515+0000 7fa315dbd740 20 client.11034 trim_cache size 0 max 16384 2024-09-21T07:50:30.519 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:50:30.519 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:50:30.519 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-09-21T07:50:30.519 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 10635, 2024-09-21T07:50:30.519 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:50:30.519 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:50:30.520 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:50:30.520 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 55.401654274000002, 2024-09-21T07:50:30.520 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 257, 2024-09-21T07:50:30.520 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 238, 2024-09-21T07:50:30.520 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 238, 2024-09-21T07:50:30.520 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 60.404418536000001 2024-09-21T07:50:30.520 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:50:30.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.515+0000 7fa30ec00640 1 -- 172.21.15.39:0/1366591702 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa2e4078b90 msgr2=0x7fa2e407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:30.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.515+0000 7fa30ec00640 1 --2- 172.21.15.39:0/1366591702 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa2e4078b90 0x7fa2e407b0a0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fa308075150 tx=0x7fa2f8005f90 comp rx=0 tx=0).stop 2024-09-21T07:50:30.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.515+0000 7fa30ec00640 1 -- 172.21.15.39:0/1366591702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa308162f10 msgr2=0x7fa3080736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:30.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.515+0000 7fa30ec00640 1 --2- 172.21.15.39:0/1366591702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa308162f10 0x7fa3080736e0 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7fa300005f40 tx=0x7fa30000ee60 comp rx=0 tx=0).stop 2024-09-21T07:50:30.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.515+0000 7fa30ec00640 1 -- 172.21.15.39:0/1366591702 shutdown_connections 2024-09-21T07:50:30.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.515+0000 7fa30ec00640 1 --2- 172.21.15.39:0/1366591702 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa2e4078b90 0x7fa2e407b0a0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.515+0000 7fa30ec00640 1 --2- 172.21.15.39:0/1366591702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa308162f10 0x7fa3080736e0 unknown :-1 s=CLOSED pgs=2340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.515+0000 7fa30ec00640 1 --2- 172.21.15.39:0/1366591702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3080b3da0 0x7fa3080caa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.515+0000 7fa30ec00640 1 --2- 172.21.15.39:0/1366591702 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa3080b3990 0x7fa3081e8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.515+0000 7fa30ec00640 1 -- 172.21.15.39:0/1366591702 >> 172.21.15.39:0/1366591702 conn(0x7fa3080b2460 msgr2=0x7fa3080ade20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:30.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.515+0000 7fa30ec00640 1 -- 172.21.15.39:0/1366591702 shutdown_connections 2024-09-21T07:50:30.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.515+0000 7fa30ec00640 1 -- 172.21.15.39:0/1366591702 wait complete. 2024-09-21T07:50:30.541 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:30.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.765+0000 7fe961800640 1 -- 172.21.15.39:0/100916673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe95c0cfd30 msgr2=0x7fe95c0d0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:30.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.765+0000 7fe961800640 1 --2- 172.21.15.39:0/100916673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe95c0cfd30 0x7fe95c0d0b70 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7fe95c0b5a10 tx=0x7fe95402aa00 comp rx=0 tx=0).stop 2024-09-21T07:50:30.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.765+0000 7fe961800640 1 -- 172.21.15.39:0/100916673 shutdown_connections 2024-09-21T07:50:30.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.765+0000 7fe961800640 1 --2- 172.21.15.39:0/100916673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe95c0cfd30 0x7fe95c0d0b70 unknown :-1 s=CLOSED pgs=2345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.765+0000 7fe961800640 1 --2- 172.21.15.39:0/100916673 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe95c0cf920 0x7fe95c0d0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.765+0000 7fe961800640 1 --2- 172.21.15.39:0/100916673 >> v2:172.21.15.148:3300/0 conn(0x7fe95c0b5240 0x7fe95c15c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.765+0000 7fe961800640 1 -- 172.21.15.39:0/100916673 >> 172.21.15.39:0/100916673 conn(0x7fe95c06cfc0 msgr2=0x7fe95c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:30.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.765+0000 7fe961800640 1 -- 172.21.15.39:0/100916673 shutdown_connections 2024-09-21T07:50:30.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.765+0000 7fe961800640 1 -- 172.21.15.39:0/100916673 wait complete. 2024-09-21T07:50:30.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.768+0000 7fe961800640 1 Processor -- start 2024-09-21T07:50:30.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.768+0000 7fe961800640 1 -- start start 2024-09-21T07:50:30.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.769+0000 7fe961800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe95c0b5240 0x7fe95c166700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.769+0000 7fe961800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe95c0cf920 0x7fe95c1514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.769+0000 7fe961800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe95c0cfd30 0x7fe95c1593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.769+0000 7fe961800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe95c0b4760 con 0x7fe95c0cfd30 2024-09-21T07:50:30.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.769+0000 7fe961800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe95c0b45e0 con 0x7fe95c0cf920 2024-09-21T07:50:30.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.769+0000 7fe961800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe95c0b48e0 con 0x7fe95c0b5240 2024-09-21T07:50:30.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.770+0000 7fe95b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe95c0cfd30 0x7fe95c1593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.770+0000 7fe95b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe95c0cfd30 0x7fe95c1593f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58624/0 (socket says 172.21.15.39:58624) 2024-09-21T07:50:30.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.770+0000 7fe95b400640 1 -- 172.21.15.39:0/3042056434 learned_addr learned my addr 172.21.15.39:0/3042056434 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:30.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.770+0000 7fe95aa00640 1 --2- 172.21.15.39:0/3042056434 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe95c0b5240 0x7fe95c166700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.770+0000 7fe95a000640 1 --2- 172.21.15.39:0/3042056434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe95c0cf920 0x7fe95c1514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.770+0000 7fe95b400640 1 -- 172.21.15.39:0/3042056434 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe95c0b5240 msgr2=0x7fe95c166700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:30.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.770+0000 7fe95b400640 1 --2- 172.21.15.39:0/3042056434 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe95c0b5240 0x7fe95c166700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.770+0000 7fe95b400640 1 -- 172.21.15.39:0/3042056434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe95c0cf920 msgr2=0x7fe95c1514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:30.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.770+0000 7fe95b400640 1 --2- 172.21.15.39:0/3042056434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe95c0cf920 0x7fe95c1514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.770+0000 7fe95b400640 1 -- 172.21.15.39:0/3042056434 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe95c150700 con 0x7fe95c0cfd30 2024-09-21T07:50:30.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.770+0000 7fe95b400640 1 --2- 172.21.15.39:0/3042056434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe95c0cfd30 0x7fe95c1593f0 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7fe95402aff0 tx=0x7fe954003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:30.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.771+0000 7fe952a00640 1 -- 172.21.15.39:0/3042056434 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe954033050 con 0x7fe95c0cfd30 2024-09-21T07:50:30.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.771+0000 7fe952a00640 1 -- 172.21.15.39:0/3042056434 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe954004450 con 0x7fe95c0cfd30 2024-09-21T07:50:30.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.771+0000 7fe961800640 1 -- 172.21.15.39:0/3042056434 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe95c167030 con 0x7fe95c0cfd30 2024-09-21T07:50:30.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.771+0000 7fe952a00640 1 -- 172.21.15.39:0/3042056434 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe954038450 con 0x7fe95c0cfd30 2024-09-21T07:50:30.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.771+0000 7fe961800640 1 -- 172.21.15.39:0/3042056434 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe95c150ca0 con 0x7fe95c0cfd30 2024-09-21T07:50:30.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.772+0000 7fe961800640 1 -- 172.21.15.39:0/3042056434 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe95c166da0 con 0x7fe95c0cfd30 2024-09-21T07:50:30.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.773+0000 7fe952a00640 1 -- 172.21.15.39:0/3042056434 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe954036070 con 0x7fe95c0cfd30 2024-09-21T07:50:30.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.774+0000 7fe952a00640 1 --2- 172.21.15.39:0/3042056434 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe93c078b90 0x7fe93c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.774+0000 7fe952a00640 1 -- 172.21.15.39:0/3042056434 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(238..238 src has 1..238) ==== 7262+0+0 (secure 0 0 0) 0x7fe9540bc750 con 0x7fe95c0cfd30 2024-09-21T07:50:30.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.775+0000 7fe95aa00640 1 --2- 172.21.15.39:0/3042056434 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe93c078b90 0x7fe93c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.775+0000 7fe95aa00640 1 --2- 172.21.15.39:0/3042056434 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe93c078b90 0x7fe93c07b0a0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fe94c004590 tx=0x7fe94c00b390 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:30.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:30.779+0000 7fe952a00640 1 -- 172.21.15.39:0/3042056434 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe954083e90 con 0x7fe95c0cfd30 2024-09-21T07:50:31.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.019+0000 7fe961800640 1 -- 172.21.15.39:0/3042056434 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe95c0d0eb0 con 0x7fe95c0cfd30 2024-09-21T07:50:31.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.019+0000 7fe952a00640 1 -- 172.21.15.39:0/3042056434 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 257 v257) ==== 95+0+5292 (secure 0 0 0) 0x7fe95c0d0eb0 con 0x7fe95c0cfd30 2024-09-21T07:50:31.021 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:50:31.021 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":257,"btime":"2024-09-21T07:50:28:886412+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9967,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":254}],"filesystems":[{"mdsmap":{"epoch":257,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:50:28.886406+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":237,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":10635,"mds_1":10632,"mds_2":8531},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8531":{"gid":8531,"name":"d","rank":2,"incarnation":253,"state":"up:active","state_seq":18,"addr":"172.21.15.148:6837/1438851059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1438851059},{"type":"v1","addr":"172.21.15.148:6837","nonce":1438851059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10632":{"gid":10632,"name":"a","rank":1,"incarnation":247,"state":"up:active","state_seq":6,"addr":"172.21.15.39:6834/4143807732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4143807732},{"type":"v1","addr":"172.21.15.39:6834","nonce":4143807732}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10635":{"gid":10635,"name":"c","rank":0,"incarnation":240,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/2824493631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2824493631},{"type":"v1","addr":"172.21.15.39:6835","nonce":2824493631}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10635,"qdb_cluster":[8531,10632,10635]},"id":13}]} 2024-09-21T07:50:31.021 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 257 2024-09-21T07:50:31.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.022+0000 7fe961800640 1 -- 172.21.15.39:0/3042056434 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe93c078b90 msgr2=0x7fe93c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:31.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.022+0000 7fe961800640 1 --2- 172.21.15.39:0/3042056434 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe93c078b90 0x7fe93c07b0a0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fe94c004590 tx=0x7fe94c00b390 comp rx=0 tx=0).stop 2024-09-21T07:50:31.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.022+0000 7fe961800640 1 -- 172.21.15.39:0/3042056434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe95c0cfd30 msgr2=0x7fe95c1593f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:31.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.022+0000 7fe961800640 1 --2- 172.21.15.39:0/3042056434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe95c0cfd30 0x7fe95c1593f0 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7fe95402aff0 tx=0x7fe954003f90 comp rx=0 tx=0).stop 2024-09-21T07:50:31.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.022+0000 7fe961800640 1 -- 172.21.15.39:0/3042056434 shutdown_connections 2024-09-21T07:50:31.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.022+0000 7fe961800640 1 --2- 172.21.15.39:0/3042056434 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe93c078b90 0x7fe93c07b0a0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:31.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.022+0000 7fe961800640 1 --2- 172.21.15.39:0/3042056434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe95c0cfd30 0x7fe95c1593f0 unknown :-1 s=CLOSED pgs=2346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:31.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.022+0000 7fe961800640 1 --2- 172.21.15.39:0/3042056434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe95c0cf920 0x7fe95c1514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:31.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.022+0000 7fe961800640 1 --2- 172.21.15.39:0/3042056434 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe95c0b5240 0x7fe95c166700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:31.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.022+0000 7fe961800640 1 -- 172.21.15.39:0/3042056434 >> 172.21.15.39:0/3042056434 conn(0x7fe95c06cfc0 msgr2=0x7fe95c15ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:31.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.022+0000 7fe961800640 1 -- 172.21.15.39:0/3042056434 shutdown_connections 2024-09-21T07:50:31.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.022+0000 7fe961800640 1 -- 172.21.15.39:0/3042056434 wait complete. 2024-09-21T07:50:31.046 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json dump snaps 2024-09-21T07:50:31.300 INFO:teuthology.orchestra.run.smithi148.stdout:{"last_created":3,"last_destroyed":4,"snaps":[{"snapid":3,"ino":1099511627778,"stamp":"2024-09-21T07:50:02.909971+0000","name":"s2","metadata":{}}]} 2024-09-21T07:50:31.300 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 3, "last_destroyed": 4, "snaps": [ { "ino": 1099511627778, "metadata": {}, "name": "s2", "snapid": 3, "stamp": "2024-09-21T07:50:02.909971+0000" } ] } 2024-09-21T07:50:31.300 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 13:0 2024-09-21T07:50:31.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.528+0000 7f5c91e00640 1 -- 172.21.15.39:0/3332901326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c8c15cc00 msgr2=0x7f5c8c15d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:31.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.528+0000 7f5c91e00640 1 --2- 172.21.15.39:0/3332901326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c8c15cc00 0x7f5c8c15d010 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c073490 tx=0x7f5c7402a560 comp rx=0 tx=0).stop 2024-09-21T07:50:31.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.528+0000 7f5c91e00640 1 -- 172.21.15.39:0/3332901326 shutdown_connections 2024-09-21T07:50:31.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.528+0000 7f5c91e00640 1 --2- 172.21.15.39:0/3332901326 >> v2:172.21.15.148:3300/0 conn(0x7f5c8c0d04b0 0x7f5c8c0d08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:31.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.528+0000 7f5c91e00640 1 --2- 172.21.15.39:0/3332901326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c8c15cc00 0x7f5c8c15d010 unknown :-1 s=CLOSED pgs=2347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:31.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.528+0000 7f5c91e00640 1 --2- 172.21.15.39:0/3332901326 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c8c0cfa20 0x7f5c8c0cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:31.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.528+0000 7f5c91e00640 1 -- 172.21.15.39:0/3332901326 >> 172.21.15.39:0/3332901326 conn(0x7f5c8c06cfc0 msgr2=0x7f5c8c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:31.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.528+0000 7f5c91e00640 1 -- 172.21.15.39:0/3332901326 shutdown_connections 2024-09-21T07:50:31.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.528+0000 7f5c91e00640 1 -- 172.21.15.39:0/3332901326 wait complete. 2024-09-21T07:50:31.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.530+0000 7f5c91e00640 1 Processor -- start 2024-09-21T07:50:31.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.530+0000 7f5c91e00640 1 -- start start 2024-09-21T07:50:31.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.532+0000 7f5c91e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c8c0cfa20 0x7f5c8c0aef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:31.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.532+0000 7f5c91e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c8c0d04b0 0x7f5c8c1ec070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:31.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.532+0000 7f5c91e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c8c15cc00 0x7f5c8c0cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:31.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.532+0000 7f5c91e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5c8c06ecf0 con 0x7f5c8c15cc00 2024-09-21T07:50:31.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.532+0000 7f5c91e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5c8c074f60 con 0x7f5c8c0d04b0 2024-09-21T07:50:31.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.532+0000 7f5c91e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f5c8c06ee70 con 0x7f5c8c0cfa20 2024-09-21T07:50:31.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.532+0000 7f5c91400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c8c15cc00 0x7f5c8c0cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:31.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.532+0000 7f5c91400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c8c15cc00 0x7f5c8c0cbac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58644/0 (socket says 172.21.15.39:58644) 2024-09-21T07:50:31.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.532+0000 7f5c90a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c8c0cfa20 0x7f5c8c0aef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:31.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.532+0000 7f5c91400640 1 -- 172.21.15.39:0/2596779342 learned_addr learned my addr 172.21.15.39:0/2596779342 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:31.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.532+0000 7f5c83e00640 1 --2- 172.21.15.39:0/2596779342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c8c0d04b0 0x7f5c8c1ec070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:31.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.532+0000 7f5c91400640 1 -- 172.21.15.39:0/2596779342 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c8c0cfa20 msgr2=0x7f5c8c0aef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:31.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.532+0000 7f5c91400640 1 --2- 172.21.15.39:0/2596779342 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c8c0cfa20 0x7f5c8c0aef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:31.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.533+0000 7f5c91400640 1 -- 172.21.15.39:0/2596779342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c8c0d04b0 msgr2=0x7f5c8c1ec070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:31.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.533+0000 7f5c91400640 1 --2- 172.21.15.39:0/2596779342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c8c0d04b0 0x7f5c8c1ec070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:31.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.533+0000 7f5c91400640 1 -- 172.21.15.39:0/2596779342 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c8c15bf40 con 0x7f5c8c15cc00 2024-09-21T07:50:31.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.533+0000 7f5c91400640 1 --2- 172.21.15.39:0/2596779342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c8c15cc00 0x7f5c8c0cbac0 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7f5c8400ea70 tx=0x7f5c8400ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:31.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.533+0000 7f5c89600640 1 -- 172.21.15.39:0/2596779342 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8400ce40 con 0x7f5c8c15cc00 2024-09-21T07:50:31.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.533+0000 7f5c89600640 1 -- 172.21.15.39:0/2596779342 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c84004c10 con 0x7f5c8c15cc00 2024-09-21T07:50:31.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.533+0000 7f5c89600640 1 -- 172.21.15.39:0/2596779342 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c84005020 con 0x7f5c8c15cc00 2024-09-21T07:50:31.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.533+0000 7f5c91e00640 1 -- 172.21.15.39:0/2596779342 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c8c0ae420 con 0x7f5c8c15cc00 2024-09-21T07:50:31.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.533+0000 7f5c91e00640 1 -- 172.21.15.39:0/2596779342 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c8c0ae9e0 con 0x7f5c8c15cc00 2024-09-21T07:50:31.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.534+0000 7f5c91e00640 1 -- 172.21.15.39:0/2596779342 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c8c15bcb0 con 0x7f5c8c15cc00 2024-09-21T07:50:31.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.536+0000 7f5c89600640 1 -- 172.21.15.39:0/2596779342 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5c840051c0 con 0x7f5c8c15cc00 2024-09-21T07:50:31.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.536+0000 7f5c89600640 1 --2- 172.21.15.39:0/2596779342 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5c68078b90 0x7f5c6807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:31.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.537+0000 7f5c89600640 1 -- 172.21.15.39:0/2596779342 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(238..238 src has 1..238) ==== 7262+0+0 (secure 0 0 0) 0x7f5c8409bff0 con 0x7f5c8c15cc00 2024-09-21T07:50:31.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.537+0000 7f5c90a00640 1 --2- 172.21.15.39:0/2596779342 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5c68078b90 0x7f5c6807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:31.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.538+0000 7f5c90a00640 1 --2- 172.21.15.39:0/2596779342 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5c68078b90 0x7f5c6807b0a0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c009b30 tx=0x7f5c7c005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:31.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.541+0000 7f5c89600640 1 -- 172.21.15.39:0/2596779342 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5c840637c0 con 0x7f5c8c15cc00 2024-09-21T07:50:31.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.816+0000 7f5c91e00640 1 -- 172.21.15.39:0/2596779342 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "13:0"} v 0) -- 0x7f5c8c0d08c0 con 0x7f5c8c15cc00 2024-09-21T07:50:31.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.819+0000 7f5c89600640 1 -- 172.21.15.39:0/2596779342 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "13:0"}]=0 failed mds gid 10635 v258) ==== 99+0+0 (secure 0 0 0) 0x7f5c840681f0 con 0x7f5c8c15cc00 2024-09-21T07:50:31.821 INFO:teuthology.orchestra.run.smithi039.stderr:failed mds gid 10635 2024-09-21T07:50:31.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.822+0000 7f5c91e00640 1 -- 172.21.15.39:0/2596779342 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5c68078b90 msgr2=0x7f5c6807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:31.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.822+0000 7f5c91e00640 1 --2- 172.21.15.39:0/2596779342 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5c68078b90 0x7f5c6807b0a0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c009b30 tx=0x7f5c7c005f90 comp rx=0 tx=0).stop 2024-09-21T07:50:31.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.822+0000 7f5c91e00640 1 -- 172.21.15.39:0/2596779342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c8c15cc00 msgr2=0x7f5c8c0cbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:31.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.822+0000 7f5c91e00640 1 --2- 172.21.15.39:0/2596779342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c8c15cc00 0x7f5c8c0cbac0 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7f5c8400ea70 tx=0x7f5c8400ef30 comp rx=0 tx=0).stop 2024-09-21T07:50:31.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.822+0000 7f5c91e00640 1 -- 172.21.15.39:0/2596779342 shutdown_connections 2024-09-21T07:50:31.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.822+0000 7f5c91e00640 1 --2- 172.21.15.39:0/2596779342 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5c68078b90 0x7f5c6807b0a0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:31.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.822+0000 7f5c91e00640 1 --2- 172.21.15.39:0/2596779342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c8c15cc00 0x7f5c8c0cbac0 unknown :-1 s=CLOSED pgs=2348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:31.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.822+0000 7f5c91e00640 1 --2- 172.21.15.39:0/2596779342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c8c0d04b0 0x7f5c8c1ec070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:31.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.822+0000 7f5c91e00640 1 --2- 172.21.15.39:0/2596779342 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5c8c0cfa20 0x7f5c8c0aef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:31.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.822+0000 7f5c91e00640 1 -- 172.21.15.39:0/2596779342 >> 172.21.15.39:0/2596779342 conn(0x7f5c8c06cfc0 msgr2=0x7f5c8c1565b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:31.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.822+0000 7f5c91e00640 1 -- 172.21.15.39:0/2596779342 shutdown_connections 2024-09-21T07:50:31.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:31.822+0000 7f5c91e00640 1 -- 172.21.15.39:0/2596779342 wait complete. 2024-09-21T07:50:31.835 INFO:tasks.ceph.mds.c.smithi039.stderr: -8870> 2024-09-21T07:49:37.078+0000 7f7ede400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:50:31.848 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 13:1 2024-09-21T07:50:31.971 INFO:tasks.ceph.mds.c.smithi039.stdout:starting mds.c at 2024-09-21T07:50:32.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.083+0000 7f351ec00640 1 -- 172.21.15.39:0/3721357848 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3518074c10 msgr2=0x7f35180cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:32.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.083+0000 7f351ec00640 1 --2- 172.21.15.39:0/3721357848 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3518074c10 0x7f35180cfb40 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7f3518072ba0 tx=0x7f350c02a540 comp rx=0 tx=0).stop 2024-09-21T07:50:32.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.083+0000 7f351ec00640 1 -- 172.21.15.39:0/3721357848 shutdown_connections 2024-09-21T07:50:32.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.083+0000 7f351ec00640 1 --2- 172.21.15.39:0/3721357848 >> v2:172.21.15.148:3300/0 conn(0x7f35180d0550 0x7f35180d0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:32.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.083+0000 7f351ec00640 1 --2- 172.21.15.39:0/3721357848 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3518074c10 0x7f35180cfb40 unknown :-1 s=CLOSED pgs=2352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:32.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.083+0000 7f351ec00640 1 -- 172.21.15.39:0/3721357848 >> 172.21.15.39:0/3721357848 conn(0x7f351806cfc0 msgr2=0x7f351806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:32.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.083+0000 7f351ec00640 1 -- 172.21.15.39:0/3721357848 shutdown_connections 2024-09-21T07:50:32.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.083+0000 7f351ec00640 1 -- 172.21.15.39:0/3721357848 wait complete. 2024-09-21T07:50:32.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.085+0000 7f351ec00640 1 Processor -- start 2024-09-21T07:50:32.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.085+0000 7f351ec00640 1 -- start start 2024-09-21T07:50:32.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.087+0000 7f351ec00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f351806ecf0 0x7f3518138a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:32.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.087+0000 7f351ec00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3518074c10 0x7f351815aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:32.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.087+0000 7f351ec00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35180d0550 0x7f351815afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:32.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.087+0000 7f351ec00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f351815d0a0 con 0x7f3518074c10 2024-09-21T07:50:32.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.087+0000 7f351ec00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f351815cf20 con 0x7f35180d0550 2024-09-21T07:50:32.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.087+0000 7f351ec00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f351815d220 con 0x7f351806ecf0 2024-09-21T07:50:32.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.087+0000 7f351ce00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3518074c10 0x7f351815aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:32.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.087+0000 7f351ce00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3518074c10 0x7f351815aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50552/0 (socket says 172.21.15.39:50552) 2024-09-21T07:50:32.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.087+0000 7f351ce00640 1 -- 172.21.15.39:0/3949044717 learned_addr learned my addr 172.21.15.39:0/3949044717 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:32.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.087+0000 7f351e200640 1 --2- 172.21.15.39:0/3949044717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35180d0550 0x7f351815afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:32.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.087+0000 7f351d800640 1 --2- 172.21.15.39:0/3949044717 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f351806ecf0 0x7f3518138a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:32.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.087+0000 7f351ce00640 1 -- 172.21.15.39:0/3949044717 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f351806ecf0 msgr2=0x7f3518138a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:32.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.087+0000 7f351ce00640 1 --2- 172.21.15.39:0/3949044717 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f351806ecf0 0x7f3518138a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:32.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.087+0000 7f351ce00640 1 -- 172.21.15.39:0/3949044717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35180d0550 msgr2=0x7f351815afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:32.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.087+0000 7f351ce00640 1 --2- 172.21.15.39:0/3949044717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35180d0550 0x7f351815afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:32.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.087+0000 7f351ce00640 1 -- 172.21.15.39:0/3949044717 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35180b48e0 con 0x7f3518074c10 2024-09-21T07:50:32.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.087+0000 7f351e200640 1 --2- 172.21.15.39:0/3949044717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35180d0550 0x7f351815afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:50:32.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.088+0000 7f351ce00640 1 --2- 172.21.15.39:0/3949044717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3518074c10 0x7f351815aa70 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7f350400c3d0 tx=0x7f350400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:32.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.088+0000 7f3515600640 1 -- 172.21.15.39:0/3949044717 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3504011790 con 0x7f3518074c10 2024-09-21T07:50:32.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.088+0000 7f3515600640 1 -- 172.21.15.39:0/3949044717 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3504017070 con 0x7f3518074c10 2024-09-21T07:50:32.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.088+0000 7f3515600640 1 -- 172.21.15.39:0/3949044717 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3504011cc0 con 0x7f3518074c10 2024-09-21T07:50:32.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.088+0000 7f351ec00640 1 -- 172.21.15.39:0/3949044717 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3518159f90 con 0x7f3518074c10 2024-09-21T07:50:32.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.088+0000 7f351ec00640 1 -- 172.21.15.39:0/3949044717 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3518138fb0 con 0x7f3518074c10 2024-09-21T07:50:32.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.089+0000 7f351ec00640 1 -- 172.21.15.39:0/3949044717 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35180b4650 con 0x7f3518074c10 2024-09-21T07:50:32.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.091+0000 7f3515600640 1 -- 172.21.15.39:0/3949044717 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f350401c460 con 0x7f3518074c10 2024-09-21T07:50:32.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.092+0000 7f3515600640 1 --2- 172.21.15.39:0/3949044717 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f34f8078b90 0x7f34f807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:32.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.092+0000 7f3515600640 1 -- 172.21.15.39:0/3949044717 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(239..239 src has 1..239) ==== 7304+0+0 (secure 0 0 0) 0x7f350409afb0 con 0x7f3518074c10 2024-09-21T07:50:32.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.092+0000 7f351d800640 1 --2- 172.21.15.39:0/3949044717 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f34f8078b90 0x7f34f807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:32.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.093+0000 7f351d800640 1 --2- 172.21.15.39:0/3949044717 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f34f8078b90 0x7f34f807b0a0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f350c034000 tx=0x7f350c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:32.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.097+0000 7f3515600640 1 -- 172.21.15.39:0/3949044717 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f35180d0ea0 con 0x7f3518074c10 2024-09-21T07:50:32.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.367+0000 7f351ec00640 1 -- 172.21.15.39:0/3949044717 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "13:1"} v 0) -- 0x7f351815d740 con 0x7f3518074c10 2024-09-21T07:50:32.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.370+0000 7f3515600640 1 -- 172.21.15.39:0/3949044717 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "13:1"}]=0 failed mds gid 10632 v260) ==== 99+0+0 (secure 0 0 0) 0x7f3504062750 con 0x7f3518074c10 2024-09-21T07:50:32.371 INFO:teuthology.orchestra.run.smithi039.stderr:failed mds gid 10632 2024-09-21T07:50:32.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.372+0000 7f351ec00640 1 -- 172.21.15.39:0/3949044717 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f34f8078b90 msgr2=0x7f34f807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:32.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.372+0000 7f351ec00640 1 --2- 172.21.15.39:0/3949044717 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f34f8078b90 0x7f34f807b0a0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f350c034000 tx=0x7f350c004730 comp rx=0 tx=0).stop 2024-09-21T07:50:32.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.372+0000 7f351ec00640 1 -- 172.21.15.39:0/3949044717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3518074c10 msgr2=0x7f351815aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:32.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.372+0000 7f351ec00640 1 --2- 172.21.15.39:0/3949044717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3518074c10 0x7f351815aa70 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7f350400c3d0 tx=0x7f350400c890 comp rx=0 tx=0).stop 2024-09-21T07:50:32.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.372+0000 7f351ec00640 1 -- 172.21.15.39:0/3949044717 shutdown_connections 2024-09-21T07:50:32.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.372+0000 7f351ec00640 1 --2- 172.21.15.39:0/3949044717 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f34f8078b90 0x7f34f807b0a0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:32.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.372+0000 7f351ec00640 1 --2- 172.21.15.39:0/3949044717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35180d0550 0x7f351815afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:32.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.372+0000 7f351ec00640 1 --2- 172.21.15.39:0/3949044717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3518074c10 0x7f351815aa70 unknown :-1 s=CLOSED pgs=2353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:32.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.372+0000 7f351ec00640 1 --2- 172.21.15.39:0/3949044717 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f351806ecf0 0x7f3518138a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:32.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.372+0000 7f351ec00640 1 -- 172.21.15.39:0/3949044717 >> 172.21.15.39:0/3949044717 conn(0x7f351806cfc0 msgr2=0x7f3518157660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:32.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.372+0000 7f351ec00640 1 -- 172.21.15.39:0/3949044717 shutdown_connections 2024-09-21T07:50:32.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.372+0000 7f351ec00640 1 -- 172.21.15.39:0/3949044717 wait complete. 2024-09-21T07:50:32.411 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:32.516 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-21T07:50:32.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.639+0000 7fa8f8e00640 1 -- 172.21.15.39:0/1542352679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8f40d0390 msgr2=0x7fa8f40d07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:32.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.639+0000 7fa8f8e00640 1 --2- 172.21.15.39:0/1542352679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8f40d0390 0x7fa8f40d07a0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec009be0 tx=0x7fa8ec02a960 comp rx=0 tx=0).stop 2024-09-21T07:50:32.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.639+0000 7fa8f8e00640 1 -- 172.21.15.39:0/1542352679 shutdown_connections 2024-09-21T07:50:32.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.639+0000 7fa8f8e00640 1 --2- 172.21.15.39:0/1542352679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8f40d0390 0x7fa8f40d07a0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:32.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.639+0000 7fa8f8e00640 1 -- 172.21.15.39:0/1542352679 >> 172.21.15.39:0/1542352679 conn(0x7fa8f406cfc0 msgr2=0x7fa8f406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:32.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.639+0000 7fa8f8e00640 1 -- 172.21.15.39:0/1542352679 shutdown_connections 2024-09-21T07:50:32.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.639+0000 7fa8f8e00640 1 -- 172.21.15.39:0/1542352679 wait complete. 2024-09-21T07:50:32.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.641+0000 7fa8f8e00640 1 Processor -- start 2024-09-21T07:50:32.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.641+0000 7fa8f8e00640 1 -- start start 2024-09-21T07:50:32.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.643+0000 7fa8f8e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8f40d0390 0x7fa8f41444b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:32.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.643+0000 7fa8f8e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8f4160250 0x7fa8f415b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:32.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.643+0000 7fa8f8e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8f4160660 0x7fa8f415bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:32.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.643+0000 7fa8f8e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f40ccf30 con 0x7fa8f40d0390 2024-09-21T07:50:32.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.643+0000 7fa8f8e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f40ccdb0 con 0x7fa8f4160660 2024-09-21T07:50:32.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.643+0000 7fa8f8e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa8f40cd0b0 con 0x7fa8f4160250 2024-09-21T07:50:32.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.643+0000 7fa8f3400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8f40d0390 0x7fa8f41444b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:32.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.643+0000 7fa8f3400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8f40d0390 0x7fa8f41444b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50572/0 (socket says 172.21.15.39:50572) 2024-09-21T07:50:32.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.643+0000 7fa8f3400640 1 -- 172.21.15.39:0/2954278548 learned_addr learned my addr 172.21.15.39:0/2954278548 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:32.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.643+0000 7fa8f2a00640 1 --2- 172.21.15.39:0/2954278548 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8f4160250 0x7fa8f415b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:32.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.643+0000 7fa8f3e00640 1 --2- 172.21.15.39:0/2954278548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8f4160660 0x7fa8f415bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:32.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.644+0000 7fa8f3400640 1 -- 172.21.15.39:0/2954278548 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8f4160250 msgr2=0x7fa8f415b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:32.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.644+0000 7fa8f3400640 1 --2- 172.21.15.39:0/2954278548 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8f4160250 0x7fa8f415b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:32.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.644+0000 7fa8f3400640 1 -- 172.21.15.39:0/2954278548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8f4160660 msgr2=0x7fa8f415bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:32.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.644+0000 7fa8f3400640 1 --2- 172.21.15.39:0/2954278548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8f4160660 0x7fa8f415bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:32.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.644+0000 7fa8f3400640 1 -- 172.21.15.39:0/2954278548 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8f40b4b40 con 0x7fa8f40d0390 2024-09-21T07:50:32.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.644+0000 7fa8f3400640 1 --2- 172.21.15.39:0/2954278548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8f40d0390 0x7fa8f41444b0 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7fa8e400c3c0 tx=0x7fa8e400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:32.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.644+0000 7fa8eb400640 1 -- 172.21.15.39:0/2954278548 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e4017070 con 0x7fa8f40d0390 2024-09-21T07:50:32.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.644+0000 7fa8eb400640 1 -- 172.21.15.39:0/2954278548 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8e40035f0 con 0x7fa8f40d0390 2024-09-21T07:50:32.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.644+0000 7fa8f8e00640 1 -- 172.21.15.39:0/2954278548 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8f40b4de0 con 0x7fa8f40d0390 2024-09-21T07:50:32.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.644+0000 7fa8eb400640 1 -- 172.21.15.39:0/2954278548 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e4003a80 con 0x7fa8f40d0390 2024-09-21T07:50:32.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.644+0000 7fa8f8e00640 1 -- 172.21.15.39:0/2954278548 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8f40b50c0 con 0x7fa8f40d0390 2024-09-21T07:50:32.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.645+0000 7fa8f8e00640 1 -- 172.21.15.39:0/2954278548 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8f4145360 con 0x7fa8f40d0390 2024-09-21T07:50:32.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.647+0000 7fa8eb400640 1 -- 172.21.15.39:0/2954278548 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa8e401a070 con 0x7fa8f40d0390 2024-09-21T07:50:32.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.648+0000 7fa8eb400640 1 --2- 172.21.15.39:0/2954278548 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa8d4078b90 0x7fa8d407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:32.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.648+0000 7fa8eb400640 1 -- 172.21.15.39:0/2954278548 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(240..240 src has 1..240) ==== 7390+0+0 (secure 0 0 0) 0x7fa8e409a750 con 0x7fa8f40d0390 2024-09-21T07:50:32.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.648+0000 7fa8f2a00640 1 --2- 172.21.15.39:0/2954278548 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa8d4078b90 0x7fa8d407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:32.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.649+0000 7fa8f2a00640 1 --2- 172.21.15.39:0/2954278548 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa8d4078b90 0x7fa8d407b0a0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0007d60 tx=0x7fa8e0007cf0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:32.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.652+0000 7fa8eb400640 1 -- 172.21.15.39:0/2954278548 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa8e4061d90 con 0x7fa8f40d0390 2024-09-21T07:50:32.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.887+0000 7fa8f8e00640 1 -- 172.21.15.39:0/2954278548 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8f40d07a0 con 0x7fa8f40d0390 2024-09-21T07:50:32.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.887+0000 7fa8eb400640 1 -- 172.21.15.39:0/2954278548 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 261 v261) ==== 95+0+4446 (secure 0 0 0) 0x7fa8e40667c0 con 0x7fa8f40d0390 2024-09-21T07:50:32.889 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:50:32.889 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":261,"btime":"2024-09-21T07:50:32:379946+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":261,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:50:32.379926+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":240,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":8531},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8531":{"gid":8531,"name":"d","rank":2,"incarnation":253,"state":"up:active","state_seq":18,"addr":"172.21.15.148:6837/1438851059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1438851059},{"type":"v1","addr":"172.21.15.148:6837","nonce":1438851059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:resolve","state_seq":3,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:replay","state_seq":1,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8531,"qdb_cluster":[8531]},"id":13}]} 2024-09-21T07:50:32.889 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 261 2024-09-21T07:50:32.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.889+0000 7fa8f8e00640 1 -- 172.21.15.39:0/2954278548 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa8d4078b90 msgr2=0x7fa8d407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:32.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.889+0000 7fa8f8e00640 1 --2- 172.21.15.39:0/2954278548 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa8d4078b90 0x7fa8d407b0a0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0007d60 tx=0x7fa8e0007cf0 comp rx=0 tx=0).stop 2024-09-21T07:50:32.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.889+0000 7fa8f8e00640 1 -- 172.21.15.39:0/2954278548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8f40d0390 msgr2=0x7fa8f41444b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:32.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.889+0000 7fa8f8e00640 1 --2- 172.21.15.39:0/2954278548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8f40d0390 0x7fa8f41444b0 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7fa8e400c3c0 tx=0x7fa8e400c880 comp rx=0 tx=0).stop 2024-09-21T07:50:32.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.889+0000 7fa8f8e00640 1 -- 172.21.15.39:0/2954278548 shutdown_connections 2024-09-21T07:50:32.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.889+0000 7fa8f8e00640 1 --2- 172.21.15.39:0/2954278548 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa8d4078b90 0x7fa8d407b0a0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:32.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.889+0000 7fa8f8e00640 1 --2- 172.21.15.39:0/2954278548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8f4160660 0x7fa8f415bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:32.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.889+0000 7fa8f8e00640 1 --2- 172.21.15.39:0/2954278548 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa8f4160250 0x7fa8f415b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:32.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.889+0000 7fa8f8e00640 1 --2- 172.21.15.39:0/2954278548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8f40d0390 0x7fa8f41444b0 unknown :-1 s=CLOSED pgs=2357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:32.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.889+0000 7fa8f8e00640 1 -- 172.21.15.39:0/2954278548 >> 172.21.15.39:0/2954278548 conn(0x7fa8f406cfc0 msgr2=0x7fa8f415f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:32.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.890+0000 7fa8f8e00640 1 -- 172.21.15.39:0/2954278548 shutdown_connections 2024-09-21T07:50:32.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:32.890+0000 7fa8f8e00640 1 -- 172.21.15.39:0/2954278548 wait complete. 2024-09-21T07:50:33.316 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 261, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:50:32.379926+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 240, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 9967, 'mds_1': 11052, 'mds_2': 8531}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8531': {'gid': 8531, 'name': 'd', 'rank': 2, 'incarnation': 253, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.148:6837/1438851059', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6835', 'nonce': 1438851059}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 1438851059}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_9967': {'gid': 9967, 'name': 'b', 'rank': 0, 'incarnation': 259, 'state': 'up:resolve', 'state_seq': 3, 'addr': '172.21.15.148:6836/3381818843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 3381818843}, {'type': 'v1', 'addr': '172.21.15.148:6836', 'nonce': 3381818843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11052': {'gid': 11052, 'name': 'c', 'rank': 1, 'incarnation': 261, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.39:6835/485194245', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 485194245}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 485194245}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8531, 'qdb_cluster': [8531]} 2024-09-21T07:50:33.316 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9967:up:resolve 2024-09-21T07:50:33.408 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:50:33.413 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:50:34.317 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:34.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.556+0000 7f71dfe00640 1 -- 172.21.15.39:0/620592032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71e006eef0 msgr2=0x7f71e00cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:34.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.556+0000 7f71dfe00640 1 --2- 172.21.15.39:0/620592032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71e006eef0 0x7f71e00cd200 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7f71d8005fd0 tx=0x7f71d802aa10 comp rx=0 tx=0).stop 2024-09-21T07:50:34.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.556+0000 7f71dfe00640 1 -- 172.21.15.39:0/620592032 shutdown_connections 2024-09-21T07:50:34.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.556+0000 7f71dfe00640 1 --2- 172.21.15.39:0/620592032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71e006eef0 0x7f71e00cd200 unknown :-1 s=CLOSED pgs=2359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:34.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.556+0000 7f71dfe00640 1 --2- 172.21.15.39:0/620592032 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f71e00b7a50 0x7f71e00b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:34.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.556+0000 7f71dfe00640 1 -- 172.21.15.39:0/620592032 >> 172.21.15.39:0/620592032 conn(0x7f71e009b5f0 msgr2=0x7f71e008c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:34.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.556+0000 7f71dfe00640 1 -- 172.21.15.39:0/620592032 shutdown_connections 2024-09-21T07:50:34.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.556+0000 7f71dfe00640 1 -- 172.21.15.39:0/620592032 wait complete. 2024-09-21T07:50:34.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.558+0000 7f71dfe00640 1 Processor -- start 2024-09-21T07:50:34.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.558+0000 7f71dfe00640 1 -- start start 2024-09-21T07:50:34.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.560+0000 7f71dfe00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f71e006eef0 0x7f71e0158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:34.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.560+0000 7f71dfe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71e00b2880 0x7f71e015e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:34.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.560+0000 7f71dfe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71e00b7a50 0x7f71e015c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:34.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.560+0000 7f71dfe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f71e00cddd0 con 0x7f71e00b7a50 2024-09-21T07:50:34.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.560+0000 7f71dfe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f71e00cdc50 con 0x7f71e00b2880 2024-09-21T07:50:34.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.560+0000 7f71dfe00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f71e00cdf50 con 0x7f71e006eef0 2024-09-21T07:50:34.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.560+0000 7f71df400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71e00b7a50 0x7f71e015c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:34.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.560+0000 7f71df400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71e00b7a50 0x7f71e015c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50586/0 (socket says 172.21.15.39:50586) 2024-09-21T07:50:34.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.560+0000 7f71df400640 1 -- 172.21.15.39:0/1685363068 learned_addr learned my addr 172.21.15.39:0/1685363068 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:34.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.560+0000 7f71dea00640 1 --2- 172.21.15.39:0/1685363068 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f71e006eef0 0x7f71e0158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:34.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.560+0000 7f71d7e00640 1 --2- 172.21.15.39:0/1685363068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71e00b2880 0x7f71e015e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:34.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.560+0000 7f71df400640 1 -- 172.21.15.39:0/1685363068 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f71e006eef0 msgr2=0x7f71e0158680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:34.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.560+0000 7f71df400640 1 --2- 172.21.15.39:0/1685363068 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f71e006eef0 0x7f71e0158680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:34.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.560+0000 7f71df400640 1 -- 172.21.15.39:0/1685363068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71e00b2880 msgr2=0x7f71e015e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:34.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.560+0000 7f71df400640 1 --2- 172.21.15.39:0/1685363068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71e00b2880 0x7f71e015e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:34.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.560+0000 7f71df400640 1 -- 172.21.15.39:0/1685363068 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e009f7d0 con 0x7f71e00b7a50 2024-09-21T07:50:34.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.560+0000 7f71dea00640 1 --2- 172.21.15.39:0/1685363068 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f71e006eef0 0x7f71e0158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:50:34.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.561+0000 7f71df400640 1 --2- 172.21.15.39:0/1685363068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71e00b7a50 0x7f71e015c330 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f71d8009d90 tx=0x7f71d8003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:34.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.561+0000 7f71d6a00640 1 -- 172.21.15.39:0/1685363068 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8033050 con 0x7f71e00b7a50 2024-09-21T07:50:34.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.561+0000 7f71d6a00640 1 -- 172.21.15.39:0/1685363068 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71d8004450 con 0x7f71e00b7a50 2024-09-21T07:50:34.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.561+0000 7f71d6a00640 1 -- 172.21.15.39:0/1685363068 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8038450 con 0x7f71e00b7a50 2024-09-21T07:50:34.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.561+0000 7f71dfe00640 1 -- 172.21.15.39:0/1685363068 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71e0158bc0 con 0x7f71e00b7a50 2024-09-21T07:50:34.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.561+0000 7f71dfe00640 1 -- 172.21.15.39:0/1685363068 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71e009fa90 con 0x7f71e00b7a50 2024-09-21T07:50:34.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.562+0000 7f71dfe00640 1 -- 172.21.15.39:0/1685363068 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71e00ce400 con 0x7f71e00b7a50 2024-09-21T07:50:34.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.564+0000 7f71d6a00640 1 -- 172.21.15.39:0/1685363068 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f71d8036070 con 0x7f71e00b7a50 2024-09-21T07:50:34.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.565+0000 7f71d6a00640 1 --2- 172.21.15.39:0/1685363068 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f71bc078b90 0x7f71bc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:34.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.565+0000 7f71d6a00640 1 -- 172.21.15.39:0/1685363068 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(240..240 src has 1..240) ==== 7390+0+0 (secure 0 0 0) 0x7f71d80bc970 con 0x7f71e00b7a50 2024-09-21T07:50:34.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.565+0000 7f71dea00640 1 --2- 172.21.15.39:0/1685363068 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f71bc078b90 0x7f71bc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:34.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.566+0000 7f71dea00640 1 --2- 172.21.15.39:0/1685363068 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f71bc078b90 0x7f71bc07b0a0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f71d0005fd0 tx=0x7f71d0005ee0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:34.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.569+0000 7f71d6a00640 1 -- 172.21.15.39:0/1685363068 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f71d8084680 con 0x7f71e00b7a50 2024-09-21T07:50:34.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.805+0000 7f71dfe00640 1 -- 172.21.15.39:0/1685363068 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71e00ce7f0 con 0x7f71e00b7a50 2024-09-21T07:50:34.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.805+0000 7f71d6a00640 1 -- 172.21.15.39:0/1685363068 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 263 v263) ==== 95+0+5274 (secure 0 0 0) 0x7f71d8083f80 con 0x7f71e00b7a50 2024-09-21T07:50:34.807 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:50:34.808 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":263,"btime":"2024-09-21T07:50:34:384426+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11067,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/109229385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":109229385},{"type":"v1","addr":"172.21.15.39:6834","nonce":109229385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":262}],"filesystems":[{"mdsmap":{"epoch":263,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:50:33.388911+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":240,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":8531},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8531":{"gid":8531,"name":"d","rank":2,"incarnation":253,"state":"up:active","state_seq":18,"addr":"172.21.15.148:6837/1438851059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1438851059},{"type":"v1","addr":"172.21.15.148:6837","nonce":1438851059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:reconnect","state_seq":4,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:reconnect","state_seq":3,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8531,"qdb_cluster":[8531]},"id":13}]} 2024-09-21T07:50:34.808 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 263 2024-09-21T07:50:34.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.808+0000 7f71dfe00640 1 -- 172.21.15.39:0/1685363068 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f71bc078b90 msgr2=0x7f71bc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:34.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.808+0000 7f71dfe00640 1 --2- 172.21.15.39:0/1685363068 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f71bc078b90 0x7f71bc07b0a0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f71d0005fd0 tx=0x7f71d0005ee0 comp rx=0 tx=0).stop 2024-09-21T07:50:34.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.808+0000 7f71dfe00640 1 -- 172.21.15.39:0/1685363068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71e00b7a50 msgr2=0x7f71e015c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:34.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.808+0000 7f71dfe00640 1 --2- 172.21.15.39:0/1685363068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71e00b7a50 0x7f71e015c330 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f71d8009d90 tx=0x7f71d8003f10 comp rx=0 tx=0).stop 2024-09-21T07:50:34.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.808+0000 7f71dfe00640 1 -- 172.21.15.39:0/1685363068 shutdown_connections 2024-09-21T07:50:34.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.808+0000 7f71dfe00640 1 --2- 172.21.15.39:0/1685363068 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f71bc078b90 0x7f71bc07b0a0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:34.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.808+0000 7f71dfe00640 1 --2- 172.21.15.39:0/1685363068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71e00b7a50 0x7f71e015c330 unknown :-1 s=CLOSED pgs=2360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:34.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.808+0000 7f71dfe00640 1 --2- 172.21.15.39:0/1685363068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71e00b2880 0x7f71e015e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:34.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.808+0000 7f71dfe00640 1 --2- 172.21.15.39:0/1685363068 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f71e006eef0 0x7f71e0158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:34.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.808+0000 7f71dfe00640 1 -- 172.21.15.39:0/1685363068 >> 172.21.15.39:0/1685363068 conn(0x7f71e009b5f0 msgr2=0x7f71e00b06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:34.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.808+0000 7f71dfe00640 1 -- 172.21.15.39:0/1685363068 shutdown_connections 2024-09-21T07:50:34.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:34.808+0000 7f71dfe00640 1 -- 172.21.15.39:0/1685363068 wait complete. 2024-09-21T07:50:35.234 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 263, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:50:33.388911+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 240, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 9967, 'mds_1': 11052, 'mds_2': 8531}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8531': {'gid': 8531, 'name': 'd', 'rank': 2, 'incarnation': 253, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.148:6837/1438851059', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6835', 'nonce': 1438851059}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 1438851059}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_9967': {'gid': 9967, 'name': 'b', 'rank': 0, 'incarnation': 259, 'state': 'up:reconnect', 'state_seq': 4, 'addr': '172.21.15.148:6836/3381818843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 3381818843}, {'type': 'v1', 'addr': '172.21.15.148:6836', 'nonce': 3381818843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11052': {'gid': 11052, 'name': 'c', 'rank': 1, 'incarnation': 261, 'state': 'up:reconnect', 'state_seq': 3, 'addr': '172.21.15.39:6835/485194245', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 485194245}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 485194245}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8531, 'qdb_cluster': [8531]} 2024-09-21T07:50:35.234 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9967:up:reconnect 2024-09-21T07:50:36.235 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:36.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.477+0000 7fb91b000640 1 -- 172.21.15.39:0/1573813735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9140ce2b0 msgr2=0x7fb91406f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:36.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.477+0000 7fb91b000640 1 --2- 172.21.15.39:0/1573813735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9140ce2b0 0x7fb91406f6a0 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7fb90c005fd0 tx=0x7fb90c02a940 comp rx=0 tx=0).stop 2024-09-21T07:50:36.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.477+0000 7fb91b000640 1 -- 172.21.15.39:0/1573813735 shutdown_connections 2024-09-21T07:50:36.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.477+0000 7fb91b000640 1 --2- 172.21.15.39:0/1573813735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9140ce2b0 0x7fb91406f6a0 unknown :-1 s=CLOSED pgs=2362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:36.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.477+0000 7fb91b000640 1 -- 172.21.15.39:0/1573813735 >> 172.21.15.39:0/1573813735 conn(0x7fb9140b2460 msgr2=0x7fb91406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:36.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.477+0000 7fb91b000640 1 -- 172.21.15.39:0/1573813735 shutdown_connections 2024-09-21T07:50:36.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.477+0000 7fb91b000640 1 -- 172.21.15.39:0/1573813735 wait complete. 2024-09-21T07:50:36.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.479+0000 7fb91b000640 1 Processor -- start 2024-09-21T07:50:36.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.479+0000 7fb91b000640 1 -- start start 2024-09-21T07:50:36.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.481+0000 7fb91b000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9140b4c40 0x7fb91414dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:36.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.481+0000 7fb91b000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9140ce2b0 0x7fb91414e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:36.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.481+0000 7fb91b000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb91415e080 0x7fb91415e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:36.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.481+0000 7fb91b000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb914070da0 con 0x7fb9140b4c40 2024-09-21T07:50:36.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.481+0000 7fb91b000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb914070c20 con 0x7fb9140ce2b0 2024-09-21T07:50:36.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.481+0000 7fb91b000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb91406fbe0 con 0x7fb91415e080 2024-09-21T07:50:36.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.481+0000 7fb913e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9140b4c40 0x7fb91414dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:36.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.481+0000 7fb913e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9140b4c40 0x7fb91414dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50596/0 (socket says 172.21.15.39:50596) 2024-09-21T07:50:36.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.481+0000 7fb913e00640 1 -- 172.21.15.39:0/1008469925 learned_addr learned my addr 172.21.15.39:0/1008469925 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:36.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.481+0000 7fb90be00640 1 --2- 172.21.15.39:0/1008469925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9140ce2b0 0x7fb91414e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:36.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.481+0000 7fb919000640 1 --2- 172.21.15.39:0/1008469925 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb91415e080 0x7fb91415e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:36.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.482+0000 7fb913e00640 1 -- 172.21.15.39:0/1008469925 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb91415e080 msgr2=0x7fb91415e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:36.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.482+0000 7fb913e00640 1 --2- 172.21.15.39:0/1008469925 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb91415e080 0x7fb91415e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:36.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.482+0000 7fb913e00640 1 -- 172.21.15.39:0/1008469925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9140ce2b0 msgr2=0x7fb91414e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:36.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.482+0000 7fb913e00640 1 --2- 172.21.15.39:0/1008469925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9140ce2b0 0x7fb91414e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:36.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.482+0000 7fb913e00640 1 -- 172.21.15.39:0/1008469925 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9141f6b30 con 0x7fb9140b4c40 2024-09-21T07:50:36.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.482+0000 7fb913e00640 1 --2- 172.21.15.39:0/1008469925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9140b4c40 0x7fb91414dd80 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7fb90400c370 tx=0x7fb90400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:36.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.482+0000 7fb910c00640 1 -- 172.21.15.39:0/1008469925 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb904017070 con 0x7fb9140b4c40 2024-09-21T07:50:36.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.482+0000 7fb910c00640 1 -- 172.21.15.39:0/1008469925 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9040035f0 con 0x7fb9140b4c40 2024-09-21T07:50:36.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.482+0000 7fb910c00640 1 -- 172.21.15.39:0/1008469925 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb904003a40 con 0x7fb9140b4c40 2024-09-21T07:50:36.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.482+0000 7fb91b000640 1 -- 172.21.15.39:0/1008469925 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb91415d5a0 con 0x7fb9140b4c40 2024-09-21T07:50:36.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.482+0000 7fb91b000640 1 -- 172.21.15.39:0/1008469925 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb91415db60 con 0x7fb9140b4c40 2024-09-21T07:50:36.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.483+0000 7fb91b000640 1 -- 172.21.15.39:0/1008469925 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9140b3130 con 0x7fb9140b4c40 2024-09-21T07:50:36.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.485+0000 7fb910c00640 1 -- 172.21.15.39:0/1008469925 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb90401a070 con 0x7fb9140b4c40 2024-09-21T07:50:36.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.486+0000 7fb910c00640 1 --2- 172.21.15.39:0/1008469925 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb8f0078b90 0x7fb8f007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:36.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.486+0000 7fb910c00640 1 -- 172.21.15.39:0/1008469925 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(240..240 src has 1..240) ==== 7390+0+0 (secure 0 0 0) 0x7fb90409a620 con 0x7fb9140b4c40 2024-09-21T07:50:36.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.486+0000 7fb90be00640 1 --2- 172.21.15.39:0/1008469925 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb8f0078b90 0x7fb8f007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:36.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.487+0000 7fb90be00640 1 --2- 172.21.15.39:0/1008469925 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb8f0078b90 0x7fb8f007b0a0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fb914156d80 tx=0x7fb8fc009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:36.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.490+0000 7fb910c00640 1 -- 172.21.15.39:0/1008469925 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb9140ce6c0 con 0x7fb9140b4c40 2024-09-21T07:50:36.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.729+0000 7fb91b000640 1 -- 172.21.15.39:0/1008469925 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb91406f800 con 0x7fb9140b4c40 2024-09-21T07:50:36.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.729+0000 7fb910c00640 1 -- 172.21.15.39:0/1008469925 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 265 v265) ==== 95+0+5279 (secure 0 0 0) 0x7fb904061d70 con 0x7fb9140b4c40 2024-09-21T07:50:36.731 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:50:36.732 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":265,"btime":"2024-09-21T07:50:36:415072+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11067,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/109229385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":109229385},{"type":"v1","addr":"172.21.15.39:6834","nonce":109229385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":262}],"filesystems":[{"mdsmap":{"epoch":265,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:50:36.415066+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":240,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":8531},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8531":{"gid":8531,"name":"d","rank":2,"incarnation":253,"state":"up:active","state_seq":18,"addr":"172.21.15.148:6837/1438851059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1438851059},{"type":"v1","addr":"172.21.15.148:6837","nonce":1438851059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[8531,9967,11052]},"id":13}]} 2024-09-21T07:50:36.732 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 265 2024-09-21T07:50:36.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.732+0000 7fb91b000640 1 -- 172.21.15.39:0/1008469925 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb8f0078b90 msgr2=0x7fb8f007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:36.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.732+0000 7fb91b000640 1 --2- 172.21.15.39:0/1008469925 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb8f0078b90 0x7fb8f007b0a0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fb914156d80 tx=0x7fb8fc009040 comp rx=0 tx=0).stop 2024-09-21T07:50:36.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.732+0000 7fb91b000640 1 -- 172.21.15.39:0/1008469925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9140b4c40 msgr2=0x7fb91414dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:36.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.732+0000 7fb91b000640 1 --2- 172.21.15.39:0/1008469925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9140b4c40 0x7fb91414dd80 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7fb90400c370 tx=0x7fb90400c830 comp rx=0 tx=0).stop 2024-09-21T07:50:36.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.732+0000 7fb91b000640 1 -- 172.21.15.39:0/1008469925 shutdown_connections 2024-09-21T07:50:36.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.732+0000 7fb91b000640 1 --2- 172.21.15.39:0/1008469925 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb8f0078b90 0x7fb8f007b0a0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:36.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.732+0000 7fb91b000640 1 --2- 172.21.15.39:0/1008469925 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb91415e080 0x7fb91415e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:36.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.732+0000 7fb91b000640 1 --2- 172.21.15.39:0/1008469925 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9140ce2b0 0x7fb91414e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:36.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.732+0000 7fb91b000640 1 --2- 172.21.15.39:0/1008469925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9140b4c40 0x7fb91414dd80 unknown :-1 s=CLOSED pgs=2363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:36.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.732+0000 7fb91b000640 1 -- 172.21.15.39:0/1008469925 >> 172.21.15.39:0/1008469925 conn(0x7fb9140b2460 msgr2=0x7fb9140ad060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:36.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.732+0000 7fb91b000640 1 -- 172.21.15.39:0/1008469925 shutdown_connections 2024-09-21T07:50:36.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:36.732+0000 7fb91b000640 1 -- 172.21.15.39:0/1008469925 wait complete. 2024-09-21T07:50:37.163 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 265, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:50:36.415066+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 240, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 9967, 'mds_1': 11052, 'mds_2': 8531}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8531': {'gid': 8531, 'name': 'd', 'rank': 2, 'incarnation': 253, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.148:6837/1438851059', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6835', 'nonce': 1438851059}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 1438851059}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_9967': {'gid': 9967, 'name': 'b', 'rank': 0, 'incarnation': 259, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.148:6836/3381818843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 3381818843}, {'type': 'v1', 'addr': '172.21.15.148:6836', 'nonce': 3381818843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11052': {'gid': 11052, 'name': 'c', 'rank': 1, 'incarnation': 261, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6835/485194245', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 485194245}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 485194245}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9967, 'qdb_cluster': [8531, 9967, 11052]} 2024-09-21T07:50:37.164 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-21T07:50:37.164 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-21T07:50:37.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.391+0000 7f2d9b200640 1 -- 172.21.15.39:0/964359091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d940cfc50 msgr2=0x7f2d940d0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:37.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.391+0000 7f2d9b200640 1 --2- 172.21.15.39:0/964359091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d940cfc50 0x7f2d940d0390 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f2d90005fd0 tx=0x7f2d9002a8a0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.391+0000 7f2d9b200640 1 -- 172.21.15.39:0/964359091 shutdown_connections 2024-09-21T07:50:37.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.391+0000 7f2d9b200640 1 --2- 172.21.15.39:0/964359091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d940cfc50 0x7f2d940d0390 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.391+0000 7f2d9b200640 1 --2- 172.21.15.39:0/964359091 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d94160660 0x7f2d940cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.391+0000 7f2d9b200640 1 -- 172.21.15.39:0/964359091 >> 172.21.15.39:0/964359091 conn(0x7f2d9406cfc0 msgr2=0x7f2d9406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:37.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.391+0000 7f2d9b200640 1 -- 172.21.15.39:0/964359091 shutdown_connections 2024-09-21T07:50:37.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.391+0000 7f2d9b200640 1 -- 172.21.15.39:0/964359091 wait complete. 2024-09-21T07:50:37.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.393+0000 7f2d9b200640 1 Processor -- start 2024-09-21T07:50:37.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.393+0000 7f2d9b200640 1 -- start start 2024-09-21T07:50:37.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.395+0000 7f2d9b200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d940cfc50 0x7f2d9415c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.395+0000 7f2d9b200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d94160250 0x7f2d94155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.395+0000 7f2d9b200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d94160660 0x7f2d94157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.395+0000 7f2d9b200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2d940cce00 con 0x7f2d94160250 2024-09-21T07:50:37.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.395+0000 7f2d9b200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2d940ccc80 con 0x7f2d94160660 2024-09-21T07:50:37.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.395+0000 7f2d9b200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f2d940ccf80 con 0x7f2d940cfc50 2024-09-21T07:50:37.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.395+0000 7f2d99400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d94160250 0x7f2d94155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.395+0000 7f2d99400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d94160250 0x7f2d94155860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50598/0 (socket says 172.21.15.39:50598) 2024-09-21T07:50:37.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.395+0000 7f2d99400640 1 -- 172.21.15.39:0/4056033449 learned_addr learned my addr 172.21.15.39:0/4056033449 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:37.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.395+0000 7f2d9a800640 1 --2- 172.21.15.39:0/4056033449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d94160660 0x7f2d94157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.395+0000 7f2d99e00640 1 --2- 172.21.15.39:0/4056033449 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d940cfc50 0x7f2d9415c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.395+0000 7f2d99400640 1 -- 172.21.15.39:0/4056033449 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d940cfc50 msgr2=0x7f2d9415c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:37.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.395+0000 7f2d99400640 1 --2- 172.21.15.39:0/4056033449 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d940cfc50 0x7f2d9415c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.395+0000 7f2d99400640 1 -- 172.21.15.39:0/4056033449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d94160660 msgr2=0x7f2d94157ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:37.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.395+0000 7f2d99400640 1 --2- 172.21.15.39:0/4056033449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d94160660 0x7f2d94157ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.395+0000 7f2d99400640 1 -- 172.21.15.39:0/4056033449 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d9415ad20 con 0x7f2d94160250 2024-09-21T07:50:37.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.396+0000 7f2d99400640 1 --2- 172.21.15.39:0/4056033449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d94160250 0x7f2d94155860 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7f2d8400e9f0 tx=0x7f2d8400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:37.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.396+0000 7f2d8e000640 1 -- 172.21.15.39:0/4056033449 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8400cd20 con 0x7f2d94160250 2024-09-21T07:50:37.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.396+0000 7f2d8e000640 1 -- 172.21.15.39:0/4056033449 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d84004020 con 0x7f2d94160250 2024-09-21T07:50:37.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.396+0000 7f2d9b200640 1 -- 172.21.15.39:0/4056033449 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d9415b250 con 0x7f2d94160250 2024-09-21T07:50:37.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.396+0000 7f2d8e000640 1 -- 172.21.15.39:0/4056033449 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d84004470 con 0x7f2d94160250 2024-09-21T07:50:37.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.396+0000 7f2d9b200640 1 -- 172.21.15.39:0/4056033449 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d9415b810 con 0x7f2d94160250 2024-09-21T07:50:37.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.399+0000 7f2d8e000640 1 -- 172.21.15.39:0/4056033449 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2d840035f0 con 0x7f2d94160250 2024-09-21T07:50:37.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.400+0000 7f2d8e000640 1 --2- 172.21.15.39:0/4056033449 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2d78078b90 0x7f2d7807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.400+0000 7f2d8e000640 1 -- 172.21.15.39:0/4056033449 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(240..240 src has 1..240) ==== 7390+0+0 (secure 0 0 0) 0x7f2d8409bb60 con 0x7f2d94160250 2024-09-21T07:50:37.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.400+0000 7f2da39f0740 1 Processor -- start 2024-09-21T07:50:37.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.400+0000 7f2d99e00640 1 --2- 172.21.15.39:0/4056033449 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2d78078b90 0x7f2d7807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.400+0000 7f2da39f0740 1 -- start start 2024-09-21T07:50:37.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.401+0000 7f2d99e00640 1 --2- 172.21.15.39:0/4056033449 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2d78078b90 0x7f2d7807b0a0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f2d88001720 tx=0x7f2d88007e90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:37.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.402+0000 7f2da39f0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56189952b270 0x5618998fff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.402+0000 7f2da39f0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5618995023b0 0x5618999004b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.402+0000 7f2da39f0740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56189950a590 0x5618998d0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.402+0000 7f2da39f0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561899855430 con 0x56189952b270 2024-09-21T07:50:37.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.402+0000 7f2da39f0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5618998455d0 con 0x5618995023b0 2024-09-21T07:50:37.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.402+0000 7f2da39f0740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x561899834a00 con 0x56189950a590 2024-09-21T07:50:37.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.402+0000 7f2d99e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56189952b270 0x5618998fff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.402+0000 7f2d99e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56189952b270 0x5618998fff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50608/0 (socket says 172.21.15.39:50608) 2024-09-21T07:50:37.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.402+0000 7f2d99e00640 1 -- 172.21.15.39:0/2129490357 learned_addr learned my addr 172.21.15.39:0/2129490357 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:37.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.402+0000 7f2d9a800640 1 --2- 172.21.15.39:0/2129490357 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56189950a590 0x5618998d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.402+0000 7f2d99400640 1 --2- 172.21.15.39:0/2129490357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5618995023b0 0x5618999004b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.403+0000 7f2d99e00640 1 -- 172.21.15.39:0/2129490357 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56189950a590 msgr2=0x5618998d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:37.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.403+0000 7f2d99e00640 1 --2- 172.21.15.39:0/2129490357 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56189950a590 0x5618998d0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.403+0000 7f2d99e00640 1 -- 172.21.15.39:0/2129490357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5618995023b0 msgr2=0x5618999004b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:37.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.403+0000 7f2d99e00640 1 --2- 172.21.15.39:0/2129490357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5618995023b0 0x5618999004b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.403+0000 7f2d99e00640 1 -- 172.21.15.39:0/2129490357 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618997fd520 con 0x56189952b270 2024-09-21T07:50:37.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.403+0000 7f2d9a800640 1 --2- 172.21.15.39:0/2129490357 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56189950a590 0x5618998d0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:50:37.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.403+0000 7f2d99e00640 1 --2- 172.21.15.39:0/2129490357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56189952b270 0x5618998fff70 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800fb20 tx=0x7f2d88012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa37c41cddae56 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:37.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.403+0000 7f2d82600640 1 -- 172.21.15.39:0/2129490357 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8801b050 con 0x56189952b270 2024-09-21T07:50:37.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.403+0000 7f2d82600640 1 -- 172.21.15.39:0/2129490357 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d88025070 con 0x56189952b270 2024-09-21T07:50:37.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.403+0000 7f2d82600640 1 -- 172.21.15.39:0/2129490357 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d88020440 con 0x56189952b270 2024-09-21T07:50:37.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.404+0000 7f2da39f0740 1 -- 172.21.15.39:0/2129490357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56189952b270 msgr2=0x5618998fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:37.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.404+0000 7f2da39f0740 1 --2- 172.21.15.39:0/2129490357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56189952b270 0x5618998fff70 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800fb20 tx=0x7f2d88012000 comp rx=0 tx=0).stop 2024-09-21T07:50:37.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.404+0000 7f2da39f0740 1 -- 172.21.15.39:0/2129490357 shutdown_connections 2024-09-21T07:50:37.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.404+0000 7f2da39f0740 1 --2- 172.21.15.39:0/2129490357 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56189950a590 0x5618998d0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.404+0000 7f2da39f0740 1 --2- 172.21.15.39:0/2129490357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5618995023b0 0x5618999004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.404+0000 7f2da39f0740 1 --2- 172.21.15.39:0/2129490357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56189952b270 0x5618998fff70 unknown :-1 s=CLOSED pgs=2365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.404+0000 7f2da39f0740 1 -- 172.21.15.39:0/2129490357 >> 172.21.15.39:0/2129490357 conn(0x56189951f3c0 msgr2=0x5618998d5f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:37.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.404+0000 7f2da39f0740 1 -- 172.21.15.39:0/2129490357 shutdown_connections 2024-09-21T07:50:37.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.404+0000 7f2da39f0740 1 -- 172.21.15.39:0/2129490357 wait complete. 2024-09-21T07:50:37.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.405+0000 7f2da39f0740 1 -- start start 2024-09-21T07:50:37.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.406+0000 7f2da39f0740 1 Processor -- start 2024-09-21T07:50:37.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.407+0000 7f2da39f0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5618995023b0 0x5618998fbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.407+0000 7f2da39f0740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56189950a590 0x5618998fce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.407+0000 7f2da39f0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56189952b270 0x5618998f6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.407+0000 7f2da39f0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5618997db330 con 0x56189952b270 2024-09-21T07:50:37.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.407+0000 7f2da39f0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561899838810 con 0x5618995023b0 2024-09-21T07:50:37.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.407+0000 7f2da39f0740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x5618995a5620 con 0x56189950a590 2024-09-21T07:50:37.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.407+0000 7f2da39f0740 10 client.0 _ll_register_callbacks cb 0x5618997e0420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:50:37.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.408+0000 7f2d9a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56189952b270 0x5618998f6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.408+0000 7f2d99e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5618995023b0 0x5618998fbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.408+0000 7f2d99e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5618995023b0 0x5618998fbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.39:47602/0 (socket says 172.21.15.39:47602) 2024-09-21T07:50:37.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.408+0000 7f2d99e00640 1 -- 172.21.15.39:0/3723149998 learned_addr learned my addr 172.21.15.39:0/3723149998 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:37.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.408+0000 7f2d9a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56189952b270 0x5618998f6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50610/0 (socket says 172.21.15.39:50610) 2024-09-21T07:50:37.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.408+0000 7f2d99400640 1 --2- 172.21.15.39:0/3723149998 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56189950a590 0x5618998fce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.408+0000 7f2d99e00640 1 -- 172.21.15.39:0/3723149998 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56189950a590 msgr2=0x5618998fce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:37.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.408+0000 7f2d99e00640 1 --2- 172.21.15.39:0/3723149998 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56189950a590 0x5618998fce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.408+0000 7f2d99e00640 1 -- 172.21.15.39:0/3723149998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56189952b270 msgr2=0x5618998f6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:37.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.408+0000 7f2d99e00640 1 --2- 172.21.15.39:0/3723149998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56189952b270 0x5618998f6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.408+0000 7f2d99e00640 1 -- 172.21.15.39:0/3723149998 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561899596850 con 0x5618995023b0 2024-09-21T07:50:37.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.408+0000 7f2d9a800640 1 --2- 172.21.15.39:0/3723149998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56189952b270 0x5618998f6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:50:37.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.409+0000 7f2d99e00640 1 --2- 172.21.15.39:0/3723149998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5618995023b0 0x5618998fbdd0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800f5a0 tx=0x7f2d88020b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e95764b72bd08362 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:37.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.409+0000 7f2d76a00640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-21T07:50:37.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.409+0000 7f2d76a00640 1 -- 172.21.15.39:0/3723149998 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d88034410 con 0x5618995023b0 2024-09-21T07:50:37.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.409+0000 7f2da39f0740 1 -- 172.21.15.39:0/3723149998 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d88025070 con 0x5618995023b0 2024-09-21T07:50:37.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.409+0000 7f2d76a00640 1 -- 172.21.15.39:0/3723149998 <== mon.1 v2:172.21.15.148:3300/0 2 ==== osd_map(240..240 src has 1..240) ==== 7390+0+0 (secure 0 0 0) 0x7f2d8800f720 con 0x5618995023b0 2024-09-21T07:50:37.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.409+0000 7f2d76a00640 1 -- 172.21.15.39:0/3723149998 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=265) ==== 24+0+0 (secure 0 0 0) 0x7f2d8801b020 con 0x5618995023b0 2024-09-21T07:50:37.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.409+0000 7f2da39f0740 10 client.10099 fetch_fsmap learned FSMap version 265 2024-09-21T07:50:37.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.409+0000 7f2da39f0740 1 -- 172.21.15.39:0/3723149998 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=265,monmap=2+}) -- 0x56189960d6b0 con 0x5618995023b0 2024-09-21T07:50:37.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.409+0000 7f2d76a00640 1 -- 172.21.15.39:0/3723149998 <== mon.1 v2:172.21.15.148:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d88025070 con 0x5618995023b0 2024-09-21T07:50:37.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.410+0000 7f2d76a00640 1 -- 172.21.15.39:0/3723149998 <== mon.1 v2:172.21.15.148:3300/0 5 ==== fsmap(e 265) ==== 3542+0+0 (secure 0 0 0) 0x7f2d880345b0 con 0x5618995023b0 2024-09-21T07:50:37.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.410+0000 7f2da39f0740 10 client.10099 fetch_fsmap finished waiting for FSMap version 265 2024-09-21T07:50:37.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.410+0000 7f2da39f0740 10 client.10099 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T07:50:37.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.410+0000 7f2da39f0740 20 client.10099 populate_metadata read hostname 'smithi039' 2024-09-21T07:50:37.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.410+0000 7f2da39f0740 1 --2- 172.21.15.39:0/3723149998 >> [v2:172.21.15.148:6835/1438851059,v1:172.21.15.148:6837/1438851059] conn(0x5618994e9e40 0x5618998f8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.410+0000 7f2da39f0740 4 client.10099 mds_command: new command op to 8531 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:50:37.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.410+0000 7f2da39f0740 1 -- 172.21.15.39:0/3723149998 --> [v2:172.21.15.148:6835/1438851059,v1:172.21.15.148:6837/1438851059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561899847520 con 0x5618994e9e40 2024-09-21T07:50:37.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.410+0000 7f2d99400640 1 --2- 172.21.15.39:0/3723149998 >> [v2:172.21.15.148:6835/1438851059,v1:172.21.15.148:6837/1438851059] conn(0x5618994e9e40 0x5618998f8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.411+0000 7f2d99400640 1 --2- 172.21.15.39:0/3723149998 >> [v2:172.21.15.148:6835/1438851059,v1:172.21.15.148:6837/1438851059] conn(0x5618994e9e40 0x5618998f8110 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=47617cea851b6b1a server_cookie=355179709ab26602 in_seq=0 out_seq=0 2024-09-21T07:50:37.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.411+0000 7f2d76a00640 10 client.10099 ms_handle_connect on v2:172.21.15.148:6835/1438851059 2024-09-21T07:50:37.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.412+0000 7f2d76a00640 1 -- 172.21.15.39:0/3723149998 <== mds.2 v2:172.21.15.148:6835/1438851059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561899847520 con 0x5618994e9e40 2024-09-21T07:50:37.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.412+0000 7f2d76a00640 10 client.10099 handle_command_reply: tid=0 2024-09-21T07:50:37.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.412+0000 7f2da39f0740 1 client.10099 shutdown 2024-09-21T07:50:37.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.412+0000 7f2da39f0740 1 -- 172.21.15.39:0/3723149998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5618995023b0 msgr2=0x5618998fbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:37.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.412+0000 7f2da39f0740 1 --2- 172.21.15.39:0/3723149998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5618995023b0 0x5618998fbdd0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800f5a0 tx=0x7f2d88020b00 comp rx=0 tx=0).stop 2024-09-21T07:50:37.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.412+0000 7f2da39f0740 1 -- 172.21.15.39:0/3723149998 shutdown_connections 2024-09-21T07:50:37.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.412+0000 7f2da39f0740 1 --2- 172.21.15.39:0/3723149998 >> [v2:172.21.15.148:6835/1438851059,v1:172.21.15.148:6837/1438851059] conn(0x5618994e9e40 0x5618998f8110 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.412+0000 7f2da39f0740 1 --2- 172.21.15.39:0/3723149998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56189952b270 0x5618998f6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.412+0000 7f2d76a00640 0 client.10099 ms_handle_reset on v2:172.21.15.148:6835/1438851059 2024-09-21T07:50:37.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.412+0000 7f2da39f0740 1 --2- 172.21.15.39:0/3723149998 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56189950a590 0x5618998fce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.412+0000 7f2da39f0740 1 --2- 172.21.15.39:0/3723149998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5618995023b0 0x5618998fbdd0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.412+0000 7f2da39f0740 1 -- 172.21.15.39:0/3723149998 >> 172.21.15.39:0/3723149998 conn(0x56189951f3c0 msgr2=0x561899911370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:37.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.412+0000 7f2da39f0740 1 -- 172.21.15.39:0/3723149998 shutdown_connections 2024-09-21T07:50:37.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.413+0000 7f2da39f0740 1 -- 172.21.15.39:0/3723149998 wait complete. 2024-09-21T07:50:37.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.413+0000 7f2da39f0740 20 client.10099 trim_cache size 0 max 16384 2024-09-21T07:50:37.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.433+0000 7f2da39f0740 1 Processor -- start 2024-09-21T07:50:37.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.433+0000 7f2da39f0740 1 -- start start 2024-09-21T07:50:37.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.434+0000 7f2da39f0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618995023b0 0x56189991e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.434+0000 7f2da39f0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56189950a590 0x56189991e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.434+0000 7f2da39f0740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56189952b270 0x56189993dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.434+0000 7f2da39f0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2d941ffd80 con 0x5618995023b0 2024-09-21T07:50:37.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.434+0000 7f2da39f0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2d94200450 con 0x56189950a590 2024-09-21T07:50:37.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.434+0000 7f2da39f0740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f2d940b5170 con 0x56189952b270 2024-09-21T07:50:37.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.434+0000 7f2d99e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618995023b0 0x56189991e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.434+0000 7f2d99e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618995023b0 0x56189991e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50616/0 (socket says 172.21.15.39:50616) 2024-09-21T07:50:37.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.434+0000 7f2d99e00640 1 -- 172.21.15.39:0/1711127923 learned_addr learned my addr 172.21.15.39:0/1711127923 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:37.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.435+0000 7f2d99400640 1 --2- 172.21.15.39:0/1711127923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56189950a590 0x56189991e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.435+0000 7f2d9a800640 1 --2- 172.21.15.39:0/1711127923 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56189952b270 0x56189993dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.435+0000 7f2d99e00640 1 -- 172.21.15.39:0/1711127923 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56189952b270 msgr2=0x56189993dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:37.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.435+0000 7f2d99e00640 1 --2- 172.21.15.39:0/1711127923 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56189952b270 0x56189993dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.435+0000 7f2d99e00640 1 -- 172.21.15.39:0/1711127923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56189950a590 msgr2=0x56189991e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:37.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.435+0000 7f2d99e00640 1 --2- 172.21.15.39:0/1711127923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56189950a590 0x56189991e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.435+0000 7f2d99e00640 1 -- 172.21.15.39:0/1711127923 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618998d6c50 con 0x5618995023b0 2024-09-21T07:50:37.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.435+0000 7f2d99400640 1 --2- 172.21.15.39:0/1711127923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56189950a590 0x56189991e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:50:37.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.435+0000 7f2d9a800640 1 --2- 172.21.15.39:0/1711127923 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56189952b270 0x56189993dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:50:37.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.435+0000 7f2d99e00640 1 --2- 172.21.15.39:0/1711127923 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618995023b0 0x56189991e000 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7f2d88007920 tx=0x7f2d88034a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6591d5387c453c43 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:37.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.435+0000 7f2d81c00640 1 -- 172.21.15.39:0/1711127923 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8800f720 con 0x5618995023b0 2024-09-21T07:50:37.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.435+0000 7f2d81c00640 1 -- 172.21.15.39:0/1711127923 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d88025040 con 0x5618995023b0 2024-09-21T07:50:37.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.435+0000 7f2d81c00640 1 -- 172.21.15.39:0/1711127923 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d880343f0 con 0x5618995023b0 2024-09-21T07:50:37.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.436+0000 7f2da39f0740 1 -- 172.21.15.39:0/1711127923 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618995023b0 msgr2=0x56189991e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:37.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.436+0000 7f2da39f0740 1 --2- 172.21.15.39:0/1711127923 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618995023b0 0x56189991e000 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7f2d88007920 tx=0x7f2d88034a60 comp rx=0 tx=0).stop 2024-09-21T07:50:37.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.436+0000 7f2da39f0740 1 -- 172.21.15.39:0/1711127923 shutdown_connections 2024-09-21T07:50:37.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.436+0000 7f2da39f0740 1 --2- 172.21.15.39:0/1711127923 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56189952b270 0x56189993dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.436+0000 7f2da39f0740 1 --2- 172.21.15.39:0/1711127923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56189950a590 0x56189991e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.436+0000 7f2da39f0740 1 --2- 172.21.15.39:0/1711127923 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618995023b0 0x56189991e000 unknown :-1 s=CLOSED pgs=2366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.436+0000 7f2da39f0740 1 -- 172.21.15.39:0/1711127923 >> 172.21.15.39:0/1711127923 conn(0x56189951f3c0 msgr2=0x56189991b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:37.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.436+0000 7f2da39f0740 1 -- 172.21.15.39:0/1711127923 shutdown_connections 2024-09-21T07:50:37.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.436+0000 7f2da39f0740 1 -- 172.21.15.39:0/1711127923 wait complete. 2024-09-21T07:50:37.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.437+0000 7f2da39f0740 1 -- start start 2024-09-21T07:50:37.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.437+0000 7f2da39f0740 1 Processor -- start 2024-09-21T07:50:37.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.439+0000 7f2da39f0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5618995023b0 0x56189999b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.439+0000 7f2da39f0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56189950a590 0x56189991e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.439+0000 7f2da39f0740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56189952b270 0x56189991e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.439+0000 7f2da39f0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56189992e040 con 0x56189950a590 2024-09-21T07:50:37.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.439+0000 7f2da39f0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5618998fd550 con 0x5618995023b0 2024-09-21T07:50:37.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.439+0000 7f2da39f0740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x561899914bf0 con 0x56189952b270 2024-09-21T07:50:37.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.439+0000 7f2da39f0740 10 client.0 _ll_register_callbacks cb 0x7f2d8400cec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:50:37.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.439+0000 7f2d99400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56189950a590 0x56189991e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.439+0000 7f2d99400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56189950a590 0x56189991e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50628/0 (socket says 172.21.15.39:50628) 2024-09-21T07:50:37.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.439+0000 7f2d99400640 1 -- 172.21.15.39:0/2713720255 learned_addr learned my addr 172.21.15.39:0/2713720255 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:37.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.439+0000 7f2d9a800640 1 --2- 172.21.15.39:0/2713720255 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56189952b270 0x56189991e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.439+0000 7f2d99e00640 1 --2- 172.21.15.39:0/2713720255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5618995023b0 0x56189999b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.440+0000 7f2d99400640 1 -- 172.21.15.39:0/2713720255 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56189952b270 msgr2=0x56189991e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:37.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.440+0000 7f2d99400640 1 --2- 172.21.15.39:0/2713720255 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56189952b270 0x56189991e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.440+0000 7f2d99400640 1 -- 172.21.15.39:0/2713720255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5618995023b0 msgr2=0x56189999b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:37.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.440+0000 7f2d99400640 1 --2- 172.21.15.39:0/2713720255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5618995023b0 0x56189999b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.440+0000 7f2d99400640 1 -- 172.21.15.39:0/2713720255 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56189999b610 con 0x56189950a590 2024-09-21T07:50:37.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.440+0000 7f2d99400640 1 --2- 172.21.15.39:0/2713720255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56189950a590 0x56189991e000 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7f2d84060520 tx=0x7f2d8406a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b723a80a30f517 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:37.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.440+0000 7f2d76a00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:50:37.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.440+0000 7f2d76a00640 1 -- 172.21.15.39:0/2713720255 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8405f520 con 0x56189950a590 2024-09-21T07:50:37.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.440+0000 7f2d76a00640 1 -- 172.21.15.39:0/2713720255 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(240..240 src has 1..240) ==== 7390+0+0 (secure 0 0 0) 0x7f2d84087db0 con 0x56189950a590 2024-09-21T07:50:37.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.440+0000 7f2da39f0740 1 -- 172.21.15.39:0/2713720255 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d88025040 con 0x56189950a590 2024-09-21T07:50:37.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.440+0000 7f2d76a00640 1 -- 172.21.15.39:0/2713720255 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=265) ==== 24+0+0 (secure 0 0 0) 0x7f2d8405f6c0 con 0x56189950a590 2024-09-21T07:50:37.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.440+0000 7f2da39f0740 10 client.11106 fetch_fsmap learned FSMap version 265 2024-09-21T07:50:37.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.440+0000 7f2da39f0740 1 -- 172.21.15.39:0/2713720255 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=265,monmap=2+}) -- 0x561899930880 con 0x56189950a590 2024-09-21T07:50:37.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.441+0000 7f2d76a00640 1 -- 172.21.15.39:0/2713720255 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d88025040 con 0x56189950a590 2024-09-21T07:50:37.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.441+0000 7f2d76a00640 1 -- 172.21.15.39:0/2713720255 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 265) ==== 3542+0+0 (secure 0 0 0) 0x7f2d8406c700 con 0x56189950a590 2024-09-21T07:50:37.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.441+0000 7f2da39f0740 10 client.11106 fetch_fsmap finished waiting for FSMap version 265 2024-09-21T07:50:37.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.441+0000 7f2da39f0740 10 client.11106 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T07:50:37.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.441+0000 7f2da39f0740 20 client.11106 populate_metadata read hostname 'smithi039' 2024-09-21T07:50:37.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.441+0000 7f2da39f0740 1 --2- 172.21.15.39:0/2713720255 >> [v2:172.21.15.148:6835/1438851059,v1:172.21.15.148:6837/1438851059] conn(0x5618998f8ad0 0x5618998d06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.441+0000 7f2da39f0740 4 client.11106 mds_command: new command op to 8531 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:50:37.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.441+0000 7f2da39f0740 1 -- 172.21.15.39:0/2713720255 --> [v2:172.21.15.148:6835/1438851059,v1:172.21.15.148:6837/1438851059] -- command(tid 0: {"prefix": "status"}) -- 0x56189992d580 con 0x5618998f8ad0 2024-09-21T07:50:37.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.441+0000 7f2d99e00640 1 --2- 172.21.15.39:0/2713720255 >> [v2:172.21.15.148:6835/1438851059,v1:172.21.15.148:6837/1438851059] conn(0x5618998f8ad0 0x5618998d06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.442+0000 7f2d99e00640 1 --2- 172.21.15.39:0/2713720255 >> [v2:172.21.15.148:6835/1438851059,v1:172.21.15.148:6837/1438851059] conn(0x5618998f8ad0 0x5618998d06d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=13222b2d83f918ee server_cookie=381450c51b34e4c4 in_seq=0 out_seq=0 2024-09-21T07:50:37.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.442+0000 7f2d76a00640 10 client.11106 ms_handle_connect on v2:172.21.15.148:6835/1438851059 2024-09-21T07:50:37.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.442+0000 7f2d76a00640 1 -- 172.21.15.39:0/2713720255 <== mds.2 v2:172.21.15.148:6835/1438851059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x56189992d580 con 0x5618998f8ad0 2024-09-21T07:50:37.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.442+0000 7f2d76a00640 10 client.11106 handle_command_reply: tid=0 2024-09-21T07:50:37.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.442+0000 7f2da39f0740 1 client.11106 shutdown 2024-09-21T07:50:37.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.442+0000 7f2da39f0740 1 -- 172.21.15.39:0/2713720255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56189950a590 msgr2=0x56189991e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:37.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.442+0000 7f2da39f0740 1 --2- 172.21.15.39:0/2713720255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56189950a590 0x56189991e000 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7f2d84060520 tx=0x7f2d8406a040 comp rx=0 tx=0).stop 2024-09-21T07:50:37.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.442+0000 7f2da39f0740 1 -- 172.21.15.39:0/2713720255 shutdown_connections 2024-09-21T07:50:37.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.442+0000 7f2da39f0740 1 --2- 172.21.15.39:0/2713720255 >> [v2:172.21.15.148:6835/1438851059,v1:172.21.15.148:6837/1438851059] conn(0x5618998f8ad0 0x5618998d06d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.442+0000 7f2da39f0740 1 --2- 172.21.15.39:0/2713720255 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56189952b270 0x56189991e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.442+0000 7f2da39f0740 1 --2- 172.21.15.39:0/2713720255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56189950a590 0x56189991e000 unknown :-1 s=CLOSED pgs=2367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.442+0000 7f2da39f0740 1 --2- 172.21.15.39:0/2713720255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5618995023b0 0x56189999b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.442+0000 7f2d76a00640 0 client.11106 ms_handle_reset on v2:172.21.15.148:6835/1438851059 2024-09-21T07:50:37.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.442+0000 7f2da39f0740 1 -- 172.21.15.39:0/2713720255 >> 172.21.15.39:0/2713720255 conn(0x56189951f3c0 msgr2=0x5618998d5f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:37.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.443+0000 7f2da39f0740 1 -- 172.21.15.39:0/2713720255 shutdown_connections 2024-09-21T07:50:37.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.443+0000 7f2da39f0740 1 -- 172.21.15.39:0/2713720255 wait complete. 2024-09-21T07:50:37.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.443+0000 7f2da39f0740 20 client.11106 trim_cache size 0 max 16384 2024-09-21T07:50:37.448 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:50:37.448 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:50:37.448 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 2, 2024-09-21T07:50:37.448 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 8531, 2024-09-21T07:50:37.448 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:50:37.448 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:50:37.449 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:50:37.449 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 12.575836938, 2024-09-21T07:50:37.449 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 265, 2024-09-21T07:50:37.449 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 240, 2024-09-21T07:50:37.449 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 240, 2024-09-21T07:50:37.449 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 67.331778716000002 2024-09-21T07:50:37.449 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:50:37.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.443+0000 7f2d9b200640 1 -- 172.21.15.39:0/4056033449 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2d78078b90 msgr2=0x7f2d7807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:37.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.443+0000 7f2d9b200640 1 --2- 172.21.15.39:0/4056033449 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2d78078b90 0x7f2d7807b0a0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f2d88001720 tx=0x7f2d88007e90 comp rx=0 tx=0).stop 2024-09-21T07:50:37.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.443+0000 7f2d9b200640 1 -- 172.21.15.39:0/4056033449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d94160250 msgr2=0x7f2d94155860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:37.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.443+0000 7f2d9b200640 1 --2- 172.21.15.39:0/4056033449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d94160250 0x7f2d94155860 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7f2d8400e9f0 tx=0x7f2d8400eeb0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.443+0000 7f2d9b200640 1 -- 172.21.15.39:0/4056033449 shutdown_connections 2024-09-21T07:50:37.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.443+0000 7f2d9b200640 1 --2- 172.21.15.39:0/4056033449 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2d78078b90 0x7f2d7807b0a0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.443+0000 7f2d9b200640 1 --2- 172.21.15.39:0/4056033449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d94160660 0x7f2d94157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.443+0000 7f2d9b200640 1 --2- 172.21.15.39:0/4056033449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d94160250 0x7f2d94155860 unknown :-1 s=CLOSED pgs=2364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.443+0000 7f2d9b200640 1 --2- 172.21.15.39:0/4056033449 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2d940cfc50 0x7f2d9415c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.443+0000 7f2d9b200640 1 -- 172.21.15.39:0/4056033449 >> 172.21.15.39:0/4056033449 conn(0x7f2d9406cfc0 msgr2=0x7f2d9406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:37.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.443+0000 7f2d9b200640 1 -- 172.21.15.39:0/4056033449 shutdown_connections 2024-09-21T07:50:37.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.444+0000 7f2d9b200640 1 -- 172.21.15.39:0/4056033449 wait complete. 2024-09-21T07:50:37.466 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-21T07:50:37.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.694+0000 7f478fe00640 1 -- 172.21.15.39:0/2766345982 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4790162b70 msgr2=0x7f479006f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:37.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.694+0000 7f478fe00640 1 --2- 172.21.15.39:0/2766345982 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4790162b70 0x7f479006f8d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4778009830 tx=0x7f477802a4a0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.694+0000 7f478fe00640 1 -- 172.21.15.39:0/2766345982 shutdown_connections 2024-09-21T07:50:37.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.694+0000 7f478fe00640 1 --2- 172.21.15.39:0/2766345982 >> v2:172.21.15.148:3300/0 conn(0x7f479006fe10 0x7f479006d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.694+0000 7f478fe00640 1 --2- 172.21.15.39:0/2766345982 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4790162b70 0x7f479006f8d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.694+0000 7f478fe00640 1 -- 172.21.15.39:0/2766345982 >> 172.21.15.39:0/2766345982 conn(0x7f47900d0c70 msgr2=0x7f47900d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:37.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.695+0000 7f478fe00640 1 -- 172.21.15.39:0/2766345982 shutdown_connections 2024-09-21T07:50:37.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.695+0000 7f478fe00640 1 -- 172.21.15.39:0/2766345982 wait complete. 2024-09-21T07:50:37.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.697+0000 7f478fe00640 1 Processor -- start 2024-09-21T07:50:37.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.697+0000 7f478fe00640 1 -- start start 2024-09-21T07:50:37.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.698+0000 7f478fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f479006fe10 0x7f4790158ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.698+0000 7f478fe00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47900b4cc0 0x7f47900ce990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.698+0000 7f478fe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47901f6880 0x7f47901f6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.698+0000 7f478fe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f479006dcb0 con 0x7f479006fe10 2024-09-21T07:50:37.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.698+0000 7f478fe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f479006db30 con 0x7f47901f6880 2024-09-21T07:50:37.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.698+0000 7f478fe00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f479006de30 con 0x7f47900b4cc0 2024-09-21T07:50:37.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.699+0000 7f478ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f479006fe10 0x7f4790158ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.699+0000 7f478ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f479006fe10 0x7f4790158ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50630/0 (socket says 172.21.15.39:50630) 2024-09-21T07:50:37.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.699+0000 7f478ea00640 1 -- 172.21.15.39:0/974036478 learned_addr learned my addr 172.21.15.39:0/974036478 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:37.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.699+0000 7f478f400640 1 --2- 172.21.15.39:0/974036478 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47901f6880 0x7f47901f6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.699+0000 7f4786000640 1 --2- 172.21.15.39:0/974036478 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47900b4cc0 0x7f47900ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.699+0000 7f478ea00640 1 -- 172.21.15.39:0/974036478 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47900b4cc0 msgr2=0x7f47900ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:37.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.699+0000 7f478ea00640 1 --2- 172.21.15.39:0/974036478 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47900b4cc0 0x7f47900ce990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.699+0000 7f478ea00640 1 -- 172.21.15.39:0/974036478 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47901f6880 msgr2=0x7f47901f6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:37.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.699+0000 7f478ea00640 1 --2- 172.21.15.39:0/974036478 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47901f6880 0x7f47901f6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.699+0000 7f478ea00640 1 -- 172.21.15.39:0/974036478 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47900ceed0 con 0x7f479006fe10 2024-09-21T07:50:37.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.699+0000 7f478f400640 1 --2- 172.21.15.39:0/974036478 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47901f6880 0x7f47901f6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:50:37.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.699+0000 7f4786000640 1 --2- 172.21.15.39:0/974036478 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47900b4cc0 0x7f47900ce990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:50:37.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.699+0000 7f478ea00640 1 --2- 172.21.15.39:0/974036478 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f479006fe10 0x7f4790158ab0 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7f478000c500 tx=0x7f478000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:37.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.700+0000 7f4787400640 1 -- 172.21.15.39:0/974036478 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4780011790 con 0x7f479006fe10 2024-09-21T07:50:37.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.700+0000 7f4787400640 1 -- 172.21.15.39:0/974036478 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4780017070 con 0x7f479006fe10 2024-09-21T07:50:37.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.700+0000 7f4787400640 1 -- 172.21.15.39:0/974036478 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4780011ce0 con 0x7f479006fe10 2024-09-21T07:50:37.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.700+0000 7f478fe00640 1 -- 172.21.15.39:0/974036478 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47900cf180 con 0x7f479006fe10 2024-09-21T07:50:37.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.700+0000 7f478fe00640 1 -- 172.21.15.39:0/974036478 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47900cf740 con 0x7f479006fe10 2024-09-21T07:50:37.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.702+0000 7f4787400640 1 -- 172.21.15.39:0/974036478 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f478001c460 con 0x7f479006fe10 2024-09-21T07:50:37.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.703+0000 7f4787400640 1 --2- 172.21.15.39:0/974036478 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f476c078b90 0x7f476c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.703+0000 7f4787400640 1 -- 172.21.15.39:0/974036478 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(240..240 src has 1..240) ==== 7390+0+0 (secure 0 0 0) 0x7f478009b2c0 con 0x7f479006fe10 2024-09-21T07:50:37.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.703+0000 7f479c658740 1 Processor -- start 2024-09-21T07:50:37.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.704+0000 7f479c658740 1 -- start start 2024-09-21T07:50:37.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.704+0000 7f4786000640 1 --2- 172.21.15.39:0/974036478 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f476c078b90 0x7f476c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.705+0000 7f4786000640 1 --2- 172.21.15.39:0/974036478 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f476c078b90 0x7f476c07b0a0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f4778034000 tx=0x7f4778004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:37.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.705+0000 7f479c658740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56335bcfd270 0x56335c0cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.705+0000 7f479c658740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335bcd43b0 0x56335c0ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.705+0000 7f479c658740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56335bcdc590 0x56335c09e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.705+0000 7f479c658740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56335c027430 con 0x56335bcdc590 2024-09-21T07:50:37.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.705+0000 7f479c658740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56335c0175d0 con 0x56335bcd43b0 2024-09-21T07:50:37.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.705+0000 7f479c658740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x56335c006a00 con 0x56335bcfd270 2024-09-21T07:50:37.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.705+0000 7f478f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56335bcdc590 0x56335c09e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.705+0000 7f478f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56335bcdc590 0x56335c09e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50642/0 (socket says 172.21.15.39:50642) 2024-09-21T07:50:37.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.705+0000 7f478f400640 1 -- 172.21.15.39:0/1899934289 learned_addr learned my addr 172.21.15.39:0/1899934289 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:37.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.706+0000 7f478ea00640 1 --2- 172.21.15.39:0/1899934289 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56335bcfd270 0x56335c0cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.706+0000 7f4786000640 1 --2- 172.21.15.39:0/1899934289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335bcd43b0 0x56335c0ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.706+0000 7f478f400640 1 -- 172.21.15.39:0/1899934289 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56335bcfd270 msgr2=0x56335c0cdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:37.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.706+0000 7f478f400640 1 --2- 172.21.15.39:0/1899934289 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56335bcfd270 0x56335c0cdb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.706+0000 7f478f400640 1 -- 172.21.15.39:0/1899934289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335bcd43b0 msgr2=0x56335c0ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:37.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.706+0000 7f478f400640 1 --2- 172.21.15.39:0/1899934289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335bcd43b0 0x56335c0ce0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.706+0000 7f478f400640 1 -- 172.21.15.39:0/1899934289 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56335bfcf520 con 0x56335bcdc590 2024-09-21T07:50:37.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.706+0000 7f478f400640 1 --2- 172.21.15.39:0/1899934289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56335bcdc590 0x56335c09e150 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7f4788009e80 tx=0x7f478800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43f195f243c5ca9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:37.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.707+0000 7f477f000640 1 -- 172.21.15.39:0/1899934289 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4788011640 con 0x56335bcdc590 2024-09-21T07:50:37.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.707+0000 7f477f000640 1 -- 172.21.15.39:0/1899934289 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47880118d0 con 0x56335bcdc590 2024-09-21T07:50:37.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.707+0000 7f477f000640 1 -- 172.21.15.39:0/1899934289 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4788011c80 con 0x56335bcdc590 2024-09-21T07:50:37.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.707+0000 7f479c658740 1 -- 172.21.15.39:0/1899934289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56335bcdc590 msgr2=0x56335c09e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:37.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.707+0000 7f479c658740 1 --2- 172.21.15.39:0/1899934289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56335bcdc590 0x56335c09e150 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7f4788009e80 tx=0x7f478800f730 comp rx=0 tx=0).stop 2024-09-21T07:50:37.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.707+0000 7f479c658740 1 -- 172.21.15.39:0/1899934289 shutdown_connections 2024-09-21T07:50:37.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.707+0000 7f479c658740 1 --2- 172.21.15.39:0/1899934289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56335bcdc590 0x56335c09e150 unknown :-1 s=CLOSED pgs=2370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.707+0000 7f479c658740 1 --2- 172.21.15.39:0/1899934289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335bcd43b0 0x56335c0ce0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.707+0000 7f479c658740 1 --2- 172.21.15.39:0/1899934289 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56335bcfd270 0x56335c0cdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.707+0000 7f479c658740 1 -- 172.21.15.39:0/1899934289 >> 172.21.15.39:0/1899934289 conn(0x56335bcf13c0 msgr2=0x56335c0a3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:37.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.708+0000 7f479c658740 1 -- 172.21.15.39:0/1899934289 shutdown_connections 2024-09-21T07:50:37.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.708+0000 7f479c658740 1 -- 172.21.15.39:0/1899934289 wait complete. 2024-09-21T07:50:37.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.709+0000 7f479c658740 1 -- start start 2024-09-21T07:50:37.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.709+0000 7f479c658740 1 Processor -- start 2024-09-21T07:50:37.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.710+0000 7f479c658740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56335bcd43b0 0x56335c0e3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.710+0000 7f479c658740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335bcdc590 0x56335c0c3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.710+0000 7f479c658740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56335bcfd270 0x56335c0fafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.710+0000 7f479c658740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56335bfad330 con 0x56335bcd43b0 2024-09-21T07:50:37.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.710+0000 7f479c658740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56335c00a810 con 0x56335bcdc590 2024-09-21T07:50:37.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.710+0000 7f479c658740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x56335bd77620 con 0x56335bcfd270 2024-09-21T07:50:37.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.710+0000 7f479c658740 10 client.0 _ll_register_callbacks cb 0x56335bfb2420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:50:37.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.711+0000 7f478ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56335bcd43b0 0x56335c0e3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.711+0000 7f478ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56335bcd43b0 0x56335c0e3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50646/0 (socket says 172.21.15.39:50646) 2024-09-21T07:50:37.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.711+0000 7f478ea00640 1 -- 172.21.15.39:0/547944512 learned_addr learned my addr 172.21.15.39:0/547944512 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:37.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.711+0000 7f4786000640 1 --2- 172.21.15.39:0/547944512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335bcdc590 0x56335c0c3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.711+0000 7f478f400640 1 --2- 172.21.15.39:0/547944512 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56335bcfd270 0x56335c0fafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.711+0000 7f478ea00640 1 -- 172.21.15.39:0/547944512 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56335bcfd270 msgr2=0x56335c0fafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:37.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.711+0000 7f478ea00640 1 --2- 172.21.15.39:0/547944512 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56335bcfd270 0x56335c0fafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.711+0000 7f478ea00640 1 -- 172.21.15.39:0/547944512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335bcdc590 msgr2=0x56335c0c3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:37.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.711+0000 7f478ea00640 1 --2- 172.21.15.39:0/547944512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335bcdc590 0x56335c0c3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.711+0000 7f478ea00640 1 -- 172.21.15.39:0/547944512 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56335bd68850 con 0x56335bcd43b0 2024-09-21T07:50:37.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.711+0000 7f478ea00640 1 --2- 172.21.15.39:0/547944512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56335bcd43b0 0x56335c0e3890 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7f4780099150 tx=0x7f478005ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a98ae17d13b050f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:37.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.711+0000 7f4773400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:50:37.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.712+0000 7f4773400640 1 -- 172.21.15.39:0/547944512 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4780061dd0 con 0x56335bcd43b0 2024-09-21T07:50:37.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.712+0000 7f4773400640 1 -- 172.21.15.39:0/547944512 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(240..240 src has 1..240) ==== 7390+0+0 (secure 0 0 0) 0x7f478005e2c0 con 0x56335bcd43b0 2024-09-21T07:50:37.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.712+0000 7f479c658740 1 -- 172.21.15.39:0/547944512 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47880118d0 con 0x56335bcd43b0 2024-09-21T07:50:37.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.712+0000 7f4773400640 1 -- 172.21.15.39:0/547944512 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=265) ==== 24+0+0 (secure 0 0 0) 0x7f4780062970 con 0x56335bcd43b0 2024-09-21T07:50:37.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.712+0000 7f479c658740 10 client.11118 fetch_fsmap learned FSMap version 265 2024-09-21T07:50:37.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.712+0000 7f479c658740 1 -- 172.21.15.39:0/547944512 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=265,monmap=2+}) -- 0x56335bddf6b0 con 0x56335bcd43b0 2024-09-21T07:50:37.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.712+0000 7f4773400640 1 -- 172.21.15.39:0/547944512 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47880118d0 con 0x56335bcd43b0 2024-09-21T07:50:37.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.712+0000 7f4773400640 1 -- 172.21.15.39:0/547944512 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 265) ==== 3542+0+0 (secure 0 0 0) 0x7f478005e4c0 con 0x56335bcd43b0 2024-09-21T07:50:37.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.712+0000 7f479c658740 10 client.11118 fetch_fsmap finished waiting for FSMap version 265 2024-09-21T07:50:37.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.712+0000 7f479c658740 10 client.11118 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T07:50:37.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.712+0000 7f479c658740 20 client.11118 populate_metadata read hostname 'smithi039' 2024-09-21T07:50:37.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.712+0000 7f479c658740 1 --2- 172.21.15.39:0/547944512 >> [v2:172.21.15.148:6834/3381818843,v1:172.21.15.148:6836/3381818843] conn(0x56335bcbbe40 0x56335c0f7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.712+0000 7f479c658740 4 client.11118 mds_command: new command op to 9967 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:50:37.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.712+0000 7f479c658740 1 -- 172.21.15.39:0/547944512 --> [v2:172.21.15.148:6834/3381818843,v1:172.21.15.148:6836/3381818843] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56335c019520 con 0x56335bcbbe40 2024-09-21T07:50:37.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.713+0000 7f4786000640 1 --2- 172.21.15.39:0/547944512 >> [v2:172.21.15.148:6834/3381818843,v1:172.21.15.148:6836/3381818843] conn(0x56335bcbbe40 0x56335c0f7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.713+0000 7f4786000640 1 --2- 172.21.15.39:0/547944512 >> [v2:172.21.15.148:6834/3381818843,v1:172.21.15.148:6836/3381818843] conn(0x56335bcbbe40 0x56335c0f7b70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45b9c067b527e289 server_cookie=d564693590e03557 in_seq=0 out_seq=0 2024-09-21T07:50:37.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.713+0000 7f4773400640 10 client.11118 ms_handle_connect on v2:172.21.15.148:6834/3381818843 2024-09-21T07:50:37.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.715+0000 7f4773400640 1 -- 172.21.15.39:0/547944512 <== mds.0 v2:172.21.15.148:6834/3381818843 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56335c019520 con 0x56335bcbbe40 2024-09-21T07:50:37.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.715+0000 7f4773400640 10 client.11118 handle_command_reply: tid=0 2024-09-21T07:50:37.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.715+0000 7f479c658740 1 client.11118 shutdown 2024-09-21T07:50:37.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.715+0000 7f479c658740 1 -- 172.21.15.39:0/547944512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56335bcd43b0 msgr2=0x56335c0e3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:37.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.715+0000 7f479c658740 1 --2- 172.21.15.39:0/547944512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56335bcd43b0 0x56335c0e3890 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7f4780099150 tx=0x7f478005ee90 comp rx=0 tx=0).stop 2024-09-21T07:50:37.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.715+0000 7f479c658740 1 -- 172.21.15.39:0/547944512 shutdown_connections 2024-09-21T07:50:37.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.715+0000 7f479c658740 1 --2- 172.21.15.39:0/547944512 >> [v2:172.21.15.148:6834/3381818843,v1:172.21.15.148:6836/3381818843] conn(0x56335bcbbe40 0x56335c0f7b70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.715+0000 7f479c658740 1 --2- 172.21.15.39:0/547944512 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56335bcfd270 0x56335c0fafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.715+0000 7f479c658740 1 --2- 172.21.15.39:0/547944512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335bcdc590 0x56335c0c3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.715+0000 7f479c658740 1 --2- 172.21.15.39:0/547944512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56335bcd43b0 0x56335c0e3890 unknown :-1 s=CLOSED pgs=2371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.715+0000 7f4773400640 0 client.11118 ms_handle_reset on v2:172.21.15.148:6834/3381818843 2024-09-21T07:50:37.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.715+0000 7f479c658740 1 -- 172.21.15.39:0/547944512 >> 172.21.15.39:0/547944512 conn(0x56335bcf13c0 msgr2=0x56335c0ebef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:37.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.715+0000 7f479c658740 1 -- 172.21.15.39:0/547944512 shutdown_connections 2024-09-21T07:50:37.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.715+0000 7f479c658740 1 -- 172.21.15.39:0/547944512 wait complete. 2024-09-21T07:50:37.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.715+0000 7f479c658740 20 client.11118 trim_cache size 0 max 16384 2024-09-21T07:50:37.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.735+0000 7f479c658740 1 Processor -- start 2024-09-21T07:50:37.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.735+0000 7f479c658740 1 -- start start 2024-09-21T07:50:37.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.736+0000 7f479c658740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56335bcd43b0 0x56335c0c3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.736+0000 7f479c658740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56335bcdc590 0x56335c0e72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.736+0000 7f479c658740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335bcfd270 0x56335c0e7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.736+0000 7f479c658740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4790204c80 con 0x56335bcd43b0 2024-09-21T07:50:37.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.736+0000 7f479c658740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4790203ee0 con 0x56335bcfd270 2024-09-21T07:50:37.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.736+0000 7f479c658740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f47902045b0 con 0x56335bcdc590 2024-09-21T07:50:37.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.737+0000 7f478ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56335bcd43b0 0x56335c0c3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.737+0000 7f478ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56335bcd43b0 0x56335c0c3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50650/0 (socket says 172.21.15.39:50650) 2024-09-21T07:50:37.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.737+0000 7f478ea00640 1 -- 172.21.15.39:0/1528774416 learned_addr learned my addr 172.21.15.39:0/1528774416 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:37.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.737+0000 7f4786000640 1 --2- 172.21.15.39:0/1528774416 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56335bcdc590 0x56335c0e72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.737+0000 7f478f400640 1 --2- 172.21.15.39:0/1528774416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335bcfd270 0x56335c0e7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.737+0000 7f478ea00640 1 -- 172.21.15.39:0/1528774416 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56335bcdc590 msgr2=0x56335c0e72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:37.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.737+0000 7f478ea00640 1 --2- 172.21.15.39:0/1528774416 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56335bcdc590 0x56335c0e72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.737+0000 7f478ea00640 1 -- 172.21.15.39:0/1528774416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335bcfd270 msgr2=0x56335c0e7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:37.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.737+0000 7f478ea00640 1 --2- 172.21.15.39:0/1528774416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335bcfd270 0x56335c0e7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.737+0000 7f478ea00640 1 -- 172.21.15.39:0/1528774416 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56335c0ce870 con 0x56335bcd43b0 2024-09-21T07:50:37.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.737+0000 7f478ea00640 1 --2- 172.21.15.39:0/1528774416 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56335bcd43b0 0x56335c0c3ae0 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7f47800622e0 tx=0x7f478006c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c542d6eeb9518745 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:37.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.737+0000 7f477e600640 1 -- 172.21.15.39:0/1528774416 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4780061dd0 con 0x56335bcd43b0 2024-09-21T07:50:37.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.738+0000 7f477e600640 1 -- 172.21.15.39:0/1528774416 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4780081020 con 0x56335bcd43b0 2024-09-21T07:50:37.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.738+0000 7f477e600640 1 -- 172.21.15.39:0/1528774416 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478000ed20 con 0x56335bcd43b0 2024-09-21T07:50:37.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.738+0000 7f479c658740 1 -- 172.21.15.39:0/1528774416 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56335bcd43b0 msgr2=0x56335c0c3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:37.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.738+0000 7f479c658740 1 --2- 172.21.15.39:0/1528774416 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56335bcd43b0 0x56335c0c3ae0 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7f47800622e0 tx=0x7f478006c300 comp rx=0 tx=0).stop 2024-09-21T07:50:37.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.738+0000 7f479c658740 1 -- 172.21.15.39:0/1528774416 shutdown_connections 2024-09-21T07:50:37.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.738+0000 7f479c658740 1 --2- 172.21.15.39:0/1528774416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335bcfd270 0x56335c0e7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.738+0000 7f479c658740 1 --2- 172.21.15.39:0/1528774416 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56335bcdc590 0x56335c0e72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.738+0000 7f479c658740 1 --2- 172.21.15.39:0/1528774416 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56335bcd43b0 0x56335c0c3ae0 unknown :-1 s=CLOSED pgs=2372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.738+0000 7f479c658740 1 -- 172.21.15.39:0/1528774416 >> 172.21.15.39:0/1528774416 conn(0x56335bcf13c0 msgr2=0x56335c099750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:37.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.738+0000 7f479c658740 1 -- 172.21.15.39:0/1528774416 shutdown_connections 2024-09-21T07:50:37.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.738+0000 7f479c658740 1 -- 172.21.15.39:0/1528774416 wait complete. 2024-09-21T07:50:37.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.739+0000 7f479c658740 1 -- start start 2024-09-21T07:50:37.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.739+0000 7f479c658740 1 Processor -- start 2024-09-21T07:50:37.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.741+0000 7f479c658740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56335bcd43b0 0x56335c167e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.741+0000 7f479c658740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56335bcdc590 0x56335c0c6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.741+0000 7f479c658740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335bcfd270 0x56335c0c68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.741+0000 7f479c658740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56335c0c78f0 con 0x56335bcdc590 2024-09-21T07:50:37.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.741+0000 7f479c658740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4790202ee0 con 0x56335bcfd270 2024-09-21T07:50:37.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.741+0000 7f479c658740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x56335c0c7ea0 con 0x56335bcd43b0 2024-09-21T07:50:37.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.741+0000 7f479c658740 10 client.0 _ll_register_callbacks cb 0x7f4780062170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:50:37.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.741+0000 7f4786000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56335bcdc590 0x56335c0c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.741+0000 7f4786000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56335bcdc590 0x56335c0c6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50662/0 (socket says 172.21.15.39:50662) 2024-09-21T07:50:37.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.741+0000 7f4786000640 1 -- 172.21.15.39:0/1404439528 learned_addr learned my addr 172.21.15.39:0/1404439528 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:37.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.741+0000 7f478f400640 1 --2- 172.21.15.39:0/1404439528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335bcfd270 0x56335c0c68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.741+0000 7f478ea00640 1 --2- 172.21.15.39:0/1404439528 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56335bcd43b0 0x56335c167e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.742+0000 7f4786000640 1 -- 172.21.15.39:0/1404439528 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56335bcd43b0 msgr2=0x56335c167e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:37.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.742+0000 7f4786000640 1 --2- 172.21.15.39:0/1404439528 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56335bcd43b0 0x56335c167e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.742+0000 7f4786000640 1 -- 172.21.15.39:0/1404439528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335bcfd270 msgr2=0x56335c0c68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:37.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.742+0000 7f4786000640 1 --2- 172.21.15.39:0/1404439528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335bcfd270 0x56335c0c68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.742+0000 7f4786000640 1 -- 172.21.15.39:0/1404439528 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56335c16c730 con 0x56335bcdc590 2024-09-21T07:50:37.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.742+0000 7f4786000640 1 --2- 172.21.15.39:0/1404439528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56335bcdc590 0x56335c0c6380 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7f4778004690 tx=0x7f477802a4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2b5875f2cc216ac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:37.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.742+0000 7f4773400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:50:37.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.742+0000 7f4773400640 1 -- 172.21.15.39:0/1404439528 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4778058060 con 0x56335bcdc590 2024-09-21T07:50:37.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.742+0000 7f4773400640 1 -- 172.21.15.39:0/1404439528 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(240..240 src has 1..240) ==== 7390+0+0 (secure 0 0 0) 0x7f4778049620 con 0x56335bcdc590 2024-09-21T07:50:37.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.742+0000 7f479c658740 1 -- 172.21.15.39:0/1404439528 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4780081020 con 0x56335bcdc590 2024-09-21T07:50:37.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.743+0000 7f4773400640 1 -- 172.21.15.39:0/1404439528 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=265) ==== 24+0+0 (secure 0 0 0) 0x7f477805e050 con 0x56335bcdc590 2024-09-21T07:50:37.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.743+0000 7f479c658740 10 client.11124 fetch_fsmap learned FSMap version 265 2024-09-21T07:50:37.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.743+0000 7f479c658740 1 -- 172.21.15.39:0/1404439528 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=265,monmap=2+}) -- 0x56335c0c6e60 con 0x56335bcdc590 2024-09-21T07:50:37.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.743+0000 7f4773400640 1 -- 172.21.15.39:0/1404439528 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4780081020 con 0x56335bcdc590 2024-09-21T07:50:37.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.743+0000 7f4773400640 1 -- 172.21.15.39:0/1404439528 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 265) ==== 3542+0+0 (secure 0 0 0) 0x7f4778061020 con 0x56335bcdc590 2024-09-21T07:50:37.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.743+0000 7f479c658740 10 client.11124 fetch_fsmap finished waiting for FSMap version 265 2024-09-21T07:50:37.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.743+0000 7f479c658740 10 client.11124 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T07:50:37.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.743+0000 7f479c658740 20 client.11124 populate_metadata read hostname 'smithi039' 2024-09-21T07:50:37.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.743+0000 7f479c658740 1 --2- 172.21.15.39:0/1404439528 >> [v2:172.21.15.148:6834/3381818843,v1:172.21.15.148:6836/3381818843] conn(0x56335c0fe210 0x56335c0fbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.743+0000 7f479c658740 4 client.11124 mds_command: new command op to 9967 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:50:37.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.743+0000 7f479c658740 1 -- 172.21.15.39:0/1404439528 --> [v2:172.21.15.148:6834/3381818843,v1:172.21.15.148:6836/3381818843] -- command(tid 0: {"prefix": "status"}) -- 0x56335c0c8390 con 0x56335c0fe210 2024-09-21T07:50:37.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.744+0000 7f478ea00640 1 --2- 172.21.15.39:0/1404439528 >> [v2:172.21.15.148:6834/3381818843,v1:172.21.15.148:6836/3381818843] conn(0x56335c0fe210 0x56335c0fbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.744+0000 7f478ea00640 1 --2- 172.21.15.39:0/1404439528 >> [v2:172.21.15.148:6834/3381818843,v1:172.21.15.148:6836/3381818843] conn(0x56335c0fe210 0x56335c0fbef0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de297886f0ad6065 server_cookie=a2df853575c99639 in_seq=0 out_seq=0 2024-09-21T07:50:37.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.744+0000 7f4773400640 10 client.11124 ms_handle_connect on v2:172.21.15.148:6834/3381818843 2024-09-21T07:50:37.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.744+0000 7f4773400640 1 -- 172.21.15.39:0/1404439528 <== mds.0 v2:172.21.15.148:6834/3381818843 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x56335c0c8390 con 0x56335c0fe210 2024-09-21T07:50:37.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.744+0000 7f4773400640 10 client.11124 handle_command_reply: tid=0 2024-09-21T07:50:37.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.744+0000 7f479c658740 1 client.11124 shutdown 2024-09-21T07:50:37.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.745+0000 7f479c658740 1 -- 172.21.15.39:0/1404439528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56335bcdc590 msgr2=0x56335c0c6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:37.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.745+0000 7f479c658740 1 --2- 172.21.15.39:0/1404439528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56335bcdc590 0x56335c0c6380 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7f4778004690 tx=0x7f477802a4a0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.745+0000 7f479c658740 1 -- 172.21.15.39:0/1404439528 shutdown_connections 2024-09-21T07:50:37.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.745+0000 7f479c658740 1 --2- 172.21.15.39:0/1404439528 >> [v2:172.21.15.148:6834/3381818843,v1:172.21.15.148:6836/3381818843] conn(0x56335c0fe210 0x56335c0fbef0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.745+0000 7f479c658740 1 --2- 172.21.15.39:0/1404439528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56335bcfd270 0x56335c0c68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.745+0000 7f479c658740 1 --2- 172.21.15.39:0/1404439528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56335bcdc590 0x56335c0c6380 unknown :-1 s=CLOSED pgs=2373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.745+0000 7f479c658740 1 --2- 172.21.15.39:0/1404439528 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56335bcd43b0 0x56335c167e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.745+0000 7f479c658740 1 -- 172.21.15.39:0/1404439528 >> 172.21.15.39:0/1404439528 conn(0x56335bcf13c0 msgr2=0x56335c0c4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:37.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.745+0000 7f4773400640 0 client.11124 ms_handle_reset on v2:172.21.15.148:6834/3381818843 2024-09-21T07:50:37.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.745+0000 7f479c658740 1 -- 172.21.15.39:0/1404439528 shutdown_connections 2024-09-21T07:50:37.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.745+0000 7f479c658740 1 -- 172.21.15.39:0/1404439528 wait complete. 2024-09-21T07:50:37.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.745+0000 7f479c658740 20 client.11124 trim_cache size 0 max 16384 2024-09-21T07:50:37.750 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:50:37.750 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:50:37.751 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-09-21T07:50:37.751 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 9967, 2024-09-21T07:50:37.751 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:50:37.751 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:50:37.751 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:50:37.751 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 5.9094182469999996, 2024-09-21T07:50:37.751 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 265, 2024-09-21T07:50:37.751 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 240, 2024-09-21T07:50:37.751 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 240, 2024-09-21T07:50:37.751 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 12.757983862 2024-09-21T07:50:37.751 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:50:37.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.746+0000 7f478fe00640 1 -- 172.21.15.39:0/974036478 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f476c078b90 msgr2=0x7f476c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:37.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.746+0000 7f478fe00640 1 --2- 172.21.15.39:0/974036478 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f476c078b90 0x7f476c07b0a0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f4778034000 tx=0x7f4778004730 comp rx=0 tx=0).stop 2024-09-21T07:50:37.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.746+0000 7f478fe00640 1 -- 172.21.15.39:0/974036478 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f479006fe10 msgr2=0x7f4790158ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:37.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.746+0000 7f478fe00640 1 --2- 172.21.15.39:0/974036478 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f479006fe10 0x7f4790158ab0 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7f478000c500 tx=0x7f478000c9c0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.746+0000 7f478fe00640 1 -- 172.21.15.39:0/974036478 shutdown_connections 2024-09-21T07:50:37.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.746+0000 7f478fe00640 1 --2- 172.21.15.39:0/974036478 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f476c078b90 0x7f476c07b0a0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.746+0000 7f478fe00640 1 --2- 172.21.15.39:0/974036478 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47901f6880 0x7f47901f6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.746+0000 7f478fe00640 1 --2- 172.21.15.39:0/974036478 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47900b4cc0 0x7f47900ce990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.746+0000 7f478fe00640 1 --2- 172.21.15.39:0/974036478 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f479006fe10 0x7f4790158ab0 unknown :-1 s=CLOSED pgs=2369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.746+0000 7f478fe00640 1 -- 172.21.15.39:0/974036478 >> 172.21.15.39:0/974036478 conn(0x7f47900d0c70 msgr2=0x7f47900afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:37.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.746+0000 7f478fe00640 1 -- 172.21.15.39:0/974036478 shutdown_connections 2024-09-21T07:50:37.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.746+0000 7f478fe00640 1 -- 172.21.15.39:0/974036478 wait complete. 2024-09-21T07:50:37.773 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T07:50:38.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.999+0000 7f1b2f200640 1 -- 172.21.15.39:0/3274830914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b300d04a0 msgr2=0x7f1b300d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:38.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.999+0000 7f1b2f200640 1 --2- 172.21.15.39:0/3274830914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b300d04a0 0x7f1b300d08b0 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7f1b18009830 tx=0x7f1b1802a510 comp rx=0 tx=0).stop 2024-09-21T07:50:38.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.999+0000 7f1b2f200640 1 -- 172.21.15.39:0/3274830914 shutdown_connections 2024-09-21T07:50:38.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.999+0000 7f1b2f200640 1 --2- 172.21.15.39:0/3274830914 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1b3006ee90 0x7f1b3006f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.999+0000 7f1b2f200640 1 --2- 172.21.15.39:0/3274830914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b300d04a0 0x7f1b300d08b0 unknown :-1 s=CLOSED pgs=2375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.999+0000 7f1b2f200640 1 -- 172.21.15.39:0/3274830914 >> 172.21.15.39:0/3274830914 conn(0x7f1b3006cfc0 msgr2=0x7f1b3006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:38.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.999+0000 7f1b2f200640 1 -- 172.21.15.39:0/3274830914 shutdown_connections 2024-09-21T07:50:38.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:37.999+0000 7f1b2f200640 1 -- 172.21.15.39:0/3274830914 wait complete. 2024-09-21T07:50:38.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.001+0000 7f1b2f200640 1 Processor -- start 2024-09-21T07:50:38.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.001+0000 7f1b2f200640 1 -- start start 2024-09-21T07:50:38.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.003+0000 7f1b2f200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1b3006ee90 0x7f1b3013cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:38.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.003+0000 7f1b2f200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b300d04a0 0x7f1b300cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:38.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.003+0000 7f1b2f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b301519d0 0x7f1b300ce230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:38.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.003+0000 7f1b2f200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1b300b3940 con 0x7f1b301519d0 2024-09-21T07:50:38.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.003+0000 7f1b2f200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1b300b3ac0 con 0x7f1b300d04a0 2024-09-21T07:50:38.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.003+0000 7f1b2f200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f1b300b3c40 con 0x7f1b3006ee90 2024-09-21T07:50:38.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.003+0000 7f1b2e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b301519d0 0x7f1b300ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:38.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.003+0000 7f1b2e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b301519d0 0x7f1b300ce230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50684/0 (socket says 172.21.15.39:50684) 2024-09-21T07:50:38.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.003+0000 7f1b2e800640 1 -- 172.21.15.39:0/1375268251 learned_addr learned my addr 172.21.15.39:0/1375268251 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:38.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.003+0000 7f1b27e00640 1 --2- 172.21.15.39:0/1375268251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b300d04a0 0x7f1b300cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:38.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.003+0000 7f1b2de00640 1 --2- 172.21.15.39:0/1375268251 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1b3006ee90 0x7f1b3013cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:38.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.004+0000 7f1b2e800640 1 -- 172.21.15.39:0/1375268251 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1b3006ee90 msgr2=0x7f1b3013cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:38.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.004+0000 7f1b2e800640 1 --2- 172.21.15.39:0/1375268251 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1b3006ee90 0x7f1b3013cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.004+0000 7f1b2e800640 1 -- 172.21.15.39:0/1375268251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b300d04a0 msgr2=0x7f1b300cdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:38.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.004+0000 7f1b2e800640 1 --2- 172.21.15.39:0/1375268251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b300d04a0 0x7f1b300cdcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.004+0000 7f1b2e800640 1 -- 172.21.15.39:0/1375268251 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b300ae500 con 0x7f1b301519d0 2024-09-21T07:50:38.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.004+0000 7f1b2de00640 1 --2- 172.21.15.39:0/1375268251 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1b3006ee90 0x7f1b3013cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:50:38.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.004+0000 7f1b2e800640 1 --2- 172.21.15.39:0/1375268251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b301519d0 0x7f1b300ce230 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800e990 tx=0x7f1b2800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:38.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.004+0000 7f1b26000640 1 -- 172.21.15.39:0/1375268251 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b28004d20 con 0x7f1b301519d0 2024-09-21T07:50:38.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.004+0000 7f1b26000640 1 -- 172.21.15.39:0/1375268251 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b2800cb00 con 0x7f1b301519d0 2024-09-21T07:50:38.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.004+0000 7f1b26000640 1 -- 172.21.15.39:0/1375268251 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b28005020 con 0x7f1b301519d0 2024-09-21T07:50:38.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.004+0000 7f1b2f200640 1 -- 172.21.15.39:0/1375268251 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b300ae7e0 con 0x7f1b301519d0 2024-09-21T07:50:38.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.004+0000 7f1b2f200640 1 -- 172.21.15.39:0/1375268251 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b300aeda0 con 0x7f1b301519d0 2024-09-21T07:50:38.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.007+0000 7f1b26000640 1 -- 172.21.15.39:0/1375268251 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1b280051c0 con 0x7f1b301519d0 2024-09-21T07:50:38.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.008+0000 7f1b26000640 1 --2- 172.21.15.39:0/1375268251 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1b0c078bc0 0x7f1b0c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:38.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.008+0000 7f1b26000640 1 -- 172.21.15.39:0/1375268251 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(240..240 src has 1..240) ==== 7390+0+0 (secure 0 0 0) 0x7f1b2809bca0 con 0x7f1b301519d0 2024-09-21T07:50:38.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.008+0000 7f1b3c1ec740 1 Processor -- start 2024-09-21T07:50:38.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.008+0000 7f1b2de00640 1 --2- 172.21.15.39:0/1375268251 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1b0c078bc0 0x7f1b0c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:38.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.008+0000 7f1b3c1ec740 1 -- start start 2024-09-21T07:50:38.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.009+0000 7f1b2de00640 1 --2- 172.21.15.39:0/1375268251 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1b0c078bc0 0x7f1b0c07b0d0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f1b200013f0 tx=0x7f1b20009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:38.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.010+0000 7f1b3c1ec740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce35fc270 0x562ce39ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:38.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.010+0000 7f1b3c1ec740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce35d33b0 0x562ce39cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:38.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.010+0000 7f1b3c1ec740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562ce35db590 0x562ce399d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:38.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.010+0000 7f1b3c1ec740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562ce3926430 con 0x562ce35fc270 2024-09-21T07:50:38.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.010+0000 7f1b3c1ec740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562ce39165d0 con 0x562ce35d33b0 2024-09-21T07:50:38.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.010+0000 7f1b3c1ec740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x562ce3905a00 con 0x562ce35db590 2024-09-21T07:50:38.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.010+0000 7f1b2de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce35fc270 0x562ce39ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:38.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.010+0000 7f1b2de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce35fc270 0x562ce39ccb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50698/0 (socket says 172.21.15.39:50698) 2024-09-21T07:50:38.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.010+0000 7f1b2de00640 1 -- 172.21.15.39:0/1856518147 learned_addr learned my addr 172.21.15.39:0/1856518147 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:38.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.010+0000 7f1b27e00640 1 --2- 172.21.15.39:0/1856518147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce35d33b0 0x562ce39cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:38.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.010+0000 7f1b2e800640 1 --2- 172.21.15.39:0/1856518147 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562ce35db590 0x562ce399d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:38.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.011+0000 7f1b2de00640 1 -- 172.21.15.39:0/1856518147 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562ce35db590 msgr2=0x562ce399d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:38.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.011+0000 7f1b2de00640 1 --2- 172.21.15.39:0/1856518147 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562ce35db590 0x562ce399d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.011+0000 7f1b2de00640 1 -- 172.21.15.39:0/1856518147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce35d33b0 msgr2=0x562ce39cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:38.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.011+0000 7f1b2de00640 1 --2- 172.21.15.39:0/1856518147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce35d33b0 0x562ce39cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.011+0000 7f1b2de00640 1 -- 172.21.15.39:0/1856518147 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ce38ce520 con 0x562ce35fc270 2024-09-21T07:50:38.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.011+0000 7f1b27e00640 1 --2- 172.21.15.39:0/1856518147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce35d33b0 0x562ce39cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:50:38.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.011+0000 7f1b2de00640 1 --2- 172.21.15.39:0/1856518147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce35fc270 0x562ce39ccb60 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f1b20001e80 tx=0x7f1b20011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9405dc19c956993 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:38.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.011+0000 7f1b1ea00640 1 -- 172.21.15.39:0/1856518147 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b20025070 con 0x562ce35fc270 2024-09-21T07:50:38.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.011+0000 7f1b1ea00640 1 -- 172.21.15.39:0/1856518147 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b20015800 con 0x562ce35fc270 2024-09-21T07:50:38.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.011+0000 7f1b1ea00640 1 -- 172.21.15.39:0/1856518147 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b20015bb0 con 0x562ce35fc270 2024-09-21T07:50:38.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.012+0000 7f1b3c1ec740 1 -- 172.21.15.39:0/1856518147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce35fc270 msgr2=0x562ce39ccb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:38.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.012+0000 7f1b3c1ec740 1 --2- 172.21.15.39:0/1856518147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce35fc270 0x562ce39ccb60 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f1b20001e80 tx=0x7f1b20011000 comp rx=0 tx=0).stop 2024-09-21T07:50:38.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.012+0000 7f1b3c1ec740 1 -- 172.21.15.39:0/1856518147 shutdown_connections 2024-09-21T07:50:38.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.012+0000 7f1b3c1ec740 1 --2- 172.21.15.39:0/1856518147 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562ce35db590 0x562ce399d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.012+0000 7f1b3c1ec740 1 --2- 172.21.15.39:0/1856518147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce35d33b0 0x562ce39cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.012+0000 7f1b3c1ec740 1 --2- 172.21.15.39:0/1856518147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce35fc270 0x562ce39ccb60 unknown :-1 s=CLOSED pgs=2377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.012+0000 7f1b3c1ec740 1 -- 172.21.15.39:0/1856518147 >> 172.21.15.39:0/1856518147 conn(0x562ce35f03c0 msgr2=0x562ce39a2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:38.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.012+0000 7f1b3c1ec740 1 -- 172.21.15.39:0/1856518147 shutdown_connections 2024-09-21T07:50:38.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.012+0000 7f1b3c1ec740 1 -- 172.21.15.39:0/1856518147 wait complete. 2024-09-21T07:50:38.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.013+0000 7f1b3c1ec740 1 -- start start 2024-09-21T07:50:38.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.014+0000 7f1b3c1ec740 1 Processor -- start 2024-09-21T07:50:38.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.015+0000 7f1b3c1ec740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce35d33b0 0x562ce39e1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:38.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.015+0000 7f1b3c1ec740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce35db590 0x562ce39c8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:38.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.015+0000 7f1b3c1ec740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562ce35fc270 0x562ce39c8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:38.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.015+0000 7f1b3c1ec740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562ce38ac330 con 0x562ce35d33b0 2024-09-21T07:50:38.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.015+0000 7f1b3c1ec740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562ce3909810 con 0x562ce35db590 2024-09-21T07:50:38.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.015+0000 7f1b3c1ec740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x562ce3676620 con 0x562ce35fc270 2024-09-21T07:50:38.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.015+0000 7f1b3c1ec740 10 client.0 _ll_register_callbacks cb 0x562ce38b1420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:50:38.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.015+0000 7f1b2de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce35d33b0 0x562ce39e1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:38.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.015+0000 7f1b2de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce35d33b0 0x562ce39e1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50702/0 (socket says 172.21.15.39:50702) 2024-09-21T07:50:38.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.015+0000 7f1b2de00640 1 -- 172.21.15.39:0/2391546065 learned_addr learned my addr 172.21.15.39:0/2391546065 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:38.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.016+0000 7f1b2de00640 1 -- 172.21.15.39:0/2391546065 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562ce35fc270 msgr2=0x562ce39c8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:38.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.016+0000 7f1b2e800640 1 --2- 172.21.15.39:0/2391546065 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562ce35fc270 0x562ce39c8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:38.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.016+0000 7f1b27e00640 1 --2- 172.21.15.39:0/2391546065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce35db590 0x562ce39c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:38.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.016+0000 7f1b2de00640 1 --2- 172.21.15.39:0/2391546065 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562ce35fc270 0x562ce39c8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.016+0000 7f1b2de00640 1 -- 172.21.15.39:0/2391546065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce35db590 msgr2=0x562ce39c8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:38.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.016+0000 7f1b2de00640 1 --2- 172.21.15.39:0/2391546065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce35db590 0x562ce39c8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.016+0000 7f1b2de00640 1 -- 172.21.15.39:0/2391546065 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ce3667850 con 0x562ce35d33b0 2024-09-21T07:50:38.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.016+0000 7f1b2e800640 1 --2- 172.21.15.39:0/2391546065 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562ce35fc270 0x562ce39c8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:50:38.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.016+0000 7f1b27e00640 1 --2- 172.21.15.39:0/2391546065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce35db590 0x562ce39c8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:50:38.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.016+0000 7f1b2de00640 1 --2- 172.21.15.39:0/2391546065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce35d33b0 0x562ce39e1870 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f1b20015be0 tx=0x7f1b20015c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81faa556d5115f5f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:38.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.016+0000 7f1b13400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:50:38.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.016+0000 7f1b13400640 1 -- 172.21.15.39:0/2391546065 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b20030060 con 0x562ce35d33b0 2024-09-21T07:50:38.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.016+0000 7f1b3c1ec740 1 -- 172.21.15.39:0/2391546065 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b20015800 con 0x562ce35d33b0 2024-09-21T07:50:38.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.016+0000 7f1b13400640 1 -- 172.21.15.39:0/2391546065 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(240..240 src has 1..240) ==== 7390+0+0 (secure 0 0 0) 0x7f1b2002b470 con 0x562ce35d33b0 2024-09-21T07:50:38.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.017+0000 7f1b13400640 1 -- 172.21.15.39:0/2391546065 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=265) ==== 24+0+0 (secure 0 0 0) 0x7f1b20036050 con 0x562ce35d33b0 2024-09-21T07:50:38.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.017+0000 7f1b3c1ec740 10 client.11139 fetch_fsmap learned FSMap version 265 2024-09-21T07:50:38.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.017+0000 7f1b3c1ec740 1 -- 172.21.15.39:0/2391546065 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=265,monmap=2+}) -- 0x562ce36de6b0 con 0x562ce35d33b0 2024-09-21T07:50:38.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.017+0000 7f1b13400640 1 -- 172.21.15.39:0/2391546065 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b20015800 con 0x562ce35d33b0 2024-09-21T07:50:38.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.017+0000 7f1b13400640 1 -- 172.21.15.39:0/2391546065 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 265) ==== 3542+0+0 (secure 0 0 0) 0x7f1b20039020 con 0x562ce35d33b0 2024-09-21T07:50:38.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.017+0000 7f1b3c1ec740 10 client.11139 fetch_fsmap finished waiting for FSMap version 265 2024-09-21T07:50:38.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.017+0000 7f1b3c1ec740 10 client.11139 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:50:38.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.017+0000 7f1b3c1ec740 20 client.11139 populate_metadata read hostname 'smithi039' 2024-09-21T07:50:38.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.017+0000 7f1b3c1ec740 1 --2- 172.21.15.39:0/2391546065 >> [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] conn(0x562ce35bae40 0x562ce39f6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:38.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.017+0000 7f1b3c1ec740 4 client.11139 mds_command: new command op to 11052 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:50:38.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.017+0000 7f1b3c1ec740 1 -- 172.21.15.39:0/2391546065 --> [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ce3918520 con 0x562ce35bae40 2024-09-21T07:50:38.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.017+0000 7f1b27e00640 1 --2- 172.21.15.39:0/2391546065 >> [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] conn(0x562ce35bae40 0x562ce39f6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:38.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.018+0000 7f1b27e00640 1 --2- 172.21.15.39:0/2391546065 >> [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] conn(0x562ce35bae40 0x562ce39f6cb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=af1e888e3eb6e3dd server_cookie=b19f2deb4c858bf6 in_seq=0 out_seq=0 2024-09-21T07:50:38.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.018+0000 7f1b13400640 10 client.11139 ms_handle_connect on v2:172.21.15.39:6833/485194245 2024-09-21T07:50:38.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.019+0000 7f1b13400640 1 -- 172.21.15.39:0/2391546065 <== mds.1 v2:172.21.15.39:6833/485194245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562ce3918520 con 0x562ce35bae40 2024-09-21T07:50:38.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.019+0000 7f1b13400640 10 client.11139 handle_command_reply: tid=0 2024-09-21T07:50:38.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.019+0000 7f1b3c1ec740 1 client.11139 shutdown 2024-09-21T07:50:38.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.019+0000 7f1b3c1ec740 1 -- 172.21.15.39:0/2391546065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce35d33b0 msgr2=0x562ce39e1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:38.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.019+0000 7f1b3c1ec740 1 --2- 172.21.15.39:0/2391546065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce35d33b0 0x562ce39e1870 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f1b20015be0 tx=0x7f1b20015c10 comp rx=0 tx=0).stop 2024-09-21T07:50:38.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.019+0000 7f1b3c1ec740 1 -- 172.21.15.39:0/2391546065 shutdown_connections 2024-09-21T07:50:38.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.019+0000 7f1b3c1ec740 1 --2- 172.21.15.39:0/2391546065 >> [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] conn(0x562ce35bae40 0x562ce39f6cb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.019+0000 7f1b3c1ec740 1 --2- 172.21.15.39:0/2391546065 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562ce35fc270 0x562ce39c8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.019+0000 7f1b3c1ec740 1 --2- 172.21.15.39:0/2391546065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce35db590 0x562ce39c8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.019+0000 7f1b3c1ec740 1 --2- 172.21.15.39:0/2391546065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce35d33b0 0x562ce39e1870 unknown :-1 s=CLOSED pgs=2378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.019+0000 7f1b13400640 0 client.11139 ms_handle_reset on v2:172.21.15.39:6833/485194245 2024-09-21T07:50:38.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.019+0000 7f1b3c1ec740 1 -- 172.21.15.39:0/2391546065 >> 172.21.15.39:0/2391546065 conn(0x562ce35f03c0 msgr2=0x562ce399e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:38.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.020+0000 7f1b3c1ec740 1 -- 172.21.15.39:0/2391546065 shutdown_connections 2024-09-21T07:50:38.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.020+0000 7f1b3c1ec740 1 -- 172.21.15.39:0/2391546065 wait complete. 2024-09-21T07:50:38.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.020+0000 7f1b3c1ec740 20 client.11139 trim_cache size 0 max 16384 2024-09-21T07:50:38.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.039+0000 7f1b3c1ec740 1 Processor -- start 2024-09-21T07:50:38.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.039+0000 7f1b3c1ec740 1 -- start start 2024-09-21T07:50:38.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.041+0000 7f1b3c1ec740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce35d33b0 0x562ce399de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:38.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.041+0000 7f1b3c1ec740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562ce35db590 0x562ce399e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:38.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.041+0000 7f1b3c1ec740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce35fc270 0x562ce3a0a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:38.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.041+0000 7f1b3c1ec740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1b302040d0 con 0x562ce35fc270 2024-09-21T07:50:38.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.041+0000 7f1b3c1ec740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1b302047a0 con 0x562ce35d33b0 2024-09-21T07:50:38.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.041+0000 7f1b3c1ec740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f1b30203090 con 0x562ce35db590 2024-09-21T07:50:38.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.041+0000 7f1b2e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce35fc270 0x562ce3a0a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:38.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.041+0000 7f1b2e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce35fc270 0x562ce3a0a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50714/0 (socket says 172.21.15.39:50714) 2024-09-21T07:50:38.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.041+0000 7f1b2e800640 1 -- 172.21.15.39:0/3076346909 learned_addr learned my addr 172.21.15.39:0/3076346909 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:38.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.041+0000 7f1b2de00640 1 --2- 172.21.15.39:0/3076346909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce35d33b0 0x562ce399de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:38.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.041+0000 7f1b27e00640 1 --2- 172.21.15.39:0/3076346909 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562ce35db590 0x562ce399e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:38.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.041+0000 7f1b2e800640 1 -- 172.21.15.39:0/3076346909 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562ce35db590 msgr2=0x562ce399e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:38.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.041+0000 7f1b2e800640 1 --2- 172.21.15.39:0/3076346909 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562ce35db590 0x562ce399e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.041+0000 7f1b2e800640 1 -- 172.21.15.39:0/3076346909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce35d33b0 msgr2=0x562ce399de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:38.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.041+0000 7f1b2e800640 1 --2- 172.21.15.39:0/3076346909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce35d33b0 0x562ce399de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.041+0000 7f1b2e800640 1 -- 172.21.15.39:0/3076346909 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ce39a3840 con 0x562ce35fc270 2024-09-21T07:50:38.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.042+0000 7f1b2e800640 1 --2- 172.21.15.39:0/3076346909 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce35fc270 0x562ce3a0a5d0 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7f1b2805f370 tx=0x7f1b2809b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ffb064732d253d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:38.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.042+0000 7f1b1e000640 1 -- 172.21.15.39:0/3076346909 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2801f050 con 0x562ce35fc270 2024-09-21T07:50:38.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.042+0000 7f1b1e000640 1 -- 172.21.15.39:0/3076346909 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b28089030 con 0x562ce35fc270 2024-09-21T07:50:38.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.042+0000 7f1b1e000640 1 -- 172.21.15.39:0/3076346909 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b280627c0 con 0x562ce35fc270 2024-09-21T07:50:38.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.043+0000 7f1b3c1ec740 1 -- 172.21.15.39:0/3076346909 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce35fc270 msgr2=0x562ce3a0a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:38.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.043+0000 7f1b3c1ec740 1 --2- 172.21.15.39:0/3076346909 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce35fc270 0x562ce3a0a5d0 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7f1b2805f370 tx=0x7f1b2809b040 comp rx=0 tx=0).stop 2024-09-21T07:50:38.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.043+0000 7f1b3c1ec740 1 -- 172.21.15.39:0/3076346909 shutdown_connections 2024-09-21T07:50:38.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.043+0000 7f1b3c1ec740 1 --2- 172.21.15.39:0/3076346909 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce35fc270 0x562ce3a0a5d0 unknown :-1 s=CLOSED pgs=2379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.043+0000 7f1b3c1ec740 1 --2- 172.21.15.39:0/3076346909 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562ce35db590 0x562ce399e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.043+0000 7f1b3c1ec740 1 --2- 172.21.15.39:0/3076346909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce35d33b0 0x562ce399de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.043+0000 7f1b3c1ec740 1 -- 172.21.15.39:0/3076346909 >> 172.21.15.39:0/3076346909 conn(0x562ce35f03c0 msgr2=0x562ce39c4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:38.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.043+0000 7f1b3c1ec740 1 -- 172.21.15.39:0/3076346909 shutdown_connections 2024-09-21T07:50:38.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.043+0000 7f1b3c1ec740 1 -- 172.21.15.39:0/3076346909 wait complete. 2024-09-21T07:50:38.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.044+0000 7f1b3c1ec740 1 -- start start 2024-09-21T07:50:38.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.044+0000 7f1b3c1ec740 1 Processor -- start 2024-09-21T07:50:38.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.045+0000 7f1b3c1ec740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562ce35d33b0 0x562ce3a5d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:38.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.045+0000 7f1b3c1ec740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce35db590 0x562ce399de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:38.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.045+0000 7f1b3c1ec740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce35fc270 0x562ce399e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:38.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.045+0000 7f1b3c1ec740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562ce39fa6d0 con 0x562ce35db590 2024-09-21T07:50:38.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.045+0000 7f1b3c1ec740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562ce39fa110 con 0x562ce35fc270 2024-09-21T07:50:38.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.045+0000 7f1b3c1ec740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x562ce39fad80 con 0x562ce35d33b0 2024-09-21T07:50:38.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.045+0000 7f1b3c1ec740 10 client.0 _ll_register_callbacks cb 0x7f1b28004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:50:38.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.046+0000 7f1b27e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce35db590 0x562ce399de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:38.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.046+0000 7f1b27e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce35db590 0x562ce399de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50722/0 (socket says 172.21.15.39:50722) 2024-09-21T07:50:38.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.046+0000 7f1b27e00640 1 -- 172.21.15.39:0/577575208 learned_addr learned my addr 172.21.15.39:0/577575208 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:38.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.046+0000 7f1b27e00640 1 -- 172.21.15.39:0/577575208 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562ce35d33b0 msgr2=0x562ce3a5d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:38.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.046+0000 7f1b2e800640 1 --2- 172.21.15.39:0/577575208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce35fc270 0x562ce399e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:38.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.046+0000 7f1b2de00640 1 --2- 172.21.15.39:0/577575208 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562ce35d33b0 0x562ce3a5d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:38.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.046+0000 7f1b27e00640 1 --2- 172.21.15.39:0/577575208 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562ce35d33b0 0x562ce3a5d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.046+0000 7f1b27e00640 1 -- 172.21.15.39:0/577575208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce35fc270 msgr2=0x562ce399e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:38.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.046+0000 7f1b27e00640 1 --2- 172.21.15.39:0/577575208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce35fc270 0x562ce399e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.046+0000 7f1b27e00640 1 -- 172.21.15.39:0/577575208 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ce3a5dc50 con 0x562ce35db590 2024-09-21T07:50:38.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.046+0000 7f1b2de00640 1 --2- 172.21.15.39:0/577575208 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562ce35d33b0 0x562ce3a5d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:50:38.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.046+0000 7f1b2e800640 1 --2- 172.21.15.39:0/577575208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce35fc270 0x562ce399e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:50:38.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.046+0000 7f1b27e00640 1 --2- 172.21.15.39:0/577575208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce35db590 0x562ce399de40 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7f1b18009920 tx=0x7f1b18004450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21eb14af198a2dad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:38.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.046+0000 7f1b13400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:50:38.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.047+0000 7f1b13400640 1 -- 172.21.15.39:0/577575208 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b18047710 con 0x562ce35db590 2024-09-21T07:50:38.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.047+0000 7f1b13400640 1 -- 172.21.15.39:0/577575208 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(240..240 src has 1..240) ==== 7390+0+0 (secure 0 0 0) 0x7f1b180478b0 con 0x562ce35db590 2024-09-21T07:50:38.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.047+0000 7f1b3c1ec740 1 -- 172.21.15.39:0/577575208 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b28089030 con 0x562ce35db590 2024-09-21T07:50:38.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.047+0000 7f1b13400640 1 -- 172.21.15.39:0/577575208 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=265) ==== 24+0+0 (secure 0 0 0) 0x7f1b18055050 con 0x562ce35db590 2024-09-21T07:50:38.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.047+0000 7f1b3c1ec740 10 client.11145 fetch_fsmap learned FSMap version 265 2024-09-21T07:50:38.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.047+0000 7f1b3c1ec740 1 -- 172.21.15.39:0/577575208 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=265,monmap=2+}) -- 0x562ce3a6a7f0 con 0x562ce35db590 2024-09-21T07:50:38.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.047+0000 7f1b13400640 1 -- 172.21.15.39:0/577575208 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b28089030 con 0x562ce35db590 2024-09-21T07:50:38.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.047+0000 7f1b13400640 1 -- 172.21.15.39:0/577575208 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 265) ==== 3542+0+0 (secure 0 0 0) 0x7f1b18058020 con 0x562ce35db590 2024-09-21T07:50:38.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.047+0000 7f1b3c1ec740 10 client.11145 fetch_fsmap finished waiting for FSMap version 265 2024-09-21T07:50:38.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.047+0000 7f1b3c1ec740 10 client.11145 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:50:38.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.047+0000 7f1b3c1ec740 20 client.11145 populate_metadata read hostname 'smithi039' 2024-09-21T07:50:38.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.047+0000 7f1b3c1ec740 1 --2- 172.21.15.39:0/577575208 >> [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] conn(0x562ce39f4f40 0x562ce39f5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:38.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.047+0000 7f1b3c1ec740 4 client.11145 mds_command: new command op to 11052 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:50:38.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.047+0000 7f1b3c1ec740 1 -- 172.21.15.39:0/577575208 --> [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] -- command(tid 0: {"prefix": "status"}) -- 0x562ce3913fd0 con 0x562ce39f4f40 2024-09-21T07:50:38.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.048+0000 7f1b2de00640 1 --2- 172.21.15.39:0/577575208 >> [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] conn(0x562ce39f4f40 0x562ce39f5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:38.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.048+0000 7f1b2de00640 1 --2- 172.21.15.39:0/577575208 >> [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] conn(0x562ce39f4f40 0x562ce39f5350 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6a1b21288dd1044a server_cookie=c42496166b63fe0a in_seq=0 out_seq=0 2024-09-21T07:50:38.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.048+0000 7f1b13400640 10 client.11145 ms_handle_connect on v2:172.21.15.39:6833/485194245 2024-09-21T07:50:38.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.048+0000 7f1b13400640 1 -- 172.21.15.39:0/577575208 <== mds.1 v2:172.21.15.39:6833/485194245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+335 (crc 0 0 0) 0x562ce3913fd0 con 0x562ce39f4f40 2024-09-21T07:50:38.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.048+0000 7f1b13400640 10 client.11145 handle_command_reply: tid=0 2024-09-21T07:50:38.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.048+0000 7f1b3c1ec740 1 client.11145 shutdown 2024-09-21T07:50:38.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.049+0000 7f1b3c1ec740 1 -- 172.21.15.39:0/577575208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce35db590 msgr2=0x562ce399de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:38.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.049+0000 7f1b3c1ec740 1 --2- 172.21.15.39:0/577575208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce35db590 0x562ce399de40 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7f1b18009920 tx=0x7f1b18004450 comp rx=0 tx=0).stop 2024-09-21T07:50:38.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.049+0000 7f1b3c1ec740 1 -- 172.21.15.39:0/577575208 shutdown_connections 2024-09-21T07:50:38.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.049+0000 7f1b3c1ec740 1 --2- 172.21.15.39:0/577575208 >> [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] conn(0x562ce39f4f40 0x562ce39f5350 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.049+0000 7f1b3c1ec740 1 --2- 172.21.15.39:0/577575208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce35fc270 0x562ce399e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.049+0000 7f1b3c1ec740 1 --2- 172.21.15.39:0/577575208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562ce35db590 0x562ce399de40 unknown :-1 s=CLOSED pgs=2380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.049+0000 7f1b13400640 0 client.11145 ms_handle_reset on v2:172.21.15.39:6833/485194245 2024-09-21T07:50:38.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.049+0000 7f1b3c1ec740 1 --2- 172.21.15.39:0/577575208 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562ce35d33b0 0x562ce3a5d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.049+0000 7f1b3c1ec740 1 -- 172.21.15.39:0/577575208 >> 172.21.15.39:0/577575208 conn(0x562ce35f03c0 msgr2=0x562ce39e8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:38.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.049+0000 7f1b3c1ec740 1 -- 172.21.15.39:0/577575208 shutdown_connections 2024-09-21T07:50:38.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.049+0000 7f1b3c1ec740 1 -- 172.21.15.39:0/577575208 wait complete. 2024-09-21T07:50:38.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.049+0000 7f1b3c1ec740 20 client.11145 trim_cache size 0 max 16384 2024-09-21T07:50:38.055 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:50:38.055 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:50:38.055 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 1, 2024-09-21T07:50:38.055 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 11052, 2024-09-21T07:50:38.055 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:50:38.055 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:50:38.055 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:50:38.055 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 5.6637224350000004, 2024-09-21T07:50:38.055 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 265, 2024-09-21T07:50:38.055 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 240, 2024-09-21T07:50:38.055 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 240, 2024-09-21T07:50:38.055 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 6.0759897260000004 2024-09-21T07:50:38.055 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:50:38.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.050+0000 7f1b2f200640 1 -- 172.21.15.39:0/1375268251 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1b0c078bc0 msgr2=0x7f1b0c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:38.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.050+0000 7f1b2f200640 1 --2- 172.21.15.39:0/1375268251 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1b0c078bc0 0x7f1b0c07b0d0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f1b200013f0 tx=0x7f1b20009040 comp rx=0 tx=0).stop 2024-09-21T07:50:38.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.050+0000 7f1b2f200640 1 -- 172.21.15.39:0/1375268251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b301519d0 msgr2=0x7f1b300ce230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:38.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.050+0000 7f1b2f200640 1 --2- 172.21.15.39:0/1375268251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b301519d0 0x7f1b300ce230 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800e990 tx=0x7f1b2800ee50 comp rx=0 tx=0).stop 2024-09-21T07:50:38.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.050+0000 7f1b2f200640 1 -- 172.21.15.39:0/1375268251 shutdown_connections 2024-09-21T07:50:38.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.050+0000 7f1b2f200640 1 --2- 172.21.15.39:0/1375268251 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1b0c078bc0 0x7f1b0c07b0d0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.050+0000 7f1b2f200640 1 --2- 172.21.15.39:0/1375268251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b301519d0 0x7f1b300ce230 unknown :-1 s=CLOSED pgs=2376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.050+0000 7f1b2f200640 1 --2- 172.21.15.39:0/1375268251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b300d04a0 0x7f1b300cdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.050+0000 7f1b2f200640 1 --2- 172.21.15.39:0/1375268251 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1b3006ee90 0x7f1b3013cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.050+0000 7f1b2f200640 1 -- 172.21.15.39:0/1375268251 >> 172.21.15.39:0/1375268251 conn(0x7f1b3006cfc0 msgr2=0x7f1b3006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:38.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.050+0000 7f1b2f200640 1 -- 172.21.15.39:0/1375268251 shutdown_connections 2024-09-21T07:50:38.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.050+0000 7f1b2f200640 1 -- 172.21.15.39:0/1375268251 wait complete. 2024-09-21T07:50:38.074 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:38.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.299+0000 7f50a5600640 1 -- 172.21.15.39:0/3668496082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50a00af430 msgr2=0x7f50a00cd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:38.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.299+0000 7f50a5600640 1 --2- 172.21.15.39:0/3668496082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50a00af430 0x7f50a00cd260 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f5090009830 tx=0x7f509002a480 comp rx=0 tx=0).stop 2024-09-21T07:50:38.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.300+0000 7f50a5600640 1 -- 172.21.15.39:0/3668496082 shutdown_connections 2024-09-21T07:50:38.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.300+0000 7f50a5600640 1 --2- 172.21.15.39:0/3668496082 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f50a006fdb0 0x7f50a00b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.300+0000 7f50a5600640 1 --2- 172.21.15.39:0/3668496082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50a00af430 0x7f50a00cd260 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.300+0000 7f50a5600640 1 -- 172.21.15.39:0/3668496082 >> 172.21.15.39:0/3668496082 conn(0x7f50a00b2460 msgr2=0x7f50a006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:38.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.300+0000 7f50a5600640 1 -- 172.21.15.39:0/3668496082 shutdown_connections 2024-09-21T07:50:38.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.300+0000 7f50a5600640 1 -- 172.21.15.39:0/3668496082 wait complete. 2024-09-21T07:50:38.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.302+0000 7f50a5600640 1 Processor -- start 2024-09-21T07:50:38.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.302+0000 7f50a5600640 1 -- start start 2024-09-21T07:50:38.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.303+0000 7f50a5600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50a006f9a0 0x7f50a0157570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:38.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.303+0000 7f50a5600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f50a006fdb0 0x7f50a00990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:38.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.304+0000 7f50a5600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50a00af430 0x7f50a00cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:38.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.304+0000 7f50a5600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f50a00cfd70 con 0x7f50a006f9a0 2024-09-21T07:50:38.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.304+0000 7f50a5600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f50a00b45f0 con 0x7f50a00af430 2024-09-21T07:50:38.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.304+0000 7f50a5600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f50a01f6cc0 con 0x7f50a006fdb0 2024-09-21T07:50:38.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.304+0000 7f509ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50a006f9a0 0x7f50a0157570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:38.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.304+0000 7f509ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50a006f9a0 0x7f50a0157570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50730/0 (socket says 172.21.15.39:50730) 2024-09-21T07:50:38.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.304+0000 7f509ea00640 1 -- 172.21.15.39:0/156762106 learned_addr learned my addr 172.21.15.39:0/156762106 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:38.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.304+0000 7f509f400640 1 --2- 172.21.15.39:0/156762106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50a00af430 0x7f50a00cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:38.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.304+0000 7f509e000640 1 --2- 172.21.15.39:0/156762106 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f50a006fdb0 0x7f50a00990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:38.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.304+0000 7f509ea00640 1 -- 172.21.15.39:0/156762106 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f50a006fdb0 msgr2=0x7f50a00990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:38.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.304+0000 7f509ea00640 1 --2- 172.21.15.39:0/156762106 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f50a006fdb0 0x7f50a00990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.304+0000 7f509ea00640 1 -- 172.21.15.39:0/156762106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50a00af430 msgr2=0x7f50a00cb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:38.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.304+0000 7f509ea00640 1 --2- 172.21.15.39:0/156762106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50a00af430 0x7f50a00cb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.304+0000 7f509ea00640 1 -- 172.21.15.39:0/156762106 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a0157ab0 con 0x7f50a006f9a0 2024-09-21T07:50:38.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.304+0000 7f509f400640 1 --2- 172.21.15.39:0/156762106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50a00af430 0x7f50a00cb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:50:38.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.305+0000 7f509ea00640 1 --2- 172.21.15.39:0/156762106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50a006f9a0 0x7f50a0157570 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7f5090009950 tx=0x7f5090033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:38.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.305+0000 7f5096a00640 1 -- 172.21.15.39:0/156762106 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5090005c90 con 0x7f50a006f9a0 2024-09-21T07:50:38.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.305+0000 7f5096a00640 1 -- 172.21.15.39:0/156762106 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5090031e50 con 0x7f50a006f9a0 2024-09-21T07:50:38.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.305+0000 7f50a5600640 1 -- 172.21.15.39:0/156762106 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50a015a530 con 0x7f50a006f9a0 2024-09-21T07:50:38.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.305+0000 7f5096a00640 1 -- 172.21.15.39:0/156762106 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5090005260 con 0x7f50a006f9a0 2024-09-21T07:50:38.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.305+0000 7f50a5600640 1 -- 172.21.15.39:0/156762106 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50a0154c50 con 0x7f50a006f9a0 2024-09-21T07:50:38.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.306+0000 7f50a5600640 1 -- 172.21.15.39:0/156762106 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50a015a920 con 0x7f50a006f9a0 2024-09-21T07:50:38.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.307+0000 7f5096a00640 1 -- 172.21.15.39:0/156762106 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5090005400 con 0x7f50a006f9a0 2024-09-21T07:50:38.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.308+0000 7f5096a00640 1 --2- 172.21.15.39:0/156762106 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5080078b90 0x7f508007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:38.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.309+0000 7f5096a00640 1 -- 172.21.15.39:0/156762106 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(240..240 src has 1..240) ==== 7390+0+0 (secure 0 0 0) 0x7f50900bc570 con 0x7f50a006f9a0 2024-09-21T07:50:38.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.309+0000 7f509e000640 1 --2- 172.21.15.39:0/156762106 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5080078b90 0x7f508007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:38.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.310+0000 7f509e000640 1 --2- 172.21.15.39:0/156762106 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5080078b90 0x7f508007b0a0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f508c007fd0 tx=0x7f508c009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:38.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.313+0000 7f5096a00640 1 -- 172.21.15.39:0/156762106 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f509003f030 con 0x7f50a006f9a0 2024-09-21T07:50:38.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.548+0000 7f50a5600640 1 -- 172.21.15.39:0/156762106 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50a00b40b0 con 0x7f50a006f9a0 2024-09-21T07:50:38.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.548+0000 7f5096a00640 1 -- 172.21.15.39:0/156762106 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 265 v265) ==== 95+0+5279 (secure 0 0 0) 0x7f50900842a0 con 0x7f50a006f9a0 2024-09-21T07:50:38.550 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:50:38.550 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":265,"btime":"2024-09-21T07:50:36:415072+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11067,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/109229385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":109229385},{"type":"v1","addr":"172.21.15.39:6834","nonce":109229385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":262}],"filesystems":[{"mdsmap":{"epoch":265,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:50:36.415066+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":240,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":8531},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8531":{"gid":8531,"name":"d","rank":2,"incarnation":253,"state":"up:active","state_seq":18,"addr":"172.21.15.148:6837/1438851059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1438851059},{"type":"v1","addr":"172.21.15.148:6837","nonce":1438851059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[8531,9967,11052]},"id":13}]} 2024-09-21T07:50:38.551 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 265 2024-09-21T07:50:38.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.550+0000 7f50a5600640 1 -- 172.21.15.39:0/156762106 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5080078b90 msgr2=0x7f508007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:38.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.550+0000 7f50a5600640 1 --2- 172.21.15.39:0/156762106 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5080078b90 0x7f508007b0a0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f508c007fd0 tx=0x7f508c009040 comp rx=0 tx=0).stop 2024-09-21T07:50:38.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.551+0000 7f50a5600640 1 -- 172.21.15.39:0/156762106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50a006f9a0 msgr2=0x7f50a0157570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:38.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.551+0000 7f50a5600640 1 --2- 172.21.15.39:0/156762106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50a006f9a0 0x7f50a0157570 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7f5090009950 tx=0x7f5090033c20 comp rx=0 tx=0).stop 2024-09-21T07:50:38.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.551+0000 7f50a5600640 1 -- 172.21.15.39:0/156762106 shutdown_connections 2024-09-21T07:50:38.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.551+0000 7f50a5600640 1 --2- 172.21.15.39:0/156762106 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5080078b90 0x7f508007b0a0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.551+0000 7f50a5600640 1 --2- 172.21.15.39:0/156762106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50a00af430 0x7f50a00cb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.551+0000 7f50a5600640 1 --2- 172.21.15.39:0/156762106 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f50a006fdb0 0x7f50a00990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.551+0000 7f50a5600640 1 --2- 172.21.15.39:0/156762106 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50a006f9a0 0x7f50a0157570 unknown :-1 s=CLOSED pgs=2381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.551+0000 7f50a5600640 1 -- 172.21.15.39:0/156762106 >> 172.21.15.39:0/156762106 conn(0x7f50a00b2460 msgr2=0x7f50a0167050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:38.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.551+0000 7f50a5600640 1 -- 172.21.15.39:0/156762106 shutdown_connections 2024-09-21T07:50:38.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.551+0000 7f50a5600640 1 -- 172.21.15.39:0/156762106 wait complete. 2024-09-21T07:50:38.574 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:active 2024-09-21T07:50:38.575 DEBUG:tasks.cephfs.filesystem:reached state 'up:active' in 0.5007283687591553s 2024-09-21T07:50:38.575 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:38.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.799+0000 7fd5eac00640 1 -- 172.21.15.39:0/3224576707 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5e40ce690 msgr2=0x7fd5e40b2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:38.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.799+0000 7fd5eac00640 1 --2- 172.21.15.39:0/3224576707 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5e40ce690 0x7fd5e40b2f70 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40b7ad0 tx=0x7fd5d802a540 comp rx=0 tx=0).stop 2024-09-21T07:50:38.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.799+0000 7fd5eac00640 1 -- 172.21.15.39:0/3224576707 shutdown_connections 2024-09-21T07:50:38.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.799+0000 7fd5eac00640 1 --2- 172.21.15.39:0/3224576707 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5e40d2030 0x7fd5e40d2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.799+0000 7fd5eac00640 1 --2- 172.21.15.39:0/3224576707 >> v2:172.21.15.148:3300/0 conn(0x7fd5e40d0060 0x7fd5e40d0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.799+0000 7fd5eac00640 1 --2- 172.21.15.39:0/3224576707 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5e40ce690 0x7fd5e40b2f70 unknown :-1 s=CLOSED pgs=2382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.799+0000 7fd5eac00640 1 -- 172.21.15.39:0/3224576707 >> 172.21.15.39:0/3224576707 conn(0x7fd5e409b5f0 msgr2=0x7fd5e408c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:38.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.799+0000 7fd5eac00640 1 -- 172.21.15.39:0/3224576707 shutdown_connections 2024-09-21T07:50:38.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.799+0000 7fd5eac00640 1 -- 172.21.15.39:0/3224576707 wait complete. 2024-09-21T07:50:38.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.801+0000 7fd5eac00640 1 Processor -- start 2024-09-21T07:50:38.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.802+0000 7fd5eac00640 1 -- start start 2024-09-21T07:50:38.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.803+0000 7fd5eac00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5e40ce690 0x7fd5e41f6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:38.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.803+0000 7fd5eac00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5e40d0060 0x7fd5e41f9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:38.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.803+0000 7fd5eac00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5e40d2030 0x7fd5e40c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:38.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.803+0000 7fd5eac00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e40cd260 con 0x7fd5e40d0060 2024-09-21T07:50:38.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.803+0000 7fd5eac00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e40cd0e0 con 0x7fd5e40ce690 2024-09-21T07:50:38.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.803+0000 7fd5eac00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd5e40cd3e0 con 0x7fd5e40d2030 2024-09-21T07:50:38.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.803+0000 7fd5e8e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5e40d0060 0x7fd5e41f9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:38.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.803+0000 7fd5e8e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5e40d0060 0x7fd5e41f9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50750/0 (socket says 172.21.15.39:50750) 2024-09-21T07:50:38.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.803+0000 7fd5e8e00640 1 -- 172.21.15.39:0/2269589030 learned_addr learned my addr 172.21.15.39:0/2269589030 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:38.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.803+0000 7fd5e9800640 1 --2- 172.21.15.39:0/2269589030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5e40ce690 0x7fd5e41f6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:38.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.804+0000 7fd5ea200640 1 --2- 172.21.15.39:0/2269589030 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5e40d2030 0x7fd5e40c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:38.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.804+0000 7fd5e8e00640 1 -- 172.21.15.39:0/2269589030 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5e40d2030 msgr2=0x7fd5e40c5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:38.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.804+0000 7fd5e8e00640 1 --2- 172.21.15.39:0/2269589030 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5e40d2030 0x7fd5e40c5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.804+0000 7fd5e8e00640 1 -- 172.21.15.39:0/2269589030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5e40ce690 msgr2=0x7fd5e41f6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:38.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.804+0000 7fd5e8e00640 1 --2- 172.21.15.39:0/2269589030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5e40ce690 0x7fd5e41f6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.804+0000 7fd5e8e00640 1 -- 172.21.15.39:0/2269589030 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e41f9e40 con 0x7fd5e40d0060 2024-09-21T07:50:38.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.804+0000 7fd5ea200640 1 --2- 172.21.15.39:0/2269589030 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5e40d2030 0x7fd5e40c5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:50:38.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.804+0000 7fd5e8e00640 1 --2- 172.21.15.39:0/2269589030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5e40d0060 0x7fd5e41f9900 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7fd5d000c9a0 tx=0x7fd5d000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:38.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.804+0000 7fd5e1600640 1 -- 172.21.15.39:0/2269589030 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0004230 con 0x7fd5e40d0060 2024-09-21T07:50:38.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.804+0000 7fd5e1600640 1 -- 172.21.15.39:0/2269589030 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd5d00044c0 con 0x7fd5e40d0060 2024-09-21T07:50:38.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.804+0000 7fd5eac00640 1 -- 172.21.15.39:0/2269589030 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e41fa350 con 0x7fd5e40d0060 2024-09-21T07:50:38.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.804+0000 7fd5e1600640 1 -- 172.21.15.39:0/2269589030 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0007bf0 con 0x7fd5e40d0060 2024-09-21T07:50:38.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.805+0000 7fd5eac00640 1 -- 172.21.15.39:0/2269589030 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e40aeff0 con 0x7fd5e40d0060 2024-09-21T07:50:38.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.806+0000 7fd5eac00640 1 -- 172.21.15.39:0/2269589030 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5e41fa0c0 con 0x7fd5e40d0060 2024-09-21T07:50:38.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.807+0000 7fd5e1600640 1 -- 172.21.15.39:0/2269589030 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd5d00035f0 con 0x7fd5e40d0060 2024-09-21T07:50:38.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.808+0000 7fd5e1600640 1 --2- 172.21.15.39:0/2269589030 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd5c4078b90 0x7fd5c407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:38.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.808+0000 7fd5e1600640 1 -- 172.21.15.39:0/2269589030 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(240..240 src has 1..240) ==== 7390+0+0 (secure 0 0 0) 0x7fd5d009b780 con 0x7fd5e40d0060 2024-09-21T07:50:38.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.808+0000 7fd5e9800640 1 --2- 172.21.15.39:0/2269589030 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd5c4078b90 0x7fd5c407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:38.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.809+0000 7fd5e9800640 1 --2- 172.21.15.39:0/2269589030 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd5c4078b90 0x7fd5c407b0a0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8033000 tx=0x7fd5d8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:38.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:38.813+0000 7fd5e1600640 1 -- 172.21.15.39:0/2269589030 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd5d0014070 con 0x7fd5e40d0060 2024-09-21T07:50:39.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.048+0000 7fd5eac00640 1 -- 172.21.15.39:0/2269589030 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5e41fa870 con 0x7fd5e40d0060 2024-09-21T07:50:39.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.049+0000 7fd5e1600640 1 -- 172.21.15.39:0/2269589030 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 265 v265) ==== 95+0+5279 (secure 0 0 0) 0x7fd5e41fa870 con 0x7fd5e40d0060 2024-09-21T07:50:39.051 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:50:39.051 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":265,"btime":"2024-09-21T07:50:36:415072+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11067,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/109229385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":109229385},{"type":"v1","addr":"172.21.15.39:6834","nonce":109229385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":262}],"filesystems":[{"mdsmap":{"epoch":265,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:50:36.415066+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":240,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":8531},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8531":{"gid":8531,"name":"d","rank":2,"incarnation":253,"state":"up:active","state_seq":18,"addr":"172.21.15.148:6837/1438851059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1438851059},{"type":"v1","addr":"172.21.15.148:6837","nonce":1438851059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[8531,9967,11052]},"id":13}]} 2024-09-21T07:50:39.051 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 265 2024-09-21T07:50:39.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.051+0000 7fd5eac00640 1 -- 172.21.15.39:0/2269589030 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd5c4078b90 msgr2=0x7fd5c407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:39.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.051+0000 7fd5eac00640 1 --2- 172.21.15.39:0/2269589030 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd5c4078b90 0x7fd5c407b0a0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8033000 tx=0x7fd5d8004730 comp rx=0 tx=0).stop 2024-09-21T07:50:39.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.051+0000 7fd5eac00640 1 -- 172.21.15.39:0/2269589030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5e40d0060 msgr2=0x7fd5e41f9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:39.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.051+0000 7fd5eac00640 1 --2- 172.21.15.39:0/2269589030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5e40d0060 0x7fd5e41f9900 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7fd5d000c9a0 tx=0x7fd5d000ce60 comp rx=0 tx=0).stop 2024-09-21T07:50:39.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.051+0000 7fd5eac00640 1 -- 172.21.15.39:0/2269589030 shutdown_connections 2024-09-21T07:50:39.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.051+0000 7fd5eac00640 1 --2- 172.21.15.39:0/2269589030 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd5c4078b90 0x7fd5c407b0a0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:39.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.051+0000 7fd5eac00640 1 --2- 172.21.15.39:0/2269589030 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5e40d2030 0x7fd5e40c5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:39.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.051+0000 7fd5eac00640 1 --2- 172.21.15.39:0/2269589030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5e40d0060 0x7fd5e41f9900 unknown :-1 s=CLOSED pgs=2383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:39.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.051+0000 7fd5eac00640 1 --2- 172.21.15.39:0/2269589030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5e40ce690 0x7fd5e41f6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:39.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.051+0000 7fd5eac00640 1 -- 172.21.15.39:0/2269589030 >> 172.21.15.39:0/2269589030 conn(0x7fd5e409b5f0 msgr2=0x7fd5e40c5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:39.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.051+0000 7fd5eac00640 1 -- 172.21.15.39:0/2269589030 shutdown_connections 2024-09-21T07:50:39.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.051+0000 7fd5eac00640 1 -- 172.21.15.39:0/2269589030 wait complete. 2024-09-21T07:50:39.075 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json dump snaps 2024-09-21T07:50:39.330 INFO:teuthology.orchestra.run.smithi148.stdout:{"last_created":3,"last_destroyed":4,"snaps":[{"snapid":3,"ino":1099511627778,"stamp":"2024-09-21T07:50:02.909971+0000","name":"s2","metadata":{}}]} 2024-09-21T07:50:39.330 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 3, "last_destroyed": 4, "snaps": [ { "ino": 1099511627778, "metadata": {}, "name": "s2", "snapid": 3, "stamp": "2024-09-21T07:50:02.909971+0000" } ] } 2024-09-21T07:50:39.330 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:39.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.558+0000 7fb19ca00640 1 -- 172.21.15.39:0/4238305375 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb1980cd6a0 msgr2=0x7fb198167440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:39.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.558+0000 7fb19ca00640 1 --2- 172.21.15.39:0/4238305375 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb1980cd6a0 0x7fb198167440 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb190009be0 tx=0x7fb19002a920 comp rx=0 tx=0).stop 2024-09-21T07:50:39.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.558+0000 7fb19ca00640 1 -- 172.21.15.39:0/4238305375 shutdown_connections 2024-09-21T07:50:39.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.558+0000 7fb19ca00640 1 --2- 172.21.15.39:0/4238305375 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb1980cd6a0 0x7fb198167440 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:39.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.558+0000 7fb19ca00640 1 -- 172.21.15.39:0/4238305375 >> 172.21.15.39:0/4238305375 conn(0x7fb1980d0cb0 msgr2=0x7fb1980d10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:39.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.558+0000 7fb19ca00640 1 -- 172.21.15.39:0/4238305375 shutdown_connections 2024-09-21T07:50:39.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.558+0000 7fb19ca00640 1 -- 172.21.15.39:0/4238305375 wait complete. 2024-09-21T07:50:39.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.560+0000 7fb19ca00640 1 Processor -- start 2024-09-21T07:50:39.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.560+0000 7fb19ca00640 1 -- start start 2024-09-21T07:50:39.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.562+0000 7fb19ca00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb1981539b0 0x7fb19815ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:39.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.562+0000 7fb19ca00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb19815ccd0 0x7fb1981493f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:39.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.562+0000 7fb19ca00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb198152f60 0x7fb19815de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:39.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.562+0000 7fb19ca00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb198167d10 con 0x7fb198152f60 2024-09-21T07:50:39.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.562+0000 7fb19ca00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb198167b90 con 0x7fb19815ccd0 2024-09-21T07:50:39.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.562+0000 7fb19ca00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb198167e90 con 0x7fb1981539b0 2024-09-21T07:50:39.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.562+0000 7fb197e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb198152f60 0x7fb19815de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:39.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.562+0000 7fb197e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb198152f60 0x7fb19815de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50764/0 (socket says 172.21.15.39:50764) 2024-09-21T07:50:39.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.562+0000 7fb197e00640 1 -- 172.21.15.39:0/270921343 learned_addr learned my addr 172.21.15.39:0/270921343 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:39.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.562+0000 7fb18fe00640 1 --2- 172.21.15.39:0/270921343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb19815ccd0 0x7fb1981493f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:39.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.562+0000 7fb197400640 1 --2- 172.21.15.39:0/270921343 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb1981539b0 0x7fb19815ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:39.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.562+0000 7fb197e00640 1 -- 172.21.15.39:0/270921343 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb1981539b0 msgr2=0x7fb19815ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:39.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.562+0000 7fb197e00640 1 --2- 172.21.15.39:0/270921343 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb1981539b0 0x7fb19815ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:39.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.562+0000 7fb197e00640 1 -- 172.21.15.39:0/270921343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb19815ccd0 msgr2=0x7fb1981493f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:39.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.562+0000 7fb197e00640 1 --2- 172.21.15.39:0/270921343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb19815ccd0 0x7fb1981493f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:39.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.562+0000 7fb197e00640 1 -- 172.21.15.39:0/270921343 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1980711c0 con 0x7fb198152f60 2024-09-21T07:50:39.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.562+0000 7fb197400640 1 --2- 172.21.15.39:0/270921343 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb1981539b0 0x7fb19815ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:50:39.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.563+0000 7fb197e00640 1 --2- 172.21.15.39:0/270921343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb198152f60 0x7fb19815de70 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7fb190005fd0 tx=0x7fb190003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:39.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.563+0000 7fb18f400640 1 -- 172.21.15.39:0/270921343 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb190030e60 con 0x7fb198152f60 2024-09-21T07:50:39.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.563+0000 7fb19ca00640 1 -- 172.21.15.39:0/270921343 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1981567d0 con 0x7fb198152f60 2024-09-21T07:50:39.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.563+0000 7fb18f400640 1 -- 172.21.15.39:0/270921343 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb19003e070 con 0x7fb198152f60 2024-09-21T07:50:39.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.563+0000 7fb18f400640 1 -- 172.21.15.39:0/270921343 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1900042a0 con 0x7fb198152f60 2024-09-21T07:50:39.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.563+0000 7fb19ca00640 1 -- 172.21.15.39:0/270921343 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb198070bd0 con 0x7fb198152f60 2024-09-21T07:50:39.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.564+0000 7fb19ca00640 1 -- 172.21.15.39:0/270921343 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1980ce070 con 0x7fb198152f60 2024-09-21T07:50:39.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.565+0000 7fb18f400640 1 -- 172.21.15.39:0/270921343 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb190004440 con 0x7fb198152f60 2024-09-21T07:50:39.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.566+0000 7fb18f400640 1 --2- 172.21.15.39:0/270921343 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb174078b90 0x7fb17407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:39.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.566+0000 7fb18f400640 1 -- 172.21.15.39:0/270921343 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(240..240 src has 1..240) ==== 7390+0+0 (secure 0 0 0) 0x7fb1900bcbf0 con 0x7fb198152f60 2024-09-21T07:50:39.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.567+0000 7fb197400640 1 --2- 172.21.15.39:0/270921343 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb174078b90 0x7fb17407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:39.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.567+0000 7fb197400640 1 --2- 172.21.15.39:0/270921343 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb174078b90 0x7fb17407b0a0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fb188007fd0 tx=0x7fb188009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:39.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.571+0000 7fb18f400640 1 -- 172.21.15.39:0/270921343 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb19002d070 con 0x7fb198152f60 2024-09-21T07:50:39.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.806+0000 7fb19ca00640 1 -- 172.21.15.39:0/270921343 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb198069630 con 0x7fb198152f60 2024-09-21T07:50:39.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.807+0000 7fb18f400640 1 -- 172.21.15.39:0/270921343 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 265 v265) ==== 95+0+5279 (secure 0 0 0) 0x7fb1900849a0 con 0x7fb198152f60 2024-09-21T07:50:39.809 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:50:39.809 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":265,"btime":"2024-09-21T07:50:36:415072+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11067,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/109229385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":109229385},{"type":"v1","addr":"172.21.15.39:6834","nonce":109229385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":262}],"filesystems":[{"mdsmap":{"epoch":265,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:50:36.415066+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":240,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":8531},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8531":{"gid":8531,"name":"d","rank":2,"incarnation":253,"state":"up:active","state_seq":18,"addr":"172.21.15.148:6837/1438851059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1438851059},{"type":"v1","addr":"172.21.15.148:6837","nonce":1438851059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[8531,9967,11052]},"id":13}]} 2024-09-21T07:50:39.809 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 265 2024-09-21T07:50:39.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.809+0000 7fb19ca00640 1 -- 172.21.15.39:0/270921343 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb174078b90 msgr2=0x7fb17407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:39.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.809+0000 7fb19ca00640 1 --2- 172.21.15.39:0/270921343 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb174078b90 0x7fb17407b0a0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fb188007fd0 tx=0x7fb188009040 comp rx=0 tx=0).stop 2024-09-21T07:50:39.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.809+0000 7fb19ca00640 1 -- 172.21.15.39:0/270921343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb198152f60 msgr2=0x7fb19815de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:39.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.809+0000 7fb19ca00640 1 --2- 172.21.15.39:0/270921343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb198152f60 0x7fb19815de70 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7fb190005fd0 tx=0x7fb190003c20 comp rx=0 tx=0).stop 2024-09-21T07:50:39.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.809+0000 7fb19ca00640 1 -- 172.21.15.39:0/270921343 shutdown_connections 2024-09-21T07:50:39.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.809+0000 7fb19ca00640 1 --2- 172.21.15.39:0/270921343 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb174078b90 0x7fb17407b0a0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:39.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.809+0000 7fb19ca00640 1 --2- 172.21.15.39:0/270921343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb198152f60 0x7fb19815de70 unknown :-1 s=CLOSED pgs=2385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:39.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.809+0000 7fb19ca00640 1 --2- 172.21.15.39:0/270921343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb19815ccd0 0x7fb1981493f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:39.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.809+0000 7fb19ca00640 1 --2- 172.21.15.39:0/270921343 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb1981539b0 0x7fb19815ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:39.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.809+0000 7fb19ca00640 1 -- 172.21.15.39:0/270921343 >> 172.21.15.39:0/270921343 conn(0x7fb1980d0cb0 msgr2=0x7fb19815c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:39.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.809+0000 7fb19ca00640 1 -- 172.21.15.39:0/270921343 shutdown_connections 2024-09-21T07:50:39.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:39.809+0000 7fb19ca00640 1 -- 172.21.15.39:0/270921343 wait complete. 2024-09-21T07:50:39.833 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps 2024-09-21T07:50:40.076 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_created":3,"last_destroyed":4,"snaps":[{"snapid":3,"ino":1099511627778,"stamp":"2024-09-21T07:50:02.909971+0000","name":"s2","metadata":{}}]} 2024-09-21T07:50:40.077 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 3, "last_destroyed": 4, "snaps": [ { "ino": 1099511627778, "metadata": {}, "name": "s2", "snapid": 3, "stamp": "2024-09-21T07:50:02.909971+0000" } ] } 2024-09-21T07:50:40.077 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:40.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.302+0000 7f873b800640 1 -- 172.21.15.39:0/4012570508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f871c004800 msgr2=0x7f871c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:40.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.302+0000 7f873b800640 1 --2- 172.21.15.39:0/4012570508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f871c004800 0x7f871c004c10 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7f872000c390 tx=0x7f871c005830 comp rx=0 tx=0).stop 2024-09-21T07:50:40.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.302+0000 7f8739800640 1 -- 172.21.15.39:0/4012570508 reap_dead start 2024-09-21T07:50:40.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.302+0000 7f873b800640 1 -- 172.21.15.39:0/4012570508 shutdown_connections 2024-09-21T07:50:40.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.302+0000 7f873b800640 1 -- 172.21.15.39:0/4012570508 >> 172.21.15.39:0/4012570508 conn(0x7f87340b2460 msgr2=0x7f873406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:40.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.302+0000 7f873b800640 1 -- 172.21.15.39:0/4012570508 shutdown_connections 2024-09-21T07:50:40.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.302+0000 7f873b800640 1 -- 172.21.15.39:0/4012570508 wait complete. 2024-09-21T07:50:40.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.305+0000 7f873b800640 1 Processor -- start 2024-09-21T07:50:40.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.305+0000 7f873b800640 1 -- start start 2024-09-21T07:50:40.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.306+0000 7f873b800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f871c004800 0x7f87341f52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:40.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.306+0000 7f873b800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87341f9db0 0x7f87340b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:40.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.306+0000 7f873b800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87340b0e20 0x7f87340cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:40.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.306+0000 7f873b800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f87340af080 con 0x7f87340b0e20 2024-09-21T07:50:40.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.306+0000 7f873b800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f87340cd3c0 con 0x7f871c004800 2024-09-21T07:50:40.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.306+0000 7f873b800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f87340af200 con 0x7f87341f9db0 2024-09-21T07:50:40.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.306+0000 7f8739800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87340b0e20 0x7f87340cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:40.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.306+0000 7f8739800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87340b0e20 0x7f87340cf0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50776/0 (socket says 172.21.15.39:50776) 2024-09-21T07:50:40.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.306+0000 7f8739800640 1 -- 172.21.15.39:0/2563125511 learned_addr learned my addr 172.21.15.39:0/2563125511 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:40.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.307+0000 7f8733e00640 1 --2- 172.21.15.39:0/2563125511 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87341f9db0 0x7f87340b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:40.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.307+0000 7f8738e00640 1 --2- 172.21.15.39:0/2563125511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f871c004800 0x7f87341f52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:40.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.307+0000 7f8739800640 1 -- 172.21.15.39:0/2563125511 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87341f9db0 msgr2=0x7f87340b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:40.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.307+0000 7f8739800640 1 --2- 172.21.15.39:0/2563125511 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87341f9db0 0x7f87340b08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:40.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.307+0000 7f8739800640 1 -- 172.21.15.39:0/2563125511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f871c004800 msgr2=0x7f87341f52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:40.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.307+0000 7f8739800640 1 --2- 172.21.15.39:0/2563125511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f871c004800 0x7f87341f52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:40.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.307+0000 7f8739800640 1 -- 172.21.15.39:0/2563125511 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87340b1230 con 0x7f87340b0e20 2024-09-21T07:50:40.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.307+0000 7f8733e00640 1 --2- 172.21.15.39:0/2563125511 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87341f9db0 0x7f87340b08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:50:40.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.307+0000 7f8739800640 1 --2- 172.21.15.39:0/2563125511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87340b0e20 0x7f87340cf0b0 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7f87340b5090 tx=0x7f872c011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:40.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.307+0000 7f8730c00640 1 -- 172.21.15.39:0/2563125511 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872c003970 con 0x7f87340b0e20 2024-09-21T07:50:40.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.307+0000 7f8730c00640 1 -- 172.21.15.39:0/2563125511 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f872c003b50 con 0x7f87340b0e20 2024-09-21T07:50:40.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.307+0000 7f873b800640 1 -- 172.21.15.39:0/2563125511 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87340cfd10 con 0x7f87340b0e20 2024-09-21T07:50:40.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.307+0000 7f8730c00640 1 -- 172.21.15.39:0/2563125511 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872c004030 con 0x7f87340b0e20 2024-09-21T07:50:40.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.308+0000 7f873b800640 1 -- 172.21.15.39:0/2563125511 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87341f95b0 con 0x7f87340b0e20 2024-09-21T07:50:40.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.309+0000 7f873b800640 1 -- 172.21.15.39:0/2563125511 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87340af680 con 0x7f87340b0e20 2024-09-21T07:50:40.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.310+0000 7f8730c00640 1 -- 172.21.15.39:0/2563125511 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f872c0041d0 con 0x7f87340b0e20 2024-09-21T07:50:40.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.311+0000 7f8730c00640 1 --2- 172.21.15.39:0/2563125511 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8714078b90 0x7f871407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:40.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.311+0000 7f8730c00640 1 -- 172.21.15.39:0/2563125511 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(240..240 src has 1..240) ==== 7390+0+0 (secure 0 0 0) 0x7f872c09c0e0 con 0x7f87340b0e20 2024-09-21T07:50:40.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.311+0000 7f8738e00640 1 --2- 172.21.15.39:0/2563125511 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8714078b90 0x7f871407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:40.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.312+0000 7f8738e00640 1 --2- 172.21.15.39:0/2563125511 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8714078b90 0x7f871407b0a0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f8728000c00 tx=0x7f8728006b10 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:40.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.315+0000 7f8730c00640 1 -- 172.21.15.39:0/2563125511 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f872c020060 con 0x7f87340b0e20 2024-09-21T07:50:40.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.553+0000 7f873b800640 1 -- 172.21.15.39:0/2563125511 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87340b4b60 con 0x7f87340b0e20 2024-09-21T07:50:40.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.554+0000 7f8730c00640 1 -- 172.21.15.39:0/2563125511 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 265 v265) ==== 95+0+5279 (secure 0 0 0) 0x7f87340b1230 con 0x7f87340b0e20 2024-09-21T07:50:40.556 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:50:40.556 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":265,"btime":"2024-09-21T07:50:36:415072+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11067,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/109229385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":109229385},{"type":"v1","addr":"172.21.15.39:6834","nonce":109229385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":262}],"filesystems":[{"mdsmap":{"epoch":265,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:50:36.415066+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":240,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":8531},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8531":{"gid":8531,"name":"d","rank":2,"incarnation":253,"state":"up:active","state_seq":18,"addr":"172.21.15.148:6837/1438851059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1438851059},{"type":"v1","addr":"172.21.15.148:6837","nonce":1438851059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[8531,9967,11052]},"id":13}]} 2024-09-21T07:50:40.556 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 265 2024-09-21T07:50:40.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.557+0000 7f873b800640 1 -- 172.21.15.39:0/2563125511 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8714078b90 msgr2=0x7f871407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:40.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.557+0000 7f873b800640 1 --2- 172.21.15.39:0/2563125511 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8714078b90 0x7f871407b0a0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f8728000c00 tx=0x7f8728006b10 comp rx=0 tx=0).stop 2024-09-21T07:50:40.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.557+0000 7f873b800640 1 -- 172.21.15.39:0/2563125511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87340b0e20 msgr2=0x7f87340cf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:40.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.557+0000 7f873b800640 1 --2- 172.21.15.39:0/2563125511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87340b0e20 0x7f87340cf0b0 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7f87340b5090 tx=0x7f872c011f80 comp rx=0 tx=0).stop 2024-09-21T07:50:40.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.557+0000 7f873b800640 1 -- 172.21.15.39:0/2563125511 shutdown_connections 2024-09-21T07:50:40.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.557+0000 7f873b800640 1 --2- 172.21.15.39:0/2563125511 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8714078b90 0x7f871407b0a0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:40.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.557+0000 7f873b800640 1 --2- 172.21.15.39:0/2563125511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87340b0e20 0x7f87340cf0b0 unknown :-1 s=CLOSED pgs=2388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:40.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.557+0000 7f873b800640 1 --2- 172.21.15.39:0/2563125511 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87341f9db0 0x7f87340b08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:40.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.557+0000 7f873b800640 1 --2- 172.21.15.39:0/2563125511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f871c004800 0x7f87341f52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:40.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.557+0000 7f873b800640 1 -- 172.21.15.39:0/2563125511 >> 172.21.15.39:0/2563125511 conn(0x7f87340b2460 msgr2=0x7f87340ae8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:40.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.557+0000 7f873b800640 1 -- 172.21.15.39:0/2563125511 shutdown_connections 2024-09-21T07:50:40.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:40.557+0000 7f873b800640 1 -- 172.21.15.39:0/2563125511 wait complete. 2024-09-21T07:50:40.581 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json dump snaps 2024-09-21T07:50:40.830 INFO:teuthology.orchestra.run.smithi148.stdout:{"last_created":3,"last_destroyed":4,"snaps":[{"snapid":3,"ino":1099511627778,"stamp":"2024-09-21T07:50:02.909971+0000","name":"s2","metadata":{}}]} 2024-09-21T07:50:40.830 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 3, "last_destroyed": 4, "snaps": [ { "ino": 1099511627778, "metadata": {}, "name": "s2", "snapid": 3, "stamp": "2024-09-21T07:50:02.909971+0000" } ] } 2024-09-21T07:50:40.830 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:40.880 INFO:tasks.ceph.mds.b.smithi148.stderr:2024-09-21T07:50:40.878+0000 7fa0be800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:50:40.880 INFO:tasks.ceph.mds.b.smithi148.stderr:2024-09-21T07:50:40.878+0000 7fa0be800640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:50:40.880 INFO:tasks.ceph.mds.b.smithi148.stderr:2024-09-21T07:50:40.878+0000 7fa0be800640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-21T07:50:41.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.056+0000 7f6a2ca00640 1 -- 172.21.15.39:0/436955156 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a0c003c30 msgr2=0x7f6a0c0040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:41.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.056+0000 7f6a2ca00640 1 --2- 172.21.15.39:0/436955156 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a0c003c30 0x7f6a0c0040a0 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7f6a1800c3c0 tx=0x7f6a0c005830 comp rx=0 tx=0).stop 2024-09-21T07:50:41.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.056+0000 7f6a2ca00640 1 -- 172.21.15.39:0/436955156 shutdown_connections 2024-09-21T07:50:41.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.056+0000 7f6a2ca00640 1 --2- 172.21.15.39:0/436955156 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a0c003c30 0x7f6a0c0040a0 unknown :-1 s=CLOSED pgs=2390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:41.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.056+0000 7f6a2ca00640 1 -- 172.21.15.39:0/436955156 >> 172.21.15.39:0/436955156 conn(0x7f6a280b2460 msgr2=0x7f6a2806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:41.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.056+0000 7f6a2ca00640 1 -- 172.21.15.39:0/436955156 shutdown_connections 2024-09-21T07:50:41.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.056+0000 7f6a2ca00640 1 -- 172.21.15.39:0/436955156 wait complete. 2024-09-21T07:50:41.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.058+0000 7f6a2ca00640 1 Processor -- start 2024-09-21T07:50:41.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.059+0000 7f6a2ca00640 1 -- start start 2024-09-21T07:50:41.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.060+0000 7f6a2ca00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a0c003c30 0x7f6a280cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:41.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.060+0000 7f6a2ca00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a281f5800 0x7f6a280ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:41.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.060+0000 7f6a2ca00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a280b3bc0 0x7f6a280cea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:41.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.060+0000 7f6a2ca00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6a28156bb0 con 0x7f6a0c003c30 2024-09-21T07:50:41.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.060+0000 7f6a2ca00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6a280cd390 con 0x7f6a280b3bc0 2024-09-21T07:50:41.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.060+0000 7f6a2ca00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f6a28156d30 con 0x7f6a281f5800 2024-09-21T07:50:41.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.060+0000 7f6a25e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a0c003c30 0x7f6a280cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:41.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.060+0000 7f6a25e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a0c003c30 0x7f6a280cdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50786/0 (socket says 172.21.15.39:50786) 2024-09-21T07:50:41.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.060+0000 7f6a25e00640 1 -- 172.21.15.39:0/3729796265 learned_addr learned my addr 172.21.15.39:0/3729796265 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:41.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.061+0000 7f6a25400640 1 --2- 172.21.15.39:0/3729796265 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a281f5800 0x7f6a280ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:41.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.061+0000 7f6a26800640 1 --2- 172.21.15.39:0/3729796265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a280b3bc0 0x7f6a280cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:41.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.061+0000 7f6a25e00640 1 -- 172.21.15.39:0/3729796265 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a281f5800 msgr2=0x7f6a280ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:41.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.061+0000 7f6a25e00640 1 --2- 172.21.15.39:0/3729796265 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a281f5800 0x7f6a280ce4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:41.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.061+0000 7f6a25e00640 1 -- 172.21.15.39:0/3729796265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a280b3bc0 msgr2=0x7f6a280cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:41.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.061+0000 7f6a25e00640 1 --2- 172.21.15.39:0/3729796265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a280b3bc0 0x7f6a280cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:41.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.061+0000 7f6a25e00640 1 -- 172.21.15.39:0/3729796265 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a280ca210 con 0x7f6a0c003c30 2024-09-21T07:50:41.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.061+0000 7f6a26800640 1 --2- 172.21.15.39:0/3729796265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a280b3bc0 0x7f6a280cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:50:41.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.061+0000 7f6a25e00640 1 --2- 172.21.15.39:0/3729796265 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a0c003c30 0x7f6a280cdf80 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7f6a1800c4e0 tx=0x7f6a18001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:41.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.061+0000 7f6a1e000640 1 -- 172.21.15.39:0/3729796265 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a180037f0 con 0x7f6a0c003c30 2024-09-21T07:50:41.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.061+0000 7f6a1e000640 1 -- 172.21.15.39:0/3729796265 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a1803a070 con 0x7f6a0c003c30 2024-09-21T07:50:41.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.061+0000 7f6a1e000640 1 -- 172.21.15.39:0/3729796265 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a18003d00 con 0x7f6a0c003c30 2024-09-21T07:50:41.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.062+0000 7f6a2ca00640 1 -- 172.21.15.39:0/3729796265 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a280b0c10 con 0x7f6a0c003c30 2024-09-21T07:50:41.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.062+0000 7f6a2ca00640 1 -- 172.21.15.39:0/3729796265 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a280c6040 con 0x7f6a0c003c30 2024-09-21T07:50:41.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.063+0000 7f6a2ca00640 1 -- 172.21.15.39:0/3729796265 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a280c4be0 con 0x7f6a0c003c30 2024-09-21T07:50:41.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.064+0000 7f6a1e000640 1 -- 172.21.15.39:0/3729796265 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6a1802b5b0 con 0x7f6a0c003c30 2024-09-21T07:50:41.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.065+0000 7f6a1e000640 1 --2- 172.21.15.39:0/3729796265 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6a08078bc0 0x7f6a0807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:41.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.065+0000 7f6a1e000640 1 -- 172.21.15.39:0/3729796265 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(240..240 src has 1..240) ==== 7390+0+0 (secure 0 0 0) 0x7f6a180bb320 con 0x7f6a0c003c30 2024-09-21T07:50:41.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.065+0000 7f6a25400640 1 --2- 172.21.15.39:0/3729796265 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6a08078bc0 0x7f6a0807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:41.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.066+0000 7f6a25400640 1 --2- 172.21.15.39:0/3729796265 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6a08078bc0 0x7f6a0807b0d0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f6a280b1a80 tx=0x7f6a14006b10 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:41.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.069+0000 7f6a1e000640 1 -- 172.21.15.39:0/3729796265 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6a280cd5b0 con 0x7f6a0c003c30 2024-09-21T07:50:41.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.302+0000 7f6a2ca00640 1 -- 172.21.15.39:0/3729796265 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a28154800 con 0x7f6a0c003c30 2024-09-21T07:50:41.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.302+0000 7f6a1e000640 1 -- 172.21.15.39:0/3729796265 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 265 v265) ==== 95+0+5279 (secure 0 0 0) 0x7f6a180830f0 con 0x7f6a0c003c30 2024-09-21T07:50:41.304 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:50:41.304 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":265,"btime":"2024-09-21T07:50:36:415072+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11067,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/109229385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":109229385},{"type":"v1","addr":"172.21.15.39:6834","nonce":109229385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":262}],"filesystems":[{"mdsmap":{"epoch":265,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:50:36.415066+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":240,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":8531},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8531":{"gid":8531,"name":"d","rank":2,"incarnation":253,"state":"up:active","state_seq":18,"addr":"172.21.15.148:6837/1438851059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1438851059},{"type":"v1","addr":"172.21.15.148:6837","nonce":1438851059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[8531,9967,11052]},"id":13}]} 2024-09-21T07:50:41.305 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 265 2024-09-21T07:50:41.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.304+0000 7f6a2ca00640 1 -- 172.21.15.39:0/3729796265 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6a08078bc0 msgr2=0x7f6a0807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:41.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.304+0000 7f6a2ca00640 1 --2- 172.21.15.39:0/3729796265 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6a08078bc0 0x7f6a0807b0d0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f6a280b1a80 tx=0x7f6a14006b10 comp rx=0 tx=0).stop 2024-09-21T07:50:41.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.304+0000 7f6a2ca00640 1 -- 172.21.15.39:0/3729796265 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a0c003c30 msgr2=0x7f6a280cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:41.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.304+0000 7f6a2ca00640 1 --2- 172.21.15.39:0/3729796265 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a0c003c30 0x7f6a280cdf80 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7f6a1800c4e0 tx=0x7f6a18001360 comp rx=0 tx=0).stop 2024-09-21T07:50:41.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.305+0000 7f6a2ca00640 1 -- 172.21.15.39:0/3729796265 shutdown_connections 2024-09-21T07:50:41.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.305+0000 7f6a2ca00640 1 --2- 172.21.15.39:0/3729796265 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6a08078bc0 0x7f6a0807b0d0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:41.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.305+0000 7f6a2ca00640 1 --2- 172.21.15.39:0/3729796265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a280b3bc0 0x7f6a280cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:41.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.305+0000 7f6a2ca00640 1 --2- 172.21.15.39:0/3729796265 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a281f5800 0x7f6a280ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:41.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.305+0000 7f6a2ca00640 1 --2- 172.21.15.39:0/3729796265 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a0c003c30 0x7f6a280cdf80 unknown :-1 s=CLOSED pgs=2391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:41.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.305+0000 7f6a2ca00640 1 -- 172.21.15.39:0/3729796265 >> 172.21.15.39:0/3729796265 conn(0x7f6a280b2460 msgr2=0x7f6a280b34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:41.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.305+0000 7f6a2ca00640 1 -- 172.21.15.39:0/3729796265 shutdown_connections 2024-09-21T07:50:41.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.305+0000 7f6a2ca00640 1 -- 172.21.15.39:0/3729796265 wait complete. 2024-09-21T07:50:41.329 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 13:2 true 2024-09-21T07:50:41.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.556+0000 7f36e7200640 1 -- 172.21.15.39:0/1316218583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36e80d1580 msgr2=0x7f36e80d1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:41.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.556+0000 7f36e7200640 1 --2- 172.21.15.39:0/1316218583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36e80d1580 0x7f36e80d1ac0 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7f36e808e1d0 tx=0x7f36d002a5f0 comp rx=0 tx=0).stop 2024-09-21T07:50:41.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.556+0000 7f36e7200640 1 -- 172.21.15.39:0/1316218583 shutdown_connections 2024-09-21T07:50:41.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.556+0000 7f36e7200640 1 --2- 172.21.15.39:0/1316218583 >> v2:172.21.15.148:3300/0 conn(0x7f36e80b3b60 0x7f36e80b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:41.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.556+0000 7f36e7200640 1 --2- 172.21.15.39:0/1316218583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36e80d1580 0x7f36e80d1ac0 unknown :-1 s=CLOSED pgs=2392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:41.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.556+0000 7f36e7200640 1 -- 172.21.15.39:0/1316218583 >> 172.21.15.39:0/1316218583 conn(0x7f36e809d7b0 msgr2=0x7f36e80d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:41.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.556+0000 7f36e7200640 1 -- 172.21.15.39:0/1316218583 shutdown_connections 2024-09-21T07:50:41.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.556+0000 7f36e7200640 1 -- 172.21.15.39:0/1316218583 wait complete. 2024-09-21T07:50:41.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.558+0000 7f36e7200640 1 Processor -- start 2024-09-21T07:50:41.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.558+0000 7f36e7200640 1 -- start start 2024-09-21T07:50:41.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.560+0000 7f36e7200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36e80b3b60 0x7f36e80cbf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:41.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.560+0000 7f36e7200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36e80b5a90 0x7f36e815c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:41.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.560+0000 7f36e7200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36e815cea0 0x7f36e81f84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:41.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.560+0000 7f36e7200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f36e80b4630 con 0x7f36e815cea0 2024-09-21T07:50:41.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.560+0000 7f36e7200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f36e80b44b0 con 0x7f36e80b5a90 2024-09-21T07:50:41.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.560+0000 7f36e7200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f36e80b47b0 con 0x7f36e80b3b60 2024-09-21T07:50:41.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.560+0000 7f36e6800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36e815cea0 0x7f36e81f84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:41.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.560+0000 7f36e6800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36e815cea0 0x7f36e81f84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50806/0 (socket says 172.21.15.39:50806) 2024-09-21T07:50:41.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.560+0000 7f36e6800640 1 -- 172.21.15.39:0/472423037 learned_addr learned my addr 172.21.15.39:0/472423037 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:41.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.560+0000 7f36e5e00640 1 --2- 172.21.15.39:0/472423037 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36e80b3b60 0x7f36e80cbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:41.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.560+0000 7f36dfe00640 1 --2- 172.21.15.39:0/472423037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36e80b5a90 0x7f36e815c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:41.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.561+0000 7f36e6800640 1 -- 172.21.15.39:0/472423037 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36e80b3b60 msgr2=0x7f36e80cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:41.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.561+0000 7f36e6800640 1 --2- 172.21.15.39:0/472423037 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36e80b3b60 0x7f36e80cbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:41.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.561+0000 7f36e6800640 1 -- 172.21.15.39:0/472423037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36e80b5a90 msgr2=0x7f36e815c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:41.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.561+0000 7f36e6800640 1 --2- 172.21.15.39:0/472423037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36e80b5a90 0x7f36e815c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:41.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.561+0000 7f36e6800640 1 -- 172.21.15.39:0/472423037 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36e80cc820 con 0x7f36e815cea0 2024-09-21T07:50:41.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.561+0000 7f36e6800640 1 --2- 172.21.15.39:0/472423037 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36e815cea0 0x7f36e81f84e0 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7f36e000ea70 tx=0x7f36e000ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:41.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.561+0000 7f36de000640 1 -- 172.21.15.39:0/472423037 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e000ce40 con 0x7f36e815cea0 2024-09-21T07:50:41.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.561+0000 7f36de000640 1 -- 172.21.15.39:0/472423037 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36e0004c10 con 0x7f36e815cea0 2024-09-21T07:50:41.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.561+0000 7f36de000640 1 -- 172.21.15.39:0/472423037 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e0005020 con 0x7f36e815cea0 2024-09-21T07:50:41.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.561+0000 7f36e7200640 1 -- 172.21.15.39:0/472423037 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36e80ccb00 con 0x7f36e815cea0 2024-09-21T07:50:41.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.562+0000 7f36e7200640 1 -- 172.21.15.39:0/472423037 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36e80cd0c0 con 0x7f36e815cea0 2024-09-21T07:50:41.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.563+0000 7f36e7200640 1 -- 172.21.15.39:0/472423037 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36e80b4c60 con 0x7f36e815cea0 2024-09-21T07:50:41.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.564+0000 7f36de000640 1 -- 172.21.15.39:0/472423037 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f36e00051c0 con 0x7f36e815cea0 2024-09-21T07:50:41.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.565+0000 7f36de000640 1 --2- 172.21.15.39:0/472423037 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f36c4078b90 0x7f36c407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:41.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.565+0000 7f36de000640 1 -- 172.21.15.39:0/472423037 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(240..240 src has 1..240) ==== 7390+0+0 (secure 0 0 0) 0x7f36e009c0a0 con 0x7f36e815cea0 2024-09-21T07:50:41.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.565+0000 7f36e5e00640 1 --2- 172.21.15.39:0/472423037 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f36c4078b90 0x7f36c407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:41.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.566+0000 7f36e5e00640 1 --2- 172.21.15.39:0/472423037 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f36c4078b90 0x7f36c407b0a0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f36d8007fd0 tx=0x7f36d8009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:41.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.569+0000 7f36de000640 1 -- 172.21.15.39:0/472423037 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f36e00637f0 con 0x7f36e815cea0 2024-09-21T07:50:41.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:41.842+0000 7f36e7200640 1 -- 172.21.15.39:0/472423037 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "13:2", "val": "true"} v 0) -- 0x7f36e80b5480 con 0x7f36e815cea0 2024-09-21T07:50:42.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:42.444+0000 7f36de000640 1 -- 172.21.15.39:0/472423037 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "13:2", "val": "true"}]=0 freezing mds.8531 v266) ==== 113+0+0 (secure 0 0 0) 0x7f36e0068220 con 0x7f36e815cea0 2024-09-21T07:50:42.447 INFO:teuthology.orchestra.run.smithi039.stderr:freezing mds.8531 2024-09-21T07:50:42.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:42.447+0000 7f36e7200640 1 -- 172.21.15.39:0/472423037 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f36c4078b90 msgr2=0x7f36c407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:42.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:42.447+0000 7f36e7200640 1 --2- 172.21.15.39:0/472423037 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f36c4078b90 0x7f36c407b0a0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f36d8007fd0 tx=0x7f36d8009040 comp rx=0 tx=0).stop 2024-09-21T07:50:42.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:42.447+0000 7f36e7200640 1 -- 172.21.15.39:0/472423037 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36e815cea0 msgr2=0x7f36e81f84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:42.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:42.447+0000 7f36e7200640 1 --2- 172.21.15.39:0/472423037 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36e815cea0 0x7f36e81f84e0 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7f36e000ea70 tx=0x7f36e000ef30 comp rx=0 tx=0).stop 2024-09-21T07:50:42.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:42.447+0000 7f36e7200640 1 -- 172.21.15.39:0/472423037 shutdown_connections 2024-09-21T07:50:42.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:42.447+0000 7f36e7200640 1 --2- 172.21.15.39:0/472423037 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f36c4078b90 0x7f36c407b0a0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:42.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:42.447+0000 7f36e7200640 1 --2- 172.21.15.39:0/472423037 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36e815cea0 0x7f36e81f84e0 unknown :-1 s=CLOSED pgs=2393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:42.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:42.447+0000 7f36e7200640 1 --2- 172.21.15.39:0/472423037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36e80b5a90 0x7f36e815c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:42.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:42.447+0000 7f36e7200640 1 --2- 172.21.15.39:0/472423037 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f36e80b3b60 0x7f36e80cbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:42.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:42.448+0000 7f36e7200640 1 -- 172.21.15.39:0/472423037 >> 172.21.15.39:0/472423037 conn(0x7f36e809d7b0 msgr2=0x7f36e806f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:42.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:42.448+0000 7f36e7200640 1 -- 172.21.15.39:0/472423037 shutdown_connections 2024-09-21T07:50:42.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:42.448+0000 7f36e7200640 1 -- 172.21.15.39:0/472423037 wait complete. 2024-09-21T07:50:42.476 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json config set mds_kill_mdstable_at 9 2024-09-21T07:50:42.722 INFO:teuthology.orchestra.run.smithi148.stdout:{"success":"mds_kill_mdstable_at = '' (not observed, change may require restart) "} 2024-09-21T07:50:42.722 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_kill_mdstable_at = '' (not observed, change may require restart) " } 2024-09-21T07:50:42.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:50:42.723 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d0/d1/dir/.snap/s3) 2024-09-21T07:50:42.727 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-09-21T07:50:42.765 INFO:tasks.ceph.mds.d.smithi148.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::handle_request(ceph::cref_t&)' thread 7fcd64000640 time 2024-09-21T07:50:42.764910+0000 2024-09-21T07:50:42.765 INFO:tasks.ceph.mds.d.smithi148.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: 68: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 9) 2024-09-21T07:50:42.766 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:50:42.766 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fcd6b05654b] 2024-09-21T07:50:42.766 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7fcd6b056758] 2024-09-21T07:50:42.766 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x301) [0x5589bcede505] 2024-09-21T07:50:42.766 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xc96) [0x5589bcb76dd4] 2024-09-21T07:50:42.766 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5589bcb7a523] 2024-09-21T07:50:42.766 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5589bcb7ae4a] 2024-09-21T07:50:42.766 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x5589bcb5dfa4] 2024-09-21T07:50:42.767 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7fcd6b1a57df] 2024-09-21T07:50:42.767 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: (DispatchQueue::entry()+0x60f) [0x7fcd6b1a60d1] 2024-09-21T07:50:42.767 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7fcd6b293f5b] 2024-09-21T07:50:42.767 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (Thread::entry_wrapper()+0x43) [0x7fcd6b02fd57] 2024-09-21T07:50:42.767 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: (Thread::_entry_func(void*)+0xd) [0x7fcd6b02fd73] 2024-09-21T07:50:42.767 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: /lib64/libc.so.6(+0x89c52) [0x7fcd6a689c52] 2024-09-21T07:50:42.767 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: /lib64/libc.so.6(+0x10ec80) [0x7fcd6a70ec80] 2024-09-21T07:50:42.767 INFO:tasks.ceph.mds.d.smithi148.stderr:2024-09-21T07:50:42.764+0000 7fcd64000640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::handle_request(ceph::cref_t&)' thread 7fcd64000640 time 2024-09-21T07:50:42.764910+0000 2024-09-21T07:50:42.767 INFO:tasks.ceph.mds.d.smithi148.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: 68: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 9) 2024-09-21T07:50:42.767 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:50:42.767 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:50:42.767 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fcd6b05654b] 2024-09-21T07:50:42.768 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7fcd6b056758] 2024-09-21T07:50:42.768 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x301) [0x5589bcede505] 2024-09-21T07:50:42.768 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xc96) [0x5589bcb76dd4] 2024-09-21T07:50:42.768 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5589bcb7a523] 2024-09-21T07:50:42.768 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5589bcb7ae4a] 2024-09-21T07:50:42.768 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x5589bcb5dfa4] 2024-09-21T07:50:42.768 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7fcd6b1a57df] 2024-09-21T07:50:42.768 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: (DispatchQueue::entry()+0x60f) [0x7fcd6b1a60d1] 2024-09-21T07:50:42.768 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7fcd6b293f5b] 2024-09-21T07:50:42.768 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (Thread::entry_wrapper()+0x43) [0x7fcd6b02fd57] 2024-09-21T07:50:42.768 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: (Thread::_entry_func(void*)+0xd) [0x7fcd6b02fd73] 2024-09-21T07:50:42.768 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: /lib64/libc.so.6(+0x89c52) [0x7fcd6a689c52] 2024-09-21T07:50:42.768 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: /lib64/libc.so.6(+0x10ec80) [0x7fcd6a70ec80] 2024-09-21T07:50:42.769 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:50:42.769 INFO:tasks.ceph.mds.d.smithi148.stderr:*** Caught signal (Aborted) ** 2024-09-21T07:50:42.769 INFO:tasks.ceph.mds.d.smithi148.stderr: in thread 7fcd64000640 thread_name:ms_dispatch 2024-09-21T07:50:42.769 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:50:42.769 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fcd6a63e6f0] 2024-09-21T07:50:42.769 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7fcd6a68b99c] 2024-09-21T07:50:42.769 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: raise() 2024-09-21T07:50:42.769 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: abort() 2024-09-21T07:50:42.769 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7fcd6b056686] 2024-09-21T07:50:42.769 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7fcd6b056758] 2024-09-21T07:50:42.770 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x301) [0x5589bcede505] 2024-09-21T07:50:42.770 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xc96) [0x5589bcb76dd4] 2024-09-21T07:50:42.770 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5589bcb7a523] 2024-09-21T07:50:42.770 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5589bcb7ae4a] 2024-09-21T07:50:42.770 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x5589bcb5dfa4] 2024-09-21T07:50:42.770 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7fcd6b1a57df] 2024-09-21T07:50:42.770 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: (DispatchQueue::entry()+0x60f) [0x7fcd6b1a60d1] 2024-09-21T07:50:42.770 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7fcd6b293f5b] 2024-09-21T07:50:42.770 INFO:tasks.ceph.mds.d.smithi148.stderr: 15: (Thread::entry_wrapper()+0x43) [0x7fcd6b02fd57] 2024-09-21T07:50:42.770 INFO:tasks.ceph.mds.d.smithi148.stderr: 16: (Thread::_entry_func(void*)+0xd) [0x7fcd6b02fd73] 2024-09-21T07:50:42.770 INFO:tasks.ceph.mds.d.smithi148.stderr: 17: /lib64/libc.so.6(+0x89c52) [0x7fcd6a689c52] 2024-09-21T07:50:42.770 INFO:tasks.ceph.mds.d.smithi148.stderr: 18: /lib64/libc.so.6(+0x10ec80) [0x7fcd6a70ec80] 2024-09-21T07:50:42.770 INFO:tasks.ceph.mds.d.smithi148.stderr:2024-09-21T07:50:42.765+0000 7fcd64000640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:50:42.770 INFO:tasks.ceph.mds.d.smithi148.stderr: in thread 7fcd64000640 thread_name:ms_dispatch 2024-09-21T07:50:42.771 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:50:42.771 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:50:42.771 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fcd6a63e6f0] 2024-09-21T07:50:42.771 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7fcd6a68b99c] 2024-09-21T07:50:42.771 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: raise() 2024-09-21T07:50:42.771 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: abort() 2024-09-21T07:50:42.771 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7fcd6b056686] 2024-09-21T07:50:42.771 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7fcd6b056758] 2024-09-21T07:50:42.771 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x301) [0x5589bcede505] 2024-09-21T07:50:42.771 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xc96) [0x5589bcb76dd4] 2024-09-21T07:50:42.771 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5589bcb7a523] 2024-09-21T07:50:42.771 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5589bcb7ae4a] 2024-09-21T07:50:42.771 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x5589bcb5dfa4] 2024-09-21T07:50:42.771 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7fcd6b1a57df] 2024-09-21T07:50:42.771 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: (DispatchQueue::entry()+0x60f) [0x7fcd6b1a60d1] 2024-09-21T07:50:42.771 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7fcd6b293f5b] 2024-09-21T07:50:42.772 INFO:tasks.ceph.mds.d.smithi148.stderr: 15: (Thread::entry_wrapper()+0x43) [0x7fcd6b02fd57] 2024-09-21T07:50:42.772 INFO:tasks.ceph.mds.d.smithi148.stderr: 16: (Thread::_entry_func(void*)+0xd) [0x7fcd6b02fd73] 2024-09-21T07:50:42.772 INFO:tasks.ceph.mds.d.smithi148.stderr: 17: /lib64/libc.so.6(+0x89c52) [0x7fcd6a689c52] 2024-09-21T07:50:42.772 INFO:tasks.ceph.mds.d.smithi148.stderr: 18: /lib64/libc.so.6(+0x10ec80) [0x7fcd6a70ec80] 2024-09-21T07:50:42.772 INFO:tasks.ceph.mds.d.smithi148.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:50:42.772 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:50:42.772 INFO:tasks.ceph.mds.d.smithi148.stderr: -3322> 2024-09-21T07:49:30.107+0000 7fcd68600640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-09-21T07:50:42.778 INFO:tasks.ceph.mds.d.smithi148.stderr: -1> 2024-09-21T07:50:42.764+0000 7fcd64000640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::handle_request(ceph::cref_t&)' thread 7fcd64000640 time 2024-09-21T07:50:42.764910+0000 2024-09-21T07:50:42.778 INFO:tasks.ceph.mds.d.smithi148.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: 68: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 9) 2024-09-21T07:50:42.778 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:50:42.778 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:50:42.778 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fcd6b05654b] 2024-09-21T07:50:42.778 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7fcd6b056758] 2024-09-21T07:50:42.778 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x301) [0x5589bcede505] 2024-09-21T07:50:42.778 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xc96) [0x5589bcb76dd4] 2024-09-21T07:50:42.778 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5589bcb7a523] 2024-09-21T07:50:42.778 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5589bcb7ae4a] 2024-09-21T07:50:42.778 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x5589bcb5dfa4] 2024-09-21T07:50:42.779 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7fcd6b1a57df] 2024-09-21T07:50:42.779 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: (DispatchQueue::entry()+0x60f) [0x7fcd6b1a60d1] 2024-09-21T07:50:42.779 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7fcd6b293f5b] 2024-09-21T07:50:42.779 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (Thread::entry_wrapper()+0x43) [0x7fcd6b02fd57] 2024-09-21T07:50:42.779 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: (Thread::_entry_func(void*)+0xd) [0x7fcd6b02fd73] 2024-09-21T07:50:42.779 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: /lib64/libc.so.6(+0x89c52) [0x7fcd6a689c52] 2024-09-21T07:50:42.779 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: /lib64/libc.so.6(+0x10ec80) [0x7fcd6a70ec80] 2024-09-21T07:50:42.779 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:50:42.779 INFO:tasks.ceph.mds.d.smithi148.stderr: 0> 2024-09-21T07:50:42.765+0000 7fcd64000640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:50:42.779 INFO:tasks.ceph.mds.d.smithi148.stderr: in thread 7fcd64000640 thread_name:ms_dispatch 2024-09-21T07:50:42.779 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:50:42.779 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:50:42.780 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fcd6a63e6f0] 2024-09-21T07:50:42.780 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7fcd6a68b99c] 2024-09-21T07:50:42.780 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: raise() 2024-09-21T07:50:42.780 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: abort() 2024-09-21T07:50:42.780 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7fcd6b056686] 2024-09-21T07:50:42.780 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7fcd6b056758] 2024-09-21T07:50:42.780 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x301) [0x5589bcede505] 2024-09-21T07:50:42.780 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xc96) [0x5589bcb76dd4] 2024-09-21T07:50:42.780 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5589bcb7a523] 2024-09-21T07:50:42.780 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5589bcb7ae4a] 2024-09-21T07:50:42.780 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x5589bcb5dfa4] 2024-09-21T07:50:42.780 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7fcd6b1a57df] 2024-09-21T07:50:42.780 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: (DispatchQueue::entry()+0x60f) [0x7fcd6b1a60d1] 2024-09-21T07:50:42.780 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7fcd6b293f5b] 2024-09-21T07:50:42.780 INFO:tasks.ceph.mds.d.smithi148.stderr: 15: (Thread::entry_wrapper()+0x43) [0x7fcd6b02fd57] 2024-09-21T07:50:42.781 INFO:tasks.ceph.mds.d.smithi148.stderr: 16: (Thread::_entry_func(void*)+0xd) [0x7fcd6b02fd73] 2024-09-21T07:50:42.781 INFO:tasks.ceph.mds.d.smithi148.stderr: 17: /lib64/libc.so.6(+0x89c52) [0x7fcd6a689c52] 2024-09-21T07:50:42.781 INFO:tasks.ceph.mds.d.smithi148.stderr: 18: /lib64/libc.so.6(+0x10ec80) [0x7fcd6a70ec80] 2024-09-21T07:50:42.781 INFO:tasks.ceph.mds.d.smithi148.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:50:42.781 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:50:42.781 INFO:tasks.ceph.mds.d.smithi148.stderr: -3322> 2024-09-21T07:49:30.107+0000 7fcd68600640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-09-21T07:50:42.785 INFO:tasks.ceph.mds.d.smithi148.stderr: -1> 2024-09-21T07:50:42.764+0000 7fcd64000640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::handle_request(ceph::cref_t&)' thread 7fcd64000640 time 2024-09-21T07:50:42.764910+0000 2024-09-21T07:50:42.785 INFO:tasks.ceph.mds.d.smithi148.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: 68: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 9) 2024-09-21T07:50:42.785 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:50:42.785 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:50:42.785 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fcd6b05654b] 2024-09-21T07:50:42.785 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7fcd6b056758] 2024-09-21T07:50:42.785 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x301) [0x5589bcede505] 2024-09-21T07:50:42.786 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xc96) [0x5589bcb76dd4] 2024-09-21T07:50:42.786 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5589bcb7a523] 2024-09-21T07:50:42.786 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5589bcb7ae4a] 2024-09-21T07:50:42.786 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x5589bcb5dfa4] 2024-09-21T07:50:42.786 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7fcd6b1a57df] 2024-09-21T07:50:42.786 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: (DispatchQueue::entry()+0x60f) [0x7fcd6b1a60d1] 2024-09-21T07:50:42.786 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7fcd6b293f5b] 2024-09-21T07:50:42.786 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (Thread::entry_wrapper()+0x43) [0x7fcd6b02fd57] 2024-09-21T07:50:42.786 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: (Thread::_entry_func(void*)+0xd) [0x7fcd6b02fd73] 2024-09-21T07:50:42.786 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: /lib64/libc.so.6(+0x89c52) [0x7fcd6a689c52] 2024-09-21T07:50:42.786 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: /lib64/libc.so.6(+0x10ec80) [0x7fcd6a70ec80] 2024-09-21T07:50:42.786 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:50:42.786 INFO:tasks.ceph.mds.d.smithi148.stderr: 0> 2024-09-21T07:50:42.765+0000 7fcd64000640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:50:42.786 INFO:tasks.ceph.mds.d.smithi148.stderr: in thread 7fcd64000640 thread_name:ms_dispatch 2024-09-21T07:50:42.787 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:50:42.787 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:50:42.787 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fcd6a63e6f0] 2024-09-21T07:50:42.787 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7fcd6a68b99c] 2024-09-21T07:50:42.787 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: raise() 2024-09-21T07:50:42.787 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: abort() 2024-09-21T07:50:42.787 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7fcd6b056686] 2024-09-21T07:50:42.787 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7fcd6b056758] 2024-09-21T07:50:42.787 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x301) [0x5589bcede505] 2024-09-21T07:50:42.787 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xc96) [0x5589bcb76dd4] 2024-09-21T07:50:42.787 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x5589bcb7a523] 2024-09-21T07:50:42.787 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x5589bcb7ae4a] 2024-09-21T07:50:42.787 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x5589bcb5dfa4] 2024-09-21T07:50:42.787 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: /usr/lib64/ceph/libceph-common.so.2(+0x3a57df) [0x7fcd6b1a57df] 2024-09-21T07:50:42.787 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: (DispatchQueue::entry()+0x60f) [0x7fcd6b1a60d1] 2024-09-21T07:50:42.788 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: /usr/lib64/ceph/libceph-common.so.2(+0x493f5b) [0x7fcd6b293f5b] 2024-09-21T07:50:42.788 INFO:tasks.ceph.mds.d.smithi148.stderr: 15: (Thread::entry_wrapper()+0x43) [0x7fcd6b02fd57] 2024-09-21T07:50:42.788 INFO:tasks.ceph.mds.d.smithi148.stderr: 16: (Thread::_entry_func(void*)+0xd) [0x7fcd6b02fd73] 2024-09-21T07:50:42.788 INFO:tasks.ceph.mds.d.smithi148.stderr: 17: /lib64/libc.so.6(+0x89c52) [0x7fcd6a689c52] 2024-09-21T07:50:42.788 INFO:tasks.ceph.mds.d.smithi148.stderr: 18: /lib64/libc.so.6(+0x10ec80) [0x7fcd6a70ec80] 2024-09-21T07:50:42.788 INFO:tasks.ceph.mds.d.smithi148.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:50:42.788 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:50:43.003 INFO:teuthology.orchestra.run.smithi039.stdout:{"mds_beacon_grace":"15.000000"} 2024-09-21T07:50:43.003 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-09-21T07:50:43.003 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:43.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.228+0000 7f5083200640 1 -- 172.21.15.39:0/3407502416 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50840d0390 msgr2=0x7f50840d07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:43.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.228+0000 7f5083200640 1 --2- 172.21.15.39:0/3407502416 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50840d0390 0x7f50840d07a0 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7f507c005fd0 tx=0x7f507c02aa90 comp rx=0 tx=0).stop 2024-09-21T07:50:43.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.228+0000 7f5083200640 1 -- 172.21.15.39:0/3407502416 shutdown_connections 2024-09-21T07:50:43.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.228+0000 7f5083200640 1 --2- 172.21.15.39:0/3407502416 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50840d0390 0x7f50840d07a0 unknown :-1 s=CLOSED pgs=2394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:43.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.228+0000 7f5083200640 1 --2- 172.21.15.39:0/3407502416 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5084160250 0x7f508415d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:43.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.228+0000 7f5083200640 1 -- 172.21.15.39:0/3407502416 >> 172.21.15.39:0/3407502416 conn(0x7f508406cfc0 msgr2=0x7f508406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:43.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.228+0000 7f5083200640 1 -- 172.21.15.39:0/3407502416 shutdown_connections 2024-09-21T07:50:43.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.228+0000 7f5083200640 1 -- 172.21.15.39:0/3407502416 wait complete. 2024-09-21T07:50:43.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.230+0000 7f5083200640 1 Processor -- start 2024-09-21T07:50:43.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.230+0000 7f5083200640 1 -- start start 2024-09-21T07:50:43.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.232+0000 7f5083200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50840d0390 0x7f50840b4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:43.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.232+0000 7f5083200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5084160250 0x7f508414ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:43.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.232+0000 7f5083200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5084160660 0x7f508415b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:43.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.232+0000 7f5083200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f50840ccdc0 con 0x7f50840d0390 2024-09-21T07:50:43.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.232+0000 7f5083200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f50840ccc40 con 0x7f5084160250 2024-09-21T07:50:43.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.232+0000 7f5083200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f50840ccf40 con 0x7f5084160660 2024-09-21T07:50:43.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.232+0000 7f5081e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50840d0390 0x7f50840b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:43.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.232+0000 7f5081e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50840d0390 0x7f50840b4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51056/0 (socket says 172.21.15.39:51056) 2024-09-21T07:50:43.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.232+0000 7f5081e00640 1 -- 172.21.15.39:0/4031238205 learned_addr learned my addr 172.21.15.39:0/4031238205 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:43.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.232+0000 7f5082800640 1 --2- 172.21.15.39:0/4031238205 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5084160660 0x7f508415b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:43.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.232+0000 7f5081400640 1 --2- 172.21.15.39:0/4031238205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5084160250 0x7f508414ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:43.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.233+0000 7f5081e00640 1 -- 172.21.15.39:0/4031238205 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5084160660 msgr2=0x7f508415b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:43.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.233+0000 7f5081e00640 1 --2- 172.21.15.39:0/4031238205 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5084160660 0x7f508415b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:43.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.233+0000 7f5081e00640 1 -- 172.21.15.39:0/4031238205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5084160250 msgr2=0x7f508414ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:43.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.233+0000 7f5081e00640 1 --2- 172.21.15.39:0/4031238205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5084160250 0x7f508414ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:43.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.233+0000 7f5081e00640 1 -- 172.21.15.39:0/4031238205 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50840b5000 con 0x7f50840d0390 2024-09-21T07:50:43.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.233+0000 7f5081e00640 1 --2- 172.21.15.39:0/4031238205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50840d0390 0x7f50840b4540 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7f507400d750 tx=0x7f507400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:43.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.233+0000 7f507a000640 1 -- 172.21.15.39:0/4031238205 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5074017070 con 0x7f50840d0390 2024-09-21T07:50:43.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.233+0000 7f507a000640 1 -- 172.21.15.39:0/4031238205 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5074004310 con 0x7f50840d0390 2024-09-21T07:50:43.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.233+0000 7f507a000640 1 -- 172.21.15.39:0/4031238205 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507400de60 con 0x7f50840d0390 2024-09-21T07:50:43.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.233+0000 7f5083200640 1 -- 172.21.15.39:0/4031238205 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50841f5cf0 con 0x7f50840d0390 2024-09-21T07:50:43.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.233+0000 7f5083200640 1 -- 172.21.15.39:0/4031238205 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50841f59f0 con 0x7f50840d0390 2024-09-21T07:50:43.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.234+0000 7f5083200640 1 -- 172.21.15.39:0/4031238205 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50840ce970 con 0x7f50840d0390 2024-09-21T07:50:43.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.236+0000 7f507a000640 1 -- 172.21.15.39:0/4031238205 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5074009c30 con 0x7f50840d0390 2024-09-21T07:50:43.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.237+0000 7f507a000640 1 --2- 172.21.15.39:0/4031238205 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5064078bc0 0x7f506407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:43.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.237+0000 7f507a000640 1 -- 172.21.15.39:0/4031238205 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(240..240 src has 1..240) ==== 7390+0+0 (secure 0 0 0) 0x7f507409b760 con 0x7f50840d0390 2024-09-21T07:50:43.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.238+0000 7f5081400640 1 --2- 172.21.15.39:0/4031238205 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5064078bc0 0x7f506407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:43.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.238+0000 7f5081400640 1 --2- 172.21.15.39:0/4031238205 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5064078bc0 0x7f506407b0d0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f50840aa770 tx=0x7f5070009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:43.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.242+0000 7f507a000640 1 -- 172.21.15.39:0/4031238205 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f50840ce970 con 0x7f50840d0390 2024-09-21T07:50:43.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.476+0000 7f5083200640 1 -- 172.21.15.39:0/4031238205 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50840cd3f0 con 0x7f50840d0390 2024-09-21T07:50:43.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.476+0000 7f507a000640 1 -- 172.21.15.39:0/4031238205 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 266 v266) ==== 95+0+5279 (secure 0 0 0) 0x7f5074020020 con 0x7f50840d0390 2024-09-21T07:50:43.479 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:50:43.479 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":266,"btime":"2024-09-21T07:50:42:436514+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11067,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/109229385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":109229385},{"type":"v1","addr":"172.21.15.39:6834","nonce":109229385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":262}],"filesystems":[{"mdsmap":{"epoch":266,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:50:41.844213+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":240,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":8531},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8531":{"gid":8531,"name":"d","rank":2,"incarnation":253,"state":"up:active","state_seq":18,"addr":"172.21.15.148:6837/1438851059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1438851059},{"type":"v1","addr":"172.21.15.148:6837","nonce":1438851059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[11052,9967,8531]},"id":13}]} 2024-09-21T07:50:43.479 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 266 2024-09-21T07:50:43.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.479+0000 7f5083200640 1 -- 172.21.15.39:0/4031238205 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5064078bc0 msgr2=0x7f506407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:43.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.479+0000 7f5083200640 1 --2- 172.21.15.39:0/4031238205 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5064078bc0 0x7f506407b0d0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f50840aa770 tx=0x7f5070009040 comp rx=0 tx=0).stop 2024-09-21T07:50:43.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.479+0000 7f5083200640 1 -- 172.21.15.39:0/4031238205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50840d0390 msgr2=0x7f50840b4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:43.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.479+0000 7f5083200640 1 --2- 172.21.15.39:0/4031238205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50840d0390 0x7f50840b4540 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7f507400d750 tx=0x7f507400dc10 comp rx=0 tx=0).stop 2024-09-21T07:50:43.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.479+0000 7f5083200640 1 -- 172.21.15.39:0/4031238205 shutdown_connections 2024-09-21T07:50:43.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.479+0000 7f5083200640 1 --2- 172.21.15.39:0/4031238205 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5064078bc0 0x7f506407b0d0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:43.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.479+0000 7f5083200640 1 --2- 172.21.15.39:0/4031238205 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5084160660 0x7f508415b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:43.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.479+0000 7f5083200640 1 --2- 172.21.15.39:0/4031238205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5084160250 0x7f508414ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:43.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.479+0000 7f5083200640 1 --2- 172.21.15.39:0/4031238205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50840d0390 0x7f50840b4540 unknown :-1 s=CLOSED pgs=2395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:43.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.479+0000 7f5083200640 1 -- 172.21.15.39:0/4031238205 >> 172.21.15.39:0/4031238205 conn(0x7f508406cfc0 msgr2=0x7f5084073f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:43.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.479+0000 7f5083200640 1 -- 172.21.15.39:0/4031238205 shutdown_connections 2024-09-21T07:50:43.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:43.479+0000 7f5083200640 1 -- 172.21.15.39:0/4031238205 wait complete. 2024-09-21T07:50:43.503 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:50:43.513 INFO:tasks.ceph.mds.d.smithi148.stderr:daemon-helper: command crashed with signal 6 2024-09-21T07:50:45.077 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-09-21T07:50:48.505 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:48.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.740+0000 7fd0cee00640 1 -- 172.21.15.39:0/167327676 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0ac003a50 msgr2=0x7fd0ac003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:48.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.740+0000 7fd0cee00640 1 --2- 172.21.15.39:0/167327676 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0ac003a50 0x7fd0ac003ec0 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7fd0b800c4a0 tx=0x7fd0ac005830 comp rx=0 tx=0).stop 2024-09-21T07:50:48.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.740+0000 7fd0cce00640 1 -- 172.21.15.39:0/167327676 reap_dead start 2024-09-21T07:50:48.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.740+0000 7fd0cee00640 1 -- 172.21.15.39:0/167327676 shutdown_connections 2024-09-21T07:50:48.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.740+0000 7fd0cee00640 1 -- 172.21.15.39:0/167327676 >> 172.21.15.39:0/167327676 conn(0x7fd0c80b2460 msgr2=0x7fd0c806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:48.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.740+0000 7fd0cee00640 1 -- 172.21.15.39:0/167327676 shutdown_connections 2024-09-21T07:50:48.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.740+0000 7fd0cee00640 1 -- 172.21.15.39:0/167327676 wait complete. 2024-09-21T07:50:48.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.743+0000 7fd0cee00640 1 Processor -- start 2024-09-21T07:50:48.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.743+0000 7fd0cee00640 1 -- start start 2024-09-21T07:50:48.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.744+0000 7fd0cee00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0ac003a50 0x7fd0c81f3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:48.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.744+0000 7fd0cee00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0c80c6b40 0x7fd0c814ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:48.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.744+0000 7fd0cee00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd0c81504b0 0x7fd0c80c3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:48.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.744+0000 7fd0cee00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c80cd400 con 0x7fd0ac003a50 2024-09-21T07:50:48.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.744+0000 7fd0cee00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c80cd280 con 0x7fd0c80c6b40 2024-09-21T07:50:48.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.744+0000 7fd0cee00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd0c80af080 con 0x7fd0c81504b0 2024-09-21T07:50:48.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.744+0000 7fd0c7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0ac003a50 0x7fd0c81f3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:48.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.745+0000 7fd0c7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0ac003a50 0x7fd0c81f3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51082/0 (socket says 172.21.15.39:51082) 2024-09-21T07:50:48.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.745+0000 7fd0c7e00640 1 -- 172.21.15.39:0/1628006328 learned_addr learned my addr 172.21.15.39:0/1628006328 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:48.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.745+0000 7fd0cce00640 1 --2- 172.21.15.39:0/1628006328 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd0c81504b0 0x7fd0c80c3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:48.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.745+0000 7fd0bfe00640 1 --2- 172.21.15.39:0/1628006328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0c80c6b40 0x7fd0c814ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:48.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.745+0000 7fd0c7e00640 1 -- 172.21.15.39:0/1628006328 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd0c81504b0 msgr2=0x7fd0c80c3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:48.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.745+0000 7fd0c7e00640 1 --2- 172.21.15.39:0/1628006328 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd0c81504b0 0x7fd0c80c3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:48.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.745+0000 7fd0c7e00640 1 -- 172.21.15.39:0/1628006328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0c80c6b40 msgr2=0x7fd0c814ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:48.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.745+0000 7fd0c7e00640 1 --2- 172.21.15.39:0/1628006328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0c80c6b40 0x7fd0c814ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:48.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.745+0000 7fd0c7e00640 1 -- 172.21.15.39:0/1628006328 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0c80c49b0 con 0x7fd0ac003a50 2024-09-21T07:50:48.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.745+0000 7fd0bfe00640 1 --2- 172.21.15.39:0/1628006328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0c80c6b40 0x7fd0c814ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:50:48.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.745+0000 7fd0c7e00640 1 --2- 172.21.15.39:0/1628006328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0ac003a50 0x7fd0c81f3070 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7fd0b800c5c0 tx=0x7fd0b8006310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:48.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.746+0000 7fd0c4c00640 1 -- 172.21.15.39:0/1628006328 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b8042070 con 0x7fd0ac003a50 2024-09-21T07:50:48.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.746+0000 7fd0c4c00640 1 -- 172.21.15.39:0/1628006328 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd0b80038e0 con 0x7fd0ac003a50 2024-09-21T07:50:48.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.746+0000 7fd0cee00640 1 -- 172.21.15.39:0/1628006328 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0c81508c0 con 0x7fd0ac003a50 2024-09-21T07:50:48.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.746+0000 7fd0c4c00640 1 -- 172.21.15.39:0/1628006328 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b8039070 con 0x7fd0ac003a50 2024-09-21T07:50:48.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.746+0000 7fd0cee00640 1 -- 172.21.15.39:0/1628006328 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0c814e5c0 con 0x7fd0ac003a50 2024-09-21T07:50:48.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.747+0000 7fd0cee00640 1 -- 172.21.15.39:0/1628006328 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0c8156bb0 con 0x7fd0ac003a50 2024-09-21T07:50:48.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.748+0000 7fd0c4c00640 1 -- 172.21.15.39:0/1628006328 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd0b8045070 con 0x7fd0ac003a50 2024-09-21T07:50:48.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.750+0000 7fd0c4c00640 1 --2- 172.21.15.39:0/1628006328 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd0a4078bc0 0x7fd0a407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:48.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.750+0000 7fd0c4c00640 1 -- 172.21.15.39:0/1628006328 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(240..240 src has 1..240) ==== 7390+0+0 (secure 0 0 0) 0x7fd0b80baec0 con 0x7fd0ac003a50 2024-09-21T07:50:48.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.750+0000 7fd0bfe00640 1 --2- 172.21.15.39:0/1628006328 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd0a4078bc0 0x7fd0a407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:48.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.751+0000 7fd0bfe00640 1 --2- 172.21.15.39:0/1628006328 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd0a4078bc0 0x7fd0a407b0d0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0009460 tx=0x7fd0b0005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:48.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.754+0000 7fd0c4c00640 1 -- 172.21.15.39:0/1628006328 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd0c81508c0 con 0x7fd0ac003a50 2024-09-21T07:50:48.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.992+0000 7fd0cee00640 1 -- 172.21.15.39:0/1628006328 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0c8074d40 con 0x7fd0ac003a50 2024-09-21T07:50:48.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.992+0000 7fd0c4c00640 1 -- 172.21.15.39:0/1628006328 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 266 v266) ==== 95+0+5279 (secure 0 0 0) 0x7fd0c80cd5b0 con 0x7fd0ac003a50 2024-09-21T07:50:48.994 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:50:48.994 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":266,"btime":"2024-09-21T07:50:42:436514+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11067,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/109229385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":109229385},{"type":"v1","addr":"172.21.15.39:6834","nonce":109229385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":262}],"filesystems":[{"mdsmap":{"epoch":266,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:50:41.844213+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":240,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":8531},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8531":{"gid":8531,"name":"d","rank":2,"incarnation":253,"state":"up:active","state_seq":18,"addr":"172.21.15.148:6837/1438851059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1438851059},{"type":"v1","addr":"172.21.15.148:6837","nonce":1438851059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[11052,9967,8531]},"id":13}]} 2024-09-21T07:50:48.995 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 266 2024-09-21T07:50:48.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.995+0000 7fd0cee00640 1 -- 172.21.15.39:0/1628006328 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd0a4078bc0 msgr2=0x7fd0a407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:48.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.995+0000 7fd0cee00640 1 --2- 172.21.15.39:0/1628006328 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd0a4078bc0 0x7fd0a407b0d0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0009460 tx=0x7fd0b0005f90 comp rx=0 tx=0).stop 2024-09-21T07:50:48.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.995+0000 7fd0cee00640 1 -- 172.21.15.39:0/1628006328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0ac003a50 msgr2=0x7fd0c81f3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:48.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.995+0000 7fd0cee00640 1 --2- 172.21.15.39:0/1628006328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0ac003a50 0x7fd0c81f3070 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7fd0b800c5c0 tx=0x7fd0b8006310 comp rx=0 tx=0).stop 2024-09-21T07:50:48.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.995+0000 7fd0cee00640 1 -- 172.21.15.39:0/1628006328 shutdown_connections 2024-09-21T07:50:48.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.995+0000 7fd0cee00640 1 --2- 172.21.15.39:0/1628006328 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd0a4078bc0 0x7fd0a407b0d0 secure :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0009460 tx=0x7fd0b0005f90 comp rx=0 tx=0).stop 2024-09-21T07:50:48.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.995+0000 7fd0cee00640 1 --2- 172.21.15.39:0/1628006328 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd0c81504b0 0x7fd0c80c3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:48.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.995+0000 7fd0cee00640 1 --2- 172.21.15.39:0/1628006328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd0c80c6b40 0x7fd0c814ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:48.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.995+0000 7fd0cee00640 1 --2- 172.21.15.39:0/1628006328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0ac003a50 0x7fd0c81f3070 unknown :-1 s=CLOSED pgs=2398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:48.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.995+0000 7fd0cee00640 1 -- 172.21.15.39:0/1628006328 >> 172.21.15.39:0/1628006328 conn(0x7fd0c80b2460 msgr2=0x7fd0c80ad060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:48.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.995+0000 7fd0cee00640 1 -- 172.21.15.39:0/1628006328 shutdown_connections 2024-09-21T07:50:48.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:48.995+0000 7fd0cee00640 1 -- 172.21.15.39:0/1628006328 wait complete. 2024-09-21T07:50:49.020 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:50:51.688 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~7s 2024-09-21T07:50:54.021 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:54.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.258+0000 7f33d6200640 1 -- 172.21.15.39:0/2971052971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33d00cf990 msgr2=0x7f33d00b40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:54.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.258+0000 7f33d6200640 1 --2- 172.21.15.39:0/2971052971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33d00cf990 0x7f33d00b40b0 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7f33d00cdd60 tx=0x7f33c802b530 comp rx=0 tx=0).stop 2024-09-21T07:50:54.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.258+0000 7f33d6200640 1 -- 172.21.15.39:0/2971052971 shutdown_connections 2024-09-21T07:50:54.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.258+0000 7f33d6200640 1 --2- 172.21.15.39:0/2971052971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33d00cf990 0x7f33d00b40b0 unknown :-1 s=CLOSED pgs=2400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:54.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.258+0000 7f33d6200640 1 --2- 172.21.15.39:0/2971052971 >> v2:172.21.15.148:3300/0 conn(0x7f33d00cd260 0x7f33d00cf450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:54.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.258+0000 7f33d6200640 1 -- 172.21.15.39:0/2971052971 >> 172.21.15.39:0/2971052971 conn(0x7f33d00b2460 msgr2=0x7f33d006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:54.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.259+0000 7f33d6200640 1 -- 172.21.15.39:0/2971052971 shutdown_connections 2024-09-21T07:50:54.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.259+0000 7f33d6200640 1 -- 172.21.15.39:0/2971052971 wait complete. 2024-09-21T07:50:54.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.261+0000 7f33d6200640 1 Processor -- start 2024-09-21T07:50:54.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.261+0000 7f33d6200640 1 -- start start 2024-09-21T07:50:54.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.262+0000 7f33d6200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33d00af430 0x7f33d009d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:54.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.262+0000 7f33d6200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33d00cd260 0x7f33d01f26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:54.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.263+0000 7f33d6200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33d00cf990 0x7f33d006ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:54.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.263+0000 7f33d6200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f33d00b45f0 con 0x7f33d00cd260 2024-09-21T07:50:54.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.263+0000 7f33d6200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f33d0071270 con 0x7f33d00cf990 2024-09-21T07:50:54.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.263+0000 7f33d6200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f33d00b4770 con 0x7f33d00af430 2024-09-21T07:50:54.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.263+0000 7f33cea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33d00cd260 0x7f33d01f26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:54.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.263+0000 7f33cea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33d00cd260 0x7f33d01f26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53768/0 (socket says 172.21.15.39:53768) 2024-09-21T07:50:54.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.263+0000 7f33cea00640 1 -- 172.21.15.39:0/261409882 learned_addr learned my addr 172.21.15.39:0/261409882 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:54.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.263+0000 7f33cf400640 1 --2- 172.21.15.39:0/261409882 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33d00af430 0x7f33d009d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:54.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.263+0000 7f33cfe00640 1 --2- 172.21.15.39:0/261409882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33d00cf990 0x7f33d006ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:54.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.263+0000 7f33cea00640 1 -- 172.21.15.39:0/261409882 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33d00af430 msgr2=0x7f33d009d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:54.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.263+0000 7f33cea00640 1 --2- 172.21.15.39:0/261409882 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33d00af430 0x7f33d009d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:54.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.263+0000 7f33cea00640 1 -- 172.21.15.39:0/261409882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33d00cf990 msgr2=0x7f33d006ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:54.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.263+0000 7f33cea00640 1 --2- 172.21.15.39:0/261409882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33d00cf990 0x7f33d006ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:54.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.263+0000 7f33cea00640 1 -- 172.21.15.39:0/261409882 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33d01f41b0 con 0x7f33d00cd260 2024-09-21T07:50:54.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.263+0000 7f33cfe00640 1 --2- 172.21.15.39:0/261409882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33d00cf990 0x7f33d006ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:50:54.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.263+0000 7f33cf400640 1 --2- 172.21.15.39:0/261409882 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33d00af430 0x7f33d009d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:50:54.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.264+0000 7f33cea00640 1 --2- 172.21.15.39:0/261409882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33d00cd260 0x7f33d01f26f0 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7f33bc00c9a0 tx=0x7f33bc00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:54.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.264+0000 7f33c7400640 1 -- 172.21.15.39:0/261409882 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc007bb0 con 0x7f33d00cd260 2024-09-21T07:50:54.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.264+0000 7f33c7400640 1 -- 172.21.15.39:0/261409882 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33bc007e40 con 0x7f33d00cd260 2024-09-21T07:50:54.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.264+0000 7f33d6200640 1 -- 172.21.15.39:0/261409882 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33d01f4490 con 0x7f33d00cd260 2024-09-21T07:50:54.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.264+0000 7f33c7400640 1 -- 172.21.15.39:0/261409882 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc00f690 con 0x7f33d00cd260 2024-09-21T07:50:54.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.264+0000 7f33d6200640 1 -- 172.21.15.39:0/261409882 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33d01f4a50 con 0x7f33d00cd260 2024-09-21T07:50:54.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.265+0000 7f33d6200640 1 -- 172.21.15.39:0/261409882 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33d00cf450 con 0x7f33d00cd260 2024-09-21T07:50:54.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.266+0000 7f33c7400640 1 -- 172.21.15.39:0/261409882 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f33bc00f830 con 0x7f33d00cd260 2024-09-21T07:50:54.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.267+0000 7f33c7400640 1 --2- 172.21.15.39:0/261409882 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f33b0078bc0 0x7f33b007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:54.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.268+0000 7f33c7400640 1 -- 172.21.15.39:0/261409882 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(240..240 src has 1..240) ==== 7390+0+0 (secure 0 0 0) 0x7f33bc09bc50 con 0x7f33d00cd260 2024-09-21T07:50:54.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.268+0000 7f33cf400640 1 --2- 172.21.15.39:0/261409882 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f33b0078bc0 0x7f33b007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:54.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.269+0000 7f33cf400640 1 --2- 172.21.15.39:0/261409882 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f33b0078bc0 0x7f33b007b0d0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f33c0000f60 tx=0x7f33c0009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:54.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.272+0000 7f33c7400640 1 -- 172.21.15.39:0/261409882 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f33bc0a1020 con 0x7f33d00cd260 2024-09-21T07:50:54.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.509+0000 7f33d6200640 1 -- 172.21.15.39:0/261409882 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33d00709f0 con 0x7f33d00cd260 2024-09-21T07:50:54.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.509+0000 7f33c7400640 1 -- 172.21.15.39:0/261409882 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 266 v266) ==== 95+0+5279 (secure 0 0 0) 0x7f33bc014030 con 0x7f33d00cd260 2024-09-21T07:50:54.512 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:50:54.513 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":266,"btime":"2024-09-21T07:50:42:436514+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11067,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/109229385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":109229385},{"type":"v1","addr":"172.21.15.39:6834","nonce":109229385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":262}],"filesystems":[{"mdsmap":{"epoch":266,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:50:41.844213+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":240,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":8531},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8531":{"gid":8531,"name":"d","rank":2,"incarnation":253,"state":"up:active","state_seq":18,"addr":"172.21.15.148:6837/1438851059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1438851059},{"type":"v1","addr":"172.21.15.148:6837","nonce":1438851059}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[11052,9967,8531]},"id":13}]} 2024-09-21T07:50:54.513 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 266 2024-09-21T07:50:54.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.512+0000 7f33d6200640 1 -- 172.21.15.39:0/261409882 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f33b0078bc0 msgr2=0x7f33b007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:54.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.512+0000 7f33d6200640 1 --2- 172.21.15.39:0/261409882 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f33b0078bc0 0x7f33b007b0d0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f33c0000f60 tx=0x7f33c0009040 comp rx=0 tx=0).stop 2024-09-21T07:50:54.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.512+0000 7f33d6200640 1 -- 172.21.15.39:0/261409882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33d00cd260 msgr2=0x7f33d01f26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:54.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.512+0000 7f33d6200640 1 --2- 172.21.15.39:0/261409882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33d00cd260 0x7f33d01f26f0 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7f33bc00c9a0 tx=0x7f33bc00ce60 comp rx=0 tx=0).stop 2024-09-21T07:50:54.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.512+0000 7f33d6200640 1 -- 172.21.15.39:0/261409882 shutdown_connections 2024-09-21T07:50:54.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.512+0000 7f33d6200640 1 --2- 172.21.15.39:0/261409882 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f33b0078bc0 0x7f33b007b0d0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:54.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.512+0000 7f33d6200640 1 --2- 172.21.15.39:0/261409882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33d00cf990 0x7f33d006ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:54.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.512+0000 7f33d6200640 1 --2- 172.21.15.39:0/261409882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33d00cd260 0x7f33d01f26f0 unknown :-1 s=CLOSED pgs=2401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:54.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.512+0000 7f33d6200640 1 --2- 172.21.15.39:0/261409882 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f33d00af430 0x7f33d009d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:54.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.512+0000 7f33d6200640 1 -- 172.21.15.39:0/261409882 >> 172.21.15.39:0/261409882 conn(0x7f33d00b2460 msgr2=0x7f33d006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:54.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.512+0000 7f33d6200640 1 -- 172.21.15.39:0/261409882 shutdown_connections 2024-09-21T07:50:54.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:54.512+0000 7f33d6200640 1 -- 172.21.15.39:0/261409882 wait complete. 2024-09-21T07:50:54.539 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:50:58.300 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~13s 2024-09-21T07:50:59.540 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:59.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.776+0000 7f3c6be00640 1 -- 172.21.15.39:0/3588063030 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3c6c145d00 msgr2=0x7f3c6c073750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:59.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.776+0000 7f3c6be00640 1 --2- 172.21.15.39:0/3588063030 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3c6c145d00 0x7f3c6c073750 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3c58009830 tx=0x7f3c5802a4b0 comp rx=0 tx=0).stop 2024-09-21T07:50:59.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.776+0000 7f3c6be00640 1 -- 172.21.15.39:0/3588063030 shutdown_connections 2024-09-21T07:50:59.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.776+0000 7f3c6be00640 1 --2- 172.21.15.39:0/3588063030 >> v2:172.21.15.148:3300/0 conn(0x7f3c6c073c90 0x7f3c6c0cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:59.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.776+0000 7f3c6be00640 1 --2- 172.21.15.39:0/3588063030 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3c6c145d00 0x7f3c6c073750 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:59.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.776+0000 7f3c6be00640 1 -- 172.21.15.39:0/3588063030 >> 172.21.15.39:0/3588063030 conn(0x7f3c6c06cfc0 msgr2=0x7f3c6c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:59.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.776+0000 7f3c6be00640 1 -- 172.21.15.39:0/3588063030 shutdown_connections 2024-09-21T07:50:59.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.776+0000 7f3c6be00640 1 -- 172.21.15.39:0/3588063030 wait complete. 2024-09-21T07:50:59.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.778+0000 7f3c6be00640 1 Processor -- start 2024-09-21T07:50:59.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.779+0000 7f3c6be00640 1 -- start start 2024-09-21T07:50:59.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.780+0000 7f3c6be00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3c6c073c90 0x7f3c6c0b2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:59.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.780+0000 7f3c6be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c6c074c00 0x7f3c6c1e7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:59.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.780+0000 7f3c6be00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c6c145d00 0x7f3c6c0ca640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:59.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.780+0000 7f3c6be00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3c6c0d0590 con 0x7f3c6c074c00 2024-09-21T07:50:59.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.780+0000 7f3c6be00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3c6c0cdc70 con 0x7f3c6c145d00 2024-09-21T07:50:59.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.780+0000 7f3c6be00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f3c6c0d0710 con 0x7f3c6c073c90 2024-09-21T07:50:59.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.780+0000 7f3c6a000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c6c074c00 0x7f3c6c1e7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:59.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.780+0000 7f3c6aa00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3c6c073c90 0x7f3c6c0b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:59.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.780+0000 7f3c6a000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c6c074c00 0x7f3c6c1e7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53780/0 (socket says 172.21.15.39:53780) 2024-09-21T07:50:59.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.780+0000 7f3c6aa00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3c6c073c90 0x7f3c6c0b2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.39:44720/0 (socket says 172.21.15.39:44720) 2024-09-21T07:50:59.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.780+0000 7f3c6a000640 1 -- 172.21.15.39:0/1705949404 learned_addr learned my addr 172.21.15.39:0/1705949404 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:50:59.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.781+0000 7f3c6b400640 1 --2- 172.21.15.39:0/1705949404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c6c145d00 0x7f3c6c0ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:59.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.781+0000 7f3c6aa00640 1 -- 172.21.15.39:0/1705949404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c6c145d00 msgr2=0x7f3c6c0ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:59.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.781+0000 7f3c6aa00640 1 --2- 172.21.15.39:0/1705949404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c6c145d00 0x7f3c6c0ca640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:59.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.781+0000 7f3c6aa00640 1 -- 172.21.15.39:0/1705949404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c6c074c00 msgr2=0x7f3c6c1e7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:59.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.781+0000 7f3c6aa00640 1 --2- 172.21.15.39:0/1705949404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c6c074c00 0x7f3c6c1e7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:59.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.781+0000 7f3c6aa00640 1 -- 172.21.15.39:0/1705949404 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c6c1e7fd0 con 0x7f3c6c073c90 2024-09-21T07:50:59.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.781+0000 7f3c6a000640 1 --2- 172.21.15.39:0/1705949404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c6c074c00 0x7f3c6c1e7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:50:59.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.781+0000 7f3c6aa00640 1 --2- 172.21.15.39:0/1705949404 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3c6c073c90 0x7f3c6c0b2720 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c00c530 tx=0x7f3c5c00c930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:59.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.782+0000 7f3c62a00640 1 -- 172.21.15.39:0/1705949404 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c011790 con 0x7f3c6c073c90 2024-09-21T07:50:59.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.782+0000 7f3c62a00640 1 -- 172.21.15.39:0/1705949404 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c5c017070 con 0x7f3c6c073c90 2024-09-21T07:50:59.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.782+0000 7f3c6be00640 1 -- 172.21.15.39:0/1705949404 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c6c131ba0 con 0x7f3c6c073c90 2024-09-21T07:50:59.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.782+0000 7f3c62a00640 1 -- 172.21.15.39:0/1705949404 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c011cc0 con 0x7f3c6c073c90 2024-09-21T07:50:59.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.782+0000 7f3c6be00640 1 -- 172.21.15.39:0/1705949404 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c6c132160 con 0x7f3c6c073c90 2024-09-21T07:50:59.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.783+0000 7f3c6be00640 1 -- 172.21.15.39:0/1705949404 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c6c0739f0 con 0x7f3c6c073c90 2024-09-21T07:50:59.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.784+0000 7f3c62a00640 1 -- 172.21.15.39:0/1705949404 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3c5c01c460 con 0x7f3c6c073c90 2024-09-21T07:50:59.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.785+0000 7f3c62a00640 1 --2- 172.21.15.39:0/1705949404 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3c4c078b90 0x7f3c4c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:59.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.785+0000 7f3c62a00640 1 -- 172.21.15.39:0/1705949404 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(240..240 src has 1..240) ==== 7390+0+0 (secure 0 0 0) 0x7f3c5c09b360 con 0x7f3c6c073c90 2024-09-21T07:50:59.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.786+0000 7f3c6a000640 1 --2- 172.21.15.39:0/1705949404 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3c4c078b90 0x7f3c4c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:59.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.786+0000 7f3c6a000640 1 --2- 172.21.15.39:0/1705949404 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3c4c078b90 0x7f3c4c07b0a0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f3c58000c00 tx=0x7f3c58004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:59.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:50:59.789+0000 7f3c62a00640 1 -- 172.21.15.39:0/1705949404 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3c6c0739f0 con 0x7f3c6c073c90 2024-09-21T07:51:00.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.027+0000 7f3c6be00640 1 -- 172.21.15.39:0/1705949404 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c6c0d0c00 con 0x7f3c6c073c90 2024-09-21T07:51:00.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.028+0000 7f3c62a00640 1 -- 172.21.15.39:0/1705949404 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 267 v267) ==== 95+0+5327 (secure 0 0 0) 0x7f3c5c063240 con 0x7f3c6c073c90 2024-09-21T07:51:00.030 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:00.030 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":267,"btime":"2024-09-21T07:50:56:563330+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11067,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/109229385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":109229385},{"type":"v1","addr":"172.21.15.39:6834","nonce":109229385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":262}],"filesystems":[{"mdsmap":{"epoch":267,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:50:56.563300+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":240,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":8531},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8531":{"gid":8531,"name":"d","rank":2,"incarnation":253,"state":"up:active","state_seq":18,"addr":"172.21.15.148:6837/1438851059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1438851059},{"type":"v1","addr":"172.21.15.148:6837","nonce":1438851059}]},"join_fscid":-1,"laggy_since":"2024-09-21T07:50:56.563300+0000","export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[8531,9967,11052]},"id":13}]} 2024-09-21T07:51:00.030 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 267 2024-09-21T07:51:00.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.030+0000 7f3c6be00640 1 -- 172.21.15.39:0/1705949404 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3c4c078b90 msgr2=0x7f3c4c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:00.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.030+0000 7f3c6be00640 1 --2- 172.21.15.39:0/1705949404 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3c4c078b90 0x7f3c4c07b0a0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f3c58000c00 tx=0x7f3c58004730 comp rx=0 tx=0).stop 2024-09-21T07:51:00.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.030+0000 7f3c6be00640 1 -- 172.21.15.39:0/1705949404 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3c6c073c90 msgr2=0x7f3c6c0b2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:00.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.030+0000 7f3c6be00640 1 --2- 172.21.15.39:0/1705949404 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3c6c073c90 0x7f3c6c0b2720 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c00c530 tx=0x7f3c5c00c930 comp rx=0 tx=0).stop 2024-09-21T07:51:00.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.030+0000 7f3c6be00640 1 -- 172.21.15.39:0/1705949404 shutdown_connections 2024-09-21T07:51:00.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.030+0000 7f3c6be00640 1 --2- 172.21.15.39:0/1705949404 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3c4c078b90 0x7f3c4c07b0a0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:00.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.030+0000 7f3c6be00640 1 --2- 172.21.15.39:0/1705949404 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c6c145d00 0x7f3c6c0ca640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:00.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.030+0000 7f3c6be00640 1 --2- 172.21.15.39:0/1705949404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c6c074c00 0x7f3c6c1e7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:00.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.030+0000 7f3c6be00640 1 --2- 172.21.15.39:0/1705949404 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3c6c073c90 0x7f3c6c0b2720 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:00.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.030+0000 7f3c6be00640 1 -- 172.21.15.39:0/1705949404 >> 172.21.15.39:0/1705949404 conn(0x7f3c6c06cfc0 msgr2=0x7f3c6c0b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:00.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.030+0000 7f3c6be00640 1 -- 172.21.15.39:0/1705949404 shutdown_connections 2024-09-21T07:51:00.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.030+0000 7f3c6be00640 1 -- 172.21.15.39:0/1705949404 wait complete. 2024-09-21T07:51:00.055 DEBUG:tasks.ceph_test_case:wait_until_true: success in 15s and 0 retries 2024-09-21T07:51:00.055 DEBUG:teuthology.orchestra.run.smithi148:> sudo sysctl -n kernel.core_pattern 2024-09-21T07:51:00.114 INFO:teuthology.orchestra.run.smithi148.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T07:51:00.117 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2024-09-21T07:51:00.117 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2024-09-21T07:51:00.183 INFO:teuthology.orchestra.run.smithi148.stdout:1726905042.45746.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i d', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2024-09-21T07:51:00.187 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1726905042.45746.core'] 2024-09-21T07:51:00.187 INFO:tasks.cephfs.cephfs_test_case:Found core file 1726905042.45746.core, deleting it 2024-09-21T07:51:00.187 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1726905042.45746.core 2024-09-21T07:51:00.378 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 13:2 2024-09-21T07:51:00.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.606+0000 7f45ec000640 1 -- 172.21.15.39:0/1809117925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45e415d830 msgr2=0x7f45e4156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:00.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.606+0000 7f45ec000640 1 --2- 172.21.15.39:0/1809117925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45e415d830 0x7f45e4156bb0 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7f45e406d880 tx=0x7f45d002a4d0 comp rx=0 tx=0).stop 2024-09-21T07:51:00.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.607+0000 7f45ec000640 1 -- 172.21.15.39:0/1809117925 shutdown_connections 2024-09-21T07:51:00.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.607+0000 7f45ec000640 1 --2- 172.21.15.39:0/1809117925 >> v2:172.21.15.148:3300/0 conn(0x7f45e40ccc60 0x7f45e40cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:00.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.607+0000 7f45ec000640 1 --2- 172.21.15.39:0/1809117925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45e415d830 0x7f45e4156bb0 unknown :-1 s=CLOSED pgs=2403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:00.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.607+0000 7f45ec000640 1 --2- 172.21.15.39:0/1809117925 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45e40b30a0 0x7f45e40cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:00.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.607+0000 7f45ec000640 1 -- 172.21.15.39:0/1809117925 >> 172.21.15.39:0/1809117925 conn(0x7f45e40b2460 msgr2=0x7f45e406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:00.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.607+0000 7f45ec000640 1 -- 172.21.15.39:0/1809117925 shutdown_connections 2024-09-21T07:51:00.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.607+0000 7f45ec000640 1 -- 172.21.15.39:0/1809117925 wait complete. 2024-09-21T07:51:00.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.609+0000 7f45ec000640 1 Processor -- start 2024-09-21T07:51:00.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.609+0000 7f45ec000640 1 -- start start 2024-09-21T07:51:00.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.611+0000 7f45ec000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45e40b30a0 0x7f45e409d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:00.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.611+0000 7f45ec000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45e40ccc60 0x7f45e40c5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:00.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.611+0000 7f45ec000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45e415d830 0x7f45e40c62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:00.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.611+0000 7f45ec000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f45e40af200 con 0x7f45e415d830 2024-09-21T07:51:00.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.611+0000 7f45ec000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f45e40af080 con 0x7f45e40b30a0 2024-09-21T07:51:00.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.611+0000 7f45ec000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f45e40af380 con 0x7f45e40ccc60 2024-09-21T07:51:00.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.611+0000 7f45ea000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45e415d830 0x7f45e40c62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:00.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.611+0000 7f45ea000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45e415d830 0x7f45e40c62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53804/0 (socket says 172.21.15.39:53804) 2024-09-21T07:51:00.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.611+0000 7f45ea000640 1 -- 172.21.15.39:0/980936694 learned_addr learned my addr 172.21.15.39:0/980936694 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:51:00.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.611+0000 7f45e9600640 1 --2- 172.21.15.39:0/980936694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45e40b30a0 0x7f45e409d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:00.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.611+0000 7f45e8c00640 1 --2- 172.21.15.39:0/980936694 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45e40ccc60 0x7f45e40c5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:00.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.611+0000 7f45ea000640 1 -- 172.21.15.39:0/980936694 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45e40ccc60 msgr2=0x7f45e40c5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:00.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.611+0000 7f45ea000640 1 --2- 172.21.15.39:0/980936694 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45e40ccc60 0x7f45e40c5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:00.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.611+0000 7f45ea000640 1 -- 172.21.15.39:0/980936694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45e40b30a0 msgr2=0x7f45e409d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:00.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.611+0000 7f45ea000640 1 --2- 172.21.15.39:0/980936694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45e40b30a0 0x7f45e409d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:00.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.611+0000 7f45ea000640 1 -- 172.21.15.39:0/980936694 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45e41f3c00 con 0x7f45e415d830 2024-09-21T07:51:00.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.612+0000 7f45ea000640 1 --2- 172.21.15.39:0/980936694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45e415d830 0x7f45e40c62d0 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7f45dc00e990 tx=0x7f45dc00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:00.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.612+0000 7f45e1600640 1 -- 172.21.15.39:0/980936694 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45dc004d20 con 0x7f45e415d830 2024-09-21T07:51:00.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.612+0000 7f45e1600640 1 -- 172.21.15.39:0/980936694 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45dc00caa0 con 0x7f45e415d830 2024-09-21T07:51:00.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.612+0000 7f45e1600640 1 -- 172.21.15.39:0/980936694 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45dc005020 con 0x7f45e415d830 2024-09-21T07:51:00.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.612+0000 7f45ec000640 1 -- 172.21.15.39:0/980936694 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45e41f3ee0 con 0x7f45e415d830 2024-09-21T07:51:00.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.612+0000 7f45ec000640 1 -- 172.21.15.39:0/980936694 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45e41f38f0 con 0x7f45e415d830 2024-09-21T07:51:00.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.613+0000 7f45ec000640 1 -- 172.21.15.39:0/980936694 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45e40af870 con 0x7f45e415d830 2024-09-21T07:51:00.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.614+0000 7f45e1600640 1 -- 172.21.15.39:0/980936694 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f45dc0051c0 con 0x7f45e415d830 2024-09-21T07:51:00.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.615+0000 7f45e1600640 1 --2- 172.21.15.39:0/980936694 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f45c4078b90 0x7f45c407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:00.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.615+0000 7f45e1600640 1 -- 172.21.15.39:0/980936694 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(240..240 src has 1..240) ==== 7390+0+0 (secure 0 0 0) 0x7f45dc09bde0 con 0x7f45e415d830 2024-09-21T07:51:00.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.616+0000 7f45e9600640 1 --2- 172.21.15.39:0/980936694 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f45c4078b90 0x7f45c407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:00.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.617+0000 7f45e9600640 1 --2- 172.21.15.39:0/980936694 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f45c4078b90 0x7f45c407b0a0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f45d800daa0 tx=0x7f45d8005d70 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:00.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.620+0000 7f45e1600640 1 -- 172.21.15.39:0/980936694 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f45dc016030 con 0x7f45e415d830 2024-09-21T07:51:00.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.900+0000 7f45ec000640 1 -- 172.21.15.39:0/980936694 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "13:2"} v 0) -- 0x7f45e4156ed0 con 0x7f45e415d830 2024-09-21T07:51:00.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.903+0000 7f45e1600640 1 -- 172.21.15.39:0/980936694 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "13:2"}]=0 failed mds gid 8531 v268) ==== 98+0+0 (secure 0 0 0) 0x7f45dc063af0 con 0x7f45e415d830 2024-09-21T07:51:00.905 INFO:teuthology.orchestra.run.smithi039.stderr:failed mds gid 8531 2024-09-21T07:51:00.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.907+0000 7f45ec000640 1 -- 172.21.15.39:0/980936694 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f45c4078b90 msgr2=0x7f45c407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:00.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.907+0000 7f45ec000640 1 --2- 172.21.15.39:0/980936694 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f45c4078b90 0x7f45c407b0a0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f45d800daa0 tx=0x7f45d8005d70 comp rx=0 tx=0).stop 2024-09-21T07:51:00.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.907+0000 7f45ec000640 1 -- 172.21.15.39:0/980936694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45e415d830 msgr2=0x7f45e40c62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:00.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.907+0000 7f45ec000640 1 --2- 172.21.15.39:0/980936694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45e415d830 0x7f45e40c62d0 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7f45dc00e990 tx=0x7f45dc00ee50 comp rx=0 tx=0).stop 2024-09-21T07:51:00.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.908+0000 7f45ec000640 1 -- 172.21.15.39:0/980936694 shutdown_connections 2024-09-21T07:51:00.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.908+0000 7f45ec000640 1 --2- 172.21.15.39:0/980936694 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f45c4078b90 0x7f45c407b0a0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:00.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.908+0000 7f45ec000640 1 --2- 172.21.15.39:0/980936694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45e415d830 0x7f45e40c62d0 unknown :-1 s=CLOSED pgs=2404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:00.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.908+0000 7f45ec000640 1 --2- 172.21.15.39:0/980936694 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45e40ccc60 0x7f45e40c5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:00.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.908+0000 7f45ec000640 1 --2- 172.21.15.39:0/980936694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45e40b30a0 0x7f45e409d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:00.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.908+0000 7f45ec000640 1 -- 172.21.15.39:0/980936694 >> 172.21.15.39:0/980936694 conn(0x7f45e40b2460 msgr2=0x7f45e4154cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:00.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.908+0000 7f45ec000640 1 -- 172.21.15.39:0/980936694 shutdown_connections 2024-09-21T07:51:00.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:00.908+0000 7f45ec000640 1 -- 172.21.15.39:0/980936694 wait complete. 2024-09-21T07:51:00.945 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T07:51:00.945 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T07:51:00.945 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T07:51:00.945 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T07:51:00.945 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:51:00.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:51:00.946 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/daemon/state.py", line 146, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi148 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d' 2024-09-21T07:51:00.946 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T07:51:00.946 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T07:51:00.950 INFO:tasks.ceph.mds.d:Started 2024-09-21T07:51:00.950 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:51:01.141 INFO:tasks.ceph.mds.d.smithi148.stdout:starting mds.d at 2024-09-21T07:51:01.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.212+0000 7fb8eee00640 1 -- 172.21.15.39:0/1363857420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8e8160250 msgr2=0x7fb8e815d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:01.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.212+0000 7fb8eee00640 1 --2- 172.21.15.39:0/1363857420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8e8160250 0x7fb8e815d570 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc005fd0 tx=0x7fb8dc02a5d0 comp rx=0 tx=0).stop 2024-09-21T07:51:01.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.212+0000 7fb8eee00640 1 -- 172.21.15.39:0/1363857420 shutdown_connections 2024-09-21T07:51:01.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.212+0000 7fb8eee00640 1 --2- 172.21.15.39:0/1363857420 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8e80d0390 0x7fb8e80d07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:01.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.212+0000 7fb8eee00640 1 --2- 172.21.15.39:0/1363857420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8e8160250 0x7fb8e815d570 unknown :-1 s=CLOSED pgs=2407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:01.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.212+0000 7fb8eee00640 1 -- 172.21.15.39:0/1363857420 >> 172.21.15.39:0/1363857420 conn(0x7fb8e806cfc0 msgr2=0x7fb8e806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:01.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.212+0000 7fb8eee00640 1 -- 172.21.15.39:0/1363857420 shutdown_connections 2024-09-21T07:51:01.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.212+0000 7fb8eee00640 1 -- 172.21.15.39:0/1363857420 wait complete. 2024-09-21T07:51:01.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.214+0000 7fb8eee00640 1 Processor -- start 2024-09-21T07:51:01.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.214+0000 7fb8eee00640 1 -- start start 2024-09-21T07:51:01.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.216+0000 7fb8eee00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8e80d0390 0x7fb8e81eb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:01.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.216+0000 7fb8eee00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8e8160250 0x7fb8e81e6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:01.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.216+0000 7fb8eee00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8e8160660 0x7fb8e81e65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:01.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.216+0000 7fb8eee00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e80ccdc0 con 0x7fb8e8160660 2024-09-21T07:51:01.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.216+0000 7fb8eee00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e80ccc40 con 0x7fb8e8160250 2024-09-21T07:51:01.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.216+0000 7fb8eee00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb8e80ccf40 con 0x7fb8e80d0390 2024-09-21T07:51:01.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.216+0000 7fb8ee400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8e8160660 0x7fb8e81e65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:01.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.216+0000 7fb8eda00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8e80d0390 0x7fb8e81eb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:01.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.216+0000 7fb8eda00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8e80d0390 0x7fb8e81eb6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.39:44766/0 (socket says 172.21.15.39:44766) 2024-09-21T07:51:01.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.216+0000 7fb8eda00640 1 -- 172.21.15.39:0/1527147642 learned_addr learned my addr 172.21.15.39:0/1527147642 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:51:01.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.216+0000 7fb8ed000640 1 --2- 172.21.15.39:0/1527147642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8e8160250 0x7fb8e81e6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:01.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.217+0000 7fb8eda00640 1 -- 172.21.15.39:0/1527147642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8e8160250 msgr2=0x7fb8e81e6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:01.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.217+0000 7fb8eda00640 1 --2- 172.21.15.39:0/1527147642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8e8160250 0x7fb8e81e6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:01.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.217+0000 7fb8eda00640 1 -- 172.21.15.39:0/1527147642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8e8160660 msgr2=0x7fb8e81e65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:01.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.217+0000 7fb8eda00640 1 --2- 172.21.15.39:0/1527147642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8e8160660 0x7fb8e81e65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:01.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.217+0000 7fb8eda00640 1 -- 172.21.15.39:0/1527147642 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e81e91d0 con 0x7fb8e80d0390 2024-09-21T07:51:01.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.217+0000 7fb8ee400640 1 --2- 172.21.15.39:0/1527147642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8e8160660 0x7fb8e81e65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:51:01.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.217+0000 7fb8eda00640 1 --2- 172.21.15.39:0/1527147642 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8e80d0390 0x7fb8e81eb6d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc034040 tx=0x7fb8dc0035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:01.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.217+0000 7fb8e5600640 1 -- 172.21.15.39:0/1527147642 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc03e030 con 0x7fb8e80d0390 2024-09-21T07:51:01.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.217+0000 7fb8e5600640 1 -- 172.21.15.39:0/1527147642 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8dc003ae0 con 0x7fb8e80d0390 2024-09-21T07:51:01.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.217+0000 7fb8e5600640 1 -- 172.21.15.39:0/1527147642 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc035040 con 0x7fb8e80d0390 2024-09-21T07:51:01.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.217+0000 7fb8eee00640 1 -- 172.21.15.39:0/1527147642 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8e81e94b0 con 0x7fb8e80d0390 2024-09-21T07:51:01.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.218+0000 7fb8eee00640 1 -- 172.21.15.39:0/1527147642 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8e81e9a70 con 0x7fb8e80d0390 2024-09-21T07:51:01.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.218+0000 7fb8eee00640 1 -- 172.21.15.39:0/1527147642 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8e80cd460 con 0x7fb8e80d0390 2024-09-21T07:51:01.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.220+0000 7fb8e5600640 1 -- 172.21.15.39:0/1527147642 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb8dc041030 con 0x7fb8e80d0390 2024-09-21T07:51:01.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.221+0000 7fb8e5600640 1 --2- 172.21.15.39:0/1527147642 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb8c8078b90 0x7fb8c807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:01.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.221+0000 7fb8e5600640 1 -- 172.21.15.39:0/1527147642 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(241..241 src has 1..241) ==== 7476+0+0 (secure 0 0 0) 0x7fb8dc0bbaa0 con 0x7fb8e80d0390 2024-09-21T07:51:01.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.221+0000 7fb8ed000640 1 --2- 172.21.15.39:0/1527147642 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb8c8078b90 0x7fb8c807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:01.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.222+0000 7fb8ed000640 1 --2- 172.21.15.39:0/1527147642 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb8c8078b90 0x7fb8c807b0a0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fb8e80b4d90 tx=0x7fb8d4009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:01.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.225+0000 7fb8e5600640 1 -- 172.21.15.39:0/1527147642 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb8dc083180 con 0x7fb8e80d0390 2024-09-21T07:51:01.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.459+0000 7fb8eee00640 1 -- 172.21.15.39:0/1527147642 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8e8073190 con 0x7fb8e80d0390 2024-09-21T07:51:01.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.459+0000 7fb8e5600640 1 -- 172.21.15.39:0/1527147642 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 269 v269) ==== 95+0+4447 (secure 0 0 0) 0x7fb8dc087bb0 con 0x7fb8e80d0390 2024-09-21T07:51:01.462 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:01.462 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":269,"btime":"2024-09-21T07:51:00:908893+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":269,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:51:00.908874+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":241,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":11067},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11067":{"gid":11067,"name":"a","rank":2,"incarnation":269,"state":"up:replay","state_seq":1,"addr":"172.21.15.39:6834/109229385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":109229385},{"type":"v1","addr":"172.21.15.39:6834","nonce":109229385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[11052,9967]},"id":13}]} 2024-09-21T07:51:01.462 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 269 2024-09-21T07:51:01.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.462+0000 7fb8eee00640 1 -- 172.21.15.39:0/1527147642 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb8c8078b90 msgr2=0x7fb8c807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:01.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.462+0000 7fb8eee00640 1 --2- 172.21.15.39:0/1527147642 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb8c8078b90 0x7fb8c807b0a0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fb8e80b4d90 tx=0x7fb8d4009040 comp rx=0 tx=0).stop 2024-09-21T07:51:01.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.462+0000 7fb8eee00640 1 -- 172.21.15.39:0/1527147642 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8e80d0390 msgr2=0x7fb8e81eb6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:01.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.462+0000 7fb8eee00640 1 --2- 172.21.15.39:0/1527147642 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8e80d0390 0x7fb8e81eb6d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc034040 tx=0x7fb8dc0035f0 comp rx=0 tx=0).stop 2024-09-21T07:51:01.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.462+0000 7fb8eee00640 1 -- 172.21.15.39:0/1527147642 shutdown_connections 2024-09-21T07:51:01.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.462+0000 7fb8eee00640 1 --2- 172.21.15.39:0/1527147642 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb8c8078b90 0x7fb8c807b0a0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:01.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.462+0000 7fb8eee00640 1 --2- 172.21.15.39:0/1527147642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8e8160660 0x7fb8e81e65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:01.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.462+0000 7fb8eee00640 1 --2- 172.21.15.39:0/1527147642 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8e8160250 0x7fb8e81e6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:01.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.462+0000 7fb8eee00640 1 --2- 172.21.15.39:0/1527147642 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8e80d0390 0x7fb8e81eb6d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:01.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.462+0000 7fb8eee00640 1 -- 172.21.15.39:0/1527147642 >> 172.21.15.39:0/1527147642 conn(0x7fb8e806cfc0 msgr2=0x7fb8e80735b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:01.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.462+0000 7fb8eee00640 1 -- 172.21.15.39:0/1527147642 shutdown_connections 2024-09-21T07:51:01.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:01.462+0000 7fb8eee00640 1 -- 172.21.15.39:0/1527147642 wait complete. 2024-09-21T07:51:01.486 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-09-21T07:51:03.514 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:51:03.519 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:51:05.012 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-09-21T07:51:05.882 INFO:tasks.ceph.mds.b.smithi148.stderr:2024-09-21T07:51:05.881+0000 7fa0be800640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-21T07:51:06.487 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:51:06.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.723+0000 7faab1200640 1 -- 172.21.15.39:0/4142311849 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa90003a30 msgr2=0x7faa90003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:06.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.723+0000 7faab1200640 1 --2- 172.21.15.39:0/4142311849 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa90003a30 0x7faa90003ea0 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7faa9c00c310 tx=0x7faa90005830 comp rx=0 tx=0).stop 2024-09-21T07:51:06.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.723+0000 7faaaa800640 1 -- 172.21.15.39:0/4142311849 reap_dead start 2024-09-21T07:51:06.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.723+0000 7faab1200640 1 -- 172.21.15.39:0/4142311849 shutdown_connections 2024-09-21T07:51:06.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.723+0000 7faab1200640 1 -- 172.21.15.39:0/4142311849 >> 172.21.15.39:0/4142311849 conn(0x7faaac06cfc0 msgr2=0x7faaac06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:06.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.723+0000 7faab1200640 1 -- 172.21.15.39:0/4142311849 shutdown_connections 2024-09-21T07:51:06.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.723+0000 7faab1200640 1 -- 172.21.15.39:0/4142311849 wait complete. 2024-09-21T07:51:06.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.726+0000 7faab1200640 1 Processor -- start 2024-09-21T07:51:06.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.726+0000 7faab1200640 1 -- start start 2024-09-21T07:51:06.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.727+0000 7faab1200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa90003a30 0x7faaac151030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:06.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.727+0000 7faab1200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faaac15e2c0 0x7faaac15e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:06.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.727+0000 7faab1200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faaac1f7990 0x7faaac1f7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:06.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.727+0000 7faab1200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7faaac0d0d40 con 0x7faaac15e2c0 2024-09-21T07:51:06.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.727+0000 7faab1200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7faaac070120 con 0x7faa90003a30 2024-09-21T07:51:06.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.727+0000 7faab1200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7faaac0d0ec0 con 0x7faaac1f7990 2024-09-21T07:51:06.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.727+0000 7faaa3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faaac15e2c0 0x7faaac15e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:06.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.727+0000 7faaa3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faaac15e2c0 0x7faaac15e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44384/0 (socket says 172.21.15.39:44384) 2024-09-21T07:51:06.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.727+0000 7faaa3e00640 1 -- 172.21.15.39:0/2734879574 learned_addr learned my addr 172.21.15.39:0/2734879574 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:51:06.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.728+0000 7faaaa800640 1 --2- 172.21.15.39:0/2734879574 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faaac1f7990 0x7faaac1f7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:06.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.728+0000 7faaa9e00640 1 --2- 172.21.15.39:0/2734879574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa90003a30 0x7faaac151030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:06.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.728+0000 7faaa3e00640 1 -- 172.21.15.39:0/2734879574 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faaac1f7990 msgr2=0x7faaac1f7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:06.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.728+0000 7faaa3e00640 1 --2- 172.21.15.39:0/2734879574 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faaac1f7990 0x7faaac1f7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:06.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.728+0000 7faaa3e00640 1 -- 172.21.15.39:0/2734879574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa90003a30 msgr2=0x7faaac151030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:06.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.728+0000 7faaa3e00640 1 --2- 172.21.15.39:0/2734879574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa90003a30 0x7faaac151030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:06.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.728+0000 7faaa3e00640 1 -- 172.21.15.39:0/2734879574 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaac0b2da0 con 0x7faaac15e2c0 2024-09-21T07:51:06.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.728+0000 7faaaa800640 1 --2- 172.21.15.39:0/2734879574 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faaac1f7990 0x7faaac1f7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:51:06.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.728+0000 7faaa9e00640 1 --2- 172.21.15.39:0/2734879574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa90003a30 0x7faaac151030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:51:06.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.728+0000 7faaa3e00640 1 --2- 172.21.15.39:0/2734879574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faaac15e2c0 0x7faaac15e6d0 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7faa94004df0 tx=0x7faa94001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:06.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.728+0000 7faaa2000640 1 -- 172.21.15.39:0/2734879574 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa94001d40 con 0x7faaac15e2c0 2024-09-21T07:51:06.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.728+0000 7faaa2000640 1 -- 172.21.15.39:0/2734879574 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa94016070 con 0x7faaac15e2c0 2024-09-21T07:51:06.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.728+0000 7faaa2000640 1 -- 172.21.15.39:0/2734879574 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa94010ac0 con 0x7faaac15e2c0 2024-09-21T07:51:06.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.728+0000 7faab1200640 1 -- 172.21.15.39:0/2734879574 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaac1f50e0 con 0x7faaac15e2c0 2024-09-21T07:51:06.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.729+0000 7faab1200640 1 -- 172.21.15.39:0/2734879574 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaac1f56a0 con 0x7faaac15e2c0 2024-09-21T07:51:06.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.729+0000 7faab1200640 1 -- 172.21.15.39:0/2734879574 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaac0cd3c0 con 0x7faaac15e2c0 2024-09-21T07:51:06.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.731+0000 7faaa2000640 1 -- 172.21.15.39:0/2734879574 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7faa94010c60 con 0x7faaac15e2c0 2024-09-21T07:51:06.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.732+0000 7faaa2000640 1 --2- 172.21.15.39:0/2734879574 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faa88078b90 0x7faa8807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:06.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.732+0000 7faaa2000640 1 -- 172.21.15.39:0/2734879574 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(241..241 src has 1..241) ==== 7476+0+0 (secure 0 0 0) 0x7faa9409bd80 con 0x7faaac15e2c0 2024-09-21T07:51:06.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.732+0000 7faaa9e00640 1 --2- 172.21.15.39:0/2734879574 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faa88078b90 0x7faa8807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:06.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.733+0000 7faaa9e00640 1 --2- 172.21.15.39:0/2734879574 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faa88078b90 0x7faa8807b0a0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7faa9c033000 tx=0x7faa9c0019f0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:06.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.737+0000 7faaa2000640 1 -- 172.21.15.39:0/2734879574 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7faaac0cd3c0 con 0x7faaac15e2c0 2024-09-21T07:51:06.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.974+0000 7faab1200640 1 -- 172.21.15.39:0/2734879574 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faaac0cf920 con 0x7faaac15e2c0 2024-09-21T07:51:06.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.975+0000 7faaa2000640 1 -- 172.21.15.39:0/2734879574 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 273 v273) ==== 95+0+5280 (secure 0 0 0) 0x7faa94063a30 con 0x7faaac15e2c0 2024-09-21T07:51:06.977 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:06.977 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":273,"btime":"2024-09-21T07:51:04:941764+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10195,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":270}],"filesystems":[{"mdsmap":{"epoch":273,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:51:04.941752+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":241,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":11067},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11067":{"gid":11067,"name":"a","rank":2,"incarnation":269,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6834/109229385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":109229385},{"type":"v1","addr":"172.21.15.39:6834","nonce":109229385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[9967,11052,11067]},"id":13}]} 2024-09-21T07:51:06.977 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 273 2024-09-21T07:51:06.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.977+0000 7faab1200640 1 -- 172.21.15.39:0/2734879574 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faa88078b90 msgr2=0x7faa8807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:06.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.977+0000 7faab1200640 1 --2- 172.21.15.39:0/2734879574 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faa88078b90 0x7faa8807b0a0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7faa9c033000 tx=0x7faa9c0019f0 comp rx=0 tx=0).stop 2024-09-21T07:51:06.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.977+0000 7faab1200640 1 -- 172.21.15.39:0/2734879574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faaac15e2c0 msgr2=0x7faaac15e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:06.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.977+0000 7faab1200640 1 --2- 172.21.15.39:0/2734879574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faaac15e2c0 0x7faaac15e6d0 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7faa94004df0 tx=0x7faa94001630 comp rx=0 tx=0).stop 2024-09-21T07:51:06.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.977+0000 7faab1200640 1 -- 172.21.15.39:0/2734879574 shutdown_connections 2024-09-21T07:51:06.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.977+0000 7faab1200640 1 --2- 172.21.15.39:0/2734879574 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7faa88078b90 0x7faa8807b0a0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:06.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.977+0000 7faab1200640 1 --2- 172.21.15.39:0/2734879574 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faaac1f7990 0x7faaac1f7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:06.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.977+0000 7faab1200640 1 --2- 172.21.15.39:0/2734879574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faaac15e2c0 0x7faaac15e6d0 unknown :-1 s=CLOSED pgs=2410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:06.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.977+0000 7faab1200640 1 --2- 172.21.15.39:0/2734879574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa90003a30 0x7faaac151030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:06.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.977+0000 7faab1200640 1 -- 172.21.15.39:0/2734879574 >> 172.21.15.39:0/2734879574 conn(0x7faaac06cfc0 msgr2=0x7faaac15ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:06.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.977+0000 7faab1200640 1 -- 172.21.15.39:0/2734879574 shutdown_connections 2024-09-21T07:51:06.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:06.978+0000 7faab1200640 1 -- 172.21.15.39:0/2734879574 wait complete. 2024-09-21T07:51:07.007 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-21T07:51:07.007 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:51:07.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.232+0000 7fe1fc800640 1 -- 172.21.15.39:0/1543336211 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1f40b3990 msgr2=0x7fe1f40b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:07.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.232+0000 7fe1fc800640 1 --2- 172.21.15.39:0/1543336211 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1f40b3990 0x7fe1f40b4450 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4005fd0 tx=0x7fe1e402a5d0 comp rx=0 tx=0).stop 2024-09-21T07:51:07.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.232+0000 7fe1fc800640 1 -- 172.21.15.39:0/1543336211 shutdown_connections 2024-09-21T07:51:07.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.232+0000 7fe1fc800640 1 --2- 172.21.15.39:0/1543336211 >> v2:172.21.15.148:3300/0 conn(0x7fe1f40b3da0 0x7fe1f40b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:07.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.232+0000 7fe1fc800640 1 --2- 172.21.15.39:0/1543336211 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1f40b3990 0x7fe1f40b4450 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:07.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.232+0000 7fe1fc800640 1 -- 172.21.15.39:0/1543336211 >> 172.21.15.39:0/1543336211 conn(0x7fe1f40b2460 msgr2=0x7fe1f406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:07.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.232+0000 7fe1fc800640 1 -- 172.21.15.39:0/1543336211 shutdown_connections 2024-09-21T07:51:07.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.232+0000 7fe1fc800640 1 -- 172.21.15.39:0/1543336211 wait complete. 2024-09-21T07:51:07.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.234+0000 7fe1fc800640 1 Processor -- start 2024-09-21T07:51:07.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.234+0000 7fe1fc800640 1 -- start start 2024-09-21T07:51:07.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.236+0000 7fe1fc800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1f40b3990 0x7fe1f40c6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:07.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.236+0000 7fe1fc800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1f40b3da0 0x7fe1f41f94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:07.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.236+0000 7fe1fc800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1f4162f10 0x7fe1f41f9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:07.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.236+0000 7fe1fc800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f40af200 con 0x7fe1f40b3da0 2024-09-21T07:51:07.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.236+0000 7fe1fc800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f40af080 con 0x7fe1f40b3990 2024-09-21T07:51:07.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.236+0000 7fe1fc800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe1f40af380 con 0x7fe1f4162f10 2024-09-21T07:51:07.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.236+0000 7fe1f9400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1f40b3da0 0x7fe1f41f94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:07.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.236+0000 7fe1f9400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1f40b3da0 0x7fe1f41f94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44398/0 (socket says 172.21.15.39:44398) 2024-09-21T07:51:07.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.236+0000 7fe1f9400640 1 -- 172.21.15.39:0/311487352 learned_addr learned my addr 172.21.15.39:0/311487352 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:51:07.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.236+0000 7fe1f9e00640 1 --2- 172.21.15.39:0/311487352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1f40b3990 0x7fe1f40c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:07.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.236+0000 7fe1fa800640 1 --2- 172.21.15.39:0/311487352 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1f4162f10 0x7fe1f41f9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:07.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.237+0000 7fe1f9400640 1 -- 172.21.15.39:0/311487352 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1f4162f10 msgr2=0x7fe1f41f9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:07.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.237+0000 7fe1f9400640 1 --2- 172.21.15.39:0/311487352 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1f4162f10 0x7fe1f41f9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:07.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.237+0000 7fe1f9400640 1 -- 172.21.15.39:0/311487352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1f40b3990 msgr2=0x7fe1f40c6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:07.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.237+0000 7fe1f9400640 1 --2- 172.21.15.39:0/311487352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1f40b3990 0x7fe1f40c6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:07.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.237+0000 7fe1f9400640 1 -- 172.21.15.39:0/311487352 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f415c3e0 con 0x7fe1f40b3da0 2024-09-21T07:51:07.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.237+0000 7fe1f9400640 1 --2- 172.21.15.39:0/311487352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1f40b3da0 0x7fe1f41f94e0 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4000c00 tx=0x7fe1e40035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:07.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.237+0000 7fe1ee000640 1 -- 172.21.15.39:0/311487352 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e403e030 con 0x7fe1f40b3da0 2024-09-21T07:51:07.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.237+0000 7fe1ee000640 1 -- 172.21.15.39:0/311487352 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1e4003ae0 con 0x7fe1f40b3da0 2024-09-21T07:51:07.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.237+0000 7fe1fc800640 1 -- 172.21.15.39:0/311487352 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1f415c6c0 con 0x7fe1f40b3da0 2024-09-21T07:51:07.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.237+0000 7fe1ee000640 1 -- 172.21.15.39:0/311487352 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4035040 con 0x7fe1f40b3da0 2024-09-21T07:51:07.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.238+0000 7fe1fc800640 1 -- 172.21.15.39:0/311487352 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1f415cc80 con 0x7fe1f40b3da0 2024-09-21T07:51:07.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.239+0000 7fe1fc800640 1 -- 172.21.15.39:0/311487352 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1f40af870 con 0x7fe1f40b3da0 2024-09-21T07:51:07.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.240+0000 7fe1ee000640 1 -- 172.21.15.39:0/311487352 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe1e4041030 con 0x7fe1f40b3da0 2024-09-21T07:51:07.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.241+0000 7fe1ee000640 1 --2- 172.21.15.39:0/311487352 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe1d8078bc0 0x7fe1d807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:07.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.241+0000 7fe1ee000640 1 -- 172.21.15.39:0/311487352 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(241..241 src has 1..241) ==== 7476+0+0 (secure 0 0 0) 0x7fe1e40bb880 con 0x7fe1f40b3da0 2024-09-21T07:51:07.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.241+0000 7fe1f9e00640 1 --2- 172.21.15.39:0/311487352 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe1d8078bc0 0x7fe1d807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:07.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.242+0000 7fe1f9e00640 1 --2- 172.21.15.39:0/311487352 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe1d8078bc0 0x7fe1d807b0d0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4075150 tx=0x7fe1e8009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:07.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.245+0000 7fe1ee000640 1 -- 172.21.15.39:0/311487352 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe1e4082ef0 con 0x7fe1f40b3da0 2024-09-21T07:51:07.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.481+0000 7fe1fc800640 1 -- 172.21.15.39:0/311487352 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1f4072d80 con 0x7fe1f40b3da0 2024-09-21T07:51:07.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.482+0000 7fe1ee000640 1 -- 172.21.15.39:0/311487352 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 273 v273) ==== 95+0+5280 (secure 0 0 0) 0x7fe1e4087920 con 0x7fe1f40b3da0 2024-09-21T07:51:07.484 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:07.484 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":273,"btime":"2024-09-21T07:51:04:941764+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10195,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":270}],"filesystems":[{"mdsmap":{"epoch":273,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:51:04.941752+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":241,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":11067},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11067":{"gid":11067,"name":"a","rank":2,"incarnation":269,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6834/109229385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":109229385},{"type":"v1","addr":"172.21.15.39:6834","nonce":109229385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[9967,11052,11067]},"id":13}]} 2024-09-21T07:51:07.484 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 273 2024-09-21T07:51:07.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.484+0000 7fe1fc800640 1 -- 172.21.15.39:0/311487352 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe1d8078bc0 msgr2=0x7fe1d807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:07.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.484+0000 7fe1fc800640 1 --2- 172.21.15.39:0/311487352 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe1d8078bc0 0x7fe1d807b0d0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4075150 tx=0x7fe1e8009040 comp rx=0 tx=0).stop 2024-09-21T07:51:07.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.484+0000 7fe1fc800640 1 -- 172.21.15.39:0/311487352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1f40b3da0 msgr2=0x7fe1f41f94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:07.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.484+0000 7fe1fc800640 1 --2- 172.21.15.39:0/311487352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1f40b3da0 0x7fe1f41f94e0 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4000c00 tx=0x7fe1e40035f0 comp rx=0 tx=0).stop 2024-09-21T07:51:07.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.484+0000 7fe1fc800640 1 -- 172.21.15.39:0/311487352 shutdown_connections 2024-09-21T07:51:07.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.484+0000 7fe1fc800640 1 --2- 172.21.15.39:0/311487352 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe1d8078bc0 0x7fe1d807b0d0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:07.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.484+0000 7fe1fc800640 1 --2- 172.21.15.39:0/311487352 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1f4162f10 0x7fe1f41f9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:07.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.484+0000 7fe1fc800640 1 --2- 172.21.15.39:0/311487352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1f40b3da0 0x7fe1f41f94e0 unknown :-1 s=CLOSED pgs=2411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:07.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.484+0000 7fe1fc800640 1 --2- 172.21.15.39:0/311487352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1f40b3990 0x7fe1f40c6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:07.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.484+0000 7fe1fc800640 1 -- 172.21.15.39:0/311487352 >> 172.21.15.39:0/311487352 conn(0x7fe1f40b2460 msgr2=0x7fe1f41600d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:07.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.485+0000 7fe1fc800640 1 -- 172.21.15.39:0/311487352 shutdown_connections 2024-09-21T07:51:07.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:07.485+0000 7fe1fc800640 1 -- 172.21.15.39:0/311487352 wait complete. 2024-09-21T07:51:07.911 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 273, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:51:04.941752+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 241, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 9967, 'mds_1': 11052, 'mds_2': 11067}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9967': {'gid': 9967, 'name': 'b', 'rank': 0, 'incarnation': 259, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.148:6836/3381818843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 3381818843}, {'type': 'v1', 'addr': '172.21.15.148:6836', 'nonce': 3381818843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11052': {'gid': 11052, 'name': 'c', 'rank': 1, 'incarnation': 261, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6835/485194245', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 485194245}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 485194245}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11067': {'gid': 11067, 'name': 'a', 'rank': 2, 'incarnation': 269, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.39:6834/109229385', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 109229385}, {'type': 'v1', 'addr': '172.21.15.39:6834', 'nonce': 109229385}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9967, 'qdb_cluster': [9967, 11052, 11067]} 2024-09-21T07:51:07.911 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-21T07:51:07.911 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-21T07:51:08.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.140+0000 7f121d200640 1 -- 172.21.15.39:0/2211691632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12180d0630 msgr2=0x7f12180d0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:08.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.140+0000 7f121d200640 1 --2- 172.21.15.39:0/2211691632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12180d0630 0x7f12180d0a40 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7f1218072900 tx=0x7f121002aa30 comp rx=0 tx=0).stop 2024-09-21T07:51:08.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.140+0000 7f121d200640 1 -- 172.21.15.39:0/2211691632 shutdown_connections 2024-09-21T07:51:08.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.140+0000 7f121d200640 1 --2- 172.21.15.39:0/2211691632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12180d0630 0x7f12180d0a40 unknown :-1 s=CLOSED pgs=2412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.140+0000 7f121d200640 1 --2- 172.21.15.39:0/2211691632 >> v2:172.21.15.148:3300/0 conn(0x7f1218073dd0 0x7f121815d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.140+0000 7f121d200640 1 -- 172.21.15.39:0/2211691632 >> 172.21.15.39:0/2211691632 conn(0x7f121806cfc0 msgr2=0x7f121806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:08.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.140+0000 7f121d200640 1 -- 172.21.15.39:0/2211691632 shutdown_connections 2024-09-21T07:51:08.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.140+0000 7f121d200640 1 -- 172.21.15.39:0/2211691632 wait complete. 2024-09-21T07:51:08.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.143+0000 7f121d200640 1 Processor -- start 2024-09-21T07:51:08.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.143+0000 7f121d200640 1 -- start start 2024-09-21T07:51:08.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.144+0000 7f121d200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1218073dd0 0x7f121814ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.144+0000 7f121d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12180d0630 0x7f12180b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.144+0000 7f121d200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f121815daf0 0x7f121815b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.144+0000 7f121d200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f12180cd1f0 con 0x7f12180d0630 2024-09-21T07:51:08.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.144+0000 7f121d200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f12180cd070 con 0x7f1218073dd0 2024-09-21T07:51:08.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.144+0000 7f121d200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f12180cd370 con 0x7f121815daf0 2024-09-21T07:51:08.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.145+0000 7f1215400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12180d0630 0x7f12180b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.145+0000 7f1215400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12180d0630 0x7f12180b4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44410/0 (socket says 172.21.15.39:44410) 2024-09-21T07:51:08.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.145+0000 7f1215400640 1 -- 172.21.15.39:0/1845416511 learned_addr learned my addr 172.21.15.39:0/1845416511 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:51:08.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.145+0000 7f1216800640 1 --2- 172.21.15.39:0/1845416511 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f121815daf0 0x7f121815b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.145+0000 7f1215e00640 1 --2- 172.21.15.39:0/1845416511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1218073dd0 0x7f121814ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.145+0000 7f1215400640 1 -- 172.21.15.39:0/1845416511 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f121815daf0 msgr2=0x7f121815b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:08.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.145+0000 7f1215400640 1 --2- 172.21.15.39:0/1845416511 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f121815daf0 0x7f121815b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.145+0000 7f1215400640 1 -- 172.21.15.39:0/1845416511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1218073dd0 msgr2=0x7f121814ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:08.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.145+0000 7f1215400640 1 --2- 172.21.15.39:0/1845416511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1218073dd0 0x7f121814ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.145+0000 7f1215400640 1 -- 172.21.15.39:0/1845416511 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12181f8da0 con 0x7f12180d0630 2024-09-21T07:51:08.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.145+0000 7f1215400640 1 --2- 172.21.15.39:0/1845416511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12180d0630 0x7f12180b4d30 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7f120400c340 tx=0x7f120400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:08.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.146+0000 7f120e000640 1 -- 172.21.15.39:0/1845416511 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12040052d0 con 0x7f12180d0630 2024-09-21T07:51:08.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.146+0000 7f120e000640 1 -- 172.21.15.39:0/1845416511 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1204007020 con 0x7f12180d0630 2024-09-21T07:51:08.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.146+0000 7f120e000640 1 -- 172.21.15.39:0/1845416511 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204007470 con 0x7f12180d0630 2024-09-21T07:51:08.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.146+0000 7f121d200640 1 -- 172.21.15.39:0/1845416511 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12181f9080 con 0x7f12180d0630 2024-09-21T07:51:08.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.146+0000 7f121d200640 1 -- 172.21.15.39:0/1845416511 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12181f9640 con 0x7f12180d0630 2024-09-21T07:51:08.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.148+0000 7f120e000640 1 -- 172.21.15.39:0/1845416511 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f12040067d0 con 0x7f12180d0630 2024-09-21T07:51:08.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.149+0000 7f120e000640 1 --2- 172.21.15.39:0/1845416511 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f11f8078b90 0x7f11f807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.149+0000 7f120e000640 1 -- 172.21.15.39:0/1845416511 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(241..241 src has 1..241) ==== 7476+0+0 (secure 0 0 0) 0x7f120409a850 con 0x7f12180d0630 2024-09-21T07:51:08.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.150+0000 7f1224208740 1 Processor -- start 2024-09-21T07:51:08.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.150+0000 7f1215e00640 1 --2- 172.21.15.39:0/1845416511 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f11f8078b90 0x7f11f807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.150+0000 7f1224208740 1 -- start start 2024-09-21T07:51:08.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.151+0000 7f1215e00640 1 --2- 172.21.15.39:0/1845416511 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f11f8078b90 0x7f11f807b0a0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f12180730e0 tx=0x7f1208005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:08.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.152+0000 7f1224208740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55713d5db270 0x55713d9ab970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.152+0000 7f1224208740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55713d5b23b0 0x55713d9abeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.152+0000 7f1224208740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55713d5ba590 0x55713d97bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.152+0000 7f1224208740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55713d905430 con 0x55713d5db270 2024-09-21T07:51:08.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.152+0000 7f1224208740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55713d8f55d0 con 0x55713d5b23b0 2024-09-21T07:51:08.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.152+0000 7f1224208740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55713d8e4a00 con 0x55713d5ba590 2024-09-21T07:51:08.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.152+0000 7f1215e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55713d5db270 0x55713d9ab970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.152+0000 7f1215e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55713d5db270 0x55713d9ab970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44412/0 (socket says 172.21.15.39:44412) 2024-09-21T07:51:08.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.152+0000 7f1215e00640 1 -- 172.21.15.39:0/2759446968 learned_addr learned my addr 172.21.15.39:0/2759446968 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:51:08.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.152+0000 7f1216800640 1 --2- 172.21.15.39:0/2759446968 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55713d5ba590 0x55713d97bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.152+0000 7f1215400640 1 --2- 172.21.15.39:0/2759446968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55713d5b23b0 0x55713d9abeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.152+0000 7f1215e00640 1 -- 172.21.15.39:0/2759446968 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55713d5ba590 msgr2=0x55713d97bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:51:08.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.152+0000 7f1215e00640 1 --2- 172.21.15.39:0/2759446968 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55713d5ba590 0x55713d97bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.152+0000 7f1215e00640 1 -- 172.21.15.39:0/2759446968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55713d5b23b0 msgr2=0x55713d9abeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:51:08.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.152+0000 7f1215e00640 1 --2- 172.21.15.39:0/2759446968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55713d5b23b0 0x55713d9abeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.152+0000 7f1215e00640 1 -- 172.21.15.39:0/2759446968 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55713d8ad520 con 0x55713d5db270 2024-09-21T07:51:08.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.153+0000 7f1215e00640 1 --2- 172.21.15.39:0/2759446968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55713d5db270 0x55713d9ab970 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f1208011e30 tx=0x7f1208014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eac694dafb96db43 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:08.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.153+0000 7f1202a00640 1 -- 172.21.15.39:0/2759446968 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120800eb30 con 0x55713d5db270 2024-09-21T07:51:08.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.153+0000 7f1202a00640 1 -- 172.21.15.39:0/2759446968 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f120800ecd0 con 0x55713d5db270 2024-09-21T07:51:08.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.153+0000 7f1202a00640 1 -- 172.21.15.39:0/2759446968 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1208026670 con 0x55713d5db270 2024-09-21T07:51:08.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.154+0000 7f1224208740 1 -- 172.21.15.39:0/2759446968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55713d5db270 msgr2=0x55713d9ab970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:08.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.154+0000 7f1224208740 1 --2- 172.21.15.39:0/2759446968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55713d5db270 0x55713d9ab970 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f1208011e30 tx=0x7f1208014000 comp rx=0 tx=0).stop 2024-09-21T07:51:08.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.154+0000 7f1224208740 1 -- 172.21.15.39:0/2759446968 shutdown_connections 2024-09-21T07:51:08.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.154+0000 7f1224208740 1 --2- 172.21.15.39:0/2759446968 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55713d5ba590 0x55713d97bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.154+0000 7f1224208740 1 --2- 172.21.15.39:0/2759446968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55713d5b23b0 0x55713d9abeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.154+0000 7f1224208740 1 --2- 172.21.15.39:0/2759446968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55713d5db270 0x55713d9ab970 unknown :-1 s=CLOSED pgs=2414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.154+0000 7f1224208740 1 -- 172.21.15.39:0/2759446968 >> 172.21.15.39:0/2759446968 conn(0x55713d5cf3c0 msgr2=0x55713d981970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:08.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.154+0000 7f1224208740 1 -- 172.21.15.39:0/2759446968 shutdown_connections 2024-09-21T07:51:08.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.154+0000 7f1224208740 1 -- 172.21.15.39:0/2759446968 wait complete. 2024-09-21T07:51:08.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.155+0000 7f1224208740 1 -- start start 2024-09-21T07:51:08.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.155+0000 7f1224208740 1 Processor -- start 2024-09-21T07:51:08.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.157+0000 7f1224208740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55713d5b23b0 0x55713d990300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.157+0000 7f1224208740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55713d5ba590 0x55713d9d9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.157+0000 7f1224208740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55713d5db270 0x55713d9c0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.157+0000 7f1224208740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55713d88b330 con 0x55713d5b23b0 2024-09-21T07:51:08.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.157+0000 7f1224208740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55713d8e8810 con 0x55713d5db270 2024-09-21T07:51:08.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.157+0000 7f1224208740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55713d655620 con 0x55713d5ba590 2024-09-21T07:51:08.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.157+0000 7f1224208740 10 client.0 _ll_register_callbacks cb 0x55713d890420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:51:08.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.157+0000 7f1215e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55713d5b23b0 0x55713d990300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.157+0000 7f1215e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55713d5b23b0 0x55713d990300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44414/0 (socket says 172.21.15.39:44414) 2024-09-21T07:51:08.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.157+0000 7f1215e00640 1 -- 172.21.15.39:0/1674380812 learned_addr learned my addr 172.21.15.39:0/1674380812 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:51:08.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.157+0000 7f1215e00640 1 -- 172.21.15.39:0/1674380812 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55713d5ba590 msgr2=0x55713d9d9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:51:08.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.157+0000 7f1215e00640 1 --2- 172.21.15.39:0/1674380812 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55713d5ba590 0x55713d9d9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.157+0000 7f1216800640 1 --2- 172.21.15.39:0/1674380812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55713d5db270 0x55713d9c0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.157+0000 7f1215e00640 1 -- 172.21.15.39:0/1674380812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55713d5db270 msgr2=0x55713d9c0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:51:08.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.157+0000 7f1215e00640 1 --2- 172.21.15.39:0/1674380812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55713d5db270 0x55713d9c0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.157+0000 7f1215e00640 1 -- 172.21.15.39:0/1674380812 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55713d646850 con 0x55713d5b23b0 2024-09-21T07:51:08.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.157+0000 7f1216800640 1 --2- 172.21.15.39:0/1674380812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55713d5db270 0x55713d9c0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:51:08.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.158+0000 7f1215e00640 1 --2- 172.21.15.39:0/1674380812 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55713d5b23b0 0x55713d990300 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f1208005b90 tx=0x7f120800e930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4967f8d56c64c1d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:08.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.158+0000 7f11f7400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:51:08.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.158+0000 7f11f7400640 1 -- 172.21.15.39:0/1674380812 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120800eb20 con 0x55713d5b23b0 2024-09-21T07:51:08.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.158+0000 7f11f7400640 1 -- 172.21.15.39:0/1674380812 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(241..241 src has 1..241) ==== 7476+0+0 (secure 0 0 0) 0x7f1208009040 con 0x55713d5b23b0 2024-09-21T07:51:08.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.158+0000 7f1224208740 1 -- 172.21.15.39:0/1674380812 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f120800ecd0 con 0x55713d5b23b0 2024-09-21T07:51:08.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.158+0000 7f11f7400640 1 -- 172.21.15.39:0/1674380812 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=273) ==== 24+0+0 (secure 0 0 0) 0x7f1208039050 con 0x55713d5b23b0 2024-09-21T07:51:08.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.158+0000 7f1224208740 10 client.11247 fetch_fsmap learned FSMap version 273 2024-09-21T07:51:08.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.158+0000 7f1224208740 1 -- 172.21.15.39:0/1674380812 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=273,monmap=2+}) -- 0x55713d6bd6b0 con 0x55713d5b23b0 2024-09-21T07:51:08.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.158+0000 7f11f7400640 1 -- 172.21.15.39:0/1674380812 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f120800ecd0 con 0x55713d5b23b0 2024-09-21T07:51:08.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.159+0000 7f11f7400640 1 -- 172.21.15.39:0/1674380812 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 273) ==== 3542+0+0 (secure 0 0 0) 0x7f1208004110 con 0x55713d5b23b0 2024-09-21T07:51:08.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.159+0000 7f1224208740 10 client.11247 fetch_fsmap finished waiting for FSMap version 273 2024-09-21T07:51:08.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.159+0000 7f1224208740 10 client.11247 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T07:51:08.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.159+0000 7f1224208740 20 client.11247 populate_metadata read hostname 'smithi039' 2024-09-21T07:51:08.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.159+0000 7f1224208740 1 --2- 172.21.15.39:0/1674380812 >> [v2:172.21.15.148:6834/3381818843,v1:172.21.15.148:6836/3381818843] conn(0x55713d599e40 0x55713d995810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.159+0000 7f1224208740 4 client.11247 mds_command: new command op to 9967 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:51:08.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.159+0000 7f1224208740 1 -- 172.21.15.39:0/1674380812 --> [v2:172.21.15.148:6834/3381818843,v1:172.21.15.148:6836/3381818843] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55713d8f7520 con 0x55713d599e40 2024-09-21T07:51:08.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.159+0000 7f1215400640 1 --2- 172.21.15.39:0/1674380812 >> [v2:172.21.15.148:6834/3381818843,v1:172.21.15.148:6836/3381818843] conn(0x55713d599e40 0x55713d995810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.160+0000 7f1215400640 1 --2- 172.21.15.39:0/1674380812 >> [v2:172.21.15.148:6834/3381818843,v1:172.21.15.148:6836/3381818843] conn(0x55713d599e40 0x55713d995810 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ca15461d4d2f689 server_cookie=a315a9e93a380cd3 in_seq=0 out_seq=0 2024-09-21T07:51:08.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.160+0000 7f11f7400640 10 client.11247 ms_handle_connect on v2:172.21.15.148:6834/3381818843 2024-09-21T07:51:08.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.161+0000 7f11f7400640 1 -- 172.21.15.39:0/1674380812 <== mds.0 v2:172.21.15.148:6834/3381818843 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55713d8f7520 con 0x55713d599e40 2024-09-21T07:51:08.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.161+0000 7f11f7400640 10 client.11247 handle_command_reply: tid=0 2024-09-21T07:51:08.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.161+0000 7f1224208740 1 client.11247 shutdown 2024-09-21T07:51:08.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.161+0000 7f1224208740 1 -- 172.21.15.39:0/1674380812 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55713d5b23b0 msgr2=0x55713d990300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:08.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.161+0000 7f1224208740 1 --2- 172.21.15.39:0/1674380812 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55713d5b23b0 0x55713d990300 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f1208005b90 tx=0x7f120800e930 comp rx=0 tx=0).stop 2024-09-21T07:51:08.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.161+0000 7f1224208740 1 -- 172.21.15.39:0/1674380812 shutdown_connections 2024-09-21T07:51:08.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.161+0000 7f1224208740 1 --2- 172.21.15.39:0/1674380812 >> [v2:172.21.15.148:6834/3381818843,v1:172.21.15.148:6836/3381818843] conn(0x55713d599e40 0x55713d995810 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.161+0000 7f1224208740 1 --2- 172.21.15.39:0/1674380812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55713d5db270 0x55713d9c0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.161+0000 7f1224208740 1 --2- 172.21.15.39:0/1674380812 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55713d5ba590 0x55713d9d9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.161+0000 7f1224208740 1 --2- 172.21.15.39:0/1674380812 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55713d5b23b0 0x55713d990300 unknown :-1 s=CLOSED pgs=2415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.161+0000 7f11f7400640 0 client.11247 ms_handle_reset on v2:172.21.15.148:6834/3381818843 2024-09-21T07:51:08.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.161+0000 7f1224208740 1 -- 172.21.15.39:0/1674380812 >> 172.21.15.39:0/1674380812 conn(0x55713d5cf3c0 msgr2=0x55713d9a2ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:08.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.161+0000 7f1224208740 1 -- 172.21.15.39:0/1674380812 shutdown_connections 2024-09-21T07:51:08.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.161+0000 7f1224208740 1 -- 172.21.15.39:0/1674380812 wait complete. 2024-09-21T07:51:08.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.161+0000 7f1224208740 20 client.11247 trim_cache size 0 max 16384 2024-09-21T07:51:08.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.181+0000 7f1224208740 1 Processor -- start 2024-09-21T07:51:08.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.181+0000 7f1224208740 1 -- start start 2024-09-21T07:51:08.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.182+0000 7f1224208740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55713d5b23b0 0x55713d97dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.182+0000 7f1224208740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55713d5ba590 0x55713d9817f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.182+0000 7f1224208740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55713d5db270 0x55713d981d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.182+0000 7f1224208740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f12182051a0 con 0x55713d5b23b0 2024-09-21T07:51:08.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.182+0000 7f1224208740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1218204400 con 0x55713d5ba590 2024-09-21T07:51:08.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.182+0000 7f1224208740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f1218204ad0 con 0x55713d5db270 2024-09-21T07:51:08.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.182+0000 7f1215e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55713d5b23b0 0x55713d97dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.182+0000 7f1215e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55713d5b23b0 0x55713d97dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44416/0 (socket says 172.21.15.39:44416) 2024-09-21T07:51:08.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.182+0000 7f1215e00640 1 -- 172.21.15.39:0/3520164014 learned_addr learned my addr 172.21.15.39:0/3520164014 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:51:08.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.182+0000 7f1215400640 1 --2- 172.21.15.39:0/3520164014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55713d5ba590 0x55713d9817f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.182+0000 7f1216800640 1 --2- 172.21.15.39:0/3520164014 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55713d5db270 0x55713d981d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.183+0000 7f1215e00640 1 -- 172.21.15.39:0/3520164014 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55713d5db270 msgr2=0x55713d981d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:51:08.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.183+0000 7f1215e00640 1 --2- 172.21.15.39:0/3520164014 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55713d5db270 0x55713d981d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.183+0000 7f1215e00640 1 -- 172.21.15.39:0/3520164014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55713d5ba590 msgr2=0x55713d9817f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:51:08.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.183+0000 7f1215e00640 1 --2- 172.21.15.39:0/3520164014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55713d5ba590 0x55713d9817f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.183+0000 7f1215e00640 1 -- 172.21.15.39:0/3520164014 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55713d982650 con 0x55713d5b23b0 2024-09-21T07:51:08.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.183+0000 7f1215e00640 1 --2- 172.21.15.39:0/3520164014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55713d5b23b0 0x55713d97dfc0 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f1208011890 tx=0x7f12080118c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6845050f75c796a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:08.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.183+0000 7f1202000640 1 -- 172.21.15.39:0/3520164014 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1208042070 con 0x55713d5b23b0 2024-09-21T07:51:08.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.183+0000 7f1202000640 1 -- 172.21.15.39:0/3520164014 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f120800e9a0 con 0x55713d5b23b0 2024-09-21T07:51:08.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.183+0000 7f1202000640 1 -- 172.21.15.39:0/3520164014 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1208026800 con 0x55713d5b23b0 2024-09-21T07:51:08.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.184+0000 7f1224208740 1 -- 172.21.15.39:0/3520164014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55713d5b23b0 msgr2=0x55713d97dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:08.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.184+0000 7f1224208740 1 --2- 172.21.15.39:0/3520164014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55713d5b23b0 0x55713d97dfc0 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f1208011890 tx=0x7f12080118c0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.184+0000 7f1224208740 1 -- 172.21.15.39:0/3520164014 shutdown_connections 2024-09-21T07:51:08.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.184+0000 7f1224208740 1 --2- 172.21.15.39:0/3520164014 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55713d5db270 0x55713d981d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.184+0000 7f1224208740 1 --2- 172.21.15.39:0/3520164014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55713d5ba590 0x55713d9817f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.184+0000 7f1224208740 1 --2- 172.21.15.39:0/3520164014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55713d5b23b0 0x55713d97dfc0 unknown :-1 s=CLOSED pgs=2416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.184+0000 7f1224208740 1 -- 172.21.15.39:0/3520164014 >> 172.21.15.39:0/3520164014 conn(0x55713d5cf3c0 msgr2=0x55713d97cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:08.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.184+0000 7f1224208740 1 -- 172.21.15.39:0/3520164014 shutdown_connections 2024-09-21T07:51:08.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.184+0000 7f1224208740 1 -- 172.21.15.39:0/3520164014 wait complete. 2024-09-21T07:51:08.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.185+0000 7f1224208740 1 -- start start 2024-09-21T07:51:08.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.185+0000 7f1224208740 1 Processor -- start 2024-09-21T07:51:08.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.187+0000 7f1224208740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55713d5b23b0 0x55713da42cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.187+0000 7f1224208740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55713d5ba590 0x55713d9c87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.187+0000 7f1224208740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55713d5db270 0x55713d9c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.187+0000 7f1224208740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55713d9a2600 con 0x55713d5b23b0 2024-09-21T07:51:08.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.187+0000 7f1224208740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1218203570 con 0x55713d5db270 2024-09-21T07:51:08.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.187+0000 7f1224208740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55713d9a6400 con 0x55713d5ba590 2024-09-21T07:51:08.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.187+0000 7f1224208740 10 client.0 _ll_register_callbacks cb 0x7f1208011d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:51:08.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.187+0000 7f1215e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55713d5b23b0 0x55713da42cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.187+0000 7f1215e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55713d5b23b0 0x55713da42cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44420/0 (socket says 172.21.15.39:44420) 2024-09-21T07:51:08.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.187+0000 7f1215e00640 1 -- 172.21.15.39:0/3578079096 learned_addr learned my addr 172.21.15.39:0/3578079096 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:51:08.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.187+0000 7f1215400640 1 --2- 172.21.15.39:0/3578079096 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55713d5ba590 0x55713d9c87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.187+0000 7f1216800640 1 --2- 172.21.15.39:0/3578079096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55713d5db270 0x55713d9c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.187+0000 7f1215e00640 1 -- 172.21.15.39:0/3578079096 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55713d5ba590 msgr2=0x55713d9c87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:51:08.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.187+0000 7f1215e00640 1 --2- 172.21.15.39:0/3578079096 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55713d5ba590 0x55713d9c87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.187+0000 7f1215e00640 1 -- 172.21.15.39:0/3578079096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55713d5db270 msgr2=0x55713d9c8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:51:08.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.187+0000 7f1215e00640 1 --2- 172.21.15.39:0/3578079096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55713d5db270 0x55713d9c8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.187+0000 7f1215e00640 1 -- 172.21.15.39:0/3578079096 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55713d9c82f0 con 0x55713d5b23b0 2024-09-21T07:51:08.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.187+0000 7f1216800640 1 --2- 172.21.15.39:0/3578079096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55713d5db270 0x55713d9c8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:51:08.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.188+0000 7f1215e00640 1 --2- 172.21.15.39:0/3578079096 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55713d5b23b0 0x55713da42cd0 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7f120800efd0 tx=0x7f120800ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6268855beed365 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:08.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.188+0000 7f11f7400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:51:08.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.188+0000 7f11f7400640 1 -- 172.21.15.39:0/3578079096 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1208042070 con 0x55713d5b23b0 2024-09-21T07:51:08.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.188+0000 7f11f7400640 1 -- 172.21.15.39:0/3578079096 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(241..241 src has 1..241) ==== 7476+0+0 (secure 0 0 0) 0x7f1208009040 con 0x55713d5b23b0 2024-09-21T07:51:08.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.188+0000 7f1224208740 1 -- 172.21.15.39:0/3578079096 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f120800e9a0 con 0x55713d5b23b0 2024-09-21T07:51:08.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.188+0000 7f11f7400640 1 -- 172.21.15.39:0/3578079096 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=273) ==== 24+0+0 (secure 0 0 0) 0x7f12080269c0 con 0x55713d5b23b0 2024-09-21T07:51:08.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.188+0000 7f1224208740 10 client.11253 fetch_fsmap learned FSMap version 273 2024-09-21T07:51:08.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.188+0000 7f1224208740 1 -- 172.21.15.39:0/3578079096 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=273,monmap=2+}) -- 0x55713da49da0 con 0x55713d5b23b0 2024-09-21T07:51:08.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.188+0000 7f11f7400640 1 -- 172.21.15.39:0/3578079096 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f120800e9a0 con 0x55713d5b23b0 2024-09-21T07:51:08.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.188+0000 7f11f7400640 1 -- 172.21.15.39:0/3578079096 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 273) ==== 3542+0+0 (secure 0 0 0) 0x7f1208026b60 con 0x55713d5b23b0 2024-09-21T07:51:08.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.188+0000 7f1224208740 10 client.11253 fetch_fsmap finished waiting for FSMap version 273 2024-09-21T07:51:08.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.188+0000 7f1224208740 10 client.11253 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T07:51:08.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.189+0000 7f1224208740 20 client.11253 populate_metadata read hostname 'smithi039' 2024-09-21T07:51:08.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.189+0000 7f1224208740 1 --2- 172.21.15.39:0/3578079096 >> [v2:172.21.15.148:6834/3381818843,v1:172.21.15.148:6836/3381818843] conn(0x55713da45e70 0x55713d9a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.189+0000 7f1224208740 4 client.11253 mds_command: new command op to 9967 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:51:08.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.189+0000 7f1224208740 1 -- 172.21.15.39:0/3578079096 --> [v2:172.21.15.148:6834/3381818843,v1:172.21.15.148:6836/3381818843] -- command(tid 0: {"prefix": "status"}) -- 0x55713d8f2fd0 con 0x55713da45e70 2024-09-21T07:51:08.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.189+0000 7f1215400640 1 --2- 172.21.15.39:0/3578079096 >> [v2:172.21.15.148:6834/3381818843,v1:172.21.15.148:6836/3381818843] conn(0x55713da45e70 0x55713d9a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.190+0000 7f1215400640 1 --2- 172.21.15.39:0/3578079096 >> [v2:172.21.15.148:6834/3381818843,v1:172.21.15.148:6836/3381818843] conn(0x55713da45e70 0x55713d9a0a20 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffdb95e6533220b6 server_cookie=9f2d8a85510ec9e9 in_seq=0 out_seq=0 2024-09-21T07:51:08.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.190+0000 7f11f7400640 10 client.11253 ms_handle_connect on v2:172.21.15.148:6834/3381818843 2024-09-21T07:51:08.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.190+0000 7f11f7400640 1 -- 172.21.15.39:0/3578079096 <== mds.0 v2:172.21.15.148:6834/3381818843 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x55713d8f2fd0 con 0x55713da45e70 2024-09-21T07:51:08.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.190+0000 7f11f7400640 10 client.11253 handle_command_reply: tid=0 2024-09-21T07:51:08.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.190+0000 7f1224208740 1 client.11253 shutdown 2024-09-21T07:51:08.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.190+0000 7f1224208740 1 -- 172.21.15.39:0/3578079096 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55713d5b23b0 msgr2=0x55713da42cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:08.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.190+0000 7f1224208740 1 --2- 172.21.15.39:0/3578079096 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55713d5b23b0 0x55713da42cd0 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7f120800efd0 tx=0x7f120800ed80 comp rx=0 tx=0).stop 2024-09-21T07:51:08.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.190+0000 7f1224208740 1 -- 172.21.15.39:0/3578079096 shutdown_connections 2024-09-21T07:51:08.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.190+0000 7f1224208740 1 --2- 172.21.15.39:0/3578079096 >> [v2:172.21.15.148:6834/3381818843,v1:172.21.15.148:6836/3381818843] conn(0x55713da45e70 0x55713d9a0a20 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.190+0000 7f1224208740 1 --2- 172.21.15.39:0/3578079096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55713d5db270 0x55713d9c8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.190+0000 7f1224208740 1 --2- 172.21.15.39:0/3578079096 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55713d5ba590 0x55713d9c87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.190+0000 7f1224208740 1 --2- 172.21.15.39:0/3578079096 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55713d5b23b0 0x55713da42cd0 unknown :-1 s=CLOSED pgs=2417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.190+0000 7f1224208740 1 -- 172.21.15.39:0/3578079096 >> 172.21.15.39:0/3578079096 conn(0x55713d5cf3c0 msgr2=0x55713d981b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:08.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.190+0000 7f11f7400640 0 client.11253 ms_handle_reset on v2:172.21.15.148:6834/3381818843 2024-09-21T07:51:08.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.190+0000 7f1224208740 1 -- 172.21.15.39:0/3578079096 shutdown_connections 2024-09-21T07:51:08.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.190+0000 7f1224208740 1 -- 172.21.15.39:0/3578079096 wait complete. 2024-09-21T07:51:08.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.191+0000 7f1224208740 20 client.11253 trim_cache size 0 max 16384 2024-09-21T07:51:08.195 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:51:08.195 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:51:08.195 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-09-21T07:51:08.196 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 9967, 2024-09-21T07:51:08.196 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:51:08.196 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:51:08.196 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:51:08.196 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 36.354852592, 2024-09-21T07:51:08.196 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 273, 2024-09-21T07:51:08.196 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 241, 2024-09-21T07:51:08.196 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 241, 2024-09-21T07:51:08.196 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 43.203418976999998 2024-09-21T07:51:08.196 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:51:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.191+0000 7f121d200640 1 -- 172.21.15.39:0/1845416511 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f11f8078b90 msgr2=0x7f11f807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.191+0000 7f121d200640 1 --2- 172.21.15.39:0/1845416511 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f11f8078b90 0x7f11f807b0a0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f12180730e0 tx=0x7f1208005f90 comp rx=0 tx=0).stop 2024-09-21T07:51:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.191+0000 7f121d200640 1 -- 172.21.15.39:0/1845416511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12180d0630 msgr2=0x7f12180b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.191+0000 7f121d200640 1 --2- 172.21.15.39:0/1845416511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12180d0630 0x7f12180b4d30 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7f120400c340 tx=0x7f120400c800 comp rx=0 tx=0).stop 2024-09-21T07:51:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.191+0000 7f121d200640 1 -- 172.21.15.39:0/1845416511 shutdown_connections 2024-09-21T07:51:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.191+0000 7f121d200640 1 --2- 172.21.15.39:0/1845416511 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f11f8078b90 0x7f11f807b0a0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.191+0000 7f121d200640 1 --2- 172.21.15.39:0/1845416511 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f121815daf0 0x7f121815b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.191+0000 7f121d200640 1 --2- 172.21.15.39:0/1845416511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12180d0630 0x7f12180b4d30 unknown :-1 s=CLOSED pgs=2413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.191+0000 7f121d200640 1 --2- 172.21.15.39:0/1845416511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1218073dd0 0x7f121814ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.191+0000 7f121d200640 1 -- 172.21.15.39:0/1845416511 >> 172.21.15.39:0/1845416511 conn(0x7f121806cfc0 msgr2=0x7f12180b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.191+0000 7f121d200640 1 -- 172.21.15.39:0/1845416511 shutdown_connections 2024-09-21T07:51:08.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.191+0000 7f121d200640 1 -- 172.21.15.39:0/1845416511 wait complete. 2024-09-21T07:51:08.218 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T07:51:08.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.443+0000 7fc49b200640 1 -- 172.21.15.39:0/3647536326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc49c0b96a0 msgr2=0x7fc49c0b59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:08.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.443+0000 7fc49b200640 1 --2- 172.21.15.39:0/3647536326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc49c0b96a0 0x7fc49c0b59e0 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7fc484005fd0 tx=0x7fc48402a5c0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.443+0000 7fc49b200640 1 -- 172.21.15.39:0/3647536326 shutdown_connections 2024-09-21T07:51:08.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.443+0000 7fc49b200640 1 --2- 172.21.15.39:0/3647536326 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc49c0d0e60 0x7fc49c0d1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.443+0000 7fc49b200640 1 --2- 172.21.15.39:0/3647536326 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc49c0b96a0 0x7fc49c0b59e0 unknown :-1 s=CLOSED pgs=2418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.443+0000 7fc49b200640 1 -- 172.21.15.39:0/3647536326 >> 172.21.15.39:0/3647536326 conn(0x7fc49c09d7d0 msgr2=0x7fc49c0d4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:08.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.443+0000 7fc49b200640 1 -- 172.21.15.39:0/3647536326 shutdown_connections 2024-09-21T07:51:08.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.443+0000 7fc49b200640 1 -- 172.21.15.39:0/3647536326 wait complete. 2024-09-21T07:51:08.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.445+0000 7fc49b200640 1 Processor -- start 2024-09-21T07:51:08.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.445+0000 7fc49b200640 1 -- start start 2024-09-21T07:51:08.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.447+0000 7fc49b200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc49c0b96a0 0x7fc49c14d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.447+0000 7fc49b200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc49c0d0e60 0x7fc49c14fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.447+0000 7fc49b200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc49c0d5060 0x7fc49c150120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.447+0000 7fc49b200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c0d2a00 con 0x7fc49c0d5060 2024-09-21T07:51:08.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.447+0000 7fc49b200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c0d2880 con 0x7fc49c0d0e60 2024-09-21T07:51:08.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.447+0000 7fc49b200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc49c0d2b80 con 0x7fc49c0b96a0 2024-09-21T07:51:08.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.447+0000 7fc49a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc49c0d5060 0x7fc49c150120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.447+0000 7fc49a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc49c0d5060 0x7fc49c150120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44436/0 (socket says 172.21.15.39:44436) 2024-09-21T07:51:08.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.447+0000 7fc49a800640 1 -- 172.21.15.39:0/3871690682 learned_addr learned my addr 172.21.15.39:0/3871690682 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:51:08.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.447+0000 7fc491400640 1 --2- 172.21.15.39:0/3871690682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc49c0d0e60 0x7fc49c14fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.447+0000 7fc499e00640 1 --2- 172.21.15.39:0/3871690682 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc49c0b96a0 0x7fc49c14d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.448+0000 7fc49a800640 1 -- 172.21.15.39:0/3871690682 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc49c0b96a0 msgr2=0x7fc49c14d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:08.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.448+0000 7fc49a800640 1 --2- 172.21.15.39:0/3871690682 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc49c0b96a0 0x7fc49c14d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.448+0000 7fc49a800640 1 -- 172.21.15.39:0/3871690682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc49c0d0e60 msgr2=0x7fc49c14fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:08.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.448+0000 7fc49a800640 1 --2- 172.21.15.39:0/3871690682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc49c0d0e60 0x7fc49c14fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.448+0000 7fc49a800640 1 -- 172.21.15.39:0/3871690682 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc49c150980 con 0x7fc49c0d5060 2024-09-21T07:51:08.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.448+0000 7fc49a800640 1 --2- 172.21.15.39:0/3871690682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc49c0d5060 0x7fc49c150120 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7fc49400ea70 tx=0x7fc49400ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:08.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.448+0000 7fc492a00640 1 -- 172.21.15.39:0/3871690682 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49400ce40 con 0x7fc49c0d5060 2024-09-21T07:51:08.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.448+0000 7fc492a00640 1 -- 172.21.15.39:0/3871690682 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc494004c10 con 0x7fc49c0d5060 2024-09-21T07:51:08.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.448+0000 7fc49b200640 1 -- 172.21.15.39:0/3871690682 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc49c14dd60 con 0x7fc49c0d5060 2024-09-21T07:51:08.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.448+0000 7fc492a00640 1 -- 172.21.15.39:0/3871690682 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc494005020 con 0x7fc49c0d5060 2024-09-21T07:51:08.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.448+0000 7fc49b200640 1 -- 172.21.15.39:0/3871690682 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc49c14e320 con 0x7fc49c0d5060 2024-09-21T07:51:08.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.451+0000 7fc492a00640 1 -- 172.21.15.39:0/3871690682 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc4940051c0 con 0x7fc49c0d5060 2024-09-21T07:51:08.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.452+0000 7fc492a00640 1 --2- 172.21.15.39:0/3871690682 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc478078bc0 0x7fc47807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.452+0000 7fc492a00640 1 -- 172.21.15.39:0/3871690682 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(241..241 src has 1..241) ==== 7476+0+0 (secure 0 0 0) 0x7fc49409c180 con 0x7fc49c0d5060 2024-09-21T07:51:08.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.452+0000 7fc4a7bcd740 1 Processor -- start 2024-09-21T07:51:08.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.452+0000 7fc499e00640 1 --2- 172.21.15.39:0/3871690682 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc478078bc0 0x7fc47807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.452+0000 7fc4a7bcd740 1 -- start start 2024-09-21T07:51:08.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.453+0000 7fc499e00640 1 --2- 172.21.15.39:0/3871690682 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc478078bc0 0x7fc47807b0d0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fc48c007fd0 tx=0x7fc48c009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:08.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.454+0000 7fc4a7bcd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559802e7a270 0x55980324ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.454+0000 7fc4a7bcd740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559802e513b0 0x55980324b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.454+0000 7fc4a7bcd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559802e59590 0x55980321b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.454+0000 7fc4a7bcd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5598031a4430 con 0x559802e7a270 2024-09-21T07:51:08.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.454+0000 7fc4a7bcd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5598031945d0 con 0x559802e59590 2024-09-21T07:51:08.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.454+0000 7fc4a7bcd740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x559803183a00 con 0x559802e513b0 2024-09-21T07:51:08.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.454+0000 7fc499e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559802e7a270 0x55980324ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.454+0000 7fc499e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559802e7a270 0x55980324ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44440/0 (socket says 172.21.15.39:44440) 2024-09-21T07:51:08.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.454+0000 7fc499e00640 1 -- 172.21.15.39:0/1366911961 learned_addr learned my addr 172.21.15.39:0/1366911961 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:51:08.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.454+0000 7fc491400640 1 --2- 172.21.15.39:0/1366911961 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559802e513b0 0x55980324b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.454+0000 7fc49a800640 1 --2- 172.21.15.39:0/1366911961 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559802e59590 0x55980321b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.454+0000 7fc499e00640 1 -- 172.21.15.39:0/1366911961 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559802e513b0 msgr2=0x55980324b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:51:08.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.454+0000 7fc499e00640 1 --2- 172.21.15.39:0/1366911961 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559802e513b0 0x55980324b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.454+0000 7fc499e00640 1 -- 172.21.15.39:0/1366911961 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559802e59590 msgr2=0x55980321b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:51:08.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.454+0000 7fc499e00640 1 --2- 172.21.15.39:0/1366911961 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559802e59590 0x55980321b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.454+0000 7fc499e00640 1 -- 172.21.15.39:0/1366911961 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55980314c520 con 0x559802e7a270 2024-09-21T07:51:08.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.455+0000 7fc499e00640 1 --2- 172.21.15.39:0/1366911961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559802e7a270 0x55980324ab60 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7fc48c00f960 tx=0x7fc48c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a30ae42db792ab server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:08.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.455+0000 7fc48a600640 1 -- 172.21.15.39:0/1366911961 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48c025070 con 0x559802e7a270 2024-09-21T07:51:08.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.455+0000 7fc48a600640 1 -- 172.21.15.39:0/1366911961 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc48c01b050 con 0x559802e7a270 2024-09-21T07:51:08.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.455+0000 7fc48a600640 1 -- 172.21.15.39:0/1366911961 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48c014e40 con 0x559802e7a270 2024-09-21T07:51:08.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.456+0000 7fc4a7bcd740 1 -- 172.21.15.39:0/1366911961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559802e7a270 msgr2=0x55980324ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:08.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.456+0000 7fc4a7bcd740 1 --2- 172.21.15.39:0/1366911961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559802e7a270 0x55980324ab60 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7fc48c00f960 tx=0x7fc48c012000 comp rx=0 tx=0).stop 2024-09-21T07:51:08.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.456+0000 7fc4a7bcd740 1 -- 172.21.15.39:0/1366911961 shutdown_connections 2024-09-21T07:51:08.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.456+0000 7fc4a7bcd740 1 --2- 172.21.15.39:0/1366911961 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559802e59590 0x55980321b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.456+0000 7fc4a7bcd740 1 --2- 172.21.15.39:0/1366911961 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559802e513b0 0x55980324b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.456+0000 7fc4a7bcd740 1 --2- 172.21.15.39:0/1366911961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559802e7a270 0x55980324ab60 unknown :-1 s=CLOSED pgs=2420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.456+0000 7fc4a7bcd740 1 -- 172.21.15.39:0/1366911961 >> 172.21.15.39:0/1366911961 conn(0x559802e6e3c0 msgr2=0x559803220b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:08.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.456+0000 7fc4a7bcd740 1 -- 172.21.15.39:0/1366911961 shutdown_connections 2024-09-21T07:51:08.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.456+0000 7fc4a7bcd740 1 -- 172.21.15.39:0/1366911961 wait complete. 2024-09-21T07:51:08.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.457+0000 7fc4a7bcd740 1 -- start start 2024-09-21T07:51:08.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.457+0000 7fc4a7bcd740 1 Processor -- start 2024-09-21T07:51:08.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.459+0000 7fc4a7bcd740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559802e513b0 0x559803247af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.459+0000 7fc4a7bcd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559802e59590 0x55980325e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.459+0000 7fc4a7bcd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559802e7a270 0x559803242d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.459+0000 7fc4a7bcd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55980312a330 con 0x559802e59590 2024-09-21T07:51:08.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.459+0000 7fc4a7bcd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559803187810 con 0x559802e7a270 2024-09-21T07:51:08.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.459+0000 7fc4a7bcd740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x559802ef4620 con 0x559802e513b0 2024-09-21T07:51:08.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.459+0000 7fc4a7bcd740 10 client.0 _ll_register_callbacks cb 0x55980312f420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:51:08.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.459+0000 7fc491400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559802e59590 0x55980325e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.459+0000 7fc491400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559802e59590 0x55980325e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44444/0 (socket says 172.21.15.39:44444) 2024-09-21T07:51:08.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.459+0000 7fc491400640 1 -- 172.21.15.39:0/938093287 learned_addr learned my addr 172.21.15.39:0/938093287 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:51:08.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.459+0000 7fc499e00640 1 --2- 172.21.15.39:0/938093287 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559802e513b0 0x559803247af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.459+0000 7fc49a800640 1 --2- 172.21.15.39:0/938093287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559802e7a270 0x559803242d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.459+0000 7fc491400640 1 -- 172.21.15.39:0/938093287 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559802e513b0 msgr2=0x559803247af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:51:08.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.459+0000 7fc491400640 1 --2- 172.21.15.39:0/938093287 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559802e513b0 0x559803247af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.459+0000 7fc491400640 1 -- 172.21.15.39:0/938093287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559802e7a270 msgr2=0x559803242d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:51:08.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.459+0000 7fc491400640 1 --2- 172.21.15.39:0/938093287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559802e7a270 0x559803242d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.459+0000 7fc491400640 1 -- 172.21.15.39:0/938093287 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559802ee5850 con 0x559802e59590 2024-09-21T07:51:08.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.459+0000 7fc49a800640 1 --2- 172.21.15.39:0/938093287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559802e7a270 0x559803242d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:51:08.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.460+0000 7fc491400640 1 --2- 172.21.15.39:0/938093287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559802e59590 0x55980325e1a0 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7fc484006010 tx=0x7fc484005680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d6bc6ec27f3ec4a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:08.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.460+0000 7fc47ea00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:51:08.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.460+0000 7fc47ea00640 1 -- 172.21.15.39:0/938093287 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc484035040 con 0x559802e59590 2024-09-21T07:51:08.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.460+0000 7fc47ea00640 1 -- 172.21.15.39:0/938093287 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(241..241 src has 1..241) ==== 7476+0+0 (secure 0 0 0) 0x7fc4840035f0 con 0x559802e59590 2024-09-21T07:51:08.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.460+0000 7fc4a7bcd740 1 -- 172.21.15.39:0/938093287 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc48c01b050 con 0x559802e59590 2024-09-21T07:51:08.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.460+0000 7fc47ea00640 1 -- 172.21.15.39:0/938093287 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=273) ==== 24+0+0 (secure 0 0 0) 0x7fc4840044d0 con 0x559802e59590 2024-09-21T07:51:08.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.460+0000 7fc4a7bcd740 10 client.11265 fetch_fsmap learned FSMap version 273 2024-09-21T07:51:08.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.460+0000 7fc4a7bcd740 1 -- 172.21.15.39:0/938093287 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=273,monmap=2+}) -- 0x559802f5c6b0 con 0x559802e59590 2024-09-21T07:51:08.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.461+0000 7fc47ea00640 1 -- 172.21.15.39:0/938093287 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc48c01b050 con 0x559802e59590 2024-09-21T07:51:08.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.461+0000 7fc47ea00640 1 -- 172.21.15.39:0/938093287 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 273) ==== 3542+0+0 (secure 0 0 0) 0x7fc4840038e0 con 0x559802e59590 2024-09-21T07:51:08.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.461+0000 7fc4a7bcd740 10 client.11265 fetch_fsmap finished waiting for FSMap version 273 2024-09-21T07:51:08.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.461+0000 7fc4a7bcd740 10 client.11265 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:51:08.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.461+0000 7fc4a7bcd740 20 client.11265 populate_metadata read hostname 'smithi039' 2024-09-21T07:51:08.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.461+0000 7fc4a7bcd740 1 --2- 172.21.15.39:0/938093287 >> [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] conn(0x559802e38e40 0x5598032757d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.461+0000 7fc4a7bcd740 4 client.11265 mds_command: new command op to 11052 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:51:08.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.461+0000 7fc4a7bcd740 1 -- 172.21.15.39:0/938093287 --> [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559803196520 con 0x559802e38e40 2024-09-21T07:51:08.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.461+0000 7fc499e00640 1 --2- 172.21.15.39:0/938093287 >> [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] conn(0x559802e38e40 0x5598032757d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.462+0000 7fc499e00640 1 --2- 172.21.15.39:0/938093287 >> [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] conn(0x559802e38e40 0x5598032757d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a3bbd5c872b5e63 server_cookie=ba029a7a19d022e7 in_seq=0 out_seq=0 2024-09-21T07:51:08.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.462+0000 7fc47ea00640 10 client.11265 ms_handle_connect on v2:172.21.15.39:6833/485194245 2024-09-21T07:51:08.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.462+0000 7fc47ea00640 1 -- 172.21.15.39:0/938093287 <== mds.1 v2:172.21.15.39:6833/485194245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559803196520 con 0x559802e38e40 2024-09-21T07:51:08.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.462+0000 7fc47ea00640 10 client.11265 handle_command_reply: tid=0 2024-09-21T07:51:08.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.462+0000 7fc4a7bcd740 1 client.11265 shutdown 2024-09-21T07:51:08.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.463+0000 7fc4a7bcd740 1 -- 172.21.15.39:0/938093287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559802e59590 msgr2=0x55980325e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:08.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.463+0000 7fc4a7bcd740 1 --2- 172.21.15.39:0/938093287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559802e59590 0x55980325e1a0 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7fc484006010 tx=0x7fc484005680 comp rx=0 tx=0).stop 2024-09-21T07:51:08.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.463+0000 7fc4a7bcd740 1 -- 172.21.15.39:0/938093287 shutdown_connections 2024-09-21T07:51:08.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.463+0000 7fc4a7bcd740 1 --2- 172.21.15.39:0/938093287 >> [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] conn(0x559802e38e40 0x5598032757d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.463+0000 7fc4a7bcd740 1 --2- 172.21.15.39:0/938093287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559802e7a270 0x559803242d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.463+0000 7fc4a7bcd740 1 --2- 172.21.15.39:0/938093287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559802e59590 0x55980325e1a0 unknown :-1 s=CLOSED pgs=2421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.463+0000 7fc4a7bcd740 1 --2- 172.21.15.39:0/938093287 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559802e513b0 0x559803247af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.463+0000 7fc4a7bcd740 1 -- 172.21.15.39:0/938093287 >> 172.21.15.39:0/938093287 conn(0x559802e6e3c0 msgr2=0x55980324ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:08.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.463+0000 7fc47ea00640 0 client.11265 ms_handle_reset on v2:172.21.15.39:6833/485194245 2024-09-21T07:51:08.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.463+0000 7fc4a7bcd740 1 -- 172.21.15.39:0/938093287 shutdown_connections 2024-09-21T07:51:08.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.463+0000 7fc4a7bcd740 1 -- 172.21.15.39:0/938093287 wait complete. 2024-09-21T07:51:08.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.463+0000 7fc4a7bcd740 20 client.11265 trim_cache size 0 max 16384 2024-09-21T07:51:08.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.482+0000 7fc4a7bcd740 1 Processor -- start 2024-09-21T07:51:08.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.482+0000 7fc4a7bcd740 1 -- start start 2024-09-21T07:51:08.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.484+0000 7fc4a7bcd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559802e6e3c0 0x559803277f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.484+0000 7fc4a7bcd740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559802e513b0 0x559803268eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.484+0000 7fc4a7bcd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559802e59590 0x5598032693f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.484+0000 7fc4a7bcd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c204380 con 0x559802e6e3c0 2024-09-21T07:51:08.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.484+0000 7fc4a7bcd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c204a50 con 0x559802e59590 2024-09-21T07:51:08.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.484+0000 7fc4a7bcd740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc49c2033d0 con 0x559802e513b0 2024-09-21T07:51:08.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.484+0000 7fc499e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559802e6e3c0 0x559803277f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.484+0000 7fc499e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559802e6e3c0 0x559803277f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44456/0 (socket says 172.21.15.39:44456) 2024-09-21T07:51:08.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.484+0000 7fc499e00640 1 -- 172.21.15.39:0/2374723516 learned_addr learned my addr 172.21.15.39:0/2374723516 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:51:08.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.484+0000 7fc491400640 1 --2- 172.21.15.39:0/2374723516 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559802e513b0 0x559803268eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.484+0000 7fc49a800640 1 --2- 172.21.15.39:0/2374723516 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559802e59590 0x5598032693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.484+0000 7fc499e00640 1 -- 172.21.15.39:0/2374723516 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559802e513b0 msgr2=0x559803268eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:51:08.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.484+0000 7fc499e00640 1 --2- 172.21.15.39:0/2374723516 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559802e513b0 0x559803268eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.484+0000 7fc499e00640 1 -- 172.21.15.39:0/2374723516 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559802e59590 msgr2=0x5598032693f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:51:08.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.484+0000 7fc499e00640 1 --2- 172.21.15.39:0/2374723516 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559802e59590 0x5598032693f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.484+0000 7fc499e00640 1 -- 172.21.15.39:0/2374723516 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559803246c50 con 0x559802e6e3c0 2024-09-21T07:51:08.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.484+0000 7fc49a800640 1 --2- 172.21.15.39:0/2374723516 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559802e59590 0x5598032693f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:51:08.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.485+0000 7fc499e00640 1 --2- 172.21.15.39:0/2374723516 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559802e6e3c0 0x559803277f20 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7fc48c0078a0 tx=0x7fc48c007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc58c0595c1834c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:08.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.485+0000 7fc489c00640 1 -- 172.21.15.39:0/2374723516 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48c039070 con 0x559802e6e3c0 2024-09-21T07:51:08.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.485+0000 7fc489c00640 1 -- 172.21.15.39:0/2374723516 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc48c031030 con 0x559802e6e3c0 2024-09-21T07:51:08.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.485+0000 7fc489c00640 1 -- 172.21.15.39:0/2374723516 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48c025040 con 0x559802e6e3c0 2024-09-21T07:51:08.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.486+0000 7fc4a7bcd740 1 -- 172.21.15.39:0/2374723516 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559802e6e3c0 msgr2=0x559803277f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:08.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.486+0000 7fc4a7bcd740 1 --2- 172.21.15.39:0/2374723516 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559802e6e3c0 0x559803277f20 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7fc48c0078a0 tx=0x7fc48c007ea0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.486+0000 7fc4a7bcd740 1 -- 172.21.15.39:0/2374723516 shutdown_connections 2024-09-21T07:51:08.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.486+0000 7fc4a7bcd740 1 --2- 172.21.15.39:0/2374723516 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559802e59590 0x5598032693f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.486+0000 7fc4a7bcd740 1 --2- 172.21.15.39:0/2374723516 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559802e513b0 0x559803268eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.486+0000 7fc4a7bcd740 1 --2- 172.21.15.39:0/2374723516 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559802e6e3c0 0x559803277f20 unknown :-1 s=CLOSED pgs=2422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.486+0000 7fc4a7bcd740 1 -- 172.21.15.39:0/2374723516 >> 172.21.15.39:0/2374723516 conn(0x559803247c20 msgr2=0x5598032481d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:08.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.486+0000 7fc4a7bcd740 1 -- 172.21.15.39:0/2374723516 shutdown_connections 2024-09-21T07:51:08.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.486+0000 7fc4a7bcd740 1 -- 172.21.15.39:0/2374723516 wait complete. 2024-09-21T07:51:08.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.487+0000 7fc4a7bcd740 1 -- start start 2024-09-21T07:51:08.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.487+0000 7fc4a7bcd740 1 Processor -- start 2024-09-21T07:51:08.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.488+0000 7fc4a7bcd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559802e513b0 0x5598032740f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.488+0000 7fc4a7bcd740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559802e59590 0x559803274630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.488+0000 7fc4a7bcd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559802e6e3c0 0x5598032e9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.488+0000 7fc4a7bcd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5598032473b0 con 0x559802e513b0 2024-09-21T07:51:08.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.488+0000 7fc4a7bcd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559803235930 con 0x559802e6e3c0 2024-09-21T07:51:08.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.488+0000 7fc4a7bcd740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x559803235f00 con 0x559802e59590 2024-09-21T07:51:08.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.488+0000 7fc4a7bcd740 10 client.0 _ll_register_callbacks cb 0x7fc484005310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:51:08.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.489+0000 7fc499e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559802e513b0 0x5598032740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.489+0000 7fc499e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559802e513b0 0x5598032740f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44458/0 (socket says 172.21.15.39:44458) 2024-09-21T07:51:08.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.489+0000 7fc499e00640 1 -- 172.21.15.39:0/2500715027 learned_addr learned my addr 172.21.15.39:0/2500715027 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:51:08.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.489+0000 7fc499e00640 1 -- 172.21.15.39:0/2500715027 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559802e59590 msgr2=0x559803274630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:51:08.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.489+0000 7fc499e00640 1 --2- 172.21.15.39:0/2500715027 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559802e59590 0x559803274630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.489+0000 7fc499e00640 1 -- 172.21.15.39:0/2500715027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559802e6e3c0 msgr2=0x5598032e9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:51:08.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.489+0000 7fc499e00640 1 --2- 172.21.15.39:0/2500715027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559802e6e3c0 0x5598032e9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.489+0000 7fc499e00640 1 -- 172.21.15.39:0/2500715027 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55980325fb70 con 0x559802e513b0 2024-09-21T07:51:08.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.489+0000 7fc499e00640 1 --2- 172.21.15.39:0/2500715027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559802e513b0 0x5598032740f0 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7fc48c037040 tx=0x7fc48c0148d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e946832483a644e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:08.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.489+0000 7fc47ea00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:51:08.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.489+0000 7fc47ea00640 1 -- 172.21.15.39:0/2500715027 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48c039070 con 0x559802e513b0 2024-09-21T07:51:08.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.490+0000 7fc4a7bcd740 1 -- 172.21.15.39:0/2500715027 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc48c031030 con 0x559802e513b0 2024-09-21T07:51:08.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.490+0000 7fc47ea00640 1 -- 172.21.15.39:0/2500715027 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(241..241 src has 1..241) ==== 7476+0+0 (secure 0 0 0) 0x7fc48c020b40 con 0x559802e513b0 2024-09-21T07:51:08.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.490+0000 7fc47ea00640 1 -- 172.21.15.39:0/2500715027 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=273) ==== 24+0+0 (secure 0 0 0) 0x7fc48c020d40 con 0x559802e513b0 2024-09-21T07:51:08.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.490+0000 7fc4a7bcd740 10 client.11271 fetch_fsmap learned FSMap version 273 2024-09-21T07:51:08.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.490+0000 7fc4a7bcd740 1 -- 172.21.15.39:0/2500715027 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=273,monmap=2+}) -- 0x5598032778e0 con 0x559802e513b0 2024-09-21T07:51:08.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.490+0000 7fc47ea00640 1 -- 172.21.15.39:0/2500715027 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc48c031030 con 0x559802e513b0 2024-09-21T07:51:08.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.490+0000 7fc47ea00640 1 -- 172.21.15.39:0/2500715027 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 273) ==== 3542+0+0 (secure 0 0 0) 0x7fc48c03e440 con 0x559802e513b0 2024-09-21T07:51:08.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.490+0000 7fc4a7bcd740 10 client.11271 fetch_fsmap finished waiting for FSMap version 273 2024-09-21T07:51:08.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.490+0000 7fc4a7bcd740 10 client.11271 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:51:08.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.490+0000 7fc4a7bcd740 20 client.11271 populate_metadata read hostname 'smithi039' 2024-09-21T07:51:08.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.490+0000 7fc4a7bcd740 1 --2- 172.21.15.39:0/2500715027 >> [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] conn(0x559802e0e280 0x5598032e5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.490+0000 7fc4a7bcd740 4 client.11271 mds_command: new command op to 11052 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:51:08.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.490+0000 7fc4a7bcd740 1 -- 172.21.15.39:0/2500715027 --> [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] -- command(tid 0: {"prefix": "status"}) -- 0x559803191fd0 con 0x559802e0e280 2024-09-21T07:51:08.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.491+0000 7fc491400640 1 --2- 172.21.15.39:0/2500715027 >> [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] conn(0x559802e0e280 0x5598032e5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.491+0000 7fc491400640 1 --2- 172.21.15.39:0/2500715027 >> [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] conn(0x559802e0e280 0x5598032e5a30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8518419762958eb6 server_cookie=98fc51fd56e6d43f in_seq=0 out_seq=0 2024-09-21T07:51:08.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.491+0000 7fc47ea00640 10 client.11271 ms_handle_connect on v2:172.21.15.39:6833/485194245 2024-09-21T07:51:08.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.491+0000 7fc47ea00640 1 -- 172.21.15.39:0/2500715027 <== mds.1 v2:172.21.15.39:6833/485194245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+335 (crc 0 0 0) 0x559803191fd0 con 0x559802e0e280 2024-09-21T07:51:08.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.491+0000 7fc47ea00640 10 client.11271 handle_command_reply: tid=0 2024-09-21T07:51:08.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.491+0000 7fc4a7bcd740 1 client.11271 shutdown 2024-09-21T07:51:08.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.491+0000 7fc4a7bcd740 1 -- 172.21.15.39:0/2500715027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559802e513b0 msgr2=0x5598032740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:08.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.491+0000 7fc4a7bcd740 1 --2- 172.21.15.39:0/2500715027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559802e513b0 0x5598032740f0 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7fc48c037040 tx=0x7fc48c0148d0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.492+0000 7fc4a7bcd740 1 -- 172.21.15.39:0/2500715027 shutdown_connections 2024-09-21T07:51:08.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.492+0000 7fc4a7bcd740 1 --2- 172.21.15.39:0/2500715027 >> [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] conn(0x559802e0e280 0x5598032e5a30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.492+0000 7fc4a7bcd740 1 --2- 172.21.15.39:0/2500715027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559802e6e3c0 0x5598032e9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.492+0000 7fc4a7bcd740 1 --2- 172.21.15.39:0/2500715027 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x559802e59590 0x559803274630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.492+0000 7fc4a7bcd740 1 --2- 172.21.15.39:0/2500715027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559802e513b0 0x5598032740f0 unknown :-1 s=CLOSED pgs=2423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.492+0000 7fc47ea00640 0 client.11271 ms_handle_reset on v2:172.21.15.39:6833/485194245 2024-09-21T07:51:08.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.492+0000 7fc4a7bcd740 1 -- 172.21.15.39:0/2500715027 >> 172.21.15.39:0/2500715027 conn(0x559803247c20 msgr2=0x559803278020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:08.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.492+0000 7fc4a7bcd740 1 -- 172.21.15.39:0/2500715027 shutdown_connections 2024-09-21T07:51:08.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.492+0000 7fc4a7bcd740 1 -- 172.21.15.39:0/2500715027 wait complete. 2024-09-21T07:51:08.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.492+0000 7fc4a7bcd740 20 client.11271 trim_cache size 0 max 16384 2024-09-21T07:51:08.496 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:51:08.497 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:51:08.497 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 1, 2024-09-21T07:51:08.497 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 11052, 2024-09-21T07:51:08.497 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:51:08.497 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:51:08.497 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:51:08.497 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 36.106658705999997, 2024-09-21T07:51:08.497 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 273, 2024-09-21T07:51:08.497 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 241, 2024-09-21T07:51:08.497 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 241, 2024-09-21T07:51:08.497 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 36.518927183999999 2024-09-21T07:51:08.497 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:51:08.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.492+0000 7fc49b200640 1 -- 172.21.15.39:0/3871690682 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc478078bc0 msgr2=0x7fc47807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:08.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.492+0000 7fc49b200640 1 --2- 172.21.15.39:0/3871690682 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc478078bc0 0x7fc47807b0d0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fc48c007fd0 tx=0x7fc48c009040 comp rx=0 tx=0).stop 2024-09-21T07:51:08.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.493+0000 7fc49b200640 1 -- 172.21.15.39:0/3871690682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc49c0d5060 msgr2=0x7fc49c150120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:08.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.493+0000 7fc49b200640 1 --2- 172.21.15.39:0/3871690682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc49c0d5060 0x7fc49c150120 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7fc49400ea70 tx=0x7fc49400ef30 comp rx=0 tx=0).stop 2024-09-21T07:51:08.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.493+0000 7fc49b200640 1 -- 172.21.15.39:0/3871690682 shutdown_connections 2024-09-21T07:51:08.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.493+0000 7fc49b200640 1 --2- 172.21.15.39:0/3871690682 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc478078bc0 0x7fc47807b0d0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.493+0000 7fc49b200640 1 --2- 172.21.15.39:0/3871690682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc49c0d5060 0x7fc49c150120 unknown :-1 s=CLOSED pgs=2419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.493+0000 7fc49b200640 1 --2- 172.21.15.39:0/3871690682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc49c0d0e60 0x7fc49c14fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.493+0000 7fc49b200640 1 --2- 172.21.15.39:0/3871690682 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc49c0b96a0 0x7fc49c14d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.493+0000 7fc49b200640 1 -- 172.21.15.39:0/3871690682 >> 172.21.15.39:0/3871690682 conn(0x7fc49c09d7d0 msgr2=0x7fc49c0b1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:08.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.493+0000 7fc49b200640 1 -- 172.21.15.39:0/3871690682 shutdown_connections 2024-09-21T07:51:08.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.493+0000 7fc49b200640 1 -- 172.21.15.39:0/3871690682 wait complete. 2024-09-21T07:51:08.518 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T07:51:08.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.745+0000 7f11ff200640 1 -- 172.21.15.39:0/3215348948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12000cdb20 msgr2=0x7f12000d1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:08.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.745+0000 7f11ff200640 1 --2- 172.21.15.39:0/3215348948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12000cdb20 0x7f12000d1c10 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f1200073cb0 tx=0x7f11f002a4d0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.745+0000 7f11ff200640 1 -- 172.21.15.39:0/3215348948 shutdown_connections 2024-09-21T07:51:08.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.745+0000 7f11ff200640 1 --2- 172.21.15.39:0/3215348948 >> v2:172.21.15.148:3300/0 conn(0x7f120006d320 0x7f12001602d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.745+0000 7f11ff200640 1 --2- 172.21.15.39:0/3215348948 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f120015d760 0x7f120006cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.745+0000 7f11ff200640 1 --2- 172.21.15.39:0/3215348948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12000cdb20 0x7f12000d1c10 unknown :-1 s=CLOSED pgs=2424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.745+0000 7f11ff200640 1 -- 172.21.15.39:0/3215348948 >> 172.21.15.39:0/3215348948 conn(0x7f12000b6f80 msgr2=0x7f120006def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:08.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.745+0000 7f11ff200640 1 -- 172.21.15.39:0/3215348948 shutdown_connections 2024-09-21T07:51:08.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.745+0000 7f11ff200640 1 -- 172.21.15.39:0/3215348948 wait complete. 2024-09-21T07:51:08.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.747+0000 7f11ff200640 1 Processor -- start 2024-09-21T07:51:08.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.747+0000 7f11ff200640 1 -- start start 2024-09-21T07:51:08.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.749+0000 7f11ff200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f120006d320 0x7f12001f57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.749+0000 7f11ff200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12000cdb20 0x7f12000aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.749+0000 7f11ff200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f120015d760 0x7f12000ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.749+0000 7f11ff200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1200160990 con 0x7f120006d320 2024-09-21T07:51:08.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.749+0000 7f11ff200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1200160810 con 0x7f12000cdb20 2024-09-21T07:51:08.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.749+0000 7f11ff200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f1200160b10 con 0x7f120015d760 2024-09-21T07:51:08.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.749+0000 7f11fde00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f120006d320 0x7f12001f57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.749+0000 7f11fde00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f120006d320 0x7f12001f57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44472/0 (socket says 172.21.15.39:44472) 2024-09-21T07:51:08.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.749+0000 7f11fde00640 1 -- 172.21.15.39:0/1146249220 learned_addr learned my addr 172.21.15.39:0/1146249220 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:51:08.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.749+0000 7f11fe800640 1 --2- 172.21.15.39:0/1146249220 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f120015d760 0x7f12000ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.749+0000 7f11f5400640 1 --2- 172.21.15.39:0/1146249220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12000cdb20 0x7f12000aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.750+0000 7f11fde00640 1 -- 172.21.15.39:0/1146249220 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f120015d760 msgr2=0x7f12000ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:08.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.750+0000 7f11fde00640 1 --2- 172.21.15.39:0/1146249220 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f120015d760 0x7f12000ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.750+0000 7f11fde00640 1 -- 172.21.15.39:0/1146249220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12000cdb20 msgr2=0x7f12000aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:08.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.750+0000 7f11fde00640 1 --2- 172.21.15.39:0/1146249220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12000cdb20 0x7f12000aeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.750+0000 7f11fde00640 1 -- 172.21.15.39:0/1146249220 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12000b4280 con 0x7f120006d320 2024-09-21T07:51:08.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.750+0000 7f11fde00640 1 --2- 172.21.15.39:0/1146249220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f120006d320 0x7f12001f57b0 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f11f0034040 tx=0x7f11f00039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:08.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.750+0000 7f11f6a00640 1 -- 172.21.15.39:0/1146249220 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f003e030 con 0x7f120006d320 2024-09-21T07:51:08.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.750+0000 7f11f6a00640 1 -- 172.21.15.39:0/1146249220 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11f00047a0 con 0x7f120006d320 2024-09-21T07:51:08.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.750+0000 7f11ff200640 1 -- 172.21.15.39:0/1146249220 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12001f6120 con 0x7f120006d320 2024-09-21T07:51:08.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.750+0000 7f11f6a00640 1 -- 172.21.15.39:0/1146249220 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f0035040 con 0x7f120006d320 2024-09-21T07:51:08.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.751+0000 7f11ff200640 1 -- 172.21.15.39:0/1146249220 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1200159030 con 0x7f120006d320 2024-09-21T07:51:08.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.754+0000 7f11f6a00640 1 -- 172.21.15.39:0/1146249220 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f11f0041030 con 0x7f120006d320 2024-09-21T07:51:08.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.755+0000 7f11f6a00640 1 --2- 172.21.15.39:0/1146249220 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f11dc078bc0 0x7f11dc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.755+0000 7f11f6a00640 1 -- 172.21.15.39:0/1146249220 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(241..241 src has 1..241) ==== 7476+0+0 (secure 0 0 0) 0x7f11f00bbbc0 con 0x7f120006d320 2024-09-21T07:51:08.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.755+0000 7f120bcc3740 1 Processor -- start 2024-09-21T07:51:08.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.755+0000 7f120bcc3740 1 -- start start 2024-09-21T07:51:08.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.755+0000 7f11f5400640 1 --2- 172.21.15.39:0/1146249220 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f11dc078bc0 0x7f11dc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.756+0000 7f11f5400640 1 --2- 172.21.15.39:0/1146249220 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f11dc078bc0 0x7f11dc07b0d0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f12000b5e40 tx=0x7f11e8009060 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:08.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.757+0000 7f120bcc3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7768c1270 0x55d776c91b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.757+0000 7f120bcc3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d7768983b0 0x55d776c920a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.757+0000 7f120bcc3740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d7768a0590 0x55d776c62150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.757+0000 7f120bcc3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d776beb430 con 0x55d7768c1270 2024-09-21T07:51:08.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.757+0000 7f120bcc3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d776bdb5d0 con 0x55d7768983b0 2024-09-21T07:51:08.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.757+0000 7f120bcc3740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55d776bcaa00 con 0x55d7768a0590 2024-09-21T07:51:08.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.757+0000 7f11fde00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7768c1270 0x55d776c91b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.757+0000 7f11f5400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d7768983b0 0x55d776c920a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.757+0000 7f11fde00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7768c1270 0x55d776c91b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44482/0 (socket says 172.21.15.39:44482) 2024-09-21T07:51:08.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.757+0000 7f11fde00640 1 -- 172.21.15.39:0/4150983528 learned_addr learned my addr 172.21.15.39:0/4150983528 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:51:08.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.757+0000 7f11fe800640 1 --2- 172.21.15.39:0/4150983528 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d7768a0590 0x55d776c62150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.757+0000 7f11f5400640 1 -- 172.21.15.39:0/4150983528 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d7768a0590 msgr2=0x55d776c62150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:51:08.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.757+0000 7f11f5400640 1 --2- 172.21.15.39:0/4150983528 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d7768a0590 0x55d776c62150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.757+0000 7f11f5400640 1 -- 172.21.15.39:0/4150983528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7768c1270 msgr2=0x55d776c91b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:51:08.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.757+0000 7f11f5400640 1 --2- 172.21.15.39:0/4150983528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7768c1270 0x55d776c91b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.757+0000 7f11f5400640 1 -- 172.21.15.39:0/4150983528 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d776b93520 con 0x55d7768983b0 2024-09-21T07:51:08.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.757+0000 7f11fde00640 1 --2- 172.21.15.39:0/4150983528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7768c1270 0x55d776c91b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:51:08.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.758+0000 7f11f5400640 1 --2- 172.21.15.39:0/4150983528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d7768983b0 0x55d776c920a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f11e800e9b0 tx=0x7f11e8011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4168c9d38792db54 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:08.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.758+0000 7f11ee600640 1 -- 172.21.15.39:0/4150983528 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e801b040 con 0x55d7768983b0 2024-09-21T07:51:08.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.758+0000 7f11ee600640 1 -- 172.21.15.39:0/4150983528 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11e8024070 con 0x55d7768983b0 2024-09-21T07:51:08.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.758+0000 7f11ee600640 1 -- 172.21.15.39:0/4150983528 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e8015c60 con 0x55d7768983b0 2024-09-21T07:51:08.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.759+0000 7f120bcc3740 1 -- 172.21.15.39:0/4150983528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d7768983b0 msgr2=0x55d776c920a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:08.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.759+0000 7f120bcc3740 1 --2- 172.21.15.39:0/4150983528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d7768983b0 0x55d776c920a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f11e800e9b0 tx=0x7f11e8011000 comp rx=0 tx=0).stop 2024-09-21T07:51:08.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.759+0000 7f120bcc3740 1 -- 172.21.15.39:0/4150983528 shutdown_connections 2024-09-21T07:51:08.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.759+0000 7f120bcc3740 1 --2- 172.21.15.39:0/4150983528 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d7768a0590 0x55d776c62150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.759+0000 7f120bcc3740 1 --2- 172.21.15.39:0/4150983528 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d7768983b0 0x55d776c920a0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.759+0000 7f120bcc3740 1 --2- 172.21.15.39:0/4150983528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7768c1270 0x55d776c91b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.759+0000 7f120bcc3740 1 -- 172.21.15.39:0/4150983528 >> 172.21.15.39:0/4150983528 conn(0x55d7768b53c0 msgr2=0x55d776c67b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:08.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.759+0000 7f120bcc3740 1 -- 172.21.15.39:0/4150983528 shutdown_connections 2024-09-21T07:51:08.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.759+0000 7f120bcc3740 1 -- 172.21.15.39:0/4150983528 wait complete. 2024-09-21T07:51:08.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.760+0000 7f120bcc3740 1 -- start start 2024-09-21T07:51:08.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.760+0000 7f120bcc3740 1 Processor -- start 2024-09-21T07:51:08.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.762+0000 7f120bcc3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7768983b0 0x55d776c75f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.762+0000 7f120bcc3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d7768a0590 0x55d776ca8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.762+0000 7f120bcc3740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d7768c1270 0x55d776ca8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.762+0000 7f120bcc3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d776b71330 con 0x55d7768983b0 2024-09-21T07:51:08.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.762+0000 7f120bcc3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d776bce810 con 0x55d7768a0590 2024-09-21T07:51:08.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.762+0000 7f120bcc3740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55d77693b620 con 0x55d7768c1270 2024-09-21T07:51:08.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.762+0000 7f120bcc3740 10 client.0 _ll_register_callbacks cb 0x55d776b76420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:51:08.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.762+0000 7f11fde00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7768983b0 0x55d776c75f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.762+0000 7f11fde00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7768983b0 0x55d776c75f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44490/0 (socket says 172.21.15.39:44490) 2024-09-21T07:51:08.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.762+0000 7f11fde00640 1 -- 172.21.15.39:0/1790786119 learned_addr learned my addr 172.21.15.39:0/1790786119 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:51:08.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.762+0000 7f11f5400640 1 --2- 172.21.15.39:0/1790786119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d7768a0590 0x55d776ca8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.762+0000 7f11fe800640 1 --2- 172.21.15.39:0/1790786119 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d7768c1270 0x55d776ca8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.762+0000 7f11fde00640 1 -- 172.21.15.39:0/1790786119 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d7768c1270 msgr2=0x55d776ca8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:51:08.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.762+0000 7f11fde00640 1 --2- 172.21.15.39:0/1790786119 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d7768c1270 0x55d776ca8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.763+0000 7f11fde00640 1 -- 172.21.15.39:0/1790786119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d7768a0590 msgr2=0x55d776ca8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:51:08.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.763+0000 7f11fde00640 1 --2- 172.21.15.39:0/1790786119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d7768a0590 0x55d776ca8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.763+0000 7f11fde00640 1 -- 172.21.15.39:0/1790786119 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d77692c850 con 0x55d7768983b0 2024-09-21T07:51:08.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.763+0000 7f11fde00640 1 --2- 172.21.15.39:0/1790786119 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7768983b0 0x55d776c75f70 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f11f0004010 tx=0x7f11f0082d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fedb8b4503be6880 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:08.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.763+0000 7f11e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:51:08.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.763+0000 7f11e2a00640 1 -- 172.21.15.39:0/1790786119 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f00a9070 con 0x55d7768983b0 2024-09-21T07:51:08.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.763+0000 7f11e2a00640 1 -- 172.21.15.39:0/1790786119 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(241..241 src has 1..241) ==== 7476+0+0 (secure 0 0 0) 0x7f11f00837a0 con 0x55d7768983b0 2024-09-21T07:51:08.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.763+0000 7f120bcc3740 1 -- 172.21.15.39:0/1790786119 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11e8024070 con 0x55d7768983b0 2024-09-21T07:51:08.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.763+0000 7f11e2a00640 1 -- 172.21.15.39:0/1790786119 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=273) ==== 24+0+0 (secure 0 0 0) 0x7f11f0048040 con 0x55d7768983b0 2024-09-21T07:51:08.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.763+0000 7f120bcc3740 10 client.11283 fetch_fsmap learned FSMap version 273 2024-09-21T07:51:08.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.763+0000 7f120bcc3740 1 -- 172.21.15.39:0/1790786119 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=273,monmap=2+}) -- 0x55d7769a36b0 con 0x55d7768983b0 2024-09-21T07:51:08.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.763+0000 7f11e2a00640 1 -- 172.21.15.39:0/1790786119 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11e8024070 con 0x55d7768983b0 2024-09-21T07:51:08.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.764+0000 7f11e2a00640 1 -- 172.21.15.39:0/1790786119 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 273) ==== 3542+0+0 (secure 0 0 0) 0x7f11f008c790 con 0x55d7768983b0 2024-09-21T07:51:08.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.764+0000 7f120bcc3740 10 client.11283 fetch_fsmap finished waiting for FSMap version 273 2024-09-21T07:51:08.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.764+0000 7f120bcc3740 10 client.11283 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:51:08.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.764+0000 7f120bcc3740 20 client.11283 populate_metadata read hostname 'smithi039' 2024-09-21T07:51:08.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.764+0000 7f120bcc3740 1 --2- 172.21.15.39:0/1790786119 >> [v2:172.21.15.39:6832/109229385,v1:172.21.15.39:6834/109229385] conn(0x55d77687fe40 0x55d776c788d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.764+0000 7f120bcc3740 4 client.11283 mds_command: new command op to 11067 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:51:08.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.764+0000 7f120bcc3740 1 -- 172.21.15.39:0/1790786119 --> [v2:172.21.15.39:6832/109229385,v1:172.21.15.39:6834/109229385] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d776bdd520 con 0x55d77687fe40 2024-09-21T07:51:08.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.764+0000 7f11f5400640 1 --2- 172.21.15.39:0/1790786119 >> [v2:172.21.15.39:6832/109229385,v1:172.21.15.39:6834/109229385] conn(0x55d77687fe40 0x55d776c788d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.765+0000 7f11f5400640 1 --2- 172.21.15.39:0/1790786119 >> [v2:172.21.15.39:6832/109229385,v1:172.21.15.39:6834/109229385] conn(0x55d77687fe40 0x55d776c788d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b910eaa326f848c9 server_cookie=751298da5b9b48f in_seq=0 out_seq=0 2024-09-21T07:51:08.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.765+0000 7f11e2a00640 10 client.11283 ms_handle_connect on v2:172.21.15.39:6832/109229385 2024-09-21T07:51:08.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.766+0000 7f11e2a00640 1 -- 172.21.15.39:0/1790786119 <== mds.2 v2:172.21.15.39:6832/109229385 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d776bdd520 con 0x55d77687fe40 2024-09-21T07:51:08.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.766+0000 7f11e2a00640 10 client.11283 handle_command_reply: tid=0 2024-09-21T07:51:08.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.766+0000 7f120bcc3740 1 client.11283 shutdown 2024-09-21T07:51:08.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.766+0000 7f120bcc3740 1 -- 172.21.15.39:0/1790786119 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7768983b0 msgr2=0x55d776c75f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:08.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.766+0000 7f120bcc3740 1 --2- 172.21.15.39:0/1790786119 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7768983b0 0x55d776c75f70 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f11f0004010 tx=0x7f11f0082d50 comp rx=0 tx=0).stop 2024-09-21T07:51:08.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.766+0000 7f120bcc3740 1 -- 172.21.15.39:0/1790786119 shutdown_connections 2024-09-21T07:51:08.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.766+0000 7f120bcc3740 1 --2- 172.21.15.39:0/1790786119 >> [v2:172.21.15.39:6832/109229385,v1:172.21.15.39:6834/109229385] conn(0x55d77687fe40 0x55d776c788d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.766+0000 7f120bcc3740 1 --2- 172.21.15.39:0/1790786119 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d7768c1270 0x55d776ca8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.766+0000 7f120bcc3740 1 --2- 172.21.15.39:0/1790786119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d7768a0590 0x55d776ca8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.766+0000 7f120bcc3740 1 --2- 172.21.15.39:0/1790786119 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7768983b0 0x55d776c75f70 unknown :-1 s=CLOSED pgs=2426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.766+0000 7f11e2a00640 0 client.11283 ms_handle_reset on v2:172.21.15.39:6832/109229385 2024-09-21T07:51:08.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.766+0000 7f120bcc3740 1 -- 172.21.15.39:0/1790786119 >> 172.21.15.39:0/1790786119 conn(0x55d7768b53c0 msgr2=0x55d776caf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:08.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.766+0000 7f120bcc3740 1 -- 172.21.15.39:0/1790786119 shutdown_connections 2024-09-21T07:51:08.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.766+0000 7f120bcc3740 1 -- 172.21.15.39:0/1790786119 wait complete. 2024-09-21T07:51:08.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.766+0000 7f120bcc3740 20 client.11283 trim_cache size 0 max 16384 2024-09-21T07:51:08.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.786+0000 7f120bcc3740 1 Processor -- start 2024-09-21T07:51:08.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.786+0000 7f120bcc3740 1 -- start start 2024-09-21T07:51:08.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.787+0000 7f120bcc3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7768983b0 0x55d776c883a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.787+0000 7f120bcc3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d7768a0590 0x55d776c902a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.787+0000 7f120bcc3740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d7768c1270 0x55d776c907e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.787+0000 7f120bcc3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1200204130 con 0x55d7768983b0 2024-09-21T07:51:08.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.787+0000 7f120bcc3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1200204800 con 0x55d7768a0590 2024-09-21T07:51:08.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.787+0000 7f120bcc3740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f1200203130 con 0x55d7768c1270 2024-09-21T07:51:08.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.787+0000 7f11fde00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7768983b0 0x55d776c883a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.787+0000 7f11fde00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7768983b0 0x55d776c883a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44496/0 (socket says 172.21.15.39:44496) 2024-09-21T07:51:08.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.787+0000 7f11fde00640 1 -- 172.21.15.39:0/204672989 learned_addr learned my addr 172.21.15.39:0/204672989 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:51:08.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.788+0000 7f11fe800640 1 --2- 172.21.15.39:0/204672989 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d7768c1270 0x55d776c907e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.788+0000 7f11f5400640 1 --2- 172.21.15.39:0/204672989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d7768a0590 0x55d776c902a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.788+0000 7f11fde00640 1 -- 172.21.15.39:0/204672989 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d7768c1270 msgr2=0x55d776c907e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:51:08.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.788+0000 7f11fde00640 1 --2- 172.21.15.39:0/204672989 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d7768c1270 0x55d776c907e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.788+0000 7f11fde00640 1 -- 172.21.15.39:0/204672989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d7768a0590 msgr2=0x55d776c902a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:51:08.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.788+0000 7f11fde00640 1 --2- 172.21.15.39:0/204672989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d7768a0590 0x55d776c902a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.788+0000 7f11fde00640 1 -- 172.21.15.39:0/204672989 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d776bcab80 con 0x55d7768983b0 2024-09-21T07:51:08.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.788+0000 7f11fde00640 1 --2- 172.21.15.39:0/204672989 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7768983b0 0x55d776c883a0 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7f11f00099c0 tx=0x7f11f0083180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71aeef8fe129ac7b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:08.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.788+0000 7f11edc00640 1 -- 172.21.15.39:0/204672989 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f00a9040 con 0x55d7768983b0 2024-09-21T07:51:08.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.788+0000 7f11edc00640 1 -- 172.21.15.39:0/204672989 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11f008cc20 con 0x55d7768983b0 2024-09-21T07:51:08.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.788+0000 7f11edc00640 1 -- 172.21.15.39:0/204672989 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f00837e0 con 0x55d7768983b0 2024-09-21T07:51:08.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.789+0000 7f120bcc3740 1 -- 172.21.15.39:0/204672989 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7768983b0 msgr2=0x55d776c883a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:08.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.789+0000 7f120bcc3740 1 --2- 172.21.15.39:0/204672989 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7768983b0 0x55d776c883a0 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7f11f00099c0 tx=0x7f11f0083180 comp rx=0 tx=0).stop 2024-09-21T07:51:08.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.789+0000 7f120bcc3740 1 -- 172.21.15.39:0/204672989 shutdown_connections 2024-09-21T07:51:08.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.789+0000 7f120bcc3740 1 --2- 172.21.15.39:0/204672989 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d7768c1270 0x55d776c907e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.789+0000 7f120bcc3740 1 --2- 172.21.15.39:0/204672989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d7768a0590 0x55d776c902a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.789+0000 7f120bcc3740 1 --2- 172.21.15.39:0/204672989 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7768983b0 0x55d776c883a0 unknown :-1 s=CLOSED pgs=2427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.789+0000 7f120bcc3740 1 -- 172.21.15.39:0/204672989 >> 172.21.15.39:0/204672989 conn(0x55d7768b53c0 msgr2=0x55d776c66b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:08.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.789+0000 7f120bcc3740 1 -- 172.21.15.39:0/204672989 shutdown_connections 2024-09-21T07:51:08.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.789+0000 7f120bcc3740 1 -- 172.21.15.39:0/204672989 wait complete. 2024-09-21T07:51:08.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.790+0000 7f120bcc3740 1 -- start start 2024-09-21T07:51:08.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.790+0000 7f120bcc3740 1 Processor -- start 2024-09-21T07:51:08.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.792+0000 7f120bcc3740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d7768983b0 0x55d776c76190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.792+0000 7f120bcc3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d7768a0590 0x55d776c90b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.792+0000 7f120bcc3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7768c1270 0x55d776d2f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.792+0000 7f120bcc3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d776b71330 con 0x55d7768c1270 2024-09-21T07:51:08.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.792+0000 7f120bcc3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d776c76e80 con 0x55d7768a0590 2024-09-21T07:51:08.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.792+0000 7f120bcc3740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55d776ca9810 con 0x55d7768983b0 2024-09-21T07:51:08.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.792+0000 7f120bcc3740 10 client.0 _ll_register_callbacks cb 0x7f11f007f1e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:51:08.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.792+0000 7f11fe800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7768c1270 0x55d776d2f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.792+0000 7f11fe800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7768c1270 0x55d776d2f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44510/0 (socket says 172.21.15.39:44510) 2024-09-21T07:51:08.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.792+0000 7f11fe800640 1 -- 172.21.15.39:0/2618836942 learned_addr learned my addr 172.21.15.39:0/2618836942 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:51:08.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.792+0000 7f11fde00640 1 --2- 172.21.15.39:0/2618836942 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d7768983b0 0x55d776c76190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.792+0000 7f11f5400640 1 --2- 172.21.15.39:0/2618836942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d7768a0590 0x55d776c90b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.792+0000 7f11fe800640 1 -- 172.21.15.39:0/2618836942 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d7768983b0 msgr2=0x55d776c76190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:51:08.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.792+0000 7f11fe800640 1 --2- 172.21.15.39:0/2618836942 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d7768983b0 0x55d776c76190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.792+0000 7f11fe800640 1 -- 172.21.15.39:0/2618836942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d7768a0590 msgr2=0x55d776c90b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:51:08.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.792+0000 7f11fe800640 1 --2- 172.21.15.39:0/2618836942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d7768a0590 0x55d776c90b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.792+0000 7f11fe800640 1 -- 172.21.15.39:0/2618836942 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d776d2f820 con 0x55d7768c1270 2024-09-21T07:51:08.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.793+0000 7f11fe800640 1 --2- 172.21.15.39:0/2618836942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7768c1270 0x55d776d2f2e0 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7f11f800aea0 tx=0x7f11f800af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=881e6dd6eb031c51 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:08.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.793+0000 7f11e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:51:08.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.793+0000 7f11e2a00640 1 -- 172.21.15.39:0/2618836942 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f8022610 con 0x55d7768c1270 2024-09-21T07:51:08.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.793+0000 7f11e2a00640 1 -- 172.21.15.39:0/2618836942 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(241..241 src has 1..241) ==== 7476+0+0 (secure 0 0 0) 0x7f11f80227b0 con 0x55d7768c1270 2024-09-21T07:51:08.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.793+0000 7f120bcc3740 1 -- 172.21.15.39:0/2618836942 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11f008cc20 con 0x55d7768c1270 2024-09-21T07:51:08.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.793+0000 7f11e2a00640 1 -- 172.21.15.39:0/2618836942 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=273) ==== 24+0+0 (secure 0 0 0) 0x7f11f802a050 con 0x55d7768c1270 2024-09-21T07:51:08.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.793+0000 7f120bcc3740 10 client.11289 fetch_fsmap learned FSMap version 273 2024-09-21T07:51:08.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.793+0000 7f120bcc3740 1 -- 172.21.15.39:0/2618836942 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=273,monmap=2+}) -- 0x55d776cc6570 con 0x55d7768c1270 2024-09-21T07:51:08.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.794+0000 7f11e2a00640 1 -- 172.21.15.39:0/2618836942 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11f008cc20 con 0x55d7768c1270 2024-09-21T07:51:08.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.794+0000 7f11e2a00640 1 -- 172.21.15.39:0/2618836942 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 273) ==== 3542+0+0 (secure 0 0 0) 0x7f11f802d020 con 0x55d7768c1270 2024-09-21T07:51:08.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.794+0000 7f120bcc3740 10 client.11289 fetch_fsmap finished waiting for FSMap version 273 2024-09-21T07:51:08.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.794+0000 7f120bcc3740 10 client.11289 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:51:08.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.794+0000 7f120bcc3740 20 client.11289 populate_metadata read hostname 'smithi039' 2024-09-21T07:51:08.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.794+0000 7f120bcc3740 1 --2- 172.21.15.39:0/2618836942 >> [v2:172.21.15.39:6832/109229385,v1:172.21.15.39:6834/109229385] conn(0x55d776c7a800 0x55d776c7ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.794+0000 7f120bcc3740 4 client.11289 mds_command: new command op to 11067 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:51:08.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.794+0000 7f120bcc3740 1 -- 172.21.15.39:0/2618836942 --> [v2:172.21.15.39:6832/109229385,v1:172.21.15.39:6834/109229385] -- command(tid 0: {"prefix": "status"}) -- 0x55d776c8add0 con 0x55d776c7a800 2024-09-21T07:51:08.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.794+0000 7f11fde00640 1 --2- 172.21.15.39:0/2618836942 >> [v2:172.21.15.39:6832/109229385,v1:172.21.15.39:6834/109229385] conn(0x55d776c7a800 0x55d776c7ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.795+0000 7f11fde00640 1 --2- 172.21.15.39:0/2618836942 >> [v2:172.21.15.39:6832/109229385,v1:172.21.15.39:6834/109229385] conn(0x55d776c7a800 0x55d776c7ac10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=501465e9960f99a server_cookie=b5c1006dcd05316c in_seq=0 out_seq=0 2024-09-21T07:51:08.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.795+0000 7f11e2a00640 10 client.11289 ms_handle_connect on v2:172.21.15.39:6832/109229385 2024-09-21T07:51:08.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.795+0000 7f11e2a00640 1 -- 172.21.15.39:0/2618836942 <== mds.2 v2:172.21.15.39:6832/109229385 1 ==== command_reply(tid 0: 0 ) ==== 8+0+335 (crc 0 0 0) 0x55d776c8add0 con 0x55d776c7a800 2024-09-21T07:51:08.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.795+0000 7f11e2a00640 10 client.11289 handle_command_reply: tid=0 2024-09-21T07:51:08.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.795+0000 7f120bcc3740 1 client.11289 shutdown 2024-09-21T07:51:08.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.795+0000 7f120bcc3740 1 -- 172.21.15.39:0/2618836942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7768c1270 msgr2=0x55d776d2f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:08.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.795+0000 7f120bcc3740 1 --2- 172.21.15.39:0/2618836942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7768c1270 0x55d776d2f2e0 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7f11f800aea0 tx=0x7f11f800af90 comp rx=0 tx=0).stop 2024-09-21T07:51:08.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.795+0000 7f120bcc3740 1 -- 172.21.15.39:0/2618836942 shutdown_connections 2024-09-21T07:51:08.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.795+0000 7f120bcc3740 1 --2- 172.21.15.39:0/2618836942 >> [v2:172.21.15.39:6832/109229385,v1:172.21.15.39:6834/109229385] conn(0x55d776c7a800 0x55d776c7ac10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.795+0000 7f120bcc3740 1 --2- 172.21.15.39:0/2618836942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d7768c1270 0x55d776d2f2e0 unknown :-1 s=CLOSED pgs=2428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.795+0000 7f120bcc3740 1 --2- 172.21.15.39:0/2618836942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d7768a0590 0x55d776c90b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.795+0000 7f120bcc3740 1 --2- 172.21.15.39:0/2618836942 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d7768983b0 0x55d776c76190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.795+0000 7f11e2a00640 0 client.11289 ms_handle_reset on v2:172.21.15.39:6832/109229385 2024-09-21T07:51:08.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.795+0000 7f120bcc3740 1 -- 172.21.15.39:0/2618836942 >> 172.21.15.39:0/2618836942 conn(0x55d7768b53c0 msgr2=0x55d776c923f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:08.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.795+0000 7f120bcc3740 1 -- 172.21.15.39:0/2618836942 shutdown_connections 2024-09-21T07:51:08.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.795+0000 7f120bcc3740 1 -- 172.21.15.39:0/2618836942 wait complete. 2024-09-21T07:51:08.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.795+0000 7f120bcc3740 20 client.11289 trim_cache size 0 max 16384 2024-09-21T07:51:08.800 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:51:08.800 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:51:08.800 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 2, 2024-09-21T07:51:08.800 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 11067, 2024-09-21T07:51:08.801 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:51:08.801 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:51:08.801 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:51:08.801 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 7.8799772910000003, 2024-09-21T07:51:08.801 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 273, 2024-09-21T07:51:08.801 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 241, 2024-09-21T07:51:08.801 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 241, 2024-09-21T07:51:08.801 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 36.277115954999999 2024-09-21T07:51:08.801 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:51:08.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.796+0000 7f11ff200640 1 -- 172.21.15.39:0/1146249220 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f11dc078bc0 msgr2=0x7f11dc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:08.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.796+0000 7f11ff200640 1 --2- 172.21.15.39:0/1146249220 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f11dc078bc0 0x7f11dc07b0d0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f12000b5e40 tx=0x7f11e8009060 comp rx=0 tx=0).stop 2024-09-21T07:51:08.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.796+0000 7f11ff200640 1 -- 172.21.15.39:0/1146249220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f120006d320 msgr2=0x7f12001f57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:08.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.796+0000 7f11ff200640 1 --2- 172.21.15.39:0/1146249220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f120006d320 0x7f12001f57b0 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f11f0034040 tx=0x7f11f00039e0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.796+0000 7f11ff200640 1 -- 172.21.15.39:0/1146249220 shutdown_connections 2024-09-21T07:51:08.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.796+0000 7f11ff200640 1 --2- 172.21.15.39:0/1146249220 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f11dc078bc0 0x7f11dc07b0d0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.796+0000 7f11ff200640 1 --2- 172.21.15.39:0/1146249220 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f120015d760 0x7f12000ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.796+0000 7f11ff200640 1 --2- 172.21.15.39:0/1146249220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12000cdb20 0x7f12000aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.796+0000 7f11ff200640 1 --2- 172.21.15.39:0/1146249220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f120006d320 0x7f12001f57b0 unknown :-1 s=CLOSED pgs=2425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.796+0000 7f11ff200640 1 -- 172.21.15.39:0/1146249220 >> 172.21.15.39:0/1146249220 conn(0x7f12000b6f80 msgr2=0x7f12001600f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:08.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.796+0000 7f11ff200640 1 -- 172.21.15.39:0/1146249220 shutdown_connections 2024-09-21T07:51:08.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:08.796+0000 7f11ff200640 1 -- 172.21.15.39:0/1146249220 wait complete. 2024-09-21T07:51:08.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:08.822 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir d0/d1/dir/.snap/*) 2024-09-21T07:51:08.875 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T07:51:08.878 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T07:51:08.972 INFO:teuthology.orchestra.run.smithi039.stdout:[global] 2024-09-21T07:51:08.972 INFO:teuthology.orchestra.run.smithi039.stdout: chdir = "" 2024-09-21T07:51:08.973 INFO:teuthology.orchestra.run.smithi039.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T07:51:08.973 INFO:teuthology.orchestra.run.smithi039.stdout: auth supported = cephx 2024-09-21T07:51:08.973 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:08.973 INFO:teuthology.orchestra.run.smithi039.stdout: filestore xattr use omap = true 2024-09-21T07:51:08.973 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:08.973 INFO:teuthology.orchestra.run.smithi039.stdout: mon clock drift allowed = 1.000 2024-09-21T07:51:08.973 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:08.973 INFO:teuthology.orchestra.run.smithi039.stdout: osd crush chooseleaf type = 0 2024-09-21T07:51:08.973 INFO:teuthology.orchestra.run.smithi039.stdout: auth debug = true 2024-09-21T07:51:08.973 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:08.973 INFO:teuthology.orchestra.run.smithi039.stdout: ms die on old message = true 2024-09-21T07:51:08.973 INFO:teuthology.orchestra.run.smithi039.stdout: ms die on bug = true 2024-09-21T07:51:08.973 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:08.973 INFO:teuthology.orchestra.run.smithi039.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T07:51:08.973 INFO:teuthology.orchestra.run.smithi039.stdout: mon pg warn max object skew = 0 2024-09-21T07:51:08.973 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:08.974 INFO:teuthology.orchestra.run.smithi039.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T07:51:08.974 INFO:teuthology.orchestra.run.smithi039.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T07:51:08.974 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:08.974 INFO:teuthology.orchestra.run.smithi039.stdout: osd pool default size = 2 2024-09-21T07:51:08.974 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:08.974 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd allow primary affinity = true 2024-09-21T07:51:08.974 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd allow pg remap = true 2024-09-21T07:51:08.974 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on legacy crush tunables = false 2024-09-21T07:51:08.974 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on crush straw calc version zero = false 2024-09-21T07:51:08.974 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on no sortbitwise = false 2024-09-21T07:51:08.974 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on osd down out interval zero = false 2024-09-21T07:51:08.974 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on too few osds = false 2024-09-21T07:51:08.974 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T07:51:08.974 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T07:51:08.974 INFO:teuthology.orchestra.run.smithi039.stdout: mon_allow_pool_size_one = true 2024-09-21T07:51:08.975 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:08.975 INFO:teuthology.orchestra.run.smithi039.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T07:51:08.975 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:08.975 INFO:teuthology.orchestra.run.smithi039.stdout: osd default data pool replay window = 5 2024-09-21T07:51:08.975 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:08.975 INFO:teuthology.orchestra.run.smithi039.stdout: mon allow pool delete = true 2024-09-21T07:51:08.975 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:08.975 INFO:teuthology.orchestra.run.smithi039.stdout: mon cluster log file level = debug 2024-09-21T07:51:08.975 INFO:teuthology.orchestra.run.smithi039.stdout: debug asserts on shutdown = true 2024-09-21T07:51:08.975 INFO:teuthology.orchestra.run.smithi039.stdout: mon health detail to clog = false 2024-09-21T07:51:08.975 INFO:teuthology.orchestra.run.smithi039.stdout: mon host = "172.21.15.39,172.21.15.148,[v2:172.21.15.148:3301,v1:172.21.15.148:6790]" 2024-09-21T07:51:08.975 INFO:teuthology.orchestra.run.smithi039.stdout: ms die on skipped message = False 2024-09-21T07:51:08.975 INFO:teuthology.orchestra.run.smithi039.stdout: fsid = 7ab9b04a-f88d-49aa-a493-e5c076637ffb 2024-09-21T07:51:08.975 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:08.975 INFO:teuthology.orchestra.run.smithi039.stdout:[osd] 2024-09-21T07:51:08.975 INFO:teuthology.orchestra.run.smithi039.stdout: osd journal size = 100 2024-09-21T07:51:08.975 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:08.976 INFO:teuthology.orchestra.run.smithi039.stdout: osd scrub load threshold = 5.0 2024-09-21T07:51:08.976 INFO:teuthology.orchestra.run.smithi039.stdout: osd scrub max interval = 600 2024-09-21T07:51:08.976 INFO:teuthology.orchestra.run.smithi039.stdout: osd mclock profile = high_recovery_ops 2024-09-21T07:51:08.976 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:08.976 INFO:teuthology.orchestra.run.smithi039.stdout: osd recover clone overlap = true 2024-09-21T07:51:08.976 INFO:teuthology.orchestra.run.smithi039.stdout: osd recovery max chunk = 1048576 2024-09-21T07:51:08.976 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:08.976 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug shutdown = true 2024-09-21T07:51:08.976 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug op order = true 2024-09-21T07:51:08.976 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug verify stray on activate = true 2024-09-21T07:51:08.976 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug trim objects = true 2024-09-21T07:51:08.976 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:08.976 INFO:teuthology.orchestra.run.smithi039.stdout: osd open classes on start = true 2024-09-21T07:51:08.976 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug pg log writeout = true 2024-09-21T07:51:08.976 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:08.976 INFO:teuthology.orchestra.run.smithi039.stdout: osd deep scrub update digest min age = 30 2024-09-21T07:51:08.977 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:08.977 INFO:teuthology.orchestra.run.smithi039.stdout: osd map max advance = 10 2024-09-21T07:51:08.977 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:08.977 INFO:teuthology.orchestra.run.smithi039.stdout: journal zero on create = true 2024-09-21T07:51:08.977 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:08.977 INFO:teuthology.orchestra.run.smithi039.stdout: filestore ondisk finisher threads = 3 2024-09-21T07:51:08.977 INFO:teuthology.orchestra.run.smithi039.stdout: filestore apply finisher threads = 3 2024-09-21T07:51:08.977 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:08.977 INFO:teuthology.orchestra.run.smithi039.stdout: bdev debug aio = true 2024-09-21T07:51:08.977 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug misdirected ops = true 2024-09-21T07:51:08.977 INFO:teuthology.orchestra.run.smithi039.stdout: bdev async discard = True 2024-09-21T07:51:08.977 INFO:teuthology.orchestra.run.smithi039.stdout: bdev enable discard = True 2024-09-21T07:51:08.977 INFO:teuthology.orchestra.run.smithi039.stdout: bluestore allocator = bitmap 2024-09-21T07:51:08.977 INFO:teuthology.orchestra.run.smithi039.stdout: bluestore block size = 96636764160 2024-09-21T07:51:08.977 INFO:teuthology.orchestra.run.smithi039.stdout: bluestore fsck on mount = True 2024-09-21T07:51:08.977 INFO:teuthology.orchestra.run.smithi039.stdout: debug bluefs = 1/20 2024-09-21T07:51:08.977 INFO:teuthology.orchestra.run.smithi039.stdout: debug bluestore = 1/20 2024-09-21T07:51:08.978 INFO:teuthology.orchestra.run.smithi039.stdout: debug ms = 1 2024-09-21T07:51:08.978 INFO:teuthology.orchestra.run.smithi039.stdout: debug osd = 20 2024-09-21T07:51:08.978 INFO:teuthology.orchestra.run.smithi039.stdout: debug rocksdb = 4/10 2024-09-21T07:51:08.978 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T07:51:08.978 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd full ratio = 0.9 2024-09-21T07:51:08.978 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd nearfull ratio = 0.8 2024-09-21T07:51:08.978 INFO:teuthology.orchestra.run.smithi039.stdout: osd failsafe full ratio = 0.95 2024-09-21T07:51:08.978 INFO:teuthology.orchestra.run.smithi039.stdout: osd objectstore = bluestore 2024-09-21T07:51:08.978 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:08.978 INFO:teuthology.orchestra.run.smithi039.stdout:[mgr] 2024-09-21T07:51:08.978 INFO:teuthology.orchestra.run.smithi039.stdout: debug ms = 1 2024-09-21T07:51:08.978 INFO:teuthology.orchestra.run.smithi039.stdout: debug mgr = 20 2024-09-21T07:51:08.978 INFO:teuthology.orchestra.run.smithi039.stdout: debug mon = 20 2024-09-21T07:51:08.978 INFO:teuthology.orchestra.run.smithi039.stdout: debug auth = 20 2024-09-21T07:51:08.978 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min pgs per osd = 4 2024-09-21T07:51:08.979 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min bytes per osd = 10 2024-09-21T07:51:08.979 INFO:teuthology.orchestra.run.smithi039.stdout: mgr/telemetry/nag = false 2024-09-21T07:51:08.979 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:08.979 INFO:teuthology.orchestra.run.smithi039.stdout:[mon] 2024-09-21T07:51:08.979 INFO:teuthology.orchestra.run.smithi039.stdout: debug ms = 1 2024-09-21T07:51:08.979 INFO:teuthology.orchestra.run.smithi039.stdout: debug mon = 20 2024-09-21T07:51:08.979 INFO:teuthology.orchestra.run.smithi039.stdout: debug paxos = 20 2024-09-21T07:51:08.979 INFO:teuthology.orchestra.run.smithi039.stdout: debug auth = 20 2024-09-21T07:51:08.979 INFO:teuthology.orchestra.run.smithi039.stdout: mon data avail warn = 5 2024-09-21T07:51:08.979 INFO:teuthology.orchestra.run.smithi039.stdout: mon mgr mkfs grace = 240 2024-09-21T07:51:08.979 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min pgs per osd = 4 2024-09-21T07:51:08.979 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd reporter subtree level = osd 2024-09-21T07:51:08.979 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd prime pg temp = true 2024-09-21T07:51:08.979 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min bytes per osd = 10 2024-09-21T07:51:08.979 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:08.979 INFO:teuthology.orchestra.run.smithi039.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T07:51:08.980 INFO:teuthology.orchestra.run.smithi039.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T07:51:08.980 INFO:teuthology.orchestra.run.smithi039.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T07:51:08.980 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:08.980 INFO:teuthology.orchestra.run.smithi039.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T07:51:08.980 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T07:51:08.980 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T07:51:08.980 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:08.980 INFO:teuthology.orchestra.run.smithi039.stdout: # 1m isn't quite enough 2024-09-21T07:51:08.980 INFO:teuthology.orchestra.run.smithi039.stdout: mon_down_mkfs_grace = 2m 2024-09-21T07:51:08.980 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:08.980 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_filestore_osds = false 2024-09-21T07:51:08.980 INFO:teuthology.orchestra.run.smithi039.stdout: mon down mkfs grace = 300 2024-09-21T07:51:08.980 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:08.980 INFO:teuthology.orchestra.run.smithi039.stdout:[client] 2024-09-21T07:51:08.980 INFO:teuthology.orchestra.run.smithi039.stdout: rgw cache enabled = true 2024-09-21T07:51:08.981 INFO:teuthology.orchestra.run.smithi039.stdout: rgw enable ops log = true 2024-09-21T07:51:08.981 INFO:teuthology.orchestra.run.smithi039.stdout: rgw enable usage log = true 2024-09-21T07:51:08.981 INFO:teuthology.orchestra.run.smithi039.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T07:51:08.981 INFO:teuthology.orchestra.run.smithi039.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T07:51:08.981 INFO:teuthology.orchestra.run.smithi039.stdout: pid file = "" 2024-09-21T07:51:08.981 INFO:teuthology.orchestra.run.smithi039.stdout:[mon.a] 2024-09-21T07:51:08.981 INFO:teuthology.orchestra.run.smithi039.stdout:[mon.b] 2024-09-21T07:51:08.981 INFO:teuthology.orchestra.run.smithi039.stdout:[mon.c] 2024-09-21T07:51:08.981 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.a] 2024-09-21T07:51:08.981 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.c] 2024-09-21T07:51:08.981 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.b] 2024-09-21T07:51:08.981 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.d] 2024-09-21T07:51:08.981 INFO:teuthology.orchestra.run.smithi039.stdout:[mds] 2024-09-21T07:51:08.981 INFO:teuthology.orchestra.run.smithi039.stdout: mds_reconnect_timeout = 5 2024-09-21T07:51:08.985 INFO:teuthology.orchestra.run.smithi148.stdout:[global] 2024-09-21T07:51:08.985 INFO:teuthology.orchestra.run.smithi148.stdout: chdir = "" 2024-09-21T07:51:08.985 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T07:51:08.985 INFO:teuthology.orchestra.run.smithi148.stdout: auth supported = cephx 2024-09-21T07:51:08.985 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:51:08.985 INFO:teuthology.orchestra.run.smithi148.stdout: filestore xattr use omap = true 2024-09-21T07:51:08.986 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:51:08.986 INFO:teuthology.orchestra.run.smithi148.stdout: mon clock drift allowed = 1.000 2024-09-21T07:51:08.986 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:51:08.986 INFO:teuthology.orchestra.run.smithi148.stdout: osd crush chooseleaf type = 0 2024-09-21T07:51:08.986 INFO:teuthology.orchestra.run.smithi148.stdout: auth debug = true 2024-09-21T07:51:08.986 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:51:08.986 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on old message = true 2024-09-21T07:51:08.986 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on bug = true 2024-09-21T07:51:08.986 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:51:08.986 INFO:teuthology.orchestra.run.smithi148.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T07:51:08.986 INFO:teuthology.orchestra.run.smithi148.stdout: mon pg warn max object skew = 0 2024-09-21T07:51:08.986 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:51:08.986 INFO:teuthology.orchestra.run.smithi148.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T07:51:08.987 INFO:teuthology.orchestra.run.smithi148.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T07:51:08.987 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:51:08.987 INFO:teuthology.orchestra.run.smithi148.stdout: osd pool default size = 2 2024-09-21T07:51:08.987 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:51:08.987 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd allow primary affinity = true 2024-09-21T07:51:08.987 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd allow pg remap = true 2024-09-21T07:51:08.987 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on legacy crush tunables = false 2024-09-21T07:51:08.987 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on crush straw calc version zero = false 2024-09-21T07:51:08.987 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on no sortbitwise = false 2024-09-21T07:51:08.987 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on osd down out interval zero = false 2024-09-21T07:51:08.987 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on too few osds = false 2024-09-21T07:51:08.987 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T07:51:08.987 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T07:51:08.987 INFO:teuthology.orchestra.run.smithi148.stdout: mon_allow_pool_size_one = true 2024-09-21T07:51:08.987 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:51:08.988 INFO:teuthology.orchestra.run.smithi148.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T07:51:08.988 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:51:08.988 INFO:teuthology.orchestra.run.smithi148.stdout: osd default data pool replay window = 5 2024-09-21T07:51:08.988 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:51:08.988 INFO:teuthology.orchestra.run.smithi148.stdout: mon allow pool delete = true 2024-09-21T07:51:08.988 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:51:08.988 INFO:teuthology.orchestra.run.smithi148.stdout: mon cluster log file level = debug 2024-09-21T07:51:08.988 INFO:teuthology.orchestra.run.smithi148.stdout: debug asserts on shutdown = true 2024-09-21T07:51:08.988 INFO:teuthology.orchestra.run.smithi148.stdout: mon health detail to clog = false 2024-09-21T07:51:08.988 INFO:teuthology.orchestra.run.smithi148.stdout: mon host = "172.21.15.39,172.21.15.148,[v2:172.21.15.148:3301,v1:172.21.15.148:6790]" 2024-09-21T07:51:08.988 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on skipped message = False 2024-09-21T07:51:08.988 INFO:teuthology.orchestra.run.smithi148.stdout: fsid = 7ab9b04a-f88d-49aa-a493-e5c076637ffb 2024-09-21T07:51:08.988 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:51:08.988 INFO:teuthology.orchestra.run.smithi148.stdout:[osd] 2024-09-21T07:51:08.988 INFO:teuthology.orchestra.run.smithi148.stdout: osd journal size = 100 2024-09-21T07:51:08.989 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:51:08.989 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub load threshold = 5.0 2024-09-21T07:51:08.989 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub max interval = 600 2024-09-21T07:51:08.989 INFO:teuthology.orchestra.run.smithi148.stdout: osd mclock profile = high_recovery_ops 2024-09-21T07:51:08.989 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:51:08.989 INFO:teuthology.orchestra.run.smithi148.stdout: osd recover clone overlap = true 2024-09-21T07:51:08.989 INFO:teuthology.orchestra.run.smithi148.stdout: osd recovery max chunk = 1048576 2024-09-21T07:51:08.989 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:51:08.989 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug shutdown = true 2024-09-21T07:51:08.989 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug op order = true 2024-09-21T07:51:08.989 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug verify stray on activate = true 2024-09-21T07:51:08.989 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug trim objects = true 2024-09-21T07:51:08.989 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:51:08.989 INFO:teuthology.orchestra.run.smithi148.stdout: osd open classes on start = true 2024-09-21T07:51:08.990 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug pg log writeout = true 2024-09-21T07:51:08.990 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:51:08.990 INFO:teuthology.orchestra.run.smithi148.stdout: osd deep scrub update digest min age = 30 2024-09-21T07:51:08.990 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:51:08.990 INFO:teuthology.orchestra.run.smithi148.stdout: osd map max advance = 10 2024-09-21T07:51:08.990 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:51:08.990 INFO:teuthology.orchestra.run.smithi148.stdout: journal zero on create = true 2024-09-21T07:51:08.990 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:51:08.990 INFO:teuthology.orchestra.run.smithi148.stdout: filestore ondisk finisher threads = 3 2024-09-21T07:51:08.990 INFO:teuthology.orchestra.run.smithi148.stdout: filestore apply finisher threads = 3 2024-09-21T07:51:08.990 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:51:08.990 INFO:teuthology.orchestra.run.smithi148.stdout: bdev debug aio = true 2024-09-21T07:51:08.991 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug misdirected ops = true 2024-09-21T07:51:08.991 INFO:teuthology.orchestra.run.smithi148.stdout: bdev async discard = True 2024-09-21T07:51:08.991 INFO:teuthology.orchestra.run.smithi148.stdout: bdev enable discard = True 2024-09-21T07:51:08.991 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore allocator = bitmap 2024-09-21T07:51:08.991 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore block size = 96636764160 2024-09-21T07:51:08.991 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore fsck on mount = True 2024-09-21T07:51:08.991 INFO:teuthology.orchestra.run.smithi148.stdout: debug bluefs = 1/20 2024-09-21T07:51:08.991 INFO:teuthology.orchestra.run.smithi148.stdout: debug bluestore = 1/20 2024-09-21T07:51:08.991 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-09-21T07:51:08.991 INFO:teuthology.orchestra.run.smithi148.stdout: debug osd = 20 2024-09-21T07:51:08.991 INFO:teuthology.orchestra.run.smithi148.stdout: debug rocksdb = 4/10 2024-09-21T07:51:08.991 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T07:51:08.991 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd full ratio = 0.9 2024-09-21T07:51:08.991 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd nearfull ratio = 0.8 2024-09-21T07:51:08.992 INFO:teuthology.orchestra.run.smithi148.stdout: osd failsafe full ratio = 0.95 2024-09-21T07:51:08.992 INFO:teuthology.orchestra.run.smithi148.stdout: osd objectstore = bluestore 2024-09-21T07:51:08.992 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:51:08.992 INFO:teuthology.orchestra.run.smithi148.stdout:[mgr] 2024-09-21T07:51:08.992 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-09-21T07:51:08.992 INFO:teuthology.orchestra.run.smithi148.stdout: debug mgr = 20 2024-09-21T07:51:08.992 INFO:teuthology.orchestra.run.smithi148.stdout: debug mon = 20 2024-09-21T07:51:08.992 INFO:teuthology.orchestra.run.smithi148.stdout: debug auth = 20 2024-09-21T07:51:08.992 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min pgs per osd = 4 2024-09-21T07:51:08.992 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min bytes per osd = 10 2024-09-21T07:51:08.992 INFO:teuthology.orchestra.run.smithi148.stdout: mgr/telemetry/nag = false 2024-09-21T07:51:08.992 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:51:08.992 INFO:teuthology.orchestra.run.smithi148.stdout:[mon] 2024-09-21T07:51:08.992 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-09-21T07:51:08.992 INFO:teuthology.orchestra.run.smithi148.stdout: debug mon = 20 2024-09-21T07:51:08.993 INFO:teuthology.orchestra.run.smithi148.stdout: debug paxos = 20 2024-09-21T07:51:08.993 INFO:teuthology.orchestra.run.smithi148.stdout: debug auth = 20 2024-09-21T07:51:08.993 INFO:teuthology.orchestra.run.smithi148.stdout: mon data avail warn = 5 2024-09-21T07:51:08.993 INFO:teuthology.orchestra.run.smithi148.stdout: mon mgr mkfs grace = 240 2024-09-21T07:51:08.993 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min pgs per osd = 4 2024-09-21T07:51:08.993 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd reporter subtree level = osd 2024-09-21T07:51:08.993 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd prime pg temp = true 2024-09-21T07:51:08.993 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min bytes per osd = 10 2024-09-21T07:51:08.993 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:51:08.993 INFO:teuthology.orchestra.run.smithi148.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T07:51:08.993 INFO:teuthology.orchestra.run.smithi148.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T07:51:08.993 INFO:teuthology.orchestra.run.smithi148.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T07:51:08.993 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:51:08.993 INFO:teuthology.orchestra.run.smithi148.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T07:51:08.993 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T07:51:08.994 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T07:51:08.994 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:51:08.994 INFO:teuthology.orchestra.run.smithi148.stdout: # 1m isn't quite enough 2024-09-21T07:51:08.994 INFO:teuthology.orchestra.run.smithi148.stdout: mon_down_mkfs_grace = 2m 2024-09-21T07:51:08.994 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:51:08.994 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_filestore_osds = false 2024-09-21T07:51:08.994 INFO:teuthology.orchestra.run.smithi148.stdout: mon down mkfs grace = 300 2024-09-21T07:51:08.994 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:51:08.994 INFO:teuthology.orchestra.run.smithi148.stdout:[client] 2024-09-21T07:51:08.994 INFO:teuthology.orchestra.run.smithi148.stdout: rgw cache enabled = true 2024-09-21T07:51:08.994 INFO:teuthology.orchestra.run.smithi148.stdout: rgw enable ops log = true 2024-09-21T07:51:08.994 INFO:teuthology.orchestra.run.smithi148.stdout: rgw enable usage log = true 2024-09-21T07:51:08.994 INFO:teuthology.orchestra.run.smithi148.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T07:51:08.994 INFO:teuthology.orchestra.run.smithi148.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T07:51:08.994 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = "" 2024-09-21T07:51:08.995 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.a] 2024-09-21T07:51:08.995 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.b] 2024-09-21T07:51:08.995 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.c] 2024-09-21T07:51:08.995 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.a] 2024-09-21T07:51:08.995 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.c] 2024-09-21T07:51:08.995 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.b] 2024-09-21T07:51:08.995 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.d] 2024-09-21T07:51:08.995 INFO:teuthology.orchestra.run.smithi148.stdout:[mds] 2024-09-21T07:51:08.995 INFO:teuthology.orchestra.run.smithi148.stdout: mds_reconnect_timeout = 5 2024-09-21T07:51:09.011 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:51:09.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.237+0000 7f2b06a00640 1 -- 172.21.15.39:0/3714169548 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2b000b4db0 msgr2=0x7f2b000d4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:09.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.237+0000 7f2b06a00640 1 --2- 172.21.15.39:0/3714169548 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2b000b4db0 0x7f2b000d4c90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2af4009850 tx=0x7f2af402a5a0 comp rx=0 tx=0).stop 2024-09-21T07:51:09.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.237+0000 7f2b06a00640 1 -- 172.21.15.39:0/3714169548 shutdown_connections 2024-09-21T07:51:09.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.237+0000 7f2b06a00640 1 --2- 172.21.15.39:0/3714169548 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2b000b4db0 0x7f2b000d4c90 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:09.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.237+0000 7f2b06a00640 1 -- 172.21.15.39:0/3714169548 >> 172.21.15.39:0/3714169548 conn(0x7f2b0008bfb0 msgr2=0x7f2b000d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:09.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.237+0000 7f2b06a00640 1 -- 172.21.15.39:0/3714169548 shutdown_connections 2024-09-21T07:51:09.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.237+0000 7f2b06a00640 1 -- 172.21.15.39:0/3714169548 wait complete. 2024-09-21T07:51:09.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.239+0000 7f2b06a00640 1 Processor -- start 2024-09-21T07:51:09.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.240+0000 7f2b06a00640 1 -- start start 2024-09-21T07:51:09.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.241+0000 7f2b06a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b000b4db0 0x7f2b0014fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:09.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.241+0000 7f2b06a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b000d2a50 0x7f2b00155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:09.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.241+0000 7f2b06a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2b000d2e60 0x7f2b00155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:09.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.241+0000 7f2b06a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2b00169f80 con 0x7f2b000b4db0 2024-09-21T07:51:09.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.241+0000 7f2b06a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2b00169e00 con 0x7f2b000d2a50 2024-09-21T07:51:09.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.241+0000 7f2b06a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f2b0016a100 con 0x7f2b000d2e60 2024-09-21T07:51:09.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.241+0000 7f2b05600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b000b4db0 0x7f2b0014fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:09.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.241+0000 7f2b05600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b000b4db0 0x7f2b0014fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44524/0 (socket says 172.21.15.39:44524) 2024-09-21T07:51:09.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.241+0000 7f2b05600640 1 -- 172.21.15.39:0/1962447395 learned_addr learned my addr 172.21.15.39:0/1962447395 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:51:09.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.242+0000 7f2b06000640 1 --2- 172.21.15.39:0/1962447395 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2b000d2e60 0x7f2b00155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:09.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.242+0000 7f2b04c00640 1 --2- 172.21.15.39:0/1962447395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b000d2a50 0x7f2b00155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:09.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.242+0000 7f2b05600640 1 -- 172.21.15.39:0/1962447395 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2b000d2e60 msgr2=0x7f2b00155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:09.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.242+0000 7f2b05600640 1 --2- 172.21.15.39:0/1962447395 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2b000d2e60 0x7f2b00155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:09.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.242+0000 7f2b05600640 1 -- 172.21.15.39:0/1962447395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b000d2a50 msgr2=0x7f2b00155aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:09.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.242+0000 7f2b05600640 1 --2- 172.21.15.39:0/1962447395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b000d2a50 0x7f2b00155aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:09.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.242+0000 7f2b05600640 1 -- 172.21.15.39:0/1962447395 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b00153ed0 con 0x7f2b000b4db0 2024-09-21T07:51:09.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.242+0000 7f2b05600640 1 --2- 172.21.15.39:0/1962447395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b000b4db0 0x7f2b0014fc40 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7f2af402ab90 tx=0x7f2af4033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:09.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.242+0000 7f2afd600640 1 -- 172.21.15.39:0/1962447395 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af403f070 con 0x7f2b000b4db0 2024-09-21T07:51:09.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.242+0000 7f2afd600640 1 -- 172.21.15.39:0/1962447395 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2af4033e10 con 0x7f2b000b4db0 2024-09-21T07:51:09.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.242+0000 7f2b06a00640 1 -- 172.21.15.39:0/1962447395 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b00153690 con 0x7f2b000b4db0 2024-09-21T07:51:09.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.242+0000 7f2afd600640 1 -- 172.21.15.39:0/1962447395 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af4005c30 con 0x7f2b000b4db0 2024-09-21T07:51:09.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.243+0000 7f2b06a00640 1 -- 172.21.15.39:0/1962447395 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b00154420 con 0x7f2b000b4db0 2024-09-21T07:51:09.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.244+0000 7f2b06a00640 1 -- 172.21.15.39:0/1962447395 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b000d4180 con 0x7f2b000b4db0 2024-09-21T07:51:09.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.245+0000 7f2afd600640 1 -- 172.21.15.39:0/1962447395 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2af4042070 con 0x7f2b000b4db0 2024-09-21T07:51:09.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.246+0000 7f2afd600640 1 --2- 172.21.15.39:0/1962447395 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2ae0078b90 0x7f2ae007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:09.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.246+0000 7f2afd600640 1 -- 172.21.15.39:0/1962447395 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(241..241 src has 1..241) ==== 7476+0+0 (secure 0 0 0) 0x7f2af40bc370 con 0x7f2b000b4db0 2024-09-21T07:51:09.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.246+0000 7f2b04c00640 1 --2- 172.21.15.39:0/1962447395 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2ae0078b90 0x7f2ae007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:09.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.247+0000 7f2b04c00640 1 --2- 172.21.15.39:0/1962447395 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2ae0078b90 0x7f2ae007b0a0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f2aec007910 tx=0x7f2aec009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:09.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.250+0000 7f2afd600640 1 -- 172.21.15.39:0/1962447395 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2b000d4180 con 0x7f2b000b4db0 2024-09-21T07:51:09.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.492+0000 7f2b06a00640 1 -- 172.21.15.39:0/1962447395 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b000d4c90 con 0x7f2b000b4db0 2024-09-21T07:51:09.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.493+0000 7f2afd600640 1 -- 172.21.15.39:0/1962447395 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 273 v273) ==== 95+0+5280 (secure 0 0 0) 0x7f2af4084020 con 0x7f2b000b4db0 2024-09-21T07:51:09.495 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:09.495 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":273,"btime":"2024-09-21T07:51:04:941764+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10195,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":270}],"filesystems":[{"mdsmap":{"epoch":273,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:51:04.941752+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":241,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":11067},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11067":{"gid":11067,"name":"a","rank":2,"incarnation":269,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6834/109229385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":109229385},{"type":"v1","addr":"172.21.15.39:6834","nonce":109229385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[9967,11052,11067]},"id":13}]} 2024-09-21T07:51:09.495 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 273 2024-09-21T07:51:09.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.495+0000 7f2b06a00640 1 -- 172.21.15.39:0/1962447395 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2ae0078b90 msgr2=0x7f2ae007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:09.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.495+0000 7f2b06a00640 1 --2- 172.21.15.39:0/1962447395 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2ae0078b90 0x7f2ae007b0a0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f2aec007910 tx=0x7f2aec009040 comp rx=0 tx=0).stop 2024-09-21T07:51:09.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.496+0000 7f2b06a00640 1 -- 172.21.15.39:0/1962447395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b000b4db0 msgr2=0x7f2b0014fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:09.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.496+0000 7f2b06a00640 1 --2- 172.21.15.39:0/1962447395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b000b4db0 0x7f2b0014fc40 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7f2af402ab90 tx=0x7f2af4033c20 comp rx=0 tx=0).stop 2024-09-21T07:51:09.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.496+0000 7f2b06a00640 1 -- 172.21.15.39:0/1962447395 shutdown_connections 2024-09-21T07:51:09.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.496+0000 7f2b06a00640 1 --2- 172.21.15.39:0/1962447395 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2ae0078b90 0x7f2ae007b0a0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:09.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.496+0000 7f2b06a00640 1 --2- 172.21.15.39:0/1962447395 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2b000d2e60 0x7f2b00155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:09.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.496+0000 7f2b06a00640 1 --2- 172.21.15.39:0/1962447395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b000d2a50 0x7f2b00155aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:09.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.496+0000 7f2b06a00640 1 --2- 172.21.15.39:0/1962447395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b000b4db0 0x7f2b0014fc40 unknown :-1 s=CLOSED pgs=2430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:09.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.496+0000 7f2b06a00640 1 -- 172.21.15.39:0/1962447395 >> 172.21.15.39:0/1962447395 conn(0x7f2b0008bfb0 msgr2=0x7f2b0006f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:09.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.496+0000 7f2b06a00640 1 -- 172.21.15.39:0/1962447395 shutdown_connections 2024-09-21T07:51:09.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.496+0000 7f2b06a00640 1 -- 172.21.15.39:0/1962447395 wait complete. 2024-09-21T07:51:09.519 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 13:2 true 2024-09-21T07:51:09.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.746+0000 7f5116600640 1 -- 172.21.15.39:0/1718893373 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50f80039e0 msgr2=0x7f50f8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:09.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.746+0000 7f5116600640 1 --2- 172.21.15.39:0/1718893373 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50f80039e0 0x7f50f8003e10 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7f5104001740 tx=0x7f50f8005830 comp rx=0 tx=0).stop 2024-09-21T07:51:09.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.746+0000 7f5115c00640 1 -- 172.21.15.39:0/1718893373 reap_dead start 2024-09-21T07:51:09.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.746+0000 7f5116600640 1 -- 172.21.15.39:0/1718893373 shutdown_connections 2024-09-21T07:51:09.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.746+0000 7f5116600640 1 -- 172.21.15.39:0/1718893373 >> 172.21.15.39:0/1718893373 conn(0x7f51100b25f0 msgr2=0x7f51100d01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:09.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.746+0000 7f5116600640 1 -- 172.21.15.39:0/1718893373 shutdown_connections 2024-09-21T07:51:09.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.746+0000 7f5116600640 1 -- 172.21.15.39:0/1718893373 wait complete. 2024-09-21T07:51:09.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.749+0000 7f5116600640 1 Processor -- start 2024-09-21T07:51:09.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.749+0000 7f5116600640 1 -- start start 2024-09-21T07:51:09.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.750+0000 7f5116600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50f80039e0 0x7f5110072dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:09.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.750+0000 7f5116600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5110073310 0x7f5110073720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:09.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.750+0000 7f5116600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51100c7120 0x7f51100c7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:09.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.750+0000 7f5116600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f51100adbe0 con 0x7f50f80039e0 2024-09-21T07:51:09.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.750+0000 7f5116600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f51100afb50 con 0x7f51100c7120 2024-09-21T07:51:09.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.750+0000 7f5116600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f51100add60 con 0x7f5110073310 2024-09-21T07:51:09.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.750+0000 7f5115200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50f80039e0 0x7f5110072dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:09.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.750+0000 7f5115200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50f80039e0 0x7f5110072dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44542/0 (socket says 172.21.15.39:44542) 2024-09-21T07:51:09.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.750+0000 7f5115200640 1 -- 172.21.15.39:0/2890723267 learned_addr learned my addr 172.21.15.39:0/2890723267 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:51:09.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.751+0000 7f510fe00640 1 --2- 172.21.15.39:0/2890723267 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5110073310 0x7f5110073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:09.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.751+0000 7f5115c00640 1 --2- 172.21.15.39:0/2890723267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51100c7120 0x7f51100c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:09.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.751+0000 7f5115200640 1 -- 172.21.15.39:0/2890723267 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5110073310 msgr2=0x7f5110073720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:09.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.751+0000 7f5115200640 1 --2- 172.21.15.39:0/2890723267 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5110073310 0x7f5110073720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:09.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.751+0000 7f5115200640 1 -- 172.21.15.39:0/2890723267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51100c7120 msgr2=0x7f51100c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:09.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.751+0000 7f5115200640 1 --2- 172.21.15.39:0/2890723267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51100c7120 0x7f51100c7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:09.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.751+0000 7f5115200640 1 -- 172.21.15.39:0/2890723267 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51101e89f0 con 0x7f50f80039e0 2024-09-21T07:51:09.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.751+0000 7f5115200640 1 --2- 172.21.15.39:0/2890723267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50f80039e0 0x7f5110072dd0 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7f5104001740 tx=0x7f51040060e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:09.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.751+0000 7f510cc00640 1 -- 172.21.15.39:0/2890723267 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5104037040 con 0x7f50f80039e0 2024-09-21T07:51:09.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.752+0000 7f510cc00640 1 -- 172.21.15.39:0/2890723267 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5104041070 con 0x7f50f80039e0 2024-09-21T07:51:09.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.752+0000 7f5116600640 1 -- 172.21.15.39:0/2890723267 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51101e8cd0 con 0x7f50f80039e0 2024-09-21T07:51:09.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.752+0000 7f510cc00640 1 -- 172.21.15.39:0/2890723267 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5104003a30 con 0x7f50f80039e0 2024-09-21T07:51:09.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.752+0000 7f5116600640 1 -- 172.21.15.39:0/2890723267 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51100c6c00 con 0x7f50f80039e0 2024-09-21T07:51:09.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.753+0000 7f5116600640 1 -- 172.21.15.39:0/2890723267 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51100b30c0 con 0x7f50f80039e0 2024-09-21T07:51:09.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.754+0000 7f510cc00640 1 -- 172.21.15.39:0/2890723267 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5104003bd0 con 0x7f50f80039e0 2024-09-21T07:51:09.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.755+0000 7f510cc00640 1 --2- 172.21.15.39:0/2890723267 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f50f0078bc0 0x7f50f007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:09.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.756+0000 7f510cc00640 1 -- 172.21.15.39:0/2890723267 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(241..241 src has 1..241) ==== 7476+0+0 (secure 0 0 0) 0x7f51040bb900 con 0x7f50f80039e0 2024-09-21T07:51:09.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.756+0000 7f510fe00640 1 --2- 172.21.15.39:0/2890723267 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f50f0078bc0 0x7f50f007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:09.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.757+0000 7f510fe00640 1 --2- 172.21.15.39:0/2890723267 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f50f0078bc0 0x7f50f007b0d0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f5110156020 tx=0x7f50fc0067a0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:09.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:09.760+0000 7f510cc00640 1 -- 172.21.15.39:0/2890723267 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f51100b30c0 con 0x7f50f80039e0 2024-09-21T07:51:10.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:10.036+0000 7f5116600640 1 -- 172.21.15.39:0/2890723267 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "13:2", "val": "true"} v 0) -- 0x7f51100aeab0 con 0x7f50f80039e0 2024-09-21T07:51:11.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:10.998+0000 7f510cc00640 1 -- 172.21.15.39:0/2890723267 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "13:2", "val": "true"}]=0 freezing mds.11067 v274) ==== 114+0+0 (secure 0 0 0) 0x7f51040835c0 con 0x7f50f80039e0 2024-09-21T07:51:11.000 INFO:teuthology.orchestra.run.smithi039.stderr:freezing mds.11067 2024-09-21T07:51:11.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.004+0000 7f5116600640 1 -- 172.21.15.39:0/2890723267 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f50f0078bc0 msgr2=0x7f50f007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:11.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.004+0000 7f5116600640 1 --2- 172.21.15.39:0/2890723267 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f50f0078bc0 0x7f50f007b0d0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f5110156020 tx=0x7f50fc0067a0 comp rx=0 tx=0).stop 2024-09-21T07:51:11.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.004+0000 7f5116600640 1 -- 172.21.15.39:0/2890723267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50f80039e0 msgr2=0x7f5110072dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:11.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.004+0000 7f5116600640 1 --2- 172.21.15.39:0/2890723267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50f80039e0 0x7f5110072dd0 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7f5104001740 tx=0x7f51040060e0 comp rx=0 tx=0).stop 2024-09-21T07:51:11.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.005+0000 7f5116600640 1 -- 172.21.15.39:0/2890723267 shutdown_connections 2024-09-21T07:51:11.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.005+0000 7f5116600640 1 --2- 172.21.15.39:0/2890723267 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f50f0078bc0 0x7f50f007b0d0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:11.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.005+0000 7f5116600640 1 --2- 172.21.15.39:0/2890723267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51100c7120 0x7f51100c7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:11.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.005+0000 7f5116600640 1 --2- 172.21.15.39:0/2890723267 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5110073310 0x7f5110073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:11.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.005+0000 7f5116600640 1 --2- 172.21.15.39:0/2890723267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50f80039e0 0x7f5110072dd0 unknown :-1 s=CLOSED pgs=2433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:11.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.005+0000 7f5116600640 1 -- 172.21.15.39:0/2890723267 >> 172.21.15.39:0/2890723267 conn(0x7f51100b25f0 msgr2=0x7f51100aed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:11.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.005+0000 7f5116600640 1 -- 172.21.15.39:0/2890723267 shutdown_connections 2024-09-21T07:51:11.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.007+0000 7f5116600640 1 -- 172.21.15.39:0/2890723267 wait complete. 2024-09-21T07:51:11.041 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_kill_mdstable_at 4 2024-09-21T07:51:11.291 INFO:teuthology.orchestra.run.smithi039.stdout:{"success":"mds_kill_mdstable_at = '' (not observed, change may require restart) "} 2024-09-21T07:51:11.291 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_kill_mdstable_at = '' (not observed, change may require restart) " } 2024-09-21T07:51:11.291 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:51:11.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.518+0000 7fe11fe00640 1 -- 172.21.15.39:0/2158745139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1200d4b40 msgr2=0x7fe1200b1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:11.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.518+0000 7fe11fe00640 1 --2- 172.21.15.39:0/2158745139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1200d4b40 0x7fe1200b1c90 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7fe10c005fd0 tx=0x7fe10c02a5d0 comp rx=0 tx=0).stop 2024-09-21T07:51:11.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.518+0000 7fe11fe00640 1 -- 172.21.15.39:0/2158745139 shutdown_connections 2024-09-21T07:51:11.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.518+0000 7fe11fe00640 1 --2- 172.21.15.39:0/2158745139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1200d4b40 0x7fe1200b1c90 unknown :-1 s=CLOSED pgs=2434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:11.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.518+0000 7fe11fe00640 1 --2- 172.21.15.39:0/2158745139 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1200d19f0 0x7fe1200d4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:11.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.518+0000 7fe11fe00640 1 -- 172.21.15.39:0/2158745139 >> 172.21.15.39:0/2158745139 conn(0x7fe12008bfb0 msgr2=0x7fe12009d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:11.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.519+0000 7fe11fe00640 1 -- 172.21.15.39:0/2158745139 shutdown_connections 2024-09-21T07:51:11.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.519+0000 7fe11fe00640 1 -- 172.21.15.39:0/2158745139 wait complete. 2024-09-21T07:51:11.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.523+0000 7fe11fe00640 1 Processor -- start 2024-09-21T07:51:11.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.523+0000 7fe11fe00640 1 -- start start 2024-09-21T07:51:11.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.524+0000 7fe11fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1200b21d0 0x7fe1200c4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:11.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.524+0000 7fe11fe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1200d19f0 0x7fe1200c5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:11.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.524+0000 7fe11fe00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe12015cd90 0x7fe1200c5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:11.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.524+0000 7fe11fe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe1200b2e70 con 0x7fe1200b21d0 2024-09-21T07:51:11.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.524+0000 7fe11fe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe1200b2cf0 con 0x7fe1200d19f0 2024-09-21T07:51:11.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.524+0000 7fe11fe00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe1200b5890 con 0x7fe12015cd90 2024-09-21T07:51:11.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.524+0000 7fe11ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1200b21d0 0x7fe1200c4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:11.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.524+0000 7fe11ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1200b21d0 0x7fe1200c4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44558/0 (socket says 172.21.15.39:44558) 2024-09-21T07:51:11.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.524+0000 7fe11ea00640 1 -- 172.21.15.39:0/3907432 learned_addr learned my addr 172.21.15.39:0/3907432 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:51:11.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.525+0000 7fe11f400640 1 --2- 172.21.15.39:0/3907432 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe12015cd90 0x7fe1200c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:11.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.525+0000 7fe11e000640 1 --2- 172.21.15.39:0/3907432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1200d19f0 0x7fe1200c5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:11.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.525+0000 7fe11ea00640 1 -- 172.21.15.39:0/3907432 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe12015cd90 msgr2=0x7fe1200c5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:11.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.525+0000 7fe11ea00640 1 --2- 172.21.15.39:0/3907432 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe12015cd90 0x7fe1200c5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:11.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.525+0000 7fe11ea00640 1 -- 172.21.15.39:0/3907432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1200d19f0 msgr2=0x7fe1200c5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:11.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.525+0000 7fe11ea00640 1 --2- 172.21.15.39:0/3907432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1200d19f0 0x7fe1200c5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:11.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.525+0000 7fe11ea00640 1 -- 172.21.15.39:0/3907432 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe12009fe50 con 0x7fe1200b21d0 2024-09-21T07:51:11.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.525+0000 7fe11ea00640 1 --2- 172.21.15.39:0/3907432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1200b21d0 0x7fe1200c4920 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7fe11000e7f0 tx=0x7fe11000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:11.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.526+0000 7fe116a00640 1 -- 172.21.15.39:0/3907432 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11000ccc0 con 0x7fe1200b21d0 2024-09-21T07:51:11.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.526+0000 7fe116a00640 1 -- 172.21.15.39:0/3907432 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe110010430 con 0x7fe1200b21d0 2024-09-21T07:51:11.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.526+0000 7fe11fe00640 1 -- 172.21.15.39:0/3907432 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1200a0130 con 0x7fe1200b21d0 2024-09-21T07:51:11.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.526+0000 7fe116a00640 1 -- 172.21.15.39:0/3907432 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe110010840 con 0x7fe1200b21d0 2024-09-21T07:51:11.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.526+0000 7fe11fe00640 1 -- 172.21.15.39:0/3907432 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1200a0c10 con 0x7fe1200b21d0 2024-09-21T07:51:11.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.527+0000 7fe11fe00640 1 -- 172.21.15.39:0/3907432 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe120069a50 con 0x7fe1200b21d0 2024-09-21T07:51:11.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.528+0000 7fe116a00640 1 -- 172.21.15.39:0/3907432 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe1100109e0 con 0x7fe1200b21d0 2024-09-21T07:51:11.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.529+0000 7fe116a00640 1 --2- 172.21.15.39:0/3907432 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe100078b90 0x7fe10007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:11.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.529+0000 7fe116a00640 1 -- 172.21.15.39:0/3907432 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(242..242 src has 1..242) ==== 7564+0+0 (secure 0 0 0) 0x7fe11009bed0 con 0x7fe1200b21d0 2024-09-21T07:51:11.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.529+0000 7fe11e000640 1 --2- 172.21.15.39:0/3907432 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe100078b90 0x7fe10007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:11.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.530+0000 7fe11e000640 1 --2- 172.21.15.39:0/3907432 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe100078b90 0x7fe10007b0a0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fe10c033000 tx=0x7fe10c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:11.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.534+0000 7fe116a00640 1 -- 172.21.15.39:0/3907432 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe1100634e0 con 0x7fe1200b21d0 2024-09-21T07:51:11.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.770+0000 7fe11fe00640 1 -- 172.21.15.39:0/3907432 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1200b1c90 con 0x7fe1200b21d0 2024-09-21T07:51:11.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.770+0000 7fe116a00640 1 -- 172.21.15.39:0/3907432 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 274 v274) ==== 95+0+5280 (secure 0 0 0) 0x7fe110067f10 con 0x7fe1200b21d0 2024-09-21T07:51:11.772 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:11.772 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":274,"btime":"2024-09-21T07:51:10:985573+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10195,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":270}],"filesystems":[{"mdsmap":{"epoch":274,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:51:10.037820+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":241,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":11067},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11067":{"gid":11067,"name":"a","rank":2,"incarnation":269,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6834/109229385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":109229385},{"type":"v1","addr":"172.21.15.39:6834","nonce":109229385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[11067,11052,9967]},"id":13}]} 2024-09-21T07:51:11.772 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 274 2024-09-21T07:51:11.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.773+0000 7fe11fe00640 1 -- 172.21.15.39:0/3907432 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe100078b90 msgr2=0x7fe10007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:11.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.773+0000 7fe11fe00640 1 --2- 172.21.15.39:0/3907432 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe100078b90 0x7fe10007b0a0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fe10c033000 tx=0x7fe10c004730 comp rx=0 tx=0).stop 2024-09-21T07:51:11.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.773+0000 7fe11fe00640 1 -- 172.21.15.39:0/3907432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1200b21d0 msgr2=0x7fe1200c4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:11.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.773+0000 7fe11fe00640 1 --2- 172.21.15.39:0/3907432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1200b21d0 0x7fe1200c4920 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7fe11000e7f0 tx=0x7fe11000ecb0 comp rx=0 tx=0).stop 2024-09-21T07:51:11.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.773+0000 7fe11fe00640 1 -- 172.21.15.39:0/3907432 shutdown_connections 2024-09-21T07:51:11.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.773+0000 7fe11fe00640 1 --2- 172.21.15.39:0/3907432 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe100078b90 0x7fe10007b0a0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:11.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.773+0000 7fe11fe00640 1 --2- 172.21.15.39:0/3907432 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe12015cd90 0x7fe1200c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:11.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.773+0000 7fe11fe00640 1 --2- 172.21.15.39:0/3907432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1200d19f0 0x7fe1200c5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:11.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.773+0000 7fe11fe00640 1 --2- 172.21.15.39:0/3907432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1200b21d0 0x7fe1200c4920 unknown :-1 s=CLOSED pgs=2435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:11.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.773+0000 7fe11fe00640 1 -- 172.21.15.39:0/3907432 >> 172.21.15.39:0/3907432 conn(0x7fe12008bfb0 msgr2=0x7fe1200d0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:11.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.773+0000 7fe11fe00640 1 -- 172.21.15.39:0/3907432 shutdown_connections 2024-09-21T07:51:11.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:11.773+0000 7fe11fe00640 1 -- 172.21.15.39:0/3907432 wait complete. 2024-09-21T07:51:11.796 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json dump snaps --server 2024-09-21T07:51:12.054 INFO:teuthology.orchestra.run.smithi148.stdout:{"last_snap":7,"last_created":5,"last_destroyed":7,"pending_noop":[],"snaps":[],"need_to_purge":{"30":[3,5,6,7]},"pending_update":[],"pending_destroy":[]} 2024-09-21T07:51:12.054 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 5, "last_destroyed": 7, "last_snap": 7, "need_to_purge": { "30": [ 3, 5, 6, 7 ] }, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [] } 2024-09-21T07:51:12.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:12.055 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d0/d2/dir/.snap/s1) 2024-09-21T07:51:12.059 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-09-21T07:51:12.097 INFO:tasks.ceph.mds.a.smithi039.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::commit(version_t, LogSegment*)' thread 7fbba4a00640 time 2024-09-21T07:51:12.096796+0000 2024-09-21T07:51:12.097 INFO:tasks.ceph.mds.a.smithi039.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: 195: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 4) 2024-09-21T07:51:12.102 INFO:tasks.ceph.mds.a.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:51:12.102 INFO:tasks.ceph.mds.a.smithi039.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fbbb3c5654b] 2024-09-21T07:51:12.102 INFO:tasks.ceph.mds.a.smithi039.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7fbbb3c56758] 2024-09-21T07:51:12.102 INFO:tasks.ceph.mds.a.smithi039.stderr: 3: (MDSTableClient::commit(unsigned long, LogSegment*)+0x333) [0x55e88f0a13d3] 2024-09-21T07:51:12.102 INFO:tasks.ceph.mds.a.smithi039.stderr: 4: (Server::_mksnap_finish(boost::intrusive_ptr const&, CInode*, SnapInfo&)+0x9f) [0x55e88ee54ac3] 2024-09-21T07:51:12.102 INFO:tasks.ceph.mds.a.smithi039.stderr: 5: ceph-mds(+0x221794) [0x55e88ee49794] 2024-09-21T07:51:12.102 INFO:tasks.ceph.mds.a.smithi039.stderr: 6: (MDSContext::complete(int)+0x7c) [0x55e88f0ef6d8] 2024-09-21T07:51:12.103 INFO:tasks.ceph.mds.a.smithi039.stderr: 7: (MDSIOContextBase::complete(int)+0x501) [0x55e88f0efd57] 2024-09-21T07:51:12.103 INFO:tasks.ceph.mds.a.smithi039.stderr: 8: (MDSLogContextBase::complete(int)+0x46) [0x55e88f0efdd4] 2024-09-21T07:51:12.103 INFO:tasks.ceph.mds.a.smithi039.stderr: 9: (Finisher::finisher_thread_entry()+0x495) [0x7fbbb3bde34b] 2024-09-21T07:51:12.103 INFO:tasks.ceph.mds.a.smithi039.stderr: 10: ceph-mds(+0x1052e1) [0x55e88ed2d2e1] 2024-09-21T07:51:12.103 INFO:tasks.ceph.mds.a.smithi039.stderr: 11: (Thread::entry_wrapper()+0x43) [0x7fbbb3c2fd57] 2024-09-21T07:51:12.103 INFO:tasks.ceph.mds.a.smithi039.stderr: 12: (Thread::_entry_func(void*)+0xd) [0x7fbbb3c2fd73] 2024-09-21T07:51:12.103 INFO:tasks.ceph.mds.a.smithi039.stderr: 13: /lib64/libc.so.6(+0x89c52) [0x7fbbb3289c52] 2024-09-21T07:51:12.103 INFO:tasks.ceph.mds.a.smithi039.stderr: 14: /lib64/libc.so.6(+0x10ec80) [0x7fbbb330ec80] 2024-09-21T07:51:12.103 INFO:tasks.ceph.mds.a.smithi039.stderr:*** Caught signal (Aborted) ** 2024-09-21T07:51:12.103 INFO:tasks.ceph.mds.a.smithi039.stderr: in thread 7fbba4a00640 thread_name:MR_Finisher 2024-09-21T07:51:12.103 INFO:tasks.ceph.mds.a.smithi039.stderr:2024-09-21T07:51:12.096+0000 7fbba4a00640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::commit(version_t, LogSegment*)' thread 7fbba4a00640 time 2024-09-21T07:51:12.096796+0000 2024-09-21T07:51:12.103 INFO:tasks.ceph.mds.a.smithi039.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: 195: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 4) 2024-09-21T07:51:12.103 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:51:12.103 INFO:tasks.ceph.mds.a.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:51:12.103 INFO:tasks.ceph.mds.a.smithi039.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fbbb3c5654b] 2024-09-21T07:51:12.104 INFO:tasks.ceph.mds.a.smithi039.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7fbbb3c56758] 2024-09-21T07:51:12.104 INFO:tasks.ceph.mds.a.smithi039.stderr: 3: (MDSTableClient::commit(unsigned long, LogSegment*)+0x333) [0x55e88f0a13d3] 2024-09-21T07:51:12.104 INFO:tasks.ceph.mds.a.smithi039.stderr: 4: (Server::_mksnap_finish(boost::intrusive_ptr const&, CInode*, SnapInfo&)+0x9f) [0x55e88ee54ac3] 2024-09-21T07:51:12.104 INFO:tasks.ceph.mds.a.smithi039.stderr: 5: ceph-mds(+0x221794) [0x55e88ee49794] 2024-09-21T07:51:12.104 INFO:tasks.ceph.mds.a.smithi039.stderr: 6: (MDSContext::complete(int)+0x7c) [0x55e88f0ef6d8] 2024-09-21T07:51:12.104 INFO:tasks.ceph.mds.a.smithi039.stderr: 7: (MDSIOContextBase::complete(int)+0x501) [0x55e88f0efd57] 2024-09-21T07:51:12.104 INFO:tasks.ceph.mds.a.smithi039.stderr: 8: (MDSLogContextBase::complete(int)+0x46) [0x55e88f0efdd4] 2024-09-21T07:51:12.104 INFO:tasks.ceph.mds.a.smithi039.stderr: 9: (Finisher::finisher_thread_entry()+0x495) [0x7fbbb3bde34b] 2024-09-21T07:51:12.104 INFO:tasks.ceph.mds.a.smithi039.stderr: 10: ceph-mds(+0x1052e1) [0x55e88ed2d2e1] 2024-09-21T07:51:12.104 INFO:tasks.ceph.mds.a.smithi039.stderr: 11: (Thread::entry_wrapper()+0x43) [0x7fbbb3c2fd57] 2024-09-21T07:51:12.104 INFO:tasks.ceph.mds.a.smithi039.stderr: 12: (Thread::_entry_func(void*)+0xd) [0x7fbbb3c2fd73] 2024-09-21T07:51:12.104 INFO:tasks.ceph.mds.a.smithi039.stderr: 13: /lib64/libc.so.6(+0x89c52) [0x7fbbb3289c52] 2024-09-21T07:51:12.105 INFO:tasks.ceph.mds.a.smithi039.stderr: 14: /lib64/libc.so.6(+0x10ec80) [0x7fbbb330ec80] 2024-09-21T07:51:12.105 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:51:12.105 INFO:tasks.ceph.mds.a.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:51:12.105 INFO:tasks.ceph.mds.a.smithi039.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fbbb323e6f0] 2024-09-21T07:51:12.105 INFO:tasks.ceph.mds.a.smithi039.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7fbbb328b99c] 2024-09-21T07:51:12.105 INFO:tasks.ceph.mds.a.smithi039.stderr: 3: raise() 2024-09-21T07:51:12.105 INFO:tasks.ceph.mds.a.smithi039.stderr: 4: abort() 2024-09-21T07:51:12.105 INFO:tasks.ceph.mds.a.smithi039.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7fbbb3c56686] 2024-09-21T07:51:12.105 INFO:tasks.ceph.mds.a.smithi039.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7fbbb3c56758] 2024-09-21T07:51:12.105 INFO:tasks.ceph.mds.a.smithi039.stderr: 7: (MDSTableClient::commit(unsigned long, LogSegment*)+0x333) [0x55e88f0a13d3] 2024-09-21T07:51:12.105 INFO:tasks.ceph.mds.a.smithi039.stderr: 8: (Server::_mksnap_finish(boost::intrusive_ptr const&, CInode*, SnapInfo&)+0x9f) [0x55e88ee54ac3] 2024-09-21T07:51:12.105 INFO:tasks.ceph.mds.a.smithi039.stderr: 9: ceph-mds(+0x221794) [0x55e88ee49794] 2024-09-21T07:51:12.106 INFO:tasks.ceph.mds.a.smithi039.stderr: 10: (MDSContext::complete(int)+0x7c) [0x55e88f0ef6d8] 2024-09-21T07:51:12.106 INFO:tasks.ceph.mds.a.smithi039.stderr: 11: (MDSIOContextBase::complete(int)+0x501) [0x55e88f0efd57] 2024-09-21T07:51:12.106 INFO:tasks.ceph.mds.a.smithi039.stderr: 12: (MDSLogContextBase::complete(int)+0x46) [0x55e88f0efdd4] 2024-09-21T07:51:12.106 INFO:tasks.ceph.mds.a.smithi039.stderr: 13: (Finisher::finisher_thread_entry()+0x495) [0x7fbbb3bde34b] 2024-09-21T07:51:12.106 INFO:tasks.ceph.mds.a.smithi039.stderr: 14: ceph-mds(+0x1052e1) [0x55e88ed2d2e1] 2024-09-21T07:51:12.106 INFO:tasks.ceph.mds.a.smithi039.stderr: 15: (Thread::entry_wrapper()+0x43) [0x7fbbb3c2fd57] 2024-09-21T07:51:12.106 INFO:tasks.ceph.mds.a.smithi039.stderr: 16: (Thread::_entry_func(void*)+0xd) [0x7fbbb3c2fd73] 2024-09-21T07:51:12.106 INFO:tasks.ceph.mds.a.smithi039.stderr: 17: /lib64/libc.so.6(+0x89c52) [0x7fbbb3289c52] 2024-09-21T07:51:12.106 INFO:tasks.ceph.mds.a.smithi039.stderr: 18: /lib64/libc.so.6(+0x10ec80) [0x7fbbb330ec80] 2024-09-21T07:51:12.106 INFO:tasks.ceph.mds.a.smithi039.stderr:2024-09-21T07:51:12.098+0000 7fbba4a00640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:51:12.106 INFO:tasks.ceph.mds.a.smithi039.stderr: in thread 7fbba4a00640 thread_name:MR_Finisher 2024-09-21T07:51:12.106 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:51:12.106 INFO:tasks.ceph.mds.a.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:51:12.106 INFO:tasks.ceph.mds.a.smithi039.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fbbb323e6f0] 2024-09-21T07:51:12.106 INFO:tasks.ceph.mds.a.smithi039.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7fbbb328b99c] 2024-09-21T07:51:12.107 INFO:tasks.ceph.mds.a.smithi039.stderr: 3: raise() 2024-09-21T07:51:12.107 INFO:tasks.ceph.mds.a.smithi039.stderr: 4: abort() 2024-09-21T07:51:12.107 INFO:tasks.ceph.mds.a.smithi039.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7fbbb3c56686] 2024-09-21T07:51:12.107 INFO:tasks.ceph.mds.a.smithi039.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7fbbb3c56758] 2024-09-21T07:51:12.107 INFO:tasks.ceph.mds.a.smithi039.stderr: 7: (MDSTableClient::commit(unsigned long, LogSegment*)+0x333) [0x55e88f0a13d3] 2024-09-21T07:51:12.107 INFO:tasks.ceph.mds.a.smithi039.stderr: 8: (Server::_mksnap_finish(boost::intrusive_ptr const&, CInode*, SnapInfo&)+0x9f) [0x55e88ee54ac3] 2024-09-21T07:51:12.107 INFO:tasks.ceph.mds.a.smithi039.stderr: 9: ceph-mds(+0x221794) [0x55e88ee49794] 2024-09-21T07:51:12.107 INFO:tasks.ceph.mds.a.smithi039.stderr: 10: (MDSContext::complete(int)+0x7c) [0x55e88f0ef6d8] 2024-09-21T07:51:12.107 INFO:tasks.ceph.mds.a.smithi039.stderr: 11: (MDSIOContextBase::complete(int)+0x501) [0x55e88f0efd57] 2024-09-21T07:51:12.107 INFO:tasks.ceph.mds.a.smithi039.stderr: 12: (MDSLogContextBase::complete(int)+0x46) [0x55e88f0efdd4] 2024-09-21T07:51:12.107 INFO:tasks.ceph.mds.a.smithi039.stderr: 13: (Finisher::finisher_thread_entry()+0x495) [0x7fbbb3bde34b] 2024-09-21T07:51:12.107 INFO:tasks.ceph.mds.a.smithi039.stderr: 14: ceph-mds(+0x1052e1) [0x55e88ed2d2e1] 2024-09-21T07:51:12.107 INFO:tasks.ceph.mds.a.smithi039.stderr: 15: (Thread::entry_wrapper()+0x43) [0x7fbbb3c2fd57] 2024-09-21T07:51:12.108 INFO:tasks.ceph.mds.a.smithi039.stderr: 16: (Thread::_entry_func(void*)+0xd) [0x7fbbb3c2fd73] 2024-09-21T07:51:12.108 INFO:tasks.ceph.mds.a.smithi039.stderr: 17: /lib64/libc.so.6(+0x89c52) [0x7fbbb3289c52] 2024-09-21T07:51:12.108 INFO:tasks.ceph.mds.a.smithi039.stderr: 18: /lib64/libc.so.6(+0x10ec80) [0x7fbbb330ec80] 2024-09-21T07:51:12.108 INFO:tasks.ceph.mds.a.smithi039.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:51:12.108 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:51:12.112 INFO:tasks.ceph.mds.a.smithi039.stderr: -1> 2024-09-21T07:51:12.096+0000 7fbba4a00640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::commit(version_t, LogSegment*)' thread 7fbba4a00640 time 2024-09-21T07:51:12.096796+0000 2024-09-21T07:51:12.112 INFO:tasks.ceph.mds.a.smithi039.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: 195: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 4) 2024-09-21T07:51:12.112 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:51:12.112 INFO:tasks.ceph.mds.a.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:51:12.112 INFO:tasks.ceph.mds.a.smithi039.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fbbb3c5654b] 2024-09-21T07:51:12.112 INFO:tasks.ceph.mds.a.smithi039.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7fbbb3c56758] 2024-09-21T07:51:12.112 INFO:tasks.ceph.mds.a.smithi039.stderr: 3: (MDSTableClient::commit(unsigned long, LogSegment*)+0x333) [0x55e88f0a13d3] 2024-09-21T07:51:12.113 INFO:tasks.ceph.mds.a.smithi039.stderr: 4: (Server::_mksnap_finish(boost::intrusive_ptr const&, CInode*, SnapInfo&)+0x9f) [0x55e88ee54ac3] 2024-09-21T07:51:12.113 INFO:tasks.ceph.mds.a.smithi039.stderr: 5: ceph-mds(+0x221794) [0x55e88ee49794] 2024-09-21T07:51:12.113 INFO:tasks.ceph.mds.a.smithi039.stderr: 6: (MDSContext::complete(int)+0x7c) [0x55e88f0ef6d8] 2024-09-21T07:51:12.113 INFO:tasks.ceph.mds.a.smithi039.stderr: 7: (MDSIOContextBase::complete(int)+0x501) [0x55e88f0efd57] 2024-09-21T07:51:12.113 INFO:tasks.ceph.mds.a.smithi039.stderr: 8: (MDSLogContextBase::complete(int)+0x46) [0x55e88f0efdd4] 2024-09-21T07:51:12.113 INFO:tasks.ceph.mds.a.smithi039.stderr: 9: (Finisher::finisher_thread_entry()+0x495) [0x7fbbb3bde34b] 2024-09-21T07:51:12.113 INFO:tasks.ceph.mds.a.smithi039.stderr: 10: ceph-mds(+0x1052e1) [0x55e88ed2d2e1] 2024-09-21T07:51:12.113 INFO:tasks.ceph.mds.a.smithi039.stderr: 11: (Thread::entry_wrapper()+0x43) [0x7fbbb3c2fd57] 2024-09-21T07:51:12.113 INFO:tasks.ceph.mds.a.smithi039.stderr: 12: (Thread::_entry_func(void*)+0xd) [0x7fbbb3c2fd73] 2024-09-21T07:51:12.113 INFO:tasks.ceph.mds.a.smithi039.stderr: 13: /lib64/libc.so.6(+0x89c52) [0x7fbbb3289c52] 2024-09-21T07:51:12.113 INFO:tasks.ceph.mds.a.smithi039.stderr: 14: /lib64/libc.so.6(+0x10ec80) [0x7fbbb330ec80] 2024-09-21T07:51:12.113 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:51:12.113 INFO:tasks.ceph.mds.a.smithi039.stderr: 0> 2024-09-21T07:51:12.098+0000 7fbba4a00640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:51:12.113 INFO:tasks.ceph.mds.a.smithi039.stderr: in thread 7fbba4a00640 thread_name:MR_Finisher 2024-09-21T07:51:12.114 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:51:12.114 INFO:tasks.ceph.mds.a.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:51:12.114 INFO:tasks.ceph.mds.a.smithi039.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fbbb323e6f0] 2024-09-21T07:51:12.114 INFO:tasks.ceph.mds.a.smithi039.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7fbbb328b99c] 2024-09-21T07:51:12.114 INFO:tasks.ceph.mds.a.smithi039.stderr: 3: raise() 2024-09-21T07:51:12.114 INFO:tasks.ceph.mds.a.smithi039.stderr: 4: abort() 2024-09-21T07:51:12.114 INFO:tasks.ceph.mds.a.smithi039.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7fbbb3c56686] 2024-09-21T07:51:12.114 INFO:tasks.ceph.mds.a.smithi039.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7fbbb3c56758] 2024-09-21T07:51:12.114 INFO:tasks.ceph.mds.a.smithi039.stderr: 7: (MDSTableClient::commit(unsigned long, LogSegment*)+0x333) [0x55e88f0a13d3] 2024-09-21T07:51:12.114 INFO:tasks.ceph.mds.a.smithi039.stderr: 8: (Server::_mksnap_finish(boost::intrusive_ptr const&, CInode*, SnapInfo&)+0x9f) [0x55e88ee54ac3] 2024-09-21T07:51:12.114 INFO:tasks.ceph.mds.a.smithi039.stderr: 9: ceph-mds(+0x221794) [0x55e88ee49794] 2024-09-21T07:51:12.114 INFO:tasks.ceph.mds.a.smithi039.stderr: 10: (MDSContext::complete(int)+0x7c) [0x55e88f0ef6d8] 2024-09-21T07:51:12.114 INFO:tasks.ceph.mds.a.smithi039.stderr: 11: (MDSIOContextBase::complete(int)+0x501) [0x55e88f0efd57] 2024-09-21T07:51:12.115 INFO:tasks.ceph.mds.a.smithi039.stderr: 12: (MDSLogContextBase::complete(int)+0x46) [0x55e88f0efdd4] 2024-09-21T07:51:12.115 INFO:tasks.ceph.mds.a.smithi039.stderr: 13: (Finisher::finisher_thread_entry()+0x495) [0x7fbbb3bde34b] 2024-09-21T07:51:12.115 INFO:tasks.ceph.mds.a.smithi039.stderr: 14: ceph-mds(+0x1052e1) [0x55e88ed2d2e1] 2024-09-21T07:51:12.115 INFO:tasks.ceph.mds.a.smithi039.stderr: 15: (Thread::entry_wrapper()+0x43) [0x7fbbb3c2fd57] 2024-09-21T07:51:12.115 INFO:tasks.ceph.mds.a.smithi039.stderr: 16: (Thread::_entry_func(void*)+0xd) [0x7fbbb3c2fd73] 2024-09-21T07:51:12.115 INFO:tasks.ceph.mds.a.smithi039.stderr: 17: /lib64/libc.so.6(+0x89c52) [0x7fbbb3289c52] 2024-09-21T07:51:12.115 INFO:tasks.ceph.mds.a.smithi039.stderr: 18: /lib64/libc.so.6(+0x10ec80) [0x7fbbb330ec80] 2024-09-21T07:51:12.115 INFO:tasks.ceph.mds.a.smithi039.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:51:12.115 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:51:12.119 INFO:tasks.ceph.mds.a.smithi039.stderr: -1> 2024-09-21T07:51:12.096+0000 7fbba4a00640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::commit(version_t, LogSegment*)' thread 7fbba4a00640 time 2024-09-21T07:51:12.096796+0000 2024-09-21T07:51:12.119 INFO:tasks.ceph.mds.a.smithi039.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: 195: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 4) 2024-09-21T07:51:12.119 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:51:12.119 INFO:tasks.ceph.mds.a.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:51:12.119 INFO:tasks.ceph.mds.a.smithi039.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fbbb3c5654b] 2024-09-21T07:51:12.119 INFO:tasks.ceph.mds.a.smithi039.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7fbbb3c56758] 2024-09-21T07:51:12.119 INFO:tasks.ceph.mds.a.smithi039.stderr: 3: (MDSTableClient::commit(unsigned long, LogSegment*)+0x333) [0x55e88f0a13d3] 2024-09-21T07:51:12.120 INFO:tasks.ceph.mds.a.smithi039.stderr: 4: (Server::_mksnap_finish(boost::intrusive_ptr const&, CInode*, SnapInfo&)+0x9f) [0x55e88ee54ac3] 2024-09-21T07:51:12.120 INFO:tasks.ceph.mds.a.smithi039.stderr: 5: ceph-mds(+0x221794) [0x55e88ee49794] 2024-09-21T07:51:12.120 INFO:tasks.ceph.mds.a.smithi039.stderr: 6: (MDSContext::complete(int)+0x7c) [0x55e88f0ef6d8] 2024-09-21T07:51:12.120 INFO:tasks.ceph.mds.a.smithi039.stderr: 7: (MDSIOContextBase::complete(int)+0x501) [0x55e88f0efd57] 2024-09-21T07:51:12.120 INFO:tasks.ceph.mds.a.smithi039.stderr: 8: (MDSLogContextBase::complete(int)+0x46) [0x55e88f0efdd4] 2024-09-21T07:51:12.120 INFO:tasks.ceph.mds.a.smithi039.stderr: 9: (Finisher::finisher_thread_entry()+0x495) [0x7fbbb3bde34b] 2024-09-21T07:51:12.120 INFO:tasks.ceph.mds.a.smithi039.stderr: 10: ceph-mds(+0x1052e1) [0x55e88ed2d2e1] 2024-09-21T07:51:12.120 INFO:tasks.ceph.mds.a.smithi039.stderr: 11: (Thread::entry_wrapper()+0x43) [0x7fbbb3c2fd57] 2024-09-21T07:51:12.120 INFO:tasks.ceph.mds.a.smithi039.stderr: 12: (Thread::_entry_func(void*)+0xd) [0x7fbbb3c2fd73] 2024-09-21T07:51:12.120 INFO:tasks.ceph.mds.a.smithi039.stderr: 13: /lib64/libc.so.6(+0x89c52) [0x7fbbb3289c52] 2024-09-21T07:51:12.120 INFO:tasks.ceph.mds.a.smithi039.stderr: 14: /lib64/libc.so.6(+0x10ec80) [0x7fbbb330ec80] 2024-09-21T07:51:12.120 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:51:12.120 INFO:tasks.ceph.mds.a.smithi039.stderr: 0> 2024-09-21T07:51:12.098+0000 7fbba4a00640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:51:12.120 INFO:tasks.ceph.mds.a.smithi039.stderr: in thread 7fbba4a00640 thread_name:MR_Finisher 2024-09-21T07:51:12.121 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:51:12.121 INFO:tasks.ceph.mds.a.smithi039.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:51:12.121 INFO:tasks.ceph.mds.a.smithi039.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fbbb323e6f0] 2024-09-21T07:51:12.121 INFO:tasks.ceph.mds.a.smithi039.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7fbbb328b99c] 2024-09-21T07:51:12.121 INFO:tasks.ceph.mds.a.smithi039.stderr: 3: raise() 2024-09-21T07:51:12.121 INFO:tasks.ceph.mds.a.smithi039.stderr: 4: abort() 2024-09-21T07:51:12.121 INFO:tasks.ceph.mds.a.smithi039.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7fbbb3c56686] 2024-09-21T07:51:12.121 INFO:tasks.ceph.mds.a.smithi039.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7fbbb3c56758] 2024-09-21T07:51:12.121 INFO:tasks.ceph.mds.a.smithi039.stderr: 7: (MDSTableClient::commit(unsigned long, LogSegment*)+0x333) [0x55e88f0a13d3] 2024-09-21T07:51:12.121 INFO:tasks.ceph.mds.a.smithi039.stderr: 8: (Server::_mksnap_finish(boost::intrusive_ptr const&, CInode*, SnapInfo&)+0x9f) [0x55e88ee54ac3] 2024-09-21T07:51:12.121 INFO:tasks.ceph.mds.a.smithi039.stderr: 9: ceph-mds(+0x221794) [0x55e88ee49794] 2024-09-21T07:51:12.121 INFO:tasks.ceph.mds.a.smithi039.stderr: 10: (MDSContext::complete(int)+0x7c) [0x55e88f0ef6d8] 2024-09-21T07:51:12.121 INFO:tasks.ceph.mds.a.smithi039.stderr: 11: (MDSIOContextBase::complete(int)+0x501) [0x55e88f0efd57] 2024-09-21T07:51:12.121 INFO:tasks.ceph.mds.a.smithi039.stderr: 12: (MDSLogContextBase::complete(int)+0x46) [0x55e88f0efdd4] 2024-09-21T07:51:12.121 INFO:tasks.ceph.mds.a.smithi039.stderr: 13: (Finisher::finisher_thread_entry()+0x495) [0x7fbbb3bde34b] 2024-09-21T07:51:12.121 INFO:tasks.ceph.mds.a.smithi039.stderr: 14: ceph-mds(+0x1052e1) [0x55e88ed2d2e1] 2024-09-21T07:51:12.122 INFO:tasks.ceph.mds.a.smithi039.stderr: 15: (Thread::entry_wrapper()+0x43) [0x7fbbb3c2fd57] 2024-09-21T07:51:12.122 INFO:tasks.ceph.mds.a.smithi039.stderr: 16: (Thread::_entry_func(void*)+0xd) [0x7fbbb3c2fd73] 2024-09-21T07:51:12.122 INFO:tasks.ceph.mds.a.smithi039.stderr: 17: /lib64/libc.so.6(+0x89c52) [0x7fbbb3289c52] 2024-09-21T07:51:12.122 INFO:tasks.ceph.mds.a.smithi039.stderr: 18: /lib64/libc.so.6(+0x10ec80) [0x7fbbb330ec80] 2024-09-21T07:51:12.122 INFO:tasks.ceph.mds.a.smithi039.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:51:12.122 INFO:tasks.ceph.mds.a.smithi039.stderr: 2024-09-21T07:51:12.339 INFO:teuthology.orchestra.run.smithi039.stdout:{"mds_beacon_grace":"15.000000"} 2024-09-21T07:51:12.339 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-09-21T07:51:12.339 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:51:12.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.606+0000 7f1fff200640 1 -- 172.21.15.39:0/166519949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ff815a460 msgr2=0x7f1ff806fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:12.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.606+0000 7f1fff200640 1 --2- 172.21.15.39:0/166519949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ff815a460 0x7f1ff806fa30 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8072f20 tx=0x7f1fe802a580 comp rx=0 tx=0).stop 2024-09-21T07:51:12.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.606+0000 7f1fff200640 1 -- 172.21.15.39:0/166519949 shutdown_connections 2024-09-21T07:51:12.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.606+0000 7f1fff200640 1 --2- 172.21.15.39:0/166519949 >> v2:172.21.15.148:3300/0 conn(0x7f1ff80d0dd0 0x7f1ff80cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:12.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.606+0000 7f1fff200640 1 --2- 172.21.15.39:0/166519949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ff815a460 0x7f1ff806fa30 unknown :-1 s=CLOSED pgs=2437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:12.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.606+0000 7f1fff200640 1 -- 172.21.15.39:0/166519949 >> 172.21.15.39:0/166519949 conn(0x7f1ff806cfc0 msgr2=0x7f1ff806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:12.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.606+0000 7f1fff200640 1 -- 172.21.15.39:0/166519949 shutdown_connections 2024-09-21T07:51:12.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.606+0000 7f1fff200640 1 -- 172.21.15.39:0/166519949 wait complete. 2024-09-21T07:51:12.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.609+0000 7f1fff200640 1 Processor -- start 2024-09-21T07:51:12.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.609+0000 7f1fff200640 1 -- start start 2024-09-21T07:51:12.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.610+0000 7f1fff200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ff80cd780 0x7f1ff81f63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:12.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.610+0000 7f1fff200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1ff80d0dd0 0x7f1ff813fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:12.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.610+0000 7f1fff200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ff815a460 0x7f1ff8142c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:12.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.610+0000 7f1fff200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff815d560 con 0x7f1ff815a460 2024-09-21T07:51:12.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.610+0000 7f1fff200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff815d3e0 con 0x7f1ff80cd780 2024-09-21T07:51:12.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.610+0000 7f1fff200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f1ff815d6e0 con 0x7f1ff80d0dd0 2024-09-21T07:51:12.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.611+0000 7f1ffd200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ff815a460 0x7f1ff8142c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:12.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.611+0000 7f1ffd200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ff815a460 0x7f1ff8142c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43718/0 (socket says 172.21.15.39:43718) 2024-09-21T07:51:12.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.611+0000 7f1ffd200640 1 -- 172.21.15.39:0/4113714527 learned_addr learned my addr 172.21.15.39:0/4113714527 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:51:12.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.611+0000 7f1ff7e00640 1 --2- 172.21.15.39:0/4113714527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ff80cd780 0x7f1ff81f63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:12.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.611+0000 7f1ff7400640 1 --2- 172.21.15.39:0/4113714527 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1ff80d0dd0 0x7f1ff813fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:12.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.611+0000 7f1ffd200640 1 -- 172.21.15.39:0/4113714527 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1ff80d0dd0 msgr2=0x7f1ff813fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:12.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.611+0000 7f1ffd200640 1 --2- 172.21.15.39:0/4113714527 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1ff80d0dd0 0x7f1ff813fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:12.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.611+0000 7f1ffd200640 1 -- 172.21.15.39:0/4113714527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ff80cd780 msgr2=0x7f1ff81f63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:12.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.611+0000 7f1ffd200640 1 --2- 172.21.15.39:0/4113714527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ff80cd780 0x7f1ff81f63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:12.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.611+0000 7f1ffd200640 1 -- 172.21.15.39:0/4113714527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff8141150 con 0x7f1ff815a460 2024-09-21T07:51:12.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.611+0000 7f1ffd200640 1 --2- 172.21.15.39:0/4113714527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ff815a460 0x7f1ff8142c70 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f1ff000caa0 tx=0x7f1ff000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:12.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.612+0000 7f1fefe00640 1 -- 172.21.15.39:0/4113714527 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff000adf0 con 0x7f1ff815a460 2024-09-21T07:51:12.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.612+0000 7f1fefe00640 1 -- 172.21.15.39:0/4113714527 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ff000f540 con 0x7f1ff815a460 2024-09-21T07:51:12.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.612+0000 7f1fefe00640 1 -- 172.21.15.39:0/4113714527 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff000f990 con 0x7f1ff815a460 2024-09-21T07:51:12.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.612+0000 7f1fff200640 1 -- 172.21.15.39:0/4113714527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ff8141430 con 0x7f1ff815a460 2024-09-21T07:51:12.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.612+0000 7f1fff200640 1 -- 172.21.15.39:0/4113714527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ff81403a0 con 0x7f1ff815a460 2024-09-21T07:51:12.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.613+0000 7f1fff200640 1 -- 172.21.15.39:0/4113714527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ff8140b60 con 0x7f1ff815a460 2024-09-21T07:51:12.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.614+0000 7f1fefe00640 1 -- 172.21.15.39:0/4113714527 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1ff000fb30 con 0x7f1ff815a460 2024-09-21T07:51:12.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.615+0000 7f1fefe00640 1 --2- 172.21.15.39:0/4113714527 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1fd8078b90 0x7f1fd807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:12.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.615+0000 7f1fefe00640 1 -- 172.21.15.39:0/4113714527 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(243..243 src has 1..243) ==== 7564+0+0 (secure 0 0 0) 0x7f1ff009be00 con 0x7f1ff815a460 2024-09-21T07:51:12.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.616+0000 7f1ff7e00640 1 --2- 172.21.15.39:0/4113714527 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1fd8078b90 0x7f1fd807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:12.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.617+0000 7f1ff7e00640 1 --2- 172.21.15.39:0/4113714527 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1fd8078b90 0x7f1fd807b0a0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8033000 tx=0x7f1fe8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:12.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.620+0000 7f1fefe00640 1 -- 172.21.15.39:0/4113714527 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1ff0014030 con 0x7f1ff815a460 2024-09-21T07:51:12.742 INFO:tasks.ceph.mds.a.smithi039.stderr:daemon-helper: command crashed with signal 6 2024-09-21T07:51:12.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.859+0000 7f1fff200640 1 -- 172.21.15.39:0/4113714527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ff80cd340 con 0x7f1ff815a460 2024-09-21T07:51:12.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.860+0000 7f1fefe00640 1 -- 172.21.15.39:0/4113714527 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 274 v274) ==== 95+0+5280 (secure 0 0 0) 0x7f1ff000fe20 con 0x7f1ff815a460 2024-09-21T07:51:12.861 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:12.861 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":274,"btime":"2024-09-21T07:51:10:985573+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10195,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":270}],"filesystems":[{"mdsmap":{"epoch":274,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:51:10.037820+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":241,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":11067},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11067":{"gid":11067,"name":"a","rank":2,"incarnation":269,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6834/109229385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":109229385},{"type":"v1","addr":"172.21.15.39:6834","nonce":109229385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[11067,11052,9967]},"id":13}]} 2024-09-21T07:51:12.861 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 274 2024-09-21T07:51:12.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.862+0000 7f1fff200640 1 -- 172.21.15.39:0/4113714527 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1fd8078b90 msgr2=0x7f1fd807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:12.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.862+0000 7f1fff200640 1 --2- 172.21.15.39:0/4113714527 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1fd8078b90 0x7f1fd807b0a0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8033000 tx=0x7f1fe8004730 comp rx=0 tx=0).stop 2024-09-21T07:51:12.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.862+0000 7f1fff200640 1 -- 172.21.15.39:0/4113714527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ff815a460 msgr2=0x7f1ff8142c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:12.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.862+0000 7f1fff200640 1 --2- 172.21.15.39:0/4113714527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ff815a460 0x7f1ff8142c70 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f1ff000caa0 tx=0x7f1ff000cf60 comp rx=0 tx=0).stop 2024-09-21T07:51:12.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.862+0000 7f1fff200640 1 -- 172.21.15.39:0/4113714527 shutdown_connections 2024-09-21T07:51:12.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.862+0000 7f1fff200640 1 --2- 172.21.15.39:0/4113714527 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1fd8078b90 0x7f1fd807b0a0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:12.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.862+0000 7f1fff200640 1 --2- 172.21.15.39:0/4113714527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ff815a460 0x7f1ff8142c70 unknown :-1 s=CLOSED pgs=2438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:12.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.862+0000 7f1fff200640 1 --2- 172.21.15.39:0/4113714527 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1ff80d0dd0 0x7f1ff813fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:12.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.862+0000 7f1fff200640 1 --2- 172.21.15.39:0/4113714527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ff80cd780 0x7f1ff81f63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:12.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.862+0000 7f1fff200640 1 -- 172.21.15.39:0/4113714527 >> 172.21.15.39:0/4113714527 conn(0x7f1ff806cfc0 msgr2=0x7f1ff815e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:12.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.863+0000 7f1fff200640 1 -- 172.21.15.39:0/4113714527 shutdown_connections 2024-09-21T07:51:12.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:12.863+0000 7f1fff200640 1 -- 172.21.15.39:0/4113714527 wait complete. 2024-09-21T07:51:12.890 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:51:17.891 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:51:18.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.133+0000 7f9f83000640 1 -- 172.21.15.39:0/2798825143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f7c0b4db0 msgr2=0x7f9f7c0d4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:18.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.133+0000 7f9f83000640 1 --2- 172.21.15.39:0/2798825143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f7c0b4db0 0x7f9f7c0d4c90 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f9f700098e0 tx=0x7f9f7002a5d0 comp rx=0 tx=0).stop 2024-09-21T07:51:18.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.133+0000 7f9f83000640 1 -- 172.21.15.39:0/2798825143 shutdown_connections 2024-09-21T07:51:18.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.133+0000 7f9f83000640 1 --2- 172.21.15.39:0/2798825143 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f7c0d2a50 0x7f9f7c0d2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:18.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.133+0000 7f9f83000640 1 --2- 172.21.15.39:0/2798825143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f7c0b4db0 0x7f9f7c0d4c90 unknown :-1 s=CLOSED pgs=2440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:18.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.133+0000 7f9f83000640 1 -- 172.21.15.39:0/2798825143 >> 172.21.15.39:0/2798825143 conn(0x7f9f7c08bfb0 msgr2=0x7f9f7c0d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:18.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.134+0000 7f9f83000640 1 -- 172.21.15.39:0/2798825143 shutdown_connections 2024-09-21T07:51:18.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.134+0000 7f9f83000640 1 -- 172.21.15.39:0/2798825143 wait complete. 2024-09-21T07:51:18.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.136+0000 7f9f83000640 1 Processor -- start 2024-09-21T07:51:18.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.136+0000 7f9f83000640 1 -- start start 2024-09-21T07:51:18.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.138+0000 7f9f83000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f7c0b4db0 0x7f9f7c1f2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:18.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.138+0000 7f9f83000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f7c0b6000 0x7f9f7c0d1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:18.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.138+0000 7f9f83000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f7c0d2a50 0x7f9f7c1f8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:18.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.138+0000 7f9f83000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7c0cee90 con 0x7f9f7c0b6000 2024-09-21T07:51:18.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.138+0000 7f9f83000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7c0b6d30 con 0x7f9f7c0b4db0 2024-09-21T07:51:18.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.138+0000 7f9f83000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f9f7c0cf010 con 0x7f9f7c0d2a50 2024-09-21T07:51:18.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.138+0000 7f9f81200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f7c0b6000 0x7f9f7c0d1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:18.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.138+0000 7f9f81200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f7c0b6000 0x7f9f7c0d1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43728/0 (socket says 172.21.15.39:43728) 2024-09-21T07:51:18.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.138+0000 7f9f81200640 1 -- 172.21.15.39:0/785120965 learned_addr learned my addr 172.21.15.39:0/785120965 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:51:18.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.138+0000 7f9f81c00640 1 --2- 172.21.15.39:0/785120965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f7c0b4db0 0x7f9f7c1f2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:18.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.138+0000 7f9f82600640 1 --2- 172.21.15.39:0/785120965 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f7c0d2a50 0x7f9f7c1f8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:18.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.138+0000 7f9f81200640 1 -- 172.21.15.39:0/785120965 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f7c0d2a50 msgr2=0x7f9f7c1f8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:18.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.138+0000 7f9f81200640 1 --2- 172.21.15.39:0/785120965 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f7c0d2a50 0x7f9f7c1f8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:18.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.138+0000 7f9f81200640 1 -- 172.21.15.39:0/785120965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f7c0b4db0 msgr2=0x7f9f7c1f2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:18.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.138+0000 7f9f81200640 1 --2- 172.21.15.39:0/785120965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f7c0b4db0 0x7f9f7c1f2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:18.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.138+0000 7f9f81200640 1 -- 172.21.15.39:0/785120965 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f7c0cbcc0 con 0x7f9f7c0b6000 2024-09-21T07:51:18.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.138+0000 7f9f82600640 1 --2- 172.21.15.39:0/785120965 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f7c0d2a50 0x7f9f7c1f8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:51:18.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.139+0000 7f9f81200640 1 --2- 172.21.15.39:0/785120965 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f7c0b6000 0x7f9f7c0d1e00 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7f9f6800d720 tx=0x7f9f6800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:18.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.139+0000 7f9f79600640 1 -- 172.21.15.39:0/785120965 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f68017070 con 0x7f9f7c0b6000 2024-09-21T07:51:18.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.139+0000 7f9f79600640 1 -- 172.21.15.39:0/785120965 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f68004310 con 0x7f9f7c0b6000 2024-09-21T07:51:18.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.139+0000 7f9f79600640 1 -- 172.21.15.39:0/785120965 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f68009c30 con 0x7f9f7c0b6000 2024-09-21T07:51:18.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.139+0000 7f9f83000640 1 -- 172.21.15.39:0/785120965 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f7c0cbfa0 con 0x7f9f7c0b6000 2024-09-21T07:51:18.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.139+0000 7f9f83000640 1 -- 172.21.15.39:0/785120965 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f7c0cc560 con 0x7f9f7c0b6000 2024-09-21T07:51:18.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.140+0000 7f9f83000640 1 -- 172.21.15.39:0/785120965 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f7c0cba30 con 0x7f9f7c0b6000 2024-09-21T07:51:18.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.142+0000 7f9f79600640 1 -- 172.21.15.39:0/785120965 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9f680035f0 con 0x7f9f7c0b6000 2024-09-21T07:51:18.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.143+0000 7f9f79600640 1 --2- 172.21.15.39:0/785120965 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9f5c078bc0 0x7f9f5c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:18.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.143+0000 7f9f79600640 1 -- 172.21.15.39:0/785120965 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(244..244 src has 1..244) ==== 7564+0+0 (secure 0 0 0) 0x7f9f6809b4b0 con 0x7f9f7c0b6000 2024-09-21T07:51:18.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.143+0000 7f9f81c00640 1 --2- 172.21.15.39:0/785120965 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9f5c078bc0 0x7f9f5c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:18.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.144+0000 7f9f81c00640 1 --2- 172.21.15.39:0/785120965 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9f5c078bc0 0x7f9f5c07b0d0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f9f70034000 tx=0x7f9f70004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:18.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.148+0000 7f9f79600640 1 -- 172.21.15.39:0/785120965 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9f68062ad0 con 0x7f9f7c0b6000 2024-09-21T07:51:18.335 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-09-21T07:51:18.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.380+0000 7f9f83000640 1 -- 172.21.15.39:0/785120965 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f7c0d4c90 con 0x7f9f7c0b6000 2024-09-21T07:51:18.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.381+0000 7f9f79600640 1 -- 172.21.15.39:0/785120965 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 274 v274) ==== 95+0+5280 (secure 0 0 0) 0x7f9f68067500 con 0x7f9f7c0b6000 2024-09-21T07:51:18.382 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:18.382 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":274,"btime":"2024-09-21T07:51:10:985573+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10195,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":270}],"filesystems":[{"mdsmap":{"epoch":274,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:51:10.037820+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":241,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":11067},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11067":{"gid":11067,"name":"a","rank":2,"incarnation":269,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6834/109229385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":109229385},{"type":"v1","addr":"172.21.15.39:6834","nonce":109229385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[11067,11052,9967]},"id":13}]} 2024-09-21T07:51:18.382 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 274 2024-09-21T07:51:18.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.383+0000 7f9f83000640 1 -- 172.21.15.39:0/785120965 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9f5c078bc0 msgr2=0x7f9f5c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:18.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.383+0000 7f9f83000640 1 --2- 172.21.15.39:0/785120965 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9f5c078bc0 0x7f9f5c07b0d0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f9f70034000 tx=0x7f9f70004730 comp rx=0 tx=0).stop 2024-09-21T07:51:18.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.383+0000 7f9f83000640 1 -- 172.21.15.39:0/785120965 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f7c0b6000 msgr2=0x7f9f7c0d1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:18.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.383+0000 7f9f83000640 1 --2- 172.21.15.39:0/785120965 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f7c0b6000 0x7f9f7c0d1e00 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7f9f6800d720 tx=0x7f9f6800dbe0 comp rx=0 tx=0).stop 2024-09-21T07:51:18.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.383+0000 7f9f83000640 1 -- 172.21.15.39:0/785120965 shutdown_connections 2024-09-21T07:51:18.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.383+0000 7f9f83000640 1 --2- 172.21.15.39:0/785120965 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9f5c078bc0 0x7f9f5c07b0d0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:18.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.383+0000 7f9f83000640 1 --2- 172.21.15.39:0/785120965 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9f7c0d2a50 0x7f9f7c1f8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:18.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.383+0000 7f9f83000640 1 --2- 172.21.15.39:0/785120965 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f7c0b6000 0x7f9f7c0d1e00 unknown :-1 s=CLOSED pgs=2441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:18.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.383+0000 7f9f83000640 1 --2- 172.21.15.39:0/785120965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f7c0b4db0 0x7f9f7c1f2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:18.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.383+0000 7f9f83000640 1 -- 172.21.15.39:0/785120965 >> 172.21.15.39:0/785120965 conn(0x7f9f7c08bfb0 msgr2=0x7f9f7c0ca620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:18.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.383+0000 7f9f83000640 1 -- 172.21.15.39:0/785120965 shutdown_connections 2024-09-21T07:51:18.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:18.383+0000 7f9f83000640 1 -- 172.21.15.39:0/785120965 wait complete. 2024-09-21T07:51:18.410 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:51:23.411 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:51:23.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.648+0000 7fd864a00640 1 -- 172.21.15.39:0/1973908344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd86015cd30 msgr2=0x7fd8600cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:23.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.648+0000 7fd864a00640 1 --2- 172.21.15.39:0/1973908344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd86015cd30 0x7fd8600cfd10 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7fd86006de00 tx=0x7fd85002a560 comp rx=0 tx=0).stop 2024-09-21T07:51:23.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.648+0000 7fd864a00640 1 -- 172.21.15.39:0/1973908344 shutdown_connections 2024-09-21T07:51:23.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.648+0000 7fd864a00640 1 --2- 172.21.15.39:0/1973908344 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd86006fc10 0x7fd8600d0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:23.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.648+0000 7fd864a00640 1 --2- 172.21.15.39:0/1973908344 >> v2:172.21.15.148:3300/0 conn(0x7fd86006f800 0x7fd8600cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:23.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.648+0000 7fd864a00640 1 --2- 172.21.15.39:0/1973908344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd86015cd30 0x7fd8600cfd10 unknown :-1 s=CLOSED pgs=2442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:23.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.648+0000 7fd864a00640 1 -- 172.21.15.39:0/1973908344 >> 172.21.15.39:0/1973908344 conn(0x7fd86006cfc0 msgr2=0x7fd86006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:23.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.648+0000 7fd864a00640 1 -- 172.21.15.39:0/1973908344 shutdown_connections 2024-09-21T07:51:23.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.648+0000 7fd864a00640 1 -- 172.21.15.39:0/1973908344 wait complete. 2024-09-21T07:51:23.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.651+0000 7fd864a00640 1 Processor -- start 2024-09-21T07:51:23.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.651+0000 7fd864a00640 1 -- start start 2024-09-21T07:51:23.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.652+0000 7fd864a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd86006f800 0x7fd8600aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:23.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.652+0000 7fd864a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd86006fc10 0x7fd8601f8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:23.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.652+0000 7fd864a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd86015cd30 0x7fd8601e92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:23.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.652+0000 7fd864a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd860073710 con 0x7fd86006fc10 2024-09-21T07:51:23.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.652+0000 7fd864a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd8600d0f60 con 0x7fd86015cd30 2024-09-21T07:51:23.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.652+0000 7fd864a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd860073890 con 0x7fd86006f800 2024-09-21T07:51:23.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.653+0000 7fd857e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd86006fc10 0x7fd8601f8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:23.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.653+0000 7fd85de00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd86006f800 0x7fd8600aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:23.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.653+0000 7fd857e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd86006fc10 0x7fd8601f8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48454/0 (socket says 172.21.15.39:48454) 2024-09-21T07:51:23.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.653+0000 7fd857e00640 1 -- 172.21.15.39:0/2018475593 learned_addr learned my addr 172.21.15.39:0/2018475593 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:51:23.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.653+0000 7fd85e800640 1 --2- 172.21.15.39:0/2018475593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd86015cd30 0x7fd8601e92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:23.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.653+0000 7fd857e00640 1 -- 172.21.15.39:0/2018475593 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd86006f800 msgr2=0x7fd8600aeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:23.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.653+0000 7fd857e00640 1 --2- 172.21.15.39:0/2018475593 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd86006f800 0x7fd8600aeb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:23.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.653+0000 7fd857e00640 1 -- 172.21.15.39:0/2018475593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd86015cd30 msgr2=0x7fd8601e92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:23.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.653+0000 7fd857e00640 1 --2- 172.21.15.39:0/2018475593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd86015cd30 0x7fd8601e92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:23.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.653+0000 7fd857e00640 1 -- 172.21.15.39:0/2018475593 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8601532f0 con 0x7fd86006fc10 2024-09-21T07:51:23.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.653+0000 7fd85de00640 1 --2- 172.21.15.39:0/2018475593 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd86006f800 0x7fd8600aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:51:23.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.654+0000 7fd857e00640 1 --2- 172.21.15.39:0/2018475593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd86006fc10 0x7fd8601f8d10 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7fd84800d6d0 tx=0x7fd84800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:23.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.654+0000 7fd856000640 1 -- 172.21.15.39:0/2018475593 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd848017070 con 0x7fd86006fc10 2024-09-21T07:51:23.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.654+0000 7fd856000640 1 -- 172.21.15.39:0/2018475593 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd848004310 con 0x7fd86006fc10 2024-09-21T07:51:23.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.654+0000 7fd864a00640 1 -- 172.21.15.39:0/2018475593 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8601f98f0 con 0x7fd86006fc10 2024-09-21T07:51:23.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.654+0000 7fd856000640 1 -- 172.21.15.39:0/2018475593 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd848009a80 con 0x7fd86006fc10 2024-09-21T07:51:23.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.654+0000 7fd864a00640 1 -- 172.21.15.39:0/2018475593 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd860154070 con 0x7fd86006fc10 2024-09-21T07:51:23.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.655+0000 7fd864a00640 1 -- 172.21.15.39:0/2018475593 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8600af9b0 con 0x7fd86006fc10 2024-09-21T07:51:23.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.656+0000 7fd856000640 1 -- 172.21.15.39:0/2018475593 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd848009c20 con 0x7fd86006fc10 2024-09-21T07:51:23.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.657+0000 7fd856000640 1 --2- 172.21.15.39:0/2018475593 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd83c078b90 0x7fd83c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:23.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.657+0000 7fd856000640 1 -- 172.21.15.39:0/2018475593 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(245..245 src has 1..245) ==== 7520+0+0 (secure 0 0 0) 0x7fd84809b4b0 con 0x7fd86006fc10 2024-09-21T07:51:23.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.658+0000 7fd85de00640 1 --2- 172.21.15.39:0/2018475593 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd83c078b90 0x7fd83c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:23.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.659+0000 7fd85de00640 1 --2- 172.21.15.39:0/2018475593 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd83c078b90 0x7fd83c07b0a0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fd850033000 tx=0x7fd850004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:23.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.662+0000 7fd856000640 1 -- 172.21.15.39:0/2018475593 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd8600af9b0 con 0x7fd86006fc10 2024-09-21T07:51:23.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.894+0000 7fd864a00640 1 -- 172.21.15.39:0/2018475593 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd860073db0 con 0x7fd86006fc10 2024-09-21T07:51:23.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.894+0000 7fd856000640 1 -- 172.21.15.39:0/2018475593 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 274 v274) ==== 95+0+5280 (secure 0 0 0) 0x7fd8601532f0 con 0x7fd86006fc10 2024-09-21T07:51:23.898 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:23.898 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":274,"btime":"2024-09-21T07:51:10:985573+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10195,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":270}],"filesystems":[{"mdsmap":{"epoch":274,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:51:10.037820+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":241,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":11067},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11067":{"gid":11067,"name":"a","rank":2,"incarnation":269,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6834/109229385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":109229385},{"type":"v1","addr":"172.21.15.39:6834","nonce":109229385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[11067,11052,9967]},"id":13}]} 2024-09-21T07:51:23.899 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 274 2024-09-21T07:51:23.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.897+0000 7fd864a00640 1 -- 172.21.15.39:0/2018475593 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd83c078b90 msgr2=0x7fd83c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:23.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.897+0000 7fd864a00640 1 --2- 172.21.15.39:0/2018475593 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd83c078b90 0x7fd83c07b0a0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fd850033000 tx=0x7fd850004730 comp rx=0 tx=0).stop 2024-09-21T07:51:23.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.897+0000 7fd864a00640 1 -- 172.21.15.39:0/2018475593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd86006fc10 msgr2=0x7fd8601f8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:23.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.897+0000 7fd864a00640 1 --2- 172.21.15.39:0/2018475593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd86006fc10 0x7fd8601f8d10 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7fd84800d6d0 tx=0x7fd84800db90 comp rx=0 tx=0).stop 2024-09-21T07:51:23.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.897+0000 7fd864a00640 1 -- 172.21.15.39:0/2018475593 shutdown_connections 2024-09-21T07:51:23.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.897+0000 7fd864a00640 1 --2- 172.21.15.39:0/2018475593 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd83c078b90 0x7fd83c07b0a0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:23.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.897+0000 7fd864a00640 1 --2- 172.21.15.39:0/2018475593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd86015cd30 0x7fd8601e92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:23.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.897+0000 7fd864a00640 1 --2- 172.21.15.39:0/2018475593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd86006fc10 0x7fd8601f8d10 unknown :-1 s=CLOSED pgs=2443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:23.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.897+0000 7fd864a00640 1 --2- 172.21.15.39:0/2018475593 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd86006f800 0x7fd8600aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:23.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.897+0000 7fd864a00640 1 -- 172.21.15.39:0/2018475593 >> 172.21.15.39:0/2018475593 conn(0x7fd86006cfc0 msgr2=0x7fd86015de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:23.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.897+0000 7fd864a00640 1 -- 172.21.15.39:0/2018475593 shutdown_connections 2024-09-21T07:51:23.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:23.897+0000 7fd864a00640 1 -- 172.21.15.39:0/2018475593 wait complete. 2024-09-21T07:51:23.922 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:51:24.946 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~7s 2024-09-21T07:51:28.923 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:51:29.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.160+0000 7f6a97200640 1 -- 172.21.15.39:0/880864966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a7c005250 msgr2=0x7f6a7c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:29.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.160+0000 7f6a97200640 1 --2- 172.21.15.39:0/880864966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a7c005250 0x7f6a7c00b9c0 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7f6a9000c610 tx=0x7f6a7c005830 comp rx=0 tx=0).stop 2024-09-21T07:51:29.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.160+0000 7f6a97200640 1 -- 172.21.15.39:0/880864966 shutdown_connections 2024-09-21T07:51:29.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.160+0000 7f6a97200640 1 --2- 172.21.15.39:0/880864966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a7c005250 0x7f6a7c00b9c0 unknown :-1 s=CLOSED pgs=2445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:29.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.160+0000 7f6a97200640 1 -- 172.21.15.39:0/880864966 >> 172.21.15.39:0/880864966 conn(0x7f6a980b2510 msgr2=0x7f6a980d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:29.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.160+0000 7f6a97200640 1 -- 172.21.15.39:0/880864966 shutdown_connections 2024-09-21T07:51:29.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.160+0000 7f6a97200640 1 -- 172.21.15.39:0/880864966 wait complete. 2024-09-21T07:51:29.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.162+0000 7f6a97200640 1 Processor -- start 2024-09-21T07:51:29.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.162+0000 7f6a97200640 1 -- start start 2024-09-21T07:51:29.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.164+0000 7f6a97200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a7c005250 0x7f6a980b0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:29.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.164+0000 7f6a97200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a981f2850 0x7f6a981f2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:29.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.164+0000 7f6a97200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a981f31a0 0x7f6a9814a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:29.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.164+0000 7f6a97200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6a980cb960 con 0x7f6a981f31a0 2024-09-21T07:51:29.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.164+0000 7f6a97200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9806fbc0 con 0x7f6a981f2850 2024-09-21T07:51:29.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.164+0000 7f6a97200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f6a980cbae0 con 0x7f6a7c005250 2024-09-21T07:51:29.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.164+0000 7f6a96800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a981f31a0 0x7f6a9814a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:29.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.164+0000 7f6a96800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a981f31a0 0x7f6a9814a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48476/0 (socket says 172.21.15.39:48476) 2024-09-21T07:51:29.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.164+0000 7f6a96800640 1 -- 172.21.15.39:0/2481438260 learned_addr learned my addr 172.21.15.39:0/2481438260 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:51:29.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.164+0000 7f6a95e00640 1 --2- 172.21.15.39:0/2481438260 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a7c005250 0x7f6a980b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:29.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.164+0000 7f6a95400640 1 --2- 172.21.15.39:0/2481438260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a981f2850 0x7f6a981f2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:29.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.165+0000 7f6a96800640 1 -- 172.21.15.39:0/2481438260 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a7c005250 msgr2=0x7f6a980b0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:29.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.165+0000 7f6a96800640 1 --2- 172.21.15.39:0/2481438260 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a7c005250 0x7f6a980b0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:29.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.165+0000 7f6a96800640 1 -- 172.21.15.39:0/2481438260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a981f2850 msgr2=0x7f6a981f2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:29.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.165+0000 7f6a96800640 1 --2- 172.21.15.39:0/2481438260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a981f2850 0x7f6a981f2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:29.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.165+0000 7f6a96800640 1 -- 172.21.15.39:0/2481438260 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a9814a0f0 con 0x7f6a981f31a0 2024-09-21T07:51:29.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.165+0000 7f6a96800640 1 --2- 172.21.15.39:0/2481438260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a981f31a0 0x7f6a9814a8b0 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7f6a90001330 tx=0x7f6a9000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:29.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.165+0000 7f6a8e000640 1 -- 172.21.15.39:0/2481438260 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a90039070 con 0x7f6a981f31a0 2024-09-21T07:51:29.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.165+0000 7f6a8e000640 1 -- 172.21.15.39:0/2481438260 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a9002bda0 con 0x7f6a981f31a0 2024-09-21T07:51:29.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.165+0000 7f6a97200640 1 -- 172.21.15.39:0/2481438260 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a9815e830 con 0x7f6a981f31a0 2024-09-21T07:51:29.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.165+0000 7f6a8e000640 1 -- 172.21.15.39:0/2481438260 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a90006280 con 0x7f6a981f31a0 2024-09-21T07:51:29.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.165+0000 7f6a97200640 1 -- 172.21.15.39:0/2481438260 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a9815edf0 con 0x7f6a981f31a0 2024-09-21T07:51:29.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.166+0000 7f6a97200640 1 -- 172.21.15.39:0/2481438260 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a9815e5a0 con 0x7f6a981f31a0 2024-09-21T07:51:29.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.168+0000 7f6a8e000640 1 -- 172.21.15.39:0/2481438260 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6a90006420 con 0x7f6a981f31a0 2024-09-21T07:51:29.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.168+0000 7f6a8e000640 1 --2- 172.21.15.39:0/2481438260 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6a78078bc0 0x7f6a7807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:29.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.169+0000 7f6a8e000640 1 -- 172.21.15.39:0/2481438260 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(245..245 src has 1..245) ==== 7520+0+0 (secure 0 0 0) 0x7f6a900bafc0 con 0x7f6a981f31a0 2024-09-21T07:51:29.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.169+0000 7f6a95e00640 1 --2- 172.21.15.39:0/2481438260 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6a78078bc0 0x7f6a7807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:29.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.170+0000 7f6a95e00640 1 --2- 172.21.15.39:0/2481438260 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6a78078bc0 0x7f6a7807b0d0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f6a88006680 tx=0x7f6a88006b70 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:29.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.174+0000 7f6a8e000640 1 -- 172.21.15.39:0/2481438260 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6a90082d40 con 0x7f6a981f31a0 2024-09-21T07:51:29.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.406+0000 7f6a97200640 1 -- 172.21.15.39:0/2481438260 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a980694e0 con 0x7f6a981f31a0 2024-09-21T07:51:29.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.407+0000 7f6a8e000640 1 -- 172.21.15.39:0/2481438260 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 274 v274) ==== 95+0+5280 (secure 0 0 0) 0x7f6a980694e0 con 0x7f6a981f31a0 2024-09-21T07:51:29.409 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:29.409 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":274,"btime":"2024-09-21T07:51:10:985573+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10195,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":270}],"filesystems":[{"mdsmap":{"epoch":274,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:51:10.037820+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":241,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":11067},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11067":{"gid":11067,"name":"a","rank":2,"incarnation":269,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6834/109229385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":109229385},{"type":"v1","addr":"172.21.15.39:6834","nonce":109229385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[11067,11052,9967]},"id":13}]} 2024-09-21T07:51:29.409 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 274 2024-09-21T07:51:29.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.409+0000 7f6a97200640 1 -- 172.21.15.39:0/2481438260 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6a78078bc0 msgr2=0x7f6a7807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:29.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.409+0000 7f6a97200640 1 --2- 172.21.15.39:0/2481438260 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6a78078bc0 0x7f6a7807b0d0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f6a88006680 tx=0x7f6a88006b70 comp rx=0 tx=0).stop 2024-09-21T07:51:29.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.409+0000 7f6a97200640 1 -- 172.21.15.39:0/2481438260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a981f31a0 msgr2=0x7f6a9814a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:29.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.409+0000 7f6a97200640 1 --2- 172.21.15.39:0/2481438260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a981f31a0 0x7f6a9814a8b0 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7f6a90001330 tx=0x7f6a9000c2d0 comp rx=0 tx=0).stop 2024-09-21T07:51:29.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.409+0000 7f6a97200640 1 -- 172.21.15.39:0/2481438260 shutdown_connections 2024-09-21T07:51:29.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.409+0000 7f6a97200640 1 --2- 172.21.15.39:0/2481438260 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6a78078bc0 0x7f6a7807b0d0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:29.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.409+0000 7f6a97200640 1 --2- 172.21.15.39:0/2481438260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a981f31a0 0x7f6a9814a8b0 unknown :-1 s=CLOSED pgs=2446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:29.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.409+0000 7f6a97200640 1 --2- 172.21.15.39:0/2481438260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a981f2850 0x7f6a981f2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:29.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.409+0000 7f6a97200640 1 --2- 172.21.15.39:0/2481438260 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a7c005250 0x7f6a980b0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:29.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.409+0000 7f6a97200640 1 -- 172.21.15.39:0/2481438260 >> 172.21.15.39:0/2481438260 conn(0x7f6a980b2510 msgr2=0x7f6a980ad0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:29.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.409+0000 7f6a97200640 1 -- 172.21.15.39:0/2481438260 shutdown_connections 2024-09-21T07:51:29.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:29.409+0000 7f6a97200640 1 -- 172.21.15.39:0/2481438260 wait complete. 2024-09-21T07:51:29.434 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:51:31.558 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~13s 2024-09-21T07:51:33.621 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:51:33.626 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:51:34.435 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:51:34.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.672+0000 7fe460400640 1 -- 172.21.15.39:0/1766796660 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe440004d40 msgr2=0x7fe440005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:34.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.672+0000 7fe460400640 1 --2- 172.21.15.39:0/1766796660 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe440004d40 0x7fe440005150 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7fe450009ed0 tx=0x7fe440005830 comp rx=0 tx=0).stop 2024-09-21T07:51:34.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.672+0000 7fe45e400640 1 -- 172.21.15.39:0/1766796660 reap_dead start 2024-09-21T07:51:34.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.672+0000 7fe460400640 1 -- 172.21.15.39:0/1766796660 shutdown_connections 2024-09-21T07:51:34.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.672+0000 7fe460400640 1 -- 172.21.15.39:0/1766796660 >> 172.21.15.39:0/1766796660 conn(0x7fe45806cfc0 msgr2=0x7fe45806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:34.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.672+0000 7fe460400640 1 -- 172.21.15.39:0/1766796660 shutdown_connections 2024-09-21T07:51:34.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.672+0000 7fe460400640 1 -- 172.21.15.39:0/1766796660 wait complete. 2024-09-21T07:51:34.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.675+0000 7fe460400640 1 Processor -- start 2024-09-21T07:51:34.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.675+0000 7fe460400640 1 -- start start 2024-09-21T07:51:34.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.676+0000 7fe460400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe440004d40 0x7fe458148e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:34.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.676+0000 7fe460400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe4581568b0 0x7fe458154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:34.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.676+0000 7fe460400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe458156cc0 0x7fe458154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:34.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.676+0000 7fe460400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe4580d0ce0 con 0x7fe458156cc0 2024-09-21T07:51:34.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.676+0000 7fe460400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe4580d0b60 con 0x7fe440004d40 2024-09-21T07:51:34.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.676+0000 7fe460400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe4580d0e60 con 0x7fe4581568b0 2024-09-21T07:51:34.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.676+0000 7fe45e400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe458156cc0 0x7fe458154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:34.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.677+0000 7fe45e400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe458156cc0 0x7fe458154ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39740/0 (socket says 172.21.15.39:39740) 2024-09-21T07:51:34.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.677+0000 7fe45e400640 1 -- 172.21.15.39:0/2193005870 learned_addr learned my addr 172.21.15.39:0/2193005870 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:51:34.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.677+0000 7fe45d000640 1 --2- 172.21.15.39:0/2193005870 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe4581568b0 0x7fe458154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:34.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.677+0000 7fe45da00640 1 --2- 172.21.15.39:0/2193005870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe440004d40 0x7fe458148e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:34.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.677+0000 7fe45e400640 1 -- 172.21.15.39:0/2193005870 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe4581568b0 msgr2=0x7fe458154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:34.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.677+0000 7fe45e400640 1 --2- 172.21.15.39:0/2193005870 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe4581568b0 0x7fe458154960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:34.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.677+0000 7fe45e400640 1 -- 172.21.15.39:0/2193005870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe440004d40 msgr2=0x7fe458148e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:34.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.677+0000 7fe45e400640 1 --2- 172.21.15.39:0/2193005870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe440004d40 0x7fe458148e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:34.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.677+0000 7fe45e400640 1 -- 172.21.15.39:0/2193005870 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4581f5590 con 0x7fe458156cc0 2024-09-21T07:51:34.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.677+0000 7fe45e400640 1 --2- 172.21.15.39:0/2193005870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe458156cc0 0x7fe458154ea0 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7fe45002af20 tx=0x7fe450033dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:34.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.677+0000 7fe455600640 1 -- 172.21.15.39:0/2193005870 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe450042070 con 0x7fe458156cc0 2024-09-21T07:51:34.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.678+0000 7fe455600640 1 -- 172.21.15.39:0/2193005870 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe450003c60 con 0x7fe458156cc0 2024-09-21T07:51:34.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.678+0000 7fe455600640 1 -- 172.21.15.39:0/2193005870 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe450038040 con 0x7fe458156cc0 2024-09-21T07:51:34.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.678+0000 7fe460400640 1 -- 172.21.15.39:0/2193005870 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4581f5870 con 0x7fe458156cc0 2024-09-21T07:51:34.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.678+0000 7fe460400640 1 -- 172.21.15.39:0/2193005870 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4581f5e30 con 0x7fe458156cc0 2024-09-21T07:51:34.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.679+0000 7fe460400640 1 -- 172.21.15.39:0/2193005870 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4581f64f0 con 0x7fe458156cc0 2024-09-21T07:51:34.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.680+0000 7fe455600640 1 -- 172.21.15.39:0/2193005870 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe450004290 con 0x7fe458156cc0 2024-09-21T07:51:34.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.681+0000 7fe455600640 1 --2- 172.21.15.39:0/2193005870 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe438078b90 0x7fe43807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:34.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.681+0000 7fe455600640 1 -- 172.21.15.39:0/2193005870 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(245..245 src has 1..245) ==== 7520+0+0 (secure 0 0 0) 0x7fe4500bba20 con 0x7fe458156cc0 2024-09-21T07:51:34.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.681+0000 7fe45da00640 1 --2- 172.21.15.39:0/2193005870 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe438078b90 0x7fe43807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:34.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.682+0000 7fe45da00640 1 --2- 172.21.15.39:0/2193005870 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe438078b90 0x7fe43807b0a0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fe44c009aa0 tx=0x7fe44c005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:34.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.686+0000 7fe455600640 1 -- 172.21.15.39:0/2193005870 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe4581f64f0 con 0x7fe458156cc0 2024-09-21T07:51:34.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.921+0000 7fe460400640 1 -- 172.21.15.39:0/2193005870 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4580d0620 con 0x7fe458156cc0 2024-09-21T07:51:34.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.922+0000 7fe455600640 1 -- 172.21.15.39:0/2193005870 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 275 v275) ==== 95+0+5328 (secure 0 0 0) 0x7fe450083080 con 0x7fe458156cc0 2024-09-21T07:51:34.924 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:34.924 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":275,"btime":"2024-09-21T07:51:31:572749+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10195,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":270}],"filesystems":[{"mdsmap":{"epoch":275,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:51:31.572717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":241,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":11067},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11067":{"gid":11067,"name":"a","rank":2,"incarnation":269,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6834/109229385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":109229385},{"type":"v1","addr":"172.21.15.39:6834","nonce":109229385}]},"join_fscid":-1,"laggy_since":"2024-09-21T07:51:31.572717+0000","export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[9967,11052,11067]},"id":13}]} 2024-09-21T07:51:34.924 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 275 2024-09-21T07:51:34.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.924+0000 7fe460400640 1 -- 172.21.15.39:0/2193005870 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe438078b90 msgr2=0x7fe43807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:34.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.924+0000 7fe460400640 1 --2- 172.21.15.39:0/2193005870 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe438078b90 0x7fe43807b0a0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fe44c009aa0 tx=0x7fe44c005f90 comp rx=0 tx=0).stop 2024-09-21T07:51:34.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.924+0000 7fe460400640 1 -- 172.21.15.39:0/2193005870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe458156cc0 msgr2=0x7fe458154ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:34.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.924+0000 7fe460400640 1 --2- 172.21.15.39:0/2193005870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe458156cc0 0x7fe458154ea0 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7fe45002af20 tx=0x7fe450033dc0 comp rx=0 tx=0).stop 2024-09-21T07:51:34.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.924+0000 7fe460400640 1 -- 172.21.15.39:0/2193005870 shutdown_connections 2024-09-21T07:51:34.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.924+0000 7fe460400640 1 --2- 172.21.15.39:0/2193005870 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe438078b90 0x7fe43807b0a0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:34.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.924+0000 7fe460400640 1 --2- 172.21.15.39:0/2193005870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe458156cc0 0x7fe458154ea0 unknown :-1 s=CLOSED pgs=2449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:34.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.924+0000 7fe460400640 1 --2- 172.21.15.39:0/2193005870 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe4581568b0 0x7fe458154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:34.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.924+0000 7fe460400640 1 --2- 172.21.15.39:0/2193005870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe440004d40 0x7fe458148e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:34.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.925+0000 7fe460400640 1 -- 172.21.15.39:0/2193005870 >> 172.21.15.39:0/2193005870 conn(0x7fe45806cfc0 msgr2=0x7fe4580ad890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:34.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.925+0000 7fe460400640 1 -- 172.21.15.39:0/2193005870 shutdown_connections 2024-09-21T07:51:34.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:34.925+0000 7fe460400640 1 -- 172.21.15.39:0/2193005870 wait complete. 2024-09-21T07:51:34.953 DEBUG:tasks.ceph_test_case:wait_until_true: success in 20s and 0 retries 2024-09-21T07:51:34.953 DEBUG:teuthology.orchestra.run.smithi039:> sudo sysctl -n kernel.core_pattern 2024-09-21T07:51:35.005 INFO:teuthology.orchestra.run.smithi039.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T07:51:35.008 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2024-09-21T07:51:35.009 DEBUG:teuthology.orchestra.run.smithi039:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2024-09-21T07:51:35.071 INFO:teuthology.orchestra.run.smithi039.stdout:1726905072.100124.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i a', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/proc/self/exe', platform: 'x86_64' 2024-09-21T07:51:35.075 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1726905072.100124.core'] 2024-09-21T07:51:35.075 INFO:tasks.cephfs.cephfs_test_case:Found core file 1726905072.100124.core, deleting it 2024-09-21T07:51:35.075 DEBUG:teuthology.orchestra.run.smithi039:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1726905072.100124.core 2024-09-21T07:51:35.171 INFO:tasks.cephfs.mount:Suspending the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:51:35.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:35.171 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 down 2024-09-21T07:51:35.236 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:51:35.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.461+0000 7fe2c8200640 1 -- 172.21.15.39:0/4228707454 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2c0151590 msgr2=0x7fe2c01519a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:35.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.461+0000 7fe2c8200640 1 --2- 172.21.15.39:0/4228707454 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2c0151590 0x7fe2c01519a0 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7fe2c00d1390 tx=0x7fe2b802a670 comp rx=0 tx=0).stop 2024-09-21T07:51:35.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.461+0000 7fe2c8200640 1 -- 172.21.15.39:0/4228707454 shutdown_connections 2024-09-21T07:51:35.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.461+0000 7fe2c8200640 1 --2- 172.21.15.39:0/4228707454 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2c0151590 0x7fe2c01519a0 unknown :-1 s=CLOSED pgs=2450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:35.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.461+0000 7fe2c8200640 1 --2- 172.21.15.39:0/4228707454 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe2c00b3370 0x7fe2c00b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:35.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.461+0000 7fe2c8200640 1 --2- 172.21.15.39:0/4228707454 >> v2:172.21.15.148:3300/0 conn(0x7fe2c015e590 0x7fe2c0070a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:35.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.461+0000 7fe2c8200640 1 -- 172.21.15.39:0/4228707454 >> 172.21.15.39:0/4228707454 conn(0x7fe2c006cfc0 msgr2=0x7fe2c006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:35.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.461+0000 7fe2c8200640 1 -- 172.21.15.39:0/4228707454 shutdown_connections 2024-09-21T07:51:35.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.461+0000 7fe2c8200640 1 -- 172.21.15.39:0/4228707454 wait complete. 2024-09-21T07:51:35.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.463+0000 7fe2c8200640 1 Processor -- start 2024-09-21T07:51:35.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.464+0000 7fe2c8200640 1 -- start start 2024-09-21T07:51:35.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.465+0000 7fe2c8200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2c00b3370 0x7fe2c01583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:35.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.465+0000 7fe2c8200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe2c0151590 0x7fe2c01f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:35.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.465+0000 7fe2c8200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2c015e590 0x7fe2c01f6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:35.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.465+0000 7fe2c8200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c015b750 con 0x7fe2c015e590 2024-09-21T07:51:35.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.465+0000 7fe2c8200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c015b5d0 con 0x7fe2c00b3370 2024-09-21T07:51:35.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.465+0000 7fe2c8200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe2c015b8d0 con 0x7fe2c0151590 2024-09-21T07:51:35.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.465+0000 7fe2c6200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2c015e590 0x7fe2c01f6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:35.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.465+0000 7fe2c6200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2c015e590 0x7fe2c01f6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39758/0 (socket says 172.21.15.39:39758) 2024-09-21T07:51:35.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.465+0000 7fe2c6200640 1 -- 172.21.15.39:0/484874666 learned_addr learned my addr 172.21.15.39:0/484874666 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:51:35.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.466+0000 7fe2c5800640 1 --2- 172.21.15.39:0/484874666 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2c00b3370 0x7fe2c01583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:35.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.466+0000 7fe2c4e00640 1 --2- 172.21.15.39:0/484874666 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe2c0151590 0x7fe2c01f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:35.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.466+0000 7fe2c6200640 1 -- 172.21.15.39:0/484874666 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe2c0151590 msgr2=0x7fe2c01f5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:35.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.466+0000 7fe2c6200640 1 --2- 172.21.15.39:0/484874666 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe2c0151590 0x7fe2c01f5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:35.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.466+0000 7fe2c6200640 1 -- 172.21.15.39:0/484874666 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2c00b3370 msgr2=0x7fe2c01583f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:35.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.466+0000 7fe2c6200640 1 --2- 172.21.15.39:0/484874666 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2c00b3370 0x7fe2c01583f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:35.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.466+0000 7fe2c6200640 1 -- 172.21.15.39:0/484874666 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c0152630 con 0x7fe2c015e590 2024-09-21T07:51:35.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.466+0000 7fe2c6200640 1 --2- 172.21.15.39:0/484874666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2c015e590 0x7fe2c01f6200 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7fe2b802ac60 tx=0x7fe2b8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:35.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.466+0000 7fe2bd600640 1 -- 172.21.15.39:0/484874666 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8004450 con 0x7fe2c015e590 2024-09-21T07:51:35.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.466+0000 7fe2bd600640 1 -- 172.21.15.39:0/484874666 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe2b80046e0 con 0x7fe2c015e590 2024-09-21T07:51:35.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.466+0000 7fe2bd600640 1 -- 172.21.15.39:0/484874666 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b803b670 con 0x7fe2c015e590 2024-09-21T07:51:35.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.466+0000 7fe2c8200640 1 -- 172.21.15.39:0/484874666 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2c0152910 con 0x7fe2c015e590 2024-09-21T07:51:35.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.467+0000 7fe2c8200640 1 -- 172.21.15.39:0/484874666 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2c00ae600 con 0x7fe2c015e590 2024-09-21T07:51:35.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.467+0000 7fe2c8200640 1 -- 172.21.15.39:0/484874666 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2c015bdf0 con 0x7fe2c015e590 2024-09-21T07:51:35.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.469+0000 7fe2bd600640 1 -- 172.21.15.39:0/484874666 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe2b803b810 con 0x7fe2c015e590 2024-09-21T07:51:35.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.470+0000 7fe2bd600640 1 --2- 172.21.15.39:0/484874666 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe2a0078bc0 0x7fe2a007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:35.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.470+0000 7fe2bd600640 1 -- 172.21.15.39:0/484874666 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(245..245 src has 1..245) ==== 7520+0+0 (secure 0 0 0) 0x7fe2b80bca90 con 0x7fe2c015e590 2024-09-21T07:51:35.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.470+0000 7fe2c5800640 1 --2- 172.21.15.39:0/484874666 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe2a0078bc0 0x7fe2a007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:35.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.471+0000 7fe2c5800640 1 --2- 172.21.15.39:0/484874666 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe2a0078bc0 0x7fe2a007b0d0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4005fd0 tx=0x7fe2b4005850 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:35.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.474+0000 7fe2bd600640 1 -- 172.21.15.39:0/484874666 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe2b803f030 con 0x7fe2c015e590 2024-09-21T07:51:35.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.713+0000 7fe2c8200640 1 -- 172.21.15.39:0/484874666 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2c006d890 con 0x7fe2c015e590 2024-09-21T07:51:35.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.714+0000 7fe2bd600640 1 -- 172.21.15.39:0/484874666 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 275 v275) ==== 95+0+5328 (secure 0 0 0) 0x7fe2b8084740 con 0x7fe2c015e590 2024-09-21T07:51:35.716 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:35.716 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":275,"btime":"2024-09-21T07:51:31:572749+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10195,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":270}],"filesystems":[{"mdsmap":{"epoch":275,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:51:31.572717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":241,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":11067},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11067":{"gid":11067,"name":"a","rank":2,"incarnation":269,"state":"up:active","state_seq":12,"addr":"172.21.15.39:6834/109229385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":109229385},{"type":"v1","addr":"172.21.15.39:6834","nonce":109229385}]},"join_fscid":-1,"laggy_since":"2024-09-21T07:51:31.572717+0000","export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[9967,11052,11067]},"id":13}]} 2024-09-21T07:51:35.716 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 275 2024-09-21T07:51:35.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.716+0000 7fe2c8200640 1 -- 172.21.15.39:0/484874666 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe2a0078bc0 msgr2=0x7fe2a007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:35.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.716+0000 7fe2c8200640 1 --2- 172.21.15.39:0/484874666 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe2a0078bc0 0x7fe2a007b0d0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4005fd0 tx=0x7fe2b4005850 comp rx=0 tx=0).stop 2024-09-21T07:51:35.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.716+0000 7fe2c8200640 1 -- 172.21.15.39:0/484874666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2c015e590 msgr2=0x7fe2c01f6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:35.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.716+0000 7fe2c8200640 1 --2- 172.21.15.39:0/484874666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2c015e590 0x7fe2c01f6200 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7fe2b802ac60 tx=0x7fe2b8003c20 comp rx=0 tx=0).stop 2024-09-21T07:51:35.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.717+0000 7fe2c8200640 1 -- 172.21.15.39:0/484874666 shutdown_connections 2024-09-21T07:51:35.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.717+0000 7fe2c8200640 1 --2- 172.21.15.39:0/484874666 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe2a0078bc0 0x7fe2a007b0d0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:35.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.717+0000 7fe2c8200640 1 --2- 172.21.15.39:0/484874666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2c015e590 0x7fe2c01f6200 unknown :-1 s=CLOSED pgs=2451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:35.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.717+0000 7fe2c8200640 1 --2- 172.21.15.39:0/484874666 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe2c0151590 0x7fe2c01f5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:35.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.717+0000 7fe2c8200640 1 --2- 172.21.15.39:0/484874666 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2c00b3370 0x7fe2c01583f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:35.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.717+0000 7fe2c8200640 1 -- 172.21.15.39:0/484874666 >> 172.21.15.39:0/484874666 conn(0x7fe2c006cfc0 msgr2=0x7fe2c00ce5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:35.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.717+0000 7fe2c8200640 1 -- 172.21.15.39:0/484874666 shutdown_connections 2024-09-21T07:51:35.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:35.717+0000 7fe2c8200640 1 -- 172.21.15.39:0/484874666 wait complete. 2024-09-21T07:51:35.741 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json dump snaps --server 2024-09-21T07:51:35.990 INFO:teuthology.orchestra.run.smithi148.stdout:{"last_snap":8,"last_created":5,"last_destroyed":7,"pending_noop":[],"snaps":[],"need_to_purge":{},"pending_update":[{"version":18,"snapinfo":{"snapid":8,"ino":1099511627780,"stamp":"2024-09-21T07:51:12.091986+0000","name":"s1","metadata":{}}}],"pending_destroy":[]} 2024-09-21T07:51:35.991 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 5, "last_destroyed": 7, "last_snap": 8, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [ { "snapInfinityo": { "ino": 1099511627780, "metadata": {}, "name": "s1", "snapid": 8, "stamp": "2024-09-21T07:51:12.091986+0000" }, "version": 18 } ], "snaps": [] } 2024-09-21T07:51:35.991 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 13:2 2024-09-21T07:51:36.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.218+0000 7fb2a8e00640 1 -- 172.21.15.39:0/4088508775 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb288005250 msgr2=0x7fb28800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:36.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.218+0000 7fb2a8e00640 1 --2- 172.21.15.39:0/4088508775 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb288005250 0x7fb28800b9c0 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7fb29c009ed0 tx=0x7fb288005830 comp rx=0 tx=0).stop 2024-09-21T07:51:36.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.219+0000 7fb2a8e00640 1 -- 172.21.15.39:0/4088508775 shutdown_connections 2024-09-21T07:51:36.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.219+0000 7fb2a8e00640 1 --2- 172.21.15.39:0/4088508775 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb288005250 0x7fb28800b9c0 unknown :-1 s=CLOSED pgs=2453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.219+0000 7fb2a8e00640 1 -- 172.21.15.39:0/4088508775 >> 172.21.15.39:0/4088508775 conn(0x7fb2a40b2510 msgr2=0x7fb2a40d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:36.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.219+0000 7fb2a8e00640 1 -- 172.21.15.39:0/4088508775 shutdown_connections 2024-09-21T07:51:36.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.219+0000 7fb2a8e00640 1 -- 172.21.15.39:0/4088508775 wait complete. 2024-09-21T07:51:36.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.221+0000 7fb2a8e00640 1 Processor -- start 2024-09-21T07:51:36.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.221+0000 7fb2a8e00640 1 -- start start 2024-09-21T07:51:36.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.222+0000 7fb2a8e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb288005250 0x7fb2a40ad0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:36.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.222+0000 7fb2a8e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2a40af680 0x7fb2a41f9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:36.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.222+0000 7fb2a8e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2a41e8260 0x7fb2a415f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:36.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.223+0000 7fb2a8e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a40cb960 con 0x7fb2a40af680 2024-09-21T07:51:36.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.223+0000 7fb2a8e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a4073720 con 0x7fb2a41e8260 2024-09-21T07:51:36.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.223+0000 7fb2a8e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb2a40cbae0 con 0x7fb288005250 2024-09-21T07:51:36.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.223+0000 7fb2a2a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2a40af680 0x7fb2a41f9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:36.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.223+0000 7fb2a2a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2a40af680 0x7fb2a41f9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39766/0 (socket says 172.21.15.39:39766) 2024-09-21T07:51:36.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.223+0000 7fb2a2a00640 1 -- 172.21.15.39:0/351826001 learned_addr learned my addr 172.21.15.39:0/351826001 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:51:36.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.223+0000 7fb2a3e00640 1 --2- 172.21.15.39:0/351826001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2a41e8260 0x7fb2a415f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:36.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.223+0000 7fb2a3400640 1 --2- 172.21.15.39:0/351826001 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb288005250 0x7fb2a40ad0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:36.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.223+0000 7fb2a2a00640 1 -- 172.21.15.39:0/351826001 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb288005250 msgr2=0x7fb2a40ad0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:36.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.223+0000 7fb2a2a00640 1 --2- 172.21.15.39:0/351826001 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb288005250 0x7fb2a40ad0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.223+0000 7fb2a2a00640 1 -- 172.21.15.39:0/351826001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2a41e8260 msgr2=0x7fb2a415f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:36.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.223+0000 7fb2a2a00640 1 --2- 172.21.15.39:0/351826001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2a41e8260 0x7fb2a415f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.223+0000 7fb2a2a00640 1 -- 172.21.15.39:0/351826001 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a41f9a40 con 0x7fb2a40af680 2024-09-21T07:51:36.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.224+0000 7fb2a2a00640 1 --2- 172.21.15.39:0/351826001 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2a40af680 0x7fb2a41f9500 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7fb29000e5c0 tx=0x7fb29000e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:36.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.224+0000 7fb29b400640 1 -- 172.21.15.39:0/351826001 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb290015070 con 0x7fb2a40af680 2024-09-21T07:51:36.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.224+0000 7fb29b400640 1 -- 172.21.15.39:0/351826001 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb2900109c0 con 0x7fb2a40af680 2024-09-21T07:51:36.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.224+0000 7fb2a8e00640 1 -- 172.21.15.39:0/351826001 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2a40c9bc0 con 0x7fb2a40af680 2024-09-21T07:51:36.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.224+0000 7fb29b400640 1 -- 172.21.15.39:0/351826001 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb290010e10 con 0x7fb2a40af680 2024-09-21T07:51:36.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.224+0000 7fb2a8e00640 1 -- 172.21.15.39:0/351826001 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2a41f8d00 con 0x7fb2a40af680 2024-09-21T07:51:36.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.225+0000 7fb2a8e00640 1 -- 172.21.15.39:0/351826001 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2a406f8a0 con 0x7fb2a40af680 2024-09-21T07:51:36.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.226+0000 7fb29b400640 1 -- 172.21.15.39:0/351826001 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb290018070 con 0x7fb2a40af680 2024-09-21T07:51:36.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.227+0000 7fb29b400640 1 --2- 172.21.15.39:0/351826001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb284078bc0 0x7fb28407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:36.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.227+0000 7fb29b400640 1 -- 172.21.15.39:0/351826001 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(245..245 src has 1..245) ==== 7520+0+0 (secure 0 0 0) 0x7fb29009b630 con 0x7fb2a40af680 2024-09-21T07:51:36.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.228+0000 7fb2a3400640 1 --2- 172.21.15.39:0/351826001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb284078bc0 0x7fb28407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:36.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.229+0000 7fb2a3400640 1 --2- 172.21.15.39:0/351826001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb284078bc0 0x7fb28407b0d0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fb294000c00 tx=0x7fb294006b70 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:36.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.232+0000 7fb29b400640 1 -- 172.21.15.39:0/351826001 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb290062d30 con 0x7fb2a40af680 2024-09-21T07:51:36.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.503+0000 7fb2a8e00640 1 -- 172.21.15.39:0/351826001 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "13:2"} v 0) -- 0x7fb2a40b18c0 con 0x7fb2a40af680 2024-09-21T07:51:36.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.506+0000 7fb29b400640 1 -- 172.21.15.39:0/351826001 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "13:2"}]=0 failed mds gid 11067 v276) ==== 99+0+0 (secure 0 0 0) 0x7fb290067760 con 0x7fb2a40af680 2024-09-21T07:51:36.508 INFO:teuthology.orchestra.run.smithi039.stderr:failed mds gid 11067 2024-09-21T07:51:36.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.509+0000 7fb2a8e00640 1 -- 172.21.15.39:0/351826001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb284078bc0 msgr2=0x7fb28407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:36.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.509+0000 7fb2a8e00640 1 --2- 172.21.15.39:0/351826001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb284078bc0 0x7fb28407b0d0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fb294000c00 tx=0x7fb294006b70 comp rx=0 tx=0).stop 2024-09-21T07:51:36.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.509+0000 7fb2a8e00640 1 -- 172.21.15.39:0/351826001 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2a40af680 msgr2=0x7fb2a41f9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:36.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.509+0000 7fb2a8e00640 1 --2- 172.21.15.39:0/351826001 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2a40af680 0x7fb2a41f9500 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7fb29000e5c0 tx=0x7fb29000e9c0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.510+0000 7fb2a8e00640 1 -- 172.21.15.39:0/351826001 shutdown_connections 2024-09-21T07:51:36.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.510+0000 7fb2a8e00640 1 --2- 172.21.15.39:0/351826001 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb284078bc0 0x7fb28407b0d0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.510+0000 7fb2a8e00640 1 --2- 172.21.15.39:0/351826001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb2a41e8260 0x7fb2a415f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.510+0000 7fb2a8e00640 1 --2- 172.21.15.39:0/351826001 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2a40af680 0x7fb2a41f9500 unknown :-1 s=CLOSED pgs=2454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.510+0000 7fb2a8e00640 1 --2- 172.21.15.39:0/351826001 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb288005250 0x7fb2a40ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.510+0000 7fb2a8e00640 1 -- 172.21.15.39:0/351826001 >> 172.21.15.39:0/351826001 conn(0x7fb2a40b2510 msgr2=0x7fb2a40738a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:36.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.510+0000 7fb2a8e00640 1 -- 172.21.15.39:0/351826001 shutdown_connections 2024-09-21T07:51:36.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.510+0000 7fb2a8e00640 1 -- 172.21.15.39:0/351826001 wait complete. 2024-09-21T07:51:36.540 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T07:51:36.540 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T07:51:36.540 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T07:51:36.540 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:51:36.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:51:36.540 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/daemon/state.py", line 146, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi039 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a' 2024-09-21T07:51:36.541 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T07:51:36.541 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T07:51:36.545 INFO:tasks.ceph.mds.a:Started 2024-09-21T07:51:36.545 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:51:36.741 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-21T07:51:36.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.773+0000 7fe591600640 1 -- 172.21.15.39:0/2473310217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe58c071540 msgr2=0x7fe58c06ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:36.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.773+0000 7fe591600640 1 --2- 172.21.15.39:0/2473310217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe58c071540 0x7fe58c06ddf0 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7fe58c06f470 tx=0x7fe57402a580 comp rx=0 tx=0).stop 2024-09-21T07:51:36.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.773+0000 7fe591600640 1 -- 172.21.15.39:0/2473310217 shutdown_connections 2024-09-21T07:51:36.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.773+0000 7fe591600640 1 --2- 172.21.15.39:0/2473310217 >> v2:172.21.15.148:3300/0 conn(0x7fe58c06e330 0x7fe58c0cd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.773+0000 7fe591600640 1 --2- 172.21.15.39:0/2473310217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe58c071540 0x7fe58c06ddf0 unknown :-1 s=CLOSED pgs=2459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.773+0000 7fe591600640 1 -- 172.21.15.39:0/2473310217 >> 172.21.15.39:0/2473310217 conn(0x7fe58c073980 msgr2=0x7fe58c073d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:36.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.773+0000 7fe591600640 1 -- 172.21.15.39:0/2473310217 shutdown_connections 2024-09-21T07:51:36.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.773+0000 7fe591600640 1 -- 172.21.15.39:0/2473310217 wait complete. 2024-09-21T07:51:36.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.775+0000 7fe591600640 1 Processor -- start 2024-09-21T07:51:36.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.776+0000 7fe591600640 1 -- start start 2024-09-21T07:51:36.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.777+0000 7fe591600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe58c06e330 0x7fe58c1f9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:36.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.777+0000 7fe591600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe58c071540 0x7fe58c14b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:36.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.777+0000 7fe591600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe58c075dc0 0x7fe58c14b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:36.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.777+0000 7fe591600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c0ce210 con 0x7fe58c071540 2024-09-21T07:51:36.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.777+0000 7fe591600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c0ce090 con 0x7fe58c06e330 2024-09-21T07:51:36.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.777+0000 7fe591600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe58c0ce390 con 0x7fe58c075dc0 2024-09-21T07:51:36.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.777+0000 7fe583e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe58c071540 0x7fe58c14b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:36.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.777+0000 7fe583e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe58c071540 0x7fe58c14b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39800/0 (socket says 172.21.15.39:39800) 2024-09-21T07:51:36.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.777+0000 7fe583e00640 1 -- 172.21.15.39:0/1822138437 learned_addr learned my addr 172.21.15.39:0/1822138437 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:51:36.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.777+0000 7fe58aa00640 1 --2- 172.21.15.39:0/1822138437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe58c06e330 0x7fe58c1f9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:36.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.778+0000 7fe58b400640 1 --2- 172.21.15.39:0/1822138437 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe58c075dc0 0x7fe58c14b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:36.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.778+0000 7fe583e00640 1 -- 172.21.15.39:0/1822138437 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe58c075dc0 msgr2=0x7fe58c14b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:36.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.778+0000 7fe583e00640 1 --2- 172.21.15.39:0/1822138437 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe58c075dc0 0x7fe58c14b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.778+0000 7fe583e00640 1 -- 172.21.15.39:0/1822138437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe58c06e330 msgr2=0x7fe58c1f9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:36.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.778+0000 7fe583e00640 1 --2- 172.21.15.39:0/1822138437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe58c06e330 0x7fe58c1f9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.778+0000 7fe583e00640 1 -- 172.21.15.39:0/1822138437 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe58c14be50 con 0x7fe58c071540 2024-09-21T07:51:36.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.778+0000 7fe58aa00640 1 --2- 172.21.15.39:0/1822138437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe58c06e330 0x7fe58c1f9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:51:36.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.778+0000 7fe58b400640 1 --2- 172.21.15.39:0/1822138437 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe58c075dc0 0x7fe58c14b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:51:36.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.778+0000 7fe583e00640 1 --2- 172.21.15.39:0/1822138437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe58c071540 0x7fe58c14b120 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7fe574034040 tx=0x7fe5740038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:36.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.778+0000 7fe582a00640 1 -- 172.21.15.39:0/1822138437 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57403e030 con 0x7fe58c071540 2024-09-21T07:51:36.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.778+0000 7fe582a00640 1 -- 172.21.15.39:0/1822138437 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe574003ca0 con 0x7fe58c071540 2024-09-21T07:51:36.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.778+0000 7fe582a00640 1 -- 172.21.15.39:0/1822138437 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe574035040 con 0x7fe58c071540 2024-09-21T07:51:36.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.778+0000 7fe591600640 1 -- 172.21.15.39:0/1822138437 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe58c14c130 con 0x7fe58c071540 2024-09-21T07:51:36.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.779+0000 7fe591600640 1 -- 172.21.15.39:0/1822138437 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe58c14c6f0 con 0x7fe58c071540 2024-09-21T07:51:36.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.780+0000 7fe591600640 1 -- 172.21.15.39:0/1822138437 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe58c0ce8b0 con 0x7fe58c071540 2024-09-21T07:51:36.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.781+0000 7fe582a00640 1 -- 172.21.15.39:0/1822138437 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe574041030 con 0x7fe58c071540 2024-09-21T07:51:36.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.783+0000 7fe582a00640 1 --2- 172.21.15.39:0/1822138437 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe568078b90 0x7fe56807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:36.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.783+0000 7fe582a00640 1 -- 172.21.15.39:0/1822138437 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(246..246 src has 1..246) ==== 7562+0+0 (secure 0 0 0) 0x7fe5740bbbd0 con 0x7fe58c071540 2024-09-21T07:51:36.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.783+0000 7fe58aa00640 1 --2- 172.21.15.39:0/1822138437 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe568078b90 0x7fe56807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:36.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.784+0000 7fe58aa00640 1 --2- 172.21.15.39:0/1822138437 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe568078b90 0x7fe56807b0a0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fe57c0016c0 tx=0x7fe57c009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:36.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:36.788+0000 7fe582a00640 1 -- 172.21.15.39:0/1822138437 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe5740838c0 con 0x7fe58c071540 2024-09-21T07:51:37.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:37.024+0000 7fe591600640 1 -- 172.21.15.39:0/1822138437 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe58c0b0120 con 0x7fe58c071540 2024-09-21T07:51:37.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:37.025+0000 7fe582a00640 1 -- 172.21.15.39:0/1822138437 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 277 v277) ==== 95+0+4450 (secure 0 0 0) 0x7fe5740831c0 con 0x7fe58c071540 2024-09-21T07:51:37.027 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:37.027 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":277,"btime":"2024-09-21T07:51:36:512216+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":277,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:51:36.512198+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":246,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":10195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10195":{"gid":10195,"name":"d","rank":2,"incarnation":277,"state":"up:replay","state_seq":1,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[11052,9967]},"id":13}]} 2024-09-21T07:51:37.027 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 277 2024-09-21T07:51:37.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:37.027+0000 7fe591600640 1 -- 172.21.15.39:0/1822138437 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe568078b90 msgr2=0x7fe56807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:37.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:37.027+0000 7fe591600640 1 --2- 172.21.15.39:0/1822138437 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe568078b90 0x7fe56807b0a0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fe57c0016c0 tx=0x7fe57c009040 comp rx=0 tx=0).stop 2024-09-21T07:51:37.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:37.027+0000 7fe591600640 1 -- 172.21.15.39:0/1822138437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe58c071540 msgr2=0x7fe58c14b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:37.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:37.027+0000 7fe591600640 1 --2- 172.21.15.39:0/1822138437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe58c071540 0x7fe58c14b120 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7fe574034040 tx=0x7fe5740038a0 comp rx=0 tx=0).stop 2024-09-21T07:51:37.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:37.027+0000 7fe591600640 1 -- 172.21.15.39:0/1822138437 shutdown_connections 2024-09-21T07:51:37.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:37.027+0000 7fe591600640 1 --2- 172.21.15.39:0/1822138437 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe568078b90 0x7fe56807b0a0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:37.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:37.027+0000 7fe591600640 1 --2- 172.21.15.39:0/1822138437 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe58c075dc0 0x7fe58c14b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:37.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:37.027+0000 7fe591600640 1 --2- 172.21.15.39:0/1822138437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe58c071540 0x7fe58c14b120 unknown :-1 s=CLOSED pgs=2460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:37.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:37.027+0000 7fe591600640 1 --2- 172.21.15.39:0/1822138437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe58c06e330 0x7fe58c1f9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:37.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:37.027+0000 7fe591600640 1 -- 172.21.15.39:0/1822138437 >> 172.21.15.39:0/1822138437 conn(0x7fe58c073980 msgr2=0x7fe58c0b5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:37.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:37.028+0000 7fe591600640 1 -- 172.21.15.39:0/1822138437 shutdown_connections 2024-09-21T07:51:37.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:37.028+0000 7fe591600640 1 -- 172.21.15.39:0/1822138437 wait complete. 2024-09-21T07:51:37.052 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-09-21T07:51:38.269 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-09-21T07:51:42.053 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:51:42.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.291+0000 7feeb3e00640 1 -- 172.21.15.39:0/78872976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee94005350 msgr2=0x7fee9400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:42.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.291+0000 7feeb3e00640 1 --2- 172.21.15.39:0/78872976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee94005350 0x7fee9400b9c0 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7feeac073230 tx=0x7fee94005830 comp rx=0 tx=0).stop 2024-09-21T07:51:42.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.291+0000 7feeb3e00640 1 -- 172.21.15.39:0/78872976 shutdown_connections 2024-09-21T07:51:42.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.291+0000 7feeb3e00640 1 --2- 172.21.15.39:0/78872976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee94005350 0x7fee9400b9c0 unknown :-1 s=CLOSED pgs=2462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:42.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.291+0000 7feeb3e00640 1 -- 172.21.15.39:0/78872976 >> 172.21.15.39:0/78872976 conn(0x7feeac06cfc0 msgr2=0x7feeac06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:42.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.291+0000 7feeb3e00640 1 -- 172.21.15.39:0/78872976 shutdown_connections 2024-09-21T07:51:42.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.291+0000 7feeb3e00640 1 -- 172.21.15.39:0/78872976 wait complete. 2024-09-21T07:51:42.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.294+0000 7feeb3e00640 1 Processor -- start 2024-09-21T07:51:42.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.294+0000 7feeb3e00640 1 -- start start 2024-09-21T07:51:42.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.295+0000 7feeb3e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee94005350 0x7feeac0af4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:42.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.295+0000 7feeb3e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeac1f79e0 0x7feeac0ade00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:42.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.295+0000 7feeb3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feeac0c6c90 0x7feeac1f6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:42.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.295+0000 7feeb3e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7feeac0cdc70 con 0x7feeac0c6c90 2024-09-21T07:51:42.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.295+0000 7feeb3e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7feeac0cdaf0 con 0x7feeac1f79e0 2024-09-21T07:51:42.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.295+0000 7feeb3e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7feeac0cddf0 con 0x7fee94005350 2024-09-21T07:51:42.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.296+0000 7feeb1e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feeac0c6c90 0x7feeac1f6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:42.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.296+0000 7feeb1e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feeac0c6c90 0x7feeac1f6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46778/0 (socket says 172.21.15.39:46778) 2024-09-21T07:51:42.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.296+0000 7feeb1e00640 1 -- 172.21.15.39:0/3575419645 learned_addr learned my addr 172.21.15.39:0/3575419645 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:51:42.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.296+0000 7feeb0a00640 1 --2- 172.21.15.39:0/3575419645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeac1f79e0 0x7feeac0ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:42.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.296+0000 7feeb1400640 1 --2- 172.21.15.39:0/3575419645 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee94005350 0x7feeac0af4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:42.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.296+0000 7feeb1e00640 1 -- 172.21.15.39:0/3575419645 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee94005350 msgr2=0x7feeac0af4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:42.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.296+0000 7feeb1e00640 1 --2- 172.21.15.39:0/3575419645 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee94005350 0x7feeac0af4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:42.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.296+0000 7feeb1e00640 1 -- 172.21.15.39:0/3575419645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeac1f79e0 msgr2=0x7feeac0ade00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:42.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.296+0000 7feeb1e00640 1 --2- 172.21.15.39:0/3575419645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeac1f79e0 0x7feeac0ade00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:42.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.296+0000 7feeb1e00640 1 -- 172.21.15.39:0/3575419645 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeac0acd60 con 0x7feeac0c6c90 2024-09-21T07:51:42.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.296+0000 7feeb0a00640 1 --2- 172.21.15.39:0/3575419645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeac1f79e0 0x7feeac0ade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:51:42.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.296+0000 7feeb1e00640 1 --2- 172.21.15.39:0/3575419645 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feeac0c6c90 0x7feeac1f6dd0 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7feea400c700 tx=0x7feea4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:42.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.297+0000 7feea9600640 1 -- 172.21.15.39:0/3575419645 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea402b7a0 con 0x7feeac0c6c90 2024-09-21T07:51:42.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.297+0000 7feea9600640 1 -- 172.21.15.39:0/3575419645 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feea402b940 con 0x7feeac0c6c90 2024-09-21T07:51:42.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.297+0000 7feea9600640 1 -- 172.21.15.39:0/3575419645 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea402bd50 con 0x7feeac0c6c90 2024-09-21T07:51:42.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.297+0000 7feeb3e00640 1 -- 172.21.15.39:0/3575419645 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeac0ad040 con 0x7feeac0c6c90 2024-09-21T07:51:42.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.297+0000 7feeb3e00640 1 -- 172.21.15.39:0/3575419645 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeac0ad600 con 0x7feeac0c6c90 2024-09-21T07:51:42.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.298+0000 7feeb3e00640 1 -- 172.21.15.39:0/3575419645 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feeac0d07e0 con 0x7feeac0c6c90 2024-09-21T07:51:42.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.299+0000 7feea9600640 1 -- 172.21.15.39:0/3575419645 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7feea403c030 con 0x7feeac0c6c90 2024-09-21T07:51:42.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.300+0000 7feea9600640 1 --2- 172.21.15.39:0/3575419645 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fee8c078b90 0x7fee8c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:42.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.300+0000 7feea9600640 1 -- 172.21.15.39:0/3575419645 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(246..246 src has 1..246) ==== 7562+0+0 (secure 0 0 0) 0x7feea40babd0 con 0x7feeac0c6c90 2024-09-21T07:51:42.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.301+0000 7feeb1400640 1 --2- 172.21.15.39:0/3575419645 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fee8c078b90 0x7fee8c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:42.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.302+0000 7feeb1400640 1 --2- 172.21.15.39:0/3575419645 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fee8c078b90 0x7fee8c07b0a0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7feea0009bd0 tx=0x7feea0005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:42.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.305+0000 7feea9600640 1 -- 172.21.15.39:0/3575419645 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7feea4082270 con 0x7feeac0c6c90 2024-09-21T07:51:42.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.539+0000 7feeb3e00640 1 -- 172.21.15.39:0/3575419645 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feeac0ce040 con 0x7feeac0c6c90 2024-09-21T07:51:42.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.540+0000 7feea9600640 1 -- 172.21.15.39:0/3575419645 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 279 v279) ==== 95+0+5280 (secure 0 0 0) 0x7feea4086ca0 con 0x7feeac0c6c90 2024-09-21T07:51:42.542 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:42.542 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":279,"btime":"2024-09-21T07:51:38:524572+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":278}],"filesystems":[{"mdsmap":{"epoch":279,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:51:37.529529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":246,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":10195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10195":{"gid":10195,"name":"d","rank":2,"incarnation":277,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[11052,9967]},"id":13}]} 2024-09-21T07:51:42.542 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 279 2024-09-21T07:51:42.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.542+0000 7feeb3e00640 1 -- 172.21.15.39:0/3575419645 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fee8c078b90 msgr2=0x7fee8c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:42.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.542+0000 7feeb3e00640 1 --2- 172.21.15.39:0/3575419645 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fee8c078b90 0x7fee8c07b0a0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7feea0009bd0 tx=0x7feea0005f90 comp rx=0 tx=0).stop 2024-09-21T07:51:42.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.542+0000 7feeb3e00640 1 -- 172.21.15.39:0/3575419645 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feeac0c6c90 msgr2=0x7feeac1f6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:42.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.542+0000 7feeb3e00640 1 --2- 172.21.15.39:0/3575419645 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feeac0c6c90 0x7feeac1f6dd0 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7feea400c700 tx=0x7feea4003ba0 comp rx=0 tx=0).stop 2024-09-21T07:51:42.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.543+0000 7feeb3e00640 1 -- 172.21.15.39:0/3575419645 shutdown_connections 2024-09-21T07:51:42.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.543+0000 7feeb3e00640 1 --2- 172.21.15.39:0/3575419645 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fee8c078b90 0x7fee8c07b0a0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:42.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.543+0000 7feeb3e00640 1 --2- 172.21.15.39:0/3575419645 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feeac0c6c90 0x7feeac1f6dd0 unknown :-1 s=CLOSED pgs=2463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:42.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.543+0000 7feeb3e00640 1 --2- 172.21.15.39:0/3575419645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feeac1f79e0 0x7feeac0ade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:42.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.543+0000 7feeb3e00640 1 --2- 172.21.15.39:0/3575419645 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fee94005350 0x7feeac0af4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:42.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.543+0000 7feeb3e00640 1 -- 172.21.15.39:0/3575419645 >> 172.21.15.39:0/3575419645 conn(0x7feeac06cfc0 msgr2=0x7feeac073680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:42.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.543+0000 7feeb3e00640 1 -- 172.21.15.39:0/3575419645 shutdown_connections 2024-09-21T07:51:42.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.543+0000 7feeb3e00640 1 -- 172.21.15.39:0/3575419645 wait complete. 2024-09-21T07:51:42.569 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-21T07:51:42.569 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:51:42.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.794+0000 7f3a72600640 1 -- 172.21.15.39:0/2449837744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a6c0d0dd0 msgr2=0x7f3a6c0cdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:42.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.794+0000 7f3a72600640 1 --2- 172.21.15.39:0/2449837744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a6c0d0dd0 0x7f3a6c0cdb90 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c072c00 tx=0x7f3a6402aa30 comp rx=0 tx=0).stop 2024-09-21T07:51:42.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.794+0000 7f3a72600640 1 -- 172.21.15.39:0/2449837744 shutdown_connections 2024-09-21T07:51:42.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.794+0000 7f3a72600640 1 --2- 172.21.15.39:0/2449837744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a6c0d0dd0 0x7f3a6c0cdb90 unknown :-1 s=CLOSED pgs=2465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:42.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.794+0000 7f3a72600640 1 --2- 172.21.15.39:0/2449837744 >> v2:172.21.15.148:3300/0 conn(0x7f3a6c15a460 0x7f3a6c06fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:42.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.794+0000 7f3a72600640 1 -- 172.21.15.39:0/2449837744 >> 172.21.15.39:0/2449837744 conn(0x7f3a6c06cfc0 msgr2=0x7f3a6c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:42.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.794+0000 7f3a72600640 1 -- 172.21.15.39:0/2449837744 shutdown_connections 2024-09-21T07:51:42.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.794+0000 7f3a72600640 1 -- 172.21.15.39:0/2449837744 wait complete. 2024-09-21T07:51:42.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.797+0000 7f3a72600640 1 Processor -- start 2024-09-21T07:51:42.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.797+0000 7f3a72600640 1 -- start start 2024-09-21T07:51:42.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.798+0000 7f3a72600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3a6c0cd780 0x7f3a6c0c5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:42.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.798+0000 7f3a72600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a6c0d0dd0 0x7f3a6c1357d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:42.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.798+0000 7f3a72600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a6c15a460 0x7f3a6c0abd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:42.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.798+0000 7f3a72600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6c15d560 con 0x7f3a6c0d0dd0 2024-09-21T07:51:42.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.798+0000 7f3a72600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6c15d3e0 con 0x7f3a6c15a460 2024-09-21T07:51:42.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.798+0000 7f3a72600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f3a6c15d6e0 con 0x7f3a6c0cd780 2024-09-21T07:51:42.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.798+0000 7f3a62a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a6c0d0dd0 0x7f3a6c1357d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:42.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.798+0000 7f3a62a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a6c0d0dd0 0x7f3a6c1357d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46798/0 (socket says 172.21.15.39:46798) 2024-09-21T07:51:42.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.798+0000 7f3a62a00640 1 -- 172.21.15.39:0/793415080 learned_addr learned my addr 172.21.15.39:0/793415080 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:51:42.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.799+0000 7f3a6be00640 1 --2- 172.21.15.39:0/793415080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a6c15a460 0x7f3a6c0abd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:42.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.799+0000 7f3a6b400640 1 --2- 172.21.15.39:0/793415080 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3a6c0cd780 0x7f3a6c0c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:42.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.799+0000 7f3a62a00640 1 -- 172.21.15.39:0/793415080 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3a6c0cd780 msgr2=0x7f3a6c0c5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:42.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.799+0000 7f3a62a00640 1 --2- 172.21.15.39:0/793415080 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3a6c0cd780 0x7f3a6c0c5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:42.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.799+0000 7f3a62a00640 1 -- 172.21.15.39:0/793415080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a6c15a460 msgr2=0x7f3a6c0abd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:42.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.799+0000 7f3a62a00640 1 --2- 172.21.15.39:0/793415080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a6c15a460 0x7f3a6c0abd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:42.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.799+0000 7f3a62a00640 1 -- 172.21.15.39:0/793415080 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a6c0aacc0 con 0x7f3a6c0d0dd0 2024-09-21T07:51:42.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.799+0000 7f3a62a00640 1 --2- 172.21.15.39:0/793415080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a6c0d0dd0 0x7f3a6c1357d0 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7f3a5400c3a0 tx=0x7f3a5400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:42.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.799+0000 7f3a63e00640 1 -- 172.21.15.39:0/793415080 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a54017070 con 0x7f3a6c0d0dd0 2024-09-21T07:51:42.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.799+0000 7f3a63e00640 1 -- 172.21.15.39:0/793415080 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a540053c0 con 0x7f3a6c0d0dd0 2024-09-21T07:51:42.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.799+0000 7f3a63e00640 1 -- 172.21.15.39:0/793415080 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a54003850 con 0x7f3a6c0d0dd0 2024-09-21T07:51:42.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.800+0000 7f3a72600640 1 -- 172.21.15.39:0/793415080 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a6c0aafa0 con 0x7f3a6c0d0dd0 2024-09-21T07:51:42.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.800+0000 7f3a72600640 1 -- 172.21.15.39:0/793415080 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a6c0ab560 con 0x7f3a6c0d0dd0 2024-09-21T07:51:42.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.801+0000 7f3a72600640 1 -- 172.21.15.39:0/793415080 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a6c15ed10 con 0x7f3a6c0d0dd0 2024-09-21T07:51:42.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.802+0000 7f3a63e00640 1 -- 172.21.15.39:0/793415080 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3a5401a070 con 0x7f3a6c0d0dd0 2024-09-21T07:51:42.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.803+0000 7f3a63e00640 1 --2- 172.21.15.39:0/793415080 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3a48078b90 0x7f3a4807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:42.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.803+0000 7f3a63e00640 1 -- 172.21.15.39:0/793415080 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(246..246 src has 1..246) ==== 7562+0+0 (secure 0 0 0) 0x7f3a5409a760 con 0x7f3a6c0d0dd0 2024-09-21T07:51:42.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.803+0000 7f3a6b400640 1 --2- 172.21.15.39:0/793415080 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3a48078b90 0x7f3a4807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:42.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.804+0000 7f3a6b400640 1 --2- 172.21.15.39:0/793415080 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3a48078b90 0x7f3a4807b0a0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c0058c0 tx=0x7f3a5c005850 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:42.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:42.807+0000 7f3a63e00640 1 -- 172.21.15.39:0/793415080 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3a540624f0 con 0x7f3a6c0d0dd0 2024-09-21T07:51:43.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:43.042+0000 7f3a72600640 1 -- 172.21.15.39:0/793415080 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a6c0af3a0 con 0x7f3a6c0d0dd0 2024-09-21T07:51:43.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:43.043+0000 7f3a63e00640 1 -- 172.21.15.39:0/793415080 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 279 v279) ==== 95+0+5280 (secure 0 0 0) 0x7f3a54061df0 con 0x7f3a6c0d0dd0 2024-09-21T07:51:43.045 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:43.045 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":279,"btime":"2024-09-21T07:51:38:524572+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":278}],"filesystems":[{"mdsmap":{"epoch":279,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:51:37.529529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":246,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":10195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10195":{"gid":10195,"name":"d","rank":2,"incarnation":277,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[11052,9967]},"id":13}]} 2024-09-21T07:51:43.045 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 279 2024-09-21T07:51:43.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:43.046+0000 7f3a72600640 1 -- 172.21.15.39:0/793415080 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3a48078b90 msgr2=0x7f3a4807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:43.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:43.046+0000 7f3a72600640 1 --2- 172.21.15.39:0/793415080 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3a48078b90 0x7f3a4807b0a0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c0058c0 tx=0x7f3a5c005850 comp rx=0 tx=0).stop 2024-09-21T07:51:43.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:43.046+0000 7f3a72600640 1 -- 172.21.15.39:0/793415080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a6c0d0dd0 msgr2=0x7f3a6c1357d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:43.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:43.046+0000 7f3a72600640 1 --2- 172.21.15.39:0/793415080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a6c0d0dd0 0x7f3a6c1357d0 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7f3a5400c3a0 tx=0x7f3a5400c860 comp rx=0 tx=0).stop 2024-09-21T07:51:43.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:43.046+0000 7f3a72600640 1 -- 172.21.15.39:0/793415080 shutdown_connections 2024-09-21T07:51:43.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:43.046+0000 7f3a72600640 1 --2- 172.21.15.39:0/793415080 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3a48078b90 0x7f3a4807b0a0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:43.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:43.046+0000 7f3a72600640 1 --2- 172.21.15.39:0/793415080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a6c15a460 0x7f3a6c0abd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:43.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:43.046+0000 7f3a72600640 1 --2- 172.21.15.39:0/793415080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a6c0d0dd0 0x7f3a6c1357d0 unknown :-1 s=CLOSED pgs=2466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:43.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:43.046+0000 7f3a72600640 1 --2- 172.21.15.39:0/793415080 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3a6c0cd780 0x7f3a6c0c5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:43.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:43.046+0000 7f3a72600640 1 -- 172.21.15.39:0/793415080 >> 172.21.15.39:0/793415080 conn(0x7f3a6c06cfc0 msgr2=0x7f3a6c15a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:43.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:43.046+0000 7f3a72600640 1 -- 172.21.15.39:0/793415080 shutdown_connections 2024-09-21T07:51:43.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:43.046+0000 7f3a72600640 1 -- 172.21.15.39:0/793415080 wait complete. 2024-09-21T07:51:43.476 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 279, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:51:37.529529+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 246, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 9967, 'mds_1': 11052, 'mds_2': 10195}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9967': {'gid': 9967, 'name': 'b', 'rank': 0, 'incarnation': 259, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.148:6836/3381818843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 3381818843}, {'type': 'v1', 'addr': '172.21.15.148:6836', 'nonce': 3381818843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10195': {'gid': 10195, 'name': 'd', 'rank': 2, 'incarnation': 277, 'state': 'up:reconnect', 'state_seq': 11, 'addr': '172.21.15.148:6837/597301351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6835', 'nonce': 597301351}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 597301351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11052': {'gid': 11052, 'name': 'c', 'rank': 1, 'incarnation': 261, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6835/485194245', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 485194245}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 485194245}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9967, 'qdb_cluster': [11052, 9967]} 2024-09-21T07:51:43.476 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10195:up:reconnect 2024-09-21T07:51:44.477 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:51:44.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.713+0000 7f8960a00640 1 -- 172.21.15.39:0/222926770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f895c0cc500 msgr2=0x7f895c0af590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:44.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.713+0000 7f8960a00640 1 --2- 172.21.15.39:0/222926770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f895c0cc500 0x7f895c0af590 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f894c009830 tx=0x7f894c02a510 comp rx=0 tx=0).stop 2024-09-21T07:51:44.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.713+0000 7f8960a00640 1 -- 172.21.15.39:0/222926770 shutdown_connections 2024-09-21T07:51:44.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.713+0000 7f8960a00640 1 --2- 172.21.15.39:0/222926770 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f895c0d0e70 0x7f895c0cca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:44.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.713+0000 7f8960a00640 1 --2- 172.21.15.39:0/222926770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f895c0cc500 0x7f895c0af590 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:44.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.713+0000 7f8960a00640 1 -- 172.21.15.39:0/222926770 >> 172.21.15.39:0/222926770 conn(0x7f895c0b2510 msgr2=0x7f895c0d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:44.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.714+0000 7f8960a00640 1 -- 172.21.15.39:0/222926770 shutdown_connections 2024-09-21T07:51:44.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.714+0000 7f8960a00640 1 -- 172.21.15.39:0/222926770 wait complete. 2024-09-21T07:51:44.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.716+0000 7f8960a00640 1 Processor -- start 2024-09-21T07:51:44.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.716+0000 7f8960a00640 1 -- start start 2024-09-21T07:51:44.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.717+0000 7f8960a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f895c0cc500 0x7f895c161b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:44.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.717+0000 7f8960a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f895c0ccf90 0x7f895c1f5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:44.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.717+0000 7f8960a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f895c0d0e70 0x7f895c09a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:44.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.717+0000 7f8960a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f895c06faf0 con 0x7f895c0cc500 2024-09-21T07:51:44.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.717+0000 7f8960a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f895c06fc70 con 0x7f895c0d0e70 2024-09-21T07:51:44.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.717+0000 7f8960a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f895c06fdf0 con 0x7f895c0ccf90 2024-09-21T07:51:44.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.718+0000 7f895b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f895c0cc500 0x7f895c161b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:44.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.718+0000 7f895b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f895c0cc500 0x7f895c161b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46808/0 (socket says 172.21.15.39:46808) 2024-09-21T07:51:44.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.718+0000 7f895b400640 1 -- 172.21.15.39:0/2300599593 learned_addr learned my addr 172.21.15.39:0/2300599593 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:51:44.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.718+0000 7f895be00640 1 --2- 172.21.15.39:0/2300599593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f895c0d0e70 0x7f895c09a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:44.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.718+0000 7f8953e00640 1 --2- 172.21.15.39:0/2300599593 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f895c0ccf90 0x7f895c1f5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:44.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.718+0000 7f895b400640 1 -- 172.21.15.39:0/2300599593 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f895c0ccf90 msgr2=0x7f895c1f5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:44.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.718+0000 7f895b400640 1 --2- 172.21.15.39:0/2300599593 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f895c0ccf90 0x7f895c1f5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:44.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.718+0000 7f895b400640 1 -- 172.21.15.39:0/2300599593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f895c0d0e70 msgr2=0x7f895c09a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:44.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.718+0000 7f895b400640 1 --2- 172.21.15.39:0/2300599593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f895c0d0e70 0x7f895c09a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:44.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.718+0000 7f895b400640 1 -- 172.21.15.39:0/2300599593 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f895c1f6170 con 0x7f895c0cc500 2024-09-21T07:51:44.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.718+0000 7f895be00640 1 --2- 172.21.15.39:0/2300599593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f895c0d0e70 0x7f895c09a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:51:44.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.718+0000 7f8953e00640 1 --2- 172.21.15.39:0/2300599593 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f895c0ccf90 0x7f895c1f5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:51:44.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.719+0000 7f895b400640 1 --2- 172.21.15.39:0/2300599593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f895c0cc500 0x7f895c161b70 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7f894c009950 tx=0x7f894c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:44.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.719+0000 7f8953400640 1 -- 172.21.15.39:0/2300599593 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894c005c90 con 0x7f895c0cc500 2024-09-21T07:51:44.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.719+0000 7f8953400640 1 -- 172.21.15.39:0/2300599593 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f894c031e50 con 0x7f895c0cc500 2024-09-21T07:51:44.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.719+0000 7f8960a00640 1 -- 172.21.15.39:0/2300599593 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f895c1f6450 con 0x7f895c0cc500 2024-09-21T07:51:44.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.719+0000 7f8953400640 1 -- 172.21.15.39:0/2300599593 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894c005260 con 0x7f895c0cc500 2024-09-21T07:51:44.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.719+0000 7f8960a00640 1 -- 172.21.15.39:0/2300599593 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f895c09b990 con 0x7f895c0cc500 2024-09-21T07:51:44.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.720+0000 7f8960a00640 1 -- 172.21.15.39:0/2300599593 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f895c0702a0 con 0x7f895c0cc500 2024-09-21T07:51:44.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.721+0000 7f8953400640 1 -- 172.21.15.39:0/2300599593 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f894c005400 con 0x7f895c0cc500 2024-09-21T07:51:44.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.722+0000 7f8953400640 1 --2- 172.21.15.39:0/2300599593 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8938078b90 0x7f893807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:44.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.722+0000 7f8953400640 1 -- 172.21.15.39:0/2300599593 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(246..246 src has 1..246) ==== 7562+0+0 (secure 0 0 0) 0x7f894c0bc6e0 con 0x7f895c0cc500 2024-09-21T07:51:44.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.723+0000 7f8953e00640 1 --2- 172.21.15.39:0/2300599593 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8938078b90 0x7f893807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:44.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.723+0000 7f8953e00640 1 --2- 172.21.15.39:0/2300599593 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8938078b90 0x7f893807b0a0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f895c15da30 tx=0x7f8944005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:44.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.727+0000 7f8953400640 1 -- 172.21.15.39:0/2300599593 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f894c03f030 con 0x7f895c0cc500 2024-09-21T07:51:44.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.963+0000 7f8960a00640 1 -- 172.21.15.39:0/2300599593 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f895c09c670 con 0x7f895c0cc500 2024-09-21T07:51:44.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.964+0000 7f8953400640 1 -- 172.21.15.39:0/2300599593 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 279 v279) ==== 95+0+5280 (secure 0 0 0) 0x7f894c0842c0 con 0x7f895c0cc500 2024-09-21T07:51:44.966 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:44.966 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":279,"btime":"2024-09-21T07:51:38:524572+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":278}],"filesystems":[{"mdsmap":{"epoch":279,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:51:37.529529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":246,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":10195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10195":{"gid":10195,"name":"d","rank":2,"incarnation":277,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[11052,9967]},"id":13}]} 2024-09-21T07:51:44.966 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 279 2024-09-21T07:51:44.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.966+0000 7f8960a00640 1 -- 172.21.15.39:0/2300599593 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8938078b90 msgr2=0x7f893807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:44.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.966+0000 7f8960a00640 1 --2- 172.21.15.39:0/2300599593 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8938078b90 0x7f893807b0a0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f895c15da30 tx=0x7f8944005f90 comp rx=0 tx=0).stop 2024-09-21T07:51:44.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.966+0000 7f8960a00640 1 -- 172.21.15.39:0/2300599593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f895c0cc500 msgr2=0x7f895c161b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:44.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.966+0000 7f8960a00640 1 --2- 172.21.15.39:0/2300599593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f895c0cc500 0x7f895c161b70 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7f894c009950 tx=0x7f894c033c20 comp rx=0 tx=0).stop 2024-09-21T07:51:44.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.966+0000 7f8960a00640 1 -- 172.21.15.39:0/2300599593 shutdown_connections 2024-09-21T07:51:44.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.966+0000 7f8960a00640 1 --2- 172.21.15.39:0/2300599593 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8938078b90 0x7f893807b0a0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:44.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.966+0000 7f8960a00640 1 --2- 172.21.15.39:0/2300599593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f895c0d0e70 0x7f895c09a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:44.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.966+0000 7f8960a00640 1 --2- 172.21.15.39:0/2300599593 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f895c0ccf90 0x7f895c1f5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:44.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.966+0000 7f8960a00640 1 --2- 172.21.15.39:0/2300599593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f895c0cc500 0x7f895c161b70 unknown :-1 s=CLOSED pgs=2467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:44.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.966+0000 7f8960a00640 1 -- 172.21.15.39:0/2300599593 >> 172.21.15.39:0/2300599593 conn(0x7f895c0b2510 msgr2=0x7f895c0ae540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:44.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.966+0000 7f8960a00640 1 -- 172.21.15.39:0/2300599593 shutdown_connections 2024-09-21T07:51:44.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:44.966+0000 7f8960a00640 1 -- 172.21.15.39:0/2300599593 wait complete. 2024-09-21T07:51:45.396 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 279, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:51:37.529529+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 246, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 9967, 'mds_1': 11052, 'mds_2': 10195}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9967': {'gid': 9967, 'name': 'b', 'rank': 0, 'incarnation': 259, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.148:6836/3381818843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 3381818843}, {'type': 'v1', 'addr': '172.21.15.148:6836', 'nonce': 3381818843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10195': {'gid': 10195, 'name': 'd', 'rank': 2, 'incarnation': 277, 'state': 'up:reconnect', 'state_seq': 11, 'addr': '172.21.15.148:6837/597301351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6835', 'nonce': 597301351}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 597301351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11052': {'gid': 11052, 'name': 'c', 'rank': 1, 'incarnation': 261, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6835/485194245', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 485194245}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 485194245}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9967, 'qdb_cluster': [11052, 9967]} 2024-09-21T07:51:45.396 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10195:up:reconnect 2024-09-21T07:51:46.397 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:51:46.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.633+0000 7f799e200640 1 -- 172.21.15.39:0/2206172735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f799806f800 msgr2=0x7f79980cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:46.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.633+0000 7f799e200640 1 --2- 172.21.15.39:0/2206172735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f799806f800 0x7f79980cd8e0 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7f7980005fd0 tx=0x7f798002a530 comp rx=0 tx=0).stop 2024-09-21T07:51:46.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.633+0000 7f799e200640 1 -- 172.21.15.39:0/2206172735 shutdown_connections 2024-09-21T07:51:46.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.633+0000 7f799e200640 1 --2- 172.21.15.39:0/2206172735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f799806f800 0x7f79980cd8e0 unknown :-1 s=CLOSED pgs=2468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:46.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.634+0000 7f799e200640 1 --2- 172.21.15.39:0/2206172735 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f799815cd30 0x7f79980cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:46.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.634+0000 7f799e200640 1 -- 172.21.15.39:0/2206172735 >> 172.21.15.39:0/2206172735 conn(0x7f799806cfc0 msgr2=0x7f799806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:46.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.634+0000 7f799e200640 1 -- 172.21.15.39:0/2206172735 shutdown_connections 2024-09-21T07:51:46.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.634+0000 7f799e200640 1 -- 172.21.15.39:0/2206172735 wait complete. 2024-09-21T07:51:46.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.636+0000 7f799e200640 1 Processor -- start 2024-09-21T07:51:46.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.636+0000 7f799e200640 1 -- start start 2024-09-21T07:51:46.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.637+0000 7f799e200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f799806f800 0x7f79981f93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:46.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.637+0000 7f799e200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f799806fc10 0x7f79980aa930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:46.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.637+0000 7f799e200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f799815cd30 0x7f79981e92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:46.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.637+0000 7f799e200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7998073890 con 0x7f799806fc10 2024-09-21T07:51:46.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.637+0000 7f799e200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7998073710 con 0x7f799806f800 2024-09-21T07:51:46.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.637+0000 7f799e200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f7998073a10 con 0x7f799815cd30 2024-09-21T07:51:46.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.638+0000 7f798fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f799806fc10 0x7f79980aa930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:46.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.638+0000 7f798fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f799806fc10 0x7f79980aa930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46818/0 (socket says 172.21.15.39:46818) 2024-09-21T07:51:46.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.638+0000 7f798fe00640 1 -- 172.21.15.39:0/2326575774 learned_addr learned my addr 172.21.15.39:0/2326575774 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:51:46.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.638+0000 7f7997e00640 1 --2- 172.21.15.39:0/2326575774 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f799815cd30 0x7f79981e92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:46.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.638+0000 7f7997400640 1 --2- 172.21.15.39:0/2326575774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f799806f800 0x7f79981f93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:46.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.638+0000 7f798fe00640 1 -- 172.21.15.39:0/2326575774 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f799815cd30 msgr2=0x7f79981e92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:46.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.638+0000 7f798fe00640 1 --2- 172.21.15.39:0/2326575774 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f799815cd30 0x7f79981e92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:46.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.638+0000 7f798fe00640 1 -- 172.21.15.39:0/2326575774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f799806f800 msgr2=0x7f79981f93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:46.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.638+0000 7f798fe00640 1 --2- 172.21.15.39:0/2326575774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f799806f800 0x7f79981f93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:46.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.638+0000 7f798fe00640 1 -- 172.21.15.39:0/2326575774 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79981f5590 con 0x7f799806fc10 2024-09-21T07:51:46.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.638+0000 7f7997e00640 1 --2- 172.21.15.39:0/2326575774 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f799815cd30 0x7f79981e92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:51:46.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.638+0000 7f7997400640 1 --2- 172.21.15.39:0/2326575774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f799806f800 0x7f79981f93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:51:46.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.639+0000 7f798fe00640 1 --2- 172.21.15.39:0/2326575774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f799806fc10 0x7f79980aa930 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7f7980006010 tx=0x7f79800038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:46.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.639+0000 7f798f400640 1 -- 172.21.15.39:0/2326575774 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798003e030 con 0x7f799806fc10 2024-09-21T07:51:46.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.639+0000 7f798f400640 1 -- 172.21.15.39:0/2326575774 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7980003d90 con 0x7f799806fc10 2024-09-21T07:51:46.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.639+0000 7f799e200640 1 -- 172.21.15.39:0/2326575774 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79981f5870 con 0x7f799806fc10 2024-09-21T07:51:46.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.639+0000 7f798f400640 1 -- 172.21.15.39:0/2326575774 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7980035040 con 0x7f799806fc10 2024-09-21T07:51:46.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.639+0000 7f799e200640 1 -- 172.21.15.39:0/2326575774 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79981f5e30 con 0x7f799806fc10 2024-09-21T07:51:46.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.640+0000 7f799e200640 1 -- 172.21.15.39:0/2326575774 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7998073ec0 con 0x7f799806fc10 2024-09-21T07:51:46.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.641+0000 7f798f400640 1 -- 172.21.15.39:0/2326575774 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7980041030 con 0x7f799806fc10 2024-09-21T07:51:46.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.642+0000 7f798f400640 1 --2- 172.21.15.39:0/2326575774 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7974078b90 0x7f797407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:46.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.642+0000 7f798f400640 1 -- 172.21.15.39:0/2326575774 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(246..246 src has 1..246) ==== 7562+0+0 (secure 0 0 0) 0x7f79800bbcf0 con 0x7f799806fc10 2024-09-21T07:51:46.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.643+0000 7f7997400640 1 --2- 172.21.15.39:0/2326575774 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7974078b90 0x7f797407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:46.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.644+0000 7f7997400640 1 --2- 172.21.15.39:0/2326575774 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7974078b90 0x7f797407b0a0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f7988005fd0 tx=0x7f798800f360 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:46.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.647+0000 7f798f400640 1 -- 172.21.15.39:0/2326575774 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7980083950 con 0x7f799806fc10 2024-09-21T07:51:46.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.882+0000 7f799e200640 1 -- 172.21.15.39:0/2326575774 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f799806e110 con 0x7f799806fc10 2024-09-21T07:51:46.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.882+0000 7f798f400640 1 -- 172.21.15.39:0/2326575774 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 279 v279) ==== 95+0+5280 (secure 0 0 0) 0x7f7980083250 con 0x7f799806fc10 2024-09-21T07:51:46.885 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:46.885 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":279,"btime":"2024-09-21T07:51:38:524572+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":278}],"filesystems":[{"mdsmap":{"epoch":279,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:51:37.529529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":246,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":10195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10195":{"gid":10195,"name":"d","rank":2,"incarnation":277,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[11052,9967]},"id":13}]} 2024-09-21T07:51:46.885 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 279 2024-09-21T07:51:46.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.885+0000 7f799e200640 1 -- 172.21.15.39:0/2326575774 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7974078b90 msgr2=0x7f797407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:46.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.885+0000 7f799e200640 1 --2- 172.21.15.39:0/2326575774 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7974078b90 0x7f797407b0a0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f7988005fd0 tx=0x7f798800f360 comp rx=0 tx=0).stop 2024-09-21T07:51:46.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.885+0000 7f799e200640 1 -- 172.21.15.39:0/2326575774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f799806fc10 msgr2=0x7f79980aa930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:46.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.885+0000 7f799e200640 1 --2- 172.21.15.39:0/2326575774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f799806fc10 0x7f79980aa930 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7f7980006010 tx=0x7f79800038a0 comp rx=0 tx=0).stop 2024-09-21T07:51:46.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.885+0000 7f799e200640 1 -- 172.21.15.39:0/2326575774 shutdown_connections 2024-09-21T07:51:46.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.885+0000 7f799e200640 1 --2- 172.21.15.39:0/2326575774 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7974078b90 0x7f797407b0a0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:46.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.885+0000 7f799e200640 1 --2- 172.21.15.39:0/2326575774 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f799815cd30 0x7f79981e92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:46.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.885+0000 7f799e200640 1 --2- 172.21.15.39:0/2326575774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f799806fc10 0x7f79980aa930 unknown :-1 s=CLOSED pgs=2469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:46.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.885+0000 7f799e200640 1 --2- 172.21.15.39:0/2326575774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f799806f800 0x7f79981f93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:46.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.885+0000 7f799e200640 1 -- 172.21.15.39:0/2326575774 >> 172.21.15.39:0/2326575774 conn(0x7f799806cfc0 msgr2=0x7f79980b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:46.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.885+0000 7f799e200640 1 -- 172.21.15.39:0/2326575774 shutdown_connections 2024-09-21T07:51:46.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:46.885+0000 7f799e200640 1 -- 172.21.15.39:0/2326575774 wait complete. 2024-09-21T07:51:47.311 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 279, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:51:37.529529+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 246, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 9967, 'mds_1': 11052, 'mds_2': 10195}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9967': {'gid': 9967, 'name': 'b', 'rank': 0, 'incarnation': 259, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.148:6836/3381818843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 3381818843}, {'type': 'v1', 'addr': '172.21.15.148:6836', 'nonce': 3381818843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10195': {'gid': 10195, 'name': 'd', 'rank': 2, 'incarnation': 277, 'state': 'up:reconnect', 'state_seq': 11, 'addr': '172.21.15.148:6837/597301351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6835', 'nonce': 597301351}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 597301351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11052': {'gid': 11052, 'name': 'c', 'rank': 1, 'incarnation': 261, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6835/485194245', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 485194245}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 485194245}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9967, 'qdb_cluster': [11052, 9967]} 2024-09-21T07:51:47.311 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10195:up:reconnect 2024-09-21T07:51:48.313 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:51:48.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.551+0000 7f4b42800640 1 -- 172.21.15.39:0/3719317285 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b3c0d09a0 msgr2=0x7f4b20004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:48.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.551+0000 7f4b42800640 1 --2- 172.21.15.39:0/3719317285 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b3c0d09a0 0x7f4b20004800 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4b24009b50 tx=0x7f4b20005830 comp rx=0 tx=0).stop 2024-09-21T07:51:48.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.551+0000 7f4b3be00640 1 -- 172.21.15.39:0/3719317285 reap_dead start 2024-09-21T07:51:48.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.551+0000 7f4b42800640 1 -- 172.21.15.39:0/3719317285 shutdown_connections 2024-09-21T07:51:48.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.551+0000 7f4b42800640 1 -- 172.21.15.39:0/3719317285 >> 172.21.15.39:0/3719317285 conn(0x7f4b3c06cfc0 msgr2=0x7f4b3c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:48.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.551+0000 7f4b42800640 1 -- 172.21.15.39:0/3719317285 shutdown_connections 2024-09-21T07:51:48.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.551+0000 7f4b42800640 1 -- 172.21.15.39:0/3719317285 wait complete. 2024-09-21T07:51:48.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.553+0000 7f4b42800640 1 Processor -- start 2024-09-21T07:51:48.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.553+0000 7f4b42800640 1 -- start start 2024-09-21T07:51:48.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.555+0000 7f4b42800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b3c0d09a0 0x7f4b3c1fdee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:48.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.555+0000 7f4b42800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b3c0b3620 0x7f4b3c13d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:48.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.555+0000 7f4b42800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b3c156630 0x7f4b3c156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:48.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.555+0000 7f4b42800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c0cc910 con 0x7f4b3c156630 2024-09-21T07:51:48.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.555+0000 7f4b42800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c0cc790 con 0x7f4b3c0b3620 2024-09-21T07:51:48.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.555+0000 7f4b42800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f4b3c0cca90 con 0x7f4b3c0d09a0 2024-09-21T07:51:48.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.555+0000 7f4b3be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b3c156630 0x7f4b3c156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:48.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.555+0000 7f4b3b400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b3c0d09a0 0x7f4b3c1fdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:48.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.555+0000 7f4b3be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b3c156630 0x7f4b3c156a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46836/0 (socket says 172.21.15.39:46836) 2024-09-21T07:51:48.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.555+0000 7f4b33e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b3c0b3620 0x7f4b3c13d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:48.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.555+0000 7f4b3be00640 1 -- 172.21.15.39:0/978307515 learned_addr learned my addr 172.21.15.39:0/978307515 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:51:48.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.556+0000 7f4b3be00640 1 -- 172.21.15.39:0/978307515 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b3c0d09a0 msgr2=0x7f4b3c1fdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:48.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.556+0000 7f4b3be00640 1 --2- 172.21.15.39:0/978307515 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b3c0d09a0 0x7f4b3c1fdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:48.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.556+0000 7f4b3be00640 1 -- 172.21.15.39:0/978307515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b3c0b3620 msgr2=0x7f4b3c13d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:48.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.556+0000 7f4b3be00640 1 --2- 172.21.15.39:0/978307515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b3c0b3620 0x7f4b3c13d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:48.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.556+0000 7f4b3be00640 1 -- 172.21.15.39:0/978307515 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b3c156f80 con 0x7f4b3c156630 2024-09-21T07:51:48.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.556+0000 7f4b33e00640 1 --2- 172.21.15.39:0/978307515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b3c0b3620 0x7f4b3c13d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:51:48.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.556+0000 7f4b3be00640 1 --2- 172.21.15.39:0/978307515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b3c156630 0x7f4b3c156a40 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c0affb0 tx=0x7f4b3400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:48.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.556+0000 7f4b33400640 1 -- 172.21.15.39:0/978307515 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b34003d80 con 0x7f4b3c156630 2024-09-21T07:51:48.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.557+0000 7f4b33400640 1 -- 172.21.15.39:0/978307515 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b34004b20 con 0x7f4b3c156630 2024-09-21T07:51:48.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.557+0000 7f4b33400640 1 -- 172.21.15.39:0/978307515 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b34004030 con 0x7f4b3c156630 2024-09-21T07:51:48.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.557+0000 7f4b42800640 1 -- 172.21.15.39:0/978307515 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b3c154a90 con 0x7f4b3c156630 2024-09-21T07:51:48.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.557+0000 7f4b42800640 1 -- 172.21.15.39:0/978307515 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b3c0b3cf0 con 0x7f4b3c156630 2024-09-21T07:51:48.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.558+0000 7f4b42800640 1 -- 172.21.15.39:0/978307515 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b3c0cdd00 con 0x7f4b3c156630 2024-09-21T07:51:48.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.559+0000 7f4b33400640 1 -- 172.21.15.39:0/978307515 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4b340041d0 con 0x7f4b3c156630 2024-09-21T07:51:48.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.560+0000 7f4b33400640 1 --2- 172.21.15.39:0/978307515 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4b18078bc0 0x7f4b1807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:48.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.560+0000 7f4b33400640 1 -- 172.21.15.39:0/978307515 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(246..246 src has 1..246) ==== 7562+0+0 (secure 0 0 0) 0x7f4b3409c310 con 0x7f4b3c156630 2024-09-21T07:51:48.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.561+0000 7f4b3b400640 1 --2- 172.21.15.39:0/978307515 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4b18078bc0 0x7f4b1807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:48.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.561+0000 7f4b3b400640 1 --2- 172.21.15.39:0/978307515 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4b18078bc0 0x7f4b1807b0d0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c001470 tx=0x7f4b2c001060 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:48.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.564+0000 7f4b33400640 1 -- 172.21.15.39:0/978307515 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4b3c0cdd00 con 0x7f4b3c156630 2024-09-21T07:51:48.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.803+0000 7f4b42800640 1 -- 172.21.15.39:0/978307515 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b3c15e660 con 0x7f4b3c156630 2024-09-21T07:51:48.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.804+0000 7f4b33400640 1 -- 172.21.15.39:0/978307515 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 279 v279) ==== 95+0+5280 (secure 0 0 0) 0x7f4b3c15e660 con 0x7f4b3c156630 2024-09-21T07:51:48.806 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:48.806 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":279,"btime":"2024-09-21T07:51:38:524572+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":278}],"filesystems":[{"mdsmap":{"epoch":279,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:51:37.529529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":246,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":10195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10195":{"gid":10195,"name":"d","rank":2,"incarnation":277,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[11052,9967]},"id":13}]} 2024-09-21T07:51:48.806 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 279 2024-09-21T07:51:48.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.806+0000 7f4b42800640 1 -- 172.21.15.39:0/978307515 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4b18078bc0 msgr2=0x7f4b1807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:48.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.806+0000 7f4b42800640 1 --2- 172.21.15.39:0/978307515 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4b18078bc0 0x7f4b1807b0d0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c001470 tx=0x7f4b2c001060 comp rx=0 tx=0).stop 2024-09-21T07:51:48.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.806+0000 7f4b42800640 1 -- 172.21.15.39:0/978307515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b3c156630 msgr2=0x7f4b3c156a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:48.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.806+0000 7f4b42800640 1 --2- 172.21.15.39:0/978307515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b3c156630 0x7f4b3c156a40 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c0affb0 tx=0x7f4b3400f790 comp rx=0 tx=0).stop 2024-09-21T07:51:48.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.806+0000 7f4b42800640 1 -- 172.21.15.39:0/978307515 shutdown_connections 2024-09-21T07:51:48.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.806+0000 7f4b42800640 1 --2- 172.21.15.39:0/978307515 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4b18078bc0 0x7f4b1807b0d0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:48.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.806+0000 7f4b42800640 1 --2- 172.21.15.39:0/978307515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b3c156630 0x7f4b3c156a40 unknown :-1 s=CLOSED pgs=2471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:48.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.806+0000 7f4b42800640 1 --2- 172.21.15.39:0/978307515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b3c0b3620 0x7f4b3c13d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:48.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.806+0000 7f4b42800640 1 --2- 172.21.15.39:0/978307515 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4b3c0d09a0 0x7f4b3c1fdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:48.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.806+0000 7f4b42800640 1 -- 172.21.15.39:0/978307515 >> 172.21.15.39:0/978307515 conn(0x7f4b3c06cfc0 msgr2=0x7f4b3c06fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:48.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.806+0000 7f4b42800640 1 -- 172.21.15.39:0/978307515 shutdown_connections 2024-09-21T07:51:48.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:48.806+0000 7f4b42800640 1 -- 172.21.15.39:0/978307515 wait complete. 2024-09-21T07:51:49.234 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 279, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:51:37.529529+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 246, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 9967, 'mds_1': 11052, 'mds_2': 10195}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9967': {'gid': 9967, 'name': 'b', 'rank': 0, 'incarnation': 259, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.148:6836/3381818843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 3381818843}, {'type': 'v1', 'addr': '172.21.15.148:6836', 'nonce': 3381818843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10195': {'gid': 10195, 'name': 'd', 'rank': 2, 'incarnation': 277, 'state': 'up:reconnect', 'state_seq': 11, 'addr': '172.21.15.148:6837/597301351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6835', 'nonce': 597301351}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 597301351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11052': {'gid': 11052, 'name': 'c', 'rank': 1, 'incarnation': 261, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6835/485194245', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 485194245}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 485194245}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9967, 'qdb_cluster': [11052, 9967]} 2024-09-21T07:51:49.234 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10195:up:reconnect 2024-09-21T07:51:50.235 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:51:50.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.470+0000 7f0e7bc00640 1 -- 172.21.15.39:0/2955847717 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e74155e80 msgr2=0x7f0e740b4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:50.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.470+0000 7f0e7bc00640 1 --2- 172.21.15.39:0/2955847717 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e74155e80 0x7f0e740b4690 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c005fd0 tx=0x7f0e6c02a8b0 comp rx=0 tx=0).stop 2024-09-21T07:51:50.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.470+0000 7f0e7bc00640 1 -- 172.21.15.39:0/2955847717 shutdown_connections 2024-09-21T07:51:50.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.470+0000 7f0e7bc00640 1 --2- 172.21.15.39:0/2955847717 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e74155e80 0x7f0e740b4690 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:50.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.470+0000 7f0e7bc00640 1 --2- 172.21.15.39:0/2955847717 >> v2:172.21.15.148:3300/0 conn(0x7f0e7415f360 0x7f0e7415bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:50.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.470+0000 7f0e7bc00640 1 -- 172.21.15.39:0/2955847717 >> 172.21.15.39:0/2955847717 conn(0x7f0e7406cfc0 msgr2=0x7f0e7406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:50.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.470+0000 7f0e7bc00640 1 -- 172.21.15.39:0/2955847717 shutdown_connections 2024-09-21T07:51:50.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.470+0000 7f0e7bc00640 1 -- 172.21.15.39:0/2955847717 wait complete. 2024-09-21T07:51:50.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.472+0000 7f0e7bc00640 1 Processor -- start 2024-09-21T07:51:50.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.472+0000 7f0e7bc00640 1 -- start start 2024-09-21T07:51:50.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.474+0000 7f0e7bc00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e74155a70 0x7f0e74142680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:50.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.474+0000 7f0e7bc00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e74155e80 0x7f0e741528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:50.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.474+0000 7f0e7bc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e7415f360 0x7f0e741f8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:50.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.474+0000 7f0e7bc00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0e740d0620 con 0x7f0e7415f360 2024-09-21T07:51:50.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.474+0000 7f0e7bc00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0e740b4ef0 con 0x7f0e74155e80 2024-09-21T07:51:50.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.474+0000 7f0e7bc00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f0e740d07a0 con 0x7f0e74155a70 2024-09-21T07:51:50.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.474+0000 7f0e79c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e7415f360 0x7f0e741f8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:50.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.474+0000 7f0e79c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e7415f360 0x7f0e741f8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46838/0 (socket says 172.21.15.39:46838) 2024-09-21T07:51:50.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.474+0000 7f0e79c00640 1 -- 172.21.15.39:0/111728754 learned_addr learned my addr 172.21.15.39:0/111728754 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:51:50.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.474+0000 7f0e79200640 1 --2- 172.21.15.39:0/111728754 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e74155a70 0x7f0e74142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:50.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.474+0000 7f0e73e00640 1 --2- 172.21.15.39:0/111728754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e74155e80 0x7f0e741528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:50.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.475+0000 7f0e79c00640 1 -- 172.21.15.39:0/111728754 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e74155a70 msgr2=0x7f0e74142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:50.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.475+0000 7f0e79c00640 1 --2- 172.21.15.39:0/111728754 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e74155a70 0x7f0e74142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:50.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.475+0000 7f0e79c00640 1 -- 172.21.15.39:0/111728754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e74155e80 msgr2=0x7f0e741528c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:50.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.475+0000 7f0e79c00640 1 --2- 172.21.15.39:0/111728754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e74155e80 0x7f0e741528c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:50.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.475+0000 7f0e79c00640 1 -- 172.21.15.39:0/111728754 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e741457d0 con 0x7f0e7415f360 2024-09-21T07:51:50.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.475+0000 7f0e79c00640 1 --2- 172.21.15.39:0/111728754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e7415f360 0x7f0e741f8a90 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c02aea0 tx=0x7f0e6c0316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:50.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.475+0000 7f0e70c00640 1 -- 172.21.15.39:0/111728754 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c031da0 con 0x7f0e7415f360 2024-09-21T07:51:50.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.475+0000 7f0e70c00640 1 -- 172.21.15.39:0/111728754 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e6c003a60 con 0x7f0e7415f360 2024-09-21T07:51:50.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.475+0000 7f0e7bc00640 1 -- 172.21.15.39:0/111728754 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e74145c40 con 0x7f0e7415f360 2024-09-21T07:51:50.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.475+0000 7f0e70c00640 1 -- 172.21.15.39:0/111728754 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c0330d0 con 0x7f0e7415f360 2024-09-21T07:51:50.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.476+0000 7f0e7bc00640 1 -- 172.21.15.39:0/111728754 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e7415d180 con 0x7f0e7415f360 2024-09-21T07:51:50.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.476+0000 7f0e7bc00640 1 -- 172.21.15.39:0/111728754 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e740d0cc0 con 0x7f0e7415f360 2024-09-21T07:51:50.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.478+0000 7f0e70c00640 1 -- 172.21.15.39:0/111728754 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0e6c033270 con 0x7f0e7415f360 2024-09-21T07:51:50.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.479+0000 7f0e70c00640 1 --2- 172.21.15.39:0/111728754 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0e54078b90 0x7f0e5407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:50.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.479+0000 7f0e70c00640 1 -- 172.21.15.39:0/111728754 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(246..246 src has 1..246) ==== 7562+0+0 (secure 0 0 0) 0x7f0e6c0bc7c0 con 0x7f0e7415f360 2024-09-21T07:51:50.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.479+0000 7f0e79200640 1 --2- 172.21.15.39:0/111728754 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0e54078b90 0x7f0e5407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:50.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.480+0000 7f0e79200640 1 --2- 172.21.15.39:0/111728754 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0e54078b90 0x7f0e5407b0a0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f0e680095f0 tx=0x7f0e68005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:50.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.483+0000 7f0e70c00640 1 -- 172.21.15.39:0/111728754 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0e6c03f030 con 0x7f0e7415f360 2024-09-21T07:51:50.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.721+0000 7f0e7bc00640 1 -- 172.21.15.39:0/111728754 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e741459a0 con 0x7f0e7415f360 2024-09-21T07:51:50.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.722+0000 7f0e70c00640 1 -- 172.21.15.39:0/111728754 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 279 v279) ==== 95+0+5280 (secure 0 0 0) 0x7f0e6c083e70 con 0x7f0e7415f360 2024-09-21T07:51:50.724 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:50.724 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":279,"btime":"2024-09-21T07:51:38:524572+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":278}],"filesystems":[{"mdsmap":{"epoch":279,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:51:37.529529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":246,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":10195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10195":{"gid":10195,"name":"d","rank":2,"incarnation":277,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[11052,9967]},"id":13}]} 2024-09-21T07:51:50.725 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 279 2024-09-21T07:51:50.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.724+0000 7f0e7bc00640 1 -- 172.21.15.39:0/111728754 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0e54078b90 msgr2=0x7f0e5407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:50.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.724+0000 7f0e7bc00640 1 --2- 172.21.15.39:0/111728754 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0e54078b90 0x7f0e5407b0a0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f0e680095f0 tx=0x7f0e68005f90 comp rx=0 tx=0).stop 2024-09-21T07:51:50.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.724+0000 7f0e7bc00640 1 -- 172.21.15.39:0/111728754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e7415f360 msgr2=0x7f0e741f8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:50.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.724+0000 7f0e7bc00640 1 --2- 172.21.15.39:0/111728754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e7415f360 0x7f0e741f8a90 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c02aea0 tx=0x7f0e6c0316f0 comp rx=0 tx=0).stop 2024-09-21T07:51:50.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.724+0000 7f0e7bc00640 1 -- 172.21.15.39:0/111728754 shutdown_connections 2024-09-21T07:51:50.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.724+0000 7f0e7bc00640 1 --2- 172.21.15.39:0/111728754 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0e54078b90 0x7f0e5407b0a0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:50.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.725+0000 7f0e7bc00640 1 --2- 172.21.15.39:0/111728754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e7415f360 0x7f0e741f8a90 unknown :-1 s=CLOSED pgs=2472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:50.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.725+0000 7f0e7bc00640 1 --2- 172.21.15.39:0/111728754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e74155e80 0x7f0e741528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:50.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.725+0000 7f0e7bc00640 1 --2- 172.21.15.39:0/111728754 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0e74155a70 0x7f0e74142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:50.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.725+0000 7f0e7bc00640 1 -- 172.21.15.39:0/111728754 >> 172.21.15.39:0/111728754 conn(0x7f0e7406cfc0 msgr2=0x7f0e7415a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:50.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.725+0000 7f0e7bc00640 1 -- 172.21.15.39:0/111728754 shutdown_connections 2024-09-21T07:51:50.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:50.725+0000 7f0e7bc00640 1 -- 172.21.15.39:0/111728754 wait complete. 2024-09-21T07:51:51.155 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 279, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:51:37.529529+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 246, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 9967, 'mds_1': 11052, 'mds_2': 10195}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9967': {'gid': 9967, 'name': 'b', 'rank': 0, 'incarnation': 259, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.148:6836/3381818843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 3381818843}, {'type': 'v1', 'addr': '172.21.15.148:6836', 'nonce': 3381818843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10195': {'gid': 10195, 'name': 'd', 'rank': 2, 'incarnation': 277, 'state': 'up:reconnect', 'state_seq': 11, 'addr': '172.21.15.148:6837/597301351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6835', 'nonce': 597301351}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 597301351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11052': {'gid': 11052, 'name': 'c', 'rank': 1, 'incarnation': 261, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6835/485194245', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 485194245}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 485194245}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9967, 'qdb_cluster': [11052, 9967]} 2024-09-21T07:51:51.156 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10195:up:reconnect 2024-09-21T07:51:52.156 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:51:52.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.393+0000 7fb363000640 1 -- 172.21.15.39:0/2564362320 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb344005250 msgr2=0x7fb34400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:52.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.393+0000 7fb363000640 1 --2- 172.21.15.39:0/2564362320 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb344005250 0x7fb34400b9c0 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7fb35400c630 tx=0x7fb344005830 comp rx=0 tx=0).stop 2024-09-21T07:51:52.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.393+0000 7fb363000640 1 -- 172.21.15.39:0/2564362320 shutdown_connections 2024-09-21T07:51:52.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.393+0000 7fb363000640 1 --2- 172.21.15.39:0/2564362320 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb344005250 0x7fb34400b9c0 unknown :-1 s=CLOSED pgs=2474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:52.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.393+0000 7fb363000640 1 -- 172.21.15.39:0/2564362320 >> 172.21.15.39:0/2564362320 conn(0x7fb35c0b2510 msgr2=0x7fb35c0d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:52.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.393+0000 7fb363000640 1 -- 172.21.15.39:0/2564362320 shutdown_connections 2024-09-21T07:51:52.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.393+0000 7fb363000640 1 -- 172.21.15.39:0/2564362320 wait complete. 2024-09-21T07:51:52.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.395+0000 7fb363000640 1 Processor -- start 2024-09-21T07:51:52.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.395+0000 7fb363000640 1 -- start start 2024-09-21T07:51:52.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.397+0000 7fb363000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb344005250 0x7fb35c1e61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:52.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.397+0000 7fb363000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb35c15ce30 0x7fb35c157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:52.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.397+0000 7fb363000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb35c1f8ca0 0x7fb35c1f90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:52.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.397+0000 7fb363000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb35c06fc30 con 0x7fb35c1f8ca0 2024-09-21T07:51:52.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.397+0000 7fb363000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb35c06fab0 con 0x7fb35c15ce30 2024-09-21T07:51:52.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.397+0000 7fb363000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb35c06fdb0 con 0x7fb344005250 2024-09-21T07:51:52.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.397+0000 7fb362600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb35c1f8ca0 0x7fb35c1f90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:52.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.397+0000 7fb362600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb35c1f8ca0 0x7fb35c1f90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46088/0 (socket says 172.21.15.39:46088) 2024-09-21T07:51:52.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.397+0000 7fb362600640 1 -- 172.21.15.39:0/1110661916 learned_addr learned my addr 172.21.15.39:0/1110661916 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:51:52.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.397+0000 7fb361200640 1 --2- 172.21.15.39:0/1110661916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb35c15ce30 0x7fb35c157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:52.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.397+0000 7fb361c00640 1 --2- 172.21.15.39:0/1110661916 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb344005250 0x7fb35c1e61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:52.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.397+0000 7fb362600640 1 -- 172.21.15.39:0/1110661916 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb344005250 msgr2=0x7fb35c1e61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:52.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.397+0000 7fb362600640 1 --2- 172.21.15.39:0/1110661916 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb344005250 0x7fb35c1e61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:52.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.397+0000 7fb362600640 1 -- 172.21.15.39:0/1110661916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb35c15ce30 msgr2=0x7fb35c157bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:52.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.397+0000 7fb362600640 1 --2- 172.21.15.39:0/1110661916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb35c15ce30 0x7fb35c157bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:52.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.397+0000 7fb362600640 1 -- 172.21.15.39:0/1110661916 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb35c1f95f0 con 0x7fb35c1f8ca0 2024-09-21T07:51:52.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.397+0000 7fb361c00640 1 --2- 172.21.15.39:0/1110661916 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb344005250 0x7fb35c1e61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:51:52.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.397+0000 7fb361200640 1 --2- 172.21.15.39:0/1110661916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb35c15ce30 0x7fb35c157bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:51:52.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.398+0000 7fb362600640 1 --2- 172.21.15.39:0/1110661916 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb35c1f8ca0 0x7fb35c1f90b0 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7fb354007010 tx=0x7fb354003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:52.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.398+0000 7fb359600640 1 -- 172.21.15.39:0/1110661916 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35402b7f0 con 0x7fb35c1f8ca0 2024-09-21T07:51:52.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.398+0000 7fb359600640 1 -- 172.21.15.39:0/1110661916 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb35402b990 con 0x7fb35c1f8ca0 2024-09-21T07:51:52.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.398+0000 7fb359600640 1 -- 172.21.15.39:0/1110661916 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35402bde0 con 0x7fb35c1f8ca0 2024-09-21T07:51:52.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.398+0000 7fb363000640 1 -- 172.21.15.39:0/1110661916 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb35c15b310 con 0x7fb35c1f8ca0 2024-09-21T07:51:52.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.398+0000 7fb363000640 1 -- 172.21.15.39:0/1110661916 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb35c15b8d0 con 0x7fb35c1f8ca0 2024-09-21T07:51:52.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.399+0000 7fb363000640 1 -- 172.21.15.39:0/1110661916 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb35c072fa0 con 0x7fb35c1f8ca0 2024-09-21T07:51:52.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.400+0000 7fb359600640 1 -- 172.21.15.39:0/1110661916 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb35403c030 con 0x7fb35c1f8ca0 2024-09-21T07:51:52.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.401+0000 7fb359600640 1 --2- 172.21.15.39:0/1110661916 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb33c078b90 0x7fb33c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:52.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.401+0000 7fb359600640 1 -- 172.21.15.39:0/1110661916 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(246..246 src has 1..246) ==== 7562+0+0 (secure 0 0 0) 0x7fb3540baa90 con 0x7fb35c1f8ca0 2024-09-21T07:51:52.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.402+0000 7fb361c00640 1 --2- 172.21.15.39:0/1110661916 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb33c078b90 0x7fb33c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:52.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.403+0000 7fb361c00640 1 --2- 172.21.15.39:0/1110661916 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb33c078b90 0x7fb33c07b0a0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fb350009bd0 tx=0x7fb350005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:52.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.406+0000 7fb359600640 1 -- 172.21.15.39:0/1110661916 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb354082780 con 0x7fb35c1f8ca0 2024-09-21T07:51:52.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.642+0000 7fb363000640 1 -- 172.21.15.39:0/1110661916 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb35c0ce380 con 0x7fb35c1f8ca0 2024-09-21T07:51:52.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.643+0000 7fb359600640 1 -- 172.21.15.39:0/1110661916 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 279 v279) ==== 95+0+5280 (secure 0 0 0) 0x7fb354082080 con 0x7fb35c1f8ca0 2024-09-21T07:51:52.645 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:52.645 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":279,"btime":"2024-09-21T07:51:38:524572+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":278}],"filesystems":[{"mdsmap":{"epoch":279,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:51:37.529529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":246,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":10195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10195":{"gid":10195,"name":"d","rank":2,"incarnation":277,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[11052,9967]},"id":13}]} 2024-09-21T07:51:52.645 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 279 2024-09-21T07:51:52.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.645+0000 7fb363000640 1 -- 172.21.15.39:0/1110661916 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb33c078b90 msgr2=0x7fb33c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:52.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.645+0000 7fb363000640 1 --2- 172.21.15.39:0/1110661916 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb33c078b90 0x7fb33c07b0a0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fb350009bd0 tx=0x7fb350005f90 comp rx=0 tx=0).stop 2024-09-21T07:51:52.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.645+0000 7fb363000640 1 -- 172.21.15.39:0/1110661916 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb35c1f8ca0 msgr2=0x7fb35c1f90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:52.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.645+0000 7fb363000640 1 --2- 172.21.15.39:0/1110661916 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb35c1f8ca0 0x7fb35c1f90b0 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7fb354007010 tx=0x7fb354003ba0 comp rx=0 tx=0).stop 2024-09-21T07:51:52.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.646+0000 7fb363000640 1 -- 172.21.15.39:0/1110661916 shutdown_connections 2024-09-21T07:51:52.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.646+0000 7fb363000640 1 --2- 172.21.15.39:0/1110661916 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb33c078b90 0x7fb33c07b0a0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:52.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.646+0000 7fb363000640 1 --2- 172.21.15.39:0/1110661916 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb35c1f8ca0 0x7fb35c1f90b0 unknown :-1 s=CLOSED pgs=2475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:52.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.646+0000 7fb363000640 1 --2- 172.21.15.39:0/1110661916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb35c15ce30 0x7fb35c157bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:52.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.646+0000 7fb363000640 1 --2- 172.21.15.39:0/1110661916 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb344005250 0x7fb35c1e61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:52.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.646+0000 7fb363000640 1 -- 172.21.15.39:0/1110661916 >> 172.21.15.39:0/1110661916 conn(0x7fb35c0b2510 msgr2=0x7fb35c0cd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:52.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.646+0000 7fb363000640 1 -- 172.21.15.39:0/1110661916 shutdown_connections 2024-09-21T07:51:52.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:52.646+0000 7fb363000640 1 -- 172.21.15.39:0/1110661916 wait complete. 2024-09-21T07:51:53.072 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 279, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:51:37.529529+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 246, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 9967, 'mds_1': 11052, 'mds_2': 10195}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9967': {'gid': 9967, 'name': 'b', 'rank': 0, 'incarnation': 259, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.148:6836/3381818843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 3381818843}, {'type': 'v1', 'addr': '172.21.15.148:6836', 'nonce': 3381818843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10195': {'gid': 10195, 'name': 'd', 'rank': 2, 'incarnation': 277, 'state': 'up:reconnect', 'state_seq': 11, 'addr': '172.21.15.148:6837/597301351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6835', 'nonce': 597301351}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 597301351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11052': {'gid': 11052, 'name': 'c', 'rank': 1, 'incarnation': 261, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6835/485194245', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 485194245}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 485194245}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9967, 'qdb_cluster': [11052, 9967]} 2024-09-21T07:51:53.072 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10195:up:reconnect 2024-09-21T07:51:54.074 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:51:54.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.310+0000 7fbce5c00640 1 -- 172.21.15.39:0/668517346 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbce00d2a50 msgr2=0x7fbce00d2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:54.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.310+0000 7fbce5c00640 1 --2- 172.21.15.39:0/668517346 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbce00d2a50 0x7fbce00d2e60 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7fbce008df30 tx=0x7fbcc802a5f0 comp rx=0 tx=0).stop 2024-09-21T07:51:54.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.310+0000 7fbce5c00640 1 -- 172.21.15.39:0/668517346 shutdown_connections 2024-09-21T07:51:54.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.310+0000 7fbce5c00640 1 --2- 172.21.15.39:0/668517346 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbce00d2a50 0x7fbce00d2e60 unknown :-1 s=CLOSED pgs=2476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:54.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.310+0000 7fbce5c00640 1 --2- 172.21.15.39:0/668517346 >> v2:172.21.15.148:3300/0 conn(0x7fbce00b4db0 0x7fbce00d4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:54.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.310+0000 7fbce5c00640 1 -- 172.21.15.39:0/668517346 >> 172.21.15.39:0/668517346 conn(0x7fbce008bfb0 msgr2=0x7fbce00d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:54.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.310+0000 7fbce5c00640 1 -- 172.21.15.39:0/668517346 shutdown_connections 2024-09-21T07:51:54.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.310+0000 7fbce5c00640 1 -- 172.21.15.39:0/668517346 wait complete. 2024-09-21T07:51:54.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.312+0000 7fbce5c00640 1 Processor -- start 2024-09-21T07:51:54.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.312+0000 7fbce5c00640 1 -- start start 2024-09-21T07:51:54.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.314+0000 7fbce5c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbce00b4db0 0x7fbce00ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:54.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.314+0000 7fbce5c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbce00b6000 0x7fbce015b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:54.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.314+0000 7fbce5c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbce00d2a50 0x7fbce0154c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:54.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.314+0000 7fbce5c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbce00cee90 con 0x7fbce00b6000 2024-09-21T07:51:54.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.314+0000 7fbce5c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbce00b6d30 con 0x7fbce00d2a50 2024-09-21T07:51:54.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.314+0000 7fbce5c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fbce00cf010 con 0x7fbce00b4db0 2024-09-21T07:51:54.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.314+0000 7fbcd7400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbce00b6000 0x7fbce015b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:54.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.314+0000 7fbcd7400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbce00b6000 0x7fbce015b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46104/0 (socket says 172.21.15.39:46104) 2024-09-21T07:51:54.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.314+0000 7fbcd7400640 1 -- 172.21.15.39:0/3274914293 learned_addr learned my addr 172.21.15.39:0/3274914293 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:51:54.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.314+0000 7fbcdfe00640 1 --2- 172.21.15.39:0/3274914293 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbce00b4db0 0x7fbce00ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:54.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.314+0000 7fbce5200640 1 --2- 172.21.15.39:0/3274914293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbce00d2a50 0x7fbce0154c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:54.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.315+0000 7fbcd7400640 1 -- 172.21.15.39:0/3274914293 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbce00b4db0 msgr2=0x7fbce00ca7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:54.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.315+0000 7fbcd7400640 1 --2- 172.21.15.39:0/3274914293 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbce00b4db0 0x7fbce00ca7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:54.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.315+0000 7fbcd7400640 1 -- 172.21.15.39:0/3274914293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbce00d2a50 msgr2=0x7fbce0154c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:54.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.315+0000 7fbcd7400640 1 --2- 172.21.15.39:0/3274914293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbce00d2a50 0x7fbce0154c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:54.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.315+0000 7fbcd7400640 1 -- 172.21.15.39:0/3274914293 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce0161830 con 0x7fbce00b6000 2024-09-21T07:51:54.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.315+0000 7fbcd7400640 1 --2- 172.21.15.39:0/3274914293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbce00b6000 0x7fbce015b790 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7fbcc802abe0 tx=0x7fbcc80049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:54.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.315+0000 7fbcdcc00640 1 -- 172.21.15.39:0/3274914293 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8031950 con 0x7fbce00b6000 2024-09-21T07:51:54.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.315+0000 7fbcdcc00640 1 -- 172.21.15.39:0/3274914293 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbcc8031be0 con 0x7fbce00b6000 2024-09-21T07:51:54.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.315+0000 7fbcdcc00640 1 -- 172.21.15.39:0/3274914293 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc80330d0 con 0x7fbce00b6000 2024-09-21T07:51:54.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.315+0000 7fbce5c00640 1 -- 172.21.15.39:0/3274914293 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbce0161550 con 0x7fbce00b6000 2024-09-21T07:51:54.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.316+0000 7fbce5c00640 1 -- 172.21.15.39:0/3274914293 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbce015af90 con 0x7fbce00b6000 2024-09-21T07:51:54.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.317+0000 7fbce5c00640 1 -- 172.21.15.39:0/3274914293 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbce01612c0 con 0x7fbce00b6000 2024-09-21T07:51:54.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.318+0000 7fbcdcc00640 1 -- 172.21.15.39:0/3274914293 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbcc8033290 con 0x7fbce00b6000 2024-09-21T07:51:54.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.319+0000 7fbcdcc00640 1 --2- 172.21.15.39:0/3274914293 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbcbc078bc0 0x7fbcbc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:54.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.319+0000 7fbcdcc00640 1 -- 172.21.15.39:0/3274914293 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(246..246 src has 1..246) ==== 7562+0+0 (secure 0 0 0) 0x7fbcc80bc740 con 0x7fbce00b6000 2024-09-21T07:51:54.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.319+0000 7fbcdfe00640 1 --2- 172.21.15.39:0/3274914293 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbcbc078bc0 0x7fbcbc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:54.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.320+0000 7fbcdfe00640 1 --2- 172.21.15.39:0/3274914293 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbcbc078bc0 0x7fbcbc07b0d0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fbcd000bb30 tx=0x7fbcd000b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:54.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.324+0000 7fbcdcc00640 1 -- 172.21.15.39:0/3274914293 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbcc803f030 con 0x7fbce00b6000 2024-09-21T07:51:54.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.560+0000 7fbce5c00640 1 -- 172.21.15.39:0/3274914293 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbce00d4c90 con 0x7fbce00b6000 2024-09-21T07:51:54.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.560+0000 7fbcdcc00640 1 -- 172.21.15.39:0/3274914293 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 279 v279) ==== 95+0+5280 (secure 0 0 0) 0x7fbcc8031770 con 0x7fbce00b6000 2024-09-21T07:51:54.562 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:54.563 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":279,"btime":"2024-09-21T07:51:38:524572+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":278}],"filesystems":[{"mdsmap":{"epoch":279,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:51:37.529529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":246,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":10195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10195":{"gid":10195,"name":"d","rank":2,"incarnation":277,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[11052,9967]},"id":13}]} 2024-09-21T07:51:54.563 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 279 2024-09-21T07:51:54.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.563+0000 7fbce5c00640 1 -- 172.21.15.39:0/3274914293 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbcbc078bc0 msgr2=0x7fbcbc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:54.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.563+0000 7fbce5c00640 1 --2- 172.21.15.39:0/3274914293 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbcbc078bc0 0x7fbcbc07b0d0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fbcd000bb30 tx=0x7fbcd000b2e0 comp rx=0 tx=0).stop 2024-09-21T07:51:54.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.563+0000 7fbce5c00640 1 -- 172.21.15.39:0/3274914293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbce00b6000 msgr2=0x7fbce015b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:54.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.563+0000 7fbce5c00640 1 --2- 172.21.15.39:0/3274914293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbce00b6000 0x7fbce015b790 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7fbcc802abe0 tx=0x7fbcc80049d0 comp rx=0 tx=0).stop 2024-09-21T07:51:54.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.563+0000 7fbce5c00640 1 -- 172.21.15.39:0/3274914293 shutdown_connections 2024-09-21T07:51:54.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.563+0000 7fbce5c00640 1 --2- 172.21.15.39:0/3274914293 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbcbc078bc0 0x7fbcbc07b0d0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:54.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.563+0000 7fbce5c00640 1 --2- 172.21.15.39:0/3274914293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbce00d2a50 0x7fbce0154c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:54.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.563+0000 7fbce5c00640 1 --2- 172.21.15.39:0/3274914293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbce00b6000 0x7fbce015b790 unknown :-1 s=CLOSED pgs=2477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:54.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.563+0000 7fbce5c00640 1 --2- 172.21.15.39:0/3274914293 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbce00b4db0 0x7fbce00ca7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:54.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.563+0000 7fbce5c00640 1 -- 172.21.15.39:0/3274914293 >> 172.21.15.39:0/3274914293 conn(0x7fbce008bfb0 msgr2=0x7fbce00d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:54.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.563+0000 7fbce5c00640 1 -- 172.21.15.39:0/3274914293 shutdown_connections 2024-09-21T07:51:54.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:54.563+0000 7fbce5c00640 1 -- 172.21.15.39:0/3274914293 wait complete. 2024-09-21T07:51:54.992 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 279, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:51:37.529529+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 246, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 9967, 'mds_1': 11052, 'mds_2': 10195}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9967': {'gid': 9967, 'name': 'b', 'rank': 0, 'incarnation': 259, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.148:6836/3381818843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 3381818843}, {'type': 'v1', 'addr': '172.21.15.148:6836', 'nonce': 3381818843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10195': {'gid': 10195, 'name': 'd', 'rank': 2, 'incarnation': 277, 'state': 'up:reconnect', 'state_seq': 11, 'addr': '172.21.15.148:6837/597301351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6835', 'nonce': 597301351}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 597301351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11052': {'gid': 11052, 'name': 'c', 'rank': 1, 'incarnation': 261, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6835/485194245', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 485194245}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 485194245}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9967, 'qdb_cluster': [11052, 9967]} 2024-09-21T07:51:54.992 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10195:up:reconnect 2024-09-21T07:51:55.994 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:51:56.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.231+0000 7f1720a00640 1 -- 172.21.15.39:0/4292641151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f171c06def0 msgr2=0x7f171c0b79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:56.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.231+0000 7f1720a00640 1 --2- 172.21.15.39:0/4292641151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f171c06def0 0x7f171c0b79f0 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7f170c0098e0 tx=0x7f170c02a5d0 comp rx=0 tx=0).stop 2024-09-21T07:51:56.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.231+0000 7f1720a00640 1 -- 172.21.15.39:0/4292641151 shutdown_connections 2024-09-21T07:51:56.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.231+0000 7f1720a00640 1 --2- 172.21.15.39:0/4292641151 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f171c0d11f0 0x7f171c0cc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:56.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.231+0000 7f1720a00640 1 --2- 172.21.15.39:0/4292641151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f171c06def0 0x7f171c0b79f0 unknown :-1 s=CLOSED pgs=2479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:56.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.231+0000 7f1720a00640 1 -- 172.21.15.39:0/4292641151 >> 172.21.15.39:0/4292641151 conn(0x7f171c0b3e40 msgr2=0x7f171c0b4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:56.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.231+0000 7f1720a00640 1 -- 172.21.15.39:0/4292641151 shutdown_connections 2024-09-21T07:51:56.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.231+0000 7f1720a00640 1 -- 172.21.15.39:0/4292641151 wait complete. 2024-09-21T07:51:56.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.233+0000 7f1720a00640 1 Processor -- start 2024-09-21T07:51:56.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.233+0000 7f1720a00640 1 -- start start 2024-09-21T07:51:56.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.235+0000 7f1720a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f171c06def0 0x7f171c15ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:56.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.235+0000 7f1720a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f171c0d11f0 0x7f171c0cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:56.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.235+0000 7f1720a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f171c0d1600 0x7f171c0cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:56.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.235+0000 7f1720a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f171c09b130 con 0x7f171c0d1600 2024-09-21T07:51:56.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.235+0000 7f1720a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f171c0cd2f0 con 0x7f171c06def0 2024-09-21T07:51:56.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.235+0000 7f1720a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f171c09b2b0 con 0x7f171c0d11f0 2024-09-21T07:51:56.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.235+0000 7f171be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f171c0d1600 0x7f171c0cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:56.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.235+0000 7f171be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f171c0d1600 0x7f171c0cbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46124/0 (socket says 172.21.15.39:46124) 2024-09-21T07:51:56.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.235+0000 7f171be00640 1 -- 172.21.15.39:0/3641678174 learned_addr learned my addr 172.21.15.39:0/3641678174 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:51:56.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.235+0000 7f171aa00640 1 --2- 172.21.15.39:0/3641678174 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f171c0d11f0 0x7f171c0cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:56.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.235+0000 7f171b400640 1 --2- 172.21.15.39:0/3641678174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f171c06def0 0x7f171c15ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:56.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.235+0000 7f171be00640 1 -- 172.21.15.39:0/3641678174 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f171c0d11f0 msgr2=0x7f171c0cb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:56.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.235+0000 7f171be00640 1 --2- 172.21.15.39:0/3641678174 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f171c0d11f0 0x7f171c0cb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:56.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.235+0000 7f171be00640 1 -- 172.21.15.39:0/3641678174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f171c06def0 msgr2=0x7f171c15ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:56.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.235+0000 7f171be00640 1 --2- 172.21.15.39:0/3641678174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f171c06def0 0x7f171c15ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:56.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.235+0000 7f171be00640 1 -- 172.21.15.39:0/3641678174 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f171c15b2e0 con 0x7f171c0d1600 2024-09-21T07:51:56.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.235+0000 7f171b400640 1 --2- 172.21.15.39:0/3641678174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f171c06def0 0x7f171c15ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:51:56.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.236+0000 7f171be00640 1 --2- 172.21.15.39:0/3641678174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f171c0d1600 0x7f171c0cbce0 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7f171400c730 tx=0x7f171400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:56.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.236+0000 7f1713400640 1 -- 172.21.15.39:0/3641678174 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1714017070 con 0x7f171c0d1600 2024-09-21T07:51:56.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.236+0000 7f1713400640 1 -- 172.21.15.39:0/3641678174 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1714003c60 con 0x7f171c0d1600 2024-09-21T07:51:56.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.236+0000 7f1713400640 1 -- 172.21.15.39:0/3641678174 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17140052d0 con 0x7f171c0d1600 2024-09-21T07:51:56.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.236+0000 7f1720a00640 1 -- 172.21.15.39:0/3641678174 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f171c0c9eb0 con 0x7f171c0d1600 2024-09-21T07:51:56.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.236+0000 7f1720a00640 1 -- 172.21.15.39:0/3641678174 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f171c0c5e80 con 0x7f171c0d1600 2024-09-21T07:51:56.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.237+0000 7f1720a00640 1 -- 172.21.15.39:0/3641678174 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f171c09b7d0 con 0x7f171c0d1600 2024-09-21T07:51:56.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.239+0000 7f1713400640 1 -- 172.21.15.39:0/3641678174 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f171401a070 con 0x7f171c0d1600 2024-09-21T07:51:56.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.239+0000 7f1713400640 1 --2- 172.21.15.39:0/3641678174 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f16fc078b90 0x7f16fc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:56.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.240+0000 7f1713400640 1 -- 172.21.15.39:0/3641678174 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(246..246 src has 1..246) ==== 7562+0+0 (secure 0 0 0) 0x7f171409abf0 con 0x7f171c0d1600 2024-09-21T07:51:56.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.240+0000 7f171b400640 1 --2- 172.21.15.39:0/3641678174 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f16fc078b90 0x7f16fc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:56.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.241+0000 7f171b400640 1 --2- 172.21.15.39:0/3641678174 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f16fc078b90 0x7f16fc07b0a0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f170c034000 tx=0x7f170c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:56.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.244+0000 7f1713400640 1 -- 172.21.15.39:0/3641678174 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1714062290 con 0x7f171c0d1600 2024-09-21T07:51:56.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.480+0000 7f1720a00640 1 -- 172.21.15.39:0/3641678174 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f171c0d1de0 con 0x7f171c0d1600 2024-09-21T07:51:56.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.481+0000 7f1713400640 1 -- 172.21.15.39:0/3641678174 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 279 v279) ==== 95+0+5280 (secure 0 0 0) 0x7f1714066cc0 con 0x7f171c0d1600 2024-09-21T07:51:56.483 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:56.483 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":279,"btime":"2024-09-21T07:51:38:524572+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":278}],"filesystems":[{"mdsmap":{"epoch":279,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:51:37.529529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":246,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":10195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10195":{"gid":10195,"name":"d","rank":2,"incarnation":277,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[11052,9967]},"id":13}]} 2024-09-21T07:51:56.483 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 279 2024-09-21T07:51:56.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.483+0000 7f1720a00640 1 -- 172.21.15.39:0/3641678174 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f16fc078b90 msgr2=0x7f16fc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:56.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.483+0000 7f1720a00640 1 --2- 172.21.15.39:0/3641678174 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f16fc078b90 0x7f16fc07b0a0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f170c034000 tx=0x7f170c004730 comp rx=0 tx=0).stop 2024-09-21T07:51:56.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.483+0000 7f1720a00640 1 -- 172.21.15.39:0/3641678174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f171c0d1600 msgr2=0x7f171c0cbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:56.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.483+0000 7f1720a00640 1 --2- 172.21.15.39:0/3641678174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f171c0d1600 0x7f171c0cbce0 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7f171400c730 tx=0x7f171400cbf0 comp rx=0 tx=0).stop 2024-09-21T07:51:56.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.483+0000 7f1720a00640 1 -- 172.21.15.39:0/3641678174 shutdown_connections 2024-09-21T07:51:56.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.483+0000 7f1720a00640 1 --2- 172.21.15.39:0/3641678174 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f16fc078b90 0x7f16fc07b0a0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:56.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.483+0000 7f1720a00640 1 --2- 172.21.15.39:0/3641678174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f171c0d1600 0x7f171c0cbce0 unknown :-1 s=CLOSED pgs=2480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:56.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.483+0000 7f1720a00640 1 --2- 172.21.15.39:0/3641678174 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f171c0d11f0 0x7f171c0cb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:56.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.483+0000 7f1720a00640 1 --2- 172.21.15.39:0/3641678174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f171c06def0 0x7f171c15ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:56.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.483+0000 7f1720a00640 1 -- 172.21.15.39:0/3641678174 >> 172.21.15.39:0/3641678174 conn(0x7f171c0b3e40 msgr2=0x7f171c15e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:56.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.483+0000 7f1720a00640 1 -- 172.21.15.39:0/3641678174 shutdown_connections 2024-09-21T07:51:56.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:56.483+0000 7f1720a00640 1 -- 172.21.15.39:0/3641678174 wait complete. 2024-09-21T07:51:56.912 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 279, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:51:37.529529+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 246, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 9967, 'mds_1': 11052, 'mds_2': 10195}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9967': {'gid': 9967, 'name': 'b', 'rank': 0, 'incarnation': 259, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.148:6836/3381818843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 3381818843}, {'type': 'v1', 'addr': '172.21.15.148:6836', 'nonce': 3381818843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10195': {'gid': 10195, 'name': 'd', 'rank': 2, 'incarnation': 277, 'state': 'up:reconnect', 'state_seq': 11, 'addr': '172.21.15.148:6837/597301351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6835', 'nonce': 597301351}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 597301351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11052': {'gid': 11052, 'name': 'c', 'rank': 1, 'incarnation': 261, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6835/485194245', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 485194245}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 485194245}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9967, 'qdb_cluster': [11052, 9967]} 2024-09-21T07:51:56.912 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10195:up:reconnect 2024-09-21T07:51:57.912 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:51:58.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.150+0000 7fafe2e00640 1 -- 172.21.15.39:0/776028075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafdc0b3370 msgr2=0x7fafdc0b3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:58.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.150+0000 7fafe2e00640 1 --2- 172.21.15.39:0/776028075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafdc0b3370 0x7fafdc0b3780 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7fafdc06fe60 tx=0x7fafc402a4d0 comp rx=0 tx=0).stop 2024-09-21T07:51:58.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.150+0000 7fafe2e00640 1 -- 172.21.15.39:0/776028075 shutdown_connections 2024-09-21T07:51:58.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.150+0000 7fafe2e00640 1 --2- 172.21.15.39:0/776028075 >> v2:172.21.15.148:3300/0 conn(0x7fafdc151590 0x7fafdc1519a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:58.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.150+0000 7fafe2e00640 1 --2- 172.21.15.39:0/776028075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafdc0b3370 0x7fafdc0b3780 unknown :-1 s=CLOSED pgs=2481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:58.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.150+0000 7fafe2e00640 1 --2- 172.21.15.39:0/776028075 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fafdc15e590 0x7fafdc070a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:58.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.150+0000 7fafe2e00640 1 -- 172.21.15.39:0/776028075 >> 172.21.15.39:0/776028075 conn(0x7fafdc06cfc0 msgr2=0x7fafdc06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:58.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.150+0000 7fafe2e00640 1 -- 172.21.15.39:0/776028075 shutdown_connections 2024-09-21T07:51:58.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.150+0000 7fafe2e00640 1 -- 172.21.15.39:0/776028075 wait complete. 2024-09-21T07:51:58.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.153+0000 7fafe2e00640 1 Processor -- start 2024-09-21T07:51:58.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.153+0000 7fafe2e00640 1 -- start start 2024-09-21T07:51:58.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.154+0000 7fafe2e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafdc0b3370 0x7fafdc1e5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:58.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.154+0000 7fafe2e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fafdc151590 0x7fafdc1502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:58.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.154+0000 7fafe2e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafdc15e590 0x7fafdc1f6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:58.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.154+0000 7fafe2e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc15b750 con 0x7fafdc0b3370 2024-09-21T07:51:58.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.154+0000 7fafe2e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc15b5d0 con 0x7fafdc15e590 2024-09-21T07:51:58.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.154+0000 7fafe2e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fafdc15b8d0 con 0x7fafdc151590 2024-09-21T07:51:58.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.154+0000 7fafdbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafdc0b3370 0x7fafdc1e5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:58.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.154+0000 7fafdbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafdc0b3370 0x7fafdc1e5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46146/0 (socket says 172.21.15.39:46146) 2024-09-21T07:51:58.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.155+0000 7fafdbe00640 1 -- 172.21.15.39:0/234364509 learned_addr learned my addr 172.21.15.39:0/234364509 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:51:58.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.155+0000 7fafe0e00640 1 --2- 172.21.15.39:0/234364509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafdc15e590 0x7fafdc1f6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:58.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.155+0000 7fafd3e00640 1 --2- 172.21.15.39:0/234364509 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fafdc151590 0x7fafdc1502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:58.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.155+0000 7fafdbe00640 1 -- 172.21.15.39:0/234364509 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fafdc151590 msgr2=0x7fafdc1502b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:58.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.155+0000 7fafdbe00640 1 --2- 172.21.15.39:0/234364509 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fafdc151590 0x7fafdc1502b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:58.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.155+0000 7fafdbe00640 1 -- 172.21.15.39:0/234364509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafdc15e590 msgr2=0x7fafdc1f6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:58.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.155+0000 7fafdbe00640 1 --2- 172.21.15.39:0/234364509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafdc15e590 0x7fafdc1f6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:58.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.155+0000 7fafdbe00640 1 -- 172.21.15.39:0/234364509 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafdc0b1160 con 0x7fafdc0b3370 2024-09-21T07:51:58.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.155+0000 7fafdbe00640 1 --2- 172.21.15.39:0/234364509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafdc0b3370 0x7fafdc1e5430 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7fafcc00ea90 tx=0x7fafcc00ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:58.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.155+0000 7fafd8c00640 1 -- 172.21.15.39:0/234364509 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafcc00cc80 con 0x7fafdc0b3370 2024-09-21T07:51:58.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.155+0000 7fafd8c00640 1 -- 172.21.15.39:0/234364509 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fafcc005020 con 0x7fafdc0b3370 2024-09-21T07:51:58.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.155+0000 7fafe2e00640 1 -- 172.21.15.39:0/234364509 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafdc0b1440 con 0x7fafdc0b3370 2024-09-21T07:51:58.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.155+0000 7fafd8c00640 1 -- 172.21.15.39:0/234364509 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafcc005470 con 0x7fafdc0b3370 2024-09-21T07:51:58.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.156+0000 7fafe2e00640 1 -- 172.21.15.39:0/234364509 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafdc0b1a00 con 0x7fafdc0b3370 2024-09-21T07:51:58.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.157+0000 7fafe2e00640 1 -- 172.21.15.39:0/234364509 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafdc15bdf0 con 0x7fafdc0b3370 2024-09-21T07:51:58.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.158+0000 7fafd8c00640 1 -- 172.21.15.39:0/234364509 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fafcc0047a0 con 0x7fafdc0b3370 2024-09-21T07:51:58.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.159+0000 7fafd8c00640 1 --2- 172.21.15.39:0/234364509 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fafb8078b90 0x7fafb807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:58.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.159+0000 7fafd8c00640 1 -- 172.21.15.39:0/234364509 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(246..246 src has 1..246) ==== 7562+0+0 (secure 0 0 0) 0x7fafcc09b760 con 0x7fafdc0b3370 2024-09-21T07:51:58.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.160+0000 7fafd3e00640 1 --2- 172.21.15.39:0/234364509 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fafb8078b90 0x7fafb807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:58.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.160+0000 7fafd3e00640 1 --2- 172.21.15.39:0/234364509 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fafb8078b90 0x7fafb807b0a0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fafc4033000 tx=0x7fafc4004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:58.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.164+0000 7fafd8c00640 1 -- 172.21.15.39:0/234364509 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fafcc063580 con 0x7fafdc0b3370 2024-09-21T07:51:58.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.424+0000 7fafe2e00640 1 -- 172.21.15.39:0/234364509 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fafdc159470 con 0x7fafdc0b3370 2024-09-21T07:51:58.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.424+0000 7fafd8c00640 1 -- 172.21.15.39:0/234364509 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 279 v279) ==== 95+0+5280 (secure 0 0 0) 0x7fafcc062e80 con 0x7fafdc0b3370 2024-09-21T07:51:58.426 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:51:58.426 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":279,"btime":"2024-09-21T07:51:38:524572+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":278}],"filesystems":[{"mdsmap":{"epoch":279,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:51:37.529529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":246,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":10195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10195":{"gid":10195,"name":"d","rank":2,"incarnation":277,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[11052,9967]},"id":13}]} 2024-09-21T07:51:58.427 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 279 2024-09-21T07:51:58.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.426+0000 7fafe2e00640 1 -- 172.21.15.39:0/234364509 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fafb8078b90 msgr2=0x7fafb807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:58.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.426+0000 7fafe2e00640 1 --2- 172.21.15.39:0/234364509 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fafb8078b90 0x7fafb807b0a0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fafc4033000 tx=0x7fafc4004730 comp rx=0 tx=0).stop 2024-09-21T07:51:58.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.426+0000 7fafe2e00640 1 -- 172.21.15.39:0/234364509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafdc0b3370 msgr2=0x7fafdc1e5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:58.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.426+0000 7fafe2e00640 1 --2- 172.21.15.39:0/234364509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafdc0b3370 0x7fafdc1e5430 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7fafcc00ea90 tx=0x7fafcc00ef50 comp rx=0 tx=0).stop 2024-09-21T07:51:58.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.427+0000 7fafe2e00640 1 -- 172.21.15.39:0/234364509 shutdown_connections 2024-09-21T07:51:58.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.427+0000 7fafe2e00640 1 --2- 172.21.15.39:0/234364509 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fafb8078b90 0x7fafb807b0a0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:58.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.427+0000 7fafe2e00640 1 --2- 172.21.15.39:0/234364509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafdc15e590 0x7fafdc1f6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:58.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.427+0000 7fafe2e00640 1 --2- 172.21.15.39:0/234364509 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fafdc151590 0x7fafdc1502b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:58.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.427+0000 7fafe2e00640 1 --2- 172.21.15.39:0/234364509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafdc0b3370 0x7fafdc1e5430 unknown :-1 s=CLOSED pgs=2482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:58.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.427+0000 7fafe2e00640 1 -- 172.21.15.39:0/234364509 >> 172.21.15.39:0/234364509 conn(0x7fafdc06cfc0 msgr2=0x7fafdc0b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:58.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.427+0000 7fafe2e00640 1 -- 172.21.15.39:0/234364509 shutdown_connections 2024-09-21T07:51:58.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:51:58.427+0000 7fafe2e00640 1 -- 172.21.15.39:0/234364509 wait complete. 2024-09-21T07:51:58.856 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 279, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:51:37.529529+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 246, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 9967, 'mds_1': 11052, 'mds_2': 10195}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9967': {'gid': 9967, 'name': 'b', 'rank': 0, 'incarnation': 259, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.148:6836/3381818843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 3381818843}, {'type': 'v1', 'addr': '172.21.15.148:6836', 'nonce': 3381818843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10195': {'gid': 10195, 'name': 'd', 'rank': 2, 'incarnation': 277, 'state': 'up:reconnect', 'state_seq': 11, 'addr': '172.21.15.148:6837/597301351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6835', 'nonce': 597301351}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 597301351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11052': {'gid': 11052, 'name': 'c', 'rank': 1, 'incarnation': 261, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6835/485194245', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 485194245}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 485194245}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9967, 'qdb_cluster': [11052, 9967]} 2024-09-21T07:51:58.856 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10195:up:reconnect 2024-09-21T07:51:59.857 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:52:00.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.096+0000 7fef85000640 1 -- 172.21.15.39:0/2458664790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef800b5890 msgr2=0x7fef800b23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:00.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.096+0000 7fef85000640 1 --2- 172.21.15.39:0/2458664790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef800b5890 0x7fef800b23a0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fef78005fd0 tx=0x7fef7802a8a0 comp rx=0 tx=0).stop 2024-09-21T07:52:00.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.096+0000 7fef85000640 1 -- 172.21.15.39:0/2458664790 shutdown_connections 2024-09-21T07:52:00.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.096+0000 7fef85000640 1 --2- 172.21.15.39:0/2458664790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef800b5890 0x7fef800b23a0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:00.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.096+0000 7fef85000640 1 --2- 172.21.15.39:0/2458664790 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fef800d19f0 0x7fef800d4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:00.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.096+0000 7fef85000640 1 -- 172.21.15.39:0/2458664790 >> 172.21.15.39:0/2458664790 conn(0x7fef8008bfb0 msgr2=0x7fef8009d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:00.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.097+0000 7fef85000640 1 -- 172.21.15.39:0/2458664790 shutdown_connections 2024-09-21T07:52:00.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.097+0000 7fef85000640 1 -- 172.21.15.39:0/2458664790 wait complete. 2024-09-21T07:52:00.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.099+0000 7fef85000640 1 Processor -- start 2024-09-21T07:52:00.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.099+0000 7fef85000640 1 -- start start 2024-09-21T07:52:00.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.101+0000 7fef85000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef800b5890 0x7fef800c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:00.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.101+0000 7fef85000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fef800d19f0 0x7fef80154c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:00.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.101+0000 7fef85000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef800d4b40 0x7fef801f8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:00.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.101+0000 7fef85000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fef800b5e20 con 0x7fef800d4b40 2024-09-21T07:52:00.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.101+0000 7fef85000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fef800b5ca0 con 0x7fef800b5890 2024-09-21T07:52:00.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.101+0000 7fef85000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fef800b28e0 con 0x7fef800d19f0 2024-09-21T07:52:00.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.101+0000 7fef7fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef800d4b40 0x7fef801f8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:00.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.101+0000 7fef7fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef800d4b40 0x7fef801f8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46158/0 (socket says 172.21.15.39:46158) 2024-09-21T07:52:00.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.101+0000 7fef7fe00640 1 -- 172.21.15.39:0/3185692945 learned_addr learned my addr 172.21.15.39:0/3185692945 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:00.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.101+0000 7fef7f400640 1 --2- 172.21.15.39:0/3185692945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef800b5890 0x7fef800c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:00.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.101+0000 7fef76a00640 1 --2- 172.21.15.39:0/3185692945 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fef800d19f0 0x7fef80154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:00.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.101+0000 7fef7fe00640 1 -- 172.21.15.39:0/3185692945 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fef800d19f0 msgr2=0x7fef80154c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:00.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.101+0000 7fef7fe00640 1 --2- 172.21.15.39:0/3185692945 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fef800d19f0 0x7fef80154c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:00.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.101+0000 7fef7fe00640 1 -- 172.21.15.39:0/3185692945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef800b5890 msgr2=0x7fef800c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:00.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.101+0000 7fef7fe00640 1 --2- 172.21.15.39:0/3185692945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef800b5890 0x7fef800c8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:00.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.101+0000 7fef7fe00640 1 -- 172.21.15.39:0/3185692945 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef800c8af0 con 0x7fef800d4b40 2024-09-21T07:52:00.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.101+0000 7fef76a00640 1 --2- 172.21.15.39:0/3185692945 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fef800d19f0 0x7fef80154c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:52:00.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.102+0000 7fef7fe00640 1 --2- 172.21.15.39:0/3185692945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef800d4b40 0x7fef801f8f60 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7fef78009b30 tx=0x7fef78032d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:00.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.102+0000 7fef77e00640 1 -- 172.21.15.39:0/3185692945 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef78041070 con 0x7fef800d4b40 2024-09-21T07:52:00.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.102+0000 7fef77e00640 1 -- 172.21.15.39:0/3185692945 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef78004450 con 0x7fef800d4b40 2024-09-21T07:52:00.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.102+0000 7fef85000640 1 -- 172.21.15.39:0/3185692945 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef800c8dd0 con 0x7fef800d4b40 2024-09-21T07:52:00.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.102+0000 7fef77e00640 1 -- 172.21.15.39:0/3185692945 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef78031430 con 0x7fef800d4b40 2024-09-21T07:52:00.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.102+0000 7fef85000640 1 -- 172.21.15.39:0/3185692945 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef800a0ba0 con 0x7fef800d4b40 2024-09-21T07:52:00.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.103+0000 7fef85000640 1 -- 172.21.15.39:0/3185692945 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef800c8860 con 0x7fef800d4b40 2024-09-21T07:52:00.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.105+0000 7fef77e00640 1 -- 172.21.15.39:0/3185692945 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fef78044070 con 0x7fef800d4b40 2024-09-21T07:52:00.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.106+0000 7fef77e00640 1 --2- 172.21.15.39:0/3185692945 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fef5c078bc0 0x7fef5c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:00.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.106+0000 7fef77e00640 1 -- 172.21.15.39:0/3185692945 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(246..246 src has 1..246) ==== 7562+0+0 (secure 0 0 0) 0x7fef7802e020 con 0x7fef800d4b40 2024-09-21T07:52:00.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.106+0000 7fef7f400640 1 --2- 172.21.15.39:0/3185692945 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fef5c078bc0 0x7fef5c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:00.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.107+0000 7fef7f400640 1 --2- 172.21.15.39:0/3185692945 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fef5c078bc0 0x7fef5c07b0d0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fef70009590 tx=0x7fef70005f30 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:00.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.110+0000 7fef77e00640 1 -- 172.21.15.39:0/3185692945 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fef7804b070 con 0x7fef800d4b40 2024-09-21T07:52:00.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.352+0000 7fef85000640 1 -- 172.21.15.39:0/3185692945 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef80069a50 con 0x7fef800d4b40 2024-09-21T07:52:00.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.352+0000 7fef77e00640 1 -- 172.21.15.39:0/3185692945 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 279 v279) ==== 95+0+5280 (secure 0 0 0) 0x7fef78083e20 con 0x7fef800d4b40 2024-09-21T07:52:00.354 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:52:00.354 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":279,"btime":"2024-09-21T07:51:38:524572+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":278}],"filesystems":[{"mdsmap":{"epoch":279,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:51:37.529529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":246,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":10195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10195":{"gid":10195,"name":"d","rank":2,"incarnation":277,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[11052,9967]},"id":13}]} 2024-09-21T07:52:00.355 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 279 2024-09-21T07:52:00.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.354+0000 7fef85000640 1 -- 172.21.15.39:0/3185692945 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fef5c078bc0 msgr2=0x7fef5c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:00.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.354+0000 7fef85000640 1 --2- 172.21.15.39:0/3185692945 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fef5c078bc0 0x7fef5c07b0d0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fef70009590 tx=0x7fef70005f30 comp rx=0 tx=0).stop 2024-09-21T07:52:00.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.354+0000 7fef85000640 1 -- 172.21.15.39:0/3185692945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef800d4b40 msgr2=0x7fef801f8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:00.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.354+0000 7fef85000640 1 --2- 172.21.15.39:0/3185692945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef800d4b40 0x7fef801f8f60 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7fef78009b30 tx=0x7fef78032d30 comp rx=0 tx=0).stop 2024-09-21T07:52:00.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.355+0000 7fef85000640 1 -- 172.21.15.39:0/3185692945 shutdown_connections 2024-09-21T07:52:00.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.355+0000 7fef85000640 1 --2- 172.21.15.39:0/3185692945 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fef5c078bc0 0x7fef5c07b0d0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:00.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.355+0000 7fef85000640 1 --2- 172.21.15.39:0/3185692945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef800d4b40 0x7fef801f8f60 unknown :-1 s=CLOSED pgs=2484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:00.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.355+0000 7fef85000640 1 --2- 172.21.15.39:0/3185692945 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fef800d19f0 0x7fef80154c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:00.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.355+0000 7fef85000640 1 --2- 172.21.15.39:0/3185692945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef800b5890 0x7fef800c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:00.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.355+0000 7fef85000640 1 -- 172.21.15.39:0/3185692945 >> 172.21.15.39:0/3185692945 conn(0x7fef8008bfb0 msgr2=0x7fef8006e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:00.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.355+0000 7fef85000640 1 -- 172.21.15.39:0/3185692945 shutdown_connections 2024-09-21T07:52:00.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:00.355+0000 7fef85000640 1 -- 172.21.15.39:0/3185692945 wait complete. 2024-09-21T07:52:00.784 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 279, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:51:37.529529+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 246, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 9967, 'mds_1': 11052, 'mds_2': 10195}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9967': {'gid': 9967, 'name': 'b', 'rank': 0, 'incarnation': 259, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.148:6836/3381818843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 3381818843}, {'type': 'v1', 'addr': '172.21.15.148:6836', 'nonce': 3381818843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10195': {'gid': 10195, 'name': 'd', 'rank': 2, 'incarnation': 277, 'state': 'up:reconnect', 'state_seq': 11, 'addr': '172.21.15.148:6837/597301351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6835', 'nonce': 597301351}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 597301351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11052': {'gid': 11052, 'name': 'c', 'rank': 1, 'incarnation': 261, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6835/485194245', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 485194245}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 485194245}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9967, 'qdb_cluster': [11052, 9967]} 2024-09-21T07:52:00.785 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10195:up:reconnect 2024-09-21T07:52:01.785 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:52:02.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.022+0000 7f5fd3600640 1 -- 172.21.15.39:0/78588287 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5fcc0cfd10 msgr2=0x7f5fcc0cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:02.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.022+0000 7f5fd3600640 1 --2- 172.21.15.39:0/78588287 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5fcc0cfd10 0x7f5fcc0cd8e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8005fd0 tx=0x7f5fb802a5d0 comp rx=0 tx=0).stop 2024-09-21T07:52:02.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.022+0000 7f5fd3600640 1 -- 172.21.15.39:0/78588287 shutdown_connections 2024-09-21T07:52:02.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.022+0000 7f5fd3600640 1 --2- 172.21.15.39:0/78588287 >> v2:172.21.15.148:3300/0 conn(0x7f5fcc0d0620 0x7f5fcc0d0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:02.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.022+0000 7f5fd3600640 1 --2- 172.21.15.39:0/78588287 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5fcc0cfd10 0x7f5fcc0cd8e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:02.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.022+0000 7f5fd3600640 1 -- 172.21.15.39:0/78588287 >> 172.21.15.39:0/78588287 conn(0x7f5fcc06cfc0 msgr2=0x7f5fcc06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:02.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.022+0000 7f5fd3600640 1 -- 172.21.15.39:0/78588287 shutdown_connections 2024-09-21T07:52:02.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.022+0000 7f5fd3600640 1 -- 172.21.15.39:0/78588287 wait complete. 2024-09-21T07:52:02.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.024+0000 7f5fd3600640 1 Processor -- start 2024-09-21T07:52:02.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.024+0000 7f5fd3600640 1 -- start start 2024-09-21T07:52:02.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.026+0000 7f5fd3600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5fcc0cfd10 0x7f5fcc13f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:02.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.026+0000 7f5fd3600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5fcc0d0620 0x7f5fcc0ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:02.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.026+0000 7f5fd3600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fcc15cd30 0x7f5fcc0aed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:02.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.026+0000 7f5fd3600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc073890 con 0x7f5fcc15cd30 2024-09-21T07:52:02.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.026+0000 7f5fd3600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc073710 con 0x7f5fcc0cfd10 2024-09-21T07:52:02.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.026+0000 7f5fd3600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f5fcc073a10 con 0x7f5fcc0d0620 2024-09-21T07:52:02.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.026+0000 7f5fd1600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fcc15cd30 0x7f5fcc0aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:02.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.026+0000 7f5fd1600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fcc15cd30 0x7f5fcc0aed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37442/0 (socket says 172.21.15.39:37442) 2024-09-21T07:52:02.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.026+0000 7f5fd1600640 1 -- 172.21.15.39:0/3024043741 learned_addr learned my addr 172.21.15.39:0/3024043741 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:02.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.027+0000 7f5fcbe00640 1 --2- 172.21.15.39:0/3024043741 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5fcc0d0620 0x7f5fcc0ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:02.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.027+0000 7f5fd0c00640 1 --2- 172.21.15.39:0/3024043741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5fcc0cfd10 0x7f5fcc13f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:02.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.027+0000 7f5fd1600640 1 -- 172.21.15.39:0/3024043741 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5fcc0d0620 msgr2=0x7f5fcc0ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:02.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.027+0000 7f5fd1600640 1 --2- 172.21.15.39:0/3024043741 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5fcc0d0620 0x7f5fcc0ae7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:02.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.027+0000 7f5fd1600640 1 -- 172.21.15.39:0/3024043741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5fcc0cfd10 msgr2=0x7f5fcc13f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:02.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.027+0000 7f5fd1600640 1 --2- 172.21.15.39:0/3024043741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5fcc0cfd10 0x7f5fcc13f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:02.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.027+0000 7f5fd1600640 1 -- 172.21.15.39:0/3024043741 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fcc1f3ba0 con 0x7f5fcc15cd30 2024-09-21T07:52:02.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.027+0000 7f5fd1600640 1 --2- 172.21.15.39:0/3024043741 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fcc15cd30 0x7f5fcc0aed20 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7f5fc400eab0 tx=0x7f5fc400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:02.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.027+0000 7f5fc8c00640 1 -- 172.21.15.39:0/3024043741 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc400ce40 con 0x7f5fcc15cd30 2024-09-21T07:52:02.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.027+0000 7f5fc8c00640 1 -- 172.21.15.39:0/3024043741 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5fc4004b20 con 0x7f5fcc15cd30 2024-09-21T07:52:02.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.027+0000 7f5fd3600640 1 -- 172.21.15.39:0/3024043741 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fcc1f3e80 con 0x7f5fcc15cd30 2024-09-21T07:52:02.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.027+0000 7f5fc8c00640 1 -- 172.21.15.39:0/3024043741 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc4005020 con 0x7f5fcc15cd30 2024-09-21T07:52:02.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.027+0000 7f5fd3600640 1 -- 172.21.15.39:0/3024043741 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fcc1f4440 con 0x7f5fcc15cd30 2024-09-21T07:52:02.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.029+0000 7f5fd3600640 1 -- 172.21.15.39:0/3024043741 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fcc1f3910 con 0x7f5fcc15cd30 2024-09-21T07:52:02.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.030+0000 7f5fc8c00640 1 -- 172.21.15.39:0/3024043741 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5fc40051c0 con 0x7f5fcc15cd30 2024-09-21T07:52:02.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.031+0000 7f5fc8c00640 1 --2- 172.21.15.39:0/3024043741 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5fac078b90 0x7f5fac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:02.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.031+0000 7f5fc8c00640 1 -- 172.21.15.39:0/3024043741 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(246..246 src has 1..246) ==== 7562+0+0 (secure 0 0 0) 0x7f5fc409c000 con 0x7f5fcc15cd30 2024-09-21T07:52:02.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.031+0000 7f5fd0c00640 1 --2- 172.21.15.39:0/3024043741 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5fac078b90 0x7f5fac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:02.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.032+0000 7f5fd0c00640 1 --2- 172.21.15.39:0/3024043741 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5fac078b90 0x7f5fac07b0a0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0007fd0 tx=0x7f5fc0009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:02.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.035+0000 7f5fc8c00640 1 -- 172.21.15.39:0/3024043741 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5fc4063e20 con 0x7f5fcc15cd30 2024-09-21T07:52:02.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.270+0000 7f5fd3600640 1 -- 172.21.15.39:0/3024043741 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5fcc073f30 con 0x7f5fcc15cd30 2024-09-21T07:52:02.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.271+0000 7f5fc8c00640 1 -- 172.21.15.39:0/3024043741 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 279 v279) ==== 95+0+5280 (secure 0 0 0) 0x7f5fc4063720 con 0x7f5fcc15cd30 2024-09-21T07:52:02.273 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:52:02.273 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":279,"btime":"2024-09-21T07:51:38:524572+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":278}],"filesystems":[{"mdsmap":{"epoch":279,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:51:37.529529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":246,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":10195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10195":{"gid":10195,"name":"d","rank":2,"incarnation":277,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[11052,9967]},"id":13}]} 2024-09-21T07:52:02.273 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 279 2024-09-21T07:52:02.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.273+0000 7f5fd3600640 1 -- 172.21.15.39:0/3024043741 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5fac078b90 msgr2=0x7f5fac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:02.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.273+0000 7f5fd3600640 1 --2- 172.21.15.39:0/3024043741 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5fac078b90 0x7f5fac07b0a0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0007fd0 tx=0x7f5fc0009040 comp rx=0 tx=0).stop 2024-09-21T07:52:02.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.273+0000 7f5fd3600640 1 -- 172.21.15.39:0/3024043741 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fcc15cd30 msgr2=0x7f5fcc0aed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:02.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.273+0000 7f5fd3600640 1 --2- 172.21.15.39:0/3024043741 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fcc15cd30 0x7f5fcc0aed20 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7f5fc400eab0 tx=0x7f5fc400ef70 comp rx=0 tx=0).stop 2024-09-21T07:52:02.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.273+0000 7f5fd3600640 1 -- 172.21.15.39:0/3024043741 shutdown_connections 2024-09-21T07:52:02.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.273+0000 7f5fd3600640 1 --2- 172.21.15.39:0/3024043741 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5fac078b90 0x7f5fac07b0a0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:02.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.273+0000 7f5fd3600640 1 --2- 172.21.15.39:0/3024043741 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fcc15cd30 0x7f5fcc0aed20 unknown :-1 s=CLOSED pgs=2485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:02.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.273+0000 7f5fd3600640 1 --2- 172.21.15.39:0/3024043741 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5fcc0d0620 0x7f5fcc0ae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:02.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.273+0000 7f5fd3600640 1 --2- 172.21.15.39:0/3024043741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5fcc0cfd10 0x7f5fcc13f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:02.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.273+0000 7f5fd3600640 1 -- 172.21.15.39:0/3024043741 >> 172.21.15.39:0/3024043741 conn(0x7f5fcc06cfc0 msgr2=0x7f5fcc15b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:02.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.274+0000 7f5fd3600640 1 -- 172.21.15.39:0/3024043741 shutdown_connections 2024-09-21T07:52:02.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:02.274+0000 7f5fd3600640 1 -- 172.21.15.39:0/3024043741 wait complete. 2024-09-21T07:52:02.703 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 279, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:51:37.529529+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 246, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 9967, 'mds_1': 11052, 'mds_2': 10195}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9967': {'gid': 9967, 'name': 'b', 'rank': 0, 'incarnation': 259, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.148:6836/3381818843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 3381818843}, {'type': 'v1', 'addr': '172.21.15.148:6836', 'nonce': 3381818843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10195': {'gid': 10195, 'name': 'd', 'rank': 2, 'incarnation': 277, 'state': 'up:reconnect', 'state_seq': 11, 'addr': '172.21.15.148:6837/597301351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6835', 'nonce': 597301351}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 597301351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11052': {'gid': 11052, 'name': 'c', 'rank': 1, 'incarnation': 261, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6835/485194245', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 485194245}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 485194245}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9967, 'qdb_cluster': [11052, 9967]} 2024-09-21T07:52:02.703 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10195:up:reconnect 2024-09-21T07:52:03.704 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:52:03.730 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:52:03.755 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:52:03.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:03.942+0000 7f392e600640 1 -- 172.21.15.39:0/3612131252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f392806cde0 msgr2=0x7f392806d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:03.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:03.942+0000 7f392e600640 1 --2- 172.21.15.39:0/3612131252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f392806cde0 0x7f392806d1f0 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7f39280cece0 tx=0x7f392002aac0 comp rx=0 tx=0).stop 2024-09-21T07:52:03.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:03.943+0000 7f392e600640 1 -- 172.21.15.39:0/3612131252 shutdown_connections 2024-09-21T07:52:03.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:03.943+0000 7f392e600640 1 --2- 172.21.15.39:0/3612131252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f392806cde0 0x7f392806d1f0 unknown :-1 s=CLOSED pgs=2487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:03.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:03.943+0000 7f392e600640 1 --2- 172.21.15.39:0/3612131252 >> v2:172.21.15.148:3300/0 conn(0x7f39280cd770 0x7f392815c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:03.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:03.943+0000 7f392e600640 1 -- 172.21.15.39:0/3612131252 >> 172.21.15.39:0/3612131252 conn(0x7f39280b2d40 msgr2=0x7f39280b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:03.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:03.943+0000 7f392e600640 1 -- 172.21.15.39:0/3612131252 shutdown_connections 2024-09-21T07:52:03.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:03.943+0000 7f392e600640 1 -- 172.21.15.39:0/3612131252 wait complete. 2024-09-21T07:52:03.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:03.945+0000 7f392e600640 1 Processor -- start 2024-09-21T07:52:03.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:03.945+0000 7f392e600640 1 -- start start 2024-09-21T07:52:03.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:03.946+0000 7f392e600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f392806cde0 0x7f39281f84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:03.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:03.948+0000 7f392e600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39280c4880 0x7f39281f8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:03.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:03.948+0000 7f392e600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39280cd770 0x7f39280af480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:03.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:03.948+0000 7f392e600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f39280b41c0 con 0x7f39280cd770 2024-09-21T07:52:03.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:03.948+0000 7f392e600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f39280b4040 con 0x7f39280c4880 2024-09-21T07:52:03.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:03.948+0000 7f392e600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f39280b4340 con 0x7f392806cde0 2024-09-21T07:52:03.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:03.948+0000 7f392dc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39280cd770 0x7f39280af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:03.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:03.948+0000 7f392dc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39280cd770 0x7f39280af480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37460/0 (socket says 172.21.15.39:37460) 2024-09-21T07:52:03.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:03.948+0000 7f392dc00640 1 -- 172.21.15.39:0/2038392538 learned_addr learned my addr 172.21.15.39:0/2038392538 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:03.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:03.948+0000 7f3927e00640 1 --2- 172.21.15.39:0/2038392538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39280c4880 0x7f39281f8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:03.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:03.948+0000 7f392d200640 1 --2- 172.21.15.39:0/2038392538 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f392806cde0 0x7f39281f84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:03.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:03.948+0000 7f392dc00640 1 -- 172.21.15.39:0/2038392538 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f392806cde0 msgr2=0x7f39281f84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:03.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:03.948+0000 7f392dc00640 1 --2- 172.21.15.39:0/2038392538 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f392806cde0 0x7f39281f84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:03.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:03.948+0000 7f392dc00640 1 -- 172.21.15.39:0/2038392538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39280c4880 msgr2=0x7f39281f8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:03.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:03.948+0000 7f392dc00640 1 --2- 172.21.15.39:0/2038392538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39280c4880 0x7f39281f8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:03.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:03.948+0000 7f392dc00640 1 -- 172.21.15.39:0/2038392538 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39281f9950 con 0x7f39280cd770 2024-09-21T07:52:03.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:03.948+0000 7f392dc00640 1 --2- 172.21.15.39:0/2038392538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39280cd770 0x7f39280af480 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7f392002b0b0 tx=0x7f3920003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:03.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:03.948+0000 7f3924c00640 1 -- 172.21.15.39:0/2038392538 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3920003e50 con 0x7f39280cd770 2024-09-21T07:52:03.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:03.948+0000 7f3924c00640 1 -- 172.21.15.39:0/2038392538 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3920004250 con 0x7f39280cd770 2024-09-21T07:52:03.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:03.948+0000 7f392e600640 1 -- 172.21.15.39:0/2038392538 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39281f9c30 con 0x7f39280cd770 2024-09-21T07:52:03.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:03.948+0000 7f3924c00640 1 -- 172.21.15.39:0/2038392538 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3920004680 con 0x7f39280cd770 2024-09-21T07:52:03.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:03.948+0000 7f392e600640 1 -- 172.21.15.39:0/2038392538 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f392815e170 con 0x7f39280cd770 2024-09-21T07:52:03.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:03.949+0000 7f392e600640 1 -- 172.21.15.39:0/2038392538 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39280b48d0 con 0x7f39280cd770 2024-09-21T07:52:03.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:03.950+0000 7f3924c00640 1 -- 172.21.15.39:0/2038392538 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3920049430 con 0x7f39280cd770 2024-09-21T07:52:03.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:03.951+0000 7f3924c00640 1 --2- 172.21.15.39:0/2038392538 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3908078d30 0x7f390807b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:03.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:03.951+0000 7f3924c00640 1 -- 172.21.15.39:0/2038392538 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(246..246 src has 1..246) ==== 7562+0+0 (secure 0 0 0) 0x7f39200bc150 con 0x7f39280cd770 2024-09-21T07:52:03.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:03.952+0000 7f392d200640 1 --2- 172.21.15.39:0/2038392538 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3908078d30 0x7f390807b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:03.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:03.953+0000 7f392d200640 1 --2- 172.21.15.39:0/2038392538 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3908078d30 0x7f390807b240 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f391c009b30 tx=0x7f391c005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:03.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:03.956+0000 7f3924c00640 1 -- 172.21.15.39:0/2038392538 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3920032030 con 0x7f39280cd770 2024-09-21T07:52:04.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:04.192+0000 7f392e600640 1 -- 172.21.15.39:0/2038392538 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3928074b30 con 0x7f39280cd770 2024-09-21T07:52:04.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:04.193+0000 7f3924c00640 1 -- 172.21.15.39:0/2038392538 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 279 v279) ==== 95+0+5280 (secure 0 0 0) 0x7f3920004820 con 0x7f39280cd770 2024-09-21T07:52:04.194 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:52:04.194 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":279,"btime":"2024-09-21T07:51:38:524572+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":278}],"filesystems":[{"mdsmap":{"epoch":279,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:51:37.529529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":246,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":10195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10195":{"gid":10195,"name":"d","rank":2,"incarnation":277,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[11052,9967]},"id":13}]} 2024-09-21T07:52:04.194 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 279 2024-09-21T07:52:04.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:04.195+0000 7f392e600640 1 -- 172.21.15.39:0/2038392538 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3908078d30 msgr2=0x7f390807b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:04.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:04.195+0000 7f392e600640 1 --2- 172.21.15.39:0/2038392538 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3908078d30 0x7f390807b240 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f391c009b30 tx=0x7f391c005f90 comp rx=0 tx=0).stop 2024-09-21T07:52:04.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:04.195+0000 7f392e600640 1 -- 172.21.15.39:0/2038392538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39280cd770 msgr2=0x7f39280af480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:04.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:04.195+0000 7f392e600640 1 --2- 172.21.15.39:0/2038392538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39280cd770 0x7f39280af480 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7f392002b0b0 tx=0x7f3920003ce0 comp rx=0 tx=0).stop 2024-09-21T07:52:04.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:04.196+0000 7f392e600640 1 -- 172.21.15.39:0/2038392538 shutdown_connections 2024-09-21T07:52:04.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:04.196+0000 7f392e600640 1 --2- 172.21.15.39:0/2038392538 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3908078d30 0x7f390807b240 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:04.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:04.196+0000 7f392e600640 1 --2- 172.21.15.39:0/2038392538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39280cd770 0x7f39280af480 unknown :-1 s=CLOSED pgs=2488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:04.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:04.196+0000 7f392e600640 1 --2- 172.21.15.39:0/2038392538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39280c4880 0x7f39281f8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:04.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:04.196+0000 7f392e600640 1 --2- 172.21.15.39:0/2038392538 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f392806cde0 0x7f39281f84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:04.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:04.196+0000 7f392e600640 1 -- 172.21.15.39:0/2038392538 >> 172.21.15.39:0/2038392538 conn(0x7f39280b2d40 msgr2=0x7f392815bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:04.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:04.196+0000 7f392e600640 1 -- 172.21.15.39:0/2038392538 shutdown_connections 2024-09-21T07:52:04.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:04.196+0000 7f392e600640 1 -- 172.21.15.39:0/2038392538 wait complete. 2024-09-21T07:52:04.624 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 279, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:51:37.529529+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 246, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 9967, 'mds_1': 11052, 'mds_2': 10195}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9967': {'gid': 9967, 'name': 'b', 'rank': 0, 'incarnation': 259, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.148:6836/3381818843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 3381818843}, {'type': 'v1', 'addr': '172.21.15.148:6836', 'nonce': 3381818843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10195': {'gid': 10195, 'name': 'd', 'rank': 2, 'incarnation': 277, 'state': 'up:reconnect', 'state_seq': 11, 'addr': '172.21.15.148:6837/597301351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6835', 'nonce': 597301351}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 597301351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11052': {'gid': 11052, 'name': 'c', 'rank': 1, 'incarnation': 261, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6835/485194245', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 485194245}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 485194245}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9967, 'qdb_cluster': [11052, 9967]} 2024-09-21T07:52:04.624 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10195:up:reconnect 2024-09-21T07:52:05.625 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:52:05.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:05.862+0000 7f8a11200640 1 -- 172.21.15.39:0/2865480810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a0c06f800 msgr2=0x7f8a0c0cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:05.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:05.862+0000 7f8a11200640 1 --2- 172.21.15.39:0/2865480810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a0c06f800 0x7f8a0c0cd8e0 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x7f89f4005fd0 tx=0x7f89f402a5d0 comp rx=0 tx=0).stop 2024-09-21T07:52:05.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:05.863+0000 7f8a11200640 1 -- 172.21.15.39:0/2865480810 shutdown_connections 2024-09-21T07:52:05.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:05.863+0000 7f8a11200640 1 --2- 172.21.15.39:0/2865480810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a0c06f800 0x7f8a0c0cd8e0 unknown :-1 s=CLOSED pgs=2489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:05.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:05.863+0000 7f8a11200640 1 --2- 172.21.15.39:0/2865480810 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a0c15cd30 0x7f8a0c0cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:05.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:05.863+0000 7f8a11200640 1 -- 172.21.15.39:0/2865480810 >> 172.21.15.39:0/2865480810 conn(0x7f8a0c06cfc0 msgr2=0x7f8a0c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:05.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:05.863+0000 7f8a11200640 1 -- 172.21.15.39:0/2865480810 shutdown_connections 2024-09-21T07:52:05.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:05.863+0000 7f8a11200640 1 -- 172.21.15.39:0/2865480810 wait complete. 2024-09-21T07:52:05.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:05.865+0000 7f8a11200640 1 Processor -- start 2024-09-21T07:52:05.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:05.865+0000 7f8a11200640 1 -- start start 2024-09-21T07:52:05.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:05.867+0000 7f8a11200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a0c06f800 0x7f8a0c14d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:05.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:05.867+0000 7f8a11200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a0c06fc10 0x7f8a0c1e92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:05.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:05.867+0000 7f8a11200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a0c15cd30 0x7f8a0c1e9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:05.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:05.867+0000 7f8a11200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c073890 con 0x7f8a0c06f800 2024-09-21T07:52:05.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:05.867+0000 7f8a11200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c073710 con 0x7f8a0c06fc10 2024-09-21T07:52:05.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:05.867+0000 7f8a11200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f8a0c073a10 con 0x7f8a0c15cd30 2024-09-21T07:52:05.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:05.867+0000 7f8a09e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a0c06f800 0x7f8a0c14d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:05.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:05.867+0000 7f8a09e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a0c06f800 0x7f8a0c14d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37486/0 (socket says 172.21.15.39:37486) 2024-09-21T07:52:05.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:05.867+0000 7f8a09e00640 1 -- 172.21.15.39:0/274194084 learned_addr learned my addr 172.21.15.39:0/274194084 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:05.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:05.867+0000 7f8a03e00640 1 --2- 172.21.15.39:0/274194084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a0c06fc10 0x7f8a0c1e92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:05.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:05.867+0000 7f8a0a800640 1 --2- 172.21.15.39:0/274194084 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a0c15cd30 0x7f8a0c1e9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:05.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:05.867+0000 7f8a09e00640 1 -- 172.21.15.39:0/274194084 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a0c15cd30 msgr2=0x7f8a0c1e9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:05.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:05.867+0000 7f8a09e00640 1 --2- 172.21.15.39:0/274194084 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a0c15cd30 0x7f8a0c1e9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:05.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:05.867+0000 7f8a09e00640 1 -- 172.21.15.39:0/274194084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a0c06fc10 msgr2=0x7f8a0c1e92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:05.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:05.867+0000 7f8a09e00640 1 --2- 172.21.15.39:0/274194084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a0c06fc10 0x7f8a0c1e92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:05.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:05.867+0000 7f8a09e00640 1 -- 172.21.15.39:0/274194084 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0c14d9c0 con 0x7f8a0c06f800 2024-09-21T07:52:05.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:05.868+0000 7f8a09e00640 1 --2- 172.21.15.39:0/274194084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a0c06f800 0x7f8a0c14d480 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00e790 tx=0x7f89fc00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:05.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:05.868+0000 7f8a02000640 1 -- 172.21.15.39:0/274194084 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc00eea0 con 0x7f8a0c06f800 2024-09-21T07:52:05.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:05.868+0000 7f8a02000640 1 -- 172.21.15.39:0/274194084 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89fc00cdb0 con 0x7f8a0c06f800 2024-09-21T07:52:05.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:05.868+0000 7f8a02000640 1 -- 172.21.15.39:0/274194084 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc010690 con 0x7f8a0c06f800 2024-09-21T07:52:05.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:05.868+0000 7f8a11200640 1 -- 172.21.15.39:0/274194084 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a0c0af120 con 0x7f8a0c06f800 2024-09-21T07:52:05.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:05.868+0000 7f8a11200640 1 -- 172.21.15.39:0/274194084 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a0c0b38d0 con 0x7f8a0c06f800 2024-09-21T07:52:05.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:05.869+0000 7f8a11200640 1 -- 172.21.15.39:0/274194084 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a0c073ec0 con 0x7f8a0c06f800 2024-09-21T07:52:05.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:05.870+0000 7f8a02000640 1 -- 172.21.15.39:0/274194084 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f89fc010830 con 0x7f8a0c06f800 2024-09-21T07:52:05.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:05.871+0000 7f8a02000640 1 --2- 172.21.15.39:0/274194084 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f89e8078b90 0x7f89e807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:05.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:05.871+0000 7f8a02000640 1 -- 172.21.15.39:0/274194084 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(246..246 src has 1..246) ==== 7562+0+0 (secure 0 0 0) 0x7f89fc09bf70 con 0x7f8a0c06f800 2024-09-21T07:52:05.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:05.872+0000 7f8a03e00640 1 --2- 172.21.15.39:0/274194084 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f89e8078b90 0x7f89e807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:05.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:05.873+0000 7f8a03e00640 1 --2- 172.21.15.39:0/274194084 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f89e8078b90 0x7f89e807b0a0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f89f4033000 tx=0x7f89f4004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:05.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:05.876+0000 7f8a02000640 1 -- 172.21.15.39:0/274194084 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8a0c14d9c0 con 0x7f8a0c06f800 2024-09-21T07:52:06.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:06.114+0000 7f8a11200640 1 -- 172.21.15.39:0/274194084 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a0c06d770 con 0x7f8a0c06f800 2024-09-21T07:52:06.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:06.115+0000 7f8a02000640 1 -- 172.21.15.39:0/274194084 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 279 v279) ==== 95+0+5280 (secure 0 0 0) 0x7f89fc063bd0 con 0x7f8a0c06f800 2024-09-21T07:52:06.116 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:52:06.116 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":279,"btime":"2024-09-21T07:51:38:524572+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":278}],"filesystems":[{"mdsmap":{"epoch":279,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:51:37.529529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":246,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":10195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10195":{"gid":10195,"name":"d","rank":2,"incarnation":277,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[11052,9967]},"id":13}]} 2024-09-21T07:52:06.116 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 279 2024-09-21T07:52:06.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:06.117+0000 7f8a11200640 1 -- 172.21.15.39:0/274194084 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f89e8078b90 msgr2=0x7f89e807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:06.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:06.117+0000 7f8a11200640 1 --2- 172.21.15.39:0/274194084 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f89e8078b90 0x7f89e807b0a0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f89f4033000 tx=0x7f89f4004730 comp rx=0 tx=0).stop 2024-09-21T07:52:06.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:06.117+0000 7f8a11200640 1 -- 172.21.15.39:0/274194084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a0c06f800 msgr2=0x7f8a0c14d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:06.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:06.117+0000 7f8a11200640 1 --2- 172.21.15.39:0/274194084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a0c06f800 0x7f8a0c14d480 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00e790 tx=0x7f89fc00ec50 comp rx=0 tx=0).stop 2024-09-21T07:52:06.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:06.117+0000 7f8a11200640 1 -- 172.21.15.39:0/274194084 shutdown_connections 2024-09-21T07:52:06.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:06.117+0000 7f8a11200640 1 --2- 172.21.15.39:0/274194084 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f89e8078b90 0x7f89e807b0a0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:06.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:06.117+0000 7f8a11200640 1 --2- 172.21.15.39:0/274194084 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8a0c15cd30 0x7f8a0c1e9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:06.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:06.117+0000 7f8a11200640 1 --2- 172.21.15.39:0/274194084 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a0c06fc10 0x7f8a0c1e92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:06.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:06.117+0000 7f8a11200640 1 --2- 172.21.15.39:0/274194084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a0c06f800 0x7f8a0c14d480 unknown :-1 s=CLOSED pgs=2490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:06.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:06.117+0000 7f8a11200640 1 -- 172.21.15.39:0/274194084 >> 172.21.15.39:0/274194084 conn(0x7f8a0c06cfc0 msgr2=0x7f8a0c15b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:06.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:06.117+0000 7f8a11200640 1 -- 172.21.15.39:0/274194084 shutdown_connections 2024-09-21T07:52:06.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:06.117+0000 7f8a11200640 1 -- 172.21.15.39:0/274194084 wait complete. 2024-09-21T07:52:06.543 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 279, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:51:37.529529+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 246, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 9967, 'mds_1': 11052, 'mds_2': 10195}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9967': {'gid': 9967, 'name': 'b', 'rank': 0, 'incarnation': 259, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.148:6836/3381818843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 3381818843}, {'type': 'v1', 'addr': '172.21.15.148:6836', 'nonce': 3381818843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10195': {'gid': 10195, 'name': 'd', 'rank': 2, 'incarnation': 277, 'state': 'up:reconnect', 'state_seq': 11, 'addr': '172.21.15.148:6837/597301351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6835', 'nonce': 597301351}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 597301351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11052': {'gid': 11052, 'name': 'c', 'rank': 1, 'incarnation': 261, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6835/485194245', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 485194245}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 485194245}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9967, 'qdb_cluster': [11052, 9967]} 2024-09-21T07:52:06.543 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10195:up:reconnect 2024-09-21T07:52:07.544 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:52:07.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:07.782+0000 7f1cd7800640 1 -- 172.21.15.39:0/1277197898 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cd006fa30 msgr2=0x7f1cd00d06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:07.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:07.782+0000 7f1cd7800640 1 --2- 172.21.15.39:0/1277197898 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cd006fa30 0x7f1cd00d06c0 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8009830 tx=0x7f1cb802a480 comp rx=0 tx=0).stop 2024-09-21T07:52:07.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:07.782+0000 7f1cd7800640 1 -- 172.21.15.39:0/1277197898 shutdown_connections 2024-09-21T07:52:07.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:07.782+0000 7f1cd7800640 1 --2- 172.21.15.39:0/1277197898 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cd006fa30 0x7f1cd00d06c0 unknown :-1 s=CLOSED pgs=2492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:07.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:07.782+0000 7f1cd7800640 1 -- 172.21.15.39:0/1277197898 >> 172.21.15.39:0/1277197898 conn(0x7f1cd006cfc0 msgr2=0x7f1cd006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:07.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:07.782+0000 7f1cd7800640 1 -- 172.21.15.39:0/1277197898 shutdown_connections 2024-09-21T07:52:07.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:07.783+0000 7f1cd7800640 1 -- 172.21.15.39:0/1277197898 wait complete. 2024-09-21T07:52:07.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:07.785+0000 7f1cd7800640 1 Processor -- start 2024-09-21T07:52:07.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:07.785+0000 7f1cd7800640 1 -- start start 2024-09-21T07:52:07.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:07.786+0000 7f1cd7800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cd006fa30 0x7f1cd00b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:07.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:07.786+0000 7f1cd7800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1cd00d0c00 0x7f1cd013b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:07.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:07.786+0000 7f1cd7800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cd015a460 0x7f1cd00c62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:07.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:07.786+0000 7f1cd7800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd015d810 con 0x7f1cd006fa30 2024-09-21T07:52:07.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:07.786+0000 7f1cd7800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd015d690 con 0x7f1cd015a460 2024-09-21T07:52:07.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:07.786+0000 7f1cd7800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f1cd015d990 con 0x7f1cd00d0c00 2024-09-21T07:52:07.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:07.786+0000 7f1cd4e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cd006fa30 0x7f1cd00b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:07.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:07.787+0000 7f1cd4e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cd006fa30 0x7f1cd00b27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37500/0 (socket says 172.21.15.39:37500) 2024-09-21T07:52:07.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:07.787+0000 7f1cd4e00640 1 -- 172.21.15.39:0/1901918637 learned_addr learned my addr 172.21.15.39:0/1901918637 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:07.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:07.787+0000 7f1cc7e00640 1 --2- 172.21.15.39:0/1901918637 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1cd00d0c00 0x7f1cd013b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:07.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:07.787+0000 7f1cd5800640 1 --2- 172.21.15.39:0/1901918637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cd015a460 0x7f1cd00c62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:07.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:07.787+0000 7f1cd4e00640 1 -- 172.21.15.39:0/1901918637 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1cd00d0c00 msgr2=0x7f1cd013b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:07.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:07.787+0000 7f1cd4e00640 1 --2- 172.21.15.39:0/1901918637 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1cd00d0c00 0x7f1cd013b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:07.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:07.787+0000 7f1cd4e00640 1 -- 172.21.15.39:0/1901918637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cd015a460 msgr2=0x7f1cd00c62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:07.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:07.787+0000 7f1cd4e00640 1 --2- 172.21.15.39:0/1901918637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cd015a460 0x7f1cd00c62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:07.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:07.787+0000 7f1cd4e00640 1 -- 172.21.15.39:0/1901918637 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd00b11e0 con 0x7f1cd006fa30 2024-09-21T07:52:07.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:07.787+0000 7f1cd4e00640 1 --2- 172.21.15.39:0/1901918637 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cd006fa30 0x7f1cd00b27b0 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000c390 tx=0x7f1cc000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:07.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:07.787+0000 7f1ccd600640 1 -- 172.21.15.39:0/1901918637 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0017070 con 0x7f1cd006fa30 2024-09-21T07:52:07.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:07.788+0000 7f1ccd600640 1 -- 172.21.15.39:0/1901918637 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1cc00035f0 con 0x7f1cd006fa30 2024-09-21T07:52:07.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:07.788+0000 7f1cd7800640 1 -- 172.21.15.39:0/1901918637 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cd00b14c0 con 0x7f1cd006fa30 2024-09-21T07:52:07.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:07.788+0000 7f1ccd600640 1 -- 172.21.15.39:0/1901918637 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0003a20 con 0x7f1cd006fa30 2024-09-21T07:52:07.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:07.788+0000 7f1cd7800640 1 -- 172.21.15.39:0/1901918637 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cd013ba70 con 0x7f1cd006fa30 2024-09-21T07:52:07.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:07.789+0000 7f1cd7800640 1 -- 172.21.15.39:0/1901918637 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cd00ce4f0 con 0x7f1cd006fa30 2024-09-21T07:52:07.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:07.790+0000 7f1ccd600640 1 -- 172.21.15.39:0/1901918637 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1cc001a070 con 0x7f1cd006fa30 2024-09-21T07:52:07.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:07.791+0000 7f1ccd600640 1 --2- 172.21.15.39:0/1901918637 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1cac078b90 0x7f1cac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:07.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:07.791+0000 7f1ccd600640 1 -- 172.21.15.39:0/1901918637 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(246..246 src has 1..246) ==== 7562+0+0 (secure 0 0 0) 0x7f1cc009a940 con 0x7f1cd006fa30 2024-09-21T07:52:07.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:07.792+0000 7f1cc7e00640 1 --2- 172.21.15.39:0/1901918637 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1cac078b90 0x7f1cac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:07.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:07.792+0000 7f1cc7e00640 1 --2- 172.21.15.39:0/1901918637 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1cac078b90 0x7f1cac07b0a0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8034000 tx=0x7f1cb8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:07.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:07.796+0000 7f1ccd600640 1 -- 172.21.15.39:0/1901918637 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1cd00ce4f0 con 0x7f1cd006fa30 2024-09-21T07:52:08.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:08.033+0000 7f1cd7800640 1 -- 172.21.15.39:0/1901918637 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1cd00d06c0 con 0x7f1cd006fa30 2024-09-21T07:52:08.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:08.034+0000 7f1ccd600640 1 -- 172.21.15.39:0/1901918637 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 279 v279) ==== 95+0+5280 (secure 0 0 0) 0x7f1cc0062520 con 0x7f1cd006fa30 2024-09-21T07:52:08.035 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:52:08.035 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":279,"btime":"2024-09-21T07:51:38:524572+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":278}],"filesystems":[{"mdsmap":{"epoch":279,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:51:37.529529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":246,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":10195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10195":{"gid":10195,"name":"d","rank":2,"incarnation":277,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[11052,9967]},"id":13}]} 2024-09-21T07:52:08.035 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 279 2024-09-21T07:52:08.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:08.036+0000 7f1cd7800640 1 -- 172.21.15.39:0/1901918637 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1cac078b90 msgr2=0x7f1cac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:08.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:08.036+0000 7f1cd7800640 1 --2- 172.21.15.39:0/1901918637 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1cac078b90 0x7f1cac07b0a0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8034000 tx=0x7f1cb8004730 comp rx=0 tx=0).stop 2024-09-21T07:52:08.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:08.036+0000 7f1cd7800640 1 -- 172.21.15.39:0/1901918637 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cd006fa30 msgr2=0x7f1cd00b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:08.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:08.036+0000 7f1cd7800640 1 --2- 172.21.15.39:0/1901918637 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cd006fa30 0x7f1cd00b27b0 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000c390 tx=0x7f1cc000c850 comp rx=0 tx=0).stop 2024-09-21T07:52:08.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:08.037+0000 7f1cd7800640 1 -- 172.21.15.39:0/1901918637 shutdown_connections 2024-09-21T07:52:08.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:08.037+0000 7f1cd7800640 1 --2- 172.21.15.39:0/1901918637 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1cac078b90 0x7f1cac07b0a0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:08.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:08.037+0000 7f1cd7800640 1 --2- 172.21.15.39:0/1901918637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cd015a460 0x7f1cd00c62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:08.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:08.037+0000 7f1cd7800640 1 --2- 172.21.15.39:0/1901918637 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1cd00d0c00 0x7f1cd013b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:08.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:08.037+0000 7f1cd7800640 1 --2- 172.21.15.39:0/1901918637 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cd006fa30 0x7f1cd00b27b0 unknown :-1 s=CLOSED pgs=2493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:08.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:08.037+0000 7f1cd7800640 1 -- 172.21.15.39:0/1901918637 >> 172.21.15.39:0/1901918637 conn(0x7f1cd006cfc0 msgr2=0x7f1cd006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:08.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:08.037+0000 7f1cd7800640 1 -- 172.21.15.39:0/1901918637 shutdown_connections 2024-09-21T07:52:08.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:08.037+0000 7f1cd7800640 1 -- 172.21.15.39:0/1901918637 wait complete. 2024-09-21T07:52:08.465 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 279, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:51:37.529529+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 246, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 9967, 'mds_1': 11052, 'mds_2': 10195}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9967': {'gid': 9967, 'name': 'b', 'rank': 0, 'incarnation': 259, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.148:6836/3381818843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 3381818843}, {'type': 'v1', 'addr': '172.21.15.148:6836', 'nonce': 3381818843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10195': {'gid': 10195, 'name': 'd', 'rank': 2, 'incarnation': 277, 'state': 'up:reconnect', 'state_seq': 11, 'addr': '172.21.15.148:6837/597301351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6835', 'nonce': 597301351}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 597301351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11052': {'gid': 11052, 'name': 'c', 'rank': 1, 'incarnation': 261, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6835/485194245', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 485194245}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 485194245}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9967, 'qdb_cluster': [11052, 9967]} 2024-09-21T07:52:08.466 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10195:up:reconnect 2024-09-21T07:52:09.467 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:52:09.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.704+0000 7fb81da00640 1 -- 172.21.15.39:0/2567091864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8180cfd10 msgr2=0x7fb8180cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:09.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.704+0000 7fb81da00640 1 --2- 172.21.15.39:0/2567091864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8180cfd10 0x7fb8180cd8e0 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7fb800005fd0 tx=0x7fb80002a520 comp rx=0 tx=0).stop 2024-09-21T07:52:09.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.704+0000 7fb81da00640 1 -- 172.21.15.39:0/2567091864 shutdown_connections 2024-09-21T07:52:09.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.704+0000 7fb81da00640 1 --2- 172.21.15.39:0/2567091864 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb8180d0620 0x7fb8180d0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:09.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.704+0000 7fb81da00640 1 --2- 172.21.15.39:0/2567091864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8180cfd10 0x7fb8180cd8e0 unknown :-1 s=CLOSED pgs=2494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:09.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.704+0000 7fb81da00640 1 -- 172.21.15.39:0/2567091864 >> 172.21.15.39:0/2567091864 conn(0x7fb81806cfc0 msgr2=0x7fb81806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:09.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.704+0000 7fb81da00640 1 -- 172.21.15.39:0/2567091864 shutdown_connections 2024-09-21T07:52:09.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.704+0000 7fb81da00640 1 -- 172.21.15.39:0/2567091864 wait complete. 2024-09-21T07:52:09.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.706+0000 7fb81da00640 1 Processor -- start 2024-09-21T07:52:09.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.707+0000 7fb81da00640 1 -- start start 2024-09-21T07:52:09.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.708+0000 7fb81da00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8180cfd10 0x7fb8181f9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:09.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.708+0000 7fb81da00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8180d0620 0x7fb81814f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:09.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.708+0000 7fb81da00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb81815cd30 0x7fb8181e92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:09.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.708+0000 7fb81da00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb818073890 con 0x7fb8180cfd10 2024-09-21T07:52:09.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.708+0000 7fb81da00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb818073710 con 0x7fb8180d0620 2024-09-21T07:52:09.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.708+0000 7fb81da00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb818073a10 con 0x7fb81815cd30 2024-09-21T07:52:09.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.708+0000 7fb816a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8180cfd10 0x7fb8181f9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:09.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.708+0000 7fb816a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8180cfd10 0x7fb8181f9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37520/0 (socket says 172.21.15.39:37520) 2024-09-21T07:52:09.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.708+0000 7fb816a00640 1 -- 172.21.15.39:0/3201377948 learned_addr learned my addr 172.21.15.39:0/3201377948 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:09.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.709+0000 7fb80fe00640 1 --2- 172.21.15.39:0/3201377948 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8180d0620 0x7fb81814f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:09.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.709+0000 7fb817400640 1 --2- 172.21.15.39:0/3201377948 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb81815cd30 0x7fb8181e92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:09.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.709+0000 7fb816a00640 1 -- 172.21.15.39:0/3201377948 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb81815cd30 msgr2=0x7fb8181e92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:09.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.709+0000 7fb816a00640 1 --2- 172.21.15.39:0/3201377948 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb81815cd30 0x7fb8181e92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:09.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.709+0000 7fb816a00640 1 -- 172.21.15.39:0/3201377948 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8180d0620 msgr2=0x7fb81814f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:09.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.709+0000 7fb816a00640 1 --2- 172.21.15.39:0/3201377948 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8180d0620 0x7fb81814f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:09.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.709+0000 7fb816a00640 1 -- 172.21.15.39:0/3201377948 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8180b3710 con 0x7fb8180cfd10 2024-09-21T07:52:09.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.709+0000 7fb816a00640 1 --2- 172.21.15.39:0/3201377948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8180cfd10 0x7fb8181f9770 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7fb80800b3f0 tx=0x7fb80800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:09.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.709+0000 7fb80ea00640 1 -- 172.21.15.39:0/3201377948 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb808015020 con 0x7fb8180cfd10 2024-09-21T07:52:09.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.710+0000 7fb80ea00640 1 -- 172.21.15.39:0/3201377948 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb808011920 con 0x7fb8180cfd10 2024-09-21T07:52:09.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.710+0000 7fb81da00640 1 -- 172.21.15.39:0/3201377948 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8181f9cb0 con 0x7fb8180cfd10 2024-09-21T07:52:09.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.710+0000 7fb80ea00640 1 -- 172.21.15.39:0/3201377948 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb808011d70 con 0x7fb8180cfd10 2024-09-21T07:52:09.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.710+0000 7fb81da00640 1 -- 172.21.15.39:0/3201377948 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8180b3cd0 con 0x7fb8180cfd10 2024-09-21T07:52:09.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.711+0000 7fb81da00640 1 -- 172.21.15.39:0/3201377948 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb818073f90 con 0x7fb8180cfd10 2024-09-21T07:52:09.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.712+0000 7fb80ea00640 1 -- 172.21.15.39:0/3201377948 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb808018020 con 0x7fb8180cfd10 2024-09-21T07:52:09.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.713+0000 7fb80ea00640 1 --2- 172.21.15.39:0/3201377948 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb7f4078bc0 0x7fb7f407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:09.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.713+0000 7fb80ea00640 1 -- 172.21.15.39:0/3201377948 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(246..246 src has 1..246) ==== 7562+0+0 (secure 0 0 0) 0x7fb80809b040 con 0x7fb8180cfd10 2024-09-21T07:52:09.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.713+0000 7fb80fe00640 1 --2- 172.21.15.39:0/3201377948 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb7f4078bc0 0x7fb7f407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:09.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.714+0000 7fb80fe00640 1 --2- 172.21.15.39:0/3201377948 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb7f4078bc0 0x7fb7f407b0d0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fb800033000 tx=0x7fb800004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:09.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.717+0000 7fb80ea00640 1 -- 172.21.15.39:0/3201377948 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb80801f030 con 0x7fb8180cfd10 2024-09-21T07:52:09.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.958+0000 7fb81da00640 1 -- 172.21.15.39:0/3201377948 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8180cd660 con 0x7fb8180cfd10 2024-09-21T07:52:09.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.959+0000 7fb80ea00640 1 -- 172.21.15.39:0/3201377948 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 279 v279) ==== 95+0+5280 (secure 0 0 0) 0x7fb808062700 con 0x7fb8180cfd10 2024-09-21T07:52:09.960 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:52:09.960 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":279,"btime":"2024-09-21T07:51:38:524572+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":278}],"filesystems":[{"mdsmap":{"epoch":279,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:51:37.529529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":246,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":10195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10195":{"gid":10195,"name":"d","rank":2,"incarnation":277,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[11052,9967]},"id":13}]} 2024-09-21T07:52:09.960 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 279 2024-09-21T07:52:09.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.961+0000 7fb81da00640 1 -- 172.21.15.39:0/3201377948 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb7f4078bc0 msgr2=0x7fb7f407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:09.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.961+0000 7fb81da00640 1 --2- 172.21.15.39:0/3201377948 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb7f4078bc0 0x7fb7f407b0d0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fb800033000 tx=0x7fb800004730 comp rx=0 tx=0).stop 2024-09-21T07:52:09.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.961+0000 7fb81da00640 1 -- 172.21.15.39:0/3201377948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8180cfd10 msgr2=0x7fb8181f9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:09.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.961+0000 7fb81da00640 1 --2- 172.21.15.39:0/3201377948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8180cfd10 0x7fb8181f9770 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7fb80800b3f0 tx=0x7fb80800b8b0 comp rx=0 tx=0).stop 2024-09-21T07:52:09.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.962+0000 7fb81da00640 1 -- 172.21.15.39:0/3201377948 shutdown_connections 2024-09-21T07:52:09.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.962+0000 7fb81da00640 1 --2- 172.21.15.39:0/3201377948 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb7f4078bc0 0x7fb7f407b0d0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:09.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.962+0000 7fb81da00640 1 --2- 172.21.15.39:0/3201377948 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb81815cd30 0x7fb8181e92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:09.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.962+0000 7fb81da00640 1 --2- 172.21.15.39:0/3201377948 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8180d0620 0x7fb81814f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:09.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.962+0000 7fb81da00640 1 --2- 172.21.15.39:0/3201377948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8180cfd10 0x7fb8181f9770 unknown :-1 s=CLOSED pgs=2495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:09.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.962+0000 7fb81da00640 1 -- 172.21.15.39:0/3201377948 >> 172.21.15.39:0/3201377948 conn(0x7fb81806cfc0 msgr2=0x7fb81806f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:09.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.962+0000 7fb81da00640 1 -- 172.21.15.39:0/3201377948 shutdown_connections 2024-09-21T07:52:09.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:09.962+0000 7fb81da00640 1 -- 172.21.15.39:0/3201377948 wait complete. 2024-09-21T07:52:10.390 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 279, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:51:37.529529+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 246, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 9967, 'mds_1': 11052, 'mds_2': 10195}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9967': {'gid': 9967, 'name': 'b', 'rank': 0, 'incarnation': 259, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.148:6836/3381818843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 3381818843}, {'type': 'v1', 'addr': '172.21.15.148:6836', 'nonce': 3381818843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10195': {'gid': 10195, 'name': 'd', 'rank': 2, 'incarnation': 277, 'state': 'up:reconnect', 'state_seq': 11, 'addr': '172.21.15.148:6837/597301351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6835', 'nonce': 597301351}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 597301351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11052': {'gid': 11052, 'name': 'c', 'rank': 1, 'incarnation': 261, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6835/485194245', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 485194245}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 485194245}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9967, 'qdb_cluster': [11052, 9967]} 2024-09-21T07:52:10.501 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10195:up:reconnect 2024-09-21T07:52:11.502 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:52:11.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.740+0000 7f4efc800640 1 -- 172.21.15.39:0/4038840671 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ef40cfd10 msgr2=0x7f4ef40cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:11.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.740+0000 7f4efc800640 1 --2- 172.21.15.39:0/4038840671 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ef40cfd10 0x7f4ef40cd8e0 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4009830 tx=0x7f4ee402a500 comp rx=0 tx=0).stop 2024-09-21T07:52:11.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.740+0000 7f4efc800640 1 -- 172.21.15.39:0/4038840671 shutdown_connections 2024-09-21T07:52:11.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.740+0000 7f4efc800640 1 --2- 172.21.15.39:0/4038840671 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4ef40d0620 0x7f4ef40d0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:11.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.740+0000 7f4efc800640 1 --2- 172.21.15.39:0/4038840671 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ef40cfd10 0x7f4ef40cd8e0 unknown :-1 s=CLOSED pgs=2497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:11.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.740+0000 7f4efc800640 1 -- 172.21.15.39:0/4038840671 >> 172.21.15.39:0/4038840671 conn(0x7f4ef406cfc0 msgr2=0x7f4ef406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:11.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.740+0000 7f4efc800640 1 -- 172.21.15.39:0/4038840671 shutdown_connections 2024-09-21T07:52:11.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.740+0000 7f4efc800640 1 -- 172.21.15.39:0/4038840671 wait complete. 2024-09-21T07:52:11.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.742+0000 7f4efc800640 1 Processor -- start 2024-09-21T07:52:11.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.743+0000 7f4efc800640 1 -- start start 2024-09-21T07:52:11.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.744+0000 7f4efc800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ef40cfd10 0x7f4ef4153dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:11.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.744+0000 7f4efc800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ef40d0620 0x7f4ef414bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:11.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.744+0000 7f4efc800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4ef415cd30 0x7f4ef40b23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:11.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.744+0000 7f4efc800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef4073890 con 0x7f4ef40d0620 2024-09-21T07:52:11.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.744+0000 7f4efc800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef4073710 con 0x7f4ef40cfd10 2024-09-21T07:52:11.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.744+0000 7f4efc800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f4ef4073a10 con 0x7f4ef415cd30 2024-09-21T07:52:11.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.744+0000 7f4ef9400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ef40d0620 0x7f4ef414bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:11.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.744+0000 7f4ef9400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ef40d0620 0x7f4ef414bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37534/0 (socket says 172.21.15.39:37534) 2024-09-21T07:52:11.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.744+0000 7f4ef9400640 1 -- 172.21.15.39:0/3814236393 learned_addr learned my addr 172.21.15.39:0/3814236393 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:11.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.745+0000 7f4efa800640 1 --2- 172.21.15.39:0/3814236393 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4ef415cd30 0x7f4ef40b23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:11.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.745+0000 7f4ef9e00640 1 --2- 172.21.15.39:0/3814236393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ef40cfd10 0x7f4ef4153dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:11.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.745+0000 7f4ef9400640 1 -- 172.21.15.39:0/3814236393 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4ef415cd30 msgr2=0x7f4ef40b23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:11.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.745+0000 7f4ef9400640 1 --2- 172.21.15.39:0/3814236393 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4ef415cd30 0x7f4ef40b23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:11.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.745+0000 7f4ef9400640 1 -- 172.21.15.39:0/3814236393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ef40cfd10 msgr2=0x7f4ef4153dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:11.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.745+0000 7f4ef9400640 1 --2- 172.21.15.39:0/3814236393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ef40cfd10 0x7f4ef4153dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:11.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.745+0000 7f4ef9400640 1 -- 172.21.15.39:0/3814236393 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ef4154610 con 0x7f4ef40d0620 2024-09-21T07:52:11.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.745+0000 7f4efa800640 1 --2- 172.21.15.39:0/3814236393 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4ef415cd30 0x7f4ef40b23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:52:11.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.745+0000 7f4ef9e00640 1 --2- 172.21.15.39:0/3814236393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ef40cfd10 0x7f4ef4153dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:52:11.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.745+0000 7f4ef9400640 1 --2- 172.21.15.39:0/3814236393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ef40d0620 0x7f4ef414bbd0 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4009950 tx=0x7f4ee4033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:11.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.745+0000 7f4eee000640 1 -- 172.21.15.39:0/3814236393 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4031d60 con 0x7f4ef40d0620 2024-09-21T07:52:11.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.745+0000 7f4eee000640 1 -- 172.21.15.39:0/3814236393 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ee4033e10 con 0x7f4ef40d0620 2024-09-21T07:52:11.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.745+0000 7f4efc800640 1 -- 172.21.15.39:0/3814236393 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ef41548f0 con 0x7f4ef40d0620 2024-09-21T07:52:11.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.745+0000 7f4eee000640 1 -- 172.21.15.39:0/3814236393 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4005260 con 0x7f4ef40d0620 2024-09-21T07:52:11.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.746+0000 7f4efc800640 1 -- 172.21.15.39:0/3814236393 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ef4154eb0 con 0x7f4ef40d0620 2024-09-21T07:52:11.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.747+0000 7f4efc800640 1 -- 172.21.15.39:0/3814236393 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ef4154380 con 0x7f4ef40d0620 2024-09-21T07:52:11.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.748+0000 7f4eee000640 1 -- 172.21.15.39:0/3814236393 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4ee4005400 con 0x7f4ef40d0620 2024-09-21T07:52:11.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.749+0000 7f4eee000640 1 --2- 172.21.15.39:0/3814236393 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4ed8078b90 0x7f4ed807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:11.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.749+0000 7f4eee000640 1 -- 172.21.15.39:0/3814236393 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(246..246 src has 1..246) ==== 7562+0+0 (secure 0 0 0) 0x7f4ee40bc680 con 0x7f4ef40d0620 2024-09-21T07:52:11.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.749+0000 7f4ef9e00640 1 --2- 172.21.15.39:0/3814236393 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4ed8078b90 0x7f4ed807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:11.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.750+0000 7f4ef9e00640 1 --2- 172.21.15.39:0/3814236393 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4ed8078b90 0x7f4ed807b0a0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8000f60 tx=0x7f4ee8009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:11.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.754+0000 7f4eee000640 1 -- 172.21.15.39:0/3814236393 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4ee403f030 con 0x7f4ef40d0620 2024-09-21T07:52:11.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.990+0000 7f4efc800640 1 -- 172.21.15.39:0/3814236393 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ef4073f50 con 0x7f4ef40d0620 2024-09-21T07:52:11.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.991+0000 7f4eee000640 1 -- 172.21.15.39:0/3814236393 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 279 v279) ==== 95+0+5280 (secure 0 0 0) 0x7f4ee4084400 con 0x7f4ef40d0620 2024-09-21T07:52:11.992 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:52:11.992 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":279,"btime":"2024-09-21T07:51:38:524572+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":278}],"filesystems":[{"mdsmap":{"epoch":279,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:51:37.529529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":246,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":10195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10195":{"gid":10195,"name":"d","rank":2,"incarnation":277,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[11052,9967]},"id":13}]} 2024-09-21T07:52:11.992 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 279 2024-09-21T07:52:11.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.994+0000 7f4efc800640 1 -- 172.21.15.39:0/3814236393 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4ed8078b90 msgr2=0x7f4ed807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:11.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.994+0000 7f4efc800640 1 --2- 172.21.15.39:0/3814236393 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4ed8078b90 0x7f4ed807b0a0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8000f60 tx=0x7f4ee8009040 comp rx=0 tx=0).stop 2024-09-21T07:52:11.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.994+0000 7f4efc800640 1 -- 172.21.15.39:0/3814236393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ef40d0620 msgr2=0x7f4ef414bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:11.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.994+0000 7f4efc800640 1 --2- 172.21.15.39:0/3814236393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ef40d0620 0x7f4ef414bbd0 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4009950 tx=0x7f4ee4033c20 comp rx=0 tx=0).stop 2024-09-21T07:52:11.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.994+0000 7f4efc800640 1 -- 172.21.15.39:0/3814236393 shutdown_connections 2024-09-21T07:52:11.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.994+0000 7f4efc800640 1 --2- 172.21.15.39:0/3814236393 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4ed8078b90 0x7f4ed807b0a0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:11.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.994+0000 7f4efc800640 1 --2- 172.21.15.39:0/3814236393 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4ef415cd30 0x7f4ef40b23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:11.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.994+0000 7f4efc800640 1 --2- 172.21.15.39:0/3814236393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ef40d0620 0x7f4ef414bbd0 unknown :-1 s=CLOSED pgs=2498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:11.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.994+0000 7f4efc800640 1 --2- 172.21.15.39:0/3814236393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ef40cfd10 0x7f4ef4153dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:11.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.994+0000 7f4efc800640 1 -- 172.21.15.39:0/3814236393 >> 172.21.15.39:0/3814236393 conn(0x7f4ef406cfc0 msgr2=0x7f4ef406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:11.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.994+0000 7f4efc800640 1 -- 172.21.15.39:0/3814236393 shutdown_connections 2024-09-21T07:52:11.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:11.994+0000 7f4efc800640 1 -- 172.21.15.39:0/3814236393 wait complete. 2024-09-21T07:52:12.421 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 279, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:51:37.529529+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 246, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 9967, 'mds_1': 11052, 'mds_2': 10195}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9967': {'gid': 9967, 'name': 'b', 'rank': 0, 'incarnation': 259, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.148:6836/3381818843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 3381818843}, {'type': 'v1', 'addr': '172.21.15.148:6836', 'nonce': 3381818843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10195': {'gid': 10195, 'name': 'd', 'rank': 2, 'incarnation': 277, 'state': 'up:reconnect', 'state_seq': 11, 'addr': '172.21.15.148:6837/597301351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6835', 'nonce': 597301351}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 597301351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11052': {'gid': 11052, 'name': 'c', 'rank': 1, 'incarnation': 261, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6835/485194245', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 485194245}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 485194245}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9967, 'qdb_cluster': [11052, 9967]} 2024-09-21T07:52:12.421 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10195:up:reconnect 2024-09-21T07:52:13.422 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:52:13.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.657+0000 7fb687200640 1 -- 172.21.15.39:0/2924214432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb68806fde0 msgr2=0x7fb688072fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:13.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.657+0000 7fb687200640 1 --2- 172.21.15.39:0/2924214432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb68806fde0 0x7fb688072fa0 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7fb680005fd0 tx=0x7fb68002aaa0 comp rx=0 tx=0).stop 2024-09-21T07:52:13.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.658+0000 7fb687200640 1 -- 172.21.15.39:0/2924214432 shutdown_connections 2024-09-21T07:52:13.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.658+0000 7fb687200640 1 --2- 172.21.15.39:0/2924214432 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb68815c830 0x7fb68806f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:13.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.658+0000 7fb687200640 1 --2- 172.21.15.39:0/2924214432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb68806fde0 0x7fb688072fa0 unknown :-1 s=CLOSED pgs=2499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:13.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.658+0000 7fb687200640 1 -- 172.21.15.39:0/2924214432 >> 172.21.15.39:0/2924214432 conn(0x7fb6880b2510 msgr2=0x7fb6880d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:13.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.658+0000 7fb687200640 1 -- 172.21.15.39:0/2924214432 shutdown_connections 2024-09-21T07:52:13.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.658+0000 7fb687200640 1 -- 172.21.15.39:0/2924214432 wait complete. 2024-09-21T07:52:13.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.660+0000 7fb687200640 1 Processor -- start 2024-09-21T07:52:13.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.660+0000 7fb687200640 1 -- start start 2024-09-21T07:52:13.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.661+0000 7fb687200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb68806fde0 0x7fb6880ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:13.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.661+0000 7fb687200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6880d0f20 0x7fb6881eac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:13.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.661+0000 7fb687200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb68815c830 0x7fb6881eb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:13.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.661+0000 7fb687200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb6880b4570 con 0x7fb6880d0f20 2024-09-21T07:52:13.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.661+0000 7fb687200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb6880b43f0 con 0x7fb68815c830 2024-09-21T07:52:13.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.661+0000 7fb687200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb6880b46f0 con 0x7fb68806fde0 2024-09-21T07:52:13.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.662+0000 7fb67fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6880d0f20 0x7fb6881eac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:13.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.662+0000 7fb67fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6880d0f20 0x7fb6881eac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40454/0 (socket says 172.21.15.39:40454) 2024-09-21T07:52:13.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.662+0000 7fb67fe00640 1 -- 172.21.15.39:0/1700614032 learned_addr learned my addr 172.21.15.39:0/1700614032 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:13.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.662+0000 7fb685e00640 1 --2- 172.21.15.39:0/1700614032 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb68806fde0 0x7fb6880ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:13.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.662+0000 7fb686800640 1 --2- 172.21.15.39:0/1700614032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb68815c830 0x7fb6881eb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:13.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.662+0000 7fb67fe00640 1 -- 172.21.15.39:0/1700614032 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb68806fde0 msgr2=0x7fb6880ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:13.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.662+0000 7fb67fe00640 1 --2- 172.21.15.39:0/1700614032 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb68806fde0 0x7fb6880ca160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:13.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.662+0000 7fb67fe00640 1 -- 172.21.15.39:0/1700614032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb68815c830 msgr2=0x7fb6881eb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:13.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.662+0000 7fb67fe00640 1 --2- 172.21.15.39:0/1700614032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb68815c830 0x7fb6881eb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:13.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.662+0000 7fb67fe00640 1 -- 172.21.15.39:0/1700614032 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6880c8710 con 0x7fb6880d0f20 2024-09-21T07:52:13.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.662+0000 7fb686800640 1 --2- 172.21.15.39:0/1700614032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb68815c830 0x7fb6881eb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:52:13.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.662+0000 7fb67fe00640 1 --2- 172.21.15.39:0/1700614032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6880d0f20 0x7fb6881eac40 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7fb67000d700 tx=0x7fb67000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:13.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.663+0000 7fb67e000640 1 -- 172.21.15.39:0/1700614032 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb670017070 con 0x7fb6880d0f20 2024-09-21T07:52:13.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.663+0000 7fb67e000640 1 -- 172.21.15.39:0/1700614032 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb670005310 con 0x7fb6880d0f20 2024-09-21T07:52:13.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.663+0000 7fb67e000640 1 -- 172.21.15.39:0/1700614032 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb670009820 con 0x7fb6880d0f20 2024-09-21T07:52:13.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.663+0000 7fb687200640 1 -- 172.21.15.39:0/1700614032 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb68815fbd0 con 0x7fb6880d0f20 2024-09-21T07:52:13.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.663+0000 7fb687200640 1 -- 172.21.15.39:0/1700614032 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6880ad300 con 0x7fb6880d0f20 2024-09-21T07:52:13.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.664+0000 7fb687200640 1 -- 172.21.15.39:0/1700614032 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6881e7070 con 0x7fb6880d0f20 2024-09-21T07:52:13.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.665+0000 7fb67e000640 1 -- 172.21.15.39:0/1700614032 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb6700099c0 con 0x7fb6880d0f20 2024-09-21T07:52:13.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.666+0000 7fb67e000640 1 --2- 172.21.15.39:0/1700614032 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb664078b90 0x7fb66407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:13.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.666+0000 7fb67e000640 1 -- 172.21.15.39:0/1700614032 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(246..246 src has 1..246) ==== 7562+0+0 (secure 0 0 0) 0x7fb67009b570 con 0x7fb6880d0f20 2024-09-21T07:52:13.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.667+0000 7fb685e00640 1 --2- 172.21.15.39:0/1700614032 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb664078b90 0x7fb66407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:13.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.668+0000 7fb685e00640 1 --2- 172.21.15.39:0/1700614032 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb664078b90 0x7fb66407b0a0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fb678001720 tx=0x7fb678009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:13.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.671+0000 7fb67e000640 1 -- 172.21.15.39:0/1700614032 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb6881e7070 con 0x7fb6880d0f20 2024-09-21T07:52:13.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.908+0000 7fb687200640 1 -- 172.21.15.39:0/1700614032 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6880b4c10 con 0x7fb6880d0f20 2024-09-21T07:52:13.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.909+0000 7fb67e000640 1 -- 172.21.15.39:0/1700614032 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 279 v279) ==== 95+0+5280 (secure 0 0 0) 0x7fb670062c10 con 0x7fb6880d0f20 2024-09-21T07:52:13.911 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:52:13.911 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":279,"btime":"2024-09-21T07:51:38:524572+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":278}],"filesystems":[{"mdsmap":{"epoch":279,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:51:37.529529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":246,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":10195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10195":{"gid":10195,"name":"d","rank":2,"incarnation":277,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[11052,9967]},"id":13}]} 2024-09-21T07:52:13.911 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 279 2024-09-21T07:52:13.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.911+0000 7fb687200640 1 -- 172.21.15.39:0/1700614032 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb664078b90 msgr2=0x7fb66407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:13.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.911+0000 7fb687200640 1 --2- 172.21.15.39:0/1700614032 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb664078b90 0x7fb66407b0a0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fb678001720 tx=0x7fb678009040 comp rx=0 tx=0).stop 2024-09-21T07:52:13.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.911+0000 7fb687200640 1 -- 172.21.15.39:0/1700614032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6880d0f20 msgr2=0x7fb6881eac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:13.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.911+0000 7fb687200640 1 --2- 172.21.15.39:0/1700614032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6880d0f20 0x7fb6881eac40 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7fb67000d700 tx=0x7fb67000dbc0 comp rx=0 tx=0).stop 2024-09-21T07:52:13.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.911+0000 7fb687200640 1 -- 172.21.15.39:0/1700614032 shutdown_connections 2024-09-21T07:52:13.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.911+0000 7fb687200640 1 --2- 172.21.15.39:0/1700614032 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb664078b90 0x7fb66407b0a0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:13.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.911+0000 7fb687200640 1 --2- 172.21.15.39:0/1700614032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb68815c830 0x7fb6881eb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:13.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.911+0000 7fb687200640 1 --2- 172.21.15.39:0/1700614032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6880d0f20 0x7fb6881eac40 unknown :-1 s=CLOSED pgs=2500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:13.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.911+0000 7fb687200640 1 --2- 172.21.15.39:0/1700614032 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb68806fde0 0x7fb6880ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:13.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.912+0000 7fb687200640 1 -- 172.21.15.39:0/1700614032 >> 172.21.15.39:0/1700614032 conn(0x7fb6880b2510 msgr2=0x7fb6881e76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:13.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.912+0000 7fb687200640 1 -- 172.21.15.39:0/1700614032 shutdown_connections 2024-09-21T07:52:13.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:13.912+0000 7fb687200640 1 -- 172.21.15.39:0/1700614032 wait complete. 2024-09-21T07:52:14.341 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 279, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:51:37.529529+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 246, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 9967, 'mds_1': 11052, 'mds_2': 10195}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9967': {'gid': 9967, 'name': 'b', 'rank': 0, 'incarnation': 259, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.148:6836/3381818843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 3381818843}, {'type': 'v1', 'addr': '172.21.15.148:6836', 'nonce': 3381818843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10195': {'gid': 10195, 'name': 'd', 'rank': 2, 'incarnation': 277, 'state': 'up:reconnect', 'state_seq': 11, 'addr': '172.21.15.148:6837/597301351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6835', 'nonce': 597301351}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 597301351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11052': {'gid': 11052, 'name': 'c', 'rank': 1, 'incarnation': 261, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6835/485194245', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 485194245}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 485194245}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9967, 'qdb_cluster': [11052, 9967]} 2024-09-21T07:52:14.341 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10195:up:reconnect 2024-09-21T07:52:15.342 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:52:15.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.575+0000 7fe0cb600640 1 -- 172.21.15.39:0/2303914330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0c415cd30 msgr2=0x7fe0c40cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:15.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.575+0000 7fe0cb600640 1 --2- 172.21.15.39:0/2303914330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0c415cd30 0x7fe0c40cfd10 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8005fd0 tx=0x7fe0b802a5f0 comp rx=0 tx=0).stop 2024-09-21T07:52:15.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.576+0000 7fe0cb600640 1 -- 172.21.15.39:0/2303914330 shutdown_connections 2024-09-21T07:52:15.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.576+0000 7fe0cb600640 1 --2- 172.21.15.39:0/2303914330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0c415cd30 0x7fe0c40cfd10 unknown :-1 s=CLOSED pgs=2502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:15.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.576+0000 7fe0cb600640 1 -- 172.21.15.39:0/2303914330 >> 172.21.15.39:0/2303914330 conn(0x7fe0c406cfc0 msgr2=0x7fe0c406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:15.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.576+0000 7fe0cb600640 1 -- 172.21.15.39:0/2303914330 shutdown_connections 2024-09-21T07:52:15.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.576+0000 7fe0cb600640 1 -- 172.21.15.39:0/2303914330 wait complete. 2024-09-21T07:52:15.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.578+0000 7fe0cb600640 1 Processor -- start 2024-09-21T07:52:15.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.578+0000 7fe0cb600640 1 -- start start 2024-09-21T07:52:15.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.579+0000 7fe0cb600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe0c406f800 0x7fe0c41464e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:15.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.580+0000 7fe0cb600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0c406fc10 0x7fe0c41f8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:15.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.580+0000 7fe0cb600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0c415cd30 0x7fe0c40b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:15.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.580+0000 7fe0cb600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c4073aa0 con 0x7fe0c415cd30 2024-09-21T07:52:15.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.580+0000 7fe0cb600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c4073920 con 0x7fe0c406fc10 2024-09-21T07:52:15.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.580+0000 7fe0cb600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe0c4073c20 con 0x7fe0c406f800 2024-09-21T07:52:15.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.580+0000 7fe0c9600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0c415cd30 0x7fe0c40b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:15.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.580+0000 7fe0c9600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0c415cd30 0x7fe0c40b1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40482/0 (socket says 172.21.15.39:40482) 2024-09-21T07:52:15.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.580+0000 7fe0c9600640 1 -- 172.21.15.39:0/2926135761 learned_addr learned my addr 172.21.15.39:0/2926135761 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:15.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.580+0000 7fe0c8c00640 1 --2- 172.21.15.39:0/2926135761 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe0c406f800 0x7fe0c41464e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:15.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.580+0000 7fe0c3e00640 1 --2- 172.21.15.39:0/2926135761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0c406fc10 0x7fe0c41f8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:15.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.580+0000 7fe0c9600640 1 -- 172.21.15.39:0/2926135761 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe0c406f800 msgr2=0x7fe0c41464e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:15.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.580+0000 7fe0c9600640 1 --2- 172.21.15.39:0/2926135761 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe0c406f800 0x7fe0c41464e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:15.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.580+0000 7fe0c9600640 1 -- 172.21.15.39:0/2926135761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0c406fc10 msgr2=0x7fe0c41f8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:15.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.580+0000 7fe0c9600640 1 --2- 172.21.15.39:0/2926135761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0c406fc10 0x7fe0c41f8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:15.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.580+0000 7fe0c9600640 1 -- 172.21.15.39:0/2926135761 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c41f6630 con 0x7fe0c415cd30 2024-09-21T07:52:15.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.581+0000 7fe0c9600640 1 --2- 172.21.15.39:0/2926135761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0c415cd30 0x7fe0c40b1b30 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc00c730 tx=0x7fe0bc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:15.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.581+0000 7fe0c0c00640 1 -- 172.21.15.39:0/2926135761 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc003b70 con 0x7fe0c415cd30 2024-09-21T07:52:15.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.581+0000 7fe0c0c00640 1 -- 172.21.15.39:0/2926135761 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe0bc003d30 con 0x7fe0c415cd30 2024-09-21T07:52:15.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.581+0000 7fe0cb600640 1 -- 172.21.15.39:0/2926135761 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0c41491f0 con 0x7fe0c415cd30 2024-09-21T07:52:15.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.581+0000 7fe0c0c00640 1 -- 172.21.15.39:0/2926135761 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc0052d0 con 0x7fe0c415cd30 2024-09-21T07:52:15.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.581+0000 7fe0cb600640 1 -- 172.21.15.39:0/2926135761 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0c41497b0 con 0x7fe0c415cd30 2024-09-21T07:52:15.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.582+0000 7fe0cb600640 1 -- 172.21.15.39:0/2926135761 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0c4147010 con 0x7fe0c415cd30 2024-09-21T07:52:15.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.584+0000 7fe0c0c00640 1 -- 172.21.15.39:0/2926135761 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe0bc005680 con 0x7fe0c415cd30 2024-09-21T07:52:15.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.585+0000 7fe0c0c00640 1 --2- 172.21.15.39:0/2926135761 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe0a4078b90 0x7fe0a407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:15.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.585+0000 7fe0c0c00640 1 -- 172.21.15.39:0/2926135761 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(246..246 src has 1..246) ==== 7562+0+0 (secure 0 0 0) 0x7fe0bc09b470 con 0x7fe0c415cd30 2024-09-21T07:52:15.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.585+0000 7fe0c8c00640 1 --2- 172.21.15.39:0/2926135761 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe0a4078b90 0x7fe0a407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:15.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.586+0000 7fe0c8c00640 1 --2- 172.21.15.39:0/2926135761 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe0a4078b90 0x7fe0a407b0a0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8033000 tx=0x7fe0b8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:15.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.589+0000 7fe0c0c00640 1 -- 172.21.15.39:0/2926135761 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe0bc0630d0 con 0x7fe0c415cd30 2024-09-21T07:52:15.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.827+0000 7fe0cb600640 1 -- 172.21.15.39:0/2926135761 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0c41f99e0 con 0x7fe0c415cd30 2024-09-21T07:52:15.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.828+0000 7fe0c0c00640 1 -- 172.21.15.39:0/2926135761 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 279 v279) ==== 95+0+5280 (secure 0 0 0) 0x7fe0bc0629d0 con 0x7fe0c415cd30 2024-09-21T07:52:15.830 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:52:15.830 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":279,"btime":"2024-09-21T07:51:38:524572+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":278}],"filesystems":[{"mdsmap":{"epoch":279,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:51:37.529529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":246,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":10195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10195":{"gid":10195,"name":"d","rank":2,"incarnation":277,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[11052,9967]},"id":13}]} 2024-09-21T07:52:15.830 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 279 2024-09-21T07:52:15.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.830+0000 7fe0cb600640 1 -- 172.21.15.39:0/2926135761 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe0a4078b90 msgr2=0x7fe0a407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:15.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.830+0000 7fe0cb600640 1 --2- 172.21.15.39:0/2926135761 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe0a4078b90 0x7fe0a407b0a0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8033000 tx=0x7fe0b8004730 comp rx=0 tx=0).stop 2024-09-21T07:52:15.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.830+0000 7fe0cb600640 1 -- 172.21.15.39:0/2926135761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0c415cd30 msgr2=0x7fe0c40b1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:15.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.830+0000 7fe0cb600640 1 --2- 172.21.15.39:0/2926135761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0c415cd30 0x7fe0c40b1b30 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc00c730 tx=0x7fe0bc00cbf0 comp rx=0 tx=0).stop 2024-09-21T07:52:15.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.830+0000 7fe0cb600640 1 -- 172.21.15.39:0/2926135761 shutdown_connections 2024-09-21T07:52:15.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.830+0000 7fe0cb600640 1 --2- 172.21.15.39:0/2926135761 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe0a4078b90 0x7fe0a407b0a0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:15.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.830+0000 7fe0cb600640 1 --2- 172.21.15.39:0/2926135761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0c415cd30 0x7fe0c40b1b30 unknown :-1 s=CLOSED pgs=2503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:15.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.830+0000 7fe0cb600640 1 --2- 172.21.15.39:0/2926135761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0c406fc10 0x7fe0c41f8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:15.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.830+0000 7fe0cb600640 1 --2- 172.21.15.39:0/2926135761 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe0c406f800 0x7fe0c41464e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:15.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.830+0000 7fe0cb600640 1 -- 172.21.15.39:0/2926135761 >> 172.21.15.39:0/2926135761 conn(0x7fe0c406cfc0 msgr2=0x7fe0c4068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:15.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.830+0000 7fe0cb600640 1 -- 172.21.15.39:0/2926135761 shutdown_connections 2024-09-21T07:52:15.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:15.830+0000 7fe0cb600640 1 -- 172.21.15.39:0/2926135761 wait complete. 2024-09-21T07:52:16.257 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 279, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:51:37.529529+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 246, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 9967, 'mds_1': 11052, 'mds_2': 10195}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9967': {'gid': 9967, 'name': 'b', 'rank': 0, 'incarnation': 259, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.148:6836/3381818843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 3381818843}, {'type': 'v1', 'addr': '172.21.15.148:6836', 'nonce': 3381818843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10195': {'gid': 10195, 'name': 'd', 'rank': 2, 'incarnation': 277, 'state': 'up:reconnect', 'state_seq': 11, 'addr': '172.21.15.148:6837/597301351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6835', 'nonce': 597301351}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 597301351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11052': {'gid': 11052, 'name': 'c', 'rank': 1, 'incarnation': 261, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6835/485194245', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 485194245}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 485194245}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9967, 'qdb_cluster': [11052, 9967]} 2024-09-21T07:52:16.257 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10195:up:reconnect 2024-09-21T07:52:17.258 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:52:17.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.496+0000 7fb79b400640 1 -- 172.21.15.39:0/2214741218 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb77c0041c0 msgr2=0x7fb77c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:17.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.496+0000 7fb79b400640 1 --2- 172.21.15.39:0/2214741218 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb77c0041c0 0x7fb77c004800 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7fb780006f90 tx=0x7fb77c005830 comp rx=0 tx=0).stop 2024-09-21T07:52:17.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.496+0000 7fb799400640 1 -- 172.21.15.39:0/2214741218 reap_dead start 2024-09-21T07:52:17.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.497+0000 7fb79b400640 1 -- 172.21.15.39:0/2214741218 shutdown_connections 2024-09-21T07:52:17.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.497+0000 7fb79b400640 1 -- 172.21.15.39:0/2214741218 >> 172.21.15.39:0/2214741218 conn(0x7fb79406cfc0 msgr2=0x7fb79406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:17.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.497+0000 7fb79b400640 1 -- 172.21.15.39:0/2214741218 shutdown_connections 2024-09-21T07:52:17.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.497+0000 7fb79b400640 1 -- 172.21.15.39:0/2214741218 wait complete. 2024-09-21T07:52:17.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.499+0000 7fb79b400640 1 Processor -- start 2024-09-21T07:52:17.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.499+0000 7fb79b400640 1 -- start start 2024-09-21T07:52:17.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.501+0000 7fb79b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb77c0041c0 0x7fb794150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:17.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.501+0000 7fb79b400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb794154930 0x7fb7940ab050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:17.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.501+0000 7fb79b400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb794154d40 0x7fb7940ab590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:17.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.501+0000 7fb79b400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb7940713c0 con 0x7fb77c0041c0 2024-09-21T07:52:17.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.501+0000 7fb79b400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb794071240 con 0x7fb794154d40 2024-09-21T07:52:17.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.501+0000 7fb79b400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb794070a00 con 0x7fb794154930 2024-09-21T07:52:17.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.501+0000 7fb798a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb77c0041c0 0x7fb794150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:17.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.501+0000 7fb798a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb77c0041c0 0x7fb794150390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40502/0 (socket says 172.21.15.39:40502) 2024-09-21T07:52:17.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.501+0000 7fb798a00640 1 -- 172.21.15.39:0/1799551582 learned_addr learned my addr 172.21.15.39:0/1799551582 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:17.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.501+0000 7fb793e00640 1 --2- 172.21.15.39:0/1799551582 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb794154930 0x7fb7940ab050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:17.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.501+0000 7fb799400640 1 --2- 172.21.15.39:0/1799551582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb794154d40 0x7fb7940ab590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:17.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.501+0000 7fb798a00640 1 -- 172.21.15.39:0/1799551582 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb794154930 msgr2=0x7fb7940ab050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:17.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.501+0000 7fb798a00640 1 --2- 172.21.15.39:0/1799551582 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb794154930 0x7fb7940ab050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:17.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.501+0000 7fb798a00640 1 -- 172.21.15.39:0/1799551582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb794154d40 msgr2=0x7fb7940ab590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:17.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.501+0000 7fb798a00640 1 --2- 172.21.15.39:0/1799551582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb794154d40 0x7fb7940ab590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:17.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.501+0000 7fb798a00640 1 -- 172.21.15.39:0/1799551582 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7940a9e60 con 0x7fb77c0041c0 2024-09-21T07:52:17.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.502+0000 7fb798a00640 1 --2- 172.21.15.39:0/1799551582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb77c0041c0 0x7fb794150390 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7fb78800ea70 tx=0x7fb78800ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:17.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.502+0000 7fb790c00640 1 -- 172.21.15.39:0/1799551582 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78800dc60 con 0x7fb77c0041c0 2024-09-21T07:52:17.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.502+0000 7fb790c00640 1 -- 172.21.15.39:0/1799551582 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb78800de00 con 0x7fb77c0041c0 2024-09-21T07:52:17.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.502+0000 7fb790c00640 1 -- 172.21.15.39:0/1799551582 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7880042b0 con 0x7fb77c0041c0 2024-09-21T07:52:17.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.502+0000 7fb79b400640 1 -- 172.21.15.39:0/1799551582 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7940aa390 con 0x7fb77c0041c0 2024-09-21T07:52:17.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.502+0000 7fb79b400640 1 -- 172.21.15.39:0/1799551582 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7940aa950 con 0x7fb77c0041c0 2024-09-21T07:52:17.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.503+0000 7fb79b400640 1 -- 172.21.15.39:0/1799551582 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb794070f00 con 0x7fb77c0041c0 2024-09-21T07:52:17.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.505+0000 7fb790c00640 1 -- 172.21.15.39:0/1799551582 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb7880035f0 con 0x7fb77c0041c0 2024-09-21T07:52:17.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.506+0000 7fb790c00640 1 --2- 172.21.15.39:0/1799551582 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb774078bc0 0x7fb77407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:17.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.506+0000 7fb790c00640 1 -- 172.21.15.39:0/1799551582 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(246..246 src has 1..246) ==== 7562+0+0 (secure 0 0 0) 0x7fb78809be60 con 0x7fb77c0041c0 2024-09-21T07:52:17.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.507+0000 7fb793e00640 1 --2- 172.21.15.39:0/1799551582 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb774078bc0 0x7fb77407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:17.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.507+0000 7fb793e00640 1 --2- 172.21.15.39:0/1799551582 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb774078bc0 0x7fb77407b0d0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fb780033000 tx=0x7fb78003b040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:17.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.511+0000 7fb790c00640 1 -- 172.21.15.39:0/1799551582 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb788063500 con 0x7fb77c0041c0 2024-09-21T07:52:17.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.749+0000 7fb79b400640 1 -- 172.21.15.39:0/1799551582 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7940cd7e0 con 0x7fb77c0041c0 2024-09-21T07:52:17.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.750+0000 7fb790c00640 1 -- 172.21.15.39:0/1799551582 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 279 v279) ==== 95+0+5280 (secure 0 0 0) 0x7fb788067f30 con 0x7fb77c0041c0 2024-09-21T07:52:17.752 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:52:17.752 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":279,"btime":"2024-09-21T07:51:38:524572+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":278}],"filesystems":[{"mdsmap":{"epoch":279,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:51:37.529529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":246,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":10195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10195":{"gid":10195,"name":"d","rank":2,"incarnation":277,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[11052,9967]},"id":13}]} 2024-09-21T07:52:17.752 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 279 2024-09-21T07:52:17.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.752+0000 7fb79b400640 1 -- 172.21.15.39:0/1799551582 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb774078bc0 msgr2=0x7fb77407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:17.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.752+0000 7fb79b400640 1 --2- 172.21.15.39:0/1799551582 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb774078bc0 0x7fb77407b0d0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fb780033000 tx=0x7fb78003b040 comp rx=0 tx=0).stop 2024-09-21T07:52:17.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.752+0000 7fb79b400640 1 -- 172.21.15.39:0/1799551582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb77c0041c0 msgr2=0x7fb794150390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:17.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.752+0000 7fb79b400640 1 --2- 172.21.15.39:0/1799551582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb77c0041c0 0x7fb794150390 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7fb78800ea70 tx=0x7fb78800ef30 comp rx=0 tx=0).stop 2024-09-21T07:52:17.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.752+0000 7fb79b400640 1 -- 172.21.15.39:0/1799551582 shutdown_connections 2024-09-21T07:52:17.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.752+0000 7fb79b400640 1 --2- 172.21.15.39:0/1799551582 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb774078bc0 0x7fb77407b0d0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:17.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.752+0000 7fb79b400640 1 --2- 172.21.15.39:0/1799551582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb794154d40 0x7fb7940ab590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:17.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.752+0000 7fb79b400640 1 --2- 172.21.15.39:0/1799551582 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb794154930 0x7fb7940ab050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:17.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.752+0000 7fb79b400640 1 --2- 172.21.15.39:0/1799551582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb77c0041c0 0x7fb794150390 unknown :-1 s=CLOSED pgs=2506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:17.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.752+0000 7fb79b400640 1 -- 172.21.15.39:0/1799551582 >> 172.21.15.39:0/1799551582 conn(0x7fb79406cfc0 msgr2=0x7fb7940d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:17.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.752+0000 7fb79b400640 1 -- 172.21.15.39:0/1799551582 shutdown_connections 2024-09-21T07:52:17.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:17.752+0000 7fb79b400640 1 -- 172.21.15.39:0/1799551582 wait complete. 2024-09-21T07:52:18.179 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 279, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:51:37.529529+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 246, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 9967, 'mds_1': 11052, 'mds_2': 10195}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9967': {'gid': 9967, 'name': 'b', 'rank': 0, 'incarnation': 259, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.148:6836/3381818843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 3381818843}, {'type': 'v1', 'addr': '172.21.15.148:6836', 'nonce': 3381818843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10195': {'gid': 10195, 'name': 'd', 'rank': 2, 'incarnation': 277, 'state': 'up:reconnect', 'state_seq': 11, 'addr': '172.21.15.148:6837/597301351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6835', 'nonce': 597301351}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 597301351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11052': {'gid': 11052, 'name': 'c', 'rank': 1, 'incarnation': 261, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6835/485194245', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 485194245}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 485194245}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9967, 'qdb_cluster': [11052, 9967]} 2024-09-21T07:52:18.179 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10195:up:reconnect 2024-09-21T07:52:19.180 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:52:19.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.416+0000 7f67c4c00640 1 -- 172.21.15.39:0/264889551 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67a4005150 msgr2=0x7f67a400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:19.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.416+0000 7f67c4c00640 1 --2- 172.21.15.39:0/264889551 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67a4005150 0x7f67a400b9c0 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7f67b8005fd0 tx=0x7f67a4005830 comp rx=0 tx=0).stop 2024-09-21T07:52:19.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.416+0000 7f67be800640 1 -- 172.21.15.39:0/264889551 reap_dead start 2024-09-21T07:52:19.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.417+0000 7f67c4c00640 1 -- 172.21.15.39:0/264889551 shutdown_connections 2024-09-21T07:52:19.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.417+0000 7f67c4c00640 1 -- 172.21.15.39:0/264889551 >> 172.21.15.39:0/264889551 conn(0x7f67c006cfc0 msgr2=0x7f67c006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:19.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.417+0000 7f67c4c00640 1 -- 172.21.15.39:0/264889551 shutdown_connections 2024-09-21T07:52:19.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.417+0000 7f67c4c00640 1 -- 172.21.15.39:0/264889551 wait complete. 2024-09-21T07:52:19.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.419+0000 7f67c4c00640 1 Processor -- start 2024-09-21T07:52:19.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.419+0000 7f67c4c00640 1 -- start start 2024-09-21T07:52:19.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.420+0000 7f67c4c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67a4005150 0x7f67c01f96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:19.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.421+0000 7f67c4c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67c00affe0 0x7f67c0157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:19.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.421+0000 7f67c4c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f67c0073150 0x7f67c0073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:19.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.421+0000 7f67c4c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f67c015c060 con 0x7f67a4005150 2024-09-21T07:52:19.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.421+0000 7f67c4c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f67c015bee0 con 0x7f67c00affe0 2024-09-21T07:52:19.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.421+0000 7f67c4c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f67c00d0770 con 0x7f67c0073150 2024-09-21T07:52:19.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.421+0000 7f67bde00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67a4005150 0x7f67c01f96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:19.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.421+0000 7f67bde00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67a4005150 0x7f67c01f96f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40516/0 (socket says 172.21.15.39:40516) 2024-09-21T07:52:19.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.421+0000 7f67bde00640 1 -- 172.21.15.39:0/515579554 learned_addr learned my addr 172.21.15.39:0/515579554 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:19.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.421+0000 7f67bde00640 1 -- 172.21.15.39:0/515579554 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f67c0073150 msgr2=0x7f67c0073560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:19.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.421+0000 7f67bde00640 1 --2- 172.21.15.39:0/515579554 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f67c0073150 0x7f67c0073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:19.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.421+0000 7f67bde00640 1 -- 172.21.15.39:0/515579554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67c00affe0 msgr2=0x7f67c0157cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:19.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.421+0000 7f67bde00640 1 --2- 172.21.15.39:0/515579554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67c00affe0 0x7f67c0157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:19.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.421+0000 7f67bde00640 1 -- 172.21.15.39:0/515579554 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c0073aa0 con 0x7f67a4005150 2024-09-21T07:52:19.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.421+0000 7f67bde00640 1 --2- 172.21.15.39:0/515579554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67a4005150 0x7f67c01f96f0 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7f67b0011a10 tx=0x7f67b0011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:19.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.422+0000 7f67b6000640 1 -- 172.21.15.39:0/515579554 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b000fcd0 con 0x7f67a4005150 2024-09-21T07:52:19.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.422+0000 7f67b6000640 1 -- 172.21.15.39:0/515579554 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f67b0016070 con 0x7f67a4005150 2024-09-21T07:52:19.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.422+0000 7f67b6000640 1 -- 172.21.15.39:0/515579554 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b0004290 con 0x7f67a4005150 2024-09-21T07:52:19.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.422+0000 7f67c4c00640 1 -- 172.21.15.39:0/515579554 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67c00c5680 con 0x7f67a4005150 2024-09-21T07:52:19.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.422+0000 7f67c4c00640 1 -- 172.21.15.39:0/515579554 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67c00c60e0 con 0x7f67a4005150 2024-09-21T07:52:19.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.423+0000 7f67c4c00640 1 -- 172.21.15.39:0/515579554 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67c00b4aa0 con 0x7f67a4005150 2024-09-21T07:52:19.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.424+0000 7f67b6000640 1 -- 172.21.15.39:0/515579554 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f67b0018070 con 0x7f67a4005150 2024-09-21T07:52:19.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.425+0000 7f67b6000640 1 --2- 172.21.15.39:0/515579554 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f679c078bc0 0x7f679c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:19.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.425+0000 7f67b6000640 1 -- 172.21.15.39:0/515579554 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(246..246 src has 1..246) ==== 7562+0+0 (secure 0 0 0) 0x7f67b009bca0 con 0x7f67a4005150 2024-09-21T07:52:19.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.426+0000 7f67b7e00640 1 --2- 172.21.15.39:0/515579554 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f679c078bc0 0x7f679c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:19.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.427+0000 7f67b7e00640 1 --2- 172.21.15.39:0/515579554 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f679c078bc0 0x7f679c07b0d0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f67a8000fb0 tx=0x7f67a8001c90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:19.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.430+0000 7f67b6000640 1 -- 172.21.15.39:0/515579554 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f67c0073aa0 con 0x7f67a4005150 2024-09-21T07:52:19.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.665+0000 7f67c4c00640 1 -- 172.21.15.39:0/515579554 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f67c006a600 con 0x7f67a4005150 2024-09-21T07:52:19.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.666+0000 7f67b6000640 1 -- 172.21.15.39:0/515579554 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 279 v279) ==== 95+0+5280 (secure 0 0 0) 0x7f67b0063900 con 0x7f67a4005150 2024-09-21T07:52:19.668 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:52:19.668 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":279,"btime":"2024-09-21T07:51:38:524572+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":278}],"filesystems":[{"mdsmap":{"epoch":279,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:51:37.529529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":246,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":10195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10195":{"gid":10195,"name":"d","rank":2,"incarnation":277,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[11052,9967]},"id":13}]} 2024-09-21T07:52:19.668 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 279 2024-09-21T07:52:19.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.668+0000 7f67c4c00640 1 -- 172.21.15.39:0/515579554 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f679c078bc0 msgr2=0x7f679c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:19.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.668+0000 7f67c4c00640 1 --2- 172.21.15.39:0/515579554 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f679c078bc0 0x7f679c07b0d0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f67a8000fb0 tx=0x7f67a8001c90 comp rx=0 tx=0).stop 2024-09-21T07:52:19.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.668+0000 7f67c4c00640 1 -- 172.21.15.39:0/515579554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67a4005150 msgr2=0x7f67c01f96f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:19.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.668+0000 7f67c4c00640 1 --2- 172.21.15.39:0/515579554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67a4005150 0x7f67c01f96f0 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7f67b0011a10 tx=0x7f67b0011ed0 comp rx=0 tx=0).stop 2024-09-21T07:52:19.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.668+0000 7f67c4c00640 1 -- 172.21.15.39:0/515579554 shutdown_connections 2024-09-21T07:52:19.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.668+0000 7f67c4c00640 1 --2- 172.21.15.39:0/515579554 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f679c078bc0 0x7f679c07b0d0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:19.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.668+0000 7f67c4c00640 1 --2- 172.21.15.39:0/515579554 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f67c0073150 0x7f67c0073560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:19.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.668+0000 7f67c4c00640 1 --2- 172.21.15.39:0/515579554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67c00affe0 0x7f67c0157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:19.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.668+0000 7f67c4c00640 1 --2- 172.21.15.39:0/515579554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67a4005150 0x7f67c01f96f0 unknown :-1 s=CLOSED pgs=2509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:19.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.668+0000 7f67c4c00640 1 -- 172.21.15.39:0/515579554 >> 172.21.15.39:0/515579554 conn(0x7f67c006cfc0 msgr2=0x7f67c015e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:19.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.668+0000 7f67c4c00640 1 -- 172.21.15.39:0/515579554 shutdown_connections 2024-09-21T07:52:19.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:19.668+0000 7f67c4c00640 1 -- 172.21.15.39:0/515579554 wait complete. 2024-09-21T07:52:20.097 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 279, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:51:37.529529+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 246, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 9967, 'mds_1': 11052, 'mds_2': 10195}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9967': {'gid': 9967, 'name': 'b', 'rank': 0, 'incarnation': 259, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.148:6836/3381818843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 3381818843}, {'type': 'v1', 'addr': '172.21.15.148:6836', 'nonce': 3381818843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10195': {'gid': 10195, 'name': 'd', 'rank': 2, 'incarnation': 277, 'state': 'up:reconnect', 'state_seq': 11, 'addr': '172.21.15.148:6837/597301351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6835', 'nonce': 597301351}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 597301351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11052': {'gid': 11052, 'name': 'c', 'rank': 1, 'incarnation': 261, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6835/485194245', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 485194245}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 485194245}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9967, 'qdb_cluster': [11052, 9967]} 2024-09-21T07:52:20.098 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10195:up:reconnect 2024-09-21T07:52:21.099 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:52:21.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.337+0000 7fbbae000640 1 -- 172.21.15.39:0/99197407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb8c0041c0 msgr2=0x7fbb8c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:21.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.337+0000 7fbbae000640 1 --2- 172.21.15.39:0/99197407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb8c0041c0 0x7fbb8c004800 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7fbb90009b50 tx=0x7fbb8c005830 comp rx=0 tx=0).stop 2024-09-21T07:52:21.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.337+0000 7fbba7e00640 1 -- 172.21.15.39:0/99197407 reap_dead start 2024-09-21T07:52:21.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.337+0000 7fbbae000640 1 -- 172.21.15.39:0/99197407 shutdown_connections 2024-09-21T07:52:21.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.337+0000 7fbbae000640 1 -- 172.21.15.39:0/99197407 >> 172.21.15.39:0/99197407 conn(0x7fbba806cfc0 msgr2=0x7fbba806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:21.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.337+0000 7fbbae000640 1 -- 172.21.15.39:0/99197407 shutdown_connections 2024-09-21T07:52:21.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.337+0000 7fbbae000640 1 -- 172.21.15.39:0/99197407 wait complete. 2024-09-21T07:52:21.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.339+0000 7fbbae000640 1 Processor -- start 2024-09-21T07:52:21.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.339+0000 7fbbae000640 1 -- start start 2024-09-21T07:52:21.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.341+0000 7fbbae000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb8c0041c0 0x7fbba814e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:21.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.341+0000 7fbbae000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbba80a9e10 0x7fbba81f6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:21.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.341+0000 7fbbae000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbba81f6cd0 0x7fbba81f70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:21.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.341+0000 7fbbae000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbba80cdee0 con 0x7fbba81f6cd0 2024-09-21T07:52:21.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.341+0000 7fbbae000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbba80cdd60 con 0x7fbb8c0041c0 2024-09-21T07:52:21.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.341+0000 7fbbae000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fbba80ce060 con 0x7fbba80a9e10 2024-09-21T07:52:21.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.341+0000 7fbba7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbba81f6cd0 0x7fbba81f70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:21.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.341+0000 7fbba7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbba81f6cd0 0x7fbba81f70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40534/0 (socket says 172.21.15.39:40534) 2024-09-21T07:52:21.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.341+0000 7fbba7e00640 1 -- 172.21.15.39:0/1930846012 learned_addr learned my addr 172.21.15.39:0/1930846012 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:21.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.341+0000 7fbb9fe00640 1 --2- 172.21.15.39:0/1930846012 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbba80a9e10 0x7fbba81f6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:21.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.341+0000 7fbba7400640 1 --2- 172.21.15.39:0/1930846012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb8c0041c0 0x7fbba814e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:21.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.341+0000 7fbba7e00640 1 -- 172.21.15.39:0/1930846012 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbba80a9e10 msgr2=0x7fbba81f6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:21.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.341+0000 7fbba7e00640 1 --2- 172.21.15.39:0/1930846012 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbba80a9e10 0x7fbba81f6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:21.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.341+0000 7fbba7e00640 1 -- 172.21.15.39:0/1930846012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb8c0041c0 msgr2=0x7fbba814e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:21.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.341+0000 7fbba7e00640 1 --2- 172.21.15.39:0/1930846012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb8c0041c0 0x7fbba814e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:21.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.341+0000 7fbba7e00640 1 -- 172.21.15.39:0/1930846012 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbba814e9d0 con 0x7fbba81f6cd0 2024-09-21T07:52:21.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.342+0000 7fbba7e00640 1 --2- 172.21.15.39:0/1930846012 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbba81f6cd0 0x7fbba81f70e0 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7fbba80731f0 tx=0x7fbba0006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:21.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.342+0000 7fbb9f400640 1 -- 172.21.15.39:0/1930846012 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba0011900 con 0x7fbba81f6cd0 2024-09-21T07:52:21.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.342+0000 7fbb9f400640 1 -- 172.21.15.39:0/1930846012 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbba0018070 con 0x7fbba81f6cd0 2024-09-21T07:52:21.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.342+0000 7fbbae000640 1 -- 172.21.15.39:0/1930846012 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbba80a9520 con 0x7fbba81f6cd0 2024-09-21T07:52:21.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.342+0000 7fbb9f400640 1 -- 172.21.15.39:0/1930846012 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba0011d20 con 0x7fbba81f6cd0 2024-09-21T07:52:21.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.342+0000 7fbbae000640 1 -- 172.21.15.39:0/1930846012 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbba80a9800 con 0x7fbba81f6cd0 2024-09-21T07:52:21.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.343+0000 7fbbae000640 1 -- 172.21.15.39:0/1930846012 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbba80d0a30 con 0x7fbba81f6cd0 2024-09-21T07:52:21.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.345+0000 7fbb9f400640 1 -- 172.21.15.39:0/1930846012 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbba001d460 con 0x7fbba81f6cd0 2024-09-21T07:52:21.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.346+0000 7fbb9f400640 1 --2- 172.21.15.39:0/1930846012 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbb84078bc0 0x7fbb8407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:21.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.346+0000 7fbb9f400640 1 -- 172.21.15.39:0/1930846012 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(246..246 src has 1..246) ==== 7562+0+0 (secure 0 0 0) 0x7fbba009bf20 con 0x7fbba81f6cd0 2024-09-21T07:52:21.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.347+0000 7fbba7400640 1 --2- 172.21.15.39:0/1930846012 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbb84078bc0 0x7fbb8407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:21.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.348+0000 7fbba7400640 1 --2- 172.21.15.39:0/1930846012 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbb84078bc0 0x7fbb8407b0d0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fbb98001860 tx=0x7fbb98001db0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:21.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.350+0000 7fbb9f400640 1 -- 172.21.15.39:0/1930846012 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbba0063b90 con 0x7fbba81f6cd0 2024-09-21T07:52:21.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.584+0000 7fbbae000640 1 -- 172.21.15.39:0/1930846012 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbba81f7680 con 0x7fbba81f6cd0 2024-09-21T07:52:21.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.584+0000 7fbb9f400640 1 -- 172.21.15.39:0/1930846012 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 279 v279) ==== 95+0+5280 (secure 0 0 0) 0x7fbba0063490 con 0x7fbba81f6cd0 2024-09-21T07:52:21.587 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:52:21.587 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":279,"btime":"2024-09-21T07:51:38:524572+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":278}],"filesystems":[{"mdsmap":{"epoch":279,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:51:37.529529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":246,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":10195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10195":{"gid":10195,"name":"d","rank":2,"incarnation":277,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[11052,9967]},"id":13}]} 2024-09-21T07:52:21.587 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 279 2024-09-21T07:52:21.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.587+0000 7fbbae000640 1 -- 172.21.15.39:0/1930846012 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbb84078bc0 msgr2=0x7fbb8407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:21.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.587+0000 7fbbae000640 1 --2- 172.21.15.39:0/1930846012 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbb84078bc0 0x7fbb8407b0d0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fbb98001860 tx=0x7fbb98001db0 comp rx=0 tx=0).stop 2024-09-21T07:52:21.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.587+0000 7fbbae000640 1 -- 172.21.15.39:0/1930846012 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbba81f6cd0 msgr2=0x7fbba81f70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:21.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.587+0000 7fbbae000640 1 --2- 172.21.15.39:0/1930846012 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbba81f6cd0 0x7fbba81f70e0 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7fbba80731f0 tx=0x7fbba0006430 comp rx=0 tx=0).stop 2024-09-21T07:52:21.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.587+0000 7fbbae000640 1 -- 172.21.15.39:0/1930846012 shutdown_connections 2024-09-21T07:52:21.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.587+0000 7fbbae000640 1 --2- 172.21.15.39:0/1930846012 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbb84078bc0 0x7fbb8407b0d0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:21.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.587+0000 7fbbae000640 1 --2- 172.21.15.39:0/1930846012 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbba81f6cd0 0x7fbba81f70e0 unknown :-1 s=CLOSED pgs=2512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:21.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.587+0000 7fbbae000640 1 --2- 172.21.15.39:0/1930846012 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbba80a9e10 0x7fbba81f6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:21.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.587+0000 7fbbae000640 1 --2- 172.21.15.39:0/1930846012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb8c0041c0 0x7fbba814e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:21.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.587+0000 7fbbae000640 1 -- 172.21.15.39:0/1930846012 >> 172.21.15.39:0/1930846012 conn(0x7fbba806cfc0 msgr2=0x7fbba806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:21.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.587+0000 7fbbae000640 1 -- 172.21.15.39:0/1930846012 shutdown_connections 2024-09-21T07:52:21.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:21.587+0000 7fbbae000640 1 -- 172.21.15.39:0/1930846012 wait complete. 2024-09-21T07:52:22.017 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 279, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:51:37.529529+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 246, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 9967, 'mds_1': 11052, 'mds_2': 10195}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9967': {'gid': 9967, 'name': 'b', 'rank': 0, 'incarnation': 259, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.148:6836/3381818843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 3381818843}, {'type': 'v1', 'addr': '172.21.15.148:6836', 'nonce': 3381818843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10195': {'gid': 10195, 'name': 'd', 'rank': 2, 'incarnation': 277, 'state': 'up:reconnect', 'state_seq': 11, 'addr': '172.21.15.148:6837/597301351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6835', 'nonce': 597301351}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 597301351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11052': {'gid': 11052, 'name': 'c', 'rank': 1, 'incarnation': 261, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6835/485194245', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 485194245}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 485194245}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9967, 'qdb_cluster': [11052, 9967]} 2024-09-21T07:52:22.017 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10195:up:reconnect 2024-09-21T07:52:23.018 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:52:23.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.256+0000 7fdbb6800640 1 -- 172.21.15.39:0/4241037248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbb00d0620 msgr2=0x7fdbb00d0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.256+0000 7fdbb6800640 1 --2- 172.21.15.39:0/4241037248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbb00d0620 0x7fdbb00d0a30 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7fdba8005fd0 tx=0x7fdba802aa90 comp rx=0 tx=0).stop 2024-09-21T07:52:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.256+0000 7fdbb6800640 1 -- 172.21.15.39:0/4241037248 shutdown_connections 2024-09-21T07:52:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.256+0000 7fdbb6800640 1 --2- 172.21.15.39:0/4241037248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbb00d0620 0x7fdbb00d0a30 unknown :-1 s=CLOSED pgs=2514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.256+0000 7fdbb6800640 1 --2- 172.21.15.39:0/4241037248 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdbb00cfd10 0x7fdbb00cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.256+0000 7fdbb6800640 1 -- 172.21.15.39:0/4241037248 >> 172.21.15.39:0/4241037248 conn(0x7fdbb006cfc0 msgr2=0x7fdbb006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.256+0000 7fdbb6800640 1 -- 172.21.15.39:0/4241037248 shutdown_connections 2024-09-21T07:52:23.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.257+0000 7fdbb6800640 1 -- 172.21.15.39:0/4241037248 wait complete. 2024-09-21T07:52:23.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.259+0000 7fdbb6800640 1 Processor -- start 2024-09-21T07:52:23.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.259+0000 7fdbb6800640 1 -- start start 2024-09-21T07:52:23.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.260+0000 7fdbb6800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbb00cfd10 0x7fdbb00ac9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:23.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.260+0000 7fdbb6800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdbb00d0620 0x7fdbb00ae350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:23.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.260+0000 7fdbb6800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbb015cd30 0x7fdbb01f6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:23.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.260+0000 7fdbb6800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb0073890 con 0x7fdbb00cfd10 2024-09-21T07:52:23.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.260+0000 7fdbb6800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb0073710 con 0x7fdbb015cd30 2024-09-21T07:52:23.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.260+0000 7fdbb6800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fdbb0073a10 con 0x7fdbb00d0620 2024-09-21T07:52:23.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.260+0000 7fdbaf400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbb00cfd10 0x7fdbb00ac9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:23.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.261+0000 7fdbaf400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbb00cfd10 0x7fdbb00ac9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35632/0 (socket says 172.21.15.39:35632) 2024-09-21T07:52:23.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.261+0000 7fdbaf400640 1 -- 172.21.15.39:0/3983137639 learned_addr learned my addr 172.21.15.39:0/3983137639 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:23.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.261+0000 7fdba6a00640 1 --2- 172.21.15.39:0/3983137639 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdbb00d0620 0x7fdbb00ae350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:23.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.261+0000 7fdbafe00640 1 --2- 172.21.15.39:0/3983137639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbb015cd30 0x7fdbb01f6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:23.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.261+0000 7fdbaf400640 1 -- 172.21.15.39:0/3983137639 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdbb00d0620 msgr2=0x7fdbb00ae350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:23.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.261+0000 7fdbaf400640 1 --2- 172.21.15.39:0/3983137639 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdbb00d0620 0x7fdbb00ae350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:23.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.261+0000 7fdbaf400640 1 -- 172.21.15.39:0/3983137639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbb015cd30 msgr2=0x7fdbb01f6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:23.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.261+0000 7fdbaf400640 1 --2- 172.21.15.39:0/3983137639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbb015cd30 0x7fdbb01f6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:23.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.261+0000 7fdbaf400640 1 -- 172.21.15.39:0/3983137639 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb00aec80 con 0x7fdbb00cfd10 2024-09-21T07:52:23.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.261+0000 7fdbafe00640 1 --2- 172.21.15.39:0/3983137639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbb015cd30 0x7fdbb01f6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:52:23.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.261+0000 7fdbaf400640 1 --2- 172.21.15.39:0/3983137639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbb00cfd10 0x7fdbb00ac9f0 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7fdba000c3f0 tx=0x7fdba000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:23.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.261+0000 7fdba7e00640 1 -- 172.21.15.39:0/3983137639 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba0017070 con 0x7fdbb00cfd10 2024-09-21T07:52:23.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.262+0000 7fdba7e00640 1 -- 172.21.15.39:0/3983137639 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdba00035f0 con 0x7fdbb00cfd10 2024-09-21T07:52:23.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.262+0000 7fdbb6800640 1 -- 172.21.15.39:0/3983137639 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb00aef60 con 0x7fdbb00cfd10 2024-09-21T07:52:23.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.262+0000 7fdba7e00640 1 -- 172.21.15.39:0/3983137639 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba0003a40 con 0x7fdbb00cfd10 2024-09-21T07:52:23.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.262+0000 7fdbb6800640 1 -- 172.21.15.39:0/3983137639 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbb01e7a40 con 0x7fdbb00cfd10 2024-09-21T07:52:23.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.263+0000 7fdbb6800640 1 -- 172.21.15.39:0/3983137639 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbb0073f30 con 0x7fdbb00cfd10 2024-09-21T07:52:23.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.264+0000 7fdba7e00640 1 -- 172.21.15.39:0/3983137639 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdba001a070 con 0x7fdbb00cfd10 2024-09-21T07:52:23.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.265+0000 7fdba7e00640 1 --2- 172.21.15.39:0/3983137639 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdb8c078bc0 0x7fdb8c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:23.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.265+0000 7fdba7e00640 1 -- 172.21.15.39:0/3983137639 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(246..246 src has 1..246) ==== 7562+0+0 (secure 0 0 0) 0x7fdba009a7d0 con 0x7fdbb00cfd10 2024-09-21T07:52:23.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.266+0000 7fdba6a00640 1 --2- 172.21.15.39:0/3983137639 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdb8c078bc0 0x7fdb8c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:23.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.266+0000 7fdba6a00640 1 --2- 172.21.15.39:0/3983137639 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdb8c078bc0 0x7fdb8c07b0d0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fdb98005fd0 tx=0x7fdb98005850 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:23.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.270+0000 7fdba7e00640 1 -- 172.21.15.39:0/3983137639 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdba00624d0 con 0x7fdbb00cfd10 2024-09-21T07:52:23.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.507+0000 7fdbb6800640 1 -- 172.21.15.39:0/3983137639 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdbb015c4a0 con 0x7fdbb00cfd10 2024-09-21T07:52:23.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.508+0000 7fdba7e00640 1 -- 172.21.15.39:0/3983137639 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 279 v279) ==== 95+0+5280 (secure 0 0 0) 0x7fdba0061dd0 con 0x7fdbb00cfd10 2024-09-21T07:52:23.510 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:52:23.510 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":279,"btime":"2024-09-21T07:51:38:524572+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":278}],"filesystems":[{"mdsmap":{"epoch":279,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:51:37.529529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":246,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":10195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10195":{"gid":10195,"name":"d","rank":2,"incarnation":277,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[11052,9967]},"id":13}]} 2024-09-21T07:52:23.510 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 279 2024-09-21T07:52:23.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.510+0000 7fdbb6800640 1 -- 172.21.15.39:0/3983137639 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdb8c078bc0 msgr2=0x7fdb8c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:23.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.510+0000 7fdbb6800640 1 --2- 172.21.15.39:0/3983137639 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdb8c078bc0 0x7fdb8c07b0d0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fdb98005fd0 tx=0x7fdb98005850 comp rx=0 tx=0).stop 2024-09-21T07:52:23.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.510+0000 7fdbb6800640 1 -- 172.21.15.39:0/3983137639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbb00cfd10 msgr2=0x7fdbb00ac9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:23.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.510+0000 7fdbb6800640 1 --2- 172.21.15.39:0/3983137639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbb00cfd10 0x7fdbb00ac9f0 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7fdba000c3f0 tx=0x7fdba000c8b0 comp rx=0 tx=0).stop 2024-09-21T07:52:23.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.511+0000 7fdbb6800640 1 -- 172.21.15.39:0/3983137639 shutdown_connections 2024-09-21T07:52:23.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.511+0000 7fdbb6800640 1 --2- 172.21.15.39:0/3983137639 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fdb8c078bc0 0x7fdb8c07b0d0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:23.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.511+0000 7fdbb6800640 1 --2- 172.21.15.39:0/3983137639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdbb015cd30 0x7fdbb01f6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:23.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.511+0000 7fdbb6800640 1 --2- 172.21.15.39:0/3983137639 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fdbb00d0620 0x7fdbb00ae350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:23.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.511+0000 7fdbb6800640 1 --2- 172.21.15.39:0/3983137639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbb00cfd10 0x7fdbb00ac9f0 unknown :-1 s=CLOSED pgs=2515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:23.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.511+0000 7fdbb6800640 1 -- 172.21.15.39:0/3983137639 >> 172.21.15.39:0/3983137639 conn(0x7fdbb006cfc0 msgr2=0x7fdbb0070f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:23.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.511+0000 7fdbb6800640 1 -- 172.21.15.39:0/3983137639 shutdown_connections 2024-09-21T07:52:23.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:23.511+0000 7fdbb6800640 1 -- 172.21.15.39:0/3983137639 wait complete. 2024-09-21T07:52:23.941 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 279, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:51:37.529529+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 246, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 9967, 'mds_1': 11052, 'mds_2': 10195}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9967': {'gid': 9967, 'name': 'b', 'rank': 0, 'incarnation': 259, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.148:6836/3381818843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 3381818843}, {'type': 'v1', 'addr': '172.21.15.148:6836', 'nonce': 3381818843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10195': {'gid': 10195, 'name': 'd', 'rank': 2, 'incarnation': 277, 'state': 'up:reconnect', 'state_seq': 11, 'addr': '172.21.15.148:6837/597301351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6835', 'nonce': 597301351}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 597301351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11052': {'gid': 11052, 'name': 'c', 'rank': 1, 'incarnation': 261, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6835/485194245', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 485194245}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 485194245}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9967, 'qdb_cluster': [11052, 9967]} 2024-09-21T07:52:23.942 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10195:up:reconnect 2024-09-21T07:52:24.942 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:52:25.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.182+0000 7f3914000640 1 -- 172.21.15.39:0/2404020907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38f4003a30 msgr2=0x7f38f4003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:25.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.182+0000 7f3914000640 1 --2- 172.21.15.39:0/2404020907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38f4003a30 0x7f38f4003ea0 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7f390000c310 tx=0x7f38f4005830 comp rx=0 tx=0).stop 2024-09-21T07:52:25.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.182+0000 7f3912000640 1 -- 172.21.15.39:0/2404020907 reap_dead start 2024-09-21T07:52:25.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.182+0000 7f3914000640 1 -- 172.21.15.39:0/2404020907 shutdown_connections 2024-09-21T07:52:25.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.182+0000 7f3914000640 1 -- 172.21.15.39:0/2404020907 >> 172.21.15.39:0/2404020907 conn(0x7f390c06cfc0 msgr2=0x7f390c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:25.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.182+0000 7f3914000640 1 -- 172.21.15.39:0/2404020907 shutdown_connections 2024-09-21T07:52:25.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.182+0000 7f3914000640 1 -- 172.21.15.39:0/2404020907 wait complete. 2024-09-21T07:52:25.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.184+0000 7f3914000640 1 Processor -- start 2024-09-21T07:52:25.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.184+0000 7f3914000640 1 -- start start 2024-09-21T07:52:25.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.186+0000 7f3914000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f38f4003a30 0x7f390c143a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:25.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.186+0000 7f3914000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f390c1463c0 0x7f390c1467d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:25.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.186+0000 7f3914000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f390c146d10 0x7f390c0ada60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:25.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.186+0000 7f3914000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f390c0cdef0 con 0x7f390c146d10 2024-09-21T07:52:25.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.186+0000 7f3914000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f390c0cdd70 con 0x7f390c1463c0 2024-09-21T07:52:25.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.186+0000 7f3914000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f390c073710 con 0x7f38f4003a30 2024-09-21T07:52:25.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.186+0000 7f3912000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f390c146d10 0x7f390c0ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:25.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.186+0000 7f3912000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f390c146d10 0x7f390c0ada60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35652/0 (socket says 172.21.15.39:35652) 2024-09-21T07:52:25.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.186+0000 7f3912000640 1 -- 172.21.15.39:0/1568769111 learned_addr learned my addr 172.21.15.39:0/1568769111 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:25.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.186+0000 7f3910c00640 1 --2- 172.21.15.39:0/1568769111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f390c1463c0 0x7f390c1467d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:25.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.186+0000 7f3911600640 1 --2- 172.21.15.39:0/1568769111 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f38f4003a30 0x7f390c143a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:25.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.187+0000 7f3912000640 1 -- 172.21.15.39:0/1568769111 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f38f4003a30 msgr2=0x7f390c143a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:25.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.187+0000 7f3912000640 1 --2- 172.21.15.39:0/1568769111 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f38f4003a30 0x7f390c143a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:25.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.187+0000 7f3912000640 1 -- 172.21.15.39:0/1568769111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f390c1463c0 msgr2=0x7f390c1467d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:25.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.187+0000 7f3912000640 1 --2- 172.21.15.39:0/1568769111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f390c1463c0 0x7f390c1467d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:25.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.187+0000 7f3912000640 1 -- 172.21.15.39:0/1568769111 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f390c1f6d60 con 0x7f390c146d10 2024-09-21T07:52:25.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.187+0000 7f3910c00640 1 --2- 172.21.15.39:0/1568769111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f390c1463c0 0x7f390c1467d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:52:25.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.187+0000 7f3912000640 1 --2- 172.21.15.39:0/1568769111 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f390c146d10 0x7f390c0ada60 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7f390c0d1210 tx=0x7f39040019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:25.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.187+0000 7f3909600640 1 -- 172.21.15.39:0/1568769111 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3904011870 con 0x7f390c146d10 2024-09-21T07:52:25.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.187+0000 7f3909600640 1 -- 172.21.15.39:0/1568769111 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3904017070 con 0x7f390c146d10 2024-09-21T07:52:25.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.187+0000 7f3914000640 1 -- 172.21.15.39:0/1568769111 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f390c1f7040 con 0x7f390c146d10 2024-09-21T07:52:25.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.187+0000 7f3909600640 1 -- 172.21.15.39:0/1568769111 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3904011da0 con 0x7f390c146d10 2024-09-21T07:52:25.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.187+0000 7f3914000640 1 -- 172.21.15.39:0/1568769111 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f390c1f7600 con 0x7f390c146d10 2024-09-21T07:52:25.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.188+0000 7f3914000640 1 -- 172.21.15.39:0/1568769111 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f390c073ca0 con 0x7f390c146d10 2024-09-21T07:52:25.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.189+0000 7f3909600640 1 -- 172.21.15.39:0/1568769111 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f390401c460 con 0x7f390c146d10 2024-09-21T07:52:25.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.190+0000 7f3909600640 1 --2- 172.21.15.39:0/1568769111 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f38ec078b90 0x7f38ec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:25.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.191+0000 7f3909600640 1 -- 172.21.15.39:0/1568769111 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(246..246 src has 1..246) ==== 7562+0+0 (secure 0 0 0) 0x7f390409bd70 con 0x7f390c146d10 2024-09-21T07:52:25.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.191+0000 7f3911600640 1 --2- 172.21.15.39:0/1568769111 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f38ec078b90 0x7f38ec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:25.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.192+0000 7f3911600640 1 --2- 172.21.15.39:0/1568769111 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f38ec078b90 0x7f38ec07b0a0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f3900033000 tx=0x7f39000019f0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:25.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.195+0000 7f3909600640 1 -- 172.21.15.39:0/1568769111 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f39040639d0 con 0x7f390c146d10 2024-09-21T07:52:25.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.433+0000 7f3914000640 1 -- 172.21.15.39:0/1568769111 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f390c1f6af0 con 0x7f390c146d10 2024-09-21T07:52:25.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.433+0000 7f3909600640 1 -- 172.21.15.39:0/1568769111 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 279 v279) ==== 95+0+5280 (secure 0 0 0) 0x7f39040632d0 con 0x7f390c146d10 2024-09-21T07:52:25.435 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:52:25.435 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":279,"btime":"2024-09-21T07:51:38:524572+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":278}],"filesystems":[{"mdsmap":{"epoch":279,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:51:37.529529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":246,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":10195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10195":{"gid":10195,"name":"d","rank":2,"incarnation":277,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[11052,9967]},"id":13}]} 2024-09-21T07:52:25.436 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 279 2024-09-21T07:52:25.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.436+0000 7f3914000640 1 -- 172.21.15.39:0/1568769111 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f38ec078b90 msgr2=0x7f38ec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:25.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.436+0000 7f3914000640 1 --2- 172.21.15.39:0/1568769111 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f38ec078b90 0x7f38ec07b0a0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f3900033000 tx=0x7f39000019f0 comp rx=0 tx=0).stop 2024-09-21T07:52:25.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.436+0000 7f3914000640 1 -- 172.21.15.39:0/1568769111 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f390c146d10 msgr2=0x7f390c0ada60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:25.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.436+0000 7f3914000640 1 --2- 172.21.15.39:0/1568769111 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f390c146d10 0x7f390c0ada60 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7f390c0d1210 tx=0x7f39040019b0 comp rx=0 tx=0).stop 2024-09-21T07:52:25.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.436+0000 7f3914000640 1 -- 172.21.15.39:0/1568769111 shutdown_connections 2024-09-21T07:52:25.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.436+0000 7f3914000640 1 --2- 172.21.15.39:0/1568769111 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f38ec078b90 0x7f38ec07b0a0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:25.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.436+0000 7f3914000640 1 --2- 172.21.15.39:0/1568769111 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f390c146d10 0x7f390c0ada60 unknown :-1 s=CLOSED pgs=2518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:25.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.436+0000 7f3914000640 1 --2- 172.21.15.39:0/1568769111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f390c1463c0 0x7f390c1467d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:25.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.436+0000 7f3914000640 1 --2- 172.21.15.39:0/1568769111 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f38f4003a30 0x7f390c143a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:25.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.436+0000 7f3914000640 1 -- 172.21.15.39:0/1568769111 >> 172.21.15.39:0/1568769111 conn(0x7f390c06cfc0 msgr2=0x7f390c0afe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:25.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.436+0000 7f3914000640 1 -- 172.21.15.39:0/1568769111 shutdown_connections 2024-09-21T07:52:25.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:25.436+0000 7f3914000640 1 -- 172.21.15.39:0/1568769111 wait complete. 2024-09-21T07:52:25.865 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 279, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:51:37.529529+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 246, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 9967, 'mds_1': 11052, 'mds_2': 10195}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9967': {'gid': 9967, 'name': 'b', 'rank': 0, 'incarnation': 259, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.148:6836/3381818843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 3381818843}, {'type': 'v1', 'addr': '172.21.15.148:6836', 'nonce': 3381818843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10195': {'gid': 10195, 'name': 'd', 'rank': 2, 'incarnation': 277, 'state': 'up:reconnect', 'state_seq': 11, 'addr': '172.21.15.148:6837/597301351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6835', 'nonce': 597301351}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 597301351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11052': {'gid': 11052, 'name': 'c', 'rank': 1, 'incarnation': 261, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6835/485194245', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 485194245}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 485194245}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9967, 'qdb_cluster': [11052, 9967]} 2024-09-21T07:52:25.865 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10195:up:reconnect 2024-09-21T07:52:26.866 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:52:27.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.106+0000 7f1f87000640 1 -- 172.21.15.39:0/1556083032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f64004800 msgr2=0x7f1f64004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:27.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.106+0000 7f1f87000640 1 --2- 172.21.15.39:0/1556083032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f64004800 0x7f1f64004c10 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7f1f80073100 tx=0x7f1f64005830 comp rx=0 tx=0).stop 2024-09-21T07:52:27.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.106+0000 7f1f87000640 1 -- 172.21.15.39:0/1556083032 shutdown_connections 2024-09-21T07:52:27.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.106+0000 7f1f87000640 1 --2- 172.21.15.39:0/1556083032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f64004800 0x7f1f64004c10 unknown :-1 s=CLOSED pgs=2520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:27.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.106+0000 7f1f87000640 1 -- 172.21.15.39:0/1556083032 >> 172.21.15.39:0/1556083032 conn(0x7f1f8006cfc0 msgr2=0x7f1f8006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:27.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.106+0000 7f1f87000640 1 -- 172.21.15.39:0/1556083032 shutdown_connections 2024-09-21T07:52:27.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.106+0000 7f1f87000640 1 -- 172.21.15.39:0/1556083032 wait complete. 2024-09-21T07:52:27.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.109+0000 7f1f87000640 1 Processor -- start 2024-09-21T07:52:27.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.109+0000 7f1f87000640 1 -- start start 2024-09-21T07:52:27.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.110+0000 7f1f87000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f64004800 0x7f1f801445c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:27.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.110+0000 7f1f87000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f8014d0d0 0x7f1f80143c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:27.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.110+0000 7f1f87000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f801594c0 0x7f1f800b4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:27.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.110+0000 7f1f87000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8006f800 con 0x7f1f8014d0d0 2024-09-21T07:52:27.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.110+0000 7f1f87000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1f800d0e90 con 0x7f1f64004800 2024-09-21T07:52:27.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.110+0000 7f1f87000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f1f8006f980 con 0x7f1f801594c0 2024-09-21T07:52:27.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.110+0000 7f1f77e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f8014d0d0 0x7f1f80143c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:27.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.111+0000 7f1f77e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f8014d0d0 0x7f1f80143c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35670/0 (socket says 172.21.15.39:35670) 2024-09-21T07:52:27.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.111+0000 7f1f77e00640 1 -- 172.21.15.39:0/3897510479 learned_addr learned my addr 172.21.15.39:0/3897510479 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:27.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.111+0000 7f1f85000640 1 --2- 172.21.15.39:0/3897510479 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f801594c0 0x7f1f800b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:27.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.111+0000 7f1f7fe00640 1 --2- 172.21.15.39:0/3897510479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f64004800 0x7f1f801445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:27.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.111+0000 7f1f77e00640 1 -- 172.21.15.39:0/3897510479 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f801594c0 msgr2=0x7f1f800b4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:27.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.111+0000 7f1f77e00640 1 --2- 172.21.15.39:0/3897510479 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f801594c0 0x7f1f800b4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:27.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.111+0000 7f1f77e00640 1 -- 172.21.15.39:0/3897510479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f64004800 msgr2=0x7f1f801445c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:27.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.111+0000 7f1f77e00640 1 --2- 172.21.15.39:0/3897510479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f64004800 0x7f1f801445c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:27.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.111+0000 7f1f77e00640 1 -- 172.21.15.39:0/3897510479 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f80158420 con 0x7f1f8014d0d0 2024-09-21T07:52:27.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.111+0000 7f1f77e00640 1 --2- 172.21.15.39:0/3897510479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f8014d0d0 0x7f1f80143c80 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7f1f68036040 tx=0x7f1f680336f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:27.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.111+0000 7f1f7cc00640 1 -- 172.21.15.39:0/3897510479 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f68038040 con 0x7f1f8014d0d0 2024-09-21T07:52:27.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.112+0000 7f1f7cc00640 1 -- 172.21.15.39:0/3897510479 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f680339d0 con 0x7f1f8014d0d0 2024-09-21T07:52:27.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.112+0000 7f1f7cc00640 1 -- 172.21.15.39:0/3897510479 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f68033e20 con 0x7f1f8014d0d0 2024-09-21T07:52:27.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.112+0000 7f1f87000640 1 -- 172.21.15.39:0/3897510479 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f80158700 con 0x7f1f8014d0d0 2024-09-21T07:52:27.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.112+0000 7f1f87000640 1 -- 172.21.15.39:0/3897510479 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f80158cc0 con 0x7f1f8014d0d0 2024-09-21T07:52:27.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.113+0000 7f1f87000640 1 -- 172.21.15.39:0/3897510479 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f800b57e0 con 0x7f1f8014d0d0 2024-09-21T07:52:27.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.114+0000 7f1f7cc00640 1 -- 172.21.15.39:0/3897510479 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1f68043020 con 0x7f1f8014d0d0 2024-09-21T07:52:27.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.115+0000 7f1f7cc00640 1 --2- 172.21.15.39:0/3897510479 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1f5c078bc0 0x7f1f5c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:27.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.115+0000 7f1f7cc00640 1 -- 172.21.15.39:0/3897510479 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(247..247 src has 1..247) ==== 7605+0+0 (secure 0 0 0) 0x7f1f680bb730 con 0x7f1f8014d0d0 2024-09-21T07:52:27.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.116+0000 7f1f7fe00640 1 --2- 172.21.15.39:0/3897510479 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1f5c078bc0 0x7f1f5c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:27.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.116+0000 7f1f7fe00640 1 --2- 172.21.15.39:0/3897510479 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1f5c078bc0 0x7f1f5c07b0d0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f1f70000c00 tx=0x7f1f70006c80 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:27.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.120+0000 7f1f7cc00640 1 -- 172.21.15.39:0/3897510479 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1f800b57e0 con 0x7f1f8014d0d0 2024-09-21T07:52:27.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.354+0000 7f1f87000640 1 -- 172.21.15.39:0/3897510479 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f8006e110 con 0x7f1f8014d0d0 2024-09-21T07:52:27.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.355+0000 7f1f7cc00640 1 -- 172.21.15.39:0/3897510479 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 279 v279) ==== 95+0+5280 (secure 0 0 0) 0x7f1f6804a020 con 0x7f1f8014d0d0 2024-09-21T07:52:27.357 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:52:27.357 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":279,"btime":"2024-09-21T07:51:38:524572+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":278}],"filesystems":[{"mdsmap":{"epoch":279,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:51:37.529529+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":246,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":10195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10195":{"gid":10195,"name":"d","rank":2,"incarnation":277,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[11052,9967]},"id":13}]} 2024-09-21T07:52:27.358 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 279 2024-09-21T07:52:27.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.357+0000 7f1f87000640 1 -- 172.21.15.39:0/3897510479 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1f5c078bc0 msgr2=0x7f1f5c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:27.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.357+0000 7f1f87000640 1 --2- 172.21.15.39:0/3897510479 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1f5c078bc0 0x7f1f5c07b0d0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f1f70000c00 tx=0x7f1f70006c80 comp rx=0 tx=0).stop 2024-09-21T07:52:27.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.357+0000 7f1f87000640 1 -- 172.21.15.39:0/3897510479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f8014d0d0 msgr2=0x7f1f80143c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:27.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.357+0000 7f1f87000640 1 --2- 172.21.15.39:0/3897510479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f8014d0d0 0x7f1f80143c80 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7f1f68036040 tx=0x7f1f680336f0 comp rx=0 tx=0).stop 2024-09-21T07:52:27.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.357+0000 7f1f87000640 1 -- 172.21.15.39:0/3897510479 shutdown_connections 2024-09-21T07:52:27.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.357+0000 7f1f87000640 1 --2- 172.21.15.39:0/3897510479 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1f5c078bc0 0x7f1f5c07b0d0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:27.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.357+0000 7f1f87000640 1 --2- 172.21.15.39:0/3897510479 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1f801594c0 0x7f1f800b4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:27.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.358+0000 7f1f87000640 1 --2- 172.21.15.39:0/3897510479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f8014d0d0 0x7f1f80143c80 unknown :-1 s=CLOSED pgs=2521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:27.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.358+0000 7f1f87000640 1 --2- 172.21.15.39:0/3897510479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f64004800 0x7f1f801445c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:27.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.358+0000 7f1f87000640 1 -- 172.21.15.39:0/3897510479 >> 172.21.15.39:0/3897510479 conn(0x7f1f8006cfc0 msgr2=0x7f1f80153670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:27.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.358+0000 7f1f87000640 1 -- 172.21.15.39:0/3897510479 shutdown_connections 2024-09-21T07:52:27.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:27.358+0000 7f1f87000640 1 -- 172.21.15.39:0/3897510479 wait complete. 2024-09-21T07:52:27.784 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 279, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:51:37.529529+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 246, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 9967, 'mds_1': 11052, 'mds_2': 10195}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9967': {'gid': 9967, 'name': 'b', 'rank': 0, 'incarnation': 259, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.148:6836/3381818843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 3381818843}, {'type': 'v1', 'addr': '172.21.15.148:6836', 'nonce': 3381818843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10195': {'gid': 10195, 'name': 'd', 'rank': 2, 'incarnation': 277, 'state': 'up:reconnect', 'state_seq': 11, 'addr': '172.21.15.148:6837/597301351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6835', 'nonce': 597301351}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 597301351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11052': {'gid': 11052, 'name': 'c', 'rank': 1, 'incarnation': 261, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6835/485194245', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 485194245}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 485194245}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9967, 'qdb_cluster': [11052, 9967]} 2024-09-21T07:52:27.784 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10195:up:reconnect 2024-09-21T07:52:28.785 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:52:29.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.016+0000 7fc95ae00640 1 -- 172.21.15.39:0/439404466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc95406f800 msgr2=0x7fc9540cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:29.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.016+0000 7fc95ae00640 1 --2- 172.21.15.39:0/439404466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc95406f800 0x7fc9540cd8e0 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 crypto rx=0x7fc93c009830 tx=0x7fc93c02a510 comp rx=0 tx=0).stop 2024-09-21T07:52:29.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.016+0000 7fc95ae00640 1 -- 172.21.15.39:0/439404466 shutdown_connections 2024-09-21T07:52:29.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.016+0000 7fc95ae00640 1 --2- 172.21.15.39:0/439404466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc95406f800 0x7fc9540cd8e0 unknown :-1 s=CLOSED pgs=2523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.016+0000 7fc95ae00640 1 --2- 172.21.15.39:0/439404466 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc95415cd30 0x7fc9540cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.016+0000 7fc95ae00640 1 -- 172.21.15.39:0/439404466 >> 172.21.15.39:0/439404466 conn(0x7fc95406cfc0 msgr2=0x7fc95406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:29.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.016+0000 7fc95ae00640 1 -- 172.21.15.39:0/439404466 shutdown_connections 2024-09-21T07:52:29.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.016+0000 7fc95ae00640 1 -- 172.21.15.39:0/439404466 wait complete. 2024-09-21T07:52:29.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.018+0000 7fc95ae00640 1 Processor -- start 2024-09-21T07:52:29.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.019+0000 7fc95ae00640 1 -- start start 2024-09-21T07:52:29.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.020+0000 7fc95ae00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc95406f800 0x7fc95414f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:29.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.020+0000 7fc95ae00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc95406fc10 0x7fc95414cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:29.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.020+0000 7fc95ae00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc95415cd30 0x7fc95413f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:29.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.020+0000 7fc95ae00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc954073890 con 0x7fc95406fc10 2024-09-21T07:52:29.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.020+0000 7fc95ae00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc954073710 con 0x7fc95415cd30 2024-09-21T07:52:29.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.020+0000 7fc95ae00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc954073a10 con 0x7fc95406f800 2024-09-21T07:52:29.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.020+0000 7fc94be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc95406fc10 0x7fc95414cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:29.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.020+0000 7fc94be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc95406fc10 0x7fc95414cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35696/0 (socket says 172.21.15.39:35696) 2024-09-21T07:52:29.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.020+0000 7fc94be00640 1 -- 172.21.15.39:0/655881509 learned_addr learned my addr 172.21.15.39:0/655881509 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:29.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.020+0000 7fc953e00640 1 --2- 172.21.15.39:0/655881509 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc95406f800 0x7fc95414f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:29.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.021+0000 7fc958e00640 1 --2- 172.21.15.39:0/655881509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc95415cd30 0x7fc95413f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:29.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.021+0000 7fc94be00640 1 -- 172.21.15.39:0/655881509 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc95406f800 msgr2=0x7fc95414f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:29.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.021+0000 7fc94be00640 1 --2- 172.21.15.39:0/655881509 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc95406f800 0x7fc95414f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.021+0000 7fc94be00640 1 -- 172.21.15.39:0/655881509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc95415cd30 msgr2=0x7fc95413f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:29.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.021+0000 7fc94be00640 1 --2- 172.21.15.39:0/655881509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc95415cd30 0x7fc95413f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.021+0000 7fc94be00640 1 -- 172.21.15.39:0/655881509 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc95414e8e0 con 0x7fc95406fc10 2024-09-21T07:52:29.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.021+0000 7fc953e00640 1 --2- 172.21.15.39:0/655881509 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc95406f800 0x7fc95414f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:52:29.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.021+0000 7fc94be00640 1 --2- 172.21.15.39:0/655881509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc95406fc10 0x7fc95414cab0 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7fc93c009950 tx=0x7fc93c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:29.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.021+0000 7fc950c00640 1 -- 172.21.15.39:0/655881509 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c031d60 con 0x7fc95406fc10 2024-09-21T07:52:29.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.021+0000 7fc950c00640 1 -- 172.21.15.39:0/655881509 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc93c033e10 con 0x7fc95406fc10 2024-09-21T07:52:29.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.021+0000 7fc95ae00640 1 -- 172.21.15.39:0/655881509 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc954150bc0 con 0x7fc95406fc10 2024-09-21T07:52:29.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.021+0000 7fc950c00640 1 -- 172.21.15.39:0/655881509 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c005260 con 0x7fc95406fc10 2024-09-21T07:52:29.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.022+0000 7fc95ae00640 1 -- 172.21.15.39:0/655881509 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc95414ee60 con 0x7fc95406fc10 2024-09-21T07:52:29.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.023+0000 7fc95ae00640 1 -- 172.21.15.39:0/655881509 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc954073ec0 con 0x7fc95406fc10 2024-09-21T07:52:29.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.024+0000 7fc950c00640 1 -- 172.21.15.39:0/655881509 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc93c005400 con 0x7fc95406fc10 2024-09-21T07:52:29.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.025+0000 7fc950c00640 1 --2- 172.21.15.39:0/655881509 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc930078bc0 0x7fc93007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:29.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.025+0000 7fc950c00640 1 -- 172.21.15.39:0/655881509 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(247..247 src has 1..247) ==== 7605+0+0 (secure 0 0 0) 0x7fc93c0bc760 con 0x7fc95406fc10 2024-09-21T07:52:29.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.025+0000 7fc953e00640 1 --2- 172.21.15.39:0/655881509 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc930078bc0 0x7fc93007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:29.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.026+0000 7fc953e00640 1 --2- 172.21.15.39:0/655881509 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc930078bc0 0x7fc93007b0d0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fc944005fd0 tx=0x7fc94400f360 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:29.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.029+0000 7fc950c00640 1 -- 172.21.15.39:0/655881509 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc93c03f030 con 0x7fc95406fc10 2024-09-21T07:52:29.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.266+0000 7fc95ae00640 1 -- 172.21.15.39:0/655881509 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc9540cfdb0 con 0x7fc95406fc10 2024-09-21T07:52:29.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.267+0000 7fc950c00640 1 -- 172.21.15.39:0/655881509 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 281 v281) ==== 95+0+5283 (secure 0 0 0) 0x7fc93c084390 con 0x7fc95406fc10 2024-09-21T07:52:29.269 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:52:29.269 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":281,"btime":"2024-09-21T07:52:29:069596+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":278}],"filesystems":[{"mdsmap":{"epoch":281,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:52:29.069589+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":246,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":10195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10195":{"gid":10195,"name":"d","rank":2,"incarnation":277,"state":"up:active","state_seq":25,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[9967,10195,11052]},"id":13}]} 2024-09-21T07:52:29.269 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 281 2024-09-21T07:52:29.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.269+0000 7fc95ae00640 1 -- 172.21.15.39:0/655881509 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc930078bc0 msgr2=0x7fc93007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:29.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.269+0000 7fc95ae00640 1 --2- 172.21.15.39:0/655881509 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc930078bc0 0x7fc93007b0d0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fc944005fd0 tx=0x7fc94400f360 comp rx=0 tx=0).stop 2024-09-21T07:52:29.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.269+0000 7fc95ae00640 1 -- 172.21.15.39:0/655881509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc95406fc10 msgr2=0x7fc95414cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:29.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.269+0000 7fc95ae00640 1 --2- 172.21.15.39:0/655881509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc95406fc10 0x7fc95414cab0 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7fc93c009950 tx=0x7fc93c033c20 comp rx=0 tx=0).stop 2024-09-21T07:52:29.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.269+0000 7fc95ae00640 1 -- 172.21.15.39:0/655881509 shutdown_connections 2024-09-21T07:52:29.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.269+0000 7fc95ae00640 1 --2- 172.21.15.39:0/655881509 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc930078bc0 0x7fc93007b0d0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.269+0000 7fc95ae00640 1 --2- 172.21.15.39:0/655881509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc95415cd30 0x7fc95413f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.269+0000 7fc95ae00640 1 --2- 172.21.15.39:0/655881509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc95406fc10 0x7fc95414cab0 unknown :-1 s=CLOSED pgs=2524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.269+0000 7fc95ae00640 1 --2- 172.21.15.39:0/655881509 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc95406f800 0x7fc95414f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.269+0000 7fc95ae00640 1 -- 172.21.15.39:0/655881509 >> 172.21.15.39:0/655881509 conn(0x7fc95406cfc0 msgr2=0x7fc954070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:29.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.269+0000 7fc95ae00640 1 -- 172.21.15.39:0/655881509 shutdown_connections 2024-09-21T07:52:29.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.270+0000 7fc95ae00640 1 -- 172.21.15.39:0/655881509 wait complete. 2024-09-21T07:52:29.697 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 281, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:52:29.069589+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 246, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 9967, 'mds_1': 11052, 'mds_2': 10195}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9967': {'gid': 9967, 'name': 'b', 'rank': 0, 'incarnation': 259, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.148:6836/3381818843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 3381818843}, {'type': 'v1', 'addr': '172.21.15.148:6836', 'nonce': 3381818843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_10195': {'gid': 10195, 'name': 'd', 'rank': 2, 'incarnation': 277, 'state': 'up:active', 'state_seq': 25, 'addr': '172.21.15.148:6837/597301351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6835', 'nonce': 597301351}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 597301351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11052': {'gid': 11052, 'name': 'c', 'rank': 1, 'incarnation': 261, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6835/485194245', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 485194245}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 485194245}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9967, 'qdb_cluster': [9967, 10195, 11052]} 2024-09-21T07:52:29.697 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-21T07:52:29.697 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-21T07:52:29.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.924+0000 7f59b3000640 1 -- 172.21.15.39:0/1739413342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59ac0cb8a0 msgr2=0x7f59ac166490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:29.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.924+0000 7f59b3000640 1 --2- 172.21.15.39:0/1739413342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59ac0cb8a0 0x7f59ac166490 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7f59a4005fd0 tx=0x7f59a402aa10 comp rx=0 tx=0).stop 2024-09-21T07:52:29.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.924+0000 7f59b3000640 1 -- 172.21.15.39:0/1739413342 shutdown_connections 2024-09-21T07:52:29.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.924+0000 7f59b3000640 1 --2- 172.21.15.39:0/1739413342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59ac0cb8a0 0x7f59ac166490 unknown :-1 s=CLOSED pgs=2526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.924+0000 7f59b3000640 1 -- 172.21.15.39:0/1739413342 >> 172.21.15.39:0/1739413342 conn(0x7f59ac0b2510 msgr2=0x7f59ac0d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:29.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.924+0000 7f59b3000640 1 -- 172.21.15.39:0/1739413342 shutdown_connections 2024-09-21T07:52:29.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.924+0000 7f59b3000640 1 -- 172.21.15.39:0/1739413342 wait complete. 2024-09-21T07:52:29.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.926+0000 7f59b3000640 1 Processor -- start 2024-09-21T07:52:29.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.927+0000 7f59b3000640 1 -- start start 2024-09-21T07:52:29.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.928+0000 7f59b3000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59ac0cb8a0 0x7f59ac14e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:29.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.928+0000 7f59b3000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59ac0cc390 0x7f59ac099490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:29.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.928+0000 7f59b3000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59ac0ce310 0x7f59ac073e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:29.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.928+0000 7f59b3000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f59ac166b50 con 0x7f59ac0cc390 2024-09-21T07:52:29.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.928+0000 7f59b3000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f59ac1669d0 con 0x7f59ac0cb8a0 2024-09-21T07:52:29.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.928+0000 7f59b3000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f59ac166cd0 con 0x7f59ac0ce310 2024-09-21T07:52:29.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.928+0000 7f59b1200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59ac0cc390 0x7f59ac099490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:29.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.928+0000 7f59b1200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59ac0cc390 0x7f59ac099490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35716/0 (socket says 172.21.15.39:35716) 2024-09-21T07:52:29.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.928+0000 7f59b1200640 1 -- 172.21.15.39:0/383578351 learned_addr learned my addr 172.21.15.39:0/383578351 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:29.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.928+0000 7f59b1c00640 1 --2- 172.21.15.39:0/383578351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59ac0cb8a0 0x7f59ac14e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:29.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.928+0000 7f59b2600640 1 --2- 172.21.15.39:0/383578351 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59ac0ce310 0x7f59ac073e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:29.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.929+0000 7f59b1200640 1 -- 172.21.15.39:0/383578351 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59ac0ce310 msgr2=0x7f59ac073e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:29.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.929+0000 7f59b1200640 1 --2- 172.21.15.39:0/383578351 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59ac0ce310 0x7f59ac073e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.929+0000 7f59b1200640 1 -- 172.21.15.39:0/383578351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59ac0cb8a0 msgr2=0x7f59ac14e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:29.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.929+0000 7f59b1200640 1 --2- 172.21.15.39:0/383578351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59ac0cb8a0 0x7f59ac14e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.929+0000 7f59b1200640 1 -- 172.21.15.39:0/383578351 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59ac15c0f0 con 0x7f59ac0cc390 2024-09-21T07:52:29.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.929+0000 7f59b1200640 1 --2- 172.21.15.39:0/383578351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59ac0cc390 0x7f59ac099490 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 crypto rx=0x7f599800c3f0 tx=0x7f599800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:29.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.929+0000 7f59a9600640 1 -- 172.21.15.39:0/383578351 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5998017070 con 0x7f59ac0cc390 2024-09-21T07:52:29.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.929+0000 7f59a9600640 1 -- 172.21.15.39:0/383578351 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f59980035f0 con 0x7f59ac0cc390 2024-09-21T07:52:29.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.929+0000 7f59a9600640 1 -- 172.21.15.39:0/383578351 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5998003a40 con 0x7f59ac0cc390 2024-09-21T07:52:29.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.929+0000 7f59b3000640 1 -- 172.21.15.39:0/383578351 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59ac15c3d0 con 0x7f59ac0cc390 2024-09-21T07:52:29.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.930+0000 7f59b3000640 1 -- 172.21.15.39:0/383578351 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59ac14ce00 con 0x7f59ac0cc390 2024-09-21T07:52:29.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.932+0000 7f59a9600640 1 -- 172.21.15.39:0/383578351 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f599801a070 con 0x7f59ac0cc390 2024-09-21T07:52:29.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.933+0000 7f59a9600640 1 --2- 172.21.15.39:0/383578351 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f598c078b90 0x7f598c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:29.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.933+0000 7f59a9600640 1 -- 172.21.15.39:0/383578351 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(247..247 src has 1..247) ==== 7605+0+0 (secure 0 0 0) 0x7f599809a7f0 con 0x7f59ac0cc390 2024-09-21T07:52:29.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.933+0000 7f59bb699740 1 Processor -- start 2024-09-21T07:52:29.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.934+0000 7f59b1c00640 1 --2- 172.21.15.39:0/383578351 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f598c078b90 0x7f598c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:29.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.934+0000 7f59bb699740 1 -- start start 2024-09-21T07:52:29.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.935+0000 7f59b1c00640 1 --2- 172.21.15.39:0/383578351 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f598c078b90 0x7f598c07b0a0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f59a0006fd0 tx=0x7f59a0009060 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:29.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.935+0000 7f59bb699740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5575862e7270 0x5575866d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:29.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.935+0000 7f59bb699740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5575862be3b0 0x557586688150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:29.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.935+0000 7f59bb699740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5575862c6590 0x557586688690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:29.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.935+0000 7f59bb699740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557586611430 con 0x5575862c6590 2024-09-21T07:52:29.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.935+0000 7f59bb699740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5575866015d0 con 0x5575862e7270 2024-09-21T07:52:29.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.935+0000 7f59bb699740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x5575865f0a00 con 0x5575862be3b0 2024-09-21T07:52:29.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.936+0000 7f59b2600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5575862c6590 0x557586688690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:29.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.936+0000 7f59b2600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5575862c6590 0x557586688690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35732/0 (socket says 172.21.15.39:35732) 2024-09-21T07:52:29.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.936+0000 7f59b2600640 1 -- 172.21.15.39:0/4201185036 learned_addr learned my addr 172.21.15.39:0/4201185036 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:29.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.936+0000 7f59b1c00640 1 --2- 172.21.15.39:0/4201185036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5575862e7270 0x5575866d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:29.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.936+0000 7f59b1200640 1 --2- 172.21.15.39:0/4201185036 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5575862be3b0 0x557586688150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:29.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.936+0000 7f59b2600640 1 -- 172.21.15.39:0/4201185036 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5575862be3b0 msgr2=0x557586688150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:52:29.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.936+0000 7f59b2600640 1 --2- 172.21.15.39:0/4201185036 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5575862be3b0 0x557586688150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.936+0000 7f59b2600640 1 -- 172.21.15.39:0/4201185036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5575862e7270 msgr2=0x5575866d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:52:29.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.936+0000 7f59b2600640 1 --2- 172.21.15.39:0/4201185036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5575862e7270 0x5575866d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.936+0000 7f59b2600640 1 -- 172.21.15.39:0/4201185036 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575865b9520 con 0x5575862c6590 2024-09-21T07:52:29.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.936+0000 7f59b2600640 1 --2- 172.21.15.39:0/4201185036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5575862c6590 0x557586688690 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7f59a402b000 tx=0x7f59a4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acc20d5c3cf0aefd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:29.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.937+0000 7f599e000640 1 -- 172.21.15.39:0/4201185036 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a4047070 con 0x5575862c6590 2024-09-21T07:52:29.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.937+0000 7f599e000640 1 -- 172.21.15.39:0/4201185036 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f59a4003db0 con 0x5575862c6590 2024-09-21T07:52:29.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.937+0000 7f599e000640 1 -- 172.21.15.39:0/4201185036 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a4033070 con 0x5575862c6590 2024-09-21T07:52:29.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.937+0000 7f59bb699740 1 -- 172.21.15.39:0/4201185036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5575862c6590 msgr2=0x557586688690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:29.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.937+0000 7f59bb699740 1 --2- 172.21.15.39:0/4201185036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5575862c6590 0x557586688690 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7f59a402b000 tx=0x7f59a4004ab0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.938+0000 7f59bb699740 1 -- 172.21.15.39:0/4201185036 shutdown_connections 2024-09-21T07:52:29.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.938+0000 7f59bb699740 1 --2- 172.21.15.39:0/4201185036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5575862c6590 0x557586688690 unknown :-1 s=CLOSED pgs=2528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.938+0000 7f59bb699740 1 --2- 172.21.15.39:0/4201185036 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5575862be3b0 0x557586688150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.938+0000 7f59bb699740 1 --2- 172.21.15.39:0/4201185036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5575862e7270 0x5575866d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.938+0000 7f59bb699740 1 -- 172.21.15.39:0/4201185036 >> 172.21.15.39:0/4201185036 conn(0x5575862db3c0 msgr2=0x55758668db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:29.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.938+0000 7f59bb699740 1 -- 172.21.15.39:0/4201185036 shutdown_connections 2024-09-21T07:52:29.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.938+0000 7f59bb699740 1 -- 172.21.15.39:0/4201185036 wait complete. 2024-09-21T07:52:29.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.939+0000 7f59bb699740 1 -- start start 2024-09-21T07:52:29.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.939+0000 7f59bb699740 1 Processor -- start 2024-09-21T07:52:29.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.940+0000 7f59bb699740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5575862be3b0 0x5575866af130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:29.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.940+0000 7f59bb699740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5575862c6590 0x5575866d5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:29.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.940+0000 7f59bb699740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5575862e7270 0x5575866d5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:29.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.940+0000 7f59bb699740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557586597330 con 0x5575862c6590 2024-09-21T07:52:29.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.940+0000 7f59bb699740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5575865f4810 con 0x5575862e7270 2024-09-21T07:52:29.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.940+0000 7f59bb699740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x557586361620 con 0x5575862be3b0 2024-09-21T07:52:29.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.940+0000 7f59bb699740 10 client.0 _ll_register_callbacks cb 0x55758659c420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:52:29.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.941+0000 7f59b1200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5575862c6590 0x5575866d5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:29.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.941+0000 7f59b1200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5575862c6590 0x5575866d5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35744/0 (socket says 172.21.15.39:35744) 2024-09-21T07:52:29.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.941+0000 7f59b1200640 1 -- 172.21.15.39:0/1418345094 learned_addr learned my addr 172.21.15.39:0/1418345094 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:29.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.941+0000 7f59b1c00640 1 --2- 172.21.15.39:0/1418345094 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5575862be3b0 0x5575866af130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:29.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.941+0000 7f59b1200640 1 -- 172.21.15.39:0/1418345094 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5575862be3b0 msgr2=0x5575866af130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:52:29.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.941+0000 7f59b2600640 1 --2- 172.21.15.39:0/1418345094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5575862e7270 0x5575866d5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:29.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.941+0000 7f59b1200640 1 --2- 172.21.15.39:0/1418345094 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5575862be3b0 0x5575866af130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.941+0000 7f59b1200640 1 -- 172.21.15.39:0/1418345094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5575862e7270 msgr2=0x5575866d5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:52:29.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.941+0000 7f59b1200640 1 --2- 172.21.15.39:0/1418345094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5575862e7270 0x5575866d5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.941+0000 7f59b1200640 1 -- 172.21.15.39:0/1418345094 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557586352850 con 0x5575862c6590 2024-09-21T07:52:29.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.941+0000 7f59b1c00640 1 --2- 172.21.15.39:0/1418345094 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5575862be3b0 0x5575866af130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:52:29.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.941+0000 7f59b1200640 1 --2- 172.21.15.39:0/1418345094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5575862c6590 0x5575866d5550 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7f599800cdb0 tx=0x7f5998061600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d822fa4c603a507 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:29.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.941+0000 7f5992a00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:52:29.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.942+0000 7f5992a00640 1 -- 172.21.15.39:0/1418345094 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59980612f0 con 0x5575862c6590 2024-09-21T07:52:29.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.942+0000 7f5992a00640 1 -- 172.21.15.39:0/1418345094 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(247..247 src has 1..247) ==== 7605+0+0 (secure 0 0 0) 0x7f599805d7e0 con 0x5575862c6590 2024-09-21T07:52:29.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.942+0000 7f59bb699740 1 -- 172.21.15.39:0/1418345094 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59a4003db0 con 0x5575862c6590 2024-09-21T07:52:29.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.942+0000 7f5992a00640 1 -- 172.21.15.39:0/1418345094 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=281) ==== 24+0+0 (secure 0 0 0) 0x7f59980668c0 con 0x5575862c6590 2024-09-21T07:52:29.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.942+0000 7f59bb699740 10 client.11559 fetch_fsmap learned FSMap version 281 2024-09-21T07:52:29.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.942+0000 7f59bb699740 1 -- 172.21.15.39:0/1418345094 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=281,monmap=2+}) -- 0x5575863c96b0 con 0x5575862c6590 2024-09-21T07:52:29.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.942+0000 7f5992a00640 1 -- 172.21.15.39:0/1418345094 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f59a4003db0 con 0x5575862c6590 2024-09-21T07:52:29.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.942+0000 7f5992a00640 1 -- 172.21.15.39:0/1418345094 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 281) ==== 3542+0+0 (secure 0 0 0) 0x7f599805d9e0 con 0x5575862c6590 2024-09-21T07:52:29.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.942+0000 7f59bb699740 10 client.11559 fetch_fsmap finished waiting for FSMap version 281 2024-09-21T07:52:29.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.942+0000 7f59bb699740 10 client.11559 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T07:52:29.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.942+0000 7f59bb699740 20 client.11559 populate_metadata read hostname 'smithi039' 2024-09-21T07:52:29.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.942+0000 7f59bb699740 1 --2- 172.21.15.39:0/1418345094 >> [v2:172.21.15.148:6834/3381818843,v1:172.21.15.148:6836/3381818843] conn(0x5575862a5e40 0x5575866e78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:29.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.942+0000 7f59bb699740 4 client.11559 mds_command: new command op to 9967 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:52:29.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.942+0000 7f59bb699740 1 -- 172.21.15.39:0/1418345094 --> [v2:172.21.15.148:6834/3381818843,v1:172.21.15.148:6836/3381818843] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557586603520 con 0x5575862a5e40 2024-09-21T07:52:29.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.943+0000 7f59b1c00640 1 --2- 172.21.15.39:0/1418345094 >> [v2:172.21.15.148:6834/3381818843,v1:172.21.15.148:6836/3381818843] conn(0x5575862a5e40 0x5575866e78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:29.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.944+0000 7f59b1c00640 1 --2- 172.21.15.39:0/1418345094 >> [v2:172.21.15.148:6834/3381818843,v1:172.21.15.148:6836/3381818843] conn(0x5575862a5e40 0x5575866e78b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae1a9cbdf1e948b3 server_cookie=a073fd01bc02e10 in_seq=0 out_seq=0 2024-09-21T07:52:29.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.944+0000 7f5992a00640 10 client.11559 ms_handle_connect on v2:172.21.15.148:6834/3381818843 2024-09-21T07:52:29.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.945+0000 7f5992a00640 1 -- 172.21.15.39:0/1418345094 <== mds.0 v2:172.21.15.148:6834/3381818843 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557586603520 con 0x5575862a5e40 2024-09-21T07:52:29.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.945+0000 7f5992a00640 10 client.11559 handle_command_reply: tid=0 2024-09-21T07:52:29.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.945+0000 7f59bb699740 1 client.11559 shutdown 2024-09-21T07:52:29.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.946+0000 7f59bb699740 1 -- 172.21.15.39:0/1418345094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5575862c6590 msgr2=0x5575866d5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:29.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.946+0000 7f59bb699740 1 --2- 172.21.15.39:0/1418345094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5575862c6590 0x5575866d5550 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7f599800cdb0 tx=0x7f5998061600 comp rx=0 tx=0).stop 2024-09-21T07:52:29.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.946+0000 7f59bb699740 1 -- 172.21.15.39:0/1418345094 shutdown_connections 2024-09-21T07:52:29.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.946+0000 7f59bb699740 1 --2- 172.21.15.39:0/1418345094 >> [v2:172.21.15.148:6834/3381818843,v1:172.21.15.148:6836/3381818843] conn(0x5575862a5e40 0x5575866e78b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.946+0000 7f59bb699740 1 --2- 172.21.15.39:0/1418345094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5575862e7270 0x5575866d5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.946+0000 7f59bb699740 1 --2- 172.21.15.39:0/1418345094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5575862c6590 0x5575866d5550 unknown :-1 s=CLOSED pgs=2529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.946+0000 7f5992a00640 0 client.11559 ms_handle_reset on v2:172.21.15.148:6834/3381818843 2024-09-21T07:52:29.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.946+0000 7f59bb699740 1 --2- 172.21.15.39:0/1418345094 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5575862be3b0 0x5575866af130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.946+0000 7f59bb699740 1 -- 172.21.15.39:0/1418345094 >> 172.21.15.39:0/1418345094 conn(0x5575862db3c0 msgr2=0x5575866b4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:29.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.946+0000 7f59bb699740 1 -- 172.21.15.39:0/1418345094 shutdown_connections 2024-09-21T07:52:29.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.946+0000 7f59bb699740 1 -- 172.21.15.39:0/1418345094 wait complete. 2024-09-21T07:52:29.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.946+0000 7f59bb699740 20 client.11559 trim_cache size 0 max 16384 2024-09-21T07:52:29.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.965+0000 7f59bb699740 1 Processor -- start 2024-09-21T07:52:29.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.965+0000 7f59bb699740 1 -- start start 2024-09-21T07:52:29.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.967+0000 7f59bb699740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5575862be3b0 0x5575866e8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:29.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.967+0000 7f59bb699740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5575862c6590 0x5575866b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:29.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.967+0000 7f59bb699740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5575862e7270 0x5575866b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:29.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.967+0000 7f59bb699740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f59ac1fff30 con 0x5575862c6590 2024-09-21T07:52:29.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.967+0000 7f59bb699740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f59ac200600 con 0x5575862be3b0 2024-09-21T07:52:29.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.967+0000 7f59bb699740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f59ac0af040 con 0x5575862e7270 2024-09-21T07:52:29.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.967+0000 7f59b1200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5575862c6590 0x5575866b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:29.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.967+0000 7f59b1200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5575862c6590 0x5575866b48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35756/0 (socket says 172.21.15.39:35756) 2024-09-21T07:52:29.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.967+0000 7f59b1200640 1 -- 172.21.15.39:0/3154221247 learned_addr learned my addr 172.21.15.39:0/3154221247 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:29.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.967+0000 7f59b2600640 1 --2- 172.21.15.39:0/3154221247 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5575862e7270 0x5575866b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:29.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.967+0000 7f59b1c00640 1 --2- 172.21.15.39:0/3154221247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5575862be3b0 0x5575866e8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:29.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.967+0000 7f59b1200640 1 -- 172.21.15.39:0/3154221247 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5575862e7270 msgr2=0x5575866b4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:52:29.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.967+0000 7f59b1200640 1 --2- 172.21.15.39:0/3154221247 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5575862e7270 0x5575866b4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.967+0000 7f59b1200640 1 -- 172.21.15.39:0/3154221247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5575862be3b0 msgr2=0x5575866e8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:52:29.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.967+0000 7f59b1200640 1 --2- 172.21.15.39:0/3154221247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5575862be3b0 0x5575866e8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.967+0000 7f59b1200640 1 -- 172.21.15.39:0/3154221247 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55758668d850 con 0x5575862c6590 2024-09-21T07:52:29.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.968+0000 7f59b1200640 1 --2- 172.21.15.39:0/3154221247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5575862c6590 0x5575866b48a0 secure :-1 s=READY pgs=2530 cs=0 l=1 rev1=1 crypto rx=0x7f599805f100 tx=0x7f5998061d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c2012982acadf17 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:29.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.968+0000 7f599d600640 1 -- 172.21.15.39:0/3154221247 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59980610f0 con 0x5575862c6590 2024-09-21T07:52:29.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.968+0000 7f599d600640 1 -- 172.21.15.39:0/3154221247 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f599805d7e0 con 0x5575862c6590 2024-09-21T07:52:29.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.968+0000 7f599d600640 1 -- 172.21.15.39:0/3154221247 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5998060b60 con 0x5575862c6590 2024-09-21T07:52:29.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.969+0000 7f59bb699740 1 -- 172.21.15.39:0/3154221247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5575862c6590 msgr2=0x5575866b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:29.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.969+0000 7f59bb699740 1 --2- 172.21.15.39:0/3154221247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5575862c6590 0x5575866b48a0 secure :-1 s=READY pgs=2530 cs=0 l=1 rev1=1 crypto rx=0x7f599805f100 tx=0x7f5998061d30 comp rx=0 tx=0).stop 2024-09-21T07:52:29.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.969+0000 7f59bb699740 1 -- 172.21.15.39:0/3154221247 shutdown_connections 2024-09-21T07:52:29.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.969+0000 7f59bb699740 1 --2- 172.21.15.39:0/3154221247 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5575862e7270 0x5575866b4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.969+0000 7f59bb699740 1 --2- 172.21.15.39:0/3154221247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5575862c6590 0x5575866b48a0 unknown :-1 s=CLOSED pgs=2530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.969+0000 7f59bb699740 1 --2- 172.21.15.39:0/3154221247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5575862be3b0 0x5575866e8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.969+0000 7f59bb699740 1 -- 172.21.15.39:0/3154221247 >> 172.21.15.39:0/3154221247 conn(0x5575862db3c0 msgr2=0x55758668d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:29.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.969+0000 7f59bb699740 1 -- 172.21.15.39:0/3154221247 shutdown_connections 2024-09-21T07:52:29.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.969+0000 7f59bb699740 1 -- 172.21.15.39:0/3154221247 wait complete. 2024-09-21T07:52:29.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.970+0000 7f59bb699740 1 -- start start 2024-09-21T07:52:29.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.970+0000 7f59bb699740 1 Processor -- start 2024-09-21T07:52:29.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.971+0000 7f59bb699740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5575862be3b0 0x55758673c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:29.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.971+0000 7f59bb699740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5575862e7270 0x5575866ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:29.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.971+0000 7f59bb699740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55758673f5b0 0x5575866b7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:29.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.971+0000 7f59bb699740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5575866e9990 con 0x5575862e7270 2024-09-21T07:52:29.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.971+0000 7f59bb699740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5575866af800 con 0x55758673f5b0 2024-09-21T07:52:29.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.971+0000 7f59bb699740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x5575866ea000 con 0x5575862be3b0 2024-09-21T07:52:29.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.971+0000 7f59bb699740 10 client.0 _ll_register_callbacks cb 0x7f5998061c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:52:29.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.972+0000 7f59b1200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5575862e7270 0x5575866ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:29.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.972+0000 7f59b1200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5575862e7270 0x5575866ad8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35772/0 (socket says 172.21.15.39:35772) 2024-09-21T07:52:29.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.972+0000 7f59b2600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55758673f5b0 0x5575866b7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:29.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.972+0000 7f59b1200640 1 -- 172.21.15.39:0/1338494527 learned_addr learned my addr 172.21.15.39:0/1338494527 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:29.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.972+0000 7f59b1c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5575862be3b0 0x55758673c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:29.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.972+0000 7f59b1200640 1 -- 172.21.15.39:0/1338494527 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5575862be3b0 msgr2=0x55758673c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:52:29.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.972+0000 7f59b1200640 1 --2- 172.21.15.39:0/1338494527 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5575862be3b0 0x55758673c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.972+0000 7f59b1200640 1 -- 172.21.15.39:0/1338494527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55758673f5b0 msgr2=0x5575866b7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:52:29.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.972+0000 7f59b1200640 1 --2- 172.21.15.39:0/1338494527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55758673f5b0 0x5575866b7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.972+0000 7f59b1200640 1 -- 172.21.15.39:0/1338494527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55758675adc0 con 0x5575862e7270 2024-09-21T07:52:29.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.972+0000 7f59b1200640 1 --2- 172.21.15.39:0/1338494527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5575862e7270 0x5575866ad8a0 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7f599800cdb0 tx=0x7f5998098590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f631e80eb4309002 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:29.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.972+0000 7f5992a00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:52:29.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.972+0000 7f5992a00640 1 -- 172.21.15.39:0/1338494527 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5998097060 con 0x5575862e7270 2024-09-21T07:52:29.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.973+0000 7f59bb699740 1 -- 172.21.15.39:0/1338494527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f599805d7e0 con 0x5575862e7270 2024-09-21T07:52:29.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.973+0000 7f5992a00640 1 -- 172.21.15.39:0/1338494527 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(247..247 src has 1..247) ==== 7605+0+0 (secure 0 0 0) 0x7f5998067a50 con 0x5575862e7270 2024-09-21T07:52:29.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.973+0000 7f5992a00640 1 -- 172.21.15.39:0/1338494527 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=281) ==== 24+0+0 (secure 0 0 0) 0x7f59980610f0 con 0x5575862e7270 2024-09-21T07:52:29.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.973+0000 7f59bb699740 10 client.11565 fetch_fsmap learned FSMap version 281 2024-09-21T07:52:29.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.973+0000 7f59bb699740 1 -- 172.21.15.39:0/1338494527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=281,monmap=2+}) -- 0x55758675adc0 con 0x5575862e7270 2024-09-21T07:52:29.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.973+0000 7f5992a00640 1 -- 172.21.15.39:0/1338494527 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f599805d7e0 con 0x5575862e7270 2024-09-21T07:52:29.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.973+0000 7f5992a00640 1 -- 172.21.15.39:0/1338494527 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 281) ==== 3542+0+0 (secure 0 0 0) 0x7f599808f020 con 0x5575862e7270 2024-09-21T07:52:29.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.973+0000 7f59bb699740 10 client.11565 fetch_fsmap finished waiting for FSMap version 281 2024-09-21T07:52:29.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.973+0000 7f59bb699740 10 client.11565 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T07:52:29.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.973+0000 7f59bb699740 20 client.11565 populate_metadata read hostname 'smithi039' 2024-09-21T07:52:29.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.973+0000 7f59bb699740 1 --2- 172.21.15.39:0/1338494527 >> [v2:172.21.15.148:6834/3381818843,v1:172.21.15.148:6836/3381818843] conn(0x5575866f8e20 0x5575866df040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:29.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.973+0000 7f59bb699740 4 client.11565 mds_command: new command op to 9967 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:52:29.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.973+0000 7f59bb699740 1 -- 172.21.15.39:0/1338494527 --> [v2:172.21.15.148:6834/3381818843,v1:172.21.15.148:6836/3381818843] -- command(tid 0: {"prefix": "status"}) -- 0x5575865fefd0 con 0x5575866f8e20 2024-09-21T07:52:29.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.974+0000 7f59b1c00640 1 --2- 172.21.15.39:0/1338494527 >> [v2:172.21.15.148:6834/3381818843,v1:172.21.15.148:6836/3381818843] conn(0x5575866f8e20 0x5575866df040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:29.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.975+0000 7f59b1c00640 1 --2- 172.21.15.39:0/1338494527 >> [v2:172.21.15.148:6834/3381818843,v1:172.21.15.148:6836/3381818843] conn(0x5575866f8e20 0x5575866df040 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78f2d0abb6fa1a07 server_cookie=a48cb2698ffa44f9 in_seq=0 out_seq=0 2024-09-21T07:52:29.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.975+0000 7f5992a00640 10 client.11565 ms_handle_connect on v2:172.21.15.148:6834/3381818843 2024-09-21T07:52:29.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.975+0000 7f5992a00640 1 -- 172.21.15.39:0/1338494527 <== mds.0 v2:172.21.15.148:6834/3381818843 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x5575865fefd0 con 0x5575866f8e20 2024-09-21T07:52:29.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.975+0000 7f5992a00640 10 client.11565 handle_command_reply: tid=0 2024-09-21T07:52:29.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.975+0000 7f59bb699740 1 client.11565 shutdown 2024-09-21T07:52:29.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.975+0000 7f59bb699740 1 -- 172.21.15.39:0/1338494527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5575862e7270 msgr2=0x5575866ad8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:29.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.975+0000 7f59bb699740 1 --2- 172.21.15.39:0/1338494527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5575862e7270 0x5575866ad8a0 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7f599800cdb0 tx=0x7f5998098590 comp rx=0 tx=0).stop 2024-09-21T07:52:29.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.975+0000 7f59bb699740 1 -- 172.21.15.39:0/1338494527 shutdown_connections 2024-09-21T07:52:29.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.975+0000 7f59bb699740 1 --2- 172.21.15.39:0/1338494527 >> [v2:172.21.15.148:6834/3381818843,v1:172.21.15.148:6836/3381818843] conn(0x5575866f8e20 0x5575866df040 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.976+0000 7f59bb699740 1 --2- 172.21.15.39:0/1338494527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55758673f5b0 0x5575866b7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.976+0000 7f59bb699740 1 --2- 172.21.15.39:0/1338494527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5575862e7270 0x5575866ad8a0 unknown :-1 s=CLOSED pgs=2531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.976+0000 7f59bb699740 1 --2- 172.21.15.39:0/1338494527 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5575862be3b0 0x55758673c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.976+0000 7f5992a00640 0 client.11565 ms_handle_reset on v2:172.21.15.148:6834/3381818843 2024-09-21T07:52:29.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.976+0000 7f59bb699740 1 -- 172.21.15.39:0/1338494527 >> 172.21.15.39:0/1338494527 conn(0x5575862db3c0 msgr2=0x5575866e8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:29.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.976+0000 7f59bb699740 1 -- 172.21.15.39:0/1338494527 shutdown_connections 2024-09-21T07:52:29.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.976+0000 7f59bb699740 1 -- 172.21.15.39:0/1338494527 wait complete. 2024-09-21T07:52:29.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.976+0000 7f59bb699740 20 client.11565 trim_cache size 0 max 16384 2024-09-21T07:52:29.987 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:52:29.988 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:52:29.988 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-09-21T07:52:29.988 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 9967, 2024-09-21T07:52:29.988 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:52:29.988 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:52:29.988 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:52:29.989 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 118.140233432, 2024-09-21T07:52:29.989 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 281, 2024-09-21T07:52:29.989 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 247, 2024-09-21T07:52:29.989 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 247, 2024-09-21T07:52:29.989 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 124.988799556 2024-09-21T07:52:29.989 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:52:29.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.976+0000 7f59b3000640 1 -- 172.21.15.39:0/383578351 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f598c078b90 msgr2=0x7f598c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:29.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.976+0000 7f59b3000640 1 --2- 172.21.15.39:0/383578351 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f598c078b90 0x7f598c07b0a0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f59a0006fd0 tx=0x7f59a0009060 comp rx=0 tx=0).stop 2024-09-21T07:52:29.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.976+0000 7f59b3000640 1 -- 172.21.15.39:0/383578351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59ac0cc390 msgr2=0x7f59ac099490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:29.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.976+0000 7f59b3000640 1 --2- 172.21.15.39:0/383578351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59ac0cc390 0x7f59ac099490 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 crypto rx=0x7f599800c3f0 tx=0x7f599800c8b0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.976+0000 7f59b3000640 1 -- 172.21.15.39:0/383578351 shutdown_connections 2024-09-21T07:52:29.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.976+0000 7f59b3000640 1 --2- 172.21.15.39:0/383578351 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f598c078b90 0x7f598c07b0a0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.976+0000 7f59b3000640 1 --2- 172.21.15.39:0/383578351 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59ac0ce310 0x7f59ac073e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.976+0000 7f59b3000640 1 --2- 172.21.15.39:0/383578351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59ac0cc390 0x7f59ac099490 unknown :-1 s=CLOSED pgs=2527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.976+0000 7f59b3000640 1 --2- 172.21.15.39:0/383578351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59ac0cb8a0 0x7f59ac14e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.976+0000 7f59b3000640 1 -- 172.21.15.39:0/383578351 >> 172.21.15.39:0/383578351 conn(0x7f59ac0b2510 msgr2=0x7f59ac0b0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:29.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.977+0000 7f59b3000640 1 -- 172.21.15.39:0/383578351 shutdown_connections 2024-09-21T07:52:29.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:29.977+0000 7f59b3000640 1 -- 172.21.15.39:0/383578351 wait complete. 2024-09-21T07:52:30.002 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-21T07:52:30.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.232+0000 7fe716000640 1 -- 172.21.15.39:0/3278068812 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7100b2ca0 msgr2=0x7fe7100b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:30.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.232+0000 7fe716000640 1 --2- 172.21.15.39:0/3278068812 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7100b2ca0 0x7fe7100b4dc0 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x7fe71008da30 tx=0x7fe70802aa10 comp rx=0 tx=0).stop 2024-09-21T07:52:30.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.232+0000 7fe716000640 1 -- 172.21.15.39:0/3278068812 shutdown_connections 2024-09-21T07:52:30.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.232+0000 7fe716000640 1 --2- 172.21.15.39:0/3278068812 >> v2:172.21.15.148:3300/0 conn(0x7fe7100d4d10 0x7fe7100b4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.232+0000 7fe716000640 1 --2- 172.21.15.39:0/3278068812 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7100b2ca0 0x7fe7100b4dc0 unknown :-1 s=CLOSED pgs=2532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.232+0000 7fe716000640 1 -- 172.21.15.39:0/3278068812 >> 172.21.15.39:0/3278068812 conn(0x7fe71008bfb0 msgr2=0x7fe71009d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:30.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.232+0000 7fe716000640 1 -- 172.21.15.39:0/3278068812 shutdown_connections 2024-09-21T07:52:30.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.232+0000 7fe716000640 1 -- 172.21.15.39:0/3278068812 wait complete. 2024-09-21T07:52:30.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.234+0000 7fe716000640 1 Processor -- start 2024-09-21T07:52:30.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.234+0000 7fe716000640 1 -- start start 2024-09-21T07:52:30.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.236+0000 7fe716000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7100b2ca0 0x7fe7100a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.236+0000 7fe716000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7100d19f0 0x7fe7101e86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.236+0000 7fe716000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7101e8c30 0x7fe7101f8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.236+0000 7fe716000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe7100b5480 con 0x7fe7100d19f0 2024-09-21T07:52:30.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.236+0000 7fe716000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe7100b5300 con 0x7fe7101e8c30 2024-09-21T07:52:30.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.236+0000 7fe716000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe7100b5600 con 0x7fe7100b2ca0 2024-09-21T07:52:30.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.236+0000 7fe707e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7100d19f0 0x7fe7101e86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.236+0000 7fe707e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7100d19f0 0x7fe7101e86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35790/0 (socket says 172.21.15.39:35790) 2024-09-21T07:52:30.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.236+0000 7fe707e00640 1 -- 172.21.15.39:0/2915293155 learned_addr learned my addr 172.21.15.39:0/2915293155 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:30.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.236+0000 7fe714c00640 1 --2- 172.21.15.39:0/2915293155 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7100b2ca0 0x7fe7100a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.236+0000 7fe715600640 1 --2- 172.21.15.39:0/2915293155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7101e8c30 0x7fe7101f8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.237+0000 7fe707e00640 1 -- 172.21.15.39:0/2915293155 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7100b2ca0 msgr2=0x7fe7100a1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:30.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.237+0000 7fe707e00640 1 --2- 172.21.15.39:0/2915293155 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7100b2ca0 0x7fe7100a1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.237+0000 7fe707e00640 1 -- 172.21.15.39:0/2915293155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7101e8c30 msgr2=0x7fe7101f8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:30.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.237+0000 7fe707e00640 1 --2- 172.21.15.39:0/2915293155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7101e8c30 0x7fe7101f8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.237+0000 7fe707e00640 1 -- 172.21.15.39:0/2915293155 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7101e97d0 con 0x7fe7100d19f0 2024-09-21T07:52:30.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.237+0000 7fe715600640 1 --2- 172.21.15.39:0/2915293155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7101e8c30 0x7fe7101f8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:52:30.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.237+0000 7fe707e00640 1 --2- 172.21.15.39:0/2915293155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7100d19f0 0x7fe7101e86f0 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7fe6f800d6d0 tx=0x7fe6f800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:30.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.237+0000 7fe70d600640 1 -- 172.21.15.39:0/2915293155 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8017070 con 0x7fe7100d19f0 2024-09-21T07:52:30.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.237+0000 7fe70d600640 1 -- 172.21.15.39:0/2915293155 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6f8005330 con 0x7fe7100d19f0 2024-09-21T07:52:30.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.237+0000 7fe70d600640 1 -- 172.21.15.39:0/2915293155 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8009820 con 0x7fe7100d19f0 2024-09-21T07:52:30.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.237+0000 7fe716000640 1 -- 172.21.15.39:0/2915293155 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7101e9040 con 0x7fe7100d19f0 2024-09-21T07:52:30.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.237+0000 7fe716000640 1 -- 172.21.15.39:0/2915293155 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7101e9d20 con 0x7fe7100d19f0 2024-09-21T07:52:30.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.240+0000 7fe70d600640 1 -- 172.21.15.39:0/2915293155 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe6f80099c0 con 0x7fe7100d19f0 2024-09-21T07:52:30.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.241+0000 7fe70d600640 1 --2- 172.21.15.39:0/2915293155 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe6ec078bc0 0x7fe6ec07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.241+0000 7fe70d600640 1 -- 172.21.15.39:0/2915293155 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(247..247 src has 1..247) ==== 7605+0+0 (secure 0 0 0) 0x7fe6f809b630 con 0x7fe7100d19f0 2024-09-21T07:52:30.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.241+0000 7fe71e826740 1 Processor -- start 2024-09-21T07:52:30.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.241+0000 7fe71e826740 1 -- start start 2024-09-21T07:52:30.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.242+0000 7fe714c00640 1 --2- 172.21.15.39:0/2915293155 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe6ec078bc0 0x7fe6ec07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.242+0000 7fe714c00640 1 --2- 172.21.15.39:0/2915293155 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe6ec078bc0 0x7fe6ec07b0d0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fe7000098c0 tx=0x7fe700009060 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:30.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.243+0000 7fe71e826740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558adf087270 0x558adf457b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.243+0000 7fe71e826740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558adf05e3b0 0x558adf4580a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.243+0000 7fe71e826740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558adf066590 0x558adf428150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.243+0000 7fe71e826740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558adf3b1430 con 0x558adf087270 2024-09-21T07:52:30.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.243+0000 7fe71e826740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558adf3a15d0 con 0x558adf066590 2024-09-21T07:52:30.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.243+0000 7fe71e826740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x558adf390a00 con 0x558adf05e3b0 2024-09-21T07:52:30.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.243+0000 7fe714c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558adf087270 0x558adf457b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.243+0000 7fe714c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558adf087270 0x558adf457b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35802/0 (socket says 172.21.15.39:35802) 2024-09-21T07:52:30.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.243+0000 7fe714c00640 1 -- 172.21.15.39:0/2827804749 learned_addr learned my addr 172.21.15.39:0/2827804749 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:30.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.243+0000 7fe707e00640 1 --2- 172.21.15.39:0/2827804749 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558adf05e3b0 0x558adf4580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.243+0000 7fe715600640 1 --2- 172.21.15.39:0/2827804749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558adf066590 0x558adf428150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.244+0000 7fe714c00640 1 -- 172.21.15.39:0/2827804749 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558adf05e3b0 msgr2=0x558adf4580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:52:30.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.244+0000 7fe714c00640 1 --2- 172.21.15.39:0/2827804749 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558adf05e3b0 0x558adf4580a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.244+0000 7fe714c00640 1 -- 172.21.15.39:0/2827804749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558adf066590 msgr2=0x558adf428150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:52:30.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.244+0000 7fe714c00640 1 --2- 172.21.15.39:0/2827804749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558adf066590 0x558adf428150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.244+0000 7fe714c00640 1 -- 172.21.15.39:0/2827804749 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558adf359520 con 0x558adf087270 2024-09-21T07:52:30.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.244+0000 7fe714c00640 1 --2- 172.21.15.39:0/2827804749 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558adf087270 0x558adf457b60 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7fe70000eb70 tx=0x7fe700011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=345a482d5508dea7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:30.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.244+0000 7fe705200640 1 -- 172.21.15.39:0/2827804749 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70001b040 con 0x558adf087270 2024-09-21T07:52:30.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.244+0000 7fe705200640 1 -- 172.21.15.39:0/2827804749 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe700024070 con 0x558adf087270 2024-09-21T07:52:30.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.244+0000 7fe705200640 1 -- 172.21.15.39:0/2827804749 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe700015d10 con 0x558adf087270 2024-09-21T07:52:30.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.245+0000 7fe71e826740 1 -- 172.21.15.39:0/2827804749 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558adf087270 msgr2=0x558adf457b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:30.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.245+0000 7fe71e826740 1 --2- 172.21.15.39:0/2827804749 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558adf087270 0x558adf457b60 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7fe70000eb70 tx=0x7fe700011000 comp rx=0 tx=0).stop 2024-09-21T07:52:30.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.245+0000 7fe71e826740 1 -- 172.21.15.39:0/2827804749 shutdown_connections 2024-09-21T07:52:30.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.245+0000 7fe71e826740 1 --2- 172.21.15.39:0/2827804749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558adf066590 0x558adf428150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.245+0000 7fe71e826740 1 --2- 172.21.15.39:0/2827804749 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558adf05e3b0 0x558adf4580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.245+0000 7fe71e826740 1 --2- 172.21.15.39:0/2827804749 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558adf087270 0x558adf457b60 unknown :-1 s=CLOSED pgs=2534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.245+0000 7fe71e826740 1 -- 172.21.15.39:0/2827804749 >> 172.21.15.39:0/2827804749 conn(0x558adf07b3c0 msgr2=0x558adf42db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:30.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.245+0000 7fe71e826740 1 -- 172.21.15.39:0/2827804749 shutdown_connections 2024-09-21T07:52:30.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.245+0000 7fe71e826740 1 -- 172.21.15.39:0/2827804749 wait complete. 2024-09-21T07:52:30.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.246+0000 7fe71e826740 1 -- start start 2024-09-21T07:52:30.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.246+0000 7fe71e826740 1 Processor -- start 2024-09-21T07:52:30.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.248+0000 7fe71e826740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558adf05e3b0 0x558adf44e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.248+0000 7fe71e826740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558adf066590 0x558adf4504c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.248+0000 7fe71e826740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558adf087270 0x558adf450a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.248+0000 7fe71e826740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558adf337330 con 0x558adf05e3b0 2024-09-21T07:52:30.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.248+0000 7fe71e826740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558adf394810 con 0x558adf087270 2024-09-21T07:52:30.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.248+0000 7fe71e826740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x558adf101620 con 0x558adf066590 2024-09-21T07:52:30.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.248+0000 7fe71e826740 10 client.0 _ll_register_callbacks cb 0x558adf33c420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:52:30.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.248+0000 7fe714c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558adf05e3b0 0x558adf44e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.248+0000 7fe714c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558adf05e3b0 0x558adf44e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35806/0 (socket says 172.21.15.39:35806) 2024-09-21T07:52:30.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.248+0000 7fe714c00640 1 -- 172.21.15.39:0/159416377 learned_addr learned my addr 172.21.15.39:0/159416377 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:30.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.249+0000 7fe707e00640 1 --2- 172.21.15.39:0/159416377 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558adf066590 0x558adf4504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.249+0000 7fe715600640 1 --2- 172.21.15.39:0/159416377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558adf087270 0x558adf450a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.249+0000 7fe714c00640 1 -- 172.21.15.39:0/159416377 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558adf066590 msgr2=0x558adf4504c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:52:30.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.249+0000 7fe714c00640 1 --2- 172.21.15.39:0/159416377 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558adf066590 0x558adf4504c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.249+0000 7fe714c00640 1 -- 172.21.15.39:0/159416377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558adf087270 msgr2=0x558adf450a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:52:30.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.249+0000 7fe714c00640 1 --2- 172.21.15.39:0/159416377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558adf087270 0x558adf450a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.249+0000 7fe714c00640 1 -- 172.21.15.39:0/159416377 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558adf0f2850 con 0x558adf05e3b0 2024-09-21T07:52:30.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.249+0000 7fe715600640 1 --2- 172.21.15.39:0/159416377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558adf087270 0x558adf450a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:52:30.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.249+0000 7fe714c00640 1 --2- 172.21.15.39:0/159416377 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558adf05e3b0 0x558adf44e4e0 secure :-1 s=READY pgs=2535 cs=0 l=1 rev1=1 crypto rx=0x7fe700015850 tx=0x7fe700016ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a8d8ad01af53bd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:30.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.249+0000 7fe6fd600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:52:30.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.249+0000 7fe6fd600640 1 -- 172.21.15.39:0/159416377 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe700004510 con 0x558adf05e3b0 2024-09-21T07:52:30.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.249+0000 7fe6fd600640 1 -- 172.21.15.39:0/159416377 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(247..247 src has 1..247) ==== 7605+0+0 (secure 0 0 0) 0x7fe700003640 con 0x558adf05e3b0 2024-09-21T07:52:30.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.249+0000 7fe71e826740 1 -- 172.21.15.39:0/159416377 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe700024070 con 0x558adf05e3b0 2024-09-21T07:52:30.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.250+0000 7fe6fd600640 1 -- 172.21.15.39:0/159416377 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=281) ==== 24+0+0 (secure 0 0 0) 0x7fe70003b050 con 0x558adf05e3b0 2024-09-21T07:52:30.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.250+0000 7fe71e826740 10 client.11577 fetch_fsmap learned FSMap version 281 2024-09-21T07:52:30.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.250+0000 7fe71e826740 1 -- 172.21.15.39:0/159416377 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=281,monmap=2+}) -- 0x558adf1696b0 con 0x558adf05e3b0 2024-09-21T07:52:30.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.250+0000 7fe6fd600640 1 -- 172.21.15.39:0/159416377 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe700024070 con 0x558adf05e3b0 2024-09-21T07:52:30.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.250+0000 7fe6fd600640 1 -- 172.21.15.39:0/159416377 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 281) ==== 3542+0+0 (secure 0 0 0) 0x7fe70003e020 con 0x558adf05e3b0 2024-09-21T07:52:30.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.250+0000 7fe71e826740 10 client.11577 fetch_fsmap finished waiting for FSMap version 281 2024-09-21T07:52:30.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.250+0000 7fe71e826740 10 client.11577 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T07:52:30.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.250+0000 7fe71e826740 20 client.11577 populate_metadata read hostname 'smithi039' 2024-09-21T07:52:30.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.250+0000 7fe71e826740 1 --2- 172.21.15.39:0/159416377 >> [v2:172.21.15.148:6835/597301351,v1:172.21.15.148:6837/597301351] conn(0x558adf045e40 0x558adf484c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.250+0000 7fe71e826740 4 client.11577 mds_command: new command op to 10195 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:52:30.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.250+0000 7fe71e826740 1 -- 172.21.15.39:0/159416377 --> [v2:172.21.15.148:6835/597301351,v1:172.21.15.148:6837/597301351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558adf3a3520 con 0x558adf045e40 2024-09-21T07:52:30.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.251+0000 7fe707e00640 1 --2- 172.21.15.39:0/159416377 >> [v2:172.21.15.148:6835/597301351,v1:172.21.15.148:6837/597301351] conn(0x558adf045e40 0x558adf484c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.251+0000 7fe707e00640 1 --2- 172.21.15.39:0/159416377 >> [v2:172.21.15.148:6835/597301351,v1:172.21.15.148:6837/597301351] conn(0x558adf045e40 0x558adf484c60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9873958c36f28809 server_cookie=e9db7c98ea38113e in_seq=0 out_seq=0 2024-09-21T07:52:30.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.251+0000 7fe6fd600640 10 client.11577 ms_handle_connect on v2:172.21.15.148:6835/597301351 2024-09-21T07:52:30.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.252+0000 7fe6fd600640 1 -- 172.21.15.39:0/159416377 <== mds.2 v2:172.21.15.148:6835/597301351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558adf3a3520 con 0x558adf045e40 2024-09-21T07:52:30.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.252+0000 7fe6fd600640 10 client.11577 handle_command_reply: tid=0 2024-09-21T07:52:30.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.253+0000 7fe71e826740 1 client.11577 shutdown 2024-09-21T07:52:30.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.253+0000 7fe71e826740 1 -- 172.21.15.39:0/159416377 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558adf05e3b0 msgr2=0x558adf44e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:30.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.253+0000 7fe71e826740 1 --2- 172.21.15.39:0/159416377 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558adf05e3b0 0x558adf44e4e0 secure :-1 s=READY pgs=2535 cs=0 l=1 rev1=1 crypto rx=0x7fe700015850 tx=0x7fe700016ee0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.253+0000 7fe71e826740 1 -- 172.21.15.39:0/159416377 shutdown_connections 2024-09-21T07:52:30.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.253+0000 7fe71e826740 1 --2- 172.21.15.39:0/159416377 >> [v2:172.21.15.148:6835/597301351,v1:172.21.15.148:6837/597301351] conn(0x558adf045e40 0x558adf484c60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.253+0000 7fe71e826740 1 --2- 172.21.15.39:0/159416377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558adf087270 0x558adf450a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.253+0000 7fe71e826740 1 --2- 172.21.15.39:0/159416377 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558adf066590 0x558adf4504c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.253+0000 7fe71e826740 1 --2- 172.21.15.39:0/159416377 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558adf05e3b0 0x558adf44e4e0 unknown :-1 s=CLOSED pgs=2535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.253+0000 7fe71e826740 1 -- 172.21.15.39:0/159416377 >> 172.21.15.39:0/159416377 conn(0x558adf07b3c0 msgr2=0x558adf44f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:30.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.253+0000 7fe6fd600640 0 client.11577 ms_handle_reset on v2:172.21.15.148:6835/597301351 2024-09-21T07:52:30.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.253+0000 7fe71e826740 1 -- 172.21.15.39:0/159416377 shutdown_connections 2024-09-21T07:52:30.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.253+0000 7fe71e826740 1 -- 172.21.15.39:0/159416377 wait complete. 2024-09-21T07:52:30.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.253+0000 7fe71e826740 20 client.11577 trim_cache size 0 max 16384 2024-09-21T07:52:30.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.272+0000 7fe71e826740 1 Processor -- start 2024-09-21T07:52:30.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.272+0000 7fe71e826740 1 -- start start 2024-09-21T07:52:30.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.274+0000 7fe71e826740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558adf05e3b0 0x558adf42d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.274+0000 7fe71e826740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558adf066590 0x558adf42df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.274+0000 7fe71e826740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558adf087270 0x558adf495120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.274+0000 7fe71e826740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe7102044a0 con 0x558adf087270 2024-09-21T07:52:30.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.274+0000 7fe71e826740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe710204b70 con 0x558adf05e3b0 2024-09-21T07:52:30.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.274+0000 7fe71e826740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe710203440 con 0x558adf066590 2024-09-21T07:52:30.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.274+0000 7fe715600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558adf087270 0x558adf495120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.274+0000 7fe715600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558adf087270 0x558adf495120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35822/0 (socket says 172.21.15.39:35822) 2024-09-21T07:52:30.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.274+0000 7fe715600640 1 -- 172.21.15.39:0/4246953299 learned_addr learned my addr 172.21.15.39:0/4246953299 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:30.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.274+0000 7fe714c00640 1 --2- 172.21.15.39:0/4246953299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558adf05e3b0 0x558adf42d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.274+0000 7fe707e00640 1 --2- 172.21.15.39:0/4246953299 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558adf066590 0x558adf42df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.274+0000 7fe715600640 1 -- 172.21.15.39:0/4246953299 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558adf066590 msgr2=0x558adf42df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:52:30.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.274+0000 7fe715600640 1 --2- 172.21.15.39:0/4246953299 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558adf066590 0x558adf42df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.274+0000 7fe715600640 1 -- 172.21.15.39:0/4246953299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558adf05e3b0 msgr2=0x558adf42d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:52:30.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.274+0000 7fe715600640 1 --2- 172.21.15.39:0/4246953299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558adf05e3b0 0x558adf42d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.275+0000 7fe715600640 1 -- 172.21.15.39:0/4246953299 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558adf456cc0 con 0x558adf087270 2024-09-21T07:52:30.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.275+0000 7fe714c00640 1 --2- 172.21.15.39:0/4246953299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558adf05e3b0 0x558adf42d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:52:30.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.275+0000 7fe707e00640 1 --2- 172.21.15.39:0/4246953299 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558adf066590 0x558adf42df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:52:30.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.275+0000 7fe715600640 1 --2- 172.21.15.39:0/4246953299 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558adf087270 0x558adf495120 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7fe70802afd0 tx=0x7fe708004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c92e609ee310fb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:30.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.275+0000 7fe6ffe00640 1 -- 172.21.15.39:0/4246953299 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe708033040 con 0x558adf087270 2024-09-21T07:52:30.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.275+0000 7fe6ffe00640 1 -- 172.21.15.39:0/4246953299 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe70802fdd0 con 0x558adf087270 2024-09-21T07:52:30.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.275+0000 7fe6ffe00640 1 -- 172.21.15.39:0/4246953299 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe708003b90 con 0x558adf087270 2024-09-21T07:52:30.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.276+0000 7fe71e826740 1 -- 172.21.15.39:0/4246953299 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558adf087270 msgr2=0x558adf495120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:30.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.276+0000 7fe71e826740 1 --2- 172.21.15.39:0/4246953299 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558adf087270 0x558adf495120 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7fe70802afd0 tx=0x7fe708004a40 comp rx=0 tx=0).stop 2024-09-21T07:52:30.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.276+0000 7fe71e826740 1 -- 172.21.15.39:0/4246953299 shutdown_connections 2024-09-21T07:52:30.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.276+0000 7fe71e826740 1 --2- 172.21.15.39:0/4246953299 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558adf087270 0x558adf495120 unknown :-1 s=CLOSED pgs=2536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.276+0000 7fe71e826740 1 --2- 172.21.15.39:0/4246953299 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558adf066590 0x558adf42df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.276+0000 7fe71e826740 1 --2- 172.21.15.39:0/4246953299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558adf05e3b0 0x558adf42d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.276+0000 7fe71e826740 1 -- 172.21.15.39:0/4246953299 >> 172.21.15.39:0/4246953299 conn(0x558adf07b3c0 msgr2=0x558adf4511b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:30.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.276+0000 7fe71e826740 1 -- 172.21.15.39:0/4246953299 shutdown_connections 2024-09-21T07:52:30.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.276+0000 7fe71e826740 1 -- 172.21.15.39:0/4246953299 wait complete. 2024-09-21T07:52:30.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.277+0000 7fe71e826740 1 -- start start 2024-09-21T07:52:30.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.277+0000 7fe71e826740 1 Processor -- start 2024-09-21T07:52:30.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.279+0000 7fe71e826740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558adf05e3b0 0x558adf4f1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.279+0000 7fe71e826740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558adf066590 0x558adf4f54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.279+0000 7fe71e826740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558adf087270 0x558adf4f59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.279+0000 7fe71e826740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558adf46cc60 con 0x558adf05e3b0 2024-09-21T07:52:30.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.279+0000 7fe71e826740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558adf469ba0 con 0x558adf066590 2024-09-21T07:52:30.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.279+0000 7fe71e826740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x558adf481f50 con 0x558adf087270 2024-09-21T07:52:30.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.279+0000 7fe71e826740 10 client.0 _ll_register_callbacks cb 0x7fe70000ea70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:52:30.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.279+0000 7fe714c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558adf05e3b0 0x558adf4f1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.279+0000 7fe714c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558adf05e3b0 0x558adf4f1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35832/0 (socket says 172.21.15.39:35832) 2024-09-21T07:52:30.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.279+0000 7fe714c00640 1 -- 172.21.15.39:0/1430844053 learned_addr learned my addr 172.21.15.39:0/1430844053 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:30.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.279+0000 7fe715600640 1 --2- 172.21.15.39:0/1430844053 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558adf087270 0x558adf4f59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.279+0000 7fe707e00640 1 --2- 172.21.15.39:0/1430844053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558adf066590 0x558adf4f54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.279+0000 7fe714c00640 1 -- 172.21.15.39:0/1430844053 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558adf087270 msgr2=0x558adf4f59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:52:30.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.279+0000 7fe714c00640 1 --2- 172.21.15.39:0/1430844053 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558adf087270 0x558adf4f59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.279+0000 7fe714c00640 1 -- 172.21.15.39:0/1430844053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558adf066590 msgr2=0x558adf4f54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:52:30.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.279+0000 7fe714c00640 1 --2- 172.21.15.39:0/1430844053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558adf066590 0x558adf4f54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.279+0000 7fe714c00640 1 -- 172.21.15.39:0/1430844053 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558adf456f70 con 0x558adf05e3b0 2024-09-21T07:52:30.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.280+0000 7fe714c00640 1 --2- 172.21.15.39:0/1430844053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558adf05e3b0 0x558adf4f1860 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 crypto rx=0x7fe700009780 tx=0x7fe700003a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=999c3ac6d93f3ba2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:30.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.280+0000 7fe6fd600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:52:30.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.280+0000 7fe6fd600640 1 -- 172.21.15.39:0/1430844053 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe700004510 con 0x558adf05e3b0 2024-09-21T07:52:30.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.280+0000 7fe6fd600640 1 -- 172.21.15.39:0/1430844053 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(247..247 src has 1..247) ==== 7605+0+0 (secure 0 0 0) 0x7fe700033410 con 0x558adf05e3b0 2024-09-21T07:52:30.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.280+0000 7fe71e826740 1 -- 172.21.15.39:0/1430844053 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe70802fdd0 con 0x558adf05e3b0 2024-09-21T07:52:30.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.280+0000 7fe6fd600640 1 -- 172.21.15.39:0/1430844053 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=281) ==== 24+0+0 (secure 0 0 0) 0x7fe70004a030 con 0x558adf05e3b0 2024-09-21T07:52:30.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.280+0000 7fe71e826740 10 client.11583 fetch_fsmap learned FSMap version 281 2024-09-21T07:52:30.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.280+0000 7fe71e826740 1 -- 172.21.15.39:0/1430844053 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=281,monmap=2+}) -- 0x558adf4733f0 con 0x558adf05e3b0 2024-09-21T07:52:30.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.280+0000 7fe6fd600640 1 -- 172.21.15.39:0/1430844053 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe70802fdd0 con 0x558adf05e3b0 2024-09-21T07:52:30.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.281+0000 7fe6fd600640 1 -- 172.21.15.39:0/1430844053 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 281) ==== 3542+0+0 (secure 0 0 0) 0x7fe70003e040 con 0x558adf05e3b0 2024-09-21T07:52:30.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.281+0000 7fe71e826740 10 client.11583 fetch_fsmap finished waiting for FSMap version 281 2024-09-21T07:52:30.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.281+0000 7fe71e826740 10 client.11583 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T07:52:30.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.281+0000 7fe71e826740 20 client.11583 populate_metadata read hostname 'smithi039' 2024-09-21T07:52:30.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.281+0000 7fe71e826740 1 --2- 172.21.15.39:0/1430844053 >> [v2:172.21.15.148:6835/597301351,v1:172.21.15.148:6837/597301351] conn(0x558adf451650 0x558adf4558d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.281+0000 7fe71e826740 4 client.11583 mds_command: new command op to 10195 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:52:30.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.281+0000 7fe71e826740 1 -- 172.21.15.39:0/1430844053 --> [v2:172.21.15.148:6835/597301351,v1:172.21.15.148:6837/597301351] -- command(tid 0: {"prefix": "status"}) -- 0x558adf39efd0 con 0x558adf451650 2024-09-21T07:52:30.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.281+0000 7fe707e00640 1 --2- 172.21.15.39:0/1430844053 >> [v2:172.21.15.148:6835/597301351,v1:172.21.15.148:6837/597301351] conn(0x558adf451650 0x558adf4558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.282+0000 7fe6fd600640 10 client.11583 ms_handle_connect on v2:172.21.15.148:6835/597301351 2024-09-21T07:52:30.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.282+0000 7fe707e00640 1 --2- 172.21.15.39:0/1430844053 >> [v2:172.21.15.148:6835/597301351,v1:172.21.15.148:6837/597301351] conn(0x558adf451650 0x558adf4558d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=643ee80aaaee219b server_cookie=68c8d1670d35f080 in_seq=0 out_seq=0 2024-09-21T07:52:30.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.282+0000 7fe6fd600640 1 -- 172.21.15.39:0/1430844053 <== mds.2 v2:172.21.15.148:6835/597301351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+335 (crc 0 0 0) 0x558adf39efd0 con 0x558adf451650 2024-09-21T07:52:30.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.282+0000 7fe6fd600640 10 client.11583 handle_command_reply: tid=0 2024-09-21T07:52:30.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.282+0000 7fe71e826740 1 client.11583 shutdown 2024-09-21T07:52:30.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.282+0000 7fe71e826740 1 -- 172.21.15.39:0/1430844053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558adf05e3b0 msgr2=0x558adf4f1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:30.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.282+0000 7fe71e826740 1 --2- 172.21.15.39:0/1430844053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558adf05e3b0 0x558adf4f1860 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 crypto rx=0x7fe700009780 tx=0x7fe700003a00 comp rx=0 tx=0).stop 2024-09-21T07:52:30.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.282+0000 7fe71e826740 1 -- 172.21.15.39:0/1430844053 shutdown_connections 2024-09-21T07:52:30.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.282+0000 7fe71e826740 1 --2- 172.21.15.39:0/1430844053 >> [v2:172.21.15.148:6835/597301351,v1:172.21.15.148:6837/597301351] conn(0x558adf451650 0x558adf4558d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.282+0000 7fe71e826740 1 --2- 172.21.15.39:0/1430844053 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558adf087270 0x558adf4f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.282+0000 7fe71e826740 1 --2- 172.21.15.39:0/1430844053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558adf066590 0x558adf4f54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.282+0000 7fe71e826740 1 --2- 172.21.15.39:0/1430844053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558adf05e3b0 0x558adf4f1860 unknown :-1 s=CLOSED pgs=2537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.282+0000 7fe71e826740 1 -- 172.21.15.39:0/1430844053 >> 172.21.15.39:0/1430844053 conn(0x558adf07b3c0 msgr2=0x558adf457910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:30.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.282+0000 7fe6fd600640 0 client.11583 ms_handle_reset on v2:172.21.15.148:6835/597301351 2024-09-21T07:52:30.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.282+0000 7fe71e826740 1 -- 172.21.15.39:0/1430844053 shutdown_connections 2024-09-21T07:52:30.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.282+0000 7fe71e826740 1 -- 172.21.15.39:0/1430844053 wait complete. 2024-09-21T07:52:30.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.283+0000 7fe71e826740 20 client.11583 trim_cache size 0 max 16384 2024-09-21T07:52:30.288 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:52:30.288 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:52:30.289 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 2, 2024-09-21T07:52:30.289 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 10195, 2024-09-21T07:52:30.289 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:52:30.289 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:52:30.289 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:52:30.289 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 53.761223278000003, 2024-09-21T07:52:30.289 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 281, 2024-09-21T07:52:30.289 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 247, 2024-09-21T07:52:30.289 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 247, 2024-09-21T07:52:30.289 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 89.139687863000006 2024-09-21T07:52:30.289 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:52:30.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.283+0000 7fe716000640 1 -- 172.21.15.39:0/2915293155 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe6ec078bc0 msgr2=0x7fe6ec07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:30.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.283+0000 7fe716000640 1 --2- 172.21.15.39:0/2915293155 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe6ec078bc0 0x7fe6ec07b0d0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fe7000098c0 tx=0x7fe700009060 comp rx=0 tx=0).stop 2024-09-21T07:52:30.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.283+0000 7fe716000640 1 -- 172.21.15.39:0/2915293155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7100d19f0 msgr2=0x7fe7101e86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:30.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.283+0000 7fe716000640 1 --2- 172.21.15.39:0/2915293155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7100d19f0 0x7fe7101e86f0 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7fe6f800d6d0 tx=0x7fe6f800db90 comp rx=0 tx=0).stop 2024-09-21T07:52:30.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.283+0000 7fe716000640 1 -- 172.21.15.39:0/2915293155 shutdown_connections 2024-09-21T07:52:30.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.283+0000 7fe716000640 1 --2- 172.21.15.39:0/2915293155 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe6ec078bc0 0x7fe6ec07b0d0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.283+0000 7fe716000640 1 --2- 172.21.15.39:0/2915293155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7101e8c30 0x7fe7101f8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.283+0000 7fe716000640 1 --2- 172.21.15.39:0/2915293155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7100d19f0 0x7fe7101e86f0 unknown :-1 s=CLOSED pgs=2533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.283+0000 7fe716000640 1 --2- 172.21.15.39:0/2915293155 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe7100b2ca0 0x7fe7100a1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.283+0000 7fe716000640 1 -- 172.21.15.39:0/2915293155 >> 172.21.15.39:0/2915293155 conn(0x7fe71008bfb0 msgr2=0x7fe7100d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:30.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.283+0000 7fe716000640 1 -- 172.21.15.39:0/2915293155 shutdown_connections 2024-09-21T07:52:30.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.283+0000 7fe716000640 1 -- 172.21.15.39:0/2915293155 wait complete. 2024-09-21T07:52:30.309 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T07:52:30.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.534+0000 7fa13bc00640 1 -- 172.21.15.39:0/3524019933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa134155a70 msgr2=0x7fa1340b39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:30.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.534+0000 7fa13bc00640 1 --2- 172.21.15.39:0/3524019933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa134155a70 0x7fa1340b39f0 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 crypto rx=0x7fa11c005fd0 tx=0x7fa11c02a5d0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.534+0000 7fa13bc00640 1 -- 172.21.15.39:0/3524019933 shutdown_connections 2024-09-21T07:52:30.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.534+0000 7fa13bc00640 1 --2- 172.21.15.39:0/3524019933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa134155a70 0x7fa1340b39f0 unknown :-1 s=CLOSED pgs=2538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.534+0000 7fa13bc00640 1 --2- 172.21.15.39:0/3524019933 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa13415f360 0x7fa13415bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.534+0000 7fa13bc00640 1 -- 172.21.15.39:0/3524019933 >> 172.21.15.39:0/3524019933 conn(0x7fa13406cfc0 msgr2=0x7fa13406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:30.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.534+0000 7fa13bc00640 1 -- 172.21.15.39:0/3524019933 shutdown_connections 2024-09-21T07:52:30.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.534+0000 7fa13bc00640 1 -- 172.21.15.39:0/3524019933 wait complete. 2024-09-21T07:52:30.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.536+0000 7fa13bc00640 1 Processor -- start 2024-09-21T07:52:30.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.537+0000 7fa13bc00640 1 -- start start 2024-09-21T07:52:30.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.538+0000 7fa13bc00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa1340b3f30 0x7fa13415b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.538+0000 7fa13bc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa134155a70 0x7fa1341527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.538+0000 7fa13bc00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa13415f360 0x7fa1341e94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.538+0000 7fa13bc00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa1340d0620 con 0x7fa134155a70 2024-09-21T07:52:30.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.538+0000 7fa13bc00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa134156150 con 0x7fa13415f360 2024-09-21T07:52:30.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.538+0000 7fa13bc00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa1340d07a0 con 0x7fa1340b3f30 2024-09-21T07:52:30.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.538+0000 7fa12be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa134155a70 0x7fa1341527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.538+0000 7fa12be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa134155a70 0x7fa1341527f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35852/0 (socket says 172.21.15.39:35852) 2024-09-21T07:52:30.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.538+0000 7fa12be00640 1 -- 172.21.15.39:0/2926059157 learned_addr learned my addr 172.21.15.39:0/2926059157 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:30.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.539+0000 7fa12be00640 1 -- 172.21.15.39:0/2926059157 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa1340b3f30 msgr2=0x7fa13415b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:30.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.539+0000 7fa12be00640 1 --2- 172.21.15.39:0/2926059157 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa1340b3f30 0x7fa13415b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.539+0000 7fa12be00640 1 -- 172.21.15.39:0/2926059157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa13415f360 msgr2=0x7fa1341e94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:30.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.539+0000 7fa12be00640 1 --2- 172.21.15.39:0/2926059157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa13415f360 0x7fa1341e94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.539+0000 7fa12be00640 1 -- 172.21.15.39:0/2926059157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1340aff50 con 0x7fa134155a70 2024-09-21T07:52:30.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.539+0000 7fa12be00640 1 --2- 172.21.15.39:0/2926059157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa134155a70 0x7fa1341527f0 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7fa11c000c00 tx=0x7fa11c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:30.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.540+0000 7fa131600640 1 -- 172.21.15.39:0/2926059157 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c03e030 con 0x7fa134155a70 2024-09-21T07:52:30.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.540+0000 7fa131600640 1 -- 172.21.15.39:0/2926059157 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa11c003ca0 con 0x7fa134155a70 2024-09-21T07:52:30.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.540+0000 7fa131600640 1 -- 172.21.15.39:0/2926059157 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c035040 con 0x7fa134155a70 2024-09-21T07:52:30.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.540+0000 7fa13bc00640 1 -- 172.21.15.39:0/2926059157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1340b0210 con 0x7fa134155a70 2024-09-21T07:52:30.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.540+0000 7fa13bc00640 1 -- 172.21.15.39:0/2926059157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1340b07d0 con 0x7fa134155a70 2024-09-21T07:52:30.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.542+0000 7fa131600640 1 -- 172.21.15.39:0/2926059157 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa11c041030 con 0x7fa134155a70 2024-09-21T07:52:30.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.543+0000 7fa131600640 1 --2- 172.21.15.39:0/2926059157 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa110078b90 0x7fa11007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.543+0000 7fa131600640 1 -- 172.21.15.39:0/2926059157 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(247..247 src has 1..247) ==== 7605+0+0 (secure 0 0 0) 0x7fa11c0bbad0 con 0x7fa134155a70 2024-09-21T07:52:30.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.544+0000 7fa142c94740 1 Processor -- start 2024-09-21T07:52:30.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.544+0000 7fa142c94740 1 -- start start 2024-09-21T07:52:30.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.544+0000 7fa139200640 1 --2- 172.21.15.39:0/2926059157 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa110078b90 0x7fa11007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.545+0000 7fa139200640 1 --2- 172.21.15.39:0/2926059157 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa110078b90 0x7fa11007b0a0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fa12400b980 tx=0x7fa124005db0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:30.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.546+0000 7fa142c94740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56439069c270 0x564390a6c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.546+0000 7fa142c94740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643906733b0 0x564390a6ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.546+0000 7fa142c94740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56439067b590 0x564390a3cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.546+0000 7fa142c94740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5643909c6430 con 0x56439067b590 2024-09-21T07:52:30.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.546+0000 7fa142c94740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5643909b65d0 con 0x5643906733b0 2024-09-21T07:52:30.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.546+0000 7fa142c94740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x5643909a5a00 con 0x56439069c270 2024-09-21T07:52:30.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.546+0000 7fa139c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56439067b590 0x564390a3cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.546+0000 7fa139c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56439067b590 0x564390a3cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35866/0 (socket says 172.21.15.39:35866) 2024-09-21T07:52:30.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.546+0000 7fa139c00640 1 -- 172.21.15.39:0/1771780018 learned_addr learned my addr 172.21.15.39:0/1771780018 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:30.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.546+0000 7fa139200640 1 --2- 172.21.15.39:0/1771780018 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56439069c270 0x564390a6c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.546+0000 7fa12be00640 1 --2- 172.21.15.39:0/1771780018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643906733b0 0x564390a6ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.547+0000 7fa139c00640 1 -- 172.21.15.39:0/1771780018 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56439069c270 msgr2=0x564390a6c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:52:30.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.547+0000 7fa139c00640 1 --2- 172.21.15.39:0/1771780018 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56439069c270 0x564390a6c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.547+0000 7fa139c00640 1 -- 172.21.15.39:0/1771780018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643906733b0 msgr2=0x564390a6ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:52:30.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.547+0000 7fa139c00640 1 --2- 172.21.15.39:0/1771780018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643906733b0 0x564390a6ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.547+0000 7fa139c00640 1 -- 172.21.15.39:0/1771780018 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56439096e520 con 0x56439067b590 2024-09-21T07:52:30.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.547+0000 7fa139200640 1 --2- 172.21.15.39:0/1771780018 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56439069c270 0x564390a6c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:52:30.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.547+0000 7fa139c00640 1 --2- 172.21.15.39:0/1771780018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56439067b590 0x564390a3cf60 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7fa12c001c80 tx=0x7fa12c002140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db71ada21ab9ca0f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:30.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.547+0000 7fa129600640 1 -- 172.21.15.39:0/1771780018 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12c017070 con 0x56439067b590 2024-09-21T07:52:30.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.547+0000 7fa129600640 1 -- 172.21.15.39:0/1771780018 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa12c00cd60 con 0x56439067b590 2024-09-21T07:52:30.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.547+0000 7fa129600640 1 -- 172.21.15.39:0/1771780018 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12c003b90 con 0x56439067b590 2024-09-21T07:52:30.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.548+0000 7fa142c94740 1 -- 172.21.15.39:0/1771780018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56439067b590 msgr2=0x564390a3cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:30.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.548+0000 7fa142c94740 1 --2- 172.21.15.39:0/1771780018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56439067b590 0x564390a3cf60 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7fa12c001c80 tx=0x7fa12c002140 comp rx=0 tx=0).stop 2024-09-21T07:52:30.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.548+0000 7fa142c94740 1 -- 172.21.15.39:0/1771780018 shutdown_connections 2024-09-21T07:52:30.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.548+0000 7fa142c94740 1 --2- 172.21.15.39:0/1771780018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56439067b590 0x564390a3cf60 unknown :-1 s=CLOSED pgs=2540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.548+0000 7fa142c94740 1 --2- 172.21.15.39:0/1771780018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643906733b0 0x564390a6ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.548+0000 7fa142c94740 1 --2- 172.21.15.39:0/1771780018 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56439069c270 0x564390a6c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.548+0000 7fa142c94740 1 -- 172.21.15.39:0/1771780018 >> 172.21.15.39:0/1771780018 conn(0x5643906903c0 msgr2=0x564390a42970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:30.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.548+0000 7fa142c94740 1 -- 172.21.15.39:0/1771780018 shutdown_connections 2024-09-21T07:52:30.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.548+0000 7fa142c94740 1 -- 172.21.15.39:0/1771780018 wait complete. 2024-09-21T07:52:30.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.549+0000 7fa142c94740 1 -- start start 2024-09-21T07:52:30.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.549+0000 7fa142c94740 1 Processor -- start 2024-09-21T07:52:30.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.551+0000 7fa142c94740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643906733b0 0x564390a81680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.551+0000 7fa142c94740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56439067b590 0x564390a82870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.551+0000 7fa142c94740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56439069c270 0x564390a82db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.551+0000 7fa142c94740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56439094c330 con 0x56439069c270 2024-09-21T07:52:30.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.551+0000 7fa142c94740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5643909a9810 con 0x5643906733b0 2024-09-21T07:52:30.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.551+0000 7fa142c94740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x564390716620 con 0x56439067b590 2024-09-21T07:52:30.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.551+0000 7fa142c94740 10 client.0 _ll_register_callbacks cb 0x564390951420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:52:30.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.551+0000 7fa139c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56439069c270 0x564390a82db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.551+0000 7fa139c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56439069c270 0x564390a82db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35870/0 (socket says 172.21.15.39:35870) 2024-09-21T07:52:30.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.551+0000 7fa139c00640 1 -- 172.21.15.39:0/151128122 learned_addr learned my addr 172.21.15.39:0/151128122 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:30.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.552+0000 7fa139200640 1 --2- 172.21.15.39:0/151128122 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643906733b0 0x564390a81680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.552+0000 7fa12be00640 1 --2- 172.21.15.39:0/151128122 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56439067b590 0x564390a82870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.552+0000 7fa139c00640 1 -- 172.21.15.39:0/151128122 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56439067b590 msgr2=0x564390a82870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:52:30.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.552+0000 7fa139c00640 1 --2- 172.21.15.39:0/151128122 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56439067b590 0x564390a82870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.552+0000 7fa139c00640 1 -- 172.21.15.39:0/151128122 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643906733b0 msgr2=0x564390a81680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:52:30.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.552+0000 7fa139c00640 1 --2- 172.21.15.39:0/151128122 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643906733b0 0x564390a81680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.552+0000 7fa139c00640 1 -- 172.21.15.39:0/151128122 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564390707850 con 0x56439069c270 2024-09-21T07:52:30.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.552+0000 7fa139c00640 1 --2- 172.21.15.39:0/151128122 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56439069c270 0x564390a82db0 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 crypto rx=0x7fa12c002230 tx=0x7fa12c00cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a9ec3cd903e29e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:30.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.552+0000 7fa122000640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:52:30.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.552+0000 7fa122000640 1 -- 172.21.15.39:0/151128122 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12c021070 con 0x56439069c270 2024-09-21T07:52:30.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.552+0000 7fa142c94740 1 -- 172.21.15.39:0/151128122 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa12c00cd60 con 0x56439069c270 2024-09-21T07:52:30.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.552+0000 7fa122000640 1 -- 172.21.15.39:0/151128122 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(247..247 src has 1..247) ==== 7605+0+0 (secure 0 0 0) 0x7fa12c003b70 con 0x56439069c270 2024-09-21T07:52:30.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.553+0000 7fa122000640 1 -- 172.21.15.39:0/151128122 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=281) ==== 24+0+0 (secure 0 0 0) 0x7fa12c017040 con 0x56439069c270 2024-09-21T07:52:30.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.553+0000 7fa142c94740 10 client.11595 fetch_fsmap learned FSMap version 281 2024-09-21T07:52:30.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.553+0000 7fa142c94740 1 -- 172.21.15.39:0/151128122 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=281,monmap=2+}) -- 0x56439077e6b0 con 0x56439069c270 2024-09-21T07:52:30.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.553+0000 7fa122000640 1 -- 172.21.15.39:0/151128122 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa12c00cd60 con 0x56439069c270 2024-09-21T07:52:30.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.553+0000 7fa122000640 1 -- 172.21.15.39:0/151128122 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 281) ==== 3542+0+0 (secure 0 0 0) 0x7fa12c005070 con 0x56439069c270 2024-09-21T07:52:30.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.553+0000 7fa142c94740 10 client.11595 fetch_fsmap finished waiting for FSMap version 281 2024-09-21T07:52:30.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.553+0000 7fa142c94740 10 client.11595 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:52:30.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.553+0000 7fa142c94740 20 client.11595 populate_metadata read hostname 'smithi039' 2024-09-21T07:52:30.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.553+0000 7fa142c94740 1 --2- 172.21.15.39:0/151128122 >> [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] conn(0x56439065ae40 0x564390a65df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.553+0000 7fa142c94740 4 client.11595 mds_command: new command op to 11052 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:52:30.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.553+0000 7fa142c94740 1 -- 172.21.15.39:0/151128122 --> [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643909b8520 con 0x56439065ae40 2024-09-21T07:52:30.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.553+0000 7fa139200640 1 --2- 172.21.15.39:0/151128122 >> [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] conn(0x56439065ae40 0x564390a65df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.554+0000 7fa139200640 1 --2- 172.21.15.39:0/151128122 >> [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] conn(0x56439065ae40 0x564390a65df0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fe3e8ebb81230ba0 server_cookie=befaeb1f1adbe6b2 in_seq=0 out_seq=0 2024-09-21T07:52:30.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.554+0000 7fa122000640 10 client.11595 ms_handle_connect on v2:172.21.15.39:6833/485194245 2024-09-21T07:52:30.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.555+0000 7fa122000640 1 -- 172.21.15.39:0/151128122 <== mds.1 v2:172.21.15.39:6833/485194245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5643909b8520 con 0x56439065ae40 2024-09-21T07:52:30.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.555+0000 7fa122000640 10 client.11595 handle_command_reply: tid=0 2024-09-21T07:52:30.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.555+0000 7fa142c94740 1 client.11595 shutdown 2024-09-21T07:52:30.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.555+0000 7fa142c94740 1 -- 172.21.15.39:0/151128122 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56439069c270 msgr2=0x564390a82db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:30.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.555+0000 7fa142c94740 1 --2- 172.21.15.39:0/151128122 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56439069c270 0x564390a82db0 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 crypto rx=0x7fa12c002230 tx=0x7fa12c00cad0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.555+0000 7fa142c94740 1 -- 172.21.15.39:0/151128122 shutdown_connections 2024-09-21T07:52:30.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.555+0000 7fa142c94740 1 --2- 172.21.15.39:0/151128122 >> [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] conn(0x56439065ae40 0x564390a65df0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.555+0000 7fa142c94740 1 --2- 172.21.15.39:0/151128122 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56439069c270 0x564390a82db0 unknown :-1 s=CLOSED pgs=2541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.555+0000 7fa142c94740 1 --2- 172.21.15.39:0/151128122 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56439067b590 0x564390a82870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.555+0000 7fa122000640 0 client.11595 ms_handle_reset on v2:172.21.15.39:6833/485194245 2024-09-21T07:52:30.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.555+0000 7fa142c94740 1 --2- 172.21.15.39:0/151128122 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643906733b0 0x564390a81680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.555+0000 7fa142c94740 1 -- 172.21.15.39:0/151128122 >> 172.21.15.39:0/151128122 conn(0x5643906903c0 msgr2=0x564390a8a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:30.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.555+0000 7fa142c94740 1 -- 172.21.15.39:0/151128122 shutdown_connections 2024-09-21T07:52:30.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.555+0000 7fa142c94740 1 -- 172.21.15.39:0/151128122 wait complete. 2024-09-21T07:52:30.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.556+0000 7fa142c94740 20 client.11595 trim_cache size 0 max 16384 2024-09-21T07:52:30.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.575+0000 7fa142c94740 1 Processor -- start 2024-09-21T07:52:30.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.575+0000 7fa142c94740 1 -- start start 2024-09-21T07:52:30.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.576+0000 7fa142c94740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5643906733b0 0x564390a65df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.576+0000 7fa142c94740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56439067b590 0x564390a429e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.576+0000 7fa142c94740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56439069c270 0x564390a66f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.576+0000 7fa142c94740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa134205260 con 0x5643906733b0 2024-09-21T07:52:30.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.576+0000 7fa142c94740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa1342044c0 con 0x56439067b590 2024-09-21T07:52:30.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.576+0000 7fa142c94740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa134204b90 con 0x56439069c270 2024-09-21T07:52:30.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.576+0000 7fa139200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5643906733b0 0x564390a65df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.577+0000 7fa139200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5643906733b0 0x564390a65df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35886/0 (socket says 172.21.15.39:35886) 2024-09-21T07:52:30.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.577+0000 7fa139200640 1 -- 172.21.15.39:0/1204598675 learned_addr learned my addr 172.21.15.39:0/1204598675 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:30.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.577+0000 7fa139c00640 1 --2- 172.21.15.39:0/1204598675 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56439069c270 0x564390a66f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.577+0000 7fa12be00640 1 --2- 172.21.15.39:0/1204598675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56439067b590 0x564390a429e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.577+0000 7fa139200640 1 -- 172.21.15.39:0/1204598675 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56439069c270 msgr2=0x564390a66f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:52:30.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.577+0000 7fa139200640 1 --2- 172.21.15.39:0/1204598675 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56439069c270 0x564390a66f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.577+0000 7fa139200640 1 -- 172.21.15.39:0/1204598675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56439067b590 msgr2=0x564390a429e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:52:30.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.577+0000 7fa139200640 1 --2- 172.21.15.39:0/1204598675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56439067b590 0x564390a429e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.577+0000 7fa139200640 1 -- 172.21.15.39:0/1204598675 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564390a43650 con 0x5643906733b0 2024-09-21T07:52:30.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.577+0000 7fa139200640 1 --2- 172.21.15.39:0/1204598675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5643906733b0 0x564390a65df0 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x564390a9ab90 tx=0x7fa124014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a12d7a5ce03d74 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:30.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.577+0000 7fa128c00640 1 -- 172.21.15.39:0/1204598675 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa124019040 con 0x5643906733b0 2024-09-21T07:52:30.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.577+0000 7fa128c00640 1 -- 172.21.15.39:0/1204598675 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa12401a040 con 0x5643906733b0 2024-09-21T07:52:30.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.577+0000 7fa128c00640 1 -- 172.21.15.39:0/1204598675 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa124003df0 con 0x5643906733b0 2024-09-21T07:52:30.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.578+0000 7fa142c94740 1 -- 172.21.15.39:0/1204598675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5643906733b0 msgr2=0x564390a65df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:30.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.578+0000 7fa142c94740 1 --2- 172.21.15.39:0/1204598675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5643906733b0 0x564390a65df0 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x564390a9ab90 tx=0x7fa124014000 comp rx=0 tx=0).stop 2024-09-21T07:52:30.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.578+0000 7fa142c94740 1 -- 172.21.15.39:0/1204598675 shutdown_connections 2024-09-21T07:52:30.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.578+0000 7fa142c94740 1 --2- 172.21.15.39:0/1204598675 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56439069c270 0x564390a66f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.578+0000 7fa142c94740 1 --2- 172.21.15.39:0/1204598675 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56439067b590 0x564390a429e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.578+0000 7fa142c94740 1 --2- 172.21.15.39:0/1204598675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5643906733b0 0x564390a65df0 unknown :-1 s=CLOSED pgs=2542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.578+0000 7fa142c94740 1 -- 172.21.15.39:0/1204598675 >> 172.21.15.39:0/1204598675 conn(0x5643906903c0 msgr2=0x564390a88a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:30.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.578+0000 7fa142c94740 1 -- 172.21.15.39:0/1204598675 shutdown_connections 2024-09-21T07:52:30.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.578+0000 7fa142c94740 1 -- 172.21.15.39:0/1204598675 wait complete. 2024-09-21T07:52:30.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.579+0000 7fa142c94740 1 -- start start 2024-09-21T07:52:30.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.579+0000 7fa142c94740 1 Processor -- start 2024-09-21T07:52:30.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.581+0000 7fa142c94740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643906733b0 0x564390b0c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.581+0000 7fa142c94740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56439067b590 0x564390b07740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.581+0000 7fa142c94740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56439069c270 0x564390aa9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.581+0000 7fa142c94740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564390a66d60 con 0x56439067b590 2024-09-21T07:52:30.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.581+0000 7fa142c94740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa13415a9f0 con 0x5643906733b0 2024-09-21T07:52:30.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.581+0000 7fa142c94740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x564390a9ac50 con 0x56439069c270 2024-09-21T07:52:30.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.581+0000 7fa142c94740 10 client.0 _ll_register_callbacks cb 0x7fa124005f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:52:30.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.581+0000 7fa12be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56439067b590 0x564390b07740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.581+0000 7fa12be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56439067b590 0x564390b07740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35900/0 (socket says 172.21.15.39:35900) 2024-09-21T07:52:30.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.581+0000 7fa12be00640 1 -- 172.21.15.39:0/3018187732 learned_addr learned my addr 172.21.15.39:0/3018187732 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:30.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.581+0000 7fa139200640 1 --2- 172.21.15.39:0/3018187732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643906733b0 0x564390b0c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.581+0000 7fa139c00640 1 --2- 172.21.15.39:0/3018187732 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56439069c270 0x564390aa9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.582+0000 7fa12be00640 1 -- 172.21.15.39:0/3018187732 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56439069c270 msgr2=0x564390aa9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:52:30.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.582+0000 7fa12be00640 1 --2- 172.21.15.39:0/3018187732 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56439069c270 0x564390aa9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.582+0000 7fa12be00640 1 -- 172.21.15.39:0/3018187732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643906733b0 msgr2=0x564390b0c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:52:30.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.582+0000 7fa12be00640 1 --2- 172.21.15.39:0/3018187732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643906733b0 0x564390b0c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.582+0000 7fa12be00640 1 -- 172.21.15.39:0/3018187732 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564390a87f30 con 0x56439067b590 2024-09-21T07:52:30.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.582+0000 7fa12be00640 1 --2- 172.21.15.39:0/3018187732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56439067b590 0x564390b07740 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7fa11c083020 tx=0x7fa11c07f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf03d59a2d83f55 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:30.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.582+0000 7fa122000640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:52:30.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.582+0000 7fa122000640 1 -- 172.21.15.39:0/3018187732 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c048040 con 0x56439067b590 2024-09-21T07:52:30.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.582+0000 7fa122000640 1 -- 172.21.15.39:0/3018187732 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(247..247 src has 1..247) ==== 7605+0+0 (secure 0 0 0) 0x7fa11c0822f0 con 0x56439067b590 2024-09-21T07:52:30.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.582+0000 7fa142c94740 1 -- 172.21.15.39:0/3018187732 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa12401a040 con 0x56439067b590 2024-09-21T07:52:30.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.582+0000 7fa122000640 1 -- 172.21.15.39:0/3018187732 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=281) ==== 24+0+0 (secure 0 0 0) 0x7fa11c08ae60 con 0x56439067b590 2024-09-21T07:52:30.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.582+0000 7fa142c94740 10 client.11601 fetch_fsmap learned FSMap version 281 2024-09-21T07:52:30.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.582+0000 7fa142c94740 1 -- 172.21.15.39:0/3018187732 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=281,monmap=2+}) -- 0x564390a68660 con 0x56439067b590 2024-09-21T07:52:30.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.583+0000 7fa122000640 1 -- 172.21.15.39:0/3018187732 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa12401a040 con 0x56439067b590 2024-09-21T07:52:30.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.583+0000 7fa122000640 1 -- 172.21.15.39:0/3018187732 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 281) ==== 3542+0+0 (secure 0 0 0) 0x7fa11c09d0b0 con 0x56439067b590 2024-09-21T07:52:30.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.583+0000 7fa142c94740 10 client.11601 fetch_fsmap finished waiting for FSMap version 281 2024-09-21T07:52:30.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.583+0000 7fa142c94740 10 client.11601 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:52:30.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.583+0000 7fa142c94740 20 client.11601 populate_metadata read hostname 'smithi039' 2024-09-21T07:52:30.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.583+0000 7fa142c94740 1 --2- 172.21.15.39:0/3018187732 >> [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] conn(0x564390a99a40 0x564390a95f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.583+0000 7fa142c94740 4 client.11601 mds_command: new command op to 11052 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:52:30.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.583+0000 7fa142c94740 1 -- 172.21.15.39:0/3018187732 --> [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] -- command(tid 0: {"prefix": "status"}) -- 0x5643909b3fd0 con 0x564390a99a40 2024-09-21T07:52:30.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.583+0000 7fa139200640 1 --2- 172.21.15.39:0/3018187732 >> [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] conn(0x564390a99a40 0x564390a95f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.584+0000 7fa139200640 1 --2- 172.21.15.39:0/3018187732 >> [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] conn(0x564390a99a40 0x564390a95f20 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ffc5a81a142bffa8 server_cookie=2f5be8133853825e in_seq=0 out_seq=0 2024-09-21T07:52:30.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.584+0000 7fa122000640 10 client.11601 ms_handle_connect on v2:172.21.15.39:6833/485194245 2024-09-21T07:52:30.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.584+0000 7fa122000640 1 -- 172.21.15.39:0/3018187732 <== mds.1 v2:172.21.15.39:6833/485194245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x5643909b3fd0 con 0x564390a99a40 2024-09-21T07:52:30.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.584+0000 7fa122000640 10 client.11601 handle_command_reply: tid=0 2024-09-21T07:52:30.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.584+0000 7fa142c94740 1 client.11601 shutdown 2024-09-21T07:52:30.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.584+0000 7fa142c94740 1 -- 172.21.15.39:0/3018187732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56439067b590 msgr2=0x564390b07740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:30.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.584+0000 7fa142c94740 1 --2- 172.21.15.39:0/3018187732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56439067b590 0x564390b07740 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7fa11c083020 tx=0x7fa11c07f580 comp rx=0 tx=0).stop 2024-09-21T07:52:30.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.584+0000 7fa142c94740 1 -- 172.21.15.39:0/3018187732 shutdown_connections 2024-09-21T07:52:30.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.584+0000 7fa142c94740 1 --2- 172.21.15.39:0/3018187732 >> [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] conn(0x564390a99a40 0x564390a95f20 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.584+0000 7fa142c94740 1 --2- 172.21.15.39:0/3018187732 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56439069c270 0x564390aa9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.584+0000 7fa142c94740 1 --2- 172.21.15.39:0/3018187732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56439067b590 0x564390b07740 unknown :-1 s=CLOSED pgs=2543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.584+0000 7fa142c94740 1 --2- 172.21.15.39:0/3018187732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643906733b0 0x564390b0c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.584+0000 7fa122000640 0 client.11601 ms_handle_reset on v2:172.21.15.39:6833/485194245 2024-09-21T07:52:30.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.584+0000 7fa142c94740 1 -- 172.21.15.39:0/3018187732 >> 172.21.15.39:0/3018187732 conn(0x5643906903c0 msgr2=0x564390aab0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:30.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.584+0000 7fa142c94740 1 -- 172.21.15.39:0/3018187732 shutdown_connections 2024-09-21T07:52:30.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.584+0000 7fa142c94740 1 -- 172.21.15.39:0/3018187732 wait complete. 2024-09-21T07:52:30.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.585+0000 7fa142c94740 20 client.11601 trim_cache size 0 max 16384 2024-09-21T07:52:30.589 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:52:30.590 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:52:30.590 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 1, 2024-09-21T07:52:30.590 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 11052, 2024-09-21T07:52:30.590 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:52:30.590 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:52:30.590 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:52:30.590 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 118.199217684, 2024-09-21T07:52:30.590 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 281, 2024-09-21T07:52:30.590 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 247, 2024-09-21T07:52:30.590 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 247, 2024-09-21T07:52:30.590 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 118.61148635799999 2024-09-21T07:52:30.590 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:52:30.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.585+0000 7fa13bc00640 1 -- 172.21.15.39:0/2926059157 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa110078b90 msgr2=0x7fa11007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:30.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.585+0000 7fa13bc00640 1 --2- 172.21.15.39:0/2926059157 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa110078b90 0x7fa11007b0a0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fa12400b980 tx=0x7fa124005db0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.585+0000 7fa13bc00640 1 -- 172.21.15.39:0/2926059157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa134155a70 msgr2=0x7fa1341527f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:30.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.585+0000 7fa13bc00640 1 --2- 172.21.15.39:0/2926059157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa134155a70 0x7fa1341527f0 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7fa11c000c00 tx=0x7fa11c0038a0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.585+0000 7fa13bc00640 1 -- 172.21.15.39:0/2926059157 shutdown_connections 2024-09-21T07:52:30.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.585+0000 7fa13bc00640 1 --2- 172.21.15.39:0/2926059157 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa110078b90 0x7fa11007b0a0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.585+0000 7fa13bc00640 1 --2- 172.21.15.39:0/2926059157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa13415f360 0x7fa1341e94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.585+0000 7fa13bc00640 1 --2- 172.21.15.39:0/2926059157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa134155a70 0x7fa1341527f0 unknown :-1 s=CLOSED pgs=2539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.585+0000 7fa13bc00640 1 --2- 172.21.15.39:0/2926059157 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa1340b3f30 0x7fa13415b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.585+0000 7fa13bc00640 1 -- 172.21.15.39:0/2926059157 >> 172.21.15.39:0/2926059157 conn(0x7fa13406cfc0 msgr2=0x7fa1340cc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:30.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.585+0000 7fa13bc00640 1 -- 172.21.15.39:0/2926059157 shutdown_connections 2024-09-21T07:52:30.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.585+0000 7fa13bc00640 1 -- 172.21.15.39:0/2926059157 wait complete. 2024-09-21T07:52:30.609 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:52:30.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.834+0000 7f0d3b200640 1 -- 172.21.15.39:0/2940551520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d3c0d0880 msgr2=0x7f0d3c0d0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:30.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.834+0000 7f0d3b200640 1 --2- 172.21.15.39:0/2940551520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d3c0d0880 0x7f0d3c0d0c90 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c0746f0 tx=0x7f0d3402a980 comp rx=0 tx=0).stop 2024-09-21T07:52:30.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.834+0000 7f0d3b200640 1 -- 172.21.15.39:0/2940551520 shutdown_connections 2024-09-21T07:52:30.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.834+0000 7f0d3b200640 1 --2- 172.21.15.39:0/2940551520 >> v2:172.21.15.148:3300/0 conn(0x7f0d3c0b1c90 0x7f0d3c0b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.834+0000 7f0d3b200640 1 --2- 172.21.15.39:0/2940551520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d3c0d0880 0x7f0d3c0d0c90 unknown :-1 s=CLOSED pgs=2544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.834+0000 7f0d3b200640 1 -- 172.21.15.39:0/2940551520 >> 172.21.15.39:0/2940551520 conn(0x7f0d3c0cee40 msgr2=0x7f0d3c0cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:30.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.835+0000 7f0d3b200640 1 -- 172.21.15.39:0/2940551520 shutdown_connections 2024-09-21T07:52:30.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.835+0000 7f0d3b200640 1 -- 172.21.15.39:0/2940551520 wait complete. 2024-09-21T07:52:30.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.837+0000 7f0d3b200640 1 Processor -- start 2024-09-21T07:52:30.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.837+0000 7f0d3b200640 1 -- start start 2024-09-21T07:52:30.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.838+0000 7f0d3b200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d3c0b1740 0x7f0d3c09cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.838+0000 7f0d3b200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d3c0b1c90 0x7f0d3c06ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.838+0000 7f0d3b200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d3c0d0880 0x7f0d3c06f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.838+0000 7f0d3b200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c0d1350 con 0x7f0d3c0b1740 2024-09-21T07:52:30.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.838+0000 7f0d3b200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c0d11d0 con 0x7f0d3c0b1c90 2024-09-21T07:52:30.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.838+0000 7f0d3b200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f0d3c0cced0 con 0x7f0d3c0d0880 2024-09-21T07:52:30.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.839+0000 7f0d39e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d3c0b1740 0x7f0d3c09cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.839+0000 7f0d39e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d3c0b1740 0x7f0d3c09cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35906/0 (socket says 172.21.15.39:35906) 2024-09-21T07:52:30.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.839+0000 7f0d39e00640 1 -- 172.21.15.39:0/3079231122 learned_addr learned my addr 172.21.15.39:0/3079231122 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:30.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.839+0000 7f0d3a800640 1 --2- 172.21.15.39:0/3079231122 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d3c0d0880 0x7f0d3c06f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.839+0000 7f0d39400640 1 --2- 172.21.15.39:0/3079231122 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d3c0b1c90 0x7f0d3c06ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.839+0000 7f0d39e00640 1 -- 172.21.15.39:0/3079231122 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d3c0d0880 msgr2=0x7f0d3c06f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:30.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.839+0000 7f0d39e00640 1 --2- 172.21.15.39:0/3079231122 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d3c0d0880 0x7f0d3c06f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.839+0000 7f0d39e00640 1 -- 172.21.15.39:0/3079231122 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d3c0b1c90 msgr2=0x7f0d3c06ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:30.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.839+0000 7f0d39e00640 1 --2- 172.21.15.39:0/3079231122 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d3c0b1c90 0x7f0d3c06ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.839+0000 7f0d39e00640 1 -- 172.21.15.39:0/3079231122 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d3c09fcd0 con 0x7f0d3c0b1740 2024-09-21T07:52:30.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.839+0000 7f0d39e00640 1 --2- 172.21.15.39:0/3079231122 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d3c0b1740 0x7f0d3c09cd10 secure :-1 s=READY pgs=2545 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00b3e0 tx=0x7f0d2c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:30.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.840+0000 7f0d32000640 1 -- 172.21.15.39:0/3079231122 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c015020 con 0x7f0d3c0b1740 2024-09-21T07:52:30.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.840+0000 7f0d32000640 1 -- 172.21.15.39:0/3079231122 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d2c011940 con 0x7f0d3c0b1740 2024-09-21T07:52:30.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.840+0000 7f0d3b200640 1 -- 172.21.15.39:0/3079231122 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d3c09ffb0 con 0x7f0d3c0b1740 2024-09-21T07:52:30.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.840+0000 7f0d32000640 1 -- 172.21.15.39:0/3079231122 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c011dd0 con 0x7f0d3c0b1740 2024-09-21T07:52:30.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.840+0000 7f0d3b200640 1 -- 172.21.15.39:0/3079231122 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d3c09c510 con 0x7f0d3c0b1740 2024-09-21T07:52:30.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.841+0000 7f0d3b200640 1 -- 172.21.15.39:0/3079231122 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d3c0b20a0 con 0x7f0d3c0b1740 2024-09-21T07:52:30.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.842+0000 7f0d32000640 1 -- 172.21.15.39:0/3079231122 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0d2c018020 con 0x7f0d3c0b1740 2024-09-21T07:52:30.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.843+0000 7f0d32000640 1 --2- 172.21.15.39:0/3079231122 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0d1c078b90 0x7f0d1c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.843+0000 7f0d32000640 1 -- 172.21.15.39:0/3079231122 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(247..247 src has 1..247) ==== 7605+0+0 (secure 0 0 0) 0x7f0d2c09b330 con 0x7f0d3c0b1740 2024-09-21T07:52:30.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.844+0000 7f0d39400640 1 --2- 172.21.15.39:0/3079231122 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0d1c078b90 0x7f0d1c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.844+0000 7f0d39400640 1 --2- 172.21.15.39:0/3079231122 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0d1c078b90 0x7f0d1c07b0a0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f0d28005fd0 tx=0x7f0d28005d80 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:30.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:30.848+0000 7f0d32000640 1 -- 172.21.15.39:0/3079231122 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0d2c062f60 con 0x7f0d3c0b1740 2024-09-21T07:52:30.890 INFO:tasks.ceph.mds.b.smithi148.stderr:2024-09-21T07:52:30.889+0000 7fa0be800640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-21T07:52:31.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.086+0000 7f0d3b200640 1 -- 172.21.15.39:0/3079231122 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d3c0afdd0 con 0x7f0d3c0b1740 2024-09-21T07:52:31.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.090+0000 7f0d32000640 1 -- 172.21.15.39:0/3079231122 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 281 v281) ==== 95+0+5283 (secure 0 0 0) 0x7f0d2c062860 con 0x7f0d3c0b1740 2024-09-21T07:52:31.092 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:52:31.093 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":281,"btime":"2024-09-21T07:52:29:069596+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":278}],"filesystems":[{"mdsmap":{"epoch":281,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:52:29.069589+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":246,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":10195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10195":{"gid":10195,"name":"d","rank":2,"incarnation":277,"state":"up:active","state_seq":25,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[9967,10195,11052]},"id":13}]} 2024-09-21T07:52:31.093 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 281 2024-09-21T07:52:31.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.093+0000 7f0d3b200640 1 -- 172.21.15.39:0/3079231122 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0d1c078b90 msgr2=0x7f0d1c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:31.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.093+0000 7f0d3b200640 1 --2- 172.21.15.39:0/3079231122 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0d1c078b90 0x7f0d1c07b0a0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f0d28005fd0 tx=0x7f0d28005d80 comp rx=0 tx=0).stop 2024-09-21T07:52:31.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.093+0000 7f0d3b200640 1 -- 172.21.15.39:0/3079231122 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d3c0b1740 msgr2=0x7f0d3c09cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:31.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.093+0000 7f0d3b200640 1 --2- 172.21.15.39:0/3079231122 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d3c0b1740 0x7f0d3c09cd10 secure :-1 s=READY pgs=2545 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00b3e0 tx=0x7f0d2c00b8a0 comp rx=0 tx=0).stop 2024-09-21T07:52:31.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.093+0000 7f0d3b200640 1 -- 172.21.15.39:0/3079231122 shutdown_connections 2024-09-21T07:52:31.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.093+0000 7f0d3b200640 1 --2- 172.21.15.39:0/3079231122 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0d1c078b90 0x7f0d1c07b0a0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:31.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.093+0000 7f0d3b200640 1 --2- 172.21.15.39:0/3079231122 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d3c0d0880 0x7f0d3c06f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:31.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.093+0000 7f0d3b200640 1 --2- 172.21.15.39:0/3079231122 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d3c0b1c90 0x7f0d3c06ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:31.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.093+0000 7f0d3b200640 1 --2- 172.21.15.39:0/3079231122 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d3c0b1740 0x7f0d3c09cd10 unknown :-1 s=CLOSED pgs=2545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:31.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.093+0000 7f0d3b200640 1 -- 172.21.15.39:0/3079231122 >> 172.21.15.39:0/3079231122 conn(0x7f0d3c0cee40 msgr2=0x7f0d3c15f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:31.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.093+0000 7f0d3b200640 1 -- 172.21.15.39:0/3079231122 shutdown_connections 2024-09-21T07:52:31.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.093+0000 7f0d3b200640 1 -- 172.21.15.39:0/3079231122 wait complete. 2024-09-21T07:52:31.117 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json dump snaps 2024-09-21T07:52:31.367 INFO:teuthology.orchestra.run.smithi148.stdout:{"last_created":8,"last_destroyed":7,"snaps":[{"snapid":8,"ino":1099511627780,"stamp":"2024-09-21T07:51:12.091986+0000","name":"s1","metadata":{}}]} 2024-09-21T07:52:31.368 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 8, "last_destroyed": 7, "snaps": [ { "ino": 1099511627780, "metadata": {}, "name": "s1", "snapid": 8, "stamp": "2024-09-21T07:51:12.091986+0000" } ] } 2024-09-21T07:52:31.368 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:52:31.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.593+0000 7f285fe00640 1 -- 172.21.15.39:0/847498740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28600b1c90 msgr2=0x7f28600b20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:31.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.593+0000 7f285fe00640 1 --2- 172.21.15.39:0/847498740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28600b1c90 0x7f28600b20a0 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7f2848005fd0 tx=0x7f284802a540 comp rx=0 tx=0).stop 2024-09-21T07:52:31.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.593+0000 7f285fe00640 1 -- 172.21.15.39:0/847498740 shutdown_connections 2024-09-21T07:52:31.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.593+0000 7f285fe00640 1 --2- 172.21.15.39:0/847498740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28600b1c90 0x7f28600b20a0 unknown :-1 s=CLOSED pgs=2547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:31.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.593+0000 7f285fe00640 1 --2- 172.21.15.39:0/847498740 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28600b1740 0x7f28600b77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:31.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.593+0000 7f285fe00640 1 -- 172.21.15.39:0/847498740 >> 172.21.15.39:0/847498740 conn(0x7f28600cee40 msgr2=0x7f28600cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:31.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.593+0000 7f285fe00640 1 -- 172.21.15.39:0/847498740 shutdown_connections 2024-09-21T07:52:31.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.593+0000 7f285fe00640 1 -- 172.21.15.39:0/847498740 wait complete. 2024-09-21T07:52:31.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.595+0000 7f285fe00640 1 Processor -- start 2024-09-21T07:52:31.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.595+0000 7f285fe00640 1 -- start start 2024-09-21T07:52:31.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.597+0000 7f285fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28600b1740 0x7f28601e9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:31.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.597+0000 7f285fe00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28600b1c90 0x7f286015eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:31.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.597+0000 7f285fe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28600d0880 0x7f28601f5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:31.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.597+0000 7f285fe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f28600cced0 con 0x7f28600b1740 2024-09-21T07:52:31.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.597+0000 7f285fe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f28600cd050 con 0x7f28600d0880 2024-09-21T07:52:31.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.597+0000 7f285fe00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f28600cd1d0 con 0x7f28600b1c90 2024-09-21T07:52:31.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.597+0000 7f285ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28600b1740 0x7f28601e9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:31.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.597+0000 7f285ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28600b1740 0x7f28601e9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35922/0 (socket says 172.21.15.39:35922) 2024-09-21T07:52:31.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.597+0000 7f285ea00640 1 -- 172.21.15.39:0/1497089869 learned_addr learned my addr 172.21.15.39:0/1497089869 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:31.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.597+0000 7f2857e00640 1 --2- 172.21.15.39:0/1497089869 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28600b1c90 0x7f286015eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:31.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.597+0000 7f285f400640 1 --2- 172.21.15.39:0/1497089869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28600d0880 0x7f28601f5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:31.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.598+0000 7f285ea00640 1 -- 172.21.15.39:0/1497089869 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28600b1c90 msgr2=0x7f286015eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:31.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.598+0000 7f285ea00640 1 --2- 172.21.15.39:0/1497089869 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28600b1c90 0x7f286015eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:31.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.598+0000 7f285ea00640 1 -- 172.21.15.39:0/1497089869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28600d0880 msgr2=0x7f28601f5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:31.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.598+0000 7f285ea00640 1 --2- 172.21.15.39:0/1497089869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28600d0880 0x7f28601f5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:31.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.598+0000 7f285ea00640 1 -- 172.21.15.39:0/1497089869 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28601f92c0 con 0x7f28600b1740 2024-09-21T07:52:31.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.598+0000 7f285f400640 1 --2- 172.21.15.39:0/1497089869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28600d0880 0x7f28601f5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:52:31.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.598+0000 7f285ea00640 1 --2- 172.21.15.39:0/1497089869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28600b1740 0x7f28601e9830 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 crypto rx=0x7f285000e7c0 tx=0x7f285000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:31.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.598+0000 7f2857400640 1 -- 172.21.15.39:0/1497089869 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285000ccc0 con 0x7f28600b1740 2024-09-21T07:52:31.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.598+0000 7f2857400640 1 -- 172.21.15.39:0/1497089869 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2850010430 con 0x7f28600b1740 2024-09-21T07:52:31.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.598+0000 7f2857400640 1 -- 172.21.15.39:0/1497089869 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2850010840 con 0x7f28600b1740 2024-09-21T07:52:31.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.598+0000 7f285fe00640 1 -- 172.21.15.39:0/1497089869 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28601f95a0 con 0x7f28600b1740 2024-09-21T07:52:31.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.598+0000 7f285fe00640 1 -- 172.21.15.39:0/1497089869 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28601f57e0 con 0x7f28600b1740 2024-09-21T07:52:31.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.599+0000 7f285fe00640 1 -- 172.21.15.39:0/1497089869 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28600b77f0 con 0x7f28600b1740 2024-09-21T07:52:31.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.601+0000 7f2857400640 1 -- 172.21.15.39:0/1497089869 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f28500109e0 con 0x7f28600b1740 2024-09-21T07:52:31.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.602+0000 7f2857400640 1 --2- 172.21.15.39:0/1497089869 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f283c078b90 0x7f283c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:31.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.602+0000 7f2857400640 1 -- 172.21.15.39:0/1497089869 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(247..247 src has 1..247) ==== 7605+0+0 (secure 0 0 0) 0x7f285009bf20 con 0x7f28600b1740 2024-09-21T07:52:31.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.603+0000 7f2857e00640 1 --2- 172.21.15.39:0/1497089869 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f283c078b90 0x7f283c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:31.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.603+0000 7f2857e00640 1 --2- 172.21.15.39:0/1497089869 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f283c078b90 0x7f283c07b0a0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f2848033000 tx=0x7f2848004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:31.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.607+0000 7f2857400640 1 -- 172.21.15.39:0/1497089869 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f28500635a0 con 0x7f28600b1740 2024-09-21T07:52:31.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.846+0000 7f285fe00640 1 -- 172.21.15.39:0/1497089869 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28600b37e0 con 0x7f28600b1740 2024-09-21T07:52:31.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.847+0000 7f2857400640 1 -- 172.21.15.39:0/1497089869 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 281 v281) ==== 95+0+5283 (secure 0 0 0) 0x7f2850067fd0 con 0x7f28600b1740 2024-09-21T07:52:31.849 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:52:31.849 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":281,"btime":"2024-09-21T07:52:29:069596+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":278}],"filesystems":[{"mdsmap":{"epoch":281,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:52:29.069589+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":246,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":10195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10195":{"gid":10195,"name":"d","rank":2,"incarnation":277,"state":"up:active","state_seq":25,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[9967,10195,11052]},"id":13}]} 2024-09-21T07:52:31.849 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 281 2024-09-21T07:52:31.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.849+0000 7f285fe00640 1 -- 172.21.15.39:0/1497089869 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f283c078b90 msgr2=0x7f283c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:31.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.849+0000 7f285fe00640 1 --2- 172.21.15.39:0/1497089869 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f283c078b90 0x7f283c07b0a0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f2848033000 tx=0x7f2848004730 comp rx=0 tx=0).stop 2024-09-21T07:52:31.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.849+0000 7f285fe00640 1 -- 172.21.15.39:0/1497089869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28600b1740 msgr2=0x7f28601e9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:31.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.849+0000 7f285fe00640 1 --2- 172.21.15.39:0/1497089869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28600b1740 0x7f28601e9830 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 crypto rx=0x7f285000e7c0 tx=0x7f285000ec80 comp rx=0 tx=0).stop 2024-09-21T07:52:31.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.849+0000 7f285fe00640 1 -- 172.21.15.39:0/1497089869 shutdown_connections 2024-09-21T07:52:31.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.849+0000 7f285fe00640 1 --2- 172.21.15.39:0/1497089869 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f283c078b90 0x7f283c07b0a0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:31.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.849+0000 7f285fe00640 1 --2- 172.21.15.39:0/1497089869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28600d0880 0x7f28601f5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:31.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.849+0000 7f285fe00640 1 --2- 172.21.15.39:0/1497089869 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f28600b1c90 0x7f286015eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:31.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.849+0000 7f285fe00640 1 --2- 172.21.15.39:0/1497089869 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28600b1740 0x7f28601e9830 unknown :-1 s=CLOSED pgs=2548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:31.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.849+0000 7f285fe00640 1 -- 172.21.15.39:0/1497089869 >> 172.21.15.39:0/1497089869 conn(0x7f28600cee40 msgr2=0x7f28600d0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:31.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.850+0000 7f285fe00640 1 -- 172.21.15.39:0/1497089869 shutdown_connections 2024-09-21T07:52:31.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:31.850+0000 7f285fe00640 1 -- 172.21.15.39:0/1497089869 wait complete. 2024-09-21T07:52:31.874 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps 2024-09-21T07:52:32.129 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_created":8,"last_destroyed":7,"snaps":[{"snapid":8,"ino":1099511627780,"stamp":"2024-09-21T07:51:12.091986+0000","name":"s1","metadata":{}}]} 2024-09-21T07:52:32.130 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 8, "last_destroyed": 7, "snaps": [ { "ino": 1099511627780, "metadata": {}, "name": "s1", "snapid": 8, "stamp": "2024-09-21T07:51:12.091986+0000" } ] } 2024-09-21T07:52:32.130 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:52:32.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.357+0000 7f4ed4000640 1 -- 172.21.15.39:0/2441279880 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4ecc0d06e0 msgr2=0x7f4ecc0d0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:32.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.357+0000 7f4ed4000640 1 --2- 172.21.15.39:0/2441279880 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4ecc0d06e0 0x7f4ecc0d0af0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4005fd0 tx=0x7f4ec402a8b0 comp rx=0 tx=0).stop 2024-09-21T07:52:32.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.357+0000 7f4ed4000640 1 -- 172.21.15.39:0/2441279880 shutdown_connections 2024-09-21T07:52:32.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.357+0000 7f4ed4000640 1 --2- 172.21.15.39:0/2441279880 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4ecc0d06e0 0x7f4ecc0d0af0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:32.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.357+0000 7f4ed4000640 1 --2- 172.21.15.39:0/2441279880 >> v2:172.21.15.148:3300/0 conn(0x7f4ecc0cfd10 0x7f4ecc0cd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:32.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.357+0000 7f4ed4000640 1 -- 172.21.15.39:0/2441279880 >> 172.21.15.39:0/2441279880 conn(0x7f4ecc06cfc0 msgr2=0x7f4ecc06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:32.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.357+0000 7f4ed4000640 1 -- 172.21.15.39:0/2441279880 shutdown_connections 2024-09-21T07:52:32.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.357+0000 7f4ed4000640 1 -- 172.21.15.39:0/2441279880 wait complete. 2024-09-21T07:52:32.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.360+0000 7f4ed4000640 1 Processor -- start 2024-09-21T07:52:32.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.360+0000 7f4ed4000640 1 -- start start 2024-09-21T07:52:32.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.361+0000 7f4ed4000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ecc0cfd10 0x7f4ecc14ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:32.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.361+0000 7f4ed4000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4ecc0d06e0 0x7f4ecc0b4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:32.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.361+0000 7f4ed4000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ecc15cd30 0x7f4ecc1e92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:32.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.361+0000 7f4ed4000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4ecc073970 con 0x7f4ecc15cd30 2024-09-21T07:52:32.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.361+0000 7f4ed4000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4ecc0737f0 con 0x7f4ecc0cfd10 2024-09-21T07:52:32.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.361+0000 7f4ed4000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f4ecc073af0 con 0x7f4ecc0d06e0 2024-09-21T07:52:32.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.362+0000 7f4ed2000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ecc15cd30 0x7f4ecc1e92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:32.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.362+0000 7f4ed2000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ecc15cd30 0x7f4ecc1e92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60648/0 (socket says 172.21.15.39:60648) 2024-09-21T07:52:32.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.362+0000 7f4ed2000640 1 -- 172.21.15.39:0/490459906 learned_addr learned my addr 172.21.15.39:0/490459906 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:32.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.362+0000 7f4ed0c00640 1 --2- 172.21.15.39:0/490459906 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4ecc0d06e0 0x7f4ecc0b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:32.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.362+0000 7f4ed1600640 1 --2- 172.21.15.39:0/490459906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ecc0cfd10 0x7f4ecc14ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:32.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.362+0000 7f4ed2000640 1 -- 172.21.15.39:0/490459906 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4ecc0d06e0 msgr2=0x7f4ecc0b4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:32.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.362+0000 7f4ed2000640 1 --2- 172.21.15.39:0/490459906 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4ecc0d06e0 0x7f4ecc0b4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:32.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.362+0000 7f4ed2000640 1 -- 172.21.15.39:0/490459906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ecc0cfd10 msgr2=0x7f4ecc14ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:32.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.362+0000 7f4ed2000640 1 --2- 172.21.15.39:0/490459906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ecc0cfd10 0x7f4ecc14ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:32.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.362+0000 7f4ed2000640 1 -- 172.21.15.39:0/490459906 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ecc155310 con 0x7f4ecc15cd30 2024-09-21T07:52:32.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.362+0000 7f4ed1600640 1 --2- 172.21.15.39:0/490459906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ecc0cfd10 0x7f4ecc14ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:52:32.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.362+0000 7f4ed2000640 1 --2- 172.21.15.39:0/490459906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ecc15cd30 0x7f4ecc1e92d0 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7f4ec402aea0 tx=0x7f4ec40316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:32.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.363+0000 7f4ec9600640 1 -- 172.21.15.39:0/490459906 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4031da0 con 0x7f4ecc15cd30 2024-09-21T07:52:32.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.363+0000 7f4ed4000640 1 -- 172.21.15.39:0/490459906 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ecc1555f0 con 0x7f4ecc15cd30 2024-09-21T07:52:32.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.363+0000 7f4ec9600640 1 -- 172.21.15.39:0/490459906 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ec4003a60 con 0x7f4ecc15cd30 2024-09-21T07:52:32.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.363+0000 7f4ec9600640 1 -- 172.21.15.39:0/490459906 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec40330d0 con 0x7f4ecc15cd30 2024-09-21T07:52:32.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.363+0000 7f4ed4000640 1 -- 172.21.15.39:0/490459906 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ecc155bb0 con 0x7f4ecc15cd30 2024-09-21T07:52:32.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.364+0000 7f4ed4000640 1 -- 172.21.15.39:0/490459906 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ecc0b5130 con 0x7f4ecc15cd30 2024-09-21T07:52:32.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.365+0000 7f4ec9600640 1 -- 172.21.15.39:0/490459906 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4ec4033270 con 0x7f4ecc15cd30 2024-09-21T07:52:32.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.366+0000 7f4ec9600640 1 --2- 172.21.15.39:0/490459906 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4eac078b90 0x7f4eac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:32.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.366+0000 7f4ec9600640 1 -- 172.21.15.39:0/490459906 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(247..247 src has 1..247) ==== 7605+0+0 (secure 0 0 0) 0x7f4ec40bc570 con 0x7f4ecc15cd30 2024-09-21T07:52:32.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.367+0000 7f4ed1600640 1 --2- 172.21.15.39:0/490459906 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4eac078b90 0x7f4eac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:32.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.368+0000 7f4ed1600640 1 --2- 172.21.15.39:0/490459906 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4eac078b90 0x7f4eac07b0a0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0007fd0 tx=0x7f4ec0007cf0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:32.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.371+0000 7f4ec9600640 1 -- 172.21.15.39:0/490459906 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4ec404a030 con 0x7f4ecc15cd30 2024-09-21T07:52:32.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.608+0000 7f4ed4000640 1 -- 172.21.15.39:0/490459906 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ecc073fa0 con 0x7f4ecc15cd30 2024-09-21T07:52:32.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.608+0000 7f4ec9600640 1 -- 172.21.15.39:0/490459906 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 281 v281) ==== 95+0+5283 (secure 0 0 0) 0x7f4ec403f030 con 0x7f4ecc15cd30 2024-09-21T07:52:32.610 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:52:32.610 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":281,"btime":"2024-09-21T07:52:29:069596+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":278}],"filesystems":[{"mdsmap":{"epoch":281,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:52:29.069589+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":246,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":10195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10195":{"gid":10195,"name":"d","rank":2,"incarnation":277,"state":"up:active","state_seq":25,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[9967,10195,11052]},"id":13}]} 2024-09-21T07:52:32.611 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 281 2024-09-21T07:52:32.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.610+0000 7f4ed4000640 1 -- 172.21.15.39:0/490459906 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4eac078b90 msgr2=0x7f4eac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:32.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.610+0000 7f4ed4000640 1 --2- 172.21.15.39:0/490459906 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4eac078b90 0x7f4eac07b0a0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0007fd0 tx=0x7f4ec0007cf0 comp rx=0 tx=0).stop 2024-09-21T07:52:32.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.610+0000 7f4ed4000640 1 -- 172.21.15.39:0/490459906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ecc15cd30 msgr2=0x7f4ecc1e92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:32.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.610+0000 7f4ed4000640 1 --2- 172.21.15.39:0/490459906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ecc15cd30 0x7f4ecc1e92d0 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7f4ec402aea0 tx=0x7f4ec40316f0 comp rx=0 tx=0).stop 2024-09-21T07:52:32.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.611+0000 7f4ed4000640 1 -- 172.21.15.39:0/490459906 shutdown_connections 2024-09-21T07:52:32.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.611+0000 7f4ed4000640 1 --2- 172.21.15.39:0/490459906 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4eac078b90 0x7f4eac07b0a0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:32.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.611+0000 7f4ed4000640 1 --2- 172.21.15.39:0/490459906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ecc15cd30 0x7f4ecc1e92d0 unknown :-1 s=CLOSED pgs=2549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:32.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.611+0000 7f4ed4000640 1 --2- 172.21.15.39:0/490459906 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4ecc0d06e0 0x7f4ecc0b4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:32.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.611+0000 7f4ed4000640 1 --2- 172.21.15.39:0/490459906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ecc0cfd10 0x7f4ecc14ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:32.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.611+0000 7f4ed4000640 1 -- 172.21.15.39:0/490459906 >> 172.21.15.39:0/490459906 conn(0x7f4ecc06cfc0 msgr2=0x7f4ecc06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:32.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.611+0000 7f4ed4000640 1 -- 172.21.15.39:0/490459906 shutdown_connections 2024-09-21T07:52:32.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:32.611+0000 7f4ed4000640 1 -- 172.21.15.39:0/490459906 wait complete. 2024-09-21T07:52:32.634 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json dump snaps 2024-09-21T07:52:32.878 INFO:teuthology.orchestra.run.smithi148.stdout:{"last_created":8,"last_destroyed":7,"snaps":[{"snapid":8,"ino":1099511627780,"stamp":"2024-09-21T07:51:12.091986+0000","name":"s1","metadata":{}}]} 2024-09-21T07:52:32.878 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 8, "last_destroyed": 7, "snaps": [ { "ino": 1099511627780, "metadata": {}, "name": "s1", "snapid": 8, "stamp": "2024-09-21T07:51:12.091986+0000" } ] } 2024-09-21T07:52:32.879 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:52:33.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.105+0000 7f38c3e00640 1 -- 172.21.15.39:0/1754602324 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f38c40d17e0 msgr2=0x7f38c40d2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:33.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.105+0000 7f38c3e00640 1 --2- 172.21.15.39:0/1754602324 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f38c40d17e0 0x7f38c40d2670 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f38b0009860 tx=0x7f38b002a4a0 comp rx=0 tx=0).stop 2024-09-21T07:52:33.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.105+0000 7f38c3e00640 1 -- 172.21.15.39:0/1754602324 shutdown_connections 2024-09-21T07:52:33.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.105+0000 7f38c3e00640 1 --2- 172.21.15.39:0/1754602324 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f38c40d17e0 0x7f38c40d2670 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:33.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.105+0000 7f38c3e00640 1 -- 172.21.15.39:0/1754602324 >> 172.21.15.39:0/1754602324 conn(0x7f38c409d960 msgr2=0x7f38c40d46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:33.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.105+0000 7f38c3e00640 1 -- 172.21.15.39:0/1754602324 shutdown_connections 2024-09-21T07:52:33.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.105+0000 7f38c3e00640 1 -- 172.21.15.39:0/1754602324 wait complete. 2024-09-21T07:52:33.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.107+0000 7f38c3e00640 1 Processor -- start 2024-09-21T07:52:33.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.107+0000 7f38c3e00640 1 -- start start 2024-09-21T07:52:33.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.109+0000 7f38c3e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38c40b3c80 0x7f38c414b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:33.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.109+0000 7f38c3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38c40b9d30 0x7f38c40ca340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:33.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.109+0000 7f38c3e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f38c40c81f0 0x7f38c40c8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:33.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.109+0000 7f38c3e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f38c406e730 con 0x7f38c40b9d30 2024-09-21T07:52:33.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.109+0000 7f38c3e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f38c406e5b0 con 0x7f38c40b3c80 2024-09-21T07:52:33.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.109+0000 7f38c3e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f38c406e8b0 con 0x7f38c40c81f0 2024-09-21T07:52:33.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.109+0000 7f38c2000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38c40b9d30 0x7f38c40ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:33.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.109+0000 7f38c2000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38c40b9d30 0x7f38c40ca340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60664/0 (socket says 172.21.15.39:60664) 2024-09-21T07:52:33.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.109+0000 7f38c2000640 1 -- 172.21.15.39:0/1705361566 learned_addr learned my addr 172.21.15.39:0/1705361566 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:33.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.109+0000 7f38c2a00640 1 --2- 172.21.15.39:0/1705361566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38c40b3c80 0x7f38c414b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:33.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.109+0000 7f38c3400640 1 --2- 172.21.15.39:0/1705361566 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f38c40c81f0 0x7f38c40c8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:33.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.110+0000 7f38c2000640 1 -- 172.21.15.39:0/1705361566 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f38c40c81f0 msgr2=0x7f38c40c8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:33.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.110+0000 7f38c2000640 1 --2- 172.21.15.39:0/1705361566 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f38c40c81f0 0x7f38c40c8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:33.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.110+0000 7f38c2000640 1 -- 172.21.15.39:0/1705361566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38c40b3c80 msgr2=0x7f38c414b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:33.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.110+0000 7f38c2000640 1 --2- 172.21.15.39:0/1705361566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38c40b3c80 0x7f38c414b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:33.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.110+0000 7f38c2000640 1 -- 172.21.15.39:0/1705361566 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c414e310 con 0x7f38c40b9d30 2024-09-21T07:52:33.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.110+0000 7f38c2a00640 1 --2- 172.21.15.39:0/1705361566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38c40b3c80 0x7f38c414b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:52:33.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.110+0000 7f38c3400640 1 --2- 172.21.15.39:0/1705361566 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f38c40c81f0 0x7f38c40c8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:52:33.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.110+0000 7f38c2000640 1 --2- 172.21.15.39:0/1705361566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38c40b9d30 0x7f38c40ca340 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 crypto rx=0x7f38b0009950 tx=0x7f38b0003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:33.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.110+0000 7f38baa00640 1 -- 172.21.15.39:0/1705361566 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b0005c90 con 0x7f38c40b9d30 2024-09-21T07:52:33.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.110+0000 7f38baa00640 1 -- 172.21.15.39:0/1705361566 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38b0005310 con 0x7f38c40b9d30 2024-09-21T07:52:33.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.110+0000 7f38baa00640 1 -- 172.21.15.39:0/1705361566 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b00047a0 con 0x7f38c40b9d30 2024-09-21T07:52:33.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.110+0000 7f38c3e00640 1 -- 172.21.15.39:0/1705361566 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38c40c8b40 con 0x7f38c40b9d30 2024-09-21T07:52:33.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.110+0000 7f38c3e00640 1 -- 172.21.15.39:0/1705361566 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38c413c0f0 con 0x7f38c40b9d30 2024-09-21T07:52:33.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.112+0000 7f38c3e00640 1 -- 172.21.15.39:0/1705361566 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38c40d2670 con 0x7f38c40b9d30 2024-09-21T07:52:33.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.113+0000 7f38baa00640 1 -- 172.21.15.39:0/1705361566 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f38b0004940 con 0x7f38c40b9d30 2024-09-21T07:52:33.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.114+0000 7f38baa00640 1 --2- 172.21.15.39:0/1705361566 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f38a4078bc0 0x7f38a407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:33.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.114+0000 7f38baa00640 1 -- 172.21.15.39:0/1705361566 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(247..247 src has 1..247) ==== 7605+0+0 (secure 0 0 0) 0x7f38b00bc4a0 con 0x7f38c40b9d30 2024-09-21T07:52:33.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.115+0000 7f38c2a00640 1 --2- 172.21.15.39:0/1705361566 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f38a4078bc0 0x7f38a407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:33.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.116+0000 7f38c2a00640 1 --2- 172.21.15.39:0/1705361566 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f38a4078bc0 0x7f38a407b0d0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f38b4000fb0 tx=0x7f38b4009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:33.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.119+0000 7f38baa00640 1 -- 172.21.15.39:0/1705361566 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f38c40c8b40 con 0x7f38c40b9d30 2024-09-21T07:52:33.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.359+0000 7f38c3e00640 1 -- 172.21.15.39:0/1705361566 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38c40caf20 con 0x7f38c40b9d30 2024-09-21T07:52:33.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.360+0000 7f38baa00640 1 -- 172.21.15.39:0/1705361566 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 281 v281) ==== 95+0+5283 (secure 0 0 0) 0x7f38c414e310 con 0x7f38c40b9d30 2024-09-21T07:52:33.362 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:52:33.363 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":281,"btime":"2024-09-21T07:52:29:069596+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":278}],"filesystems":[{"mdsmap":{"epoch":281,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:52:29.069589+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":246,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":10195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10195":{"gid":10195,"name":"d","rank":2,"incarnation":277,"state":"up:active","state_seq":25,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[9967,10195,11052]},"id":13}]} 2024-09-21T07:52:33.363 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 281 2024-09-21T07:52:33.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.362+0000 7f38c3e00640 1 -- 172.21.15.39:0/1705361566 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f38a4078bc0 msgr2=0x7f38a407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:33.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.362+0000 7f38c3e00640 1 --2- 172.21.15.39:0/1705361566 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f38a4078bc0 0x7f38a407b0d0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f38b4000fb0 tx=0x7f38b4009040 comp rx=0 tx=0).stop 2024-09-21T07:52:33.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.362+0000 7f38c3e00640 1 -- 172.21.15.39:0/1705361566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38c40b9d30 msgr2=0x7f38c40ca340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:33.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.362+0000 7f38c3e00640 1 --2- 172.21.15.39:0/1705361566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38c40b9d30 0x7f38c40ca340 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 crypto rx=0x7f38b0009950 tx=0x7f38b0003b90 comp rx=0 tx=0).stop 2024-09-21T07:52:33.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.363+0000 7f38c3e00640 1 -- 172.21.15.39:0/1705361566 shutdown_connections 2024-09-21T07:52:33.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.363+0000 7f38c3e00640 1 --2- 172.21.15.39:0/1705361566 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f38a4078bc0 0x7f38a407b0d0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:33.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.363+0000 7f38c3e00640 1 --2- 172.21.15.39:0/1705361566 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f38c40c81f0 0x7f38c40c8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:33.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.363+0000 7f38c3e00640 1 --2- 172.21.15.39:0/1705361566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38c40b9d30 0x7f38c40ca340 unknown :-1 s=CLOSED pgs=2551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:33.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.363+0000 7f38c3e00640 1 --2- 172.21.15.39:0/1705361566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38c40b3c80 0x7f38c414b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:33.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.363+0000 7f38c3e00640 1 -- 172.21.15.39:0/1705361566 >> 172.21.15.39:0/1705361566 conn(0x7f38c409d960 msgr2=0x7f38c415ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:33.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.363+0000 7f38c3e00640 1 -- 172.21.15.39:0/1705361566 shutdown_connections 2024-09-21T07:52:33.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.363+0000 7f38c3e00640 1 -- 172.21.15.39:0/1705361566 wait complete. 2024-09-21T07:52:33.390 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json dump snaps --server 2024-09-21T07:52:33.629 INFO:teuthology.orchestra.run.smithi148.stdout:{"last_snap":8,"last_created":8,"last_destroyed":7,"pending_noop":[],"snaps":[{"snapid":8,"ino":1099511627780,"stamp":"2024-09-21T07:51:12.091986+0000","name":"s1","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2024-09-21T07:52:33.629 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 8, "last_destroyed": 7, "last_snap": 8, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627780, "metadata": {}, "name": "s1", "snapid": 8, "stamp": "2024-09-21T07:51:12.091986+0000" } ] } 2024-09-21T07:52:33.629 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T07:52:33.629 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:52:33.847 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:52:33.851 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:52:33.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.853+0000 7fec09600640 1 -- 172.21.15.39:0/2457042607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec040d0bd0 msgr2=0x7fec040b9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:33.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.853+0000 7fec09600640 1 --2- 172.21.15.39:0/2457042607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec040d0bd0 0x7fec040b9eb0 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7fec040b4180 tx=0x7febf402a560 comp rx=0 tx=0).stop 2024-09-21T07:52:33.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.854+0000 7fec09600640 1 -- 172.21.15.39:0/2457042607 shutdown_connections 2024-09-21T07:52:33.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.854+0000 7fec09600640 1 --2- 172.21.15.39:0/2457042607 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec040b61d0 0x7fec040b65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:33.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.854+0000 7fec09600640 1 --2- 172.21.15.39:0/2457042607 >> v2:172.21.15.148:3300/0 conn(0x7fec040d2c80 0x7fec040d3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:33.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.854+0000 7fec09600640 1 --2- 172.21.15.39:0/2457042607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec040d0bd0 0x7fec040b9eb0 unknown :-1 s=CLOSED pgs=2552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:33.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.854+0000 7fec09600640 1 -- 172.21.15.39:0/2457042607 >> 172.21.15.39:0/2457042607 conn(0x7fec040d4750 msgr2=0x7fec040d4b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:33.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.854+0000 7fec09600640 1 -- 172.21.15.39:0/2457042607 shutdown_connections 2024-09-21T07:52:33.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.854+0000 7fec09600640 1 -- 172.21.15.39:0/2457042607 wait complete. 2024-09-21T07:52:33.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.857+0000 7fec09600640 1 Processor -- start 2024-09-21T07:52:33.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.857+0000 7fec09600640 1 -- start start 2024-09-21T07:52:33.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.859+0000 7fec09600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec040b61d0 0x7fec0414a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:33.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.859+0000 7fec09600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec040d0bd0 0x7fec0414cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:33.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.859+0000 7fec09600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec040d2c80 0x7fec0414d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:33.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.859+0000 7fec09600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fec0406e190 con 0x7fec040d2c80 2024-09-21T07:52:33.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.859+0000 7fec09600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fec0406e010 con 0x7fec040d0bd0 2024-09-21T07:52:33.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.859+0000 7fec09600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fec0406e310 con 0x7fec040b61d0 2024-09-21T07:52:33.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.859+0000 7fec03400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec040d2c80 0x7fec0414d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:33.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.859+0000 7fec03400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec040d2c80 0x7fec0414d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60680/0 (socket says 172.21.15.39:60680) 2024-09-21T07:52:33.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.859+0000 7fec03400640 1 -- 172.21.15.39:0/3001110256 learned_addr learned my addr 172.21.15.39:0/3001110256 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:33.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.859+0000 7fec02000640 1 --2- 172.21.15.39:0/3001110256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec040d0bd0 0x7fec0414cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:33.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.859+0000 7fec02a00640 1 --2- 172.21.15.39:0/3001110256 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec040b61d0 0x7fec0414a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:33.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.859+0000 7fec03400640 1 -- 172.21.15.39:0/3001110256 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec040b61d0 msgr2=0x7fec0414a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:33.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.859+0000 7fec03400640 1 --2- 172.21.15.39:0/3001110256 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec040b61d0 0x7fec0414a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:33.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.860+0000 7fec03400640 1 -- 172.21.15.39:0/3001110256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec040d0bd0 msgr2=0x7fec0414cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:33.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.860+0000 7fec03400640 1 --2- 172.21.15.39:0/3001110256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec040d0bd0 0x7fec0414cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:33.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.860+0000 7fec03400640 1 -- 172.21.15.39:0/3001110256 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec041f62b0 con 0x7fec040d2c80 2024-09-21T07:52:33.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.860+0000 7fec03400640 1 --2- 172.21.15.39:0/3001110256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec040d2c80 0x7fec0414d140 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7febfc005f40 tx=0x7febfc00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:33.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.860+0000 7febfaa00640 1 -- 172.21.15.39:0/3001110256 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc004d20 con 0x7fec040d2c80 2024-09-21T07:52:33.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.860+0000 7febfaa00640 1 -- 172.21.15.39:0/3001110256 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7febfc00cb20 con 0x7fec040d2c80 2024-09-21T07:52:33.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.860+0000 7fec09600640 1 -- 172.21.15.39:0/3001110256 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec041f6590 con 0x7fec040d2c80 2024-09-21T07:52:33.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.860+0000 7febfaa00640 1 -- 172.21.15.39:0/3001110256 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc005020 con 0x7fec040d2c80 2024-09-21T07:52:33.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.860+0000 7fec09600640 1 -- 172.21.15.39:0/3001110256 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec041f6b50 con 0x7fec040d2c80 2024-09-21T07:52:33.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.862+0000 7fec09600640 1 -- 172.21.15.39:0/3001110256 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec040b9eb0 con 0x7fec040d2c80 2024-09-21T07:52:33.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.863+0000 7febfaa00640 1 -- 172.21.15.39:0/3001110256 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7febfc0051c0 con 0x7fec040d2c80 2024-09-21T07:52:33.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.864+0000 7febfaa00640 1 --2- 172.21.15.39:0/3001110256 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7febe4078b90 0x7febe407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:33.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.864+0000 7febfaa00640 1 -- 172.21.15.39:0/3001110256 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(247..247 src has 1..247) ==== 7605+0+0 (secure 0 0 0) 0x7febfc09bf70 con 0x7fec040d2c80 2024-09-21T07:52:33.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.864+0000 7fec02a00640 1 --2- 172.21.15.39:0/3001110256 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7febe4078b90 0x7febe407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:33.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.865+0000 7fec02a00640 1 --2- 172.21.15.39:0/3001110256 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7febe4078b90 0x7febe407b0a0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7febf4033000 tx=0x7febf4004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:33.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:33.868+0000 7febfaa00640 1 -- 172.21.15.39:0/3001110256 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7febfc063550 con 0x7fec040d2c80 2024-09-21T07:52:34.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:34.109+0000 7fec09600640 1 -- 172.21.15.39:0/3001110256 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec0406fdf0 con 0x7fec040d2c80 2024-09-21T07:52:34.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:34.109+0000 7febfaa00640 1 -- 172.21.15.39:0/3001110256 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 281 v281) ==== 95+0+5283 (secure 0 0 0) 0x7febfc067f80 con 0x7fec040d2c80 2024-09-21T07:52:34.111 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:52:34.112 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":281,"btime":"2024-09-21T07:52:29:069596+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":278}],"filesystems":[{"mdsmap":{"epoch":281,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:52:29.069589+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":246,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":10195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10195":{"gid":10195,"name":"d","rank":2,"incarnation":277,"state":"up:active","state_seq":25,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[9967,10195,11052]},"id":13}]} 2024-09-21T07:52:34.112 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 281 2024-09-21T07:52:34.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:34.112+0000 7fec09600640 1 -- 172.21.15.39:0/3001110256 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7febe4078b90 msgr2=0x7febe407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:34.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:34.112+0000 7fec09600640 1 --2- 172.21.15.39:0/3001110256 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7febe4078b90 0x7febe407b0a0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7febf4033000 tx=0x7febf4004730 comp rx=0 tx=0).stop 2024-09-21T07:52:34.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:34.112+0000 7fec09600640 1 -- 172.21.15.39:0/3001110256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec040d2c80 msgr2=0x7fec0414d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:34.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:34.112+0000 7fec09600640 1 --2- 172.21.15.39:0/3001110256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec040d2c80 0x7fec0414d140 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7febfc005f40 tx=0x7febfc00ee60 comp rx=0 tx=0).stop 2024-09-21T07:52:34.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:34.112+0000 7fec09600640 1 -- 172.21.15.39:0/3001110256 shutdown_connections 2024-09-21T07:52:34.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:34.112+0000 7fec09600640 1 --2- 172.21.15.39:0/3001110256 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7febe4078b90 0x7febe407b0a0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:34.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:34.112+0000 7fec09600640 1 --2- 172.21.15.39:0/3001110256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec040d2c80 0x7fec0414d140 unknown :-1 s=CLOSED pgs=2553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:34.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:34.112+0000 7fec09600640 1 --2- 172.21.15.39:0/3001110256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fec040d0bd0 0x7fec0414cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:34.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:34.112+0000 7fec09600640 1 --2- 172.21.15.39:0/3001110256 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fec040b61d0 0x7fec0414a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:34.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:34.112+0000 7fec09600640 1 -- 172.21.15.39:0/3001110256 >> 172.21.15.39:0/3001110256 conn(0x7fec040d4750 msgr2=0x7fec040b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:34.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:34.112+0000 7fec09600640 1 -- 172.21.15.39:0/3001110256 shutdown_connections 2024-09-21T07:52:34.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:34.112+0000 7fec09600640 1 -- 172.21.15.39:0/3001110256 wait complete. 2024-09-21T07:52:34.139 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json dump snaps --server 2024-09-21T07:52:34.379 INFO:teuthology.orchestra.run.smithi148.stdout:{"last_snap":8,"last_created":8,"last_destroyed":7,"pending_noop":[],"snaps":[{"snapid":8,"ino":1099511627780,"stamp":"2024-09-21T07:51:12.091986+0000","name":"s1","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2024-09-21T07:52:34.380 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 8, "last_destroyed": 7, "last_snap": 8, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627780, "metadata": {}, "name": "s1", "snapid": 8, "stamp": "2024-09-21T07:51:12.091986+0000" } ] } 2024-09-21T07:52:34.380 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:52:34.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:34.380 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-09-21T07:52:39.186 INFO:teuthology.orchestra.run.smithi039.stderr:mkdir: cannot create directory ‘d0/d2/dir/.snap/s1’: Permission denied 2024-09-21T07:52:39.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:52:39.187 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:52:39.188 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:52:39.225 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:52:39.225 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:52:39.302 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:52:39.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.572+0000 7f9d8ee00640 1 -- 172.21.15.39:0/2064087538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d6c003a00 msgr2=0x7f9d6c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:39.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.572+0000 7f9d8ee00640 1 --2- 172.21.15.39:0/2064087538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d6c003a00 0x7f9d6c003e70 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7f9d7800c290 tx=0x7f9d6c005830 comp rx=0 tx=0).stop 2024-09-21T07:52:39.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.572+0000 7f9d8ce00640 1 -- 172.21.15.39:0/2064087538 reap_dead start 2024-09-21T07:52:39.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.572+0000 7f9d8ee00640 1 -- 172.21.15.39:0/2064087538 shutdown_connections 2024-09-21T07:52:39.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.572+0000 7f9d8ee00640 1 -- 172.21.15.39:0/2064087538 >> 172.21.15.39:0/2064087538 conn(0x7f9d8806cfc0 msgr2=0x7f9d8806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:39.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.572+0000 7f9d8ee00640 1 -- 172.21.15.39:0/2064087538 shutdown_connections 2024-09-21T07:52:39.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.572+0000 7f9d8ee00640 1 -- 172.21.15.39:0/2064087538 wait complete. 2024-09-21T07:52:39.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.574+0000 7f9d8ee00640 1 Processor -- start 2024-09-21T07:52:39.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.574+0000 7f9d8ee00640 1 -- start start 2024-09-21T07:52:39.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.576+0000 7f9d8ee00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d6c003a00 0x7f9d881e89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:39.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.576+0000 7f9d8ee00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9d881e8f30 0x7f9d8813f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:39.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.576+0000 7f9d8ee00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d8813fe90 0x7f9d881f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:39.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.576+0000 7f9d8ee00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9d880cdcf0 con 0x7f9d6c003a00 2024-09-21T07:52:39.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.576+0000 7f9d8ee00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9d880cdb70 con 0x7f9d8813fe90 2024-09-21T07:52:39.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.576+0000 7f9d8ee00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f9d880cde70 con 0x7f9d881e8f30 2024-09-21T07:52:39.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.576+0000 7f9d87e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d6c003a00 0x7f9d881e89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:39.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.576+0000 7f9d87e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d6c003a00 0x7f9d881e89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60700/0 (socket says 172.21.15.39:60700) 2024-09-21T07:52:39.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.576+0000 7f9d87e00640 1 -- 172.21.15.39:0/4128380388 learned_addr learned my addr 172.21.15.39:0/4128380388 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:39.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.576+0000 7f9d7fe00640 1 --2- 172.21.15.39:0/4128380388 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9d881e8f30 0x7f9d8813f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:39.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.576+0000 7f9d8ce00640 1 --2- 172.21.15.39:0/4128380388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d8813fe90 0x7f9d881f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:39.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.577+0000 7f9d87e00640 1 -- 172.21.15.39:0/4128380388 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9d881e8f30 msgr2=0x7f9d8813f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:39.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.577+0000 7f9d87e00640 1 --2- 172.21.15.39:0/4128380388 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9d881e8f30 0x7f9d8813f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:39.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.577+0000 7f9d87e00640 1 -- 172.21.15.39:0/4128380388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d8813fe90 msgr2=0x7f9d881f6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:39.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.577+0000 7f9d87e00640 1 --2- 172.21.15.39:0/4128380388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d8813fe90 0x7f9d881f6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:39.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.577+0000 7f9d87e00640 1 -- 172.21.15.39:0/4128380388 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d881e9340 con 0x7f9d6c003a00 2024-09-21T07:52:39.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.577+0000 7f9d8ce00640 1 --2- 172.21.15.39:0/4128380388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d8813fe90 0x7f9d881f6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:52:39.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.577+0000 7f9d87e00640 1 --2- 172.21.15.39:0/4128380388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d6c003a00 0x7f9d881e89f0 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7f9d7800c3b0 tx=0x7f9d78006d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:39.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.577+0000 7f9d84c00640 1 -- 172.21.15.39:0/4128380388 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d78031040 con 0x7f9d6c003a00 2024-09-21T07:52:39.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.577+0000 7f9d84c00640 1 -- 172.21.15.39:0/4128380388 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d780035f0 con 0x7f9d6c003a00 2024-09-21T07:52:39.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.577+0000 7f9d8ee00640 1 -- 172.21.15.39:0/4128380388 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d88142830 con 0x7f9d6c003a00 2024-09-21T07:52:39.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.577+0000 7f9d84c00640 1 -- 172.21.15.39:0/4128380388 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d78003a00 con 0x7f9d6c003a00 2024-09-21T07:52:39.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.577+0000 7f9d8ee00640 1 -- 172.21.15.39:0/4128380388 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d880a9760 con 0x7f9d6c003a00 2024-09-21T07:52:39.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.578+0000 7f9d8ee00640 1 -- 172.21.15.39:0/4128380388 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d880d0a30 con 0x7f9d6c003a00 2024-09-21T07:52:39.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.580+0000 7f9d84c00640 1 -- 172.21.15.39:0/4128380388 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9d78034070 con 0x7f9d6c003a00 2024-09-21T07:52:39.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.581+0000 7f9d84c00640 1 --2- 172.21.15.39:0/4128380388 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9d64078b90 0x7f9d6407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:39.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.581+0000 7f9d84c00640 1 -- 172.21.15.39:0/4128380388 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(247..247 src has 1..247) ==== 7605+0+0 (secure 0 0 0) 0x7f9d780bb680 con 0x7f9d6c003a00 2024-09-21T07:52:39.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.581+0000 7f9d7fe00640 1 --2- 172.21.15.39:0/4128380388 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9d64078b90 0x7f9d6407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:39.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.582+0000 7f9d7fe00640 1 --2- 172.21.15.39:0/4128380388 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9d64078b90 0x7f9d6407b0a0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f9d88146060 tx=0x7f9d70005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:39.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.585+0000 7f9d84c00640 1 -- 172.21.15.39:0/4128380388 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9d881e9340 con 0x7f9d6c003a00 2024-09-21T07:52:39.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.794+0000 7f9d8ee00640 1 -- 172.21.15.39:0/4128380388 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9d88071470 con 0x7f9d6c003a00 2024-09-21T07:52:39.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.795+0000 7f9d84c00640 1 -- 172.21.15.39:0/4128380388 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 11 entries v247) ==== 81+0+669 (secure 0 0 0) 0x7f9d78083340 con 0x7f9d6c003a00 2024-09-21T07:52:39.797 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/106826898 2024-09-21T08:52:26.149462+0000 2024-09-21T07:52:39.797 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/109229385 2024-09-22T07:51:36.504781+0000 2024-09-21T07:52:39.797 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/109229385 2024-09-22T07:51:36.504781+0000 2024-09-21T07:52:39.797 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/1438851059 2024-09-22T07:51:00.902180+0000 2024-09-21T07:52:39.797 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6835/1438851059 2024-09-22T07:51:00.902180+0000 2024-09-21T07:52:39.797 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/2824493631 2024-09-22T07:50:31.817924+0000 2024-09-21T07:52:39.797 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6833/2824493631 2024-09-22T07:50:31.817924+0000 2024-09-21T07:52:39.797 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/4143807732 2024-09-22T07:50:32.369206+0000 2024-09-21T07:52:39.798 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/4143807732 2024-09-22T07:50:32.369206+0000 2024-09-21T07:52:39.798 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/3497801035 2024-09-22T07:50:24.850152+0000 2024-09-21T07:52:39.798 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6834/3497801035 2024-09-22T07:50:24.850152+0000 2024-09-21T07:52:39.798 INFO:teuthology.orchestra.run.smithi039.stderr:listed 11 entries 2024-09-21T07:52:39.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.797+0000 7f9d8ee00640 1 -- 172.21.15.39:0/4128380388 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9d64078b90 msgr2=0x7f9d6407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:39.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.797+0000 7f9d8ee00640 1 --2- 172.21.15.39:0/4128380388 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9d64078b90 0x7f9d6407b0a0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f9d88146060 tx=0x7f9d70005f90 comp rx=0 tx=0).stop 2024-09-21T07:52:39.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.797+0000 7f9d8ee00640 1 -- 172.21.15.39:0/4128380388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d6c003a00 msgr2=0x7f9d881e89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:39.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.797+0000 7f9d8ee00640 1 --2- 172.21.15.39:0/4128380388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d6c003a00 0x7f9d881e89f0 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7f9d7800c3b0 tx=0x7f9d78006d70 comp rx=0 tx=0).stop 2024-09-21T07:52:39.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.797+0000 7f9d8ee00640 1 -- 172.21.15.39:0/4128380388 shutdown_connections 2024-09-21T07:52:39.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.797+0000 7f9d8ee00640 1 --2- 172.21.15.39:0/4128380388 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9d64078b90 0x7f9d6407b0a0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:39.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.797+0000 7f9d8ee00640 1 --2- 172.21.15.39:0/4128380388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d8813fe90 0x7f9d881f6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:39.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.797+0000 7f9d8ee00640 1 --2- 172.21.15.39:0/4128380388 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9d881e8f30 0x7f9d8813f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:39.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.797+0000 7f9d8ee00640 1 --2- 172.21.15.39:0/4128380388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d6c003a00 0x7f9d881e89f0 unknown :-1 s=CLOSED pgs=2556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:39.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.797+0000 7f9d8ee00640 1 -- 172.21.15.39:0/4128380388 >> 172.21.15.39:0/4128380388 conn(0x7f9d8806cfc0 msgr2=0x7f9d8806f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:39.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.797+0000 7f9d8ee00640 1 -- 172.21.15.39:0/4128380388 shutdown_connections 2024-09-21T07:52:39.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:39.797+0000 7f9d8ee00640 1 -- 172.21.15.39:0/4128380388 wait complete. 2024-09-21T07:52:39.821 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T07:52:39.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:39.821 DEBUG:teuthology.orchestra.run.smithi039:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T07:52:39.880 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T07:52:39.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:39.881 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:52:39.917 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:52:39.917 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:52:39.993 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:52:39.994 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:52:40.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.260+0000 7fc8c7400640 1 -- 172.21.15.39:0/3066103541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8c015a460 msgr2=0x7fc8c006fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:40.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.260+0000 7fc8c7400640 1 --2- 172.21.15.39:0/3066103541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8c015a460 0x7fc8c006fa30 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0005fd0 tx=0x7fc8b002a540 comp rx=0 tx=0).stop 2024-09-21T07:52:40.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.260+0000 7fc8c7400640 1 -- 172.21.15.39:0/3066103541 shutdown_connections 2024-09-21T07:52:40.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.260+0000 7fc8c7400640 1 --2- 172.21.15.39:0/3066103541 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc8c00cd780 0x7fc8c00cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:40.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.260+0000 7fc8c7400640 1 --2- 172.21.15.39:0/3066103541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8c015a460 0x7fc8c006fa30 unknown :-1 s=CLOSED pgs=2557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:40.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.260+0000 7fc8c7400640 1 -- 172.21.15.39:0/3066103541 >> 172.21.15.39:0/3066103541 conn(0x7fc8c006cfc0 msgr2=0x7fc8c006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:40.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.260+0000 7fc8c7400640 1 -- 172.21.15.39:0/3066103541 shutdown_connections 2024-09-21T07:52:40.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.260+0000 7fc8c7400640 1 -- 172.21.15.39:0/3066103541 wait complete. 2024-09-21T07:52:40.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.262+0000 7fc8c7400640 1 Processor -- start 2024-09-21T07:52:40.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.263+0000 7fc8c7400640 1 -- start start 2024-09-21T07:52:40.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.264+0000 7fc8c7400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8c00cd780 0x7fc8c00aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:40.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.264+0000 7fc8c7400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc8c00d06c0 0x7fc8c013ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:40.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.264+0000 7fc8c7400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8c015a460 0x7fc8c01f6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:40.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.264+0000 7fc8c7400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c015d5d0 con 0x7fc8c00cd780 2024-09-21T07:52:40.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.264+0000 7fc8c7400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c015d450 con 0x7fc8c015a460 2024-09-21T07:52:40.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.264+0000 7fc8c7400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc8c015d750 con 0x7fc8c00d06c0 2024-09-21T07:52:40.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.264+0000 7fc8c4a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8c00cd780 0x7fc8c00aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:40.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.264+0000 7fc8c4a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8c00cd780 0x7fc8c00aaa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60718/0 (socket says 172.21.15.39:60718) 2024-09-21T07:52:40.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.264+0000 7fc8c4a00640 1 -- 172.21.15.39:0/1532919580 learned_addr learned my addr 172.21.15.39:0/1532919580 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:40.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.265+0000 7fc8b7e00640 1 --2- 172.21.15.39:0/1532919580 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc8c00d06c0 0x7fc8c013ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:40.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.265+0000 7fc8c5400640 1 --2- 172.21.15.39:0/1532919580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8c015a460 0x7fc8c01f6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:40.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.265+0000 7fc8c4a00640 1 -- 172.21.15.39:0/1532919580 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc8c00d06c0 msgr2=0x7fc8c013ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:40.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.265+0000 7fc8c4a00640 1 --2- 172.21.15.39:0/1532919580 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc8c00d06c0 0x7fc8c013ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:40.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.265+0000 7fc8c4a00640 1 -- 172.21.15.39:0/1532919580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8c015a460 msgr2=0x7fc8c01f6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:40.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.265+0000 7fc8c4a00640 1 --2- 172.21.15.39:0/1532919580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8c015a460 0x7fc8c01f6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:40.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.265+0000 7fc8c4a00640 1 -- 172.21.15.39:0/1532919580 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c00ab3a0 con 0x7fc8c00cd780 2024-09-21T07:52:40.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.265+0000 7fc8c5400640 1 --2- 172.21.15.39:0/1532919580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8c015a460 0x7fc8c01f6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:52:40.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.265+0000 7fc8c4a00640 1 --2- 172.21.15.39:0/1532919580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8c00cd780 0x7fc8c00aaa70 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0034040 tx=0x7fc8b00038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:40.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.265+0000 7fc8bd600640 1 -- 172.21.15.39:0/1532919580 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b003e030 con 0x7fc8c00cd780 2024-09-21T07:52:40.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.266+0000 7fc8bd600640 1 -- 172.21.15.39:0/1532919580 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8b0003d90 con 0x7fc8c00cd780 2024-09-21T07:52:40.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.266+0000 7fc8c7400640 1 -- 172.21.15.39:0/1532919580 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8c00ab680 con 0x7fc8c00cd780 2024-09-21T07:52:40.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.266+0000 7fc8bd600640 1 -- 172.21.15.39:0/1532919580 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0035040 con 0x7fc8c00cd780 2024-09-21T07:52:40.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.266+0000 7fc8c7400640 1 -- 172.21.15.39:0/1532919580 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8c00abc40 con 0x7fc8c00cd780 2024-09-21T07:52:40.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.267+0000 7fc8c7400640 1 -- 172.21.15.39:0/1532919580 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8c015ed80 con 0x7fc8c00cd780 2024-09-21T07:52:40.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.268+0000 7fc8bd600640 1 -- 172.21.15.39:0/1532919580 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc8b0041030 con 0x7fc8c00cd780 2024-09-21T07:52:40.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.270+0000 7fc8bd600640 1 --2- 172.21.15.39:0/1532919580 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc89c078b90 0x7fc89c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:40.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.270+0000 7fc8bd600640 1 -- 172.21.15.39:0/1532919580 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(247..247 src has 1..247) ==== 7605+0+0 (secure 0 0 0) 0x7fc8b00bbf40 con 0x7fc8c00cd780 2024-09-21T07:52:40.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.270+0000 7fc8b7e00640 1 --2- 172.21.15.39:0/1532919580 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc89c078b90 0x7fc89c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:40.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.271+0000 7fc8b7e00640 1 --2- 172.21.15.39:0/1532919580 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc89c078b90 0x7fc89c07b0a0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8005fd0 tx=0x7fc8a8005850 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:40.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.274+0000 7fc8bd600640 1 -- 172.21.15.39:0/1532919580 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc8b0083ca0 con 0x7fc8c00cd780 2024-09-21T07:52:40.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.510+0000 7fc8c7400640 1 -- 172.21.15.39:0/1532919580 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc8c0159640 con 0x7fc8c00cd780 2024-09-21T07:52:40.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.510+0000 7fc8bd600640 1 -- 172.21.15.39:0/1532919580 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v281) ==== 53+0+73 (secure 0 0 0) 0x7fc8b00835a0 con 0x7fc8c00cd780 2024-09-21T07:52:40.513 INFO:teuthology.orchestra.run.smithi039.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:52:40.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.513+0000 7fc8c7400640 1 -- 172.21.15.39:0/1532919580 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc89c078b90 msgr2=0x7fc89c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:40.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.513+0000 7fc8c7400640 1 --2- 172.21.15.39:0/1532919580 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc89c078b90 0x7fc89c07b0a0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8005fd0 tx=0x7fc8a8005850 comp rx=0 tx=0).stop 2024-09-21T07:52:40.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.513+0000 7fc8c7400640 1 -- 172.21.15.39:0/1532919580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8c00cd780 msgr2=0x7fc8c00aaa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:40.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.513+0000 7fc8c7400640 1 --2- 172.21.15.39:0/1532919580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8c00cd780 0x7fc8c00aaa70 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0034040 tx=0x7fc8b00038a0 comp rx=0 tx=0).stop 2024-09-21T07:52:40.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.513+0000 7fc8c7400640 1 -- 172.21.15.39:0/1532919580 shutdown_connections 2024-09-21T07:52:40.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.513+0000 7fc8c7400640 1 --2- 172.21.15.39:0/1532919580 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc89c078b90 0x7fc89c07b0a0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:40.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.513+0000 7fc8c7400640 1 --2- 172.21.15.39:0/1532919580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8c015a460 0x7fc8c01f6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:40.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.513+0000 7fc8c7400640 1 --2- 172.21.15.39:0/1532919580 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc8c00d06c0 0x7fc8c013ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:40.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.513+0000 7fc8c7400640 1 --2- 172.21.15.39:0/1532919580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8c00cd780 0x7fc8c00aaa70 unknown :-1 s=CLOSED pgs=2558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:40.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.513+0000 7fc8c7400640 1 -- 172.21.15.39:0/1532919580 >> 172.21.15.39:0/1532919580 conn(0x7fc8c006cfc0 msgr2=0x7fc8c00d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:40.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.513+0000 7fc8c7400640 1 -- 172.21.15.39:0/1532919580 shutdown_connections 2024-09-21T07:52:40.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40.513+0000 7fc8c7400640 1 -- 172.21.15.39:0/1532919580 wait complete. 2024-09-21T07:52:40.536 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:52:40.536 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:52:40.536 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi039.front.sepia.ceph.com 2024-09-21T07:52:40.536 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T07:52:40.537 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:52:40.537 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:52:40.537 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:52:40.537 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:52:40.537 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:52:40.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:40.537 DEBUG:teuthology.orchestra.run.smithi039:> ip addr 2024-09-21T07:52:40.573 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:52:40.574 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:52:40.574 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:52:40.574 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:52:40.574 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2024-09-21T07:52:40.574 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:52:40.574 INFO:teuthology.orchestra.run.smithi039.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:52:40.574 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:52:40.574 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f0 2024-09-21T07:52:40.574 INFO:teuthology.orchestra.run.smithi039.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:52:40.574 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c7 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:52:40.574 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f1 2024-09-21T07:52:40.574 INFO:teuthology.orchestra.run.smithi039.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:52:40.575 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:70 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:52:40.575 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f0 2024-09-21T07:52:40.575 INFO:teuthology.orchestra.run.smithi039.stdout: inet 172.21.15.39/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T07:52:40.575 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft 41558sec preferred_lft 41558sec 2024-09-21T07:52:40.575 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec4:7aff:febd:1370/64 scope link 2024-09-21T07:52:40.575 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:52:40.575 INFO:teuthology.orchestra.run.smithi039.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:52:40.575 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:71 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:52:40.575 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f1 2024-09-21T07:52:40.575 INFO:teuthology.orchestra.run.smithi039.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:52:40.575 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff 2024-09-21T07:52:40.575 INFO:teuthology.orchestra.run.smithi039.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:52:40.575 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:52:40.575 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::b079:ddff:fe0e:6333/64 scope link 2024-09-21T07:52:40.575 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:52:40.576 INFO:teuthology.orchestra.run.smithi039.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:52:40.576 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:52:40.576 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::2ca7:98ff:fe8a:d0ed/64 scope link 2024-09-21T07:52:40.576 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:52:40.576 INFO:teuthology.orchestra.run.smithi039.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:52:40.576 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether ee:10:53:01:0c:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:52:40.576 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec10:53ff:fe01:c44/64 scope link 2024-09-21T07:52:40.576 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:52:40.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:40.576 DEBUG:teuthology.orchestra.run.smithi039:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:52:40.668 INFO:teuthology.orchestra.run.smithi039.stdout:1 2024-09-21T07:52:40.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:40.672 DEBUG:teuthology.orchestra.run.smithi039:> route 2024-09-21T07:52:40.707 INFO:teuthology.orchestra.run.smithi039.stdout:Kernel IP routing table 2024-09-21T07:52:40.707 INFO:teuthology.orchestra.run.smithi039.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:52:40.707 INFO:teuthology.orchestra.run.smithi039.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T07:52:40.707 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T07:52:40.708 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:52:40.708 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:52:40.708 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-21T07:52:40.708 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T07:52:40.708 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T07:52:40.708 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T07:52:40.708 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-21T07:52:40.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:52:40.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:52:40.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:40.905 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-09-21T07:52:40.939 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:52:40.939 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T07:52:40.940 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:52:40.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:40.940 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-09-21T07:52:41.037 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:52:41.037 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T07:52:41.071 INFO:teuthology.orchestra.run.smithi039.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:52:41.072 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:52:41.072 DEBUG:teuthology.orchestra.run.smithi039:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:52:41.147 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-21T07:52:41.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:41.147 DEBUG:teuthology.orchestra.run.smithi039:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:52:41.345 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:52:41.346 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:52:41.380 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:52:41.380 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:52:41.380 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: resolved to: "172.21.15.39:3300,172.21.15.148:3300,172.21.15.148:3301" 2024-09-21T07:52:41.380 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: trying mount with new device syntax: 0@7ab9b04a-f88d-49aa-a493-e5c076637ffb.cephfs=/ 2024-09-21T07:52:41.381 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.39:3300/172.21.15.148:3300/172.21.15.148:3301" will pass to kernel 2024-09-21T07:52:41.382 INFO:teuthology.orchestra.run.smithi039.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T07:52:41.382 INFO:teuthology.orchestra.run.smithi039.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:52:41.382 INFO:teuthology.orchestra.run.smithi039.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:52:41.382 INFO:teuthology.orchestra.run.smithi039.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:52:41.382 INFO:teuthology.orchestra.run.smithi039.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:52:41.385 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:52:41.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:41.385 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:52:41.442 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:52:41.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:52:41.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:52:41.536 INFO:teuthology.orchestra.run.smithi039.stdout:client9242 2024-09-21T07:52:41.536 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:52:41.536 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:52:41.572 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.13:0 flush journal 2024-09-21T07:52:41.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.840+0000 7fc811800640 1 -- 172.21.15.39:0/528754036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc80c160940 msgr2=0x7fc80c0cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:41.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.840+0000 7fc811800640 1 --2- 172.21.15.39:0/528754036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc80c160940 0x7fc80c0cc430 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4005fd0 tx=0x7fc7f402a5f0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.840+0000 7fc811800640 1 -- 172.21.15.39:0/528754036 shutdown_connections 2024-09-21T07:52:41.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.840+0000 7fc811800640 1 --2- 172.21.15.39:0/528754036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc80c160940 0x7fc80c0cc430 unknown :-1 s=CLOSED pgs=2560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.840+0000 7fc811800640 1 -- 172.21.15.39:0/528754036 >> 172.21.15.39:0/528754036 conn(0x7fc80c0b25f0 msgr2=0x7fc80c0d01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:41.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.840+0000 7fc811800640 1 -- 172.21.15.39:0/528754036 shutdown_connections 2024-09-21T07:52:41.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.840+0000 7fc811800640 1 -- 172.21.15.39:0/528754036 wait complete. 2024-09-21T07:52:41.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.842+0000 7fc811800640 1 Processor -- start 2024-09-21T07:52:41.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.842+0000 7fc811800640 1 -- start start 2024-09-21T07:52:41.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.844+0000 7fc811800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc80c0af690 0x7fc80c1f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:41.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.844+0000 7fc811800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc80c156d60 0x7fc80c0c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:41.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.844+0000 7fc811800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc80c1f61e0 0x7fc80c0ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:41.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.844+0000 7fc811800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c0ae150 con 0x7fc80c1f61e0 2024-09-21T07:52:41.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.844+0000 7fc811800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c0adfd0 con 0x7fc80c0af690 2024-09-21T07:52:41.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.844+0000 7fc811800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc80c0ae2d0 con 0x7fc80c156d60 2024-09-21T07:52:41.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.844+0000 7fc810e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc80c1f61e0 0x7fc80c0ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:41.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.844+0000 7fc810e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc80c1f61e0 0x7fc80c0ce9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60750/0 (socket says 172.21.15.39:60750) 2024-09-21T07:52:41.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.844+0000 7fc810e00640 1 -- 172.21.15.39:0/1956003294 learned_addr learned my addr 172.21.15.39:0/1956003294 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:41.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.844+0000 7fc803400640 1 --2- 172.21.15.39:0/1956003294 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc80c156d60 0x7fc80c0c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:41.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.844+0000 7fc80be00640 1 --2- 172.21.15.39:0/1956003294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc80c0af690 0x7fc80c1f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:41.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.844+0000 7fc810e00640 1 -- 172.21.15.39:0/1956003294 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc80c156d60 msgr2=0x7fc80c0c97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:41.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.844+0000 7fc810e00640 1 --2- 172.21.15.39:0/1956003294 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc80c156d60 0x7fc80c0c97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.844+0000 7fc810e00640 1 -- 172.21.15.39:0/1956003294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc80c0af690 msgr2=0x7fc80c1f5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:41.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.844+0000 7fc810e00640 1 --2- 172.21.15.39:0/1956003294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc80c0af690 0x7fc80c1f5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.844+0000 7fc810e00640 1 -- 172.21.15.39:0/1956003294 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc80c1f8b80 con 0x7fc80c1f61e0 2024-09-21T07:52:41.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.845+0000 7fc810e00640 1 --2- 172.21.15.39:0/1956003294 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc80c1f61e0 0x7fc80c0ce9c0 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 crypto rx=0x7fc80400c700 tx=0x7fc80400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:41.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.845+0000 7fc808c00640 1 -- 172.21.15.39:0/1956003294 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc804003b70 con 0x7fc80c1f61e0 2024-09-21T07:52:41.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.845+0000 7fc808c00640 1 -- 172.21.15.39:0/1956003294 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc804003d10 con 0x7fc80c1f61e0 2024-09-21T07:52:41.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.845+0000 7fc811800640 1 -- 172.21.15.39:0/1956003294 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc80c1f8e60 con 0x7fc80c1f61e0 2024-09-21T07:52:41.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.845+0000 7fc808c00640 1 -- 172.21.15.39:0/1956003294 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80400aab0 con 0x7fc80c1f61e0 2024-09-21T07:52:41.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.845+0000 7fc811800640 1 -- 172.21.15.39:0/1956003294 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc80c1f9420 con 0x7fc80c1f61e0 2024-09-21T07:52:41.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.847+0000 7fc808c00640 1 -- 172.21.15.39:0/1956003294 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc80400ac50 con 0x7fc80c1f61e0 2024-09-21T07:52:41.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.848+0000 7fc808c00640 1 --2- 172.21.15.39:0/1956003294 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc7e8078bf0 0x7fc7e807b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:41.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.849+0000 7fc808c00640 1 -- 172.21.15.39:0/1956003294 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(247..247 src has 1..247) ==== 7605+0+0 (secure 0 0 0) 0x7fc80409b380 con 0x7fc80c1f61e0 2024-09-21T07:52:41.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.849+0000 7fc819e20740 1 Processor -- start 2024-09-21T07:52:41.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.849+0000 7fc80be00640 1 --2- 172.21.15.39:0/1956003294 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc7e8078bf0 0x7fc7e807b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:41.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.849+0000 7fc819e20740 1 -- start start 2024-09-21T07:52:41.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.850+0000 7fc80be00640 1 --2- 172.21.15.39:0/1956003294 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc7e8078bf0 0x7fc7e807b100 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc000f60 tx=0x7fc7fc009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:41.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.850+0000 7fc819e20740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603f234c390 0x5603f273a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:41.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.850+0000 7fc819e20740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603f23234d0 0x5603f271d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:41.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.850+0000 7fc819e20740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603f232b6b0 0x5603f271d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:41.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.850+0000 7fc819e20740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5603f26531c0 con 0x5603f23234d0 2024-09-21T07:52:41.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.850+0000 7fc819e20740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5603f2666870 con 0x5603f232b6b0 2024-09-21T07:52:41.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.850+0000 7fc819e20740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x5603f2655ca0 con 0x5603f234c390 2024-09-21T07:52:41.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.851+0000 7fc803400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603f23234d0 0x5603f271d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:41.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.851+0000 7fc803400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603f23234d0 0x5603f271d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60752/0 (socket says 172.21.15.39:60752) 2024-09-21T07:52:41.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.851+0000 7fc803400640 1 -- 172.21.15.39:0/4217087774 learned_addr learned my addr 172.21.15.39:0/4217087774 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:41.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.851+0000 7fc810e00640 1 --2- 172.21.15.39:0/4217087774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603f232b6b0 0x5603f271d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:41.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.851+0000 7fc80be00640 1 --2- 172.21.15.39:0/4217087774 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603f234c390 0x5603f273a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:41.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.851+0000 7fc803400640 1 -- 172.21.15.39:0/4217087774 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603f234c390 msgr2=0x5603f273a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:52:41.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.851+0000 7fc803400640 1 --2- 172.21.15.39:0/4217087774 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603f234c390 0x5603f273a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.851+0000 7fc803400640 1 -- 172.21.15.39:0/4217087774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603f232b6b0 msgr2=0x5603f271d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:52:41.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.851+0000 7fc803400640 1 --2- 172.21.15.39:0/4217087774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603f232b6b0 0x5603f271d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.851+0000 7fc803400640 1 -- 172.21.15.39:0/4217087774 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603f242e7d0 con 0x5603f23234d0 2024-09-21T07:52:41.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.851+0000 7fc80be00640 1 --2- 172.21.15.39:0/4217087774 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603f234c390 0x5603f273a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:52:41.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.851+0000 7fc803400640 1 --2- 172.21.15.39:0/4217087774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603f23234d0 0x5603f271d400 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4005fd0 tx=0x7fc7f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=badc8698dca20017 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:41.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.852+0000 7fc800c00640 1 -- 172.21.15.39:0/4217087774 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4040030 con 0x5603f23234d0 2024-09-21T07:52:41.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.852+0000 7fc800c00640 1 -- 172.21.15.39:0/4217087774 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7f4005580 con 0x5603f23234d0 2024-09-21T07:52:41.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.852+0000 7fc800c00640 1 -- 172.21.15.39:0/4217087774 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4035040 con 0x5603f23234d0 2024-09-21T07:52:41.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.852+0000 7fc819e20740 1 -- 172.21.15.39:0/4217087774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603f23234d0 msgr2=0x5603f271d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:41.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.852+0000 7fc819e20740 1 --2- 172.21.15.39:0/4217087774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603f23234d0 0x5603f271d400 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4005fd0 tx=0x7fc7f40046c0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.853+0000 7fc819e20740 1 -- 172.21.15.39:0/4217087774 shutdown_connections 2024-09-21T07:52:41.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.853+0000 7fc819e20740 1 --2- 172.21.15.39:0/4217087774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603f232b6b0 0x5603f271d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.853+0000 7fc819e20740 1 --2- 172.21.15.39:0/4217087774 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603f23234d0 0x5603f271d400 unknown :-1 s=CLOSED pgs=2562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.853+0000 7fc819e20740 1 --2- 172.21.15.39:0/4217087774 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603f234c390 0x5603f273a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.853+0000 7fc819e20740 1 -- 172.21.15.39:0/4217087774 >> 172.21.15.39:0/4217087774 conn(0x5603f23404e0 msgr2=0x5603f26f3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:41.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.853+0000 7fc819e20740 1 -- 172.21.15.39:0/4217087774 shutdown_connections 2024-09-21T07:52:41.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.853+0000 7fc819e20740 1 -- 172.21.15.39:0/4217087774 wait complete. 2024-09-21T07:52:41.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.854+0000 7fc819e20740 1 -- start start 2024-09-21T07:52:41.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.854+0000 7fc819e20740 1 Processor -- start 2024-09-21T07:52:41.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.855+0000 7fc819e20740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603f23234d0 0x5603f2732c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:41.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.855+0000 7fc819e20740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603f232b6b0 0x5603f2748660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:41.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.855+0000 7fc819e20740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603f234c390 0x5603f2716a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:41.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.855+0000 7fc819e20740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5603f25fc410 con 0x5603f23234d0 2024-09-21T07:52:41.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.855+0000 7fc819e20740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5603f2659ab0 con 0x5603f234c390 2024-09-21T07:52:41.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.855+0000 7fc819e20740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x5603f23c6740 con 0x5603f232b6b0 2024-09-21T07:52:41.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.855+0000 7fc819e20740 10 client.0 _ll_register_callbacks cb 0x5603f25d9360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:52:41.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.856+0000 7fc80be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603f23234d0 0x5603f2732c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:41.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.856+0000 7fc80be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603f23234d0 0x5603f2732c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60756/0 (socket says 172.21.15.39:60756) 2024-09-21T07:52:41.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.856+0000 7fc80be00640 1 -- 172.21.15.39:0/2476398459 learned_addr learned my addr 172.21.15.39:0/2476398459 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:41.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.856+0000 7fc803400640 1 --2- 172.21.15.39:0/2476398459 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603f232b6b0 0x5603f2748660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:41.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.856+0000 7fc810e00640 1 --2- 172.21.15.39:0/2476398459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603f234c390 0x5603f2716a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:41.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.856+0000 7fc80be00640 1 -- 172.21.15.39:0/2476398459 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603f232b6b0 msgr2=0x5603f2748660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:52:41.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.856+0000 7fc80be00640 1 --2- 172.21.15.39:0/2476398459 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603f232b6b0 0x5603f2748660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.856+0000 7fc80be00640 1 -- 172.21.15.39:0/2476398459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603f234c390 msgr2=0x5603f2716a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:52:41.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.856+0000 7fc80be00640 1 --2- 172.21.15.39:0/2476398459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603f234c390 0x5603f2716a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.856+0000 7fc80be00640 1 -- 172.21.15.39:0/2476398459 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603f23b7970 con 0x5603f23234d0 2024-09-21T07:52:41.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.857+0000 7fc80be00640 1 --2- 172.21.15.39:0/2476398459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603f23234d0 0x5603f2732c80 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc003950 tx=0x7fc7fc000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb79bbd7fc53a886 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:41.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.857+0000 7fc7f9600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:52:41.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.857+0000 7fc7f9600640 1 -- 172.21.15.39:0/2476398459 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc027060 con 0x5603f23234d0 2024-09-21T07:52:41.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.857+0000 7fc819e20740 1 -- 172.21.15.39:0/2476398459 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7f4005580 con 0x5603f23234d0 2024-09-21T07:52:41.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.857+0000 7fc7f9600640 1 -- 172.21.15.39:0/2476398459 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(247..247 src has 1..247) ==== 7605+0+0 (secure 0 0 0) 0x7fc7fc010040 con 0x5603f23234d0 2024-09-21T07:52:41.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.857+0000 7fc7f9600640 1 -- 172.21.15.39:0/2476398459 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=281) ==== 24+0+0 (secure 0 0 0) 0x7fc7fc02d020 con 0x5603f23234d0 2024-09-21T07:52:41.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.857+0000 7fc819e20740 10 client.11658 fetch_fsmap learned FSMap version 281 2024-09-21T07:52:41.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.857+0000 7fc819e20740 1 -- 172.21.15.39:0/2476398459 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=281,monmap=2+}) -- 0x5603f23956a0 con 0x5603f23234d0 2024-09-21T07:52:41.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.857+0000 7fc7f9600640 1 -- 172.21.15.39:0/2476398459 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7f4005580 con 0x5603f23234d0 2024-09-21T07:52:41.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.858+0000 7fc7f9600640 1 -- 172.21.15.39:0/2476398459 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 281) ==== 3542+0+0 (secure 0 0 0) 0x7fc7fc00ec40 con 0x5603f23234d0 2024-09-21T07:52:41.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.858+0000 7fc819e20740 10 client.11658 fetch_fsmap finished waiting for FSMap version 281 2024-09-21T07:52:41.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.858+0000 7fc819e20740 10 client.11658 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.b 2024-09-21T07:52:41.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.858+0000 7fc819e20740 20 client.11658 populate_metadata read hostname 'smithi039' 2024-09-21T07:52:41.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.858+0000 7fc819e20740 1 --2- 172.21.15.39:0/2476398459 >> [v2:172.21.15.148:6834/3381818843,v1:172.21.15.148:6836/3381818843] conn(0x5603f230af60 0x5603f2748ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:41.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.858+0000 7fc819e20740 4 client.11658 mds_command: new command op to 9967 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:52:41.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.858+0000 7fc819e20740 1 -- 172.21.15.39:0/2476398459 --> [v2:172.21.15.148:6834/3381818843,v1:172.21.15.148:6836/3381818843] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603f26687c0 con 0x5603f230af60 2024-09-21T07:52:41.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.858+0000 7fc803400640 1 --2- 172.21.15.39:0/2476398459 >> [v2:172.21.15.148:6834/3381818843,v1:172.21.15.148:6836/3381818843] conn(0x5603f230af60 0x5603f2748ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:41.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.859+0000 7fc803400640 1 --2- 172.21.15.39:0/2476398459 >> [v2:172.21.15.148:6834/3381818843,v1:172.21.15.148:6836/3381818843] conn(0x5603f230af60 0x5603f2748ba0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83831bd01b23cc91 server_cookie=f7866d2356b886cc in_seq=0 out_seq=0 2024-09-21T07:52:41.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.859+0000 7fc7f9600640 10 client.11658 ms_handle_connect on v2:172.21.15.148:6834/3381818843 2024-09-21T07:52:41.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.860+0000 7fc7f9600640 1 -- 172.21.15.39:0/2476398459 <== mds.0 v2:172.21.15.148:6834/3381818843 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5603f26687c0 con 0x5603f230af60 2024-09-21T07:52:41.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.860+0000 7fc7f9600640 10 client.11658 handle_command_reply: tid=0 2024-09-21T07:52:41.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.860+0000 7fc819e20740 1 client.11658 shutdown 2024-09-21T07:52:41.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.860+0000 7fc819e20740 1 -- 172.21.15.39:0/2476398459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603f23234d0 msgr2=0x5603f2732c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:41.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.860+0000 7fc819e20740 1 --2- 172.21.15.39:0/2476398459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603f23234d0 0x5603f2732c80 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc003950 tx=0x7fc7fc000ed0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.860+0000 7fc819e20740 1 -- 172.21.15.39:0/2476398459 shutdown_connections 2024-09-21T07:52:41.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.860+0000 7fc819e20740 1 --2- 172.21.15.39:0/2476398459 >> [v2:172.21.15.148:6834/3381818843,v1:172.21.15.148:6836/3381818843] conn(0x5603f230af60 0x5603f2748ba0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.860+0000 7fc819e20740 1 --2- 172.21.15.39:0/2476398459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603f234c390 0x5603f2716a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.860+0000 7fc819e20740 1 --2- 172.21.15.39:0/2476398459 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603f232b6b0 0x5603f2748660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.860+0000 7fc819e20740 1 --2- 172.21.15.39:0/2476398459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603f23234d0 0x5603f2732c80 unknown :-1 s=CLOSED pgs=2563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.860+0000 7fc7f9600640 0 client.11658 ms_handle_reset on v2:172.21.15.148:6834/3381818843 2024-09-21T07:52:41.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.860+0000 7fc819e20740 1 -- 172.21.15.39:0/2476398459 >> 172.21.15.39:0/2476398459 conn(0x5603f23404e0 msgr2=0x5603f26ec830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:41.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.860+0000 7fc819e20740 1 -- 172.21.15.39:0/2476398459 shutdown_connections 2024-09-21T07:52:41.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.860+0000 7fc819e20740 1 -- 172.21.15.39:0/2476398459 wait complete. 2024-09-21T07:52:41.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.861+0000 7fc819e20740 20 client.11658 trim_cache size 0 max 16384 2024-09-21T07:52:41.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.876+0000 7fc819e20740 1 Processor -- start 2024-09-21T07:52:41.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.876+0000 7fc819e20740 1 -- start start 2024-09-21T07:52:41.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.878+0000 7fc819e20740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603f23234d0 0x5603f274a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:41.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.878+0000 7fc819e20740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603f232b6b0 0x5603f26ee900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:41.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.878+0000 7fc819e20740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603f234c390 0x5603f27465f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:41.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.878+0000 7fc819e20740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c204d20 con 0x5603f234c390 2024-09-21T07:52:41.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.878+0000 7fc819e20740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c203f80 con 0x5603f23234d0 2024-09-21T07:52:41.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.878+0000 7fc819e20740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc80c204650 con 0x5603f232b6b0 2024-09-21T07:52:41.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.878+0000 7fc810e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603f234c390 0x5603f27465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:41.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.878+0000 7fc810e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603f234c390 0x5603f27465f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60762/0 (socket says 172.21.15.39:60762) 2024-09-21T07:52:41.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.878+0000 7fc810e00640 1 -- 172.21.15.39:0/589359038 learned_addr learned my addr 172.21.15.39:0/589359038 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:41.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.878+0000 7fc810e00640 1 -- 172.21.15.39:0/589359038 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603f232b6b0 msgr2=0x5603f26ee900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:52:41.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.878+0000 7fc80be00640 1 --2- 172.21.15.39:0/589359038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603f23234d0 0x5603f274a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:41.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.878+0000 7fc803400640 1 --2- 172.21.15.39:0/589359038 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603f232b6b0 0x5603f26ee900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:41.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.878+0000 7fc810e00640 1 --2- 172.21.15.39:0/589359038 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603f232b6b0 0x5603f26ee900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.878+0000 7fc810e00640 1 -- 172.21.15.39:0/589359038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603f23234d0 msgr2=0x5603f274a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:52:41.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.878+0000 7fc810e00640 1 --2- 172.21.15.39:0/589359038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603f23234d0 0x5603f274a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.878+0000 7fc810e00640 1 -- 172.21.15.39:0/589359038 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603f26ec830 con 0x5603f234c390 2024-09-21T07:52:41.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.878+0000 7fc80be00640 1 --2- 172.21.15.39:0/589359038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603f23234d0 0x5603f274a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:52:41.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.879+0000 7fc803400640 1 --2- 172.21.15.39:0/589359038 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603f232b6b0 0x5603f26ee900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:52:41.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.879+0000 7fc810e00640 1 --2- 172.21.15.39:0/589359038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603f234c390 0x5603f27465f0 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7fc80400a430 tx=0x7fc804005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b295989c2c58b577 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:41.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.879+0000 7fc7fbe00640 1 -- 172.21.15.39:0/589359038 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8040628b0 con 0x5603f234c390 2024-09-21T07:52:41.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.879+0000 7fc7fbe00640 1 -- 172.21.15.39:0/589359038 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc804067390 con 0x5603f234c390 2024-09-21T07:52:41.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.879+0000 7fc7fbe00640 1 -- 172.21.15.39:0/589359038 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80406c2d0 con 0x5603f234c390 2024-09-21T07:52:41.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.880+0000 7fc819e20740 1 -- 172.21.15.39:0/589359038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603f234c390 msgr2=0x5603f27465f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:41.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.880+0000 7fc819e20740 1 --2- 172.21.15.39:0/589359038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603f234c390 0x5603f27465f0 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7fc80400a430 tx=0x7fc804005040 comp rx=0 tx=0).stop 2024-09-21T07:52:41.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.880+0000 7fc819e20740 1 -- 172.21.15.39:0/589359038 shutdown_connections 2024-09-21T07:52:41.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.880+0000 7fc819e20740 1 --2- 172.21.15.39:0/589359038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603f234c390 0x5603f27465f0 unknown :-1 s=CLOSED pgs=2564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.880+0000 7fc819e20740 1 --2- 172.21.15.39:0/589359038 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603f232b6b0 0x5603f26ee900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.880+0000 7fc819e20740 1 --2- 172.21.15.39:0/589359038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603f23234d0 0x5603f274a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.880+0000 7fc819e20740 1 -- 172.21.15.39:0/589359038 >> 172.21.15.39:0/589359038 conn(0x5603f23404e0 msgr2=0x5603f27315c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:41.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.880+0000 7fc819e20740 1 -- 172.21.15.39:0/589359038 shutdown_connections 2024-09-21T07:52:41.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.880+0000 7fc819e20740 1 -- 172.21.15.39:0/589359038 wait complete. 2024-09-21T07:52:41.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.881+0000 7fc819e20740 1 -- start start 2024-09-21T07:52:41.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.881+0000 7fc819e20740 1 Processor -- start 2024-09-21T07:52:41.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.883+0000 7fc819e20740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603f23234d0 0x5603f27b7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:41.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.883+0000 7fc819e20740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603f232b6b0 0x5603f273a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:41.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.883+0000 7fc819e20740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603f234c390 0x5603f27338e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:41.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.883+0000 7fc819e20740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c202f70 con 0x5603f232b6b0 2024-09-21T07:52:41.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.883+0000 7fc819e20740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5603f26eee40 con 0x5603f23234d0 2024-09-21T07:52:41.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.883+0000 7fc819e20740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x5603f273b000 con 0x5603f234c390 2024-09-21T07:52:41.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.883+0000 7fc819e20740 10 client.0 _ll_register_callbacks cb 0x7fc7fc001360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:52:41.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.883+0000 7fc803400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603f232b6b0 0x5603f273a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:41.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.883+0000 7fc803400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603f232b6b0 0x5603f273a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60774/0 (socket says 172.21.15.39:60774) 2024-09-21T07:52:41.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.883+0000 7fc803400640 1 -- 172.21.15.39:0/468302456 learned_addr learned my addr 172.21.15.39:0/468302456 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:41.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.883+0000 7fc80be00640 1 --2- 172.21.15.39:0/468302456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603f23234d0 0x5603f27b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:41.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.883+0000 7fc810e00640 1 --2- 172.21.15.39:0/468302456 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603f234c390 0x5603f27338e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:41.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.883+0000 7fc803400640 1 -- 172.21.15.39:0/468302456 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603f234c390 msgr2=0x5603f27338e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:52:41.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.883+0000 7fc803400640 1 --2- 172.21.15.39:0/468302456 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603f234c390 0x5603f27338e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.883+0000 7fc803400640 1 -- 172.21.15.39:0/468302456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603f23234d0 msgr2=0x5603f27b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:52:41.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.883+0000 7fc803400640 1 --2- 172.21.15.39:0/468302456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603f23234d0 0x5603f27b7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.883+0000 7fc803400640 1 -- 172.21.15.39:0/468302456 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603f26f13f0 con 0x5603f232b6b0 2024-09-21T07:52:41.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.884+0000 7fc803400640 1 --2- 172.21.15.39:0/468302456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603f232b6b0 0x5603f273a630 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7fc7f40099f0 tx=0x7fc7f4005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e09140453e0848 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:41.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.884+0000 7fc7f9600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:52:41.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.884+0000 7fc7f9600640 1 -- 172.21.15.39:0/468302456 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4035040 con 0x5603f232b6b0 2024-09-21T07:52:41.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.884+0000 7fc7f9600640 1 -- 172.21.15.39:0/468302456 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(247..247 src has 1..247) ==== 7605+0+0 (secure 0 0 0) 0x7fc7f40035f0 con 0x5603f232b6b0 2024-09-21T07:52:41.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.884+0000 7fc819e20740 1 -- 172.21.15.39:0/468302456 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc804067390 con 0x5603f232b6b0 2024-09-21T07:52:41.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.884+0000 7fc7f9600640 1 -- 172.21.15.39:0/468302456 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=281) ==== 24+0+0 (secure 0 0 0) 0x7fc7f40037f0 con 0x5603f232b6b0 2024-09-21T07:52:41.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.884+0000 7fc819e20740 10 client.11664 fetch_fsmap learned FSMap version 281 2024-09-21T07:52:41.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.884+0000 7fc819e20740 1 -- 172.21.15.39:0/468302456 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=281,monmap=2+}) -- 0x5603f273ad40 con 0x5603f232b6b0 2024-09-21T07:52:41.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.884+0000 7fc7f9600640 1 -- 172.21.15.39:0/468302456 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc804067390 con 0x5603f232b6b0 2024-09-21T07:52:41.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.884+0000 7fc7f9600640 1 -- 172.21.15.39:0/468302456 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 281) ==== 3542+0+0 (secure 0 0 0) 0x7fc7f403a040 con 0x5603f232b6b0 2024-09-21T07:52:41.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.885+0000 7fc819e20740 10 client.11664 fetch_fsmap finished waiting for FSMap version 281 2024-09-21T07:52:41.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.885+0000 7fc819e20740 10 client.11664 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.b 2024-09-21T07:52:41.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.885+0000 7fc819e20740 20 client.11664 populate_metadata read hostname 'smithi039' 2024-09-21T07:52:41.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.885+0000 7fc819e20740 1 --2- 172.21.15.39:0/468302456 >> [v2:172.21.15.148:6834/3381818843,v1:172.21.15.148:6836/3381818843] conn(0x5603f26f2940 0x5603f27465f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:41.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.885+0000 7fc819e20740 4 client.11664 mds_command: new command op to 9967 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:52:41.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.885+0000 7fc819e20740 1 -- 172.21.15.39:0/468302456 --> [v2:172.21.15.148:6834/3381818843,v1:172.21.15.148:6836/3381818843] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5603f26ef580 con 0x5603f26f2940 2024-09-21T07:52:41.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.885+0000 7fc80be00640 1 --2- 172.21.15.39:0/468302456 >> [v2:172.21.15.148:6834/3381818843,v1:172.21.15.148:6836/3381818843] conn(0x5603f26f2940 0x5603f27465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:41.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.886+0000 7fc80be00640 1 --2- 172.21.15.39:0/468302456 >> [v2:172.21.15.148:6834/3381818843,v1:172.21.15.148:6836/3381818843] conn(0x5603f26f2940 0x5603f27465f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dc3b947226439e3 server_cookie=c5a1160b0edb5b63 in_seq=0 out_seq=0 2024-09-21T07:52:41.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.886+0000 7fc7f9600640 10 client.11664 ms_handle_connect on v2:172.21.15.148:6834/3381818843 2024-09-21T07:52:41.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.899+0000 7fc7f9600640 1 -- 172.21.15.39:0/468302456 <== mds.0 v2:172.21.15.148:6834/3381818843 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5603f26ef580 con 0x5603f26f2940 2024-09-21T07:52:41.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.899+0000 7fc7f9600640 10 client.11664 handle_command_reply: tid=0 2024-09-21T07:52:41.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.899+0000 7fc819e20740 1 client.11664 shutdown 2024-09-21T07:52:41.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.899+0000 7fc819e20740 1 -- 172.21.15.39:0/468302456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603f232b6b0 msgr2=0x5603f273a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:41.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.899+0000 7fc819e20740 1 --2- 172.21.15.39:0/468302456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603f232b6b0 0x5603f273a630 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7fc7f40099f0 tx=0x7fc7f4005ee0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.899+0000 7fc819e20740 1 -- 172.21.15.39:0/468302456 shutdown_connections 2024-09-21T07:52:41.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.899+0000 7fc819e20740 1 --2- 172.21.15.39:0/468302456 >> [v2:172.21.15.148:6834/3381818843,v1:172.21.15.148:6836/3381818843] conn(0x5603f26f2940 0x5603f27465f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.899+0000 7fc819e20740 1 --2- 172.21.15.39:0/468302456 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5603f234c390 0x5603f27338e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.899+0000 7fc819e20740 1 --2- 172.21.15.39:0/468302456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5603f232b6b0 0x5603f273a630 unknown :-1 s=CLOSED pgs=2565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.899+0000 7fc819e20740 1 --2- 172.21.15.39:0/468302456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5603f23234d0 0x5603f27b7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.899+0000 7fc7f9600640 0 client.11664 ms_handle_reset on v2:172.21.15.148:6834/3381818843 2024-09-21T07:52:41.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.899+0000 7fc819e20740 1 -- 172.21.15.39:0/468302456 >> 172.21.15.39:0/468302456 conn(0x5603f23404e0 msgr2=0x5603f2731630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:41.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.899+0000 7fc819e20740 1 -- 172.21.15.39:0/468302456 shutdown_connections 2024-09-21T07:52:41.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.899+0000 7fc819e20740 1 -- 172.21.15.39:0/468302456 wait complete. 2024-09-21T07:52:41.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.899+0000 7fc819e20740 20 client.11664 trim_cache size 0 max 16384 2024-09-21T07:52:41.903 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:52:41.903 INFO:teuthology.orchestra.run.smithi039.stdout: "message": "", 2024-09-21T07:52:41.903 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0 2024-09-21T07:52:41.903 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:52:41.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.900+0000 7fc811800640 1 -- 172.21.15.39:0/1956003294 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc7e8078bf0 msgr2=0x7fc7e807b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:41.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.900+0000 7fc811800640 1 --2- 172.21.15.39:0/1956003294 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc7e8078bf0 0x7fc7e807b100 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc000f60 tx=0x7fc7fc009040 comp rx=0 tx=0).stop 2024-09-21T07:52:41.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.900+0000 7fc811800640 1 -- 172.21.15.39:0/1956003294 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc80c1f61e0 msgr2=0x7fc80c0ce9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:41.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.900+0000 7fc811800640 1 --2- 172.21.15.39:0/1956003294 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc80c1f61e0 0x7fc80c0ce9c0 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 crypto rx=0x7fc80400c700 tx=0x7fc80400cbc0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.900+0000 7fc811800640 1 -- 172.21.15.39:0/1956003294 shutdown_connections 2024-09-21T07:52:41.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.900+0000 7fc811800640 1 --2- 172.21.15.39:0/1956003294 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc7e8078bf0 0x7fc7e807b100 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.900+0000 7fc811800640 1 --2- 172.21.15.39:0/1956003294 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc80c1f61e0 0x7fc80c0ce9c0 unknown :-1 s=CLOSED pgs=2561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.900+0000 7fc811800640 1 --2- 172.21.15.39:0/1956003294 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc80c156d60 0x7fc80c0c97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.900+0000 7fc811800640 1 --2- 172.21.15.39:0/1956003294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc80c0af690 0x7fc80c1f5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.900+0000 7fc811800640 1 -- 172.21.15.39:0/1956003294 >> 172.21.15.39:0/1956003294 conn(0x7fc80c0b25f0 msgr2=0x7fc80c0708d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:41.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.900+0000 7fc811800640 1 -- 172.21.15.39:0/1956003294 shutdown_connections 2024-09-21T07:52:41.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:41.900+0000 7fc811800640 1 -- 172.21.15.39:0/1956003294 wait complete. 2024-09-21T07:52:41.928 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:52:42.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.152+0000 7f1ae2400640 1 -- 172.21.15.39:0/3743323204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1adc0d0f20 msgr2=0x7f1adc15c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:42.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.152+0000 7f1ae2400640 1 --2- 172.21.15.39:0/3743323204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1adc0d0f20 0x7f1adc15c830 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0009830 tx=0x7f1ad002a5b0 comp rx=0 tx=0).stop 2024-09-21T07:52:42.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.152+0000 7f1ae2400640 1 -- 172.21.15.39:0/3743323204 shutdown_connections 2024-09-21T07:52:42.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.152+0000 7f1ae2400640 1 --2- 172.21.15.39:0/3743323204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1adc0d0f20 0x7f1adc15c830 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:42.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.152+0000 7f1ae2400640 1 -- 172.21.15.39:0/3743323204 >> 172.21.15.39:0/3743323204 conn(0x7f1adc0b2510 msgr2=0x7f1adc0d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:42.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.152+0000 7f1ae2400640 1 -- 172.21.15.39:0/3743323204 shutdown_connections 2024-09-21T07:52:42.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.152+0000 7f1ae2400640 1 -- 172.21.15.39:0/3743323204 wait complete. 2024-09-21T07:52:42.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.154+0000 7f1ae2400640 1 Processor -- start 2024-09-21T07:52:42.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.154+0000 7f1ae2400640 1 -- start start 2024-09-21T07:52:42.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.156+0000 7f1ae2400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1adc0cb960 0x7f1adc1f6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:42.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.156+0000 7f1ae2400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1adc0cbd70 0x7f1adc1582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:42.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.156+0000 7f1ae2400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1adc0d0f20 0x7f1adc1f5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:42.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.156+0000 7f1ae2400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1adc0b4780 con 0x7f1adc0cbd70 2024-09-21T07:52:42.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.156+0000 7f1ae2400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1adc0b4600 con 0x7f1adc0d0f20 2024-09-21T07:52:42.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.156+0000 7f1ae2400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f1adc0b4900 con 0x7f1adc0cb960 2024-09-21T07:52:42.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.156+0000 7f1adbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1adc0cbd70 0x7f1adc1582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:42.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.156+0000 7f1adbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1adc0cbd70 0x7f1adc1582c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34656/0 (socket says 172.21.15.39:34656) 2024-09-21T07:52:42.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.156+0000 7f1adbe00640 1 -- 172.21.15.39:0/2107690313 learned_addr learned my addr 172.21.15.39:0/2107690313 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:42.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.156+0000 7f1ae1a00640 1 --2- 172.21.15.39:0/2107690313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1adc0d0f20 0x7f1adc1f5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:42.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.156+0000 7f1ae1000640 1 --2- 172.21.15.39:0/2107690313 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1adc0cb960 0x7f1adc1f6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:42.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.157+0000 7f1adbe00640 1 -- 172.21.15.39:0/2107690313 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1adc0cb960 msgr2=0x7f1adc1f6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:42.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.157+0000 7f1adbe00640 1 --2- 172.21.15.39:0/2107690313 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1adc0cb960 0x7f1adc1f6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:42.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.157+0000 7f1adbe00640 1 -- 172.21.15.39:0/2107690313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1adc0d0f20 msgr2=0x7f1adc1f5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:42.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.157+0000 7f1adbe00640 1 --2- 172.21.15.39:0/2107690313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1adc0d0f20 0x7f1adc1f5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:42.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.157+0000 7f1adbe00640 1 -- 172.21.15.39:0/2107690313 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1adc09dbd0 con 0x7f1adc0cbd70 2024-09-21T07:52:42.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.157+0000 7f1adbe00640 1 --2- 172.21.15.39:0/2107690313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1adc0cbd70 0x7f1adc1582c0 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8001220 tx=0x7f1ac800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:42.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.157+0000 7f1ad8c00640 1 -- 172.21.15.39:0/2107690313 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac8011790 con 0x7f1adc0cbd70 2024-09-21T07:52:42.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.157+0000 7f1ad8c00640 1 -- 172.21.15.39:0/2107690313 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ac8017070 con 0x7f1adc0cbd70 2024-09-21T07:52:42.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.157+0000 7f1ad8c00640 1 -- 172.21.15.39:0/2107690313 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac8011cc0 con 0x7f1adc0cbd70 2024-09-21T07:52:42.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.157+0000 7f1ae2400640 1 -- 172.21.15.39:0/2107690313 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1adc09de70 con 0x7f1adc0cbd70 2024-09-21T07:52:42.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.157+0000 7f1ae2400640 1 -- 172.21.15.39:0/2107690313 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1adc1f6c40 con 0x7f1adc0cbd70 2024-09-21T07:52:42.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.158+0000 7f1ae2400640 1 -- 172.21.15.39:0/2107690313 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1adc1f96e0 con 0x7f1adc0cbd70 2024-09-21T07:52:42.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.160+0000 7f1ad8c00640 1 -- 172.21.15.39:0/2107690313 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1ac801c460 con 0x7f1adc0cbd70 2024-09-21T07:52:42.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.160+0000 7f1ad8c00640 1 --2- 172.21.15.39:0/2107690313 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1abc078bc0 0x7f1abc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:42.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.161+0000 7f1ad8c00640 1 -- 172.21.15.39:0/2107690313 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(247..247 src has 1..247) ==== 7605+0+0 (secure 0 0 0) 0x7f1ac809b110 con 0x7f1adc0cbd70 2024-09-21T07:52:42.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.161+0000 7f1ae1000640 1 --2- 172.21.15.39:0/2107690313 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1abc078bc0 0x7f1abc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:42.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.162+0000 7f1ae1000640 1 --2- 172.21.15.39:0/2107690313 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1abc078bc0 0x7f1abc07b0d0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0034000 tx=0x7f1ad0004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:42.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.165+0000 7f1ad8c00640 1 -- 172.21.15.39:0/2107690313 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1ac8062780 con 0x7f1adc0cbd70 2024-09-21T07:52:42.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.399+0000 7f1ae2400640 1 -- 172.21.15.39:0/2107690313 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1adc15c830 con 0x7f1adc0cbd70 2024-09-21T07:52:42.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.400+0000 7f1ad8c00640 1 -- 172.21.15.39:0/2107690313 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 281 v281) ==== 95+0+5283 (secure 0 0 0) 0x7f1ac80671b0 con 0x7f1adc0cbd70 2024-09-21T07:52:42.402 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:52:42.402 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":281,"btime":"2024-09-21T07:52:29:069596+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":278}],"filesystems":[{"mdsmap":{"epoch":281,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:52:29.069589+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":246,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":10195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10195":{"gid":10195,"name":"d","rank":2,"incarnation":277,"state":"up:active","state_seq":25,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[9967,10195,11052]},"id":13}]} 2024-09-21T07:52:42.402 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 281 2024-09-21T07:52:42.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.402+0000 7f1ae2400640 1 -- 172.21.15.39:0/2107690313 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1abc078bc0 msgr2=0x7f1abc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:42.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.402+0000 7f1ae2400640 1 --2- 172.21.15.39:0/2107690313 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1abc078bc0 0x7f1abc07b0d0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0034000 tx=0x7f1ad0004730 comp rx=0 tx=0).stop 2024-09-21T07:52:42.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.402+0000 7f1ae2400640 1 -- 172.21.15.39:0/2107690313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1adc0cbd70 msgr2=0x7f1adc1582c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:42.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.402+0000 7f1ae2400640 1 --2- 172.21.15.39:0/2107690313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1adc0cbd70 0x7f1adc1582c0 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8001220 tx=0x7f1ac800c890 comp rx=0 tx=0).stop 2024-09-21T07:52:42.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.402+0000 7f1ae2400640 1 -- 172.21.15.39:0/2107690313 shutdown_connections 2024-09-21T07:52:42.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.402+0000 7f1ae2400640 1 --2- 172.21.15.39:0/2107690313 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1abc078bc0 0x7f1abc07b0d0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:42.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.402+0000 7f1ae2400640 1 --2- 172.21.15.39:0/2107690313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1adc0d0f20 0x7f1adc1f5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:42.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.402+0000 7f1ae2400640 1 --2- 172.21.15.39:0/2107690313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1adc0cbd70 0x7f1adc1582c0 unknown :-1 s=CLOSED pgs=2567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:42.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.402+0000 7f1ae2400640 1 --2- 172.21.15.39:0/2107690313 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1adc0cb960 0x7f1adc1f6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:42.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.402+0000 7f1ae2400640 1 -- 172.21.15.39:0/2107690313 >> 172.21.15.39:0/2107690313 conn(0x7f1adc0b2510 msgr2=0x7f1adc0cedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:42.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.403+0000 7f1ae2400640 1 -- 172.21.15.39:0/2107690313 shutdown_connections 2024-09-21T07:52:42.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.403+0000 7f1ae2400640 1 -- 172.21.15.39:0/2107690313 wait complete. 2024-09-21T07:52:42.427 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 13:2 true 2024-09-21T07:52:42.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.654+0000 7fc3d1a00640 1 -- 172.21.15.39:0/617712179 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3cc156d60 msgr2=0x7fc3cc0cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:42.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.654+0000 7fc3d1a00640 1 --2- 172.21.15.39:0/617712179 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3cc156d60 0x7fc3cc0cc430 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc072a30 tx=0x7fc3bc02a560 comp rx=0 tx=0).stop 2024-09-21T07:52:42.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.654+0000 7fc3d1a00640 1 -- 172.21.15.39:0/617712179 shutdown_connections 2024-09-21T07:52:42.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.654+0000 7fc3d1a00640 1 --2- 172.21.15.39:0/617712179 >> v2:172.21.15.148:3300/0 conn(0x7fc3cc160770 0x7fc3cc15cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:42.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.654+0000 7fc3d1a00640 1 --2- 172.21.15.39:0/617712179 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc3cc0af690 0x7fc3cc160230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:42.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.654+0000 7fc3d1a00640 1 --2- 172.21.15.39:0/617712179 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3cc156d60 0x7fc3cc0cc430 unknown :-1 s=CLOSED pgs=2568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:42.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.654+0000 7fc3d1a00640 1 -- 172.21.15.39:0/617712179 >> 172.21.15.39:0/617712179 conn(0x7fc3cc0b25f0 msgr2=0x7fc3cc0d01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:42.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.654+0000 7fc3d1a00640 1 -- 172.21.15.39:0/617712179 shutdown_connections 2024-09-21T07:52:42.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.654+0000 7fc3d1a00640 1 -- 172.21.15.39:0/617712179 wait complete. 2024-09-21T07:52:42.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.656+0000 7fc3d1a00640 1 Processor -- start 2024-09-21T07:52:42.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.656+0000 7fc3d1a00640 1 -- start start 2024-09-21T07:52:42.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.658+0000 7fc3d1a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3cc0af690 0x7fc3cc1f5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:42.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.658+0000 7fc3d1a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3cc156d60 0x7fc3cc073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:42.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.658+0000 7fc3d1a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc3cc160770 0x7fc3cc0743e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:42.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.658+0000 7fc3d1a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc15d500 con 0x7fc3cc156d60 2024-09-21T07:52:42.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.658+0000 7fc3d1a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc15d380 con 0x7fc3cc0af690 2024-09-21T07:52:42.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.658+0000 7fc3d1a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc3cc15d680 con 0x7fc3cc160770 2024-09-21T07:52:42.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.658+0000 7fc3c3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3cc156d60 0x7fc3cc073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:42.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.658+0000 7fc3c3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3cc156d60 0x7fc3cc073ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34686/0 (socket says 172.21.15.39:34686) 2024-09-21T07:52:42.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.658+0000 7fc3c3e00640 1 -- 172.21.15.39:0/3705338075 learned_addr learned my addr 172.21.15.39:0/3705338075 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:42.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.658+0000 7fc3d1000640 1 --2- 172.21.15.39:0/3705338075 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc3cc160770 0x7fc3cc0743e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:42.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.658+0000 7fc3cbe00640 1 --2- 172.21.15.39:0/3705338075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3cc0af690 0x7fc3cc1f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:42.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.658+0000 7fc3c3e00640 1 -- 172.21.15.39:0/3705338075 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc3cc160770 msgr2=0x7fc3cc0743e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:42.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.658+0000 7fc3c3e00640 1 --2- 172.21.15.39:0/3705338075 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc3cc160770 0x7fc3cc0743e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:42.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.658+0000 7fc3c3e00640 1 -- 172.21.15.39:0/3705338075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3cc0af690 msgr2=0x7fc3cc1f5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:42.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.658+0000 7fc3c3e00640 1 --2- 172.21.15.39:0/3705338075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3cc0af690 0x7fc3cc1f5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:42.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.658+0000 7fc3c3e00640 1 -- 172.21.15.39:0/3705338075 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3cc0c3ef0 con 0x7fc3cc156d60 2024-09-21T07:52:42.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.659+0000 7fc3c3e00640 1 --2- 172.21.15.39:0/3705338075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3cc156d60 0x7fc3cc073ea0 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7fc3b400d660 tx=0x7fc3b400db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:42.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.659+0000 7fc3c8c00640 1 -- 172.21.15.39:0/3705338075 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4017070 con 0x7fc3cc156d60 2024-09-21T07:52:42.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.659+0000 7fc3c8c00640 1 -- 172.21.15.39:0/3705338075 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc3b4005310 con 0x7fc3cc156d60 2024-09-21T07:52:42.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.659+0000 7fc3d1a00640 1 -- 172.21.15.39:0/3705338075 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3cc1f8f90 con 0x7fc3cc156d60 2024-09-21T07:52:42.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.659+0000 7fc3c8c00640 1 -- 172.21.15.39:0/3705338075 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4009820 con 0x7fc3cc156d60 2024-09-21T07:52:42.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.659+0000 7fc3d1a00640 1 -- 172.21.15.39:0/3705338075 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3cc1f9550 con 0x7fc3cc156d60 2024-09-21T07:52:42.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.660+0000 7fc3d1a00640 1 -- 172.21.15.39:0/3705338075 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3cc0cc430 con 0x7fc3cc156d60 2024-09-21T07:52:42.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.661+0000 7fc3c8c00640 1 -- 172.21.15.39:0/3705338075 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc3b40099c0 con 0x7fc3cc156d60 2024-09-21T07:52:42.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.662+0000 7fc3c8c00640 1 --2- 172.21.15.39:0/3705338075 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc3a8078bc0 0x7fc3a807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:42.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.663+0000 7fc3c8c00640 1 -- 172.21.15.39:0/3705338075 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(247..247 src has 1..247) ==== 7605+0+0 (secure 0 0 0) 0x7fc3b409b510 con 0x7fc3cc156d60 2024-09-21T07:52:42.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.663+0000 7fc3cbe00640 1 --2- 172.21.15.39:0/3705338075 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc3a8078bc0 0x7fc3a807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:42.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.664+0000 7fc3cbe00640 1 --2- 172.21.15.39:0/3705338075 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc3a8078bc0 0x7fc3a807b0d0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc033000 tx=0x7fc3bc004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:42.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.667+0000 7fc3c8c00640 1 -- 172.21.15.39:0/3705338075 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc3b40631d0 con 0x7fc3cc156d60 2024-09-21T07:52:42.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:42.940+0000 7fc3d1a00640 1 -- 172.21.15.39:0/3705338075 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "13:2", "val": "true"} v 0) -- 0x7fc3cc051040 con 0x7fc3cc156d60 2024-09-21T07:52:43.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.133+0000 7fc3c8c00640 1 -- 172.21.15.39:0/3705338075 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "13:2", "val": "true"}]=0 freezing mds.10195 v282) ==== 114+0+0 (secure 0 0 0) 0x7fc3b4062ad0 con 0x7fc3cc156d60 2024-09-21T07:52:43.135 INFO:teuthology.orchestra.run.smithi039.stderr:freezing mds.10195 2024-09-21T07:52:43.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.136+0000 7fc3d1a00640 1 -- 172.21.15.39:0/3705338075 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc3a8078bc0 msgr2=0x7fc3a807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:43.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.136+0000 7fc3d1a00640 1 --2- 172.21.15.39:0/3705338075 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc3a8078bc0 0x7fc3a807b0d0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc033000 tx=0x7fc3bc004730 comp rx=0 tx=0).stop 2024-09-21T07:52:43.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.136+0000 7fc3d1a00640 1 -- 172.21.15.39:0/3705338075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3cc156d60 msgr2=0x7fc3cc073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:43.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.136+0000 7fc3d1a00640 1 --2- 172.21.15.39:0/3705338075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3cc156d60 0x7fc3cc073ea0 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7fc3b400d660 tx=0x7fc3b400db20 comp rx=0 tx=0).stop 2024-09-21T07:52:43.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.136+0000 7fc3d1a00640 1 -- 172.21.15.39:0/3705338075 shutdown_connections 2024-09-21T07:52:43.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.136+0000 7fc3d1a00640 1 --2- 172.21.15.39:0/3705338075 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc3a8078bc0 0x7fc3a807b0d0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:43.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.136+0000 7fc3d1a00640 1 --2- 172.21.15.39:0/3705338075 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc3cc160770 0x7fc3cc0743e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:43.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.136+0000 7fc3d1a00640 1 --2- 172.21.15.39:0/3705338075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3cc156d60 0x7fc3cc073ea0 unknown :-1 s=CLOSED pgs=2569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:43.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.136+0000 7fc3d1a00640 1 --2- 172.21.15.39:0/3705338075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3cc0af690 0x7fc3cc1f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:43.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.136+0000 7fc3d1a00640 1 -- 172.21.15.39:0/3705338075 >> 172.21.15.39:0/3705338075 conn(0x7fc3cc0b25f0 msgr2=0x7fc3cc0ae5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:43.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.136+0000 7fc3d1a00640 1 -- 172.21.15.39:0/3705338075 shutdown_connections 2024-09-21T07:52:43.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.136+0000 7fc3d1a00640 1 -- 172.21.15.39:0/3705338075 wait complete. 2024-09-21T07:52:43.165 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json config set mds_kill_mdstable_at 4 2024-09-21T07:52:43.406 INFO:teuthology.orchestra.run.smithi148.stdout:{"success":"mds_kill_mdstable_at = '' (not observed, change may require restart) "} 2024-09-21T07:52:43.406 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_kill_mdstable_at = '' (not observed, change may require restart) " } 2024-09-21T07:52:43.406 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:52:43.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.632+0000 7fde66000640 1 -- 172.21.15.39:0/2510672539 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde48004800 msgr2=0x7fde48004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:43.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.632+0000 7fde66000640 1 --2- 172.21.15.39:0/2510672539 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde48004800 0x7fde48004c10 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7fde600b3440 tx=0x7fde48005830 comp rx=0 tx=0).stop 2024-09-21T07:52:43.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.632+0000 7fde65600640 1 -- 172.21.15.39:0/2510672539 reap_dead start 2024-09-21T07:52:43.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.632+0000 7fde66000640 1 -- 172.21.15.39:0/2510672539 shutdown_connections 2024-09-21T07:52:43.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.632+0000 7fde66000640 1 -- 172.21.15.39:0/2510672539 >> 172.21.15.39:0/2510672539 conn(0x7fde600b2510 msgr2=0x7fde600d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:43.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.632+0000 7fde66000640 1 -- 172.21.15.39:0/2510672539 shutdown_connections 2024-09-21T07:52:43.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.632+0000 7fde66000640 1 -- 172.21.15.39:0/2510672539 wait complete. 2024-09-21T07:52:43.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.634+0000 7fde66000640 1 Processor -- start 2024-09-21T07:52:43.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.634+0000 7fde66000640 1 -- start start 2024-09-21T07:52:43.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.636+0000 7fde66000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde48004800 0x7fde600c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:43.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.636+0000 7fde66000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde60069270 0x7fde600c5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:43.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.636+0000 7fde66000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde600c58d0 0x7fde6015d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:43.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.636+0000 7fde66000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fde600cb030 con 0x7fde600c58d0 2024-09-21T07:52:43.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.636+0000 7fde66000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fde600caeb0 con 0x7fde48004800 2024-09-21T07:52:43.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.636+0000 7fde66000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fde600cb1b0 con 0x7fde60069270 2024-09-21T07:52:43.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.636+0000 7fde65600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde600c58d0 0x7fde6015d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:43.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.636+0000 7fde65600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde600c58d0 0x7fde6015d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34692/0 (socket says 172.21.15.39:34692) 2024-09-21T07:52:43.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.636+0000 7fde65600640 1 -- 172.21.15.39:0/3725660449 learned_addr learned my addr 172.21.15.39:0/3725660449 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:43.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.636+0000 7fde5fe00640 1 --2- 172.21.15.39:0/3725660449 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde60069270 0x7fde600c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:43.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.636+0000 7fde64c00640 1 --2- 172.21.15.39:0/3725660449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde48004800 0x7fde600c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:43.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.637+0000 7fde65600640 1 -- 172.21.15.39:0/3725660449 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde60069270 msgr2=0x7fde600c5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:43.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.637+0000 7fde65600640 1 --2- 172.21.15.39:0/3725660449 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde60069270 0x7fde600c5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:43.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.637+0000 7fde65600640 1 -- 172.21.15.39:0/3725660449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde48004800 msgr2=0x7fde600c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:43.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.637+0000 7fde65600640 1 --2- 172.21.15.39:0/3725660449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde48004800 0x7fde600c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:43.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.637+0000 7fde65600640 1 -- 172.21.15.39:0/3725660449 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde6015e840 con 0x7fde600c58d0 2024-09-21T07:52:43.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.637+0000 7fde64c00640 1 --2- 172.21.15.39:0/3725660449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde48004800 0x7fde600c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:52:43.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.637+0000 7fde65600640 1 --2- 172.21.15.39:0/3725660449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde600c58d0 0x7fde6015d540 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 crypto rx=0x7fde600ccc70 tx=0x7fde5800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:43.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.637+0000 7fde5cc00640 1 -- 172.21.15.39:0/3725660449 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde58004b20 con 0x7fde600c58d0 2024-09-21T07:52:43.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.637+0000 7fde5cc00640 1 -- 172.21.15.39:0/3725660449 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde58004d00 con 0x7fde600c58d0 2024-09-21T07:52:43.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.637+0000 7fde5cc00640 1 -- 172.21.15.39:0/3725660449 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde58003bd0 con 0x7fde600c58d0 2024-09-21T07:52:43.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.637+0000 7fde66000640 1 -- 172.21.15.39:0/3725660449 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde600c4eb0 con 0x7fde600c58d0 2024-09-21T07:52:43.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.637+0000 7fde66000640 1 -- 172.21.15.39:0/3725660449 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde6015e560 con 0x7fde600c58d0 2024-09-21T07:52:43.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.639+0000 7fde66000640 1 -- 172.21.15.39:0/3725660449 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde6015a0a0 con 0x7fde600c58d0 2024-09-21T07:52:43.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.640+0000 7fde5cc00640 1 -- 172.21.15.39:0/3725660449 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fde58004030 con 0x7fde600c58d0 2024-09-21T07:52:43.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.641+0000 7fde5cc00640 1 --2- 172.21.15.39:0/3725660449 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fde40078b90 0x7fde4007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:43.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.641+0000 7fde5cc00640 1 -- 172.21.15.39:0/3725660449 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(247..247 src has 1..247) ==== 7605+0+0 (secure 0 0 0) 0x7fde5809bb80 con 0x7fde600c58d0 2024-09-21T07:52:43.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.641+0000 7fde64c00640 1 --2- 172.21.15.39:0/3725660449 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fde40078b90 0x7fde4007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:43.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.642+0000 7fde64c00640 1 --2- 172.21.15.39:0/3725660449 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fde40078b90 0x7fde4007b0a0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fde54001330 tx=0x7fde540068a0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:43.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.645+0000 7fde5cc00640 1 -- 172.21.15.39:0/3725660449 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fde58063290 con 0x7fde600c58d0 2024-09-21T07:52:43.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.880+0000 7fde66000640 1 -- 172.21.15.39:0/3725660449 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde600ce720 con 0x7fde600c58d0 2024-09-21T07:52:43.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.881+0000 7fde5cc00640 1 -- 172.21.15.39:0/3725660449 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 282 v282) ==== 95+0+5283 (secure 0 0 0) 0x7fde58067cc0 con 0x7fde600c58d0 2024-09-21T07:52:43.883 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:52:43.883 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":282,"btime":"2024-09-21T07:52:43:132032+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":278}],"filesystems":[{"mdsmap":{"epoch":282,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:52:42.942455+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":246,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":10195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10195":{"gid":10195,"name":"d","rank":2,"incarnation":277,"state":"up:active","state_seq":25,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[11052,10195,9967]},"id":13}]} 2024-09-21T07:52:43.883 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 282 2024-09-21T07:52:43.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.883+0000 7fde66000640 1 -- 172.21.15.39:0/3725660449 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fde40078b90 msgr2=0x7fde4007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:43.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.883+0000 7fde66000640 1 --2- 172.21.15.39:0/3725660449 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fde40078b90 0x7fde4007b0a0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fde54001330 tx=0x7fde540068a0 comp rx=0 tx=0).stop 2024-09-21T07:52:43.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.883+0000 7fde66000640 1 -- 172.21.15.39:0/3725660449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde600c58d0 msgr2=0x7fde6015d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:43.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.883+0000 7fde66000640 1 --2- 172.21.15.39:0/3725660449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde600c58d0 0x7fde6015d540 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 crypto rx=0x7fde600ccc70 tx=0x7fde5800f3c0 comp rx=0 tx=0).stop 2024-09-21T07:52:43.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.883+0000 7fde66000640 1 -- 172.21.15.39:0/3725660449 shutdown_connections 2024-09-21T07:52:43.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.883+0000 7fde66000640 1 --2- 172.21.15.39:0/3725660449 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fde40078b90 0x7fde4007b0a0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:43.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.883+0000 7fde66000640 1 --2- 172.21.15.39:0/3725660449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde600c58d0 0x7fde6015d540 unknown :-1 s=CLOSED pgs=2572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:43.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.883+0000 7fde66000640 1 --2- 172.21.15.39:0/3725660449 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde60069270 0x7fde600c5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:43.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.883+0000 7fde66000640 1 --2- 172.21.15.39:0/3725660449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde48004800 0x7fde600c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:43.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.883+0000 7fde66000640 1 -- 172.21.15.39:0/3725660449 >> 172.21.15.39:0/3725660449 conn(0x7fde600b2510 msgr2=0x7fde60070350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:43.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.884+0000 7fde66000640 1 -- 172.21.15.39:0/3725660449 shutdown_connections 2024-09-21T07:52:43.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:43.884+0000 7fde66000640 1 -- 172.21.15.39:0/3725660449 wait complete. 2024-09-21T07:52:43.908 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json dump snaps --server 2024-09-21T07:52:44.148 INFO:teuthology.orchestra.run.smithi148.stdout:{"last_snap":8,"last_created":8,"last_destroyed":7,"pending_noop":[],"snaps":[{"snapid":8,"ino":1099511627780,"stamp":"2024-09-21T07:51:12.091986+0000","name":"s1","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2024-09-21T07:52:44.148 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 8, "last_destroyed": 7, "last_snap": 8, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627780, "metadata": {}, "name": "s1", "snapid": 8, "stamp": "2024-09-21T07:51:12.091986+0000" } ] } 2024-09-21T07:52:44.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:44.148 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d0/d2/dir/.snap/s2) 2024-09-21T07:52:44.153 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-09-21T07:52:44.196 INFO:tasks.ceph.mds.d.smithi148.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::commit(version_t, LogSegment*)' thread 7f3f39e00640 time 2024-09-21T07:52:44.196178+0000 2024-09-21T07:52:44.196 INFO:tasks.ceph.mds.d.smithi148.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: 195: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 4) 2024-09-21T07:52:44.197 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:52:44.197 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f3f4905654b] 2024-09-21T07:52:44.197 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f3f49056758] 2024-09-21T07:52:44.197 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: (MDSTableClient::commit(unsigned long, LogSegment*)+0x333) [0x55b9c97413d3] 2024-09-21T07:52:44.197 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: (Server::_mksnap_finish(boost::intrusive_ptr const&, CInode*, SnapInfo&)+0x9f) [0x55b9c94f4ac3] 2024-09-21T07:52:44.197 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: ceph-mds(+0x221794) [0x55b9c94e9794] 2024-09-21T07:52:44.197 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: (MDSContext::complete(int)+0x7c) [0x55b9c978f6d8] 2024-09-21T07:52:44.197 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSIOContextBase::complete(int)+0x501) [0x55b9c978fd57] 2024-09-21T07:52:44.197 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: (MDSLogContextBase::complete(int)+0x46) [0x55b9c978fdd4] 2024-09-21T07:52:44.198 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: (Finisher::finisher_thread_entry()+0x495) [0x7f3f48fde34b] 2024-09-21T07:52:44.198 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: ceph-mds(+0x1052e1) [0x55b9c93cd2e1] 2024-09-21T07:52:44.198 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (Thread::entry_wrapper()+0x43) [0x7f3f4902fd57] 2024-09-21T07:52:44.198 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: (Thread::_entry_func(void*)+0xd) [0x7f3f4902fd73] 2024-09-21T07:52:44.198 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: /lib64/libc.so.6(+0x89c52) [0x7f3f48689c52] 2024-09-21T07:52:44.198 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: /lib64/libc.so.6(+0x10ec80) [0x7f3f4870ec80] 2024-09-21T07:52:44.198 INFO:tasks.ceph.mds.d.smithi148.stderr:*** Caught signal (Aborted) ** 2024-09-21T07:52:44.198 INFO:tasks.ceph.mds.d.smithi148.stderr: in thread 7f3f39e00640 thread_name:MR_Finisher 2024-09-21T07:52:44.198 INFO:tasks.ceph.mds.d.smithi148.stderr:2024-09-21T07:52:44.195+0000 7f3f39e00640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::commit(version_t, LogSegment*)' thread 7f3f39e00640 time 2024-09-21T07:52:44.196178+0000 2024-09-21T07:52:44.198 INFO:tasks.ceph.mds.d.smithi148.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: 195: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 4) 2024-09-21T07:52:44.198 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:52:44.198 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:52:44.198 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f3f4905654b] 2024-09-21T07:52:44.198 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f3f49056758] 2024-09-21T07:52:44.199 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: (MDSTableClient::commit(unsigned long, LogSegment*)+0x333) [0x55b9c97413d3] 2024-09-21T07:52:44.199 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: (Server::_mksnap_finish(boost::intrusive_ptr const&, CInode*, SnapInfo&)+0x9f) [0x55b9c94f4ac3] 2024-09-21T07:52:44.199 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: ceph-mds(+0x221794) [0x55b9c94e9794] 2024-09-21T07:52:44.199 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: (MDSContext::complete(int)+0x7c) [0x55b9c978f6d8] 2024-09-21T07:52:44.199 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSIOContextBase::complete(int)+0x501) [0x55b9c978fd57] 2024-09-21T07:52:44.199 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: (MDSLogContextBase::complete(int)+0x46) [0x55b9c978fdd4] 2024-09-21T07:52:44.199 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: (Finisher::finisher_thread_entry()+0x495) [0x7f3f48fde34b] 2024-09-21T07:52:44.199 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: ceph-mds(+0x1052e1) [0x55b9c93cd2e1] 2024-09-21T07:52:44.199 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (Thread::entry_wrapper()+0x43) [0x7f3f4902fd57] 2024-09-21T07:52:44.199 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: (Thread::_entry_func(void*)+0xd) [0x7f3f4902fd73] 2024-09-21T07:52:44.199 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: /lib64/libc.so.6(+0x89c52) [0x7f3f48689c52] 2024-09-21T07:52:44.199 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: /lib64/libc.so.6(+0x10ec80) [0x7f3f4870ec80] 2024-09-21T07:52:44.199 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:52:44.200 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:52:44.200 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f3f4863e6f0] 2024-09-21T07:52:44.200 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f3f4868b99c] 2024-09-21T07:52:44.201 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: raise() 2024-09-21T07:52:44.201 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: abort() 2024-09-21T07:52:44.201 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7f3f49056686] 2024-09-21T07:52:44.201 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f3f49056758] 2024-09-21T07:52:44.201 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSTableClient::commit(unsigned long, LogSegment*)+0x333) [0x55b9c97413d3] 2024-09-21T07:52:44.201 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: (Server::_mksnap_finish(boost::intrusive_ptr const&, CInode*, SnapInfo&)+0x9f) [0x55b9c94f4ac3] 2024-09-21T07:52:44.201 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: ceph-mds(+0x221794) [0x55b9c94e9794] 2024-09-21T07:52:44.201 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: (MDSContext::complete(int)+0x7c) [0x55b9c978f6d8] 2024-09-21T07:52:44.201 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (MDSIOContextBase::complete(int)+0x501) [0x55b9c978fd57] 2024-09-21T07:52:44.201 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: (MDSLogContextBase::complete(int)+0x46) [0x55b9c978fdd4] 2024-09-21T07:52:44.201 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: (Finisher::finisher_thread_entry()+0x495) [0x7f3f48fde34b] 2024-09-21T07:52:44.201 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: ceph-mds(+0x1052e1) [0x55b9c93cd2e1] 2024-09-21T07:52:44.202 INFO:tasks.ceph.mds.d.smithi148.stderr: 15: (Thread::entry_wrapper()+0x43) [0x7f3f4902fd57] 2024-09-21T07:52:44.202 INFO:tasks.ceph.mds.d.smithi148.stderr: 16: (Thread::_entry_func(void*)+0xd) [0x7f3f4902fd73] 2024-09-21T07:52:44.202 INFO:tasks.ceph.mds.d.smithi148.stderr: 17: /lib64/libc.so.6(+0x89c52) [0x7f3f48689c52] 2024-09-21T07:52:44.202 INFO:tasks.ceph.mds.d.smithi148.stderr: 18: /lib64/libc.so.6(+0x10ec80) [0x7f3f4870ec80] 2024-09-21T07:52:44.202 INFO:tasks.ceph.mds.d.smithi148.stderr:2024-09-21T07:52:44.196+0000 7f3f39e00640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:52:44.202 INFO:tasks.ceph.mds.d.smithi148.stderr: in thread 7f3f39e00640 thread_name:MR_Finisher 2024-09-21T07:52:44.202 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:52:44.202 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:52:44.202 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f3f4863e6f0] 2024-09-21T07:52:44.202 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f3f4868b99c] 2024-09-21T07:52:44.202 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: raise() 2024-09-21T07:52:44.202 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: abort() 2024-09-21T07:52:44.202 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7f3f49056686] 2024-09-21T07:52:44.202 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f3f49056758] 2024-09-21T07:52:44.203 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSTableClient::commit(unsigned long, LogSegment*)+0x333) [0x55b9c97413d3] 2024-09-21T07:52:44.203 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: (Server::_mksnap_finish(boost::intrusive_ptr const&, CInode*, SnapInfo&)+0x9f) [0x55b9c94f4ac3] 2024-09-21T07:52:44.203 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: ceph-mds(+0x221794) [0x55b9c94e9794] 2024-09-21T07:52:44.203 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: (MDSContext::complete(int)+0x7c) [0x55b9c978f6d8] 2024-09-21T07:52:44.203 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (MDSIOContextBase::complete(int)+0x501) [0x55b9c978fd57] 2024-09-21T07:52:44.203 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: (MDSLogContextBase::complete(int)+0x46) [0x55b9c978fdd4] 2024-09-21T07:52:44.203 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: (Finisher::finisher_thread_entry()+0x495) [0x7f3f48fde34b] 2024-09-21T07:52:44.203 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: ceph-mds(+0x1052e1) [0x55b9c93cd2e1] 2024-09-21T07:52:44.203 INFO:tasks.ceph.mds.d.smithi148.stderr: 15: (Thread::entry_wrapper()+0x43) [0x7f3f4902fd57] 2024-09-21T07:52:44.203 INFO:tasks.ceph.mds.d.smithi148.stderr: 16: (Thread::_entry_func(void*)+0xd) [0x7f3f4902fd73] 2024-09-21T07:52:44.203 INFO:tasks.ceph.mds.d.smithi148.stderr: 17: /lib64/libc.so.6(+0x89c52) [0x7f3f48689c52] 2024-09-21T07:52:44.203 INFO:tasks.ceph.mds.d.smithi148.stderr: 18: /lib64/libc.so.6(+0x10ec80) [0x7f3f4870ec80] 2024-09-21T07:52:44.203 INFO:tasks.ceph.mds.d.smithi148.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:52:44.203 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:52:44.209 INFO:tasks.ceph.mds.d.smithi148.stderr: -1> 2024-09-21T07:52:44.195+0000 7f3f39e00640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::commit(version_t, LogSegment*)' thread 7f3f39e00640 time 2024-09-21T07:52:44.196178+0000 2024-09-21T07:52:44.210 INFO:tasks.ceph.mds.d.smithi148.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: 195: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 4) 2024-09-21T07:52:44.210 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:52:44.210 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:52:44.210 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f3f4905654b] 2024-09-21T07:52:44.210 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f3f49056758] 2024-09-21T07:52:44.210 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: (MDSTableClient::commit(unsigned long, LogSegment*)+0x333) [0x55b9c97413d3] 2024-09-21T07:52:44.210 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: (Server::_mksnap_finish(boost::intrusive_ptr const&, CInode*, SnapInfo&)+0x9f) [0x55b9c94f4ac3] 2024-09-21T07:52:44.210 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: ceph-mds(+0x221794) [0x55b9c94e9794] 2024-09-21T07:52:44.210 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: (MDSContext::complete(int)+0x7c) [0x55b9c978f6d8] 2024-09-21T07:52:44.210 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSIOContextBase::complete(int)+0x501) [0x55b9c978fd57] 2024-09-21T07:52:44.210 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: (MDSLogContextBase::complete(int)+0x46) [0x55b9c978fdd4] 2024-09-21T07:52:44.210 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: (Finisher::finisher_thread_entry()+0x495) [0x7f3f48fde34b] 2024-09-21T07:52:44.210 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: ceph-mds(+0x1052e1) [0x55b9c93cd2e1] 2024-09-21T07:52:44.210 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (Thread::entry_wrapper()+0x43) [0x7f3f4902fd57] 2024-09-21T07:52:44.211 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: (Thread::_entry_func(void*)+0xd) [0x7f3f4902fd73] 2024-09-21T07:52:44.211 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: /lib64/libc.so.6(+0x89c52) [0x7f3f48689c52] 2024-09-21T07:52:44.211 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: /lib64/libc.so.6(+0x10ec80) [0x7f3f4870ec80] 2024-09-21T07:52:44.211 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:52:44.211 INFO:tasks.ceph.mds.d.smithi148.stderr: 0> 2024-09-21T07:52:44.196+0000 7f3f39e00640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:52:44.211 INFO:tasks.ceph.mds.d.smithi148.stderr: in thread 7f3f39e00640 thread_name:MR_Finisher 2024-09-21T07:52:44.211 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:52:44.211 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:52:44.211 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f3f4863e6f0] 2024-09-21T07:52:44.211 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f3f4868b99c] 2024-09-21T07:52:44.211 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: raise() 2024-09-21T07:52:44.211 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: abort() 2024-09-21T07:52:44.211 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7f3f49056686] 2024-09-21T07:52:44.211 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f3f49056758] 2024-09-21T07:52:44.211 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSTableClient::commit(unsigned long, LogSegment*)+0x333) [0x55b9c97413d3] 2024-09-21T07:52:44.211 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: (Server::_mksnap_finish(boost::intrusive_ptr const&, CInode*, SnapInfo&)+0x9f) [0x55b9c94f4ac3] 2024-09-21T07:52:44.212 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: ceph-mds(+0x221794) [0x55b9c94e9794] 2024-09-21T07:52:44.212 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: (MDSContext::complete(int)+0x7c) [0x55b9c978f6d8] 2024-09-21T07:52:44.212 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (MDSIOContextBase::complete(int)+0x501) [0x55b9c978fd57] 2024-09-21T07:52:44.212 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: (MDSLogContextBase::complete(int)+0x46) [0x55b9c978fdd4] 2024-09-21T07:52:44.212 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: (Finisher::finisher_thread_entry()+0x495) [0x7f3f48fde34b] 2024-09-21T07:52:44.212 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: ceph-mds(+0x1052e1) [0x55b9c93cd2e1] 2024-09-21T07:52:44.212 INFO:tasks.ceph.mds.d.smithi148.stderr: 15: (Thread::entry_wrapper()+0x43) [0x7f3f4902fd57] 2024-09-21T07:52:44.212 INFO:tasks.ceph.mds.d.smithi148.stderr: 16: (Thread::_entry_func(void*)+0xd) [0x7f3f4902fd73] 2024-09-21T07:52:44.212 INFO:tasks.ceph.mds.d.smithi148.stderr: 17: /lib64/libc.so.6(+0x89c52) [0x7f3f48689c52] 2024-09-21T07:52:44.212 INFO:tasks.ceph.mds.d.smithi148.stderr: 18: /lib64/libc.so.6(+0x10ec80) [0x7f3f4870ec80] 2024-09-21T07:52:44.212 INFO:tasks.ceph.mds.d.smithi148.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:52:44.212 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:52:44.216 INFO:tasks.ceph.mds.d.smithi148.stderr: -1> 2024-09-21T07:52:44.195+0000 7f3f39e00640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::commit(version_t, LogSegment*)' thread 7f3f39e00640 time 2024-09-21T07:52:44.196178+0000 2024-09-21T07:52:44.216 INFO:tasks.ceph.mds.d.smithi148.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: 195: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 4) 2024-09-21T07:52:44.216 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:52:44.216 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:52:44.216 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f3f4905654b] 2024-09-21T07:52:44.216 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f3f49056758] 2024-09-21T07:52:44.216 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: (MDSTableClient::commit(unsigned long, LogSegment*)+0x333) [0x55b9c97413d3] 2024-09-21T07:52:44.217 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: (Server::_mksnap_finish(boost::intrusive_ptr const&, CInode*, SnapInfo&)+0x9f) [0x55b9c94f4ac3] 2024-09-21T07:52:44.217 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: ceph-mds(+0x221794) [0x55b9c94e9794] 2024-09-21T07:52:44.217 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: (MDSContext::complete(int)+0x7c) [0x55b9c978f6d8] 2024-09-21T07:52:44.217 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSIOContextBase::complete(int)+0x501) [0x55b9c978fd57] 2024-09-21T07:52:44.217 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: (MDSLogContextBase::complete(int)+0x46) [0x55b9c978fdd4] 2024-09-21T07:52:44.217 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: (Finisher::finisher_thread_entry()+0x495) [0x7f3f48fde34b] 2024-09-21T07:52:44.217 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: ceph-mds(+0x1052e1) [0x55b9c93cd2e1] 2024-09-21T07:52:44.217 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (Thread::entry_wrapper()+0x43) [0x7f3f4902fd57] 2024-09-21T07:52:44.217 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: (Thread::_entry_func(void*)+0xd) [0x7f3f4902fd73] 2024-09-21T07:52:44.217 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: /lib64/libc.so.6(+0x89c52) [0x7f3f48689c52] 2024-09-21T07:52:44.217 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: /lib64/libc.so.6(+0x10ec80) [0x7f3f4870ec80] 2024-09-21T07:52:44.217 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:52:44.217 INFO:tasks.ceph.mds.d.smithi148.stderr: 0> 2024-09-21T07:52:44.196+0000 7f3f39e00640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:52:44.217 INFO:tasks.ceph.mds.d.smithi148.stderr: in thread 7f3f39e00640 thread_name:MR_Finisher 2024-09-21T07:52:44.218 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:52:44.218 INFO:tasks.ceph.mds.d.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:52:44.218 INFO:tasks.ceph.mds.d.smithi148.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f3f4863e6f0] 2024-09-21T07:52:44.218 INFO:tasks.ceph.mds.d.smithi148.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f3f4868b99c] 2024-09-21T07:52:44.218 INFO:tasks.ceph.mds.d.smithi148.stderr: 3: raise() 2024-09-21T07:52:44.218 INFO:tasks.ceph.mds.d.smithi148.stderr: 4: abort() 2024-09-21T07:52:44.218 INFO:tasks.ceph.mds.d.smithi148.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7f3f49056686] 2024-09-21T07:52:44.218 INFO:tasks.ceph.mds.d.smithi148.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7f3f49056758] 2024-09-21T07:52:44.218 INFO:tasks.ceph.mds.d.smithi148.stderr: 7: (MDSTableClient::commit(unsigned long, LogSegment*)+0x333) [0x55b9c97413d3] 2024-09-21T07:52:44.218 INFO:tasks.ceph.mds.d.smithi148.stderr: 8: (Server::_mksnap_finish(boost::intrusive_ptr const&, CInode*, SnapInfo&)+0x9f) [0x55b9c94f4ac3] 2024-09-21T07:52:44.218 INFO:tasks.ceph.mds.d.smithi148.stderr: 9: ceph-mds(+0x221794) [0x55b9c94e9794] 2024-09-21T07:52:44.218 INFO:tasks.ceph.mds.d.smithi148.stderr: 10: (MDSContext::complete(int)+0x7c) [0x55b9c978f6d8] 2024-09-21T07:52:44.218 INFO:tasks.ceph.mds.d.smithi148.stderr: 11: (MDSIOContextBase::complete(int)+0x501) [0x55b9c978fd57] 2024-09-21T07:52:44.218 INFO:tasks.ceph.mds.d.smithi148.stderr: 12: (MDSLogContextBase::complete(int)+0x46) [0x55b9c978fdd4] 2024-09-21T07:52:44.218 INFO:tasks.ceph.mds.d.smithi148.stderr: 13: (Finisher::finisher_thread_entry()+0x495) [0x7f3f48fde34b] 2024-09-21T07:52:44.218 INFO:tasks.ceph.mds.d.smithi148.stderr: 14: ceph-mds(+0x1052e1) [0x55b9c93cd2e1] 2024-09-21T07:52:44.219 INFO:tasks.ceph.mds.d.smithi148.stderr: 15: (Thread::entry_wrapper()+0x43) [0x7f3f4902fd57] 2024-09-21T07:52:44.219 INFO:tasks.ceph.mds.d.smithi148.stderr: 16: (Thread::_entry_func(void*)+0xd) [0x7f3f4902fd73] 2024-09-21T07:52:44.219 INFO:tasks.ceph.mds.d.smithi148.stderr: 17: /lib64/libc.so.6(+0x89c52) [0x7f3f48689c52] 2024-09-21T07:52:44.219 INFO:tasks.ceph.mds.d.smithi148.stderr: 18: /lib64/libc.so.6(+0x10ec80) [0x7f3f4870ec80] 2024-09-21T07:52:44.219 INFO:tasks.ceph.mds.d.smithi148.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:52:44.219 INFO:tasks.ceph.mds.d.smithi148.stderr: 2024-09-21T07:52:44.427 INFO:teuthology.orchestra.run.smithi039.stdout:{"mds_beacon_grace":"15.000000"} 2024-09-21T07:52:44.427 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-09-21T07:52:44.428 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:52:44.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.651+0000 7f0d1a600640 1 -- 172.21.15.39:0/337892489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d1415cd30 msgr2=0x7f0d140cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:44.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.651+0000 7f0d1a600640 1 --2- 172.21.15.39:0/337892489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d1415cd30 0x7f0d140cfd10 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009830 tx=0x7f0d0402a520 comp rx=0 tx=0).stop 2024-09-21T07:52:44.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.652+0000 7f0d1a600640 1 -- 172.21.15.39:0/337892489 shutdown_connections 2024-09-21T07:52:44.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.652+0000 7f0d1a600640 1 --2- 172.21.15.39:0/337892489 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d1406f800 0x7f0d140cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:44.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.652+0000 7f0d1a600640 1 --2- 172.21.15.39:0/337892489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d1415cd30 0x7f0d140cfd10 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:44.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.652+0000 7f0d1a600640 1 -- 172.21.15.39:0/337892489 >> 172.21.15.39:0/337892489 conn(0x7f0d1406cfc0 msgr2=0x7f0d1406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:44.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.652+0000 7f0d1a600640 1 -- 172.21.15.39:0/337892489 shutdown_connections 2024-09-21T07:52:44.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.652+0000 7f0d1a600640 1 -- 172.21.15.39:0/337892489 wait complete. 2024-09-21T07:52:44.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.654+0000 7f0d1a600640 1 Processor -- start 2024-09-21T07:52:44.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.654+0000 7f0d1a600640 1 -- start start 2024-09-21T07:52:44.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.656+0000 7f0d1a600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d1406f800 0x7f0d141594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:44.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.656+0000 7f0d1a600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d1406fc10 0x7f0d141f8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:44.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.656+0000 7f0d1a600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d1415cd30 0x7f0d141f9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:44.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.656+0000 7f0d1a600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0d14073890 con 0x7f0d1415cd30 2024-09-21T07:52:44.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.656+0000 7f0d1a600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0d14073710 con 0x7f0d1406fc10 2024-09-21T07:52:44.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.656+0000 7f0d1a600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f0d14073a10 con 0x7f0d1406f800 2024-09-21T07:52:44.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.656+0000 7f0d13e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d1415cd30 0x7f0d141f9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:44.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.656+0000 7f0d13e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d1415cd30 0x7f0d141f9250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34706/0 (socket says 172.21.15.39:34706) 2024-09-21T07:52:44.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.656+0000 7f0d13e00640 1 -- 172.21.15.39:0/1088593994 learned_addr learned my addr 172.21.15.39:0/1088593994 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:44.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.656+0000 7f0d13400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d1406f800 0x7f0d141594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:44.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.656+0000 7f0d0be00640 1 --2- 172.21.15.39:0/1088593994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d1406fc10 0x7f0d141f8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:44.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.657+0000 7f0d13e00640 1 -- 172.21.15.39:0/1088593994 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d1406f800 msgr2=0x7f0d141594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:44.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.657+0000 7f0d13e00640 1 --2- 172.21.15.39:0/1088593994 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d1406f800 0x7f0d141594c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:44.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.657+0000 7f0d13e00640 1 -- 172.21.15.39:0/1088593994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d1406fc10 msgr2=0x7f0d141f8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:44.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.657+0000 7f0d13e00640 1 --2- 172.21.15.39:0/1088593994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d1406fc10 0x7f0d141f8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:44.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.657+0000 7f0d13e00640 1 -- 172.21.15.39:0/1088593994 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d1414eae0 con 0x7f0d1415cd30 2024-09-21T07:52:44.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.657+0000 7f0d0be00640 1 --2- 172.21.15.39:0/1088593994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d1406fc10 0x7f0d141f8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:52:44.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.657+0000 7f0d13e00640 1 --2- 172.21.15.39:0/1088593994 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d1415cd30 0x7f0d141f9250 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00c700 tx=0x7f0d0c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:44.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.657+0000 7f0d0b400640 1 -- 172.21.15.39:0/1088593994 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c003b70 con 0x7f0d1415cd30 2024-09-21T07:52:44.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.657+0000 7f0d0b400640 1 -- 172.21.15.39:0/1088593994 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d0c003d10 con 0x7f0d1415cd30 2024-09-21T07:52:44.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.657+0000 7f0d1a600640 1 -- 172.21.15.39:0/1088593994 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d1414edc0 con 0x7f0d1415cd30 2024-09-21T07:52:44.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.657+0000 7f0d0b400640 1 -- 172.21.15.39:0/1088593994 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c00aab0 con 0x7f0d1415cd30 2024-09-21T07:52:44.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.657+0000 7f0d1a600640 1 -- 172.21.15.39:0/1088593994 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d1414f380 con 0x7f0d1415cd30 2024-09-21T07:52:44.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.659+0000 7f0d1a600640 1 -- 172.21.15.39:0/1088593994 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d140b1b00 con 0x7f0d1415cd30 2024-09-21T07:52:44.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.660+0000 7f0d0b400640 1 -- 172.21.15.39:0/1088593994 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0d0c00ac50 con 0x7f0d1415cd30 2024-09-21T07:52:44.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.661+0000 7f0d0b400640 1 --2- 172.21.15.39:0/1088593994 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0cf0078b90 0x7f0cf007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:44.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.662+0000 7f0d0b400640 1 -- 172.21.15.39:0/1088593994 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(247..247 src has 1..247) ==== 7605+0+0 (secure 0 0 0) 0x7f0d0c09b170 con 0x7f0d1415cd30 2024-09-21T07:52:44.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.662+0000 7f0d13400640 1 --2- 172.21.15.39:0/1088593994 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0cf0078b90 0x7f0cf007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:44.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.663+0000 7f0d13400640 1 --2- 172.21.15.39:0/1088593994 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0cf0078b90 0x7f0cf007b0a0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f0d04034000 tx=0x7f0d04004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:44.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.666+0000 7f0d0b400640 1 -- 172.21.15.39:0/1088593994 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0d0c0627e0 con 0x7f0d1415cd30 2024-09-21T07:52:44.798 INFO:tasks.ceph.mds.d.smithi148.stderr:daemon-helper: command crashed with signal 6 2024-09-21T07:52:44.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.902+0000 7f0d1a600640 1 -- 172.21.15.39:0/1088593994 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d1406a790 con 0x7f0d1415cd30 2024-09-21T07:52:44.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.902+0000 7f0d0b400640 1 -- 172.21.15.39:0/1088593994 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 282 v282) ==== 95+0+5283 (secure 0 0 0) 0x7f0d0c067210 con 0x7f0d1415cd30 2024-09-21T07:52:44.904 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:52:44.905 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":282,"btime":"2024-09-21T07:52:43:132032+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":278}],"filesystems":[{"mdsmap":{"epoch":282,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:52:42.942455+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":246,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":10195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10195":{"gid":10195,"name":"d","rank":2,"incarnation":277,"state":"up:active","state_seq":25,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[11052,10195,9967]},"id":13}]} 2024-09-21T07:52:44.905 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 282 2024-09-21T07:52:44.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.905+0000 7f0d1a600640 1 -- 172.21.15.39:0/1088593994 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0cf0078b90 msgr2=0x7f0cf007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:44.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.905+0000 7f0d1a600640 1 --2- 172.21.15.39:0/1088593994 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0cf0078b90 0x7f0cf007b0a0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f0d04034000 tx=0x7f0d04004730 comp rx=0 tx=0).stop 2024-09-21T07:52:44.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.905+0000 7f0d1a600640 1 -- 172.21.15.39:0/1088593994 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d1415cd30 msgr2=0x7f0d141f9250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:44.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.905+0000 7f0d1a600640 1 --2- 172.21.15.39:0/1088593994 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d1415cd30 0x7f0d141f9250 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00c700 tx=0x7f0d0c00cbc0 comp rx=0 tx=0).stop 2024-09-21T07:52:44.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.905+0000 7f0d1a600640 1 -- 172.21.15.39:0/1088593994 shutdown_connections 2024-09-21T07:52:44.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.905+0000 7f0d1a600640 1 --2- 172.21.15.39:0/1088593994 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0cf0078b90 0x7f0cf007b0a0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:44.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.905+0000 7f0d1a600640 1 --2- 172.21.15.39:0/1088593994 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d1415cd30 0x7f0d141f9250 unknown :-1 s=CLOSED pgs=2573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:44.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.905+0000 7f0d1a600640 1 --2- 172.21.15.39:0/1088593994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0d1406fc10 0x7f0d141f8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:44.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.905+0000 7f0d1a600640 1 --2- 172.21.15.39:0/1088593994 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0d1406f800 0x7f0d141594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:44.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.905+0000 7f0d1a600640 1 -- 172.21.15.39:0/1088593994 >> 172.21.15.39:0/1088593994 conn(0x7f0d1406cfc0 msgr2=0x7f0d1415bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:44.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.905+0000 7f0d1a600640 1 -- 172.21.15.39:0/1088593994 shutdown_connections 2024-09-21T07:52:44.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:44.905+0000 7f0d1a600640 1 -- 172.21.15.39:0/1088593994 wait complete. 2024-09-21T07:52:44.932 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:52:45.369 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-09-21T07:52:49.934 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:52:50.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.170+0000 7fa01cc00640 1 -- 172.21.15.39:0/3488973859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0180cbd70 msgr2=0x7fa018072fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:50.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.170+0000 7fa01cc00640 1 --2- 172.21.15.39:0/3488973859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0180cbd70 0x7fa018072fa0 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7fa010005fd0 tx=0x7fa01002aa90 comp rx=0 tx=0).stop 2024-09-21T07:52:50.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.170+0000 7fa01cc00640 1 -- 172.21.15.39:0/3488973859 shutdown_connections 2024-09-21T07:52:50.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.170+0000 7fa01cc00640 1 --2- 172.21.15.39:0/3488973859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0180cbd70 0x7fa018072fa0 unknown :-1 s=CLOSED pgs=2574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:50.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.170+0000 7fa01cc00640 1 --2- 172.21.15.39:0/3488973859 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa0180d0f20 0x7fa01815c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:50.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.170+0000 7fa01cc00640 1 -- 172.21.15.39:0/3488973859 >> 172.21.15.39:0/3488973859 conn(0x7fa0180b2510 msgr2=0x7fa0180d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:50.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.170+0000 7fa01cc00640 1 -- 172.21.15.39:0/3488973859 shutdown_connections 2024-09-21T07:52:50.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.170+0000 7fa01cc00640 1 -- 172.21.15.39:0/3488973859 wait complete. 2024-09-21T07:52:50.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.172+0000 7fa01cc00640 1 Processor -- start 2024-09-21T07:52:50.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.172+0000 7fa01cc00640 1 -- start start 2024-09-21T07:52:50.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.174+0000 7fa01cc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0180cb960 0x7fa0180c3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:50.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.174+0000 7fa01cc00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0180cbd70 0x7fa0180c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:50.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.174+0000 7fa01cc00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa0180d0f20 0x7fa0180ae260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:50.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.174+0000 7fa01cc00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa0180b4570 con 0x7fa0180cb960 2024-09-21T07:52:50.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.174+0000 7fa01cc00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa0180b43f0 con 0x7fa0180cbd70 2024-09-21T07:52:50.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.174+0000 7fa01cc00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa0180b46f0 con 0x7fa0180d0f20 2024-09-21T07:52:50.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.174+0000 7fa017400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0180cb960 0x7fa0180c3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:50.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.174+0000 7fa017400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0180cb960 0x7fa0180c3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34738/0 (socket says 172.21.15.39:34738) 2024-09-21T07:52:50.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.174+0000 7fa017400640 1 -- 172.21.15.39:0/1128853982 learned_addr learned my addr 172.21.15.39:0/1128853982 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:50.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.174+0000 7fa017e00640 1 --2- 172.21.15.39:0/1128853982 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa0180d0f20 0x7fa0180ae260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:50.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.174+0000 7fa016a00640 1 --2- 172.21.15.39:0/1128853982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0180cbd70 0x7fa0180c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:50.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.175+0000 7fa017400640 1 -- 172.21.15.39:0/1128853982 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa0180d0f20 msgr2=0x7fa0180ae260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:50.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.175+0000 7fa017400640 1 --2- 172.21.15.39:0/1128853982 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa0180d0f20 0x7fa0180ae260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:50.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.175+0000 7fa017400640 1 -- 172.21.15.39:0/1128853982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0180cbd70 msgr2=0x7fa0180c9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:50.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.175+0000 7fa017400640 1 --2- 172.21.15.39:0/1128853982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0180cbd70 0x7fa0180c9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:50.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.175+0000 7fa017400640 1 -- 172.21.15.39:0/1128853982 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa01814c220 con 0x7fa0180cb960 2024-09-21T07:52:50.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.175+0000 7fa017400640 1 --2- 172.21.15.39:0/1128853982 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0180cb960 0x7fa0180c3d70 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 crypto rx=0x7fa00800d6d0 tx=0x7fa00800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:50.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.175+0000 7fa00f400640 1 -- 172.21.15.39:0/1128853982 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa008017070 con 0x7fa0180cb960 2024-09-21T07:52:50.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.175+0000 7fa00f400640 1 -- 172.21.15.39:0/1128853982 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa008005330 con 0x7fa0180cb960 2024-09-21T07:52:50.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.175+0000 7fa01cc00640 1 -- 172.21.15.39:0/1128853982 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa01814c4e0 con 0x7fa0180cb960 2024-09-21T07:52:50.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.175+0000 7fa00f400640 1 -- 172.21.15.39:0/1128853982 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa008009820 con 0x7fa0180cb960 2024-09-21T07:52:50.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.175+0000 7fa01cc00640 1 -- 172.21.15.39:0/1128853982 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa01814caa0 con 0x7fa0180cb960 2024-09-21T07:52:50.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.176+0000 7fa01cc00640 1 -- 172.21.15.39:0/1128853982 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0180b4c10 con 0x7fa0180cb960 2024-09-21T07:52:50.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.178+0000 7fa00f400640 1 -- 172.21.15.39:0/1128853982 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa008009bd0 con 0x7fa0180cb960 2024-09-21T07:52:50.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.179+0000 7fa00f400640 1 --2- 172.21.15.39:0/1128853982 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9ff8078b90 0x7f9ff807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:50.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.179+0000 7fa00f400640 1 -- 172.21.15.39:0/1128853982 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(247..247 src has 1..247) ==== 7605+0+0 (secure 0 0 0) 0x7fa00809b840 con 0x7fa0180cb960 2024-09-21T07:52:50.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.179+0000 7fa016a00640 1 --2- 172.21.15.39:0/1128853982 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9ff8078b90 0x7f9ff807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:50.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.180+0000 7fa016a00640 1 --2- 172.21.15.39:0/1128853982 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9ff8078b90 0x7f9ff807b0a0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fa018074fa0 tx=0x7fa004009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:50.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.184+0000 7fa00f400640 1 -- 172.21.15.39:0/1128853982 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa008062e30 con 0x7fa0180cb960 2024-09-21T07:52:50.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.417+0000 7fa01cc00640 1 -- 172.21.15.39:0/1128853982 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa018074fd0 con 0x7fa0180cb960 2024-09-21T07:52:50.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.418+0000 7fa00f400640 1 -- 172.21.15.39:0/1128853982 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 282 v282) ==== 95+0+5283 (secure 0 0 0) 0x7fa008067860 con 0x7fa0180cb960 2024-09-21T07:52:50.420 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:52:50.420 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":282,"btime":"2024-09-21T07:52:43:132032+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":278}],"filesystems":[{"mdsmap":{"epoch":282,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:52:42.942455+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":246,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":10195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10195":{"gid":10195,"name":"d","rank":2,"incarnation":277,"state":"up:active","state_seq":25,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[11052,10195,9967]},"id":13}]} 2024-09-21T07:52:50.420 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 282 2024-09-21T07:52:50.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.420+0000 7fa01cc00640 1 -- 172.21.15.39:0/1128853982 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9ff8078b90 msgr2=0x7f9ff807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:50.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.420+0000 7fa01cc00640 1 --2- 172.21.15.39:0/1128853982 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9ff8078b90 0x7f9ff807b0a0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fa018074fa0 tx=0x7fa004009040 comp rx=0 tx=0).stop 2024-09-21T07:52:50.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.420+0000 7fa01cc00640 1 -- 172.21.15.39:0/1128853982 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0180cb960 msgr2=0x7fa0180c3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:50.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.420+0000 7fa01cc00640 1 --2- 172.21.15.39:0/1128853982 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0180cb960 0x7fa0180c3d70 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 crypto rx=0x7fa00800d6d0 tx=0x7fa00800db90 comp rx=0 tx=0).stop 2024-09-21T07:52:50.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.420+0000 7fa01cc00640 1 -- 172.21.15.39:0/1128853982 shutdown_connections 2024-09-21T07:52:50.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.420+0000 7fa01cc00640 1 --2- 172.21.15.39:0/1128853982 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9ff8078b90 0x7f9ff807b0a0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:50.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.420+0000 7fa01cc00640 1 --2- 172.21.15.39:0/1128853982 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa0180d0f20 0x7fa0180ae260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:50.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.420+0000 7fa01cc00640 1 --2- 172.21.15.39:0/1128853982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0180cbd70 0x7fa0180c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:50.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.420+0000 7fa01cc00640 1 --2- 172.21.15.39:0/1128853982 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa0180cb960 0x7fa0180c3d70 unknown :-1 s=CLOSED pgs=2575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:50.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.420+0000 7fa01cc00640 1 -- 172.21.15.39:0/1128853982 >> 172.21.15.39:0/1128853982 conn(0x7fa0180b2510 msgr2=0x7fa01815bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:50.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.420+0000 7fa01cc00640 1 -- 172.21.15.39:0/1128853982 shutdown_connections 2024-09-21T07:52:50.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:50.421+0000 7fa01cc00640 1 -- 172.21.15.39:0/1128853982 wait complete. 2024-09-21T07:52:50.448 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:52:51.980 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~7s 2024-09-21T07:52:55.449 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:52:55.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.689+0000 7f3047e00640 1 -- 172.21.15.39:0/465363553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f304015cd30 msgr2=0x7f30400cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:55.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.689+0000 7f3047e00640 1 --2- 172.21.15.39:0/465363553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f304015cd30 0x7f30400cfd10 secure :-1 s=READY pgs=2576 cs=0 l=1 rev1=1 crypto rx=0x7f304006de00 tx=0x7f303402a560 comp rx=0 tx=0).stop 2024-09-21T07:52:55.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.689+0000 7f3047e00640 1 -- 172.21.15.39:0/465363553 shutdown_connections 2024-09-21T07:52:55.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.689+0000 7f3047e00640 1 --2- 172.21.15.39:0/465363553 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f304006fc10 0x7f30400d0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:55.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.689+0000 7f3047e00640 1 --2- 172.21.15.39:0/465363553 >> v2:172.21.15.148:3300/0 conn(0x7f304006f800 0x7f30400cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:55.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.689+0000 7f3047e00640 1 --2- 172.21.15.39:0/465363553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f304015cd30 0x7f30400cfd10 unknown :-1 s=CLOSED pgs=2576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:55.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.689+0000 7f3047e00640 1 -- 172.21.15.39:0/465363553 >> 172.21.15.39:0/465363553 conn(0x7f304006cfc0 msgr2=0x7f304006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:55.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.689+0000 7f3047e00640 1 -- 172.21.15.39:0/465363553 shutdown_connections 2024-09-21T07:52:55.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.689+0000 7f3047e00640 1 -- 172.21.15.39:0/465363553 wait complete. 2024-09-21T07:52:55.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.691+0000 7f3047e00640 1 Processor -- start 2024-09-21T07:52:55.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.691+0000 7f3047e00640 1 -- start start 2024-09-21T07:52:55.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.693+0000 7f3047e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f304006f800 0x7f304014ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:55.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.693+0000 7f3047e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f304006fc10 0x7f30401f8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:55.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.693+0000 7f3047e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f304015cd30 0x7f30401f9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:55.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.693+0000 7f3047e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3040073710 con 0x7f304006f800 2024-09-21T07:52:55.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.693+0000 7f3047e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f30400d0f60 con 0x7f304015cd30 2024-09-21T07:52:55.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.693+0000 7f3047e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f3040073890 con 0x7f304006fc10 2024-09-21T07:52:55.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.693+0000 7f3045400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f304006f800 0x7f304014ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:55.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.693+0000 7f3045400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f304006f800 0x7f304014ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41388/0 (socket says 172.21.15.39:41388) 2024-09-21T07:52:55.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.693+0000 7f3045400640 1 -- 172.21.15.39:0/3536444086 learned_addr learned my addr 172.21.15.39:0/3536444086 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:52:55.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.693+0000 7f3044a00640 1 --2- 172.21.15.39:0/3536444086 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f304006fc10 0x7f30401f8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:55.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.693+0000 7f3045e00640 1 --2- 172.21.15.39:0/3536444086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f304015cd30 0x7f30401f9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:55.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.694+0000 7f3045400640 1 -- 172.21.15.39:0/3536444086 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f304006fc10 msgr2=0x7f30401f8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:55.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.694+0000 7f3045400640 1 --2- 172.21.15.39:0/3536444086 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f304006fc10 0x7f30401f8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:55.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.694+0000 7f3045400640 1 -- 172.21.15.39:0/3536444086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f304015cd30 msgr2=0x7f30401f9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:55.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.694+0000 7f3045400640 1 --2- 172.21.15.39:0/3536444086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f304015cd30 0x7f30401f9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:55.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.694+0000 7f3045400640 1 -- 172.21.15.39:0/3536444086 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3040156010 con 0x7f304006f800 2024-09-21T07:52:55.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.694+0000 7f3045400640 1 --2- 172.21.15.39:0/3536444086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f304006f800 0x7f304014ed30 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7f3034034040 tx=0x7f3034003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:55.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.694+0000 7f303d600640 1 -- 172.21.15.39:0/3536444086 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303403e030 con 0x7f304006f800 2024-09-21T07:52:55.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.694+0000 7f303d600640 1 -- 172.21.15.39:0/3536444086 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3034003e00 con 0x7f304006f800 2024-09-21T07:52:55.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.694+0000 7f303d600640 1 -- 172.21.15.39:0/3536444086 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3034035040 con 0x7f304006f800 2024-09-21T07:52:55.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.694+0000 7f3047e00640 1 -- 172.21.15.39:0/3536444086 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30401f9d20 con 0x7f304006f800 2024-09-21T07:52:55.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.694+0000 7f3047e00640 1 -- 172.21.15.39:0/3536444086 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30400b01d0 con 0x7f304006f800 2024-09-21T07:52:55.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.696+0000 7f3047e00640 1 -- 172.21.15.39:0/3536444086 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3040073db0 con 0x7f304006f800 2024-09-21T07:52:55.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.697+0000 7f303d600640 1 -- 172.21.15.39:0/3536444086 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3034041030 con 0x7f304006f800 2024-09-21T07:52:55.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.698+0000 7f303d600640 1 --2- 172.21.15.39:0/3536444086 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3020078b90 0x7f302007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:55.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.698+0000 7f303d600640 1 -- 172.21.15.39:0/3536444086 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(247..247 src has 1..247) ==== 7605+0+0 (secure 0 0 0) 0x7f30340bbc40 con 0x7f304006f800 2024-09-21T07:52:55.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.698+0000 7f3044a00640 1 --2- 172.21.15.39:0/3536444086 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3020078b90 0x7f302007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:55.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.699+0000 7f3044a00640 1 --2- 172.21.15.39:0/3536444086 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3020078b90 0x7f302007b0a0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f3040154b50 tx=0x7f302c00b390 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:55.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.703+0000 7f303d600640 1 -- 172.21.15.39:0/3536444086 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f30340832b0 con 0x7f304006f800 2024-09-21T07:52:55.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.942+0000 7f3047e00640 1 -- 172.21.15.39:0/3536444086 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3040153500 con 0x7f304006f800 2024-09-21T07:52:55.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.942+0000 7f303d600640 1 -- 172.21.15.39:0/3536444086 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 282 v282) ==== 95+0+5283 (secure 0 0 0) 0x7f3034087ce0 con 0x7f304006f800 2024-09-21T07:52:55.943 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:52:55.944 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":282,"btime":"2024-09-21T07:52:43:132032+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":278}],"filesystems":[{"mdsmap":{"epoch":282,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:52:42.942455+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":246,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":10195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10195":{"gid":10195,"name":"d","rank":2,"incarnation":277,"state":"up:active","state_seq":25,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[11052,10195,9967]},"id":13}]} 2024-09-21T07:52:55.944 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 282 2024-09-21T07:52:55.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.945+0000 7f3047e00640 1 -- 172.21.15.39:0/3536444086 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3020078b90 msgr2=0x7f302007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:55.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.945+0000 7f3047e00640 1 --2- 172.21.15.39:0/3536444086 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3020078b90 0x7f302007b0a0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f3040154b50 tx=0x7f302c00b390 comp rx=0 tx=0).stop 2024-09-21T07:52:55.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.945+0000 7f3047e00640 1 -- 172.21.15.39:0/3536444086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f304006f800 msgr2=0x7f304014ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:55.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.945+0000 7f3047e00640 1 --2- 172.21.15.39:0/3536444086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f304006f800 0x7f304014ed30 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7f3034034040 tx=0x7f3034003910 comp rx=0 tx=0).stop 2024-09-21T07:52:55.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.945+0000 7f3047e00640 1 -- 172.21.15.39:0/3536444086 shutdown_connections 2024-09-21T07:52:55.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.945+0000 7f3047e00640 1 --2- 172.21.15.39:0/3536444086 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3020078b90 0x7f302007b0a0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:55.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.945+0000 7f3047e00640 1 --2- 172.21.15.39:0/3536444086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f304015cd30 0x7f30401f9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:55.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.945+0000 7f3047e00640 1 --2- 172.21.15.39:0/3536444086 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f304006fc10 0x7f30401f8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:55.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.945+0000 7f3047e00640 1 --2- 172.21.15.39:0/3536444086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f304006f800 0x7f304014ed30 unknown :-1 s=CLOSED pgs=2577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:55.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.945+0000 7f3047e00640 1 -- 172.21.15.39:0/3536444086 >> 172.21.15.39:0/3536444086 conn(0x7f304006cfc0 msgr2=0x7f304015de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:55.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.945+0000 7f3047e00640 1 -- 172.21.15.39:0/3536444086 shutdown_connections 2024-09-21T07:52:55.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:52:55.945+0000 7f3047e00640 1 -- 172.21.15.39:0/3536444086 wait complete. 2024-09-21T07:52:55.970 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:52:58.591 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~13s 2024-09-21T07:53:00.971 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:53:01.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.208+0000 7f3b6be00640 1 -- 172.21.15.39:0/846803900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b6c0ce310 msgr2=0x7f3b6c0ce720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:01.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.208+0000 7f3b6be00640 1 --2- 172.21.15.39:0/846803900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b6c0ce310 0x7f3b6c0ce720 secure :-1 s=READY pgs=2578 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c0710f0 tx=0x7f3b5802a4d0 comp rx=0 tx=0).stop 2024-09-21T07:53:01.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.209+0000 7f3b6be00640 1 -- 172.21.15.39:0/846803900 shutdown_connections 2024-09-21T07:53:01.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.209+0000 7f3b6be00640 1 --2- 172.21.15.39:0/846803900 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b6c06f000 0x7f3b6c06f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:01.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.209+0000 7f3b6be00640 1 --2- 172.21.15.39:0/846803900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b6c0ce310 0x7f3b6c0ce720 unknown :-1 s=CLOSED pgs=2578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:01.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.209+0000 7f3b6be00640 1 --2- 172.21.15.39:0/846803900 >> v2:172.21.15.148:3300/0 conn(0x7f3b6c0cc390 0x7f3b6c0af600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:01.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.209+0000 7f3b6be00640 1 -- 172.21.15.39:0/846803900 >> 172.21.15.39:0/846803900 conn(0x7f3b6c0b2510 msgr2=0x7f3b6c0d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:01.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.209+0000 7f3b6be00640 1 -- 172.21.15.39:0/846803900 shutdown_connections 2024-09-21T07:53:01.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.209+0000 7f3b6be00640 1 -- 172.21.15.39:0/846803900 wait complete. 2024-09-21T07:53:01.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.211+0000 7f3b6be00640 1 Processor -- start 2024-09-21T07:53:01.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.211+0000 7f3b6be00640 1 -- start start 2024-09-21T07:53:01.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.212+0000 7f3b6be00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b6c06f000 0x7f3b6c099a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:01.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.213+0000 7f3b6be00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b6c0cc390 0x7f3b6c15e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:01.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.213+0000 7f3b6be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b6c0ce310 0x7f3b6c072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:01.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.213+0000 7f3b6be00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3b6c06fe80 con 0x7f3b6c0ce310 2024-09-21T07:53:01.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.213+0000 7f3b6be00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3b6c06fd00 con 0x7f3b6c06f000 2024-09-21T07:53:01.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.213+0000 7f3b6be00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f3b6c070000 con 0x7f3b6c0cc390 2024-09-21T07:53:01.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.213+0000 7f3b6b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b6c0ce310 0x7f3b6c072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:01.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.213+0000 7f3b6b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b6c0ce310 0x7f3b6c072df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41396/0 (socket says 172.21.15.39:41396) 2024-09-21T07:53:01.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.213+0000 7f3b6b400640 1 -- 172.21.15.39:0/2446966306 learned_addr learned my addr 172.21.15.39:0/2446966306 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:01.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.213+0000 7f3b6aa00640 1 --2- 172.21.15.39:0/2446966306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b6c06f000 0x7f3b6c099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:01.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.213+0000 7f3b6a000640 1 --2- 172.21.15.39:0/2446966306 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b6c0cc390 0x7f3b6c15e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:01.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.213+0000 7f3b6b400640 1 -- 172.21.15.39:0/2446966306 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b6c0cc390 msgr2=0x7f3b6c15e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:01.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.213+0000 7f3b6b400640 1 --2- 172.21.15.39:0/2446966306 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b6c0cc390 0x7f3b6c15e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:01.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.213+0000 7f3b6b400640 1 -- 172.21.15.39:0/2446966306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b6c06f000 msgr2=0x7f3b6c099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:01.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.213+0000 7f3b6b400640 1 --2- 172.21.15.39:0/2446966306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b6c06f000 0x7f3b6c099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:01.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.213+0000 7f3b6b400640 1 -- 172.21.15.39:0/2446966306 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b6c074a50 con 0x7f3b6c0ce310 2024-09-21T07:53:01.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.213+0000 7f3b6aa00640 1 --2- 172.21.15.39:0/2446966306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b6c06f000 0x7f3b6c099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:53:01.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.214+0000 7f3b6b400640 1 --2- 172.21.15.39:0/2446966306 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b6c0ce310 0x7f3b6c072df0 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7f3b6400d9b0 tx=0x7f3b6400de70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:01.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.214+0000 7f3b62a00640 1 -- 172.21.15.39:0/2446966306 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b64016070 con 0x7f3b6c0ce310 2024-09-21T07:53:01.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.214+0000 7f3b62a00640 1 -- 172.21.15.39:0/2446966306 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b6400bce0 con 0x7f3b6c0ce310 2024-09-21T07:53:01.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.214+0000 7f3b62a00640 1 -- 172.21.15.39:0/2446966306 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b64003bd0 con 0x7f3b6c0ce310 2024-09-21T07:53:01.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.214+0000 7f3b6be00640 1 -- 172.21.15.39:0/2446966306 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b6c0b1a80 con 0x7f3b6c0ce310 2024-09-21T07:53:01.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.214+0000 7f3b6be00640 1 -- 172.21.15.39:0/2446966306 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b6c15b320 con 0x7f3b6c0ce310 2024-09-21T07:53:01.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.215+0000 7f3b6be00640 1 -- 172.21.15.39:0/2446966306 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b6c0af600 con 0x7f3b6c0ce310 2024-09-21T07:53:01.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.216+0000 7f3b62a00640 1 -- 172.21.15.39:0/2446966306 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3b64010460 con 0x7f3b6c0ce310 2024-09-21T07:53:01.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.217+0000 7f3b62a00640 1 --2- 172.21.15.39:0/2446966306 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3b4c078b90 0x7f3b4c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:01.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.218+0000 7f3b62a00640 1 -- 172.21.15.39:0/2446966306 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(247..247 src has 1..247) ==== 7605+0+0 (secure 0 0 0) 0x7f3b6409ba70 con 0x7f3b6c0ce310 2024-09-21T07:53:01.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.218+0000 7f3b6aa00640 1 --2- 172.21.15.39:0/2446966306 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3b4c078b90 0x7f3b4c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:01.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.219+0000 7f3b6aa00640 1 --2- 172.21.15.39:0/2446966306 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3b4c078b90 0x7f3b4c07b0a0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c009b80 tx=0x7f3b5c005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:01.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.222+0000 7f3b62a00640 1 -- 172.21.15.39:0/2446966306 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3b6c074a50 con 0x7f3b6c0ce310 2024-09-21T07:53:01.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.457+0000 7f3b6be00640 1 -- 172.21.15.39:0/2446966306 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b6c0cb740 con 0x7f3b6c0ce310 2024-09-21T07:53:01.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.457+0000 7f3b62a00640 1 -- 172.21.15.39:0/2446966306 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 282 v282) ==== 95+0+5283 (secure 0 0 0) 0x7f3b640630e0 con 0x7f3b6c0ce310 2024-09-21T07:53:01.459 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:53:01.459 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":282,"btime":"2024-09-21T07:52:43:132032+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":278}],"filesystems":[{"mdsmap":{"epoch":282,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:52:42.942455+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":246,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":10195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10195":{"gid":10195,"name":"d","rank":2,"incarnation":277,"state":"up:active","state_seq":25,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[11052,10195,9967]},"id":13}]} 2024-09-21T07:53:01.459 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 282 2024-09-21T07:53:01.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.460+0000 7f3b6be00640 1 -- 172.21.15.39:0/2446966306 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3b4c078b90 msgr2=0x7f3b4c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:01.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.460+0000 7f3b6be00640 1 --2- 172.21.15.39:0/2446966306 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3b4c078b90 0x7f3b4c07b0a0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c009b80 tx=0x7f3b5c005f90 comp rx=0 tx=0).stop 2024-09-21T07:53:01.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.460+0000 7f3b6be00640 1 -- 172.21.15.39:0/2446966306 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b6c0ce310 msgr2=0x7f3b6c072df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:01.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.460+0000 7f3b6be00640 1 --2- 172.21.15.39:0/2446966306 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b6c0ce310 0x7f3b6c072df0 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7f3b6400d9b0 tx=0x7f3b6400de70 comp rx=0 tx=0).stop 2024-09-21T07:53:01.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.460+0000 7f3b6be00640 1 -- 172.21.15.39:0/2446966306 shutdown_connections 2024-09-21T07:53:01.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.460+0000 7f3b6be00640 1 --2- 172.21.15.39:0/2446966306 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3b4c078b90 0x7f3b4c07b0a0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:01.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.460+0000 7f3b6be00640 1 --2- 172.21.15.39:0/2446966306 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b6c0ce310 0x7f3b6c072df0 unknown :-1 s=CLOSED pgs=2579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:01.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.460+0000 7f3b6be00640 1 --2- 172.21.15.39:0/2446966306 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3b6c0cc390 0x7f3b6c15e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:01.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.460+0000 7f3b6be00640 1 --2- 172.21.15.39:0/2446966306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b6c06f000 0x7f3b6c099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:01.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.460+0000 7f3b6be00640 1 -- 172.21.15.39:0/2446966306 >> 172.21.15.39:0/2446966306 conn(0x7f3b6c0b2510 msgr2=0x7f3b6c06cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:01.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.460+0000 7f3b6be00640 1 -- 172.21.15.39:0/2446966306 shutdown_connections 2024-09-21T07:53:01.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:01.460+0000 7f3b6be00640 1 -- 172.21.15.39:0/2446966306 wait complete. 2024-09-21T07:53:01.485 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:53:03.947 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:53:03.951 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:53:05.202 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~20s 2024-09-21T07:53:06.486 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:53:06.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.722+0000 7fcb64a00640 1 -- 172.21.15.39:0/2032110889 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb60161fe0 msgr2=0x7fcb6006f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:06.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.722+0000 7fcb64a00640 1 --2- 172.21.15.39:0/2032110889 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb60161fe0 0x7fcb6006f7d0 secure :-1 s=READY pgs=2581 cs=0 l=1 rev1=1 crypto rx=0x7fcb6008ede0 tx=0x7fcb4c02a5f0 comp rx=0 tx=0).stop 2024-09-21T07:53:06.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.722+0000 7fcb64a00640 1 -- 172.21.15.39:0/2032110889 shutdown_connections 2024-09-21T07:53:06.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.722+0000 7fcb64a00640 1 --2- 172.21.15.39:0/2032110889 >> v2:172.21.15.148:3300/0 conn(0x7fcb600d0d20 0x7fcb600d1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:06.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.722+0000 7fcb64a00640 1 --2- 172.21.15.39:0/2032110889 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb60161fe0 0x7fcb6006f7d0 unknown :-1 s=CLOSED pgs=2581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:06.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.722+0000 7fcb64a00640 1 -- 172.21.15.39:0/2032110889 >> 172.21.15.39:0/2032110889 conn(0x7fcb6008bfb0 msgr2=0x7fcb6009d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:06.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.722+0000 7fcb64a00640 1 -- 172.21.15.39:0/2032110889 shutdown_connections 2024-09-21T07:53:06.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.722+0000 7fcb64a00640 1 -- 172.21.15.39:0/2032110889 wait complete. 2024-09-21T07:53:06.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.724+0000 7fcb64a00640 1 Processor -- start 2024-09-21T07:53:06.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.724+0000 7fcb64a00640 1 -- start start 2024-09-21T07:53:06.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.726+0000 7fcb64a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb600b96b0 0x7fcb600ca530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:06.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.726+0000 7fcb64a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcb600d0d20 0x7fcb600caa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:06.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.726+0000 7fcb64a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb60161fe0 0x7fcb601f6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:06.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.726+0000 7fcb64a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcb600d26a0 con 0x7fcb600b96b0 2024-09-21T07:53:06.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.726+0000 7fcb64a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcb600d2520 con 0x7fcb60161fe0 2024-09-21T07:53:06.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.726+0000 7fcb64a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fcb600d2820 con 0x7fcb600d0d20 2024-09-21T07:53:06.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.726+0000 7fcb5e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb600b96b0 0x7fcb600ca530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:06.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.726+0000 7fcb5e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb600b96b0 0x7fcb600ca530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57350/0 (socket says 172.21.15.39:57350) 2024-09-21T07:53:06.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.726+0000 7fcb5e800640 1 -- 172.21.15.39:0/3618081684 learned_addr learned my addr 172.21.15.39:0/3618081684 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:06.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.726+0000 7fcb5f200640 1 --2- 172.21.15.39:0/3618081684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb60161fe0 0x7fcb601f6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:06.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.726+0000 7fcb5de00640 1 --2- 172.21.15.39:0/3618081684 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcb600d0d20 0x7fcb600caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:06.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.727+0000 7fcb5e800640 1 -- 172.21.15.39:0/3618081684 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcb600d0d20 msgr2=0x7fcb600caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:06.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.727+0000 7fcb5e800640 1 --2- 172.21.15.39:0/3618081684 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcb600d0d20 0x7fcb600caa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:06.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.727+0000 7fcb5e800640 1 -- 172.21.15.39:0/3618081684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb60161fe0 msgr2=0x7fcb601f6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:06.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.727+0000 7fcb5e800640 1 --2- 172.21.15.39:0/3618081684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb60161fe0 0x7fcb601f6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:06.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.727+0000 7fcb5e800640 1 -- 172.21.15.39:0/3618081684 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb601f6210 con 0x7fcb600b96b0 2024-09-21T07:53:06.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.727+0000 7fcb5e800640 1 --2- 172.21.15.39:0/3618081684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb600b96b0 0x7fcb600ca530 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000c3f0 tx=0x7fcb5000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:06.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.727+0000 7fcb56a00640 1 -- 172.21.15.39:0/3618081684 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb50017070 con 0x7fcb600b96b0 2024-09-21T07:53:06.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.727+0000 7fcb56a00640 1 -- 172.21.15.39:0/3618081684 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb500035f0 con 0x7fcb600b96b0 2024-09-21T07:53:06.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.727+0000 7fcb56a00640 1 -- 172.21.15.39:0/3618081684 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb50003a40 con 0x7fcb600b96b0 2024-09-21T07:53:06.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.727+0000 7fcb64a00640 1 -- 172.21.15.39:0/3618081684 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb601f51e0 con 0x7fcb600b96b0 2024-09-21T07:53:06.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.727+0000 7fcb64a00640 1 -- 172.21.15.39:0/3618081684 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb601f5450 con 0x7fcb600b96b0 2024-09-21T07:53:06.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.729+0000 7fcb64a00640 1 -- 172.21.15.39:0/3618081684 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb600d2cd0 con 0x7fcb600b96b0 2024-09-21T07:53:06.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.730+0000 7fcb56a00640 1 -- 172.21.15.39:0/3618081684 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcb5001a070 con 0x7fcb600b96b0 2024-09-21T07:53:06.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.731+0000 7fcb56a00640 1 --2- 172.21.15.39:0/3618081684 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcb40078bc0 0x7fcb4007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:06.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.732+0000 7fcb56a00640 1 -- 172.21.15.39:0/3618081684 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(247..247 src has 1..247) ==== 7605+0+0 (secure 0 0 0) 0x7fcb5009a750 con 0x7fcb600b96b0 2024-09-21T07:53:06.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.732+0000 7fcb5de00640 1 --2- 172.21.15.39:0/3618081684 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcb40078bc0 0x7fcb4007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:06.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.733+0000 7fcb5de00640 1 --2- 172.21.15.39:0/3618081684 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcb40078bc0 0x7fcb4007b0d0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c033000 tx=0x7fcb4c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:06.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.736+0000 7fcb56a00640 1 -- 172.21.15.39:0/3618081684 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcb50061d30 con 0x7fcb600b96b0 2024-09-21T07:53:06.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.973+0000 7fcb64a00640 1 -- 172.21.15.39:0/3618081684 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb600c8880 con 0x7fcb600b96b0 2024-09-21T07:53:06.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.974+0000 7fcb56a00640 1 -- 172.21.15.39:0/3618081684 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 283 v283) ==== 95+0+5331 (secure 0 0 0) 0x7fcb50066760 con 0x7fcb600b96b0 2024-09-21T07:53:06.976 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:53:06.976 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":283,"btime":"2024-09-21T07:53:01:593143+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":278}],"filesystems":[{"mdsmap":{"epoch":283,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:53:01.593117+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":246,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":10195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10195":{"gid":10195,"name":"d","rank":2,"incarnation":277,"state":"up:active","state_seq":25,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"laggy_since":"2024-09-21T07:53:01.593117+0000","export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[9967,10195,11052]},"id":13}]} 2024-09-21T07:53:06.976 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 283 2024-09-21T07:53:06.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.976+0000 7fcb64a00640 1 -- 172.21.15.39:0/3618081684 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcb40078bc0 msgr2=0x7fcb4007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:06.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.976+0000 7fcb64a00640 1 --2- 172.21.15.39:0/3618081684 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcb40078bc0 0x7fcb4007b0d0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c033000 tx=0x7fcb4c004730 comp rx=0 tx=0).stop 2024-09-21T07:53:06.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.976+0000 7fcb64a00640 1 -- 172.21.15.39:0/3618081684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb600b96b0 msgr2=0x7fcb600ca530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:06.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.976+0000 7fcb64a00640 1 --2- 172.21.15.39:0/3618081684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb600b96b0 0x7fcb600ca530 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000c3f0 tx=0x7fcb5000c8b0 comp rx=0 tx=0).stop 2024-09-21T07:53:06.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.976+0000 7fcb64a00640 1 -- 172.21.15.39:0/3618081684 shutdown_connections 2024-09-21T07:53:06.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.976+0000 7fcb64a00640 1 --2- 172.21.15.39:0/3618081684 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcb40078bc0 0x7fcb4007b0d0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:06.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.976+0000 7fcb64a00640 1 --2- 172.21.15.39:0/3618081684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb60161fe0 0x7fcb601f6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:06.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.976+0000 7fcb64a00640 1 --2- 172.21.15.39:0/3618081684 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcb600d0d20 0x7fcb600caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:06.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.976+0000 7fcb64a00640 1 --2- 172.21.15.39:0/3618081684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb600b96b0 0x7fcb600ca530 unknown :-1 s=CLOSED pgs=2582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:06.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.976+0000 7fcb64a00640 1 -- 172.21.15.39:0/3618081684 >> 172.21.15.39:0/3618081684 conn(0x7fcb6008bfb0 msgr2=0x7fcb6015ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:06.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.976+0000 7fcb64a00640 1 -- 172.21.15.39:0/3618081684 shutdown_connections 2024-09-21T07:53:06.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:06.976+0000 7fcb64a00640 1 -- 172.21.15.39:0/3618081684 wait complete. 2024-09-21T07:53:07.004 DEBUG:tasks.ceph_test_case:wait_until_true: success in 20s and 0 retries 2024-09-21T07:53:07.005 DEBUG:teuthology.orchestra.run.smithi148:> sudo sysctl -n kernel.core_pattern 2024-09-21T07:53:07.063 INFO:teuthology.orchestra.run.smithi148.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T07:53:07.065 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2024-09-21T07:53:07.066 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2024-09-21T07:53:07.131 INFO:teuthology.orchestra.run.smithi148.stdout:1726905164.46199.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i d', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2024-09-21T07:53:07.135 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1726905164.46199.core'] 2024-09-21T07:53:07.135 INFO:tasks.cephfs.cephfs_test_case:Found core file 1726905164.46199.core, deleting it 2024-09-21T07:53:07.135 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1726905164.46199.core 2024-09-21T07:53:07.227 INFO:tasks.cephfs.mount:Suspending the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:53:07.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:53:07.227 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 down 2024-09-21T07:53:07.291 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:53:07.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.516+0000 7f47d0a00640 1 -- 172.21.15.39:0/2726635750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47cc0af490 msgr2=0x7f47cc06f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:07.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.516+0000 7f47d0a00640 1 --2- 172.21.15.39:0/2726635750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47cc0af490 0x7f47cc06f6c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f47c4005fd0 tx=0x7f47c402a780 comp rx=0 tx=0).stop 2024-09-21T07:53:07.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.516+0000 7f47d0a00640 1 -- 172.21.15.39:0/2726635750 shutdown_connections 2024-09-21T07:53:07.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.516+0000 7f47d0a00640 1 --2- 172.21.15.39:0/2726635750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47cc0af490 0x7f47cc06f6c0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:07.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.516+0000 7f47d0a00640 1 --2- 172.21.15.39:0/2726635750 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47cc0b43f0 0x7f47cc0ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:07.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.516+0000 7f47d0a00640 1 -- 172.21.15.39:0/2726635750 >> 172.21.15.39:0/2726635750 conn(0x7f47cc0b2510 msgr2=0x7f47cc0d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:07.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.516+0000 7f47d0a00640 1 -- 172.21.15.39:0/2726635750 shutdown_connections 2024-09-21T07:53:07.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.516+0000 7f47d0a00640 1 -- 172.21.15.39:0/2726635750 wait complete. 2024-09-21T07:53:07.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.518+0000 7f47d0a00640 1 Processor -- start 2024-09-21T07:53:07.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.519+0000 7f47d0a00640 1 -- start start 2024-09-21T07:53:07.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.520+0000 7f47d0a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47cc0af490 0x7f47cc0ca5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:07.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.520+0000 7f47d0a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47cc0b43f0 0x7f47cc0733f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:07.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.520+0000 7f47d0a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47cc15abe0 0x7f47cc09da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:07.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.520+0000 7f47d0a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc0cfba0 con 0x7f47cc0b43f0 2024-09-21T07:53:07.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.520+0000 7f47d0a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc0cfd20 con 0x7f47cc15abe0 2024-09-21T07:53:07.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.520+0000 7f47d0a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f47cc0cfea0 con 0x7f47cc0af490 2024-09-21T07:53:07.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.520+0000 7f47ca800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47cc0af490 0x7f47cc0ca5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:07.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.520+0000 7f47c3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47cc0b43f0 0x7f47cc0733f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:07.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.520+0000 7f47ca800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47cc0af490 0x7f47cc0ca5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.39:38906/0 (socket says 172.21.15.39:38906) 2024-09-21T07:53:07.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.520+0000 7f47ca800640 1 -- 172.21.15.39:0/2834640211 learned_addr learned my addr 172.21.15.39:0/2834640211 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:07.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.520+0000 7f47cb200640 1 --2- 172.21.15.39:0/2834640211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47cc15abe0 0x7f47cc09da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:07.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.521+0000 7f47ca800640 1 -- 172.21.15.39:0/2834640211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47cc15abe0 msgr2=0x7f47cc09da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:07.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.521+0000 7f47ca800640 1 --2- 172.21.15.39:0/2834640211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47cc15abe0 0x7f47cc09da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:07.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.521+0000 7f47ca800640 1 -- 172.21.15.39:0/2834640211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47cc0b43f0 msgr2=0x7f47cc0733f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:07.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.521+0000 7f47ca800640 1 --2- 172.21.15.39:0/2834640211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47cc0b43f0 0x7f47cc0733f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:07.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.521+0000 7f47ca800640 1 -- 172.21.15.39:0/2834640211 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47cc0b10b0 con 0x7f47cc0af490 2024-09-21T07:53:07.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.521+0000 7f47c3e00640 1 --2- 172.21.15.39:0/2834640211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47cc0b43f0 0x7f47cc0733f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:53:07.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.521+0000 7f47ca800640 1 --2- 172.21.15.39:0/2834640211 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47cc0af490 0x7f47cc0ca5e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00ea20 tx=0x7f47bc00eee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:07.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.521+0000 7f47c2a00640 1 -- 172.21.15.39:0/2834640211 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc00ce40 con 0x7f47cc0af490 2024-09-21T07:53:07.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.522+0000 7f47c2a00640 1 -- 172.21.15.39:0/2834640211 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47bc004020 con 0x7f47cc0af490 2024-09-21T07:53:07.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.522+0000 7f47d0a00640 1 -- 172.21.15.39:0/2834640211 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47cc0b1390 con 0x7f47cc0af490 2024-09-21T07:53:07.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.522+0000 7f47c2a00640 1 -- 172.21.15.39:0/2834640211 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc004470 con 0x7f47cc0af490 2024-09-21T07:53:07.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.522+0000 7f47d0a00640 1 -- 172.21.15.39:0/2834640211 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f47cc0cab20 con 0x7f47cc0af490 2024-09-21T07:53:07.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.523+0000 7f47d0a00640 1 -- 172.21.15.39:0/2834640211 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47cc06f6c0 con 0x7f47cc0af490 2024-09-21T07:53:07.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.524+0000 7f47c2a00640 1 -- 172.21.15.39:0/2834640211 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f47bc03bd10 con 0x7f47cc0af490 2024-09-21T07:53:07.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.525+0000 7f47c2a00640 1 --2- 172.21.15.39:0/2834640211 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f47a8078b90 0x7f47a807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:07.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.525+0000 7f47c2a00640 1 -- 172.21.15.39:0/2834640211 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(247..247 src has 1..247) ==== 7605+0+0 (secure 0 0 0) 0x7f47bc09bd80 con 0x7f47cc0af490 2024-09-21T07:53:07.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.525+0000 7f47c3e00640 1 --2- 172.21.15.39:0/2834640211 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f47a8078b90 0x7f47a807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:07.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.526+0000 7f47c3e00640 1 --2- 172.21.15.39:0/2834640211 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f47a8078b90 0x7f47a807b0a0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f47cc0cc3e0 tx=0x7f47b4007f70 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:07.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.528+0000 7f47c2a00640 1 -- 172.21.15.39:0/2834640211 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f47bc063500 con 0x7f47cc0af490 2024-09-21T07:53:07.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.777+0000 7f47d0a00640 1 -- 172.21.15.39:0/2834640211 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47cc0753d0 con 0x7f47cc0af490 2024-09-21T07:53:07.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.778+0000 7f47c2a00640 1 -- 172.21.15.39:0/2834640211 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 283 v283) ==== 95+0+5331 (secure 0 0 0) 0x7f47bc067f30 con 0x7f47cc0af490 2024-09-21T07:53:07.781 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:53:07.781 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":283,"btime":"2024-09-21T07:53:01:593143+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11364,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":278}],"filesystems":[{"mdsmap":{"epoch":283,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:53:01.593117+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":246,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":9967,"mds_1":11052,"mds_2":10195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9967":{"gid":9967,"name":"b","rank":0,"incarnation":259,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6836/3381818843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3381818843},{"type":"v1","addr":"172.21.15.148:6836","nonce":3381818843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_10195":{"gid":10195,"name":"d","rank":2,"incarnation":277,"state":"up:active","state_seq":25,"addr":"172.21.15.148:6837/597301351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":597301351},{"type":"v1","addr":"172.21.15.148:6837","nonce":597301351}]},"join_fscid":-1,"laggy_since":"2024-09-21T07:53:01.593117+0000","export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9967,"qdb_cluster":[9967,10195,11052]},"id":13}]} 2024-09-21T07:53:07.781 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 283 2024-09-21T07:53:07.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.781+0000 7f47d0a00640 1 -- 172.21.15.39:0/2834640211 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f47a8078b90 msgr2=0x7f47a807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:07.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.781+0000 7f47d0a00640 1 --2- 172.21.15.39:0/2834640211 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f47a8078b90 0x7f47a807b0a0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f47cc0cc3e0 tx=0x7f47b4007f70 comp rx=0 tx=0).stop 2024-09-21T07:53:07.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.781+0000 7f47d0a00640 1 -- 172.21.15.39:0/2834640211 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47cc0af490 msgr2=0x7f47cc0ca5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:07.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.781+0000 7f47d0a00640 1 --2- 172.21.15.39:0/2834640211 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47cc0af490 0x7f47cc0ca5e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00ea20 tx=0x7f47bc00eee0 comp rx=0 tx=0).stop 2024-09-21T07:53:07.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.781+0000 7f47d0a00640 1 -- 172.21.15.39:0/2834640211 shutdown_connections 2024-09-21T07:53:07.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.781+0000 7f47d0a00640 1 --2- 172.21.15.39:0/2834640211 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f47a8078b90 0x7f47a807b0a0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:07.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.781+0000 7f47d0a00640 1 --2- 172.21.15.39:0/2834640211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47cc15abe0 0x7f47cc09da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:07.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.781+0000 7f47d0a00640 1 --2- 172.21.15.39:0/2834640211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47cc0b43f0 0x7f47cc0733f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:07.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.781+0000 7f47d0a00640 1 --2- 172.21.15.39:0/2834640211 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f47cc0af490 0x7f47cc0ca5e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:07.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.781+0000 7f47d0a00640 1 -- 172.21.15.39:0/2834640211 >> 172.21.15.39:0/2834640211 conn(0x7f47cc0b2510 msgr2=0x7f47cc0ad590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:07.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.781+0000 7f47d0a00640 1 -- 172.21.15.39:0/2834640211 shutdown_connections 2024-09-21T07:53:07.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:07.781+0000 7f47d0a00640 1 -- 172.21.15.39:0/2834640211 wait complete. 2024-09-21T07:53:07.806 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json dump snaps --server 2024-09-21T07:53:08.045 INFO:teuthology.orchestra.run.smithi148.stdout:{"last_snap":9,"last_created":8,"last_destroyed":7,"pending_noop":[],"snaps":[{"snapid":8,"ino":1099511627780,"stamp":"2024-09-21T07:51:12.091986+0000","name":"s1","metadata":{}}],"need_to_purge":{},"pending_update":[{"version":20,"snapinfo":{"snapid":9,"ino":1099511627780,"stamp":"2024-09-21T07:52:44.191763+0000","name":"s2","metadata":{}}}],"pending_destroy":[]} 2024-09-21T07:53:08.046 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 8, "last_destroyed": 7, "last_snap": 9, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [ { "snapInfinityo": { "ino": 1099511627780, "metadata": {}, "name": "s2", "snapid": 9, "stamp": "2024-09-21T07:52:44.191763+0000" }, "version": 20 } ], "snaps": [ { "ino": 1099511627780, "metadata": {}, "name": "s1", "snapid": 8, "stamp": "2024-09-21T07:51:12.091986+0000" } ] } 2024-09-21T07:53:08.046 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 13:0 2024-09-21T07:53:08.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.272+0000 7fc96b800640 1 -- 172.21.15.39:0/2097420035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc94c003a30 msgr2=0x7fc94c003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:08.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.272+0000 7fc96b800640 1 --2- 172.21.15.39:0/2097420035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc94c003a30 0x7fc94c003ec0 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 crypto rx=0x7fc958006fd0 tx=0x7fc94c005830 comp rx=0 tx=0).stop 2024-09-21T07:53:08.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.273+0000 7fc969800640 1 -- 172.21.15.39:0/2097420035 reap_dead start 2024-09-21T07:53:08.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.273+0000 7fc96b800640 1 -- 172.21.15.39:0/2097420035 shutdown_connections 2024-09-21T07:53:08.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.273+0000 7fc96b800640 1 -- 172.21.15.39:0/2097420035 >> 172.21.15.39:0/2097420035 conn(0x7fc96406cfc0 msgr2=0x7fc96406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:08.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.273+0000 7fc96b800640 1 -- 172.21.15.39:0/2097420035 shutdown_connections 2024-09-21T07:53:08.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.273+0000 7fc96b800640 1 -- 172.21.15.39:0/2097420035 wait complete. 2024-09-21T07:53:08.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.275+0000 7fc96b800640 1 Processor -- start 2024-09-21T07:53:08.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.275+0000 7fc96b800640 1 -- start start 2024-09-21T07:53:08.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.276+0000 7fc96b800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc94c003a30 0x7fc964150990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:08.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.276+0000 7fc96b800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9640c96f0 0x7fc9640cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:08.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.277+0000 7fc96b800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc9640cb930 0x7fc9641597b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:08.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.277+0000 7fc96b800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc96415b5d0 con 0x7fc94c003a30 2024-09-21T07:53:08.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.277+0000 7fc96b800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc9640714a0 con 0x7fc9640c96f0 2024-09-21T07:53:08.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.277+0000 7fc96b800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc96415b750 con 0x7fc9640cb930 2024-09-21T07:53:08.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.277+0000 7fc968e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc94c003a30 0x7fc964150990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:08.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.277+0000 7fc968e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc94c003a30 0x7fc964150990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57388/0 (socket says 172.21.15.39:57388) 2024-09-21T07:53:08.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.277+0000 7fc968e00640 1 -- 172.21.15.39:0/2175045159 learned_addr learned my addr 172.21.15.39:0/2175045159 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:08.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.277+0000 7fc963e00640 1 --2- 172.21.15.39:0/2175045159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9640c96f0 0x7fc9640cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:08.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.277+0000 7fc969800640 1 --2- 172.21.15.39:0/2175045159 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc9640cb930 0x7fc9641597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:08.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.277+0000 7fc968e00640 1 -- 172.21.15.39:0/2175045159 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc9640cb930 msgr2=0x7fc9641597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:08.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.277+0000 7fc968e00640 1 --2- 172.21.15.39:0/2175045159 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc9640cb930 0x7fc9641597b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:08.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.277+0000 7fc968e00640 1 -- 172.21.15.39:0/2175045159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9640c96f0 msgr2=0x7fc9640cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:08.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.277+0000 7fc968e00640 1 --2- 172.21.15.39:0/2175045159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9640c96f0 0x7fc9640cb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:08.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.277+0000 7fc968e00640 1 -- 172.21.15.39:0/2175045159 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9640cbd40 con 0x7fc94c003a30 2024-09-21T07:53:08.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.278+0000 7fc968e00640 1 --2- 172.21.15.39:0/2175045159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc94c003a30 0x7fc964150990 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 crypto rx=0x7fc958000f60 tx=0x7fc958001d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:08.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.278+0000 7fc960c00640 1 -- 172.21.15.39:0/2175045159 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc958031040 con 0x7fc94c003a30 2024-09-21T07:53:08.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.278+0000 7fc960c00640 1 -- 172.21.15.39:0/2175045159 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc958003610 con 0x7fc94c003a30 2024-09-21T07:53:08.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.278+0000 7fc960c00640 1 -- 172.21.15.39:0/2175045159 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc958003a20 con 0x7fc94c003a30 2024-09-21T07:53:08.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.278+0000 7fc96b800640 1 -- 172.21.15.39:0/2175045159 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9641f9270 con 0x7fc94c003a30 2024-09-21T07:53:08.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.278+0000 7fc96b800640 1 -- 172.21.15.39:0/2175045159 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9641f9830 con 0x7fc94c003a30 2024-09-21T07:53:08.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.279+0000 7fc96b800640 1 -- 172.21.15.39:0/2175045159 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc96415bc70 con 0x7fc94c003a30 2024-09-21T07:53:08.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.280+0000 7fc960c00640 1 -- 172.21.15.39:0/2175045159 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc958034070 con 0x7fc94c003a30 2024-09-21T07:53:08.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.281+0000 7fc960c00640 1 --2- 172.21.15.39:0/2175045159 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc944078b90 0x7fc94407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:08.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.281+0000 7fc960c00640 1 -- 172.21.15.39:0/2175045159 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(247..247 src has 1..247) ==== 7605+0+0 (secure 0 0 0) 0x7fc9580bb440 con 0x7fc94c003a30 2024-09-21T07:53:08.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.282+0000 7fc963e00640 1 --2- 172.21.15.39:0/2175045159 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc944078b90 0x7fc94407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:08.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.282+0000 7fc963e00640 1 --2- 172.21.15.39:0/2175045159 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc944078b90 0x7fc94407b0a0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fc9640cc6e0 tx=0x7fc950006b70 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:08.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.286+0000 7fc960c00640 1 -- 172.21.15.39:0/2175045159 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc958082ab0 con 0x7fc94c003a30 2024-09-21T07:53:08.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.563+0000 7fc96b800640 1 -- 172.21.15.39:0/2175045159 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "13:0"} v 0) -- 0x7fc9641513e0 con 0x7fc94c003a30 2024-09-21T07:53:08.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.567+0000 7fc960c00640 1 -- 172.21.15.39:0/2175045159 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "13:0"}]=0 failed mds gid 9967 v284) ==== 98+0+0 (secure 0 0 0) 0x7fc9580874e0 con 0x7fc94c003a30 2024-09-21T07:53:08.568 INFO:teuthology.orchestra.run.smithi039.stderr:failed mds gid 9967 2024-09-21T07:53:08.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.570+0000 7fc96b800640 1 -- 172.21.15.39:0/2175045159 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc944078b90 msgr2=0x7fc94407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:08.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.570+0000 7fc96b800640 1 --2- 172.21.15.39:0/2175045159 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc944078b90 0x7fc94407b0a0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fc9640cc6e0 tx=0x7fc950006b70 comp rx=0 tx=0).stop 2024-09-21T07:53:08.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.570+0000 7fc96b800640 1 -- 172.21.15.39:0/2175045159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc94c003a30 msgr2=0x7fc964150990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:08.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.570+0000 7fc96b800640 1 --2- 172.21.15.39:0/2175045159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc94c003a30 0x7fc964150990 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 crypto rx=0x7fc958000f60 tx=0x7fc958001d90 comp rx=0 tx=0).stop 2024-09-21T07:53:08.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.572+0000 7fc96b800640 1 -- 172.21.15.39:0/2175045159 shutdown_connections 2024-09-21T07:53:08.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.572+0000 7fc96b800640 1 --2- 172.21.15.39:0/2175045159 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc944078b90 0x7fc94407b0a0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:08.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.572+0000 7fc96b800640 1 --2- 172.21.15.39:0/2175045159 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc9640cb930 0x7fc9641597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:08.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.572+0000 7fc96b800640 1 --2- 172.21.15.39:0/2175045159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9640c96f0 0x7fc9640cb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:08.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.572+0000 7fc96b800640 1 --2- 172.21.15.39:0/2175045159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc94c003a30 0x7fc964150990 unknown :-1 s=CLOSED pgs=2586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:08.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.572+0000 7fc96b800640 1 -- 172.21.15.39:0/2175045159 >> 172.21.15.39:0/2175045159 conn(0x7fc96406cfc0 msgr2=0x7fc9640cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:08.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.572+0000 7fc96b800640 1 -- 172.21.15.39:0/2175045159 shutdown_connections 2024-09-21T07:53:08.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.572+0000 7fc96b800640 1 -- 172.21.15.39:0/2175045159 wait complete. 2024-09-21T07:53:08.588 INFO:tasks.ceph.mds.b.smithi148.stderr: -8199> 2024-09-21T07:51:05.881+0000 7fa0be800640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-21T07:53:08.595 INFO:tasks.ceph.mds.b.smithi148.stderr: -3159> 2024-09-21T07:52:30.889+0000 7fa0be800640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-21T07:53:08.605 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 13:2 2024-09-21T07:53:08.715 INFO:tasks.ceph.mds.b.smithi148.stdout:starting mds.b at 2024-09-21T07:53:08.771 INFO:tasks.ceph.mds.c.smithi039.stderr:2024-09-21T07:53:08.769+0000 7f824ee00640 -1 --2- [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] >> [v2:172.21.15.148:6835/597301351,v1:172.21.15.148:6837/597301351] conn(0x5599fb2cd680 0x5599fb486680 unknown :-1 s=BANNER_CONNECTING pgs=14 cs=2 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.148:6835/597301351,v1:172.21.15.148:6837/597301351] is using msgr V1 protocol 2024-09-21T07:53:08.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.832+0000 7f56d8000640 1 -- 172.21.15.39:0/2232348140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56d00cfd10 msgr2=0x7f56d00cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:08.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.832+0000 7f56d8000640 1 --2- 172.21.15.39:0/2232348140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56d00cfd10 0x7f56d00cd8e0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f56bc009830 tx=0x7f56bc02a490 comp rx=0 tx=0).stop 2024-09-21T07:53:08.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.832+0000 7f56d8000640 1 -- 172.21.15.39:0/2232348140 shutdown_connections 2024-09-21T07:53:08.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.832+0000 7f56d8000640 1 --2- 172.21.15.39:0/2232348140 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56d00d0620 0x7f56d00d0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:08.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.832+0000 7f56d8000640 1 --2- 172.21.15.39:0/2232348140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56d00cfd10 0x7f56d00cd8e0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:08.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.832+0000 7f56d8000640 1 -- 172.21.15.39:0/2232348140 >> 172.21.15.39:0/2232348140 conn(0x7f56d006cfc0 msgr2=0x7f56d006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:08.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.832+0000 7f56d8000640 1 -- 172.21.15.39:0/2232348140 shutdown_connections 2024-09-21T07:53:08.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.832+0000 7f56d8000640 1 -- 172.21.15.39:0/2232348140 wait complete. 2024-09-21T07:53:08.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.835+0000 7f56d8000640 1 Processor -- start 2024-09-21T07:53:08.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.835+0000 7f56d8000640 1 -- start start 2024-09-21T07:53:08.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.836+0000 7f56d8000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56d00cfd10 0x7f56d01531c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:08.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.836+0000 7f56d8000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56d00d0620 0x7f56d014bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:08.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.836+0000 7f56d8000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56d015cd30 0x7f56d0157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:08.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.836+0000 7f56d8000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f56d0073890 con 0x7f56d00cfd10 2024-09-21T07:53:08.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.836+0000 7f56d8000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f56d0073710 con 0x7f56d015cd30 2024-09-21T07:53:08.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.836+0000 7f56d8000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f56d0073a10 con 0x7f56d00d0620 2024-09-21T07:53:08.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.837+0000 7f56d5600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56d00cfd10 0x7f56d01531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:08.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.837+0000 7f56d5600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56d00cfd10 0x7f56d01531c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57392/0 (socket says 172.21.15.39:57392) 2024-09-21T07:53:08.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.837+0000 7f56d5600640 1 -- 172.21.15.39:0/139527378 learned_addr learned my addr 172.21.15.39:0/139527378 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:08.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.837+0000 7f56d6000640 1 --2- 172.21.15.39:0/139527378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56d015cd30 0x7f56d0157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:08.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.837+0000 7f56d4c00640 1 --2- 172.21.15.39:0/139527378 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56d00d0620 0x7f56d014bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:08.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.837+0000 7f56d5600640 1 -- 172.21.15.39:0/139527378 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56d00d0620 msgr2=0x7f56d014bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:08.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.837+0000 7f56d5600640 1 --2- 172.21.15.39:0/139527378 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56d00d0620 0x7f56d014bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:08.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.837+0000 7f56d5600640 1 -- 172.21.15.39:0/139527378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56d015cd30 msgr2=0x7f56d0157360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:08.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.837+0000 7f56d5600640 1 --2- 172.21.15.39:0/139527378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56d015cd30 0x7f56d0157360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:08.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.837+0000 7f56d5600640 1 -- 172.21.15.39:0/139527378 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d00b2d90 con 0x7f56d00cfd10 2024-09-21T07:53:08.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.837+0000 7f56d6000640 1 --2- 172.21.15.39:0/139527378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56d015cd30 0x7f56d0157360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:53:08.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.837+0000 7f56d4c00640 1 --2- 172.21.15.39:0/139527378 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56d00d0620 0x7f56d014bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:53:08.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.837+0000 7f56d5600640 1 --2- 172.21.15.39:0/139527378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56d00cfd10 0x7f56d01531c0 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 crypto rx=0x7f56c400c500 tx=0x7f56c400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:08.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.838+0000 7f56cd600640 1 -- 172.21.15.39:0/139527378 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c4016020 con 0x7f56d00cfd10 2024-09-21T07:53:08.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.838+0000 7f56cd600640 1 -- 172.21.15.39:0/139527378 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56c4012940 con 0x7f56d00cfd10 2024-09-21T07:53:08.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.838+0000 7f56cd600640 1 -- 172.21.15.39:0/139527378 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c4012d90 con 0x7f56d00cfd10 2024-09-21T07:53:08.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.838+0000 7f56d8000640 1 -- 172.21.15.39:0/139527378 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56d00b2210 con 0x7f56d00cfd10 2024-09-21T07:53:08.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.838+0000 7f56d8000640 1 -- 172.21.15.39:0/139527378 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56d00b27d0 con 0x7f56d00cfd10 2024-09-21T07:53:08.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.839+0000 7f56d8000640 1 -- 172.21.15.39:0/139527378 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56d00ced60 con 0x7f56d00cfd10 2024-09-21T07:53:08.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.840+0000 7f56cd600640 1 -- 172.21.15.39:0/139527378 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f56c4019020 con 0x7f56d00cfd10 2024-09-21T07:53:08.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.841+0000 7f56cd600640 1 --2- 172.21.15.39:0/139527378 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f56b0078b90 0x7f56b007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:08.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.841+0000 7f56cd600640 1 -- 172.21.15.39:0/139527378 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(248..248 src has 1..248) ==== 7691+0+0 (secure 0 0 0) 0x7f56c409b300 con 0x7f56d00cfd10 2024-09-21T07:53:08.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.842+0000 7f56d4c00640 1 --2- 172.21.15.39:0/139527378 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f56b0078b90 0x7f56b007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:08.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.842+0000 7f56d4c00640 1 --2- 172.21.15.39:0/139527378 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f56b0078b90 0x7f56b007b0a0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f56bc034000 tx=0x7f56bc004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:08.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:08.846+0000 7f56cd600640 1 -- 172.21.15.39:0/139527378 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f56d00ced60 con 0x7f56d00cfd10 2024-09-21T07:53:09.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.118+0000 7f56d8000640 1 -- 172.21.15.39:0/139527378 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "13:2"} v 0) -- 0x7f56d0073f90 con 0x7f56d00cfd10 2024-09-21T07:53:09.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.121+0000 7f56cd600640 1 -- 172.21.15.39:0/139527378 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "13:2"}]=0 failed mds gid 10195 v286) ==== 99+0+0 (secure 0 0 0) 0x7f56c4020030 con 0x7f56d00cfd10 2024-09-21T07:53:09.123 INFO:teuthology.orchestra.run.smithi039.stderr:failed mds gid 10195 2024-09-21T07:53:09.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.125+0000 7f56d8000640 1 -- 172.21.15.39:0/139527378 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f56b0078b90 msgr2=0x7f56b007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:09.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.125+0000 7f56d8000640 1 --2- 172.21.15.39:0/139527378 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f56b0078b90 0x7f56b007b0a0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f56bc034000 tx=0x7f56bc004730 comp rx=0 tx=0).stop 2024-09-21T07:53:09.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.125+0000 7f56d8000640 1 -- 172.21.15.39:0/139527378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56d00cfd10 msgr2=0x7f56d01531c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:09.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.125+0000 7f56d8000640 1 --2- 172.21.15.39:0/139527378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56d00cfd10 0x7f56d01531c0 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 crypto rx=0x7f56c400c500 tx=0x7f56c400c9c0 comp rx=0 tx=0).stop 2024-09-21T07:53:09.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.125+0000 7f56d8000640 1 -- 172.21.15.39:0/139527378 shutdown_connections 2024-09-21T07:53:09.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.125+0000 7f56d8000640 1 --2- 172.21.15.39:0/139527378 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f56b0078b90 0x7f56b007b0a0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:09.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.125+0000 7f56d8000640 1 --2- 172.21.15.39:0/139527378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56d015cd30 0x7f56d0157360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:09.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.125+0000 7f56d8000640 1 --2- 172.21.15.39:0/139527378 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56d00d0620 0x7f56d014bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:09.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.125+0000 7f56d8000640 1 --2- 172.21.15.39:0/139527378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56d00cfd10 0x7f56d01531c0 unknown :-1 s=CLOSED pgs=2589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:09.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.125+0000 7f56d8000640 1 -- 172.21.15.39:0/139527378 >> 172.21.15.39:0/139527378 conn(0x7f56d006cfc0 msgr2=0x7f56d006f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:09.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.126+0000 7f56d8000640 1 -- 172.21.15.39:0/139527378 shutdown_connections 2024-09-21T07:53:09.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.126+0000 7f56d8000640 1 -- 172.21.15.39:0/139527378 wait complete. 2024-09-21T07:53:09.153 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T07:53:09.153 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T07:53:09.153 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T07:53:09.153 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:53:09.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:53:09.153 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/daemon/state.py", line 146, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi148 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d' 2024-09-21T07:53:09.154 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T07:53:09.154 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T07:53:09.158 INFO:tasks.ceph.mds.d:Started 2024-09-21T07:53:09.158 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:53:09.346 INFO:tasks.ceph.mds.d.smithi148.stdout:starting mds.d at 2024-09-21T07:53:09.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.382+0000 7f2dcda00640 1 -- 172.21.15.39:0/2230275020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2dc815fba0 msgr2=0x7f2dc80d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:09.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.382+0000 7f2dcda00640 1 --2- 172.21.15.39:0/2230275020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2dc815fba0 0x7f2dc80d0be0 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0005fd0 tx=0x7f2dc002aa90 comp rx=0 tx=0).stop 2024-09-21T07:53:09.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.382+0000 7f2dcda00640 1 -- 172.21.15.39:0/2230275020 shutdown_connections 2024-09-21T07:53:09.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.382+0000 7f2dcda00640 1 --2- 172.21.15.39:0/2230275020 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2dc815f790 0x7f2dc80d06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:09.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.382+0000 7f2dcda00640 1 --2- 172.21.15.39:0/2230275020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2dc815fba0 0x7f2dc80d0be0 unknown :-1 s=CLOSED pgs=2590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:09.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.382+0000 7f2dcda00640 1 -- 172.21.15.39:0/2230275020 >> 172.21.15.39:0/2230275020 conn(0x7f2dc806cfc0 msgr2=0x7f2dc806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:09.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.382+0000 7f2dcda00640 1 -- 172.21.15.39:0/2230275020 shutdown_connections 2024-09-21T07:53:09.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.383+0000 7f2dcda00640 1 -- 172.21.15.39:0/2230275020 wait complete. 2024-09-21T07:53:09.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.385+0000 7f2dcda00640 1 Processor -- start 2024-09-21T07:53:09.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.385+0000 7f2dcda00640 1 -- start start 2024-09-21T07:53:09.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.386+0000 7f2dcda00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2dc815f790 0x7f2dc8153800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:09.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.386+0000 7f2dcda00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2dc815fba0 0x7f2dc81f6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:09.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.386+0000 7f2dcda00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2dc8153d40 0x7f2dc814f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:09.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.386+0000 7f2dcda00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc80ae550 con 0x7f2dc815f790 2024-09-21T07:53:09.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.386+0000 7f2dcda00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc80ae3d0 con 0x7f2dc815fba0 2024-09-21T07:53:09.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.386+0000 7f2dcda00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f2dc80ae6d0 con 0x7f2dc8153d40 2024-09-21T07:53:09.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.387+0000 7f2dc6a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2dc815f790 0x7f2dc8153800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:09.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.387+0000 7f2dc6a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2dc815f790 0x7f2dc8153800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57418/0 (socket says 172.21.15.39:57418) 2024-09-21T07:53:09.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.387+0000 7f2dc6a00640 1 -- 172.21.15.39:0/1105054187 learned_addr learned my addr 172.21.15.39:0/1105054187 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:09.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.387+0000 7f2dbe000640 1 --2- 172.21.15.39:0/1105054187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2dc815fba0 0x7f2dc81f6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:09.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.387+0000 7f2dc7400640 1 --2- 172.21.15.39:0/1105054187 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2dc8153d40 0x7f2dc814f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:09.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.387+0000 7f2dc6a00640 1 -- 172.21.15.39:0/1105054187 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2dc8153d40 msgr2=0x7f2dc814f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:09.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.387+0000 7f2dc6a00640 1 --2- 172.21.15.39:0/1105054187 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2dc8153d40 0x7f2dc814f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:09.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.387+0000 7f2dc6a00640 1 -- 172.21.15.39:0/1105054187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2dc815fba0 msgr2=0x7f2dc81f6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:09.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.387+0000 7f2dc6a00640 1 --2- 172.21.15.39:0/1105054187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2dc815fba0 0x7f2dc81f6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:09.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.387+0000 7f2dc6a00640 1 -- 172.21.15.39:0/1105054187 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dc8154450 con 0x7f2dc815f790 2024-09-21T07:53:09.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.387+0000 7f2dc6a00640 1 --2- 172.21.15.39:0/1105054187 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2dc815f790 0x7f2dc8153800 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7f2db800c3a0 tx=0x7f2db800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:09.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.388+0000 7f2dbf400640 1 -- 172.21.15.39:0/1105054187 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db8017070 con 0x7f2dc815f790 2024-09-21T07:53:09.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.388+0000 7f2dbf400640 1 -- 172.21.15.39:0/1105054187 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2db80053c0 con 0x7f2dc815f790 2024-09-21T07:53:09.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.388+0000 7f2dbf400640 1 -- 172.21.15.39:0/1105054187 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db8003850 con 0x7f2dc815f790 2024-09-21T07:53:09.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.388+0000 7f2dcda00640 1 -- 172.21.15.39:0/1105054187 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dc814dc60 con 0x7f2dc815f790 2024-09-21T07:53:09.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.388+0000 7f2dcda00640 1 -- 172.21.15.39:0/1105054187 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dc814e220 con 0x7f2dc815f790 2024-09-21T07:53:09.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.389+0000 7f2dcda00640 1 -- 172.21.15.39:0/1105054187 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dc80b47c0 con 0x7f2dc815f790 2024-09-21T07:53:09.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.390+0000 7f2dbf400640 1 -- 172.21.15.39:0/1105054187 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2db801a070 con 0x7f2dc815f790 2024-09-21T07:53:09.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.391+0000 7f2dbf400640 1 --2- 172.21.15.39:0/1105054187 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2da4078bc0 0x7f2da407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:09.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.391+0000 7f2dbf400640 1 -- 172.21.15.39:0/1105054187 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(249..249 src has 1..249) ==== 7777+0+0 (secure 0 0 0) 0x7f2db809a840 con 0x7f2dc815f790 2024-09-21T07:53:09.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.392+0000 7f2dbe000640 1 --2- 172.21.15.39:0/1105054187 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2da4078bc0 0x7f2da407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:09.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.392+0000 7f2dbe000640 1 --2- 172.21.15.39:0/1105054187 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2da4078bc0 0x7f2da407b0d0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8155440 tx=0x7f2db000b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:09.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.396+0000 7f2dbf400640 1 -- 172.21.15.39:0/1105054187 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2db8061d70 con 0x7f2dc815f790 2024-09-21T07:53:09.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.637+0000 7f2dcda00640 1 -- 172.21.15.39:0/1105054187 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2dc8161760 con 0x7f2dc815f790 2024-09-21T07:53:09.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.638+0000 7f2dbf400640 1 -- 172.21.15.39:0/1105054187 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 287 v287) ==== 95+0+4451 (secure 0 0 0) 0x7f2db80667a0 con 0x7f2dc815f790 2024-09-21T07:53:09.640 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:53:09.640 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":287,"btime":"2024-09-21T07:53:09:127140+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":287,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:53:09.127123+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":249,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":11364,"mds_1":11052,"mds_2":10621},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10621":{"gid":10621,"name":"b","rank":2,"incarnation":287,"state":"up:replay","state_seq":1,"addr":"172.21.15.148:6835/2191156088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2191156088},{"type":"v1","addr":"172.21.15.148:6835","nonce":2191156088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11364":{"gid":11364,"name":"a","rank":0,"incarnation":285,"state":"up:resolve","state_seq":24,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11052,"qdb_cluster":[11052]},"id":13}]} 2024-09-21T07:53:09.640 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 287 2024-09-21T07:53:09.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.640+0000 7f2dcda00640 1 -- 172.21.15.39:0/1105054187 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2da4078bc0 msgr2=0x7f2da407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:09.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.640+0000 7f2dcda00640 1 --2- 172.21.15.39:0/1105054187 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2da4078bc0 0x7f2da407b0d0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8155440 tx=0x7f2db000b2e0 comp rx=0 tx=0).stop 2024-09-21T07:53:09.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.640+0000 7f2dcda00640 1 -- 172.21.15.39:0/1105054187 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2dc815f790 msgr2=0x7f2dc8153800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:09.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.640+0000 7f2dcda00640 1 --2- 172.21.15.39:0/1105054187 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2dc815f790 0x7f2dc8153800 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7f2db800c3a0 tx=0x7f2db800c860 comp rx=0 tx=0).stop 2024-09-21T07:53:09.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.641+0000 7f2dcda00640 1 -- 172.21.15.39:0/1105054187 shutdown_connections 2024-09-21T07:53:09.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.641+0000 7f2dcda00640 1 --2- 172.21.15.39:0/1105054187 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2da4078bc0 0x7f2da407b0d0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:09.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.641+0000 7f2dcda00640 1 --2- 172.21.15.39:0/1105054187 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2dc8153d40 0x7f2dc814f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:09.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.641+0000 7f2dcda00640 1 --2- 172.21.15.39:0/1105054187 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2dc815fba0 0x7f2dc81f6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:09.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.641+0000 7f2dcda00640 1 --2- 172.21.15.39:0/1105054187 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2dc815f790 0x7f2dc8153800 unknown :-1 s=CLOSED pgs=2591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:09.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.641+0000 7f2dcda00640 1 -- 172.21.15.39:0/1105054187 >> 172.21.15.39:0/1105054187 conn(0x7f2dc806cfc0 msgr2=0x7f2dc80d0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:09.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.641+0000 7f2dcda00640 1 -- 172.21.15.39:0/1105054187 shutdown_connections 2024-09-21T07:53:09.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:09.641+0000 7f2dcda00640 1 -- 172.21.15.39:0/1105054187 wait complete. 2024-09-21T07:53:09.664 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-09-21T07:53:11.915 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-09-21T07:53:14.665 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:53:14.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:14.903+0000 7f1d87200640 1 -- 172.21.15.39:0/1670374237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d6c004800 msgr2=0x7f1d6c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:14.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:14.903+0000 7f1d87200640 1 --2- 172.21.15.39:0/1670374237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d6c004800 0x7f1d6c004c10 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7f1d7000c310 tx=0x7f1d6c005830 comp rx=0 tx=0).stop 2024-09-21T07:53:14.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:14.903+0000 7f1d87200640 1 -- 172.21.15.39:0/1670374237 shutdown_connections 2024-09-21T07:53:14.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:14.903+0000 7f1d87200640 1 --2- 172.21.15.39:0/1670374237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d6c004800 0x7f1d6c004c10 unknown :-1 s=CLOSED pgs=2593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:14.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:14.903+0000 7f1d87200640 1 -- 172.21.15.39:0/1670374237 >> 172.21.15.39:0/1670374237 conn(0x7f1d880b2510 msgr2=0x7f1d880d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:14.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:14.903+0000 7f1d87200640 1 -- 172.21.15.39:0/1670374237 shutdown_connections 2024-09-21T07:53:14.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:14.903+0000 7f1d87200640 1 -- 172.21.15.39:0/1670374237 wait complete. 2024-09-21T07:53:14.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:14.905+0000 7f1d87200640 1 Processor -- start 2024-09-21T07:53:14.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:14.905+0000 7f1d87200640 1 -- start start 2024-09-21T07:53:14.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:14.906+0000 7f1d87200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d6c004800 0x7f1d881e6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:14.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:14.907+0000 7f1d87200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d88154330 0x7f1d881e65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:14.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:14.907+0000 7f1d87200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d880ad0b0 0x7f1d880b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:14.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:14.907+0000 7f1d87200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8806fc30 con 0x7f1d88154330 2024-09-21T07:53:14.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:14.907+0000 7f1d87200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8806fab0 con 0x7f1d880ad0b0 2024-09-21T07:53:14.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:14.907+0000 7f1d87200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f1d8806fdb0 con 0x7f1d6c004800 2024-09-21T07:53:14.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:14.907+0000 7f1d7fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d88154330 0x7f1d881e65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:14.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:14.907+0000 7f1d7fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d88154330 0x7f1d881e65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45080/0 (socket says 172.21.15.39:45080) 2024-09-21T07:53:14.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:14.907+0000 7f1d7fe00640 1 -- 172.21.15.39:0/2224492 learned_addr learned my addr 172.21.15.39:0/2224492 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:14.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:14.907+0000 7f1d85e00640 1 --2- 172.21.15.39:0/2224492 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d6c004800 0x7f1d881e6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:14.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:14.907+0000 7f1d86800640 1 --2- 172.21.15.39:0/2224492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d880ad0b0 0x7f1d880b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:14.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:14.907+0000 7f1d7fe00640 1 -- 172.21.15.39:0/2224492 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d6c004800 msgr2=0x7f1d881e6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:14.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:14.907+0000 7f1d7fe00640 1 --2- 172.21.15.39:0/2224492 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d6c004800 0x7f1d881e6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:14.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:14.907+0000 7f1d7fe00640 1 -- 172.21.15.39:0/2224492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d880ad0b0 msgr2=0x7f1d880b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:14.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:14.907+0000 7f1d7fe00640 1 --2- 172.21.15.39:0/2224492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d880ad0b0 0x7f1d880b05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:14.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:14.907+0000 7f1d7fe00640 1 -- 172.21.15.39:0/2224492 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d881eb150 con 0x7f1d88154330 2024-09-21T07:53:14.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:14.908+0000 7f1d7fe00640 1 --2- 172.21.15.39:0/2224492 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d88154330 0x7f1d881e65c0 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7f1d7000c430 tx=0x7f1d70001da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:14.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:14.908+0000 7f1d7e000640 1 -- 172.21.15.39:0/2224492 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d70031040 con 0x7f1d88154330 2024-09-21T07:53:14.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:14.908+0000 7f1d7e000640 1 -- 172.21.15.39:0/2224492 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d700065e0 con 0x7f1d88154330 2024-09-21T07:53:14.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:14.908+0000 7f1d87200640 1 -- 172.21.15.39:0/2224492 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d88074a60 con 0x7f1d88154330 2024-09-21T07:53:14.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:14.908+0000 7f1d7e000640 1 -- 172.21.15.39:0/2224492 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d70003a40 con 0x7f1d88154330 2024-09-21T07:53:14.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:14.908+0000 7f1d87200640 1 -- 172.21.15.39:0/2224492 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d880afdb0 con 0x7f1d88154330 2024-09-21T07:53:14.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:14.909+0000 7f1d87200640 1 -- 172.21.15.39:0/2224492 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d88072fa0 con 0x7f1d88154330 2024-09-21T07:53:14.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:14.910+0000 7f1d7e000640 1 -- 172.21.15.39:0/2224492 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1d7003e030 con 0x7f1d88154330 2024-09-21T07:53:14.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:14.911+0000 7f1d7e000640 1 --2- 172.21.15.39:0/2224492 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1d64078b90 0x7f1d6407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:14.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:14.911+0000 7f1d7e000640 1 -- 172.21.15.39:0/2224492 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(249..249 src has 1..249) ==== 7777+0+0 (secure 0 0 0) 0x7f1d700bb450 con 0x7f1d88154330 2024-09-21T07:53:14.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:14.912+0000 7f1d85e00640 1 --2- 172.21.15.39:0/2224492 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1d64078b90 0x7f1d6407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:14.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:14.912+0000 7f1d85e00640 1 --2- 172.21.15.39:0/2224492 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1d64078b90 0x7f1d6407b0a0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f1d78009c10 tx=0x7f1d78005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:14.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:14.916+0000 7f1d7e000640 1 -- 172.21.15.39:0/2224492 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1d70082fe0 con 0x7f1d88154330 2024-09-21T07:53:15.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.150+0000 7f1d87200640 1 -- 172.21.15.39:0/2224492 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d8806df80 con 0x7f1d88154330 2024-09-21T07:53:15.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.150+0000 7f1d7e000640 1 -- 172.21.15.39:0/2224492 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 289 v289) ==== 95+0+5285 (secure 0 0 0) 0x7f1d700828e0 con 0x7f1d88154330 2024-09-21T07:53:15.153 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:53:15.153 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":289,"btime":"2024-09-21T07:53:11:133316+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10633,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3495264983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3495264983},{"type":"v1","addr":"172.21.15.148:6837","nonce":3495264983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":288}],"filesystems":[{"mdsmap":{"epoch":289,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:53:10.137442+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":249,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":11364,"mds_1":11052,"mds_2":10621},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10621":{"gid":10621,"name":"b","rank":2,"incarnation":287,"state":"up:reconnect","state_seq":3,"addr":"172.21.15.148:6835/2191156088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2191156088},{"type":"v1","addr":"172.21.15.148:6835","nonce":2191156088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11364":{"gid":11364,"name":"a","rank":0,"incarnation":285,"state":"up:reconnect","state_seq":25,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11052,"qdb_cluster":[11052]},"id":13}]} 2024-09-21T07:53:15.153 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 289 2024-09-21T07:53:15.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.153+0000 7f1d87200640 1 -- 172.21.15.39:0/2224492 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1d64078b90 msgr2=0x7f1d6407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:15.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.153+0000 7f1d87200640 1 --2- 172.21.15.39:0/2224492 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1d64078b90 0x7f1d6407b0a0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f1d78009c10 tx=0x7f1d78005f90 comp rx=0 tx=0).stop 2024-09-21T07:53:15.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.153+0000 7f1d87200640 1 -- 172.21.15.39:0/2224492 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d88154330 msgr2=0x7f1d881e65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:15.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.153+0000 7f1d87200640 1 --2- 172.21.15.39:0/2224492 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d88154330 0x7f1d881e65c0 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7f1d7000c430 tx=0x7f1d70001da0 comp rx=0 tx=0).stop 2024-09-21T07:53:15.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.153+0000 7f1d87200640 1 -- 172.21.15.39:0/2224492 shutdown_connections 2024-09-21T07:53:15.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.153+0000 7f1d87200640 1 --2- 172.21.15.39:0/2224492 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1d64078b90 0x7f1d6407b0a0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:15.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.153+0000 7f1d87200640 1 --2- 172.21.15.39:0/2224492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d880ad0b0 0x7f1d880b05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:15.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.153+0000 7f1d87200640 1 --2- 172.21.15.39:0/2224492 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d88154330 0x7f1d881e65c0 unknown :-1 s=CLOSED pgs=2594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:15.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.153+0000 7f1d87200640 1 --2- 172.21.15.39:0/2224492 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f1d6c004800 0x7f1d881e6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:15.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.153+0000 7f1d87200640 1 -- 172.21.15.39:0/2224492 >> 172.21.15.39:0/2224492 conn(0x7f1d880b2510 msgr2=0x7f1d880b43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:15.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.153+0000 7f1d87200640 1 -- 172.21.15.39:0/2224492 shutdown_connections 2024-09-21T07:53:15.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.153+0000 7f1d87200640 1 -- 172.21.15.39:0/2224492 wait complete. 2024-09-21T07:53:15.181 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-21T07:53:15.181 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:53:15.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.407+0000 7ff9c3e00640 1 -- 172.21.15.39:0/4258213647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9a8003a50 msgr2=0x7ff9a8003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:15.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.407+0000 7ff9c3e00640 1 --2- 172.21.15.39:0/4258213647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9a8003a50 0x7ff9a8003ec0 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7ff9b40016c0 tx=0x7ff9a8005830 comp rx=0 tx=0).stop 2024-09-21T07:53:15.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.407+0000 7ff9c3e00640 1 -- 172.21.15.39:0/4258213647 shutdown_connections 2024-09-21T07:53:15.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.407+0000 7ff9c3e00640 1 --2- 172.21.15.39:0/4258213647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9a8003a50 0x7ff9a8003ec0 unknown :-1 s=CLOSED pgs=2596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:15.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.408+0000 7ff9c3e00640 1 -- 172.21.15.39:0/4258213647 >> 172.21.15.39:0/4258213647 conn(0x7ff9c40b2510 msgr2=0x7ff9c40d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:15.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.408+0000 7ff9c3e00640 1 -- 172.21.15.39:0/4258213647 shutdown_connections 2024-09-21T07:53:15.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.408+0000 7ff9c3e00640 1 -- 172.21.15.39:0/4258213647 wait complete. 2024-09-21T07:53:15.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.410+0000 7ff9c3e00640 1 Processor -- start 2024-09-21T07:53:15.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.410+0000 7ff9c3e00640 1 -- start start 2024-09-21T07:53:15.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.411+0000 7ff9c3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9a8003a50 0x7ff9c4092860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:15.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.412+0000 7ff9c3e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9c4099750 0x7ff9c4158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:15.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.412+0000 7ff9c3e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff9c4159250 0x7ff9c4159660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:15.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.412+0000 7ff9c3e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c40af990 con 0x7ff9a8003a50 2024-09-21T07:53:15.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.412+0000 7ff9c3e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c40af810 con 0x7ff9c4099750 2024-09-21T07:53:15.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.412+0000 7ff9c3e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7ff9c40afb10 con 0x7ff9c4159250 2024-09-21T07:53:15.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.412+0000 7ff9c2a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9a8003a50 0x7ff9c4092860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:15.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.412+0000 7ff9c2a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9a8003a50 0x7ff9c4092860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45096/0 (socket says 172.21.15.39:45096) 2024-09-21T07:53:15.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.412+0000 7ff9c2a00640 1 -- 172.21.15.39:0/3017351843 learned_addr learned my addr 172.21.15.39:0/3017351843 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:15.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.412+0000 7ff9c2000640 1 --2- 172.21.15.39:0/3017351843 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9c4099750 0x7ff9c4158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:15.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.412+0000 7ff9c3400640 1 --2- 172.21.15.39:0/3017351843 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff9c4159250 0x7ff9c4159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:15.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.412+0000 7ff9c2a00640 1 -- 172.21.15.39:0/3017351843 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff9c4159250 msgr2=0x7ff9c4159660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:15.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.412+0000 7ff9c2a00640 1 --2- 172.21.15.39:0/3017351843 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff9c4159250 0x7ff9c4159660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:15.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.412+0000 7ff9c2a00640 1 -- 172.21.15.39:0/3017351843 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9c4099750 msgr2=0x7ff9c4158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:15.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.412+0000 7ff9c2a00640 1 --2- 172.21.15.39:0/3017351843 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9c4099750 0x7ff9c4158d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:15.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.412+0000 7ff9c2a00640 1 -- 172.21.15.39:0/3017351843 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c4159ba0 con 0x7ff9a8003a50 2024-09-21T07:53:15.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.413+0000 7ff9c2a00640 1 --2- 172.21.15.39:0/3017351843 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9a8003a50 0x7ff9c4092860 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 crypto rx=0x7ff9b40017e0 tx=0x7ff9b40063a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:15.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.413+0000 7ff9baa00640 1 -- 172.21.15.39:0/3017351843 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4042070 con 0x7ff9a8003a50 2024-09-21T07:53:15.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.413+0000 7ff9baa00640 1 -- 172.21.15.39:0/3017351843 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9b4039070 con 0x7ff9a8003a50 2024-09-21T07:53:15.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.413+0000 7ff9c3e00640 1 -- 172.21.15.39:0/3017351843 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9c4156070 con 0x7ff9a8003a50 2024-09-21T07:53:15.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.413+0000 7ff9baa00640 1 -- 172.21.15.39:0/3017351843 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4003d70 con 0x7ff9a8003a50 2024-09-21T07:53:15.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.413+0000 7ff9c3e00640 1 -- 172.21.15.39:0/3017351843 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9c4156630 con 0x7ff9a8003a50 2024-09-21T07:53:15.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.414+0000 7ff9c3e00640 1 -- 172.21.15.39:0/3017351843 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9c40cea10 con 0x7ff9a8003a50 2024-09-21T07:53:15.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.415+0000 7ff9baa00640 1 -- 172.21.15.39:0/3017351843 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff9b403b070 con 0x7ff9a8003a50 2024-09-21T07:53:15.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.416+0000 7ff9baa00640 1 --2- 172.21.15.39:0/3017351843 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff9a4078b90 0x7ff9a407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:15.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.416+0000 7ff9baa00640 1 -- 172.21.15.39:0/3017351843 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(249..249 src has 1..249) ==== 7777+0+0 (secure 0 0 0) 0x7ff9b40bad10 con 0x7ff9a8003a50 2024-09-21T07:53:15.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.417+0000 7ff9c2000640 1 --2- 172.21.15.39:0/3017351843 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff9a4078b90 0x7ff9a407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:15.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.417+0000 7ff9c2000640 1 --2- 172.21.15.39:0/3017351843 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff9a4078b90 0x7ff9a407b0a0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0001d60 tx=0x7ff9b0001cf0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:15.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.421+0000 7ff9baa00640 1 -- 172.21.15.39:0/3017351843 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff9b40c0020 con 0x7ff9a8003a50 2024-09-21T07:53:15.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.655+0000 7ff9c3e00640 1 -- 172.21.15.39:0/3017351843 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9c4070c30 con 0x7ff9a8003a50 2024-09-21T07:53:15.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.656+0000 7ff9baa00640 1 -- 172.21.15.39:0/3017351843 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 289 v289) ==== 95+0+5285 (secure 0 0 0) 0x7ff9b4082890 con 0x7ff9a8003a50 2024-09-21T07:53:15.658 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:53:15.658 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":289,"btime":"2024-09-21T07:53:11:133316+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10633,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3495264983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3495264983},{"type":"v1","addr":"172.21.15.148:6837","nonce":3495264983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":288}],"filesystems":[{"mdsmap":{"epoch":289,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:53:10.137442+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":249,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":11364,"mds_1":11052,"mds_2":10621},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10621":{"gid":10621,"name":"b","rank":2,"incarnation":287,"state":"up:reconnect","state_seq":3,"addr":"172.21.15.148:6835/2191156088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2191156088},{"type":"v1","addr":"172.21.15.148:6835","nonce":2191156088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11364":{"gid":11364,"name":"a","rank":0,"incarnation":285,"state":"up:reconnect","state_seq":25,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11052,"qdb_cluster":[11052]},"id":13}]} 2024-09-21T07:53:15.658 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 289 2024-09-21T07:53:15.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.658+0000 7ff9c3e00640 1 -- 172.21.15.39:0/3017351843 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff9a4078b90 msgr2=0x7ff9a407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:15.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.658+0000 7ff9c3e00640 1 --2- 172.21.15.39:0/3017351843 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff9a4078b90 0x7ff9a407b0a0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0001d60 tx=0x7ff9b0001cf0 comp rx=0 tx=0).stop 2024-09-21T07:53:15.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.658+0000 7ff9c3e00640 1 -- 172.21.15.39:0/3017351843 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9a8003a50 msgr2=0x7ff9c4092860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:15.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.658+0000 7ff9c3e00640 1 --2- 172.21.15.39:0/3017351843 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9a8003a50 0x7ff9c4092860 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 crypto rx=0x7ff9b40017e0 tx=0x7ff9b40063a0 comp rx=0 tx=0).stop 2024-09-21T07:53:15.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.658+0000 7ff9c3e00640 1 -- 172.21.15.39:0/3017351843 shutdown_connections 2024-09-21T07:53:15.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.658+0000 7ff9c3e00640 1 --2- 172.21.15.39:0/3017351843 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff9a4078b90 0x7ff9a407b0a0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:15.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.658+0000 7ff9c3e00640 1 --2- 172.21.15.39:0/3017351843 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff9c4159250 0x7ff9c4159660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:15.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.658+0000 7ff9c3e00640 1 --2- 172.21.15.39:0/3017351843 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff9c4099750 0x7ff9c4158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:15.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.658+0000 7ff9c3e00640 1 --2- 172.21.15.39:0/3017351843 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9a8003a50 0x7ff9c4092860 unknown :-1 s=CLOSED pgs=2597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:15.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.658+0000 7ff9c3e00640 1 -- 172.21.15.39:0/3017351843 >> 172.21.15.39:0/3017351843 conn(0x7ff9c40b2510 msgr2=0x7ff9c40cd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:15.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.658+0000 7ff9c3e00640 1 -- 172.21.15.39:0/3017351843 shutdown_connections 2024-09-21T07:53:15.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:15.659+0000 7ff9c3e00640 1 -- 172.21.15.39:0/3017351843 wait complete. 2024-09-21T07:53:16.088 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 289, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:53:10.137442+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 249, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 11364, 'mds_1': 11052, 'mds_2': 10621}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10621': {'gid': 10621, 'name': 'b', 'rank': 2, 'incarnation': 287, 'state': 'up:reconnect', 'state_seq': 3, 'addr': '172.21.15.148:6835/2191156088', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 2191156088}, {'type': 'v1', 'addr': '172.21.15.148:6835', 'nonce': 2191156088}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11052': {'gid': 11052, 'name': 'c', 'rank': 1, 'incarnation': 261, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6835/485194245', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 485194245}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 485194245}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11364': {'gid': 11364, 'name': 'a', 'rank': 0, 'incarnation': 285, 'state': 'up:reconnect', 'state_seq': 25, 'addr': '172.21.15.39:6834/2161818466', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 2161818466}, {'type': 'v1', 'addr': '172.21.15.39:6834', 'nonce': 2161818466}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 11052, 'qdb_cluster': [11052]} 2024-09-21T07:53:16.088 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10621:up:reconnect 2024-09-21T07:53:17.089 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:53:17.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.330+0000 7f00f8e00640 1 -- 172.21.15.39:0/3631980480 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00d8004d90 msgr2=0x7f00d8005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:17.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.330+0000 7f00f8e00640 1 --2- 172.21.15.39:0/3631980480 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00d8004d90 0x7f00d8005220 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7f00ec001ea0 tx=0x7f00d8005830 comp rx=0 tx=0).stop 2024-09-21T07:53:17.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.330+0000 7f00f2800640 1 -- 172.21.15.39:0/3631980480 reap_dead start 2024-09-21T07:53:17.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.331+0000 7f00f8e00640 1 -- 172.21.15.39:0/3631980480 shutdown_connections 2024-09-21T07:53:17.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.331+0000 7f00f8e00640 1 -- 172.21.15.39:0/3631980480 >> 172.21.15.39:0/3631980480 conn(0x7f00f406cfc0 msgr2=0x7f00f406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:17.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.331+0000 7f00f8e00640 1 -- 172.21.15.39:0/3631980480 shutdown_connections 2024-09-21T07:53:17.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.331+0000 7f00f8e00640 1 -- 172.21.15.39:0/3631980480 wait complete. 2024-09-21T07:53:17.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.333+0000 7f00f8e00640 1 Processor -- start 2024-09-21T07:53:17.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.333+0000 7f00f8e00640 1 -- start start 2024-09-21T07:53:17.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.334+0000 7f00f8e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f00d8004d90 0x7f00f41f92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:17.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.334+0000 7f00f8e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00f40b1ed0 0x7f00f40b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:17.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.334+0000 7f00f8e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00f4145a20 0x7f00f4145e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:17.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.335+0000 7f00f8e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f00f40aebb0 con 0x7f00f40b1ed0 2024-09-21T07:53:17.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.335+0000 7f00f8e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f00f40aea30 con 0x7f00f4145a20 2024-09-21T07:53:17.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.335+0000 7f00f8e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f00f40d06e0 con 0x7f00d8004d90 2024-09-21T07:53:17.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.335+0000 7f00ebe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00f40b1ed0 0x7f00f40b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:17.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.335+0000 7f00ebe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00f40b1ed0 0x7f00f40b22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45112/0 (socket says 172.21.15.39:45112) 2024-09-21T07:53:17.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.335+0000 7f00ebe00640 1 -- 172.21.15.39:0/992978411 learned_addr learned my addr 172.21.15.39:0/992978411 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:17.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.335+0000 7f00f1e00640 1 --2- 172.21.15.39:0/992978411 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f00d8004d90 0x7f00f41f92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:17.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.335+0000 7f00f2800640 1 --2- 172.21.15.39:0/992978411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00f4145a20 0x7f00f4145e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:17.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.335+0000 7f00ebe00640 1 -- 172.21.15.39:0/992978411 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f00d8004d90 msgr2=0x7f00f41f92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:17.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.335+0000 7f00ebe00640 1 --2- 172.21.15.39:0/992978411 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f00d8004d90 0x7f00f41f92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:17.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.335+0000 7f00ebe00640 1 -- 172.21.15.39:0/992978411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00f4145a20 msgr2=0x7f00f4145e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:17.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.335+0000 7f00ebe00640 1 --2- 172.21.15.39:0/992978411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00f4145a20 0x7f00f4145e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:17.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.335+0000 7f00ebe00640 1 -- 172.21.15.39:0/992978411 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f4146370 con 0x7f00f40b1ed0 2024-09-21T07:53:17.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.335+0000 7f00f1e00640 1 --2- 172.21.15.39:0/992978411 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f00d8004d90 0x7f00f41f92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:53:17.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.335+0000 7f00f2800640 1 --2- 172.21.15.39:0/992978411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00f4145a20 0x7f00f4145e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:53:17.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.336+0000 7f00ebe00640 1 --2- 172.21.15.39:0/992978411 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00f40b1ed0 0x7f00f40b22e0 secure :-1 s=READY pgs=2600 cs=0 l=1 rev1=1 crypto rx=0x7f00dc011a80 tx=0x7f00dc011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:17.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.336+0000 7f00ea000640 1 -- 172.21.15.39:0/992978411 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc00fe10 con 0x7f00f40b1ed0 2024-09-21T07:53:17.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.336+0000 7f00ea000640 1 -- 172.21.15.39:0/992978411 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00dc016070 con 0x7f00f40b1ed0 2024-09-21T07:53:17.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.336+0000 7f00ea000640 1 -- 172.21.15.39:0/992978411 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc012690 con 0x7f00f40b1ed0 2024-09-21T07:53:17.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.336+0000 7f00f8e00640 1 -- 172.21.15.39:0/992978411 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00f41f5300 con 0x7f00f40b1ed0 2024-09-21T07:53:17.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.336+0000 7f00f8e00640 1 -- 172.21.15.39:0/992978411 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00f4159a40 con 0x7f00f40b1ed0 2024-09-21T07:53:17.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.337+0000 7f00f8e00640 1 -- 172.21.15.39:0/992978411 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00f41597b0 con 0x7f00f40b1ed0 2024-09-21T07:53:17.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.338+0000 7f00ea000640 1 -- 172.21.15.39:0/992978411 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f00dc018030 con 0x7f00f40b1ed0 2024-09-21T07:53:17.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.339+0000 7f00ea000640 1 --2- 172.21.15.39:0/992978411 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f00d0078b90 0x7f00d007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:17.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.339+0000 7f00ea000640 1 -- 172.21.15.39:0/992978411 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(250..250 src has 1..250) ==== 7820+0+0 (secure 0 0 0) 0x7f00dc09bf20 con 0x7f00f40b1ed0 2024-09-21T07:53:17.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.340+0000 7f00f1e00640 1 --2- 172.21.15.39:0/992978411 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f00d0078b90 0x7f00d007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:17.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.341+0000 7f00f1e00640 1 --2- 172.21.15.39:0/992978411 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f00d0078b90 0x7f00d007b0a0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f00e4006aa0 tx=0x7f00e4001df0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:17.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.344+0000 7f00ea000640 1 -- 172.21.15.39:0/992978411 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f00f4146370 con 0x7f00f40b1ed0 2024-09-21T07:53:17.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.580+0000 7f00f8e00640 1 -- 172.21.15.39:0/992978411 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00f41594e0 con 0x7f00f40b1ed0 2024-09-21T07:53:17.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.581+0000 7f00ea000640 1 -- 172.21.15.39:0/992978411 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 289 v289) ==== 95+0+5285 (secure 0 0 0) 0x7f00dc063b10 con 0x7f00f40b1ed0 2024-09-21T07:53:17.583 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:53:17.583 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":289,"btime":"2024-09-21T07:53:11:133316+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10633,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3495264983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3495264983},{"type":"v1","addr":"172.21.15.148:6837","nonce":3495264983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":288}],"filesystems":[{"mdsmap":{"epoch":289,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:53:10.137442+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":249,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":11364,"mds_1":11052,"mds_2":10621},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10621":{"gid":10621,"name":"b","rank":2,"incarnation":287,"state":"up:reconnect","state_seq":3,"addr":"172.21.15.148:6835/2191156088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2191156088},{"type":"v1","addr":"172.21.15.148:6835","nonce":2191156088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11364":{"gid":11364,"name":"a","rank":0,"incarnation":285,"state":"up:reconnect","state_seq":25,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11052,"qdb_cluster":[11052]},"id":13}]} 2024-09-21T07:53:17.584 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 289 2024-09-21T07:53:17.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.583+0000 7f00f8e00640 1 -- 172.21.15.39:0/992978411 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f00d0078b90 msgr2=0x7f00d007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:17.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.583+0000 7f00f8e00640 1 --2- 172.21.15.39:0/992978411 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f00d0078b90 0x7f00d007b0a0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f00e4006aa0 tx=0x7f00e4001df0 comp rx=0 tx=0).stop 2024-09-21T07:53:17.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.583+0000 7f00f8e00640 1 -- 172.21.15.39:0/992978411 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00f40b1ed0 msgr2=0x7f00f40b22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:17.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.583+0000 7f00f8e00640 1 --2- 172.21.15.39:0/992978411 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00f40b1ed0 0x7f00f40b22e0 secure :-1 s=READY pgs=2600 cs=0 l=1 rev1=1 crypto rx=0x7f00dc011a80 tx=0x7f00dc011f40 comp rx=0 tx=0).stop 2024-09-21T07:53:17.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.583+0000 7f00f8e00640 1 -- 172.21.15.39:0/992978411 shutdown_connections 2024-09-21T07:53:17.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.583+0000 7f00f8e00640 1 --2- 172.21.15.39:0/992978411 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f00d0078b90 0x7f00d007b0a0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:17.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.583+0000 7f00f8e00640 1 --2- 172.21.15.39:0/992978411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f00f4145a20 0x7f00f4145e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:17.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.583+0000 7f00f8e00640 1 --2- 172.21.15.39:0/992978411 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00f40b1ed0 0x7f00f40b22e0 unknown :-1 s=CLOSED pgs=2600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:17.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.583+0000 7f00f8e00640 1 --2- 172.21.15.39:0/992978411 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f00d8004d90 0x7f00f41f92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:17.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.584+0000 7f00f8e00640 1 -- 172.21.15.39:0/992978411 >> 172.21.15.39:0/992978411 conn(0x7f00f406cfc0 msgr2=0x7f00f406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:17.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.584+0000 7f00f8e00640 1 -- 172.21.15.39:0/992978411 shutdown_connections 2024-09-21T07:53:17.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:17.584+0000 7f00f8e00640 1 -- 172.21.15.39:0/992978411 wait complete. 2024-09-21T07:53:18.012 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 289, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:53:10.137442+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 249, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 11364, 'mds_1': 11052, 'mds_2': 10621}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10621': {'gid': 10621, 'name': 'b', 'rank': 2, 'incarnation': 287, 'state': 'up:reconnect', 'state_seq': 3, 'addr': '172.21.15.148:6835/2191156088', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 2191156088}, {'type': 'v1', 'addr': '172.21.15.148:6835', 'nonce': 2191156088}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11052': {'gid': 11052, 'name': 'c', 'rank': 1, 'incarnation': 261, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6835/485194245', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 485194245}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 485194245}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11364': {'gid': 11364, 'name': 'a', 'rank': 0, 'incarnation': 285, 'state': 'up:reconnect', 'state_seq': 25, 'addr': '172.21.15.39:6834/2161818466', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 2161818466}, {'type': 'v1', 'addr': '172.21.15.39:6834', 'nonce': 2161818466}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 11052, 'qdb_cluster': [11052]} 2024-09-21T07:53:18.012 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10621:up:reconnect 2024-09-21T07:53:19.013 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:53:19.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.253+0000 7feb13e00640 1 -- 172.21.15.39:0/4130030498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb140b4db0 msgr2=0x7feb140d4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:19.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.253+0000 7feb13e00640 1 --2- 172.21.15.39:0/4130030498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb140b4db0 0x7feb140d4c90 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 crypto rx=0x7feb04009820 tx=0x7feb0402a510 comp rx=0 tx=0).stop 2024-09-21T07:53:19.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.253+0000 7feb13e00640 1 -- 172.21.15.39:0/4130030498 shutdown_connections 2024-09-21T07:53:19.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.253+0000 7feb13e00640 1 --2- 172.21.15.39:0/4130030498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb140b4db0 0x7feb140d4c90 unknown :-1 s=CLOSED pgs=2602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:19.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.253+0000 7feb13e00640 1 -- 172.21.15.39:0/4130030498 >> 172.21.15.39:0/4130030498 conn(0x7feb1408bfb0 msgr2=0x7feb140d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:19.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.253+0000 7feb13e00640 1 -- 172.21.15.39:0/4130030498 shutdown_connections 2024-09-21T07:53:19.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.253+0000 7feb13e00640 1 -- 172.21.15.39:0/4130030498 wait complete. 2024-09-21T07:53:19.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.255+0000 7feb13e00640 1 Processor -- start 2024-09-21T07:53:19.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.255+0000 7feb13e00640 1 -- start start 2024-09-21T07:53:19.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.257+0000 7feb13e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb140b4db0 0x7feb140cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:19.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.257+0000 7feb13e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb140d2a50 0x7feb14152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:19.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.257+0000 7feb13e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb140d2e60 0x7feb14153a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:19.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.257+0000 7feb13e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7feb14169f80 con 0x7feb140d2a50 2024-09-21T07:53:19.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.257+0000 7feb13e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7feb14169e00 con 0x7feb140b4db0 2024-09-21T07:53:19.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.257+0000 7feb13e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7feb1416a100 con 0x7feb140d2e60 2024-09-21T07:53:19.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.257+0000 7feb0be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb140d2a50 0x7feb14152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:19.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.257+0000 7feb0be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb140d2a50 0x7feb14152620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45132/0 (socket says 172.21.15.39:45132) 2024-09-21T07:53:19.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.257+0000 7feb0be00640 1 -- 172.21.15.39:0/1776755564 learned_addr learned my addr 172.21.15.39:0/1776755564 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:19.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.257+0000 7feb12a00640 1 --2- 172.21.15.39:0/1776755564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb140b4db0 0x7feb140cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:19.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.257+0000 7feb0be00640 1 -- 172.21.15.39:0/1776755564 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb140d2e60 msgr2=0x7feb14153a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:19.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.257+0000 7feb0be00640 1 --2- 172.21.15.39:0/1776755564 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb140d2e60 0x7feb14153a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:19.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.257+0000 7feb0be00640 1 -- 172.21.15.39:0/1776755564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb140b4db0 msgr2=0x7feb140cb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:19.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.257+0000 7feb0be00640 1 --2- 172.21.15.39:0/1776755564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb140b4db0 0x7feb140cb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:19.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.257+0000 7feb0be00640 1 -- 172.21.15.39:0/1776755564 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb140cbb00 con 0x7feb140d2a50 2024-09-21T07:53:19.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.258+0000 7feb0be00640 1 --2- 172.21.15.39:0/1776755564 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb140d2a50 0x7feb14152620 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7feafc00b3e0 tx=0x7feafc00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:19.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.258+0000 7feb0b400640 1 -- 172.21.15.39:0/1776755564 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc015020 con 0x7feb140d2a50 2024-09-21T07:53:19.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.258+0000 7feb0b400640 1 -- 172.21.15.39:0/1776755564 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feafc011920 con 0x7feb140d2a50 2024-09-21T07:53:19.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.258+0000 7feb13e00640 1 -- 172.21.15.39:0/1776755564 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb140ca730 con 0x7feb140d2a50 2024-09-21T07:53:19.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.258+0000 7feb0b400640 1 -- 172.21.15.39:0/1776755564 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc011d70 con 0x7feb140d2a50 2024-09-21T07:53:19.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.258+0000 7feb13e00640 1 -- 172.21.15.39:0/1776755564 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb140ca9d0 con 0x7feb140d2a50 2024-09-21T07:53:19.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.259+0000 7feb13e00640 1 -- 172.21.15.39:0/1776755564 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb140b5cc0 con 0x7feb140d2a50 2024-09-21T07:53:19.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.261+0000 7feb0b400640 1 -- 172.21.15.39:0/1776755564 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7feafc018020 con 0x7feb140d2a50 2024-09-21T07:53:19.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.262+0000 7feb0b400640 1 --2- 172.21.15.39:0/1776755564 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feaf0078bc0 0x7feaf007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:19.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.262+0000 7feb0b400640 1 -- 172.21.15.39:0/1776755564 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(250..250 src has 1..250) ==== 7820+0+0 (secure 0 0 0) 0x7feafc09b2c0 con 0x7feb140d2a50 2024-09-21T07:53:19.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.262+0000 7feb12a00640 1 --2- 172.21.15.39:0/1776755564 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feaf0078bc0 0x7feaf007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:19.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.263+0000 7feb12a00640 1 --2- 172.21.15.39:0/1776755564 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feaf0078bc0 0x7feaf007b0d0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7feb04034000 tx=0x7feb04004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:19.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.266+0000 7feb0b400640 1 -- 172.21.15.39:0/1776755564 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7feb140b5cc0 con 0x7feb140d2a50 2024-09-21T07:53:19.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.500+0000 7feb13e00640 1 -- 172.21.15.39:0/1776755564 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb140c9a40 con 0x7feb140d2a50 2024-09-21T07:53:19.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.501+0000 7feb0b400640 1 -- 172.21.15.39:0/1776755564 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 291 v291) ==== 95+0+5291 (secure 0 0 0) 0x7feafc062f50 con 0x7feb140d2a50 2024-09-21T07:53:19.503 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:53:19.503 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":291,"btime":"2024-09-21T07:53:19:187136+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10633,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3495264983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3495264983},{"type":"v1","addr":"172.21.15.148:6837","nonce":3495264983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":288}],"filesystems":[{"mdsmap":{"epoch":291,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:53:19.187128+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":249,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":11364,"mds_1":11052,"mds_2":10621},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10621":{"gid":10621,"name":"b","rank":2,"incarnation":287,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6835/2191156088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2191156088},{"type":"v1","addr":"172.21.15.148:6835","nonce":2191156088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11364":{"gid":11364,"name":"a","rank":0,"incarnation":285,"state":"up:active","state_seq":28,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11364,"qdb_cluster":[10621,11052,11364]},"id":13}]} 2024-09-21T07:53:19.503 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 291 2024-09-21T07:53:19.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.503+0000 7feb13e00640 1 -- 172.21.15.39:0/1776755564 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feaf0078bc0 msgr2=0x7feaf007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:19.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.503+0000 7feb13e00640 1 --2- 172.21.15.39:0/1776755564 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feaf0078bc0 0x7feaf007b0d0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7feb04034000 tx=0x7feb04004730 comp rx=0 tx=0).stop 2024-09-21T07:53:19.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.503+0000 7feb13e00640 1 -- 172.21.15.39:0/1776755564 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb140d2a50 msgr2=0x7feb14152620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:19.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.503+0000 7feb13e00640 1 --2- 172.21.15.39:0/1776755564 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb140d2a50 0x7feb14152620 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7feafc00b3e0 tx=0x7feafc00b8a0 comp rx=0 tx=0).stop 2024-09-21T07:53:19.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.503+0000 7feb13e00640 1 -- 172.21.15.39:0/1776755564 shutdown_connections 2024-09-21T07:53:19.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.503+0000 7feb13e00640 1 --2- 172.21.15.39:0/1776755564 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feaf0078bc0 0x7feaf007b0d0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:19.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.503+0000 7feb13e00640 1 --2- 172.21.15.39:0/1776755564 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb140d2e60 0x7feb14153a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:19.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.503+0000 7feb13e00640 1 --2- 172.21.15.39:0/1776755564 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb140d2a50 0x7feb14152620 unknown :-1 s=CLOSED pgs=2603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:19.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.503+0000 7feb13e00640 1 --2- 172.21.15.39:0/1776755564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb140b4db0 0x7feb140cb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:19.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.503+0000 7feb13e00640 1 -- 172.21.15.39:0/1776755564 >> 172.21.15.39:0/1776755564 conn(0x7feb1408bfb0 msgr2=0x7feb14157640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:19.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.504+0000 7feb13e00640 1 -- 172.21.15.39:0/1776755564 shutdown_connections 2024-09-21T07:53:19.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:19.504+0000 7feb13e00640 1 -- 172.21.15.39:0/1776755564 wait complete. 2024-09-21T07:53:19.932 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 291, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:53:19.187128+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 249, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 11364, 'mds_1': 11052, 'mds_2': 10621}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10621': {'gid': 10621, 'name': 'b', 'rank': 2, 'incarnation': 287, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.148:6835/2191156088', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 2191156088}, {'type': 'v1', 'addr': '172.21.15.148:6835', 'nonce': 2191156088}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11052': {'gid': 11052, 'name': 'c', 'rank': 1, 'incarnation': 261, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.39:6835/485194245', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 485194245}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 485194245}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11364': {'gid': 11364, 'name': 'a', 'rank': 0, 'incarnation': 285, 'state': 'up:active', 'state_seq': 28, 'addr': '172.21.15.39:6834/2161818466', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 2161818466}, {'type': 'v1', 'addr': '172.21.15.39:6834', 'nonce': 2161818466}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 11364, 'qdb_cluster': [10621, 11052, 11364]} 2024-09-21T07:53:19.932 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-21T07:53:19.933 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-21T07:53:20.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.161+0000 7f46f9e00640 1 -- 172.21.15.39:0/3435666638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46dc004d90 msgr2=0x7f46dc005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:20.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.161+0000 7f46f9e00640 1 --2- 172.21.15.39:0/3435666638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46dc004d90 0x7f46dc005220 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 crypto rx=0x7f46ec005fd0 tx=0x7f46dc005830 comp rx=0 tx=0).stop 2024-09-21T07:53:20.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.161+0000 7f46f9400640 1 -- 172.21.15.39:0/3435666638 reap_dead start 2024-09-21T07:53:20.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.162+0000 7f46f9e00640 1 -- 172.21.15.39:0/3435666638 shutdown_connections 2024-09-21T07:53:20.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.162+0000 7f46f9e00640 1 -- 172.21.15.39:0/3435666638 >> 172.21.15.39:0/3435666638 conn(0x7f46f40b2510 msgr2=0x7f46f40d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:20.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.162+0000 7f46f9e00640 1 -- 172.21.15.39:0/3435666638 shutdown_connections 2024-09-21T07:53:20.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.162+0000 7f46f9e00640 1 -- 172.21.15.39:0/3435666638 wait complete. 2024-09-21T07:53:20.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.164+0000 7f46f9e00640 1 Processor -- start 2024-09-21T07:53:20.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.164+0000 7f46f9e00640 1 -- start start 2024-09-21T07:53:20.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.165+0000 7f46f9e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46dc004d90 0x7f46f415abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.165+0000 7f46f9e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46f40ca0a0 0x7f46f4073890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.166+0000 7f46f9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46f4073dd0 0x7f46f40c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.166+0000 7f46f9e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f46f41658b0 con 0x7f46f4073dd0 2024-09-21T07:53:20.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.166+0000 7f46f9e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f46f40cd730 con 0x7f46f40ca0a0 2024-09-21T07:53:20.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.166+0000 7f46f9e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f46f4165a30 con 0x7f46dc004d90 2024-09-21T07:53:20.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.166+0000 7f46f9400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46f4073dd0 0x7f46f40c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.166+0000 7f46f9400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46f4073dd0 0x7f46f40c8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45158/0 (socket says 172.21.15.39:45158) 2024-09-21T07:53:20.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.166+0000 7f46f9400640 1 -- 172.21.15.39:0/4018009416 learned_addr learned my addr 172.21.15.39:0/4018009416 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:20.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.166+0000 7f46f8a00640 1 --2- 172.21.15.39:0/4018009416 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46dc004d90 0x7f46f415abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.166+0000 7f46f3e00640 1 --2- 172.21.15.39:0/4018009416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46f40ca0a0 0x7f46f4073890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.166+0000 7f46f9400640 1 -- 172.21.15.39:0/4018009416 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46dc004d90 msgr2=0x7f46f415abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:20.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.166+0000 7f46f9400640 1 --2- 172.21.15.39:0/4018009416 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46dc004d90 0x7f46f415abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.166+0000 7f46f9400640 1 -- 172.21.15.39:0/4018009416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46f40ca0a0 msgr2=0x7f46f4073890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:20.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.166+0000 7f46f9400640 1 --2- 172.21.15.39:0/4018009416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46f40ca0a0 0x7f46f4073890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.166+0000 7f46f9400640 1 -- 172.21.15.39:0/4018009416 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f40741e0 con 0x7f46f4073dd0 2024-09-21T07:53:20.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.166+0000 7f46f3e00640 1 --2- 172.21.15.39:0/4018009416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46f40ca0a0 0x7f46f4073890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:53:20.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.167+0000 7f46f9400640 1 --2- 172.21.15.39:0/4018009416 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46f4073dd0 0x7f46f40c8bc0 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 crypto rx=0x7f46ec02aea0 tx=0x7f46ec033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:20.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.167+0000 7f46f0c00640 1 -- 172.21.15.39:0/4018009416 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec042070 con 0x7f46f4073dd0 2024-09-21T07:53:20.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.167+0000 7f46f0c00640 1 -- 172.21.15.39:0/4018009416 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46ec003970 con 0x7f46f4073dd0 2024-09-21T07:53:20.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.167+0000 7f46f9e00640 1 -- 172.21.15.39:0/4018009416 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46f415ebb0 con 0x7f46f4073dd0 2024-09-21T07:53:20.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.167+0000 7f46f0c00640 1 -- 172.21.15.39:0/4018009416 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec038040 con 0x7f46f4073dd0 2024-09-21T07:53:20.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.167+0000 7f46f9e00640 1 -- 172.21.15.39:0/4018009416 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46f40c6530 con 0x7f46f4073dd0 2024-09-21T07:53:20.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.170+0000 7f46f0c00640 1 -- 172.21.15.39:0/4018009416 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f46ec004030 con 0x7f46f4073dd0 2024-09-21T07:53:20.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.171+0000 7f46f0c00640 1 --2- 172.21.15.39:0/4018009416 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f46d4078b90 0x7f46d407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.172+0000 7f46f0c00640 1 -- 172.21.15.39:0/4018009416 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(250..250 src has 1..250) ==== 7820+0+0 (secure 0 0 0) 0x7f46ec0bba30 con 0x7f46f4073dd0 2024-09-21T07:53:20.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.172+0000 7f470246c740 1 Processor -- start 2024-09-21T07:53:20.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.172+0000 7f470246c740 1 -- start start 2024-09-21T07:53:20.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.172+0000 7f46f8a00640 1 --2- 172.21.15.39:0/4018009416 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f46d4078b90 0x7f46d407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.173+0000 7f46f8a00640 1 --2- 172.21.15.39:0/4018009416 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f46d4078b90 0x7f46d407b0a0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f46f4161330 tx=0x7f46e80015f0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:20.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.173+0000 7f470246c740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d47293a270 0x55d472d0ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.173+0000 7f470246c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d4729113b0 0x55d472d0b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.173+0000 7f470246c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d472919590 0x55d472cdb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.173+0000 7f470246c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d472c64430 con 0x55d472919590 2024-09-21T07:53:20.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.173+0000 7f470246c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d472c545d0 con 0x55d4729113b0 2024-09-21T07:53:20.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.173+0000 7f470246c740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55d472c43a00 con 0x55d47293a270 2024-09-21T07:53:20.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.174+0000 7f46f9400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d472919590 0x55d472cdb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.174+0000 7f46f9400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d472919590 0x55d472cdb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45168/0 (socket says 172.21.15.39:45168) 2024-09-21T07:53:20.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.174+0000 7f46f9400640 1 -- 172.21.15.39:0/3828549462 learned_addr learned my addr 172.21.15.39:0/3828549462 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:20.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.174+0000 7f46f8a00640 1 --2- 172.21.15.39:0/3828549462 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d47293a270 0x55d472d0ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.174+0000 7f46f3e00640 1 --2- 172.21.15.39:0/3828549462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d4729113b0 0x55d472d0b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.174+0000 7f46f9400640 1 -- 172.21.15.39:0/3828549462 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d47293a270 msgr2=0x55d472d0ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:53:20.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.174+0000 7f46f9400640 1 --2- 172.21.15.39:0/3828549462 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d47293a270 0x55d472d0ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.174+0000 7f46f9400640 1 -- 172.21.15.39:0/3828549462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d4729113b0 msgr2=0x55d472d0b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:53:20.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.174+0000 7f46f9400640 1 --2- 172.21.15.39:0/3828549462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d4729113b0 0x55d472d0b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.174+0000 7f46f9400640 1 -- 172.21.15.39:0/3828549462 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d472c0c520 con 0x55d472919590 2024-09-21T07:53:20.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.174+0000 7f46f9400640 1 --2- 172.21.15.39:0/3828549462 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d472919590 0x55d472cdb150 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7f46ec009c40 tx=0x7f46ec0826b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5577a93ef033877a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:20.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.175+0000 7f46e5200640 1 -- 172.21.15.39:0/3828549462 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec082f40 con 0x55d472919590 2024-09-21T07:53:20.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.175+0000 7f46e5200640 1 -- 172.21.15.39:0/3828549462 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46ec08c8b0 con 0x55d472919590 2024-09-21T07:53:20.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.175+0000 7f46e5200640 1 -- 172.21.15.39:0/3828549462 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec0a9170 con 0x55d472919590 2024-09-21T07:53:20.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.175+0000 7f470246c740 1 -- 172.21.15.39:0/3828549462 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d472919590 msgr2=0x55d472cdb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:20.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.175+0000 7f470246c740 1 --2- 172.21.15.39:0/3828549462 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d472919590 0x55d472cdb150 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7f46ec009c40 tx=0x7f46ec0826b0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.176+0000 7f470246c740 1 -- 172.21.15.39:0/3828549462 shutdown_connections 2024-09-21T07:53:20.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.176+0000 7f470246c740 1 --2- 172.21.15.39:0/3828549462 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d472919590 0x55d472cdb150 unknown :-1 s=CLOSED pgs=2607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.176+0000 7f470246c740 1 --2- 172.21.15.39:0/3828549462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d4729113b0 0x55d472d0b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.176+0000 7f470246c740 1 --2- 172.21.15.39:0/3828549462 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d47293a270 0x55d472d0ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.176+0000 7f470246c740 1 -- 172.21.15.39:0/3828549462 >> 172.21.15.39:0/3828549462 conn(0x55d47292e3c0 msgr2=0x55d472ce0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:20.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.176+0000 7f470246c740 1 -- 172.21.15.39:0/3828549462 shutdown_connections 2024-09-21T07:53:20.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.176+0000 7f470246c740 1 -- 172.21.15.39:0/3828549462 wait complete. 2024-09-21T07:53:20.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.177+0000 7f470246c740 1 -- start start 2024-09-21T07:53:20.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.177+0000 7f470246c740 1 Processor -- start 2024-09-21T07:53:20.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.178+0000 7f470246c740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d4729113b0 0x55d472d08260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.178+0000 7f470246c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d472919590 0x55d472d1f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.178+0000 7f470246c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d47293a270 0x55d472d380d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.179+0000 7f470246c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d472bea330 con 0x55d47293a270 2024-09-21T07:53:20.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.179+0000 7f470246c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d472c47810 con 0x55d472919590 2024-09-21T07:53:20.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.179+0000 7f470246c740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55d4729b4620 con 0x55d4729113b0 2024-09-21T07:53:20.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.179+0000 7f470246c740 10 client.0 _ll_register_callbacks cb 0x55d472bef420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:53:20.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.179+0000 7f46f9400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d47293a270 0x55d472d380d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.179+0000 7f46f9400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d47293a270 0x55d472d380d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45172/0 (socket says 172.21.15.39:45172) 2024-09-21T07:53:20.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.179+0000 7f46f9400640 1 -- 172.21.15.39:0/3617320181 learned_addr learned my addr 172.21.15.39:0/3617320181 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:20.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.179+0000 7f46f8a00640 1 --2- 172.21.15.39:0/3617320181 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d4729113b0 0x55d472d08260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.179+0000 7f46f3e00640 1 --2- 172.21.15.39:0/3617320181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d472919590 0x55d472d1f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.179+0000 7f46f9400640 1 -- 172.21.15.39:0/3617320181 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d4729113b0 msgr2=0x55d472d08260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:53:20.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.179+0000 7f46f9400640 1 --2- 172.21.15.39:0/3617320181 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d4729113b0 0x55d472d08260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.179+0000 7f46f9400640 1 -- 172.21.15.39:0/3617320181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d472919590 msgr2=0x55d472d1f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:53:20.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.179+0000 7f46f9400640 1 --2- 172.21.15.39:0/3617320181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d472919590 0x55d472d1f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.179+0000 7f46f9400640 1 -- 172.21.15.39:0/3617320181 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4729a5850 con 0x55d47293a270 2024-09-21T07:53:20.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.180+0000 7f46f9400640 1 --2- 172.21.15.39:0/3617320181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d47293a270 0x55d472d380d0 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7f46ec082bf0 tx=0x7f46ec005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11da557ee2dd07ad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:20.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.180+0000 7f46d9600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:53:20.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.180+0000 7f46d9600640 1 -- 172.21.15.39:0/3617320181 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec082f40 con 0x55d47293a270 2024-09-21T07:53:20.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.180+0000 7f46d9600640 1 -- 172.21.15.39:0/3617320181 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(250..250 src has 1..250) ==== 7820+0+0 (secure 0 0 0) 0x7f46ec07f1b0 con 0x55d47293a270 2024-09-21T07:53:20.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.180+0000 7f470246c740 1 -- 172.21.15.39:0/3617320181 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46ec08c8b0 con 0x55d47293a270 2024-09-21T07:53:20.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.180+0000 7f46d9600640 1 -- 172.21.15.39:0/3617320181 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=291) ==== 24+0+0 (secure 0 0 0) 0x7f46ec0a51a0 con 0x55d47293a270 2024-09-21T07:53:20.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.180+0000 7f470246c740 10 client.11787 fetch_fsmap learned FSMap version 291 2024-09-21T07:53:20.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.180+0000 7f470246c740 1 -- 172.21.15.39:0/3617320181 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=291,monmap=2+}) -- 0x55d472a1c6b0 con 0x55d47293a270 2024-09-21T07:53:20.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.180+0000 7f46d9600640 1 -- 172.21.15.39:0/3617320181 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46ec08c8b0 con 0x55d47293a270 2024-09-21T07:53:20.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.180+0000 7f46d9600640 1 -- 172.21.15.39:0/3617320181 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 291) ==== 3542+0+0 (secure 0 0 0) 0x7f46ec03b070 con 0x55d47293a270 2024-09-21T07:53:20.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.180+0000 7f470246c740 10 client.11787 fetch_fsmap finished waiting for FSMap version 291 2024-09-21T07:53:20.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.180+0000 7f470246c740 10 client.11787 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T07:53:20.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.180+0000 7f470246c740 20 client.11787 populate_metadata read hostname 'smithi039' 2024-09-21T07:53:20.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.180+0000 7f470246c740 1 --2- 172.21.15.39:0/3617320181 >> [v2:172.21.15.148:6834/2191156088,v1:172.21.15.148:6835/2191156088] conn(0x55d4728f8e40 0x55d472d423e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.180+0000 7f470246c740 4 client.11787 mds_command: new command op to 10621 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:53:20.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.180+0000 7f470246c740 1 -- 172.21.15.39:0/3617320181 --> [v2:172.21.15.148:6834/2191156088,v1:172.21.15.148:6835/2191156088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d472c56520 con 0x55d4728f8e40 2024-09-21T07:53:20.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.181+0000 7f46f8a00640 1 --2- 172.21.15.39:0/3617320181 >> [v2:172.21.15.148:6834/2191156088,v1:172.21.15.148:6835/2191156088] conn(0x55d4728f8e40 0x55d472d423e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.182+0000 7f46f8a00640 1 --2- 172.21.15.39:0/3617320181 >> [v2:172.21.15.148:6834/2191156088,v1:172.21.15.148:6835/2191156088] conn(0x55d4728f8e40 0x55d472d423e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b55c207e9144e749 server_cookie=14b334482f8e8d58 in_seq=0 out_seq=0 2024-09-21T07:53:20.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.182+0000 7f46d9600640 10 client.11787 ms_handle_connect on v2:172.21.15.148:6834/2191156088 2024-09-21T07:53:20.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.183+0000 7f46d9600640 1 -- 172.21.15.39:0/3617320181 <== mds.2 v2:172.21.15.148:6834/2191156088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d472c56520 con 0x55d4728f8e40 2024-09-21T07:53:20.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.183+0000 7f46d9600640 10 client.11787 handle_command_reply: tid=0 2024-09-21T07:53:20.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.183+0000 7f470246c740 1 client.11787 shutdown 2024-09-21T07:53:20.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.184+0000 7f470246c740 1 -- 172.21.15.39:0/3617320181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d47293a270 msgr2=0x55d472d380d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:20.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.184+0000 7f470246c740 1 --2- 172.21.15.39:0/3617320181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d47293a270 0x55d472d380d0 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7f46ec082bf0 tx=0x7f46ec005c50 comp rx=0 tx=0).stop 2024-09-21T07:53:20.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.184+0000 7f470246c740 1 -- 172.21.15.39:0/3617320181 shutdown_connections 2024-09-21T07:53:20.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.184+0000 7f470246c740 1 --2- 172.21.15.39:0/3617320181 >> [v2:172.21.15.148:6834/2191156088,v1:172.21.15.148:6835/2191156088] conn(0x55d4728f8e40 0x55d472d423e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.184+0000 7f470246c740 1 --2- 172.21.15.39:0/3617320181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d47293a270 0x55d472d380d0 unknown :-1 s=CLOSED pgs=2608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.184+0000 7f470246c740 1 --2- 172.21.15.39:0/3617320181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d472919590 0x55d472d1f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.184+0000 7f470246c740 1 --2- 172.21.15.39:0/3617320181 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d4729113b0 0x55d472d08260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.184+0000 7f46d9600640 0 client.11787 ms_handle_reset on v2:172.21.15.148:6834/2191156088 2024-09-21T07:53:20.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.184+0000 7f470246c740 1 -- 172.21.15.39:0/3617320181 >> 172.21.15.39:0/3617320181 conn(0x55d47292e3c0 msgr2=0x55d472d26a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:20.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.184+0000 7f470246c740 1 -- 172.21.15.39:0/3617320181 shutdown_connections 2024-09-21T07:53:20.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.184+0000 7f470246c740 1 -- 172.21.15.39:0/3617320181 wait complete. 2024-09-21T07:53:20.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.184+0000 7f470246c740 20 client.11787 trim_cache size 0 max 16384 2024-09-21T07:53:20.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.204+0000 7f470246c740 1 Processor -- start 2024-09-21T07:53:20.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.204+0000 7f470246c740 1 -- start start 2024-09-21T07:53:20.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.206+0000 7f470246c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4729113b0 0x55d472d26a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.206+0000 7f470246c740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d472919590 0x55d472d26f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.206+0000 7f470246c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d47293a270 0x55d472d0a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.206+0000 7f470246c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f46f4202630 con 0x55d4729113b0 2024-09-21T07:53:20.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.206+0000 7f470246c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f46f4202d00 con 0x55d47293a270 2024-09-21T07:53:20.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.206+0000 7f470246c740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f46f42015f0 con 0x55d472919590 2024-09-21T07:53:20.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.206+0000 7f46f8a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4729113b0 0x55d472d26a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.206+0000 7f46f8a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4729113b0 0x55d472d26a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45182/0 (socket says 172.21.15.39:45182) 2024-09-21T07:53:20.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.206+0000 7f46f8a00640 1 -- 172.21.15.39:0/3249188025 learned_addr learned my addr 172.21.15.39:0/3249188025 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:20.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.206+0000 7f46f9400640 1 --2- 172.21.15.39:0/3249188025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d47293a270 0x55d472d0a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.206+0000 7f46f3e00640 1 --2- 172.21.15.39:0/3249188025 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d472919590 0x55d472d26f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.206+0000 7f46f8a00640 1 -- 172.21.15.39:0/3249188025 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d472919590 msgr2=0x55d472d26f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:53:20.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.206+0000 7f46f8a00640 1 --2- 172.21.15.39:0/3249188025 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d472919590 0x55d472d26f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.206+0000 7f46f8a00640 1 -- 172.21.15.39:0/3249188025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d47293a270 msgr2=0x55d472d0a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:53:20.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.207+0000 7f46f8a00640 1 --2- 172.21.15.39:0/3249188025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d47293a270 0x55d472d0a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.207+0000 7f46f8a00640 1 -- 172.21.15.39:0/3249188025 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d472ce1840 con 0x55d4729113b0 2024-09-21T07:53:20.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.207+0000 7f46f8a00640 1 --2- 172.21.15.39:0/3249188025 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4729113b0 0x55d472d26a10 secure :-1 s=READY pgs=2609 cs=0 l=1 rev1=1 crypto rx=0x7f46e8006490 tx=0x7f46e8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9567c2c948adcf8b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:20.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.207+0000 7f46dbe00640 1 -- 172.21.15.39:0/3249188025 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e8004ce0 con 0x55d4729113b0 2024-09-21T07:53:20.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.207+0000 7f46dbe00640 1 -- 172.21.15.39:0/3249188025 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46e8004e80 con 0x55d4729113b0 2024-09-21T07:53:20.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.207+0000 7f46dbe00640 1 -- 172.21.15.39:0/3249188025 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e801f660 con 0x55d4729113b0 2024-09-21T07:53:20.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.208+0000 7f470246c740 1 -- 172.21.15.39:0/3249188025 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4729113b0 msgr2=0x55d472d26a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:20.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.208+0000 7f470246c740 1 --2- 172.21.15.39:0/3249188025 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4729113b0 0x55d472d26a10 secure :-1 s=READY pgs=2609 cs=0 l=1 rev1=1 crypto rx=0x7f46e8006490 tx=0x7f46e8015000 comp rx=0 tx=0).stop 2024-09-21T07:53:20.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.208+0000 7f470246c740 1 -- 172.21.15.39:0/3249188025 shutdown_connections 2024-09-21T07:53:20.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.208+0000 7f470246c740 1 --2- 172.21.15.39:0/3249188025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d47293a270 0x55d472d0a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.208+0000 7f470246c740 1 --2- 172.21.15.39:0/3249188025 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d472919590 0x55d472d26f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.208+0000 7f470246c740 1 --2- 172.21.15.39:0/3249188025 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4729113b0 0x55d472d26a10 unknown :-1 s=CLOSED pgs=2609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.208+0000 7f470246c740 1 -- 172.21.15.39:0/3249188025 >> 172.21.15.39:0/3249188025 conn(0x55d47292e3c0 msgr2=0x55d472cde5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:20.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.208+0000 7f470246c740 1 -- 172.21.15.39:0/3249188025 shutdown_connections 2024-09-21T07:53:20.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.208+0000 7f470246c740 1 -- 172.21.15.39:0/3249188025 wait complete. 2024-09-21T07:53:20.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.209+0000 7f470246c740 1 -- start start 2024-09-21T07:53:20.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.209+0000 7f470246c740 1 Processor -- start 2024-09-21T07:53:20.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.211+0000 7f470246c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4729113b0 0x55d472d9b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.211+0000 7f470246c740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d472919590 0x55d472d048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.211+0000 7f470246c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d47293a270 0x55d472d04df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.211+0000 7f470246c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d4729b4620 con 0x55d4729113b0 2024-09-21T07:53:20.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.211+0000 7f470246c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d472bea330 con 0x55d47293a270 2024-09-21T07:53:20.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.211+0000 7f470246c740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55d472c47810 con 0x55d472919590 2024-09-21T07:53:20.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.211+0000 7f470246c740 10 client.0 _ll_register_callbacks cb 0x7f46e8000ff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:53:20.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.211+0000 7f46f8a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4729113b0 0x55d472d9b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.211+0000 7f46f8a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4729113b0 0x55d472d9b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45190/0 (socket says 172.21.15.39:45190) 2024-09-21T07:53:20.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.211+0000 7f46f8a00640 1 -- 172.21.15.39:0/2653329241 learned_addr learned my addr 172.21.15.39:0/2653329241 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:20.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.211+0000 7f46f3e00640 1 --2- 172.21.15.39:0/2653329241 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d472919590 0x55d472d048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.211+0000 7f46f9400640 1 --2- 172.21.15.39:0/2653329241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d47293a270 0x55d472d04df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.211+0000 7f46f8a00640 1 -- 172.21.15.39:0/2653329241 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d472919590 msgr2=0x55d472d048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:53:20.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.211+0000 7f46f8a00640 1 --2- 172.21.15.39:0/2653329241 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d472919590 0x55d472d048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.211+0000 7f46f8a00640 1 -- 172.21.15.39:0/2653329241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d47293a270 msgr2=0x55d472d04df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:53:20.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.211+0000 7f46f8a00640 1 --2- 172.21.15.39:0/2653329241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d47293a270 0x55d472d04df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.211+0000 7f46f8a00640 1 -- 172.21.15.39:0/2653329241 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d472d08c80 con 0x55d4729113b0 2024-09-21T07:53:20.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.212+0000 7f46f8a00640 1 --2- 172.21.15.39:0/2653329241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4729113b0 0x55d472d9b670 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x7f46e8004aa0 tx=0x7f46e8004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=727774fcb76705cb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:20.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.212+0000 7f46d9600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:53:20.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.212+0000 7f46d9600640 1 -- 172.21.15.39:0/2653329241 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e803b640 con 0x55d4729113b0 2024-09-21T07:53:20.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.212+0000 7f46d9600640 1 -- 172.21.15.39:0/2653329241 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(250..250 src has 1..250) ==== 7820+0+0 (secure 0 0 0) 0x7f46e8031440 con 0x55d4729113b0 2024-09-21T07:53:20.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.212+0000 7f470246c740 1 -- 172.21.15.39:0/2653329241 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46e8004e80 con 0x55d4729113b0 2024-09-21T07:53:20.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.212+0000 7f46d9600640 1 -- 172.21.15.39:0/2653329241 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=291) ==== 24+0+0 (secure 0 0 0) 0x7f46e803b7e0 con 0x55d4729113b0 2024-09-21T07:53:20.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.212+0000 7f470246c740 10 client.11793 fetch_fsmap learned FSMap version 291 2024-09-21T07:53:20.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.212+0000 7f470246c740 1 -- 172.21.15.39:0/2653329241 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=291,monmap=2+}) -- 0x55d472da5260 con 0x55d4729113b0 2024-09-21T07:53:20.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.213+0000 7f46d9600640 1 -- 172.21.15.39:0/2653329241 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46e8004e80 con 0x55d4729113b0 2024-09-21T07:53:20.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.213+0000 7f46d9600640 1 -- 172.21.15.39:0/2653329241 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 291) ==== 3542+0+0 (secure 0 0 0) 0x7f46e8039070 con 0x55d4729113b0 2024-09-21T07:53:20.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.213+0000 7f470246c740 10 client.11793 fetch_fsmap finished waiting for FSMap version 291 2024-09-21T07:53:20.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.213+0000 7f470246c740 10 client.11793 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T07:53:20.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.213+0000 7f470246c740 20 client.11793 populate_metadata read hostname 'smithi039' 2024-09-21T07:53:20.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.213+0000 7f470246c740 1 --2- 172.21.15.39:0/2653329241 >> [v2:172.21.15.148:6834/2191156088,v1:172.21.15.148:6835/2191156088] conn(0x55d472d9e8e0 0x55d472d27890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.213+0000 7f470246c740 4 client.11793 mds_command: new command op to 10621 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:53:20.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.213+0000 7f470246c740 1 -- 172.21.15.39:0/2653329241 --> [v2:172.21.15.148:6834/2191156088,v1:172.21.15.148:6835/2191156088] -- command(tid 0: {"prefix": "status"}) -- 0x55d472c51fd0 con 0x55d472d9e8e0 2024-09-21T07:53:20.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.213+0000 7f46f3e00640 1 --2- 172.21.15.39:0/2653329241 >> [v2:172.21.15.148:6834/2191156088,v1:172.21.15.148:6835/2191156088] conn(0x55d472d9e8e0 0x55d472d27890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.214+0000 7f46f3e00640 1 --2- 172.21.15.39:0/2653329241 >> [v2:172.21.15.148:6834/2191156088,v1:172.21.15.148:6835/2191156088] conn(0x55d472d9e8e0 0x55d472d27890 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b671492e3534bb46 server_cookie=82db793c227d111d in_seq=0 out_seq=0 2024-09-21T07:53:20.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.214+0000 7f46d9600640 10 client.11793 ms_handle_connect on v2:172.21.15.148:6834/2191156088 2024-09-21T07:53:20.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.214+0000 7f46d9600640 1 -- 172.21.15.39:0/2653329241 <== mds.2 v2:172.21.15.148:6834/2191156088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+329 (crc 0 0 0) 0x55d472c51fd0 con 0x55d472d9e8e0 2024-09-21T07:53:20.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.214+0000 7f46d9600640 10 client.11793 handle_command_reply: tid=0 2024-09-21T07:53:20.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.214+0000 7f470246c740 1 client.11793 shutdown 2024-09-21T07:53:20.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.215+0000 7f470246c740 1 -- 172.21.15.39:0/2653329241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4729113b0 msgr2=0x55d472d9b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:20.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.215+0000 7f470246c740 1 --2- 172.21.15.39:0/2653329241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4729113b0 0x55d472d9b670 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x7f46e8004aa0 tx=0x7f46e8004760 comp rx=0 tx=0).stop 2024-09-21T07:53:20.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.215+0000 7f470246c740 1 -- 172.21.15.39:0/2653329241 shutdown_connections 2024-09-21T07:53:20.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.215+0000 7f470246c740 1 --2- 172.21.15.39:0/2653329241 >> [v2:172.21.15.148:6834/2191156088,v1:172.21.15.148:6835/2191156088] conn(0x55d472d9e8e0 0x55d472d27890 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.215+0000 7f470246c740 1 --2- 172.21.15.39:0/2653329241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d47293a270 0x55d472d04df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.215+0000 7f470246c740 1 --2- 172.21.15.39:0/2653329241 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55d472919590 0x55d472d048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.215+0000 7f470246c740 1 --2- 172.21.15.39:0/2653329241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d4729113b0 0x55d472d9b670 unknown :-1 s=CLOSED pgs=2610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.215+0000 7f46d9600640 0 client.11793 ms_handle_reset on v2:172.21.15.148:6834/2191156088 2024-09-21T07:53:20.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.215+0000 7f470246c740 1 -- 172.21.15.39:0/2653329241 >> 172.21.15.39:0/2653329241 conn(0x55d47292e3c0 msgr2=0x55d472cde5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:20.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.215+0000 7f470246c740 1 -- 172.21.15.39:0/2653329241 shutdown_connections 2024-09-21T07:53:20.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.215+0000 7f470246c740 1 -- 172.21.15.39:0/2653329241 wait complete. 2024-09-21T07:53:20.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.215+0000 7f470246c740 20 client.11793 trim_cache size 0 max 16384 2024-09-21T07:53:20.220 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:53:20.220 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:53:20.221 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 2, 2024-09-21T07:53:20.221 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 10621, 2024-09-21T07:53:20.221 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:53:20.221 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:53:20.221 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:53:20.221 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 11.084107969, 2024-09-21T07:53:20.221 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 291, 2024-09-21T07:53:20.221 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 250, 2024-09-21T07:53:20.221 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 250, 2024-09-21T07:53:20.221 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 11.497908556000001 2024-09-21T07:53:20.221 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:53:20.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.216+0000 7f46f9e00640 1 -- 172.21.15.39:0/4018009416 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f46d4078b90 msgr2=0x7f46d407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:20.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.216+0000 7f46f9e00640 1 --2- 172.21.15.39:0/4018009416 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f46d4078b90 0x7f46d407b0a0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f46f4161330 tx=0x7f46e80015f0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.216+0000 7f46f9e00640 1 -- 172.21.15.39:0/4018009416 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46f4073dd0 msgr2=0x7f46f40c8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:20.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.216+0000 7f46f9e00640 1 --2- 172.21.15.39:0/4018009416 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46f4073dd0 0x7f46f40c8bc0 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 crypto rx=0x7f46ec02aea0 tx=0x7f46ec033ba0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.216+0000 7f46f9e00640 1 -- 172.21.15.39:0/4018009416 shutdown_connections 2024-09-21T07:53:20.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.216+0000 7f46f9e00640 1 --2- 172.21.15.39:0/4018009416 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f46d4078b90 0x7f46d407b0a0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.216+0000 7f46f9e00640 1 --2- 172.21.15.39:0/4018009416 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46f4073dd0 0x7f46f40c8bc0 unknown :-1 s=CLOSED pgs=2606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.216+0000 7f46f9e00640 1 --2- 172.21.15.39:0/4018009416 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46f40ca0a0 0x7f46f4073890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.216+0000 7f46f9e00640 1 --2- 172.21.15.39:0/4018009416 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46dc004d90 0x7f46f415abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.216+0000 7f46f9e00640 1 -- 172.21.15.39:0/4018009416 >> 172.21.15.39:0/4018009416 conn(0x7f46f40b2510 msgr2=0x7f46f4072f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:20.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.216+0000 7f46f9e00640 1 -- 172.21.15.39:0/4018009416 shutdown_connections 2024-09-21T07:53:20.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.216+0000 7f46f9e00640 1 -- 172.21.15.39:0/4018009416 wait complete. 2024-09-21T07:53:20.239 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T07:53:20.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.465+0000 7f402e800640 1 -- 172.21.15.39:0/627872257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4028155a70 msgr2=0x7f40280b39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:20.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.465+0000 7f402e800640 1 --2- 172.21.15.39:0/627872257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4028155a70 0x7f40280b39f0 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 crypto rx=0x7f4028072c00 tx=0x7f401002a4d0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.465+0000 7f402e800640 1 -- 172.21.15.39:0/627872257 shutdown_connections 2024-09-21T07:53:20.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.465+0000 7f402e800640 1 --2- 172.21.15.39:0/627872257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4028155a70 0x7f40280b39f0 unknown :-1 s=CLOSED pgs=2611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.465+0000 7f402e800640 1 --2- 172.21.15.39:0/627872257 >> v2:172.21.15.148:3300/0 conn(0x7f402815f360 0x7f402815bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.465+0000 7f402e800640 1 -- 172.21.15.39:0/627872257 >> 172.21.15.39:0/627872257 conn(0x7f402806cfc0 msgr2=0x7f402806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:20.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.465+0000 7f402e800640 1 -- 172.21.15.39:0/627872257 shutdown_connections 2024-09-21T07:53:20.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.465+0000 7f402e800640 1 -- 172.21.15.39:0/627872257 wait complete. 2024-09-21T07:53:20.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.467+0000 7f402e800640 1 Processor -- start 2024-09-21T07:53:20.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.468+0000 7f402e800640 1 -- start start 2024-09-21T07:53:20.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.469+0000 7f402e800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40280b3f30 0x7f4028156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.469+0000 7f402e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4028155a70 0x7f4028151ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.469+0000 7f402e800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f402815f360 0x7f402814e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.469+0000 7f402e800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f40280d0620 con 0x7f4028155a70 2024-09-21T07:53:20.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.469+0000 7f402e800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f40281560d0 con 0x7f40280b3f30 2024-09-21T07:53:20.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.469+0000 7f402e800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f40280d07a0 con 0x7f402815f360 2024-09-21T07:53:20.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.469+0000 7f401fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4028155a70 0x7f4028151ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.469+0000 7f401fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4028155a70 0x7f4028151ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45222/0 (socket says 172.21.15.39:45222) 2024-09-21T07:53:20.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.469+0000 7f401fe00640 1 -- 172.21.15.39:0/2038378424 learned_addr learned my addr 172.21.15.39:0/2038378424 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:20.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.470+0000 7f4027400640 1 --2- 172.21.15.39:0/2038378424 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40280b3f30 0x7f4028156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.470+0000 7f4027e00640 1 --2- 172.21.15.39:0/2038378424 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f402815f360 0x7f402814e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.470+0000 7f401fe00640 1 -- 172.21.15.39:0/2038378424 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f402815f360 msgr2=0x7f402814e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:20.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.470+0000 7f401fe00640 1 --2- 172.21.15.39:0/2038378424 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f402815f360 0x7f402814e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.470+0000 7f401fe00640 1 -- 172.21.15.39:0/2038378424 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40280b3f30 msgr2=0x7f4028156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:20.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.470+0000 7f401fe00640 1 --2- 172.21.15.39:0/2038378424 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40280b3f30 0x7f4028156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.470+0000 7f401fe00640 1 -- 172.21.15.39:0/2038378424 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4028152010 con 0x7f4028155a70 2024-09-21T07:53:20.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.470+0000 7f4027e00640 1 --2- 172.21.15.39:0/2038378424 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f402815f360 0x7f402814e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:53:20.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.470+0000 7f401fe00640 1 --2- 172.21.15.39:0/2038378424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4028155a70 0x7f4028151ad0 secure :-1 s=READY pgs=2612 cs=0 l=1 rev1=1 crypto rx=0x7f4010034040 tx=0x7f40100039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:20.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.471+0000 7f401f400640 1 -- 172.21.15.39:0/2038378424 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401003e030 con 0x7f4028155a70 2024-09-21T07:53:20.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.471+0000 7f401f400640 1 -- 172.21.15.39:0/2038378424 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40100047a0 con 0x7f4028155a70 2024-09-21T07:53:20.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.471+0000 7f401f400640 1 -- 172.21.15.39:0/2038378424 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4010035040 con 0x7f4028155a70 2024-09-21T07:53:20.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.471+0000 7f402e800640 1 -- 172.21.15.39:0/2038378424 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f402815b980 con 0x7f4028155a70 2024-09-21T07:53:20.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.471+0000 7f402e800640 1 -- 172.21.15.39:0/2038378424 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f402815bf40 con 0x7f4028155a70 2024-09-21T07:53:20.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.473+0000 7f401f400640 1 -- 172.21.15.39:0/2038378424 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4010041030 con 0x7f4028155a70 2024-09-21T07:53:20.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.474+0000 7f401f400640 1 --2- 172.21.15.39:0/2038378424 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4004078b90 0x7f400407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.474+0000 7f401f400640 1 -- 172.21.15.39:0/2038378424 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(250..250 src has 1..250) ==== 7820+0+0 (secure 0 0 0) 0x7f40100bbdf0 con 0x7f4028155a70 2024-09-21T07:53:20.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.475+0000 7f4027400640 1 --2- 172.21.15.39:0/2038378424 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4004078b90 0x7f400407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.475+0000 7f4035a1d740 1 Processor -- start 2024-09-21T07:53:20.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.475+0000 7f4035a1d740 1 -- start start 2024-09-21T07:53:20.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.475+0000 7f4027400640 1 --2- 172.21.15.39:0/2038378424 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4004078b90 0x7f400407b0a0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f4018005fd0 tx=0x7f401800a250 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:20.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.476+0000 7f4035a1d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae4b634270 0x55ae4ba04970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.476+0000 7f4035a1d740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae4b60b3b0 0x55ae4ba04eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.476+0000 7f4035a1d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4b613590 0x55ae4b9d4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.476+0000 7f4035a1d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ae4b95e430 con 0x55ae4b613590 2024-09-21T07:53:20.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.476+0000 7f4035a1d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ae4b94e5d0 con 0x55ae4b634270 2024-09-21T07:53:20.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.476+0000 7f4035a1d740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55ae4b93da00 con 0x55ae4b60b3b0 2024-09-21T07:53:20.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.477+0000 7f4027e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4b613590 0x55ae4b9d4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.477+0000 7f4027e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4b613590 0x55ae4b9d4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45228/0 (socket says 172.21.15.39:45228) 2024-09-21T07:53:20.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.477+0000 7f4027e00640 1 -- 172.21.15.39:0/1836786421 learned_addr learned my addr 172.21.15.39:0/1836786421 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:20.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.477+0000 7f4027400640 1 --2- 172.21.15.39:0/1836786421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae4b634270 0x55ae4ba04970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.477+0000 7f401fe00640 1 --2- 172.21.15.39:0/1836786421 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae4b60b3b0 0x55ae4ba04eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.477+0000 7f4027e00640 1 -- 172.21.15.39:0/1836786421 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae4b60b3b0 msgr2=0x55ae4ba04eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:53:20.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.477+0000 7f4027e00640 1 --2- 172.21.15.39:0/1836786421 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae4b60b3b0 0x55ae4ba04eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.477+0000 7f4027e00640 1 -- 172.21.15.39:0/1836786421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae4b634270 msgr2=0x55ae4ba04970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:53:20.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.477+0000 7f4027e00640 1 --2- 172.21.15.39:0/1836786421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae4b634270 0x55ae4ba04970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.477+0000 7f4027e00640 1 -- 172.21.15.39:0/1836786421 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae4b906520 con 0x55ae4b613590 2024-09-21T07:53:20.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.477+0000 7f4027e00640 1 --2- 172.21.15.39:0/1836786421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4b613590 0x55ae4b9d4f60 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7f4020009fd0 tx=0x7f402000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a915198aac45523 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:20.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.478+0000 7f4017e00640 1 -- 172.21.15.39:0/1836786421 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4020003970 con 0x55ae4b613590 2024-09-21T07:53:20.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.478+0000 7f4017e00640 1 -- 172.21.15.39:0/1836786421 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4020003b10 con 0x55ae4b613590 2024-09-21T07:53:20.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.478+0000 7f4017e00640 1 -- 172.21.15.39:0/1836786421 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4020004020 con 0x55ae4b613590 2024-09-21T07:53:20.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.478+0000 7f4035a1d740 1 -- 172.21.15.39:0/1836786421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4b613590 msgr2=0x55ae4b9d4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:20.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.479+0000 7f4035a1d740 1 --2- 172.21.15.39:0/1836786421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4b613590 0x55ae4b9d4f60 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7f4020009fd0 tx=0x7f402000c890 comp rx=0 tx=0).stop 2024-09-21T07:53:20.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.479+0000 7f4035a1d740 1 -- 172.21.15.39:0/1836786421 shutdown_connections 2024-09-21T07:53:20.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.479+0000 7f4035a1d740 1 --2- 172.21.15.39:0/1836786421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4b613590 0x55ae4b9d4f60 unknown :-1 s=CLOSED pgs=2613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.479+0000 7f4035a1d740 1 --2- 172.21.15.39:0/1836786421 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae4b60b3b0 0x55ae4ba04eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.479+0000 7f4035a1d740 1 --2- 172.21.15.39:0/1836786421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae4b634270 0x55ae4ba04970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.479+0000 7f4035a1d740 1 -- 172.21.15.39:0/1836786421 >> 172.21.15.39:0/1836786421 conn(0x55ae4b6283c0 msgr2=0x55ae4b9da970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:20.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.479+0000 7f4035a1d740 1 -- 172.21.15.39:0/1836786421 shutdown_connections 2024-09-21T07:53:20.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.479+0000 7f4035a1d740 1 -- 172.21.15.39:0/1836786421 wait complete. 2024-09-21T07:53:20.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.480+0000 7f4035a1d740 1 -- start start 2024-09-21T07:53:20.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.480+0000 7f4035a1d740 1 Processor -- start 2024-09-21T07:53:20.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.481+0000 7f4035a1d740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae4b60b3b0 0x55ae4ba20010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.481+0000 7f4035a1d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae4b613590 0x55ae4b9fafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.481+0000 7f4035a1d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4b634270 0x55ae4ba31ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.481+0000 7f4035a1d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ae4b8e4330 con 0x55ae4b634270 2024-09-21T07:53:20.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.481+0000 7f4035a1d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ae4b941810 con 0x55ae4b613590 2024-09-21T07:53:20.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.481+0000 7f4035a1d740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55ae4b6ae620 con 0x55ae4b60b3b0 2024-09-21T07:53:20.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.482+0000 7f4035a1d740 10 client.0 _ll_register_callbacks cb 0x55ae4b8e9420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:53:20.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.482+0000 7f4027e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4b634270 0x55ae4ba31ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.482+0000 7f4027e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4b634270 0x55ae4ba31ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45236/0 (socket says 172.21.15.39:45236) 2024-09-21T07:53:20.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.482+0000 7f4027e00640 1 -- 172.21.15.39:0/2001360922 learned_addr learned my addr 172.21.15.39:0/2001360922 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:20.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.482+0000 7f4027400640 1 --2- 172.21.15.39:0/2001360922 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae4b60b3b0 0x55ae4ba20010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.482+0000 7f401fe00640 1 --2- 172.21.15.39:0/2001360922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae4b613590 0x55ae4b9fafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.482+0000 7f4027e00640 1 -- 172.21.15.39:0/2001360922 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae4b60b3b0 msgr2=0x55ae4ba20010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:53:20.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.482+0000 7f4027e00640 1 --2- 172.21.15.39:0/2001360922 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae4b60b3b0 0x55ae4ba20010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.482+0000 7f4027e00640 1 -- 172.21.15.39:0/2001360922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae4b613590 msgr2=0x55ae4b9fafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:53:20.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.482+0000 7f4027e00640 1 --2- 172.21.15.39:0/2001360922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae4b613590 0x55ae4b9fafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.482+0000 7f4027e00640 1 -- 172.21.15.39:0/2001360922 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae4b69f850 con 0x55ae4b634270 2024-09-21T07:53:20.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.482+0000 7f4027400640 1 --2- 172.21.15.39:0/2001360922 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae4b60b3b0 0x55ae4ba20010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:53:20.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.482+0000 7f4027e00640 1 --2- 172.21.15.39:0/2001360922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4b634270 0x55ae4ba31ed0 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 crypto rx=0x7f4020017040 tx=0x7f402000ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=320cbdeafcdedd91 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:20.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.482+0000 7f4014c00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:53:20.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.483+0000 7f4014c00640 1 -- 172.21.15.39:0/2001360922 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4020024020 con 0x55ae4b634270 2024-09-21T07:53:20.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.483+0000 7f4035a1d740 1 -- 172.21.15.39:0/2001360922 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4020003b10 con 0x55ae4b634270 2024-09-21T07:53:20.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.483+0000 7f4014c00640 1 -- 172.21.15.39:0/2001360922 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(250..250 src has 1..250) ==== 7820+0+0 (secure 0 0 0) 0x7f4020012040 con 0x55ae4b634270 2024-09-21T07:53:20.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.483+0000 7f4014c00640 1 -- 172.21.15.39:0/2001360922 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=291) ==== 24+0+0 (secure 0 0 0) 0x7f4020003970 con 0x55ae4b634270 2024-09-21T07:53:20.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.483+0000 7f4035a1d740 10 client.11805 fetch_fsmap learned FSMap version 291 2024-09-21T07:53:20.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.483+0000 7f4035a1d740 1 -- 172.21.15.39:0/2001360922 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=291,monmap=2+}) -- 0x55ae4b7166b0 con 0x55ae4b634270 2024-09-21T07:53:20.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.483+0000 7f4014c00640 1 -- 172.21.15.39:0/2001360922 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4020003b10 con 0x55ae4b634270 2024-09-21T07:53:20.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.483+0000 7f4014c00640 1 -- 172.21.15.39:0/2001360922 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 291) ==== 3542+0+0 (secure 0 0 0) 0x7f402002c070 con 0x55ae4b634270 2024-09-21T07:53:20.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.483+0000 7f4035a1d740 10 client.11805 fetch_fsmap finished waiting for FSMap version 291 2024-09-21T07:53:20.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.483+0000 7f4035a1d740 10 client.11805 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:53:20.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.484+0000 7f4035a1d740 20 client.11805 populate_metadata read hostname 'smithi039' 2024-09-21T07:53:20.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.484+0000 7f4035a1d740 1 --2- 172.21.15.39:0/2001360922 >> [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] conn(0x55ae4b5f2e40 0x55ae4b9fff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.484+0000 7f4035a1d740 4 client.11805 mds_command: new command op to 11052 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:53:20.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.484+0000 7f4035a1d740 1 -- 172.21.15.39:0/2001360922 --> [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae4b950520 con 0x55ae4b5f2e40 2024-09-21T07:53:20.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.484+0000 7f4027400640 1 --2- 172.21.15.39:0/2001360922 >> [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] conn(0x55ae4b5f2e40 0x55ae4b9fff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.484+0000 7f4027400640 1 --2- 172.21.15.39:0/2001360922 >> [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] conn(0x55ae4b5f2e40 0x55ae4b9fff30 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d9ee18fad59f1996 server_cookie=e7d12319876df1cb in_seq=0 out_seq=0 2024-09-21T07:53:20.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.484+0000 7f4014c00640 10 client.11805 ms_handle_connect on v2:172.21.15.39:6833/485194245 2024-09-21T07:53:20.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.485+0000 7f4014c00640 1 -- 172.21.15.39:0/2001360922 <== mds.1 v2:172.21.15.39:6833/485194245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ae4b950520 con 0x55ae4b5f2e40 2024-09-21T07:53:20.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.485+0000 7f4014c00640 10 client.11805 handle_command_reply: tid=0 2024-09-21T07:53:20.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.485+0000 7f4035a1d740 1 client.11805 shutdown 2024-09-21T07:53:20.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.485+0000 7f4035a1d740 1 -- 172.21.15.39:0/2001360922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4b634270 msgr2=0x55ae4ba31ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:20.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.485+0000 7f4035a1d740 1 --2- 172.21.15.39:0/2001360922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4b634270 0x55ae4ba31ed0 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 crypto rx=0x7f4020017040 tx=0x7f402000ac60 comp rx=0 tx=0).stop 2024-09-21T07:53:20.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.485+0000 7f4035a1d740 1 -- 172.21.15.39:0/2001360922 shutdown_connections 2024-09-21T07:53:20.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.485+0000 7f4035a1d740 1 --2- 172.21.15.39:0/2001360922 >> [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] conn(0x55ae4b5f2e40 0x55ae4b9fff30 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.485+0000 7f4035a1d740 1 --2- 172.21.15.39:0/2001360922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4b634270 0x55ae4ba31ed0 unknown :-1 s=CLOSED pgs=2614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.485+0000 7f4035a1d740 1 --2- 172.21.15.39:0/2001360922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae4b613590 0x55ae4b9fafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.485+0000 7f4035a1d740 1 --2- 172.21.15.39:0/2001360922 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae4b60b3b0 0x55ae4ba20010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.485+0000 7f4035a1d740 1 -- 172.21.15.39:0/2001360922 >> 172.21.15.39:0/2001360922 conn(0x55ae4b6283c0 msgr2=0x55ae4ba22d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:20.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.485+0000 7f4014c00640 0 client.11805 ms_handle_reset on v2:172.21.15.39:6833/485194245 2024-09-21T07:53:20.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.486+0000 7f4035a1d740 1 -- 172.21.15.39:0/2001360922 shutdown_connections 2024-09-21T07:53:20.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.486+0000 7f4035a1d740 1 -- 172.21.15.39:0/2001360922 wait complete. 2024-09-21T07:53:20.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.486+0000 7f4035a1d740 20 client.11805 trim_cache size 0 max 16384 2024-09-21T07:53:20.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.505+0000 7f4035a1d740 1 Processor -- start 2024-09-21T07:53:20.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.505+0000 7f4035a1d740 1 -- start start 2024-09-21T07:53:20.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.507+0000 7f4035a1d740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae4b60b3b0 0x55ae4ba1af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.507+0000 7f4035a1d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae4b613590 0x55ae4ba1b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.507+0000 7f4035a1d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4b634270 0x55ae4ba37e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.507+0000 7f4035a1d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4028204840 con 0x55ae4b634270 2024-09-21T07:53:20.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.507+0000 7f4035a1d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4028204f10 con 0x55ae4b613590 2024-09-21T07:53:20.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.507+0000 7f4035a1d740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f4028203830 con 0x55ae4b60b3b0 2024-09-21T07:53:20.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.507+0000 7f4027e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4b634270 0x55ae4ba37e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.507+0000 7f4027e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4b634270 0x55ae4ba37e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45246/0 (socket says 172.21.15.39:45246) 2024-09-21T07:53:20.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.507+0000 7f4027e00640 1 -- 172.21.15.39:0/3785427050 learned_addr learned my addr 172.21.15.39:0/3785427050 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:20.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.507+0000 7f4027e00640 1 -- 172.21.15.39:0/3785427050 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae4b60b3b0 msgr2=0x55ae4ba1af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:53:20.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.507+0000 7f4027e00640 1 --2- 172.21.15.39:0/3785427050 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae4b60b3b0 0x55ae4ba1af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.507+0000 7f4027e00640 1 -- 172.21.15.39:0/3785427050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae4b613590 msgr2=0x55ae4ba1b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:53:20.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.507+0000 7f4027e00640 1 --2- 172.21.15.39:0/3785427050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae4b613590 0x55ae4ba1b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.507+0000 7f4027e00640 1 -- 172.21.15.39:0/3785427050 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae4b9da0a0 con 0x55ae4b634270 2024-09-21T07:53:20.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.508+0000 7f4027e00640 1 --2- 172.21.15.39:0/3785427050 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4b634270 0x55ae4ba37e60 secure :-1 s=READY pgs=2615 cs=0 l=1 rev1=1 crypto rx=0x7f402000a290 tx=0x7f402000a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c22cd00a0f6c988e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:20.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.508+0000 7f4017400640 1 -- 172.21.15.39:0/3785427050 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4020024070 con 0x55ae4b634270 2024-09-21T07:53:20.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.508+0000 7f4017400640 1 -- 172.21.15.39:0/3785427050 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4020020470 con 0x55ae4b634270 2024-09-21T07:53:20.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.508+0000 7f4017400640 1 -- 172.21.15.39:0/3785427050 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40200206c0 con 0x55ae4b634270 2024-09-21T07:53:20.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.509+0000 7f4035a1d740 1 -- 172.21.15.39:0/3785427050 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4b634270 msgr2=0x55ae4ba37e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:20.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.509+0000 7f4035a1d740 1 --2- 172.21.15.39:0/3785427050 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4b634270 0x55ae4ba37e60 secure :-1 s=READY pgs=2615 cs=0 l=1 rev1=1 crypto rx=0x7f402000a290 tx=0x7f402000a2c0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.509+0000 7f4035a1d740 1 -- 172.21.15.39:0/3785427050 shutdown_connections 2024-09-21T07:53:20.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.509+0000 7f4035a1d740 1 --2- 172.21.15.39:0/3785427050 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4b634270 0x55ae4ba37e60 unknown :-1 s=CLOSED pgs=2615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.509+0000 7f4035a1d740 1 --2- 172.21.15.39:0/3785427050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae4b613590 0x55ae4ba1b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.509+0000 7f4035a1d740 1 --2- 172.21.15.39:0/3785427050 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae4b60b3b0 0x55ae4ba1af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.509+0000 7f4035a1d740 1 -- 172.21.15.39:0/3785427050 >> 172.21.15.39:0/3785427050 conn(0x55ae4b6283c0 msgr2=0x55ae4ba19570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:20.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.509+0000 7f4035a1d740 1 -- 172.21.15.39:0/3785427050 shutdown_connections 2024-09-21T07:53:20.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.509+0000 7f4035a1d740 1 -- 172.21.15.39:0/3785427050 wait complete. 2024-09-21T07:53:20.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.510+0000 7f4035a1d740 1 -- start start 2024-09-21T07:53:20.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.510+0000 7f4035a1d740 1 Processor -- start 2024-09-21T07:53:20.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.511+0000 7f4035a1d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae4b60b3b0 0x55ae4baa2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.511+0000 7f4035a1d740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae4b613590 0x55ae4ba9c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.511+0000 7f4035a1d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4b634270 0x55ae4ba1ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.511+0000 7f4035a1d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ae4ba1a330 con 0x55ae4b634270 2024-09-21T07:53:20.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.511+0000 7f4035a1d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ae4ba00f60 con 0x55ae4b60b3b0 2024-09-21T07:53:20.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.511+0000 7f4035a1d740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55ae4b9fded0 con 0x55ae4b613590 2024-09-21T07:53:20.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.511+0000 7f4035a1d740 10 client.0 _ll_register_callbacks cb 0x7f4020009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:53:20.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.512+0000 7f4027e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4b634270 0x55ae4ba1ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.512+0000 7f4027e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4b634270 0x55ae4ba1ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45254/0 (socket says 172.21.15.39:45254) 2024-09-21T07:53:20.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.512+0000 7f4027e00640 1 -- 172.21.15.39:0/1538031182 learned_addr learned my addr 172.21.15.39:0/1538031182 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:20.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.512+0000 7f401fe00640 1 --2- 172.21.15.39:0/1538031182 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae4b613590 0x55ae4ba9c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.512+0000 7f4027400640 1 --2- 172.21.15.39:0/1538031182 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae4b60b3b0 0x55ae4baa2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.512+0000 7f4027e00640 1 -- 172.21.15.39:0/1538031182 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae4b613590 msgr2=0x55ae4ba9c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:53:20.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.512+0000 7f4027e00640 1 --2- 172.21.15.39:0/1538031182 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae4b613590 0x55ae4ba9c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.512+0000 7f4027e00640 1 -- 172.21.15.39:0/1538031182 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae4b60b3b0 msgr2=0x55ae4baa2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:53:20.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.512+0000 7f4027e00640 1 --2- 172.21.15.39:0/1538031182 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae4b60b3b0 0x55ae4baa2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.512+0000 7f4027e00640 1 -- 172.21.15.39:0/1538031182 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae4b9d3fb0 con 0x55ae4b634270 2024-09-21T07:53:20.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.512+0000 7f4027e00640 1 --2- 172.21.15.39:0/1538031182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4b634270 0x55ae4ba1ae80 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7f4020017040 tx=0x7f4020020ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b0d85349d14b8da server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:20.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.512+0000 7f4014c00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:53:20.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.513+0000 7f4014c00640 1 -- 172.21.15.39:0/1538031182 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402003d060 con 0x55ae4b634270 2024-09-21T07:53:20.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.513+0000 7f4014c00640 1 -- 172.21.15.39:0/1538031182 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(250..250 src has 1..250) ==== 7820+0+0 (secure 0 0 0) 0x7f402002e670 con 0x55ae4b634270 2024-09-21T07:53:20.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.513+0000 7f4035a1d740 1 -- 172.21.15.39:0/1538031182 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4020020470 con 0x55ae4b634270 2024-09-21T07:53:20.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.513+0000 7f4014c00640 1 -- 172.21.15.39:0/1538031182 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=291) ==== 24+0+0 (secure 0 0 0) 0x7f4020024040 con 0x55ae4b634270 2024-09-21T07:53:20.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.513+0000 7f4035a1d740 10 client.11811 fetch_fsmap learned FSMap version 291 2024-09-21T07:53:20.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.513+0000 7f4035a1d740 1 -- 172.21.15.39:0/1538031182 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=291,monmap=2+}) -- 0x55ae4b9d54c0 con 0x55ae4b634270 2024-09-21T07:53:20.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.513+0000 7f4014c00640 1 -- 172.21.15.39:0/1538031182 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4020020470 con 0x55ae4b634270 2024-09-21T07:53:20.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.513+0000 7f4014c00640 1 -- 172.21.15.39:0/1538031182 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 291) ==== 3542+0+0 (secure 0 0 0) 0x7f402002c040 con 0x55ae4b634270 2024-09-21T07:53:20.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.513+0000 7f4035a1d740 10 client.11811 fetch_fsmap finished waiting for FSMap version 291 2024-09-21T07:53:20.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.513+0000 7f4035a1d740 10 client.11811 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:53:20.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.513+0000 7f4035a1d740 20 client.11811 populate_metadata read hostname 'smithi039' 2024-09-21T07:53:20.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.513+0000 7f4035a1d740 1 --2- 172.21.15.39:0/1538031182 >> [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] conn(0x55ae4ba9f9b0 0x55ae4b9fc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.513+0000 7f4035a1d740 4 client.11811 mds_command: new command op to 11052 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:53:20.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.513+0000 7f4035a1d740 1 -- 172.21.15.39:0/1538031182 --> [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] -- command(tid 0: {"prefix": "status"}) -- 0x55ae4b94bfd0 con 0x55ae4ba9f9b0 2024-09-21T07:53:20.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.514+0000 7f4027400640 1 --2- 172.21.15.39:0/1538031182 >> [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] conn(0x55ae4ba9f9b0 0x55ae4b9fc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.514+0000 7f4014c00640 10 client.11811 ms_handle_connect on v2:172.21.15.39:6833/485194245 2024-09-21T07:53:20.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.514+0000 7f4027400640 1 --2- 172.21.15.39:0/1538031182 >> [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] conn(0x55ae4ba9f9b0 0x55ae4b9fc8a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b74a7c6d193687b5 server_cookie=2e3219a87d9d56e4 in_seq=0 out_seq=0 2024-09-21T07:53:20.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.515+0000 7f4014c00640 1 -- 172.21.15.39:0/1538031182 <== mds.1 v2:172.21.15.39:6833/485194245 1 ==== command_reply(tid 0: 0 ) ==== 8+0+335 (crc 0 0 0) 0x55ae4b94bfd0 con 0x55ae4ba9f9b0 2024-09-21T07:53:20.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.515+0000 7f4014c00640 10 client.11811 handle_command_reply: tid=0 2024-09-21T07:53:20.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.515+0000 7f4035a1d740 1 client.11811 shutdown 2024-09-21T07:53:20.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.515+0000 7f4035a1d740 1 -- 172.21.15.39:0/1538031182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4b634270 msgr2=0x55ae4ba1ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:20.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.515+0000 7f4035a1d740 1 --2- 172.21.15.39:0/1538031182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4b634270 0x55ae4ba1ae80 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7f4020017040 tx=0x7f4020020ce0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.515+0000 7f4035a1d740 1 -- 172.21.15.39:0/1538031182 shutdown_connections 2024-09-21T07:53:20.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.515+0000 7f4035a1d740 1 --2- 172.21.15.39:0/1538031182 >> [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] conn(0x55ae4ba9f9b0 0x55ae4b9fc8a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.515+0000 7f4035a1d740 1 --2- 172.21.15.39:0/1538031182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae4b634270 0x55ae4ba1ae80 unknown :-1 s=CLOSED pgs=2616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.515+0000 7f4035a1d740 1 --2- 172.21.15.39:0/1538031182 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55ae4b613590 0x55ae4ba9c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.515+0000 7f4014c00640 0 client.11811 ms_handle_reset on v2:172.21.15.39:6833/485194245 2024-09-21T07:53:20.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.515+0000 7f4035a1d740 1 --2- 172.21.15.39:0/1538031182 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ae4b60b3b0 0x55ae4baa2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.515+0000 7f4035a1d740 1 -- 172.21.15.39:0/1538031182 >> 172.21.15.39:0/1538031182 conn(0x55ae4b6283c0 msgr2=0x55ae4ba425a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:20.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.515+0000 7f4035a1d740 1 -- 172.21.15.39:0/1538031182 shutdown_connections 2024-09-21T07:53:20.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.515+0000 7f4035a1d740 1 -- 172.21.15.39:0/1538031182 wait complete. 2024-09-21T07:53:20.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.515+0000 7f4035a1d740 20 client.11811 trim_cache size 0 max 16384 2024-09-21T07:53:20.520 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:53:20.520 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:53:20.520 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 1, 2024-09-21T07:53:20.520 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 11052, 2024-09-21T07:53:20.520 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:53:20.520 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:53:20.521 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:53:20.521 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 168.12990186499999, 2024-09-21T07:53:20.521 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 291, 2024-09-21T07:53:20.521 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 250, 2024-09-21T07:53:20.521 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 250, 2024-09-21T07:53:20.521 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 168.54217004200001 2024-09-21T07:53:20.521 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:53:20.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.516+0000 7f402e800640 1 -- 172.21.15.39:0/2038378424 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4004078b90 msgr2=0x7f400407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:20.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.516+0000 7f402e800640 1 --2- 172.21.15.39:0/2038378424 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4004078b90 0x7f400407b0a0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f4018005fd0 tx=0x7f401800a250 comp rx=0 tx=0).stop 2024-09-21T07:53:20.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.516+0000 7f402e800640 1 -- 172.21.15.39:0/2038378424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4028155a70 msgr2=0x7f4028151ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:20.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.516+0000 7f402e800640 1 --2- 172.21.15.39:0/2038378424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4028155a70 0x7f4028151ad0 secure :-1 s=READY pgs=2612 cs=0 l=1 rev1=1 crypto rx=0x7f4010034040 tx=0x7f40100039e0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.516+0000 7f402e800640 1 -- 172.21.15.39:0/2038378424 shutdown_connections 2024-09-21T07:53:20.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.516+0000 7f402e800640 1 --2- 172.21.15.39:0/2038378424 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4004078b90 0x7f400407b0a0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.516+0000 7f402e800640 1 --2- 172.21.15.39:0/2038378424 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f402815f360 0x7f402814e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.516+0000 7f402e800640 1 --2- 172.21.15.39:0/2038378424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4028155a70 0x7f4028151ad0 unknown :-1 s=CLOSED pgs=2612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.516+0000 7f402e800640 1 --2- 172.21.15.39:0/2038378424 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40280b3f30 0x7f4028156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.516+0000 7f402e800640 1 -- 172.21.15.39:0/2038378424 >> 172.21.15.39:0/2038378424 conn(0x7f402806cfc0 msgr2=0x7f40280cf080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:20.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.516+0000 7f402e800640 1 -- 172.21.15.39:0/2038378424 shutdown_connections 2024-09-21T07:53:20.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.516+0000 7f402e800640 1 -- 172.21.15.39:0/2038378424 wait complete. 2024-09-21T07:53:20.542 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T07:53:20.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.768+0000 7eff66e00640 1 -- 172.21.15.39:0/3012423409 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff6006d4f0 msgr2=0x7eff600b67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:20.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.768+0000 7eff66e00640 1 --2- 172.21.15.39:0/3012423409 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff6006d4f0 0x7eff600b67d0 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 crypto rx=0x7eff600755c0 tx=0x7eff4802a4d0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.768+0000 7eff66e00640 1 -- 172.21.15.39:0/3012423409 shutdown_connections 2024-09-21T07:53:20.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.768+0000 7eff66e00640 1 --2- 172.21.15.39:0/3012423409 >> v2:172.21.15.148:3300/0 conn(0x7eff6015c0c0 0x7eff6015c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.768+0000 7eff66e00640 1 --2- 172.21.15.39:0/3012423409 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff6006d4f0 0x7eff600b67d0 unknown :-1 s=CLOSED pgs=2617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.768+0000 7eff66e00640 1 -- 172.21.15.39:0/3012423409 >> 172.21.15.39:0/3012423409 conn(0x7eff600b3010 msgr2=0x7eff600b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:20.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.768+0000 7eff66e00640 1 -- 172.21.15.39:0/3012423409 shutdown_connections 2024-09-21T07:53:20.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.768+0000 7eff66e00640 1 -- 172.21.15.39:0/3012423409 wait complete. 2024-09-21T07:53:20.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.770+0000 7eff66e00640 1 Processor -- start 2024-09-21T07:53:20.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.770+0000 7eff66e00640 1 -- start start 2024-09-21T07:53:20.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.772+0000 7eff66e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff600cceb0 0x7eff601f54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.772+0000 7eff66e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7eff6015c0c0 0x7eff601f6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.772+0000 7eff66e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff600b4ee0 0x7eff600b52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.772+0000 7eff66e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7eff6015cda0 con 0x7eff600cceb0 2024-09-21T07:53:20.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.772+0000 7eff66e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7eff6015cc20 con 0x7eff600b4ee0 2024-09-21T07:53:20.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.772+0000 7eff66e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7eff600b3d30 con 0x7eff6015c0c0 2024-09-21T07:53:20.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.772+0000 7eff5fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff600cceb0 0x7eff601f54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.772+0000 7eff5fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff600cceb0 0x7eff601f54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45278/0 (socket says 172.21.15.39:45278) 2024-09-21T07:53:20.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.772+0000 7eff5fe00640 1 -- 172.21.15.39:0/1716501900 learned_addr learned my addr 172.21.15.39:0/1716501900 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:20.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.772+0000 7eff64e00640 1 --2- 172.21.15.39:0/1716501900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff600b4ee0 0x7eff600b52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.772+0000 7eff5fe00640 1 -- 172.21.15.39:0/1716501900 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7eff6015c0c0 msgr2=0x7eff601f6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:20.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.772+0000 7eff5fe00640 1 --2- 172.21.15.39:0/1716501900 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7eff6015c0c0 0x7eff601f6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.772+0000 7eff5fe00640 1 -- 172.21.15.39:0/1716501900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff600b4ee0 msgr2=0x7eff600b52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:20.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.772+0000 7eff5fe00640 1 --2- 172.21.15.39:0/1716501900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff600b4ee0 0x7eff600b52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.772+0000 7eff5fe00640 1 -- 172.21.15.39:0/1716501900 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff600b5830 con 0x7eff600cceb0 2024-09-21T07:53:20.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.773+0000 7eff64e00640 1 --2- 172.21.15.39:0/1716501900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff600b4ee0 0x7eff600b52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:53:20.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.773+0000 7eff5fe00640 1 --2- 172.21.15.39:0/1716501900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff600cceb0 0x7eff601f54d0 secure :-1 s=READY pgs=2618 cs=0 l=1 rev1=1 crypto rx=0x7eff5000b3e0 tx=0x7eff5000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:20.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.773+0000 7eff5cc00640 1 -- 172.21.15.39:0/1716501900 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff50015020 con 0x7eff600cceb0 2024-09-21T07:53:20.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.773+0000 7eff5cc00640 1 -- 172.21.15.39:0/1716501900 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff50011940 con 0x7eff600cceb0 2024-09-21T07:53:20.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.773+0000 7eff5cc00640 1 -- 172.21.15.39:0/1716501900 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff50011d90 con 0x7eff600cceb0 2024-09-21T07:53:20.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.773+0000 7eff66e00640 1 -- 172.21.15.39:0/1716501900 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff6015e4b0 con 0x7eff600cceb0 2024-09-21T07:53:20.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.773+0000 7eff66e00640 1 -- 172.21.15.39:0/1716501900 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff6015ea70 con 0x7eff600cceb0 2024-09-21T07:53:20.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.776+0000 7eff5cc00640 1 -- 172.21.15.39:0/1716501900 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7eff50018020 con 0x7eff600cceb0 2024-09-21T07:53:20.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.777+0000 7eff5cc00640 1 --2- 172.21.15.39:0/1716501900 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7eff3c078bc0 0x7eff3c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.778+0000 7eff5cc00640 1 -- 172.21.15.39:0/1716501900 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(250..250 src has 1..250) ==== 7820+0+0 (secure 0 0 0) 0x7eff5009b570 con 0x7eff600cceb0 2024-09-21T07:53:20.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.778+0000 7eff6e007740 1 Processor -- start 2024-09-21T07:53:20.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.778+0000 7eff6e007740 1 -- start start 2024-09-21T07:53:20.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.778+0000 7eff57e00640 1 --2- 172.21.15.39:0/1716501900 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7eff3c078bc0 0x7eff3c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.779+0000 7eff57e00640 1 --2- 172.21.15.39:0/1716501900 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7eff3c078bc0 0x7eff3c07b0d0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7eff48033000 tx=0x7eff480046c0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:20.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.780+0000 7eff6e007740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56287daa6270 0x56287de76970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.780+0000 7eff6e007740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56287da7d3b0 0x56287de76eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.780+0000 7eff6e007740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56287da85590 0x56287de46f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.780+0000 7eff6e007740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56287ddd0430 con 0x56287da7d3b0 2024-09-21T07:53:20.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.780+0000 7eff6e007740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56287ddc05d0 con 0x56287daa6270 2024-09-21T07:53:20.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.780+0000 7eff6e007740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x56287ddafa00 con 0x56287da85590 2024-09-21T07:53:20.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.780+0000 7eff57e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56287da7d3b0 0x56287de76eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.780+0000 7eff57e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56287da7d3b0 0x56287de76eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45292/0 (socket says 172.21.15.39:45292) 2024-09-21T07:53:20.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.780+0000 7eff57e00640 1 -- 172.21.15.39:0/3701730247 learned_addr learned my addr 172.21.15.39:0/3701730247 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:20.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.780+0000 7eff5fe00640 1 --2- 172.21.15.39:0/3701730247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56287daa6270 0x56287de76970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.780+0000 7eff64e00640 1 --2- 172.21.15.39:0/3701730247 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56287da85590 0x56287de46f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.780+0000 7eff57e00640 1 -- 172.21.15.39:0/3701730247 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56287da85590 msgr2=0x56287de46f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:53:20.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.780+0000 7eff57e00640 1 --2- 172.21.15.39:0/3701730247 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56287da85590 0x56287de46f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.780+0000 7eff57e00640 1 -- 172.21.15.39:0/3701730247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56287daa6270 msgr2=0x56287de76970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:53:20.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.780+0000 7eff57e00640 1 --2- 172.21.15.39:0/3701730247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56287daa6270 0x56287de76970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.780+0000 7eff57e00640 1 -- 172.21.15.39:0/3701730247 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56287dd78520 con 0x56287da7d3b0 2024-09-21T07:53:20.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.780+0000 7eff64e00640 1 --2- 172.21.15.39:0/3701730247 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56287da85590 0x56287de46f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:53:20.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.781+0000 7eff57e00640 1 --2- 172.21.15.39:0/3701730247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56287da7d3b0 0x56287de76eb0 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x7eff4802aac0 tx=0x7eff4803a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=600c8a8dae88202b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:20.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.781+0000 7eff54c00640 1 -- 172.21.15.39:0/3701730247 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff48035040 con 0x56287da7d3b0 2024-09-21T07:53:20.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.781+0000 7eff54c00640 1 -- 172.21.15.39:0/3701730247 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff4804b070 con 0x56287da7d3b0 2024-09-21T07:53:20.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.781+0000 7eff54c00640 1 -- 172.21.15.39:0/3701730247 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff48003960 con 0x56287da7d3b0 2024-09-21T07:53:20.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.782+0000 7eff6e007740 1 -- 172.21.15.39:0/3701730247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56287da7d3b0 msgr2=0x56287de76eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:20.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.782+0000 7eff6e007740 1 --2- 172.21.15.39:0/3701730247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56287da7d3b0 0x56287de76eb0 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x7eff4802aac0 tx=0x7eff4803a000 comp rx=0 tx=0).stop 2024-09-21T07:53:20.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.782+0000 7eff6e007740 1 -- 172.21.15.39:0/3701730247 shutdown_connections 2024-09-21T07:53:20.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.782+0000 7eff6e007740 1 --2- 172.21.15.39:0/3701730247 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56287da85590 0x56287de46f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.782+0000 7eff6e007740 1 --2- 172.21.15.39:0/3701730247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56287da7d3b0 0x56287de76eb0 unknown :-1 s=CLOSED pgs=2619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.782+0000 7eff6e007740 1 --2- 172.21.15.39:0/3701730247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56287daa6270 0x56287de76970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.782+0000 7eff6e007740 1 -- 172.21.15.39:0/3701730247 >> 172.21.15.39:0/3701730247 conn(0x56287da9a3c0 msgr2=0x56287de4c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:20.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.782+0000 7eff6e007740 1 -- 172.21.15.39:0/3701730247 shutdown_connections 2024-09-21T07:53:20.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.782+0000 7eff6e007740 1 -- 172.21.15.39:0/3701730247 wait complete. 2024-09-21T07:53:20.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.783+0000 7eff6e007740 1 -- start start 2024-09-21T07:53:20.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.783+0000 7eff6e007740 1 Processor -- start 2024-09-21T07:53:20.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.785+0000 7eff6e007740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56287da7d3b0 0x56287de6f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.785+0000 7eff6e007740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56287da85590 0x56287de6cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.785+0000 7eff6e007740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56287daa6270 0x56287de47c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.785+0000 7eff6e007740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56287dd56330 con 0x56287da7d3b0 2024-09-21T07:53:20.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.785+0000 7eff6e007740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56287ddb3810 con 0x56287daa6270 2024-09-21T07:53:20.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.785+0000 7eff6e007740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x56287db20620 con 0x56287da85590 2024-09-21T07:53:20.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.785+0000 7eff6e007740 10 client.0 _ll_register_callbacks cb 0x56287dd5b420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:53:20.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.785+0000 7eff5fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56287da7d3b0 0x56287de6f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.785+0000 7eff5fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56287da7d3b0 0x56287de6f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45304/0 (socket says 172.21.15.39:45304) 2024-09-21T07:53:20.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.785+0000 7eff5fe00640 1 -- 172.21.15.39:0/2974394616 learned_addr learned my addr 172.21.15.39:0/2974394616 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:20.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.785+0000 7eff57e00640 1 --2- 172.21.15.39:0/2974394616 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56287da85590 0x56287de6cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.785+0000 7eff64e00640 1 --2- 172.21.15.39:0/2974394616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56287daa6270 0x56287de47c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.785+0000 7eff5fe00640 1 -- 172.21.15.39:0/2974394616 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56287da85590 msgr2=0x56287de6cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:53:20.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.785+0000 7eff5fe00640 1 --2- 172.21.15.39:0/2974394616 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56287da85590 0x56287de6cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.785+0000 7eff5fe00640 1 -- 172.21.15.39:0/2974394616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56287daa6270 msgr2=0x56287de47c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:53:20.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.785+0000 7eff5fe00640 1 --2- 172.21.15.39:0/2974394616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56287daa6270 0x56287de47c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.785+0000 7eff5fe00640 1 -- 172.21.15.39:0/2974394616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56287db11850 con 0x56287da7d3b0 2024-09-21T07:53:20.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.785+0000 7eff64e00640 1 --2- 172.21.15.39:0/2974394616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56287daa6270 0x56287de47c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:53:20.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.786+0000 7eff5fe00640 1 --2- 172.21.15.39:0/2974394616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56287da7d3b0 0x56287de6f9b0 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 crypto rx=0x7eff5005fc40 tx=0x7eff500624e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cebb7b1db237e5c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:20.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.786+0000 7eff4d600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:53:20.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.786+0000 7eff4d600640 1 -- 172.21.15.39:0/2974394616 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff500674b0 con 0x56287da7d3b0 2024-09-21T07:53:20.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.786+0000 7eff4d600640 1 -- 172.21.15.39:0/2974394616 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(250..250 src has 1..250) ==== 7820+0+0 (secure 0 0 0) 0x7eff5009b0b0 con 0x56287da7d3b0 2024-09-21T07:53:20.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.786+0000 7eff6e007740 1 -- 172.21.15.39:0/2974394616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff4804b070 con 0x56287da7d3b0 2024-09-21T07:53:20.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.786+0000 7eff4d600640 1 -- 172.21.15.39:0/2974394616 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=291) ==== 24+0+0 (secure 0 0 0) 0x7eff50083050 con 0x56287da7d3b0 2024-09-21T07:53:20.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.786+0000 7eff6e007740 10 client.11823 fetch_fsmap learned FSMap version 291 2024-09-21T07:53:20.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.786+0000 7eff6e007740 1 -- 172.21.15.39:0/2974394616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=291,monmap=2+}) -- 0x56287db886b0 con 0x56287da7d3b0 2024-09-21T07:53:20.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.786+0000 7eff4d600640 1 -- 172.21.15.39:0/2974394616 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff4804b070 con 0x56287da7d3b0 2024-09-21T07:53:20.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.786+0000 7eff4d600640 1 -- 172.21.15.39:0/2974394616 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 291) ==== 3542+0+0 (secure 0 0 0) 0x7eff50078b80 con 0x56287da7d3b0 2024-09-21T07:53:20.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.786+0000 7eff6e007740 10 client.11823 fetch_fsmap finished waiting for FSMap version 291 2024-09-21T07:53:20.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.786+0000 7eff6e007740 10 client.11823 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:53:20.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.786+0000 7eff6e007740 20 client.11823 populate_metadata read hostname 'smithi039' 2024-09-21T07:53:20.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.786+0000 7eff6e007740 1 --2- 172.21.15.39:0/2974394616 >> [v2:172.21.15.39:6832/2161818466,v1:172.21.15.39:6834/2161818466] conn(0x56287da64e40 0x56287de898e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.786+0000 7eff6e007740 4 client.11823 mds_command: new command op to 11364 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:53:20.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.786+0000 7eff6e007740 1 -- 172.21.15.39:0/2974394616 --> [v2:172.21.15.39:6832/2161818466,v1:172.21.15.39:6834/2161818466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56287ddc2520 con 0x56287da64e40 2024-09-21T07:53:20.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.787+0000 7eff57e00640 1 --2- 172.21.15.39:0/2974394616 >> [v2:172.21.15.39:6832/2161818466,v1:172.21.15.39:6834/2161818466] conn(0x56287da64e40 0x56287de898e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.787+0000 7eff57e00640 1 --2- 172.21.15.39:0/2974394616 >> [v2:172.21.15.39:6832/2161818466,v1:172.21.15.39:6834/2161818466] conn(0x56287da64e40 0x56287de898e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad3d6f9903e04c1e server_cookie=1b53c5c7c46ce17f in_seq=0 out_seq=0 2024-09-21T07:53:20.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.787+0000 7eff4d600640 10 client.11823 ms_handle_connect on v2:172.21.15.39:6832/2161818466 2024-09-21T07:53:20.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.788+0000 7eff4d600640 1 -- 172.21.15.39:0/2974394616 <== mds.0 v2:172.21.15.39:6832/2161818466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56287ddc2520 con 0x56287da64e40 2024-09-21T07:53:20.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.788+0000 7eff4d600640 10 client.11823 handle_command_reply: tid=0 2024-09-21T07:53:20.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.788+0000 7eff6e007740 1 client.11823 shutdown 2024-09-21T07:53:20.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.789+0000 7eff6e007740 1 -- 172.21.15.39:0/2974394616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56287da7d3b0 msgr2=0x56287de6f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:20.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.789+0000 7eff6e007740 1 --2- 172.21.15.39:0/2974394616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56287da7d3b0 0x56287de6f9b0 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 crypto rx=0x7eff5005fc40 tx=0x7eff500624e0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.789+0000 7eff6e007740 1 -- 172.21.15.39:0/2974394616 shutdown_connections 2024-09-21T07:53:20.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.789+0000 7eff6e007740 1 --2- 172.21.15.39:0/2974394616 >> [v2:172.21.15.39:6832/2161818466,v1:172.21.15.39:6834/2161818466] conn(0x56287da64e40 0x56287de898e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.789+0000 7eff6e007740 1 --2- 172.21.15.39:0/2974394616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56287daa6270 0x56287de47c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.789+0000 7eff6e007740 1 --2- 172.21.15.39:0/2974394616 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56287da85590 0x56287de6cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.789+0000 7eff6e007740 1 --2- 172.21.15.39:0/2974394616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56287da7d3b0 0x56287de6f9b0 unknown :-1 s=CLOSED pgs=2620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.789+0000 7eff4d600640 0 client.11823 ms_handle_reset on v2:172.21.15.39:6832/2161818466 2024-09-21T07:53:20.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.789+0000 7eff6e007740 1 -- 172.21.15.39:0/2974394616 >> 172.21.15.39:0/2974394616 conn(0x56287da9a3c0 msgr2=0x56287de87f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:20.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.789+0000 7eff6e007740 1 -- 172.21.15.39:0/2974394616 shutdown_connections 2024-09-21T07:53:20.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.789+0000 7eff6e007740 1 -- 172.21.15.39:0/2974394616 wait complete. 2024-09-21T07:53:20.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.789+0000 7eff6e007740 20 client.11823 trim_cache size 0 max 16384 2024-09-21T07:53:20.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.808+0000 7eff6e007740 1 Processor -- start 2024-09-21T07:53:20.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.809+0000 7eff6e007740 1 -- start start 2024-09-21T07:53:20.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.810+0000 7eff6e007740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56287da7d3b0 0x56287de925e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.810+0000 7eff6e007740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56287da85590 0x56287dea30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.810+0000 7eff6e007740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56287daa6270 0x56287de93670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.810+0000 7eff6e007740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7eff602046e0 con 0x56287da85590 2024-09-21T07:53:20.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.810+0000 7eff6e007740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7eff60204db0 con 0x56287da7d3b0 2024-09-21T07:53:20.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.810+0000 7eff6e007740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7eff602036f0 con 0x56287daa6270 2024-09-21T07:53:20.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.810+0000 7eff57e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56287da85590 0x56287dea30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.810+0000 7eff57e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56287da85590 0x56287dea30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45316/0 (socket says 172.21.15.39:45316) 2024-09-21T07:53:20.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.810+0000 7eff57e00640 1 -- 172.21.15.39:0/1769037545 learned_addr learned my addr 172.21.15.39:0/1769037545 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:20.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.810+0000 7eff64e00640 1 --2- 172.21.15.39:0/1769037545 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56287daa6270 0x56287de93670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.810+0000 7eff5fe00640 1 --2- 172.21.15.39:0/1769037545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56287da7d3b0 0x56287de925e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.811+0000 7eff57e00640 1 -- 172.21.15.39:0/1769037545 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56287daa6270 msgr2=0x56287de93670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:53:20.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.811+0000 7eff57e00640 1 --2- 172.21.15.39:0/1769037545 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56287daa6270 0x56287de93670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.811+0000 7eff57e00640 1 -- 172.21.15.39:0/1769037545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56287da7d3b0 msgr2=0x56287de925e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:53:20.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.811+0000 7eff57e00640 1 --2- 172.21.15.39:0/1769037545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56287da7d3b0 0x56287de925e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.811+0000 7eff57e00640 1 -- 172.21.15.39:0/1769037545 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56287de4d650 con 0x56287da85590 2024-09-21T07:53:20.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.811+0000 7eff5fe00640 1 --2- 172.21.15.39:0/1769037545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56287da7d3b0 0x56287de925e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:53:20.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.811+0000 7eff57e00640 1 --2- 172.21.15.39:0/1769037545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56287da85590 0x56287dea30d0 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 crypto rx=0x7eff48009850 tx=0x7eff48004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ed872f2cd8c897 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:20.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.811+0000 7eff4fe00640 1 -- 172.21.15.39:0/1769037545 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff48035040 con 0x56287da85590 2024-09-21T07:53:20.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.811+0000 7eff4fe00640 1 -- 172.21.15.39:0/1769037545 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff4804b070 con 0x56287da85590 2024-09-21T07:53:20.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.811+0000 7eff4fe00640 1 -- 172.21.15.39:0/1769037545 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff480047a0 con 0x56287da85590 2024-09-21T07:53:20.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.812+0000 7eff6e007740 1 -- 172.21.15.39:0/1769037545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56287da85590 msgr2=0x56287dea30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:20.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.812+0000 7eff6e007740 1 --2- 172.21.15.39:0/1769037545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56287da85590 0x56287dea30d0 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 crypto rx=0x7eff48009850 tx=0x7eff48004730 comp rx=0 tx=0).stop 2024-09-21T07:53:20.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.812+0000 7eff6e007740 1 -- 172.21.15.39:0/1769037545 shutdown_connections 2024-09-21T07:53:20.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.812+0000 7eff6e007740 1 --2- 172.21.15.39:0/1769037545 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56287daa6270 0x56287de93670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.812+0000 7eff6e007740 1 --2- 172.21.15.39:0/1769037545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56287da85590 0x56287dea30d0 unknown :-1 s=CLOSED pgs=2621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.812+0000 7eff6e007740 1 --2- 172.21.15.39:0/1769037545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56287da7d3b0 0x56287de925e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.812+0000 7eff6e007740 1 -- 172.21.15.39:0/1769037545 >> 172.21.15.39:0/1769037545 conn(0x56287da9a3c0 msgr2=0x56287de75b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:20.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.812+0000 7eff6e007740 1 -- 172.21.15.39:0/1769037545 shutdown_connections 2024-09-21T07:53:20.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.812+0000 7eff6e007740 1 -- 172.21.15.39:0/1769037545 wait complete. 2024-09-21T07:53:20.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.813+0000 7eff6e007740 1 -- start start 2024-09-21T07:53:20.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.813+0000 7eff6e007740 1 Processor -- start 2024-09-21T07:53:20.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.815+0000 7eff6e007740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56287da7d3b0 0x56287df14860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.815+0000 7eff6e007740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56287da85590 0x56287de45c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.815+0000 7eff6e007740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56287daa6270 0x56287de74df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.815+0000 7eff6e007740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56287dea42b0 con 0x56287daa6270 2024-09-21T07:53:20.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.815+0000 7eff6e007740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56287de6ecd0 con 0x56287da7d3b0 2024-09-21T07:53:20.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.815+0000 7eff6e007740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x56287de70560 con 0x56287da85590 2024-09-21T07:53:20.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.815+0000 7eff6e007740 10 client.0 _ll_register_callbacks cb 0x7eff500623b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:53:20.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.815+0000 7eff64e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56287daa6270 0x56287de74df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.815+0000 7eff64e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56287daa6270 0x56287de74df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45324/0 (socket says 172.21.15.39:45324) 2024-09-21T07:53:20.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.815+0000 7eff64e00640 1 -- 172.21.15.39:0/3551132543 learned_addr learned my addr 172.21.15.39:0/3551132543 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:20.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.815+0000 7eff5fe00640 1 --2- 172.21.15.39:0/3551132543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56287da7d3b0 0x56287df14860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.815+0000 7eff57e00640 1 --2- 172.21.15.39:0/3551132543 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56287da85590 0x56287de45c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.815+0000 7eff64e00640 1 -- 172.21.15.39:0/3551132543 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56287da85590 msgr2=0x56287de45c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:53:20.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.815+0000 7eff64e00640 1 --2- 172.21.15.39:0/3551132543 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56287da85590 0x56287de45c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.815+0000 7eff64e00640 1 -- 172.21.15.39:0/3551132543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56287da7d3b0 msgr2=0x56287df14860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:53:20.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.815+0000 7eff64e00640 1 --2- 172.21.15.39:0/3551132543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56287da7d3b0 0x56287df14860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.815+0000 7eff64e00640 1 -- 172.21.15.39:0/3551132543 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56287df15160 con 0x56287daa6270 2024-09-21T07:53:20.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.816+0000 7eff64e00640 1 --2- 172.21.15.39:0/3551132543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56287daa6270 0x56287de74df0 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7eff58012950 tx=0x7eff58012e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ae0e83abc13937 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:20.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.816+0000 7eff4d600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:53:20.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.816+0000 7eff4d600640 1 -- 172.21.15.39:0/3551132543 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff58010b20 con 0x56287daa6270 2024-09-21T07:53:20.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.816+0000 7eff4d600640 1 -- 172.21.15.39:0/3551132543 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(250..250 src has 1..250) ==== 7820+0+0 (secure 0 0 0) 0x7eff58010cc0 con 0x56287daa6270 2024-09-21T07:53:20.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.816+0000 7eff6e007740 1 -- 172.21.15.39:0/3551132543 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff4804b070 con 0x56287daa6270 2024-09-21T07:53:20.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.816+0000 7eff4d600640 1 -- 172.21.15.39:0/3551132543 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=291) ==== 24+0+0 (secure 0 0 0) 0x7eff5801b3f0 con 0x56287daa6270 2024-09-21T07:53:20.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.816+0000 7eff6e007740 10 client.11829 fetch_fsmap learned FSMap version 291 2024-09-21T07:53:20.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.816+0000 7eff6e007740 1 -- 172.21.15.39:0/3551132543 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=291,monmap=2+}) -- 0x56287de46450 con 0x56287daa6270 2024-09-21T07:53:20.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.816+0000 7eff4d600640 1 -- 172.21.15.39:0/3551132543 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff4804b070 con 0x56287daa6270 2024-09-21T07:53:20.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.817+0000 7eff4d600640 1 -- 172.21.15.39:0/3551132543 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 291) ==== 3542+0+0 (secure 0 0 0) 0x7eff58023030 con 0x56287daa6270 2024-09-21T07:53:20.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.817+0000 7eff6e007740 10 client.11829 fetch_fsmap finished waiting for FSMap version 291 2024-09-21T07:53:20.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.817+0000 7eff6e007740 10 client.11829 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:53:20.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.817+0000 7eff6e007740 20 client.11829 populate_metadata read hostname 'smithi039' 2024-09-21T07:53:20.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.817+0000 7eff6e007740 1 --2- 172.21.15.39:0/3551132543 >> [v2:172.21.15.39:6832/2161818466,v1:172.21.15.39:6834/2161818466] conn(0x56287de93d20 0x56287de6c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.817+0000 7eff6e007740 4 client.11829 mds_command: new command op to 11364 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:53:20.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.817+0000 7eff6e007740 1 -- 172.21.15.39:0/3551132543 --> [v2:172.21.15.39:6832/2161818466,v1:172.21.15.39:6834/2161818466] -- command(tid 0: {"prefix": "status"}) -- 0x56287ddbdfd0 con 0x56287de93d20 2024-09-21T07:53:20.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.817+0000 7eff5fe00640 1 --2- 172.21.15.39:0/3551132543 >> [v2:172.21.15.39:6832/2161818466,v1:172.21.15.39:6834/2161818466] conn(0x56287de93d20 0x56287de6c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.817+0000 7eff5fe00640 1 --2- 172.21.15.39:0/3551132543 >> [v2:172.21.15.39:6832/2161818466,v1:172.21.15.39:6834/2161818466] conn(0x56287de93d20 0x56287de6c700 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d97788690764c08 server_cookie=59dee7f40b224b50 in_seq=0 out_seq=0 2024-09-21T07:53:20.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.817+0000 7eff4d600640 10 client.11829 ms_handle_connect on v2:172.21.15.39:6832/2161818466 2024-09-21T07:53:20.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.818+0000 7eff4d600640 1 -- 172.21.15.39:0/3551132543 <== mds.0 v2:172.21.15.39:6832/2161818466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x56287ddbdfd0 con 0x56287de93d20 2024-09-21T07:53:20.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.818+0000 7eff4d600640 10 client.11829 handle_command_reply: tid=0 2024-09-21T07:53:20.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.818+0000 7eff6e007740 1 client.11829 shutdown 2024-09-21T07:53:20.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.818+0000 7eff6e007740 1 -- 172.21.15.39:0/3551132543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56287daa6270 msgr2=0x56287de74df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:20.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.818+0000 7eff6e007740 1 --2- 172.21.15.39:0/3551132543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56287daa6270 0x56287de74df0 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7eff58012950 tx=0x7eff58012e10 comp rx=0 tx=0).stop 2024-09-21T07:53:20.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.818+0000 7eff6e007740 1 -- 172.21.15.39:0/3551132543 shutdown_connections 2024-09-21T07:53:20.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.818+0000 7eff6e007740 1 --2- 172.21.15.39:0/3551132543 >> [v2:172.21.15.39:6832/2161818466,v1:172.21.15.39:6834/2161818466] conn(0x56287de93d20 0x56287de6c700 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.818+0000 7eff6e007740 1 --2- 172.21.15.39:0/3551132543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56287daa6270 0x56287de74df0 unknown :-1 s=CLOSED pgs=2622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.818+0000 7eff6e007740 1 --2- 172.21.15.39:0/3551132543 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x56287da85590 0x56287de45c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.818+0000 7eff6e007740 1 --2- 172.21.15.39:0/3551132543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56287da7d3b0 0x56287df14860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.818+0000 7eff4d600640 0 client.11829 ms_handle_reset on v2:172.21.15.39:6832/2161818466 2024-09-21T07:53:20.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.818+0000 7eff6e007740 1 -- 172.21.15.39:0/3551132543 >> 172.21.15.39:0/3551132543 conn(0x56287da9a3c0 msgr2=0x56287de89090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:20.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.818+0000 7eff6e007740 1 -- 172.21.15.39:0/3551132543 shutdown_connections 2024-09-21T07:53:20.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.818+0000 7eff6e007740 1 -- 172.21.15.39:0/3551132543 wait complete. 2024-09-21T07:53:20.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.818+0000 7eff6e007740 20 client.11829 trim_cache size 0 max 16384 2024-09-21T07:53:20.824 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:53:20.824 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:53:20.824 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-09-21T07:53:20.824 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 11364, 2024-09-21T07:53:20.824 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:53:20.824 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:53:20.824 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:53:20.824 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 12.242981555, 2024-09-21T07:53:20.824 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 291, 2024-09-21T07:53:20.825 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 250, 2024-09-21T07:53:20.825 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 250, 2024-09-21T07:53:20.825 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 104.075221082 2024-09-21T07:53:20.825 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:53:20.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.819+0000 7eff66e00640 1 -- 172.21.15.39:0/1716501900 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7eff3c078bc0 msgr2=0x7eff3c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:20.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.819+0000 7eff66e00640 1 --2- 172.21.15.39:0/1716501900 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7eff3c078bc0 0x7eff3c07b0d0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7eff48033000 tx=0x7eff480046c0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.819+0000 7eff66e00640 1 -- 172.21.15.39:0/1716501900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff600cceb0 msgr2=0x7eff601f54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:20.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.819+0000 7eff66e00640 1 --2- 172.21.15.39:0/1716501900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff600cceb0 0x7eff601f54d0 secure :-1 s=READY pgs=2618 cs=0 l=1 rev1=1 crypto rx=0x7eff5000b3e0 tx=0x7eff5000b8a0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.819+0000 7eff66e00640 1 -- 172.21.15.39:0/1716501900 shutdown_connections 2024-09-21T07:53:20.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.819+0000 7eff66e00640 1 --2- 172.21.15.39:0/1716501900 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7eff3c078bc0 0x7eff3c07b0d0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.819+0000 7eff66e00640 1 --2- 172.21.15.39:0/1716501900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff600b4ee0 0x7eff600b52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.819+0000 7eff66e00640 1 --2- 172.21.15.39:0/1716501900 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7eff6015c0c0 0x7eff601f6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.819+0000 7eff66e00640 1 --2- 172.21.15.39:0/1716501900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff600cceb0 0x7eff601f54d0 unknown :-1 s=CLOSED pgs=2618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.819+0000 7eff66e00640 1 -- 172.21.15.39:0/1716501900 >> 172.21.15.39:0/1716501900 conn(0x7eff600b3010 msgr2=0x7eff600b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:20.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.819+0000 7eff66e00640 1 -- 172.21.15.39:0/1716501900 shutdown_connections 2024-09-21T07:53:20.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:20.819+0000 7eff66e00640 1 -- 172.21.15.39:0/1716501900 wait complete. 2024-09-21T07:53:20.843 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:53:21.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.068+0000 7fbd75a00640 1 -- 172.21.15.39:0/3943416088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd7006fc10 msgr2=0x7fbd700d0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:21.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.068+0000 7fbd75a00640 1 --2- 172.21.15.39:0/3943416088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd7006fc10 0x7fbd700d0620 secure :-1 s=READY pgs=2624 cs=0 l=1 rev1=1 crypto rx=0x7fbd700d1240 tx=0x7fbd6802aa50 comp rx=0 tx=0).stop 2024-09-21T07:53:21.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.068+0000 7fbd75a00640 1 -- 172.21.15.39:0/3943416088 shutdown_connections 2024-09-21T07:53:21.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.068+0000 7fbd75a00640 1 --2- 172.21.15.39:0/3943416088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd7006fc10 0x7fbd700d0620 unknown :-1 s=CLOSED pgs=2624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:21.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.068+0000 7fbd75a00640 1 --2- 172.21.15.39:0/3943416088 >> v2:172.21.15.148:3300/0 conn(0x7fbd7015cd30 0x7fbd700cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:21.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.068+0000 7fbd75a00640 1 -- 172.21.15.39:0/3943416088 >> 172.21.15.39:0/3943416088 conn(0x7fbd7006cfc0 msgr2=0x7fbd7006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:21.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.069+0000 7fbd75a00640 1 -- 172.21.15.39:0/3943416088 shutdown_connections 2024-09-21T07:53:21.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.069+0000 7fbd75a00640 1 -- 172.21.15.39:0/3943416088 wait complete. 2024-09-21T07:53:21.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.071+0000 7fbd75a00640 1 Processor -- start 2024-09-21T07:53:21.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.071+0000 7fbd75a00640 1 -- start start 2024-09-21T07:53:21.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.072+0000 7fbd75a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd7006f800 0x7fbd701505b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:21.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.072+0000 7fbd75a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbd7006fc10 0x7fbd700b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:21.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.072+0000 7fbd75a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd7015cd30 0x7fbd700b4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:21.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.072+0000 7fbd75a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbd70073890 con 0x7fbd7006f800 2024-09-21T07:53:21.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.072+0000 7fbd75a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbd70073710 con 0x7fbd7015cd30 2024-09-21T07:53:21.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.072+0000 7fbd75a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fbd70073a10 con 0x7fbd7006fc10 2024-09-21T07:53:21.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.073+0000 7fbd6ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd7006f800 0x7fbd701505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:21.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.073+0000 7fbd6ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd7006f800 0x7fbd701505b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45348/0 (socket says 172.21.15.39:45348) 2024-09-21T07:53:21.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.073+0000 7fbd6ea00640 1 -- 172.21.15.39:0/3554200621 learned_addr learned my addr 172.21.15.39:0/3554200621 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:21.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.073+0000 7fbd6ea00640 1 -- 172.21.15.39:0/3554200621 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbd7006fc10 msgr2=0x7fbd700b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:21.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.073+0000 7fbd6ea00640 1 --2- 172.21.15.39:0/3554200621 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbd7006fc10 0x7fbd700b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:21.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.073+0000 7fbd6ea00640 1 -- 172.21.15.39:0/3554200621 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd7015cd30 msgr2=0x7fbd700b4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:21.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.073+0000 7fbd6ea00640 1 --2- 172.21.15.39:0/3554200621 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd7015cd30 0x7fbd700b4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:21.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.073+0000 7fbd6ea00640 1 -- 172.21.15.39:0/3554200621 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd700b5410 con 0x7fbd7006f800 2024-09-21T07:53:21.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.073+0000 7fbd6ea00640 1 --2- 172.21.15.39:0/3554200621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd7006f800 0x7fbd701505b0 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 crypto rx=0x7fbd6000b410 tx=0x7fbd6000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:21.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.074+0000 7fbd66a00640 1 -- 172.21.15.39:0/3554200621 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd60015020 con 0x7fbd7006f800 2024-09-21T07:53:21.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.074+0000 7fbd66a00640 1 -- 172.21.15.39:0/3554200621 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd60011920 con 0x7fbd7006f800 2024-09-21T07:53:21.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.074+0000 7fbd66a00640 1 -- 172.21.15.39:0/3554200621 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd60011d70 con 0x7fbd7006f800 2024-09-21T07:53:21.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.074+0000 7fbd75a00640 1 -- 172.21.15.39:0/3554200621 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd700aef90 con 0x7fbd7006f800 2024-09-21T07:53:21.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.074+0000 7fbd75a00640 1 -- 172.21.15.39:0/3554200621 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd701f7230 con 0x7fbd7006f800 2024-09-21T07:53:21.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.075+0000 7fbd75a00640 1 -- 172.21.15.39:0/3554200621 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd700af4b0 con 0x7fbd7006f800 2024-09-21T07:53:21.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.076+0000 7fbd66a00640 1 -- 172.21.15.39:0/3554200621 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbd60018020 con 0x7fbd7006f800 2024-09-21T07:53:21.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.077+0000 7fbd66a00640 1 --2- 172.21.15.39:0/3554200621 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbd4c078bc0 0x7fbd4c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:21.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.077+0000 7fbd66a00640 1 -- 172.21.15.39:0/3554200621 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(250..250 src has 1..250) ==== 7820+0+0 (secure 0 0 0) 0x7fbd6009ae70 con 0x7fbd7006f800 2024-09-21T07:53:21.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.078+0000 7fbd67e00640 1 --2- 172.21.15.39:0/3554200621 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbd4c078bc0 0x7fbd4c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:21.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.079+0000 7fbd67e00640 1 --2- 172.21.15.39:0/3554200621 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbd4c078bc0 0x7fbd4c07b0d0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fbd58000f60 tx=0x7fbd580079b0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:21.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.082+0000 7fbd66a00640 1 -- 172.21.15.39:0/3554200621 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbd700af4b0 con 0x7fbd7006f800 2024-09-21T07:53:21.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.315+0000 7fbd75a00640 1 -- 172.21.15.39:0/3554200621 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd700b3470 con 0x7fbd7006f800 2024-09-21T07:53:21.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.316+0000 7fbd66a00640 1 -- 172.21.15.39:0/3554200621 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 291 v291) ==== 95+0+5291 (secure 0 0 0) 0x7fbd6001f070 con 0x7fbd7006f800 2024-09-21T07:53:21.318 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:53:21.318 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":291,"btime":"2024-09-21T07:53:19:187136+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10633,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3495264983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3495264983},{"type":"v1","addr":"172.21.15.148:6837","nonce":3495264983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":288}],"filesystems":[{"mdsmap":{"epoch":291,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:53:19.187128+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":249,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":11364,"mds_1":11052,"mds_2":10621},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10621":{"gid":10621,"name":"b","rank":2,"incarnation":287,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6835/2191156088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2191156088},{"type":"v1","addr":"172.21.15.148:6835","nonce":2191156088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11364":{"gid":11364,"name":"a","rank":0,"incarnation":285,"state":"up:active","state_seq":28,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11364,"qdb_cluster":[10621,11052,11364]},"id":13}]} 2024-09-21T07:53:21.319 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 291 2024-09-21T07:53:21.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.318+0000 7fbd75a00640 1 -- 172.21.15.39:0/3554200621 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbd4c078bc0 msgr2=0x7fbd4c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:21.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.319+0000 7fbd75a00640 1 --2- 172.21.15.39:0/3554200621 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbd4c078bc0 0x7fbd4c07b0d0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fbd58000f60 tx=0x7fbd580079b0 comp rx=0 tx=0).stop 2024-09-21T07:53:21.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.319+0000 7fbd75a00640 1 -- 172.21.15.39:0/3554200621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd7006f800 msgr2=0x7fbd701505b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:21.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.319+0000 7fbd75a00640 1 --2- 172.21.15.39:0/3554200621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd7006f800 0x7fbd701505b0 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 crypto rx=0x7fbd6000b410 tx=0x7fbd6000b8d0 comp rx=0 tx=0).stop 2024-09-21T07:53:21.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.319+0000 7fbd75a00640 1 -- 172.21.15.39:0/3554200621 shutdown_connections 2024-09-21T07:53:21.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.319+0000 7fbd75a00640 1 --2- 172.21.15.39:0/3554200621 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fbd4c078bc0 0x7fbd4c07b0d0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:21.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.319+0000 7fbd75a00640 1 --2- 172.21.15.39:0/3554200621 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbd7015cd30 0x7fbd700b4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:21.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.319+0000 7fbd75a00640 1 --2- 172.21.15.39:0/3554200621 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fbd7006fc10 0x7fbd700b3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:21.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.319+0000 7fbd75a00640 1 --2- 172.21.15.39:0/3554200621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd7006f800 0x7fbd701505b0 unknown :-1 s=CLOSED pgs=2625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:21.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.319+0000 7fbd75a00640 1 -- 172.21.15.39:0/3554200621 >> 172.21.15.39:0/3554200621 conn(0x7fbd7006cfc0 msgr2=0x7fbd700b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:21.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.319+0000 7fbd75a00640 1 -- 172.21.15.39:0/3554200621 shutdown_connections 2024-09-21T07:53:21.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.319+0000 7fbd75a00640 1 -- 172.21.15.39:0/3554200621 wait complete. 2024-09-21T07:53:21.346 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump snaps 2024-09-21T07:53:21.595 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_created":9,"last_destroyed":7,"snaps":[{"snapid":8,"ino":1099511627780,"stamp":"2024-09-21T07:51:12.091986+0000","name":"s1","metadata":{}},{"snapid":9,"ino":1099511627780,"stamp":"2024-09-21T07:52:44.191763+0000","name":"s2","metadata":{}}]} 2024-09-21T07:53:21.596 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 9, "last_destroyed": 7, "snaps": [ { "ino": 1099511627780, "metadata": {}, "name": "s1", "snapid": 8, "stamp": "2024-09-21T07:51:12.091986+0000" }, { "ino": 1099511627780, "metadata": {}, "name": "s2", "snapid": 9, "stamp": "2024-09-21T07:52:44.191763+0000" } ] } 2024-09-21T07:53:21.596 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:53:21.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.825+0000 7f6683200640 1 -- 172.21.15.39:0/1276861734 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6660003c10 msgr2=0x7f6660004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:21.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.825+0000 7f6683200640 1 --2- 172.21.15.39:0/1276861734 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6660003c10 0x7f6660004060 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 crypto rx=0x7f666c001830 tx=0x7f6660005830 comp rx=0 tx=0).stop 2024-09-21T07:53:21.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.825+0000 7f6681200640 1 -- 172.21.15.39:0/1276861734 reap_dead start 2024-09-21T07:53:21.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.825+0000 7f6683200640 1 -- 172.21.15.39:0/1276861734 shutdown_connections 2024-09-21T07:53:21.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.825+0000 7f6683200640 1 -- 172.21.15.39:0/1276861734 >> 172.21.15.39:0/1276861734 conn(0x7f667c06cfc0 msgr2=0x7f667c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:21.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.825+0000 7f6683200640 1 -- 172.21.15.39:0/1276861734 shutdown_connections 2024-09-21T07:53:21.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.825+0000 7f6683200640 1 -- 172.21.15.39:0/1276861734 wait complete. 2024-09-21T07:53:21.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.827+0000 7f6683200640 1 Processor -- start 2024-09-21T07:53:21.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.827+0000 7f6683200640 1 -- start start 2024-09-21T07:53:21.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.829+0000 7f6683200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6660003c10 0x7f667c0aee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:21.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.829+0000 7f6683200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f667c0ad890 0x7f667c1f96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:21.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.829+0000 7f6683200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f667c14eba0 0x7f667c14d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:21.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.829+0000 7f6683200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f667c0cdf40 con 0x7f6660003c10 2024-09-21T07:53:21.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.829+0000 7f6683200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f667c0cddc0 con 0x7f667c0ad890 2024-09-21T07:53:21.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.829+0000 7f6683200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f667c073710 con 0x7f667c14eba0 2024-09-21T07:53:21.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.829+0000 7f667be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6660003c10 0x7f667c0aee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:21.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.829+0000 7f667be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6660003c10 0x7f667c0aee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45362/0 (socket says 172.21.15.39:45362) 2024-09-21T07:53:21.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.829+0000 7f667be00640 1 -- 172.21.15.39:0/1253054531 learned_addr learned my addr 172.21.15.39:0/1253054531 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:21.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.829+0000 7f667b400640 1 --2- 172.21.15.39:0/1253054531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f667c0ad890 0x7f667c1f96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:21.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.829+0000 7f6681200640 1 --2- 172.21.15.39:0/1253054531 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f667c14eba0 0x7f667c14d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:21.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.830+0000 7f667be00640 1 -- 172.21.15.39:0/1253054531 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f667c14eba0 msgr2=0x7f667c14d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:21.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.830+0000 7f667be00640 1 --2- 172.21.15.39:0/1253054531 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f667c14eba0 0x7f667c14d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:21.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.830+0000 7f667be00640 1 -- 172.21.15.39:0/1253054531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f667c0ad890 msgr2=0x7f667c1f96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:21.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.830+0000 7f667be00640 1 --2- 172.21.15.39:0/1253054531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f667c0ad890 0x7f667c1f96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:21.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.830+0000 7f667be00640 1 -- 172.21.15.39:0/1253054531 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f667c0adda0 con 0x7f6660003c10 2024-09-21T07:53:21.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.830+0000 7f667be00640 1 --2- 172.21.15.39:0/1253054531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6660003c10 0x7f667c0aee40 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 crypto rx=0x7f666c001950 tx=0x7f666c0069d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:21.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.830+0000 7f6673e00640 1 -- 172.21.15.39:0/1253054531 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666c043070 con 0x7f6660003c10 2024-09-21T07:53:21.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.830+0000 7f6673e00640 1 -- 172.21.15.39:0/1253054531 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f666c0038e0 con 0x7f6660003c10 2024-09-21T07:53:21.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.830+0000 7f6683200640 1 -- 172.21.15.39:0/1253054531 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f667c0ae080 con 0x7f6660003c10 2024-09-21T07:53:21.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.830+0000 7f6673e00640 1 -- 172.21.15.39:0/1253054531 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666c003cd0 con 0x7f6660003c10 2024-09-21T07:53:21.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.830+0000 7f6683200640 1 -- 172.21.15.39:0/1253054531 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f667c0ae640 con 0x7f6660003c10 2024-09-21T07:53:21.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.832+0000 7f6683200640 1 -- 172.21.15.39:0/1253054531 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f667c073ca0 con 0x7f6660003c10 2024-09-21T07:53:21.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.833+0000 7f6673e00640 1 -- 172.21.15.39:0/1253054531 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f666c046070 con 0x7f6660003c10 2024-09-21T07:53:21.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.834+0000 7f6673e00640 1 --2- 172.21.15.39:0/1253054531 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f665c078bc0 0x7f665c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:21.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.834+0000 7f6673e00640 1 -- 172.21.15.39:0/1253054531 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(250..250 src has 1..250) ==== 7820+0+0 (secure 0 0 0) 0x7f666c0bb300 con 0x7f6660003c10 2024-09-21T07:53:21.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.834+0000 7f667b400640 1 --2- 172.21.15.39:0/1253054531 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f665c078bc0 0x7f665c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:21.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.835+0000 7f667b400640 1 --2- 172.21.15.39:0/1253054531 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f665c078bc0 0x7f665c07b0d0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f667c0ccb20 tx=0x7f6668005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:21.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:21.839+0000 7f6673e00640 1 -- 172.21.15.39:0/1253054531 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f666c082a00 con 0x7f6660003c10 2024-09-21T07:53:22.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.073+0000 7f6683200640 1 -- 172.21.15.39:0/1253054531 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f667c0cd370 con 0x7f6660003c10 2024-09-21T07:53:22.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.073+0000 7f6673e00640 1 -- 172.21.15.39:0/1253054531 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 291 v291) ==== 95+0+5291 (secure 0 0 0) 0x7f666c087430 con 0x7f6660003c10 2024-09-21T07:53:22.076 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:53:22.076 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":291,"btime":"2024-09-21T07:53:19:187136+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10633,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3495264983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3495264983},{"type":"v1","addr":"172.21.15.148:6837","nonce":3495264983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":288}],"filesystems":[{"mdsmap":{"epoch":291,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:53:19.187128+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":249,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":11364,"mds_1":11052,"mds_2":10621},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10621":{"gid":10621,"name":"b","rank":2,"incarnation":287,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6835/2191156088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2191156088},{"type":"v1","addr":"172.21.15.148:6835","nonce":2191156088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11364":{"gid":11364,"name":"a","rank":0,"incarnation":285,"state":"up:active","state_seq":28,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11364,"qdb_cluster":[10621,11052,11364]},"id":13}]} 2024-09-21T07:53:22.076 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 291 2024-09-21T07:53:22.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.076+0000 7f6683200640 1 -- 172.21.15.39:0/1253054531 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f665c078bc0 msgr2=0x7f665c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:22.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.076+0000 7f6683200640 1 --2- 172.21.15.39:0/1253054531 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f665c078bc0 0x7f665c07b0d0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f667c0ccb20 tx=0x7f6668005f90 comp rx=0 tx=0).stop 2024-09-21T07:53:22.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.076+0000 7f6683200640 1 -- 172.21.15.39:0/1253054531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6660003c10 msgr2=0x7f667c0aee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:22.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.076+0000 7f6683200640 1 --2- 172.21.15.39:0/1253054531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6660003c10 0x7f667c0aee40 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 crypto rx=0x7f666c001950 tx=0x7f666c0069d0 comp rx=0 tx=0).stop 2024-09-21T07:53:22.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.076+0000 7f6683200640 1 -- 172.21.15.39:0/1253054531 shutdown_connections 2024-09-21T07:53:22.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.076+0000 7f6683200640 1 --2- 172.21.15.39:0/1253054531 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f665c078bc0 0x7f665c07b0d0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:22.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.076+0000 7f6683200640 1 --2- 172.21.15.39:0/1253054531 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f667c14eba0 0x7f667c14d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:22.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.076+0000 7f6683200640 1 --2- 172.21.15.39:0/1253054531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f667c0ad890 0x7f667c1f96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:22.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.076+0000 7f6683200640 1 --2- 172.21.15.39:0/1253054531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6660003c10 0x7f667c0aee40 unknown :-1 s=CLOSED pgs=2628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:22.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.076+0000 7f6683200640 1 -- 172.21.15.39:0/1253054531 >> 172.21.15.39:0/1253054531 conn(0x7f667c06cfc0 msgr2=0x7f667c0b2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:22.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.076+0000 7f6683200640 1 -- 172.21.15.39:0/1253054531 shutdown_connections 2024-09-21T07:53:22.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.076+0000 7f6683200640 1 -- 172.21.15.39:0/1253054531 wait complete. 2024-09-21T07:53:22.103 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps 2024-09-21T07:53:22.347 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_created":9,"last_destroyed":7,"snaps":[{"snapid":8,"ino":1099511627780,"stamp":"2024-09-21T07:51:12.091986+0000","name":"s1","metadata":{}},{"snapid":9,"ino":1099511627780,"stamp":"2024-09-21T07:52:44.191763+0000","name":"s2","metadata":{}}]} 2024-09-21T07:53:22.348 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 9, "last_destroyed": 7, "snaps": [ { "ino": 1099511627780, "metadata": {}, "name": "s1", "snapid": 8, "stamp": "2024-09-21T07:51:12.091986+0000" }, { "ino": 1099511627780, "metadata": {}, "name": "s2", "snapid": 9, "stamp": "2024-09-21T07:52:44.191763+0000" } ] } 2024-09-21T07:53:22.348 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:53:22.538 INFO:tasks.ceph.mds.a.smithi039.stderr:2024-09-21T07:53:22.536+0000 7fd91b600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:53:22.538 INFO:tasks.ceph.mds.a.smithi039.stderr:2024-09-21T07:53:22.536+0000 7fd91b600640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:53:22.538 INFO:tasks.ceph.mds.a.smithi039.stderr:2024-09-21T07:53:22.536+0000 7fd91b600640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-21T07:53:22.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.573+0000 7ff78a200640 1 -- 172.21.15.39:0/4196561751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff78406f000 msgr2=0x7ff78406f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:22.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.573+0000 7ff78a200640 1 --2- 172.21.15.39:0/4196561751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff78406f000 0x7ff78406f410 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x7ff7840710f0 tx=0x7ff77c02a980 comp rx=0 tx=0).stop 2024-09-21T07:53:22.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.573+0000 7ff78a200640 1 -- 172.21.15.39:0/4196561751 shutdown_connections 2024-09-21T07:53:22.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.574+0000 7ff78a200640 1 --2- 172.21.15.39:0/4196561751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff78406f000 0x7ff78406f410 unknown :-1 s=CLOSED pgs=2629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:22.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.574+0000 7ff78a200640 1 --2- 172.21.15.39:0/4196561751 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff7840ce310 0x7ff7840ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:22.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.574+0000 7ff78a200640 1 --2- 172.21.15.39:0/4196561751 >> v2:172.21.15.148:3300/0 conn(0x7ff7840cc390 0x7ff7840af600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:22.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.574+0000 7ff78a200640 1 -- 172.21.15.39:0/4196561751 >> 172.21.15.39:0/4196561751 conn(0x7ff7840b2510 msgr2=0x7ff7840d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:22.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.574+0000 7ff78a200640 1 -- 172.21.15.39:0/4196561751 shutdown_connections 2024-09-21T07:53:22.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.574+0000 7ff78a200640 1 -- 172.21.15.39:0/4196561751 wait complete. 2024-09-21T07:53:22.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.576+0000 7ff78a200640 1 Processor -- start 2024-09-21T07:53:22.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.576+0000 7ff78a200640 1 -- start start 2024-09-21T07:53:22.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.577+0000 7ff78a200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff78406f000 0x7ff7841552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:22.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.577+0000 7ff78a200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff7840cc390 0x7ff784073e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:22.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.577+0000 7ff78a200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7840ce310 0x7ff784074380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:22.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.577+0000 7ff78a200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff78406fe80 con 0x7ff78406f000 2024-09-21T07:53:22.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.577+0000 7ff78a200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff78406fd00 con 0x7ff7840ce310 2024-09-21T07:53:22.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.577+0000 7ff78a200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7ff784070000 con 0x7ff7840cc390 2024-09-21T07:53:22.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.578+0000 7ff788e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff78406f000 0x7ff7841552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:22.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.578+0000 7ff788e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff78406f000 0x7ff7841552e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56608/0 (socket says 172.21.15.39:56608) 2024-09-21T07:53:22.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.578+0000 7ff788e00640 1 -- 172.21.15.39:0/3669817282 learned_addr learned my addr 172.21.15.39:0/3669817282 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:22.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.578+0000 7ff788e00640 1 -- 172.21.15.39:0/3669817282 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff7840cc390 msgr2=0x7ff784073e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:22.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.578+0000 7ff788e00640 1 --2- 172.21.15.39:0/3669817282 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff7840cc390 0x7ff784073e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:22.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.578+0000 7ff788e00640 1 -- 172.21.15.39:0/3669817282 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7840ce310 msgr2=0x7ff784074380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:22.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.578+0000 7ff788e00640 1 --2- 172.21.15.39:0/3669817282 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7840ce310 0x7ff784074380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:22.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.578+0000 7ff788e00640 1 -- 172.21.15.39:0/3669817282 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff78415bf50 con 0x7ff78406f000 2024-09-21T07:53:22.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.578+0000 7ff788e00640 1 --2- 172.21.15.39:0/3669817282 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff78406f000 0x7ff7841552e0 secure :-1 s=READY pgs=2630 cs=0 l=1 rev1=1 crypto rx=0x7ff77800c9a0 tx=0x7ff77800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:22.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.579+0000 7ff780c00640 1 -- 172.21.15.39:0/3669817282 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff778007bb0 con 0x7ff78406f000 2024-09-21T07:53:22.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.579+0000 7ff780c00640 1 -- 172.21.15.39:0/3669817282 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff778007e40 con 0x7ff78406f000 2024-09-21T07:53:22.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.579+0000 7ff780c00640 1 -- 172.21.15.39:0/3669817282 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77800f6b0 con 0x7ff78406f000 2024-09-21T07:53:22.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.579+0000 7ff78a200640 1 -- 172.21.15.39:0/3669817282 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff784155820 con 0x7ff78406f000 2024-09-21T07:53:22.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.579+0000 7ff78a200640 1 -- 172.21.15.39:0/3669817282 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7840ae760 con 0x7ff78406f000 2024-09-21T07:53:22.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.580+0000 7ff78a200640 1 -- 172.21.15.39:0/3669817282 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7840d11b0 con 0x7ff78406f000 2024-09-21T07:53:22.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.581+0000 7ff780c00640 1 -- 172.21.15.39:0/3669817282 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff77800f850 con 0x7ff78406f000 2024-09-21T07:53:22.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.582+0000 7ff780c00640 1 --2- 172.21.15.39:0/3669817282 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff764078bc0 0x7ff76407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:22.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.582+0000 7ff780c00640 1 -- 172.21.15.39:0/3669817282 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(250..250 src has 1..250) ==== 7820+0+0 (secure 0 0 0) 0x7ff77809bd70 con 0x7ff78406f000 2024-09-21T07:53:22.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.583+0000 7ff783e00640 1 --2- 172.21.15.39:0/3669817282 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff764078bc0 0x7ff76407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:22.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.585+0000 7ff783e00640 1 --2- 172.21.15.39:0/3669817282 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff764078bc0 0x7ff76407b0d0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7ff7700058c0 tx=0x7ff770005850 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:22.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.586+0000 7ff780c00640 1 -- 172.21.15.39:0/3669817282 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff7840d11b0 con 0x7ff78406f000 2024-09-21T07:53:22.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.821+0000 7ff78a200640 1 -- 172.21.15.39:0/3669817282 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7840af600 con 0x7ff78406f000 2024-09-21T07:53:22.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.821+0000 7ff780c00640 1 -- 172.21.15.39:0/3669817282 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 291 v291) ==== 95+0+5291 (secure 0 0 0) 0x7ff778014030 con 0x7ff78406f000 2024-09-21T07:53:22.824 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:53:22.824 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":291,"btime":"2024-09-21T07:53:19:187136+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10633,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3495264983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3495264983},{"type":"v1","addr":"172.21.15.148:6837","nonce":3495264983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":288}],"filesystems":[{"mdsmap":{"epoch":291,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:53:19.187128+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":249,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":11364,"mds_1":11052,"mds_2":10621},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10621":{"gid":10621,"name":"b","rank":2,"incarnation":287,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6835/2191156088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2191156088},{"type":"v1","addr":"172.21.15.148:6835","nonce":2191156088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11364":{"gid":11364,"name":"a","rank":0,"incarnation":285,"state":"up:active","state_seq":28,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11364,"qdb_cluster":[10621,11052,11364]},"id":13}]} 2024-09-21T07:53:22.824 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 291 2024-09-21T07:53:22.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.824+0000 7ff78a200640 1 -- 172.21.15.39:0/3669817282 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff764078bc0 msgr2=0x7ff76407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:22.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.824+0000 7ff78a200640 1 --2- 172.21.15.39:0/3669817282 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff764078bc0 0x7ff76407b0d0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7ff7700058c0 tx=0x7ff770005850 comp rx=0 tx=0).stop 2024-09-21T07:53:22.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.824+0000 7ff78a200640 1 -- 172.21.15.39:0/3669817282 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff78406f000 msgr2=0x7ff7841552e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:22.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.824+0000 7ff78a200640 1 --2- 172.21.15.39:0/3669817282 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff78406f000 0x7ff7841552e0 secure :-1 s=READY pgs=2630 cs=0 l=1 rev1=1 crypto rx=0x7ff77800c9a0 tx=0x7ff77800ce60 comp rx=0 tx=0).stop 2024-09-21T07:53:22.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.824+0000 7ff78a200640 1 -- 172.21.15.39:0/3669817282 shutdown_connections 2024-09-21T07:53:22.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.824+0000 7ff78a200640 1 --2- 172.21.15.39:0/3669817282 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff764078bc0 0x7ff76407b0d0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:22.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.824+0000 7ff78a200640 1 --2- 172.21.15.39:0/3669817282 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff7840ce310 0x7ff784074380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:22.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.824+0000 7ff78a200640 1 --2- 172.21.15.39:0/3669817282 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff7840cc390 0x7ff784073e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:22.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.824+0000 7ff78a200640 1 --2- 172.21.15.39:0/3669817282 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff78406f000 0x7ff7841552e0 unknown :-1 s=CLOSED pgs=2630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:22.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.824+0000 7ff78a200640 1 -- 172.21.15.39:0/3669817282 >> 172.21.15.39:0/3669817282 conn(0x7ff7840b2510 msgr2=0x7ff7840ad520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:22.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.824+0000 7ff78a200640 1 -- 172.21.15.39:0/3669817282 shutdown_connections 2024-09-21T07:53:22.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:22.824+0000 7ff78a200640 1 -- 172.21.15.39:0/3669817282 wait complete. 2024-09-21T07:53:22.847 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json dump snaps 2024-09-21T07:53:23.094 INFO:teuthology.orchestra.run.smithi148.stdout:{"last_created":9,"last_destroyed":7,"snaps":[{"snapid":8,"ino":1099511627780,"stamp":"2024-09-21T07:51:12.091986+0000","name":"s1","metadata":{}},{"snapid":9,"ino":1099511627780,"stamp":"2024-09-21T07:52:44.191763+0000","name":"s2","metadata":{}}]} 2024-09-21T07:53:23.095 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 9, "last_destroyed": 7, "snaps": [ { "ino": 1099511627780, "metadata": {}, "name": "s1", "snapid": 8, "stamp": "2024-09-21T07:51:12.091986+0000" }, { "ino": 1099511627780, "metadata": {}, "name": "s2", "snapid": 9, "stamp": "2024-09-21T07:52:44.191763+0000" } ] } 2024-09-21T07:53:23.095 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:53:23.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.320+0000 7f4157000640 1 -- 172.21.15.39:0/4273666675 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41500b43f0 msgr2=0x7f41500ca890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:23.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.320+0000 7f4157000640 1 --2- 172.21.15.39:0/4273666675 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41500b43f0 0x7f41500ca890 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f414400a970 tx=0x7f414402b730 comp rx=0 tx=0).stop 2024-09-21T07:53:23.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.320+0000 7f4157000640 1 -- 172.21.15.39:0/4273666675 shutdown_connections 2024-09-21T07:53:23.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.320+0000 7f4157000640 1 --2- 172.21.15.39:0/4273666675 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41500b43f0 0x7f41500ca890 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:23.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.320+0000 7f4157000640 1 -- 172.21.15.39:0/4273666675 >> 172.21.15.39:0/4273666675 conn(0x7f41500b2510 msgr2=0x7f41500d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:23.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.320+0000 7f4157000640 1 -- 172.21.15.39:0/4273666675 shutdown_connections 2024-09-21T07:53:23.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.320+0000 7f4157000640 1 -- 172.21.15.39:0/4273666675 wait complete. 2024-09-21T07:53:23.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.323+0000 7f4157000640 1 Processor -- start 2024-09-21T07:53:23.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.323+0000 7f4157000640 1 -- start start 2024-09-21T07:53:23.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.324+0000 7f4157000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41500af490 0x7f415014d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:23.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.324+0000 7f4157000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41500b43f0 0x7f415009d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:23.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.324+0000 7f4157000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f415015abe0 0x7f415009d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:23.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.324+0000 7f4157000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f41500cfd20 con 0x7f415015abe0 2024-09-21T07:53:23.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.324+0000 7f4157000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f41500cfba0 con 0x7f41500af490 2024-09-21T07:53:23.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.324+0000 7f4157000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f41500cfea0 con 0x7f41500b43f0 2024-09-21T07:53:23.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.324+0000 7f4156600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f415015abe0 0x7f415009d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:23.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.324+0000 7f4156600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f415015abe0 0x7f415009d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56610/0 (socket says 172.21.15.39:56610) 2024-09-21T07:53:23.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.325+0000 7f4156600640 1 -- 172.21.15.39:0/2225916690 learned_addr learned my addr 172.21.15.39:0/2225916690 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:23.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.325+0000 7f4155200640 1 --2- 172.21.15.39:0/2225916690 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41500b43f0 0x7f415009d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:23.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.325+0000 7f4155c00640 1 --2- 172.21.15.39:0/2225916690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41500af490 0x7f415014d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:23.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.325+0000 7f4156600640 1 -- 172.21.15.39:0/2225916690 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41500b43f0 msgr2=0x7f415009d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:23.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.325+0000 7f4156600640 1 --2- 172.21.15.39:0/2225916690 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41500b43f0 0x7f415009d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:23.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.325+0000 7f4156600640 1 -- 172.21.15.39:0/2225916690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41500af490 msgr2=0x7f415014d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:23.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.325+0000 7f4156600640 1 --2- 172.21.15.39:0/2225916690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41500af490 0x7f415014d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:23.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.325+0000 7f4156600640 1 -- 172.21.15.39:0/2225916690 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f415014e1b0 con 0x7f415015abe0 2024-09-21T07:53:23.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.325+0000 7f4156600640 1 --2- 172.21.15.39:0/2225916690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f415015abe0 0x7f415009d840 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7f414800c700 tx=0x7f414800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:23.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.325+0000 7f414d600640 1 -- 172.21.15.39:0/2225916690 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4148003b70 con 0x7f415015abe0 2024-09-21T07:53:23.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.325+0000 7f414d600640 1 -- 172.21.15.39:0/2225916690 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4148003d10 con 0x7f415015abe0 2024-09-21T07:53:23.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.325+0000 7f414d600640 1 -- 172.21.15.39:0/2225916690 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414800aab0 con 0x7f415015abe0 2024-09-21T07:53:23.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.325+0000 7f4157000640 1 -- 172.21.15.39:0/2225916690 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41501f35c0 con 0x7f415015abe0 2024-09-21T07:53:23.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.326+0000 7f4157000640 1 -- 172.21.15.39:0/2225916690 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f415014b300 con 0x7f415015abe0 2024-09-21T07:53:23.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.327+0000 7f4157000640 1 -- 172.21.15.39:0/2225916690 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41500ca890 con 0x7f415015abe0 2024-09-21T07:53:23.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.328+0000 7f414d600640 1 -- 172.21.15.39:0/2225916690 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f414800ac50 con 0x7f415015abe0 2024-09-21T07:53:23.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.329+0000 7f414d600640 1 --2- 172.21.15.39:0/2225916690 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4130078bc0 0x7f413007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:23.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.329+0000 7f414d600640 1 -- 172.21.15.39:0/2225916690 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(250..250 src has 1..250) ==== 7820+0+0 (secure 0 0 0) 0x7f414809b350 con 0x7f415015abe0 2024-09-21T07:53:23.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.329+0000 7f4155c00640 1 --2- 172.21.15.39:0/2225916690 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4130078bc0 0x7f413007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:23.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.330+0000 7f4155c00640 1 --2- 172.21.15.39:0/2225916690 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4130078bc0 0x7f413007b0d0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f4144035000 tx=0x7f4144004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:23.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.334+0000 7f414d600640 1 -- 172.21.15.39:0/2225916690 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4148062f50 con 0x7f415015abe0 2024-09-21T07:53:23.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.567+0000 7f4157000640 1 -- 172.21.15.39:0/2225916690 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f415006f740 con 0x7f415015abe0 2024-09-21T07:53:23.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.568+0000 7f414d600640 1 -- 172.21.15.39:0/2225916690 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 291 v291) ==== 95+0+5291 (secure 0 0 0) 0x7f4148062850 con 0x7f415015abe0 2024-09-21T07:53:23.570 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:53:23.570 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":291,"btime":"2024-09-21T07:53:19:187136+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10633,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3495264983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3495264983},{"type":"v1","addr":"172.21.15.148:6837","nonce":3495264983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":288}],"filesystems":[{"mdsmap":{"epoch":291,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:53:19.187128+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":249,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":11364,"mds_1":11052,"mds_2":10621},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10621":{"gid":10621,"name":"b","rank":2,"incarnation":287,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6835/2191156088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2191156088},{"type":"v1","addr":"172.21.15.148:6835","nonce":2191156088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11364":{"gid":11364,"name":"a","rank":0,"incarnation":285,"state":"up:active","state_seq":28,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11364,"qdb_cluster":[10621,11052,11364]},"id":13}]} 2024-09-21T07:53:23.570 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 291 2024-09-21T07:53:23.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.570+0000 7f4157000640 1 -- 172.21.15.39:0/2225916690 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4130078bc0 msgr2=0x7f413007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:23.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.570+0000 7f4157000640 1 --2- 172.21.15.39:0/2225916690 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4130078bc0 0x7f413007b0d0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f4144035000 tx=0x7f4144004730 comp rx=0 tx=0).stop 2024-09-21T07:53:23.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.570+0000 7f4157000640 1 -- 172.21.15.39:0/2225916690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f415015abe0 msgr2=0x7f415009d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:23.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.570+0000 7f4157000640 1 --2- 172.21.15.39:0/2225916690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f415015abe0 0x7f415009d840 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7f414800c700 tx=0x7f414800cbc0 comp rx=0 tx=0).stop 2024-09-21T07:53:23.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.570+0000 7f4157000640 1 -- 172.21.15.39:0/2225916690 shutdown_connections 2024-09-21T07:53:23.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.570+0000 7f4157000640 1 --2- 172.21.15.39:0/2225916690 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4130078bc0 0x7f413007b0d0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:23.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.570+0000 7f4157000640 1 --2- 172.21.15.39:0/2225916690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f415015abe0 0x7f415009d840 unknown :-1 s=CLOSED pgs=2632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:23.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.570+0000 7f4157000640 1 --2- 172.21.15.39:0/2225916690 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f41500b43f0 0x7f415009d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:23.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.570+0000 7f4157000640 1 --2- 172.21.15.39:0/2225916690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41500af490 0x7f415014d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:23.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.570+0000 7f4157000640 1 -- 172.21.15.39:0/2225916690 >> 172.21.15.39:0/2225916690 conn(0x7f41500b2510 msgr2=0x7f415015a290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:23.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.570+0000 7f4157000640 1 -- 172.21.15.39:0/2225916690 shutdown_connections 2024-09-21T07:53:23.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:23.570+0000 7f4157000640 1 -- 172.21.15.39:0/2225916690 wait complete. 2024-09-21T07:53:23.594 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump snaps --server 2024-09-21T07:53:23.838 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_snap":9,"last_created":9,"last_destroyed":7,"pending_noop":[],"snaps":[{"snapid":8,"ino":1099511627780,"stamp":"2024-09-21T07:51:12.091986+0000","name":"s1","metadata":{}},{"snapid":9,"ino":1099511627780,"stamp":"2024-09-21T07:52:44.191763+0000","name":"s2","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2024-09-21T07:53:23.838 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 9, "last_destroyed": 7, "last_snap": 9, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627780, "metadata": {}, "name": "s1", "snapid": 8, "stamp": "2024-09-21T07:51:12.091986+0000" }, { "ino": 1099511627780, "metadata": {}, "name": "s2", "snapid": 9, "stamp": "2024-09-21T07:52:44.191763+0000" } ] } 2024-09-21T07:53:23.838 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T07:53:23.838 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:53:24.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.064+0000 7f9779400640 1 -- 172.21.15.39:0/1186705175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97740b9eb0 msgr2=0x7f97740b61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:24.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.064+0000 7f9779400640 1 --2- 172.21.15.39:0/1186705175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97740b9eb0 0x7f97740b61d0 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 crypto rx=0x7f975c009830 tx=0x7f975c02a510 comp rx=0 tx=0).stop 2024-09-21T07:53:24.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.064+0000 7f9779400640 1 -- 172.21.15.39:0/1186705175 shutdown_connections 2024-09-21T07:53:24.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.064+0000 7f9779400640 1 --2- 172.21.15.39:0/1186705175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97740b9eb0 0x7f97740b61d0 unknown :-1 s=CLOSED pgs=2634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:24.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.064+0000 7f9779400640 1 -- 172.21.15.39:0/1186705175 >> 172.21.15.39:0/1186705175 conn(0x7f97740d4750 msgr2=0x7f97740d4b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:24.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.064+0000 7f9779400640 1 -- 172.21.15.39:0/1186705175 shutdown_connections 2024-09-21T07:53:24.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.064+0000 7f9779400640 1 -- 172.21.15.39:0/1186705175 wait complete. 2024-09-21T07:53:24.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.067+0000 7f9779400640 1 Processor -- start 2024-09-21T07:53:24.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.067+0000 7f9779400640 1 -- start start 2024-09-21T07:53:24.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.068+0000 7f9779400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97740b6710 0x7f977414e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:24.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.068+0000 7f9779400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f97740b9eb0 0x7f97741f9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:24.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.068+0000 7f9779400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97740d0bd0 0x7f97741f97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:24.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.068+0000 7f9779400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f977406e720 con 0x7f97740b6710 2024-09-21T07:53:24.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.068+0000 7f9779400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f977406e8a0 con 0x7f97740d0bd0 2024-09-21T07:53:24.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.068+0000 7f9779400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f977406ea20 con 0x7f97740b9eb0 2024-09-21T07:53:24.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.068+0000 7f9772800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97740b6710 0x7f977414e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:24.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.069+0000 7f9772800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97740b6710 0x7f977414e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56636/0 (socket says 172.21.15.39:56636) 2024-09-21T07:53:24.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.069+0000 7f9772800640 1 -- 172.21.15.39:0/1658519791 learned_addr learned my addr 172.21.15.39:0/1658519791 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:24.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.069+0000 7f9773200640 1 --2- 172.21.15.39:0/1658519791 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97740d0bd0 0x7f97741f97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:24.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.069+0000 7f9769e00640 1 --2- 172.21.15.39:0/1658519791 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f97740b9eb0 0x7f97741f9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:24.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.069+0000 7f9772800640 1 -- 172.21.15.39:0/1658519791 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f97740b9eb0 msgr2=0x7f97741f9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:24.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.069+0000 7f9772800640 1 --2- 172.21.15.39:0/1658519791 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f97740b9eb0 0x7f97741f9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:24.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.069+0000 7f9772800640 1 -- 172.21.15.39:0/1658519791 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97740d0bd0 msgr2=0x7f97741f97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:24.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.069+0000 7f9772800640 1 --2- 172.21.15.39:0/1658519791 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97740d0bd0 0x7f97741f97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:24.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.069+0000 7f9772800640 1 -- 172.21.15.39:0/1658519791 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97741f6ad0 con 0x7f97740b6710 2024-09-21T07:53:24.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.069+0000 7f9773200640 1 --2- 172.21.15.39:0/1658519791 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97740d0bd0 0x7f97741f97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:53:24.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.069+0000 7f9772800640 1 --2- 172.21.15.39:0/1658519791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97740b6710 0x7f977414e480 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 crypto rx=0x7f976400c3c0 tx=0x7f976400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:24.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.069+0000 7f976b400640 1 -- 172.21.15.39:0/1658519791 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9764017070 con 0x7f97740b6710 2024-09-21T07:53:24.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.070+0000 7f976b400640 1 -- 172.21.15.39:0/1658519791 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f97640035f0 con 0x7f97740b6710 2024-09-21T07:53:24.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.070+0000 7f976b400640 1 -- 172.21.15.39:0/1658519791 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9764003a80 con 0x7f97740b6710 2024-09-21T07:53:24.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.070+0000 7f9779400640 1 -- 172.21.15.39:0/1658519791 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f977415f650 con 0x7f97740b6710 2024-09-21T07:53:24.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.070+0000 7f9779400640 1 -- 172.21.15.39:0/1658519791 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f977415f930 con 0x7f97740b6710 2024-09-21T07:53:24.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.071+0000 7f9779400640 1 -- 172.21.15.39:0/1658519791 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97740b61d0 con 0x7f97740b6710 2024-09-21T07:53:24.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.072+0000 7f976b400640 1 -- 172.21.15.39:0/1658519791 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f976401a070 con 0x7f97740b6710 2024-09-21T07:53:24.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.073+0000 7f976b400640 1 --2- 172.21.15.39:0/1658519791 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9750078b90 0x7f975007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:24.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.074+0000 7f976b400640 1 -- 172.21.15.39:0/1658519791 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(250..250 src has 1..250) ==== 7820+0+0 (secure 0 0 0) 0x7f976409a930 con 0x7f97740b6710 2024-09-21T07:53:24.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.074+0000 7f9769e00640 1 --2- 172.21.15.39:0/1658519791 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9750078b90 0x7f975007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:24.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.075+0000 7f9769e00640 1 --2- 172.21.15.39:0/1658519791 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9750078b90 0x7f975007b0a0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f975c034000 tx=0x7f975c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:24.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.078+0000 7f976b400640 1 -- 172.21.15.39:0/1658519791 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f97740b6b20 con 0x7f97740b6710 2024-09-21T07:53:24.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.312+0000 7f9779400640 1 -- 172.21.15.39:0/1658519791 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97740b4ba0 con 0x7f97740b6710 2024-09-21T07:53:24.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.313+0000 7f976b400640 1 -- 172.21.15.39:0/1658519791 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 291 v291) ==== 95+0+5291 (secure 0 0 0) 0x7f9764062490 con 0x7f97740b6710 2024-09-21T07:53:24.315 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:53:24.315 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":291,"btime":"2024-09-21T07:53:19:187136+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10633,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3495264983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3495264983},{"type":"v1","addr":"172.21.15.148:6837","nonce":3495264983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":288}],"filesystems":[{"mdsmap":{"epoch":291,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:53:19.187128+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":249,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":11364,"mds_1":11052,"mds_2":10621},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10621":{"gid":10621,"name":"b","rank":2,"incarnation":287,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6835/2191156088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2191156088},{"type":"v1","addr":"172.21.15.148:6835","nonce":2191156088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11364":{"gid":11364,"name":"a","rank":0,"incarnation":285,"state":"up:active","state_seq":28,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11364,"qdb_cluster":[10621,11052,11364]},"id":13}]} 2024-09-21T07:53:24.315 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 291 2024-09-21T07:53:24.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.315+0000 7f9779400640 1 -- 172.21.15.39:0/1658519791 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9750078b90 msgr2=0x7f975007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:24.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.315+0000 7f9779400640 1 --2- 172.21.15.39:0/1658519791 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9750078b90 0x7f975007b0a0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f975c034000 tx=0x7f975c004730 comp rx=0 tx=0).stop 2024-09-21T07:53:24.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.315+0000 7f9779400640 1 -- 172.21.15.39:0/1658519791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97740b6710 msgr2=0x7f977414e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:24.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.315+0000 7f9779400640 1 --2- 172.21.15.39:0/1658519791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97740b6710 0x7f977414e480 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 crypto rx=0x7f976400c3c0 tx=0x7f976400c880 comp rx=0 tx=0).stop 2024-09-21T07:53:24.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.315+0000 7f9779400640 1 -- 172.21.15.39:0/1658519791 shutdown_connections 2024-09-21T07:53:24.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.315+0000 7f9779400640 1 --2- 172.21.15.39:0/1658519791 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9750078b90 0x7f975007b0a0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:24.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.315+0000 7f9779400640 1 --2- 172.21.15.39:0/1658519791 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97740d0bd0 0x7f97741f97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:24.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.315+0000 7f9779400640 1 --2- 172.21.15.39:0/1658519791 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f97740b9eb0 0x7f97741f9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:24.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.315+0000 7f9779400640 1 --2- 172.21.15.39:0/1658519791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97740b6710 0x7f977414e480 unknown :-1 s=CLOSED pgs=2635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:24.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.315+0000 7f9779400640 1 -- 172.21.15.39:0/1658519791 >> 172.21.15.39:0/1658519791 conn(0x7f97740d4750 msgr2=0x7f97740b33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:24.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.315+0000 7f9779400640 1 -- 172.21.15.39:0/1658519791 shutdown_connections 2024-09-21T07:53:24.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:24.315+0000 7f9779400640 1 -- 172.21.15.39:0/1658519791 wait complete. 2024-09-21T07:53:24.343 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump snaps --server 2024-09-21T07:53:24.588 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_snap":9,"last_created":9,"last_destroyed":7,"pending_noop":[],"snaps":[{"snapid":8,"ino":1099511627780,"stamp":"2024-09-21T07:51:12.091986+0000","name":"s1","metadata":{}},{"snapid":9,"ino":1099511627780,"stamp":"2024-09-21T07:52:44.191763+0000","name":"s2","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2024-09-21T07:53:24.588 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 9, "last_destroyed": 7, "last_snap": 9, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627780, "metadata": {}, "name": "s1", "snapid": 8, "stamp": "2024-09-21T07:51:12.091986+0000" }, { "ino": 1099511627780, "metadata": {}, "name": "s2", "snapid": 9, "stamp": "2024-09-21T07:52:44.191763+0000" } ] } 2024-09-21T07:53:24.588 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:53:24.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:53:24.588 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-09-21T07:53:34.053 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:53:34.058 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:53:34.489 INFO:teuthology.orchestra.run.smithi039.stderr:mkdir: cannot create directory ‘d0/d2/dir/.snap/s2’: Permission denied 2024-09-21T07:53:34.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:53:34.490 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:53:34.490 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:53:34.528 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:53:34.528 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:53:34.605 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:53:34.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:34.871+0000 7f3848c00640 1 -- 172.21.15.39:0/2103717252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f384415d890 msgr2=0x7f384415a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:34.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:34.871+0000 7f3848c00640 1 --2- 172.21.15.39:0/2103717252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f384415d890 0x7f384415a2f0 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 crypto rx=0x7f3844073150 tx=0x7f383402a560 comp rx=0 tx=0).stop 2024-09-21T07:53:34.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:34.872+0000 7f3848c00640 1 -- 172.21.15.39:0/2103717252 shutdown_connections 2024-09-21T07:53:34.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:34.872+0000 7f3848c00640 1 --2- 172.21.15.39:0/2103717252 >> v2:172.21.15.148:3300/0 conn(0x7f3844160210 0x7f3844160620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:34.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:34.872+0000 7f3848c00640 1 --2- 172.21.15.39:0/2103717252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f384415d890 0x7f384415a2f0 unknown :-1 s=CLOSED pgs=2637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:34.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:34.872+0000 7f3848c00640 1 -- 172.21.15.39:0/2103717252 >> 172.21.15.39:0/2103717252 conn(0x7f384406cde0 msgr2=0x7f384406d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:34.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:34.872+0000 7f3848c00640 1 -- 172.21.15.39:0/2103717252 shutdown_connections 2024-09-21T07:53:34.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:34.872+0000 7f3848c00640 1 -- 172.21.15.39:0/2103717252 wait complete. 2024-09-21T07:53:34.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:34.874+0000 7f3848c00640 1 Processor -- start 2024-09-21T07:53:34.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:34.874+0000 7f3848c00640 1 -- start start 2024-09-21T07:53:34.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:34.875+0000 7f3848c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38440b2710 0x7f38441499e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:34.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:34.875+0000 7f3848c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f384415d890 0x7f38441f5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:34.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:34.875+0000 7f3848c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3844160210 0x7f384414cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:34.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:34.875+0000 7f3848c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f384406f800 con 0x7f384415d890 2024-09-21T07:53:34.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:34.875+0000 7f3848c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f384406f680 con 0x7f38440b2710 2024-09-21T07:53:34.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:34.875+0000 7f3848c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f384406f980 con 0x7f3844160210 2024-09-21T07:53:34.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:34.876+0000 7f3842a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f384415d890 0x7f38441f5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:34.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:34.876+0000 7f3842a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f384415d890 0x7f38441f5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60766/0 (socket says 172.21.15.39:60766) 2024-09-21T07:53:34.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:34.876+0000 7f3842a00640 1 -- 172.21.15.39:0/3667502773 learned_addr learned my addr 172.21.15.39:0/3667502773 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:34.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:34.876+0000 7f3843400640 1 --2- 172.21.15.39:0/3667502773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38440b2710 0x7f38441499e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:34.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:34.876+0000 7f3843e00640 1 --2- 172.21.15.39:0/3667502773 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3844160210 0x7f384414cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:34.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:34.876+0000 7f3842a00640 1 -- 172.21.15.39:0/3667502773 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3844160210 msgr2=0x7f384414cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:34.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:34.876+0000 7f3842a00640 1 --2- 172.21.15.39:0/3667502773 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3844160210 0x7f384414cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:34.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:34.876+0000 7f3842a00640 1 -- 172.21.15.39:0/3667502773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38440b2710 msgr2=0x7f38441499e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:34.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:34.876+0000 7f3842a00640 1 --2- 172.21.15.39:0/3667502773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38440b2710 0x7f38441499e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:34.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:34.876+0000 7f3842a00640 1 -- 172.21.15.39:0/3667502773 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38441f6020 con 0x7f384415d890 2024-09-21T07:53:34.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:34.877+0000 7f3842a00640 1 --2- 172.21.15.39:0/3667502773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f384415d890 0x7f38441f5580 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 crypto rx=0x7f383000c3a0 tx=0x7f383000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:34.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:34.877+0000 7f383b400640 1 -- 172.21.15.39:0/3667502773 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3830017070 con 0x7f384415d890 2024-09-21T07:53:34.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:34.877+0000 7f383b400640 1 -- 172.21.15.39:0/3667502773 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38300053c0 con 0x7f384415d890 2024-09-21T07:53:34.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:34.877+0000 7f383b400640 1 -- 172.21.15.39:0/3667502773 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3830003850 con 0x7f384415d890 2024-09-21T07:53:34.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:34.877+0000 7f3848c00640 1 -- 172.21.15.39:0/3667502773 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38441f6300 con 0x7f384415d890 2024-09-21T07:53:34.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:34.877+0000 7f3848c00640 1 -- 172.21.15.39:0/3667502773 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f384414c600 con 0x7f384415d890 2024-09-21T07:53:34.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:34.878+0000 7f3848c00640 1 -- 172.21.15.39:0/3667502773 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f384406fea0 con 0x7f384415d890 2024-09-21T07:53:34.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:34.879+0000 7f383b400640 1 -- 172.21.15.39:0/3667502773 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f383001a070 con 0x7f384415d890 2024-09-21T07:53:34.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:34.880+0000 7f383b400640 1 --2- 172.21.15.39:0/3667502773 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3824078b90 0x7f382407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:34.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:34.881+0000 7f383b400640 1 -- 172.21.15.39:0/3667502773 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(250..250 src has 1..250) ==== 7820+0+0 (secure 0 0 0) 0x7f383009a8a0 con 0x7f384415d890 2024-09-21T07:53:34.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:34.881+0000 7f3843400640 1 --2- 172.21.15.39:0/3667502773 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3824078b90 0x7f382407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:34.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:34.882+0000 7f3843400640 1 --2- 172.21.15.39:0/3667502773 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3824078b90 0x7f382407b0a0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f3834033000 tx=0x7f3834004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:34.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:34.885+0000 7f383b400640 1 -- 172.21.15.39:0/3667502773 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3844071020 con 0x7f384415d890 2024-09-21T07:53:35.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.094+0000 7f3848c00640 1 -- 172.21.15.39:0/3667502773 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f38441f5db0 con 0x7f384415d890 2024-09-21T07:53:35.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.095+0000 7f383b400640 1 -- 172.21.15.39:0/3667502773 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v250) ==== 81+0+973 (secure 0 0 0) 0x7f3830061e40 con 0x7f384415d890 2024-09-21T07:53:35.097 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/4061644382 2024-09-21T08:53:16.749620+0000 2024-09-21T07:53:35.097 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/597301351 2024-09-22T07:53:09.120490+0000 2024-09-21T07:53:35.097 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6835/597301351 2024-09-22T07:53:09.120490+0000 2024-09-21T07:53:35.097 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/3381818843 2024-09-22T07:53:08.565426+0000 2024-09-21T07:53:35.098 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6834/3381818843 2024-09-22T07:53:08.565426+0000 2024-09-21T07:53:35.098 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/106826898 2024-09-21T08:52:26.149462+0000 2024-09-21T07:53:35.098 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/109229385 2024-09-22T07:51:36.504781+0000 2024-09-21T07:53:35.098 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/109229385 2024-09-22T07:51:36.504781+0000 2024-09-21T07:53:35.098 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/1438851059 2024-09-22T07:51:00.902180+0000 2024-09-21T07:53:35.098 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6835/1438851059 2024-09-22T07:51:00.902180+0000 2024-09-21T07:53:35.098 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/2824493631 2024-09-22T07:50:31.817924+0000 2024-09-21T07:53:35.098 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6833/2824493631 2024-09-22T07:50:31.817924+0000 2024-09-21T07:53:35.098 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/4143807732 2024-09-22T07:50:32.369206+0000 2024-09-21T07:53:35.098 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/4143807732 2024-09-22T07:50:32.369206+0000 2024-09-21T07:53:35.098 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/3497801035 2024-09-22T07:50:24.850152+0000 2024-09-21T07:53:35.099 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6834/3497801035 2024-09-22T07:50:24.850152+0000 2024-09-21T07:53:35.099 INFO:teuthology.orchestra.run.smithi039.stderr:listed 16 entries 2024-09-21T07:53:35.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.097+0000 7f3848c00640 1 -- 172.21.15.39:0/3667502773 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3824078b90 msgr2=0x7f382407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:35.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.097+0000 7f3848c00640 1 --2- 172.21.15.39:0/3667502773 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3824078b90 0x7f382407b0a0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f3834033000 tx=0x7f3834004730 comp rx=0 tx=0).stop 2024-09-21T07:53:35.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.097+0000 7f3848c00640 1 -- 172.21.15.39:0/3667502773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f384415d890 msgr2=0x7f38441f5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:35.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.097+0000 7f3848c00640 1 --2- 172.21.15.39:0/3667502773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f384415d890 0x7f38441f5580 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 crypto rx=0x7f383000c3a0 tx=0x7f383000c860 comp rx=0 tx=0).stop 2024-09-21T07:53:35.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.097+0000 7f3848c00640 1 -- 172.21.15.39:0/3667502773 shutdown_connections 2024-09-21T07:53:35.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.097+0000 7f3848c00640 1 --2- 172.21.15.39:0/3667502773 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3824078b90 0x7f382407b0a0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:35.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.097+0000 7f3848c00640 1 --2- 172.21.15.39:0/3667502773 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3844160210 0x7f384414cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:35.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.097+0000 7f3848c00640 1 --2- 172.21.15.39:0/3667502773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f384415d890 0x7f38441f5580 unknown :-1 s=CLOSED pgs=2638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:35.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.097+0000 7f3848c00640 1 --2- 172.21.15.39:0/3667502773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38440b2710 0x7f38441499e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:35.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.097+0000 7f3848c00640 1 -- 172.21.15.39:0/3667502773 >> 172.21.15.39:0/3667502773 conn(0x7f384406cde0 msgr2=0x7f38440ce430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:35.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.098+0000 7f3848c00640 1 -- 172.21.15.39:0/3667502773 shutdown_connections 2024-09-21T07:53:35.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.098+0000 7f3848c00640 1 -- 172.21.15.39:0/3667502773 wait complete. 2024-09-21T07:53:35.123 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T07:53:35.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:53:35.123 DEBUG:teuthology.orchestra.run.smithi039:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T07:53:35.202 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T07:53:35.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:53:35.203 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:53:35.239 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:53:35.239 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:53:35.314 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:53:35.315 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:53:35.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.580+0000 7f853d600640 1 -- 172.21.15.39:0/2835479950 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8538155bc0 msgr2=0x7f85380d06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:35.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.580+0000 7f853d600640 1 --2- 172.21.15.39:0/2835479950 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8538155bc0 0x7f85380d06c0 secure :-1 s=READY pgs=2639 cs=0 l=1 rev1=1 crypto rx=0x7f8520005fd0 tx=0x7f852002a5d0 comp rx=0 tx=0).stop 2024-09-21T07:53:35.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.580+0000 7f853d600640 1 -- 172.21.15.39:0/2835479950 shutdown_connections 2024-09-21T07:53:35.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.580+0000 7f853d600640 1 --2- 172.21.15.39:0/2835479950 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8538155bc0 0x7f85380d06c0 unknown :-1 s=CLOSED pgs=2639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:35.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.580+0000 7f853d600640 1 --2- 172.21.15.39:0/2835479950 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f85381557b0 0x7f85380b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:35.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.580+0000 7f853d600640 1 -- 172.21.15.39:0/2835479950 >> 172.21.15.39:0/2835479950 conn(0x7f853806cfc0 msgr2=0x7f853806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:35.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.581+0000 7f853d600640 1 -- 172.21.15.39:0/2835479950 shutdown_connections 2024-09-21T07:53:35.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.581+0000 7f853d600640 1 -- 172.21.15.39:0/2835479950 wait complete. 2024-09-21T07:53:35.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.583+0000 7f853d600640 1 Processor -- start 2024-09-21T07:53:35.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.583+0000 7f853d600640 1 -- start start 2024-09-21T07:53:35.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.584+0000 7f853d600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85380b4840 0x7f853815a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:35.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.584+0000 7f853d600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f85381557b0 0x7f8538145470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:35.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.584+0000 7f853d600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8538155bc0 0x7f8538142a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:35.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.584+0000 7f853d600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f85380d0fc0 con 0x7f85380b4840 2024-09-21T07:53:35.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.584+0000 7f853d600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f85380d0e40 con 0x7f8538155bc0 2024-09-21T07:53:35.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.584+0000 7f853d600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f85380add50 con 0x7f85381557b0 2024-09-21T07:53:35.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.584+0000 7f8536a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85380b4840 0x7f853815a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:35.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.585+0000 7f8536a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85380b4840 0x7f853815a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60778/0 (socket says 172.21.15.39:60778) 2024-09-21T07:53:35.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.585+0000 7f8536a00640 1 -- 172.21.15.39:0/1950064308 learned_addr learned my addr 172.21.15.39:0/1950064308 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:35.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.585+0000 7f8537400640 1 --2- 172.21.15.39:0/1950064308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8538155bc0 0x7f8538142a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:35.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.585+0000 7f852fe00640 1 --2- 172.21.15.39:0/1950064308 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f85381557b0 0x7f8538145470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:35.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.585+0000 7f8536a00640 1 -- 172.21.15.39:0/1950064308 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f85381557b0 msgr2=0x7f8538145470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:35.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.585+0000 7f8536a00640 1 --2- 172.21.15.39:0/1950064308 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f85381557b0 0x7f8538145470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:35.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.585+0000 7f8536a00640 1 -- 172.21.15.39:0/1950064308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8538155bc0 msgr2=0x7f8538142a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:35.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.585+0000 7f8536a00640 1 --2- 172.21.15.39:0/1950064308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8538155bc0 0x7f8538142a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:35.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.585+0000 7f8536a00640 1 -- 172.21.15.39:0/1950064308 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8538145fb0 con 0x7f85380b4840 2024-09-21T07:53:35.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.586+0000 7f8536a00640 1 --2- 172.21.15.39:0/1950064308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85380b4840 0x7f853815a030 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 crypto rx=0x7f852800d740 tx=0x7f852800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:35.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.586+0000 7f852ea00640 1 -- 172.21.15.39:0/1950064308 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8528017070 con 0x7f85380b4840 2024-09-21T07:53:35.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.586+0000 7f852ea00640 1 -- 172.21.15.39:0/1950064308 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8528009d20 con 0x7f85380b4840 2024-09-21T07:53:35.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.586+0000 7f852ea00640 1 -- 172.21.15.39:0/1950064308 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85280106c0 con 0x7f85380b4840 2024-09-21T07:53:35.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.586+0000 7f853d600640 1 -- 172.21.15.39:0/1950064308 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8538146290 con 0x7f85380b4840 2024-09-21T07:53:35.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.586+0000 7f853d600640 1 -- 172.21.15.39:0/1950064308 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8538159b10 con 0x7f85380b4840 2024-09-21T07:53:35.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.587+0000 7f853d600640 1 -- 172.21.15.39:0/1950064308 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85380b4300 con 0x7f85380b4840 2024-09-21T07:53:35.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.588+0000 7f852ea00640 1 -- 172.21.15.39:0/1950064308 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f852801a070 con 0x7f85380b4840 2024-09-21T07:53:35.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.589+0000 7f852ea00640 1 --2- 172.21.15.39:0/1950064308 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8514078b90 0x7f851407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:35.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.590+0000 7f852ea00640 1 -- 172.21.15.39:0/1950064308 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(250..250 src has 1..250) ==== 7820+0+0 (secure 0 0 0) 0x7f852809b710 con 0x7f85380b4840 2024-09-21T07:53:35.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.590+0000 7f852fe00640 1 --2- 172.21.15.39:0/1950064308 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8514078b90 0x7f851407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:35.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.591+0000 7f852fe00640 1 --2- 172.21.15.39:0/1950064308 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8514078b90 0x7f851407b0a0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f8520033000 tx=0x7f8520004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:35.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.594+0000 7f852ea00640 1 -- 172.21.15.39:0/1950064308 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8528062c20 con 0x7f85380b4840 2024-09-21T07:53:35.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.832+0000 7f853d600640 1 -- 172.21.15.39:0/1950064308 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f85380cfbd0 con 0x7f85380b4840 2024-09-21T07:53:35.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.833+0000 7f852ea00640 1 -- 172.21.15.39:0/1950064308 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v291) ==== 53+0+73 (secure 0 0 0) 0x7f8528067650 con 0x7f85380b4840 2024-09-21T07:53:35.835 INFO:teuthology.orchestra.run.smithi039.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:53:35.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.835+0000 7f853d600640 1 -- 172.21.15.39:0/1950064308 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8514078b90 msgr2=0x7f851407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:35.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.835+0000 7f853d600640 1 --2- 172.21.15.39:0/1950064308 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8514078b90 0x7f851407b0a0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f8520033000 tx=0x7f8520004730 comp rx=0 tx=0).stop 2024-09-21T07:53:35.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.835+0000 7f853d600640 1 -- 172.21.15.39:0/1950064308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85380b4840 msgr2=0x7f853815a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:35.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.835+0000 7f853d600640 1 --2- 172.21.15.39:0/1950064308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85380b4840 0x7f853815a030 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 crypto rx=0x7f852800d740 tx=0x7f852800dc00 comp rx=0 tx=0).stop 2024-09-21T07:53:35.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.835+0000 7f853d600640 1 -- 172.21.15.39:0/1950064308 shutdown_connections 2024-09-21T07:53:35.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.835+0000 7f853d600640 1 --2- 172.21.15.39:0/1950064308 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8514078b90 0x7f851407b0a0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:35.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.835+0000 7f853d600640 1 --2- 172.21.15.39:0/1950064308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8538155bc0 0x7f8538142a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:35.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.835+0000 7f853d600640 1 --2- 172.21.15.39:0/1950064308 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f85381557b0 0x7f8538145470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:35.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.835+0000 7f853d600640 1 --2- 172.21.15.39:0/1950064308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85380b4840 0x7f853815a030 unknown :-1 s=CLOSED pgs=2640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:35.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.835+0000 7f853d600640 1 -- 172.21.15.39:0/1950064308 >> 172.21.15.39:0/1950064308 conn(0x7f853806cfc0 msgr2=0x7f853806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:35.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.836+0000 7f853d600640 1 -- 172.21.15.39:0/1950064308 shutdown_connections 2024-09-21T07:53:35.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:35.836+0000 7f853d600640 1 -- 172.21.15.39:0/1950064308 wait complete. 2024-09-21T07:53:35.863 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:53:35.863 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:53:35.863 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi039.front.sepia.ceph.com 2024-09-21T07:53:35.863 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T07:53:35.863 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:53:35.863 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:53:35.863 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:53:35.863 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:53:35.863 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:53:35.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:53:35.864 DEBUG:teuthology.orchestra.run.smithi039:> ip addr 2024-09-21T07:53:35.899 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:53:35.899 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:53:35.900 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:53:35.900 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:53:35.900 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2024-09-21T07:53:35.900 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:53:35.900 INFO:teuthology.orchestra.run.smithi039.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:53:35.900 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:53:35.900 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f0 2024-09-21T07:53:35.900 INFO:teuthology.orchestra.run.smithi039.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:53:35.900 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c7 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:53:35.900 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f1 2024-09-21T07:53:35.900 INFO:teuthology.orchestra.run.smithi039.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:53:35.900 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:70 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:53:35.900 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f0 2024-09-21T07:53:35.901 INFO:teuthology.orchestra.run.smithi039.stdout: inet 172.21.15.39/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T07:53:35.901 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft 41503sec preferred_lft 41503sec 2024-09-21T07:53:35.901 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec4:7aff:febd:1370/64 scope link 2024-09-21T07:53:35.901 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:53:35.901 INFO:teuthology.orchestra.run.smithi039.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:53:35.901 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:71 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:53:35.901 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f1 2024-09-21T07:53:35.901 INFO:teuthology.orchestra.run.smithi039.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:53:35.901 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff 2024-09-21T07:53:35.901 INFO:teuthology.orchestra.run.smithi039.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:53:35.901 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:53:35.901 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::b079:ddff:fe0e:6333/64 scope link 2024-09-21T07:53:35.901 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:53:35.901 INFO:teuthology.orchestra.run.smithi039.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:53:35.902 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:53:35.902 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::2ca7:98ff:fe8a:d0ed/64 scope link 2024-09-21T07:53:35.902 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:53:35.902 INFO:teuthology.orchestra.run.smithi039.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:53:35.902 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether ee:10:53:01:0c:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:53:35.902 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec10:53ff:fe01:c44/64 scope link 2024-09-21T07:53:35.902 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:53:35.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:53:35.902 DEBUG:teuthology.orchestra.run.smithi039:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:53:35.993 INFO:teuthology.orchestra.run.smithi039.stdout:1 2024-09-21T07:53:35.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:53:35.996 DEBUG:teuthology.orchestra.run.smithi039:> route 2024-09-21T07:53:36.032 INFO:teuthology.orchestra.run.smithi039.stdout:Kernel IP routing table 2024-09-21T07:53:36.032 INFO:teuthology.orchestra.run.smithi039.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:53:36.032 INFO:teuthology.orchestra.run.smithi039.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T07:53:36.032 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T07:53:36.032 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:53:36.033 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:53:36.033 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-21T07:53:36.033 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T07:53:36.033 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T07:53:36.033 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T07:53:36.033 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-21T07:53:36.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:53:36.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:53:36.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:53:36.230 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-09-21T07:53:36.265 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:53:36.265 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T07:53:36.265 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:53:36.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:53:36.265 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-09-21T07:53:36.362 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:53:36.362 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T07:53:36.397 INFO:teuthology.orchestra.run.smithi039.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:53:36.397 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:53:36.397 DEBUG:teuthology.orchestra.run.smithi039:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:53:36.471 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-21T07:53:36.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:53:36.471 DEBUG:teuthology.orchestra.run.smithi039:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:53:36.671 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:53:36.671 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:53:36.705 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:53:36.705 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:53:36.705 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: resolved to: "172.21.15.39:3300,172.21.15.148:3300,172.21.15.148:3301" 2024-09-21T07:53:36.705 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: trying mount with new device syntax: 0@7ab9b04a-f88d-49aa-a493-e5c076637ffb.cephfs=/ 2024-09-21T07:53:36.705 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.39:3300/172.21.15.148:3300/172.21.15.148:3301" will pass to kernel 2024-09-21T07:53:36.706 INFO:teuthology.orchestra.run.smithi039.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T07:53:36.707 INFO:teuthology.orchestra.run.smithi039.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:53:36.707 INFO:teuthology.orchestra.run.smithi039.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:53:36.707 INFO:teuthology.orchestra.run.smithi039.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:53:36.707 INFO:teuthology.orchestra.run.smithi039.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:53:36.710 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:53:36.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:53:36.710 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:53:36.766 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:53:36.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:36 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:53:36.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:53:36.860 INFO:teuthology.orchestra.run.smithi039.stdout:client9401 2024-09-21T07:53:36.860 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:53:36.860 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:53:36.895 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.13:0 flush journal 2024-09-21T07:53:37.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.162+0000 7f5122c00640 1 -- 172.21.15.39:0/2900172063 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f511c15cf60 msgr2=0x7f511c0b32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:37.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.162+0000 7f5122c00640 1 --2- 172.21.15.39:0/2900172063 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f511c15cf60 0x7f511c0b32b0 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 crypto rx=0x7f510c009830 tx=0x7f510c02a480 comp rx=0 tx=0).stop 2024-09-21T07:53:37.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.163+0000 7f5122c00640 1 -- 172.21.15.39:0/2900172063 shutdown_connections 2024-09-21T07:53:37.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.163+0000 7f5122c00640 1 --2- 172.21.15.39:0/2900172063 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f511c15d950 0x7f511c0cbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.163+0000 7f5122c00640 1 --2- 172.21.15.39:0/2900172063 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f511c15cf60 0x7f511c0b32b0 unknown :-1 s=CLOSED pgs=2642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.163+0000 7f5122c00640 1 -- 172.21.15.39:0/2900172063 >> 172.21.15.39:0/2900172063 conn(0x7f511c06d440 msgr2=0x7f511c0b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:37.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.163+0000 7f5122c00640 1 -- 172.21.15.39:0/2900172063 shutdown_connections 2024-09-21T07:53:37.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.163+0000 7f5122c00640 1 -- 172.21.15.39:0/2900172063 wait complete. 2024-09-21T07:53:37.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.165+0000 7f5122c00640 1 Processor -- start 2024-09-21T07:53:37.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.165+0000 7f5122c00640 1 -- start start 2024-09-21T07:53:37.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.167+0000 7f5122c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f511c0cce70 0x7f511c15f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:37.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.167+0000 7f5122c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f511c15cf60 0x7f511c15ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:37.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.167+0000 7f5122c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f511c15d950 0x7f511c1f8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:37.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.167+0000 7f5122c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f511c06f9e0 con 0x7f511c15cf60 2024-09-21T07:53:37.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.167+0000 7f5122c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f511c06f860 con 0x7f511c15d950 2024-09-21T07:53:37.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.167+0000 7f5122c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f511c06fb60 con 0x7f511c0cce70 2024-09-21T07:53:37.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.167+0000 7f511b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f511c15cf60 0x7f511c15ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:37.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.167+0000 7f511b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f511c15cf60 0x7f511c15ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60806/0 (socket says 172.21.15.39:60806) 2024-09-21T07:53:37.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.167+0000 7f511b400640 1 -- 172.21.15.39:0/36260990 learned_addr learned my addr 172.21.15.39:0/36260990 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:37.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.167+0000 7f511be00640 1 --2- 172.21.15.39:0/36260990 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f511c0cce70 0x7f511c15f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:37.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.167+0000 7f5120c00640 1 --2- 172.21.15.39:0/36260990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f511c15d950 0x7f511c1f8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:37.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.167+0000 7f511b400640 1 -- 172.21.15.39:0/36260990 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f511c0cce70 msgr2=0x7f511c15f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:37.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.167+0000 7f511b400640 1 --2- 172.21.15.39:0/36260990 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f511c0cce70 0x7f511c15f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.167+0000 7f511b400640 1 -- 172.21.15.39:0/36260990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f511c15d950 msgr2=0x7f511c1f8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:37.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.167+0000 7f511b400640 1 --2- 172.21.15.39:0/36260990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f511c15d950 0x7f511c1f8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.168+0000 7f511b400640 1 -- 172.21.15.39:0/36260990 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f511c1f9aa0 con 0x7f511c15cf60 2024-09-21T07:53:37.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.168+0000 7f5120c00640 1 --2- 172.21.15.39:0/36260990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f511c15d950 0x7f511c1f8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:53:37.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.168+0000 7f511b400640 1 --2- 172.21.15.39:0/36260990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f511c15cf60 0x7f511c15ae00 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 crypto rx=0x7f510800d740 tx=0x7f510800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:37.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.168+0000 7f5113e00640 1 -- 172.21.15.39:0/36260990 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5108017070 con 0x7f511c15cf60 2024-09-21T07:53:37.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.168+0000 7f5113e00640 1 -- 172.21.15.39:0/36260990 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5108009d20 con 0x7f511c15cf60 2024-09-21T07:53:37.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.168+0000 7f5122c00640 1 -- 172.21.15.39:0/36260990 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f511c160c60 con 0x7f511c15cf60 2024-09-21T07:53:37.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.168+0000 7f5113e00640 1 -- 172.21.15.39:0/36260990 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51080106c0 con 0x7f511c15cf60 2024-09-21T07:53:37.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.168+0000 7f5122c00640 1 -- 172.21.15.39:0/36260990 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f511c0c9110 con 0x7f511c15cf60 2024-09-21T07:53:37.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.171+0000 7f5113e00640 1 -- 172.21.15.39:0/36260990 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f510801a070 con 0x7f511c15cf60 2024-09-21T07:53:37.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.172+0000 7f5113e00640 1 --2- 172.21.15.39:0/36260990 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f50fc078b90 0x7f50fc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:37.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.172+0000 7f5113e00640 1 -- 172.21.15.39:0/36260990 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(250..250 src has 1..250) ==== 7820+0+0 (secure 0 0 0) 0x7f510809b6b0 con 0x7f511c15cf60 2024-09-21T07:53:37.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.172+0000 7f5129e21740 1 Processor -- start 2024-09-21T07:53:37.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.172+0000 7f5129e21740 1 -- start start 2024-09-21T07:53:37.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.173+0000 7f511be00640 1 --2- 172.21.15.39:0/36260990 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f50fc078b90 0x7f50fc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:37.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.174+0000 7f5129e21740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e074ca390 0x561e078b8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:37.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.174+0000 7f511be00640 1 --2- 172.21.15.39:0/36260990 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f50fc078b90 0x7f50fc07b0a0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f510c034000 tx=0x7f510c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:37.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.174+0000 7f5129e21740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e074a14d0 0x561e0789b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:37.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.174+0000 7f5129e21740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561e074a96b0 0x561e0789b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:37.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.174+0000 7f5129e21740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561e077d11c0 con 0x561e074ca390 2024-09-21T07:53:37.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.174+0000 7f5129e21740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561e077e4870 con 0x561e074a14d0 2024-09-21T07:53:37.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.174+0000 7f5129e21740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x561e077d3ca0 con 0x561e074a96b0 2024-09-21T07:53:37.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.174+0000 7f511be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e074ca390 0x561e078b8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:37.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.174+0000 7f511be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e074ca390 0x561e078b8550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60818/0 (socket says 172.21.15.39:60818) 2024-09-21T07:53:37.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.174+0000 7f511be00640 1 -- 172.21.15.39:0/1235115019 learned_addr learned my addr 172.21.15.39:0/1235115019 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:37.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.174+0000 7f511b400640 1 --2- 172.21.15.39:0/1235115019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e074a14d0 0x561e0789b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:37.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.174+0000 7f5120c00640 1 --2- 172.21.15.39:0/1235115019 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561e074a96b0 0x561e0789b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:37.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.174+0000 7f511be00640 1 -- 172.21.15.39:0/1235115019 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561e074a96b0 msgr2=0x561e0789b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:53:37.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.174+0000 7f511be00640 1 --2- 172.21.15.39:0/1235115019 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561e074a96b0 0x561e0789b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.175+0000 7f511be00640 1 -- 172.21.15.39:0/1235115019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e074a14d0 msgr2=0x561e0789b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:53:37.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.175+0000 7f511be00640 1 --2- 172.21.15.39:0/1235115019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e074a14d0 0x561e0789b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.175+0000 7f511be00640 1 -- 172.21.15.39:0/1235115019 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e075ac7d0 con 0x561e074ca390 2024-09-21T07:53:37.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.175+0000 7f511be00640 1 --2- 172.21.15.39:0/1235115019 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e074ca390 0x561e078b8550 secure :-1 s=READY pgs=2644 cs=0 l=1 rev1=1 crypto rx=0x7f510c004530 tx=0x7f510c03e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b4b48b72a352a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:37.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.175+0000 7f5107e00640 1 -- 172.21.15.39:0/1235115019 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510c005220 con 0x561e074ca390 2024-09-21T07:53:37.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.175+0000 7f5107e00640 1 -- 172.21.15.39:0/1235115019 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f510c0054b0 con 0x561e074ca390 2024-09-21T07:53:37.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.175+0000 7f5107e00640 1 -- 172.21.15.39:0/1235115019 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510c003a10 con 0x561e074ca390 2024-09-21T07:53:37.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.176+0000 7f5129e21740 1 -- 172.21.15.39:0/1235115019 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e074ca390 msgr2=0x561e078b8550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:37.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.176+0000 7f5129e21740 1 --2- 172.21.15.39:0/1235115019 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e074ca390 0x561e078b8550 secure :-1 s=READY pgs=2644 cs=0 l=1 rev1=1 crypto rx=0x7f510c004530 tx=0x7f510c03e040 comp rx=0 tx=0).stop 2024-09-21T07:53:37.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.176+0000 7f5129e21740 1 -- 172.21.15.39:0/1235115019 shutdown_connections 2024-09-21T07:53:37.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.176+0000 7f5129e21740 1 --2- 172.21.15.39:0/1235115019 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561e074a96b0 0x561e0789b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.176+0000 7f5129e21740 1 --2- 172.21.15.39:0/1235115019 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e074a14d0 0x561e0789b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.176+0000 7f5129e21740 1 --2- 172.21.15.39:0/1235115019 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e074ca390 0x561e078b8550 unknown :-1 s=CLOSED pgs=2644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.176+0000 7f5129e21740 1 -- 172.21.15.39:0/1235115019 >> 172.21.15.39:0/1235115019 conn(0x561e074be4e0 msgr2=0x561e07871440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:37.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.176+0000 7f5129e21740 1 -- 172.21.15.39:0/1235115019 shutdown_connections 2024-09-21T07:53:37.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.176+0000 7f5129e21740 1 -- 172.21.15.39:0/1235115019 wait complete. 2024-09-21T07:53:37.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.177+0000 7f5129e21740 1 -- start start 2024-09-21T07:53:37.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.177+0000 7f5129e21740 1 Processor -- start 2024-09-21T07:53:37.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.179+0000 7f5129e21740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e074a14d0 0x561e078b6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:37.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.179+0000 7f5129e21740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e074a96b0 0x561e07889230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:37.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.179+0000 7f5129e21740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561e074ca390 0x561e07871200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:37.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.179+0000 7f5129e21740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561e0777a410 con 0x561e074a96b0 2024-09-21T07:53:37.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.179+0000 7f5129e21740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561e077d7ab0 con 0x561e074a14d0 2024-09-21T07:53:37.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.179+0000 7f5129e21740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x561e07544740 con 0x561e074ca390 2024-09-21T07:53:37.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.179+0000 7f5129e21740 10 client.0 _ll_register_callbacks cb 0x561e07757360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:53:37.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.179+0000 7f511b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e074a96b0 0x561e07889230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:37.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.179+0000 7f511b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e074a96b0 0x561e07889230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60830/0 (socket says 172.21.15.39:60830) 2024-09-21T07:53:37.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.179+0000 7f511b400640 1 -- 172.21.15.39:0/742573336 learned_addr learned my addr 172.21.15.39:0/742573336 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:37.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.179+0000 7f511be00640 1 --2- 172.21.15.39:0/742573336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e074a14d0 0x561e078b6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:37.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.179+0000 7f5120c00640 1 --2- 172.21.15.39:0/742573336 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561e074ca390 0x561e07871200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:37.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.180+0000 7f511b400640 1 -- 172.21.15.39:0/742573336 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561e074ca390 msgr2=0x561e07871200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:53:37.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.180+0000 7f511b400640 1 --2- 172.21.15.39:0/742573336 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561e074ca390 0x561e07871200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.180+0000 7f511b400640 1 -- 172.21.15.39:0/742573336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e074a14d0 msgr2=0x561e078b6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:53:37.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.180+0000 7f511b400640 1 --2- 172.21.15.39:0/742573336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e074a14d0 0x561e078b6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.180+0000 7f511b400640 1 -- 172.21.15.39:0/742573336 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e07535970 con 0x561e074a96b0 2024-09-21T07:53:37.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.180+0000 7f511be00640 1 --2- 172.21.15.39:0/742573336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e074a14d0 0x561e078b6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:53:37.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.180+0000 7f5120c00640 1 --2- 172.21.15.39:0/742573336 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561e074ca390 0x561e07871200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:53:37.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.180+0000 7f511b400640 1 --2- 172.21.15.39:0/742573336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e074a96b0 0x561e07889230 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 crypto rx=0x7f5108099300 tx=0x7f51080624b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd5d7e7e94c563bd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:37.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.180+0000 7f5104c00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:53:37.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.180+0000 7f5104c00640 1 -- 172.21.15.39:0/742573336 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5108021040 con 0x561e074a96b0 2024-09-21T07:53:37.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.180+0000 7f5104c00640 1 -- 172.21.15.39:0/742573336 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(250..250 src has 1..250) ==== 7820+0+0 (secure 0 0 0) 0x7f5108067100 con 0x561e074a96b0 2024-09-21T07:53:37.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.180+0000 7f5129e21740 1 -- 172.21.15.39:0/742573336 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f510c0054b0 con 0x561e074a96b0 2024-09-21T07:53:37.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.180+0000 7f5104c00640 1 -- 172.21.15.39:0/742573336 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=291) ==== 24+0+0 (secure 0 0 0) 0x7f510805eaf0 con 0x561e074a96b0 2024-09-21T07:53:37.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.180+0000 7f5129e21740 10 client.11892 fetch_fsmap learned FSMap version 291 2024-09-21T07:53:37.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.180+0000 7f5129e21740 1 -- 172.21.15.39:0/742573336 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=291,monmap=2+}) -- 0x561e075136a0 con 0x561e074a96b0 2024-09-21T07:53:37.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.180+0000 7f5104c00640 1 -- 172.21.15.39:0/742573336 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f510c0054b0 con 0x561e074a96b0 2024-09-21T07:53:37.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.181+0000 7f5104c00640 1 -- 172.21.15.39:0/742573336 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 291) ==== 3542+0+0 (secure 0 0 0) 0x7f5108067300 con 0x561e074a96b0 2024-09-21T07:53:37.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.181+0000 7f5129e21740 10 client.11892 fetch_fsmap finished waiting for FSMap version 291 2024-09-21T07:53:37.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.181+0000 7f5129e21740 10 client.11892 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.a 2024-09-21T07:53:37.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.181+0000 7f5129e21740 20 client.11892 populate_metadata read hostname 'smithi039' 2024-09-21T07:53:37.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.181+0000 7f5129e21740 1 --2- 172.21.15.39:0/742573336 >> [v2:172.21.15.39:6832/2161818466,v1:172.21.15.39:6834/2161818466] conn(0x561e07488f60 0x561e078c4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:37.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.181+0000 7f5129e21740 4 client.11892 mds_command: new command op to 11364 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:53:37.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.181+0000 7f5129e21740 1 -- 172.21.15.39:0/742573336 --> [v2:172.21.15.39:6832/2161818466,v1:172.21.15.39:6834/2161818466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e077e67c0 con 0x561e07488f60 2024-09-21T07:53:37.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.181+0000 7f511be00640 1 --2- 172.21.15.39:0/742573336 >> [v2:172.21.15.39:6832/2161818466,v1:172.21.15.39:6834/2161818466] conn(0x561e07488f60 0x561e078c4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:37.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.182+0000 7f511be00640 1 --2- 172.21.15.39:0/742573336 >> [v2:172.21.15.39:6832/2161818466,v1:172.21.15.39:6834/2161818466] conn(0x561e07488f60 0x561e078c4e90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4397f8d84e8140d server_cookie=e10c72b56c4f804c in_seq=0 out_seq=0 2024-09-21T07:53:37.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.182+0000 7f5104c00640 10 client.11892 ms_handle_connect on v2:172.21.15.39:6832/2161818466 2024-09-21T07:53:37.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.182+0000 7f5104c00640 1 -- 172.21.15.39:0/742573336 <== mds.0 v2:172.21.15.39:6832/2161818466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561e077e67c0 con 0x561e07488f60 2024-09-21T07:53:37.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.182+0000 7f5104c00640 10 client.11892 handle_command_reply: tid=0 2024-09-21T07:53:37.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.182+0000 7f5129e21740 1 client.11892 shutdown 2024-09-21T07:53:37.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.183+0000 7f5129e21740 1 -- 172.21.15.39:0/742573336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e074a96b0 msgr2=0x561e07889230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:37.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.183+0000 7f5129e21740 1 --2- 172.21.15.39:0/742573336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e074a96b0 0x561e07889230 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 crypto rx=0x7f5108099300 tx=0x7f51080624b0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.183+0000 7f5129e21740 1 -- 172.21.15.39:0/742573336 shutdown_connections 2024-09-21T07:53:37.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.183+0000 7f5129e21740 1 --2- 172.21.15.39:0/742573336 >> [v2:172.21.15.39:6832/2161818466,v1:172.21.15.39:6834/2161818466] conn(0x561e07488f60 0x561e078c4e90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.183+0000 7f5129e21740 1 --2- 172.21.15.39:0/742573336 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561e074ca390 0x561e07871200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.183+0000 7f5129e21740 1 --2- 172.21.15.39:0/742573336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e074a96b0 0x561e07889230 unknown :-1 s=CLOSED pgs=2645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.183+0000 7f5104c00640 0 client.11892 ms_handle_reset on v2:172.21.15.39:6832/2161818466 2024-09-21T07:53:37.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.183+0000 7f5129e21740 1 --2- 172.21.15.39:0/742573336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e074a14d0 0x561e078b6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.183+0000 7f5129e21740 1 -- 172.21.15.39:0/742573336 >> 172.21.15.39:0/742573336 conn(0x561e074be4e0 msgr2=0x561e0786a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:37.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.183+0000 7f5129e21740 1 -- 172.21.15.39:0/742573336 shutdown_connections 2024-09-21T07:53:37.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.183+0000 7f5129e21740 1 -- 172.21.15.39:0/742573336 wait complete. 2024-09-21T07:53:37.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.183+0000 7f5129e21740 20 client.11892 trim_cache size 0 max 16384 2024-09-21T07:53:37.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.199+0000 7f5129e21740 1 Processor -- start 2024-09-21T07:53:37.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.199+0000 7f5129e21740 1 -- start start 2024-09-21T07:53:37.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.200+0000 7f5129e21740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561e074a14d0 0x561e078d9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:37.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.200+0000 7f5129e21740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e074a96b0 0x561e07888f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:37.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.200+0000 7f5129e21740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e074ca390 0x561e07872380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:37.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.200+0000 7f5129e21740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f511c205430 con 0x561e074a96b0 2024-09-21T07:53:37.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.200+0000 7f5129e21740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f511c204690 con 0x561e074ca390 2024-09-21T07:53:37.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.200+0000 7f5129e21740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f511c204d60 con 0x561e074a14d0 2024-09-21T07:53:37.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.200+0000 7f511b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e074a96b0 0x561e07888f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:37.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.200+0000 7f511b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e074a96b0 0x561e07888f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60844/0 (socket says 172.21.15.39:60844) 2024-09-21T07:53:37.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.200+0000 7f511b400640 1 -- 172.21.15.39:0/2723199833 learned_addr learned my addr 172.21.15.39:0/2723199833 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:37.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.201+0000 7f511be00640 1 --2- 172.21.15.39:0/2723199833 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561e074a14d0 0x561e078d9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:37.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.201+0000 7f511b400640 1 -- 172.21.15.39:0/2723199833 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561e074a14d0 msgr2=0x561e078d9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:53:37.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.201+0000 7f511b400640 1 --2- 172.21.15.39:0/2723199833 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561e074a14d0 0x561e078d9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.201+0000 7f511b400640 1 -- 172.21.15.39:0/2723199833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e074ca390 msgr2=0x561e07872380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:53:37.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.201+0000 7f511b400640 1 --2- 172.21.15.39:0/2723199833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e074ca390 0x561e07872380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.201+0000 7f511b400640 1 -- 172.21.15.39:0/2723199833 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e0786dfa0 con 0x561e074a96b0 2024-09-21T07:53:37.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.201+0000 7f511b400640 1 --2- 172.21.15.39:0/2723199833 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e074a96b0 0x561e07888f90 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 crypto rx=0x7f5108062950 tx=0x7f510800b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5cb19a97b17bc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:37.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.201+0000 7f5107400640 1 -- 172.21.15.39:0/2723199833 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510804e070 con 0x561e074a96b0 2024-09-21T07:53:37.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.201+0000 7f5107400640 1 -- 172.21.15.39:0/2723199833 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f510805e5a0 con 0x561e074a96b0 2024-09-21T07:53:37.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.201+0000 7f5107400640 1 -- 172.21.15.39:0/2723199833 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5108067100 con 0x561e074a96b0 2024-09-21T07:53:37.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.202+0000 7f5129e21740 1 -- 172.21.15.39:0/2723199833 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e074a96b0 msgr2=0x561e07888f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:37.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.202+0000 7f5129e21740 1 --2- 172.21.15.39:0/2723199833 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e074a96b0 0x561e07888f90 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 crypto rx=0x7f5108062950 tx=0x7f510800b4c0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.202+0000 7f5129e21740 1 -- 172.21.15.39:0/2723199833 shutdown_connections 2024-09-21T07:53:37.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.202+0000 7f5129e21740 1 --2- 172.21.15.39:0/2723199833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e074ca390 0x561e07872380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.202+0000 7f5129e21740 1 --2- 172.21.15.39:0/2723199833 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e074a96b0 0x561e07888f90 unknown :-1 s=CLOSED pgs=2646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.202+0000 7f5129e21740 1 --2- 172.21.15.39:0/2723199833 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561e074a14d0 0x561e078d9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.202+0000 7f5129e21740 1 -- 172.21.15.39:0/2723199833 >> 172.21.15.39:0/2723199833 conn(0x561e074be4e0 msgr2=0x561e078c7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:37.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.202+0000 7f5129e21740 1 -- 172.21.15.39:0/2723199833 shutdown_connections 2024-09-21T07:53:37.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.202+0000 7f5129e21740 1 -- 172.21.15.39:0/2723199833 wait complete. 2024-09-21T07:53:37.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.203+0000 7f5129e21740 1 -- start start 2024-09-21T07:53:37.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.203+0000 7f5129e21740 1 Processor -- start 2024-09-21T07:53:37.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.205+0000 7f5129e21740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561e074a14d0 0x561e07889880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:37.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.205+0000 7f5129e21740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e074a96b0 0x561e078b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:37.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.205+0000 7f5129e21740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e074ca390 0x561e07872380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:37.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.205+0000 7f5129e21740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561e078b28c0 con 0x561e074a96b0 2024-09-21T07:53:37.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.205+0000 7f5129e21740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f511c1f6210 con 0x561e074ca390 2024-09-21T07:53:37.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.205+0000 7f5129e21740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x561e078b7860 con 0x561e074a14d0 2024-09-21T07:53:37.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.205+0000 7f5129e21740 10 client.0 _ll_register_callbacks cb 0x7f5108062630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:53:37.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.205+0000 7f511b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e074a96b0 0x561e078b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:37.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.205+0000 7f511b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e074a96b0 0x561e078b0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60856/0 (socket says 172.21.15.39:60856) 2024-09-21T07:53:37.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.205+0000 7f511b400640 1 -- 172.21.15.39:0/2718980267 learned_addr learned my addr 172.21.15.39:0/2718980267 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:37.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.205+0000 7f511be00640 1 --2- 172.21.15.39:0/2718980267 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561e074a14d0 0x561e07889880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:37.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.205+0000 7f5120c00640 1 --2- 172.21.15.39:0/2718980267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e074ca390 0x561e07872380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:37.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.205+0000 7f511b400640 1 -- 172.21.15.39:0/2718980267 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561e074a14d0 msgr2=0x561e07889880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:53:37.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.205+0000 7f511b400640 1 --2- 172.21.15.39:0/2718980267 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561e074a14d0 0x561e07889880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.205+0000 7f511b400640 1 -- 172.21.15.39:0/2718980267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e074ca390 msgr2=0x561e07872380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:53:37.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.206+0000 7f511b400640 1 --2- 172.21.15.39:0/2718980267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e074ca390 0x561e07872380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.206+0000 7f511b400640 1 -- 172.21.15.39:0/2718980267 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e07870910 con 0x561e074a96b0 2024-09-21T07:53:37.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.206+0000 7f511b400640 1 --2- 172.21.15.39:0/2718980267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e074a96b0 0x561e078b0ea0 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 crypto rx=0x7f510805fec0 tx=0x7f51080622d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e7a2f36957c94e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:37.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.206+0000 7f5104c00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:53:37.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.206+0000 7f5104c00640 1 -- 172.21.15.39:0/2718980267 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5108021070 con 0x561e074a96b0 2024-09-21T07:53:37.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.206+0000 7f5104c00640 1 -- 172.21.15.39:0/2718980267 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(250..250 src has 1..250) ==== 7820+0+0 (secure 0 0 0) 0x7f5108061920 con 0x561e074a96b0 2024-09-21T07:53:37.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.206+0000 7f5129e21740 1 -- 172.21.15.39:0/2718980267 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f510805e5a0 con 0x561e074a96b0 2024-09-21T07:53:37.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.206+0000 7f5104c00640 1 -- 172.21.15.39:0/2718980267 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=291) ==== 24+0+0 (secure 0 0 0) 0x7f510800de50 con 0x561e074a96b0 2024-09-21T07:53:37.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.206+0000 7f5129e21740 10 client.11898 fetch_fsmap learned FSMap version 291 2024-09-21T07:53:37.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.206+0000 7f5129e21740 1 -- 172.21.15.39:0/2718980267 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=291,monmap=2+}) -- 0x561e0788a450 con 0x561e074a96b0 2024-09-21T07:53:37.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.206+0000 7f5104c00640 1 -- 172.21.15.39:0/2718980267 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f510805e5a0 con 0x561e074a96b0 2024-09-21T07:53:37.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.207+0000 7f5104c00640 1 -- 172.21.15.39:0/2718980267 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 291) ==== 3542+0+0 (secure 0 0 0) 0x7f5108061b20 con 0x561e074a96b0 2024-09-21T07:53:37.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.207+0000 7f5129e21740 10 client.11898 fetch_fsmap finished waiting for FSMap version 291 2024-09-21T07:53:37.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.207+0000 7f5129e21740 10 client.11898 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.a 2024-09-21T07:53:37.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.207+0000 7f5129e21740 20 client.11898 populate_metadata read hostname 'smithi039' 2024-09-21T07:53:37.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.207+0000 7f5129e21740 1 --2- 172.21.15.39:0/2718980267 >> [v2:172.21.15.39:6832/2161818466,v1:172.21.15.39:6834/2161818466] conn(0x561e07935150 0x561e078728c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:37.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.207+0000 7f5129e21740 4 client.11898 mds_command: new command op to 11364 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:53:37.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.207+0000 7f5129e21740 1 -- 172.21.15.39:0/2718980267 --> [v2:172.21.15.39:6832/2161818466,v1:172.21.15.39:6834/2161818466] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561e078b5c00 con 0x561e07935150 2024-09-21T07:53:37.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.207+0000 7f511be00640 1 --2- 172.21.15.39:0/2718980267 >> [v2:172.21.15.39:6832/2161818466,v1:172.21.15.39:6834/2161818466] conn(0x561e07935150 0x561e078728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:37.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.207+0000 7f511be00640 1 --2- 172.21.15.39:0/2718980267 >> [v2:172.21.15.39:6832/2161818466,v1:172.21.15.39:6834/2161818466] conn(0x561e07935150 0x561e078728c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a7a80dfb75cf219 server_cookie=be0053d016febbdb in_seq=0 out_seq=0 2024-09-21T07:53:37.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.207+0000 7f5104c00640 10 client.11898 ms_handle_connect on v2:172.21.15.39:6832/2161818466 2024-09-21T07:53:37.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.219+0000 7f5104c00640 1 -- 172.21.15.39:0/2718980267 <== mds.0 v2:172.21.15.39:6832/2161818466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561e078b5c00 con 0x561e07935150 2024-09-21T07:53:37.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.219+0000 7f5104c00640 10 client.11898 handle_command_reply: tid=0 2024-09-21T07:53:37.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.219+0000 7f5129e21740 1 client.11898 shutdown 2024-09-21T07:53:37.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.219+0000 7f5129e21740 1 -- 172.21.15.39:0/2718980267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e074a96b0 msgr2=0x561e078b0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:37.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.219+0000 7f5129e21740 1 --2- 172.21.15.39:0/2718980267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e074a96b0 0x561e078b0ea0 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 crypto rx=0x7f510805fec0 tx=0x7f51080622d0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.220+0000 7f5129e21740 1 -- 172.21.15.39:0/2718980267 shutdown_connections 2024-09-21T07:53:37.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.220+0000 7f5129e21740 1 --2- 172.21.15.39:0/2718980267 >> [v2:172.21.15.39:6832/2161818466,v1:172.21.15.39:6834/2161818466] conn(0x561e07935150 0x561e078728c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.220+0000 7f5129e21740 1 --2- 172.21.15.39:0/2718980267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e074ca390 0x561e07872380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.220+0000 7f5129e21740 1 --2- 172.21.15.39:0/2718980267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561e074a96b0 0x561e078b0ea0 unknown :-1 s=CLOSED pgs=2647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.220+0000 7f5129e21740 1 --2- 172.21.15.39:0/2718980267 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x561e074a14d0 0x561e07889880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.220+0000 7f5104c00640 0 client.11898 ms_handle_reset on v2:172.21.15.39:6832/2161818466 2024-09-21T07:53:37.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.220+0000 7f5129e21740 1 -- 172.21.15.39:0/2718980267 >> 172.21.15.39:0/2718980267 conn(0x561e074be4e0 msgr2=0x561e078b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:37.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.220+0000 7f5129e21740 1 -- 172.21.15.39:0/2718980267 shutdown_connections 2024-09-21T07:53:37.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.220+0000 7f5129e21740 1 -- 172.21.15.39:0/2718980267 wait complete. 2024-09-21T07:53:37.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.220+0000 7f5129e21740 20 client.11898 trim_cache size 0 max 16384 2024-09-21T07:53:37.224 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:53:37.224 INFO:teuthology.orchestra.run.smithi039.stdout: "message": "", 2024-09-21T07:53:37.224 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0 2024-09-21T07:53:37.224 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:53:37.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.221+0000 7f5122c00640 1 -- 172.21.15.39:0/36260990 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f50fc078b90 msgr2=0x7f50fc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:37.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.221+0000 7f5122c00640 1 --2- 172.21.15.39:0/36260990 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f50fc078b90 0x7f50fc07b0a0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f510c034000 tx=0x7f510c004730 comp rx=0 tx=0).stop 2024-09-21T07:53:37.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.221+0000 7f5122c00640 1 -- 172.21.15.39:0/36260990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f511c15cf60 msgr2=0x7f511c15ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:37.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.221+0000 7f5122c00640 1 --2- 172.21.15.39:0/36260990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f511c15cf60 0x7f511c15ae00 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 crypto rx=0x7f510800d740 tx=0x7f510800dc00 comp rx=0 tx=0).stop 2024-09-21T07:53:37.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.221+0000 7f5122c00640 1 -- 172.21.15.39:0/36260990 shutdown_connections 2024-09-21T07:53:37.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.221+0000 7f5122c00640 1 --2- 172.21.15.39:0/36260990 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f50fc078b90 0x7f50fc07b0a0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.221+0000 7f5122c00640 1 --2- 172.21.15.39:0/36260990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f511c15d950 0x7f511c1f8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.221+0000 7f5122c00640 1 --2- 172.21.15.39:0/36260990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f511c15cf60 0x7f511c15ae00 unknown :-1 s=CLOSED pgs=2643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.221+0000 7f5122c00640 1 --2- 172.21.15.39:0/36260990 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f511c0cce70 0x7f511c15f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.221+0000 7f5122c00640 1 -- 172.21.15.39:0/36260990 >> 172.21.15.39:0/36260990 conn(0x7f511c06d440 msgr2=0x7f511c0b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:37.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.221+0000 7f5122c00640 1 -- 172.21.15.39:0/36260990 shutdown_connections 2024-09-21T07:53:37.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.221+0000 7f5122c00640 1 -- 172.21.15.39:0/36260990 wait complete. 2024-09-21T07:53:37.247 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:53:37.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.472+0000 7f4601200640 1 -- 172.21.15.39:0/827395032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45fc0b3f30 msgr2=0x7f45fc0b4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:37.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.472+0000 7f4601200640 1 --2- 172.21.15.39:0/827395032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45fc0b3f30 0x7f45fc0b4690 secure :-1 s=READY pgs=2648 cs=0 l=1 rev1=1 crypto rx=0x7f45fc0b5a30 tx=0x7f45f402a980 comp rx=0 tx=0).stop 2024-09-21T07:53:37.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.472+0000 7f4601200640 1 -- 172.21.15.39:0/827395032 shutdown_connections 2024-09-21T07:53:37.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.472+0000 7f4601200640 1 --2- 172.21.15.39:0/827395032 >> v2:172.21.15.148:3300/0 conn(0x7f45fc15bd70 0x7f45fc0b39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.472+0000 7f4601200640 1 --2- 172.21.15.39:0/827395032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45fc0b3f30 0x7f45fc0b4690 unknown :-1 s=CLOSED pgs=2648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.472+0000 7f4601200640 1 -- 172.21.15.39:0/827395032 >> 172.21.15.39:0/827395032 conn(0x7f45fc06cfc0 msgr2=0x7f45fc06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:37.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.472+0000 7f4601200640 1 -- 172.21.15.39:0/827395032 shutdown_connections 2024-09-21T07:53:37.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.472+0000 7f4601200640 1 -- 172.21.15.39:0/827395032 wait complete. 2024-09-21T07:53:37.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.475+0000 7f4601200640 1 Processor -- start 2024-09-21T07:53:37.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.475+0000 7f4601200640 1 -- start start 2024-09-21T07:53:37.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.476+0000 7f4601200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45fc0b3f30 0x7f45fc1458b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:37.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.476+0000 7f4601200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45fc15bd70 0x7f45fc145df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:37.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.476+0000 7f4601200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45fc15f360 0x7f45fc0b22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:37.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.476+0000 7f4601200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc0b4f60 con 0x7f45fc0b3f30 2024-09-21T07:53:37.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.476+0000 7f4601200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc0b4de0 con 0x7f45fc15bd70 2024-09-21T07:53:37.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.476+0000 7f4601200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f45fc0d0620 con 0x7f45fc15f360 2024-09-21T07:53:37.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.476+0000 7f45f9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45fc0b3f30 0x7f45fc1458b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:37.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.477+0000 7f45f9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45fc0b3f30 0x7f45fc1458b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60878/0 (socket says 172.21.15.39:60878) 2024-09-21T07:53:37.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.477+0000 7f45f9e00640 1 -- 172.21.15.39:0/3108896135 learned_addr learned my addr 172.21.15.39:0/3108896135 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:37.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.477+0000 7f45f3e00640 1 --2- 172.21.15.39:0/3108896135 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45fc15bd70 0x7f45fc145df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:37.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.477+0000 7f45fa800640 1 --2- 172.21.15.39:0/3108896135 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45fc15f360 0x7f45fc0b22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:37.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.477+0000 7f45f9e00640 1 -- 172.21.15.39:0/3108896135 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45fc15f360 msgr2=0x7f45fc0b22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:37.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.477+0000 7f45f9e00640 1 --2- 172.21.15.39:0/3108896135 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45fc15f360 0x7f45fc0b22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.477+0000 7f45f9e00640 1 -- 172.21.15.39:0/3108896135 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45fc15bd70 msgr2=0x7f45fc145df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:37.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.477+0000 7f45f9e00640 1 --2- 172.21.15.39:0/3108896135 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45fc15bd70 0x7f45fc145df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.477+0000 7f45f9e00640 1 -- 172.21.15.39:0/3108896135 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45fc14f800 con 0x7f45fc0b3f30 2024-09-21T07:53:37.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.477+0000 7f45f3e00640 1 --2- 172.21.15.39:0/3108896135 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45fc15bd70 0x7f45fc145df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:53:37.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.477+0000 7f45f9e00640 1 --2- 172.21.15.39:0/3108896135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45fc0b3f30 0x7f45fc1458b0 secure :-1 s=READY pgs=2649 cs=0 l=1 rev1=1 crypto rx=0x7f45ec00b3e0 tx=0x7f45ec00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:37.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.477+0000 7f45f2000640 1 -- 172.21.15.39:0/3108896135 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec015020 con 0x7f45fc0b3f30 2024-09-21T07:53:37.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.477+0000 7f45f2000640 1 -- 172.21.15.39:0/3108896135 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45ec011940 con 0x7f45fc0b3f30 2024-09-21T07:53:37.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.478+0000 7f4601200640 1 -- 172.21.15.39:0/3108896135 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45fc14fd00 con 0x7f45fc0b3f30 2024-09-21T07:53:37.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.478+0000 7f45f2000640 1 -- 172.21.15.39:0/3108896135 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec011d90 con 0x7f45fc0b3f30 2024-09-21T07:53:37.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.478+0000 7f4601200640 1 -- 172.21.15.39:0/3108896135 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45fc1502c0 con 0x7f45fc0b3f30 2024-09-21T07:53:37.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.479+0000 7f4601200640 1 -- 172.21.15.39:0/3108896135 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45fc076aa0 con 0x7f45fc0b3f30 2024-09-21T07:53:37.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.480+0000 7f45f2000640 1 -- 172.21.15.39:0/3108896135 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f45ec018020 con 0x7f45fc0b3f30 2024-09-21T07:53:37.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.481+0000 7f45f2000640 1 --2- 172.21.15.39:0/3108896135 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f45d8078b90 0x7f45d807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:37.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.481+0000 7f45f2000640 1 -- 172.21.15.39:0/3108896135 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(250..250 src has 1..250) ==== 7820+0+0 (secure 0 0 0) 0x7f45ec09b4f0 con 0x7f45fc0b3f30 2024-09-21T07:53:37.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.481+0000 7f45f3e00640 1 --2- 172.21.15.39:0/3108896135 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f45d8078b90 0x7f45d807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:37.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.482+0000 7f45f3e00640 1 --2- 172.21.15.39:0/3108896135 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f45d8078b90 0x7f45d807b0a0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f45fc159480 tx=0x7f45e4008040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:37.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.485+0000 7f45f2000640 1 -- 172.21.15.39:0/3108896135 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f45fc076aa0 con 0x7f45fc0b3f30 2024-09-21T07:53:37.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.720+0000 7f4601200640 1 -- 172.21.15.39:0/3108896135 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45fc0b4690 con 0x7f45fc0b3f30 2024-09-21T07:53:37.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.721+0000 7f45f2000640 1 -- 172.21.15.39:0/3108896135 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 291 v291) ==== 95+0+5291 (secure 0 0 0) 0x7f45ec01f020 con 0x7f45fc0b3f30 2024-09-21T07:53:37.723 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:53:37.723 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":291,"btime":"2024-09-21T07:53:19:187136+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10633,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3495264983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3495264983},{"type":"v1","addr":"172.21.15.148:6837","nonce":3495264983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":288}],"filesystems":[{"mdsmap":{"epoch":291,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:53:19.187128+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":249,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":11364,"mds_1":11052,"mds_2":10621},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10621":{"gid":10621,"name":"b","rank":2,"incarnation":287,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6835/2191156088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2191156088},{"type":"v1","addr":"172.21.15.148:6835","nonce":2191156088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11364":{"gid":11364,"name":"a","rank":0,"incarnation":285,"state":"up:active","state_seq":28,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11364,"qdb_cluster":[10621,11052,11364]},"id":13}]} 2024-09-21T07:53:37.723 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 291 2024-09-21T07:53:37.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.723+0000 7f4601200640 1 -- 172.21.15.39:0/3108896135 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f45d8078b90 msgr2=0x7f45d807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:37.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.723+0000 7f4601200640 1 --2- 172.21.15.39:0/3108896135 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f45d8078b90 0x7f45d807b0a0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f45fc159480 tx=0x7f45e4008040 comp rx=0 tx=0).stop 2024-09-21T07:53:37.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.723+0000 7f4601200640 1 -- 172.21.15.39:0/3108896135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45fc0b3f30 msgr2=0x7f45fc1458b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:37.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.723+0000 7f4601200640 1 --2- 172.21.15.39:0/3108896135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45fc0b3f30 0x7f45fc1458b0 secure :-1 s=READY pgs=2649 cs=0 l=1 rev1=1 crypto rx=0x7f45ec00b3e0 tx=0x7f45ec00b8a0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.724+0000 7f4601200640 1 -- 172.21.15.39:0/3108896135 shutdown_connections 2024-09-21T07:53:37.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.724+0000 7f4601200640 1 --2- 172.21.15.39:0/3108896135 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f45d8078b90 0x7f45d807b0a0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.724+0000 7f4601200640 1 --2- 172.21.15.39:0/3108896135 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f45fc15f360 0x7f45fc0b22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.724+0000 7f4601200640 1 --2- 172.21.15.39:0/3108896135 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f45fc15bd70 0x7f45fc145df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.724+0000 7f4601200640 1 --2- 172.21.15.39:0/3108896135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45fc0b3f30 0x7f45fc1458b0 unknown :-1 s=CLOSED pgs=2649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.724+0000 7f4601200640 1 -- 172.21.15.39:0/3108896135 >> 172.21.15.39:0/3108896135 conn(0x7f45fc06cfc0 msgr2=0x7f45fc1583b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:37.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.724+0000 7f4601200640 1 -- 172.21.15.39:0/3108896135 shutdown_connections 2024-09-21T07:53:37.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.724+0000 7f4601200640 1 -- 172.21.15.39:0/3108896135 wait complete. 2024-09-21T07:53:37.747 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 13:2 true 2024-09-21T07:53:37.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.974+0000 7f3096c00640 1 -- 172.21.15.39:0/792377537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30900b32b0 msgr2=0x7f30900cbb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:37.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.974+0000 7f3096c00640 1 --2- 172.21.15.39:0/792377537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30900b32b0 0x7f30900cbb10 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 crypto rx=0x7f30900cb650 tx=0x7f307802a560 comp rx=0 tx=0).stop 2024-09-21T07:53:37.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.975+0000 7f3096c00640 1 -- 172.21.15.39:0/792377537 shutdown_connections 2024-09-21T07:53:37.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.975+0000 7f3096c00640 1 --2- 172.21.15.39:0/792377537 >> v2:172.21.15.148:3300/0 conn(0x7f30900cce70 0x7f30900cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.975+0000 7f3096c00640 1 --2- 172.21.15.39:0/792377537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30900b32b0 0x7f30900cbb10 unknown :-1 s=CLOSED pgs=2651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.975+0000 7f3096c00640 1 -- 172.21.15.39:0/792377537 >> 172.21.15.39:0/792377537 conn(0x7f309006d440 msgr2=0x7f30900b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:37.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.975+0000 7f3096c00640 1 -- 172.21.15.39:0/792377537 shutdown_connections 2024-09-21T07:53:37.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.975+0000 7f3096c00640 1 -- 172.21.15.39:0/792377537 wait complete. 2024-09-21T07:53:37.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.977+0000 7f3096c00640 1 Processor -- start 2024-09-21T07:53:37.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.977+0000 7f3096c00640 1 -- start start 2024-09-21T07:53:37.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.979+0000 7f3096c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30900b32b0 0x7f30901f8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:37.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.979+0000 7f3096c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f30900cce70 0x7f30900ce190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:37.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.979+0000 7f3096c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f309015cf60 0x7f30900ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:37.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.979+0000 7f3096c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f309006f9e0 con 0x7f309015cf60 2024-09-21T07:53:37.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.979+0000 7f3096c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f309006f860 con 0x7f30900b32b0 2024-09-21T07:53:37.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.979+0000 7f3096c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f309006fb60 con 0x7f30900cce70 2024-09-21T07:53:37.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.979+0000 7f3094c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f309015cf60 0x7f30900ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:37.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.979+0000 7f3094c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f309015cf60 0x7f30900ce6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60892/0 (socket says 172.21.15.39:60892) 2024-09-21T07:53:37.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.979+0000 7f3094c00640 1 -- 172.21.15.39:0/3956055013 learned_addr learned my addr 172.21.15.39:0/3956055013 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:37.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.979+0000 7f308fe00640 1 --2- 172.21.15.39:0/3956055013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30900b32b0 0x7f30901f8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:37.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.979+0000 7f3087400640 1 --2- 172.21.15.39:0/3956055013 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f30900cce70 0x7f30900ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:37.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.979+0000 7f3094c00640 1 -- 172.21.15.39:0/3956055013 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f30900cce70 msgr2=0x7f30900ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:37.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.979+0000 7f3094c00640 1 --2- 172.21.15.39:0/3956055013 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f30900cce70 0x7f30900ce190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.979+0000 7f3094c00640 1 -- 172.21.15.39:0/3956055013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30900b32b0 msgr2=0x7f30901f8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:37.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.979+0000 7f3094c00640 1 --2- 172.21.15.39:0/3956055013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30900b32b0 0x7f30901f8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.979+0000 7f3094c00640 1 -- 172.21.15.39:0/3956055013 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30900c9520 con 0x7f309015cf60 2024-09-21T07:53:37.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.980+0000 7f3094c00640 1 --2- 172.21.15.39:0/3956055013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f309015cf60 0x7f30900ce6d0 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 crypto rx=0x7f308800cb00 tx=0x7f308800a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:37.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.980+0000 7f308cc00640 1 -- 172.21.15.39:0/3956055013 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308800f450 con 0x7f309015cf60 2024-09-21T07:53:37.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.980+0000 7f308cc00640 1 -- 172.21.15.39:0/3956055013 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f308800f700 con 0x7f309015cf60 2024-09-21T07:53:37.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.980+0000 7f308cc00640 1 -- 172.21.15.39:0/3956055013 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308800fb50 con 0x7f309015cf60 2024-09-21T07:53:37.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.980+0000 7f3096c00640 1 -- 172.21.15.39:0/3956055013 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30900c9800 con 0x7f309015cf60 2024-09-21T07:53:37.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.980+0000 7f3096c00640 1 -- 172.21.15.39:0/3956055013 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3090158420 con 0x7f309015cf60 2024-09-21T07:53:37.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.981+0000 7f3096c00640 1 -- 172.21.15.39:0/3956055013 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3090070080 con 0x7f309015cf60 2024-09-21T07:53:37.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.982+0000 7f308cc00640 1 -- 172.21.15.39:0/3956055013 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f308800fcf0 con 0x7f309015cf60 2024-09-21T07:53:37.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.984+0000 7f308cc00640 1 --2- 172.21.15.39:0/3956055013 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f306c078b90 0x7f306c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:37.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.984+0000 7f308cc00640 1 -- 172.21.15.39:0/3956055013 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(250..250 src has 1..250) ==== 7820+0+0 (secure 0 0 0) 0x7f308809c140 con 0x7f309015cf60 2024-09-21T07:53:37.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.984+0000 7f308fe00640 1 --2- 172.21.15.39:0/3956055013 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f306c078b90 0x7f306c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:37.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.985+0000 7f308fe00640 1 --2- 172.21.15.39:0/3956055013 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f306c078b90 0x7f306c07b0a0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f3080007fd0 tx=0x7f3080009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:37.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:37.989+0000 7f308cc00640 1 -- 172.21.15.39:0/3956055013 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3088014030 con 0x7f309015cf60 2024-09-21T07:53:38.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.263+0000 7f3096c00640 1 -- 172.21.15.39:0/3956055013 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "13:2", "val": "true"} v 0) -- 0x7f30900753a0 con 0x7f309015cf60 2024-09-21T07:53:38.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.296+0000 7f308cc00640 1 -- 172.21.15.39:0/3956055013 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "13:2", "val": "true"}]=0 freezing mds.10621 v292) ==== 114+0+0 (secure 0 0 0) 0x7f308809b0a0 con 0x7f309015cf60 2024-09-21T07:53:38.298 INFO:teuthology.orchestra.run.smithi039.stderr:freezing mds.10621 2024-09-21T07:53:38.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.299+0000 7f3096c00640 1 -- 172.21.15.39:0/3956055013 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f306c078b90 msgr2=0x7f306c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:38.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.299+0000 7f3096c00640 1 --2- 172.21.15.39:0/3956055013 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f306c078b90 0x7f306c07b0a0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f3080007fd0 tx=0x7f3080009040 comp rx=0 tx=0).stop 2024-09-21T07:53:38.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.299+0000 7f3096c00640 1 -- 172.21.15.39:0/3956055013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f309015cf60 msgr2=0x7f30900ce6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:38.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.299+0000 7f3096c00640 1 --2- 172.21.15.39:0/3956055013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f309015cf60 0x7f30900ce6d0 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 crypto rx=0x7f308800cb00 tx=0x7f308800a250 comp rx=0 tx=0).stop 2024-09-21T07:53:38.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.299+0000 7f3096c00640 1 -- 172.21.15.39:0/3956055013 shutdown_connections 2024-09-21T07:53:38.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.299+0000 7f3096c00640 1 --2- 172.21.15.39:0/3956055013 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f306c078b90 0x7f306c07b0a0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:38.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.299+0000 7f3096c00640 1 --2- 172.21.15.39:0/3956055013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f309015cf60 0x7f30900ce6d0 unknown :-1 s=CLOSED pgs=2652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:38.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.299+0000 7f3096c00640 1 --2- 172.21.15.39:0/3956055013 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f30900cce70 0x7f30900ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:38.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.299+0000 7f3096c00640 1 --2- 172.21.15.39:0/3956055013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30900b32b0 0x7f30901f8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:38.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.299+0000 7f3096c00640 1 -- 172.21.15.39:0/3956055013 >> 172.21.15.39:0/3956055013 conn(0x7f309006d440 msgr2=0x7f30900d03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:38.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.299+0000 7f3096c00640 1 -- 172.21.15.39:0/3956055013 shutdown_connections 2024-09-21T07:53:38.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.299+0000 7f3096c00640 1 -- 172.21.15.39:0/3956055013 wait complete. 2024-09-21T07:53:38.322 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json config set mds_kill_mdstable_at 4 2024-09-21T07:53:38.567 INFO:teuthology.orchestra.run.smithi148.stdout:{"success":"mds_kill_mdstable_at = '' (not observed, change may require restart) "} 2024-09-21T07:53:38.567 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_kill_mdstable_at = '' (not observed, change may require restart) " } 2024-09-21T07:53:38.568 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:53:38.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.794+0000 7efd61400640 1 -- 172.21.15.39:0/556925301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd5c0b3c80 msgr2=0x7efd5c15fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:38.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.794+0000 7efd61400640 1 --2- 172.21.15.39:0/556925301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd5c0b3c80 0x7efd5c15fa50 secure :-1 s=READY pgs=2653 cs=0 l=1 rev1=1 crypto rx=0x7efd5c072a50 tx=0x7efd4402a5f0 comp rx=0 tx=0).stop 2024-09-21T07:53:38.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.794+0000 7efd61400640 1 -- 172.21.15.39:0/556925301 shutdown_connections 2024-09-21T07:53:38.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.794+0000 7efd61400640 1 --2- 172.21.15.39:0/556925301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd5c0b3c80 0x7efd5c15fa50 unknown :-1 s=CLOSED pgs=2653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:38.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.794+0000 7efd61400640 1 --2- 172.21.15.39:0/556925301 >> v2:172.21.15.148:3300/0 conn(0x7efd5c0b4cc0 0x7efd5c0b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:38.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.794+0000 7efd61400640 1 -- 172.21.15.39:0/556925301 >> 172.21.15.39:0/556925301 conn(0x7efd5c0b2510 msgr2=0x7efd5c0d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:38.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.794+0000 7efd61400640 1 -- 172.21.15.39:0/556925301 shutdown_connections 2024-09-21T07:53:38.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.794+0000 7efd61400640 1 -- 172.21.15.39:0/556925301 wait complete. 2024-09-21T07:53:38.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.796+0000 7efd61400640 1 Processor -- start 2024-09-21T07:53:38.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.797+0000 7efd61400640 1 -- start start 2024-09-21T07:53:38.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.798+0000 7efd61400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efd5c0b3c80 0x7efd5c0c3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:38.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.798+0000 7efd61400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd5c0b4cc0 0x7efd5c1f60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:38.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.798+0000 7efd61400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd5c0cc180 0x7efd5c15dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:38.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.798+0000 7efd61400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c06f180 con 0x7efd5c0b4cc0 2024-09-21T07:53:38.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.798+0000 7efd61400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c06f000 con 0x7efd5c0cc180 2024-09-21T07:53:38.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.798+0000 7efd61400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7efd5c06f300 con 0x7efd5c0b3c80 2024-09-21T07:53:38.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.798+0000 7efd53400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd5c0b4cc0 0x7efd5c1f60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:38.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.798+0000 7efd53400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd5c0b4cc0 0x7efd5c1f60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60916/0 (socket says 172.21.15.39:60916) 2024-09-21T07:53:38.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.798+0000 7efd53400640 1 -- 172.21.15.39:0/2601533860 learned_addr learned my addr 172.21.15.39:0/2601533860 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:38.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.798+0000 7efd5be00640 1 --2- 172.21.15.39:0/2601533860 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efd5c0b3c80 0x7efd5c0c3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:38.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.798+0000 7efd60a00640 1 --2- 172.21.15.39:0/2601533860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd5c0cc180 0x7efd5c15dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:38.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.799+0000 7efd53400640 1 -- 172.21.15.39:0/2601533860 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efd5c0b3c80 msgr2=0x7efd5c0c3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:38.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.799+0000 7efd53400640 1 --2- 172.21.15.39:0/2601533860 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efd5c0b3c80 0x7efd5c0c3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:38.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.799+0000 7efd53400640 1 -- 172.21.15.39:0/2601533860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd5c0cc180 msgr2=0x7efd5c15dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:38.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.799+0000 7efd53400640 1 --2- 172.21.15.39:0/2601533860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd5c0cc180 0x7efd5c15dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:38.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.799+0000 7efd53400640 1 -- 172.21.15.39:0/2601533860 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd5c0c4310 con 0x7efd5c0b4cc0 2024-09-21T07:53:38.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.799+0000 7efd53400640 1 --2- 172.21.15.39:0/2601533860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd5c0b4cc0 0x7efd5c1f60c0 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 crypto rx=0x7efd44034040 tx=0x7efd44003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:38.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.799+0000 7efd58c00640 1 -- 172.21.15.39:0/2601533860 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4403e030 con 0x7efd5c0b4cc0 2024-09-21T07:53:38.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.799+0000 7efd58c00640 1 -- 172.21.15.39:0/2601533860 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd44003d10 con 0x7efd5c0b4cc0 2024-09-21T07:53:38.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.799+0000 7efd61400640 1 -- 172.21.15.39:0/2601533860 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd5c0c45f0 con 0x7efd5c0b4cc0 2024-09-21T07:53:38.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.800+0000 7efd58c00640 1 -- 172.21.15.39:0/2601533860 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd44035040 con 0x7efd5c0b4cc0 2024-09-21T07:53:38.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.800+0000 7efd61400640 1 -- 172.21.15.39:0/2601533860 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd5c0c4bb0 con 0x7efd5c0b4cc0 2024-09-21T07:53:38.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.801+0000 7efd61400640 1 -- 172.21.15.39:0/2601533860 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd5c15efa0 con 0x7efd5c0b4cc0 2024-09-21T07:53:38.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.802+0000 7efd58c00640 1 -- 172.21.15.39:0/2601533860 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7efd44041030 con 0x7efd5c0b4cc0 2024-09-21T07:53:38.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.803+0000 7efd58c00640 1 --2- 172.21.15.39:0/2601533860 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efd38078bc0 0x7efd3807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:38.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.804+0000 7efd58c00640 1 -- 172.21.15.39:0/2601533860 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(250..250 src has 1..250) ==== 7820+0+0 (secure 0 0 0) 0x7efd440bbc70 con 0x7efd5c0b4cc0 2024-09-21T07:53:38.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.804+0000 7efd5be00640 1 --2- 172.21.15.39:0/2601533860 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efd38078bc0 0x7efd3807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:38.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.804+0000 7efd5be00640 1 --2- 172.21.15.39:0/2601533860 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efd38078bc0 0x7efd3807b0d0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7efd4c009b30 tx=0x7efd4c005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:38.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:38.808+0000 7efd58c00640 1 -- 172.21.15.39:0/2601533860 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7efd44083210 con 0x7efd5c0b4cc0 2024-09-21T07:53:39.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.042+0000 7efd61400640 1 -- 172.21.15.39:0/2601533860 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd5c0b3740 con 0x7efd5c0b4cc0 2024-09-21T07:53:39.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.042+0000 7efd58c00640 1 -- 172.21.15.39:0/2601533860 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 292 v292) ==== 95+0+5291 (secure 0 0 0) 0x7efd44087c40 con 0x7efd5c0b4cc0 2024-09-21T07:53:39.044 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:53:39.045 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":292,"btime":"2024-09-21T07:53:38:290402+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10633,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3495264983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3495264983},{"type":"v1","addr":"172.21.15.148:6837","nonce":3495264983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":288}],"filesystems":[{"mdsmap":{"epoch":292,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:53:38.264700+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":249,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":11364,"mds_1":11052,"mds_2":10621},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10621":{"gid":10621,"name":"b","rank":2,"incarnation":287,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6835/2191156088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2191156088},{"type":"v1","addr":"172.21.15.148:6835","nonce":2191156088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11364":{"gid":11364,"name":"a","rank":0,"incarnation":285,"state":"up:active","state_seq":28,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11364,"qdb_cluster":[11364,11052,10621]},"id":13}]} 2024-09-21T07:53:39.045 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 292 2024-09-21T07:53:39.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.045+0000 7efd61400640 1 -- 172.21.15.39:0/2601533860 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efd38078bc0 msgr2=0x7efd3807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:39.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.045+0000 7efd61400640 1 --2- 172.21.15.39:0/2601533860 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efd38078bc0 0x7efd3807b0d0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7efd4c009b30 tx=0x7efd4c005f90 comp rx=0 tx=0).stop 2024-09-21T07:53:39.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.045+0000 7efd61400640 1 -- 172.21.15.39:0/2601533860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd5c0b4cc0 msgr2=0x7efd5c1f60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:39.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.045+0000 7efd61400640 1 --2- 172.21.15.39:0/2601533860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd5c0b4cc0 0x7efd5c1f60c0 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 crypto rx=0x7efd44034040 tx=0x7efd44003820 comp rx=0 tx=0).stop 2024-09-21T07:53:39.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.045+0000 7efd61400640 1 -- 172.21.15.39:0/2601533860 shutdown_connections 2024-09-21T07:53:39.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.045+0000 7efd61400640 1 --2- 172.21.15.39:0/2601533860 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7efd38078bc0 0x7efd3807b0d0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:39.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.045+0000 7efd61400640 1 --2- 172.21.15.39:0/2601533860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd5c0cc180 0x7efd5c15dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:39.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.045+0000 7efd61400640 1 --2- 172.21.15.39:0/2601533860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efd5c0b4cc0 0x7efd5c1f60c0 unknown :-1 s=CLOSED pgs=2654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:39.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.045+0000 7efd61400640 1 --2- 172.21.15.39:0/2601533860 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7efd5c0b3c80 0x7efd5c0c3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:39.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.045+0000 7efd61400640 1 -- 172.21.15.39:0/2601533860 >> 172.21.15.39:0/2601533860 conn(0x7efd5c0b2510 msgr2=0x7efd5c0ad0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:39.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.045+0000 7efd61400640 1 -- 172.21.15.39:0/2601533860 shutdown_connections 2024-09-21T07:53:39.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.045+0000 7efd61400640 1 -- 172.21.15.39:0/2601533860 wait complete. 2024-09-21T07:53:39.069 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump snaps --server 2024-09-21T07:53:39.314 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_snap":9,"last_created":9,"last_destroyed":7,"pending_noop":[],"snaps":[{"snapid":8,"ino":1099511627780,"stamp":"2024-09-21T07:51:12.091986+0000","name":"s1","metadata":{}},{"snapid":9,"ino":1099511627780,"stamp":"2024-09-21T07:52:44.191763+0000","name":"s2","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2024-09-21T07:53:39.314 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 9, "last_destroyed": 7, "last_snap": 9, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627780, "metadata": {}, "name": "s1", "snapid": 8, "stamp": "2024-09-21T07:51:12.091986+0000" }, { "ino": 1099511627780, "metadata": {}, "name": "s2", "snapid": 9, "stamp": "2024-09-21T07:52:44.191763+0000" } ] } 2024-09-21T07:53:39.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:53:39.314 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d0/d2/dir/.snap/s3) 2024-09-21T07:53:39.318 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-09-21T07:53:39.361 INFO:tasks.ceph.mds.b.smithi148.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::commit(version_t, LogSegment*)' thread 7fbe2b600640 time 2024-09-21T07:53:39.360454+0000 2024-09-21T07:53:39.361 INFO:tasks.ceph.mds.b.smithi148.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: 195: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 4) 2024-09-21T07:53:39.361 INFO:tasks.ceph.mds.b.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:53:39.361 INFO:tasks.ceph.mds.b.smithi148.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fbe3a85654b] 2024-09-21T07:53:39.362 INFO:tasks.ceph.mds.b.smithi148.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7fbe3a856758] 2024-09-21T07:53:39.362 INFO:tasks.ceph.mds.b.smithi148.stderr: 3: (MDSTableClient::commit(unsigned long, LogSegment*)+0x333) [0x56405e7bc3d3] 2024-09-21T07:53:39.362 INFO:tasks.ceph.mds.b.smithi148.stderr: 4: (Server::_mksnap_finish(boost::intrusive_ptr const&, CInode*, SnapInfo&)+0x9f) [0x56405e56fac3] 2024-09-21T07:53:39.362 INFO:tasks.ceph.mds.b.smithi148.stderr: 5: ceph-mds(+0x221794) [0x56405e564794] 2024-09-21T07:53:39.362 INFO:tasks.ceph.mds.b.smithi148.stderr: 6: (MDSContext::complete(int)+0x7c) [0x56405e80a6d8] 2024-09-21T07:53:39.362 INFO:tasks.ceph.mds.b.smithi148.stderr: 7: (MDSIOContextBase::complete(int)+0x501) [0x56405e80ad57] 2024-09-21T07:53:39.362 INFO:tasks.ceph.mds.b.smithi148.stderr: 8: (MDSLogContextBase::complete(int)+0x46) [0x56405e80add4] 2024-09-21T07:53:39.362 INFO:tasks.ceph.mds.b.smithi148.stderr: 9: (Finisher::finisher_thread_entry()+0x495) [0x7fbe3a7de34b] 2024-09-21T07:53:39.362 INFO:tasks.ceph.mds.b.smithi148.stderr: 10: ceph-mds(+0x1052e1) [0x56405e4482e1] 2024-09-21T07:53:39.362 INFO:tasks.ceph.mds.b.smithi148.stderr: 11: (Thread::entry_wrapper()+0x43) [0x7fbe3a82fd57] 2024-09-21T07:53:39.362 INFO:tasks.ceph.mds.b.smithi148.stderr: 12: (Thread::_entry_func(void*)+0xd) [0x7fbe3a82fd73] 2024-09-21T07:53:39.362 INFO:tasks.ceph.mds.b.smithi148.stderr: 13: /lib64/libc.so.6(+0x89c52) [0x7fbe39e89c52] 2024-09-21T07:53:39.362 INFO:tasks.ceph.mds.b.smithi148.stderr: 14: /lib64/libc.so.6(+0x10ec80) [0x7fbe39f0ec80] 2024-09-21T07:53:39.363 INFO:tasks.ceph.mds.b.smithi148.stderr:*** Caught signal (Aborted) ** 2024-09-21T07:53:39.363 INFO:tasks.ceph.mds.b.smithi148.stderr: in thread 7fbe2b600640 thread_name:MR_Finisher 2024-09-21T07:53:39.363 INFO:tasks.ceph.mds.b.smithi148.stderr:2024-09-21T07:53:39.360+0000 7fbe2b600640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::commit(version_t, LogSegment*)' thread 7fbe2b600640 time 2024-09-21T07:53:39.360454+0000 2024-09-21T07:53:39.363 INFO:tasks.ceph.mds.b.smithi148.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: 195: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 4) 2024-09-21T07:53:39.363 INFO:tasks.ceph.mds.b.smithi148.stderr: 2024-09-21T07:53:39.363 INFO:tasks.ceph.mds.b.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:53:39.363 INFO:tasks.ceph.mds.b.smithi148.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fbe3a85654b] 2024-09-21T07:53:39.363 INFO:tasks.ceph.mds.b.smithi148.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7fbe3a856758] 2024-09-21T07:53:39.363 INFO:tasks.ceph.mds.b.smithi148.stderr: 3: (MDSTableClient::commit(unsigned long, LogSegment*)+0x333) [0x56405e7bc3d3] 2024-09-21T07:53:39.363 INFO:tasks.ceph.mds.b.smithi148.stderr: 4: (Server::_mksnap_finish(boost::intrusive_ptr const&, CInode*, SnapInfo&)+0x9f) [0x56405e56fac3] 2024-09-21T07:53:39.363 INFO:tasks.ceph.mds.b.smithi148.stderr: 5: ceph-mds(+0x221794) [0x56405e564794] 2024-09-21T07:53:39.363 INFO:tasks.ceph.mds.b.smithi148.stderr: 6: (MDSContext::complete(int)+0x7c) [0x56405e80a6d8] 2024-09-21T07:53:39.363 INFO:tasks.ceph.mds.b.smithi148.stderr: 7: (MDSIOContextBase::complete(int)+0x501) [0x56405e80ad57] 2024-09-21T07:53:39.363 INFO:tasks.ceph.mds.b.smithi148.stderr: 8: (MDSLogContextBase::complete(int)+0x46) [0x56405e80add4] 2024-09-21T07:53:39.363 INFO:tasks.ceph.mds.b.smithi148.stderr: 9: (Finisher::finisher_thread_entry()+0x495) [0x7fbe3a7de34b] 2024-09-21T07:53:39.364 INFO:tasks.ceph.mds.b.smithi148.stderr: 10: ceph-mds(+0x1052e1) [0x56405e4482e1] 2024-09-21T07:53:39.364 INFO:tasks.ceph.mds.b.smithi148.stderr: 11: (Thread::entry_wrapper()+0x43) [0x7fbe3a82fd57] 2024-09-21T07:53:39.364 INFO:tasks.ceph.mds.b.smithi148.stderr: 12: (Thread::_entry_func(void*)+0xd) [0x7fbe3a82fd73] 2024-09-21T07:53:39.364 INFO:tasks.ceph.mds.b.smithi148.stderr: 13: /lib64/libc.so.6(+0x89c52) [0x7fbe39e89c52] 2024-09-21T07:53:39.364 INFO:tasks.ceph.mds.b.smithi148.stderr: 14: /lib64/libc.so.6(+0x10ec80) [0x7fbe39f0ec80] 2024-09-21T07:53:39.364 INFO:tasks.ceph.mds.b.smithi148.stderr: 2024-09-21T07:53:39.365 INFO:tasks.ceph.mds.b.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:53:39.365 INFO:tasks.ceph.mds.b.smithi148.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fbe39e3e6f0] 2024-09-21T07:53:39.365 INFO:tasks.ceph.mds.b.smithi148.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7fbe39e8b99c] 2024-09-21T07:53:39.365 INFO:tasks.ceph.mds.b.smithi148.stderr: 3: raise() 2024-09-21T07:53:39.365 INFO:tasks.ceph.mds.b.smithi148.stderr: 4: abort() 2024-09-21T07:53:39.365 INFO:tasks.ceph.mds.b.smithi148.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7fbe3a856686] 2024-09-21T07:53:39.365 INFO:tasks.ceph.mds.b.smithi148.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7fbe3a856758] 2024-09-21T07:53:39.365 INFO:tasks.ceph.mds.b.smithi148.stderr: 7: (MDSTableClient::commit(unsigned long, LogSegment*)+0x333) [0x56405e7bc3d3] 2024-09-21T07:53:39.365 INFO:tasks.ceph.mds.b.smithi148.stderr: 8: (Server::_mksnap_finish(boost::intrusive_ptr const&, CInode*, SnapInfo&)+0x9f) [0x56405e56fac3] 2024-09-21T07:53:39.365 INFO:tasks.ceph.mds.b.smithi148.stderr: 9: ceph-mds(+0x221794) [0x56405e564794] 2024-09-21T07:53:39.365 INFO:tasks.ceph.mds.b.smithi148.stderr: 10: (MDSContext::complete(int)+0x7c) [0x56405e80a6d8] 2024-09-21T07:53:39.366 INFO:tasks.ceph.mds.b.smithi148.stderr: 11: (MDSIOContextBase::complete(int)+0x501) [0x56405e80ad57] 2024-09-21T07:53:39.366 INFO:tasks.ceph.mds.b.smithi148.stderr: 12: (MDSLogContextBase::complete(int)+0x46) [0x56405e80add4] 2024-09-21T07:53:39.366 INFO:tasks.ceph.mds.b.smithi148.stderr: 13: (Finisher::finisher_thread_entry()+0x495) [0x7fbe3a7de34b] 2024-09-21T07:53:39.366 INFO:tasks.ceph.mds.b.smithi148.stderr: 14: ceph-mds(+0x1052e1) [0x56405e4482e1] 2024-09-21T07:53:39.366 INFO:tasks.ceph.mds.b.smithi148.stderr: 15: (Thread::entry_wrapper()+0x43) [0x7fbe3a82fd57] 2024-09-21T07:53:39.366 INFO:tasks.ceph.mds.b.smithi148.stderr: 16: (Thread::_entry_func(void*)+0xd) [0x7fbe3a82fd73] 2024-09-21T07:53:39.366 INFO:tasks.ceph.mds.b.smithi148.stderr: 17: /lib64/libc.so.6(+0x89c52) [0x7fbe39e89c52] 2024-09-21T07:53:39.366 INFO:tasks.ceph.mds.b.smithi148.stderr: 18: /lib64/libc.so.6(+0x10ec80) [0x7fbe39f0ec80] 2024-09-21T07:53:39.366 INFO:tasks.ceph.mds.b.smithi148.stderr:2024-09-21T07:53:39.361+0000 7fbe2b600640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:53:39.366 INFO:tasks.ceph.mds.b.smithi148.stderr: in thread 7fbe2b600640 thread_name:MR_Finisher 2024-09-21T07:53:39.366 INFO:tasks.ceph.mds.b.smithi148.stderr: 2024-09-21T07:53:39.366 INFO:tasks.ceph.mds.b.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:53:39.366 INFO:tasks.ceph.mds.b.smithi148.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fbe39e3e6f0] 2024-09-21T07:53:39.366 INFO:tasks.ceph.mds.b.smithi148.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7fbe39e8b99c] 2024-09-21T07:53:39.366 INFO:tasks.ceph.mds.b.smithi148.stderr: 3: raise() 2024-09-21T07:53:39.367 INFO:tasks.ceph.mds.b.smithi148.stderr: 4: abort() 2024-09-21T07:53:39.367 INFO:tasks.ceph.mds.b.smithi148.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7fbe3a856686] 2024-09-21T07:53:39.367 INFO:tasks.ceph.mds.b.smithi148.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7fbe3a856758] 2024-09-21T07:53:39.367 INFO:tasks.ceph.mds.b.smithi148.stderr: 7: (MDSTableClient::commit(unsigned long, LogSegment*)+0x333) [0x56405e7bc3d3] 2024-09-21T07:53:39.367 INFO:tasks.ceph.mds.b.smithi148.stderr: 8: (Server::_mksnap_finish(boost::intrusive_ptr const&, CInode*, SnapInfo&)+0x9f) [0x56405e56fac3] 2024-09-21T07:53:39.367 INFO:tasks.ceph.mds.b.smithi148.stderr: 9: ceph-mds(+0x221794) [0x56405e564794] 2024-09-21T07:53:39.367 INFO:tasks.ceph.mds.b.smithi148.stderr: 10: (MDSContext::complete(int)+0x7c) [0x56405e80a6d8] 2024-09-21T07:53:39.367 INFO:tasks.ceph.mds.b.smithi148.stderr: 11: (MDSIOContextBase::complete(int)+0x501) [0x56405e80ad57] 2024-09-21T07:53:39.367 INFO:tasks.ceph.mds.b.smithi148.stderr: 12: (MDSLogContextBase::complete(int)+0x46) [0x56405e80add4] 2024-09-21T07:53:39.367 INFO:tasks.ceph.mds.b.smithi148.stderr: 13: (Finisher::finisher_thread_entry()+0x495) [0x7fbe3a7de34b] 2024-09-21T07:53:39.367 INFO:tasks.ceph.mds.b.smithi148.stderr: 14: ceph-mds(+0x1052e1) [0x56405e4482e1] 2024-09-21T07:53:39.367 INFO:tasks.ceph.mds.b.smithi148.stderr: 15: (Thread::entry_wrapper()+0x43) [0x7fbe3a82fd57] 2024-09-21T07:53:39.367 INFO:tasks.ceph.mds.b.smithi148.stderr: 16: (Thread::_entry_func(void*)+0xd) [0x7fbe3a82fd73] 2024-09-21T07:53:39.367 INFO:tasks.ceph.mds.b.smithi148.stderr: 17: /lib64/libc.so.6(+0x89c52) [0x7fbe39e89c52] 2024-09-21T07:53:39.368 INFO:tasks.ceph.mds.b.smithi148.stderr: 18: /lib64/libc.so.6(+0x10ec80) [0x7fbe39f0ec80] 2024-09-21T07:53:39.368 INFO:tasks.ceph.mds.b.smithi148.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:53:39.368 INFO:tasks.ceph.mds.b.smithi148.stderr: 2024-09-21T07:53:39.372 INFO:tasks.ceph.mds.b.smithi148.stderr: -1> 2024-09-21T07:53:39.360+0000 7fbe2b600640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::commit(version_t, LogSegment*)' thread 7fbe2b600640 time 2024-09-21T07:53:39.360454+0000 2024-09-21T07:53:39.372 INFO:tasks.ceph.mds.b.smithi148.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: 195: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 4) 2024-09-21T07:53:39.373 INFO:tasks.ceph.mds.b.smithi148.stderr: 2024-09-21T07:53:39.373 INFO:tasks.ceph.mds.b.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:53:39.373 INFO:tasks.ceph.mds.b.smithi148.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fbe3a85654b] 2024-09-21T07:53:39.373 INFO:tasks.ceph.mds.b.smithi148.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7fbe3a856758] 2024-09-21T07:53:39.373 INFO:tasks.ceph.mds.b.smithi148.stderr: 3: (MDSTableClient::commit(unsigned long, LogSegment*)+0x333) [0x56405e7bc3d3] 2024-09-21T07:53:39.373 INFO:tasks.ceph.mds.b.smithi148.stderr: 4: (Server::_mksnap_finish(boost::intrusive_ptr const&, CInode*, SnapInfo&)+0x9f) [0x56405e56fac3] 2024-09-21T07:53:39.373 INFO:tasks.ceph.mds.b.smithi148.stderr: 5: ceph-mds(+0x221794) [0x56405e564794] 2024-09-21T07:53:39.373 INFO:tasks.ceph.mds.b.smithi148.stderr: 6: (MDSContext::complete(int)+0x7c) [0x56405e80a6d8] 2024-09-21T07:53:39.373 INFO:tasks.ceph.mds.b.smithi148.stderr: 7: (MDSIOContextBase::complete(int)+0x501) [0x56405e80ad57] 2024-09-21T07:53:39.373 INFO:tasks.ceph.mds.b.smithi148.stderr: 8: (MDSLogContextBase::complete(int)+0x46) [0x56405e80add4] 2024-09-21T07:53:39.373 INFO:tasks.ceph.mds.b.smithi148.stderr: 9: (Finisher::finisher_thread_entry()+0x495) [0x7fbe3a7de34b] 2024-09-21T07:53:39.373 INFO:tasks.ceph.mds.b.smithi148.stderr: 10: ceph-mds(+0x1052e1) [0x56405e4482e1] 2024-09-21T07:53:39.373 INFO:tasks.ceph.mds.b.smithi148.stderr: 11: (Thread::entry_wrapper()+0x43) [0x7fbe3a82fd57] 2024-09-21T07:53:39.373 INFO:tasks.ceph.mds.b.smithi148.stderr: 12: (Thread::_entry_func(void*)+0xd) [0x7fbe3a82fd73] 2024-09-21T07:53:39.373 INFO:tasks.ceph.mds.b.smithi148.stderr: 13: /lib64/libc.so.6(+0x89c52) [0x7fbe39e89c52] 2024-09-21T07:53:39.374 INFO:tasks.ceph.mds.b.smithi148.stderr: 14: /lib64/libc.so.6(+0x10ec80) [0x7fbe39f0ec80] 2024-09-21T07:53:39.374 INFO:tasks.ceph.mds.b.smithi148.stderr: 2024-09-21T07:53:39.374 INFO:tasks.ceph.mds.b.smithi148.stderr: 0> 2024-09-21T07:53:39.361+0000 7fbe2b600640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:53:39.374 INFO:tasks.ceph.mds.b.smithi148.stderr: in thread 7fbe2b600640 thread_name:MR_Finisher 2024-09-21T07:53:39.374 INFO:tasks.ceph.mds.b.smithi148.stderr: 2024-09-21T07:53:39.374 INFO:tasks.ceph.mds.b.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:53:39.374 INFO:tasks.ceph.mds.b.smithi148.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fbe39e3e6f0] 2024-09-21T07:53:39.374 INFO:tasks.ceph.mds.b.smithi148.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7fbe39e8b99c] 2024-09-21T07:53:39.374 INFO:tasks.ceph.mds.b.smithi148.stderr: 3: raise() 2024-09-21T07:53:39.374 INFO:tasks.ceph.mds.b.smithi148.stderr: 4: abort() 2024-09-21T07:53:39.374 INFO:tasks.ceph.mds.b.smithi148.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7fbe3a856686] 2024-09-21T07:53:39.374 INFO:tasks.ceph.mds.b.smithi148.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7fbe3a856758] 2024-09-21T07:53:39.374 INFO:tasks.ceph.mds.b.smithi148.stderr: 7: (MDSTableClient::commit(unsigned long, LogSegment*)+0x333) [0x56405e7bc3d3] 2024-09-21T07:53:39.374 INFO:tasks.ceph.mds.b.smithi148.stderr: 8: (Server::_mksnap_finish(boost::intrusive_ptr const&, CInode*, SnapInfo&)+0x9f) [0x56405e56fac3] 2024-09-21T07:53:39.375 INFO:tasks.ceph.mds.b.smithi148.stderr: 9: ceph-mds(+0x221794) [0x56405e564794] 2024-09-21T07:53:39.375 INFO:tasks.ceph.mds.b.smithi148.stderr: 10: (MDSContext::complete(int)+0x7c) [0x56405e80a6d8] 2024-09-21T07:53:39.375 INFO:tasks.ceph.mds.b.smithi148.stderr: 11: (MDSIOContextBase::complete(int)+0x501) [0x56405e80ad57] 2024-09-21T07:53:39.375 INFO:tasks.ceph.mds.b.smithi148.stderr: 12: (MDSLogContextBase::complete(int)+0x46) [0x56405e80add4] 2024-09-21T07:53:39.375 INFO:tasks.ceph.mds.b.smithi148.stderr: 13: (Finisher::finisher_thread_entry()+0x495) [0x7fbe3a7de34b] 2024-09-21T07:53:39.375 INFO:tasks.ceph.mds.b.smithi148.stderr: 14: ceph-mds(+0x1052e1) [0x56405e4482e1] 2024-09-21T07:53:39.375 INFO:tasks.ceph.mds.b.smithi148.stderr: 15: (Thread::entry_wrapper()+0x43) [0x7fbe3a82fd57] 2024-09-21T07:53:39.375 INFO:tasks.ceph.mds.b.smithi148.stderr: 16: (Thread::_entry_func(void*)+0xd) [0x7fbe3a82fd73] 2024-09-21T07:53:39.375 INFO:tasks.ceph.mds.b.smithi148.stderr: 17: /lib64/libc.so.6(+0x89c52) [0x7fbe39e89c52] 2024-09-21T07:53:39.375 INFO:tasks.ceph.mds.b.smithi148.stderr: 18: /lib64/libc.so.6(+0x10ec80) [0x7fbe39f0ec80] 2024-09-21T07:53:39.375 INFO:tasks.ceph.mds.b.smithi148.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:53:39.375 INFO:tasks.ceph.mds.b.smithi148.stderr: 2024-09-21T07:53:39.379 INFO:tasks.ceph.mds.b.smithi148.stderr: -1> 2024-09-21T07:53:39.360+0000 7fbe2b600640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: In function 'void MDSTableClient::commit(version_t, LogSegment*)' thread 7fbe2b600640 time 2024-09-21T07:53:39.360454+0000 2024-09-21T07:53:39.380 INFO:tasks.ceph.mds.b.smithi148.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-5120-g18432218/rpm/el9/BUILD/ceph-19.3.0-5120-g18432218/src/mds/MDSTableClient.cc: 195: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 4) 2024-09-21T07:53:39.380 INFO:tasks.ceph.mds.b.smithi148.stderr: 2024-09-21T07:53:39.380 INFO:tasks.ceph.mds.b.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:53:39.380 INFO:tasks.ceph.mds.b.smithi148.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fbe3a85654b] 2024-09-21T07:53:39.380 INFO:tasks.ceph.mds.b.smithi148.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7fbe3a856758] 2024-09-21T07:53:39.380 INFO:tasks.ceph.mds.b.smithi148.stderr: 3: (MDSTableClient::commit(unsigned long, LogSegment*)+0x333) [0x56405e7bc3d3] 2024-09-21T07:53:39.380 INFO:tasks.ceph.mds.b.smithi148.stderr: 4: (Server::_mksnap_finish(boost::intrusive_ptr const&, CInode*, SnapInfo&)+0x9f) [0x56405e56fac3] 2024-09-21T07:53:39.380 INFO:tasks.ceph.mds.b.smithi148.stderr: 5: ceph-mds(+0x221794) [0x56405e564794] 2024-09-21T07:53:39.380 INFO:tasks.ceph.mds.b.smithi148.stderr: 6: (MDSContext::complete(int)+0x7c) [0x56405e80a6d8] 2024-09-21T07:53:39.380 INFO:tasks.ceph.mds.b.smithi148.stderr: 7: (MDSIOContextBase::complete(int)+0x501) [0x56405e80ad57] 2024-09-21T07:53:39.380 INFO:tasks.ceph.mds.b.smithi148.stderr: 8: (MDSLogContextBase::complete(int)+0x46) [0x56405e80add4] 2024-09-21T07:53:39.380 INFO:tasks.ceph.mds.b.smithi148.stderr: 9: (Finisher::finisher_thread_entry()+0x495) [0x7fbe3a7de34b] 2024-09-21T07:53:39.380 INFO:tasks.ceph.mds.b.smithi148.stderr: 10: ceph-mds(+0x1052e1) [0x56405e4482e1] 2024-09-21T07:53:39.380 INFO:tasks.ceph.mds.b.smithi148.stderr: 11: (Thread::entry_wrapper()+0x43) [0x7fbe3a82fd57] 2024-09-21T07:53:39.381 INFO:tasks.ceph.mds.b.smithi148.stderr: 12: (Thread::_entry_func(void*)+0xd) [0x7fbe3a82fd73] 2024-09-21T07:53:39.381 INFO:tasks.ceph.mds.b.smithi148.stderr: 13: /lib64/libc.so.6(+0x89c52) [0x7fbe39e89c52] 2024-09-21T07:53:39.381 INFO:tasks.ceph.mds.b.smithi148.stderr: 14: /lib64/libc.so.6(+0x10ec80) [0x7fbe39f0ec80] 2024-09-21T07:53:39.381 INFO:tasks.ceph.mds.b.smithi148.stderr: 2024-09-21T07:53:39.381 INFO:tasks.ceph.mds.b.smithi148.stderr: 0> 2024-09-21T07:53:39.361+0000 7fbe2b600640 -1 *** Caught signal (Aborted) ** 2024-09-21T07:53:39.381 INFO:tasks.ceph.mds.b.smithi148.stderr: in thread 7fbe2b600640 thread_name:MR_Finisher 2024-09-21T07:53:39.381 INFO:tasks.ceph.mds.b.smithi148.stderr: 2024-09-21T07:53:39.381 INFO:tasks.ceph.mds.b.smithi148.stderr: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T07:53:39.381 INFO:tasks.ceph.mds.b.smithi148.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fbe39e3e6f0] 2024-09-21T07:53:39.381 INFO:tasks.ceph.mds.b.smithi148.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7fbe39e8b99c] 2024-09-21T07:53:39.381 INFO:tasks.ceph.mds.b.smithi148.stderr: 3: raise() 2024-09-21T07:53:39.381 INFO:tasks.ceph.mds.b.smithi148.stderr: 4: abort() 2024-09-21T07:53:39.381 INFO:tasks.ceph.mds.b.smithi148.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x262) [0x7fbe3a856686] 2024-09-21T07:53:39.381 INFO:tasks.ceph.mds.b.smithi148.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x256758) [0x7fbe3a856758] 2024-09-21T07:53:39.382 INFO:tasks.ceph.mds.b.smithi148.stderr: 7: (MDSTableClient::commit(unsigned long, LogSegment*)+0x333) [0x56405e7bc3d3] 2024-09-21T07:53:39.382 INFO:tasks.ceph.mds.b.smithi148.stderr: 8: (Server::_mksnap_finish(boost::intrusive_ptr const&, CInode*, SnapInfo&)+0x9f) [0x56405e56fac3] 2024-09-21T07:53:39.382 INFO:tasks.ceph.mds.b.smithi148.stderr: 9: ceph-mds(+0x221794) [0x56405e564794] 2024-09-21T07:53:39.382 INFO:tasks.ceph.mds.b.smithi148.stderr: 10: (MDSContext::complete(int)+0x7c) [0x56405e80a6d8] 2024-09-21T07:53:39.382 INFO:tasks.ceph.mds.b.smithi148.stderr: 11: (MDSIOContextBase::complete(int)+0x501) [0x56405e80ad57] 2024-09-21T07:53:39.382 INFO:tasks.ceph.mds.b.smithi148.stderr: 12: (MDSLogContextBase::complete(int)+0x46) [0x56405e80add4] 2024-09-21T07:53:39.382 INFO:tasks.ceph.mds.b.smithi148.stderr: 13: (Finisher::finisher_thread_entry()+0x495) [0x7fbe3a7de34b] 2024-09-21T07:53:39.382 INFO:tasks.ceph.mds.b.smithi148.stderr: 14: ceph-mds(+0x1052e1) [0x56405e4482e1] 2024-09-21T07:53:39.382 INFO:tasks.ceph.mds.b.smithi148.stderr: 15: (Thread::entry_wrapper()+0x43) [0x7fbe3a82fd57] 2024-09-21T07:53:39.382 INFO:tasks.ceph.mds.b.smithi148.stderr: 16: (Thread::_entry_func(void*)+0xd) [0x7fbe3a82fd73] 2024-09-21T07:53:39.382 INFO:tasks.ceph.mds.b.smithi148.stderr: 17: /lib64/libc.so.6(+0x89c52) [0x7fbe39e89c52] 2024-09-21T07:53:39.382 INFO:tasks.ceph.mds.b.smithi148.stderr: 18: /lib64/libc.so.6(+0x10ec80) [0x7fbe39f0ec80] 2024-09-21T07:53:39.382 INFO:tasks.ceph.mds.b.smithi148.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-21T07:53:39.382 INFO:tasks.ceph.mds.b.smithi148.stderr: 2024-09-21T07:53:39.595 INFO:teuthology.orchestra.run.smithi039.stdout:{"mds_beacon_grace":"15.000000"} 2024-09-21T07:53:39.595 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-09-21T07:53:39.595 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:53:39.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.819+0000 7fd5c5000640 1 -- 172.21.15.39:0/395747047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5c00d0f20 msgr2=0x7fd5c015c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:39.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.819+0000 7fd5c5000640 1 --2- 172.21.15.39:0/395747047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5c00d0f20 0x7fd5c015c830 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0009830 tx=0x7fd5b002a510 comp rx=0 tx=0).stop 2024-09-21T07:53:39.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.819+0000 7fd5c5000640 1 -- 172.21.15.39:0/395747047 shutdown_connections 2024-09-21T07:53:39.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.819+0000 7fd5c5000640 1 --2- 172.21.15.39:0/395747047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5c00d0f20 0x7fd5c015c830 unknown :-1 s=CLOSED pgs=2656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:39.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.819+0000 7fd5c5000640 1 -- 172.21.15.39:0/395747047 >> 172.21.15.39:0/395747047 conn(0x7fd5c00b2510 msgr2=0x7fd5c00d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:39.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.819+0000 7fd5c5000640 1 -- 172.21.15.39:0/395747047 shutdown_connections 2024-09-21T07:53:39.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.819+0000 7fd5c5000640 1 -- 172.21.15.39:0/395747047 wait complete. 2024-09-21T07:53:39.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.821+0000 7fd5c5000640 1 Processor -- start 2024-09-21T07:53:39.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.822+0000 7fd5c5000640 1 -- start start 2024-09-21T07:53:39.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.823+0000 7fd5c5000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5c00cb960 0x7fd5c00ca150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:39.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.823+0000 7fd5c5000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5c00cbd70 0x7fd5c00c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:39.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.823+0000 7fd5c5000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5c00d0f20 0x7fd5c009d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:39.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.823+0000 7fd5c5000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c00b4780 con 0x7fd5c00d0f20 2024-09-21T07:53:39.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.823+0000 7fd5c5000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c00b4600 con 0x7fd5c00cbd70 2024-09-21T07:53:39.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.823+0000 7fd5c5000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd5c00b4900 con 0x7fd5c00cb960 2024-09-21T07:53:39.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.823+0000 7fd5bfe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5c00d0f20 0x7fd5c009d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:39.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.823+0000 7fd5bfe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5c00d0f20 0x7fd5c009d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60938/0 (socket says 172.21.15.39:60938) 2024-09-21T07:53:39.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.823+0000 7fd5bfe00640 1 -- 172.21.15.39:0/907389206 learned_addr learned my addr 172.21.15.39:0/907389206 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:39.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.824+0000 7fd5bf400640 1 --2- 172.21.15.39:0/907389206 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5c00cb960 0x7fd5c00ca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:39.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.824+0000 7fd5bfe00640 1 -- 172.21.15.39:0/907389206 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5c00cb960 msgr2=0x7fd5c00ca150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:39.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.824+0000 7fd5bfe00640 1 --2- 172.21.15.39:0/907389206 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5c00cb960 0x7fd5c00ca150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:39.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.824+0000 7fd5bfe00640 1 -- 172.21.15.39:0/907389206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5c00cbd70 msgr2=0x7fd5c00c59f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T07:53:39.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.824+0000 7fd5bfe00640 1 --2- 172.21.15.39:0/907389206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5c00cbd70 0x7fd5c00c59f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:39.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.824+0000 7fd5bfe00640 1 -- 172.21.15.39:0/907389206 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c00c3e10 con 0x7fd5c00d0f20 2024-09-21T07:53:39.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.824+0000 7fd5bfe00640 1 --2- 172.21.15.39:0/907389206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5c00d0f20 0x7fd5c009d750 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800b9c0 tx=0x7fd5b800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:39.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.824+0000 7fd5b7400640 1 -- 172.21.15.39:0/907389206 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8019070 con 0x7fd5c00d0f20 2024-09-21T07:53:39.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.824+0000 7fd5b7400640 1 -- 172.21.15.39:0/907389206 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd5b800cc90 con 0x7fd5c00d0f20 2024-09-21T07:53:39.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.824+0000 7fd5c5000640 1 -- 172.21.15.39:0/907389206 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5c01f5fc0 con 0x7fd5c00d0f20 2024-09-21T07:53:39.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.825+0000 7fd5b7400640 1 -- 172.21.15.39:0/907389206 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b80146a0 con 0x7fd5c00d0f20 2024-09-21T07:53:39.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.825+0000 7fd5c5000640 1 -- 172.21.15.39:0/907389206 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5c01f6580 con 0x7fd5c00d0f20 2024-09-21T07:53:39.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.826+0000 7fd5c5000640 1 -- 172.21.15.39:0/907389206 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5c01f5d10 con 0x7fd5c00d0f20 2024-09-21T07:53:39.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.827+0000 7fd5b7400640 1 -- 172.21.15.39:0/907389206 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd5b801c070 con 0x7fd5c00d0f20 2024-09-21T07:53:39.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.828+0000 7fd5b7400640 1 --2- 172.21.15.39:0/907389206 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd59c078b90 0x7fd59c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:39.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.828+0000 7fd5b7400640 1 -- 172.21.15.39:0/907389206 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(250..250 src has 1..250) ==== 7820+0+0 (secure 0 0 0) 0x7fd5b809b320 con 0x7fd5c00d0f20 2024-09-21T07:53:39.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.828+0000 7fd5bf400640 1 --2- 172.21.15.39:0/907389206 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd59c078b90 0x7fd59c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:39.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.829+0000 7fd5bf400640 1 --2- 172.21.15.39:0/907389206 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd59c078b90 0x7fd59c07b0a0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0034000 tx=0x7fd5b0004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:39.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:39.832+0000 7fd5b7400640 1 -- 172.21.15.39:0/907389206 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd5b8062830 con 0x7fd5c00d0f20 2024-09-21T07:53:39.977 INFO:tasks.ceph.mds.b.smithi148.stderr:daemon-helper: command crashed with signal 6 2024-09-21T07:53:40.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:40.070+0000 7fd5c5000640 1 -- 172.21.15.39:0/907389206 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5c015c830 con 0x7fd5c00d0f20 2024-09-21T07:53:40.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:40.071+0000 7fd5b7400640 1 -- 172.21.15.39:0/907389206 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 292 v292) ==== 95+0+5291 (secure 0 0 0) 0x7fd5b8067260 con 0x7fd5c00d0f20 2024-09-21T07:53:40.073 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:53:40.073 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":292,"btime":"2024-09-21T07:53:38:290402+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10633,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3495264983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3495264983},{"type":"v1","addr":"172.21.15.148:6837","nonce":3495264983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":288}],"filesystems":[{"mdsmap":{"epoch":292,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:53:38.264700+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":249,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":11364,"mds_1":11052,"mds_2":10621},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10621":{"gid":10621,"name":"b","rank":2,"incarnation":287,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6835/2191156088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2191156088},{"type":"v1","addr":"172.21.15.148:6835","nonce":2191156088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11364":{"gid":11364,"name":"a","rank":0,"incarnation":285,"state":"up:active","state_seq":28,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11364,"qdb_cluster":[11364,11052,10621]},"id":13}]} 2024-09-21T07:53:40.073 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 292 2024-09-21T07:53:40.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:40.073+0000 7fd5c5000640 1 -- 172.21.15.39:0/907389206 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd59c078b90 msgr2=0x7fd59c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:40.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:40.073+0000 7fd5c5000640 1 --2- 172.21.15.39:0/907389206 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd59c078b90 0x7fd59c07b0a0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0034000 tx=0x7fd5b0004730 comp rx=0 tx=0).stop 2024-09-21T07:53:40.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:40.073+0000 7fd5c5000640 1 -- 172.21.15.39:0/907389206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5c00d0f20 msgr2=0x7fd5c009d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:40.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:40.073+0000 7fd5c5000640 1 --2- 172.21.15.39:0/907389206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5c00d0f20 0x7fd5c009d750 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800b9c0 tx=0x7fd5b800be80 comp rx=0 tx=0).stop 2024-09-21T07:53:40.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:40.074+0000 7fd5c5000640 1 -- 172.21.15.39:0/907389206 shutdown_connections 2024-09-21T07:53:40.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:40.074+0000 7fd5c5000640 1 --2- 172.21.15.39:0/907389206 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd59c078b90 0x7fd59c07b0a0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:40.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:40.074+0000 7fd5c5000640 1 --2- 172.21.15.39:0/907389206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5c00d0f20 0x7fd5c009d750 unknown :-1 s=CLOSED pgs=2657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:40.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:40.074+0000 7fd5c5000640 1 --2- 172.21.15.39:0/907389206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5c00cbd70 0x7fd5c00c59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:40.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:40.074+0000 7fd5c5000640 1 --2- 172.21.15.39:0/907389206 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd5c00cb960 0x7fd5c00ca150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:40.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:40.074+0000 7fd5c5000640 1 -- 172.21.15.39:0/907389206 >> 172.21.15.39:0/907389206 conn(0x7fd5c00b2510 msgr2=0x7fd5c00742d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:40.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:40.074+0000 7fd5c5000640 1 -- 172.21.15.39:0/907389206 shutdown_connections 2024-09-21T07:53:40.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:40.074+0000 7fd5c5000640 1 -- 172.21.15.39:0/907389206 wait complete. 2024-09-21T07:53:40.100 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:53:45.102 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:53:45.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.337+0000 7f82b5a00640 1 -- 172.21.15.39:0/114346041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82b015f790 msgr2=0x7f82b00d06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:45.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.337+0000 7f82b5a00640 1 --2- 172.21.15.39:0/114346041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82b015f790 0x7f82b00d06e0 secure :-1 s=READY pgs=2658 cs=0 l=1 rev1=1 crypto rx=0x7f8298005fd0 tx=0x7f829802a530 comp rx=0 tx=0).stop 2024-09-21T07:53:45.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.337+0000 7f82b5a00640 1 -- 172.21.15.39:0/114346041 shutdown_connections 2024-09-21T07:53:45.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.337+0000 7f82b5a00640 1 --2- 172.21.15.39:0/114346041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82b015f790 0x7f82b00d06e0 unknown :-1 s=CLOSED pgs=2658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:45.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.337+0000 7f82b5a00640 1 --2- 172.21.15.39:0/114346041 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82b00b47c0 0x7f82b0155800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:45.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.337+0000 7f82b5a00640 1 -- 172.21.15.39:0/114346041 >> 172.21.15.39:0/114346041 conn(0x7f82b006cfc0 msgr2=0x7f82b006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:45.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.337+0000 7f82b5a00640 1 -- 172.21.15.39:0/114346041 shutdown_connections 2024-09-21T07:53:45.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.337+0000 7f82b5a00640 1 -- 172.21.15.39:0/114346041 wait complete. 2024-09-21T07:53:45.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.339+0000 7f82b5a00640 1 Processor -- start 2024-09-21T07:53:45.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.340+0000 7f82b5a00640 1 -- start start 2024-09-21T07:53:45.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.341+0000 7f82b5a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82b015f790 0x7f82b014ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:45.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.341+0000 7f82b5a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82b015fba0 0x7f82b0158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:45.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.341+0000 7f82b5a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82b015d510 0x7f82b015d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:45.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.341+0000 7f82b5a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f82b0162ce0 con 0x7f82b015f790 2024-09-21T07:53:45.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.341+0000 7f82b5a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f82b00d1080 con 0x7f82b015fba0 2024-09-21T07:53:45.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.341+0000 7f82b5a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f82b0162e60 con 0x7f82b015d510 2024-09-21T07:53:45.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.341+0000 7f82aea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82b015f790 0x7f82b014ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:45.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.341+0000 7f82aea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82b015f790 0x7f82b014ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54780/0 (socket says 172.21.15.39:54780) 2024-09-21T07:53:45.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.341+0000 7f82aea00640 1 -- 172.21.15.39:0/753946255 learned_addr learned my addr 172.21.15.39:0/753946255 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:45.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.342+0000 7f82aea00640 1 -- 172.21.15.39:0/753946255 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82b015d510 msgr2=0x7f82b015d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:45.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.342+0000 7f82af400640 1 --2- 172.21.15.39:0/753946255 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82b015d510 0x7f82b015d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:45.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.342+0000 7f82a6000640 1 --2- 172.21.15.39:0/753946255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82b015fba0 0x7f82b0158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:45.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.342+0000 7f82aea00640 1 --2- 172.21.15.39:0/753946255 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82b015d510 0x7f82b015d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:45.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.342+0000 7f82aea00640 1 -- 172.21.15.39:0/753946255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82b015fba0 msgr2=0x7f82b0158ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:45.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.342+0000 7f82aea00640 1 --2- 172.21.15.39:0/753946255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82b015fba0 0x7f82b0158ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:45.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.342+0000 7f82aea00640 1 -- 172.21.15.39:0/753946255 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82b0157160 con 0x7f82b015f790 2024-09-21T07:53:45.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.342+0000 7f82af400640 1 --2- 172.21.15.39:0/753946255 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82b015d510 0x7f82b015d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:53:45.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.342+0000 7f82a6000640 1 --2- 172.21.15.39:0/753946255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82b015fba0 0x7f82b0158ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:53:45.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.342+0000 7f82aea00640 1 --2- 172.21.15.39:0/753946255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82b015f790 0x7f82b014ad80 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 crypto rx=0x7f82a000d6d0 tx=0x7f82a000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:45.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.342+0000 7f82a7400640 1 -- 172.21.15.39:0/753946255 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a0017070 con 0x7f82b015f790 2024-09-21T07:53:45.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.342+0000 7f82a7400640 1 -- 172.21.15.39:0/753946255 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82a0005330 con 0x7f82b015f790 2024-09-21T07:53:45.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.342+0000 7f82a7400640 1 -- 172.21.15.39:0/753946255 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a0009820 con 0x7f82b015f790 2024-09-21T07:53:45.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.342+0000 7f82b5a00640 1 -- 172.21.15.39:0/753946255 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82b015de60 con 0x7f82b015f790 2024-09-21T07:53:45.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.343+0000 7f82b5a00640 1 -- 172.21.15.39:0/753946255 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82b01f92a0 con 0x7f82b015f790 2024-09-21T07:53:45.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.344+0000 7f82b5a00640 1 -- 172.21.15.39:0/753946255 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82b0163380 con 0x7f82b015f790 2024-09-21T07:53:45.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.345+0000 7f82a7400640 1 -- 172.21.15.39:0/753946255 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f82a0009bd0 con 0x7f82b015f790 2024-09-21T07:53:45.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.346+0000 7f82a7400640 1 --2- 172.21.15.39:0/753946255 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f828c078bc0 0x7f828c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:45.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.346+0000 7f82a7400640 1 -- 172.21.15.39:0/753946255 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(250..250 src has 1..250) ==== 7820+0+0 (secure 0 0 0) 0x7f82a009b980 con 0x7f82b015f790 2024-09-21T07:53:45.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.346+0000 7f82a6000640 1 --2- 172.21.15.39:0/753946255 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f828c078bc0 0x7f828c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:45.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.348+0000 7f82a6000640 1 --2- 172.21.15.39:0/753946255 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f828c078bc0 0x7f828c07b0d0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f8298033000 tx=0x7f8298004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:45.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.351+0000 7f82a7400640 1 -- 172.21.15.39:0/753946255 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f82b015de60 con 0x7f82b015f790 2024-09-21T07:53:45.374 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-09-21T07:53:45.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.586+0000 7f82b5a00640 1 -- 172.21.15.39:0/753946255 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82b00702a0 con 0x7f82b015f790 2024-09-21T07:53:45.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.587+0000 7f82a7400640 1 -- 172.21.15.39:0/753946255 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 292 v292) ==== 95+0+5291 (secure 0 0 0) 0x7f82a00634e0 con 0x7f82b015f790 2024-09-21T07:53:45.589 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:53:45.589 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":292,"btime":"2024-09-21T07:53:38:290402+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10633,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3495264983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3495264983},{"type":"v1","addr":"172.21.15.148:6837","nonce":3495264983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":288}],"filesystems":[{"mdsmap":{"epoch":292,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:53:38.264700+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":249,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":11364,"mds_1":11052,"mds_2":10621},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10621":{"gid":10621,"name":"b","rank":2,"incarnation":287,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6835/2191156088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2191156088},{"type":"v1","addr":"172.21.15.148:6835","nonce":2191156088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11364":{"gid":11364,"name":"a","rank":0,"incarnation":285,"state":"up:active","state_seq":28,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11364,"qdb_cluster":[11364,11052,10621]},"id":13}]} 2024-09-21T07:53:45.589 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 292 2024-09-21T07:53:45.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.589+0000 7f82b5a00640 1 -- 172.21.15.39:0/753946255 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f828c078bc0 msgr2=0x7f828c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:45.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.589+0000 7f82b5a00640 1 --2- 172.21.15.39:0/753946255 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f828c078bc0 0x7f828c07b0d0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f8298033000 tx=0x7f8298004730 comp rx=0 tx=0).stop 2024-09-21T07:53:45.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.589+0000 7f82b5a00640 1 -- 172.21.15.39:0/753946255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82b015f790 msgr2=0x7f82b014ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:45.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.589+0000 7f82b5a00640 1 --2- 172.21.15.39:0/753946255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82b015f790 0x7f82b014ad80 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 crypto rx=0x7f82a000d6d0 tx=0x7f82a000db90 comp rx=0 tx=0).stop 2024-09-21T07:53:45.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.589+0000 7f82b5a00640 1 -- 172.21.15.39:0/753946255 shutdown_connections 2024-09-21T07:53:45.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.589+0000 7f82b5a00640 1 --2- 172.21.15.39:0/753946255 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f828c078bc0 0x7f828c07b0d0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:45.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.589+0000 7f82b5a00640 1 --2- 172.21.15.39:0/753946255 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82b015d510 0x7f82b015d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:45.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.589+0000 7f82b5a00640 1 --2- 172.21.15.39:0/753946255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82b015fba0 0x7f82b0158ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:45.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.589+0000 7f82b5a00640 1 --2- 172.21.15.39:0/753946255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82b015f790 0x7f82b014ad80 unknown :-1 s=CLOSED pgs=2659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:45.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.589+0000 7f82b5a00640 1 -- 172.21.15.39:0/753946255 >> 172.21.15.39:0/753946255 conn(0x7f82b006cfc0 msgr2=0x7f82b015cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:45.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.589+0000 7f82b5a00640 1 -- 172.21.15.39:0/753946255 shutdown_connections 2024-09-21T07:53:45.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:45.590+0000 7f82b5a00640 1 -- 172.21.15.39:0/753946255 wait complete. 2024-09-21T07:53:45.616 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:53:50.617 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:53:50.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:50.856+0000 7f6e11a00640 1 -- 172.21.15.39:0/2744061154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6df0004d40 msgr2=0x7f6df0005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:50.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:50.856+0000 7f6e11a00640 1 --2- 172.21.15.39:0/2744061154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6df0004d40 0x7f6df0005150 secure :-1 s=READY pgs=2661 cs=0 l=1 rev1=1 crypto rx=0x7f6e0400c770 tx=0x7f6df0005830 comp rx=0 tx=0).stop 2024-09-21T07:53:50.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:50.856+0000 7f6e11000640 1 -- 172.21.15.39:0/2744061154 reap_dead start 2024-09-21T07:53:50.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:50.856+0000 7f6e11a00640 1 -- 172.21.15.39:0/2744061154 shutdown_connections 2024-09-21T07:53:50.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:50.856+0000 7f6e11a00640 1 -- 172.21.15.39:0/2744061154 >> 172.21.15.39:0/2744061154 conn(0x7f6e0c0b2510 msgr2=0x7f6e0c0d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:50.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:50.856+0000 7f6e11a00640 1 -- 172.21.15.39:0/2744061154 shutdown_connections 2024-09-21T07:53:50.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:50.856+0000 7f6e11a00640 1 -- 172.21.15.39:0/2744061154 wait complete. 2024-09-21T07:53:50.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:50.858+0000 7f6e11a00640 1 Processor -- start 2024-09-21T07:53:50.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:50.858+0000 7f6e11a00640 1 -- start start 2024-09-21T07:53:50.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:50.860+0000 7f6e11a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6df0004d40 0x7f6e0c09da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:50.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:50.860+0000 7f6e11a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e0c0c9f70 0x7f6e0c0c9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:50.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:50.860+0000 7f6e11a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e0c15a150 0x7f6e0c15a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:50.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:50.860+0000 7f6e11a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0c0cb9b0 con 0x7f6df0004d40 2024-09-21T07:53:50.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:50.860+0000 7f6e11a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0c0cd650 con 0x7f6e0c15a150 2024-09-21T07:53:50.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:50.860+0000 7f6e11a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f6e0c0cbb30 con 0x7f6e0c0c9f70 2024-09-21T07:53:50.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:50.860+0000 7f6e0be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6df0004d40 0x7f6e0c09da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:50.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:50.860+0000 7f6e0be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6df0004d40 0x7f6e0c09da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54808/0 (socket says 172.21.15.39:54808) 2024-09-21T07:53:50.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:50.860+0000 7f6e0be00640 1 -- 172.21.15.39:0/122331113 learned_addr learned my addr 172.21.15.39:0/122331113 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:50.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:50.860+0000 7f6e0be00640 1 -- 172.21.15.39:0/122331113 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e0c0c9f70 msgr2=0x7f6e0c0c9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:50.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:50.860+0000 7f6e03400640 1 --2- 172.21.15.39:0/122331113 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e0c0c9f70 0x7f6e0c0c9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:50.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:50.860+0000 7f6e11000640 1 --2- 172.21.15.39:0/122331113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e0c15a150 0x7f6e0c15a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:50.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:50.860+0000 7f6e0be00640 1 --2- 172.21.15.39:0/122331113 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e0c0c9f70 0x7f6e0c0c9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:50.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:50.860+0000 7f6e0be00640 1 -- 172.21.15.39:0/122331113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e0c15a150 msgr2=0x7f6e0c15a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:50.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:50.860+0000 7f6e0be00640 1 --2- 172.21.15.39:0/122331113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e0c15a150 0x7f6e0c15a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:50.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:50.860+0000 7f6e0be00640 1 -- 172.21.15.39:0/122331113 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e0c15aaa0 con 0x7f6df0004d40 2024-09-21T07:53:50.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:50.860+0000 7f6e11000640 1 --2- 172.21.15.39:0/122331113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e0c15a150 0x7f6e0c15a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:53:50.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:50.860+0000 7f6e03400640 1 --2- 172.21.15.39:0/122331113 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e0c0c9f70 0x7f6e0c0c9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:53:50.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:50.861+0000 7f6e0be00640 1 --2- 172.21.15.39:0/122331113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6df0004d40 0x7f6e0c09da90 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc011ae0 tx=0x7f6dfc009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:50.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:50.861+0000 7f6e08c00640 1 -- 172.21.15.39:0/122331113 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc00fcb0 con 0x7f6df0004d40 2024-09-21T07:53:50.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:50.861+0000 7f6e08c00640 1 -- 172.21.15.39:0/122331113 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6dfc00fe50 con 0x7f6df0004d40 2024-09-21T07:53:50.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:50.861+0000 7f6e08c00640 1 -- 172.21.15.39:0/122331113 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc004280 con 0x7f6df0004d40 2024-09-21T07:53:50.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:50.861+0000 7f6e11a00640 1 -- 172.21.15.39:0/122331113 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e0c158ca0 con 0x7f6df0004d40 2024-09-21T07:53:50.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:50.861+0000 7f6e11a00640 1 -- 172.21.15.39:0/122331113 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e0c159260 con 0x7f6df0004d40 2024-09-21T07:53:50.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:50.862+0000 7f6e11a00640 1 -- 172.21.15.39:0/122331113 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e0c070400 con 0x7f6df0004d40 2024-09-21T07:53:50.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:50.863+0000 7f6e08c00640 1 -- 172.21.15.39:0/122331113 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6dfc0035f0 con 0x7f6df0004d40 2024-09-21T07:53:50.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:50.865+0000 7f6e08c00640 1 --2- 172.21.15.39:0/122331113 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6de8078b90 0x7f6de807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:50.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:50.865+0000 7f6e08c00640 1 -- 172.21.15.39:0/122331113 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(250..250 src has 1..250) ==== 7820+0+0 (secure 0 0 0) 0x7f6dfc09c1e0 con 0x7f6df0004d40 2024-09-21T07:53:50.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:50.865+0000 7f6e03400640 1 --2- 172.21.15.39:0/122331113 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6de8078b90 0x7f6de807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:50.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:50.866+0000 7f6e03400640 1 --2- 172.21.15.39:0/122331113 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6de8078b90 0x7f6de807b0a0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c09d850 tx=0x7f6df4005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:50.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:50.870+0000 7f6e08c00640 1 -- 172.21.15.39:0/122331113 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6e0c070400 con 0x7f6df0004d40 2024-09-21T07:53:51.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:51.106+0000 7f6e11a00640 1 -- 172.21.15.39:0/122331113 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e0c160320 con 0x7f6df0004d40 2024-09-21T07:53:51.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:51.107+0000 7f6e08c00640 1 -- 172.21.15.39:0/122331113 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 292 v292) ==== 95+0+5291 (secure 0 0 0) 0x7f6e0c15aaa0 con 0x7f6df0004d40 2024-09-21T07:53:51.108 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:53:51.108 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":292,"btime":"2024-09-21T07:53:38:290402+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10633,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3495264983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3495264983},{"type":"v1","addr":"172.21.15.148:6837","nonce":3495264983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":288}],"filesystems":[{"mdsmap":{"epoch":292,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:53:38.264700+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":249,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":11364,"mds_1":11052,"mds_2":10621},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10621":{"gid":10621,"name":"b","rank":2,"incarnation":287,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6835/2191156088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2191156088},{"type":"v1","addr":"172.21.15.148:6835","nonce":2191156088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11364":{"gid":11364,"name":"a","rank":0,"incarnation":285,"state":"up:active","state_seq":28,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11364,"qdb_cluster":[11364,11052,10621]},"id":13}]} 2024-09-21T07:53:51.108 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 292 2024-09-21T07:53:51.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:51.110+0000 7f6e11a00640 1 -- 172.21.15.39:0/122331113 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6de8078b90 msgr2=0x7f6de807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:51.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:51.110+0000 7f6e11a00640 1 --2- 172.21.15.39:0/122331113 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6de8078b90 0x7f6de807b0a0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c09d850 tx=0x7f6df4005f90 comp rx=0 tx=0).stop 2024-09-21T07:53:51.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:51.110+0000 7f6e11a00640 1 -- 172.21.15.39:0/122331113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6df0004d40 msgr2=0x7f6e0c09da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:51.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:51.110+0000 7f6e11a00640 1 --2- 172.21.15.39:0/122331113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6df0004d40 0x7f6e0c09da90 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc011ae0 tx=0x7f6dfc009f10 comp rx=0 tx=0).stop 2024-09-21T07:53:51.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:51.110+0000 7f6e11a00640 1 -- 172.21.15.39:0/122331113 shutdown_connections 2024-09-21T07:53:51.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:51.110+0000 7f6e11a00640 1 --2- 172.21.15.39:0/122331113 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6de8078b90 0x7f6de807b0a0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:51.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:51.110+0000 7f6e11a00640 1 --2- 172.21.15.39:0/122331113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e0c15a150 0x7f6e0c15a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:51.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:51.110+0000 7f6e11a00640 1 --2- 172.21.15.39:0/122331113 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e0c0c9f70 0x7f6e0c0c9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:51.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:51.110+0000 7f6e11a00640 1 --2- 172.21.15.39:0/122331113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6df0004d40 0x7f6e0c09da90 unknown :-1 s=CLOSED pgs=2662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:51.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:51.110+0000 7f6e11a00640 1 -- 172.21.15.39:0/122331113 >> 172.21.15.39:0/122331113 conn(0x7f6e0c0b2510 msgr2=0x7f6e0c0b4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:51.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:51.110+0000 7f6e11a00640 1 -- 172.21.15.39:0/122331113 shutdown_connections 2024-09-21T07:53:51.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:51.110+0000 7f6e11a00640 1 -- 172.21.15.39:0/122331113 wait complete. 2024-09-21T07:53:51.136 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-21T07:53:51.986 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~7s 2024-09-21T07:53:56.137 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:53:56.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.371+0000 7fa4ed200640 1 -- 172.21.15.39:0/2082906366 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4e80cfd10 msgr2=0x7fa4e80cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:56.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.372+0000 7fa4ed200640 1 --2- 172.21.15.39:0/2082906366 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4e80cfd10 0x7fa4e80cd8e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0009860 tx=0x7fa4d002a5a0 comp rx=0 tx=0).stop 2024-09-21T07:53:56.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.372+0000 7fa4ed200640 1 -- 172.21.15.39:0/2082906366 shutdown_connections 2024-09-21T07:53:56.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.372+0000 7fa4ed200640 1 --2- 172.21.15.39:0/2082906366 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4e80cfd10 0x7fa4e80cd8e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:56.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.372+0000 7fa4ed200640 1 -- 172.21.15.39:0/2082906366 >> 172.21.15.39:0/2082906366 conn(0x7fa4e806cfc0 msgr2=0x7fa4e806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:56.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.372+0000 7fa4ed200640 1 -- 172.21.15.39:0/2082906366 shutdown_connections 2024-09-21T07:53:56.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.372+0000 7fa4ed200640 1 -- 172.21.15.39:0/2082906366 wait complete. 2024-09-21T07:53:56.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.374+0000 7fa4ed200640 1 Processor -- start 2024-09-21T07:53:56.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.374+0000 7fa4ed200640 1 -- start start 2024-09-21T07:53:56.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.376+0000 7fa4ed200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4e80cfd10 0x7fa4e81e5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:56.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.376+0000 7fa4ed200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4e80d0620 0x7fa4e80b22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:56.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.376+0000 7fa4ed200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4e815cd30 0x7fa4e81f4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:56.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.376+0000 7fa4ed200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e8073aa0 con 0x7fa4e815cd30 2024-09-21T07:53:56.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.376+0000 7fa4ed200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e8073920 con 0x7fa4e80d0620 2024-09-21T07:53:56.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.376+0000 7fa4ed200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa4e8073c20 con 0x7fa4e80cfd10 2024-09-21T07:53:56.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.376+0000 7fa4e6800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4e815cd30 0x7fa4e81f4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:56.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.376+0000 7fa4e6800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4e815cd30 0x7fa4e81f4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:32904/0 (socket says 172.21.15.39:32904) 2024-09-21T07:53:56.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.376+0000 7fa4e6800640 1 -- 172.21.15.39:0/4173203496 learned_addr learned my addr 172.21.15.39:0/4173203496 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:56.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.376+0000 7fa4dfe00640 1 --2- 172.21.15.39:0/4173203496 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4e80d0620 0x7fa4e80b22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:56.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.376+0000 7fa4e5e00640 1 --2- 172.21.15.39:0/4173203496 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4e80cfd10 0x7fa4e81e5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:56.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.376+0000 7fa4e6800640 1 -- 172.21.15.39:0/4173203496 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4e80cfd10 msgr2=0x7fa4e81e5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:56.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.376+0000 7fa4e6800640 1 --2- 172.21.15.39:0/4173203496 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4e80cfd10 0x7fa4e81e5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:56.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.376+0000 7fa4e6800640 1 -- 172.21.15.39:0/4173203496 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4e80d0620 msgr2=0x7fa4e80b22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:56.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.376+0000 7fa4e6800640 1 --2- 172.21.15.39:0/4173203496 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4e80d0620 0x7fa4e80b22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:56.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.376+0000 7fa4e6800640 1 -- 172.21.15.39:0/4173203496 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4e81e7940 con 0x7fa4e815cd30 2024-09-21T07:53:56.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.377+0000 7fa4e6800640 1 --2- 172.21.15.39:0/4173203496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4e815cd30 0x7fa4e81f4a90 secure :-1 s=READY pgs=2664 cs=0 l=1 rev1=1 crypto rx=0x7fa4e000cb10 tx=0x7fa4e000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:56.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.377+0000 7fa4de000640 1 -- 172.21.15.39:0/4173203496 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0016020 con 0x7fa4e815cd30 2024-09-21T07:53:56.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.377+0000 7fa4de000640 1 -- 172.21.15.39:0/4173203496 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4e000ad80 con 0x7fa4e815cd30 2024-09-21T07:53:56.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.377+0000 7fa4de000640 1 -- 172.21.15.39:0/4173203496 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0012680 con 0x7fa4e815cd30 2024-09-21T07:53:56.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.377+0000 7fa4ed200640 1 -- 172.21.15.39:0/4173203496 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4e81e7e40 con 0x7fa4e815cd30 2024-09-21T07:53:56.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.377+0000 7fa4ed200640 1 -- 172.21.15.39:0/4173203496 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4e81e8400 con 0x7fa4e815cd30 2024-09-21T07:53:56.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.378+0000 7fa4ed200640 1 -- 172.21.15.39:0/4173203496 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4e815e9c0 con 0x7fa4e815cd30 2024-09-21T07:53:56.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.379+0000 7fa4de000640 1 -- 172.21.15.39:0/4173203496 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa4e0019020 con 0x7fa4e815cd30 2024-09-21T07:53:56.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.380+0000 7fa4de000640 1 --2- 172.21.15.39:0/4173203496 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa4c4078b90 0x7fa4c407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:56.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.381+0000 7fa4de000640 1 -- 172.21.15.39:0/4173203496 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(250..250 src has 1..250) ==== 7820+0+0 (secure 0 0 0) 0x7fa4e009bbe0 con 0x7fa4e815cd30 2024-09-21T07:53:56.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.381+0000 7fa4e5e00640 1 --2- 172.21.15.39:0/4173203496 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa4c4078b90 0x7fa4c407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:56.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.382+0000 7fa4e5e00640 1 --2- 172.21.15.39:0/4173203496 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa4c4078b90 0x7fa4c407b0a0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8001120 tx=0x7fa4d8009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:56.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.385+0000 7fa4de000640 1 -- 172.21.15.39:0/4173203496 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa4e0020020 con 0x7fa4e815cd30 2024-09-21T07:53:56.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.622+0000 7fa4ed200640 1 -- 172.21.15.39:0/4173203496 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4e81e7bb0 con 0x7fa4e815cd30 2024-09-21T07:53:56.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.623+0000 7fa4de000640 1 -- 172.21.15.39:0/4173203496 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 293 v293) ==== 95+0+5339 (secure 0 0 0) 0x7fa4e00630f0 con 0x7fa4e815cd30 2024-09-21T07:53:56.625 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:53:56.625 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":293,"btime":"2024-09-21T07:53:56:605307+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10633,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3495264983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3495264983},{"type":"v1","addr":"172.21.15.148:6837","nonce":3495264983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":288}],"filesystems":[{"mdsmap":{"epoch":293,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:53:56.605285+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":249,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":11364,"mds_1":11052,"mds_2":10621},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10621":{"gid":10621,"name":"b","rank":2,"incarnation":287,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6835/2191156088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2191156088},{"type":"v1","addr":"172.21.15.148:6835","nonce":2191156088}]},"join_fscid":-1,"laggy_since":"2024-09-21T07:53:56.605284+0000","export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11364":{"gid":11364,"name":"a","rank":0,"incarnation":285,"state":"up:active","state_seq":28,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11364,"qdb_cluster":[10621,11052,11364]},"id":13}]} 2024-09-21T07:53:56.625 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 293 2024-09-21T07:53:56.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.625+0000 7fa4ed200640 1 -- 172.21.15.39:0/4173203496 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa4c4078b90 msgr2=0x7fa4c407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:56.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.625+0000 7fa4ed200640 1 --2- 172.21.15.39:0/4173203496 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa4c4078b90 0x7fa4c407b0a0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8001120 tx=0x7fa4d8009040 comp rx=0 tx=0).stop 2024-09-21T07:53:56.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.625+0000 7fa4ed200640 1 -- 172.21.15.39:0/4173203496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4e815cd30 msgr2=0x7fa4e81f4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:56.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.625+0000 7fa4ed200640 1 --2- 172.21.15.39:0/4173203496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4e815cd30 0x7fa4e81f4a90 secure :-1 s=READY pgs=2664 cs=0 l=1 rev1=1 crypto rx=0x7fa4e000cb10 tx=0x7fa4e000a060 comp rx=0 tx=0).stop 2024-09-21T07:53:56.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.625+0000 7fa4ed200640 1 -- 172.21.15.39:0/4173203496 shutdown_connections 2024-09-21T07:53:56.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.625+0000 7fa4ed200640 1 --2- 172.21.15.39:0/4173203496 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa4c4078b90 0x7fa4c407b0a0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:56.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.625+0000 7fa4ed200640 1 --2- 172.21.15.39:0/4173203496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4e815cd30 0x7fa4e81f4a90 unknown :-1 s=CLOSED pgs=2664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:56.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.625+0000 7fa4ed200640 1 --2- 172.21.15.39:0/4173203496 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4e80d0620 0x7fa4e80b22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:56.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.625+0000 7fa4ed200640 1 --2- 172.21.15.39:0/4173203496 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa4e80cfd10 0x7fa4e81e5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:56.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.625+0000 7fa4ed200640 1 -- 172.21.15.39:0/4173203496 >> 172.21.15.39:0/4173203496 conn(0x7fa4e806cfc0 msgr2=0x7fa4e80ccef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:56.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.625+0000 7fa4ed200640 1 -- 172.21.15.39:0/4173203496 shutdown_connections 2024-09-21T07:53:56.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:56.625+0000 7fa4ed200640 1 -- 172.21.15.39:0/4173203496 wait complete. 2024-09-21T07:53:56.653 DEBUG:tasks.ceph_test_case:wait_until_true: success in 15s and 0 retries 2024-09-21T07:53:56.653 DEBUG:teuthology.orchestra.run.smithi148:> sudo sysctl -n kernel.core_pattern 2024-09-21T07:53:56.712 INFO:teuthology.orchestra.run.smithi148.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T07:53:56.715 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2024-09-21T07:53:56.715 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2024-09-21T07:53:56.783 INFO:teuthology.orchestra.run.smithi148.stdout:1726905219.45745.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i b', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/proc/self/exe', platform: 'x86_64' 2024-09-21T07:53:56.786 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1726905219.45745.core'] 2024-09-21T07:53:56.787 INFO:tasks.cephfs.cephfs_test_case:Found core file 1726905219.45745.core, deleting it 2024-09-21T07:53:56.787 DEBUG:teuthology.orchestra.run.smithi148:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1726905219.45745.core 2024-09-21T07:53:56.878 INFO:tasks.cephfs.mount:Suspending the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:53:56.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:53:56.878 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 down 2024-09-21T07:53:56.941 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:53:57.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.165+0000 7fd71f200640 1 -- 172.21.15.39:0/1997831056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7180b3370 msgr2=0x7fd7180b3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:57.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.165+0000 7fd71f200640 1 --2- 172.21.15.39:0/1997831056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7180b3370 0x7fd7180b3780 secure :-1 s=READY pgs=2666 cs=0 l=1 rev1=1 crypto rx=0x7fd704009830 tx=0x7fd70402a480 comp rx=0 tx=0).stop 2024-09-21T07:53:57.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.166+0000 7fd71f200640 1 -- 172.21.15.39:0/1997831056 shutdown_connections 2024-09-21T07:53:57.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.166+0000 7fd71f200640 1 --2- 172.21.15.39:0/1997831056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7180b3370 0x7fd7180b3780 unknown :-1 s=CLOSED pgs=2666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:57.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.166+0000 7fd71f200640 1 --2- 172.21.15.39:0/1997831056 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd71815e590 0x7fd718070a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:57.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.166+0000 7fd71f200640 1 -- 172.21.15.39:0/1997831056 >> 172.21.15.39:0/1997831056 conn(0x7fd71806cfc0 msgr2=0x7fd71806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:57.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.166+0000 7fd71f200640 1 -- 172.21.15.39:0/1997831056 shutdown_connections 2024-09-21T07:53:57.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.166+0000 7fd71f200640 1 -- 172.21.15.39:0/1997831056 wait complete. 2024-09-21T07:53:57.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.168+0000 7fd71f200640 1 Processor -- start 2024-09-21T07:53:57.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.168+0000 7fd71f200640 1 -- start start 2024-09-21T07:53:57.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.169+0000 7fd71f200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7180b3370 0x7fd718153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:57.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.169+0000 7fd71f200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd718151590 0x7fd7180cee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:57.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.169+0000 7fd71f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd71815e590 0x7fd718159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:57.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.169+0000 7fd71f200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd7181519a0 con 0x7fd71815e590 2024-09-21T07:53:57.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.169+0000 7fd71f200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd718071450 con 0x7fd7180b3370 2024-09-21T07:53:57.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.169+0000 7fd71f200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd718151b20 con 0x7fd718151590 2024-09-21T07:53:57.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.170+0000 7fd71d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd71815e590 0x7fd718159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:57.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.170+0000 7fd71d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd71815e590 0x7fd718159bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:32936/0 (socket says 172.21.15.39:32936) 2024-09-21T07:53:57.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.170+0000 7fd71d200640 1 -- 172.21.15.39:0/963071105 learned_addr learned my addr 172.21.15.39:0/963071105 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:57.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.170+0000 7fd717400640 1 --2- 172.21.15.39:0/963071105 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd718151590 0x7fd7180cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:57.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.170+0000 7fd717e00640 1 --2- 172.21.15.39:0/963071105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7180b3370 0x7fd718153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:57.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.170+0000 7fd71d200640 1 -- 172.21.15.39:0/963071105 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd718151590 msgr2=0x7fd7180cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:57.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.170+0000 7fd71d200640 1 --2- 172.21.15.39:0/963071105 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd718151590 0x7fd7180cee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:57.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.170+0000 7fd71d200640 1 -- 172.21.15.39:0/963071105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7180b3370 msgr2=0x7fd718153d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:57.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.170+0000 7fd71d200640 1 --2- 172.21.15.39:0/963071105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7180b3370 0x7fd718153d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:57.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.170+0000 7fd71d200640 1 -- 172.21.15.39:0/963071105 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd718153290 con 0x7fd71815e590 2024-09-21T07:53:57.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.171+0000 7fd71d200640 1 --2- 172.21.15.39:0/963071105 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd71815e590 0x7fd718159bc0 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 crypto rx=0x7fd71000c760 tx=0x7fd71000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:57.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.171+0000 7fd70fe00640 1 -- 172.21.15.39:0/963071105 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd710017070 con 0x7fd71815e590 2024-09-21T07:53:57.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.171+0000 7fd70fe00640 1 -- 172.21.15.39:0/963071105 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd71000ce10 con 0x7fd71815e590 2024-09-21T07:53:57.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.171+0000 7fd71f200640 1 -- 172.21.15.39:0/963071105 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd718153570 con 0x7fd71815e590 2024-09-21T07:53:57.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.171+0000 7fd70fe00640 1 -- 172.21.15.39:0/963071105 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7100052b0 con 0x7fd71815e590 2024-09-21T07:53:57.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.171+0000 7fd71f200640 1 -- 172.21.15.39:0/963071105 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd71815de50 con 0x7fd71815e590 2024-09-21T07:53:57.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.172+0000 7fd71f200640 1 -- 172.21.15.39:0/963071105 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7180b3780 con 0x7fd71815e590 2024-09-21T07:53:57.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.173+0000 7fd70fe00640 1 -- 172.21.15.39:0/963071105 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd71001a070 con 0x7fd71815e590 2024-09-21T07:53:57.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.174+0000 7fd70fe00640 1 --2- 172.21.15.39:0/963071105 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd6f8078b90 0x7fd6f807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:57.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.174+0000 7fd70fe00640 1 -- 172.21.15.39:0/963071105 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(250..250 src has 1..250) ==== 7820+0+0 (secure 0 0 0) 0x7fd71009aae0 con 0x7fd71815e590 2024-09-21T07:53:57.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.175+0000 7fd717e00640 1 --2- 172.21.15.39:0/963071105 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd6f8078b90 0x7fd6f807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:57.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.176+0000 7fd717e00640 1 --2- 172.21.15.39:0/963071105 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd6f8078b90 0x7fd6f807b0a0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fd708009b30 tx=0x7fd708005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:57.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.179+0000 7fd70fe00640 1 -- 172.21.15.39:0/963071105 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd710062090 con 0x7fd71815e590 2024-09-21T07:53:57.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.415+0000 7fd71f200640 1 -- 172.21.15.39:0/963071105 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7180ce3a0 con 0x7fd71815e590 2024-09-21T07:53:57.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.416+0000 7fd70fe00640 1 -- 172.21.15.39:0/963071105 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 293 v293) ==== 95+0+5339 (secure 0 0 0) 0x7fd710066ac0 con 0x7fd71815e590 2024-09-21T07:53:57.418 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:53:57.418 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":293,"btime":"2024-09-21T07:53:56:605307+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10633,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3495264983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3495264983},{"type":"v1","addr":"172.21.15.148:6837","nonce":3495264983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":288}],"filesystems":[{"mdsmap":{"epoch":293,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:53:56.605285+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":249,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":11364,"mds_1":11052,"mds_2":10621},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10621":{"gid":10621,"name":"b","rank":2,"incarnation":287,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6835/2191156088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":2191156088},{"type":"v1","addr":"172.21.15.148:6835","nonce":2191156088}]},"join_fscid":-1,"laggy_since":"2024-09-21T07:53:56.605284+0000","export_targets":[],"features":4540719139924082687,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11052":{"gid":11052,"name":"c","rank":1,"incarnation":261,"state":"up:active","state_seq":5,"addr":"172.21.15.39:6835/485194245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":485194245},{"type":"v1","addr":"172.21.15.39:6835","nonce":485194245}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11364":{"gid":11364,"name":"a","rank":0,"incarnation":285,"state":"up:active","state_seq":28,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11364,"qdb_cluster":[10621,11052,11364]},"id":13}]} 2024-09-21T07:53:57.418 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 293 2024-09-21T07:53:57.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.418+0000 7fd71f200640 1 -- 172.21.15.39:0/963071105 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd6f8078b90 msgr2=0x7fd6f807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:57.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.418+0000 7fd71f200640 1 --2- 172.21.15.39:0/963071105 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd6f8078b90 0x7fd6f807b0a0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fd708009b30 tx=0x7fd708005f90 comp rx=0 tx=0).stop 2024-09-21T07:53:57.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.418+0000 7fd71f200640 1 -- 172.21.15.39:0/963071105 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd71815e590 msgr2=0x7fd718159bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:57.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.418+0000 7fd71f200640 1 --2- 172.21.15.39:0/963071105 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd71815e590 0x7fd718159bc0 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 crypto rx=0x7fd71000c760 tx=0x7fd71000cc20 comp rx=0 tx=0).stop 2024-09-21T07:53:57.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.418+0000 7fd71f200640 1 -- 172.21.15.39:0/963071105 shutdown_connections 2024-09-21T07:53:57.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.418+0000 7fd71f200640 1 --2- 172.21.15.39:0/963071105 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd6f8078b90 0x7fd6f807b0a0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:57.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.418+0000 7fd71f200640 1 --2- 172.21.15.39:0/963071105 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd71815e590 0x7fd718159bc0 unknown :-1 s=CLOSED pgs=2667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:57.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.418+0000 7fd71f200640 1 --2- 172.21.15.39:0/963071105 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd718151590 0x7fd7180cee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:57.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.418+0000 7fd71f200640 1 --2- 172.21.15.39:0/963071105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd7180b3370 0x7fd718153d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:57.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.418+0000 7fd71f200640 1 -- 172.21.15.39:0/963071105 >> 172.21.15.39:0/963071105 conn(0x7fd71806cfc0 msgr2=0x7fd7180affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:57.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.418+0000 7fd71f200640 1 -- 172.21.15.39:0/963071105 shutdown_connections 2024-09-21T07:53:57.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.418+0000 7fd71f200640 1 -- 172.21.15.39:0/963071105 wait complete. 2024-09-21T07:53:57.442 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump snaps --server 2024-09-21T07:53:57.685 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_snap":10,"last_created":9,"last_destroyed":7,"pending_noop":[],"snaps":[{"snapid":8,"ino":1099511627780,"stamp":"2024-09-21T07:51:12.091986+0000","name":"s1","metadata":{}},{"snapid":9,"ino":1099511627780,"stamp":"2024-09-21T07:52:44.191763+0000","name":"s2","metadata":{}}],"need_to_purge":{},"pending_update":[{"version":22,"snapinfo":{"snapid":10,"ino":1099511627780,"stamp":"2024-09-21T07:53:39.355926+0000","name":"s3","metadata":{}}}],"pending_destroy":[]} 2024-09-21T07:53:57.686 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 9, "last_destroyed": 7, "last_snap": 10, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [ { "snapInfinityo": { "ino": 1099511627780, "metadata": {}, "name": "s3", "snapid": 10, "stamp": "2024-09-21T07:53:39.355926+0000" }, "version": 22 } ], "snaps": [ { "ino": 1099511627780, "metadata": {}, "name": "s1", "snapid": 8, "stamp": "2024-09-21T07:51:12.091986+0000" }, { "ino": 1099511627780, "metadata": {}, "name": "s2", "snapid": 9, "stamp": "2024-09-21T07:52:44.191763+0000" } ] } 2024-09-21T07:53:57.686 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 13:1 2024-09-21T07:53:57.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.912+0000 7fceaf200640 1 -- 172.21.15.39:0/3292159873 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce94004d90 msgr2=0x7fce94005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:57.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.912+0000 7fceaf200640 1 --2- 172.21.15.39:0/3292159873 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce94004d90 0x7fce94005220 secure :-1 s=READY pgs=2669 cs=0 l=1 rev1=1 crypto rx=0x7fceb00755c0 tx=0x7fce94005830 comp rx=0 tx=0).stop 2024-09-21T07:53:57.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.912+0000 7fceae800640 1 -- 172.21.15.39:0/3292159873 reap_dead start 2024-09-21T07:53:57.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.912+0000 7fceaf200640 1 -- 172.21.15.39:0/3292159873 shutdown_connections 2024-09-21T07:53:57.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.912+0000 7fceaf200640 1 -- 172.21.15.39:0/3292159873 >> 172.21.15.39:0/3292159873 conn(0x7fceb00b2510 msgr2=0x7fceb00d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:57.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.913+0000 7fceaf200640 1 -- 172.21.15.39:0/3292159873 shutdown_connections 2024-09-21T07:53:57.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.913+0000 7fceaf200640 1 -- 172.21.15.39:0/3292159873 wait complete. 2024-09-21T07:53:57.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.915+0000 7fceaf200640 1 Processor -- start 2024-09-21T07:53:57.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.915+0000 7fceaf200640 1 -- start start 2024-09-21T07:53:57.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.916+0000 7fceaf200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce94004d90 0x7fceb009db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:57.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.916+0000 7fceaf200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fceb0159480 0x7fceb0159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:57.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.916+0000 7fceaf200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fceb01f6b80 0x7fceb01f6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:57.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.916+0000 7fceaf200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fceb0073740 con 0x7fce94004d90 2024-09-21T07:53:57.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.916+0000 7fceaf200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fceb00735c0 con 0x7fceb01f6b80 2024-09-21T07:53:57.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.916+0000 7fceaf200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fceb00b43f0 con 0x7fceb0159480 2024-09-21T07:53:57.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.917+0000 7fceade00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce94004d90 0x7fceb009db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:57.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.917+0000 7fceade00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce94004d90 0x7fceb009db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:32962/0 (socket says 172.21.15.39:32962) 2024-09-21T07:53:57.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.917+0000 7fceade00640 1 -- 172.21.15.39:0/1145260293 learned_addr learned my addr 172.21.15.39:0/1145260293 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:57.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.917+0000 7fceae800640 1 --2- 172.21.15.39:0/1145260293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fceb01f6b80 0x7fceb01f6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:57.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.917+0000 7fcead400640 1 --2- 172.21.15.39:0/1145260293 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fceb0159480 0x7fceb0159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:57.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.917+0000 7fceade00640 1 -- 172.21.15.39:0/1145260293 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fceb0159480 msgr2=0x7fceb0159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:57.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.917+0000 7fceade00640 1 --2- 172.21.15.39:0/1145260293 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fceb0159480 0x7fceb0159890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:57.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.917+0000 7fceade00640 1 -- 172.21.15.39:0/1145260293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fceb01f6b80 msgr2=0x7fceb01f6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:57.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.917+0000 7fceade00640 1 --2- 172.21.15.39:0/1145260293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fceb01f6b80 0x7fceb01f6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:57.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.917+0000 7fceade00640 1 -- 172.21.15.39:0/1145260293 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb01f74d0 con 0x7fce94004d90 2024-09-21T07:53:57.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.917+0000 7fceade00640 1 --2- 172.21.15.39:0/1145260293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce94004d90 0x7fceb009db90 secure :-1 s=READY pgs=2670 cs=0 l=1 rev1=1 crypto rx=0x7fcea0006630 tx=0x7fcea00039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:57.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.918+0000 7fcea6000640 1 -- 172.21.15.39:0/1145260293 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea0015070 con 0x7fce94004d90 2024-09-21T07:53:57.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.918+0000 7fcea6000640 1 -- 172.21.15.39:0/1145260293 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcea0011920 con 0x7fce94004d90 2024-09-21T07:53:57.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.918+0000 7fcea6000640 1 -- 172.21.15.39:0/1145260293 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea0011d70 con 0x7fce94004d90 2024-09-21T07:53:57.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.918+0000 7fceaf200640 1 -- 172.21.15.39:0/1145260293 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceb015e9c0 con 0x7fce94004d90 2024-09-21T07:53:57.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.918+0000 7fceaf200640 1 -- 172.21.15.39:0/1145260293 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceb015ef40 con 0x7fce94004d90 2024-09-21T07:53:57.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.919+0000 7fceaf200640 1 -- 172.21.15.39:0/1145260293 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fceb00b4980 con 0x7fce94004d90 2024-09-21T07:53:57.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.920+0000 7fcea6000640 1 -- 172.21.15.39:0/1145260293 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcea0018070 con 0x7fce94004d90 2024-09-21T07:53:57.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.921+0000 7fcea6000640 1 --2- 172.21.15.39:0/1145260293 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fce90078b90 0x7fce9007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:57.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.921+0000 7fcea6000640 1 -- 172.21.15.39:0/1145260293 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(250..250 src has 1..250) ==== 7820+0+0 (secure 0 0 0) 0x7fcea009c3e0 con 0x7fce94004d90 2024-09-21T07:53:57.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.922+0000 7fcead400640 1 --2- 172.21.15.39:0/1145260293 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fce90078b90 0x7fce9007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:57.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.922+0000 7fcead400640 1 --2- 172.21.15.39:0/1145260293 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fce90078b90 0x7fce9007b0a0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fceb00b3ea0 tx=0x7fce9c006b10 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:57.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:57.926+0000 7fcea6000640 1 -- 172.21.15.39:0/1145260293 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fceb01f74d0 con 0x7fce94004d90 2024-09-21T07:53:58.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.199+0000 7fceaf200640 1 -- 172.21.15.39:0/1145260293 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "13:1"} v 0) -- 0x7fceb0074200 con 0x7fce94004d90 2024-09-21T07:53:58.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.203+0000 7fcea6000640 1 -- 172.21.15.39:0/1145260293 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "13:1"}]=0 failed mds gid 11052 v294) ==== 99+0+0 (secure 0 0 0) 0x7fcea0063900 con 0x7fce94004d90 2024-09-21T07:53:58.205 INFO:teuthology.orchestra.run.smithi039.stderr:failed mds gid 11052 2024-09-21T07:53:58.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.206+0000 7fceaf200640 1 -- 172.21.15.39:0/1145260293 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fce90078b90 msgr2=0x7fce9007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:58.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.206+0000 7fceaf200640 1 --2- 172.21.15.39:0/1145260293 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fce90078b90 0x7fce9007b0a0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fceb00b3ea0 tx=0x7fce9c006b10 comp rx=0 tx=0).stop 2024-09-21T07:53:58.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.207+0000 7fceaf200640 1 -- 172.21.15.39:0/1145260293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce94004d90 msgr2=0x7fceb009db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:58.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.207+0000 7fceaf200640 1 --2- 172.21.15.39:0/1145260293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce94004d90 0x7fceb009db90 secure :-1 s=READY pgs=2670 cs=0 l=1 rev1=1 crypto rx=0x7fcea0006630 tx=0x7fcea00039c0 comp rx=0 tx=0).stop 2024-09-21T07:53:58.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.207+0000 7fceaf200640 1 -- 172.21.15.39:0/1145260293 shutdown_connections 2024-09-21T07:53:58.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.207+0000 7fceaf200640 1 --2- 172.21.15.39:0/1145260293 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fce90078b90 0x7fce9007b0a0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:58.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.207+0000 7fceaf200640 1 --2- 172.21.15.39:0/1145260293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fceb01f6b80 0x7fceb01f6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:58.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.207+0000 7fceaf200640 1 --2- 172.21.15.39:0/1145260293 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fceb0159480 0x7fceb0159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:58.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.207+0000 7fceaf200640 1 --2- 172.21.15.39:0/1145260293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce94004d90 0x7fceb009db90 unknown :-1 s=CLOSED pgs=2670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:58.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.207+0000 7fceaf200640 1 -- 172.21.15.39:0/1145260293 >> 172.21.15.39:0/1145260293 conn(0x7fceb00b2510 msgr2=0x7fceb0074d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:58.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.207+0000 7fceaf200640 1 -- 172.21.15.39:0/1145260293 shutdown_connections 2024-09-21T07:53:58.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.207+0000 7fceaf200640 1 -- 172.21.15.39:0/1145260293 wait complete. 2024-09-21T07:53:58.228 INFO:tasks.ceph.mds.c.smithi039.stderr: -2223> 2024-09-21T07:53:08.769+0000 7f824ee00640 -1 --2- [v2:172.21.15.39:6833/485194245,v1:172.21.15.39:6835/485194245] >> [v2:172.21.15.148:6835/597301351,v1:172.21.15.148:6837/597301351] conn(0x5599fb2cd680 0x5599fb486680 unknown :-1 s=BANNER_CONNECTING pgs=14 cs=2 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.148:6835/597301351,v1:172.21.15.148:6837/597301351] is using msgr V1 protocol 2024-09-21T07:53:58.240 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 13:2 2024-09-21T07:53:58.354 INFO:tasks.ceph.mds.c.smithi039.stdout:starting mds.c at 2024-09-21T07:53:58.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.472+0000 7f32c6800640 1 -- 172.21.15.39:0/1467982650 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32c015cd30 msgr2=0x7f32c00cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:58.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.472+0000 7f32c6800640 1 --2- 172.21.15.39:0/1467982650 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32c015cd30 0x7f32c00cfd10 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f32b0009830 tx=0x7f32b002a530 comp rx=0 tx=0).stop 2024-09-21T07:53:58.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.472+0000 7f32c6800640 1 -- 172.21.15.39:0/1467982650 shutdown_connections 2024-09-21T07:53:58.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.472+0000 7f32c6800640 1 --2- 172.21.15.39:0/1467982650 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32c015cd30 0x7f32c00cfd10 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:58.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.472+0000 7f32c6800640 1 -- 172.21.15.39:0/1467982650 >> 172.21.15.39:0/1467982650 conn(0x7f32c006cfc0 msgr2=0x7f32c006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:58.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.472+0000 7f32c6800640 1 -- 172.21.15.39:0/1467982650 shutdown_connections 2024-09-21T07:53:58.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.472+0000 7f32c6800640 1 -- 172.21.15.39:0/1467982650 wait complete. 2024-09-21T07:53:58.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.474+0000 7f32c6800640 1 Processor -- start 2024-09-21T07:53:58.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.474+0000 7f32c6800640 1 -- start start 2024-09-21T07:53:58.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.476+0000 7f32c6800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32c006f800 0x7f32c01f9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:58.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.476+0000 7f32c6800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32c006fc10 0x7f32c0159910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:58.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.476+0000 7f32c6800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32c015cd30 0x7f32c00aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:58.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.476+0000 7f32c6800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f32c0073aa0 con 0x7f32c006fc10 2024-09-21T07:53:58.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.476+0000 7f32c6800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f32c0073920 con 0x7f32c015cd30 2024-09-21T07:53:58.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.476+0000 7f32c6800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f32c0073c20 con 0x7f32c006f800 2024-09-21T07:53:58.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.476+0000 7f32b6a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32c006fc10 0x7f32c0159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:58.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.476+0000 7f32b6a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32c006fc10 0x7f32c0159910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33004/0 (socket says 172.21.15.39:33004) 2024-09-21T07:53:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.476+0000 7f32b6a00640 1 -- 172.21.15.39:0/4215838394 learned_addr learned my addr 172.21.15.39:0/4215838394 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.476+0000 7f32bf400640 1 --2- 172.21.15.39:0/4215838394 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32c006f800 0x7f32c01f9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.476+0000 7f32bfe00640 1 --2- 172.21.15.39:0/4215838394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32c015cd30 0x7f32c00aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.476+0000 7f32b6a00640 1 -- 172.21.15.39:0/4215838394 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32c006f800 msgr2=0x7f32c01f9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.476+0000 7f32b6a00640 1 --2- 172.21.15.39:0/4215838394 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32c006f800 0x7f32c01f9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.476+0000 7f32b6a00640 1 -- 172.21.15.39:0/4215838394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32c015cd30 msgr2=0x7f32c00aeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.476+0000 7f32b6a00640 1 --2- 172.21.15.39:0/4215838394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32c015cd30 0x7f32c00aeb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.476+0000 7f32b6a00640 1 -- 172.21.15.39:0/4215838394 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c01f5c20 con 0x7f32c006fc10 2024-09-21T07:53:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.476+0000 7f32bfe00640 1 --2- 172.21.15.39:0/4215838394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32c015cd30 0x7f32c00aeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:53:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.477+0000 7f32b6a00640 1 --2- 172.21.15.39:0/4215838394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32c006fc10 0x7f32c0159910 secure :-1 s=READY pgs=2675 cs=0 l=1 rev1=1 crypto rx=0x7f32a800c3c0 tx=0x7f32a800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.477+0000 7f32b7e00640 1 -- 172.21.15.39:0/4215838394 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a8017070 con 0x7f32c006fc10 2024-09-21T07:53:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.477+0000 7f32b7e00640 1 -- 172.21.15.39:0/4215838394 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32a80053c0 con 0x7f32c006fc10 2024-09-21T07:53:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.477+0000 7f32b7e00640 1 -- 172.21.15.39:0/4215838394 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a8003850 con 0x7f32c006fc10 2024-09-21T07:53:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.477+0000 7f32c6800640 1 -- 172.21.15.39:0/4215838394 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32c01f5f00 con 0x7f32c006fc10 2024-09-21T07:53:58.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.477+0000 7f32c6800640 1 -- 172.21.15.39:0/4215838394 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32c01f69e0 con 0x7f32c006fc10 2024-09-21T07:53:58.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.478+0000 7f32c6800640 1 -- 172.21.15.39:0/4215838394 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32c00740d0 con 0x7f32c006fc10 2024-09-21T07:53:58.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.479+0000 7f32b7e00640 1 -- 172.21.15.39:0/4215838394 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f32a801a070 con 0x7f32c006fc10 2024-09-21T07:53:58.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.480+0000 7f32b7e00640 1 --2- 172.21.15.39:0/4215838394 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f329c078b90 0x7f329c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:58.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.481+0000 7f32b7e00640 1 -- 172.21.15.39:0/4215838394 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(251..251 src has 1..251) ==== 7906+0+0 (secure 0 0 0) 0x7f32a809a940 con 0x7f32c006fc10 2024-09-21T07:53:58.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.481+0000 7f32bf400640 1 --2- 172.21.15.39:0/4215838394 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f329c078b90 0x7f329c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:58.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.482+0000 7f32bf400640 1 --2- 172.21.15.39:0/4215838394 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f329c078b90 0x7f329c07b0a0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f32b0034000 tx=0x7f32b0004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:58.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.485+0000 7f32b7e00640 1 -- 172.21.15.39:0/4215838394 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f32a8061df0 con 0x7f32c006fc10 2024-09-21T07:53:58.597 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~13s 2024-09-21T07:53:58.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.761+0000 7f32c6800640 1 -- 172.21.15.39:0/4215838394 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "13:2"} v 0) -- 0x7f32c0159f30 con 0x7f32c006fc10 2024-09-21T07:53:58.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.764+0000 7f32b7e00640 1 -- 172.21.15.39:0/4215838394 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "13:2"}]=0 failed mds gid 10621 v296) ==== 99+0+0 (secure 0 0 0) 0x7f32a8066820 con 0x7f32c006fc10 2024-09-21T07:53:58.766 INFO:teuthology.orchestra.run.smithi039.stderr:failed mds gid 10621 2024-09-21T07:53:58.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.767+0000 7f32c6800640 1 -- 172.21.15.39:0/4215838394 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f329c078b90 msgr2=0x7f329c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:58.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.767+0000 7f32c6800640 1 --2- 172.21.15.39:0/4215838394 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f329c078b90 0x7f329c07b0a0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f32b0034000 tx=0x7f32b0004730 comp rx=0 tx=0).stop 2024-09-21T07:53:58.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.767+0000 7f32c6800640 1 -- 172.21.15.39:0/4215838394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32c006fc10 msgr2=0x7f32c0159910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:58.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.767+0000 7f32c6800640 1 --2- 172.21.15.39:0/4215838394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32c006fc10 0x7f32c0159910 secure :-1 s=READY pgs=2675 cs=0 l=1 rev1=1 crypto rx=0x7f32a800c3c0 tx=0x7f32a800c880 comp rx=0 tx=0).stop 2024-09-21T07:53:58.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.767+0000 7f32c6800640 1 -- 172.21.15.39:0/4215838394 shutdown_connections 2024-09-21T07:53:58.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.767+0000 7f32c6800640 1 --2- 172.21.15.39:0/4215838394 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f329c078b90 0x7f329c07b0a0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:58.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.767+0000 7f32c6800640 1 --2- 172.21.15.39:0/4215838394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32c015cd30 0x7f32c00aeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:58.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.767+0000 7f32c6800640 1 --2- 172.21.15.39:0/4215838394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32c006fc10 0x7f32c0159910 unknown :-1 s=CLOSED pgs=2675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:58.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.767+0000 7f32c6800640 1 --2- 172.21.15.39:0/4215838394 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32c006f800 0x7f32c01f9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:58.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.767+0000 7f32c6800640 1 -- 172.21.15.39:0/4215838394 >> 172.21.15.39:0/4215838394 conn(0x7f32c006cfc0 msgr2=0x7f32c015bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:58.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.767+0000 7f32c6800640 1 -- 172.21.15.39:0/4215838394 shutdown_connections 2024-09-21T07:53:58.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:58.767+0000 7f32c6800640 1 -- 172.21.15.39:0/4215838394 wait complete. 2024-09-21T07:53:58.801 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T07:53:58.801 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T07:53:58.801 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T07:53:58.801 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:53:58.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:53:58.802 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/daemon/state.py", line 146, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi148 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b' 2024-09-21T07:53:58.802 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T07:53:58.802 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T07:53:58.806 INFO:tasks.ceph.mds.b:Started 2024-09-21T07:53:58.807 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:53:58.999 INFO:tasks.ceph.mds.b.smithi148.stdout:starting mds.b at 2024-09-21T07:53:59.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.033+0000 7f12a2e00640 1 -- 172.21.15.39:0/3039691954 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f129c06fde0 msgr2=0x7f129c072fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:59.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.033+0000 7f12a2e00640 1 --2- 172.21.15.39:0/3039691954 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f129c06fde0 0x7f129c072fa0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1294005fd0 tx=0x7f129402aaa0 comp rx=0 tx=0).stop 2024-09-21T07:53:59.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.033+0000 7f12a2e00640 1 -- 172.21.15.39:0/3039691954 shutdown_connections 2024-09-21T07:53:59.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.033+0000 7f12a2e00640 1 --2- 172.21.15.39:0/3039691954 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f129c06fde0 0x7f129c072fa0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:59.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.033+0000 7f12a2e00640 1 --2- 172.21.15.39:0/3039691954 >> v2:172.21.15.148:3300/0 conn(0x7f129c15c830 0x7f129c06f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:59.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.033+0000 7f12a2e00640 1 -- 172.21.15.39:0/3039691954 >> 172.21.15.39:0/3039691954 conn(0x7f129c0b2510 msgr2=0x7f129c0d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:59.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.033+0000 7f12a2e00640 1 -- 172.21.15.39:0/3039691954 shutdown_connections 2024-09-21T07:53:59.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.033+0000 7f12a2e00640 1 -- 172.21.15.39:0/3039691954 wait complete. 2024-09-21T07:53:59.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.036+0000 7f12a2e00640 1 Processor -- start 2024-09-21T07:53:59.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.036+0000 7f12a2e00640 1 -- start start 2024-09-21T07:53:59.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.037+0000 7f12a2e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f129c06fde0 0x7f129c1e8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:59.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.037+0000 7f12a2e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f129c0d0f20 0x7f129c1ea890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:59.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.037+0000 7f12a2e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f129c15c830 0x7f129c1f8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:59.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.038+0000 7f12a2e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f129c0b4570 con 0x7f129c0d0f20 2024-09-21T07:53:59.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.038+0000 7f12a2e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f129c0b43f0 con 0x7f129c06fde0 2024-09-21T07:53:59.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.038+0000 7f12a2e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f129c0b46f0 con 0x7f129c15c830 2024-09-21T07:53:59.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.038+0000 7f12a1a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f129c06fde0 0x7f129c1e8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:59.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.038+0000 7f12a1000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f129c0d0f20 0x7f129c1ea890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:59.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.038+0000 7f12a1a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f129c06fde0 0x7f129c1e8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.39:44022/0 (socket says 172.21.15.39:44022) 2024-09-21T07:53:59.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.038+0000 7f12a2400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f129c15c830 0x7f129c1f8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:59.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.038+0000 7f12a1a00640 1 -- 172.21.15.39:0/4266154225 learned_addr learned my addr 172.21.15.39:0/4266154225 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:53:59.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.038+0000 7f12a2400640 1 -- 172.21.15.39:0/4266154225 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f129c06fde0 msgr2=0x7f129c1e8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:59.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.038+0000 7f12a2400640 1 --2- 172.21.15.39:0/4266154225 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f129c06fde0 0x7f129c1e8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:59.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.038+0000 7f12a2400640 1 -- 172.21.15.39:0/4266154225 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f129c0d0f20 msgr2=0x7f129c1ea890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:59.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.038+0000 7f12a2400640 1 --2- 172.21.15.39:0/4266154225 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f129c0d0f20 0x7f129c1ea890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:59.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.038+0000 7f12a2400640 1 -- 172.21.15.39:0/4266154225 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f129c1e7c50 con 0x7f129c15c830 2024-09-21T07:53:59.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.039+0000 7f12a2400640 1 --2- 172.21.15.39:0/4266154225 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f129c15c830 0x7f129c1f8e30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1294032000 tx=0x7f1294003ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:59.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.039+0000 7f1299600640 1 -- 172.21.15.39:0/4266154225 <== mon.2 v2:172.21.15.148:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1294044070 con 0x7f129c15c830 2024-09-21T07:53:59.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.039+0000 7f1299600640 1 -- 172.21.15.39:0/4266154225 <== mon.2 v2:172.21.15.148:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1294003e50 con 0x7f129c15c830 2024-09-21T07:53:59.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.039+0000 7f1299600640 1 -- 172.21.15.39:0/4266154225 <== mon.2 v2:172.21.15.148:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12940044b0 con 0x7f129c15c830 2024-09-21T07:53:59.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.039+0000 7f12a2e00640 1 -- 172.21.15.39:0/4266154225 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f129c1e7f30 con 0x7f129c15c830 2024-09-21T07:53:59.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.039+0000 7f12a2e00640 1 -- 172.21.15.39:0/4266154225 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f129c1e84f0 con 0x7f129c15c830 2024-09-21T07:53:59.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.040+0000 7f12a2e00640 1 -- 172.21.15.39:0/4266154225 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f129c0b4c10 con 0x7f129c15c830 2024-09-21T07:53:59.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.042+0000 7f1299600640 1 -- 172.21.15.39:0/4266154225 <== mon.2 v2:172.21.15.148:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f129402c040 con 0x7f129c15c830 2024-09-21T07:53:59.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.043+0000 7f1299600640 1 --2- 172.21.15.39:0/4266154225 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f127c078b90 0x7f127c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:59.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.043+0000 7f1299600640 1 -- 172.21.15.39:0/4266154225 <== mon.2 v2:172.21.15.148:3301/0 5 ==== osd_map(252..252 src has 1..252) ==== 7992+0+0 (secure 0 0 0) 0x7f12940bc0d0 con 0x7f129c15c830 2024-09-21T07:53:59.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.043+0000 7f12a1a00640 1 --2- 172.21.15.39:0/4266154225 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f127c078b90 0x7f127c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:59.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.044+0000 7f12a1a00640 1 --2- 172.21.15.39:0/4266154225 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f127c078b90 0x7f127c07b0a0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f12900017e0 tx=0x7f1290009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:59.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.047+0000 7f1299600640 1 -- 172.21.15.39:0/4266154225 <== mon.2 v2:172.21.15.148:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1294032030 con 0x7f129c15c830 2024-09-21T07:53:59.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.289+0000 7f12a2e00640 1 -- 172.21.15.39:0/4266154225 --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f129c159040 con 0x7f129c15c830 2024-09-21T07:53:59.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.290+0000 7f1299600640 1 -- 172.21.15.39:0/4266154225 <== mon.2 v2:172.21.15.148:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 297 v297) ==== 95+0+4452 (secure 0 0 0) 0x7f12940835c0 con 0x7f129c15c830 2024-09-21T07:53:59.293 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:53:59.293 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":297,"btime":"2024-09-21T07:53:58:770965+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":297,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:53:58.770946+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":252,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":11364,"mds_1":10633,"mds_2":11970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10633":{"gid":10633,"name":"d","rank":1,"incarnation":295,"state":"up:resolve","state_seq":14,"addr":"172.21.15.148:6837/3495264983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3495264983},{"type":"v1","addr":"172.21.15.148:6837","nonce":3495264983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11364":{"gid":11364,"name":"a","rank":0,"incarnation":285,"state":"up:active","state_seq":28,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11970":{"gid":11970,"name":"c","rank":2,"incarnation":297,"state":"up:replay","state_seq":1,"addr":"172.21.15.39:6835/732408880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":732408880},{"type":"v1","addr":"172.21.15.39:6835","nonce":732408880}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11364,"qdb_cluster":[11364]},"id":13}]} 2024-09-21T07:53:59.293 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 297 2024-09-21T07:53:59.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.293+0000 7f12a2e00640 1 -- 172.21.15.39:0/4266154225 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f127c078b90 msgr2=0x7f127c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:59.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.293+0000 7f12a2e00640 1 --2- 172.21.15.39:0/4266154225 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f127c078b90 0x7f127c07b0a0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f12900017e0 tx=0x7f1290009040 comp rx=0 tx=0).stop 2024-09-21T07:53:59.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.293+0000 7f12a2e00640 1 -- 172.21.15.39:0/4266154225 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f129c15c830 msgr2=0x7f129c1f8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:59.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.293+0000 7f12a2e00640 1 --2- 172.21.15.39:0/4266154225 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f129c15c830 0x7f129c1f8e30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1294032000 tx=0x7f1294003ce0 comp rx=0 tx=0).stop 2024-09-21T07:53:59.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.293+0000 7f12a2e00640 1 -- 172.21.15.39:0/4266154225 shutdown_connections 2024-09-21T07:53:59.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.293+0000 7f12a2e00640 1 --2- 172.21.15.39:0/4266154225 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f127c078b90 0x7f127c07b0a0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:59.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.293+0000 7f12a2e00640 1 --2- 172.21.15.39:0/4266154225 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f129c15c830 0x7f129c1f8e30 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:59.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.293+0000 7f12a2e00640 1 --2- 172.21.15.39:0/4266154225 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f129c0d0f20 0x7f129c1ea890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:59.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.293+0000 7f12a2e00640 1 --2- 172.21.15.39:0/4266154225 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f129c06fde0 0x7f129c1e8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:59.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.293+0000 7f12a2e00640 1 -- 172.21.15.39:0/4266154225 >> 172.21.15.39:0/4266154225 conn(0x7f129c0b2510 msgr2=0x7f129c06d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:59.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.293+0000 7f12a2e00640 1 -- 172.21.15.39:0/4266154225 shutdown_connections 2024-09-21T07:53:59.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:53:59.293+0000 7f12a2e00640 1 -- 172.21.15.39:0/4266154225 wait complete. 2024-09-21T07:53:59.321 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-09-21T07:54:04.167 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:54:04.173 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:54:04.322 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:54:04.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.550+0000 7f6a79200640 1 -- 172.21.15.39:0/520454886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a7415cd30 msgr2=0x7f6a740cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:04.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.550+0000 7f6a79200640 1 --2- 172.21.15.39:0/520454886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a7415cd30 0x7f6a740cfd10 secure :-1 s=READY pgs=2677 cs=0 l=1 rev1=1 crypto rx=0x7f6a64009830 tx=0x7f6a6402a510 comp rx=0 tx=0).stop 2024-09-21T07:54:04.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.550+0000 7f6a79200640 1 -- 172.21.15.39:0/520454886 shutdown_connections 2024-09-21T07:54:04.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.550+0000 7f6a79200640 1 --2- 172.21.15.39:0/520454886 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a7406f800 0x7f6a740cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:04.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.550+0000 7f6a79200640 1 --2- 172.21.15.39:0/520454886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a7415cd30 0x7f6a740cfd10 unknown :-1 s=CLOSED pgs=2677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:04.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.550+0000 7f6a79200640 1 -- 172.21.15.39:0/520454886 >> 172.21.15.39:0/520454886 conn(0x7f6a7406cfc0 msgr2=0x7f6a7406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:04.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.551+0000 7f6a79200640 1 -- 172.21.15.39:0/520454886 shutdown_connections 2024-09-21T07:54:04.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.551+0000 7f6a79200640 1 -- 172.21.15.39:0/520454886 wait complete. 2024-09-21T07:54:04.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.553+0000 7f6a79200640 1 Processor -- start 2024-09-21T07:54:04.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.553+0000 7f6a79200640 1 -- start start 2024-09-21T07:54:04.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.554+0000 7f6a79200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a7406f800 0x7f6a74159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:04.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.554+0000 7f6a79200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a7406fc10 0x7f6a741f6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:04.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.554+0000 7f6a79200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a7415cd30 0x7f6a741f72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:04.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.554+0000 7f6a79200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6a74073890 con 0x7f6a7415cd30 2024-09-21T07:54:04.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.554+0000 7f6a79200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6a74073710 con 0x7f6a7406f800 2024-09-21T07:54:04.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.554+0000 7f6a79200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f6a74073a10 con 0x7f6a7406fc10 2024-09-21T07:54:04.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.555+0000 7f6a72800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a7415cd30 0x7f6a741f72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:04.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.555+0000 7f6a72800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a7415cd30 0x7f6a741f72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55250/0 (socket says 172.21.15.39:55250) 2024-09-21T07:54:04.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.555+0000 7f6a72800640 1 -- 172.21.15.39:0/623580631 learned_addr learned my addr 172.21.15.39:0/623580631 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:04.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.555+0000 7f6a71e00640 1 --2- 172.21.15.39:0/623580631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a7406f800 0x7f6a74159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:04.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.555+0000 7f6a69400640 1 --2- 172.21.15.39:0/623580631 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a7406fc10 0x7f6a741f6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:04.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.555+0000 7f6a72800640 1 -- 172.21.15.39:0/623580631 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a7406fc10 msgr2=0x7f6a741f6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:04.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.555+0000 7f6a72800640 1 --2- 172.21.15.39:0/623580631 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a7406fc10 0x7f6a741f6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:04.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.555+0000 7f6a72800640 1 -- 172.21.15.39:0/623580631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a7406f800 msgr2=0x7f6a74159410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:04.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.555+0000 7f6a72800640 1 --2- 172.21.15.39:0/623580631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a7406f800 0x7f6a74159410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:04.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.555+0000 7f6a72800640 1 -- 172.21.15.39:0/623580631 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a740ac910 con 0x7f6a7415cd30 2024-09-21T07:54:04.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.556+0000 7f6a72800640 1 --2- 172.21.15.39:0/623580631 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a7415cd30 0x7f6a741f72c0 secure :-1 s=READY pgs=2678 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c00c760 tx=0x7f6a6c00cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:04.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.556+0000 7f6a6aa00640 1 -- 172.21.15.39:0/623580631 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c003b70 con 0x7f6a7415cd30 2024-09-21T07:54:04.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.556+0000 7f6a6aa00640 1 -- 172.21.15.39:0/623580631 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a6c003d10 con 0x7f6a7415cd30 2024-09-21T07:54:04.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.556+0000 7f6a6aa00640 1 -- 172.21.15.39:0/623580631 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c0052d0 con 0x7f6a7415cd30 2024-09-21T07:54:04.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.556+0000 7f6a79200640 1 -- 172.21.15.39:0/623580631 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a740acbf0 con 0x7f6a7415cd30 2024-09-21T07:54:04.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.556+0000 7f6a79200640 1 -- 172.21.15.39:0/623580631 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a740ad1b0 con 0x7f6a7415cd30 2024-09-21T07:54:04.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.557+0000 7f6a79200640 1 -- 172.21.15.39:0/623580631 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a74073ec0 con 0x7f6a7415cd30 2024-09-21T07:54:04.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.558+0000 7f6a6aa00640 1 -- 172.21.15.39:0/623580631 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6a6c005470 con 0x7f6a7415cd30 2024-09-21T07:54:04.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.560+0000 7f6a6aa00640 1 --2- 172.21.15.39:0/623580631 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6a50078b90 0x7f6a5007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:04.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.560+0000 7f6a6aa00640 1 -- 172.21.15.39:0/623580631 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(252..252 src has 1..252) ==== 7992+0+0 (secure 0 0 0) 0x7f6a6c09b1e0 con 0x7f6a7415cd30 2024-09-21T07:54:04.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.560+0000 7f6a71e00640 1 --2- 172.21.15.39:0/623580631 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6a50078b90 0x7f6a5007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:04.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.561+0000 7f6a71e00640 1 --2- 172.21.15.39:0/623580631 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6a50078b90 0x7f6a5007b0a0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f6a64034000 tx=0x7f6a64004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:04.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.564+0000 7f6a6aa00640 1 -- 172.21.15.39:0/623580631 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6a6c062770 con 0x7f6a7415cd30 2024-09-21T07:54:04.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.803+0000 7f6a79200640 1 -- 172.21.15.39:0/623580631 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a74153390 con 0x7f6a7415cd30 2024-09-21T07:54:04.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.803+0000 7f6a6aa00640 1 -- 172.21.15.39:0/623580631 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 299 v299) ==== 95+0+5285 (secure 0 0 0) 0x7f6a6c0671a0 con 0x7f6a7415cd30 2024-09-21T07:54:04.805 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:04.805 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":299,"btime":"2024-09-21T07:54:00:777100+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9467,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3658109005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3658109005},{"type":"v1","addr":"172.21.15.148:6835","nonce":3658109005}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":298}],"filesystems":[{"mdsmap":{"epoch":299,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:53:59.781289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":252,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":11364,"mds_1":10633,"mds_2":11970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10633":{"gid":10633,"name":"d","rank":1,"incarnation":295,"state":"up:reconnect","state_seq":15,"addr":"172.21.15.148:6837/3495264983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3495264983},{"type":"v1","addr":"172.21.15.148:6837","nonce":3495264983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11364":{"gid":11364,"name":"a","rank":0,"incarnation":285,"state":"up:active","state_seq":28,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11970":{"gid":11970,"name":"c","rank":2,"incarnation":297,"state":"up:reconnect","state_seq":3,"addr":"172.21.15.39:6835/732408880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":732408880},{"type":"v1","addr":"172.21.15.39:6835","nonce":732408880}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11364,"qdb_cluster":[11364]},"id":13}]} 2024-09-21T07:54:04.806 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 299 2024-09-21T07:54:04.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.806+0000 7f6a79200640 1 -- 172.21.15.39:0/623580631 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6a50078b90 msgr2=0x7f6a5007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:04.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.806+0000 7f6a79200640 1 --2- 172.21.15.39:0/623580631 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6a50078b90 0x7f6a5007b0a0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f6a64034000 tx=0x7f6a64004730 comp rx=0 tx=0).stop 2024-09-21T07:54:04.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.806+0000 7f6a79200640 1 -- 172.21.15.39:0/623580631 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a7415cd30 msgr2=0x7f6a741f72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:04.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.806+0000 7f6a79200640 1 --2- 172.21.15.39:0/623580631 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a7415cd30 0x7f6a741f72c0 secure :-1 s=READY pgs=2678 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c00c760 tx=0x7f6a6c00cb60 comp rx=0 tx=0).stop 2024-09-21T07:54:04.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.806+0000 7f6a79200640 1 -- 172.21.15.39:0/623580631 shutdown_connections 2024-09-21T07:54:04.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.806+0000 7f6a79200640 1 --2- 172.21.15.39:0/623580631 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6a50078b90 0x7f6a5007b0a0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:04.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.806+0000 7f6a79200640 1 --2- 172.21.15.39:0/623580631 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a7415cd30 0x7f6a741f72c0 unknown :-1 s=CLOSED pgs=2678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:04.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.806+0000 7f6a79200640 1 --2- 172.21.15.39:0/623580631 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a7406fc10 0x7f6a741f6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:04.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.806+0000 7f6a79200640 1 --2- 172.21.15.39:0/623580631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a7406f800 0x7f6a74159410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:04.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.806+0000 7f6a79200640 1 -- 172.21.15.39:0/623580631 >> 172.21.15.39:0/623580631 conn(0x7f6a7406cfc0 msgr2=0x7f6a7415bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:04.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.806+0000 7f6a79200640 1 -- 172.21.15.39:0/623580631 shutdown_connections 2024-09-21T07:54:04.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:04.806+0000 7f6a79200640 1 -- 172.21.15.39:0/623580631 wait complete. 2024-09-21T07:54:04.835 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-21T07:54:04.835 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:54:05.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.061+0000 7f020fe00640 1 -- 172.21.15.39:0/1056544468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f021006df60 msgr2=0x7f021006eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:05.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.061+0000 7f020fe00640 1 --2- 172.21.15.39:0/1056544468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f021006df60 0x7f021006eb10 secure :-1 s=READY pgs=2680 cs=0 l=1 rev1=1 crypto rx=0x7f0208005fd0 tx=0x7f020802a8b0 comp rx=0 tx=0).stop 2024-09-21T07:54:05.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.061+0000 7f020fe00640 1 -- 172.21.15.39:0/1056544468 shutdown_connections 2024-09-21T07:54:05.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.061+0000 7f020fe00640 1 --2- 172.21.15.39:0/1056544468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f021006df60 0x7f021006eb10 unknown :-1 s=CLOSED pgs=2680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:05.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.061+0000 7f020fe00640 1 --2- 172.21.15.39:0/1056544468 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f02100d3000 0x7f021006e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:05.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.061+0000 7f020fe00640 1 -- 172.21.15.39:0/1056544468 >> 172.21.15.39:0/1056544468 conn(0x7f021009d910 msgr2=0x7f021008bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:05.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.061+0000 7f020fe00640 1 -- 172.21.15.39:0/1056544468 shutdown_connections 2024-09-21T07:54:05.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.061+0000 7f020fe00640 1 -- 172.21.15.39:0/1056544468 wait complete. 2024-09-21T07:54:05.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.063+0000 7f020fe00640 1 Processor -- start 2024-09-21T07:54:05.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.063+0000 7f020fe00640 1 -- start start 2024-09-21T07:54:05.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.065+0000 7f020fe00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f021006df60 0x7f02100b23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:05.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.065+0000 7f020fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02100d1c30 0x7f02100cde20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:05.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.065+0000 7f020fe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02100b28f0 0x7f02101f8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:05.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.065+0000 7f020fe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f021006f3e0 con 0x7f02100d1c30 2024-09-21T07:54:05.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.065+0000 7f020fe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f021006f260 con 0x7f02100b28f0 2024-09-21T07:54:05.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.065+0000 7f020fe00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f021006f560 con 0x7f021006df60 2024-09-21T07:54:05.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.065+0000 7f0206000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02100d1c30 0x7f02100cde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:05.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.065+0000 7f0206000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02100d1c30 0x7f02100cde20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55264/0 (socket says 172.21.15.39:55264) 2024-09-21T07:54:05.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.065+0000 7f0206000640 1 -- 172.21.15.39:0/3008441314 learned_addr learned my addr 172.21.15.39:0/3008441314 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:05.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.065+0000 7f020f400640 1 --2- 172.21.15.39:0/3008441314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02100b28f0 0x7f02101f8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:05.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.065+0000 7f020ea00640 1 --2- 172.21.15.39:0/3008441314 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f021006df60 0x7f02100b23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:05.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.065+0000 7f0206000640 1 -- 172.21.15.39:0/3008441314 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f021006df60 msgr2=0x7f02100b23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:05.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.065+0000 7f0206000640 1 --2- 172.21.15.39:0/3008441314 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f021006df60 0x7f02100b23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:05.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.065+0000 7f0206000640 1 -- 172.21.15.39:0/3008441314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02100b28f0 msgr2=0x7f02101f8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:05.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.065+0000 7f0206000640 1 --2- 172.21.15.39:0/3008441314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02100b28f0 0x7f02101f8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:05.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.065+0000 7f0206000640 1 -- 172.21.15.39:0/3008441314 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02101f55b0 con 0x7f02100d1c30 2024-09-21T07:54:05.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.065+0000 7f020f400640 1 --2- 172.21.15.39:0/3008441314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02100b28f0 0x7f02101f8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:54:05.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.065+0000 7f020ea00640 1 --2- 172.21.15.39:0/3008441314 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f021006df60 0x7f02100b23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:54:05.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.066+0000 7f0206000640 1 --2- 172.21.15.39:0/3008441314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02100d1c30 0x7f02100cde20 secure :-1 s=READY pgs=2681 cs=0 l=1 rev1=1 crypto rx=0x7f01f800d6a0 tx=0x7f01f800db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:05.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.066+0000 7f0207400640 1 -- 172.21.15.39:0/3008441314 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f8017070 con 0x7f02100d1c30 2024-09-21T07:54:05.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.066+0000 7f0207400640 1 -- 172.21.15.39:0/3008441314 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01f8009b10 con 0x7f02100d1c30 2024-09-21T07:54:05.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.066+0000 7f0207400640 1 -- 172.21.15.39:0/3008441314 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f8010460 con 0x7f02100d1c30 2024-09-21T07:54:05.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.066+0000 7f020fe00640 1 -- 172.21.15.39:0/3008441314 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02101f5850 con 0x7f02100d1c30 2024-09-21T07:54:05.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.066+0000 7f020fe00640 1 -- 172.21.15.39:0/3008441314 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02101f5e10 con 0x7f02100d1c30 2024-09-21T07:54:05.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.067+0000 7f020fe00640 1 -- 172.21.15.39:0/3008441314 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02100b95b0 con 0x7f02100d1c30 2024-09-21T07:54:05.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.069+0000 7f0207400640 1 -- 172.21.15.39:0/3008441314 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f01f801a070 con 0x7f02100d1c30 2024-09-21T07:54:05.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.069+0000 7f0207400640 1 --2- 172.21.15.39:0/3008441314 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f01ec078b90 0x7f01ec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:05.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.070+0000 7f0207400640 1 -- 172.21.15.39:0/3008441314 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(252..252 src has 1..252) ==== 7992+0+0 (secure 0 0 0) 0x7f01f809b800 con 0x7f02100d1c30 2024-09-21T07:54:05.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.070+0000 7f020ea00640 1 --2- 172.21.15.39:0/3008441314 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f01ec078b90 0x7f01ec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:05.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.071+0000 7f020ea00640 1 --2- 172.21.15.39:0/3008441314 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f01ec078b90 0x7f01ec07b0a0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f0200007fd0 tx=0x7f0200009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:05.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.074+0000 7f0207400640 1 -- 172.21.15.39:0/3008441314 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f02100b95b0 con 0x7f02100d1c30 2024-09-21T07:54:05.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.307+0000 7f020fe00640 1 -- 172.21.15.39:0/3008441314 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f021006e5d0 con 0x7f02100d1c30 2024-09-21T07:54:05.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.308+0000 7f0207400640 1 -- 172.21.15.39:0/3008441314 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 299 v299) ==== 95+0+5285 (secure 0 0 0) 0x7f01f80632b0 con 0x7f02100d1c30 2024-09-21T07:54:05.310 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:05.310 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":299,"btime":"2024-09-21T07:54:00:777100+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9467,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3658109005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3658109005},{"type":"v1","addr":"172.21.15.148:6835","nonce":3658109005}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":298}],"filesystems":[{"mdsmap":{"epoch":299,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:53:59.781289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":252,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":11364,"mds_1":10633,"mds_2":11970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10633":{"gid":10633,"name":"d","rank":1,"incarnation":295,"state":"up:reconnect","state_seq":15,"addr":"172.21.15.148:6837/3495264983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3495264983},{"type":"v1","addr":"172.21.15.148:6837","nonce":3495264983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11364":{"gid":11364,"name":"a","rank":0,"incarnation":285,"state":"up:active","state_seq":28,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11970":{"gid":11970,"name":"c","rank":2,"incarnation":297,"state":"up:reconnect","state_seq":3,"addr":"172.21.15.39:6835/732408880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":732408880},{"type":"v1","addr":"172.21.15.39:6835","nonce":732408880}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11364,"qdb_cluster":[11364]},"id":13}]} 2024-09-21T07:54:05.310 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 299 2024-09-21T07:54:05.311 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-09-21T07:54:05.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.310+0000 7f020fe00640 1 -- 172.21.15.39:0/3008441314 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f01ec078b90 msgr2=0x7f01ec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:05.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.310+0000 7f020fe00640 1 --2- 172.21.15.39:0/3008441314 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f01ec078b90 0x7f01ec07b0a0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f0200007fd0 tx=0x7f0200009040 comp rx=0 tx=0).stop 2024-09-21T07:54:05.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.310+0000 7f020fe00640 1 -- 172.21.15.39:0/3008441314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02100d1c30 msgr2=0x7f02100cde20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:05.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.310+0000 7f020fe00640 1 --2- 172.21.15.39:0/3008441314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02100d1c30 0x7f02100cde20 secure :-1 s=READY pgs=2681 cs=0 l=1 rev1=1 crypto rx=0x7f01f800d6a0 tx=0x7f01f800db60 comp rx=0 tx=0).stop 2024-09-21T07:54:05.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.310+0000 7f020fe00640 1 -- 172.21.15.39:0/3008441314 shutdown_connections 2024-09-21T07:54:05.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.310+0000 7f020fe00640 1 --2- 172.21.15.39:0/3008441314 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f01ec078b90 0x7f01ec07b0a0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:05.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.310+0000 7f020fe00640 1 --2- 172.21.15.39:0/3008441314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02100b28f0 0x7f02101f8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:05.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.310+0000 7f020fe00640 1 --2- 172.21.15.39:0/3008441314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02100d1c30 0x7f02100cde20 unknown :-1 s=CLOSED pgs=2681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:05.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.310+0000 7f020fe00640 1 --2- 172.21.15.39:0/3008441314 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f021006df60 0x7f02100b23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:05.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.310+0000 7f020fe00640 1 -- 172.21.15.39:0/3008441314 >> 172.21.15.39:0/3008441314 conn(0x7f021009d910 msgr2=0x7f02100d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:05.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.310+0000 7f020fe00640 1 -- 172.21.15.39:0/3008441314 shutdown_connections 2024-09-21T07:54:05.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:05.311+0000 7f020fe00640 1 -- 172.21.15.39:0/3008441314 wait complete. 2024-09-21T07:54:05.737 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 299, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:53:59.781289+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 252, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 11364, 'mds_1': 10633, 'mds_2': 11970}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10633': {'gid': 10633, 'name': 'd', 'rank': 1, 'incarnation': 295, 'state': 'up:reconnect', 'state_seq': 15, 'addr': '172.21.15.148:6837/3495264983', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 3495264983}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 3495264983}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11364': {'gid': 11364, 'name': 'a', 'rank': 0, 'incarnation': 285, 'state': 'up:active', 'state_seq': 28, 'addr': '172.21.15.39:6834/2161818466', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 2161818466}, {'type': 'v1', 'addr': '172.21.15.39:6834', 'nonce': 2161818466}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11970': {'gid': 11970, 'name': 'c', 'rank': 2, 'incarnation': 297, 'state': 'up:reconnect', 'state_seq': 3, 'addr': '172.21.15.39:6835/732408880', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 732408880}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 732408880}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 11364, 'qdb_cluster': [11364]} 2024-09-21T07:54:05.737 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10633:up:reconnect 2024-09-21T07:54:06.737 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:54:06.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:06.974+0000 7faa1d600640 1 -- 172.21.15.39:0/2081458337 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa180b4690 msgr2=0x7faa180b4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:06.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:06.974+0000 7faa1d600640 1 --2- 172.21.15.39:0/2081458337 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa180b4690 0x7faa180b4aa0 secure :-1 s=READY pgs=2683 cs=0 l=1 rev1=1 crypto rx=0x7faa10005fd0 tx=0x7faa1002a8b0 comp rx=0 tx=0).stop 2024-09-21T07:54:06.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:06.974+0000 7faa1d600640 1 -- 172.21.15.39:0/2081458337 shutdown_connections 2024-09-21T07:54:06.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:06.974+0000 7faa1d600640 1 --2- 172.21.15.39:0/2081458337 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa180b4690 0x7faa180b4aa0 unknown :-1 s=CLOSED pgs=2683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:06.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:06.974+0000 7faa1d600640 1 -- 172.21.15.39:0/2081458337 >> 172.21.15.39:0/2081458337 conn(0x7faa1806cfc0 msgr2=0x7faa1806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:06.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:06.974+0000 7faa1d600640 1 -- 172.21.15.39:0/2081458337 shutdown_connections 2024-09-21T07:54:06.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:06.974+0000 7faa1d600640 1 -- 172.21.15.39:0/2081458337 wait complete. 2024-09-21T07:54:06.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:06.977+0000 7faa1d600640 1 Processor -- start 2024-09-21T07:54:06.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:06.977+0000 7faa1d600640 1 -- start start 2024-09-21T07:54:06.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:06.978+0000 7faa1d600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faa180b4690 0x7faa1814af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:06.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:06.978+0000 7faa1d600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa1815bd70 0x7faa180b2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:06.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:06.978+0000 7faa1d600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa1815f360 0x7faa180b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:06.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:06.978+0000 7faa1d600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7faa180d07a0 con 0x7faa1815f360 2024-09-21T07:54:06.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:06.978+0000 7faa1d600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7faa180d0620 con 0x7faa1815bd70 2024-09-21T07:54:06.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:06.978+0000 7faa1d600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7faa180d0920 con 0x7faa180b4690 2024-09-21T07:54:06.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:06.978+0000 7faa17400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa1815f360 0x7faa180b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:06.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:06.978+0000 7faa17400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa1815f360 0x7faa180b29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55274/0 (socket says 172.21.15.39:55274) 2024-09-21T07:54:06.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:06.978+0000 7faa17400640 1 -- 172.21.15.39:0/3835295287 learned_addr learned my addr 172.21.15.39:0/3835295287 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:06.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:06.979+0000 7faa16a00640 1 --2- 172.21.15.39:0/3835295287 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faa180b4690 0x7faa1814af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:06.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:06.979+0000 7faa0fe00640 1 --2- 172.21.15.39:0/3835295287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa1815bd70 0x7faa180b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:06.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:06.979+0000 7faa17400640 1 -- 172.21.15.39:0/3835295287 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faa180b4690 msgr2=0x7faa1814af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:06.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:06.979+0000 7faa17400640 1 --2- 172.21.15.39:0/3835295287 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faa180b4690 0x7faa1814af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:06.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:06.979+0000 7faa17400640 1 -- 172.21.15.39:0/3835295287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa1815bd70 msgr2=0x7faa180b2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:06.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:06.979+0000 7faa17400640 1 --2- 172.21.15.39:0/3835295287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa1815bd70 0x7faa180b2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:06.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:06.979+0000 7faa17400640 1 -- 172.21.15.39:0/3835295287 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa18159450 con 0x7faa1815f360 2024-09-21T07:54:06.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:06.979+0000 7faa17400640 1 --2- 172.21.15.39:0/3835295287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa1815f360 0x7faa180b29a0 secure :-1 s=READY pgs=2684 cs=0 l=1 rev1=1 crypto rx=0x7faa10009cd0 tx=0x7faa10003a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:06.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:06.979+0000 7faa0ea00640 1 -- 172.21.15.39:0/3835295287 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa10031b90 con 0x7faa1815f360 2024-09-21T07:54:06.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:06.980+0000 7faa0ea00640 1 -- 172.21.15.39:0/3835295287 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa10031e20 con 0x7faa1815f360 2024-09-21T07:54:06.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:06.980+0000 7faa0ea00640 1 -- 172.21.15.39:0/3835295287 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa100332f0 con 0x7faa1815f360 2024-09-21T07:54:06.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:06.980+0000 7faa1d600640 1 -- 172.21.15.39:0/3835295287 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa18153220 con 0x7faa1815f360 2024-09-21T07:54:06.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:06.980+0000 7faa1d600640 1 -- 172.21.15.39:0/3835295287 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa18158c50 con 0x7faa1815f360 2024-09-21T07:54:06.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:06.981+0000 7faa1d600640 1 -- 172.21.15.39:0/3835295287 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa18152f90 con 0x7faa1815f360 2024-09-21T07:54:06.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:06.982+0000 7faa0ea00640 1 -- 172.21.15.39:0/3835295287 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7faa10033490 con 0x7faa1815f360 2024-09-21T07:54:06.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:06.983+0000 7faa0ea00640 1 --2- 172.21.15.39:0/3835295287 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa9f4078bc0 0x7fa9f407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:06.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:06.983+0000 7faa0ea00640 1 -- 172.21.15.39:0/3835295287 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(252..252 src has 1..252) ==== 7992+0+0 (secure 0 0 0) 0x7faa100bcc20 con 0x7faa1815f360 2024-09-21T07:54:06.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:06.983+0000 7faa16a00640 1 --2- 172.21.15.39:0/3835295287 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa9f4078bc0 0x7fa9f407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:06.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:06.984+0000 7faa16a00640 1 --2- 172.21.15.39:0/3835295287 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa9f4078bc0 0x7fa9f407b0d0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7faa08000f60 tx=0x7faa08009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:06.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:06.988+0000 7faa0ea00640 1 -- 172.21.15.39:0/3835295287 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7faa1003f030 con 0x7faa1815f360 2024-09-21T07:54:07.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:07.227+0000 7faa1d600640 1 -- 172.21.15.39:0/3835295287 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa180d0e40 con 0x7faa1815f360 2024-09-21T07:54:07.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:07.227+0000 7faa0ea00640 1 -- 172.21.15.39:0/3835295287 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 299 v299) ==== 95+0+5285 (secure 0 0 0) 0x7faa10084000 con 0x7faa1815f360 2024-09-21T07:54:07.229 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:07.230 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":299,"btime":"2024-09-21T07:54:00:777100+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9467,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3658109005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3658109005},{"type":"v1","addr":"172.21.15.148:6835","nonce":3658109005}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":298}],"filesystems":[{"mdsmap":{"epoch":299,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:53:59.781289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":252,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":11364,"mds_1":10633,"mds_2":11970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10633":{"gid":10633,"name":"d","rank":1,"incarnation":295,"state":"up:reconnect","state_seq":15,"addr":"172.21.15.148:6837/3495264983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3495264983},{"type":"v1","addr":"172.21.15.148:6837","nonce":3495264983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11364":{"gid":11364,"name":"a","rank":0,"incarnation":285,"state":"up:active","state_seq":28,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11970":{"gid":11970,"name":"c","rank":2,"incarnation":297,"state":"up:reconnect","state_seq":3,"addr":"172.21.15.39:6835/732408880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":732408880},{"type":"v1","addr":"172.21.15.39:6835","nonce":732408880}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11364,"qdb_cluster":[11364]},"id":13}]} 2024-09-21T07:54:07.230 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 299 2024-09-21T07:54:07.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:07.230+0000 7faa1d600640 1 -- 172.21.15.39:0/3835295287 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa9f4078bc0 msgr2=0x7fa9f407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:07.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:07.230+0000 7faa1d600640 1 --2- 172.21.15.39:0/3835295287 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa9f4078bc0 0x7fa9f407b0d0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7faa08000f60 tx=0x7faa08009040 comp rx=0 tx=0).stop 2024-09-21T07:54:07.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:07.230+0000 7faa1d600640 1 -- 172.21.15.39:0/3835295287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa1815f360 msgr2=0x7faa180b29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:07.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:07.230+0000 7faa1d600640 1 --2- 172.21.15.39:0/3835295287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa1815f360 0x7faa180b29a0 secure :-1 s=READY pgs=2684 cs=0 l=1 rev1=1 crypto rx=0x7faa10009cd0 tx=0x7faa10003a10 comp rx=0 tx=0).stop 2024-09-21T07:54:07.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:07.230+0000 7faa1d600640 1 -- 172.21.15.39:0/3835295287 shutdown_connections 2024-09-21T07:54:07.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:07.230+0000 7faa1d600640 1 --2- 172.21.15.39:0/3835295287 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa9f4078bc0 0x7fa9f407b0d0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:07.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:07.230+0000 7faa1d600640 1 --2- 172.21.15.39:0/3835295287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa1815f360 0x7faa180b29a0 unknown :-1 s=CLOSED pgs=2684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:07.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:07.230+0000 7faa1d600640 1 --2- 172.21.15.39:0/3835295287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa1815bd70 0x7faa180b2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:07.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:07.230+0000 7faa1d600640 1 --2- 172.21.15.39:0/3835295287 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7faa180b4690 0x7faa1814af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:07.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:07.230+0000 7faa1d600640 1 -- 172.21.15.39:0/3835295287 >> 172.21.15.39:0/3835295287 conn(0x7faa1806cfc0 msgr2=0x7faa18153620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:07.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:07.230+0000 7faa1d600640 1 -- 172.21.15.39:0/3835295287 shutdown_connections 2024-09-21T07:54:07.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:07.230+0000 7faa1d600640 1 -- 172.21.15.39:0/3835295287 wait complete. 2024-09-21T07:54:07.658 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 299, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:53:59.781289+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 252, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 11364, 'mds_1': 10633, 'mds_2': 11970}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10633': {'gid': 10633, 'name': 'd', 'rank': 1, 'incarnation': 295, 'state': 'up:reconnect', 'state_seq': 15, 'addr': '172.21.15.148:6837/3495264983', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 3495264983}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 3495264983}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11364': {'gid': 11364, 'name': 'a', 'rank': 0, 'incarnation': 285, 'state': 'up:active', 'state_seq': 28, 'addr': '172.21.15.39:6834/2161818466', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 2161818466}, {'type': 'v1', 'addr': '172.21.15.39:6834', 'nonce': 2161818466}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11970': {'gid': 11970, 'name': 'c', 'rank': 2, 'incarnation': 297, 'state': 'up:reconnect', 'state_seq': 3, 'addr': '172.21.15.39:6835/732408880', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 732408880}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 732408880}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 11364, 'qdb_cluster': [11364]} 2024-09-21T07:54:07.658 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10633:up:reconnect 2024-09-21T07:54:08.658 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:54:08.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.896+0000 7f6083000640 1 -- 172.21.15.39:0/1590923514 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607c0ccfa0 msgr2=0x7f607c0cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:08.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.896+0000 7f6083000640 1 --2- 172.21.15.39:0/1590923514 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607c0ccfa0 0x7f607c0cd3b0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f6074005fd0 tx=0x7f607402a980 comp rx=0 tx=0).stop 2024-09-21T07:54:08.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.896+0000 7f6083000640 1 -- 172.21.15.39:0/1590923514 shutdown_connections 2024-09-21T07:54:08.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.896+0000 7f6083000640 1 --2- 172.21.15.39:0/1590923514 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607c0ccfa0 0x7f607c0cd3b0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:08.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.896+0000 7f6083000640 1 --2- 172.21.15.39:0/1590923514 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f607c09ba60 0x7f607c09aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:08.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.896+0000 7f6083000640 1 -- 172.21.15.39:0/1590923514 >> 172.21.15.39:0/1590923514 conn(0x7f607c0b3e40 msgr2=0x7f607c0b4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:08.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.896+0000 7f6083000640 1 -- 172.21.15.39:0/1590923514 shutdown_connections 2024-09-21T07:54:08.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.896+0000 7f6083000640 1 -- 172.21.15.39:0/1590923514 wait complete. 2024-09-21T07:54:08.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.898+0000 7f6083000640 1 Processor -- start 2024-09-21T07:54:08.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.898+0000 7f6083000640 1 -- start start 2024-09-21T07:54:08.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.900+0000 7f6083000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607c09ba60 0x7f607c0b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:08.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.900+0000 7f6083000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f607c0ccfa0 0x7f607c1506d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:08.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.900+0000 7f6083000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f607c15fe20 0x7f607c1e93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:08.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.900+0000 7f6083000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f607c0cdc80 con 0x7f607c15fe20 2024-09-21T07:54:08.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.900+0000 7f6083000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f607c0cdb00 con 0x7f607c09ba60 2024-09-21T07:54:08.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.900+0000 7f6083000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f607c0cde00 con 0x7f607c0ccfa0 2024-09-21T07:54:08.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.900+0000 7f6081c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607c09ba60 0x7f607c0b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:08.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.900+0000 7f6082600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f607c15fe20 0x7f607c1e93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:08.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.901+0000 7f6081200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f607c0ccfa0 0x7f607c1506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:08.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.901+0000 7f6081c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607c09ba60 0x7f607c0b0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.39:41336/0 (socket says 172.21.15.39:41336) 2024-09-21T07:54:08.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.901+0000 7f6082600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f607c15fe20 0x7f607c1e93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55284/0 (socket says 172.21.15.39:55284) 2024-09-21T07:54:08.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.901+0000 7f6081c00640 1 -- 172.21.15.39:0/3924679341 learned_addr learned my addr 172.21.15.39:0/3924679341 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:08.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.901+0000 7f6081200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f607c0ccfa0 0x7f607c1506d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3301/0 says I am v2:172.21.15.39:58042/0 (socket says 172.21.15.39:58042) 2024-09-21T07:54:08.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.901+0000 7f6081c00640 1 -- 172.21.15.39:0/3924679341 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f607c0ccfa0 msgr2=0x7f607c1506d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:08.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.901+0000 7f6081c00640 1 --2- 172.21.15.39:0/3924679341 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f607c0ccfa0 0x7f607c1506d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:08.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.901+0000 7f6081c00640 1 -- 172.21.15.39:0/3924679341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f607c15fe20 msgr2=0x7f607c1e93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:08.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.901+0000 7f6081c00640 1 --2- 172.21.15.39:0/3924679341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f607c15fe20 0x7f607c1e93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:08.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.901+0000 7f6081c00640 1 -- 172.21.15.39:0/3924679341 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f607c06e980 con 0x7f607c09ba60 2024-09-21T07:54:08.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.902+0000 7f6081c00640 1 --2- 172.21.15.39:0/3924679341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607c09ba60 0x7f607c0b0a50 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f607000e9f0 tx=0x7f607000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:08.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.902+0000 7f6079600640 1 -- 172.21.15.39:0/3924679341 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607000cd80 con 0x7f607c09ba60 2024-09-21T07:54:08.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.902+0000 7f6083000640 1 -- 172.21.15.39:0/3924679341 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f607c06ec60 con 0x7f607c09ba60 2024-09-21T07:54:08.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.902+0000 7f6079600640 1 -- 172.21.15.39:0/3924679341 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6070004020 con 0x7f607c09ba60 2024-09-21T07:54:08.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.902+0000 7f6079600640 1 -- 172.21.15.39:0/3924679341 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6070004470 con 0x7f607c09ba60 2024-09-21T07:54:08.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.902+0000 7f6083000640 1 -- 172.21.15.39:0/3924679341 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f607c06f220 con 0x7f607c09ba60 2024-09-21T07:54:08.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.903+0000 7f6083000640 1 -- 172.21.15.39:0/3924679341 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f607c0cd3b0 con 0x7f607c09ba60 2024-09-21T07:54:08.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.905+0000 7f6079600640 1 -- 172.21.15.39:0/3924679341 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6070005030 con 0x7f607c09ba60 2024-09-21T07:54:08.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.906+0000 7f6079600640 1 --2- 172.21.15.39:0/3924679341 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f605c078bc0 0x7f605c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:08.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.906+0000 7f6079600640 1 -- 172.21.15.39:0/3924679341 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(252..252 src has 1..252) ==== 7992+0+0 (secure 0 0 0) 0x7f607009c4c0 con 0x7f607c09ba60 2024-09-21T07:54:08.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.906+0000 7f6081200640 1 --2- 172.21.15.39:0/3924679341 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f605c078bc0 0x7f605c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:08.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.907+0000 7f6081200640 1 --2- 172.21.15.39:0/3924679341 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f605c078bc0 0x7f605c07b0d0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f607c0b66b0 tx=0x7f6068009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:08.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:08.910+0000 7f6079600640 1 -- 172.21.15.39:0/3924679341 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6070063920 con 0x7f607c09ba60 2024-09-21T07:54:09.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:09.152+0000 7f6083000640 1 -- 172.21.15.39:0/3924679341 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f607c0cc2e0 con 0x7f607c09ba60 2024-09-21T07:54:09.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:09.153+0000 7f6079600640 1 -- 172.21.15.39:0/3924679341 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 299 v299) ==== 95+0+5285 (secure 0 0 0) 0x7f6070068350 con 0x7f607c09ba60 2024-09-21T07:54:09.155 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:09.155 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":299,"btime":"2024-09-21T07:54:00:777100+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9467,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3658109005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3658109005},{"type":"v1","addr":"172.21.15.148:6835","nonce":3658109005}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":298}],"filesystems":[{"mdsmap":{"epoch":299,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:53:59.781289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":252,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":11364,"mds_1":10633,"mds_2":11970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10633":{"gid":10633,"name":"d","rank":1,"incarnation":295,"state":"up:reconnect","state_seq":15,"addr":"172.21.15.148:6837/3495264983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3495264983},{"type":"v1","addr":"172.21.15.148:6837","nonce":3495264983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11364":{"gid":11364,"name":"a","rank":0,"incarnation":285,"state":"up:active","state_seq":28,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11970":{"gid":11970,"name":"c","rank":2,"incarnation":297,"state":"up:reconnect","state_seq":3,"addr":"172.21.15.39:6835/732408880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":732408880},{"type":"v1","addr":"172.21.15.39:6835","nonce":732408880}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11364,"qdb_cluster":[11364]},"id":13}]} 2024-09-21T07:54:09.155 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 299 2024-09-21T07:54:09.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:09.155+0000 7f6083000640 1 -- 172.21.15.39:0/3924679341 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f605c078bc0 msgr2=0x7f605c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:09.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:09.155+0000 7f6083000640 1 --2- 172.21.15.39:0/3924679341 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f605c078bc0 0x7f605c07b0d0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f607c0b66b0 tx=0x7f6068009040 comp rx=0 tx=0).stop 2024-09-21T07:54:09.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:09.155+0000 7f6083000640 1 -- 172.21.15.39:0/3924679341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607c09ba60 msgr2=0x7f607c0b0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:09.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:09.155+0000 7f6083000640 1 --2- 172.21.15.39:0/3924679341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607c09ba60 0x7f607c0b0a50 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f607000e9f0 tx=0x7f607000eeb0 comp rx=0 tx=0).stop 2024-09-21T07:54:09.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:09.155+0000 7f6083000640 1 -- 172.21.15.39:0/3924679341 shutdown_connections 2024-09-21T07:54:09.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:09.155+0000 7f6083000640 1 --2- 172.21.15.39:0/3924679341 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f605c078bc0 0x7f605c07b0d0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:09.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:09.155+0000 7f6083000640 1 --2- 172.21.15.39:0/3924679341 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f607c15fe20 0x7f607c1e93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:09.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:09.155+0000 7f6083000640 1 --2- 172.21.15.39:0/3924679341 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f607c0ccfa0 0x7f607c1506d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:09.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:09.155+0000 7f6083000640 1 --2- 172.21.15.39:0/3924679341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f607c09ba60 0x7f607c0b0a50 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:09.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:09.155+0000 7f6083000640 1 -- 172.21.15.39:0/3924679341 >> 172.21.15.39:0/3924679341 conn(0x7f607c0b3e40 msgr2=0x7f607c0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:09.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:09.156+0000 7f6083000640 1 -- 172.21.15.39:0/3924679341 shutdown_connections 2024-09-21T07:54:09.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:09.156+0000 7f6083000640 1 -- 172.21.15.39:0/3924679341 wait complete. 2024-09-21T07:54:09.584 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 299, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:53:59.781289+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 252, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 11364, 'mds_1': 10633, 'mds_2': 11970}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10633': {'gid': 10633, 'name': 'd', 'rank': 1, 'incarnation': 295, 'state': 'up:reconnect', 'state_seq': 15, 'addr': '172.21.15.148:6837/3495264983', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 3495264983}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 3495264983}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11364': {'gid': 11364, 'name': 'a', 'rank': 0, 'incarnation': 285, 'state': 'up:active', 'state_seq': 28, 'addr': '172.21.15.39:6834/2161818466', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 2161818466}, {'type': 'v1', 'addr': '172.21.15.39:6834', 'nonce': 2161818466}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11970': {'gid': 11970, 'name': 'c', 'rank': 2, 'incarnation': 297, 'state': 'up:reconnect', 'state_seq': 3, 'addr': '172.21.15.39:6835/732408880', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 732408880}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 732408880}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 11364, 'qdb_cluster': [11364]} 2024-09-21T07:54:09.584 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10633:up:reconnect 2024-09-21T07:54:10.585 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:54:10.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:10.820+0000 7f1717e00640 1 -- 172.21.15.39:0/1261575202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17180d0880 msgr2=0x7f17180d0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:10.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:10.820+0000 7f1717e00640 1 --2- 172.21.15.39:0/1261575202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17180d0880 0x7f17180d0c90 secure :-1 s=READY pgs=2685 cs=0 l=1 rev1=1 crypto rx=0x7f1718073e70 tx=0x7f171002a980 comp rx=0 tx=0).stop 2024-09-21T07:54:10.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:10.820+0000 7f1717e00640 1 -- 172.21.15.39:0/1261575202 shutdown_connections 2024-09-21T07:54:10.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:10.820+0000 7f1717e00640 1 --2- 172.21.15.39:0/1261575202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17180d0880 0x7f17180d0c90 unknown :-1 s=CLOSED pgs=2685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:10.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:10.820+0000 7f1717e00640 1 --2- 172.21.15.39:0/1261575202 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f17180b1c90 0x7f17180b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:10.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:10.820+0000 7f1717e00640 1 --2- 172.21.15.39:0/1261575202 >> v2:172.21.15.148:3300/0 conn(0x7f17180b1740 0x7f17180b77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:10.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:10.820+0000 7f1717e00640 1 -- 172.21.15.39:0/1261575202 >> 172.21.15.39:0/1261575202 conn(0x7f17180cee40 msgr2=0x7f17180cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:10.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:10.821+0000 7f1717e00640 1 -- 172.21.15.39:0/1261575202 shutdown_connections 2024-09-21T07:54:10.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:10.821+0000 7f1717e00640 1 -- 172.21.15.39:0/1261575202 wait complete. 2024-09-21T07:54:10.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:10.823+0000 7f1717e00640 1 Processor -- start 2024-09-21T07:54:10.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:10.823+0000 7f1717e00640 1 -- start start 2024-09-21T07:54:10.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:10.824+0000 7f1717e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f17180b1740 0x7f17181f95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:10.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:10.824+0000 7f1717e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17180b1c90 0x7f171815d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:10.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:10.824+0000 7f1717e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17180d0880 0x7f17181f5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:10.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:10.825+0000 7f1717e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f17180b7eb0 con 0x7f17180d0880 2024-09-21T07:54:10.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:10.825+0000 7f1717e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f17180b7d30 con 0x7f17180b1c90 2024-09-21T07:54:10.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:10.825+0000 7f1717e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f1718160980 con 0x7f17180b1740 2024-09-21T07:54:10.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:10.825+0000 7f1717400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17180d0880 0x7f17181f5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:10.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:10.825+0000 7f1717400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17180d0880 0x7f17181f5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55302/0 (socket says 172.21.15.39:55302) 2024-09-21T07:54:10.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:10.825+0000 7f1717400640 1 -- 172.21.15.39:0/3010771264 learned_addr learned my addr 172.21.15.39:0/3010771264 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:10.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:10.825+0000 7f1717400640 1 -- 172.21.15.39:0/3010771264 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f17180b1740 msgr2=0x7f17181f95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:10.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:10.825+0000 7f1717400640 1 --2- 172.21.15.39:0/3010771264 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f17180b1740 0x7f17181f95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:10.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:10.825+0000 7f1717400640 1 -- 172.21.15.39:0/3010771264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17180b1c90 msgr2=0x7f171815d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:10.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:10.825+0000 7f1716000640 1 --2- 172.21.15.39:0/3010771264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17180b1c90 0x7f171815d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:10.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:10.825+0000 7f1717400640 1 --2- 172.21.15.39:0/3010771264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17180b1c90 0x7f171815d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:10.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:10.825+0000 7f1717400640 1 -- 172.21.15.39:0/3010771264 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17181f5ff0 con 0x7f17180d0880 2024-09-21T07:54:10.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:10.825+0000 7f1716000640 1 --2- 172.21.15.39:0/3010771264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17180b1c90 0x7f171815d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:54:10.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:10.826+0000 7f1717400640 1 --2- 172.21.15.39:0/3010771264 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17180d0880 0x7f17181f5500 secure :-1 s=READY pgs=2686 cs=0 l=1 rev1=1 crypto rx=0x7f1710009d60 tx=0x7f1710003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:10.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:10.826+0000 7f170ea00640 1 -- 172.21.15.39:0/3010771264 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1710034040 con 0x7f17180d0880 2024-09-21T07:54:10.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:10.826+0000 7f170ea00640 1 -- 172.21.15.39:0/3010771264 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1710004540 con 0x7f17180d0880 2024-09-21T07:54:10.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:10.826+0000 7f170ea00640 1 -- 172.21.15.39:0/3010771264 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1710045480 con 0x7f17180d0880 2024-09-21T07:54:10.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:10.826+0000 7f1717e00640 1 -- 172.21.15.39:0/3010771264 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17181f62d0 con 0x7f17180d0880 2024-09-21T07:54:10.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:10.826+0000 7f1717e00640 1 -- 172.21.15.39:0/3010771264 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17181f8dd0 con 0x7f17180d0880 2024-09-21T07:54:10.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:10.827+0000 7f1717e00640 1 -- 172.21.15.39:0/3010771264 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17181f5d60 con 0x7f17180d0880 2024-09-21T07:54:10.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:10.828+0000 7f170ea00640 1 -- 172.21.15.39:0/3010771264 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1710037020 con 0x7f17180d0880 2024-09-21T07:54:10.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:10.829+0000 7f170ea00640 1 --2- 172.21.15.39:0/3010771264 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f16f8078b90 0x7f16f807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:10.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:10.829+0000 7f170ea00640 1 -- 172.21.15.39:0/3010771264 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(253..253 src has 1..253) ==== 8035+0+0 (secure 0 0 0) 0x7f17100bd060 con 0x7f17180d0880 2024-09-21T07:54:10.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:10.830+0000 7f1716a00640 1 --2- 172.21.15.39:0/3010771264 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f16f8078b90 0x7f16f807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:10.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:10.831+0000 7f1716a00640 1 --2- 172.21.15.39:0/3010771264 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f16f8078b90 0x7f16f807b0a0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f1708005d20 tx=0x7f1708005c30 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:10.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:10.834+0000 7f170ea00640 1 -- 172.21.15.39:0/3010771264 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1710084af0 con 0x7f17180d0880 2024-09-21T07:54:11.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.070+0000 7f1717e00640 1 -- 172.21.15.39:0/3010771264 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f171815d650 con 0x7f17180d0880 2024-09-21T07:54:11.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.071+0000 7f170ea00640 1 -- 172.21.15.39:0/3010771264 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 301 v301) ==== 95+0+5291 (secure 0 0 0) 0x7f17100843f0 con 0x7f17180d0880 2024-09-21T07:54:11.073 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:11.074 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":301,"btime":"2024-09-21T07:54:11:059441+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9467,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3658109005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3658109005},{"type":"v1","addr":"172.21.15.148:6835","nonce":3658109005}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":298}],"filesystems":[{"mdsmap":{"epoch":301,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:54:11.059432+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":252,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":11364,"mds_1":10633,"mds_2":11970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10633":{"gid":10633,"name":"d","rank":1,"incarnation":295,"state":"up:active","state_seq":19,"addr":"172.21.15.148:6837/3495264983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3495264983},{"type":"v1","addr":"172.21.15.148:6837","nonce":3495264983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11364":{"gid":11364,"name":"a","rank":0,"incarnation":285,"state":"up:active","state_seq":28,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11970":{"gid":11970,"name":"c","rank":2,"incarnation":297,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6835/732408880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":732408880},{"type":"v1","addr":"172.21.15.39:6835","nonce":732408880}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11364,"qdb_cluster":[10633,11364,11970]},"id":13}]} 2024-09-21T07:54:11.074 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 301 2024-09-21T07:54:11.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.074+0000 7f1717e00640 1 -- 172.21.15.39:0/3010771264 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f16f8078b90 msgr2=0x7f16f807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:11.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.074+0000 7f1717e00640 1 --2- 172.21.15.39:0/3010771264 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f16f8078b90 0x7f16f807b0a0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f1708005d20 tx=0x7f1708005c30 comp rx=0 tx=0).stop 2024-09-21T07:54:11.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.074+0000 7f1717e00640 1 -- 172.21.15.39:0/3010771264 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17180d0880 msgr2=0x7f17181f5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:11.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.074+0000 7f1717e00640 1 --2- 172.21.15.39:0/3010771264 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17180d0880 0x7f17181f5500 secure :-1 s=READY pgs=2686 cs=0 l=1 rev1=1 crypto rx=0x7f1710009d60 tx=0x7f1710003f10 comp rx=0 tx=0).stop 2024-09-21T07:54:11.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.074+0000 7f1717e00640 1 -- 172.21.15.39:0/3010771264 shutdown_connections 2024-09-21T07:54:11.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.074+0000 7f1717e00640 1 --2- 172.21.15.39:0/3010771264 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f16f8078b90 0x7f16f807b0a0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.074+0000 7f1717e00640 1 --2- 172.21.15.39:0/3010771264 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17180d0880 0x7f17181f5500 unknown :-1 s=CLOSED pgs=2686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.074+0000 7f1717e00640 1 --2- 172.21.15.39:0/3010771264 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f17180b1c90 0x7f171815d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.074+0000 7f1717e00640 1 --2- 172.21.15.39:0/3010771264 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f17180b1740 0x7f17181f95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.074+0000 7f1717e00640 1 -- 172.21.15.39:0/3010771264 >> 172.21.15.39:0/3010771264 conn(0x7f17180cee40 msgr2=0x7f17180afa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:11.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.074+0000 7f1717e00640 1 -- 172.21.15.39:0/3010771264 shutdown_connections 2024-09-21T07:54:11.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.074+0000 7f1717e00640 1 -- 172.21.15.39:0/3010771264 wait complete. 2024-09-21T07:54:11.503 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 301, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2024-09-21T07:49:34.636616+0000', 'modified': '2024-09-21T07:54:11.059432+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 252, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 11364, 'mds_1': 10633, 'mds_2': 11970}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10633': {'gid': 10633, 'name': 'd', 'rank': 1, 'incarnation': 295, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.148:6837/3495264983', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 3495264983}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 3495264983}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11364': {'gid': 11364, 'name': 'a', 'rank': 0, 'incarnation': 285, 'state': 'up:active', 'state_seq': 28, 'addr': '172.21.15.39:6834/2161818466', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 2161818466}, {'type': 'v1', 'addr': '172.21.15.39:6834', 'nonce': 2161818466}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_11970': {'gid': 11970, 'name': 'c', 'rank': 2, 'incarnation': 297, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.39:6835/732408880', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 732408880}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 732408880}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 11364, 'qdb_cluster': [10633, 11364, 11970]} 2024-09-21T07:54:11.503 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-21T07:54:11.504 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-21T07:54:11.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.735+0000 7f58bf800640 1 -- 172.21.15.39:0/103056988 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58b815cd30 msgr2=0x7f58b80cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:11.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.735+0000 7f58bf800640 1 --2- 172.21.15.39:0/103056988 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58b815cd30 0x7f58b80cfd10 secure :-1 s=READY pgs=2688 cs=0 l=1 rev1=1 crypto rx=0x7f58a8009830 tx=0x7f58a802a510 comp rx=0 tx=0).stop 2024-09-21T07:54:11.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.735+0000 7f58bf800640 1 -- 172.21.15.39:0/103056988 shutdown_connections 2024-09-21T07:54:11.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.735+0000 7f58bf800640 1 --2- 172.21.15.39:0/103056988 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f58b806fc10 0x7f58b80d0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.735+0000 7f58bf800640 1 --2- 172.21.15.39:0/103056988 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58b815cd30 0x7f58b80cfd10 unknown :-1 s=CLOSED pgs=2688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.735+0000 7f58bf800640 1 -- 172.21.15.39:0/103056988 >> 172.21.15.39:0/103056988 conn(0x7f58b806cfc0 msgr2=0x7f58b806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:11.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.736+0000 7f58bf800640 1 -- 172.21.15.39:0/103056988 shutdown_connections 2024-09-21T07:54:11.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.736+0000 7f58bf800640 1 -- 172.21.15.39:0/103056988 wait complete. 2024-09-21T07:54:11.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.738+0000 7f58bf800640 1 Processor -- start 2024-09-21T07:54:11.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.738+0000 7f58bf800640 1 -- start start 2024-09-21T07:54:11.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.739+0000 7f58bf800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58b806f800 0x7f58b81ef7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:11.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.739+0000 7f58bf800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58b806fc10 0x7f58b80b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:11.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.739+0000 7f58bf800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f58b815cd30 0x7f58b81f8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:11.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.739+0000 7f58bf800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f58b8073890 con 0x7f58b806fc10 2024-09-21T07:54:11.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.739+0000 7f58bf800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f58b8073710 con 0x7f58b806f800 2024-09-21T07:54:11.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.739+0000 7f58bf800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f58b8073a10 con 0x7f58b815cd30 2024-09-21T07:54:11.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.740+0000 7f58afe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58b806fc10 0x7f58b80b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:11.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.740+0000 7f58afe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58b806fc10 0x7f58b80b1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55324/0 (socket says 172.21.15.39:55324) 2024-09-21T07:54:11.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.740+0000 7f58afe00640 1 -- 172.21.15.39:0/238325658 learned_addr learned my addr 172.21.15.39:0/238325658 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:11.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.740+0000 7f58bce00640 1 --2- 172.21.15.39:0/238325658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58b806f800 0x7f58b81ef7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:11.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.740+0000 7f58bd800640 1 --2- 172.21.15.39:0/238325658 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f58b815cd30 0x7f58b81f8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:11.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.740+0000 7f58afe00640 1 -- 172.21.15.39:0/238325658 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f58b815cd30 msgr2=0x7f58b81f8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:11.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.740+0000 7f58afe00640 1 --2- 172.21.15.39:0/238325658 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f58b815cd30 0x7f58b81f8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.740+0000 7f58afe00640 1 -- 172.21.15.39:0/238325658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58b806f800 msgr2=0x7f58b81ef7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:11.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.740+0000 7f58afe00640 1 --2- 172.21.15.39:0/238325658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58b806f800 0x7f58b81ef7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.740+0000 7f58afe00640 1 -- 172.21.15.39:0/238325658 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b80b0bd0 con 0x7f58b806fc10 2024-09-21T07:54:11.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.741+0000 7f58afe00640 1 --2- 172.21.15.39:0/238325658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58b806fc10 0x7f58b80b1d60 secure :-1 s=READY pgs=2689 cs=0 l=1 rev1=1 crypto rx=0x7f58a000c3f0 tx=0x7f58a000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:11.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.741+0000 7f58b5600640 1 -- 172.21.15.39:0/238325658 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a0017070 con 0x7f58b806fc10 2024-09-21T07:54:11.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.741+0000 7f58b5600640 1 -- 172.21.15.39:0/238325658 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58a00035f0 con 0x7f58b806fc10 2024-09-21T07:54:11.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.741+0000 7f58b5600640 1 -- 172.21.15.39:0/238325658 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a0003a40 con 0x7f58b806fc10 2024-09-21T07:54:11.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.741+0000 7f58bf800640 1 -- 172.21.15.39:0/238325658 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58b80b0eb0 con 0x7f58b806fc10 2024-09-21T07:54:11.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.741+0000 7f58bf800640 1 -- 172.21.15.39:0/238325658 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58b80b1470 con 0x7f58b806fc10 2024-09-21T07:54:11.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.744+0000 7f58b5600640 1 -- 172.21.15.39:0/238325658 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f58a001a070 con 0x7f58b806fc10 2024-09-21T07:54:11.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.745+0000 7f58b5600640 1 --2- 172.21.15.39:0/238325658 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5894078b90 0x7f589407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:11.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.745+0000 7f58b5600640 1 -- 172.21.15.39:0/238325658 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(253..253 src has 1..253) ==== 8035+0+0 (secure 0 0 0) 0x7f58a009a900 con 0x7f58b806fc10 2024-09-21T07:54:11.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.745+0000 7f58c6a52740 1 Processor -- start 2024-09-21T07:54:11.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.745+0000 7f58bce00640 1 --2- 172.21.15.39:0/238325658 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5894078b90 0x7f589407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:11.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.745+0000 7f58c6a52740 1 -- start start 2024-09-21T07:54:11.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.746+0000 7f58bce00640 1 --2- 172.21.15.39:0/238325658 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5894078b90 0x7f589407b0a0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f58a8034000 tx=0x7f58a8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:11.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.747+0000 7f58c6a52740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846f6d270 0x55884733d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:11.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.747+0000 7f58c6a52740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558846f443b0 0x55884733deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:11.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.747+0000 7f58c6a52740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558846f4c590 0x55884730df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:11.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.747+0000 7f58c6a52740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558847297430 con 0x558846f6d270 2024-09-21T07:54:11.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.747+0000 7f58c6a52740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5588472875d0 con 0x558846f443b0 2024-09-21T07:54:11.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.747+0000 7f58c6a52740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x558847276a00 con 0x558846f4c590 2024-09-21T07:54:11.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.747+0000 7f58bce00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846f6d270 0x55884733d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:11.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.747+0000 7f58bce00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846f6d270 0x55884733d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55330/0 (socket says 172.21.15.39:55330) 2024-09-21T07:54:11.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.747+0000 7f58bce00640 1 -- 172.21.15.39:0/1161735401 learned_addr learned my addr 172.21.15.39:0/1161735401 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:11.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.747+0000 7f58afe00640 1 --2- 172.21.15.39:0/1161735401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558846f443b0 0x55884733deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:11.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.747+0000 7f58bd800640 1 --2- 172.21.15.39:0/1161735401 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558846f4c590 0x55884730df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:11.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.747+0000 7f58bce00640 1 -- 172.21.15.39:0/1161735401 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558846f4c590 msgr2=0x55884730df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:11.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.747+0000 7f58bce00640 1 --2- 172.21.15.39:0/1161735401 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558846f4c590 0x55884730df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.748+0000 7f58bce00640 1 -- 172.21.15.39:0/1161735401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558846f443b0 msgr2=0x55884733deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:11.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.748+0000 7f58bce00640 1 --2- 172.21.15.39:0/1161735401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558846f443b0 0x55884733deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.748+0000 7f58bce00640 1 -- 172.21.15.39:0/1161735401 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55884723f520 con 0x558846f6d270 2024-09-21T07:54:11.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.748+0000 7f58bce00640 1 --2- 172.21.15.39:0/1161735401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846f6d270 0x55884733d970 secure :-1 s=READY pgs=2690 cs=0 l=1 rev1=1 crypto rx=0x7f58a802ab00 tx=0x7f58a803e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e63daf9a1ed0428 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:11.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.748+0000 7f58ad600640 1 -- 172.21.15.39:0/1161735401 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a8003b40 con 0x558846f6d270 2024-09-21T07:54:11.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.748+0000 7f58ad600640 1 -- 172.21.15.39:0/1161735401 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58a8003ce0 con 0x558846f6d270 2024-09-21T07:54:11.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.748+0000 7f58ad600640 1 -- 172.21.15.39:0/1161735401 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a8045660 con 0x558846f6d270 2024-09-21T07:54:11.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.749+0000 7f58c6a52740 1 -- 172.21.15.39:0/1161735401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846f6d270 msgr2=0x55884733d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:11.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.749+0000 7f58c6a52740 1 --2- 172.21.15.39:0/1161735401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846f6d270 0x55884733d970 secure :-1 s=READY pgs=2690 cs=0 l=1 rev1=1 crypto rx=0x7f58a802ab00 tx=0x7f58a803e040 comp rx=0 tx=0).stop 2024-09-21T07:54:11.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.749+0000 7f58c6a52740 1 -- 172.21.15.39:0/1161735401 shutdown_connections 2024-09-21T07:54:11.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.749+0000 7f58c6a52740 1 --2- 172.21.15.39:0/1161735401 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558846f4c590 0x55884730df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.749+0000 7f58c6a52740 1 --2- 172.21.15.39:0/1161735401 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558846f443b0 0x55884733deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.749+0000 7f58c6a52740 1 --2- 172.21.15.39:0/1161735401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846f6d270 0x55884733d970 unknown :-1 s=CLOSED pgs=2690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.749+0000 7f58c6a52740 1 -- 172.21.15.39:0/1161735401 >> 172.21.15.39:0/1161735401 conn(0x558846f613c0 msgr2=0x558847313970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:11.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.749+0000 7f58c6a52740 1 -- 172.21.15.39:0/1161735401 shutdown_connections 2024-09-21T07:54:11.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.749+0000 7f58c6a52740 1 -- 172.21.15.39:0/1161735401 wait complete. 2024-09-21T07:54:11.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.750+0000 7f58c6a52740 1 -- start start 2024-09-21T07:54:11.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.750+0000 7f58c6a52740 1 Processor -- start 2024-09-21T07:54:11.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.752+0000 7f58c6a52740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558846f443b0 0x5588473535c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:11.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.752+0000 7f58c6a52740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846f4c590 0x55884736b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:11.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.752+0000 7f58c6a52740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558846f6d270 0x558847359820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:11.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.752+0000 7f58c6a52740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55884721d330 con 0x558846f4c590 2024-09-21T07:54:11.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.752+0000 7f58c6a52740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55884727a810 con 0x558846f6d270 2024-09-21T07:54:11.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.752+0000 7f58c6a52740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x558846fe7620 con 0x558846f443b0 2024-09-21T07:54:11.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.752+0000 7f58c6a52740 10 client.0 _ll_register_callbacks cb 0x558847222420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:54:11.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.752+0000 7f58afe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846f4c590 0x55884736b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:11.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.752+0000 7f58afe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846f4c590 0x55884736b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55344/0 (socket says 172.21.15.39:55344) 2024-09-21T07:54:11.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.752+0000 7f58afe00640 1 -- 172.21.15.39:0/1214050605 learned_addr learned my addr 172.21.15.39:0/1214050605 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:11.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.752+0000 7f58bd800640 1 --2- 172.21.15.39:0/1214050605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558846f6d270 0x558847359820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:11.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.752+0000 7f58bce00640 1 --2- 172.21.15.39:0/1214050605 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558846f443b0 0x5588473535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:11.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.753+0000 7f58afe00640 1 -- 172.21.15.39:0/1214050605 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558846f443b0 msgr2=0x5588473535c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:11.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.753+0000 7f58afe00640 1 --2- 172.21.15.39:0/1214050605 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558846f443b0 0x5588473535c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.753+0000 7f58afe00640 1 -- 172.21.15.39:0/1214050605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558846f6d270 msgr2=0x558847359820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:11.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.753+0000 7f58afe00640 1 --2- 172.21.15.39:0/1214050605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558846f6d270 0x558847359820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.753+0000 7f58afe00640 1 -- 172.21.15.39:0/1214050605 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558846fd8850 con 0x558846f4c590 2024-09-21T07:54:11.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.753+0000 7f58afe00640 1 --2- 172.21.15.39:0/1214050605 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846f4c590 0x55884736b300 secure :-1 s=READY pgs=2691 cs=0 l=1 rev1=1 crypto rx=0x7f58a000c3c0 tx=0x7f58a0061560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b90e2c47bec67d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:11.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.753+0000 7f58a6000640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:54:11.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.753+0000 7f58a6000640 1 -- 172.21.15.39:0/1214050605 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a0066820 con 0x558846f4c590 2024-09-21T07:54:11.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.753+0000 7f58a6000640 1 -- 172.21.15.39:0/1214050605 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(253..253 src has 1..253) ==== 8035+0+0 (secure 0 0 0) 0x7f58a0069db0 con 0x558846f4c590 2024-09-21T07:54:11.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.753+0000 7f58c6a52740 1 -- 172.21.15.39:0/1214050605 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58a8003ce0 con 0x558846f4c590 2024-09-21T07:54:11.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.753+0000 7f58a6000640 1 -- 172.21.15.39:0/1214050605 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=301) ==== 24+0+0 (secure 0 0 0) 0x7f58a006b760 con 0x558846f4c590 2024-09-21T07:54:11.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.753+0000 7f58c6a52740 10 client.12036 fetch_fsmap learned FSMap version 301 2024-09-21T07:54:11.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.753+0000 7f58c6a52740 1 -- 172.21.15.39:0/1214050605 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=301,monmap=2+}) -- 0x55884704f6b0 con 0x558846f4c590 2024-09-21T07:54:11.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.754+0000 7f58a6000640 1 -- 172.21.15.39:0/1214050605 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58a8003ce0 con 0x558846f4c590 2024-09-21T07:54:11.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.754+0000 7f58a6000640 1 -- 172.21.15.39:0/1214050605 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 301) ==== 3542+0+0 (secure 0 0 0) 0x7f58a0061050 con 0x558846f4c590 2024-09-21T07:54:11.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.754+0000 7f58c6a52740 10 client.12036 fetch_fsmap finished waiting for FSMap version 301 2024-09-21T07:54:11.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.754+0000 7f58c6a52740 10 client.12036 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T07:54:11.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.754+0000 7f58c6a52740 20 client.12036 populate_metadata read hostname 'smithi039' 2024-09-21T07:54:11.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.754+0000 7f58c6a52740 1 --2- 172.21.15.39:0/1214050605 >> [v2:172.21.15.148:6836/3495264983,v1:172.21.15.148:6837/3495264983] conn(0x558846f2be40 0x558847339550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:11.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.754+0000 7f58c6a52740 4 client.12036 mds_command: new command op to 10633 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:54:11.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.754+0000 7f58c6a52740 1 -- 172.21.15.39:0/1214050605 --> [v2:172.21.15.148:6836/3495264983,v1:172.21.15.148:6837/3495264983] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558847289520 con 0x558846f2be40 2024-09-21T07:54:11.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.754+0000 7f58bce00640 1 --2- 172.21.15.39:0/1214050605 >> [v2:172.21.15.148:6836/3495264983,v1:172.21.15.148:6837/3495264983] conn(0x558846f2be40 0x558847339550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:11.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.755+0000 7f58bce00640 1 --2- 172.21.15.39:0/1214050605 >> [v2:172.21.15.148:6836/3495264983,v1:172.21.15.148:6837/3495264983] conn(0x558846f2be40 0x558847339550 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=13c607b58d3070e6 server_cookie=8c64f3fbbe0a94c0 in_seq=0 out_seq=0 2024-09-21T07:54:11.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.755+0000 7f58a6000640 10 client.12036 ms_handle_connect on v2:172.21.15.148:6836/3495264983 2024-09-21T07:54:11.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.756+0000 7f58a6000640 1 -- 172.21.15.39:0/1214050605 <== mds.1 v2:172.21.15.148:6836/3495264983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558847289520 con 0x558846f2be40 2024-09-21T07:54:11.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.756+0000 7f58a6000640 10 client.12036 handle_command_reply: tid=0 2024-09-21T07:54:11.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.756+0000 7f58c6a52740 1 client.12036 shutdown 2024-09-21T07:54:11.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.756+0000 7f58c6a52740 1 -- 172.21.15.39:0/1214050605 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846f4c590 msgr2=0x55884736b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:11.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.756+0000 7f58c6a52740 1 --2- 172.21.15.39:0/1214050605 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846f4c590 0x55884736b300 secure :-1 s=READY pgs=2691 cs=0 l=1 rev1=1 crypto rx=0x7f58a000c3c0 tx=0x7f58a0061560 comp rx=0 tx=0).stop 2024-09-21T07:54:11.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.756+0000 7f58c6a52740 1 -- 172.21.15.39:0/1214050605 shutdown_connections 2024-09-21T07:54:11.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.756+0000 7f58c6a52740 1 --2- 172.21.15.39:0/1214050605 >> [v2:172.21.15.148:6836/3495264983,v1:172.21.15.148:6837/3495264983] conn(0x558846f2be40 0x558847339550 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.756+0000 7f58c6a52740 1 --2- 172.21.15.39:0/1214050605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558846f6d270 0x558847359820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.756+0000 7f58a6000640 0 client.12036 ms_handle_reset on v2:172.21.15.148:6836/3495264983 2024-09-21T07:54:11.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.756+0000 7f58c6a52740 1 --2- 172.21.15.39:0/1214050605 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846f4c590 0x55884736b300 unknown :-1 s=CLOSED pgs=2691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.756+0000 7f58c6a52740 1 --2- 172.21.15.39:0/1214050605 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558846f443b0 0x5588473535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.756+0000 7f58c6a52740 1 -- 172.21.15.39:0/1214050605 >> 172.21.15.39:0/1214050605 conn(0x558846f613c0 msgr2=0x558847334ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:11.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.757+0000 7f58c6a52740 1 -- 172.21.15.39:0/1214050605 shutdown_connections 2024-09-21T07:54:11.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.757+0000 7f58c6a52740 1 -- 172.21.15.39:0/1214050605 wait complete. 2024-09-21T07:54:11.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.757+0000 7f58c6a52740 20 client.12036 trim_cache size 0 max 16384 2024-09-21T07:54:11.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.776+0000 7f58c6a52740 1 Processor -- start 2024-09-21T07:54:11.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.776+0000 7f58c6a52740 1 -- start start 2024-09-21T07:54:11.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.777+0000 7f58c6a52740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558846f443b0 0x558847334900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:11.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.777+0000 7f58c6a52740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558846f4c590 0x558847334e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:11.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.777+0000 7f58c6a52740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846f6d270 0x55884733c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:11.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.777+0000 7f58c6a52740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f58b8204600 con 0x558846f6d270 2024-09-21T07:54:11.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.777+0000 7f58c6a52740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f58b8204cd0 con 0x558846f443b0 2024-09-21T07:54:11.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.777+0000 7f58c6a52740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f58b8203720 con 0x558846f4c590 2024-09-21T07:54:11.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.778+0000 7f58bd800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846f6d270 0x55884733c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:11.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.778+0000 7f58bd800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846f6d270 0x55884733c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55354/0 (socket says 172.21.15.39:55354) 2024-09-21T07:54:11.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.778+0000 7f58bd800640 1 -- 172.21.15.39:0/1238561849 learned_addr learned my addr 172.21.15.39:0/1238561849 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:11.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.778+0000 7f58bce00640 1 --2- 172.21.15.39:0/1238561849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558846f443b0 0x558847334900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:11.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.778+0000 7f58afe00640 1 --2- 172.21.15.39:0/1238561849 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558846f4c590 0x558847334e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:11.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.778+0000 7f58bd800640 1 -- 172.21.15.39:0/1238561849 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558846f4c590 msgr2=0x558847334e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:11.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.778+0000 7f58bd800640 1 --2- 172.21.15.39:0/1238561849 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558846f4c590 0x558847334e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.778+0000 7f58bd800640 1 -- 172.21.15.39:0/1238561849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558846f443b0 msgr2=0x558847334900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:11.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.778+0000 7f58bd800640 1 --2- 172.21.15.39:0/1238561849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558846f443b0 0x558847334900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.778+0000 7f58bd800640 1 -- 172.21.15.39:0/1238561849 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558847314650 con 0x558846f6d270 2024-09-21T07:54:11.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.778+0000 7f58bd800640 1 --2- 172.21.15.39:0/1238561849 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846f6d270 0x55884733c6b0 secure :-1 s=READY pgs=2692 cs=0 l=1 rev1=1 crypto rx=0x7f58b000afc0 tx=0x7f58b00108c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36f9afd1a959d17 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:11.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.779+0000 7f58acc00640 1 -- 172.21.15.39:0/1238561849 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b0004d20 con 0x558846f6d270 2024-09-21T07:54:11.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.779+0000 7f58acc00640 1 -- 172.21.15.39:0/1238561849 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58b000f6f0 con 0x558846f6d270 2024-09-21T07:54:11.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.779+0000 7f58acc00640 1 -- 172.21.15.39:0/1238561849 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b000fb40 con 0x558846f6d270 2024-09-21T07:54:11.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.780+0000 7f58c6a52740 1 -- 172.21.15.39:0/1238561849 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846f6d270 msgr2=0x55884733c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:11.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.780+0000 7f58c6a52740 1 --2- 172.21.15.39:0/1238561849 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846f6d270 0x55884733c6b0 secure :-1 s=READY pgs=2692 cs=0 l=1 rev1=1 crypto rx=0x7f58b000afc0 tx=0x7f58b00108c0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.780+0000 7f58c6a52740 1 -- 172.21.15.39:0/1238561849 shutdown_connections 2024-09-21T07:54:11.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.780+0000 7f58c6a52740 1 --2- 172.21.15.39:0/1238561849 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846f6d270 0x55884733c6b0 unknown :-1 s=CLOSED pgs=2692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.780+0000 7f58c6a52740 1 --2- 172.21.15.39:0/1238561849 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558846f4c590 0x558847334e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.780+0000 7f58c6a52740 1 --2- 172.21.15.39:0/1238561849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558846f443b0 0x558847334900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.780+0000 7f58c6a52740 1 -- 172.21.15.39:0/1238561849 >> 172.21.15.39:0/1238561849 conn(0x558846f613c0 msgr2=0x558847313940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:11.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.780+0000 7f58c6a52740 1 -- 172.21.15.39:0/1238561849 shutdown_connections 2024-09-21T07:54:11.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.780+0000 7f58c6a52740 1 -- 172.21.15.39:0/1238561849 wait complete. 2024-09-21T07:54:11.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.781+0000 7f58c6a52740 1 -- start start 2024-09-21T07:54:11.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.781+0000 7f58c6a52740 1 Processor -- start 2024-09-21T07:54:11.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.783+0000 7f58c6a52740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558846f443b0 0x558847334af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:11.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.783+0000 7f58c6a52740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846f4c590 0x558847369fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:11.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.783+0000 7f58c6a52740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558846f6d270 0x55884736a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:11.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.783+0000 7f58c6a52740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558847336b60 con 0x558846f4c590 2024-09-21T07:54:11.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.783+0000 7f58c6a52740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55884736b840 con 0x558846f443b0 2024-09-21T07:54:11.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.783+0000 7f58c6a52740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x558847337180 con 0x558846f6d270 2024-09-21T07:54:11.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.783+0000 7f58c6a52740 10 client.0 _ll_register_callbacks cb 0x7f58a0065040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:54:11.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.783+0000 7f58afe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846f4c590 0x558847369fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:11.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.783+0000 7f58afe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846f4c590 0x558847369fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55360/0 (socket says 172.21.15.39:55360) 2024-09-21T07:54:11.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.783+0000 7f58afe00640 1 -- 172.21.15.39:0/941648308 learned_addr learned my addr 172.21.15.39:0/941648308 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:11.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.783+0000 7f58bce00640 1 --2- 172.21.15.39:0/941648308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558846f443b0 0x558847334af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:11.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.783+0000 7f58bd800640 1 --2- 172.21.15.39:0/941648308 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558846f6d270 0x55884736a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:11.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.783+0000 7f58afe00640 1 -- 172.21.15.39:0/941648308 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558846f6d270 msgr2=0x55884736a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:11.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.783+0000 7f58afe00640 1 --2- 172.21.15.39:0/941648308 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558846f6d270 0x55884736a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.783+0000 7f58afe00640 1 -- 172.21.15.39:0/941648308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558846f443b0 msgr2=0x558847334af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:11.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.783+0000 7f58afe00640 1 --2- 172.21.15.39:0/941648308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558846f443b0 0x558847334af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.783+0000 7f58afe00640 1 -- 172.21.15.39:0/941648308 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558847314bd0 con 0x558846f4c590 2024-09-21T07:54:11.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.784+0000 7f58afe00640 1 --2- 172.21.15.39:0/941648308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846f4c590 0x558847369fc0 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 crypto rx=0x7f58a005e370 tx=0x7f58a005e520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ccf6e8279bd0d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:11.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.784+0000 7f58a6000640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:54:11.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.784+0000 7f58a6000640 1 -- 172.21.15.39:0/941648308 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a0066820 con 0x558846f4c590 2024-09-21T07:54:11.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.784+0000 7f58a6000640 1 -- 172.21.15.39:0/941648308 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(253..253 src has 1..253) ==== 8035+0+0 (secure 0 0 0) 0x7f58a0069db0 con 0x558846f4c590 2024-09-21T07:54:11.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.784+0000 7f58c6a52740 1 -- 172.21.15.39:0/941648308 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58b000f6f0 con 0x558846f4c590 2024-09-21T07:54:11.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.784+0000 7f58a6000640 1 -- 172.21.15.39:0/941648308 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=301) ==== 24+0+0 (secure 0 0 0) 0x7f58a005d950 con 0x558846f4c590 2024-09-21T07:54:11.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.784+0000 7f58c6a52740 10 client.12042 fetch_fsmap learned FSMap version 301 2024-09-21T07:54:11.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.784+0000 7f58c6a52740 1 -- 172.21.15.39:0/941648308 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=301,monmap=2+}) -- 0x558847369aa0 con 0x558846f4c590 2024-09-21T07:54:11.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.784+0000 7f58a6000640 1 -- 172.21.15.39:0/941648308 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58b000f6f0 con 0x558846f4c590 2024-09-21T07:54:11.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.784+0000 7f58a6000640 1 -- 172.21.15.39:0/941648308 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 301) ==== 3542+0+0 (secure 0 0 0) 0x7f58a00980a0 con 0x558846f4c590 2024-09-21T07:54:11.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.785+0000 7f58c6a52740 10 client.12042 fetch_fsmap finished waiting for FSMap version 301 2024-09-21T07:54:11.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.785+0000 7f58c6a52740 10 client.12042 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T07:54:11.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.785+0000 7f58c6a52740 20 client.12042 populate_metadata read hostname 'smithi039' 2024-09-21T07:54:11.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.785+0000 7f58c6a52740 1 --2- 172.21.15.39:0/941648308 >> [v2:172.21.15.148:6836/3495264983,v1:172.21.15.148:6837/3495264983] conn(0x5588473dba20 0x5588473dbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:11.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.785+0000 7f58c6a52740 4 client.12042 mds_command: new command op to 10633 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:54:11.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.785+0000 7f58c6a52740 1 -- 172.21.15.39:0/941648308 --> [v2:172.21.15.148:6836/3495264983,v1:172.21.15.148:6837/3495264983] -- command(tid 0: {"prefix": "status"}) -- 0x558847284fd0 con 0x5588473dba20 2024-09-21T07:54:11.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.785+0000 7f58bce00640 1 --2- 172.21.15.39:0/941648308 >> [v2:172.21.15.148:6836/3495264983,v1:172.21.15.148:6837/3495264983] conn(0x5588473dba20 0x5588473dbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:11.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.786+0000 7f58bce00640 1 --2- 172.21.15.39:0/941648308 >> [v2:172.21.15.148:6836/3495264983,v1:172.21.15.148:6837/3495264983] conn(0x5588473dba20 0x5588473dbe30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6215b6de76dc2306 server_cookie=ae05410efdd4cb9 in_seq=0 out_seq=0 2024-09-21T07:54:11.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.786+0000 7f58a6000640 10 client.12042 ms_handle_connect on v2:172.21.15.148:6836/3495264983 2024-09-21T07:54:11.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.786+0000 7f58a6000640 1 -- 172.21.15.39:0/941648308 <== mds.1 v2:172.21.15.148:6836/3495264983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+335 (crc 0 0 0) 0x558847284fd0 con 0x5588473dba20 2024-09-21T07:54:11.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.786+0000 7f58a6000640 10 client.12042 handle_command_reply: tid=0 2024-09-21T07:54:11.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.786+0000 7f58c6a52740 1 client.12042 shutdown 2024-09-21T07:54:11.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.786+0000 7f58c6a52740 1 -- 172.21.15.39:0/941648308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846f4c590 msgr2=0x558847369fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:11.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.786+0000 7f58c6a52740 1 --2- 172.21.15.39:0/941648308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846f4c590 0x558847369fc0 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 crypto rx=0x7f58a005e370 tx=0x7f58a005e520 comp rx=0 tx=0).stop 2024-09-21T07:54:11.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.786+0000 7f58c6a52740 1 -- 172.21.15.39:0/941648308 shutdown_connections 2024-09-21T07:54:11.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.786+0000 7f58c6a52740 1 --2- 172.21.15.39:0/941648308 >> [v2:172.21.15.148:6836/3495264983,v1:172.21.15.148:6837/3495264983] conn(0x5588473dba20 0x5588473dbe30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.786+0000 7f58c6a52740 1 --2- 172.21.15.39:0/941648308 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x558846f6d270 0x55884736a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.786+0000 7f58c6a52740 1 --2- 172.21.15.39:0/941648308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558846f4c590 0x558847369fc0 unknown :-1 s=CLOSED pgs=2693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.786+0000 7f58c6a52740 1 --2- 172.21.15.39:0/941648308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558846f443b0 0x558847334af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.786+0000 7f58a6000640 0 client.12042 ms_handle_reset on v2:172.21.15.148:6836/3495264983 2024-09-21T07:54:11.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.786+0000 7f58c6a52740 1 -- 172.21.15.39:0/941648308 >> 172.21.15.39:0/941648308 conn(0x558846f613c0 msgr2=0x558847353f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:11.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.787+0000 7f58c6a52740 1 -- 172.21.15.39:0/941648308 shutdown_connections 2024-09-21T07:54:11.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.787+0000 7f58c6a52740 1 -- 172.21.15.39:0/941648308 wait complete. 2024-09-21T07:54:11.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.787+0000 7f58c6a52740 20 client.12042 trim_cache size 0 max 16384 2024-09-21T07:54:11.792 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:54:11.792 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:54:11.792 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 1, 2024-09-21T07:54:11.792 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 10633, 2024-09-21T07:54:11.792 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:54:11.792 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:54:11.792 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:54:11.792 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 13.573053723999999, 2024-09-21T07:54:11.792 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 301, 2024-09-21T07:54:11.792 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 253, 2024-09-21T07:54:11.793 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 253, 2024-09-21T07:54:11.793 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 62.438238220999999 2024-09-21T07:54:11.793 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:54:11.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.787+0000 7f58bf800640 1 -- 172.21.15.39:0/238325658 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5894078b90 msgr2=0x7f589407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:11.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.787+0000 7f58bf800640 1 --2- 172.21.15.39:0/238325658 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5894078b90 0x7f589407b0a0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f58a8034000 tx=0x7f58a8004730 comp rx=0 tx=0).stop 2024-09-21T07:54:11.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.787+0000 7f58bf800640 1 -- 172.21.15.39:0/238325658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58b806fc10 msgr2=0x7f58b80b1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:11.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.787+0000 7f58bf800640 1 --2- 172.21.15.39:0/238325658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58b806fc10 0x7f58b80b1d60 secure :-1 s=READY pgs=2689 cs=0 l=1 rev1=1 crypto rx=0x7f58a000c3f0 tx=0x7f58a000c8b0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.787+0000 7f58bf800640 1 -- 172.21.15.39:0/238325658 shutdown_connections 2024-09-21T07:54:11.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.787+0000 7f58bf800640 1 --2- 172.21.15.39:0/238325658 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5894078b90 0x7f589407b0a0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.787+0000 7f58bf800640 1 --2- 172.21.15.39:0/238325658 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f58b815cd30 0x7f58b81f8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.787+0000 7f58bf800640 1 --2- 172.21.15.39:0/238325658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58b806fc10 0x7f58b80b1d60 unknown :-1 s=CLOSED pgs=2689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.787+0000 7f58bf800640 1 --2- 172.21.15.39:0/238325658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58b806f800 0x7f58b81ef7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.787+0000 7f58bf800640 1 -- 172.21.15.39:0/238325658 >> 172.21.15.39:0/238325658 conn(0x7f58b806cfc0 msgr2=0x7f58b806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:11.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.788+0000 7f58bf800640 1 -- 172.21.15.39:0/238325658 shutdown_connections 2024-09-21T07:54:11.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:11.788+0000 7f58bf800640 1 -- 172.21.15.39:0/238325658 wait complete. 2024-09-21T07:54:11.815 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T07:54:12.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.042+0000 7f525f800640 1 -- 172.21.15.39:0/1998129435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f523c005350 msgr2=0x7f523c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:12.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.042+0000 7f525f800640 1 --2- 172.21.15.39:0/1998129435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f523c005350 0x7f523c00b9c0 secure :-1 s=READY pgs=2695 cs=0 l=1 rev1=1 crypto rx=0x7f5250005fd0 tx=0x7f523c005830 comp rx=0 tx=0).stop 2024-09-21T07:54:12.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.042+0000 7f525f800640 1 -- 172.21.15.39:0/1998129435 shutdown_connections 2024-09-21T07:54:12.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.042+0000 7f525f800640 1 --2- 172.21.15.39:0/1998129435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f523c005350 0x7f523c00b9c0 unknown :-1 s=CLOSED pgs=2695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.042+0000 7f525f800640 1 -- 172.21.15.39:0/1998129435 >> 172.21.15.39:0/1998129435 conn(0x7f525806cfc0 msgr2=0x7f525806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:12.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.042+0000 7f525f800640 1 -- 172.21.15.39:0/1998129435 shutdown_connections 2024-09-21T07:54:12.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.042+0000 7f525f800640 1 -- 172.21.15.39:0/1998129435 wait complete. 2024-09-21T07:54:12.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.044+0000 7f525f800640 1 Processor -- start 2024-09-21T07:54:12.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.045+0000 7f525f800640 1 -- start start 2024-09-21T07:54:12.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.046+0000 7f525f800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f523c005350 0x7f52581e6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.046+0000 7f525f800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f525814dc60 0x7f5258140b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.046+0000 7f525f800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5258150690 0x7f5258150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.046+0000 7f525f800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f52580cdcc0 con 0x7f523c005350 2024-09-21T07:54:12.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.046+0000 7f525f800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f52580cdb40 con 0x7f5258150690 2024-09-21T07:54:12.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.046+0000 7f525f800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f52580cde40 con 0x7f525814dc60 2024-09-21T07:54:12.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.046+0000 7f525ce00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f523c005350 0x7f52581e6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.046+0000 7f525ce00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f523c005350 0x7f52581e6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50228/0 (socket says 172.21.15.39:50228) 2024-09-21T07:54:12.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.046+0000 7f525ce00640 1 -- 172.21.15.39:0/448601088 learned_addr learned my addr 172.21.15.39:0/448601088 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:12.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.047+0000 7f525d800640 1 --2- 172.21.15.39:0/448601088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5258150690 0x7f5258150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.047+0000 7f524fe00640 1 --2- 172.21.15.39:0/448601088 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f525814dc60 0x7f5258140b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.047+0000 7f525ce00640 1 -- 172.21.15.39:0/448601088 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f525814dc60 msgr2=0x7f5258140b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:12.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.047+0000 7f525ce00640 1 --2- 172.21.15.39:0/448601088 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f525814dc60 0x7f5258140b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.047+0000 7f525ce00640 1 -- 172.21.15.39:0/448601088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5258150690 msgr2=0x7f5258150aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:12.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.047+0000 7f525ce00640 1 --2- 172.21.15.39:0/448601088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5258150690 0x7f5258150aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.047+0000 7f525ce00640 1 -- 172.21.15.39:0/448601088 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5258150fe0 con 0x7f523c005350 2024-09-21T07:54:12.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.047+0000 7f525ce00640 1 --2- 172.21.15.39:0/448601088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f523c005350 0x7f52581e6dc0 secure :-1 s=READY pgs=2696 cs=0 l=1 rev1=1 crypto rx=0x7f52480066b0 tx=0x7f52480039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:12.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.047+0000 7f5255600640 1 -- 172.21.15.39:0/448601088 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5248015070 con 0x7f523c005350 2024-09-21T07:54:12.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.048+0000 7f5255600640 1 -- 172.21.15.39:0/448601088 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5248011920 con 0x7f523c005350 2024-09-21T07:54:12.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.048+0000 7f525f800640 1 -- 172.21.15.39:0/448601088 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5258152d80 con 0x7f523c005350 2024-09-21T07:54:12.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.048+0000 7f5255600640 1 -- 172.21.15.39:0/448601088 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5248011db0 con 0x7f523c005350 2024-09-21T07:54:12.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.048+0000 7f525f800640 1 -- 172.21.15.39:0/448601088 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5258153340 con 0x7f523c005350 2024-09-21T07:54:12.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.050+0000 7f5255600640 1 -- 172.21.15.39:0/448601088 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5248018070 con 0x7f523c005350 2024-09-21T07:54:12.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.051+0000 7f5255600640 1 --2- 172.21.15.39:0/448601088 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5234078bc0 0x7f523407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.051+0000 7f5255600640 1 -- 172.21.15.39:0/448601088 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(253..253 src has 1..253) ==== 8035+0+0 (secure 0 0 0) 0x7f524809c250 con 0x7f523c005350 2024-09-21T07:54:12.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.051+0000 7f5266842740 1 Processor -- start 2024-09-21T07:54:12.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.051+0000 7f524fe00640 1 --2- 172.21.15.39:0/448601088 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5234078bc0 0x7f523407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.052+0000 7f5266842740 1 -- start start 2024-09-21T07:54:12.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.052+0000 7f524fe00640 1 --2- 172.21.15.39:0/448601088 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5234078bc0 0x7f523407b0d0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f5240000c00 tx=0x7f5240006b70 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:12.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.053+0000 7f5266842740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2480b2270 0x55c248482970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.053+0000 7f5266842740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2480893b0 0x55c248482eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.053+0000 7f5266842740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c248091590 0x55c248452f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.053+0000 7f5266842740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c2483dc430 con 0x55c2480893b0 2024-09-21T07:54:12.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.053+0000 7f5266842740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c2483cc5d0 con 0x55c2480b2270 2024-09-21T07:54:12.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.053+0000 7f5266842740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55c2483bba00 con 0x55c248091590 2024-09-21T07:54:12.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.053+0000 7f524fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2480893b0 0x55c248482eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.053+0000 7f524fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2480893b0 0x55c248482eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50236/0 (socket says 172.21.15.39:50236) 2024-09-21T07:54:12.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.053+0000 7f524fe00640 1 -- 172.21.15.39:0/2457539576 learned_addr learned my addr 172.21.15.39:0/2457539576 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:12.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.054+0000 7f525ce00640 1 --2- 172.21.15.39:0/2457539576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2480b2270 0x55c248482970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.054+0000 7f525d800640 1 --2- 172.21.15.39:0/2457539576 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c248091590 0x55c248452f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.054+0000 7f524fe00640 1 -- 172.21.15.39:0/2457539576 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c248091590 msgr2=0x55c248452f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:12.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.054+0000 7f524fe00640 1 --2- 172.21.15.39:0/2457539576 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c248091590 0x55c248452f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.054+0000 7f524fe00640 1 -- 172.21.15.39:0/2457539576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2480b2270 msgr2=0x55c248482970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:12.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.054+0000 7f524fe00640 1 --2- 172.21.15.39:0/2457539576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2480b2270 0x55c248482970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.054+0000 7f524fe00640 1 -- 172.21.15.39:0/2457539576 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c248384520 con 0x55c2480893b0 2024-09-21T07:54:12.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.054+0000 7f525d800640 1 --2- 172.21.15.39:0/2457539576 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c248091590 0x55c248452f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:12.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.054+0000 7f524fe00640 1 --2- 172.21.15.39:0/2457539576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2480893b0 0x55c248482eb0 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 crypto rx=0x7f5240012920 tx=0x7f5240015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4b24274b3b0665 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:12.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.054+0000 7f524d600640 1 -- 172.21.15.39:0/2457539576 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524000f960 con 0x55c2480893b0 2024-09-21T07:54:12.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.054+0000 7f524d600640 1 -- 172.21.15.39:0/2457539576 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5240024020 con 0x55c2480893b0 2024-09-21T07:54:12.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.054+0000 7f524d600640 1 -- 172.21.15.39:0/2457539576 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524000fd00 con 0x55c2480893b0 2024-09-21T07:54:12.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.055+0000 7f5266842740 1 -- 172.21.15.39:0/2457539576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2480893b0 msgr2=0x55c248482eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:12.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.055+0000 7f5266842740 1 --2- 172.21.15.39:0/2457539576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2480893b0 0x55c248482eb0 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 crypto rx=0x7f5240012920 tx=0x7f5240015000 comp rx=0 tx=0).stop 2024-09-21T07:54:12.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.055+0000 7f5266842740 1 -- 172.21.15.39:0/2457539576 shutdown_connections 2024-09-21T07:54:12.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.055+0000 7f5266842740 1 --2- 172.21.15.39:0/2457539576 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c248091590 0x55c248452f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.055+0000 7f5266842740 1 --2- 172.21.15.39:0/2457539576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2480893b0 0x55c248482eb0 unknown :-1 s=CLOSED pgs=2697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.055+0000 7f5266842740 1 --2- 172.21.15.39:0/2457539576 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2480b2270 0x55c248482970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.055+0000 7f5266842740 1 -- 172.21.15.39:0/2457539576 >> 172.21.15.39:0/2457539576 conn(0x55c2480a63c0 msgr2=0x55c248458970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:12.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.055+0000 7f5266842740 1 -- 172.21.15.39:0/2457539576 shutdown_connections 2024-09-21T07:54:12.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.055+0000 7f5266842740 1 -- 172.21.15.39:0/2457539576 wait complete. 2024-09-21T07:54:12.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.056+0000 7f5266842740 1 -- start start 2024-09-21T07:54:12.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.057+0000 7f5266842740 1 Processor -- start 2024-09-21T07:54:12.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.058+0000 7f5266842740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c2480893b0 0x55c248497430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.058+0000 7f5266842740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c248091590 0x55c2484b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.058+0000 7f5266842740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2480b2270 0x55c2484b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.058+0000 7f5266842740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c248362330 con 0x55c248091590 2024-09-21T07:54:12.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.058+0000 7f5266842740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c2483bf810 con 0x55c2480b2270 2024-09-21T07:54:12.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.058+0000 7f5266842740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55c24812c620 con 0x55c2480893b0 2024-09-21T07:54:12.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.058+0000 7f5266842740 10 client.0 _ll_register_callbacks cb 0x55c248367420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:54:12.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.059+0000 7f524fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c248091590 0x55c2484b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.059+0000 7f524fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c248091590 0x55c2484b14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50246/0 (socket says 172.21.15.39:50246) 2024-09-21T07:54:12.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.059+0000 7f524fe00640 1 -- 172.21.15.39:0/1281818854 learned_addr learned my addr 172.21.15.39:0/1281818854 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:12.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.059+0000 7f525ce00640 1 --2- 172.21.15.39:0/1281818854 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c2480893b0 0x55c248497430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.059+0000 7f525d800640 1 --2- 172.21.15.39:0/1281818854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2480b2270 0x55c2484b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.059+0000 7f524fe00640 1 -- 172.21.15.39:0/1281818854 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c2480893b0 msgr2=0x55c248497430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:12.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.059+0000 7f524fe00640 1 --2- 172.21.15.39:0/1281818854 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c2480893b0 0x55c248497430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.059+0000 7f524fe00640 1 -- 172.21.15.39:0/1281818854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2480b2270 msgr2=0x55c2484b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:12.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.059+0000 7f524fe00640 1 --2- 172.21.15.39:0/1281818854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2480b2270 0x55c2484b19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.059+0000 7f524fe00640 1 -- 172.21.15.39:0/1281818854 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c24811d850 con 0x55c248091590 2024-09-21T07:54:12.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.059+0000 7f525d800640 1 --2- 172.21.15.39:0/1281818854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2480b2270 0x55c2484b19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:54:12.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.059+0000 7f524fe00640 1 --2- 172.21.15.39:0/1281818854 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c248091590 0x55c2484b14a0 secure :-1 s=READY pgs=2698 cs=0 l=1 rev1=1 crypto rx=0x7f5240022040 tx=0x7f52400125c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b72e083981a8b18 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:12.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.059+0000 7f5246000640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:54:12.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.060+0000 7f5246000640 1 -- 172.21.15.39:0/1281818854 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5240023070 con 0x55c248091590 2024-09-21T07:54:12.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.060+0000 7f5266842740 1 -- 172.21.15.39:0/1281818854 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5240024020 con 0x55c248091590 2024-09-21T07:54:12.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.060+0000 7f5246000640 1 -- 172.21.15.39:0/1281818854 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(253..253 src has 1..253) ==== 8035+0+0 (secure 0 0 0) 0x7f524002a990 con 0x55c248091590 2024-09-21T07:54:12.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.060+0000 7f5246000640 1 -- 172.21.15.39:0/1281818854 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=301) ==== 24+0+0 (secure 0 0 0) 0x7f524002ab90 con 0x55c248091590 2024-09-21T07:54:12.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.060+0000 7f5266842740 10 client.12054 fetch_fsmap learned FSMap version 301 2024-09-21T07:54:12.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.060+0000 7f5266842740 1 -- 172.21.15.39:0/1281818854 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=301,monmap=2+}) -- 0x55c2481946b0 con 0x55c248091590 2024-09-21T07:54:12.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.060+0000 7f5246000640 1 -- 172.21.15.39:0/1281818854 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5240024020 con 0x55c248091590 2024-09-21T07:54:12.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.060+0000 7f5246000640 1 -- 172.21.15.39:0/1281818854 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 301) ==== 3542+0+0 (secure 0 0 0) 0x7f5240031030 con 0x55c248091590 2024-09-21T07:54:12.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.060+0000 7f5266842740 10 client.12054 fetch_fsmap finished waiting for FSMap version 301 2024-09-21T07:54:12.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.060+0000 7f5266842740 10 client.12054 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:54:12.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.060+0000 7f5266842740 20 client.12054 populate_metadata read hostname 'smithi039' 2024-09-21T07:54:12.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.060+0000 7f5266842740 1 --2- 172.21.15.39:0/1281818854 >> [v2:172.21.15.39:6832/2161818466,v1:172.21.15.39:6834/2161818466] conn(0x55c248070e40 0x55c24846d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.060+0000 7f5266842740 4 client.12054 mds_command: new command op to 11364 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:54:12.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.060+0000 7f5266842740 1 -- 172.21.15.39:0/1281818854 --> [v2:172.21.15.39:6832/2161818466,v1:172.21.15.39:6834/2161818466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2483ce520 con 0x55c248070e40 2024-09-21T07:54:12.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.061+0000 7f525ce00640 1 --2- 172.21.15.39:0/1281818854 >> [v2:172.21.15.39:6832/2161818466,v1:172.21.15.39:6834/2161818466] conn(0x55c248070e40 0x55c24846d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.061+0000 7f525ce00640 1 --2- 172.21.15.39:0/1281818854 >> [v2:172.21.15.39:6832/2161818466,v1:172.21.15.39:6834/2161818466] conn(0x55c248070e40 0x55c24846d210 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24d3446cc115f5ef server_cookie=be471cea645d2ba7 in_seq=0 out_seq=0 2024-09-21T07:54:12.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.061+0000 7f5246000640 10 client.12054 ms_handle_connect on v2:172.21.15.39:6832/2161818466 2024-09-21T07:54:12.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.062+0000 7f5246000640 1 -- 172.21.15.39:0/1281818854 <== mds.0 v2:172.21.15.39:6832/2161818466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c2483ce520 con 0x55c248070e40 2024-09-21T07:54:12.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.062+0000 7f5246000640 10 client.12054 handle_command_reply: tid=0 2024-09-21T07:54:12.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.062+0000 7f5266842740 1 client.12054 shutdown 2024-09-21T07:54:12.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.062+0000 7f5266842740 1 -- 172.21.15.39:0/1281818854 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c248091590 msgr2=0x55c2484b14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:12.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.062+0000 7f5266842740 1 --2- 172.21.15.39:0/1281818854 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c248091590 0x55c2484b14a0 secure :-1 s=READY pgs=2698 cs=0 l=1 rev1=1 crypto rx=0x7f5240022040 tx=0x7f52400125c0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.062+0000 7f5266842740 1 -- 172.21.15.39:0/1281818854 shutdown_connections 2024-09-21T07:54:12.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.062+0000 7f5266842740 1 --2- 172.21.15.39:0/1281818854 >> [v2:172.21.15.39:6832/2161818466,v1:172.21.15.39:6834/2161818466] conn(0x55c248070e40 0x55c24846d210 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.062+0000 7f5266842740 1 --2- 172.21.15.39:0/1281818854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2480b2270 0x55c2484b19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.062+0000 7f5266842740 1 --2- 172.21.15.39:0/1281818854 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c248091590 0x55c2484b14a0 unknown :-1 s=CLOSED pgs=2698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.062+0000 7f5266842740 1 --2- 172.21.15.39:0/1281818854 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c2480893b0 0x55c248497430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.062+0000 7f5246000640 0 client.12054 ms_handle_reset on v2:172.21.15.39:6832/2161818466 2024-09-21T07:54:12.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.062+0000 7f5266842740 1 -- 172.21.15.39:0/1281818854 >> 172.21.15.39:0/1281818854 conn(0x55c2480a63c0 msgr2=0x55c24849f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:12.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.063+0000 7f5266842740 1 -- 172.21.15.39:0/1281818854 shutdown_connections 2024-09-21T07:54:12.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.063+0000 7f5266842740 1 -- 172.21.15.39:0/1281818854 wait complete. 2024-09-21T07:54:12.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.063+0000 7f5266842740 20 client.12054 trim_cache size 0 max 16384 2024-09-21T07:54:12.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.082+0000 7f5266842740 1 Processor -- start 2024-09-21T07:54:12.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.083+0000 7f5266842740 1 -- start start 2024-09-21T07:54:12.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.084+0000 7f5266842740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c2480893b0 0x55c2484b1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.084+0000 7f5266842740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c248091590 0x55c248496060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.084+0000 7f5266842740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2480b2270 0x55c2484965a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.084+0000 7f5266842740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5258200a80 con 0x55c248091590 2024-09-21T07:54:12.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.084+0000 7f5266842740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5258201150 con 0x55c2480b2270 2024-09-21T07:54:12.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.084+0000 7f5266842740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f52580b5330 con 0x55c2480893b0 2024-09-21T07:54:12.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.084+0000 7f524fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c248091590 0x55c248496060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.084+0000 7f524fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c248091590 0x55c248496060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50260/0 (socket says 172.21.15.39:50260) 2024-09-21T07:54:12.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.084+0000 7f524fe00640 1 -- 172.21.15.39:0/1323462035 learned_addr learned my addr 172.21.15.39:0/1323462035 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:12.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.084+0000 7f525d800640 1 --2- 172.21.15.39:0/1323462035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2480b2270 0x55c2484965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.084+0000 7f525ce00640 1 --2- 172.21.15.39:0/1323462035 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c2480893b0 0x55c2484b1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.085+0000 7f524fe00640 1 -- 172.21.15.39:0/1323462035 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c2480893b0 msgr2=0x55c2484b1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:12.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.085+0000 7f524fe00640 1 --2- 172.21.15.39:0/1323462035 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c2480893b0 0x55c2484b1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.085+0000 7f524fe00640 1 -- 172.21.15.39:0/1323462035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2480b2270 msgr2=0x55c2484965a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:12.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.085+0000 7f524fe00640 1 --2- 172.21.15.39:0/1323462035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2480b2270 0x55c2484965a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.085+0000 7f524fe00640 1 -- 172.21.15.39:0/1323462035 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c248456740 con 0x55c248091590 2024-09-21T07:54:12.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.085+0000 7f525d800640 1 --2- 172.21.15.39:0/1323462035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2480b2270 0x55c2484965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:12.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.085+0000 7f524fe00640 1 --2- 172.21.15.39:0/1323462035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c248091590 0x55c248496060 secure :-1 s=READY pgs=2699 cs=0 l=1 rev1=1 crypto rx=0x7f524002af20 tx=0x7f5240004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aa4f84ebec1a7d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:12.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.085+0000 7f524cc00640 1 -- 172.21.15.39:0/1323462035 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5240023070 con 0x55c248091590 2024-09-21T07:54:12.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.085+0000 7f524cc00640 1 -- 172.21.15.39:0/1323462035 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f524002a790 con 0x55c248091590 2024-09-21T07:54:12.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.085+0000 7f524cc00640 1 -- 172.21.15.39:0/1323462035 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524001f540 con 0x55c248091590 2024-09-21T07:54:12.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.086+0000 7f5266842740 1 -- 172.21.15.39:0/1323462035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c248091590 msgr2=0x55c248496060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:12.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.086+0000 7f5266842740 1 --2- 172.21.15.39:0/1323462035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c248091590 0x55c248496060 secure :-1 s=READY pgs=2699 cs=0 l=1 rev1=1 crypto rx=0x7f524002af20 tx=0x7f5240004f60 comp rx=0 tx=0).stop 2024-09-21T07:54:12.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.086+0000 7f5266842740 1 -- 172.21.15.39:0/1323462035 shutdown_connections 2024-09-21T07:54:12.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.086+0000 7f5266842740 1 --2- 172.21.15.39:0/1323462035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2480b2270 0x55c2484965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.086+0000 7f5266842740 1 --2- 172.21.15.39:0/1323462035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c248091590 0x55c248496060 unknown :-1 s=CLOSED pgs=2699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.086+0000 7f5266842740 1 --2- 172.21.15.39:0/1323462035 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c2480893b0 0x55c2484b1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.086+0000 7f5266842740 1 -- 172.21.15.39:0/1323462035 >> 172.21.15.39:0/1323462035 conn(0x55c2480a63c0 msgr2=0x55c24849e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:12.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.086+0000 7f5266842740 1 -- 172.21.15.39:0/1323462035 shutdown_connections 2024-09-21T07:54:12.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.086+0000 7f5266842740 1 -- 172.21.15.39:0/1323462035 wait complete. 2024-09-21T07:54:12.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.087+0000 7f5266842740 1 -- start start 2024-09-21T07:54:12.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.087+0000 7f5266842740 1 Processor -- start 2024-09-21T07:54:12.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.089+0000 7f5266842740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2480893b0 0x55c248507650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.089+0000 7f5266842740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c248091590 0x55c248496060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.089+0000 7f5266842740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2480b2270 0x55c2484965a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.089+0000 7f5266842740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c2484539a0 con 0x55c2480b2270 2024-09-21T07:54:12.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.089+0000 7f5266842740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c24846cf30 con 0x55c2480893b0 2024-09-21T07:54:12.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.089+0000 7f5266842740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x55c24846c2a0 con 0x55c248091590 2024-09-21T07:54:12.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.089+0000 7f5266842740 10 client.0 _ll_register_callbacks cb 0x7f524000fc90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:54:12.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.089+0000 7f525d800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2480b2270 0x55c2484965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.089+0000 7f525d800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2480b2270 0x55c2484965a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50272/0 (socket says 172.21.15.39:50272) 2024-09-21T07:54:12.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.089+0000 7f525d800640 1 -- 172.21.15.39:0/219587453 learned_addr learned my addr 172.21.15.39:0/219587453 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:12.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.089+0000 7f524fe00640 1 --2- 172.21.15.39:0/219587453 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c248091590 0x55c248496060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.089+0000 7f525ce00640 1 --2- 172.21.15.39:0/219587453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2480893b0 0x55c248507650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.090+0000 7f525d800640 1 -- 172.21.15.39:0/219587453 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c248091590 msgr2=0x55c248496060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:12.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.090+0000 7f525d800640 1 --2- 172.21.15.39:0/219587453 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c248091590 0x55c248496060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.090+0000 7f525d800640 1 -- 172.21.15.39:0/219587453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2480893b0 msgr2=0x55c248507650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:12.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.090+0000 7f525d800640 1 --2- 172.21.15.39:0/219587453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2480893b0 0x55c248507650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.090+0000 7f525d800640 1 -- 172.21.15.39:0/219587453 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c248459650 con 0x55c2480b2270 2024-09-21T07:54:12.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.090+0000 7f525d800640 1 --2- 172.21.15.39:0/219587453 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2480b2270 0x55c2484965a0 secure :-1 s=READY pgs=2700 cs=0 l=1 rev1=1 crypto rx=0x7f5250009f90 tx=0x7f5250003de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ef55f03628e9fb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:12.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.090+0000 7f5246000640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:54:12.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.090+0000 7f5246000640 1 -- 172.21.15.39:0/219587453 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5250009ad0 con 0x55c2480b2270 2024-09-21T07:54:12.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.090+0000 7f5266842740 1 -- 172.21.15.39:0/219587453 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f524002a790 con 0x55c2480b2270 2024-09-21T07:54:12.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.090+0000 7f5246000640 1 -- 172.21.15.39:0/219587453 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(253..253 src has 1..253) ==== 8035+0+0 (secure 0 0 0) 0x7f5250004450 con 0x55c2480b2270 2024-09-21T07:54:12.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.091+0000 7f5246000640 1 -- 172.21.15.39:0/219587453 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=301) ==== 24+0+0 (secure 0 0 0) 0x7f5250004650 con 0x55c2480b2270 2024-09-21T07:54:12.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.091+0000 7f5266842740 10 client.12060 fetch_fsmap learned FSMap version 301 2024-09-21T07:54:12.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.091+0000 7f5266842740 1 -- 172.21.15.39:0/219587453 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=301,monmap=2+}) -- 0x55c248524d10 con 0x55c2480b2270 2024-09-21T07:54:12.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.091+0000 7f5246000640 1 -- 172.21.15.39:0/219587453 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f524002a790 con 0x55c2480b2270 2024-09-21T07:54:12.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.091+0000 7f5246000640 1 -- 172.21.15.39:0/219587453 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 301) ==== 3542+0+0 (secure 0 0 0) 0x7f5250051020 con 0x55c2480b2270 2024-09-21T07:54:12.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.091+0000 7f5266842740 10 client.12060 fetch_fsmap finished waiting for FSMap version 301 2024-09-21T07:54:12.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.091+0000 7f5266842740 10 client.12060 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:54:12.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.091+0000 7f5266842740 20 client.12060 populate_metadata read hostname 'smithi039' 2024-09-21T07:54:12.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.091+0000 7f5266842740 1 --2- 172.21.15.39:0/219587453 >> [v2:172.21.15.39:6832/2161818466,v1:172.21.15.39:6834/2161818466] conn(0x55c248498840 0x55c248521860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.091+0000 7f5266842740 4 client.12060 mds_command: new command op to 11364 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:54:12.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.091+0000 7f5266842740 1 -- 172.21.15.39:0/219587453 --> [v2:172.21.15.39:6832/2161818466,v1:172.21.15.39:6834/2161818466] -- command(tid 0: {"prefix": "status"}) -- 0x55c2483c9fd0 con 0x55c248498840 2024-09-21T07:54:12.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.091+0000 7f525ce00640 1 --2- 172.21.15.39:0/219587453 >> [v2:172.21.15.39:6832/2161818466,v1:172.21.15.39:6834/2161818466] conn(0x55c248498840 0x55c248521860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.092+0000 7f525ce00640 1 --2- 172.21.15.39:0/219587453 >> [v2:172.21.15.39:6832/2161818466,v1:172.21.15.39:6834/2161818466] conn(0x55c248498840 0x55c248521860 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cef914f049ba83d0 server_cookie=fffba47443e8ffe3 in_seq=0 out_seq=0 2024-09-21T07:54:12.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.092+0000 7f5246000640 10 client.12060 ms_handle_connect on v2:172.21.15.39:6832/2161818466 2024-09-21T07:54:12.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.092+0000 7f5246000640 1 -- 172.21.15.39:0/219587453 <== mds.0 v2:172.21.15.39:6832/2161818466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+335 (crc 0 0 0) 0x55c2483c9fd0 con 0x55c248498840 2024-09-21T07:54:12.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.092+0000 7f5246000640 10 client.12060 handle_command_reply: tid=0 2024-09-21T07:54:12.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.092+0000 7f5266842740 1 client.12060 shutdown 2024-09-21T07:54:12.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.092+0000 7f5266842740 1 -- 172.21.15.39:0/219587453 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2480b2270 msgr2=0x55c2484965a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:12.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.092+0000 7f5266842740 1 --2- 172.21.15.39:0/219587453 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2480b2270 0x55c2484965a0 secure :-1 s=READY pgs=2700 cs=0 l=1 rev1=1 crypto rx=0x7f5250009f90 tx=0x7f5250003de0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.092+0000 7f5266842740 1 -- 172.21.15.39:0/219587453 shutdown_connections 2024-09-21T07:54:12.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.092+0000 7f5266842740 1 --2- 172.21.15.39:0/219587453 >> [v2:172.21.15.39:6832/2161818466,v1:172.21.15.39:6834/2161818466] conn(0x55c248498840 0x55c248521860 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.092+0000 7f5266842740 1 --2- 172.21.15.39:0/219587453 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2480b2270 0x55c2484965a0 unknown :-1 s=CLOSED pgs=2700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.092+0000 7f5266842740 1 --2- 172.21.15.39:0/219587453 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55c248091590 0x55c248496060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.092+0000 7f5246000640 0 client.12060 ms_handle_reset on v2:172.21.15.39:6832/2161818466 2024-09-21T07:54:12.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.092+0000 7f5266842740 1 --2- 172.21.15.39:0/219587453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2480893b0 0x55c248507650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.092+0000 7f5266842740 1 -- 172.21.15.39:0/219587453 >> 172.21.15.39:0/219587453 conn(0x55c2480a63c0 msgr2=0x55c248453f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:12.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.093+0000 7f5266842740 1 -- 172.21.15.39:0/219587453 shutdown_connections 2024-09-21T07:54:12.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.093+0000 7f5266842740 1 -- 172.21.15.39:0/219587453 wait complete. 2024-09-21T07:54:12.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.093+0000 7f5266842740 20 client.12060 trim_cache size 0 max 16384 2024-09-21T07:54:12.097 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:54:12.098 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:54:12.098 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-09-21T07:54:12.098 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 11364, 2024-09-21T07:54:12.098 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:54:12.098 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:54:12.098 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:54:12.098 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 63.517381516999997, 2024-09-21T07:54:12.098 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 301, 2024-09-21T07:54:12.098 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 253, 2024-09-21T07:54:12.098 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 253, 2024-09-21T07:54:12.098 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 155.34962117200001 2024-09-21T07:54:12.098 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:54:12.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.093+0000 7f525f800640 1 -- 172.21.15.39:0/448601088 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5234078bc0 msgr2=0x7f523407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:12.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.093+0000 7f525f800640 1 --2- 172.21.15.39:0/448601088 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5234078bc0 0x7f523407b0d0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f5240000c00 tx=0x7f5240006b70 comp rx=0 tx=0).stop 2024-09-21T07:54:12.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.093+0000 7f525f800640 1 -- 172.21.15.39:0/448601088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f523c005350 msgr2=0x7f52581e6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:12.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.093+0000 7f525f800640 1 --2- 172.21.15.39:0/448601088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f523c005350 0x7f52581e6dc0 secure :-1 s=READY pgs=2696 cs=0 l=1 rev1=1 crypto rx=0x7f52480066b0 tx=0x7f52480039c0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.093+0000 7f525f800640 1 -- 172.21.15.39:0/448601088 shutdown_connections 2024-09-21T07:54:12.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.093+0000 7f525f800640 1 --2- 172.21.15.39:0/448601088 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5234078bc0 0x7f523407b0d0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.093+0000 7f525f800640 1 --2- 172.21.15.39:0/448601088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5258150690 0x7f5258150aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.093+0000 7f525f800640 1 --2- 172.21.15.39:0/448601088 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f525814dc60 0x7f5258140b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.093+0000 7f525f800640 1 --2- 172.21.15.39:0/448601088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f523c005350 0x7f52581e6dc0 unknown :-1 s=CLOSED pgs=2696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.093+0000 7f525f800640 1 -- 172.21.15.39:0/448601088 >> 172.21.15.39:0/448601088 conn(0x7f525806cfc0 msgr2=0x7f5258155660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:12.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.094+0000 7f525f800640 1 -- 172.21.15.39:0/448601088 shutdown_connections 2024-09-21T07:54:12.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.094+0000 7f525f800640 1 -- 172.21.15.39:0/448601088 wait complete. 2024-09-21T07:54:12.118 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T07:54:12.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.345+0000 7fae29e00640 1 -- 172.21.15.39:0/2482322866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae240d0620 msgr2=0x7fae240d0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:12.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.345+0000 7fae29e00640 1 --2- 172.21.15.39:0/2482322866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae240d0620 0x7fae240d0a30 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fae1c005fd0 tx=0x7fae1c02a840 comp rx=0 tx=0).stop 2024-09-21T07:54:12.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.345+0000 7fae29e00640 1 -- 172.21.15.39:0/2482322866 shutdown_connections 2024-09-21T07:54:12.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.345+0000 7fae29e00640 1 --2- 172.21.15.39:0/2482322866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae240d0620 0x7fae240d0a30 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.345+0000 7fae29e00640 1 -- 172.21.15.39:0/2482322866 >> 172.21.15.39:0/2482322866 conn(0x7fae2406cfc0 msgr2=0x7fae2406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:12.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.345+0000 7fae29e00640 1 -- 172.21.15.39:0/2482322866 shutdown_connections 2024-09-21T07:54:12.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.345+0000 7fae29e00640 1 -- 172.21.15.39:0/2482322866 wait complete. 2024-09-21T07:54:12.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.347+0000 7fae29e00640 1 Processor -- start 2024-09-21T07:54:12.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.347+0000 7fae29e00640 1 -- start start 2024-09-21T07:54:12.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.349+0000 7fae29e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae240cfd10 0x7fae24158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.349+0000 7fae29e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae240d0620 0x7fae240aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.349+0000 7fae29e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae2415cd30 0x7fae2414cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.349+0000 7fae29e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fae24073aa0 con 0x7fae240d0620 2024-09-21T07:54:12.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.349+0000 7fae29e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fae24073920 con 0x7fae2415cd30 2024-09-21T07:54:12.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.349+0000 7fae29e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fae24073c20 con 0x7fae240cfd10 2024-09-21T07:54:12.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.349+0000 7fae22000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae240d0620 0x7fae240aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.349+0000 7fae22000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae240d0620 0x7fae240aeac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50282/0 (socket says 172.21.15.39:50282) 2024-09-21T07:54:12.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.349+0000 7fae22000640 1 -- 172.21.15.39:0/1480072199 learned_addr learned my addr 172.21.15.39:0/1480072199 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:12.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.349+0000 7fae23400640 1 --2- 172.21.15.39:0/1480072199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae2415cd30 0x7fae2414cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.349+0000 7fae22a00640 1 --2- 172.21.15.39:0/1480072199 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae240cfd10 0x7fae24158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.350+0000 7fae22000640 1 -- 172.21.15.39:0/1480072199 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae240cfd10 msgr2=0x7fae24158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:12.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.350+0000 7fae22000640 1 --2- 172.21.15.39:0/1480072199 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae240cfd10 0x7fae24158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.350+0000 7fae22000640 1 -- 172.21.15.39:0/1480072199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae2415cd30 msgr2=0x7fae2414cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:12.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.350+0000 7fae22000640 1 --2- 172.21.15.39:0/1480072199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae2415cd30 0x7fae2414cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.350+0000 7fae22000640 1 -- 172.21.15.39:0/1480072199 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae24158a80 con 0x7fae240d0620 2024-09-21T07:54:12.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.350+0000 7fae22000640 1 --2- 172.21.15.39:0/1480072199 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae240d0620 0x7fae240aeac0 secure :-1 s=READY pgs=2702 cs=0 l=1 rev1=1 crypto rx=0x7fae10001220 tx=0x7fae1000cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:12.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.350+0000 7fae1b400640 1 -- 172.21.15.39:0/1480072199 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae10016020 con 0x7fae240d0620 2024-09-21T07:54:12.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.350+0000 7fae1b400640 1 -- 172.21.15.39:0/1480072199 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae1000ce50 con 0x7fae240d0620 2024-09-21T07:54:12.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.350+0000 7fae29e00640 1 -- 172.21.15.39:0/1480072199 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae241f6980 con 0x7fae240d0620 2024-09-21T07:54:12.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.350+0000 7fae1b400640 1 -- 172.21.15.39:0/1480072199 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1000abb0 con 0x7fae240d0620 2024-09-21T07:54:12.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.350+0000 7fae29e00640 1 -- 172.21.15.39:0/1480072199 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae24149a00 con 0x7fae240d0620 2024-09-21T07:54:12.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.353+0000 7fae1b400640 1 -- 172.21.15.39:0/1480072199 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fae10019020 con 0x7fae240d0620 2024-09-21T07:54:12.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.354+0000 7fae1b400640 1 --2- 172.21.15.39:0/1480072199 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae04078b90 0x7fae0407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.354+0000 7fae1b400640 1 -- 172.21.15.39:0/1480072199 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(253..253 src has 1..253) ==== 8035+0+0 (secure 0 0 0) 0x7fae1009b5e0 con 0x7fae240d0620 2024-09-21T07:54:12.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.354+0000 7fae22a00640 1 --2- 172.21.15.39:0/1480072199 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae04078b90 0x7fae0407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.354+0000 7fae30dc1740 1 Processor -- start 2024-09-21T07:54:12.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.355+0000 7fae30dc1740 1 -- start start 2024-09-21T07:54:12.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.355+0000 7fae22a00640 1 --2- 172.21.15.39:0/1480072199 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae04078b90 0x7fae0407b0a0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fae14000f60 tx=0x7fae14009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:12.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.356+0000 7fae30dc1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5593799c6270 0x559379d96970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.356+0000 7fae30dc1740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55937999d3b0 0x559379d96eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.356+0000 7fae30dc1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593799a5590 0x559379d66f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.356+0000 7fae30dc1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559379cf0430 con 0x5593799c6270 2024-09-21T07:54:12.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.356+0000 7fae30dc1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559379ce05d0 con 0x5593799a5590 2024-09-21T07:54:12.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.356+0000 7fae30dc1740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x559379ccfa00 con 0x55937999d3b0 2024-09-21T07:54:12.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.356+0000 7fae22a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5593799c6270 0x559379d96970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.356+0000 7fae22a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5593799c6270 0x559379d96970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50292/0 (socket says 172.21.15.39:50292) 2024-09-21T07:54:12.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.356+0000 7fae22a00640 1 -- 172.21.15.39:0/308496450 learned_addr learned my addr 172.21.15.39:0/308496450 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:12.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.357+0000 7fae23400640 1 --2- 172.21.15.39:0/308496450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593799a5590 0x559379d66f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.357+0000 7fae22000640 1 --2- 172.21.15.39:0/308496450 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55937999d3b0 0x559379d96eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.357+0000 7fae22a00640 1 -- 172.21.15.39:0/308496450 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55937999d3b0 msgr2=0x559379d96eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:12.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.357+0000 7fae22a00640 1 --2- 172.21.15.39:0/308496450 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55937999d3b0 0x559379d96eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.357+0000 7fae22a00640 1 -- 172.21.15.39:0/308496450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593799a5590 msgr2=0x559379d66f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:12.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.357+0000 7fae22a00640 1 --2- 172.21.15.39:0/308496450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593799a5590 0x559379d66f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.357+0000 7fae22a00640 1 -- 172.21.15.39:0/308496450 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559379c98520 con 0x5593799c6270 2024-09-21T07:54:12.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.357+0000 7fae22000640 1 --2- 172.21.15.39:0/308496450 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55937999d3b0 0x559379d96eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:12.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.357+0000 7fae23400640 1 --2- 172.21.15.39:0/308496450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593799a5590 0x559379d66f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:12.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.357+0000 7fae22a00640 1 --2- 172.21.15.39:0/308496450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5593799c6270 0x559379d96970 secure :-1 s=READY pgs=2703 cs=0 l=1 rev1=1 crypto rx=0x7fae140015d0 tx=0x7fae14012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=389734c73bb81584 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:12.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.357+0000 7fae0f400640 1 -- 172.21.15.39:0/308496450 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae14025070 con 0x5593799c6270 2024-09-21T07:54:12.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.358+0000 7fae0f400640 1 -- 172.21.15.39:0/308496450 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae1401b050 con 0x5593799c6270 2024-09-21T07:54:12.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.358+0000 7fae0f400640 1 -- 172.21.15.39:0/308496450 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae14014de0 con 0x5593799c6270 2024-09-21T07:54:12.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.358+0000 7fae30dc1740 1 -- 172.21.15.39:0/308496450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5593799c6270 msgr2=0x559379d96970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:12.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.358+0000 7fae30dc1740 1 --2- 172.21.15.39:0/308496450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5593799c6270 0x559379d96970 secure :-1 s=READY pgs=2703 cs=0 l=1 rev1=1 crypto rx=0x7fae140015d0 tx=0x7fae14012000 comp rx=0 tx=0).stop 2024-09-21T07:54:12.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.358+0000 7fae30dc1740 1 -- 172.21.15.39:0/308496450 shutdown_connections 2024-09-21T07:54:12.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.358+0000 7fae30dc1740 1 --2- 172.21.15.39:0/308496450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593799a5590 0x559379d66f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.358+0000 7fae30dc1740 1 --2- 172.21.15.39:0/308496450 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x55937999d3b0 0x559379d96eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.358+0000 7fae30dc1740 1 --2- 172.21.15.39:0/308496450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5593799c6270 0x559379d96970 unknown :-1 s=CLOSED pgs=2703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.358+0000 7fae30dc1740 1 -- 172.21.15.39:0/308496450 >> 172.21.15.39:0/308496450 conn(0x5593799ba3c0 msgr2=0x559379d6c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:12.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.358+0000 7fae30dc1740 1 -- 172.21.15.39:0/308496450 shutdown_connections 2024-09-21T07:54:12.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.358+0000 7fae30dc1740 1 -- 172.21.15.39:0/308496450 wait complete. 2024-09-21T07:54:12.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.359+0000 7fae30dc1740 1 -- start start 2024-09-21T07:54:12.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.360+0000 7fae30dc1740 1 Processor -- start 2024-09-21T07:54:12.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.361+0000 7fae30dc1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55937999d3b0 0x559379d8d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.361+0000 7fae30dc1740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5593799a5590 0x559379dc4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.361+0000 7fae30dc1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593799c6270 0x559379dc47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.361+0000 7fae30dc1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559379c76330 con 0x55937999d3b0 2024-09-21T07:54:12.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.361+0000 7fae30dc1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559379cd3810 con 0x5593799c6270 2024-09-21T07:54:12.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.361+0000 7fae30dc1740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x559379a40620 con 0x5593799a5590 2024-09-21T07:54:12.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.361+0000 7fae30dc1740 10 client.0 _ll_register_callbacks cb 0x559379c7b420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:54:12.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.361+0000 7fae22a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55937999d3b0 0x559379d8d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.362+0000 7fae22a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55937999d3b0 0x559379d8d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50302/0 (socket says 172.21.15.39:50302) 2024-09-21T07:54:12.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.362+0000 7fae22a00640 1 -- 172.21.15.39:0/2023403626 learned_addr learned my addr 172.21.15.39:0/2023403626 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:12.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.362+0000 7fae23400640 1 --2- 172.21.15.39:0/2023403626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593799c6270 0x559379dc47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.362+0000 7fae22000640 1 --2- 172.21.15.39:0/2023403626 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5593799a5590 0x559379dc4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.362+0000 7fae22a00640 1 -- 172.21.15.39:0/2023403626 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5593799a5590 msgr2=0x559379dc4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:12.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.362+0000 7fae22a00640 1 --2- 172.21.15.39:0/2023403626 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5593799a5590 0x559379dc4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.362+0000 7fae22a00640 1 -- 172.21.15.39:0/2023403626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593799c6270 msgr2=0x559379dc47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:12.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.362+0000 7fae22a00640 1 --2- 172.21.15.39:0/2023403626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593799c6270 0x559379dc47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.362+0000 7fae22a00640 1 -- 172.21.15.39:0/2023403626 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559379a31850 con 0x55937999d3b0 2024-09-21T07:54:12.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.362+0000 7fae23400640 1 --2- 172.21.15.39:0/2023403626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593799c6270 0x559379dc47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:12.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.362+0000 7fae22a00640 1 --2- 172.21.15.39:0/2023403626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55937999d3b0 0x559379d8d0d0 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 crypto rx=0x7fae14010fd0 tx=0x7fae14020bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db2518fcd4862169 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:12.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.362+0000 7fae03e00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:54:12.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.363+0000 7fae03e00640 1 -- 172.21.15.39:0/2023403626 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae14025050 con 0x55937999d3b0 2024-09-21T07:54:12.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.363+0000 7fae03e00640 1 -- 172.21.15.39:0/2023403626 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(253..253 src has 1..253) ==== 8035+0+0 (secure 0 0 0) 0x7fae1402c640 con 0x55937999d3b0 2024-09-21T07:54:12.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.363+0000 7fae30dc1740 1 -- 172.21.15.39:0/2023403626 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae1401b050 con 0x55937999d3b0 2024-09-21T07:54:12.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.363+0000 7fae03e00640 1 -- 172.21.15.39:0/2023403626 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=301) ==== 24+0+0 (secure 0 0 0) 0x7fae1402c840 con 0x55937999d3b0 2024-09-21T07:54:12.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.363+0000 7fae30dc1740 10 client.12072 fetch_fsmap learned FSMap version 301 2024-09-21T07:54:12.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.363+0000 7fae30dc1740 1 -- 172.21.15.39:0/2023403626 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=301,monmap=2+}) -- 0x559379aa86b0 con 0x55937999d3b0 2024-09-21T07:54:12.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.363+0000 7fae03e00640 1 -- 172.21.15.39:0/2023403626 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae1401b050 con 0x55937999d3b0 2024-09-21T07:54:12.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.363+0000 7fae03e00640 1 -- 172.21.15.39:0/2023403626 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 301) ==== 3542+0+0 (secure 0 0 0) 0x7fae14034030 con 0x55937999d3b0 2024-09-21T07:54:12.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.363+0000 7fae30dc1740 10 client.12072 fetch_fsmap finished waiting for FSMap version 301 2024-09-21T07:54:12.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.363+0000 7fae30dc1740 10 client.12072 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:54:12.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.363+0000 7fae30dc1740 20 client.12072 populate_metadata read hostname 'smithi039' 2024-09-21T07:54:12.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.363+0000 7fae30dc1740 1 --2- 172.21.15.39:0/2023403626 >> [v2:172.21.15.39:6833/732408880,v1:172.21.15.39:6835/732408880] conn(0x559379984e40 0x559379d8eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.363+0000 7fae30dc1740 4 client.12072 mds_command: new command op to 11970 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:54:12.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.363+0000 7fae30dc1740 1 -- 172.21.15.39:0/2023403626 --> [v2:172.21.15.39:6833/732408880,v1:172.21.15.39:6835/732408880] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559379ce2520 con 0x559379984e40 2024-09-21T07:54:12.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.364+0000 7fae22000640 1 --2- 172.21.15.39:0/2023403626 >> [v2:172.21.15.39:6833/732408880,v1:172.21.15.39:6835/732408880] conn(0x559379984e40 0x559379d8eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.364+0000 7fae22000640 1 --2- 172.21.15.39:0/2023403626 >> [v2:172.21.15.39:6833/732408880,v1:172.21.15.39:6835/732408880] conn(0x559379984e40 0x559379d8eb10 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3fafd3a2e2205abe server_cookie=1be23e89d5671025 in_seq=0 out_seq=0 2024-09-21T07:54:12.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.364+0000 7fae03e00640 10 client.12072 ms_handle_connect on v2:172.21.15.39:6833/732408880 2024-09-21T07:54:12.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.365+0000 7fae03e00640 1 -- 172.21.15.39:0/2023403626 <== mds.2 v2:172.21.15.39:6833/732408880 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559379ce2520 con 0x559379984e40 2024-09-21T07:54:12.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.365+0000 7fae03e00640 10 client.12072 handle_command_reply: tid=0 2024-09-21T07:54:12.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.365+0000 7fae30dc1740 1 client.12072 shutdown 2024-09-21T07:54:12.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.365+0000 7fae30dc1740 1 -- 172.21.15.39:0/2023403626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55937999d3b0 msgr2=0x559379d8d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:12.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.365+0000 7fae30dc1740 1 --2- 172.21.15.39:0/2023403626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55937999d3b0 0x559379d8d0d0 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 crypto rx=0x7fae14010fd0 tx=0x7fae14020bb0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.365+0000 7fae30dc1740 1 -- 172.21.15.39:0/2023403626 shutdown_connections 2024-09-21T07:54:12.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.365+0000 7fae30dc1740 1 --2- 172.21.15.39:0/2023403626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5593799c6270 0x559379dc47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.365+0000 7fae30dc1740 1 --2- 172.21.15.39:0/2023403626 >> [v2:172.21.15.39:6833/732408880,v1:172.21.15.39:6835/732408880] conn(0x559379984e40 0x559379d8eb10 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.365+0000 7fae30dc1740 1 --2- 172.21.15.39:0/2023403626 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5593799a5590 0x559379dc4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.365+0000 7fae30dc1740 1 --2- 172.21.15.39:0/2023403626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55937999d3b0 0x559379d8d0d0 unknown :-1 s=CLOSED pgs=2704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.366+0000 7fae30dc1740 1 -- 172.21.15.39:0/2023403626 >> 172.21.15.39:0/2023403626 conn(0x5593799ba3c0 msgr2=0x559379db4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:12.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.366+0000 7fae03e00640 0 client.12072 ms_handle_reset on v2:172.21.15.39:6833/732408880 2024-09-21T07:54:12.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.366+0000 7fae30dc1740 1 -- 172.21.15.39:0/2023403626 shutdown_connections 2024-09-21T07:54:12.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.366+0000 7fae30dc1740 1 -- 172.21.15.39:0/2023403626 wait complete. 2024-09-21T07:54:12.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.366+0000 7fae30dc1740 20 client.12072 trim_cache size 0 max 16384 2024-09-21T07:54:12.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.385+0000 7fae30dc1740 1 Processor -- start 2024-09-21T07:54:12.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.385+0000 7fae30dc1740 1 -- start start 2024-09-21T07:54:12.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.387+0000 7fae30dc1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55937999d3b0 0x559379dc4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.387+0000 7fae30dc1740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5593799a5590 0x559379d65c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.387+0000 7fae30dc1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5593799c6270 0x559379d661a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.387+0000 7fae30dc1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fae24204890 con 0x5593799c6270 2024-09-21T07:54:12.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.387+0000 7fae30dc1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fae24204f60 con 0x55937999d3b0 2024-09-21T07:54:12.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.387+0000 7fae30dc1740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fae24149100 con 0x5593799a5590 2024-09-21T07:54:12.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.387+0000 7fae23400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5593799c6270 0x559379d661a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.387+0000 7fae23400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5593799c6270 0x559379d661a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50308/0 (socket says 172.21.15.39:50308) 2024-09-21T07:54:12.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.387+0000 7fae23400640 1 -- 172.21.15.39:0/3194408866 learned_addr learned my addr 172.21.15.39:0/3194408866 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:12.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.387+0000 7fae22000640 1 --2- 172.21.15.39:0/3194408866 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5593799a5590 0x559379d65c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.387+0000 7fae22a00640 1 --2- 172.21.15.39:0/3194408866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55937999d3b0 0x559379dc4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.387+0000 7fae23400640 1 -- 172.21.15.39:0/3194408866 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5593799a5590 msgr2=0x559379d65c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:12.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.387+0000 7fae23400640 1 --2- 172.21.15.39:0/3194408866 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5593799a5590 0x559379d65c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.387+0000 7fae23400640 1 -- 172.21.15.39:0/3194408866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55937999d3b0 msgr2=0x559379dc4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:12.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.387+0000 7fae23400640 1 --2- 172.21.15.39:0/3194408866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55937999d3b0 0x559379dc4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.388+0000 7fae23400640 1 -- 172.21.15.39:0/3194408866 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559379d95d70 con 0x5593799c6270 2024-09-21T07:54:12.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.388+0000 7fae23400640 1 --2- 172.21.15.39:0/3194408866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5593799c6270 0x559379d661a0 secure :-1 s=READY pgs=2705 cs=0 l=1 rev1=1 crypto rx=0x7fae1c009cd0 tx=0x7fae1c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aade9b7d4df249e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:12.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.388+0000 7fae0ea00640 1 -- 172.21.15.39:0/3194408866 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1c003d70 con 0x5593799c6270 2024-09-21T07:54:12.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.388+0000 7fae0ea00640 1 -- 172.21.15.39:0/3194408866 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae1c004040 con 0x5593799c6270 2024-09-21T07:54:12.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.388+0000 7fae0ea00640 1 -- 172.21.15.39:0/3194408866 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1c004470 con 0x5593799c6270 2024-09-21T07:54:12.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.389+0000 7fae30dc1740 1 -- 172.21.15.39:0/3194408866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5593799c6270 msgr2=0x559379d661a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:12.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.389+0000 7fae30dc1740 1 --2- 172.21.15.39:0/3194408866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5593799c6270 0x559379d661a0 secure :-1 s=READY pgs=2705 cs=0 l=1 rev1=1 crypto rx=0x7fae1c009cd0 tx=0x7fae1c004a40 comp rx=0 tx=0).stop 2024-09-21T07:54:12.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.389+0000 7fae30dc1740 1 -- 172.21.15.39:0/3194408866 shutdown_connections 2024-09-21T07:54:12.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.389+0000 7fae30dc1740 1 --2- 172.21.15.39:0/3194408866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5593799c6270 0x559379d661a0 unknown :-1 s=CLOSED pgs=2705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.389+0000 7fae30dc1740 1 --2- 172.21.15.39:0/3194408866 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5593799a5590 0x559379d65c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.389+0000 7fae30dc1740 1 --2- 172.21.15.39:0/3194408866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55937999d3b0 0x559379dc4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.389+0000 7fae30dc1740 1 -- 172.21.15.39:0/3194408866 >> 172.21.15.39:0/3194408866 conn(0x5593799ba3c0 msgr2=0x559379d95660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:12.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.389+0000 7fae30dc1740 1 -- 172.21.15.39:0/3194408866 shutdown_connections 2024-09-21T07:54:12.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.389+0000 7fae30dc1740 1 -- 172.21.15.39:0/3194408866 wait complete. 2024-09-21T07:54:12.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.390+0000 7fae30dc1740 1 -- start start 2024-09-21T07:54:12.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.390+0000 7fae30dc1740 1 Processor -- start 2024-09-21T07:54:12.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.392+0000 7fae30dc1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55937999d3b0 0x559379dc9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.392+0000 7fae30dc1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5593799a5590 0x559379e34e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.392+0000 7fae30dc1740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5593799c6270 0x559379da9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.392+0000 7fae30dc1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559379dab4d0 con 0x5593799a5590 2024-09-21T07:54:12.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.392+0000 7fae30dc1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559379d8d610 con 0x55937999d3b0 2024-09-21T07:54:12.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.392+0000 7fae30dc1740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x559379d92640 con 0x5593799c6270 2024-09-21T07:54:12.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.392+0000 7fae30dc1740 10 client.0 _ll_register_callbacks cb 0x7fae14020740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:54:12.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.392+0000 7fae22000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5593799a5590 0x559379e34e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.392+0000 7fae22000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5593799a5590 0x559379e34e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50312/0 (socket says 172.21.15.39:50312) 2024-09-21T07:54:12.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.392+0000 7fae22000640 1 -- 172.21.15.39:0/4091794040 learned_addr learned my addr 172.21.15.39:0/4091794040 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:12.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.392+0000 7fae22a00640 1 --2- 172.21.15.39:0/4091794040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55937999d3b0 0x559379dc9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.392+0000 7fae23400640 1 --2- 172.21.15.39:0/4091794040 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5593799c6270 0x559379da9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.392+0000 7fae22000640 1 -- 172.21.15.39:0/4091794040 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5593799c6270 msgr2=0x559379da9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:12.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.392+0000 7fae22000640 1 --2- 172.21.15.39:0/4091794040 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5593799c6270 0x559379da9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.392+0000 7fae22000640 1 -- 172.21.15.39:0/4091794040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55937999d3b0 msgr2=0x559379dc9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:12.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.392+0000 7fae22000640 1 --2- 172.21.15.39:0/4091794040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55937999d3b0 0x559379dc9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.392+0000 7fae22000640 1 -- 172.21.15.39:0/4091794040 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559379dc8820 con 0x5593799a5590 2024-09-21T07:54:12.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.392+0000 7fae23400640 1 --2- 172.21.15.39:0/4091794040 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5593799c6270 0x559379da9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:12.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.393+0000 7fae22000640 1 --2- 172.21.15.39:0/4091794040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5593799a5590 0x559379e34e00 secure :-1 s=READY pgs=2706 cs=0 l=1 rev1=1 crypto rx=0x7fae10062720 tx=0x7fae10001550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30892fed3b2ed697 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:12.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.393+0000 7fae03e00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:54:12.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.393+0000 7fae03e00640 1 -- 172.21.15.39:0/4091794040 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1000ad50 con 0x5593799a5590 2024-09-21T07:54:12.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.393+0000 7fae03e00640 1 -- 172.21.15.39:0/4091794040 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(253..253 src has 1..253) ==== 8035+0+0 (secure 0 0 0) 0x7fae1005ec90 con 0x5593799a5590 2024-09-21T07:54:12.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.393+0000 7fae30dc1740 1 -- 172.21.15.39:0/4091794040 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae1c004040 con 0x5593799a5590 2024-09-21T07:54:12.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.393+0000 7fae03e00640 1 -- 172.21.15.39:0/4091794040 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=301) ==== 24+0+0 (secure 0 0 0) 0x7fae1009b0b0 con 0x5593799a5590 2024-09-21T07:54:12.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.393+0000 7fae30dc1740 10 client.12078 fetch_fsmap learned FSMap version 301 2024-09-21T07:54:12.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.393+0000 7fae30dc1740 1 -- 172.21.15.39:0/4091794040 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=301,monmap=2+}) -- 0x559379dc95a0 con 0x5593799a5590 2024-09-21T07:54:12.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.393+0000 7fae03e00640 1 -- 172.21.15.39:0/4091794040 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae1c004040 con 0x5593799a5590 2024-09-21T07:54:12.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.394+0000 7fae03e00640 1 -- 172.21.15.39:0/4091794040 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 301) ==== 3542+0+0 (secure 0 0 0) 0x7fae1009b250 con 0x5593799a5590 2024-09-21T07:54:12.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.394+0000 7fae30dc1740 10 client.12078 fetch_fsmap finished waiting for FSMap version 301 2024-09-21T07:54:12.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.394+0000 7fae30dc1740 10 client.12078 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:54:12.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.394+0000 7fae30dc1740 20 client.12078 populate_metadata read hostname 'smithi039' 2024-09-21T07:54:12.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.394+0000 7fae30dc1740 1 --2- 172.21.15.39:0/4091794040 >> [v2:172.21.15.39:6833/732408880,v1:172.21.15.39:6835/732408880] conn(0x559379d65c60 0x559379d66070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.394+0000 7fae30dc1740 4 client.12078 mds_command: new command op to 11970 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:54:12.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.394+0000 7fae30dc1740 1 -- 172.21.15.39:0/4091794040 --> [v2:172.21.15.39:6833/732408880,v1:172.21.15.39:6835/732408880] -- command(tid 0: {"prefix": "status"}) -- 0x559379cddfd0 con 0x559379d65c60 2024-09-21T07:54:12.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.394+0000 7fae22a00640 1 --2- 172.21.15.39:0/4091794040 >> [v2:172.21.15.39:6833/732408880,v1:172.21.15.39:6835/732408880] conn(0x559379d65c60 0x559379d66070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.394+0000 7fae22a00640 1 --2- 172.21.15.39:0/4091794040 >> [v2:172.21.15.39:6833/732408880,v1:172.21.15.39:6835/732408880] conn(0x559379d65c60 0x559379d66070 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a8102fa75d12f4da server_cookie=74f649a0d6e06bc2 in_seq=0 out_seq=0 2024-09-21T07:54:12.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.394+0000 7fae03e00640 10 client.12078 ms_handle_connect on v2:172.21.15.39:6833/732408880 2024-09-21T07:54:12.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.395+0000 7fae03e00640 1 -- 172.21.15.39:0/4091794040 <== mds.2 v2:172.21.15.39:6833/732408880 1 ==== command_reply(tid 0: 0 ) ==== 8+0+323 (crc 0 0 0) 0x559379cddfd0 con 0x559379d65c60 2024-09-21T07:54:12.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.395+0000 7fae03e00640 10 client.12078 handle_command_reply: tid=0 2024-09-21T07:54:12.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.395+0000 7fae30dc1740 1 client.12078 shutdown 2024-09-21T07:54:12.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.395+0000 7fae30dc1740 1 -- 172.21.15.39:0/4091794040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5593799a5590 msgr2=0x559379e34e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:12.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.395+0000 7fae30dc1740 1 --2- 172.21.15.39:0/4091794040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5593799a5590 0x559379e34e00 secure :-1 s=READY pgs=2706 cs=0 l=1 rev1=1 crypto rx=0x7fae10062720 tx=0x7fae10001550 comp rx=0 tx=0).stop 2024-09-21T07:54:12.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.395+0000 7fae30dc1740 1 -- 172.21.15.39:0/4091794040 shutdown_connections 2024-09-21T07:54:12.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.395+0000 7fae30dc1740 1 --2- 172.21.15.39:0/4091794040 >> [v2:172.21.15.39:6833/732408880,v1:172.21.15.39:6835/732408880] conn(0x559379d65c60 0x559379d66070 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.395+0000 7fae30dc1740 1 --2- 172.21.15.39:0/4091794040 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5593799c6270 0x559379da9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.395+0000 7fae30dc1740 1 --2- 172.21.15.39:0/4091794040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5593799a5590 0x559379e34e00 unknown :-1 s=CLOSED pgs=2706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.395+0000 7fae30dc1740 1 --2- 172.21.15.39:0/4091794040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55937999d3b0 0x559379dc9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.395+0000 7fae30dc1740 1 -- 172.21.15.39:0/4091794040 >> 172.21.15.39:0/4091794040 conn(0x5593799ba3c0 msgr2=0x559379d8c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:12.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.395+0000 7fae03e00640 0 client.12078 ms_handle_reset on v2:172.21.15.39:6833/732408880 2024-09-21T07:54:12.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.395+0000 7fae30dc1740 1 -- 172.21.15.39:0/4091794040 shutdown_connections 2024-09-21T07:54:12.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.395+0000 7fae30dc1740 1 -- 172.21.15.39:0/4091794040 wait complete. 2024-09-21T07:54:12.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.395+0000 7fae30dc1740 20 client.12078 trim_cache size 0 max 16384 2024-09-21T07:54:12.402 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:54:12.402 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:54:12.402 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 2, 2024-09-21T07:54:12.402 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 11970, 2024-09-21T07:54:12.403 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:54:12.403 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:54:12.403 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:54:12.403 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 13.618745212, 2024-09-21T07:54:12.403 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 301, 2024-09-21T07:54:12.403 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 253, 2024-09-21T07:54:12.403 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 253, 2024-09-21T07:54:12.403 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 14.037226382 2024-09-21T07:54:12.403 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:54:12.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.396+0000 7fae29e00640 1 -- 172.21.15.39:0/1480072199 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae04078b90 msgr2=0x7fae0407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:12.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.396+0000 7fae29e00640 1 --2- 172.21.15.39:0/1480072199 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae04078b90 0x7fae0407b0a0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fae14000f60 tx=0x7fae14009040 comp rx=0 tx=0).stop 2024-09-21T07:54:12.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.396+0000 7fae29e00640 1 -- 172.21.15.39:0/1480072199 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae240d0620 msgr2=0x7fae240aeac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:12.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.396+0000 7fae29e00640 1 --2- 172.21.15.39:0/1480072199 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae240d0620 0x7fae240aeac0 secure :-1 s=READY pgs=2702 cs=0 l=1 rev1=1 crypto rx=0x7fae10001220 tx=0x7fae1000cb10 comp rx=0 tx=0).stop 2024-09-21T07:54:12.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.396+0000 7fae29e00640 1 -- 172.21.15.39:0/1480072199 shutdown_connections 2024-09-21T07:54:12.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.396+0000 7fae29e00640 1 --2- 172.21.15.39:0/1480072199 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae04078b90 0x7fae0407b0a0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.396+0000 7fae29e00640 1 --2- 172.21.15.39:0/1480072199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae2415cd30 0x7fae2414cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.396+0000 7fae29e00640 1 --2- 172.21.15.39:0/1480072199 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae240d0620 0x7fae240aeac0 unknown :-1 s=CLOSED pgs=2702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.396+0000 7fae29e00640 1 --2- 172.21.15.39:0/1480072199 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae240cfd10 0x7fae24158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.396+0000 7fae29e00640 1 -- 172.21.15.39:0/1480072199 >> 172.21.15.39:0/1480072199 conn(0x7fae2406cfc0 msgr2=0x7fae2415ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:12.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.396+0000 7fae29e00640 1 -- 172.21.15.39:0/1480072199 shutdown_connections 2024-09-21T07:54:12.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.396+0000 7fae29e00640 1 -- 172.21.15.39:0/1480072199 wait complete. 2024-09-21T07:54:12.421 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:54:12.541 INFO:tasks.ceph.mds.a.smithi039.stderr:2024-09-21T07:54:12.540+0000 7fd91b600640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:54:12.542 INFO:tasks.ceph.mds.a.smithi039.stderr:2024-09-21T07:54:12.540+0000 7fd91b600640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-21T07:54:12.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.645+0000 7f2285600640 1 -- 172.21.15.39:0/717546735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f228006fc10 msgr2=0x7f22800d0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:12.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.645+0000 7f2285600640 1 --2- 172.21.15.39:0/717546735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f228006fc10 0x7f22800d0620 secure :-1 s=READY pgs=2707 cs=0 l=1 rev1=1 crypto rx=0x7f22800b56e0 tx=0x7f227802aa30 comp rx=0 tx=0).stop 2024-09-21T07:54:12.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.645+0000 7f2285600640 1 -- 172.21.15.39:0/717546735 shutdown_connections 2024-09-21T07:54:12.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.645+0000 7f2285600640 1 --2- 172.21.15.39:0/717546735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f228006fc10 0x7f22800d0620 unknown :-1 s=CLOSED pgs=2707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.645+0000 7f2285600640 1 --2- 172.21.15.39:0/717546735 >> v2:172.21.15.148:3300/0 conn(0x7f228006f800 0x7f22800cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.645+0000 7f2285600640 1 --2- 172.21.15.39:0/717546735 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f228015cd30 0x7f22800cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.645+0000 7f2285600640 1 -- 172.21.15.39:0/717546735 >> 172.21.15.39:0/717546735 conn(0x7f228006cfc0 msgr2=0x7f228006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:12.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.645+0000 7f2285600640 1 -- 172.21.15.39:0/717546735 shutdown_connections 2024-09-21T07:54:12.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.645+0000 7f2285600640 1 -- 172.21.15.39:0/717546735 wait complete. 2024-09-21T07:54:12.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.647+0000 7f2285600640 1 Processor -- start 2024-09-21T07:54:12.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.648+0000 7f2285600640 1 -- start start 2024-09-21T07:54:12.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.649+0000 7f2285600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f228006f800 0x7f2280156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.649+0000 7f2285600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f228006fc10 0x7f22801454d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.649+0000 7f2285600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f228015cd30 0x7f2280158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.649+0000 7f2285600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2280073710 con 0x7f228006f800 2024-09-21T07:54:12.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.649+0000 7f2285600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f22800d0f10 con 0x7f228006fc10 2024-09-21T07:54:12.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.649+0000 7f2285600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f2280073890 con 0x7f228015cd30 2024-09-21T07:54:12.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.649+0000 7f227ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f228006f800 0x7f2280156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.649+0000 7f227ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f228006f800 0x7f2280156160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50332/0 (socket says 172.21.15.39:50332) 2024-09-21T07:54:12.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.649+0000 7f227ea00640 1 -- 172.21.15.39:0/2127514657 learned_addr learned my addr 172.21.15.39:0/2127514657 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:12.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.650+0000 7f2277e00640 1 --2- 172.21.15.39:0/2127514657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f228006fc10 0x7f22801454d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.650+0000 7f227f400640 1 --2- 172.21.15.39:0/2127514657 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f228015cd30 0x7f2280158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.650+0000 7f227ea00640 1 -- 172.21.15.39:0/2127514657 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f228015cd30 msgr2=0x7f2280158d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:12.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.650+0000 7f227ea00640 1 --2- 172.21.15.39:0/2127514657 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f228015cd30 0x7f2280158d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.650+0000 7f227ea00640 1 -- 172.21.15.39:0/2127514657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f228006fc10 msgr2=0x7f22801454d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:12.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.650+0000 7f227ea00640 1 --2- 172.21.15.39:0/2127514657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f228006fc10 0x7f22801454d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.650+0000 7f227ea00640 1 -- 172.21.15.39:0/2127514657 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2280146350 con 0x7f228006f800 2024-09-21T07:54:12.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.650+0000 7f227ea00640 1 --2- 172.21.15.39:0/2127514657 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f228006f800 0x7f2280156160 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 crypto rx=0x7f227000e7c0 tx=0x7f227000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:12.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.650+0000 7f2276a00640 1 -- 172.21.15.39:0/2127514657 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227000ccc0 con 0x7f228006f800 2024-09-21T07:54:12.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.650+0000 7f2276a00640 1 -- 172.21.15.39:0/2127514657 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f227000ce80 con 0x7f228006f800 2024-09-21T07:54:12.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.650+0000 7f2276a00640 1 -- 172.21.15.39:0/2127514657 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2270005280 con 0x7f228006f800 2024-09-21T07:54:12.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.651+0000 7f2285600640 1 -- 172.21.15.39:0/2127514657 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2280146630 con 0x7f228006f800 2024-09-21T07:54:12.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.651+0000 7f2285600640 1 -- 172.21.15.39:0/2127514657 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2280146bf0 con 0x7f228006f800 2024-09-21T07:54:12.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.652+0000 7f2285600640 1 -- 172.21.15.39:0/2127514657 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22800d0620 con 0x7f228006f800 2024-09-21T07:54:12.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.653+0000 7f2276a00640 1 -- 172.21.15.39:0/2127514657 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2270005420 con 0x7f228006f800 2024-09-21T07:54:12.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.654+0000 7f2276a00640 1 --2- 172.21.15.39:0/2127514657 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f225c078b90 0x7f225c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.655+0000 7f2276a00640 1 -- 172.21.15.39:0/2127514657 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(253..253 src has 1..253) ==== 8035+0+0 (secure 0 0 0) 0x7f227009c0f0 con 0x7f228006f800 2024-09-21T07:54:12.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.655+0000 7f2277e00640 1 --2- 172.21.15.39:0/2127514657 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f225c078b90 0x7f225c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.656+0000 7f2277e00640 1 --2- 172.21.15.39:0/2127514657 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f225c078b90 0x7f225c07b0a0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f2280155c30 tx=0x7f2268005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:12.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.659+0000 7f2276a00640 1 -- 172.21.15.39:0/2127514657 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2270063af0 con 0x7f228006f800 2024-09-21T07:54:12.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.899+0000 7f2285600640 1 -- 172.21.15.39:0/2127514657 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f228015c2f0 con 0x7f228006f800 2024-09-21T07:54:12.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.899+0000 7f2276a00640 1 -- 172.21.15.39:0/2127514657 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 301 v301) ==== 95+0+5291 (secure 0 0 0) 0x7f22700633f0 con 0x7f228006f800 2024-09-21T07:54:12.901 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:12.902 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":301,"btime":"2024-09-21T07:54:11:059441+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9467,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3658109005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3658109005},{"type":"v1","addr":"172.21.15.148:6835","nonce":3658109005}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":298}],"filesystems":[{"mdsmap":{"epoch":301,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:54:11.059432+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":252,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":11364,"mds_1":10633,"mds_2":11970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10633":{"gid":10633,"name":"d","rank":1,"incarnation":295,"state":"up:active","state_seq":19,"addr":"172.21.15.148:6837/3495264983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3495264983},{"type":"v1","addr":"172.21.15.148:6837","nonce":3495264983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11364":{"gid":11364,"name":"a","rank":0,"incarnation":285,"state":"up:active","state_seq":28,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11970":{"gid":11970,"name":"c","rank":2,"incarnation":297,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6835/732408880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":732408880},{"type":"v1","addr":"172.21.15.39:6835","nonce":732408880}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11364,"qdb_cluster":[10633,11364,11970]},"id":13}]} 2024-09-21T07:54:12.902 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 301 2024-09-21T07:54:12.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.902+0000 7f2285600640 1 -- 172.21.15.39:0/2127514657 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f225c078b90 msgr2=0x7f225c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:12.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.902+0000 7f2285600640 1 --2- 172.21.15.39:0/2127514657 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f225c078b90 0x7f225c07b0a0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f2280155c30 tx=0x7f2268005f90 comp rx=0 tx=0).stop 2024-09-21T07:54:12.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.902+0000 7f2285600640 1 -- 172.21.15.39:0/2127514657 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f228006f800 msgr2=0x7f2280156160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:12.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.902+0000 7f2285600640 1 --2- 172.21.15.39:0/2127514657 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f228006f800 0x7f2280156160 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 crypto rx=0x7f227000e7c0 tx=0x7f227000ec80 comp rx=0 tx=0).stop 2024-09-21T07:54:12.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.902+0000 7f2285600640 1 -- 172.21.15.39:0/2127514657 shutdown_connections 2024-09-21T07:54:12.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.902+0000 7f2285600640 1 --2- 172.21.15.39:0/2127514657 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f225c078b90 0x7f225c07b0a0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.902+0000 7f2285600640 1 --2- 172.21.15.39:0/2127514657 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f228015cd30 0x7f2280158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.902+0000 7f2285600640 1 --2- 172.21.15.39:0/2127514657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f228006fc10 0x7f22801454d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.902+0000 7f2285600640 1 --2- 172.21.15.39:0/2127514657 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f228006f800 0x7f2280156160 unknown :-1 s=CLOSED pgs=2708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.902+0000 7f2285600640 1 -- 172.21.15.39:0/2127514657 >> 172.21.15.39:0/2127514657 conn(0x7f228006cfc0 msgr2=0x7f22800aee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:12.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.902+0000 7f2285600640 1 -- 172.21.15.39:0/2127514657 shutdown_connections 2024-09-21T07:54:12.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:12.902+0000 7f2285600640 1 -- 172.21.15.39:0/2127514657 wait complete. 2024-09-21T07:54:12.929 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump snaps 2024-09-21T07:54:13.178 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_created":10,"last_destroyed":7,"snaps":[{"snapid":8,"ino":1099511627780,"stamp":"2024-09-21T07:51:12.091986+0000","name":"s1","metadata":{}},{"snapid":9,"ino":1099511627780,"stamp":"2024-09-21T07:52:44.191763+0000","name":"s2","metadata":{}},{"snapid":10,"ino":1099511627780,"stamp":"2024-09-21T07:53:39.355926+0000","name":"s3","metadata":{}}]} 2024-09-21T07:54:13.178 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 10, "last_destroyed": 7, "snaps": [ { "ino": 1099511627780, "metadata": {}, "name": "s1", "snapid": 8, "stamp": "2024-09-21T07:51:12.091986+0000" }, { "ino": 1099511627780, "metadata": {}, "name": "s2", "snapid": 9, "stamp": "2024-09-21T07:52:44.191763+0000" }, { "ino": 1099511627780, "metadata": {}, "name": "s3", "snapid": 10, "stamp": "2024-09-21T07:53:39.355926+0000" } ] } 2024-09-21T07:54:13.179 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:54:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.404+0000 7f2767200640 1 -- 172.21.15.39:0/3230039243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2744004800 msgr2=0x7f2744004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.404+0000 7f2767200640 1 --2- 172.21.15.39:0/3230039243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2744004800 0x7f2744004c10 secure :-1 s=READY pgs=2710 cs=0 l=1 rev1=1 crypto rx=0x7f274800c290 tx=0x7f2744005830 comp rx=0 tx=0).stop 2024-09-21T07:54:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.404+0000 7f2765200640 1 -- 172.21.15.39:0/3230039243 reap_dead start 2024-09-21T07:54:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.405+0000 7f2767200640 1 -- 172.21.15.39:0/3230039243 shutdown_connections 2024-09-21T07:54:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.405+0000 7f2767200640 1 -- 172.21.15.39:0/3230039243 >> 172.21.15.39:0/3230039243 conn(0x7f276006cfc0 msgr2=0x7f276006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.405+0000 7f2767200640 1 -- 172.21.15.39:0/3230039243 shutdown_connections 2024-09-21T07:54:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.405+0000 7f2767200640 1 -- 172.21.15.39:0/3230039243 wait complete. 2024-09-21T07:54:13.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.407+0000 7f2767200640 1 Processor -- start 2024-09-21T07:54:13.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.407+0000 7f2767200640 1 -- start start 2024-09-21T07:54:13.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.408+0000 7f2767200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2744004800 0x7f27600b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:13.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.408+0000 7f2767200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2760148e80 0x7f2760152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:13.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.408+0000 7f2767200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2760152640 0x7f27600b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:13.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.409+0000 7f2767200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f27600d0ef0 con 0x7f2760152640 2024-09-21T07:54:13.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.409+0000 7f2767200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f27600d0d70 con 0x7f2744004800 2024-09-21T07:54:13.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.409+0000 7f2767200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f27600d0150 con 0x7f2760148e80 2024-09-21T07:54:13.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.409+0000 7f2765200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2760152640 0x7f27600b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.409+0000 7f2765200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2760152640 0x7f27600b3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50348/0 (socket says 172.21.15.39:50348) 2024-09-21T07:54:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.409+0000 7f2765200640 1 -- 172.21.15.39:0/1235954990 learned_addr learned my addr 172.21.15.39:0/1235954990 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.409+0000 7f275fe00640 1 --2- 172.21.15.39:0/1235954990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2744004800 0x7f27600b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.409+0000 7f2757400640 1 --2- 172.21.15.39:0/1235954990 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2760148e80 0x7f2760152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.409+0000 7f2765200640 1 -- 172.21.15.39:0/1235954990 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2760148e80 msgr2=0x7f2760152100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.409+0000 7f2765200640 1 --2- 172.21.15.39:0/1235954990 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2760148e80 0x7f2760152100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.409+0000 7f2765200640 1 -- 172.21.15.39:0/1235954990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2744004800 msgr2=0x7f27600b4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.409+0000 7f2765200640 1 --2- 172.21.15.39:0/1235954990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2744004800 0x7f27600b4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.409+0000 7f2765200640 1 -- 172.21.15.39:0/1235954990 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27600b5460 con 0x7f2760152640 2024-09-21T07:54:13.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.410+0000 7f2765200640 1 --2- 172.21.15.39:0/1235954990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2760152640 0x7f27600b3a30 secure :-1 s=READY pgs=2711 cs=0 l=1 rev1=1 crypto rx=0x7f27600730e0 tx=0x7f2758011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:13.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.410+0000 7f275cc00640 1 -- 172.21.15.39:0/1235954990 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2758004b20 con 0x7f2760152640 2024-09-21T07:54:13.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.410+0000 7f275cc00640 1 -- 172.21.15.39:0/1235954990 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2758004cc0 con 0x7f2760152640 2024-09-21T07:54:13.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.410+0000 7f275cc00640 1 -- 172.21.15.39:0/1235954990 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2758003bd0 con 0x7f2760152640 2024-09-21T07:54:13.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.410+0000 7f2767200640 1 -- 172.21.15.39:0/1235954990 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2760152a50 con 0x7f2760152640 2024-09-21T07:54:13.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.410+0000 7f2767200640 1 -- 172.21.15.39:0/1235954990 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27601f9a50 con 0x7f2760152640 2024-09-21T07:54:13.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.411+0000 7f2767200640 1 -- 172.21.15.39:0/1235954990 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27600d0620 con 0x7f2760152640 2024-09-21T07:54:13.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.412+0000 7f275cc00640 1 -- 172.21.15.39:0/1235954990 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2758004030 con 0x7f2760152640 2024-09-21T07:54:13.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.413+0000 7f275cc00640 1 --2- 172.21.15.39:0/1235954990 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f273c078b90 0x7f273c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:13.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.414+0000 7f275cc00640 1 -- 172.21.15.39:0/1235954990 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(253..253 src has 1..253) ==== 8035+0+0 (secure 0 0 0) 0x7f275809be30 con 0x7f2760152640 2024-09-21T07:54:13.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.414+0000 7f275fe00640 1 --2- 172.21.15.39:0/1235954990 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f273c078b90 0x7f273c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:13.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.415+0000 7f275fe00640 1 --2- 172.21.15.39:0/1235954990 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f273c078b90 0x7f273c07b0a0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f2750009b30 tx=0x7f2750005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:13.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.418+0000 7f275cc00640 1 -- 172.21.15.39:0/1235954990 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2760152a50 con 0x7f2760152640 2024-09-21T07:54:13.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.651+0000 7f2767200640 1 -- 172.21.15.39:0/1235954990 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27601f91b0 con 0x7f2760152640 2024-09-21T07:54:13.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.652+0000 7f275cc00640 1 -- 172.21.15.39:0/1235954990 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 301 v301) ==== 95+0+5291 (secure 0 0 0) 0x7f2758063390 con 0x7f2760152640 2024-09-21T07:54:13.654 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:13.654 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":301,"btime":"2024-09-21T07:54:11:059441+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9467,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3658109005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3658109005},{"type":"v1","addr":"172.21.15.148:6835","nonce":3658109005}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":298}],"filesystems":[{"mdsmap":{"epoch":301,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:54:11.059432+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":252,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":11364,"mds_1":10633,"mds_2":11970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10633":{"gid":10633,"name":"d","rank":1,"incarnation":295,"state":"up:active","state_seq":19,"addr":"172.21.15.148:6837/3495264983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3495264983},{"type":"v1","addr":"172.21.15.148:6837","nonce":3495264983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11364":{"gid":11364,"name":"a","rank":0,"incarnation":285,"state":"up:active","state_seq":28,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11970":{"gid":11970,"name":"c","rank":2,"incarnation":297,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6835/732408880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":732408880},{"type":"v1","addr":"172.21.15.39:6835","nonce":732408880}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11364,"qdb_cluster":[10633,11364,11970]},"id":13}]} 2024-09-21T07:54:13.654 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 301 2024-09-21T07:54:13.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.654+0000 7f2767200640 1 -- 172.21.15.39:0/1235954990 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f273c078b90 msgr2=0x7f273c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:13.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.654+0000 7f2767200640 1 --2- 172.21.15.39:0/1235954990 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f273c078b90 0x7f273c07b0a0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f2750009b30 tx=0x7f2750005f90 comp rx=0 tx=0).stop 2024-09-21T07:54:13.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.654+0000 7f2767200640 1 -- 172.21.15.39:0/1235954990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2760152640 msgr2=0x7f27600b3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:13.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.654+0000 7f2767200640 1 --2- 172.21.15.39:0/1235954990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2760152640 0x7f27600b3a30 secure :-1 s=READY pgs=2711 cs=0 l=1 rev1=1 crypto rx=0x7f27600730e0 tx=0x7f2758011f80 comp rx=0 tx=0).stop 2024-09-21T07:54:13.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.654+0000 7f2767200640 1 -- 172.21.15.39:0/1235954990 shutdown_connections 2024-09-21T07:54:13.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.654+0000 7f2767200640 1 --2- 172.21.15.39:0/1235954990 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f273c078b90 0x7f273c07b0a0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:13.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.654+0000 7f2767200640 1 --2- 172.21.15.39:0/1235954990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2760152640 0x7f27600b3a30 unknown :-1 s=CLOSED pgs=2711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:13.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.654+0000 7f2767200640 1 --2- 172.21.15.39:0/1235954990 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f2760148e80 0x7f2760152100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:13.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.654+0000 7f2767200640 1 --2- 172.21.15.39:0/1235954990 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2744004800 0x7f27600b4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:13.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.654+0000 7f2767200640 1 -- 172.21.15.39:0/1235954990 >> 172.21.15.39:0/1235954990 conn(0x7f276006cfc0 msgr2=0x7f2760070f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:13.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.654+0000 7f2767200640 1 -- 172.21.15.39:0/1235954990 shutdown_connections 2024-09-21T07:54:13.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:13.654+0000 7f2767200640 1 -- 172.21.15.39:0/1235954990 wait complete. 2024-09-21T07:54:13.681 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json dump snaps 2024-09-21T07:54:13.928 INFO:teuthology.orchestra.run.smithi148.stdout:{"last_created":10,"last_destroyed":7,"snaps":[{"snapid":8,"ino":1099511627780,"stamp":"2024-09-21T07:51:12.091986+0000","name":"s1","metadata":{}},{"snapid":9,"ino":1099511627780,"stamp":"2024-09-21T07:52:44.191763+0000","name":"s2","metadata":{}},{"snapid":10,"ino":1099511627780,"stamp":"2024-09-21T07:53:39.355926+0000","name":"s3","metadata":{}}]} 2024-09-21T07:54:13.928 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 10, "last_destroyed": 7, "snaps": [ { "ino": 1099511627780, "metadata": {}, "name": "s1", "snapid": 8, "stamp": "2024-09-21T07:51:12.091986+0000" }, { "ino": 1099511627780, "metadata": {}, "name": "s2", "snapid": 9, "stamp": "2024-09-21T07:52:44.191763+0000" }, { "ino": 1099511627780, "metadata": {}, "name": "s3", "snapid": 10, "stamp": "2024-09-21T07:53:39.355926+0000" } ] } 2024-09-21T07:54:13.928 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:54:14.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.156+0000 7fea9b200640 1 -- 172.21.15.39:0/2565152996 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea9c161fe0 msgr2=0x7fea9c06f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:14.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.156+0000 7fea9b200640 1 --2- 172.21.15.39:0/2565152996 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea9c161fe0 0x7fea9c06f7d0 secure :-1 s=READY pgs=2713 cs=0 l=1 rev1=1 crypto rx=0x7fea9c08ede0 tx=0x7fea8402a540 comp rx=0 tx=0).stop 2024-09-21T07:54:14.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.156+0000 7fea9b200640 1 -- 172.21.15.39:0/2565152996 shutdown_connections 2024-09-21T07:54:14.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.156+0000 7fea9b200640 1 --2- 172.21.15.39:0/2565152996 >> v2:172.21.15.148:3300/0 conn(0x7fea9c0d0d20 0x7fea9c0d1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:14.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.156+0000 7fea9b200640 1 --2- 172.21.15.39:0/2565152996 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea9c161fe0 0x7fea9c06f7d0 unknown :-1 s=CLOSED pgs=2713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:14.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.156+0000 7fea9b200640 1 -- 172.21.15.39:0/2565152996 >> 172.21.15.39:0/2565152996 conn(0x7fea9c08bfb0 msgr2=0x7fea9c09d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:14.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.156+0000 7fea9b200640 1 -- 172.21.15.39:0/2565152996 shutdown_connections 2024-09-21T07:54:14.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.156+0000 7fea9b200640 1 -- 172.21.15.39:0/2565152996 wait complete. 2024-09-21T07:54:14.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.158+0000 7fea9b200640 1 Processor -- start 2024-09-21T07:54:14.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.158+0000 7fea9b200640 1 -- start start 2024-09-21T07:54:14.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.160+0000 7fea9b200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea9c0b96b0 0x7fea9c0ca960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:14.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.160+0000 7fea9b200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea9c0d0d20 0x7fea9c15bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:14.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.160+0000 7fea9b200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea9c161fe0 0x7fea9c0d4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:14.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.160+0000 7fea9b200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fea9c0d26a0 con 0x7fea9c0b96b0 2024-09-21T07:54:14.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.160+0000 7fea9b200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fea9c0d2520 con 0x7fea9c0d0d20 2024-09-21T07:54:14.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.160+0000 7fea9b200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fea9c0d2820 con 0x7fea9c161fe0 2024-09-21T07:54:14.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.160+0000 7fea99e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea9c0b96b0 0x7fea9c0ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:14.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.160+0000 7fea99e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea9c0b96b0 0x7fea9c0ca960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50362/0 (socket says 172.21.15.39:50362) 2024-09-21T07:54:14.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.160+0000 7fea99e00640 1 -- 172.21.15.39:0/952753181 learned_addr learned my addr 172.21.15.39:0/952753181 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:14.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.160+0000 7fea91400640 1 --2- 172.21.15.39:0/952753181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea9c0d0d20 0x7fea9c15bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:14.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.160+0000 7fea9a800640 1 --2- 172.21.15.39:0/952753181 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea9c161fe0 0x7fea9c0d4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:14.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.160+0000 7fea99e00640 1 -- 172.21.15.39:0/952753181 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea9c161fe0 msgr2=0x7fea9c0d4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:14.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.160+0000 7fea99e00640 1 --2- 172.21.15.39:0/952753181 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea9c161fe0 0x7fea9c0d4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:14.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.160+0000 7fea99e00640 1 -- 172.21.15.39:0/952753181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea9c0d0d20 msgr2=0x7fea9c15bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:14.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.160+0000 7fea99e00640 1 --2- 172.21.15.39:0/952753181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea9c0d0d20 0x7fea9c15bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:14.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.160+0000 7fea99e00640 1 -- 172.21.15.39:0/952753181 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea9c1f5800 con 0x7fea9c0b96b0 2024-09-21T07:54:14.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.160+0000 7fea9a800640 1 --2- 172.21.15.39:0/952753181 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea9c161fe0 0x7fea9c0d4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:14.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.161+0000 7fea99e00640 1 --2- 172.21.15.39:0/952753181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea9c0b96b0 0x7fea9c0ca960 secure :-1 s=READY pgs=2714 cs=0 l=1 rev1=1 crypto rx=0x7fea8c00c3c0 tx=0x7fea8c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:14.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.161+0000 7fea92a00640 1 -- 172.21.15.39:0/952753181 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c017070 con 0x7fea9c0b96b0 2024-09-21T07:54:14.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.161+0000 7fea92a00640 1 -- 172.21.15.39:0/952753181 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea8c0035f0 con 0x7fea9c0b96b0 2024-09-21T07:54:14.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.161+0000 7fea9b200640 1 -- 172.21.15.39:0/952753181 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea9c1f5ae0 con 0x7fea9c0b96b0 2024-09-21T07:54:14.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.161+0000 7fea92a00640 1 -- 172.21.15.39:0/952753181 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c003a80 con 0x7fea9c0b96b0 2024-09-21T07:54:14.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.161+0000 7fea9b200640 1 -- 172.21.15.39:0/952753181 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea9c1f60a0 con 0x7fea9c0b96b0 2024-09-21T07:54:14.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.162+0000 7fea9b200640 1 -- 172.21.15.39:0/952753181 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea9c0d2cd0 con 0x7fea9c0b96b0 2024-09-21T07:54:14.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.164+0000 7fea92a00640 1 -- 172.21.15.39:0/952753181 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fea8c01a070 con 0x7fea9c0b96b0 2024-09-21T07:54:14.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.164+0000 7fea92a00640 1 --2- 172.21.15.39:0/952753181 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fea78078bc0 0x7fea7807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:14.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.165+0000 7fea92a00640 1 -- 172.21.15.39:0/952753181 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(253..253 src has 1..253) ==== 8035+0+0 (secure 0 0 0) 0x7fea8c09a7d0 con 0x7fea9c0b96b0 2024-09-21T07:54:14.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.165+0000 7fea91400640 1 --2- 172.21.15.39:0/952753181 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fea78078bc0 0x7fea7807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:14.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.166+0000 7fea91400640 1 --2- 172.21.15.39:0/952753181 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fea78078bc0 0x7fea7807b0d0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fea84034000 tx=0x7fea84004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:14.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.169+0000 7fea92a00640 1 -- 172.21.15.39:0/952753181 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fea8c061ca0 con 0x7fea9c0b96b0 2024-09-21T07:54:14.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.409+0000 7fea9b200640 1 -- 172.21.15.39:0/952753181 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea9c0d1430 con 0x7fea9c0b96b0 2024-09-21T07:54:14.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.410+0000 7fea92a00640 1 -- 172.21.15.39:0/952753181 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 301 v301) ==== 95+0+5291 (secure 0 0 0) 0x7fea8c0666d0 con 0x7fea9c0b96b0 2024-09-21T07:54:14.412 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:14.412 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":301,"btime":"2024-09-21T07:54:11:059441+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9467,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3658109005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3658109005},{"type":"v1","addr":"172.21.15.148:6835","nonce":3658109005}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":298}],"filesystems":[{"mdsmap":{"epoch":301,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:54:11.059432+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":252,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":11364,"mds_1":10633,"mds_2":11970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10633":{"gid":10633,"name":"d","rank":1,"incarnation":295,"state":"up:active","state_seq":19,"addr":"172.21.15.148:6837/3495264983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3495264983},{"type":"v1","addr":"172.21.15.148:6837","nonce":3495264983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11364":{"gid":11364,"name":"a","rank":0,"incarnation":285,"state":"up:active","state_seq":28,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11970":{"gid":11970,"name":"c","rank":2,"incarnation":297,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6835/732408880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":732408880},{"type":"v1","addr":"172.21.15.39:6835","nonce":732408880}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11364,"qdb_cluster":[10633,11364,11970]},"id":13}]} 2024-09-21T07:54:14.413 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 301 2024-09-21T07:54:14.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.412+0000 7fea9b200640 1 -- 172.21.15.39:0/952753181 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fea78078bc0 msgr2=0x7fea7807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:14.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.412+0000 7fea9b200640 1 --2- 172.21.15.39:0/952753181 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fea78078bc0 0x7fea7807b0d0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fea84034000 tx=0x7fea84004730 comp rx=0 tx=0).stop 2024-09-21T07:54:14.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.412+0000 7fea9b200640 1 -- 172.21.15.39:0/952753181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea9c0b96b0 msgr2=0x7fea9c0ca960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:14.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.412+0000 7fea9b200640 1 --2- 172.21.15.39:0/952753181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea9c0b96b0 0x7fea9c0ca960 secure :-1 s=READY pgs=2714 cs=0 l=1 rev1=1 crypto rx=0x7fea8c00c3c0 tx=0x7fea8c00c880 comp rx=0 tx=0).stop 2024-09-21T07:54:14.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.412+0000 7fea9b200640 1 -- 172.21.15.39:0/952753181 shutdown_connections 2024-09-21T07:54:14.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.412+0000 7fea9b200640 1 --2- 172.21.15.39:0/952753181 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fea78078bc0 0x7fea7807b0d0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:14.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.412+0000 7fea9b200640 1 --2- 172.21.15.39:0/952753181 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fea9c161fe0 0x7fea9c0d4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:14.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.412+0000 7fea9b200640 1 --2- 172.21.15.39:0/952753181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea9c0d0d20 0x7fea9c15bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:14.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.412+0000 7fea9b200640 1 --2- 172.21.15.39:0/952753181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea9c0b96b0 0x7fea9c0ca960 unknown :-1 s=CLOSED pgs=2714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:14.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.412+0000 7fea9b200640 1 -- 172.21.15.39:0/952753181 >> 172.21.15.39:0/952753181 conn(0x7fea9c08bfb0 msgr2=0x7fea9c068f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:14.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.412+0000 7fea9b200640 1 -- 172.21.15.39:0/952753181 shutdown_connections 2024-09-21T07:54:14.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.412+0000 7fea9b200640 1 -- 172.21.15.39:0/952753181 wait complete. 2024-09-21T07:54:14.439 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump snaps 2024-09-21T07:54:14.683 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_created":10,"last_destroyed":7,"snaps":[{"snapid":8,"ino":1099511627780,"stamp":"2024-09-21T07:51:12.091986+0000","name":"s1","metadata":{}},{"snapid":9,"ino":1099511627780,"stamp":"2024-09-21T07:52:44.191763+0000","name":"s2","metadata":{}},{"snapid":10,"ino":1099511627780,"stamp":"2024-09-21T07:53:39.355926+0000","name":"s3","metadata":{}}]} 2024-09-21T07:54:14.683 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 10, "last_destroyed": 7, "snaps": [ { "ino": 1099511627780, "metadata": {}, "name": "s1", "snapid": 8, "stamp": "2024-09-21T07:51:12.091986+0000" }, { "ino": 1099511627780, "metadata": {}, "name": "s2", "snapid": 9, "stamp": "2024-09-21T07:52:44.191763+0000" }, { "ino": 1099511627780, "metadata": {}, "name": "s3", "snapid": 10, "stamp": "2024-09-21T07:53:39.355926+0000" } ] } 2024-09-21T07:54:14.684 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:54:14.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.908+0000 7f7312800640 1 -- 172.21.15.39:0/1894049354 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f730c15fc30 msgr2=0x7f730c160320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:14.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.908+0000 7f7312800640 1 --2- 172.21.15.39:0/1894049354 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f730c15fc30 0x7f730c160320 secure :-1 s=READY pgs=2716 cs=0 l=1 rev1=1 crypto rx=0x7f730c0c3ca0 tx=0x7f730002a560 comp rx=0 tx=0).stop 2024-09-21T07:54:14.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.908+0000 7f7312800640 1 -- 172.21.15.39:0/1894049354 shutdown_connections 2024-09-21T07:54:14.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.908+0000 7f7312800640 1 --2- 172.21.15.39:0/1894049354 >> v2:172.21.15.148:3300/0 conn(0x7f730c160860 0x7f730c0cb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:14.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.908+0000 7f7312800640 1 --2- 172.21.15.39:0/1894049354 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f730c15fc30 0x7f730c160320 unknown :-1 s=CLOSED pgs=2716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:14.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.908+0000 7f7312800640 1 -- 172.21.15.39:0/1894049354 >> 172.21.15.39:0/1894049354 conn(0x7f730c0b2510 msgr2=0x7f730c0d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:14.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.909+0000 7f7312800640 1 -- 172.21.15.39:0/1894049354 shutdown_connections 2024-09-21T07:54:14.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.909+0000 7f7312800640 1 -- 172.21.15.39:0/1894049354 wait complete. 2024-09-21T07:54:14.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.911+0000 7f7312800640 1 Processor -- start 2024-09-21T07:54:14.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.911+0000 7f7312800640 1 -- start start 2024-09-21T07:54:14.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.913+0000 7f7312800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f730c0cbef0 0x7f730c0ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:14.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.913+0000 7f7312800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f730c15fc30 0x7f730c0aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:14.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.913+0000 7f7312800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f730c160860 0x7f730c1f93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:14.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.913+0000 7f7312800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f730c0b37e0 con 0x7f730c160860 2024-09-21T07:54:14.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.913+0000 7f7312800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f730c0cd650 con 0x7f730c0cbef0 2024-09-21T07:54:14.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.913+0000 7f7312800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f730c0b3960 con 0x7f730c15fc30 2024-09-21T07:54:14.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.913+0000 7f7311e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f730c160860 0x7f730c1f93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:14.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.913+0000 7f7311e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f730c160860 0x7f730c1f93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50378/0 (socket says 172.21.15.39:50378) 2024-09-21T07:54:14.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.913+0000 7f7311e00640 1 -- 172.21.15.39:0/3884994470 learned_addr learned my addr 172.21.15.39:0/3884994470 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:14.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.913+0000 7f7311400640 1 --2- 172.21.15.39:0/3884994470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f730c0cbef0 0x7f730c0ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:14.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.913+0000 7f7310a00640 1 --2- 172.21.15.39:0/3884994470 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f730c15fc30 0x7f730c0aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:14.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.913+0000 7f7311e00640 1 -- 172.21.15.39:0/3884994470 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f730c15fc30 msgr2=0x7f730c0aebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:14.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.913+0000 7f7311e00640 1 --2- 172.21.15.39:0/3884994470 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f730c15fc30 0x7f730c0aebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:14.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.913+0000 7f7311e00640 1 -- 172.21.15.39:0/3884994470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f730c0cbef0 msgr2=0x7f730c0ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:14.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.913+0000 7f7311e00640 1 --2- 172.21.15.39:0/3884994470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f730c0cbef0 0x7f730c0ae6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:14.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.913+0000 7f7311e00640 1 -- 172.21.15.39:0/3884994470 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f730c1f6130 con 0x7f730c160860 2024-09-21T07:54:14.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.914+0000 7f7311e00640 1 --2- 172.21.15.39:0/3884994470 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f730c160860 0x7f730c1f93b0 secure :-1 s=READY pgs=2717 cs=0 l=1 rev1=1 crypto rx=0x7f730400c700 tx=0x7f730400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:14.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.914+0000 7f7309600640 1 -- 172.21.15.39:0/3884994470 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7304003b70 con 0x7f730c160860 2024-09-21T07:54:14.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.914+0000 7f7309600640 1 -- 172.21.15.39:0/3884994470 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7304003d10 con 0x7f730c160860 2024-09-21T07:54:14.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.914+0000 7f7309600640 1 -- 172.21.15.39:0/3884994470 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730400aab0 con 0x7f730c160860 2024-09-21T07:54:14.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.914+0000 7f7312800640 1 -- 172.21.15.39:0/3884994470 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f730c1f6410 con 0x7f730c160860 2024-09-21T07:54:14.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.914+0000 7f7312800640 1 -- 172.21.15.39:0/3884994470 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f730c1f69d0 con 0x7f730c160860 2024-09-21T07:54:14.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.915+0000 7f7312800640 1 -- 172.21.15.39:0/3884994470 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f730c1f5ea0 con 0x7f730c160860 2024-09-21T07:54:14.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.917+0000 7f7309600640 1 -- 172.21.15.39:0/3884994470 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f730400ac50 con 0x7f730c160860 2024-09-21T07:54:14.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.917+0000 7f7309600640 1 --2- 172.21.15.39:0/3884994470 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f72ec078b90 0x7f72ec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:14.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.918+0000 7f7309600640 1 -- 172.21.15.39:0/3884994470 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(253..253 src has 1..253) ==== 8035+0+0 (secure 0 0 0) 0x7f730409b390 con 0x7f730c160860 2024-09-21T07:54:14.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.918+0000 7f7311400640 1 --2- 172.21.15.39:0/3884994470 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f72ec078b90 0x7f72ec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:14.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.919+0000 7f7311400640 1 --2- 172.21.15.39:0/3884994470 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f72ec078b90 0x7f72ec07b0a0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f7300033000 tx=0x7f7300004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:14.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:14.922+0000 7f7309600640 1 -- 172.21.15.39:0/3884994470 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f730c160c70 con 0x7f730c160860 2024-09-21T07:54:15.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.164+0000 7f7312800640 1 -- 172.21.15.39:0/3884994470 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f730c0b3e90 con 0x7f730c160860 2024-09-21T07:54:15.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.164+0000 7f7309600640 1 -- 172.21.15.39:0/3884994470 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 301 v301) ==== 95+0+5291 (secure 0 0 0) 0x7f73040627c0 con 0x7f730c160860 2024-09-21T07:54:15.166 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:15.167 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":301,"btime":"2024-09-21T07:54:11:059441+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9467,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3658109005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3658109005},{"type":"v1","addr":"172.21.15.148:6835","nonce":3658109005}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":298}],"filesystems":[{"mdsmap":{"epoch":301,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:54:11.059432+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":252,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":11364,"mds_1":10633,"mds_2":11970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10633":{"gid":10633,"name":"d","rank":1,"incarnation":295,"state":"up:active","state_seq":19,"addr":"172.21.15.148:6837/3495264983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3495264983},{"type":"v1","addr":"172.21.15.148:6837","nonce":3495264983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11364":{"gid":11364,"name":"a","rank":0,"incarnation":285,"state":"up:active","state_seq":28,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11970":{"gid":11970,"name":"c","rank":2,"incarnation":297,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6835/732408880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":732408880},{"type":"v1","addr":"172.21.15.39:6835","nonce":732408880}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11364,"qdb_cluster":[10633,11364,11970]},"id":13}]} 2024-09-21T07:54:15.167 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 301 2024-09-21T07:54:15.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.167+0000 7f7312800640 1 -- 172.21.15.39:0/3884994470 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f72ec078b90 msgr2=0x7f72ec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:15.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.167+0000 7f7312800640 1 --2- 172.21.15.39:0/3884994470 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f72ec078b90 0x7f72ec07b0a0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f7300033000 tx=0x7f7300004730 comp rx=0 tx=0).stop 2024-09-21T07:54:15.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.167+0000 7f7312800640 1 -- 172.21.15.39:0/3884994470 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f730c160860 msgr2=0x7f730c1f93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:15.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.167+0000 7f7312800640 1 --2- 172.21.15.39:0/3884994470 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f730c160860 0x7f730c1f93b0 secure :-1 s=READY pgs=2717 cs=0 l=1 rev1=1 crypto rx=0x7f730400c700 tx=0x7f730400cbc0 comp rx=0 tx=0).stop 2024-09-21T07:54:15.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.167+0000 7f7312800640 1 -- 172.21.15.39:0/3884994470 shutdown_connections 2024-09-21T07:54:15.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.167+0000 7f7312800640 1 --2- 172.21.15.39:0/3884994470 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f72ec078b90 0x7f72ec07b0a0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:15.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.167+0000 7f7312800640 1 --2- 172.21.15.39:0/3884994470 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f730c160860 0x7f730c1f93b0 unknown :-1 s=CLOSED pgs=2717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:15.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.167+0000 7f7312800640 1 --2- 172.21.15.39:0/3884994470 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f730c15fc30 0x7f730c0aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:15.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.167+0000 7f7312800640 1 --2- 172.21.15.39:0/3884994470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f730c0cbef0 0x7f730c0ae6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:15.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.167+0000 7f7312800640 1 -- 172.21.15.39:0/3884994470 >> 172.21.15.39:0/3884994470 conn(0x7f730c0b2510 msgr2=0x7f730c0afd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:15.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.167+0000 7f7312800640 1 -- 172.21.15.39:0/3884994470 shutdown_connections 2024-09-21T07:54:15.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.167+0000 7f7312800640 1 -- 172.21.15.39:0/3884994470 wait complete. 2024-09-21T07:54:15.192 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump snaps --server 2024-09-21T07:54:15.438 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_snap":10,"last_created":10,"last_destroyed":7,"pending_noop":[],"snaps":[{"snapid":8,"ino":1099511627780,"stamp":"2024-09-21T07:51:12.091986+0000","name":"s1","metadata":{}},{"snapid":9,"ino":1099511627780,"stamp":"2024-09-21T07:52:44.191763+0000","name":"s2","metadata":{}},{"snapid":10,"ino":1099511627780,"stamp":"2024-09-21T07:53:39.355926+0000","name":"s3","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2024-09-21T07:54:15.439 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 10, "last_destroyed": 7, "last_snap": 10, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627780, "metadata": {}, "name": "s1", "snapid": 8, "stamp": "2024-09-21T07:51:12.091986+0000" }, { "ino": 1099511627780, "metadata": {}, "name": "s2", "snapid": 9, "stamp": "2024-09-21T07:52:44.191763+0000" }, { "ino": 1099511627780, "metadata": {}, "name": "s3", "snapid": 10, "stamp": "2024-09-21T07:53:39.355926+0000" } ] } 2024-09-21T07:54:15.439 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T07:54:15.439 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:54:15.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.665+0000 7f0a51200640 1 -- 172.21.15.39:0/3310003186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a30004800 msgr2=0x7f0a30004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:15.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.665+0000 7f0a51200640 1 --2- 172.21.15.39:0/3310003186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a30004800 0x7f0a30004c10 secure :-1 s=READY pgs=2719 cs=0 l=1 rev1=1 crypto rx=0x7f0a38009b50 tx=0x7f0a30005830 comp rx=0 tx=0).stop 2024-09-21T07:54:15.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.665+0000 7f0a4be00640 1 -- 172.21.15.39:0/3310003186 reap_dead start 2024-09-21T07:54:15.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.665+0000 7f0a51200640 1 -- 172.21.15.39:0/3310003186 shutdown_connections 2024-09-21T07:54:15.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.665+0000 7f0a51200640 1 -- 172.21.15.39:0/3310003186 >> 172.21.15.39:0/3310003186 conn(0x7f0a4c08bfb0 msgr2=0x7f0a4c09d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:15.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.666+0000 7f0a51200640 1 -- 172.21.15.39:0/3310003186 shutdown_connections 2024-09-21T07:54:15.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.666+0000 7f0a51200640 1 -- 172.21.15.39:0/3310003186 wait complete. 2024-09-21T07:54:15.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.668+0000 7f0a51200640 1 Processor -- start 2024-09-21T07:54:15.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.668+0000 7f0a51200640 1 -- start start 2024-09-21T07:54:15.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.669+0000 7f0a51200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a30004800 0x7f0a4c14e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:15.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.669+0000 7f0a51200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a4c1f8c50 0x7f0a4c1f9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:15.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.669+0000 7f0a51200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0a4c1f95a0 0x7f0a4c0c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:15.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.669+0000 7f0a51200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4c0d2310 con 0x7f0a4c1f8c50 2024-09-21T07:54:15.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.669+0000 7f0a51200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4c06fd90 con 0x7f0a30004800 2024-09-21T07:54:15.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.669+0000 7f0a51200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f0a4c0d2490 con 0x7f0a4c1f95a0 2024-09-21T07:54:15.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.670+0000 7f0a4aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a4c1f8c50 0x7f0a4c1f9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:15.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.670+0000 7f0a4aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a4c1f8c50 0x7f0a4c1f9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50396/0 (socket says 172.21.15.39:50396) 2024-09-21T07:54:15.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.670+0000 7f0a4aa00640 1 -- 172.21.15.39:0/1673478010 learned_addr learned my addr 172.21.15.39:0/1673478010 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:15.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.670+0000 7f0a4be00640 1 --2- 172.21.15.39:0/1673478010 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0a4c1f95a0 0x7f0a4c0c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:15.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.670+0000 7f0a4b400640 1 --2- 172.21.15.39:0/1673478010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a30004800 0x7f0a4c14e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:15.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.670+0000 7f0a4aa00640 1 -- 172.21.15.39:0/1673478010 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0a4c1f95a0 msgr2=0x7f0a4c0c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:15.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.670+0000 7f0a4aa00640 1 --2- 172.21.15.39:0/1673478010 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0a4c1f95a0 0x7f0a4c0c91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:15.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.670+0000 7f0a4aa00640 1 -- 172.21.15.39:0/1673478010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a30004800 msgr2=0x7f0a4c14e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:15.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.670+0000 7f0a4aa00640 1 --2- 172.21.15.39:0/1673478010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a30004800 0x7f0a4c14e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:15.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.670+0000 7f0a4aa00640 1 -- 172.21.15.39:0/1673478010 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a4c0ce550 con 0x7f0a4c1f8c50 2024-09-21T07:54:15.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.670+0000 7f0a4b400640 1 --2- 172.21.15.39:0/1673478010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a30004800 0x7f0a4c14e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:54:15.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.671+0000 7f0a4aa00640 1 --2- 172.21.15.39:0/1673478010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a4c1f8c50 0x7f0a4c1f9060 secure :-1 s=READY pgs=2720 cs=0 l=1 rev1=1 crypto rx=0x7f0a38036040 tx=0x7f0a38033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:15.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.671+0000 7f0a43400640 1 -- 172.21.15.39:0/1673478010 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a38043070 con 0x7f0a4c1f8c50 2024-09-21T07:54:15.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.671+0000 7f0a43400640 1 -- 172.21.15.39:0/1673478010 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a38004320 con 0x7f0a4c1f8c50 2024-09-21T07:54:15.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.671+0000 7f0a51200640 1 -- 172.21.15.39:0/1673478010 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a4c0b6490 con 0x7f0a4c1f8c50 2024-09-21T07:54:15.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.671+0000 7f0a43400640 1 -- 172.21.15.39:0/1673478010 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a38038040 con 0x7f0a4c1f8c50 2024-09-21T07:54:15.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.671+0000 7f0a51200640 1 -- 172.21.15.39:0/1673478010 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a4c0b6a50 con 0x7f0a4c1f8c50 2024-09-21T07:54:15.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.672+0000 7f0a51200640 1 -- 172.21.15.39:0/1673478010 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a4c0b40f0 con 0x7f0a4c1f8c50 2024-09-21T07:54:15.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.673+0000 7f0a43400640 1 -- 172.21.15.39:0/1673478010 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0a3802b8a0 con 0x7f0a4c1f8c50 2024-09-21T07:54:15.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.674+0000 7f0a43400640 1 --2- 172.21.15.39:0/1673478010 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0a2c078b90 0x7f0a2c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:15.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.674+0000 7f0a43400640 1 -- 172.21.15.39:0/1673478010 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(253..253 src has 1..253) ==== 8035+0+0 (secure 0 0 0) 0x7f0a380bbd80 con 0x7f0a4c1f8c50 2024-09-21T07:54:15.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.675+0000 7f0a4b400640 1 --2- 172.21.15.39:0/1673478010 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0a2c078b90 0x7f0a2c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:15.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.676+0000 7f0a4b400640 1 --2- 172.21.15.39:0/1673478010 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0a2c078b90 0x7f0a2c07b0a0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c001a10 tx=0x7f0a3c001e30 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:15.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.679+0000 7f0a43400640 1 -- 172.21.15.39:0/1673478010 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0a38083130 con 0x7f0a4c1f8c50 2024-09-21T07:54:15.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.915+0000 7f0a51200640 1 -- 172.21.15.39:0/1673478010 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a4c0d5120 con 0x7f0a4c1f8c50 2024-09-21T07:54:15.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.916+0000 7f0a43400640 1 -- 172.21.15.39:0/1673478010 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 301 v301) ==== 95+0+5291 (secure 0 0 0) 0x7f0a4c0d5120 con 0x7f0a4c1f8c50 2024-09-21T07:54:15.918 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:15.918 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":301,"btime":"2024-09-21T07:54:11:059441+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9467,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3658109005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3658109005},{"type":"v1","addr":"172.21.15.148:6835","nonce":3658109005}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":298}],"filesystems":[{"mdsmap":{"epoch":301,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:54:11.059432+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":252,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":11364,"mds_1":10633,"mds_2":11970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10633":{"gid":10633,"name":"d","rank":1,"incarnation":295,"state":"up:active","state_seq":19,"addr":"172.21.15.148:6837/3495264983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3495264983},{"type":"v1","addr":"172.21.15.148:6837","nonce":3495264983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11364":{"gid":11364,"name":"a","rank":0,"incarnation":285,"state":"up:active","state_seq":28,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11970":{"gid":11970,"name":"c","rank":2,"incarnation":297,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6835/732408880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":732408880},{"type":"v1","addr":"172.21.15.39:6835","nonce":732408880}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11364,"qdb_cluster":[10633,11364,11970]},"id":13}]} 2024-09-21T07:54:15.918 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 301 2024-09-21T07:54:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.918+0000 7f0a51200640 1 -- 172.21.15.39:0/1673478010 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0a2c078b90 msgr2=0x7f0a2c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.918+0000 7f0a51200640 1 --2- 172.21.15.39:0/1673478010 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0a2c078b90 0x7f0a2c07b0a0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c001a10 tx=0x7f0a3c001e30 comp rx=0 tx=0).stop 2024-09-21T07:54:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.918+0000 7f0a51200640 1 -- 172.21.15.39:0/1673478010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a4c1f8c50 msgr2=0x7f0a4c1f9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.918+0000 7f0a51200640 1 --2- 172.21.15.39:0/1673478010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a4c1f8c50 0x7f0a4c1f9060 secure :-1 s=READY pgs=2720 cs=0 l=1 rev1=1 crypto rx=0x7f0a38036040 tx=0x7f0a38033670 comp rx=0 tx=0).stop 2024-09-21T07:54:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.918+0000 7f0a51200640 1 -- 172.21.15.39:0/1673478010 shutdown_connections 2024-09-21T07:54:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.918+0000 7f0a51200640 1 --2- 172.21.15.39:0/1673478010 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0a2c078b90 0x7f0a2c07b0a0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.918+0000 7f0a51200640 1 --2- 172.21.15.39:0/1673478010 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f0a4c1f95a0 0x7f0a4c0c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.918+0000 7f0a51200640 1 --2- 172.21.15.39:0/1673478010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a4c1f8c50 0x7f0a4c1f9060 unknown :-1 s=CLOSED pgs=2720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.918+0000 7f0a51200640 1 --2- 172.21.15.39:0/1673478010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a30004800 0x7f0a4c14e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.918+0000 7f0a51200640 1 -- 172.21.15.39:0/1673478010 >> 172.21.15.39:0/1673478010 conn(0x7f0a4c08bfb0 msgr2=0x7f0a4c0d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.919+0000 7f0a51200640 1 -- 172.21.15.39:0/1673478010 shutdown_connections 2024-09-21T07:54:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:15.919+0000 7f0a51200640 1 -- 172.21.15.39:0/1673478010 wait complete. 2024-09-21T07:54:15.943 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump snaps --server 2024-09-21T07:54:16.187 INFO:teuthology.orchestra.run.smithi039.stdout:{"last_snap":10,"last_created":10,"last_destroyed":7,"pending_noop":[],"snaps":[{"snapid":8,"ino":1099511627780,"stamp":"2024-09-21T07:51:12.091986+0000","name":"s1","metadata":{}},{"snapid":9,"ino":1099511627780,"stamp":"2024-09-21T07:52:44.191763+0000","name":"s2","metadata":{}},{"snapid":10,"ino":1099511627780,"stamp":"2024-09-21T07:53:39.355926+0000","name":"s3","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2024-09-21T07:54:16.187 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 10, "last_destroyed": 7, "last_snap": 10, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627780, "metadata": {}, "name": "s1", "snapid": 8, "stamp": "2024-09-21T07:51:12.091986+0000" }, { "ino": 1099511627780, "metadata": {}, "name": "s2", "snapid": 9, "stamp": "2024-09-21T07:52:44.191763+0000" }, { "ino": 1099511627780, "metadata": {}, "name": "s3", "snapid": 10, "stamp": "2024-09-21T07:53:39.355926+0000" } ] } 2024-09-21T07:54:16.187 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:54:16.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:16.188 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-09-21T07:54:16.747 INFO:teuthology.orchestra.run.smithi039.stderr:mkdir: cannot create directory ‘d0/d2/dir/.snap/s3’: Permission denied 2024-09-21T07:54:16.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:54:16.749 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:54:16.749 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:16.785 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:54:16.785 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:16.861 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:54:17.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.126+0000 7f3576800640 1 -- 172.21.15.39:0/292090841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35700cfd10 msgr2=0x7f35700cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:17.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.126+0000 7f3576800640 1 --2- 172.21.15.39:0/292090841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35700cfd10 0x7f35700cd8e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f3558009830 tx=0x7f355802a4b0 comp rx=0 tx=0).stop 2024-09-21T07:54:17.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.126+0000 7f3576800640 1 -- 172.21.15.39:0/292090841 shutdown_connections 2024-09-21T07:54:17.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.126+0000 7f3576800640 1 --2- 172.21.15.39:0/292090841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35700cfd10 0x7f35700cd8e0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:17.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.126+0000 7f3576800640 1 -- 172.21.15.39:0/292090841 >> 172.21.15.39:0/292090841 conn(0x7f357006cfc0 msgr2=0x7f357006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:17.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.126+0000 7f3576800640 1 -- 172.21.15.39:0/292090841 shutdown_connections 2024-09-21T07:54:17.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.126+0000 7f3576800640 1 -- 172.21.15.39:0/292090841 wait complete. 2024-09-21T07:54:17.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.128+0000 7f3576800640 1 Processor -- start 2024-09-21T07:54:17.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.128+0000 7f3576800640 1 -- start start 2024-09-21T07:54:17.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.130+0000 7f3576800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35700cfd10 0x7f357014d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:17.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.130+0000 7f3576800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35700d0620 0x7f35701e92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:17.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.130+0000 7f3576800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f357015cd30 0x7f35701e9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:17.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.130+0000 7f3576800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3570073aa0 con 0x7f35700d0620 2024-09-21T07:54:17.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.130+0000 7f3576800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3570073920 con 0x7f357015cd30 2024-09-21T07:54:17.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.130+0000 7f3576800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f3570073c20 con 0x7f35700cfd10 2024-09-21T07:54:17.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.130+0000 7f3566a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35700d0620 0x7f35701e92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:17.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.130+0000 7f3566a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35700d0620 0x7f35701e92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50398/0 (socket says 172.21.15.39:50398) 2024-09-21T07:54:17.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.130+0000 7f3566a00640 1 -- 172.21.15.39:0/2675757359 learned_addr learned my addr 172.21.15.39:0/2675757359 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:17.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.130+0000 7f356f400640 1 --2- 172.21.15.39:0/2675757359 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35700cfd10 0x7f357014d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:17.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.130+0000 7f356fe00640 1 --2- 172.21.15.39:0/2675757359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f357015cd30 0x7f35701e9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:17.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.131+0000 7f3566a00640 1 -- 172.21.15.39:0/2675757359 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35700cfd10 msgr2=0x7f357014d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:17.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.131+0000 7f3566a00640 1 --2- 172.21.15.39:0/2675757359 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35700cfd10 0x7f357014d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:17.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.131+0000 7f3566a00640 1 -- 172.21.15.39:0/2675757359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f357015cd30 msgr2=0x7f35701e9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:17.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.131+0000 7f3566a00640 1 --2- 172.21.15.39:0/2675757359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f357015cd30 0x7f35701e9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:17.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.131+0000 7f3566a00640 1 -- 172.21.15.39:0/2675757359 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3570155130 con 0x7f35700d0620 2024-09-21T07:54:17.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.131+0000 7f3566a00640 1 --2- 172.21.15.39:0/2675757359 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35700d0620 0x7f35701e92d0 secure :-1 s=READY pgs=2722 cs=0 l=1 rev1=1 crypto rx=0x7f355802aaa0 tx=0x7f3558003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:17.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.131+0000 7f3567e00640 1 -- 172.21.15.39:0/2675757359 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3558005c90 con 0x7f35700d0620 2024-09-21T07:54:17.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.131+0000 7f3567e00640 1 -- 172.21.15.39:0/2675757359 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3558005020 con 0x7f35700d0620 2024-09-21T07:54:17.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.131+0000 7f3576800640 1 -- 172.21.15.39:0/2675757359 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3570155410 con 0x7f35700d0620 2024-09-21T07:54:17.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.131+0000 7f3567e00640 1 -- 172.21.15.39:0/2675757359 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3558005450 con 0x7f35700d0620 2024-09-21T07:54:17.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.132+0000 7f3576800640 1 -- 172.21.15.39:0/2675757359 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35701559d0 con 0x7f35700d0620 2024-09-21T07:54:17.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.133+0000 7f3576800640 1 -- 172.21.15.39:0/2675757359 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3570074080 con 0x7f35700d0620 2024-09-21T07:54:17.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.134+0000 7f3567e00640 1 -- 172.21.15.39:0/2675757359 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f35580317c0 con 0x7f35700d0620 2024-09-21T07:54:17.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.135+0000 7f3567e00640 1 --2- 172.21.15.39:0/2675757359 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f354c078b90 0x7f354c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:17.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.135+0000 7f3567e00640 1 -- 172.21.15.39:0/2675757359 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(253..253 src has 1..253) ==== 8035+0+0 (secure 0 0 0) 0x7f35580bc560 con 0x7f35700d0620 2024-09-21T07:54:17.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.136+0000 7f356f400640 1 --2- 172.21.15.39:0/2675757359 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f354c078b90 0x7f354c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:17.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.136+0000 7f356f400640 1 --2- 172.21.15.39:0/2675757359 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f354c078b90 0x7f354c07b0a0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f3560000f60 tx=0x7f3560009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:17.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.140+0000 7f3567e00640 1 -- 172.21.15.39:0/2675757359 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f355803f030 con 0x7f35700d0620 2024-09-21T07:54:17.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.348+0000 7f3576800640 1 -- 172.21.15.39:0/2675757359 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3570151930 con 0x7f35700d0620 2024-09-21T07:54:17.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.348+0000 7f3567e00640 1 -- 172.21.15.39:0/2675757359 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 21 entries v253) ==== 81+0+1274 (secure 0 0 0) 0x7f3558083ac0 con 0x7f35700d0620 2024-09-21T07:54:17.351 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/249900061 2024-09-21T08:54:08.362327+0000 2024-09-21T07:54:17.351 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6835/2191156088 2024-09-22T07:53:58.762864+0000 2024-09-21T07:54:17.351 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/485194245 2024-09-22T07:53:58.201516+0000 2024-09-21T07:54:17.351 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6833/485194245 2024-09-22T07:53:58.201516+0000 2024-09-21T07:54:17.351 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/4061644382 2024-09-21T08:53:16.749620+0000 2024-09-21T07:54:17.351 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/597301351 2024-09-22T07:53:09.120490+0000 2024-09-21T07:54:17.351 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6834/2191156088 2024-09-22T07:53:58.762864+0000 2024-09-21T07:54:17.351 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6835/597301351 2024-09-22T07:53:09.120490+0000 2024-09-21T07:54:17.351 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/3381818843 2024-09-22T07:53:08.565426+0000 2024-09-21T07:54:17.351 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6834/3381818843 2024-09-22T07:53:08.565426+0000 2024-09-21T07:54:17.351 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/106826898 2024-09-21T08:52:26.149462+0000 2024-09-21T07:54:17.352 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/109229385 2024-09-22T07:51:36.504781+0000 2024-09-21T07:54:17.352 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/109229385 2024-09-22T07:51:36.504781+0000 2024-09-21T07:54:17.352 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/1438851059 2024-09-22T07:51:00.902180+0000 2024-09-21T07:54:17.352 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6835/1438851059 2024-09-22T07:51:00.902180+0000 2024-09-21T07:54:17.352 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/2824493631 2024-09-22T07:50:31.817924+0000 2024-09-21T07:54:17.352 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6833/2824493631 2024-09-22T07:50:31.817924+0000 2024-09-21T07:54:17.352 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/4143807732 2024-09-22T07:50:32.369206+0000 2024-09-21T07:54:17.352 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/4143807732 2024-09-22T07:50:32.369206+0000 2024-09-21T07:54:17.352 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/3497801035 2024-09-22T07:50:24.850152+0000 2024-09-21T07:54:17.352 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6834/3497801035 2024-09-22T07:50:24.850152+0000 2024-09-21T07:54:17.353 INFO:teuthology.orchestra.run.smithi039.stderr:listed 21 entries 2024-09-21T07:54:17.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.351+0000 7f3576800640 1 -- 172.21.15.39:0/2675757359 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f354c078b90 msgr2=0x7f354c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:17.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.351+0000 7f3576800640 1 --2- 172.21.15.39:0/2675757359 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f354c078b90 0x7f354c07b0a0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f3560000f60 tx=0x7f3560009040 comp rx=0 tx=0).stop 2024-09-21T07:54:17.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.351+0000 7f3576800640 1 -- 172.21.15.39:0/2675757359 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35700d0620 msgr2=0x7f35701e92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:17.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.351+0000 7f3576800640 1 --2- 172.21.15.39:0/2675757359 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35700d0620 0x7f35701e92d0 secure :-1 s=READY pgs=2722 cs=0 l=1 rev1=1 crypto rx=0x7f355802aaa0 tx=0x7f3558003c10 comp rx=0 tx=0).stop 2024-09-21T07:54:17.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.351+0000 7f3576800640 1 -- 172.21.15.39:0/2675757359 shutdown_connections 2024-09-21T07:54:17.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.351+0000 7f3576800640 1 --2- 172.21.15.39:0/2675757359 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f354c078b90 0x7f354c07b0a0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:17.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.351+0000 7f3576800640 1 --2- 172.21.15.39:0/2675757359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f357015cd30 0x7f35701e9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:17.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.351+0000 7f3576800640 1 --2- 172.21.15.39:0/2675757359 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35700d0620 0x7f35701e92d0 unknown :-1 s=CLOSED pgs=2722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:17.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.351+0000 7f3576800640 1 --2- 172.21.15.39:0/2675757359 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f35700cfd10 0x7f357014d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:17.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.351+0000 7f3576800640 1 -- 172.21.15.39:0/2675757359 >> 172.21.15.39:0/2675757359 conn(0x7f357006cfc0 msgr2=0x7f357006f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:17.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.351+0000 7f3576800640 1 -- 172.21.15.39:0/2675757359 shutdown_connections 2024-09-21T07:54:17.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.351+0000 7f3576800640 1 -- 172.21.15.39:0/2675757359 wait complete. 2024-09-21T07:54:17.379 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T07:54:17.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:17.379 DEBUG:teuthology.orchestra.run.smithi039:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T07:54:17.437 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T07:54:17.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:17.437 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:54:17.473 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:54:17.474 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:17.549 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:54:17.549 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:54:17.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.815+0000 7fe3da400640 1 -- 172.21.15.39:0/2642001556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3d40cb960 msgr2=0x7fe3d40cbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:17.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.815+0000 7fe3da400640 1 --2- 172.21.15.39:0/2642001556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3d40cb960 0x7fe3d40cbd70 secure :-1 s=READY pgs=2724 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0009830 tx=0x7fe3c002a510 comp rx=0 tx=0).stop 2024-09-21T07:54:17.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.815+0000 7fe3da400640 1 -- 172.21.15.39:0/2642001556 shutdown_connections 2024-09-21T07:54:17.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.815+0000 7fe3da400640 1 --2- 172.21.15.39:0/2642001556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3d40cb960 0x7fe3d40cbd70 unknown :-1 s=CLOSED pgs=2724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:17.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.815+0000 7fe3da400640 1 --2- 172.21.15.39:0/2642001556 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3d40d0f20 0x7fe3d415c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:17.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.815+0000 7fe3da400640 1 -- 172.21.15.39:0/2642001556 >> 172.21.15.39:0/2642001556 conn(0x7fe3d40b2510 msgr2=0x7fe3d40d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:17.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.815+0000 7fe3da400640 1 -- 172.21.15.39:0/2642001556 shutdown_connections 2024-09-21T07:54:17.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.815+0000 7fe3da400640 1 -- 172.21.15.39:0/2642001556 wait complete. 2024-09-21T07:54:17.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.817+0000 7fe3da400640 1 Processor -- start 2024-09-21T07:54:17.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.817+0000 7fe3da400640 1 -- start start 2024-09-21T07:54:17.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.819+0000 7fe3da400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3d406f8a0 0x7fe3d4150340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:17.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.819+0000 7fe3da400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3d40cb960 0x7fe3d409d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:17.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.819+0000 7fe3da400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3d40d0f20 0x7fe3d409b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:17.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.819+0000 7fe3da400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d40b4570 con 0x7fe3d40d0f20 2024-09-21T07:54:17.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.819+0000 7fe3da400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d40b43f0 con 0x7fe3d406f8a0 2024-09-21T07:54:17.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.819+0000 7fe3da400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe3d40b46f0 con 0x7fe3d40cb960 2024-09-21T07:54:17.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.819+0000 7fe3d9a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3d40d0f20 0x7fe3d409b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:17.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.819+0000 7fe3d9a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3d40d0f20 0x7fe3d409b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50426/0 (socket says 172.21.15.39:50426) 2024-09-21T07:54:17.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.819+0000 7fe3d9a00640 1 -- 172.21.15.39:0/918107648 learned_addr learned my addr 172.21.15.39:0/918107648 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:17.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.819+0000 7fe3d3e00640 1 --2- 172.21.15.39:0/918107648 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3d40cb960 0x7fe3d409d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:17.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.819+0000 7fe3d9000640 1 --2- 172.21.15.39:0/918107648 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3d406f8a0 0x7fe3d4150340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:17.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.820+0000 7fe3d9a00640 1 -- 172.21.15.39:0/918107648 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3d40cb960 msgr2=0x7fe3d409d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:17.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.820+0000 7fe3d9a00640 1 --2- 172.21.15.39:0/918107648 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3d40cb960 0x7fe3d409d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:17.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.820+0000 7fe3d9a00640 1 -- 172.21.15.39:0/918107648 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3d406f8a0 msgr2=0x7fe3d4150340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:17.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.820+0000 7fe3d9a00640 1 --2- 172.21.15.39:0/918107648 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3d406f8a0 0x7fe3d4150340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:17.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.820+0000 7fe3d9a00640 1 -- 172.21.15.39:0/918107648 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d41f32d0 con 0x7fe3d40d0f20 2024-09-21T07:54:17.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.820+0000 7fe3d9a00640 1 --2- 172.21.15.39:0/918107648 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3d40d0f20 0x7fe3d409b980 secure :-1 s=READY pgs=2725 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc00c730 tx=0x7fe3cc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:17.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.820+0000 7fe3d0c00640 1 -- 172.21.15.39:0/918107648 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc003b70 con 0x7fe3d40d0f20 2024-09-21T07:54:17.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.820+0000 7fe3d0c00640 1 -- 172.21.15.39:0/918107648 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3cc003d30 con 0x7fe3d40d0f20 2024-09-21T07:54:17.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.820+0000 7fe3da400640 1 -- 172.21.15.39:0/918107648 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3d41f6830 con 0x7fe3d40d0f20 2024-09-21T07:54:17.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.820+0000 7fe3d0c00640 1 -- 172.21.15.39:0/918107648 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc0052d0 con 0x7fe3d40d0f20 2024-09-21T07:54:17.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.820+0000 7fe3da400640 1 -- 172.21.15.39:0/918107648 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3d41f5cb0 con 0x7fe3d40d0f20 2024-09-21T07:54:17.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.821+0000 7fe3da400640 1 -- 172.21.15.39:0/918107648 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3d41f5a10 con 0x7fe3d40d0f20 2024-09-21T07:54:17.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.823+0000 7fe3d0c00640 1 -- 172.21.15.39:0/918107648 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe3cc005470 con 0x7fe3d40d0f20 2024-09-21T07:54:17.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.824+0000 7fe3d0c00640 1 --2- 172.21.15.39:0/918107648 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe3b4078b90 0x7fe3b407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:17.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.824+0000 7fe3d0c00640 1 -- 172.21.15.39:0/918107648 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(253..253 src has 1..253) ==== 8035+0+0 (secure 0 0 0) 0x7fe3cc09b430 con 0x7fe3d40d0f20 2024-09-21T07:54:17.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.824+0000 7fe3d9000640 1 --2- 172.21.15.39:0/918107648 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe3b4078b90 0x7fe3b407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:17.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.825+0000 7fe3d9000640 1 --2- 172.21.15.39:0/918107648 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe3b4078b90 0x7fe3b407b0a0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800bb00 tx=0x7fe3c8005e30 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:17.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:17.828+0000 7fe3d0c00640 1 -- 172.21.15.39:0/918107648 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe3d41f5a10 con 0x7fe3d40d0f20 2024-09-21T07:54:18.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:18.062+0000 7fe3da400640 1 -- 172.21.15.39:0/918107648 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe3d40b4c50 con 0x7fe3d40d0f20 2024-09-21T07:54:18.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:18.062+0000 7fe3d0c00640 1 -- 172.21.15.39:0/918107648 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v301) ==== 53+0+73 (secure 0 0 0) 0x7fe3cc062fd0 con 0x7fe3d40d0f20 2024-09-21T07:54:18.066 INFO:teuthology.orchestra.run.smithi039.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:54:18.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:18.065+0000 7fe3da400640 1 -- 172.21.15.39:0/918107648 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe3b4078b90 msgr2=0x7fe3b407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:18.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:18.065+0000 7fe3da400640 1 --2- 172.21.15.39:0/918107648 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe3b4078b90 0x7fe3b407b0a0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800bb00 tx=0x7fe3c8005e30 comp rx=0 tx=0).stop 2024-09-21T07:54:18.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:18.065+0000 7fe3da400640 1 -- 172.21.15.39:0/918107648 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3d40d0f20 msgr2=0x7fe3d409b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:18.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:18.065+0000 7fe3da400640 1 --2- 172.21.15.39:0/918107648 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3d40d0f20 0x7fe3d409b980 secure :-1 s=READY pgs=2725 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc00c730 tx=0x7fe3cc00cbf0 comp rx=0 tx=0).stop 2024-09-21T07:54:18.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:18.065+0000 7fe3da400640 1 -- 172.21.15.39:0/918107648 shutdown_connections 2024-09-21T07:54:18.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:18.065+0000 7fe3da400640 1 --2- 172.21.15.39:0/918107648 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe3b4078b90 0x7fe3b407b0a0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:18.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:18.065+0000 7fe3da400640 1 --2- 172.21.15.39:0/918107648 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3d40d0f20 0x7fe3d409b980 unknown :-1 s=CLOSED pgs=2725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:18.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:18.065+0000 7fe3da400640 1 --2- 172.21.15.39:0/918107648 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3d40cb960 0x7fe3d409d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:18.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:18.065+0000 7fe3da400640 1 --2- 172.21.15.39:0/918107648 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3d406f8a0 0x7fe3d4150340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:18.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:18.065+0000 7fe3da400640 1 -- 172.21.15.39:0/918107648 >> 172.21.15.39:0/918107648 conn(0x7fe3d40b2510 msgr2=0x7fe3d415e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:18.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:18.065+0000 7fe3da400640 1 -- 172.21.15.39:0/918107648 shutdown_connections 2024-09-21T07:54:18.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:18.065+0000 7fe3da400640 1 -- 172.21.15.39:0/918107648 wait complete. 2024-09-21T07:54:18.093 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:54:18.093 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:54:18.093 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi039.front.sepia.ceph.com 2024-09-21T07:54:18.093 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T07:54:18.093 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:54:18.093 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:54:18.093 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:54:18.093 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:54:18.093 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:54:18.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:18.093 DEBUG:teuthology.orchestra.run.smithi039:> ip addr 2024-09-21T07:54:18.129 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:54:18.130 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:54:18.130 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:54:18.130 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:18.130 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2024-09-21T07:54:18.130 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:18.130 INFO:teuthology.orchestra.run.smithi039.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:54:18.130 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:54:18.130 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f0 2024-09-21T07:54:18.130 INFO:teuthology.orchestra.run.smithi039.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:54:18.130 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c7 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:54:18.130 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f1 2024-09-21T07:54:18.130 INFO:teuthology.orchestra.run.smithi039.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:54:18.130 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:70 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:54:18.131 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f0 2024-09-21T07:54:18.131 INFO:teuthology.orchestra.run.smithi039.stdout: inet 172.21.15.39/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T07:54:18.131 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft 41461sec preferred_lft 41461sec 2024-09-21T07:54:18.131 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec4:7aff:febd:1370/64 scope link 2024-09-21T07:54:18.131 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:18.131 INFO:teuthology.orchestra.run.smithi039.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:54:18.131 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:71 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:54:18.131 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f1 2024-09-21T07:54:18.131 INFO:teuthology.orchestra.run.smithi039.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:54:18.131 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff 2024-09-21T07:54:18.131 INFO:teuthology.orchestra.run.smithi039.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:54:18.131 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:18.131 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::b079:ddff:fe0e:6333/64 scope link 2024-09-21T07:54:18.131 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:18.131 INFO:teuthology.orchestra.run.smithi039.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:54:18.132 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:54:18.132 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::2ca7:98ff:fe8a:d0ed/64 scope link tentative 2024-09-21T07:54:18.132 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:18.132 INFO:teuthology.orchestra.run.smithi039.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:54:18.132 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether ee:10:53:01:0c:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:54:18.132 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec10:53ff:fe01:c44/64 scope link 2024-09-21T07:54:18.132 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:18.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:18.132 DEBUG:teuthology.orchestra.run.smithi039:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:54:18.223 INFO:teuthology.orchestra.run.smithi039.stdout:1 2024-09-21T07:54:18.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:18.226 DEBUG:teuthology.orchestra.run.smithi039:> route 2024-09-21T07:54:18.263 INFO:teuthology.orchestra.run.smithi039.stdout:Kernel IP routing table 2024-09-21T07:54:18.263 INFO:teuthology.orchestra.run.smithi039.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:54:18.263 INFO:teuthology.orchestra.run.smithi039.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T07:54:18.263 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T07:54:18.263 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:54:18.263 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:54:18.264 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-21T07:54:18.264 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T07:54:18.264 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T07:54:18.264 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T07:54:18.264 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-21T07:54:18.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:54:18.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:54:18.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:18.461 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-09-21T07:54:18.495 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:54:18.495 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T07:54:18.495 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:54:18.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:18.496 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-09-21T07:54:18.591 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:54:18.591 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T07:54:18.626 INFO:teuthology.orchestra.run.smithi039.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:54:18.626 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:54:18.626 DEBUG:teuthology.orchestra.run.smithi039:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:54:18.701 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-21T07:54:18.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:18.702 DEBUG:teuthology.orchestra.run.smithi039:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:54:18.901 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:54:18.901 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:54:18.936 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:54:18.936 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:54:18.936 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: resolved to: "172.21.15.39:3300,172.21.15.148:3300,172.21.15.148:3301" 2024-09-21T07:54:18.937 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: trying mount with new device syntax: 0@7ab9b04a-f88d-49aa-a493-e5c076637ffb.cephfs=/ 2024-09-21T07:54:18.937 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.39:3300/172.21.15.148:3300/172.21.15.148:3301" will pass to kernel 2024-09-21T07:54:18.938 INFO:teuthology.orchestra.run.smithi039.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T07:54:18.938 INFO:teuthology.orchestra.run.smithi039.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:54:18.938 INFO:teuthology.orchestra.run.smithi039.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:54:18.938 INFO:teuthology.orchestra.run.smithi039.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:54:18.938 INFO:teuthology.orchestra.run.smithi039.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:54:18.941 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:54:18.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:18.941 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:54:18.996 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:54:19.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:54:19.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:54:19.091 INFO:teuthology.orchestra.run.smithi039.stdout:client9569 2024-09-21T07:54:19.092 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:54:19.092 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:19.127 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.13:0 flush journal 2024-09-21T07:54:19.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.395+0000 7f46d0e00640 1 -- 172.21.15.39:0/1404552926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46cc0cce70 msgr2=0x7f46cc0cd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:19.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.395+0000 7f46d0e00640 1 --2- 172.21.15.39:0/1404552926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46cc0cce70 0x7f46cc0cd280 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f46c4005fd0 tx=0x7f46c402aa10 comp rx=0 tx=0).stop 2024-09-21T07:54:19.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.395+0000 7f46d0e00640 1 -- 172.21.15.39:0/1404552926 shutdown_connections 2024-09-21T07:54:19.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.395+0000 7f46d0e00640 1 --2- 172.21.15.39:0/1404552926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46cc0cce70 0x7f46cc0cd280 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.395+0000 7f46d0e00640 1 --2- 172.21.15.39:0/1404552926 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46cc0b32b0 0x7f46cc0cbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.395+0000 7f46d0e00640 1 -- 172.21.15.39:0/1404552926 >> 172.21.15.39:0/1404552926 conn(0x7f46cc06d440 msgr2=0x7f46cc0b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:19.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.396+0000 7f46d0e00640 1 -- 172.21.15.39:0/1404552926 shutdown_connections 2024-09-21T07:54:19.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.396+0000 7f46d0e00640 1 -- 172.21.15.39:0/1404552926 wait complete. 2024-09-21T07:54:19.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.398+0000 7f46d0e00640 1 Processor -- start 2024-09-21T07:54:19.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.398+0000 7f46d0e00640 1 -- start start 2024-09-21T07:54:19.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.399+0000 7f46d0e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46cc0b32b0 0x7f46cc0c9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:19.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.400+0000 7f46d0e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46cc0cce70 0x7f46cc0ce9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:19.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.400+0000 7f46d0e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46cc15cf60 0x7f46cc0b0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:19.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.400+0000 7f46d0e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc06f9e0 con 0x7f46cc15cf60 2024-09-21T07:54:19.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.400+0000 7f46d0e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc06f860 con 0x7f46cc0cce70 2024-09-21T07:54:19.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.400+0000 7f46d0e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f46cc06fb60 con 0x7f46cc0b32b0 2024-09-21T07:54:19.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.400+0000 7f46ca800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46cc15cf60 0x7f46cc0b0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:19.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.400+0000 7f46c3e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46cc0cce70 0x7f46cc0ce9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:19.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.400+0000 7f46ca800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46cc15cf60 0x7f46cc0b0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50428/0 (socket says 172.21.15.39:50428) 2024-09-21T07:54:19.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.400+0000 7f46ca800640 1 -- 172.21.15.39:0/1409983495 learned_addr learned my addr 172.21.15.39:0/1409983495 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:19.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.400+0000 7f46c9e00640 1 --2- 172.21.15.39:0/1409983495 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46cc0b32b0 0x7f46cc0c9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:19.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.400+0000 7f46ca800640 1 -- 172.21.15.39:0/1409983495 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46cc0b32b0 msgr2=0x7f46cc0c9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:19.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.400+0000 7f46ca800640 1 --2- 172.21.15.39:0/1409983495 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46cc0b32b0 0x7f46cc0c9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.400+0000 7f46ca800640 1 -- 172.21.15.39:0/1409983495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46cc0cce70 msgr2=0x7f46cc0ce9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:19.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.400+0000 7f46ca800640 1 --2- 172.21.15.39:0/1409983495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46cc0cce70 0x7f46cc0ce9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.400+0000 7f46ca800640 1 -- 172.21.15.39:0/1409983495 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46cc158fc0 con 0x7f46cc15cf60 2024-09-21T07:54:19.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.401+0000 7f46ca800640 1 --2- 172.21.15.39:0/1409983495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46cc15cf60 0x7f46cc0b0e30 secure :-1 s=READY pgs=2726 cs=0 l=1 rev1=1 crypto rx=0x7f46c4006010 tx=0x7f46c4003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:19.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.401+0000 7f46c2000640 1 -- 172.21.15.39:0/1409983495 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c4046070 con 0x7f46cc15cf60 2024-09-21T07:54:19.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.401+0000 7f46c2000640 1 -- 172.21.15.39:0/1409983495 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46c4004540 con 0x7f46cc15cf60 2024-09-21T07:54:19.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.401+0000 7f46c2000640 1 -- 172.21.15.39:0/1409983495 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c4033070 con 0x7f46cc15cf60 2024-09-21T07:54:19.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.401+0000 7f46d0e00640 1 -- 172.21.15.39:0/1409983495 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46cc1592a0 con 0x7f46cc15cf60 2024-09-21T07:54:19.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.401+0000 7f46d0e00640 1 -- 172.21.15.39:0/1409983495 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46cc159860 con 0x7f46cc15cf60 2024-09-21T07:54:19.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.404+0000 7f46c2000640 1 -- 172.21.15.39:0/1409983495 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f46c4049070 con 0x7f46cc15cf60 2024-09-21T07:54:19.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.405+0000 7f46c2000640 1 --2- 172.21.15.39:0/1409983495 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f46a8078b90 0x7f46a807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:19.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.406+0000 7f46c2000640 1 -- 172.21.15.39:0/1409983495 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(253..253 src has 1..253) ==== 8035+0+0 (secure 0 0 0) 0x7f46c40bc730 con 0x7f46cc15cf60 2024-09-21T07:54:19.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.406+0000 7f46c9e00640 1 --2- 172.21.15.39:0/1409983495 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f46a8078b90 0x7f46a807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:19.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.406+0000 7f46d7dcf740 1 Processor -- start 2024-09-21T07:54:19.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.406+0000 7f46d7dcf740 1 -- start start 2024-09-21T07:54:19.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.407+0000 7f46c9e00640 1 --2- 172.21.15.39:0/1409983495 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f46a8078b90 0x7f46a807b0a0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f46bc000f60 tx=0x7f46bc009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:19.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.407+0000 7f46d7dcf740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5618229e5390 0x561822dd3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:19.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.407+0000 7f46d7dcf740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618229bc4d0 0x561822db6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:19.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.408+0000 7f46d7dcf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5618229c46b0 0x561822db6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:19.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.408+0000 7f46d7dcf740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561822cec1c0 con 0x5618229bc4d0 2024-09-21T07:54:19.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.408+0000 7f46d7dcf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561822cff870 con 0x5618229c46b0 2024-09-21T07:54:19.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.408+0000 7f46d7dcf740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x561822ceeca0 con 0x5618229e5390 2024-09-21T07:54:19.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.408+0000 7f46c3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618229bc4d0 0x561822db6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:19.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.408+0000 7f46c3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618229bc4d0 0x561822db6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50442/0 (socket says 172.21.15.39:50442) 2024-09-21T07:54:19.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.408+0000 7f46c3e00640 1 -- 172.21.15.39:0/2739682747 learned_addr learned my addr 172.21.15.39:0/2739682747 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:19.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.408+0000 7f46c9e00640 1 --2- 172.21.15.39:0/2739682747 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5618229e5390 0x561822dd3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:19.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.408+0000 7f46ca800640 1 --2- 172.21.15.39:0/2739682747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5618229c46b0 0x561822db6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:19.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.408+0000 7f46c3e00640 1 -- 172.21.15.39:0/2739682747 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5618229e5390 msgr2=0x561822dd3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:19.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.408+0000 7f46c3e00640 1 --2- 172.21.15.39:0/2739682747 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5618229e5390 0x561822dd3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.408+0000 7f46c3e00640 1 -- 172.21.15.39:0/2739682747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5618229c46b0 msgr2=0x561822db6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:19.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.408+0000 7f46c3e00640 1 --2- 172.21.15.39:0/2739682747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5618229c46b0 0x561822db6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.408+0000 7f46c3e00640 1 -- 172.21.15.39:0/2739682747 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561822ac77d0 con 0x5618229bc4d0 2024-09-21T07:54:19.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.408+0000 7f46c9e00640 1 --2- 172.21.15.39:0/2739682747 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5618229e5390 0x561822dd3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:19.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.408+0000 7f46ca800640 1 --2- 172.21.15.39:0/2739682747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5618229c46b0 0x561822db6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:19.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.409+0000 7f46c3e00640 1 --2- 172.21.15.39:0/2739682747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618229bc4d0 0x561822db6220 secure :-1 s=READY pgs=2727 cs=0 l=1 rev1=1 crypto rx=0x7f46b4009c20 tx=0x7f46b4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=919ac421c91c78ba server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:19.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.409+0000 7f46baa00640 1 -- 172.21.15.39:0/2739682747 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b4004250 con 0x5618229bc4d0 2024-09-21T07:54:19.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.409+0000 7f46baa00640 1 -- 172.21.15.39:0/2739682747 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46b40043f0 con 0x5618229bc4d0 2024-09-21T07:54:19.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.409+0000 7f46baa00640 1 -- 172.21.15.39:0/2739682747 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b4003850 con 0x5618229bc4d0 2024-09-21T07:54:19.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.410+0000 7f46d7dcf740 1 -- 172.21.15.39:0/2739682747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618229bc4d0 msgr2=0x561822db6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:19.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.410+0000 7f46d7dcf740 1 --2- 172.21.15.39:0/2739682747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618229bc4d0 0x561822db6220 secure :-1 s=READY pgs=2727 cs=0 l=1 rev1=1 crypto rx=0x7f46b4009c20 tx=0x7f46b4011810 comp rx=0 tx=0).stop 2024-09-21T07:54:19.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.410+0000 7f46d7dcf740 1 -- 172.21.15.39:0/2739682747 shutdown_connections 2024-09-21T07:54:19.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.410+0000 7f46d7dcf740 1 --2- 172.21.15.39:0/2739682747 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5618229c46b0 0x561822db6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.410+0000 7f46d7dcf740 1 --2- 172.21.15.39:0/2739682747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618229bc4d0 0x561822db6220 unknown :-1 s=CLOSED pgs=2727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.410+0000 7f46d7dcf740 1 --2- 172.21.15.39:0/2739682747 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5618229e5390 0x561822dd3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.410+0000 7f46d7dcf740 1 -- 172.21.15.39:0/2739682747 >> 172.21.15.39:0/2739682747 conn(0x5618229d94e0 msgr2=0x561822d8c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:19.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.410+0000 7f46d7dcf740 1 -- 172.21.15.39:0/2739682747 shutdown_connections 2024-09-21T07:54:19.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.410+0000 7f46d7dcf740 1 -- 172.21.15.39:0/2739682747 wait complete. 2024-09-21T07:54:19.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.411+0000 7f46d7dcf740 1 -- start start 2024-09-21T07:54:19.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.411+0000 7f46d7dcf740 1 Processor -- start 2024-09-21T07:54:19.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.413+0000 7f46d7dcf740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618229bc4d0 0x561822dd1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:19.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.413+0000 7f46d7dcf740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5618229c46b0 0x561822d8c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:19.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.413+0000 7f46d7dcf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5618229e5390 0x561822de3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:19.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.413+0000 7f46d7dcf740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561822c95410 con 0x5618229bc4d0 2024-09-21T07:54:19.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.413+0000 7f46d7dcf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561822cf2ab0 con 0x5618229e5390 2024-09-21T07:54:19.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.413+0000 7f46d7dcf740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x561822a5f740 con 0x5618229c46b0 2024-09-21T07:54:19.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.413+0000 7f46d7dcf740 10 client.0 _ll_register_callbacks cb 0x561822c72360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:54:19.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.413+0000 7f46c9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618229bc4d0 0x561822dd1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:19.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.413+0000 7f46c9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618229bc4d0 0x561822dd1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50456/0 (socket says 172.21.15.39:50456) 2024-09-21T07:54:19.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.413+0000 7f46c9e00640 1 -- 172.21.15.39:0/1582024179 learned_addr learned my addr 172.21.15.39:0/1582024179 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:19.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.413+0000 7f46ca800640 1 --2- 172.21.15.39:0/1582024179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5618229e5390 0x561822de3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:19.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.413+0000 7f46c3e00640 1 --2- 172.21.15.39:0/1582024179 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5618229c46b0 0x561822d8c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:19.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.413+0000 7f46c9e00640 1 -- 172.21.15.39:0/1582024179 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5618229c46b0 msgr2=0x561822d8c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:19.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.413+0000 7f46c9e00640 1 --2- 172.21.15.39:0/1582024179 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5618229c46b0 0x561822d8c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.413+0000 7f46c9e00640 1 -- 172.21.15.39:0/1582024179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5618229e5390 msgr2=0x561822de3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:19.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.413+0000 7f46c9e00640 1 --2- 172.21.15.39:0/1582024179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5618229e5390 0x561822de3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.413+0000 7f46c9e00640 1 -- 172.21.15.39:0/1582024179 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561822a50970 con 0x5618229bc4d0 2024-09-21T07:54:19.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.414+0000 7f46c9e00640 1 --2- 172.21.15.39:0/1582024179 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618229bc4d0 0x561822dd1370 secure :-1 s=READY pgs=2728 cs=0 l=1 rev1=1 crypto rx=0x7f46bc0146d0 tx=0x7f46bc001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d478a23925fc0b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:19.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.414+0000 7f46af400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:54:19.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.414+0000 7f46af400640 1 -- 172.21.15.39:0/1582024179 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc013680 con 0x5618229bc4d0 2024-09-21T07:54:19.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.414+0000 7f46d7dcf740 1 -- 172.21.15.39:0/1582024179 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46b40043f0 con 0x5618229bc4d0 2024-09-21T07:54:19.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.414+0000 7f46af400640 1 -- 172.21.15.39:0/1582024179 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(253..253 src has 1..253) ==== 8035+0+0 (secure 0 0 0) 0x7f46bc013820 con 0x5618229bc4d0 2024-09-21T07:54:19.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.414+0000 7f46af400640 1 -- 172.21.15.39:0/1582024179 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=301) ==== 24+0+0 (secure 0 0 0) 0x7f46bc013a20 con 0x5618229bc4d0 2024-09-21T07:54:19.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.414+0000 7f46d7dcf740 10 client.12138 fetch_fsmap learned FSMap version 301 2024-09-21T07:54:19.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.414+0000 7f46d7dcf740 1 -- 172.21.15.39:0/1582024179 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=301,monmap=2+}) -- 0x561822a2e6a0 con 0x5618229bc4d0 2024-09-21T07:54:19.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.415+0000 7f46af400640 1 -- 172.21.15.39:0/1582024179 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46b40043f0 con 0x5618229bc4d0 2024-09-21T07:54:19.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.415+0000 7f46af400640 1 -- 172.21.15.39:0/1582024179 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 301) ==== 3542+0+0 (secure 0 0 0) 0x7f46bc020070 con 0x5618229bc4d0 2024-09-21T07:54:19.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.415+0000 7f46d7dcf740 10 client.12138 fetch_fsmap finished waiting for FSMap version 301 2024-09-21T07:54:19.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.415+0000 7f46d7dcf740 10 client.12138 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.a 2024-09-21T07:54:19.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.415+0000 7f46d7dcf740 20 client.12138 populate_metadata read hostname 'smithi039' 2024-09-21T07:54:19.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.415+0000 7f46d7dcf740 1 --2- 172.21.15.39:0/1582024179 >> [v2:172.21.15.39:6832/2161818466,v1:172.21.15.39:6834/2161818466] conn(0x5618229a3f60 0x561822da4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:19.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.415+0000 7f46d7dcf740 4 client.12138 mds_command: new command op to 11364 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:54:19.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.415+0000 7f46d7dcf740 1 -- 172.21.15.39:0/1582024179 --> [v2:172.21.15.39:6832/2161818466,v1:172.21.15.39:6834/2161818466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561822d017c0 con 0x5618229a3f60 2024-09-21T07:54:19.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.415+0000 7f46c3e00640 1 --2- 172.21.15.39:0/1582024179 >> [v2:172.21.15.39:6832/2161818466,v1:172.21.15.39:6834/2161818466] conn(0x5618229a3f60 0x561822da4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:19.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.416+0000 7f46c3e00640 1 --2- 172.21.15.39:0/1582024179 >> [v2:172.21.15.39:6832/2161818466,v1:172.21.15.39:6834/2161818466] conn(0x5618229a3f60 0x561822da4040 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9eb3d34fe51f7375 server_cookie=f31e46f54513eefb in_seq=0 out_seq=0 2024-09-21T07:54:19.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.416+0000 7f46af400640 10 client.12138 ms_handle_connect on v2:172.21.15.39:6832/2161818466 2024-09-21T07:54:19.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.416+0000 7f46af400640 1 -- 172.21.15.39:0/1582024179 <== mds.0 v2:172.21.15.39:6832/2161818466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561822d017c0 con 0x5618229a3f60 2024-09-21T07:54:19.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.416+0000 7f46af400640 10 client.12138 handle_command_reply: tid=0 2024-09-21T07:54:19.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.417+0000 7f46d7dcf740 1 client.12138 shutdown 2024-09-21T07:54:19.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.417+0000 7f46d7dcf740 1 -- 172.21.15.39:0/1582024179 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618229bc4d0 msgr2=0x561822dd1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:19.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.417+0000 7f46d7dcf740 1 --2- 172.21.15.39:0/1582024179 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618229bc4d0 0x561822dd1370 secure :-1 s=READY pgs=2728 cs=0 l=1 rev1=1 crypto rx=0x7f46bc0146d0 tx=0x7f46bc001080 comp rx=0 tx=0).stop 2024-09-21T07:54:19.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.417+0000 7f46d7dcf740 1 -- 172.21.15.39:0/1582024179 shutdown_connections 2024-09-21T07:54:19.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.417+0000 7f46d7dcf740 1 --2- 172.21.15.39:0/1582024179 >> [v2:172.21.15.39:6832/2161818466,v1:172.21.15.39:6834/2161818466] conn(0x5618229a3f60 0x561822da4040 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.417+0000 7f46d7dcf740 1 --2- 172.21.15.39:0/1582024179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5618229e5390 0x561822de3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.417+0000 7f46d7dcf740 1 --2- 172.21.15.39:0/1582024179 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5618229c46b0 0x561822d8c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.417+0000 7f46d7dcf740 1 --2- 172.21.15.39:0/1582024179 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618229bc4d0 0x561822dd1370 unknown :-1 s=CLOSED pgs=2728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.417+0000 7f46af400640 0 client.12138 ms_handle_reset on v2:172.21.15.39:6832/2161818466 2024-09-21T07:54:19.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.417+0000 7f46d7dcf740 1 -- 172.21.15.39:0/1582024179 >> 172.21.15.39:0/1582024179 conn(0x5618229d94e0 msgr2=0x561822da59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:19.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.417+0000 7f46d7dcf740 1 -- 172.21.15.39:0/1582024179 shutdown_connections 2024-09-21T07:54:19.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.417+0000 7f46d7dcf740 1 -- 172.21.15.39:0/1582024179 wait complete. 2024-09-21T07:54:19.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.417+0000 7f46d7dcf740 20 client.12138 trim_cache size 0 max 16384 2024-09-21T07:54:19.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.433+0000 7f46d7dcf740 1 Processor -- start 2024-09-21T07:54:19.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.433+0000 7f46d7dcf740 1 -- start start 2024-09-21T07:54:19.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.434+0000 7f46d7dcf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5618229bc4d0 0x561822d8d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:19.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.434+0000 7f46d7dcf740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5618229c46b0 0x561822dad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:19.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.434+0000 7f46d7dcf740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618229e5390 0x561822dadef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:19.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.434+0000 7f46d7dcf740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc2058a0 con 0x5618229e5390 2024-09-21T07:54:19.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.434+0000 7f46d7dcf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc204b00 con 0x5618229bc4d0 2024-09-21T07:54:19.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.434+0000 7f46d7dcf740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f46cc2051d0 con 0x5618229c46b0 2024-09-21T07:54:19.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.434+0000 7f46ca800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618229e5390 0x561822dadef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:19.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.434+0000 7f46ca800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618229e5390 0x561822dadef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50466/0 (socket says 172.21.15.39:50466) 2024-09-21T07:54:19.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.435+0000 7f46ca800640 1 -- 172.21.15.39:0/1365186365 learned_addr learned my addr 172.21.15.39:0/1365186365 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:19.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.435+0000 7f46c3e00640 1 --2- 172.21.15.39:0/1365186365 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5618229c46b0 0x561822dad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:19.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.435+0000 7f46c9e00640 1 --2- 172.21.15.39:0/1365186365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5618229bc4d0 0x561822d8d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:19.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.435+0000 7f46ca800640 1 -- 172.21.15.39:0/1365186365 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5618229c46b0 msgr2=0x561822dad9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:19.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.435+0000 7f46ca800640 1 --2- 172.21.15.39:0/1365186365 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5618229c46b0 0x561822dad9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.435+0000 7f46ca800640 1 -- 172.21.15.39:0/1365186365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5618229bc4d0 msgr2=0x561822d8d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:19.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.435+0000 7f46ca800640 1 --2- 172.21.15.39:0/1365186365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5618229bc4d0 0x561822d8d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.435+0000 7f46ca800640 1 -- 172.21.15.39:0/1365186365 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561822d88fa0 con 0x5618229e5390 2024-09-21T07:54:19.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.435+0000 7f46c9e00640 1 --2- 172.21.15.39:0/1365186365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5618229bc4d0 0x561822d8d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:19.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.435+0000 7f46ca800640 1 --2- 172.21.15.39:0/1365186365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618229e5390 0x561822dadef0 secure :-1 s=READY pgs=2729 cs=0 l=1 rev1=1 crypto rx=0x7f46c4080dd0 tx=0x7f46c40bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d6d5c2c77c9d763 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:19.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.435+0000 7f46ba000640 1 -- 172.21.15.39:0/1365186365 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c4083b60 con 0x5618229e5390 2024-09-21T07:54:19.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.435+0000 7f46ba000640 1 -- 172.21.15.39:0/1365186365 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46c4088590 con 0x5618229e5390 2024-09-21T07:54:19.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.435+0000 7f46ba000640 1 -- 172.21.15.39:0/1365186365 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c408d4d0 con 0x5618229e5390 2024-09-21T07:54:19.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.436+0000 7f46d7dcf740 1 -- 172.21.15.39:0/1365186365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618229e5390 msgr2=0x561822dadef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:19.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.436+0000 7f46d7dcf740 1 --2- 172.21.15.39:0/1365186365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618229e5390 0x561822dadef0 secure :-1 s=READY pgs=2729 cs=0 l=1 rev1=1 crypto rx=0x7f46c4080dd0 tx=0x7f46c40bc040 comp rx=0 tx=0).stop 2024-09-21T07:54:19.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.436+0000 7f46d7dcf740 1 -- 172.21.15.39:0/1365186365 shutdown_connections 2024-09-21T07:54:19.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.436+0000 7f46d7dcf740 1 --2- 172.21.15.39:0/1365186365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618229e5390 0x561822dadef0 unknown :-1 s=CLOSED pgs=2729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.436+0000 7f46d7dcf740 1 --2- 172.21.15.39:0/1365186365 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5618229c46b0 0x561822dad9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.436+0000 7f46d7dcf740 1 --2- 172.21.15.39:0/1365186365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5618229bc4d0 0x561822d8d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.436+0000 7f46d7dcf740 1 -- 172.21.15.39:0/1365186365 >> 172.21.15.39:0/1365186365 conn(0x5618229d94e0 msgr2=0x561822d8d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:19.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.436+0000 7f46d7dcf740 1 -- 172.21.15.39:0/1365186365 shutdown_connections 2024-09-21T07:54:19.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.436+0000 7f46d7dcf740 1 -- 172.21.15.39:0/1365186365 wait complete. 2024-09-21T07:54:19.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.437+0000 7f46d7dcf740 1 -- start start 2024-09-21T07:54:19.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.437+0000 7f46d7dcf740 1 Processor -- start 2024-09-21T07:54:19.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.439+0000 7f46d7dcf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5618229bc4d0 0x561822e49ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:19.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.439+0000 7f46d7dcf740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5618229c46b0 0x561822de1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:19.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.439+0000 7f46d7dcf740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618229e5390 0x561822d8d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:19.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.439+0000 7f46d7dcf740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561822d8c850 con 0x5618229e5390 2024-09-21T07:54:19.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.439+0000 7f46d7dcf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc203b20 con 0x5618229bc4d0 2024-09-21T07:54:19.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.439+0000 7f46d7dcf740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x561822dcae20 con 0x5618229c46b0 2024-09-21T07:54:19.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.439+0000 7f46d7dcf740 10 client.0 _ll_register_callbacks cb 0x7f46bc017040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:54:19.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.439+0000 7f46ca800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618229e5390 0x561822d8d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:19.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.439+0000 7f46ca800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618229e5390 0x561822d8d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50478/0 (socket says 172.21.15.39:50478) 2024-09-21T07:54:19.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.439+0000 7f46ca800640 1 -- 172.21.15.39:0/3443740688 learned_addr learned my addr 172.21.15.39:0/3443740688 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:19.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.439+0000 7f46c3e00640 1 --2- 172.21.15.39:0/3443740688 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5618229c46b0 0x561822de1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:19.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.439+0000 7f46c9e00640 1 --2- 172.21.15.39:0/3443740688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5618229bc4d0 0x561822e49ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:19.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.439+0000 7f46ca800640 1 -- 172.21.15.39:0/3443740688 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5618229c46b0 msgr2=0x561822de1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:19.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.439+0000 7f46ca800640 1 --2- 172.21.15.39:0/3443740688 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5618229c46b0 0x561822de1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.440+0000 7f46ca800640 1 -- 172.21.15.39:0/3443740688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5618229bc4d0 msgr2=0x561822e49ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:19.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.440+0000 7f46ca800640 1 --2- 172.21.15.39:0/3443740688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5618229bc4d0 0x561822e49ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.440+0000 7f46ca800640 1 -- 172.21.15.39:0/3443740688 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561822e49ff0 con 0x5618229e5390 2024-09-21T07:54:19.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.440+0000 7f46c9e00640 1 --2- 172.21.15.39:0/3443740688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5618229bc4d0 0x561822e49ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:19.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.440+0000 7f46ca800640 1 --2- 172.21.15.39:0/3443740688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618229e5390 0x561822d8d120 secure :-1 s=READY pgs=2730 cs=0 l=1 rev1=1 crypto rx=0x7f46c406f260 tx=0x7f46c40833d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69893fdc3d5bbe17 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:19.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.440+0000 7f46af400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:54:19.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.440+0000 7f46af400640 1 -- 172.21.15.39:0/3443740688 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c409d020 con 0x5618229e5390 2024-09-21T07:54:19.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.440+0000 7f46af400640 1 -- 172.21.15.39:0/3443740688 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(253..253 src has 1..253) ==== 8035+0+0 (secure 0 0 0) 0x7f46c4082fc0 con 0x5618229e5390 2024-09-21T07:54:19.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.440+0000 7f46d7dcf740 1 -- 172.21.15.39:0/3443740688 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46c4088590 con 0x5618229e5390 2024-09-21T07:54:19.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.440+0000 7f46af400640 1 -- 172.21.15.39:0/3443740688 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=301) ==== 24+0+0 (secure 0 0 0) 0x7f46c4088360 con 0x5618229e5390 2024-09-21T07:54:19.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.440+0000 7f46d7dcf740 10 client.12144 fetch_fsmap learned FSMap version 301 2024-09-21T07:54:19.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.440+0000 7f46d7dcf740 1 -- 172.21.15.39:0/3443740688 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=301,monmap=2+}) -- 0x561822e53f50 con 0x5618229e5390 2024-09-21T07:54:19.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.440+0000 7f46af400640 1 -- 172.21.15.39:0/3443740688 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46c4088590 con 0x5618229e5390 2024-09-21T07:54:19.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.440+0000 7f46af400640 1 -- 172.21.15.39:0/3443740688 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 301) ==== 3542+0+0 (secure 0 0 0) 0x7f46c4040030 con 0x5618229e5390 2024-09-21T07:54:19.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.441+0000 7f46d7dcf740 10 client.12144 fetch_fsmap finished waiting for FSMap version 301 2024-09-21T07:54:19.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.441+0000 7f46d7dcf740 10 client.12144 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.a 2024-09-21T07:54:19.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.441+0000 7f46d7dcf740 20 client.12144 populate_metadata read hostname 'smithi039' 2024-09-21T07:54:19.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.441+0000 7f46d7dcf740 1 --2- 172.21.15.39:0/3443740688 >> [v2:172.21.15.39:6832/2161818466,v1:172.21.15.39:6834/2161818466] conn(0x561822d8d660 0x561822e53120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:19.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.441+0000 7f46d7dcf740 4 client.12144 mds_command: new command op to 11364 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:54:19.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.441+0000 7f46d7dcf740 1 -- 172.21.15.39:0/3443740688 --> [v2:172.21.15.39:6832/2161818466,v1:172.21.15.39:6834/2161818466] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561822cfd270 con 0x561822d8d660 2024-09-21T07:54:19.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.441+0000 7f46c9e00640 1 --2- 172.21.15.39:0/3443740688 >> [v2:172.21.15.39:6832/2161818466,v1:172.21.15.39:6834/2161818466] conn(0x561822d8d660 0x561822e53120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:19.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.441+0000 7f46c9e00640 1 --2- 172.21.15.39:0/3443740688 >> [v2:172.21.15.39:6832/2161818466,v1:172.21.15.39:6834/2161818466] conn(0x561822d8d660 0x561822e53120 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea1b24098f4d0984 server_cookie=817b6192909a7c35 in_seq=0 out_seq=0 2024-09-21T07:54:19.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.441+0000 7f46af400640 10 client.12144 ms_handle_connect on v2:172.21.15.39:6832/2161818466 2024-09-21T07:54:19.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.450+0000 7f46af400640 1 -- 172.21.15.39:0/3443740688 <== mds.0 v2:172.21.15.39:6832/2161818466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561822cfd270 con 0x561822d8d660 2024-09-21T07:54:19.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.450+0000 7f46af400640 10 client.12144 handle_command_reply: tid=0 2024-09-21T07:54:19.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.450+0000 7f46d7dcf740 1 client.12144 shutdown 2024-09-21T07:54:19.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.451+0000 7f46d7dcf740 1 -- 172.21.15.39:0/3443740688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618229e5390 msgr2=0x561822d8d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:19.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.451+0000 7f46d7dcf740 1 --2- 172.21.15.39:0/3443740688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618229e5390 0x561822d8d120 secure :-1 s=READY pgs=2730 cs=0 l=1 rev1=1 crypto rx=0x7f46c406f260 tx=0x7f46c40833d0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.451+0000 7f46d7dcf740 1 -- 172.21.15.39:0/3443740688 shutdown_connections 2024-09-21T07:54:19.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.451+0000 7f46d7dcf740 1 --2- 172.21.15.39:0/3443740688 >> [v2:172.21.15.39:6832/2161818466,v1:172.21.15.39:6834/2161818466] conn(0x561822d8d660 0x561822e53120 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.451+0000 7f46d7dcf740 1 --2- 172.21.15.39:0/3443740688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5618229e5390 0x561822d8d120 unknown :-1 s=CLOSED pgs=2730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.451+0000 7f46d7dcf740 1 --2- 172.21.15.39:0/3443740688 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x5618229c46b0 0x561822de1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.451+0000 7f46af400640 0 client.12144 ms_handle_reset on v2:172.21.15.39:6832/2161818466 2024-09-21T07:54:19.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.451+0000 7f46d7dcf740 1 --2- 172.21.15.39:0/3443740688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5618229bc4d0 0x561822e49ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.451+0000 7f46d7dcf740 1 -- 172.21.15.39:0/3443740688 >> 172.21.15.39:0/3443740688 conn(0x5618229d94e0 msgr2=0x561822da5c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:19.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.451+0000 7f46d7dcf740 1 -- 172.21.15.39:0/3443740688 shutdown_connections 2024-09-21T07:54:19.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.451+0000 7f46d7dcf740 1 -- 172.21.15.39:0/3443740688 wait complete. 2024-09-21T07:54:19.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.451+0000 7f46d7dcf740 20 client.12144 trim_cache size 0 max 16384 2024-09-21T07:54:19.454 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:54:19.454 INFO:teuthology.orchestra.run.smithi039.stdout: "message": "", 2024-09-21T07:54:19.455 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0 2024-09-21T07:54:19.455 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:54:19.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.452+0000 7f46d0e00640 1 -- 172.21.15.39:0/1409983495 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f46a8078b90 msgr2=0x7f46a807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:19.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.452+0000 7f46d0e00640 1 --2- 172.21.15.39:0/1409983495 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f46a8078b90 0x7f46a807b0a0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f46bc000f60 tx=0x7f46bc009040 comp rx=0 tx=0).stop 2024-09-21T07:54:19.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.452+0000 7f46d0e00640 1 -- 172.21.15.39:0/1409983495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46cc15cf60 msgr2=0x7f46cc0b0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:19.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.452+0000 7f46d0e00640 1 --2- 172.21.15.39:0/1409983495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46cc15cf60 0x7f46cc0b0e30 secure :-1 s=READY pgs=2726 cs=0 l=1 rev1=1 crypto rx=0x7f46c4006010 tx=0x7f46c4003f10 comp rx=0 tx=0).stop 2024-09-21T07:54:19.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.452+0000 7f46d0e00640 1 -- 172.21.15.39:0/1409983495 shutdown_connections 2024-09-21T07:54:19.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.452+0000 7f46d0e00640 1 --2- 172.21.15.39:0/1409983495 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f46a8078b90 0x7f46a807b0a0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.452+0000 7f46d0e00640 1 --2- 172.21.15.39:0/1409983495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46cc15cf60 0x7f46cc0b0e30 unknown :-1 s=CLOSED pgs=2726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.452+0000 7f46d0e00640 1 --2- 172.21.15.39:0/1409983495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46cc0cce70 0x7f46cc0ce9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.452+0000 7f46d0e00640 1 --2- 172.21.15.39:0/1409983495 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f46cc0b32b0 0x7f46cc0c9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.452+0000 7f46d0e00640 1 -- 172.21.15.39:0/1409983495 >> 172.21.15.39:0/1409983495 conn(0x7f46cc06d440 msgr2=0x7f46cc0aec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:19.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.452+0000 7f46d0e00640 1 -- 172.21.15.39:0/1409983495 shutdown_connections 2024-09-21T07:54:19.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:19.452+0000 7f46d0e00640 1 -- 172.21.15.39:0/1409983495 wait complete. 2024-09-21T07:54:19.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:19.476 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir d0/d2/dir/.snap/*) 2024-09-21T07:54:19.536 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:54:19.536 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:54:19.620 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:54:19.719 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:54:19.720 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:54:19.720 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:54:19.720 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:19.755 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:54:19.755 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:19.830 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:54:20.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.096+0000 7f9c3e000640 1 -- 172.21.15.39:0/4183773063 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9c3815cd30 msgr2=0x7f9c380cfb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:20.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.096+0000 7f9c3e000640 1 --2- 172.21.15.39:0/4183773063 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9c3815cd30 0x7f9c380cfb30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9c28005fd0 tx=0x7f9c2802a840 comp rx=0 tx=0).stop 2024-09-21T07:54:20.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.096+0000 7f9c3e000640 1 -- 172.21.15.39:0/4183773063 shutdown_connections 2024-09-21T07:54:20.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.096+0000 7f9c3e000640 1 --2- 172.21.15.39:0/4183773063 >> v2:172.21.15.148:3300/0 conn(0x7f9c3806fc10 0x7f9c380d06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:20.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.096+0000 7f9c3e000640 1 --2- 172.21.15.39:0/4183773063 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9c3815cd30 0x7f9c380cfb30 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:20.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.096+0000 7f9c3e000640 1 -- 172.21.15.39:0/4183773063 >> 172.21.15.39:0/4183773063 conn(0x7f9c3806cfc0 msgr2=0x7f9c3806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:20.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.096+0000 7f9c3e000640 1 -- 172.21.15.39:0/4183773063 shutdown_connections 2024-09-21T07:54:20.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.096+0000 7f9c3e000640 1 -- 172.21.15.39:0/4183773063 wait complete. 2024-09-21T07:54:20.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.098+0000 7f9c3e000640 1 Processor -- start 2024-09-21T07:54:20.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.098+0000 7f9c3e000640 1 -- start start 2024-09-21T07:54:20.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.100+0000 7f9c3e000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9c3806f800 0x7f9c381e4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:20.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.100+0000 7f9c3e000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c3806fc10 0x7f9c38145e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:20.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.100+0000 7f9c3e000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c3815cd30 0x7f9c38155500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:20.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.100+0000 7f9c3e000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9c38073890 con 0x7f9c3806fc10 2024-09-21T07:54:20.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.100+0000 7f9c3e000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9c38073710 con 0x7f9c3815cd30 2024-09-21T07:54:20.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.100+0000 7f9c3e000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f9c38073a10 con 0x7f9c3806f800 2024-09-21T07:54:20.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.100+0000 7f9c2ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c3806fc10 0x7f9c38145e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:20.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.100+0000 7f9c2ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c3806fc10 0x7f9c38145e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50480/0 (socket says 172.21.15.39:50480) 2024-09-21T07:54:20.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.100+0000 7f9c2ea00640 1 -- 172.21.15.39:0/2034434751 learned_addr learned my addr 172.21.15.39:0/2034434751 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:20.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.100+0000 7f9c37e00640 1 --2- 172.21.15.39:0/2034434751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c3815cd30 0x7f9c38155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:20.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.100+0000 7f9c37400640 1 --2- 172.21.15.39:0/2034434751 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9c3806f800 0x7f9c381e4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:20.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.101+0000 7f9c2ea00640 1 -- 172.21.15.39:0/2034434751 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9c3806f800 msgr2=0x7f9c381e4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:20.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.101+0000 7f9c2ea00640 1 --2- 172.21.15.39:0/2034434751 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9c3806f800 0x7f9c381e4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:20.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.101+0000 7f9c2ea00640 1 -- 172.21.15.39:0/2034434751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c3815cd30 msgr2=0x7f9c38155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:20.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.101+0000 7f9c2ea00640 1 --2- 172.21.15.39:0/2034434751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c3815cd30 0x7f9c38155500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:20.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.101+0000 7f9c2ea00640 1 -- 172.21.15.39:0/2034434751 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c38154460 con 0x7f9c3806fc10 2024-09-21T07:54:20.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.101+0000 7f9c2ea00640 1 --2- 172.21.15.39:0/2034434751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c3806fc10 0x7f9c38145e30 secure :-1 s=READY pgs=2732 cs=0 l=1 rev1=1 crypto rx=0x7f9c2000c530 tx=0x7f9c2000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:20.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.101+0000 7f9c2fe00640 1 -- 172.21.15.39:0/2034434751 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c20016020 con 0x7f9c3806fc10 2024-09-21T07:54:20.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.101+0000 7f9c2fe00640 1 -- 172.21.15.39:0/2034434751 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c20012940 con 0x7f9c3806fc10 2024-09-21T07:54:20.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.101+0000 7f9c2fe00640 1 -- 172.21.15.39:0/2034434751 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c20012d90 con 0x7f9c3806fc10 2024-09-21T07:54:20.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.101+0000 7f9c3e000640 1 -- 172.21.15.39:0/2034434751 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c38154740 con 0x7f9c3806fc10 2024-09-21T07:54:20.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.101+0000 7f9c3e000640 1 -- 172.21.15.39:0/2034434751 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c38154d00 con 0x7f9c3806fc10 2024-09-21T07:54:20.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.103+0000 7f9c3e000640 1 -- 172.21.15.39:0/2034434751 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c38073f50 con 0x7f9c3806fc10 2024-09-21T07:54:20.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.104+0000 7f9c2fe00640 1 -- 172.21.15.39:0/2034434751 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9c20019020 con 0x7f9c3806fc10 2024-09-21T07:54:20.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.105+0000 7f9c2fe00640 1 --2- 172.21.15.39:0/2034434751 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9c14078b90 0x7f9c1407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:20.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.105+0000 7f9c2fe00640 1 -- 172.21.15.39:0/2034434751 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(253..253 src has 1..253) ==== 8035+0+0 (secure 0 0 0) 0x7f9c2009b5b0 con 0x7f9c3806fc10 2024-09-21T07:54:20.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.105+0000 7f9c37400640 1 --2- 172.21.15.39:0/2034434751 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9c14078b90 0x7f9c1407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:20.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.106+0000 7f9c37400640 1 --2- 172.21.15.39:0/2034434751 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9c14078b90 0x7f9c1407b0a0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f9c28033000 tx=0x7f9c28004ab0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:20.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.109+0000 7f9c2fe00640 1 -- 172.21.15.39:0/2034434751 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9c20020020 con 0x7f9c3806fc10 2024-09-21T07:54:20.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.320+0000 7f9c3e000640 1 -- 172.21.15.39:0/2034434751 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9c380cfd40 con 0x7f9c3806fc10 2024-09-21T07:54:20.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.321+0000 7f9c2fe00640 1 -- 172.21.15.39:0/2034434751 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 21 entries v253) ==== 81+0+1274 (secure 0 0 0) 0x7f9c200629e0 con 0x7f9c3806fc10 2024-09-21T07:54:20.323 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/249900061 2024-09-21T08:54:08.362327+0000 2024-09-21T07:54:20.323 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6835/2191156088 2024-09-22T07:53:58.762864+0000 2024-09-21T07:54:20.323 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/485194245 2024-09-22T07:53:58.201516+0000 2024-09-21T07:54:20.323 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6833/485194245 2024-09-22T07:53:58.201516+0000 2024-09-21T07:54:20.324 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/4061644382 2024-09-21T08:53:16.749620+0000 2024-09-21T07:54:20.324 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/597301351 2024-09-22T07:53:09.120490+0000 2024-09-21T07:54:20.324 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6834/2191156088 2024-09-22T07:53:58.762864+0000 2024-09-21T07:54:20.324 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6835/597301351 2024-09-22T07:53:09.120490+0000 2024-09-21T07:54:20.324 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/3381818843 2024-09-22T07:53:08.565426+0000 2024-09-21T07:54:20.324 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6834/3381818843 2024-09-22T07:53:08.565426+0000 2024-09-21T07:54:20.324 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/106826898 2024-09-21T08:52:26.149462+0000 2024-09-21T07:54:20.324 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/109229385 2024-09-22T07:51:36.504781+0000 2024-09-21T07:54:20.324 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/109229385 2024-09-22T07:51:36.504781+0000 2024-09-21T07:54:20.324 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6837/1438851059 2024-09-22T07:51:00.902180+0000 2024-09-21T07:54:20.324 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6835/1438851059 2024-09-22T07:51:00.902180+0000 2024-09-21T07:54:20.324 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6835/2824493631 2024-09-22T07:50:31.817924+0000 2024-09-21T07:54:20.324 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6833/2824493631 2024-09-22T07:50:31.817924+0000 2024-09-21T07:54:20.324 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6834/4143807732 2024-09-22T07:50:32.369206+0000 2024-09-21T07:54:20.325 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6832/4143807732 2024-09-22T07:50:32.369206+0000 2024-09-21T07:54:20.325 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6836/3497801035 2024-09-22T07:50:24.850152+0000 2024-09-21T07:54:20.325 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.148:6834/3497801035 2024-09-22T07:50:24.850152+0000 2024-09-21T07:54:20.325 INFO:teuthology.orchestra.run.smithi039.stderr:listed 21 entries 2024-09-21T07:54:20.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.324+0000 7f9c3e000640 1 -- 172.21.15.39:0/2034434751 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9c14078b90 msgr2=0x7f9c1407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:20.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.324+0000 7f9c3e000640 1 --2- 172.21.15.39:0/2034434751 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9c14078b90 0x7f9c1407b0a0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f9c28033000 tx=0x7f9c28004ab0 comp rx=0 tx=0).stop 2024-09-21T07:54:20.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.324+0000 7f9c3e000640 1 -- 172.21.15.39:0/2034434751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c3806fc10 msgr2=0x7f9c38145e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:20.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.324+0000 7f9c3e000640 1 --2- 172.21.15.39:0/2034434751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c3806fc10 0x7f9c38145e30 secure :-1 s=READY pgs=2732 cs=0 l=1 rev1=1 crypto rx=0x7f9c2000c530 tx=0x7f9c2000c9f0 comp rx=0 tx=0).stop 2024-09-21T07:54:20.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.324+0000 7f9c3e000640 1 -- 172.21.15.39:0/2034434751 shutdown_connections 2024-09-21T07:54:20.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.324+0000 7f9c3e000640 1 --2- 172.21.15.39:0/2034434751 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9c14078b90 0x7f9c1407b0a0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:20.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.324+0000 7f9c3e000640 1 --2- 172.21.15.39:0/2034434751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9c3815cd30 0x7f9c38155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:20.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.324+0000 7f9c3e000640 1 --2- 172.21.15.39:0/2034434751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c3806fc10 0x7f9c38145e30 unknown :-1 s=CLOSED pgs=2732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:20.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.324+0000 7f9c3e000640 1 --2- 172.21.15.39:0/2034434751 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9c3806f800 0x7f9c381e4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:20.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.324+0000 7f9c3e000640 1 -- 172.21.15.39:0/2034434751 >> 172.21.15.39:0/2034434751 conn(0x7f9c3806cfc0 msgr2=0x7f9c3815bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:20.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.324+0000 7f9c3e000640 1 -- 172.21.15.39:0/2034434751 shutdown_connections 2024-09-21T07:54:20.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.324+0000 7f9c3e000640 1 -- 172.21.15.39:0/2034434751 wait complete. 2024-09-21T07:54:20.347 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T07:54:20.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:20.348 DEBUG:teuthology.orchestra.run.smithi039:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T07:54:20.411 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T07:54:20.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:20.412 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:54:20.448 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:54:20.448 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:54:20.448 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:20.488 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:54:20.489 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:54:20.489 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:54:20.489 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:20.524 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:54:20.524 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:54:20.524 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:54:20.524 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:20.566 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:54:20.567 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:54:20.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.790+0000 7f9d7f200640 1 -- 172.21.15.39:0/3022056281 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d8015c520 msgr2=0x7f9d800b3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:20.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.790+0000 7f9d7f200640 1 --2- 172.21.15.39:0/3022056281 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d8015c520 0x7f9d800b3ae0 secure :-1 s=READY pgs=2733 cs=0 l=1 rev1=1 crypto rx=0x7f9d70005fd0 tx=0x7f9d7002a670 comp rx=0 tx=0).stop 2024-09-21T07:54:20.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.790+0000 7f9d7f200640 1 -- 172.21.15.39:0/3022056281 shutdown_connections 2024-09-21T07:54:20.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.790+0000 7f9d7f200640 1 --2- 172.21.15.39:0/3022056281 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9d800cf1e0 0x7f9d800b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:20.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.790+0000 7f9d7f200640 1 --2- 172.21.15.39:0/3022056281 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d8015c520 0x7f9d800b3ae0 unknown :-1 s=CLOSED pgs=2733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:20.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.790+0000 7f9d7f200640 1 -- 172.21.15.39:0/3022056281 >> 172.21.15.39:0/3022056281 conn(0x7f9d80074050 msgr2=0x7f9d80074490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:20.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.790+0000 7f9d7f200640 1 -- 172.21.15.39:0/3022056281 shutdown_connections 2024-09-21T07:54:20.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.791+0000 7f9d7f200640 1 -- 172.21.15.39:0/3022056281 wait complete. 2024-09-21T07:54:20.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.793+0000 7f9d7f200640 1 Processor -- start 2024-09-21T07:54:20.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.793+0000 7f9d7f200640 1 -- start start 2024-09-21T07:54:20.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.794+0000 7f9d7f200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9d800b4020 0x7f9d801f9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:20.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.794+0000 7f9d7f200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d800cf1e0 0x7f9d8015b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:20.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.794+0000 7f9d7f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d8015c520 0x7f9d80153af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:20.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.794+0000 7f9d7f200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9d801609e0 con 0x7f9d8015c520 2024-09-21T07:54:20.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.794+0000 7f9d7f200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9d80160860 con 0x7f9d800cf1e0 2024-09-21T07:54:20.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.794+0000 7f9d7f200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f9d80160b60 con 0x7f9d800b4020 2024-09-21T07:54:20.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.795+0000 7f9d7e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d8015c520 0x7f9d80153af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:20.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.795+0000 7f9d7e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d8015c520 0x7f9d80153af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50504/0 (socket says 172.21.15.39:50504) 2024-09-21T07:54:20.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.795+0000 7f9d7e800640 1 -- 172.21.15.39:0/683016544 learned_addr learned my addr 172.21.15.39:0/683016544 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:20.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.795+0000 7f9d7de00640 1 --2- 172.21.15.39:0/683016544 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9d800b4020 0x7f9d801f9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:20.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.795+0000 7f9d75400640 1 --2- 172.21.15.39:0/683016544 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d800cf1e0 0x7f9d8015b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:20.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.795+0000 7f9d7e800640 1 -- 172.21.15.39:0/683016544 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9d800b4020 msgr2=0x7f9d801f9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:20.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.795+0000 7f9d7e800640 1 --2- 172.21.15.39:0/683016544 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9d800b4020 0x7f9d801f9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:20.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.795+0000 7f9d7e800640 1 -- 172.21.15.39:0/683016544 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d800cf1e0 msgr2=0x7f9d8015b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:20.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.795+0000 7f9d7e800640 1 --2- 172.21.15.39:0/683016544 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d800cf1e0 0x7f9d8015b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:20.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.795+0000 7f9d7e800640 1 -- 172.21.15.39:0/683016544 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d8015bdc0 con 0x7f9d8015c520 2024-09-21T07:54:20.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.796+0000 7f9d7e800640 1 --2- 172.21.15.39:0/683016544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d8015c520 0x7f9d80153af0 secure :-1 s=READY pgs=2734 cs=0 l=1 rev1=1 crypto rx=0x7f9d7800ea70 tx=0x7f9d7800ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:20.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.796+0000 7f9d76a00640 1 -- 172.21.15.39:0/683016544 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7800cde0 con 0x7f9d8015c520 2024-09-21T07:54:20.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.796+0000 7f9d76a00640 1 -- 172.21.15.39:0/683016544 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d78004c10 con 0x7f9d8015c520 2024-09-21T07:54:20.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.796+0000 7f9d7f200640 1 -- 172.21.15.39:0/683016544 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d80156c30 con 0x7f9d8015c520 2024-09-21T07:54:20.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.796+0000 7f9d76a00640 1 -- 172.21.15.39:0/683016544 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d78005020 con 0x7f9d8015c520 2024-09-21T07:54:20.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.796+0000 7f9d7f200640 1 -- 172.21.15.39:0/683016544 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d801f6010 con 0x7f9d8015c520 2024-09-21T07:54:20.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.797+0000 7f9d7f200640 1 -- 172.21.15.39:0/683016544 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d800b4430 con 0x7f9d8015c520 2024-09-21T07:54:20.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.798+0000 7f9d76a00640 1 -- 172.21.15.39:0/683016544 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9d780051c0 con 0x7f9d8015c520 2024-09-21T07:54:20.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.799+0000 7f9d76a00640 1 --2- 172.21.15.39:0/683016544 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9d5c078bc0 0x7f9d5c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:20.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.800+0000 7f9d76a00640 1 -- 172.21.15.39:0/683016544 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(253..253 src has 1..253) ==== 8035+0+0 (secure 0 0 0) 0x7f9d7809c320 con 0x7f9d8015c520 2024-09-21T07:54:20.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.800+0000 7f9d7de00640 1 --2- 172.21.15.39:0/683016544 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9d5c078bc0 0x7f9d5c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:20.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.801+0000 7f9d7de00640 1 --2- 172.21.15.39:0/683016544 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9d5c078bc0 0x7f9d5c07b0d0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f9d70033000 tx=0x7f9d70004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:20.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:20.804+0000 7f9d76a00640 1 -- 172.21.15.39:0/683016544 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9d78063750 con 0x7f9d8015c520 2024-09-21T07:54:21.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.039+0000 7f9d7f200640 1 -- 172.21.15.39:0/683016544 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d801569c0 con 0x7f9d8015c520 2024-09-21T07:54:21.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.039+0000 7f9d76a00640 1 -- 172.21.15.39:0/683016544 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 301 v301) ==== 95+0+5291 (secure 0 0 0) 0x7f9d78068180 con 0x7f9d8015c520 2024-09-21T07:54:21.041 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:21.042 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":301,"btime":"2024-09-21T07:54:11:059441+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9467,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3658109005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3658109005},{"type":"v1","addr":"172.21.15.148:6835","nonce":3658109005}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":298}],"filesystems":[{"mdsmap":{"epoch":301,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:54:11.059432+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":252,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":11364,"mds_1":10633,"mds_2":11970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10633":{"gid":10633,"name":"d","rank":1,"incarnation":295,"state":"up:active","state_seq":19,"addr":"172.21.15.148:6837/3495264983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3495264983},{"type":"v1","addr":"172.21.15.148:6837","nonce":3495264983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11364":{"gid":11364,"name":"a","rank":0,"incarnation":285,"state":"up:active","state_seq":28,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11970":{"gid":11970,"name":"c","rank":2,"incarnation":297,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6835/732408880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":732408880},{"type":"v1","addr":"172.21.15.39:6835","nonce":732408880}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11364,"qdb_cluster":[10633,11364,11970]},"id":13}]} 2024-09-21T07:54:21.042 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 301 2024-09-21T07:54:21.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.042+0000 7f9d7f200640 1 -- 172.21.15.39:0/683016544 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9d5c078bc0 msgr2=0x7f9d5c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:21.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.042+0000 7f9d7f200640 1 --2- 172.21.15.39:0/683016544 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9d5c078bc0 0x7f9d5c07b0d0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f9d70033000 tx=0x7f9d70004730 comp rx=0 tx=0).stop 2024-09-21T07:54:21.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.042+0000 7f9d7f200640 1 -- 172.21.15.39:0/683016544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d8015c520 msgr2=0x7f9d80153af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:21.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.042+0000 7f9d7f200640 1 --2- 172.21.15.39:0/683016544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d8015c520 0x7f9d80153af0 secure :-1 s=READY pgs=2734 cs=0 l=1 rev1=1 crypto rx=0x7f9d7800ea70 tx=0x7f9d7800ef30 comp rx=0 tx=0).stop 2024-09-21T07:54:21.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.042+0000 7f9d7f200640 1 -- 172.21.15.39:0/683016544 shutdown_connections 2024-09-21T07:54:21.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.042+0000 7f9d7f200640 1 --2- 172.21.15.39:0/683016544 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9d5c078bc0 0x7f9d5c07b0d0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:21.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.042+0000 7f9d7f200640 1 --2- 172.21.15.39:0/683016544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d8015c520 0x7f9d80153af0 unknown :-1 s=CLOSED pgs=2734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:21.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.042+0000 7f9d7f200640 1 --2- 172.21.15.39:0/683016544 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d800cf1e0 0x7f9d8015b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:21.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.042+0000 7f9d7f200640 1 --2- 172.21.15.39:0/683016544 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f9d800b4020 0x7f9d801f9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:21.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.042+0000 7f9d7f200640 1 -- 172.21.15.39:0/683016544 >> 172.21.15.39:0/683016544 conn(0x7f9d80074050 msgr2=0x7f9d800aebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:21.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.042+0000 7f9d7f200640 1 -- 172.21.15.39:0/683016544 shutdown_connections 2024-09-21T07:54:21.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.042+0000 7f9d7f200640 1 -- 172.21.15.39:0/683016544 wait complete. 2024-09-21T07:54:21.067 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:54:21.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.291+0000 7f82bdc00640 1 -- 172.21.15.39:0/2093432927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82b80b3f30 msgr2=0x7f82b80b4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:21.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.291+0000 7f82bdc00640 1 --2- 172.21.15.39:0/2093432927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82b80b3f30 0x7f82b80b4690 secure :-1 s=READY pgs=2735 cs=0 l=1 rev1=1 crypto rx=0x7f82b80b5a30 tx=0x7f82b002aa30 comp rx=0 tx=0).stop 2024-09-21T07:54:21.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.291+0000 7f82bdc00640 1 -- 172.21.15.39:0/2093432927 shutdown_connections 2024-09-21T07:54:21.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.291+0000 7f82bdc00640 1 --2- 172.21.15.39:0/2093432927 >> v2:172.21.15.148:3300/0 conn(0x7f82b815bd70 0x7f82b80b39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:21.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.291+0000 7f82bdc00640 1 --2- 172.21.15.39:0/2093432927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82b80b3f30 0x7f82b80b4690 unknown :-1 s=CLOSED pgs=2735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:21.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.291+0000 7f82bdc00640 1 -- 172.21.15.39:0/2093432927 >> 172.21.15.39:0/2093432927 conn(0x7f82b806cfc0 msgr2=0x7f82b806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:21.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.291+0000 7f82bdc00640 1 -- 172.21.15.39:0/2093432927 shutdown_connections 2024-09-21T07:54:21.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.291+0000 7f82bdc00640 1 -- 172.21.15.39:0/2093432927 wait complete. 2024-09-21T07:54:21.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.293+0000 7f82bdc00640 1 Processor -- start 2024-09-21T07:54:21.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.293+0000 7f82bdc00640 1 -- start start 2024-09-21T07:54:21.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.295+0000 7f82bdc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82b80b3f30 0x7f82b8147a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:21.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.295+0000 7f82bdc00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82b815bd70 0x7f82b814a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:21.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.295+0000 7f82bdc00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82b815f360 0x7f82b814a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:21.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.295+0000 7f82bdc00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f82b80b4f60 con 0x7f82b80b3f30 2024-09-21T07:54:21.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.295+0000 7f82bdc00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f82b80b4de0 con 0x7f82b815bd70 2024-09-21T07:54:21.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.295+0000 7f82bdc00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f82b80d0620 con 0x7f82b815f360 2024-09-21T07:54:21.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.295+0000 7f82b6a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82b80b3f30 0x7f82b8147a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:21.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.295+0000 7f82b6a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82b80b3f30 0x7f82b8147a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50534/0 (socket says 172.21.15.39:50534) 2024-09-21T07:54:21.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.295+0000 7f82b6a00640 1 -- 172.21.15.39:0/2121430227 learned_addr learned my addr 172.21.15.39:0/2121430227 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:21.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.295+0000 7f82b6000640 1 --2- 172.21.15.39:0/2121430227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82b815bd70 0x7f82b814a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:21.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.295+0000 7f82b7400640 1 --2- 172.21.15.39:0/2121430227 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82b815f360 0x7f82b814a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:21.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.295+0000 7f82b6a00640 1 -- 172.21.15.39:0/2121430227 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82b815f360 msgr2=0x7f82b814a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:21.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.295+0000 7f82b6a00640 1 --2- 172.21.15.39:0/2121430227 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82b815f360 0x7f82b814a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:21.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.295+0000 7f82b6a00640 1 -- 172.21.15.39:0/2121430227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82b815bd70 msgr2=0x7f82b814a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:21.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.295+0000 7f82b6a00640 1 --2- 172.21.15.39:0/2121430227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82b815bd70 0x7f82b814a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:21.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.295+0000 7f82b6a00640 1 -- 172.21.15.39:0/2121430227 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82b8148350 con 0x7f82b80b3f30 2024-09-21T07:54:21.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.296+0000 7f82b6a00640 1 --2- 172.21.15.39:0/2121430227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82b80b3f30 0x7f82b8147a20 secure :-1 s=READY pgs=2736 cs=0 l=1 rev1=1 crypto rx=0x7f82a800c3a0 tx=0x7f82a800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:21.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.296+0000 7f82aea00640 1 -- 172.21.15.39:0/2121430227 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a8017070 con 0x7f82b80b3f30 2024-09-21T07:54:21.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.296+0000 7f82aea00640 1 -- 172.21.15.39:0/2121430227 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82a80053c0 con 0x7f82b80b3f30 2024-09-21T07:54:21.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.296+0000 7f82aea00640 1 -- 172.21.15.39:0/2121430227 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a8003850 con 0x7f82b80b3f30 2024-09-21T07:54:21.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.296+0000 7f82bdc00640 1 -- 172.21.15.39:0/2121430227 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82b8148630 con 0x7f82b80b3f30 2024-09-21T07:54:21.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.296+0000 7f82bdc00640 1 -- 172.21.15.39:0/2121430227 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82b80abf10 con 0x7f82b80b3f30 2024-09-21T07:54:21.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.297+0000 7f82bdc00640 1 -- 172.21.15.39:0/2121430227 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82b80cd460 con 0x7f82b80b3f30 2024-09-21T07:54:21.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.298+0000 7f82aea00640 1 -- 172.21.15.39:0/2121430227 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f82a801a070 con 0x7f82b80b3f30 2024-09-21T07:54:21.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.299+0000 7f82aea00640 1 --2- 172.21.15.39:0/2121430227 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8298078b90 0x7f829807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:21.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.299+0000 7f82aea00640 1 -- 172.21.15.39:0/2121430227 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(253..253 src has 1..253) ==== 8035+0+0 (secure 0 0 0) 0x7f82a809aa40 con 0x7f82b80b3f30 2024-09-21T07:54:21.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.300+0000 7f82b6000640 1 --2- 172.21.15.39:0/2121430227 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8298078b90 0x7f829807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:21.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.301+0000 7f82b6000640 1 --2- 172.21.15.39:0/2121430227 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8298078b90 0x7f829807b0a0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f82a4005fd0 tx=0x7f82a4005850 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:21.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.304+0000 7f82aea00640 1 -- 172.21.15.39:0/2121430227 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f82b80cd460 con 0x7f82b80b3f30 2024-09-21T07:54:21.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.539+0000 7f82bdc00640 1 -- 172.21.15.39:0/2121430227 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82b80ac140 con 0x7f82b80b3f30 2024-09-21T07:54:21.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.539+0000 7f82aea00640 1 -- 172.21.15.39:0/2121430227 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 301 v301) ==== 95+0+5291 (secure 0 0 0) 0x7f82b80ac140 con 0x7f82b80b3f30 2024-09-21T07:54:21.542 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:21.542 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":301,"btime":"2024-09-21T07:54:11:059441+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9467,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3658109005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3658109005},{"type":"v1","addr":"172.21.15.148:6835","nonce":3658109005}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":298}],"filesystems":[{"mdsmap":{"epoch":301,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:54:11.059432+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":252,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":11364,"mds_1":10633,"mds_2":11970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10633":{"gid":10633,"name":"d","rank":1,"incarnation":295,"state":"up:active","state_seq":19,"addr":"172.21.15.148:6837/3495264983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3495264983},{"type":"v1","addr":"172.21.15.148:6837","nonce":3495264983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11364":{"gid":11364,"name":"a","rank":0,"incarnation":285,"state":"up:active","state_seq":28,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11970":{"gid":11970,"name":"c","rank":2,"incarnation":297,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6835/732408880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":732408880},{"type":"v1","addr":"172.21.15.39:6835","nonce":732408880}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11364,"qdb_cluster":[10633,11364,11970]},"id":13}]} 2024-09-21T07:54:21.542 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 301 2024-09-21T07:54:21.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.542+0000 7f82bdc00640 1 -- 172.21.15.39:0/2121430227 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8298078b90 msgr2=0x7f829807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:21.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.542+0000 7f82bdc00640 1 --2- 172.21.15.39:0/2121430227 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8298078b90 0x7f829807b0a0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f82a4005fd0 tx=0x7f82a4005850 comp rx=0 tx=0).stop 2024-09-21T07:54:21.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.542+0000 7f82bdc00640 1 -- 172.21.15.39:0/2121430227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82b80b3f30 msgr2=0x7f82b8147a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:21.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.542+0000 7f82bdc00640 1 --2- 172.21.15.39:0/2121430227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82b80b3f30 0x7f82b8147a20 secure :-1 s=READY pgs=2736 cs=0 l=1 rev1=1 crypto rx=0x7f82a800c3a0 tx=0x7f82a800c860 comp rx=0 tx=0).stop 2024-09-21T07:54:21.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.542+0000 7f82bdc00640 1 -- 172.21.15.39:0/2121430227 shutdown_connections 2024-09-21T07:54:21.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.542+0000 7f82bdc00640 1 --2- 172.21.15.39:0/2121430227 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8298078b90 0x7f829807b0a0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:21.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.542+0000 7f82bdc00640 1 --2- 172.21.15.39:0/2121430227 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f82b815f360 0x7f82b814a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:21.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.542+0000 7f82bdc00640 1 --2- 172.21.15.39:0/2121430227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f82b815bd70 0x7f82b814a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:21.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.542+0000 7f82bdc00640 1 --2- 172.21.15.39:0/2121430227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82b80b3f30 0x7f82b8147a20 unknown :-1 s=CLOSED pgs=2736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:21.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.542+0000 7f82bdc00640 1 -- 172.21.15.39:0/2121430227 >> 172.21.15.39:0/2121430227 conn(0x7f82b806cfc0 msgr2=0x7f82b80cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:21.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.542+0000 7f82bdc00640 1 -- 172.21.15.39:0/2121430227 shutdown_connections 2024-09-21T07:54:21.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.542+0000 7f82bdc00640 1 -- 172.21.15.39:0/2121430227 wait complete. 2024-09-21T07:54:21.566 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:54:21.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.791+0000 7f3421000640 1 -- 172.21.15.39:0/724678420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3400003fd0 msgr2=0x7f3400004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:21.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.791+0000 7f3421000640 1 --2- 172.21.15.39:0/724678420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3400003fd0 0x7f3400004800 secure :-1 s=READY pgs=2738 cs=0 l=1 rev1=1 crypto rx=0x7f341c09e400 tx=0x7f3400005830 comp rx=0 tx=0).stop 2024-09-21T07:54:21.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.791+0000 7f341be00640 1 -- 172.21.15.39:0/724678420 reap_dead start 2024-09-21T07:54:21.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.791+0000 7f3421000640 1 -- 172.21.15.39:0/724678420 shutdown_connections 2024-09-21T07:54:21.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.791+0000 7f3421000640 1 -- 172.21.15.39:0/724678420 >> 172.21.15.39:0/724678420 conn(0x7f341c0b2510 msgr2=0x7f341c0d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:21.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.791+0000 7f3421000640 1 -- 172.21.15.39:0/724678420 shutdown_connections 2024-09-21T07:54:21.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.792+0000 7f3421000640 1 -- 172.21.15.39:0/724678420 wait complete. 2024-09-21T07:54:21.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.794+0000 7f3421000640 1 Processor -- start 2024-09-21T07:54:21.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.794+0000 7f3421000640 1 -- start start 2024-09-21T07:54:21.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.795+0000 7f3421000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3400003fd0 0x7f341c074da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:21.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.795+0000 7f3421000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f341c160570 0x7f341c1f87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:21.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.795+0000 7f3421000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f341c160ac0 0x7f341c1f8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:21.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.796+0000 7f3421000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f341c0af710 con 0x7f341c160570 2024-09-21T07:54:21.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.796+0000 7f3421000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f341c0af590 con 0x7f341c160ac0 2024-09-21T07:54:21.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.796+0000 7f3421000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f341c0af890 con 0x7f3400003fd0 2024-09-21T07:54:21.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.796+0000 7f3413e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f341c160570 0x7f341c1f87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:21.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.796+0000 7f3413e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f341c160570 0x7f341c1f87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50548/0 (socket says 172.21.15.39:50548) 2024-09-21T07:54:21.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.796+0000 7f3413e00640 1 -- 172.21.15.39:0/448863654 learned_addr learned my addr 172.21.15.39:0/448863654 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:21.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.796+0000 7f341b400640 1 --2- 172.21.15.39:0/448863654 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3400003fd0 0x7f341c074da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:21.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.796+0000 7f341be00640 1 --2- 172.21.15.39:0/448863654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f341c160ac0 0x7f341c1f8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:21.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.796+0000 7f3413e00640 1 -- 172.21.15.39:0/448863654 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3400003fd0 msgr2=0x7f341c074da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:21.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.796+0000 7f3413e00640 1 --2- 172.21.15.39:0/448863654 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3400003fd0 0x7f341c074da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:21.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.796+0000 7f3413e00640 1 -- 172.21.15.39:0/448863654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f341c160ac0 msgr2=0x7f341c1f8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:21.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.796+0000 7f3413e00640 1 --2- 172.21.15.39:0/448863654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f341c160ac0 0x7f341c1f8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:21.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.796+0000 7f3413e00640 1 -- 172.21.15.39:0/448863654 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f341c072f60 con 0x7f341c160570 2024-09-21T07:54:21.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.797+0000 7f3413e00640 1 --2- 172.21.15.39:0/448863654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f341c160570 0x7f341c1f87c0 secure :-1 s=READY pgs=2739 cs=0 l=1 rev1=1 crypto rx=0x7f340402b070 tx=0x7f3404001e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:21.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.797+0000 7f3413400640 1 -- 172.21.15.39:0/448863654 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3404032030 con 0x7f341c160570 2024-09-21T07:54:21.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.797+0000 7f3413400640 1 -- 172.21.15.39:0/448863654 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3404003af0 con 0x7f341c160570 2024-09-21T07:54:21.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.797+0000 7f3421000640 1 -- 172.21.15.39:0/448863654 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f341c0752e0 con 0x7f341c160570 2024-09-21T07:54:21.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.797+0000 7f3413400640 1 -- 172.21.15.39:0/448863654 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3404037420 con 0x7f341c160570 2024-09-21T07:54:21.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.797+0000 7f3421000640 1 -- 172.21.15.39:0/448863654 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f341c1f95d0 con 0x7f341c160570 2024-09-21T07:54:21.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.798+0000 7f3421000640 1 -- 172.21.15.39:0/448863654 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f341c0cf0f0 con 0x7f341c160570 2024-09-21T07:54:21.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.800+0000 7f3413400640 1 -- 172.21.15.39:0/448863654 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3404035020 con 0x7f341c160570 2024-09-21T07:54:21.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.800+0000 7f3413400640 1 --2- 172.21.15.39:0/448863654 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f33f8078bc0 0x7f33f807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:21.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.801+0000 7f3413400640 1 -- 172.21.15.39:0/448863654 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(253..253 src has 1..253) ==== 8035+0+0 (secure 0 0 0) 0x7f34040bb6d0 con 0x7f341c160570 2024-09-21T07:54:21.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.801+0000 7f341b400640 1 --2- 172.21.15.39:0/448863654 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f33f8078bc0 0x7f33f807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:21.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.802+0000 7f341b400640 1 --2- 172.21.15.39:0/448863654 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f33f8078bc0 0x7f33f807b0d0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f340c009af0 tx=0x7f340c005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:21.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:21.805+0000 7f3413400640 1 -- 172.21.15.39:0/448863654 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3404046030 con 0x7f341c160570 2024-09-21T07:54:22.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.000+0000 7f3421000640 1 -- 172.21.15.39:0/448863654 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f341c0b4850 con 0x7f341c160570 2024-09-21T07:54:22.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.001+0000 7f3413400640 1 -- 172.21.15.39:0/448863654 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v253) ==== 74+0+24048 (secure 0 0 0) 0x7f340403d070 con 0x7f341c160570 2024-09-21T07:54:22.004 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:22.004 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":253,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:54:09.039210+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:49:32.570124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"232","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-09-21T07:49:33.592246+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"243","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":7,"snap_epoch":243,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"29.4","mappings":[{"from":6,"to":0}]},{"pgid":"29.19","mappings":[{"from":6,"to":0}]},{"pgid":"29.1a","mappings":[{"from":6,"to":0}]},{"pgid":"29.1e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/249900061":"2024-09-21T08:54:08.362327+0000","172.21.15.148:6835/2191156088":"2024-09-22T07:53:58.762864+0000","172.21.15.39:6835/485194245":"2024-09-22T07:53:58.201516+0000","172.21.15.39:6833/485194245":"2024-09-22T07:53:58.201516+0000","172.21.15.39:0/4061644382":"2024-09-21T08:53:16.749620+0000","172.21.15.148:6837/597301351":"2024-09-22T07:53:09.120490+0000","172.21.15.148:6834/2191156088":"2024-09-22T07:53:58.762864+0000","172.21.15.148:6835/597301351":"2024-09-22T07:53:09.120490+0000","172.21.15.148:6836/3381818843":"2024-09-22T07:53:08.565426+0000","172.21.15.148:6834/3381818843":"2024-09-22T07:53:08.565426+0000","172.21.15.39:0/106826898":"2024-09-21T08:52:26.149462+0000","172.21.15.39:6834/109229385":"2024-09-22T07:51:36.504781+0000","172.21.15.39:6832/109229385":"2024-09-22T07:51:36.504781+0000","172.21.15.148:6837/1438851059":"2024-09-22T07:51:00.902180+0000","172.21.15.148:6835/1438851059":"2024-09-22T07:51:00.902180+0000","172.21.15.39:6835/2824493631":"2024-09-22T07:50:31.817924+0000","172.21.15.39:6833/2824493631":"2024-09-22T07:50:31.817924+0000","172.21.15.39:6834/4143807732":"2024-09-22T07:50:32.369206+0000","172.21.15.39:6832/4143807732":"2024-09-22T07:50:32.369206+0000","172.21.15.148:6836/3497801035":"2024-09-22T07:50:24.850152+0000","172.21.15.148:6834/3497801035":"2024-09-22T07:50:24.850152+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]},{"pool":20,"snaps":[{"begin":80,"length":23},{"begin":181,"length":23}]},{"pool":22,"snaps":[{"begin":100,"length":1},{"begin":199,"length":1}]},{"pool":24,"snaps":[{"begin":2,"length":2}]},{"pool":26,"snaps":[{"begin":2,"length":2}]},{"pool":28,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:54:22.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.005+0000 7f3421000640 1 -- 172.21.15.39:0/448863654 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f33f8078bc0 msgr2=0x7f33f807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:22.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.005+0000 7f3421000640 1 --2- 172.21.15.39:0/448863654 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f33f8078bc0 0x7f33f807b0d0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f340c009af0 tx=0x7f340c005f90 comp rx=0 tx=0).stop 2024-09-21T07:54:22.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.005+0000 7f3421000640 1 -- 172.21.15.39:0/448863654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f341c160570 msgr2=0x7f341c1f87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:22.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.005+0000 7f3421000640 1 --2- 172.21.15.39:0/448863654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f341c160570 0x7f341c1f87c0 secure :-1 s=READY pgs=2739 cs=0 l=1 rev1=1 crypto rx=0x7f340402b070 tx=0x7f3404001e20 comp rx=0 tx=0).stop 2024-09-21T07:54:22.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.005+0000 7f3421000640 1 -- 172.21.15.39:0/448863654 shutdown_connections 2024-09-21T07:54:22.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.005+0000 7f3421000640 1 --2- 172.21.15.39:0/448863654 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f33f8078bc0 0x7f33f807b0d0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:22.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.005+0000 7f3421000640 1 --2- 172.21.15.39:0/448863654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f341c160ac0 0x7f341c1f8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:22.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.005+0000 7f3421000640 1 --2- 172.21.15.39:0/448863654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f341c160570 0x7f341c1f87c0 unknown :-1 s=CLOSED pgs=2739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:22.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.005+0000 7f3421000640 1 --2- 172.21.15.39:0/448863654 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3400003fd0 0x7f341c074da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:22.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.005+0000 7f3421000640 1 -- 172.21.15.39:0/448863654 >> 172.21.15.39:0/448863654 conn(0x7f341c0b2510 msgr2=0x7f341c15e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:22.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.005+0000 7f3421000640 1 -- 172.21.15.39:0/448863654 shutdown_connections 2024-09-21T07:54:22.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.005+0000 7f3421000640 1 -- 172.21.15.39:0/448863654 wait complete. 2024-09-21T07:54:22.029 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T07:54:22.029 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:54:22.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.265+0000 7fc92ee00640 1 -- 172.21.15.39:0/2167926228 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc92815c520 msgr2=0x7fc9280b3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:22.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.265+0000 7fc92ee00640 1 --2- 172.21.15.39:0/2167926228 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc92815c520 0x7fc9280b3ae0 secure :-1 s=READY pgs=2740 cs=0 l=1 rev1=1 crypto rx=0x7fc9280763c0 tx=0x7fc91c02a5f0 comp rx=0 tx=0).stop 2024-09-21T07:54:22.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.265+0000 7fc92ee00640 1 -- 172.21.15.39:0/2167926228 shutdown_connections 2024-09-21T07:54:22.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.265+0000 7fc92ee00640 1 --2- 172.21.15.39:0/2167926228 >> v2:172.21.15.148:3300/0 conn(0x7fc9280cf1e0 0x7fc9280b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:22.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.265+0000 7fc92ee00640 1 --2- 172.21.15.39:0/2167926228 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc92815c520 0x7fc9280b3ae0 unknown :-1 s=CLOSED pgs=2740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:22.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.265+0000 7fc92ee00640 1 -- 172.21.15.39:0/2167926228 >> 172.21.15.39:0/2167926228 conn(0x7fc928074050 msgr2=0x7fc928074490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:22.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.265+0000 7fc92ee00640 1 -- 172.21.15.39:0/2167926228 shutdown_connections 2024-09-21T07:54:22.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.265+0000 7fc92ee00640 1 -- 172.21.15.39:0/2167926228 wait complete. 2024-09-21T07:54:22.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.267+0000 7fc92ee00640 1 Processor -- start 2024-09-21T07:54:22.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.268+0000 7fc92ee00640 1 -- start start 2024-09-21T07:54:22.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.269+0000 7fc92ee00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc9280b4020 0x7fc9281f66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:22.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.269+0000 7fc92ee00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9280cf1e0 0x7fc928146250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:22.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.269+0000 7fc92ee00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc92815c520 0x7fc9281f90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:22.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.269+0000 7fc92ee00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc9281609e0 con 0x7fc9280cf1e0 2024-09-21T07:54:22.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.269+0000 7fc92ee00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc928160860 con 0x7fc92815c520 2024-09-21T07:54:22.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.269+0000 7fc92ee00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc928160b60 con 0x7fc9280b4020 2024-09-21T07:54:22.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.269+0000 7fc92d000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9280cf1e0 0x7fc928146250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:22.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.269+0000 7fc92d000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9280cf1e0 0x7fc928146250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40974/0 (socket says 172.21.15.39:40974) 2024-09-21T07:54:22.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.269+0000 7fc92d000640 1 -- 172.21.15.39:0/1436501707 learned_addr learned my addr 172.21.15.39:0/1436501707 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:22.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.269+0000 7fc92da00640 1 --2- 172.21.15.39:0/1436501707 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc9280b4020 0x7fc9281f66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:22.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.270+0000 7fc92e400640 1 --2- 172.21.15.39:0/1436501707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc92815c520 0x7fc9281f90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:22.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.270+0000 7fc92d000640 1 -- 172.21.15.39:0/1436501707 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc9280b4020 msgr2=0x7fc9281f66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:22.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.270+0000 7fc92d000640 1 --2- 172.21.15.39:0/1436501707 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc9280b4020 0x7fc9281f66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:22.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.270+0000 7fc92d000640 1 -- 172.21.15.39:0/1436501707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc92815c520 msgr2=0x7fc9281f90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:22.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.270+0000 7fc92d000640 1 --2- 172.21.15.39:0/1436501707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc92815c520 0x7fc9281f90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:22.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.270+0000 7fc92d000640 1 -- 172.21.15.39:0/1436501707 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9281f9c90 con 0x7fc9280cf1e0 2024-09-21T07:54:22.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.270+0000 7fc92d000640 1 --2- 172.21.15.39:0/1436501707 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9280cf1e0 0x7fc928146250 secure :-1 s=READY pgs=2741 cs=0 l=1 rev1=1 crypto rx=0x7fc91400b3f0 tx=0x7fc91400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:22.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.271+0000 7fc925600640 1 -- 172.21.15.39:0/1436501707 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc914015020 con 0x7fc9280cf1e0 2024-09-21T07:54:22.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.271+0000 7fc925600640 1 -- 172.21.15.39:0/1436501707 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc914011920 con 0x7fc9280cf1e0 2024-09-21T07:54:22.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.271+0000 7fc925600640 1 -- 172.21.15.39:0/1436501707 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc914011d70 con 0x7fc9280cf1e0 2024-09-21T07:54:22.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.271+0000 7fc92ee00640 1 -- 172.21.15.39:0/1436501707 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9281f9f70 con 0x7fc9280cf1e0 2024-09-21T07:54:22.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.271+0000 7fc92ee00640 1 -- 172.21.15.39:0/1436501707 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9281f5eb0 con 0x7fc9280cf1e0 2024-09-21T07:54:22.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.272+0000 7fc92ee00640 1 -- 172.21.15.39:0/1436501707 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9281f9a00 con 0x7fc9280cf1e0 2024-09-21T07:54:22.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.273+0000 7fc925600640 1 -- 172.21.15.39:0/1436501707 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc914018020 con 0x7fc9280cf1e0 2024-09-21T07:54:22.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.274+0000 7fc925600640 1 --2- 172.21.15.39:0/1436501707 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc908078bc0 0x7fc90807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:22.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.274+0000 7fc925600640 1 -- 172.21.15.39:0/1436501707 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(254..254 src has 1..254) ==== 8091+0+0 (secure 0 0 0) 0x7fc91409b200 con 0x7fc9280cf1e0 2024-09-21T07:54:22.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.274+0000 7fc92da00640 1 --2- 172.21.15.39:0/1436501707 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc908078bc0 0x7fc90807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:22.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.275+0000 7fc92da00640 1 --2- 172.21.15.39:0/1436501707 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc908078bc0 0x7fc90807b0d0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fc91c033000 tx=0x7fc91c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:22.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.278+0000 7fc925600640 1 -- 172.21.15.39:0/1436501707 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc91401f030 con 0x7fc9280cf1e0 2024-09-21T07:54:22.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.525+0000 7fc92ee00640 1 -- 172.21.15.39:0/1436501707 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc92815f9e0 con 0x7fc9280cf1e0 2024-09-21T07:54:22.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.526+0000 7fc925600640 1 -- 172.21.15.39:0/1436501707 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 301 v301) ==== 95+0+5291 (secure 0 0 0) 0x7fc914062690 con 0x7fc9280cf1e0 2024-09-21T07:54:22.528 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:22.528 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":301,"btime":"2024-09-21T07:54:11:059441+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9467,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3658109005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3658109005},{"type":"v1","addr":"172.21.15.148:6835","nonce":3658109005}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":298}],"filesystems":[{"mdsmap":{"epoch":301,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:54:11.059432+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":252,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":11364,"mds_1":10633,"mds_2":11970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10633":{"gid":10633,"name":"d","rank":1,"incarnation":295,"state":"up:active","state_seq":19,"addr":"172.21.15.148:6837/3495264983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3495264983},{"type":"v1","addr":"172.21.15.148:6837","nonce":3495264983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11364":{"gid":11364,"name":"a","rank":0,"incarnation":285,"state":"up:active","state_seq":28,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11970":{"gid":11970,"name":"c","rank":2,"incarnation":297,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6835/732408880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":732408880},{"type":"v1","addr":"172.21.15.39:6835","nonce":732408880}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11364,"qdb_cluster":[10633,11364,11970]},"id":13}]} 2024-09-21T07:54:22.528 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 301 2024-09-21T07:54:22.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.529+0000 7fc92ee00640 1 -- 172.21.15.39:0/1436501707 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc908078bc0 msgr2=0x7fc90807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:22.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.529+0000 7fc92ee00640 1 --2- 172.21.15.39:0/1436501707 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc908078bc0 0x7fc90807b0d0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fc91c033000 tx=0x7fc91c004730 comp rx=0 tx=0).stop 2024-09-21T07:54:22.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.529+0000 7fc92ee00640 1 -- 172.21.15.39:0/1436501707 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9280cf1e0 msgr2=0x7fc928146250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:22.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.530+0000 7fc92ee00640 1 --2- 172.21.15.39:0/1436501707 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9280cf1e0 0x7fc928146250 secure :-1 s=READY pgs=2741 cs=0 l=1 rev1=1 crypto rx=0x7fc91400b3f0 tx=0x7fc91400b8b0 comp rx=0 tx=0).stop 2024-09-21T07:54:22.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.530+0000 7fc92ee00640 1 -- 172.21.15.39:0/1436501707 shutdown_connections 2024-09-21T07:54:22.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.530+0000 7fc92ee00640 1 --2- 172.21.15.39:0/1436501707 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc908078bc0 0x7fc90807b0d0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:22.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.530+0000 7fc92ee00640 1 --2- 172.21.15.39:0/1436501707 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc92815c520 0x7fc9281f90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:22.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.530+0000 7fc92ee00640 1 --2- 172.21.15.39:0/1436501707 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9280cf1e0 0x7fc928146250 unknown :-1 s=CLOSED pgs=2741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:22.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.530+0000 7fc92ee00640 1 --2- 172.21.15.39:0/1436501707 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc9280b4020 0x7fc9281f66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:22.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.530+0000 7fc92ee00640 1 -- 172.21.15.39:0/1436501707 >> 172.21.15.39:0/1436501707 conn(0x7fc928074050 msgr2=0x7fc928069210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:22.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.530+0000 7fc92ee00640 1 -- 172.21.15.39:0/1436501707 shutdown_connections 2024-09-21T07:54:22.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.530+0000 7fc92ee00640 1 -- 172.21.15.39:0/1436501707 wait complete. 2024-09-21T07:54:22.557 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:54:22.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.784+0000 7fd21dc00640 1 -- 172.21.15.39:0/3197917916 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1fc004db0 msgr2=0x7fd1fc005280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:22.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.784+0000 7fd21dc00640 1 --2- 172.21.15.39:0/3197917916 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1fc004db0 0x7fd1fc005280 secure :-1 s=READY pgs=2743 cs=0 l=1 rev1=1 crypto rx=0x7fd21000e3d0 tx=0x7fd1fc005830 comp rx=0 tx=0).stop 2024-09-21T07:54:22.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.784+0000 7fd217400640 1 -- 172.21.15.39:0/3197917916 reap_dead start 2024-09-21T07:54:22.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.784+0000 7fd21dc00640 1 -- 172.21.15.39:0/3197917916 shutdown_connections 2024-09-21T07:54:22.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.784+0000 7fd21dc00640 1 -- 172.21.15.39:0/3197917916 >> 172.21.15.39:0/3197917916 conn(0x7fd21806cfc0 msgr2=0x7fd21806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:22.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.785+0000 7fd21dc00640 1 -- 172.21.15.39:0/3197917916 shutdown_connections 2024-09-21T07:54:22.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.785+0000 7fd21dc00640 1 -- 172.21.15.39:0/3197917916 wait complete. 2024-09-21T07:54:22.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.787+0000 7fd21dc00640 1 Processor -- start 2024-09-21T07:54:22.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.787+0000 7fd21dc00640 1 -- start start 2024-09-21T07:54:22.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.788+0000 7fd21dc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1fc004db0 0x7fd2181e9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:22.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.788+0000 7fd21dc00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd21815ea30 0x7fd2181e6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:22.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.788+0000 7fd21dc00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd2181e73f0 0x7fd2180b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:22.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.788+0000 7fd21dc00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd2180b4e30 con 0x7fd1fc004db0 2024-09-21T07:54:22.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.788+0000 7fd21dc00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd2180b4cb0 con 0x7fd21815ea30 2024-09-21T07:54:22.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.788+0000 7fd21dc00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd2180b4260 con 0x7fd2181e73f0 2024-09-21T07:54:22.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.789+0000 7fd216a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1fc004db0 0x7fd2181e9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:22.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.789+0000 7fd216a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1fc004db0 0x7fd2181e9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41002/0 (socket says 172.21.15.39:41002) 2024-09-21T07:54:22.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.789+0000 7fd216a00640 1 -- 172.21.15.39:0/3637782967 learned_addr learned my addr 172.21.15.39:0/3637782967 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:22.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.789+0000 7fd217400640 1 --2- 172.21.15.39:0/3637782967 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd2181e73f0 0x7fd2180b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:22.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.789+0000 7fd216000640 1 --2- 172.21.15.39:0/3637782967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd21815ea30 0x7fd2181e6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:22.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.789+0000 7fd216a00640 1 -- 172.21.15.39:0/3637782967 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd2181e73f0 msgr2=0x7fd2180b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:22.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.789+0000 7fd216a00640 1 --2- 172.21.15.39:0/3637782967 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd2181e73f0 0x7fd2180b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:22.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.789+0000 7fd216a00640 1 -- 172.21.15.39:0/3637782967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd21815ea30 msgr2=0x7fd2181e6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:22.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.789+0000 7fd216a00640 1 --2- 172.21.15.39:0/3637782967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd21815ea30 0x7fd2181e6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:22.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.789+0000 7fd216a00640 1 -- 172.21.15.39:0/3637782967 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2181e7800 con 0x7fd1fc004db0 2024-09-21T07:54:22.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.789+0000 7fd216a00640 1 --2- 172.21.15.39:0/3637782967 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1fc004db0 0x7fd2181e9a70 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 crypto rx=0x7fd2080066b0 tx=0x7fd2080039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:22.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.790+0000 7fd20ea00640 1 -- 172.21.15.39:0/3637782967 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd208015070 con 0x7fd1fc004db0 2024-09-21T07:54:22.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.790+0000 7fd20ea00640 1 -- 172.21.15.39:0/3637782967 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd208011900 con 0x7fd1fc004db0 2024-09-21T07:54:22.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.790+0000 7fd20ea00640 1 -- 172.21.15.39:0/3637782967 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd208011d10 con 0x7fd1fc004db0 2024-09-21T07:54:22.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.790+0000 7fd21dc00640 1 -- 172.21.15.39:0/3637782967 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2181e8fa0 con 0x7fd1fc004db0 2024-09-21T07:54:22.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.790+0000 7fd21dc00640 1 -- 172.21.15.39:0/3637782967 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2181f91a0 con 0x7fd1fc004db0 2024-09-21T07:54:22.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.791+0000 7fd21dc00640 1 -- 172.21.15.39:0/3637782967 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2181e9170 con 0x7fd1fc004db0 2024-09-21T07:54:22.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.792+0000 7fd20ea00640 1 -- 172.21.15.39:0/3637782967 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd208018070 con 0x7fd1fc004db0 2024-09-21T07:54:22.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.793+0000 7fd20ea00640 1 --2- 172.21.15.39:0/3637782967 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd1f8078bc0 0x7fd1f807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:22.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.793+0000 7fd20ea00640 1 -- 172.21.15.39:0/3637782967 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(254..254 src has 1..254) ==== 8091+0+0 (secure 0 0 0) 0x7fd20809c3d0 con 0x7fd1fc004db0 2024-09-21T07:54:22.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.794+0000 7fd216000640 1 --2- 172.21.15.39:0/3637782967 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd1f8078bc0 0x7fd1f807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:22.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.795+0000 7fd216000640 1 --2- 172.21.15.39:0/3637782967 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd1f8078bc0 0x7fd1f807b0d0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fd2181e6b50 tx=0x7fd204006b10 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:22.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:22.798+0000 7fd20ea00640 1 -- 172.21.15.39:0/3637782967 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd2181e7800 con 0x7fd1fc004db0 2024-09-21T07:54:23.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.035+0000 7fd21dc00640 1 -- 172.21.15.39:0/3637782967 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd2180b39f0 con 0x7fd1fc004db0 2024-09-21T07:54:23.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.036+0000 7fd20ea00640 1 -- 172.21.15.39:0/3637782967 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 301 v301) ==== 95+0+5291 (secure 0 0 0) 0x7fd208063ed0 con 0x7fd1fc004db0 2024-09-21T07:54:23.038 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:23.038 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":301,"btime":"2024-09-21T07:54:11:059441+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9467,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3658109005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3658109005},{"type":"v1","addr":"172.21.15.148:6835","nonce":3658109005}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":298}],"filesystems":[{"mdsmap":{"epoch":301,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-09-21T07:49:34.636616+0000","modified":"2024-09-21T07:54:11.059432+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":252,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":11364,"mds_1":10633,"mds_2":11970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10633":{"gid":10633,"name":"d","rank":1,"incarnation":295,"state":"up:active","state_seq":19,"addr":"172.21.15.148:6837/3495264983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3495264983},{"type":"v1","addr":"172.21.15.148:6837","nonce":3495264983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11364":{"gid":11364,"name":"a","rank":0,"incarnation":285,"state":"up:active","state_seq":28,"addr":"172.21.15.39:6834/2161818466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2161818466},{"type":"v1","addr":"172.21.15.39:6834","nonce":2161818466}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_11970":{"gid":11970,"name":"c","rank":2,"incarnation":297,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6835/732408880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":732408880},{"type":"v1","addr":"172.21.15.39:6835","nonce":732408880}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":11364,"qdb_cluster":[10633,11364,11970]},"id":13}]} 2024-09-21T07:54:23.038 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 301 2024-09-21T07:54:23.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.038+0000 7fd21dc00640 1 -- 172.21.15.39:0/3637782967 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd1f8078bc0 msgr2=0x7fd1f807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:23.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.038+0000 7fd21dc00640 1 --2- 172.21.15.39:0/3637782967 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd1f8078bc0 0x7fd1f807b0d0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fd2181e6b50 tx=0x7fd204006b10 comp rx=0 tx=0).stop 2024-09-21T07:54:23.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.038+0000 7fd21dc00640 1 -- 172.21.15.39:0/3637782967 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1fc004db0 msgr2=0x7fd2181e9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:23.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.038+0000 7fd21dc00640 1 --2- 172.21.15.39:0/3637782967 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1fc004db0 0x7fd2181e9a70 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 crypto rx=0x7fd2080066b0 tx=0x7fd2080039c0 comp rx=0 tx=0).stop 2024-09-21T07:54:23.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.038+0000 7fd21dc00640 1 -- 172.21.15.39:0/3637782967 shutdown_connections 2024-09-21T07:54:23.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.038+0000 7fd21dc00640 1 --2- 172.21.15.39:0/3637782967 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd1f8078bc0 0x7fd1f807b0d0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:23.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.038+0000 7fd21dc00640 1 --2- 172.21.15.39:0/3637782967 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd2181e73f0 0x7fd2180b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:23.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.038+0000 7fd21dc00640 1 --2- 172.21.15.39:0/3637782967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd21815ea30 0x7fd2181e6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:23.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.038+0000 7fd21dc00640 1 --2- 172.21.15.39:0/3637782967 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1fc004db0 0x7fd2181e9a70 unknown :-1 s=CLOSED pgs=2744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:23.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.038+0000 7fd21dc00640 1 -- 172.21.15.39:0/3637782967 >> 172.21.15.39:0/3637782967 conn(0x7fd21806cfc0 msgr2=0x7fd2180cf110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:23.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.039+0000 7fd21dc00640 1 -- 172.21.15.39:0/3637782967 shutdown_connections 2024-09-21T07:54:23.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.039+0000 7fd21dc00640 1 -- 172.21.15.39:0/3637782967 wait complete. 2024-09-21T07:54:23.067 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:54:23.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.296+0000 7f5a23e00640 1 -- 172.21.15.39:0/727610826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a2415fc30 msgr2=0x7f5a24160320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:23.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.296+0000 7f5a23e00640 1 --2- 172.21.15.39:0/727610826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a2415fc30 0x7f5a24160320 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f5a14005fd0 tx=0x7f5a1402a540 comp rx=0 tx=0).stop 2024-09-21T07:54:23.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.296+0000 7f5a23e00640 1 -- 172.21.15.39:0/727610826 shutdown_connections 2024-09-21T07:54:23.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.296+0000 7f5a23e00640 1 --2- 172.21.15.39:0/727610826 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5a240cc0c0 0x7f5a2406f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:23.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.296+0000 7f5a23e00640 1 --2- 172.21.15.39:0/727610826 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a2415fc30 0x7f5a24160320 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:23.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.296+0000 7f5a23e00640 1 -- 172.21.15.39:0/727610826 >> 172.21.15.39:0/727610826 conn(0x7f5a240b2510 msgr2=0x7f5a240d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:23.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.297+0000 7f5a23e00640 1 -- 172.21.15.39:0/727610826 shutdown_connections 2024-09-21T07:54:23.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.297+0000 7f5a23e00640 1 -- 172.21.15.39:0/727610826 wait complete. 2024-09-21T07:54:23.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.299+0000 7f5a23e00640 1 Processor -- start 2024-09-21T07:54:23.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.299+0000 7f5a23e00640 1 -- start start 2024-09-21T07:54:23.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.300+0000 7f5a23e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a240cc0c0 0x7f5a241ef930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:23.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.300+0000 7f5a23e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a2415fc30 0x7f5a2415e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:23.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.300+0000 7f5a23e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5a24160860 0x7f5a2415bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:23.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.300+0000 7f5a23e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2406f750 con 0x7f5a2415fc30 2024-09-21T07:54:23.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.300+0000 7f5a23e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5a240cd770 con 0x7f5a240cc0c0 2024-09-21T07:54:23.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.300+0000 7f5a23e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f5a2406f8d0 con 0x7f5a24160860 2024-09-21T07:54:23.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.301+0000 7f5a1be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a2415fc30 0x7f5a2415e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:23.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.301+0000 7f5a1be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a2415fc30 0x7f5a2415e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41008/0 (socket says 172.21.15.39:41008) 2024-09-21T07:54:23.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.301+0000 7f5a1be00640 1 -- 172.21.15.39:0/1695341857 learned_addr learned my addr 172.21.15.39:0/1695341857 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:23.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.301+0000 7f5a23400640 1 --2- 172.21.15.39:0/1695341857 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5a24160860 0x7f5a2415bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:23.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.301+0000 7f5a22a00640 1 --2- 172.21.15.39:0/1695341857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a240cc0c0 0x7f5a241ef930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:23.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.301+0000 7f5a1be00640 1 -- 172.21.15.39:0/1695341857 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5a24160860 msgr2=0x7f5a2415bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:23.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.301+0000 7f5a1be00640 1 --2- 172.21.15.39:0/1695341857 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5a24160860 0x7f5a2415bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:23.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.301+0000 7f5a1be00640 1 -- 172.21.15.39:0/1695341857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a240cc0c0 msgr2=0x7f5a241ef930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:23.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.301+0000 7f5a1be00640 1 --2- 172.21.15.39:0/1695341857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a240cc0c0 0x7f5a241ef930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:23.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.301+0000 7f5a1be00640 1 -- 172.21.15.39:0/1695341857 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a2415e690 con 0x7f5a2415fc30 2024-09-21T07:54:23.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.302+0000 7f5a1be00640 1 --2- 172.21.15.39:0/1695341857 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a2415fc30 0x7f5a2415e150 secure :-1 s=READY pgs=2745 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c00c370 tx=0x7f5a0c00c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:23.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.302+0000 7f5a1b400640 1 -- 172.21.15.39:0/1695341857 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c017070 con 0x7f5a2415fc30 2024-09-21T07:54:23.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.302+0000 7f5a1b400640 1 -- 172.21.15.39:0/1695341857 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a0c0053c0 con 0x7f5a2415fc30 2024-09-21T07:54:23.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.302+0000 7f5a23e00640 1 -- 172.21.15.39:0/1695341857 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a2415a540 con 0x7f5a2415fc30 2024-09-21T07:54:23.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.302+0000 7f5a1b400640 1 -- 172.21.15.39:0/1695341857 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c003850 con 0x7f5a2415fc30 2024-09-21T07:54:23.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.302+0000 7f5a23e00640 1 -- 172.21.15.39:0/1695341857 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a240ce3f0 con 0x7f5a2415fc30 2024-09-21T07:54:23.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.303+0000 7f5a23e00640 1 -- 172.21.15.39:0/1695341857 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a2406fdf0 con 0x7f5a2415fc30 2024-09-21T07:54:23.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.304+0000 7f5a1b400640 1 -- 172.21.15.39:0/1695341857 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5a0c01a070 con 0x7f5a2415fc30 2024-09-21T07:54:23.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.306+0000 7f5a1b400640 1 --2- 172.21.15.39:0/1695341857 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5a00078b90 0x7f5a0007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:23.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.306+0000 7f5a1b400640 1 -- 172.21.15.39:0/1695341857 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(254..254 src has 1..254) ==== 8091+0+0 (secure 0 0 0) 0x7f5a0c09a930 con 0x7f5a2415fc30 2024-09-21T07:54:23.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.306+0000 7f5a22a00640 1 --2- 172.21.15.39:0/1695341857 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5a00078b90 0x7f5a0007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:23.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.307+0000 7f5a22a00640 1 --2- 172.21.15.39:0/1695341857 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5a00078b90 0x7f5a0007b0a0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f5a14033000 tx=0x7f5a14004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:23.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.311+0000 7f5a1b400640 1 -- 172.21.15.39:0/1695341857 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5a24160c70 con 0x7f5a2415fc30 2024-09-21T07:54:23.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.512+0000 7f5a23e00640 1 -- 172.21.15.39:0/1695341857 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5a240d1080 con 0x7f5a2415fc30 2024-09-21T07:54:23.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.514+0000 7f5a1b400640 1 -- 172.21.15.39:0/1695341857 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v254) ==== 74+0+24138 (secure 0 0 0) 0x7f5a0c061d30 con 0x7f5a2415fc30 2024-09-21T07:54:23.516 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:23.516 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":254,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:54:22.125934+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:49:32.570124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"232","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-09-21T07:49:33.592246+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"254","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":13,"snap_epoch":254,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"29.4","mappings":[{"from":6,"to":0}]},{"pgid":"29.19","mappings":[{"from":6,"to":0}]},{"pgid":"29.1a","mappings":[{"from":6,"to":0}]},{"pgid":"29.1e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:0/249900061":"2024-09-21T08:54:08.362327+0000","172.21.15.148:6835/2191156088":"2024-09-22T07:53:58.762864+0000","172.21.15.39:6835/485194245":"2024-09-22T07:53:58.201516+0000","172.21.15.39:6833/485194245":"2024-09-22T07:53:58.201516+0000","172.21.15.39:0/4061644382":"2024-09-21T08:53:16.749620+0000","172.21.15.148:6837/597301351":"2024-09-22T07:53:09.120490+0000","172.21.15.148:6834/2191156088":"2024-09-22T07:53:58.762864+0000","172.21.15.148:6835/597301351":"2024-09-22T07:53:09.120490+0000","172.21.15.148:6836/3381818843":"2024-09-22T07:53:08.565426+0000","172.21.15.148:6834/3381818843":"2024-09-22T07:53:08.565426+0000","172.21.15.39:0/106826898":"2024-09-21T08:52:26.149462+0000","172.21.15.39:6834/109229385":"2024-09-22T07:51:36.504781+0000","172.21.15.39:6832/109229385":"2024-09-22T07:51:36.504781+0000","172.21.15.148:6837/1438851059":"2024-09-22T07:51:00.902180+0000","172.21.15.148:6835/1438851059":"2024-09-22T07:51:00.902180+0000","172.21.15.39:6835/2824493631":"2024-09-22T07:50:31.817924+0000","172.21.15.39:6833/2824493631":"2024-09-22T07:50:31.817924+0000","172.21.15.39:6834/4143807732":"2024-09-22T07:50:32.369206+0000","172.21.15.39:6832/4143807732":"2024-09-22T07:50:32.369206+0000","172.21.15.148:6836/3497801035":"2024-09-22T07:50:24.850152+0000","172.21.15.148:6834/3497801035":"2024-09-22T07:50:24.850152+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]},{"pool":20,"snaps":[{"begin":80,"length":23},{"begin":181,"length":23}]},{"pool":22,"snaps":[{"begin":100,"length":1},{"begin":199,"length":1}]},{"pool":24,"snaps":[{"begin":2,"length":2}]},{"pool":26,"snaps":[{"begin":2,"length":2}]},{"pool":28,"snaps":[{"begin":2,"length":2}]},{"pool":30,"snaps":[{"begin":8,"length":6}]}],"new_removed_snaps":[{"pool":30,"snaps":[{"begin":8,"length":6}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:54:23.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.516+0000 7f5a23e00640 1 -- 172.21.15.39:0/1695341857 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5a00078b90 msgr2=0x7f5a0007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.516+0000 7f5a23e00640 1 --2- 172.21.15.39:0/1695341857 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5a00078b90 0x7f5a0007b0a0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f5a14033000 tx=0x7f5a14004730 comp rx=0 tx=0).stop 2024-09-21T07:54:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.516+0000 7f5a23e00640 1 -- 172.21.15.39:0/1695341857 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a2415fc30 msgr2=0x7f5a2415e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.516+0000 7f5a23e00640 1 --2- 172.21.15.39:0/1695341857 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a2415fc30 0x7f5a2415e150 secure :-1 s=READY pgs=2745 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c00c370 tx=0x7f5a0c00c770 comp rx=0 tx=0).stop 2024-09-21T07:54:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.516+0000 7f5a23e00640 1 -- 172.21.15.39:0/1695341857 shutdown_connections 2024-09-21T07:54:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.516+0000 7f5a23e00640 1 --2- 172.21.15.39:0/1695341857 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5a00078b90 0x7f5a0007b0a0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.516+0000 7f5a23e00640 1 --2- 172.21.15.39:0/1695341857 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f5a24160860 0x7f5a2415bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.516+0000 7f5a23e00640 1 --2- 172.21.15.39:0/1695341857 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a2415fc30 0x7f5a2415e150 unknown :-1 s=CLOSED pgs=2745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.516+0000 7f5a23e00640 1 --2- 172.21.15.39:0/1695341857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a240cc0c0 0x7f5a241ef930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.516+0000 7f5a23e00640 1 -- 172.21.15.39:0/1695341857 >> 172.21.15.39:0/1695341857 conn(0x7f5a240b2510 msgr2=0x7f5a24072f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.517+0000 7f5a23e00640 1 -- 172.21.15.39:0/1695341857 shutdown_connections 2024-09-21T07:54:23.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.517+0000 7f5a23e00640 1 -- 172.21.15.39:0/1695341857 wait complete. 2024-09-21T07:54:23.541 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T07:54:23.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.766+0000 7fe1f7e00640 1 -- 172.21.15.39:0/3170829617 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1f80d0f20 msgr2=0x7fe1f815c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:23.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.766+0000 7fe1f7e00640 1 --2- 172.21.15.39:0/3170829617 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1f80d0f20 0x7fe1f815c830 secure :-1 s=READY pgs=2747 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8005fd0 tx=0x7fe1e802a670 comp rx=0 tx=0).stop 2024-09-21T07:54:23.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.766+0000 7fe1f7e00640 1 -- 172.21.15.39:0/3170829617 shutdown_connections 2024-09-21T07:54:23.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.766+0000 7fe1f7e00640 1 --2- 172.21.15.39:0/3170829617 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1f80d0f20 0x7fe1f815c830 unknown :-1 s=CLOSED pgs=2747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:23.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.766+0000 7fe1f7e00640 1 -- 172.21.15.39:0/3170829617 >> 172.21.15.39:0/3170829617 conn(0x7fe1f80b2510 msgr2=0x7fe1f80d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:23.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.767+0000 7fe1f7e00640 1 -- 172.21.15.39:0/3170829617 shutdown_connections 2024-09-21T07:54:23.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.767+0000 7fe1f7e00640 1 -- 172.21.15.39:0/3170829617 wait complete. 2024-09-21T07:54:23.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.769+0000 7fe1f7e00640 1 Processor -- start 2024-09-21T07:54:23.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.769+0000 7fe1f7e00640 1 -- start start 2024-09-21T07:54:23.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.770+0000 7fe1f7e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1f80cb960 0x7fe1f81f5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:23.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.770+0000 7fe1f7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1f80cbd70 0x7fe1f8074260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:23.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.770+0000 7fe1f7e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1f80d0f20 0x7fe1f80747a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:23.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.770+0000 7fe1f7e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f80b4780 con 0x7fe1f80cbd70 2024-09-21T07:54:23.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.770+0000 7fe1f7e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f80b4600 con 0x7fe1f80cb960 2024-09-21T07:54:23.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.770+0000 7fe1f7e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe1f80b4900 con 0x7fe1f80d0f20 2024-09-21T07:54:23.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.771+0000 7fe1f6000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1f80cbd70 0x7fe1f8074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:23.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.771+0000 7fe1f6000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1f80cbd70 0x7fe1f8074260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41028/0 (socket says 172.21.15.39:41028) 2024-09-21T07:54:23.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.771+0000 7fe1f6000640 1 -- 172.21.15.39:0/1815067951 learned_addr learned my addr 172.21.15.39:0/1815067951 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:23.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.771+0000 7fe1f6a00640 1 --2- 172.21.15.39:0/1815067951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1f80cb960 0x7fe1f81f5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:23.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.771+0000 7fe1f7400640 1 --2- 172.21.15.39:0/1815067951 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1f80d0f20 0x7fe1f80747a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:23.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.771+0000 7fe1f6000640 1 -- 172.21.15.39:0/1815067951 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1f80d0f20 msgr2=0x7fe1f80747a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:23.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.771+0000 7fe1f6000640 1 --2- 172.21.15.39:0/1815067951 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1f80d0f20 0x7fe1f80747a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:23.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.771+0000 7fe1f6000640 1 -- 172.21.15.39:0/1815067951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1f80cb960 msgr2=0x7fe1f81f5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:23.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.771+0000 7fe1f6000640 1 --2- 172.21.15.39:0/1815067951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1f80cb960 0x7fe1f81f5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:23.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.771+0000 7fe1f6000640 1 -- 172.21.15.39:0/1815067951 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f80cfe40 con 0x7fe1f80cbd70 2024-09-21T07:54:23.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.771+0000 7fe1f6000640 1 --2- 172.21.15.39:0/1815067951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1f80cbd70 0x7fe1f8074260 secure :-1 s=READY pgs=2748 cs=0 l=1 rev1=1 crypto rx=0x7fe1e400c3a0 tx=0x7fe1e400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:23.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.772+0000 7fe1eea00640 1 -- 172.21.15.39:0/1815067951 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4017070 con 0x7fe1f80cbd70 2024-09-21T07:54:23.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.772+0000 7fe1eea00640 1 -- 172.21.15.39:0/1815067951 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1e40053c0 con 0x7fe1f80cbd70 2024-09-21T07:54:23.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.772+0000 7fe1eea00640 1 -- 172.21.15.39:0/1815067951 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4003850 con 0x7fe1f80cbd70 2024-09-21T07:54:23.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.772+0000 7fe1f7e00640 1 -- 172.21.15.39:0/1815067951 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1f80cdf70 con 0x7fe1f80cbd70 2024-09-21T07:54:23.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.772+0000 7fe1f7e00640 1 -- 172.21.15.39:0/1815067951 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1f80ce530 con 0x7fe1f80cbd70 2024-09-21T07:54:23.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.773+0000 7fe1f7e00640 1 -- 172.21.15.39:0/1815067951 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1f80cfbb0 con 0x7fe1f80cbd70 2024-09-21T07:54:23.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.774+0000 7fe1eea00640 1 -- 172.21.15.39:0/1815067951 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe1e401a070 con 0x7fe1f80cbd70 2024-09-21T07:54:23.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.775+0000 7fe1eea00640 1 --2- 172.21.15.39:0/1815067951 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe1d8078b90 0x7fe1d807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:23.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.775+0000 7fe1eea00640 1 -- 172.21.15.39:0/1815067951 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(254..254 src has 1..254) ==== 8091+0+0 (secure 0 0 0) 0x7fe1e409aa70 con 0x7fe1f80cbd70 2024-09-21T07:54:23.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.776+0000 7fe1f6a00640 1 --2- 172.21.15.39:0/1815067951 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe1d8078b90 0x7fe1d807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:23.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.776+0000 7fe1f6a00640 1 --2- 172.21.15.39:0/1815067951 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe1d8078b90 0x7fe1d807b0a0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8033000 tx=0x7fe1e8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:23.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:23.779+0000 7fe1eea00640 1 -- 172.21.15.39:0/1815067951 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe1e4062550 con 0x7fe1f80cbd70 2024-09-21T07:54:24.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.019+0000 7fe1f7e00640 1 -- 172.21.15.39:0/1815067951 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe1f815c830 con 0x7fe1f80cbd70 2024-09-21T07:54:24.022 INFO:tasks.ceph.mon.a.smithi039.stderr:2024-09-21T07:54:24.020+0000 7fabab600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T07:54:24.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.023+0000 7fe1eea00640 1 -- 172.21.15.39:0/1815067951 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v302) ==== 161+0+0 (secure 0 0 0) 0x7fe1e4061e50 con 0x7fe1f80cbd70 2024-09-21T07:54:24.025 INFO:teuthology.orchestra.run.smithi039.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T07:54:24.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.027+0000 7fe1f7e00640 1 -- 172.21.15.39:0/1815067951 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe1d8078b90 msgr2=0x7fe1d807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:24.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.027+0000 7fe1f7e00640 1 --2- 172.21.15.39:0/1815067951 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe1d8078b90 0x7fe1d807b0a0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8033000 tx=0x7fe1e8004730 comp rx=0 tx=0).stop 2024-09-21T07:54:24.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.027+0000 7fe1f7e00640 1 -- 172.21.15.39:0/1815067951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1f80cbd70 msgr2=0x7fe1f8074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:24.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.027+0000 7fe1f7e00640 1 --2- 172.21.15.39:0/1815067951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1f80cbd70 0x7fe1f8074260 secure :-1 s=READY pgs=2748 cs=0 l=1 rev1=1 crypto rx=0x7fe1e400c3a0 tx=0x7fe1e400c860 comp rx=0 tx=0).stop 2024-09-21T07:54:24.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.028+0000 7fe1f7e00640 1 -- 172.21.15.39:0/1815067951 shutdown_connections 2024-09-21T07:54:24.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.028+0000 7fe1f7e00640 1 --2- 172.21.15.39:0/1815067951 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe1d8078b90 0x7fe1d807b0a0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:24.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.028+0000 7fe1f7e00640 1 --2- 172.21.15.39:0/1815067951 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe1f80d0f20 0x7fe1f80747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:24.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.028+0000 7fe1f7e00640 1 --2- 172.21.15.39:0/1815067951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1f80cbd70 0x7fe1f8074260 secure :-1 s=CLOSED pgs=2748 cs=0 l=1 rev1=1 crypto rx=0x7fe1e400c3a0 tx=0x7fe1e400c860 comp rx=0 tx=0).stop 2024-09-21T07:54:24.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.028+0000 7fe1f7e00640 1 --2- 172.21.15.39:0/1815067951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1f80cb960 0x7fe1f81f5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:24.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.028+0000 7fe1f7e00640 1 -- 172.21.15.39:0/1815067951 >> 172.21.15.39:0/1815067951 conn(0x7fe1f80b2510 msgr2=0x7fe1f815fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:24.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.028+0000 7fe1f7e00640 1 -- 172.21.15.39:0/1815067951 shutdown_connections 2024-09-21T07:54:24.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.029+0000 7fe1f7e00640 1 -- 172.21.15.39:0/1815067951 wait complete. 2024-09-21T07:54:24.043 INFO:tasks.ceph.mds.a.smithi039.stderr: -5873> 2024-09-21T07:53:22.536+0000 7fd91b600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:54:24.043 INFO:tasks.ceph.mds.a.smithi039.stderr: -5871> 2024-09-21T07:53:22.536+0000 7fd91b600640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:54:24.043 INFO:tasks.ceph.mds.a.smithi039.stderr: -5869> 2024-09-21T07:53:22.536+0000 7fd91b600640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-21T07:54:24.048 INFO:tasks.ceph.mds.a.smithi039.stderr: -1882> 2024-09-21T07:54:12.540+0000 7fd91b600640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T07:54:24.048 INFO:tasks.ceph.mds.a.smithi039.stderr: -1880> 2024-09-21T07:54:12.540+0000 7fd91b600640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-21T07:54:24.058 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T07:54:24.158 INFO:tasks.ceph.mds.c.smithi039.stdout:starting mds.c at 2024-09-21T07:54:24.159 INFO:tasks.ceph.mds.d.smithi148.stdout:starting mds.d at 2024-09-21T07:54:24.175 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-21T07:54:24.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.293+0000 7f5256e00640 1 -- 172.21.15.39:0/2578389110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52500cce70 msgr2=0x7f52500cd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:24.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.293+0000 7f5256e00640 1 --2- 172.21.15.39:0/2578389110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52500cce70 0x7f52500cd280 secure :-1 s=READY pgs=2755 cs=0 l=1 rev1=1 crypto rx=0x7f5248005fd0 tx=0x7f524802a8b0 comp rx=0 tx=0).stop 2024-09-21T07:54:24.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.293+0000 7f5256e00640 1 -- 172.21.15.39:0/2578389110 shutdown_connections 2024-09-21T07:54:24.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.293+0000 7f5256e00640 1 --2- 172.21.15.39:0/2578389110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52500cce70 0x7f52500cd280 unknown :-1 s=CLOSED pgs=2755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:24.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.293+0000 7f5256e00640 1 -- 172.21.15.39:0/2578389110 >> 172.21.15.39:0/2578389110 conn(0x7f525006d440 msgr2=0x7f52500b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:24.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.293+0000 7f5256e00640 1 -- 172.21.15.39:0/2578389110 shutdown_connections 2024-09-21T07:54:24.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.293+0000 7f5256e00640 1 -- 172.21.15.39:0/2578389110 wait complete. 2024-09-21T07:54:24.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.296+0000 7f5256e00640 1 Processor -- start 2024-09-21T07:54:24.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.296+0000 7f5256e00640 1 -- start start 2024-09-21T07:54:24.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.297+0000 7f5256e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52500b32b0 0x7f52501f5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:24.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.297+0000 7f5256e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52500cce70 0x7f525009d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:24.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.297+0000 7f5256e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f525015cf60 0x7f5250099520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:24.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.297+0000 7f5256e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f525006fb20 con 0x7f52500cce70 2024-09-21T07:54:24.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.297+0000 7f5256e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f525006f9a0 con 0x7f52500b32b0 2024-09-21T07:54:24.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.297+0000 7f5256e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f525006fca0 con 0x7f525015cf60 2024-09-21T07:54:24.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.297+0000 7f5247400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52500cce70 0x7f525009d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:24.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.297+0000 7f5247400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52500cce70 0x7f525009d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41088/0 (socket says 172.21.15.39:41088) 2024-09-21T07:54:24.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.298+0000 7f5247400640 1 -- 172.21.15.39:0/2444805137 learned_addr learned my addr 172.21.15.39:0/2444805137 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:24.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.298+0000 7f524fe00640 1 --2- 172.21.15.39:0/2444805137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52500b32b0 0x7f52501f5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:24.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.298+0000 7f5254e00640 1 --2- 172.21.15.39:0/2444805137 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f525015cf60 0x7f5250099520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:24.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.298+0000 7f5247400640 1 -- 172.21.15.39:0/2444805137 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f525015cf60 msgr2=0x7f5250099520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:24.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.298+0000 7f5247400640 1 --2- 172.21.15.39:0/2444805137 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f525015cf60 0x7f5250099520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:24.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.298+0000 7f5247400640 1 -- 172.21.15.39:0/2444805137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52500b32b0 msgr2=0x7f52501f5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:24.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.298+0000 7f5247400640 1 --2- 172.21.15.39:0/2444805137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52500b32b0 0x7f52501f5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:24.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.298+0000 7f5247400640 1 -- 172.21.15.39:0/2444805137 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f525009c770 con 0x7f52500cce70 2024-09-21T07:54:24.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.298+0000 7f5247400640 1 --2- 172.21.15.39:0/2444805137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52500cce70 0x7f525009d7d0 secure :-1 s=READY pgs=2756 cs=0 l=1 rev1=1 crypto rx=0x7f523800c400 tx=0x7f523800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:24.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.298+0000 7f524cc00640 1 -- 172.21.15.39:0/2444805137 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5238017070 con 0x7f52500cce70 2024-09-21T07:54:24.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.298+0000 7f524cc00640 1 -- 172.21.15.39:0/2444805137 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52380053c0 con 0x7f52500cce70 2024-09-21T07:54:24.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.298+0000 7f524cc00640 1 -- 172.21.15.39:0/2444805137 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5238003850 con 0x7f52500cce70 2024-09-21T07:54:24.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.298+0000 7f5256e00640 1 -- 172.21.15.39:0/2444805137 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f525009ca30 con 0x7f52500cce70 2024-09-21T07:54:24.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.299+0000 7f5256e00640 1 -- 172.21.15.39:0/2444805137 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f525009cff0 con 0x7f52500cce70 2024-09-21T07:54:24.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.300+0000 7f5256e00640 1 -- 172.21.15.39:0/2444805137 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5250070150 con 0x7f52500cce70 2024-09-21T07:54:24.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.301+0000 7f524cc00640 1 -- 172.21.15.39:0/2444805137 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f523801a070 con 0x7f52500cce70 2024-09-21T07:54:24.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.302+0000 7f524cc00640 1 --2- 172.21.15.39:0/2444805137 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f522c078bc0 0x7f522c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:24.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.302+0000 7f524cc00640 1 -- 172.21.15.39:0/2444805137 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(255..255 src has 1..255) ==== 8321+0+0 (secure 0 0 0) 0x7f523809ac30 con 0x7f52500cce70 2024-09-21T07:54:24.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.302+0000 7f524fe00640 1 --2- 172.21.15.39:0/2444805137 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f522c078bc0 0x7f522c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:24.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.303+0000 7f524fe00640 1 --2- 172.21.15.39:0/2444805137 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f522c078bc0 0x7f522c07b0d0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f52400013f0 tx=0x7f5240009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:24.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.306+0000 7f524cc00640 1 -- 172.21.15.39:0/2444805137 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5238062580 con 0x7f52500cce70 2024-09-21T07:54:24.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:24.543+0000 7f5256e00640 1 -- 172.21.15.39:0/2444805137 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f52500753a0 con 0x7f52500cce70 2024-09-21T07:54:25.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.156+0000 7f524cc00640 1 -- 172.21.15.39:0/2444805137 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v303) ==== 104+0+0 (secure 0 0 0) 0x7f5238061e80 con 0x7f52500cce70 2024-09-21T07:54:25.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.159+0000 7f5256e00640 1 -- 172.21.15.39:0/2444805137 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f522c078bc0 msgr2=0x7f522c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:25.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.159+0000 7f5256e00640 1 --2- 172.21.15.39:0/2444805137 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f522c078bc0 0x7f522c07b0d0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f52400013f0 tx=0x7f5240009040 comp rx=0 tx=0).stop 2024-09-21T07:54:25.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.159+0000 7f5256e00640 1 -- 172.21.15.39:0/2444805137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52500cce70 msgr2=0x7f525009d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:25.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.159+0000 7f5256e00640 1 --2- 172.21.15.39:0/2444805137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52500cce70 0x7f525009d7d0 secure :-1 s=READY pgs=2756 cs=0 l=1 rev1=1 crypto rx=0x7f523800c400 tx=0x7f523800c8c0 comp rx=0 tx=0).stop 2024-09-21T07:54:25.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.159+0000 7f5256e00640 1 -- 172.21.15.39:0/2444805137 shutdown_connections 2024-09-21T07:54:25.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.159+0000 7f5256e00640 1 --2- 172.21.15.39:0/2444805137 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f522c078bc0 0x7f522c07b0d0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:25.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.159+0000 7f5256e00640 1 --2- 172.21.15.39:0/2444805137 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f525015cf60 0x7f5250099520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:25.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.159+0000 7f5256e00640 1 --2- 172.21.15.39:0/2444805137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52500cce70 0x7f525009d7d0 unknown :-1 s=CLOSED pgs=2756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:25.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.159+0000 7f5256e00640 1 --2- 172.21.15.39:0/2444805137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52500b32b0 0x7f52501f5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:25.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.159+0000 7f5256e00640 1 -- 172.21.15.39:0/2444805137 >> 172.21.15.39:0/2444805137 conn(0x7f525006d440 msgr2=0x7f52500b1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:25.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.159+0000 7f5256e00640 1 -- 172.21.15.39:0/2444805137 shutdown_connections 2024-09-21T07:54:25.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.159+0000 7f5256e00640 1 -- 172.21.15.39:0/2444805137 wait complete. 2024-09-21T07:54:25.189 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T07:54:25.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.421+0000 7f01f6c00640 1 -- 172.21.15.39:0/4113590512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01f0153080 msgr2=0x7f01f0153490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:25.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.421+0000 7f01f6c00640 1 --2- 172.21.15.39:0/4113590512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01f0153080 0x7f01f0153490 secure :-1 s=READY pgs=2757 cs=0 l=1 rev1=1 crypto rx=0x7f01f0160390 tx=0x7f01d802a4f0 comp rx=0 tx=0).stop 2024-09-21T07:54:25.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.422+0000 7f01f6c00640 1 -- 172.21.15.39:0/4113590512 shutdown_connections 2024-09-21T07:54:25.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.422+0000 7f01f6c00640 1 --2- 172.21.15.39:0/4113590512 >> v2:172.21.15.148:3300/0 conn(0x7f01f01539d0 0x7f01f0160450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:25.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.422+0000 7f01f6c00640 1 --2- 172.21.15.39:0/4113590512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01f0153080 0x7f01f0153490 unknown :-1 s=CLOSED pgs=2757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:25.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.422+0000 7f01f6c00640 1 --2- 172.21.15.39:0/4113590512 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f01f00cfc80 0x7f01f00af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:25.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.422+0000 7f01f6c00640 1 -- 172.21.15.39:0/4113590512 >> 172.21.15.39:0/4113590512 conn(0x7f01f006d7a0 msgr2=0x7f01f00b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:25.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.422+0000 7f01f6c00640 1 -- 172.21.15.39:0/4113590512 shutdown_connections 2024-09-21T07:54:25.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.422+0000 7f01f6c00640 1 -- 172.21.15.39:0/4113590512 wait complete. 2024-09-21T07:54:25.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.424+0000 7f01f6c00640 1 Processor -- start 2024-09-21T07:54:25.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.424+0000 7f01f6c00640 1 -- start start 2024-09-21T07:54:25.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.425+0000 7f01f6c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01f00cfc80 0x7f01f0147ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:25.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.425+0000 7f01f6c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f01f0153080 0x7f01f01f6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:25.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.425+0000 7f01f6c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01f01539d0 0x7f01f01f73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:25.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.426+0000 7f01f6c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f01f0160990 con 0x7f01f00cfc80 2024-09-21T07:54:25.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.426+0000 7f01f6c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f01f0070b40 con 0x7f01f01539d0 2024-09-21T07:54:25.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.426+0000 7f01f6c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f01f0160b10 con 0x7f01f0153080 2024-09-21T07:54:25.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.426+0000 7f01efe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01f00cfc80 0x7f01f0147ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:25.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.426+0000 7f01efe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01f00cfc80 0x7f01f0147ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41098/0 (socket says 172.21.15.39:41098) 2024-09-21T07:54:25.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.426+0000 7f01efe00640 1 -- 172.21.15.39:0/2524947738 learned_addr learned my addr 172.21.15.39:0/2524947738 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:25.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.426+0000 7f01e7400640 1 --2- 172.21.15.39:0/2524947738 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f01f0153080 0x7f01f01f6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:25.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.426+0000 7f01efe00640 1 -- 172.21.15.39:0/2524947738 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f01f0153080 msgr2=0x7f01f01f6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:25.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.426+0000 7f01f4c00640 1 --2- 172.21.15.39:0/2524947738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01f01539d0 0x7f01f01f73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:25.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.426+0000 7f01efe00640 1 --2- 172.21.15.39:0/2524947738 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f01f0153080 0x7f01f01f6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:25.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.426+0000 7f01efe00640 1 -- 172.21.15.39:0/2524947738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01f01539d0 msgr2=0x7f01f01f73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:25.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.426+0000 7f01efe00640 1 --2- 172.21.15.39:0/2524947738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01f01539d0 0x7f01f01f73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:25.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.426+0000 7f01efe00640 1 -- 172.21.15.39:0/2524947738 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f015c1a0 con 0x7f01f00cfc80 2024-09-21T07:54:25.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.426+0000 7f01e7400640 1 --2- 172.21.15.39:0/2524947738 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f01f0153080 0x7f01f01f6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:54:25.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.426+0000 7f01f4c00640 1 --2- 172.21.15.39:0/2524947738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01f01539d0 0x7f01f01f73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:54:25.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.427+0000 7f01efe00640 1 --2- 172.21.15.39:0/2524947738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01f00cfc80 0x7f01f0147ad0 secure :-1 s=READY pgs=2758 cs=0 l=1 rev1=1 crypto rx=0x7f01e000d690 tx=0x7f01e000db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:25.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.427+0000 7f01ecc00640 1 -- 172.21.15.39:0/2524947738 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e0017070 con 0x7f01f00cfc80 2024-09-21T07:54:25.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.427+0000 7f01ecc00640 1 -- 172.21.15.39:0/2524947738 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01e0005310 con 0x7f01f00cfc80 2024-09-21T07:54:25.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.427+0000 7f01f6c00640 1 -- 172.21.15.39:0/2524947738 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01f015c480 con 0x7f01f00cfc80 2024-09-21T07:54:25.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.427+0000 7f01ecc00640 1 -- 172.21.15.39:0/2524947738 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e0009820 con 0x7f01f00cfc80 2024-09-21T07:54:25.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.427+0000 7f01f6c00640 1 -- 172.21.15.39:0/2524947738 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01f015ca40 con 0x7f01f00cfc80 2024-09-21T07:54:25.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.428+0000 7f01f6c00640 1 -- 172.21.15.39:0/2524947738 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01f0161040 con 0x7f01f00cfc80 2024-09-21T07:54:25.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.429+0000 7f01ecc00640 1 -- 172.21.15.39:0/2524947738 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f01e00099c0 con 0x7f01f00cfc80 2024-09-21T07:54:25.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.430+0000 7f01ecc00640 1 --2- 172.21.15.39:0/2524947738 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f01cc078bc0 0x7f01cc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:25.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.430+0000 7f01ecc00640 1 -- 172.21.15.39:0/2524947738 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(255..255 src has 1..255) ==== 8321+0+0 (secure 0 0 0) 0x7f01e009b8d0 con 0x7f01f00cfc80 2024-09-21T07:54:25.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.431+0000 7f01e7400640 1 --2- 172.21.15.39:0/2524947738 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f01cc078bc0 0x7f01cc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:25.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.431+0000 7f01e7400640 1 --2- 172.21.15.39:0/2524947738 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f01cc078bc0 0x7f01cc07b0d0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f01d8033000 tx=0x7f01d8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:25.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.435+0000 7f01ecc00640 1 -- 172.21.15.39:0/2524947738 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f01e0063220 con 0x7f01f00cfc80 2024-09-21T07:54:25.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:25.633+0000 7f01f6c00640 1 -- 172.21.15.39:0/2524947738 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f01f00b0a20 con 0x7f01f00cfc80 2024-09-21T07:54:26.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.161+0000 7f01ecc00640 1 -- 172.21.15.39:0/2524947738 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v256) ==== 181+0+0 (secure 0 0 0) 0x7f01e0062b20 con 0x7f01f00cfc80 2024-09-21T07:54:26.163 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' removed 2024-09-21T07:54:26.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.164+0000 7f01f6c00640 1 -- 172.21.15.39:0/2524947738 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f01cc078bc0 msgr2=0x7f01cc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:26.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.164+0000 7f01f6c00640 1 --2- 172.21.15.39:0/2524947738 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f01cc078bc0 0x7f01cc07b0d0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f01d8033000 tx=0x7f01d8004730 comp rx=0 tx=0).stop 2024-09-21T07:54:26.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.164+0000 7f01f6c00640 1 -- 172.21.15.39:0/2524947738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01f00cfc80 msgr2=0x7f01f0147ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:26.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.164+0000 7f01f6c00640 1 --2- 172.21.15.39:0/2524947738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01f00cfc80 0x7f01f0147ad0 secure :-1 s=READY pgs=2758 cs=0 l=1 rev1=1 crypto rx=0x7f01e000d690 tx=0x7f01e000db50 comp rx=0 tx=0).stop 2024-09-21T07:54:26.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.164+0000 7f01f6c00640 1 -- 172.21.15.39:0/2524947738 shutdown_connections 2024-09-21T07:54:26.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.164+0000 7f01f6c00640 1 --2- 172.21.15.39:0/2524947738 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f01cc078bc0 0x7f01cc07b0d0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:26.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.164+0000 7f01f6c00640 1 --2- 172.21.15.39:0/2524947738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01f01539d0 0x7f01f01f73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:26.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.164+0000 7f01f6c00640 1 --2- 172.21.15.39:0/2524947738 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f01f0153080 0x7f01f01f6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:26.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.164+0000 7f01f6c00640 1 --2- 172.21.15.39:0/2524947738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01f00cfc80 0x7f01f0147ad0 unknown :-1 s=CLOSED pgs=2758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:26.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.164+0000 7f01f6c00640 1 -- 172.21.15.39:0/2524947738 >> 172.21.15.39:0/2524947738 conn(0x7f01f006d7a0 msgr2=0x7f01f00d0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:26.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.164+0000 7f01f6c00640 1 -- 172.21.15.39:0/2524947738 shutdown_connections 2024-09-21T07:54:26.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.164+0000 7f01f6c00640 1 -- 172.21.15.39:0/2524947738 wait complete. 2024-09-21T07:54:26.193 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T07:54:26.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.422+0000 7f491a000640 1 -- 172.21.15.39:0/1672674342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49140cfc80 msgr2=0x7f49140af2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:26.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.422+0000 7f491a000640 1 --2- 172.21.15.39:0/1672674342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49140cfc80 0x7f49140af2e0 secure :-1 s=READY pgs=2759 cs=0 l=1 rev1=1 crypto rx=0x7f4914160390 tx=0x7f490402a4f0 comp rx=0 tx=0).stop 2024-09-21T07:54:26.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.422+0000 7f491a000640 1 -- 172.21.15.39:0/1672674342 shutdown_connections 2024-09-21T07:54:26.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.422+0000 7f491a000640 1 --2- 172.21.15.39:0/1672674342 >> v2:172.21.15.148:3300/0 conn(0x7f49141539d0 0x7f4914160450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:26.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.422+0000 7f491a000640 1 --2- 172.21.15.39:0/1672674342 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4914153080 0x7f4914153490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:26.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.422+0000 7f491a000640 1 --2- 172.21.15.39:0/1672674342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49140cfc80 0x7f49140af2e0 unknown :-1 s=CLOSED pgs=2759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:26.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.422+0000 7f491a000640 1 -- 172.21.15.39:0/1672674342 >> 172.21.15.39:0/1672674342 conn(0x7f491406d7a0 msgr2=0x7f49140b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:26.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.422+0000 7f491a000640 1 -- 172.21.15.39:0/1672674342 shutdown_connections 2024-09-21T07:54:26.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.422+0000 7f491a000640 1 -- 172.21.15.39:0/1672674342 wait complete. 2024-09-21T07:54:26.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.425+0000 7f491a000640 1 Processor -- start 2024-09-21T07:54:26.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.425+0000 7f491a000640 1 -- start start 2024-09-21T07:54:26.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.426+0000 7f491a000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49140cfc80 0x7f491413f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:26.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.426+0000 7f491a000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4914153080 0x7f49140b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:26.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.426+0000 7f491a000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49141539d0 0x7f49140b4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:26.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.426+0000 7f491a000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4914160990 con 0x7f49141539d0 2024-09-21T07:54:26.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.426+0000 7f491a000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4914070b40 con 0x7f4914153080 2024-09-21T07:54:26.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.426+0000 7f491a000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f4914160b10 con 0x7f49140cfc80 2024-09-21T07:54:26.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.426+0000 7f4913e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49141539d0 0x7f49140b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:26.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.426+0000 7f4913e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49141539d0 0x7f49140b4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41106/0 (socket says 172.21.15.39:41106) 2024-09-21T07:54:26.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.426+0000 7f4913e00640 1 -- 172.21.15.39:0/1771755495 learned_addr learned my addr 172.21.15.39:0/1771755495 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:26.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.427+0000 7f4913400640 1 --2- 172.21.15.39:0/1771755495 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49140cfc80 0x7f491413f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:26.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.427+0000 7f490be00640 1 --2- 172.21.15.39:0/1771755495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4914153080 0x7f49140b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:26.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.427+0000 7f4913e00640 1 -- 172.21.15.39:0/1771755495 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49140cfc80 msgr2=0x7f491413f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:26.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.427+0000 7f4913e00640 1 --2- 172.21.15.39:0/1771755495 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49140cfc80 0x7f491413f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:26.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.427+0000 7f4913e00640 1 -- 172.21.15.39:0/1771755495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4914153080 msgr2=0x7f49140b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:26.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.427+0000 7f4913e00640 1 --2- 172.21.15.39:0/1771755495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4914153080 0x7f49140b3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:26.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.427+0000 7f4913e00640 1 -- 172.21.15.39:0/1771755495 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4914149240 con 0x7f49141539d0 2024-09-21T07:54:26.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.427+0000 7f4913400640 1 --2- 172.21.15.39:0/1771755495 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49140cfc80 0x7f491413f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:26.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.427+0000 7f4913e00640 1 --2- 172.21.15.39:0/1771755495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49141539d0 0x7f49140b4450 secure :-1 s=READY pgs=2760 cs=0 l=1 rev1=1 crypto rx=0x7f490c00e990 tx=0x7f490c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:26.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.427+0000 7f490b400640 1 -- 172.21.15.39:0/1771755495 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c004d20 con 0x7f49141539d0 2024-09-21T07:54:26.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.428+0000 7f490b400640 1 -- 172.21.15.39:0/1771755495 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f490c00caa0 con 0x7f49141539d0 2024-09-21T07:54:26.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.428+0000 7f490b400640 1 -- 172.21.15.39:0/1771755495 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c005020 con 0x7f49141539d0 2024-09-21T07:54:26.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.428+0000 7f491a000640 1 -- 172.21.15.39:0/1771755495 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4914154b10 con 0x7f49141539d0 2024-09-21T07:54:26.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.428+0000 7f491a000640 1 -- 172.21.15.39:0/1771755495 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49141550d0 con 0x7f49141539d0 2024-09-21T07:54:26.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.429+0000 7f491a000640 1 -- 172.21.15.39:0/1771755495 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4914161040 con 0x7f49141539d0 2024-09-21T07:54:26.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.430+0000 7f490b400640 1 -- 172.21.15.39:0/1771755495 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f490c0051c0 con 0x7f49141539d0 2024-09-21T07:54:26.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.431+0000 7f490b400640 1 --2- 172.21.15.39:0/1771755495 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f48f0078b90 0x7f48f007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:26.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.431+0000 7f490b400640 1 -- 172.21.15.39:0/1771755495 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(256..256 src has 1..256) ==== 7767+0+0 (secure 0 0 0) 0x7f490c09be80 con 0x7f49141539d0 2024-09-21T07:54:26.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.432+0000 7f4913400640 1 --2- 172.21.15.39:0/1771755495 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f48f0078b90 0x7f48f007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:26.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.432+0000 7f4913400640 1 --2- 172.21.15.39:0/1771755495 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f48f0078b90 0x7f48f007b0a0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f4904033000 tx=0x7f4904004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:26.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.436+0000 7f490b400640 1 -- 172.21.15.39:0/1771755495 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f490c016030 con 0x7f49141539d0 2024-09-21T07:54:26.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:26.630+0000 7f491a000640 1 -- 172.21.15.39:0/1771755495 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f49140008d0 con 0x7f49141539d0 2024-09-21T07:54:27.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.164+0000 7f490b400640 1 -- 172.21.15.39:0/1771755495 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v257) ==== 169+0+0 (secure 0 0 0) 0x7f490c063aa0 con 0x7f49141539d0 2024-09-21T07:54:27.166 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' removed 2024-09-21T07:54:27.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.167+0000 7f491a000640 1 -- 172.21.15.39:0/1771755495 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f48f0078b90 msgr2=0x7f48f007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:27.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.167+0000 7f491a000640 1 --2- 172.21.15.39:0/1771755495 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f48f0078b90 0x7f48f007b0a0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f4904033000 tx=0x7f4904004730 comp rx=0 tx=0).stop 2024-09-21T07:54:27.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.167+0000 7f491a000640 1 -- 172.21.15.39:0/1771755495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49141539d0 msgr2=0x7f49140b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:27.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.167+0000 7f491a000640 1 --2- 172.21.15.39:0/1771755495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49141539d0 0x7f49140b4450 secure :-1 s=READY pgs=2760 cs=0 l=1 rev1=1 crypto rx=0x7f490c00e990 tx=0x7f490c00ee50 comp rx=0 tx=0).stop 2024-09-21T07:54:27.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.167+0000 7f491a000640 1 -- 172.21.15.39:0/1771755495 shutdown_connections 2024-09-21T07:54:27.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.167+0000 7f491a000640 1 --2- 172.21.15.39:0/1771755495 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f48f0078b90 0x7f48f007b0a0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:27.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.167+0000 7f491a000640 1 --2- 172.21.15.39:0/1771755495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49141539d0 0x7f49140b4450 unknown :-1 s=CLOSED pgs=2760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:27.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.167+0000 7f491a000640 1 --2- 172.21.15.39:0/1771755495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4914153080 0x7f49140b3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:27.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.167+0000 7f491a000640 1 --2- 172.21.15.39:0/1771755495 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49140cfc80 0x7f491413f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:27.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.167+0000 7f491a000640 1 -- 172.21.15.39:0/1771755495 >> 172.21.15.39:0/1771755495 conn(0x7f491406d7a0 msgr2=0x7f491415ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:27.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.167+0000 7f491a000640 1 -- 172.21.15.39:0/1771755495 shutdown_connections 2024-09-21T07:54:27.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.167+0000 7f491a000640 1 -- 172.21.15.39:0/1771755495 wait complete. 2024-09-21T07:54:27.202 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T07:54:27.205 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T07:54:27.302 INFO:teuthology.orchestra.run.smithi039.stdout:[global] 2024-09-21T07:54:27.302 INFO:teuthology.orchestra.run.smithi039.stdout: chdir = "" 2024-09-21T07:54:27.302 INFO:teuthology.orchestra.run.smithi039.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T07:54:27.302 INFO:teuthology.orchestra.run.smithi039.stdout: auth supported = cephx 2024-09-21T07:54:27.302 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:27.302 INFO:teuthology.orchestra.run.smithi039.stdout: filestore xattr use omap = true 2024-09-21T07:54:27.302 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:27.302 INFO:teuthology.orchestra.run.smithi039.stdout: mon clock drift allowed = 1.000 2024-09-21T07:54:27.302 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:27.302 INFO:teuthology.orchestra.run.smithi039.stdout: osd crush chooseleaf type = 0 2024-09-21T07:54:27.302 INFO:teuthology.orchestra.run.smithi039.stdout: auth debug = true 2024-09-21T07:54:27.302 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:27.303 INFO:teuthology.orchestra.run.smithi039.stdout: ms die on old message = true 2024-09-21T07:54:27.303 INFO:teuthology.orchestra.run.smithi039.stdout: ms die on bug = true 2024-09-21T07:54:27.303 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:27.303 INFO:teuthology.orchestra.run.smithi039.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T07:54:27.303 INFO:teuthology.orchestra.run.smithi039.stdout: mon pg warn max object skew = 0 2024-09-21T07:54:27.303 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:27.303 INFO:teuthology.orchestra.run.smithi039.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T07:54:27.303 INFO:teuthology.orchestra.run.smithi039.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T07:54:27.303 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:27.303 INFO:teuthology.orchestra.run.smithi039.stdout: osd pool default size = 2 2024-09-21T07:54:27.303 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:27.303 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd allow primary affinity = true 2024-09-21T07:54:27.303 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd allow pg remap = true 2024-09-21T07:54:27.303 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on legacy crush tunables = false 2024-09-21T07:54:27.303 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on crush straw calc version zero = false 2024-09-21T07:54:27.304 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on no sortbitwise = false 2024-09-21T07:54:27.304 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on osd down out interval zero = false 2024-09-21T07:54:27.304 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on too few osds = false 2024-09-21T07:54:27.304 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T07:54:27.304 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T07:54:27.304 INFO:teuthology.orchestra.run.smithi039.stdout: mon_allow_pool_size_one = true 2024-09-21T07:54:27.304 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:27.304 INFO:teuthology.orchestra.run.smithi039.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T07:54:27.304 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:27.304 INFO:teuthology.orchestra.run.smithi039.stdout: osd default data pool replay window = 5 2024-09-21T07:54:27.304 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:27.304 INFO:teuthology.orchestra.run.smithi039.stdout: mon allow pool delete = true 2024-09-21T07:54:27.304 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:27.304 INFO:teuthology.orchestra.run.smithi039.stdout: mon cluster log file level = debug 2024-09-21T07:54:27.305 INFO:teuthology.orchestra.run.smithi039.stdout: debug asserts on shutdown = true 2024-09-21T07:54:27.305 INFO:teuthology.orchestra.run.smithi039.stdout: mon health detail to clog = false 2024-09-21T07:54:27.305 INFO:teuthology.orchestra.run.smithi039.stdout: mon host = "172.21.15.39,172.21.15.148,[v2:172.21.15.148:3301,v1:172.21.15.148:6790]" 2024-09-21T07:54:27.305 INFO:teuthology.orchestra.run.smithi039.stdout: ms die on skipped message = False 2024-09-21T07:54:27.305 INFO:teuthology.orchestra.run.smithi039.stdout: fsid = 7ab9b04a-f88d-49aa-a493-e5c076637ffb 2024-09-21T07:54:27.305 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:27.305 INFO:teuthology.orchestra.run.smithi039.stdout:[osd] 2024-09-21T07:54:27.305 INFO:teuthology.orchestra.run.smithi039.stdout: osd journal size = 100 2024-09-21T07:54:27.305 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:27.305 INFO:teuthology.orchestra.run.smithi039.stdout: osd scrub load threshold = 5.0 2024-09-21T07:54:27.305 INFO:teuthology.orchestra.run.smithi039.stdout: osd scrub max interval = 600 2024-09-21T07:54:27.305 INFO:teuthology.orchestra.run.smithi039.stdout: osd mclock profile = high_recovery_ops 2024-09-21T07:54:27.305 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:27.305 INFO:teuthology.orchestra.run.smithi039.stdout: osd recover clone overlap = true 2024-09-21T07:54:27.305 INFO:teuthology.orchestra.run.smithi039.stdout: osd recovery max chunk = 1048576 2024-09-21T07:54:27.305 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:27.306 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug shutdown = true 2024-09-21T07:54:27.306 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug op order = true 2024-09-21T07:54:27.306 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug verify stray on activate = true 2024-09-21T07:54:27.306 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug trim objects = true 2024-09-21T07:54:27.306 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:27.306 INFO:teuthology.orchestra.run.smithi039.stdout: osd open classes on start = true 2024-09-21T07:54:27.306 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug pg log writeout = true 2024-09-21T07:54:27.306 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:27.306 INFO:teuthology.orchestra.run.smithi039.stdout: osd deep scrub update digest min age = 30 2024-09-21T07:54:27.306 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:27.306 INFO:teuthology.orchestra.run.smithi039.stdout: osd map max advance = 10 2024-09-21T07:54:27.306 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:27.306 INFO:teuthology.orchestra.run.smithi039.stdout: journal zero on create = true 2024-09-21T07:54:27.306 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:27.306 INFO:teuthology.orchestra.run.smithi039.stdout: filestore ondisk finisher threads = 3 2024-09-21T07:54:27.306 INFO:teuthology.orchestra.run.smithi039.stdout: filestore apply finisher threads = 3 2024-09-21T07:54:27.307 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:27.307 INFO:teuthology.orchestra.run.smithi039.stdout: bdev debug aio = true 2024-09-21T07:54:27.307 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug misdirected ops = true 2024-09-21T07:54:27.307 INFO:teuthology.orchestra.run.smithi039.stdout: bdev async discard = True 2024-09-21T07:54:27.307 INFO:teuthology.orchestra.run.smithi039.stdout: bdev enable discard = True 2024-09-21T07:54:27.307 INFO:teuthology.orchestra.run.smithi039.stdout: bluestore allocator = bitmap 2024-09-21T07:54:27.307 INFO:teuthology.orchestra.run.smithi039.stdout: bluestore block size = 96636764160 2024-09-21T07:54:27.307 INFO:teuthology.orchestra.run.smithi039.stdout: bluestore fsck on mount = True 2024-09-21T07:54:27.307 INFO:teuthology.orchestra.run.smithi039.stdout: debug bluefs = 1/20 2024-09-21T07:54:27.307 INFO:teuthology.orchestra.run.smithi039.stdout: debug bluestore = 1/20 2024-09-21T07:54:27.307 INFO:teuthology.orchestra.run.smithi039.stdout: debug ms = 1 2024-09-21T07:54:27.307 INFO:teuthology.orchestra.run.smithi039.stdout: debug osd = 20 2024-09-21T07:54:27.307 INFO:teuthology.orchestra.run.smithi039.stdout: debug rocksdb = 4/10 2024-09-21T07:54:27.307 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T07:54:27.307 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd full ratio = 0.9 2024-09-21T07:54:27.308 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd nearfull ratio = 0.8 2024-09-21T07:54:27.308 INFO:teuthology.orchestra.run.smithi039.stdout: osd failsafe full ratio = 0.95 2024-09-21T07:54:27.308 INFO:teuthology.orchestra.run.smithi039.stdout: osd objectstore = bluestore 2024-09-21T07:54:27.308 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:27.308 INFO:teuthology.orchestra.run.smithi039.stdout:[mgr] 2024-09-21T07:54:27.308 INFO:teuthology.orchestra.run.smithi039.stdout: debug ms = 1 2024-09-21T07:54:27.308 INFO:teuthology.orchestra.run.smithi039.stdout: debug mgr = 20 2024-09-21T07:54:27.308 INFO:teuthology.orchestra.run.smithi039.stdout: debug mon = 20 2024-09-21T07:54:27.308 INFO:teuthology.orchestra.run.smithi039.stdout: debug auth = 20 2024-09-21T07:54:27.308 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min pgs per osd = 4 2024-09-21T07:54:27.308 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min bytes per osd = 10 2024-09-21T07:54:27.308 INFO:teuthology.orchestra.run.smithi039.stdout: mgr/telemetry/nag = false 2024-09-21T07:54:27.308 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:27.308 INFO:teuthology.orchestra.run.smithi039.stdout:[mon] 2024-09-21T07:54:27.308 INFO:teuthology.orchestra.run.smithi039.stdout: debug ms = 1 2024-09-21T07:54:27.308 INFO:teuthology.orchestra.run.smithi039.stdout: debug mon = 20 2024-09-21T07:54:27.309 INFO:teuthology.orchestra.run.smithi039.stdout: debug paxos = 20 2024-09-21T07:54:27.309 INFO:teuthology.orchestra.run.smithi039.stdout: debug auth = 20 2024-09-21T07:54:27.309 INFO:teuthology.orchestra.run.smithi039.stdout: mon data avail warn = 5 2024-09-21T07:54:27.309 INFO:teuthology.orchestra.run.smithi039.stdout: mon mgr mkfs grace = 240 2024-09-21T07:54:27.309 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min pgs per osd = 4 2024-09-21T07:54:27.309 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd reporter subtree level = osd 2024-09-21T07:54:27.309 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd prime pg temp = true 2024-09-21T07:54:27.309 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min bytes per osd = 10 2024-09-21T07:54:27.309 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:27.309 INFO:teuthology.orchestra.run.smithi039.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T07:54:27.309 INFO:teuthology.orchestra.run.smithi039.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T07:54:27.309 INFO:teuthology.orchestra.run.smithi039.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T07:54:27.309 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:27.309 INFO:teuthology.orchestra.run.smithi039.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T07:54:27.310 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T07:54:27.310 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T07:54:27.310 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:27.310 INFO:teuthology.orchestra.run.smithi039.stdout: # 1m isn't quite enough 2024-09-21T07:54:27.310 INFO:teuthology.orchestra.run.smithi039.stdout: mon_down_mkfs_grace = 2m 2024-09-21T07:54:27.310 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:27.310 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_filestore_osds = false 2024-09-21T07:54:27.310 INFO:teuthology.orchestra.run.smithi039.stdout: mon down mkfs grace = 300 2024-09-21T07:54:27.310 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:27.310 INFO:teuthology.orchestra.run.smithi039.stdout:[client] 2024-09-21T07:54:27.310 INFO:teuthology.orchestra.run.smithi039.stdout: rgw cache enabled = true 2024-09-21T07:54:27.310 INFO:teuthology.orchestra.run.smithi039.stdout: rgw enable ops log = true 2024-09-21T07:54:27.310 INFO:teuthology.orchestra.run.smithi039.stdout: rgw enable usage log = true 2024-09-21T07:54:27.310 INFO:teuthology.orchestra.run.smithi039.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T07:54:27.310 INFO:teuthology.orchestra.run.smithi039.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T07:54:27.310 INFO:teuthology.orchestra.run.smithi039.stdout: pid file = "" 2024-09-21T07:54:27.311 INFO:teuthology.orchestra.run.smithi039.stdout:[mon.a] 2024-09-21T07:54:27.311 INFO:teuthology.orchestra.run.smithi039.stdout:[mon.b] 2024-09-21T07:54:27.311 INFO:teuthology.orchestra.run.smithi039.stdout:[mon.c] 2024-09-21T07:54:27.311 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.a] 2024-09-21T07:54:27.311 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.c] 2024-09-21T07:54:27.311 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.b] 2024-09-21T07:54:27.311 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.d] 2024-09-21T07:54:27.311 INFO:teuthology.orchestra.run.smithi039.stdout:[mds] 2024-09-21T07:54:27.311 INFO:teuthology.orchestra.run.smithi148.stdout:[global] 2024-09-21T07:54:27.311 INFO:teuthology.orchestra.run.smithi148.stdout: chdir = "" 2024-09-21T07:54:27.312 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T07:54:27.312 INFO:teuthology.orchestra.run.smithi148.stdout: auth supported = cephx 2024-09-21T07:54:27.312 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:54:27.312 INFO:teuthology.orchestra.run.smithi148.stdout: filestore xattr use omap = true 2024-09-21T07:54:27.312 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:54:27.312 INFO:teuthology.orchestra.run.smithi148.stdout: mon clock drift allowed = 1.000 2024-09-21T07:54:27.312 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:54:27.312 INFO:teuthology.orchestra.run.smithi148.stdout: osd crush chooseleaf type = 0 2024-09-21T07:54:27.312 INFO:teuthology.orchestra.run.smithi148.stdout: auth debug = true 2024-09-21T07:54:27.312 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:54:27.312 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on old message = true 2024-09-21T07:54:27.312 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on bug = true 2024-09-21T07:54:27.312 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:54:27.312 INFO:teuthology.orchestra.run.smithi148.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T07:54:27.312 INFO:teuthology.orchestra.run.smithi148.stdout: mon pg warn max object skew = 0 2024-09-21T07:54:27.313 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:54:27.313 INFO:teuthology.orchestra.run.smithi148.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T07:54:27.313 INFO:teuthology.orchestra.run.smithi148.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T07:54:27.313 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:54:27.313 INFO:teuthology.orchestra.run.smithi148.stdout: osd pool default size = 2 2024-09-21T07:54:27.313 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:54:27.313 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd allow primary affinity = true 2024-09-21T07:54:27.313 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd allow pg remap = true 2024-09-21T07:54:27.313 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on legacy crush tunables = false 2024-09-21T07:54:27.313 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on crush straw calc version zero = false 2024-09-21T07:54:27.313 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on no sortbitwise = false 2024-09-21T07:54:27.313 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on osd down out interval zero = false 2024-09-21T07:54:27.313 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on too few osds = false 2024-09-21T07:54:27.313 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T07:54:27.313 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T07:54:27.313 INFO:teuthology.orchestra.run.smithi148.stdout: mon_allow_pool_size_one = true 2024-09-21T07:54:27.314 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:54:27.314 INFO:teuthology.orchestra.run.smithi148.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T07:54:27.314 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:54:27.314 INFO:teuthology.orchestra.run.smithi148.stdout: osd default data pool replay window = 5 2024-09-21T07:54:27.314 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:54:27.314 INFO:teuthology.orchestra.run.smithi148.stdout: mon allow pool delete = true 2024-09-21T07:54:27.314 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:54:27.314 INFO:teuthology.orchestra.run.smithi148.stdout: mon cluster log file level = debug 2024-09-21T07:54:27.314 INFO:teuthology.orchestra.run.smithi148.stdout: debug asserts on shutdown = true 2024-09-21T07:54:27.314 INFO:teuthology.orchestra.run.smithi148.stdout: mon health detail to clog = false 2024-09-21T07:54:27.314 INFO:teuthology.orchestra.run.smithi148.stdout: mon host = "172.21.15.39,172.21.15.148,[v2:172.21.15.148:3301,v1:172.21.15.148:6790]" 2024-09-21T07:54:27.314 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on skipped message = False 2024-09-21T07:54:27.314 INFO:teuthology.orchestra.run.smithi148.stdout: fsid = 7ab9b04a-f88d-49aa-a493-e5c076637ffb 2024-09-21T07:54:27.314 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:54:27.315 INFO:teuthology.orchestra.run.smithi148.stdout:[osd] 2024-09-21T07:54:27.315 INFO:teuthology.orchestra.run.smithi148.stdout: osd journal size = 100 2024-09-21T07:54:27.315 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:54:27.315 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub load threshold = 5.0 2024-09-21T07:54:27.315 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub max interval = 600 2024-09-21T07:54:27.315 INFO:teuthology.orchestra.run.smithi148.stdout: osd mclock profile = high_recovery_ops 2024-09-21T07:54:27.315 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:54:27.315 INFO:teuthology.orchestra.run.smithi148.stdout: osd recover clone overlap = true 2024-09-21T07:54:27.315 INFO:teuthology.orchestra.run.smithi148.stdout: osd recovery max chunk = 1048576 2024-09-21T07:54:27.315 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:54:27.315 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug shutdown = true 2024-09-21T07:54:27.315 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug op order = true 2024-09-21T07:54:27.315 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug verify stray on activate = true 2024-09-21T07:54:27.315 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug trim objects = true 2024-09-21T07:54:27.315 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:54:27.315 INFO:teuthology.orchestra.run.smithi148.stdout: osd open classes on start = true 2024-09-21T07:54:27.316 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug pg log writeout = true 2024-09-21T07:54:27.316 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:54:27.316 INFO:teuthology.orchestra.run.smithi148.stdout: osd deep scrub update digest min age = 30 2024-09-21T07:54:27.316 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:54:27.316 INFO:teuthology.orchestra.run.smithi148.stdout: osd map max advance = 10 2024-09-21T07:54:27.316 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:54:27.316 INFO:teuthology.orchestra.run.smithi148.stdout: journal zero on create = true 2024-09-21T07:54:27.316 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:54:27.316 INFO:teuthology.orchestra.run.smithi148.stdout: filestore ondisk finisher threads = 3 2024-09-21T07:54:27.316 INFO:teuthology.orchestra.run.smithi148.stdout: filestore apply finisher threads = 3 2024-09-21T07:54:27.316 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:54:27.316 INFO:teuthology.orchestra.run.smithi148.stdout: bdev debug aio = true 2024-09-21T07:54:27.316 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug misdirected ops = true 2024-09-21T07:54:27.316 INFO:teuthology.orchestra.run.smithi148.stdout: bdev async discard = True 2024-09-21T07:54:27.316 INFO:teuthology.orchestra.run.smithi148.stdout: bdev enable discard = True 2024-09-21T07:54:27.316 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore allocator = bitmap 2024-09-21T07:54:27.317 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore block size = 96636764160 2024-09-21T07:54:27.317 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore fsck on mount = True 2024-09-21T07:54:27.317 INFO:teuthology.orchestra.run.smithi148.stdout: debug bluefs = 1/20 2024-09-21T07:54:27.317 INFO:teuthology.orchestra.run.smithi148.stdout: debug bluestore = 1/20 2024-09-21T07:54:27.317 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-09-21T07:54:27.317 INFO:teuthology.orchestra.run.smithi148.stdout: debug osd = 20 2024-09-21T07:54:27.317 INFO:teuthology.orchestra.run.smithi148.stdout: debug rocksdb = 4/10 2024-09-21T07:54:27.317 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T07:54:27.317 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd full ratio = 0.9 2024-09-21T07:54:27.317 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd nearfull ratio = 0.8 2024-09-21T07:54:27.317 INFO:teuthology.orchestra.run.smithi148.stdout: osd failsafe full ratio = 0.95 2024-09-21T07:54:27.317 INFO:teuthology.orchestra.run.smithi148.stdout: osd objectstore = bluestore 2024-09-21T07:54:27.317 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:54:27.317 INFO:teuthology.orchestra.run.smithi148.stdout:[mgr] 2024-09-21T07:54:27.317 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-09-21T07:54:27.317 INFO:teuthology.orchestra.run.smithi148.stdout: debug mgr = 20 2024-09-21T07:54:27.318 INFO:teuthology.orchestra.run.smithi148.stdout: debug mon = 20 2024-09-21T07:54:27.318 INFO:teuthology.orchestra.run.smithi148.stdout: debug auth = 20 2024-09-21T07:54:27.318 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min pgs per osd = 4 2024-09-21T07:54:27.318 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min bytes per osd = 10 2024-09-21T07:54:27.318 INFO:teuthology.orchestra.run.smithi148.stdout: mgr/telemetry/nag = false 2024-09-21T07:54:27.318 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:54:27.318 INFO:teuthology.orchestra.run.smithi148.stdout:[mon] 2024-09-21T07:54:27.318 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-09-21T07:54:27.318 INFO:teuthology.orchestra.run.smithi148.stdout: debug mon = 20 2024-09-21T07:54:27.318 INFO:teuthology.orchestra.run.smithi148.stdout: debug paxos = 20 2024-09-21T07:54:27.318 INFO:teuthology.orchestra.run.smithi148.stdout: debug auth = 20 2024-09-21T07:54:27.318 INFO:teuthology.orchestra.run.smithi148.stdout: mon data avail warn = 5 2024-09-21T07:54:27.318 INFO:teuthology.orchestra.run.smithi148.stdout: mon mgr mkfs grace = 240 2024-09-21T07:54:27.318 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min pgs per osd = 4 2024-09-21T07:54:27.319 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd reporter subtree level = osd 2024-09-21T07:54:27.319 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd prime pg temp = true 2024-09-21T07:54:27.319 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min bytes per osd = 10 2024-09-21T07:54:27.319 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:54:27.319 INFO:teuthology.orchestra.run.smithi148.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T07:54:27.319 INFO:teuthology.orchestra.run.smithi148.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T07:54:27.319 INFO:teuthology.orchestra.run.smithi148.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T07:54:27.319 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:54:27.319 INFO:teuthology.orchestra.run.smithi148.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T07:54:27.319 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T07:54:27.319 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T07:54:27.319 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:54:27.319 INFO:teuthology.orchestra.run.smithi148.stdout: # 1m isn't quite enough 2024-09-21T07:54:27.319 INFO:teuthology.orchestra.run.smithi148.stdout: mon_down_mkfs_grace = 2m 2024-09-21T07:54:27.319 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:54:27.320 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_filestore_osds = false 2024-09-21T07:54:27.320 INFO:teuthology.orchestra.run.smithi148.stdout: mon down mkfs grace = 300 2024-09-21T07:54:27.320 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-21T07:54:27.320 INFO:teuthology.orchestra.run.smithi148.stdout:[client] 2024-09-21T07:54:27.320 INFO:teuthology.orchestra.run.smithi148.stdout: rgw cache enabled = true 2024-09-21T07:54:27.320 INFO:teuthology.orchestra.run.smithi148.stdout: rgw enable ops log = true 2024-09-21T07:54:27.320 INFO:teuthology.orchestra.run.smithi148.stdout: rgw enable usage log = true 2024-09-21T07:54:27.320 INFO:teuthology.orchestra.run.smithi148.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T07:54:27.320 INFO:teuthology.orchestra.run.smithi148.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T07:54:27.320 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = "" 2024-09-21T07:54:27.320 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.a] 2024-09-21T07:54:27.320 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.b] 2024-09-21T07:54:27.320 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.c] 2024-09-21T07:54:27.320 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.a] 2024-09-21T07:54:27.320 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.c] 2024-09-21T07:54:27.321 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.b] 2024-09-21T07:54:27.321 INFO:teuthology.orchestra.run.smithi148.stdout:[mds.d] 2024-09-21T07:54:27.321 INFO:teuthology.orchestra.run.smithi148.stdout:[mds] 2024-09-21T07:54:27.334 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T07:54:27.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.564+0000 7fe52b200640 1 -- 172.21.15.39:0/958024239 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5240b30a0 msgr2=0x7fe5240cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:27.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.564+0000 7fe52b200640 1 --2- 172.21.15.39:0/958024239 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5240b30a0 0x7fe5240cb8a0 secure :-1 s=READY pgs=2761 cs=0 l=1 rev1=1 crypto rx=0x7fe52406d880 tx=0x7fe51402a4d0 comp rx=0 tx=0).stop 2024-09-21T07:54:27.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.564+0000 7fe52b200640 1 -- 172.21.15.39:0/958024239 shutdown_connections 2024-09-21T07:54:27.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.564+0000 7fe52b200640 1 --2- 172.21.15.39:0/958024239 >> v2:172.21.15.148:3300/0 conn(0x7fe5240ccc60 0x7fe5240cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:27.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.564+0000 7fe52b200640 1 --2- 172.21.15.39:0/958024239 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe52415d830 0x7fe524156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:27.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.564+0000 7fe52b200640 1 --2- 172.21.15.39:0/958024239 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5240b30a0 0x7fe5240cb8a0 unknown :-1 s=CLOSED pgs=2761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:27.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.564+0000 7fe52b200640 1 -- 172.21.15.39:0/958024239 >> 172.21.15.39:0/958024239 conn(0x7fe5240b2460 msgr2=0x7fe52406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:27.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.565+0000 7fe52b200640 1 -- 172.21.15.39:0/958024239 shutdown_connections 2024-09-21T07:54:27.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.565+0000 7fe52b200640 1 -- 172.21.15.39:0/958024239 wait complete. 2024-09-21T07:54:27.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.567+0000 7fe52b200640 1 Processor -- start 2024-09-21T07:54:27.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.567+0000 7fe52b200640 1 -- start start 2024-09-21T07:54:27.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.568+0000 7fe52b200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5240b30a0 0x7fe5240cf850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:27.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.568+0000 7fe52b200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe5240ccc60 0x7fe524073f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:27.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.568+0000 7fe52b200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe52415d830 0x7fe5240744a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:27.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.568+0000 7fe52b200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe5240af200 con 0x7fe52415d830 2024-09-21T07:54:27.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.568+0000 7fe52b200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe5240af080 con 0x7fe5240b30a0 2024-09-21T07:54:27.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.568+0000 7fe52b200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe5240af380 con 0x7fe5240ccc60 2024-09-21T07:54:27.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.569+0000 7fe523e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5240b30a0 0x7fe5240cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:27.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.569+0000 7fe529200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe52415d830 0x7fe5240744a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:27.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.569+0000 7fe523e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5240b30a0 0x7fe5240cf850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.39:47702/0 (socket says 172.21.15.39:47702) 2024-09-21T07:54:27.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.569+0000 7fe523400640 1 --2- 172.21.15.39:0/529600796 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe5240ccc60 0x7fe524073f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:27.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.569+0000 7fe523e00640 1 -- 172.21.15.39:0/529600796 learned_addr learned my addr 172.21.15.39:0/529600796 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:27.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.569+0000 7fe529200640 1 -- 172.21.15.39:0/529600796 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe5240ccc60 msgr2=0x7fe524073f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:27.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.569+0000 7fe529200640 1 --2- 172.21.15.39:0/529600796 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe5240ccc60 0x7fe524073f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:27.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.569+0000 7fe529200640 1 -- 172.21.15.39:0/529600796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5240b30a0 msgr2=0x7fe5240cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:27.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.569+0000 7fe529200640 1 --2- 172.21.15.39:0/529600796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5240b30a0 0x7fe5240cf850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:27.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.569+0000 7fe529200640 1 -- 172.21.15.39:0/529600796 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5240c3e20 con 0x7fe52415d830 2024-09-21T07:54:27.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.569+0000 7fe529200640 1 --2- 172.21.15.39:0/529600796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe52415d830 0x7fe5240744a0 secure :-1 s=READY pgs=2762 cs=0 l=1 rev1=1 crypto rx=0x7fe51c005fd0 tx=0x7fe51c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:27.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.570+0000 7fe51be00640 1 -- 172.21.15.39:0/529600796 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c004d20 con 0x7fe52415d830 2024-09-21T07:54:27.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.570+0000 7fe51be00640 1 -- 172.21.15.39:0/529600796 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe51c00ca60 con 0x7fe52415d830 2024-09-21T07:54:27.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.570+0000 7fe52b200640 1 -- 172.21.15.39:0/529600796 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5240c4100 con 0x7fe52415d830 2024-09-21T07:54:27.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.570+0000 7fe51be00640 1 -- 172.21.15.39:0/529600796 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c005020 con 0x7fe52415d830 2024-09-21T07:54:27.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.570+0000 7fe52b200640 1 -- 172.21.15.39:0/529600796 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5240c5e20 con 0x7fe52415d830 2024-09-21T07:54:27.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.571+0000 7fe52b200640 1 -- 172.21.15.39:0/529600796 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5240af870 con 0x7fe52415d830 2024-09-21T07:54:27.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.572+0000 7fe51be00640 1 -- 172.21.15.39:0/529600796 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe51c0051c0 con 0x7fe52415d830 2024-09-21T07:54:27.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.573+0000 7fe51be00640 1 --2- 172.21.15.39:0/529600796 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe504078b90 0x7fe50407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:27.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.573+0000 7fe51be00640 1 -- 172.21.15.39:0/529600796 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(257..257 src has 1..257) ==== 7353+0+0 (secure 0 0 0) 0x7fe51c09bb70 con 0x7fe52415d830 2024-09-21T07:54:27.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.574+0000 7fe523e00640 1 --2- 172.21.15.39:0/529600796 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe504078b90 0x7fe50407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:27.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.575+0000 7fe523e00640 1 --2- 172.21.15.39:0/529600796 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe504078b90 0x7fe50407b0a0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fe514009230 tx=0x7fe514004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:27.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.578+0000 7fe51be00640 1 -- 172.21.15.39:0/529600796 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe51c016030 con 0x7fe52415d830 2024-09-21T07:54:27.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.775+0000 7fe52b200640 1 -- 172.21.15.39:0/529600796 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7fe52406ff70 con 0x7fe52415d830 2024-09-21T07:54:27.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.779+0000 7fe51be00640 1 -- 172.21.15.39:0/529600796 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v56) ==== 71+0+0 (secure 0 0 0) 0x7fe51c063250 con 0x7fe52415d830 2024-09-21T07:54:27.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.782+0000 7fe52b200640 1 -- 172.21.15.39:0/529600796 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe504078b90 msgr2=0x7fe50407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:27.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.782+0000 7fe52b200640 1 --2- 172.21.15.39:0/529600796 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe504078b90 0x7fe50407b0a0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fe514009230 tx=0x7fe514004730 comp rx=0 tx=0).stop 2024-09-21T07:54:27.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.782+0000 7fe52b200640 1 -- 172.21.15.39:0/529600796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe52415d830 msgr2=0x7fe5240744a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:27.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.782+0000 7fe52b200640 1 --2- 172.21.15.39:0/529600796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe52415d830 0x7fe5240744a0 secure :-1 s=READY pgs=2762 cs=0 l=1 rev1=1 crypto rx=0x7fe51c005fd0 tx=0x7fe51c00edf0 comp rx=0 tx=0).stop 2024-09-21T07:54:27.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.782+0000 7fe52b200640 1 -- 172.21.15.39:0/529600796 shutdown_connections 2024-09-21T07:54:27.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.782+0000 7fe52b200640 1 --2- 172.21.15.39:0/529600796 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe504078b90 0x7fe50407b0a0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:27.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.782+0000 7fe52b200640 1 --2- 172.21.15.39:0/529600796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe52415d830 0x7fe5240744a0 unknown :-1 s=CLOSED pgs=2762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:27.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.782+0000 7fe52b200640 1 --2- 172.21.15.39:0/529600796 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe5240ccc60 0x7fe524073f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:27.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.782+0000 7fe52b200640 1 --2- 172.21.15.39:0/529600796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe5240b30a0 0x7fe5240cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:27.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.782+0000 7fe52b200640 1 -- 172.21.15.39:0/529600796 >> 172.21.15.39:0/529600796 conn(0x7fe5240b2460 msgr2=0x7fe5240ad7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:27.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.782+0000 7fe52b200640 1 -- 172.21.15.39:0/529600796 shutdown_connections 2024-09-21T07:54:27.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:27.782+0000 7fe52b200640 1 -- 172.21.15.39:0/529600796 wait complete. 2024-09-21T07:54:27.807 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_snapclient_cache' 2024-09-21T07:54:28.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:28.034+0000 7f2749000640 1 -- 172.21.15.39:0/2333981027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2728004800 msgr2=0x7f2728004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:28.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:28.034+0000 7f2749000640 1 --2- 172.21.15.39:0/2333981027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2728004800 0x7f2728004c10 secure :-1 s=READY pgs=2764 cs=0 l=1 rev1=1 crypto rx=0x7f273000c290 tx=0x7f2728005830 comp rx=0 tx=0).stop 2024-09-21T07:54:28.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:28.035+0000 7f2743e00640 1 -- 172.21.15.39:0/2333981027 reap_dead start 2024-09-21T07:54:28.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:28.035+0000 7f2749000640 1 -- 172.21.15.39:0/2333981027 shutdown_connections 2024-09-21T07:54:28.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:28.035+0000 7f2749000640 1 -- 172.21.15.39:0/2333981027 >> 172.21.15.39:0/2333981027 conn(0x7f27440d0c70 msgr2=0x7f27440d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:28.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:28.035+0000 7f2749000640 1 -- 172.21.15.39:0/2333981027 shutdown_connections 2024-09-21T07:54:28.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:28.035+0000 7f2749000640 1 -- 172.21.15.39:0/2333981027 wait complete. 2024-09-21T07:54:28.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:28.037+0000 7f2749000640 1 Processor -- start 2024-09-21T07:54:28.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:28.037+0000 7f2749000640 1 -- start start 2024-09-21T07:54:28.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:28.039+0000 7f2749000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2728004800 0x7f27441f5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:28.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:28.039+0000 7f2749000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f274414b5a0 0x7f274414b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:28.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:28.039+0000 7f2749000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2744156140 0x7f2744156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:28.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:28.039+0000 7f2749000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f27440ce660 con 0x7f2728004800 2024-09-21T07:54:28.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:28.039+0000 7f2749000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f27440cd840 con 0x7f2744156140 2024-09-21T07:54:28.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:28.039+0000 7f2749000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f274415ccb0 con 0x7f274414b5a0 2024-09-21T07:54:28.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:28.039+0000 7f2743400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2728004800 0x7f27441f5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:28.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:28.039+0000 7f2743400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2728004800 0x7f27441f5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41120/0 (socket says 172.21.15.39:41120) 2024-09-21T07:54:28.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:28.039+0000 7f2743400640 1 -- 172.21.15.39:0/1222669204 learned_addr learned my addr 172.21.15.39:0/1222669204 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:28.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:28.039+0000 7f2742a00640 1 --2- 172.21.15.39:0/1222669204 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f274414b5a0 0x7f274414b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:28.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:28.039+0000 7f2743e00640 1 --2- 172.21.15.39:0/1222669204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2744156140 0x7f2744156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:28.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:28.039+0000 7f2743400640 1 -- 172.21.15.39:0/1222669204 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f274414b5a0 msgr2=0x7f274414b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:28.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:28.039+0000 7f2743400640 1 --2- 172.21.15.39:0/1222669204 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f274414b5a0 0x7f274414b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:28.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:28.039+0000 7f2743400640 1 -- 172.21.15.39:0/1222669204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2744156140 msgr2=0x7f2744156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:28.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:28.039+0000 7f2743400640 1 --2- 172.21.15.39:0/1222669204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2744156140 0x7f2744156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:28.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:28.039+0000 7f2743400640 1 -- 172.21.15.39:0/1222669204 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2744156a90 con 0x7f2728004800 2024-09-21T07:54:28.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:28.040+0000 7f2743400640 1 --2- 172.21.15.39:0/1222669204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2728004800 0x7f27441f5ea0 secure :-1 s=READY pgs=2765 cs=0 l=1 rev1=1 crypto rx=0x7f27340119e0 tx=0x7f2734011ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:28.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:28.040+0000 7f273b400640 1 -- 172.21.15.39:0/1222669204 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273401f070 con 0x7f2728004800 2024-09-21T07:54:28.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:28.040+0000 7f273b400640 1 -- 172.21.15.39:0/1222669204 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2734012450 con 0x7f2728004800 2024-09-21T07:54:28.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:28.040+0000 7f2749000640 1 -- 172.21.15.39:0/1222669204 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27441f33f0 con 0x7f2728004800 2024-09-21T07:54:28.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:28.040+0000 7f273b400640 1 -- 172.21.15.39:0/1222669204 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2734012800 con 0x7f2728004800 2024-09-21T07:54:28.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:28.040+0000 7f2749000640 1 -- 172.21.15.39:0/1222669204 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27441f39b0 con 0x7f2728004800 2024-09-21T07:54:28.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:28.041+0000 7f2749000640 1 -- 172.21.15.39:0/1222669204 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2744153870 con 0x7f2728004800 2024-09-21T07:54:28.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:28.042+0000 7f273b400640 1 -- 172.21.15.39:0/1222669204 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2734022070 con 0x7f2728004800 2024-09-21T07:54:28.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:28.043+0000 7f273b400640 1 --2- 172.21.15.39:0/1222669204 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2724078b90 0x7f272407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:28.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:28.043+0000 7f273b400640 1 -- 172.21.15.39:0/1222669204 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(257..257 src has 1..257) ==== 7353+0+0 (secure 0 0 0) 0x7f273409c030 con 0x7f2728004800 2024-09-21T07:54:28.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:28.044+0000 7f2742a00640 1 --2- 172.21.15.39:0/1222669204 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2724078b90 0x7f272407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:28.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:28.045+0000 7f2742a00640 1 --2- 172.21.15.39:0/1222669204 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2724078b90 0x7f272407b0a0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f2730033000 tx=0x7f2744160570 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:28.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:28.048+0000 7f273b400640 1 -- 172.21.15.39:0/1222669204 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2744156a90 con 0x7f2728004800 2024-09-21T07:54:28.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:28.320+0000 7f2749000640 1 -- 172.21.15.39:0/1222669204 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_snapclient_cache"]} v 0) -- 0x7f274414f5a0 con 0x7f2728004800 2024-09-21T07:54:29.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.185+0000 7f273b400640 1 -- 172.21.15.39:0/1222669204 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_snapclient_cache"]}]=0 v1029) ==== 142+0+0 (secure 0 0 0) 0x7f27340637b0 con 0x7f2728004800 2024-09-21T07:54:29.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.187+0000 7f2749000640 1 -- 172.21.15.39:0/1222669204 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2724078b90 msgr2=0x7f272407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:29.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.187+0000 7f2749000640 1 --2- 172.21.15.39:0/1222669204 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2724078b90 0x7f272407b0a0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f2730033000 tx=0x7f2744160570 comp rx=0 tx=0).stop 2024-09-21T07:54:29.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.187+0000 7f2749000640 1 -- 172.21.15.39:0/1222669204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2728004800 msgr2=0x7f27441f5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:29.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.187+0000 7f2749000640 1 --2- 172.21.15.39:0/1222669204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2728004800 0x7f27441f5ea0 secure :-1 s=READY pgs=2765 cs=0 l=1 rev1=1 crypto rx=0x7f27340119e0 tx=0x7f2734011ea0 comp rx=0 tx=0).stop 2024-09-21T07:54:29.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.187+0000 7f2749000640 1 -- 172.21.15.39:0/1222669204 shutdown_connections 2024-09-21T07:54:29.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.187+0000 7f2749000640 1 --2- 172.21.15.39:0/1222669204 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2724078b90 0x7f272407b0a0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:29.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.187+0000 7f2749000640 1 --2- 172.21.15.39:0/1222669204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2744156140 0x7f2744156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:29.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.187+0000 7f2749000640 1 --2- 172.21.15.39:0/1222669204 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f274414b5a0 0x7f274414b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:29.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.187+0000 7f2749000640 1 --2- 172.21.15.39:0/1222669204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2728004800 0x7f27441f5ea0 unknown :-1 s=CLOSED pgs=2765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:29.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.187+0000 7f2749000640 1 -- 172.21.15.39:0/1222669204 >> 172.21.15.39:0/1222669204 conn(0x7f27440d0c70 msgr2=0x7f274415be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:29.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.188+0000 7f2749000640 1 -- 172.21.15.39:0/1222669204 shutdown_connections 2024-09-21T07:54:29.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.188+0000 7f2749000640 1 -- 172.21.15.39:0/1222669204 wait complete. 2024-09-21T07:54:29.214 INFO:tasks.cephfs_test_runner:check if snapclient cache gets synced properly ... ok 2024-09-21T07:54:29.214 INFO:tasks.cephfs_test_runner:Starting test: test_snapshot_check_access (tasks.cephfs.test_snapshots.TestSnapshots) 2024-09-21T07:54:29.215 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:54:29.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.440+0000 7fde19c00640 1 -- 172.21.15.39:0/3460323179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde140b5a90 msgr2=0x7fde140d1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:29.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.440+0000 7fde19c00640 1 --2- 172.21.15.39:0/3460323179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde140b5a90 0x7fde140d1ac0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fde04009820 tx=0x7fde0402a4a0 comp rx=0 tx=0).stop 2024-09-21T07:54:29.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.440+0000 7fde19c00640 1 -- 172.21.15.39:0/3460323179 shutdown_connections 2024-09-21T07:54:29.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.440+0000 7fde19c00640 1 --2- 172.21.15.39:0/3460323179 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde140b3b60 0x7fde140b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:29.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.440+0000 7fde19c00640 1 --2- 172.21.15.39:0/3460323179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde140b5a90 0x7fde140d1ac0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:29.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.440+0000 7fde19c00640 1 -- 172.21.15.39:0/3460323179 >> 172.21.15.39:0/3460323179 conn(0x7fde1409d7b0 msgr2=0x7fde140d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:29.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.440+0000 7fde19c00640 1 -- 172.21.15.39:0/3460323179 shutdown_connections 2024-09-21T07:54:29.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.440+0000 7fde19c00640 1 -- 172.21.15.39:0/3460323179 wait complete. 2024-09-21T07:54:29.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.442+0000 7fde19c00640 1 Processor -- start 2024-09-21T07:54:29.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.442+0000 7fde19c00640 1 -- start start 2024-09-21T07:54:29.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.444+0000 7fde19c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde140b3b60 0x7fde140c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:29.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.444+0000 7fde19c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde140b5a90 0x7fde1414c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:29.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.444+0000 7fde19c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde140d0e70 0x7fde14161000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:29.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.444+0000 7fde19c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fde140b4630 con 0x7fde140d0e70 2024-09-21T07:54:29.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.444+0000 7fde19c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fde140b44b0 con 0x7fde140b3b60 2024-09-21T07:54:29.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.444+0000 7fde19c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fde140b47b0 con 0x7fde140b5a90 2024-09-21T07:54:29.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.444+0000 7fde19200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde140d0e70 0x7fde14161000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:29.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.444+0000 7fde19200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde140d0e70 0x7fde14161000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41128/0 (socket says 172.21.15.39:41128) 2024-09-21T07:54:29.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.444+0000 7fde19200640 1 -- 172.21.15.39:0/24016201 learned_addr learned my addr 172.21.15.39:0/24016201 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:29.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.444+0000 7fde13e00640 1 --2- 172.21.15.39:0/24016201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde140b3b60 0x7fde140c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:29.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.444+0000 7fde0b400640 1 --2- 172.21.15.39:0/24016201 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde140b5a90 0x7fde1414c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:29.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.445+0000 7fde19200640 1 -- 172.21.15.39:0/24016201 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde140b5a90 msgr2=0x7fde1414c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:29.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.445+0000 7fde19200640 1 --2- 172.21.15.39:0/24016201 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde140b5a90 0x7fde1414c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:29.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.445+0000 7fde19200640 1 -- 172.21.15.39:0/24016201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde140b3b60 msgr2=0x7fde140c7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:29.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.445+0000 7fde19200640 1 --2- 172.21.15.39:0/24016201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde140b3b60 0x7fde140c7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:29.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.445+0000 7fde19200640 1 -- 172.21.15.39:0/24016201 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde1414d460 con 0x7fde140d0e70 2024-09-21T07:54:29.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.445+0000 7fde19200640 1 --2- 172.21.15.39:0/24016201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde140d0e70 0x7fde14161000 secure :-1 s=READY pgs=2767 cs=0 l=1 rev1=1 crypto rx=0x7fde0c00ea70 tx=0x7fde0c00ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:29.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.445+0000 7fde10c00640 1 -- 172.21.15.39:0/24016201 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0c00ce40 con 0x7fde140d0e70 2024-09-21T07:54:29.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.445+0000 7fde10c00640 1 -- 172.21.15.39:0/24016201 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde0c004c10 con 0x7fde140d0e70 2024-09-21T07:54:29.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.445+0000 7fde10c00640 1 -- 172.21.15.39:0/24016201 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0c005020 con 0x7fde140d0e70 2024-09-21T07:54:29.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.445+0000 7fde19c00640 1 -- 172.21.15.39:0/24016201 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde140b2370 con 0x7fde140d0e70 2024-09-21T07:54:29.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.445+0000 7fde19c00640 1 -- 172.21.15.39:0/24016201 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde140b2930 con 0x7fde140d0e70 2024-09-21T07:54:29.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.447+0000 7fde19c00640 1 -- 172.21.15.39:0/24016201 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde1414d1d0 con 0x7fde140d0e70 2024-09-21T07:54:29.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.448+0000 7fde10c00640 1 -- 172.21.15.39:0/24016201 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fde0c0051c0 con 0x7fde140d0e70 2024-09-21T07:54:29.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.449+0000 7fde10c00640 1 --2- 172.21.15.39:0/24016201 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fddf0078b90 0x7fddf007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:29.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.449+0000 7fde10c00640 1 -- 172.21.15.39:0/24016201 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(257..257 src has 1..257) ==== 7353+0+0 (secure 0 0 0) 0x7fde0c09c120 con 0x7fde140d0e70 2024-09-21T07:54:29.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.449+0000 7fde13e00640 1 --2- 172.21.15.39:0/24016201 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fddf0078b90 0x7fddf007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:29.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.450+0000 7fde13e00640 1 --2- 172.21.15.39:0/24016201 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fddf0078b90 0x7fddf007b0a0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fde04034000 tx=0x7fde04004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:29.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.453+0000 7fde10c00640 1 -- 172.21.15.39:0/24016201 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fde0c063810 con 0x7fde140d0e70 2024-09-21T07:54:29.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.653+0000 7fde19c00640 1 -- 172.21.15.39:0/24016201 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fde140b2160 con 0x7fde140d0e70 2024-09-21T07:54:29.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.654+0000 7fde10c00640 1 -- 172.21.15.39:0/24016201 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v257) ==== 74+0+20517 (secure 0 0 0) 0x7fde0c068240 con 0x7fde140d0e70 2024-09-21T07:54:29.657 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:29.657 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":257,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:54:27.162592+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":234,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.148:6836/3495264983":"2024-09-22T07:54:24.021472+0000","172.21.15.39:6834/2161818466":"2024-09-22T07:54:24.021491+0000","172.21.15.39:6835/732408880":"2024-09-22T07:54:24.021497+0000","172.21.15.39:6833/732408880":"2024-09-22T07:54:24.021497+0000","172.21.15.39:0/249900061":"2024-09-21T08:54:08.362327+0000","172.21.15.148:6837/3495264983":"2024-09-22T07:54:24.021472+0000","172.21.15.39:6832/2161818466":"2024-09-22T07:54:24.021491+0000","172.21.15.148:6835/2191156088":"2024-09-22T07:53:58.762864+0000","172.21.15.39:6835/485194245":"2024-09-22T07:53:58.201516+0000","172.21.15.39:6833/485194245":"2024-09-22T07:53:58.201516+0000","172.21.15.39:0/4061644382":"2024-09-21T08:53:16.749620+0000","172.21.15.148:6837/597301351":"2024-09-22T07:53:09.120490+0000","172.21.15.148:6834/2191156088":"2024-09-22T07:53:58.762864+0000","172.21.15.148:6835/597301351":"2024-09-22T07:53:09.120490+0000","172.21.15.148:6836/3381818843":"2024-09-22T07:53:08.565426+0000","172.21.15.148:6834/3381818843":"2024-09-22T07:53:08.565426+0000","172.21.15.39:0/106826898":"2024-09-21T08:52:26.149462+0000","172.21.15.39:6834/109229385":"2024-09-22T07:51:36.504781+0000","172.21.15.39:6832/109229385":"2024-09-22T07:51:36.504781+0000","172.21.15.148:6837/1438851059":"2024-09-22T07:51:00.902180+0000","172.21.15.148:6835/1438851059":"2024-09-22T07:51:00.902180+0000","172.21.15.39:6835/2824493631":"2024-09-22T07:50:31.817924+0000","172.21.15.39:6833/2824493631":"2024-09-22T07:50:31.817924+0000","172.21.15.39:6834/4143807732":"2024-09-22T07:50:32.369206+0000","172.21.15.39:6832/4143807732":"2024-09-22T07:50:32.369206+0000","172.21.15.148:6836/3497801035":"2024-09-22T07:50:24.850152+0000","172.21.15.148:6834/3497801035":"2024-09-22T07:50:24.850152+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]},{"pool":20,"snaps":[{"begin":80,"length":23},{"begin":181,"length":23}]},{"pool":22,"snaps":[{"begin":100,"length":1},{"begin":199,"length":1}]},{"pool":24,"snaps":[{"begin":2,"length":2}]},{"pool":26,"snaps":[{"begin":2,"length":2}]},{"pool":28,"snaps":[{"begin":2,"length":2}]},{"pool":30,"snaps":[{"begin":8,"length":6}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:54:29.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.657+0000 7fde19c00640 1 -- 172.21.15.39:0/24016201 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fddf0078b90 msgr2=0x7fddf007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:29.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.657+0000 7fde19c00640 1 --2- 172.21.15.39:0/24016201 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fddf0078b90 0x7fddf007b0a0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fde04034000 tx=0x7fde04004730 comp rx=0 tx=0).stop 2024-09-21T07:54:29.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.657+0000 7fde19c00640 1 -- 172.21.15.39:0/24016201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde140d0e70 msgr2=0x7fde14161000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:29.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.657+0000 7fde19c00640 1 --2- 172.21.15.39:0/24016201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde140d0e70 0x7fde14161000 secure :-1 s=READY pgs=2767 cs=0 l=1 rev1=1 crypto rx=0x7fde0c00ea70 tx=0x7fde0c00ef30 comp rx=0 tx=0).stop 2024-09-21T07:54:29.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.657+0000 7fde19c00640 1 -- 172.21.15.39:0/24016201 shutdown_connections 2024-09-21T07:54:29.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.657+0000 7fde19c00640 1 --2- 172.21.15.39:0/24016201 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fddf0078b90 0x7fddf007b0a0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:29.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.657+0000 7fde19c00640 1 --2- 172.21.15.39:0/24016201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde140d0e70 0x7fde14161000 unknown :-1 s=CLOSED pgs=2767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:29.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.657+0000 7fde19c00640 1 --2- 172.21.15.39:0/24016201 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fde140b5a90 0x7fde1414c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:29.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.657+0000 7fde19c00640 1 --2- 172.21.15.39:0/24016201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde140b3b60 0x7fde140c7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:29.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.657+0000 7fde19c00640 1 -- 172.21.15.39:0/24016201 >> 172.21.15.39:0/24016201 conn(0x7fde1409d7b0 msgr2=0x7fde1406e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:29.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.657+0000 7fde19c00640 1 -- 172.21.15.39:0/24016201 shutdown_connections 2024-09-21T07:54:29.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.657+0000 7fde19c00640 1 -- 172.21.15.39:0/24016201 wait complete. 2024-09-21T07:54:29.684 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T07:33:01.569433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'snap-test-pool', 'create_time': '2024-09-21T07:34:22.032185+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 1, 'snap_epoch': 37, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'test_data_pool', 'create_time': '2024-09-21T07:35:25.604532+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 1, 'snap_epoch': 66, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_metadata_pool', 'create_time': '2024-09-21T07:35:26.327607+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T07:54:29.684 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T07:54:29.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.912+0000 7fb5a8e00640 1 -- 172.21.15.39:0/4168039954 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5a415d890 msgr2=0x7fb5a406ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:29.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.912+0000 7fb5a8e00640 1 --2- 172.21.15.39:0/4168039954 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5a415d890 0x7fb5a406ef90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb59c009be0 tx=0x7fb59c02a820 comp rx=0 tx=0).stop 2024-09-21T07:54:29.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.912+0000 7fb5a8e00640 1 -- 172.21.15.39:0/4168039954 shutdown_connections 2024-09-21T07:54:29.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.912+0000 7fb5a8e00640 1 --2- 172.21.15.39:0/4168039954 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5a415d890 0x7fb5a406ef90 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:29.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.912+0000 7fb5a8e00640 1 -- 172.21.15.39:0/4168039954 >> 172.21.15.39:0/4168039954 conn(0x7fb5a406d7a0 msgr2=0x7fb5a40b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:29.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.912+0000 7fb5a8e00640 1 -- 172.21.15.39:0/4168039954 shutdown_connections 2024-09-21T07:54:29.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.912+0000 7fb5a8e00640 1 -- 172.21.15.39:0/4168039954 wait complete. 2024-09-21T07:54:29.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.914+0000 7fb5a8e00640 1 Processor -- start 2024-09-21T07:54:29.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.914+0000 7fb5a8e00640 1 -- start start 2024-09-21T07:54:29.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.916+0000 7fb5a8e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5a40706f0 0x7fb5a41e4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:29.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.916+0000 7fb5a8e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5a40af830 0x7fb5a41e52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:29.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.916+0000 7fb5a8e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5a415d890 0x7fb5a4074360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:29.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.916+0000 7fb5a8e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a40b38e0 con 0x7fb5a415d890 2024-09-21T07:54:29.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.916+0000 7fb5a8e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a40b3760 con 0x7fb5a40af830 2024-09-21T07:54:29.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.916+0000 7fb5a8e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fb5a40b3a60 con 0x7fb5a40706f0 2024-09-21T07:54:29.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.916+0000 7fb5a2800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5a415d890 0x7fb5a4074360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:29.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.916+0000 7fb5a2800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5a415d890 0x7fb5a4074360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41140/0 (socket says 172.21.15.39:41140) 2024-09-21T07:54:29.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.916+0000 7fb5a2800640 1 -- 172.21.15.39:0/2211611030 learned_addr learned my addr 172.21.15.39:0/2211611030 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:29.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.916+0000 7fb5a1e00640 1 --2- 172.21.15.39:0/2211611030 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5a40706f0 0x7fb5a41e4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:29.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.916+0000 7fb59be00640 1 --2- 172.21.15.39:0/2211611030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5a40af830 0x7fb5a41e52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:29.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.917+0000 7fb5a2800640 1 -- 172.21.15.39:0/2211611030 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5a40706f0 msgr2=0x7fb5a41e4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:29.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.917+0000 7fb5a2800640 1 --2- 172.21.15.39:0/2211611030 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5a40706f0 0x7fb5a41e4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:29.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.917+0000 7fb5a2800640 1 -- 172.21.15.39:0/2211611030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5a40af830 msgr2=0x7fb5a41e52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:29.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.917+0000 7fb5a2800640 1 --2- 172.21.15.39:0/2211611030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5a40af830 0x7fb5a41e52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:29.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.917+0000 7fb5a2800640 1 -- 172.21.15.39:0/2211611030 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5a40735a0 con 0x7fb5a415d890 2024-09-21T07:54:29.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.917+0000 7fb5a2800640 1 --2- 172.21.15.39:0/2211611030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5a415d890 0x7fb5a4074360 secure :-1 s=READY pgs=2769 cs=0 l=1 rev1=1 crypto rx=0x7fb59c005fd0 tx=0x7fb59c032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:29.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.917+0000 7fb59a000640 1 -- 172.21.15.39:0/2211611030 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c041020 con 0x7fb5a415d890 2024-09-21T07:54:29.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.917+0000 7fb59a000640 1 -- 172.21.15.39:0/2211611030 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb59c032e50 con 0x7fb5a415d890 2024-09-21T07:54:29.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.917+0000 7fb59a000640 1 -- 172.21.15.39:0/2211611030 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c036040 con 0x7fb5a415d890 2024-09-21T07:54:29.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.917+0000 7fb5a8e00640 1 -- 172.21.15.39:0/2211611030 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5a4073880 con 0x7fb5a415d890 2024-09-21T07:54:29.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.918+0000 7fb5a8e00640 1 -- 172.21.15.39:0/2211611030 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5a409c8d0 con 0x7fb5a415d890 2024-09-21T07:54:29.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.919+0000 7fb5a8e00640 1 -- 172.21.15.39:0/2211611030 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5a40732f0 con 0x7fb5a415d890 2024-09-21T07:54:29.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.920+0000 7fb59a000640 1 -- 172.21.15.39:0/2211611030 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb59c044020 con 0x7fb5a415d890 2024-09-21T07:54:29.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.921+0000 7fb59a000640 1 --2- 172.21.15.39:0/2211611030 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb580078bc0 0x7fb58007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:29.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.921+0000 7fb59a000640 1 -- 172.21.15.39:0/2211611030 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(257..257 src has 1..257) ==== 7353+0+0 (secure 0 0 0) 0x7fb59c0bbfb0 con 0x7fb5a415d890 2024-09-21T07:54:29.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.922+0000 7fb5a1e00640 1 --2- 172.21.15.39:0/2211611030 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb580078bc0 0x7fb58007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:29.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.923+0000 7fb5a1e00640 1 --2- 172.21.15.39:0/2211611030 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb580078bc0 0x7fb58007b0d0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fb5940013c0 tx=0x7fb594001720 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:29.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:29.926+0000 7fb59a000640 1 -- 172.21.15.39:0/2211611030 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb59c04b070 con 0x7fb5a415d890 2024-09-21T07:54:30.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.123+0000 7fb5a8e00640 1 -- 172.21.15.39:0/2211611030 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb5a406ef90 con 0x7fb5a415d890 2024-09-21T07:54:30.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.124+0000 7fb59a000640 1 -- 172.21.15.39:0/2211611030 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v257) ==== 93+0+10 (secure 0 0 0) 0x7fb59c083690 con 0x7fb5a415d890 2024-09-21T07:54:30.126 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-09-21T07:54:30.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.126+0000 7fb5a8e00640 1 -- 172.21.15.39:0/2211611030 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb580078bc0 msgr2=0x7fb58007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:30.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.126+0000 7fb5a8e00640 1 --2- 172.21.15.39:0/2211611030 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb580078bc0 0x7fb58007b0d0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fb5940013c0 tx=0x7fb594001720 comp rx=0 tx=0).stop 2024-09-21T07:54:30.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.126+0000 7fb5a8e00640 1 -- 172.21.15.39:0/2211611030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5a415d890 msgr2=0x7fb5a4074360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:30.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.126+0000 7fb5a8e00640 1 --2- 172.21.15.39:0/2211611030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5a415d890 0x7fb5a4074360 secure :-1 s=READY pgs=2769 cs=0 l=1 rev1=1 crypto rx=0x7fb59c005fd0 tx=0x7fb59c032c60 comp rx=0 tx=0).stop 2024-09-21T07:54:30.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.126+0000 7fb5a8e00640 1 -- 172.21.15.39:0/2211611030 shutdown_connections 2024-09-21T07:54:30.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.126+0000 7fb5a8e00640 1 --2- 172.21.15.39:0/2211611030 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fb580078bc0 0x7fb58007b0d0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:30.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.126+0000 7fb5a8e00640 1 --2- 172.21.15.39:0/2211611030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5a415d890 0x7fb5a4074360 unknown :-1 s=CLOSED pgs=2769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:30.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.126+0000 7fb5a8e00640 1 --2- 172.21.15.39:0/2211611030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb5a40af830 0x7fb5a41e52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:30.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.126+0000 7fb5a8e00640 1 --2- 172.21.15.39:0/2211611030 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fb5a40706f0 0x7fb5a41e4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:30.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.126+0000 7fb5a8e00640 1 -- 172.21.15.39:0/2211611030 >> 172.21.15.39:0/2211611030 conn(0x7fb5a406d7a0 msgr2=0x7fb5a4159340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:30.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.127+0000 7fb5a8e00640 1 -- 172.21.15.39:0/2211611030 shutdown_connections 2024-09-21T07:54:30.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.127+0000 7fb5a8e00640 1 -- 172.21.15.39:0/2211611030 wait complete. 2024-09-21T07:54:30.151 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2024-09-21T07:54:30.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.377+0000 7f3a31a00640 1 -- 172.21.15.39:0/1234440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a2c15daf0 msgr2=0x7f3a2c0cf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:30.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.377+0000 7f3a31a00640 1 --2- 172.21.15.39:0/1234440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a2c15daf0 0x7f3a2c0cf450 secure :-1 s=READY pgs=2770 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c0ce9d0 tx=0x7f3a1402a560 comp rx=0 tx=0).stop 2024-09-21T07:54:30.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.377+0000 7f3a31a00640 1 -- 172.21.15.39:0/1234440 shutdown_connections 2024-09-21T07:54:30.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.377+0000 7f3a31a00640 1 --2- 172.21.15.39:0/1234440 >> v2:172.21.15.148:3300/0 conn(0x7f3a2c0cf990 0x7f3a2c0d0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:30.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.377+0000 7f3a31a00640 1 --2- 172.21.15.39:0/1234440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a2c15daf0 0x7f3a2c0cf450 unknown :-1 s=CLOSED pgs=2770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:30.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.377+0000 7f3a31a00640 1 --2- 172.21.15.39:0/1234440 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3a2c073dd0 0x7f3a2c15d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:30.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.377+0000 7f3a31a00640 1 -- 172.21.15.39:0/1234440 >> 172.21.15.39:0/1234440 conn(0x7f3a2c06cfc0 msgr2=0x7f3a2c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:30.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.377+0000 7f3a31a00640 1 -- 172.21.15.39:0/1234440 shutdown_connections 2024-09-21T07:54:30.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.378+0000 7f3a31a00640 1 -- 172.21.15.39:0/1234440 wait complete. 2024-09-21T07:54:30.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.380+0000 7f3a31a00640 1 Processor -- start 2024-09-21T07:54:30.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.380+0000 7f3a31a00640 1 -- start start 2024-09-21T07:54:30.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.381+0000 7f3a31a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a2c073dd0 0x7f3a2c0a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:30.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.381+0000 7f3a31a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3a2c0cf990 0x7f3a2c13e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:30.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.381+0000 7f3a31a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a2c15daf0 0x7f3a2c13eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:30.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.381+0000 7f3a31a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2c160740 con 0x7f3a2c15daf0 2024-09-21T07:54:30.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.381+0000 7f3a31a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2c1605c0 con 0x7f3a2c073dd0 2024-09-21T07:54:30.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.381+0000 7f3a31a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f3a2c1608c0 con 0x7f3a2c0cf990 2024-09-21T07:54:30.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.382+0000 7f3a2b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a2c15daf0 0x7f3a2c13eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:30.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.382+0000 7f3a2b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a2c15daf0 0x7f3a2c13eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41154/0 (socket says 172.21.15.39:41154) 2024-09-21T07:54:30.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.382+0000 7f3a2b400640 1 -- 172.21.15.39:0/3507877039 learned_addr learned my addr 172.21.15.39:0/3507877039 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:30.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.382+0000 7f3a2aa00640 1 --2- 172.21.15.39:0/3507877039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a2c073dd0 0x7f3a2c0a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:30.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.382+0000 7f3a23e00640 1 --2- 172.21.15.39:0/3507877039 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3a2c0cf990 0x7f3a2c13e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:30.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.382+0000 7f3a2b400640 1 -- 172.21.15.39:0/3507877039 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3a2c0cf990 msgr2=0x7f3a2c13e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:30.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.382+0000 7f3a2b400640 1 --2- 172.21.15.39:0/3507877039 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3a2c0cf990 0x7f3a2c13e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:30.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.382+0000 7f3a2b400640 1 -- 172.21.15.39:0/3507877039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a2c073dd0 msgr2=0x7f3a2c0a94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:30.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.382+0000 7f3a2b400640 1 --2- 172.21.15.39:0/3507877039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a2c073dd0 0x7f3a2c0a94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:30.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.382+0000 7f3a2b400640 1 -- 172.21.15.39:0/3507877039 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a2c13f410 con 0x7f3a2c15daf0 2024-09-21T07:54:30.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.382+0000 7f3a2aa00640 1 --2- 172.21.15.39:0/3507877039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a2c073dd0 0x7f3a2c0a94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:30.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.382+0000 7f3a2b400640 1 --2- 172.21.15.39:0/3507877039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a2c15daf0 0x7f3a2c13eae0 secure :-1 s=READY pgs=2771 cs=0 l=1 rev1=1 crypto rx=0x7f3a2400d9e0 tx=0x7f3a2400dea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:30.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.383+0000 7f3a22a00640 1 -- 172.21.15.39:0/3507877039 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a24017070 con 0x7f3a2c15daf0 2024-09-21T07:54:30.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.383+0000 7f3a22a00640 1 -- 172.21.15.39:0/3507877039 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a2400be30 con 0x7f3a2c15daf0 2024-09-21T07:54:30.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.383+0000 7f3a31a00640 1 -- 172.21.15.39:0/3507877039 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a2c1e8d70 con 0x7f3a2c15daf0 2024-09-21T07:54:30.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.383+0000 7f3a22a00640 1 -- 172.21.15.39:0/3507877039 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a24003bb0 con 0x7f3a2c15daf0 2024-09-21T07:54:30.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.383+0000 7f3a31a00640 1 -- 172.21.15.39:0/3507877039 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a2c1e9330 con 0x7f3a2c15daf0 2024-09-21T07:54:30.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.384+0000 7f3a31a00640 1 -- 172.21.15.39:0/3507877039 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a2c0d0630 con 0x7f3a2c15daf0 2024-09-21T07:54:30.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.385+0000 7f3a22a00640 1 -- 172.21.15.39:0/3507877039 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3a24010460 con 0x7f3a2c15daf0 2024-09-21T07:54:30.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.386+0000 7f3a22a00640 1 --2- 172.21.15.39:0/3507877039 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3a08078b90 0x7f3a0807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:30.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.386+0000 7f3a22a00640 1 -- 172.21.15.39:0/3507877039 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(257..257 src has 1..257) ==== 7353+0+0 (secure 0 0 0) 0x7f3a2409b700 con 0x7f3a2c15daf0 2024-09-21T07:54:30.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.387+0000 7f3a2aa00640 1 --2- 172.21.15.39:0/3507877039 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3a08078b90 0x7f3a0807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:30.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.388+0000 7f3a2aa00640 1 --2- 172.21.15.39:0/3507877039 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3a08078b90 0x7f3a0807b0a0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c0730e0 tx=0x7f3a1c005e70 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:30.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.391+0000 7f3a22a00640 1 -- 172.21.15.39:0/3507877039 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3a24062f10 con 0x7f3a2c15daf0 2024-09-21T07:54:30.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.586+0000 7f3a31a00640 1 -- 172.21.15.39:0/3507877039 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) -- 0x7f3a2c13f1a0 con 0x7f3a2c15daf0 2024-09-21T07:54:30.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.587+0000 7f3a22a00640 1 -- 172.21.15.39:0/3507877039 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v257) ==== 103+0+11 (secure 0 0 0) 0x7f3a24067940 con 0x7f3a2c15daf0 2024-09-21T07:54:30.589 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-21T07:54:30.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.589+0000 7f3a31a00640 1 -- 172.21.15.39:0/3507877039 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3a08078b90 msgr2=0x7f3a0807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:30.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.589+0000 7f3a31a00640 1 --2- 172.21.15.39:0/3507877039 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3a08078b90 0x7f3a0807b0a0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c0730e0 tx=0x7f3a1c005e70 comp rx=0 tx=0).stop 2024-09-21T07:54:30.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.589+0000 7f3a31a00640 1 -- 172.21.15.39:0/3507877039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a2c15daf0 msgr2=0x7f3a2c13eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:30.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.589+0000 7f3a31a00640 1 --2- 172.21.15.39:0/3507877039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a2c15daf0 0x7f3a2c13eae0 secure :-1 s=READY pgs=2771 cs=0 l=1 rev1=1 crypto rx=0x7f3a2400d9e0 tx=0x7f3a2400dea0 comp rx=0 tx=0).stop 2024-09-21T07:54:30.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.589+0000 7f3a31a00640 1 -- 172.21.15.39:0/3507877039 shutdown_connections 2024-09-21T07:54:30.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.589+0000 7f3a31a00640 1 --2- 172.21.15.39:0/3507877039 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3a08078b90 0x7f3a0807b0a0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:30.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.589+0000 7f3a31a00640 1 --2- 172.21.15.39:0/3507877039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3a2c15daf0 0x7f3a2c13eae0 unknown :-1 s=CLOSED pgs=2771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:30.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.589+0000 7f3a31a00640 1 --2- 172.21.15.39:0/3507877039 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3a2c0cf990 0x7f3a2c13e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:30.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.589+0000 7f3a31a00640 1 --2- 172.21.15.39:0/3507877039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a2c073dd0 0x7f3a2c0a94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:30.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.589+0000 7f3a31a00640 1 -- 172.21.15.39:0/3507877039 >> 172.21.15.39:0/3507877039 conn(0x7f3a2c06cfc0 msgr2=0x7f3a2c15edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:30.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.589+0000 7f3a31a00640 1 -- 172.21.15.39:0/3507877039 shutdown_connections 2024-09-21T07:54:30.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.590+0000 7f3a31a00640 1 -- 172.21.15.39:0/3507877039 wait complete. 2024-09-21T07:54:30.617 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2024-09-21T07:54:30.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.845+0000 7f3c72e00640 1 -- 172.21.15.39:0/2833807455 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3c6c0b6bd0 msgr2=0x7f3c6c0b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:30.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.845+0000 7f3c72e00640 1 --2- 172.21.15.39:0/2833807455 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3c6c0b6bd0 0x7f3c6c0b5450 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3c64009be0 tx=0x7f3c6402a920 comp rx=0 tx=0).stop 2024-09-21T07:54:30.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.846+0000 7f3c72e00640 1 -- 172.21.15.39:0/2833807455 shutdown_connections 2024-09-21T07:54:30.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.846+0000 7f3c72e00640 1 --2- 172.21.15.39:0/2833807455 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3c6c0b6bd0 0x7f3c6c0b5450 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:30.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.846+0000 7f3c72e00640 1 -- 172.21.15.39:0/2833807455 >> 172.21.15.39:0/2833807455 conn(0x7f3c6c09d7d0 msgr2=0x7f3c6c0d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:30.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.846+0000 7f3c72e00640 1 -- 172.21.15.39:0/2833807455 shutdown_connections 2024-09-21T07:54:30.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.846+0000 7f3c72e00640 1 -- 172.21.15.39:0/2833807455 wait complete. 2024-09-21T07:54:30.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.848+0000 7f3c72e00640 1 Processor -- start 2024-09-21T07:54:30.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.848+0000 7f3c72e00640 1 -- start start 2024-09-21T07:54:30.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.849+0000 7f3c72e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c6c0b6bd0 0x7f3c6c0ce140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:30.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.849+0000 7f3c72e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c6c0b97e0 0x7f3c6c0a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:30.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.849+0000 7f3c72e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3c6c0d2ef0 0x7f3c6c06eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:30.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.850+0000 7f3c72e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3c6c169e90 con 0x7f3c6c0b97e0 2024-09-21T07:54:30.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.850+0000 7f3c72e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3c6c169d10 con 0x7f3c6c0b6bd0 2024-09-21T07:54:30.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.850+0000 7f3c72e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f3c6c16a010 con 0x7f3c6c0d2ef0 2024-09-21T07:54:30.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.850+0000 7f3c71000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c6c0b97e0 0x7f3c6c0a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:30.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.850+0000 7f3c71000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c6c0b97e0 0x7f3c6c0a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41168/0 (socket says 172.21.15.39:41168) 2024-09-21T07:54:30.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.850+0000 7f3c71000640 1 -- 172.21.15.39:0/1783888204 learned_addr learned my addr 172.21.15.39:0/1783888204 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:30.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.850+0000 7f3c72400640 1 --2- 172.21.15.39:0/1783888204 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3c6c0d2ef0 0x7f3c6c06eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:30.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.850+0000 7f3c71a00640 1 --2- 172.21.15.39:0/1783888204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c6c0b6bd0 0x7f3c6c0ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:30.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.850+0000 7f3c71000640 1 -- 172.21.15.39:0/1783888204 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3c6c0d2ef0 msgr2=0x7f3c6c06eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:30.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.850+0000 7f3c71000640 1 --2- 172.21.15.39:0/1783888204 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3c6c0d2ef0 0x7f3c6c06eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:30.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.850+0000 7f3c71000640 1 -- 172.21.15.39:0/1783888204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c6c0b6bd0 msgr2=0x7f3c6c0ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:30.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.850+0000 7f3c71000640 1 --2- 172.21.15.39:0/1783888204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c6c0b6bd0 0x7f3c6c0ce140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:30.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.850+0000 7f3c71000640 1 -- 172.21.15.39:0/1783888204 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c6c0cd580 con 0x7f3c6c0b97e0 2024-09-21T07:54:30.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.850+0000 7f3c71a00640 1 --2- 172.21.15.39:0/1783888204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c6c0b6bd0 0x7f3c6c0ce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:30.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.850+0000 7f3c72400640 1 --2- 172.21.15.39:0/1783888204 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3c6c0d2ef0 0x7f3c6c06eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:30.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.851+0000 7f3c71000640 1 --2- 172.21.15.39:0/1783888204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c6c0b97e0 0x7f3c6c0a0740 secure :-1 s=READY pgs=2773 cs=0 l=1 rev1=1 crypto rx=0x7f3c5800c700 tx=0x7f3c5800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:30.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.851+0000 7f3c69600640 1 -- 172.21.15.39:0/1783888204 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c58016020 con 0x7f3c6c0b97e0 2024-09-21T07:54:30.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.851+0000 7f3c69600640 1 -- 172.21.15.39:0/1783888204 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c5800a950 con 0x7f3c6c0b97e0 2024-09-21T07:54:30.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.851+0000 7f3c72e00640 1 -- 172.21.15.39:0/1783888204 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c6c146420 con 0x7f3c6c0b97e0 2024-09-21T07:54:30.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.851+0000 7f3c72e00640 1 -- 172.21.15.39:0/1783888204 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c6c1469e0 con 0x7f3c6c0b97e0 2024-09-21T07:54:30.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.851+0000 7f3c69600640 1 -- 172.21.15.39:0/1783888204 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5800ada0 con 0x7f3c6c0b97e0 2024-09-21T07:54:30.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.852+0000 7f3c72e00640 1 -- 172.21.15.39:0/1783888204 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c48005290 con 0x7f3c6c0b97e0 2024-09-21T07:54:30.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.854+0000 7f3c69600640 1 -- 172.21.15.39:0/1783888204 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3c58019020 con 0x7f3c6c0b97e0 2024-09-21T07:54:30.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.855+0000 7f3c69600640 1 --2- 172.21.15.39:0/1783888204 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3c4c078b90 0x7f3c4c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:30.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.855+0000 7f3c69600640 1 -- 172.21.15.39:0/1783888204 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(257..257 src has 1..257) ==== 7353+0+0 (secure 0 0 0) 0x7f3c5809b140 con 0x7f3c6c0b97e0 2024-09-21T07:54:30.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.855+0000 7f3c71a00640 1 --2- 172.21.15.39:0/1783888204 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3c4c078b90 0x7f3c4c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:30.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.857+0000 7f3c71a00640 1 --2- 172.21.15.39:0/1783888204 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3c4c078b90 0x7f3c4c07b0a0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f3c60007fd0 tx=0x7f3c60009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:30.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:30.860+0000 7f3c69600640 1 -- 172.21.15.39:0/1783888204 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3c58020030 con 0x7f3c6c0b97e0 2024-09-21T07:54:31.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.059+0000 7f3c72e00640 1 -- 172.21.15.39:0/1783888204 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) -- 0x7f3c48005d90 con 0x7f3c6c0b97e0 2024-09-21T07:54:31.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.059+0000 7f3c69600640 1 -- 172.21.15.39:0/1783888204 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v257) ==== 103+0+11 (secure 0 0 0) 0x7f3c6c0d38f0 con 0x7f3c6c0b97e0 2024-09-21T07:54:31.061 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-21T07:54:31.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.062+0000 7f3c72e00640 1 -- 172.21.15.39:0/1783888204 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3c4c078b90 msgr2=0x7f3c4c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:31.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.062+0000 7f3c72e00640 1 --2- 172.21.15.39:0/1783888204 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3c4c078b90 0x7f3c4c07b0a0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f3c60007fd0 tx=0x7f3c60009040 comp rx=0 tx=0).stop 2024-09-21T07:54:31.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.062+0000 7f3c72e00640 1 -- 172.21.15.39:0/1783888204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c6c0b97e0 msgr2=0x7f3c6c0a0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:31.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.062+0000 7f3c72e00640 1 --2- 172.21.15.39:0/1783888204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c6c0b97e0 0x7f3c6c0a0740 secure :-1 s=READY pgs=2773 cs=0 l=1 rev1=1 crypto rx=0x7f3c5800c700 tx=0x7f3c5800cbc0 comp rx=0 tx=0).stop 2024-09-21T07:54:31.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.062+0000 7f3c72e00640 1 -- 172.21.15.39:0/1783888204 shutdown_connections 2024-09-21T07:54:31.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.062+0000 7f3c72e00640 1 --2- 172.21.15.39:0/1783888204 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3c4c078b90 0x7f3c4c07b0a0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:31.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.062+0000 7f3c72e00640 1 --2- 172.21.15.39:0/1783888204 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3c6c0d2ef0 0x7f3c6c06eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:31.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.062+0000 7f3c72e00640 1 --2- 172.21.15.39:0/1783888204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c6c0b97e0 0x7f3c6c0a0740 unknown :-1 s=CLOSED pgs=2773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:31.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.062+0000 7f3c72e00640 1 --2- 172.21.15.39:0/1783888204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c6c0b6bd0 0x7f3c6c0ce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:31.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.062+0000 7f3c72e00640 1 -- 172.21.15.39:0/1783888204 >> 172.21.15.39:0/1783888204 conn(0x7f3c6c09d7d0 msgr2=0x7f3c6c0b84f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:31.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.062+0000 7f3c72e00640 1 -- 172.21.15.39:0/1783888204 shutdown_connections 2024-09-21T07:54:31.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.062+0000 7f3c72e00640 1 -- 172.21.15.39:0/1783888204 wait complete. 2024-09-21T07:54:31.087 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2024-09-21T07:54:31.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.320+0000 7f8bad600640 1 -- 172.21.15.39:0/767234619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b8c004800 msgr2=0x7f8b8c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:31.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.320+0000 7f8bad600640 1 --2- 172.21.15.39:0/767234619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b8c004800 0x7f8b8c004c10 secure :-1 s=READY pgs=2775 cs=0 l=1 rev1=1 crypto rx=0x7f8b90001830 tx=0x7f8b8c005830 comp rx=0 tx=0).stop 2024-09-21T07:54:31.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.320+0000 7f8ba7400640 1 -- 172.21.15.39:0/767234619 reap_dead start 2024-09-21T07:54:31.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.320+0000 7f8bad600640 1 -- 172.21.15.39:0/767234619 shutdown_connections 2024-09-21T07:54:31.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.320+0000 7f8bad600640 1 -- 172.21.15.39:0/767234619 >> 172.21.15.39:0/767234619 conn(0x7f8ba80b2460 msgr2=0x7f8ba806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:31.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.320+0000 7f8bad600640 1 -- 172.21.15.39:0/767234619 shutdown_connections 2024-09-21T07:54:31.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.320+0000 7f8bad600640 1 -- 172.21.15.39:0/767234619 wait complete. 2024-09-21T07:54:31.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.323+0000 7f8bad600640 1 Processor -- start 2024-09-21T07:54:31.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.323+0000 7f8bad600640 1 -- start start 2024-09-21T07:54:31.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.324+0000 7f8bad600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b8c004800 0x7f8ba815ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:31.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.324+0000 7f8bad600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ba815f7d0 0x7f8ba81501c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:31.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.324+0000 7f8bad600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ba80c7300 0x7f8ba80c7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:31.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.324+0000 7f8bad600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba80af080 con 0x7f8ba815f7d0 2024-09-21T07:54:31.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.324+0000 7f8bad600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba80cd360 con 0x7f8b8c004800 2024-09-21T07:54:31.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.324+0000 7f8bad600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f8ba80af200 con 0x7f8ba80c7300 2024-09-21T07:54:31.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.325+0000 7f8b9fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ba815f7d0 0x7f8ba81501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:31.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.325+0000 7f8b9fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ba815f7d0 0x7f8ba81501c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41180/0 (socket says 172.21.15.39:41180) 2024-09-21T07:54:31.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.325+0000 7f8b9fe00640 1 -- 172.21.15.39:0/3673597105 learned_addr learned my addr 172.21.15.39:0/3673597105 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:31.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.325+0000 7f8ba6a00640 1 --2- 172.21.15.39:0/3673597105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b8c004800 0x7f8ba815ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:31.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.325+0000 7f8ba7400640 1 --2- 172.21.15.39:0/3673597105 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ba80c7300 0x7f8ba80c7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:31.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.325+0000 7f8b9fe00640 1 -- 172.21.15.39:0/3673597105 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ba80c7300 msgr2=0x7f8ba80c7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:31.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.325+0000 7f8b9fe00640 1 --2- 172.21.15.39:0/3673597105 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ba80c7300 0x7f8ba80c7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:31.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.325+0000 7f8b9fe00640 1 -- 172.21.15.39:0/3673597105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b8c004800 msgr2=0x7f8ba815ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:31.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.325+0000 7f8b9fe00640 1 --2- 172.21.15.39:0/3673597105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b8c004800 0x7f8ba815ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:31.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.325+0000 7f8b9fe00640 1 -- 172.21.15.39:0/3673597105 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba815dca0 con 0x7f8ba815f7d0 2024-09-21T07:54:31.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.325+0000 7f8b9fe00640 1 --2- 172.21.15.39:0/3673597105 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ba815f7d0 0x7f8ba81501c0 secure :-1 s=READY pgs=2776 cs=0 l=1 rev1=1 crypto rx=0x7f8b90000f60 tx=0x7f8b90006940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:31.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.326+0000 7f8b9ea00640 1 -- 172.21.15.39:0/3673597105 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b90043070 con 0x7f8ba815f7d0 2024-09-21T07:54:31.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.326+0000 7f8b9ea00640 1 -- 172.21.15.39:0/3673597105 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b900038e0 con 0x7f8ba815f7d0 2024-09-21T07:54:31.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.326+0000 7f8b9ea00640 1 -- 172.21.15.39:0/3673597105 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b90003d30 con 0x7f8ba815f7d0 2024-09-21T07:54:31.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.326+0000 7f8bad600640 1 -- 172.21.15.39:0/3673597105 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ba815df80 con 0x7f8ba815f7d0 2024-09-21T07:54:31.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.326+0000 7f8bad600640 1 -- 172.21.15.39:0/3673597105 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ba815e540 con 0x7f8ba815f7d0 2024-09-21T07:54:31.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.327+0000 7f8bad600640 1 -- 172.21.15.39:0/3673597105 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ba80d0cc0 con 0x7f8ba815f7d0 2024-09-21T07:54:31.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.328+0000 7f8b9ea00640 1 -- 172.21.15.39:0/3673597105 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8b90033480 con 0x7f8ba815f7d0 2024-09-21T07:54:31.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.329+0000 7f8b9ea00640 1 --2- 172.21.15.39:0/3673597105 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8b84078b90 0x7f8b8407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:31.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.329+0000 7f8b9ea00640 1 -- 172.21.15.39:0/3673597105 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(257..257 src has 1..257) ==== 7353+0+0 (secure 0 0 0) 0x7f8b900bb3a0 con 0x7f8ba815f7d0 2024-09-21T07:54:31.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.330+0000 7f8ba6a00640 1 --2- 172.21.15.39:0/3673597105 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8b84078b90 0x7f8b8407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:31.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.330+0000 7f8ba6a00640 1 --2- 172.21.15.39:0/3673597105 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8b84078b90 0x7f8b8407b0a0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f8b98001410 tx=0x7f8b98001320 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:31.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.334+0000 7f8b9ea00640 1 -- 172.21.15.39:0/3673597105 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8ba80d0cc0 con 0x7f8ba815f7d0 2024-09-21T07:54:31.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.528+0000 7f8bad600640 1 -- 172.21.15.39:0/3673597105 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) -- 0x7f8ba815da30 con 0x7f8ba815f7d0 2024-09-21T07:54:31.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.528+0000 7f8b9ea00640 1 -- 172.21.15.39:0/3673597105 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v257) ==== 107+0+11 (secure 0 0 0) 0x7f8b90083160 con 0x7f8ba815f7d0 2024-09-21T07:54:31.531 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-09-21T07:54:31.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.531+0000 7f8bad600640 1 -- 172.21.15.39:0/3673597105 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8b84078b90 msgr2=0x7f8b8407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:31.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.531+0000 7f8bad600640 1 --2- 172.21.15.39:0/3673597105 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8b84078b90 0x7f8b8407b0a0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f8b98001410 tx=0x7f8b98001320 comp rx=0 tx=0).stop 2024-09-21T07:54:31.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.531+0000 7f8bad600640 1 -- 172.21.15.39:0/3673597105 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ba815f7d0 msgr2=0x7f8ba81501c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:31.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.531+0000 7f8bad600640 1 --2- 172.21.15.39:0/3673597105 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ba815f7d0 0x7f8ba81501c0 secure :-1 s=READY pgs=2776 cs=0 l=1 rev1=1 crypto rx=0x7f8b90000f60 tx=0x7f8b90006940 comp rx=0 tx=0).stop 2024-09-21T07:54:31.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.531+0000 7f8bad600640 1 -- 172.21.15.39:0/3673597105 shutdown_connections 2024-09-21T07:54:31.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.531+0000 7f8bad600640 1 --2- 172.21.15.39:0/3673597105 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8b84078b90 0x7f8b8407b0a0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:31.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.531+0000 7f8bad600640 1 --2- 172.21.15.39:0/3673597105 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8ba80c7300 0x7f8ba80c7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:31.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.531+0000 7f8bad600640 1 --2- 172.21.15.39:0/3673597105 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ba815f7d0 0x7f8ba81501c0 unknown :-1 s=CLOSED pgs=2776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:31.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.531+0000 7f8bad600640 1 --2- 172.21.15.39:0/3673597105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b8c004800 0x7f8ba815ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:31.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.531+0000 7f8bad600640 1 -- 172.21.15.39:0/3673597105 >> 172.21.15.39:0/3673597105 conn(0x7f8ba80b2460 msgr2=0x7f8ba806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:31.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.531+0000 7f8bad600640 1 -- 172.21.15.39:0/3673597105 shutdown_connections 2024-09-21T07:54:31.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.531+0000 7f8bad600640 1 -- 172.21.15.39:0/3673597105 wait complete. 2024-09-21T07:54:31.559 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_snapshot_check_access' 2024-09-21T07:54:31.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.784+0000 7fd47f200640 1 -- 172.21.15.39:0/2576407614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4800ce690 msgr2=0x7fd4800b2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:31.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.784+0000 7fd47f200640 1 --2- 172.21.15.39:0/2576407614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4800ce690 0x7fd4800b2f70 secure :-1 s=READY pgs=2777 cs=0 l=1 rev1=1 crypto rx=0x7fd4800ccc60 tx=0x7fd47002a5f0 comp rx=0 tx=0).stop 2024-09-21T07:54:31.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.784+0000 7fd47f200640 1 -- 172.21.15.39:0/2576407614 shutdown_connections 2024-09-21T07:54:31.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.784+0000 7fd47f200640 1 --2- 172.21.15.39:0/2576407614 >> v2:172.21.15.148:3300/0 conn(0x7fd4800d2740 0x7fd4800d0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:31.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.784+0000 7fd47f200640 1 --2- 172.21.15.39:0/2576407614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4800ce690 0x7fd4800b2f70 unknown :-1 s=CLOSED pgs=2777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:31.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.784+0000 7fd47f200640 1 -- 172.21.15.39:0/2576407614 >> 172.21.15.39:0/2576407614 conn(0x7fd48009b5f0 msgr2=0x7fd48008c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:31.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.784+0000 7fd47f200640 1 -- 172.21.15.39:0/2576407614 shutdown_connections 2024-09-21T07:54:31.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.784+0000 7fd47f200640 1 -- 172.21.15.39:0/2576407614 wait complete. 2024-09-21T07:54:31.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.786+0000 7fd47f200640 1 Processor -- start 2024-09-21T07:54:31.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.786+0000 7fd47f200640 1 -- start start 2024-09-21T07:54:31.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.788+0000 7fd47f200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4800ce690 0x7fd48014f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:31.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.788+0000 7fd47f200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd4800d0060 0x7fd48014fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:31.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.788+0000 7fd47f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4800d2740 0x7fd480150000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:31.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.788+0000 7fd47f200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd4800cd290 con 0x7fd4800d2740 2024-09-21T07:54:31.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.788+0000 7fd47f200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd4800cd110 con 0x7fd4800ce690 2024-09-21T07:54:31.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.788+0000 7fd47f200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd4800cd410 con 0x7fd4800d0060 2024-09-21T07:54:31.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.788+0000 7fd47e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4800d2740 0x7fd480150000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:31.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.788+0000 7fd47e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4800d2740 0x7fd480150000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41196/0 (socket says 172.21.15.39:41196) 2024-09-21T07:54:31.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.788+0000 7fd47e800640 1 -- 172.21.15.39:0/1559402946 learned_addr learned my addr 172.21.15.39:0/1559402946 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:31.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.788+0000 7fd47de00640 1 --2- 172.21.15.39:0/1559402946 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4800ce690 0x7fd48014f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:31.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.788+0000 7fd47d400640 1 --2- 172.21.15.39:0/1559402946 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd4800d0060 0x7fd48014fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:31.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.788+0000 7fd47e800640 1 -- 172.21.15.39:0/1559402946 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd4800d0060 msgr2=0x7fd48014fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:31.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.788+0000 7fd47e800640 1 --2- 172.21.15.39:0/1559402946 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd4800d0060 0x7fd48014fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:31.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.788+0000 7fd47e800640 1 -- 172.21.15.39:0/1559402946 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4800ce690 msgr2=0x7fd48014f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:31.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.788+0000 7fd47e800640 1 --2- 172.21.15.39:0/1559402946 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4800ce690 0x7fd48014f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:31.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.788+0000 7fd47e800640 1 -- 172.21.15.39:0/1559402946 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd480152a60 con 0x7fd4800d2740 2024-09-21T07:54:31.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.789+0000 7fd47e800640 1 --2- 172.21.15.39:0/1559402946 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4800d2740 0x7fd480150000 secure :-1 s=READY pgs=2778 cs=0 l=1 rev1=1 crypto rx=0x7fd47800ea40 tx=0x7fd47800ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:31.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.789+0000 7fd476000640 1 -- 172.21.15.39:0/1559402946 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47800cde0 con 0x7fd4800d2740 2024-09-21T07:54:31.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.789+0000 7fd476000640 1 -- 172.21.15.39:0/1559402946 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd478004c10 con 0x7fd4800d2740 2024-09-21T07:54:31.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.789+0000 7fd47f200640 1 -- 172.21.15.39:0/1559402946 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd480150540 con 0x7fd4800d2740 2024-09-21T07:54:31.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.789+0000 7fd476000640 1 -- 172.21.15.39:0/1559402946 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478005020 con 0x7fd4800d2740 2024-09-21T07:54:31.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.789+0000 7fd47f200640 1 -- 172.21.15.39:0/1559402946 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4800cacc0 con 0x7fd4800d2740 2024-09-21T07:54:31.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.790+0000 7fd47f200640 1 -- 172.21.15.39:0/1559402946 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4800cd8c0 con 0x7fd4800d2740 2024-09-21T07:54:31.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.792+0000 7fd476000640 1 -- 172.21.15.39:0/1559402946 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd4780051c0 con 0x7fd4800d2740 2024-09-21T07:54:31.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.792+0000 7fd476000640 1 --2- 172.21.15.39:0/1559402946 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd460078b90 0x7fd46007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:31.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.793+0000 7fd476000640 1 -- 172.21.15.39:0/1559402946 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(257..257 src has 1..257) ==== 7353+0+0 (secure 0 0 0) 0x7fd47809c0d0 con 0x7fd4800d2740 2024-09-21T07:54:31.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.793+0000 7fd47de00640 1 --2- 172.21.15.39:0/1559402946 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd460078b90 0x7fd46007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:31.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.794+0000 7fd47de00640 1 --2- 172.21.15.39:0/1559402946 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd460078b90 0x7fd46007b0a0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fd470034000 tx=0x7fd470004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:31.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:31.797+0000 7fd476000640 1 -- 172.21.15.39:0/1559402946 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd478063e90 con 0x7fd4800d2740 2024-09-21T07:54:32.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.067+0000 7fd47f200640 1 -- 172.21.15.39:0/1559402946 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_snapshot_check_access"]} v 0) -- 0x7fd4800ca770 con 0x7fd4800d2740 2024-09-21T07:54:32.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.214+0000 7fd476000640 1 -- 172.21.15.39:0/1559402946 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_snapshot_check_access"]}]=0 v1032) ==== 150+0+0 (secure 0 0 0) 0x7fd478063790 con 0x7fd4800d2740 2024-09-21T07:54:32.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.215+0000 7fd47f200640 1 -- 172.21.15.39:0/1559402946 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd460078b90 msgr2=0x7fd46007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:32.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.215+0000 7fd47f200640 1 --2- 172.21.15.39:0/1559402946 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd460078b90 0x7fd46007b0a0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fd470034000 tx=0x7fd470004730 comp rx=0 tx=0).stop 2024-09-21T07:54:32.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.215+0000 7fd47f200640 1 -- 172.21.15.39:0/1559402946 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4800d2740 msgr2=0x7fd480150000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:32.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.215+0000 7fd47f200640 1 --2- 172.21.15.39:0/1559402946 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4800d2740 0x7fd480150000 secure :-1 s=READY pgs=2778 cs=0 l=1 rev1=1 crypto rx=0x7fd47800ea40 tx=0x7fd47800ef00 comp rx=0 tx=0).stop 2024-09-21T07:54:32.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.216+0000 7fd47f200640 1 -- 172.21.15.39:0/1559402946 shutdown_connections 2024-09-21T07:54:32.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.216+0000 7fd47f200640 1 --2- 172.21.15.39:0/1559402946 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd460078b90 0x7fd46007b0a0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:32.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.216+0000 7fd47f200640 1 --2- 172.21.15.39:0/1559402946 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4800d2740 0x7fd480150000 unknown :-1 s=CLOSED pgs=2778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:32.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.216+0000 7fd47f200640 1 --2- 172.21.15.39:0/1559402946 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd4800d0060 0x7fd48014fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:32.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.216+0000 7fd47f200640 1 --2- 172.21.15.39:0/1559402946 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4800ce690 0x7fd48014f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:32.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.216+0000 7fd47f200640 1 -- 172.21.15.39:0/1559402946 >> 172.21.15.39:0/1559402946 conn(0x7fd48009b5f0 msgr2=0x7fd4800cf460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:32.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.216+0000 7fd47f200640 1 -- 172.21.15.39:0/1559402946 shutdown_connections 2024-09-21T07:54:32.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.216+0000 7fd47f200640 1 -- 172.21.15.39:0/1559402946 wait complete. 2024-09-21T07:54:32.244 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T07:54:32.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.469+0000 7f9861000640 1 -- 172.21.15.39:0/3974565117 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f985c0cf450 msgr2=0x7f985c0cf860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:32.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.469+0000 7f9861000640 1 --2- 172.21.15.39:0/3974565117 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f985c0cf450 0x7f985c0cf860 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9848009830 tx=0x7f984802a520 comp rx=0 tx=0).stop 2024-09-21T07:54:32.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.469+0000 7f9861000640 1 -- 172.21.15.39:0/3974565117 shutdown_connections 2024-09-21T07:54:32.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.469+0000 7f9861000640 1 --2- 172.21.15.39:0/3974565117 >> v2:172.21.15.148:3300/0 conn(0x7f985c0d0630 0x7f985c0d0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:32.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.469+0000 7f9861000640 1 --2- 172.21.15.39:0/3974565117 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f985c0cf450 0x7f985c0cf860 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:32.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.469+0000 7f9861000640 1 -- 172.21.15.39:0/3974565117 >> 172.21.15.39:0/3974565117 conn(0x7f985c06cfc0 msgr2=0x7f985c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:32.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.469+0000 7f9861000640 1 -- 172.21.15.39:0/3974565117 shutdown_connections 2024-09-21T07:54:32.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.469+0000 7f9861000640 1 -- 172.21.15.39:0/3974565117 wait complete. 2024-09-21T07:54:32.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.471+0000 7f9861000640 1 Processor -- start 2024-09-21T07:54:32.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.471+0000 7f9861000640 1 -- start start 2024-09-21T07:54:32.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.473+0000 7f9861000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f985c073dd0 0x7f985c1e95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:32.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.473+0000 7f9861000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f985c0cf450 0x7f985c1e9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:32.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.473+0000 7f9861000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f985c0d0630 0x7f985c1f9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:32.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.473+0000 7f9861000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f985c0cd1e0 con 0x7f985c073dd0 2024-09-21T07:54:32.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.473+0000 7f9861000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f985c0cd060 con 0x7f985c0d0630 2024-09-21T07:54:32.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.473+0000 7f9861000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f985c0cd360 con 0x7f985c0cf450 2024-09-21T07:54:32.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.473+0000 7f9859e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f985c073dd0 0x7f985c1e95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:32.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.473+0000 7f9859e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f985c073dd0 0x7f985c1e95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37990/0 (socket says 172.21.15.39:37990) 2024-09-21T07:54:32.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.473+0000 7f9859e00640 1 -- 172.21.15.39:0/3230387490 learned_addr learned my addr 172.21.15.39:0/3230387490 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:32.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.473+0000 7f9859400640 1 --2- 172.21.15.39:0/3230387490 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f985c0cf450 0x7f985c1e9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:32.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.473+0000 7f985a800640 1 --2- 172.21.15.39:0/3230387490 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f985c0d0630 0x7f985c1f9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:32.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.474+0000 7f9859e00640 1 -- 172.21.15.39:0/3230387490 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f985c0cf450 msgr2=0x7f985c1e9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:32.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.474+0000 7f9859e00640 1 --2- 172.21.15.39:0/3230387490 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f985c0cf450 0x7f985c1e9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:32.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.474+0000 7f9859e00640 1 -- 172.21.15.39:0/3230387490 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f985c0d0630 msgr2=0x7f985c1f9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:32.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.474+0000 7f9859e00640 1 --2- 172.21.15.39:0/3230387490 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f985c0d0630 0x7f985c1f9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:32.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.474+0000 7f9859e00640 1 -- 172.21.15.39:0/3230387490 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f985c1515b0 con 0x7f985c073dd0 2024-09-21T07:54:32.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.474+0000 7f9859e00640 1 --2- 172.21.15.39:0/3230387490 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f985c073dd0 0x7f985c1e95a0 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 crypto rx=0x7f984c00c340 tx=0x7f984c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:32.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.474+0000 7f9852000640 1 -- 172.21.15.39:0/3230387490 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984c0052d0 con 0x7f985c073dd0 2024-09-21T07:54:32.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.474+0000 7f9852000640 1 -- 172.21.15.39:0/3230387490 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f984c007020 con 0x7f985c073dd0 2024-09-21T07:54:32.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.474+0000 7f9852000640 1 -- 172.21.15.39:0/3230387490 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984c007470 con 0x7f985c073dd0 2024-09-21T07:54:32.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.474+0000 7f9861000640 1 -- 172.21.15.39:0/3230387490 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f985c1ea020 con 0x7f985c073dd0 2024-09-21T07:54:32.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.474+0000 7f9861000640 1 -- 172.21.15.39:0/3230387490 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f985c151a00 con 0x7f985c073dd0 2024-09-21T07:54:32.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.476+0000 7f9861000640 1 -- 172.21.15.39:0/3230387490 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f985c0cd810 con 0x7f985c073dd0 2024-09-21T07:54:32.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.477+0000 7f9852000640 1 -- 172.21.15.39:0/3230387490 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f984c0067d0 con 0x7f985c073dd0 2024-09-21T07:54:32.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.478+0000 7f9852000640 1 --2- 172.21.15.39:0/3230387490 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f983c078b90 0x7f983c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:32.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.478+0000 7f9852000640 1 -- 172.21.15.39:0/3230387490 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(257..257 src has 1..257) ==== 7353+0+0 (secure 0 0 0) 0x7f984c09a7c0 con 0x7f985c073dd0 2024-09-21T07:54:32.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.478+0000 7f9859400640 1 --2- 172.21.15.39:0/3230387490 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f983c078b90 0x7f983c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:32.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.479+0000 7f9859400640 1 --2- 172.21.15.39:0/3230387490 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f983c078b90 0x7f983c07b0a0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f9848034000 tx=0x7f9848004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:32.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.482+0000 7f9852000640 1 -- 172.21.15.39:0/3230387490 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f985c1ea020 con 0x7f985c073dd0 2024-09-21T07:54:32.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.680+0000 7f9861000640 1 -- 172.21.15.39:0/3230387490 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f985c050ff0 con 0x7f985c073dd0 2024-09-21T07:54:32.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.680+0000 7f9852000640 1 -- 172.21.15.39:0/3230387490 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v56) ==== 59+0+2294 (secure 0 0 0) 0x7f985c1515b0 con 0x7f985c073dd0 2024-09-21T07:54:32.682 INFO:teuthology.orchestra.run.smithi039.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T07:54:32.682 INFO:teuthology.orchestra.run.smithi039.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T07:54:32.683 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced client_mount_timeout 30 2024-09-21T07:54:32.683 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_client 20/20 2024-09-21T07:54:32.683 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_mgr 20/20 2024-09-21T07:54:32.683 INFO:teuthology.orchestra.run.smithi039.stdout:mgr advanced debug_ms 1/1 2024-09-21T07:54:32.683 INFO:teuthology.orchestra.run.smithi039.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T07:54:32.683 INFO:teuthology.orchestra.run.smithi039.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T07:54:32.683 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3421.881759 2024-09-21T07:54:32.683 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3520.252978 2024-09-21T07:54:32.683 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3511.183316 2024-09-21T07:54:32.683 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3829.631847 2024-09-21T07:54:32.683 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 5474.545872 2024-09-21T07:54:32.683 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4013.628559 2024-09-21T07:54:32.683 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4230.187288 2024-09-21T07:54:32.683 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4148.983107 2024-09-21T07:54:32.684 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_mds 20/20 2024-09-21T07:54:32.684 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T07:54:32.684 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced debug_ms 2/2 2024-09-21T07:54:32.684 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_debug_frag true 2024-09-21T07:54:32.684 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_debug_scatterstat true 2024-09-21T07:54:32.684 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T07:54:32.684 INFO:teuthology.orchestra.run.smithi039.stdout:mds dev mds_verify_scatter true 2024-09-21T07:54:32.684 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T07:54:32.684 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T07:54:32.684 INFO:teuthology.orchestra.run.smithi039.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T07:54:32.684 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced client_mount_timeout 600 2024-09-21T07:54:32.684 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced debug_client 20/20 2024-09-21T07:54:32.684 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced debug_ms 1/1 2024-09-21T07:54:32.684 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T07:54:32.685 INFO:teuthology.orchestra.run.smithi039.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T07:54:32.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.683+0000 7f9861000640 1 -- 172.21.15.39:0/3230387490 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f983c078b90 msgr2=0x7f983c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:32.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.683+0000 7f9861000640 1 --2- 172.21.15.39:0/3230387490 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f983c078b90 0x7f983c07b0a0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f9848034000 tx=0x7f9848004730 comp rx=0 tx=0).stop 2024-09-21T07:54:32.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.683+0000 7f9861000640 1 -- 172.21.15.39:0/3230387490 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f985c073dd0 msgr2=0x7f985c1e95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:32.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.683+0000 7f9861000640 1 --2- 172.21.15.39:0/3230387490 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f985c073dd0 0x7f985c1e95a0 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 crypto rx=0x7f984c00c340 tx=0x7f984c00c800 comp rx=0 tx=0).stop 2024-09-21T07:54:32.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.683+0000 7f9861000640 1 -- 172.21.15.39:0/3230387490 shutdown_connections 2024-09-21T07:54:32.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.683+0000 7f9861000640 1 --2- 172.21.15.39:0/3230387490 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f983c078b90 0x7f983c07b0a0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:32.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.683+0000 7f9861000640 1 --2- 172.21.15.39:0/3230387490 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f985c0d0630 0x7f985c1f9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:32.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.683+0000 7f9861000640 1 --2- 172.21.15.39:0/3230387490 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f985c0cf450 0x7f985c1e9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:32.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.683+0000 7f9861000640 1 --2- 172.21.15.39:0/3230387490 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f985c073dd0 0x7f985c1e95a0 unknown :-1 s=CLOSED pgs=2779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:32.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.683+0000 7f9861000640 1 -- 172.21.15.39:0/3230387490 >> 172.21.15.39:0/3230387490 conn(0x7f985c06cfc0 msgr2=0x7f985c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:32.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.683+0000 7f9861000640 1 -- 172.21.15.39:0/3230387490 shutdown_connections 2024-09-21T07:54:32.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.683+0000 7f9861000640 1 -- 172.21.15.39:0/3230387490 wait complete. 2024-09-21T07:54:32.707 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T07:54:32.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.935+0000 7f75eae00640 1 -- 172.21.15.39:0/2943828957 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75e40b6c90 msgr2=0x7f75e40cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:32.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.935+0000 7f75eae00640 1 --2- 172.21.15.39:0/2943828957 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75e40b6c90 0x7f75e40cf7e0 secure :-1 s=READY pgs=2780 cs=0 l=1 rev1=1 crypto rx=0x7f75e40b5490 tx=0x7f75d802a4a0 comp rx=0 tx=0).stop 2024-09-21T07:54:32.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.935+0000 7f75eae00640 1 -- 172.21.15.39:0/2943828957 shutdown_connections 2024-09-21T07:54:32.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.935+0000 7f75eae00640 1 --2- 172.21.15.39:0/2943828957 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f75e40d25a0 0x7f75e40d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:32.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.935+0000 7f75eae00640 1 --2- 172.21.15.39:0/2943828957 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75e40b6c90 0x7f75e40cf7e0 unknown :-1 s=CLOSED pgs=2780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:32.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.935+0000 7f75eae00640 1 -- 172.21.15.39:0/2943828957 >> 172.21.15.39:0/2943828957 conn(0x7f75e409d7b0 msgr2=0x7f75e40d44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:32.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.935+0000 7f75eae00640 1 -- 172.21.15.39:0/2943828957 shutdown_connections 2024-09-21T07:54:32.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.935+0000 7f75eae00640 1 -- 172.21.15.39:0/2943828957 wait complete. 2024-09-21T07:54:32.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.937+0000 7f75eae00640 1 Processor -- start 2024-09-21T07:54:32.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.937+0000 7f75eae00640 1 -- start start 2024-09-21T07:54:32.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.939+0000 7f75eae00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75e40b5f60 0x7f75e4164c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:32.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.939+0000 7f75eae00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f75e40b6c90 0x7f75e41653a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:32.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.939+0000 7f75eae00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75e40d25a0 0x7f75e41494f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:32.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.939+0000 7f75eae00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f75e40cfea0 con 0x7f75e40b5f60 2024-09-21T07:54:32.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.939+0000 7f75eae00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f75e40cfd20 con 0x7f75e40d25a0 2024-09-21T07:54:32.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.939+0000 7f75eae00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f75e40b9e60 con 0x7f75e40b6c90 2024-09-21T07:54:32.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.939+0000 7f75e9a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75e40b5f60 0x7f75e4164c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:32.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.939+0000 7f75e9a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75e40b5f60 0x7f75e4164c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38012/0 (socket says 172.21.15.39:38012) 2024-09-21T07:54:32.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.939+0000 7f75e9a00640 1 -- 172.21.15.39:0/3518080885 learned_addr learned my addr 172.21.15.39:0/3518080885 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:32.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.939+0000 7f75e9000640 1 --2- 172.21.15.39:0/3518080885 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f75e40b6c90 0x7f75e41653a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:32.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.939+0000 7f75ea400640 1 --2- 172.21.15.39:0/3518080885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75e40d25a0 0x7f75e41494f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:32.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.940+0000 7f75e9a00640 1 -- 172.21.15.39:0/3518080885 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f75e40b6c90 msgr2=0x7f75e41653a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:32.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.940+0000 7f75e9a00640 1 --2- 172.21.15.39:0/3518080885 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f75e40b6c90 0x7f75e41653a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:32.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.940+0000 7f75e9a00640 1 -- 172.21.15.39:0/3518080885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75e40d25a0 msgr2=0x7f75e41494f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:32.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.940+0000 7f75e9a00640 1 --2- 172.21.15.39:0/3518080885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75e40d25a0 0x7f75e41494f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:32.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.940+0000 7f75e9a00640 1 -- 172.21.15.39:0/3518080885 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75e41658e0 con 0x7f75e40b5f60 2024-09-21T07:54:32.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.940+0000 7f75e9a00640 1 --2- 172.21.15.39:0/3518080885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75e40b5f60 0x7f75e4164c10 secure :-1 s=READY pgs=2781 cs=0 l=1 rev1=1 crypto rx=0x7f75d8000c00 tx=0x7f75d80039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:32.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.940+0000 7f75e1600640 1 -- 172.21.15.39:0/3518080885 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d803e030 con 0x7f75e40b5f60 2024-09-21T07:54:32.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.940+0000 7f75e1600640 1 -- 172.21.15.39:0/3518080885 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75d80047a0 con 0x7f75e40b5f60 2024-09-21T07:54:32.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.940+0000 7f75eae00640 1 -- 172.21.15.39:0/3518080885 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75e415df40 con 0x7f75e40b5f60 2024-09-21T07:54:32.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.940+0000 7f75e1600640 1 -- 172.21.15.39:0/3518080885 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d8035040 con 0x7f75e40b5f60 2024-09-21T07:54:32.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.941+0000 7f75eae00640 1 -- 172.21.15.39:0/3518080885 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75e414a0f0 con 0x7f75e40b5f60 2024-09-21T07:54:32.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.942+0000 7f75eae00640 1 -- 172.21.15.39:0/3518080885 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75e40c7d20 con 0x7f75e40b5f60 2024-09-21T07:54:32.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.943+0000 7f75e1600640 1 -- 172.21.15.39:0/3518080885 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f75d8041030 con 0x7f75e40b5f60 2024-09-21T07:54:32.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.944+0000 7f75e1600640 1 --2- 172.21.15.39:0/3518080885 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f75c4078bc0 0x7f75c407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:32.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.944+0000 7f75e1600640 1 -- 172.21.15.39:0/3518080885 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(257..257 src has 1..257) ==== 7353+0+0 (secure 0 0 0) 0x7f75d80bbba0 con 0x7f75e40b5f60 2024-09-21T07:54:32.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.944+0000 7f75e9000640 1 --2- 172.21.15.39:0/3518080885 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f75c4078bc0 0x7f75c407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:32.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.945+0000 7f75e9000640 1 --2- 172.21.15.39:0/3518080885 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f75c4078bc0 0x7f75c407b0d0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f75e406de30 tx=0x7f75d0009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:32.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:32.948+0000 7f75e1600640 1 -- 172.21.15.39:0/3518080885 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f75d8083310 con 0x7f75e40b5f60 2024-09-21T07:54:33.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.145+0000 7f75eae00640 1 -- 172.21.15.39:0/3518080885 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f75e41634b0 con 0x7f75e40b5f60 2024-09-21T07:54:33.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.150+0000 7f75e1600640 1 -- 172.21.15.39:0/3518080885 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v57)=0 v57) ==== 122+0+0 (secure 0 0 0) 0x7f75d8087d40 con 0x7f75e40b5f60 2024-09-21T07:54:33.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.152+0000 7f75eae00640 1 -- 172.21.15.39:0/3518080885 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f75c4078bc0 msgr2=0x7f75c407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:33.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.152+0000 7f75eae00640 1 --2- 172.21.15.39:0/3518080885 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f75c4078bc0 0x7f75c407b0d0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f75e406de30 tx=0x7f75d0009040 comp rx=0 tx=0).stop 2024-09-21T07:54:33.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.152+0000 7f75eae00640 1 -- 172.21.15.39:0/3518080885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75e40b5f60 msgr2=0x7f75e4164c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:33.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.152+0000 7f75eae00640 1 --2- 172.21.15.39:0/3518080885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75e40b5f60 0x7f75e4164c10 secure :-1 s=READY pgs=2781 cs=0 l=1 rev1=1 crypto rx=0x7f75d8000c00 tx=0x7f75d80039e0 comp rx=0 tx=0).stop 2024-09-21T07:54:33.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.152+0000 7f75eae00640 1 -- 172.21.15.39:0/3518080885 shutdown_connections 2024-09-21T07:54:33.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.152+0000 7f75eae00640 1 --2- 172.21.15.39:0/3518080885 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f75c4078bc0 0x7f75c407b0d0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:33.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.152+0000 7f75eae00640 1 --2- 172.21.15.39:0/3518080885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75e40d25a0 0x7f75e41494f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:33.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.152+0000 7f75eae00640 1 --2- 172.21.15.39:0/3518080885 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f75e40b6c90 0x7f75e41653a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:33.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.152+0000 7f75eae00640 1 --2- 172.21.15.39:0/3518080885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75e40b5f60 0x7f75e4164c10 unknown :-1 s=CLOSED pgs=2781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:33.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.152+0000 7f75eae00640 1 -- 172.21.15.39:0/3518080885 >> 172.21.15.39:0/3518080885 conn(0x7f75e409d7b0 msgr2=0x7f75e415f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:33.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.152+0000 7f75eae00640 1 -- 172.21.15.39:0/3518080885 shutdown_connections 2024-09-21T07:54:33.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.153+0000 7f75eae00640 1 -- 172.21.15.39:0/3518080885 wait complete. 2024-09-21T07:54:33.178 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:54:33.178 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:54:33.259 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:54:33.339 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:54:33.339 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:54:33.339 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:33.375 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:54:33.375 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:54:33.375 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:33.411 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:54:33.411 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:54:33.411 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:33.450 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:54:33.451 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:54:33.451 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:33.486 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:54:33.486 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T07:54:33.486 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:54:33.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.716+0000 7f4a62800640 1 -- 172.21.15.39:0/2075453177 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a5c0b5240 msgr2=0x7f4a5c15c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:33.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.716+0000 7f4a62800640 1 --2- 172.21.15.39:0/2075453177 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a5c0b5240 0x7f4a5c15c690 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c009830 tx=0x7f4a4c02a520 comp rx=0 tx=0).stop 2024-09-21T07:54:33.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.716+0000 7f4a62800640 1 -- 172.21.15.39:0/2075453177 shutdown_connections 2024-09-21T07:54:33.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.716+0000 7f4a62800640 1 --2- 172.21.15.39:0/2075453177 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a5c0b5240 0x7f4a5c15c690 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:33.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.716+0000 7f4a62800640 1 -- 172.21.15.39:0/2075453177 >> 172.21.15.39:0/2075453177 conn(0x7f4a5c06cfc0 msgr2=0x7f4a5c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:33.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.716+0000 7f4a62800640 1 -- 172.21.15.39:0/2075453177 shutdown_connections 2024-09-21T07:54:33.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.716+0000 7f4a62800640 1 -- 172.21.15.39:0/2075453177 wait complete. 2024-09-21T07:54:33.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.718+0000 7f4a62800640 1 Processor -- start 2024-09-21T07:54:33.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.719+0000 7f4a62800640 1 -- start start 2024-09-21T07:54:33.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.720+0000 7f4a62800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a5c0b5240 0x7f4a5c0b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:33.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.720+0000 7f4a62800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a5c0cf920 0x7f4a5c1f5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:33.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.720+0000 7f4a62800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a5c0b1080 0x7f4a5c0b1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:33.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.720+0000 7f4a62800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c0b4ae0 con 0x7f4a5c0cf920 2024-09-21T07:54:33.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.720+0000 7f4a62800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c0cff40 con 0x7f4a5c0b5240 2024-09-21T07:54:33.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.720+0000 7f4a62800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f4a5c0b4c60 con 0x7f4a5c0b1080 2024-09-21T07:54:33.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.720+0000 7f4a53e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a5c0cf920 0x7f4a5c1f5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:33.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.720+0000 7f4a53e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a5c0cf920 0x7f4a5c1f5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38026/0 (socket says 172.21.15.39:38026) 2024-09-21T07:54:33.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.720+0000 7f4a53e00640 1 -- 172.21.15.39:0/4287622230 learned_addr learned my addr 172.21.15.39:0/4287622230 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:33.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.720+0000 7f4a5b400640 1 --2- 172.21.15.39:0/4287622230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a5c0b5240 0x7f4a5c0b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:33.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.720+0000 7f4a5be00640 1 --2- 172.21.15.39:0/4287622230 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a5c0b1080 0x7f4a5c0b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:33.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.721+0000 7f4a53e00640 1 -- 172.21.15.39:0/4287622230 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a5c0b1080 msgr2=0x7f4a5c0b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:33.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.721+0000 7f4a53e00640 1 --2- 172.21.15.39:0/4287622230 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a5c0b1080 0x7f4a5c0b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:33.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.721+0000 7f4a53e00640 1 -- 172.21.15.39:0/4287622230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a5c0b5240 msgr2=0x7f4a5c0b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:33.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.721+0000 7f4a53e00640 1 --2- 172.21.15.39:0/4287622230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a5c0b5240 0x7f4a5c0b1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:33.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.721+0000 7f4a53e00640 1 -- 172.21.15.39:0/4287622230 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a5c0b19d0 con 0x7f4a5c0cf920 2024-09-21T07:54:33.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.721+0000 7f4a53e00640 1 --2- 172.21.15.39:0/4287622230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a5c0cf920 0x7f4a5c1f5d60 secure :-1 s=READY pgs=2783 cs=0 l=1 rev1=1 crypto rx=0x7f4a4400c400 tx=0x7f4a4400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:33.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.721+0000 7f4a53400640 1 -- 172.21.15.39:0/4287622230 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a44017070 con 0x7f4a5c0cf920 2024-09-21T07:54:33.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.721+0000 7f4a53400640 1 -- 172.21.15.39:0/4287622230 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a440053c0 con 0x7f4a5c0cf920 2024-09-21T07:54:33.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.721+0000 7f4a53400640 1 -- 172.21.15.39:0/4287622230 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a44003850 con 0x7f4a5c0cf920 2024-09-21T07:54:33.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.721+0000 7f4a62800640 1 -- 172.21.15.39:0/4287622230 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a5c1e3ed0 con 0x7f4a5c0cf920 2024-09-21T07:54:33.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.722+0000 7f4a62800640 1 -- 172.21.15.39:0/4287622230 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a5c1f67d0 con 0x7f4a5c0cf920 2024-09-21T07:54:33.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.722+0000 7f4a62800640 1 -- 172.21.15.39:0/4287622230 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a5c15c690 con 0x7f4a5c0cf920 2024-09-21T07:54:33.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.724+0000 7f4a53400640 1 -- 172.21.15.39:0/4287622230 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4a4401a070 con 0x7f4a5c0cf920 2024-09-21T07:54:33.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.725+0000 7f4a53400640 1 --2- 172.21.15.39:0/4287622230 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4a38078b90 0x7f4a3807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:33.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.725+0000 7f4a53400640 1 -- 172.21.15.39:0/4287622230 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(257..257 src has 1..257) ==== 7353+0+0 (secure 0 0 0) 0x7f4a4409a770 con 0x7f4a5c0cf920 2024-09-21T07:54:33.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.725+0000 7f4a5b400640 1 --2- 172.21.15.39:0/4287622230 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4a38078b90 0x7f4a3807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:33.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.726+0000 7f4a5b400640 1 --2- 172.21.15.39:0/4287622230 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4a38078b90 0x7f4a3807b0a0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c034000 tx=0x7f4a4c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:33.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.729+0000 7f4a53400640 1 -- 172.21.15.39:0/4287622230 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4a5c0b19d0 con 0x7f4a5c0cf920 2024-09-21T07:54:33.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.962+0000 7f4a62800640 1 -- 172.21.15.39:0/4287622230 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a5c156000 con 0x7f4a5c0cf920 2024-09-21T07:54:33.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.963+0000 7f4a53400640 1 -- 172.21.15.39:0/4287622230 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 303 v303) ==== 95+0+3921 (secure 0 0 0) 0x7f4a44061e50 con 0x7f4a5c0cf920 2024-09-21T07:54:33.965 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:33.965 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":303,"btime":"2024-09-21T07:54:25:147921+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9467,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/3658109005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":3658109005},{"type":"v1","addr":"172.21.15.148:6835","nonce":3658109005}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":298},{"gid":9635,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/4138851560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":4138851560},{"type":"v1","addr":"172.21.15.148:6837","nonce":4138851560}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":303},{"gid":12201,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/2437051680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2437051680},{"type":"v1","addr":"172.21.15.39:6833","nonce":2437051680}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":303},{"gid":12210,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/501411360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":501411360},{"type":"v1","addr":"172.21.15.39:6835","nonce":501411360}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":303}],"filesystems":[]} 2024-09-21T07:54:33.965 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 303 2024-09-21T07:54:33.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.965+0000 7f4a62800640 1 -- 172.21.15.39:0/4287622230 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4a38078b90 msgr2=0x7f4a3807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:33.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.965+0000 7f4a62800640 1 --2- 172.21.15.39:0/4287622230 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4a38078b90 0x7f4a3807b0a0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c034000 tx=0x7f4a4c004730 comp rx=0 tx=0).stop 2024-09-21T07:54:33.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.965+0000 7f4a62800640 1 -- 172.21.15.39:0/4287622230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a5c0cf920 msgr2=0x7f4a5c1f5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:33.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.965+0000 7f4a62800640 1 --2- 172.21.15.39:0/4287622230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a5c0cf920 0x7f4a5c1f5d60 secure :-1 s=READY pgs=2783 cs=0 l=1 rev1=1 crypto rx=0x7f4a4400c400 tx=0x7f4a4400c8c0 comp rx=0 tx=0).stop 2024-09-21T07:54:33.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.965+0000 7f4a62800640 1 -- 172.21.15.39:0/4287622230 shutdown_connections 2024-09-21T07:54:33.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.965+0000 7f4a62800640 1 --2- 172.21.15.39:0/4287622230 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4a38078b90 0x7f4a3807b0a0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:33.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.965+0000 7f4a62800640 1 --2- 172.21.15.39:0/4287622230 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4a5c0b1080 0x7f4a5c0b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:33.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.965+0000 7f4a62800640 1 --2- 172.21.15.39:0/4287622230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a5c0cf920 0x7f4a5c1f5d60 unknown :-1 s=CLOSED pgs=2783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:33.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.965+0000 7f4a62800640 1 --2- 172.21.15.39:0/4287622230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a5c0b5240 0x7f4a5c0b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:33.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.965+0000 7f4a62800640 1 -- 172.21.15.39:0/4287622230 >> 172.21.15.39:0/4287622230 conn(0x7f4a5c06cfc0 msgr2=0x7f4a5c15e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:33.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.966+0000 7f4a62800640 1 -- 172.21.15.39:0/4287622230 shutdown_connections 2024-09-21T07:54:33.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:33.966+0000 7f4a62800640 1 -- 172.21.15.39:0/4287622230 wait complete. 2024-09-21T07:54:33.993 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T07:54:33.993 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T07:54:33.994 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T07:54:33.994 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:54:33.994 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T07:54:33.994 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T07:54:33.994 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T07:54:33.994 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:54:33.994 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T07:54:33.994 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T07:54:33.995 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T07:54:33.995 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:54:33.995 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T07:54:33.995 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T07:54:33.995 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T07:54:33.995 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:54:34.017 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T07:54:34.017 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T07:54:34.019 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T07:54:34.019 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T07:54:34.020 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T07:54:34.020 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T07:54:34.020 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T07:54:34.021 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T07:54:34.064 INFO:tasks.ceph.mds.c:Started 2024-09-21T07:54:34.064 DEBUG:teuthology.parallel:result is None 2024-09-21T07:54:34.065 INFO:tasks.ceph.mds.d:Started 2024-09-21T07:54:34.065 DEBUG:teuthology.parallel:result is None 2024-09-21T07:54:34.067 INFO:tasks.ceph.mds.a:Started 2024-09-21T07:54:34.067 DEBUG:teuthology.parallel:result is None 2024-09-21T07:54:34.068 INFO:tasks.ceph.mds.b:Started 2024-09-21T07:54:34.068 DEBUG:teuthology.parallel:result is None 2024-09-21T07:54:34.068 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T07:54:34.263 INFO:tasks.ceph.mds.b.smithi148.stdout:starting mds.b at 2024-09-21T07:54:34.263 INFO:tasks.ceph.mds.d.smithi148.stdout:starting mds.d at 2024-09-21T07:54:34.264 INFO:tasks.ceph.mds.a.smithi039.stdout:starting mds.a at 2024-09-21T07:54:34.265 INFO:tasks.ceph.mds.c.smithi039.stdout:starting mds.c at 2024-09-21T07:54:34.265 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:54:34.270 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:54:34.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:34.303+0000 7f021c800640 1 -- 172.21.15.39:0/3896507185 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02140b30a0 msgr2=0x7f02140cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:34.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:34.303+0000 7f021c800640 1 --2- 172.21.15.39:0/3896507185 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02140b30a0 0x7f02140cb8a0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f0208009830 tx=0x7f020802a530 comp rx=0 tx=0).stop 2024-09-21T07:54:34.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:34.303+0000 7f021c800640 1 -- 172.21.15.39:0/3896507185 shutdown_connections 2024-09-21T07:54:34.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:34.303+0000 7f021c800640 1 --2- 172.21.15.39:0/3896507185 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02140b30a0 0x7f02140cb8a0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:34.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:34.303+0000 7f021c800640 1 -- 172.21.15.39:0/3896507185 >> 172.21.15.39:0/3896507185 conn(0x7f02140b2460 msgr2=0x7f021406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:34.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:34.303+0000 7f021c800640 1 -- 172.21.15.39:0/3896507185 shutdown_connections 2024-09-21T07:54:34.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:34.303+0000 7f021c800640 1 -- 172.21.15.39:0/3896507185 wait complete. 2024-09-21T07:54:34.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:34.306+0000 7f021c800640 1 Processor -- start 2024-09-21T07:54:34.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:34.306+0000 7f021c800640 1 -- start start 2024-09-21T07:54:34.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:34.307+0000 7f021c800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f02140b30a0 0x7f02141f54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:34.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:34.307+0000 7f021c800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02140ccc60 0x7f02140ce790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:34.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:34.307+0000 7f021c800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f021415d830 0x7f02140cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:34.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:34.307+0000 7f021c800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f02140af520 con 0x7f021415d830 2024-09-21T07:54:34.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:34.307+0000 7f021c800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f02140af3a0 con 0x7f02140ccc60 2024-09-21T07:54:34.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:34.307+0000 7f021c800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f02140af6a0 con 0x7f02140b30a0 2024-09-21T07:54:34.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:34.308+0000 7f021a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f021415d830 0x7f02140cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:34.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:34.308+0000 7f0219400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02140ccc60 0x7f02140ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:34.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:34.308+0000 7f021a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f021415d830 0x7f02140cecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38072/0 (socket says 172.21.15.39:38072) 2024-09-21T07:54:34.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:34.308+0000 7f021a800640 1 -- 172.21.15.39:0/4010429561 learned_addr learned my addr 172.21.15.39:0/4010429561 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:34.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:34.308+0000 7f0219e00640 1 --2- 172.21.15.39:0/4010429561 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f02140b30a0 0x7f02141f54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:34.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:34.308+0000 7f0219400640 1 -- 172.21.15.39:0/4010429561 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f02140b30a0 msgr2=0x7f02141f54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:34.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:34.308+0000 7f0219400640 1 --2- 172.21.15.39:0/4010429561 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f02140b30a0 0x7f02141f54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:34.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:34.308+0000 7f0219400640 1 -- 172.21.15.39:0/4010429561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f021415d830 msgr2=0x7f02140cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:34.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:34.308+0000 7f0219400640 1 --2- 172.21.15.39:0/4010429561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f021415d830 0x7f02140cecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:34.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:34.308+0000 7f0219400640 1 -- 172.21.15.39:0/4010429561 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02140b17c0 con 0x7f02140ccc60 2024-09-21T07:54:34.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:34.308+0000 7f021a800640 1 --2- 172.21.15.39:0/4010429561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f021415d830 0x7f02140cecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:54:34.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:34.309+0000 7f0219400640 1 --2- 172.21.15.39:0/4010429561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02140ccc60 0x7f02140ce790 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f020400c3c0 tx=0x7f020400c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:34.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:34.309+0000 7f020e000640 1 -- 172.21.15.39:0/4010429561 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0204011790 con 0x7f02140ccc60 2024-09-21T07:54:34.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:34.309+0000 7f020e000640 1 -- 172.21.15.39:0/4010429561 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0204017070 con 0x7f02140ccc60 2024-09-21T07:54:34.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:34.309+0000 7f020e000640 1 -- 172.21.15.39:0/4010429561 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0204011d20 con 0x7f02140ccc60 2024-09-21T07:54:34.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:34.309+0000 7f021c800640 1 -- 172.21.15.39:0/4010429561 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02140b1a80 con 0x7f02140ccc60 2024-09-21T07:54:34.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:34.309+0000 7f021c800640 1 -- 172.21.15.39:0/4010429561 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02140ce260 con 0x7f02140ccc60 2024-09-21T07:54:34.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:34.310+0000 7f021c800640 1 -- 172.21.15.39:0/4010429561 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02140cb8a0 con 0x7f02140ccc60 2024-09-21T07:54:34.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:34.311+0000 7f020e000640 1 -- 172.21.15.39:0/4010429561 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f020401c460 con 0x7f02140ccc60 2024-09-21T07:54:34.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:34.312+0000 7f020e000640 1 --2- 172.21.15.39:0/4010429561 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f01f8078bc0 0x7f01f807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:34.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:34.313+0000 7f020e000640 1 -- 172.21.15.39:0/4010429561 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(257..257 src has 1..257) ==== 7353+0+0 (secure 0 0 0) 0x7f020409af80 con 0x7f02140ccc60 2024-09-21T07:54:34.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:34.313+0000 7f0219e00640 1 --2- 172.21.15.39:0/4010429561 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f01f8078bc0 0x7f01f807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:34.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:34.314+0000 7f0219e00640 1 --2- 172.21.15.39:0/4010429561 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f01f8078bc0 0x7f01f807b0d0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f0208034000 tx=0x7f0208004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:34.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:34.318+0000 7f020e000640 1 -- 172.21.15.39:0/4010429561 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f02040626f0 con 0x7f02140ccc60 2024-09-21T07:54:34.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:34.529+0000 7f021c800640 1 -- 172.21.15.39:0/4010429561 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f02140ad8e0 con 0x7f02140ccc60 2024-09-21T07:54:35.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.246+0000 7f020e000640 1 -- 172.21.15.39:0/4010429561 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v259) ==== 97+0+0 (secure 0 0 0) 0x7f0204067120 con 0x7f02140ccc60 2024-09-21T07:54:35.249 INFO:teuthology.orchestra.run.smithi039.stderr: removed all blocklist entries 2024-09-21T07:54:35.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.249+0000 7f021c800640 1 -- 172.21.15.39:0/4010429561 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f01f8078bc0 msgr2=0x7f01f807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:35.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.249+0000 7f021c800640 1 --2- 172.21.15.39:0/4010429561 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f01f8078bc0 0x7f01f807b0d0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f0208034000 tx=0x7f0208004730 comp rx=0 tx=0).stop 2024-09-21T07:54:35.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.249+0000 7f021c800640 1 -- 172.21.15.39:0/4010429561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02140ccc60 msgr2=0x7f02140ce790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:35.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.249+0000 7f021c800640 1 --2- 172.21.15.39:0/4010429561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02140ccc60 0x7f02140ce790 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f020400c3c0 tx=0x7f020400c880 comp rx=0 tx=0).stop 2024-09-21T07:54:35.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.249+0000 7f021c800640 1 -- 172.21.15.39:0/4010429561 shutdown_connections 2024-09-21T07:54:35.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.249+0000 7f021c800640 1 --2- 172.21.15.39:0/4010429561 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f01f8078bc0 0x7f01f807b0d0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:35.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.249+0000 7f021c800640 1 --2- 172.21.15.39:0/4010429561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f021415d830 0x7f02140cecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:35.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.249+0000 7f021c800640 1 --2- 172.21.15.39:0/4010429561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02140ccc60 0x7f02140ce790 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:35.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.249+0000 7f021c800640 1 --2- 172.21.15.39:0/4010429561 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f02140b30a0 0x7f02141f54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:35.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.249+0000 7f021c800640 1 -- 172.21.15.39:0/4010429561 >> 172.21.15.39:0/4010429561 conn(0x7f02140b2460 msgr2=0x7f0214068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:35.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.250+0000 7f021c800640 1 -- 172.21.15.39:0/4010429561 shutdown_connections 2024-09-21T07:54:35.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.250+0000 7f021c800640 1 -- 172.21.15.39:0/4010429561 wait complete. 2024-09-21T07:54:35.279 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T07:54:35.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.510+0000 7f6215800640 1 -- 172.21.15.39:0/2259230279 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f621015d830 msgr2=0x7f6210156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:35.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.510+0000 7f6215800640 1 --2- 172.21.15.39:0/2259230279 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f621015d830 0x7f6210156bb0 secure :-1 s=READY pgs=2791 cs=0 l=1 rev1=1 crypto rx=0x7f6210071e60 tx=0x7f61f802a4a0 comp rx=0 tx=0).stop 2024-09-21T07:54:35.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.510+0000 7f6215800640 1 -- 172.21.15.39:0/2259230279 shutdown_connections 2024-09-21T07:54:35.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.510+0000 7f6215800640 1 --2- 172.21.15.39:0/2259230279 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f621015d830 0x7f6210156bb0 unknown :-1 s=CLOSED pgs=2791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:35.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.510+0000 7f6215800640 1 --2- 172.21.15.39:0/2259230279 >> v2:172.21.15.148:3300/0 conn(0x7f62100b30a0 0x7f62100cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:35.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.510+0000 7f6215800640 1 -- 172.21.15.39:0/2259230279 >> 172.21.15.39:0/2259230279 conn(0x7f62100b2460 msgr2=0x7f621006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:35.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.510+0000 7f6215800640 1 -- 172.21.15.39:0/2259230279 shutdown_connections 2024-09-21T07:54:35.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.510+0000 7f6215800640 1 -- 172.21.15.39:0/2259230279 wait complete. 2024-09-21T07:54:35.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.512+0000 7f6215800640 1 Processor -- start 2024-09-21T07:54:35.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.512+0000 7f6215800640 1 -- start start 2024-09-21T07:54:35.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.514+0000 7f6215800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62100b30a0 0x7f62101f5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:35.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.514+0000 7f6215800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62100ccc60 0x7f62101f8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:35.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.514+0000 7f6215800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f621015d830 0x7f62101f9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:35.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.514+0000 7f6215800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f62100af410 con 0x7f62100ccc60 2024-09-21T07:54:35.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.514+0000 7f6215800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f62100af290 con 0x7f62100b30a0 2024-09-21T07:54:35.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.514+0000 7f6215800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f62100af590 con 0x7f621015d830 2024-09-21T07:54:35.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.514+0000 7f6207e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62100ccc60 0x7f62101f8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:35.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.514+0000 7f6207e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62100ccc60 0x7f62101f8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38088/0 (socket says 172.21.15.39:38088) 2024-09-21T07:54:35.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.514+0000 7f6207e00640 1 -- 172.21.15.39:0/1261287058 learned_addr learned my addr 172.21.15.39:0/1261287058 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:35.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.514+0000 7f620ea00640 1 --2- 172.21.15.39:0/1261287058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62100b30a0 0x7f62101f5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:35.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.514+0000 7f620f400640 1 --2- 172.21.15.39:0/1261287058 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f621015d830 0x7f62101f9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:35.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.515+0000 7f6207e00640 1 -- 172.21.15.39:0/1261287058 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f621015d830 msgr2=0x7f62101f9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:35.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.515+0000 7f6207e00640 1 --2- 172.21.15.39:0/1261287058 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f621015d830 0x7f62101f9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:35.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.515+0000 7f6207e00640 1 -- 172.21.15.39:0/1261287058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62100b30a0 msgr2=0x7f62101f5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:35.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.515+0000 7f6207e00640 1 --2- 172.21.15.39:0/1261287058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62100b30a0 0x7f62101f5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:35.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.515+0000 7f6207e00640 1 -- 172.21.15.39:0/1261287058 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6210156e70 con 0x7f62100ccc60 2024-09-21T07:54:35.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.515+0000 7f6207e00640 1 --2- 172.21.15.39:0/1261287058 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62100ccc60 0x7f62101f8ed0 secure :-1 s=READY pgs=2792 cs=0 l=1 rev1=1 crypto rx=0x7f61f80098f0 tx=0x7f61f8033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:35.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.515+0000 7f6206a00640 1 -- 172.21.15.39:0/1261287058 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f8005c90 con 0x7f62100ccc60 2024-09-21T07:54:35.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.515+0000 7f6206a00640 1 -- 172.21.15.39:0/1261287058 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61f8031e50 con 0x7f62100ccc60 2024-09-21T07:54:35.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.515+0000 7f6206a00640 1 -- 172.21.15.39:0/1261287058 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f8005260 con 0x7f62100ccc60 2024-09-21T07:54:35.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.515+0000 7f6215800640 1 -- 172.21.15.39:0/1261287058 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6210156bb0 con 0x7f62100ccc60 2024-09-21T07:54:35.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.515+0000 7f6215800640 1 -- 172.21.15.39:0/1261287058 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62100cdf80 con 0x7f62100ccc60 2024-09-21T07:54:35.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.516+0000 7f6215800640 1 -- 172.21.15.39:0/1261287058 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f621006d240 con 0x7f62100ccc60 2024-09-21T07:54:35.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.518+0000 7f6206a00640 1 -- 172.21.15.39:0/1261287058 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f61f8005400 con 0x7f62100ccc60 2024-09-21T07:54:35.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.519+0000 7f6206a00640 1 --2- 172.21.15.39:0/1261287058 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f61ec078b90 0x7f61ec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:35.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.519+0000 7f6206a00640 1 -- 172.21.15.39:0/1261287058 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(261..261 src has 1..261) ==== 6192+0+0 (secure 0 0 0) 0x7f61f80bbff0 con 0x7f62100ccc60 2024-09-21T07:54:35.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.519+0000 7f620ea00640 1 --2- 172.21.15.39:0/1261287058 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f61ec078b90 0x7f61ec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:35.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.520+0000 7f620ea00640 1 --2- 172.21.15.39:0/1261287058 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f61ec078b90 0x7f61ec07b0a0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f6200009b30 tx=0x7f6200005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:35.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.523+0000 7f6206a00640 1 -- 172.21.15.39:0/1261287058 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f61f803f030 con 0x7f62100ccc60 2024-09-21T07:54:35.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.789+0000 7f6215800640 1 -- 172.21.15.39:0/1261287058 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f62100afa40 con 0x7f62100ccc60 2024-09-21T07:54:35.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.790+0000 7f6206a00640 1 -- 172.21.15.39:0/1261287058 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v25) ==== 80+0+5304 (secure 0 0 0) 0x7f61f8083c30 con 0x7f62100ccc60 2024-09-21T07:54:35.792 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:35.792 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:54:35.792 INFO:teuthology.orchestra.run.smithi039.stdout: "auth_dump": [ 2024-09-21T07:54:35.792 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:54:35.792 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.a", 2024-09-21T07:54:35.792 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5m+P6EERAABQqmlkrocSi1+hwc73Adrg==", 2024-09-21T07:54:35.792 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:54:35.792 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:54:35.792 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:54:35.793 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:54:35.793 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:54:35.793 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:54:35.793 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:54:35.793 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:54:35.793 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.b", 2024-09-21T07:54:35.793 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5m5/GeLRAAfE8fr5dbitQa89rNK9pZ2Q==", 2024-09-21T07:54:35.793 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:54:35.793 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:54:35.793 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:54:35.793 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:54:35.793 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:54:35.793 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:54:35.793 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:54:35.793 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:54:35.794 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.c", 2024-09-21T07:54:35.794 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5msbx6HxAALsF8ORvIoJpIAFZ5uxmOEw==", 2024-09-21T07:54:35.794 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:54:35.794 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:54:35.794 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:54:35.794 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:54:35.794 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:54:35.794 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:54:35.794 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:54:35.794 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:54:35.794 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mds.d", 2024-09-21T07:54:35.794 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mIrcBABAApKAB/7FkZ5glNTxlbkIsBw==", 2024-09-21T07:54:35.794 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:54:35.795 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:54:35.795 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:54:35.795 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:54:35.795 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:54:35.795 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:54:35.795 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:54:35.795 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:54:35.795 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.0", 2024-09-21T07:54:35.795 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB7du5msw81DhAAmVimqN4x9K3YpT0nbovppQ==", 2024-09-21T07:54:35.795 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:54:35.795 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:54:35.795 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:54:35.795 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:54:35.795 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:54:35.796 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:54:35.796 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:54:35.796 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.1", 2024-09-21T07:54:35.796 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB+du5mdnCIOhAAtVB4ThehQwb0zRriyKcaxA==", 2024-09-21T07:54:35.796 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:54:35.796 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:54:35.796 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:54:35.796 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:54:35.796 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:54:35.796 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:54:35.796 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:54:35.796 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.2", 2024-09-21T07:54:35.796 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCCdu5mTfWxDhAAadJCbPN5xBx/f9L0v2yTdg==", 2024-09-21T07:54:35.796 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:54:35.796 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:54:35.796 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:54:35.797 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:54:35.797 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:54:35.797 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:54:35.797 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:54:35.797 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.3", 2024-09-21T07:54:35.797 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCFdu5mL1E2HxAAyOhY97uUXEw5L1CcDeONVA==", 2024-09-21T07:54:35.797 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:54:35.797 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:54:35.797 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:54:35.797 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:54:35.797 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:54:35.797 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:54:35.797 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:54:35.797 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.4", 2024-09-21T07:54:35.797 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCLdu5m6PY/IhAAgvcTgm+6uWA7lIG4tcTIEg==", 2024-09-21T07:54:35.797 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:54:35.798 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:54:35.798 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:54:35.798 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:54:35.798 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:54:35.798 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:54:35.798 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:54:35.798 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.5", 2024-09-21T07:54:35.798 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCOdu5mE/wlMxAA3JBg4xH+N/PPC1Uv2ZVrYQ==", 2024-09-21T07:54:35.798 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:54:35.798 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:54:35.798 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:54:35.798 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:54:35.798 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:54:35.799 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:54:35.799 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:54:35.799 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.6", 2024-09-21T07:54:35.799 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCSdu5mC3TgBxAAwsArjeIdFcApY3/1OJffEg==", 2024-09-21T07:54:35.799 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:54:35.799 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:54:35.799 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:54:35.799 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:54:35.799 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:54:35.799 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:54:35.799 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:54:35.799 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "osd.7", 2024-09-21T07:54:35.799 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQCVdu5mSWHWFBAA9qPHWRdsvrB74inrp1lj4Q==", 2024-09-21T07:54:35.799 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:54:35.800 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow profile osd", 2024-09-21T07:54:35.800 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile osd", 2024-09-21T07:54:35.800 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:54:35.800 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:54:35.800 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:54:35.800 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:54:35.800 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.0", 2024-09-21T07:54:35.800 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mGfb1DBAA/aSMKR5or3xy53rB/iUjpA==", 2024-09-21T07:54:35.800 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:54:35.800 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:54:35.800 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow r", 2024-09-21T07:54:35.800 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T07:54:35.800 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:54:35.800 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:54:35.800 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:54:35.800 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.1", 2024-09-21T07:54:35.801 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mEkciIhAAid+bxJEr4P7vMnZ42PAy5w==", 2024-09-21T07:54:35.801 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:54:35.801 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:54:35.801 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow r", 2024-09-21T07:54:35.801 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow rw", 2024-09-21T07:54:35.801 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rwx" 2024-09-21T07:54:35.801 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:54:35.801 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:54:35.801 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:54:35.801 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.2", 2024-09-21T07:54:35.801 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mSu2oFxAAgjy2m/wWKh6xaoh+YDoU1Q==", 2024-09-21T07:54:35.801 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:54:35.801 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:54:35.801 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow r", 2024-09-21T07:54:35.802 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow rw", 2024-09-21T07:54:35.802 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rwx" 2024-09-21T07:54:35.802 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:54:35.802 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:54:35.802 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:54:35.802 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.3", 2024-09-21T07:54:35.802 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQB0du5mCHy0LBAAZneF2jUWgfuyGehZFbOxDg==", 2024-09-21T07:54:35.802 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:54:35.802 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow", 2024-09-21T07:54:35.802 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow r", 2024-09-21T07:54:35.802 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow rw", 2024-09-21T07:54:35.802 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow rwx" 2024-09-21T07:54:35.802 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:54:35.802 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:54:35.802 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:54:35.803 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "client.admin", 2024-09-21T07:54:35.803 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBydu5m9LInERAAAlDpqb94f2Qb9VYJJwTBcA==", 2024-09-21T07:54:35.803 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:54:35.803 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow *", 2024-09-21T07:54:35.803 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr": "allow *", 2024-09-21T07:54:35.803 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow *", 2024-09-21T07:54:35.803 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:54:35.803 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:54:35.803 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:54:35.803 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:54:35.803 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mgr.x", 2024-09-21T07:54:35.803 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBzdu5mA2GRBhAAkyCKdgH6laHy8xNt080dXg==", 2024-09-21T07:54:35.803 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:54:35.803 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow *", 2024-09-21T07:54:35.804 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile mgr", 2024-09-21T07:54:35.804 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:54:35.804 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:54:35.804 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:54:35.804 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:54:35.804 INFO:teuthology.orchestra.run.smithi039.stdout: "entity": "mgr.y", 2024-09-21T07:54:35.804 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "AQBydu5mr+8aNxAA7TFYNms2WsXUfa4ITTJOHw==", 2024-09-21T07:54:35.804 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": { 2024-09-21T07:54:35.804 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": "allow *", 2024-09-21T07:54:35.804 INFO:teuthology.orchestra.run.smithi039.stdout: "mon": "allow profile mgr", 2024-09-21T07:54:35.804 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "allow *" 2024-09-21T07:54:35.804 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:54:35.805 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:54:35.805 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-21T07:54:35.805 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:54:35.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.792+0000 7f6215800640 1 -- 172.21.15.39:0/1261287058 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f61ec078b90 msgr2=0x7f61ec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:35.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.792+0000 7f6215800640 1 --2- 172.21.15.39:0/1261287058 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f61ec078b90 0x7f61ec07b0a0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f6200009b30 tx=0x7f6200005f90 comp rx=0 tx=0).stop 2024-09-21T07:54:35.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.792+0000 7f6215800640 1 -- 172.21.15.39:0/1261287058 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62100ccc60 msgr2=0x7f62101f8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:35.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.792+0000 7f6215800640 1 --2- 172.21.15.39:0/1261287058 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62100ccc60 0x7f62101f8ed0 secure :-1 s=READY pgs=2792 cs=0 l=1 rev1=1 crypto rx=0x7f61f80098f0 tx=0x7f61f8033c20 comp rx=0 tx=0).stop 2024-09-21T07:54:35.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.792+0000 7f6215800640 1 -- 172.21.15.39:0/1261287058 shutdown_connections 2024-09-21T07:54:35.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.792+0000 7f6215800640 1 --2- 172.21.15.39:0/1261287058 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f61ec078b90 0x7f61ec07b0a0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:35.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.792+0000 7f6215800640 1 --2- 172.21.15.39:0/1261287058 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f621015d830 0x7f62101f9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:35.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.792+0000 7f6215800640 1 --2- 172.21.15.39:0/1261287058 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62100ccc60 0x7f62101f8ed0 unknown :-1 s=CLOSED pgs=2792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:35.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.792+0000 7f6215800640 1 --2- 172.21.15.39:0/1261287058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f62100b30a0 0x7f62101f5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:35.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.792+0000 7f6215800640 1 -- 172.21.15.39:0/1261287058 >> 172.21.15.39:0/1261287058 conn(0x7f62100b2460 msgr2=0x7f621006f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:35.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.792+0000 7f6215800640 1 -- 172.21.15.39:0/1261287058 shutdown_connections 2024-09-21T07:54:35.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:35.793+0000 7f6215800640 1 -- 172.21.15.39:0/1261287058 wait complete. 2024-09-21T07:54:35.819 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T07:54:36.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.043+0000 7f7ba1600640 1 -- 172.21.15.39:0/670495563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b9c0d06d0 msgr2=0x7f7b9c0d0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:36.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.043+0000 7f7ba1600640 1 --2- 172.21.15.39:0/670495563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b9c0d06d0 0x7f7b9c0d0ae0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f7b94005fd0 tx=0x7f7b9402a8b0 comp rx=0 tx=0).stop 2024-09-21T07:54:36.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.043+0000 7f7ba1600640 1 -- 172.21.15.39:0/670495563 shutdown_connections 2024-09-21T07:54:36.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.043+0000 7f7ba1600640 1 --2- 172.21.15.39:0/670495563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b9c0d06d0 0x7f7b9c0d0ae0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:36.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.043+0000 7f7ba1600640 1 --2- 172.21.15.39:0/670495563 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b9c15d3a0 0x7f7b9c15d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:36.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.043+0000 7f7ba1600640 1 -- 172.21.15.39:0/670495563 >> 172.21.15.39:0/670495563 conn(0x7f7b9c06cfc0 msgr2=0x7f7b9c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:36.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.043+0000 7f7ba1600640 1 -- 172.21.15.39:0/670495563 shutdown_connections 2024-09-21T07:54:36.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.043+0000 7f7ba1600640 1 -- 172.21.15.39:0/670495563 wait complete. 2024-09-21T07:54:36.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.046+0000 7f7ba1600640 1 Processor -- start 2024-09-21T07:54:36.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.046+0000 7f7ba1600640 1 -- start start 2024-09-21T07:54:36.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.047+0000 7f7ba1600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b9c0cf450 0x7f7b9c1431d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:36.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.047+0000 7f7ba1600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b9c0d06d0 0x7f7b9c1564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:36.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.047+0000 7f7ba1600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b9c15d3a0 0x7f7b9c1439b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:36.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.047+0000 7f7ba1600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c0cd700 con 0x7f7b9c0cf450 2024-09-21T07:54:36.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.047+0000 7f7ba1600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c0cd580 con 0x7f7b9c15d3a0 2024-09-21T07:54:36.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.047+0000 7f7ba1600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f7b9c0cd880 con 0x7f7b9c0d06d0 2024-09-21T07:54:36.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.047+0000 7f7b9aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b9c0cf450 0x7f7b9c1431d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:36.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.048+0000 7f7b9aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b9c0cf450 0x7f7b9c1431d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38092/0 (socket says 172.21.15.39:38092) 2024-09-21T07:54:36.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.048+0000 7f7b9aa00640 1 -- 172.21.15.39:0/2909629850 learned_addr learned my addr 172.21.15.39:0/2909629850 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:36.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.048+0000 7f7b9b400640 1 --2- 172.21.15.39:0/2909629850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b9c15d3a0 0x7f7b9c1439b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:36.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.048+0000 7f7b93e00640 1 --2- 172.21.15.39:0/2909629850 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b9c0d06d0 0x7f7b9c1564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:36.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.048+0000 7f7b9aa00640 1 -- 172.21.15.39:0/2909629850 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b9c0d06d0 msgr2=0x7f7b9c1564d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:36.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.048+0000 7f7b9aa00640 1 --2- 172.21.15.39:0/2909629850 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b9c0d06d0 0x7f7b9c1564d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:36.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.048+0000 7f7b9aa00640 1 -- 172.21.15.39:0/2909629850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b9c15d3a0 msgr2=0x7f7b9c1439b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:36.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.048+0000 7f7b9aa00640 1 --2- 172.21.15.39:0/2909629850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b9c15d3a0 0x7f7b9c1439b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:36.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.048+0000 7f7b9aa00640 1 -- 172.21.15.39:0/2909629850 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b9c154fc0 con 0x7f7b9c0cf450 2024-09-21T07:54:36.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.048+0000 7f7b9b400640 1 --2- 172.21.15.39:0/2909629850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b9c15d3a0 0x7f7b9c1439b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:36.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.048+0000 7f7b9aa00640 1 --2- 172.21.15.39:0/2909629850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b9c0cf450 0x7f7b9c1431d0 secure :-1 s=READY pgs=2793 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c00c760 tx=0x7f7b8c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:36.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.049+0000 7f7b92a00640 1 -- 172.21.15.39:0/2909629850 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c016020 con 0x7f7b9c0cf450 2024-09-21T07:54:36.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.049+0000 7f7b92a00640 1 -- 172.21.15.39:0/2909629850 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b8c00a950 con 0x7f7b9c0cf450 2024-09-21T07:54:36.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.049+0000 7f7ba1600640 1 -- 172.21.15.39:0/2909629850 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b9c155430 con 0x7f7b9c0cf450 2024-09-21T07:54:36.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.049+0000 7f7b92a00640 1 -- 172.21.15.39:0/2909629850 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c00ada0 con 0x7f7b9c0cf450 2024-09-21T07:54:36.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.049+0000 7f7ba1600640 1 -- 172.21.15.39:0/2909629850 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b9c1559f0 con 0x7f7b9c0cf450 2024-09-21T07:54:36.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.050+0000 7f7ba1600640 1 -- 172.21.15.39:0/2909629850 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b9c0cf030 con 0x7f7b9c0cf450 2024-09-21T07:54:36.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.051+0000 7f7b92a00640 1 -- 172.21.15.39:0/2909629850 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7b8c019020 con 0x7f7b9c0cf450 2024-09-21T07:54:36.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.052+0000 7f7b92a00640 1 --2- 172.21.15.39:0/2909629850 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7b78078bc0 0x7f7b7807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:36.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.052+0000 7f7b92a00640 1 -- 172.21.15.39:0/2909629850 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(261..261 src has 1..261) ==== 6192+0+0 (secure 0 0 0) 0x7f7b8c09adc0 con 0x7f7b9c0cf450 2024-09-21T07:54:36.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.053+0000 7f7b93e00640 1 --2- 172.21.15.39:0/2909629850 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7b78078bc0 0x7f7b7807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:36.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.053+0000 7f7b93e00640 1 --2- 172.21.15.39:0/2909629850 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7b78078bc0 0x7f7b7807b0d0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c0b0d10 tx=0x7f7b84005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:36.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.057+0000 7f7b92a00640 1 -- 172.21.15.39:0/2909629850 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7b9c0cf030 con 0x7f7b9c0cf450 2024-09-21T07:54:36.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.258+0000 7f7ba1600640 1 -- 172.21.15.39:0/2909629850 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f7b9c0cdda0 con 0x7f7b9c0cf450 2024-09-21T07:54:36.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.258+0000 7f7b92a00640 1 -- 172.21.15.39:0/2909629850 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v261) ==== 84+0+287 (secure 0 0 0) 0x7f7b8c020020 con 0x7f7b9c0cf450 2024-09-21T07:54:36.260 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:36.260 INFO:teuthology.orchestra.run.smithi039.stdout:[ 2024-09-21T07:54:36.260 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:54:36.261 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 1, 2024-09-21T07:54:36.261 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": ".mgr" 2024-09-21T07:54:36.261 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:54:36.261 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:54:36.261 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 6, 2024-09-21T07:54:36.261 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": "snap-test-pool" 2024-09-21T07:54:36.261 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:54:36.261 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:54:36.261 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 11, 2024-09-21T07:54:36.261 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": "test_data_pool" 2024-09-21T07:54:36.261 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-21T07:54:36.261 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-21T07:54:36.261 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 12, 2024-09-21T07:54:36.261 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": "test_metadata_pool" 2024-09-21T07:54:36.262 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-21T07:54:36.262 INFO:teuthology.orchestra.run.smithi039.stdout:] 2024-09-21T07:54:36.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.261+0000 7f7ba1600640 1 -- 172.21.15.39:0/2909629850 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7b78078bc0 msgr2=0x7f7b7807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:36.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.261+0000 7f7ba1600640 1 --2- 172.21.15.39:0/2909629850 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7b78078bc0 0x7f7b7807b0d0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c0b0d10 tx=0x7f7b84005f90 comp rx=0 tx=0).stop 2024-09-21T07:54:36.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.261+0000 7f7ba1600640 1 -- 172.21.15.39:0/2909629850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b9c0cf450 msgr2=0x7f7b9c1431d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:36.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.261+0000 7f7ba1600640 1 --2- 172.21.15.39:0/2909629850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b9c0cf450 0x7f7b9c1431d0 secure :-1 s=READY pgs=2793 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c00c760 tx=0x7f7b8c00cc20 comp rx=0 tx=0).stop 2024-09-21T07:54:36.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.261+0000 7f7ba1600640 1 -- 172.21.15.39:0/2909629850 shutdown_connections 2024-09-21T07:54:36.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.261+0000 7f7ba1600640 1 --2- 172.21.15.39:0/2909629850 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7b78078bc0 0x7f7b7807b0d0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:36.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.261+0000 7f7ba1600640 1 --2- 172.21.15.39:0/2909629850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7b9c15d3a0 0x7f7b9c1439b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:36.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.261+0000 7f7ba1600640 1 --2- 172.21.15.39:0/2909629850 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7b9c0d06d0 0x7f7b9c1564d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:36.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.261+0000 7f7ba1600640 1 --2- 172.21.15.39:0/2909629850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b9c0cf450 0x7f7b9c1431d0 unknown :-1 s=CLOSED pgs=2793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:36.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.261+0000 7f7ba1600640 1 -- 172.21.15.39:0/2909629850 >> 172.21.15.39:0/2909629850 conn(0x7f7b9c06cfc0 msgr2=0x7f7b9c073130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:36.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.261+0000 7f7ba1600640 1 -- 172.21.15.39:0/2909629850 shutdown_connections 2024-09-21T07:54:36.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.261+0000 7f7ba1600640 1 -- 172.21.15.39:0/2909629850 wait complete. 2024-09-21T07:54:36.287 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T07:54:36.288 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T07:54:36.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.518+0000 7fe329a00640 1 -- 172.21.15.39:0/3223475612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3240b5cc0 msgr2=0x7fe3240cf060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:36.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.518+0000 7fe329a00640 1 --2- 172.21.15.39:0/3223475612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3240b5cc0 0x7fe3240cf060 secure :-1 s=READY pgs=2795 cs=0 l=1 rev1=1 crypto rx=0x7fe32408d950 tx=0x7fe31002a530 comp rx=0 tx=0).stop 2024-09-21T07:54:36.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.518+0000 7fe329a00640 1 -- 172.21.15.39:0/3223475612 shutdown_connections 2024-09-21T07:54:36.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.518+0000 7fe329a00640 1 --2- 172.21.15.39:0/3223475612 >> v2:172.21.15.148:3300/0 conn(0x7fe3240cf5a0 0x7fe3240cf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:36.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.518+0000 7fe329a00640 1 --2- 172.21.15.39:0/3223475612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3240b5cc0 0x7fe3240cf060 unknown :-1 s=CLOSED pgs=2795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:36.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.518+0000 7fe329a00640 1 -- 172.21.15.39:0/3223475612 >> 172.21.15.39:0/3223475612 conn(0x7fe32408c000 msgr2=0x7fe3240d46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:36.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.518+0000 7fe329a00640 1 -- 172.21.15.39:0/3223475612 shutdown_connections 2024-09-21T07:54:36.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.518+0000 7fe329a00640 1 -- 172.21.15.39:0/3223475612 wait complete. 2024-09-21T07:54:36.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.520+0000 7fe329a00640 1 Processor -- start 2024-09-21T07:54:36.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.520+0000 7fe329a00640 1 -- start start 2024-09-21T07:54:36.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.522+0000 7fe329a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3240b5cc0 0x7fe3241e2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:36.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.522+0000 7fe329a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3240cf5a0 0x7fe3240ce190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:36.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.522+0000 7fe329a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3240d4e00 0x7fe3240b96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:36.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.522+0000 7fe329a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe3240b5170 con 0x7fe3240d4e00 2024-09-21T07:54:36.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.522+0000 7fe329a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe3240b7110 con 0x7fe3240cf5a0 2024-09-21T07:54:36.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.522+0000 7fe329a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe3240cfef0 con 0x7fe3240b5cc0 2024-09-21T07:54:36.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.522+0000 7fe329000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3240d4e00 0x7fe3240b96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:36.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.522+0000 7fe329000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3240d4e00 0x7fe3240b96f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38106/0 (socket says 172.21.15.39:38106) 2024-09-21T07:54:36.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.522+0000 7fe329000640 1 -- 172.21.15.39:0/2303046855 learned_addr learned my addr 172.21.15.39:0/2303046855 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:36.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.522+0000 7fe323e00640 1 --2- 172.21.15.39:0/2303046855 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3240b5cc0 0x7fe3241e2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:36.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.522+0000 7fe323400640 1 --2- 172.21.15.39:0/2303046855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3240cf5a0 0x7fe3240ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:36.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.523+0000 7fe329000640 1 -- 172.21.15.39:0/2303046855 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3240b5cc0 msgr2=0x7fe3241e2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:36.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.523+0000 7fe329000640 1 --2- 172.21.15.39:0/2303046855 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3240b5cc0 0x7fe3241e2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:36.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.523+0000 7fe329000640 1 -- 172.21.15.39:0/2303046855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3240cf5a0 msgr2=0x7fe3240ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:36.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.523+0000 7fe329000640 1 --2- 172.21.15.39:0/2303046855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3240cf5a0 0x7fe3240ce190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:36.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.523+0000 7fe329000640 1 -- 172.21.15.39:0/2303046855 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3240c92f0 con 0x7fe3240d4e00 2024-09-21T07:54:36.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.523+0000 7fe323400640 1 --2- 172.21.15.39:0/2303046855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3240cf5a0 0x7fe3240ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:36.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.523+0000 7fe329000640 1 --2- 172.21.15.39:0/2303046855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3240d4e00 0x7fe3240b96f0 secure :-1 s=READY pgs=2796 cs=0 l=1 rev1=1 crypto rx=0x7fe31c00d9a0 tx=0x7fe31c00de60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:36.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.523+0000 7fe31be00640 1 -- 172.21.15.39:0/2303046855 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c017070 con 0x7fe3240d4e00 2024-09-21T07:54:36.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.523+0000 7fe31be00640 1 -- 172.21.15.39:0/2303046855 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe31c00bda0 con 0x7fe3240d4e00 2024-09-21T07:54:36.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.523+0000 7fe329a00640 1 -- 172.21.15.39:0/2303046855 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3241f5a40 con 0x7fe3240d4e00 2024-09-21T07:54:36.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.523+0000 7fe31be00640 1 -- 172.21.15.39:0/2303046855 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c003bd0 con 0x7fe3240d4e00 2024-09-21T07:54:36.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.524+0000 7fe329a00640 1 -- 172.21.15.39:0/2303046855 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3241f6000 con 0x7fe3240d4e00 2024-09-21T07:54:36.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.524+0000 7fe329a00640 1 -- 172.21.15.39:0/2303046855 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3240cf060 con 0x7fe3240d4e00 2024-09-21T07:54:36.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.526+0000 7fe31be00640 1 -- 172.21.15.39:0/2303046855 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe31c010460 con 0x7fe3240d4e00 2024-09-21T07:54:36.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.527+0000 7fe31be00640 1 --2- 172.21.15.39:0/2303046855 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe304078b90 0x7fe30407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:36.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.527+0000 7fe31be00640 1 -- 172.21.15.39:0/2303046855 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(261..261 src has 1..261) ==== 6192+0+0 (secure 0 0 0) 0x7fe31c09b620 con 0x7fe3240d4e00 2024-09-21T07:54:36.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.527+0000 7fe323e00640 1 --2- 172.21.15.39:0/2303046855 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe304078b90 0x7fe30407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:36.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.528+0000 7fe323e00640 1 --2- 172.21.15.39:0/2303046855 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe304078b90 0x7fe30407b0a0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fe314001760 tx=0x7fe314009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:36.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.530+0000 7fe31be00640 1 -- 172.21.15.39:0/2303046855 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe31c0632c0 con 0x7fe3240d4e00 2024-09-21T07:54:36.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:36.726+0000 7fe329a00640 1 -- 172.21.15.39:0/2303046855 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe32406f780 con 0x7fe3240d4e00 2024-09-21T07:54:37.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.259+0000 7fe31be00640 1 -- 172.21.15.39:0/2303046855 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v262) ==== 138+0+0 (secure 0 0 0) 0x7fe31c067cf0 con 0x7fe3240d4e00 2024-09-21T07:54:37.261 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' created 2024-09-21T07:54:37.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.262+0000 7fe329a00640 1 -- 172.21.15.39:0/2303046855 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe304078b90 msgr2=0x7fe30407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:37.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.262+0000 7fe329a00640 1 --2- 172.21.15.39:0/2303046855 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe304078b90 0x7fe30407b0a0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fe314001760 tx=0x7fe314009040 comp rx=0 tx=0).stop 2024-09-21T07:54:37.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.262+0000 7fe329a00640 1 -- 172.21.15.39:0/2303046855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3240d4e00 msgr2=0x7fe3240b96f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:37.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.262+0000 7fe329a00640 1 --2- 172.21.15.39:0/2303046855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3240d4e00 0x7fe3240b96f0 secure :-1 s=READY pgs=2796 cs=0 l=1 rev1=1 crypto rx=0x7fe31c00d9a0 tx=0x7fe31c00de60 comp rx=0 tx=0).stop 2024-09-21T07:54:37.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.262+0000 7fe329a00640 1 -- 172.21.15.39:0/2303046855 shutdown_connections 2024-09-21T07:54:37.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.262+0000 7fe329a00640 1 --2- 172.21.15.39:0/2303046855 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe304078b90 0x7fe30407b0a0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:37.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.262+0000 7fe329a00640 1 --2- 172.21.15.39:0/2303046855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3240d4e00 0x7fe3240b96f0 unknown :-1 s=CLOSED pgs=2796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:37.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.262+0000 7fe329a00640 1 --2- 172.21.15.39:0/2303046855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe3240cf5a0 0x7fe3240ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:37.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.262+0000 7fe329a00640 1 --2- 172.21.15.39:0/2303046855 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe3240b5cc0 0x7fe3241e2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:37.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.262+0000 7fe329a00640 1 -- 172.21.15.39:0/2303046855 >> 172.21.15.39:0/2303046855 conn(0x7fe32408c000 msgr2=0x7fe3240b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:37.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.262+0000 7fe329a00640 1 -- 172.21.15.39:0/2303046855 shutdown_connections 2024-09-21T07:54:37.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.262+0000 7fe329a00640 1 -- 172.21.15.39:0/2303046855 wait complete. 2024-09-21T07:54:37.292 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T07:54:37.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.527+0000 7f12aca00640 1 -- 172.21.15.39:0/1847705376 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f128c004800 msgr2=0x7f128c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:37.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.527+0000 7f12aca00640 1 --2- 172.21.15.39:0/1847705376 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f128c004800 0x7f128c004c10 secure :-1 s=READY pgs=2798 cs=0 l=1 rev1=1 crypto rx=0x7f1294009cd0 tx=0x7f128c005830 comp rx=0 tx=0).stop 2024-09-21T07:54:37.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.527+0000 7f12a6800640 1 -- 172.21.15.39:0/1847705376 reap_dead start 2024-09-21T07:54:37.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.527+0000 7f12aca00640 1 -- 172.21.15.39:0/1847705376 shutdown_connections 2024-09-21T07:54:37.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.527+0000 7f12aca00640 1 -- 172.21.15.39:0/1847705376 >> 172.21.15.39:0/1847705376 conn(0x7f12a80b2460 msgr2=0x7f12a806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:37.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.527+0000 7f12aca00640 1 -- 172.21.15.39:0/1847705376 shutdown_connections 2024-09-21T07:54:37.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.527+0000 7f12aca00640 1 -- 172.21.15.39:0/1847705376 wait complete. 2024-09-21T07:54:37.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.529+0000 7f12aca00640 1 Processor -- start 2024-09-21T07:54:37.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.529+0000 7f12aca00640 1 -- start start 2024-09-21T07:54:37.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.531+0000 7f12aca00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f128c004800 0x7f12a81f4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:37.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.531+0000 7f12aca00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12a80c49b0 0x7f12a80cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:37.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.531+0000 7f12aca00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f12a80cf6e0 0x7f12a815b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:37.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.531+0000 7f12aca00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f12a80cbc30 con 0x7f12a80c49b0 2024-09-21T07:54:37.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.531+0000 7f12aca00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f12a80cbab0 con 0x7f128c004800 2024-09-21T07:54:37.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.531+0000 7f12aca00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f12a80af080 con 0x7f12a80cf6e0 2024-09-21T07:54:37.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.531+0000 7f12a5400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12a80c49b0 0x7f12a80cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:37.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.531+0000 7f12a5400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12a80c49b0 0x7f12a80cf1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38124/0 (socket says 172.21.15.39:38124) 2024-09-21T07:54:37.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.531+0000 7f12a5400640 1 -- 172.21.15.39:0/1331214772 learned_addr learned my addr 172.21.15.39:0/1331214772 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:37.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.531+0000 7f12a5e00640 1 --2- 172.21.15.39:0/1331214772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f128c004800 0x7f12a81f4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:37.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.531+0000 7f12a6800640 1 --2- 172.21.15.39:0/1331214772 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f12a80cf6e0 0x7f12a815b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:37.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.532+0000 7f12a5400640 1 -- 172.21.15.39:0/1331214772 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f12a80cf6e0 msgr2=0x7f12a815b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:37.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.532+0000 7f12a5400640 1 --2- 172.21.15.39:0/1331214772 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f12a80cf6e0 0x7f12a815b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:37.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.532+0000 7f12a5400640 1 -- 172.21.15.39:0/1331214772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f128c004800 msgr2=0x7f12a81f4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:37.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.532+0000 7f12a5400640 1 --2- 172.21.15.39:0/1331214772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f128c004800 0x7f12a81f4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:37.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.532+0000 7f12a5400640 1 -- 172.21.15.39:0/1331214772 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12a80cfed0 con 0x7f12a80c49b0 2024-09-21T07:54:37.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.532+0000 7f12a5400640 1 --2- 172.21.15.39:0/1331214772 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12a80c49b0 0x7f12a80cf1a0 secure :-1 s=READY pgs=2799 cs=0 l=1 rev1=1 crypto rx=0x7f1294009c20 tx=0x7f12940038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:37.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.532+0000 7f129e000640 1 -- 172.21.15.39:0/1331214772 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1294038040 con 0x7f12a80c49b0 2024-09-21T07:54:37.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.532+0000 7f129e000640 1 -- 172.21.15.39:0/1331214772 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1294041070 con 0x7f12a80c49b0 2024-09-21T07:54:37.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.532+0000 7f129e000640 1 -- 172.21.15.39:0/1331214772 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1294003c90 con 0x7f12a80c49b0 2024-09-21T07:54:37.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.532+0000 7f12aca00640 1 -- 172.21.15.39:0/1331214772 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12a80c3bf0 con 0x7f12a80c49b0 2024-09-21T07:54:37.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.533+0000 7f12aca00640 1 -- 172.21.15.39:0/1331214772 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12a80c41b0 con 0x7f12a80c49b0 2024-09-21T07:54:37.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.534+0000 7f12aca00640 1 -- 172.21.15.39:0/1331214772 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12a8156bb0 con 0x7f12a80c49b0 2024-09-21T07:54:37.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.535+0000 7f129e000640 1 -- 172.21.15.39:0/1331214772 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1294046460 con 0x7f12a80c49b0 2024-09-21T07:54:37.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.536+0000 7f129e000640 1 --2- 172.21.15.39:0/1331214772 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1288078bc0 0x7f128807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:37.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.537+0000 7f129e000640 1 -- 172.21.15.39:0/1331214772 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(262..262 src has 1..262) ==== 6562+0+0 (secure 0 0 0) 0x7f12940bbed0 con 0x7f12a80c49b0 2024-09-21T07:54:37.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.537+0000 7f12a5e00640 1 --2- 172.21.15.39:0/1331214772 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1288078bc0 0x7f128807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:37.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.538+0000 7f12a5e00640 1 --2- 172.21.15.39:0/1331214772 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1288078bc0 0x7f128807b0d0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f1298005eb0 tx=0x7f1298005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:37.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.542+0000 7f129e000640 1 -- 172.21.15.39:0/1331214772 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f12940838c0 con 0x7f12a80c49b0 2024-09-21T07:54:37.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:37.740+0000 7f12aca00640 1 -- 172.21.15.39:0/1331214772 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f12a80cfc60 con 0x7f12a80c49b0 2024-09-21T07:54:38.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.267+0000 7f129e000640 1 -- 172.21.15.39:0/1331214772 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v263) ==== 170+0+0 (secure 0 0 0) 0x7f12940882f0 con 0x7f12a80c49b0 2024-09-21T07:54:38.270 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' created 2024-09-21T07:54:38.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.275+0000 7f12aca00640 1 -- 172.21.15.39:0/1331214772 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1288078bc0 msgr2=0x7f128807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:38.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.275+0000 7f12aca00640 1 --2- 172.21.15.39:0/1331214772 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1288078bc0 0x7f128807b0d0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f1298005eb0 tx=0x7f1298005f90 comp rx=0 tx=0).stop 2024-09-21T07:54:38.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.275+0000 7f12aca00640 1 -- 172.21.15.39:0/1331214772 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12a80c49b0 msgr2=0x7f12a80cf1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:38.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.275+0000 7f12aca00640 1 --2- 172.21.15.39:0/1331214772 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12a80c49b0 0x7f12a80cf1a0 secure :-1 s=READY pgs=2799 cs=0 l=1 rev1=1 crypto rx=0x7f1294009c20 tx=0x7f12940038a0 comp rx=0 tx=0).stop 2024-09-21T07:54:38.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.275+0000 7f12aca00640 1 -- 172.21.15.39:0/1331214772 shutdown_connections 2024-09-21T07:54:38.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.275+0000 7f12aca00640 1 --2- 172.21.15.39:0/1331214772 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f1288078bc0 0x7f128807b0d0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:38.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.275+0000 7f12aca00640 1 --2- 172.21.15.39:0/1331214772 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f12a80cf6e0 0x7f12a815b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:38.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.275+0000 7f12aca00640 1 --2- 172.21.15.39:0/1331214772 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12a80c49b0 0x7f12a80cf1a0 unknown :-1 s=CLOSED pgs=2799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:38.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.275+0000 7f12aca00640 1 --2- 172.21.15.39:0/1331214772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f128c004800 0x7f12a81f4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:38.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.275+0000 7f12aca00640 1 -- 172.21.15.39:0/1331214772 >> 172.21.15.39:0/1331214772 conn(0x7f12a80b2460 msgr2=0x7f12a80b15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:38.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.276+0000 7f12aca00640 1 -- 172.21.15.39:0/1331214772 shutdown_connections 2024-09-21T07:54:38.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.276+0000 7f12aca00640 1 -- 172.21.15.39:0/1331214772 wait complete. 2024-09-21T07:54:38.317 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T07:54:38.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.548+0000 7f905e600640 1 -- 172.21.15.39:0/758445061 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f903c004d40 msgr2=0x7f903c005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:38.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.548+0000 7f905e600640 1 --2- 172.21.15.39:0/758445061 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f903c004d40 0x7f903c005150 secure :-1 s=READY pgs=2801 cs=0 l=1 rev1=1 crypto rx=0x7f9050009ed0 tx=0x7f903c005830 comp rx=0 tx=0).stop 2024-09-21T07:54:38.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.548+0000 7f9057e00640 1 -- 172.21.15.39:0/758445061 reap_dead start 2024-09-21T07:54:38.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.548+0000 7f905e600640 1 -- 172.21.15.39:0/758445061 shutdown_connections 2024-09-21T07:54:38.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.548+0000 7f905e600640 1 -- 172.21.15.39:0/758445061 >> 172.21.15.39:0/758445061 conn(0x7f905806cfc0 msgr2=0x7f905806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:38.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.549+0000 7f905e600640 1 -- 172.21.15.39:0/758445061 shutdown_connections 2024-09-21T07:54:38.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.549+0000 7f905e600640 1 -- 172.21.15.39:0/758445061 wait complete. 2024-09-21T07:54:38.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.551+0000 7f905e600640 1 Processor -- start 2024-09-21T07:54:38.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.551+0000 7f905e600640 1 -- start start 2024-09-21T07:54:38.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.552+0000 7f905e600640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f903c004d40 0x7f90580b4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:38.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.552+0000 7f905e600640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f90581f9800 0x7f90581f9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:38.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.552+0000 7f905e600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f905815b840 0x7f905815bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:38.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.552+0000 7f905e600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f90580ccda0 con 0x7f905815b840 2024-09-21T07:54:38.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.552+0000 7f905e600640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f90580ccc20 con 0x7f903c004d40 2024-09-21T07:54:38.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.552+0000 7f905e600640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f90580ccf20 con 0x7f90581f9800 2024-09-21T07:54:38.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.553+0000 7f9057e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f905815b840 0x7f905815bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:38.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.553+0000 7f9057e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f905815b840 0x7f905815bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38150/0 (socket says 172.21.15.39:38150) 2024-09-21T07:54:38.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.553+0000 7f9057e00640 1 -- 172.21.15.39:0/751741250 learned_addr learned my addr 172.21.15.39:0/751741250 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:38.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.553+0000 7f9057400640 1 --2- 172.21.15.39:0/751741250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f903c004d40 0x7f90580b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:38.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.553+0000 7f904fe00640 1 --2- 172.21.15.39:0/751741250 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f90581f9800 0x7f90581f9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:38.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.553+0000 7f9057e00640 1 -- 172.21.15.39:0/751741250 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f90581f9800 msgr2=0x7f90581f9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:38.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.553+0000 7f9057e00640 1 --2- 172.21.15.39:0/751741250 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f90581f9800 0x7f90581f9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:38.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.553+0000 7f9057e00640 1 -- 172.21.15.39:0/751741250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f903c004d40 msgr2=0x7f90580b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:38.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.553+0000 7f9057e00640 1 --2- 172.21.15.39:0/751741250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f903c004d40 0x7f90580b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:38.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.553+0000 7f9057e00640 1 -- 172.21.15.39:0/751741250 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f905815c190 con 0x7f905815b840 2024-09-21T07:54:38.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.553+0000 7f904fe00640 1 --2- 172.21.15.39:0/751741250 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f90581f9800 0x7f90581f9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:38.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.553+0000 7f9057e00640 1 --2- 172.21.15.39:0/751741250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f905815b840 0x7f905815bc50 secure :-1 s=READY pgs=2802 cs=0 l=1 rev1=1 crypto rx=0x7f9050036040 tx=0x7f9050033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:38.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.554+0000 7f904f400640 1 -- 172.21.15.39:0/751741250 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9050043070 con 0x7f905815b840 2024-09-21T07:54:38.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.554+0000 7f904f400640 1 -- 172.21.15.39:0/751741250 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9050033d00 con 0x7f905815b840 2024-09-21T07:54:38.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.554+0000 7f904f400640 1 -- 172.21.15.39:0/751741250 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9050038040 con 0x7f905815b840 2024-09-21T07:54:38.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.554+0000 7f905e600640 1 -- 172.21.15.39:0/751741250 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90580b34c0 con 0x7f905815b840 2024-09-21T07:54:38.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.554+0000 7f905e600640 1 -- 172.21.15.39:0/751741250 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90581f6140 con 0x7f905815b840 2024-09-21T07:54:38.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.555+0000 7f905e600640 1 -- 172.21.15.39:0/751741250 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f905815cae0 con 0x7f905815b840 2024-09-21T07:54:38.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.556+0000 7f904f400640 1 -- 172.21.15.39:0/751741250 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9050003bd0 con 0x7f905815b840 2024-09-21T07:54:38.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.557+0000 7f904f400640 1 --2- 172.21.15.39:0/751741250 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9034078b90 0x7f903407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:38.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.557+0000 7f904f400640 1 -- 172.21.15.39:0/751741250 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(263..263 src has 1..263) ==== 6944+0+0 (secure 0 0 0) 0x7f90500bb3d0 con 0x7f905815b840 2024-09-21T07:54:38.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.558+0000 7f9057400640 1 --2- 172.21.15.39:0/751741250 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9034078b90 0x7f903407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:38.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.559+0000 7f9057400640 1 --2- 172.21.15.39:0/751741250 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9034078b90 0x7f903407b0a0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f90580730e0 tx=0x7f9048005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:38.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.563+0000 7f904f400640 1 -- 172.21.15.39:0/751741250 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f905815cae0 con 0x7f905815b840 2024-09-21T07:54:38.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:38.802+0000 7f905e600640 1 -- 172.21.15.39:0/751741250 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f90580697d0 con 0x7f905815b840 2024-09-21T07:54:39.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.270+0000 7f904f400640 1 -- 172.21.15.39:0/751741250 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 31 and data pool 32 v306) ==== 174+0+0 (secure 0 0 0) 0x7f90580697d0 con 0x7f905815b840 2024-09-21T07:54:39.272 INFO:teuthology.orchestra.run.smithi039.stderr:new fs with metadata pool 31 and data pool 32 2024-09-21T07:54:39.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.275+0000 7f905e600640 1 -- 172.21.15.39:0/751741250 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9034078b90 msgr2=0x7f903407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:39.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.275+0000 7f905e600640 1 --2- 172.21.15.39:0/751741250 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9034078b90 0x7f903407b0a0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f90580730e0 tx=0x7f9048005f90 comp rx=0 tx=0).stop 2024-09-21T07:54:39.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.275+0000 7f905e600640 1 -- 172.21.15.39:0/751741250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f905815b840 msgr2=0x7f905815bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:39.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.275+0000 7f905e600640 1 --2- 172.21.15.39:0/751741250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f905815b840 0x7f905815bc50 secure :-1 s=READY pgs=2802 cs=0 l=1 rev1=1 crypto rx=0x7f9050036040 tx=0x7f9050033c10 comp rx=0 tx=0).stop 2024-09-21T07:54:39.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.276+0000 7f905e600640 1 -- 172.21.15.39:0/751741250 shutdown_connections 2024-09-21T07:54:39.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.276+0000 7f905e600640 1 --2- 172.21.15.39:0/751741250 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f9034078b90 0x7f903407b0a0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:39.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.276+0000 7f905e600640 1 --2- 172.21.15.39:0/751741250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f905815b840 0x7f905815bc50 unknown :-1 s=CLOSED pgs=2802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:39.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.276+0000 7f905e600640 1 --2- 172.21.15.39:0/751741250 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f90581f9800 0x7f90581f9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:39.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.276+0000 7f905e600640 1 --2- 172.21.15.39:0/751741250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f903c004d40 0x7f90580b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:39.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.276+0000 7f905e600640 1 -- 172.21.15.39:0/751741250 >> 172.21.15.39:0/751741250 conn(0x7f905806cfc0 msgr2=0x7f90580b2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:39.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.277+0000 7f905e600640 1 -- 172.21.15.39:0/751741250 shutdown_connections 2024-09-21T07:54:39.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.277+0000 7f905e600640 1 -- 172.21.15.39:0/751741250 wait complete. 2024-09-21T07:54:39.330 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:54:39.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.556+0000 7fa3ef400640 1 -- 172.21.15.39:0/744804068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3e80b4b60 msgr2=0x7fa3e80af080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:39.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.556+0000 7fa3ef400640 1 --2- 172.21.15.39:0/744804068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3e80b4b60 0x7fa3e80af080 secure :-1 s=READY pgs=2803 cs=0 l=1 rev1=1 crypto rx=0x7fa3e80729e0 tx=0x7fa3e002aa30 comp rx=0 tx=0).stop 2024-09-21T07:54:39.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.556+0000 7fa3ef400640 1 -- 172.21.15.39:0/744804068 shutdown_connections 2024-09-21T07:54:39.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.556+0000 7fa3ef400640 1 --2- 172.21.15.39:0/744804068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3e80b4b60 0x7fa3e80af080 unknown :-1 s=CLOSED pgs=2803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:39.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.556+0000 7fa3ef400640 1 --2- 172.21.15.39:0/744804068 >> v2:172.21.15.148:3300/0 conn(0x7fa3e8162f10 0x7fa3e80b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:39.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.556+0000 7fa3ef400640 1 -- 172.21.15.39:0/744804068 >> 172.21.15.39:0/744804068 conn(0x7fa3e80b2460 msgr2=0x7fa3e806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:39.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.556+0000 7fa3ef400640 1 -- 172.21.15.39:0/744804068 shutdown_connections 2024-09-21T07:54:39.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.556+0000 7fa3ef400640 1 -- 172.21.15.39:0/744804068 wait complete. 2024-09-21T07:54:39.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.558+0000 7fa3ef400640 1 Processor -- start 2024-09-21T07:54:39.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.559+0000 7fa3ef400640 1 -- start start 2024-09-21T07:54:39.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.560+0000 7fa3ef400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa3e80b3990 0x7fa3e809d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:39.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.560+0000 7fa3ef400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3e8162f10 0x7fa3e81f9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:39.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.560+0000 7fa3ef400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3e815d3c0 0x7fa3e81f9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:39.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.560+0000 7fa3ef400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e80b3f20 con 0x7fa3e8162f10 2024-09-21T07:54:39.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.560+0000 7fa3ef400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e80b3da0 con 0x7fa3e815d3c0 2024-09-21T07:54:39.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.560+0000 7fa3ef400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fa3e80b40a0 con 0x7fa3e80b3990 2024-09-21T07:54:39.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.560+0000 7fa3e7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3e8162f10 0x7fa3e81f9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:39.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.560+0000 7fa3e7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3e8162f10 0x7fa3e81f9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38168/0 (socket says 172.21.15.39:38168) 2024-09-21T07:54:39.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.560+0000 7fa3e7e00640 1 -- 172.21.15.39:0/4137772129 learned_addr learned my addr 172.21.15.39:0/4137772129 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:39.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.561+0000 7fa3eca00640 1 --2- 172.21.15.39:0/4137772129 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa3e80b3990 0x7fa3e809d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:39.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.561+0000 7fa3ed400640 1 --2- 172.21.15.39:0/4137772129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3e815d3c0 0x7fa3e81f9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:39.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.561+0000 7fa3e7e00640 1 -- 172.21.15.39:0/4137772129 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa3e80b3990 msgr2=0x7fa3e809d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:39.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.561+0000 7fa3e7e00640 1 --2- 172.21.15.39:0/4137772129 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa3e80b3990 0x7fa3e809d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:39.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.561+0000 7fa3e7e00640 1 -- 172.21.15.39:0/4137772129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3e815d3c0 msgr2=0x7fa3e81f9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:39.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.561+0000 7fa3e7e00640 1 --2- 172.21.15.39:0/4137772129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3e815d3c0 0x7fa3e81f9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:39.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.561+0000 7fa3e7e00640 1 -- 172.21.15.39:0/4137772129 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e81e8660 con 0x7fa3e8162f10 2024-09-21T07:54:39.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.561+0000 7fa3e7e00640 1 --2- 172.21.15.39:0/4137772129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3e8162f10 0x7fa3e81f9040 secure :-1 s=READY pgs=2804 cs=0 l=1 rev1=1 crypto rx=0x7fa3d400c340 tx=0x7fa3d400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:39.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.561+0000 7fa3e4c00640 1 -- 172.21.15.39:0/4137772129 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4017070 con 0x7fa3e8162f10 2024-09-21T07:54:39.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.562+0000 7fa3ef400640 1 -- 172.21.15.39:0/4137772129 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3e80b0990 con 0x7fa3e8162f10 2024-09-21T07:54:39.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.562+0000 7fa3e4c00640 1 -- 172.21.15.39:0/4137772129 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3d40053c0 con 0x7fa3e8162f10 2024-09-21T07:54:39.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.562+0000 7fa3e4c00640 1 -- 172.21.15.39:0/4137772129 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4003850 con 0x7fa3e8162f10 2024-09-21T07:54:39.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.562+0000 7fa3ef400640 1 -- 172.21.15.39:0/4137772129 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3e80b0f50 con 0x7fa3e8162f10 2024-09-21T07:54:39.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.563+0000 7fa3ef400640 1 -- 172.21.15.39:0/4137772129 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3e8075280 con 0x7fa3e8162f10 2024-09-21T07:54:39.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.564+0000 7fa3e4c00640 1 -- 172.21.15.39:0/4137772129 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa3d401a070 con 0x7fa3e8162f10 2024-09-21T07:54:39.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.565+0000 7fa3e4c00640 1 --2- 172.21.15.39:0/4137772129 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa3c8078bc0 0x7fa3c807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:39.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.566+0000 7fa3e4c00640 1 -- 172.21.15.39:0/4137772129 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(264..264 src has 1..264) ==== 7044+0+0 (secure 0 0 0) 0x7fa3d409a590 con 0x7fa3e8162f10 2024-09-21T07:54:39.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.566+0000 7fa3eca00640 1 --2- 172.21.15.39:0/4137772129 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa3c8078bc0 0x7fa3c807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:39.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.567+0000 7fa3eca00640 1 --2- 172.21.15.39:0/4137772129 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa3c8078bc0 0x7fa3c807b0d0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc005fd0 tx=0x7fa3dc005850 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:39.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.570+0000 7fa3e4c00640 1 -- 172.21.15.39:0/4137772129 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa3d4062490 con 0x7fa3e8162f10 2024-09-21T07:54:39.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.771+0000 7fa3ef400640 1 -- 172.21.15.39:0/4137772129 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa3e80cdb70 con 0x7fa3e8162f10 2024-09-21T07:54:39.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.772+0000 7fa3e4c00640 1 -- 172.21.15.39:0/4137772129 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v264) ==== 74+0+22609 (secure 0 0 0) 0x7fa3d4061d90 con 0x7fa3e8162f10 2024-09-21T07:54:39.774 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:39.774 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":264,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:54:38.804569+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:54:36.740920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"264","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2024-09-21T07:54:37.755494+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"264","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]},{"pool":20,"snaps":[{"begin":80,"length":23},{"begin":181,"length":23}]},{"pool":22,"snaps":[{"begin":100,"length":1},{"begin":199,"length":1}]},{"pool":24,"snaps":[{"begin":2,"length":2}]},{"pool":26,"snaps":[{"begin":2,"length":2}]},{"pool":28,"snaps":[{"begin":2,"length":2}]},{"pool":30,"snaps":[{"begin":8,"length":6}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:54:39.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.775+0000 7fa3ef400640 1 -- 172.21.15.39:0/4137772129 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa3c8078bc0 msgr2=0x7fa3c807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:39.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.775+0000 7fa3ef400640 1 --2- 172.21.15.39:0/4137772129 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa3c8078bc0 0x7fa3c807b0d0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc005fd0 tx=0x7fa3dc005850 comp rx=0 tx=0).stop 2024-09-21T07:54:39.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.775+0000 7fa3ef400640 1 -- 172.21.15.39:0/4137772129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3e8162f10 msgr2=0x7fa3e81f9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:39.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.775+0000 7fa3ef400640 1 --2- 172.21.15.39:0/4137772129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3e8162f10 0x7fa3e81f9040 secure :-1 s=READY pgs=2804 cs=0 l=1 rev1=1 crypto rx=0x7fa3d400c340 tx=0x7fa3d400c800 comp rx=0 tx=0).stop 2024-09-21T07:54:39.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.775+0000 7fa3ef400640 1 -- 172.21.15.39:0/4137772129 shutdown_connections 2024-09-21T07:54:39.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.775+0000 7fa3ef400640 1 --2- 172.21.15.39:0/4137772129 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fa3c8078bc0 0x7fa3c807b0d0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:39.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.775+0000 7fa3ef400640 1 --2- 172.21.15.39:0/4137772129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa3e815d3c0 0x7fa3e81f9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:39.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.775+0000 7fa3ef400640 1 --2- 172.21.15.39:0/4137772129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3e8162f10 0x7fa3e81f9040 unknown :-1 s=CLOSED pgs=2804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:39.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.775+0000 7fa3ef400640 1 --2- 172.21.15.39:0/4137772129 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fa3e80b3990 0x7fa3e809d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:39.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.775+0000 7fa3ef400640 1 -- 172.21.15.39:0/4137772129 >> 172.21.15.39:0/4137772129 conn(0x7fa3e80b2460 msgr2=0x7fa3e806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:39.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.775+0000 7fa3ef400640 1 -- 172.21.15.39:0/4137772129 shutdown_connections 2024-09-21T07:54:39.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:39.775+0000 7fa3ef400640 1 -- 172.21.15.39:0/4137772129 wait complete. 2024-09-21T07:54:39.802 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:54:40.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.051+0000 7f203d000640 1 -- 172.21.15.39:0/76676158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2038072da0 msgr2=0x7f2038073700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:40.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.051+0000 7f203d000640 1 --2- 172.21.15.39:0/76676158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2038072da0 0x7f2038073700 secure :-1 s=READY pgs=2806 cs=0 l=1 rev1=1 crypto rx=0x7f2028009830 tx=0x7f202802a490 comp rx=0 tx=0).stop 2024-09-21T07:54:40.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.051+0000 7f203d000640 1 -- 172.21.15.39:0/76676158 shutdown_connections 2024-09-21T07:54:40.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.051+0000 7f203d000640 1 --2- 172.21.15.39:0/76676158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2038072da0 0x7f2038073700 unknown :-1 s=CLOSED pgs=2806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:40.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.051+0000 7f203d000640 1 -- 172.21.15.39:0/76676158 >> 172.21.15.39:0/76676158 conn(0x7f203806cde0 msgr2=0x7f203806d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:40.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.051+0000 7f203d000640 1 -- 172.21.15.39:0/76676158 shutdown_connections 2024-09-21T07:54:40.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.051+0000 7f203d000640 1 -- 172.21.15.39:0/76676158 wait complete. 2024-09-21T07:54:40.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.053+0000 7f203d000640 1 Processor -- start 2024-09-21T07:54:40.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.053+0000 7f203d000640 1 -- start start 2024-09-21T07:54:40.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.055+0000 7f203d000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2038072da0 0x7f20380c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:40.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.055+0000 7f203d000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20380b3870 0x7f2038155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:40.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.055+0000 7f203d000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20380d0b70 0x7f20381557d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:40.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.055+0000 7f203d000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f20380ccc10 con 0x7f2038072da0 2024-09-21T07:54:40.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.055+0000 7f203d000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2038073c40 con 0x7f20380b3870 2024-09-21T07:54:40.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.055+0000 7f203d000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f20381565d0 con 0x7f20380d0b70 2024-09-21T07:54:40.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.055+0000 7f2037400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2038072da0 0x7f20380c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:40.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.055+0000 7f2037400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2038072da0 0x7f20380c7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38194/0 (socket says 172.21.15.39:38194) 2024-09-21T07:54:40.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.055+0000 7f2037400640 1 -- 172.21.15.39:0/1152415630 learned_addr learned my addr 172.21.15.39:0/1152415630 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:40.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.055+0000 7f2036a00640 1 --2- 172.21.15.39:0/1152415630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20380b3870 0x7f2038155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:40.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.055+0000 7f2037400640 1 -- 172.21.15.39:0/1152415630 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20380d0b70 msgr2=0x7f20381557d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:40.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.055+0000 7f2037e00640 1 --2- 172.21.15.39:0/1152415630 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20380d0b70 0x7f20381557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:40.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.055+0000 7f2037400640 1 --2- 172.21.15.39:0/1152415630 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20380d0b70 0x7f20381557d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:40.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.055+0000 7f2037400640 1 -- 172.21.15.39:0/1152415630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20380b3870 msgr2=0x7f2038155290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:40.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.055+0000 7f2037400640 1 --2- 172.21.15.39:0/1152415630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20380b3870 0x7f2038155290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:40.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.056+0000 7f2037400640 1 -- 172.21.15.39:0/1152415630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20381f83f0 con 0x7f2038072da0 2024-09-21T07:54:40.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.056+0000 7f2036a00640 1 --2- 172.21.15.39:0/1152415630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20380b3870 0x7f2038155290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:54:40.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.056+0000 7f2037e00640 1 --2- 172.21.15.39:0/1152415630 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20380d0b70 0x7f20381557d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:54:40.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.056+0000 7f2037400640 1 --2- 172.21.15.39:0/1152415630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2038072da0 0x7f20380c7f50 secure :-1 s=READY pgs=2807 cs=0 l=1 rev1=1 crypto rx=0x7f2028000c00 tx=0x7f2028033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:40.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.056+0000 7f202f400640 1 -- 172.21.15.39:0/1152415630 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2028031d60 con 0x7f2038072da0 2024-09-21T07:54:40.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.056+0000 7f202f400640 1 -- 172.21.15.39:0/1152415630 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2028033070 con 0x7f2038072da0 2024-09-21T07:54:40.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.056+0000 7f202f400640 1 -- 172.21.15.39:0/1152415630 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20280334c0 con 0x7f2038072da0 2024-09-21T07:54:40.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.056+0000 7f203d000640 1 -- 172.21.15.39:0/1152415630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20381f86d0 con 0x7f2038072da0 2024-09-21T07:54:40.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.056+0000 7f203d000640 1 -- 172.21.15.39:0/1152415630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2038154d70 con 0x7f2038072da0 2024-09-21T07:54:40.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.057+0000 7f203d000640 1 -- 172.21.15.39:0/1152415630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20381f8910 con 0x7f2038072da0 2024-09-21T07:54:40.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.059+0000 7f202f400640 1 -- 172.21.15.39:0/1152415630 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2028033660 con 0x7f2038072da0 2024-09-21T07:54:40.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.060+0000 7f202f400640 1 --2- 172.21.15.39:0/1152415630 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2018078b90 0x7f201807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:40.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.060+0000 7f202f400640 1 -- 172.21.15.39:0/1152415630 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(264..264 src has 1..264) ==== 7044+0+0 (secure 0 0 0) 0x7f20280bc8a0 con 0x7f2038072da0 2024-09-21T07:54:40.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.061+0000 7f2036a00640 1 --2- 172.21.15.39:0/1152415630 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2018078b90 0x7f201807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:40.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.062+0000 7f2036a00640 1 --2- 172.21.15.39:0/1152415630 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2018078b90 0x7f201807b0a0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f20240046b0 tx=0x7f2024009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:40.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.065+0000 7f202f400640 1 -- 172.21.15.39:0/1152415630 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f20381f8910 con 0x7f2038072da0 2024-09-21T07:54:40.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.263+0000 7f203d000640 1 -- 172.21.15.39:0/1152415630 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2038073700 con 0x7f2038072da0 2024-09-21T07:54:40.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.264+0000 7f202f400640 1 -- 172.21.15.39:0/1152415630 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v264) ==== 74+0+22609 (secure 0 0 0) 0x7f202803f030 con 0x7f2038072da0 2024-09-21T07:54:40.265 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:40.266 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":264,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:54:38.804569+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:54:36.740920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"264","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2024-09-21T07:54:37.755494+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"264","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]},{"pool":20,"snaps":[{"begin":80,"length":23},{"begin":181,"length":23}]},{"pool":22,"snaps":[{"begin":100,"length":1},{"begin":199,"length":1}]},{"pool":24,"snaps":[{"begin":2,"length":2}]},{"pool":26,"snaps":[{"begin":2,"length":2}]},{"pool":28,"snaps":[{"begin":2,"length":2}]},{"pool":30,"snaps":[{"begin":8,"length":6}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:54:40.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.267+0000 7f203d000640 1 -- 172.21.15.39:0/1152415630 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2018078b90 msgr2=0x7f201807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:40.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.267+0000 7f203d000640 1 --2- 172.21.15.39:0/1152415630 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2018078b90 0x7f201807b0a0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f20240046b0 tx=0x7f2024009040 comp rx=0 tx=0).stop 2024-09-21T07:54:40.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.267+0000 7f203d000640 1 -- 172.21.15.39:0/1152415630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2038072da0 msgr2=0x7f20380c7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:40.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.267+0000 7f203d000640 1 --2- 172.21.15.39:0/1152415630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2038072da0 0x7f20380c7f50 secure :-1 s=READY pgs=2807 cs=0 l=1 rev1=1 crypto rx=0x7f2028000c00 tx=0x7f2028033d60 comp rx=0 tx=0).stop 2024-09-21T07:54:40.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.267+0000 7f203d000640 1 -- 172.21.15.39:0/1152415630 shutdown_connections 2024-09-21T07:54:40.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.267+0000 7f203d000640 1 --2- 172.21.15.39:0/1152415630 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2018078b90 0x7f201807b0a0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:40.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.267+0000 7f203d000640 1 --2- 172.21.15.39:0/1152415630 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f20380d0b70 0x7f20381557d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:40.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.267+0000 7f203d000640 1 --2- 172.21.15.39:0/1152415630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20380b3870 0x7f2038155290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:40.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.267+0000 7f203d000640 1 --2- 172.21.15.39:0/1152415630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2038072da0 0x7f20380c7f50 unknown :-1 s=CLOSED pgs=2807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:40.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.267+0000 7f203d000640 1 -- 172.21.15.39:0/1152415630 >> 172.21.15.39:0/1152415630 conn(0x7f203806cde0 msgr2=0x7f2038169590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:40.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.267+0000 7f203d000640 1 -- 172.21.15.39:0/1152415630 shutdown_connections 2024-09-21T07:54:40.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.267+0000 7f203d000640 1 -- 172.21.15.39:0/1152415630 wait complete. 2024-09-21T07:54:40.293 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T07:54:40.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.521+0000 7f43d3200640 1 -- 172.21.15.39:0/3892166352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43d4160250 msgr2=0x7f43d415d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:40.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.521+0000 7f43d3200640 1 --2- 172.21.15.39:0/3892166352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43d4160250 0x7f43d415d570 secure :-1 s=READY pgs=2808 cs=0 l=1 rev1=1 crypto rx=0x7f43d406dd50 tx=0x7f43c402a580 comp rx=0 tx=0).stop 2024-09-21T07:54:40.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.522+0000 7f43d3200640 1 -- 172.21.15.39:0/3892166352 shutdown_connections 2024-09-21T07:54:40.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.522+0000 7f43d3200640 1 --2- 172.21.15.39:0/3892166352 >> v2:172.21.15.148:3300/0 conn(0x7f43d40cfc50 0x7f43d40d0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:40.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.522+0000 7f43d3200640 1 --2- 172.21.15.39:0/3892166352 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43d4160660 0x7f43d40cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:40.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.522+0000 7f43d3200640 1 --2- 172.21.15.39:0/3892166352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43d4160250 0x7f43d415d570 unknown :-1 s=CLOSED pgs=2808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:40.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.522+0000 7f43d3200640 1 -- 172.21.15.39:0/3892166352 >> 172.21.15.39:0/3892166352 conn(0x7f43d406cfc0 msgr2=0x7f43d406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:40.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.522+0000 7f43d3200640 1 -- 172.21.15.39:0/3892166352 shutdown_connections 2024-09-21T07:54:40.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.522+0000 7f43d3200640 1 -- 172.21.15.39:0/3892166352 wait complete. 2024-09-21T07:54:40.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.524+0000 7f43d3200640 1 Processor -- start 2024-09-21T07:54:40.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.524+0000 7f43d3200640 1 -- start start 2024-09-21T07:54:40.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.525+0000 7f43d3200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43d40cfc50 0x7f43d40b2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:40.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.525+0000 7f43d3200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43d4160250 0x7f43d41460d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:40.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.525+0000 7f43d3200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43d4160660 0x7f43d4146610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:40.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.526+0000 7f43d3200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f43d40ccbb0 con 0x7f43d40cfc50 2024-09-21T07:54:40.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.526+0000 7f43d3200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f43d40cca30 con 0x7f43d4160250 2024-09-21T07:54:40.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.526+0000 7f43d3200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f43d40ccd30 con 0x7f43d4160660 2024-09-21T07:54:40.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.526+0000 7f43d1e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43d40cfc50 0x7f43d40b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:40.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.526+0000 7f43d1e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43d40cfc50 0x7f43d40b2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38212/0 (socket says 172.21.15.39:38212) 2024-09-21T07:54:40.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.526+0000 7f43d1e00640 1 -- 172.21.15.39:0/2803193922 learned_addr learned my addr 172.21.15.39:0/2803193922 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:40.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.526+0000 7f43cbe00640 1 --2- 172.21.15.39:0/2803193922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43d4160250 0x7f43d41460d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:40.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.526+0000 7f43d1e00640 1 -- 172.21.15.39:0/2803193922 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43d4160660 msgr2=0x7f43d4146610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:40.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.526+0000 7f43d2800640 1 --2- 172.21.15.39:0/2803193922 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43d4160660 0x7f43d4146610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:40.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.526+0000 7f43d1e00640 1 --2- 172.21.15.39:0/2803193922 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43d4160660 0x7f43d4146610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:40.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.526+0000 7f43d1e00640 1 -- 172.21.15.39:0/2803193922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43d4160250 msgr2=0x7f43d41460d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:40.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.526+0000 7f43d1e00640 1 --2- 172.21.15.39:0/2803193922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43d4160250 0x7f43d41460d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:40.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.526+0000 7f43d1e00640 1 -- 172.21.15.39:0/2803193922 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d41e6440 con 0x7f43d40cfc50 2024-09-21T07:54:40.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.526+0000 7f43d2800640 1 --2- 172.21.15.39:0/2803193922 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43d4160660 0x7f43d4146610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:54:40.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.527+0000 7f43d1e00640 1 --2- 172.21.15.39:0/2803193922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43d40cfc50 0x7f43d40b2060 secure :-1 s=READY pgs=2809 cs=0 l=1 rev1=1 crypto rx=0x7f43c4000c00 tx=0x7f43c40039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:40.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.527+0000 7f43ca000640 1 -- 172.21.15.39:0/2803193922 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c403e030 con 0x7f43d40cfc50 2024-09-21T07:54:40.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.527+0000 7f43ca000640 1 -- 172.21.15.39:0/2803193922 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43c40047a0 con 0x7f43d40cfc50 2024-09-21T07:54:40.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.527+0000 7f43d3200640 1 -- 172.21.15.39:0/2803193922 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43d40b1580 con 0x7f43d40cfc50 2024-09-21T07:54:40.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.527+0000 7f43ca000640 1 -- 172.21.15.39:0/2803193922 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c4035040 con 0x7f43d40cfc50 2024-09-21T07:54:40.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.527+0000 7f43d3200640 1 -- 172.21.15.39:0/2803193922 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43d40b1b40 con 0x7f43d40cfc50 2024-09-21T07:54:40.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.528+0000 7f43d3200640 1 -- 172.21.15.39:0/2803193922 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43d40aec90 con 0x7f43d40cfc50 2024-09-21T07:54:40.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.529+0000 7f43ca000640 1 -- 172.21.15.39:0/2803193922 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f43c4041030 con 0x7f43d40cfc50 2024-09-21T07:54:40.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.530+0000 7f43ca000640 1 --2- 172.21.15.39:0/2803193922 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f43b0078bc0 0x7f43b007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:40.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.530+0000 7f43ca000640 1 -- 172.21.15.39:0/2803193922 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(265..265 src has 1..265) ==== 7044+0+0 (secure 0 0 0) 0x7f43c40bbc00 con 0x7f43d40cfc50 2024-09-21T07:54:40.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.531+0000 7f43cbe00640 1 --2- 172.21.15.39:0/2803193922 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f43b0078bc0 0x7f43b007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:40.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.532+0000 7f43cbe00640 1 --2- 172.21.15.39:0/2803193922 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f43b0078bc0 0x7f43b007b0d0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f43d414c670 tx=0x7f43bc008040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:40.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.535+0000 7f43ca000640 1 -- 172.21.15.39:0/2803193922 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f43d40aec90 con 0x7f43d40cfc50 2024-09-21T07:54:40.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:40.773+0000 7f43d3200640 1 -- 172.21.15.39:0/2803193922 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f43d40cd260 con 0x7f43d40cfc50 2024-09-21T07:54:41.249 INFO:tasks.ceph.mds.c.smithi039.stderr:2024-09-21T07:54:41.248+0000 7fa436800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:54:41.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.306+0000 7f43ca000640 1 -- 172.21.15.39:0/2803193922 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v309) ==== 118+0+0 (secure 0 0 0) 0x7f43c4083a60 con 0x7f43d40cfc50 2024-09-21T07:54:41.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.309+0000 7f43d3200640 1 -- 172.21.15.39:0/2803193922 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f43b0078bc0 msgr2=0x7f43b007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:41.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.309+0000 7f43d3200640 1 --2- 172.21.15.39:0/2803193922 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f43b0078bc0 0x7f43b007b0d0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f43d414c670 tx=0x7f43bc008040 comp rx=0 tx=0).stop 2024-09-21T07:54:41.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.309+0000 7f43d3200640 1 -- 172.21.15.39:0/2803193922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43d40cfc50 msgr2=0x7f43d40b2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:41.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.309+0000 7f43d3200640 1 --2- 172.21.15.39:0/2803193922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43d40cfc50 0x7f43d40b2060 secure :-1 s=READY pgs=2809 cs=0 l=1 rev1=1 crypto rx=0x7f43c4000c00 tx=0x7f43c40039e0 comp rx=0 tx=0).stop 2024-09-21T07:54:41.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.309+0000 7f43d3200640 1 -- 172.21.15.39:0/2803193922 shutdown_connections 2024-09-21T07:54:41.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.309+0000 7f43d3200640 1 --2- 172.21.15.39:0/2803193922 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f43b0078bc0 0x7f43b007b0d0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:41.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.309+0000 7f43d3200640 1 --2- 172.21.15.39:0/2803193922 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f43d4160660 0x7f43d4146610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:41.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.309+0000 7f43d3200640 1 --2- 172.21.15.39:0/2803193922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43d4160250 0x7f43d41460d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:41.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.309+0000 7f43d3200640 1 --2- 172.21.15.39:0/2803193922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43d40cfc50 0x7f43d40b2060 unknown :-1 s=CLOSED pgs=2809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:41.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.309+0000 7f43d3200640 1 -- 172.21.15.39:0/2803193922 >> 172.21.15.39:0/2803193922 conn(0x7f43d406cfc0 msgr2=0x7f43d40ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:41.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.309+0000 7f43d3200640 1 -- 172.21.15.39:0/2803193922 shutdown_connections 2024-09-21T07:54:41.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.309+0000 7f43d3200640 1 -- 172.21.15.39:0/2803193922 wait complete. 2024-09-21T07:54:41.333 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T07:54:41.333 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T07:54:41.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.562+0000 7fefc8800640 1 -- 172.21.15.39:0/318786636 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefc00b30a0 msgr2=0x7fefc00cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:41.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.562+0000 7fefc8800640 1 --2- 172.21.15.39:0/318786636 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefc00b30a0 0x7fefc00cb8a0 secure :-1 s=READY pgs=2811 cs=0 l=1 rev1=1 crypto rx=0x7fefc00cbde0 tx=0x7fefb402a4a0 comp rx=0 tx=0).stop 2024-09-21T07:54:41.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.562+0000 7fefc8800640 1 -- 172.21.15.39:0/318786636 shutdown_connections 2024-09-21T07:54:41.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.562+0000 7fefc8800640 1 --2- 172.21.15.39:0/318786636 >> v2:172.21.15.148:3300/0 conn(0x7fefc00ccc60 0x7fefc00cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:41.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.562+0000 7fefc8800640 1 --2- 172.21.15.39:0/318786636 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefc00b30a0 0x7fefc00cb8a0 unknown :-1 s=CLOSED pgs=2811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:41.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.562+0000 7fefc8800640 1 -- 172.21.15.39:0/318786636 >> 172.21.15.39:0/318786636 conn(0x7fefc00b2460 msgr2=0x7fefc006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:41.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.562+0000 7fefc8800640 1 -- 172.21.15.39:0/318786636 shutdown_connections 2024-09-21T07:54:41.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.562+0000 7fefc8800640 1 -- 172.21.15.39:0/318786636 wait complete. 2024-09-21T07:54:41.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.564+0000 7fefc8800640 1 Processor -- start 2024-09-21T07:54:41.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.564+0000 7fefc8800640 1 -- start start 2024-09-21T07:54:41.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.566+0000 7fefc8800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefc00b30a0 0x7fefc015bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:41.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.566+0000 7fefc8800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefc00ccc60 0x7fefc0155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:41.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.566+0000 7fefc8800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefc015d830 0x7fefc00c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:41.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.566+0000 7fefc8800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fefc00af410 con 0x7fefc00ccc60 2024-09-21T07:54:41.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.566+0000 7fefc8800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fefc00af290 con 0x7fefc015d830 2024-09-21T07:54:41.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.566+0000 7fefc8800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fefc00af590 con 0x7fefc00b30a0 2024-09-21T07:54:41.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.566+0000 7fefc5400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefc00ccc60 0x7fefc0155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:41.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.566+0000 7fefc5400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefc00ccc60 0x7fefc0155dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38230/0 (socket says 172.21.15.39:38230) 2024-09-21T07:54:41.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.566+0000 7fefc5400640 1 -- 172.21.15.39:0/1127856613 learned_addr learned my addr 172.21.15.39:0/1127856613 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:41.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.566+0000 7fefc5e00640 1 --2- 172.21.15.39:0/1127856613 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefc00b30a0 0x7fefc015bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:41.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.566+0000 7fefc6800640 1 --2- 172.21.15.39:0/1127856613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefc015d830 0x7fefc00c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:41.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.566+0000 7fefc5400640 1 -- 172.21.15.39:0/1127856613 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefc00b30a0 msgr2=0x7fefc015bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:41.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.566+0000 7fefc5400640 1 --2- 172.21.15.39:0/1127856613 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefc00b30a0 0x7fefc015bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:41.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.566+0000 7fefc5400640 1 -- 172.21.15.39:0/1127856613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefc015d830 msgr2=0x7fefc00c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:41.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.566+0000 7fefc5400640 1 --2- 172.21.15.39:0/1127856613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefc015d830 0x7fefc00c7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:41.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.566+0000 7fefc5400640 1 -- 172.21.15.39:0/1127856613 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc01f9470 con 0x7fefc00ccc60 2024-09-21T07:54:41.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.566+0000 7fefc5e00640 1 --2- 172.21.15.39:0/1127856613 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefc00b30a0 0x7fefc015bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:54:41.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.567+0000 7fefc5400640 1 --2- 172.21.15.39:0/1127856613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefc00ccc60 0x7fefc0155dd0 secure :-1 s=READY pgs=2812 cs=0 l=1 rev1=1 crypto rx=0x7fefb000c3c0 tx=0x7fefb000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:41.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.567+0000 7fefba000640 1 -- 172.21.15.39:0/1127856613 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb0017070 con 0x7fefc00ccc60 2024-09-21T07:54:41.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.567+0000 7fefba000640 1 -- 172.21.15.39:0/1127856613 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fefb00035f0 con 0x7fefc00ccc60 2024-09-21T07:54:41.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.567+0000 7fefba000640 1 -- 172.21.15.39:0/1127856613 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb0003a80 con 0x7fefc00ccc60 2024-09-21T07:54:41.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.567+0000 7fefc8800640 1 -- 172.21.15.39:0/1127856613 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc01f9750 con 0x7fefc00ccc60 2024-09-21T07:54:41.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.567+0000 7fefc8800640 1 -- 172.21.15.39:0/1127856613 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc01f9d10 con 0x7fefc00ccc60 2024-09-21T07:54:41.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.568+0000 7fefc8800640 1 -- 172.21.15.39:0/1127856613 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefc00afa40 con 0x7fefc00ccc60 2024-09-21T07:54:41.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.570+0000 7fefba000640 1 -- 172.21.15.39:0/1127856613 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fefb001a070 con 0x7fefc00ccc60 2024-09-21T07:54:41.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.570+0000 7fefba000640 1 --2- 172.21.15.39:0/1127856613 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fefa4078bc0 0x7fefa407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:41.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.571+0000 7fefba000640 1 -- 172.21.15.39:0/1127856613 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(265..265 src has 1..265) ==== 7044+0+0 (secure 0 0 0) 0x7fefb009a3f0 con 0x7fefc00ccc60 2024-09-21T07:54:41.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.571+0000 7fefc5e00640 1 --2- 172.21.15.39:0/1127856613 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fefa4078bc0 0x7fefa407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:41.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.572+0000 7fefc5e00640 1 --2- 172.21.15.39:0/1127856613 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fefa4078bc0 0x7fefa407b0d0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fefb4034000 tx=0x7fefb4004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:41.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.575+0000 7fefba000640 1 -- 172.21.15.39:0/1127856613 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fefc00cd5b0 con 0x7fefc00ccc60 2024-09-21T07:54:41.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:41.813+0000 7fefc8800640 1 -- 172.21.15.39:0/1127856613 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fefc00ca680 con 0x7fefc00ccc60 2024-09-21T07:54:42.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.318+0000 7fefba000640 1 -- 172.21.15.39:0/1127856613 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v310) ==== 122+0+0 (secure 0 0 0) 0x7fefb0062410 con 0x7fefc00ccc60 2024-09-21T07:54:42.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.321+0000 7fefc8800640 1 -- 172.21.15.39:0/1127856613 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fefa4078bc0 msgr2=0x7fefa407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:42.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.321+0000 7fefc8800640 1 --2- 172.21.15.39:0/1127856613 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fefa4078bc0 0x7fefa407b0d0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fefb4034000 tx=0x7fefb4004730 comp rx=0 tx=0).stop 2024-09-21T07:54:42.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.321+0000 7fefc8800640 1 -- 172.21.15.39:0/1127856613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefc00ccc60 msgr2=0x7fefc0155dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:42.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.321+0000 7fefc8800640 1 --2- 172.21.15.39:0/1127856613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefc00ccc60 0x7fefc0155dd0 secure :-1 s=READY pgs=2812 cs=0 l=1 rev1=1 crypto rx=0x7fefb000c3c0 tx=0x7fefb000c880 comp rx=0 tx=0).stop 2024-09-21T07:54:42.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.321+0000 7fefc8800640 1 -- 172.21.15.39:0/1127856613 shutdown_connections 2024-09-21T07:54:42.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.321+0000 7fefc8800640 1 --2- 172.21.15.39:0/1127856613 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fefa4078bc0 0x7fefa407b0d0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:42.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.321+0000 7fefc8800640 1 --2- 172.21.15.39:0/1127856613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefc015d830 0x7fefc00c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:42.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.321+0000 7fefc8800640 1 --2- 172.21.15.39:0/1127856613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefc00ccc60 0x7fefc0155dd0 unknown :-1 s=CLOSED pgs=2812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:42.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.321+0000 7fefc8800640 1 --2- 172.21.15.39:0/1127856613 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fefc00b30a0 0x7fefc015bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:42.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.321+0000 7fefc8800640 1 -- 172.21.15.39:0/1127856613 >> 172.21.15.39:0/1127856613 conn(0x7fefc00b2460 msgr2=0x7fefc0156bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:42.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.321+0000 7fefc8800640 1 -- 172.21.15.39:0/1127856613 shutdown_connections 2024-09-21T07:54:42.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.321+0000 7fefc8800640 1 -- 172.21.15.39:0/1127856613 wait complete. 2024-09-21T07:54:42.350 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T07:54:42.350 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:54:42.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.578+0000 7f94ef000640 1 -- 172.21.15.39:0/4102398137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94e80b3da0 msgr2=0x7f94e80b4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:42.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.578+0000 7f94ef000640 1 --2- 172.21.15.39:0/4102398137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94e80b3da0 0x7f94e80b4990 secure :-1 s=READY pgs=2814 cs=0 l=1 rev1=1 crypto rx=0x7f94e80b2e40 tx=0x7f94e002aac0 comp rx=0 tx=0).stop 2024-09-21T07:54:42.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.578+0000 7f94ef000640 1 -- 172.21.15.39:0/4102398137 shutdown_connections 2024-09-21T07:54:42.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.578+0000 7f94ef000640 1 --2- 172.21.15.39:0/4102398137 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94e80b3da0 0x7f94e80b4990 unknown :-1 s=CLOSED pgs=2814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:42.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.578+0000 7f94ef000640 1 --2- 172.21.15.39:0/4102398137 >> v2:172.21.15.148:3300/0 conn(0x7f94e80b3990 0x7f94e80b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:42.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.578+0000 7f94ef000640 1 -- 172.21.15.39:0/4102398137 >> 172.21.15.39:0/4102398137 conn(0x7f94e80b2460 msgr2=0x7f94e806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:42.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.578+0000 7f94ef000640 1 -- 172.21.15.39:0/4102398137 shutdown_connections 2024-09-21T07:54:42.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.578+0000 7f94ef000640 1 -- 172.21.15.39:0/4102398137 wait complete. 2024-09-21T07:54:42.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.580+0000 7f94ef000640 1 Processor -- start 2024-09-21T07:54:42.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.580+0000 7f94ef000640 1 -- start start 2024-09-21T07:54:42.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.582+0000 7f94ef000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94e80b3990 0x7f94e81f9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:42.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.582+0000 7f94ef000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f94e80b3da0 0x7f94e814f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:42.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.582+0000 7f94ef000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94e8162f10 0x7f94e80b0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:42.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.582+0000 7f94ef000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f94e80af200 con 0x7f94e8162f10 2024-09-21T07:54:42.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.582+0000 7f94ef000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f94e80af080 con 0x7f94e80b3990 2024-09-21T07:54:42.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.582+0000 7f94ef000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f94e80af380 con 0x7f94e80b3da0 2024-09-21T07:54:42.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.582+0000 7f94ed000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94e8162f10 0x7f94e80b0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:42.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.582+0000 7f94ed000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94e8162f10 0x7f94e80b0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47274/0 (socket says 172.21.15.39:47274) 2024-09-21T07:54:42.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.582+0000 7f94ed000640 1 -- 172.21.15.39:0/1602257382 learned_addr learned my addr 172.21.15.39:0/1602257382 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:42.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.582+0000 7f94df400640 1 --2- 172.21.15.39:0/1602257382 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f94e80b3da0 0x7f94e814f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:42.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.582+0000 7f94e7e00640 1 --2- 172.21.15.39:0/1602257382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94e80b3990 0x7f94e81f9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:42.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.583+0000 7f94ed000640 1 -- 172.21.15.39:0/1602257382 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f94e80b3da0 msgr2=0x7f94e814f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:42.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.583+0000 7f94ed000640 1 --2- 172.21.15.39:0/1602257382 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f94e80b3da0 0x7f94e814f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:42.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.583+0000 7f94ed000640 1 -- 172.21.15.39:0/1602257382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94e80b3990 msgr2=0x7f94e81f9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:42.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.583+0000 7f94ed000640 1 --2- 172.21.15.39:0/1602257382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94e80b3990 0x7f94e81f9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:42.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.583+0000 7f94ed000640 1 -- 172.21.15.39:0/1602257382 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94e81f9e90 con 0x7f94e8162f10 2024-09-21T07:54:42.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.583+0000 7f94e7e00640 1 --2- 172.21.15.39:0/1602257382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94e80b3990 0x7f94e81f9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:42.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.583+0000 7f94ed000640 1 --2- 172.21.15.39:0/1602257382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94e8162f10 0x7f94e80b0c50 secure :-1 s=READY pgs=2815 cs=0 l=1 rev1=1 crypto rx=0x7f94e0009d60 tx=0x7f94e0003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:42.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.583+0000 7f94e4c00640 1 -- 172.21.15.39:0/1602257382 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e0044070 con 0x7f94e8162f10 2024-09-21T07:54:42.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.583+0000 7f94e4c00640 1 -- 172.21.15.39:0/1602257382 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f94e0004250 con 0x7f94e8162f10 2024-09-21T07:54:42.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.583+0000 7f94e4c00640 1 -- 172.21.15.39:0/1602257382 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e00046a0 con 0x7f94e8162f10 2024-09-21T07:54:42.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.583+0000 7f94ef000640 1 -- 172.21.15.39:0/1602257382 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94e8074ee0 con 0x7f94e8162f10 2024-09-21T07:54:42.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.583+0000 7f94ef000640 1 -- 172.21.15.39:0/1602257382 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94e8074c00 con 0x7f94e8162f10 2024-09-21T07:54:42.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.585+0000 7f94ef000640 1 -- 172.21.15.39:0/1602257382 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94e80cdb70 con 0x7f94e8162f10 2024-09-21T07:54:42.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.586+0000 7f94e4c00640 1 -- 172.21.15.39:0/1602257382 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f94e0036400 con 0x7f94e8162f10 2024-09-21T07:54:42.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.587+0000 7f94e4c00640 1 --2- 172.21.15.39:0/1602257382 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f94c4078b90 0x7f94c407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:42.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.587+0000 7f94e4c00640 1 -- 172.21.15.39:0/1602257382 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(265..265 src has 1..265) ==== 7044+0+0 (secure 0 0 0) 0x7f94e00bb9e0 con 0x7f94e8162f10 2024-09-21T07:54:42.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.587+0000 7f94e7e00640 1 --2- 172.21.15.39:0/1602257382 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f94c4078b90 0x7f94c407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:42.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.588+0000 7f94e7e00640 1 --2- 172.21.15.39:0/1602257382 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f94c4078b90 0x7f94c407b0a0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f94d8000f60 tx=0x7f94d8009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:42.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.592+0000 7f94e4c00640 1 -- 172.21.15.39:0/1602257382 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f94e81f9e90 con 0x7f94e8162f10 2024-09-21T07:54:42.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.826+0000 7f94ef000640 1 -- 172.21.15.39:0/1602257382 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f94e80b3240 con 0x7f94e8162f10 2024-09-21T07:54:42.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.826+0000 7f94e4c00640 1 -- 172.21.15.39:0/1602257382 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 310 v310) ==== 95+0+5243 (secure 0 0 0) 0x7f94e0032030 con 0x7f94e8162f10 2024-09-21T07:54:42.827 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:42.827 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":310,"btime":"2024-09-21T07:54:42:307372+0000","default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9725,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1018044536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1018044536},{"type":"v1","addr":"172.21.15.148:6837","nonce":1018044536}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":305},{"gid":11125,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1771887993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1771887993},{"type":"v1","addr":"172.21.15.148:6836","nonce":1771887993}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":304},{"gid":11131,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/649113662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":649113662},{"type":"v1","addr":"172.21.15.39:6835","nonce":649113662}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":305}],"filesystems":[{"mdsmap":{"epoch":310,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:54:38.804512+0000","modified":"2024-09-21T07:54:41.814517+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":12315},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12315":{"gid":12315,"name":"c","rank":0,"incarnation":306,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/3094159467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3094159467},{"type":"v1","addr":"172.21.15.39:6834","nonce":3094159467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":12315,"qdb_cluster":[12315]},"id":14}]} 2024-09-21T07:54:42.828 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 310 2024-09-21T07:54:42.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.829+0000 7f94ef000640 1 -- 172.21.15.39:0/1602257382 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f94c4078b90 msgr2=0x7f94c407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:42.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.829+0000 7f94ef000640 1 --2- 172.21.15.39:0/1602257382 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f94c4078b90 0x7f94c407b0a0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f94d8000f60 tx=0x7f94d8009040 comp rx=0 tx=0).stop 2024-09-21T07:54:42.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.829+0000 7f94ef000640 1 -- 172.21.15.39:0/1602257382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94e8162f10 msgr2=0x7f94e80b0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:42.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.829+0000 7f94ef000640 1 --2- 172.21.15.39:0/1602257382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94e8162f10 0x7f94e80b0c50 secure :-1 s=READY pgs=2815 cs=0 l=1 rev1=1 crypto rx=0x7f94e0009d60 tx=0x7f94e0003ce0 comp rx=0 tx=0).stop 2024-09-21T07:54:42.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.829+0000 7f94ef000640 1 -- 172.21.15.39:0/1602257382 shutdown_connections 2024-09-21T07:54:42.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.829+0000 7f94ef000640 1 --2- 172.21.15.39:0/1602257382 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f94c4078b90 0x7f94c407b0a0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:42.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.829+0000 7f94ef000640 1 --2- 172.21.15.39:0/1602257382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94e8162f10 0x7f94e80b0c50 unknown :-1 s=CLOSED pgs=2815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:42.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.829+0000 7f94ef000640 1 --2- 172.21.15.39:0/1602257382 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f94e80b3da0 0x7f94e814f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:42.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.829+0000 7f94ef000640 1 --2- 172.21.15.39:0/1602257382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94e80b3990 0x7f94e81f9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:42.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.829+0000 7f94ef000640 1 -- 172.21.15.39:0/1602257382 >> 172.21.15.39:0/1602257382 conn(0x7f94e80b2460 msgr2=0x7f94e81616b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:42.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.829+0000 7f94ef000640 1 -- 172.21.15.39:0/1602257382 shutdown_connections 2024-09-21T07:54:42.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:42.829+0000 7f94ef000640 1 -- 172.21.15.39:0/1602257382 wait complete. 2024-09-21T07:54:42.852 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:54:43.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.079+0000 7f69a6200640 1 -- 172.21.15.39:0/1025975002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6984003a00 msgr2=0x7f6984003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:43.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.079+0000 7f69a6200640 1 --2- 172.21.15.39:0/1025975002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6984003a00 0x7f6984003e70 secure :-1 s=READY pgs=2817 cs=0 l=1 rev1=1 crypto rx=0x7f6990009b50 tx=0x7f6984005830 comp rx=0 tx=0).stop 2024-09-21T07:54:43.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.079+0000 7f699fe00640 1 -- 172.21.15.39:0/1025975002 reap_dead start 2024-09-21T07:54:43.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.079+0000 7f69a6200640 1 -- 172.21.15.39:0/1025975002 shutdown_connections 2024-09-21T07:54:43.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.079+0000 7f69a6200640 1 -- 172.21.15.39:0/1025975002 >> 172.21.15.39:0/1025975002 conn(0x7f69a00b2460 msgr2=0x7f69a006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:43.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.079+0000 7f69a6200640 1 -- 172.21.15.39:0/1025975002 shutdown_connections 2024-09-21T07:54:43.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.079+0000 7f69a6200640 1 -- 172.21.15.39:0/1025975002 wait complete. 2024-09-21T07:54:43.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.081+0000 7f69a6200640 1 Processor -- start 2024-09-21T07:54:43.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.081+0000 7f69a6200640 1 -- start start 2024-09-21T07:54:43.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.083+0000 7f69a6200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6984003a00 0x7f69a01f2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:43.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.083+0000 7f69a6200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69a009afd0 0x7f69a00c6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:43.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.083+0000 7f69a6200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69a00c86f0 0x7f69a00c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:43.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.083+0000 7f69a6200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f69a0157210 con 0x7f69a00c86f0 2024-09-21T07:54:43.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.083+0000 7f69a6200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f69a0157090 con 0x7f69a009afd0 2024-09-21T07:54:43.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.083+0000 7f69a6200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f69a00af080 con 0x7f6984003a00 2024-09-21T07:54:43.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.083+0000 7f699fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69a00c86f0 0x7f69a00c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:43.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.083+0000 7f699fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69a00c86f0 0x7f69a00c8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47286/0 (socket says 172.21.15.39:47286) 2024-09-21T07:54:43.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.083+0000 7f699fe00640 1 -- 172.21.15.39:0/418017446 learned_addr learned my addr 172.21.15.39:0/418017446 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:43.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.083+0000 7f699f400640 1 --2- 172.21.15.39:0/418017446 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6984003a00 0x7f69a01f2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:43.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.084+0000 7f6997e00640 1 --2- 172.21.15.39:0/418017446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69a009afd0 0x7f69a00c6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:43.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.084+0000 7f699fe00640 1 -- 172.21.15.39:0/418017446 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6984003a00 msgr2=0x7f69a01f2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:43.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.084+0000 7f699fe00640 1 --2- 172.21.15.39:0/418017446 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6984003a00 0x7f69a01f2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:43.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.084+0000 7f699fe00640 1 -- 172.21.15.39:0/418017446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69a009afd0 msgr2=0x7f69a00c6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:43.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.084+0000 7f699fe00640 1 --2- 172.21.15.39:0/418017446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69a009afd0 0x7f69a00c6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:43.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.084+0000 7f699fe00640 1 -- 172.21.15.39:0/418017446 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a00c7650 con 0x7f69a00c86f0 2024-09-21T07:54:43.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.084+0000 7f699fe00640 1 --2- 172.21.15.39:0/418017446 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69a00c86f0 0x7f69a00c8b00 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 crypto rx=0x7f69a00cb690 tx=0x7f699800ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:43.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.084+0000 7f6997400640 1 -- 172.21.15.39:0/418017446 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699800cc20 con 0x7f69a00c86f0 2024-09-21T07:54:43.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.084+0000 7f6997400640 1 -- 172.21.15.39:0/418017446 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6998018070 con 0x7f69a00c86f0 2024-09-21T07:54:43.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.084+0000 7f6997400640 1 -- 172.21.15.39:0/418017446 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69980116c0 con 0x7f69a00c86f0 2024-09-21T07:54:43.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.084+0000 7f69a6200640 1 -- 172.21.15.39:0/418017446 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69a00c7930 con 0x7f69a00c86f0 2024-09-21T07:54:43.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.085+0000 7f69a6200640 1 -- 172.21.15.39:0/418017446 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69a00c7ef0 con 0x7f69a00c86f0 2024-09-21T07:54:43.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.085+0000 7f69a6200640 1 -- 172.21.15.39:0/418017446 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69a00cb8a0 con 0x7f69a00c86f0 2024-09-21T07:54:43.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.087+0000 7f6997400640 1 -- 172.21.15.39:0/418017446 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6998011860 con 0x7f69a00c86f0 2024-09-21T07:54:43.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.088+0000 7f6997400640 1 --2- 172.21.15.39:0/418017446 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f697c078bc0 0x7f697c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:43.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.088+0000 7f6997400640 1 -- 172.21.15.39:0/418017446 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(265..265 src has 1..265) ==== 7044+0+0 (secure 0 0 0) 0x7f699809b640 con 0x7f69a00c86f0 2024-09-21T07:54:43.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.088+0000 7f699f400640 1 --2- 172.21.15.39:0/418017446 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f697c078bc0 0x7f697c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:43.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.089+0000 7f699f400640 1 --2- 172.21.15.39:0/418017446 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f697c078bc0 0x7f697c07b0d0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f6990035000 tx=0x7f69900046c0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:43.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.092+0000 7f6997400640 1 -- 172.21.15.39:0/418017446 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6998063530 con 0x7f69a00c86f0 2024-09-21T07:54:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.287+0000 7f69a6200640 1 -- 172.21.15.39:0/418017446 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f69a00ad3e0 con 0x7f69a00c86f0 2024-09-21T07:54:43.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.289+0000 7f6997400640 1 -- 172.21.15.39:0/418017446 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v265) ==== 74+0+22596 (secure 0 0 0) 0x7f6998062e30 con 0x7f69a00c86f0 2024-09-21T07:54:43.290 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:43.290 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":265,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:54:40.270721+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:54:36.740920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"264","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2024-09-21T07:54:37.755494+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"265","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]},{"pool":20,"snaps":[{"begin":80,"length":23},{"begin":181,"length":23}]},{"pool":22,"snaps":[{"begin":100,"length":1},{"begin":199,"length":1}]},{"pool":24,"snaps":[{"begin":2,"length":2}]},{"pool":26,"snaps":[{"begin":2,"length":2}]},{"pool":28,"snaps":[{"begin":2,"length":2}]},{"pool":30,"snaps":[{"begin":8,"length":6}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:54:43.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.291+0000 7f69a6200640 1 -- 172.21.15.39:0/418017446 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f697c078bc0 msgr2=0x7f697c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:43.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.291+0000 7f69a6200640 1 --2- 172.21.15.39:0/418017446 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f697c078bc0 0x7f697c07b0d0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f6990035000 tx=0x7f69900046c0 comp rx=0 tx=0).stop 2024-09-21T07:54:43.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.291+0000 7f69a6200640 1 -- 172.21.15.39:0/418017446 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69a00c86f0 msgr2=0x7f69a00c8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:43.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.291+0000 7f69a6200640 1 --2- 172.21.15.39:0/418017446 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69a00c86f0 0x7f69a00c8b00 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 crypto rx=0x7f69a00cb690 tx=0x7f699800ed10 comp rx=0 tx=0).stop 2024-09-21T07:54:43.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.291+0000 7f69a6200640 1 -- 172.21.15.39:0/418017446 shutdown_connections 2024-09-21T07:54:43.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.291+0000 7f69a6200640 1 --2- 172.21.15.39:0/418017446 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f697c078bc0 0x7f697c07b0d0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:43.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.291+0000 7f69a6200640 1 --2- 172.21.15.39:0/418017446 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69a00c86f0 0x7f69a00c8b00 unknown :-1 s=CLOSED pgs=2818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:43.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.291+0000 7f69a6200640 1 --2- 172.21.15.39:0/418017446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f69a009afd0 0x7f69a00c6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:43.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.291+0000 7f69a6200640 1 --2- 172.21.15.39:0/418017446 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6984003a00 0x7f69a01f2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:43.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.291+0000 7f69a6200640 1 -- 172.21.15.39:0/418017446 >> 172.21.15.39:0/418017446 conn(0x7f69a00b2460 msgr2=0x7f69a00b15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:43.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.291+0000 7f69a6200640 1 -- 172.21.15.39:0/418017446 shutdown_connections 2024-09-21T07:54:43.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.292+0000 7f69a6200640 1 -- 172.21.15.39:0/418017446 wait complete. 2024-09-21T07:54:43.314 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T07:54:43.315 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:54:43.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.542+0000 7f80e1400640 1 -- 172.21.15.39:0/2940440134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80dc0d0630 msgr2=0x7f80dc0d0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:43.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.542+0000 7f80e1400640 1 --2- 172.21.15.39:0/2940440134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80dc0d0630 0x7f80dc0d0a40 secure :-1 s=READY pgs=2820 cs=0 l=1 rev1=1 crypto rx=0x7f80d4005fd0 tx=0x7f80d402a8b0 comp rx=0 tx=0).stop 2024-09-21T07:54:43.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.543+0000 7f80e1400640 1 -- 172.21.15.39:0/2940440134 shutdown_connections 2024-09-21T07:54:43.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.543+0000 7f80e1400640 1 --2- 172.21.15.39:0/2940440134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80dc0d0630 0x7f80dc0d0a40 unknown :-1 s=CLOSED pgs=2820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:43.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.543+0000 7f80e1400640 1 --2- 172.21.15.39:0/2940440134 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80dc0cf450 0x7f80dc0cf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:43.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.543+0000 7f80e1400640 1 -- 172.21.15.39:0/2940440134 >> 172.21.15.39:0/2940440134 conn(0x7f80dc06cfc0 msgr2=0x7f80dc06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:43.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.543+0000 7f80e1400640 1 -- 172.21.15.39:0/2940440134 shutdown_connections 2024-09-21T07:54:43.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.543+0000 7f80e1400640 1 -- 172.21.15.39:0/2940440134 wait complete. 2024-09-21T07:54:43.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.545+0000 7f80e1400640 1 Processor -- start 2024-09-21T07:54:43.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.545+0000 7f80e1400640 1 -- start start 2024-09-21T07:54:43.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.547+0000 7f80e1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80dc073dd0 0x7f80dc0b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:43.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.547+0000 7f80e1400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80dc0cf450 0x7f80dc0b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:43.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.547+0000 7f80e1400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80dc0d0630 0x7f80dc14d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:43.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.547+0000 7f80e1400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f80dc0cd1a0 con 0x7f80dc073dd0 2024-09-21T07:54:43.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.547+0000 7f80e1400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f80dc0cd020 con 0x7f80dc0cf450 2024-09-21T07:54:43.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.547+0000 7f80e1400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f80dc0cd320 con 0x7f80dc0d0630 2024-09-21T07:54:43.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.547+0000 7f80da800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80dc073dd0 0x7f80dc0b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:43.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.547+0000 7f80da800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80dc073dd0 0x7f80dc0b46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47304/0 (socket says 172.21.15.39:47304) 2024-09-21T07:54:43.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.547+0000 7f80da800640 1 -- 172.21.15.39:0/3772251570 learned_addr learned my addr 172.21.15.39:0/3772251570 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:43.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.547+0000 7f80db200640 1 --2- 172.21.15.39:0/3772251570 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80dc0d0630 0x7f80dc14d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:43.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.547+0000 7f80d9e00640 1 --2- 172.21.15.39:0/3772251570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80dc0cf450 0x7f80dc0b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:43.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.547+0000 7f80da800640 1 -- 172.21.15.39:0/3772251570 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80dc0d0630 msgr2=0x7f80dc14d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:43.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.547+0000 7f80da800640 1 --2- 172.21.15.39:0/3772251570 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80dc0d0630 0x7f80dc14d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:43.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.547+0000 7f80da800640 1 -- 172.21.15.39:0/3772251570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80dc0cf450 msgr2=0x7f80dc0b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:43.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.547+0000 7f80da800640 1 --2- 172.21.15.39:0/3772251570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80dc0cf450 0x7f80dc0b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:43.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.547+0000 7f80da800640 1 -- 172.21.15.39:0/3772251570 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80dc0b54b0 con 0x7f80dc073dd0 2024-09-21T07:54:43.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.547+0000 7f80db200640 1 --2- 172.21.15.39:0/3772251570 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80dc0d0630 0x7f80dc14d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:43.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.548+0000 7f80da800640 1 --2- 172.21.15.39:0/3772251570 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80dc073dd0 0x7f80dc0b46a0 secure :-1 s=READY pgs=2821 cs=0 l=1 rev1=1 crypto rx=0x7f80cc00c370 tx=0x7f80cc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:43.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.548+0000 7f80d2a00640 1 -- 172.21.15.39:0/3772251570 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc011790 con 0x7f80dc073dd0 2024-09-21T07:54:43.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.548+0000 7f80d2a00640 1 -- 172.21.15.39:0/3772251570 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80cc017070 con 0x7f80dc073dd0 2024-09-21T07:54:43.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.548+0000 7f80d2a00640 1 -- 172.21.15.39:0/3772251570 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc011d20 con 0x7f80dc073dd0 2024-09-21T07:54:43.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.548+0000 7f80e1400640 1 -- 172.21.15.39:0/3772251570 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80dc14d950 con 0x7f80dc073dd0 2024-09-21T07:54:43.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.548+0000 7f80e1400640 1 -- 172.21.15.39:0/3772251570 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80dc0b4ef0 con 0x7f80dc073dd0 2024-09-21T07:54:43.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.549+0000 7f80e1400640 1 -- 172.21.15.39:0/3772251570 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80dc0cd7d0 con 0x7f80dc073dd0 2024-09-21T07:54:43.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.550+0000 7f80d2a00640 1 -- 172.21.15.39:0/3772251570 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f80cc01c460 con 0x7f80dc073dd0 2024-09-21T07:54:43.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.551+0000 7f80d2a00640 1 --2- 172.21.15.39:0/3772251570 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f80bc078bc0 0x7f80bc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:43.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.552+0000 7f80d2a00640 1 -- 172.21.15.39:0/3772251570 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(265..265 src has 1..265) ==== 7044+0+0 (secure 0 0 0) 0x7f80cc09b000 con 0x7f80dc073dd0 2024-09-21T07:54:43.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.552+0000 7f80d9e00640 1 --2- 172.21.15.39:0/3772251570 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f80bc078bc0 0x7f80bc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:43.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.553+0000 7f80d9e00640 1 --2- 172.21.15.39:0/3772251570 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f80bc078bc0 0x7f80bc07b0d0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f80c8005fd0 tx=0x7f80c8005850 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:43.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.556+0000 7f80d2a00640 1 -- 172.21.15.39:0/3772251570 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f80dc0d0f80 con 0x7f80dc073dd0 2024-09-21T07:54:43.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.747+0000 7f80e1400640 1 -- 172.21.15.39:0/3772251570 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f80dc0b00d0 con 0x7f80bc078bc0 2024-09-21T07:54:43.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.757+0000 7f80d2a00640 1 -- 172.21.15.39:0/3772251570 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+486407 (secure 0 0 0) 0x7f80dc0b00d0 con 0x7f80bc078bc0 2024-09-21T07:54:43.764 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:43.767 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:54:43.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.763+0000 7f80e1400640 1 -- 172.21.15.39:0/3772251570 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f80bc078bc0 msgr2=0x7f80bc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:43.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.763+0000 7f80e1400640 1 --2- 172.21.15.39:0/3772251570 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f80bc078bc0 0x7f80bc07b0d0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f80c8005fd0 tx=0x7f80c8005850 comp rx=0 tx=0).stop 2024-09-21T07:54:43.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.763+0000 7f80e1400640 1 -- 172.21.15.39:0/3772251570 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80dc073dd0 msgr2=0x7f80dc0b46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:43.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.763+0000 7f80e1400640 1 --2- 172.21.15.39:0/3772251570 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80dc073dd0 0x7f80dc0b46a0 secure :-1 s=READY pgs=2821 cs=0 l=1 rev1=1 crypto rx=0x7f80cc00c370 tx=0x7f80cc00c830 comp rx=0 tx=0).stop 2024-09-21T07:54:43.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.763+0000 7f80e1400640 1 -- 172.21.15.39:0/3772251570 shutdown_connections 2024-09-21T07:54:43.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.763+0000 7f80e1400640 1 --2- 172.21.15.39:0/3772251570 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f80bc078bc0 0x7f80bc07b0d0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:43.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.763+0000 7f80e1400640 1 --2- 172.21.15.39:0/3772251570 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80dc0d0630 0x7f80dc14d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:43.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.763+0000 7f80e1400640 1 --2- 172.21.15.39:0/3772251570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80dc0cf450 0x7f80dc0b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:43.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.763+0000 7f80e1400640 1 --2- 172.21.15.39:0/3772251570 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80dc073dd0 0x7f80dc0b46a0 unknown :-1 s=CLOSED pgs=2821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:43.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.763+0000 7f80e1400640 1 -- 172.21.15.39:0/3772251570 >> 172.21.15.39:0/3772251570 conn(0x7f80dc06cfc0 msgr2=0x7f80dc06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:43.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.763+0000 7f80e1400640 1 -- 172.21.15.39:0/3772251570 shutdown_connections 2024-09-21T07:54:43.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:43.763+0000 7f80e1400640 1 -- 172.21.15.39:0/3772251570 wait complete. 2024-09-21T07:54:43.786 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":910,"stamp":"2024-09-21T07:54:41.978906+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592654,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":116,"num_write_kb":1634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":292,"acting":292,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":239,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":457080,"kb_used_data":27048,"kb_used_omap":12,"kb_used_meta":429811,"kb_avail":754517640,"statfs":{"total":773094113280,"available":772626063360,"internally_reserved":0,"allocated":27697152,"data_stored":15597486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":440127024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2286,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.715899"},"pg_stats":[{"pgid":"32.1f","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.576786+0000","last_change":"2024-09-21T07:54:39.304408+0000","last_active":"2024-09-21T07:54:40.576786+0000","last_peered":"2024-09-21T07:54:40.576786+0000","last_clean":"2024-09-21T07:54:40.576786+0000","last_became_active":"2024-09-21T07:54:39.301322+0000","last_became_peered":"2024-09-21T07:54:39.301322+0000","last_unstale":"2024-09-21T07:54:40.576786+0000","last_undegraded":"2024-09-21T07:54:40.576786+0000","last_fullsized":"2024-09-21T07:54:40.576786+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:28.727090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.576497+0000","last_change":"2024-09-21T07:54:39.318701+0000","last_active":"2024-09-21T07:54:40.576497+0000","last_peered":"2024-09-21T07:54:40.576497+0000","last_clean":"2024-09-21T07:54:40.576497+0000","last_became_active":"2024-09-21T07:54:39.318375+0000","last_became_peered":"2024-09-21T07:54:39.318375+0000","last_unstale":"2024-09-21T07:54:40.576497+0000","last_undegraded":"2024-09-21T07:54:40.576497+0000","last_fullsized":"2024-09-21T07:54:40.576497+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:58:21.776964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.577059+0000","last_change":"2024-09-21T07:54:39.308208+0000","last_active":"2024-09-21T07:54:40.577059+0000","last_peered":"2024-09-21T07:54:40.577059+0000","last_clean":"2024-09-21T07:54:40.577059+0000","last_became_active":"2024-09-21T07:54:39.307863+0000","last_became_peered":"2024-09-21T07:54:39.307863+0000","last_unstale":"2024-09-21T07:54:40.577059+0000","last_undegraded":"2024-09-21T07:54:40.577059+0000","last_fullsized":"2024-09-21T07:54:40.577059+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:58.582247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.576546+0000","last_change":"2024-09-21T07:54:39.301716+0000","last_active":"2024-09-21T07:54:40.576546+0000","last_peered":"2024-09-21T07:54:40.576546+0000","last_clean":"2024-09-21T07:54:40.576546+0000","last_became_active":"2024-09-21T07:54:39.301428+0000","last_became_peered":"2024-09-21T07:54:39.301428+0000","last_unstale":"2024-09-21T07:54:40.576546+0000","last_undegraded":"2024-09-21T07:54:40.576546+0000","last_fullsized":"2024-09-21T07:54:40.576546+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:04.990166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.608973+0000","last_change":"2024-09-21T07:54:39.298698+0000","last_active":"2024-09-21T07:54:40.608973+0000","last_peered":"2024-09-21T07:54:40.608973+0000","last_clean":"2024-09-21T07:54:40.608973+0000","last_became_active":"2024-09-21T07:54:39.296272+0000","last_became_peered":"2024-09-21T07:54:39.296272+0000","last_unstale":"2024-09-21T07:54:40.608973+0000","last_undegraded":"2024-09-21T07:54:40.608973+0000","last_fullsized":"2024-09-21T07:54:40.608973+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:13.600267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.610018+0000","last_change":"2024-09-21T07:54:39.313058+0000","last_active":"2024-09-21T07:54:40.610018+0000","last_peered":"2024-09-21T07:54:40.610018+0000","last_clean":"2024-09-21T07:54:40.610018+0000","last_became_active":"2024-09-21T07:54:39.312661+0000","last_became_peered":"2024-09-21T07:54:39.312661+0000","last_unstale":"2024-09-21T07:54:40.610018+0000","last_undegraded":"2024-09-21T07:54:40.610018+0000","last_fullsized":"2024-09-21T07:54:40.610018+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:18.895423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.609571+0000","last_change":"2024-09-21T07:54:39.301660+0000","last_active":"2024-09-21T07:54:40.609571+0000","last_peered":"2024-09-21T07:54:40.609571+0000","last_clean":"2024-09-21T07:54:40.609571+0000","last_became_active":"2024-09-21T07:54:39.301318+0000","last_became_peered":"2024-09-21T07:54:39.301318+0000","last_unstale":"2024-09-21T07:54:40.609571+0000","last_undegraded":"2024-09-21T07:54:40.609571+0000","last_fullsized":"2024-09-21T07:54:40.609571+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:06.819293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.607896+0000","last_change":"2024-09-21T07:54:39.293648+0000","last_active":"2024-09-21T07:54:40.607896+0000","last_peered":"2024-09-21T07:54:40.607896+0000","last_clean":"2024-09-21T07:54:40.607896+0000","last_became_active":"2024-09-21T07:54:39.291378+0000","last_became_peered":"2024-09-21T07:54:39.291378+0000","last_unstale":"2024-09-21T07:54:40.607896+0000","last_undegraded":"2024-09-21T07:54:40.607896+0000","last_fullsized":"2024-09-21T07:54:40.607896+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:29.745606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.576947+0000","last_change":"2024-09-21T07:54:39.310000+0000","last_active":"2024-09-21T07:54:40.576947+0000","last_peered":"2024-09-21T07:54:40.576947+0000","last_clean":"2024-09-21T07:54:40.576947+0000","last_became_active":"2024-09-21T07:54:39.307318+0000","last_became_peered":"2024-09-21T07:54:39.307318+0000","last_unstale":"2024-09-21T07:54:40.576947+0000","last_undegraded":"2024-09-21T07:54:40.576947+0000","last_fullsized":"2024-09-21T07:54:40.576947+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:02:43.076786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.576561+0000","last_change":"2024-09-21T07:54:39.309549+0000","last_active":"2024-09-21T07:54:40.576561+0000","last_peered":"2024-09-21T07:54:40.576561+0000","last_clean":"2024-09-21T07:54:40.576561+0000","last_became_active":"2024-09-21T07:54:39.309223+0000","last_became_peered":"2024-09-21T07:54:39.309223+0000","last_unstale":"2024-09-21T07:54:40.576561+0000","last_undegraded":"2024-09-21T07:54:40.576561+0000","last_fullsized":"2024-09-21T07:54:40.576561+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:17:11.002037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.576068+0000","last_change":"2024-09-21T07:54:39.307331+0000","last_active":"2024-09-21T07:54:40.576068+0000","last_peered":"2024-09-21T07:54:40.576068+0000","last_clean":"2024-09-21T07:54:40.576068+0000","last_became_active":"2024-09-21T07:54:39.306929+0000","last_became_peered":"2024-09-21T07:54:39.306929+0000","last_unstale":"2024-09-21T07:54:40.576068+0000","last_undegraded":"2024-09-21T07:54:40.576068+0000","last_fullsized":"2024-09-21T07:54:40.576068+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:01:26.853678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"0'0","reported_seq":1,"reported_epoch":262,"state":"creating+peering","last_fresh":"2024-09-21T07:54:37.265312+0000","last_change":"2024-09-21T07:54:37.265312+0000","last_active":"2024-09-21T07:54:37.257240+0000","last_peered":"2024-09-21T07:54:37.257240+0000","last_clean":"2024-09-21T07:54:37.257240+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:37.265312+0000","last_undegraded":"2024-09-21T07:54:37.265312+0000","last_fullsized":"2024-09-21T07:54:37.265312+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":389,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.273343+0000","last_change":"2024-09-21T07:35:27.879605+0000","last_active":"2024-09-21T07:54:37.273343+0000","last_peered":"2024-09-21T07:54:37.273343+0000","last_clean":"2024-09-21T07:54:37.273343+0000","last_became_active":"2024-09-21T07:35:27.879274+0000","last_became_peered":"2024-09-21T07:35:27.879274+0000","last_unstale":"2024-09-21T07:54:37.273343+0000","last_undegraded":"2024-09-21T07:54:37.273343+0000","last_fullsized":"2024-09-21T07:54:37.273343+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:24.241636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":404,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.576369+0000","last_change":"2024-09-21T07:35:29.491336+0000","last_active":"2024-09-21T07:54:40.576369+0000","last_peered":"2024-09-21T07:54:40.576369+0000","last_clean":"2024-09-21T07:54:40.576369+0000","last_became_active":"2024-09-21T07:35:26.866484+0000","last_became_peered":"2024-09-21T07:35:26.866484+0000","last_unstale":"2024-09-21T07:54:40.576369+0000","last_undegraded":"2024-09-21T07:54:40.576369+0000","last_fullsized":"2024-09-21T07:54:40.576369+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:10.449665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076010100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":575,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.272057+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:54:37.272057+0000","last_peered":"2024-09-21T07:54:37.272057+0000","last_clean":"2024-09-21T07:54:37.272057+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:54:37.272057+0000","last_undegraded":"2024-09-21T07:54:37.272057+0000","last_fullsized":"2024-09-21T07:54:37.272057+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":466,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.267080+0000","last_change":"2024-09-21T07:34:24.539248+0000","last_active":"2024-09-21T07:54:37.267080+0000","last_peered":"2024-09-21T07:54:37.267080+0000","last_clean":"2024-09-21T07:54:37.267080+0000","last_became_active":"2024-09-21T07:34:23.273465+0000","last_became_peered":"2024-09-21T07:34:23.273465+0000","last_unstale":"2024-09-21T07:54:37.267080+0000","last_undegraded":"2024-09-21T07:54:37.267080+0000","last_fullsized":"2024-09-21T07:54:37.267080+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:55.517452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060627700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.1b","version":"0'0","reported_seq":1,"reported_epoch":262,"state":"creating+peering","last_fresh":"2024-09-21T07:54:37.271350+0000","last_change":"2024-09-21T07:54:37.271350+0000","last_active":"2024-09-21T07:54:37.257240+0000","last_peered":"2024-09-21T07:54:37.257240+0000","last_clean":"2024-09-21T07:54:37.257240+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:37.271350+0000","last_undegraded":"2024-09-21T07:54:37.271350+0000","last_fullsized":"2024-09-21T07:54:37.271350+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":398,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.576701+0000","last_change":"2024-09-21T07:35:27.864492+0000","last_active":"2024-09-21T07:54:40.576701+0000","last_peered":"2024-09-21T07:54:40.576701+0000","last_clean":"2024-09-21T07:54:40.576701+0000","last_became_active":"2024-09-21T07:35:27.864176+0000","last_became_peered":"2024-09-21T07:35:27.864176+0000","last_unstale":"2024-09-21T07:54:40.576701+0000","last_undegraded":"2024-09-21T07:54:40.576701+0000","last_fullsized":"2024-09-21T07:54:40.576701+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:05.153467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":396,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.274107+0000","last_change":"2024-09-21T07:35:29.493678+0000","last_active":"2024-09-21T07:54:37.274107+0000","last_peered":"2024-09-21T07:54:37.274107+0000","last_clean":"2024-09-21T07:54:37.274107+0000","last_became_active":"2024-09-21T07:35:26.870749+0000","last_became_peered":"2024-09-21T07:35:26.870749+0000","last_unstale":"2024-09-21T07:54:37.274107+0000","last_undegraded":"2024-09-21T07:54:37.274107+0000","last_fullsized":"2024-09-21T07:54:37.274107+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:55.367877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010159570000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":448,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.610370+0000","last_change":"2024-09-21T07:34:24.597274+0000","last_active":"2024-09-21T07:54:40.610370+0000","last_peered":"2024-09-21T07:54:40.610370+0000","last_clean":"2024-09-21T07:54:40.610370+0000","last_became_active":"2024-09-21T07:34:23.279560+0000","last_became_peered":"2024-09-21T07:34:23.279560+0000","last_unstale":"2024-09-21T07:54:40.610370+0000","last_undegraded":"2024-09-21T07:54:40.610370+0000","last_fullsized":"2024-09-21T07:54:40.610370+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:15.845265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059699799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.1a","version":"0'0","reported_seq":1,"reported_epoch":262,"state":"creating+peering","last_fresh":"2024-09-21T07:54:37.268603+0000","last_change":"2024-09-21T07:54:37.268603+0000","last_active":"2024-09-21T07:54:37.257240+0000","last_peered":"2024-09-21T07:54:37.257240+0000","last_clean":"2024-09-21T07:54:37.257240+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:37.268603+0000","last_undegraded":"2024-09-21T07:54:37.268603+0000","last_fullsized":"2024-09-21T07:54:37.268603+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":392,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.270720+0000","last_change":"2024-09-21T07:35:27.874387+0000","last_active":"2024-09-21T07:54:37.270720+0000","last_peered":"2024-09-21T07:54:37.270720+0000","last_clean":"2024-09-21T07:54:37.270720+0000","last_became_active":"2024-09-21T07:35:27.873277+0000","last_became_peered":"2024-09-21T07:35:27.873277+0000","last_unstale":"2024-09-21T07:54:37.270720+0000","last_undegraded":"2024-09-21T07:54:37.270720+0000","last_fullsized":"2024-09-21T07:54:37.270720+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:04.581463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":398,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.608866+0000","last_change":"2024-09-21T07:35:29.387094+0000","last_active":"2024-09-21T07:54:40.608866+0000","last_peered":"2024-09-21T07:54:40.608866+0000","last_clean":"2024-09-21T07:54:40.608866+0000","last_became_active":"2024-09-21T07:35:26.874757+0000","last_became_peered":"2024-09-21T07:35:26.874757+0000","last_unstale":"2024-09-21T07:54:40.608866+0000","last_undegraded":"2024-09-21T07:54:40.608866+0000","last_fullsized":"2024-09-21T07:54:40.608866+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:33.542750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061065700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":449,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.273002+0000","last_change":"2024-09-21T07:34:24.259906+0000","last_active":"2024-09-21T07:54:37.273002+0000","last_peered":"2024-09-21T07:54:37.273002+0000","last_clean":"2024-09-21T07:54:37.273002+0000","last_became_active":"2024-09-21T07:34:23.635282+0000","last_became_peered":"2024-09-21T07:34:23.635282+0000","last_unstale":"2024-09-21T07:54:37.273002+0000","last_undegraded":"2024-09-21T07:54:37.273002+0000","last_fullsized":"2024-09-21T07:54:37.273002+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:00.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001527664,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":1,"reported_epoch":262,"state":"creating+peering","last_fresh":"2024-09-21T07:54:37.274971+0000","last_change":"2024-09-21T07:54:37.274971+0000","last_active":"2024-09-21T07:54:37.257240+0000","last_peered":"2024-09-21T07:54:37.257240+0000","last_clean":"2024-09-21T07:54:37.257240+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:37.274971+0000","last_undegraded":"2024-09-21T07:54:37.274971+0000","last_fullsized":"2024-09-21T07:54:37.274971+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":392,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.271829+0000","last_change":"2024-09-21T07:35:27.873645+0000","last_active":"2024-09-21T07:54:37.271829+0000","last_peered":"2024-09-21T07:54:37.271829+0000","last_clean":"2024-09-21T07:54:37.271829+0000","last_became_active":"2024-09-21T07:35:27.873245+0000","last_became_peered":"2024-09-21T07:35:27.873245+0000","last_unstale":"2024-09-21T07:54:37.271829+0000","last_undegraded":"2024-09-21T07:54:37.271829+0000","last_fullsized":"2024-09-21T07:54:37.271829+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:55.090953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":399,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.272626+0000","last_change":"2024-09-21T07:35:28.861744+0000","last_active":"2024-09-21T07:54:37.272626+0000","last_peered":"2024-09-21T07:54:37.272626+0000","last_clean":"2024-09-21T07:54:37.272626+0000","last_became_active":"2024-09-21T07:35:26.876462+0000","last_became_peered":"2024-09-21T07:35:26.876462+0000","last_unstale":"2024-09-21T07:54:37.272626+0000","last_undegraded":"2024-09-21T07:54:37.272626+0000","last_fullsized":"2024-09-21T07:54:37.272626+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:48.632568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012020920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":450,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.273114+0000","last_change":"2024-09-21T07:34:25.085454+0000","last_active":"2024-09-21T07:54:37.273114+0000","last_peered":"2024-09-21T07:54:37.273114+0000","last_clean":"2024-09-21T07:54:37.273114+0000","last_became_active":"2024-09-21T07:34:23.279205+0000","last_became_peered":"2024-09-21T07:34:23.279205+0000","last_unstale":"2024-09-21T07:54:37.273114+0000","last_undegraded":"2024-09-21T07:54:37.273114+0000","last_fullsized":"2024-09-21T07:54:37.273114+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:54.231666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095320599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":15,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.575922+0000","last_change":"2024-09-21T07:54:38.295170+0000","last_active":"2024-09-21T07:54:40.575922+0000","last_peered":"2024-09-21T07:54:40.575922+0000","last_clean":"2024-09-21T07:54:40.575922+0000","last_became_active":"2024-09-21T07:54:38.294876+0000","last_became_peered":"2024-09-21T07:54:38.294876+0000","last_unstale":"2024-09-21T07:54:40.575922+0000","last_undegraded":"2024-09-21T07:54:40.575922+0000","last_fullsized":"2024-09-21T07:54:40.575922+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:36:36.419308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":397,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.576587+0000","last_change":"2024-09-21T07:35:27.861373+0000","last_active":"2024-09-21T07:54:40.576587+0000","last_peered":"2024-09-21T07:54:40.576587+0000","last_clean":"2024-09-21T07:54:40.576587+0000","last_became_active":"2024-09-21T07:35:27.861016+0000","last_became_peered":"2024-09-21T07:35:27.861016+0000","last_unstale":"2024-09-21T07:54:40.576587+0000","last_undegraded":"2024-09-21T07:54:40.576587+0000","last_fullsized":"2024-09-21T07:54:40.576587+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:56.043763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":399,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.276141+0000","last_change":"2024-09-21T07:35:29.053677+0000","last_active":"2024-09-21T07:54:37.276141+0000","last_peered":"2024-09-21T07:54:37.276141+0000","last_clean":"2024-09-21T07:54:37.276141+0000","last_became_active":"2024-09-21T07:35:26.875496+0000","last_became_peered":"2024-09-21T07:35:26.875496+0000","last_unstale":"2024-09-21T07:54:37.276141+0000","last_undegraded":"2024-09-21T07:54:37.276141+0000","last_fullsized":"2024-09-21T07:54:37.276141+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:04.850784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065859000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":451,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.275995+0000","last_change":"2024-09-21T07:34:25.028521+0000","last_active":"2024-09-21T07:54:37.275995+0000","last_peered":"2024-09-21T07:54:37.275995+0000","last_clean":"2024-09-21T07:54:37.275995+0000","last_became_active":"2024-09-21T07:34:23.635467+0000","last_became_peered":"2024-09-21T07:34:23.635467+0000","last_unstale":"2024-09-21T07:54:37.275995+0000","last_undegraded":"2024-09-21T07:54:37.275995+0000","last_fullsized":"2024-09-21T07:54:37.275995+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:33.729123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000660558,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.1f","version":"264'4","reported_seq":19,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.576853+0000","last_change":"2024-09-21T07:54:38.293144+0000","last_active":"2024-09-21T07:54:40.576853+0000","last_peered":"2024-09-21T07:54:40.576853+0000","last_clean":"2024-09-21T07:54:40.576853+0000","last_became_active":"2024-09-21T07:54:38.292073+0000","last_became_peered":"2024-09-21T07:54:38.292073+0000","last_unstale":"2024-09-21T07:54:40.576853+0000","last_undegraded":"2024-09-21T07:54:40.576853+0000","last_fullsized":"2024-09-21T07:54:40.576853+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:46:02.586538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":391,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.273933+0000","last_change":"2024-09-21T07:35:27.871201+0000","last_active":"2024-09-21T07:54:37.273933+0000","last_peered":"2024-09-21T07:54:37.273933+0000","last_clean":"2024-09-21T07:54:37.273933+0000","last_became_active":"2024-09-21T07:35:27.870816+0000","last_became_peered":"2024-09-21T07:35:27.870816+0000","last_unstale":"2024-09-21T07:54:37.273933+0000","last_undegraded":"2024-09-21T07:54:37.273933+0000","last_fullsized":"2024-09-21T07:54:37.273933+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:04.787450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":404,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.577350+0000","last_change":"2024-09-21T07:35:29.491874+0000","last_active":"2024-09-21T07:54:40.577350+0000","last_peered":"2024-09-21T07:54:40.577350+0000","last_clean":"2024-09-21T07:54:40.577350+0000","last_became_active":"2024-09-21T07:35:26.866381+0000","last_became_peered":"2024-09-21T07:35:26.866381+0000","last_unstale":"2024-09-21T07:54:40.577350+0000","last_undegraded":"2024-09-21T07:54:40.577350+0000","last_fullsized":"2024-09-21T07:54:40.577350+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:36.404647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073235399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":460,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.576793+0000","last_change":"2024-09-21T07:34:24.260046+0000","last_active":"2024-09-21T07:54:40.576793+0000","last_peered":"2024-09-21T07:54:40.576793+0000","last_clean":"2024-09-21T07:54:40.576793+0000","last_became_active":"2024-09-21T07:34:23.273431+0000","last_became_peered":"2024-09-21T07:34:23.273431+0000","last_unstale":"2024-09-21T07:54:40.576793+0000","last_undegraded":"2024-09-21T07:54:40.576793+0000","last_fullsized":"2024-09-21T07:54:40.576793+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:18.876057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019803310000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.1d","version":"264'2","reported_seq":17,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.608681+0000","last_change":"2024-09-21T07:54:38.301636+0000","last_active":"2024-09-21T07:54:40.608681+0000","last_peered":"2024-09-21T07:54:40.608681+0000","last_clean":"2024-09-21T07:54:40.608681+0000","last_became_active":"2024-09-21T07:54:38.301255+0000","last_became_peered":"2024-09-21T07:54:38.301255+0000","last_unstale":"2024-09-21T07:54:40.608681+0000","last_undegraded":"2024-09-21T07:54:40.608681+0000","last_fullsized":"2024-09-21T07:54:40.608681+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:58:44.038214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":391,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.273942+0000","last_change":"2024-09-21T07:35:27.871736+0000","last_active":"2024-09-21T07:54:37.273942+0000","last_peered":"2024-09-21T07:54:37.273942+0000","last_clean":"2024-09-21T07:54:37.273942+0000","last_became_active":"2024-09-21T07:35:27.871232+0000","last_became_peered":"2024-09-21T07:35:27.871232+0000","last_unstale":"2024-09-21T07:54:37.273942+0000","last_undegraded":"2024-09-21T07:54:37.273942+0000","last_fullsized":"2024-09-21T07:54:37.273942+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:04.419301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":405,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.576197+0000","last_change":"2024-09-21T07:35:29.491246+0000","last_active":"2024-09-21T07:54:40.576197+0000","last_peered":"2024-09-21T07:54:40.576197+0000","last_clean":"2024-09-21T07:54:40.576197+0000","last_became_active":"2024-09-21T07:35:26.866783+0000","last_became_peered":"2024-09-21T07:35:26.866783+0000","last_unstale":"2024-09-21T07:54:40.576197+0000","last_undegraded":"2024-09-21T07:54:40.576197+0000","last_fullsized":"2024-09-21T07:54:40.576197+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:28.881914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098474099999999992,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":451,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.276339+0000","last_change":"2024-09-21T07:34:25.028640+0000","last_active":"2024-09-21T07:54:37.276339+0000","last_peered":"2024-09-21T07:54:37.276339+0000","last_clean":"2024-09-21T07:54:37.276339+0000","last_became_active":"2024-09-21T07:34:23.260950+0000","last_became_peered":"2024-09-21T07:34:23.260950+0000","last_unstale":"2024-09-21T07:54:37.276339+0000","last_undegraded":"2024-09-21T07:54:37.276339+0000","last_fullsized":"2024-09-21T07:54:37.276339+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:23:42.121398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055135099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.577301+0000","last_change":"2024-09-21T07:54:39.319086+0000","last_active":"2024-09-21T07:54:40.577301+0000","last_peered":"2024-09-21T07:54:40.577301+0000","last_clean":"2024-09-21T07:54:40.577301+0000","last_became_active":"2024-09-21T07:54:39.318793+0000","last_became_peered":"2024-09-21T07:54:39.318793+0000","last_unstale":"2024-09-21T07:54:40.577301+0000","last_undegraded":"2024-09-21T07:54:40.577301+0000","last_fullsized":"2024-09-21T07:54:40.577301+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:35:55.662137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1c","version":"0'0","reported_seq":1,"reported_epoch":262,"state":"creating+peering","last_fresh":"2024-09-21T07:54:37.270526+0000","last_change":"2024-09-21T07:54:37.270526+0000","last_active":"2024-09-21T07:54:37.257240+0000","last_peered":"2024-09-21T07:54:37.257240+0000","last_clean":"2024-09-21T07:54:37.257240+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:37.270526+0000","last_undegraded":"2024-09-21T07:54:37.270526+0000","last_fullsized":"2024-09-21T07:54:37.270526+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":394,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.274786+0000","last_change":"2024-09-21T07:35:27.872027+0000","last_active":"2024-09-21T07:54:37.274786+0000","last_peered":"2024-09-21T07:54:37.274786+0000","last_clean":"2024-09-21T07:54:37.274786+0000","last_became_active":"2024-09-21T07:35:27.870426+0000","last_became_peered":"2024-09-21T07:35:27.870426+0000","last_unstale":"2024-09-21T07:54:37.274786+0000","last_undegraded":"2024-09-21T07:54:37.274786+0000","last_fullsized":"2024-09-21T07:54:37.274786+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":405,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.576088+0000","last_change":"2024-09-21T07:35:29.491008+0000","last_active":"2024-09-21T07:54:40.576088+0000","last_peered":"2024-09-21T07:54:40.576088+0000","last_clean":"2024-09-21T07:54:40.576088+0000","last_became_active":"2024-09-21T07:35:26.866750+0000","last_became_peered":"2024-09-21T07:35:26.866750+0000","last_unstale":"2024-09-21T07:54:40.576088+0000","last_undegraded":"2024-09-21T07:54:40.576088+0000","last_fullsized":"2024-09-21T07:54:40.576088+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:25.490406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083529700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":450,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.273638+0000","last_change":"2024-09-21T07:34:24.260665+0000","last_active":"2024-09-21T07:54:37.273638+0000","last_peered":"2024-09-21T07:54:37.273638+0000","last_clean":"2024-09-21T07:54:37.273638+0000","last_became_active":"2024-09-21T07:34:23.635909+0000","last_became_peered":"2024-09-21T07:34:23.635909+0000","last_unstale":"2024-09-21T07:54:37.273638+0000","last_undegraded":"2024-09-21T07:54:37.273638+0000","last_fullsized":"2024-09-21T07:54:37.273638+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:44.536885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.13","version":"0'0","reported_seq":1,"reported_epoch":262,"state":"creating+peering","last_fresh":"2024-09-21T07:54:37.270684+0000","last_change":"2024-09-21T07:54:37.270684+0000","last_active":"2024-09-21T07:54:37.257240+0000","last_peered":"2024-09-21T07:54:37.257240+0000","last_clean":"2024-09-21T07:54:37.257240+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:37.270684+0000","last_undegraded":"2024-09-21T07:54:37.270684+0000","last_fullsized":"2024-09-21T07:54:37.270684+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":392,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.608119+0000","last_change":"2024-09-21T07:35:27.874772+0000","last_active":"2024-09-21T07:54:40.608119+0000","last_peered":"2024-09-21T07:54:40.608119+0000","last_clean":"2024-09-21T07:54:40.608119+0000","last_became_active":"2024-09-21T07:35:27.874434+0000","last_became_peered":"2024-09-21T07:35:27.874434+0000","last_unstale":"2024-09-21T07:54:40.608119+0000","last_undegraded":"2024-09-21T07:54:40.608119+0000","last_fullsized":"2024-09-21T07:54:40.608119+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:31.998755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":413,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.266958+0000","last_change":"2024-09-21T07:35:28.859153+0000","last_active":"2024-09-21T07:54:37.266958+0000","last_peered":"2024-09-21T07:54:37.266958+0000","last_clean":"2024-09-21T07:54:37.266958+0000","last_became_active":"2024-09-21T07:35:26.874428+0000","last_became_peered":"2024-09-21T07:35:26.874428+0000","last_unstale":"2024-09-21T07:54:37.266958+0000","last_undegraded":"2024-09-21T07:54:37.266958+0000","last_fullsized":"2024-09-21T07:54:37.266958+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:04.487621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048557299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":466,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.266677+0000","last_change":"2024-09-21T07:34:24.538845+0000","last_active":"2024-09-21T07:54:37.266677+0000","last_peered":"2024-09-21T07:54:37.266677+0000","last_clean":"2024-09-21T07:54:37.266677+0000","last_became_active":"2024-09-21T07:34:23.636372+0000","last_became_peered":"2024-09-21T07:34:23.636372+0000","last_unstale":"2024-09-21T07:54:37.266677+0000","last_undegraded":"2024-09-21T07:54:37.266677+0000","last_fullsized":"2024-09-21T07:54:37.266677+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:29.393813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841451,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":1,"reported_epoch":262,"state":"creating+peering","last_fresh":"2024-09-21T07:54:37.271670+0000","last_change":"2024-09-21T07:54:37.271670+0000","last_active":"2024-09-21T07:54:37.257240+0000","last_peered":"2024-09-21T07:54:37.257240+0000","last_clean":"2024-09-21T07:54:37.257240+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:37.271670+0000","last_undegraded":"2024-09-21T07:54:37.271670+0000","last_fullsized":"2024-09-21T07:54:37.271670+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":395,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.273442+0000","last_change":"2024-09-21T07:35:27.877801+0000","last_active":"2024-09-21T07:54:37.273442+0000","last_peered":"2024-09-21T07:54:37.273442+0000","last_clean":"2024-09-21T07:54:37.273442+0000","last_became_active":"2024-09-21T07:35:27.877464+0000","last_became_peered":"2024-09-21T07:35:27.877464+0000","last_unstale":"2024-09-21T07:54:37.273442+0000","last_undegraded":"2024-09-21T07:54:37.273442+0000","last_fullsized":"2024-09-21T07:54:37.273442+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:08.814929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":398,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.274869+0000","last_change":"2024-09-21T07:35:28.863732+0000","last_active":"2024-09-21T07:54:37.274869+0000","last_peered":"2024-09-21T07:54:37.274869+0000","last_clean":"2024-09-21T07:54:37.274869+0000","last_became_active":"2024-09-21T07:35:26.877392+0000","last_became_peered":"2024-09-21T07:35:26.877392+0000","last_unstale":"2024-09-21T07:54:37.274869+0000","last_undegraded":"2024-09-21T07:54:37.274869+0000","last_fullsized":"2024-09-21T07:54:37.274869+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071425700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":459,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.577376+0000","last_change":"2024-09-21T07:34:24.259686+0000","last_active":"2024-09-21T07:54:40.577376+0000","last_peered":"2024-09-21T07:54:40.577376+0000","last_clean":"2024-09-21T07:54:40.577376+0000","last_became_active":"2024-09-21T07:34:23.274405+0000","last_became_peered":"2024-09-21T07:34:23.274405+0000","last_unstale":"2024-09-21T07:54:40.577376+0000","last_undegraded":"2024-09-21T07:54:40.577376+0000","last_fullsized":"2024-09-21T07:54:40.577376+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:26.304693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022009920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":1,"reported_epoch":262,"state":"creating+peering","last_fresh":"2024-09-21T07:54:37.270571+0000","last_change":"2024-09-21T07:54:37.270571+0000","last_active":"2024-09-21T07:54:37.257240+0000","last_peered":"2024-09-21T07:54:37.257240+0000","last_clean":"2024-09-21T07:54:37.257240+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:37.270571+0000","last_undegraded":"2024-09-21T07:54:37.270571+0000","last_fullsized":"2024-09-21T07:54:37.270571+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":392,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.610246+0000","last_change":"2024-09-21T07:35:27.867336+0000","last_active":"2024-09-21T07:54:40.610246+0000","last_peered":"2024-09-21T07:54:40.610246+0000","last_clean":"2024-09-21T07:54:40.610246+0000","last_became_active":"2024-09-21T07:35:27.867015+0000","last_became_peered":"2024-09-21T07:35:27.867015+0000","last_unstale":"2024-09-21T07:54:40.610246+0000","last_undegraded":"2024-09-21T07:54:40.610246+0000","last_fullsized":"2024-09-21T07:54:40.610246+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:59.214139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":397,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.273524+0000","last_change":"2024-09-21T07:35:29.493137+0000","last_active":"2024-09-21T07:54:37.273524+0000","last_peered":"2024-09-21T07:54:37.273524+0000","last_clean":"2024-09-21T07:54:37.273524+0000","last_became_active":"2024-09-21T07:35:26.872230+0000","last_became_peered":"2024-09-21T07:35:26.872230+0000","last_unstale":"2024-09-21T07:54:37.273524+0000","last_undegraded":"2024-09-21T07:54:37.273524+0000","last_fullsized":"2024-09-21T07:54:37.273524+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:24.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001197449,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":450,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.272368+0000","last_change":"2024-09-21T07:34:24.258754+0000","last_active":"2024-09-21T07:54:37.272368+0000","last_peered":"2024-09-21T07:54:37.272368+0000","last_clean":"2024-09-21T07:54:37.272368+0000","last_became_active":"2024-09-21T07:34:23.272657+0000","last_became_peered":"2024-09-21T07:34:23.272657+0000","last_unstale":"2024-09-21T07:54:37.272368+0000","last_undegraded":"2024-09-21T07:54:37.272368+0000","last_fullsized":"2024-09-21T07:54:37.272368+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:05.477099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067143200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.10","version":"0'0","reported_seq":1,"reported_epoch":262,"state":"creating+peering","last_fresh":"2024-09-21T07:54:37.269611+0000","last_change":"2024-09-21T07:54:37.269611+0000","last_active":"2024-09-21T07:54:37.257240+0000","last_peered":"2024-09-21T07:54:37.257240+0000","last_clean":"2024-09-21T07:54:37.257240+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:37.269611+0000","last_undegraded":"2024-09-21T07:54:37.269611+0000","last_fullsized":"2024-09-21T07:54:37.269611+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":392,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.273328+0000","last_change":"2024-09-21T07:35:27.876021+0000","last_active":"2024-09-21T07:54:37.273328+0000","last_peered":"2024-09-21T07:54:37.273328+0000","last_clean":"2024-09-21T07:54:37.273328+0000","last_became_active":"2024-09-21T07:35:27.875724+0000","last_became_peered":"2024-09-21T07:35:27.875724+0000","last_unstale":"2024-09-21T07:54:37.273328+0000","last_undegraded":"2024-09-21T07:54:37.273328+0000","last_fullsized":"2024-09-21T07:54:37.273328+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:03.022812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":397,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.273193+0000","last_change":"2024-09-21T07:35:29.492570+0000","last_active":"2024-09-21T07:54:37.273193+0000","last_peered":"2024-09-21T07:54:37.273193+0000","last_clean":"2024-09-21T07:54:37.273193+0000","last_became_active":"2024-09-21T07:35:26.874316+0000","last_became_peered":"2024-09-21T07:35:26.874316+0000","last_unstale":"2024-09-21T07:54:37.273193+0000","last_undegraded":"2024-09-21T07:54:37.273193+0000","last_fullsized":"2024-09-21T07:54:37.273193+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:16.512323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011990760000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":450,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.273235+0000","last_change":"2024-09-21T07:34:25.085232+0000","last_active":"2024-09-21T07:54:37.273235+0000","last_peered":"2024-09-21T07:54:37.273235+0000","last_clean":"2024-09-21T07:54:37.273235+0000","last_became_active":"2024-09-21T07:34:23.277807+0000","last_became_peered":"2024-09-21T07:34:23.277807+0000","last_unstale":"2024-09-21T07:54:37.273235+0000","last_undegraded":"2024-09-21T07:54:37.273235+0000","last_fullsized":"2024-09-21T07:54:37.273235+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:36.362919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086619099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":1,"reported_epoch":262,"state":"creating+peering","last_fresh":"2024-09-21T07:54:37.270550+0000","last_change":"2024-09-21T07:54:37.270550+0000","last_active":"2024-09-21T07:54:37.257240+0000","last_peered":"2024-09-21T07:54:37.257240+0000","last_clean":"2024-09-21T07:54:37.257240+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:37.270550+0000","last_undegraded":"2024-09-21T07:54:37.270550+0000","last_fullsized":"2024-09-21T07:54:37.270550+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":395,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.274354+0000","last_change":"2024-09-21T07:35:27.872798+0000","last_active":"2024-09-21T07:54:37.274354+0000","last_peered":"2024-09-21T07:54:37.274354+0000","last_clean":"2024-09-21T07:54:37.274354+0000","last_became_active":"2024-09-21T07:35:27.872483+0000","last_became_peered":"2024-09-21T07:35:27.872483+0000","last_unstale":"2024-09-21T07:54:37.274354+0000","last_undegraded":"2024-09-21T07:54:37.274354+0000","last_fullsized":"2024-09-21T07:54:37.274354+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:11.933469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":402,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.274100+0000","last_change":"2024-09-21T07:35:29.206723+0000","last_active":"2024-09-21T07:54:37.274100+0000","last_peered":"2024-09-21T07:54:37.274100+0000","last_clean":"2024-09-21T07:54:37.274100+0000","last_became_active":"2024-09-21T07:35:26.872377+0000","last_became_peered":"2024-09-21T07:35:26.872377+0000","last_unstale":"2024-09-21T07:54:37.274100+0000","last_undegraded":"2024-09-21T07:54:37.274100+0000","last_fullsized":"2024-09-21T07:54:37.274100+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:29.086862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083077200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":448,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.609137+0000","last_change":"2024-09-21T07:34:24.597672+0000","last_active":"2024-09-21T07:54:40.609137+0000","last_peered":"2024-09-21T07:54:40.609137+0000","last_clean":"2024-09-21T07:54:40.609137+0000","last_became_active":"2024-09-21T07:34:23.267770+0000","last_became_peered":"2024-09-21T07:34:23.267770+0000","last_unstale":"2024-09-21T07:54:40.609137+0000","last_undegraded":"2024-09-21T07:54:40.609137+0000","last_fullsized":"2024-09-21T07:54:40.609137+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:51.629476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086378500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.608080+0000","last_change":"2024-09-21T07:54:39.298211+0000","last_active":"2024-09-21T07:54:40.608080+0000","last_peered":"2024-09-21T07:54:40.608080+0000","last_clean":"2024-09-21T07:54:40.608080+0000","last_became_active":"2024-09-21T07:54:39.297877+0000","last_became_peered":"2024-09-21T07:54:39.297877+0000","last_unstale":"2024-09-21T07:54:40.608080+0000","last_undegraded":"2024-09-21T07:54:40.608080+0000","last_fullsized":"2024-09-21T07:54:40.608080+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:13.790472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":1,"reported_epoch":262,"state":"creating+peering","last_fresh":"2024-09-21T07:54:37.274644+0000","last_change":"2024-09-21T07:54:37.274644+0000","last_active":"2024-09-21T07:54:37.257240+0000","last_peered":"2024-09-21T07:54:37.257240+0000","last_clean":"2024-09-21T07:54:37.257240+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:37.274644+0000","last_undegraded":"2024-09-21T07:54:37.274644+0000","last_fullsized":"2024-09-21T07:54:37.274644+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":395,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.274469+0000","last_change":"2024-09-21T07:35:27.870121+0000","last_active":"2024-09-21T07:54:37.274469+0000","last_peered":"2024-09-21T07:54:37.274469+0000","last_clean":"2024-09-21T07:54:37.274469+0000","last_became_active":"2024-09-21T07:35:27.869811+0000","last_became_peered":"2024-09-21T07:35:27.869811+0000","last_unstale":"2024-09-21T07:54:37.274469+0000","last_undegraded":"2024-09-21T07:54:37.274469+0000","last_fullsized":"2024-09-21T07:54:37.274469+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:31.778461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":400,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.271732+0000","last_change":"2024-09-21T07:35:28.860699+0000","last_active":"2024-09-21T07:54:37.271732+0000","last_peered":"2024-09-21T07:54:37.271732+0000","last_clean":"2024-09-21T07:54:37.271732+0000","last_became_active":"2024-09-21T07:35:26.876779+0000","last_became_peered":"2024-09-21T07:35:26.876779+0000","last_unstale":"2024-09-21T07:54:37.271732+0000","last_undegraded":"2024-09-21T07:54:37.271732+0000","last_fullsized":"2024-09-21T07:54:37.271732+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:28.286403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000913188,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":452,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.272836+0000","last_change":"2024-09-21T07:34:24.260181+0000","last_active":"2024-09-21T07:54:37.272836+0000","last_peered":"2024-09-21T07:54:37.272836+0000","last_clean":"2024-09-21T07:54:37.272836+0000","last_became_active":"2024-09-21T07:34:23.275580+0000","last_became_peered":"2024-09-21T07:34:23.275580+0000","last_unstale":"2024-09-21T07:54:37.272836+0000","last_undegraded":"2024-09-21T07:54:37.272836+0000","last_fullsized":"2024-09-21T07:54:37.272836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:49.857622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014706980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.15","version":"0'0","reported_seq":1,"reported_epoch":262,"state":"creating+peering","last_fresh":"2024-09-21T07:54:37.270609+0000","last_change":"2024-09-21T07:54:37.270609+0000","last_active":"2024-09-21T07:54:37.257240+0000","last_peered":"2024-09-21T07:54:37.257240+0000","last_clean":"2024-09-21T07:54:37.257240+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:37.270609+0000","last_undegraded":"2024-09-21T07:54:37.270609+0000","last_fullsized":"2024-09-21T07:54:37.270609+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":392,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.276656+0000","last_change":"2024-09-21T07:35:27.875393+0000","last_active":"2024-09-21T07:54:37.276656+0000","last_peered":"2024-09-21T07:54:37.276656+0000","last_clean":"2024-09-21T07:54:37.276656+0000","last_became_active":"2024-09-21T07:35:27.874671+0000","last_became_peered":"2024-09-21T07:35:27.874671+0000","last_unstale":"2024-09-21T07:54:37.276656+0000","last_undegraded":"2024-09-21T07:54:37.276656+0000","last_fullsized":"2024-09-21T07:54:37.276656+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":402,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.273622+0000","last_change":"2024-09-21T07:35:29.206296+0000","last_active":"2024-09-21T07:54:37.273622+0000","last_peered":"2024-09-21T07:54:37.273622+0000","last_clean":"2024-09-21T07:54:37.273622+0000","last_became_active":"2024-09-21T07:35:26.874575+0000","last_became_peered":"2024-09-21T07:35:26.874575+0000","last_unstale":"2024-09-21T07:54:37.273622+0000","last_undegraded":"2024-09-21T07:54:37.273622+0000","last_fullsized":"2024-09-21T07:54:37.273622+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:22.771154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061011599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":459,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.576852+0000","last_change":"2024-09-21T07:34:24.259112+0000","last_active":"2024-09-21T07:54:40.576852+0000","last_peered":"2024-09-21T07:54:40.576852+0000","last_clean":"2024-09-21T07:54:40.576852+0000","last_became_active":"2024-09-21T07:34:23.635444+0000","last_became_peered":"2024-09-21T07:34:23.635444+0000","last_unstale":"2024-09-21T07:54:40.576852+0000","last_undegraded":"2024-09-21T07:54:40.576852+0000","last_fullsized":"2024-09-21T07:54:40.576852+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:46.255385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001952636,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.14","version":"264'6","reported_seq":21,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.576724+0000","last_change":"2024-09-21T07:54:38.301495+0000","last_active":"2024-09-21T07:54:40.576724+0000","last_peered":"2024-09-21T07:54:40.576724+0000","last_clean":"2024-09-21T07:54:40.576724+0000","last_became_active":"2024-09-21T07:54:38.300865+0000","last_became_peered":"2024-09-21T07:54:38.300865+0000","last_unstale":"2024-09-21T07:54:40.576724+0000","last_undegraded":"2024-09-21T07:54:40.576724+0000","last_fullsized":"2024-09-21T07:54:40.576724+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:26:07.479418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":392,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.277186+0000","last_change":"2024-09-21T07:35:27.878356+0000","last_active":"2024-09-21T07:54:37.277186+0000","last_peered":"2024-09-21T07:54:37.277186+0000","last_clean":"2024-09-21T07:54:37.277186+0000","last_became_active":"2024-09-21T07:35:27.877982+0000","last_became_peered":"2024-09-21T07:35:27.877982+0000","last_unstale":"2024-09-21T07:54:37.277186+0000","last_undegraded":"2024-09-21T07:54:37.277186+0000","last_fullsized":"2024-09-21T07:54:37.277186+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":400,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.275899+0000","last_change":"2024-09-21T07:35:29.053131+0000","last_active":"2024-09-21T07:54:37.275899+0000","last_peered":"2024-09-21T07:54:37.275899+0000","last_clean":"2024-09-21T07:54:37.275899+0000","last_became_active":"2024-09-21T07:35:26.870516+0000","last_became_peered":"2024-09-21T07:35:26.870516+0000","last_unstale":"2024-09-21T07:54:37.275899+0000","last_undegraded":"2024-09-21T07:54:37.275899+0000","last_fullsized":"2024-09-21T07:54:37.275899+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:10.924491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070705699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":466,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.267495+0000","last_change":"2024-09-21T07:34:24.539152+0000","last_active":"2024-09-21T07:54:37.267495+0000","last_peered":"2024-09-21T07:54:37.267495+0000","last_clean":"2024-09-21T07:54:37.267495+0000","last_became_active":"2024-09-21T07:34:23.259573+0000","last_became_peered":"2024-09-21T07:34:23.259573+0000","last_unstale":"2024-09-21T07:54:37.267495+0000","last_undegraded":"2024-09-21T07:54:37.267495+0000","last_fullsized":"2024-09-21T07:54:37.267495+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:26.986900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058108400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.b","version":"0'0","reported_seq":1,"reported_epoch":262,"state":"creating+peering","last_fresh":"2024-09-21T07:54:37.275230+0000","last_change":"2024-09-21T07:54:37.275230+0000","last_active":"2024-09-21T07:54:37.257240+0000","last_peered":"2024-09-21T07:54:37.257240+0000","last_clean":"2024-09-21T07:54:37.257240+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:37.275230+0000","last_undegraded":"2024-09-21T07:54:37.275230+0000","last_fullsized":"2024-09-21T07:54:37.275230+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":391,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.275860+0000","last_change":"2024-09-21T07:35:27.873726+0000","last_active":"2024-09-21T07:54:37.275860+0000","last_peered":"2024-09-21T07:54:37.275860+0000","last_clean":"2024-09-21T07:54:37.275860+0000","last_became_active":"2024-09-21T07:35:27.871421+0000","last_became_peered":"2024-09-21T07:35:27.871421+0000","last_unstale":"2024-09-21T07:54:37.275860+0000","last_undegraded":"2024-09-21T07:54:37.275860+0000","last_fullsized":"2024-09-21T07:54:37.275860+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:00.168915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":400,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.272071+0000","last_change":"2024-09-21T07:35:28.861032+0000","last_active":"2024-09-21T07:54:37.272071+0000","last_peered":"2024-09-21T07:54:37.272071+0000","last_clean":"2024-09-21T07:54:37.272071+0000","last_became_active":"2024-09-21T07:35:26.873149+0000","last_became_peered":"2024-09-21T07:35:26.873149+0000","last_unstale":"2024-09-21T07:54:37.272071+0000","last_undegraded":"2024-09-21T07:54:37.272071+0000","last_fullsized":"2024-09-21T07:54:37.272071+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:59.435981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011988369999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":449,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.272502+0000","last_change":"2024-09-21T07:34:24.259902+0000","last_active":"2024-09-21T07:54:37.272502+0000","last_peered":"2024-09-21T07:54:37.272502+0000","last_clean":"2024-09-21T07:54:37.272502+0000","last_became_active":"2024-09-21T07:34:23.271902+0000","last_became_peered":"2024-09-21T07:34:23.271902+0000","last_unstale":"2024-09-21T07:54:37.272502+0000","last_undegraded":"2024-09-21T07:54:37.272502+0000","last_fullsized":"2024-09-21T07:54:37.272502+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:59.439525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001686358,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":15,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.608498+0000","last_change":"2024-09-21T07:54:38.297862+0000","last_active":"2024-09-21T07:54:40.608498+0000","last_peered":"2024-09-21T07:54:40.608498+0000","last_clean":"2024-09-21T07:54:40.608498+0000","last_became_active":"2024-09-21T07:54:38.297508+0000","last_became_peered":"2024-09-21T07:54:38.297508+0000","last_unstale":"2024-09-21T07:54:40.608498+0000","last_undegraded":"2024-09-21T07:54:40.608498+0000","last_fullsized":"2024-09-21T07:54:40.608498+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:34.200002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":390,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.272890+0000","last_change":"2024-09-21T07:35:27.872637+0000","last_active":"2024-09-21T07:54:37.272890+0000","last_peered":"2024-09-21T07:54:37.272890+0000","last_clean":"2024-09-21T07:54:37.272890+0000","last_became_active":"2024-09-21T07:35:27.872149+0000","last_became_peered":"2024-09-21T07:35:27.872149+0000","last_unstale":"2024-09-21T07:54:37.272890+0000","last_undegraded":"2024-09-21T07:54:37.272890+0000","last_fullsized":"2024-09-21T07:54:37.272890+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:27.697872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":397,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.273327+0000","last_change":"2024-09-21T07:35:29.493362+0000","last_active":"2024-09-21T07:54:37.273327+0000","last_peered":"2024-09-21T07:54:37.273327+0000","last_clean":"2024-09-21T07:54:37.273327+0000","last_became_active":"2024-09-21T07:35:26.873018+0000","last_became_peered":"2024-09-21T07:35:26.873018+0000","last_unstale":"2024-09-21T07:54:37.273327+0000","last_undegraded":"2024-09-21T07:54:37.273327+0000","last_fullsized":"2024-09-21T07:54:37.273327+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:02.460557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012496510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":458,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.577284+0000","last_change":"2024-09-21T07:34:24.259662+0000","last_active":"2024-09-21T07:54:40.577284+0000","last_peered":"2024-09-21T07:54:40.577284+0000","last_clean":"2024-09-21T07:54:40.577284+0000","last_became_active":"2024-09-21T07:34:23.278812+0000","last_became_peered":"2024-09-21T07:34:23.278812+0000","last_unstale":"2024-09-21T07:54:40.577284+0000","last_undegraded":"2024-09-21T07:54:40.577284+0000","last_fullsized":"2024-09-21T07:54:40.577284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:49.829974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356463,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.608601+0000","last_change":"2024-09-21T07:54:39.304798+0000","last_active":"2024-09-21T07:54:40.608601+0000","last_peered":"2024-09-21T07:54:40.608601+0000","last_clean":"2024-09-21T07:54:40.608601+0000","last_became_active":"2024-09-21T07:54:39.301715+0000","last_became_peered":"2024-09-21T07:54:39.301715+0000","last_unstale":"2024-09-21T07:54:40.608601+0000","last_undegraded":"2024-09-21T07:54:40.608601+0000","last_fullsized":"2024-09-21T07:54:40.608601+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:52:10.733255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":15,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.608228+0000","last_change":"2024-09-21T07:54:38.294367+0000","last_active":"2024-09-21T07:54:40.608228+0000","last_peered":"2024-09-21T07:54:40.608228+0000","last_clean":"2024-09-21T07:54:40.608228+0000","last_became_active":"2024-09-21T07:54:38.293370+0000","last_became_peered":"2024-09-21T07:54:38.293370+0000","last_unstale":"2024-09-21T07:54:40.608228+0000","last_undegraded":"2024-09-21T07:54:40.608228+0000","last_fullsized":"2024-09-21T07:54:40.608228+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:37.732712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":390,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.610243+0000","last_change":"2024-09-21T07:35:27.868448+0000","last_active":"2024-09-21T07:54:40.610243+0000","last_peered":"2024-09-21T07:54:40.610243+0000","last_clean":"2024-09-21T07:54:40.610243+0000","last_became_active":"2024-09-21T07:35:27.868105+0000","last_became_peered":"2024-09-21T07:35:27.868105+0000","last_unstale":"2024-09-21T07:54:40.610243+0000","last_undegraded":"2024-09-21T07:54:40.610243+0000","last_fullsized":"2024-09-21T07:54:40.610243+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:41.468702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":395,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.273218+0000","last_change":"2024-09-21T07:35:29.492974+0000","last_active":"2024-09-21T07:54:37.273218+0000","last_peered":"2024-09-21T07:54:37.273218+0000","last_clean":"2024-09-21T07:54:37.273218+0000","last_became_active":"2024-09-21T07:35:26.872165+0000","last_became_peered":"2024-09-21T07:35:26.872165+0000","last_unstale":"2024-09-21T07:54:37.273218+0000","last_undegraded":"2024-09-21T07:54:37.273218+0000","last_fullsized":"2024-09-21T07:54:37.273218+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:59.912520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013795179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":449,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.273059+0000","last_change":"2024-09-21T07:34:25.084861+0000","last_active":"2024-09-21T07:54:37.273059+0000","last_peered":"2024-09-21T07:54:37.273059+0000","last_clean":"2024-09-21T07:54:37.273059+0000","last_became_active":"2024-09-21T07:34:23.277708+0000","last_became_peered":"2024-09-21T07:34:23.277708+0000","last_unstale":"2024-09-21T07:54:37.273059+0000","last_undegraded":"2024-09-21T07:54:37.273059+0000","last_fullsized":"2024-09-21T07:54:37.273059+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:40.816488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000597566,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.609775+0000","last_change":"2024-09-21T07:54:39.309318+0000","last_active":"2024-09-21T07:54:40.609775+0000","last_peered":"2024-09-21T07:54:40.609775+0000","last_clean":"2024-09-21T07:54:40.609775+0000","last_became_active":"2024-09-21T07:54:39.308999+0000","last_became_peered":"2024-09-21T07:54:39.308999+0000","last_unstale":"2024-09-21T07:54:40.609775+0000","last_undegraded":"2024-09-21T07:54:40.609775+0000","last_fullsized":"2024-09-21T07:54:40.609775+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:10.506431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":1,"reported_epoch":262,"state":"creating+peering","last_fresh":"2024-09-21T07:54:37.268333+0000","last_change":"2024-09-21T07:54:37.268333+0000","last_active":"2024-09-21T07:54:37.257240+0000","last_peered":"2024-09-21T07:54:37.257240+0000","last_clean":"2024-09-21T07:54:37.257240+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:37.268333+0000","last_undegraded":"2024-09-21T07:54:37.268333+0000","last_fullsized":"2024-09-21T07:54:37.268333+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":390,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.609344+0000","last_change":"2024-09-21T07:35:27.874640+0000","last_active":"2024-09-21T07:54:40.609344+0000","last_peered":"2024-09-21T07:54:40.609344+0000","last_clean":"2024-09-21T07:54:40.609344+0000","last_became_active":"2024-09-21T07:35:27.874265+0000","last_became_peered":"2024-09-21T07:35:27.874265+0000","last_unstale":"2024-09-21T07:54:40.609344+0000","last_undegraded":"2024-09-21T07:54:40.609344+0000","last_fullsized":"2024-09-21T07:54:40.609344+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:45.754654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":400,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.274460+0000","last_change":"2024-09-21T07:35:29.207392+0000","last_active":"2024-09-21T07:54:37.274460+0000","last_peered":"2024-09-21T07:54:37.274460+0000","last_clean":"2024-09-21T07:54:37.274460+0000","last_became_active":"2024-09-21T07:35:26.870917+0000","last_became_peered":"2024-09-21T07:35:26.870917+0000","last_unstale":"2024-09-21T07:54:37.274460+0000","last_undegraded":"2024-09-21T07:54:37.274460+0000","last_fullsized":"2024-09-21T07:54:37.274460+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:59.159937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059198499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":459,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.576318+0000","last_change":"2024-09-21T07:34:24.258709+0000","last_active":"2024-09-21T07:54:40.576318+0000","last_peered":"2024-09-21T07:54:40.576318+0000","last_clean":"2024-09-21T07:54:40.576318+0000","last_became_active":"2024-09-21T07:34:23.279042+0000","last_became_peered":"2024-09-21T07:34:23.279042+0000","last_unstale":"2024-09-21T07:54:40.576318+0000","last_undegraded":"2024-09-21T07:54:40.576318+0000","last_fullsized":"2024-09-21T07:54:40.576318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:01.762117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019282100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.f","version":"264'1","reported_seq":16,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.576973+0000","last_change":"2024-09-21T07:54:38.299153+0000","last_active":"2024-09-21T07:54:40.576973+0000","last_peered":"2024-09-21T07:54:40.576973+0000","last_clean":"2024-09-21T07:54:40.576973+0000","last_became_active":"2024-09-21T07:54:38.297607+0000","last_became_peered":"2024-09-21T07:54:38.297607+0000","last_unstale":"2024-09-21T07:54:40.576973+0000","last_undegraded":"2024-09-21T07:54:40.576973+0000","last_fullsized":"2024-09-21T07:54:40.576973+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:04.792467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":390,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.273626+0000","last_change":"2024-09-21T07:35:27.870260+0000","last_active":"2024-09-21T07:54:37.273626+0000","last_peered":"2024-09-21T07:54:37.273626+0000","last_clean":"2024-09-21T07:54:37.273626+0000","last_became_active":"2024-09-21T07:35:27.869448+0000","last_became_peered":"2024-09-21T07:35:27.869448+0000","last_unstale":"2024-09-21T07:54:37.273626+0000","last_undegraded":"2024-09-21T07:54:37.273626+0000","last_fullsized":"2024-09-21T07:54:37.273626+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:16.946790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":397,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.609233+0000","last_change":"2024-09-21T07:35:29.386367+0000","last_active":"2024-09-21T07:54:40.609233+0000","last_peered":"2024-09-21T07:54:40.609233+0000","last_clean":"2024-09-21T07:54:40.609233+0000","last_became_active":"2024-09-21T07:35:26.879238+0000","last_became_peered":"2024-09-21T07:35:26.879238+0000","last_unstale":"2024-09-21T07:54:40.609233+0000","last_undegraded":"2024-09-21T07:54:40.609233+0000","last_fullsized":"2024-09-21T07:54:40.609233+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:16.528260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092223100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":449,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.275105+0000","last_change":"2024-09-21T07:34:25.085635+0000","last_active":"2024-09-21T07:54:37.275105+0000","last_peered":"2024-09-21T07:54:37.275105+0000","last_clean":"2024-09-21T07:54:37.275105+0000","last_became_active":"2024-09-21T07:34:23.273566+0000","last_became_peered":"2024-09-21T07:34:23.273566+0000","last_unstale":"2024-09-21T07:54:37.275105+0000","last_undegraded":"2024-09-21T07:54:37.275105+0000","last_fullsized":"2024-09-21T07:54:37.275105+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:14.142272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079897300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.e","version":"0'0","reported_seq":1,"reported_epoch":262,"state":"creating+peering","last_fresh":"2024-09-21T07:54:37.269189+0000","last_change":"2024-09-21T07:54:37.269189+0000","last_active":"2024-09-21T07:54:37.257240+0000","last_peered":"2024-09-21T07:54:37.257240+0000","last_clean":"2024-09-21T07:54:37.257240+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:37.269189+0000","last_undegraded":"2024-09-21T07:54:37.269189+0000","last_fullsized":"2024-09-21T07:54:37.269189+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":390,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.273961+0000","last_change":"2024-09-21T07:35:27.878728+0000","last_active":"2024-09-21T07:54:37.273961+0000","last_peered":"2024-09-21T07:54:37.273961+0000","last_clean":"2024-09-21T07:54:37.273961+0000","last_became_active":"2024-09-21T07:35:27.878335+0000","last_became_peered":"2024-09-21T07:35:27.878335+0000","last_unstale":"2024-09-21T07:54:37.273961+0000","last_undegraded":"2024-09-21T07:54:37.273961+0000","last_fullsized":"2024-09-21T07:54:37.273961+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:25:37.731771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":412,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.266449+0000","last_change":"2024-09-21T07:35:28.858418+0000","last_active":"2024-09-21T07:54:37.266449+0000","last_peered":"2024-09-21T07:54:37.266449+0000","last_clean":"2024-09-21T07:54:37.266449+0000","last_became_active":"2024-09-21T07:35:26.876944+0000","last_became_peered":"2024-09-21T07:35:26.876944+0000","last_unstale":"2024-09-21T07:54:37.266449+0000","last_undegraded":"2024-09-21T07:54:37.266449+0000","last_fullsized":"2024-09-21T07:54:37.266449+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:04.514304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090862799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":447,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.610090+0000","last_change":"2024-09-21T07:34:24.597926+0000","last_active":"2024-09-21T07:54:40.610090+0000","last_peered":"2024-09-21T07:54:40.610090+0000","last_clean":"2024-09-21T07:54:40.610090+0000","last_became_active":"2024-09-21T07:34:23.268926+0000","last_became_peered":"2024-09-21T07:34:23.268926+0000","last_unstale":"2024-09-21T07:54:40.610090+0000","last_undegraded":"2024-09-21T07:54:40.610090+0000","last_fullsized":"2024-09-21T07:54:40.610090+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:46.190752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101851,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.d","version":"264'1","reported_seq":16,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.608728+0000","last_change":"2024-09-21T07:54:38.300028+0000","last_active":"2024-09-21T07:54:40.608728+0000","last_peered":"2024-09-21T07:54:40.608728+0000","last_clean":"2024-09-21T07:54:40.608728+0000","last_became_active":"2024-09-21T07:54:38.299574+0000","last_became_peered":"2024-09-21T07:54:38.299574+0000","last_unstale":"2024-09-21T07:54:40.608728+0000","last_undegraded":"2024-09-21T07:54:40.608728+0000","last_fullsized":"2024-09-21T07:54:40.608728+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:56.870183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":392,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.609905+0000","last_change":"2024-09-21T07:35:27.877426+0000","last_active":"2024-09-21T07:54:40.609905+0000","last_peered":"2024-09-21T07:54:40.609905+0000","last_clean":"2024-09-21T07:54:40.609905+0000","last_became_active":"2024-09-21T07:35:27.877050+0000","last_became_peered":"2024-09-21T07:35:27.877050+0000","last_unstale":"2024-09-21T07:54:40.609905+0000","last_undegraded":"2024-09-21T07:54:40.609905+0000","last_fullsized":"2024-09-21T07:54:40.609905+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:36.268334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":397,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.272917+0000","last_change":"2024-09-21T07:35:28.862273+0000","last_active":"2024-09-21T07:54:37.272917+0000","last_peered":"2024-09-21T07:54:37.272917+0000","last_clean":"2024-09-21T07:54:37.272917+0000","last_became_active":"2024-09-21T07:35:26.874011+0000","last_became_peered":"2024-09-21T07:35:26.874011+0000","last_unstale":"2024-09-21T07:54:37.272917+0000","last_undegraded":"2024-09-21T07:54:37.272917+0000","last_fullsized":"2024-09-21T07:54:37.272917+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:11.986950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071644799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":451,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.272263+0000","last_change":"2024-09-21T07:34:24.259794+0000","last_active":"2024-09-21T07:54:37.272263+0000","last_peered":"2024-09-21T07:54:37.272263+0000","last_clean":"2024-09-21T07:54:37.272263+0000","last_became_active":"2024-09-21T07:34:23.265679+0000","last_became_peered":"2024-09-21T07:34:23.265679+0000","last_unstale":"2024-09-21T07:54:37.272263+0000","last_undegraded":"2024-09-21T07:54:37.272263+0000","last_fullsized":"2024-09-21T07:54:37.272263+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:02.575796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001298518,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":1,"reported_epoch":262,"state":"creating+peering","last_fresh":"2024-09-21T07:54:37.270876+0000","last_change":"2024-09-21T07:54:37.270876+0000","last_active":"2024-09-21T07:54:37.257240+0000","last_peered":"2024-09-21T07:54:37.257240+0000","last_clean":"2024-09-21T07:54:37.257240+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:37.270876+0000","last_undegraded":"2024-09-21T07:54:37.270876+0000","last_fullsized":"2024-09-21T07:54:37.270876+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":395,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.274677+0000","last_change":"2024-09-21T07:35:27.877725+0000","last_active":"2024-09-21T07:54:37.274677+0000","last_peered":"2024-09-21T07:54:37.274677+0000","last_clean":"2024-09-21T07:54:37.274677+0000","last_became_active":"2024-09-21T07:35:27.877398+0000","last_became_peered":"2024-09-21T07:35:27.877398+0000","last_unstale":"2024-09-21T07:54:37.274677+0000","last_undegraded":"2024-09-21T07:54:37.274677+0000","last_fullsized":"2024-09-21T07:54:37.274677+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:05.859762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":398,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.270342+0000","last_change":"2024-09-21T07:35:28.860612+0000","last_active":"2024-09-21T07:54:37.270342+0000","last_peered":"2024-09-21T07:54:37.270342+0000","last_clean":"2024-09-21T07:54:37.270342+0000","last_became_active":"2024-09-21T07:35:26.876562+0000","last_became_peered":"2024-09-21T07:35:26.876562+0000","last_unstale":"2024-09-21T07:54:37.270342+0000","last_undegraded":"2024-09-21T07:54:37.270342+0000","last_fullsized":"2024-09-21T07:54:37.270342+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:06.824331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012252470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":449,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.273578+0000","last_change":"2024-09-21T07:34:24.260330+0000","last_active":"2024-09-21T07:54:37.273578+0000","last_peered":"2024-09-21T07:54:37.273578+0000","last_clean":"2024-09-21T07:54:37.273578+0000","last_became_active":"2024-09-21T07:34:23.637210+0000","last_became_peered":"2024-09-21T07:34:23.637210+0000","last_unstale":"2024-09-21T07:54:37.273578+0000","last_undegraded":"2024-09-21T07:54:37.273578+0000","last_fullsized":"2024-09-21T07:54:37.273578+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.488451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001537093,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":15,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.609100+0000","last_change":"2024-09-21T07:54:38.300817+0000","last_active":"2024-09-21T07:54:40.609100+0000","last_peered":"2024-09-21T07:54:40.609100+0000","last_clean":"2024-09-21T07:54:40.609100+0000","last_became_active":"2024-09-21T07:54:38.300443+0000","last_became_peered":"2024-09-21T07:54:38.300443+0000","last_unstale":"2024-09-21T07:54:40.609100+0000","last_undegraded":"2024-09-21T07:54:40.609100+0000","last_fullsized":"2024-09-21T07:54:40.609100+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:50.932301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":391,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.272952+0000","last_change":"2024-09-21T07:35:27.874757+0000","last_active":"2024-09-21T07:54:37.272952+0000","last_peered":"2024-09-21T07:54:37.272952+0000","last_clean":"2024-09-21T07:54:37.272952+0000","last_became_active":"2024-09-21T07:35:27.874360+0000","last_became_peered":"2024-09-21T07:35:27.874360+0000","last_unstale":"2024-09-21T07:54:37.272952+0000","last_undegraded":"2024-09-21T07:54:37.272952+0000","last_fullsized":"2024-09-21T07:54:37.272952+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:17.646912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":395,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.273602+0000","last_change":"2024-09-21T07:35:29.493431+0000","last_active":"2024-09-21T07:54:37.273602+0000","last_peered":"2024-09-21T07:54:37.273602+0000","last_clean":"2024-09-21T07:54:37.273602+0000","last_became_active":"2024-09-21T07:35:26.873691+0000","last_became_peered":"2024-09-21T07:35:26.873691+0000","last_unstale":"2024-09-21T07:54:37.273602+0000","last_undegraded":"2024-09-21T07:54:37.273602+0000","last_fullsized":"2024-09-21T07:54:37.273602+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:51.705051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001162464,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":447,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.610382+0000","last_change":"2024-09-21T07:34:24.597286+0000","last_active":"2024-09-21T07:54:40.610382+0000","last_peered":"2024-09-21T07:54:40.610382+0000","last_clean":"2024-09-21T07:54:40.610382+0000","last_became_active":"2024-09-21T07:34:23.267133+0000","last_became_peered":"2024-09-21T07:34:23.267133+0000","last_unstale":"2024-09-21T07:54:40.610382+0000","last_undegraded":"2024-09-21T07:54:40.610382+0000","last_fullsized":"2024-09-21T07:54:40.610382+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:18.257010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.576344+0000","last_change":"2024-09-21T07:54:39.299363+0000","last_active":"2024-09-21T07:54:40.576344+0000","last_peered":"2024-09-21T07:54:40.576344+0000","last_clean":"2024-09-21T07:54:40.576344+0000","last_became_active":"2024-09-21T07:54:39.298900+0000","last_became_peered":"2024-09-21T07:54:39.298900+0000","last_unstale":"2024-09-21T07:54:40.576344+0000","last_undegraded":"2024-09-21T07:54:40.576344+0000","last_fullsized":"2024-09-21T07:54:40.576344+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:19.143810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":1,"reported_epoch":262,"state":"creating+peering","last_fresh":"2024-09-21T07:54:37.271169+0000","last_change":"2024-09-21T07:54:37.271169+0000","last_active":"2024-09-21T07:54:37.257240+0000","last_peered":"2024-09-21T07:54:37.257240+0000","last_clean":"2024-09-21T07:54:37.257240+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:37.271169+0000","last_undegraded":"2024-09-21T07:54:37.271169+0000","last_fullsized":"2024-09-21T07:54:37.271169+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":406,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.266319+0000","last_change":"2024-09-21T07:35:27.864458+0000","last_active":"2024-09-21T07:54:37.266319+0000","last_peered":"2024-09-21T07:54:37.266319+0000","last_clean":"2024-09-21T07:54:37.266319+0000","last_became_active":"2024-09-21T07:35:27.864136+0000","last_became_peered":"2024-09-21T07:35:27.864136+0000","last_unstale":"2024-09-21T07:54:37.266319+0000","last_undegraded":"2024-09-21T07:54:37.266319+0000","last_fullsized":"2024-09-21T07:54:37.266319+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:20.162240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":412,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.267064+0000","last_change":"2024-09-21T07:35:28.858784+0000","last_active":"2024-09-21T07:54:37.267064+0000","last_peered":"2024-09-21T07:54:37.267064+0000","last_clean":"2024-09-21T07:54:37.267064+0000","last_became_active":"2024-09-21T07:35:26.876890+0000","last_became_peered":"2024-09-21T07:35:26.876890+0000","last_unstale":"2024-09-21T07:54:37.267064+0000","last_undegraded":"2024-09-21T07:54:37.267064+0000","last_fullsized":"2024-09-21T07:54:37.267064+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:40.738327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064543499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":458,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.577228+0000","last_change":"2024-09-21T07:34:24.258712+0000","last_active":"2024-09-21T07:54:40.577228+0000","last_peered":"2024-09-21T07:54:40.577228+0000","last_clean":"2024-09-21T07:54:40.577228+0000","last_became_active":"2024-09-21T07:34:23.274692+0000","last_became_peered":"2024-09-21T07:34:23.274692+0000","last_unstale":"2024-09-21T07:54:40.577228+0000","last_undegraded":"2024-09-21T07:54:40.577228+0000","last_fullsized":"2024-09-21T07:54:40.577228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:43.279454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018051479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":1,"reported_epoch":262,"state":"creating+peering","last_fresh":"2024-09-21T07:54:37.274434+0000","last_change":"2024-09-21T07:54:37.274434+0000","last_active":"2024-09-21T07:54:37.257240+0000","last_peered":"2024-09-21T07:54:37.257240+0000","last_clean":"2024-09-21T07:54:37.257240+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:37.274434+0000","last_undegraded":"2024-09-21T07:54:37.274434+0000","last_fullsized":"2024-09-21T07:54:37.274434+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":398,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.576254+0000","last_change":"2024-09-21T07:35:27.878020+0000","last_active":"2024-09-21T07:54:40.576254+0000","last_peered":"2024-09-21T07:54:40.576254+0000","last_clean":"2024-09-21T07:54:40.576254+0000","last_became_active":"2024-09-21T07:35:27.877532+0000","last_became_peered":"2024-09-21T07:35:27.877532+0000","last_unstale":"2024-09-21T07:54:40.576254+0000","last_undegraded":"2024-09-21T07:54:40.576254+0000","last_fullsized":"2024-09-21T07:54:40.576254+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:19.756073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":398,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.272520+0000","last_change":"2024-09-21T07:35:28.861049+0000","last_active":"2024-09-21T07:54:37.272520+0000","last_peered":"2024-09-21T07:54:37.272520+0000","last_clean":"2024-09-21T07:54:37.272520+0000","last_became_active":"2024-09-21T07:35:26.876819+0000","last_became_peered":"2024-09-21T07:35:26.876819+0000","last_unstale":"2024-09-21T07:54:37.272520+0000","last_undegraded":"2024-09-21T07:54:37.272520+0000","last_fullsized":"2024-09-21T07:54:37.272520+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:19.633803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055780599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":449,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.272917+0000","last_change":"2024-09-21T07:34:25.084849+0000","last_active":"2024-09-21T07:54:37.272917+0000","last_peered":"2024-09-21T07:54:37.272917+0000","last_clean":"2024-09-21T07:54:37.272917+0000","last_became_active":"2024-09-21T07:34:23.277713+0000","last_became_peered":"2024-09-21T07:34:23.277713+0000","last_unstale":"2024-09-21T07:54:37.272917+0000","last_undegraded":"2024-09-21T07:54:37.272917+0000","last_fullsized":"2024-09-21T07:54:37.272917+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:05.811810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059079499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.609524+0000","last_change":"2024-09-21T07:54:39.300046+0000","last_active":"2024-09-21T07:54:40.609524+0000","last_peered":"2024-09-21T07:54:40.609524+0000","last_clean":"2024-09-21T07:54:40.609524+0000","last_became_active":"2024-09-21T07:54:39.299733+0000","last_became_peered":"2024-09-21T07:54:39.299733+0000","last_unstale":"2024-09-21T07:54:40.609524+0000","last_undegraded":"2024-09-21T07:54:40.609524+0000","last_fullsized":"2024-09-21T07:54:40.609524+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:05:28.970944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":15,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.608040+0000","last_change":"2024-09-21T07:54:38.290947+0000","last_active":"2024-09-21T07:54:40.608040+0000","last_peered":"2024-09-21T07:54:40.608040+0000","last_clean":"2024-09-21T07:54:40.608040+0000","last_became_active":"2024-09-21T07:54:38.290609+0000","last_became_peered":"2024-09-21T07:54:38.290609+0000","last_unstale":"2024-09-21T07:54:40.608040+0000","last_undegraded":"2024-09-21T07:54:40.608040+0000","last_fullsized":"2024-09-21T07:54:40.608040+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:40.430199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":392,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.276189+0000","last_change":"2024-09-21T07:35:27.884248+0000","last_active":"2024-09-21T07:54:37.276189+0000","last_peered":"2024-09-21T07:54:37.276189+0000","last_clean":"2024-09-21T07:54:37.276189+0000","last_became_active":"2024-09-21T07:35:27.883861+0000","last_became_peered":"2024-09-21T07:35:27.883861+0000","last_unstale":"2024-09-21T07:54:37.276189+0000","last_undegraded":"2024-09-21T07:54:37.276189+0000","last_fullsized":"2024-09-21T07:54:37.276189+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":401,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.274431+0000","last_change":"2024-09-21T07:35:29.207135+0000","last_active":"2024-09-21T07:54:37.274431+0000","last_peered":"2024-09-21T07:54:37.274431+0000","last_clean":"2024-09-21T07:54:37.274431+0000","last_became_active":"2024-09-21T07:35:26.872787+0000","last_became_peered":"2024-09-21T07:35:26.872787+0000","last_unstale":"2024-09-21T07:54:37.274431+0000","last_undegraded":"2024-09-21T07:54:37.274431+0000","last_fullsized":"2024-09-21T07:54:37.274431+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:27.543217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011157509999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":465,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.266283+0000","last_change":"2024-09-21T07:34:24.538610+0000","last_active":"2024-09-21T07:54:37.266283+0000","last_peered":"2024-09-21T07:54:37.266283+0000","last_clean":"2024-09-21T07:54:37.266283+0000","last_became_active":"2024-09-21T07:34:23.259700+0000","last_became_peered":"2024-09-21T07:34:23.259700+0000","last_unstale":"2024-09-21T07:54:37.266283+0000","last_undegraded":"2024-09-21T07:54:37.266283+0000","last_fullsized":"2024-09-21T07:54:37.266283+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:57.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070522399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.7","version":"0'0","reported_seq":1,"reported_epoch":262,"state":"creating+peering","last_fresh":"2024-09-21T07:54:37.275445+0000","last_change":"2024-09-21T07:54:37.275445+0000","last_active":"2024-09-21T07:54:37.257240+0000","last_peered":"2024-09-21T07:54:37.257240+0000","last_clean":"2024-09-21T07:54:37.257240+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:37.275445+0000","last_undegraded":"2024-09-21T07:54:37.275445+0000","last_fullsized":"2024-09-21T07:54:37.275445+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":406,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.266833+0000","last_change":"2024-09-21T07:35:27.868024+0000","last_active":"2024-09-21T07:54:37.266833+0000","last_peered":"2024-09-21T07:54:37.266833+0000","last_clean":"2024-09-21T07:54:37.266833+0000","last_became_active":"2024-09-21T07:35:27.867339+0000","last_became_peered":"2024-09-21T07:35:27.867339+0000","last_unstale":"2024-09-21T07:54:37.266833+0000","last_undegraded":"2024-09-21T07:54:37.266833+0000","last_fullsized":"2024-09-21T07:54:37.266833+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":413,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.266706+0000","last_change":"2024-09-21T07:35:28.858505+0000","last_active":"2024-09-21T07:54:37.266706+0000","last_peered":"2024-09-21T07:54:37.266706+0000","last_clean":"2024-09-21T07:54:37.266706+0000","last_became_active":"2024-09-21T07:35:26.877320+0000","last_became_peered":"2024-09-21T07:35:26.877320+0000","last_unstale":"2024-09-21T07:54:37.266706+0000","last_undegraded":"2024-09-21T07:54:37.266706+0000","last_fullsized":"2024-09-21T07:54:37.266706+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:53.672782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244119999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":449,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.609207+0000","last_change":"2024-09-21T07:34:24.597597+0000","last_active":"2024-09-21T07:54:40.609207+0000","last_peered":"2024-09-21T07:54:40.609207+0000","last_clean":"2024-09-21T07:54:40.609207+0000","last_became_active":"2024-09-21T07:34:23.635204+0000","last_became_peered":"2024-09-21T07:34:23.635204+0000","last_unstale":"2024-09-21T07:54:40.609207+0000","last_undegraded":"2024-09-21T07:54:40.609207+0000","last_fullsized":"2024-09-21T07:54:40.609207+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:48.644789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081139199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.6","version":"0'0","reported_seq":1,"reported_epoch":262,"state":"creating+peering","last_fresh":"2024-09-21T07:54:37.271736+0000","last_change":"2024-09-21T07:54:37.271736+0000","last_active":"2024-09-21T07:54:37.257240+0000","last_peered":"2024-09-21T07:54:37.257240+0000","last_clean":"2024-09-21T07:54:37.257240+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:37.271736+0000","last_undegraded":"2024-09-21T07:54:37.271736+0000","last_fullsized":"2024-09-21T07:54:37.271736+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":393,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.274446+0000","last_change":"2024-09-21T07:35:27.877503+0000","last_active":"2024-09-21T07:54:37.274446+0000","last_peered":"2024-09-21T07:54:37.274446+0000","last_clean":"2024-09-21T07:54:37.274446+0000","last_became_active":"2024-09-21T07:35:27.875775+0000","last_became_peered":"2024-09-21T07:35:27.875775+0000","last_unstale":"2024-09-21T07:54:37.274446+0000","last_undegraded":"2024-09-21T07:54:37.274446+0000","last_fullsized":"2024-09-21T07:54:37.274446+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:49.291023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":396,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.272056+0000","last_change":"2024-09-21T07:35:29.491412+0000","last_active":"2024-09-21T07:54:37.272056+0000","last_peered":"2024-09-21T07:54:37.272056+0000","last_clean":"2024-09-21T07:54:37.272056+0000","last_became_active":"2024-09-21T07:35:26.876404+0000","last_became_peered":"2024-09-21T07:35:26.876404+0000","last_unstale":"2024-09-21T07:54:37.272056+0000","last_undegraded":"2024-09-21T07:54:37.272056+0000","last_fullsized":"2024-09-21T07:54:37.272056+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063918000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":459,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.577083+0000","last_change":"2024-09-21T07:34:24.259026+0000","last_active":"2024-09-21T07:54:40.577083+0000","last_peered":"2024-09-21T07:54:40.577083+0000","last_clean":"2024-09-21T07:54:40.577083+0000","last_became_active":"2024-09-21T07:34:23.635456+0000","last_became_peered":"2024-09-21T07:34:23.635456+0000","last_unstale":"2024-09-21T07:54:40.577083+0000","last_undegraded":"2024-09-21T07:54:40.577083+0000","last_fullsized":"2024-09-21T07:54:40.577083+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:09.592752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018654419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":1,"reported_epoch":262,"state":"creating+peering","last_fresh":"2024-09-21T07:54:37.271424+0000","last_change":"2024-09-21T07:54:37.271424+0000","last_active":"2024-09-21T07:54:37.257240+0000","last_peered":"2024-09-21T07:54:37.257240+0000","last_clean":"2024-09-21T07:54:37.257240+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:37.271424+0000","last_undegraded":"2024-09-21T07:54:37.271424+0000","last_fullsized":"2024-09-21T07:54:37.271424+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":393,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.274582+0000","last_change":"2024-09-21T07:35:27.871984+0000","last_active":"2024-09-21T07:54:37.274582+0000","last_peered":"2024-09-21T07:54:37.274582+0000","last_clean":"2024-09-21T07:54:37.274582+0000","last_became_active":"2024-09-21T07:35:27.870336+0000","last_became_peered":"2024-09-21T07:35:27.870336+0000","last_unstale":"2024-09-21T07:54:37.274582+0000","last_undegraded":"2024-09-21T07:54:37.274582+0000","last_fullsized":"2024-09-21T07:54:37.274582+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:57.120479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":399,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.271284+0000","last_change":"2024-09-21T07:35:28.860351+0000","last_active":"2024-09-21T07:54:37.271284+0000","last_peered":"2024-09-21T07:54:37.271284+0000","last_clean":"2024-09-21T07:54:37.271284+0000","last_became_active":"2024-09-21T07:35:26.872046+0000","last_became_peered":"2024-09-21T07:35:26.872046+0000","last_unstale":"2024-09-21T07:54:37.271284+0000","last_undegraded":"2024-09-21T07:54:37.271284+0000","last_fullsized":"2024-09-21T07:54:37.271284+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:47.074327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098384399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":449,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.273322+0000","last_change":"2024-09-21T07:34:24.260300+0000","last_active":"2024-09-21T07:54:37.273322+0000","last_peered":"2024-09-21T07:54:37.273322+0000","last_clean":"2024-09-21T07:54:37.273322+0000","last_became_active":"2024-09-21T07:34:23.270128+0000","last_became_peered":"2024-09-21T07:34:23.270128+0000","last_unstale":"2024-09-21T07:54:37.273322+0000","last_undegraded":"2024-09-21T07:54:37.273322+0000","last_fullsized":"2024-09-21T07:54:37.273322+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:25.260931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00176651,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:54:38.260838+0000","last_change":"2024-09-21T07:54:38.260838+0000","last_active":"2024-09-21T07:54:38.260838+0000","last_peered":"2024-09-21T07:54:38.260838+0000","last_clean":"2024-09-21T07:54:38.260838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:38.260838+0000","last_undegraded":"2024-09-21T07:54:38.260838+0000","last_fullsized":"2024-09-21T07:54:38.260838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":1,"reported_epoch":262,"state":"creating+peering","last_fresh":"2024-09-21T07:54:37.272303+0000","last_change":"2024-09-21T07:54:37.272303+0000","last_active":"2024-09-21T07:54:37.257240+0000","last_peered":"2024-09-21T07:54:37.257240+0000","last_clean":"2024-09-21T07:54:37.257240+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:54:37.272303+0000","last_undegraded":"2024-09-21T07:54:37.272303+0000","last_fullsized":"2024-09-21T07:54:37.272303+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":393,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.274710+0000","last_change":"2024-09-21T07:35:27.877861+0000","last_active":"2024-09-21T07:54:37.274710+0000","last_peered":"2024-09-21T07:54:37.274710+0000","last_clean":"2024-09-21T07:54:37.274710+0000","last_became_active":"2024-09-21T07:35:27.877434+0000","last_became_peered":"2024-09-21T07:35:27.877434+0000","last_unstale":"2024-09-21T07:54:37.274710+0000","last_undegraded":"2024-09-21T07:54:37.274710+0000","last_fullsized":"2024-09-21T07:54:37.274710+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":398,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.273236+0000","last_change":"2024-09-21T07:35:28.862954+0000","last_active":"2024-09-21T07:54:37.273236+0000","last_peered":"2024-09-21T07:54:37.273236+0000","last_clean":"2024-09-21T07:54:37.273236+0000","last_became_active":"2024-09-21T07:35:26.870613+0000","last_became_peered":"2024-09-21T07:35:26.870613+0000","last_unstale":"2024-09-21T07:54:37.273236+0000","last_undegraded":"2024-09-21T07:54:37.273236+0000","last_fullsized":"2024-09-21T07:54:37.273236+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:06.901043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059842100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":450,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-21T07:54:37.276498+0000","last_change":"2024-09-21T07:34:25.028904+0000","last_active":"2024-09-21T07:54:37.276498+0000","last_peered":"2024-09-21T07:54:37.276498+0000","last_clean":"2024-09-21T07:54:37.276498+0000","last_became_active":"2024-09-21T07:34:23.260942+0000","last_became_peered":"2024-09-21T07:34:23.260942+0000","last_unstale":"2024-09-21T07:54:37.276498+0000","last_undegraded":"2024-09-21T07:54:37.276498+0000","last_fullsized":"2024-09-21T07:54:37.276498+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:22.709328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060553899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":2286,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14,"ondisk_log_size":14,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":34,"acting":34,"num_store_stats":2},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509702,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71100,"kb_used_data":3808,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94300740,"statfs":{"total":96636764160,"available":96563957760,"internally_reserved":0,"allocated":3899392,"data_stored":2389015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509704,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70524,"kb_used_data":3228,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301316,"statfs":{"total":96636764160,"available":96564547584,"internally_reserved":0,"allocated":3305472,"data_stored":1798647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509702,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52464,"kb_used_data":3216,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319376,"statfs":{"total":96636764160,"available":96583041024,"internally_reserved":0,"allocated":3293184,"data_stored":1791495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509703,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30264,"kb_used_data":3228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341576,"statfs":{"total":96636764160,"available":96605773824,"internally_reserved":0,"allocated":3305472,"data_stored":1798647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509704,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48292,"kb_used_data":3216,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323548,"statfs":{"total":96636764160,"available":96587313152,"internally_reserved":0,"allocated":3293184,"data_stored":1791495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509704,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53040,"kb_used_data":3796,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318800,"statfs":{"total":96636764160,"available":96582451200,"internally_reserved":0,"allocated":3887104,"data_stored":2381863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542408,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56664,"kb_used_data":3268,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315176,"statfs":{"total":96636764160,"available":96578740224,"internally_reserved":0,"allocated":3346432,"data_stored":1822072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542407,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74732,"kb_used_data":3288,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297108,"statfs":{"total":96636764160,"available":96560238592,"internally_reserved":0,"allocated":3366912,"data_stored":1824252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:54:43.798 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:54:44.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.053+0000 7fe512200640 1 -- 172.21.15.39:0/3054108198 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe50c0d0630 msgr2=0x7fe50c0d0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:44.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.053+0000 7fe512200640 1 --2- 172.21.15.39:0/3054108198 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe50c0d0630 0x7fe50c0d0a40 secure :-1 s=READY pgs=2822 cs=0 l=1 rev1=1 crypto rx=0x7fe50c072900 tx=0x7fe50402aa30 comp rx=0 tx=0).stop 2024-09-21T07:54:44.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.053+0000 7fe512200640 1 -- 172.21.15.39:0/3054108198 shutdown_connections 2024-09-21T07:54:44.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.053+0000 7fe512200640 1 --2- 172.21.15.39:0/3054108198 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe50c0d0630 0x7fe50c0d0a40 unknown :-1 s=CLOSED pgs=2822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.053+0000 7fe512200640 1 --2- 172.21.15.39:0/3054108198 >> v2:172.21.15.148:3300/0 conn(0x7fe50c073dd0 0x7fe50c15d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.053+0000 7fe512200640 1 -- 172.21.15.39:0/3054108198 >> 172.21.15.39:0/3054108198 conn(0x7fe50c06cfc0 msgr2=0x7fe50c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:44.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.053+0000 7fe512200640 1 -- 172.21.15.39:0/3054108198 shutdown_connections 2024-09-21T07:54:44.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.054+0000 7fe512200640 1 -- 172.21.15.39:0/3054108198 wait complete. 2024-09-21T07:54:44.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.056+0000 7fe512200640 1 Processor -- start 2024-09-21T07:54:44.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.056+0000 7fe512200640 1 -- start start 2024-09-21T07:54:44.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.057+0000 7fe512200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe50c073dd0 0x7fe50c151770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:44.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.057+0000 7fe512200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe50c0d0630 0x7fe50c15aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:44.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.057+0000 7fe512200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe50c15daf0 0x7fe50c1e6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:44.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.057+0000 7fe512200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c0cd1b0 con 0x7fe50c0d0630 2024-09-21T07:54:44.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.057+0000 7fe512200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c0cd030 con 0x7fe50c15daf0 2024-09-21T07:54:44.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.057+0000 7fe512200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe50c0cd330 con 0x7fe50c073dd0 2024-09-21T07:54:44.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.057+0000 7fe502a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe50c0d0630 0x7fe50c15aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:44.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.058+0000 7fe502a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe50c0d0630 0x7fe50c15aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47326/0 (socket says 172.21.15.39:47326) 2024-09-21T07:54:44.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.058+0000 7fe502a00640 1 -- 172.21.15.39:0/4082406088 learned_addr learned my addr 172.21.15.39:0/4082406088 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:44.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.058+0000 7fe50b400640 1 --2- 172.21.15.39:0/4082406088 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe50c073dd0 0x7fe50c151770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:44.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.058+0000 7fe50be00640 1 --2- 172.21.15.39:0/4082406088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe50c15daf0 0x7fe50c1e6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:44.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.058+0000 7fe502a00640 1 -- 172.21.15.39:0/4082406088 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe50c073dd0 msgr2=0x7fe50c151770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:44.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.058+0000 7fe502a00640 1 --2- 172.21.15.39:0/4082406088 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe50c073dd0 0x7fe50c151770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.058+0000 7fe502a00640 1 -- 172.21.15.39:0/4082406088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe50c15daf0 msgr2=0x7fe50c1e6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:44.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.058+0000 7fe502a00640 1 --2- 172.21.15.39:0/4082406088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe50c15daf0 0x7fe50c1e6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.058+0000 7fe502a00640 1 -- 172.21.15.39:0/4082406088 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe50c1e5a70 con 0x7fe50c0d0630 2024-09-21T07:54:44.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.058+0000 7fe50b400640 1 --2- 172.21.15.39:0/4082406088 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe50c073dd0 0x7fe50c151770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:44.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.058+0000 7fe50be00640 1 --2- 172.21.15.39:0/4082406088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe50c15daf0 0x7fe50c1e6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:44.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.058+0000 7fe502a00640 1 --2- 172.21.15.39:0/4082406088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe50c0d0630 0x7fe50c15aa90 secure :-1 s=READY pgs=2823 cs=0 l=1 rev1=1 crypto rx=0x7fe4f400c340 tx=0x7fe4f400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:44.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.058+0000 7fe503e00640 1 -- 172.21.15.39:0/4082406088 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f40052d0 con 0x7fe50c0d0630 2024-09-21T07:54:44.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.059+0000 7fe503e00640 1 -- 172.21.15.39:0/4082406088 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4f4007020 con 0x7fe50c0d0630 2024-09-21T07:54:44.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.059+0000 7fe512200640 1 -- 172.21.15.39:0/4082406088 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe50c1e5d50 con 0x7fe50c0d0630 2024-09-21T07:54:44.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.059+0000 7fe503e00640 1 -- 172.21.15.39:0/4082406088 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4007470 con 0x7fe50c0d0630 2024-09-21T07:54:44.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.059+0000 7fe512200640 1 -- 172.21.15.39:0/4082406088 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe50c151d10 con 0x7fe50c0d0630 2024-09-21T07:54:44.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.060+0000 7fe512200640 1 -- 172.21.15.39:0/4082406088 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe50c15b030 con 0x7fe50c0d0630 2024-09-21T07:54:44.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.061+0000 7fe503e00640 1 -- 172.21.15.39:0/4082406088 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe4f40067d0 con 0x7fe50c0d0630 2024-09-21T07:54:44.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.062+0000 7fe503e00640 1 --2- 172.21.15.39:0/4082406088 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe4e8078bc0 0x7fe4e807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:44.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.062+0000 7fe503e00640 1 -- 172.21.15.39:0/4082406088 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(265..265 src has 1..265) ==== 7044+0+0 (secure 0 0 0) 0x7fe4f409a760 con 0x7fe50c0d0630 2024-09-21T07:54:44.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.062+0000 7fe50b400640 1 --2- 172.21.15.39:0/4082406088 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe4e8078bc0 0x7fe4e807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:44.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.063+0000 7fe50b400640 1 --2- 172.21.15.39:0/4082406088 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe4e8078bc0 0x7fe4e807b0d0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc005fd0 tx=0x7fe4fc00f360 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:44.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.066+0000 7fe503e00640 1 -- 172.21.15.39:0/4082406088 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe50c0d0f80 con 0x7fe50c0d0630 2024-09-21T07:54:44.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.261+0000 7fe512200640 1 -- 172.21.15.39:0/4082406088 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe50c0cd850 con 0x7fe4e8078bc0 2024-09-21T07:54:44.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.271+0000 7fe503e00640 1 -- 172.21.15.39:0/4082406088 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+495243 (secure 0 0 0) 0x7fe50c0cd850 con 0x7fe4e8078bc0 2024-09-21T07:54:44.280 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:44.284 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:54:44.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.277+0000 7fe512200640 1 -- 172.21.15.39:0/4082406088 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe4e8078bc0 msgr2=0x7fe4e807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:44.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.277+0000 7fe512200640 1 --2- 172.21.15.39:0/4082406088 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe4e8078bc0 0x7fe4e807b0d0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc005fd0 tx=0x7fe4fc00f360 comp rx=0 tx=0).stop 2024-09-21T07:54:44.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.277+0000 7fe512200640 1 -- 172.21.15.39:0/4082406088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe50c0d0630 msgr2=0x7fe50c15aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:44.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.277+0000 7fe512200640 1 --2- 172.21.15.39:0/4082406088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe50c0d0630 0x7fe50c15aa90 secure :-1 s=READY pgs=2823 cs=0 l=1 rev1=1 crypto rx=0x7fe4f400c340 tx=0x7fe4f400c800 comp rx=0 tx=0).stop 2024-09-21T07:54:44.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.277+0000 7fe512200640 1 -- 172.21.15.39:0/4082406088 shutdown_connections 2024-09-21T07:54:44.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.277+0000 7fe512200640 1 --2- 172.21.15.39:0/4082406088 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe4e8078bc0 0x7fe4e807b0d0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.277+0000 7fe512200640 1 --2- 172.21.15.39:0/4082406088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe50c15daf0 0x7fe50c1e6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.277+0000 7fe512200640 1 --2- 172.21.15.39:0/4082406088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe50c0d0630 0x7fe50c15aa90 unknown :-1 s=CLOSED pgs=2823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.277+0000 7fe512200640 1 --2- 172.21.15.39:0/4082406088 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe50c073dd0 0x7fe50c151770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.277+0000 7fe512200640 1 -- 172.21.15.39:0/4082406088 >> 172.21.15.39:0/4082406088 conn(0x7fe50c06cfc0 msgr2=0x7fe50c0b0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:44.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.277+0000 7fe512200640 1 -- 172.21.15.39:0/4082406088 shutdown_connections 2024-09-21T07:54:44.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.277+0000 7fe512200640 1 -- 172.21.15.39:0/4082406088 wait complete. 2024-09-21T07:54:44.303 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":911,"stamp":"2024-09-21T07:54:43.979589+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":355,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":461504,"kb_used_data":27336,"kb_used_omap":12,"kb_used_meta":433971,"kb_avail":754513216,"statfs":{"total":773094113280,"available":772621533184,"internally_reserved":0,"allocated":27992064,"data_stored":15761572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":444386864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002223"},"pg_stats":[{"pgid":"32.1f","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.576786+0000","last_change":"2024-09-21T07:54:39.304408+0000","last_active":"2024-09-21T07:54:40.576786+0000","last_peered":"2024-09-21T07:54:40.576786+0000","last_clean":"2024-09-21T07:54:40.576786+0000","last_became_active":"2024-09-21T07:54:39.301322+0000","last_became_peered":"2024-09-21T07:54:39.301322+0000","last_unstale":"2024-09-21T07:54:40.576786+0000","last_undegraded":"2024-09-21T07:54:40.576786+0000","last_fullsized":"2024-09-21T07:54:40.576786+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:28.727090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.279349+0000","last_change":"2024-09-21T07:54:39.305324+0000","last_active":"2024-09-21T07:54:40.279349+0000","last_peered":"2024-09-21T07:54:40.279349+0000","last_clean":"2024-09-21T07:54:40.279349+0000","last_became_active":"2024-09-21T07:54:39.300955+0000","last_became_peered":"2024-09-21T07:54:39.300955+0000","last_unstale":"2024-09-21T07:54:40.279349+0000","last_undegraded":"2024-09-21T07:54:40.279349+0000","last_fullsized":"2024-09-21T07:54:40.279349+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:50:49.717146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.276153+0000","last_change":"2024-09-21T07:54:39.279903+0000","last_active":"2024-09-21T07:54:40.276153+0000","last_peered":"2024-09-21T07:54:40.276153+0000","last_clean":"2024-09-21T07:54:40.276153+0000","last_became_active":"2024-09-21T07:54:39.279364+0000","last_became_peered":"2024-09-21T07:54:39.279364+0000","last_unstale":"2024-09-21T07:54:40.276153+0000","last_undegraded":"2024-09-21T07:54:40.276153+0000","last_fullsized":"2024-09-21T07:54:40.276153+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:17:01.984642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.275675+0000","last_change":"2024-09-21T07:54:39.296077+0000","last_active":"2024-09-21T07:54:40.275675+0000","last_peered":"2024-09-21T07:54:40.275675+0000","last_clean":"2024-09-21T07:54:40.275675+0000","last_became_active":"2024-09-21T07:54:39.295751+0000","last_became_peered":"2024-09-21T07:54:39.295751+0000","last_unstale":"2024-09-21T07:54:40.275675+0000","last_undegraded":"2024-09-21T07:54:40.275675+0000","last_fullsized":"2024-09-21T07:54:40.275675+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:35:20.197640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.611377+0000","last_change":"2024-09-21T07:54:39.306439+0000","last_active":"2024-09-21T07:54:40.611377+0000","last_peered":"2024-09-21T07:54:40.611377+0000","last_clean":"2024-09-21T07:54:40.611377+0000","last_became_active":"2024-09-21T07:54:39.304472+0000","last_became_peered":"2024-09-21T07:54:39.304472+0000","last_unstale":"2024-09-21T07:54:40.611377+0000","last_undegraded":"2024-09-21T07:54:40.611377+0000","last_fullsized":"2024-09-21T07:54:40.611377+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:22:49.121098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.576497+0000","last_change":"2024-09-21T07:54:39.318701+0000","last_active":"2024-09-21T07:54:40.576497+0000","last_peered":"2024-09-21T07:54:40.576497+0000","last_clean":"2024-09-21T07:54:40.576497+0000","last_became_active":"2024-09-21T07:54:39.318375+0000","last_became_peered":"2024-09-21T07:54:39.318375+0000","last_unstale":"2024-09-21T07:54:40.576497+0000","last_undegraded":"2024-09-21T07:54:40.576497+0000","last_fullsized":"2024-09-21T07:54:40.576497+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:58:21.776964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.278302+0000","last_change":"2024-09-21T07:54:39.306728+0000","last_active":"2024-09-21T07:54:40.278302+0000","last_peered":"2024-09-21T07:54:40.278302+0000","last_clean":"2024-09-21T07:54:40.278302+0000","last_became_active":"2024-09-21T07:54:39.306128+0000","last_became_peered":"2024-09-21T07:54:39.306128+0000","last_unstale":"2024-09-21T07:54:40.278302+0000","last_undegraded":"2024-09-21T07:54:40.278302+0000","last_fullsized":"2024-09-21T07:54:40.278302+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:21:23.134965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.608036+0000","last_change":"2024-09-21T07:54:39.297758+0000","last_active":"2024-09-21T07:54:40.608036+0000","last_peered":"2024-09-21T07:54:40.608036+0000","last_clean":"2024-09-21T07:54:40.608036+0000","last_became_active":"2024-09-21T07:54:39.297289+0000","last_became_peered":"2024-09-21T07:54:39.297289+0000","last_unstale":"2024-09-21T07:54:40.608036+0000","last_undegraded":"2024-09-21T07:54:40.608036+0000","last_fullsized":"2024-09-21T07:54:40.608036+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:22:34.976301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.577059+0000","last_change":"2024-09-21T07:54:39.308208+0000","last_active":"2024-09-21T07:54:40.577059+0000","last_peered":"2024-09-21T07:54:40.577059+0000","last_clean":"2024-09-21T07:54:40.577059+0000","last_became_active":"2024-09-21T07:54:39.307863+0000","last_became_peered":"2024-09-21T07:54:39.307863+0000","last_unstale":"2024-09-21T07:54:40.577059+0000","last_undegraded":"2024-09-21T07:54:40.577059+0000","last_fullsized":"2024-09-21T07:54:40.577059+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:58.582247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.276334+0000","last_change":"2024-09-21T07:54:39.297191+0000","last_active":"2024-09-21T07:54:40.276334+0000","last_peered":"2024-09-21T07:54:40.276334+0000","last_clean":"2024-09-21T07:54:40.276334+0000","last_became_active":"2024-09-21T07:54:39.296855+0000","last_became_peered":"2024-09-21T07:54:39.296855+0000","last_unstale":"2024-09-21T07:54:40.276334+0000","last_undegraded":"2024-09-21T07:54:40.276334+0000","last_fullsized":"2024-09-21T07:54:40.276334+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:02.525178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.276763+0000","last_change":"2024-09-21T07:54:39.287521+0000","last_active":"2024-09-21T07:54:40.276763+0000","last_peered":"2024-09-21T07:54:40.276763+0000","last_clean":"2024-09-21T07:54:40.276763+0000","last_became_active":"2024-09-21T07:54:39.287198+0000","last_became_peered":"2024-09-21T07:54:39.287198+0000","last_unstale":"2024-09-21T07:54:40.276763+0000","last_undegraded":"2024-09-21T07:54:40.276763+0000","last_fullsized":"2024-09-21T07:54:40.276763+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:21:07.183461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.576546+0000","last_change":"2024-09-21T07:54:39.301716+0000","last_active":"2024-09-21T07:54:40.576546+0000","last_peered":"2024-09-21T07:54:40.576546+0000","last_clean":"2024-09-21T07:54:40.576546+0000","last_became_active":"2024-09-21T07:54:39.301428+0000","last_became_peered":"2024-09-21T07:54:39.301428+0000","last_unstale":"2024-09-21T07:54:40.576546+0000","last_undegraded":"2024-09-21T07:54:40.576546+0000","last_fullsized":"2024-09-21T07:54:40.576546+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:04.990166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.608973+0000","last_change":"2024-09-21T07:54:39.298698+0000","last_active":"2024-09-21T07:54:40.608973+0000","last_peered":"2024-09-21T07:54:40.608973+0000","last_clean":"2024-09-21T07:54:40.608973+0000","last_became_active":"2024-09-21T07:54:39.296272+0000","last_became_peered":"2024-09-21T07:54:39.296272+0000","last_unstale":"2024-09-21T07:54:40.608973+0000","last_undegraded":"2024-09-21T07:54:40.608973+0000","last_fullsized":"2024-09-21T07:54:40.608973+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:13.600267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.607227+0000","last_change":"2024-09-21T07:54:39.305166+0000","last_active":"2024-09-21T07:54:40.607227+0000","last_peered":"2024-09-21T07:54:40.607227+0000","last_clean":"2024-09-21T07:54:40.607227+0000","last_became_active":"2024-09-21T07:54:39.304494+0000","last_became_peered":"2024-09-21T07:54:39.304494+0000","last_unstale":"2024-09-21T07:54:40.607227+0000","last_undegraded":"2024-09-21T07:54:40.607227+0000","last_fullsized":"2024-09-21T07:54:40.607227+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:32.974694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.610789+0000","last_change":"2024-09-21T07:54:39.312293+0000","last_active":"2024-09-21T07:54:40.610789+0000","last_peered":"2024-09-21T07:54:40.610789+0000","last_clean":"2024-09-21T07:54:40.610789+0000","last_became_active":"2024-09-21T07:54:39.311656+0000","last_became_peered":"2024-09-21T07:54:39.311656+0000","last_unstale":"2024-09-21T07:54:40.610789+0000","last_undegraded":"2024-09-21T07:54:40.610789+0000","last_fullsized":"2024-09-21T07:54:40.610789+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:32.974694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.275386+0000","last_change":"2024-09-21T07:54:39.307549+0000","last_active":"2024-09-21T07:54:40.275386+0000","last_peered":"2024-09-21T07:54:40.275386+0000","last_clean":"2024-09-21T07:54:40.275386+0000","last_became_active":"2024-09-21T07:54:39.306029+0000","last_became_peered":"2024-09-21T07:54:39.306029+0000","last_unstale":"2024-09-21T07:54:40.275386+0000","last_undegraded":"2024-09-21T07:54:40.275386+0000","last_fullsized":"2024-09-21T07:54:40.275386+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:14:53.998739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.610018+0000","last_change":"2024-09-21T07:54:39.313058+0000","last_active":"2024-09-21T07:54:40.610018+0000","last_peered":"2024-09-21T07:54:40.610018+0000","last_clean":"2024-09-21T07:54:40.610018+0000","last_became_active":"2024-09-21T07:54:39.312661+0000","last_became_peered":"2024-09-21T07:54:39.312661+0000","last_unstale":"2024-09-21T07:54:40.610018+0000","last_undegraded":"2024-09-21T07:54:40.610018+0000","last_fullsized":"2024-09-21T07:54:40.610018+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:18.895423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.276505+0000","last_change":"2024-09-21T07:54:39.292371+0000","last_active":"2024-09-21T07:54:40.276505+0000","last_peered":"2024-09-21T07:54:40.276505+0000","last_clean":"2024-09-21T07:54:40.276505+0000","last_became_active":"2024-09-21T07:54:39.292025+0000","last_became_peered":"2024-09-21T07:54:39.292025+0000","last_unstale":"2024-09-21T07:54:40.276505+0000","last_undegraded":"2024-09-21T07:54:40.276505+0000","last_fullsized":"2024-09-21T07:54:40.276505+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:36.341737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.611350+0000","last_change":"2024-09-21T07:54:39.311747+0000","last_active":"2024-09-21T07:54:40.611350+0000","last_peered":"2024-09-21T07:54:40.611350+0000","last_clean":"2024-09-21T07:54:40.611350+0000","last_became_active":"2024-09-21T07:54:39.311386+0000","last_became_peered":"2024-09-21T07:54:39.311386+0000","last_unstale":"2024-09-21T07:54:40.611350+0000","last_undegraded":"2024-09-21T07:54:40.611350+0000","last_fullsized":"2024-09-21T07:54:40.611350+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:04.183280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.279097+0000","last_change":"2024-09-21T07:54:39.314744+0000","last_active":"2024-09-21T07:54:40.279097+0000","last_peered":"2024-09-21T07:54:40.279097+0000","last_clean":"2024-09-21T07:54:40.279097+0000","last_became_active":"2024-09-21T07:54:39.306377+0000","last_became_peered":"2024-09-21T07:54:39.306377+0000","last_unstale":"2024-09-21T07:54:40.279097+0000","last_undegraded":"2024-09-21T07:54:40.279097+0000","last_fullsized":"2024-09-21T07:54:40.279097+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:37.435057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.278478+0000","last_change":"2024-09-21T07:54:39.306839+0000","last_active":"2024-09-21T07:54:40.278478+0000","last_peered":"2024-09-21T07:54:40.278478+0000","last_clean":"2024-09-21T07:54:40.278478+0000","last_became_active":"2024-09-21T07:54:39.306499+0000","last_became_peered":"2024-09-21T07:54:39.306499+0000","last_unstale":"2024-09-21T07:54:40.278478+0000","last_undegraded":"2024-09-21T07:54:40.278478+0000","last_fullsized":"2024-09-21T07:54:40.278478+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:11:31.207230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.609571+0000","last_change":"2024-09-21T07:54:39.301660+0000","last_active":"2024-09-21T07:54:40.609571+0000","last_peered":"2024-09-21T07:54:40.609571+0000","last_clean":"2024-09-21T07:54:40.609571+0000","last_became_active":"2024-09-21T07:54:39.301318+0000","last_became_peered":"2024-09-21T07:54:39.301318+0000","last_unstale":"2024-09-21T07:54:40.609571+0000","last_undegraded":"2024-09-21T07:54:40.609571+0000","last_fullsized":"2024-09-21T07:54:40.609571+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:06.819293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.610851+0000","last_change":"2024-09-21T07:54:39.311970+0000","last_active":"2024-09-21T07:54:40.610851+0000","last_peered":"2024-09-21T07:54:40.610851+0000","last_clean":"2024-09-21T07:54:40.610851+0000","last_became_active":"2024-09-21T07:54:39.310956+0000","last_became_peered":"2024-09-21T07:54:39.310956+0000","last_unstale":"2024-09-21T07:54:40.610851+0000","last_undegraded":"2024-09-21T07:54:40.610851+0000","last_fullsized":"2024-09-21T07:54:40.610851+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:55.356231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.607896+0000","last_change":"2024-09-21T07:54:39.293648+0000","last_active":"2024-09-21T07:54:40.607896+0000","last_peered":"2024-09-21T07:54:40.607896+0000","last_clean":"2024-09-21T07:54:40.607896+0000","last_became_active":"2024-09-21T07:54:39.291378+0000","last_became_peered":"2024-09-21T07:54:39.291378+0000","last_unstale":"2024-09-21T07:54:40.607896+0000","last_undegraded":"2024-09-21T07:54:40.607896+0000","last_fullsized":"2024-09-21T07:54:40.607896+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:29.745606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.279549+0000","last_change":"2024-09-21T07:54:39.300016+0000","last_active":"2024-09-21T07:54:40.279549+0000","last_peered":"2024-09-21T07:54:40.279549+0000","last_clean":"2024-09-21T07:54:40.279549+0000","last_became_active":"2024-09-21T07:54:39.299608+0000","last_became_peered":"2024-09-21T07:54:39.299608+0000","last_unstale":"2024-09-21T07:54:40.279549+0000","last_undegraded":"2024-09-21T07:54:40.279549+0000","last_fullsized":"2024-09-21T07:54:40.279549+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:56.645276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.279195+0000","last_change":"2024-09-21T07:54:39.299241+0000","last_active":"2024-09-21T07:54:40.279195+0000","last_peered":"2024-09-21T07:54:40.279195+0000","last_clean":"2024-09-21T07:54:40.279195+0000","last_became_active":"2024-09-21T07:54:39.298909+0000","last_became_peered":"2024-09-21T07:54:39.298909+0000","last_unstale":"2024-09-21T07:54:40.279195+0000","last_undegraded":"2024-09-21T07:54:40.279195+0000","last_fullsized":"2024-09-21T07:54:40.279195+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:18:37.875128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.611512+0000","last_change":"2024-09-21T07:54:39.293411+0000","last_active":"2024-09-21T07:54:40.611512+0000","last_peered":"2024-09-21T07:54:40.611512+0000","last_clean":"2024-09-21T07:54:40.611512+0000","last_became_active":"2024-09-21T07:54:39.292261+0000","last_became_peered":"2024-09-21T07:54:39.292261+0000","last_unstale":"2024-09-21T07:54:40.611512+0000","last_undegraded":"2024-09-21T07:54:40.611512+0000","last_fullsized":"2024-09-21T07:54:40.611512+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:00.526708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.576947+0000","last_change":"2024-09-21T07:54:39.310000+0000","last_active":"2024-09-21T07:54:40.576947+0000","last_peered":"2024-09-21T07:54:40.576947+0000","last_clean":"2024-09-21T07:54:40.576947+0000","last_became_active":"2024-09-21T07:54:39.307318+0000","last_became_peered":"2024-09-21T07:54:39.307318+0000","last_unstale":"2024-09-21T07:54:40.576947+0000","last_undegraded":"2024-09-21T07:54:40.576947+0000","last_fullsized":"2024-09-21T07:54:40.576947+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:02:43.076786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.607292+0000","last_change":"2024-09-21T07:54:39.304640+0000","last_active":"2024-09-21T07:54:40.607292+0000","last_peered":"2024-09-21T07:54:40.607292+0000","last_clean":"2024-09-21T07:54:40.607292+0000","last_became_active":"2024-09-21T07:54:39.304330+0000","last_became_peered":"2024-09-21T07:54:39.304330+0000","last_unstale":"2024-09-21T07:54:40.607292+0000","last_undegraded":"2024-09-21T07:54:40.607292+0000","last_fullsized":"2024-09-21T07:54:40.607292+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:55.356231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.607129+0000","last_change":"2024-09-21T07:54:39.305471+0000","last_active":"2024-09-21T07:54:40.607129+0000","last_peered":"2024-09-21T07:54:40.607129+0000","last_clean":"2024-09-21T07:54:40.607129+0000","last_became_active":"2024-09-21T07:54:39.304877+0000","last_became_peered":"2024-09-21T07:54:39.304877+0000","last_unstale":"2024-09-21T07:54:40.607129+0000","last_undegraded":"2024-09-21T07:54:40.607129+0000","last_fullsized":"2024-09-21T07:54:40.607129+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:01:39.749335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.278498+0000","last_change":"2024-09-21T07:54:39.314907+0000","last_active":"2024-09-21T07:54:40.278498+0000","last_peered":"2024-09-21T07:54:40.278498+0000","last_clean":"2024-09-21T07:54:40.278498+0000","last_became_active":"2024-09-21T07:54:39.314534+0000","last_became_peered":"2024-09-21T07:54:39.314534+0000","last_unstale":"2024-09-21T07:54:40.278498+0000","last_undegraded":"2024-09-21T07:54:40.278498+0000","last_fullsized":"2024-09-21T07:54:40.278498+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:09:43.367330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.576561+0000","last_change":"2024-09-21T07:54:39.309549+0000","last_active":"2024-09-21T07:54:40.576561+0000","last_peered":"2024-09-21T07:54:40.576561+0000","last_clean":"2024-09-21T07:54:40.576561+0000","last_became_active":"2024-09-21T07:54:39.309223+0000","last_became_peered":"2024-09-21T07:54:39.309223+0000","last_unstale":"2024-09-21T07:54:40.576561+0000","last_undegraded":"2024-09-21T07:54:40.576561+0000","last_fullsized":"2024-09-21T07:54:40.576561+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:17:11.002037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.576068+0000","last_change":"2024-09-21T07:54:39.307331+0000","last_active":"2024-09-21T07:54:40.576068+0000","last_peered":"2024-09-21T07:54:40.576068+0000","last_clean":"2024-09-21T07:54:40.576068+0000","last_became_active":"2024-09-21T07:54:39.306929+0000","last_became_peered":"2024-09-21T07:54:39.306929+0000","last_unstale":"2024-09-21T07:54:40.576068+0000","last_undegraded":"2024-09-21T07:54:40.576068+0000","last_fullsized":"2024-09-21T07:54:40.576068+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:01:26.853678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"264'1","reported_seq":16,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.279281+0000","last_change":"2024-09-21T07:54:38.289438+0000","last_active":"2024-09-21T07:54:40.279281+0000","last_peered":"2024-09-21T07:54:40.279281+0000","last_clean":"2024-09-21T07:54:40.279281+0000","last_became_active":"2024-09-21T07:54:38.289089+0000","last_became_peered":"2024-09-21T07:54:38.289089+0000","last_unstale":"2024-09-21T07:54:40.279281+0000","last_undegraded":"2024-09-21T07:54:40.279281+0000","last_fullsized":"2024-09-21T07:54:40.279281+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:45.528995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":395,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.610955+0000","last_change":"2024-09-21T07:35:27.879605+0000","last_active":"2024-09-21T07:54:40.610955+0000","last_peered":"2024-09-21T07:54:40.610955+0000","last_clean":"2024-09-21T07:54:40.610955+0000","last_became_active":"2024-09-21T07:35:27.879274+0000","last_became_peered":"2024-09-21T07:35:27.879274+0000","last_unstale":"2024-09-21T07:54:40.610955+0000","last_undegraded":"2024-09-21T07:54:40.610955+0000","last_fullsized":"2024-09-21T07:54:40.610955+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:24.241636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":404,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.576369+0000","last_change":"2024-09-21T07:35:29.491336+0000","last_active":"2024-09-21T07:54:40.576369+0000","last_peered":"2024-09-21T07:54:40.576369+0000","last_clean":"2024-09-21T07:54:40.576369+0000","last_became_active":"2024-09-21T07:35:26.866484+0000","last_became_peered":"2024-09-21T07:35:26.866484+0000","last_unstale":"2024-09-21T07:54:40.576369+0000","last_undegraded":"2024-09-21T07:54:40.576369+0000","last_fullsized":"2024-09-21T07:54:40.576369+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:10.449665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076010100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":581,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.609696+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:54:40.609696+0000","last_peered":"2024-09-21T07:54:40.609696+0000","last_clean":"2024-09-21T07:54:40.609696+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:54:40.609696+0000","last_undegraded":"2024-09-21T07:54:40.609696+0000","last_fullsized":"2024-09-21T07:54:40.609696+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":472,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.279010+0000","last_change":"2024-09-21T07:34:24.539248+0000","last_active":"2024-09-21T07:54:40.279010+0000","last_peered":"2024-09-21T07:54:40.279010+0000","last_clean":"2024-09-21T07:54:40.279010+0000","last_became_active":"2024-09-21T07:34:23.273465+0000","last_became_peered":"2024-09-21T07:34:23.273465+0000","last_unstale":"2024-09-21T07:54:40.279010+0000","last_undegraded":"2024-09-21T07:54:40.279010+0000","last_fullsized":"2024-09-21T07:54:40.279010+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:55.517452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060627700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.607619+0000","last_change":"2024-09-21T07:54:39.308317+0000","last_active":"2024-09-21T07:54:40.607619+0000","last_peered":"2024-09-21T07:54:40.607619+0000","last_clean":"2024-09-21T07:54:40.607619+0000","last_became_active":"2024-09-21T07:54:39.303047+0000","last_became_peered":"2024-09-21T07:54:39.303047+0000","last_unstale":"2024-09-21T07:54:40.607619+0000","last_undegraded":"2024-09-21T07:54:40.607619+0000","last_fullsized":"2024-09-21T07:54:40.607619+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:23.733083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1b","version":"264'1","reported_seq":16,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.608945+0000","last_change":"2024-09-21T07:54:38.293823+0000","last_active":"2024-09-21T07:54:40.608945+0000","last_peered":"2024-09-21T07:54:40.608945+0000","last_clean":"2024-09-21T07:54:40.608945+0000","last_became_active":"2024-09-21T07:54:38.292081+0000","last_became_peered":"2024-09-21T07:54:38.292081+0000","last_unstale":"2024-09-21T07:54:40.608945+0000","last_undegraded":"2024-09-21T07:54:40.608945+0000","last_fullsized":"2024-09-21T07:54:40.608945+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:20:01.647695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":398,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.576701+0000","last_change":"2024-09-21T07:35:27.864492+0000","last_active":"2024-09-21T07:54:40.576701+0000","last_peered":"2024-09-21T07:54:40.576701+0000","last_clean":"2024-09-21T07:54:40.576701+0000","last_became_active":"2024-09-21T07:35:27.864176+0000","last_became_peered":"2024-09-21T07:35:27.864176+0000","last_unstale":"2024-09-21T07:54:40.576701+0000","last_undegraded":"2024-09-21T07:54:40.576701+0000","last_fullsized":"2024-09-21T07:54:40.576701+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:05.153467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":402,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.611778+0000","last_change":"2024-09-21T07:35:29.493678+0000","last_active":"2024-09-21T07:54:40.611778+0000","last_peered":"2024-09-21T07:54:40.611778+0000","last_clean":"2024-09-21T07:54:40.611778+0000","last_became_active":"2024-09-21T07:35:26.870749+0000","last_became_peered":"2024-09-21T07:35:26.870749+0000","last_unstale":"2024-09-21T07:54:40.611778+0000","last_undegraded":"2024-09-21T07:54:40.611778+0000","last_fullsized":"2024-09-21T07:54:40.611778+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:55.367877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010159570000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":448,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.610370+0000","last_change":"2024-09-21T07:34:24.597274+0000","last_active":"2024-09-21T07:54:40.610370+0000","last_peered":"2024-09-21T07:54:40.610370+0000","last_clean":"2024-09-21T07:54:40.610370+0000","last_became_active":"2024-09-21T07:34:23.279560+0000","last_became_peered":"2024-09-21T07:34:23.279560+0000","last_unstale":"2024-09-21T07:54:40.610370+0000","last_undegraded":"2024-09-21T07:54:40.610370+0000","last_fullsized":"2024-09-21T07:54:40.610370+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:15.845265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059699799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.279239+0000","last_change":"2024-09-21T07:54:39.305924+0000","last_active":"2024-09-21T07:54:40.279239+0000","last_peered":"2024-09-21T07:54:40.279239+0000","last_clean":"2024-09-21T07:54:40.279239+0000","last_became_active":"2024-09-21T07:54:39.305577+0000","last_became_peered":"2024-09-21T07:54:39.305577+0000","last_unstale":"2024-09-21T07:54:40.279239+0000","last_undegraded":"2024-09-21T07:54:40.279239+0000","last_fullsized":"2024-09-21T07:54:40.279239+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:18.375464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1a","version":"264'1","reported_seq":16,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.607216+0000","last_change":"2024-09-21T07:54:38.297279+0000","last_active":"2024-09-21T07:54:40.607216+0000","last_peered":"2024-09-21T07:54:40.607216+0000","last_clean":"2024-09-21T07:54:40.607216+0000","last_became_active":"2024-09-21T07:54:38.291494+0000","last_became_peered":"2024-09-21T07:54:38.291494+0000","last_unstale":"2024-09-21T07:54:40.607216+0000","last_undegraded":"2024-09-21T07:54:40.607216+0000","last_fullsized":"2024-09-21T07:54:40.607216+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:20:01.647695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":398,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.278425+0000","last_change":"2024-09-21T07:35:27.874387+0000","last_active":"2024-09-21T07:54:40.278425+0000","last_peered":"2024-09-21T07:54:40.278425+0000","last_clean":"2024-09-21T07:54:40.278425+0000","last_became_active":"2024-09-21T07:35:27.873277+0000","last_became_peered":"2024-09-21T07:35:27.873277+0000","last_unstale":"2024-09-21T07:54:40.278425+0000","last_undegraded":"2024-09-21T07:54:40.278425+0000","last_fullsized":"2024-09-21T07:54:40.278425+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:04.581463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":398,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.608866+0000","last_change":"2024-09-21T07:35:29.387094+0000","last_active":"2024-09-21T07:54:40.608866+0000","last_peered":"2024-09-21T07:54:40.608866+0000","last_clean":"2024-09-21T07:54:40.608866+0000","last_became_active":"2024-09-21T07:35:26.874757+0000","last_became_peered":"2024-09-21T07:35:26.874757+0000","last_unstale":"2024-09-21T07:54:40.608866+0000","last_undegraded":"2024-09-21T07:54:40.608866+0000","last_fullsized":"2024-09-21T07:54:40.608866+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:33.542750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061065700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":455,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.610335+0000","last_change":"2024-09-21T07:34:24.259906+0000","last_active":"2024-09-21T07:54:40.610335+0000","last_peered":"2024-09-21T07:54:40.610335+0000","last_clean":"2024-09-21T07:54:40.610335+0000","last_became_active":"2024-09-21T07:34:23.635282+0000","last_became_peered":"2024-09-21T07:34:23.635282+0000","last_unstale":"2024-09-21T07:54:40.610335+0000","last_undegraded":"2024-09-21T07:54:40.610335+0000","last_fullsized":"2024-09-21T07:54:40.610335+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:00.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001527664,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.276236+0000","last_change":"2024-09-21T07:54:39.301243+0000","last_active":"2024-09-21T07:54:40.276236+0000","last_peered":"2024-09-21T07:54:40.276236+0000","last_clean":"2024-09-21T07:54:40.276236+0000","last_became_active":"2024-09-21T07:54:39.300398+0000","last_became_peered":"2024-09-21T07:54:39.300398+0000","last_unstale":"2024-09-21T07:54:40.276236+0000","last_undegraded":"2024-09-21T07:54:40.276236+0000","last_fullsized":"2024-09-21T07:54:40.276236+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:56.374935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":15,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.275269+0000","last_change":"2024-09-21T07:54:38.282109+0000","last_active":"2024-09-21T07:54:40.275269+0000","last_peered":"2024-09-21T07:54:40.275269+0000","last_clean":"2024-09-21T07:54:40.275269+0000","last_became_active":"2024-09-21T07:54:38.281784+0000","last_became_peered":"2024-09-21T07:54:38.281784+0000","last_unstale":"2024-09-21T07:54:40.275269+0000","last_undegraded":"2024-09-21T07:54:40.275269+0000","last_fullsized":"2024-09-21T07:54:40.275269+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:32.416735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":398,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.278726+0000","last_change":"2024-09-21T07:35:27.873645+0000","last_active":"2024-09-21T07:54:40.278726+0000","last_peered":"2024-09-21T07:54:40.278726+0000","last_clean":"2024-09-21T07:54:40.278726+0000","last_became_active":"2024-09-21T07:35:27.873245+0000","last_became_peered":"2024-09-21T07:35:27.873245+0000","last_unstale":"2024-09-21T07:54:40.278726+0000","last_undegraded":"2024-09-21T07:54:40.278726+0000","last_fullsized":"2024-09-21T07:54:40.278726+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:55.090953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":405,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.279032+0000","last_change":"2024-09-21T07:35:28.861744+0000","last_active":"2024-09-21T07:54:40.279032+0000","last_peered":"2024-09-21T07:54:40.279032+0000","last_clean":"2024-09-21T07:54:40.279032+0000","last_became_active":"2024-09-21T07:35:26.876462+0000","last_became_peered":"2024-09-21T07:35:26.876462+0000","last_unstale":"2024-09-21T07:54:40.279032+0000","last_undegraded":"2024-09-21T07:54:40.279032+0000","last_fullsized":"2024-09-21T07:54:40.279032+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:48.632568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012020920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":456,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.607042+0000","last_change":"2024-09-21T07:34:25.085454+0000","last_active":"2024-09-21T07:54:40.607042+0000","last_peered":"2024-09-21T07:54:40.607042+0000","last_clean":"2024-09-21T07:54:40.607042+0000","last_became_active":"2024-09-21T07:34:23.279205+0000","last_became_peered":"2024-09-21T07:34:23.279205+0000","last_unstale":"2024-09-21T07:54:40.607042+0000","last_undegraded":"2024-09-21T07:54:40.607042+0000","last_fullsized":"2024-09-21T07:54:40.607042+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:54.231666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095320599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.608457+0000","last_change":"2024-09-21T07:54:39.298473+0000","last_active":"2024-09-21T07:54:40.608457+0000","last_peered":"2024-09-21T07:54:40.608457+0000","last_clean":"2024-09-21T07:54:40.608457+0000","last_became_active":"2024-09-21T07:54:39.296833+0000","last_became_peered":"2024-09-21T07:54:39.296833+0000","last_unstale":"2024-09-21T07:54:40.608457+0000","last_undegraded":"2024-09-21T07:54:40.608457+0000","last_fullsized":"2024-09-21T07:54:40.608457+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:20:52.886679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":15,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.575922+0000","last_change":"2024-09-21T07:54:38.295170+0000","last_active":"2024-09-21T07:54:40.575922+0000","last_peered":"2024-09-21T07:54:40.575922+0000","last_clean":"2024-09-21T07:54:40.575922+0000","last_became_active":"2024-09-21T07:54:38.294876+0000","last_became_peered":"2024-09-21T07:54:38.294876+0000","last_unstale":"2024-09-21T07:54:40.575922+0000","last_undegraded":"2024-09-21T07:54:40.575922+0000","last_fullsized":"2024-09-21T07:54:40.575922+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:36:36.419308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":397,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.576587+0000","last_change":"2024-09-21T07:35:27.861373+0000","last_active":"2024-09-21T07:54:40.576587+0000","last_peered":"2024-09-21T07:54:40.576587+0000","last_clean":"2024-09-21T07:54:40.576587+0000","last_became_active":"2024-09-21T07:35:27.861016+0000","last_became_peered":"2024-09-21T07:35:27.861016+0000","last_unstale":"2024-09-21T07:54:40.576587+0000","last_undegraded":"2024-09-21T07:54:40.576587+0000","last_fullsized":"2024-09-21T07:54:40.576587+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:56.043763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":405,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.275918+0000","last_change":"2024-09-21T07:35:29.053677+0000","last_active":"2024-09-21T07:54:40.275918+0000","last_peered":"2024-09-21T07:54:40.275918+0000","last_clean":"2024-09-21T07:54:40.275918+0000","last_became_active":"2024-09-21T07:35:26.875496+0000","last_became_peered":"2024-09-21T07:35:26.875496+0000","last_unstale":"2024-09-21T07:54:40.275918+0000","last_undegraded":"2024-09-21T07:54:40.275918+0000","last_fullsized":"2024-09-21T07:54:40.275918+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:04.850784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065859000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":457,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.275475+0000","last_change":"2024-09-21T07:34:25.028521+0000","last_active":"2024-09-21T07:54:40.275475+0000","last_peered":"2024-09-21T07:54:40.275475+0000","last_clean":"2024-09-21T07:54:40.275475+0000","last_became_active":"2024-09-21T07:34:23.635467+0000","last_became_peered":"2024-09-21T07:34:23.635467+0000","last_unstale":"2024-09-21T07:54:40.275475+0000","last_undegraded":"2024-09-21T07:54:40.275475+0000","last_fullsized":"2024-09-21T07:54:40.275475+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:33.729123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000660558,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.278334+0000","last_change":"2024-09-21T07:54:39.303549+0000","last_active":"2024-09-21T07:54:40.278334+0000","last_peered":"2024-09-21T07:54:40.278334+0000","last_clean":"2024-09-21T07:54:40.278334+0000","last_became_active":"2024-09-21T07:54:39.303130+0000","last_became_peered":"2024-09-21T07:54:39.303130+0000","last_unstale":"2024-09-21T07:54:40.278334+0000","last_undegraded":"2024-09-21T07:54:40.278334+0000","last_fullsized":"2024-09-21T07:54:40.278334+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:25:41.997333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1f","version":"264'4","reported_seq":19,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.576853+0000","last_change":"2024-09-21T07:54:38.293144+0000","last_active":"2024-09-21T07:54:40.576853+0000","last_peered":"2024-09-21T07:54:40.576853+0000","last_clean":"2024-09-21T07:54:40.576853+0000","last_became_active":"2024-09-21T07:54:38.292073+0000","last_became_peered":"2024-09-21T07:54:38.292073+0000","last_unstale":"2024-09-21T07:54:40.576853+0000","last_undegraded":"2024-09-21T07:54:40.576853+0000","last_fullsized":"2024-09-21T07:54:40.576853+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:46:02.586538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":397,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.607849+0000","last_change":"2024-09-21T07:35:27.871201+0000","last_active":"2024-09-21T07:54:40.607849+0000","last_peered":"2024-09-21T07:54:40.607849+0000","last_clean":"2024-09-21T07:54:40.607849+0000","last_became_active":"2024-09-21T07:35:27.870816+0000","last_became_peered":"2024-09-21T07:35:27.870816+0000","last_unstale":"2024-09-21T07:54:40.607849+0000","last_undegraded":"2024-09-21T07:54:40.607849+0000","last_fullsized":"2024-09-21T07:54:40.607849+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:04.787450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":404,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.577350+0000","last_change":"2024-09-21T07:35:29.491874+0000","last_active":"2024-09-21T07:54:40.577350+0000","last_peered":"2024-09-21T07:54:40.577350+0000","last_clean":"2024-09-21T07:54:40.577350+0000","last_became_active":"2024-09-21T07:35:26.866381+0000","last_became_peered":"2024-09-21T07:35:26.866381+0000","last_unstale":"2024-09-21T07:54:40.577350+0000","last_undegraded":"2024-09-21T07:54:40.577350+0000","last_fullsized":"2024-09-21T07:54:40.577350+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:36.404647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073235399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":460,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.576793+0000","last_change":"2024-09-21T07:34:24.260046+0000","last_active":"2024-09-21T07:54:40.576793+0000","last_peered":"2024-09-21T07:54:40.576793+0000","last_clean":"2024-09-21T07:54:40.576793+0000","last_became_active":"2024-09-21T07:34:23.273431+0000","last_became_peered":"2024-09-21T07:34:23.273431+0000","last_unstale":"2024-09-21T07:54:40.576793+0000","last_undegraded":"2024-09-21T07:54:40.576793+0000","last_fullsized":"2024-09-21T07:54:40.576793+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:18.876057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019803310000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.610091+0000","last_change":"2024-09-21T07:54:39.315001+0000","last_active":"2024-09-21T07:54:40.610091+0000","last_peered":"2024-09-21T07:54:40.610091+0000","last_clean":"2024-09-21T07:54:40.610091+0000","last_became_active":"2024-09-21T07:54:39.314170+0000","last_became_peered":"2024-09-21T07:54:39.314170+0000","last_unstale":"2024-09-21T07:54:40.610091+0000","last_undegraded":"2024-09-21T07:54:40.610091+0000","last_fullsized":"2024-09-21T07:54:40.610091+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:01:39.749335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.1d","version":"264'2","reported_seq":17,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.608681+0000","last_change":"2024-09-21T07:54:38.301636+0000","last_active":"2024-09-21T07:54:40.608681+0000","last_peered":"2024-09-21T07:54:40.608681+0000","last_clean":"2024-09-21T07:54:40.608681+0000","last_became_active":"2024-09-21T07:54:38.301255+0000","last_became_peered":"2024-09-21T07:54:38.301255+0000","last_unstale":"2024-09-21T07:54:40.608681+0000","last_undegraded":"2024-09-21T07:54:40.608681+0000","last_fullsized":"2024-09-21T07:54:40.608681+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:58:44.038214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":397,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.607745+0000","last_change":"2024-09-21T07:35:27.871736+0000","last_active":"2024-09-21T07:54:40.607745+0000","last_peered":"2024-09-21T07:54:40.607745+0000","last_clean":"2024-09-21T07:54:40.607745+0000","last_became_active":"2024-09-21T07:35:27.871232+0000","last_became_peered":"2024-09-21T07:35:27.871232+0000","last_unstale":"2024-09-21T07:54:40.607745+0000","last_undegraded":"2024-09-21T07:54:40.607745+0000","last_fullsized":"2024-09-21T07:54:40.607745+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:04.419301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":405,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.576197+0000","last_change":"2024-09-21T07:35:29.491246+0000","last_active":"2024-09-21T07:54:40.576197+0000","last_peered":"2024-09-21T07:54:40.576197+0000","last_clean":"2024-09-21T07:54:40.576197+0000","last_became_active":"2024-09-21T07:35:26.866783+0000","last_became_peered":"2024-09-21T07:35:26.866783+0000","last_unstale":"2024-09-21T07:54:40.576197+0000","last_undegraded":"2024-09-21T07:54:40.576197+0000","last_fullsized":"2024-09-21T07:54:40.576197+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:28.881914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098474099999999992,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":457,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.276031+0000","last_change":"2024-09-21T07:34:25.028640+0000","last_active":"2024-09-21T07:54:40.276031+0000","last_peered":"2024-09-21T07:54:40.276031+0000","last_clean":"2024-09-21T07:54:40.276031+0000","last_became_active":"2024-09-21T07:34:23.260950+0000","last_became_peered":"2024-09-21T07:34:23.260950+0000","last_unstale":"2024-09-21T07:54:40.276031+0000","last_undegraded":"2024-09-21T07:54:40.276031+0000","last_fullsized":"2024-09-21T07:54:40.276031+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:23:42.121398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055135099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.577301+0000","last_change":"2024-09-21T07:54:39.319086+0000","last_active":"2024-09-21T07:54:40.577301+0000","last_peered":"2024-09-21T07:54:40.577301+0000","last_clean":"2024-09-21T07:54:40.577301+0000","last_became_active":"2024-09-21T07:54:39.318793+0000","last_became_peered":"2024-09-21T07:54:39.318793+0000","last_unstale":"2024-09-21T07:54:40.577301+0000","last_undegraded":"2024-09-21T07:54:40.577301+0000","last_fullsized":"2024-09-21T07:54:40.577301+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:35:55.662137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1c","version":"264'2","reported_seq":17,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.607690+0000","last_change":"2024-09-21T07:54:38.294951+0000","last_active":"2024-09-21T07:54:40.607690+0000","last_peered":"2024-09-21T07:54:40.607690+0000","last_clean":"2024-09-21T07:54:40.607690+0000","last_became_active":"2024-09-21T07:54:38.294646+0000","last_became_peered":"2024-09-21T07:54:38.294646+0000","last_unstale":"2024-09-21T07:54:40.607690+0000","last_undegraded":"2024-09-21T07:54:40.607690+0000","last_fullsized":"2024-09-21T07:54:40.607690+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:40.961515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":400,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.609344+0000","last_change":"2024-09-21T07:35:27.872027+0000","last_active":"2024-09-21T07:54:40.609344+0000","last_peered":"2024-09-21T07:54:40.609344+0000","last_clean":"2024-09-21T07:54:40.609344+0000","last_became_active":"2024-09-21T07:35:27.870426+0000","last_became_peered":"2024-09-21T07:35:27.870426+0000","last_unstale":"2024-09-21T07:54:40.609344+0000","last_undegraded":"2024-09-21T07:54:40.609344+0000","last_fullsized":"2024-09-21T07:54:40.609344+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":405,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.576088+0000","last_change":"2024-09-21T07:35:29.491008+0000","last_active":"2024-09-21T07:54:40.576088+0000","last_peered":"2024-09-21T07:54:40.576088+0000","last_clean":"2024-09-21T07:54:40.576088+0000","last_became_active":"2024-09-21T07:35:26.866750+0000","last_became_peered":"2024-09-21T07:35:26.866750+0000","last_unstale":"2024-09-21T07:54:40.576088+0000","last_undegraded":"2024-09-21T07:54:40.576088+0000","last_fullsized":"2024-09-21T07:54:40.576088+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:25.490406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083529700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":456,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.611505+0000","last_change":"2024-09-21T07:34:24.260665+0000","last_active":"2024-09-21T07:54:40.611505+0000","last_peered":"2024-09-21T07:54:40.611505+0000","last_clean":"2024-09-21T07:54:40.611505+0000","last_became_active":"2024-09-21T07:34:23.635909+0000","last_became_peered":"2024-09-21T07:34:23.635909+0000","last_unstale":"2024-09-21T07:54:40.611505+0000","last_undegraded":"2024-09-21T07:54:40.611505+0000","last_fullsized":"2024-09-21T07:54:40.611505+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:44.536885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.278859+0000","last_change":"2024-09-21T07:54:39.299848+0000","last_active":"2024-09-21T07:54:40.278859+0000","last_peered":"2024-09-21T07:54:40.278859+0000","last_clean":"2024-09-21T07:54:40.278859+0000","last_became_active":"2024-09-21T07:54:39.299449+0000","last_became_peered":"2024-09-21T07:54:39.299449+0000","last_unstale":"2024-09-21T07:54:40.278859+0000","last_undegraded":"2024-09-21T07:54:40.278859+0000","last_fullsized":"2024-09-21T07:54:40.278859+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:31.147400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.13","version":"264'2","reported_seq":17,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.609076+0000","last_change":"2024-09-21T07:54:38.297909+0000","last_active":"2024-09-21T07:54:40.609076+0000","last_peered":"2024-09-21T07:54:40.609076+0000","last_clean":"2024-09-21T07:54:40.609076+0000","last_became_active":"2024-09-21T07:54:38.297460+0000","last_became_peered":"2024-09-21T07:54:38.297460+0000","last_unstale":"2024-09-21T07:54:40.609076+0000","last_undegraded":"2024-09-21T07:54:40.609076+0000","last_fullsized":"2024-09-21T07:54:40.609076+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:40.961515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":392,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.608119+0000","last_change":"2024-09-21T07:35:27.874772+0000","last_active":"2024-09-21T07:54:40.608119+0000","last_peered":"2024-09-21T07:54:40.608119+0000","last_clean":"2024-09-21T07:54:40.608119+0000","last_became_active":"2024-09-21T07:35:27.874434+0000","last_became_peered":"2024-09-21T07:35:27.874434+0000","last_unstale":"2024-09-21T07:54:40.608119+0000","last_undegraded":"2024-09-21T07:54:40.608119+0000","last_fullsized":"2024-09-21T07:54:40.608119+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:31.998755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":419,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.278973+0000","last_change":"2024-09-21T07:35:28.859153+0000","last_active":"2024-09-21T07:54:40.278973+0000","last_peered":"2024-09-21T07:54:40.278973+0000","last_clean":"2024-09-21T07:54:40.278973+0000","last_became_active":"2024-09-21T07:35:26.874428+0000","last_became_peered":"2024-09-21T07:35:26.874428+0000","last_unstale":"2024-09-21T07:54:40.278973+0000","last_undegraded":"2024-09-21T07:54:40.278973+0000","last_fullsized":"2024-09-21T07:54:40.278973+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:04.487621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048557299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":472,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.278211+0000","last_change":"2024-09-21T07:34:24.538845+0000","last_active":"2024-09-21T07:54:40.278211+0000","last_peered":"2024-09-21T07:54:40.278211+0000","last_clean":"2024-09-21T07:54:40.278211+0000","last_became_active":"2024-09-21T07:34:23.636372+0000","last_became_peered":"2024-09-21T07:34:23.636372+0000","last_unstale":"2024-09-21T07:54:40.278211+0000","last_undegraded":"2024-09-21T07:54:40.278211+0000","last_fullsized":"2024-09-21T07:54:40.278211+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:29.393813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841451,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.278992+0000","last_change":"2024-09-21T07:54:39.292835+0000","last_active":"2024-09-21T07:54:40.278992+0000","last_peered":"2024-09-21T07:54:40.278992+0000","last_clean":"2024-09-21T07:54:40.278992+0000","last_became_active":"2024-09-21T07:54:39.292437+0000","last_became_peered":"2024-09-21T07:54:39.292437+0000","last_unstale":"2024-09-21T07:54:40.278992+0000","last_undegraded":"2024-09-21T07:54:40.278992+0000","last_fullsized":"2024-09-21T07:54:40.278992+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:26:47.084476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":15,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.608769+0000","last_change":"2024-09-21T07:54:38.293728+0000","last_active":"2024-09-21T07:54:40.608769+0000","last_peered":"2024-09-21T07:54:40.608769+0000","last_clean":"2024-09-21T07:54:40.608769+0000","last_became_active":"2024-09-21T07:54:38.292594+0000","last_became_peered":"2024-09-21T07:54:38.292594+0000","last_unstale":"2024-09-21T07:54:40.608769+0000","last_undegraded":"2024-09-21T07:54:40.608769+0000","last_fullsized":"2024-09-21T07:54:40.608769+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:23.891425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":401,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.609869+0000","last_change":"2024-09-21T07:35:27.877801+0000","last_active":"2024-09-21T07:54:40.609869+0000","last_peered":"2024-09-21T07:54:40.609869+0000","last_clean":"2024-09-21T07:54:40.609869+0000","last_became_active":"2024-09-21T07:35:27.877464+0000","last_became_peered":"2024-09-21T07:35:27.877464+0000","last_unstale":"2024-09-21T07:54:40.609869+0000","last_undegraded":"2024-09-21T07:54:40.609869+0000","last_fullsized":"2024-09-21T07:54:40.609869+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:08.814929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":404,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.607890+0000","last_change":"2024-09-21T07:35:28.863732+0000","last_active":"2024-09-21T07:54:40.607890+0000","last_peered":"2024-09-21T07:54:40.607890+0000","last_clean":"2024-09-21T07:54:40.607890+0000","last_became_active":"2024-09-21T07:35:26.877392+0000","last_became_peered":"2024-09-21T07:35:26.877392+0000","last_unstale":"2024-09-21T07:54:40.607890+0000","last_undegraded":"2024-09-21T07:54:40.607890+0000","last_fullsized":"2024-09-21T07:54:40.607890+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071425700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":459,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.577376+0000","last_change":"2024-09-21T07:34:24.259686+0000","last_active":"2024-09-21T07:54:40.577376+0000","last_peered":"2024-09-21T07:54:40.577376+0000","last_clean":"2024-09-21T07:54:40.577376+0000","last_became_active":"2024-09-21T07:34:23.274405+0000","last_became_peered":"2024-09-21T07:34:23.274405+0000","last_unstale":"2024-09-21T07:54:40.577376+0000","last_undegraded":"2024-09-21T07:54:40.577376+0000","last_fullsized":"2024-09-21T07:54:40.577376+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:26.304693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022009920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.609588+0000","last_change":"2024-09-21T07:54:39.303233+0000","last_active":"2024-09-21T07:54:40.609588+0000","last_peered":"2024-09-21T07:54:40.609588+0000","last_clean":"2024-09-21T07:54:40.609588+0000","last_became_active":"2024-09-21T07:54:39.302505+0000","last_became_peered":"2024-09-21T07:54:39.302505+0000","last_unstale":"2024-09-21T07:54:40.609588+0000","last_undegraded":"2024-09-21T07:54:40.609588+0000","last_fullsized":"2024-09-21T07:54:40.609588+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:17:48.459911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":15,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.609694+0000","last_change":"2024-09-21T07:54:38.297444+0000","last_active":"2024-09-21T07:54:40.609694+0000","last_peered":"2024-09-21T07:54:40.609694+0000","last_clean":"2024-09-21T07:54:40.609694+0000","last_became_active":"2024-09-21T07:54:38.297074+0000","last_became_peered":"2024-09-21T07:54:38.297074+0000","last_unstale":"2024-09-21T07:54:40.609694+0000","last_undegraded":"2024-09-21T07:54:40.609694+0000","last_fullsized":"2024-09-21T07:54:40.609694+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:29:53.971323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":392,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.610246+0000","last_change":"2024-09-21T07:35:27.867336+0000","last_active":"2024-09-21T07:54:40.610246+0000","last_peered":"2024-09-21T07:54:40.610246+0000","last_clean":"2024-09-21T07:54:40.610246+0000","last_became_active":"2024-09-21T07:35:27.867015+0000","last_became_peered":"2024-09-21T07:35:27.867015+0000","last_unstale":"2024-09-21T07:54:40.610246+0000","last_undegraded":"2024-09-21T07:54:40.610246+0000","last_fullsized":"2024-09-21T07:54:40.610246+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:59.214139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":403,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.611064+0000","last_change":"2024-09-21T07:35:29.493137+0000","last_active":"2024-09-21T07:54:40.611064+0000","last_peered":"2024-09-21T07:54:40.611064+0000","last_clean":"2024-09-21T07:54:40.611064+0000","last_became_active":"2024-09-21T07:35:26.872230+0000","last_became_peered":"2024-09-21T07:35:26.872230+0000","last_unstale":"2024-09-21T07:54:40.611064+0000","last_undegraded":"2024-09-21T07:54:40.611064+0000","last_fullsized":"2024-09-21T07:54:40.611064+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:24.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001197449,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":456,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.609826+0000","last_change":"2024-09-21T07:34:24.258754+0000","last_active":"2024-09-21T07:54:40.609826+0000","last_peered":"2024-09-21T07:54:40.609826+0000","last_clean":"2024-09-21T07:54:40.609826+0000","last_became_active":"2024-09-21T07:34:23.272657+0000","last_became_peered":"2024-09-21T07:34:23.272657+0000","last_unstale":"2024-09-21T07:54:40.609826+0000","last_undegraded":"2024-09-21T07:54:40.609826+0000","last_fullsized":"2024-09-21T07:54:40.609826+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:05.477099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067143200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.608749+0000","last_change":"2024-09-21T07:54:39.309368+0000","last_active":"2024-09-21T07:54:40.608749+0000","last_peered":"2024-09-21T07:54:40.608749+0000","last_clean":"2024-09-21T07:54:40.608749+0000","last_became_active":"2024-09-21T07:54:39.308997+0000","last_became_peered":"2024-09-21T07:54:39.308997+0000","last_unstale":"2024-09-21T07:54:40.608749+0000","last_undegraded":"2024-09-21T07:54:40.608749+0000","last_fullsized":"2024-09-21T07:54:40.608749+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:04.054287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"264'8","reported_seq":23,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.278241+0000","last_change":"2024-09-21T07:54:38.282575+0000","last_active":"2024-09-21T07:54:40.278241+0000","last_peered":"2024-09-21T07:54:40.278241+0000","last_clean":"2024-09-21T07:54:40.278241+0000","last_became_active":"2024-09-21T07:54:38.282278+0000","last_became_peered":"2024-09-21T07:54:38.282278+0000","last_unstale":"2024-09-21T07:54:40.278241+0000","last_undegraded":"2024-09-21T07:54:40.278241+0000","last_fullsized":"2024-09-21T07:54:40.278241+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:03:11.392443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":398,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.607288+0000","last_change":"2024-09-21T07:35:27.876021+0000","last_active":"2024-09-21T07:54:40.607288+0000","last_peered":"2024-09-21T07:54:40.607288+0000","last_clean":"2024-09-21T07:54:40.607288+0000","last_became_active":"2024-09-21T07:35:27.875724+0000","last_became_peered":"2024-09-21T07:35:27.875724+0000","last_unstale":"2024-09-21T07:54:40.607288+0000","last_undegraded":"2024-09-21T07:54:40.607288+0000","last_fullsized":"2024-09-21T07:54:40.607288+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:03.022812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":403,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.610247+0000","last_change":"2024-09-21T07:35:29.492570+0000","last_active":"2024-09-21T07:54:40.610247+0000","last_peered":"2024-09-21T07:54:40.610247+0000","last_clean":"2024-09-21T07:54:40.610247+0000","last_became_active":"2024-09-21T07:35:26.874316+0000","last_became_peered":"2024-09-21T07:35:26.874316+0000","last_unstale":"2024-09-21T07:54:40.610247+0000","last_undegraded":"2024-09-21T07:54:40.610247+0000","last_fullsized":"2024-09-21T07:54:40.610247+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:16.512323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011990760000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":456,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.607212+0000","last_change":"2024-09-21T07:34:25.085232+0000","last_active":"2024-09-21T07:54:40.607212+0000","last_peered":"2024-09-21T07:54:40.607212+0000","last_clean":"2024-09-21T07:54:40.607212+0000","last_became_active":"2024-09-21T07:34:23.277807+0000","last_became_peered":"2024-09-21T07:34:23.277807+0000","last_unstale":"2024-09-21T07:54:40.607212+0000","last_undegraded":"2024-09-21T07:54:40.607212+0000","last_fullsized":"2024-09-21T07:54:40.607212+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:36.362919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086619099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.278464+0000","last_change":"2024-09-21T07:54:39.308517+0000","last_active":"2024-09-21T07:54:40.278464+0000","last_peered":"2024-09-21T07:54:40.278464+0000","last_clean":"2024-09-21T07:54:40.278464+0000","last_became_active":"2024-09-21T07:54:39.307697+0000","last_became_peered":"2024-09-21T07:54:39.307697+0000","last_unstale":"2024-09-21T07:54:40.278464+0000","last_undegraded":"2024-09-21T07:54:40.278464+0000","last_fullsized":"2024-09-21T07:54:40.278464+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:31.507095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":15,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.607896+0000","last_change":"2024-09-21T07:54:38.292164+0000","last_active":"2024-09-21T07:54:40.607896+0000","last_peered":"2024-09-21T07:54:40.607896+0000","last_clean":"2024-09-21T07:54:40.607896+0000","last_became_active":"2024-09-21T07:54:38.291754+0000","last_became_peered":"2024-09-21T07:54:38.291754+0000","last_unstale":"2024-09-21T07:54:40.607896+0000","last_undegraded":"2024-09-21T07:54:40.607896+0000","last_fullsized":"2024-09-21T07:54:40.607896+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:50.643950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":401,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.610188+0000","last_change":"2024-09-21T07:35:27.872798+0000","last_active":"2024-09-21T07:54:40.610188+0000","last_peered":"2024-09-21T07:54:40.610188+0000","last_clean":"2024-09-21T07:54:40.610188+0000","last_became_active":"2024-09-21T07:35:27.872483+0000","last_became_peered":"2024-09-21T07:35:27.872483+0000","last_unstale":"2024-09-21T07:54:40.610188+0000","last_undegraded":"2024-09-21T07:54:40.610188+0000","last_fullsized":"2024-09-21T07:54:40.610188+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:11.933469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":408,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.609462+0000","last_change":"2024-09-21T07:35:29.206723+0000","last_active":"2024-09-21T07:54:40.609462+0000","last_peered":"2024-09-21T07:54:40.609462+0000","last_clean":"2024-09-21T07:54:40.609462+0000","last_became_active":"2024-09-21T07:35:26.872377+0000","last_became_peered":"2024-09-21T07:35:26.872377+0000","last_unstale":"2024-09-21T07:54:40.609462+0000","last_undegraded":"2024-09-21T07:54:40.609462+0000","last_fullsized":"2024-09-21T07:54:40.609462+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:29.086862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083077200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":448,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.609137+0000","last_change":"2024-09-21T07:34:24.597672+0000","last_active":"2024-09-21T07:54:40.609137+0000","last_peered":"2024-09-21T07:54:40.609137+0000","last_clean":"2024-09-21T07:54:40.609137+0000","last_became_active":"2024-09-21T07:34:23.267770+0000","last_became_peered":"2024-09-21T07:34:23.267770+0000","last_unstale":"2024-09-21T07:54:40.609137+0000","last_undegraded":"2024-09-21T07:54:40.609137+0000","last_fullsized":"2024-09-21T07:54:40.609137+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:51.629476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086378500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.608080+0000","last_change":"2024-09-21T07:54:39.298211+0000","last_active":"2024-09-21T07:54:40.608080+0000","last_peered":"2024-09-21T07:54:40.608080+0000","last_clean":"2024-09-21T07:54:40.608080+0000","last_became_active":"2024-09-21T07:54:39.297877+0000","last_became_peered":"2024-09-21T07:54:39.297877+0000","last_unstale":"2024-09-21T07:54:40.608080+0000","last_undegraded":"2024-09-21T07:54:40.608080+0000","last_fullsized":"2024-09-21T07:54:40.608080+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:13.790472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":15,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.276427+0000","last_change":"2024-09-21T07:54:38.289593+0000","last_active":"2024-09-21T07:54:40.276427+0000","last_peered":"2024-09-21T07:54:40.276427+0000","last_clean":"2024-09-21T07:54:40.276427+0000","last_became_active":"2024-09-21T07:54:38.289290+0000","last_became_peered":"2024-09-21T07:54:38.289290+0000","last_unstale":"2024-09-21T07:54:40.276427+0000","last_undegraded":"2024-09-21T07:54:40.276427+0000","last_fullsized":"2024-09-21T07:54:40.276427+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:22:38.980285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":401,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.611029+0000","last_change":"2024-09-21T07:35:27.870121+0000","last_active":"2024-09-21T07:54:40.611029+0000","last_peered":"2024-09-21T07:54:40.611029+0000","last_clean":"2024-09-21T07:54:40.611029+0000","last_became_active":"2024-09-21T07:35:27.869811+0000","last_became_peered":"2024-09-21T07:35:27.869811+0000","last_unstale":"2024-09-21T07:54:40.611029+0000","last_undegraded":"2024-09-21T07:54:40.611029+0000","last_fullsized":"2024-09-21T07:54:40.611029+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:31.778461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":406,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.278645+0000","last_change":"2024-09-21T07:35:28.860699+0000","last_active":"2024-09-21T07:54:40.278645+0000","last_peered":"2024-09-21T07:54:40.278645+0000","last_clean":"2024-09-21T07:54:40.278645+0000","last_became_active":"2024-09-21T07:35:26.876779+0000","last_became_peered":"2024-09-21T07:35:26.876779+0000","last_unstale":"2024-09-21T07:54:40.278645+0000","last_undegraded":"2024-09-21T07:54:40.278645+0000","last_fullsized":"2024-09-21T07:54:40.278645+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:28.286403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000913188,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":458,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.279623+0000","last_change":"2024-09-21T07:34:24.260181+0000","last_active":"2024-09-21T07:54:40.279623+0000","last_peered":"2024-09-21T07:54:40.279623+0000","last_clean":"2024-09-21T07:54:40.279623+0000","last_became_active":"2024-09-21T07:34:23.275580+0000","last_became_peered":"2024-09-21T07:34:23.275580+0000","last_unstale":"2024-09-21T07:54:40.279623+0000","last_undegraded":"2024-09-21T07:54:40.279623+0000","last_fullsized":"2024-09-21T07:54:40.279623+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:49.857622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014706980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.275576+0000","last_change":"2024-09-21T07:54:39.289917+0000","last_active":"2024-09-21T07:54:40.275576+0000","last_peered":"2024-09-21T07:54:40.275576+0000","last_clean":"2024-09-21T07:54:40.275576+0000","last_became_active":"2024-09-21T07:54:39.289615+0000","last_became_peered":"2024-09-21T07:54:39.289615+0000","last_unstale":"2024-09-21T07:54:40.275576+0000","last_undegraded":"2024-09-21T07:54:40.275576+0000","last_fullsized":"2024-09-21T07:54:40.275576+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:44.945371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"264'3","reported_seq":18,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.610775+0000","last_change":"2024-09-21T07:54:38.294791+0000","last_active":"2024-09-21T07:54:40.610775+0000","last_peered":"2024-09-21T07:54:40.610775+0000","last_clean":"2024-09-21T07:54:40.610775+0000","last_became_active":"2024-09-21T07:54:38.294199+0000","last_became_peered":"2024-09-21T07:54:38.294199+0000","last_unstale":"2024-09-21T07:54:40.610775+0000","last_undegraded":"2024-09-21T07:54:40.610775+0000","last_fullsized":"2024-09-21T07:54:40.610775+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:47.007101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":398,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.276530+0000","last_change":"2024-09-21T07:35:27.875393+0000","last_active":"2024-09-21T07:54:40.276530+0000","last_peered":"2024-09-21T07:54:40.276530+0000","last_clean":"2024-09-21T07:54:40.276530+0000","last_became_active":"2024-09-21T07:35:27.874671+0000","last_became_peered":"2024-09-21T07:35:27.874671+0000","last_unstale":"2024-09-21T07:54:40.276530+0000","last_undegraded":"2024-09-21T07:54:40.276530+0000","last_fullsized":"2024-09-21T07:54:40.276530+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":408,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.610045+0000","last_change":"2024-09-21T07:35:29.206296+0000","last_active":"2024-09-21T07:54:40.610045+0000","last_peered":"2024-09-21T07:54:40.610045+0000","last_clean":"2024-09-21T07:54:40.610045+0000","last_became_active":"2024-09-21T07:35:26.874575+0000","last_became_peered":"2024-09-21T07:35:26.874575+0000","last_unstale":"2024-09-21T07:54:40.610045+0000","last_undegraded":"2024-09-21T07:54:40.610045+0000","last_fullsized":"2024-09-21T07:54:40.610045+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:22.771154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061011599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":459,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.576852+0000","last_change":"2024-09-21T07:34:24.259112+0000","last_active":"2024-09-21T07:54:40.576852+0000","last_peered":"2024-09-21T07:54:40.576852+0000","last_clean":"2024-09-21T07:54:40.576852+0000","last_became_active":"2024-09-21T07:34:23.635444+0000","last_became_peered":"2024-09-21T07:34:23.635444+0000","last_unstale":"2024-09-21T07:54:40.576852+0000","last_undegraded":"2024-09-21T07:54:40.576852+0000","last_fullsized":"2024-09-21T07:54:40.576852+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:46.255385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001952636,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.278755+0000","last_change":"2024-09-21T07:54:39.299702+0000","last_active":"2024-09-21T07:54:40.278755+0000","last_peered":"2024-09-21T07:54:40.278755+0000","last_clean":"2024-09-21T07:54:40.278755+0000","last_became_active":"2024-09-21T07:54:39.297293+0000","last_became_peered":"2024-09-21T07:54:39.297293+0000","last_unstale":"2024-09-21T07:54:40.278755+0000","last_undegraded":"2024-09-21T07:54:40.278755+0000","last_fullsized":"2024-09-21T07:54:40.278755+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:33.404382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.14","version":"264'6","reported_seq":21,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.576724+0000","last_change":"2024-09-21T07:54:38.301495+0000","last_active":"2024-09-21T07:54:40.576724+0000","last_peered":"2024-09-21T07:54:40.576724+0000","last_clean":"2024-09-21T07:54:40.576724+0000","last_became_active":"2024-09-21T07:54:38.300865+0000","last_became_peered":"2024-09-21T07:54:38.300865+0000","last_unstale":"2024-09-21T07:54:40.576724+0000","last_undegraded":"2024-09-21T07:54:40.576724+0000","last_fullsized":"2024-09-21T07:54:40.576724+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:26:07.479418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":398,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.276787+0000","last_change":"2024-09-21T07:35:27.878356+0000","last_active":"2024-09-21T07:54:40.276787+0000","last_peered":"2024-09-21T07:54:40.276787+0000","last_clean":"2024-09-21T07:54:40.276787+0000","last_became_active":"2024-09-21T07:35:27.877982+0000","last_became_peered":"2024-09-21T07:35:27.877982+0000","last_unstale":"2024-09-21T07:54:40.276787+0000","last_undegraded":"2024-09-21T07:54:40.276787+0000","last_fullsized":"2024-09-21T07:54:40.276787+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":406,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.275534+0000","last_change":"2024-09-21T07:35:29.053131+0000","last_active":"2024-09-21T07:54:40.275534+0000","last_peered":"2024-09-21T07:54:40.275534+0000","last_clean":"2024-09-21T07:54:40.275534+0000","last_became_active":"2024-09-21T07:35:26.870516+0000","last_became_peered":"2024-09-21T07:35:26.870516+0000","last_unstale":"2024-09-21T07:54:40.275534+0000","last_undegraded":"2024-09-21T07:54:40.275534+0000","last_fullsized":"2024-09-21T07:54:40.275534+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:10.924491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070705699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":472,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.279567+0000","last_change":"2024-09-21T07:34:24.539152+0000","last_active":"2024-09-21T07:54:40.279567+0000","last_peered":"2024-09-21T07:54:40.279567+0000","last_clean":"2024-09-21T07:54:40.279567+0000","last_became_active":"2024-09-21T07:34:23.259573+0000","last_became_peered":"2024-09-21T07:34:23.259573+0000","last_unstale":"2024-09-21T07:54:40.279567+0000","last_undegraded":"2024-09-21T07:54:40.279567+0000","last_fullsized":"2024-09-21T07:54:40.279567+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:26.986900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058108400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.276105+0000","last_change":"2024-09-21T07:54:39.304018+0000","last_active":"2024-09-21T07:54:40.276105+0000","last_peered":"2024-09-21T07:54:40.276105+0000","last_clean":"2024-09-21T07:54:40.276105+0000","last_became_active":"2024-09-21T07:54:39.303403+0000","last_became_peered":"2024-09-21T07:54:39.303403+0000","last_unstale":"2024-09-21T07:54:40.276105+0000","last_undegraded":"2024-09-21T07:54:40.276105+0000","last_fullsized":"2024-09-21T07:54:40.276105+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:59.702369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.b","version":"264'2","reported_seq":17,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.275616+0000","last_change":"2024-09-21T07:54:38.293172+0000","last_active":"2024-09-21T07:54:40.275616+0000","last_peered":"2024-09-21T07:54:40.275616+0000","last_clean":"2024-09-21T07:54:40.275616+0000","last_became_active":"2024-09-21T07:54:38.292866+0000","last_became_peered":"2024-09-21T07:54:38.292866+0000","last_unstale":"2024-09-21T07:54:40.275616+0000","last_undegraded":"2024-09-21T07:54:40.275616+0000","last_fullsized":"2024-09-21T07:54:40.275616+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:12.033481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":397,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.275280+0000","last_change":"2024-09-21T07:35:27.873726+0000","last_active":"2024-09-21T07:54:40.275280+0000","last_peered":"2024-09-21T07:54:40.275280+0000","last_clean":"2024-09-21T07:54:40.275280+0000","last_became_active":"2024-09-21T07:35:27.871421+0000","last_became_peered":"2024-09-21T07:35:27.871421+0000","last_unstale":"2024-09-21T07:54:40.275280+0000","last_undegraded":"2024-09-21T07:54:40.275280+0000","last_fullsized":"2024-09-21T07:54:40.275280+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:00.168915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":406,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.279480+0000","last_change":"2024-09-21T07:35:28.861032+0000","last_active":"2024-09-21T07:54:40.279480+0000","last_peered":"2024-09-21T07:54:40.279480+0000","last_clean":"2024-09-21T07:54:40.279480+0000","last_became_active":"2024-09-21T07:35:26.873149+0000","last_became_peered":"2024-09-21T07:35:26.873149+0000","last_unstale":"2024-09-21T07:54:40.279480+0000","last_undegraded":"2024-09-21T07:54:40.279480+0000","last_fullsized":"2024-09-21T07:54:40.279480+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:59.435981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011988369999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":455,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.610301+0000","last_change":"2024-09-21T07:34:24.259902+0000","last_active":"2024-09-21T07:54:40.610301+0000","last_peered":"2024-09-21T07:54:40.610301+0000","last_clean":"2024-09-21T07:54:40.610301+0000","last_became_active":"2024-09-21T07:34:23.271902+0000","last_became_peered":"2024-09-21T07:34:23.271902+0000","last_unstale":"2024-09-21T07:54:40.610301+0000","last_undegraded":"2024-09-21T07:54:40.610301+0000","last_fullsized":"2024-09-21T07:54:40.610301+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:59.439525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001686358,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.278965+0000","last_change":"2024-09-21T07:54:39.302455+0000","last_active":"2024-09-21T07:54:40.278965+0000","last_peered":"2024-09-21T07:54:40.278965+0000","last_clean":"2024-09-21T07:54:40.278965+0000","last_became_active":"2024-09-21T07:54:39.302144+0000","last_became_peered":"2024-09-21T07:54:39.302144+0000","last_unstale":"2024-09-21T07:54:40.278965+0000","last_undegraded":"2024-09-21T07:54:40.278965+0000","last_fullsized":"2024-09-21T07:54:40.278965+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:39.075911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":15,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.608498+0000","last_change":"2024-09-21T07:54:38.297862+0000","last_active":"2024-09-21T07:54:40.608498+0000","last_peered":"2024-09-21T07:54:40.608498+0000","last_clean":"2024-09-21T07:54:40.608498+0000","last_became_active":"2024-09-21T07:54:38.297508+0000","last_became_peered":"2024-09-21T07:54:38.297508+0000","last_unstale":"2024-09-21T07:54:40.608498+0000","last_undegraded":"2024-09-21T07:54:40.608498+0000","last_fullsized":"2024-09-21T07:54:40.608498+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:34.200002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":396,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.606884+0000","last_change":"2024-09-21T07:35:27.872637+0000","last_active":"2024-09-21T07:54:40.606884+0000","last_peered":"2024-09-21T07:54:40.606884+0000","last_clean":"2024-09-21T07:54:40.606884+0000","last_became_active":"2024-09-21T07:35:27.872149+0000","last_became_peered":"2024-09-21T07:35:27.872149+0000","last_unstale":"2024-09-21T07:54:40.606884+0000","last_undegraded":"2024-09-21T07:54:40.606884+0000","last_fullsized":"2024-09-21T07:54:40.606884+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:27.697872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":403,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.609668+0000","last_change":"2024-09-21T07:35:29.493362+0000","last_active":"2024-09-21T07:54:40.609668+0000","last_peered":"2024-09-21T07:54:40.609668+0000","last_clean":"2024-09-21T07:54:40.609668+0000","last_became_active":"2024-09-21T07:35:26.873018+0000","last_became_peered":"2024-09-21T07:35:26.873018+0000","last_unstale":"2024-09-21T07:54:40.609668+0000","last_undegraded":"2024-09-21T07:54:40.609668+0000","last_fullsized":"2024-09-21T07:54:40.609668+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:02.460557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012496510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":458,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.577284+0000","last_change":"2024-09-21T07:34:24.259662+0000","last_active":"2024-09-21T07:54:40.577284+0000","last_peered":"2024-09-21T07:54:40.577284+0000","last_clean":"2024-09-21T07:54:40.577284+0000","last_became_active":"2024-09-21T07:34:23.278812+0000","last_became_peered":"2024-09-21T07:34:23.278812+0000","last_unstale":"2024-09-21T07:54:40.577284+0000","last_undegraded":"2024-09-21T07:54:40.577284+0000","last_fullsized":"2024-09-21T07:54:40.577284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:49.829974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356463,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.608601+0000","last_change":"2024-09-21T07:54:39.304798+0000","last_active":"2024-09-21T07:54:40.608601+0000","last_peered":"2024-09-21T07:54:40.608601+0000","last_clean":"2024-09-21T07:54:40.608601+0000","last_became_active":"2024-09-21T07:54:39.301715+0000","last_became_peered":"2024-09-21T07:54:39.301715+0000","last_unstale":"2024-09-21T07:54:40.608601+0000","last_undegraded":"2024-09-21T07:54:40.608601+0000","last_fullsized":"2024-09-21T07:54:40.608601+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:52:10.733255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":15,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.608228+0000","last_change":"2024-09-21T07:54:38.294367+0000","last_active":"2024-09-21T07:54:40.608228+0000","last_peered":"2024-09-21T07:54:40.608228+0000","last_clean":"2024-09-21T07:54:40.608228+0000","last_became_active":"2024-09-21T07:54:38.293370+0000","last_became_peered":"2024-09-21T07:54:38.293370+0000","last_unstale":"2024-09-21T07:54:40.608228+0000","last_undegraded":"2024-09-21T07:54:40.608228+0000","last_fullsized":"2024-09-21T07:54:40.608228+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:37.732712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":390,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.610243+0000","last_change":"2024-09-21T07:35:27.868448+0000","last_active":"2024-09-21T07:54:40.610243+0000","last_peered":"2024-09-21T07:54:40.610243+0000","last_clean":"2024-09-21T07:54:40.610243+0000","last_became_active":"2024-09-21T07:35:27.868105+0000","last_became_peered":"2024-09-21T07:35:27.868105+0000","last_unstale":"2024-09-21T07:54:40.610243+0000","last_undegraded":"2024-09-21T07:54:40.610243+0000","last_fullsized":"2024-09-21T07:54:40.610243+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:41.468702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":401,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.610826+0000","last_change":"2024-09-21T07:35:29.492974+0000","last_active":"2024-09-21T07:54:40.610826+0000","last_peered":"2024-09-21T07:54:40.610826+0000","last_clean":"2024-09-21T07:54:40.610826+0000","last_became_active":"2024-09-21T07:35:26.872165+0000","last_became_peered":"2024-09-21T07:35:26.872165+0000","last_unstale":"2024-09-21T07:54:40.610826+0000","last_undegraded":"2024-09-21T07:54:40.610826+0000","last_fullsized":"2024-09-21T07:54:40.610826+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:59.912520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013795179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":455,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.606954+0000","last_change":"2024-09-21T07:34:25.084861+0000","last_active":"2024-09-21T07:54:40.606954+0000","last_peered":"2024-09-21T07:54:40.606954+0000","last_clean":"2024-09-21T07:54:40.606954+0000","last_became_active":"2024-09-21T07:34:23.277708+0000","last_became_peered":"2024-09-21T07:34:23.277708+0000","last_unstale":"2024-09-21T07:54:40.606954+0000","last_undegraded":"2024-09-21T07:54:40.606954+0000","last_fullsized":"2024-09-21T07:54:40.606954+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:40.816488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000597566,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.609775+0000","last_change":"2024-09-21T07:54:39.309318+0000","last_active":"2024-09-21T07:54:40.609775+0000","last_peered":"2024-09-21T07:54:40.609775+0000","last_clean":"2024-09-21T07:54:40.609775+0000","last_became_active":"2024-09-21T07:54:39.308999+0000","last_became_peered":"2024-09-21T07:54:39.308999+0000","last_unstale":"2024-09-21T07:54:40.609775+0000","last_undegraded":"2024-09-21T07:54:40.609775+0000","last_fullsized":"2024-09-21T07:54:40.609775+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:10.506431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":15,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.606781+0000","last_change":"2024-09-21T07:54:38.295903+0000","last_active":"2024-09-21T07:54:40.606781+0000","last_peered":"2024-09-21T07:54:40.606781+0000","last_clean":"2024-09-21T07:54:40.606781+0000","last_became_active":"2024-09-21T07:54:38.295615+0000","last_became_peered":"2024-09-21T07:54:38.295615+0000","last_unstale":"2024-09-21T07:54:40.606781+0000","last_undegraded":"2024-09-21T07:54:40.606781+0000","last_fullsized":"2024-09-21T07:54:40.606781+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:37:46.585131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":390,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.609344+0000","last_change":"2024-09-21T07:35:27.874640+0000","last_active":"2024-09-21T07:54:40.609344+0000","last_peered":"2024-09-21T07:54:40.609344+0000","last_clean":"2024-09-21T07:54:40.609344+0000","last_became_active":"2024-09-21T07:35:27.874265+0000","last_became_peered":"2024-09-21T07:35:27.874265+0000","last_unstale":"2024-09-21T07:54:40.609344+0000","last_undegraded":"2024-09-21T07:54:40.609344+0000","last_fullsized":"2024-09-21T07:54:40.609344+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:45.754654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":406,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.610638+0000","last_change":"2024-09-21T07:35:29.207392+0000","last_active":"2024-09-21T07:54:40.610638+0000","last_peered":"2024-09-21T07:54:40.610638+0000","last_clean":"2024-09-21T07:54:40.610638+0000","last_became_active":"2024-09-21T07:35:26.870917+0000","last_became_peered":"2024-09-21T07:35:26.870917+0000","last_unstale":"2024-09-21T07:54:40.610638+0000","last_undegraded":"2024-09-21T07:54:40.610638+0000","last_fullsized":"2024-09-21T07:54:40.610638+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:59.159937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059198499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":459,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.576318+0000","last_change":"2024-09-21T07:34:24.258709+0000","last_active":"2024-09-21T07:54:40.576318+0000","last_peered":"2024-09-21T07:54:40.576318+0000","last_clean":"2024-09-21T07:54:40.576318+0000","last_became_active":"2024-09-21T07:34:23.279042+0000","last_became_peered":"2024-09-21T07:34:23.279042+0000","last_unstale":"2024-09-21T07:54:40.576318+0000","last_undegraded":"2024-09-21T07:54:40.576318+0000","last_fullsized":"2024-09-21T07:54:40.576318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:01.762117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019282100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.608445+0000","last_change":"2024-09-21T07:54:39.309690+0000","last_active":"2024-09-21T07:54:40.608445+0000","last_peered":"2024-09-21T07:54:40.608445+0000","last_clean":"2024-09-21T07:54:40.608445+0000","last_became_active":"2024-09-21T07:54:39.309336+0000","last_became_peered":"2024-09-21T07:54:39.309336+0000","last_unstale":"2024-09-21T07:54:40.608445+0000","last_undegraded":"2024-09-21T07:54:40.608445+0000","last_fullsized":"2024-09-21T07:54:40.608445+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:52.276707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"264'1","reported_seq":16,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.576973+0000","last_change":"2024-09-21T07:54:38.299153+0000","last_active":"2024-09-21T07:54:40.576973+0000","last_peered":"2024-09-21T07:54:40.576973+0000","last_clean":"2024-09-21T07:54:40.576973+0000","last_became_active":"2024-09-21T07:54:38.297607+0000","last_became_peered":"2024-09-21T07:54:38.297607+0000","last_unstale":"2024-09-21T07:54:40.576973+0000","last_undegraded":"2024-09-21T07:54:40.576973+0000","last_fullsized":"2024-09-21T07:54:40.576973+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:04.792467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":396,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.607744+0000","last_change":"2024-09-21T07:35:27.870260+0000","last_active":"2024-09-21T07:54:40.607744+0000","last_peered":"2024-09-21T07:54:40.607744+0000","last_clean":"2024-09-21T07:54:40.607744+0000","last_became_active":"2024-09-21T07:35:27.869448+0000","last_became_peered":"2024-09-21T07:35:27.869448+0000","last_unstale":"2024-09-21T07:54:40.607744+0000","last_undegraded":"2024-09-21T07:54:40.607744+0000","last_fullsized":"2024-09-21T07:54:40.607744+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:16.946790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":397,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.609233+0000","last_change":"2024-09-21T07:35:29.386367+0000","last_active":"2024-09-21T07:54:40.609233+0000","last_peered":"2024-09-21T07:54:40.609233+0000","last_clean":"2024-09-21T07:54:40.609233+0000","last_became_active":"2024-09-21T07:35:26.879238+0000","last_became_peered":"2024-09-21T07:35:26.879238+0000","last_unstale":"2024-09-21T07:54:40.609233+0000","last_undegraded":"2024-09-21T07:54:40.609233+0000","last_fullsized":"2024-09-21T07:54:40.609233+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:16.528260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092223100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":455,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.607962+0000","last_change":"2024-09-21T07:34:25.085635+0000","last_active":"2024-09-21T07:54:40.607962+0000","last_peered":"2024-09-21T07:54:40.607962+0000","last_clean":"2024-09-21T07:54:40.607962+0000","last_became_active":"2024-09-21T07:34:23.273566+0000","last_became_peered":"2024-09-21T07:34:23.273566+0000","last_unstale":"2024-09-21T07:54:40.607962+0000","last_undegraded":"2024-09-21T07:54:40.607962+0000","last_fullsized":"2024-09-21T07:54:40.607962+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:14.142272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079897300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.609618+0000","last_change":"2024-09-21T07:54:39.300826+0000","last_active":"2024-09-21T07:54:40.609618+0000","last_peered":"2024-09-21T07:54:40.609618+0000","last_clean":"2024-09-21T07:54:40.609618+0000","last_became_active":"2024-09-21T07:54:39.300291+0000","last_became_peered":"2024-09-21T07:54:39.300291+0000","last_unstale":"2024-09-21T07:54:40.609618+0000","last_undegraded":"2024-09-21T07:54:40.609618+0000","last_fullsized":"2024-09-21T07:54:40.609618+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:29:58.217668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.e","version":"264'2","reported_seq":17,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.279419+0000","last_change":"2024-09-21T07:54:38.280222+0000","last_active":"2024-09-21T07:54:40.279419+0000","last_peered":"2024-09-21T07:54:40.279419+0000","last_clean":"2024-09-21T07:54:40.279419+0000","last_became_active":"2024-09-21T07:54:38.279317+0000","last_became_peered":"2024-09-21T07:54:38.279317+0000","last_unstale":"2024-09-21T07:54:40.279419+0000","last_undegraded":"2024-09-21T07:54:40.279419+0000","last_fullsized":"2024-09-21T07:54:40.279419+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:23:15.938331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":396,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.607751+0000","last_change":"2024-09-21T07:35:27.878728+0000","last_active":"2024-09-21T07:54:40.607751+0000","last_peered":"2024-09-21T07:54:40.607751+0000","last_clean":"2024-09-21T07:54:40.607751+0000","last_became_active":"2024-09-21T07:35:27.878335+0000","last_became_peered":"2024-09-21T07:35:27.878335+0000","last_unstale":"2024-09-21T07:54:40.607751+0000","last_undegraded":"2024-09-21T07:54:40.607751+0000","last_fullsized":"2024-09-21T07:54:40.607751+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:25:37.731771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":418,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.278073+0000","last_change":"2024-09-21T07:35:28.858418+0000","last_active":"2024-09-21T07:54:40.278073+0000","last_peered":"2024-09-21T07:54:40.278073+0000","last_clean":"2024-09-21T07:54:40.278073+0000","last_became_active":"2024-09-21T07:35:26.876944+0000","last_became_peered":"2024-09-21T07:35:26.876944+0000","last_unstale":"2024-09-21T07:54:40.278073+0000","last_undegraded":"2024-09-21T07:54:40.278073+0000","last_fullsized":"2024-09-21T07:54:40.278073+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:04.514304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090862799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":447,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.610090+0000","last_change":"2024-09-21T07:34:24.597926+0000","last_active":"2024-09-21T07:54:40.610090+0000","last_peered":"2024-09-21T07:54:40.610090+0000","last_clean":"2024-09-21T07:54:40.610090+0000","last_became_active":"2024-09-21T07:34:23.268926+0000","last_became_peered":"2024-09-21T07:34:23.268926+0000","last_unstale":"2024-09-21T07:54:40.610090+0000","last_undegraded":"2024-09-21T07:54:40.610090+0000","last_fullsized":"2024-09-21T07:54:40.610090+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:46.190752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101851,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.278654+0000","last_change":"2024-09-21T07:54:39.304280+0000","last_active":"2024-09-21T07:54:40.278654+0000","last_peered":"2024-09-21T07:54:40.278654+0000","last_clean":"2024-09-21T07:54:40.278654+0000","last_became_active":"2024-09-21T07:54:39.303971+0000","last_became_peered":"2024-09-21T07:54:39.303971+0000","last_unstale":"2024-09-21T07:54:40.278654+0000","last_undegraded":"2024-09-21T07:54:40.278654+0000","last_fullsized":"2024-09-21T07:54:40.278654+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:51:40.839772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"264'1","reported_seq":16,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.608728+0000","last_change":"2024-09-21T07:54:38.300028+0000","last_active":"2024-09-21T07:54:40.608728+0000","last_peered":"2024-09-21T07:54:40.608728+0000","last_clean":"2024-09-21T07:54:40.608728+0000","last_became_active":"2024-09-21T07:54:38.299574+0000","last_became_peered":"2024-09-21T07:54:38.299574+0000","last_unstale":"2024-09-21T07:54:40.608728+0000","last_undegraded":"2024-09-21T07:54:40.608728+0000","last_fullsized":"2024-09-21T07:54:40.608728+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:56.870183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":392,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.609905+0000","last_change":"2024-09-21T07:35:27.877426+0000","last_active":"2024-09-21T07:54:40.609905+0000","last_peered":"2024-09-21T07:54:40.609905+0000","last_clean":"2024-09-21T07:54:40.609905+0000","last_became_active":"2024-09-21T07:35:27.877050+0000","last_became_peered":"2024-09-21T07:35:27.877050+0000","last_unstale":"2024-09-21T07:54:40.609905+0000","last_undegraded":"2024-09-21T07:54:40.609905+0000","last_fullsized":"2024-09-21T07:54:40.609905+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:36.268334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":403,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.606892+0000","last_change":"2024-09-21T07:35:28.862273+0000","last_active":"2024-09-21T07:54:40.606892+0000","last_peered":"2024-09-21T07:54:40.606892+0000","last_clean":"2024-09-21T07:54:40.606892+0000","last_became_active":"2024-09-21T07:35:26.874011+0000","last_became_peered":"2024-09-21T07:35:26.874011+0000","last_unstale":"2024-09-21T07:54:40.606892+0000","last_undegraded":"2024-09-21T07:54:40.606892+0000","last_fullsized":"2024-09-21T07:54:40.606892+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:11.986950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071644799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":457,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.278951+0000","last_change":"2024-09-21T07:34:24.259794+0000","last_active":"2024-09-21T07:54:40.278951+0000","last_peered":"2024-09-21T07:54:40.278951+0000","last_clean":"2024-09-21T07:54:40.278951+0000","last_became_active":"2024-09-21T07:34:23.265679+0000","last_became_peered":"2024-09-21T07:34:23.265679+0000","last_unstale":"2024-09-21T07:54:40.278951+0000","last_undegraded":"2024-09-21T07:54:40.278951+0000","last_fullsized":"2024-09-21T07:54:40.278951+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:02.575796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001298518,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.275723+0000","last_change":"2024-09-21T07:54:39.298024+0000","last_active":"2024-09-21T07:54:40.275723+0000","last_peered":"2024-09-21T07:54:40.275723+0000","last_clean":"2024-09-21T07:54:40.275723+0000","last_became_active":"2024-09-21T07:54:39.297141+0000","last_became_peered":"2024-09-21T07:54:39.297141+0000","last_unstale":"2024-09-21T07:54:40.275723+0000","last_undegraded":"2024-09-21T07:54:40.275723+0000","last_fullsized":"2024-09-21T07:54:40.275723+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:25.278550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":15,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.609733+0000","last_change":"2024-09-21T07:54:38.293577+0000","last_active":"2024-09-21T07:54:40.609733+0000","last_peered":"2024-09-21T07:54:40.609733+0000","last_clean":"2024-09-21T07:54:40.609733+0000","last_became_active":"2024-09-21T07:54:38.293222+0000","last_became_peered":"2024-09-21T07:54:38.293222+0000","last_unstale":"2024-09-21T07:54:40.609733+0000","last_undegraded":"2024-09-21T07:54:40.609733+0000","last_fullsized":"2024-09-21T07:54:40.609733+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:37:46.585131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":401,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.608905+0000","last_change":"2024-09-21T07:35:27.877725+0000","last_active":"2024-09-21T07:54:40.608905+0000","last_peered":"2024-09-21T07:54:40.608905+0000","last_clean":"2024-09-21T07:54:40.608905+0000","last_became_active":"2024-09-21T07:35:27.877398+0000","last_became_peered":"2024-09-21T07:35:27.877398+0000","last_unstale":"2024-09-21T07:54:40.608905+0000","last_undegraded":"2024-09-21T07:54:40.608905+0000","last_fullsized":"2024-09-21T07:54:40.608905+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:05.859762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":404,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.278443+0000","last_change":"2024-09-21T07:35:28.860612+0000","last_active":"2024-09-21T07:54:40.278443+0000","last_peered":"2024-09-21T07:54:40.278443+0000","last_clean":"2024-09-21T07:54:40.278443+0000","last_became_active":"2024-09-21T07:35:26.876562+0000","last_became_peered":"2024-09-21T07:35:26.876562+0000","last_unstale":"2024-09-21T07:54:40.278443+0000","last_undegraded":"2024-09-21T07:54:40.278443+0000","last_fullsized":"2024-09-21T07:54:40.278443+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:06.824331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012252470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":455,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.611427+0000","last_change":"2024-09-21T07:34:24.260330+0000","last_active":"2024-09-21T07:54:40.611427+0000","last_peered":"2024-09-21T07:54:40.611427+0000","last_clean":"2024-09-21T07:54:40.611427+0000","last_became_active":"2024-09-21T07:34:23.637210+0000","last_became_peered":"2024-09-21T07:34:23.637210+0000","last_unstale":"2024-09-21T07:54:40.611427+0000","last_undegraded":"2024-09-21T07:54:40.611427+0000","last_fullsized":"2024-09-21T07:54:40.611427+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.488451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001537093,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.607559+0000","last_change":"2024-09-21T07:54:39.295447+0000","last_active":"2024-09-21T07:54:40.607559+0000","last_peered":"2024-09-21T07:54:40.607559+0000","last_clean":"2024-09-21T07:54:40.607559+0000","last_became_active":"2024-09-21T07:54:39.295112+0000","last_became_peered":"2024-09-21T07:54:39.295112+0000","last_unstale":"2024-09-21T07:54:40.607559+0000","last_undegraded":"2024-09-21T07:54:40.607559+0000","last_fullsized":"2024-09-21T07:54:40.607559+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:04.183280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":15,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.609100+0000","last_change":"2024-09-21T07:54:38.300817+0000","last_active":"2024-09-21T07:54:40.609100+0000","last_peered":"2024-09-21T07:54:40.609100+0000","last_clean":"2024-09-21T07:54:40.609100+0000","last_became_active":"2024-09-21T07:54:38.300443+0000","last_became_peered":"2024-09-21T07:54:38.300443+0000","last_unstale":"2024-09-21T07:54:40.609100+0000","last_undegraded":"2024-09-21T07:54:40.609100+0000","last_fullsized":"2024-09-21T07:54:40.609100+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:50.932301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":397,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.606801+0000","last_change":"2024-09-21T07:35:27.874757+0000","last_active":"2024-09-21T07:54:40.606801+0000","last_peered":"2024-09-21T07:54:40.606801+0000","last_clean":"2024-09-21T07:54:40.606801+0000","last_became_active":"2024-09-21T07:35:27.874360+0000","last_became_peered":"2024-09-21T07:35:27.874360+0000","last_unstale":"2024-09-21T07:54:40.606801+0000","last_undegraded":"2024-09-21T07:54:40.606801+0000","last_fullsized":"2024-09-21T07:54:40.606801+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:17.646912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":401,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.611730+0000","last_change":"2024-09-21T07:35:29.493431+0000","last_active":"2024-09-21T07:54:40.611730+0000","last_peered":"2024-09-21T07:54:40.611730+0000","last_clean":"2024-09-21T07:54:40.611730+0000","last_became_active":"2024-09-21T07:35:26.873691+0000","last_became_peered":"2024-09-21T07:35:26.873691+0000","last_unstale":"2024-09-21T07:54:40.611730+0000","last_undegraded":"2024-09-21T07:54:40.611730+0000","last_fullsized":"2024-09-21T07:54:40.611730+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:51.705051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001162464,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":447,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.610382+0000","last_change":"2024-09-21T07:34:24.597286+0000","last_active":"2024-09-21T07:54:40.610382+0000","last_peered":"2024-09-21T07:54:40.610382+0000","last_clean":"2024-09-21T07:54:40.610382+0000","last_became_active":"2024-09-21T07:34:23.267133+0000","last_became_peered":"2024-09-21T07:34:23.267133+0000","last_unstale":"2024-09-21T07:54:40.610382+0000","last_undegraded":"2024-09-21T07:54:40.610382+0000","last_fullsized":"2024-09-21T07:54:40.610382+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:18.257010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.576344+0000","last_change":"2024-09-21T07:54:39.299363+0000","last_active":"2024-09-21T07:54:40.576344+0000","last_peered":"2024-09-21T07:54:40.576344+0000","last_clean":"2024-09-21T07:54:40.576344+0000","last_became_active":"2024-09-21T07:54:39.298900+0000","last_became_peered":"2024-09-21T07:54:39.298900+0000","last_unstale":"2024-09-21T07:54:40.576344+0000","last_undegraded":"2024-09-21T07:54:40.576344+0000","last_fullsized":"2024-09-21T07:54:40.576344+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:19.143810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":15,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.608895+0000","last_change":"2024-09-21T07:54:38.295244+0000","last_active":"2024-09-21T07:54:40.608895+0000","last_peered":"2024-09-21T07:54:40.608895+0000","last_clean":"2024-09-21T07:54:40.608895+0000","last_became_active":"2024-09-21T07:54:38.294875+0000","last_became_peered":"2024-09-21T07:54:38.294875+0000","last_unstale":"2024-09-21T07:54:40.608895+0000","last_undegraded":"2024-09-21T07:54:40.608895+0000","last_fullsized":"2024-09-21T07:54:40.608895+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:10.353533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":412,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.277967+0000","last_change":"2024-09-21T07:35:27.864458+0000","last_active":"2024-09-21T07:54:40.277967+0000","last_peered":"2024-09-21T07:54:40.277967+0000","last_clean":"2024-09-21T07:54:40.277967+0000","last_became_active":"2024-09-21T07:35:27.864136+0000","last_became_peered":"2024-09-21T07:35:27.864136+0000","last_unstale":"2024-09-21T07:54:40.277967+0000","last_undegraded":"2024-09-21T07:54:40.277967+0000","last_fullsized":"2024-09-21T07:54:40.277967+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:20.162240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":418,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.279424+0000","last_change":"2024-09-21T07:35:28.858784+0000","last_active":"2024-09-21T07:54:40.279424+0000","last_peered":"2024-09-21T07:54:40.279424+0000","last_clean":"2024-09-21T07:54:40.279424+0000","last_became_active":"2024-09-21T07:35:26.876890+0000","last_became_peered":"2024-09-21T07:35:26.876890+0000","last_unstale":"2024-09-21T07:54:40.279424+0000","last_undegraded":"2024-09-21T07:54:40.279424+0000","last_fullsized":"2024-09-21T07:54:40.279424+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:40.738327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064543499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":458,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.577228+0000","last_change":"2024-09-21T07:34:24.258712+0000","last_active":"2024-09-21T07:54:40.577228+0000","last_peered":"2024-09-21T07:54:40.577228+0000","last_clean":"2024-09-21T07:54:40.577228+0000","last_became_active":"2024-09-21T07:34:23.274692+0000","last_became_peered":"2024-09-21T07:34:23.274692+0000","last_unstale":"2024-09-21T07:54:40.577228+0000","last_undegraded":"2024-09-21T07:54:40.577228+0000","last_fullsized":"2024-09-21T07:54:40.577228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:43.279454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018051479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.279376+0000","last_change":"2024-09-21T07:54:39.304144+0000","last_active":"2024-09-21T07:54:40.279376+0000","last_peered":"2024-09-21T07:54:40.279376+0000","last_clean":"2024-09-21T07:54:40.279376+0000","last_became_active":"2024-09-21T07:54:39.300572+0000","last_became_peered":"2024-09-21T07:54:39.300572+0000","last_unstale":"2024-09-21T07:54:40.279376+0000","last_undegraded":"2024-09-21T07:54:40.279376+0000","last_fullsized":"2024-09-21T07:54:40.279376+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:53:06.173546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":15,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.275376+0000","last_change":"2024-09-21T07:54:38.297698+0000","last_active":"2024-09-21T07:54:40.275376+0000","last_peered":"2024-09-21T07:54:40.275376+0000","last_clean":"2024-09-21T07:54:40.275376+0000","last_became_active":"2024-09-21T07:54:38.297391+0000","last_became_peered":"2024-09-21T07:54:38.297391+0000","last_unstale":"2024-09-21T07:54:40.275376+0000","last_undegraded":"2024-09-21T07:54:40.275376+0000","last_fullsized":"2024-09-21T07:54:40.275376+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:54.433097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":398,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.576254+0000","last_change":"2024-09-21T07:35:27.878020+0000","last_active":"2024-09-21T07:54:40.576254+0000","last_peered":"2024-09-21T07:54:40.576254+0000","last_clean":"2024-09-21T07:54:40.576254+0000","last_became_active":"2024-09-21T07:35:27.877532+0000","last_became_peered":"2024-09-21T07:35:27.877532+0000","last_unstale":"2024-09-21T07:54:40.576254+0000","last_undegraded":"2024-09-21T07:54:40.576254+0000","last_fullsized":"2024-09-21T07:54:40.576254+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:19.756073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":404,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.278998+0000","last_change":"2024-09-21T07:35:28.861049+0000","last_active":"2024-09-21T07:54:40.278998+0000","last_peered":"2024-09-21T07:54:40.278998+0000","last_clean":"2024-09-21T07:54:40.278998+0000","last_became_active":"2024-09-21T07:35:26.876819+0000","last_became_peered":"2024-09-21T07:35:26.876819+0000","last_unstale":"2024-09-21T07:54:40.278998+0000","last_undegraded":"2024-09-21T07:54:40.278998+0000","last_fullsized":"2024-09-21T07:54:40.278998+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:19.633803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055780599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":455,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.606930+0000","last_change":"2024-09-21T07:34:25.084849+0000","last_active":"2024-09-21T07:54:40.606930+0000","last_peered":"2024-09-21T07:54:40.606930+0000","last_clean":"2024-09-21T07:54:40.606930+0000","last_became_active":"2024-09-21T07:34:23.277713+0000","last_became_peered":"2024-09-21T07:34:23.277713+0000","last_unstale":"2024-09-21T07:54:40.606930+0000","last_undegraded":"2024-09-21T07:54:40.606930+0000","last_fullsized":"2024-09-21T07:54:40.606930+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:05.811810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059079499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.609524+0000","last_change":"2024-09-21T07:54:39.300046+0000","last_active":"2024-09-21T07:54:40.609524+0000","last_peered":"2024-09-21T07:54:40.609524+0000","last_clean":"2024-09-21T07:54:40.609524+0000","last_became_active":"2024-09-21T07:54:39.299733+0000","last_became_peered":"2024-09-21T07:54:39.299733+0000","last_unstale":"2024-09-21T07:54:40.609524+0000","last_undegraded":"2024-09-21T07:54:40.609524+0000","last_fullsized":"2024-09-21T07:54:40.609524+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:05:28.970944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":15,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.608040+0000","last_change":"2024-09-21T07:54:38.290947+0000","last_active":"2024-09-21T07:54:40.608040+0000","last_peered":"2024-09-21T07:54:40.608040+0000","last_clean":"2024-09-21T07:54:40.608040+0000","last_became_active":"2024-09-21T07:54:38.290609+0000","last_became_peered":"2024-09-21T07:54:38.290609+0000","last_unstale":"2024-09-21T07:54:40.608040+0000","last_undegraded":"2024-09-21T07:54:40.608040+0000","last_fullsized":"2024-09-21T07:54:40.608040+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:40.430199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":398,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.275790+0000","last_change":"2024-09-21T07:35:27.884248+0000","last_active":"2024-09-21T07:54:40.275790+0000","last_peered":"2024-09-21T07:54:40.275790+0000","last_clean":"2024-09-21T07:54:40.275790+0000","last_became_active":"2024-09-21T07:35:27.883861+0000","last_became_peered":"2024-09-21T07:35:27.883861+0000","last_unstale":"2024-09-21T07:54:40.275790+0000","last_undegraded":"2024-09-21T07:54:40.275790+0000","last_fullsized":"2024-09-21T07:54:40.275790+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":407,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.610385+0000","last_change":"2024-09-21T07:35:29.207135+0000","last_active":"2024-09-21T07:54:40.610385+0000","last_peered":"2024-09-21T07:54:40.610385+0000","last_clean":"2024-09-21T07:54:40.610385+0000","last_became_active":"2024-09-21T07:35:26.872787+0000","last_became_peered":"2024-09-21T07:35:26.872787+0000","last_unstale":"2024-09-21T07:54:40.610385+0000","last_undegraded":"2024-09-21T07:54:40.610385+0000","last_fullsized":"2024-09-21T07:54:40.610385+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:27.543217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011157509999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":471,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.278382+0000","last_change":"2024-09-21T07:34:24.538610+0000","last_active":"2024-09-21T07:54:40.278382+0000","last_peered":"2024-09-21T07:54:40.278382+0000","last_clean":"2024-09-21T07:54:40.278382+0000","last_became_active":"2024-09-21T07:34:23.259700+0000","last_became_peered":"2024-09-21T07:34:23.259700+0000","last_unstale":"2024-09-21T07:54:40.278382+0000","last_undegraded":"2024-09-21T07:54:40.278382+0000","last_fullsized":"2024-09-21T07:54:40.278382+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:57.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070522399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.278639+0000","last_change":"2024-09-21T07:54:39.307601+0000","last_active":"2024-09-21T07:54:40.278639+0000","last_peered":"2024-09-21T07:54:40.278639+0000","last_clean":"2024-09-21T07:54:40.278639+0000","last_became_active":"2024-09-21T07:54:39.307281+0000","last_became_peered":"2024-09-21T07:54:39.307281+0000","last_unstale":"2024-09-21T07:54:40.278639+0000","last_undegraded":"2024-09-21T07:54:40.278639+0000","last_fullsized":"2024-09-21T07:54:40.278639+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:36.709627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.7","version":"264'2","reported_seq":17,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.276745+0000","last_change":"2024-09-21T07:54:38.294852+0000","last_active":"2024-09-21T07:54:40.276745+0000","last_peered":"2024-09-21T07:54:40.276745+0000","last_clean":"2024-09-21T07:54:40.276745+0000","last_became_active":"2024-09-21T07:54:38.294558+0000","last_became_peered":"2024-09-21T07:54:38.294558+0000","last_unstale":"2024-09-21T07:54:40.276745+0000","last_undegraded":"2024-09-21T07:54:40.276745+0000","last_fullsized":"2024-09-21T07:54:40.276745+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:31:49.919294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":412,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.278071+0000","last_change":"2024-09-21T07:35:27.868024+0000","last_active":"2024-09-21T07:54:40.278071+0000","last_peered":"2024-09-21T07:54:40.278071+0000","last_clean":"2024-09-21T07:54:40.278071+0000","last_became_active":"2024-09-21T07:35:27.867339+0000","last_became_peered":"2024-09-21T07:35:27.867339+0000","last_unstale":"2024-09-21T07:54:40.278071+0000","last_undegraded":"2024-09-21T07:54:40.278071+0000","last_fullsized":"2024-09-21T07:54:40.278071+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":419,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.278759+0000","last_change":"2024-09-21T07:35:28.858505+0000","last_active":"2024-09-21T07:54:40.278759+0000","last_peered":"2024-09-21T07:54:40.278759+0000","last_clean":"2024-09-21T07:54:40.278759+0000","last_became_active":"2024-09-21T07:35:26.877320+0000","last_became_peered":"2024-09-21T07:35:26.877320+0000","last_unstale":"2024-09-21T07:54:40.278759+0000","last_undegraded":"2024-09-21T07:54:40.278759+0000","last_fullsized":"2024-09-21T07:54:40.278759+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:53.672782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244119999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":449,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.609207+0000","last_change":"2024-09-21T07:34:24.597597+0000","last_active":"2024-09-21T07:54:40.609207+0000","last_peered":"2024-09-21T07:54:40.609207+0000","last_clean":"2024-09-21T07:54:40.609207+0000","last_became_active":"2024-09-21T07:34:23.635204+0000","last_became_peered":"2024-09-21T07:34:23.635204+0000","last_unstale":"2024-09-21T07:54:40.609207+0000","last_undegraded":"2024-09-21T07:54:40.609207+0000","last_fullsized":"2024-09-21T07:54:40.609207+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:48.644789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081139199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.610023+0000","last_change":"2024-09-21T07:54:39.317640+0000","last_active":"2024-09-21T07:54:40.610023+0000","last_peered":"2024-09-21T07:54:40.610023+0000","last_clean":"2024-09-21T07:54:40.610023+0000","last_became_active":"2024-09-21T07:54:39.317409+0000","last_became_peered":"2024-09-21T07:54:39.317409+0000","last_unstale":"2024-09-21T07:54:40.610023+0000","last_undegraded":"2024-09-21T07:54:40.610023+0000","last_fullsized":"2024-09-21T07:54:40.610023+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:48:04.606170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"265'1","reported_seq":16,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.611015+0000","last_change":"2024-09-21T07:54:38.286711+0000","last_active":"2024-09-21T07:54:40.611015+0000","last_peered":"2024-09-21T07:54:40.611015+0000","last_clean":"2024-09-21T07:54:40.611015+0000","last_became_active":"2024-09-21T07:54:38.286384+0000","last_became_peered":"2024-09-21T07:54:38.286384+0000","last_unstale":"2024-09-21T07:54:40.611015+0000","last_undegraded":"2024-09-21T07:54:40.611015+0000","last_fullsized":"2024-09-21T07:54:40.611015+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:17:11.383593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":399,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.610915+0000","last_change":"2024-09-21T07:35:27.877503+0000","last_active":"2024-09-21T07:54:40.610915+0000","last_peered":"2024-09-21T07:54:40.610915+0000","last_clean":"2024-09-21T07:54:40.610915+0000","last_became_active":"2024-09-21T07:35:27.875775+0000","last_became_peered":"2024-09-21T07:35:27.875775+0000","last_unstale":"2024-09-21T07:54:40.610915+0000","last_undegraded":"2024-09-21T07:54:40.610915+0000","last_fullsized":"2024-09-21T07:54:40.610915+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:49.291023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":402,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.610175+0000","last_change":"2024-09-21T07:35:29.491412+0000","last_active":"2024-09-21T07:54:40.610175+0000","last_peered":"2024-09-21T07:54:40.610175+0000","last_clean":"2024-09-21T07:54:40.610175+0000","last_became_active":"2024-09-21T07:35:26.876404+0000","last_became_peered":"2024-09-21T07:35:26.876404+0000","last_unstale":"2024-09-21T07:54:40.610175+0000","last_undegraded":"2024-09-21T07:54:40.610175+0000","last_fullsized":"2024-09-21T07:54:40.610175+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063918000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":459,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.577083+0000","last_change":"2024-09-21T07:34:24.259026+0000","last_active":"2024-09-21T07:54:40.577083+0000","last_peered":"2024-09-21T07:54:40.577083+0000","last_clean":"2024-09-21T07:54:40.577083+0000","last_became_active":"2024-09-21T07:34:23.635456+0000","last_became_peered":"2024-09-21T07:34:23.635456+0000","last_unstale":"2024-09-21T07:54:40.577083+0000","last_undegraded":"2024-09-21T07:54:40.577083+0000","last_fullsized":"2024-09-21T07:54:40.577083+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:09.592752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018654419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.275554+0000","last_change":"2024-09-21T07:54:39.288520+0000","last_active":"2024-09-21T07:54:40.275554+0000","last_peered":"2024-09-21T07:54:40.275554+0000","last_clean":"2024-09-21T07:54:40.275554+0000","last_became_active":"2024-09-21T07:54:39.288231+0000","last_became_peered":"2024-09-21T07:54:39.288231+0000","last_unstale":"2024-09-21T07:54:40.275554+0000","last_undegraded":"2024-09-21T07:54:40.275554+0000","last_fullsized":"2024-09-21T07:54:40.275554+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:46:43.627859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":15,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.607651+0000","last_change":"2024-09-21T07:54:38.291828+0000","last_active":"2024-09-21T07:54:40.607651+0000","last_peered":"2024-09-21T07:54:40.607651+0000","last_clean":"2024-09-21T07:54:40.607651+0000","last_became_active":"2024-09-21T07:54:38.291465+0000","last_became_peered":"2024-09-21T07:54:38.291465+0000","last_unstale":"2024-09-21T07:54:40.607651+0000","last_undegraded":"2024-09-21T07:54:40.607651+0000","last_fullsized":"2024-09-21T07:54:40.607651+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:27:35.710995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":399,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.611150+0000","last_change":"2024-09-21T07:35:27.871984+0000","last_active":"2024-09-21T07:54:40.611150+0000","last_peered":"2024-09-21T07:54:40.611150+0000","last_clean":"2024-09-21T07:54:40.611150+0000","last_became_active":"2024-09-21T07:35:27.870336+0000","last_became_peered":"2024-09-21T07:35:27.870336+0000","last_unstale":"2024-09-21T07:54:40.611150+0000","last_undegraded":"2024-09-21T07:54:40.611150+0000","last_fullsized":"2024-09-21T07:54:40.611150+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:57.120479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":405,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.278533+0000","last_change":"2024-09-21T07:35:28.860351+0000","last_active":"2024-09-21T07:54:40.278533+0000","last_peered":"2024-09-21T07:54:40.278533+0000","last_clean":"2024-09-21T07:54:40.278533+0000","last_became_active":"2024-09-21T07:35:26.872046+0000","last_became_peered":"2024-09-21T07:35:26.872046+0000","last_unstale":"2024-09-21T07:54:40.278533+0000","last_undegraded":"2024-09-21T07:54:40.278533+0000","last_fullsized":"2024-09-21T07:54:40.278533+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:47.074327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098384399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":455,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.610460+0000","last_change":"2024-09-21T07:34:24.260300+0000","last_active":"2024-09-21T07:54:40.610460+0000","last_peered":"2024-09-21T07:54:40.610460+0000","last_clean":"2024-09-21T07:54:40.610460+0000","last_became_active":"2024-09-21T07:34:23.270128+0000","last_became_peered":"2024-09-21T07:34:23.270128+0000","last_unstale":"2024-09-21T07:54:40.610460+0000","last_undegraded":"2024-09-21T07:54:40.610460+0000","last_fullsized":"2024-09-21T07:54:40.610460+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:25.260931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00176651,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":13,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.609290+0000","last_change":"2024-09-21T07:54:39.311793+0000","last_active":"2024-09-21T07:54:40.609290+0000","last_peered":"2024-09-21T07:54:40.609290+0000","last_clean":"2024-09-21T07:54:40.609290+0000","last_became_active":"2024-09-21T07:54:39.311263+0000","last_became_peered":"2024-09-21T07:54:39.311263+0000","last_unstale":"2024-09-21T07:54:40.609290+0000","last_undegraded":"2024-09-21T07:54:40.609290+0000","last_fullsized":"2024-09-21T07:54:40.609290+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":263,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:38.260838+0000","last_clean_scrub_stamp":"2024-09-21T07:54:38.260838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:06.443196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":15,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.607628+0000","last_change":"2024-09-21T07:54:38.297996+0000","last_active":"2024-09-21T07:54:40.607628+0000","last_peered":"2024-09-21T07:54:40.607628+0000","last_clean":"2024-09-21T07:54:40.607628+0000","last_became_active":"2024-09-21T07:54:38.297618+0000","last_became_peered":"2024-09-21T07:54:38.297618+0000","last_unstale":"2024-09-21T07:54:40.607628+0000","last_undegraded":"2024-09-21T07:54:40.607628+0000","last_fullsized":"2024-09-21T07:54:40.607628+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":262,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.257240+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.257240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:14:14.124838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":399,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.609033+0000","last_change":"2024-09-21T07:35:27.877861+0000","last_active":"2024-09-21T07:54:40.609033+0000","last_peered":"2024-09-21T07:54:40.609033+0000","last_clean":"2024-09-21T07:54:40.609033+0000","last_became_active":"2024-09-21T07:35:27.877434+0000","last_became_peered":"2024-09-21T07:35:27.877434+0000","last_unstale":"2024-09-21T07:54:40.609033+0000","last_undegraded":"2024-09-21T07:54:40.609033+0000","last_fullsized":"2024-09-21T07:54:40.609033+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":404,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.607069+0000","last_change":"2024-09-21T07:35:28.862954+0000","last_active":"2024-09-21T07:54:40.607069+0000","last_peered":"2024-09-21T07:54:40.607069+0000","last_clean":"2024-09-21T07:54:40.607069+0000","last_became_active":"2024-09-21T07:35:26.870613+0000","last_became_peered":"2024-09-21T07:35:26.870613+0000","last_unstale":"2024-09-21T07:54:40.607069+0000","last_undegraded":"2024-09-21T07:54:40.607069+0000","last_fullsized":"2024-09-21T07:54:40.607069+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:06.901043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059842100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":456,"reported_epoch":265,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.276535+0000","last_change":"2024-09-21T07:34:25.028904+0000","last_active":"2024-09-21T07:54:40.276535+0000","last_peered":"2024-09-21T07:54:40.276535+0000","last_clean":"2024-09-21T07:54:40.276535+0000","last_became_active":"2024-09-21T07:34:23.260942+0000","last_became_peered":"2024-09-21T07:34:23.260942+0000","last_unstale":"2024-09-21T07:54:40.276535+0000","last_undegraded":"2024-09-21T07:54:40.276535+0000","last_fullsized":"2024-09-21T07:54:40.276535+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:22.709328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060553899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509703,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71136,"kb_used_data":3852,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94300704,"statfs":{"total":96636764160,"available":96563920896,"internally_reserved":0,"allocated":3944448,"data_stored":2412494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509705,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70556,"kb_used_data":3272,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301284,"statfs":{"total":96636764160,"available":96564514816,"internally_reserved":0,"allocated":3350528,"data_stored":1822114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509703,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52508,"kb_used_data":3264,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319332,"statfs":{"total":96636764160,"available":96582995968,"internally_reserved":0,"allocated":3342336,"data_stored":1822002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509704,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30308,"kb_used_data":3272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341532,"statfs":{"total":96636764160,"available":96605728768,"internally_reserved":0,"allocated":3350528,"data_stored":1822654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509705,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48348,"kb_used_data":3264,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323492,"statfs":{"total":96636764160,"available":96587255808,"internally_reserved":0,"allocated":3342336,"data_stored":1822002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509705,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57252,"kb_used_data":3856,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314588,"statfs":{"total":96636764160,"available":96578138112,"internally_reserved":0,"allocated":3948544,"data_stored":2413982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542408,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56664,"kb_used_data":3268,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315176,"statfs":{"total":96636764160,"available":96578740224,"internally_reserved":0,"allocated":3346432,"data_stored":1822072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542407,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74732,"kb_used_data":3288,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297108,"statfs":{"total":96636764160,"available":96560238592,"internally_reserved":0,"allocated":3366912,"data_stored":1824252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:54:44.311 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T07:54:44.311 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T07:54:44.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.581+0000 7f6857e00640 1 -- 172.21.15.39:0/2015023895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68580d0030 msgr2=0x7f68580b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:44.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.581+0000 7f6857e00640 1 --2- 172.21.15.39:0/2015023895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68580d0030 0x7f68580b4460 secure :-1 s=READY pgs=2825 cs=0 l=1 rev1=1 crypto rx=0x7f6848005fd0 tx=0x7f684802a5f0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.581+0000 7f6857e00640 1 -- 172.21.15.39:0/2015023895 shutdown_connections 2024-09-21T07:54:44.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.581+0000 7f6857e00640 1 --2- 172.21.15.39:0/2015023895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68580d0030 0x7f68580b4460 unknown :-1 s=CLOSED pgs=2825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.581+0000 7f6857e00640 1 -- 172.21.15.39:0/2015023895 >> 172.21.15.39:0/2015023895 conn(0x7f68580d0cb0 msgr2=0x7f68580d10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:44.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.581+0000 7f6857e00640 1 -- 172.21.15.39:0/2015023895 shutdown_connections 2024-09-21T07:54:44.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.581+0000 7f6857e00640 1 -- 172.21.15.39:0/2015023895 wait complete. 2024-09-21T07:54:44.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.584+0000 7f6857e00640 1 Processor -- start 2024-09-21T07:54:44.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.584+0000 7f6857e00640 1 -- start start 2024-09-21T07:54:44.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.585+0000 7f6857e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f685806d4f0 0x7f685815cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:44.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.585+0000 7f6857e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68580b49a0 0x7f68581f59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:44.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.585+0000 7f6857e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68580d0030 0x7f685815dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:44.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.585+0000 7f6857e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f68580706f0 con 0x7f68580b49a0 2024-09-21T07:54:44.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.585+0000 7f6857e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6858070570 con 0x7f685806d4f0 2024-09-21T07:54:44.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.585+0000 7f6857e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f6858070870 con 0x7f68580d0030 2024-09-21T07:54:44.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.585+0000 7f684fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68580b49a0 0x7f68581f59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:44.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.585+0000 7f684fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68580b49a0 0x7f68581f59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47354/0 (socket says 172.21.15.39:47354) 2024-09-21T07:54:44.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.586+0000 7f684fe00640 1 -- 172.21.15.39:0/2425128119 learned_addr learned my addr 172.21.15.39:0/2425128119 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:44.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.586+0000 7f6856a00640 1 --2- 172.21.15.39:0/2425128119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f685806d4f0 0x7f685815cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:44.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.586+0000 7f6857400640 1 --2- 172.21.15.39:0/2425128119 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68580d0030 0x7f685815dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:44.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.586+0000 7f684fe00640 1 -- 172.21.15.39:0/2425128119 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68580d0030 msgr2=0x7f685815dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:44.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.586+0000 7f684fe00640 1 --2- 172.21.15.39:0/2425128119 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68580d0030 0x7f685815dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.586+0000 7f684fe00640 1 -- 172.21.15.39:0/2425128119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f685806d4f0 msgr2=0x7f685815cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:44.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.586+0000 7f684fe00640 1 --2- 172.21.15.39:0/2425128119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f685806d4f0 0x7f685815cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.586+0000 7f684fe00640 1 -- 172.21.15.39:0/2425128119 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68581f64e0 con 0x7f68580b49a0 2024-09-21T07:54:44.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.586+0000 7f684fe00640 1 --2- 172.21.15.39:0/2425128119 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68580b49a0 0x7f68581f59d0 secure :-1 s=READY pgs=2826 cs=0 l=1 rev1=1 crypto rx=0x7f684000c400 tx=0x7f684000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:44.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.587+0000 7f684f400640 1 -- 172.21.15.39:0/2425128119 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6840017070 con 0x7f68580b49a0 2024-09-21T07:54:44.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.587+0000 7f684f400640 1 -- 172.21.15.39:0/2425128119 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68400053c0 con 0x7f68580b49a0 2024-09-21T07:54:44.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.587+0000 7f6857e00640 1 -- 172.21.15.39:0/2425128119 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68581f67c0 con 0x7f68580b49a0 2024-09-21T07:54:44.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.587+0000 7f684f400640 1 -- 172.21.15.39:0/2425128119 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6840003850 con 0x7f68580b49a0 2024-09-21T07:54:44.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.587+0000 7f6857e00640 1 -- 172.21.15.39:0/2425128119 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6858159a70 con 0x7f68580b49a0 2024-09-21T07:54:44.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.588+0000 7f6857e00640 1 -- 172.21.15.39:0/2425128119 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68580b4460 con 0x7f68580b49a0 2024-09-21T07:54:44.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.589+0000 7f684f400640 1 -- 172.21.15.39:0/2425128119 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f684001a070 con 0x7f68580b49a0 2024-09-21T07:54:44.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.590+0000 7f684f400640 1 --2- 172.21.15.39:0/2425128119 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6834078b90 0x7f683407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:44.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.590+0000 7f684f400640 1 -- 172.21.15.39:0/2425128119 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(265..265 src has 1..265) ==== 7044+0+0 (secure 0 0 0) 0x7f684009a690 con 0x7f68580b49a0 2024-09-21T07:54:44.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.591+0000 7f6856a00640 1 --2- 172.21.15.39:0/2425128119 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6834078b90 0x7f683407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:44.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.592+0000 7f6856a00640 1 --2- 172.21.15.39:0/2425128119 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6834078b90 0x7f683407b0a0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f6848033000 tx=0x7f6848004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:44.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.595+0000 7f684f400640 1 -- 172.21.15.39:0/2425128119 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f68580b4db0 con 0x7f68580b49a0 2024-09-21T07:54:44.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.866+0000 7f6857e00640 1 -- 172.21.15.39:0/2425128119 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f6858159580 con 0x7f68580b49a0 2024-09-21T07:54:44.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.869+0000 7f684f400640 1 -- 172.21.15.39:0/2425128119 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v26) ==== 190+0+147 (secure 0 0 0) 0x7f6840062580 con 0x7f68580b49a0 2024-09-21T07:54:44.871 INFO:teuthology.orchestra.run.smithi039.stdout:[client.0] 2024-09-21T07:54:44.871 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQB0du5mGfb1DBAA/aSMKR5or3xy53rB/iUjpA== 2024-09-21T07:54:44.871 INFO:teuthology.orchestra.run.smithi039.stdout: caps mds = "allow" 2024-09-21T07:54:44.871 INFO:teuthology.orchestra.run.smithi039.stdout: caps mon = "allow r" 2024-09-21T07:54:44.871 INFO:teuthology.orchestra.run.smithi039.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T07:54:44.871 INFO:teuthology.orchestra.run.smithi039.stderr:updated caps for client.0 2024-09-21T07:54:44.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.872+0000 7f6857e00640 1 -- 172.21.15.39:0/2425128119 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6834078b90 msgr2=0x7f683407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:44.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.872+0000 7f6857e00640 1 --2- 172.21.15.39:0/2425128119 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6834078b90 0x7f683407b0a0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f6848033000 tx=0x7f6848004730 comp rx=0 tx=0).stop 2024-09-21T07:54:44.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.872+0000 7f6857e00640 1 -- 172.21.15.39:0/2425128119 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68580b49a0 msgr2=0x7f68581f59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:44.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.872+0000 7f6857e00640 1 --2- 172.21.15.39:0/2425128119 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68580b49a0 0x7f68581f59d0 secure :-1 s=READY pgs=2826 cs=0 l=1 rev1=1 crypto rx=0x7f684000c400 tx=0x7f684000c8c0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.872+0000 7f6857e00640 1 -- 172.21.15.39:0/2425128119 shutdown_connections 2024-09-21T07:54:44.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.872+0000 7f6857e00640 1 --2- 172.21.15.39:0/2425128119 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6834078b90 0x7f683407b0a0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.872+0000 7f6857e00640 1 --2- 172.21.15.39:0/2425128119 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f68580d0030 0x7f685815dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.872+0000 7f6857e00640 1 --2- 172.21.15.39:0/2425128119 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68580b49a0 0x7f68581f59d0 unknown :-1 s=CLOSED pgs=2826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.872+0000 7f6857e00640 1 --2- 172.21.15.39:0/2425128119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f685806d4f0 0x7f685815cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.872+0000 7f6857e00640 1 -- 172.21.15.39:0/2425128119 >> 172.21.15.39:0/2425128119 conn(0x7f68580d0cb0 msgr2=0x7f68580afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:44.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.872+0000 7f6857e00640 1 -- 172.21.15.39:0/2425128119 shutdown_connections 2024-09-21T07:54:44.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:44.872+0000 7f6857e00640 1 -- 172.21.15.39:0/2425128119 wait complete. 2024-09-21T07:54:44.897 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:54:45.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.124+0000 7f3046a00640 1 -- 172.21.15.39:0/179129717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f304015f170 msgr2=0x7f304015f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:45.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.124+0000 7f3046a00640 1 --2- 172.21.15.39:0/179129717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f304015f170 0x7f304015f580 secure :-1 s=READY pgs=2828 cs=0 l=1 rev1=1 crypto rx=0x7f302c005fd0 tx=0x7f302c02a540 comp rx=0 tx=0).stop 2024-09-21T07:54:45.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.124+0000 7f3046a00640 1 -- 172.21.15.39:0/179129717 shutdown_connections 2024-09-21T07:54:45.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.124+0000 7f3046a00640 1 --2- 172.21.15.39:0/179129717 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f304006fee0 0x7f30400702f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:45.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.124+0000 7f3046a00640 1 --2- 172.21.15.39:0/179129717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f304015f170 0x7f304015f580 unknown :-1 s=CLOSED pgs=2828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:45.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.124+0000 7f3046a00640 1 -- 172.21.15.39:0/179129717 >> 172.21.15.39:0/179129717 conn(0x7f30400b6f80 msgr2=0x7f304006def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:45.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.124+0000 7f3046a00640 1 -- 172.21.15.39:0/179129717 shutdown_connections 2024-09-21T07:54:45.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.124+0000 7f3046a00640 1 -- 172.21.15.39:0/179129717 wait complete. 2024-09-21T07:54:45.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.127+0000 7f3046a00640 1 Processor -- start 2024-09-21T07:54:45.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.127+0000 7f3046a00640 1 -- start start 2024-09-21T07:54:45.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.128+0000 7f3046a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f304006fee0 0x7f3040143db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:45.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.128+0000 7f3046a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f304015c570 0x7f30400b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:45.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.128+0000 7f3046a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f304015f170 0x7f30401e93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:45.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.128+0000 7f3046a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f30400cd2b0 con 0x7f304015f170 2024-09-21T07:54:45.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.128+0000 7f3046a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f30400cd130 con 0x7f304006fee0 2024-09-21T07:54:45.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.128+0000 7f3046a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f30400cd430 con 0x7f304015c570 2024-09-21T07:54:45.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.129+0000 7f3046000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f304015f170 0x7f30401e93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:45.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.129+0000 7f3046000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f304015f170 0x7f30401e93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47366/0 (socket says 172.21.15.39:47366) 2024-09-21T07:54:45.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.129+0000 7f3046000640 1 -- 172.21.15.39:0/1563458770 learned_addr learned my addr 172.21.15.39:0/1563458770 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:45.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.129+0000 7f3045600640 1 --2- 172.21.15.39:0/1563458770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f304006fee0 0x7f3040143db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:45.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.129+0000 7f3044c00640 1 --2- 172.21.15.39:0/1563458770 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f304015c570 0x7f30400b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:45.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.129+0000 7f3046000640 1 -- 172.21.15.39:0/1563458770 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f304015c570 msgr2=0x7f30400b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:45.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.129+0000 7f3046000640 1 --2- 172.21.15.39:0/1563458770 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f304015c570 0x7f30400b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:45.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.129+0000 7f3046000640 1 -- 172.21.15.39:0/1563458770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f304006fee0 msgr2=0x7f3040143db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:45.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.129+0000 7f3046000640 1 --2- 172.21.15.39:0/1563458770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f304006fee0 0x7f3040143db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:45.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.129+0000 7f3046000640 1 -- 172.21.15.39:0/1563458770 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30401583d0 con 0x7f304015f170 2024-09-21T07:54:45.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.129+0000 7f3046000640 1 --2- 172.21.15.39:0/1563458770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f304015f170 0x7f30401e93c0 secure :-1 s=READY pgs=2829 cs=0 l=1 rev1=1 crypto rx=0x7f303800e990 tx=0x7f303800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:45.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.130+0000 7f303d600640 1 -- 172.21.15.39:0/1563458770 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3038004d20 con 0x7f304015f170 2024-09-21T07:54:45.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.130+0000 7f303d600640 1 -- 172.21.15.39:0/1563458770 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f303800cb00 con 0x7f304015f170 2024-09-21T07:54:45.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.130+0000 7f303d600640 1 -- 172.21.15.39:0/1563458770 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3038005020 con 0x7f304015f170 2024-09-21T07:54:45.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.130+0000 7f3046a00640 1 -- 172.21.15.39:0/1563458770 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30401586b0 con 0x7f304015f170 2024-09-21T07:54:45.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.130+0000 7f3046a00640 1 -- 172.21.15.39:0/1563458770 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3040158c70 con 0x7f304015f170 2024-09-21T07:54:45.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.131+0000 7f3046a00640 1 -- 172.21.15.39:0/1563458770 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3040070490 con 0x7f304015f170 2024-09-21T07:54:45.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.132+0000 7f303d600640 1 -- 172.21.15.39:0/1563458770 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f30380051c0 con 0x7f304015f170 2024-09-21T07:54:45.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.133+0000 7f303d600640 1 --2- 172.21.15.39:0/1563458770 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3020078b90 0x7f302007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:45.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.133+0000 7f303d600640 1 -- 172.21.15.39:0/1563458770 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(265..265 src has 1..265) ==== 7044+0+0 (secure 0 0 0) 0x7f303809bb10 con 0x7f304015f170 2024-09-21T07:54:45.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.134+0000 7f3045600640 1 --2- 172.21.15.39:0/1563458770 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3020078b90 0x7f302007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:45.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.135+0000 7f3045600640 1 --2- 172.21.15.39:0/1563458770 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3020078b90 0x7f302007b0a0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f3034000f60 tx=0x7f3034009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:45.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.138+0000 7f303d600640 1 -- 172.21.15.39:0/1563458770 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3038016030 con 0x7f304015f170 2024-09-21T07:54:45.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.374+0000 7f3046a00640 1 -- 172.21.15.39:0/1563458770 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30400cd950 con 0x7f304015f170 2024-09-21T07:54:45.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.375+0000 7f303d600640 1 -- 172.21.15.39:0/1563458770 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 310 v310) ==== 95+0+5243 (secure 0 0 0) 0x7f3038063450 con 0x7f304015f170 2024-09-21T07:54:45.376 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:45.376 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":310,"btime":"2024-09-21T07:54:42:307372+0000","default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9725,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1018044536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1018044536},{"type":"v1","addr":"172.21.15.148:6837","nonce":1018044536}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":305},{"gid":11125,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1771887993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1771887993},{"type":"v1","addr":"172.21.15.148:6836","nonce":1771887993}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":304},{"gid":11131,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/649113662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":649113662},{"type":"v1","addr":"172.21.15.39:6835","nonce":649113662}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":305}],"filesystems":[{"mdsmap":{"epoch":310,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:54:38.804512+0000","modified":"2024-09-21T07:54:41.814517+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":12315},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12315":{"gid":12315,"name":"c","rank":0,"incarnation":306,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/3094159467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3094159467},{"type":"v1","addr":"172.21.15.39:6834","nonce":3094159467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":12315,"qdb_cluster":[12315]},"id":14}]} 2024-09-21T07:54:45.377 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 310 2024-09-21T07:54:45.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.378+0000 7f3046a00640 1 -- 172.21.15.39:0/1563458770 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3020078b90 msgr2=0x7f302007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:45.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.378+0000 7f3046a00640 1 --2- 172.21.15.39:0/1563458770 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3020078b90 0x7f302007b0a0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f3034000f60 tx=0x7f3034009040 comp rx=0 tx=0).stop 2024-09-21T07:54:45.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.378+0000 7f3046a00640 1 -- 172.21.15.39:0/1563458770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f304015f170 msgr2=0x7f30401e93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:45.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.378+0000 7f3046a00640 1 --2- 172.21.15.39:0/1563458770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f304015f170 0x7f30401e93c0 secure :-1 s=READY pgs=2829 cs=0 l=1 rev1=1 crypto rx=0x7f303800e990 tx=0x7f303800ee50 comp rx=0 tx=0).stop 2024-09-21T07:54:45.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.378+0000 7f3046a00640 1 -- 172.21.15.39:0/1563458770 shutdown_connections 2024-09-21T07:54:45.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.378+0000 7f3046a00640 1 --2- 172.21.15.39:0/1563458770 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3020078b90 0x7f302007b0a0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:45.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.378+0000 7f3046a00640 1 --2- 172.21.15.39:0/1563458770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f304015f170 0x7f30401e93c0 unknown :-1 s=CLOSED pgs=2829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:45.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.378+0000 7f3046a00640 1 --2- 172.21.15.39:0/1563458770 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f304015c570 0x7f30400b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:45.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.378+0000 7f3046a00640 1 --2- 172.21.15.39:0/1563458770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f304006fee0 0x7f3040143db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:45.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.378+0000 7f3046a00640 1 -- 172.21.15.39:0/1563458770 >> 172.21.15.39:0/1563458770 conn(0x7f30400b6f80 msgr2=0x7f3040074370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:45.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.378+0000 7f3046a00640 1 -- 172.21.15.39:0/1563458770 shutdown_connections 2024-09-21T07:54:45.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:45.378+0000 7f3046a00640 1 -- 172.21.15.39:0/1563458770 wait complete. 2024-09-21T07:54:45.803 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 310, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T07:54:38.804512+0000', 'modified': '2024-09-21T07:54:41.814517+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 12315}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_12315': {'gid': 12315, 'name': 'c', 'rank': 0, 'incarnation': 306, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.39:6834/3094159467', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 3094159467}, {'type': 'v1', 'addr': '172.21.15.39:6834', 'nonce': 3094159467}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [32], 'metadata_pool': 31, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 12315, 'qdb_cluster': [12315]} 2024-09-21T07:54:45.803 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T07:54:45.804 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T07:54:46.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.032+0000 7fe552e00640 1 -- 172.21.15.39:0/3959664867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe54c15cc00 msgr2=0x7fe54c15d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:46.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.032+0000 7fe552e00640 1 --2- 172.21.15.39:0/3959664867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe54c15cc00 0x7fe54c15d010 secure :-1 s=READY pgs=2831 cs=0 l=1 rev1=1 crypto rx=0x7fe54c0d0170 tx=0x7fe53802a5f0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.032+0000 7fe552e00640 1 -- 172.21.15.39:0/3959664867 shutdown_connections 2024-09-21T07:54:46.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.032+0000 7fe552e00640 1 --2- 172.21.15.39:0/3959664867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe54c15cc00 0x7fe54c15d010 unknown :-1 s=CLOSED pgs=2831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.032+0000 7fe552e00640 1 --2- 172.21.15.39:0/3959664867 >> v2:172.21.15.148:3300/0 conn(0x7fe54c0cfa20 0x7fe54c0cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.032+0000 7fe552e00640 1 -- 172.21.15.39:0/3959664867 >> 172.21.15.39:0/3959664867 conn(0x7fe54c06cfc0 msgr2=0x7fe54c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:46.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.032+0000 7fe552e00640 1 -- 172.21.15.39:0/3959664867 shutdown_connections 2024-09-21T07:54:46.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.032+0000 7fe552e00640 1 -- 172.21.15.39:0/3959664867 wait complete. 2024-09-21T07:54:46.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.034+0000 7fe552e00640 1 Processor -- start 2024-09-21T07:54:46.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.034+0000 7fe552e00640 1 -- start start 2024-09-21T07:54:46.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.036+0000 7fe552e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe54c0cfa20 0x7fe54c0ae0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:46.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.036+0000 7fe552e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe54c0d04b0 0x7fe54c13cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:46.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.036+0000 7fe552e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe54c15cc00 0x7fe54c0b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:46.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.036+0000 7fe552e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c06ee70 con 0x7fe54c15cc00 2024-09-21T07:54:46.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.036+0000 7fe552e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c06ecf0 con 0x7fe54c0cfa20 2024-09-21T07:54:46.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.036+0000 7fe552e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe54c06eff0 con 0x7fe54c0d04b0 2024-09-21T07:54:46.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.036+0000 7fe552400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe54c15cc00 0x7fe54c0b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:46.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.036+0000 7fe552400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe54c15cc00 0x7fe54c0b32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47382/0 (socket says 172.21.15.39:47382) 2024-09-21T07:54:46.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.036+0000 7fe552400640 1 -- 172.21.15.39:0/1125717888 learned_addr learned my addr 172.21.15.39:0/1125717888 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:46.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.036+0000 7fe551a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe54c0cfa20 0x7fe54c0ae0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:46.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.036+0000 7fe551000640 1 --2- 172.21.15.39:0/1125717888 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe54c0d04b0 0x7fe54c13cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:46.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.037+0000 7fe552400640 1 -- 172.21.15.39:0/1125717888 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe54c0d04b0 msgr2=0x7fe54c13cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:46.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.037+0000 7fe552400640 1 --2- 172.21.15.39:0/1125717888 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe54c0d04b0 0x7fe54c13cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.037+0000 7fe552400640 1 -- 172.21.15.39:0/1125717888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe54c0cfa20 msgr2=0x7fe54c0ae0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:46.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.037+0000 7fe552400640 1 --2- 172.21.15.39:0/1125717888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe54c0cfa20 0x7fe54c0ae0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.037+0000 7fe552400640 1 -- 172.21.15.39:0/1125717888 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe54c13ab70 con 0x7fe54c15cc00 2024-09-21T07:54:46.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.037+0000 7fe552400640 1 --2- 172.21.15.39:0/1125717888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe54c15cc00 0x7fe54c0b32f0 secure :-1 s=READY pgs=2832 cs=0 l=1 rev1=1 crypto rx=0x7fe54400c730 tx=0x7fe54400cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:46.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.037+0000 7fe549600640 1 -- 172.21.15.39:0/1125717888 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe544003b70 con 0x7fe54c15cc00 2024-09-21T07:54:46.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.037+0000 7fe549600640 1 -- 172.21.15.39:0/1125717888 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe544003d10 con 0x7fe54c15cc00 2024-09-21T07:54:46.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.037+0000 7fe549600640 1 -- 172.21.15.39:0/1125717888 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5440052d0 con 0x7fe54c15cc00 2024-09-21T07:54:46.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.037+0000 7fe552e00640 1 -- 172.21.15.39:0/1125717888 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe54c13ae50 con 0x7fe54c15cc00 2024-09-21T07:54:46.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.038+0000 7fe552e00640 1 -- 172.21.15.39:0/1125717888 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe54c13b410 con 0x7fe54c15cc00 2024-09-21T07:54:46.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.041+0000 7fe549600640 1 -- 172.21.15.39:0/1125717888 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe544005470 con 0x7fe54c15cc00 2024-09-21T07:54:46.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.042+0000 7fe549600640 1 --2- 172.21.15.39:0/1125717888 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe52c078bc0 0x7fe52c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:46.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.042+0000 7fe55b5cc740 1 Processor -- start 2024-09-21T07:54:46.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.042+0000 7fe549600640 1 -- 172.21.15.39:0/1125717888 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(265..265 src has 1..265) ==== 7044+0+0 (secure 0 0 0) 0x7fe54409af90 con 0x7fe54c15cc00 2024-09-21T07:54:46.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.042+0000 7fe55b5cc740 1 -- start start 2024-09-21T07:54:46.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.043+0000 7fe551a00640 1 --2- 172.21.15.39:0/1125717888 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe52c078bc0 0x7fe52c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:46.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.043+0000 7fe551a00640 1 --2- 172.21.15.39:0/1125717888 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe52c078bc0 0x7fe52c07b0d0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fe54000bb50 tx=0x7fe54000b2e0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:46.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.044+0000 7fe55b5cc740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562bef7b6270 0x562befb86b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:46.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.044+0000 7fe55b5cc740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562bef78d3b0 0x562befb870a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:46.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.044+0000 7fe55b5cc740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562bef795590 0x562befb57150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:46.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.044+0000 7fe55b5cc740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562befae0430 con 0x562bef7b6270 2024-09-21T07:54:46.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.044+0000 7fe55b5cc740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562befad05d0 con 0x562bef78d3b0 2024-09-21T07:54:46.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.044+0000 7fe55b5cc740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x562befabfa00 con 0x562bef795590 2024-09-21T07:54:46.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.044+0000 7fe551a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562bef7b6270 0x562befb86b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:46.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.044+0000 7fe551a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562bef7b6270 0x562befb86b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47390/0 (socket says 172.21.15.39:47390) 2024-09-21T07:54:46.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.044+0000 7fe551a00640 1 -- 172.21.15.39:0/277496232 learned_addr learned my addr 172.21.15.39:0/277496232 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:46.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.044+0000 7fe552400640 1 --2- 172.21.15.39:0/277496232 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562bef795590 0x562befb57150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:46.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.044+0000 7fe551000640 1 --2- 172.21.15.39:0/277496232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562bef78d3b0 0x562befb870a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:46.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.045+0000 7fe551a00640 1 -- 172.21.15.39:0/277496232 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562bef795590 msgr2=0x562befb57150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:46.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.045+0000 7fe551a00640 1 --2- 172.21.15.39:0/277496232 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562bef795590 0x562befb57150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.045+0000 7fe551a00640 1 -- 172.21.15.39:0/277496232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562bef78d3b0 msgr2=0x562befb870a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:46.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.045+0000 7fe551a00640 1 --2- 172.21.15.39:0/277496232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562bef78d3b0 0x562befb870a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.045+0000 7fe551a00640 1 -- 172.21.15.39:0/277496232 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562befa88520 con 0x562bef7b6270 2024-09-21T07:54:46.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.045+0000 7fe552400640 1 --2- 172.21.15.39:0/277496232 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562bef795590 0x562befb57150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:54:46.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.045+0000 7fe551a00640 1 --2- 172.21.15.39:0/277496232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562bef7b6270 0x562befb86b60 secure :-1 s=READY pgs=2833 cs=0 l=1 rev1=1 crypto rx=0x7fe540005d00 tx=0x7fe540013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14401ec2aa46c225 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:46.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.045+0000 7fe53e000640 1 -- 172.21.15.39:0/277496232 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54001e620 con 0x562bef7b6270 2024-09-21T07:54:46.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.045+0000 7fe53e000640 1 -- 172.21.15.39:0/277496232 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe540011e50 con 0x562bef7b6270 2024-09-21T07:54:46.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.045+0000 7fe53e000640 1 -- 172.21.15.39:0/277496232 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54001eaf0 con 0x562bef7b6270 2024-09-21T07:54:46.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.046+0000 7fe55b5cc740 1 -- 172.21.15.39:0/277496232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562bef7b6270 msgr2=0x562befb86b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:46.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.046+0000 7fe55b5cc740 1 --2- 172.21.15.39:0/277496232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562bef7b6270 0x562befb86b60 secure :-1 s=READY pgs=2833 cs=0 l=1 rev1=1 crypto rx=0x7fe540005d00 tx=0x7fe540013000 comp rx=0 tx=0).stop 2024-09-21T07:54:46.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.046+0000 7fe55b5cc740 1 -- 172.21.15.39:0/277496232 shutdown_connections 2024-09-21T07:54:46.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.046+0000 7fe55b5cc740 1 --2- 172.21.15.39:0/277496232 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562bef795590 0x562befb57150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.046+0000 7fe55b5cc740 1 --2- 172.21.15.39:0/277496232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562bef78d3b0 0x562befb870a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.046+0000 7fe55b5cc740 1 --2- 172.21.15.39:0/277496232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562bef7b6270 0x562befb86b60 unknown :-1 s=CLOSED pgs=2833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.046+0000 7fe55b5cc740 1 -- 172.21.15.39:0/277496232 >> 172.21.15.39:0/277496232 conn(0x562bef7aa3c0 msgr2=0x562befb5cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:46.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.046+0000 7fe55b5cc740 1 -- 172.21.15.39:0/277496232 shutdown_connections 2024-09-21T07:54:46.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.046+0000 7fe55b5cc740 1 -- 172.21.15.39:0/277496232 wait complete. 2024-09-21T07:54:46.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.047+0000 7fe55b5cc740 1 -- start start 2024-09-21T07:54:46.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.047+0000 7fe55b5cc740 1 Processor -- start 2024-09-21T07:54:46.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.049+0000 7fe55b5cc740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562bef78d3b0 0x562befb999e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:46.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.049+0000 7fe55b5cc740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562bef795590 0x562befb58030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:46.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.049+0000 7fe55b5cc740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562bef7b6270 0x562befb9b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:46.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.049+0000 7fe55b5cc740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562befa66330 con 0x562bef7b6270 2024-09-21T07:54:46.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.049+0000 7fe55b5cc740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562befac3810 con 0x562bef78d3b0 2024-09-21T07:54:46.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.049+0000 7fe55b5cc740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x562bef830620 con 0x562bef795590 2024-09-21T07:54:46.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.049+0000 7fe55b5cc740 10 client.0 _ll_register_callbacks cb 0x562befa6b420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:54:46.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.049+0000 7fe552400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562bef7b6270 0x562befb9b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:46.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.049+0000 7fe552400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562bef7b6270 0x562befb9b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47394/0 (socket says 172.21.15.39:47394) 2024-09-21T07:54:46.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.049+0000 7fe552400640 1 -- 172.21.15.39:0/2169476210 learned_addr learned my addr 172.21.15.39:0/2169476210 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:46.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.049+0000 7fe551a00640 1 --2- 172.21.15.39:0/2169476210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562bef78d3b0 0x562befb999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:46.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.049+0000 7fe551000640 1 --2- 172.21.15.39:0/2169476210 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562bef795590 0x562befb58030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:46.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.050+0000 7fe552400640 1 -- 172.21.15.39:0/2169476210 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562bef795590 msgr2=0x562befb58030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:46.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.050+0000 7fe552400640 1 --2- 172.21.15.39:0/2169476210 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562bef795590 0x562befb58030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.050+0000 7fe552400640 1 -- 172.21.15.39:0/2169476210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562bef78d3b0 msgr2=0x562befb999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:46.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.050+0000 7fe552400640 1 --2- 172.21.15.39:0/2169476210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562bef78d3b0 0x562befb999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.050+0000 7fe552400640 1 -- 172.21.15.39:0/2169476210 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bef821850 con 0x562bef7b6270 2024-09-21T07:54:46.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.050+0000 7fe552400640 1 --2- 172.21.15.39:0/2169476210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562bef7b6270 0x562befb9b520 secure :-1 s=READY pgs=2834 cs=0 l=1 rev1=1 crypto rx=0x7fe5440624a0 tx=0x7fe5440622b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b263e8d9310b1de6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:46.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.050+0000 7fe532a00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:54:46.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.050+0000 7fe532a00640 1 -- 172.21.15.39:0/2169476210 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54406c110 con 0x562bef7b6270 2024-09-21T07:54:46.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.050+0000 7fe532a00640 1 -- 172.21.15.39:0/2169476210 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(265..265 src has 1..265) ==== 7044+0+0 (secure 0 0 0) 0x7fe544061a00 con 0x562bef7b6270 2024-09-21T07:54:46.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.050+0000 7fe55b5cc740 1 -- 172.21.15.39:0/2169476210 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe540011e50 con 0x562bef7b6270 2024-09-21T07:54:46.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.050+0000 7fe532a00640 1 -- 172.21.15.39:0/2169476210 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=310) ==== 24+0+0 (secure 0 0 0) 0x7fe544061c00 con 0x562bef7b6270 2024-09-21T07:54:46.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.050+0000 7fe55b5cc740 10 client.12450 fetch_fsmap learned FSMap version 310 2024-09-21T07:54:46.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.051+0000 7fe55b5cc740 1 -- 172.21.15.39:0/2169476210 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=310,monmap=2+}) -- 0x562bef8986b0 con 0x562bef7b6270 2024-09-21T07:54:46.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.051+0000 7fe532a00640 1 -- 172.21.15.39:0/2169476210 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe540011e50 con 0x562bef7b6270 2024-09-21T07:54:46.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.051+0000 7fe532a00640 1 -- 172.21.15.39:0/2169476210 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 310) ==== 3502+0+0 (secure 0 0 0) 0x7fe54405e0f0 con 0x562bef7b6270 2024-09-21T07:54:46.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.051+0000 7fe55b5cc740 10 client.12450 fetch_fsmap finished waiting for FSMap version 310 2024-09-21T07:54:46.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.051+0000 7fe55b5cc740 10 client.12450 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:54:46.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.051+0000 7fe55b5cc740 20 client.12450 populate_metadata read hostname 'smithi039' 2024-09-21T07:54:46.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.051+0000 7fe55b5cc740 1 --2- 172.21.15.39:0/2169476210 >> [v2:172.21.15.39:6832/3094159467,v1:172.21.15.39:6834/3094159467] conn(0x562bef774e40 0x562befb7eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:46.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.051+0000 7fe55b5cc740 4 client.12450 mds_command: new command op to 12315 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:54:46.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.051+0000 7fe55b5cc740 1 -- 172.21.15.39:0/2169476210 --> [v2:172.21.15.39:6832/3094159467,v1:172.21.15.39:6834/3094159467] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562befad2520 con 0x562bef774e40 2024-09-21T07:54:46.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.051+0000 7fe551a00640 1 --2- 172.21.15.39:0/2169476210 >> [v2:172.21.15.39:6832/3094159467,v1:172.21.15.39:6834/3094159467] conn(0x562bef774e40 0x562befb7eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:46.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.052+0000 7fe532a00640 10 client.12450 ms_handle_connect on v2:172.21.15.39:6832/3094159467 2024-09-21T07:54:46.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.052+0000 7fe551a00640 1 --2- 172.21.15.39:0/2169476210 >> [v2:172.21.15.39:6832/3094159467,v1:172.21.15.39:6834/3094159467] conn(0x562bef774e40 0x562befb7eda0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d4b32ad32ad376b server_cookie=6eb57f4a609ba4d7 in_seq=0 out_seq=0 2024-09-21T07:54:46.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.053+0000 7fe532a00640 1 -- 172.21.15.39:0/2169476210 <== mds.0 v2:172.21.15.39:6832/3094159467 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562befad2520 con 0x562bef774e40 2024-09-21T07:54:46.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.053+0000 7fe532a00640 10 client.12450 handle_command_reply: tid=0 2024-09-21T07:54:46.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.053+0000 7fe55b5cc740 1 client.12450 shutdown 2024-09-21T07:54:46.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.053+0000 7fe55b5cc740 1 -- 172.21.15.39:0/2169476210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562bef7b6270 msgr2=0x562befb9b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:46.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.053+0000 7fe55b5cc740 1 --2- 172.21.15.39:0/2169476210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562bef7b6270 0x562befb9b520 secure :-1 s=READY pgs=2834 cs=0 l=1 rev1=1 crypto rx=0x7fe5440624a0 tx=0x7fe5440622b0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.053+0000 7fe55b5cc740 1 -- 172.21.15.39:0/2169476210 shutdown_connections 2024-09-21T07:54:46.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.053+0000 7fe55b5cc740 1 --2- 172.21.15.39:0/2169476210 >> [v2:172.21.15.39:6832/3094159467,v1:172.21.15.39:6834/3094159467] conn(0x562bef774e40 0x562befb7eda0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.053+0000 7fe55b5cc740 1 --2- 172.21.15.39:0/2169476210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562bef7b6270 0x562befb9b520 unknown :-1 s=CLOSED pgs=2834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.053+0000 7fe55b5cc740 1 --2- 172.21.15.39:0/2169476210 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562bef795590 0x562befb58030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.053+0000 7fe55b5cc740 1 --2- 172.21.15.39:0/2169476210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562bef78d3b0 0x562befb999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.053+0000 7fe55b5cc740 1 -- 172.21.15.39:0/2169476210 >> 172.21.15.39:0/2169476210 conn(0x562bef7aa3c0 msgr2=0x562befb981f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:46.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.053+0000 7fe532a00640 0 client.12450 ms_handle_reset on v2:172.21.15.39:6832/3094159467 2024-09-21T07:54:46.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.053+0000 7fe55b5cc740 1 -- 172.21.15.39:0/2169476210 shutdown_connections 2024-09-21T07:54:46.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.053+0000 7fe55b5cc740 1 -- 172.21.15.39:0/2169476210 wait complete. 2024-09-21T07:54:46.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.053+0000 7fe55b5cc740 20 client.12450 trim_cache size 0 max 16384 2024-09-21T07:54:46.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.073+0000 7fe55b5cc740 1 Processor -- start 2024-09-21T07:54:46.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.073+0000 7fe55b5cc740 1 -- start start 2024-09-21T07:54:46.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.075+0000 7fe55b5cc740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562bef78d3b0 0x562befba2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:46.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.075+0000 7fe55b5cc740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562bef795590 0x562befb9aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:46.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.075+0000 7fe55b5cc740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562bef7b6270 0x562befb7c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:46.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.075+0000 7fe55b5cc740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c2038d0 con 0x562bef7b6270 2024-09-21T07:54:46.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.075+0000 7fe55b5cc740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c13c240 con 0x562bef795590 2024-09-21T07:54:46.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.075+0000 7fe55b5cc740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe54c2034a0 con 0x562bef78d3b0 2024-09-21T07:54:46.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.075+0000 7fe552400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562bef7b6270 0x562befb7c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:46.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.075+0000 7fe552400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562bef7b6270 0x562befb7c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47396/0 (socket says 172.21.15.39:47396) 2024-09-21T07:54:46.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.075+0000 7fe552400640 1 -- 172.21.15.39:0/2304325237 learned_addr learned my addr 172.21.15.39:0/2304325237 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:46.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.075+0000 7fe551000640 1 --2- 172.21.15.39:0/2304325237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562bef795590 0x562befb9aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:46.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.075+0000 7fe551a00640 1 --2- 172.21.15.39:0/2304325237 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562bef78d3b0 0x562befba2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:46.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.075+0000 7fe552400640 1 -- 172.21.15.39:0/2304325237 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562bef78d3b0 msgr2=0x562befba2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:46.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.075+0000 7fe552400640 1 --2- 172.21.15.39:0/2304325237 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562bef78d3b0 0x562befba2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.075+0000 7fe552400640 1 -- 172.21.15.39:0/2304325237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562bef795590 msgr2=0x562befb9aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:46.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.075+0000 7fe552400640 1 --2- 172.21.15.39:0/2304325237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562bef795590 0x562befb9aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.075+0000 7fe552400640 1 -- 172.21.15.39:0/2304325237 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562befb5d840 con 0x562bef7b6270 2024-09-21T07:54:46.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.076+0000 7fe552400640 1 --2- 172.21.15.39:0/2304325237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562bef7b6270 0x562befb7c8f0 secure :-1 s=READY pgs=2835 cs=0 l=1 rev1=1 crypto rx=0x7fe54405ed20 tx=0x7fe54400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa978728f9fae008 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:46.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.076+0000 7fe53d600640 1 -- 172.21.15.39:0/2304325237 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54406c110 con 0x562bef7b6270 2024-09-21T07:54:46.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.076+0000 7fe53d600640 1 -- 172.21.15.39:0/2304325237 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe54406be90 con 0x562bef7b6270 2024-09-21T07:54:46.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.076+0000 7fe53d600640 1 -- 172.21.15.39:0/2304325237 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe544062570 con 0x562bef7b6270 2024-09-21T07:54:46.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.077+0000 7fe55b5cc740 1 -- 172.21.15.39:0/2304325237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562bef7b6270 msgr2=0x562befb7c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:46.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.077+0000 7fe55b5cc740 1 --2- 172.21.15.39:0/2304325237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562bef7b6270 0x562befb7c8f0 secure :-1 s=READY pgs=2835 cs=0 l=1 rev1=1 crypto rx=0x7fe54405ed20 tx=0x7fe54400cf90 comp rx=0 tx=0).stop 2024-09-21T07:54:46.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.077+0000 7fe55b5cc740 1 -- 172.21.15.39:0/2304325237 shutdown_connections 2024-09-21T07:54:46.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.077+0000 7fe55b5cc740 1 --2- 172.21.15.39:0/2304325237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562bef7b6270 0x562befb7c8f0 unknown :-1 s=CLOSED pgs=2835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.077+0000 7fe55b5cc740 1 --2- 172.21.15.39:0/2304325237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562bef795590 0x562befb9aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.077+0000 7fe55b5cc740 1 --2- 172.21.15.39:0/2304325237 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562bef78d3b0 0x562befba2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.077+0000 7fe55b5cc740 1 -- 172.21.15.39:0/2304325237 >> 172.21.15.39:0/2304325237 conn(0x562bef7aa3c0 msgr2=0x562befb7ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:46.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.077+0000 7fe55b5cc740 1 -- 172.21.15.39:0/2304325237 shutdown_connections 2024-09-21T07:54:46.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.077+0000 7fe55b5cc740 1 -- 172.21.15.39:0/2304325237 wait complete. 2024-09-21T07:54:46.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.078+0000 7fe55b5cc740 1 -- start start 2024-09-21T07:54:46.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.078+0000 7fe55b5cc740 1 Processor -- start 2024-09-21T07:54:46.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.079+0000 7fe55b5cc740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562bef78d3b0 0x562befc25d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:46.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.079+0000 7fe55b5cc740 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562bef795590 0x562befc21d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:46.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.079+0000 7fe55b5cc740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562bef7b6270 0x562befc185b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:46.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.079+0000 7fe55b5cc740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562befb81130 con 0x562bef78d3b0 2024-09-21T07:54:46.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.079+0000 7fe55b5cc740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562befb80c90 con 0x562bef7b6270 2024-09-21T07:54:46.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.079+0000 7fe55b5cc740 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x562befbb15f0 con 0x562bef795590 2024-09-21T07:54:46.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.079+0000 7fe55b5cc740 10 client.0 _ll_register_callbacks cb 0x7fe544061f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:54:46.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.080+0000 7fe551a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562bef78d3b0 0x562befc25d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:46.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.080+0000 7fe551a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562bef78d3b0 0x562befc25d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47408/0 (socket says 172.21.15.39:47408) 2024-09-21T07:54:46.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.080+0000 7fe551a00640 1 -- 172.21.15.39:0/1362193241 learned_addr learned my addr 172.21.15.39:0/1362193241 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:46.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.080+0000 7fe551a00640 1 -- 172.21.15.39:0/1362193241 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562bef795590 msgr2=0x562befc21d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:46.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.080+0000 7fe551a00640 1 --2- 172.21.15.39:0/1362193241 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562bef795590 0x562befc21d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.080+0000 7fe551a00640 1 -- 172.21.15.39:0/1362193241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562bef7b6270 msgr2=0x562befc185b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:46.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.080+0000 7fe551a00640 1 --2- 172.21.15.39:0/1362193241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562bef7b6270 0x562befc185b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.080+0000 7fe551a00640 1 -- 172.21.15.39:0/1362193241 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562befc26250 con 0x562bef78d3b0 2024-09-21T07:54:46.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.080+0000 7fe551a00640 1 --2- 172.21.15.39:0/1362193241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562bef78d3b0 0x562befc25d10 secure :-1 s=READY pgs=2836 cs=0 l=1 rev1=1 crypto rx=0x7fe540005cd0 tx=0x7fe5400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7778a24143e82e3a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:46.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.080+0000 7fe532a00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-09-21T07:54:46.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.081+0000 7fe532a00640 1 -- 172.21.15.39:0/1362193241 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54003f020 con 0x562bef78d3b0 2024-09-21T07:54:46.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.081+0000 7fe532a00640 1 -- 172.21.15.39:0/1362193241 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(265..265 src has 1..265) ==== 7044+0+0 (secure 0 0 0) 0x7fe54001eab0 con 0x562bef78d3b0 2024-09-21T07:54:46.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.081+0000 7fe55b5cc740 1 -- 172.21.15.39:0/1362193241 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe54406be90 con 0x562bef78d3b0 2024-09-21T07:54:46.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.081+0000 7fe532a00640 1 -- 172.21.15.39:0/1362193241 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=310) ==== 24+0+0 (secure 0 0 0) 0x7fe540049050 con 0x562bef78d3b0 2024-09-21T07:54:46.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.081+0000 7fe55b5cc740 10 client.12456 fetch_fsmap learned FSMap version 310 2024-09-21T07:54:46.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.081+0000 7fe55b5cc740 1 -- 172.21.15.39:0/1362193241 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=310,monmap=2+}) -- 0x562befc22490 con 0x562bef78d3b0 2024-09-21T07:54:46.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.081+0000 7fe532a00640 1 -- 172.21.15.39:0/1362193241 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe54406be90 con 0x562bef78d3b0 2024-09-21T07:54:46.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.081+0000 7fe532a00640 1 -- 172.21.15.39:0/1362193241 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 310) ==== 3502+0+0 (secure 0 0 0) 0x7fe54004c020 con 0x562bef78d3b0 2024-09-21T07:54:46.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.081+0000 7fe55b5cc740 10 client.12456 fetch_fsmap finished waiting for FSMap version 310 2024-09-21T07:54:46.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.081+0000 7fe55b5cc740 10 client.12456 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:54:46.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.081+0000 7fe55b5cc740 20 client.12456 populate_metadata read hostname 'smithi039' 2024-09-21T07:54:46.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.081+0000 7fe55b5cc740 1 --2- 172.21.15.39:0/1362193241 >> [v2:172.21.15.39:6832/3094159467,v1:172.21.15.39:6834/3094159467] conn(0x562befb86c20 0x562befbaf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:46.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.081+0000 7fe55b5cc740 4 client.12456 mds_command: new command op to 12315 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:54:46.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.081+0000 7fe55b5cc740 1 -- 172.21.15.39:0/1362193241 --> [v2:172.21.15.39:6832/3094159467,v1:172.21.15.39:6834/3094159467] -- command(tid 0: {"prefix": "status"}) -- 0x562befacdfd0 con 0x562befb86c20 2024-09-21T07:54:46.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.082+0000 7fe551000640 1 --2- 172.21.15.39:0/1362193241 >> [v2:172.21.15.39:6832/3094159467,v1:172.21.15.39:6834/3094159467] conn(0x562befb86c20 0x562befbaf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:46.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.082+0000 7fe551000640 1 --2- 172.21.15.39:0/1362193241 >> [v2:172.21.15.39:6832/3094159467,v1:172.21.15.39:6834/3094159467] conn(0x562befb86c20 0x562befbaf760 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ece420a736ad7c9 server_cookie=bd12a0b52fa22419 in_seq=0 out_seq=0 2024-09-21T07:54:46.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.082+0000 7fe532a00640 10 client.12456 ms_handle_connect on v2:172.21.15.39:6832/3094159467 2024-09-21T07:54:46.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.082+0000 7fe532a00640 1 -- 172.21.15.39:0/1362193241 <== mds.0 v2:172.21.15.39:6832/3094159467 1 ==== command_reply(tid 0: 0 ) ==== 8+0+329 (crc 0 0 0) 0x562befacdfd0 con 0x562befb86c20 2024-09-21T07:54:46.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.082+0000 7fe532a00640 10 client.12456 handle_command_reply: tid=0 2024-09-21T07:54:46.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.082+0000 7fe55b5cc740 1 client.12456 shutdown 2024-09-21T07:54:46.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.083+0000 7fe55b5cc740 1 -- 172.21.15.39:0/1362193241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562bef78d3b0 msgr2=0x562befc25d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:46.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.083+0000 7fe55b5cc740 1 --2- 172.21.15.39:0/1362193241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562bef78d3b0 0x562befc25d10 secure :-1 s=READY pgs=2836 cs=0 l=1 rev1=1 crypto rx=0x7fe540005cd0 tx=0x7fe5400046c0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.083+0000 7fe55b5cc740 1 -- 172.21.15.39:0/1362193241 shutdown_connections 2024-09-21T07:54:46.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.083+0000 7fe55b5cc740 1 --2- 172.21.15.39:0/1362193241 >> [v2:172.21.15.39:6832/3094159467,v1:172.21.15.39:6834/3094159467] conn(0x562befb86c20 0x562befbaf760 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.083+0000 7fe55b5cc740 1 --2- 172.21.15.39:0/1362193241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562bef7b6270 0x562befc185b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.083+0000 7fe55b5cc740 1 --2- 172.21.15.39:0/1362193241 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x562bef795590 0x562befc21d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.083+0000 7fe55b5cc740 1 --2- 172.21.15.39:0/1362193241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562bef78d3b0 0x562befc25d10 unknown :-1 s=CLOSED pgs=2836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.083+0000 7fe532a00640 0 client.12456 ms_handle_reset on v2:172.21.15.39:6832/3094159467 2024-09-21T07:54:46.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.083+0000 7fe55b5cc740 1 -- 172.21.15.39:0/1362193241 >> 172.21.15.39:0/1362193241 conn(0x562bef7aa3c0 msgr2=0x562befb7ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:46.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.083+0000 7fe55b5cc740 1 -- 172.21.15.39:0/1362193241 shutdown_connections 2024-09-21T07:54:46.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.083+0000 7fe55b5cc740 1 -- 172.21.15.39:0/1362193241 wait complete. 2024-09-21T07:54:46.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.083+0000 7fe55b5cc740 20 client.12456 trim_cache size 0 max 16384 2024-09-21T07:54:46.087 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:54:46.087 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "7ab9b04a-f88d-49aa-a493-e5c076637ffb", 2024-09-21T07:54:46.087 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-09-21T07:54:46.087 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 12315, 2024-09-21T07:54:46.087 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-09-21T07:54:46.087 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-09-21T07:54:46.087 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-09-21T07:54:46.087 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 6.8111072769999996, 2024-09-21T07:54:46.087 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 310, 2024-09-21T07:54:46.087 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 265, 2024-09-21T07:54:46.087 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 265, 2024-09-21T07:54:46.087 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 11.814509291 2024-09-21T07:54:46.088 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:54:46.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.084+0000 7fe552e00640 1 -- 172.21.15.39:0/1125717888 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe52c078bc0 msgr2=0x7fe52c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:46.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.084+0000 7fe552e00640 1 --2- 172.21.15.39:0/1125717888 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe52c078bc0 0x7fe52c07b0d0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fe54000bb50 tx=0x7fe54000b2e0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.084+0000 7fe552e00640 1 -- 172.21.15.39:0/1125717888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe54c15cc00 msgr2=0x7fe54c0b32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:46.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.084+0000 7fe552e00640 1 --2- 172.21.15.39:0/1125717888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe54c15cc00 0x7fe54c0b32f0 secure :-1 s=READY pgs=2832 cs=0 l=1 rev1=1 crypto rx=0x7fe54400c730 tx=0x7fe54400cb30 comp rx=0 tx=0).stop 2024-09-21T07:54:46.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.084+0000 7fe552e00640 1 -- 172.21.15.39:0/1125717888 shutdown_connections 2024-09-21T07:54:46.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.084+0000 7fe552e00640 1 --2- 172.21.15.39:0/1125717888 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe52c078bc0 0x7fe52c07b0d0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.084+0000 7fe552e00640 1 --2- 172.21.15.39:0/1125717888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe54c15cc00 0x7fe54c0b32f0 unknown :-1 s=CLOSED pgs=2832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.084+0000 7fe552e00640 1 --2- 172.21.15.39:0/1125717888 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe54c0d04b0 0x7fe54c13cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.084+0000 7fe552e00640 1 --2- 172.21.15.39:0/1125717888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe54c0cfa20 0x7fe54c0ae0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.084+0000 7fe552e00640 1 -- 172.21.15.39:0/1125717888 >> 172.21.15.39:0/1125717888 conn(0x7fe54c06cfc0 msgr2=0x7fe54c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:46.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.084+0000 7fe552e00640 1 -- 172.21.15.39:0/1125717888 shutdown_connections 2024-09-21T07:54:46.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.084+0000 7fe552e00640 1 -- 172.21.15.39:0/1125717888 wait complete. 2024-09-21T07:54:46.110 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:54:46.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.340+0000 7fae47000640 1 -- 172.21.15.39:0/1071161481 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae400b30a0 msgr2=0x7fae400cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:46.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.340+0000 7fae47000640 1 --2- 172.21.15.39:0/1071161481 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae400b30a0 0x7fae400cb8a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fae30009860 tx=0x7fae3002a510 comp rx=0 tx=0).stop 2024-09-21T07:54:46.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.340+0000 7fae47000640 1 -- 172.21.15.39:0/1071161481 shutdown_connections 2024-09-21T07:54:46.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.340+0000 7fae47000640 1 --2- 172.21.15.39:0/1071161481 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae400b30a0 0x7fae400cb8a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.340+0000 7fae47000640 1 -- 172.21.15.39:0/1071161481 >> 172.21.15.39:0/1071161481 conn(0x7fae400b2460 msgr2=0x7fae4006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:46.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.340+0000 7fae47000640 1 -- 172.21.15.39:0/1071161481 shutdown_connections 2024-09-21T07:54:46.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.340+0000 7fae47000640 1 -- 172.21.15.39:0/1071161481 wait complete. 2024-09-21T07:54:46.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.342+0000 7fae47000640 1 Processor -- start 2024-09-21T07:54:46.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.342+0000 7fae47000640 1 -- start start 2024-09-21T07:54:46.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.344+0000 7fae47000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae400b30a0 0x7fae400c3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:46.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.344+0000 7fae47000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae400ccc60 0x7fae401f6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:46.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.344+0000 7fae47000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae4015d830 0x7fae401f7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:46.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.344+0000 7fae47000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fae400af5b0 con 0x7fae400b30a0 2024-09-21T07:54:46.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.344+0000 7fae47000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fae400af430 con 0x7fae400ccc60 2024-09-21T07:54:46.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.344+0000 7fae47000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fae400af730 con 0x7fae4015d830 2024-09-21T07:54:46.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.344+0000 7fae3fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae400b30a0 0x7fae400c3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:46.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.344+0000 7fae3fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae400b30a0 0x7fae400c3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47424/0 (socket says 172.21.15.39:47424) 2024-09-21T07:54:46.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.344+0000 7fae3fe00640 1 -- 172.21.15.39:0/1842743619 learned_addr learned my addr 172.21.15.39:0/1842743619 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:46.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.344+0000 7fae3f400640 1 --2- 172.21.15.39:0/1842743619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae400ccc60 0x7fae401f6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:46.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.344+0000 7fae45000640 1 --2- 172.21.15.39:0/1842743619 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae4015d830 0x7fae401f7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:46.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.344+0000 7fae3fe00640 1 -- 172.21.15.39:0/1842743619 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae4015d830 msgr2=0x7fae401f7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:46.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.344+0000 7fae3fe00640 1 --2- 172.21.15.39:0/1842743619 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae4015d830 0x7fae401f7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.344+0000 7fae3fe00640 1 -- 172.21.15.39:0/1842743619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae400ccc60 msgr2=0x7fae401f6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:46.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.344+0000 7fae3fe00640 1 --2- 172.21.15.39:0/1842743619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae400ccc60 0x7fae401f6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.344+0000 7fae3fe00640 1 -- 172.21.15.39:0/1842743619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae400c49d0 con 0x7fae400b30a0 2024-09-21T07:54:46.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.344+0000 7fae3f400640 1 --2- 172.21.15.39:0/1842743619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae400ccc60 0x7fae401f6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:54:46.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.344+0000 7fae45000640 1 --2- 172.21.15.39:0/1842743619 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae4015d830 0x7fae401f7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:54:46.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.345+0000 7fae3fe00640 1 --2- 172.21.15.39:0/1842743619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae400b30a0 0x7fae400c3d80 secure :-1 s=READY pgs=2838 cs=0 l=1 rev1=1 crypto rx=0x7fae3002ab00 tx=0x7fae30033c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:46.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.345+0000 7fae37e00640 1 -- 172.21.15.39:0/1842743619 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae30031d60 con 0x7fae400b30a0 2024-09-21T07:54:46.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.345+0000 7fae37e00640 1 -- 172.21.15.39:0/1842743619 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae30005020 con 0x7fae400b30a0 2024-09-21T07:54:46.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.345+0000 7fae47000640 1 -- 172.21.15.39:0/1842743619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae400c4cb0 con 0x7fae400b30a0 2024-09-21T07:54:46.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.345+0000 7fae37e00640 1 -- 172.21.15.39:0/1842743619 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae30005470 con 0x7fae400b30a0 2024-09-21T07:54:46.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.345+0000 7fae47000640 1 -- 172.21.15.39:0/1842743619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae400c5270 con 0x7fae400b30a0 2024-09-21T07:54:46.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.346+0000 7fae47000640 1 -- 172.21.15.39:0/1842743619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae400cb8a0 con 0x7fae400b30a0 2024-09-21T07:54:46.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.347+0000 7fae37e00640 1 -- 172.21.15.39:0/1842743619 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fae300047a0 con 0x7fae400b30a0 2024-09-21T07:54:46.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.348+0000 7fae37e00640 1 --2- 172.21.15.39:0/1842743619 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae20078b90 0x7fae2007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:46.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.349+0000 7fae37e00640 1 -- 172.21.15.39:0/1842743619 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(265..265 src has 1..265) ==== 7044+0+0 (secure 0 0 0) 0x7fae300bc5c0 con 0x7fae400b30a0 2024-09-21T07:54:46.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.349+0000 7fae3f400640 1 --2- 172.21.15.39:0/1842743619 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae20078b90 0x7fae2007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:46.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.350+0000 7fae3f400640 1 --2- 172.21.15.39:0/1842743619 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae20078b90 0x7fae2007b0a0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fae400c5da0 tx=0x7fae2c009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:46.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.353+0000 7fae37e00640 1 -- 172.21.15.39:0/1842743619 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fae3003f030 con 0x7fae400b30a0 2024-09-21T07:54:46.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.590+0000 7fae47000640 1 -- 172.21.15.39:0/1842743619 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fae40156e00 con 0x7fae400b30a0 2024-09-21T07:54:46.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.590+0000 7fae37e00640 1 -- 172.21.15.39:0/1842743619 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v310) ==== 53+0+73 (secure 0 0 0) 0x7fae30084420 con 0x7fae400b30a0 2024-09-21T07:54:46.591 INFO:teuthology.orchestra.run.smithi039.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:54:46.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.593+0000 7fae47000640 1 -- 172.21.15.39:0/1842743619 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae20078b90 msgr2=0x7fae2007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:46.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.593+0000 7fae47000640 1 --2- 172.21.15.39:0/1842743619 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae20078b90 0x7fae2007b0a0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fae400c5da0 tx=0x7fae2c009040 comp rx=0 tx=0).stop 2024-09-21T07:54:46.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.593+0000 7fae47000640 1 -- 172.21.15.39:0/1842743619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae400b30a0 msgr2=0x7fae400c3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:46.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.593+0000 7fae47000640 1 --2- 172.21.15.39:0/1842743619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae400b30a0 0x7fae400c3d80 secure :-1 s=READY pgs=2838 cs=0 l=1 rev1=1 crypto rx=0x7fae3002ab00 tx=0x7fae30033c90 comp rx=0 tx=0).stop 2024-09-21T07:54:46.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.593+0000 7fae47000640 1 -- 172.21.15.39:0/1842743619 shutdown_connections 2024-09-21T07:54:46.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.593+0000 7fae47000640 1 --2- 172.21.15.39:0/1842743619 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fae20078b90 0x7fae2007b0a0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.593+0000 7fae47000640 1 --2- 172.21.15.39:0/1842743619 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fae4015d830 0x7fae401f7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.593+0000 7fae47000640 1 --2- 172.21.15.39:0/1842743619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae400ccc60 0x7fae401f6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.593+0000 7fae47000640 1 --2- 172.21.15.39:0/1842743619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae400b30a0 0x7fae400c3d80 unknown :-1 s=CLOSED pgs=2838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.593+0000 7fae47000640 1 -- 172.21.15.39:0/1842743619 >> 172.21.15.39:0/1842743619 conn(0x7fae400b2460 msgr2=0x7fae40068d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:46.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.593+0000 7fae47000640 1 -- 172.21.15.39:0/1842743619 shutdown_connections 2024-09-21T07:54:46.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46.593+0000 7fae47000640 1 -- 172.21.15.39:0/1842743619 wait complete. 2024-09-21T07:54:46.615 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:54:46.615 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:54:46.616 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi039.front.sepia.ceph.com 2024-09-21T07:54:46.616 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T07:54:46.616 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:54:46.616 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:54:46.616 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:54:46.616 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:54:46.616 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:54:46.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:46.616 DEBUG:teuthology.orchestra.run.smithi039:> ip addr 2024-09-21T07:54:46.652 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:54:46.653 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:54:46.653 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:54:46.653 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:46.653 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2024-09-21T07:54:46.653 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:46.653 INFO:teuthology.orchestra.run.smithi039.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:54:46.653 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:54:46.653 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f0 2024-09-21T07:54:46.653 INFO:teuthology.orchestra.run.smithi039.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:54:46.653 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c7 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:54:46.653 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f1 2024-09-21T07:54:46.653 INFO:teuthology.orchestra.run.smithi039.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:54:46.653 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:70 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:54:46.653 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f0 2024-09-21T07:54:46.654 INFO:teuthology.orchestra.run.smithi039.stdout: inet 172.21.15.39/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T07:54:46.654 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft 41432sec preferred_lft 41432sec 2024-09-21T07:54:46.654 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec4:7aff:febd:1370/64 scope link 2024-09-21T07:54:46.654 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:46.654 INFO:teuthology.orchestra.run.smithi039.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:54:46.654 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:71 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:54:46.654 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f1 2024-09-21T07:54:46.654 INFO:teuthology.orchestra.run.smithi039.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:54:46.654 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff 2024-09-21T07:54:46.654 INFO:teuthology.orchestra.run.smithi039.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:54:46.654 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:46.654 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::b079:ddff:fe0e:6333/64 scope link 2024-09-21T07:54:46.654 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:46.655 INFO:teuthology.orchestra.run.smithi039.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:54:46.655 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:54:46.655 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::2ca7:98ff:fe8a:d0ed/64 scope link 2024-09-21T07:54:46.655 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:46.655 INFO:teuthology.orchestra.run.smithi039.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:54:46.655 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether ee:10:53:01:0c:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:54:46.655 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec10:53ff:fe01:c44/64 scope link 2024-09-21T07:54:46.655 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:46.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:46.655 DEBUG:teuthology.orchestra.run.smithi039:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:54:46.746 INFO:teuthology.orchestra.run.smithi039.stdout:1 2024-09-21T07:54:46.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:46.749 DEBUG:teuthology.orchestra.run.smithi039:> route 2024-09-21T07:54:46.785 INFO:teuthology.orchestra.run.smithi039.stdout:Kernel IP routing table 2024-09-21T07:54:46.785 INFO:teuthology.orchestra.run.smithi039.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:54:46.785 INFO:teuthology.orchestra.run.smithi039.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T07:54:46.785 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T07:54:46.785 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:54:46.786 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:54:46.786 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-21T07:54:46.786 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T07:54:46.786 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T07:54:46.786 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T07:54:46.786 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-21T07:54:46.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:54:46.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:54:46.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:46.984 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-09-21T07:54:47.019 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:54:47.020 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T07:54:47.020 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:54:47.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:47.020 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-09-21T07:54:47.117 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:54:47.117 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T07:54:47.152 INFO:teuthology.orchestra.run.smithi039.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:54:47.152 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:54:47.152 DEBUG:teuthology.orchestra.run.smithi039:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:54:47.226 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-21T07:54:47.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:47.226 DEBUG:teuthology.orchestra.run.smithi039:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:54:47.428 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:54:47.429 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:54:47.461 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:54:47.461 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:54:47.461 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: resolved to: "172.21.15.39:3300,172.21.15.148:3300,172.21.15.148:3301" 2024-09-21T07:54:47.461 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: trying mount with new device syntax: 0@7ab9b04a-f88d-49aa-a493-e5c076637ffb.cephfs=/ 2024-09-21T07:54:47.462 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.39:3300/172.21.15.148:3300/172.21.15.148:3301" will pass to kernel 2024-09-21T07:54:47.463 INFO:teuthology.orchestra.run.smithi039.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T07:54:47.463 INFO:teuthology.orchestra.run.smithi039.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:54:47.463 INFO:teuthology.orchestra.run.smithi039.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:54:47.463 INFO:teuthology.orchestra.run.smithi039.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:54:47.463 INFO:teuthology.orchestra.run.smithi039.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:54:47.466 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:54:47.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:47.467 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:54:47.522 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:54:47.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:54:47.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:54:47.621 INFO:teuthology.orchestra.run.smithi039.stdout:client12462 2024-09-21T07:54:47.621 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:54:47.621 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:47.657 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get mds_max_snaps_per_dir 2024-09-21T07:54:47.945 INFO:teuthology.orchestra.run.smithi039.stdout:{"mds_max_snaps_per_dir":"100"} 2024-09-21T07:54:47.945 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_max_snaps_per_dir": "100" } 2024-09-21T07:54:47.945 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'mkdir -p dir1/dir2') 2024-09-21T07:54:48.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:54:48.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:54:48.046 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:54:48.046 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:48.081 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:54:48.081 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:48.156 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:54:48.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.423+0000 7f7eba000640 1 -- 172.21.15.39:0/1549596088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7eb415d830 msgr2=0x7f7eb4156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:48.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.423+0000 7f7eba000640 1 --2- 172.21.15.39:0/1549596088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7eb415d830 0x7f7eb4156bb0 secure :-1 s=READY pgs=2840 cs=0 l=1 rev1=1 crypto rx=0x7f7eb406d880 tx=0x7f7e9c02a4d0 comp rx=0 tx=0).stop 2024-09-21T07:54:48.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.423+0000 7f7eba000640 1 -- 172.21.15.39:0/1549596088 shutdown_connections 2024-09-21T07:54:48.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.423+0000 7f7eba000640 1 --2- 172.21.15.39:0/1549596088 >> v2:172.21.15.148:3300/0 conn(0x7f7eb40ccc60 0x7f7eb40cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:48.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.423+0000 7f7eba000640 1 --2- 172.21.15.39:0/1549596088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7eb415d830 0x7f7eb4156bb0 unknown :-1 s=CLOSED pgs=2840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:48.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.423+0000 7f7eba000640 1 --2- 172.21.15.39:0/1549596088 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7eb40b30a0 0x7f7eb40cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:48.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.423+0000 7f7eba000640 1 -- 172.21.15.39:0/1549596088 >> 172.21.15.39:0/1549596088 conn(0x7f7eb40b2460 msgr2=0x7f7eb406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:48.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.423+0000 7f7eba000640 1 -- 172.21.15.39:0/1549596088 shutdown_connections 2024-09-21T07:54:48.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.423+0000 7f7eba000640 1 -- 172.21.15.39:0/1549596088 wait complete. 2024-09-21T07:54:48.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.425+0000 7f7eba000640 1 Processor -- start 2024-09-21T07:54:48.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.425+0000 7f7eba000640 1 -- start start 2024-09-21T07:54:48.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.427+0000 7f7eba000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7eb40b30a0 0x7f7eb4074580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:48.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.427+0000 7f7eba000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7eb40ccc60 0x7f7eb4074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:48.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.427+0000 7f7eba000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7eb415d830 0x7f7eb40931f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:48.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.427+0000 7f7eba000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb40af200 con 0x7f7eb415d830 2024-09-21T07:54:48.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.427+0000 7f7eba000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb40af080 con 0x7f7eb40ccc60 2024-09-21T07:54:48.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.427+0000 7f7eba000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f7eb40af380 con 0x7f7eb40b30a0 2024-09-21T07:54:48.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.427+0000 7f7eb3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7eb415d830 0x7f7eb40931f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:48.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.427+0000 7f7eb3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7eb415d830 0x7f7eb40931f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47442/0 (socket says 172.21.15.39:47442) 2024-09-21T07:54:48.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.427+0000 7f7eabe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7eb40ccc60 0x7f7eb4074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:48.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.427+0000 7f7eb3e00640 1 -- 172.21.15.39:0/3317765952 learned_addr learned my addr 172.21.15.39:0/3317765952 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:48.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.427+0000 7f7eb3400640 1 --2- 172.21.15.39:0/3317765952 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7eb40b30a0 0x7f7eb4074580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:48.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.428+0000 7f7eb3e00640 1 -- 172.21.15.39:0/3317765952 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7eb40b30a0 msgr2=0x7f7eb4074580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:48.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.428+0000 7f7eb3e00640 1 --2- 172.21.15.39:0/3317765952 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7eb40b30a0 0x7f7eb4074580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:48.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.428+0000 7f7eb3e00640 1 -- 172.21.15.39:0/3317765952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7eb40ccc60 msgr2=0x7f7eb4074ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:48.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.428+0000 7f7eb3e00640 1 --2- 172.21.15.39:0/3317765952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7eb40ccc60 0x7f7eb4074ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:48.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.428+0000 7f7eb3e00640 1 -- 172.21.15.39:0/3317765952 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb40c6c10 con 0x7f7eb415d830 2024-09-21T07:54:48.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.428+0000 7f7eabe00640 1 --2- 172.21.15.39:0/3317765952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7eb40ccc60 0x7f7eb4074ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:54:48.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.428+0000 7f7eb3e00640 1 --2- 172.21.15.39:0/3317765952 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7eb415d830 0x7f7eb40931f0 secure :-1 s=READY pgs=2841 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00ea70 tx=0x7f7eac00ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:48.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.428+0000 7f7eab400640 1 -- 172.21.15.39:0/3317765952 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac00ce40 con 0x7f7eb415d830 2024-09-21T07:54:48.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.428+0000 7f7eab400640 1 -- 172.21.15.39:0/3317765952 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7eac004c10 con 0x7f7eb415d830 2024-09-21T07:54:48.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.428+0000 7f7eab400640 1 -- 172.21.15.39:0/3317765952 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac005020 con 0x7f7eb415d830 2024-09-21T07:54:48.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.428+0000 7f7eba000640 1 -- 172.21.15.39:0/3317765952 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eb4094160 con 0x7f7eb415d830 2024-09-21T07:54:48.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.428+0000 7f7eba000640 1 -- 172.21.15.39:0/3317765952 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eb40c50b0 con 0x7f7eb415d830 2024-09-21T07:54:48.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.429+0000 7f7eba000640 1 -- 172.21.15.39:0/3317765952 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eb4091720 con 0x7f7eb415d830 2024-09-21T07:54:48.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.431+0000 7f7eab400640 1 -- 172.21.15.39:0/3317765952 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7eac0051c0 con 0x7f7eb415d830 2024-09-21T07:54:48.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.432+0000 7f7eab400640 1 --2- 172.21.15.39:0/3317765952 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7e90078bc0 0x7f7e9007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:48.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.432+0000 7f7eab400640 1 -- 172.21.15.39:0/3317765952 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(265..265 src has 1..265) ==== 7044+0+0 (secure 0 0 0) 0x7f7eac09bf80 con 0x7f7eb415d830 2024-09-21T07:54:48.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.432+0000 7f7eb3400640 1 --2- 172.21.15.39:0/3317765952 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7e90078bc0 0x7f7e9007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:48.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.433+0000 7f7eb3400640 1 --2- 172.21.15.39:0/3317765952 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7e90078bc0 0x7f7e9007b0d0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4005fd0 tx=0x7f7ea400b260 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:48.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.436+0000 7f7eab400640 1 -- 172.21.15.39:0/3317765952 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7eac063790 con 0x7f7eb415d830 2024-09-21T07:54:48.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.644+0000 7f7eba000640 1 -- 172.21.15.39:0/3317765952 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7eb40af870 con 0x7f7eb415d830 2024-09-21T07:54:48.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.644+0000 7f7eab400640 1 -- 172.21.15.39:0/3317765952 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v265) ==== 80+0+0 (secure 0 0 0) 0x7f7eac0681c0 con 0x7f7eb415d830 2024-09-21T07:54:48.646 INFO:teuthology.orchestra.run.smithi039.stderr:listed 0 entries 2024-09-21T07:54:48.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.647+0000 7f7eba000640 1 -- 172.21.15.39:0/3317765952 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7e90078bc0 msgr2=0x7f7e9007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:48.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.647+0000 7f7eba000640 1 --2- 172.21.15.39:0/3317765952 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7e90078bc0 0x7f7e9007b0d0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4005fd0 tx=0x7f7ea400b260 comp rx=0 tx=0).stop 2024-09-21T07:54:48.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.647+0000 7f7eba000640 1 -- 172.21.15.39:0/3317765952 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7eb415d830 msgr2=0x7f7eb40931f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:48.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.647+0000 7f7eba000640 1 --2- 172.21.15.39:0/3317765952 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7eb415d830 0x7f7eb40931f0 secure :-1 s=READY pgs=2841 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00ea70 tx=0x7f7eac00ef30 comp rx=0 tx=0).stop 2024-09-21T07:54:48.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.647+0000 7f7eba000640 1 -- 172.21.15.39:0/3317765952 shutdown_connections 2024-09-21T07:54:48.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.647+0000 7f7eba000640 1 --2- 172.21.15.39:0/3317765952 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7e90078bc0 0x7f7e9007b0d0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:48.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.647+0000 7f7eba000640 1 --2- 172.21.15.39:0/3317765952 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7eb415d830 0x7f7eb40931f0 unknown :-1 s=CLOSED pgs=2841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:48.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.647+0000 7f7eba000640 1 --2- 172.21.15.39:0/3317765952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7eb40ccc60 0x7f7eb4074ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:48.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.647+0000 7f7eba000640 1 --2- 172.21.15.39:0/3317765952 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7eb40b30a0 0x7f7eb4074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:48.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.647+0000 7f7eba000640 1 -- 172.21.15.39:0/3317765952 >> 172.21.15.39:0/3317765952 conn(0x7f7eb40b2460 msgr2=0x7f7eb415e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:48.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.647+0000 7f7eba000640 1 -- 172.21.15.39:0/3317765952 shutdown_connections 2024-09-21T07:54:48.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:48.647+0000 7f7eba000640 1 -- 172.21.15.39:0/3317765952 wait complete. 2024-09-21T07:54:48.671 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T07:54:48.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:48.671 DEBUG:teuthology.orchestra.run.smithi039:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T07:54:48.735 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T07:54:48.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:48.735 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:54:48.771 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs authorize cephfs client.foo /dir1 rws 2024-09-21T07:54:49.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.042+0000 7f629c000640 1 -- 172.21.15.39:0/4123922310 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f629406f970 msgr2=0x7f629415d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:49.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.042+0000 7f629c000640 1 --2- 172.21.15.39:0/4123922310 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f629406f970 0x7f629415d340 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f628c005fd0 tx=0x7f628c02a8b0 comp rx=0 tx=0).stop 2024-09-21T07:54:49.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.042+0000 7f629c000640 1 -- 172.21.15.39:0/4123922310 shutdown_connections 2024-09-21T07:54:49.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.042+0000 7f629c000640 1 --2- 172.21.15.39:0/4123922310 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f629406f970 0x7f629415d340 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:49.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.042+0000 7f629c000640 1 --2- 172.21.15.39:0/4123922310 >> v2:172.21.15.148:3300/0 conn(0x7f6294070bc0 0x7f629406f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:49.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.042+0000 7f629c000640 1 -- 172.21.15.39:0/4123922310 >> 172.21.15.39:0/4123922310 conn(0x7f62940b2640 msgr2=0x7f62940d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:49.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.042+0000 7f629c000640 1 -- 172.21.15.39:0/4123922310 shutdown_connections 2024-09-21T07:54:49.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.042+0000 7f629c000640 1 -- 172.21.15.39:0/4123922310 wait complete. 2024-09-21T07:54:49.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.044+0000 7f629c000640 1 Processor -- start 2024-09-21T07:54:49.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.044+0000 7f629c000640 1 -- start start 2024-09-21T07:54:49.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.046+0000 7f629c000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f629406f560 0x7f62941f8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:49.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.046+0000 7f629c000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f629406f970 0x7f6294074b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:49.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.046+0000 7f629c000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6294070bc0 0x7f629415a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:49.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.046+0000 7f629c000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f62940cd4f0 con 0x7f6294070bc0 2024-09-21T07:54:49.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.046+0000 7f629c000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f62940b50b0 con 0x7f629406f560 2024-09-21T07:54:49.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.046+0000 7f629c000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f62940cd670 con 0x7f629406f970 2024-09-21T07:54:49.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.046+0000 7f629a000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6294070bc0 0x7f629415a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:49.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.046+0000 7f629a000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6294070bc0 0x7f629415a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47458/0 (socket says 172.21.15.39:47458) 2024-09-21T07:54:49.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.046+0000 7f629a000640 1 -- 172.21.15.39:0/3051049999 learned_addr learned my addr 172.21.15.39:0/3051049999 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:49.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.046+0000 7f6299600640 1 --2- 172.21.15.39:0/3051049999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f629406f560 0x7f62941f8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:49.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.046+0000 7f6298c00640 1 --2- 172.21.15.39:0/3051049999 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f629406f970 0x7f6294074b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:49.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.047+0000 7f629a000640 1 -- 172.21.15.39:0/3051049999 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f629406f970 msgr2=0x7f6294074b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:49.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.047+0000 7f629a000640 1 --2- 172.21.15.39:0/3051049999 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f629406f970 0x7f6294074b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:49.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.047+0000 7f629a000640 1 -- 172.21.15.39:0/3051049999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f629406f560 msgr2=0x7f62941f8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:49.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.047+0000 7f629a000640 1 --2- 172.21.15.39:0/3051049999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f629406f560 0x7f62941f8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:49.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.047+0000 7f629a000640 1 -- 172.21.15.39:0/3051049999 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62940ceda0 con 0x7f6294070bc0 2024-09-21T07:54:49.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.047+0000 7f6298c00640 1 --2- 172.21.15.39:0/3051049999 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f629406f970 0x7f6294074b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:49.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.047+0000 7f629a000640 1 --2- 172.21.15.39:0/3051049999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6294070bc0 0x7f629415a960 secure :-1 s=READY pgs=2842 cs=0 l=1 rev1=1 crypto rx=0x7f628c009cd0 tx=0x7f628c031650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:49.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.047+0000 7f6291600640 1 -- 172.21.15.39:0/3051049999 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628c003e00 con 0x7f6294070bc0 2024-09-21T07:54:49.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.047+0000 7f6291600640 1 -- 172.21.15.39:0/3051049999 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f628c031a80 con 0x7f6294070bc0 2024-09-21T07:54:49.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.047+0000 7f6291600640 1 -- 172.21.15.39:0/3051049999 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628c033070 con 0x7f6294070bc0 2024-09-21T07:54:49.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.047+0000 7f629c000640 1 -- 172.21.15.39:0/3051049999 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62940cf080 con 0x7f6294070bc0 2024-09-21T07:54:49.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.048+0000 7f629c000640 1 -- 172.21.15.39:0/3051049999 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62940cf640 con 0x7f6294070bc0 2024-09-21T07:54:49.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.048+0000 7f629c000640 1 -- 172.21.15.39:0/3051049999 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f629415db70 con 0x7f6294070bc0 2024-09-21T07:54:49.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.050+0000 7f6291600640 1 -- 172.21.15.39:0/3051049999 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f628c033210 con 0x7f6294070bc0 2024-09-21T07:54:49.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.051+0000 7f6291600640 1 --2- 172.21.15.39:0/3051049999 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6274078b90 0x7f627407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:49.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.051+0000 7f6291600640 1 -- 172.21.15.39:0/3051049999 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(265..265 src has 1..265) ==== 7044+0+0 (secure 0 0 0) 0x7f628c0bc600 con 0x7f6294070bc0 2024-09-21T07:54:49.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.052+0000 7f6299600640 1 --2- 172.21.15.39:0/3051049999 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6274078b90 0x7f627407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:49.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.052+0000 7f6299600640 1 --2- 172.21.15.39:0/3051049999 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6274078b90 0x7f627407b0a0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f6288009b30 tx=0x7f6288005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:49.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.056+0000 7f6291600640 1 -- 172.21.15.39:0/3051049999 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f628c03f030 con 0x7f6294070bc0 2024-09-21T07:54:49.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.292+0000 7f629c000640 1 -- 172.21.15.39:0/3051049999 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs authorize", "filesystem": "cephfs", "entity": "client.foo", "caps": ["/dir1", "rws"]} v 0) -- 0x7f62940b1960 con 0x7f6294070bc0 2024-09-21T07:54:49.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.295+0000 7f6291600640 1 -- 172.21.15.39:0/3051049999 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs authorize", "filesystem": "cephfs", "entity": "client.foo", "caps": ["/dir1", "rws"]}]=0 v28) ==== 134+0+192 (secure 0 0 0) 0x7f628c084580 con 0x7f6294070bc0 2024-09-21T07:54:49.297 INFO:teuthology.orchestra.run.smithi039.stdout:[client.foo] 2024-09-21T07:54:49.297 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQDJe+5mkXqMERAAf9/MNKnsDriSJPozFr9VXg== 2024-09-21T07:54:49.297 INFO:teuthology.orchestra.run.smithi039.stdout: caps mds = "allow rws fsname=cephfs path=/dir1" 2024-09-21T07:54:49.297 INFO:teuthology.orchestra.run.smithi039.stdout: caps mon = "allow r fsname=cephfs" 2024-09-21T07:54:49.298 INFO:teuthology.orchestra.run.smithi039.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T07:54:49.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.298+0000 7f629c000640 1 -- 172.21.15.39:0/3051049999 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6274078b90 msgr2=0x7f627407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:49.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.298+0000 7f629c000640 1 --2- 172.21.15.39:0/3051049999 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6274078b90 0x7f627407b0a0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f6288009b30 tx=0x7f6288005f90 comp rx=0 tx=0).stop 2024-09-21T07:54:49.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.298+0000 7f629c000640 1 -- 172.21.15.39:0/3051049999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6294070bc0 msgr2=0x7f629415a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:49.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.298+0000 7f629c000640 1 --2- 172.21.15.39:0/3051049999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6294070bc0 0x7f629415a960 secure :-1 s=READY pgs=2842 cs=0 l=1 rev1=1 crypto rx=0x7f628c009cd0 tx=0x7f628c031650 comp rx=0 tx=0).stop 2024-09-21T07:54:49.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.298+0000 7f629c000640 1 -- 172.21.15.39:0/3051049999 shutdown_connections 2024-09-21T07:54:49.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.298+0000 7f629c000640 1 --2- 172.21.15.39:0/3051049999 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6274078b90 0x7f627407b0a0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:49.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.298+0000 7f629c000640 1 --2- 172.21.15.39:0/3051049999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6294070bc0 0x7f629415a960 unknown :-1 s=CLOSED pgs=2842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:49.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.298+0000 7f629c000640 1 --2- 172.21.15.39:0/3051049999 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f629406f970 0x7f6294074b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:49.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.298+0000 7f629c000640 1 --2- 172.21.15.39:0/3051049999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f629406f560 0x7f62941f8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:49.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.298+0000 7f629c000640 1 -- 172.21.15.39:0/3051049999 >> 172.21.15.39:0/3051049999 conn(0x7f62940b2640 msgr2=0x7f629415c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:49.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.298+0000 7f629c000640 1 -- 172.21.15.39:0/3051049999 shutdown_connections 2024-09-21T07:54:49.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.298+0000 7f629c000640 1 -- 172.21.15.39:0/3051049999 wait complete. 2024-09-21T07:54:49.321 DEBUG:teuthology.orchestra.run.smithi039:> mktemp 2024-09-21T07:54:49.356 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/tmp.RchFpsx2Yp 2024-09-21T07:54:49.357 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:54:49.357 DEBUG:teuthology.orchestra.run.smithi039:> dd of=/tmp/tmp.RchFpsx2Yp 2024-09-21T07:54:49.431 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:54:49.432 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:49.507 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:54:49.507 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T07:54:49.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:49.507 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:54:49.581 INFO:teuthology.orchestra.run.smithi039.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-21T07:54:49.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:54:49.582 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:54:49.582 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:49.658 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:54:49.658 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:54:49.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.922+0000 7f6e2f200640 1 -- 172.21.15.39:0/2749828058 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e2815d760 msgr2=0x7f6e2806cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:49.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.922+0000 7f6e2f200640 1 --2- 172.21.15.39:0/2749828058 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e2815d760 0x7f6e2806cde0 secure :-1 s=READY pgs=2844 cs=0 l=1 rev1=1 crypto rx=0x7f6e18009830 tx=0x7f6e1802a480 comp rx=0 tx=0).stop 2024-09-21T07:54:49.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.922+0000 7f6e2f200640 1 -- 172.21.15.39:0/2749828058 shutdown_connections 2024-09-21T07:54:49.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.922+0000 7f6e2f200640 1 --2- 172.21.15.39:0/2749828058 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e2815d760 0x7f6e2806cde0 unknown :-1 s=CLOSED pgs=2844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:49.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.922+0000 7f6e2f200640 1 --2- 172.21.15.39:0/2749828058 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e280cdb20 0x7f6e280d1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:49.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.922+0000 7f6e2f200640 1 -- 172.21.15.39:0/2749828058 >> 172.21.15.39:0/2749828058 conn(0x7f6e280b6f80 msgr2=0x7f6e2806def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:49.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.922+0000 7f6e2f200640 1 -- 172.21.15.39:0/2749828058 shutdown_connections 2024-09-21T07:54:49.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.922+0000 7f6e2f200640 1 -- 172.21.15.39:0/2749828058 wait complete. 2024-09-21T07:54:49.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.925+0000 7f6e2f200640 1 Processor -- start 2024-09-21T07:54:49.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.925+0000 7f6e2f200640 1 -- start start 2024-09-21T07:54:49.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.926+0000 7f6e2f200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e2806d320 0x7f6e280ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:49.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.926+0000 7f6e2f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e280cdb20 0x7f6e280aed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:49.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.926+0000 7f6e2f200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e2815d760 0x7f6e281f50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:49.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.926+0000 7f6e2f200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6e281609e0 con 0x7f6e280cdb20 2024-09-21T07:54:49.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.926+0000 7f6e2f200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6e280d04b0 con 0x7f6e2815d760 2024-09-21T07:54:49.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.926+0000 7f6e2f200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f6e28160b60 con 0x7f6e2806d320 2024-09-21T07:54:49.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.926+0000 7f6e2d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e280cdb20 0x7f6e280aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:49.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.926+0000 7f6e2d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e280cdb20 0x7f6e280aed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47468/0 (socket says 172.21.15.39:47468) 2024-09-21T07:54:49.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.926+0000 7f6e2d400640 1 -- 172.21.15.39:0/2621709287 learned_addr learned my addr 172.21.15.39:0/2621709287 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:49.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.927+0000 7f6e2de00640 1 --2- 172.21.15.39:0/2621709287 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e2806d320 0x7f6e280ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:49.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.927+0000 7f6e2d400640 1 -- 172.21.15.39:0/2621709287 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e2806d320 msgr2=0x7f6e280ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:49.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.927+0000 7f6e2d400640 1 --2- 172.21.15.39:0/2621709287 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e2806d320 0x7f6e280ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:49.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.927+0000 7f6e2d400640 1 -- 172.21.15.39:0/2621709287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e2815d760 msgr2=0x7f6e281f50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:49.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.927+0000 7f6e2e800640 1 --2- 172.21.15.39:0/2621709287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e2815d760 0x7f6e281f50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:49.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.927+0000 7f6e2d400640 1 --2- 172.21.15.39:0/2621709287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e2815d760 0x7f6e281f50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:49.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.927+0000 7f6e2d400640 1 -- 172.21.15.39:0/2621709287 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e280af260 con 0x7f6e280cdb20 2024-09-21T07:54:49.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.927+0000 7f6e2de00640 1 --2- 172.21.15.39:0/2621709287 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e2806d320 0x7f6e280ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:54:49.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.927+0000 7f6e2e800640 1 --2- 172.21.15.39:0/2621709287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e2815d760 0x7f6e281f50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:54:49.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.927+0000 7f6e2d400640 1 --2- 172.21.15.39:0/2621709287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e280cdb20 0x7f6e280aed20 secure :-1 s=READY pgs=2845 cs=0 l=1 rev1=1 crypto rx=0x7f6e18009950 tx=0x7f6e18033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:49.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.928+0000 7f6e22000640 1 -- 172.21.15.39:0/2621709287 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e18005c90 con 0x7f6e280cdb20 2024-09-21T07:54:49.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.928+0000 7f6e22000640 1 -- 172.21.15.39:0/2621709287 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e18031e50 con 0x7f6e280cdb20 2024-09-21T07:54:49.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.928+0000 7f6e22000640 1 -- 172.21.15.39:0/2621709287 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e18005260 con 0x7f6e280cdb20 2024-09-21T07:54:49.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.928+0000 7f6e2f200640 1 -- 172.21.15.39:0/2621709287 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e280b5bc0 con 0x7f6e280cdb20 2024-09-21T07:54:49.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.928+0000 7f6e2f200640 1 -- 172.21.15.39:0/2621709287 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e280b6180 con 0x7f6e280cdb20 2024-09-21T07:54:49.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.929+0000 7f6e2f200640 1 -- 172.21.15.39:0/2621709287 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e280d1800 con 0x7f6e280cdb20 2024-09-21T07:54:49.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.930+0000 7f6e22000640 1 -- 172.21.15.39:0/2621709287 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6e18005400 con 0x7f6e280cdb20 2024-09-21T07:54:49.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.931+0000 7f6e22000640 1 --2- 172.21.15.39:0/2621709287 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6e0c078b90 0x7f6e0c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:49.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.931+0000 7f6e22000640 1 -- 172.21.15.39:0/2621709287 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(265..265 src has 1..265) ==== 7044+0+0 (secure 0 0 0) 0x7f6e180bc3e0 con 0x7f6e280cdb20 2024-09-21T07:54:49.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.932+0000 7f6e2de00640 1 --2- 172.21.15.39:0/2621709287 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6e0c078b90 0x7f6e0c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:49.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.932+0000 7f6e2de00640 1 --2- 172.21.15.39:0/2621709287 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6e0c078b90 0x7f6e0c07b0a0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c005fd0 tx=0x7f6e1c005d20 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:49.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:49.935+0000 7f6e22000640 1 -- 172.21.15.39:0/2621709287 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6e280d1800 con 0x7f6e280cdb20 2024-09-21T07:54:50.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:50.170+0000 7f6e2f200640 1 -- 172.21.15.39:0/2621709287 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6e280b5950 con 0x7f6e280cdb20 2024-09-21T07:54:50.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:50.170+0000 7f6e22000640 1 -- 172.21.15.39:0/2621709287 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v310) ==== 53+0+73 (secure 0 0 0) 0x7f6e1803f030 con 0x7f6e280cdb20 2024-09-21T07:54:50.173 INFO:teuthology.orchestra.run.smithi039.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:54:50.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:50.173+0000 7f6e2f200640 1 -- 172.21.15.39:0/2621709287 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6e0c078b90 msgr2=0x7f6e0c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:50.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:50.173+0000 7f6e2f200640 1 --2- 172.21.15.39:0/2621709287 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6e0c078b90 0x7f6e0c07b0a0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c005fd0 tx=0x7f6e1c005d20 comp rx=0 tx=0).stop 2024-09-21T07:54:50.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:50.173+0000 7f6e2f200640 1 -- 172.21.15.39:0/2621709287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e280cdb20 msgr2=0x7f6e280aed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:50.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:50.173+0000 7f6e2f200640 1 --2- 172.21.15.39:0/2621709287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e280cdb20 0x7f6e280aed20 secure :-1 s=READY pgs=2845 cs=0 l=1 rev1=1 crypto rx=0x7f6e18009950 tx=0x7f6e18033c20 comp rx=0 tx=0).stop 2024-09-21T07:54:50.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:50.173+0000 7f6e2f200640 1 -- 172.21.15.39:0/2621709287 shutdown_connections 2024-09-21T07:54:50.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:50.173+0000 7f6e2f200640 1 --2- 172.21.15.39:0/2621709287 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6e0c078b90 0x7f6e0c07b0a0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:50.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:50.173+0000 7f6e2f200640 1 --2- 172.21.15.39:0/2621709287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e2815d760 0x7f6e281f50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:50.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:50.173+0000 7f6e2f200640 1 --2- 172.21.15.39:0/2621709287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e280cdb20 0x7f6e280aed20 unknown :-1 s=CLOSED pgs=2845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:50.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:50.173+0000 7f6e2f200640 1 --2- 172.21.15.39:0/2621709287 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6e2806d320 0x7f6e280ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:50.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:50.173+0000 7f6e2f200640 1 -- 172.21.15.39:0/2621709287 >> 172.21.15.39:0/2621709287 conn(0x7f6e280b6f80 msgr2=0x7f6e280700c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:50.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:50.173+0000 7f6e2f200640 1 -- 172.21.15.39:0/2621709287 shutdown_connections 2024-09-21T07:54:50.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:50.173+0000 7f6e2f200640 1 -- 172.21.15.39:0/2621709287 wait complete. 2024-09-21T07:54:50.196 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:54:50.196 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:54:50.197 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi039.front.sepia.ceph.com 2024-09-21T07:54:50.197 INFO:tasks.cephfs.mount:self.client.name = client.foo 2024-09-21T07:54:50.197 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:54:50.197 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:54:50.197 INFO:tasks.cephfs.mount:self.cephfs_mntpt = /dir1 2024-09-21T07:54:50.197 INFO:tasks.cephfs.mount:self.client_keyring_path = /tmp/tmp.RchFpsx2Yp 2024-09-21T07:54:50.197 DEBUG:teuthology.orchestra.run.smithi039:> mktemp 2024-09-21T07:54:50.232 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/tmp.LtCNlP1H8s 2024-09-21T07:54:50.232 DEBUG:teuthology.orchestra.run.smithi039:> sudo cp /tmp/tmp.RchFpsx2Yp /tmp/tmp.LtCNlP1H8s 2024-09-21T07:54:50.329 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0666 /tmp/tmp.LtCNlP1H8s 2024-09-21T07:54:50.423 DEBUG:teuthology.orchestra.remote:smithi039:/tmp/tmp.LtCNlP1H8s is 192B 2024-09-21T07:54:50.465 DEBUG:teuthology.orchestra.run.smithi039:> rm -fr /tmp/tmp.LtCNlP1H8s 2024-09-21T07:54:50.500 INFO:tasks.cephfs.mount:keyring content - [client.foo] key = AQDJe+5mkXqMERAAf9/MNKnsDriSJPozFr9VXg== caps mds = "allow rws fsname=cephfs path=/dir1" caps mon = "allow r fsname=cephfs" caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T07:54:50.501 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:54:50.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:50.501 DEBUG:teuthology.orchestra.run.smithi039:> ip addr 2024-09-21T07:54:50.576 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:54:50.576 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:54:50.576 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:54:50.576 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:50.576 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2024-09-21T07:54:50.577 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:50.577 INFO:teuthology.orchestra.run.smithi039.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:54:50.577 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c6 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:54:50.577 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f0 2024-09-21T07:54:50.577 INFO:teuthology.orchestra.run.smithi039.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:54:50.577 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c7 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:54:50.577 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f1 2024-09-21T07:54:50.577 INFO:teuthology.orchestra.run.smithi039.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:54:50.577 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:70 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:54:50.577 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f0 2024-09-21T07:54:50.577 INFO:teuthology.orchestra.run.smithi039.stdout: inet 172.21.15.39/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T07:54:50.577 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft 41428sec preferred_lft 41428sec 2024-09-21T07:54:50.577 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec4:7aff:febd:1370/64 scope link 2024-09-21T07:54:50.577 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:50.577 INFO:teuthology.orchestra.run.smithi039.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:54:50.578 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:71 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:54:50.578 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f1 2024-09-21T07:54:50.578 INFO:teuthology.orchestra.run.smithi039.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:54:50.578 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff 2024-09-21T07:54:50.578 INFO:teuthology.orchestra.run.smithi039.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:54:50.578 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:50.578 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::b079:ddff:fe0e:6333/64 scope link 2024-09-21T07:54:50.578 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:50.578 INFO:teuthology.orchestra.run.smithi039.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:54:50.578 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 2e:a7:98:8a:d0:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:54:50.578 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::2ca7:98ff:fe8a:d0ed/64 scope link 2024-09-21T07:54:50.578 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:50.578 INFO:teuthology.orchestra.run.smithi039.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:54:50.579 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether ee:10:53:01:0c:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:54:50.579 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec10:53ff:fe01:c44/64 scope link 2024-09-21T07:54:50.579 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:50.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:50.579 DEBUG:teuthology.orchestra.run.smithi039:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:54:50.670 INFO:teuthology.orchestra.run.smithi039.stdout:1 2024-09-21T07:54:50.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:50.673 DEBUG:teuthology.orchestra.run.smithi039:> route 2024-09-21T07:54:50.709 INFO:teuthology.orchestra.run.smithi039.stdout:Kernel IP routing table 2024-09-21T07:54:50.710 INFO:teuthology.orchestra.run.smithi039.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:54:50.710 INFO:teuthology.orchestra.run.smithi039.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T07:54:50.710 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T07:54:50.710 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:54:50.710 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:54:50.710 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-09-21T07:54:50.710 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T07:54:50.710 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T07:54:50.710 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T07:54:50.710 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-09-21T07:54:50.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:54:50.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:54:50.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:50.907 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-09-21T07:54:50.942 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:54:50.942 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T07:54:50.943 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:54:50.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:50.943 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-09-21T07:54:51.039 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:54:51.040 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T07:54:51.075 INFO:teuthology.orchestra.run.smithi039.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:54:51.076 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:54:51.076 DEBUG:teuthology.orchestra.run.smithi039:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:54:51.151 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:54:51.151 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/tmp/tmp.RchFpsx2Yp of=/dev/stdout 2024-09-21T07:54:51.226 INFO:tasks.cephfs.kernel_mount:mounting using device: foo@.cephfs=/dir1 2024-09-21T07:54:51.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:51.227 DEBUG:teuthology.orchestra.run.smithi039:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph foo@.cephfs=/dir1 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,secret=AQDJe+5mkXqMERAAf9/MNKnsDriSJPozFr9VXg==,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:54:51.426 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,secret=AQDJe+5mkXqMERAAf9/MNKnsDriSJPozFr9VXg==,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:54:51.426 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:54:51.426 INFO:teuthology.orchestra.run.smithi039.stdout:keyring.get_secret failed 2024-09-21T07:54:51.459 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,secret=AQDJe+5mkXqMERAAf9/MNKnsDriSJPozFr9VXg==,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-21T07:54:51.460 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-21T07:54:51.460 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: resolved to: "172.21.15.39:3300,172.21.15.148:3300,172.21.15.148:3301" 2024-09-21T07:54:51.460 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: trying mount with new device syntax: foo@7ab9b04a-f88d-49aa-a493-e5c076637ffb.cephfs=/dir1 2024-09-21T07:54:51.460 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=foo,ms_mode=prefer-crc,key=foo,mon_addr=172.21.15.39:3300/172.21.15.148:3300/172.21.15.148:3301" will pass to kernel 2024-09-21T07:54:51.461 INFO:teuthology.orchestra.run.smithi039.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T07:54:51.461 INFO:teuthology.orchestra.run.smithi039.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:54:51.461 INFO:teuthology.orchestra.run.smithi039.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:54:51.461 INFO:teuthology.orchestra.run.smithi039.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:54:51.462 INFO:teuthology.orchestra.run.smithi039.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:54:51.464 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:54:51.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:51.464 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:54:51.521 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:54:51.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:54:51.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:54:51.611 INFO:teuthology.orchestra.run.smithi039.stdout:client9821 2024-09-21T07:54:51.611 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:54:51.611 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:51.647 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'pushd dir2; dd if=/dev/urandom of=file bs=4k count=1;') 2024-09-21T07:54:51.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:54:51.767 INFO:teuthology.orchestra.run.smithi039.stdout:~/cephtest/mnt.0/dir2 ~/cephtest/mnt.0 2024-09-21T07:54:51.776 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-09-21T07:54:51.776 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-09-21T07:54:51.776 INFO:teuthology.orchestra.run.smithi039.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00308973 s, 1.3 MB/s 2024-09-21T07:54:51.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:54:51.785 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'mkdir .snap/one') 2024-09-21T07:54:51.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:54:51.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:54:51.900 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'rm -rf dir2') 2024-09-21T07:54:51.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:54:51.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:54:51.996 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat .snap/one/dir2/file') 2024-09-21T07:54:52.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:54:52.087 INFO:teuthology.orchestra.run.smithi039.stdout: File: .snap/one/dir2/file 2024-09-21T07:54:52.087 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 4096 Blocks: 8 IO Block: 4194304 regular file 2024-09-21T07:54:52.087 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 2ch/44d Inode: 1099511627778 Links: 1 2024-09-21T07:54:52.087 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1333/ ubuntu) 2024-09-21T07:54:52.087 INFO:teuthology.orchestra.run.smithi039.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-09-21T07:54:52.087 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-09-21 07:54:51.768914573 +0000 2024-09-21T07:54:52.087 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-09-21 07:54:51.774914457 +0000 2024-09-21T07:54:52.087 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-09-21 07:54:51.774914457 +0000 2024-09-21T07:54:52.087 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: 2024-09-21 07:54:51.768914573 +0000 2024-09-21T07:54:52.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:54:52.095 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:54:52.096 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:54:52.191 DEBUG:teuthology.orchestra.run.smithi148:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:54:52.301 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:54:52.301 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:54:52.301 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:54:52.301 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:52.338 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:54:52.338 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:52.413 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:54:52.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.678+0000 7f10eb200640 1 -- 172.21.15.39:0/631119103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10ec0d04b0 msgr2=0x7f10ec0d08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:52.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.678+0000 7f10eb200640 1 --2- 172.21.15.39:0/631119103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10ec0d04b0 0x7f10ec0d08c0 secure :-1 s=READY pgs=2846 cs=0 l=1 rev1=1 crypto rx=0x7f10ec06dcc0 tx=0x7f10e402aac0 comp rx=0 tx=0).stop 2024-09-21T07:54:52.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.678+0000 7f10eb200640 1 -- 172.21.15.39:0/631119103 shutdown_connections 2024-09-21T07:54:52.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.678+0000 7f10eb200640 1 --2- 172.21.15.39:0/631119103 >> v2:172.21.15.148:3300/0 conn(0x7f10ec15cc00 0x7f10ec15d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:52.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.678+0000 7f10eb200640 1 --2- 172.21.15.39:0/631119103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10ec0d04b0 0x7f10ec0d08c0 unknown :-1 s=CLOSED pgs=2846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:52.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.678+0000 7f10eb200640 1 -- 172.21.15.39:0/631119103 >> 172.21.15.39:0/631119103 conn(0x7f10ec06cfc0 msgr2=0x7f10ec06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:52.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.678+0000 7f10eb200640 1 -- 172.21.15.39:0/631119103 shutdown_connections 2024-09-21T07:54:52.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.678+0000 7f10eb200640 1 -- 172.21.15.39:0/631119103 wait complete. 2024-09-21T07:54:52.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.680+0000 7f10eb200640 1 Processor -- start 2024-09-21T07:54:52.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.680+0000 7f10eb200640 1 -- start start 2024-09-21T07:54:52.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.682+0000 7f10eb200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10ec0cfa20 0x7f10ec141050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:52.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.682+0000 7f10eb200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f10ec0d04b0 0x7f10ec15b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:52.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.682+0000 7f10eb200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10ec15cc00 0x7f10ec0b4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:52.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.682+0000 7f10eb200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f10ec06ee70 con 0x7f10ec15cc00 2024-09-21T07:54:52.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.682+0000 7f10eb200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f10ec06ecf0 con 0x7f10ec0cfa20 2024-09-21T07:54:52.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.682+0000 7f10eb200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f10ec06eff0 con 0x7f10ec0d04b0 2024-09-21T07:54:52.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.682+0000 7f10ea800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10ec15cc00 0x7f10ec0b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:52.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.682+0000 7f10ea800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10ec15cc00 0x7f10ec0b4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51610/0 (socket says 172.21.15.39:51610) 2024-09-21T07:54:52.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.682+0000 7f10ea800640 1 -- 172.21.15.39:0/3504958382 learned_addr learned my addr 172.21.15.39:0/3504958382 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:52.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.682+0000 7f10e9e00640 1 --2- 172.21.15.39:0/3504958382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10ec0cfa20 0x7f10ec141050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:52.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.682+0000 7f10e3e00640 1 --2- 172.21.15.39:0/3504958382 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f10ec0d04b0 0x7f10ec15b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:52.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.683+0000 7f10ea800640 1 -- 172.21.15.39:0/3504958382 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f10ec0d04b0 msgr2=0x7f10ec15b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:52.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.683+0000 7f10ea800640 1 --2- 172.21.15.39:0/3504958382 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f10ec0d04b0 0x7f10ec15b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:52.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.683+0000 7f10ea800640 1 -- 172.21.15.39:0/3504958382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10ec0cfa20 msgr2=0x7f10ec141050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:52.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.683+0000 7f10ea800640 1 --2- 172.21.15.39:0/3504958382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10ec0cfa20 0x7f10ec141050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:52.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.683+0000 7f10ea800640 1 -- 172.21.15.39:0/3504958382 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10ec0b3920 con 0x7f10ec15cc00 2024-09-21T07:54:52.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.683+0000 7f10ea800640 1 --2- 172.21.15.39:0/3504958382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10ec15cc00 0x7f10ec0b4010 secure :-1 s=READY pgs=2847 cs=0 l=1 rev1=1 crypto rx=0x7f10e4009d60 tx=0x7f10e4003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:52.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.683+0000 7f10e2000640 1 -- 172.21.15.39:0/3504958382 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e4044070 con 0x7f10ec15cc00 2024-09-21T07:54:52.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.683+0000 7f10e2000640 1 -- 172.21.15.39:0/3504958382 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10e4004250 con 0x7f10ec15cc00 2024-09-21T07:54:52.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.683+0000 7f10eb200640 1 -- 172.21.15.39:0/3504958382 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10ec15b910 con 0x7f10ec15cc00 2024-09-21T07:54:52.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.683+0000 7f10e2000640 1 -- 172.21.15.39:0/3504958382 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e4004680 con 0x7f10ec15cc00 2024-09-21T07:54:52.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.683+0000 7f10eb200640 1 -- 172.21.15.39:0/3504958382 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10ec15bed0 con 0x7f10ec15cc00 2024-09-21T07:54:52.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.684+0000 7f10eb200640 1 -- 172.21.15.39:0/3504958382 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10ec06f4a0 con 0x7f10ec15cc00 2024-09-21T07:54:52.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.686+0000 7f10e2000640 1 -- 172.21.15.39:0/3504958382 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f10e4036400 con 0x7f10ec15cc00 2024-09-21T07:54:52.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.687+0000 7f10e2000640 1 --2- 172.21.15.39:0/3504958382 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f10c8078bc0 0x7f10c807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:52.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.687+0000 7f10e2000640 1 -- 172.21.15.39:0/3504958382 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(265..265 src has 1..265) ==== 7044+0+0 (secure 0 0 0) 0x7f10e40bb960 con 0x7f10ec15cc00 2024-09-21T07:54:52.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.687+0000 7f10e9e00640 1 --2- 172.21.15.39:0/3504958382 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f10c8078bc0 0x7f10c807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:52.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.688+0000 7f10e9e00640 1 --2- 172.21.15.39:0/3504958382 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f10c8078bc0 0x7f10c807b0d0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f10dc001720 tx=0x7f10dc009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:52.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.692+0000 7f10e2000640 1 -- 172.21.15.39:0/3504958382 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f10e4032030 con 0x7f10ec15cc00 2024-09-21T07:54:52.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.904+0000 7f10eb200640 1 -- 172.21.15.39:0/3504958382 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f10ec06fff0 con 0x7f10ec15cc00 2024-09-21T07:54:52.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.904+0000 7f10e2000640 1 -- 172.21.15.39:0/3504958382 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v265) ==== 80+0+0 (secure 0 0 0) 0x7f10e4004820 con 0x7f10ec15cc00 2024-09-21T07:54:52.906 INFO:teuthology.orchestra.run.smithi039.stderr:listed 0 entries 2024-09-21T07:54:52.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.907+0000 7f10eb200640 1 -- 172.21.15.39:0/3504958382 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f10c8078bc0 msgr2=0x7f10c807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:52.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.907+0000 7f10eb200640 1 --2- 172.21.15.39:0/3504958382 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f10c8078bc0 0x7f10c807b0d0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f10dc001720 tx=0x7f10dc009040 comp rx=0 tx=0).stop 2024-09-21T07:54:52.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.907+0000 7f10eb200640 1 -- 172.21.15.39:0/3504958382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10ec15cc00 msgr2=0x7f10ec0b4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:52.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.907+0000 7f10eb200640 1 --2- 172.21.15.39:0/3504958382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10ec15cc00 0x7f10ec0b4010 secure :-1 s=READY pgs=2847 cs=0 l=1 rev1=1 crypto rx=0x7f10e4009d60 tx=0x7f10e4003f10 comp rx=0 tx=0).stop 2024-09-21T07:54:52.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.907+0000 7f10eb200640 1 -- 172.21.15.39:0/3504958382 shutdown_connections 2024-09-21T07:54:52.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.907+0000 7f10eb200640 1 --2- 172.21.15.39:0/3504958382 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f10c8078bc0 0x7f10c807b0d0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:52.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.907+0000 7f10eb200640 1 --2- 172.21.15.39:0/3504958382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10ec15cc00 0x7f10ec0b4010 unknown :-1 s=CLOSED pgs=2847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:52.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.907+0000 7f10eb200640 1 --2- 172.21.15.39:0/3504958382 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f10ec0d04b0 0x7f10ec15b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:52.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.907+0000 7f10eb200640 1 --2- 172.21.15.39:0/3504958382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10ec0cfa20 0x7f10ec141050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:52.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.907+0000 7f10eb200640 1 -- 172.21.15.39:0/3504958382 >> 172.21.15.39:0/3504958382 conn(0x7f10ec06cfc0 msgr2=0x7f10ec0afdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:52.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.907+0000 7f10eb200640 1 -- 172.21.15.39:0/3504958382 shutdown_connections 2024-09-21T07:54:52.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:52.907+0000 7f10eb200640 1 -- 172.21.15.39:0/3504958382 wait complete. 2024-09-21T07:54:52.930 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.foo... 2024-09-21T07:54:52.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:52.931 DEBUG:teuthology.orchestra.run.smithi039:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T07:54:52.992 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T07:54:52.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:52.993 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:54:53.028 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:54:53.028 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:54:53.029 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:53.068 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:54:53.068 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:54:53.068 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:54:53.068 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:53.106 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:54:53.106 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:54:53.106 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:54:53.106 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:53.145 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:54:53.146 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:54:53.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.372+0000 7f8099200640 1 -- 172.21.15.39:0/3691836395 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80940b6c90 msgr2=0x7f80940cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:53.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.372+0000 7f8099200640 1 --2- 172.21.15.39:0/3691836395 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80940b6c90 0x7f80940cf7e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8084005fd0 tx=0x7f808402a4b0 comp rx=0 tx=0).stop 2024-09-21T07:54:53.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.372+0000 7f8099200640 1 -- 172.21.15.39:0/3691836395 shutdown_connections 2024-09-21T07:54:53.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.372+0000 7f8099200640 1 --2- 172.21.15.39:0/3691836395 >> v2:172.21.15.148:3300/0 conn(0x7f80940b5f60 0x7f80940b6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:53.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.372+0000 7f8099200640 1 --2- 172.21.15.39:0/3691836395 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80940b6c90 0x7f80940cf7e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:53.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.372+0000 7f8099200640 1 -- 172.21.15.39:0/3691836395 >> 172.21.15.39:0/3691836395 conn(0x7f809409d7b0 msgr2=0x7f80940d44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:53.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.372+0000 7f8099200640 1 -- 172.21.15.39:0/3691836395 shutdown_connections 2024-09-21T07:54:53.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.372+0000 7f8099200640 1 -- 172.21.15.39:0/3691836395 wait complete. 2024-09-21T07:54:53.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.374+0000 7f8099200640 1 Processor -- start 2024-09-21T07:54:53.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.374+0000 7f8099200640 1 -- start start 2024-09-21T07:54:53.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.376+0000 7f8099200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80940b5f60 0x7f809414ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:53.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.376+0000 7f8099200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80940b6c90 0x7f80941641e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:53.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.376+0000 7f8099200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80940d25a0 0x7f809415f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:53.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.376+0000 7f8099200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f80940cff30 con 0x7f80940b5f60 2024-09-21T07:54:53.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.376+0000 7f8099200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f80940b9a40 con 0x7f80940b6c90 2024-09-21T07:54:53.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.376+0000 7f8099200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f80940b9e60 con 0x7f80940d25a0 2024-09-21T07:54:53.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.376+0000 7f8093400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80940b5f60 0x7f809414ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:53.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.376+0000 7f8093400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80940b5f60 0x7f809414ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51618/0 (socket says 172.21.15.39:51618) 2024-09-21T07:54:53.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.376+0000 7f8093400640 1 -- 172.21.15.39:0/3105449704 learned_addr learned my addr 172.21.15.39:0/3105449704 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:53.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.376+0000 7f8092a00640 1 --2- 172.21.15.39:0/3105449704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80940b6c90 0x7f80941641e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:53.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.376+0000 7f8093e00640 1 --2- 172.21.15.39:0/3105449704 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80940d25a0 0x7f809415f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:53.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.377+0000 7f8093400640 1 -- 172.21.15.39:0/3105449704 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80940d25a0 msgr2=0x7f809415f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:53.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.377+0000 7f8093400640 1 --2- 172.21.15.39:0/3105449704 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80940d25a0 0x7f809415f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:53.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.377+0000 7f8093400640 1 -- 172.21.15.39:0/3105449704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80940b6c90 msgr2=0x7f80941641e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:53.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.377+0000 7f8093400640 1 --2- 172.21.15.39:0/3105449704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80940b6c90 0x7f80941641e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:53.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.377+0000 7f8093400640 1 -- 172.21.15.39:0/3105449704 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80941f8c30 con 0x7f80940b5f60 2024-09-21T07:54:53.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.377+0000 7f8093400640 1 --2- 172.21.15.39:0/3105449704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80940b5f60 0x7f809414ea20 secure :-1 s=READY pgs=2848 cs=0 l=1 rev1=1 crypto rx=0x7f8084034040 tx=0x7f80840038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:53.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.377+0000 7f808b400640 1 -- 172.21.15.39:0/3105449704 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808403e030 con 0x7f80940b5f60 2024-09-21T07:54:53.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.377+0000 7f808b400640 1 -- 172.21.15.39:0/3105449704 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8084003d90 con 0x7f80940b5f60 2024-09-21T07:54:53.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.377+0000 7f8099200640 1 -- 172.21.15.39:0/3105449704 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80941f8f10 con 0x7f80940b5f60 2024-09-21T07:54:53.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.377+0000 7f808b400640 1 -- 172.21.15.39:0/3105449704 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8084035040 con 0x7f80940b5f60 2024-09-21T07:54:53.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.378+0000 7f8099200640 1 -- 172.21.15.39:0/3105449704 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80941f94d0 con 0x7f80940b5f60 2024-09-21T07:54:53.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.379+0000 7f8099200640 1 -- 172.21.15.39:0/3105449704 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80940b6370 con 0x7f80940b5f60 2024-09-21T07:54:53.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.380+0000 7f808b400640 1 -- 172.21.15.39:0/3105449704 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8084041030 con 0x7f80940b5f60 2024-09-21T07:54:53.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.381+0000 7f808b400640 1 --2- 172.21.15.39:0/3105449704 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8074078b90 0x7f807407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:53.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.381+0000 7f808b400640 1 -- 172.21.15.39:0/3105449704 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(265..265 src has 1..265) ==== 7044+0+0 (secure 0 0 0) 0x7f80840bbd30 con 0x7f80940b5f60 2024-09-21T07:54:53.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.381+0000 7f8092a00640 1 --2- 172.21.15.39:0/3105449704 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8074078b90 0x7f807407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:53.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.382+0000 7f8092a00640 1 --2- 172.21.15.39:0/3105449704 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8074078b90 0x7f807407b0a0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f80940d4550 tx=0x7f8080005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:53.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.385+0000 7f808b400640 1 -- 172.21.15.39:0/3105449704 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f80840835d0 con 0x7f80940b5f60 2024-09-21T07:54:53.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.625+0000 7f8099200640 1 -- 172.21.15.39:0/3105449704 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80940d4860 con 0x7f80940b5f60 2024-09-21T07:54:53.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.626+0000 7f808b400640 1 -- 172.21.15.39:0/3105449704 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 310 v310) ==== 95+0+5243 (secure 0 0 0) 0x7f8084088000 con 0x7f80940b5f60 2024-09-21T07:54:53.628 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:53.628 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":310,"btime":"2024-09-21T07:54:42:307372+0000","default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9725,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1018044536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1018044536},{"type":"v1","addr":"172.21.15.148:6837","nonce":1018044536}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":305},{"gid":11125,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1771887993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1771887993},{"type":"v1","addr":"172.21.15.148:6836","nonce":1771887993}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":304},{"gid":11131,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/649113662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":649113662},{"type":"v1","addr":"172.21.15.39:6835","nonce":649113662}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":305}],"filesystems":[{"mdsmap":{"epoch":310,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:54:38.804512+0000","modified":"2024-09-21T07:54:41.814517+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":12315},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12315":{"gid":12315,"name":"c","rank":0,"incarnation":306,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/3094159467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3094159467},{"type":"v1","addr":"172.21.15.39:6834","nonce":3094159467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":12315,"qdb_cluster":[12315]},"id":14}]} 2024-09-21T07:54:53.628 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 310 2024-09-21T07:54:53.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.628+0000 7f8099200640 1 -- 172.21.15.39:0/3105449704 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8074078b90 msgr2=0x7f807407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:53.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.628+0000 7f8099200640 1 --2- 172.21.15.39:0/3105449704 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8074078b90 0x7f807407b0a0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f80940d4550 tx=0x7f8080005f90 comp rx=0 tx=0).stop 2024-09-21T07:54:53.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.628+0000 7f8099200640 1 -- 172.21.15.39:0/3105449704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80940b5f60 msgr2=0x7f809414ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:53.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.628+0000 7f8099200640 1 --2- 172.21.15.39:0/3105449704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80940b5f60 0x7f809414ea20 secure :-1 s=READY pgs=2848 cs=0 l=1 rev1=1 crypto rx=0x7f8084034040 tx=0x7f80840038a0 comp rx=0 tx=0).stop 2024-09-21T07:54:53.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.628+0000 7f8099200640 1 -- 172.21.15.39:0/3105449704 shutdown_connections 2024-09-21T07:54:53.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.628+0000 7f8099200640 1 --2- 172.21.15.39:0/3105449704 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8074078b90 0x7f807407b0a0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:53.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.628+0000 7f8099200640 1 --2- 172.21.15.39:0/3105449704 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f80940d25a0 0x7f809415f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:53.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.628+0000 7f8099200640 1 --2- 172.21.15.39:0/3105449704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f80940b6c90 0x7f80941641e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:53.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.628+0000 7f8099200640 1 --2- 172.21.15.39:0/3105449704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f80940b5f60 0x7f809414ea20 unknown :-1 s=CLOSED pgs=2848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:53.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.628+0000 7f8099200640 1 -- 172.21.15.39:0/3105449704 >> 172.21.15.39:0/3105449704 conn(0x7f809409d7b0 msgr2=0x7f80940b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:53.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.628+0000 7f8099200640 1 -- 172.21.15.39:0/3105449704 shutdown_connections 2024-09-21T07:54:53.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.628+0000 7f8099200640 1 -- 172.21.15.39:0/3105449704 wait complete. 2024-09-21T07:54:53.653 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:54:53.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.876+0000 7f5277e00640 1 -- 172.21.15.39:0/49109825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52700ccc60 msgr2=0x7f52700cd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:53.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.876+0000 7f5277e00640 1 --2- 172.21.15.39:0/49109825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52700ccc60 0x7f52700cd070 secure :-1 s=READY pgs=2849 cs=0 l=1 rev1=1 crypto rx=0x7f5270075160 tx=0x7f526802aa30 comp rx=0 tx=0).stop 2024-09-21T07:54:53.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.876+0000 7f5277e00640 1 -- 172.21.15.39:0/49109825 shutdown_connections 2024-09-21T07:54:53.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.876+0000 7f5277e00640 1 --2- 172.21.15.39:0/49109825 >> v2:172.21.15.148:3300/0 conn(0x7f52700cb8a0 0x7f5270156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:53.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.876+0000 7f5277e00640 1 --2- 172.21.15.39:0/49109825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52700ccc60 0x7f52700cd070 unknown :-1 s=CLOSED pgs=2849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:53.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.876+0000 7f5277e00640 1 -- 172.21.15.39:0/49109825 >> 172.21.15.39:0/49109825 conn(0x7f52700b2460 msgr2=0x7f527006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:53.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.876+0000 7f5277e00640 1 -- 172.21.15.39:0/49109825 shutdown_connections 2024-09-21T07:54:53.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.876+0000 7f5277e00640 1 -- 172.21.15.39:0/49109825 wait complete. 2024-09-21T07:54:53.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.879+0000 7f5277e00640 1 Processor -- start 2024-09-21T07:54:53.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.879+0000 7f5277e00640 1 -- start start 2024-09-21T07:54:53.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.880+0000 7f5277e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52700b30a0 0x7f52700c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:53.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.880+0000 7f5277e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52700cb8a0 0x7f52701f8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:53.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.880+0000 7f5277e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52700ccc60 0x7f527015e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:53.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.880+0000 7f5277e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f52700af410 con 0x7f52700ccc60 2024-09-21T07:54:53.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.880+0000 7f5277e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f52700af290 con 0x7f52700cb8a0 2024-09-21T07:54:53.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.880+0000 7f5277e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f52700af590 con 0x7f52700b30a0 2024-09-21T07:54:53.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.880+0000 7f5275e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52700ccc60 0x7f527015e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:53.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.881+0000 7f5275e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52700ccc60 0x7f527015e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51634/0 (socket says 172.21.15.39:51634) 2024-09-21T07:54:53.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.881+0000 7f5275e00640 1 -- 172.21.15.39:0/829919494 learned_addr learned my addr 172.21.15.39:0/829919494 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:53.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.881+0000 7f5274a00640 1 --2- 172.21.15.39:0/829919494 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52700cb8a0 0x7f52701f8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:53.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.881+0000 7f5275e00640 1 -- 172.21.15.39:0/829919494 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52700b30a0 msgr2=0x7f52700c9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:53.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.881+0000 7f5275e00640 1 --2- 172.21.15.39:0/829919494 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52700b30a0 0x7f52700c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:53.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.881+0000 7f5275e00640 1 -- 172.21.15.39:0/829919494 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52700cb8a0 msgr2=0x7f52701f8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:53.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.881+0000 7f5275e00640 1 --2- 172.21.15.39:0/829919494 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52700cb8a0 0x7f52701f8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:53.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.881+0000 7f5275e00640 1 -- 172.21.15.39:0/829919494 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52700ce120 con 0x7f52700ccc60 2024-09-21T07:54:53.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.881+0000 7f5274a00640 1 --2- 172.21.15.39:0/829919494 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52700cb8a0 0x7f52701f8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:54:53.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.881+0000 7f5275e00640 1 --2- 172.21.15.39:0/829919494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52700ccc60 0x7f527015e810 secure :-1 s=READY pgs=2850 cs=0 l=1 rev1=1 crypto rx=0x7f5268005fd0 tx=0x7f5268003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:53.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.882+0000 7f526d600640 1 -- 172.21.15.39:0/829919494 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5268033050 con 0x7f52700ccc60 2024-09-21T07:54:53.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.882+0000 7f526d600640 1 -- 172.21.15.39:0/829919494 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5268004540 con 0x7f52700ccc60 2024-09-21T07:54:53.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.882+0000 7f526d600640 1 -- 172.21.15.39:0/829919494 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5268038450 con 0x7f52700ccc60 2024-09-21T07:54:53.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.882+0000 7f5277e00640 1 -- 172.21.15.39:0/829919494 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52700ce400 con 0x7f52700ccc60 2024-09-21T07:54:53.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.882+0000 7f5277e00640 1 -- 172.21.15.39:0/829919494 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52700ce9c0 con 0x7f52700ccc60 2024-09-21T07:54:53.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.883+0000 7f5277e00640 1 -- 172.21.15.39:0/829919494 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52700afab0 con 0x7f52700ccc60 2024-09-21T07:54:53.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.884+0000 7f526d600640 1 -- 172.21.15.39:0/829919494 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5268036070 con 0x7f52700ccc60 2024-09-21T07:54:53.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.885+0000 7f526d600640 1 --2- 172.21.15.39:0/829919494 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5250078bc0 0x7f525007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:53.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.885+0000 7f526d600640 1 -- 172.21.15.39:0/829919494 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(265..265 src has 1..265) ==== 7044+0+0 (secure 0 0 0) 0x7f52680bc5d0 con 0x7f52700ccc60 2024-09-21T07:54:53.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.886+0000 7f5275400640 1 --2- 172.21.15.39:0/829919494 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5250078bc0 0x7f525007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:53.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.886+0000 7f5275400640 1 --2- 172.21.15.39:0/829919494 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5250078bc0 0x7f525007b0d0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f5264001510 tx=0x7f5264007930 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:53.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:53.889+0000 7f526d600640 1 -- 172.21.15.39:0/829919494 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5268084560 con 0x7f52700ccc60 2024-09-21T07:54:54.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.126+0000 7f5277e00640 1 -- 172.21.15.39:0/829919494 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52700cd070 con 0x7f52700ccc60 2024-09-21T07:54:54.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.126+0000 7f526d600640 1 -- 172.21.15.39:0/829919494 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 310 v310) ==== 95+0+5243 (secure 0 0 0) 0x7f5268083e60 con 0x7f52700ccc60 2024-09-21T07:54:54.128 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:54.128 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":310,"btime":"2024-09-21T07:54:42:307372+0000","default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9725,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1018044536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1018044536},{"type":"v1","addr":"172.21.15.148:6837","nonce":1018044536}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":305},{"gid":11125,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1771887993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1771887993},{"type":"v1","addr":"172.21.15.148:6836","nonce":1771887993}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":304},{"gid":11131,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/649113662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":649113662},{"type":"v1","addr":"172.21.15.39:6835","nonce":649113662}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":305}],"filesystems":[{"mdsmap":{"epoch":310,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:54:38.804512+0000","modified":"2024-09-21T07:54:41.814517+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":12315},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12315":{"gid":12315,"name":"c","rank":0,"incarnation":306,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/3094159467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3094159467},{"type":"v1","addr":"172.21.15.39:6834","nonce":3094159467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":12315,"qdb_cluster":[12315]},"id":14}]} 2024-09-21T07:54:54.129 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 310 2024-09-21T07:54:54.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.128+0000 7f5277e00640 1 -- 172.21.15.39:0/829919494 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5250078bc0 msgr2=0x7f525007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:54.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.128+0000 7f5277e00640 1 --2- 172.21.15.39:0/829919494 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5250078bc0 0x7f525007b0d0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f5264001510 tx=0x7f5264007930 comp rx=0 tx=0).stop 2024-09-21T07:54:54.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.128+0000 7f5277e00640 1 -- 172.21.15.39:0/829919494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52700ccc60 msgr2=0x7f527015e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:54.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.128+0000 7f5277e00640 1 --2- 172.21.15.39:0/829919494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52700ccc60 0x7f527015e810 secure :-1 s=READY pgs=2850 cs=0 l=1 rev1=1 crypto rx=0x7f5268005fd0 tx=0x7f5268003f90 comp rx=0 tx=0).stop 2024-09-21T07:54:54.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.129+0000 7f5277e00640 1 -- 172.21.15.39:0/829919494 shutdown_connections 2024-09-21T07:54:54.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.129+0000 7f5277e00640 1 --2- 172.21.15.39:0/829919494 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5250078bc0 0x7f525007b0d0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:54.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.129+0000 7f5277e00640 1 --2- 172.21.15.39:0/829919494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52700ccc60 0x7f527015e810 unknown :-1 s=CLOSED pgs=2850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:54.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.129+0000 7f5277e00640 1 --2- 172.21.15.39:0/829919494 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f52700cb8a0 0x7f52701f8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:54.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.129+0000 7f5277e00640 1 --2- 172.21.15.39:0/829919494 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f52700b30a0 0x7f52700c9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:54.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.129+0000 7f5277e00640 1 -- 172.21.15.39:0/829919494 >> 172.21.15.39:0/829919494 conn(0x7f52700b2460 msgr2=0x7f52701553c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:54.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.129+0000 7f5277e00640 1 -- 172.21.15.39:0/829919494 shutdown_connections 2024-09-21T07:54:54.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.129+0000 7f5277e00640 1 -- 172.21.15.39:0/829919494 wait complete. 2024-09-21T07:54:54.153 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:54:54.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.379+0000 7f265a400640 1 -- 172.21.15.39:0/4120504692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26540ccc60 msgr2=0x7f26540cd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:54.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.379+0000 7f265a400640 1 --2- 172.21.15.39:0/4120504692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26540ccc60 0x7f26540cd070 secure :-1 s=READY pgs=2852 cs=0 l=1 rev1=1 crypto rx=0x7f2654075160 tx=0x7f264c02aa30 comp rx=0 tx=0).stop 2024-09-21T07:54:54.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.379+0000 7f265a400640 1 -- 172.21.15.39:0/4120504692 shutdown_connections 2024-09-21T07:54:54.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.379+0000 7f265a400640 1 --2- 172.21.15.39:0/4120504692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26540ccc60 0x7f26540cd070 unknown :-1 s=CLOSED pgs=2852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:54.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.379+0000 7f265a400640 1 --2- 172.21.15.39:0/4120504692 >> v2:172.21.15.148:3300/0 conn(0x7f26540cb8a0 0x7f2654156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:54.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.379+0000 7f265a400640 1 -- 172.21.15.39:0/4120504692 >> 172.21.15.39:0/4120504692 conn(0x7f26540b2460 msgr2=0x7f265406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:54.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.380+0000 7f265a400640 1 -- 172.21.15.39:0/4120504692 shutdown_connections 2024-09-21T07:54:54.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.380+0000 7f265a400640 1 -- 172.21.15.39:0/4120504692 wait complete. 2024-09-21T07:54:54.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.382+0000 7f265a400640 1 Processor -- start 2024-09-21T07:54:54.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.382+0000 7f265a400640 1 -- start start 2024-09-21T07:54:54.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.383+0000 7f265a400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26540b30a0 0x7f26541f5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:54.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.383+0000 7f265a400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26540cb8a0 0x7f26540b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:54.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.384+0000 7f265a400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f26540ccc60 0x7f26540b1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:54.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.384+0000 7f265a400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f26540af410 con 0x7f26540cb8a0 2024-09-21T07:54:54.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.384+0000 7f265a400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f26540af290 con 0x7f26540b30a0 2024-09-21T07:54:54.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.384+0000 7f265a400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f26540af590 con 0x7f26540ccc60 2024-09-21T07:54:54.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.384+0000 7f2652a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26540cb8a0 0x7f26540b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:54.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.384+0000 7f2652a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26540cb8a0 0x7f26540b0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51650/0 (socket says 172.21.15.39:51650) 2024-09-21T07:54:54.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.384+0000 7f2652a00640 1 -- 172.21.15.39:0/2395665969 learned_addr learned my addr 172.21.15.39:0/2395665969 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:54.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.384+0000 7f2652a00640 1 -- 172.21.15.39:0/2395665969 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f26540ccc60 msgr2=0x7f26540b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:54.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.384+0000 7f2652a00640 1 --2- 172.21.15.39:0/2395665969 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f26540ccc60 0x7f26540b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:54.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.384+0000 7f2652a00640 1 -- 172.21.15.39:0/2395665969 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26540b30a0 msgr2=0x7f26541f5190 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T07:54:54.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.384+0000 7f2652a00640 1 --2- 172.21.15.39:0/2395665969 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26540b30a0 0x7f26541f5190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:54.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.384+0000 7f2652a00640 1 -- 172.21.15.39:0/2395665969 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26541f9020 con 0x7f26540cb8a0 2024-09-21T07:54:54.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.385+0000 7f2652a00640 1 --2- 172.21.15.39:0/2395665969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26540cb8a0 0x7f26540b0ed0 secure :-1 s=READY pgs=2853 cs=0 l=1 rev1=1 crypto rx=0x7f264000d740 tx=0x7f264000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:54.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.385+0000 7f264b400640 1 -- 172.21.15.39:0/2395665969 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2640017070 con 0x7f26540cb8a0 2024-09-21T07:54:54.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.385+0000 7f264b400640 1 -- 172.21.15.39:0/2395665969 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2640009ab0 con 0x7f26540cb8a0 2024-09-21T07:54:54.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.385+0000 7f264b400640 1 -- 172.21.15.39:0/2395665969 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2640010460 con 0x7f26540cb8a0 2024-09-21T07:54:54.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.385+0000 7f265a400640 1 -- 172.21.15.39:0/2395665969 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26541f9300 con 0x7f26540cb8a0 2024-09-21T07:54:54.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.385+0000 7f265a400640 1 -- 172.21.15.39:0/2395665969 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26541f98c0 con 0x7f26540cb8a0 2024-09-21T07:54:54.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.386+0000 7f265a400640 1 -- 172.21.15.39:0/2395665969 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26540afab0 con 0x7f26540cb8a0 2024-09-21T07:54:54.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.387+0000 7f264b400640 1 -- 172.21.15.39:0/2395665969 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f264001a070 con 0x7f26540cb8a0 2024-09-21T07:54:54.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.388+0000 7f264b400640 1 --2- 172.21.15.39:0/2395665969 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2634078b90 0x7f263407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:54.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.388+0000 7f264b400640 1 -- 172.21.15.39:0/2395665969 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(265..265 src has 1..265) ==== 7044+0+0 (secure 0 0 0) 0x7f264009b430 con 0x7f26540cb8a0 2024-09-21T07:54:54.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.389+0000 7f2653400640 1 --2- 172.21.15.39:0/2395665969 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2634078b90 0x7f263407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:54.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.391+0000 7f2653400640 1 --2- 172.21.15.39:0/2395665969 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2634078b90 0x7f263407b0a0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f26440014a0 tx=0x7f2644007930 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:54.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.393+0000 7f264b400640 1 -- 172.21.15.39:0/2395665969 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f26400632d0 con 0x7f26540cb8a0 2024-09-21T07:54:54.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.592+0000 7f265a400640 1 -- 172.21.15.39:0/2395665969 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f26540b40c0 con 0x7f26540cb8a0 2024-09-21T07:54:54.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.594+0000 7f264b400640 1 -- 172.21.15.39:0/2395665969 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v265) ==== 74+0+22596 (secure 0 0 0) 0x7f2640062bd0 con 0x7f26540cb8a0 2024-09-21T07:54:54.596 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:54.596 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":265,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:54:40.270721+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:54:36.740920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"264","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2024-09-21T07:54:37.755494+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"265","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]},{"pool":20,"snaps":[{"begin":80,"length":23},{"begin":181,"length":23}]},{"pool":22,"snaps":[{"begin":100,"length":1},{"begin":199,"length":1}]},{"pool":24,"snaps":[{"begin":2,"length":2}]},{"pool":26,"snaps":[{"begin":2,"length":2}]},{"pool":28,"snaps":[{"begin":2,"length":2}]},{"pool":30,"snaps":[{"begin":8,"length":6}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:54:54.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.596+0000 7f265a400640 1 -- 172.21.15.39:0/2395665969 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2634078b90 msgr2=0x7f263407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:54.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.596+0000 7f265a400640 1 --2- 172.21.15.39:0/2395665969 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2634078b90 0x7f263407b0a0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f26440014a0 tx=0x7f2644007930 comp rx=0 tx=0).stop 2024-09-21T07:54:54.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.597+0000 7f265a400640 1 -- 172.21.15.39:0/2395665969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26540cb8a0 msgr2=0x7f26540b0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:54.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.597+0000 7f265a400640 1 --2- 172.21.15.39:0/2395665969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26540cb8a0 0x7f26540b0ed0 secure :-1 s=READY pgs=2853 cs=0 l=1 rev1=1 crypto rx=0x7f264000d740 tx=0x7f264000dc00 comp rx=0 tx=0).stop 2024-09-21T07:54:54.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.597+0000 7f265a400640 1 -- 172.21.15.39:0/2395665969 shutdown_connections 2024-09-21T07:54:54.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.597+0000 7f265a400640 1 --2- 172.21.15.39:0/2395665969 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2634078b90 0x7f263407b0a0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:54.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.597+0000 7f265a400640 1 --2- 172.21.15.39:0/2395665969 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f26540ccc60 0x7f26540b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:54.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.597+0000 7f265a400640 1 --2- 172.21.15.39:0/2395665969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26540cb8a0 0x7f26540b0ed0 unknown :-1 s=CLOSED pgs=2853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:54.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.597+0000 7f265a400640 1 --2- 172.21.15.39:0/2395665969 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f26540b30a0 0x7f26541f5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:54.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.597+0000 7f265a400640 1 -- 172.21.15.39:0/2395665969 >> 172.21.15.39:0/2395665969 conn(0x7f26540b2460 msgr2=0x7f265415f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:54.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.597+0000 7f265a400640 1 -- 172.21.15.39:0/2395665969 shutdown_connections 2024-09-21T07:54:54.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.597+0000 7f265a400640 1 -- 172.21.15.39:0/2395665969 wait complete. 2024-09-21T07:54:54.625 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T07:54:54.625 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:54:54.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.850+0000 7f9005800640 1 -- 172.21.15.39:0/613585379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90001519d0 msgr2=0x7f9000073f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:54.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.850+0000 7f9005800640 1 --2- 172.21.15.39:0/613585379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90001519d0 0x7f9000073f30 secure :-1 s=READY pgs=2854 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0005fd0 tx=0x7f8ff002a5d0 comp rx=0 tx=0).stop 2024-09-21T07:54:54.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.850+0000 7f9005800640 1 -- 172.21.15.39:0/613585379 shutdown_connections 2024-09-21T07:54:54.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.850+0000 7f9005800640 1 --2- 172.21.15.39:0/613585379 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f90000d04a0 0x7f90000d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:54.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.850+0000 7f9005800640 1 --2- 172.21.15.39:0/613585379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90001519d0 0x7f9000073f30 unknown :-1 s=CLOSED pgs=2854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:54.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.850+0000 7f9005800640 1 -- 172.21.15.39:0/613585379 >> 172.21.15.39:0/613585379 conn(0x7f900006cfc0 msgr2=0x7f900006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:54.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.850+0000 7f9005800640 1 -- 172.21.15.39:0/613585379 shutdown_connections 2024-09-21T07:54:54.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.850+0000 7f9005800640 1 -- 172.21.15.39:0/613585379 wait complete. 2024-09-21T07:54:54.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.852+0000 7f9005800640 1 Processor -- start 2024-09-21T07:54:54.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.852+0000 7f9005800640 1 -- start start 2024-09-21T07:54:54.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.854+0000 7f9005800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f900006ee90 0x7f9000159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:54.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.854+0000 7f9005800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90000d04a0 0x7f9000159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:54.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.854+0000 7f9005800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f90001519d0 0x7f90001537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:54.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.854+0000 7f9005800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f90000b3940 con 0x7f90000d04a0 2024-09-21T07:54:54.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.854+0000 7f9005800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f90000b3ac0 con 0x7f900006ee90 2024-09-21T07:54:54.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.854+0000 7f9005800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f90000b3c40 con 0x7f90001519d0 2024-09-21T07:54:54.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.854+0000 7f8ff7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90000d04a0 0x7f9000159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:54.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.854+0000 7f8ff7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90000d04a0 0x7f9000159d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51666/0 (socket says 172.21.15.39:51666) 2024-09-21T07:54:54.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.854+0000 7f8ff7e00640 1 -- 172.21.15.39:0/3183098907 learned_addr learned my addr 172.21.15.39:0/3183098907 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:54.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.854+0000 7f8fffe00640 1 --2- 172.21.15.39:0/3183098907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f900006ee90 0x7f9000159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:54.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.854+0000 7f9004e00640 1 --2- 172.21.15.39:0/3183098907 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f90001519d0 0x7f90001537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:54.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.854+0000 7f8ff7e00640 1 -- 172.21.15.39:0/3183098907 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f90001519d0 msgr2=0x7f90001537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:54.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.854+0000 7f8ff7e00640 1 --2- 172.21.15.39:0/3183098907 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f90001519d0 0x7f90001537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:54.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.854+0000 7f8ff7e00640 1 -- 172.21.15.39:0/3183098907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f900006ee90 msgr2=0x7f9000159830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:54.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.854+0000 7f8ff7e00640 1 --2- 172.21.15.39:0/3183098907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f900006ee90 0x7f9000159830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:54.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.854+0000 7f8ff7e00640 1 -- 172.21.15.39:0/3183098907 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9000151e80 con 0x7f90000d04a0 2024-09-21T07:54:54.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.854+0000 7f8fffe00640 1 --2- 172.21.15.39:0/3183098907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f900006ee90 0x7f9000159830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:54.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.854+0000 7f9004e00640 1 --2- 172.21.15.39:0/3183098907 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f90001519d0 0x7f90001537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:54.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.855+0000 7f8ff7e00640 1 --2- 172.21.15.39:0/3183098907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90000d04a0 0x7f9000159d70 secure :-1 s=READY pgs=2855 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800d6d0 tx=0x7f8fe800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:54.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.855+0000 7f8ffcc00640 1 -- 172.21.15.39:0/3183098907 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8017070 con 0x7f90000d04a0 2024-09-21T07:54:54.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.855+0000 7f8ffcc00640 1 -- 172.21.15.39:0/3183098907 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8fe8005330 con 0x7f90000d04a0 2024-09-21T07:54:54.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.855+0000 7f8ffcc00640 1 -- 172.21.15.39:0/3183098907 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe80097c0 con 0x7f90000d04a0 2024-09-21T07:54:54.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.855+0000 7f9005800640 1 -- 172.21.15.39:0/3183098907 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9000152160 con 0x7f90000d04a0 2024-09-21T07:54:54.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.855+0000 7f9005800640 1 -- 172.21.15.39:0/3183098907 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90001f9830 con 0x7f90000d04a0 2024-09-21T07:54:54.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.856+0000 7f9005800640 1 -- 172.21.15.39:0/3183098907 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f900014afe0 con 0x7f90000d04a0 2024-09-21T07:54:54.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.858+0000 7f8ffcc00640 1 -- 172.21.15.39:0/3183098907 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8fe8009b70 con 0x7f90000d04a0 2024-09-21T07:54:54.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.859+0000 7f8ffcc00640 1 --2- 172.21.15.39:0/3183098907 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8fdc078bc0 0x7f8fdc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:54.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.859+0000 7f8ffcc00640 1 -- 172.21.15.39:0/3183098907 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(265..265 src has 1..265) ==== 7044+0+0 (secure 0 0 0) 0x7f8fe809b4e0 con 0x7f90000d04a0 2024-09-21T07:54:54.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.859+0000 7f8fffe00640 1 --2- 172.21.15.39:0/3183098907 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8fdc078bc0 0x7f8fdc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:54.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.860+0000 7f8fffe00640 1 --2- 172.21.15.39:0/3183098907 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8fdc078bc0 0x7f8fdc07b0d0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0033000 tx=0x7f8ff0004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:54.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:54.864+0000 7f8ffcc00640 1 -- 172.21.15.39:0/3183098907 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8fe8063470 con 0x7f90000d04a0 2024-09-21T07:54:55.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.103+0000 7f9005800640 1 -- 172.21.15.39:0/3183098907 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9000152330 con 0x7f90000d04a0 2024-09-21T07:54:55.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.104+0000 7f8ffcc00640 1 -- 172.21.15.39:0/3183098907 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 310 v310) ==== 95+0+5243 (secure 0 0 0) 0x7f8fe8062d70 con 0x7f90000d04a0 2024-09-21T07:54:55.106 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:55.107 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":310,"btime":"2024-09-21T07:54:42:307372+0000","default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9725,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1018044536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1018044536},{"type":"v1","addr":"172.21.15.148:6837","nonce":1018044536}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":305},{"gid":11125,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1771887993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1771887993},{"type":"v1","addr":"172.21.15.148:6836","nonce":1771887993}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":304},{"gid":11131,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/649113662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":649113662},{"type":"v1","addr":"172.21.15.39:6835","nonce":649113662}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":305}],"filesystems":[{"mdsmap":{"epoch":310,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:54:38.804512+0000","modified":"2024-09-21T07:54:41.814517+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":12315},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12315":{"gid":12315,"name":"c","rank":0,"incarnation":306,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/3094159467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3094159467},{"type":"v1","addr":"172.21.15.39:6834","nonce":3094159467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":12315,"qdb_cluster":[12315]},"id":14}]} 2024-09-21T07:54:55.107 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 310 2024-09-21T07:54:55.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.107+0000 7f9005800640 1 -- 172.21.15.39:0/3183098907 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8fdc078bc0 msgr2=0x7f8fdc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:55.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.107+0000 7f9005800640 1 --2- 172.21.15.39:0/3183098907 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8fdc078bc0 0x7f8fdc07b0d0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0033000 tx=0x7f8ff0004730 comp rx=0 tx=0).stop 2024-09-21T07:54:55.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.107+0000 7f9005800640 1 -- 172.21.15.39:0/3183098907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90000d04a0 msgr2=0x7f9000159d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:55.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.107+0000 7f9005800640 1 --2- 172.21.15.39:0/3183098907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90000d04a0 0x7f9000159d70 secure :-1 s=READY pgs=2855 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800d6d0 tx=0x7f8fe800db90 comp rx=0 tx=0).stop 2024-09-21T07:54:55.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.107+0000 7f9005800640 1 -- 172.21.15.39:0/3183098907 shutdown_connections 2024-09-21T07:54:55.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.107+0000 7f9005800640 1 --2- 172.21.15.39:0/3183098907 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8fdc078bc0 0x7f8fdc07b0d0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:55.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.107+0000 7f9005800640 1 --2- 172.21.15.39:0/3183098907 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f90001519d0 0x7f90001537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:55.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.107+0000 7f9005800640 1 --2- 172.21.15.39:0/3183098907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90000d04a0 0x7f9000159d70 unknown :-1 s=CLOSED pgs=2855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:55.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.107+0000 7f9005800640 1 --2- 172.21.15.39:0/3183098907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f900006ee90 0x7f9000159830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:55.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.107+0000 7f9005800640 1 -- 172.21.15.39:0/3183098907 >> 172.21.15.39:0/3183098907 conn(0x7f900006cfc0 msgr2=0x7f900014a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:55.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.107+0000 7f9005800640 1 -- 172.21.15.39:0/3183098907 shutdown_connections 2024-09-21T07:54:55.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.107+0000 7f9005800640 1 -- 172.21.15.39:0/3183098907 wait complete. 2024-09-21T07:54:55.136 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:54:55.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.365+0000 7f32c9a00640 1 -- 172.21.15.39:0/1058677015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32c40ccc60 msgr2=0x7f32c40cd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:55.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.365+0000 7f32c9a00640 1 --2- 172.21.15.39:0/1058677015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32c40ccc60 0x7f32c40cd070 secure :-1 s=READY pgs=2857 cs=0 l=1 rev1=1 crypto rx=0x7f32bc005fd0 tx=0x7f32bc02a820 comp rx=0 tx=0).stop 2024-09-21T07:54:55.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.365+0000 7f32c9a00640 1 -- 172.21.15.39:0/1058677015 shutdown_connections 2024-09-21T07:54:55.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.365+0000 7f32c9a00640 1 --2- 172.21.15.39:0/1058677015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32c40ccc60 0x7f32c40cd070 unknown :-1 s=CLOSED pgs=2857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:55.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.365+0000 7f32c9a00640 1 --2- 172.21.15.39:0/1058677015 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32c40b30a0 0x7f32c40cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:55.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.365+0000 7f32c9a00640 1 -- 172.21.15.39:0/1058677015 >> 172.21.15.39:0/1058677015 conn(0x7f32c40b2460 msgr2=0x7f32c406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:55.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.365+0000 7f32c9a00640 1 -- 172.21.15.39:0/1058677015 shutdown_connections 2024-09-21T07:54:55.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.365+0000 7f32c9a00640 1 -- 172.21.15.39:0/1058677015 wait complete. 2024-09-21T07:54:55.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.367+0000 7f32c9a00640 1 Processor -- start 2024-09-21T07:54:55.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.368+0000 7f32c9a00640 1 -- start start 2024-09-21T07:54:55.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.369+0000 7f32c9a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32c40b30a0 0x7f32c41f5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:55.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.369+0000 7f32c9a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32c40ccc60 0x7f32c4074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:55.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.369+0000 7f32c9a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32c415d830 0x7f32c40745b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:55.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.369+0000 7f32c9a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f32c40af410 con 0x7f32c415d830 2024-09-21T07:54:55.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.369+0000 7f32c9a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f32c40af290 con 0x7f32c40b30a0 2024-09-21T07:54:55.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.369+0000 7f32c9a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f32c40af590 con 0x7f32c40ccc60 2024-09-21T07:54:55.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.369+0000 7f32c3400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32c415d830 0x7f32c40745b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:55.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.370+0000 7f32c3400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32c415d830 0x7f32c40745b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51678/0 (socket says 172.21.15.39:51678) 2024-09-21T07:54:55.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.370+0000 7f32c3400640 1 -- 172.21.15.39:0/3190986957 learned_addr learned my addr 172.21.15.39:0/3190986957 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:55.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.370+0000 7f32c2a00640 1 --2- 172.21.15.39:0/3190986957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32c40b30a0 0x7f32c41f5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:55.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.370+0000 7f32c2000640 1 --2- 172.21.15.39:0/3190986957 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32c40ccc60 0x7f32c4074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:55.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.370+0000 7f32c3400640 1 -- 172.21.15.39:0/3190986957 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32c40ccc60 msgr2=0x7f32c4074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:55.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.370+0000 7f32c3400640 1 --2- 172.21.15.39:0/3190986957 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32c40ccc60 0x7f32c4074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:55.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.370+0000 7f32c3400640 1 -- 172.21.15.39:0/3190986957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32c40b30a0 msgr2=0x7f32c41f5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:55.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.370+0000 7f32c3400640 1 --2- 172.21.15.39:0/3190986957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32c40b30a0 0x7f32c41f5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:55.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.370+0000 7f32c3400640 1 -- 172.21.15.39:0/3190986957 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c415d3e0 con 0x7f32c415d830 2024-09-21T07:54:55.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.370+0000 7f32c3400640 1 --2- 172.21.15.39:0/3190986957 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32c415d830 0x7f32c40745b0 secure :-1 s=READY pgs=2858 cs=0 l=1 rev1=1 crypto rx=0x7f32bc009cd0 tx=0x7f32bc032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:55.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.373+0000 7f32baa00640 1 -- 172.21.15.39:0/3190986957 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc037040 con 0x7f32c415d830 2024-09-21T07:54:55.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.373+0000 7f32baa00640 1 -- 172.21.15.39:0/3190986957 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32bc042070 con 0x7f32c415d830 2024-09-21T07:54:55.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.374+0000 7f32c9a00640 1 -- 172.21.15.39:0/3190986957 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32c40c5460 con 0x7f32c415d830 2024-09-21T07:54:55.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.374+0000 7f32baa00640 1 -- 172.21.15.39:0/3190986957 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc003e60 con 0x7f32c415d830 2024-09-21T07:54:55.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.374+0000 7f32c9a00640 1 -- 172.21.15.39:0/3190986957 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32c40b10b0 con 0x7f32c415d830 2024-09-21T07:54:55.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.375+0000 7f32c9a00640 1 -- 172.21.15.39:0/3190986957 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32c40c5740 con 0x7f32c415d830 2024-09-21T07:54:55.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.376+0000 7f32baa00640 1 -- 172.21.15.39:0/3190986957 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f32bc004280 con 0x7f32c415d830 2024-09-21T07:54:55.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.377+0000 7f32baa00640 1 --2- 172.21.15.39:0/3190986957 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f32a4078bc0 0x7f32a407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:55.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.377+0000 7f32baa00640 1 -- 172.21.15.39:0/3190986957 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(265..265 src has 1..265) ==== 7044+0+0 (secure 0 0 0) 0x7f32bc0bbf20 con 0x7f32c415d830 2024-09-21T07:54:55.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.377+0000 7f32c2a00640 1 --2- 172.21.15.39:0/3190986957 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f32a4078bc0 0x7f32a407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:55.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.378+0000 7f32c2a00640 1 --2- 172.21.15.39:0/3190986957 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f32a4078bc0 0x7f32a407b0d0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f32b4009b30 tx=0x7f32b4005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:55.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.381+0000 7f32baa00640 1 -- 172.21.15.39:0/3190986957 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f32bc0837c0 con 0x7f32c415d830 2024-09-21T07:54:55.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.614+0000 7f32c9a00640 1 -- 172.21.15.39:0/3190986957 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32c40afa40 con 0x7f32c415d830 2024-09-21T07:54:55.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.615+0000 7f32baa00640 1 -- 172.21.15.39:0/3190986957 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 310 v310) ==== 95+0+5243 (secure 0 0 0) 0x7f32bc0881f0 con 0x7f32c415d830 2024-09-21T07:54:55.617 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:55.617 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":310,"btime":"2024-09-21T07:54:42:307372+0000","default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9725,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1018044536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1018044536},{"type":"v1","addr":"172.21.15.148:6837","nonce":1018044536}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":305},{"gid":11125,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1771887993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1771887993},{"type":"v1","addr":"172.21.15.148:6836","nonce":1771887993}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":304},{"gid":11131,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/649113662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":649113662},{"type":"v1","addr":"172.21.15.39:6835","nonce":649113662}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":305}],"filesystems":[{"mdsmap":{"epoch":310,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:54:38.804512+0000","modified":"2024-09-21T07:54:41.814517+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":12315},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12315":{"gid":12315,"name":"c","rank":0,"incarnation":306,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/3094159467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3094159467},{"type":"v1","addr":"172.21.15.39:6834","nonce":3094159467}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":12315,"qdb_cluster":[12315]},"id":14}]} 2024-09-21T07:54:55.617 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 310 2024-09-21T07:54:55.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.617+0000 7f32c9a00640 1 -- 172.21.15.39:0/3190986957 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f32a4078bc0 msgr2=0x7f32a407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:55.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.617+0000 7f32c9a00640 1 --2- 172.21.15.39:0/3190986957 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f32a4078bc0 0x7f32a407b0d0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f32b4009b30 tx=0x7f32b4005f90 comp rx=0 tx=0).stop 2024-09-21T07:54:55.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.617+0000 7f32c9a00640 1 -- 172.21.15.39:0/3190986957 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32c415d830 msgr2=0x7f32c40745b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:55.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.617+0000 7f32c9a00640 1 --2- 172.21.15.39:0/3190986957 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32c415d830 0x7f32c40745b0 secure :-1 s=READY pgs=2858 cs=0 l=1 rev1=1 crypto rx=0x7f32bc009cd0 tx=0x7f32bc032d10 comp rx=0 tx=0).stop 2024-09-21T07:54:55.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.618+0000 7f32c9a00640 1 -- 172.21.15.39:0/3190986957 shutdown_connections 2024-09-21T07:54:55.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.618+0000 7f32c9a00640 1 --2- 172.21.15.39:0/3190986957 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f32a4078bc0 0x7f32a407b0d0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:55.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.618+0000 7f32c9a00640 1 --2- 172.21.15.39:0/3190986957 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f32c415d830 0x7f32c40745b0 unknown :-1 s=CLOSED pgs=2858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:55.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.618+0000 7f32c9a00640 1 --2- 172.21.15.39:0/3190986957 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f32c40ccc60 0x7f32c4074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:55.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.618+0000 7f32c9a00640 1 --2- 172.21.15.39:0/3190986957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32c40b30a0 0x7f32c41f5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:55.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.618+0000 7f32c9a00640 1 -- 172.21.15.39:0/3190986957 >> 172.21.15.39:0/3190986957 conn(0x7f32c40b2460 msgr2=0x7f32c40b34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:55.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.618+0000 7f32c9a00640 1 -- 172.21.15.39:0/3190986957 shutdown_connections 2024-09-21T07:54:55.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.618+0000 7f32c9a00640 1 -- 172.21.15.39:0/3190986957 wait complete. 2024-09-21T07:54:55.645 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:54:55.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.870+0000 7ff839200640 1 -- 172.21.15.39:0/1794202230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff818003a00 msgr2=0x7ff818003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:55.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.870+0000 7ff839200640 1 --2- 172.21.15.39:0/1794202230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff818003a00 0x7ff818003e30 secure :-1 s=READY pgs=2860 cs=0 l=1 rev1=1 crypto rx=0x7ff82400c290 tx=0x7ff818005830 comp rx=0 tx=0).stop 2024-09-21T07:54:55.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.871+0000 7ff839200640 1 -- 172.21.15.39:0/1794202230 shutdown_connections 2024-09-21T07:54:55.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.871+0000 7ff839200640 1 --2- 172.21.15.39:0/1794202230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff818003a00 0x7ff818003e30 unknown :-1 s=CLOSED pgs=2860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:55.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.871+0000 7ff839200640 1 -- 172.21.15.39:0/1794202230 >> 172.21.15.39:0/1794202230 conn(0x7ff83406cfc0 msgr2=0x7ff83406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:55.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.871+0000 7ff839200640 1 -- 172.21.15.39:0/1794202230 shutdown_connections 2024-09-21T07:54:55.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.871+0000 7ff839200640 1 -- 172.21.15.39:0/1794202230 wait complete. 2024-09-21T07:54:55.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.873+0000 7ff839200640 1 Processor -- start 2024-09-21T07:54:55.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.873+0000 7ff839200640 1 -- start start 2024-09-21T07:54:55.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.874+0000 7ff839200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff818003a00 0x7ff83413c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:55.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.874+0000 7ff839200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff834159480 0x7ff834150350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:55.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.875+0000 7ff839200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff8340b21a0 0x7ff8340b25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:55.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.875+0000 7ff839200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff83415d900 con 0x7ff834159480 2024-09-21T07:54:55.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.875+0000 7ff839200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff83415d780 con 0x7ff818003a00 2024-09-21T07:54:55.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.875+0000 7ff839200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7ff83415da80 con 0x7ff8340b21a0 2024-09-21T07:54:55.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.875+0000 7ff832a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff834159480 0x7ff834150350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:55.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.875+0000 7ff832a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff834159480 0x7ff834150350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51702/0 (socket says 172.21.15.39:51702) 2024-09-21T07:54:55.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.875+0000 7ff832a00640 1 -- 172.21.15.39:0/3325518411 learned_addr learned my addr 172.21.15.39:0/3325518411 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:55.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.875+0000 7ff833400640 1 --2- 172.21.15.39:0/3325518411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff818003a00 0x7ff83413c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:55.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.875+0000 7ff832a00640 1 -- 172.21.15.39:0/3325518411 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff8340b21a0 msgr2=0x7ff8340b25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:55.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.875+0000 7ff832a00640 1 --2- 172.21.15.39:0/3325518411 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff8340b21a0 0x7ff8340b25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:55.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.875+0000 7ff832a00640 1 -- 172.21.15.39:0/3325518411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff818003a00 msgr2=0x7ff83413c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:55.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.875+0000 7ff832a00640 1 --2- 172.21.15.39:0/3325518411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff818003a00 0x7ff83413c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:55.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.875+0000 7ff832a00640 1 -- 172.21.15.39:0/3325518411 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8340b2af0 con 0x7ff834159480 2024-09-21T07:54:55.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.876+0000 7ff832a00640 1 --2- 172.21.15.39:0/3325518411 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff834159480 0x7ff834150350 secure :-1 s=READY pgs=2861 cs=0 l=1 rev1=1 crypto rx=0x7ff820011a10 tx=0x7ff820011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:55.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.876+0000 7ff82b400640 1 -- 172.21.15.39:0/3325518411 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82001e070 con 0x7ff834159480 2024-09-21T07:54:55.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.876+0000 7ff82b400640 1 -- 172.21.15.39:0/3325518411 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff820004460 con 0x7ff834159480 2024-09-21T07:54:55.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.876+0000 7ff839200640 1 -- 172.21.15.39:0/3325518411 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff83413cca0 con 0x7ff834159480 2024-09-21T07:54:55.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.876+0000 7ff82b400640 1 -- 172.21.15.39:0/3325518411 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82000fc00 con 0x7ff834159480 2024-09-21T07:54:55.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.876+0000 7ff839200640 1 -- 172.21.15.39:0/3325518411 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff83413d260 con 0x7ff834159480 2024-09-21T07:54:55.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.877+0000 7ff839200640 1 -- 172.21.15.39:0/3325518411 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8340d0680 con 0x7ff834159480 2024-09-21T07:54:55.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.879+0000 7ff82b400640 1 -- 172.21.15.39:0/3325518411 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff820021070 con 0x7ff834159480 2024-09-21T07:54:55.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.880+0000 7ff82b400640 1 --2- 172.21.15.39:0/3325518411 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff814078b90 0x7ff81407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:55.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.880+0000 7ff82b400640 1 -- 172.21.15.39:0/3325518411 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(265..265 src has 1..265) ==== 7044+0+0 (secure 0 0 0) 0x7ff82009c260 con 0x7ff834159480 2024-09-21T07:54:55.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.881+0000 7ff833400640 1 --2- 172.21.15.39:0/3325518411 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff814078b90 0x7ff81407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:55.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.883+0000 7ff82b400640 1 -- 172.21.15.39:0/3325518411 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff8340b2af0 con 0x7ff834159480 2024-09-21T07:54:55.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:55.884+0000 7ff833400640 1 --2- 172.21.15.39:0/3325518411 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff814078b90 0x7ff81407b0a0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7ff824033000 tx=0x7ff834161000 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:56.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.083+0000 7ff839200640 1 -- 172.21.15.39:0/3325518411 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff83413ca30 con 0x7ff834159480 2024-09-21T07:54:56.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.085+0000 7ff82b400640 1 -- 172.21.15.39:0/3325518411 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v265) ==== 74+0+22596 (secure 0 0 0) 0x7ff820063d10 con 0x7ff834159480 2024-09-21T07:54:56.087 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:54:56.087 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":265,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:54:40.270721+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:54:36.740920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"264","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2024-09-21T07:54:37.755494+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"265","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]},{"pool":20,"snaps":[{"begin":80,"length":23},{"begin":181,"length":23}]},{"pool":22,"snaps":[{"begin":100,"length":1},{"begin":199,"length":1}]},{"pool":24,"snaps":[{"begin":2,"length":2}]},{"pool":26,"snaps":[{"begin":2,"length":2}]},{"pool":28,"snaps":[{"begin":2,"length":2}]},{"pool":30,"snaps":[{"begin":8,"length":6}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:54:56.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.087+0000 7ff839200640 1 -- 172.21.15.39:0/3325518411 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff814078b90 msgr2=0x7ff81407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:56.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.087+0000 7ff839200640 1 --2- 172.21.15.39:0/3325518411 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff814078b90 0x7ff81407b0a0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7ff824033000 tx=0x7ff834161000 comp rx=0 tx=0).stop 2024-09-21T07:54:56.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.088+0000 7ff839200640 1 -- 172.21.15.39:0/3325518411 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff834159480 msgr2=0x7ff834150350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:56.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.088+0000 7ff839200640 1 --2- 172.21.15.39:0/3325518411 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff834159480 0x7ff834150350 secure :-1 s=READY pgs=2861 cs=0 l=1 rev1=1 crypto rx=0x7ff820011a10 tx=0x7ff820011ed0 comp rx=0 tx=0).stop 2024-09-21T07:54:56.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.088+0000 7ff839200640 1 -- 172.21.15.39:0/3325518411 shutdown_connections 2024-09-21T07:54:56.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.088+0000 7ff839200640 1 --2- 172.21.15.39:0/3325518411 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff814078b90 0x7ff81407b0a0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:56.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.088+0000 7ff839200640 1 --2- 172.21.15.39:0/3325518411 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff8340b21a0 0x7ff8340b25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:56.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.088+0000 7ff839200640 1 --2- 172.21.15.39:0/3325518411 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff834159480 0x7ff834150350 unknown :-1 s=CLOSED pgs=2861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:56.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.088+0000 7ff839200640 1 --2- 172.21.15.39:0/3325518411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff818003a00 0x7ff83413c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:56.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.088+0000 7ff839200640 1 -- 172.21.15.39:0/3325518411 >> 172.21.15.39:0/3325518411 conn(0x7ff83406cfc0 msgr2=0x7ff83415de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:56.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.088+0000 7ff839200640 1 -- 172.21.15.39:0/3325518411 shutdown_connections 2024-09-21T07:54:56.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.088+0000 7ff839200640 1 -- 172.21.15.39:0/3325518411 wait complete. 2024-09-21T07:54:56.113 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T07:54:56.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.339+0000 7f50b2c00640 1 -- 172.21.15.39:0/906976204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50ac0b5450 msgr2=0x7f50ac0d27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:56.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.339+0000 7f50b2c00640 1 --2- 172.21.15.39:0/906976204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50ac0b5450 0x7f50ac0d27e0 secure :-1 s=READY pgs=2862 cs=0 l=1 rev1=1 crypto rx=0x7f5098005fd0 tx=0x7f509802a620 comp rx=0 tx=0).stop 2024-09-21T07:54:56.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.339+0000 7f50b2c00640 1 -- 172.21.15.39:0/906976204 shutdown_connections 2024-09-21T07:54:56.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.339+0000 7f50b2c00640 1 --2- 172.21.15.39:0/906976204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50ac0b5450 0x7f50ac0d27e0 unknown :-1 s=CLOSED pgs=2862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:56.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.339+0000 7f50b2c00640 1 --2- 172.21.15.39:0/906976204 >> v2:172.21.15.148:3300/0 conn(0x7f50ac0b97e0 0x7f50ac0b6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:56.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.339+0000 7f50b2c00640 1 -- 172.21.15.39:0/906976204 >> 172.21.15.39:0/906976204 conn(0x7f50ac09d7d0 msgr2=0x7f50ac0d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:56.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.339+0000 7f50b2c00640 1 -- 172.21.15.39:0/906976204 shutdown_connections 2024-09-21T07:54:56.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.339+0000 7f50b2c00640 1 -- 172.21.15.39:0/906976204 wait complete. 2024-09-21T07:54:56.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.341+0000 7f50b2c00640 1 Processor -- start 2024-09-21T07:54:56.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.341+0000 7f50b2c00640 1 -- start start 2024-09-21T07:54:56.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.343+0000 7f50b2c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50ac0b5450 0x7f50ac0ce310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:56.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.343+0000 7f50b2c00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f50ac0b97e0 0x7f50ac15d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:56.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.343+0000 7f50b2c00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50ac0d2d20 0x7f50ac0ca200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:56.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.343+0000 7f50b2c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac0cecd0 con 0x7f50ac0b5450 2024-09-21T07:54:56.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.343+0000 7f50b2c00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac0ceb50 con 0x7f50ac0d2d20 2024-09-21T07:54:56.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.343+0000 7f50b2c00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f50ac0cee50 con 0x7f50ac0b97e0 2024-09-21T07:54:56.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.343+0000 7f50b1800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50ac0b5450 0x7f50ac0ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:56.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.343+0000 7f50b1800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50ac0b5450 0x7f50ac0ce310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51708/0 (socket says 172.21.15.39:51708) 2024-09-21T07:54:56.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.343+0000 7f50b1800640 1 -- 172.21.15.39:0/575898591 learned_addr learned my addr 172.21.15.39:0/575898591 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:56.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.343+0000 7f50b2200640 1 --2- 172.21.15.39:0/575898591 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50ac0d2d20 0x7f50ac0ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:56.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.343+0000 7f50b0e00640 1 --2- 172.21.15.39:0/575898591 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f50ac0b97e0 0x7f50ac15d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:56.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.343+0000 7f50b1800640 1 -- 172.21.15.39:0/575898591 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f50ac0b97e0 msgr2=0x7f50ac15d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:56.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.343+0000 7f50b1800640 1 --2- 172.21.15.39:0/575898591 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f50ac0b97e0 0x7f50ac15d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:56.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.344+0000 7f50b1800640 1 -- 172.21.15.39:0/575898591 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50ac0d2d20 msgr2=0x7f50ac0ca200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:56.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.344+0000 7f50b1800640 1 --2- 172.21.15.39:0/575898591 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50ac0d2d20 0x7f50ac0ca200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:56.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.344+0000 7f50b1800640 1 -- 172.21.15.39:0/575898591 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50ac1f5df0 con 0x7f50ac0b5450 2024-09-21T07:54:56.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.344+0000 7f50b1800640 1 --2- 172.21.15.39:0/575898591 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50ac0b5450 0x7f50ac0ce310 secure :-1 s=READY pgs=2863 cs=0 l=1 rev1=1 crypto rx=0x7f50a000d6d0 tx=0x7f50a000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:56.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.344+0000 7f50a9600640 1 -- 172.21.15.39:0/575898591 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a0017070 con 0x7f50ac0b5450 2024-09-21T07:54:56.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.344+0000 7f50a9600640 1 -- 172.21.15.39:0/575898591 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50a0005330 con 0x7f50ac0b5450 2024-09-21T07:54:56.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.344+0000 7f50a9600640 1 -- 172.21.15.39:0/575898591 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a0009820 con 0x7f50ac0b5450 2024-09-21T07:54:56.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.344+0000 7f50b2c00640 1 -- 172.21.15.39:0/575898591 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50ac1f60d0 con 0x7f50ac0b5450 2024-09-21T07:54:56.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.344+0000 7f50b2c00640 1 -- 172.21.15.39:0/575898591 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50ac0ce850 con 0x7f50ac0b5450 2024-09-21T07:54:56.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.345+0000 7f50b2c00640 1 -- 172.21.15.39:0/575898591 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50ac0d27e0 con 0x7f50ac0b5450 2024-09-21T07:54:56.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.346+0000 7f50a9600640 1 -- 172.21.15.39:0/575898591 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f50a0009bd0 con 0x7f50ac0b5450 2024-09-21T07:54:56.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.347+0000 7f50a9600640 1 --2- 172.21.15.39:0/575898591 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f508c078b90 0x7f508c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:56.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.347+0000 7f50a9600640 1 -- 172.21.15.39:0/575898591 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(265..265 src has 1..265) ==== 7044+0+0 (secure 0 0 0) 0x7f50a009b590 con 0x7f50ac0b5450 2024-09-21T07:54:56.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.348+0000 7f50b0e00640 1 --2- 172.21.15.39:0/575898591 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f508c078b90 0x7f508c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:56.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.349+0000 7f50b0e00640 1 --2- 172.21.15.39:0/575898591 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f508c078b90 0x7f508c07b0a0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f5098033000 tx=0x7f5098004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:56.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.352+0000 7f50a9600640 1 -- 172.21.15.39:0/575898591 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f50a0063480 con 0x7f50ac0b5450 2024-09-21T07:54:56.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.597+0000 7f50b2c00640 1 -- 172.21.15.39:0/575898591 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f50ac06f3d0 con 0x7f50ac0b5450 2024-09-21T07:54:56.599 INFO:tasks.ceph.mon.a.smithi039.stderr:2024-09-21T07:54:56.598+0000 7fabab600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T07:54:56.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.600+0000 7f50a9600640 1 -- 172.21.15.39:0/575898591 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v311) ==== 161+0+0 (secure 0 0 0) 0x7f50a0062d80 con 0x7f50ac0b5450 2024-09-21T07:54:56.602 INFO:teuthology.orchestra.run.smithi039.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T07:54:56.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.603+0000 7f50b2c00640 1 -- 172.21.15.39:0/575898591 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f508c078b90 msgr2=0x7f508c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:56.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.603+0000 7f50b2c00640 1 --2- 172.21.15.39:0/575898591 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f508c078b90 0x7f508c07b0a0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f5098033000 tx=0x7f5098004730 comp rx=0 tx=0).stop 2024-09-21T07:54:56.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.603+0000 7f50b2c00640 1 -- 172.21.15.39:0/575898591 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50ac0b5450 msgr2=0x7f50ac0ce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:56.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.603+0000 7f50b2c00640 1 --2- 172.21.15.39:0/575898591 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50ac0b5450 0x7f50ac0ce310 secure :-1 s=READY pgs=2863 cs=0 l=1 rev1=1 crypto rx=0x7f50a000d6d0 tx=0x7f50a000db90 comp rx=0 tx=0).stop 2024-09-21T07:54:56.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.605+0000 7f50b2c00640 1 -- 172.21.15.39:0/575898591 shutdown_connections 2024-09-21T07:54:56.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.605+0000 7f50b2c00640 1 --2- 172.21.15.39:0/575898591 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f508c078b90 0x7f508c07b0a0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:56.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.605+0000 7f50b2c00640 1 --2- 172.21.15.39:0/575898591 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f50ac0d2d20 0x7f50ac0ca200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:56.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.605+0000 7f50b2c00640 1 --2- 172.21.15.39:0/575898591 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f50ac0b97e0 0x7f50ac15d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:56.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.605+0000 7f50b2c00640 1 --2- 172.21.15.39:0/575898591 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50ac0b5450 0x7f50ac0ce310 unknown :-1 s=CLOSED pgs=2863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:56.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.605+0000 7f50b2c00640 1 -- 172.21.15.39:0/575898591 >> 172.21.15.39:0/575898591 conn(0x7f50ac09d7d0 msgr2=0x7f50ac15ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:56.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.605+0000 7f50b2c00640 1 -- 172.21.15.39:0/575898591 shutdown_connections 2024-09-21T07:54:56.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.606+0000 7f50b2c00640 1 -- 172.21.15.39:0/575898591 wait complete. 2024-09-21T07:54:56.618 INFO:tasks.ceph.mds.c.smithi039.stderr: -6643> 2024-09-21T07:54:41.248+0000 7fa436800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:54:56.637 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T07:54:56.749 INFO:tasks.ceph.mds.c.smithi039.stdout:starting mds.c at 2024-09-21T07:54:56.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.868+0000 7f2173200640 1 -- 172.21.15.39:0/3704261374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21740d1580 msgr2=0x7f21740d1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:56.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.868+0000 7f2173200640 1 --2- 172.21.15.39:0/3704261374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21740d1580 0x7f21740d1ac0 secure :-1 s=READY pgs=2867 cs=0 l=1 rev1=1 crypto rx=0x7f215c009830 tx=0x7f215c02a5c0 comp rx=0 tx=0).stop 2024-09-21T07:54:56.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.868+0000 7f2173200640 1 -- 172.21.15.39:0/3704261374 shutdown_connections 2024-09-21T07:54:56.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.868+0000 7f2173200640 1 --2- 172.21.15.39:0/3704261374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21740d1580 0x7f21740d1ac0 unknown :-1 s=CLOSED pgs=2867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:56.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.868+0000 7f2173200640 1 -- 172.21.15.39:0/3704261374 >> 172.21.15.39:0/3704261374 conn(0x7f217409d7b0 msgr2=0x7f21740d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:56.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.868+0000 7f2173200640 1 -- 172.21.15.39:0/3704261374 shutdown_connections 2024-09-21T07:54:56.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.868+0000 7f2173200640 1 -- 172.21.15.39:0/3704261374 wait complete. 2024-09-21T07:54:56.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.870+0000 7f2173200640 1 Processor -- start 2024-09-21T07:54:56.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.870+0000 7f2173200640 1 -- start start 2024-09-21T07:54:56.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.872+0000 7f2173200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21740b3b60 0x7f21741f5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:56.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.872+0000 7f2173200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21740b5a90 0x7f21740b2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:56.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.872+0000 7f2173200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f21740b2880 0x7f21741f6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:56.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.872+0000 7f2173200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f21740b4800 con 0x7f21740b3b60 2024-09-21T07:54:56.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.872+0000 7f2173200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f21740b4680 con 0x7f21740b5a90 2024-09-21T07:54:56.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.872+0000 7f2173200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f21740b4980 con 0x7f21740b2880 2024-09-21T07:54:56.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.872+0000 7f2171e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21740b3b60 0x7f21741f5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:56.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.872+0000 7f2171e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21740b3b60 0x7f21741f5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51752/0 (socket says 172.21.15.39:51752) 2024-09-21T07:54:56.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.872+0000 7f2171e00640 1 -- 172.21.15.39:0/1565667016 learned_addr learned my addr 172.21.15.39:0/1565667016 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:56.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.872+0000 7f216be00640 1 --2- 172.21.15.39:0/1565667016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21740b5a90 0x7f21740b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:56.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.873+0000 7f2172800640 1 --2- 172.21.15.39:0/1565667016 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f21740b2880 0x7f21741f6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:56.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.873+0000 7f2171e00640 1 -- 172.21.15.39:0/1565667016 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f21740b2880 msgr2=0x7f21741f6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:56.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.873+0000 7f2171e00640 1 --2- 172.21.15.39:0/1565667016 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f21740b2880 0x7f21741f6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:56.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.873+0000 7f2171e00640 1 -- 172.21.15.39:0/1565667016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21740b5a90 msgr2=0x7f21740b2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:56.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.873+0000 7f2171e00640 1 --2- 172.21.15.39:0/1565667016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21740b5a90 0x7f21740b2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:56.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.873+0000 7f2171e00640 1 -- 172.21.15.39:0/1565667016 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f217415cbb0 con 0x7f21740b3b60 2024-09-21T07:54:56.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.873+0000 7f216be00640 1 --2- 172.21.15.39:0/1565667016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21740b5a90 0x7f21740b2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:56.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.873+0000 7f2171e00640 1 --2- 172.21.15.39:0/1565667016 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21740b3b60 0x7f21741f5d20 secure :-1 s=READY pgs=2868 cs=0 l=1 rev1=1 crypto rx=0x7f216400c390 tx=0x7f216400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:56.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.873+0000 7f216a000640 1 -- 172.21.15.39:0/1565667016 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2164017070 con 0x7f21740b3b60 2024-09-21T07:54:56.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.873+0000 7f216a000640 1 -- 172.21.15.39:0/1565667016 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21640035f0 con 0x7f21740b3b60 2024-09-21T07:54:56.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.873+0000 7f216a000640 1 -- 172.21.15.39:0/1565667016 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2164003a20 con 0x7f21740b3b60 2024-09-21T07:54:56.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.873+0000 7f2173200640 1 -- 172.21.15.39:0/1565667016 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f217415ce90 con 0x7f21740b3b60 2024-09-21T07:54:56.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.874+0000 7f2173200640 1 -- 172.21.15.39:0/1565667016 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f217415d450 con 0x7f21740b3b60 2024-09-21T07:54:56.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.875+0000 7f2173200640 1 -- 172.21.15.39:0/1565667016 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21740b42a0 con 0x7f21740b3b60 2024-09-21T07:54:56.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.876+0000 7f216a000640 1 -- 172.21.15.39:0/1565667016 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f216401a070 con 0x7f21740b3b60 2024-09-21T07:54:56.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.878+0000 7f216a000640 1 --2- 172.21.15.39:0/1565667016 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2150078bc0 0x7f215007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:56.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.878+0000 7f216a000640 1 -- 172.21.15.39:0/1565667016 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(267..267 src has 1..267) ==== 7217+0+0 (secure 0 0 0) 0x7f216409a630 con 0x7f21740b3b60 2024-09-21T07:54:56.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.878+0000 7f216be00640 1 --2- 172.21.15.39:0/1565667016 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2150078bc0 0x7f215007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:56.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.881+0000 7f216be00640 1 --2- 172.21.15.39:0/1565667016 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2150078bc0 0x7f215007b0d0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f215c034000 tx=0x7f215c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:56.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:56.883+0000 7f216a000640 1 -- 172.21.15.39:0/1565667016 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f21740b42a0 con 0x7f21740b3b60 2024-09-21T07:54:57.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.120+0000 7f2173200640 1 -- 172.21.15.39:0/1565667016 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f21740d1ac0 con 0x7f21740b3b60 2024-09-21T07:54:57.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.606+0000 7f216a000640 1 -- 172.21.15.39:0/1565667016 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v312) ==== 104+0+0 (secure 0 0 0) 0x7f2164062470 con 0x7f21740b3b60 2024-09-21T07:54:57.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.608+0000 7f2173200640 1 -- 172.21.15.39:0/1565667016 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2150078bc0 msgr2=0x7f215007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:57.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.608+0000 7f2173200640 1 --2- 172.21.15.39:0/1565667016 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2150078bc0 0x7f215007b0d0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f215c034000 tx=0x7f215c004730 comp rx=0 tx=0).stop 2024-09-21T07:54:57.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.608+0000 7f2173200640 1 -- 172.21.15.39:0/1565667016 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21740b3b60 msgr2=0x7f21741f5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:57.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.608+0000 7f2173200640 1 --2- 172.21.15.39:0/1565667016 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21740b3b60 0x7f21741f5d20 secure :-1 s=READY pgs=2868 cs=0 l=1 rev1=1 crypto rx=0x7f216400c390 tx=0x7f216400c850 comp rx=0 tx=0).stop 2024-09-21T07:54:57.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.609+0000 7f2173200640 1 -- 172.21.15.39:0/1565667016 shutdown_connections 2024-09-21T07:54:57.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.609+0000 7f2173200640 1 --2- 172.21.15.39:0/1565667016 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f2150078bc0 0x7f215007b0d0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:57.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.609+0000 7f2173200640 1 --2- 172.21.15.39:0/1565667016 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f21740b2880 0x7f21741f6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:57.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.609+0000 7f2173200640 1 --2- 172.21.15.39:0/1565667016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21740b5a90 0x7f21740b2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:57.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.609+0000 7f2173200640 1 --2- 172.21.15.39:0/1565667016 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21740b3b60 0x7f21741f5d20 unknown :-1 s=CLOSED pgs=2868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:57.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.609+0000 7f2173200640 1 -- 172.21.15.39:0/1565667016 >> 172.21.15.39:0/1565667016 conn(0x7f217409d7b0 msgr2=0x7f2174161830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:57.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.609+0000 7f2173200640 1 -- 172.21.15.39:0/1565667016 shutdown_connections 2024-09-21T07:54:57.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.609+0000 7f2173200640 1 -- 172.21.15.39:0/1565667016 wait complete. 2024-09-21T07:54:57.636 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T07:54:57.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.868+0000 7fe6bbe00640 1 -- 172.21.15.39:0/2358690840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6bc0cf990 msgr2=0x7fe6bc0cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:57.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.868+0000 7fe6bbe00640 1 --2- 172.21.15.39:0/2358690840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6bc0cf990 0x7fe6bc0cfda0 secure :-1 s=READY pgs=2870 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4005fd0 tx=0x7fe6b402a8b0 comp rx=0 tx=0).stop 2024-09-21T07:54:57.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.868+0000 7fe6bbe00640 1 -- 172.21.15.39:0/2358690840 shutdown_connections 2024-09-21T07:54:57.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.868+0000 7fe6bbe00640 1 --2- 172.21.15.39:0/2358690840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6bc0cf990 0x7fe6bc0cfda0 unknown :-1 s=CLOSED pgs=2870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:57.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.868+0000 7fe6bbe00640 1 --2- 172.21.15.39:0/2358690840 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe6bc071bf0 0x7fe6bc152290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:57.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.868+0000 7fe6bbe00640 1 -- 172.21.15.39:0/2358690840 >> 172.21.15.39:0/2358690840 conn(0x7fe6bc0d1420 msgr2=0x7fe6bc073720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:57.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.868+0000 7fe6bbe00640 1 -- 172.21.15.39:0/2358690840 shutdown_connections 2024-09-21T07:54:57.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.868+0000 7fe6bbe00640 1 -- 172.21.15.39:0/2358690840 wait complete. 2024-09-21T07:54:57.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.870+0000 7fe6bbe00640 1 Processor -- start 2024-09-21T07:54:57.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.870+0000 7fe6bbe00640 1 -- start start 2024-09-21T07:54:57.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.872+0000 7fe6bbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6bc071bf0 0x7fe6bc157ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:57.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.872+0000 7fe6bbe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6bc0cf990 0x7fe6bc143d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:57.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.872+0000 7fe6bbe00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe6bc15f060 0x7fe6bc156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:57.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.872+0000 7fe6bbe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe6bc0d02e0 con 0x7fe6bc071bf0 2024-09-21T07:54:57.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.872+0000 7fe6bbe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe6bc06fb80 con 0x7fe6bc0cf990 2024-09-21T07:54:57.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.872+0000 7fe6bbe00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fe6bc0d0460 con 0x7fe6bc15f060 2024-09-21T07:54:57.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.872+0000 7fe6baa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6bc071bf0 0x7fe6bc157ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:57.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.872+0000 7fe6baa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6bc071bf0 0x7fe6bc157ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51782/0 (socket says 172.21.15.39:51782) 2024-09-21T07:54:57.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.872+0000 7fe6baa00640 1 -- 172.21.15.39:0/3924389609 learned_addr learned my addr 172.21.15.39:0/3924389609 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:57.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.872+0000 7fe6bb400640 1 --2- 172.21.15.39:0/3924389609 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe6bc15f060 0x7fe6bc156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:57.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.872+0000 7fe6b2000640 1 --2- 172.21.15.39:0/3924389609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6bc0cf990 0x7fe6bc143d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:57.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.873+0000 7fe6baa00640 1 -- 172.21.15.39:0/3924389609 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe6bc15f060 msgr2=0x7fe6bc156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:57.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.873+0000 7fe6baa00640 1 --2- 172.21.15.39:0/3924389609 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe6bc15f060 0x7fe6bc156f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:57.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.873+0000 7fe6baa00640 1 -- 172.21.15.39:0/3924389609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6bc0cf990 msgr2=0x7fe6bc143d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:57.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.873+0000 7fe6baa00640 1 --2- 172.21.15.39:0/3924389609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6bc0cf990 0x7fe6bc143d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:57.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.873+0000 7fe6baa00640 1 -- 172.21.15.39:0/3924389609 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6bc1f6350 con 0x7fe6bc071bf0 2024-09-21T07:54:57.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.873+0000 7fe6b2000640 1 --2- 172.21.15.39:0/3924389609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6bc0cf990 0x7fe6bc143d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:57.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.873+0000 7fe6baa00640 1 --2- 172.21.15.39:0/3924389609 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6bc071bf0 0x7fe6bc157ee0 secure :-1 s=READY pgs=2871 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac00d770 tx=0x7fe6ac00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:57.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.873+0000 7fe6b3400640 1 -- 172.21.15.39:0/3924389609 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac017070 con 0x7fe6bc071bf0 2024-09-21T07:54:57.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.873+0000 7fe6b3400640 1 -- 172.21.15.39:0/3924389609 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6ac009cc0 con 0x7fe6bc071bf0 2024-09-21T07:54:57.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.873+0000 7fe6bbe00640 1 -- 172.21.15.39:0/3924389609 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6bc1f5590 con 0x7fe6bc071bf0 2024-09-21T07:54:57.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.873+0000 7fe6b3400640 1 -- 172.21.15.39:0/3924389609 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac0106c0 con 0x7fe6bc071bf0 2024-09-21T07:54:57.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.873+0000 7fe6bbe00640 1 -- 172.21.15.39:0/3924389609 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6bc1f5b50 con 0x7fe6bc071bf0 2024-09-21T07:54:57.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.874+0000 7fe6bbe00640 1 -- 172.21.15.39:0/3924389609 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6bc152290 con 0x7fe6bc071bf0 2024-09-21T07:54:57.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.876+0000 7fe6b3400640 1 -- 172.21.15.39:0/3924389609 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe6ac01a070 con 0x7fe6bc071bf0 2024-09-21T07:54:57.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.877+0000 7fe6b3400640 1 --2- 172.21.15.39:0/3924389609 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe698078b90 0x7fe69807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:57.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.878+0000 7fe6b3400640 1 -- 172.21.15.39:0/3924389609 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(267..267 src has 1..267) ==== 7217+0+0 (secure 0 0 0) 0x7fe6ac09b400 con 0x7fe6bc071bf0 2024-09-21T07:54:57.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.878+0000 7fe6b2000640 1 --2- 172.21.15.39:0/3924389609 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe698078b90 0x7fe69807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:57.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.879+0000 7fe6b2000640 1 --2- 172.21.15.39:0/3924389609 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe698078b90 0x7fe69807b0a0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc151910 tx=0x7fe6a4009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:57.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:57.882+0000 7fe6b3400640 1 -- 172.21.15.39:0/3924389609 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe6ac062be0 con 0x7fe6bc071bf0 2024-09-21T07:54:58.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.085+0000 7fe6bbe00640 1 -- 172.21.15.39:0/3924389609 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe6bc0b4fa0 con 0x7fe6bc071bf0 2024-09-21T07:54:58.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.608+0000 7fe6b3400640 1 -- 172.21.15.39:0/3924389609 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v268) ==== 181+0+0 (secure 0 0 0) 0x7fe6ac067610 con 0x7fe6bc071bf0 2024-09-21T07:54:58.610 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' removed 2024-09-21T07:54:58.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.610+0000 7fe6bbe00640 1 -- 172.21.15.39:0/3924389609 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe698078b90 msgr2=0x7fe69807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:58.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.610+0000 7fe6bbe00640 1 --2- 172.21.15.39:0/3924389609 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe698078b90 0x7fe69807b0a0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc151910 tx=0x7fe6a4009040 comp rx=0 tx=0).stop 2024-09-21T07:54:58.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.610+0000 7fe6bbe00640 1 -- 172.21.15.39:0/3924389609 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6bc071bf0 msgr2=0x7fe6bc157ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:58.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.610+0000 7fe6bbe00640 1 --2- 172.21.15.39:0/3924389609 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6bc071bf0 0x7fe6bc157ee0 secure :-1 s=READY pgs=2871 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac00d770 tx=0x7fe6ac00dc30 comp rx=0 tx=0).stop 2024-09-21T07:54:58.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.611+0000 7fe6bbe00640 1 -- 172.21.15.39:0/3924389609 shutdown_connections 2024-09-21T07:54:58.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.611+0000 7fe6bbe00640 1 --2- 172.21.15.39:0/3924389609 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fe698078b90 0x7fe69807b0a0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:58.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.611+0000 7fe6bbe00640 1 --2- 172.21.15.39:0/3924389609 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fe6bc15f060 0x7fe6bc156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:58.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.611+0000 7fe6bbe00640 1 --2- 172.21.15.39:0/3924389609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6bc0cf990 0x7fe6bc143d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:58.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.611+0000 7fe6bbe00640 1 --2- 172.21.15.39:0/3924389609 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6bc071bf0 0x7fe6bc157ee0 unknown :-1 s=CLOSED pgs=2871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:58.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.611+0000 7fe6bbe00640 1 -- 172.21.15.39:0/3924389609 >> 172.21.15.39:0/3924389609 conn(0x7fe6bc0d1420 msgr2=0x7fe6bc06d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:58.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.611+0000 7fe6bbe00640 1 -- 172.21.15.39:0/3924389609 shutdown_connections 2024-09-21T07:54:58.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.611+0000 7fe6bbe00640 1 -- 172.21.15.39:0/3924389609 wait complete. 2024-09-21T07:54:58.641 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T07:54:58.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.876+0000 7f4927200640 1 -- 172.21.15.39:0/2859663916 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f492806f4b0 msgr2=0x7f49280732b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:58.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.876+0000 7f4927200640 1 --2- 172.21.15.39:0/2859663916 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f492806f4b0 0x7f49280732b0 secure :-1 s=READY pgs=2873 cs=0 l=1 rev1=1 crypto rx=0x7f4910009830 tx=0x7f491002a510 comp rx=0 tx=0).stop 2024-09-21T07:54:58.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.876+0000 7f4927200640 1 -- 172.21.15.39:0/2859663916 shutdown_connections 2024-09-21T07:54:58.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.876+0000 7f4927200640 1 --2- 172.21.15.39:0/2859663916 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f492806f4b0 0x7f49280732b0 unknown :-1 s=CLOSED pgs=2873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:58.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.876+0000 7f4927200640 1 --2- 172.21.15.39:0/2859663916 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49280b4db0 0x7f492806ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:58.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.876+0000 7f4927200640 1 -- 172.21.15.39:0/2859663916 >> 172.21.15.39:0/2859663916 conn(0x7f492806d660 msgr2=0x7f49280b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:58.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.876+0000 7f4927200640 1 -- 172.21.15.39:0/2859663916 shutdown_connections 2024-09-21T07:54:58.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.876+0000 7f4927200640 1 -- 172.21.15.39:0/2859663916 wait complete. 2024-09-21T07:54:58.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.878+0000 7f4927200640 1 Processor -- start 2024-09-21T07:54:58.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.878+0000 7f4927200640 1 -- start start 2024-09-21T07:54:58.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.880+0000 7f4927200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f492806f4b0 0x7f4928154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:58.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.880+0000 7f4927200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49280737f0 0x7f492815dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:58.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.880+0000 7f4927200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49280b4db0 0x7f4928158000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:58.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.880+0000 7f4927200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4928155b80 con 0x7f492806f4b0 2024-09-21T07:54:58.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.880+0000 7f4927200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4928155a00 con 0x7f49280737f0 2024-09-21T07:54:58.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.880+0000 7f4927200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f4928155d00 con 0x7f49280b4db0 2024-09-21T07:54:58.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.880+0000 7f4925e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f492806f4b0 0x7f4928154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:58.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.880+0000 7f4925e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f492806f4b0 0x7f4928154050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51796/0 (socket says 172.21.15.39:51796) 2024-09-21T07:54:58.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.880+0000 7f4925e00640 1 -- 172.21.15.39:0/2330145393 learned_addr learned my addr 172.21.15.39:0/2330145393 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:58.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.880+0000 7f491d400640 1 --2- 172.21.15.39:0/2330145393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49280737f0 0x7f492815dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:58.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.880+0000 7f4926800640 1 --2- 172.21.15.39:0/2330145393 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49280b4db0 0x7f4928158000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:58.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.881+0000 7f4925e00640 1 -- 172.21.15.39:0/2330145393 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49280b4db0 msgr2=0x7f4928158000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:58.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.881+0000 7f4925e00640 1 --2- 172.21.15.39:0/2330145393 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49280b4db0 0x7f4928158000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:58.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.881+0000 7f4925e00640 1 -- 172.21.15.39:0/2330145393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49280737f0 msgr2=0x7f492815dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:58.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.881+0000 7f4925e00640 1 --2- 172.21.15.39:0/2330145393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49280737f0 0x7f492815dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:58.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.881+0000 7f4925e00640 1 -- 172.21.15.39:0/2330145393 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49281f5900 con 0x7f492806f4b0 2024-09-21T07:54:58.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.881+0000 7f491d400640 1 --2- 172.21.15.39:0/2330145393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49280737f0 0x7f492815dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:58.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.881+0000 7f4925e00640 1 --2- 172.21.15.39:0/2330145393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f492806f4b0 0x7f4928154050 secure :-1 s=READY pgs=2874 cs=0 l=1 rev1=1 crypto rx=0x7f491800ea20 tx=0x7f491800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:58.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.881+0000 7f491ea00640 1 -- 172.21.15.39:0/2330145393 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491800cde0 con 0x7f492806f4b0 2024-09-21T07:54:58.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.881+0000 7f491ea00640 1 -- 172.21.15.39:0/2330145393 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4918011560 con 0x7f492806f4b0 2024-09-21T07:54:58.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.881+0000 7f491ea00640 1 -- 172.21.15.39:0/2330145393 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4918011990 con 0x7f492806f4b0 2024-09-21T07:54:58.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.881+0000 7f4927200640 1 -- 172.21.15.39:0/2330145393 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49281f5be0 con 0x7f492806f4b0 2024-09-21T07:54:58.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.881+0000 7f4927200640 1 -- 172.21.15.39:0/2330145393 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49281f61a0 con 0x7f492806f4b0 2024-09-21T07:54:58.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.882+0000 7f4927200640 1 -- 172.21.15.39:0/2330145393 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49280732b0 con 0x7f492806f4b0 2024-09-21T07:54:58.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.884+0000 7f491ea00640 1 -- 172.21.15.39:0/2330145393 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4918011b30 con 0x7f492806f4b0 2024-09-21T07:54:58.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.885+0000 7f491ea00640 1 --2- 172.21.15.39:0/2330145393 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4904078b90 0x7f490407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:58.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.885+0000 7f491ea00640 1 -- 172.21.15.39:0/2330145393 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(268..268 src has 1..268) ==== 6779+0+0 (secure 0 0 0) 0x7f491809bed0 con 0x7f492806f4b0 2024-09-21T07:54:58.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.885+0000 7f491d400640 1 --2- 172.21.15.39:0/2330145393 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4904078b90 0x7f490407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:58.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.886+0000 7f491d400640 1 --2- 172.21.15.39:0/2330145393 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4904078b90 0x7f490407b0a0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f4910034000 tx=0x7f4910004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:58.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:58.889+0000 7f491ea00640 1 -- 172.21.15.39:0/2330145393 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4918016030 con 0x7f492806f4b0 2024-09-21T07:54:59.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.085+0000 7f4927200640 1 -- 172.21.15.39:0/2330145393 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4928154590 con 0x7f492806f4b0 2024-09-21T07:54:59.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.625+0000 7f491ea00640 1 -- 172.21.15.39:0/2330145393 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v269) ==== 169+0+0 (secure 0 0 0) 0x7f4918011e20 con 0x7f492806f4b0 2024-09-21T07:54:59.628 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' removed 2024-09-21T07:54:59.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.628+0000 7f4927200640 1 -- 172.21.15.39:0/2330145393 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4904078b90 msgr2=0x7f490407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:59.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.628+0000 7f4927200640 1 --2- 172.21.15.39:0/2330145393 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4904078b90 0x7f490407b0a0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f4910034000 tx=0x7f4910004730 comp rx=0 tx=0).stop 2024-09-21T07:54:59.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.628+0000 7f4927200640 1 -- 172.21.15.39:0/2330145393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f492806f4b0 msgr2=0x7f4928154050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:59.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.628+0000 7f4927200640 1 --2- 172.21.15.39:0/2330145393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f492806f4b0 0x7f4928154050 secure :-1 s=READY pgs=2874 cs=0 l=1 rev1=1 crypto rx=0x7f491800ea20 tx=0x7f491800eee0 comp rx=0 tx=0).stop 2024-09-21T07:54:59.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.630+0000 7f4927200640 1 -- 172.21.15.39:0/2330145393 shutdown_connections 2024-09-21T07:54:59.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.630+0000 7f4927200640 1 --2- 172.21.15.39:0/2330145393 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4904078b90 0x7f490407b0a0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:59.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.630+0000 7f4927200640 1 --2- 172.21.15.39:0/2330145393 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f49280b4db0 0x7f4928158000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:59.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.630+0000 7f4927200640 1 --2- 172.21.15.39:0/2330145393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f49280737f0 0x7f492815dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:59.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.630+0000 7f4927200640 1 --2- 172.21.15.39:0/2330145393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f492806f4b0 0x7f4928154050 unknown :-1 s=CLOSED pgs=2874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:59.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.630+0000 7f4927200640 1 -- 172.21.15.39:0/2330145393 >> 172.21.15.39:0/2330145393 conn(0x7f492806d660 msgr2=0x7f49280aeaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:59.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.631+0000 7f4927200640 1 -- 172.21.15.39:0/2330145393 shutdown_connections 2024-09-21T07:54:59.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.631+0000 7f4927200640 1 -- 172.21.15.39:0/2330145393 wait complete. 2024-09-21T07:54:59.661 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T07:54:59.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.887+0000 7f54adc00640 1 -- 172.21.15.39:0/3928674296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54a815d830 msgr2=0x7f54a8156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:59.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.887+0000 7f54adc00640 1 --2- 172.21.15.39:0/3928674296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54a815d830 0x7f54a8156bb0 secure :-1 s=READY pgs=2876 cs=0 l=1 rev1=1 crypto rx=0x7f5490009830 tx=0x7f549002a480 comp rx=0 tx=0).stop 2024-09-21T07:54:59.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.887+0000 7f54adc00640 1 -- 172.21.15.39:0/3928674296 shutdown_connections 2024-09-21T07:54:59.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.887+0000 7f54adc00640 1 --2- 172.21.15.39:0/3928674296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54a815d830 0x7f54a8156bb0 unknown :-1 s=CLOSED pgs=2876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:59.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.887+0000 7f54adc00640 1 --2- 172.21.15.39:0/3928674296 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f54a80b30a0 0x7f54a80cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:59.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.887+0000 7f54adc00640 1 -- 172.21.15.39:0/3928674296 >> 172.21.15.39:0/3928674296 conn(0x7f54a80b2460 msgr2=0x7f54a806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:59.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.887+0000 7f54adc00640 1 -- 172.21.15.39:0/3928674296 shutdown_connections 2024-09-21T07:54:59.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.888+0000 7f54adc00640 1 -- 172.21.15.39:0/3928674296 wait complete. 2024-09-21T07:54:59.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.890+0000 7f54adc00640 1 Processor -- start 2024-09-21T07:54:59.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.890+0000 7f54adc00640 1 -- start start 2024-09-21T07:54:59.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.891+0000 7f54adc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54a80b30a0 0x7f54a80c6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:59.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.891+0000 7f54adc00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f54a80ccc60 0x7f54a8155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:59.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.891+0000 7f54adc00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54a815d830 0x7f54a80c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:59.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.891+0000 7f54adc00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f54a80af450 con 0x7f54a80b30a0 2024-09-21T07:54:59.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.891+0000 7f54adc00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f54a80af2d0 con 0x7f54a815d830 2024-09-21T07:54:59.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.891+0000 7f54adc00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f54a80af5d0 con 0x7f54a80ccc60 2024-09-21T07:54:59.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.892+0000 7f54a6a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54a80b30a0 0x7f54a80c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:59.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.892+0000 7f54a6a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54a80b30a0 0x7f54a80c6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51818/0 (socket says 172.21.15.39:51818) 2024-09-21T07:54:59.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.892+0000 7f54a6a00640 1 -- 172.21.15.39:0/2541724632 learned_addr learned my addr 172.21.15.39:0/2541724632 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:54:59.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.892+0000 7f549fe00640 1 --2- 172.21.15.39:0/2541724632 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f54a80ccc60 0x7f54a8155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:59.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.892+0000 7f54a7400640 1 --2- 172.21.15.39:0/2541724632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54a815d830 0x7f54a80c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:59.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.892+0000 7f54a6a00640 1 -- 172.21.15.39:0/2541724632 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f54a80ccc60 msgr2=0x7f54a8155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:59.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.892+0000 7f54a6a00640 1 --2- 172.21.15.39:0/2541724632 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f54a80ccc60 0x7f54a8155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:59.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.892+0000 7f54a6a00640 1 -- 172.21.15.39:0/2541724632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54a815d830 msgr2=0x7f54a80c8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:59.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.892+0000 7f54a6a00640 1 --2- 172.21.15.39:0/2541724632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54a815d830 0x7f54a80c8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:59.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.892+0000 7f54a6a00640 1 -- 172.21.15.39:0/2541724632 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a81560e0 con 0x7f54a80b30a0 2024-09-21T07:54:59.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.892+0000 7f54a7400640 1 --2- 172.21.15.39:0/2541724632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54a815d830 0x7f54a80c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:54:59.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.893+0000 7f54a6a00640 1 --2- 172.21.15.39:0/2541724632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54a80b30a0 0x7f54a80c6960 secure :-1 s=READY pgs=2877 cs=0 l=1 rev1=1 crypto rx=0x7f549800d750 tx=0x7f549800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:59.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.893+0000 7f549ea00640 1 -- 172.21.15.39:0/2541724632 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5498017070 con 0x7f54a80b30a0 2024-09-21T07:54:59.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.893+0000 7f549ea00640 1 -- 172.21.15.39:0/2541724632 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5498004310 con 0x7f54a80b30a0 2024-09-21T07:54:59.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.893+0000 7f549ea00640 1 -- 172.21.15.39:0/2541724632 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549800de60 con 0x7f54a80b30a0 2024-09-21T07:54:59.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.893+0000 7f54adc00640 1 -- 172.21.15.39:0/2541724632 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54a80c76a0 con 0x7f54a80b30a0 2024-09-21T07:54:59.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.893+0000 7f54adc00640 1 -- 172.21.15.39:0/2541724632 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54a80c7c60 con 0x7f54a80b30a0 2024-09-21T07:54:59.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.894+0000 7f54adc00640 1 -- 172.21.15.39:0/2541724632 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54a80afa30 con 0x7f54a80b30a0 2024-09-21T07:54:59.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.895+0000 7f549ea00640 1 -- 172.21.15.39:0/2541724632 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5498009c30 con 0x7f54a80b30a0 2024-09-21T07:54:59.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.896+0000 7f549ea00640 1 --2- 172.21.15.39:0/2541724632 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5484078b90 0x7f548407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:59.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.896+0000 7f549ea00640 1 -- 172.21.15.39:0/2541724632 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(269..269 src has 1..269) ==== 6278+0+0 (secure 0 0 0) 0x7f549809b2f0 con 0x7f54a80b30a0 2024-09-21T07:54:59.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.897+0000 7f549fe00640 1 --2- 172.21.15.39:0/2541724632 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5484078b90 0x7f548407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:59.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.898+0000 7f549fe00640 1 --2- 172.21.15.39:0/2541724632 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5484078b90 0x7f548407b0a0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f5490034000 tx=0x7f5490004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:59.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:54:59.901+0000 7f549ea00640 1 -- 172.21.15.39:0/2541724632 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5498020020 con 0x7f54a80b30a0 2024-09-21T07:55:00.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.102+0000 7f54adc00640 1 -- 172.21.15.39:0/2541724632 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f54a806ce80 con 0x7f54a80b30a0 2024-09-21T07:55:00.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.108+0000 7f549ea00640 1 -- 172.21.15.39:0/2541724632 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v58) ==== 71+0+0 (secure 0 0 0) 0x7f5498063570 con 0x7f54a80b30a0 2024-09-21T07:55:00.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.110+0000 7f54adc00640 1 -- 172.21.15.39:0/2541724632 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5484078b90 msgr2=0x7f548407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:00.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.110+0000 7f54adc00640 1 --2- 172.21.15.39:0/2541724632 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5484078b90 0x7f548407b0a0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f5490034000 tx=0x7f5490004730 comp rx=0 tx=0).stop 2024-09-21T07:55:00.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.110+0000 7f54adc00640 1 -- 172.21.15.39:0/2541724632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54a80b30a0 msgr2=0x7f54a80c6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:00.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.110+0000 7f54adc00640 1 --2- 172.21.15.39:0/2541724632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54a80b30a0 0x7f54a80c6960 secure :-1 s=READY pgs=2877 cs=0 l=1 rev1=1 crypto rx=0x7f549800d750 tx=0x7f549800dc10 comp rx=0 tx=0).stop 2024-09-21T07:55:00.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.110+0000 7f54adc00640 1 -- 172.21.15.39:0/2541724632 shutdown_connections 2024-09-21T07:55:00.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.110+0000 7f54adc00640 1 --2- 172.21.15.39:0/2541724632 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f5484078b90 0x7f548407b0a0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:00.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.110+0000 7f54adc00640 1 --2- 172.21.15.39:0/2541724632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f54a815d830 0x7f54a80c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:00.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.110+0000 7f54adc00640 1 --2- 172.21.15.39:0/2541724632 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f54a80ccc60 0x7f54a8155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:00.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.110+0000 7f54adc00640 1 --2- 172.21.15.39:0/2541724632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54a80b30a0 0x7f54a80c6960 unknown :-1 s=CLOSED pgs=2877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:00.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.110+0000 7f54adc00640 1 -- 172.21.15.39:0/2541724632 >> 172.21.15.39:0/2541724632 conn(0x7f54a80b2460 msgr2=0x7f54a80adbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:00.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.111+0000 7f54adc00640 1 -- 172.21.15.39:0/2541724632 shutdown_connections 2024-09-21T07:55:00.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.111+0000 7f54adc00640 1 -- 172.21.15.39:0/2541724632 wait complete. 2024-09-21T07:55:00.139 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_snapshot_check_access' 2024-09-21T07:55:00.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.371+0000 7f6c3b200640 1 -- 172.21.15.39:0/791671356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c20004800 msgr2=0x7f6c20004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:00.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.371+0000 7f6c3b200640 1 --2- 172.21.15.39:0/791671356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c20004800 0x7f6c20004c10 secure :-1 s=READY pgs=2879 cs=0 l=1 rev1=1 crypto rx=0x7f6c24009b50 tx=0x7f6c20005830 comp rx=0 tx=0).stop 2024-09-21T07:55:00.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.371+0000 7f6c3a800640 1 -- 172.21.15.39:0/791671356 reap_dead start 2024-09-21T07:55:00.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.371+0000 7f6c3b200640 1 -- 172.21.15.39:0/791671356 shutdown_connections 2024-09-21T07:55:00.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.371+0000 7f6c3b200640 1 -- 172.21.15.39:0/791671356 >> 172.21.15.39:0/791671356 conn(0x7f6c3c06cfc0 msgr2=0x7f6c3c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:00.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.371+0000 7f6c3b200640 1 -- 172.21.15.39:0/791671356 shutdown_connections 2024-09-21T07:55:00.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.371+0000 7f6c3b200640 1 -- 172.21.15.39:0/791671356 wait complete. 2024-09-21T07:55:00.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.374+0000 7f6c3b200640 1 Processor -- start 2024-09-21T07:55:00.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.374+0000 7f6c3b200640 1 -- start start 2024-09-21T07:55:00.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.375+0000 7f6c3b200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c20004800 0x7f6c3c13ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:00.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.375+0000 7f6c3b200640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6c3c0aa5f0 0x7f6c3c0aaa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:00.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.375+0000 7f6c3b200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c3c0aaf40 0x7f6c3c0adba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:00.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.375+0000 7f6c3b200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c15d3a0 con 0x7f6c3c0aaf40 2024-09-21T07:55:00.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.375+0000 7f6c3b200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c15d220 con 0x7f6c20004800 2024-09-21T07:55:00.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.375+0000 7f6c3b200640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f6c3c06ecf0 con 0x7f6c3c0aa5f0 2024-09-21T07:55:00.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.375+0000 7f6c3a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c3c0aaf40 0x7f6c3c0adba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:00.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.375+0000 7f6c3a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c3c0aaf40 0x7f6c3c0adba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51842/0 (socket says 172.21.15.39:51842) 2024-09-21T07:55:00.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.375+0000 7f6c3a800640 1 -- 172.21.15.39:0/1623804486 learned_addr learned my addr 172.21.15.39:0/1623804486 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:55:00.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.376+0000 7f6c39e00640 1 --2- 172.21.15.39:0/1623804486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c20004800 0x7f6c3c13ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:00.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.376+0000 7f6c3a800640 1 -- 172.21.15.39:0/1623804486 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6c3c0aa5f0 msgr2=0x7f6c3c0aaa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:00.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.376+0000 7f6c3a800640 1 --2- 172.21.15.39:0/1623804486 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6c3c0aa5f0 0x7f6c3c0aaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:00.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.376+0000 7f6c3a800640 1 -- 172.21.15.39:0/1623804486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c20004800 msgr2=0x7f6c3c13ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:00.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.376+0000 7f6c3a800640 1 --2- 172.21.15.39:0/1623804486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c20004800 0x7f6c3c13ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:00.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.376+0000 7f6c3a800640 1 -- 172.21.15.39:0/1623804486 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c3c15c1d0 con 0x7f6c3c0aaf40 2024-09-21T07:55:00.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.376+0000 7f6c3a800640 1 --2- 172.21.15.39:0/1623804486 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c3c0aaf40 0x7f6c3c0adba0 secure :-1 s=READY pgs=2880 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c15d550 tx=0x7f6c3400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:00.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.376+0000 7f6c32000640 1 -- 172.21.15.39:0/1623804486 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c34017070 con 0x7f6c3c0aaf40 2024-09-21T07:55:00.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.376+0000 7f6c32000640 1 -- 172.21.15.39:0/1623804486 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c3400ad10 con 0x7f6c3c0aaf40 2024-09-21T07:55:00.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.377+0000 7f6c32000640 1 -- 172.21.15.39:0/1623804486 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c340072d0 con 0x7f6c3c0aaf40 2024-09-21T07:55:00.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.377+0000 7f6c3b200640 1 -- 172.21.15.39:0/1623804486 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c3c0ace20 con 0x7f6c3c0aaf40 2024-09-21T07:55:00.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.377+0000 7f6c3b200640 1 -- 172.21.15.39:0/1623804486 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c3c0ad0c0 con 0x7f6c3c0aaf40 2024-09-21T07:55:00.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.378+0000 7f6c3b200640 1 -- 172.21.15.39:0/1623804486 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c3c0c5400 con 0x7f6c3c0aaf40 2024-09-21T07:55:00.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.379+0000 7f6c32000640 1 -- 172.21.15.39:0/1623804486 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6c3401a070 con 0x7f6c3c0aaf40 2024-09-21T07:55:00.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.380+0000 7f6c32000640 1 --2- 172.21.15.39:0/1623804486 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6c18078bc0 0x7f6c1807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:00.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.380+0000 7f6c32000640 1 -- 172.21.15.39:0/1623804486 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(269..269 src has 1..269) ==== 6278+0+0 (secure 0 0 0) 0x7f6c3409ac70 con 0x7f6c3c0aaf40 2024-09-21T07:55:00.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.380+0000 7f6c39e00640 1 --2- 172.21.15.39:0/1623804486 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6c18078bc0 0x7f6c1807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:00.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.381+0000 7f6c39e00640 1 --2- 172.21.15.39:0/1623804486 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6c18078bc0 0x7f6c1807b0d0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c000c00 tx=0x7f6c2c006b70 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:00.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.385+0000 7f6c32000640 1 -- 172.21.15.39:0/1623804486 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6c3c15c1d0 con 0x7f6c3c0aaf40 2024-09-21T07:55:00.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:00.657+0000 7f6c3b200640 1 -- 172.21.15.39:0/1623804486 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_snapshot_check_access"]} v 0) -- 0x7f6c3c06f250 con 0x7f6c3c0aaf40 2024-09-21T07:55:01.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:01.635+0000 7f6c32000640 1 -- 172.21.15.39:0/1623804486 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_snapshot_check_access"]}]=0 v1059) ==== 147+0+0 (secure 0 0 0) 0x7f6c34062e60 con 0x7f6c3c0aaf40 2024-09-21T07:55:01.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:01.638+0000 7f6c3b200640 1 -- 172.21.15.39:0/1623804486 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6c18078bc0 msgr2=0x7f6c1807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:01.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:01.638+0000 7f6c3b200640 1 --2- 172.21.15.39:0/1623804486 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6c18078bc0 0x7f6c1807b0d0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c000c00 tx=0x7f6c2c006b70 comp rx=0 tx=0).stop 2024-09-21T07:55:01.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:01.638+0000 7f6c3b200640 1 -- 172.21.15.39:0/1623804486 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c3c0aaf40 msgr2=0x7f6c3c0adba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:01.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:01.638+0000 7f6c3b200640 1 --2- 172.21.15.39:0/1623804486 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c3c0aaf40 0x7f6c3c0adba0 secure :-1 s=READY pgs=2880 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c15d550 tx=0x7f6c3400cf70 comp rx=0 tx=0).stop 2024-09-21T07:55:01.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:01.638+0000 7f6c3b200640 1 -- 172.21.15.39:0/1623804486 shutdown_connections 2024-09-21T07:55:01.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:01.639+0000 7f6c3b200640 1 --2- 172.21.15.39:0/1623804486 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f6c18078bc0 0x7f6c1807b0d0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:01.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:01.639+0000 7f6c3b200640 1 --2- 172.21.15.39:0/1623804486 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c3c0aaf40 0x7f6c3c0adba0 unknown :-1 s=CLOSED pgs=2880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:01.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:01.639+0000 7f6c3b200640 1 --2- 172.21.15.39:0/1623804486 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6c3c0aa5f0 0x7f6c3c0aaa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:01.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:01.639+0000 7f6c3b200640 1 --2- 172.21.15.39:0/1623804486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c20004800 0x7f6c3c13ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:01.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:01.639+0000 7f6c3b200640 1 -- 172.21.15.39:0/1623804486 >> 172.21.15.39:0/1623804486 conn(0x7f6c3c06cfc0 msgr2=0x7f6c3c0a94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:01.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:01.639+0000 7f6c3b200640 1 -- 172.21.15.39:0/1623804486 shutdown_connections 2024-09-21T07:55:01.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:01.639+0000 7f6c3b200640 1 -- 172.21.15.39:0/1623804486 wait complete. 2024-09-21T07:55:01.671 INFO:tasks.cephfs_test_runner:test_snapshot_check_access (tasks.cephfs.test_snapshots.TestSnapshots) ... ok 2024-09-21T07:55:01.671 INFO:tasks.cephfs_test_runner: 2024-09-21T07:55:01.672 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-21T07:55:01.672 INFO:tasks.cephfs_test_runner:Ran 13 tests in 1276.213s 2024-09-21T07:55:01.672 INFO:tasks.cephfs_test_runner: 2024-09-21T07:55:01.672 INFO:tasks.cephfs_test_runner:OK 2024-09-21T07:55:01.672 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-21T07:55:01.682 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-21T07:55:01.690 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-21T07:55:01.690 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:55:01.690 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:55:01.728 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:55:01.728 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-21T07:55:01.728 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:55:01.804 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052632,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497980k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:55:01.804 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:55:01.805 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:55:01.845 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:55:01.845 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-21T07:55:01.845 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:55:01.923 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:55:01.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:01.923 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-09-21T07:55:01.960 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:55:01.961 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T07:55:01.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:01.961 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:55:02.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:02.060 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:55:02.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:02.334 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link delete ceph-brx 2024-09-21T07:55:02.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:02.631 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-09-21T07:55:02.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:02.668 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link delete ceph-brx 2024-09-21T07:55:02.767 INFO:teuthology.orchestra.run.smithi039.stderr:Cannot find device "ceph-brx" 2024-09-21T07:55:02.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:55:02.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:02.770 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-09-21T07:55:02.811 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T07:55:02.811 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T07:55:02.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:02.812 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T07:55:02.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:02.915 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T07:55:03.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:03.194 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link delete ceph-brx 2024-09-21T07:55:03.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:03.473 DEBUG:teuthology.orchestra.run.smithi148:> ip netns list 2024-09-21T07:55:03.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:03.508 DEBUG:teuthology.orchestra.run.smithi148:> sudo ip link delete ceph-brx 2024-09-21T07:55:03.608 INFO:teuthology.orchestra.run.smithi148.stderr:Cannot find device "ceph-brx" 2024-09-21T07:55:03.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:55:03.611 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-21T07:55:03.620 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T07:55:03.620 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:55:03.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:03.858+0000 7fd41d800640 1 -- 172.21.15.39:0/1059167514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4180b5240 msgr2=0x7fd41815c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:03.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:03.858+0000 7fd41d800640 1 --2- 172.21.15.39:0/1059167514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4180b5240 0x7fd41815c690 secure :-1 s=READY pgs=2882 cs=0 l=1 rev1=1 crypto rx=0x7fd4180b0000 tx=0x7fd40802a4a0 comp rx=0 tx=0).stop 2024-09-21T07:55:03.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:03.858+0000 7fd41d800640 1 -- 172.21.15.39:0/1059167514 shutdown_connections 2024-09-21T07:55:03.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:03.858+0000 7fd41d800640 1 --2- 172.21.15.39:0/1059167514 >> v2:172.21.15.148:3300/0 conn(0x7fd4180d0d40 0x7fd4180b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:03.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:03.858+0000 7fd41d800640 1 --2- 172.21.15.39:0/1059167514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4180b5240 0x7fd41815c690 unknown :-1 s=CLOSED pgs=2882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:03.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:03.858+0000 7fd41d800640 1 -- 172.21.15.39:0/1059167514 >> 172.21.15.39:0/1059167514 conn(0x7fd41806cfc0 msgr2=0x7fd41806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:03.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:03.858+0000 7fd41d800640 1 -- 172.21.15.39:0/1059167514 shutdown_connections 2024-09-21T07:55:03.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:03.858+0000 7fd41d800640 1 -- 172.21.15.39:0/1059167514 wait complete. 2024-09-21T07:55:03.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:03.861+0000 7fd41d800640 1 Processor -- start 2024-09-21T07:55:03.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:03.861+0000 7fd41d800640 1 -- start start 2024-09-21T07:55:03.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:03.862+0000 7fd41d800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd4180b5240 0x7fd41814fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:03.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:03.862+0000 7fd41d800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4180cf920 0x7fd418151060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:03.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:03.862+0000 7fd41d800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd41814ed10 0x7fd4181515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:03.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:03.862+0000 7fd41d800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd418070100 con 0x7fd4180cf920 2024-09-21T07:55:03.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:03.862+0000 7fd41d800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd4180cff40 con 0x7fd41814ed10 2024-09-21T07:55:03.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:03.862+0000 7fd41d800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd4180b4910 con 0x7fd4180b5240 2024-09-21T07:55:03.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:03.862+0000 7fd40fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4180cf920 0x7fd418151060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:03.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:03.863+0000 7fd40fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4180cf920 0x7fd418151060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42368/0 (socket says 172.21.15.39:42368) 2024-09-21T07:55:03.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:03.863+0000 7fd416a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd4180b5240 0x7fd41814fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:03.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:03.863+0000 7fd40fe00640 1 -- 172.21.15.39:0/3285769079 learned_addr learned my addr 172.21.15.39:0/3285769079 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:55:03.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:03.863+0000 7fd417400640 1 --2- 172.21.15.39:0/3285769079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd41814ed10 0x7fd4181515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:03.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:03.863+0000 7fd40fe00640 1 -- 172.21.15.39:0/3285769079 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd4180b5240 msgr2=0x7fd41814fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:03.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:03.863+0000 7fd40fe00640 1 --2- 172.21.15.39:0/3285769079 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd4180b5240 0x7fd41814fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:03.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:03.863+0000 7fd40fe00640 1 -- 172.21.15.39:0/3285769079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd41814ed10 msgr2=0x7fd4181515a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:03.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:03.863+0000 7fd40fe00640 1 --2- 172.21.15.39:0/3285769079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd41814ed10 0x7fd4181515a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:03.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:03.863+0000 7fd40fe00640 1 -- 172.21.15.39:0/3285769079 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4181f3490 con 0x7fd4180cf920 2024-09-21T07:55:03.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:03.863+0000 7fd40fe00640 1 --2- 172.21.15.39:0/3285769079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4180cf920 0x7fd418151060 secure :-1 s=READY pgs=2883 cs=0 l=1 rev1=1 crypto rx=0x7fd40000c3a0 tx=0x7fd40000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:03.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:03.863+0000 7fd40ea00640 1 -- 172.21.15.39:0/3285769079 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd400011790 con 0x7fd4180cf920 2024-09-21T07:55:03.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:03.864+0000 7fd40ea00640 1 -- 172.21.15.39:0/3285769079 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd400017070 con 0x7fd4180cf920 2024-09-21T07:55:03.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:03.864+0000 7fd40ea00640 1 -- 172.21.15.39:0/3285769079 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd400011ce0 con 0x7fd4180cf920 2024-09-21T07:55:03.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:03.864+0000 7fd41d800640 1 -- 172.21.15.39:0/3285769079 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4181f3770 con 0x7fd4180cf920 2024-09-21T07:55:03.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:03.864+0000 7fd41d800640 1 -- 172.21.15.39:0/3285769079 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4181f3d30 con 0x7fd4180cf920 2024-09-21T07:55:03.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:03.865+0000 7fd41d800640 1 -- 172.21.15.39:0/3285769079 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4180b4e30 con 0x7fd4180cf920 2024-09-21T07:55:03.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:03.866+0000 7fd40ea00640 1 -- 172.21.15.39:0/3285769079 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd40001c460 con 0x7fd4180cf920 2024-09-21T07:55:03.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:03.867+0000 7fd40ea00640 1 --2- 172.21.15.39:0/3285769079 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd3f4078b90 0x7fd3f407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:03.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:03.867+0000 7fd40ea00640 1 -- 172.21.15.39:0/3285769079 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(269..269 src has 1..269) ==== 6278+0+0 (secure 0 0 0) 0x7fd40009abf0 con 0x7fd4180cf920 2024-09-21T07:55:03.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:03.867+0000 7fd416a00640 1 --2- 172.21.15.39:0/3285769079 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd3f4078b90 0x7fd3f407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:03.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:03.868+0000 7fd416a00640 1 --2- 172.21.15.39:0/3285769079 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd3f4078b90 0x7fd3f407b0a0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fd408034000 tx=0x7fd408004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:03.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:03.871+0000 7fd40ea00640 1 -- 172.21.15.39:0/3285769079 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd400062820 con 0x7fd4180cf920 2024-09-21T07:55:04.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.068+0000 7fd41d800640 1 -- 172.21.15.39:0/3285769079 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd4180b3740 con 0x7fd3f4078b90 2024-09-21T07:55:04.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.073+0000 7fd40ea00640 1 -- 172.21.15.39:0/3285769079 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+256262 (secure 0 0 0) 0x7fd4180b3740 con 0x7fd3f4078b90 2024-09-21T07:55:04.078 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:55:04.079 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:55:04.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.077+0000 7fd41d800640 1 -- 172.21.15.39:0/3285769079 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd3f4078b90 msgr2=0x7fd3f407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:04.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.077+0000 7fd41d800640 1 --2- 172.21.15.39:0/3285769079 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd3f4078b90 0x7fd3f407b0a0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fd408034000 tx=0x7fd408004730 comp rx=0 tx=0).stop 2024-09-21T07:55:04.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.078+0000 7fd41d800640 1 -- 172.21.15.39:0/3285769079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4180cf920 msgr2=0x7fd418151060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:04.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.078+0000 7fd41d800640 1 --2- 172.21.15.39:0/3285769079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4180cf920 0x7fd418151060 secure :-1 s=READY pgs=2883 cs=0 l=1 rev1=1 crypto rx=0x7fd40000c3a0 tx=0x7fd40000c860 comp rx=0 tx=0).stop 2024-09-21T07:55:04.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.078+0000 7fd41d800640 1 -- 172.21.15.39:0/3285769079 shutdown_connections 2024-09-21T07:55:04.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.078+0000 7fd41d800640 1 --2- 172.21.15.39:0/3285769079 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd3f4078b90 0x7fd3f407b0a0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:04.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.078+0000 7fd41d800640 1 --2- 172.21.15.39:0/3285769079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd41814ed10 0x7fd4181515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:04.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.078+0000 7fd41d800640 1 --2- 172.21.15.39:0/3285769079 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4180cf920 0x7fd418151060 unknown :-1 s=CLOSED pgs=2883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:04.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.078+0000 7fd41d800640 1 --2- 172.21.15.39:0/3285769079 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd4180b5240 0x7fd41814fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:04.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.078+0000 7fd41d800640 1 -- 172.21.15.39:0/3285769079 >> 172.21.15.39:0/3285769079 conn(0x7fd41806cfc0 msgr2=0x7fd4181e8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:04.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.078+0000 7fd41d800640 1 -- 172.21.15.39:0/3285769079 shutdown_connections 2024-09-21T07:55:04.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.078+0000 7fd41d800640 1 -- 172.21.15.39:0/3285769079 wait complete. 2024-09-21T07:55:04.105 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":925,"stamp":"2024-09-21T07:55:03.986673+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":211,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":466044,"kb_used_data":27720,"kb_used_omap":12,"kb_used_meta":438131,"kb_avail":754508676,"statfs":{"total":773094113280,"available":772616884224,"internally_reserved":0,"allocated":28385280,"data_stored":15991320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":448646704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12736,"num_objects":-25,"num_object_clones":-1,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":-1,"num_read":-1,"num_read_kb":-1,"num_write":-60,"num_write_kb":-31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.383374"},"pg_stats":[{"pgid":"12.d","version":"0'0","reported_seq":403,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.315418+0000","last_change":"2024-09-21T07:35:27.879605+0000","last_active":"2024-09-21T07:55:00.315418+0000","last_peered":"2024-09-21T07:55:00.315418+0000","last_clean":"2024-09-21T07:55:00.315418+0000","last_became_active":"2024-09-21T07:35:27.879274+0000","last_became_peered":"2024-09-21T07:35:27.879274+0000","last_unstale":"2024-09-21T07:55:00.315418+0000","last_undegraded":"2024-09-21T07:55:00.315418+0000","last_fullsized":"2024-09-21T07:55:00.315418+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:24.241636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":413,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.312401+0000","last_change":"2024-09-21T07:35:29.491336+0000","last_active":"2024-09-21T07:55:00.312401+0000","last_peered":"2024-09-21T07:55:00.312401+0000","last_clean":"2024-09-21T07:55:00.312401+0000","last_became_active":"2024-09-21T07:35:26.866484+0000","last_became_peered":"2024-09-21T07:35:26.866484+0000","last_unstale":"2024-09-21T07:55:00.312401+0000","last_undegraded":"2024-09-21T07:55:00.312401+0000","last_fullsized":"2024-09-21T07:55:00.312401+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:10.449665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076010100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":589,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.311307+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:55:00.311307+0000","last_peered":"2024-09-21T07:55:00.311307+0000","last_clean":"2024-09-21T07:55:00.311307+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:55:00.311307+0000","last_undegraded":"2024-09-21T07:55:00.311307+0000","last_fullsized":"2024-09-21T07:55:00.311307+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":480,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.632658+0000","last_change":"2024-09-21T07:34:24.539248+0000","last_active":"2024-09-21T07:54:59.632658+0000","last_peered":"2024-09-21T07:54:59.632658+0000","last_clean":"2024-09-21T07:54:59.632658+0000","last_became_active":"2024-09-21T07:34:23.273465+0000","last_became_peered":"2024-09-21T07:34:23.273465+0000","last_unstale":"2024-09-21T07:54:59.632658+0000","last_undegraded":"2024-09-21T07:54:59.632658+0000","last_fullsized":"2024-09-21T07:54:59.632658+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:55.517452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060627700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":407,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.312347+0000","last_change":"2024-09-21T07:35:27.864492+0000","last_active":"2024-09-21T07:55:00.312347+0000","last_peered":"2024-09-21T07:55:00.312347+0000","last_clean":"2024-09-21T07:55:00.312347+0000","last_became_active":"2024-09-21T07:35:27.864176+0000","last_became_peered":"2024-09-21T07:35:27.864176+0000","last_unstale":"2024-09-21T07:55:00.312347+0000","last_undegraded":"2024-09-21T07:55:00.312347+0000","last_fullsized":"2024-09-21T07:55:00.312347+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:05.153467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":410,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.314353+0000","last_change":"2024-09-21T07:35:29.493678+0000","last_active":"2024-09-21T07:55:00.314353+0000","last_peered":"2024-09-21T07:55:00.314353+0000","last_clean":"2024-09-21T07:55:00.314353+0000","last_became_active":"2024-09-21T07:35:26.870749+0000","last_became_peered":"2024-09-21T07:35:26.870749+0000","last_unstale":"2024-09-21T07:55:00.314353+0000","last_undegraded":"2024-09-21T07:55:00.314353+0000","last_fullsized":"2024-09-21T07:55:00.314353+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:55.367877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010159570000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":456,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.175411+0000","last_change":"2024-09-21T07:34:24.597274+0000","last_active":"2024-09-21T07:55:00.175411+0000","last_peered":"2024-09-21T07:55:00.175411+0000","last_clean":"2024-09-21T07:55:00.175411+0000","last_became_active":"2024-09-21T07:34:23.279560+0000","last_became_peered":"2024-09-21T07:34:23.279560+0000","last_unstale":"2024-09-21T07:55:00.175411+0000","last_undegraded":"2024-09-21T07:55:00.175411+0000","last_fullsized":"2024-09-21T07:55:00.175411+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:15.845265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059699799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":407,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.631402+0000","last_change":"2024-09-21T07:35:27.874387+0000","last_active":"2024-09-21T07:54:59.631402+0000","last_peered":"2024-09-21T07:54:59.631402+0000","last_clean":"2024-09-21T07:54:59.631402+0000","last_became_active":"2024-09-21T07:35:27.873277+0000","last_became_peered":"2024-09-21T07:35:27.873277+0000","last_unstale":"2024-09-21T07:54:59.631402+0000","last_undegraded":"2024-09-21T07:54:59.631402+0000","last_fullsized":"2024-09-21T07:54:59.631402+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:04.581463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":406,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.175552+0000","last_change":"2024-09-21T07:35:29.387094+0000","last_active":"2024-09-21T07:55:00.175552+0000","last_peered":"2024-09-21T07:55:00.175552+0000","last_clean":"2024-09-21T07:55:00.175552+0000","last_became_active":"2024-09-21T07:35:26.874757+0000","last_became_peered":"2024-09-21T07:35:26.874757+0000","last_unstale":"2024-09-21T07:55:00.175552+0000","last_undegraded":"2024-09-21T07:55:00.175552+0000","last_fullsized":"2024-09-21T07:55:00.175552+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:33.542750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061065700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":463,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.313869+0000","last_change":"2024-09-21T07:34:24.259906+0000","last_active":"2024-09-21T07:55:00.313869+0000","last_peered":"2024-09-21T07:55:00.313869+0000","last_clean":"2024-09-21T07:55:00.313869+0000","last_became_active":"2024-09-21T07:34:23.635282+0000","last_became_peered":"2024-09-21T07:34:23.635282+0000","last_unstale":"2024-09-21T07:55:00.313869+0000","last_undegraded":"2024-09-21T07:55:00.313869+0000","last_fullsized":"2024-09-21T07:55:00.313869+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:00.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001527664,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":407,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.634628+0000","last_change":"2024-09-21T07:35:27.873645+0000","last_active":"2024-09-21T07:54:59.634628+0000","last_peered":"2024-09-21T07:54:59.634628+0000","last_clean":"2024-09-21T07:54:59.634628+0000","last_became_active":"2024-09-21T07:35:27.873245+0000","last_became_peered":"2024-09-21T07:35:27.873245+0000","last_unstale":"2024-09-21T07:54:59.634628+0000","last_undegraded":"2024-09-21T07:54:59.634628+0000","last_fullsized":"2024-09-21T07:54:59.634628+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:55.090953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":413,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.633609+0000","last_change":"2024-09-21T07:35:28.861744+0000","last_active":"2024-09-21T07:54:59.633609+0000","last_peered":"2024-09-21T07:54:59.633609+0000","last_clean":"2024-09-21T07:54:59.633609+0000","last_became_active":"2024-09-21T07:35:26.876462+0000","last_became_peered":"2024-09-21T07:35:26.876462+0000","last_unstale":"2024-09-21T07:54:59.633609+0000","last_undegraded":"2024-09-21T07:54:59.633609+0000","last_fullsized":"2024-09-21T07:54:59.633609+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:48.632568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012020920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":464,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.630668+0000","last_change":"2024-09-21T07:34:25.085454+0000","last_active":"2024-09-21T07:54:59.630668+0000","last_peered":"2024-09-21T07:54:59.630668+0000","last_clean":"2024-09-21T07:54:59.630668+0000","last_became_active":"2024-09-21T07:34:23.279205+0000","last_became_peered":"2024-09-21T07:34:23.279205+0000","last_unstale":"2024-09-21T07:54:59.630668+0000","last_undegraded":"2024-09-21T07:54:59.630668+0000","last_fullsized":"2024-09-21T07:54:59.630668+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:54.231666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095320599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":405,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.314238+0000","last_change":"2024-09-21T07:35:27.861373+0000","last_active":"2024-09-21T07:55:00.314238+0000","last_peered":"2024-09-21T07:55:00.314238+0000","last_clean":"2024-09-21T07:55:00.314238+0000","last_became_active":"2024-09-21T07:35:27.861016+0000","last_became_peered":"2024-09-21T07:35:27.861016+0000","last_unstale":"2024-09-21T07:55:00.314238+0000","last_undegraded":"2024-09-21T07:55:00.314238+0000","last_fullsized":"2024-09-21T07:55:00.314238+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:56.043763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":413,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.311346+0000","last_change":"2024-09-21T07:35:29.053677+0000","last_active":"2024-09-21T07:55:00.311346+0000","last_peered":"2024-09-21T07:55:00.311346+0000","last_clean":"2024-09-21T07:55:00.311346+0000","last_became_active":"2024-09-21T07:35:26.875496+0000","last_became_peered":"2024-09-21T07:35:26.875496+0000","last_unstale":"2024-09-21T07:55:00.311346+0000","last_undegraded":"2024-09-21T07:55:00.311346+0000","last_fullsized":"2024-09-21T07:55:00.311346+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:04.850784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065859000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":465,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.310605+0000","last_change":"2024-09-21T07:34:25.028521+0000","last_active":"2024-09-21T07:55:00.310605+0000","last_peered":"2024-09-21T07:55:00.310605+0000","last_clean":"2024-09-21T07:55:00.310605+0000","last_became_active":"2024-09-21T07:34:23.635467+0000","last_became_peered":"2024-09-21T07:34:23.635467+0000","last_unstale":"2024-09-21T07:55:00.310605+0000","last_undegraded":"2024-09-21T07:55:00.310605+0000","last_fullsized":"2024-09-21T07:55:00.310605+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:33.729123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000660558,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":405,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.633153+0000","last_change":"2024-09-21T07:35:27.871201+0000","last_active":"2024-09-21T07:54:59.633153+0000","last_peered":"2024-09-21T07:54:59.633153+0000","last_clean":"2024-09-21T07:54:59.633153+0000","last_became_active":"2024-09-21T07:35:27.870816+0000","last_became_peered":"2024-09-21T07:35:27.870816+0000","last_unstale":"2024-09-21T07:54:59.633153+0000","last_undegraded":"2024-09-21T07:54:59.633153+0000","last_fullsized":"2024-09-21T07:54:59.633153+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:04.787450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":412,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.314136+0000","last_change":"2024-09-21T07:35:29.491874+0000","last_active":"2024-09-21T07:55:00.314136+0000","last_peered":"2024-09-21T07:55:00.314136+0000","last_clean":"2024-09-21T07:55:00.314136+0000","last_became_active":"2024-09-21T07:35:26.866381+0000","last_became_peered":"2024-09-21T07:35:26.866381+0000","last_unstale":"2024-09-21T07:55:00.314136+0000","last_undegraded":"2024-09-21T07:55:00.314136+0000","last_fullsized":"2024-09-21T07:55:00.314136+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:36.404647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073235399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":468,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.313603+0000","last_change":"2024-09-21T07:34:24.260046+0000","last_active":"2024-09-21T07:55:00.313603+0000","last_peered":"2024-09-21T07:55:00.313603+0000","last_clean":"2024-09-21T07:55:00.313603+0000","last_became_active":"2024-09-21T07:34:23.273431+0000","last_became_peered":"2024-09-21T07:34:23.273431+0000","last_unstale":"2024-09-21T07:55:00.313603+0000","last_undegraded":"2024-09-21T07:55:00.313603+0000","last_fullsized":"2024-09-21T07:55:00.313603+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:18.876057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019803310000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":405,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.632132+0000","last_change":"2024-09-21T07:35:27.871736+0000","last_active":"2024-09-21T07:54:59.632132+0000","last_peered":"2024-09-21T07:54:59.632132+0000","last_clean":"2024-09-21T07:54:59.632132+0000","last_became_active":"2024-09-21T07:35:27.871232+0000","last_became_peered":"2024-09-21T07:35:27.871232+0000","last_unstale":"2024-09-21T07:54:59.632132+0000","last_undegraded":"2024-09-21T07:54:59.632132+0000","last_fullsized":"2024-09-21T07:54:59.632132+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:04.419301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":414,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.311567+0000","last_change":"2024-09-21T07:35:29.491246+0000","last_active":"2024-09-21T07:55:00.311567+0000","last_peered":"2024-09-21T07:55:00.311567+0000","last_clean":"2024-09-21T07:55:00.311567+0000","last_became_active":"2024-09-21T07:35:26.866783+0000","last_became_peered":"2024-09-21T07:35:26.866783+0000","last_unstale":"2024-09-21T07:55:00.311567+0000","last_undegraded":"2024-09-21T07:55:00.311567+0000","last_fullsized":"2024-09-21T07:55:00.311567+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:28.881914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098474099999999992,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":465,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.311456+0000","last_change":"2024-09-21T07:34:25.028640+0000","last_active":"2024-09-21T07:55:00.311456+0000","last_peered":"2024-09-21T07:55:00.311456+0000","last_clean":"2024-09-21T07:55:00.311456+0000","last_became_active":"2024-09-21T07:34:23.260950+0000","last_became_peered":"2024-09-21T07:34:23.260950+0000","last_unstale":"2024-09-21T07:55:00.311456+0000","last_undegraded":"2024-09-21T07:55:00.311456+0000","last_fullsized":"2024-09-21T07:55:00.311456+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:23:42.121398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055135099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":408,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.319249+0000","last_change":"2024-09-21T07:35:27.872027+0000","last_active":"2024-09-21T07:55:00.319249+0000","last_peered":"2024-09-21T07:55:00.319249+0000","last_clean":"2024-09-21T07:55:00.319249+0000","last_became_active":"2024-09-21T07:35:27.870426+0000","last_became_peered":"2024-09-21T07:35:27.870426+0000","last_unstale":"2024-09-21T07:55:00.319249+0000","last_undegraded":"2024-09-21T07:55:00.319249+0000","last_fullsized":"2024-09-21T07:55:00.319249+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":414,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.312990+0000","last_change":"2024-09-21T07:35:29.491008+0000","last_active":"2024-09-21T07:55:00.312990+0000","last_peered":"2024-09-21T07:55:00.312990+0000","last_clean":"2024-09-21T07:55:00.312990+0000","last_became_active":"2024-09-21T07:35:26.866750+0000","last_became_peered":"2024-09-21T07:35:26.866750+0000","last_unstale":"2024-09-21T07:55:00.312990+0000","last_undegraded":"2024-09-21T07:55:00.312990+0000","last_fullsized":"2024-09-21T07:55:00.312990+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:25.490406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083529700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":464,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.315758+0000","last_change":"2024-09-21T07:34:24.260665+0000","last_active":"2024-09-21T07:55:00.315758+0000","last_peered":"2024-09-21T07:55:00.315758+0000","last_clean":"2024-09-21T07:55:00.315758+0000","last_became_active":"2024-09-21T07:34:23.635909+0000","last_became_peered":"2024-09-21T07:34:23.635909+0000","last_unstale":"2024-09-21T07:55:00.315758+0000","last_undegraded":"2024-09-21T07:55:00.315758+0000","last_fullsized":"2024-09-21T07:55:00.315758+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:44.536885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":400,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.173356+0000","last_change":"2024-09-21T07:35:27.874772+0000","last_active":"2024-09-21T07:55:00.173356+0000","last_peered":"2024-09-21T07:55:00.173356+0000","last_clean":"2024-09-21T07:55:00.173356+0000","last_became_active":"2024-09-21T07:35:27.874434+0000","last_became_peered":"2024-09-21T07:35:27.874434+0000","last_unstale":"2024-09-21T07:55:00.173356+0000","last_undegraded":"2024-09-21T07:55:00.173356+0000","last_fullsized":"2024-09-21T07:55:00.173356+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:31.998755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":428,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.632463+0000","last_change":"2024-09-21T07:35:28.859153+0000","last_active":"2024-09-21T07:54:59.632463+0000","last_peered":"2024-09-21T07:54:59.632463+0000","last_clean":"2024-09-21T07:54:59.632463+0000","last_became_active":"2024-09-21T07:35:26.874428+0000","last_became_peered":"2024-09-21T07:35:26.874428+0000","last_unstale":"2024-09-21T07:54:59.632463+0000","last_undegraded":"2024-09-21T07:54:59.632463+0000","last_fullsized":"2024-09-21T07:54:59.632463+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:04.487621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048557299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":480,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.630653+0000","last_change":"2024-09-21T07:34:24.538845+0000","last_active":"2024-09-21T07:54:59.630653+0000","last_peered":"2024-09-21T07:54:59.630653+0000","last_clean":"2024-09-21T07:54:59.630653+0000","last_became_active":"2024-09-21T07:34:23.636372+0000","last_became_peered":"2024-09-21T07:34:23.636372+0000","last_unstale":"2024-09-21T07:54:59.630653+0000","last_undegraded":"2024-09-21T07:54:59.630653+0000","last_fullsized":"2024-09-21T07:54:59.630653+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:29.393813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841451,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":409,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.312263+0000","last_change":"2024-09-21T07:35:27.877801+0000","last_active":"2024-09-21T07:55:00.312263+0000","last_peered":"2024-09-21T07:55:00.312263+0000","last_clean":"2024-09-21T07:55:00.312263+0000","last_became_active":"2024-09-21T07:35:27.877464+0000","last_became_peered":"2024-09-21T07:35:27.877464+0000","last_unstale":"2024-09-21T07:55:00.312263+0000","last_undegraded":"2024-09-21T07:55:00.312263+0000","last_fullsized":"2024-09-21T07:55:00.312263+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:08.814929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":413,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.632257+0000","last_change":"2024-09-21T07:35:28.863732+0000","last_active":"2024-09-21T07:54:59.632257+0000","last_peered":"2024-09-21T07:54:59.632257+0000","last_clean":"2024-09-21T07:54:59.632257+0000","last_became_active":"2024-09-21T07:35:26.877392+0000","last_became_peered":"2024-09-21T07:35:26.877392+0000","last_unstale":"2024-09-21T07:54:59.632257+0000","last_undegraded":"2024-09-21T07:54:59.632257+0000","last_fullsized":"2024-09-21T07:54:59.632257+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071425700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":467,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.314344+0000","last_change":"2024-09-21T07:34:24.259686+0000","last_active":"2024-09-21T07:55:00.314344+0000","last_peered":"2024-09-21T07:55:00.314344+0000","last_clean":"2024-09-21T07:55:00.314344+0000","last_became_active":"2024-09-21T07:34:23.274405+0000","last_became_peered":"2024-09-21T07:34:23.274405+0000","last_unstale":"2024-09-21T07:55:00.314344+0000","last_undegraded":"2024-09-21T07:55:00.314344+0000","last_fullsized":"2024-09-21T07:55:00.314344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:26.304693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022009920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":400,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.175037+0000","last_change":"2024-09-21T07:35:27.867336+0000","last_active":"2024-09-21T07:55:00.175037+0000","last_peered":"2024-09-21T07:55:00.175037+0000","last_clean":"2024-09-21T07:55:00.175037+0000","last_became_active":"2024-09-21T07:35:27.867015+0000","last_became_peered":"2024-09-21T07:35:27.867015+0000","last_unstale":"2024-09-21T07:55:00.175037+0000","last_undegraded":"2024-09-21T07:55:00.175037+0000","last_fullsized":"2024-09-21T07:55:00.175037+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:59.214139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":411,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.315550+0000","last_change":"2024-09-21T07:35:29.493137+0000","last_active":"2024-09-21T07:55:00.315550+0000","last_peered":"2024-09-21T07:55:00.315550+0000","last_clean":"2024-09-21T07:55:00.315550+0000","last_became_active":"2024-09-21T07:35:26.872230+0000","last_became_peered":"2024-09-21T07:35:26.872230+0000","last_unstale":"2024-09-21T07:55:00.315550+0000","last_undegraded":"2024-09-21T07:55:00.315550+0000","last_fullsized":"2024-09-21T07:55:00.315550+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:24.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001197449,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":464,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.311160+0000","last_change":"2024-09-21T07:34:24.258754+0000","last_active":"2024-09-21T07:55:00.311160+0000","last_peered":"2024-09-21T07:55:00.311160+0000","last_clean":"2024-09-21T07:55:00.311160+0000","last_became_active":"2024-09-21T07:34:23.272657+0000","last_became_peered":"2024-09-21T07:34:23.272657+0000","last_unstale":"2024-09-21T07:55:00.311160+0000","last_undegraded":"2024-09-21T07:55:00.311160+0000","last_fullsized":"2024-09-21T07:55:00.311160+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:05.477099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067143200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":406,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.632348+0000","last_change":"2024-09-21T07:35:27.876021+0000","last_active":"2024-09-21T07:54:59.632348+0000","last_peered":"2024-09-21T07:54:59.632348+0000","last_clean":"2024-09-21T07:54:59.632348+0000","last_became_active":"2024-09-21T07:35:27.875724+0000","last_became_peered":"2024-09-21T07:35:27.875724+0000","last_unstale":"2024-09-21T07:54:59.632348+0000","last_undegraded":"2024-09-21T07:54:59.632348+0000","last_fullsized":"2024-09-21T07:54:59.632348+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:03.022812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":411,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.313993+0000","last_change":"2024-09-21T07:35:29.492570+0000","last_active":"2024-09-21T07:55:00.313993+0000","last_peered":"2024-09-21T07:55:00.313993+0000","last_clean":"2024-09-21T07:55:00.313993+0000","last_became_active":"2024-09-21T07:35:26.874316+0000","last_became_peered":"2024-09-21T07:35:26.874316+0000","last_unstale":"2024-09-21T07:55:00.313993+0000","last_undegraded":"2024-09-21T07:55:00.313993+0000","last_fullsized":"2024-09-21T07:55:00.313993+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:16.512323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011990760000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":464,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.633378+0000","last_change":"2024-09-21T07:34:25.085232+0000","last_active":"2024-09-21T07:54:59.633378+0000","last_peered":"2024-09-21T07:54:59.633378+0000","last_clean":"2024-09-21T07:54:59.633378+0000","last_became_active":"2024-09-21T07:34:23.277807+0000","last_became_peered":"2024-09-21T07:34:23.277807+0000","last_unstale":"2024-09-21T07:54:59.633378+0000","last_undegraded":"2024-09-21T07:54:59.633378+0000","last_fullsized":"2024-09-21T07:54:59.633378+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:36.362919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086619099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":409,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.316773+0000","last_change":"2024-09-21T07:35:27.872798+0000","last_active":"2024-09-21T07:55:00.316773+0000","last_peered":"2024-09-21T07:55:00.316773+0000","last_clean":"2024-09-21T07:55:00.316773+0000","last_became_active":"2024-09-21T07:35:27.872483+0000","last_became_peered":"2024-09-21T07:35:27.872483+0000","last_unstale":"2024-09-21T07:55:00.316773+0000","last_undegraded":"2024-09-21T07:55:00.316773+0000","last_fullsized":"2024-09-21T07:55:00.316773+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:11.933469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":416,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.315971+0000","last_change":"2024-09-21T07:35:29.206723+0000","last_active":"2024-09-21T07:55:00.315971+0000","last_peered":"2024-09-21T07:55:00.315971+0000","last_clean":"2024-09-21T07:55:00.315971+0000","last_became_active":"2024-09-21T07:35:26.872377+0000","last_became_peered":"2024-09-21T07:35:26.872377+0000","last_unstale":"2024-09-21T07:55:00.315971+0000","last_undegraded":"2024-09-21T07:55:00.315971+0000","last_fullsized":"2024-09-21T07:55:00.315971+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:29.086862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083077200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":456,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.175826+0000","last_change":"2024-09-21T07:34:24.597672+0000","last_active":"2024-09-21T07:55:00.175826+0000","last_peered":"2024-09-21T07:55:00.175826+0000","last_clean":"2024-09-21T07:55:00.175826+0000","last_became_active":"2024-09-21T07:34:23.267770+0000","last_became_peered":"2024-09-21T07:34:23.267770+0000","last_unstale":"2024-09-21T07:55:00.175826+0000","last_undegraded":"2024-09-21T07:55:00.175826+0000","last_fullsized":"2024-09-21T07:55:00.175826+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:51.629476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086378500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":409,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.316038+0000","last_change":"2024-09-21T07:35:27.870121+0000","last_active":"2024-09-21T07:55:00.316038+0000","last_peered":"2024-09-21T07:55:00.316038+0000","last_clean":"2024-09-21T07:55:00.316038+0000","last_became_active":"2024-09-21T07:35:27.869811+0000","last_became_peered":"2024-09-21T07:35:27.869811+0000","last_unstale":"2024-09-21T07:55:00.316038+0000","last_undegraded":"2024-09-21T07:55:00.316038+0000","last_fullsized":"2024-09-21T07:55:00.316038+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:31.778461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":414,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.634294+0000","last_change":"2024-09-21T07:35:28.860699+0000","last_active":"2024-09-21T07:54:59.634294+0000","last_peered":"2024-09-21T07:54:59.634294+0000","last_clean":"2024-09-21T07:54:59.634294+0000","last_became_active":"2024-09-21T07:35:26.876779+0000","last_became_peered":"2024-09-21T07:35:26.876779+0000","last_unstale":"2024-09-21T07:54:59.634294+0000","last_undegraded":"2024-09-21T07:54:59.634294+0000","last_fullsized":"2024-09-21T07:54:59.634294+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:28.286403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000913188,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":466,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.634607+0000","last_change":"2024-09-21T07:34:24.260181+0000","last_active":"2024-09-21T07:54:59.634607+0000","last_peered":"2024-09-21T07:54:59.634607+0000","last_clean":"2024-09-21T07:54:59.634607+0000","last_became_active":"2024-09-21T07:34:23.275580+0000","last_became_peered":"2024-09-21T07:34:23.275580+0000","last_unstale":"2024-09-21T07:54:59.634607+0000","last_undegraded":"2024-09-21T07:54:59.634607+0000","last_fullsized":"2024-09-21T07:54:59.634607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:49.857622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014706980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":407,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.314738+0000","last_change":"2024-09-21T07:35:27.875393+0000","last_active":"2024-09-21T07:55:00.314738+0000","last_peered":"2024-09-21T07:55:00.314738+0000","last_clean":"2024-09-21T07:55:00.314738+0000","last_became_active":"2024-09-21T07:35:27.874671+0000","last_became_peered":"2024-09-21T07:35:27.874671+0000","last_unstale":"2024-09-21T07:55:00.314738+0000","last_undegraded":"2024-09-21T07:55:00.314738+0000","last_fullsized":"2024-09-21T07:55:00.314738+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":416,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.312471+0000","last_change":"2024-09-21T07:35:29.206296+0000","last_active":"2024-09-21T07:55:00.312471+0000","last_peered":"2024-09-21T07:55:00.312471+0000","last_clean":"2024-09-21T07:55:00.312471+0000","last_became_active":"2024-09-21T07:35:26.874575+0000","last_became_peered":"2024-09-21T07:35:26.874575+0000","last_unstale":"2024-09-21T07:55:00.312471+0000","last_undegraded":"2024-09-21T07:55:00.312471+0000","last_fullsized":"2024-09-21T07:55:00.312471+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:22.771154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061011599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":467,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.315376+0000","last_change":"2024-09-21T07:34:24.259112+0000","last_active":"2024-09-21T07:55:00.315376+0000","last_peered":"2024-09-21T07:55:00.315376+0000","last_clean":"2024-09-21T07:55:00.315376+0000","last_became_active":"2024-09-21T07:34:23.635444+0000","last_became_peered":"2024-09-21T07:34:23.635444+0000","last_unstale":"2024-09-21T07:55:00.315376+0000","last_undegraded":"2024-09-21T07:55:00.315376+0000","last_fullsized":"2024-09-21T07:55:00.315376+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:46.255385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001952636,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":407,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.312824+0000","last_change":"2024-09-21T07:35:27.878356+0000","last_active":"2024-09-21T07:55:00.312824+0000","last_peered":"2024-09-21T07:55:00.312824+0000","last_clean":"2024-09-21T07:55:00.312824+0000","last_became_active":"2024-09-21T07:35:27.877982+0000","last_became_peered":"2024-09-21T07:35:27.877982+0000","last_unstale":"2024-09-21T07:55:00.312824+0000","last_undegraded":"2024-09-21T07:55:00.312824+0000","last_fullsized":"2024-09-21T07:55:00.312824+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":414,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.310325+0000","last_change":"2024-09-21T07:35:29.053131+0000","last_active":"2024-09-21T07:55:00.310325+0000","last_peered":"2024-09-21T07:55:00.310325+0000","last_clean":"2024-09-21T07:55:00.310325+0000","last_became_active":"2024-09-21T07:35:26.870516+0000","last_became_peered":"2024-09-21T07:35:26.870516+0000","last_unstale":"2024-09-21T07:55:00.310325+0000","last_undegraded":"2024-09-21T07:55:00.310325+0000","last_fullsized":"2024-09-21T07:55:00.310325+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:10.924491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070705699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":480,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.632868+0000","last_change":"2024-09-21T07:34:24.539152+0000","last_active":"2024-09-21T07:54:59.632868+0000","last_peered":"2024-09-21T07:54:59.632868+0000","last_clean":"2024-09-21T07:54:59.632868+0000","last_became_active":"2024-09-21T07:34:23.259573+0000","last_became_peered":"2024-09-21T07:34:23.259573+0000","last_unstale":"2024-09-21T07:54:59.632868+0000","last_undegraded":"2024-09-21T07:54:59.632868+0000","last_fullsized":"2024-09-21T07:54:59.632868+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:26.986900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058108400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":406,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.310193+0000","last_change":"2024-09-21T07:35:27.873726+0000","last_active":"2024-09-21T07:55:00.310193+0000","last_peered":"2024-09-21T07:55:00.310193+0000","last_clean":"2024-09-21T07:55:00.310193+0000","last_became_active":"2024-09-21T07:35:27.871421+0000","last_became_peered":"2024-09-21T07:35:27.871421+0000","last_unstale":"2024-09-21T07:55:00.310193+0000","last_undegraded":"2024-09-21T07:55:00.310193+0000","last_fullsized":"2024-09-21T07:55:00.310193+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:00.168915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":414,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.634353+0000","last_change":"2024-09-21T07:35:28.861032+0000","last_active":"2024-09-21T07:54:59.634353+0000","last_peered":"2024-09-21T07:54:59.634353+0000","last_clean":"2024-09-21T07:54:59.634353+0000","last_became_active":"2024-09-21T07:35:26.873149+0000","last_became_peered":"2024-09-21T07:35:26.873149+0000","last_unstale":"2024-09-21T07:54:59.634353+0000","last_undegraded":"2024-09-21T07:54:59.634353+0000","last_fullsized":"2024-09-21T07:54:59.634353+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:59.435981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011988369999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":463,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.311971+0000","last_change":"2024-09-21T07:34:24.259902+0000","last_active":"2024-09-21T07:55:00.311971+0000","last_peered":"2024-09-21T07:55:00.311971+0000","last_clean":"2024-09-21T07:55:00.311971+0000","last_became_active":"2024-09-21T07:34:23.271902+0000","last_became_peered":"2024-09-21T07:34:23.271902+0000","last_unstale":"2024-09-21T07:55:00.311971+0000","last_undegraded":"2024-09-21T07:55:00.311971+0000","last_fullsized":"2024-09-21T07:55:00.311971+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:59.439525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001686358,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":405,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.631813+0000","last_change":"2024-09-21T07:35:27.872637+0000","last_active":"2024-09-21T07:54:59.631813+0000","last_peered":"2024-09-21T07:54:59.631813+0000","last_clean":"2024-09-21T07:54:59.631813+0000","last_became_active":"2024-09-21T07:35:27.872149+0000","last_became_peered":"2024-09-21T07:35:27.872149+0000","last_unstale":"2024-09-21T07:54:59.631813+0000","last_undegraded":"2024-09-21T07:54:59.631813+0000","last_fullsized":"2024-09-21T07:54:59.631813+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:27.697872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":411,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.314113+0000","last_change":"2024-09-21T07:35:29.493362+0000","last_active":"2024-09-21T07:55:00.314113+0000","last_peered":"2024-09-21T07:55:00.314113+0000","last_clean":"2024-09-21T07:55:00.314113+0000","last_became_active":"2024-09-21T07:35:26.873018+0000","last_became_peered":"2024-09-21T07:35:26.873018+0000","last_unstale":"2024-09-21T07:55:00.314113+0000","last_undegraded":"2024-09-21T07:55:00.314113+0000","last_fullsized":"2024-09-21T07:55:00.314113+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:02.460557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012496510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":466,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.314038+0000","last_change":"2024-09-21T07:34:24.259662+0000","last_active":"2024-09-21T07:55:00.314038+0000","last_peered":"2024-09-21T07:55:00.314038+0000","last_clean":"2024-09-21T07:55:00.314038+0000","last_became_active":"2024-09-21T07:34:23.278812+0000","last_became_peered":"2024-09-21T07:34:23.278812+0000","last_unstale":"2024-09-21T07:55:00.314038+0000","last_undegraded":"2024-09-21T07:55:00.314038+0000","last_fullsized":"2024-09-21T07:55:00.314038+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:49.829974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356463,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":398,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.174789+0000","last_change":"2024-09-21T07:35:27.868448+0000","last_active":"2024-09-21T07:55:00.174789+0000","last_peered":"2024-09-21T07:55:00.174789+0000","last_clean":"2024-09-21T07:55:00.174789+0000","last_became_active":"2024-09-21T07:35:27.868105+0000","last_became_peered":"2024-09-21T07:35:27.868105+0000","last_unstale":"2024-09-21T07:55:00.174789+0000","last_undegraded":"2024-09-21T07:55:00.174789+0000","last_fullsized":"2024-09-21T07:55:00.174789+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:41.468702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":409,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.315283+0000","last_change":"2024-09-21T07:35:29.492974+0000","last_active":"2024-09-21T07:55:00.315283+0000","last_peered":"2024-09-21T07:55:00.315283+0000","last_clean":"2024-09-21T07:55:00.315283+0000","last_became_active":"2024-09-21T07:35:26.872165+0000","last_became_peered":"2024-09-21T07:35:26.872165+0000","last_unstale":"2024-09-21T07:55:00.315283+0000","last_undegraded":"2024-09-21T07:55:00.315283+0000","last_fullsized":"2024-09-21T07:55:00.315283+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:59.912520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013795179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":463,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.630639+0000","last_change":"2024-09-21T07:34:25.084861+0000","last_active":"2024-09-21T07:54:59.630639+0000","last_peered":"2024-09-21T07:54:59.630639+0000","last_clean":"2024-09-21T07:54:59.630639+0000","last_became_active":"2024-09-21T07:34:23.277708+0000","last_became_peered":"2024-09-21T07:34:23.277708+0000","last_unstale":"2024-09-21T07:54:59.630639+0000","last_undegraded":"2024-09-21T07:54:59.630639+0000","last_fullsized":"2024-09-21T07:54:59.630639+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:40.816488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000597566,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":398,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.175249+0000","last_change":"2024-09-21T07:35:27.874640+0000","last_active":"2024-09-21T07:55:00.175249+0000","last_peered":"2024-09-21T07:55:00.175249+0000","last_clean":"2024-09-21T07:55:00.175249+0000","last_became_active":"2024-09-21T07:35:27.874265+0000","last_became_peered":"2024-09-21T07:35:27.874265+0000","last_unstale":"2024-09-21T07:55:00.175249+0000","last_undegraded":"2024-09-21T07:55:00.175249+0000","last_fullsized":"2024-09-21T07:55:00.175249+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:45.754654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":414,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.318433+0000","last_change":"2024-09-21T07:35:29.207392+0000","last_active":"2024-09-21T07:55:00.318433+0000","last_peered":"2024-09-21T07:55:00.318433+0000","last_clean":"2024-09-21T07:55:00.318433+0000","last_became_active":"2024-09-21T07:35:26.870917+0000","last_became_peered":"2024-09-21T07:35:26.870917+0000","last_unstale":"2024-09-21T07:55:00.318433+0000","last_undegraded":"2024-09-21T07:55:00.318433+0000","last_fullsized":"2024-09-21T07:55:00.318433+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:59.159937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059198499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":467,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.313609+0000","last_change":"2024-09-21T07:34:24.258709+0000","last_active":"2024-09-21T07:55:00.313609+0000","last_peered":"2024-09-21T07:55:00.313609+0000","last_clean":"2024-09-21T07:55:00.313609+0000","last_became_active":"2024-09-21T07:34:23.279042+0000","last_became_peered":"2024-09-21T07:34:23.279042+0000","last_unstale":"2024-09-21T07:55:00.313609+0000","last_undegraded":"2024-09-21T07:55:00.313609+0000","last_fullsized":"2024-09-21T07:55:00.313609+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:01.762117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019282100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":404,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.633291+0000","last_change":"2024-09-21T07:35:27.870260+0000","last_active":"2024-09-21T07:54:59.633291+0000","last_peered":"2024-09-21T07:54:59.633291+0000","last_clean":"2024-09-21T07:54:59.633291+0000","last_became_active":"2024-09-21T07:35:27.869448+0000","last_became_peered":"2024-09-21T07:35:27.869448+0000","last_unstale":"2024-09-21T07:54:59.633291+0000","last_undegraded":"2024-09-21T07:54:59.633291+0000","last_fullsized":"2024-09-21T07:54:59.633291+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:16.946790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":405,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.175086+0000","last_change":"2024-09-21T07:35:29.386367+0000","last_active":"2024-09-21T07:55:00.175086+0000","last_peered":"2024-09-21T07:55:00.175086+0000","last_clean":"2024-09-21T07:55:00.175086+0000","last_became_active":"2024-09-21T07:35:26.879238+0000","last_became_peered":"2024-09-21T07:35:26.879238+0000","last_unstale":"2024-09-21T07:55:00.175086+0000","last_undegraded":"2024-09-21T07:55:00.175086+0000","last_fullsized":"2024-09-21T07:55:00.175086+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:16.528260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092223100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":463,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.632428+0000","last_change":"2024-09-21T07:34:25.085635+0000","last_active":"2024-09-21T07:54:59.632428+0000","last_peered":"2024-09-21T07:54:59.632428+0000","last_clean":"2024-09-21T07:54:59.632428+0000","last_became_active":"2024-09-21T07:34:23.273566+0000","last_became_peered":"2024-09-21T07:34:23.273566+0000","last_unstale":"2024-09-21T07:54:59.632428+0000","last_undegraded":"2024-09-21T07:54:59.632428+0000","last_fullsized":"2024-09-21T07:54:59.632428+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:14.142272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079897300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":404,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.633141+0000","last_change":"2024-09-21T07:35:27.878728+0000","last_active":"2024-09-21T07:54:59.633141+0000","last_peered":"2024-09-21T07:54:59.633141+0000","last_clean":"2024-09-21T07:54:59.633141+0000","last_became_active":"2024-09-21T07:35:27.878335+0000","last_became_peered":"2024-09-21T07:35:27.878335+0000","last_unstale":"2024-09-21T07:54:59.633141+0000","last_undegraded":"2024-09-21T07:54:59.633141+0000","last_fullsized":"2024-09-21T07:54:59.633141+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:25:37.731771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":426,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.630524+0000","last_change":"2024-09-21T07:35:28.858418+0000","last_active":"2024-09-21T07:54:59.630524+0000","last_peered":"2024-09-21T07:54:59.630524+0000","last_clean":"2024-09-21T07:54:59.630524+0000","last_became_active":"2024-09-21T07:35:26.876944+0000","last_became_peered":"2024-09-21T07:35:26.876944+0000","last_unstale":"2024-09-21T07:54:59.630524+0000","last_undegraded":"2024-09-21T07:54:59.630524+0000","last_fullsized":"2024-09-21T07:54:59.630524+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:04.514304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090862799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":455,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.176081+0000","last_change":"2024-09-21T07:34:24.597926+0000","last_active":"2024-09-21T07:55:00.176081+0000","last_peered":"2024-09-21T07:55:00.176081+0000","last_clean":"2024-09-21T07:55:00.176081+0000","last_became_active":"2024-09-21T07:34:23.268926+0000","last_became_peered":"2024-09-21T07:34:23.268926+0000","last_unstale":"2024-09-21T07:55:00.176081+0000","last_undegraded":"2024-09-21T07:55:00.176081+0000","last_fullsized":"2024-09-21T07:55:00.176081+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:46.190752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101851,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":400,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.175363+0000","last_change":"2024-09-21T07:35:27.877426+0000","last_active":"2024-09-21T07:55:00.175363+0000","last_peered":"2024-09-21T07:55:00.175363+0000","last_clean":"2024-09-21T07:55:00.175363+0000","last_became_active":"2024-09-21T07:35:27.877050+0000","last_became_peered":"2024-09-21T07:35:27.877050+0000","last_unstale":"2024-09-21T07:55:00.175363+0000","last_undegraded":"2024-09-21T07:55:00.175363+0000","last_fullsized":"2024-09-21T07:55:00.175363+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:36.268334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":412,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.631943+0000","last_change":"2024-09-21T07:35:28.862273+0000","last_active":"2024-09-21T07:54:59.631943+0000","last_peered":"2024-09-21T07:54:59.631943+0000","last_clean":"2024-09-21T07:54:59.631943+0000","last_became_active":"2024-09-21T07:35:26.874011+0000","last_became_peered":"2024-09-21T07:35:26.874011+0000","last_unstale":"2024-09-21T07:54:59.631943+0000","last_undegraded":"2024-09-21T07:54:59.631943+0000","last_fullsized":"2024-09-21T07:54:59.631943+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:11.986950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071644799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":465,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.637034+0000","last_change":"2024-09-21T07:34:24.259794+0000","last_active":"2024-09-21T07:54:59.637034+0000","last_peered":"2024-09-21T07:54:59.637034+0000","last_clean":"2024-09-21T07:54:59.637034+0000","last_became_active":"2024-09-21T07:34:23.265679+0000","last_became_peered":"2024-09-21T07:34:23.265679+0000","last_unstale":"2024-09-21T07:54:59.637034+0000","last_undegraded":"2024-09-21T07:54:59.637034+0000","last_fullsized":"2024-09-21T07:54:59.637034+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:02.575796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001298518,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":409,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.317882+0000","last_change":"2024-09-21T07:35:27.877725+0000","last_active":"2024-09-21T07:55:00.317882+0000","last_peered":"2024-09-21T07:55:00.317882+0000","last_clean":"2024-09-21T07:55:00.317882+0000","last_became_active":"2024-09-21T07:35:27.877398+0000","last_became_peered":"2024-09-21T07:35:27.877398+0000","last_unstale":"2024-09-21T07:55:00.317882+0000","last_undegraded":"2024-09-21T07:55:00.317882+0000","last_fullsized":"2024-09-21T07:55:00.317882+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:05.859762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":413,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.633091+0000","last_change":"2024-09-21T07:35:28.860612+0000","last_active":"2024-09-21T07:54:59.633091+0000","last_peered":"2024-09-21T07:54:59.633091+0000","last_clean":"2024-09-21T07:54:59.633091+0000","last_became_active":"2024-09-21T07:35:26.876562+0000","last_became_peered":"2024-09-21T07:35:26.876562+0000","last_unstale":"2024-09-21T07:54:59.633091+0000","last_undegraded":"2024-09-21T07:54:59.633091+0000","last_fullsized":"2024-09-21T07:54:59.633091+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:06.824331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012252470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":463,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.315655+0000","last_change":"2024-09-21T07:34:24.260330+0000","last_active":"2024-09-21T07:55:00.315655+0000","last_peered":"2024-09-21T07:55:00.315655+0000","last_clean":"2024-09-21T07:55:00.315655+0000","last_became_active":"2024-09-21T07:34:23.637210+0000","last_became_peered":"2024-09-21T07:34:23.637210+0000","last_unstale":"2024-09-21T07:55:00.315655+0000","last_undegraded":"2024-09-21T07:55:00.315655+0000","last_fullsized":"2024-09-21T07:55:00.315655+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.488451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001537093,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":405,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.630514+0000","last_change":"2024-09-21T07:35:27.874757+0000","last_active":"2024-09-21T07:54:59.630514+0000","last_peered":"2024-09-21T07:54:59.630514+0000","last_clean":"2024-09-21T07:54:59.630514+0000","last_became_active":"2024-09-21T07:35:27.874360+0000","last_became_peered":"2024-09-21T07:35:27.874360+0000","last_unstale":"2024-09-21T07:54:59.630514+0000","last_undegraded":"2024-09-21T07:54:59.630514+0000","last_fullsized":"2024-09-21T07:54:59.630514+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:17.646912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":410,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.314189+0000","last_change":"2024-09-21T07:35:29.493431+0000","last_active":"2024-09-21T07:55:00.314189+0000","last_peered":"2024-09-21T07:55:00.314189+0000","last_clean":"2024-09-21T07:55:00.314189+0000","last_became_active":"2024-09-21T07:35:26.873691+0000","last_became_peered":"2024-09-21T07:35:26.873691+0000","last_unstale":"2024-09-21T07:55:00.314189+0000","last_undegraded":"2024-09-21T07:55:00.314189+0000","last_fullsized":"2024-09-21T07:55:00.314189+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:51.705051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001162464,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":455,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.175180+0000","last_change":"2024-09-21T07:34:24.597286+0000","last_active":"2024-09-21T07:55:00.175180+0000","last_peered":"2024-09-21T07:55:00.175180+0000","last_clean":"2024-09-21T07:55:00.175180+0000","last_became_active":"2024-09-21T07:34:23.267133+0000","last_became_peered":"2024-09-21T07:34:23.267133+0000","last_unstale":"2024-09-21T07:55:00.175180+0000","last_undegraded":"2024-09-21T07:55:00.175180+0000","last_fullsized":"2024-09-21T07:55:00.175180+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:18.257010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":420,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.630258+0000","last_change":"2024-09-21T07:35:27.864458+0000","last_active":"2024-09-21T07:54:59.630258+0000","last_peered":"2024-09-21T07:54:59.630258+0000","last_clean":"2024-09-21T07:54:59.630258+0000","last_became_active":"2024-09-21T07:35:27.864136+0000","last_became_peered":"2024-09-21T07:35:27.864136+0000","last_unstale":"2024-09-21T07:54:59.630258+0000","last_undegraded":"2024-09-21T07:54:59.630258+0000","last_fullsized":"2024-09-21T07:54:59.630258+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:20.162240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":427,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.632799+0000","last_change":"2024-09-21T07:35:28.858784+0000","last_active":"2024-09-21T07:54:59.632799+0000","last_peered":"2024-09-21T07:54:59.632799+0000","last_clean":"2024-09-21T07:54:59.632799+0000","last_became_active":"2024-09-21T07:35:26.876890+0000","last_became_peered":"2024-09-21T07:35:26.876890+0000","last_unstale":"2024-09-21T07:54:59.632799+0000","last_undegraded":"2024-09-21T07:54:59.632799+0000","last_fullsized":"2024-09-21T07:54:59.632799+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:40.738327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064543499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":466,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.313937+0000","last_change":"2024-09-21T07:34:24.258712+0000","last_active":"2024-09-21T07:55:00.313937+0000","last_peered":"2024-09-21T07:55:00.313937+0000","last_clean":"2024-09-21T07:55:00.313937+0000","last_became_active":"2024-09-21T07:34:23.274692+0000","last_became_peered":"2024-09-21T07:34:23.274692+0000","last_unstale":"2024-09-21T07:55:00.313937+0000","last_undegraded":"2024-09-21T07:55:00.313937+0000","last_fullsized":"2024-09-21T07:55:00.313937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:43.279454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018051479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":406,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.312289+0000","last_change":"2024-09-21T07:35:27.878020+0000","last_active":"2024-09-21T07:55:00.312289+0000","last_peered":"2024-09-21T07:55:00.312289+0000","last_clean":"2024-09-21T07:55:00.312289+0000","last_became_active":"2024-09-21T07:35:27.877532+0000","last_became_peered":"2024-09-21T07:35:27.877532+0000","last_unstale":"2024-09-21T07:55:00.312289+0000","last_undegraded":"2024-09-21T07:55:00.312289+0000","last_fullsized":"2024-09-21T07:55:00.312289+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:19.756073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":413,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.633494+0000","last_change":"2024-09-21T07:35:28.861049+0000","last_active":"2024-09-21T07:54:59.633494+0000","last_peered":"2024-09-21T07:54:59.633494+0000","last_clean":"2024-09-21T07:54:59.633494+0000","last_became_active":"2024-09-21T07:35:26.876819+0000","last_became_peered":"2024-09-21T07:35:26.876819+0000","last_unstale":"2024-09-21T07:54:59.633494+0000","last_undegraded":"2024-09-21T07:54:59.633494+0000","last_fullsized":"2024-09-21T07:54:59.633494+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:19.633803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055780599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":463,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.630568+0000","last_change":"2024-09-21T07:34:25.084849+0000","last_active":"2024-09-21T07:54:59.630568+0000","last_peered":"2024-09-21T07:54:59.630568+0000","last_clean":"2024-09-21T07:54:59.630568+0000","last_became_active":"2024-09-21T07:34:23.277713+0000","last_became_peered":"2024-09-21T07:34:23.277713+0000","last_unstale":"2024-09-21T07:54:59.630568+0000","last_undegraded":"2024-09-21T07:54:59.630568+0000","last_fullsized":"2024-09-21T07:54:59.630568+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:05.811810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059079499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":406,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.312851+0000","last_change":"2024-09-21T07:35:27.884248+0000","last_active":"2024-09-21T07:55:00.312851+0000","last_peered":"2024-09-21T07:55:00.312851+0000","last_clean":"2024-09-21T07:55:00.312851+0000","last_became_active":"2024-09-21T07:35:27.883861+0000","last_became_peered":"2024-09-21T07:35:27.883861+0000","last_unstale":"2024-09-21T07:55:00.312851+0000","last_undegraded":"2024-09-21T07:55:00.312851+0000","last_fullsized":"2024-09-21T07:55:00.312851+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":415,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.318311+0000","last_change":"2024-09-21T07:35:29.207135+0000","last_active":"2024-09-21T07:55:00.318311+0000","last_peered":"2024-09-21T07:55:00.318311+0000","last_clean":"2024-09-21T07:55:00.318311+0000","last_became_active":"2024-09-21T07:35:26.872787+0000","last_became_peered":"2024-09-21T07:35:26.872787+0000","last_unstale":"2024-09-21T07:55:00.318311+0000","last_undegraded":"2024-09-21T07:55:00.318311+0000","last_fullsized":"2024-09-21T07:55:00.318311+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:27.543217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011157509999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":479,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.630798+0000","last_change":"2024-09-21T07:34:24.538610+0000","last_active":"2024-09-21T07:54:59.630798+0000","last_peered":"2024-09-21T07:54:59.630798+0000","last_clean":"2024-09-21T07:54:59.630798+0000","last_became_active":"2024-09-21T07:34:23.259700+0000","last_became_peered":"2024-09-21T07:34:23.259700+0000","last_unstale":"2024-09-21T07:54:59.630798+0000","last_undegraded":"2024-09-21T07:54:59.630798+0000","last_fullsized":"2024-09-21T07:54:59.630798+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:57.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070522399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":420,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.630936+0000","last_change":"2024-09-21T07:35:27.868024+0000","last_active":"2024-09-21T07:54:59.630936+0000","last_peered":"2024-09-21T07:54:59.630936+0000","last_clean":"2024-09-21T07:54:59.630936+0000","last_became_active":"2024-09-21T07:35:27.867339+0000","last_became_peered":"2024-09-21T07:35:27.867339+0000","last_unstale":"2024-09-21T07:54:59.630936+0000","last_undegraded":"2024-09-21T07:54:59.630936+0000","last_fullsized":"2024-09-21T07:54:59.630936+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":427,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.631794+0000","last_change":"2024-09-21T07:35:28.858505+0000","last_active":"2024-09-21T07:54:59.631794+0000","last_peered":"2024-09-21T07:54:59.631794+0000","last_clean":"2024-09-21T07:54:59.631794+0000","last_became_active":"2024-09-21T07:35:26.877320+0000","last_became_peered":"2024-09-21T07:35:26.877320+0000","last_unstale":"2024-09-21T07:54:59.631794+0000","last_undegraded":"2024-09-21T07:54:59.631794+0000","last_fullsized":"2024-09-21T07:54:59.631794+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:53.672782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244119999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":457,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.175837+0000","last_change":"2024-09-21T07:34:24.597597+0000","last_active":"2024-09-21T07:55:00.175837+0000","last_peered":"2024-09-21T07:55:00.175837+0000","last_clean":"2024-09-21T07:55:00.175837+0000","last_became_active":"2024-09-21T07:34:23.635204+0000","last_became_peered":"2024-09-21T07:34:23.635204+0000","last_unstale":"2024-09-21T07:55:00.175837+0000","last_undegraded":"2024-09-21T07:55:00.175837+0000","last_fullsized":"2024-09-21T07:55:00.175837+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:48.644789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081139199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":408,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.316004+0000","last_change":"2024-09-21T07:35:27.877503+0000","last_active":"2024-09-21T07:55:00.316004+0000","last_peered":"2024-09-21T07:55:00.316004+0000","last_clean":"2024-09-21T07:55:00.316004+0000","last_became_active":"2024-09-21T07:35:27.875775+0000","last_became_peered":"2024-09-21T07:35:27.875775+0000","last_unstale":"2024-09-21T07:55:00.316004+0000","last_undegraded":"2024-09-21T07:55:00.316004+0000","last_fullsized":"2024-09-21T07:55:00.316004+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:49.291023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":410,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.311850+0000","last_change":"2024-09-21T07:35:29.491412+0000","last_active":"2024-09-21T07:55:00.311850+0000","last_peered":"2024-09-21T07:55:00.311850+0000","last_clean":"2024-09-21T07:55:00.311850+0000","last_became_active":"2024-09-21T07:35:26.876404+0000","last_became_peered":"2024-09-21T07:35:26.876404+0000","last_unstale":"2024-09-21T07:55:00.311850+0000","last_undegraded":"2024-09-21T07:55:00.311850+0000","last_fullsized":"2024-09-21T07:55:00.311850+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063918000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":467,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.314021+0000","last_change":"2024-09-21T07:34:24.259026+0000","last_active":"2024-09-21T07:55:00.314021+0000","last_peered":"2024-09-21T07:55:00.314021+0000","last_clean":"2024-09-21T07:55:00.314021+0000","last_became_active":"2024-09-21T07:34:23.635456+0000","last_became_peered":"2024-09-21T07:34:23.635456+0000","last_unstale":"2024-09-21T07:55:00.314021+0000","last_undegraded":"2024-09-21T07:55:00.314021+0000","last_fullsized":"2024-09-21T07:55:00.314021+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:09.592752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018654419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":408,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.316264+0000","last_change":"2024-09-21T07:35:27.871984+0000","last_active":"2024-09-21T07:55:00.316264+0000","last_peered":"2024-09-21T07:55:00.316264+0000","last_clean":"2024-09-21T07:55:00.316264+0000","last_became_active":"2024-09-21T07:35:27.870336+0000","last_became_peered":"2024-09-21T07:35:27.870336+0000","last_unstale":"2024-09-21T07:55:00.316264+0000","last_undegraded":"2024-09-21T07:55:00.316264+0000","last_fullsized":"2024-09-21T07:55:00.316264+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:57.120479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":413,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.631715+0000","last_change":"2024-09-21T07:35:28.860351+0000","last_active":"2024-09-21T07:54:59.631715+0000","last_peered":"2024-09-21T07:54:59.631715+0000","last_clean":"2024-09-21T07:54:59.631715+0000","last_became_active":"2024-09-21T07:35:26.872046+0000","last_became_peered":"2024-09-21T07:35:26.872046+0000","last_unstale":"2024-09-21T07:54:59.631715+0000","last_undegraded":"2024-09-21T07:54:59.631715+0000","last_fullsized":"2024-09-21T07:54:59.631715+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:47.074327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098384399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":463,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.315458+0000","last_change":"2024-09-21T07:34:24.260300+0000","last_active":"2024-09-21T07:55:00.315458+0000","last_peered":"2024-09-21T07:55:00.315458+0000","last_clean":"2024-09-21T07:55:00.315458+0000","last_became_active":"2024-09-21T07:34:23.270128+0000","last_became_peered":"2024-09-21T07:34:23.270128+0000","last_unstale":"2024-09-21T07:55:00.315458+0000","last_undegraded":"2024-09-21T07:55:00.315458+0000","last_fullsized":"2024-09-21T07:55:00.315458+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:25.260931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00176651,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":408,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.318010+0000","last_change":"2024-09-21T07:35:27.877861+0000","last_active":"2024-09-21T07:55:00.318010+0000","last_peered":"2024-09-21T07:55:00.318010+0000","last_clean":"2024-09-21T07:55:00.318010+0000","last_became_active":"2024-09-21T07:35:27.877434+0000","last_became_peered":"2024-09-21T07:35:27.877434+0000","last_unstale":"2024-09-21T07:55:00.318010+0000","last_undegraded":"2024-09-21T07:55:00.318010+0000","last_fullsized":"2024-09-21T07:55:00.318010+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":412,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.630743+0000","last_change":"2024-09-21T07:35:28.862954+0000","last_active":"2024-09-21T07:54:59.630743+0000","last_peered":"2024-09-21T07:54:59.630743+0000","last_clean":"2024-09-21T07:54:59.630743+0000","last_became_active":"2024-09-21T07:35:26.870613+0000","last_became_peered":"2024-09-21T07:35:26.870613+0000","last_unstale":"2024-09-21T07:54:59.630743+0000","last_undegraded":"2024-09-21T07:54:59.630743+0000","last_fullsized":"2024-09-21T07:54:59.630743+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:06.901043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059842100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":464,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.314518+0000","last_change":"2024-09-21T07:34:25.028904+0000","last_active":"2024-09-21T07:55:00.314518+0000","last_peered":"2024-09-21T07:55:00.314518+0000","last_clean":"2024-09-21T07:55:00.314518+0000","last_became_active":"2024-09-21T07:34:23.260942+0000","last_became_peered":"2024-09-21T07:34:23.260942+0000","last_unstale":"2024-09-21T07:55:00.314518+0000","last_undegraded":"2024-09-21T07:55:00.314518+0000","last_fullsized":"2024-09-21T07:55:00.314518+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:22.709328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060553899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509707,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71188,"kb_used_data":3900,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94300652,"statfs":{"total":96636764160,"available":96563867648,"internally_reserved":0,"allocated":3993600,"data_stored":2441691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509709,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74768,"kb_used_data":3320,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297072,"statfs":{"total":96636764160,"available":96560201728,"internally_reserved":0,"allocated":3399680,"data_stored":1851323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509707,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52556,"kb_used_data":3320,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319284,"statfs":{"total":96636764160,"available":96582946816,"internally_reserved":0,"allocated":3399680,"data_stored":1851323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509708,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30348,"kb_used_data":3320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341492,"statfs":{"total":96636764160,"available":96605687808,"internally_reserved":0,"allocated":3399680,"data_stored":1851323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509709,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48396,"kb_used_data":3320,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323444,"statfs":{"total":96636764160,"available":96587206656,"internally_reserved":0,"allocated":3399680,"data_stored":1851323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509709,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57300,"kb_used_data":3900,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314540,"statfs":{"total":96636764160,"available":96578088960,"internally_reserved":0,"allocated":3993600,"data_stored":2441691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542412,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56720,"kb_used_data":3320,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315120,"statfs":{"total":96636764160,"available":96578682880,"internally_reserved":0,"allocated":3399680,"data_stored":1851323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542411,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74768,"kb_used_data":3320,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297072,"statfs":{"total":96636764160,"available":96560201728,"internally_reserved":0,"allocated":3399680,"data_stored":1851323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:55:04.109 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:55:04.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.370+0000 7fc680400640 1 -- 172.21.15.39:0/3278709409 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6780b5240 msgr2=0x7fc67815c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:04.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.370+0000 7fc680400640 1 --2- 172.21.15.39:0/3278709409 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6780b5240 0x7fc67815c690 secure :-1 s=READY pgs=2885 cs=0 l=1 rev1=1 crypto rx=0x7fc67806dd90 tx=0x7fc66c02a4a0 comp rx=0 tx=0).stop 2024-09-21T07:55:04.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.370+0000 7fc680400640 1 -- 172.21.15.39:0/3278709409 shutdown_connections 2024-09-21T07:55:04.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.370+0000 7fc680400640 1 --2- 172.21.15.39:0/3278709409 >> v2:172.21.15.148:3300/0 conn(0x7fc6780cf920 0x7fc6780d0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:04.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.370+0000 7fc680400640 1 --2- 172.21.15.39:0/3278709409 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6780b5240 0x7fc67815c690 unknown :-1 s=CLOSED pgs=2885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:04.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.370+0000 7fc680400640 1 -- 172.21.15.39:0/3278709409 >> 172.21.15.39:0/3278709409 conn(0x7fc67806cfc0 msgr2=0x7fc67806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:04.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.370+0000 7fc680400640 1 -- 172.21.15.39:0/3278709409 shutdown_connections 2024-09-21T07:55:04.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.370+0000 7fc680400640 1 -- 172.21.15.39:0/3278709409 wait complete. 2024-09-21T07:55:04.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.373+0000 7fc680400640 1 Processor -- start 2024-09-21T07:55:04.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.373+0000 7fc680400640 1 -- start start 2024-09-21T07:55:04.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.374+0000 7fc680400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc6780b5240 0x7fc678139ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:04.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.374+0000 7fc680400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6780cf920 0x7fc6781e4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:04.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.374+0000 7fc680400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6780cfd30 0x7fc67815d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:04.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.374+0000 7fc680400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc6780d0cf0 con 0x7fc6780cfd30 2024-09-21T07:55:04.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.374+0000 7fc680400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc6780d0b70 con 0x7fc6780cf920 2024-09-21T07:55:04.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.374+0000 7fc680400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fc6780d0e70 con 0x7fc6780b5240 2024-09-21T07:55:04.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.375+0000 7fc67d000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6780cf920 0x7fc6781e4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:04.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.375+0000 7fc67e400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6780cfd30 0x7fc67815d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:04.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.375+0000 7fc67d000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6780cf920 0x7fc6781e4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.39:33724/0 (socket says 172.21.15.39:33724) 2024-09-21T07:55:04.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.375+0000 7fc67d000640 1 -- 172.21.15.39:0/224446433 learned_addr learned my addr 172.21.15.39:0/224446433 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:55:04.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.375+0000 7fc67da00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc6780b5240 0x7fc678139ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:04.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.375+0000 7fc67e400640 1 -- 172.21.15.39:0/224446433 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc6780b5240 msgr2=0x7fc678139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:04.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.375+0000 7fc67e400640 1 --2- 172.21.15.39:0/224446433 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc6780b5240 0x7fc678139ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:04.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.375+0000 7fc67e400640 1 -- 172.21.15.39:0/224446433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6780cf920 msgr2=0x7fc6781e4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:04.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.375+0000 7fc67e400640 1 --2- 172.21.15.39:0/224446433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6780cf920 0x7fc6781e4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:04.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.375+0000 7fc67e400640 1 -- 172.21.15.39:0/224446433 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6781e60b0 con 0x7fc6780cfd30 2024-09-21T07:55:04.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.375+0000 7fc67d000640 1 --2- 172.21.15.39:0/224446433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6780cf920 0x7fc6781e4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:55:04.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.375+0000 7fc67e400640 1 --2- 172.21.15.39:0/224446433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6780cfd30 0x7fc67815d900 secure :-1 s=READY pgs=2886 cs=0 l=1 rev1=1 crypto rx=0x7fc67000c700 tx=0x7fc67000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:04.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.376+0000 7fc675600640 1 -- 172.21.15.39:0/224446433 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc670017070 con 0x7fc6780cfd30 2024-09-21T07:55:04.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.376+0000 7fc675600640 1 -- 172.21.15.39:0/224446433 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc670003c60 con 0x7fc6780cfd30 2024-09-21T07:55:04.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.376+0000 7fc675600640 1 -- 172.21.15.39:0/224446433 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6700052d0 con 0x7fc6780cfd30 2024-09-21T07:55:04.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.376+0000 7fc680400640 1 -- 172.21.15.39:0/224446433 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6781e6390 con 0x7fc6780cfd30 2024-09-21T07:55:04.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.376+0000 7fc680400640 1 -- 172.21.15.39:0/224446433 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6781e6950 con 0x7fc6780cfd30 2024-09-21T07:55:04.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.377+0000 7fc680400640 1 -- 172.21.15.39:0/224446433 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6780b4ca0 con 0x7fc6780cfd30 2024-09-21T07:55:04.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.378+0000 7fc675600640 1 -- 172.21.15.39:0/224446433 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc67001a070 con 0x7fc6780cfd30 2024-09-21T07:55:04.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.379+0000 7fc675600640 1 --2- 172.21.15.39:0/224446433 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc658078b90 0x7fc65807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:04.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.380+0000 7fc675600640 1 -- 172.21.15.39:0/224446433 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(269..269 src has 1..269) ==== 6278+0+0 (secure 0 0 0) 0x7fc67009a600 con 0x7fc6780cfd30 2024-09-21T07:55:04.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.380+0000 7fc67da00640 1 --2- 172.21.15.39:0/224446433 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc658078b90 0x7fc65807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:04.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.381+0000 7fc67da00640 1 --2- 172.21.15.39:0/224446433 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc658078b90 0x7fc65807b0a0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fc66c034000 tx=0x7fc66c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:04.386 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:55:04.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.384+0000 7fc675600640 1 -- 172.21.15.39:0/224446433 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc670062110 con 0x7fc6780cfd30 2024-09-21T07:55:04.389 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:55:04.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.580+0000 7fc680400640 1 -- 172.21.15.39:0/224446433 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc67813a0e0 con 0x7fc658078b90 2024-09-21T07:55:04.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.585+0000 7fc675600640 1 -- 172.21.15.39:0/224446433 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+256262 (secure 0 0 0) 0x7fc67813a0e0 con 0x7fc658078b90 2024-09-21T07:55:04.589 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:55:04.591 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:55:04.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.589+0000 7fc680400640 1 -- 172.21.15.39:0/224446433 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc658078b90 msgr2=0x7fc65807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:04.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.589+0000 7fc680400640 1 --2- 172.21.15.39:0/224446433 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc658078b90 0x7fc65807b0a0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fc66c034000 tx=0x7fc66c004730 comp rx=0 tx=0).stop 2024-09-21T07:55:04.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.589+0000 7fc680400640 1 -- 172.21.15.39:0/224446433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6780cfd30 msgr2=0x7fc67815d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:04.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.589+0000 7fc680400640 1 --2- 172.21.15.39:0/224446433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6780cfd30 0x7fc67815d900 secure :-1 s=READY pgs=2886 cs=0 l=1 rev1=1 crypto rx=0x7fc67000c700 tx=0x7fc67000cbc0 comp rx=0 tx=0).stop 2024-09-21T07:55:04.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.589+0000 7fc680400640 1 -- 172.21.15.39:0/224446433 shutdown_connections 2024-09-21T07:55:04.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.589+0000 7fc680400640 1 --2- 172.21.15.39:0/224446433 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fc658078b90 0x7fc65807b0a0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:04.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.589+0000 7fc680400640 1 --2- 172.21.15.39:0/224446433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6780cfd30 0x7fc67815d900 unknown :-1 s=CLOSED pgs=2886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:04.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.589+0000 7fc680400640 1 --2- 172.21.15.39:0/224446433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6780cf920 0x7fc6781e4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:04.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.589+0000 7fc680400640 1 --2- 172.21.15.39:0/224446433 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fc6780b5240 0x7fc678139ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:04.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.589+0000 7fc680400640 1 -- 172.21.15.39:0/224446433 >> 172.21.15.39:0/224446433 conn(0x7fc67806cfc0 msgr2=0x7fc67806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:04.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.589+0000 7fc680400640 1 -- 172.21.15.39:0/224446433 shutdown_connections 2024-09-21T07:55:04.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.589+0000 7fc680400640 1 -- 172.21.15.39:0/224446433 wait complete. 2024-09-21T07:55:04.616 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":925,"stamp":"2024-09-21T07:55:03.986673+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":211,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":466044,"kb_used_data":27720,"kb_used_omap":12,"kb_used_meta":438131,"kb_avail":754508676,"statfs":{"total":773094113280,"available":772616884224,"internally_reserved":0,"allocated":28385280,"data_stored":15991320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":448646704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12736,"num_objects":-25,"num_object_clones":-1,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":-1,"num_read":-1,"num_read_kb":-1,"num_write":-60,"num_write_kb":-31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.383374"},"pg_stats":[{"pgid":"12.d","version":"0'0","reported_seq":403,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.315418+0000","last_change":"2024-09-21T07:35:27.879605+0000","last_active":"2024-09-21T07:55:00.315418+0000","last_peered":"2024-09-21T07:55:00.315418+0000","last_clean":"2024-09-21T07:55:00.315418+0000","last_became_active":"2024-09-21T07:35:27.879274+0000","last_became_peered":"2024-09-21T07:35:27.879274+0000","last_unstale":"2024-09-21T07:55:00.315418+0000","last_undegraded":"2024-09-21T07:55:00.315418+0000","last_fullsized":"2024-09-21T07:55:00.315418+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:24.241636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":413,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.312401+0000","last_change":"2024-09-21T07:35:29.491336+0000","last_active":"2024-09-21T07:55:00.312401+0000","last_peered":"2024-09-21T07:55:00.312401+0000","last_clean":"2024-09-21T07:55:00.312401+0000","last_became_active":"2024-09-21T07:35:26.866484+0000","last_became_peered":"2024-09-21T07:35:26.866484+0000","last_unstale":"2024-09-21T07:55:00.312401+0000","last_undegraded":"2024-09-21T07:55:00.312401+0000","last_fullsized":"2024-09-21T07:55:00.312401+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:10.449665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076010100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":589,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.311307+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:55:00.311307+0000","last_peered":"2024-09-21T07:55:00.311307+0000","last_clean":"2024-09-21T07:55:00.311307+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:55:00.311307+0000","last_undegraded":"2024-09-21T07:55:00.311307+0000","last_fullsized":"2024-09-21T07:55:00.311307+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":480,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.632658+0000","last_change":"2024-09-21T07:34:24.539248+0000","last_active":"2024-09-21T07:54:59.632658+0000","last_peered":"2024-09-21T07:54:59.632658+0000","last_clean":"2024-09-21T07:54:59.632658+0000","last_became_active":"2024-09-21T07:34:23.273465+0000","last_became_peered":"2024-09-21T07:34:23.273465+0000","last_unstale":"2024-09-21T07:54:59.632658+0000","last_undegraded":"2024-09-21T07:54:59.632658+0000","last_fullsized":"2024-09-21T07:54:59.632658+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:55.517452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060627700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":407,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.312347+0000","last_change":"2024-09-21T07:35:27.864492+0000","last_active":"2024-09-21T07:55:00.312347+0000","last_peered":"2024-09-21T07:55:00.312347+0000","last_clean":"2024-09-21T07:55:00.312347+0000","last_became_active":"2024-09-21T07:35:27.864176+0000","last_became_peered":"2024-09-21T07:35:27.864176+0000","last_unstale":"2024-09-21T07:55:00.312347+0000","last_undegraded":"2024-09-21T07:55:00.312347+0000","last_fullsized":"2024-09-21T07:55:00.312347+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:05.153467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":410,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.314353+0000","last_change":"2024-09-21T07:35:29.493678+0000","last_active":"2024-09-21T07:55:00.314353+0000","last_peered":"2024-09-21T07:55:00.314353+0000","last_clean":"2024-09-21T07:55:00.314353+0000","last_became_active":"2024-09-21T07:35:26.870749+0000","last_became_peered":"2024-09-21T07:35:26.870749+0000","last_unstale":"2024-09-21T07:55:00.314353+0000","last_undegraded":"2024-09-21T07:55:00.314353+0000","last_fullsized":"2024-09-21T07:55:00.314353+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:55.367877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010159570000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":456,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.175411+0000","last_change":"2024-09-21T07:34:24.597274+0000","last_active":"2024-09-21T07:55:00.175411+0000","last_peered":"2024-09-21T07:55:00.175411+0000","last_clean":"2024-09-21T07:55:00.175411+0000","last_became_active":"2024-09-21T07:34:23.279560+0000","last_became_peered":"2024-09-21T07:34:23.279560+0000","last_unstale":"2024-09-21T07:55:00.175411+0000","last_undegraded":"2024-09-21T07:55:00.175411+0000","last_fullsized":"2024-09-21T07:55:00.175411+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:15.845265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059699799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":407,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.631402+0000","last_change":"2024-09-21T07:35:27.874387+0000","last_active":"2024-09-21T07:54:59.631402+0000","last_peered":"2024-09-21T07:54:59.631402+0000","last_clean":"2024-09-21T07:54:59.631402+0000","last_became_active":"2024-09-21T07:35:27.873277+0000","last_became_peered":"2024-09-21T07:35:27.873277+0000","last_unstale":"2024-09-21T07:54:59.631402+0000","last_undegraded":"2024-09-21T07:54:59.631402+0000","last_fullsized":"2024-09-21T07:54:59.631402+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:04.581463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":406,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.175552+0000","last_change":"2024-09-21T07:35:29.387094+0000","last_active":"2024-09-21T07:55:00.175552+0000","last_peered":"2024-09-21T07:55:00.175552+0000","last_clean":"2024-09-21T07:55:00.175552+0000","last_became_active":"2024-09-21T07:35:26.874757+0000","last_became_peered":"2024-09-21T07:35:26.874757+0000","last_unstale":"2024-09-21T07:55:00.175552+0000","last_undegraded":"2024-09-21T07:55:00.175552+0000","last_fullsized":"2024-09-21T07:55:00.175552+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:33.542750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061065700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":463,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.313869+0000","last_change":"2024-09-21T07:34:24.259906+0000","last_active":"2024-09-21T07:55:00.313869+0000","last_peered":"2024-09-21T07:55:00.313869+0000","last_clean":"2024-09-21T07:55:00.313869+0000","last_became_active":"2024-09-21T07:34:23.635282+0000","last_became_peered":"2024-09-21T07:34:23.635282+0000","last_unstale":"2024-09-21T07:55:00.313869+0000","last_undegraded":"2024-09-21T07:55:00.313869+0000","last_fullsized":"2024-09-21T07:55:00.313869+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:00.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001527664,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":407,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.634628+0000","last_change":"2024-09-21T07:35:27.873645+0000","last_active":"2024-09-21T07:54:59.634628+0000","last_peered":"2024-09-21T07:54:59.634628+0000","last_clean":"2024-09-21T07:54:59.634628+0000","last_became_active":"2024-09-21T07:35:27.873245+0000","last_became_peered":"2024-09-21T07:35:27.873245+0000","last_unstale":"2024-09-21T07:54:59.634628+0000","last_undegraded":"2024-09-21T07:54:59.634628+0000","last_fullsized":"2024-09-21T07:54:59.634628+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:55.090953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":413,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.633609+0000","last_change":"2024-09-21T07:35:28.861744+0000","last_active":"2024-09-21T07:54:59.633609+0000","last_peered":"2024-09-21T07:54:59.633609+0000","last_clean":"2024-09-21T07:54:59.633609+0000","last_became_active":"2024-09-21T07:35:26.876462+0000","last_became_peered":"2024-09-21T07:35:26.876462+0000","last_unstale":"2024-09-21T07:54:59.633609+0000","last_undegraded":"2024-09-21T07:54:59.633609+0000","last_fullsized":"2024-09-21T07:54:59.633609+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:48.632568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012020920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":464,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.630668+0000","last_change":"2024-09-21T07:34:25.085454+0000","last_active":"2024-09-21T07:54:59.630668+0000","last_peered":"2024-09-21T07:54:59.630668+0000","last_clean":"2024-09-21T07:54:59.630668+0000","last_became_active":"2024-09-21T07:34:23.279205+0000","last_became_peered":"2024-09-21T07:34:23.279205+0000","last_unstale":"2024-09-21T07:54:59.630668+0000","last_undegraded":"2024-09-21T07:54:59.630668+0000","last_fullsized":"2024-09-21T07:54:59.630668+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:54.231666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095320599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":405,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.314238+0000","last_change":"2024-09-21T07:35:27.861373+0000","last_active":"2024-09-21T07:55:00.314238+0000","last_peered":"2024-09-21T07:55:00.314238+0000","last_clean":"2024-09-21T07:55:00.314238+0000","last_became_active":"2024-09-21T07:35:27.861016+0000","last_became_peered":"2024-09-21T07:35:27.861016+0000","last_unstale":"2024-09-21T07:55:00.314238+0000","last_undegraded":"2024-09-21T07:55:00.314238+0000","last_fullsized":"2024-09-21T07:55:00.314238+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:56.043763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":413,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.311346+0000","last_change":"2024-09-21T07:35:29.053677+0000","last_active":"2024-09-21T07:55:00.311346+0000","last_peered":"2024-09-21T07:55:00.311346+0000","last_clean":"2024-09-21T07:55:00.311346+0000","last_became_active":"2024-09-21T07:35:26.875496+0000","last_became_peered":"2024-09-21T07:35:26.875496+0000","last_unstale":"2024-09-21T07:55:00.311346+0000","last_undegraded":"2024-09-21T07:55:00.311346+0000","last_fullsized":"2024-09-21T07:55:00.311346+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:04.850784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065859000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":465,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.310605+0000","last_change":"2024-09-21T07:34:25.028521+0000","last_active":"2024-09-21T07:55:00.310605+0000","last_peered":"2024-09-21T07:55:00.310605+0000","last_clean":"2024-09-21T07:55:00.310605+0000","last_became_active":"2024-09-21T07:34:23.635467+0000","last_became_peered":"2024-09-21T07:34:23.635467+0000","last_unstale":"2024-09-21T07:55:00.310605+0000","last_undegraded":"2024-09-21T07:55:00.310605+0000","last_fullsized":"2024-09-21T07:55:00.310605+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:33.729123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000660558,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":405,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.633153+0000","last_change":"2024-09-21T07:35:27.871201+0000","last_active":"2024-09-21T07:54:59.633153+0000","last_peered":"2024-09-21T07:54:59.633153+0000","last_clean":"2024-09-21T07:54:59.633153+0000","last_became_active":"2024-09-21T07:35:27.870816+0000","last_became_peered":"2024-09-21T07:35:27.870816+0000","last_unstale":"2024-09-21T07:54:59.633153+0000","last_undegraded":"2024-09-21T07:54:59.633153+0000","last_fullsized":"2024-09-21T07:54:59.633153+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:04.787450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":412,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.314136+0000","last_change":"2024-09-21T07:35:29.491874+0000","last_active":"2024-09-21T07:55:00.314136+0000","last_peered":"2024-09-21T07:55:00.314136+0000","last_clean":"2024-09-21T07:55:00.314136+0000","last_became_active":"2024-09-21T07:35:26.866381+0000","last_became_peered":"2024-09-21T07:35:26.866381+0000","last_unstale":"2024-09-21T07:55:00.314136+0000","last_undegraded":"2024-09-21T07:55:00.314136+0000","last_fullsized":"2024-09-21T07:55:00.314136+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:36.404647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073235399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":468,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.313603+0000","last_change":"2024-09-21T07:34:24.260046+0000","last_active":"2024-09-21T07:55:00.313603+0000","last_peered":"2024-09-21T07:55:00.313603+0000","last_clean":"2024-09-21T07:55:00.313603+0000","last_became_active":"2024-09-21T07:34:23.273431+0000","last_became_peered":"2024-09-21T07:34:23.273431+0000","last_unstale":"2024-09-21T07:55:00.313603+0000","last_undegraded":"2024-09-21T07:55:00.313603+0000","last_fullsized":"2024-09-21T07:55:00.313603+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:18.876057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019803310000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":405,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.632132+0000","last_change":"2024-09-21T07:35:27.871736+0000","last_active":"2024-09-21T07:54:59.632132+0000","last_peered":"2024-09-21T07:54:59.632132+0000","last_clean":"2024-09-21T07:54:59.632132+0000","last_became_active":"2024-09-21T07:35:27.871232+0000","last_became_peered":"2024-09-21T07:35:27.871232+0000","last_unstale":"2024-09-21T07:54:59.632132+0000","last_undegraded":"2024-09-21T07:54:59.632132+0000","last_fullsized":"2024-09-21T07:54:59.632132+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:04.419301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":414,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.311567+0000","last_change":"2024-09-21T07:35:29.491246+0000","last_active":"2024-09-21T07:55:00.311567+0000","last_peered":"2024-09-21T07:55:00.311567+0000","last_clean":"2024-09-21T07:55:00.311567+0000","last_became_active":"2024-09-21T07:35:26.866783+0000","last_became_peered":"2024-09-21T07:35:26.866783+0000","last_unstale":"2024-09-21T07:55:00.311567+0000","last_undegraded":"2024-09-21T07:55:00.311567+0000","last_fullsized":"2024-09-21T07:55:00.311567+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:28.881914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098474099999999992,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":465,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.311456+0000","last_change":"2024-09-21T07:34:25.028640+0000","last_active":"2024-09-21T07:55:00.311456+0000","last_peered":"2024-09-21T07:55:00.311456+0000","last_clean":"2024-09-21T07:55:00.311456+0000","last_became_active":"2024-09-21T07:34:23.260950+0000","last_became_peered":"2024-09-21T07:34:23.260950+0000","last_unstale":"2024-09-21T07:55:00.311456+0000","last_undegraded":"2024-09-21T07:55:00.311456+0000","last_fullsized":"2024-09-21T07:55:00.311456+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:23:42.121398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055135099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":408,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.319249+0000","last_change":"2024-09-21T07:35:27.872027+0000","last_active":"2024-09-21T07:55:00.319249+0000","last_peered":"2024-09-21T07:55:00.319249+0000","last_clean":"2024-09-21T07:55:00.319249+0000","last_became_active":"2024-09-21T07:35:27.870426+0000","last_became_peered":"2024-09-21T07:35:27.870426+0000","last_unstale":"2024-09-21T07:55:00.319249+0000","last_undegraded":"2024-09-21T07:55:00.319249+0000","last_fullsized":"2024-09-21T07:55:00.319249+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":414,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.312990+0000","last_change":"2024-09-21T07:35:29.491008+0000","last_active":"2024-09-21T07:55:00.312990+0000","last_peered":"2024-09-21T07:55:00.312990+0000","last_clean":"2024-09-21T07:55:00.312990+0000","last_became_active":"2024-09-21T07:35:26.866750+0000","last_became_peered":"2024-09-21T07:35:26.866750+0000","last_unstale":"2024-09-21T07:55:00.312990+0000","last_undegraded":"2024-09-21T07:55:00.312990+0000","last_fullsized":"2024-09-21T07:55:00.312990+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:25.490406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083529700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":464,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.315758+0000","last_change":"2024-09-21T07:34:24.260665+0000","last_active":"2024-09-21T07:55:00.315758+0000","last_peered":"2024-09-21T07:55:00.315758+0000","last_clean":"2024-09-21T07:55:00.315758+0000","last_became_active":"2024-09-21T07:34:23.635909+0000","last_became_peered":"2024-09-21T07:34:23.635909+0000","last_unstale":"2024-09-21T07:55:00.315758+0000","last_undegraded":"2024-09-21T07:55:00.315758+0000","last_fullsized":"2024-09-21T07:55:00.315758+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:44.536885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":400,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.173356+0000","last_change":"2024-09-21T07:35:27.874772+0000","last_active":"2024-09-21T07:55:00.173356+0000","last_peered":"2024-09-21T07:55:00.173356+0000","last_clean":"2024-09-21T07:55:00.173356+0000","last_became_active":"2024-09-21T07:35:27.874434+0000","last_became_peered":"2024-09-21T07:35:27.874434+0000","last_unstale":"2024-09-21T07:55:00.173356+0000","last_undegraded":"2024-09-21T07:55:00.173356+0000","last_fullsized":"2024-09-21T07:55:00.173356+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:31.998755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":428,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.632463+0000","last_change":"2024-09-21T07:35:28.859153+0000","last_active":"2024-09-21T07:54:59.632463+0000","last_peered":"2024-09-21T07:54:59.632463+0000","last_clean":"2024-09-21T07:54:59.632463+0000","last_became_active":"2024-09-21T07:35:26.874428+0000","last_became_peered":"2024-09-21T07:35:26.874428+0000","last_unstale":"2024-09-21T07:54:59.632463+0000","last_undegraded":"2024-09-21T07:54:59.632463+0000","last_fullsized":"2024-09-21T07:54:59.632463+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:04.487621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048557299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":480,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.630653+0000","last_change":"2024-09-21T07:34:24.538845+0000","last_active":"2024-09-21T07:54:59.630653+0000","last_peered":"2024-09-21T07:54:59.630653+0000","last_clean":"2024-09-21T07:54:59.630653+0000","last_became_active":"2024-09-21T07:34:23.636372+0000","last_became_peered":"2024-09-21T07:34:23.636372+0000","last_unstale":"2024-09-21T07:54:59.630653+0000","last_undegraded":"2024-09-21T07:54:59.630653+0000","last_fullsized":"2024-09-21T07:54:59.630653+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:29.393813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841451,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":409,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.312263+0000","last_change":"2024-09-21T07:35:27.877801+0000","last_active":"2024-09-21T07:55:00.312263+0000","last_peered":"2024-09-21T07:55:00.312263+0000","last_clean":"2024-09-21T07:55:00.312263+0000","last_became_active":"2024-09-21T07:35:27.877464+0000","last_became_peered":"2024-09-21T07:35:27.877464+0000","last_unstale":"2024-09-21T07:55:00.312263+0000","last_undegraded":"2024-09-21T07:55:00.312263+0000","last_fullsized":"2024-09-21T07:55:00.312263+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:08.814929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":413,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.632257+0000","last_change":"2024-09-21T07:35:28.863732+0000","last_active":"2024-09-21T07:54:59.632257+0000","last_peered":"2024-09-21T07:54:59.632257+0000","last_clean":"2024-09-21T07:54:59.632257+0000","last_became_active":"2024-09-21T07:35:26.877392+0000","last_became_peered":"2024-09-21T07:35:26.877392+0000","last_unstale":"2024-09-21T07:54:59.632257+0000","last_undegraded":"2024-09-21T07:54:59.632257+0000","last_fullsized":"2024-09-21T07:54:59.632257+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071425700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":467,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.314344+0000","last_change":"2024-09-21T07:34:24.259686+0000","last_active":"2024-09-21T07:55:00.314344+0000","last_peered":"2024-09-21T07:55:00.314344+0000","last_clean":"2024-09-21T07:55:00.314344+0000","last_became_active":"2024-09-21T07:34:23.274405+0000","last_became_peered":"2024-09-21T07:34:23.274405+0000","last_unstale":"2024-09-21T07:55:00.314344+0000","last_undegraded":"2024-09-21T07:55:00.314344+0000","last_fullsized":"2024-09-21T07:55:00.314344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:26.304693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022009920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":400,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.175037+0000","last_change":"2024-09-21T07:35:27.867336+0000","last_active":"2024-09-21T07:55:00.175037+0000","last_peered":"2024-09-21T07:55:00.175037+0000","last_clean":"2024-09-21T07:55:00.175037+0000","last_became_active":"2024-09-21T07:35:27.867015+0000","last_became_peered":"2024-09-21T07:35:27.867015+0000","last_unstale":"2024-09-21T07:55:00.175037+0000","last_undegraded":"2024-09-21T07:55:00.175037+0000","last_fullsized":"2024-09-21T07:55:00.175037+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:59.214139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":411,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.315550+0000","last_change":"2024-09-21T07:35:29.493137+0000","last_active":"2024-09-21T07:55:00.315550+0000","last_peered":"2024-09-21T07:55:00.315550+0000","last_clean":"2024-09-21T07:55:00.315550+0000","last_became_active":"2024-09-21T07:35:26.872230+0000","last_became_peered":"2024-09-21T07:35:26.872230+0000","last_unstale":"2024-09-21T07:55:00.315550+0000","last_undegraded":"2024-09-21T07:55:00.315550+0000","last_fullsized":"2024-09-21T07:55:00.315550+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:24.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001197449,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":464,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.311160+0000","last_change":"2024-09-21T07:34:24.258754+0000","last_active":"2024-09-21T07:55:00.311160+0000","last_peered":"2024-09-21T07:55:00.311160+0000","last_clean":"2024-09-21T07:55:00.311160+0000","last_became_active":"2024-09-21T07:34:23.272657+0000","last_became_peered":"2024-09-21T07:34:23.272657+0000","last_unstale":"2024-09-21T07:55:00.311160+0000","last_undegraded":"2024-09-21T07:55:00.311160+0000","last_fullsized":"2024-09-21T07:55:00.311160+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:05.477099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067143200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":406,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.632348+0000","last_change":"2024-09-21T07:35:27.876021+0000","last_active":"2024-09-21T07:54:59.632348+0000","last_peered":"2024-09-21T07:54:59.632348+0000","last_clean":"2024-09-21T07:54:59.632348+0000","last_became_active":"2024-09-21T07:35:27.875724+0000","last_became_peered":"2024-09-21T07:35:27.875724+0000","last_unstale":"2024-09-21T07:54:59.632348+0000","last_undegraded":"2024-09-21T07:54:59.632348+0000","last_fullsized":"2024-09-21T07:54:59.632348+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:03.022812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":411,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.313993+0000","last_change":"2024-09-21T07:35:29.492570+0000","last_active":"2024-09-21T07:55:00.313993+0000","last_peered":"2024-09-21T07:55:00.313993+0000","last_clean":"2024-09-21T07:55:00.313993+0000","last_became_active":"2024-09-21T07:35:26.874316+0000","last_became_peered":"2024-09-21T07:35:26.874316+0000","last_unstale":"2024-09-21T07:55:00.313993+0000","last_undegraded":"2024-09-21T07:55:00.313993+0000","last_fullsized":"2024-09-21T07:55:00.313993+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:16.512323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011990760000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":464,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.633378+0000","last_change":"2024-09-21T07:34:25.085232+0000","last_active":"2024-09-21T07:54:59.633378+0000","last_peered":"2024-09-21T07:54:59.633378+0000","last_clean":"2024-09-21T07:54:59.633378+0000","last_became_active":"2024-09-21T07:34:23.277807+0000","last_became_peered":"2024-09-21T07:34:23.277807+0000","last_unstale":"2024-09-21T07:54:59.633378+0000","last_undegraded":"2024-09-21T07:54:59.633378+0000","last_fullsized":"2024-09-21T07:54:59.633378+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:36.362919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086619099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":409,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.316773+0000","last_change":"2024-09-21T07:35:27.872798+0000","last_active":"2024-09-21T07:55:00.316773+0000","last_peered":"2024-09-21T07:55:00.316773+0000","last_clean":"2024-09-21T07:55:00.316773+0000","last_became_active":"2024-09-21T07:35:27.872483+0000","last_became_peered":"2024-09-21T07:35:27.872483+0000","last_unstale":"2024-09-21T07:55:00.316773+0000","last_undegraded":"2024-09-21T07:55:00.316773+0000","last_fullsized":"2024-09-21T07:55:00.316773+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:11.933469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":416,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.315971+0000","last_change":"2024-09-21T07:35:29.206723+0000","last_active":"2024-09-21T07:55:00.315971+0000","last_peered":"2024-09-21T07:55:00.315971+0000","last_clean":"2024-09-21T07:55:00.315971+0000","last_became_active":"2024-09-21T07:35:26.872377+0000","last_became_peered":"2024-09-21T07:35:26.872377+0000","last_unstale":"2024-09-21T07:55:00.315971+0000","last_undegraded":"2024-09-21T07:55:00.315971+0000","last_fullsized":"2024-09-21T07:55:00.315971+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:29.086862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083077200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":456,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.175826+0000","last_change":"2024-09-21T07:34:24.597672+0000","last_active":"2024-09-21T07:55:00.175826+0000","last_peered":"2024-09-21T07:55:00.175826+0000","last_clean":"2024-09-21T07:55:00.175826+0000","last_became_active":"2024-09-21T07:34:23.267770+0000","last_became_peered":"2024-09-21T07:34:23.267770+0000","last_unstale":"2024-09-21T07:55:00.175826+0000","last_undegraded":"2024-09-21T07:55:00.175826+0000","last_fullsized":"2024-09-21T07:55:00.175826+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:51.629476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086378500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":409,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.316038+0000","last_change":"2024-09-21T07:35:27.870121+0000","last_active":"2024-09-21T07:55:00.316038+0000","last_peered":"2024-09-21T07:55:00.316038+0000","last_clean":"2024-09-21T07:55:00.316038+0000","last_became_active":"2024-09-21T07:35:27.869811+0000","last_became_peered":"2024-09-21T07:35:27.869811+0000","last_unstale":"2024-09-21T07:55:00.316038+0000","last_undegraded":"2024-09-21T07:55:00.316038+0000","last_fullsized":"2024-09-21T07:55:00.316038+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:31.778461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":414,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.634294+0000","last_change":"2024-09-21T07:35:28.860699+0000","last_active":"2024-09-21T07:54:59.634294+0000","last_peered":"2024-09-21T07:54:59.634294+0000","last_clean":"2024-09-21T07:54:59.634294+0000","last_became_active":"2024-09-21T07:35:26.876779+0000","last_became_peered":"2024-09-21T07:35:26.876779+0000","last_unstale":"2024-09-21T07:54:59.634294+0000","last_undegraded":"2024-09-21T07:54:59.634294+0000","last_fullsized":"2024-09-21T07:54:59.634294+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:28.286403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000913188,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":466,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.634607+0000","last_change":"2024-09-21T07:34:24.260181+0000","last_active":"2024-09-21T07:54:59.634607+0000","last_peered":"2024-09-21T07:54:59.634607+0000","last_clean":"2024-09-21T07:54:59.634607+0000","last_became_active":"2024-09-21T07:34:23.275580+0000","last_became_peered":"2024-09-21T07:34:23.275580+0000","last_unstale":"2024-09-21T07:54:59.634607+0000","last_undegraded":"2024-09-21T07:54:59.634607+0000","last_fullsized":"2024-09-21T07:54:59.634607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:49.857622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014706980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":407,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.314738+0000","last_change":"2024-09-21T07:35:27.875393+0000","last_active":"2024-09-21T07:55:00.314738+0000","last_peered":"2024-09-21T07:55:00.314738+0000","last_clean":"2024-09-21T07:55:00.314738+0000","last_became_active":"2024-09-21T07:35:27.874671+0000","last_became_peered":"2024-09-21T07:35:27.874671+0000","last_unstale":"2024-09-21T07:55:00.314738+0000","last_undegraded":"2024-09-21T07:55:00.314738+0000","last_fullsized":"2024-09-21T07:55:00.314738+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":416,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.312471+0000","last_change":"2024-09-21T07:35:29.206296+0000","last_active":"2024-09-21T07:55:00.312471+0000","last_peered":"2024-09-21T07:55:00.312471+0000","last_clean":"2024-09-21T07:55:00.312471+0000","last_became_active":"2024-09-21T07:35:26.874575+0000","last_became_peered":"2024-09-21T07:35:26.874575+0000","last_unstale":"2024-09-21T07:55:00.312471+0000","last_undegraded":"2024-09-21T07:55:00.312471+0000","last_fullsized":"2024-09-21T07:55:00.312471+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:22.771154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061011599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":467,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.315376+0000","last_change":"2024-09-21T07:34:24.259112+0000","last_active":"2024-09-21T07:55:00.315376+0000","last_peered":"2024-09-21T07:55:00.315376+0000","last_clean":"2024-09-21T07:55:00.315376+0000","last_became_active":"2024-09-21T07:34:23.635444+0000","last_became_peered":"2024-09-21T07:34:23.635444+0000","last_unstale":"2024-09-21T07:55:00.315376+0000","last_undegraded":"2024-09-21T07:55:00.315376+0000","last_fullsized":"2024-09-21T07:55:00.315376+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:46.255385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001952636,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":407,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.312824+0000","last_change":"2024-09-21T07:35:27.878356+0000","last_active":"2024-09-21T07:55:00.312824+0000","last_peered":"2024-09-21T07:55:00.312824+0000","last_clean":"2024-09-21T07:55:00.312824+0000","last_became_active":"2024-09-21T07:35:27.877982+0000","last_became_peered":"2024-09-21T07:35:27.877982+0000","last_unstale":"2024-09-21T07:55:00.312824+0000","last_undegraded":"2024-09-21T07:55:00.312824+0000","last_fullsized":"2024-09-21T07:55:00.312824+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":414,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.310325+0000","last_change":"2024-09-21T07:35:29.053131+0000","last_active":"2024-09-21T07:55:00.310325+0000","last_peered":"2024-09-21T07:55:00.310325+0000","last_clean":"2024-09-21T07:55:00.310325+0000","last_became_active":"2024-09-21T07:35:26.870516+0000","last_became_peered":"2024-09-21T07:35:26.870516+0000","last_unstale":"2024-09-21T07:55:00.310325+0000","last_undegraded":"2024-09-21T07:55:00.310325+0000","last_fullsized":"2024-09-21T07:55:00.310325+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:10.924491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070705699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":480,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.632868+0000","last_change":"2024-09-21T07:34:24.539152+0000","last_active":"2024-09-21T07:54:59.632868+0000","last_peered":"2024-09-21T07:54:59.632868+0000","last_clean":"2024-09-21T07:54:59.632868+0000","last_became_active":"2024-09-21T07:34:23.259573+0000","last_became_peered":"2024-09-21T07:34:23.259573+0000","last_unstale":"2024-09-21T07:54:59.632868+0000","last_undegraded":"2024-09-21T07:54:59.632868+0000","last_fullsized":"2024-09-21T07:54:59.632868+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:26.986900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058108400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":406,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.310193+0000","last_change":"2024-09-21T07:35:27.873726+0000","last_active":"2024-09-21T07:55:00.310193+0000","last_peered":"2024-09-21T07:55:00.310193+0000","last_clean":"2024-09-21T07:55:00.310193+0000","last_became_active":"2024-09-21T07:35:27.871421+0000","last_became_peered":"2024-09-21T07:35:27.871421+0000","last_unstale":"2024-09-21T07:55:00.310193+0000","last_undegraded":"2024-09-21T07:55:00.310193+0000","last_fullsized":"2024-09-21T07:55:00.310193+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:00.168915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":414,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.634353+0000","last_change":"2024-09-21T07:35:28.861032+0000","last_active":"2024-09-21T07:54:59.634353+0000","last_peered":"2024-09-21T07:54:59.634353+0000","last_clean":"2024-09-21T07:54:59.634353+0000","last_became_active":"2024-09-21T07:35:26.873149+0000","last_became_peered":"2024-09-21T07:35:26.873149+0000","last_unstale":"2024-09-21T07:54:59.634353+0000","last_undegraded":"2024-09-21T07:54:59.634353+0000","last_fullsized":"2024-09-21T07:54:59.634353+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:59.435981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011988369999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":463,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.311971+0000","last_change":"2024-09-21T07:34:24.259902+0000","last_active":"2024-09-21T07:55:00.311971+0000","last_peered":"2024-09-21T07:55:00.311971+0000","last_clean":"2024-09-21T07:55:00.311971+0000","last_became_active":"2024-09-21T07:34:23.271902+0000","last_became_peered":"2024-09-21T07:34:23.271902+0000","last_unstale":"2024-09-21T07:55:00.311971+0000","last_undegraded":"2024-09-21T07:55:00.311971+0000","last_fullsized":"2024-09-21T07:55:00.311971+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:59.439525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001686358,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":405,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.631813+0000","last_change":"2024-09-21T07:35:27.872637+0000","last_active":"2024-09-21T07:54:59.631813+0000","last_peered":"2024-09-21T07:54:59.631813+0000","last_clean":"2024-09-21T07:54:59.631813+0000","last_became_active":"2024-09-21T07:35:27.872149+0000","last_became_peered":"2024-09-21T07:35:27.872149+0000","last_unstale":"2024-09-21T07:54:59.631813+0000","last_undegraded":"2024-09-21T07:54:59.631813+0000","last_fullsized":"2024-09-21T07:54:59.631813+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:27.697872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":411,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.314113+0000","last_change":"2024-09-21T07:35:29.493362+0000","last_active":"2024-09-21T07:55:00.314113+0000","last_peered":"2024-09-21T07:55:00.314113+0000","last_clean":"2024-09-21T07:55:00.314113+0000","last_became_active":"2024-09-21T07:35:26.873018+0000","last_became_peered":"2024-09-21T07:35:26.873018+0000","last_unstale":"2024-09-21T07:55:00.314113+0000","last_undegraded":"2024-09-21T07:55:00.314113+0000","last_fullsized":"2024-09-21T07:55:00.314113+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:02.460557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012496510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":466,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.314038+0000","last_change":"2024-09-21T07:34:24.259662+0000","last_active":"2024-09-21T07:55:00.314038+0000","last_peered":"2024-09-21T07:55:00.314038+0000","last_clean":"2024-09-21T07:55:00.314038+0000","last_became_active":"2024-09-21T07:34:23.278812+0000","last_became_peered":"2024-09-21T07:34:23.278812+0000","last_unstale":"2024-09-21T07:55:00.314038+0000","last_undegraded":"2024-09-21T07:55:00.314038+0000","last_fullsized":"2024-09-21T07:55:00.314038+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:49.829974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356463,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":398,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.174789+0000","last_change":"2024-09-21T07:35:27.868448+0000","last_active":"2024-09-21T07:55:00.174789+0000","last_peered":"2024-09-21T07:55:00.174789+0000","last_clean":"2024-09-21T07:55:00.174789+0000","last_became_active":"2024-09-21T07:35:27.868105+0000","last_became_peered":"2024-09-21T07:35:27.868105+0000","last_unstale":"2024-09-21T07:55:00.174789+0000","last_undegraded":"2024-09-21T07:55:00.174789+0000","last_fullsized":"2024-09-21T07:55:00.174789+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:41.468702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":409,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.315283+0000","last_change":"2024-09-21T07:35:29.492974+0000","last_active":"2024-09-21T07:55:00.315283+0000","last_peered":"2024-09-21T07:55:00.315283+0000","last_clean":"2024-09-21T07:55:00.315283+0000","last_became_active":"2024-09-21T07:35:26.872165+0000","last_became_peered":"2024-09-21T07:35:26.872165+0000","last_unstale":"2024-09-21T07:55:00.315283+0000","last_undegraded":"2024-09-21T07:55:00.315283+0000","last_fullsized":"2024-09-21T07:55:00.315283+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:59.912520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013795179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":463,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.630639+0000","last_change":"2024-09-21T07:34:25.084861+0000","last_active":"2024-09-21T07:54:59.630639+0000","last_peered":"2024-09-21T07:54:59.630639+0000","last_clean":"2024-09-21T07:54:59.630639+0000","last_became_active":"2024-09-21T07:34:23.277708+0000","last_became_peered":"2024-09-21T07:34:23.277708+0000","last_unstale":"2024-09-21T07:54:59.630639+0000","last_undegraded":"2024-09-21T07:54:59.630639+0000","last_fullsized":"2024-09-21T07:54:59.630639+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:40.816488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000597566,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":398,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.175249+0000","last_change":"2024-09-21T07:35:27.874640+0000","last_active":"2024-09-21T07:55:00.175249+0000","last_peered":"2024-09-21T07:55:00.175249+0000","last_clean":"2024-09-21T07:55:00.175249+0000","last_became_active":"2024-09-21T07:35:27.874265+0000","last_became_peered":"2024-09-21T07:35:27.874265+0000","last_unstale":"2024-09-21T07:55:00.175249+0000","last_undegraded":"2024-09-21T07:55:00.175249+0000","last_fullsized":"2024-09-21T07:55:00.175249+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:45.754654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":414,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.318433+0000","last_change":"2024-09-21T07:35:29.207392+0000","last_active":"2024-09-21T07:55:00.318433+0000","last_peered":"2024-09-21T07:55:00.318433+0000","last_clean":"2024-09-21T07:55:00.318433+0000","last_became_active":"2024-09-21T07:35:26.870917+0000","last_became_peered":"2024-09-21T07:35:26.870917+0000","last_unstale":"2024-09-21T07:55:00.318433+0000","last_undegraded":"2024-09-21T07:55:00.318433+0000","last_fullsized":"2024-09-21T07:55:00.318433+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:59.159937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059198499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":467,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.313609+0000","last_change":"2024-09-21T07:34:24.258709+0000","last_active":"2024-09-21T07:55:00.313609+0000","last_peered":"2024-09-21T07:55:00.313609+0000","last_clean":"2024-09-21T07:55:00.313609+0000","last_became_active":"2024-09-21T07:34:23.279042+0000","last_became_peered":"2024-09-21T07:34:23.279042+0000","last_unstale":"2024-09-21T07:55:00.313609+0000","last_undegraded":"2024-09-21T07:55:00.313609+0000","last_fullsized":"2024-09-21T07:55:00.313609+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:01.762117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019282100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":404,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.633291+0000","last_change":"2024-09-21T07:35:27.870260+0000","last_active":"2024-09-21T07:54:59.633291+0000","last_peered":"2024-09-21T07:54:59.633291+0000","last_clean":"2024-09-21T07:54:59.633291+0000","last_became_active":"2024-09-21T07:35:27.869448+0000","last_became_peered":"2024-09-21T07:35:27.869448+0000","last_unstale":"2024-09-21T07:54:59.633291+0000","last_undegraded":"2024-09-21T07:54:59.633291+0000","last_fullsized":"2024-09-21T07:54:59.633291+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:16.946790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":405,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.175086+0000","last_change":"2024-09-21T07:35:29.386367+0000","last_active":"2024-09-21T07:55:00.175086+0000","last_peered":"2024-09-21T07:55:00.175086+0000","last_clean":"2024-09-21T07:55:00.175086+0000","last_became_active":"2024-09-21T07:35:26.879238+0000","last_became_peered":"2024-09-21T07:35:26.879238+0000","last_unstale":"2024-09-21T07:55:00.175086+0000","last_undegraded":"2024-09-21T07:55:00.175086+0000","last_fullsized":"2024-09-21T07:55:00.175086+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:16.528260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092223100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":463,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.632428+0000","last_change":"2024-09-21T07:34:25.085635+0000","last_active":"2024-09-21T07:54:59.632428+0000","last_peered":"2024-09-21T07:54:59.632428+0000","last_clean":"2024-09-21T07:54:59.632428+0000","last_became_active":"2024-09-21T07:34:23.273566+0000","last_became_peered":"2024-09-21T07:34:23.273566+0000","last_unstale":"2024-09-21T07:54:59.632428+0000","last_undegraded":"2024-09-21T07:54:59.632428+0000","last_fullsized":"2024-09-21T07:54:59.632428+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:14.142272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079897300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":404,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.633141+0000","last_change":"2024-09-21T07:35:27.878728+0000","last_active":"2024-09-21T07:54:59.633141+0000","last_peered":"2024-09-21T07:54:59.633141+0000","last_clean":"2024-09-21T07:54:59.633141+0000","last_became_active":"2024-09-21T07:35:27.878335+0000","last_became_peered":"2024-09-21T07:35:27.878335+0000","last_unstale":"2024-09-21T07:54:59.633141+0000","last_undegraded":"2024-09-21T07:54:59.633141+0000","last_fullsized":"2024-09-21T07:54:59.633141+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:25:37.731771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":426,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.630524+0000","last_change":"2024-09-21T07:35:28.858418+0000","last_active":"2024-09-21T07:54:59.630524+0000","last_peered":"2024-09-21T07:54:59.630524+0000","last_clean":"2024-09-21T07:54:59.630524+0000","last_became_active":"2024-09-21T07:35:26.876944+0000","last_became_peered":"2024-09-21T07:35:26.876944+0000","last_unstale":"2024-09-21T07:54:59.630524+0000","last_undegraded":"2024-09-21T07:54:59.630524+0000","last_fullsized":"2024-09-21T07:54:59.630524+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:04.514304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090862799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":455,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.176081+0000","last_change":"2024-09-21T07:34:24.597926+0000","last_active":"2024-09-21T07:55:00.176081+0000","last_peered":"2024-09-21T07:55:00.176081+0000","last_clean":"2024-09-21T07:55:00.176081+0000","last_became_active":"2024-09-21T07:34:23.268926+0000","last_became_peered":"2024-09-21T07:34:23.268926+0000","last_unstale":"2024-09-21T07:55:00.176081+0000","last_undegraded":"2024-09-21T07:55:00.176081+0000","last_fullsized":"2024-09-21T07:55:00.176081+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:46.190752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101851,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":400,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.175363+0000","last_change":"2024-09-21T07:35:27.877426+0000","last_active":"2024-09-21T07:55:00.175363+0000","last_peered":"2024-09-21T07:55:00.175363+0000","last_clean":"2024-09-21T07:55:00.175363+0000","last_became_active":"2024-09-21T07:35:27.877050+0000","last_became_peered":"2024-09-21T07:35:27.877050+0000","last_unstale":"2024-09-21T07:55:00.175363+0000","last_undegraded":"2024-09-21T07:55:00.175363+0000","last_fullsized":"2024-09-21T07:55:00.175363+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:36.268334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":412,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.631943+0000","last_change":"2024-09-21T07:35:28.862273+0000","last_active":"2024-09-21T07:54:59.631943+0000","last_peered":"2024-09-21T07:54:59.631943+0000","last_clean":"2024-09-21T07:54:59.631943+0000","last_became_active":"2024-09-21T07:35:26.874011+0000","last_became_peered":"2024-09-21T07:35:26.874011+0000","last_unstale":"2024-09-21T07:54:59.631943+0000","last_undegraded":"2024-09-21T07:54:59.631943+0000","last_fullsized":"2024-09-21T07:54:59.631943+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:11.986950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071644799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":465,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.637034+0000","last_change":"2024-09-21T07:34:24.259794+0000","last_active":"2024-09-21T07:54:59.637034+0000","last_peered":"2024-09-21T07:54:59.637034+0000","last_clean":"2024-09-21T07:54:59.637034+0000","last_became_active":"2024-09-21T07:34:23.265679+0000","last_became_peered":"2024-09-21T07:34:23.265679+0000","last_unstale":"2024-09-21T07:54:59.637034+0000","last_undegraded":"2024-09-21T07:54:59.637034+0000","last_fullsized":"2024-09-21T07:54:59.637034+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:02.575796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001298518,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":409,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.317882+0000","last_change":"2024-09-21T07:35:27.877725+0000","last_active":"2024-09-21T07:55:00.317882+0000","last_peered":"2024-09-21T07:55:00.317882+0000","last_clean":"2024-09-21T07:55:00.317882+0000","last_became_active":"2024-09-21T07:35:27.877398+0000","last_became_peered":"2024-09-21T07:35:27.877398+0000","last_unstale":"2024-09-21T07:55:00.317882+0000","last_undegraded":"2024-09-21T07:55:00.317882+0000","last_fullsized":"2024-09-21T07:55:00.317882+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:05.859762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":413,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.633091+0000","last_change":"2024-09-21T07:35:28.860612+0000","last_active":"2024-09-21T07:54:59.633091+0000","last_peered":"2024-09-21T07:54:59.633091+0000","last_clean":"2024-09-21T07:54:59.633091+0000","last_became_active":"2024-09-21T07:35:26.876562+0000","last_became_peered":"2024-09-21T07:35:26.876562+0000","last_unstale":"2024-09-21T07:54:59.633091+0000","last_undegraded":"2024-09-21T07:54:59.633091+0000","last_fullsized":"2024-09-21T07:54:59.633091+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:06.824331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012252470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":463,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.315655+0000","last_change":"2024-09-21T07:34:24.260330+0000","last_active":"2024-09-21T07:55:00.315655+0000","last_peered":"2024-09-21T07:55:00.315655+0000","last_clean":"2024-09-21T07:55:00.315655+0000","last_became_active":"2024-09-21T07:34:23.637210+0000","last_became_peered":"2024-09-21T07:34:23.637210+0000","last_unstale":"2024-09-21T07:55:00.315655+0000","last_undegraded":"2024-09-21T07:55:00.315655+0000","last_fullsized":"2024-09-21T07:55:00.315655+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.488451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001537093,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":405,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.630514+0000","last_change":"2024-09-21T07:35:27.874757+0000","last_active":"2024-09-21T07:54:59.630514+0000","last_peered":"2024-09-21T07:54:59.630514+0000","last_clean":"2024-09-21T07:54:59.630514+0000","last_became_active":"2024-09-21T07:35:27.874360+0000","last_became_peered":"2024-09-21T07:35:27.874360+0000","last_unstale":"2024-09-21T07:54:59.630514+0000","last_undegraded":"2024-09-21T07:54:59.630514+0000","last_fullsized":"2024-09-21T07:54:59.630514+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:17.646912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":410,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.314189+0000","last_change":"2024-09-21T07:35:29.493431+0000","last_active":"2024-09-21T07:55:00.314189+0000","last_peered":"2024-09-21T07:55:00.314189+0000","last_clean":"2024-09-21T07:55:00.314189+0000","last_became_active":"2024-09-21T07:35:26.873691+0000","last_became_peered":"2024-09-21T07:35:26.873691+0000","last_unstale":"2024-09-21T07:55:00.314189+0000","last_undegraded":"2024-09-21T07:55:00.314189+0000","last_fullsized":"2024-09-21T07:55:00.314189+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:51.705051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001162464,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":455,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.175180+0000","last_change":"2024-09-21T07:34:24.597286+0000","last_active":"2024-09-21T07:55:00.175180+0000","last_peered":"2024-09-21T07:55:00.175180+0000","last_clean":"2024-09-21T07:55:00.175180+0000","last_became_active":"2024-09-21T07:34:23.267133+0000","last_became_peered":"2024-09-21T07:34:23.267133+0000","last_unstale":"2024-09-21T07:55:00.175180+0000","last_undegraded":"2024-09-21T07:55:00.175180+0000","last_fullsized":"2024-09-21T07:55:00.175180+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:18.257010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":420,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.630258+0000","last_change":"2024-09-21T07:35:27.864458+0000","last_active":"2024-09-21T07:54:59.630258+0000","last_peered":"2024-09-21T07:54:59.630258+0000","last_clean":"2024-09-21T07:54:59.630258+0000","last_became_active":"2024-09-21T07:35:27.864136+0000","last_became_peered":"2024-09-21T07:35:27.864136+0000","last_unstale":"2024-09-21T07:54:59.630258+0000","last_undegraded":"2024-09-21T07:54:59.630258+0000","last_fullsized":"2024-09-21T07:54:59.630258+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:20.162240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":427,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.632799+0000","last_change":"2024-09-21T07:35:28.858784+0000","last_active":"2024-09-21T07:54:59.632799+0000","last_peered":"2024-09-21T07:54:59.632799+0000","last_clean":"2024-09-21T07:54:59.632799+0000","last_became_active":"2024-09-21T07:35:26.876890+0000","last_became_peered":"2024-09-21T07:35:26.876890+0000","last_unstale":"2024-09-21T07:54:59.632799+0000","last_undegraded":"2024-09-21T07:54:59.632799+0000","last_fullsized":"2024-09-21T07:54:59.632799+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:40.738327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064543499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":466,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.313937+0000","last_change":"2024-09-21T07:34:24.258712+0000","last_active":"2024-09-21T07:55:00.313937+0000","last_peered":"2024-09-21T07:55:00.313937+0000","last_clean":"2024-09-21T07:55:00.313937+0000","last_became_active":"2024-09-21T07:34:23.274692+0000","last_became_peered":"2024-09-21T07:34:23.274692+0000","last_unstale":"2024-09-21T07:55:00.313937+0000","last_undegraded":"2024-09-21T07:55:00.313937+0000","last_fullsized":"2024-09-21T07:55:00.313937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:43.279454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018051479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":406,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.312289+0000","last_change":"2024-09-21T07:35:27.878020+0000","last_active":"2024-09-21T07:55:00.312289+0000","last_peered":"2024-09-21T07:55:00.312289+0000","last_clean":"2024-09-21T07:55:00.312289+0000","last_became_active":"2024-09-21T07:35:27.877532+0000","last_became_peered":"2024-09-21T07:35:27.877532+0000","last_unstale":"2024-09-21T07:55:00.312289+0000","last_undegraded":"2024-09-21T07:55:00.312289+0000","last_fullsized":"2024-09-21T07:55:00.312289+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:19.756073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":413,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.633494+0000","last_change":"2024-09-21T07:35:28.861049+0000","last_active":"2024-09-21T07:54:59.633494+0000","last_peered":"2024-09-21T07:54:59.633494+0000","last_clean":"2024-09-21T07:54:59.633494+0000","last_became_active":"2024-09-21T07:35:26.876819+0000","last_became_peered":"2024-09-21T07:35:26.876819+0000","last_unstale":"2024-09-21T07:54:59.633494+0000","last_undegraded":"2024-09-21T07:54:59.633494+0000","last_fullsized":"2024-09-21T07:54:59.633494+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:19.633803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055780599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":463,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.630568+0000","last_change":"2024-09-21T07:34:25.084849+0000","last_active":"2024-09-21T07:54:59.630568+0000","last_peered":"2024-09-21T07:54:59.630568+0000","last_clean":"2024-09-21T07:54:59.630568+0000","last_became_active":"2024-09-21T07:34:23.277713+0000","last_became_peered":"2024-09-21T07:34:23.277713+0000","last_unstale":"2024-09-21T07:54:59.630568+0000","last_undegraded":"2024-09-21T07:54:59.630568+0000","last_fullsized":"2024-09-21T07:54:59.630568+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:05.811810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059079499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":406,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.312851+0000","last_change":"2024-09-21T07:35:27.884248+0000","last_active":"2024-09-21T07:55:00.312851+0000","last_peered":"2024-09-21T07:55:00.312851+0000","last_clean":"2024-09-21T07:55:00.312851+0000","last_became_active":"2024-09-21T07:35:27.883861+0000","last_became_peered":"2024-09-21T07:35:27.883861+0000","last_unstale":"2024-09-21T07:55:00.312851+0000","last_undegraded":"2024-09-21T07:55:00.312851+0000","last_fullsized":"2024-09-21T07:55:00.312851+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":415,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.318311+0000","last_change":"2024-09-21T07:35:29.207135+0000","last_active":"2024-09-21T07:55:00.318311+0000","last_peered":"2024-09-21T07:55:00.318311+0000","last_clean":"2024-09-21T07:55:00.318311+0000","last_became_active":"2024-09-21T07:35:26.872787+0000","last_became_peered":"2024-09-21T07:35:26.872787+0000","last_unstale":"2024-09-21T07:55:00.318311+0000","last_undegraded":"2024-09-21T07:55:00.318311+0000","last_fullsized":"2024-09-21T07:55:00.318311+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:27.543217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011157509999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":479,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.630798+0000","last_change":"2024-09-21T07:34:24.538610+0000","last_active":"2024-09-21T07:54:59.630798+0000","last_peered":"2024-09-21T07:54:59.630798+0000","last_clean":"2024-09-21T07:54:59.630798+0000","last_became_active":"2024-09-21T07:34:23.259700+0000","last_became_peered":"2024-09-21T07:34:23.259700+0000","last_unstale":"2024-09-21T07:54:59.630798+0000","last_undegraded":"2024-09-21T07:54:59.630798+0000","last_fullsized":"2024-09-21T07:54:59.630798+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:57.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070522399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":420,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.630936+0000","last_change":"2024-09-21T07:35:27.868024+0000","last_active":"2024-09-21T07:54:59.630936+0000","last_peered":"2024-09-21T07:54:59.630936+0000","last_clean":"2024-09-21T07:54:59.630936+0000","last_became_active":"2024-09-21T07:35:27.867339+0000","last_became_peered":"2024-09-21T07:35:27.867339+0000","last_unstale":"2024-09-21T07:54:59.630936+0000","last_undegraded":"2024-09-21T07:54:59.630936+0000","last_fullsized":"2024-09-21T07:54:59.630936+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":427,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.631794+0000","last_change":"2024-09-21T07:35:28.858505+0000","last_active":"2024-09-21T07:54:59.631794+0000","last_peered":"2024-09-21T07:54:59.631794+0000","last_clean":"2024-09-21T07:54:59.631794+0000","last_became_active":"2024-09-21T07:35:26.877320+0000","last_became_peered":"2024-09-21T07:35:26.877320+0000","last_unstale":"2024-09-21T07:54:59.631794+0000","last_undegraded":"2024-09-21T07:54:59.631794+0000","last_fullsized":"2024-09-21T07:54:59.631794+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:53.672782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244119999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":457,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.175837+0000","last_change":"2024-09-21T07:34:24.597597+0000","last_active":"2024-09-21T07:55:00.175837+0000","last_peered":"2024-09-21T07:55:00.175837+0000","last_clean":"2024-09-21T07:55:00.175837+0000","last_became_active":"2024-09-21T07:34:23.635204+0000","last_became_peered":"2024-09-21T07:34:23.635204+0000","last_unstale":"2024-09-21T07:55:00.175837+0000","last_undegraded":"2024-09-21T07:55:00.175837+0000","last_fullsized":"2024-09-21T07:55:00.175837+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:48.644789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081139199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":408,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.316004+0000","last_change":"2024-09-21T07:35:27.877503+0000","last_active":"2024-09-21T07:55:00.316004+0000","last_peered":"2024-09-21T07:55:00.316004+0000","last_clean":"2024-09-21T07:55:00.316004+0000","last_became_active":"2024-09-21T07:35:27.875775+0000","last_became_peered":"2024-09-21T07:35:27.875775+0000","last_unstale":"2024-09-21T07:55:00.316004+0000","last_undegraded":"2024-09-21T07:55:00.316004+0000","last_fullsized":"2024-09-21T07:55:00.316004+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:49.291023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":410,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.311850+0000","last_change":"2024-09-21T07:35:29.491412+0000","last_active":"2024-09-21T07:55:00.311850+0000","last_peered":"2024-09-21T07:55:00.311850+0000","last_clean":"2024-09-21T07:55:00.311850+0000","last_became_active":"2024-09-21T07:35:26.876404+0000","last_became_peered":"2024-09-21T07:35:26.876404+0000","last_unstale":"2024-09-21T07:55:00.311850+0000","last_undegraded":"2024-09-21T07:55:00.311850+0000","last_fullsized":"2024-09-21T07:55:00.311850+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063918000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":467,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.314021+0000","last_change":"2024-09-21T07:34:24.259026+0000","last_active":"2024-09-21T07:55:00.314021+0000","last_peered":"2024-09-21T07:55:00.314021+0000","last_clean":"2024-09-21T07:55:00.314021+0000","last_became_active":"2024-09-21T07:34:23.635456+0000","last_became_peered":"2024-09-21T07:34:23.635456+0000","last_unstale":"2024-09-21T07:55:00.314021+0000","last_undegraded":"2024-09-21T07:55:00.314021+0000","last_fullsized":"2024-09-21T07:55:00.314021+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:09.592752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018654419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":408,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.316264+0000","last_change":"2024-09-21T07:35:27.871984+0000","last_active":"2024-09-21T07:55:00.316264+0000","last_peered":"2024-09-21T07:55:00.316264+0000","last_clean":"2024-09-21T07:55:00.316264+0000","last_became_active":"2024-09-21T07:35:27.870336+0000","last_became_peered":"2024-09-21T07:35:27.870336+0000","last_unstale":"2024-09-21T07:55:00.316264+0000","last_undegraded":"2024-09-21T07:55:00.316264+0000","last_fullsized":"2024-09-21T07:55:00.316264+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:57.120479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":413,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.631715+0000","last_change":"2024-09-21T07:35:28.860351+0000","last_active":"2024-09-21T07:54:59.631715+0000","last_peered":"2024-09-21T07:54:59.631715+0000","last_clean":"2024-09-21T07:54:59.631715+0000","last_became_active":"2024-09-21T07:35:26.872046+0000","last_became_peered":"2024-09-21T07:35:26.872046+0000","last_unstale":"2024-09-21T07:54:59.631715+0000","last_undegraded":"2024-09-21T07:54:59.631715+0000","last_fullsized":"2024-09-21T07:54:59.631715+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:47.074327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098384399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":463,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.315458+0000","last_change":"2024-09-21T07:34:24.260300+0000","last_active":"2024-09-21T07:55:00.315458+0000","last_peered":"2024-09-21T07:55:00.315458+0000","last_clean":"2024-09-21T07:55:00.315458+0000","last_became_active":"2024-09-21T07:34:23.270128+0000","last_became_peered":"2024-09-21T07:34:23.270128+0000","last_unstale":"2024-09-21T07:55:00.315458+0000","last_undegraded":"2024-09-21T07:55:00.315458+0000","last_fullsized":"2024-09-21T07:55:00.315458+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:25.260931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00176651,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":408,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.318010+0000","last_change":"2024-09-21T07:35:27.877861+0000","last_active":"2024-09-21T07:55:00.318010+0000","last_peered":"2024-09-21T07:55:00.318010+0000","last_clean":"2024-09-21T07:55:00.318010+0000","last_became_active":"2024-09-21T07:35:27.877434+0000","last_became_peered":"2024-09-21T07:35:27.877434+0000","last_unstale":"2024-09-21T07:55:00.318010+0000","last_undegraded":"2024-09-21T07:55:00.318010+0000","last_fullsized":"2024-09-21T07:55:00.318010+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":412,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.630743+0000","last_change":"2024-09-21T07:35:28.862954+0000","last_active":"2024-09-21T07:54:59.630743+0000","last_peered":"2024-09-21T07:54:59.630743+0000","last_clean":"2024-09-21T07:54:59.630743+0000","last_became_active":"2024-09-21T07:35:26.870613+0000","last_became_peered":"2024-09-21T07:35:26.870613+0000","last_unstale":"2024-09-21T07:54:59.630743+0000","last_undegraded":"2024-09-21T07:54:59.630743+0000","last_fullsized":"2024-09-21T07:54:59.630743+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:06.901043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059842100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":464,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.314518+0000","last_change":"2024-09-21T07:34:25.028904+0000","last_active":"2024-09-21T07:55:00.314518+0000","last_peered":"2024-09-21T07:55:00.314518+0000","last_clean":"2024-09-21T07:55:00.314518+0000","last_became_active":"2024-09-21T07:34:23.260942+0000","last_became_peered":"2024-09-21T07:34:23.260942+0000","last_unstale":"2024-09-21T07:55:00.314518+0000","last_undegraded":"2024-09-21T07:55:00.314518+0000","last_fullsized":"2024-09-21T07:55:00.314518+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:22.709328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060553899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509707,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71188,"kb_used_data":3900,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94300652,"statfs":{"total":96636764160,"available":96563867648,"internally_reserved":0,"allocated":3993600,"data_stored":2441691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509709,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74768,"kb_used_data":3320,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297072,"statfs":{"total":96636764160,"available":96560201728,"internally_reserved":0,"allocated":3399680,"data_stored":1851323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509707,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52556,"kb_used_data":3320,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319284,"statfs":{"total":96636764160,"available":96582946816,"internally_reserved":0,"allocated":3399680,"data_stored":1851323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509708,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30348,"kb_used_data":3320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341492,"statfs":{"total":96636764160,"available":96605687808,"internally_reserved":0,"allocated":3399680,"data_stored":1851323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509709,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48396,"kb_used_data":3320,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323444,"statfs":{"total":96636764160,"available":96587206656,"internally_reserved":0,"allocated":3399680,"data_stored":1851323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509709,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57300,"kb_used_data":3900,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314540,"statfs":{"total":96636764160,"available":96578088960,"internally_reserved":0,"allocated":3993600,"data_stored":2441691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542412,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56720,"kb_used_data":3320,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315120,"statfs":{"total":96636764160,"available":96578682880,"internally_reserved":0,"allocated":3399680,"data_stored":1851323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542411,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74768,"kb_used_data":3320,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297072,"statfs":{"total":96636764160,"available":96560201728,"internally_reserved":0,"allocated":3399680,"data_stored":1851323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:55:04.620 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T07:55:04.620 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:55:04.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.880+0000 7f3e41a00640 1 -- 172.21.15.39:0/2759872645 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e3c0cfa20 msgr2=0x7f3e3c0cd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:04.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.880+0000 7f3e41a00640 1 --2- 172.21.15.39:0/2759872645 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e3c0cfa20 0x7f3e3c0cd5a0 secure :-1 s=READY pgs=2888 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c06dcc0 tx=0x7f3e2c02a530 comp rx=0 tx=0).stop 2024-09-21T07:55:04.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.880+0000 7f3e41a00640 1 -- 172.21.15.39:0/2759872645 shutdown_connections 2024-09-21T07:55:04.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.880+0000 7f3e41a00640 1 --2- 172.21.15.39:0/2759872645 >> v2:172.21.15.148:3300/0 conn(0x7f3e3c15cc00 0x7f3e3c15d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:04.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.880+0000 7f3e41a00640 1 --2- 172.21.15.39:0/2759872645 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e3c0cfa20 0x7f3e3c0cd5a0 unknown :-1 s=CLOSED pgs=2888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:04.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.880+0000 7f3e41a00640 1 -- 172.21.15.39:0/2759872645 >> 172.21.15.39:0/2759872645 conn(0x7f3e3c06cfc0 msgr2=0x7f3e3c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:04.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.881+0000 7f3e41a00640 1 -- 172.21.15.39:0/2759872645 shutdown_connections 2024-09-21T07:55:04.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.881+0000 7f3e41a00640 1 -- 172.21.15.39:0/2759872645 wait complete. 2024-09-21T07:55:04.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.883+0000 7f3e41a00640 1 Processor -- start 2024-09-21T07:55:04.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.883+0000 7f3e41a00640 1 -- start start 2024-09-21T07:55:04.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.884+0000 7f3e41a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3e3c0cfa20 0x7f3e3c144780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:04.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.884+0000 7f3e41a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e3c0d04b0 0x7f3e3c155e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:04.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.884+0000 7f3e41a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e3c15cc00 0x7f3e3c156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:04.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.884+0000 7f3e41a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3c06ee70 con 0x7f3e3c15cc00 2024-09-21T07:55:04.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.884+0000 7f3e41a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3c06ecf0 con 0x7f3e3c0d04b0 2024-09-21T07:55:04.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.884+0000 7f3e41a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f3e3c06eff0 con 0x7f3e3c0cfa20 2024-09-21T07:55:04.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.885+0000 7f3e41000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e3c15cc00 0x7f3e3c156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:04.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.885+0000 7f3e41000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e3c15cc00 0x7f3e3c156360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42418/0 (socket says 172.21.15.39:42418) 2024-09-21T07:55:04.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.885+0000 7f3e3be00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3e3c0cfa20 0x7f3e3c144780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:04.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.885+0000 7f3e41000640 1 -- 172.21.15.39:0/4027965488 learned_addr learned my addr 172.21.15.39:0/4027965488 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:55:04.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.885+0000 7f3e33400640 1 --2- 172.21.15.39:0/4027965488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e3c0d04b0 0x7f3e3c155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:04.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.885+0000 7f3e41000640 1 -- 172.21.15.39:0/4027965488 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3e3c0cfa20 msgr2=0x7f3e3c144780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:04.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.885+0000 7f3e41000640 1 --2- 172.21.15.39:0/4027965488 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3e3c0cfa20 0x7f3e3c144780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:04.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.885+0000 7f3e41000640 1 -- 172.21.15.39:0/4027965488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e3c0d04b0 msgr2=0x7f3e3c155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:04.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.885+0000 7f3e41000640 1 --2- 172.21.15.39:0/4027965488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e3c0d04b0 0x7f3e3c155e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:04.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.885+0000 7f3e41000640 1 -- 172.21.15.39:0/4027965488 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e3c1f97f0 con 0x7f3e3c15cc00 2024-09-21T07:55:04.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.886+0000 7f3e41000640 1 --2- 172.21.15.39:0/4027965488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e3c15cc00 0x7f3e3c156360 secure :-1 s=READY pgs=2889 cs=0 l=1 rev1=1 crypto rx=0x7f3e3400c6f0 tx=0x7f3e3400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:04.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.886+0000 7f3e38c00640 1 -- 172.21.15.39:0/4027965488 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e34017070 con 0x7f3e3c15cc00 2024-09-21T07:55:04.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.886+0000 7f3e38c00640 1 -- 172.21.15.39:0/4027965488 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e34003c90 con 0x7f3e3c15cc00 2024-09-21T07:55:04.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.886+0000 7f3e38c00640 1 -- 172.21.15.39:0/4027965488 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e34003e60 con 0x7f3e3c15cc00 2024-09-21T07:55:04.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.886+0000 7f3e41a00640 1 -- 172.21.15.39:0/4027965488 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e3c1e3cc0 con 0x7f3e3c15cc00 2024-09-21T07:55:04.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.886+0000 7f3e41a00640 1 -- 172.21.15.39:0/4027965488 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e3c1f8d10 con 0x7f3e3c15cc00 2024-09-21T07:55:04.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.887+0000 7f3e41a00640 1 -- 172.21.15.39:0/4027965488 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e3c06f480 con 0x7f3e3c15cc00 2024-09-21T07:55:04.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.888+0000 7f3e38c00640 1 -- 172.21.15.39:0/4027965488 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3e3401a070 con 0x7f3e3c15cc00 2024-09-21T07:55:04.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.889+0000 7f3e38c00640 1 --2- 172.21.15.39:0/4027965488 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3e18078b90 0x7f3e1807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:04.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.889+0000 7f3e38c00640 1 -- 172.21.15.39:0/4027965488 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(269..269 src has 1..269) ==== 6278+0+0 (secure 0 0 0) 0x7f3e3409a580 con 0x7f3e3c15cc00 2024-09-21T07:55:04.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.890+0000 7f3e3be00640 1 --2- 172.21.15.39:0/4027965488 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3e18078b90 0x7f3e1807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:04.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.890+0000 7f3e3be00640 1 --2- 172.21.15.39:0/4027965488 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3e18078b90 0x7f3e1807b0a0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c034000 tx=0x7f3e2c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:04.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:04.894+0000 7f3e38c00640 1 -- 172.21.15.39:0/4027965488 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3e34062090 con 0x7f3e3c15cc00 2024-09-21T07:55:05.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.091+0000 7f3e41a00640 1 -- 172.21.15.39:0/4027965488 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3e3c0cf620 con 0x7f3e18078b90 2024-09-21T07:55:05.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.097+0000 7f3e38c00640 1 -- 172.21.15.39:0/4027965488 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+256262 (secure 0 0 0) 0x7f3e3c0cf620 con 0x7f3e18078b90 2024-09-21T07:55:05.101 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:55:05.103 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:55:05.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.101+0000 7f3e41a00640 1 -- 172.21.15.39:0/4027965488 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3e18078b90 msgr2=0x7f3e1807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:05.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.101+0000 7f3e41a00640 1 --2- 172.21.15.39:0/4027965488 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3e18078b90 0x7f3e1807b0a0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c034000 tx=0x7f3e2c004730 comp rx=0 tx=0).stop 2024-09-21T07:55:05.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.101+0000 7f3e41a00640 1 -- 172.21.15.39:0/4027965488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e3c15cc00 msgr2=0x7f3e3c156360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:05.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.101+0000 7f3e41a00640 1 --2- 172.21.15.39:0/4027965488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e3c15cc00 0x7f3e3c156360 secure :-1 s=READY pgs=2889 cs=0 l=1 rev1=1 crypto rx=0x7f3e3400c6f0 tx=0x7f3e3400cbb0 comp rx=0 tx=0).stop 2024-09-21T07:55:05.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.101+0000 7f3e41a00640 1 -- 172.21.15.39:0/4027965488 shutdown_connections 2024-09-21T07:55:05.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.101+0000 7f3e41a00640 1 --2- 172.21.15.39:0/4027965488 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3e18078b90 0x7f3e1807b0a0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:05.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.101+0000 7f3e41a00640 1 --2- 172.21.15.39:0/4027965488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e3c15cc00 0x7f3e3c156360 unknown :-1 s=CLOSED pgs=2889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:05.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.101+0000 7f3e41a00640 1 --2- 172.21.15.39:0/4027965488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3e3c0d04b0 0x7f3e3c155e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:05.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.101+0000 7f3e41a00640 1 --2- 172.21.15.39:0/4027965488 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3e3c0cfa20 0x7f3e3c144780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:05.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.101+0000 7f3e41a00640 1 -- 172.21.15.39:0/4027965488 >> 172.21.15.39:0/4027965488 conn(0x7f3e3c06cfc0 msgr2=0x7f3e3c15c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:05.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.101+0000 7f3e41a00640 1 -- 172.21.15.39:0/4027965488 shutdown_connections 2024-09-21T07:55:05.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.102+0000 7f3e41a00640 1 -- 172.21.15.39:0/4027965488 wait complete. 2024-09-21T07:55:05.125 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":925,"stamp":"2024-09-21T07:55:03.986673+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":211,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":466044,"kb_used_data":27720,"kb_used_omap":12,"kb_used_meta":438131,"kb_avail":754508676,"statfs":{"total":773094113280,"available":772616884224,"internally_reserved":0,"allocated":28385280,"data_stored":15991320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":448646704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12736,"num_objects":-25,"num_object_clones":-1,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":-1,"num_read":-1,"num_read_kb":-1,"num_write":-60,"num_write_kb":-31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.383374"},"pg_stats":[{"pgid":"12.d","version":"0'0","reported_seq":403,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.315418+0000","last_change":"2024-09-21T07:35:27.879605+0000","last_active":"2024-09-21T07:55:00.315418+0000","last_peered":"2024-09-21T07:55:00.315418+0000","last_clean":"2024-09-21T07:55:00.315418+0000","last_became_active":"2024-09-21T07:35:27.879274+0000","last_became_peered":"2024-09-21T07:35:27.879274+0000","last_unstale":"2024-09-21T07:55:00.315418+0000","last_undegraded":"2024-09-21T07:55:00.315418+0000","last_fullsized":"2024-09-21T07:55:00.315418+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:24.241636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":413,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.312401+0000","last_change":"2024-09-21T07:35:29.491336+0000","last_active":"2024-09-21T07:55:00.312401+0000","last_peered":"2024-09-21T07:55:00.312401+0000","last_clean":"2024-09-21T07:55:00.312401+0000","last_became_active":"2024-09-21T07:35:26.866484+0000","last_became_peered":"2024-09-21T07:35:26.866484+0000","last_unstale":"2024-09-21T07:55:00.312401+0000","last_undegraded":"2024-09-21T07:55:00.312401+0000","last_fullsized":"2024-09-21T07:55:00.312401+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:10.449665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076010100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":589,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.311307+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:55:00.311307+0000","last_peered":"2024-09-21T07:55:00.311307+0000","last_clean":"2024-09-21T07:55:00.311307+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:55:00.311307+0000","last_undegraded":"2024-09-21T07:55:00.311307+0000","last_fullsized":"2024-09-21T07:55:00.311307+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":480,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.632658+0000","last_change":"2024-09-21T07:34:24.539248+0000","last_active":"2024-09-21T07:54:59.632658+0000","last_peered":"2024-09-21T07:54:59.632658+0000","last_clean":"2024-09-21T07:54:59.632658+0000","last_became_active":"2024-09-21T07:34:23.273465+0000","last_became_peered":"2024-09-21T07:34:23.273465+0000","last_unstale":"2024-09-21T07:54:59.632658+0000","last_undegraded":"2024-09-21T07:54:59.632658+0000","last_fullsized":"2024-09-21T07:54:59.632658+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:55.517452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060627700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":407,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.312347+0000","last_change":"2024-09-21T07:35:27.864492+0000","last_active":"2024-09-21T07:55:00.312347+0000","last_peered":"2024-09-21T07:55:00.312347+0000","last_clean":"2024-09-21T07:55:00.312347+0000","last_became_active":"2024-09-21T07:35:27.864176+0000","last_became_peered":"2024-09-21T07:35:27.864176+0000","last_unstale":"2024-09-21T07:55:00.312347+0000","last_undegraded":"2024-09-21T07:55:00.312347+0000","last_fullsized":"2024-09-21T07:55:00.312347+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:05.153467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":410,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.314353+0000","last_change":"2024-09-21T07:35:29.493678+0000","last_active":"2024-09-21T07:55:00.314353+0000","last_peered":"2024-09-21T07:55:00.314353+0000","last_clean":"2024-09-21T07:55:00.314353+0000","last_became_active":"2024-09-21T07:35:26.870749+0000","last_became_peered":"2024-09-21T07:35:26.870749+0000","last_unstale":"2024-09-21T07:55:00.314353+0000","last_undegraded":"2024-09-21T07:55:00.314353+0000","last_fullsized":"2024-09-21T07:55:00.314353+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:55.367877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010159570000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":456,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.175411+0000","last_change":"2024-09-21T07:34:24.597274+0000","last_active":"2024-09-21T07:55:00.175411+0000","last_peered":"2024-09-21T07:55:00.175411+0000","last_clean":"2024-09-21T07:55:00.175411+0000","last_became_active":"2024-09-21T07:34:23.279560+0000","last_became_peered":"2024-09-21T07:34:23.279560+0000","last_unstale":"2024-09-21T07:55:00.175411+0000","last_undegraded":"2024-09-21T07:55:00.175411+0000","last_fullsized":"2024-09-21T07:55:00.175411+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:15.845265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059699799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":407,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.631402+0000","last_change":"2024-09-21T07:35:27.874387+0000","last_active":"2024-09-21T07:54:59.631402+0000","last_peered":"2024-09-21T07:54:59.631402+0000","last_clean":"2024-09-21T07:54:59.631402+0000","last_became_active":"2024-09-21T07:35:27.873277+0000","last_became_peered":"2024-09-21T07:35:27.873277+0000","last_unstale":"2024-09-21T07:54:59.631402+0000","last_undegraded":"2024-09-21T07:54:59.631402+0000","last_fullsized":"2024-09-21T07:54:59.631402+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:04.581463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":406,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.175552+0000","last_change":"2024-09-21T07:35:29.387094+0000","last_active":"2024-09-21T07:55:00.175552+0000","last_peered":"2024-09-21T07:55:00.175552+0000","last_clean":"2024-09-21T07:55:00.175552+0000","last_became_active":"2024-09-21T07:35:26.874757+0000","last_became_peered":"2024-09-21T07:35:26.874757+0000","last_unstale":"2024-09-21T07:55:00.175552+0000","last_undegraded":"2024-09-21T07:55:00.175552+0000","last_fullsized":"2024-09-21T07:55:00.175552+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:33.542750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061065700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":463,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.313869+0000","last_change":"2024-09-21T07:34:24.259906+0000","last_active":"2024-09-21T07:55:00.313869+0000","last_peered":"2024-09-21T07:55:00.313869+0000","last_clean":"2024-09-21T07:55:00.313869+0000","last_became_active":"2024-09-21T07:34:23.635282+0000","last_became_peered":"2024-09-21T07:34:23.635282+0000","last_unstale":"2024-09-21T07:55:00.313869+0000","last_undegraded":"2024-09-21T07:55:00.313869+0000","last_fullsized":"2024-09-21T07:55:00.313869+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:00.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001527664,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":407,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.634628+0000","last_change":"2024-09-21T07:35:27.873645+0000","last_active":"2024-09-21T07:54:59.634628+0000","last_peered":"2024-09-21T07:54:59.634628+0000","last_clean":"2024-09-21T07:54:59.634628+0000","last_became_active":"2024-09-21T07:35:27.873245+0000","last_became_peered":"2024-09-21T07:35:27.873245+0000","last_unstale":"2024-09-21T07:54:59.634628+0000","last_undegraded":"2024-09-21T07:54:59.634628+0000","last_fullsized":"2024-09-21T07:54:59.634628+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:55.090953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":413,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.633609+0000","last_change":"2024-09-21T07:35:28.861744+0000","last_active":"2024-09-21T07:54:59.633609+0000","last_peered":"2024-09-21T07:54:59.633609+0000","last_clean":"2024-09-21T07:54:59.633609+0000","last_became_active":"2024-09-21T07:35:26.876462+0000","last_became_peered":"2024-09-21T07:35:26.876462+0000","last_unstale":"2024-09-21T07:54:59.633609+0000","last_undegraded":"2024-09-21T07:54:59.633609+0000","last_fullsized":"2024-09-21T07:54:59.633609+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:48.632568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012020920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":464,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.630668+0000","last_change":"2024-09-21T07:34:25.085454+0000","last_active":"2024-09-21T07:54:59.630668+0000","last_peered":"2024-09-21T07:54:59.630668+0000","last_clean":"2024-09-21T07:54:59.630668+0000","last_became_active":"2024-09-21T07:34:23.279205+0000","last_became_peered":"2024-09-21T07:34:23.279205+0000","last_unstale":"2024-09-21T07:54:59.630668+0000","last_undegraded":"2024-09-21T07:54:59.630668+0000","last_fullsized":"2024-09-21T07:54:59.630668+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:54.231666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095320599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":405,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.314238+0000","last_change":"2024-09-21T07:35:27.861373+0000","last_active":"2024-09-21T07:55:00.314238+0000","last_peered":"2024-09-21T07:55:00.314238+0000","last_clean":"2024-09-21T07:55:00.314238+0000","last_became_active":"2024-09-21T07:35:27.861016+0000","last_became_peered":"2024-09-21T07:35:27.861016+0000","last_unstale":"2024-09-21T07:55:00.314238+0000","last_undegraded":"2024-09-21T07:55:00.314238+0000","last_fullsized":"2024-09-21T07:55:00.314238+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:56.043763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":413,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.311346+0000","last_change":"2024-09-21T07:35:29.053677+0000","last_active":"2024-09-21T07:55:00.311346+0000","last_peered":"2024-09-21T07:55:00.311346+0000","last_clean":"2024-09-21T07:55:00.311346+0000","last_became_active":"2024-09-21T07:35:26.875496+0000","last_became_peered":"2024-09-21T07:35:26.875496+0000","last_unstale":"2024-09-21T07:55:00.311346+0000","last_undegraded":"2024-09-21T07:55:00.311346+0000","last_fullsized":"2024-09-21T07:55:00.311346+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:04.850784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065859000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":465,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.310605+0000","last_change":"2024-09-21T07:34:25.028521+0000","last_active":"2024-09-21T07:55:00.310605+0000","last_peered":"2024-09-21T07:55:00.310605+0000","last_clean":"2024-09-21T07:55:00.310605+0000","last_became_active":"2024-09-21T07:34:23.635467+0000","last_became_peered":"2024-09-21T07:34:23.635467+0000","last_unstale":"2024-09-21T07:55:00.310605+0000","last_undegraded":"2024-09-21T07:55:00.310605+0000","last_fullsized":"2024-09-21T07:55:00.310605+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:33.729123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000660558,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":405,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.633153+0000","last_change":"2024-09-21T07:35:27.871201+0000","last_active":"2024-09-21T07:54:59.633153+0000","last_peered":"2024-09-21T07:54:59.633153+0000","last_clean":"2024-09-21T07:54:59.633153+0000","last_became_active":"2024-09-21T07:35:27.870816+0000","last_became_peered":"2024-09-21T07:35:27.870816+0000","last_unstale":"2024-09-21T07:54:59.633153+0000","last_undegraded":"2024-09-21T07:54:59.633153+0000","last_fullsized":"2024-09-21T07:54:59.633153+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:04.787450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":412,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.314136+0000","last_change":"2024-09-21T07:35:29.491874+0000","last_active":"2024-09-21T07:55:00.314136+0000","last_peered":"2024-09-21T07:55:00.314136+0000","last_clean":"2024-09-21T07:55:00.314136+0000","last_became_active":"2024-09-21T07:35:26.866381+0000","last_became_peered":"2024-09-21T07:35:26.866381+0000","last_unstale":"2024-09-21T07:55:00.314136+0000","last_undegraded":"2024-09-21T07:55:00.314136+0000","last_fullsized":"2024-09-21T07:55:00.314136+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:36.404647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073235399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":468,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.313603+0000","last_change":"2024-09-21T07:34:24.260046+0000","last_active":"2024-09-21T07:55:00.313603+0000","last_peered":"2024-09-21T07:55:00.313603+0000","last_clean":"2024-09-21T07:55:00.313603+0000","last_became_active":"2024-09-21T07:34:23.273431+0000","last_became_peered":"2024-09-21T07:34:23.273431+0000","last_unstale":"2024-09-21T07:55:00.313603+0000","last_undegraded":"2024-09-21T07:55:00.313603+0000","last_fullsized":"2024-09-21T07:55:00.313603+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:18.876057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019803310000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":405,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.632132+0000","last_change":"2024-09-21T07:35:27.871736+0000","last_active":"2024-09-21T07:54:59.632132+0000","last_peered":"2024-09-21T07:54:59.632132+0000","last_clean":"2024-09-21T07:54:59.632132+0000","last_became_active":"2024-09-21T07:35:27.871232+0000","last_became_peered":"2024-09-21T07:35:27.871232+0000","last_unstale":"2024-09-21T07:54:59.632132+0000","last_undegraded":"2024-09-21T07:54:59.632132+0000","last_fullsized":"2024-09-21T07:54:59.632132+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:04.419301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":414,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.311567+0000","last_change":"2024-09-21T07:35:29.491246+0000","last_active":"2024-09-21T07:55:00.311567+0000","last_peered":"2024-09-21T07:55:00.311567+0000","last_clean":"2024-09-21T07:55:00.311567+0000","last_became_active":"2024-09-21T07:35:26.866783+0000","last_became_peered":"2024-09-21T07:35:26.866783+0000","last_unstale":"2024-09-21T07:55:00.311567+0000","last_undegraded":"2024-09-21T07:55:00.311567+0000","last_fullsized":"2024-09-21T07:55:00.311567+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:28.881914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098474099999999992,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":465,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.311456+0000","last_change":"2024-09-21T07:34:25.028640+0000","last_active":"2024-09-21T07:55:00.311456+0000","last_peered":"2024-09-21T07:55:00.311456+0000","last_clean":"2024-09-21T07:55:00.311456+0000","last_became_active":"2024-09-21T07:34:23.260950+0000","last_became_peered":"2024-09-21T07:34:23.260950+0000","last_unstale":"2024-09-21T07:55:00.311456+0000","last_undegraded":"2024-09-21T07:55:00.311456+0000","last_fullsized":"2024-09-21T07:55:00.311456+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:23:42.121398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055135099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":408,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.319249+0000","last_change":"2024-09-21T07:35:27.872027+0000","last_active":"2024-09-21T07:55:00.319249+0000","last_peered":"2024-09-21T07:55:00.319249+0000","last_clean":"2024-09-21T07:55:00.319249+0000","last_became_active":"2024-09-21T07:35:27.870426+0000","last_became_peered":"2024-09-21T07:35:27.870426+0000","last_unstale":"2024-09-21T07:55:00.319249+0000","last_undegraded":"2024-09-21T07:55:00.319249+0000","last_fullsized":"2024-09-21T07:55:00.319249+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":414,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.312990+0000","last_change":"2024-09-21T07:35:29.491008+0000","last_active":"2024-09-21T07:55:00.312990+0000","last_peered":"2024-09-21T07:55:00.312990+0000","last_clean":"2024-09-21T07:55:00.312990+0000","last_became_active":"2024-09-21T07:35:26.866750+0000","last_became_peered":"2024-09-21T07:35:26.866750+0000","last_unstale":"2024-09-21T07:55:00.312990+0000","last_undegraded":"2024-09-21T07:55:00.312990+0000","last_fullsized":"2024-09-21T07:55:00.312990+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:25.490406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083529700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":464,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.315758+0000","last_change":"2024-09-21T07:34:24.260665+0000","last_active":"2024-09-21T07:55:00.315758+0000","last_peered":"2024-09-21T07:55:00.315758+0000","last_clean":"2024-09-21T07:55:00.315758+0000","last_became_active":"2024-09-21T07:34:23.635909+0000","last_became_peered":"2024-09-21T07:34:23.635909+0000","last_unstale":"2024-09-21T07:55:00.315758+0000","last_undegraded":"2024-09-21T07:55:00.315758+0000","last_fullsized":"2024-09-21T07:55:00.315758+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:44.536885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":400,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.173356+0000","last_change":"2024-09-21T07:35:27.874772+0000","last_active":"2024-09-21T07:55:00.173356+0000","last_peered":"2024-09-21T07:55:00.173356+0000","last_clean":"2024-09-21T07:55:00.173356+0000","last_became_active":"2024-09-21T07:35:27.874434+0000","last_became_peered":"2024-09-21T07:35:27.874434+0000","last_unstale":"2024-09-21T07:55:00.173356+0000","last_undegraded":"2024-09-21T07:55:00.173356+0000","last_fullsized":"2024-09-21T07:55:00.173356+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:31.998755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":428,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.632463+0000","last_change":"2024-09-21T07:35:28.859153+0000","last_active":"2024-09-21T07:54:59.632463+0000","last_peered":"2024-09-21T07:54:59.632463+0000","last_clean":"2024-09-21T07:54:59.632463+0000","last_became_active":"2024-09-21T07:35:26.874428+0000","last_became_peered":"2024-09-21T07:35:26.874428+0000","last_unstale":"2024-09-21T07:54:59.632463+0000","last_undegraded":"2024-09-21T07:54:59.632463+0000","last_fullsized":"2024-09-21T07:54:59.632463+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:04.487621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048557299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":480,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.630653+0000","last_change":"2024-09-21T07:34:24.538845+0000","last_active":"2024-09-21T07:54:59.630653+0000","last_peered":"2024-09-21T07:54:59.630653+0000","last_clean":"2024-09-21T07:54:59.630653+0000","last_became_active":"2024-09-21T07:34:23.636372+0000","last_became_peered":"2024-09-21T07:34:23.636372+0000","last_unstale":"2024-09-21T07:54:59.630653+0000","last_undegraded":"2024-09-21T07:54:59.630653+0000","last_fullsized":"2024-09-21T07:54:59.630653+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:29.393813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841451,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":409,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.312263+0000","last_change":"2024-09-21T07:35:27.877801+0000","last_active":"2024-09-21T07:55:00.312263+0000","last_peered":"2024-09-21T07:55:00.312263+0000","last_clean":"2024-09-21T07:55:00.312263+0000","last_became_active":"2024-09-21T07:35:27.877464+0000","last_became_peered":"2024-09-21T07:35:27.877464+0000","last_unstale":"2024-09-21T07:55:00.312263+0000","last_undegraded":"2024-09-21T07:55:00.312263+0000","last_fullsized":"2024-09-21T07:55:00.312263+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:08.814929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":413,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.632257+0000","last_change":"2024-09-21T07:35:28.863732+0000","last_active":"2024-09-21T07:54:59.632257+0000","last_peered":"2024-09-21T07:54:59.632257+0000","last_clean":"2024-09-21T07:54:59.632257+0000","last_became_active":"2024-09-21T07:35:26.877392+0000","last_became_peered":"2024-09-21T07:35:26.877392+0000","last_unstale":"2024-09-21T07:54:59.632257+0000","last_undegraded":"2024-09-21T07:54:59.632257+0000","last_fullsized":"2024-09-21T07:54:59.632257+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071425700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":467,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.314344+0000","last_change":"2024-09-21T07:34:24.259686+0000","last_active":"2024-09-21T07:55:00.314344+0000","last_peered":"2024-09-21T07:55:00.314344+0000","last_clean":"2024-09-21T07:55:00.314344+0000","last_became_active":"2024-09-21T07:34:23.274405+0000","last_became_peered":"2024-09-21T07:34:23.274405+0000","last_unstale":"2024-09-21T07:55:00.314344+0000","last_undegraded":"2024-09-21T07:55:00.314344+0000","last_fullsized":"2024-09-21T07:55:00.314344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:26.304693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022009920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":400,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.175037+0000","last_change":"2024-09-21T07:35:27.867336+0000","last_active":"2024-09-21T07:55:00.175037+0000","last_peered":"2024-09-21T07:55:00.175037+0000","last_clean":"2024-09-21T07:55:00.175037+0000","last_became_active":"2024-09-21T07:35:27.867015+0000","last_became_peered":"2024-09-21T07:35:27.867015+0000","last_unstale":"2024-09-21T07:55:00.175037+0000","last_undegraded":"2024-09-21T07:55:00.175037+0000","last_fullsized":"2024-09-21T07:55:00.175037+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:59.214139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":411,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.315550+0000","last_change":"2024-09-21T07:35:29.493137+0000","last_active":"2024-09-21T07:55:00.315550+0000","last_peered":"2024-09-21T07:55:00.315550+0000","last_clean":"2024-09-21T07:55:00.315550+0000","last_became_active":"2024-09-21T07:35:26.872230+0000","last_became_peered":"2024-09-21T07:35:26.872230+0000","last_unstale":"2024-09-21T07:55:00.315550+0000","last_undegraded":"2024-09-21T07:55:00.315550+0000","last_fullsized":"2024-09-21T07:55:00.315550+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:24.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001197449,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":464,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.311160+0000","last_change":"2024-09-21T07:34:24.258754+0000","last_active":"2024-09-21T07:55:00.311160+0000","last_peered":"2024-09-21T07:55:00.311160+0000","last_clean":"2024-09-21T07:55:00.311160+0000","last_became_active":"2024-09-21T07:34:23.272657+0000","last_became_peered":"2024-09-21T07:34:23.272657+0000","last_unstale":"2024-09-21T07:55:00.311160+0000","last_undegraded":"2024-09-21T07:55:00.311160+0000","last_fullsized":"2024-09-21T07:55:00.311160+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:05.477099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067143200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":406,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.632348+0000","last_change":"2024-09-21T07:35:27.876021+0000","last_active":"2024-09-21T07:54:59.632348+0000","last_peered":"2024-09-21T07:54:59.632348+0000","last_clean":"2024-09-21T07:54:59.632348+0000","last_became_active":"2024-09-21T07:35:27.875724+0000","last_became_peered":"2024-09-21T07:35:27.875724+0000","last_unstale":"2024-09-21T07:54:59.632348+0000","last_undegraded":"2024-09-21T07:54:59.632348+0000","last_fullsized":"2024-09-21T07:54:59.632348+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:03.022812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":411,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.313993+0000","last_change":"2024-09-21T07:35:29.492570+0000","last_active":"2024-09-21T07:55:00.313993+0000","last_peered":"2024-09-21T07:55:00.313993+0000","last_clean":"2024-09-21T07:55:00.313993+0000","last_became_active":"2024-09-21T07:35:26.874316+0000","last_became_peered":"2024-09-21T07:35:26.874316+0000","last_unstale":"2024-09-21T07:55:00.313993+0000","last_undegraded":"2024-09-21T07:55:00.313993+0000","last_fullsized":"2024-09-21T07:55:00.313993+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:16.512323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011990760000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":464,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.633378+0000","last_change":"2024-09-21T07:34:25.085232+0000","last_active":"2024-09-21T07:54:59.633378+0000","last_peered":"2024-09-21T07:54:59.633378+0000","last_clean":"2024-09-21T07:54:59.633378+0000","last_became_active":"2024-09-21T07:34:23.277807+0000","last_became_peered":"2024-09-21T07:34:23.277807+0000","last_unstale":"2024-09-21T07:54:59.633378+0000","last_undegraded":"2024-09-21T07:54:59.633378+0000","last_fullsized":"2024-09-21T07:54:59.633378+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:36.362919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086619099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":409,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.316773+0000","last_change":"2024-09-21T07:35:27.872798+0000","last_active":"2024-09-21T07:55:00.316773+0000","last_peered":"2024-09-21T07:55:00.316773+0000","last_clean":"2024-09-21T07:55:00.316773+0000","last_became_active":"2024-09-21T07:35:27.872483+0000","last_became_peered":"2024-09-21T07:35:27.872483+0000","last_unstale":"2024-09-21T07:55:00.316773+0000","last_undegraded":"2024-09-21T07:55:00.316773+0000","last_fullsized":"2024-09-21T07:55:00.316773+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:11.933469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":416,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.315971+0000","last_change":"2024-09-21T07:35:29.206723+0000","last_active":"2024-09-21T07:55:00.315971+0000","last_peered":"2024-09-21T07:55:00.315971+0000","last_clean":"2024-09-21T07:55:00.315971+0000","last_became_active":"2024-09-21T07:35:26.872377+0000","last_became_peered":"2024-09-21T07:35:26.872377+0000","last_unstale":"2024-09-21T07:55:00.315971+0000","last_undegraded":"2024-09-21T07:55:00.315971+0000","last_fullsized":"2024-09-21T07:55:00.315971+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:29.086862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083077200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":456,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.175826+0000","last_change":"2024-09-21T07:34:24.597672+0000","last_active":"2024-09-21T07:55:00.175826+0000","last_peered":"2024-09-21T07:55:00.175826+0000","last_clean":"2024-09-21T07:55:00.175826+0000","last_became_active":"2024-09-21T07:34:23.267770+0000","last_became_peered":"2024-09-21T07:34:23.267770+0000","last_unstale":"2024-09-21T07:55:00.175826+0000","last_undegraded":"2024-09-21T07:55:00.175826+0000","last_fullsized":"2024-09-21T07:55:00.175826+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:51.629476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086378500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":409,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.316038+0000","last_change":"2024-09-21T07:35:27.870121+0000","last_active":"2024-09-21T07:55:00.316038+0000","last_peered":"2024-09-21T07:55:00.316038+0000","last_clean":"2024-09-21T07:55:00.316038+0000","last_became_active":"2024-09-21T07:35:27.869811+0000","last_became_peered":"2024-09-21T07:35:27.869811+0000","last_unstale":"2024-09-21T07:55:00.316038+0000","last_undegraded":"2024-09-21T07:55:00.316038+0000","last_fullsized":"2024-09-21T07:55:00.316038+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:31.778461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":414,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.634294+0000","last_change":"2024-09-21T07:35:28.860699+0000","last_active":"2024-09-21T07:54:59.634294+0000","last_peered":"2024-09-21T07:54:59.634294+0000","last_clean":"2024-09-21T07:54:59.634294+0000","last_became_active":"2024-09-21T07:35:26.876779+0000","last_became_peered":"2024-09-21T07:35:26.876779+0000","last_unstale":"2024-09-21T07:54:59.634294+0000","last_undegraded":"2024-09-21T07:54:59.634294+0000","last_fullsized":"2024-09-21T07:54:59.634294+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:28.286403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000913188,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":466,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.634607+0000","last_change":"2024-09-21T07:34:24.260181+0000","last_active":"2024-09-21T07:54:59.634607+0000","last_peered":"2024-09-21T07:54:59.634607+0000","last_clean":"2024-09-21T07:54:59.634607+0000","last_became_active":"2024-09-21T07:34:23.275580+0000","last_became_peered":"2024-09-21T07:34:23.275580+0000","last_unstale":"2024-09-21T07:54:59.634607+0000","last_undegraded":"2024-09-21T07:54:59.634607+0000","last_fullsized":"2024-09-21T07:54:59.634607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:49.857622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014706980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":407,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.314738+0000","last_change":"2024-09-21T07:35:27.875393+0000","last_active":"2024-09-21T07:55:00.314738+0000","last_peered":"2024-09-21T07:55:00.314738+0000","last_clean":"2024-09-21T07:55:00.314738+0000","last_became_active":"2024-09-21T07:35:27.874671+0000","last_became_peered":"2024-09-21T07:35:27.874671+0000","last_unstale":"2024-09-21T07:55:00.314738+0000","last_undegraded":"2024-09-21T07:55:00.314738+0000","last_fullsized":"2024-09-21T07:55:00.314738+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":416,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.312471+0000","last_change":"2024-09-21T07:35:29.206296+0000","last_active":"2024-09-21T07:55:00.312471+0000","last_peered":"2024-09-21T07:55:00.312471+0000","last_clean":"2024-09-21T07:55:00.312471+0000","last_became_active":"2024-09-21T07:35:26.874575+0000","last_became_peered":"2024-09-21T07:35:26.874575+0000","last_unstale":"2024-09-21T07:55:00.312471+0000","last_undegraded":"2024-09-21T07:55:00.312471+0000","last_fullsized":"2024-09-21T07:55:00.312471+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:22.771154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061011599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":467,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.315376+0000","last_change":"2024-09-21T07:34:24.259112+0000","last_active":"2024-09-21T07:55:00.315376+0000","last_peered":"2024-09-21T07:55:00.315376+0000","last_clean":"2024-09-21T07:55:00.315376+0000","last_became_active":"2024-09-21T07:34:23.635444+0000","last_became_peered":"2024-09-21T07:34:23.635444+0000","last_unstale":"2024-09-21T07:55:00.315376+0000","last_undegraded":"2024-09-21T07:55:00.315376+0000","last_fullsized":"2024-09-21T07:55:00.315376+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:46.255385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001952636,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":407,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.312824+0000","last_change":"2024-09-21T07:35:27.878356+0000","last_active":"2024-09-21T07:55:00.312824+0000","last_peered":"2024-09-21T07:55:00.312824+0000","last_clean":"2024-09-21T07:55:00.312824+0000","last_became_active":"2024-09-21T07:35:27.877982+0000","last_became_peered":"2024-09-21T07:35:27.877982+0000","last_unstale":"2024-09-21T07:55:00.312824+0000","last_undegraded":"2024-09-21T07:55:00.312824+0000","last_fullsized":"2024-09-21T07:55:00.312824+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":414,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.310325+0000","last_change":"2024-09-21T07:35:29.053131+0000","last_active":"2024-09-21T07:55:00.310325+0000","last_peered":"2024-09-21T07:55:00.310325+0000","last_clean":"2024-09-21T07:55:00.310325+0000","last_became_active":"2024-09-21T07:35:26.870516+0000","last_became_peered":"2024-09-21T07:35:26.870516+0000","last_unstale":"2024-09-21T07:55:00.310325+0000","last_undegraded":"2024-09-21T07:55:00.310325+0000","last_fullsized":"2024-09-21T07:55:00.310325+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:10.924491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070705699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":480,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.632868+0000","last_change":"2024-09-21T07:34:24.539152+0000","last_active":"2024-09-21T07:54:59.632868+0000","last_peered":"2024-09-21T07:54:59.632868+0000","last_clean":"2024-09-21T07:54:59.632868+0000","last_became_active":"2024-09-21T07:34:23.259573+0000","last_became_peered":"2024-09-21T07:34:23.259573+0000","last_unstale":"2024-09-21T07:54:59.632868+0000","last_undegraded":"2024-09-21T07:54:59.632868+0000","last_fullsized":"2024-09-21T07:54:59.632868+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:26.986900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058108400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":406,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.310193+0000","last_change":"2024-09-21T07:35:27.873726+0000","last_active":"2024-09-21T07:55:00.310193+0000","last_peered":"2024-09-21T07:55:00.310193+0000","last_clean":"2024-09-21T07:55:00.310193+0000","last_became_active":"2024-09-21T07:35:27.871421+0000","last_became_peered":"2024-09-21T07:35:27.871421+0000","last_unstale":"2024-09-21T07:55:00.310193+0000","last_undegraded":"2024-09-21T07:55:00.310193+0000","last_fullsized":"2024-09-21T07:55:00.310193+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:00.168915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":414,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.634353+0000","last_change":"2024-09-21T07:35:28.861032+0000","last_active":"2024-09-21T07:54:59.634353+0000","last_peered":"2024-09-21T07:54:59.634353+0000","last_clean":"2024-09-21T07:54:59.634353+0000","last_became_active":"2024-09-21T07:35:26.873149+0000","last_became_peered":"2024-09-21T07:35:26.873149+0000","last_unstale":"2024-09-21T07:54:59.634353+0000","last_undegraded":"2024-09-21T07:54:59.634353+0000","last_fullsized":"2024-09-21T07:54:59.634353+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:59.435981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011988369999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":463,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.311971+0000","last_change":"2024-09-21T07:34:24.259902+0000","last_active":"2024-09-21T07:55:00.311971+0000","last_peered":"2024-09-21T07:55:00.311971+0000","last_clean":"2024-09-21T07:55:00.311971+0000","last_became_active":"2024-09-21T07:34:23.271902+0000","last_became_peered":"2024-09-21T07:34:23.271902+0000","last_unstale":"2024-09-21T07:55:00.311971+0000","last_undegraded":"2024-09-21T07:55:00.311971+0000","last_fullsized":"2024-09-21T07:55:00.311971+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:59.439525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001686358,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":405,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.631813+0000","last_change":"2024-09-21T07:35:27.872637+0000","last_active":"2024-09-21T07:54:59.631813+0000","last_peered":"2024-09-21T07:54:59.631813+0000","last_clean":"2024-09-21T07:54:59.631813+0000","last_became_active":"2024-09-21T07:35:27.872149+0000","last_became_peered":"2024-09-21T07:35:27.872149+0000","last_unstale":"2024-09-21T07:54:59.631813+0000","last_undegraded":"2024-09-21T07:54:59.631813+0000","last_fullsized":"2024-09-21T07:54:59.631813+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:27.697872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":411,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.314113+0000","last_change":"2024-09-21T07:35:29.493362+0000","last_active":"2024-09-21T07:55:00.314113+0000","last_peered":"2024-09-21T07:55:00.314113+0000","last_clean":"2024-09-21T07:55:00.314113+0000","last_became_active":"2024-09-21T07:35:26.873018+0000","last_became_peered":"2024-09-21T07:35:26.873018+0000","last_unstale":"2024-09-21T07:55:00.314113+0000","last_undegraded":"2024-09-21T07:55:00.314113+0000","last_fullsized":"2024-09-21T07:55:00.314113+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:02.460557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012496510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":466,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.314038+0000","last_change":"2024-09-21T07:34:24.259662+0000","last_active":"2024-09-21T07:55:00.314038+0000","last_peered":"2024-09-21T07:55:00.314038+0000","last_clean":"2024-09-21T07:55:00.314038+0000","last_became_active":"2024-09-21T07:34:23.278812+0000","last_became_peered":"2024-09-21T07:34:23.278812+0000","last_unstale":"2024-09-21T07:55:00.314038+0000","last_undegraded":"2024-09-21T07:55:00.314038+0000","last_fullsized":"2024-09-21T07:55:00.314038+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:49.829974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356463,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":398,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.174789+0000","last_change":"2024-09-21T07:35:27.868448+0000","last_active":"2024-09-21T07:55:00.174789+0000","last_peered":"2024-09-21T07:55:00.174789+0000","last_clean":"2024-09-21T07:55:00.174789+0000","last_became_active":"2024-09-21T07:35:27.868105+0000","last_became_peered":"2024-09-21T07:35:27.868105+0000","last_unstale":"2024-09-21T07:55:00.174789+0000","last_undegraded":"2024-09-21T07:55:00.174789+0000","last_fullsized":"2024-09-21T07:55:00.174789+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:41.468702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":409,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.315283+0000","last_change":"2024-09-21T07:35:29.492974+0000","last_active":"2024-09-21T07:55:00.315283+0000","last_peered":"2024-09-21T07:55:00.315283+0000","last_clean":"2024-09-21T07:55:00.315283+0000","last_became_active":"2024-09-21T07:35:26.872165+0000","last_became_peered":"2024-09-21T07:35:26.872165+0000","last_unstale":"2024-09-21T07:55:00.315283+0000","last_undegraded":"2024-09-21T07:55:00.315283+0000","last_fullsized":"2024-09-21T07:55:00.315283+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:59.912520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013795179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":463,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.630639+0000","last_change":"2024-09-21T07:34:25.084861+0000","last_active":"2024-09-21T07:54:59.630639+0000","last_peered":"2024-09-21T07:54:59.630639+0000","last_clean":"2024-09-21T07:54:59.630639+0000","last_became_active":"2024-09-21T07:34:23.277708+0000","last_became_peered":"2024-09-21T07:34:23.277708+0000","last_unstale":"2024-09-21T07:54:59.630639+0000","last_undegraded":"2024-09-21T07:54:59.630639+0000","last_fullsized":"2024-09-21T07:54:59.630639+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:40.816488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000597566,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":398,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.175249+0000","last_change":"2024-09-21T07:35:27.874640+0000","last_active":"2024-09-21T07:55:00.175249+0000","last_peered":"2024-09-21T07:55:00.175249+0000","last_clean":"2024-09-21T07:55:00.175249+0000","last_became_active":"2024-09-21T07:35:27.874265+0000","last_became_peered":"2024-09-21T07:35:27.874265+0000","last_unstale":"2024-09-21T07:55:00.175249+0000","last_undegraded":"2024-09-21T07:55:00.175249+0000","last_fullsized":"2024-09-21T07:55:00.175249+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:45.754654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":414,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.318433+0000","last_change":"2024-09-21T07:35:29.207392+0000","last_active":"2024-09-21T07:55:00.318433+0000","last_peered":"2024-09-21T07:55:00.318433+0000","last_clean":"2024-09-21T07:55:00.318433+0000","last_became_active":"2024-09-21T07:35:26.870917+0000","last_became_peered":"2024-09-21T07:35:26.870917+0000","last_unstale":"2024-09-21T07:55:00.318433+0000","last_undegraded":"2024-09-21T07:55:00.318433+0000","last_fullsized":"2024-09-21T07:55:00.318433+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:59.159937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059198499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":467,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.313609+0000","last_change":"2024-09-21T07:34:24.258709+0000","last_active":"2024-09-21T07:55:00.313609+0000","last_peered":"2024-09-21T07:55:00.313609+0000","last_clean":"2024-09-21T07:55:00.313609+0000","last_became_active":"2024-09-21T07:34:23.279042+0000","last_became_peered":"2024-09-21T07:34:23.279042+0000","last_unstale":"2024-09-21T07:55:00.313609+0000","last_undegraded":"2024-09-21T07:55:00.313609+0000","last_fullsized":"2024-09-21T07:55:00.313609+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:01.762117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019282100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":404,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.633291+0000","last_change":"2024-09-21T07:35:27.870260+0000","last_active":"2024-09-21T07:54:59.633291+0000","last_peered":"2024-09-21T07:54:59.633291+0000","last_clean":"2024-09-21T07:54:59.633291+0000","last_became_active":"2024-09-21T07:35:27.869448+0000","last_became_peered":"2024-09-21T07:35:27.869448+0000","last_unstale":"2024-09-21T07:54:59.633291+0000","last_undegraded":"2024-09-21T07:54:59.633291+0000","last_fullsized":"2024-09-21T07:54:59.633291+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:16.946790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":405,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.175086+0000","last_change":"2024-09-21T07:35:29.386367+0000","last_active":"2024-09-21T07:55:00.175086+0000","last_peered":"2024-09-21T07:55:00.175086+0000","last_clean":"2024-09-21T07:55:00.175086+0000","last_became_active":"2024-09-21T07:35:26.879238+0000","last_became_peered":"2024-09-21T07:35:26.879238+0000","last_unstale":"2024-09-21T07:55:00.175086+0000","last_undegraded":"2024-09-21T07:55:00.175086+0000","last_fullsized":"2024-09-21T07:55:00.175086+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:16.528260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092223100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":463,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.632428+0000","last_change":"2024-09-21T07:34:25.085635+0000","last_active":"2024-09-21T07:54:59.632428+0000","last_peered":"2024-09-21T07:54:59.632428+0000","last_clean":"2024-09-21T07:54:59.632428+0000","last_became_active":"2024-09-21T07:34:23.273566+0000","last_became_peered":"2024-09-21T07:34:23.273566+0000","last_unstale":"2024-09-21T07:54:59.632428+0000","last_undegraded":"2024-09-21T07:54:59.632428+0000","last_fullsized":"2024-09-21T07:54:59.632428+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:14.142272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079897300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":404,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.633141+0000","last_change":"2024-09-21T07:35:27.878728+0000","last_active":"2024-09-21T07:54:59.633141+0000","last_peered":"2024-09-21T07:54:59.633141+0000","last_clean":"2024-09-21T07:54:59.633141+0000","last_became_active":"2024-09-21T07:35:27.878335+0000","last_became_peered":"2024-09-21T07:35:27.878335+0000","last_unstale":"2024-09-21T07:54:59.633141+0000","last_undegraded":"2024-09-21T07:54:59.633141+0000","last_fullsized":"2024-09-21T07:54:59.633141+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:25:37.731771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":426,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.630524+0000","last_change":"2024-09-21T07:35:28.858418+0000","last_active":"2024-09-21T07:54:59.630524+0000","last_peered":"2024-09-21T07:54:59.630524+0000","last_clean":"2024-09-21T07:54:59.630524+0000","last_became_active":"2024-09-21T07:35:26.876944+0000","last_became_peered":"2024-09-21T07:35:26.876944+0000","last_unstale":"2024-09-21T07:54:59.630524+0000","last_undegraded":"2024-09-21T07:54:59.630524+0000","last_fullsized":"2024-09-21T07:54:59.630524+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:04.514304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090862799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":455,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.176081+0000","last_change":"2024-09-21T07:34:24.597926+0000","last_active":"2024-09-21T07:55:00.176081+0000","last_peered":"2024-09-21T07:55:00.176081+0000","last_clean":"2024-09-21T07:55:00.176081+0000","last_became_active":"2024-09-21T07:34:23.268926+0000","last_became_peered":"2024-09-21T07:34:23.268926+0000","last_unstale":"2024-09-21T07:55:00.176081+0000","last_undegraded":"2024-09-21T07:55:00.176081+0000","last_fullsized":"2024-09-21T07:55:00.176081+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:46.190752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101851,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":400,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.175363+0000","last_change":"2024-09-21T07:35:27.877426+0000","last_active":"2024-09-21T07:55:00.175363+0000","last_peered":"2024-09-21T07:55:00.175363+0000","last_clean":"2024-09-21T07:55:00.175363+0000","last_became_active":"2024-09-21T07:35:27.877050+0000","last_became_peered":"2024-09-21T07:35:27.877050+0000","last_unstale":"2024-09-21T07:55:00.175363+0000","last_undegraded":"2024-09-21T07:55:00.175363+0000","last_fullsized":"2024-09-21T07:55:00.175363+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:36.268334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":412,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.631943+0000","last_change":"2024-09-21T07:35:28.862273+0000","last_active":"2024-09-21T07:54:59.631943+0000","last_peered":"2024-09-21T07:54:59.631943+0000","last_clean":"2024-09-21T07:54:59.631943+0000","last_became_active":"2024-09-21T07:35:26.874011+0000","last_became_peered":"2024-09-21T07:35:26.874011+0000","last_unstale":"2024-09-21T07:54:59.631943+0000","last_undegraded":"2024-09-21T07:54:59.631943+0000","last_fullsized":"2024-09-21T07:54:59.631943+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:11.986950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071644799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":465,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.637034+0000","last_change":"2024-09-21T07:34:24.259794+0000","last_active":"2024-09-21T07:54:59.637034+0000","last_peered":"2024-09-21T07:54:59.637034+0000","last_clean":"2024-09-21T07:54:59.637034+0000","last_became_active":"2024-09-21T07:34:23.265679+0000","last_became_peered":"2024-09-21T07:34:23.265679+0000","last_unstale":"2024-09-21T07:54:59.637034+0000","last_undegraded":"2024-09-21T07:54:59.637034+0000","last_fullsized":"2024-09-21T07:54:59.637034+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:02.575796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001298518,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":409,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.317882+0000","last_change":"2024-09-21T07:35:27.877725+0000","last_active":"2024-09-21T07:55:00.317882+0000","last_peered":"2024-09-21T07:55:00.317882+0000","last_clean":"2024-09-21T07:55:00.317882+0000","last_became_active":"2024-09-21T07:35:27.877398+0000","last_became_peered":"2024-09-21T07:35:27.877398+0000","last_unstale":"2024-09-21T07:55:00.317882+0000","last_undegraded":"2024-09-21T07:55:00.317882+0000","last_fullsized":"2024-09-21T07:55:00.317882+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:05.859762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":413,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.633091+0000","last_change":"2024-09-21T07:35:28.860612+0000","last_active":"2024-09-21T07:54:59.633091+0000","last_peered":"2024-09-21T07:54:59.633091+0000","last_clean":"2024-09-21T07:54:59.633091+0000","last_became_active":"2024-09-21T07:35:26.876562+0000","last_became_peered":"2024-09-21T07:35:26.876562+0000","last_unstale":"2024-09-21T07:54:59.633091+0000","last_undegraded":"2024-09-21T07:54:59.633091+0000","last_fullsized":"2024-09-21T07:54:59.633091+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:06.824331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012252470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":463,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.315655+0000","last_change":"2024-09-21T07:34:24.260330+0000","last_active":"2024-09-21T07:55:00.315655+0000","last_peered":"2024-09-21T07:55:00.315655+0000","last_clean":"2024-09-21T07:55:00.315655+0000","last_became_active":"2024-09-21T07:34:23.637210+0000","last_became_peered":"2024-09-21T07:34:23.637210+0000","last_unstale":"2024-09-21T07:55:00.315655+0000","last_undegraded":"2024-09-21T07:55:00.315655+0000","last_fullsized":"2024-09-21T07:55:00.315655+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.488451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001537093,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":405,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.630514+0000","last_change":"2024-09-21T07:35:27.874757+0000","last_active":"2024-09-21T07:54:59.630514+0000","last_peered":"2024-09-21T07:54:59.630514+0000","last_clean":"2024-09-21T07:54:59.630514+0000","last_became_active":"2024-09-21T07:35:27.874360+0000","last_became_peered":"2024-09-21T07:35:27.874360+0000","last_unstale":"2024-09-21T07:54:59.630514+0000","last_undegraded":"2024-09-21T07:54:59.630514+0000","last_fullsized":"2024-09-21T07:54:59.630514+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:17.646912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":410,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.314189+0000","last_change":"2024-09-21T07:35:29.493431+0000","last_active":"2024-09-21T07:55:00.314189+0000","last_peered":"2024-09-21T07:55:00.314189+0000","last_clean":"2024-09-21T07:55:00.314189+0000","last_became_active":"2024-09-21T07:35:26.873691+0000","last_became_peered":"2024-09-21T07:35:26.873691+0000","last_unstale":"2024-09-21T07:55:00.314189+0000","last_undegraded":"2024-09-21T07:55:00.314189+0000","last_fullsized":"2024-09-21T07:55:00.314189+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:51.705051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001162464,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":455,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.175180+0000","last_change":"2024-09-21T07:34:24.597286+0000","last_active":"2024-09-21T07:55:00.175180+0000","last_peered":"2024-09-21T07:55:00.175180+0000","last_clean":"2024-09-21T07:55:00.175180+0000","last_became_active":"2024-09-21T07:34:23.267133+0000","last_became_peered":"2024-09-21T07:34:23.267133+0000","last_unstale":"2024-09-21T07:55:00.175180+0000","last_undegraded":"2024-09-21T07:55:00.175180+0000","last_fullsized":"2024-09-21T07:55:00.175180+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:18.257010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":420,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.630258+0000","last_change":"2024-09-21T07:35:27.864458+0000","last_active":"2024-09-21T07:54:59.630258+0000","last_peered":"2024-09-21T07:54:59.630258+0000","last_clean":"2024-09-21T07:54:59.630258+0000","last_became_active":"2024-09-21T07:35:27.864136+0000","last_became_peered":"2024-09-21T07:35:27.864136+0000","last_unstale":"2024-09-21T07:54:59.630258+0000","last_undegraded":"2024-09-21T07:54:59.630258+0000","last_fullsized":"2024-09-21T07:54:59.630258+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:20.162240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":427,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.632799+0000","last_change":"2024-09-21T07:35:28.858784+0000","last_active":"2024-09-21T07:54:59.632799+0000","last_peered":"2024-09-21T07:54:59.632799+0000","last_clean":"2024-09-21T07:54:59.632799+0000","last_became_active":"2024-09-21T07:35:26.876890+0000","last_became_peered":"2024-09-21T07:35:26.876890+0000","last_unstale":"2024-09-21T07:54:59.632799+0000","last_undegraded":"2024-09-21T07:54:59.632799+0000","last_fullsized":"2024-09-21T07:54:59.632799+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:40.738327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064543499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":466,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.313937+0000","last_change":"2024-09-21T07:34:24.258712+0000","last_active":"2024-09-21T07:55:00.313937+0000","last_peered":"2024-09-21T07:55:00.313937+0000","last_clean":"2024-09-21T07:55:00.313937+0000","last_became_active":"2024-09-21T07:34:23.274692+0000","last_became_peered":"2024-09-21T07:34:23.274692+0000","last_unstale":"2024-09-21T07:55:00.313937+0000","last_undegraded":"2024-09-21T07:55:00.313937+0000","last_fullsized":"2024-09-21T07:55:00.313937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:43.279454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018051479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":406,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.312289+0000","last_change":"2024-09-21T07:35:27.878020+0000","last_active":"2024-09-21T07:55:00.312289+0000","last_peered":"2024-09-21T07:55:00.312289+0000","last_clean":"2024-09-21T07:55:00.312289+0000","last_became_active":"2024-09-21T07:35:27.877532+0000","last_became_peered":"2024-09-21T07:35:27.877532+0000","last_unstale":"2024-09-21T07:55:00.312289+0000","last_undegraded":"2024-09-21T07:55:00.312289+0000","last_fullsized":"2024-09-21T07:55:00.312289+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:19.756073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":413,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.633494+0000","last_change":"2024-09-21T07:35:28.861049+0000","last_active":"2024-09-21T07:54:59.633494+0000","last_peered":"2024-09-21T07:54:59.633494+0000","last_clean":"2024-09-21T07:54:59.633494+0000","last_became_active":"2024-09-21T07:35:26.876819+0000","last_became_peered":"2024-09-21T07:35:26.876819+0000","last_unstale":"2024-09-21T07:54:59.633494+0000","last_undegraded":"2024-09-21T07:54:59.633494+0000","last_fullsized":"2024-09-21T07:54:59.633494+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:19.633803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055780599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":463,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.630568+0000","last_change":"2024-09-21T07:34:25.084849+0000","last_active":"2024-09-21T07:54:59.630568+0000","last_peered":"2024-09-21T07:54:59.630568+0000","last_clean":"2024-09-21T07:54:59.630568+0000","last_became_active":"2024-09-21T07:34:23.277713+0000","last_became_peered":"2024-09-21T07:34:23.277713+0000","last_unstale":"2024-09-21T07:54:59.630568+0000","last_undegraded":"2024-09-21T07:54:59.630568+0000","last_fullsized":"2024-09-21T07:54:59.630568+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:05.811810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059079499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":406,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.312851+0000","last_change":"2024-09-21T07:35:27.884248+0000","last_active":"2024-09-21T07:55:00.312851+0000","last_peered":"2024-09-21T07:55:00.312851+0000","last_clean":"2024-09-21T07:55:00.312851+0000","last_became_active":"2024-09-21T07:35:27.883861+0000","last_became_peered":"2024-09-21T07:35:27.883861+0000","last_unstale":"2024-09-21T07:55:00.312851+0000","last_undegraded":"2024-09-21T07:55:00.312851+0000","last_fullsized":"2024-09-21T07:55:00.312851+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":415,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.318311+0000","last_change":"2024-09-21T07:35:29.207135+0000","last_active":"2024-09-21T07:55:00.318311+0000","last_peered":"2024-09-21T07:55:00.318311+0000","last_clean":"2024-09-21T07:55:00.318311+0000","last_became_active":"2024-09-21T07:35:26.872787+0000","last_became_peered":"2024-09-21T07:35:26.872787+0000","last_unstale":"2024-09-21T07:55:00.318311+0000","last_undegraded":"2024-09-21T07:55:00.318311+0000","last_fullsized":"2024-09-21T07:55:00.318311+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:27.543217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011157509999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":479,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.630798+0000","last_change":"2024-09-21T07:34:24.538610+0000","last_active":"2024-09-21T07:54:59.630798+0000","last_peered":"2024-09-21T07:54:59.630798+0000","last_clean":"2024-09-21T07:54:59.630798+0000","last_became_active":"2024-09-21T07:34:23.259700+0000","last_became_peered":"2024-09-21T07:34:23.259700+0000","last_unstale":"2024-09-21T07:54:59.630798+0000","last_undegraded":"2024-09-21T07:54:59.630798+0000","last_fullsized":"2024-09-21T07:54:59.630798+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:57.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070522399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":420,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.630936+0000","last_change":"2024-09-21T07:35:27.868024+0000","last_active":"2024-09-21T07:54:59.630936+0000","last_peered":"2024-09-21T07:54:59.630936+0000","last_clean":"2024-09-21T07:54:59.630936+0000","last_became_active":"2024-09-21T07:35:27.867339+0000","last_became_peered":"2024-09-21T07:35:27.867339+0000","last_unstale":"2024-09-21T07:54:59.630936+0000","last_undegraded":"2024-09-21T07:54:59.630936+0000","last_fullsized":"2024-09-21T07:54:59.630936+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":427,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.631794+0000","last_change":"2024-09-21T07:35:28.858505+0000","last_active":"2024-09-21T07:54:59.631794+0000","last_peered":"2024-09-21T07:54:59.631794+0000","last_clean":"2024-09-21T07:54:59.631794+0000","last_became_active":"2024-09-21T07:35:26.877320+0000","last_became_peered":"2024-09-21T07:35:26.877320+0000","last_unstale":"2024-09-21T07:54:59.631794+0000","last_undegraded":"2024-09-21T07:54:59.631794+0000","last_fullsized":"2024-09-21T07:54:59.631794+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:53.672782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244119999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":457,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.175837+0000","last_change":"2024-09-21T07:34:24.597597+0000","last_active":"2024-09-21T07:55:00.175837+0000","last_peered":"2024-09-21T07:55:00.175837+0000","last_clean":"2024-09-21T07:55:00.175837+0000","last_became_active":"2024-09-21T07:34:23.635204+0000","last_became_peered":"2024-09-21T07:34:23.635204+0000","last_unstale":"2024-09-21T07:55:00.175837+0000","last_undegraded":"2024-09-21T07:55:00.175837+0000","last_fullsized":"2024-09-21T07:55:00.175837+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:48.644789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081139199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":408,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.316004+0000","last_change":"2024-09-21T07:35:27.877503+0000","last_active":"2024-09-21T07:55:00.316004+0000","last_peered":"2024-09-21T07:55:00.316004+0000","last_clean":"2024-09-21T07:55:00.316004+0000","last_became_active":"2024-09-21T07:35:27.875775+0000","last_became_peered":"2024-09-21T07:35:27.875775+0000","last_unstale":"2024-09-21T07:55:00.316004+0000","last_undegraded":"2024-09-21T07:55:00.316004+0000","last_fullsized":"2024-09-21T07:55:00.316004+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:49.291023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":410,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.311850+0000","last_change":"2024-09-21T07:35:29.491412+0000","last_active":"2024-09-21T07:55:00.311850+0000","last_peered":"2024-09-21T07:55:00.311850+0000","last_clean":"2024-09-21T07:55:00.311850+0000","last_became_active":"2024-09-21T07:35:26.876404+0000","last_became_peered":"2024-09-21T07:35:26.876404+0000","last_unstale":"2024-09-21T07:55:00.311850+0000","last_undegraded":"2024-09-21T07:55:00.311850+0000","last_fullsized":"2024-09-21T07:55:00.311850+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063918000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":467,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.314021+0000","last_change":"2024-09-21T07:34:24.259026+0000","last_active":"2024-09-21T07:55:00.314021+0000","last_peered":"2024-09-21T07:55:00.314021+0000","last_clean":"2024-09-21T07:55:00.314021+0000","last_became_active":"2024-09-21T07:34:23.635456+0000","last_became_peered":"2024-09-21T07:34:23.635456+0000","last_unstale":"2024-09-21T07:55:00.314021+0000","last_undegraded":"2024-09-21T07:55:00.314021+0000","last_fullsized":"2024-09-21T07:55:00.314021+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:09.592752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018654419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":408,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.316264+0000","last_change":"2024-09-21T07:35:27.871984+0000","last_active":"2024-09-21T07:55:00.316264+0000","last_peered":"2024-09-21T07:55:00.316264+0000","last_clean":"2024-09-21T07:55:00.316264+0000","last_became_active":"2024-09-21T07:35:27.870336+0000","last_became_peered":"2024-09-21T07:35:27.870336+0000","last_unstale":"2024-09-21T07:55:00.316264+0000","last_undegraded":"2024-09-21T07:55:00.316264+0000","last_fullsized":"2024-09-21T07:55:00.316264+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:57.120479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":413,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.631715+0000","last_change":"2024-09-21T07:35:28.860351+0000","last_active":"2024-09-21T07:54:59.631715+0000","last_peered":"2024-09-21T07:54:59.631715+0000","last_clean":"2024-09-21T07:54:59.631715+0000","last_became_active":"2024-09-21T07:35:26.872046+0000","last_became_peered":"2024-09-21T07:35:26.872046+0000","last_unstale":"2024-09-21T07:54:59.631715+0000","last_undegraded":"2024-09-21T07:54:59.631715+0000","last_fullsized":"2024-09-21T07:54:59.631715+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:47.074327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098384399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":463,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.315458+0000","last_change":"2024-09-21T07:34:24.260300+0000","last_active":"2024-09-21T07:55:00.315458+0000","last_peered":"2024-09-21T07:55:00.315458+0000","last_clean":"2024-09-21T07:55:00.315458+0000","last_became_active":"2024-09-21T07:34:23.270128+0000","last_became_peered":"2024-09-21T07:34:23.270128+0000","last_unstale":"2024-09-21T07:55:00.315458+0000","last_undegraded":"2024-09-21T07:55:00.315458+0000","last_fullsized":"2024-09-21T07:55:00.315458+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:25.260931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00176651,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":408,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.318010+0000","last_change":"2024-09-21T07:35:27.877861+0000","last_active":"2024-09-21T07:55:00.318010+0000","last_peered":"2024-09-21T07:55:00.318010+0000","last_clean":"2024-09-21T07:55:00.318010+0000","last_became_active":"2024-09-21T07:35:27.877434+0000","last_became_peered":"2024-09-21T07:35:27.877434+0000","last_unstale":"2024-09-21T07:55:00.318010+0000","last_undegraded":"2024-09-21T07:55:00.318010+0000","last_fullsized":"2024-09-21T07:55:00.318010+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":412,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.630743+0000","last_change":"2024-09-21T07:35:28.862954+0000","last_active":"2024-09-21T07:54:59.630743+0000","last_peered":"2024-09-21T07:54:59.630743+0000","last_clean":"2024-09-21T07:54:59.630743+0000","last_became_active":"2024-09-21T07:35:26.870613+0000","last_became_peered":"2024-09-21T07:35:26.870613+0000","last_unstale":"2024-09-21T07:54:59.630743+0000","last_undegraded":"2024-09-21T07:54:59.630743+0000","last_fullsized":"2024-09-21T07:54:59.630743+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:06.901043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059842100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":464,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.314518+0000","last_change":"2024-09-21T07:34:25.028904+0000","last_active":"2024-09-21T07:55:00.314518+0000","last_peered":"2024-09-21T07:55:00.314518+0000","last_clean":"2024-09-21T07:55:00.314518+0000","last_became_active":"2024-09-21T07:34:23.260942+0000","last_became_peered":"2024-09-21T07:34:23.260942+0000","last_unstale":"2024-09-21T07:55:00.314518+0000","last_undegraded":"2024-09-21T07:55:00.314518+0000","last_fullsized":"2024-09-21T07:55:00.314518+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:22.709328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060553899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509707,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71188,"kb_used_data":3900,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94300652,"statfs":{"total":96636764160,"available":96563867648,"internally_reserved":0,"allocated":3993600,"data_stored":2441691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509709,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74768,"kb_used_data":3320,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297072,"statfs":{"total":96636764160,"available":96560201728,"internally_reserved":0,"allocated":3399680,"data_stored":1851323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509707,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52556,"kb_used_data":3320,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319284,"statfs":{"total":96636764160,"available":96582946816,"internally_reserved":0,"allocated":3399680,"data_stored":1851323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509708,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30348,"kb_used_data":3320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341492,"statfs":{"total":96636764160,"available":96605687808,"internally_reserved":0,"allocated":3399680,"data_stored":1851323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509709,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48396,"kb_used_data":3320,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323444,"statfs":{"total":96636764160,"available":96587206656,"internally_reserved":0,"allocated":3399680,"data_stored":1851323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509709,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57300,"kb_used_data":3900,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314540,"statfs":{"total":96636764160,"available":96578088960,"internally_reserved":0,"allocated":3993600,"data_stored":2441691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542412,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56720,"kb_used_data":3320,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315120,"statfs":{"total":96636764160,"available":96578682880,"internally_reserved":0,"allocated":3399680,"data_stored":1851323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542411,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74768,"kb_used_data":3320,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297072,"statfs":{"total":96636764160,"available":96560201728,"internally_reserved":0,"allocated":3399680,"data_stored":1851323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:55:05.129 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:55:05.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.389+0000 7fad37e00640 1 -- 172.21.15.39:0/1230793479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad300b30a0 msgr2=0x7fad300cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:05.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.389+0000 7fad37e00640 1 --2- 172.21.15.39:0/1230793479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad300b30a0 0x7fad300cb8a0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fad24009830 tx=0x7fad2402a480 comp rx=0 tx=0).stop 2024-09-21T07:55:05.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.389+0000 7fad37e00640 1 -- 172.21.15.39:0/1230793479 shutdown_connections 2024-09-21T07:55:05.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.389+0000 7fad37e00640 1 --2- 172.21.15.39:0/1230793479 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fad3015d830 0x7fad30156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:05.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.389+0000 7fad37e00640 1 --2- 172.21.15.39:0/1230793479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad300b30a0 0x7fad300cb8a0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:05.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.389+0000 7fad37e00640 1 -- 172.21.15.39:0/1230793479 >> 172.21.15.39:0/1230793479 conn(0x7fad300b2460 msgr2=0x7fad3006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:05.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.389+0000 7fad37e00640 1 -- 172.21.15.39:0/1230793479 shutdown_connections 2024-09-21T07:55:05.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.389+0000 7fad37e00640 1 -- 172.21.15.39:0/1230793479 wait complete. 2024-09-21T07:55:05.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.391+0000 7fad37e00640 1 Processor -- start 2024-09-21T07:55:05.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.391+0000 7fad37e00640 1 -- start start 2024-09-21T07:55:05.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.393+0000 7fad37e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad300b30a0 0x7fad300c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:05.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.393+0000 7fad37e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fad300ccc60 0x7fad300c5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:05.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.393+0000 7fad37e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad3015d830 0x7fad300c6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:05.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.393+0000 7fad37e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fad300af450 con 0x7fad3015d830 2024-09-21T07:55:05.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.393+0000 7fad37e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fad300af2d0 con 0x7fad300b30a0 2024-09-21T07:55:05.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.393+0000 7fad37e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fad300af5d0 con 0x7fad300ccc60 2024-09-21T07:55:05.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.393+0000 7fad35e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad3015d830 0x7fad300c6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:05.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.393+0000 7fad35e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad3015d830 0x7fad300c6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42422/0 (socket says 172.21.15.39:42422) 2024-09-21T07:55:05.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.393+0000 7fad35e00640 1 -- 172.21.15.39:0/1073173906 learned_addr learned my addr 172.21.15.39:0/1073173906 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:55:05.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.393+0000 7fad35400640 1 --2- 172.21.15.39:0/1073173906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad300b30a0 0x7fad300c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:05.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.393+0000 7fad34a00640 1 --2- 172.21.15.39:0/1073173906 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fad300ccc60 0x7fad300c5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:05.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.394+0000 7fad35e00640 1 -- 172.21.15.39:0/1073173906 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fad300ccc60 msgr2=0x7fad300c5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:05.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.394+0000 7fad35e00640 1 --2- 172.21.15.39:0/1073173906 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fad300ccc60 0x7fad300c5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:05.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.394+0000 7fad35e00640 1 -- 172.21.15.39:0/1073173906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad300b30a0 msgr2=0x7fad300c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:05.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.394+0000 7fad35e00640 1 --2- 172.21.15.39:0/1073173906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad300b30a0 0x7fad300c9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:05.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.394+0000 7fad35e00640 1 -- 172.21.15.39:0/1073173906 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad30156580 con 0x7fad3015d830 2024-09-21T07:55:05.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.394+0000 7fad35e00640 1 --2- 172.21.15.39:0/1073173906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad3015d830 0x7fad300c6440 secure :-1 s=READY pgs=2890 cs=0 l=1 rev1=1 crypto rx=0x7fad2800cae0 tx=0x7fad2800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:05.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.394+0000 7fad2d600640 1 -- 172.21.15.39:0/1073173906 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad28016020 con 0x7fad3015d830 2024-09-21T07:55:05.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.394+0000 7fad2d600640 1 -- 172.21.15.39:0/1073173906 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad2800ad30 con 0x7fad3015d830 2024-09-21T07:55:05.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.394+0000 7fad37e00640 1 -- 172.21.15.39:0/1073173906 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad300c43f0 con 0x7fad3015d830 2024-09-21T07:55:05.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.394+0000 7fad2d600640 1 -- 172.21.15.39:0/1073173906 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad28012680 con 0x7fad3015d830 2024-09-21T07:55:05.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.395+0000 7fad37e00640 1 -- 172.21.15.39:0/1073173906 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad300c49b0 con 0x7fad3015d830 2024-09-21T07:55:05.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.396+0000 7fad37e00640 1 -- 172.21.15.39:0/1073173906 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad300afa30 con 0x7fad3015d830 2024-09-21T07:55:05.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.397+0000 7fad2d600640 1 -- 172.21.15.39:0/1073173906 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fad28019020 con 0x7fad3015d830 2024-09-21T07:55:05.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.398+0000 7fad2d600640 1 --2- 172.21.15.39:0/1073173906 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fad10078b90 0x7fad1007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:05.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.399+0000 7fad2d600640 1 -- 172.21.15.39:0/1073173906 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(269..269 src has 1..269) ==== 6278+0+0 (secure 0 0 0) 0x7fad2809b3e0 con 0x7fad3015d830 2024-09-21T07:55:05.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.399+0000 7fad35400640 1 --2- 172.21.15.39:0/1073173906 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fad10078b90 0x7fad1007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:05.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.400+0000 7fad35400640 1 --2- 172.21.15.39:0/1073173906 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fad10078b90 0x7fad1007b0a0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fad24034000 tx=0x7fad24004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:05.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.403+0000 7fad2d600640 1 -- 172.21.15.39:0/1073173906 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fad28020020 con 0x7fad3015d830 2024-09-21T07:55:05.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.598+0000 7fad37e00640 1 -- 172.21.15.39:0/1073173906 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad3015d610 con 0x7fad3015d830 2024-09-21T07:55:05.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.600+0000 7fad2d600640 1 -- 172.21.15.39:0/1073173906 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v269) ==== 74+0+18899 (secure 0 0 0) 0x7fad30156580 con 0x7fad3015d830 2024-09-21T07:55:05.676 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:55:05.676 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":269,"fsid":"7ab9b04a-f88d-49aa-a493-e5c076637ffb","created":"2024-09-21T07:32:46.357679+0000","modified":"2024-09-21T07:54:59.624772+0000","last_up_change":"2024-09-21T07:33:02.438318+0000","last_in_change":"2024-09-21T07:32:52.128828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:33:01.569433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"snap-test-pool","create_time":"2024-09-21T07:34:22.032185+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":37,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"test_data_pool","create_time":"2024-09-21T07:35:25.604532+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":1,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_metadata_pool","create_time":"2024-09-21T07:35:26.327607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0218dc78-8164-452b-8728-359d39705d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":266,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6809","nonce":4245419123}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6811","nonce":4245419123}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6815","nonce":4245419123}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4245419123},{"type":"v1","addr":"172.21.15.39:6813","nonce":4245419123}]},"public_addr":"172.21.15.39:6809/4245419123","cluster_addr":"172.21.15.39:6811/4245419123","heartbeat_back_addr":"172.21.15.39:6815/4245419123","heartbeat_front_addr":"172.21.15.39:6813/4245419123","state":["exists","up"]},{"osd":1,"uuid":"3c181f18-28e5-40a2-91e1-494f088472a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6825","nonce":1628627569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6827","nonce":1628627569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6831","nonce":1628627569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":1628627569},{"type":"v1","addr":"172.21.15.39:6829","nonce":1628627569}]},"public_addr":"172.21.15.39:6825/1628627569","cluster_addr":"172.21.15.39:6827/1628627569","heartbeat_back_addr":"172.21.15.39:6831/1628627569","heartbeat_front_addr":"172.21.15.39:6829/1628627569","state":["exists","up"]},{"osd":2,"uuid":"950170d9-946a-4f47-84c8-bece5afd5417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6817","nonce":3897193606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6819","nonce":3897193606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6823","nonce":3897193606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":3897193606},{"type":"v1","addr":"172.21.15.39:6821","nonce":3897193606}]},"public_addr":"172.21.15.39:6817/3897193606","cluster_addr":"172.21.15.39:6819/3897193606","heartbeat_back_addr":"172.21.15.39:6823/3897193606","heartbeat_front_addr":"172.21.15.39:6821/3897193606","state":["exists","up"]},{"osd":3,"uuid":"a8319049-b790-44d1-9cfb-ab2c2cc16953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":266,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6801","nonce":3213199177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6803","nonce":3213199177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6807","nonce":3213199177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3213199177},{"type":"v1","addr":"172.21.15.39:6805","nonce":3213199177}]},"public_addr":"172.21.15.39:6801/3213199177","cluster_addr":"172.21.15.39:6803/3213199177","heartbeat_back_addr":"172.21.15.39:6807/3213199177","heartbeat_front_addr":"172.21.15.39:6805/3213199177","state":["exists","up"]},{"osd":4,"uuid":"17c4fdd3-3d30-4e97-8319-7084cead1295","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":266,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6803","nonce":1437509971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6805","nonce":1437509971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6809","nonce":1437509971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1437509971},{"type":"v1","addr":"172.21.15.148:6807","nonce":1437509971}]},"public_addr":"172.21.15.148:6803/1437509971","cluster_addr":"172.21.15.148:6805/1437509971","heartbeat_back_addr":"172.21.15.148:6809/1437509971","heartbeat_front_addr":"172.21.15.148:6807/1437509971","state":["exists","up"]},{"osd":5,"uuid":"e46b3734-6d7b-4d66-8934-2799d12ba363","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6827","nonce":1837190792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6829","nonce":1837190792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6833","nonce":1837190792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1837190792},{"type":"v1","addr":"172.21.15.148:6831","nonce":1837190792}]},"public_addr":"172.21.15.148:6827/1837190792","cluster_addr":"172.21.15.148:6829/1837190792","heartbeat_back_addr":"172.21.15.148:6833/1837190792","heartbeat_front_addr":"172.21.15.148:6831/1837190792","state":["exists","up"]},{"osd":6,"uuid":"68bfcd21-68a0-450f-8646-dca1877516cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6811","nonce":65974579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6813","nonce":65974579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6817","nonce":65974579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":65974579},{"type":"v1","addr":"172.21.15.148:6815","nonce":65974579}]},"public_addr":"172.21.15.148:6811/65974579","cluster_addr":"172.21.15.148:6813/65974579","heartbeat_back_addr":"172.21.15.148:6817/65974579","heartbeat_front_addr":"172.21.15.148:6815/65974579","state":["exists","up"]},{"osd":7,"uuid":"bb4c236d-8a7b-45a9-8cef-303a8b1adb7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":263,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6819","nonce":1438255756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6821","nonce":1438255756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6825","nonce":1438255756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1438255756},{"type":"v1","addr":"172.21.15.148:6823","nonce":1438255756}]},"public_addr":"172.21.15.148:6819/1438255756","cluster_addr":"172.21.15.148:6821/1438255756","heartbeat_back_addr":"172.21.15.148:6825/1438255756","heartbeat_front_addr":"172.21.15.148:6823/1438255756","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.984813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.836602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.097018+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.433651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:58.496275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.518055+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.022608+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:32:59.176053+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6834/3094159467":"2024-09-22T07:54:56.598869+0000","172.21.15.39:6832/3094159467":"2024-09-22T07:54:56.598869+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":14,"snaps":[{"begin":2,"length":2}]},{"pool":18,"snaps":[{"begin":75,"length":27},{"begin":175,"length":27}]},{"pool":20,"snaps":[{"begin":80,"length":23},{"begin":181,"length":23}]},{"pool":22,"snaps":[{"begin":100,"length":1},{"begin":199,"length":1}]},{"pool":24,"snaps":[{"begin":2,"length":2}]},{"pool":26,"snaps":[{"begin":2,"length":2}]},{"pool":28,"snaps":[{"begin":2,"length":2}]},{"pool":30,"snaps":[{"begin":8,"length":6}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:55:05.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.602+0000 7fad37e00640 1 -- 172.21.15.39:0/1073173906 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fad10078b90 msgr2=0x7fad1007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:05.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.602+0000 7fad37e00640 1 --2- 172.21.15.39:0/1073173906 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fad10078b90 0x7fad1007b0a0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fad24034000 tx=0x7fad24004730 comp rx=0 tx=0).stop 2024-09-21T07:55:05.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.602+0000 7fad37e00640 1 -- 172.21.15.39:0/1073173906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad3015d830 msgr2=0x7fad300c6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:05.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.602+0000 7fad37e00640 1 --2- 172.21.15.39:0/1073173906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad3015d830 0x7fad300c6440 secure :-1 s=READY pgs=2890 cs=0 l=1 rev1=1 crypto rx=0x7fad2800cae0 tx=0x7fad2800a060 comp rx=0 tx=0).stop 2024-09-21T07:55:05.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.602+0000 7fad37e00640 1 -- 172.21.15.39:0/1073173906 shutdown_connections 2024-09-21T07:55:05.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.602+0000 7fad37e00640 1 --2- 172.21.15.39:0/1073173906 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fad10078b90 0x7fad1007b0a0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:05.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.602+0000 7fad37e00640 1 --2- 172.21.15.39:0/1073173906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad3015d830 0x7fad300c6440 unknown :-1 s=CLOSED pgs=2890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:05.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.602+0000 7fad37e00640 1 --2- 172.21.15.39:0/1073173906 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fad300ccc60 0x7fad300c5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:05.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.602+0000 7fad37e00640 1 --2- 172.21.15.39:0/1073173906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad300b30a0 0x7fad300c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:05.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.602+0000 7fad37e00640 1 -- 172.21.15.39:0/1073173906 >> 172.21.15.39:0/1073173906 conn(0x7fad300b2460 msgr2=0x7fad300ad540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:05.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.602+0000 7fad37e00640 1 -- 172.21.15.39:0/1073173906 shutdown_connections 2024-09-21T07:55:05.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:05.602+0000 7fad37e00640 1 -- 172.21.15.39:0/1073173906 wait complete. 2024-09-21T07:55:06.680 INFO:tasks.ceph:Scrubbing osd.0 2024-09-21T07:55:06.680 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-21T07:55:06.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.921+0000 7f787a800640 1 -- 172.21.15.39:0/522235486 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f787406f020 msgr2=0x7f787406f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:06.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.921+0000 7f787a800640 1 --2- 172.21.15.39:0/522235486 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f787406f020 0x7f787406f430 secure :-1 s=READY pgs=2892 cs=0 l=1 rev1=1 crypto rx=0x7f786c005fd0 tx=0x7f786c02a940 comp rx=0 tx=0).stop 2024-09-21T07:55:06.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.921+0000 7f787a800640 1 -- 172.21.15.39:0/522235486 shutdown_connections 2024-09-21T07:55:06.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.921+0000 7f787a800640 1 --2- 172.21.15.39:0/522235486 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f787406f020 0x7f787406f430 unknown :-1 s=CLOSED pgs=2892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:06.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.921+0000 7f787a800640 1 --2- 172.21.15.39:0/522235486 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78740cc190 0x7f78740cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:06.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.921+0000 7f787a800640 1 -- 172.21.15.39:0/522235486 >> 172.21.15.39:0/522235486 conn(0x7f78740b2640 msgr2=0x7f78740d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:06.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.921+0000 7f787a800640 1 -- 172.21.15.39:0/522235486 shutdown_connections 2024-09-21T07:55:06.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.921+0000 7f787a800640 1 -- 172.21.15.39:0/522235486 wait complete. 2024-09-21T07:55:06.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.923+0000 7f787a800640 1 Processor -- start 2024-09-21T07:55:06.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.923+0000 7f787a800640 1 -- start start 2024-09-21T07:55:06.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.925+0000 7f787a800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f787406f020 0x7f78740c6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:06.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.925+0000 7f787a800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78740b35f0 0x7f787409d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:06.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.925+0000 7f787a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78740cc190 0x7f787415d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:06.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.925+0000 7f787a800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f787406fe10 con 0x7f78740cc190 2024-09-21T07:55:06.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.925+0000 7f787a800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f787406fc90 con 0x7f787406f020 2024-09-21T07:55:06.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.925+0000 7f787a800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f787406ff90 con 0x7f78740b35f0 2024-09-21T07:55:06.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.925+0000 7f7873e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78740cc190 0x7f787415d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:06.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.925+0000 7f7873e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78740cc190 0x7f787415d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42448/0 (socket says 172.21.15.39:42448) 2024-09-21T07:55:06.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.925+0000 7f7873e00640 1 -- 172.21.15.39:0/3946732597 learned_addr learned my addr 172.21.15.39:0/3946732597 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:55:06.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.925+0000 7f786be00640 1 --2- 172.21.15.39:0/3946732597 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78740b35f0 0x7f787409d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:06.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.925+0000 7f7873400640 1 --2- 172.21.15.39:0/3946732597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f787406f020 0x7f78740c6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:06.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.926+0000 7f7873e00640 1 -- 172.21.15.39:0/3946732597 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78740b35f0 msgr2=0x7f787409d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:06.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.926+0000 7f7873e00640 1 --2- 172.21.15.39:0/3946732597 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f78740b35f0 0x7f787409d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:06.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.926+0000 7f7873e00640 1 -- 172.21.15.39:0/3946732597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f787406f020 msgr2=0x7f78740c6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:06.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.926+0000 7f7873e00640 1 --2- 172.21.15.39:0/3946732597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f787406f020 0x7f78740c6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:06.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.926+0000 7f7873e00640 1 -- 172.21.15.39:0/3946732597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78740c9480 con 0x7f78740cc190 2024-09-21T07:55:06.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.926+0000 7f7873e00640 1 --2- 172.21.15.39:0/3946732597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78740cc190 0x7f787415d270 secure :-1 s=READY pgs=2893 cs=0 l=1 rev1=1 crypto rx=0x7f786c02af30 tx=0x7f786c031ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:06.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.926+0000 7f786b400640 1 -- 172.21.15.39:0/3946732597 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c03f030 con 0x7f78740cc190 2024-09-21T07:55:06.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.926+0000 7f786b400640 1 -- 172.21.15.39:0/3946732597 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f786c004250 con 0x7f78740cc190 2024-09-21T07:55:06.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.926+0000 7f786b400640 1 -- 172.21.15.39:0/3946732597 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c004680 con 0x7f78740cc190 2024-09-21T07:55:06.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.926+0000 7f787a800640 1 -- 172.21.15.39:0/3946732597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78740c9760 con 0x7f78740cc190 2024-09-21T07:55:06.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.926+0000 7f787a800640 1 -- 172.21.15.39:0/3946732597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78740c9d20 con 0x7f78740cc190 2024-09-21T07:55:06.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.927+0000 7f787a800640 1 -- 172.21.15.39:0/3946732597 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f787406f430 con 0x7f78740cc190 2024-09-21T07:55:06.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.929+0000 7f786b400640 1 -- 172.21.15.39:0/3946732597 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f786c042030 con 0x7f78740cc190 2024-09-21T07:55:06.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.930+0000 7f786b400640 1 --2- 172.21.15.39:0/3946732597 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7850078bc0 0x7f785007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:06.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.930+0000 7f786b400640 1 -- 172.21.15.39:0/3946732597 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(269..269 src has 1..269) ==== 6278+0+0 (secure 0 0 0) 0x7f786c0bc120 con 0x7f78740cc190 2024-09-21T07:55:06.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.930+0000 7f786b400640 1 --2- 172.21.15.39:0/3946732597 >> [v2:172.21.15.39:6808/4245419123,v1:172.21.15.39:6809/4245419123] conn(0x7f7850081490 0x7f7850083980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:06.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.930+0000 7f786b400640 1 -- 172.21.15.39:0/3946732597 --> [v2:172.21.15.39:6808/4245419123,v1:172.21.15.39:6809/4245419123] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7850084050 con 0x7f7850081490 2024-09-21T07:55:06.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.930+0000 7f786b400640 1 -- 172.21.15.39:0/3946732597 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=269) ==== 24+0+0 (secure 0 0 0) 0x7f786c0c3050 con 0x7f78740cc190 2024-09-21T07:55:06.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.930+0000 7f786be00640 1 --2- 172.21.15.39:0/3946732597 >> [v2:172.21.15.39:6808/4245419123,v1:172.21.15.39:6809/4245419123] conn(0x7f7850081490 0x7f7850083980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:06.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.931+0000 7f7873400640 1 --2- 172.21.15.39:0/3946732597 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7850078bc0 0x7f785007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:06.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.931+0000 7f786be00640 1 --2- 172.21.15.39:0/3946732597 >> [v2:172.21.15.39:6808/4245419123,v1:172.21.15.39:6809/4245419123] conn(0x7f7850081490 0x7f7850083980 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:06.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.932+0000 7f7873400640 1 --2- 172.21.15.39:0/3946732597 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7850078bc0 0x7f785007b0d0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f7864005fd0 tx=0x7f7864005850 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:06.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.932+0000 7f786b400640 1 -- 172.21.15.39:0/3946732597 <== osd.0 v2:172.21.15.39:6808/4245419123 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7850084050 con 0x7f7850081490 2024-09-21T07:55:06.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.945+0000 7f787a800640 1 -- 172.21.15.39:0/3946732597 --> [v2:172.21.15.39:6808/4245419123,v1:172.21.15.39:6809/4245419123] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f78740c6dd0 con 0x7f7850081490 2024-09-21T07:55:06.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.945+0000 7f786b400640 1 -- 172.21.15.39:0/3946732597 <== osd.0 v2:172.21.15.39:6808/4245419123 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f78740c6dd0 con 0x7f7850081490 2024-09-21T07:55:06.948 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:55:06.948 INFO:teuthology.orchestra.run.smithi039.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T07:55:06.948 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:55:06.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.946+0000 7f787a800640 1 -- 172.21.15.39:0/3946732597 >> [v2:172.21.15.39:6808/4245419123,v1:172.21.15.39:6809/4245419123] conn(0x7f7850081490 msgr2=0x7f7850083980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:06.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.946+0000 7f787a800640 1 --2- 172.21.15.39:0/3946732597 >> [v2:172.21.15.39:6808/4245419123,v1:172.21.15.39:6809/4245419123] conn(0x7f7850081490 0x7f7850083980 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:06.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.946+0000 7f787a800640 1 -- 172.21.15.39:0/3946732597 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7850078bc0 msgr2=0x7f785007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:06.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.946+0000 7f787a800640 1 --2- 172.21.15.39:0/3946732597 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7850078bc0 0x7f785007b0d0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f7864005fd0 tx=0x7f7864005850 comp rx=0 tx=0).stop 2024-09-21T07:55:06.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.946+0000 7f787a800640 1 -- 172.21.15.39:0/3946732597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78740cc190 msgr2=0x7f787415d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:06.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.946+0000 7f787a800640 1 --2- 172.21.15.39:0/3946732597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78740cc190 0x7f787415d270 secure :-1 s=READY pgs=2893 cs=0 l=1 rev1=1 crypto rx=0x7f786c02af30 tx=0x7f786c031ea0 comp rx=0 tx=0).stop 2024-09-21T07:55:06.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.946+0000 7f7873e00640 1 -- 172.21.15.39:0/3946732597 reap_dead start 2024-09-21T07:55:06.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.946+0000 7f787a800640 1 -- 172.21.15.39:0/3946732597 shutdown_connections 2024-09-21T07:55:06.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.946+0000 7f787a800640 1 -- 172.21.15.39:0/3946732597 >> 172.21.15.39:0/3946732597 conn(0x7f78740b2640 msgr2=0x7f7874074e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:06.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.946+0000 7f787a800640 1 -- 172.21.15.39:0/3946732597 shutdown_connections 2024-09-21T07:55:06.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:06.946+0000 7f787a800640 1 -- 172.21.15.39:0/3946732597 wait complete. 2024-09-21T07:55:06.969 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-21T07:55:07.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.195+0000 7f59f0800640 1 -- 172.21.15.39:0/4087573945 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59e80cb8a0 msgr2=0x7f59e8156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:07.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.195+0000 7f59f0800640 1 --2- 172.21.15.39:0/4087573945 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59e80cb8a0 0x7f59e8156bb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f59d8009830 tx=0x7f59d802a480 comp rx=0 tx=0).stop 2024-09-21T07:55:07.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.195+0000 7f59f0800640 1 -- 172.21.15.39:0/4087573945 shutdown_connections 2024-09-21T07:55:07.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.195+0000 7f59f0800640 1 --2- 172.21.15.39:0/4087573945 >> v2:172.21.15.148:3300/0 conn(0x7f59e80ccc60 0x7f59e80cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:07.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.195+0000 7f59f0800640 1 --2- 172.21.15.39:0/4087573945 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59e80cb8a0 0x7f59e8156bb0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:07.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.195+0000 7f59f0800640 1 -- 172.21.15.39:0/4087573945 >> 172.21.15.39:0/4087573945 conn(0x7f59e80b2460 msgr2=0x7f59e806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:07.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.195+0000 7f59f0800640 1 -- 172.21.15.39:0/4087573945 shutdown_connections 2024-09-21T07:55:07.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.195+0000 7f59f0800640 1 -- 172.21.15.39:0/4087573945 wait complete. 2024-09-21T07:55:07.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.197+0000 7f59f0800640 1 Processor -- start 2024-09-21T07:55:07.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.197+0000 7f59f0800640 1 -- start start 2024-09-21T07:55:07.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.199+0000 7f59f0800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59e80b30a0 0x7f59e81f52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:07.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.199+0000 7f59f0800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59e80cb8a0 0x7f59e815e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:07.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.199+0000 7f59f0800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59e80ccc60 0x7f59e815edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:07.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.199+0000 7f59f0800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f59e80af410 con 0x7f59e80b30a0 2024-09-21T07:55:07.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.199+0000 7f59f0800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f59e80af290 con 0x7f59e80cb8a0 2024-09-21T07:55:07.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.199+0000 7f59f0800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f59e80af590 con 0x7f59e80ccc60 2024-09-21T07:55:07.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.199+0000 7f59ede00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59e80b30a0 0x7f59e81f52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:07.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.199+0000 7f59ede00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59e80b30a0 0x7f59e81f52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42458/0 (socket says 172.21.15.39:42458) 2024-09-21T07:55:07.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.199+0000 7f59ede00640 1 -- 172.21.15.39:0/3522912917 learned_addr learned my addr 172.21.15.39:0/3522912917 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:55:07.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.199+0000 7f59ed400640 1 --2- 172.21.15.39:0/3522912917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59e80cb8a0 0x7f59e815e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:07.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.200+0000 7f59ede00640 1 -- 172.21.15.39:0/3522912917 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59e80ccc60 msgr2=0x7f59e815edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:07.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.200+0000 7f59ede00640 1 --2- 172.21.15.39:0/3522912917 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59e80ccc60 0x7f59e815edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:07.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.200+0000 7f59ede00640 1 -- 172.21.15.39:0/3522912917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59e80cb8a0 msgr2=0x7f59e815e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:07.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.200+0000 7f59ede00640 1 --2- 172.21.15.39:0/3522912917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59e80cb8a0 0x7f59e815e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:07.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.200+0000 7f59ede00640 1 -- 172.21.15.39:0/3522912917 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59e80c7cc0 con 0x7f59e80b30a0 2024-09-21T07:55:07.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.200+0000 7f59ede00640 1 --2- 172.21.15.39:0/3522912917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59e80b30a0 0x7f59e81f52b0 secure :-1 s=READY pgs=2894 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00c760 tx=0x7f59dc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:07.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.200+0000 7f59e2000640 1 -- 172.21.15.39:0/3522912917 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59dc016020 con 0x7f59e80b30a0 2024-09-21T07:55:07.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.200+0000 7f59e2000640 1 -- 172.21.15.39:0/3522912917 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f59dc00a950 con 0x7f59e80b30a0 2024-09-21T07:55:07.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.200+0000 7f59f0800640 1 -- 172.21.15.39:0/3522912917 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59e81f57f0 con 0x7f59e80b30a0 2024-09-21T07:55:07.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.200+0000 7f59e2000640 1 -- 172.21.15.39:0/3522912917 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59dc00ada0 con 0x7f59e80b30a0 2024-09-21T07:55:07.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.200+0000 7f59f0800640 1 -- 172.21.15.39:0/3522912917 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59e80c7f60 con 0x7f59e80b30a0 2024-09-21T07:55:07.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.201+0000 7f59f0800640 1 -- 172.21.15.39:0/3522912917 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59e80afae0 con 0x7f59e80b30a0 2024-09-21T07:55:07.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.203+0000 7f59e2000640 1 -- 172.21.15.39:0/3522912917 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f59dc019020 con 0x7f59e80b30a0 2024-09-21T07:55:07.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.204+0000 7f59e2000640 1 --2- 172.21.15.39:0/3522912917 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f59cc078bc0 0x7f59cc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:07.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.204+0000 7f59e2000640 1 -- 172.21.15.39:0/3522912917 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(269..269 src has 1..269) ==== 6278+0+0 (secure 0 0 0) 0x7f59dc09b270 con 0x7f59e80b30a0 2024-09-21T07:55:07.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.204+0000 7f59ed400640 1 --2- 172.21.15.39:0/3522912917 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f59cc078bc0 0x7f59cc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:07.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.205+0000 7f59ed400640 1 --2- 172.21.15.39:0/3522912917 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f59cc078bc0 0x7f59cc07b0d0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f59d8034000 tx=0x7f59d8004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:07.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.207+0000 7f59e2000640 1 -- 172.21.15.39:0/3522912917 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f59dc020020 con 0x7f59e80b30a0 2024-09-21T07:55:07.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.406+0000 7f59f0800640 1 -- 172.21.15.39:0/3522912917 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f59e80adac0 con 0x7f59cc078bc0 2024-09-21T07:55:07.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.407+0000 7f59e2000640 1 -- 172.21.15.39:0/3522912917 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f59e80adac0 con 0x7f59cc078bc0 2024-09-21T07:55:07.409 INFO:teuthology.orchestra.run.smithi039.stderr:instructed osd(s) 0 to deep-scrub 2024-09-21T07:55:07.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.409+0000 7f59f0800640 1 -- 172.21.15.39:0/3522912917 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f59cc078bc0 msgr2=0x7f59cc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:07.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.409+0000 7f59f0800640 1 --2- 172.21.15.39:0/3522912917 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f59cc078bc0 0x7f59cc07b0d0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f59d8034000 tx=0x7f59d8004730 comp rx=0 tx=0).stop 2024-09-21T07:55:07.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.409+0000 7f59f0800640 1 -- 172.21.15.39:0/3522912917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59e80b30a0 msgr2=0x7f59e81f52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:07.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.409+0000 7f59f0800640 1 --2- 172.21.15.39:0/3522912917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59e80b30a0 0x7f59e81f52b0 secure :-1 s=READY pgs=2894 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00c760 tx=0x7f59dc00cc20 comp rx=0 tx=0).stop 2024-09-21T07:55:07.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.409+0000 7f59f0800640 1 -- 172.21.15.39:0/3522912917 shutdown_connections 2024-09-21T07:55:07.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.409+0000 7f59f0800640 1 --2- 172.21.15.39:0/3522912917 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f59cc078bc0 0x7f59cc07b0d0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:07.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.409+0000 7f59f0800640 1 --2- 172.21.15.39:0/3522912917 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f59e80ccc60 0x7f59e815edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:07.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.409+0000 7f59f0800640 1 --2- 172.21.15.39:0/3522912917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59e80cb8a0 0x7f59e815e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:07.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.409+0000 7f59f0800640 1 --2- 172.21.15.39:0/3522912917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59e80b30a0 0x7f59e81f52b0 unknown :-1 s=CLOSED pgs=2894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:07.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.409+0000 7f59f0800640 1 -- 172.21.15.39:0/3522912917 >> 172.21.15.39:0/3522912917 conn(0x7f59e80b2460 msgr2=0x7f59e80b34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:07.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.409+0000 7f59f0800640 1 -- 172.21.15.39:0/3522912917 shutdown_connections 2024-09-21T07:55:07.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.409+0000 7f59f0800640 1 -- 172.21.15.39:0/3522912917 wait complete. 2024-09-21T07:55:07.433 INFO:tasks.ceph:Scrubbing osd.1 2024-09-21T07:55:07.433 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-21T07:55:07.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.661+0000 7f8d38a00640 1 -- 172.21.15.39:0/1581944185 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d34070bc0 msgr2=0x7f8d3406f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:07.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.661+0000 7f8d38a00640 1 --2- 172.21.15.39:0/1581944185 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d34070bc0 0x7f8d3406f020 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8d24005fd0 tx=0x7f8d2402a5d0 comp rx=0 tx=0).stop 2024-09-21T07:55:07.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.661+0000 7f8d38a00640 1 -- 172.21.15.39:0/1581944185 shutdown_connections 2024-09-21T07:55:07.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.661+0000 7f8d38a00640 1 --2- 172.21.15.39:0/1581944185 >> v2:172.21.15.148:3300/0 conn(0x7f8d3406f560 0x7f8d3406f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:07.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.661+0000 7f8d38a00640 1 --2- 172.21.15.39:0/1581944185 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d34070bc0 0x7f8d3406f020 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:07.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.661+0000 7f8d38a00640 1 -- 172.21.15.39:0/1581944185 >> 172.21.15.39:0/1581944185 conn(0x7f8d340b2640 msgr2=0x7f8d340d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:07.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.661+0000 7f8d38a00640 1 -- 172.21.15.39:0/1581944185 shutdown_connections 2024-09-21T07:55:07.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.661+0000 7f8d38a00640 1 -- 172.21.15.39:0/1581944185 wait complete. 2024-09-21T07:55:07.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.663+0000 7f8d38a00640 1 Processor -- start 2024-09-21T07:55:07.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.663+0000 7f8d38a00640 1 -- start start 2024-09-21T07:55:07.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.665+0000 7f8d38a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d3406f560 0x7f8d340c7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:07.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.665+0000 7f8d38a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d34070bc0 0x7f8d3415fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:07.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.665+0000 7f8d38a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d3415cc30 0x7f8d340ae300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:07.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.665+0000 7f8d38a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8d340cd670 con 0x7f8d3406f560 2024-09-21T07:55:07.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.665+0000 7f8d38a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8d340cd4f0 con 0x7f8d34070bc0 2024-09-21T07:55:07.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.665+0000 7f8d38a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f8d3415d750 con 0x7f8d3415cc30 2024-09-21T07:55:07.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.665+0000 7f8d31e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d3406f560 0x7f8d340c7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:07.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.665+0000 7f8d31e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d3406f560 0x7f8d340c7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42472/0 (socket says 172.21.15.39:42472) 2024-09-21T07:55:07.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.665+0000 7f8d31e00640 1 -- 172.21.15.39:0/1768361828 learned_addr learned my addr 172.21.15.39:0/1768361828 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:55:07.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.665+0000 7f8d32800640 1 --2- 172.21.15.39:0/1768361828 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d3415cc30 0x7f8d340ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:07.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.666+0000 7f8d31400640 1 --2- 172.21.15.39:0/1768361828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d34070bc0 0x7f8d3415fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:07.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.666+0000 7f8d31e00640 1 -- 172.21.15.39:0/1768361828 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d3415cc30 msgr2=0x7f8d340ae300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:07.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.666+0000 7f8d31e00640 1 --2- 172.21.15.39:0/1768361828 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d3415cc30 0x7f8d340ae300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:07.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.666+0000 7f8d31e00640 1 -- 172.21.15.39:0/1768361828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d34070bc0 msgr2=0x7f8d3415fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:07.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.666+0000 7f8d31e00640 1 --2- 172.21.15.39:0/1768361828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d34070bc0 0x7f8d3415fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:07.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.666+0000 7f8d31e00640 1 -- 172.21.15.39:0/1768361828 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d341f9d60 con 0x7f8d3406f560 2024-09-21T07:55:07.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.666+0000 7f8d31400640 1 --2- 172.21.15.39:0/1768361828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d34070bc0 0x7f8d3415fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:55:07.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.666+0000 7f8d32800640 1 --2- 172.21.15.39:0/1768361828 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f8d3415cc30 0x7f8d340ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:55:07.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.666+0000 7f8d31e00640 1 --2- 172.21.15.39:0/1768361828 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d3406f560 0x7f8d340c7370 secure :-1 s=READY pgs=2895 cs=0 l=1 rev1=1 crypto rx=0x7f8d24034040 tx=0x7f8d240035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:07.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.672+0000 7f8d2a000640 1 -- 172.21.15.39:0/1768361828 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d2403e030 con 0x7f8d3406f560 2024-09-21T07:55:07.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.672+0000 7f8d2a000640 1 -- 172.21.15.39:0/1768361828 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d24003ae0 con 0x7f8d3406f560 2024-09-21T07:55:07.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.672+0000 7f8d38a00640 1 -- 172.21.15.39:0/1768361828 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d341f94c0 con 0x7f8d3406f560 2024-09-21T07:55:07.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.673+0000 7f8d2a000640 1 -- 172.21.15.39:0/1768361828 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d24035040 con 0x7f8d3406f560 2024-09-21T07:55:07.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.673+0000 7f8d38a00640 1 -- 172.21.15.39:0/1768361828 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d341f9a80 con 0x7f8d3406f560 2024-09-21T07:55:07.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.673+0000 7f8d38a00640 1 -- 172.21.15.39:0/1768361828 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8d3415dc00 con 0x7f8d3406f560 2024-09-21T07:55:07.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.675+0000 7f8d2a000640 1 -- 172.21.15.39:0/1768361828 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8d24041030 con 0x7f8d3406f560 2024-09-21T07:55:07.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.676+0000 7f8d2a000640 1 --2- 172.21.15.39:0/1768361828 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8d14078b90 0x7f8d1407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:07.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.676+0000 7f8d2a000640 1 -- 172.21.15.39:0/1768361828 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(269..269 src has 1..269) ==== 6278+0+0 (secure 0 0 0) 0x7f8d240bbb10 con 0x7f8d3406f560 2024-09-21T07:55:07.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.676+0000 7f8d2a000640 1 --2- 172.21.15.39:0/1768361828 >> [v2:172.21.15.39:6824/1628627569,v1:172.21.15.39:6825/1628627569] conn(0x7f8d14081460 0x7f8d14083950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:07.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.676+0000 7f8d2a000640 1 -- 172.21.15.39:0/1768361828 --> [v2:172.21.15.39:6824/1628627569,v1:172.21.15.39:6825/1628627569] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8d2403be50 con 0x7f8d14081460 2024-09-21T07:55:07.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.676+0000 7f8d2a000640 1 -- 172.21.15.39:0/1768361828 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=269) ==== 24+0+0 (secure 0 0 0) 0x7f8d240bbfa0 con 0x7f8d3406f560 2024-09-21T07:55:07.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.676+0000 7f8d31400640 1 --2- 172.21.15.39:0/1768361828 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8d14078b90 0x7f8d1407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:07.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.677+0000 7f8d32800640 1 --2- 172.21.15.39:0/1768361828 >> [v2:172.21.15.39:6824/1628627569,v1:172.21.15.39:6825/1628627569] conn(0x7f8d14081460 0x7f8d14083950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:07.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.677+0000 7f8d32800640 1 --2- 172.21.15.39:0/1768361828 >> [v2:172.21.15.39:6824/1628627569,v1:172.21.15.39:6825/1628627569] conn(0x7f8d14081460 0x7f8d14083950 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:07.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.677+0000 7f8d31400640 1 --2- 172.21.15.39:0/1768361828 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8d14078b90 0x7f8d1407b0a0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f8d340c8810 tx=0x7f8d20005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:07.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.678+0000 7f8d2a000640 1 -- 172.21.15.39:0/1768361828 <== osd.1 v2:172.21.15.39:6824/1628627569 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d2403be50 con 0x7f8d14081460 2024-09-21T07:55:07.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.691+0000 7f8d38a00640 1 -- 172.21.15.39:0/1768361828 --> [v2:172.21.15.39:6824/1628627569,v1:172.21.15.39:6825/1628627569] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f8d340b1b60 con 0x7f8d14081460 2024-09-21T07:55:07.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.692+0000 7f8d2a000640 1 -- 172.21.15.39:0/1768361828 <== osd.1 v2:172.21.15.39:6824/1628627569 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f8d340b1b60 con 0x7f8d14081460 2024-09-21T07:55:07.694 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:55:07.694 INFO:teuthology.orchestra.run.smithi039.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T07:55:07.694 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:55:07.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.692+0000 7f8d38a00640 1 -- 172.21.15.39:0/1768361828 >> [v2:172.21.15.39:6824/1628627569,v1:172.21.15.39:6825/1628627569] conn(0x7f8d14081460 msgr2=0x7f8d14083950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:07.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.692+0000 7f8d38a00640 1 --2- 172.21.15.39:0/1768361828 >> [v2:172.21.15.39:6824/1628627569,v1:172.21.15.39:6825/1628627569] conn(0x7f8d14081460 0x7f8d14083950 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:07.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.692+0000 7f8d38a00640 1 -- 172.21.15.39:0/1768361828 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8d14078b90 msgr2=0x7f8d1407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:07.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.692+0000 7f8d38a00640 1 --2- 172.21.15.39:0/1768361828 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f8d14078b90 0x7f8d1407b0a0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f8d340c8810 tx=0x7f8d20005f90 comp rx=0 tx=0).stop 2024-09-21T07:55:07.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.692+0000 7f8d38a00640 1 -- 172.21.15.39:0/1768361828 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d3406f560 msgr2=0x7f8d340c7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:07.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.692+0000 7f8d38a00640 1 --2- 172.21.15.39:0/1768361828 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d3406f560 0x7f8d340c7370 secure :-1 s=READY pgs=2895 cs=0 l=1 rev1=1 crypto rx=0x7f8d24034040 tx=0x7f8d240035f0 comp rx=0 tx=0).stop 2024-09-21T07:55:07.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.692+0000 7f8d32800640 1 -- 172.21.15.39:0/1768361828 reap_dead start 2024-09-21T07:55:07.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.692+0000 7f8d38a00640 1 -- 172.21.15.39:0/1768361828 shutdown_connections 2024-09-21T07:55:07.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.692+0000 7f8d38a00640 1 -- 172.21.15.39:0/1768361828 >> 172.21.15.39:0/1768361828 conn(0x7f8d340b2640 msgr2=0x7f8d340b0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:07.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.692+0000 7f8d38a00640 1 -- 172.21.15.39:0/1768361828 shutdown_connections 2024-09-21T07:55:07.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.692+0000 7f8d38a00640 1 -- 172.21.15.39:0/1768361828 wait complete. 2024-09-21T07:55:07.715 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-21T07:55:07.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.940+0000 7f7505a00640 1 -- 172.21.15.39:0/3999195659 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75000b30a0 msgr2=0x7f75000cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:07.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.940+0000 7f7505a00640 1 --2- 172.21.15.39:0/3999195659 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75000b30a0 0x7f75000cb8a0 secure :-1 s=READY pgs=2896 cs=0 l=1 rev1=1 crypto rx=0x7f74f0005fd0 tx=0x7f74f002a4d0 comp rx=0 tx=0).stop 2024-09-21T07:55:07.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.940+0000 7f7505a00640 1 -- 172.21.15.39:0/3999195659 shutdown_connections 2024-09-21T07:55:07.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.940+0000 7f7505a00640 1 --2- 172.21.15.39:0/3999195659 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f75000ccc60 0x7f75000cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:07.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.940+0000 7f7505a00640 1 --2- 172.21.15.39:0/3999195659 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75000b30a0 0x7f75000cb8a0 unknown :-1 s=CLOSED pgs=2896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:07.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.940+0000 7f7505a00640 1 -- 172.21.15.39:0/3999195659 >> 172.21.15.39:0/3999195659 conn(0x7f75000b2460 msgr2=0x7f750006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:07.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.940+0000 7f7505a00640 1 -- 172.21.15.39:0/3999195659 shutdown_connections 2024-09-21T07:55:07.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.940+0000 7f7505a00640 1 -- 172.21.15.39:0/3999195659 wait complete. 2024-09-21T07:55:07.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.942+0000 7f7505a00640 1 Processor -- start 2024-09-21T07:55:07.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.942+0000 7f7505a00640 1 -- start start 2024-09-21T07:55:07.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.944+0000 7f7505a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f75000b30a0 0x7f7500099920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:07.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.944+0000 7f7505a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75000ccc60 0x7f75000c8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:07.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.944+0000 7f7505a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f750015d830 0x7f75000cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:07.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.944+0000 7f7505a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f75000af410 con 0x7f75000ccc60 2024-09-21T07:55:07.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.944+0000 7f7505a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f75000af290 con 0x7f750015d830 2024-09-21T07:55:07.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.944+0000 7f7505a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f75000af590 con 0x7f75000b30a0 2024-09-21T07:55:07.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.944+0000 7f74f7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75000ccc60 0x7f75000c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:07.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.944+0000 7f74f7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75000ccc60 0x7f75000c8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42484/0 (socket says 172.21.15.39:42484) 2024-09-21T07:55:07.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.944+0000 7f74f7e00640 1 -- 172.21.15.39:0/3043525387 learned_addr learned my addr 172.21.15.39:0/3043525387 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:55:07.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.944+0000 7f74ff400640 1 --2- 172.21.15.39:0/3043525387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f750015d830 0x7f75000cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:07.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.944+0000 7f74fea00640 1 --2- 172.21.15.39:0/3043525387 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f75000b30a0 0x7f7500099920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:07.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.945+0000 7f74f7e00640 1 -- 172.21.15.39:0/3043525387 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f75000b30a0 msgr2=0x7f7500099920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:07.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.945+0000 7f74f7e00640 1 --2- 172.21.15.39:0/3043525387 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f75000b30a0 0x7f7500099920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:07.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.945+0000 7f74f7e00640 1 -- 172.21.15.39:0/3043525387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f750015d830 msgr2=0x7f75000cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:07.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.945+0000 7f74f7e00640 1 --2- 172.21.15.39:0/3043525387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f750015d830 0x7f75000cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:07.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.945+0000 7f74f7e00640 1 -- 172.21.15.39:0/3043525387 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75000cea30 con 0x7f75000ccc60 2024-09-21T07:55:07.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.945+0000 7f74f7e00640 1 --2- 172.21.15.39:0/3043525387 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75000ccc60 0x7f75000c8240 secure :-1 s=READY pgs=2897 cs=0 l=1 rev1=1 crypto rx=0x7f74e800b410 tx=0x7f74e800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:07.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.945+0000 7f74f6a00640 1 -- 172.21.15.39:0/3043525387 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e8015020 con 0x7f75000ccc60 2024-09-21T07:55:07.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.945+0000 7f74f6a00640 1 -- 172.21.15.39:0/3043525387 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f74e8011920 con 0x7f75000ccc60 2024-09-21T07:55:07.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.945+0000 7f7505a00640 1 -- 172.21.15.39:0/3043525387 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75000c68c0 con 0x7f75000ccc60 2024-09-21T07:55:07.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.945+0000 7f74f6a00640 1 -- 172.21.15.39:0/3043525387 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e8011d70 con 0x7f75000ccc60 2024-09-21T07:55:07.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.945+0000 7f7505a00640 1 -- 172.21.15.39:0/3043525387 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75000c6ba0 con 0x7f75000ccc60 2024-09-21T07:55:07.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.946+0000 7f7505a00640 1 -- 172.21.15.39:0/3043525387 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75000afa70 con 0x7f75000ccc60 2024-09-21T07:55:07.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.947+0000 7f74f6a00640 1 -- 172.21.15.39:0/3043525387 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f74e8018020 con 0x7f75000ccc60 2024-09-21T07:55:07.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.948+0000 7f74f6a00640 1 --2- 172.21.15.39:0/3043525387 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f74dc078bc0 0x7f74dc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:07.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.949+0000 7f74f6a00640 1 -- 172.21.15.39:0/3043525387 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(269..269 src has 1..269) ==== 6278+0+0 (secure 0 0 0) 0x7f74e809af20 con 0x7f75000ccc60 2024-09-21T07:55:07.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.949+0000 7f74fea00640 1 --2- 172.21.15.39:0/3043525387 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f74dc078bc0 0x7f74dc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:07.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.951+0000 7f74fea00640 1 --2- 172.21.15.39:0/3043525387 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f74dc078bc0 0x7f74dc07b0d0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f74f0033000 tx=0x7f74f0004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:07.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:07.953+0000 7f74f6a00640 1 -- 172.21.15.39:0/3043525387 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f74e801f020 con 0x7f75000ccc60 2024-09-21T07:55:08.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.153+0000 7f7505a00640 1 -- 172.21.15.39:0/3043525387 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f75000adaa0 con 0x7f74dc078bc0 2024-09-21T07:55:08.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.154+0000 7f74f6a00640 1 -- 172.21.15.39:0/3043525387 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f75000adaa0 con 0x7f74dc078bc0 2024-09-21T07:55:08.156 INFO:teuthology.orchestra.run.smithi039.stderr:instructed osd(s) 1 to deep-scrub 2024-09-21T07:55:08.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.157+0000 7f7505a00640 1 -- 172.21.15.39:0/3043525387 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f74dc078bc0 msgr2=0x7f74dc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:08.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.157+0000 7f7505a00640 1 --2- 172.21.15.39:0/3043525387 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f74dc078bc0 0x7f74dc07b0d0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f74f0033000 tx=0x7f74f0004730 comp rx=0 tx=0).stop 2024-09-21T07:55:08.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.157+0000 7f7505a00640 1 -- 172.21.15.39:0/3043525387 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75000ccc60 msgr2=0x7f75000c8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:08.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.157+0000 7f7505a00640 1 --2- 172.21.15.39:0/3043525387 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75000ccc60 0x7f75000c8240 secure :-1 s=READY pgs=2897 cs=0 l=1 rev1=1 crypto rx=0x7f74e800b410 tx=0x7f74e800b8d0 comp rx=0 tx=0).stop 2024-09-21T07:55:08.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.157+0000 7f7505a00640 1 -- 172.21.15.39:0/3043525387 shutdown_connections 2024-09-21T07:55:08.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.157+0000 7f7505a00640 1 --2- 172.21.15.39:0/3043525387 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f74dc078bc0 0x7f74dc07b0d0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:08.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.157+0000 7f7505a00640 1 --2- 172.21.15.39:0/3043525387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f750015d830 0x7f75000cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:08.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.157+0000 7f7505a00640 1 --2- 172.21.15.39:0/3043525387 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75000ccc60 0x7f75000c8240 unknown :-1 s=CLOSED pgs=2897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:08.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.157+0000 7f7505a00640 1 --2- 172.21.15.39:0/3043525387 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f75000b30a0 0x7f7500099920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:08.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.157+0000 7f7505a00640 1 -- 172.21.15.39:0/3043525387 >> 172.21.15.39:0/3043525387 conn(0x7f75000b2460 msgr2=0x7f750006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:08.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.157+0000 7f7505a00640 1 -- 172.21.15.39:0/3043525387 shutdown_connections 2024-09-21T07:55:08.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.157+0000 7f7505a00640 1 -- 172.21.15.39:0/3043525387 wait complete. 2024-09-21T07:55:08.185 INFO:tasks.ceph:Scrubbing osd.2 2024-09-21T07:55:08.186 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-21T07:55:08.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.415+0000 7f4567800640 1 -- 172.21.15.39:0/3457174350 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4560070bc0 msgr2=0x7f456006f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:08.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.415+0000 7f4567800640 1 --2- 172.21.15.39:0/3457174350 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4560070bc0 0x7f456006f020 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f4554009830 tx=0x7f455402a510 comp rx=0 tx=0).stop 2024-09-21T07:55:08.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.416+0000 7f4567800640 1 -- 172.21.15.39:0/3457174350 shutdown_connections 2024-09-21T07:55:08.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.416+0000 7f4567800640 1 --2- 172.21.15.39:0/3457174350 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f456006f970 0x7f456015d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:08.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.416+0000 7f4567800640 1 --2- 172.21.15.39:0/3457174350 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4560070bc0 0x7f456006f020 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:08.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.416+0000 7f4567800640 1 -- 172.21.15.39:0/3457174350 >> 172.21.15.39:0/3457174350 conn(0x7f45600b2640 msgr2=0x7f45600d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:08.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.416+0000 7f4567800640 1 -- 172.21.15.39:0/3457174350 shutdown_connections 2024-09-21T07:55:08.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.416+0000 7f4567800640 1 -- 172.21.15.39:0/3457174350 wait complete. 2024-09-21T07:55:08.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.418+0000 7f4567800640 1 Processor -- start 2024-09-21T07:55:08.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.418+0000 7f4567800640 1 -- start start 2024-09-21T07:55:08.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.419+0000 7f4567800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f456006f560 0x7f456009da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:08.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.419+0000 7f4567800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f456006f970 0x7f45600c4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:08.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.420+0000 7f4567800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4560070bc0 0x7f45600c4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:08.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.420+0000 7f4567800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f45600cd4f0 con 0x7f456006f970 2024-09-21T07:55:08.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.420+0000 7f4567800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f45600b50b0 con 0x7f456006f560 2024-09-21T07:55:08.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.420+0000 7f4567800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f45600cd670 con 0x7f4560070bc0 2024-09-21T07:55:08.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.420+0000 7f455fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f456006f970 0x7f45600c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:08.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.420+0000 7f455fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f456006f970 0x7f45600c4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42486/0 (socket says 172.21.15.39:42486) 2024-09-21T07:55:08.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.420+0000 7f455fe00640 1 -- 172.21.15.39:0/605750895 learned_addr learned my addr 172.21.15.39:0/605750895 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:55:08.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.420+0000 7f4564e00640 1 --2- 172.21.15.39:0/605750895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f456006f560 0x7f456009da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:08.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.420+0000 7f4565800640 1 --2- 172.21.15.39:0/605750895 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4560070bc0 0x7f45600c4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:08.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.420+0000 7f455fe00640 1 -- 172.21.15.39:0/605750895 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4560070bc0 msgr2=0x7f45600c4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:08.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.420+0000 7f455fe00640 1 --2- 172.21.15.39:0/605750895 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4560070bc0 0x7f45600c4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:08.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.420+0000 7f455fe00640 1 -- 172.21.15.39:0/605750895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f456006f560 msgr2=0x7f456009da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:08.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.420+0000 7f455fe00640 1 --2- 172.21.15.39:0/605750895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f456006f560 0x7f456009da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:08.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.420+0000 7f455fe00640 1 -- 172.21.15.39:0/605750895 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45600c67c0 con 0x7f456006f970 2024-09-21T07:55:08.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.421+0000 7f455fe00640 1 --2- 172.21.15.39:0/605750895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f456006f970 0x7f45600c4690 secure :-1 s=READY pgs=2898 cs=0 l=1 rev1=1 crypto rx=0x7f454c00c430 tx=0x7f454c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:08.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.421+0000 7f455cc00640 1 -- 172.21.15.39:0/605750895 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c011790 con 0x7f456006f970 2024-09-21T07:55:08.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.421+0000 7f455cc00640 1 -- 172.21.15.39:0/605750895 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f454c017070 con 0x7f456006f970 2024-09-21T07:55:08.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.421+0000 7f4567800640 1 -- 172.21.15.39:0/605750895 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45600c6cc0 con 0x7f456006f970 2024-09-21T07:55:08.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.421+0000 7f455cc00640 1 -- 172.21.15.39:0/605750895 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c011ce0 con 0x7f456006f970 2024-09-21T07:55:08.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.421+0000 7f4567800640 1 -- 172.21.15.39:0/605750895 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45600aebd0 con 0x7f456006f970 2024-09-21T07:55:08.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.422+0000 7f4567800640 1 -- 172.21.15.39:0/605750895 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f45600ae6e0 con 0x7f456006f970 2024-09-21T07:55:08.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.423+0000 7f455cc00640 1 -- 172.21.15.39:0/605750895 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f454c01c460 con 0x7f456006f970 2024-09-21T07:55:08.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.424+0000 7f455cc00640 1 --2- 172.21.15.39:0/605750895 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4540078b90 0x7f454007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:08.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.425+0000 7f455cc00640 1 -- 172.21.15.39:0/605750895 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(269..269 src has 1..269) ==== 6278+0+0 (secure 0 0 0) 0x7f454c09ae60 con 0x7f456006f970 2024-09-21T07:55:08.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.425+0000 7f455cc00640 1 --2- 172.21.15.39:0/605750895 >> [v2:172.21.15.39:6816/3897193606,v1:172.21.15.39:6817/3897193606] conn(0x7f4540081400 0x7f45400838f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:08.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.425+0000 7f455cc00640 1 -- 172.21.15.39:0/605750895 --> [v2:172.21.15.39:6816/3897193606,v1:172.21.15.39:6817/3897193606] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f454c009e50 con 0x7f4540081400 2024-09-21T07:55:08.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.425+0000 7f455cc00640 1 -- 172.21.15.39:0/605750895 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=269) ==== 24+0+0 (secure 0 0 0) 0x7f454c09d480 con 0x7f456006f970 2024-09-21T07:55:08.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.425+0000 7f4564e00640 1 --2- 172.21.15.39:0/605750895 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4540078b90 0x7f454007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:08.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.425+0000 7f4565800640 1 --2- 172.21.15.39:0/605750895 >> [v2:172.21.15.39:6816/3897193606,v1:172.21.15.39:6817/3897193606] conn(0x7f4540081400 0x7f45400838f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:08.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.426+0000 7f4565800640 1 --2- 172.21.15.39:0/605750895 >> [v2:172.21.15.39:6816/3897193606,v1:172.21.15.39:6817/3897193606] conn(0x7f4540081400 0x7f45400838f0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:08.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.426+0000 7f455cc00640 1 -- 172.21.15.39:0/605750895 <== osd.2 v2:172.21.15.39:6816/3897193606 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f454c009e50 con 0x7f4540081400 2024-09-21T07:55:08.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.427+0000 7f4564e00640 1 --2- 172.21.15.39:0/605750895 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4540078b90 0x7f454007b0a0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f4554034000 tx=0x7f4554004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:08.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.440+0000 7f4567800640 1 -- 172.21.15.39:0/605750895 --> [v2:172.21.15.39:6816/3897193606,v1:172.21.15.39:6817/3897193606] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f45600c6a70 con 0x7f4540081400 2024-09-21T07:55:08.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.440+0000 7f455cc00640 1 -- 172.21.15.39:0/605750895 <== osd.2 v2:172.21.15.39:6816/3897193606 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f45600c6a70 con 0x7f4540081400 2024-09-21T07:55:08.442 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:55:08.442 INFO:teuthology.orchestra.run.smithi039.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T07:55:08.442 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:55:08.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.440+0000 7f4567800640 1 -- 172.21.15.39:0/605750895 >> [v2:172.21.15.39:6816/3897193606,v1:172.21.15.39:6817/3897193606] conn(0x7f4540081400 msgr2=0x7f45400838f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:08.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.440+0000 7f4567800640 1 --2- 172.21.15.39:0/605750895 >> [v2:172.21.15.39:6816/3897193606,v1:172.21.15.39:6817/3897193606] conn(0x7f4540081400 0x7f45400838f0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:08.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.440+0000 7f4567800640 1 -- 172.21.15.39:0/605750895 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4540078b90 msgr2=0x7f454007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:08.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.440+0000 7f4567800640 1 --2- 172.21.15.39:0/605750895 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4540078b90 0x7f454007b0a0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f4554034000 tx=0x7f4554004730 comp rx=0 tx=0).stop 2024-09-21T07:55:08.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.440+0000 7f4567800640 1 -- 172.21.15.39:0/605750895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f456006f970 msgr2=0x7f45600c4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:08.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.441+0000 7f4567800640 1 --2- 172.21.15.39:0/605750895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f456006f970 0x7f45600c4690 secure :-1 s=READY pgs=2898 cs=0 l=1 rev1=1 crypto rx=0x7f454c00c430 tx=0x7f454c00c8f0 comp rx=0 tx=0).stop 2024-09-21T07:55:08.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.441+0000 7f4565800640 1 -- 172.21.15.39:0/605750895 reap_dead start 2024-09-21T07:55:08.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.441+0000 7f4567800640 1 -- 172.21.15.39:0/605750895 shutdown_connections 2024-09-21T07:55:08.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.441+0000 7f4567800640 1 -- 172.21.15.39:0/605750895 >> 172.21.15.39:0/605750895 conn(0x7f45600b2640 msgr2=0x7f456015c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:08.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.441+0000 7f4567800640 1 -- 172.21.15.39:0/605750895 shutdown_connections 2024-09-21T07:55:08.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.441+0000 7f4567800640 1 -- 172.21.15.39:0/605750895 wait complete. 2024-09-21T07:55:08.463 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-21T07:55:08.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.690+0000 7f3f66400640 1 -- 172.21.15.39:0/1476146484 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f44003f90 msgr2=0x7f3f44004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:08.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.690+0000 7f3f66400640 1 --2- 172.21.15.39:0/1476146484 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f44003f90 0x7f3f44004800 secure :-1 s=READY pgs=2900 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c009b20 tx=0x7f3f44005830 comp rx=0 tx=0).stop 2024-09-21T07:55:08.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.690+0000 7f3f5fe00640 1 -- 172.21.15.39:0/1476146484 reap_dead start 2024-09-21T07:55:08.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.690+0000 7f3f66400640 1 -- 172.21.15.39:0/1476146484 shutdown_connections 2024-09-21T07:55:08.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.690+0000 7f3f66400640 1 -- 172.21.15.39:0/1476146484 >> 172.21.15.39:0/1476146484 conn(0x7f3f600b2460 msgr2=0x7f3f6006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:08.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.691+0000 7f3f66400640 1 -- 172.21.15.39:0/1476146484 shutdown_connections 2024-09-21T07:55:08.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.691+0000 7f3f66400640 1 -- 172.21.15.39:0/1476146484 wait complete. 2024-09-21T07:55:08.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.693+0000 7f3f66400640 1 Processor -- start 2024-09-21T07:55:08.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.693+0000 7f3f66400640 1 -- start start 2024-09-21T07:55:08.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.694+0000 7f3f66400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f44003f90 0x7f3f601f8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:08.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.694+0000 7f3f66400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3f6015b820 0x7f3f600c6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:08.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.694+0000 7f3f66400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f60161110 0x7f3f60161520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:08.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.694+0000 7f3f66400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3f600af910 con 0x7f3f60161110 2024-09-21T07:55:08.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.694+0000 7f3f66400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3f600af790 con 0x7f3f44003f90 2024-09-21T07:55:08.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.694+0000 7f3f66400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f3f600afa90 con 0x7f3f6015b820 2024-09-21T07:55:08.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.695+0000 7f3f5fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f60161110 0x7f3f60161520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:08.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.695+0000 7f3f5fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f60161110 0x7f3f60161520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42502/0 (socket says 172.21.15.39:42502) 2024-09-21T07:55:08.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.695+0000 7f3f5fe00640 1 -- 172.21.15.39:0/1798787138 learned_addr learned my addr 172.21.15.39:0/1798787138 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:55:08.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.695+0000 7f3f5f400640 1 --2- 172.21.15.39:0/1798787138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f44003f90 0x7f3f601f8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:08.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.695+0000 7f3f5ea00640 1 --2- 172.21.15.39:0/1798787138 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3f6015b820 0x7f3f600c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:08.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.695+0000 7f3f5fe00640 1 -- 172.21.15.39:0/1798787138 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3f6015b820 msgr2=0x7f3f600c6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:08.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.695+0000 7f3f5fe00640 1 --2- 172.21.15.39:0/1798787138 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3f6015b820 0x7f3f600c6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:08.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.695+0000 7f3f5fe00640 1 -- 172.21.15.39:0/1798787138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f44003f90 msgr2=0x7f3f601f8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:08.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.695+0000 7f3f5fe00640 1 --2- 172.21.15.39:0/1798787138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f44003f90 0x7f3f601f8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:08.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.695+0000 7f3f5fe00640 1 -- 172.21.15.39:0/1798787138 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f60161a60 con 0x7f3f60161110 2024-09-21T07:55:08.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.695+0000 7f3f5fe00640 1 --2- 172.21.15.39:0/1798787138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f60161110 0x7f3f60161520 secure :-1 s=READY pgs=2901 cs=0 l=1 rev1=1 crypto rx=0x7f3f600b2e40 tx=0x7f3f58006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:08.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.696+0000 7f3f57400640 1 -- 172.21.15.39:0/1798787138 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f58011900 con 0x7f3f60161110 2024-09-21T07:55:08.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.696+0000 7f3f57400640 1 -- 172.21.15.39:0/1798787138 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f58011aa0 con 0x7f3f60161110 2024-09-21T07:55:08.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.696+0000 7f3f66400640 1 -- 172.21.15.39:0/1798787138 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f601f9310 con 0x7f3f60161110 2024-09-21T07:55:08.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.696+0000 7f3f57400640 1 -- 172.21.15.39:0/1798787138 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f58011e50 con 0x7f3f60161110 2024-09-21T07:55:08.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.696+0000 7f3f66400640 1 -- 172.21.15.39:0/1798787138 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f601f98d0 con 0x7f3f60161110 2024-09-21T07:55:08.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.697+0000 7f3f66400640 1 -- 172.21.15.39:0/1798787138 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f600af080 con 0x7f3f60161110 2024-09-21T07:55:08.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.698+0000 7f3f57400640 1 -- 172.21.15.39:0/1798787138 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3f5801d470 con 0x7f3f60161110 2024-09-21T07:55:08.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.700+0000 7f3f57400640 1 --2- 172.21.15.39:0/1798787138 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3f40078d30 0x7f3f4007b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:08.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.700+0000 7f3f57400640 1 -- 172.21.15.39:0/1798787138 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(269..269 src has 1..269) ==== 6278+0+0 (secure 0 0 0) 0x7f3f5809b760 con 0x7f3f60161110 2024-09-21T07:55:08.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.700+0000 7f3f5f400640 1 --2- 172.21.15.39:0/1798787138 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3f40078d30 0x7f3f4007b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:08.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.701+0000 7f3f5f400640 1 --2- 172.21.15.39:0/1798787138 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3f40078d30 0x7f3f4007b240 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f3f60075150 tx=0x7f3f50006720 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:08.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.704+0000 7f3f57400640 1 -- 172.21.15.39:0/1798787138 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3f60161a60 con 0x7f3f60161110 2024-09-21T07:55:08.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.901+0000 7f3f66400640 1 -- 172.21.15.39:0/1798787138 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f3f600cdb70 con 0x7f3f40078d30 2024-09-21T07:55:08.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.901+0000 7f3f57400640 1 -- 172.21.15.39:0/1798787138 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3f600cdb70 con 0x7f3f40078d30 2024-09-21T07:55:08.903 INFO:teuthology.orchestra.run.smithi039.stderr:instructed osd(s) 2 to deep-scrub 2024-09-21T07:55:08.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.904+0000 7f3f66400640 1 -- 172.21.15.39:0/1798787138 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3f40078d30 msgr2=0x7f3f4007b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:08.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.904+0000 7f3f66400640 1 --2- 172.21.15.39:0/1798787138 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3f40078d30 0x7f3f4007b240 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f3f60075150 tx=0x7f3f50006720 comp rx=0 tx=0).stop 2024-09-21T07:55:08.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.904+0000 7f3f66400640 1 -- 172.21.15.39:0/1798787138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f60161110 msgr2=0x7f3f60161520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:08.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.904+0000 7f3f66400640 1 --2- 172.21.15.39:0/1798787138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f60161110 0x7f3f60161520 secure :-1 s=READY pgs=2901 cs=0 l=1 rev1=1 crypto rx=0x7f3f600b2e40 tx=0x7f3f58006430 comp rx=0 tx=0).stop 2024-09-21T07:55:08.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.904+0000 7f3f66400640 1 -- 172.21.15.39:0/1798787138 shutdown_connections 2024-09-21T07:55:08.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.904+0000 7f3f66400640 1 --2- 172.21.15.39:0/1798787138 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3f40078d30 0x7f3f4007b240 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:08.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.904+0000 7f3f66400640 1 --2- 172.21.15.39:0/1798787138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f60161110 0x7f3f60161520 unknown :-1 s=CLOSED pgs=2901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:08.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.904+0000 7f3f66400640 1 --2- 172.21.15.39:0/1798787138 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3f6015b820 0x7f3f600c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:08.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.904+0000 7f3f66400640 1 --2- 172.21.15.39:0/1798787138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f44003f90 0x7f3f601f8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:08.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.904+0000 7f3f66400640 1 -- 172.21.15.39:0/1798787138 >> 172.21.15.39:0/1798787138 conn(0x7f3f600b2460 msgr2=0x7f3f600d0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:08.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.904+0000 7f3f66400640 1 -- 172.21.15.39:0/1798787138 shutdown_connections 2024-09-21T07:55:08.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:08.904+0000 7f3f66400640 1 -- 172.21.15.39:0/1798787138 wait complete. 2024-09-21T07:55:08.930 INFO:tasks.ceph:Scrubbing osd.3 2024-09-21T07:55:08.931 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-21T07:55:09.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.162+0000 7f034fe00640 1 -- 172.21.15.39:0/3049382985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0330005250 msgr2=0x7f033000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:09.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.162+0000 7f034fe00640 1 --2- 172.21.15.39:0/3049382985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0330005250 0x7f033000b9c0 secure :-1 s=READY pgs=2903 cs=0 l=1 rev1=1 crypto rx=0x7f03480755c0 tx=0x7f0330005830 comp rx=0 tx=0).stop 2024-09-21T07:55:09.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.162+0000 7f034fe00640 1 -- 172.21.15.39:0/3049382985 shutdown_connections 2024-09-21T07:55:09.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.162+0000 7f034fe00640 1 --2- 172.21.15.39:0/3049382985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0330005250 0x7f033000b9c0 unknown :-1 s=CLOSED pgs=2903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:09.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.162+0000 7f034fe00640 1 -- 172.21.15.39:0/3049382985 >> 172.21.15.39:0/3049382985 conn(0x7f03480b2640 msgr2=0x7f03480d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:09.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.162+0000 7f034fe00640 1 -- 172.21.15.39:0/3049382985 shutdown_connections 2024-09-21T07:55:09.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.162+0000 7f034fe00640 1 -- 172.21.15.39:0/3049382985 wait complete. 2024-09-21T07:55:09.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.164+0000 7f034fe00640 1 Processor -- start 2024-09-21T07:55:09.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.164+0000 7f034fe00640 1 -- start start 2024-09-21T07:55:09.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.166+0000 7f034fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0330005250 0x7f0348155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:09.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.166+0000 7f034fe00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03480aee30 0x7f03480ad0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:09.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.166+0000 7f034fe00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0348156890 0x7f03480c6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:09.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.166+0000 7f034fe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f03480b3b80 con 0x7f0330005250 2024-09-21T07:55:09.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.166+0000 7f034fe00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f03480b3a00 con 0x7f0348156890 2024-09-21T07:55:09.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.166+0000 7f034fe00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f03480b3d00 con 0x7f03480aee30 2024-09-21T07:55:09.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.166+0000 7f034d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0330005250 0x7f0348155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:09.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.166+0000 7f034d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0330005250 0x7f0348155d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42506/0 (socket says 172.21.15.39:42506) 2024-09-21T07:55:09.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.166+0000 7f034d400640 1 -- 172.21.15.39:0/3457794870 learned_addr learned my addr 172.21.15.39:0/3457794870 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:55:09.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.166+0000 7f034de00640 1 --2- 172.21.15.39:0/3457794870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0348156890 0x7f03480c6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:09.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.166+0000 7f034ca00640 1 --2- 172.21.15.39:0/3457794870 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03480aee30 0x7f03480ad0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:09.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.166+0000 7f034d400640 1 -- 172.21.15.39:0/3457794870 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03480aee30 msgr2=0x7f03480ad0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:09.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.166+0000 7f034d400640 1 --2- 172.21.15.39:0/3457794870 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03480aee30 0x7f03480ad0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:09.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.166+0000 7f034d400640 1 -- 172.21.15.39:0/3457794870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0348156890 msgr2=0x7f03480c6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:09.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.166+0000 7f034d400640 1 --2- 172.21.15.39:0/3457794870 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0348156890 0x7f03480c6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:09.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.166+0000 7f034d400640 1 -- 172.21.15.39:0/3457794870 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03480ad610 con 0x7f0330005250 2024-09-21T07:55:09.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.166+0000 7f034ca00640 1 --2- 172.21.15.39:0/3457794870 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f03480aee30 0x7f03480ad0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:55:09.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.167+0000 7f034d400640 1 --2- 172.21.15.39:0/3457794870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0330005250 0x7f0348155d20 secure :-1 s=READY pgs=2904 cs=0 l=1 rev1=1 crypto rx=0x7f033c011ab0 tx=0x7f033c011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:09.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.167+0000 7f0345600640 1 -- 172.21.15.39:0/3457794870 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033c01e070 con 0x7f0330005250 2024-09-21T07:55:09.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.167+0000 7f0345600640 1 -- 172.21.15.39:0/3457794870 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f033c00fe70 con 0x7f0330005250 2024-09-21T07:55:09.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.167+0000 7f0345600640 1 -- 172.21.15.39:0/3457794870 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033c0042b0 con 0x7f0330005250 2024-09-21T07:55:09.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.167+0000 7f034fe00640 1 -- 172.21.15.39:0/3457794870 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03480c62b0 con 0x7f0330005250 2024-09-21T07:55:09.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.167+0000 7f034fe00640 1 -- 172.21.15.39:0/3457794870 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03480c45b0 con 0x7f0330005250 2024-09-21T07:55:09.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.168+0000 7f034fe00640 1 -- 172.21.15.39:0/3457794870 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f03480753d0 con 0x7f0330005250 2024-09-21T07:55:09.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.169+0000 7f0345600640 1 -- 172.21.15.39:0/3457794870 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f033c021070 con 0x7f0330005250 2024-09-21T07:55:09.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.170+0000 7f0345600640 1 --2- 172.21.15.39:0/3457794870 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0328078b90 0x7f032807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:09.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.170+0000 7f0345600640 1 -- 172.21.15.39:0/3457794870 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(269..269 src has 1..269) ==== 6278+0+0 (secure 0 0 0) 0x7f033c09c510 con 0x7f0330005250 2024-09-21T07:55:09.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.171+0000 7f0345600640 1 --2- 172.21.15.39:0/3457794870 >> [v2:172.21.15.39:6800/3213199177,v1:172.21.15.39:6801/3213199177] conn(0x7f0328081400 0x7f03280838f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:09.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.171+0000 7f0345600640 1 -- 172.21.15.39:0/3457794870 --> [v2:172.21.15.39:6800/3213199177,v1:172.21.15.39:6801/3213199177] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f033c001d30 con 0x7f0328081400 2024-09-21T07:55:09.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.171+0000 7f0345600640 1 -- 172.21.15.39:0/3457794870 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=269) ==== 24+0+0 (secure 0 0 0) 0x7f033c09c8c0 con 0x7f0330005250 2024-09-21T07:55:09.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.171+0000 7f034ca00640 1 --2- 172.21.15.39:0/3457794870 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0328078b90 0x7f032807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:09.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.171+0000 7f034de00640 1 --2- 172.21.15.39:0/3457794870 >> [v2:172.21.15.39:6800/3213199177,v1:172.21.15.39:6801/3213199177] conn(0x7f0328081400 0x7f03280838f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:09.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.172+0000 7f034de00640 1 --2- 172.21.15.39:0/3457794870 >> [v2:172.21.15.39:6800/3213199177,v1:172.21.15.39:6801/3213199177] conn(0x7f0328081400 0x7f03280838f0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:09.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.172+0000 7f034ca00640 1 --2- 172.21.15.39:0/3457794870 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0328078b90 0x7f032807b0a0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f034815cd30 tx=0x7f0334006b70 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:09.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.172+0000 7f0345600640 1 -- 172.21.15.39:0/3457794870 <== osd.3 v2:172.21.15.39:6800/3213199177 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f033c001d30 con 0x7f0328081400 2024-09-21T07:55:09.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.186+0000 7f034fe00640 1 -- 172.21.15.39:0/3457794870 --> [v2:172.21.15.39:6800/3213199177,v1:172.21.15.39:6801/3213199177] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0348070940 con 0x7f0328081400 2024-09-21T07:55:09.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.186+0000 7f0345600640 1 -- 172.21.15.39:0/3457794870 <== osd.3 v2:172.21.15.39:6800/3213199177 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0348070940 con 0x7f0328081400 2024-09-21T07:55:09.188 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:55:09.188 INFO:teuthology.orchestra.run.smithi039.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T07:55:09.188 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:55:09.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.186+0000 7f034fe00640 1 -- 172.21.15.39:0/3457794870 >> [v2:172.21.15.39:6800/3213199177,v1:172.21.15.39:6801/3213199177] conn(0x7f0328081400 msgr2=0x7f03280838f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:09.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.186+0000 7f034fe00640 1 --2- 172.21.15.39:0/3457794870 >> [v2:172.21.15.39:6800/3213199177,v1:172.21.15.39:6801/3213199177] conn(0x7f0328081400 0x7f03280838f0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:09.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.186+0000 7f034fe00640 1 -- 172.21.15.39:0/3457794870 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0328078b90 msgr2=0x7f032807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:09.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.186+0000 7f034fe00640 1 --2- 172.21.15.39:0/3457794870 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f0328078b90 0x7f032807b0a0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f034815cd30 tx=0x7f0334006b70 comp rx=0 tx=0).stop 2024-09-21T07:55:09.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.187+0000 7f034fe00640 1 -- 172.21.15.39:0/3457794870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0330005250 msgr2=0x7f0348155d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:09.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.187+0000 7f034fe00640 1 --2- 172.21.15.39:0/3457794870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0330005250 0x7f0348155d20 secure :-1 s=READY pgs=2904 cs=0 l=1 rev1=1 crypto rx=0x7f033c011ab0 tx=0x7f033c011f70 comp rx=0 tx=0).stop 2024-09-21T07:55:09.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.187+0000 7f034de00640 1 -- 172.21.15.39:0/3457794870 reap_dead start 2024-09-21T07:55:09.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.187+0000 7f034fe00640 1 -- 172.21.15.39:0/3457794870 shutdown_connections 2024-09-21T07:55:09.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.187+0000 7f034fe00640 1 -- 172.21.15.39:0/3457794870 >> 172.21.15.39:0/3457794870 conn(0x7f03480b2640 msgr2=0x7f03480ccd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:09.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.187+0000 7f034fe00640 1 -- 172.21.15.39:0/3457794870 shutdown_connections 2024-09-21T07:55:09.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.187+0000 7f034fe00640 1 -- 172.21.15.39:0/3457794870 wait complete. 2024-09-21T07:55:09.209 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-21T07:55:09.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.436+0000 7f38cc400640 1 -- 172.21.15.39:0/1139007821 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f38c415d830 msgr2=0x7f38c4156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:09.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.436+0000 7f38cc400640 1 --2- 172.21.15.39:0/1139007821 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f38c415d830 0x7f38c4156bb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f38b0009830 tx=0x7f38b002a480 comp rx=0 tx=0).stop 2024-09-21T07:55:09.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.436+0000 7f38cc400640 1 -- 172.21.15.39:0/1139007821 shutdown_connections 2024-09-21T07:55:09.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.436+0000 7f38cc400640 1 --2- 172.21.15.39:0/1139007821 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f38c415d830 0x7f38c4156bb0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:09.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.436+0000 7f38cc400640 1 --2- 172.21.15.39:0/1139007821 >> v2:172.21.15.148:3300/0 conn(0x7f38c40b30a0 0x7f38c40cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:09.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.436+0000 7f38cc400640 1 -- 172.21.15.39:0/1139007821 >> 172.21.15.39:0/1139007821 conn(0x7f38c40b2460 msgr2=0x7f38c406cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:09.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.436+0000 7f38cc400640 1 -- 172.21.15.39:0/1139007821 shutdown_connections 2024-09-21T07:55:09.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.436+0000 7f38cc400640 1 -- 172.21.15.39:0/1139007821 wait complete. 2024-09-21T07:55:09.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.438+0000 7f38cc400640 1 Processor -- start 2024-09-21T07:55:09.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.438+0000 7f38cc400640 1 -- start start 2024-09-21T07:55:09.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.440+0000 7f38cc400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38c40b30a0 0x7f38c40c3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:09.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.440+0000 7f38cc400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f38c40ccc60 0x7f38c40b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:09.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.440+0000 7f38cc400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38c415d830 0x7f38c40b1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:09.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.440+0000 7f38cc400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f38c40af450 con 0x7f38c40b30a0 2024-09-21T07:55:09.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.440+0000 7f38cc400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f38c40af2d0 con 0x7f38c415d830 2024-09-21T07:55:09.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.440+0000 7f38cc400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f38c40af5d0 con 0x7f38c40ccc60 2024-09-21T07:55:09.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.440+0000 7f38c9a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38c40b30a0 0x7f38c40c3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:09.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.440+0000 7f38c9a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38c40b30a0 0x7f38c40c3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42518/0 (socket says 172.21.15.39:42518) 2024-09-21T07:55:09.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.440+0000 7f38c9a00640 1 -- 172.21.15.39:0/2700909928 learned_addr learned my addr 172.21.15.39:0/2700909928 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:55:09.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.441+0000 7f38c9000640 1 --2- 172.21.15.39:0/2700909928 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f38c40ccc60 0x7f38c40b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:09.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.441+0000 7f38c9a00640 1 -- 172.21.15.39:0/2700909928 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f38c40ccc60 msgr2=0x7f38c40b0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:09.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.441+0000 7f38c9a00640 1 --2- 172.21.15.39:0/2700909928 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f38c40ccc60 0x7f38c40b0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:09.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.441+0000 7f38ca400640 1 --2- 172.21.15.39:0/2700909928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38c415d830 0x7f38c40b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:09.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.441+0000 7f38c9a00640 1 -- 172.21.15.39:0/2700909928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38c415d830 msgr2=0x7f38c40b1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:09.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.441+0000 7f38c9a00640 1 --2- 172.21.15.39:0/2700909928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38c415d830 0x7f38c40b1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:09.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.441+0000 7f38c9a00640 1 -- 172.21.15.39:0/2700909928 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c41f41e0 con 0x7f38c40b30a0 2024-09-21T07:55:09.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.441+0000 7f38ca400640 1 --2- 172.21.15.39:0/2700909928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38c415d830 0x7f38c40b1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:55:09.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.441+0000 7f38c9a00640 1 --2- 172.21.15.39:0/2700909928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38c40b30a0 0x7f38c40c3d80 secure :-1 s=READY pgs=2906 cs=0 l=1 rev1=1 crypto rx=0x7f38b800e790 tx=0x7f38b800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:09.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.441+0000 7f38c1600640 1 -- 172.21.15.39:0/2700909928 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b800eea0 con 0x7f38c40b30a0 2024-09-21T07:55:09.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.442+0000 7f38c1600640 1 -- 172.21.15.39:0/2700909928 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38b800cdb0 con 0x7f38c40b30a0 2024-09-21T07:55:09.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.442+0000 7f38cc400640 1 -- 172.21.15.39:0/2700909928 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38c41f44c0 con 0x7f38c40b30a0 2024-09-21T07:55:09.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.442+0000 7f38c1600640 1 -- 172.21.15.39:0/2700909928 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b8010690 con 0x7f38c40b30a0 2024-09-21T07:55:09.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.442+0000 7f38cc400640 1 -- 172.21.15.39:0/2700909928 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38c41f4a80 con 0x7f38c40b30a0 2024-09-21T07:55:09.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.443+0000 7f38cc400640 1 -- 172.21.15.39:0/2700909928 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38c40b0bb0 con 0x7f38c40b30a0 2024-09-21T07:55:09.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.444+0000 7f38c1600640 1 -- 172.21.15.39:0/2700909928 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f38b8010830 con 0x7f38c40b30a0 2024-09-21T07:55:09.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.445+0000 7f38c1600640 1 --2- 172.21.15.39:0/2700909928 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f38a4078b90 0x7f38a407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:09.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.445+0000 7f38c1600640 1 -- 172.21.15.39:0/2700909928 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(269..269 src has 1..269) ==== 6278+0+0 (secure 0 0 0) 0x7f38b809ba00 con 0x7f38c40b30a0 2024-09-21T07:55:09.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.445+0000 7f38c9000640 1 --2- 172.21.15.39:0/2700909928 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f38a4078b90 0x7f38a407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:09.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.446+0000 7f38c9000640 1 --2- 172.21.15.39:0/2700909928 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f38a4078b90 0x7f38a407b0a0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f38b0034000 tx=0x7f38b0004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:09.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.449+0000 7f38c1600640 1 -- 172.21.15.39:0/2700909928 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f38b8063b80 con 0x7f38c40b30a0 2024-09-21T07:55:09.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.652+0000 7f38cc400640 1 -- 172.21.15.39:0/2700909928 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f38c409e3d0 con 0x7f38a4078b90 2024-09-21T07:55:09.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.653+0000 7f38c1600640 1 -- 172.21.15.39:0/2700909928 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f38c409e3d0 con 0x7f38a4078b90 2024-09-21T07:55:09.655 INFO:teuthology.orchestra.run.smithi039.stderr:instructed osd(s) 3 to deep-scrub 2024-09-21T07:55:09.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.656+0000 7f38cc400640 1 -- 172.21.15.39:0/2700909928 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f38a4078b90 msgr2=0x7f38a407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:09.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.656+0000 7f38cc400640 1 --2- 172.21.15.39:0/2700909928 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f38a4078b90 0x7f38a407b0a0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f38b0034000 tx=0x7f38b0004730 comp rx=0 tx=0).stop 2024-09-21T07:55:09.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.656+0000 7f38cc400640 1 -- 172.21.15.39:0/2700909928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38c40b30a0 msgr2=0x7f38c40c3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:09.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.656+0000 7f38cc400640 1 --2- 172.21.15.39:0/2700909928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38c40b30a0 0x7f38c40c3d80 secure :-1 s=READY pgs=2906 cs=0 l=1 rev1=1 crypto rx=0x7f38b800e790 tx=0x7f38b800ec50 comp rx=0 tx=0).stop 2024-09-21T07:55:09.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.656+0000 7f38cc400640 1 -- 172.21.15.39:0/2700909928 shutdown_connections 2024-09-21T07:55:09.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.656+0000 7f38cc400640 1 --2- 172.21.15.39:0/2700909928 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f38a4078b90 0x7f38a407b0a0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:09.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.656+0000 7f38cc400640 1 --2- 172.21.15.39:0/2700909928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38c415d830 0x7f38c40b1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:09.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.656+0000 7f38cc400640 1 --2- 172.21.15.39:0/2700909928 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f38c40ccc60 0x7f38c40b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:09.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.656+0000 7f38cc400640 1 --2- 172.21.15.39:0/2700909928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38c40b30a0 0x7f38c40c3d80 unknown :-1 s=CLOSED pgs=2906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:09.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.656+0000 7f38cc400640 1 -- 172.21.15.39:0/2700909928 >> 172.21.15.39:0/2700909928 conn(0x7f38c40b2460 msgr2=0x7f38c40add20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:09.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.656+0000 7f38cc400640 1 -- 172.21.15.39:0/2700909928 shutdown_connections 2024-09-21T07:55:09.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.656+0000 7f38cc400640 1 -- 172.21.15.39:0/2700909928 wait complete. 2024-09-21T07:55:09.680 INFO:tasks.ceph:Scrubbing osd.4 2024-09-21T07:55:09.680 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-21T07:55:09.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.912+0000 7f3f3ee00640 1 -- 172.21.15.39:0/2845995741 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3f38071bf0 msgr2=0x7f3f38152290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:09.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.912+0000 7f3f3ee00640 1 --2- 172.21.15.39:0/2845995741 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3f38071bf0 0x7f3f38152290 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c009830 tx=0x7f3f2c02a480 comp rx=0 tx=0).stop 2024-09-21T07:55:09.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.912+0000 7f3f3ee00640 1 -- 172.21.15.39:0/2845995741 shutdown_connections 2024-09-21T07:55:09.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.912+0000 7f3f3ee00640 1 --2- 172.21.15.39:0/2845995741 >> v2:172.21.15.148:3300/0 conn(0x7f3f380cf990 0x7f3f380cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:09.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.912+0000 7f3f3ee00640 1 --2- 172.21.15.39:0/2845995741 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3f38071bf0 0x7f3f38152290 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:09.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.912+0000 7f3f3ee00640 1 -- 172.21.15.39:0/2845995741 >> 172.21.15.39:0/2845995741 conn(0x7f3f380d1420 msgr2=0x7f3f38073720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:09.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.912+0000 7f3f3ee00640 1 -- 172.21.15.39:0/2845995741 shutdown_connections 2024-09-21T07:55:09.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.912+0000 7f3f3ee00640 1 -- 172.21.15.39:0/2845995741 wait complete. 2024-09-21T07:55:09.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.914+0000 7f3f3ee00640 1 Processor -- start 2024-09-21T07:55:09.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.914+0000 7f3f3ee00640 1 -- start start 2024-09-21T07:55:09.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.916+0000 7f3f3ee00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f38071bf0 0x7f3f3814b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:09.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.916+0000 7f3f3ee00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f380cf990 0x7f3f3813cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:09.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.916+0000 7f3f3ee00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3f3815f060 0x7f3f3813d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:09.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.916+0000 7f3f3ee00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3f380d02e0 con 0x7f3f380cf990 2024-09-21T07:55:09.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.916+0000 7f3f3ee00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3806fb80 con 0x7f3f38071bf0 2024-09-21T07:55:09.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.916+0000 7f3f3ee00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f3f380d0460 con 0x7f3f3815f060 2024-09-21T07:55:09.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.916+0000 7f3f3d000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f380cf990 0x7f3f3813cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:09.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.916+0000 7f3f3d000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f380cf990 0x7f3f3813cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42530/0 (socket says 172.21.15.39:42530) 2024-09-21T07:55:09.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.916+0000 7f3f3d000640 1 -- 172.21.15.39:0/3863655429 learned_addr learned my addr 172.21.15.39:0/3863655429 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:55:09.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.916+0000 7f3f3da00640 1 --2- 172.21.15.39:0/3863655429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f38071bf0 0x7f3f3814b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:09.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.917+0000 7f3f3d000640 1 -- 172.21.15.39:0/3863655429 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3f3815f060 msgr2=0x7f3f3813d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:09.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.917+0000 7f3f3d000640 1 --2- 172.21.15.39:0/3863655429 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f3f3815f060 0x7f3f3813d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:09.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.917+0000 7f3f3d000640 1 -- 172.21.15.39:0/3863655429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f38071bf0 msgr2=0x7f3f3814b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:09.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.917+0000 7f3f3d000640 1 --2- 172.21.15.39:0/3863655429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3f38071bf0 0x7f3f3814b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:09.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.917+0000 7f3f3d000640 1 -- 172.21.15.39:0/3863655429 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f381547d0 con 0x7f3f380cf990 2024-09-21T07:55:09.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.917+0000 7f3f3d000640 1 --2- 172.21.15.39:0/3863655429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f380cf990 0x7f3f3813cef0 secure :-1 s=READY pgs=2907 cs=0 l=1 rev1=1 crypto rx=0x7f3f2400c390 tx=0x7f3f2400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:09.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.917+0000 7f3f35600640 1 -- 172.21.15.39:0/3863655429 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f24017070 con 0x7f3f380cf990 2024-09-21T07:55:09.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.917+0000 7f3f35600640 1 -- 172.21.15.39:0/3863655429 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f240035f0 con 0x7f3f380cf990 2024-09-21T07:55:09.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.917+0000 7f3f3ee00640 1 -- 172.21.15.39:0/3863655429 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f38154d00 con 0x7f3f380cf990 2024-09-21T07:55:09.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.917+0000 7f3f35600640 1 -- 172.21.15.39:0/3863655429 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f24003a20 con 0x7f3f380cf990 2024-09-21T07:55:09.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.918+0000 7f3f3ee00640 1 -- 172.21.15.39:0/3863655429 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f381552c0 con 0x7f3f380cf990 2024-09-21T07:55:09.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.918+0000 7f3f3ee00640 1 -- 172.21.15.39:0/3863655429 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3f38155910 con 0x7f3f380cf990 2024-09-21T07:55:09.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.920+0000 7f3f35600640 1 -- 172.21.15.39:0/3863655429 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3f2401a070 con 0x7f3f380cf990 2024-09-21T07:55:09.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.921+0000 7f3f35600640 1 --2- 172.21.15.39:0/3863655429 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3f18078b90 0x7f3f1807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:09.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.921+0000 7f3f35600640 1 -- 172.21.15.39:0/3863655429 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(269..269 src has 1..269) ==== 6278+0+0 (secure 0 0 0) 0x7f3f2409a210 con 0x7f3f380cf990 2024-09-21T07:55:09.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.921+0000 7f3f35600640 1 --2- 172.21.15.39:0/3863655429 >> [v2:172.21.15.148:6802/1437509971,v1:172.21.15.148:6803/1437509971] conn(0x7f3f18081460 0x7f3f18083950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:09.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.921+0000 7f3f35600640 1 -- 172.21.15.39:0/3863655429 --> [v2:172.21.15.148:6802/1437509971,v1:172.21.15.148:6803/1437509971] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3f18084020 con 0x7f3f18081460 2024-09-21T07:55:09.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.921+0000 7f3f35600640 1 -- 172.21.15.39:0/3863655429 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=269) ==== 24+0+0 (secure 0 0 0) 0x7f3f240a1050 con 0x7f3f380cf990 2024-09-21T07:55:09.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.921+0000 7f3f3da00640 1 --2- 172.21.15.39:0/3863655429 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3f18078b90 0x7f3f1807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:09.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.922+0000 7f3f3e400640 1 --2- 172.21.15.39:0/3863655429 >> [v2:172.21.15.148:6802/1437509971,v1:172.21.15.148:6803/1437509971] conn(0x7f3f18081460 0x7f3f18083950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:09.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.922+0000 7f3f3da00640 1 --2- 172.21.15.39:0/3863655429 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3f18078b90 0x7f3f1807b0a0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c034000 tx=0x7f3f2c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:09.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.922+0000 7f3f3e400640 1 --2- 172.21.15.39:0/3863655429 >> [v2:172.21.15.148:6802/1437509971,v1:172.21.15.148:6803/1437509971] conn(0x7f3f18081460 0x7f3f18083950 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:09.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.923+0000 7f3f35600640 1 -- 172.21.15.39:0/3863655429 <== osd.4 v2:172.21.15.148:6802/1437509971 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3f18084020 con 0x7f3f18081460 2024-09-21T07:55:09.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.936+0000 7f3f3ee00640 1 -- 172.21.15.39:0/3863655429 --> [v2:172.21.15.148:6802/1437509971,v1:172.21.15.148:6803/1437509971] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f3f38154ab0 con 0x7f3f18081460 2024-09-21T07:55:09.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.937+0000 7f3f35600640 1 -- 172.21.15.39:0/3863655429 <== osd.4 v2:172.21.15.148:6802/1437509971 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f3f38154ab0 con 0x7f3f18081460 2024-09-21T07:55:09.939 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:55:09.939 INFO:teuthology.orchestra.run.smithi039.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T07:55:09.939 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:55:09.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.938+0000 7f3f3ee00640 1 -- 172.21.15.39:0/3863655429 >> [v2:172.21.15.148:6802/1437509971,v1:172.21.15.148:6803/1437509971] conn(0x7f3f18081460 msgr2=0x7f3f18083950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:09.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.938+0000 7f3f3ee00640 1 --2- 172.21.15.39:0/3863655429 >> [v2:172.21.15.148:6802/1437509971,v1:172.21.15.148:6803/1437509971] conn(0x7f3f18081460 0x7f3f18083950 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:09.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.938+0000 7f3f3ee00640 1 -- 172.21.15.39:0/3863655429 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3f18078b90 msgr2=0x7f3f1807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:09.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.938+0000 7f3f3ee00640 1 --2- 172.21.15.39:0/3863655429 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f3f18078b90 0x7f3f1807b0a0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c034000 tx=0x7f3f2c004730 comp rx=0 tx=0).stop 2024-09-21T07:55:09.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.938+0000 7f3f3ee00640 1 -- 172.21.15.39:0/3863655429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f380cf990 msgr2=0x7f3f3813cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:09.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.938+0000 7f3f3ee00640 1 --2- 172.21.15.39:0/3863655429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f380cf990 0x7f3f3813cef0 secure :-1 s=READY pgs=2907 cs=0 l=1 rev1=1 crypto rx=0x7f3f2400c390 tx=0x7f3f2400c850 comp rx=0 tx=0).stop 2024-09-21T07:55:09.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.938+0000 7f3f3e400640 1 -- 172.21.15.39:0/3863655429 reap_dead start 2024-09-21T07:55:09.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.938+0000 7f3f3ee00640 1 -- 172.21.15.39:0/3863655429 shutdown_connections 2024-09-21T07:55:09.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.938+0000 7f3f3ee00640 1 -- 172.21.15.39:0/3863655429 >> 172.21.15.39:0/3863655429 conn(0x7f3f380d1420 msgr2=0x7f3f380cf1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:09.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.938+0000 7f3f3ee00640 1 -- 172.21.15.39:0/3863655429 shutdown_connections 2024-09-21T07:55:09.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:09.938+0000 7f3f3ee00640 1 -- 172.21.15.39:0/3863655429 wait complete. 2024-09-21T07:55:09.959 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-21T07:55:10.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.185+0000 7f56f7a00640 1 -- 172.21.15.39:0/2449695344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56f015d830 msgr2=0x7f56f0156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:10.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.185+0000 7f56f7a00640 1 --2- 172.21.15.39:0/2449695344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56f015d830 0x7f56f0156bb0 secure :-1 s=READY pgs=2909 cs=0 l=1 rev1=1 crypto rx=0x7f56f0071e60 tx=0x7f56dc02a4a0 comp rx=0 tx=0).stop 2024-09-21T07:55:10.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.185+0000 7f56f7a00640 1 -- 172.21.15.39:0/2449695344 shutdown_connections 2024-09-21T07:55:10.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.185+0000 7f56f7a00640 1 --2- 172.21.15.39:0/2449695344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56f015d830 0x7f56f0156bb0 unknown :-1 s=CLOSED pgs=2909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:10.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.185+0000 7f56f7a00640 1 --2- 172.21.15.39:0/2449695344 >> v2:172.21.15.148:3300/0 conn(0x7f56f00b30a0 0x7f56f00cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:10.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.185+0000 7f56f7a00640 1 -- 172.21.15.39:0/2449695344 >> 172.21.15.39:0/2449695344 conn(0x7f56f00b2460 msgr2=0x7f56f006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:10.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.185+0000 7f56f7a00640 1 -- 172.21.15.39:0/2449695344 shutdown_connections 2024-09-21T07:55:10.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.185+0000 7f56f7a00640 1 -- 172.21.15.39:0/2449695344 wait complete. 2024-09-21T07:55:10.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.188+0000 7f56f7a00640 1 Processor -- start 2024-09-21T07:55:10.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.188+0000 7f56f7a00640 1 -- start start 2024-09-21T07:55:10.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.189+0000 7f56f7a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56f00b30a0 0x7f56f0154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:10.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.189+0000 7f56f7a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56f00ccc60 0x7f56f01f8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:10.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.189+0000 7f56f7a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56f015d830 0x7f56f01f93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:10.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.189+0000 7f56f7a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f56f00af410 con 0x7f56f00b30a0 2024-09-21T07:55:10.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.189+0000 7f56f7a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f56f00af290 con 0x7f56f015d830 2024-09-21T07:55:10.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.189+0000 7f56f7a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f56f00af590 con 0x7f56f00ccc60 2024-09-21T07:55:10.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.189+0000 7f56f5000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56f00b30a0 0x7f56f0154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:10.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.189+0000 7f56f5000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56f00b30a0 0x7f56f0154ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42554/0 (socket says 172.21.15.39:42554) 2024-09-21T07:55:10.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.189+0000 7f56f5000640 1 -- 172.21.15.39:0/137211198 learned_addr learned my addr 172.21.15.39:0/137211198 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:55:10.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.190+0000 7f56f5a00640 1 --2- 172.21.15.39:0/137211198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56f015d830 0x7f56f01f93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:10.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.190+0000 7f56f5000640 1 -- 172.21.15.39:0/137211198 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56f00ccc60 msgr2=0x7f56f01f8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:10.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.190+0000 7f56f5000640 1 --2- 172.21.15.39:0/137211198 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56f00ccc60 0x7f56f01f8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:10.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.190+0000 7f56f5000640 1 -- 172.21.15.39:0/137211198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56f015d830 msgr2=0x7f56f01f93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:10.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.190+0000 7f56f5000640 1 --2- 172.21.15.39:0/137211198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56f015d830 0x7f56f01f93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:10.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.190+0000 7f56f5000640 1 -- 172.21.15.39:0/137211198 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f00cff70 con 0x7f56f00b30a0 2024-09-21T07:55:10.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.190+0000 7f56f5a00640 1 --2- 172.21.15.39:0/137211198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56f015d830 0x7f56f01f93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:55:10.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.190+0000 7f56f5000640 1 --2- 172.21.15.39:0/137211198 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56f00b30a0 0x7f56f0154ad0 secure :-1 s=READY pgs=2910 cs=0 l=1 rev1=1 crypto rx=0x7f56e400d740 tx=0x7f56e400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:10.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.190+0000 7f56ecc00640 1 -- 172.21.15.39:0/137211198 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e4017070 con 0x7f56f00b30a0 2024-09-21T07:55:10.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.190+0000 7f56ecc00640 1 -- 172.21.15.39:0/137211198 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56e4009d20 con 0x7f56f00b30a0 2024-09-21T07:55:10.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.190+0000 7f56f7a00640 1 -- 172.21.15.39:0/137211198 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56f0155170 con 0x7f56f00b30a0 2024-09-21T07:55:10.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.191+0000 7f56ecc00640 1 -- 172.21.15.39:0/137211198 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e40106c0 con 0x7f56f00b30a0 2024-09-21T07:55:10.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.191+0000 7f56f7a00640 1 -- 172.21.15.39:0/137211198 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56f0155730 con 0x7f56f00b30a0 2024-09-21T07:55:10.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.192+0000 7f56f7a00640 1 -- 172.21.15.39:0/137211198 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56f006a680 con 0x7f56f00b30a0 2024-09-21T07:55:10.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.193+0000 7f56ecc00640 1 -- 172.21.15.39:0/137211198 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f56e401a070 con 0x7f56f00b30a0 2024-09-21T07:55:10.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.194+0000 7f56ecc00640 1 --2- 172.21.15.39:0/137211198 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f56d0078bc0 0x7f56d007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:10.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.194+0000 7f56ecc00640 1 -- 172.21.15.39:0/137211198 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(269..269 src has 1..269) ==== 6278+0+0 (secure 0 0 0) 0x7f56e409b070 con 0x7f56f00b30a0 2024-09-21T07:55:10.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.194+0000 7f56efe00640 1 --2- 172.21.15.39:0/137211198 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f56d0078bc0 0x7f56d007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:10.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.195+0000 7f56efe00640 1 --2- 172.21.15.39:0/137211198 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f56d0078bc0 0x7f56d007b0d0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f56dc034000 tx=0x7f56dc0046c0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:10.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.198+0000 7f56ecc00640 1 -- 172.21.15.39:0/137211198 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f56f006a680 con 0x7f56f00b30a0 2024-09-21T07:55:10.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.396+0000 7f56f7a00640 1 -- 172.21.15.39:0/137211198 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f56f00cdf80 con 0x7f56d0078bc0 2024-09-21T07:55:10.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.397+0000 7f56ecc00640 1 -- 172.21.15.39:0/137211198 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f56f00cdf80 con 0x7f56d0078bc0 2024-09-21T07:55:10.399 INFO:teuthology.orchestra.run.smithi039.stderr:instructed osd(s) 4 to deep-scrub 2024-09-21T07:55:10.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.399+0000 7f56f7a00640 1 -- 172.21.15.39:0/137211198 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f56d0078bc0 msgr2=0x7f56d007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:10.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.400+0000 7f56f7a00640 1 --2- 172.21.15.39:0/137211198 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f56d0078bc0 0x7f56d007b0d0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f56dc034000 tx=0x7f56dc0046c0 comp rx=0 tx=0).stop 2024-09-21T07:55:10.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.400+0000 7f56f7a00640 1 -- 172.21.15.39:0/137211198 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56f00b30a0 msgr2=0x7f56f0154ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:10.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.400+0000 7f56f7a00640 1 --2- 172.21.15.39:0/137211198 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56f00b30a0 0x7f56f0154ad0 secure :-1 s=READY pgs=2910 cs=0 l=1 rev1=1 crypto rx=0x7f56e400d740 tx=0x7f56e400dc00 comp rx=0 tx=0).stop 2024-09-21T07:55:10.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.400+0000 7f56f7a00640 1 -- 172.21.15.39:0/137211198 shutdown_connections 2024-09-21T07:55:10.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.400+0000 7f56f7a00640 1 --2- 172.21.15.39:0/137211198 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f56d0078bc0 0x7f56d007b0d0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:10.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.400+0000 7f56f7a00640 1 --2- 172.21.15.39:0/137211198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56f015d830 0x7f56f01f93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:10.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.400+0000 7f56f7a00640 1 --2- 172.21.15.39:0/137211198 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f56f00ccc60 0x7f56f01f8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:10.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.400+0000 7f56f7a00640 1 --2- 172.21.15.39:0/137211198 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56f00b30a0 0x7f56f0154ad0 unknown :-1 s=CLOSED pgs=2910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:10.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.400+0000 7f56f7a00640 1 -- 172.21.15.39:0/137211198 >> 172.21.15.39:0/137211198 conn(0x7f56f00b2460 msgr2=0x7f56f006f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:10.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.400+0000 7f56f7a00640 1 -- 172.21.15.39:0/137211198 shutdown_connections 2024-09-21T07:55:10.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.400+0000 7f56f7a00640 1 -- 172.21.15.39:0/137211198 wait complete. 2024-09-21T07:55:10.424 INFO:tasks.ceph:Scrubbing osd.5 2024-09-21T07:55:10.424 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-21T07:55:10.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.653+0000 7fd368800640 1 -- 172.21.15.39:0/814195605 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd348004800 msgr2=0x7fd348004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:10.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.653+0000 7fd368800640 1 --2- 172.21.15.39:0/814195605 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd348004800 0x7fd348004c10 secure :-1 s=READY pgs=2912 cs=0 l=1 rev1=1 crypto rx=0x7fd3600cad40 tx=0x7fd348005830 comp rx=0 tx=0).stop 2024-09-21T07:55:10.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.653+0000 7fd366800640 1 -- 172.21.15.39:0/814195605 reap_dead start 2024-09-21T07:55:10.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.653+0000 7fd368800640 1 -- 172.21.15.39:0/814195605 shutdown_connections 2024-09-21T07:55:10.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.653+0000 7fd368800640 1 -- 172.21.15.39:0/814195605 >> 172.21.15.39:0/814195605 conn(0x7fd3600b2640 msgr2=0x7fd3600d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:10.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.653+0000 7fd368800640 1 -- 172.21.15.39:0/814195605 shutdown_connections 2024-09-21T07:55:10.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.653+0000 7fd368800640 1 -- 172.21.15.39:0/814195605 wait complete. 2024-09-21T07:55:10.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.655+0000 7fd368800640 1 Processor -- start 2024-09-21T07:55:10.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.655+0000 7fd368800640 1 -- start start 2024-09-21T07:55:10.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.657+0000 7fd368800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd348004800 0x7fd3601f9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:10.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.657+0000 7fd368800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3601f54b0 0x7fd3601f58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:10.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.657+0000 7fd368800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd3600ae610 0x7fd3600aea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:10.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.657+0000 7fd368800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd3600b33f0 con 0x7fd348004800 2024-09-21T07:55:10.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.657+0000 7fd368800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd3600b50d0 con 0x7fd3601f54b0 2024-09-21T07:55:10.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.657+0000 7fd368800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fd3600b3570 con 0x7fd3600ae610 2024-09-21T07:55:10.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.657+0000 7fd365e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd348004800 0x7fd3601f9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:10.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.657+0000 7fd365e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd348004800 0x7fd3601f9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42582/0 (socket says 172.21.15.39:42582) 2024-09-21T07:55:10.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.657+0000 7fd365e00640 1 -- 172.21.15.39:0/3485228135 learned_addr learned my addr 172.21.15.39:0/3485228135 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:55:10.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.657+0000 7fd365400640 1 --2- 172.21.15.39:0/3485228135 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3601f54b0 0x7fd3601f58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:10.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.657+0000 7fd365e00640 1 -- 172.21.15.39:0/3485228135 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd3600ae610 msgr2=0x7fd3600aea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:10.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.657+0000 7fd365e00640 1 --2- 172.21.15.39:0/3485228135 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fd3600ae610 0x7fd3600aea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:10.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.657+0000 7fd365e00640 1 -- 172.21.15.39:0/3485228135 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3601f54b0 msgr2=0x7fd3601f58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:10.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.657+0000 7fd365e00640 1 --2- 172.21.15.39:0/3485228135 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3601f54b0 0x7fd3601f58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:10.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.657+0000 7fd365e00640 1 -- 172.21.15.39:0/3485228135 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3601fa2f0 con 0x7fd348004800 2024-09-21T07:55:10.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.658+0000 7fd365e00640 1 --2- 172.21.15.39:0/3485228135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd348004800 0x7fd3601f9db0 secure :-1 s=READY pgs=2913 cs=0 l=1 rev1=1 crypto rx=0x7fd354006600 tx=0x7fd3540039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:10.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.658+0000 7fd35a000640 1 -- 172.21.15.39:0/3485228135 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd354015070 con 0x7fd348004800 2024-09-21T07:55:10.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.658+0000 7fd35a000640 1 -- 172.21.15.39:0/3485228135 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd354011860 con 0x7fd348004800 2024-09-21T07:55:10.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.658+0000 7fd35a000640 1 -- 172.21.15.39:0/3485228135 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd354011cb0 con 0x7fd348004800 2024-09-21T07:55:10.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.658+0000 7fd368800640 1 -- 172.21.15.39:0/3485228135 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3600aef60 con 0x7fd348004800 2024-09-21T07:55:10.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.658+0000 7fd368800640 1 -- 172.21.15.39:0/3485228135 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3600c8020 con 0x7fd348004800 2024-09-21T07:55:10.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.659+0000 7fd368800640 1 -- 172.21.15.39:0/3485228135 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd3600b38f0 con 0x7fd348004800 2024-09-21T07:55:10.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.661+0000 7fd35a000640 1 -- 172.21.15.39:0/3485228135 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd354018070 con 0x7fd348004800 2024-09-21T07:55:10.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.661+0000 7fd35a000640 1 --2- 172.21.15.39:0/3485228135 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd344078b90 0x7fd34407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:10.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.662+0000 7fd35a000640 1 -- 172.21.15.39:0/3485228135 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(269..269 src has 1..269) ==== 6278+0+0 (secure 0 0 0) 0x7fd35409bc90 con 0x7fd348004800 2024-09-21T07:55:10.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.662+0000 7fd35a000640 1 --2- 172.21.15.39:0/3485228135 >> [v2:172.21.15.148:6826/1837190792,v1:172.21.15.148:6827/1837190792] conn(0x7fd344081400 0x7fd3440838f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:10.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.662+0000 7fd35a000640 1 -- 172.21.15.39:0/3485228135 --> [v2:172.21.15.148:6826/1837190792,v1:172.21.15.148:6827/1837190792] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd354011e50 con 0x7fd344081400 2024-09-21T07:55:10.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.662+0000 7fd35a000640 1 -- 172.21.15.39:0/3485228135 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=269) ==== 24+0+0 (secure 0 0 0) 0x7fd3540a1070 con 0x7fd348004800 2024-09-21T07:55:10.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.662+0000 7fd365400640 1 --2- 172.21.15.39:0/3485228135 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd344078b90 0x7fd34407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:10.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.662+0000 7fd366800640 1 --2- 172.21.15.39:0/3485228135 >> [v2:172.21.15.148:6826/1837190792,v1:172.21.15.148:6827/1837190792] conn(0x7fd344081400 0x7fd3440838f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:10.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.663+0000 7fd365400640 1 --2- 172.21.15.39:0/3485228135 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd344078b90 0x7fd34407b0a0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fd350035000 tx=0x7fd35002d040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:10.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.663+0000 7fd366800640 1 --2- 172.21.15.39:0/3485228135 >> [v2:172.21.15.148:6826/1837190792,v1:172.21.15.148:6827/1837190792] conn(0x7fd344081400 0x7fd3440838f0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:10.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.664+0000 7fd35a000640 1 -- 172.21.15.39:0/3485228135 <== osd.5 v2:172.21.15.148:6826/1837190792 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd354011e50 con 0x7fd344081400 2024-09-21T07:55:10.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.678+0000 7fd368800640 1 -- 172.21.15.39:0/3485228135 --> [v2:172.21.15.148:6826/1837190792,v1:172.21.15.148:6827/1837190792] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd3600b0c00 con 0x7fd344081400 2024-09-21T07:55:10.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.679+0000 7fd35a000640 1 -- 172.21.15.39:0/3485228135 <== osd.5 v2:172.21.15.148:6826/1837190792 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd3600b0c00 con 0x7fd344081400 2024-09-21T07:55:10.681 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:55:10.681 INFO:teuthology.orchestra.run.smithi039.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T07:55:10.681 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:55:10.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.679+0000 7fd368800640 1 -- 172.21.15.39:0/3485228135 >> [v2:172.21.15.148:6826/1837190792,v1:172.21.15.148:6827/1837190792] conn(0x7fd344081400 msgr2=0x7fd3440838f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:10.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.679+0000 7fd368800640 1 --2- 172.21.15.39:0/3485228135 >> [v2:172.21.15.148:6826/1837190792,v1:172.21.15.148:6827/1837190792] conn(0x7fd344081400 0x7fd3440838f0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:10.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.680+0000 7fd368800640 1 -- 172.21.15.39:0/3485228135 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd344078b90 msgr2=0x7fd34407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:10.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.680+0000 7fd368800640 1 --2- 172.21.15.39:0/3485228135 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fd344078b90 0x7fd34407b0a0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fd350035000 tx=0x7fd35002d040 comp rx=0 tx=0).stop 2024-09-21T07:55:10.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.680+0000 7fd368800640 1 -- 172.21.15.39:0/3485228135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd348004800 msgr2=0x7fd3601f9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:10.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.680+0000 7fd368800640 1 --2- 172.21.15.39:0/3485228135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd348004800 0x7fd3601f9db0 secure :-1 s=READY pgs=2913 cs=0 l=1 rev1=1 crypto rx=0x7fd354006600 tx=0x7fd3540039c0 comp rx=0 tx=0).stop 2024-09-21T07:55:10.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.680+0000 7fd366800640 1 -- 172.21.15.39:0/3485228135 reap_dead start 2024-09-21T07:55:10.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.680+0000 7fd368800640 1 -- 172.21.15.39:0/3485228135 shutdown_connections 2024-09-21T07:55:10.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.680+0000 7fd368800640 1 -- 172.21.15.39:0/3485228135 >> 172.21.15.39:0/3485228135 conn(0x7fd3600b2640 msgr2=0x7fd3600b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:10.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.680+0000 7fd368800640 1 -- 172.21.15.39:0/3485228135 shutdown_connections 2024-09-21T07:55:10.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.680+0000 7fd368800640 1 -- 172.21.15.39:0/3485228135 wait complete. 2024-09-21T07:55:10.708 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-21T07:55:10.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.936+0000 7fac16a00640 1 -- 172.21.15.39:0/889619471 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac100b30a0 msgr2=0x7fac100cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:10.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.936+0000 7fac16a00640 1 --2- 172.21.15.39:0/889619471 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac100b30a0 0x7fac100cb8a0 secure :-1 s=READY pgs=2914 cs=0 l=1 rev1=1 crypto rx=0x7fac100cbde0 tx=0x7fac0002a4d0 comp rx=0 tx=0).stop 2024-09-21T07:55:10.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.936+0000 7fac16a00640 1 -- 172.21.15.39:0/889619471 shutdown_connections 2024-09-21T07:55:10.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.936+0000 7fac16a00640 1 --2- 172.21.15.39:0/889619471 >> v2:172.21.15.148:3300/0 conn(0x7fac100ccc60 0x7fac100cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:10.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.936+0000 7fac16a00640 1 --2- 172.21.15.39:0/889619471 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac100b30a0 0x7fac100cb8a0 unknown :-1 s=CLOSED pgs=2914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:10.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.936+0000 7fac16a00640 1 -- 172.21.15.39:0/889619471 >> 172.21.15.39:0/889619471 conn(0x7fac100b2460 msgr2=0x7fac1006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:10.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.936+0000 7fac16a00640 1 -- 172.21.15.39:0/889619471 shutdown_connections 2024-09-21T07:55:10.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.936+0000 7fac16a00640 1 -- 172.21.15.39:0/889619471 wait complete. 2024-09-21T07:55:10.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.938+0000 7fac16a00640 1 Processor -- start 2024-09-21T07:55:10.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.939+0000 7fac16a00640 1 -- start start 2024-09-21T07:55:10.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.940+0000 7fac16a00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fac100b30a0 0x7fac100c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:10.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.940+0000 7fac16a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac100ccc60 0x7fac101f3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:10.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.940+0000 7fac16a00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac1015d830 0x7fac100c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:10.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.940+0000 7fac16a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fac100af410 con 0x7fac100ccc60 2024-09-21T07:55:10.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.940+0000 7fac16a00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fac100af290 con 0x7fac1015d830 2024-09-21T07:55:10.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.940+0000 7fac16a00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fac100af590 con 0x7fac100b30a0 2024-09-21T07:55:10.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.940+0000 7fac07e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac100ccc60 0x7fac101f3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:10.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.940+0000 7fac07e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac100ccc60 0x7fac101f3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42590/0 (socket says 172.21.15.39:42590) 2024-09-21T07:55:10.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.940+0000 7fac07e00640 1 -- 172.21.15.39:0/3231360257 learned_addr learned my addr 172.21.15.39:0/3231360257 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:55:10.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.941+0000 7fac07e00640 1 -- 172.21.15.39:0/3231360257 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fac100b30a0 msgr2=0x7fac100c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:10.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.941+0000 7fac07e00640 1 --2- 172.21.15.39:0/3231360257 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fac100b30a0 0x7fac100c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:10.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.941+0000 7fac07e00640 1 -- 172.21.15.39:0/3231360257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac1015d830 msgr2=0x7fac100c8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:10.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.941+0000 7fac14a00640 1 --2- 172.21.15.39:0/3231360257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac1015d830 0x7fac100c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:10.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.941+0000 7fac07e00640 1 --2- 172.21.15.39:0/3231360257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac1015d830 0x7fac100c8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:10.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.941+0000 7fac07e00640 1 -- 172.21.15.39:0/3231360257 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac100c7c00 con 0x7fac100ccc60 2024-09-21T07:55:10.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.941+0000 7fac14a00640 1 --2- 172.21.15.39:0/3231360257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac1015d830 0x7fac100c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:55:10.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.941+0000 7fac07e00640 1 --2- 172.21.15.39:0/3231360257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac100ccc60 0x7fac101f3100 secure :-1 s=READY pgs=2915 cs=0 l=1 rev1=1 crypto rx=0x7fabf800b410 tx=0x7fabf800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:10.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.941+0000 7fac0cc00640 1 -- 172.21.15.39:0/3231360257 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf8015020 con 0x7fac100ccc60 2024-09-21T07:55:10.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.941+0000 7fac0cc00640 1 -- 172.21.15.39:0/3231360257 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fabf8011920 con 0x7fac100ccc60 2024-09-21T07:55:10.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.941+0000 7fac0cc00640 1 -- 172.21.15.39:0/3231360257 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf8011d70 con 0x7fac100ccc60 2024-09-21T07:55:10.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.941+0000 7fac16a00640 1 -- 172.21.15.39:0/3231360257 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac100c7ee0 con 0x7fac100ccc60 2024-09-21T07:55:10.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.942+0000 7fac16a00640 1 -- 172.21.15.39:0/3231360257 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac100c84a0 con 0x7fac100ccc60 2024-09-21T07:55:10.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.943+0000 7fac16a00640 1 -- 172.21.15.39:0/3231360257 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac100c7970 con 0x7fac100ccc60 2024-09-21T07:55:10.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.944+0000 7fac0cc00640 1 -- 172.21.15.39:0/3231360257 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fabf8018020 con 0x7fac100ccc60 2024-09-21T07:55:10.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.945+0000 7fac0cc00640 1 --2- 172.21.15.39:0/3231360257 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fabec078b90 0x7fabec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:10.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.945+0000 7fac0cc00640 1 -- 172.21.15.39:0/3231360257 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(269..269 src has 1..269) ==== 6278+0+0 (secure 0 0 0) 0x7fabf809afb0 con 0x7fac100ccc60 2024-09-21T07:55:10.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.945+0000 7fac0fe00640 1 --2- 172.21.15.39:0/3231360257 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fabec078b90 0x7fabec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:10.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.946+0000 7fac0fe00640 1 --2- 172.21.15.39:0/3231360257 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fabec078b90 0x7fabec07b0a0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fac00033000 tx=0x7fac000046c0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:10.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:10.949+0000 7fac0cc00640 1 -- 172.21.15.39:0/3231360257 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fabf801f020 con 0x7fac100ccc60 2024-09-21T07:55:11.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.150+0000 7fac16a00640 1 -- 172.21.15.39:0/3231360257 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fac100afac0 con 0x7fabec078b90 2024-09-21T07:55:11.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.151+0000 7fac0cc00640 1 -- 172.21.15.39:0/3231360257 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fac100afac0 con 0x7fabec078b90 2024-09-21T07:55:11.153 INFO:teuthology.orchestra.run.smithi039.stderr:instructed osd(s) 5 to deep-scrub 2024-09-21T07:55:11.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.153+0000 7fac16a00640 1 -- 172.21.15.39:0/3231360257 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fabec078b90 msgr2=0x7fabec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:11.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.153+0000 7fac16a00640 1 --2- 172.21.15.39:0/3231360257 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fabec078b90 0x7fabec07b0a0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fac00033000 tx=0x7fac000046c0 comp rx=0 tx=0).stop 2024-09-21T07:55:11.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.153+0000 7fac16a00640 1 -- 172.21.15.39:0/3231360257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac100ccc60 msgr2=0x7fac101f3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:11.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.153+0000 7fac16a00640 1 --2- 172.21.15.39:0/3231360257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac100ccc60 0x7fac101f3100 secure :-1 s=READY pgs=2915 cs=0 l=1 rev1=1 crypto rx=0x7fabf800b410 tx=0x7fabf800b8d0 comp rx=0 tx=0).stop 2024-09-21T07:55:11.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.154+0000 7fac16a00640 1 -- 172.21.15.39:0/3231360257 shutdown_connections 2024-09-21T07:55:11.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.154+0000 7fac16a00640 1 --2- 172.21.15.39:0/3231360257 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fabec078b90 0x7fabec07b0a0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:11.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.154+0000 7fac16a00640 1 --2- 172.21.15.39:0/3231360257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac1015d830 0x7fac100c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:11.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.154+0000 7fac16a00640 1 --2- 172.21.15.39:0/3231360257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac100ccc60 0x7fac101f3100 unknown :-1 s=CLOSED pgs=2915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:11.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.154+0000 7fac16a00640 1 --2- 172.21.15.39:0/3231360257 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fac100b30a0 0x7fac100c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:11.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.154+0000 7fac16a00640 1 -- 172.21.15.39:0/3231360257 >> 172.21.15.39:0/3231360257 conn(0x7fac100b2460 msgr2=0x7fac1006f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:11.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.154+0000 7fac16a00640 1 -- 172.21.15.39:0/3231360257 shutdown_connections 2024-09-21T07:55:11.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.154+0000 7fac16a00640 1 -- 172.21.15.39:0/3231360257 wait complete. 2024-09-21T07:55:11.178 INFO:tasks.ceph:Scrubbing osd.6 2024-09-21T07:55:11.178 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-21T07:55:11.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.408+0000 7feb77800640 1 -- 172.21.15.39:0/622389072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb7006f730 msgr2=0x7feb700b33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:11.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.408+0000 7feb77800640 1 --2- 172.21.15.39:0/622389072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb7006f730 0x7feb700b33f0 secure :-1 s=READY pgs=2917 cs=0 l=1 rev1=1 crypto rx=0x7feb70070510 tx=0x7feb5c02a540 comp rx=0 tx=0).stop 2024-09-21T07:55:11.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.408+0000 7feb77800640 1 -- 172.21.15.39:0/622389072 shutdown_connections 2024-09-21T07:55:11.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.408+0000 7feb77800640 1 --2- 172.21.15.39:0/622389072 >> v2:172.21.15.148:3300/0 conn(0x7feb7006fb40 0x7feb700b3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:11.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.408+0000 7feb77800640 1 --2- 172.21.15.39:0/622389072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb7006f730 0x7feb700b33f0 unknown :-1 s=CLOSED pgs=2917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:11.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.408+0000 7feb77800640 1 -- 172.21.15.39:0/622389072 >> 172.21.15.39:0/622389072 conn(0x7feb700b2640 msgr2=0x7feb700d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:11.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.409+0000 7feb77800640 1 -- 172.21.15.39:0/622389072 shutdown_connections 2024-09-21T07:55:11.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.409+0000 7feb77800640 1 -- 172.21.15.39:0/622389072 wait complete. 2024-09-21T07:55:11.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.411+0000 7feb77800640 1 Processor -- start 2024-09-21T07:55:11.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.411+0000 7feb77800640 1 -- start start 2024-09-21T07:55:11.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.412+0000 7feb77800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb7006f730 0x7feb701f5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:11.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.412+0000 7feb77800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb7006fb40 0x7feb7015c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:11.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.412+0000 7feb77800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb70070bc0 0x7feb70074120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:11.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.412+0000 7feb77800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7feb700cd670 con 0x7feb7006fb40 2024-09-21T07:55:11.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.412+0000 7feb77800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7feb700cd4f0 con 0x7feb70070bc0 2024-09-21T07:55:11.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.412+0000 7feb77800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7feb700b3e70 con 0x7feb7006f730 2024-09-21T07:55:11.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.413+0000 7feb6fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb7006fb40 0x7feb7015c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:11.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.413+0000 7feb6fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb7006fb40 0x7feb7015c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42620/0 (socket says 172.21.15.39:42620) 2024-09-21T07:55:11.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.413+0000 7feb6fe00640 1 -- 172.21.15.39:0/980022513 learned_addr learned my addr 172.21.15.39:0/980022513 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:55:11.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.413+0000 7feb74e00640 1 --2- 172.21.15.39:0/980022513 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb7006f730 0x7feb701f5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:11.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.413+0000 7feb75800640 1 --2- 172.21.15.39:0/980022513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb70070bc0 0x7feb70074120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:11.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.413+0000 7feb6fe00640 1 -- 172.21.15.39:0/980022513 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb7006f730 msgr2=0x7feb701f5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:11.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.413+0000 7feb6fe00640 1 --2- 172.21.15.39:0/980022513 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7feb7006f730 0x7feb701f5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:11.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.413+0000 7feb6fe00640 1 -- 172.21.15.39:0/980022513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb70070bc0 msgr2=0x7feb70074120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:11.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.413+0000 7feb6fe00640 1 --2- 172.21.15.39:0/980022513 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb70070bc0 0x7feb70074120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:11.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.413+0000 7feb6fe00640 1 -- 172.21.15.39:0/980022513 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb7015c930 con 0x7feb7006fb40 2024-09-21T07:55:11.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.414+0000 7feb6fe00640 1 --2- 172.21.15.39:0/980022513 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb7006fb40 0x7feb7015c3f0 secure :-1 s=READY pgs=2918 cs=0 l=1 rev1=1 crypto rx=0x7feb5c02ab30 tx=0x7feb5c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:11.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.414+0000 7feb6cc00640 1 -- 172.21.15.39:0/980022513 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5c031d60 con 0x7feb7006fb40 2024-09-21T07:55:11.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.414+0000 7feb6cc00640 1 -- 172.21.15.39:0/980022513 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb5c033e10 con 0x7feb7006fb40 2024-09-21T07:55:11.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.414+0000 7feb6cc00640 1 -- 172.21.15.39:0/980022513 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5c005260 con 0x7feb7006fb40 2024-09-21T07:55:11.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.414+0000 7feb77800640 1 -- 172.21.15.39:0/980022513 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb700c9080 con 0x7feb7006fb40 2024-09-21T07:55:11.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.414+0000 7feb77800640 1 -- 172.21.15.39:0/980022513 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb700ae7a0 con 0x7feb7006fb40 2024-09-21T07:55:11.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.415+0000 7feb77800640 1 -- 172.21.15.39:0/980022513 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7feb700ae2b0 con 0x7feb7006fb40 2024-09-21T07:55:11.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.416+0000 7feb6cc00640 1 -- 172.21.15.39:0/980022513 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7feb5c005400 con 0x7feb7006fb40 2024-09-21T07:55:11.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.417+0000 7feb6cc00640 1 --2- 172.21.15.39:0/980022513 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feb50078b90 0x7feb5007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:11.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.417+0000 7feb6cc00640 1 -- 172.21.15.39:0/980022513 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(269..269 src has 1..269) ==== 6278+0+0 (secure 0 0 0) 0x7feb5c0bc120 con 0x7feb7006fb40 2024-09-21T07:55:11.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.417+0000 7feb6cc00640 1 --2- 172.21.15.39:0/980022513 >> [v2:172.21.15.148:6810/65974579,v1:172.21.15.148:6811/65974579] conn(0x7feb50081460 0x7feb50083950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:11.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.418+0000 7feb6cc00640 1 -- 172.21.15.39:0/980022513 --> [v2:172.21.15.148:6810/65974579,v1:172.21.15.148:6811/65974579] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feb5c003e50 con 0x7feb50081460 2024-09-21T07:55:11.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.418+0000 7feb6cc00640 1 -- 172.21.15.39:0/980022513 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=269) ==== 24+0+0 (secure 0 0 0) 0x7feb5c0c3050 con 0x7feb7006fb40 2024-09-21T07:55:11.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.418+0000 7feb74e00640 1 --2- 172.21.15.39:0/980022513 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feb50078b90 0x7feb5007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:11.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.418+0000 7feb75800640 1 --2- 172.21.15.39:0/980022513 >> [v2:172.21.15.148:6810/65974579,v1:172.21.15.148:6811/65974579] conn(0x7feb50081460 0x7feb50083950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:11.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.418+0000 7feb74e00640 1 --2- 172.21.15.39:0/980022513 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feb50078b90 0x7feb5007b0a0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7feb64000f60 tx=0x7feb64009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:11.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.419+0000 7feb75800640 1 --2- 172.21.15.39:0/980022513 >> [v2:172.21.15.148:6810/65974579,v1:172.21.15.148:6811/65974579] conn(0x7feb50081460 0x7feb50083950 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:11.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.420+0000 7feb6cc00640 1 -- 172.21.15.39:0/980022513 <== osd.6 v2:172.21.15.148:6810/65974579 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feb5c003e50 con 0x7feb50081460 2024-09-21T07:55:11.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.434+0000 7feb77800640 1 -- 172.21.15.39:0/980022513 --> [v2:172.21.15.148:6810/65974579,v1:172.21.15.148:6811/65974579] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7feb700b43c0 con 0x7feb50081460 2024-09-21T07:55:11.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.435+0000 7feb6cc00640 1 -- 172.21.15.39:0/980022513 <== osd.6 v2:172.21.15.148:6810/65974579 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7feb700b43c0 con 0x7feb50081460 2024-09-21T07:55:11.437 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:55:11.437 INFO:teuthology.orchestra.run.smithi039.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T07:55:11.437 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:55:11.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.435+0000 7feb77800640 1 -- 172.21.15.39:0/980022513 >> [v2:172.21.15.148:6810/65974579,v1:172.21.15.148:6811/65974579] conn(0x7feb50081460 msgr2=0x7feb50083950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:11.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.435+0000 7feb77800640 1 --2- 172.21.15.39:0/980022513 >> [v2:172.21.15.148:6810/65974579,v1:172.21.15.148:6811/65974579] conn(0x7feb50081460 0x7feb50083950 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:11.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.435+0000 7feb77800640 1 -- 172.21.15.39:0/980022513 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feb50078b90 msgr2=0x7feb5007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:11.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.435+0000 7feb77800640 1 --2- 172.21.15.39:0/980022513 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7feb50078b90 0x7feb5007b0a0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7feb64000f60 tx=0x7feb64009040 comp rx=0 tx=0).stop 2024-09-21T07:55:11.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.435+0000 7feb77800640 1 -- 172.21.15.39:0/980022513 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb7006fb40 msgr2=0x7feb7015c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:11.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.435+0000 7feb77800640 1 --2- 172.21.15.39:0/980022513 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb7006fb40 0x7feb7015c3f0 secure :-1 s=READY pgs=2918 cs=0 l=1 rev1=1 crypto rx=0x7feb5c02ab30 tx=0x7feb5c033c20 comp rx=0 tx=0).stop 2024-09-21T07:55:11.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.436+0000 7feb75800640 1 -- 172.21.15.39:0/980022513 reap_dead start 2024-09-21T07:55:11.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.436+0000 7feb77800640 1 -- 172.21.15.39:0/980022513 shutdown_connections 2024-09-21T07:55:11.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.436+0000 7feb77800640 1 -- 172.21.15.39:0/980022513 >> 172.21.15.39:0/980022513 conn(0x7feb700b2640 msgr2=0x7feb700afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:11.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.436+0000 7feb77800640 1 -- 172.21.15.39:0/980022513 shutdown_connections 2024-09-21T07:55:11.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.436+0000 7feb77800640 1 -- 172.21.15.39:0/980022513 wait complete. 2024-09-21T07:55:11.458 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-21T07:55:11.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.682+0000 7f87eb000640 1 -- 172.21.15.39:0/4169963039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87e4160660 msgr2=0x7f87e40cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:11.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.682+0000 7f87eb000640 1 --2- 172.21.15.39:0/4169963039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87e4160660 0x7f87e40cf710 secure :-1 s=READY pgs=2919 cs=0 l=1 rev1=1 crypto rx=0x7f87d0005fd0 tx=0x7f87d002a5d0 comp rx=0 tx=0).stop 2024-09-21T07:55:11.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.682+0000 7f87eb000640 1 -- 172.21.15.39:0/4169963039 shutdown_connections 2024-09-21T07:55:11.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.682+0000 7f87eb000640 1 --2- 172.21.15.39:0/4169963039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87e4160660 0x7f87e40cf710 unknown :-1 s=CLOSED pgs=2919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:11.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.682+0000 7f87eb000640 1 --2- 172.21.15.39:0/4169963039 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87e4160250 0x7f87e415d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:11.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.682+0000 7f87eb000640 1 -- 172.21.15.39:0/4169963039 >> 172.21.15.39:0/4169963039 conn(0x7f87e406cfc0 msgr2=0x7f87e406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:11.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.683+0000 7f87eb000640 1 -- 172.21.15.39:0/4169963039 shutdown_connections 2024-09-21T07:55:11.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.683+0000 7f87eb000640 1 -- 172.21.15.39:0/4169963039 wait complete. 2024-09-21T07:55:11.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.685+0000 7f87eb000640 1 Processor -- start 2024-09-21T07:55:11.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.685+0000 7f87eb000640 1 -- start start 2024-09-21T07:55:11.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.686+0000 7f87eb000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87e40cfc50 0x7f87e4145b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:11.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.686+0000 7f87eb000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87e4160250 0x7f87e4159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:11.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.686+0000 7f87eb000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87e4160660 0x7f87e415edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:11.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.686+0000 7f87eb000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f87e40cce00 con 0x7f87e4160250 2024-09-21T07:55:11.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.686+0000 7f87eb000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f87e40ccc80 con 0x7f87e40cfc50 2024-09-21T07:55:11.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.686+0000 7f87eb000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f87e40ccf80 con 0x7f87e4160660 2024-09-21T07:55:11.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.687+0000 7f87e9200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87e4160250 0x7f87e4159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:11.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.687+0000 7f87e9200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87e4160250 0x7f87e4159fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42638/0 (socket says 172.21.15.39:42638) 2024-09-21T07:55:11.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.687+0000 7f87e9200640 1 -- 172.21.15.39:0/1245824701 learned_addr learned my addr 172.21.15.39:0/1245824701 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:55:11.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.687+0000 7f87e9c00640 1 --2- 172.21.15.39:0/1245824701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87e40cfc50 0x7f87e4145b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:11.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.687+0000 7f87ea600640 1 --2- 172.21.15.39:0/1245824701 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87e4160660 0x7f87e415edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:11.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.687+0000 7f87e9200640 1 -- 172.21.15.39:0/1245824701 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87e4160660 msgr2=0x7f87e415edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:11.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.687+0000 7f87e9200640 1 --2- 172.21.15.39:0/1245824701 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87e4160660 0x7f87e415edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:11.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.687+0000 7f87e9200640 1 -- 172.21.15.39:0/1245824701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87e40cfc50 msgr2=0x7f87e4145b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:11.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.687+0000 7f87e9200640 1 --2- 172.21.15.39:0/1245824701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87e40cfc50 0x7f87e4145b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:11.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.687+0000 7f87e9200640 1 -- 172.21.15.39:0/1245824701 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e415a4e0 con 0x7f87e4160250 2024-09-21T07:55:11.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.687+0000 7f87ea600640 1 --2- 172.21.15.39:0/1245824701 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87e4160660 0x7f87e415edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:55:11.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.687+0000 7f87e9200640 1 --2- 172.21.15.39:0/1245824701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87e4160250 0x7f87e4159fa0 secure :-1 s=READY pgs=2920 cs=0 l=1 rev1=1 crypto rx=0x7f87d0006010 tx=0x7f87d00038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:11.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.688+0000 7f87e1600640 1 -- 172.21.15.39:0/1245824701 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d003e030 con 0x7f87e4160250 2024-09-21T07:55:11.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.688+0000 7f87e1600640 1 -- 172.21.15.39:0/1245824701 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87d0003ca0 con 0x7f87e4160250 2024-09-21T07:55:11.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.688+0000 7f87eb000640 1 -- 172.21.15.39:0/1245824701 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87e4146450 con 0x7f87e4160250 2024-09-21T07:55:11.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.688+0000 7f87e1600640 1 -- 172.21.15.39:0/1245824701 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d0035040 con 0x7f87e4160250 2024-09-21T07:55:11.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.688+0000 7f87eb000640 1 -- 172.21.15.39:0/1245824701 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87e4146a10 con 0x7f87e4160250 2024-09-21T07:55:11.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.689+0000 7f87eb000640 1 -- 172.21.15.39:0/1245824701 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87e40cd430 con 0x7f87e4160250 2024-09-21T07:55:11.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.691+0000 7f87e1600640 1 -- 172.21.15.39:0/1245824701 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f87d0041030 con 0x7f87e4160250 2024-09-21T07:55:11.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.692+0000 7f87e1600640 1 --2- 172.21.15.39:0/1245824701 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f87c4078bc0 0x7f87c407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:11.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.692+0000 7f87e1600640 1 -- 172.21.15.39:0/1245824701 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(269..269 src has 1..269) ==== 6278+0+0 (secure 0 0 0) 0x7f87d00bb620 con 0x7f87e4160250 2024-09-21T07:55:11.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.692+0000 7f87e9c00640 1 --2- 172.21.15.39:0/1245824701 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f87c4078bc0 0x7f87c407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:11.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.693+0000 7f87e9c00640 1 --2- 172.21.15.39:0/1245824701 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f87c4078bc0 0x7f87c407b0d0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f87d8005fd0 tx=0x7f87d8005850 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:11.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.700+0000 7f87e1600640 1 -- 172.21.15.39:0/1245824701 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f87d0083810 con 0x7f87e4160250 2024-09-21T07:55:11.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.895+0000 7f87eb000640 1 -- 172.21.15.39:0/1245824701 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f87e40b28a0 con 0x7f87c4078bc0 2024-09-21T07:55:11.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.896+0000 7f87e1600640 1 -- 172.21.15.39:0/1245824701 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f87e40b28a0 con 0x7f87c4078bc0 2024-09-21T07:55:11.898 INFO:teuthology.orchestra.run.smithi039.stderr:instructed osd(s) 6 to deep-scrub 2024-09-21T07:55:11.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.898+0000 7f87eb000640 1 -- 172.21.15.39:0/1245824701 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f87c4078bc0 msgr2=0x7f87c407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:11.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.898+0000 7f87eb000640 1 --2- 172.21.15.39:0/1245824701 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f87c4078bc0 0x7f87c407b0d0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f87d8005fd0 tx=0x7f87d8005850 comp rx=0 tx=0).stop 2024-09-21T07:55:11.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.898+0000 7f87eb000640 1 -- 172.21.15.39:0/1245824701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87e4160250 msgr2=0x7f87e4159fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:11.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.898+0000 7f87eb000640 1 --2- 172.21.15.39:0/1245824701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87e4160250 0x7f87e4159fa0 secure :-1 s=READY pgs=2920 cs=0 l=1 rev1=1 crypto rx=0x7f87d0006010 tx=0x7f87d00038a0 comp rx=0 tx=0).stop 2024-09-21T07:55:11.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.898+0000 7f87eb000640 1 -- 172.21.15.39:0/1245824701 shutdown_connections 2024-09-21T07:55:11.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.898+0000 7f87eb000640 1 --2- 172.21.15.39:0/1245824701 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f87c4078bc0 0x7f87c407b0d0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:11.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.898+0000 7f87eb000640 1 --2- 172.21.15.39:0/1245824701 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f87e4160660 0x7f87e415edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:11.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.898+0000 7f87eb000640 1 --2- 172.21.15.39:0/1245824701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87e4160250 0x7f87e4159fa0 unknown :-1 s=CLOSED pgs=2920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:11.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.898+0000 7f87eb000640 1 --2- 172.21.15.39:0/1245824701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87e40cfc50 0x7f87e4145b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:11.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.898+0000 7f87eb000640 1 -- 172.21.15.39:0/1245824701 >> 172.21.15.39:0/1245824701 conn(0x7f87e406cfc0 msgr2=0x7f87e40cd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:11.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.898+0000 7f87eb000640 1 -- 172.21.15.39:0/1245824701 shutdown_connections 2024-09-21T07:55:11.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:11.898+0000 7f87eb000640 1 -- 172.21.15.39:0/1245824701 wait complete. 2024-09-21T07:55:11.926 INFO:tasks.ceph:Scrubbing osd.7 2024-09-21T07:55:11.926 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-21T07:55:12.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.158+0000 7fcd2dc00640 1 -- 172.21.15.39:0/120093632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd2815d440 msgr2=0x7fcd2815d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:12.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.158+0000 7fcd2dc00640 1 --2- 172.21.15.39:0/120093632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd2815d440 0x7fcd2815d850 secure :-1 s=READY pgs=2922 cs=0 l=1 rev1=1 crypto rx=0x7fcd20009be0 tx=0x7fcd2002a920 comp rx=0 tx=0).stop 2024-09-21T07:55:12.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.158+0000 7fcd2dc00640 1 -- 172.21.15.39:0/120093632 shutdown_connections 2024-09-21T07:55:12.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.158+0000 7fcd2dc00640 1 --2- 172.21.15.39:0/120093632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd2815d440 0x7fcd2815d850 unknown :-1 s=CLOSED pgs=2922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:12.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.158+0000 7fcd2dc00640 1 --2- 172.21.15.39:0/120093632 >> v2:172.21.15.148:3300/0 conn(0x7fcd281539f0 0x7fcd280b4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:12.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.158+0000 7fcd2dc00640 1 -- 172.21.15.39:0/120093632 >> 172.21.15.39:0/120093632 conn(0x7fcd2806d660 msgr2=0x7fcd280b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:12.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.158+0000 7fcd2dc00640 1 -- 172.21.15.39:0/120093632 shutdown_connections 2024-09-21T07:55:12.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.158+0000 7fcd2dc00640 1 -- 172.21.15.39:0/120093632 wait complete. 2024-09-21T07:55:12.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.160+0000 7fcd2dc00640 1 Processor -- start 2024-09-21T07:55:12.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.160+0000 7fcd2dc00640 1 -- start start 2024-09-21T07:55:12.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.162+0000 7fcd2dc00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd281539f0 0x7fcd281445c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:12.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.162+0000 7fcd2dc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd2815d440 0x7fcd28153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:12.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.162+0000 7fcd2dc00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcd28160340 0x7fcd281543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:12.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.162+0000 7fcd2dc00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcd280ad670 con 0x7fcd2815d440 2024-09-21T07:55:12.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.162+0000 7fcd2dc00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcd280ad4f0 con 0x7fcd281539f0 2024-09-21T07:55:12.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.162+0000 7fcd2dc00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7fcd280ad7f0 con 0x7fcd28160340 2024-09-21T07:55:12.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.162+0000 7fcd27400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd2815d440 0x7fcd28153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:12.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.162+0000 7fcd27400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd2815d440 0x7fcd28153e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35236/0 (socket says 172.21.15.39:35236) 2024-09-21T07:55:12.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.162+0000 7fcd27400640 1 -- 172.21.15.39:0/4053584769 learned_addr learned my addr 172.21.15.39:0/4053584769 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:55:12.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.162+0000 7fcd27e00640 1 --2- 172.21.15.39:0/4053584769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd281539f0 0x7fcd281445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:12.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.162+0000 7fcd2d200640 1 --2- 172.21.15.39:0/4053584769 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcd28160340 0x7fcd281543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:12.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.162+0000 7fcd27400640 1 -- 172.21.15.39:0/4053584769 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcd28160340 msgr2=0x7fcd281543b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:12.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.162+0000 7fcd27400640 1 --2- 172.21.15.39:0/4053584769 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7fcd28160340 0x7fcd281543b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:12.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.162+0000 7fcd27400640 1 -- 172.21.15.39:0/4053584769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd281539f0 msgr2=0x7fcd281445c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:12.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.162+0000 7fcd27400640 1 --2- 172.21.15.39:0/4053584769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd281539f0 0x7fcd281445c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:12.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.162+0000 7fcd27400640 1 -- 172.21.15.39:0/4053584769 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd281571b0 con 0x7fcd2815d440 2024-09-21T07:55:12.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.162+0000 7fcd27e00640 1 --2- 172.21.15.39:0/4053584769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd281539f0 0x7fcd281445c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:55:12.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.163+0000 7fcd27400640 1 --2- 172.21.15.39:0/4053584769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd2815d440 0x7fcd28153e70 secure :-1 s=READY pgs=2923 cs=0 l=1 rev1=1 crypto rx=0x7fcd1400c400 tx=0x7fcd1400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:12.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.163+0000 7fcd1fe00640 1 -- 172.21.15.39:0/4053584769 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd14011790 con 0x7fcd2815d440 2024-09-21T07:55:12.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.163+0000 7fcd1fe00640 1 -- 172.21.15.39:0/4053584769 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd14017070 con 0x7fcd2815d440 2024-09-21T07:55:12.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.163+0000 7fcd1fe00640 1 -- 172.21.15.39:0/4053584769 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd14011cc0 con 0x7fcd2815d440 2024-09-21T07:55:12.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.163+0000 7fcd2dc00640 1 -- 172.21.15.39:0/4053584769 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd2813e6a0 con 0x7fcd2815d440 2024-09-21T07:55:12.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.163+0000 7fcd2dc00640 1 -- 172.21.15.39:0/4053584769 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd2813ec60 con 0x7fcd2815d440 2024-09-21T07:55:12.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.164+0000 7fcd2dc00640 1 -- 172.21.15.39:0/4053584769 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcd280b4c40 con 0x7fcd2815d440 2024-09-21T07:55:12.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.165+0000 7fcd1fe00640 1 -- 172.21.15.39:0/4053584769 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcd1401c460 con 0x7fcd2815d440 2024-09-21T07:55:12.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.166+0000 7fcd1fe00640 1 --2- 172.21.15.39:0/4053584769 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcd08078b90 0x7fcd0807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:12.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.167+0000 7fcd1fe00640 1 -- 172.21.15.39:0/4053584769 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(269..269 src has 1..269) ==== 6278+0+0 (secure 0 0 0) 0x7fcd1409b810 con 0x7fcd2815d440 2024-09-21T07:55:12.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.167+0000 7fcd1fe00640 1 --2- 172.21.15.39:0/4053584769 >> [v2:172.21.15.148:6818/1438255756,v1:172.21.15.148:6819/1438255756] conn(0x7fcd08081400 0x7fcd080838f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:12.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.167+0000 7fcd1fe00640 1 -- 172.21.15.39:0/4053584769 --> [v2:172.21.15.148:6818/1438255756,v1:172.21.15.148:6819/1438255756] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcd14010e50 con 0x7fcd08081400 2024-09-21T07:55:12.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.167+0000 7fcd1fe00640 1 -- 172.21.15.39:0/4053584769 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=269) ==== 24+0+0 (secure 0 0 0) 0x7fcd1409bbc0 con 0x7fcd2815d440 2024-09-21T07:55:12.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.167+0000 7fcd27e00640 1 --2- 172.21.15.39:0/4053584769 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcd08078b90 0x7fcd0807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:12.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.167+0000 7fcd2d200640 1 --2- 172.21.15.39:0/4053584769 >> [v2:172.21.15.148:6818/1438255756,v1:172.21.15.148:6819/1438255756] conn(0x7fcd08081400 0x7fcd080838f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:12.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.168+0000 7fcd27e00640 1 --2- 172.21.15.39:0/4053584769 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcd08078b90 0x7fcd0807b0a0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fcd18009b80 tx=0x7fcd18005f90 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:12.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.168+0000 7fcd2d200640 1 --2- 172.21.15.39:0/4053584769 >> [v2:172.21.15.148:6818/1438255756,v1:172.21.15.148:6819/1438255756] conn(0x7fcd08081400 0x7fcd080838f0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:12.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.169+0000 7fcd1fe00640 1 -- 172.21.15.39:0/4053584769 <== osd.7 v2:172.21.15.148:6818/1438255756 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcd14010e50 con 0x7fcd08081400 2024-09-21T07:55:12.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.182+0000 7fcd2dc00640 1 -- 172.21.15.39:0/4053584769 --> [v2:172.21.15.148:6818/1438255756,v1:172.21.15.148:6819/1438255756] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fcd28156f60 con 0x7fcd08081400 2024-09-21T07:55:12.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.183+0000 7fcd1fe00640 1 -- 172.21.15.39:0/4053584769 <== osd.7 v2:172.21.15.148:6818/1438255756 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fcd28156f60 con 0x7fcd08081400 2024-09-21T07:55:12.185 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-21T07:55:12.185 INFO:teuthology.orchestra.run.smithi039.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T07:55:12.185 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-21T07:55:12.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.183+0000 7fcd2dc00640 1 -- 172.21.15.39:0/4053584769 >> [v2:172.21.15.148:6818/1438255756,v1:172.21.15.148:6819/1438255756] conn(0x7fcd08081400 msgr2=0x7fcd080838f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:12.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.183+0000 7fcd2dc00640 1 --2- 172.21.15.39:0/4053584769 >> [v2:172.21.15.148:6818/1438255756,v1:172.21.15.148:6819/1438255756] conn(0x7fcd08081400 0x7fcd080838f0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:12.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.183+0000 7fcd2dc00640 1 -- 172.21.15.39:0/4053584769 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcd08078b90 msgr2=0x7fcd0807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:12.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.183+0000 7fcd2dc00640 1 --2- 172.21.15.39:0/4053584769 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7fcd08078b90 0x7fcd0807b0a0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fcd18009b80 tx=0x7fcd18005f90 comp rx=0 tx=0).stop 2024-09-21T07:55:12.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.183+0000 7fcd2dc00640 1 -- 172.21.15.39:0/4053584769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd2815d440 msgr2=0x7fcd28153e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:12.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.183+0000 7fcd2dc00640 1 --2- 172.21.15.39:0/4053584769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd2815d440 0x7fcd28153e70 secure :-1 s=READY pgs=2923 cs=0 l=1 rev1=1 crypto rx=0x7fcd1400c400 tx=0x7fcd1400c8c0 comp rx=0 tx=0).stop 2024-09-21T07:55:12.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.183+0000 7fcd2d200640 1 -- 172.21.15.39:0/4053584769 reap_dead start 2024-09-21T07:55:12.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.183+0000 7fcd2dc00640 1 -- 172.21.15.39:0/4053584769 shutdown_connections 2024-09-21T07:55:12.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.183+0000 7fcd2dc00640 1 -- 172.21.15.39:0/4053584769 >> 172.21.15.39:0/4053584769 conn(0x7fcd2806d660 msgr2=0x7fcd2815ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:12.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.184+0000 7fcd2dc00640 1 -- 172.21.15.39:0/4053584769 shutdown_connections 2024-09-21T07:55:12.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.184+0000 7fcd2dc00640 1 -- 172.21.15.39:0/4053584769 wait complete. 2024-09-21T07:55:12.205 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-21T07:55:12.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.429+0000 7f4f61400640 1 -- 172.21.15.39:0/2538965050 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f5c0b5920 msgr2=0x7f4f5c0b39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:12.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.429+0000 7f4f61400640 1 --2- 172.21.15.39:0/2538965050 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f5c0b5920 0x7f4f5c0b39e0 secure :-1 s=READY pgs=2924 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c0730f0 tx=0x7f4f5402aa30 comp rx=0 tx=0).stop 2024-09-21T07:55:12.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.429+0000 7f4f61400640 1 -- 172.21.15.39:0/2538965050 shutdown_connections 2024-09-21T07:55:12.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.429+0000 7f4f61400640 1 --2- 172.21.15.39:0/2538965050 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f5c0b5920 0x7f4f5c0b39e0 unknown :-1 s=CLOSED pgs=2924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:12.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.429+0000 7f4f61400640 1 --2- 172.21.15.39:0/2538965050 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4f5c15f2d0 0x7f4f5c0b53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:12.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.429+0000 7f4f61400640 1 --2- 172.21.15.39:0/2538965050 >> v2:172.21.15.148:3300/0 conn(0x7f4f5c15bce0 0x7f4f5c0d0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:12.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.429+0000 7f4f61400640 1 -- 172.21.15.39:0/2538965050 >> 172.21.15.39:0/2538965050 conn(0x7f4f5c0d0cb0 msgr2=0x7f4f5c0d10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:12.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.429+0000 7f4f61400640 1 -- 172.21.15.39:0/2538965050 shutdown_connections 2024-09-21T07:55:12.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.429+0000 7f4f61400640 1 -- 172.21.15.39:0/2538965050 wait complete. 2024-09-21T07:55:12.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.431+0000 7f4f61400640 1 Processor -- start 2024-09-21T07:55:12.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.432+0000 7f4f61400640 1 -- start start 2024-09-21T07:55:12.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.433+0000 7f4f61400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f5c0b5920 0x7f4f5c1513a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:12.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.433+0000 7f4f61400640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f5c15bce0 0x7f4f5c165bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:12.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.433+0000 7f4f61400640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4f5c15f2d0 0x7f4f5c1498a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:12.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.433+0000 7f4f61400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5c0b42b0 con 0x7f4f5c0b5920 2024-09-21T07:55:12.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.433+0000 7f4f61400640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5c0b4130 con 0x7f4f5c15bce0 2024-09-21T07:55:12.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.433+0000 7f4f61400640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f4f5c0b4430 con 0x7f4f5c15f2d0 2024-09-21T07:55:12.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.433+0000 7f4f5be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f5c0b5920 0x7f4f5c1513a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:12.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.433+0000 7f4f5be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f5c0b5920 0x7f4f5c1513a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35256/0 (socket says 172.21.15.39:35256) 2024-09-21T07:55:12.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.433+0000 7f4f5be00640 1 -- 172.21.15.39:0/816822651 learned_addr learned my addr 172.21.15.39:0/816822651 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:55:12.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.433+0000 7f4f53e00640 1 --2- 172.21.15.39:0/816822651 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f5c15bce0 0x7f4f5c165bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:12.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.433+0000 7f4f60a00640 1 --2- 172.21.15.39:0/816822651 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4f5c15f2d0 0x7f4f5c1498a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:12.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.434+0000 7f4f5be00640 1 -- 172.21.15.39:0/816822651 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4f5c15f2d0 msgr2=0x7f4f5c1498a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:12.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.434+0000 7f4f5be00640 1 --2- 172.21.15.39:0/816822651 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4f5c15f2d0 0x7f4f5c1498a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:12.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.434+0000 7f4f5be00640 1 -- 172.21.15.39:0/816822651 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f5c15bce0 msgr2=0x7f4f5c165bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:12.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.434+0000 7f4f5be00640 1 --2- 172.21.15.39:0/816822651 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f5c15bce0 0x7f4f5c165bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:12.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.434+0000 7f4f5be00640 1 -- 172.21.15.39:0/816822651 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f5c166130 con 0x7f4f5c0b5920 2024-09-21T07:55:12.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.434+0000 7f4f53e00640 1 --2- 172.21.15.39:0/816822651 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f5c15bce0 0x7f4f5c165bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:55:12.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.434+0000 7f4f5be00640 1 --2- 172.21.15.39:0/816822651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f5c0b5920 0x7f4f5c1513a0 secure :-1 s=READY pgs=2925 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c00e740 tx=0x7f4f4c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:12.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.434+0000 7f4f58c00640 1 -- 172.21.15.39:0/816822651 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c00ccc0 con 0x7f4f5c0b5920 2024-09-21T07:55:12.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.434+0000 7f4f58c00640 1 -- 172.21.15.39:0/816822651 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f4c00ee50 con 0x7f4f5c0b5920 2024-09-21T07:55:12.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.434+0000 7f4f61400640 1 -- 172.21.15.39:0/816822651 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f5c168430 con 0x7f4f5c0b5920 2024-09-21T07:55:12.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.434+0000 7f4f58c00640 1 -- 172.21.15.39:0/816822651 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c010670 con 0x7f4f5c0b5920 2024-09-21T07:55:12.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.434+0000 7f4f61400640 1 -- 172.21.15.39:0/816822651 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f5c168150 con 0x7f4f5c0b5920 2024-09-21T07:55:12.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.435+0000 7f4f61400640 1 -- 172.21.15.39:0/816822651 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f5c0d0540 con 0x7f4f5c0b5920 2024-09-21T07:55:12.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.437+0000 7f4f58c00640 1 -- 172.21.15.39:0/816822651 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4f4c010810 con 0x7f4f5c0b5920 2024-09-21T07:55:12.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.438+0000 7f4f58c00640 1 --2- 172.21.15.39:0/816822651 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4f38078b90 0x7f4f3807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:12.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.438+0000 7f4f58c00640 1 -- 172.21.15.39:0/816822651 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(269..269 src has 1..269) ==== 6278+0+0 (secure 0 0 0) 0x7f4f4c09b8f0 con 0x7f4f5c0b5920 2024-09-21T07:55:12.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.438+0000 7f4f53e00640 1 --2- 172.21.15.39:0/816822651 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4f38078b90 0x7f4f3807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:12.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.439+0000 7f4f53e00640 1 --2- 172.21.15.39:0/816822651 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4f38078b90 0x7f4f3807b0a0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c152550 tx=0x7f4f44005ef0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:12.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.442+0000 7f4f58c00640 1 -- 172.21.15.39:0/816822651 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4f5c166130 con 0x7f4f5c0b5920 2024-09-21T07:55:12.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.640+0000 7f4f61400640 1 -- 172.21.15.39:0/816822651 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f4f5c0b3c30 con 0x7f4f38078b90 2024-09-21T07:55:12.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.640+0000 7f4f58c00640 1 -- 172.21.15.39:0/816822651 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f4f5c0b3c30 con 0x7f4f38078b90 2024-09-21T07:55:12.642 INFO:teuthology.orchestra.run.smithi039.stderr:instructed osd(s) 7 to deep-scrub 2024-09-21T07:55:12.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.643+0000 7f4f61400640 1 -- 172.21.15.39:0/816822651 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4f38078b90 msgr2=0x7f4f3807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:12.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.643+0000 7f4f61400640 1 --2- 172.21.15.39:0/816822651 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4f38078b90 0x7f4f3807b0a0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c152550 tx=0x7f4f44005ef0 comp rx=0 tx=0).stop 2024-09-21T07:55:12.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.643+0000 7f4f61400640 1 -- 172.21.15.39:0/816822651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f5c0b5920 msgr2=0x7f4f5c1513a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:12.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.643+0000 7f4f61400640 1 --2- 172.21.15.39:0/816822651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f5c0b5920 0x7f4f5c1513a0 secure :-1 s=READY pgs=2925 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c00e740 tx=0x7f4f4c00ec00 comp rx=0 tx=0).stop 2024-09-21T07:55:12.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.643+0000 7f4f61400640 1 -- 172.21.15.39:0/816822651 shutdown_connections 2024-09-21T07:55:12.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.643+0000 7f4f61400640 1 --2- 172.21.15.39:0/816822651 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f4f38078b90 0x7f4f3807b0a0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:12.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.643+0000 7f4f61400640 1 --2- 172.21.15.39:0/816822651 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f4f5c15f2d0 0x7f4f5c1498a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:12.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.643+0000 7f4f61400640 1 --2- 172.21.15.39:0/816822651 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f5c15bce0 0x7f4f5c165bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:12.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.643+0000 7f4f61400640 1 --2- 172.21.15.39:0/816822651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f5c0b5920 0x7f4f5c1513a0 unknown :-1 s=CLOSED pgs=2925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:12.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.643+0000 7f4f61400640 1 -- 172.21.15.39:0/816822651 >> 172.21.15.39:0/816822651 conn(0x7f4f5c0d0cb0 msgr2=0x7f4f5c14e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:12.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.643+0000 7f4f61400640 1 -- 172.21.15.39:0/816822651 shutdown_connections 2024-09-21T07:55:12.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.643+0000 7f4f61400640 1 -- 172.21.15.39:0/816822651 wait complete. 2024-09-21T07:55:12.667 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:55:12.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.894+0000 7ff69f800640 1 -- 172.21.15.39:0/2072137406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff698070bc0 msgr2=0x7ff6980d06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:12.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.894+0000 7ff69f800640 1 --2- 172.21.15.39:0/2072137406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff698070bc0 0x7ff6980d06b0 secure :-1 s=READY pgs=2926 cs=0 l=1 rev1=1 crypto rx=0x7ff6980cdd30 tx=0x7ff68c02a580 comp rx=0 tx=0).stop 2024-09-21T07:55:12.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.894+0000 7ff69f800640 1 -- 172.21.15.39:0/2072137406 shutdown_connections 2024-09-21T07:55:12.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.894+0000 7ff69f800640 1 --2- 172.21.15.39:0/2072137406 >> v2:172.21.15.148:3300/0 conn(0x7ff6980d0bf0 0x7ff6980cf450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:12.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.894+0000 7ff69f800640 1 --2- 172.21.15.39:0/2072137406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff698070bc0 0x7ff6980d06b0 unknown :-1 s=CLOSED pgs=2926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:12.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.894+0000 7ff69f800640 1 -- 172.21.15.39:0/2072137406 >> 172.21.15.39:0/2072137406 conn(0x7ff69806cfc0 msgr2=0x7ff69806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:12.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.894+0000 7ff69f800640 1 -- 172.21.15.39:0/2072137406 shutdown_connections 2024-09-21T07:55:12.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.894+0000 7ff69f800640 1 -- 172.21.15.39:0/2072137406 wait complete. 2024-09-21T07:55:12.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.896+0000 7ff69f800640 1 Processor -- start 2024-09-21T07:55:12.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.896+0000 7ff69f800640 1 -- start start 2024-09-21T07:55:12.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.897+0000 7ff69f800640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff698070bc0 0x7ff69814a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:12.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.898+0000 7ff69f800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6980cf990 0x7ff6980b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:12.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.898+0000 7ff69f800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6980d0bf0 0x7ff6980a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:12.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.898+0000 7ff69f800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff6980cffb0 con 0x7ff6980cf990 2024-09-21T07:55:12.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.898+0000 7ff69f800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff698160a30 con 0x7ff6980d0bf0 2024-09-21T07:55:12.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.898+0000 7ff69f800640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7ff6981f62e0 con 0x7ff698070bc0 2024-09-21T07:55:12.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.898+0000 7ff697e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6980cf990 0x7ff6980b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:12.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.898+0000 7ff697e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6980cf990 0x7ff6980b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35276/0 (socket says 172.21.15.39:35276) 2024-09-21T07:55:12.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.898+0000 7ff697e00640 1 -- 172.21.15.39:0/3826385792 learned_addr learned my addr 172.21.15.39:0/3826385792 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:55:12.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.898+0000 7ff69ce00640 1 --2- 172.21.15.39:0/3826385792 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff698070bc0 0x7ff69814a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:12.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.898+0000 7ff69d800640 1 --2- 172.21.15.39:0/3826385792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6980d0bf0 0x7ff6980a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:12.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.898+0000 7ff697e00640 1 -- 172.21.15.39:0/3826385792 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff698070bc0 msgr2=0x7ff69814a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:12.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.898+0000 7ff697e00640 1 --2- 172.21.15.39:0/3826385792 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff698070bc0 0x7ff69814a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:12.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.898+0000 7ff697e00640 1 -- 172.21.15.39:0/3826385792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6980d0bf0 msgr2=0x7ff6980a94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:12.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.898+0000 7ff697e00640 1 --2- 172.21.15.39:0/3826385792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6980d0bf0 0x7ff6980a94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:12.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.898+0000 7ff697e00640 1 -- 172.21.15.39:0/3826385792 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff698154870 con 0x7ff6980cf990 2024-09-21T07:55:12.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.898+0000 7ff69ce00640 1 --2- 172.21.15.39:0/3826385792 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff698070bc0 0x7ff69814a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:55:12.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.898+0000 7ff69d800640 1 --2- 172.21.15.39:0/3826385792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6980d0bf0 0x7ff6980a94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:55:12.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.899+0000 7ff697e00640 1 --2- 172.21.15.39:0/3826385792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6980cf990 0x7ff6980b5050 secure :-1 s=READY pgs=2927 cs=0 l=1 rev1=1 crypto rx=0x7ff68400d750 tx=0x7ff68400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:12.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.899+0000 7ff694c00640 1 -- 172.21.15.39:0/3826385792 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff684017070 con 0x7ff6980cf990 2024-09-21T07:55:12.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.899+0000 7ff694c00640 1 -- 172.21.15.39:0/3826385792 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff684009b10 con 0x7ff6980cf990 2024-09-21T07:55:12.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.899+0000 7ff694c00640 1 -- 172.21.15.39:0/3826385792 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff684010460 con 0x7ff6980cf990 2024-09-21T07:55:12.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.899+0000 7ff69f800640 1 -- 172.21.15.39:0/3826385792 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff698154b50 con 0x7ff6980cf990 2024-09-21T07:55:12.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.899+0000 7ff69f800640 1 -- 172.21.15.39:0/3826385792 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6980b4850 con 0x7ff6980cf990 2024-09-21T07:55:12.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.900+0000 7ff69f800640 1 -- 172.21.15.39:0/3826385792 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff698160f00 con 0x7ff6980cf990 2024-09-21T07:55:12.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.901+0000 7ff694c00640 1 -- 172.21.15.39:0/3826385792 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff68401a070 con 0x7ff6980cf990 2024-09-21T07:55:12.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.902+0000 7ff694c00640 1 --2- 172.21.15.39:0/3826385792 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff678078b90 0x7ff67807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:12.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.902+0000 7ff694c00640 1 -- 172.21.15.39:0/3826385792 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(269..269 src has 1..269) ==== 6278+0+0 (secure 0 0 0) 0x7ff68409b360 con 0x7ff6980cf990 2024-09-21T07:55:12.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.903+0000 7ff69ce00640 1 --2- 172.21.15.39:0/3826385792 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff678078b90 0x7ff67807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:12.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.904+0000 7ff69ce00640 1 --2- 172.21.15.39:0/3826385792 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff678078b90 0x7ff67807b0a0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7ff68c033000 tx=0x7ff68c004730 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:12.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:12.907+0000 7ff694c00640 1 -- 172.21.15.39:0/3826385792 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff684063680 con 0x7ff6980cf990 2024-09-21T07:55:13.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:13.101+0000 7ff69f800640 1 -- 172.21.15.39:0/3826385792 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff69815d7f0 con 0x7ff678078b90 2024-09-21T07:55:13.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:13.106+0000 7ff694c00640 1 -- 172.21.15.39:0/3826385792 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+256142 (secure 0 0 0) 0x7ff68c004040 con 0x7ff678078b90 2024-09-21T07:55:13.111 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:55:13.112 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:55:13.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:13.110+0000 7ff69f800640 1 -- 172.21.15.39:0/3826385792 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff678078b90 msgr2=0x7ff67807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:13.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:13.110+0000 7ff69f800640 1 --2- 172.21.15.39:0/3826385792 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff678078b90 0x7ff67807b0a0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7ff68c033000 tx=0x7ff68c004730 comp rx=0 tx=0).stop 2024-09-21T07:55:13.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:13.110+0000 7ff69f800640 1 -- 172.21.15.39:0/3826385792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6980cf990 msgr2=0x7ff6980b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:13.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:13.110+0000 7ff69f800640 1 --2- 172.21.15.39:0/3826385792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6980cf990 0x7ff6980b5050 secure :-1 s=READY pgs=2927 cs=0 l=1 rev1=1 crypto rx=0x7ff68400d750 tx=0x7ff68400dc10 comp rx=0 tx=0).stop 2024-09-21T07:55:13.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:13.110+0000 7ff69f800640 1 -- 172.21.15.39:0/3826385792 shutdown_connections 2024-09-21T07:55:13.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:13.110+0000 7ff69f800640 1 --2- 172.21.15.39:0/3826385792 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7ff678078b90 0x7ff67807b0a0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:13.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:13.110+0000 7ff69f800640 1 --2- 172.21.15.39:0/3826385792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6980d0bf0 0x7ff6980a94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:13.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:13.110+0000 7ff69f800640 1 --2- 172.21.15.39:0/3826385792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6980cf990 0x7ff6980b5050 unknown :-1 s=CLOSED pgs=2927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:13.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:13.110+0000 7ff69f800640 1 --2- 172.21.15.39:0/3826385792 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7ff698070bc0 0x7ff69814a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:13.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:13.110+0000 7ff69f800640 1 -- 172.21.15.39:0/3826385792 >> 172.21.15.39:0/3826385792 conn(0x7ff69806cfc0 msgr2=0x7ff69806fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:13.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:13.110+0000 7ff69f800640 1 -- 172.21.15.39:0/3826385792 shutdown_connections 2024-09-21T07:55:13.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:13.110+0000 7ff69f800640 1 -- 172.21.15.39:0/3826385792 wait complete. 2024-09-21T07:55:13.134 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":929,"stamp":"2024-09-21T07:55:11.988825+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":466028,"kb_used_data":27720,"kb_used_omap":12,"kb_used_meta":438131,"kb_avail":754508692,"statfs":{"total":773094113280,"available":772616900608,"internally_reserved":0,"allocated":28385280,"data_stored":15991320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":448646704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003128"},"pg_stats":[{"pgid":"12.d","version":"0'0","reported_seq":403,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.315418+0000","last_change":"2024-09-21T07:35:27.879605+0000","last_active":"2024-09-21T07:55:00.315418+0000","last_peered":"2024-09-21T07:55:00.315418+0000","last_clean":"2024-09-21T07:55:00.315418+0000","last_became_active":"2024-09-21T07:35:27.879274+0000","last_became_peered":"2024-09-21T07:35:27.879274+0000","last_unstale":"2024-09-21T07:55:00.315418+0000","last_undegraded":"2024-09-21T07:55:00.315418+0000","last_fullsized":"2024-09-21T07:55:00.315418+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:24.241636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":413,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.312401+0000","last_change":"2024-09-21T07:35:29.491336+0000","last_active":"2024-09-21T07:55:00.312401+0000","last_peered":"2024-09-21T07:55:00.312401+0000","last_clean":"2024-09-21T07:55:00.312401+0000","last_became_active":"2024-09-21T07:35:26.866484+0000","last_became_peered":"2024-09-21T07:35:26.866484+0000","last_unstale":"2024-09-21T07:55:00.312401+0000","last_undegraded":"2024-09-21T07:55:00.312401+0000","last_fullsized":"2024-09-21T07:55:00.312401+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:10.449665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076010100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":590,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.311370+0000","last_change":"2024-09-21T07:33:04.017469+0000","last_active":"2024-09-21T07:55:00.311370+0000","last_peered":"2024-09-21T07:55:00.311370+0000","last_clean":"2024-09-21T07:55:00.311370+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:55:00.311370+0000","last_undegraded":"2024-09-21T07:55:00.311370+0000","last_fullsized":"2024-09-21T07:55:00.311370+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:33:02.438318+0000","last_clean_scrub_stamp":"2024-09-21T07:33:02.438318+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:34.520989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":480,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.632658+0000","last_change":"2024-09-21T07:34:24.539248+0000","last_active":"2024-09-21T07:54:59.632658+0000","last_peered":"2024-09-21T07:54:59.632658+0000","last_clean":"2024-09-21T07:54:59.632658+0000","last_became_active":"2024-09-21T07:34:23.273465+0000","last_became_peered":"2024-09-21T07:34:23.273465+0000","last_unstale":"2024-09-21T07:54:59.632658+0000","last_undegraded":"2024-09-21T07:54:59.632658+0000","last_fullsized":"2024-09-21T07:54:59.632658+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:55.517452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060627700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":407,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.312347+0000","last_change":"2024-09-21T07:35:27.864492+0000","last_active":"2024-09-21T07:55:00.312347+0000","last_peered":"2024-09-21T07:55:00.312347+0000","last_clean":"2024-09-21T07:55:00.312347+0000","last_became_active":"2024-09-21T07:35:27.864176+0000","last_became_peered":"2024-09-21T07:35:27.864176+0000","last_unstale":"2024-09-21T07:55:00.312347+0000","last_undegraded":"2024-09-21T07:55:00.312347+0000","last_fullsized":"2024-09-21T07:55:00.312347+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:05.153467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":410,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.314353+0000","last_change":"2024-09-21T07:35:29.493678+0000","last_active":"2024-09-21T07:55:00.314353+0000","last_peered":"2024-09-21T07:55:00.314353+0000","last_clean":"2024-09-21T07:55:00.314353+0000","last_became_active":"2024-09-21T07:35:26.870749+0000","last_became_peered":"2024-09-21T07:35:26.870749+0000","last_unstale":"2024-09-21T07:55:00.314353+0000","last_undegraded":"2024-09-21T07:55:00.314353+0000","last_fullsized":"2024-09-21T07:55:00.314353+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:55.367877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010159570000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":456,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.175411+0000","last_change":"2024-09-21T07:34:24.597274+0000","last_active":"2024-09-21T07:55:00.175411+0000","last_peered":"2024-09-21T07:55:00.175411+0000","last_clean":"2024-09-21T07:55:00.175411+0000","last_became_active":"2024-09-21T07:34:23.279560+0000","last_became_peered":"2024-09-21T07:34:23.279560+0000","last_unstale":"2024-09-21T07:55:00.175411+0000","last_undegraded":"2024-09-21T07:55:00.175411+0000","last_fullsized":"2024-09-21T07:55:00.175411+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:15.845265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059699799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":407,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.631402+0000","last_change":"2024-09-21T07:35:27.874387+0000","last_active":"2024-09-21T07:54:59.631402+0000","last_peered":"2024-09-21T07:54:59.631402+0000","last_clean":"2024-09-21T07:54:59.631402+0000","last_became_active":"2024-09-21T07:35:27.873277+0000","last_became_peered":"2024-09-21T07:35:27.873277+0000","last_unstale":"2024-09-21T07:54:59.631402+0000","last_undegraded":"2024-09-21T07:54:59.631402+0000","last_fullsized":"2024-09-21T07:54:59.631402+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:04.581463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":406,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.175552+0000","last_change":"2024-09-21T07:35:29.387094+0000","last_active":"2024-09-21T07:55:00.175552+0000","last_peered":"2024-09-21T07:55:00.175552+0000","last_clean":"2024-09-21T07:55:00.175552+0000","last_became_active":"2024-09-21T07:35:26.874757+0000","last_became_peered":"2024-09-21T07:35:26.874757+0000","last_unstale":"2024-09-21T07:55:00.175552+0000","last_undegraded":"2024-09-21T07:55:00.175552+0000","last_fullsized":"2024-09-21T07:55:00.175552+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:33.542750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061065700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":463,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.313869+0000","last_change":"2024-09-21T07:34:24.259906+0000","last_active":"2024-09-21T07:55:00.313869+0000","last_peered":"2024-09-21T07:55:00.313869+0000","last_clean":"2024-09-21T07:55:00.313869+0000","last_became_active":"2024-09-21T07:34:23.635282+0000","last_became_peered":"2024-09-21T07:34:23.635282+0000","last_unstale":"2024-09-21T07:55:00.313869+0000","last_undegraded":"2024-09-21T07:55:00.313869+0000","last_fullsized":"2024-09-21T07:55:00.313869+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:00.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001527664,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":407,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.634628+0000","last_change":"2024-09-21T07:35:27.873645+0000","last_active":"2024-09-21T07:54:59.634628+0000","last_peered":"2024-09-21T07:54:59.634628+0000","last_clean":"2024-09-21T07:54:59.634628+0000","last_became_active":"2024-09-21T07:35:27.873245+0000","last_became_peered":"2024-09-21T07:35:27.873245+0000","last_unstale":"2024-09-21T07:54:59.634628+0000","last_undegraded":"2024-09-21T07:54:59.634628+0000","last_fullsized":"2024-09-21T07:54:59.634628+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:55.090953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":413,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.633609+0000","last_change":"2024-09-21T07:35:28.861744+0000","last_active":"2024-09-21T07:54:59.633609+0000","last_peered":"2024-09-21T07:54:59.633609+0000","last_clean":"2024-09-21T07:54:59.633609+0000","last_became_active":"2024-09-21T07:35:26.876462+0000","last_became_peered":"2024-09-21T07:35:26.876462+0000","last_unstale":"2024-09-21T07:54:59.633609+0000","last_undegraded":"2024-09-21T07:54:59.633609+0000","last_fullsized":"2024-09-21T07:54:59.633609+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:48.632568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012020920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":464,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.630668+0000","last_change":"2024-09-21T07:34:25.085454+0000","last_active":"2024-09-21T07:54:59.630668+0000","last_peered":"2024-09-21T07:54:59.630668+0000","last_clean":"2024-09-21T07:54:59.630668+0000","last_became_active":"2024-09-21T07:34:23.279205+0000","last_became_peered":"2024-09-21T07:34:23.279205+0000","last_unstale":"2024-09-21T07:54:59.630668+0000","last_undegraded":"2024-09-21T07:54:59.630668+0000","last_fullsized":"2024-09-21T07:54:59.630668+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:54.231666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095320599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":405,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.314238+0000","last_change":"2024-09-21T07:35:27.861373+0000","last_active":"2024-09-21T07:55:00.314238+0000","last_peered":"2024-09-21T07:55:00.314238+0000","last_clean":"2024-09-21T07:55:00.314238+0000","last_became_active":"2024-09-21T07:35:27.861016+0000","last_became_peered":"2024-09-21T07:35:27.861016+0000","last_unstale":"2024-09-21T07:55:00.314238+0000","last_undegraded":"2024-09-21T07:55:00.314238+0000","last_fullsized":"2024-09-21T07:55:00.314238+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:56.043763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":413,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.311346+0000","last_change":"2024-09-21T07:35:29.053677+0000","last_active":"2024-09-21T07:55:00.311346+0000","last_peered":"2024-09-21T07:55:00.311346+0000","last_clean":"2024-09-21T07:55:00.311346+0000","last_became_active":"2024-09-21T07:35:26.875496+0000","last_became_peered":"2024-09-21T07:35:26.875496+0000","last_unstale":"2024-09-21T07:55:00.311346+0000","last_undegraded":"2024-09-21T07:55:00.311346+0000","last_fullsized":"2024-09-21T07:55:00.311346+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:04.850784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065859000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":465,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.310605+0000","last_change":"2024-09-21T07:34:25.028521+0000","last_active":"2024-09-21T07:55:00.310605+0000","last_peered":"2024-09-21T07:55:00.310605+0000","last_clean":"2024-09-21T07:55:00.310605+0000","last_became_active":"2024-09-21T07:34:23.635467+0000","last_became_peered":"2024-09-21T07:34:23.635467+0000","last_unstale":"2024-09-21T07:55:00.310605+0000","last_undegraded":"2024-09-21T07:55:00.310605+0000","last_fullsized":"2024-09-21T07:55:00.310605+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:33.729123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000660558,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":405,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.633153+0000","last_change":"2024-09-21T07:35:27.871201+0000","last_active":"2024-09-21T07:54:59.633153+0000","last_peered":"2024-09-21T07:54:59.633153+0000","last_clean":"2024-09-21T07:54:59.633153+0000","last_became_active":"2024-09-21T07:35:27.870816+0000","last_became_peered":"2024-09-21T07:35:27.870816+0000","last_unstale":"2024-09-21T07:54:59.633153+0000","last_undegraded":"2024-09-21T07:54:59.633153+0000","last_fullsized":"2024-09-21T07:54:59.633153+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:04.787450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":412,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.314136+0000","last_change":"2024-09-21T07:35:29.491874+0000","last_active":"2024-09-21T07:55:00.314136+0000","last_peered":"2024-09-21T07:55:00.314136+0000","last_clean":"2024-09-21T07:55:00.314136+0000","last_became_active":"2024-09-21T07:35:26.866381+0000","last_became_peered":"2024-09-21T07:35:26.866381+0000","last_unstale":"2024-09-21T07:55:00.314136+0000","last_undegraded":"2024-09-21T07:55:00.314136+0000","last_fullsized":"2024-09-21T07:55:00.314136+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:36.404647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073235399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":476,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:11.012278+0000","last_change":"2024-09-21T07:55:11.012278+0000","last_active":"2024-09-21T07:55:11.012278+0000","last_peered":"2024-09-21T07:55:11.012278+0000","last_clean":"2024-09-21T07:55:11.012278+0000","last_became_active":"2024-09-21T07:34:23.273431+0000","last_became_peered":"2024-09-21T07:34:23.273431+0000","last_unstale":"2024-09-21T07:55:11.012278+0000","last_undegraded":"2024-09-21T07:55:11.012278+0000","last_fullsized":"2024-09-21T07:55:11.012278+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:11.012203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:11.012203+0000","last_clean_scrub_stamp":"2024-09-21T07:55:11.012203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:43.236568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019803310000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":405,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.632132+0000","last_change":"2024-09-21T07:35:27.871736+0000","last_active":"2024-09-21T07:54:59.632132+0000","last_peered":"2024-09-21T07:54:59.632132+0000","last_clean":"2024-09-21T07:54:59.632132+0000","last_became_active":"2024-09-21T07:35:27.871232+0000","last_became_peered":"2024-09-21T07:35:27.871232+0000","last_unstale":"2024-09-21T07:54:59.632132+0000","last_undegraded":"2024-09-21T07:54:59.632132+0000","last_fullsized":"2024-09-21T07:54:59.632132+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:04.419301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":414,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.311567+0000","last_change":"2024-09-21T07:35:29.491246+0000","last_active":"2024-09-21T07:55:00.311567+0000","last_peered":"2024-09-21T07:55:00.311567+0000","last_clean":"2024-09-21T07:55:00.311567+0000","last_became_active":"2024-09-21T07:35:26.866783+0000","last_became_peered":"2024-09-21T07:35:26.866783+0000","last_unstale":"2024-09-21T07:55:00.311567+0000","last_undegraded":"2024-09-21T07:55:00.311567+0000","last_fullsized":"2024-09-21T07:55:00.311567+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:28.881914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098474099999999992,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":465,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.311456+0000","last_change":"2024-09-21T07:34:25.028640+0000","last_active":"2024-09-21T07:55:00.311456+0000","last_peered":"2024-09-21T07:55:00.311456+0000","last_clean":"2024-09-21T07:55:00.311456+0000","last_became_active":"2024-09-21T07:34:23.260950+0000","last_became_peered":"2024-09-21T07:34:23.260950+0000","last_unstale":"2024-09-21T07:55:00.311456+0000","last_undegraded":"2024-09-21T07:55:00.311456+0000","last_fullsized":"2024-09-21T07:55:00.311456+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:23:42.121398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055135099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":408,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.319249+0000","last_change":"2024-09-21T07:35:27.872027+0000","last_active":"2024-09-21T07:55:00.319249+0000","last_peered":"2024-09-21T07:55:00.319249+0000","last_clean":"2024-09-21T07:55:00.319249+0000","last_became_active":"2024-09-21T07:35:27.870426+0000","last_became_peered":"2024-09-21T07:35:27.870426+0000","last_unstale":"2024-09-21T07:55:00.319249+0000","last_undegraded":"2024-09-21T07:55:00.319249+0000","last_fullsized":"2024-09-21T07:55:00.319249+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":414,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.312990+0000","last_change":"2024-09-21T07:35:29.491008+0000","last_active":"2024-09-21T07:55:00.312990+0000","last_peered":"2024-09-21T07:55:00.312990+0000","last_clean":"2024-09-21T07:55:00.312990+0000","last_became_active":"2024-09-21T07:35:26.866750+0000","last_became_peered":"2024-09-21T07:35:26.866750+0000","last_unstale":"2024-09-21T07:55:00.312990+0000","last_undegraded":"2024-09-21T07:55:00.312990+0000","last_fullsized":"2024-09-21T07:55:00.312990+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:25.490406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083529700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":464,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.315758+0000","last_change":"2024-09-21T07:34:24.260665+0000","last_active":"2024-09-21T07:55:00.315758+0000","last_peered":"2024-09-21T07:55:00.315758+0000","last_clean":"2024-09-21T07:55:00.315758+0000","last_became_active":"2024-09-21T07:34:23.635909+0000","last_became_peered":"2024-09-21T07:34:23.635909+0000","last_unstale":"2024-09-21T07:55:00.315758+0000","last_undegraded":"2024-09-21T07:55:00.315758+0000","last_fullsized":"2024-09-21T07:55:00.315758+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:44.536885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":400,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.173356+0000","last_change":"2024-09-21T07:35:27.874772+0000","last_active":"2024-09-21T07:55:00.173356+0000","last_peered":"2024-09-21T07:55:00.173356+0000","last_clean":"2024-09-21T07:55:00.173356+0000","last_became_active":"2024-09-21T07:35:27.874434+0000","last_became_peered":"2024-09-21T07:35:27.874434+0000","last_unstale":"2024-09-21T07:55:00.173356+0000","last_undegraded":"2024-09-21T07:55:00.173356+0000","last_fullsized":"2024-09-21T07:55:00.173356+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:31.998755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":428,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.632463+0000","last_change":"2024-09-21T07:35:28.859153+0000","last_active":"2024-09-21T07:54:59.632463+0000","last_peered":"2024-09-21T07:54:59.632463+0000","last_clean":"2024-09-21T07:54:59.632463+0000","last_became_active":"2024-09-21T07:35:26.874428+0000","last_became_peered":"2024-09-21T07:35:26.874428+0000","last_unstale":"2024-09-21T07:54:59.632463+0000","last_undegraded":"2024-09-21T07:54:59.632463+0000","last_fullsized":"2024-09-21T07:54:59.632463+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:04.487621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048557299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":480,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.630653+0000","last_change":"2024-09-21T07:34:24.538845+0000","last_active":"2024-09-21T07:54:59.630653+0000","last_peered":"2024-09-21T07:54:59.630653+0000","last_clean":"2024-09-21T07:54:59.630653+0000","last_became_active":"2024-09-21T07:34:23.636372+0000","last_became_peered":"2024-09-21T07:34:23.636372+0000","last_unstale":"2024-09-21T07:54:59.630653+0000","last_undegraded":"2024-09-21T07:54:59.630653+0000","last_fullsized":"2024-09-21T07:54:59.630653+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:29.393813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841451,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":409,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.312263+0000","last_change":"2024-09-21T07:35:27.877801+0000","last_active":"2024-09-21T07:55:00.312263+0000","last_peered":"2024-09-21T07:55:00.312263+0000","last_clean":"2024-09-21T07:55:00.312263+0000","last_became_active":"2024-09-21T07:35:27.877464+0000","last_became_peered":"2024-09-21T07:35:27.877464+0000","last_unstale":"2024-09-21T07:55:00.312263+0000","last_undegraded":"2024-09-21T07:55:00.312263+0000","last_fullsized":"2024-09-21T07:55:00.312263+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:08.814929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":413,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.632257+0000","last_change":"2024-09-21T07:35:28.863732+0000","last_active":"2024-09-21T07:54:59.632257+0000","last_peered":"2024-09-21T07:54:59.632257+0000","last_clean":"2024-09-21T07:54:59.632257+0000","last_became_active":"2024-09-21T07:35:26.877392+0000","last_became_peered":"2024-09-21T07:35:26.877392+0000","last_unstale":"2024-09-21T07:54:59.632257+0000","last_undegraded":"2024-09-21T07:54:59.632257+0000","last_fullsized":"2024-09-21T07:54:59.632257+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071425700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":467,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.314344+0000","last_change":"2024-09-21T07:34:24.259686+0000","last_active":"2024-09-21T07:55:00.314344+0000","last_peered":"2024-09-21T07:55:00.314344+0000","last_clean":"2024-09-21T07:55:00.314344+0000","last_became_active":"2024-09-21T07:34:23.274405+0000","last_became_peered":"2024-09-21T07:34:23.274405+0000","last_unstale":"2024-09-21T07:55:00.314344+0000","last_undegraded":"2024-09-21T07:55:00.314344+0000","last_fullsized":"2024-09-21T07:55:00.314344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:26.304693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022009920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":400,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.175037+0000","last_change":"2024-09-21T07:35:27.867336+0000","last_active":"2024-09-21T07:55:00.175037+0000","last_peered":"2024-09-21T07:55:00.175037+0000","last_clean":"2024-09-21T07:55:00.175037+0000","last_became_active":"2024-09-21T07:35:27.867015+0000","last_became_peered":"2024-09-21T07:35:27.867015+0000","last_unstale":"2024-09-21T07:55:00.175037+0000","last_undegraded":"2024-09-21T07:55:00.175037+0000","last_fullsized":"2024-09-21T07:55:00.175037+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:59.214139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":411,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.315550+0000","last_change":"2024-09-21T07:35:29.493137+0000","last_active":"2024-09-21T07:55:00.315550+0000","last_peered":"2024-09-21T07:55:00.315550+0000","last_clean":"2024-09-21T07:55:00.315550+0000","last_became_active":"2024-09-21T07:35:26.872230+0000","last_became_peered":"2024-09-21T07:35:26.872230+0000","last_unstale":"2024-09-21T07:55:00.315550+0000","last_undegraded":"2024-09-21T07:55:00.315550+0000","last_fullsized":"2024-09-21T07:55:00.315550+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:24.437929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001197449,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":464,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.311160+0000","last_change":"2024-09-21T07:34:24.258754+0000","last_active":"2024-09-21T07:55:00.311160+0000","last_peered":"2024-09-21T07:55:00.311160+0000","last_clean":"2024-09-21T07:55:00.311160+0000","last_became_active":"2024-09-21T07:34:23.272657+0000","last_became_peered":"2024-09-21T07:34:23.272657+0000","last_unstale":"2024-09-21T07:55:00.311160+0000","last_undegraded":"2024-09-21T07:55:00.311160+0000","last_fullsized":"2024-09-21T07:55:00.311160+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:05.477099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067143200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":406,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.632348+0000","last_change":"2024-09-21T07:35:27.876021+0000","last_active":"2024-09-21T07:54:59.632348+0000","last_peered":"2024-09-21T07:54:59.632348+0000","last_clean":"2024-09-21T07:54:59.632348+0000","last_became_active":"2024-09-21T07:35:27.875724+0000","last_became_peered":"2024-09-21T07:35:27.875724+0000","last_unstale":"2024-09-21T07:54:59.632348+0000","last_undegraded":"2024-09-21T07:54:59.632348+0000","last_fullsized":"2024-09-21T07:54:59.632348+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:03.022812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":411,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.313993+0000","last_change":"2024-09-21T07:35:29.492570+0000","last_active":"2024-09-21T07:55:00.313993+0000","last_peered":"2024-09-21T07:55:00.313993+0000","last_clean":"2024-09-21T07:55:00.313993+0000","last_became_active":"2024-09-21T07:35:26.874316+0000","last_became_peered":"2024-09-21T07:35:26.874316+0000","last_unstale":"2024-09-21T07:55:00.313993+0000","last_undegraded":"2024-09-21T07:55:00.313993+0000","last_fullsized":"2024-09-21T07:55:00.313993+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:16.512323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011990760000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":464,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.633378+0000","last_change":"2024-09-21T07:34:25.085232+0000","last_active":"2024-09-21T07:54:59.633378+0000","last_peered":"2024-09-21T07:54:59.633378+0000","last_clean":"2024-09-21T07:54:59.633378+0000","last_became_active":"2024-09-21T07:34:23.277807+0000","last_became_peered":"2024-09-21T07:34:23.277807+0000","last_unstale":"2024-09-21T07:54:59.633378+0000","last_undegraded":"2024-09-21T07:54:59.633378+0000","last_fullsized":"2024-09-21T07:54:59.633378+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:36.362919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086619099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":409,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.316773+0000","last_change":"2024-09-21T07:35:27.872798+0000","last_active":"2024-09-21T07:55:00.316773+0000","last_peered":"2024-09-21T07:55:00.316773+0000","last_clean":"2024-09-21T07:55:00.316773+0000","last_became_active":"2024-09-21T07:35:27.872483+0000","last_became_peered":"2024-09-21T07:35:27.872483+0000","last_unstale":"2024-09-21T07:55:00.316773+0000","last_undegraded":"2024-09-21T07:55:00.316773+0000","last_fullsized":"2024-09-21T07:55:00.316773+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:11.933469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":416,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.315971+0000","last_change":"2024-09-21T07:35:29.206723+0000","last_active":"2024-09-21T07:55:00.315971+0000","last_peered":"2024-09-21T07:55:00.315971+0000","last_clean":"2024-09-21T07:55:00.315971+0000","last_became_active":"2024-09-21T07:35:26.872377+0000","last_became_peered":"2024-09-21T07:35:26.872377+0000","last_unstale":"2024-09-21T07:55:00.315971+0000","last_undegraded":"2024-09-21T07:55:00.315971+0000","last_fullsized":"2024-09-21T07:55:00.315971+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:29.086862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083077200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":464,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:11.320650+0000","last_change":"2024-09-21T07:55:11.320650+0000","last_active":"2024-09-21T07:55:11.320650+0000","last_peered":"2024-09-21T07:55:11.320650+0000","last_clean":"2024-09-21T07:55:11.320650+0000","last_became_active":"2024-09-21T07:34:23.267770+0000","last_became_peered":"2024-09-21T07:34:23.267770+0000","last_unstale":"2024-09-21T07:55:11.320650+0000","last_undegraded":"2024-09-21T07:55:11.320650+0000","last_fullsized":"2024-09-21T07:55:11.320650+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:11.320555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:11.320555+0000","last_clean_scrub_stamp":"2024-09-21T07:55:11.320555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:38.127843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086378500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":409,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.316038+0000","last_change":"2024-09-21T07:35:27.870121+0000","last_active":"2024-09-21T07:55:00.316038+0000","last_peered":"2024-09-21T07:55:00.316038+0000","last_clean":"2024-09-21T07:55:00.316038+0000","last_became_active":"2024-09-21T07:35:27.869811+0000","last_became_peered":"2024-09-21T07:35:27.869811+0000","last_unstale":"2024-09-21T07:55:00.316038+0000","last_undegraded":"2024-09-21T07:55:00.316038+0000","last_fullsized":"2024-09-21T07:55:00.316038+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:31.778461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":414,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.634294+0000","last_change":"2024-09-21T07:35:28.860699+0000","last_active":"2024-09-21T07:54:59.634294+0000","last_peered":"2024-09-21T07:54:59.634294+0000","last_clean":"2024-09-21T07:54:59.634294+0000","last_became_active":"2024-09-21T07:35:26.876779+0000","last_became_peered":"2024-09-21T07:35:26.876779+0000","last_unstale":"2024-09-21T07:54:59.634294+0000","last_undegraded":"2024-09-21T07:54:59.634294+0000","last_fullsized":"2024-09-21T07:54:59.634294+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:28.286403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000913188,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":466,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.634607+0000","last_change":"2024-09-21T07:34:24.260181+0000","last_active":"2024-09-21T07:54:59.634607+0000","last_peered":"2024-09-21T07:54:59.634607+0000","last_clean":"2024-09-21T07:54:59.634607+0000","last_became_active":"2024-09-21T07:34:23.275580+0000","last_became_peered":"2024-09-21T07:34:23.275580+0000","last_unstale":"2024-09-21T07:54:59.634607+0000","last_undegraded":"2024-09-21T07:54:59.634607+0000","last_fullsized":"2024-09-21T07:54:59.634607+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:49.857622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014706980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":407,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.314738+0000","last_change":"2024-09-21T07:35:27.875393+0000","last_active":"2024-09-21T07:55:00.314738+0000","last_peered":"2024-09-21T07:55:00.314738+0000","last_clean":"2024-09-21T07:55:00.314738+0000","last_became_active":"2024-09-21T07:35:27.874671+0000","last_became_peered":"2024-09-21T07:35:27.874671+0000","last_unstale":"2024-09-21T07:55:00.314738+0000","last_undegraded":"2024-09-21T07:55:00.314738+0000","last_fullsized":"2024-09-21T07:55:00.314738+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:56.459560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":416,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.312471+0000","last_change":"2024-09-21T07:35:29.206296+0000","last_active":"2024-09-21T07:55:00.312471+0000","last_peered":"2024-09-21T07:55:00.312471+0000","last_clean":"2024-09-21T07:55:00.312471+0000","last_became_active":"2024-09-21T07:35:26.874575+0000","last_became_peered":"2024-09-21T07:35:26.874575+0000","last_unstale":"2024-09-21T07:55:00.312471+0000","last_undegraded":"2024-09-21T07:55:00.312471+0000","last_fullsized":"2024-09-21T07:55:00.312471+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:22.771154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061011599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":467,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.315376+0000","last_change":"2024-09-21T07:34:24.259112+0000","last_active":"2024-09-21T07:55:00.315376+0000","last_peered":"2024-09-21T07:55:00.315376+0000","last_clean":"2024-09-21T07:55:00.315376+0000","last_became_active":"2024-09-21T07:34:23.635444+0000","last_became_peered":"2024-09-21T07:34:23.635444+0000","last_unstale":"2024-09-21T07:55:00.315376+0000","last_undegraded":"2024-09-21T07:55:00.315376+0000","last_fullsized":"2024-09-21T07:55:00.315376+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:46.255385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001952636,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":407,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.312824+0000","last_change":"2024-09-21T07:35:27.878356+0000","last_active":"2024-09-21T07:55:00.312824+0000","last_peered":"2024-09-21T07:55:00.312824+0000","last_clean":"2024-09-21T07:55:00.312824+0000","last_became_active":"2024-09-21T07:35:27.877982+0000","last_became_peered":"2024-09-21T07:35:27.877982+0000","last_unstale":"2024-09-21T07:55:00.312824+0000","last_undegraded":"2024-09-21T07:55:00.312824+0000","last_fullsized":"2024-09-21T07:55:00.312824+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":414,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.310325+0000","last_change":"2024-09-21T07:35:29.053131+0000","last_active":"2024-09-21T07:55:00.310325+0000","last_peered":"2024-09-21T07:55:00.310325+0000","last_clean":"2024-09-21T07:55:00.310325+0000","last_became_active":"2024-09-21T07:35:26.870516+0000","last_became_peered":"2024-09-21T07:35:26.870516+0000","last_unstale":"2024-09-21T07:55:00.310325+0000","last_undegraded":"2024-09-21T07:55:00.310325+0000","last_fullsized":"2024-09-21T07:55:00.310325+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:10.924491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070705699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":480,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.632868+0000","last_change":"2024-09-21T07:34:24.539152+0000","last_active":"2024-09-21T07:54:59.632868+0000","last_peered":"2024-09-21T07:54:59.632868+0000","last_clean":"2024-09-21T07:54:59.632868+0000","last_became_active":"2024-09-21T07:34:23.259573+0000","last_became_peered":"2024-09-21T07:34:23.259573+0000","last_unstale":"2024-09-21T07:54:59.632868+0000","last_undegraded":"2024-09-21T07:54:59.632868+0000","last_fullsized":"2024-09-21T07:54:59.632868+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:05:26.986900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058108400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":406,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.310193+0000","last_change":"2024-09-21T07:35:27.873726+0000","last_active":"2024-09-21T07:55:00.310193+0000","last_peered":"2024-09-21T07:55:00.310193+0000","last_clean":"2024-09-21T07:55:00.310193+0000","last_became_active":"2024-09-21T07:35:27.871421+0000","last_became_peered":"2024-09-21T07:35:27.871421+0000","last_unstale":"2024-09-21T07:55:00.310193+0000","last_undegraded":"2024-09-21T07:55:00.310193+0000","last_fullsized":"2024-09-21T07:55:00.310193+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:00.168915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":414,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.634353+0000","last_change":"2024-09-21T07:35:28.861032+0000","last_active":"2024-09-21T07:54:59.634353+0000","last_peered":"2024-09-21T07:54:59.634353+0000","last_clean":"2024-09-21T07:54:59.634353+0000","last_became_active":"2024-09-21T07:35:26.873149+0000","last_became_peered":"2024-09-21T07:35:26.873149+0000","last_unstale":"2024-09-21T07:54:59.634353+0000","last_undegraded":"2024-09-21T07:54:59.634353+0000","last_fullsized":"2024-09-21T07:54:59.634353+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:59.435981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011988369999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":463,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.311971+0000","last_change":"2024-09-21T07:34:24.259902+0000","last_active":"2024-09-21T07:55:00.311971+0000","last_peered":"2024-09-21T07:55:00.311971+0000","last_clean":"2024-09-21T07:55:00.311971+0000","last_became_active":"2024-09-21T07:34:23.271902+0000","last_became_peered":"2024-09-21T07:34:23.271902+0000","last_unstale":"2024-09-21T07:55:00.311971+0000","last_undegraded":"2024-09-21T07:55:00.311971+0000","last_fullsized":"2024-09-21T07:55:00.311971+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:59.439525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001686358,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":405,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.631813+0000","last_change":"2024-09-21T07:35:27.872637+0000","last_active":"2024-09-21T07:54:59.631813+0000","last_peered":"2024-09-21T07:54:59.631813+0000","last_clean":"2024-09-21T07:54:59.631813+0000","last_became_active":"2024-09-21T07:35:27.872149+0000","last_became_peered":"2024-09-21T07:35:27.872149+0000","last_unstale":"2024-09-21T07:54:59.631813+0000","last_undegraded":"2024-09-21T07:54:59.631813+0000","last_fullsized":"2024-09-21T07:54:59.631813+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:27.697872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":411,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.314113+0000","last_change":"2024-09-21T07:35:29.493362+0000","last_active":"2024-09-21T07:55:00.314113+0000","last_peered":"2024-09-21T07:55:00.314113+0000","last_clean":"2024-09-21T07:55:00.314113+0000","last_became_active":"2024-09-21T07:35:26.873018+0000","last_became_peered":"2024-09-21T07:35:26.873018+0000","last_unstale":"2024-09-21T07:55:00.314113+0000","last_undegraded":"2024-09-21T07:55:00.314113+0000","last_fullsized":"2024-09-21T07:55:00.314113+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:02.460557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012496510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":474,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:10.043061+0000","last_change":"2024-09-21T07:55:10.043061+0000","last_active":"2024-09-21T07:55:10.043061+0000","last_peered":"2024-09-21T07:55:10.043061+0000","last_clean":"2024-09-21T07:55:10.043061+0000","last_became_active":"2024-09-21T07:34:23.278812+0000","last_became_peered":"2024-09-21T07:34:23.278812+0000","last_unstale":"2024-09-21T07:55:10.043061+0000","last_undegraded":"2024-09-21T07:55:10.043061+0000","last_fullsized":"2024-09-21T07:55:10.043061+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:10.042970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:10.042970+0000","last_clean_scrub_stamp":"2024-09-21T07:55:10.042970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:56:01.384383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356463,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":398,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.174789+0000","last_change":"2024-09-21T07:35:27.868448+0000","last_active":"2024-09-21T07:55:00.174789+0000","last_peered":"2024-09-21T07:55:00.174789+0000","last_clean":"2024-09-21T07:55:00.174789+0000","last_became_active":"2024-09-21T07:35:27.868105+0000","last_became_peered":"2024-09-21T07:35:27.868105+0000","last_unstale":"2024-09-21T07:55:00.174789+0000","last_undegraded":"2024-09-21T07:55:00.174789+0000","last_fullsized":"2024-09-21T07:55:00.174789+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:41.468702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":409,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.315283+0000","last_change":"2024-09-21T07:35:29.492974+0000","last_active":"2024-09-21T07:55:00.315283+0000","last_peered":"2024-09-21T07:55:00.315283+0000","last_clean":"2024-09-21T07:55:00.315283+0000","last_became_active":"2024-09-21T07:35:26.872165+0000","last_became_peered":"2024-09-21T07:35:26.872165+0000","last_unstale":"2024-09-21T07:55:00.315283+0000","last_undegraded":"2024-09-21T07:55:00.315283+0000","last_fullsized":"2024-09-21T07:55:00.315283+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:59.912520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013795179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":463,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.630639+0000","last_change":"2024-09-21T07:34:25.084861+0000","last_active":"2024-09-21T07:54:59.630639+0000","last_peered":"2024-09-21T07:54:59.630639+0000","last_clean":"2024-09-21T07:54:59.630639+0000","last_became_active":"2024-09-21T07:34:23.277708+0000","last_became_peered":"2024-09-21T07:34:23.277708+0000","last_unstale":"2024-09-21T07:54:59.630639+0000","last_undegraded":"2024-09-21T07:54:59.630639+0000","last_fullsized":"2024-09-21T07:54:59.630639+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:40.816488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000597566,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":398,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.175249+0000","last_change":"2024-09-21T07:35:27.874640+0000","last_active":"2024-09-21T07:55:00.175249+0000","last_peered":"2024-09-21T07:55:00.175249+0000","last_clean":"2024-09-21T07:55:00.175249+0000","last_became_active":"2024-09-21T07:35:27.874265+0000","last_became_peered":"2024-09-21T07:35:27.874265+0000","last_unstale":"2024-09-21T07:55:00.175249+0000","last_undegraded":"2024-09-21T07:55:00.175249+0000","last_fullsized":"2024-09-21T07:55:00.175249+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:45.754654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":414,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.318433+0000","last_change":"2024-09-21T07:35:29.207392+0000","last_active":"2024-09-21T07:55:00.318433+0000","last_peered":"2024-09-21T07:55:00.318433+0000","last_clean":"2024-09-21T07:55:00.318433+0000","last_became_active":"2024-09-21T07:35:26.870917+0000","last_became_peered":"2024-09-21T07:35:26.870917+0000","last_unstale":"2024-09-21T07:55:00.318433+0000","last_undegraded":"2024-09-21T07:55:00.318433+0000","last_fullsized":"2024-09-21T07:55:00.318433+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:59.159937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059198499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":467,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.313609+0000","last_change":"2024-09-21T07:34:24.258709+0000","last_active":"2024-09-21T07:55:00.313609+0000","last_peered":"2024-09-21T07:55:00.313609+0000","last_clean":"2024-09-21T07:55:00.313609+0000","last_became_active":"2024-09-21T07:34:23.279042+0000","last_became_peered":"2024-09-21T07:34:23.279042+0000","last_unstale":"2024-09-21T07:55:00.313609+0000","last_undegraded":"2024-09-21T07:55:00.313609+0000","last_fullsized":"2024-09-21T07:55:00.313609+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:50:01.762117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019282100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":404,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.633291+0000","last_change":"2024-09-21T07:35:27.870260+0000","last_active":"2024-09-21T07:54:59.633291+0000","last_peered":"2024-09-21T07:54:59.633291+0000","last_clean":"2024-09-21T07:54:59.633291+0000","last_became_active":"2024-09-21T07:35:27.869448+0000","last_became_peered":"2024-09-21T07:35:27.869448+0000","last_unstale":"2024-09-21T07:54:59.633291+0000","last_undegraded":"2024-09-21T07:54:59.633291+0000","last_fullsized":"2024-09-21T07:54:59.633291+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:16.946790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":405,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.175086+0000","last_change":"2024-09-21T07:35:29.386367+0000","last_active":"2024-09-21T07:55:00.175086+0000","last_peered":"2024-09-21T07:55:00.175086+0000","last_clean":"2024-09-21T07:55:00.175086+0000","last_became_active":"2024-09-21T07:35:26.879238+0000","last_became_peered":"2024-09-21T07:35:26.879238+0000","last_unstale":"2024-09-21T07:55:00.175086+0000","last_undegraded":"2024-09-21T07:55:00.175086+0000","last_fullsized":"2024-09-21T07:55:00.175086+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:16.528260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092223100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":463,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.632428+0000","last_change":"2024-09-21T07:34:25.085635+0000","last_active":"2024-09-21T07:54:59.632428+0000","last_peered":"2024-09-21T07:54:59.632428+0000","last_clean":"2024-09-21T07:54:59.632428+0000","last_became_active":"2024-09-21T07:34:23.273566+0000","last_became_peered":"2024-09-21T07:34:23.273566+0000","last_unstale":"2024-09-21T07:54:59.632428+0000","last_undegraded":"2024-09-21T07:54:59.632428+0000","last_fullsized":"2024-09-21T07:54:59.632428+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:06:14.142272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079897300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":404,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.633141+0000","last_change":"2024-09-21T07:35:27.878728+0000","last_active":"2024-09-21T07:54:59.633141+0000","last_peered":"2024-09-21T07:54:59.633141+0000","last_clean":"2024-09-21T07:54:59.633141+0000","last_became_active":"2024-09-21T07:35:27.878335+0000","last_became_peered":"2024-09-21T07:35:27.878335+0000","last_unstale":"2024-09-21T07:54:59.633141+0000","last_undegraded":"2024-09-21T07:54:59.633141+0000","last_fullsized":"2024-09-21T07:54:59.633141+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:25:37.731771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":426,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.630524+0000","last_change":"2024-09-21T07:35:28.858418+0000","last_active":"2024-09-21T07:54:59.630524+0000","last_peered":"2024-09-21T07:54:59.630524+0000","last_clean":"2024-09-21T07:54:59.630524+0000","last_became_active":"2024-09-21T07:35:26.876944+0000","last_became_peered":"2024-09-21T07:35:26.876944+0000","last_unstale":"2024-09-21T07:54:59.630524+0000","last_undegraded":"2024-09-21T07:54:59.630524+0000","last_fullsized":"2024-09-21T07:54:59.630524+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:04.514304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090862799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":455,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.176081+0000","last_change":"2024-09-21T07:34:24.597926+0000","last_active":"2024-09-21T07:55:00.176081+0000","last_peered":"2024-09-21T07:55:00.176081+0000","last_clean":"2024-09-21T07:55:00.176081+0000","last_became_active":"2024-09-21T07:34:23.268926+0000","last_became_peered":"2024-09-21T07:34:23.268926+0000","last_unstale":"2024-09-21T07:55:00.176081+0000","last_undegraded":"2024-09-21T07:55:00.176081+0000","last_fullsized":"2024-09-21T07:55:00.176081+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:46.190752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101851,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":400,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.175363+0000","last_change":"2024-09-21T07:35:27.877426+0000","last_active":"2024-09-21T07:55:00.175363+0000","last_peered":"2024-09-21T07:55:00.175363+0000","last_clean":"2024-09-21T07:55:00.175363+0000","last_became_active":"2024-09-21T07:35:27.877050+0000","last_became_peered":"2024-09-21T07:35:27.877050+0000","last_unstale":"2024-09-21T07:55:00.175363+0000","last_undegraded":"2024-09-21T07:55:00.175363+0000","last_fullsized":"2024-09-21T07:55:00.175363+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:36.268334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":412,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.631943+0000","last_change":"2024-09-21T07:35:28.862273+0000","last_active":"2024-09-21T07:54:59.631943+0000","last_peered":"2024-09-21T07:54:59.631943+0000","last_clean":"2024-09-21T07:54:59.631943+0000","last_became_active":"2024-09-21T07:35:26.874011+0000","last_became_peered":"2024-09-21T07:35:26.874011+0000","last_unstale":"2024-09-21T07:54:59.631943+0000","last_undegraded":"2024-09-21T07:54:59.631943+0000","last_fullsized":"2024-09-21T07:54:59.631943+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:11.986950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071644799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":465,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.637034+0000","last_change":"2024-09-21T07:34:24.259794+0000","last_active":"2024-09-21T07:54:59.637034+0000","last_peered":"2024-09-21T07:54:59.637034+0000","last_clean":"2024-09-21T07:54:59.637034+0000","last_became_active":"2024-09-21T07:34:23.265679+0000","last_became_peered":"2024-09-21T07:34:23.265679+0000","last_unstale":"2024-09-21T07:54:59.637034+0000","last_undegraded":"2024-09-21T07:54:59.637034+0000","last_fullsized":"2024-09-21T07:54:59.637034+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:02.575796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001298518,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":409,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.317882+0000","last_change":"2024-09-21T07:35:27.877725+0000","last_active":"2024-09-21T07:55:00.317882+0000","last_peered":"2024-09-21T07:55:00.317882+0000","last_clean":"2024-09-21T07:55:00.317882+0000","last_became_active":"2024-09-21T07:35:27.877398+0000","last_became_peered":"2024-09-21T07:35:27.877398+0000","last_unstale":"2024-09-21T07:55:00.317882+0000","last_undegraded":"2024-09-21T07:55:00.317882+0000","last_fullsized":"2024-09-21T07:55:00.317882+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:05.859762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":413,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.633091+0000","last_change":"2024-09-21T07:35:28.860612+0000","last_active":"2024-09-21T07:54:59.633091+0000","last_peered":"2024-09-21T07:54:59.633091+0000","last_clean":"2024-09-21T07:54:59.633091+0000","last_became_active":"2024-09-21T07:35:26.876562+0000","last_became_peered":"2024-09-21T07:35:26.876562+0000","last_unstale":"2024-09-21T07:54:59.633091+0000","last_undegraded":"2024-09-21T07:54:59.633091+0000","last_fullsized":"2024-09-21T07:54:59.633091+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:06.824331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012252470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":463,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.315655+0000","last_change":"2024-09-21T07:34:24.260330+0000","last_active":"2024-09-21T07:55:00.315655+0000","last_peered":"2024-09-21T07:55:00.315655+0000","last_clean":"2024-09-21T07:55:00.315655+0000","last_became_active":"2024-09-21T07:34:23.637210+0000","last_became_peered":"2024-09-21T07:34:23.637210+0000","last_unstale":"2024-09-21T07:55:00.315655+0000","last_undegraded":"2024-09-21T07:55:00.315655+0000","last_fullsized":"2024-09-21T07:55:00.315655+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:02.488451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001537093,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":405,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.630514+0000","last_change":"2024-09-21T07:35:27.874757+0000","last_active":"2024-09-21T07:54:59.630514+0000","last_peered":"2024-09-21T07:54:59.630514+0000","last_clean":"2024-09-21T07:54:59.630514+0000","last_became_active":"2024-09-21T07:35:27.874360+0000","last_became_peered":"2024-09-21T07:35:27.874360+0000","last_unstale":"2024-09-21T07:54:59.630514+0000","last_undegraded":"2024-09-21T07:54:59.630514+0000","last_fullsized":"2024-09-21T07:54:59.630514+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:17.646912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":410,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.314189+0000","last_change":"2024-09-21T07:35:29.493431+0000","last_active":"2024-09-21T07:55:00.314189+0000","last_peered":"2024-09-21T07:55:00.314189+0000","last_clean":"2024-09-21T07:55:00.314189+0000","last_became_active":"2024-09-21T07:35:26.873691+0000","last_became_peered":"2024-09-21T07:35:26.873691+0000","last_unstale":"2024-09-21T07:55:00.314189+0000","last_undegraded":"2024-09-21T07:55:00.314189+0000","last_fullsized":"2024-09-21T07:55:00.314189+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:51.705051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001162464,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":455,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.175180+0000","last_change":"2024-09-21T07:34:24.597286+0000","last_active":"2024-09-21T07:55:00.175180+0000","last_peered":"2024-09-21T07:55:00.175180+0000","last_clean":"2024-09-21T07:55:00.175180+0000","last_became_active":"2024-09-21T07:34:23.267133+0000","last_became_peered":"2024-09-21T07:34:23.267133+0000","last_unstale":"2024-09-21T07:55:00.175180+0000","last_undegraded":"2024-09-21T07:55:00.175180+0000","last_fullsized":"2024-09-21T07:55:00.175180+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:18.257010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":420,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.630258+0000","last_change":"2024-09-21T07:35:27.864458+0000","last_active":"2024-09-21T07:54:59.630258+0000","last_peered":"2024-09-21T07:54:59.630258+0000","last_clean":"2024-09-21T07:54:59.630258+0000","last_became_active":"2024-09-21T07:35:27.864136+0000","last_became_peered":"2024-09-21T07:35:27.864136+0000","last_unstale":"2024-09-21T07:54:59.630258+0000","last_undegraded":"2024-09-21T07:54:59.630258+0000","last_fullsized":"2024-09-21T07:54:59.630258+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:20.162240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":427,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.632799+0000","last_change":"2024-09-21T07:35:28.858784+0000","last_active":"2024-09-21T07:54:59.632799+0000","last_peered":"2024-09-21T07:54:59.632799+0000","last_clean":"2024-09-21T07:54:59.632799+0000","last_became_active":"2024-09-21T07:35:26.876890+0000","last_became_peered":"2024-09-21T07:35:26.876890+0000","last_unstale":"2024-09-21T07:54:59.632799+0000","last_undegraded":"2024-09-21T07:54:59.632799+0000","last_fullsized":"2024-09-21T07:54:59.632799+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:40.738327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064543499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":466,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.313937+0000","last_change":"2024-09-21T07:34:24.258712+0000","last_active":"2024-09-21T07:55:00.313937+0000","last_peered":"2024-09-21T07:55:00.313937+0000","last_clean":"2024-09-21T07:55:00.313937+0000","last_became_active":"2024-09-21T07:34:23.274692+0000","last_became_peered":"2024-09-21T07:34:23.274692+0000","last_unstale":"2024-09-21T07:55:00.313937+0000","last_undegraded":"2024-09-21T07:55:00.313937+0000","last_fullsized":"2024-09-21T07:55:00.313937+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:43.279454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018051479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":406,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.312289+0000","last_change":"2024-09-21T07:35:27.878020+0000","last_active":"2024-09-21T07:55:00.312289+0000","last_peered":"2024-09-21T07:55:00.312289+0000","last_clean":"2024-09-21T07:55:00.312289+0000","last_became_active":"2024-09-21T07:35:27.877532+0000","last_became_peered":"2024-09-21T07:35:27.877532+0000","last_unstale":"2024-09-21T07:55:00.312289+0000","last_undegraded":"2024-09-21T07:55:00.312289+0000","last_fullsized":"2024-09-21T07:55:00.312289+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:19.756073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":413,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.633494+0000","last_change":"2024-09-21T07:35:28.861049+0000","last_active":"2024-09-21T07:54:59.633494+0000","last_peered":"2024-09-21T07:54:59.633494+0000","last_clean":"2024-09-21T07:54:59.633494+0000","last_became_active":"2024-09-21T07:35:26.876819+0000","last_became_peered":"2024-09-21T07:35:26.876819+0000","last_unstale":"2024-09-21T07:54:59.633494+0000","last_undegraded":"2024-09-21T07:54:59.633494+0000","last_fullsized":"2024-09-21T07:54:59.633494+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:19.633803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055780599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":463,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.630568+0000","last_change":"2024-09-21T07:34:25.084849+0000","last_active":"2024-09-21T07:54:59.630568+0000","last_peered":"2024-09-21T07:54:59.630568+0000","last_clean":"2024-09-21T07:54:59.630568+0000","last_became_active":"2024-09-21T07:34:23.277713+0000","last_became_peered":"2024-09-21T07:34:23.277713+0000","last_unstale":"2024-09-21T07:54:59.630568+0000","last_undegraded":"2024-09-21T07:54:59.630568+0000","last_fullsized":"2024-09-21T07:54:59.630568+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:05.811810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059079499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":406,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.312851+0000","last_change":"2024-09-21T07:35:27.884248+0000","last_active":"2024-09-21T07:55:00.312851+0000","last_peered":"2024-09-21T07:55:00.312851+0000","last_clean":"2024-09-21T07:55:00.312851+0000","last_became_active":"2024-09-21T07:35:27.883861+0000","last_became_peered":"2024-09-21T07:35:27.883861+0000","last_unstale":"2024-09-21T07:55:00.312851+0000","last_undegraded":"2024-09-21T07:55:00.312851+0000","last_fullsized":"2024-09-21T07:55:00.312851+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":415,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.318311+0000","last_change":"2024-09-21T07:35:29.207135+0000","last_active":"2024-09-21T07:55:00.318311+0000","last_peered":"2024-09-21T07:55:00.318311+0000","last_clean":"2024-09-21T07:55:00.318311+0000","last_became_active":"2024-09-21T07:35:26.872787+0000","last_became_peered":"2024-09-21T07:35:26.872787+0000","last_unstale":"2024-09-21T07:55:00.318311+0000","last_undegraded":"2024-09-21T07:55:00.318311+0000","last_fullsized":"2024-09-21T07:55:00.318311+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:27.543217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011157509999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":479,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.630798+0000","last_change":"2024-09-21T07:34:24.538610+0000","last_active":"2024-09-21T07:54:59.630798+0000","last_peered":"2024-09-21T07:54:59.630798+0000","last_clean":"2024-09-21T07:54:59.630798+0000","last_became_active":"2024-09-21T07:34:23.259700+0000","last_became_peered":"2024-09-21T07:34:23.259700+0000","last_unstale":"2024-09-21T07:54:59.630798+0000","last_undegraded":"2024-09-21T07:54:59.630798+0000","last_fullsized":"2024-09-21T07:54:59.630798+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:57.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070522399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":420,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.630936+0000","last_change":"2024-09-21T07:35:27.868024+0000","last_active":"2024-09-21T07:54:59.630936+0000","last_peered":"2024-09-21T07:54:59.630936+0000","last_clean":"2024-09-21T07:54:59.630936+0000","last_became_active":"2024-09-21T07:35:27.867339+0000","last_became_peered":"2024-09-21T07:35:27.867339+0000","last_unstale":"2024-09-21T07:54:59.630936+0000","last_undegraded":"2024-09-21T07:54:59.630936+0000","last_fullsized":"2024-09-21T07:54:59.630936+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:09.181656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":427,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.631794+0000","last_change":"2024-09-21T07:35:28.858505+0000","last_active":"2024-09-21T07:54:59.631794+0000","last_peered":"2024-09-21T07:54:59.631794+0000","last_clean":"2024-09-21T07:54:59.631794+0000","last_became_active":"2024-09-21T07:35:26.877320+0000","last_became_peered":"2024-09-21T07:35:26.877320+0000","last_unstale":"2024-09-21T07:54:59.631794+0000","last_undegraded":"2024-09-21T07:54:59.631794+0000","last_fullsized":"2024-09-21T07:54:59.631794+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:53.672782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244119999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":457,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.175837+0000","last_change":"2024-09-21T07:34:24.597597+0000","last_active":"2024-09-21T07:55:00.175837+0000","last_peered":"2024-09-21T07:55:00.175837+0000","last_clean":"2024-09-21T07:55:00.175837+0000","last_became_active":"2024-09-21T07:34:23.635204+0000","last_became_peered":"2024-09-21T07:34:23.635204+0000","last_unstale":"2024-09-21T07:55:00.175837+0000","last_undegraded":"2024-09-21T07:55:00.175837+0000","last_fullsized":"2024-09-21T07:55:00.175837+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:48.644789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081139199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":408,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.316004+0000","last_change":"2024-09-21T07:35:27.877503+0000","last_active":"2024-09-21T07:55:00.316004+0000","last_peered":"2024-09-21T07:55:00.316004+0000","last_clean":"2024-09-21T07:55:00.316004+0000","last_became_active":"2024-09-21T07:35:27.875775+0000","last_became_peered":"2024-09-21T07:35:27.875775+0000","last_unstale":"2024-09-21T07:55:00.316004+0000","last_undegraded":"2024-09-21T07:55:00.316004+0000","last_fullsized":"2024-09-21T07:55:00.316004+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:49.291023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":410,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.311850+0000","last_change":"2024-09-21T07:35:29.491412+0000","last_active":"2024-09-21T07:55:00.311850+0000","last_peered":"2024-09-21T07:55:00.311850+0000","last_clean":"2024-09-21T07:55:00.311850+0000","last_became_active":"2024-09-21T07:35:26.876404+0000","last_became_peered":"2024-09-21T07:35:26.876404+0000","last_unstale":"2024-09-21T07:55:00.311850+0000","last_undegraded":"2024-09-21T07:55:00.311850+0000","last_fullsized":"2024-09-21T07:55:00.311850+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:07.252410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063918000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":467,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.314021+0000","last_change":"2024-09-21T07:34:24.259026+0000","last_active":"2024-09-21T07:55:00.314021+0000","last_peered":"2024-09-21T07:55:00.314021+0000","last_clean":"2024-09-21T07:55:00.314021+0000","last_became_active":"2024-09-21T07:34:23.635456+0000","last_became_peered":"2024-09-21T07:34:23.635456+0000","last_unstale":"2024-09-21T07:55:00.314021+0000","last_undegraded":"2024-09-21T07:55:00.314021+0000","last_fullsized":"2024-09-21T07:55:00.314021+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:09.592752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018654419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":408,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.316264+0000","last_change":"2024-09-21T07:35:27.871984+0000","last_active":"2024-09-21T07:55:00.316264+0000","last_peered":"2024-09-21T07:55:00.316264+0000","last_clean":"2024-09-21T07:55:00.316264+0000","last_became_active":"2024-09-21T07:35:27.870336+0000","last_became_peered":"2024-09-21T07:35:27.870336+0000","last_unstale":"2024-09-21T07:55:00.316264+0000","last_undegraded":"2024-09-21T07:55:00.316264+0000","last_fullsized":"2024-09-21T07:55:00.316264+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:57.120479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":413,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.631715+0000","last_change":"2024-09-21T07:35:28.860351+0000","last_active":"2024-09-21T07:54:59.631715+0000","last_peered":"2024-09-21T07:54:59.631715+0000","last_clean":"2024-09-21T07:54:59.631715+0000","last_became_active":"2024-09-21T07:35:26.872046+0000","last_became_peered":"2024-09-21T07:35:26.872046+0000","last_unstale":"2024-09-21T07:54:59.631715+0000","last_undegraded":"2024-09-21T07:54:59.631715+0000","last_fullsized":"2024-09-21T07:54:59.631715+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:47.074327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098384399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":463,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.315458+0000","last_change":"2024-09-21T07:34:24.260300+0000","last_active":"2024-09-21T07:55:00.315458+0000","last_peered":"2024-09-21T07:55:00.315458+0000","last_clean":"2024-09-21T07:55:00.315458+0000","last_became_active":"2024-09-21T07:34:23.270128+0000","last_became_peered":"2024-09-21T07:34:23.270128+0000","last_unstale":"2024-09-21T07:55:00.315458+0000","last_undegraded":"2024-09-21T07:55:00.315458+0000","last_fullsized":"2024-09-21T07:55:00.315458+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:25.260931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00176651,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":408,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.318010+0000","last_change":"2024-09-21T07:35:27.877861+0000","last_active":"2024-09-21T07:55:00.318010+0000","last_peered":"2024-09-21T07:55:00.318010+0000","last_clean":"2024-09-21T07:55:00.318010+0000","last_became_active":"2024-09-21T07:35:27.877434+0000","last_became_peered":"2024-09-21T07:35:27.877434+0000","last_unstale":"2024-09-21T07:55:00.318010+0000","last_undegraded":"2024-09-21T07:55:00.318010+0000","last_fullsized":"2024-09-21T07:55:00.318010+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:26.844971+0000","last_clean_scrub_stamp":"2024-09-21T07:35:26.844971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:22.145565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":412,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:54:59.630743+0000","last_change":"2024-09-21T07:35:28.862954+0000","last_active":"2024-09-21T07:54:59.630743+0000","last_peered":"2024-09-21T07:54:59.630743+0000","last_clean":"2024-09-21T07:54:59.630743+0000","last_became_active":"2024-09-21T07:35:26.870613+0000","last_became_peered":"2024-09-21T07:35:26.870613+0000","last_unstale":"2024-09-21T07:54:59.630743+0000","last_undegraded":"2024-09-21T07:54:59.630743+0000","last_fullsized":"2024-09-21T07:54:59.630743+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:35:25.835993+0000","last_clean_scrub_stamp":"2024-09-21T07:35:25.835993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:06.901043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059842100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":464,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:00.314518+0000","last_change":"2024-09-21T07:34:25.028904+0000","last_active":"2024-09-21T07:55:00.314518+0000","last_peered":"2024-09-21T07:55:00.314518+0000","last_clean":"2024-09-21T07:55:00.314518+0000","last_became_active":"2024-09-21T07:34:23.260942+0000","last_became_peered":"2024-09-21T07:34:23.260942+0000","last_unstale":"2024-09-21T07:55:00.314518+0000","last_undegraded":"2024-09-21T07:55:00.314518+0000","last_fullsized":"2024-09-21T07:55:00.314518+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:34:22.238474+0000","last_clean_scrub_stamp":"2024-09-21T07:34:22.238474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:09:22.709328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060553899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509708,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71188,"kb_used_data":3900,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94300652,"statfs":{"total":96636764160,"available":96563867648,"internally_reserved":0,"allocated":3993600,"data_stored":2441691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509710,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74768,"kb_used_data":3320,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297072,"statfs":{"total":96636764160,"available":96560201728,"internally_reserved":0,"allocated":3399680,"data_stored":1851323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509708,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52556,"kb_used_data":3320,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319284,"statfs":{"total":96636764160,"available":96582946816,"internally_reserved":0,"allocated":3399680,"data_stored":1851323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509709,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30348,"kb_used_data":3320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341492,"statfs":{"total":96636764160,"available":96605687808,"internally_reserved":0,"allocated":3399680,"data_stored":1851323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509710,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48396,"kb_used_data":3320,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323444,"statfs":{"total":96636764160,"available":96587206656,"internally_reserved":0,"allocated":3399680,"data_stored":1851323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509710,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57300,"kb_used_data":3900,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314540,"statfs":{"total":96636764160,"available":96578088960,"internally_reserved":0,"allocated":3993600,"data_stored":2441691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542414,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56712,"kb_used_data":3320,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315128,"statfs":{"total":96636764160,"available":96578691072,"internally_reserved":0,"allocated":3399680,"data_stored":1851323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542413,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74760,"kb_used_data":3320,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297080,"statfs":{"total":96636764160,"available":96560209920,"internally_reserved":0,"allocated":3399680,"data_stored":1851323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:55:13.139 INFO:tasks.ceph:pgid 12.d last_scrub_stamp 2024-09-21T07:35:26.844971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=26, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.140 INFO:tasks.ceph:pgid 11.a last_scrub_stamp 2024-09-21T07:35:25.835993+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=25, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.140 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-21T07:33:02.438318+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=33, tm_sec=2, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.140 INFO:tasks.ceph:pgid 6.7 last_scrub_stamp 2024-09-21T07:34:22.238474+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=34, tm_sec=22, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.140 INFO:tasks.ceph:pgid 12.8 last_scrub_stamp 2024-09-21T07:35:26.844971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=26, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.140 INFO:tasks.ceph:pgid 11.f last_scrub_stamp 2024-09-21T07:35:25.835993+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=25, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.140 INFO:tasks.ceph:pgid 6.2 last_scrub_stamp 2024-09-21T07:34:22.238474+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=34, tm_sec=22, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.140 INFO:tasks.ceph:pgid 12.9 last_scrub_stamp 2024-09-21T07:35:26.844971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=26, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.140 INFO:tasks.ceph:pgid 11.e last_scrub_stamp 2024-09-21T07:35:25.835993+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=25, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.141 INFO:tasks.ceph:pgid 6.3 last_scrub_stamp 2024-09-21T07:34:22.238474+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=34, tm_sec=22, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.141 INFO:tasks.ceph:pgid 12.a last_scrub_stamp 2024-09-21T07:35:26.844971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=26, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.141 INFO:tasks.ceph:pgid 11.d last_scrub_stamp 2024-09-21T07:35:25.835993+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=25, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.141 INFO:tasks.ceph:pgid 6.0 last_scrub_stamp 2024-09-21T07:34:22.238474+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=34, tm_sec=22, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.141 INFO:tasks.ceph:pgid 12.b last_scrub_stamp 2024-09-21T07:35:26.844971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=26, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.141 INFO:tasks.ceph:pgid 11.c last_scrub_stamp 2024-09-21T07:35:25.835993+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=25, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.141 INFO:tasks.ceph:pgid 6.1 last_scrub_stamp 2024-09-21T07:34:22.238474+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=34, tm_sec=22, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.141 INFO:tasks.ceph:pgid 12.c last_scrub_stamp 2024-09-21T07:35:26.844971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=26, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.141 INFO:tasks.ceph:pgid 11.b last_scrub_stamp 2024-09-21T07:35:25.835993+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=25, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.142 INFO:tasks.ceph:pgid 12.e last_scrub_stamp 2024-09-21T07:35:26.844971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=26, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.142 INFO:tasks.ceph:pgid 11.9 last_scrub_stamp 2024-09-21T07:35:25.835993+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=25, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.142 INFO:tasks.ceph:pgid 6.4 last_scrub_stamp 2024-09-21T07:34:22.238474+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=34, tm_sec=22, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.142 INFO:tasks.ceph:pgid 12.f last_scrub_stamp 2024-09-21T07:35:26.844971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=26, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.142 INFO:tasks.ceph:pgid 11.8 last_scrub_stamp 2024-09-21T07:35:25.835993+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=25, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.142 INFO:tasks.ceph:pgid 6.5 last_scrub_stamp 2024-09-21T07:34:22.238474+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=34, tm_sec=22, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.142 INFO:tasks.ceph:pgid 12.0 last_scrub_stamp 2024-09-21T07:35:26.844971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=26, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.142 INFO:tasks.ceph:pgid 11.7 last_scrub_stamp 2024-09-21T07:35:25.835993+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=25, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.143 INFO:tasks.ceph:pgid 6.a last_scrub_stamp 2024-09-21T07:34:22.238474+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=34, tm_sec=22, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.143 INFO:tasks.ceph:pgid 12.1 last_scrub_stamp 2024-09-21T07:35:26.844971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=26, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.143 INFO:tasks.ceph:pgid 11.6 last_scrub_stamp 2024-09-21T07:35:25.835993+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=25, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.143 INFO:tasks.ceph:pgid 6.b last_scrub_stamp 2024-09-21T07:34:22.238474+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=34, tm_sec=22, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.143 INFO:tasks.ceph:pgid 12.2 last_scrub_stamp 2024-09-21T07:35:26.844971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=26, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.143 INFO:tasks.ceph:pgid 11.5 last_scrub_stamp 2024-09-21T07:35:25.835993+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=25, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.143 INFO:tasks.ceph:pgid 6.8 last_scrub_stamp 2024-09-21T07:34:22.238474+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=34, tm_sec=22, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.143 INFO:tasks.ceph:pgid 12.3 last_scrub_stamp 2024-09-21T07:35:26.844971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=26, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.143 INFO:tasks.ceph:pgid 11.4 last_scrub_stamp 2024-09-21T07:35:25.835993+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=25, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.143 INFO:tasks.ceph:pgid 6.9 last_scrub_stamp 2024-09-21T07:34:22.238474+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=34, tm_sec=22, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.144 INFO:tasks.ceph:pgid 12.4 last_scrub_stamp 2024-09-21T07:35:26.844971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=26, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.144 INFO:tasks.ceph:pgid 11.3 last_scrub_stamp 2024-09-21T07:35:25.835993+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=25, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.144 INFO:tasks.ceph:pgid 12.5 last_scrub_stamp 2024-09-21T07:35:26.844971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=26, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.144 INFO:tasks.ceph:pgid 11.2 last_scrub_stamp 2024-09-21T07:35:25.835993+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=25, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.144 INFO:tasks.ceph:pgid 6.f last_scrub_stamp 2024-09-21T07:34:22.238474+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=34, tm_sec=22, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.144 INFO:tasks.ceph:pgid 12.6 last_scrub_stamp 2024-09-21T07:35:26.844971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=26, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.144 INFO:tasks.ceph:pgid 11.1 last_scrub_stamp 2024-09-21T07:35:25.835993+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=25, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.144 INFO:tasks.ceph:pgid 6.c last_scrub_stamp 2024-09-21T07:34:22.238474+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=34, tm_sec=22, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.144 INFO:tasks.ceph:pgid 12.7 last_scrub_stamp 2024-09-21T07:35:26.844971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=26, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.145 INFO:tasks.ceph:pgid 11.0 last_scrub_stamp 2024-09-21T07:35:25.835993+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=25, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.145 INFO:tasks.ceph:pgid 6.d last_scrub_stamp 2024-09-21T07:34:22.238474+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=34, tm_sec=22, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.145 INFO:tasks.ceph:pgid 12.18 last_scrub_stamp 2024-09-21T07:35:26.844971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=26, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.145 INFO:tasks.ceph:pgid 11.1f last_scrub_stamp 2024-09-21T07:35:25.835993+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=25, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.145 INFO:tasks.ceph:pgid 6.12 last_scrub_stamp 2024-09-21T07:34:22.238474+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=34, tm_sec=22, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.145 INFO:tasks.ceph:pgid 12.19 last_scrub_stamp 2024-09-21T07:35:26.844971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=26, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.145 INFO:tasks.ceph:pgid 11.1e last_scrub_stamp 2024-09-21T07:35:25.835993+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=25, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.145 INFO:tasks.ceph:pgid 12.1a last_scrub_stamp 2024-09-21T07:35:26.844971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=26, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.145 INFO:tasks.ceph:pgid 11.1d last_scrub_stamp 2024-09-21T07:35:25.835993+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=25, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.145 INFO:tasks.ceph:pgid 6.10 last_scrub_stamp 2024-09-21T07:34:22.238474+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=34, tm_sec=22, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.146 INFO:tasks.ceph:pgid 12.1b last_scrub_stamp 2024-09-21T07:35:26.844971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=26, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.146 INFO:tasks.ceph:pgid 11.1c last_scrub_stamp 2024-09-21T07:35:25.835993+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=25, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.146 INFO:tasks.ceph:pgid 6.11 last_scrub_stamp 2024-09-21T07:34:22.238474+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=34, tm_sec=22, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.146 INFO:tasks.ceph:pgid 12.1c last_scrub_stamp 2024-09-21T07:35:26.844971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=26, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.146 INFO:tasks.ceph:pgid 11.1b last_scrub_stamp 2024-09-21T07:35:25.835993+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=25, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.146 INFO:tasks.ceph:pgid 6.16 last_scrub_stamp 2024-09-21T07:34:22.238474+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=34, tm_sec=22, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.146 INFO:tasks.ceph:pgid 12.1d last_scrub_stamp 2024-09-21T07:35:26.844971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=26, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.146 INFO:tasks.ceph:pgid 11.1a last_scrub_stamp 2024-09-21T07:35:25.835993+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=25, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.146 INFO:tasks.ceph:pgid 6.17 last_scrub_stamp 2024-09-21T07:34:22.238474+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=34, tm_sec=22, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.147 INFO:tasks.ceph:pgid 12.1e last_scrub_stamp 2024-09-21T07:35:26.844971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=26, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.147 INFO:tasks.ceph:pgid 11.19 last_scrub_stamp 2024-09-21T07:35:25.835993+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=25, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.147 INFO:tasks.ceph:pgid 6.14 last_scrub_stamp 2024-09-21T07:34:22.238474+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=34, tm_sec=22, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.147 INFO:tasks.ceph:pgid 12.1f last_scrub_stamp 2024-09-21T07:35:26.844971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=26, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.147 INFO:tasks.ceph:pgid 11.18 last_scrub_stamp 2024-09-21T07:35:25.835993+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=25, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.147 INFO:tasks.ceph:pgid 6.15 last_scrub_stamp 2024-09-21T07:34:22.238474+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=34, tm_sec=22, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.147 INFO:tasks.ceph:pgid 12.10 last_scrub_stamp 2024-09-21T07:35:26.844971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=26, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.147 INFO:tasks.ceph:pgid 11.17 last_scrub_stamp 2024-09-21T07:35:25.835993+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=25, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.147 INFO:tasks.ceph:pgid 6.1a last_scrub_stamp 2024-09-21T07:34:22.238474+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=34, tm_sec=22, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.147 INFO:tasks.ceph:pgid 12.11 last_scrub_stamp 2024-09-21T07:35:26.844971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=26, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.148 INFO:tasks.ceph:pgid 11.16 last_scrub_stamp 2024-09-21T07:35:25.835993+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=25, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.148 INFO:tasks.ceph:pgid 6.1b last_scrub_stamp 2024-09-21T07:34:22.238474+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=34, tm_sec=22, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.148 INFO:tasks.ceph:pgid 12.12 last_scrub_stamp 2024-09-21T07:35:26.844971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=26, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.148 INFO:tasks.ceph:pgid 11.15 last_scrub_stamp 2024-09-21T07:35:25.835993+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=25, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.148 INFO:tasks.ceph:pgid 6.18 last_scrub_stamp 2024-09-21T07:34:22.238474+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=34, tm_sec=22, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.148 INFO:tasks.ceph:pgid 12.13 last_scrub_stamp 2024-09-21T07:35:26.844971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=26, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.148 INFO:tasks.ceph:pgid 11.14 last_scrub_stamp 2024-09-21T07:35:25.835993+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=25, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.148 INFO:tasks.ceph:pgid 6.19 last_scrub_stamp 2024-09-21T07:34:22.238474+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=34, tm_sec=22, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.148 INFO:tasks.ceph:pgid 12.14 last_scrub_stamp 2024-09-21T07:35:26.844971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=26, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.149 INFO:tasks.ceph:pgid 11.13 last_scrub_stamp 2024-09-21T07:35:25.835993+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=25, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.149 INFO:tasks.ceph:pgid 6.1e last_scrub_stamp 2024-09-21T07:34:22.238474+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=34, tm_sec=22, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.149 INFO:tasks.ceph:pgid 12.15 last_scrub_stamp 2024-09-21T07:35:26.844971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=26, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.149 INFO:tasks.ceph:pgid 11.12 last_scrub_stamp 2024-09-21T07:35:25.835993+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=25, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.149 INFO:tasks.ceph:pgid 6.1f last_scrub_stamp 2024-09-21T07:34:22.238474+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=34, tm_sec=22, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.149 INFO:tasks.ceph:pgid 12.16 last_scrub_stamp 2024-09-21T07:35:26.844971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=26, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.149 INFO:tasks.ceph:pgid 11.11 last_scrub_stamp 2024-09-21T07:35:25.835993+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=25, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.149 INFO:tasks.ceph:pgid 6.1c last_scrub_stamp 2024-09-21T07:34:22.238474+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=34, tm_sec=22, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.149 INFO:tasks.ceph:pgid 12.17 last_scrub_stamp 2024-09-21T07:35:26.844971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=26, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.149 INFO:tasks.ceph:pgid 11.10 last_scrub_stamp 2024-09-21T07:35:25.835993+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=35, tm_sec=25, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.150 INFO:tasks.ceph:pgid 6.1d last_scrub_stamp 2024-09-21T07:34:22.238474+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=34, tm_sec=22, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=55, tm_sec=5, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:55:13.150 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-21T07:55:33.151 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:55:33.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.387+0000 7f7842000640 1 -- 172.21.15.39:0/344637849 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f783c15c690 msgr2=0x7f783c0d0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:33.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.387+0000 7f7842000640 1 --2- 172.21.15.39:0/344637849 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f783c15c690 0x7f783c0d0630 secure :-1 s=READY pgs=2929 cs=0 l=1 rev1=1 crypto rx=0x7f7824009830 tx=0x7f782402a530 comp rx=0 tx=0).stop 2024-09-21T07:55:33.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.387+0000 7f7842000640 1 -- 172.21.15.39:0/344637849 shutdown_connections 2024-09-21T07:55:33.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.387+0000 7f7842000640 1 --2- 172.21.15.39:0/344637849 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f783c15c690 0x7f783c0d0630 unknown :-1 s=CLOSED pgs=2929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:33.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.387+0000 7f7842000640 1 -- 172.21.15.39:0/344637849 >> 172.21.15.39:0/344637849 conn(0x7f783c06cfc0 msgr2=0x7f783c06d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:33.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.387+0000 7f7842000640 1 -- 172.21.15.39:0/344637849 shutdown_connections 2024-09-21T07:55:33.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.387+0000 7f7842000640 1 -- 172.21.15.39:0/344637849 wait complete. 2024-09-21T07:55:33.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.390+0000 7f7842000640 1 Processor -- start 2024-09-21T07:55:33.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.390+0000 7f7842000640 1 -- start start 2024-09-21T07:55:33.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.391+0000 7f7842000640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f783c0b5240 0x7f783c1e9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:33.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.391+0000 7f7842000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f783c0d0b70 0x7f783c13d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:33.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.391+0000 7f7842000640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f783c15c690 0x7f783c13de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:33.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.391+0000 7f7842000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f783c070120 con 0x7f783c0d0b70 2024-09-21T07:55:33.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.391+0000 7f7842000640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f783c0b4ae0 con 0x7f783c0b5240 2024-09-21T07:55:33.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.391+0000 7f7842000640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f783c0b4c60 con 0x7f783c15c690 2024-09-21T07:55:33.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.391+0000 7f7833e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f783c0d0b70 0x7f783c13d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:33.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.392+0000 7f7833e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f783c0d0b70 0x7f783c13d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38856/0 (socket says 172.21.15.39:38856) 2024-09-21T07:55:33.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.392+0000 7f7833e00640 1 -- 172.21.15.39:0/3007800564 learned_addr learned my addr 172.21.15.39:0/3007800564 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:55:33.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.392+0000 7f783be00640 1 --2- 172.21.15.39:0/3007800564 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f783c15c690 0x7f783c13de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:33.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.392+0000 7f783b400640 1 --2- 172.21.15.39:0/3007800564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f783c0b5240 0x7f783c1e9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:33.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.392+0000 7f7833e00640 1 -- 172.21.15.39:0/3007800564 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f783c15c690 msgr2=0x7f783c13de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:33.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.392+0000 7f7833e00640 1 --2- 172.21.15.39:0/3007800564 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f783c15c690 0x7f783c13de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:33.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.392+0000 7f7833e00640 1 -- 172.21.15.39:0/3007800564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f783c0b5240 msgr2=0x7f783c1e9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:33.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.392+0000 7f7833e00640 1 --2- 172.21.15.39:0/3007800564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f783c0b5240 0x7f783c1e9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:33.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.392+0000 7f7833e00640 1 -- 172.21.15.39:0/3007800564 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f783c156520 con 0x7f783c0d0b70 2024-09-21T07:55:33.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.392+0000 7f7833e00640 1 --2- 172.21.15.39:0/3007800564 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f783c0d0b70 0x7f783c13d500 secure :-1 s=READY pgs=2930 cs=0 l=1 rev1=1 crypto rx=0x7f7824009950 tx=0x7f7824033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:33.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.393+0000 7f7833400640 1 -- 172.21.15.39:0/3007800564 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7824031b60 con 0x7f783c0d0b70 2024-09-21T07:55:33.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.393+0000 7f7833400640 1 -- 172.21.15.39:0/3007800564 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7824031df0 con 0x7f783c0d0b70 2024-09-21T07:55:33.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.393+0000 7f7842000640 1 -- 172.21.15.39:0/3007800564 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f783c1e8f70 con 0x7f783c0d0b70 2024-09-21T07:55:33.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.393+0000 7f7833400640 1 -- 172.21.15.39:0/3007800564 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7824005280 con 0x7f783c0d0b70 2024-09-21T07:55:33.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.393+0000 7f7842000640 1 -- 172.21.15.39:0/3007800564 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f783c1e9530 con 0x7f783c0d0b70 2024-09-21T07:55:33.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.394+0000 7f7842000640 1 -- 172.21.15.39:0/3007800564 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f783c156290 con 0x7f783c0d0b70 2024-09-21T07:55:33.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.395+0000 7f7833400640 1 -- 172.21.15.39:0/3007800564 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7824005420 con 0x7f783c0d0b70 2024-09-21T07:55:33.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.396+0000 7f7833400640 1 --2- 172.21.15.39:0/3007800564 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7818078bc0 0x7f781807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:33.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.397+0000 7f7833400640 1 -- 172.21.15.39:0/3007800564 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(269..269 src has 1..269) ==== 6278+0+0 (secure 0 0 0) 0x7f78240bc260 con 0x7f783c0d0b70 2024-09-21T07:55:33.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.397+0000 7f783b400640 1 --2- 172.21.15.39:0/3007800564 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7818078bc0 0x7f781807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:33.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.398+0000 7f783b400640 1 --2- 172.21.15.39:0/3007800564 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7818078bc0 0x7f781807b0d0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f782c000f60 tx=0x7f782c009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:33.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.401+0000 7f7833400640 1 -- 172.21.15.39:0/3007800564 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f782403f030 con 0x7f783c0d0b70 2024-09-21T07:55:33.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.594+0000 7f7842000640 1 -- 172.21.15.39:0/3007800564 --> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f783c156800 con 0x7f7818078bc0 2024-09-21T07:55:33.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.599+0000 7f7833400640 1 -- 172.21.15.39:0/3007800564 <== mgr.4115 v2:172.21.15.148:6800/896204996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+256149 (secure 0 0 0) 0x7f783c156800 con 0x7f7818078bc0 2024-09-21T07:55:33.603 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:55:33.604 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-21T07:55:33.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.603+0000 7f7842000640 1 -- 172.21.15.39:0/3007800564 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7818078bc0 msgr2=0x7f781807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:33.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.603+0000 7f7842000640 1 --2- 172.21.15.39:0/3007800564 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7818078bc0 0x7f781807b0d0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f782c000f60 tx=0x7f782c009040 comp rx=0 tx=0).stop 2024-09-21T07:55:33.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.603+0000 7f7842000640 1 -- 172.21.15.39:0/3007800564 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f783c0d0b70 msgr2=0x7f783c13d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:33.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.603+0000 7f7842000640 1 --2- 172.21.15.39:0/3007800564 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f783c0d0b70 0x7f783c13d500 secure :-1 s=READY pgs=2930 cs=0 l=1 rev1=1 crypto rx=0x7f7824009950 tx=0x7f7824033c10 comp rx=0 tx=0).stop 2024-09-21T07:55:33.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.603+0000 7f7842000640 1 -- 172.21.15.39:0/3007800564 shutdown_connections 2024-09-21T07:55:33.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.603+0000 7f7842000640 1 --2- 172.21.15.39:0/3007800564 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7818078bc0 0x7f781807b0d0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:33.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.603+0000 7f7842000640 1 --2- 172.21.15.39:0/3007800564 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f783c15c690 0x7f783c13de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:33.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.603+0000 7f7842000640 1 --2- 172.21.15.39:0/3007800564 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f783c0d0b70 0x7f783c13d500 unknown :-1 s=CLOSED pgs=2930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:33.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.603+0000 7f7842000640 1 --2- 172.21.15.39:0/3007800564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f783c0b5240 0x7f783c1e9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:33.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.603+0000 7f7842000640 1 -- 172.21.15.39:0/3007800564 >> 172.21.15.39:0/3007800564 conn(0x7f783c06cfc0 msgr2=0x7f783c154cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:33.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.603+0000 7f7842000640 1 -- 172.21.15.39:0/3007800564 shutdown_connections 2024-09-21T07:55:33.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.603+0000 7f7842000640 1 -- 172.21.15.39:0/3007800564 wait complete. 2024-09-21T07:55:33.630 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":939,"stamp":"2024-09-21T07:55:31.995462+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":470152,"kb_used_data":27720,"kb_used_omap":12,"kb_used_meta":442291,"kb_avail":754504568,"statfs":{"total":773094113280,"available":772612677632,"internally_reserved":0,"allocated":28385280,"data_stored":15991320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":452906544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003722"},"pg_stats":[{"pgid":"12.d","version":"0'0","reported_seq":411,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:26.898376+0000","last_change":"2024-09-21T07:55:26.898376+0000","last_active":"2024-09-21T07:55:26.898376+0000","last_peered":"2024-09-21T07:55:26.898376+0000","last_clean":"2024-09-21T07:55:26.898376+0000","last_became_active":"2024-09-21T07:35:27.879274+0000","last_became_peered":"2024-09-21T07:35:27.879274+0000","last_unstale":"2024-09-21T07:55:26.898376+0000","last_undegraded":"2024-09-21T07:55:26.898376+0000","last_fullsized":"2024-09-21T07:55:26.898376+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:26.898257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:26.898257+0000","last_clean_scrub_stamp":"2024-09-21T07:55:26.898257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:50:29.092392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":421,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:19.918781+0000","last_change":"2024-09-21T07:55:19.918781+0000","last_active":"2024-09-21T07:55:19.918781+0000","last_peered":"2024-09-21T07:55:19.918781+0000","last_clean":"2024-09-21T07:55:19.918781+0000","last_became_active":"2024-09-21T07:35:26.866484+0000","last_became_peered":"2024-09-21T07:35:26.866484+0000","last_unstale":"2024-09-21T07:55:19.918781+0000","last_undegraded":"2024-09-21T07:55:19.918781+0000","last_fullsized":"2024-09-21T07:55:19.918781+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:19.918685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:19.918685+0000","last_clean_scrub_stamp":"2024-09-21T07:55:19.918685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:44.450858+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00076010100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"269'74","reported_seq":600,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:13.788390+0000","last_change":"2024-09-21T07:55:13.788390+0000","last_active":"2024-09-21T07:55:13.788390+0000","last_peered":"2024-09-21T07:55:13.788390+0000","last_clean":"2024-09-21T07:55:13.788390+0000","last_became_active":"2024-09-21T07:33:04.017106+0000","last_became_peered":"2024-09-21T07:33:04.017106+0000","last_unstale":"2024-09-21T07:55:13.788390+0000","last_undegraded":"2024-09-21T07:55:13.788390+0000","last_fullsized":"2024-09-21T07:55:13.788390+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'74","last_scrub_stamp":"2024-09-21T07:55:13.788289+0000","last_deep_scrub":"269'74","last_deep_scrub_stamp":"2024-09-21T07:55:13.788289+0000","last_clean_scrub_stamp":"2024-09-21T07:55:13.788289+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:10:08.679387+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":488,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:11.429597+0000","last_change":"2024-09-21T07:55:11.429597+0000","last_active":"2024-09-21T07:55:11.429597+0000","last_peered":"2024-09-21T07:55:11.429597+0000","last_clean":"2024-09-21T07:55:11.429597+0000","last_became_active":"2024-09-21T07:34:23.273465+0000","last_became_peered":"2024-09-21T07:34:23.273465+0000","last_unstale":"2024-09-21T07:55:11.429597+0000","last_undegraded":"2024-09-21T07:55:11.429597+0000","last_fullsized":"2024-09-21T07:55:11.429597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:11.429464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:11.429464+0000","last_clean_scrub_stamp":"2024-09-21T07:55:11.429464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:01:52.977180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060627700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":415,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:20.954740+0000","last_change":"2024-09-21T07:55:20.954740+0000","last_active":"2024-09-21T07:55:20.954740+0000","last_peered":"2024-09-21T07:55:20.954740+0000","last_clean":"2024-09-21T07:55:20.954740+0000","last_became_active":"2024-09-21T07:35:27.864176+0000","last_became_peered":"2024-09-21T07:35:27.864176+0000","last_unstale":"2024-09-21T07:55:20.954740+0000","last_undegraded":"2024-09-21T07:55:20.954740+0000","last_fullsized":"2024-09-21T07:55:20.954740+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.954642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.954642+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.954642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:52.720142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":418,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:22.901232+0000","last_change":"2024-09-21T07:55:22.901232+0000","last_active":"2024-09-21T07:55:22.901232+0000","last_peered":"2024-09-21T07:55:22.901232+0000","last_clean":"2024-09-21T07:55:22.901232+0000","last_became_active":"2024-09-21T07:35:26.870749+0000","last_became_peered":"2024-09-21T07:35:26.870749+0000","last_unstale":"2024-09-21T07:55:22.901232+0000","last_undegraded":"2024-09-21T07:55:22.901232+0000","last_fullsized":"2024-09-21T07:55:22.901232+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:22.901133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:22.901133+0000","last_clean_scrub_stamp":"2024-09-21T07:55:22.901133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:34.785531+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0010159570000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":465,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:13.267904+0000","last_change":"2024-09-21T07:55:13.267807+0000","last_active":"2024-09-21T07:55:13.267904+0000","last_peered":"2024-09-21T07:55:13.267904+0000","last_clean":"2024-09-21T07:55:13.267904+0000","last_became_active":"2024-09-21T07:34:23.279560+0000","last_became_peered":"2024-09-21T07:34:23.279560+0000","last_unstale":"2024-09-21T07:55:13.267904+0000","last_undegraded":"2024-09-21T07:55:13.267904+0000","last_fullsized":"2024-09-21T07:55:13.267904+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:13.267707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:13.267707+0000","last_clean_scrub_stamp":"2024-09-21T07:55:13.267707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:05.889203+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00059699799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":415,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:16.311918+0000","last_change":"2024-09-21T07:55:16.311918+0000","last_active":"2024-09-21T07:55:16.311918+0000","last_peered":"2024-09-21T07:55:16.311918+0000","last_clean":"2024-09-21T07:55:16.311918+0000","last_became_active":"2024-09-21T07:35:27.873277+0000","last_became_peered":"2024-09-21T07:35:27.873277+0000","last_unstale":"2024-09-21T07:55:16.311918+0000","last_undegraded":"2024-09-21T07:55:16.311918+0000","last_fullsized":"2024-09-21T07:55:16.311918+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:16.311820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:16.311820+0000","last_clean_scrub_stamp":"2024-09-21T07:55:16.311820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:51.451641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":414,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:16.244616+0000","last_change":"2024-09-21T07:55:16.244616+0000","last_active":"2024-09-21T07:55:16.244616+0000","last_peered":"2024-09-21T07:55:16.244616+0000","last_clean":"2024-09-21T07:55:16.244616+0000","last_became_active":"2024-09-21T07:35:26.874757+0000","last_became_peered":"2024-09-21T07:35:26.874757+0000","last_unstale":"2024-09-21T07:55:16.244616+0000","last_undegraded":"2024-09-21T07:55:16.244616+0000","last_fullsized":"2024-09-21T07:55:16.244616+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:16.244474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:16.244474+0000","last_clean_scrub_stamp":"2024-09-21T07:55:16.244474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:40:36.638643+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00061065700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":472,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:12.769278+0000","last_change":"2024-09-21T07:55:12.769278+0000","last_active":"2024-09-21T07:55:12.769278+0000","last_peered":"2024-09-21T07:55:12.769278+0000","last_clean":"2024-09-21T07:55:12.769278+0000","last_became_active":"2024-09-21T07:34:23.635282+0000","last_became_peered":"2024-09-21T07:34:23.635282+0000","last_unstale":"2024-09-21T07:55:12.769278+0000","last_undegraded":"2024-09-21T07:55:12.769278+0000","last_fullsized":"2024-09-21T07:55:12.769278+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:12.769185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:12.769185+0000","last_clean_scrub_stamp":"2024-09-21T07:55:12.769185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:37.259226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001527664,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":415,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:17.287213+0000","last_change":"2024-09-21T07:55:17.287213+0000","last_active":"2024-09-21T07:55:17.287213+0000","last_peered":"2024-09-21T07:55:17.287213+0000","last_clean":"2024-09-21T07:55:17.287213+0000","last_became_active":"2024-09-21T07:35:27.873245+0000","last_became_peered":"2024-09-21T07:35:27.873245+0000","last_unstale":"2024-09-21T07:55:17.287213+0000","last_undegraded":"2024-09-21T07:55:17.287213+0000","last_fullsized":"2024-09-21T07:55:17.287213+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:17.287085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:17.287085+0000","last_clean_scrub_stamp":"2024-09-21T07:55:17.287085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:15:42.682707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":421,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:11.430868+0000","last_change":"2024-09-21T07:55:11.430868+0000","last_active":"2024-09-21T07:55:11.430868+0000","last_peered":"2024-09-21T07:55:11.430868+0000","last_clean":"2024-09-21T07:55:11.430868+0000","last_became_active":"2024-09-21T07:35:26.876462+0000","last_became_peered":"2024-09-21T07:35:26.876462+0000","last_unstale":"2024-09-21T07:55:11.430868+0000","last_undegraded":"2024-09-21T07:55:11.430868+0000","last_fullsized":"2024-09-21T07:55:11.430868+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:11.430793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:11.430793+0000","last_clean_scrub_stamp":"2024-09-21T07:55:11.430793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:44.019619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012020920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":472,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:08.747185+0000","last_change":"2024-09-21T07:55:08.747185+0000","last_active":"2024-09-21T07:55:08.747185+0000","last_peered":"2024-09-21T07:55:08.747185+0000","last_clean":"2024-09-21T07:55:08.747185+0000","last_became_active":"2024-09-21T07:34:23.279205+0000","last_became_peered":"2024-09-21T07:34:23.279205+0000","last_unstale":"2024-09-21T07:55:08.747185+0000","last_undegraded":"2024-09-21T07:55:08.747185+0000","last_fullsized":"2024-09-21T07:55:08.747185+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:08.747085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:08.747085+0000","last_clean_scrub_stamp":"2024-09-21T07:55:08.747085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:10:03.638183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095320599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":413,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:21.957304+0000","last_change":"2024-09-21T07:55:21.957304+0000","last_active":"2024-09-21T07:55:21.957304+0000","last_peered":"2024-09-21T07:55:21.957304+0000","last_clean":"2024-09-21T07:55:21.957304+0000","last_became_active":"2024-09-21T07:35:27.861016+0000","last_became_peered":"2024-09-21T07:35:27.861016+0000","last_unstale":"2024-09-21T07:55:21.957304+0000","last_undegraded":"2024-09-21T07:55:21.957304+0000","last_fullsized":"2024-09-21T07:55:21.957304+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.957203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.957203+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.957203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:25.222657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":421,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:12.969988+0000","last_change":"2024-09-21T07:55:12.969988+0000","last_active":"2024-09-21T07:55:12.969988+0000","last_peered":"2024-09-21T07:55:12.969988+0000","last_clean":"2024-09-21T07:55:12.969988+0000","last_became_active":"2024-09-21T07:35:26.875496+0000","last_became_peered":"2024-09-21T07:35:26.875496+0000","last_unstale":"2024-09-21T07:55:12.969988+0000","last_undegraded":"2024-09-21T07:55:12.969988+0000","last_fullsized":"2024-09-21T07:55:12.969988+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:12.969903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:12.969903+0000","last_clean_scrub_stamp":"2024-09-21T07:55:12.969903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:37.537965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065859000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":474,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:11.007123+0000","last_change":"2024-09-21T07:55:11.007070+0000","last_active":"2024-09-21T07:55:11.007123+0000","last_peered":"2024-09-21T07:55:11.007123+0000","last_clean":"2024-09-21T07:55:11.007123+0000","last_became_active":"2024-09-21T07:34:23.635467+0000","last_became_peered":"2024-09-21T07:34:23.635467+0000","last_unstale":"2024-09-21T07:55:11.007123+0000","last_undegraded":"2024-09-21T07:55:11.007123+0000","last_fullsized":"2024-09-21T07:55:11.007123+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:11.006986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:11.006986+0000","last_clean_scrub_stamp":"2024-09-21T07:55:11.006986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:55:57.347820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000660558,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":413,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:18.775264+0000","last_change":"2024-09-21T07:55:18.775264+0000","last_active":"2024-09-21T07:55:18.775264+0000","last_peered":"2024-09-21T07:55:18.775264+0000","last_clean":"2024-09-21T07:55:18.775264+0000","last_became_active":"2024-09-21T07:35:27.870816+0000","last_became_peered":"2024-09-21T07:35:27.870816+0000","last_unstale":"2024-09-21T07:55:18.775264+0000","last_undegraded":"2024-09-21T07:55:18.775264+0000","last_fullsized":"2024-09-21T07:55:18.775264+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:18.775146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:18.775146+0000","last_clean_scrub_stamp":"2024-09-21T07:55:18.775146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:11:09.902558+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":420,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:18.962245+0000","last_change":"2024-09-21T07:55:18.962245+0000","last_active":"2024-09-21T07:55:18.962245+0000","last_peered":"2024-09-21T07:55:18.962245+0000","last_clean":"2024-09-21T07:55:18.962245+0000","last_became_active":"2024-09-21T07:35:26.866381+0000","last_became_peered":"2024-09-21T07:35:26.866381+0000","last_unstale":"2024-09-21T07:55:18.962245+0000","last_undegraded":"2024-09-21T07:55:18.962245+0000","last_fullsized":"2024-09-21T07:55:18.962245+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:18.962127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:18.962127+0000","last_clean_scrub_stamp":"2024-09-21T07:55:18.962127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:58.440259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073235399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":476,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:11.012278+0000","last_change":"2024-09-21T07:55:11.012278+0000","last_active":"2024-09-21T07:55:11.012278+0000","last_peered":"2024-09-21T07:55:11.012278+0000","last_clean":"2024-09-21T07:55:11.012278+0000","last_became_active":"2024-09-21T07:34:23.273431+0000","last_became_peered":"2024-09-21T07:34:23.273431+0000","last_unstale":"2024-09-21T07:55:11.012278+0000","last_undegraded":"2024-09-21T07:55:11.012278+0000","last_fullsized":"2024-09-21T07:55:11.012278+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:11.012203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:11.012203+0000","last_clean_scrub_stamp":"2024-09-21T07:55:11.012203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:43.236568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019803310000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":413,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:16.730313+0000","last_change":"2024-09-21T07:55:16.730313+0000","last_active":"2024-09-21T07:55:16.730313+0000","last_peered":"2024-09-21T07:55:16.730313+0000","last_clean":"2024-09-21T07:55:16.730313+0000","last_became_active":"2024-09-21T07:35:27.871232+0000","last_became_peered":"2024-09-21T07:35:27.871232+0000","last_unstale":"2024-09-21T07:55:16.730313+0000","last_undegraded":"2024-09-21T07:55:16.730313+0000","last_fullsized":"2024-09-21T07:55:16.730313+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:16.730209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:16.730209+0000","last_clean_scrub_stamp":"2024-09-21T07:55:16.730209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:09.871973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":422,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:17.959675+0000","last_change":"2024-09-21T07:55:17.959675+0000","last_active":"2024-09-21T07:55:17.959675+0000","last_peered":"2024-09-21T07:55:17.959675+0000","last_clean":"2024-09-21T07:55:17.959675+0000","last_became_active":"2024-09-21T07:35:26.866783+0000","last_became_peered":"2024-09-21T07:35:26.866783+0000","last_unstale":"2024-09-21T07:55:17.959675+0000","last_undegraded":"2024-09-21T07:55:17.959675+0000","last_fullsized":"2024-09-21T07:55:17.959675+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:17.959574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:17.959574+0000","last_clean_scrub_stamp":"2024-09-21T07:55:17.959574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:14.198021+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00098474099999999992,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":474,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:08.972380+0000","last_change":"2024-09-21T07:55:08.972264+0000","last_active":"2024-09-21T07:55:08.972380+0000","last_peered":"2024-09-21T07:55:08.972380+0000","last_clean":"2024-09-21T07:55:08.972380+0000","last_became_active":"2024-09-21T07:34:23.260950+0000","last_became_peered":"2024-09-21T07:34:23.260950+0000","last_unstale":"2024-09-21T07:55:08.972380+0000","last_undegraded":"2024-09-21T07:55:08.972380+0000","last_fullsized":"2024-09-21T07:55:08.972380+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:08.972170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:08.972170+0000","last_clean_scrub_stamp":"2024-09-21T07:55:08.972170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:11.038703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055135099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":416,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:19.445315+0000","last_change":"2024-09-21T07:55:19.445315+0000","last_active":"2024-09-21T07:55:19.445315+0000","last_peered":"2024-09-21T07:55:19.445315+0000","last_clean":"2024-09-21T07:55:19.445315+0000","last_became_active":"2024-09-21T07:35:27.870426+0000","last_became_peered":"2024-09-21T07:35:27.870426+0000","last_unstale":"2024-09-21T07:55:19.445315+0000","last_undegraded":"2024-09-21T07:55:19.445315+0000","last_fullsized":"2024-09-21T07:55:19.445315+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:19.445211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:19.445211+0000","last_clean_scrub_stamp":"2024-09-21T07:55:19.445211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:38:28.773103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":422,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:15.933754+0000","last_change":"2024-09-21T07:55:15.933754+0000","last_active":"2024-09-21T07:55:15.933754+0000","last_peered":"2024-09-21T07:55:15.933754+0000","last_clean":"2024-09-21T07:55:15.933754+0000","last_became_active":"2024-09-21T07:35:26.866750+0000","last_became_peered":"2024-09-21T07:35:26.866750+0000","last_unstale":"2024-09-21T07:55:15.933754+0000","last_undegraded":"2024-09-21T07:55:15.933754+0000","last_fullsized":"2024-09-21T07:55:15.933754+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:15.933655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:15.933655+0000","last_clean_scrub_stamp":"2024-09-21T07:55:15.933655+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:47.834615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083529700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":473,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:14.736371+0000","last_change":"2024-09-21T07:55:14.736308+0000","last_active":"2024-09-21T07:55:14.736371+0000","last_peered":"2024-09-21T07:55:14.736371+0000","last_clean":"2024-09-21T07:55:14.736371+0000","last_became_active":"2024-09-21T07:34:23.635909+0000","last_became_peered":"2024-09-21T07:34:23.635909+0000","last_unstale":"2024-09-21T07:55:14.736371+0000","last_undegraded":"2024-09-21T07:55:14.736371+0000","last_fullsized":"2024-09-21T07:55:14.736371+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:14.736215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:14.736215+0000","last_clean_scrub_stamp":"2024-09-21T07:55:14.736215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:42.234652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":409,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:18.244743+0000","last_change":"2024-09-21T07:55:18.244681+0000","last_active":"2024-09-21T07:55:18.244743+0000","last_peered":"2024-09-21T07:55:18.244743+0000","last_clean":"2024-09-21T07:55:18.244743+0000","last_became_active":"2024-09-21T07:35:27.874434+0000","last_became_peered":"2024-09-21T07:35:27.874434+0000","last_unstale":"2024-09-21T07:55:18.244743+0000","last_undegraded":"2024-09-21T07:55:18.244743+0000","last_fullsized":"2024-09-21T07:55:18.244743+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:18.244577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:18.244577+0000","last_clean_scrub_stamp":"2024-09-21T07:55:18.244577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:44:12.654897+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":436,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:14.401228+0000","last_change":"2024-09-21T07:55:14.401228+0000","last_active":"2024-09-21T07:55:14.401228+0000","last_peered":"2024-09-21T07:55:14.401228+0000","last_clean":"2024-09-21T07:55:14.401228+0000","last_became_active":"2024-09-21T07:35:26.874428+0000","last_became_peered":"2024-09-21T07:35:26.874428+0000","last_unstale":"2024-09-21T07:55:14.401228+0000","last_undegraded":"2024-09-21T07:55:14.401228+0000","last_fullsized":"2024-09-21T07:55:14.401228+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:14.401094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:14.401094+0000","last_clean_scrub_stamp":"2024-09-21T07:55:14.401094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:48.100463+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00048557299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":488,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:13.449250+0000","last_change":"2024-09-21T07:55:13.449250+0000","last_active":"2024-09-21T07:55:13.449250+0000","last_peered":"2024-09-21T07:55:13.449250+0000","last_clean":"2024-09-21T07:55:13.449250+0000","last_became_active":"2024-09-21T07:34:23.636372+0000","last_became_peered":"2024-09-21T07:34:23.636372+0000","last_unstale":"2024-09-21T07:55:13.449250+0000","last_undegraded":"2024-09-21T07:55:13.449250+0000","last_fullsized":"2024-09-21T07:55:13.449250+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:13.449135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:13.449135+0000","last_clean_scrub_stamp":"2024-09-21T07:55:13.449135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:40.947572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841451,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":417,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:17.470821+0000","last_change":"2024-09-21T07:55:17.470821+0000","last_active":"2024-09-21T07:55:17.470821+0000","last_peered":"2024-09-21T07:55:17.470821+0000","last_clean":"2024-09-21T07:55:17.470821+0000","last_became_active":"2024-09-21T07:35:27.877464+0000","last_became_peered":"2024-09-21T07:35:27.877464+0000","last_unstale":"2024-09-21T07:55:17.470821+0000","last_undegraded":"2024-09-21T07:55:17.470821+0000","last_fullsized":"2024-09-21T07:55:17.470821+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:17.470699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:17.470699+0000","last_clean_scrub_stamp":"2024-09-21T07:55:17.470699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:25.742455+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":421,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:13.669543+0000","last_change":"2024-09-21T07:55:13.669543+0000","last_active":"2024-09-21T07:55:13.669543+0000","last_peered":"2024-09-21T07:55:13.669543+0000","last_clean":"2024-09-21T07:55:13.669543+0000","last_became_active":"2024-09-21T07:35:26.877392+0000","last_became_peered":"2024-09-21T07:35:26.877392+0000","last_unstale":"2024-09-21T07:55:13.669543+0000","last_undegraded":"2024-09-21T07:55:13.669543+0000","last_fullsized":"2024-09-21T07:55:13.669543+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:13.669462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:13.669462+0000","last_clean_scrub_stamp":"2024-09-21T07:55:13.669462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:01:41.224374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071425700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":475,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:11.983525+0000","last_change":"2024-09-21T07:55:11.983525+0000","last_active":"2024-09-21T07:55:11.983525+0000","last_peered":"2024-09-21T07:55:11.983525+0000","last_clean":"2024-09-21T07:55:11.983525+0000","last_became_active":"2024-09-21T07:34:23.274405+0000","last_became_peered":"2024-09-21T07:34:23.274405+0000","last_unstale":"2024-09-21T07:55:11.983525+0000","last_undegraded":"2024-09-21T07:55:11.983525+0000","last_fullsized":"2024-09-21T07:55:11.983525+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:11.983444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:11.983444+0000","last_clean_scrub_stamp":"2024-09-21T07:55:11.983444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:31:19.413871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022009920000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":408,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:19.262497+0000","last_change":"2024-09-21T07:55:19.262497+0000","last_active":"2024-09-21T07:55:19.262497+0000","last_peered":"2024-09-21T07:55:19.262497+0000","last_clean":"2024-09-21T07:55:19.262497+0000","last_became_active":"2024-09-21T07:35:27.867015+0000","last_became_peered":"2024-09-21T07:35:27.867015+0000","last_unstale":"2024-09-21T07:55:19.262497+0000","last_undegraded":"2024-09-21T07:55:19.262497+0000","last_fullsized":"2024-09-21T07:55:19.262497+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:19.262362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:19.262362+0000","last_clean_scrub_stamp":"2024-09-21T07:55:19.262362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:26.214583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":419,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:20.853068+0000","last_change":"2024-09-21T07:55:20.853068+0000","last_active":"2024-09-21T07:55:20.853068+0000","last_peered":"2024-09-21T07:55:20.853068+0000","last_clean":"2024-09-21T07:55:20.853068+0000","last_became_active":"2024-09-21T07:35:26.872230+0000","last_became_peered":"2024-09-21T07:35:26.872230+0000","last_unstale":"2024-09-21T07:55:20.853068+0000","last_undegraded":"2024-09-21T07:55:20.853068+0000","last_fullsized":"2024-09-21T07:55:20.853068+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.852964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.852964+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.852964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:50.642993+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.001197449,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":472,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:16.779234+0000","last_change":"2024-09-21T07:55:16.779234+0000","last_active":"2024-09-21T07:55:16.779234+0000","last_peered":"2024-09-21T07:55:16.779234+0000","last_clean":"2024-09-21T07:55:16.779234+0000","last_became_active":"2024-09-21T07:34:23.272657+0000","last_became_peered":"2024-09-21T07:34:23.272657+0000","last_unstale":"2024-09-21T07:55:16.779234+0000","last_undegraded":"2024-09-21T07:55:16.779234+0000","last_fullsized":"2024-09-21T07:55:16.779234+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:16.779122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:16.779122+0000","last_clean_scrub_stamp":"2024-09-21T07:55:16.779122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:19.543921+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00067143200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":414,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:14.719450+0000","last_change":"2024-09-21T07:55:14.719450+0000","last_active":"2024-09-21T07:55:14.719450+0000","last_peered":"2024-09-21T07:55:14.719450+0000","last_clean":"2024-09-21T07:55:14.719450+0000","last_became_active":"2024-09-21T07:35:27.875724+0000","last_became_peered":"2024-09-21T07:35:27.875724+0000","last_unstale":"2024-09-21T07:55:14.719450+0000","last_undegraded":"2024-09-21T07:55:14.719450+0000","last_fullsized":"2024-09-21T07:55:14.719450+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:14.719326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:14.719326+0000","last_clean_scrub_stamp":"2024-09-21T07:55:14.719326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:40.683039+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":419,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:19.829198+0000","last_change":"2024-09-21T07:55:19.829198+0000","last_active":"2024-09-21T07:55:19.829198+0000","last_peered":"2024-09-21T07:55:19.829198+0000","last_clean":"2024-09-21T07:55:19.829198+0000","last_became_active":"2024-09-21T07:35:26.874316+0000","last_became_peered":"2024-09-21T07:35:26.874316+0000","last_unstale":"2024-09-21T07:55:19.829198+0000","last_undegraded":"2024-09-21T07:55:19.829198+0000","last_fullsized":"2024-09-21T07:55:19.829198+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:19.829113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:19.829113+0000","last_clean_scrub_stamp":"2024-09-21T07:55:19.829113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:45.792825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011990760000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":472,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:07.789563+0000","last_change":"2024-09-21T07:55:07.789563+0000","last_active":"2024-09-21T07:55:07.789563+0000","last_peered":"2024-09-21T07:55:07.789563+0000","last_clean":"2024-09-21T07:55:07.789563+0000","last_became_active":"2024-09-21T07:34:23.277807+0000","last_became_peered":"2024-09-21T07:34:23.277807+0000","last_unstale":"2024-09-21T07:55:07.789563+0000","last_undegraded":"2024-09-21T07:55:07.789563+0000","last_fullsized":"2024-09-21T07:55:07.789563+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:07.789479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:07.789479+0000","last_clean_scrub_stamp":"2024-09-21T07:55:07.789479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:32.279520+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00086619099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":417,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:16.476950+0000","last_change":"2024-09-21T07:55:16.476950+0000","last_active":"2024-09-21T07:55:16.476950+0000","last_peered":"2024-09-21T07:55:16.476950+0000","last_clean":"2024-09-21T07:55:16.476950+0000","last_became_active":"2024-09-21T07:35:27.872483+0000","last_became_peered":"2024-09-21T07:35:27.872483+0000","last_unstale":"2024-09-21T07:55:16.476950+0000","last_undegraded":"2024-09-21T07:55:16.476950+0000","last_fullsized":"2024-09-21T07:55:16.476950+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:16.476825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:16.476825+0000","last_clean_scrub_stamp":"2024-09-21T07:55:16.476825+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:16.047137+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":424,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:12.500371+0000","last_change":"2024-09-21T07:55:12.500371+0000","last_active":"2024-09-21T07:55:12.500371+0000","last_peered":"2024-09-21T07:55:12.500371+0000","last_clean":"2024-09-21T07:55:12.500371+0000","last_became_active":"2024-09-21T07:35:26.872377+0000","last_became_peered":"2024-09-21T07:35:26.872377+0000","last_unstale":"2024-09-21T07:55:12.500371+0000","last_undegraded":"2024-09-21T07:55:12.500371+0000","last_fullsized":"2024-09-21T07:55:12.500371+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:12.500275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:12.500275+0000","last_clean_scrub_stamp":"2024-09-21T07:55:12.500275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:13.042070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083077200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":464,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:11.320650+0000","last_change":"2024-09-21T07:55:11.320650+0000","last_active":"2024-09-21T07:55:11.320650+0000","last_peered":"2024-09-21T07:55:11.320650+0000","last_clean":"2024-09-21T07:55:11.320650+0000","last_became_active":"2024-09-21T07:34:23.267770+0000","last_became_peered":"2024-09-21T07:34:23.267770+0000","last_unstale":"2024-09-21T07:55:11.320650+0000","last_undegraded":"2024-09-21T07:55:11.320650+0000","last_fullsized":"2024-09-21T07:55:11.320650+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:11.320555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:11.320555+0000","last_clean_scrub_stamp":"2024-09-21T07:55:11.320555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:38.127843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086378500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":417,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:18.431163+0000","last_change":"2024-09-21T07:55:18.431163+0000","last_active":"2024-09-21T07:55:18.431163+0000","last_peered":"2024-09-21T07:55:18.431163+0000","last_clean":"2024-09-21T07:55:18.431163+0000","last_became_active":"2024-09-21T07:35:27.869811+0000","last_became_peered":"2024-09-21T07:35:27.869811+0000","last_unstale":"2024-09-21T07:55:18.431163+0000","last_undegraded":"2024-09-21T07:55:18.431163+0000","last_fullsized":"2024-09-21T07:55:18.431163+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:18.431062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:18.431062+0000","last_clean_scrub_stamp":"2024-09-21T07:55:18.431062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:56:44.739214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":422,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:10.463928+0000","last_change":"2024-09-21T07:55:10.463928+0000","last_active":"2024-09-21T07:55:10.463928+0000","last_peered":"2024-09-21T07:55:10.463928+0000","last_clean":"2024-09-21T07:55:10.463928+0000","last_became_active":"2024-09-21T07:35:26.876779+0000","last_became_peered":"2024-09-21T07:35:26.876779+0000","last_unstale":"2024-09-21T07:55:10.463928+0000","last_undegraded":"2024-09-21T07:55:10.463928+0000","last_fullsized":"2024-09-21T07:55:10.463928+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:10.463839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:10.463839+0000","last_clean_scrub_stamp":"2024-09-21T07:55:10.463839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:37:14.140610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000913188,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":474,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:12.418184+0000","last_change":"2024-09-21T07:55:12.418184+0000","last_active":"2024-09-21T07:55:12.418184+0000","last_peered":"2024-09-21T07:55:12.418184+0000","last_clean":"2024-09-21T07:55:12.418184+0000","last_became_active":"2024-09-21T07:34:23.275580+0000","last_became_peered":"2024-09-21T07:34:23.275580+0000","last_unstale":"2024-09-21T07:55:12.418184+0000","last_undegraded":"2024-09-21T07:55:12.418184+0000","last_fullsized":"2024-09-21T07:55:12.418184+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:12.418046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:12.418046+0000","last_clean_scrub_stamp":"2024-09-21T07:55:12.418046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:09.103347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014706980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":415,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:16.969908+0000","last_change":"2024-09-21T07:55:16.969908+0000","last_active":"2024-09-21T07:55:16.969908+0000","last_peered":"2024-09-21T07:55:16.969908+0000","last_clean":"2024-09-21T07:55:16.969908+0000","last_became_active":"2024-09-21T07:35:27.874671+0000","last_became_peered":"2024-09-21T07:35:27.874671+0000","last_unstale":"2024-09-21T07:55:16.969908+0000","last_undegraded":"2024-09-21T07:55:16.969908+0000","last_fullsized":"2024-09-21T07:55:16.969908+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:16.969807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:16.969807+0000","last_clean_scrub_stamp":"2024-09-21T07:55:16.969807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:52.262709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":424,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:13.463403+0000","last_change":"2024-09-21T07:55:13.463403+0000","last_active":"2024-09-21T07:55:13.463403+0000","last_peered":"2024-09-21T07:55:13.463403+0000","last_clean":"2024-09-21T07:55:13.463403+0000","last_became_active":"2024-09-21T07:35:26.874575+0000","last_became_peered":"2024-09-21T07:35:26.874575+0000","last_unstale":"2024-09-21T07:55:13.463403+0000","last_undegraded":"2024-09-21T07:55:13.463403+0000","last_fullsized":"2024-09-21T07:55:13.463403+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:13.463275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:13.463275+0000","last_clean_scrub_stamp":"2024-09-21T07:55:13.463275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:30.416090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061011599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":475,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:13.945641+0000","last_change":"2024-09-21T07:55:13.945641+0000","last_active":"2024-09-21T07:55:13.945641+0000","last_peered":"2024-09-21T07:55:13.945641+0000","last_clean":"2024-09-21T07:55:13.945641+0000","last_became_active":"2024-09-21T07:34:23.635444+0000","last_became_peered":"2024-09-21T07:34:23.635444+0000","last_unstale":"2024-09-21T07:55:13.945641+0000","last_undegraded":"2024-09-21T07:55:13.945641+0000","last_fullsized":"2024-09-21T07:55:13.945641+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:13.945556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:13.945556+0000","last_clean_scrub_stamp":"2024-09-21T07:55:13.945556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:20:34.922519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001952636,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":415,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:13.922006+0000","last_change":"2024-09-21T07:55:13.922006+0000","last_active":"2024-09-21T07:55:13.922006+0000","last_peered":"2024-09-21T07:55:13.922006+0000","last_clean":"2024-09-21T07:55:13.922006+0000","last_became_active":"2024-09-21T07:35:27.877982+0000","last_became_peered":"2024-09-21T07:35:27.877982+0000","last_unstale":"2024-09-21T07:55:13.922006+0000","last_undegraded":"2024-09-21T07:55:13.922006+0000","last_fullsized":"2024-09-21T07:55:13.922006+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:13.921912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:13.921912+0000","last_clean_scrub_stamp":"2024-09-21T07:55:13.921912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:41.172662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":423,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:09.963544+0000","last_change":"2024-09-21T07:55:09.963492+0000","last_active":"2024-09-21T07:55:09.963544+0000","last_peered":"2024-09-21T07:55:09.963544+0000","last_clean":"2024-09-21T07:55:09.963544+0000","last_became_active":"2024-09-21T07:35:26.870516+0000","last_became_peered":"2024-09-21T07:35:26.870516+0000","last_unstale":"2024-09-21T07:55:09.963544+0000","last_undegraded":"2024-09-21T07:55:09.963544+0000","last_fullsized":"2024-09-21T07:55:09.963544+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:09.963408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:09.963408+0000","last_clean_scrub_stamp":"2024-09-21T07:55:09.963408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:33.703206+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00070705699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":488,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:12.418963+0000","last_change":"2024-09-21T07:55:12.418963+0000","last_active":"2024-09-21T07:55:12.418963+0000","last_peered":"2024-09-21T07:55:12.418963+0000","last_clean":"2024-09-21T07:55:12.418963+0000","last_became_active":"2024-09-21T07:34:23.259573+0000","last_became_peered":"2024-09-21T07:34:23.259573+0000","last_unstale":"2024-09-21T07:55:12.418963+0000","last_undegraded":"2024-09-21T07:55:12.418963+0000","last_fullsized":"2024-09-21T07:55:12.418963+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:12.418868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:12.418868+0000","last_clean_scrub_stamp":"2024-09-21T07:55:12.418868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:10:07.309966+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00058108400000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":414,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:15.929142+0000","last_change":"2024-09-21T07:55:15.929142+0000","last_active":"2024-09-21T07:55:15.929142+0000","last_peered":"2024-09-21T07:55:15.929142+0000","last_clean":"2024-09-21T07:55:15.929142+0000","last_became_active":"2024-09-21T07:35:27.871421+0000","last_became_peered":"2024-09-21T07:35:27.871421+0000","last_unstale":"2024-09-21T07:55:15.929142+0000","last_undegraded":"2024-09-21T07:55:15.929142+0000","last_fullsized":"2024-09-21T07:55:15.929142+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:15.929038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:15.929038+0000","last_clean_scrub_stamp":"2024-09-21T07:55:15.929038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:53.141040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":422,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:15.352318+0000","last_change":"2024-09-21T07:55:15.352318+0000","last_active":"2024-09-21T07:55:15.352318+0000","last_peered":"2024-09-21T07:55:15.352318+0000","last_clean":"2024-09-21T07:55:15.352318+0000","last_became_active":"2024-09-21T07:35:26.873149+0000","last_became_peered":"2024-09-21T07:35:26.873149+0000","last_unstale":"2024-09-21T07:55:15.352318+0000","last_undegraded":"2024-09-21T07:55:15.352318+0000","last_fullsized":"2024-09-21T07:55:15.352318+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:15.352213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:15.352213+0000","last_clean_scrub_stamp":"2024-09-21T07:55:15.352213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:40.578102+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0011988369999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":472,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:17.825068+0000","last_change":"2024-09-21T07:55:17.825009+0000","last_active":"2024-09-21T07:55:17.825068+0000","last_peered":"2024-09-21T07:55:17.825068+0000","last_clean":"2024-09-21T07:55:17.825068+0000","last_became_active":"2024-09-21T07:34:23.271902+0000","last_became_peered":"2024-09-21T07:34:23.271902+0000","last_unstale":"2024-09-21T07:55:17.825068+0000","last_undegraded":"2024-09-21T07:55:17.825068+0000","last_fullsized":"2024-09-21T07:55:17.825068+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:17.824913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:17.824913+0000","last_clean_scrub_stamp":"2024-09-21T07:55:17.824913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:22.078210+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.001686358,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":413,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:19.795739+0000","last_change":"2024-09-21T07:55:19.795739+0000","last_active":"2024-09-21T07:55:19.795739+0000","last_peered":"2024-09-21T07:55:19.795739+0000","last_clean":"2024-09-21T07:55:19.795739+0000","last_became_active":"2024-09-21T07:35:27.872149+0000","last_became_peered":"2024-09-21T07:35:27.872149+0000","last_unstale":"2024-09-21T07:55:19.795739+0000","last_undegraded":"2024-09-21T07:55:19.795739+0000","last_fullsized":"2024-09-21T07:55:19.795739+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:19.795638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:19.795638+0000","last_clean_scrub_stamp":"2024-09-21T07:55:19.795638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:51:57.499733+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":419,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:25.913017+0000","last_change":"2024-09-21T07:55:25.913017+0000","last_active":"2024-09-21T07:55:25.913017+0000","last_peered":"2024-09-21T07:55:25.913017+0000","last_clean":"2024-09-21T07:55:25.913017+0000","last_became_active":"2024-09-21T07:35:26.873018+0000","last_became_peered":"2024-09-21T07:35:26.873018+0000","last_unstale":"2024-09-21T07:55:25.913017+0000","last_undegraded":"2024-09-21T07:55:25.913017+0000","last_fullsized":"2024-09-21T07:55:25.913017+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:25.912917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:25.912917+0000","last_clean_scrub_stamp":"2024-09-21T07:55:25.912917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:03.056664+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0012496510000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":475,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:10.043113+0000","last_change":"2024-09-21T07:55:10.043061+0000","last_active":"2024-09-21T07:55:10.043113+0000","last_peered":"2024-09-21T07:55:10.043113+0000","last_clean":"2024-09-21T07:55:10.043113+0000","last_became_active":"2024-09-21T07:34:23.278812+0000","last_became_peered":"2024-09-21T07:34:23.278812+0000","last_unstale":"2024-09-21T07:55:10.043113+0000","last_undegraded":"2024-09-21T07:55:10.043113+0000","last_fullsized":"2024-09-21T07:55:10.043113+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:10.042970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:10.042970+0000","last_clean_scrub_stamp":"2024-09-21T07:55:10.042970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:56:01.384383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356463,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":407,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:21.257728+0000","last_change":"2024-09-21T07:55:21.257652+0000","last_active":"2024-09-21T07:55:21.257728+0000","last_peered":"2024-09-21T07:55:21.257728+0000","last_clean":"2024-09-21T07:55:21.257728+0000","last_became_active":"2024-09-21T07:35:27.868105+0000","last_became_peered":"2024-09-21T07:35:27.868105+0000","last_unstale":"2024-09-21T07:55:21.257728+0000","last_undegraded":"2024-09-21T07:55:21.257728+0000","last_fullsized":"2024-09-21T07:55:21.257728+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.257501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.257501+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.257501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:02:54.822245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":418,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.886950+0000","last_change":"2024-09-21T07:55:24.886879+0000","last_active":"2024-09-21T07:55:24.886950+0000","last_peered":"2024-09-21T07:55:24.886950+0000","last_clean":"2024-09-21T07:55:24.886950+0000","last_became_active":"2024-09-21T07:35:26.872165+0000","last_became_peered":"2024-09-21T07:35:26.872165+0000","last_unstale":"2024-09-21T07:55:24.886950+0000","last_undegraded":"2024-09-21T07:55:24.886950+0000","last_fullsized":"2024-09-21T07:55:24.886950+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:24.886776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:24.886776+0000","last_clean_scrub_stamp":"2024-09-21T07:55:24.886776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:40:51.102519+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0013795179999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":472,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:10.677498+0000","last_change":"2024-09-21T07:55:10.677449+0000","last_active":"2024-09-21T07:55:10.677498+0000","last_peered":"2024-09-21T07:55:10.677498+0000","last_clean":"2024-09-21T07:55:10.677498+0000","last_became_active":"2024-09-21T07:34:23.277708+0000","last_became_peered":"2024-09-21T07:34:23.277708+0000","last_unstale":"2024-09-21T07:55:10.677498+0000","last_undegraded":"2024-09-21T07:55:10.677498+0000","last_fullsized":"2024-09-21T07:55:10.677498+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:10.677356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:10.677356+0000","last_clean_scrub_stamp":"2024-09-21T07:55:10.677356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:44.376724+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.000597566,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":407,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:20.250352+0000","last_change":"2024-09-21T07:55:20.250294+0000","last_active":"2024-09-21T07:55:20.250352+0000","last_peered":"2024-09-21T07:55:20.250352+0000","last_clean":"2024-09-21T07:55:20.250352+0000","last_became_active":"2024-09-21T07:35:27.874265+0000","last_became_peered":"2024-09-21T07:35:27.874265+0000","last_unstale":"2024-09-21T07:55:20.250352+0000","last_undegraded":"2024-09-21T07:55:20.250352+0000","last_fullsized":"2024-09-21T07:55:20.250352+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.250197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.250197+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.250197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:03.549671+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":423,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:15.502372+0000","last_change":"2024-09-21T07:55:15.502277+0000","last_active":"2024-09-21T07:55:15.502372+0000","last_peered":"2024-09-21T07:55:15.502372+0000","last_clean":"2024-09-21T07:55:15.502372+0000","last_became_active":"2024-09-21T07:35:26.870917+0000","last_became_peered":"2024-09-21T07:35:26.870917+0000","last_unstale":"2024-09-21T07:55:15.502372+0000","last_undegraded":"2024-09-21T07:55:15.502372+0000","last_fullsized":"2024-09-21T07:55:15.502372+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:15.502154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:15.502154+0000","last_clean_scrub_stamp":"2024-09-21T07:55:15.502154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:29:18.541882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059198499999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":476,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:12.978682+0000","last_change":"2024-09-21T07:55:12.978633+0000","last_active":"2024-09-21T07:55:12.978682+0000","last_peered":"2024-09-21T07:55:12.978682+0000","last_clean":"2024-09-21T07:55:12.978682+0000","last_became_active":"2024-09-21T07:34:23.279042+0000","last_became_peered":"2024-09-21T07:34:23.279042+0000","last_unstale":"2024-09-21T07:55:12.978682+0000","last_undegraded":"2024-09-21T07:55:12.978682+0000","last_fullsized":"2024-09-21T07:55:12.978682+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:12.978551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:12.978551+0000","last_clean_scrub_stamp":"2024-09-21T07:55:12.978551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:39.690980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019282100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":413,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:20.790095+0000","last_change":"2024-09-21T07:55:20.790042+0000","last_active":"2024-09-21T07:55:20.790095+0000","last_peered":"2024-09-21T07:55:20.790095+0000","last_clean":"2024-09-21T07:55:20.790095+0000","last_became_active":"2024-09-21T07:35:27.869448+0000","last_became_peered":"2024-09-21T07:35:27.869448+0000","last_unstale":"2024-09-21T07:55:20.790095+0000","last_undegraded":"2024-09-21T07:55:20.790095+0000","last_fullsized":"2024-09-21T07:55:20.790095+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.789943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.789943+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.789943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:28:57.022656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":414,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:17.272874+0000","last_change":"2024-09-21T07:55:17.272806+0000","last_active":"2024-09-21T07:55:17.272874+0000","last_peered":"2024-09-21T07:55:17.272874+0000","last_clean":"2024-09-21T07:55:17.272874+0000","last_became_active":"2024-09-21T07:35:26.879238+0000","last_became_peered":"2024-09-21T07:35:26.879238+0000","last_unstale":"2024-09-21T07:55:17.272874+0000","last_undegraded":"2024-09-21T07:55:17.272874+0000","last_fullsized":"2024-09-21T07:55:17.272874+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:17.272705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:17.272705+0000","last_clean_scrub_stamp":"2024-09-21T07:55:17.272705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:52.496801+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00092223100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":471,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:09.717492+0000","last_change":"2024-09-21T07:55:09.717492+0000","last_active":"2024-09-21T07:55:09.717492+0000","last_peered":"2024-09-21T07:55:09.717492+0000","last_clean":"2024-09-21T07:55:09.717492+0000","last_became_active":"2024-09-21T07:34:23.273566+0000","last_became_peered":"2024-09-21T07:34:23.273566+0000","last_unstale":"2024-09-21T07:55:09.717492+0000","last_undegraded":"2024-09-21T07:55:09.717492+0000","last_fullsized":"2024-09-21T07:55:09.717492+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:09.717400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:09.717400+0000","last_clean_scrub_stamp":"2024-09-21T07:55:09.717400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:37.215837+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00079897300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":413,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:21.811245+0000","last_change":"2024-09-21T07:55:21.811173+0000","last_active":"2024-09-21T07:55:21.811245+0000","last_peered":"2024-09-21T07:55:21.811245+0000","last_clean":"2024-09-21T07:55:21.811245+0000","last_became_active":"2024-09-21T07:35:27.878335+0000","last_became_peered":"2024-09-21T07:35:27.878335+0000","last_unstale":"2024-09-21T07:55:21.811245+0000","last_undegraded":"2024-09-21T07:55:21.811245+0000","last_fullsized":"2024-09-21T07:55:21.811245+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.811064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.811064+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.811064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:55:33.206991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":435,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:18.429308+0000","last_change":"2024-09-21T07:55:18.429240+0000","last_active":"2024-09-21T07:55:18.429308+0000","last_peered":"2024-09-21T07:55:18.429308+0000","last_clean":"2024-09-21T07:55:18.429308+0000","last_became_active":"2024-09-21T07:35:26.876944+0000","last_became_peered":"2024-09-21T07:35:26.876944+0000","last_unstale":"2024-09-21T07:55:18.429308+0000","last_undegraded":"2024-09-21T07:55:18.429308+0000","last_fullsized":"2024-09-21T07:55:18.429308+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:18.429120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:18.429120+0000","last_clean_scrub_stamp":"2024-09-21T07:55:18.429120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:44.392833+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00090862799999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":463,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:12.297434+0000","last_change":"2024-09-21T07:55:12.297434+0000","last_active":"2024-09-21T07:55:12.297434+0000","last_peered":"2024-09-21T07:55:12.297434+0000","last_clean":"2024-09-21T07:55:12.297434+0000","last_became_active":"2024-09-21T07:34:23.268926+0000","last_became_peered":"2024-09-21T07:34:23.268926+0000","last_unstale":"2024-09-21T07:55:12.297434+0000","last_undegraded":"2024-09-21T07:55:12.297434+0000","last_fullsized":"2024-09-21T07:55:12.297434+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:12.297320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:12.297320+0000","last_clean_scrub_stamp":"2024-09-21T07:55:12.297320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:51:14.634262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001101851,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":409,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:22.287036+0000","last_change":"2024-09-21T07:55:22.286963+0000","last_active":"2024-09-21T07:55:22.287036+0000","last_peered":"2024-09-21T07:55:22.287036+0000","last_clean":"2024-09-21T07:55:22.287036+0000","last_became_active":"2024-09-21T07:35:27.877050+0000","last_became_peered":"2024-09-21T07:35:27.877050+0000","last_unstale":"2024-09-21T07:55:22.287036+0000","last_undegraded":"2024-09-21T07:55:22.287036+0000","last_fullsized":"2024-09-21T07:55:22.287036+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:22.286855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:22.286855+0000","last_clean_scrub_stamp":"2024-09-21T07:55:22.286855+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:22.898233+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":420,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:12.694035+0000","last_change":"2024-09-21T07:55:12.694035+0000","last_active":"2024-09-21T07:55:12.694035+0000","last_peered":"2024-09-21T07:55:12.694035+0000","last_clean":"2024-09-21T07:55:12.694035+0000","last_became_active":"2024-09-21T07:35:26.874011+0000","last_became_peered":"2024-09-21T07:35:26.874011+0000","last_unstale":"2024-09-21T07:55:12.694035+0000","last_undegraded":"2024-09-21T07:55:12.694035+0000","last_fullsized":"2024-09-21T07:55:12.694035+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:12.693945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:12.693945+0000","last_clean_scrub_stamp":"2024-09-21T07:55:12.693945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:42:45.534980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071644799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":474,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:09.507899+0000","last_change":"2024-09-21T07:55:09.507791+0000","last_active":"2024-09-21T07:55:09.507899+0000","last_peered":"2024-09-21T07:55:09.507899+0000","last_clean":"2024-09-21T07:55:09.507899+0000","last_became_active":"2024-09-21T07:34:23.265679+0000","last_became_peered":"2024-09-21T07:34:23.265679+0000","last_unstale":"2024-09-21T07:55:09.507899+0000","last_undegraded":"2024-09-21T07:55:09.507899+0000","last_fullsized":"2024-09-21T07:55:09.507899+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:09.507691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:09.507691+0000","last_clean_scrub_stamp":"2024-09-21T07:55:09.507691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:18.341956+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.001298518,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":417,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.348958+0000","last_change":"2024-09-21T07:55:23.348958+0000","last_active":"2024-09-21T07:55:23.348958+0000","last_peered":"2024-09-21T07:55:23.348958+0000","last_clean":"2024-09-21T07:55:23.348958+0000","last_became_active":"2024-09-21T07:35:27.877398+0000","last_became_peered":"2024-09-21T07:35:27.877398+0000","last_unstale":"2024-09-21T07:55:23.348958+0000","last_undegraded":"2024-09-21T07:55:23.348958+0000","last_fullsized":"2024-09-21T07:55:23.348958+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:23.348846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:23.348846+0000","last_clean_scrub_stamp":"2024-09-21T07:55:23.348846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:21.802602+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":421,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:18.292131+0000","last_change":"2024-09-21T07:55:18.292131+0000","last_active":"2024-09-21T07:55:18.292131+0000","last_peered":"2024-09-21T07:55:18.292131+0000","last_clean":"2024-09-21T07:55:18.292131+0000","last_became_active":"2024-09-21T07:35:26.876562+0000","last_became_peered":"2024-09-21T07:35:26.876562+0000","last_unstale":"2024-09-21T07:55:18.292131+0000","last_undegraded":"2024-09-21T07:55:18.292131+0000","last_fullsized":"2024-09-21T07:55:18.292131+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:18.292035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:18.292035+0000","last_clean_scrub_stamp":"2024-09-21T07:55:18.292035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:34:57.409573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012252470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":471,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:15.741723+0000","last_change":"2024-09-21T07:55:15.741723+0000","last_active":"2024-09-21T07:55:15.741723+0000","last_peered":"2024-09-21T07:55:15.741723+0000","last_clean":"2024-09-21T07:55:15.741723+0000","last_became_active":"2024-09-21T07:34:23.637210+0000","last_became_peered":"2024-09-21T07:34:23.637210+0000","last_unstale":"2024-09-21T07:55:15.741723+0000","last_undegraded":"2024-09-21T07:55:15.741723+0000","last_fullsized":"2024-09-21T07:55:15.741723+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:15.741623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:15.741623+0000","last_clean_scrub_stamp":"2024-09-21T07:55:15.741623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:49.440992+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.001537093,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":413,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:17.749355+0000","last_change":"2024-09-21T07:55:17.749355+0000","last_active":"2024-09-21T07:55:17.749355+0000","last_peered":"2024-09-21T07:55:17.749355+0000","last_clean":"2024-09-21T07:55:17.749355+0000","last_became_active":"2024-09-21T07:35:27.874360+0000","last_became_peered":"2024-09-21T07:35:27.874360+0000","last_unstale":"2024-09-21T07:55:17.749355+0000","last_undegraded":"2024-09-21T07:55:17.749355+0000","last_fullsized":"2024-09-21T07:55:17.749355+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:17.749251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:17.749251+0000","last_clean_scrub_stamp":"2024-09-21T07:55:17.749251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:03:20.753736+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":418,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.893197+0000","last_change":"2024-09-21T07:55:23.893197+0000","last_active":"2024-09-21T07:55:23.893197+0000","last_peered":"2024-09-21T07:55:23.893197+0000","last_clean":"2024-09-21T07:55:23.893197+0000","last_became_active":"2024-09-21T07:35:26.873691+0000","last_became_peered":"2024-09-21T07:35:26.873691+0000","last_unstale":"2024-09-21T07:55:23.893197+0000","last_undegraded":"2024-09-21T07:55:23.893197+0000","last_fullsized":"2024-09-21T07:55:23.893197+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:23.893086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:23.893086+0000","last_clean_scrub_stamp":"2024-09-21T07:55:23.893086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:11:15.020498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001162464,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":463,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:14.259370+0000","last_change":"2024-09-21T07:55:14.259370+0000","last_active":"2024-09-21T07:55:14.259370+0000","last_peered":"2024-09-21T07:55:14.259370+0000","last_clean":"2024-09-21T07:55:14.259370+0000","last_became_active":"2024-09-21T07:34:23.267133+0000","last_became_peered":"2024-09-21T07:34:23.267133+0000","last_unstale":"2024-09-21T07:55:14.259370+0000","last_undegraded":"2024-09-21T07:55:14.259370+0000","last_fullsized":"2024-09-21T07:55:14.259370+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:14.259277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:14.259277+0000","last_clean_scrub_stamp":"2024-09-21T07:55:14.259277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:43:26.527673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":428,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:19.412158+0000","last_change":"2024-09-21T07:55:19.412158+0000","last_active":"2024-09-21T07:55:19.412158+0000","last_peered":"2024-09-21T07:55:19.412158+0000","last_clean":"2024-09-21T07:55:19.412158+0000","last_became_active":"2024-09-21T07:35:27.864136+0000","last_became_peered":"2024-09-21T07:35:27.864136+0000","last_unstale":"2024-09-21T07:55:19.412158+0000","last_undegraded":"2024-09-21T07:55:19.412158+0000","last_fullsized":"2024-09-21T07:55:19.412158+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:19.412059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:19.412059+0000","last_clean_scrub_stamp":"2024-09-21T07:55:19.412059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:11.853319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":435,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:17.451707+0000","last_change":"2024-09-21T07:55:17.451707+0000","last_active":"2024-09-21T07:55:17.451707+0000","last_peered":"2024-09-21T07:55:17.451707+0000","last_clean":"2024-09-21T07:55:17.451707+0000","last_became_active":"2024-09-21T07:35:26.876890+0000","last_became_peered":"2024-09-21T07:35:26.876890+0000","last_unstale":"2024-09-21T07:55:17.451707+0000","last_undegraded":"2024-09-21T07:55:17.451707+0000","last_fullsized":"2024-09-21T07:55:17.451707+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:17.451594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:17.451594+0000","last_clean_scrub_stamp":"2024-09-21T07:55:17.451594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:44:26.319933+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00064543499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":474,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:16.934142+0000","last_change":"2024-09-21T07:55:16.934142+0000","last_active":"2024-09-21T07:55:16.934142+0000","last_peered":"2024-09-21T07:55:16.934142+0000","last_clean":"2024-09-21T07:55:16.934142+0000","last_became_active":"2024-09-21T07:34:23.274692+0000","last_became_peered":"2024-09-21T07:34:23.274692+0000","last_unstale":"2024-09-21T07:55:16.934142+0000","last_undegraded":"2024-09-21T07:55:16.934142+0000","last_fullsized":"2024-09-21T07:55:16.934142+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:16.934022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:16.934022+0000","last_clean_scrub_stamp":"2024-09-21T07:55:16.934022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:24.639669+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0018051479999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":414,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:22.986345+0000","last_change":"2024-09-21T07:55:22.986345+0000","last_active":"2024-09-21T07:55:22.986345+0000","last_peered":"2024-09-21T07:55:22.986345+0000","last_clean":"2024-09-21T07:55:22.986345+0000","last_became_active":"2024-09-21T07:35:27.877532+0000","last_became_peered":"2024-09-21T07:35:27.877532+0000","last_unstale":"2024-09-21T07:55:22.986345+0000","last_undegraded":"2024-09-21T07:55:22.986345+0000","last_fullsized":"2024-09-21T07:55:22.986345+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:22.986239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:22.986239+0000","last_clean_scrub_stamp":"2024-09-21T07:55:22.986239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:11:17.593106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":421,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:14.352254+0000","last_change":"2024-09-21T07:55:14.352254+0000","last_active":"2024-09-21T07:55:14.352254+0000","last_peered":"2024-09-21T07:55:14.352254+0000","last_clean":"2024-09-21T07:55:14.352254+0000","last_became_active":"2024-09-21T07:35:26.876819+0000","last_became_peered":"2024-09-21T07:35:26.876819+0000","last_unstale":"2024-09-21T07:55:14.352254+0000","last_undegraded":"2024-09-21T07:55:14.352254+0000","last_fullsized":"2024-09-21T07:55:14.352254+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:14.352168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:14.352168+0000","last_clean_scrub_stamp":"2024-09-21T07:55:14.352168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:00.081893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055780599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":471,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:11.672362+0000","last_change":"2024-09-21T07:55:11.672362+0000","last_active":"2024-09-21T07:55:11.672362+0000","last_peered":"2024-09-21T07:55:11.672362+0000","last_clean":"2024-09-21T07:55:11.672362+0000","last_became_active":"2024-09-21T07:34:23.277713+0000","last_became_peered":"2024-09-21T07:34:23.277713+0000","last_unstale":"2024-09-21T07:55:11.672362+0000","last_undegraded":"2024-09-21T07:55:11.672362+0000","last_fullsized":"2024-09-21T07:55:11.672362+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:11.672281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:11.672281+0000","last_clean_scrub_stamp":"2024-09-21T07:55:11.672281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:14.437080+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00059079499999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":414,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:14.968257+0000","last_change":"2024-09-21T07:55:14.968257+0000","last_active":"2024-09-21T07:55:14.968257+0000","last_peered":"2024-09-21T07:55:14.968257+0000","last_clean":"2024-09-21T07:55:14.968257+0000","last_became_active":"2024-09-21T07:35:27.883861+0000","last_became_peered":"2024-09-21T07:35:27.883861+0000","last_unstale":"2024-09-21T07:55:14.968257+0000","last_undegraded":"2024-09-21T07:55:14.968257+0000","last_fullsized":"2024-09-21T07:55:14.968257+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:14.968146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:14.968146+0000","last_clean_scrub_stamp":"2024-09-21T07:55:14.968146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:22.621394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":423,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:14.472464+0000","last_change":"2024-09-21T07:55:14.472464+0000","last_active":"2024-09-21T07:55:14.472464+0000","last_peered":"2024-09-21T07:55:14.472464+0000","last_clean":"2024-09-21T07:55:14.472464+0000","last_became_active":"2024-09-21T07:35:26.872787+0000","last_became_peered":"2024-09-21T07:35:26.872787+0000","last_unstale":"2024-09-21T07:55:14.472464+0000","last_undegraded":"2024-09-21T07:55:14.472464+0000","last_fullsized":"2024-09-21T07:55:14.472464+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:14.472366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:14.472366+0000","last_clean_scrub_stamp":"2024-09-21T07:55:14.472366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:31:47.871306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011157509999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":487,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:15.437022+0000","last_change":"2024-09-21T07:55:15.437022+0000","last_active":"2024-09-21T07:55:15.437022+0000","last_peered":"2024-09-21T07:55:15.437022+0000","last_clean":"2024-09-21T07:55:15.437022+0000","last_became_active":"2024-09-21T07:34:23.259700+0000","last_became_peered":"2024-09-21T07:34:23.259700+0000","last_unstale":"2024-09-21T07:55:15.437022+0000","last_undegraded":"2024-09-21T07:55:15.437022+0000","last_fullsized":"2024-09-21T07:55:15.437022+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:15.436928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:15.436928+0000","last_clean_scrub_stamp":"2024-09-21T07:55:15.436928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:18.201727+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00070522399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":428,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:20.366311+0000","last_change":"2024-09-21T07:55:20.366311+0000","last_active":"2024-09-21T07:55:20.366311+0000","last_peered":"2024-09-21T07:55:20.366311+0000","last_clean":"2024-09-21T07:55:20.366311+0000","last_became_active":"2024-09-21T07:35:27.867339+0000","last_became_peered":"2024-09-21T07:35:27.867339+0000","last_unstale":"2024-09-21T07:55:20.366311+0000","last_undegraded":"2024-09-21T07:55:20.366311+0000","last_fullsized":"2024-09-21T07:55:20.366311+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.366198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.366198+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.366198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:25.406540+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":435,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:16.483582+0000","last_change":"2024-09-21T07:55:16.483582+0000","last_active":"2024-09-21T07:55:16.483582+0000","last_peered":"2024-09-21T07:55:16.483582+0000","last_clean":"2024-09-21T07:55:16.483582+0000","last_became_active":"2024-09-21T07:35:26.877320+0000","last_became_peered":"2024-09-21T07:35:26.877320+0000","last_unstale":"2024-09-21T07:55:16.483582+0000","last_undegraded":"2024-09-21T07:55:16.483582+0000","last_fullsized":"2024-09-21T07:55:16.483582+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:16.483465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:16.483465+0000","last_clean_scrub_stamp":"2024-09-21T07:55:16.483465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:42:49.324501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010244119999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":465,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:15.247684+0000","last_change":"2024-09-21T07:55:15.247684+0000","last_active":"2024-09-21T07:55:15.247684+0000","last_peered":"2024-09-21T07:55:15.247684+0000","last_clean":"2024-09-21T07:55:15.247684+0000","last_became_active":"2024-09-21T07:34:23.635204+0000","last_became_peered":"2024-09-21T07:34:23.635204+0000","last_unstale":"2024-09-21T07:55:15.247684+0000","last_undegraded":"2024-09-21T07:55:15.247684+0000","last_fullsized":"2024-09-21T07:55:15.247684+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:15.247594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:15.247594+0000","last_clean_scrub_stamp":"2024-09-21T07:55:15.247594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:43:07.580359+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00081139199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":416,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:21.365678+0000","last_change":"2024-09-21T07:55:21.365678+0000","last_active":"2024-09-21T07:55:21.365678+0000","last_peered":"2024-09-21T07:55:21.365678+0000","last_clean":"2024-09-21T07:55:21.365678+0000","last_became_active":"2024-09-21T07:35:27.875775+0000","last_became_peered":"2024-09-21T07:35:27.875775+0000","last_unstale":"2024-09-21T07:55:21.365678+0000","last_undegraded":"2024-09-21T07:55:21.365678+0000","last_fullsized":"2024-09-21T07:55:21.365678+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.365569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.365569+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.365569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:02:25.069844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":418,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:21.858132+0000","last_change":"2024-09-21T07:55:21.858132+0000","last_active":"2024-09-21T07:55:21.858132+0000","last_peered":"2024-09-21T07:55:21.858132+0000","last_clean":"2024-09-21T07:55:21.858132+0000","last_became_active":"2024-09-21T07:35:26.876404+0000","last_became_peered":"2024-09-21T07:35:26.876404+0000","last_unstale":"2024-09-21T07:55:21.858132+0000","last_undegraded":"2024-09-21T07:55:21.858132+0000","last_fullsized":"2024-09-21T07:55:21.858132+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.858027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.858027+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.858027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:14.999791+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00063918000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":476,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:14.919667+0000","last_change":"2024-09-21T07:55:14.919603+0000","last_active":"2024-09-21T07:55:14.919667+0000","last_peered":"2024-09-21T07:55:14.919667+0000","last_clean":"2024-09-21T07:55:14.919667+0000","last_became_active":"2024-09-21T07:34:23.635456+0000","last_became_peered":"2024-09-21T07:34:23.635456+0000","last_unstale":"2024-09-21T07:55:14.919667+0000","last_undegraded":"2024-09-21T07:55:14.919667+0000","last_fullsized":"2024-09-21T07:55:14.919667+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:14.919507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:14.919507+0000","last_clean_scrub_stamp":"2024-09-21T07:55:14.919507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:32:27.567184+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0018654419999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":416,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:20.409393+0000","last_change":"2024-09-21T07:55:20.409393+0000","last_active":"2024-09-21T07:55:20.409393+0000","last_peered":"2024-09-21T07:55:20.409393+0000","last_clean":"2024-09-21T07:55:20.409393+0000","last_became_active":"2024-09-21T07:35:27.870336+0000","last_became_peered":"2024-09-21T07:35:27.870336+0000","last_unstale":"2024-09-21T07:55:20.409393+0000","last_undegraded":"2024-09-21T07:55:20.409393+0000","last_fullsized":"2024-09-21T07:55:20.409393+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.409258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.409258+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.409258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:20:44.799712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":421,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:13.371529+0000","last_change":"2024-09-21T07:55:13.371529+0000","last_active":"2024-09-21T07:55:13.371529+0000","last_peered":"2024-09-21T07:55:13.371529+0000","last_clean":"2024-09-21T07:55:13.371529+0000","last_became_active":"2024-09-21T07:35:26.872046+0000","last_became_peered":"2024-09-21T07:35:26.872046+0000","last_unstale":"2024-09-21T07:55:13.371529+0000","last_undegraded":"2024-09-21T07:55:13.371529+0000","last_fullsized":"2024-09-21T07:55:13.371529+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:13.371444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:13.371444+0000","last_clean_scrub_stamp":"2024-09-21T07:55:13.371444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:35:58.286096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098384399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":471,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:18.851104+0000","last_change":"2024-09-21T07:55:18.851104+0000","last_active":"2024-09-21T07:55:18.851104+0000","last_peered":"2024-09-21T07:55:18.851104+0000","last_clean":"2024-09-21T07:55:18.851104+0000","last_became_active":"2024-09-21T07:34:23.270128+0000","last_became_peered":"2024-09-21T07:34:23.270128+0000","last_unstale":"2024-09-21T07:55:18.851104+0000","last_undegraded":"2024-09-21T07:55:18.851104+0000","last_fullsized":"2024-09-21T07:55:18.851104+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:18.851009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:18.851009+0000","last_clean_scrub_stamp":"2024-09-21T07:55:18.851009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:01:46.405921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00176651,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":416,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:22.364480+0000","last_change":"2024-09-21T07:55:22.364480+0000","last_active":"2024-09-21T07:55:22.364480+0000","last_peered":"2024-09-21T07:55:22.364480+0000","last_clean":"2024-09-21T07:55:22.364480+0000","last_became_active":"2024-09-21T07:35:27.877434+0000","last_became_peered":"2024-09-21T07:35:27.877434+0000","last_unstale":"2024-09-21T07:55:22.364480+0000","last_undegraded":"2024-09-21T07:55:22.364480+0000","last_fullsized":"2024-09-21T07:55:22.364480+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:22.364381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:22.364381+0000","last_clean_scrub_stamp":"2024-09-21T07:55:22.364381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:14:59.231979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":420,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:15.693932+0000","last_change":"2024-09-21T07:55:15.693932+0000","last_active":"2024-09-21T07:55:15.693932+0000","last_peered":"2024-09-21T07:55:15.693932+0000","last_clean":"2024-09-21T07:55:15.693932+0000","last_became_active":"2024-09-21T07:35:26.870613+0000","last_became_peered":"2024-09-21T07:35:26.870613+0000","last_unstale":"2024-09-21T07:55:15.693932+0000","last_undegraded":"2024-09-21T07:55:15.693932+0000","last_fullsized":"2024-09-21T07:55:15.693932+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:15.693835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:15.693835+0000","last_clean_scrub_stamp":"2024-09-21T07:55:15.693835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:45.483864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059842100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":472,"reported_epoch":269,"state":"active+clean","last_fresh":"2024-09-21T07:55:11.978090+0000","last_change":"2024-09-21T07:55:11.978090+0000","last_active":"2024-09-21T07:55:11.978090+0000","last_peered":"2024-09-21T07:55:11.978090+0000","last_clean":"2024-09-21T07:55:11.978090+0000","last_became_active":"2024-09-21T07:34:23.260942+0000","last_became_peered":"2024-09-21T07:34:23.260942+0000","last_unstale":"2024-09-21T07:55:11.978090+0000","last_undegraded":"2024-09-21T07:55:11.978090+0000","last_fullsized":"2024-09-21T07:55:11.978090+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:11.978008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:11.978008+0000","last_clean_scrub_stamp":"2024-09-21T07:55:11.978008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:17.665409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060553899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509712,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71180,"kb_used_data":3900,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94300660,"statfs":{"total":96636764160,"available":96563875840,"internally_reserved":0,"allocated":3993600,"data_stored":2441691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509714,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74760,"kb_used_data":3320,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297080,"statfs":{"total":96636764160,"available":96560209920,"internally_reserved":0,"allocated":3399680,"data_stored":1851323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509712,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52552,"kb_used_data":3320,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319288,"statfs":{"total":96636764160,"available":96582950912,"internally_reserved":0,"allocated":3399680,"data_stored":1851323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509713,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30344,"kb_used_data":3320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341496,"statfs":{"total":96636764160,"available":96605691904,"internally_reserved":0,"allocated":3399680,"data_stored":1851323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509714,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48392,"kb_used_data":3320,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323448,"statfs":{"total":96636764160,"available":96587210752,"internally_reserved":0,"allocated":3399680,"data_stored":1851323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509714,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57292,"kb_used_data":3900,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314548,"statfs":{"total":96636764160,"available":96578097152,"internally_reserved":0,"allocated":3993600,"data_stored":2441691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542418,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56712,"kb_used_data":3320,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315128,"statfs":{"total":96636764160,"available":96578691072,"internally_reserved":0,"allocated":3399680,"data_stored":1851323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542417,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78920,"kb_used_data":3320,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94292920,"statfs":{"total":96636764160,"available":96555950080,"internally_reserved":0,"allocated":3399680,"data_stored":1851323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:55:33.635 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-21T07:55:33.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.890+0000 7f7dc7e00640 1 -- 172.21.15.39:0/3218914800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7dc8160250 msgr2=0x7f7dc815d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:33.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.890+0000 7f7dc7e00640 1 --2- 172.21.15.39:0/3218914800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7dc8160250 0x7f7dc815d570 secure :-1 s=READY pgs=2931 cs=0 l=1 rev1=1 crypto rx=0x7f7db8005fd0 tx=0x7f7db802a530 comp rx=0 tx=0).stop 2024-09-21T07:55:33.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.890+0000 7f7dc7e00640 1 -- 172.21.15.39:0/3218914800 shutdown_connections 2024-09-21T07:55:33.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.890+0000 7f7dc7e00640 1 --2- 172.21.15.39:0/3218914800 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7dc8160660 0x7f7dc80cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:33.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.890+0000 7f7dc7e00640 1 --2- 172.21.15.39:0/3218914800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7dc8160250 0x7f7dc815d570 unknown :-1 s=CLOSED pgs=2931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:33.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.890+0000 7f7dc7e00640 1 -- 172.21.15.39:0/3218914800 >> 172.21.15.39:0/3218914800 conn(0x7f7dc806cfc0 msgr2=0x7f7dc806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:33.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.890+0000 7f7dc7e00640 1 -- 172.21.15.39:0/3218914800 shutdown_connections 2024-09-21T07:55:33.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.890+0000 7f7dc7e00640 1 -- 172.21.15.39:0/3218914800 wait complete. 2024-09-21T07:55:33.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.893+0000 7f7dc7e00640 1 Processor -- start 2024-09-21T07:55:33.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.893+0000 7f7dc7e00640 1 -- start start 2024-09-21T07:55:33.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.894+0000 7f7dc7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7dc80cfc50 0x7f7dc8145780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:33.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.894+0000 7f7dc7e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7dc8160250 0x7f7dc81f4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:33.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.894+0000 7f7dc7e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7dc8160660 0x7f7dc81f4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:33.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.894+0000 7f7dc7e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc80cce00 con 0x7f7dc80cfc50 2024-09-21T07:55:33.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.894+0000 7f7dc7e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc80ccc80 con 0x7f7dc8160250 2024-09-21T07:55:33.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.894+0000 7f7dc7e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f7dc80ccf80 con 0x7f7dc8160660 2024-09-21T07:55:33.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.895+0000 7f7dc6a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7dc80cfc50 0x7f7dc8145780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:33.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.895+0000 7f7dc6a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7dc80cfc50 0x7f7dc8145780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38878/0 (socket says 172.21.15.39:38878) 2024-09-21T07:55:33.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.895+0000 7f7dc6a00640 1 -- 172.21.15.39:0/4281838507 learned_addr learned my addr 172.21.15.39:0/4281838507 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:55:33.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.895+0000 7f7dc7400640 1 --2- 172.21.15.39:0/4281838507 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7dc8160660 0x7f7dc81f4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:33.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.895+0000 7f7dbe000640 1 --2- 172.21.15.39:0/4281838507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7dc8160250 0x7f7dc81f4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:33.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.895+0000 7f7dc6a00640 1 -- 172.21.15.39:0/4281838507 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7dc8160660 msgr2=0x7f7dc81f4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:33.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.895+0000 7f7dc6a00640 1 --2- 172.21.15.39:0/4281838507 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7dc8160660 0x7f7dc81f4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:33.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.895+0000 7f7dc6a00640 1 -- 172.21.15.39:0/4281838507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7dc8160250 msgr2=0x7f7dc81f4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:33.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.895+0000 7f7dc6a00640 1 --2- 172.21.15.39:0/4281838507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7dc8160250 0x7f7dc81f4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:33.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.895+0000 7f7dc6a00640 1 -- 172.21.15.39:0/4281838507 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dc80b1930 con 0x7f7dc80cfc50 2024-09-21T07:55:33.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.895+0000 7f7dc6a00640 1 --2- 172.21.15.39:0/4281838507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7dc80cfc50 0x7f7dc8145780 secure :-1 s=READY pgs=2932 cs=0 l=1 rev1=1 crypto rx=0x7f7db8034040 tx=0x7f7db8003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:33.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.896+0000 7f7dbf400640 1 -- 172.21.15.39:0/4281838507 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db803e030 con 0x7f7dc80cfc50 2024-09-21T07:55:33.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.896+0000 7f7dbf400640 1 -- 172.21.15.39:0/4281838507 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7db8003e00 con 0x7f7dc80cfc50 2024-09-21T07:55:33.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.896+0000 7f7dbf400640 1 -- 172.21.15.39:0/4281838507 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db8035040 con 0x7f7dc80cfc50 2024-09-21T07:55:33.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.896+0000 7f7dc7e00640 1 -- 172.21.15.39:0/4281838507 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dc80b1c10 con 0x7f7dc80cfc50 2024-09-21T07:55:33.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.896+0000 7f7dc7e00640 1 -- 172.21.15.39:0/4281838507 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dc80b21d0 con 0x7f7dc80cfc50 2024-09-21T07:55:33.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.897+0000 7f7dc7e00640 1 -- 172.21.15.39:0/4281838507 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dc80cd430 con 0x7f7dc80cfc50 2024-09-21T07:55:33.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.898+0000 7f7dbf400640 1 -- 172.21.15.39:0/4281838507 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7db8041030 con 0x7f7dc80cfc50 2024-09-21T07:55:33.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.899+0000 7f7dbf400640 1 --2- 172.21.15.39:0/4281838507 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7da4078b90 0x7f7da407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:33.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.899+0000 7f7dbf400640 1 -- 172.21.15.39:0/4281838507 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(269..269 src has 1..269) ==== 6278+0+0 (secure 0 0 0) 0x7f7db80bb820 con 0x7f7dc80cfc50 2024-09-21T07:55:33.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.899+0000 7f7dbe000640 1 --2- 172.21.15.39:0/4281838507 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7da4078b90 0x7f7da407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:33.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.900+0000 7f7dbe000640 1 --2- 172.21.15.39:0/4281838507 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7da4078b90 0x7f7da407b0a0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f7db0005fd0 tx=0x7f7db0005850 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:33.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:33.904+0000 7f7dbf400640 1 -- 172.21.15.39:0/4281838507 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7db8083980 con 0x7f7dc80cfc50 2024-09-21T07:55:34.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.105+0000 7f7dc7e00640 1 -- 172.21.15.39:0/4281838507 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f7dc8073190 con 0x7f7dc80cfc50 2024-09-21T07:55:34.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.109+0000 7f7dbf400640 1 -- 172.21.15.39:0/4281838507 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v59)=0 v59) ==== 123+0+0 (secure 0 0 0) 0x7f7db8083280 con 0x7f7dc80cfc50 2024-09-21T07:55:34.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.110+0000 7f7dbf400640 1 -- 172.21.15.39:0/4281838507 <== mon.0 v2:172.21.15.39:3300/0 8 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f7db8082d90 con 0x7f7dc80cfc50 2024-09-21T07:55:34.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.112+0000 7f7dc7e00640 1 -- 172.21.15.39:0/4281838507 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7da4078b90 msgr2=0x7f7da407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:34.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.112+0000 7f7dc7e00640 1 --2- 172.21.15.39:0/4281838507 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7da4078b90 0x7f7da407b0a0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f7db0005fd0 tx=0x7f7db0005850 comp rx=0 tx=0).stop 2024-09-21T07:55:34.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.112+0000 7f7dc7e00640 1 -- 172.21.15.39:0/4281838507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7dc80cfc50 msgr2=0x7f7dc8145780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:34.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.112+0000 7f7dc7e00640 1 --2- 172.21.15.39:0/4281838507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7dc80cfc50 0x7f7dc8145780 secure :-1 s=READY pgs=2932 cs=0 l=1 rev1=1 crypto rx=0x7f7db8034040 tx=0x7f7db8003910 comp rx=0 tx=0).stop 2024-09-21T07:55:34.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.112+0000 7f7dc7e00640 1 -- 172.21.15.39:0/4281838507 shutdown_connections 2024-09-21T07:55:34.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.112+0000 7f7dc7e00640 1 --2- 172.21.15.39:0/4281838507 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f7da4078b90 0x7f7da407b0a0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:34.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.112+0000 7f7dc7e00640 1 --2- 172.21.15.39:0/4281838507 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f7dc8160660 0x7f7dc81f4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:34.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.112+0000 7f7dc7e00640 1 --2- 172.21.15.39:0/4281838507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7dc8160250 0x7f7dc81f4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:34.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.112+0000 7f7dc7e00640 1 --2- 172.21.15.39:0/4281838507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7dc80cfc50 0x7f7dc8145780 unknown :-1 s=CLOSED pgs=2932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:34.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.112+0000 7f7dc7e00640 1 -- 172.21.15.39:0/4281838507 >> 172.21.15.39:0/4281838507 conn(0x7f7dc806cfc0 msgr2=0x7f7dc8068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:34.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.112+0000 7f7dc7e00640 1 -- 172.21.15.39:0/4281838507 shutdown_connections 2024-09-21T07:55:34.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.112+0000 7f7dc7e00640 1 -- 172.21.15.39:0/4281838507 wait complete. 2024-09-21T07:55:34.139 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T07:55:34.139 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:55:34.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.365+0000 7f6a13e00640 1 -- 172.21.15.39:0/3177944123 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a140b5a90 msgr2=0x7f6a140d1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:34.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.365+0000 7f6a13e00640 1 --2- 172.21.15.39:0/3177944123 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a140b5a90 0x7f6a140d1ac0 secure :-1 s=READY pgs=2934 cs=0 l=1 rev1=1 crypto rx=0x7f6a04009820 tx=0x7f6a0402a5a0 comp rx=0 tx=0).stop 2024-09-21T07:55:34.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.365+0000 7f6a13e00640 1 -- 172.21.15.39:0/3177944123 shutdown_connections 2024-09-21T07:55:34.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.365+0000 7f6a13e00640 1 --2- 172.21.15.39:0/3177944123 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a140b5a90 0x7f6a140d1ac0 unknown :-1 s=CLOSED pgs=2934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:34.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.365+0000 7f6a13e00640 1 -- 172.21.15.39:0/3177944123 >> 172.21.15.39:0/3177944123 conn(0x7f6a1409d7b0 msgr2=0x7f6a140d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:34.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.365+0000 7f6a13e00640 1 -- 172.21.15.39:0/3177944123 shutdown_connections 2024-09-21T07:55:34.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.365+0000 7f6a13e00640 1 -- 172.21.15.39:0/3177944123 wait complete. 2024-09-21T07:55:34.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.367+0000 7f6a13e00640 1 Processor -- start 2024-09-21T07:55:34.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.367+0000 7f6a13e00640 1 -- start start 2024-09-21T07:55:34.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.369+0000 7f6a13e00640 1 --2- >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a140b3b60 0x7f6a141f5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:34.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.369+0000 7f6a13e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a140b5a90 0x7f6a1409e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:34.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.369+0000 7f6a13e00640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a140d0e70 0x7f6a1409e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:34.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.369+0000 7f6a13e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6a140b4800 con 0x7f6a140b5a90 2024-09-21T07:55:34.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.369+0000 7f6a13e00640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6a140b4680 con 0x7f6a140d0e70 2024-09-21T07:55:34.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.369+0000 7f6a13e00640 1 -- --> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] -- mon_getmap magic: 0 -- 0x7f6a140b4980 con 0x7f6a140b3b60 2024-09-21T07:55:34.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.369+0000 7f6a0be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a140b5a90 0x7f6a1409e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:34.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.369+0000 7f6a0be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a140b5a90 0x7f6a1409e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38902/0 (socket says 172.21.15.39:38902) 2024-09-21T07:55:34.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.369+0000 7f6a0be00640 1 -- 172.21.15.39:0/1000763159 learned_addr learned my addr 172.21.15.39:0/1000763159 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-09-21T07:55:34.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.369+0000 7f6a12a00640 1 --2- 172.21.15.39:0/1000763159 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a140b3b60 0x7f6a141f5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:34.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.369+0000 7f6a13400640 1 --2- 172.21.15.39:0/1000763159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a140d0e70 0x7f6a1409e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:34.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.370+0000 7f6a0be00640 1 -- 172.21.15.39:0/1000763159 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a140b3b60 msgr2=0x7f6a141f5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:34.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.370+0000 7f6a0be00640 1 --2- 172.21.15.39:0/1000763159 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a140b3b60 0x7f6a141f5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:34.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.370+0000 7f6a0be00640 1 -- 172.21.15.39:0/1000763159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a140d0e70 msgr2=0x7f6a1409e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:34.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.370+0000 7f6a0be00640 1 --2- 172.21.15.39:0/1000763159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a140d0e70 0x7f6a1409e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:34.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.370+0000 7f6a0be00640 1 -- 172.21.15.39:0/1000763159 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a141f6570 con 0x7f6a140b5a90 2024-09-21T07:55:34.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.370+0000 7f6a12a00640 1 --2- 172.21.15.39:0/1000763159 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a140b3b60 0x7f6a141f5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:55:34.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.370+0000 7f6a0be00640 1 --2- 172.21.15.39:0/1000763159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a140b5a90 0x7f6a1409e120 secure :-1 s=READY pgs=2935 cs=0 l=1 rev1=1 crypto rx=0x7f69fc00c3c0 tx=0x7f69fc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:34.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.370+0000 7f6a0aa00640 1 -- 172.21.15.39:0/1000763159 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc017070 con 0x7f6a140b5a90 2024-09-21T07:55:34.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.370+0000 7f6a0aa00640 1 -- 172.21.15.39:0/1000763159 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f69fc0035f0 con 0x7f6a140b5a90 2024-09-21T07:55:34.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.370+0000 7f6a0aa00640 1 -- 172.21.15.39:0/1000763159 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc003aa0 con 0x7f6a140b5a90 2024-09-21T07:55:34.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.370+0000 7f6a13e00640 1 -- 172.21.15.39:0/1000763159 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a141f6850 con 0x7f6a140b5a90 2024-09-21T07:55:34.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.370+0000 7f6a13e00640 1 -- 172.21.15.39:0/1000763159 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a141f6e10 con 0x7f6a140b5a90 2024-09-21T07:55:34.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.371+0000 7f6a13e00640 1 -- 172.21.15.39:0/1000763159 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a140d1ac0 con 0x7f6a140b5a90 2024-09-21T07:55:34.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.373+0000 7f6a0aa00640 1 -- 172.21.15.39:0/1000763159 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f69fc01a070 con 0x7f6a140b5a90 2024-09-21T07:55:34.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.374+0000 7f6a0aa00640 1 --2- 172.21.15.39:0/1000763159 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f69f0078b90 0x7f69f007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:34.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.374+0000 7f6a0aa00640 1 -- 172.21.15.39:0/1000763159 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(269..269 src has 1..269) ==== 6278+0+0 (secure 0 0 0) 0x7f69fc09a3d0 con 0x7f6a140b5a90 2024-09-21T07:55:34.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.374+0000 7f6a12a00640 1 --2- 172.21.15.39:0/1000763159 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f69f0078b90 0x7f69f007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:34.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.375+0000 7f6a12a00640 1 --2- 172.21.15.39:0/1000763159 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f69f0078b90 0x7f69f007b0a0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f6a04034000 tx=0x7f6a040046c0 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:34.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.378+0000 7f6a0aa00640 1 -- 172.21.15.39:0/1000763159 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f69fc0625c0 con 0x7f6a140b5a90 2024-09-21T07:55:34.494 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:55:34.499 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:55:34.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.615+0000 7f6a13e00640 1 -- 172.21.15.39:0/1000763159 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a141f6300 con 0x7f6a140b5a90 2024-09-21T07:55:34.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.615+0000 7f6a0aa00640 1 -- 172.21.15.39:0/1000763159 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 312 v312) ==== 95+0+3922 (secure 0 0 0) 0x7f69fc061ec0 con 0x7f6a140b5a90 2024-09-21T07:55:34.616 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-21T07:55:34.616 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":312,"btime":"2024-09-21T07:54:57:601574+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9725,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/1018044536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6835","nonce":1018044536},{"type":"v1","addr":"172.21.15.148:6837","nonce":1018044536}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":305},{"gid":11125,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6836/1771887993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":1771887993},{"type":"v1","addr":"172.21.15.148:6836","nonce":1771887993}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":304},{"gid":11131,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/649113662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":649113662},{"type":"v1","addr":"172.21.15.39:6835","nonce":649113662}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":305},{"gid":12537,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2557992328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2557992328},{"type":"v1","addr":"172.21.15.39:6834","nonce":2557992328}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":312}],"filesystems":[]} 2024-09-21T07:55:34.617 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 312 2024-09-21T07:55:34.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.618+0000 7f6a13e00640 1 -- 172.21.15.39:0/1000763159 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f69f0078b90 msgr2=0x7f69f007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:34.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.618+0000 7f6a13e00640 1 --2- 172.21.15.39:0/1000763159 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f69f0078b90 0x7f69f007b0a0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f6a04034000 tx=0x7f6a040046c0 comp rx=0 tx=0).stop 2024-09-21T07:55:34.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.618+0000 7f6a13e00640 1 -- 172.21.15.39:0/1000763159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a140b5a90 msgr2=0x7f6a1409e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:34.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.618+0000 7f6a13e00640 1 --2- 172.21.15.39:0/1000763159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a140b5a90 0x7f6a1409e120 secure :-1 s=READY pgs=2935 cs=0 l=1 rev1=1 crypto rx=0x7f69fc00c3c0 tx=0x7f69fc00c880 comp rx=0 tx=0).stop 2024-09-21T07:55:34.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.618+0000 7f6a13e00640 1 -- 172.21.15.39:0/1000763159 shutdown_connections 2024-09-21T07:55:34.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.618+0000 7f6a13e00640 1 --2- 172.21.15.39:0/1000763159 >> [v2:172.21.15.148:6800/896204996,v1:172.21.15.148:6801/896204996] conn(0x7f69f0078b90 0x7f69f007b0a0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:34.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.618+0000 7f6a13e00640 1 --2- 172.21.15.39:0/1000763159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a140d0e70 0x7f6a1409e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:34.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.618+0000 7f6a13e00640 1 --2- 172.21.15.39:0/1000763159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a140b5a90 0x7f6a1409e120 unknown :-1 s=CLOSED pgs=2935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:34.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.618+0000 7f6a13e00640 1 --2- 172.21.15.39:0/1000763159 >> [v2:172.21.15.148:3301/0,v1:172.21.15.148:6790/0] conn(0x7f6a140b3b60 0x7f6a141f5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:34.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.618+0000 7f6a13e00640 1 -- 172.21.15.39:0/1000763159 >> 172.21.15.39:0/1000763159 conn(0x7f6a1409d7b0 msgr2=0x7f6a140d4360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:34.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.618+0000 7f6a13e00640 1 -- 172.21.15.39:0/1000763159 shutdown_connections 2024-09-21T07:55:34.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-21T07:55:34.618+0000 7f6a13e00640 1 -- 172.21.15.39:0/1000763159 wait complete. 2024-09-21T07:55:34.645 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-21T07:55:34.646 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-21T07:55:34.646 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T07:55:34.646 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:55:34.706 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T07:55:34.706 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T07:55:34.706 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:55:34.770 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T07:55:34.770 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T07:55:34.770 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:55:34.795 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T07:55:34.795 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T07:55:34.795 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:55:34.860 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T07:55:34.860 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-21T07:55:34.861 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-21T07:55:34.861 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:55:34.916 INFO:tasks.ceph.osd.0:Stopped 2024-09-21T07:55:34.916 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-21T07:55:34.916 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:55:34.988 INFO:tasks.ceph.osd.1:Stopped 2024-09-21T07:55:34.988 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-21T07:55:34.988 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:55:35.046 INFO:tasks.ceph.osd.2:Stopped 2024-09-21T07:55:35.046 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-21T07:55:35.046 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:55:35.117 INFO:tasks.ceph.osd.3:Stopped 2024-09-21T07:55:35.117 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-21T07:55:35.117 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:55:35.175 INFO:tasks.ceph.osd.4:Stopped 2024-09-21T07:55:35.175 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-21T07:55:35.175 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:55:35.238 INFO:tasks.ceph.osd.5:Stopped 2024-09-21T07:55:35.238 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-21T07:55:35.239 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:55:35.309 INFO:tasks.ceph.osd.6:Stopped 2024-09-21T07:55:35.309 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-21T07:55:35.309 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:55:35.387 INFO:tasks.ceph.osd.7:Stopped 2024-09-21T07:55:35.387 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-21T07:55:35.388 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-21T07:55:35.388 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:55:35.462 INFO:tasks.ceph.mgr.y:Stopped 2024-09-21T07:55:35.462 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-21T07:55:35.462 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:55:35.536 INFO:tasks.ceph.mgr.x:Stopped 2024-09-21T07:55:35.536 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-21T07:55:35.536 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-21T07:55:35.537 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:55:35.569 INFO:tasks.ceph.mon.a:Stopped 2024-09-21T07:55:35.570 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-21T07:55:35.570 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:55:35.598 INFO:tasks.ceph.mon.b:Stopped 2024-09-21T07:55:35.598 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-21T07:55:35.598 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:55:35.667 INFO:tasks.ceph.mon.c:Stopped 2024-09-21T07:55:35.667 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-21T07:55:35.667 DEBUG:teuthology.orchestra.run.smithi039:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'evicting unresponsive client' | egrep -v RECENT_CRASH | head -n 1 2024-09-21T07:55:35.747 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T07:55:35.747 DEBUG:teuthology.orchestra.run.smithi039:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-21T07:55:36.055 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T07:55:36.055 DEBUG:teuthology.orchestra.run.smithi039:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-21T07:55:36.168 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T07:55:36.168 DEBUG:teuthology.orchestra.run.smithi039:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-21T07:55:36.275 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T07:55:36.275 DEBUG:teuthology.orchestra.run.smithi039:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-21T07:55:36.419 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi148.front.sepia.ceph.com 2024-09-21T07:55:36.419 DEBUG:teuthology.orchestra.run.smithi148:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-21T07:55:36.628 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi148.front.sepia.ceph.com 2024-09-21T07:55:36.629 DEBUG:teuthology.orchestra.run.smithi148:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-21T07:55:36.769 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi148.front.sepia.ceph.com 2024-09-21T07:55:36.769 DEBUG:teuthology.orchestra.run.smithi148:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-21T07:55:36.895 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi148.front.sepia.ceph.com 2024-09-21T07:55:36.895 DEBUG:teuthology.orchestra.run.smithi148:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-21T07:55:37.054 INFO:tasks.ceph:Archiving mon data... 2024-09-21T07:55:37.056 DEBUG:teuthology.misc:Transferring archived files from smithi039:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913602/data/mon.a.tgz 2024-09-21T07:55:37.056 DEBUG:teuthology.orchestra.run.smithi039:> mktemp 2024-09-21T07:55:37.092 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/tmp.Ih1qiwdcPD 2024-09-21T07:55:37.093 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.Ih1qiwdcPD 2024-09-21T07:55:37.332 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0666 /tmp/tmp.Ih1qiwdcPD 2024-09-21T07:55:37.426 DEBUG:teuthology.orchestra.remote:smithi039:/tmp/tmp.Ih1qiwdcPD is 629KB 2024-09-21T07:55:37.519 DEBUG:teuthology.orchestra.run.smithi039:> rm -fr /tmp/tmp.Ih1qiwdcPD 2024-09-21T07:55:37.553 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913602/data/mon.b.tgz 2024-09-21T07:55:37.553 DEBUG:teuthology.orchestra.run.smithi148:> mktemp 2024-09-21T07:55:37.589 INFO:teuthology.orchestra.run.smithi148.stdout:/tmp/tmp.HLmymCIDmO 2024-09-21T07:55:37.589 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.HLmymCIDmO 2024-09-21T07:55:37.850 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0666 /tmp/tmp.HLmymCIDmO 2024-09-21T07:55:37.943 DEBUG:teuthology.orchestra.remote:smithi148:/tmp/tmp.HLmymCIDmO is 646KB 2024-09-21T07:55:38.034 DEBUG:teuthology.orchestra.run.smithi148:> rm -fr /tmp/tmp.HLmymCIDmO 2024-09-21T07:55:38.069 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913602/data/mon.c.tgz 2024-09-21T07:55:38.069 DEBUG:teuthology.orchestra.run.smithi148:> mktemp 2024-09-21T07:55:38.145 INFO:teuthology.orchestra.run.smithi148.stdout:/tmp/tmp.7Tm1CKfJHq 2024-09-21T07:55:38.145 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.7Tm1CKfJHq 2024-09-21T07:55:38.375 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0666 /tmp/tmp.7Tm1CKfJHq 2024-09-21T07:55:38.469 DEBUG:teuthology.orchestra.remote:smithi148:/tmp/tmp.7Tm1CKfJHq is 646KB 2024-09-21T07:55:38.560 DEBUG:teuthology.orchestra.run.smithi148:> rm -fr /tmp/tmp.7Tm1CKfJHq 2024-09-21T07:55:38.596 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-21T07:55:38.596 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T07:55:38.600 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T07:55:38.728 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-21T07:55:38.763 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-21T07:55:38.764 INFO:tasks.ceph:Archiving crash dumps... 2024-09-21T07:55:38.765 DEBUG:teuthology.misc:Transferring archived files from smithi039:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913602/remote/smithi039/crash 2024-09-21T07:55:38.766 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T07:55:38.994 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913602/remote/smithi148/crash 2024-09-21T07:55:38.994 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T07:55:39.172 INFO:tasks.ceph:Shutting down logrotate 2024-09-21T07:55:39.173 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:55:39.178 DEBUG:teuthology.orchestra.run.smithi148:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:55:39.319 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-21T07:55:39.375 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-21T07:55:39.431 INFO:tasks.ceph:Compressing logs... 2024-09-21T07:55:39.431 DEBUG:teuthology.orchestra.run.smithi039:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T07:55:39.435 DEBUG:teuthology.orchestra.run.smithi148:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T07:55:39.485 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107891.log 2024-09-21T07:55:39.485 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-21T07:55:39.486 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.107891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67985.log 2024-09-21T07:55:39.487 INFO:teuthology.orchestra.run.smithi039.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.107891.log.gz 2024-09-21T07:55:39.487 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59656.log 2024-09-21T07:55:39.488 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.67985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57284.log 2024-09-21T07:55:39.489 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.67985.log.gz 2024-09-21T07:55:39.489 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41367.log 2024-09-21T07:55:39.489 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.59656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45305.log 2024-09-21T07:55:39.491 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-21T07:55:39.491 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.1.41367.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-21T07:55:39.491 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41367.log.gz 2024-09-21T07:55:39.492 INFO:teuthology.orchestra.run.smithi039.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.59656.log.gz 2024-09-21T07:55:39.492 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.57284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51552.log 2024-09-21T07:55:39.492 INFO:teuthology.orchestra.run.smithi039.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.57284.log.gz 2024-09-21T07:55:39.492 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.45305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52875.log 2024-09-21T07:55:39.492 INFO:teuthology.orchestra.run.smithi039.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45305.log.gz 2024-09-21T07:55:39.492 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51552.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51552.log.gz 2024-09-21T07:55:39.492 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110013.log 2024-09-21T07:55:39.494 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108703.log 2024-09-21T07:55:39.494 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52875.log.gz 2024-09-21T07:55:39.494 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.110013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54960.log 2024-09-21T07:55:39.494 INFO:teuthology.orchestra.run.smithi039.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.110013.log.gz 2024-09-21T07:55:39.495 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-21T07:55:39.495 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35985.log 2024-09-21T07:55:39.495 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35905.log 2024-09-21T07:55:39.495 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.35985.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.35985.log.gz 2024-09-21T07:55:39.495 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-21T07:55:39.495 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35785.log 2024-09-21T07:55:39.496 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.35905.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.35905.log.gz 2024-09-21T07:55:39.497 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35945.log 2024-09-21T07:55:39.498 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.108703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36921.log 2024-09-21T07:55:39.498 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.108703.log.gz 2024-09-21T07:55:39.498 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54960.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54960.log.gz 2024-09-21T07:55:39.498 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59142.log 2024-09-21T07:55:39.498 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67385.log 2024-09-21T07:55:39.498 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36921.log.gz 2024-09-21T07:55:39.498 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.59142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91772.log 2024-09-21T07:55:39.498 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.59142.log.gz 2024-09-21T07:55:39.498 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.67385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.72580.log 2024-09-21T07:55:39.500 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.35785.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-21T07:55:39.500 INFO:teuthology.orchestra.run.smithi148.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.35785.log.gz 2024-09-21T07:55:39.500 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.35945.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-21T07:55:39.500 INFO:teuthology.orchestra.run.smithi148.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.35945.log.gz 2024-09-21T07:55:39.500 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-21T07:55:39.501 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67385.log.gz 2024-09-21T07:55:39.501 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.91772.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.91772.log.gz 2024-09-21T07:55:39.501 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44012.log 2024-09-21T07:55:39.501 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.0.72580.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.72580.log.gz 2024-09-21T07:55:39.501 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109776.log 2024-09-21T07:55:39.501 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.44012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54792.log 2024-09-21T07:55:39.501 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44012.log.gz 2024-09-21T07:55:39.502 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-21T07:55:39.503 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.109776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53762.log 2024-09-21T07:55:39.503 INFO:teuthology.orchestra.run.smithi039.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.109776.log.gz 2024-09-21T07:55:39.503 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85519.log 2024-09-21T07:55:39.503 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54792.log.gz 2024-09-21T07:55:39.503 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53762.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53762.log.gz 2024-09-21T07:55:39.503 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35881.log 2024-09-21T07:55:39.504 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-osd.5.log: /var/log/ceph/ceph.log: 94.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-21T07:55:39.504 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35865.log 2024-09-21T07:55:39.504 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.85519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94590.log 2024-09-21T07:55:39.504 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.85519.log.gz 2024-09-21T07:55:39.505 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58474.log 2024-09-21T07:55:39.505 INFO:teuthology.orchestra.run.smithi039.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.35881.log.gz 2024-09-21T07:55:39.506 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.94590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.74892.log 2024-09-21T07:55:39.506 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.94590.log.gz 2024-09-21T07:55:39.507 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.35865.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.35865.log.gz 2024-09-21T07:55:39.507 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36145.log 2024-09-21T07:55:39.507 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.58474.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58474.log.gz 2024-09-21T07:55:39.507 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.57664.log 2024-09-21T07:55:39.507 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.0.74892.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.74892.log.gz 2024-09-21T07:55:39.508 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61216.log 2024-09-21T07:55:39.508 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.0.57664.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.57664.log.gz 2024-09-21T07:55:39.508 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62299.log 2024-09-21T07:55:39.509 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.61216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54666.log 2024-09-21T07:55:39.509 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61216.log.gz 2024-09-21T07:55:39.510 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.62299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103923.log 2024-09-21T07:55:39.511 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62299.log.gz 2024-09-21T07:55:39.511 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.36145.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.36145.log.gz 2024-09-21T07:55:39.511 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-21T07:55:39.512 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43690.log 2024-09-21T07:55:39.512 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54666.log.gz 2024-09-21T07:55:39.512 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.103923.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.103923.log.gz 2024-09-21T07:55:39.512 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45639.log 2024-09-21T07:55:39.513 INFO:teuthology.orchestra.run.smithi148.stderr: 92.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-21T07:55:39.513 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-21T07:55:39.513 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74410.log 2024-09-21T07:55:39.513 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43690.log.gz 2024-09-21T07:55:39.514 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.45639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94846.log 2024-09-21T07:55:39.514 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45639.log.gz 2024-09-21T07:55:39.514 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.74410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55002.log 2024-09-21T07:55:39.514 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74410.log.gz 2024-09-21T07:55:39.515 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.94846.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.94846.log.gz 2024-09-21T07:55:39.515 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71328.log 2024-09-21T07:55:39.516 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-21T07:55:39.517 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.55002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106464.log 2024-09-21T07:55:39.517 INFO:teuthology.orchestra.run.smithi039.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.55002.log.gz 2024-09-21T07:55:39.517 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.71328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115133.log 2024-09-21T07:55:39.517 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71328.log.gz 2024-09-21T07:55:39.518 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.106464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46121.log 2024-09-21T07:55:39.518 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.115133.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.106464.log.gz 2024-09-21T07:55:39.519 INFO:teuthology.orchestra.run.smithi039.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.115133.log.gz 2024-09-21T07:55:39.519 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59389.log 2024-09-21T07:55:39.519 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54439.log 2024-09-21T07:55:39.520 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46121.log.gz 2024-09-21T07:55:39.520 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.59389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80185.log 2024-09-21T07:55:39.520 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59389.log.gz 2024-09-21T07:55:39.521 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36047.log 2024-09-21T07:55:39.522 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41782.log 2024-09-21T07:55:39.523 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99611.log 2024-09-21T07:55:39.523 INFO:teuthology.orchestra.run.smithi039.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54439.log.gz 2024-09-21T07:55:39.523 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.80185.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.80185.log.gz 2024-09-21T07:55:39.523 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43686.log 2024-09-21T07:55:39.523 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.99611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.107242.log 2024-09-21T07:55:39.523 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.99611.log.gz 2024-09-21T07:55:39.524 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.36047.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.36047.log.gz 2024-09-21T07:55:39.524 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35825.log 2024-09-21T07:55:39.525 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78740.log 2024-09-21T07:55:39.525 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43686.log.gz 2024-09-21T07:55:39.525 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.0.107242.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.107242.log.gz 2024-09-21T07:55:39.525 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63573.log 2024-09-21T07:55:39.525 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.78740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68655.log 2024-09-21T07:55:39.526 INFO:teuthology.orchestra.run.smithi039.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.78740.log.gz 2024-09-21T07:55:39.526 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72658.log 2024-09-21T07:55:39.526 INFO:teuthology.orchestra.run.smithi039.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.63573.log.gz 2024-09-21T07:55:39.527 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93274.log 2024-09-21T07:55:39.527 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.68655.log.gz 2024-09-21T07:55:39.528 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.72658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106338.log 2024-09-21T07:55:39.528 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.93274.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.72658.log.gz 2024-09-21T07:55:39.529 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.93274.log.gz 2024-09-21T07:55:39.529 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96202.log 2024-09-21T07:55:39.530 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.3.41782.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.41782.log.gz 2024-09-21T07:55:39.530 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.106338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61874.log 2024-09-21T07:55:39.530 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.106338.log.gz 2024-09-21T07:55:39.531 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.96202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45387.log 2024-09-21T07:55:39.531 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.96202.log.gz 2024-09-21T07:55:39.531 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.61874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98499.log 2024-09-21T07:55:39.532 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61874.log.gz 2024-09-21T07:55:39.532 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.45387.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45387.log.gz 2024-09-21T07:55:39.532 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63134.log 2024-09-21T07:55:39.533 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph-client.admin.35825.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.35825.log.gz 2024-09-21T07:55:39.533 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.98499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42794.log 2024-09-21T07:55:39.533 INFO:teuthology.orchestra.run.smithi039.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.98499.log.gz 2024-09-21T07:55:39.534 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110079.log 2024-09-21T07:55:39.534 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.63134.log.gz 2024-09-21T07:55:39.535 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105395.log 2024-09-21T07:55:39.535 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42794.log.gz 2024-09-21T07:55:39.536 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.110079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85130.log 2024-09-21T07:55:39.536 INFO:teuthology.orchestra.run.smithi039.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.110079.log.gz 2024-09-21T07:55:39.536 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.105395.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.105395.log.gz 2024-09-21T07:55:39.537 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110311.log 2024-09-21T07:55:39.538 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.85130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92512.log 2024-09-21T07:55:39.538 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.85130.log.gz 2024-09-21T07:55:39.538 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.110311.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.110311.log.gz 2024-09-21T07:55:39.539 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97067.log 2024-09-21T07:55:39.539 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.92512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56380.log 2024-09-21T07:55:39.540 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.92512.log.gz 2024-09-21T07:55:39.540 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.97067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91898.log 2024-09-21T07:55:39.540 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.97067.log.gz 2024-09-21T07:55:39.541 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108017.log 2024-09-21T07:55:39.541 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56380.log.gz 2024-09-21T07:55:39.542 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.91898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79221.log 2024-09-21T07:55:39.542 INFO:teuthology.orchestra.run.smithi039.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.91898.log.gz 2024-09-21T07:55:39.543 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.108017.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.108017.log.gz 2024-09-21T07:55:39.543 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57028.log 2024-09-21T07:55:39.544 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.79221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66840.log 2024-09-21T07:55:39.544 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.79221.log.gz 2024-09-21T07:55:39.544 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.57028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96024.log 2024-09-21T07:55:39.545 INFO:teuthology.orchestra.run.smithi039.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57028.log.gz 2024-09-21T07:55:39.545 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.66840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56170.log 2024-09-21T07:55:39.546 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66840.log.gz 2024-09-21T07:55:39.546 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.96024.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.96024.log.gz 2024-09-21T07:55:39.546 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86118.log 2024-09-21T07:55:39.547 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49434.log 2024-09-21T07:55:39.547 INFO:teuthology.orchestra.run.smithi039.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56170.log.gz 2024-09-21T07:55:39.548 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.86118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93900.log 2024-09-21T07:55:39.548 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.86118.log.gz 2024-09-21T07:55:39.549 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95436.log 2024-09-21T07:55:39.549 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49434.log.gz 2024-09-21T07:55:39.550 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.93900.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.93900.log.gz 2024-09-21T07:55:39.550 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50945.log 2024-09-21T07:55:39.551 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.95436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62984.log 2024-09-21T07:55:39.551 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.95436.log.gz 2024-09-21T07:55:39.551 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.50945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99485.log 2024-09-21T07:55:39.552 INFO:teuthology.orchestra.run.smithi039.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50945.log.gz 2024-09-21T07:55:39.552 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.62984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45750.log 2024-09-21T07:55:39.553 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.99485.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.62984.log.gz 2024-09-21T07:55:39.553 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.99485.log.gz 2024-09-21T07:55:39.553 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109498.log 2024-09-21T07:55:39.554 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.45750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104171.log 2024-09-21T07:55:39.554 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45750.log.gz 2024-09-21T07:55:39.555 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.109498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85992.log 2024-09-21T07:55:39.555 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.109498.log.gz 2024-09-21T07:55:39.556 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.104171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85298.log 2024-09-21T07:55:39.556 INFO:teuthology.orchestra.run.smithi039.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.104171.log.gz 2024-09-21T07:55:39.557 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.85992.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.85992.log.gz 2024-09-21T07:55:39.557 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51384.log 2024-09-21T07:55:39.558 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.85298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65360.log 2024-09-21T07:55:39.558 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.85298.log.gz 2024-09-21T07:55:39.558 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57116.log 2024-09-21T07:55:39.559 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51384.log.gz 2024-09-21T07:55:39.559 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.65360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68697.log 2024-09-21T07:55:39.560 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65360.log.gz 2024-09-21T07:55:39.560 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.57116.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57116.log.gz 2024-09-21T07:55:39.560 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78782.log 2024-09-21T07:55:39.561 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61000.log 2024-09-21T07:55:39.561 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.68697.log.gz 2024-09-21T07:55:39.562 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.78782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66594.log 2024-09-21T07:55:39.562 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.78782.log.gz 2024-09-21T07:55:39.563 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.61000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111448.log 2024-09-21T07:55:39.563 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61000.log.gz 2024-09-21T07:55:39.564 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.66594.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.66594.log.gz 2024-09-21T07:55:39.564 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93816.log 2024-09-21T07:55:39.565 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.111448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91257.log 2024-09-21T07:55:39.565 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.111448.log.gz 2024-09-21T07:55:39.565 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.93816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58884.log 2024-09-21T07:55:39.565 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.93816.log.gz 2024-09-21T07:55:39.566 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.91257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106596.log 2024-09-21T07:55:39.567 INFO:teuthology.orchestra.run.smithi039.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.91257.log.gz 2024-09-21T07:55:39.567 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.58884.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.58884.log.gz 2024-09-21T07:55:39.567 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43684.log 2024-09-21T07:55:39.568 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.106596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73505.log 2024-09-21T07:55:39.569 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43684.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.106596.log.gz 2024-09-21T07:55:39.569 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63092.log 2024-09-21T07:55:39.569 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43684.log.gz 2024-09-21T07:55:39.570 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.73505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35692.log 2024-09-21T07:55:39.570 INFO:teuthology.orchestra.run.smithi039.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.73505.log.gz 2024-09-21T07:55:39.571 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63092.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.63092.log.gz 2024-09-21T07:55:39.571 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104741.log 2024-09-21T07:55:39.572 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35692.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35692.log.gz 2024-09-21T07:55:39.572 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96972.log 2024-09-21T07:55:39.572 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.104741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112436.log 2024-09-21T07:55:39.572 INFO:teuthology.orchestra.run.smithi039.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.104741.log.gz 2024-09-21T07:55:39.573 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.96972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73080.log 2024-09-21T07:55:39.573 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.96972.log.gz 2024-09-21T07:55:39.574 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.112436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52372.log 2024-09-21T07:55:39.574 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.112436.log.gz 2024-09-21T07:55:39.575 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.73080.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.73080.log.gz 2024-09-21T07:55:39.575 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49788.log 2024-09-21T07:55:39.576 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90579.log 2024-09-21T07:55:39.576 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52372.log.gz 2024-09-21T07:55:39.577 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69534.log 2024-09-21T07:55:39.577 INFO:teuthology.orchestra.run.smithi039.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49788.log.gz 2024-09-21T07:55:39.578 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.90579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42626.log 2024-09-21T07:55:39.578 INFO:teuthology.orchestra.run.smithi039.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.90579.log.gz 2024-09-21T07:55:39.578 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.69534.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69534.log.gz 2024-09-21T07:55:39.579 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109602.log 2024-09-21T07:55:39.579 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91856.log 2024-09-21T07:55:39.579 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42626.log.gz 2024-09-21T07:55:39.580 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.109602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106086.log 2024-09-21T07:55:39.580 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.109602.log.gz 2024-09-21T07:55:39.581 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.91856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59100.log 2024-09-21T07:55:39.581 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.91856.log.gz 2024-09-21T07:55:39.582 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.106086.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.106086.log.gz 2024-09-21T07:55:39.582 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99340.log 2024-09-21T07:55:39.583 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.59100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92665.log 2024-09-21T07:55:39.583 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59100.log.gz 2024-09-21T07:55:39.584 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.99340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102232.log 2024-09-21T07:55:39.584 INFO:teuthology.orchestra.run.smithi039.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.99340.log.gz 2024-09-21T07:55:39.584 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.92665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52074.log 2024-09-21T07:55:39.585 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.102232.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.92665.log.gz 2024-09-21T07:55:39.585 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.102232.log.gz 2024-09-21T07:55:39.585 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99130.log 2024-09-21T07:55:39.586 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43354.log 2024-09-21T07:55:39.586 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52074.log.gz 2024-09-21T07:55:39.587 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.99130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73811.log 2024-09-21T07:55:39.587 INFO:teuthology.orchestra.run.smithi039.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.99130.log.gz 2024-09-21T07:55:39.588 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43005.log 2024-09-21T07:55:39.588 INFO:teuthology.orchestra.run.smithi039.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43354.log.gz 2024-09-21T07:55:39.589 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.73811.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.73811.log.gz 2024-09-21T07:55:39.589 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79347.log 2024-09-21T07:55:39.590 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49521.log 2024-09-21T07:55:39.590 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43005.log.gz 2024-09-21T07:55:39.590 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.79347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64878.log 2024-09-21T07:55:39.591 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.79347.log.gz 2024-09-21T07:55:39.591 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56935.log 2024-09-21T07:55:39.592 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49521.log.gz 2024-09-21T07:55:39.592 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.64878.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.64878.log.gz 2024-09-21T07:55:39.592 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46434.log 2024-09-21T07:55:39.593 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63969.log 2024-09-21T07:55:39.593 INFO:teuthology.orchestra.run.smithi039.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.56935.log.gz 2024-09-21T07:55:39.594 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42584.log 2024-09-21T07:55:39.594 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46434.log.gz 2024-09-21T07:55:39.595 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111770.log 2024-09-21T07:55:39.595 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.63969.log.gz 2024-09-21T07:55:39.596 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42584.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42584.log.gz 2024-09-21T07:55:39.596 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49658.log 2024-09-21T07:55:39.597 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.111770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97494.log 2024-09-21T07:55:39.597 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.111770.log.gz 2024-09-21T07:55:39.598 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62257.log 2024-09-21T07:55:39.598 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49658.log.gz 2024-09-21T07:55:39.598 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.97494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35813.log 2024-09-21T07:55:39.599 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.97494.log.gz 2024-09-21T07:55:39.599 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.62257.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62257.log.gz 2024-09-21T07:55:39.599 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91730.log 2024-09-21T07:55:39.600 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66393.log 2024-09-21T07:55:39.600 INFO:teuthology.orchestra.run.smithi039.stderr: 79.9% -- replaced with /var/log/ceph/ceph-client.admin.35813.log.gz 2024-09-21T07:55:39.601 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.91730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70230.log 2024-09-21T07:55:39.601 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.91730.log.gz 2024-09-21T07:55:39.602 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.66393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53075.log 2024-09-21T07:55:39.602 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.66393.log.gz 2024-09-21T07:55:39.603 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.70230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73674.log 2024-09-21T07:55:39.603 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.70230.log.gz 2024-09-21T07:55:39.604 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53075.log: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.53075.log.gz 2024-09-21T07:55:39.604 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61323.log 2024-09-21T07:55:39.604 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.73674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84859.log 2024-09-21T07:55:39.605 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.73674.log.gz 2024-09-21T07:55:39.605 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.61323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54918.log 2024-09-21T07:55:39.605 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61323.log.gz 2024-09-21T07:55:39.606 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.84859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82629.log 2024-09-21T07:55:39.606 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.84859.log.gz 2024-09-21T07:55:39.607 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60152.log 2024-09-21T07:55:39.607 INFO:teuthology.orchestra.run.smithi039.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.54918.log.gz 2024-09-21T07:55:39.608 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.82629.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.82629.log.gz 2024-09-21T07:55:39.608 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114357.log 2024-09-21T07:55:39.609 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84607.log 2024-09-21T07:55:39.609 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60152.log.gz 2024-09-21T07:55:39.609 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.114357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69090.log 2024-09-21T07:55:39.610 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.114357.log.gz 2024-09-21T07:55:39.610 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.84607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85172.log 2024-09-21T07:55:39.611 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.84607.log.gz 2024-09-21T07:55:39.611 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.69090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114965.log 2024-09-21T07:55:39.611 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69090.log.gz 2024-09-21T07:55:39.612 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.85172.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.85172.log.gz 2024-09-21T07:55:39.612 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57476.log 2024-09-21T07:55:39.613 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.114965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101315.log 2024-09-21T07:55:39.613 INFO:teuthology.orchestra.run.smithi039.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.114965.log.gz 2024-09-21T07:55:39.614 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.57476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44096.log 2024-09-21T07:55:39.614 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57476.log.gz 2024-09-21T07:55:39.615 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.101315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93400.log 2024-09-21T07:55:39.615 INFO:teuthology.orchestra.run.smithi039.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.101315.log.gz 2024-09-21T07:55:39.616 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.44096.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44096.log.gz 2024-09-21T07:55:39.616 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97750.log 2024-09-21T07:55:39.616 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.93400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69864.log 2024-09-21T07:55:39.617 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.93400.log.gz 2024-09-21T07:55:39.617 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.97750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60378.log 2024-09-21T07:55:39.617 INFO:teuthology.orchestra.run.smithi039.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.97750.log.gz 2024-09-21T07:55:39.618 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.69864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91337.log 2024-09-21T07:55:39.619 INFO:teuthology.orchestra.run.smithi039.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.69864.log.gz 2024-09-21T07:55:39.619 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111326.log 2024-09-21T07:55:39.619 INFO:teuthology.orchestra.run.smithi039.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.60378.log.gz 2024-09-21T07:55:39.620 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.91337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52752.log 2024-09-21T07:55:39.620 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.91337.log.gz 2024-09-21T07:55:39.621 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.111326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54396.log 2024-09-21T07:55:39.621 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.111326.log.gz 2024-09-21T07:55:39.622 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.0.52752.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.52752.log.gz 2024-09-21T07:55:39.622 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114839.log 2024-09-21T07:55:39.623 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94497.log 2024-09-21T07:55:39.623 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54396.log.gz 2024-09-21T07:55:39.623 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.114839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111854.log 2024-09-21T07:55:39.624 INFO:teuthology.orchestra.run.smithi039.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.114839.log.gz 2024-09-21T07:55:39.624 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.94497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54483.log 2024-09-21T07:55:39.624 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.94497.log.gz 2024-09-21T07:55:39.625 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.111854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52564.log 2024-09-21T07:55:39.625 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.111854.log.gz 2024-09-21T07:55:39.626 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54483.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54483.log.gz 2024-09-21T07:55:39.626 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46794.log 2024-09-21T07:55:39.627 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52564.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-21T07:55:39.627 INFO:teuthology.orchestra.run.smithi039.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52564.log.gz 2024-09-21T07:55:39.628 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90682.log 2024-09-21T07:55:39.628 INFO:teuthology.orchestra.run.smithi039.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46794.log.gz 2024-09-21T07:55:39.629 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111938.log 2024-09-21T07:55:39.630 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.90682.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.90682.log.gz 2024-09-21T07:55:39.630 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107320.log 2024-09-21T07:55:39.631 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.111938.log: gzip -5 --verbose -- 87.3% /var/log/ceph/ceph-client.admin.46920.log -- replaced with /var/log/ceph/ceph-client.admin.111938.log.gz 2024-09-21T07:55:39.631 INFO:teuthology.orchestra.run.smithi039.stderr: 2024-09-21T07:55:39.631 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.107320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108345.log 2024-09-21T07:55:39.632 INFO:teuthology.orchestra.run.smithi039.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.107320.log.gz 2024-09-21T07:55:39.632 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46920.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46920.log.gz 2024-09-21T07:55:39.633 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104256.log 2024-09-21T07:55:39.634 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.108345.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.108345.log.gz 2024-09-21T07:55:39.634 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55731.log 2024-09-21T07:55:39.634 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.104256.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.104256.log.gz 2024-09-21T07:55:39.635 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96286.log 2024-09-21T07:55:39.635 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.55731.log: gzip -5 --verbose 87.5% -- -- replaced with /var/log/ceph/ceph-client.admin.55731.log.gz /var/log/ceph/ceph-client.admin.112113.log 2024-09-21T07:55:39.635 INFO:teuthology.orchestra.run.smithi039.stderr: 2024-09-21T07:55:39.636 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.96286.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.96286.log.gz 2024-09-21T07:55:39.636 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68210.log 2024-09-21T07:55:39.637 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.112113.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.112113.log.gz 2024-09-21T07:55:39.637 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97960.log 2024-09-21T07:55:39.638 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68210.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.68210.log.gz 2024-09-21T07:55:39.638 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57823.log 2024-09-21T07:55:39.639 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.97960.log: gzip -5 --verbose 87.2% -- -- replaced with /var/log/ceph/ceph-client.admin.97960.log.gz /var/log/ceph/ceph-client.admin.74536.log 2024-09-21T07:55:39.639 INFO:teuthology.orchestra.run.smithi039.stderr: 2024-09-21T07:55:39.640 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.57823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74145.log 2024-09-21T07:55:39.640 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57823.log.gz 2024-09-21T07:55:39.641 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.74536.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.74536.log.gz 2024-09-21T07:55:39.641 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51979.log 2024-09-21T07:55:39.642 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.74145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93648.log 2024-09-21T07:55:39.642 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.74145.log.gz 2024-09-21T07:55:39.643 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51979.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.51979.log.gz 2024-09-21T07:55:39.643 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69202.log 2024-09-21T07:55:39.644 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.93648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63719.log 2024-09-21T07:55:39.644 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.93648.log.gz 2024-09-21T07:55:39.645 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.69202.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.69202.log.gz 2024-09-21T07:55:39.645 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51426.log 2024-09-21T07:55:39.646 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43006.log 2024-09-21T07:55:39.646 INFO:teuthology.orchestra.run.smithi039.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.63719.log.gz 2024-09-21T07:55:39.647 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51426.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51426.log.gz 2024-09-21T07:55:39.647 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47651.log 2024-09-21T07:55:39.648 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60009.log 2024-09-21T07:55:39.648 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43006.log.gz 2024-09-21T07:55:39.649 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47651.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47651.log.gz 2024-09-21T07:55:39.649 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109818.log 2024-09-21T07:55:39.650 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35740.log 2024-09-21T07:55:39.650 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.60009.log.gz 2024-09-21T07:55:39.651 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.109818.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.109818.log.gz 2024-09-21T07:55:39.651 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50082.log 2024-09-21T07:55:39.652 INFO:teuthology.orchestra.run.smithi148.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-21T07:55:39.652 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56986.log 2024-09-21T07:55:39.652 INFO:teuthology.orchestra.run.smithi039.stderr: 77.0% -- replaced with /var/log/ceph/ceph-client.admin.35740.log.gz 2024-09-21T07:55:39.653 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.50082.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50082.log.gz 2024-09-21T07:55:39.653 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96720.log 2024-09-21T07:55:39.654 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71689.log 2024-09-21T07:55:39.654 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56986.log.gz 2024-09-21T07:55:39.655 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.96720.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.96720.log.gz 2024-09-21T07:55:39.655 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52023.log 2024-09-21T07:55:39.656 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.71689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58333.log 2024-09-21T07:55:39.656 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71689.log.gz 2024-09-21T07:55:39.657 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52023.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52023.log.gz 2024-09-21T07:55:39.657 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102539.log 2024-09-21T07:55:39.658 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.58333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108213.log 2024-09-21T07:55:39.658 INFO:teuthology.orchestra.run.smithi039.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58333.log.gz 2024-09-21T07:55:39.659 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.102539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97666.log 2024-09-21T07:55:39.659 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.102539.log.gz 2024-09-21T07:55:39.660 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.108213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50274.log 2024-09-21T07:55:39.660 INFO:teuthology.orchestra.run.smithi039.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.108213.log.gz 2024-09-21T07:55:39.661 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.97666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68434.log 2024-09-21T07:55:39.661 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.97666.log.gz 2024-09-21T07:55:39.662 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.50274.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50274.log.gz 2024-09-21T07:55:39.662 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51300.log 2024-09-21T07:55:39.663 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107592.log 2024-09-21T07:55:39.663 INFO:teuthology.orchestra.run.smithi039.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.68434.log.gz 2024-09-21T07:55:39.664 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51300.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51300.log.gz 2024-09-21T07:55:39.664 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46836.log 2024-09-21T07:55:39.665 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.107592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69780.log 2024-09-21T07:55:39.666 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.107592.log.gz 2024-09-21T07:55:39.666 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46836.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46836.log.gz 2024-09-21T07:55:39.667 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49914.log 2024-09-21T07:55:39.667 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.69780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62195.log 2024-09-21T07:55:39.668 INFO:teuthology.orchestra.run.smithi039.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.69780.log.gz 2024-09-21T07:55:39.668 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49914.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49914.log.gz 2024-09-21T07:55:39.669 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72188.log 2024-09-21T07:55:39.669 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.62195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91418.log 2024-09-21T07:55:39.670 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.62195.log.gz 2024-09-21T07:55:39.671 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.72188.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.72188.log.gz 2024-09-21T07:55:39.671 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114737.log 2024-09-21T07:55:39.672 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.91418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102356.log 2024-09-21T07:55:39.672 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.91418.log.gz 2024-09-21T07:55:39.673 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.114737.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.114737.log.gz 2024-09-21T07:55:39.673 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97229.log 2024-09-21T07:55:39.674 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.102356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106128.log 2024-09-21T07:55:39.674 INFO:teuthology.orchestra.run.smithi039.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.102356.log.gz 2024-09-21T07:55:39.675 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.97229.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.97229.log.gz 2024-09-21T07:55:39.675 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98583.log 2024-09-21T07:55:39.676 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.106128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104699.log 2024-09-21T07:55:39.676 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.106128.log.gz 2024-09-21T07:55:39.677 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.98583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67559.log 2024-09-21T07:55:39.677 INFO:teuthology.orchestra.run.smithi039.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.98583.log.gz 2024-09-21T07:55:39.678 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.104699.log: gzip 87.2% -- replaced with /var/log/ceph/ceph-client.admin.104699.log.gz 2024-09-21T07:55:39.678 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.91982.log 2024-09-21T07:55:39.679 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.67559.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67559.log.gz 2024-09-21T07:55:39.679 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106002.log 2024-09-21T07:55:39.680 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.91982.log: gzip 87.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.91982.log.gz --verbose 2024-09-21T07:55:39.680 INFO:teuthology.orchestra.run.smithi039.stderr: -- /var/log/ceph/ceph-client.admin.61610.log 2024-09-21T07:55:39.680 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.106002.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.106002.log.gz 2024-09-21T07:55:39.681 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56506.log 2024-09-21T07:55:39.681 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.61610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47143.log 2024-09-21T07:55:39.682 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61610.log.gz 2024-09-21T07:55:39.682 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56601.log 2024-09-21T07:55:39.682 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56506.log.gz 2024-09-21T07:55:39.683 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47143.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47143.log.gz 2024-09-21T07:55:39.683 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73379.log 2024-09-21T07:55:39.684 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105774.log 2024-09-21T07:55:39.684 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56601.log.gz 2024-09-21T07:55:39.685 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.73379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57074.log 2024-09-21T07:55:39.685 INFO:teuthology.orchestra.run.smithi039.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.73379.log.gz 2024-09-21T07:55:39.686 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.105774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84649.log 2024-09-21T07:55:39.686 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.105774.log.gz 2024-09-21T07:55:39.687 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.57074.log: gzip 87.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.57074.log.gz 2024-09-21T07:55:39.687 INFO:teuthology.orchestra.run.smithi039.stderr: --verbose -- /var/log/ceph/ceph-client.admin.45513.log 2024-09-21T07:55:39.688 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.84649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48884.log 2024-09-21T07:55:39.688 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.84649.log.gz 2024-09-21T07:55:39.689 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.45513.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45513.log.gz 2024-09-21T07:55:39.689 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115511.log 2024-09-21T07:55:39.690 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79792.log 2024-09-21T07:55:39.690 INFO:teuthology.orchestra.run.smithi039.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48884.log.gz 2024-09-21T07:55:39.691 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.115511.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.115511.log.gz 2024-09-21T07:55:39.691 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85904.log 2024-09-21T07:55:39.692 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.79792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56464.log 2024-09-21T07:55:39.692 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.79792.log.gz 2024-09-21T07:55:39.692 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.85904.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.85904.log.gzgzip 2024-09-21T07:55:39.693 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.65663.log 2024-09-21T07:55:39.693 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35765.log 2024-09-21T07:55:39.694 INFO:teuthology.orchestra.run.smithi039.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56464.log.gz 2024-09-21T07:55:39.694 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.65663.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.65663.log.gz 2024-09-21T07:55:39.694 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103125.log 2024-09-21T07:55:39.695 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35765.log: 78.2% -- replaced with /var/log/ceph/ceph-client.admin.35765.log.gz 2024-09-21T07:55:39.695 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72912.log 2024-09-21T07:55:39.696 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.103125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36221.log 2024-09-21T07:55:39.697 INFO:teuthology.orchestra.run.smithi039.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.103125.log.gz 2024-09-21T07:55:39.697 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.72912.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.72912.log.gz 2024-09-21T07:55:39.697 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55983.log 2024-09-21T07:55:39.698 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69596.log 86.2% 2024-09-21T07:55:39.698 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36221.log.gz 2024-09-21T07:55:39.699 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.55983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60336.log 2024-09-21T07:55:39.699 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55983.log.gz 2024-09-21T07:55:39.700 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.69596.log: gzip -5 --verbose -- 87.3% /var/log/ceph/ceph-client.admin.51689.log 2024-09-21T07:55:39.700 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.69596.log.gz 2024-09-21T07:55:39.701 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60336.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60336.log.gz 2024-09-21T07:55:39.701 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85722.log 2024-09-21T07:55:39.702 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51689.log: gzip 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51689.log.gz 2024-09-21T07:55:39.702 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36280.log 2024-09-21T07:55:39.703 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.85722.log: 87.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.85722.log.gz 2024-09-21T07:55:39.703 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.48553.log 2024-09-21T07:55:39.704 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36280.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.36280.log.gz 2024-09-21T07:55:39.704 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93358.log 2024-09-21T07:55:39.704 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48553.log: gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48553.log.gz 2024-09-21T07:55:39.705 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.58600.log 2024-09-21T07:55:39.705 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.93358.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.93358.log.gz 2024-09-21T07:55:39.706 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109129.log 2024-09-21T07:55:39.706 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.58600.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58600.log.gz 2024-09-21T07:55:39.706 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111368.log 2024-09-21T07:55:39.707 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.109129.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.109129.log.gz 2024-09-21T07:55:39.707 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106380.log 2024-09-21T07:55:39.708 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.111368.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.111368.log.gz 2024-09-21T07:55:39.708 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104967.log 2024-09-21T07:55:39.709 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.106380.log: 87.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.106380.log.gz 2024-09-21T07:55:39.709 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.49830.log 2024-09-21T07:55:39.710 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.104967.log: gzip 87.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.104967.log.gz 2024-09-21T07:55:39.710 INFO:teuthology.orchestra.run.smithi039.stderr: --verbose -- /var/log/ceph/ceph-client.admin.72146.log 2024-09-21T07:55:39.711 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49830.log: 87.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.49830.log.gz 2024-09-21T07:55:39.711 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.68089.log 2024-09-21T07:55:39.712 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.72146.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.72146.log.gzgzip 2024-09-21T07:55:39.712 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.90369.log 2024-09-21T07:55:39.713 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68089.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.68089.log.gz 2024-09-21T07:55:39.713 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56044.log 2024-09-21T07:55:39.714 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.90369.log: gzip -5 --verbose 87.1% -- -- replaced with /var/log/ceph/ceph-client.admin.90369.log.gz 2024-09-21T07:55:39.714 INFO:teuthology.orchestra.run.smithi039.stderr: /var/log/ceph/ceph-client.admin.112524.log 2024-09-21T07:55:39.715 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47567.log 2024-09-21T07:55:39.715 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56044.log.gz 2024-09-21T07:55:39.715 INFO:teuthology.orchestra.run.smithi148.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-21T07:55:39.716 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.112524.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.112524.log.gz 2024-09-21T07:55:39.716 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99424.log 2024-09-21T07:55:39.716 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47567.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47567.log.gz 2024-09-21T07:55:39.717 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41888.log 2024-09-21T07:55:39.717 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.99424.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.99424.log.gz 2024-09-21T07:55:39.717 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72777.log 2024-09-21T07:55:39.718 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.41888.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41888.log.gz 2024-09-21T07:55:39.718 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108279.log 2024-09-21T07:55:39.719 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.72777.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.72777.log.gz 2024-09-21T07:55:39.719 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67211.log 2024-09-21T07:55:39.720 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.108279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.66170.log 2024-09-21T07:55:39.721 INFO:teuthology.orchestra.run.smithi039.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.108279.log.gz 2024-09-21T07:55:39.721 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.67211.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67211.log.gz 2024-09-21T07:55:39.721 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100935.log 2024-09-21T07:55:39.722 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.0.66170.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.66170.log.gz 2024-09-21T07:55:39.722 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109560.log 2024-09-21T07:55:39.723 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.100935.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.100935.log.gz 2024-09-21T07:55:39.723 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100203.log 2024-09-21T07:55:39.724 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.109560.log: gzip -5 87.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.109560.log.gz 2024-09-21T07:55:39.724 INFO:teuthology.orchestra.run.smithi039.stderr: -- /var/log/ceph/ceph-client.admin.63501.log 2024-09-21T07:55:39.725 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.100203.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.100203.log.gz 2024-09-21T07:55:39.725 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90850.log 2024-09-21T07:55:39.726 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115049.log 2024-09-21T07:55:39.726 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.63501.log.gz 2024-09-21T07:55:39.727 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.90850.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.90850.log.gz 2024-09-21T07:55:39.727 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58780.log 2024-09-21T07:55:39.728 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.115049.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.115049.log.gz 2024-09-21T07:55:39.728 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99298.log 2024-09-21T07:55:39.728 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.58780.log: 87.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.58780.log.gz 2024-09-21T07:55:39.729 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47231.log 2024-09-21T07:55:39.729 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.99298.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.99298.log.gz 2024-09-21T07:55:39.730 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68784.log 2024-09-21T07:55:39.730 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63256.log 2024-09-21T07:55:39.730 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47231.log.gz 2024-09-21T07:55:39.731 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102891.log 2024-09-21T07:55:39.732 INFO:teuthology.orchestra.run.smithi039.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.68784.log.gz 2024-09-21T07:55:39.732 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63256.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.63256.log.gz 2024-09-21T07:55:39.732 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52498.log 2024-09-21T07:55:39.733 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.102891.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.102891.log.gz 2024-09-21T07:55:39.733 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112253.log 2024-09-21T07:55:39.734 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106808.log 2024-09-21T07:55:39.734 INFO:teuthology.orchestra.run.smithi039.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.52498.log.gz 2024-09-21T07:55:39.735 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.112253.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.112253.log.gzgzip 2024-09-21T07:55:39.735 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.55431.log 2024-09-21T07:55:39.736 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.106808.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.106808.log.gz 2024-09-21T07:55:39.736 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92461.log 2024-09-21T07:55:39.737 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.55431.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55431.log.gz 2024-09-21T07:55:39.737 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102046.log 2024-09-21T07:55:39.738 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.92461.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.92461.log.gz 2024-09-21T07:55:39.738 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62512.log 2024-09-21T07:55:39.739 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.102046.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.102046.log.gz 2024-09-21T07:55:39.739 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67447.log 2024-09-21T07:55:39.740 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.62512.log: gzip 87.4% -- replaced with /var/log/ceph/ceph-client.admin.62512.log.gz 2024-09-21T07:55:39.740 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.74196.log 2024-09-21T07:55:39.741 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.67447.log: 87.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.67447.log.gz 2024-09-21T07:55:39.741 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.65579.log 2024-09-21T07:55:39.742 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.74196.log: gzip 87.4% -- replaced with /var/log/ceph/ceph-client.admin.74196.log.gz -5 2024-09-21T07:55:39.742 INFO:teuthology.orchestra.run.smithi039.stderr: --verbose -- /var/log/ceph/ceph-client.admin.73337.log 2024-09-21T07:55:39.743 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.65579.log: 87.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.65579.log.gz 2024-09-21T07:55:39.743 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.100332.log 2024-09-21T07:55:39.743 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.73337.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.73337.log.gz 2024-09-21T07:55:39.744 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105564.log 2024-09-21T07:55:39.744 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.100332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57368.log 87.4% 2024-09-21T07:55:39.744 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.100332.log.gz 2024-09-21T07:55:39.745 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.105564.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.105564.log.gz 2024-09-21T07:55:39.745 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111158.log 2024-09-21T07:55:39.746 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.57368.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57368.log.gz 2024-09-21T07:55:39.746 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69638.log 2024-09-21T07:55:39.747 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.111158.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.111158.log.gz 2024-09-21T07:55:39.747 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51510.log 2024-09-21T07:55:39.748 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.69638.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.69638.log.gz 2024-09-21T07:55:39.748 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106044.log 2024-09-21T07:55:39.749 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51510.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51510.log.gz 2024-09-21T07:55:39.749 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36011.log 2024-09-21T07:55:39.750 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.106044.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.106044.log.gz 2024-09-21T07:55:39.750 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44054.log 2024-09-21T07:55:39.751 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99569.log 2024-09-21T07:55:39.751 INFO:teuthology.orchestra.run.smithi039.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.36011.log.gz 2024-09-21T07:55:39.752 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.44054.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44054.log.gz 2024-09-21T07:55:39.752 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56422.log 2024-09-21T07:55:39.753 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.99569.log: gzip 87.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.99569.log.gz --verbose 2024-09-21T07:55:39.753 INFO:teuthology.orchestra.run.smithi039.stderr: -- /var/log/ceph/ceph-client.admin.47189.log 2024-09-21T07:55:39.754 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56422.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56422.log.gz 2024-09-21T07:55:39.754 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53888.log 2024-09-21T07:55:39.755 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47189.log: gzip 87.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.47189.log.gz --verbose 2024-09-21T07:55:39.755 INFO:teuthology.orchestra.run.smithi039.stderr: -- /var/log/ceph/ceph-client.admin.36058.log 2024-09-21T07:55:39.755 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53888.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.53888.log.gz 2024-09-21T07:55:39.756 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90233.log 2024-09-21T07:55:39.756 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36058.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.36058.log.gz 2024-09-21T07:55:39.757 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70168.log 2024-09-21T07:55:39.757 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.90233.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.90233.log.gz 2024-09-21T07:55:39.757 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71936.log 2024-09-21T07:55:39.758 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.70168.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.70168.log.gz 2024-09-21T07:55:39.758 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103965.log 2024-09-21T07:55:39.759 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.71936.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71936.log.gz 2024-09-21T07:55:39.759 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36585.log 2024-09-21T07:55:39.760 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.103965.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.103965.log.gz 2024-09-21T07:55:39.760 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46532.log 2024-09-21T07:55:39.761 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36585.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.36585.log.gz 2024-09-21T07:55:39.761 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91547.log 2024-09-21T07:55:39.762 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46532.log: gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46532.log.gz 2024-09-21T07:55:39.762 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.67919.log 2024-09-21T07:55:39.763 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.91547.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.91547.log.gzgzip 2024-09-21T07:55:39.763 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.86286.log 2024-09-21T07:55:39.764 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.67919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63299.log 2024-09-21T07:55:39.764 INFO:teuthology.orchestra.run.smithi039.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.67919.log.gz 2024-09-21T07:55:39.765 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.86286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42275.log 2024-09-21T07:55:39.765 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.86286.log.gz 2024-09-21T07:55:39.766 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63299.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.63299.log.gz 2024-09-21T07:55:39.766 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79305.log 2024-09-21T07:55:39.767 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114273.log 2024-09-21T07:55:39.767 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42275.log.gz 2024-09-21T07:55:39.768 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.79305.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.79305.log.gz 2024-09-21T07:55:39.768 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113340.log 2024-09-21T07:55:39.769 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.114273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114147.log 2024-09-21T07:55:39.769 INFO:teuthology.orchestra.run.smithi039.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.114273.log.gz 2024-09-21T07:55:39.770 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.113340.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.113340.log.gz 2024-09-21T07:55:39.770 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107054.log 2024-09-21T07:55:39.771 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.114147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80395.log 2024-09-21T07:55:39.771 INFO:teuthology.orchestra.run.smithi039.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.114147.log.gz 2024-09-21T07:55:39.772 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.107054.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.107054.log.gz 2024-09-21T07:55:39.772 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105009.log 2024-09-21T07:55:39.773 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.80395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73225.log 2024-09-21T07:55:39.774 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.80395.log.gz 2024-09-21T07:55:39.774 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.105009.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.105009.log.gz 2024-09-21T07:55:39.774 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93316.log 2024-09-21T07:55:39.775 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.73225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103492.log 2024-09-21T07:55:39.776 INFO:teuthology.orchestra.run.smithi039.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.73225.log.gz 2024-09-21T07:55:39.776 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.93316.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.93316.log.gz 2024-09-21T07:55:39.776 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104214.log 2024-09-21T07:55:39.777 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.103492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99779.log 2024-09-21T07:55:39.778 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.103492.log.gz 2024-09-21T07:55:39.778 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.104214.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.104214.log.gz 2024-09-21T07:55:39.779 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44138.log 2024-09-21T07:55:39.779 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.99779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72392.log 2024-09-21T07:55:39.780 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.99779.log.gz 2024-09-21T07:55:39.781 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.44138.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44138.log.gz 2024-09-21T07:55:39.781 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104086.log 2024-09-21T07:55:39.781 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.72392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49956.log 2024-09-21T07:55:39.782 INFO:teuthology.orchestra.run.smithi039.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.72392.log.gz 2024-09-21T07:55:39.783 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.104086.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.104086.log.gz 2024-09-21T07:55:39.783 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61935.log 2024-09-21T07:55:39.784 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42189.log 2024-09-21T07:55:39.784 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49956.log.gz 2024-09-21T07:55:39.785 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.61935.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61935.log.gz 2024-09-21T07:55:39.785 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115259.log 2024-09-21T07:55:39.786 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.86600.log 2024-09-21T07:55:39.786 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42189.log.gz 2024-09-21T07:55:39.787 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.115259.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.115259.log.gz 2024-09-21T07:55:39.787 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54834.log 2024-09-21T07:55:39.788 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.0.86600.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.86600.log.gz 2024-09-21T07:55:39.788 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107838.log 2024-09-21T07:55:39.789 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54834.log: gzip -5 87.1% --verbose -- replaced with /var/log/ceph/ceph-client.admin.54834.log.gz -- 2024-09-21T07:55:39.789 INFO:teuthology.orchestra.run.smithi039.stderr: /var/log/ceph/ceph-client.admin.113463.log 2024-09-21T07:55:39.790 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.107838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66435.log 2024-09-21T07:55:39.790 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.107838.log.gz 2024-09-21T07:55:39.791 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.113463.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.113463.log.gz 2024-09-21T07:55:39.791 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43970.log 2024-09-21T07:55:39.792 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.66435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67025.log 2024-09-21T07:55:39.792 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66435.log.gz 2024-09-21T07:55:39.793 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50462.log 2024-09-21T07:55:39.793 INFO:teuthology.orchestra.run.smithi039.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43970.log.gz 2024-09-21T07:55:39.794 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.67025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58251.log 2024-09-21T07:55:39.794 INFO:teuthology.orchestra.run.smithi039.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.67025.log.gz 2024-09-21T07:55:39.794 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.0.50462.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50462.log.gz 2024-09-21T07:55:39.795 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105291.log 2024-09-21T07:55:39.796 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.58251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100785.log 2024-09-21T07:55:39.796 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.58251.log.gz 2024-09-21T07:55:39.797 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.105291.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.105291.log.gz 2024-09-21T07:55:39.797 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114315.log 2024-09-21T07:55:39.798 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.100785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94323.log 2024-09-21T07:55:39.798 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.100785.log.gz 2024-09-21T07:55:39.799 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.114315.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.114315.log.gz 2024-09-21T07:55:39.799 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79704.log 2024-09-21T07:55:39.800 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.94323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47101.log 2024-09-21T07:55:39.801 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.94323.log.gz 2024-09-21T07:55:39.801 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.79704.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.79704.log.gz 2024-09-21T07:55:39.801 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74704.log 2024-09-21T07:55:39.802 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51891.log 2024-09-21T07:55:39.803 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47101.log.gz 2024-09-21T07:55:39.803 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.74704.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.74704.log.gz 2024-09-21T07:55:39.805 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49704.log 2024-09-21T07:55:39.806 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109354.log 2024-09-21T07:55:39.807 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51891.log.gz 2024-09-21T07:55:39.807 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49704.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49704.log.gz 2024-09-21T07:55:39.807 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111686.log 2024-09-21T07:55:39.807 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.109354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59263.log 2024-09-21T07:55:39.808 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.109354.log.gz 2024-09-21T07:55:39.808 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.111686.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.111686.log.gz 2024-09-21T07:55:39.808 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70374.log 2024-09-21T07:55:39.808 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.59263.log: 87.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.59263.log.gz 2024-09-21T07:55:39.809 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.58719.log 2024-09-21T07:55:39.809 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.70374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107933.log 2024-09-21T07:55:39.810 INFO:teuthology.orchestra.run.smithi039.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.70374.log.gz 2024-09-21T07:55:39.811 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.58719.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.58719.log.gz 2024-09-21T07:55:39.811 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42231.log 2024-09-21T07:55:39.812 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.107933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62081.log 2024-09-21T07:55:39.812 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.107933.log.gz 2024-09-21T07:55:39.813 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42231.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.42231.log.gz 2024-09-21T07:55:39.813 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64963.log 2024-09-21T07:55:39.814 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.62081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.80733.log 2024-09-21T07:55:39.814 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.62081.log.gz 2024-09-21T07:55:39.815 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.64963.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.64963.log.gz 2024-09-21T07:55:39.815 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97147.log 2024-09-21T07:55:39.816 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.0.80733.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.80733.log.gz 2024-09-21T07:55:39.816 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106960.log 2024-09-21T07:55:39.817 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.97147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103819.log 2024-09-21T07:55:39.817 INFO:teuthology.orchestra.run.smithi039.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.97147.log.gz 2024-09-21T07:55:39.818 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.106960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101264.log 2024-09-21T07:55:39.818 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.106960.log.gz 2024-09-21T07:55:39.819 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.103819.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.103819.log.gz 2024-09-21T07:55:39.819 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109171.log 2024-09-21T07:55:39.820 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.101264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94972.log 2024-09-21T07:55:39.820 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.101264.log.gz 2024-09-21T07:55:39.821 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.109171.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.109171.log.gz 2024-09-21T07:55:39.821 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57242.log 2024-09-21T07:55:39.822 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.94972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112650.log 2024-09-21T07:55:39.823 INFO:teuthology.orchestra.run.smithi039.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.94972.log.gz 2024-09-21T07:55:39.823 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.57242.log: 87.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.57242.log.gz -5 2024-09-21T07:55:39.823 INFO:teuthology.orchestra.run.smithi039.stderr: --verbose -- /var/log/ceph/ceph-client.admin.58992.log 2024-09-21T07:55:39.824 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.112650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66636.log 2024-09-21T07:55:39.824 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.112650.log.gz 2024-09-21T07:55:39.825 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.58992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79887.log 2024-09-21T07:55:39.826 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.66636.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.58992.log.gz 2024-09-21T07:55:39.826 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66636.log.gz 2024-09-21T07:55:39.826 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48197.log 2024-09-21T07:55:39.827 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.79887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42458.log 2024-09-21T07:55:39.827 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.79887.log.gz 2024-09-21T07:55:39.828 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93138.log 2024-09-21T07:55:39.828 INFO:teuthology.orchestra.run.smithi039.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.48197.log.gz 2024-09-21T07:55:39.829 INFO:teuthology.orchestra.run.smithi148.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-21T07:55:39.829 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42458.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42458.log.gz 2024-09-21T07:55:39.830 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79389.log 2024-09-21T07:55:39.830 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.93138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43017.log 2024-09-21T07:55:39.831 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.93138.log.gz 2024-09-21T07:55:39.831 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.79389.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.79389.log.gz 2024-09-21T07:55:39.832 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62123.log 2024-09-21T07:55:39.832 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56128.log 2024-09-21T07:55:39.832 INFO:teuthology.orchestra.run.smithi039.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.43017.log.gz 2024-09-21T07:55:39.833 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.62123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56803.log 2024-09-21T07:55:39.834 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62123.log.gz 2024-09-21T07:55:39.834 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56128.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56128.log.gz 2024-09-21T07:55:39.835 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105960.log 2024-09-21T07:55:39.835 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112692.log 2024-09-21T07:55:39.836 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56803.log.gz 2024-09-21T07:55:39.836 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.105960.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.105960.log.gz 2024-09-21T07:55:39.837 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70734.log 2024-09-21T07:55:39.838 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.112692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67841.log 2024-09-21T07:55:39.838 INFO:teuthology.orchestra.run.smithi039.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.112692.log.gz 2024-09-21T07:55:39.839 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.70734.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.70734.log.gz 2024-09-21T07:55:39.839 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105918.log 2024-09-21T07:55:39.839 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.0.67841.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.67841.log.gz 2024-09-21T07:55:39.839 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93858.log 2024-09-21T07:55:39.840 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.105918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96552.log 2024-09-21T07:55:39.841 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.105918.log.gz 2024-09-21T07:55:39.842 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.93858.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.93858.log.gz 2024-09-21T07:55:39.842 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108469.log 2024-09-21T07:55:39.843 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.96552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106662.log 2024-09-21T07:55:39.843 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.96552.log.gz 2024-09-21T07:55:39.844 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.108469.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.108469.log.gz 2024-09-21T07:55:39.844 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84901.log 2024-09-21T07:55:39.845 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.106662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48926.log 2024-09-21T07:55:39.845 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.106662.log.gz 2024-09-21T07:55:39.846 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.84901.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.84901.log.gz 2024-09-21T07:55:39.846 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112734.log 2024-09-21T07:55:39.847 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59967.log 2024-09-21T07:55:39.847 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48926.log.gz 2024-09-21T07:55:39.848 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.112734.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.112734.log.gz 2024-09-21T07:55:39.848 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54876.log 2024-09-21T07:55:39.849 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.59967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47609.log 2024-09-21T07:55:39.849 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59967.log.gz 2024-09-21T07:55:39.850 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54876.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54876.log.gz 2024-09-21T07:55:39.850 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65796.log 2024-09-21T07:55:39.851 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36826.log 2024-09-21T07:55:39.851 INFO:teuthology.orchestra.run.smithi039.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47609.log.gz 2024-09-21T07:55:39.852 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.65796.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65796.log.gz 2024-09-21T07:55:39.852 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55068.log 2024-09-21T07:55:39.853 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93606.log 2024-09-21T07:55:39.853 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36826.log.gz 2024-09-21T07:55:39.854 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.55068.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55068.log.gz 2024-09-21T07:55:39.855 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73267.log 2024-09-21T07:55:39.855 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.93606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100869.log 2024-09-21T07:55:39.856 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.93606.log.gz 2024-09-21T07:55:39.856 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.73267.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.73267.log.gz 2024-09-21T07:55:39.857 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85382.log 2024-09-21T07:55:39.857 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.100869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101468.log 2024-09-21T07:55:39.858 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.85382.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.100869.log.gz 2024-09-21T07:55:39.858 INFO:teuthology.orchestra.run.smithi039.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.85382.log.gz 2024-09-21T07:55:39.859 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42500.log 2024-09-21T07:55:39.859 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.101468.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-21T07:55:39.860 INFO:teuthology.orchestra.run.smithi039.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.101468.log.gz 2024-09-21T07:55:39.861 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42500.log: gzip 87.1% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.42500.log.gz -- 2024-09-21T07:55:39.861 INFO:teuthology.orchestra.run.smithi039.stderr: /var/log/ceph/ceph-client.admin.60954.log 2024-09-21T07:55:39.862 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68372.log 2024-09-21T07:55:39.863 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60954.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60954.log.gz 2024-09-21T07:55:39.863 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48842.log 2024-09-21T07:55:39.864 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68372.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68372.log.gz 2024-09-21T07:55:39.864 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73971.log 2024-09-21T07:55:39.865 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85088.log 2024-09-21T07:55:39.865 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48842.log.gz 2024-09-21T07:55:39.866 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.73971.log: 89.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-21T07:55:39.866 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.73971.log.gz 2024-09-21T07:55:39.867 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58375.log 2024-09-21T07:55:39.867 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.85088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55941.log 2024-09-21T07:55:39.868 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.58375.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.85088.log.gz 2024-09-21T07:55:39.868 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93442.log 2024-09-21T07:55:39.868 INFO:teuthology.orchestra.run.smithi039.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.58375.log.gz 2024-09-21T07:55:39.869 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.55941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45597.log 2024-09-21T07:55:39.869 INFO:teuthology.orchestra.run.smithi039.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55941.log.gz 2024-09-21T07:55:39.870 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.93442.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.93442.log.gz 2024-09-21T07:55:39.870 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115217.log 2024-09-21T07:55:39.871 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.45597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79053.log 2024-09-21T07:55:39.871 INFO:teuthology.orchestra.run.smithi039.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.45597.log.gz 2024-09-21T07:55:39.872 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.115217.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.115217.log.gz 2024-09-21T07:55:39.872 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115595.log 2024-09-21T07:55:39.873 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.79053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113970.log 2024-09-21T07:55:39.873 INFO:teuthology.orchestra.run.smithi039.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.79053.log.gz 2024-09-21T07:55:39.874 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.115595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97624.log 2024-09-21T07:55:39.874 INFO:teuthology.orchestra.run.smithi039.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.115595.log.gz 2024-09-21T07:55:39.875 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.113970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78969.log 2024-09-21T07:55:39.875 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.113970.log.gz 2024-09-21T07:55:39.876 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.97624.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.97624.log.gz 2024-09-21T07:55:39.877 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55595.log 2024-09-21T07:55:39.877 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.78969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54193.log 2024-09-21T07:55:39.878 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.78969.log.gz 2024-09-21T07:55:39.879 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.55595.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55595.log.gz 2024-09-21T07:55:39.879 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106704.log 2024-09-21T07:55:39.880 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111116.log 2024-09-21T07:55:39.880 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54193.log.gz 2024-09-21T07:55:39.880 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.106704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94804.log 2024-09-21T07:55:39.881 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.111116.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.106704.log.gz 2024-09-21T07:55:39.882 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.111116.log.gz 2024-09-21T07:55:39.882 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60213.log 2024-09-21T07:55:39.883 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.94804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53720.log 2024-09-21T07:55:39.883 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.94804.log.gz 2024-09-21T07:55:39.884 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53930.log 2024-09-21T07:55:39.884 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.60213.log.gz 2024-09-21T07:55:39.884 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53720.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53720.log.gz 2024-09-21T07:55:39.885 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101966.log 2024-09-21T07:55:39.885 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99256.log 2024-09-21T07:55:39.886 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53930.log.gz 2024-09-21T07:55:39.887 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.101966.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.101966.log.gz 2024-09-21T07:55:39.887 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101222.log 2024-09-21T07:55:39.888 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.99256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105732.log 2024-09-21T07:55:39.888 INFO:teuthology.orchestra.run.smithi039.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.99256.log.gz 2024-09-21T07:55:39.889 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.101222.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.101222.log.gz 2024-09-21T07:55:39.889 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79011.log 2024-09-21T07:55:39.890 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.105732.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.105732.log.gz 2024-09-21T07:55:39.890 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79263.log 2024-09-21T07:55:39.891 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.79011.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.79011.log.gz 2024-09-21T07:55:39.891 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43007.log 2024-09-21T07:55:39.892 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.79263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50987.log 2024-09-21T07:55:39.892 INFO:teuthology.orchestra.run.smithi039.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.79263.log.gz 2024-09-21T07:55:39.893 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43007.log: 87.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.43007.log.gz 2024-09-21T07:55:39.893 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.68487.log 2024-09-21T07:55:39.894 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.50987.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50987.log.gz 2024-09-21T07:55:39.894 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73463.log 2024-09-21T07:55:39.894 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68487.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.68487.log.gz 2024-09-21T07:55:39.895 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84513.log 2024-09-21T07:55:39.895 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.73463.log: gzip 87.7% -- replaced with /var/log/ceph/ceph-client.admin.73463.log.gz -5 2024-09-21T07:55:39.895 INFO:teuthology.orchestra.run.smithi039.stderr: --verbose -- /var/log/ceph/ceph-client.admin.52288.log 2024-09-21T07:55:39.896 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.84513.log: gzip 87.7% -- replaced with /var/log/ceph/ceph-client.admin.84513.log.gz 2024-09-21T07:55:39.896 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.35645.log 2024-09-21T07:55:39.897 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52288.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.52288.log.gz 2024-09-21T07:55:39.897 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.79975.log 2024-09-21T07:55:39.898 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35645.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35645.log.gz 2024-09-21T07:55:39.898 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112860.log 2024-09-21T07:55:39.899 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.79975.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.79975.log.gz 2024-09-21T07:55:39.899 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99695.log 2024-09-21T07:55:39.900 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.112860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67319.log 2024-09-21T07:55:39.900 INFO:teuthology.orchestra.run.smithi039.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.112860.log.gz 2024-09-21T07:55:39.901 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.99695.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.99695.log.gz 2024-09-21T07:55:39.901 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105606.log 2024-09-21T07:55:39.902 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.67319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61997.log 2024-09-21T07:55:39.902 INFO:teuthology.orchestra.run.smithi039.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.67319.log.gz 2024-09-21T07:55:39.903 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.105606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110207.log 2024-09-21T07:55:39.903 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.105606.log.gz 2024-09-21T07:55:39.904 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.61997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42836.log 2024-09-21T07:55:39.904 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61997.log.gz 2024-09-21T07:55:39.905 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.110207.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.110207.log.gz 2024-09-21T07:55:39.905 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108797.log 2024-09-21T07:55:39.906 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42836.log: gzip -5 87.4% --verbose -- replaced with /var/log/ceph/ceph-client.admin.42836.log.gz -- 2024-09-21T07:55:39.906 INFO:teuthology.orchestra.run.smithi039.stderr: /var/log/ceph/ceph-client.admin.96370.log 2024-09-21T07:55:39.906 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.108797.log: 87.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.108797.log.gz 2024-09-21T07:55:39.907 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.foo.113767.log 2024-09-21T07:55:39.907 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.96370.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.96370.log.gz 2024-09-21T07:55:39.907 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47315.log 2024-09-21T07:55:39.908 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.foo.113767.log: 0.0% -- replaced with /var/log/ceph/ceph-client.foo.113767.log.gz 2024-09-21T07:55:39.908 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110981.log 2024-09-21T07:55:39.909 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47315.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47315.log.gz 2024-09-21T07:55:39.909 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64077.log 2024-09-21T07:55:39.910 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.110981.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.110981.log.gz 2024-09-21T07:55:39.910 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59784.log 2024-09-21T07:55:39.911 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.64077.log: 87.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.64077.log.gz 2024-09-21T07:55:39.911 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.92066.log 2024-09-21T07:55:39.912 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.59784.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59784.log.gz 2024-09-21T07:55:39.912 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48141.log 2024-09-21T07:55:39.913 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.92066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102477.log 2024-09-21T07:55:39.913 INFO:teuthology.orchestra.run.smithi039.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.92066.log.gz 2024-09-21T07:55:39.914 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101402.log 89.9% 2024-09-21T07:55:39.914 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48141.log.gz 2024-09-21T07:55:39.915 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.102477.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.102477.log.gz 2024-09-21T07:55:39.915 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68613.log 2024-09-21T07:55:39.916 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.101402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112385.log 2024-09-21T07:55:39.916 INFO:teuthology.orchestra.run.smithi039.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.101402.log.gz 2024-09-21T07:55:39.917 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68613.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.68613.log.gz 2024-09-21T07:55:39.917 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95385.log 2024-09-21T07:55:39.918 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.112385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97271.log 2024-09-21T07:55:39.918 INFO:teuthology.orchestra.run.smithi039.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.112385.log.gz 2024-09-21T07:55:39.919 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.95385.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.95385.log.gz 2024-09-21T07:55:39.919 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53449.log 2024-09-21T07:55:39.920 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.97271.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.97271.log.gz 2024-09-21T07:55:39.920 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111728.log 2024-09-21T07:55:39.921 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112033.log 2024-09-21T07:55:39.921 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53449.log.gz 2024-09-21T07:55:39.922 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.111728.log: 87.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.111728.log.gz 2024-09-21T07:55:39.922 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.112478.log 2024-09-21T07:55:39.923 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.112033.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.112033.log.gz 2024-09-21T07:55:39.923 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70332.log 2024-09-21T07:55:39.923 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.112478.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.112478.log.gz 2024-09-21T07:55:39.923 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.105188.log 2024-09-21T07:55:39.924 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.70332.log: gzip 87.1% -- replaced with /var/log/ceph/ceph-client.admin.70332.log.gz 2024-09-21T07:55:39.924 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.104456.log 2024-09-21T07:55:39.925 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.105188.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.105188.log.gz 2024-09-21T07:55:39.925 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48800.log 2024-09-21T07:55:39.926 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.104456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73164.log 2024-09-21T07:55:39.927 INFO:teuthology.orchestra.run.smithi039.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.104456.log.gz 2024-09-21T07:55:39.927 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48800.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.48800.log.gz 2024-09-21T07:55:39.927 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71751.log 2024-09-21T07:55:39.928 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.73164.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.73164.log.gz 2024-09-21T07:55:39.928 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48427.log 2024-09-21T07:55:39.929 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.71751.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.71751.log.gz 2024-09-21T07:55:39.929 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49052.log 2024-09-21T07:55:39.930 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48427.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48427.log.gz 2024-09-21T07:55:39.930 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56254.log 2024-09-21T07:55:39.931 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49052.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49052.log.gz 2024-09-21T07:55:39.931 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114460.log 2024-09-21T07:55:39.932 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56254.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.56254.log.gz 2024-09-21T07:55:39.932 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57200.log 2024-09-21T07:55:39.933 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.114460.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.114460.log.gz 2024-09-21T07:55:39.933 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105816.log 2024-09-21T07:55:39.934 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.57200.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57200.log.gz 2024-09-21T07:55:39.934 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72870.log 2024-09-21T07:55:39.935 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.105816.log: 87.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.105816.log.gz 2024-09-21T07:55:39.935 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.93526.log 2024-09-21T07:55:39.936 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.72870.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.72870.log.gz 2024-09-21T07:55:39.936 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71793.log 2024-09-21T07:55:39.937 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.93526.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.93526.log.gz 2024-09-21T07:55:39.937 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46659.log 2024-09-21T07:55:39.938 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.71793.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71793.log.gz 2024-09-21T07:55:39.938 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108407.log 2024-09-21T07:55:39.939 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46659.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46659.log.gz 2024-09-21T07:55:39.939 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100288.log 2024-09-21T07:55:39.939 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.108407.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.108407.log.gz 2024-09-21T07:55:39.939 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48758.log 2024-09-21T07:55:39.940 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.100288.log: gzip 87.3% -- replaced with /var/log/ceph/ceph-client.admin.100288.log.gz -5 2024-09-21T07:55:39.940 INFO:teuthology.orchestra.run.smithi039.stderr: --verbose -- /var/log/ceph/ceph-client.admin.93232.log 2024-09-21T07:55:39.941 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48758.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48758.log.gz 2024-09-21T07:55:39.941 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68739.log 2024-09-21T07:55:39.942 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.93232.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.93232.log.gz 2024-09-21T07:55:39.942 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97792.log 2024-09-21T07:55:39.943 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68739.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.68739.log.gz 2024-09-21T07:55:39.943 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106296.log 2024-09-21T07:55:39.944 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.97792.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.97792.log.gz 2024-09-21T07:55:39.944 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91684.log 2024-09-21T07:55:39.945 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.106296.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.106296.log.gz 2024-09-21T07:55:39.945 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102933.log 2024-09-21T07:55:39.946 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.91684.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.91684.log.gz 2024-09-21T07:55:39.946 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51113.log 2024-09-21T07:55:39.947 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.102933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91814.log 2024-09-21T07:55:39.947 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.102933.log.gz 2024-09-21T07:55:39.948 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51113.log: gzip 87.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.51113.log.gz 2024-09-21T07:55:39.948 INFO:teuthology.orchestra.run.smithi039.stderr: --verbose -- /var/log/ceph/ceph-client.admin.91591.log 2024-09-21T07:55:39.949 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.91814.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.91814.log.gz 2024-09-21T07:55:39.949 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59722.log 2024-09-21T07:55:39.950 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.91591.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.91591.log.gz 2024-09-21T07:55:39.950 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60594.log 2024-09-21T07:55:39.951 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.59722.log: gzip 87.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.59722.log.gz --verbose 2024-09-21T07:55:39.951 INFO:teuthology.orchestra.run.smithi039.stderr: -- /var/log/ceph/ceph-client.0.55256.log 2024-09-21T07:55:39.951 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60594.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.60594.log.gz 2024-09-21T07:55:39.951 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53323.log 2024-09-21T07:55:39.952 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.0.55256.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.55256.log.gz 2024-09-21T07:55:39.952 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46392.log 2024-09-21T07:55:39.953 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53323.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53323.log.gz 2024-09-21T07:55:39.953 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73038.log 2024-09-21T07:55:39.954 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46392.log: gzip -5 87.2% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.46392.log.gz 2024-09-21T07:55:39.954 INFO:teuthology.orchestra.run.smithi039.stderr: /var/log/ceph/ceph-client.admin.105867.log 2024-09-21T07:55:39.955 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.73038.log: gzip 87.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.73038.log.gz 2024-09-21T07:55:39.955 INFO:teuthology.orchestra.run.smithi039.stderr: --verbose -- /var/log/ceph/ceph-client.admin.90411.log 2024-09-21T07:55:39.956 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.105867.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.105867.log.gz 2024-09-21T07:55:39.956 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53533.log 2024-09-21T07:55:39.957 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.90411.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.90411.log.gz 2024-09-21T07:55:39.957 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74238.log 2024-09-21T07:55:39.958 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53533.log: gzip 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53533.log.gz 2024-09-21T07:55:39.958 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36286.log 2024-09-21T07:55:39.959 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.74238.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74238.log.gz 2024-09-21T07:55:39.959 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58027.log 2024-09-21T07:55:39.960 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36286.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.36286.log.gz 2024-09-21T07:55:39.960 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94548.log 2024-09-21T07:55:39.961 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.58027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90766.log 2024-09-21T07:55:39.961 INFO:teuthology.orchestra.run.smithi039.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.58027.log.gz 2024-09-21T07:55:39.962 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.94548.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.94548.log.gz 2024-09-21T07:55:39.962 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56296.log 2024-09-21T07:55:39.963 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.90766.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.90766.log.gz 2024-09-21T07:55:39.963 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96636.log 2024-09-21T07:55:39.963 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56296.log: gzip 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56296.log.gz 2024-09-21T07:55:39.964 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.109396.log 2024-09-21T07:55:39.964 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.96636.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.96636.log.gzgzip 2024-09-21T07:55:39.964 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.71083.log 2024-09-21T07:55:39.965 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.109396.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.109396.log.gz 2024-09-21T07:55:39.965 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71997.log 2024-09-21T07:55:39.966 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.71083.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71083.log.gz 2024-09-21T07:55:39.966 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36197.log 2024-09-21T07:55:39.967 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.71997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79582.log 2024-09-21T07:55:39.967 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.71997.log.gz 2024-09-21T07:55:39.968 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36197.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.36197.log.gz 2024-09-21T07:55:39.968 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48385.log 2024-09-21T07:55:39.969 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.79582.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.79582.log.gz 2024-09-21T07:55:39.969 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95343.log 2024-09-21T07:55:39.970 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48385.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48385.log.gz 2024-09-21T07:55:39.970 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68312.log 2024-09-21T07:55:39.971 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.95343.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.95343.log.gz 2024-09-21T07:55:39.971 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50885.log 2024-09-21T07:55:39.972 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66963.log 2024-09-21T07:55:39.972 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.68312.log.gz 2024-09-21T07:55:39.973 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.50885.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50885.log.gz 2024-09-21T07:55:39.973 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94068.log 2024-09-21T07:55:39.974 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.66963.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66963.log.gz 2024-09-21T07:55:39.974 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61813.log 2024-09-21T07:55:39.975 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.94068.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.94068.log.gz 2024-09-21T07:55:39.975 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44336.log 2024-09-21T07:55:39.975 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.61813.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61813.log.gz 2024-09-21T07:55:39.976 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100701.log 2024-09-21T07:55:39.976 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.44336.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44336.log.gz 2024-09-21T07:55:39.977 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64921.log 2024-09-21T07:55:39.977 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.100701.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.100701.log.gz 2024-09-21T07:55:39.977 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78478.log 2024-09-21T07:55:39.978 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.64921.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.64921.log.gz 2024-09-21T07:55:39.978 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64189.log 2024-09-21T07:55:39.979 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.78478.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.78478.log.gz 2024-09-21T07:55:39.979 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71000.log 2024-09-21T07:55:39.980 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.64189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111896.log 2024-09-21T07:55:39.980 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.64189.log.gz 2024-09-21T07:55:39.981 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.71000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62906.log 2024-09-21T07:55:39.981 INFO:teuthology.orchestra.run.smithi039.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.71000.log.gz 2024-09-21T07:55:39.982 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.111896.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.111896.log.gz 2024-09-21T07:55:39.982 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103017.log 2024-09-21T07:55:39.983 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.0.62906.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.62906.log.gz 2024-09-21T07:55:39.983 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73891.log 2024-09-21T07:55:39.984 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.103017.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.103017.log.gz 2024-09-21T07:55:39.984 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73547.log 2024-09-21T07:55:39.985 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.73891.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.73891.log.gz 2024-09-21T07:55:39.985 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96930.log 2024-09-21T07:55:39.986 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.73547.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.73547.log.gz 2024-09-21T07:55:39.986 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115343.log 2024-09-21T07:55:39.987 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.96930.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.96930.log.gz 2024-09-21T07:55:39.987 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94163.log 2024-09-21T07:55:39.988 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.115343.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.115343.log.gz 2024-09-21T07:55:39.988 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104309.log 2024-09-21T07:55:39.989 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.94163.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.94163.log.gz 2024-09-21T07:55:39.989 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63615.log 2024-09-21T07:55:39.990 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.104309.log: gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.104309.log.gz -5 2024-09-21T07:55:39.990 INFO:teuthology.orchestra.run.smithi039.stderr: --verbose -- /var/log/ceph/ceph-client.admin.54098.log 2024-09-21T07:55:39.991 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63615.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.63615.log.gz 2024-09-21T07:55:39.991 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80143.log 2024-09-21T07:55:39.991 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54098.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54098.log.gz 2024-09-21T07:55:39.991 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105690.log 2024-09-21T07:55:39.992 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.80143.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.80143.log.gz 2024-09-21T07:55:39.992 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86160.log 2024-09-21T07:55:39.993 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.105690.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.105690.log.gz 2024-09-21T07:55:39.993 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60552.log 2024-09-21T07:55:39.994 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.86160.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.86160.log.gz 2024-09-21T07:55:39.994 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53281.log 2024-09-21T07:55:39.995 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60552.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.60552.log.gz 2024-09-21T07:55:39.995 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63341.log 2024-09-21T07:55:39.996 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113263.log 2024-09-21T07:55:39.996 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53281.log.gz 2024-09-21T07:55:39.997 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63341.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.63341.log.gz 2024-09-21T07:55:39.997 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56338.log 2024-09-21T07:55:39.998 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.113263.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.113263.log.gz 2024-09-21T07:55:39.998 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55899.log 2024-09-21T07:55:39.999 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56338.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56338.log.gz 2024-09-21T07:55:39.999 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62383.log 2024-09-21T07:55:40.000 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.55899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59347.log 2024-09-21T07:55:40.000 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55899.log.gz 2024-09-21T07:55:40.001 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.62383.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62383.log.gz 2024-09-21T07:55:40.001 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85946.log 2024-09-21T07:55:40.002 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.59347.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59347.log.gz 2024-09-21T07:55:40.002 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108059.log 2024-09-21T07:55:40.003 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.85946.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.85946.log.gz 2024-09-21T07:55:40.003 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61526.log 2024-09-21T07:55:40.003 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.108059.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.108059.log.gz 2024-09-21T07:55:40.004 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48119.log 2024-09-21T07:55:40.004 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.61526.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.61526.log.gz 2024-09-21T07:55:40.005 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102663.log 2024-09-21T07:55:40.005 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48119.log: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.48119.log.gz 2024-09-21T07:55:40.006 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62428.log 2024-09-21T07:55:40.006 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.102663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104925.log 2024-09-21T07:55:40.007 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.102663.log.gz 2024-09-21T07:55:40.007 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.62428.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.62428.log.gz 2024-09-21T07:55:40.007 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45429.log 2024-09-21T07:55:40.008 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.104925.log: gzip 87.4% -- replaced with /var/log/ceph/ceph-client.admin.104925.log.gz 2024-09-21T07:55:40.008 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.69983.log 2024-09-21T07:55:40.009 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.45429.log: gzip -5 --verbose 87.4% -- -- replaced with /var/log/ceph/ceph-client.admin.45429.log.gz /var/log/ceph/ceph-client.admin.59548.log 2024-09-21T07:55:40.009 INFO:teuthology.orchestra.run.smithi039.stderr: 2024-09-21T07:55:40.010 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.69983.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.69983.log.gz 2024-09-21T07:55:40.010 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109734.log 2024-09-21T07:55:40.011 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.59548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108147.log 2024-09-21T07:55:40.011 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59548.log.gz 2024-09-21T07:55:40.012 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.109734.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.109734.log.gz 2024-09-21T07:55:40.012 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114418.log 2024-09-21T07:55:40.013 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.108147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66678.log 2024-09-21T07:55:40.013 INFO:teuthology.orchestra.run.smithi039.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.108147.log.gz 2024-09-21T07:55:40.014 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.114418.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.114418.log.gz 2024-09-21T07:55:40.014 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54620.log 2024-09-21T07:55:40.015 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.66678.log: gzip -5 --verbose 87.4% -- -- replaced with /var/log/ceph/ceph-client.admin.66678.log.gz 2024-09-21T07:55:40.015 INFO:teuthology.orchestra.run.smithi039.stderr: /var/log/ceph/ceph-client.admin.70045.log 2024-09-21T07:55:40.015 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54620.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54620.log.gz 2024-09-21T07:55:40.016 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74284.log 2024-09-21T07:55:40.016 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.70045.log: 87.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.70045.log.gz 2024-09-21T07:55:40.016 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.99947.log 2024-09-21T07:55:40.017 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.74284.log: 87.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.74284.log.gz 2024-09-21T07:55:40.017 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.50166.log 2024-09-21T07:55:40.018 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.99947.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.99947.log.gz 2024-09-21T07:55:40.018 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53972.log 2024-09-21T07:55:40.019 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.50166.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50166.log.gz 2024-09-21T07:55:40.019 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94243.log 2024-09-21T07:55:40.020 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53972.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53972.log.gz 2024-09-21T07:55:40.020 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104803.log 2024-09-21T07:55:40.021 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.94243.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.94243.log.gz 2024-09-21T07:55:40.021 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63657.log 2024-09-21T07:55:40.022 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.104803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84817.log 2024-09-21T07:55:40.022 INFO:teuthology.orchestra.run.smithi039.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.104803.log.gz 2024-09-21T07:55:40.023 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63657.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.63657.log.gz 2024-09-21T07:55:40.023 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60051.log 2024-09-21T07:55:40.024 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.84817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115719.log 2024-09-21T07:55:40.024 INFO:teuthology.orchestra.run.smithi039.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.84817.log.gz 2024-09-21T07:55:40.025 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60051.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60051.log.gz 2024-09-21T07:55:40.025 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42412.log 2024-09-21T07:55:40.026 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.115719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48595.log 2024-09-21T07:55:40.026 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.115719.log.gz 2024-09-21T07:55:40.027 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42412.log: 87.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.42412.log.gz 2024-09-21T07:55:40.027 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.100743.log 2024-09-21T07:55:40.027 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114502.log 2024-09-21T07:55:40.028 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48595.log.gz 2024-09-21T07:55:40.028 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.100743.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.100743.log.gz 2024-09-21T07:55:40.029 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109312.log 2024-09-21T07:55:40.029 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.114502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96244.log 2024-09-21T07:55:40.030 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.114502.log.gz 2024-09-21T07:55:40.030 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.109312.log: gzip 86.9% -5 -- replaced with /var/log/ceph/ceph-client.admin.109312.log.gz 2024-09-21T07:55:40.030 INFO:teuthology.orchestra.run.smithi039.stderr: --verbose -- /var/log/ceph/ceph-client.admin.93942.log 2024-09-21T07:55:40.031 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.96244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.108985.log 2024-09-21T07:55:40.031 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.96244.log.gz 2024-09-21T07:55:40.032 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.93942.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.93942.log.gz 2024-09-21T07:55:40.032 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44672.log 2024-09-21T07:55:40.033 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.0.108985.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.108985.log.gz 2024-09-21T07:55:40.033 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36084.log 2024-09-21T07:55:40.034 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.0.44672.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44672.log.gz 2024-09-21T07:55:40.034 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91460.log 2024-09-21T07:55:40.035 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36084.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-21T07:55:40.035 INFO:teuthology.orchestra.run.smithi039.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.36084.log.gz 2024-09-21T07:55:40.036 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.91460.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.91460.log.gz 2024-09-21T07:55:40.036 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84775.log 2024-09-21T07:55:40.037 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70290.log 2024-09-21T07:55:40.038 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.84775.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.84775.log.gz 2024-09-21T07:55:40.038 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91162.log 2024-09-21T07:55:40.039 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.70290.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.70290.log.gz 2024-09-21T07:55:40.039 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106766.log 2024-09-21T07:55:40.040 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.91162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110373.log 2024-09-21T07:55:40.041 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.91162.log.gz 2024-09-21T07:55:40.042 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.106766.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.106766.log.gz 2024-09-21T07:55:40.042 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90808.log 2024-09-21T07:55:40.043 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.110373.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.110373.log.gz 2024-09-21T07:55:40.043 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42542.log 2024-09-21T07:55:40.044 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.90808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55689.log 2024-09-21T07:55:40.044 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.90808.log.gz 2024-09-21T07:55:40.045 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42542.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42542.log.gz 2024-09-21T07:55:40.045 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43688.log 2024-09-21T07:55:40.046 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.55689.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.55689.log.gz 2024-09-21T07:55:40.046 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65888.log 2024-09-21T07:55:40.047 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107975.log 2024-09-21T07:55:40.047 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43688.log.gz 2024-09-21T07:55:40.048 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.65888.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65888.log.gz 2024-09-21T07:55:40.048 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74057.log 2024-09-21T07:55:40.049 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.107975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44180.log 2024-09-21T07:55:40.050 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.107975.log.gz 2024-09-21T07:55:40.051 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.74057.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.74057.log.gz 2024-09-21T07:55:40.051 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94453.log 2024-09-21T07:55:40.051 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.44180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114105.log 2024-09-21T07:55:40.053 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.94453.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.44180.log.gz 2024-09-21T07:55:40.053 INFO:teuthology.orchestra.run.smithi039.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.94453.log.gz 2024-09-21T07:55:40.053 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64471.log 2024-09-21T07:55:40.054 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.114105.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.114105.log.gz 2024-09-21T07:55:40.054 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47441.log 2024-09-21T07:55:40.055 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.0.64471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74013.log 2024-09-21T07:55:40.055 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.64471.log.gz 2024-09-21T07:55:40.056 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47441.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47441.log.gz 2024-09-21T07:55:40.056 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35989.log 2024-09-21T07:55:40.057 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.74013.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.74013.log.gz 2024-09-21T07:55:40.057 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78866.log 2024-09-21T07:55:40.058 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63927.log 2024-09-21T07:55:40.059 INFO:teuthology.orchestra.run.smithi039.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.35989.log.gz 2024-09-21T07:55:40.060 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.78866.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.78866.log.gz 2024-09-21T07:55:40.060 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52162.log 2024-09-21T07:55:40.060 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63927.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.63927.log.gz 2024-09-21T07:55:40.061 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102725.log 2024-09-21T07:55:40.062 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99214.log 2024-09-21T07:55:40.062 INFO:teuthology.orchestra.run.smithi039.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52162.log.gz 2024-09-21T07:55:40.063 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.102725.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.102725.log.gz 2024-09-21T07:55:40.063 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97540.log 2024-09-21T07:55:40.064 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46878.log 2024-09-21T07:55:40.064 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.99214.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.99214.log.gz 2024-09-21T07:55:40.065 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.97540.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.97540.log.gz 2024-09-21T07:55:40.065 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47006.log 2024-09-21T07:55:40.066 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69390.log 2024-09-21T07:55:40.066 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46878.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46878.log.gz 2024-09-21T07:55:40.067 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47006.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47006.log.gz 2024-09-21T07:55:40.067 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35850.log 2024-09-21T07:55:40.068 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53678.log 2024-09-21T07:55:40.068 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.0.69390.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.69390.log.gz 2024-09-21T07:55:40.069 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35850.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.35850.log.gz 2024-09-21T07:55:40.069 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72104.log 2024-09-21T07:55:40.070 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54527.log 2024-09-21T07:55:40.071 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53678.log.gz 2024-09-21T07:55:40.071 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.72104.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.72104.log.gz 2024-09-21T07:55:40.071 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59221.log 2024-09-21T07:55:40.072 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53804.log 2024-09-21T07:55:40.073 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54527.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54527.log.gz 2024-09-21T07:55:40.073 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.59221.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.59221.log.gz 2024-09-21T07:55:40.073 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91503.log 2024-09-21T07:55:40.074 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98110.log 2024-09-21T07:55:40.074 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53804.log.gz 2024-09-21T07:55:40.075 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.91503.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.91503.log.gz 2024-09-21T07:55:40.075 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85599.log 2024-09-21T07:55:40.076 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.98110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62341.log 2024-09-21T07:55:40.077 INFO:teuthology.orchestra.run.smithi039.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.98110.log.gz 2024-09-21T07:55:40.077 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.85599.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.85599.log.gz 2024-09-21T07:55:40.077 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43010.log 2024-09-21T07:55:40.078 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.62341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49311.log 2024-09-21T07:55:40.079 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62341.log.gz 2024-09-21T07:55:40.079 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43010.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43010.log.gz 2024-09-21T07:55:40.079 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52456.log 2024-09-21T07:55:40.080 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109438.log 2024-09-21T07:55:40.081 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49311.log.gz 2024-09-21T07:55:40.081 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52456.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52456.log.gz 2024-09-21T07:55:40.082 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80269.log 2024-09-21T07:55:40.083 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47947.log 2024-09-21T07:55:40.083 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.109438.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.109438.log.gz 2024-09-21T07:55:40.084 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.80269.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.80269.log.gz 2024-09-21T07:55:40.084 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51029.log 2024-09-21T07:55:40.085 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.0.47947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104522.log 2024-09-21T07:55:40.085 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47947.log.gz 2024-09-21T07:55:40.086 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51029.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51029.log.gz 2024-09-21T07:55:40.086 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78656.log 2024-09-21T07:55:40.087 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.104522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45828.log 2024-09-21T07:55:40.088 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.78656.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.104522.log.gz 2024-09-21T07:55:40.088 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.78656.log.gz 2024-09-21T07:55:40.088 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50750.log 2024-09-21T07:55:40.089 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.45828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85424.log 2024-09-21T07:55:40.089 INFO:teuthology.orchestra.run.smithi039.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45828.log.gz 2024-09-21T07:55:40.090 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.50750.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50750.log.gz 2024-09-21T07:55:40.090 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67067.log 2024-09-21T07:55:40.091 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94720.log 2024-09-21T07:55:40.091 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.85424.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.85424.log.gz 2024-09-21T07:55:40.092 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.67067.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67067.log.gz 2024-09-21T07:55:40.092 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.92320.log 2024-09-21T07:55:40.093 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111812.log 2024-09-21T07:55:40.093 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.94720.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.94720.log.gz 2024-09-21T07:55:40.093 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.0.92320.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.92320.log.gz 2024-09-21T07:55:40.094 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115637.log 2024-09-21T07:55:40.095 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80227.log 2024-09-21T07:55:40.095 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.111812.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.111812.log.gz 2024-09-21T07:55:40.096 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.115637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70640.log 2024-09-21T07:55:40.096 INFO:teuthology.orchestra.run.smithi039.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.115637.log.gz 2024-09-21T07:55:40.097 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.80227.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.80227.log.gz 2024-09-21T07:55:40.097 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96678.log 2024-09-21T07:55:40.098 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.70640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112608.log 2024-09-21T07:55:40.099 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.70640.log.gz 2024-09-21T07:55:40.099 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.96678.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.96678.log.gz 2024-09-21T07:55:40.100 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70524.log 2024-09-21T07:55:40.100 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.112608.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.112608.log.gz 2024-09-21T07:55:40.101 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46752.log 2024-09-21T07:55:40.102 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.70524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65526.log 2024-09-21T07:55:40.103 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46752.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.70524.log.gz 2024-09-21T07:55:40.103 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65747.log 87.4% 2024-09-21T07:55:40.103 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46752.log.gz 2024-09-21T07:55:40.104 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.65526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90453.log 2024-09-21T07:55:40.104 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.65526.log.gz 2024-09-21T07:55:40.105 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.65747.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.65747.log.gz 2024-09-21T07:55:40.105 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88401.log 2024-09-21T07:55:40.106 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.90453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60510.log 2024-09-21T07:55:40.107 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.90453.log.gz 2024-09-21T07:55:40.107 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.88401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46163.log 2024-09-21T07:55:40.108 INFO:teuthology.orchestra.run.smithi039.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.88401.log.gz 2024-09-21T07:55:40.109 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60510.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.60510.log.gz 2024-09-21T07:55:40.109 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111242.log 2024-09-21T07:55:40.109 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84733.log 2024-09-21T07:55:40.110 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46163.log.gz 2024-09-21T07:55:40.111 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.111242.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.111242.log.gz 2024-09-21T07:55:40.111 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78698.log 2024-09-21T07:55:40.112 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.84733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79748.log 2024-09-21T07:55:40.112 INFO:teuthology.orchestra.run.smithi039.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.84733.log.gz 2024-09-21T07:55:40.113 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.78698.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.78698.log.gz 2024-09-21T07:55:40.113 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65705.log 2024-09-21T07:55:40.114 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.79748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97876.log 2024-09-21T07:55:40.115 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.79748.log.gz 2024-09-21T07:55:40.115 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.65705.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.65705.log.gz 2024-09-21T07:55:40.115 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102601.log 2024-09-21T07:55:40.116 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.97876.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.97876.log.gz 2024-09-21T07:55:40.117 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65238.log 2024-09-21T07:55:40.117 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.102601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36625.log 2024-09-21T07:55:40.118 INFO:teuthology.orchestra.run.smithi039.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.102601.log.gz 2024-09-21T07:55:40.119 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.65238.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65238.log.gz 2024-09-21T07:55:40.119 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94930.log 2024-09-21T07:55:40.120 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68270.log 2024-09-21T07:55:40.120 INFO:teuthology.orchestra.run.smithi039.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.36625.log.gz 2024-09-21T07:55:40.121 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.94930.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.94930.log.gz 2024-09-21T07:55:40.121 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93984.log 2024-09-21T07:55:40.122 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57874.log 2024-09-21T07:55:40.123 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68270.log.gz 2024-09-21T07:55:40.123 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.93984.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.93984.log.gz 2024-09-21T07:55:40.124 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109213.log 2024-09-21T07:55:40.124 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.57874.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57874.log.gz 2024-09-21T07:55:40.125 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61427.log 2024-09-21T07:55:40.126 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.109213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80833.log 2024-09-21T07:55:40.127 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.61427.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.109213.log.gz 2024-09-21T07:55:40.127 INFO:teuthology.orchestra.run.smithi039.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.61427.log.gz 2024-09-21T07:55:40.127 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99036.log 2024-09-21T07:55:40.128 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.80833.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.80833.log.gz 2024-09-21T07:55:40.128 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69737.log 2024-09-21T07:55:40.129 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.99036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115007.log 2024-09-21T07:55:40.130 INFO:teuthology.orchestra.run.smithi039.stderr: 87.8%/var/log/ceph/ceph-client.admin.69737.log: -- replaced with /var/log/ceph/ceph-client.admin.99036.log.gz 2024-09-21T07:55:40.130 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69737.log.gz 2024-09-21T07:55:40.130 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102849.log 2024-09-21T07:55:40.132 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.115007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94888.log 2024-09-21T07:55:40.132 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.115007.log.gz 2024-09-21T07:55:40.133 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.102849.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.102849.log.gz 2024-09-21T07:55:40.133 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78614.log 2024-09-21T07:55:40.134 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.94888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59305.log 2024-09-21T07:55:40.134 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.94888.log.gz 2024-09-21T07:55:40.135 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.78614.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.78614.log.gz 2024-09-21T07:55:40.135 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104414.log 2024-09-21T07:55:40.136 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.59305.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59305.log.gz 2024-09-21T07:55:40.136 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92599.log 2024-09-21T07:55:40.137 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.104414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58189.log 2024-09-21T07:55:40.138 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.104414.log.gz 2024-09-21T07:55:40.138 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.92599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48637.log 2024-09-21T07:55:40.139 INFO:teuthology.orchestra.run.smithi039.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.92599.log.gz 2024-09-21T07:55:40.140 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.58189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86244.log 2024-09-21T07:55:40.140 INFO:teuthology.orchestra.run.smithi039.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.58189.log.gz 2024-09-21T07:55:40.141 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50208.log 2024-09-21T07:55:40.141 INFO:teuthology.orchestra.run.smithi039.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.48637.log.gz 2024-09-21T07:55:40.142 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.86244.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.86244.log.gz 2024-09-21T07:55:40.142 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61150.log 2024-09-21T07:55:40.143 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.50208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111644.log 2024-09-21T07:55:40.143 INFO:teuthology.orchestra.run.smithi039.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.50208.log.gz 2024-09-21T07:55:40.144 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.61150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48027.log 2024-09-21T07:55:40.145 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.111644.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.61150.log.gz 2024-09-21T07:55:40.145 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.111644.log.gz 2024-09-21T07:55:40.146 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53636.log 2024-09-21T07:55:40.147 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71532.log 2024-09-21T07:55:40.147 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48027.log.gz 2024-09-21T07:55:40.148 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53636.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53636.log.gz 2024-09-21T07:55:40.148 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59506.log 2024-09-21T07:55:40.149 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.71532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111200.log 2024-09-21T07:55:40.149 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71532.log.gz 2024-09-21T07:55:40.150 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.59506.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59506.log.gz 2024-09-21T07:55:40.150 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100445.log 2024-09-21T07:55:40.151 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.111200.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.111200.log.gz 2024-09-21T07:55:40.151 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43018.log 2024-09-21T07:55:40.152 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.100445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48060.log 2024-09-21T07:55:40.152 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.100445.log.gz 2024-09-21T07:55:40.154 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65982.log 2024-09-21T07:55:40.154 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43018.log.gz 2024-09-21T07:55:40.155 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48060.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.48060.log.gz 2024-09-21T07:55:40.155 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93774.log 2024-09-21T07:55:40.155 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.65982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107470.log 2024-09-21T07:55:40.156 INFO:teuthology.orchestra.run.smithi039.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.65982.log.gz 2024-09-21T07:55:40.157 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.93774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73122.log 2024-09-21T07:55:40.157 INFO:teuthology.orchestra.run.smithi039.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.93774.log.gz 2024-09-21T07:55:40.158 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.107470.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.107470.log.gz 2024-09-21T07:55:40.158 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36234.log 2024-09-21T07:55:40.159 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.73122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47483.log 2024-09-21T07:55:40.159 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.73122.log.gz 2024-09-21T07:55:40.160 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71835.log 2024-09-21T07:55:40.160 INFO:teuthology.orchestra.run.smithi039.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.36234.log.gz 2024-09-21T07:55:40.161 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47483.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47483.log.gz 2024-09-21T07:55:40.162 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107549.log 2024-09-21T07:55:40.163 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.71835.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-21T07:55:40.163 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71835.log.gz 2024-09-21T07:55:40.164 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.107549.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.107549.log.gz 2024-09-21T07:55:40.164 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115427.log 2024-09-21T07:55:40.165 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79179.log 2024-09-21T07:55:40.166 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.115427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100617.log 2024-09-21T07:55:40.166 INFO:teuthology.orchestra.run.smithi039.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.115427.log.gz 2024-09-21T07:55:40.167 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.79179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66534.log 2024-09-21T07:55:40.168 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.79179.log.gz 2024-09-21T07:55:40.168 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.100617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115091.log 2024-09-21T07:55:40.169 INFO:teuthology.orchestra.run.smithi039.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.100617.log.gz 2024-09-21T07:55:40.169 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.66534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97582.log 2024-09-21T07:55:40.170 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.66534.log.gz 2024-09-21T07:55:40.171 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.115091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103861.log 2024-09-21T07:55:40.171 INFO:teuthology.orchestra.run.smithi039.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.115091.log.gz 2024-09-21T07:55:40.172 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.97582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62624.log 2024-09-21T07:55:40.172 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.97582.log.gz 2024-09-21T07:55:40.174 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.103861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59442.log 2024-09-21T07:55:40.174 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.103861.log.gz 2024-09-21T07:55:40.175 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.62624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49998.log 2024-09-21T07:55:40.175 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62624.log.gz 2024-09-21T07:55:40.176 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.59442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36183.log 2024-09-21T07:55:40.176 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59442.log.gz 2024-09-21T07:55:40.177 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58926.log 2024-09-21T07:55:40.177 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49998.log.gz 2024-09-21T07:55:40.179 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70922.log 2024-09-21T07:55:40.179 INFO:teuthology.orchestra.run.smithi039.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.36183.log.gz 2024-09-21T07:55:40.180 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.58926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52978.log 2024-09-21T07:55:40.180 INFO:teuthology.orchestra.run.smithi039.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.58926.log.gz 2024-09-21T07:55:40.180 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.0.70922.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.70922.log.gz 2024-09-21T07:55:40.181 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110845.log 2024-09-21T07:55:40.182 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59058.log 2024-09-21T07:55:40.182 INFO:teuthology.orchestra.run.smithi039.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.52978.log.gz 2024-09-21T07:55:40.183 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.110845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105437.log 2024-09-21T07:55:40.184 INFO:teuthology.orchestra.run.smithi039.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.110845.log.gz 2024-09-21T07:55:40.184 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.59058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107634.log 2024-09-21T07:55:40.185 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59058.log.gz 2024-09-21T07:55:40.186 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.105437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45918.log 2024-09-21T07:55:40.186 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.105437.log.gz 2024-09-21T07:55:40.187 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.107634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47759.log 2024-09-21T07:55:40.187 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.107634.log.gz 2024-09-21T07:55:40.188 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.45918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58842.log 2024-09-21T07:55:40.188 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45918.log.gz 2024-09-21T07:55:40.189 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112211.log 2024-09-21T07:55:40.189 INFO:teuthology.orchestra.run.smithi039.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47759.log.gz 2024-09-21T07:55:40.191 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.58842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98649.log 2024-09-21T07:55:40.191 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58842.log.gz 2024-09-21T07:55:40.192 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.112211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46037.log 2024-09-21T07:55:40.192 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.112211.log.gz 2024-09-21T07:55:40.193 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.98649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63843.log 2024-09-21T07:55:40.193 INFO:teuthology.orchestra.run.smithi039.stderr: 91.1%/var/log/ceph/ceph-client.admin.46037.log: -- replaced with /var/log/ceph/ceph-client.admin.98649.log.gz 2024-09-21T07:55:40.195 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62039.log 2024-09-21T07:55:40.195 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46037.log.gz 2024-09-21T07:55:40.195 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111284.log 2024-09-21T07:55:40.196 INFO:teuthology.orchestra.run.smithi039.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.63843.log.gz 2024-09-21T07:55:40.197 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.62039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105069.log 2024-09-21T07:55:40.197 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.62039.log.gz 2024-09-21T07:55:40.198 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.111284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70458.log 2024-09-21T07:55:40.198 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.111284.log.gz 2024-09-21T07:55:40.199 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.105069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109674.log 2024-09-21T07:55:40.199 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.105069.log.gz 2024-09-21T07:55:40.200 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.70458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51594.log 2024-09-21T07:55:40.200 INFO:teuthology.orchestra.run.smithi039.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.70458.log.gz 2024-09-21T07:55:40.201 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.109674.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.109674.log.gz 2024-09-21T07:55:40.201 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45555.log 2024-09-21T07:55:40.203 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51594.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51594.log.gz 2024-09-21T07:55:40.203 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.45555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103299.log 2024-09-21T07:55:40.204 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45555.log.gz 2024-09-21T07:55:40.204 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49094.log 2024-09-21T07:55:40.206 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.103299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61652.log 2024-09-21T07:55:40.206 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.103299.log.gz 2024-09-21T07:55:40.207 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107796.log 2024-09-21T07:55:40.207 INFO:teuthology.orchestra.run.smithi039.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49094.log.gz 2024-09-21T07:55:40.208 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.61652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84691.log 2024-09-21T07:55:40.208 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61652.log.gz 2024-09-21T07:55:40.209 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.107796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79836.log 2024-09-21T07:55:40.209 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.107796.log.gz 2024-09-21T07:55:40.211 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.84691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115677.log 2024-09-21T07:55:40.211 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.84691.log.gz 2024-09-21T07:55:40.214 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.79836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60901.log 2024-09-21T07:55:40.214 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.79836.log.gz 2024-09-21T07:55:40.215 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.115677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103191.log 2024-09-21T07:55:40.215 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.115677.log.gz 2024-09-21T07:55:40.215 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115469.log 2024-09-21T07:55:40.215 INFO:teuthology.orchestra.run.smithi039.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.60901.log.gz 2024-09-21T07:55:40.216 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.103191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104129.log 2024-09-21T07:55:40.217 INFO:teuthology.orchestra.run.smithi039.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.103191.log.gz 2024-09-21T07:55:40.217 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.115469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42710.log 2024-09-21T07:55:40.217 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.115469.log.gz 2024-09-21T07:55:40.218 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.104129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69822.log 2024-09-21T07:55:40.218 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.104129.log.gz 2024-09-21T07:55:40.219 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57410.log 2024-09-21T07:55:40.219 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42710.log.gz 2024-09-21T07:55:40.220 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.69822.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-21T07:55:40.221 INFO:teuthology.orchestra.run.smithi039.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.69822.log.gz 2024-09-21T07:55:40.222 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.57410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61385.log 2024-09-21T07:55:40.223 INFO:teuthology.orchestra.run.smithi039.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.57410.log.gz 2024-09-21T07:55:40.223 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65106.log 2024-09-21T07:55:40.224 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.61385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36277.log 2024-09-21T07:55:40.224 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61385.log.gz 2024-09-21T07:55:40.225 INFO:teuthology.orchestra.run.smithi148.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-21T07:55:40.226 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.65106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99863.log 2024-09-21T07:55:40.226 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.65106.log.gz 2024-09-21T07:55:40.227 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71594.log 2024-09-21T07:55:40.227 INFO:teuthology.orchestra.run.smithi039.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.36277.log.gz 2024-09-21T07:55:40.228 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.99863.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.99863.log.gz 2024-09-21T07:55:40.228 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99905.log 2024-09-21T07:55:40.229 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.71594.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.71594.log.gz 2024-09-21T07:55:40.230 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100394.log 2024-09-21T07:55:40.231 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.99905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42065.log 2024-09-21T07:55:40.231 INFO:teuthology.orchestra.run.smithi039.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.99905.log.gz 2024-09-21T07:55:40.232 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.100394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53187.log 2024-09-21T07:55:40.232 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.100394.log.gz 2024-09-21T07:55:40.234 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42065.log: gzip -5 --verbose 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42065.log.gz 2024-09-21T07:55:40.234 INFO:teuthology.orchestra.run.smithi039.stderr: -- /var/log/ceph/ceph-client.admin.43352.log 2024-09-21T07:55:40.235 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53187.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53187.log.gz 2024-09-21T07:55:40.235 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61042.log 2024-09-21T07:55:40.236 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47357.log 2024-09-21T07:55:40.236 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43352.log.gz 2024-09-21T07:55:40.237 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.61042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110485.log 2024-09-21T07:55:40.238 INFO:teuthology.orchestra.run.smithi039.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.61042.log.gz 2024-09-21T07:55:40.238 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47357.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47357.log.gz 2024-09-21T07:55:40.239 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80311.log 2024-09-21T07:55:40.240 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.110485.log: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100575.log 2024-09-21T07:55:40.240 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.110485.log.gz 2024-09-21T07:55:40.241 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.80311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71229.log 2024-09-21T07:55:40.241 INFO:teuthology.orchestra.run.smithi039.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.80311.log.gz 2024-09-21T07:55:40.242 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.100575.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.100575.log.gz 2024-09-21T07:55:40.243 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97313.log 2024-09-21T07:55:40.244 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.71229.log: 87.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51468.log 2024-09-21T07:55:40.244 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.71229.log.gz 2024-09-21T07:55:40.245 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.97313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50124.log 2024-09-21T07:55:40.245 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.97313.log.gz 2024-09-21T07:55:40.247 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66331.log 2024-09-21T07:55:40.247 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51468.log.gz 2024-09-21T07:55:40.248 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.50124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50559.log 2024-09-21T07:55:40.248 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50124.log.gz 2024-09-21T07:55:40.249 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.66331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43680.log 2024-09-21T07:55:40.249 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66331.log.gz 2024-09-21T07:55:40.250 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.50559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84962.log 2024-09-21T07:55:40.251 INFO:teuthology.orchestra.run.smithi039.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.50559.log.gz 2024-09-21T07:55:40.252 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64836.log 2024-09-21T07:55:40.252 INFO:teuthology.orchestra.run.smithi039.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43680.log.gz 2024-09-21T07:55:40.253 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.84962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60715.log 2024-09-21T07:55:40.253 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.84962.log.gz 2024-09-21T07:55:40.254 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.64836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114189.log 2024-09-21T07:55:40.254 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.64836.log.gz 2024-09-21T07:55:40.255 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56086.log 2024-09-21T07:55:40.256 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.60715.log.gz 2024-09-21T07:55:40.257 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.114189.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.114189.log.gz 2024-09-21T07:55:40.257 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46205.log 2024-09-21T07:55:40.258 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56086.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56086.log.gz 2024-09-21T07:55:40.258 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97834.log 2024-09-21T07:55:40.260 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66248.log 2024-09-21T07:55:40.260 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46205.log.gz 2024-09-21T07:55:40.261 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.97834.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.97834.log.gz 2024-09-21T07:55:40.262 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.113114.log 2024-09-21T07:55:40.263 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.66248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96160.log 2024-09-21T07:55:40.263 INFO:teuthology.orchestra.run.smithi039.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.66248.log.gz 2024-09-21T07:55:40.263 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.0.113114.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.113114.log.gz 2024-09-21T07:55:40.264 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95038.log 2024-09-21T07:55:40.265 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.96160.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.96160.log.gz 2024-09-21T07:55:40.266 INFO:teuthology.orchestra.run.smithi039.stderr:gzip/var/log/ceph/ceph-client.admin.95038.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.108101.log 2024-09-21T07:55:40.266 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.95038.log.gz 2024-09-21T07:55:40.267 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96510.log 2024-09-21T07:55:40.269 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.108101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92557.log 2024-09-21T07:55:40.269 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.108101.log.gz 2024-09-21T07:55:40.270 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.96510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60799.log 2024-09-21T07:55:40.270 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.96510.log.gz 2024-09-21T07:55:40.271 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.92557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49136.log 2024-09-21T07:55:40.271 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.92557.log.gz 2024-09-21T07:55:40.272 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56847.log 2024-09-21T07:55:40.273 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60799.log.gz 2024-09-21T07:55:40.274 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43344.log 2024-09-21T07:55:40.274 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49136.log.gz 2024-09-21T07:55:40.275 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80437.log 2024-09-21T07:55:40.275 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56847.log.gz 2024-09-21T07:55:40.276 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43344.log: gzip 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43344.log.gz -5 2024-09-21T07:55:40.276 INFO:teuthology.orchestra.run.smithi039.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50843.log 2024-09-21T07:55:40.277 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.80437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65166.log 2024-09-21T07:55:40.278 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.80437.log.gz 2024-09-21T07:55:40.279 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.50843.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50843.log.gzgzip 2024-09-21T07:55:40.279 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-21T07:55:40.280 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.65166.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.65166.log.gz 2024-09-21T07:55:40.280 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78572.log 2024-09-21T07:55:40.282 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50522.log 2024-09-21T07:55:40.283 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.78572.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.78572.log.gz 2024-09-21T07:55:40.284 INFO:teuthology.orchestra.run.smithi039.stderr:gzip/var/log/ceph/ceph-client.admin.50522.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.64677.log 2024-09-21T07:55:40.284 INFO:teuthology.orchestra.run.smithi039.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.50522.log.gz 2024-09-21T07:55:40.285 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49392.log 2024-09-21T07:55:40.287 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.64677.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.64677.log.gz 2024-09-21T07:55:40.287 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115385.log 2024-09-21T07:55:40.288 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49392.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49392.log.gz 2024-09-21T07:55:40.289 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97452.log 2024-09-21T07:55:40.290 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.115385.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.115385.log.gz 2024-09-21T07:55:40.291 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114544.log 2024-09-21T07:55:40.292 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.97452.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.97452.log.gz 2024-09-21T07:55:40.292 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79484.log 2024-09-21T07:55:40.294 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.114544.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.114544.log.gz 2024-09-21T07:55:40.295 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52330.log 2024-09-21T07:55:40.295 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.79484.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.79484.log.gz 2024-09-21T07:55:40.296 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48291.log 2024-09-21T07:55:40.298 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52330.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.52330.log.gz 2024-09-21T07:55:40.298 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103579.log 2024-09-21T07:55:40.299 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48291.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48291.log.gz 2024-09-21T07:55:40.300 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100245.log 2024-09-21T07:55:40.300 INFO:teuthology.orchestra.run.smithi039.stderr: 94.7% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-21T07:55:40.301 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.103579.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.103579.log.gz 2024-09-21T07:55:40.301 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64015.log 2024-09-21T07:55:40.303 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.100245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96412.log 87.6% 2024-09-21T07:55:40.303 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.100245.log.gz 2024-09-21T07:55:40.304 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.64015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91940.log 2024-09-21T07:55:40.304 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.64015.log.gz 2024-09-21T07:55:40.305 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.96412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61084.log 2024-09-21T07:55:40.305 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.96412.log.gz 2024-09-21T07:55:40.306 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.91940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48511.log 2024-09-21T07:55:40.307 INFO:teuthology.orchestra.run.smithi039.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.91940.log.gz 2024-09-21T07:55:40.308 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.61084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95481.log 2024-09-21T07:55:40.309 INFO:teuthology.orchestra.run.smithi039.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.61084.log.gz 2024-09-21T07:55:40.309 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48511.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48511.log.gz 2024-09-21T07:55:40.309 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43886.log 2024-09-21T07:55:40.310 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.95481.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.95481.log.gz 2024-09-21T07:55:40.310 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71472.log 2024-09-21T07:55:40.312 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65005.log 2024-09-21T07:55:40.312 INFO:teuthology.orchestra.run.smithi039.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43886.log.gz 2024-09-21T07:55:40.313 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.71472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43577.log 2024-09-21T07:55:40.313 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71472.log.gz 2024-09-21T07:55:40.314 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.65005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80479.log 2024-09-21T07:55:40.314 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.65005.log.gz 2024-09-21T07:55:40.316 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43577.log: gzip -5 --verbose -- 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43577.log.gz 2024-09-21T07:55:40.316 INFO:teuthology.orchestra.run.smithi039.stderr: /var/log/ceph/ceph-client.admin.95589.log 2024-09-21T07:55:40.317 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.80479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100042.log 90.9% -- replaced with /var/log/ceph/ceph-client.admin.80479.log.gz 2024-09-21T07:55:40.317 INFO:teuthology.orchestra.run.smithi039.stderr: 2024-09-21T07:55:40.319 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.95589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57326.log 2024-09-21T07:55:40.319 INFO:teuthology.orchestra.run.smithi039.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.95589.log.gz 2024-09-21T07:55:40.320 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.100042.log: gzip -5 --verbose 87.5% -- replaced with /var/log/ceph/ceph-client.admin.100042.log.gz 2024-09-21T07:55:40.320 INFO:teuthology.orchestra.run.smithi039.stderr: -- /var/log/ceph/ceph-client.admin.74620.log 2024-09-21T07:55:40.321 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.57326.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57326.log.gz 2024-09-21T07:55:40.322 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54750.log 2024-09-21T07:55:40.323 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.74620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63384.log 2024-09-21T07:55:40.323 INFO:teuthology.orchestra.run.smithi039.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.74620.log.gz 2024-09-21T07:55:40.324 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54750.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54750.log.gz 2024-09-21T07:55:40.324 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36264.log 2024-09-21T07:55:40.326 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63384.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-21T07:55:40.326 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.63384.log.gz 2024-09-21T07:55:40.327 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54578.log 2024-09-21T07:55:40.327 INFO:teuthology.orchestra.run.smithi039.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36264.log.gz 2024-09-21T07:55:40.329 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.54578.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.54578.log.gz 2024-09-21T07:55:40.330 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35662.log 2024-09-21T07:55:40.331 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52204.log 2024-09-21T07:55:40.332 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35662.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35662.log.gz 2024-09-21T07:55:40.333 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111560.log 2024-09-21T07:55:40.334 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52204.log: 87.4% 87.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-21T07:55:40.334 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52204.log.gz 2024-09-21T07:55:40.335 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101360.log 2024-09-21T07:55:40.336 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.111560.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.111560.log.gz 2024-09-21T07:55:40.336 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99821.log 2024-09-21T07:55:40.337 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.101360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71370.log 2024-09-21T07:55:40.337 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.101360.log.gz 2024-09-21T07:55:40.339 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.99821.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.99821.log.gz 2024-09-21T07:55:40.339 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63780.log 2024-09-21T07:55:40.340 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.71370.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.71370.log.gz 2024-09-21T07:55:40.340 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47525.log 2024-09-21T07:55:40.342 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51216.log 2024-09-21T07:55:40.342 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.63780.log.gz 2024-09-21T07:55:40.343 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50656.log 2024-09-21T07:55:40.343 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47525.log.gz 2024-09-21T07:55:40.344 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93484.log 2024-09-21T07:55:40.344 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51216.log.gz 2024-09-21T07:55:40.345 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.50656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96888.log 2024-09-21T07:55:40.346 INFO:teuthology.orchestra.run.smithi039.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.50656.log.gz 2024-09-21T07:55:40.347 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.93484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67169.log 2024-09-21T07:55:40.347 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.93484.log.gz 2024-09-21T07:55:40.348 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.96888.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.96888.log.gz 2024-09-21T07:55:40.348 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103257.log 2024-09-21T07:55:40.349 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.67169.log: gzip 87.2% -- replaced with /var/log/ceph/ceph-client.admin.67169.log.gz -5 2024-09-21T07:55:40.350 INFO:teuthology.orchestra.run.smithi039.stderr: --verbose -- /var/log/ceph/ceph-client.0.110767.log 2024-09-21T07:55:40.351 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.103257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43349.log 2024-09-21T07:55:40.351 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.103257.log.gz 2024-09-21T07:55:40.351 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.0.110767.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.110767.log.gz 2024-09-21T07:55:40.352 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73421.log 2024-09-21T07:55:40.353 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43349.log: gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43349.log.gz 2024-09-21T07:55:40.354 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.103534.log 2024-09-21T07:55:40.354 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.73421.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.73421.log.gz 2024-09-21T07:55:40.355 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80059.log 2024-09-21T07:55:40.356 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.103534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100827.log 2024-09-21T07:55:40.356 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.103534.log.gz 2024-09-21T07:55:40.357 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.80059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69468.log 2024-09-21T07:55:40.357 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.80059.log.gz 2024-09-21T07:55:40.358 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.100827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56212.log 2024-09-21T07:55:40.359 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.100827.log.gz 2024-09-21T07:55:40.360 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.69468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80101.log 2024-09-21T07:55:40.360 INFO:teuthology.orchestra.run.smithi039.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.69468.log.gz 2024-09-21T07:55:40.361 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56212.log: gzip 87.1% -5 --verbose -- /var/log/ceph/ceph-client.admin.46247.log 2024-09-21T07:55:40.361 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56212.log.gz 2024-09-21T07:55:40.362 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.80101.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.80101.log.gz 2024-09-21T07:55:40.364 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46247.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46247.log.gz 2024-09-21T07:55:40.364 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90934.log 2024-09-21T07:55:40.365 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49477.log 2024-09-21T07:55:40.367 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.90934.log: gzip -5 --verbose -- 87.4% -- replaced with /var/log/ceph/ceph-client.admin.90934.log.gz 2024-09-21T07:55:40.367 INFO:teuthology.orchestra.run.smithi039.stderr: /var/log/ceph/ceph-client.admin.47273.log 2024-09-21T07:55:40.368 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49477.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49477.log.gz 2024-09-21T07:55:40.368 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47050.log 2024-09-21T07:55:40.369 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100533.log 2024-09-21T07:55:40.369 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47273.log.gz 2024-09-21T07:55:40.371 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47050.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47050.log.gz 2024-09-21T07:55:40.371 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.100533.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.100533.log.gz 2024-09-21T07:55:40.372 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62718.log 2024-09-21T07:55:40.373 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62470.log 2024-09-21T07:55:40.374 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.62718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67109.log 2024-09-21T07:55:40.375 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62718.log.gz 2024-09-21T07:55:40.376 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.62470.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.62470.log.gz 2024-09-21T07:55:40.376 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85256.log 2024-09-21T07:55:40.377 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.67109.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.67109.log.gz 2024-09-21T07:55:40.377 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90724.log 2024-09-21T07:55:40.378 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.85256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102787.log 2024-09-21T07:55:40.379 INFO:teuthology.orchestra.run.smithi039.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.85256.log.gz 2024-09-21T07:55:40.380 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.90724.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.90724.log.gz 2024-09-21T07:55:40.380 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56891.log/var/log/ceph/ceph-client.admin.102787.log: 2024-09-21T07:55:40.381 INFO:teuthology.orchestra.run.smithi039.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.102787.log.gz 2024-09-21T07:55:40.382 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73631.log 2024-09-21T07:55:40.383 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52900.log 2024-09-21T07:55:40.383 INFO:teuthology.orchestra.run.smithi039.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.56891.log.gz 2024-09-21T07:55:40.385 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.73631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64737.log 2024-09-21T07:55:40.385 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.73631.log.gz 2024-09-21T07:55:40.386 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52900.log: 89.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.98298.log 2024-09-21T07:55:40.386 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52900.log.gz 2024-09-21T07:55:40.387 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.64737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110145.log 2024-09-21T07:55:40.387 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.64737.log.gz 2024-09-21T07:55:40.388 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.0.98298.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.98298.log.gz 2024-09-21T07:55:40.388 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51258.log 2024-09-21T07:55:40.389 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.110145.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.110145.log.gz 2024-09-21T07:55:40.389 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91642.log 2024-09-21T07:55:40.391 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115553.log 2024-09-21T07:55:40.391 INFO:teuthology.orchestra.run.smithi039.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.51258.log.gz 2024-09-21T07:55:40.392 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.91642.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.91642.log.gz 2024-09-21T07:55:40.393 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114923.log 2024-09-21T07:55:40.393 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.115553.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.115553.log.gz 2024-09-21T07:55:40.394 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96804.log 2024-09-21T07:55:40.395 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.114923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97708.log 2024-09-21T07:55:40.395 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.114923.log.gz 2024-09-21T07:55:40.396 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.96804.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.96804.log.gz 2024-09-21T07:55:40.396 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86202.log 2024-09-21T07:55:40.398 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.97708.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.97708.log.gzgzip 2024-09-21T07:55:40.398 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.52246.log 2024-09-21T07:55:40.399 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.86202.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.86202.log.gz 2024-09-21T07:55:40.399 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57919.log 2024-09-21T07:55:40.401 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105479.log 2024-09-21T07:55:40.401 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52246.log.gz 2024-09-21T07:55:40.402 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.57919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59590.log 2024-09-21T07:55:40.402 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57919.log.gz 2024-09-21T07:55:40.403 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.105479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74101.log 2024-09-21T07:55:40.403 INFO:teuthology.orchestra.run.smithi039.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.105479.log.gz 2024-09-21T07:55:40.405 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.59590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86346.log 2024-09-21T07:55:40.405 INFO:teuthology.orchestra.run.smithi039.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.59590.log.gz 2024-09-21T07:55:40.405 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.74101.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.74101.log.gz 2024-09-21T07:55:40.406 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96594.log 2024-09-21T07:55:40.407 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.86346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111490.log 2024-09-21T07:55:40.408 INFO:teuthology.orchestra.run.smithi039.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.86346.log.gz 2024-09-21T07:55:40.409 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.96594.log: gzip -5 --verbose 87.7% -- replaced with /var/log/ceph/ceph-client.admin.96594.log.gz 2024-09-21T07:55:40.409 INFO:teuthology.orchestra.run.smithi039.stderr: -- /var/log/ceph/ceph-client.admin.48968.log 2024-09-21T07:55:40.409 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.111490.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.111490.log.gz 2024-09-21T07:55:40.410 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85340.log 2024-09-21T07:55:40.411 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90976.log 2024-09-21T07:55:40.411 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48968.log.gz 2024-09-21T07:55:40.412 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.85340.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.85340.log.gz 2024-09-21T07:55:40.413 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.90976.log: -- /var/log/ceph/ceph-client.admin.58127.log 2024-09-21T07:55:40.413 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.90976.log.gz 2024-09-21T07:55:40.414 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71647.log 2024-09-21T07:55:40.416 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.58127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104882.log 2024-09-21T07:55:40.416 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.58127.log.gz 2024-09-21T07:55:40.417 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.71647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61712.log 2024-09-21T07:55:40.417 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71647.log.gz 2024-09-21T07:55:40.418 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.104882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46331.log 2024-09-21T07:55:40.418 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.104882.log.gz 2024-09-21T07:55:40.423 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.61712.log: gzip -5 --verbose 87.1% -- replaced with /var/log/ceph/ceph-client.admin.61712.log.gz 2024-09-21T07:55:40.423 INFO:teuthology.orchestra.run.smithi039.stderr: -- /var/log/ceph/ceph-client.admin.90537.log 2024-09-21T07:55:40.424 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46331.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46331.log.gz 2024-09-21T07:55:40.424 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54273.log 2024-09-21T07:55:40.424 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.90537.log: gzip -5 --verbose 87.4% -- replaced with /var/log/ceph/ceph-client.admin.90537.log.gz 2024-09-21T07:55:40.424 INFO:teuthology.orchestra.run.smithi039.stderr: -- /var/log/ceph/ceph-client.admin.49010.log 2024-09-21T07:55:40.424 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54273.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54273.log.gz 2024-09-21T07:55:40.424 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78927.log 2024-09-21T07:55:40.425 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71412.log 2024-09-21T07:55:40.425 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49010.log.gz 2024-09-21T07:55:40.426 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.78927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.101123.log 2024-09-21T07:55:40.427 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.78927.log.gz 2024-09-21T07:55:40.427 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.71412.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71412.log.gz 2024-09-21T07:55:40.428 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94636.log 2024-09-21T07:55:40.429 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.0.101123.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.101123.log.gz 2024-09-21T07:55:40.429 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63214.log 2024-09-21T07:55:40.430 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.94636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107386.log 2024-09-21T07:55:40.431 INFO:teuthology.orchestra.run.smithi039.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.94636.log.gz 2024-09-21T07:55:40.432 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63214.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.63214.log.gz 2024-09-21T07:55:40.432 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104007.log 2024-09-21T07:55:40.433 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.107386.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.107386.log.gz 2024-09-21T07:55:40.434 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43012.log 2024-09-21T07:55:40.435 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.104007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44825.log 2024-09-21T07:55:40.435 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.104007.log.gz 2024-09-21T07:55:40.436 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43012.log: gzip -5 87.2% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.43012.log.gz /var/log/ceph/ceph-client.admin.52956.log 2024-09-21T07:55:40.436 INFO:teuthology.orchestra.run.smithi039.stderr: 2024-09-21T07:55:40.437 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.44825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68978.log 2024-09-21T07:55:40.438 INFO:teuthology.orchestra.run.smithi039.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44825.log.gz 2024-09-21T07:55:40.439 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52956.log: gzip 85.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.52956.log.gz --verbose 2024-09-21T07:55:40.439 INFO:teuthology.orchestra.run.smithi039.stderr: -- /var/log/ceph/ceph-client.admin.86034.log 2024-09-21T07:55:40.440 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43016.log 2024-09-21T07:55:40.440 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.68978.log.gz 2024-09-21T07:55:40.442 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.86034.log: gzip -5 --verbose 87.5% -- replaced with /var/log/ceph/ceph-client.admin.86034.log.gz 2024-09-21T07:55:40.442 INFO:teuthology.orchestra.run.smithi039.stderr: -- /var/log/ceph/ceph-client.admin.109860.log 2024-09-21T07:55:40.443 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43016.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43016.log.gz 2024-09-21T07:55:40.443 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85809.log 2024-09-21T07:55:40.444 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.109860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99172.log 2024-09-21T07:55:40.444 INFO:teuthology.orchestra.run.smithi039.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.109860.log.gz 2024-09-21T07:55:40.445 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.85809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109063.log 2024-09-21T07:55:40.445 INFO:teuthology.orchestra.run.smithi039.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.85809.log.gz 2024-09-21T07:55:40.447 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.99172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96762.log 2024-09-21T07:55:40.447 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.99172.log.gz 2024-09-21T07:55:40.448 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.109063.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.109063.log.gz 2024-09-21T07:55:40.449 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51935.log 2024-09-21T07:55:40.450 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.96762.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.96762.log.gz 2024-09-21T07:55:40.450 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72954.log 2024-09-21T07:55:40.451 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68916.log 2024-09-21T07:55:40.451 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51935.log.gz 2024-09-21T07:55:40.452 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.72954.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.72954.log.gz 2024-09-21T07:55:40.453 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54014.log 2024-09-21T07:55:40.454 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68916.log: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67653.log 2024-09-21T07:55:40.454 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.68916.log.gz 2024-09-21T07:55:40.455 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56681.log 2024-09-21T07:55:40.455 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54014.log.gz 2024-09-21T07:55:40.456 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.67653.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.67653.log.gz 2024-09-21T07:55:40.457 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97401.log 2024-09-21T07:55:40.457 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56681.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56681.log.gz 2024-09-21T07:55:40.458 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48469.log 2024-09-21T07:55:40.460 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.97401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49231.log 2024-09-21T07:55:40.460 INFO:teuthology.orchestra.run.smithi039.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.97401.log.gz 2024-09-21T07:55:40.462 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48469.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48469.log.gz 2024-09-21T07:55:40.462 INFO:teuthology.orchestra.run.smithi039.stderr:gzip/var/log/ceph/ceph-client.admin.49231.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.114231.log 2024-09-21T07:55:40.463 INFO:teuthology.orchestra.run.smithi039.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49231.log.gz 2024-09-21T07:55:40.463 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43341.log 2024-09-21T07:55:40.464 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.114231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102108.log 2024-09-21T07:55:40.464 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.114231.log.gz 2024-09-21T07:55:40.465 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43341.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43341.log.gz 2024-09-21T07:55:40.466 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42370.log 2024-09-21T07:55:40.467 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.102108.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.102108.log.gz 2024-09-21T07:55:40.467 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43345.log 2024-09-21T07:55:40.468 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105353.log 2024-09-21T07:55:40.469 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42370.log.gz 2024-09-21T07:55:40.470 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101887.log 2024-09-21T07:55:40.470 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43345.log.gz 2024-09-21T07:55:40.472 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.105353.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.105353.log.gz 2024-09-21T07:55:40.472 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115175.log/var/log/ceph/ceph-client.admin.101887.log: 2024-09-21T07:55:40.472 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.101887.log.gz 2024-09-21T07:55:40.473 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86076.log 2024-09-21T07:55:40.474 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.115175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99382.log 2024-09-21T07:55:40.474 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.115175.log.gz 2024-09-21T07:55:40.475 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.86076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70107.log 2024-09-21T07:55:40.476 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.86076.log.gz 2024-09-21T07:55:40.479 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.99382.log: gzip -5 87.0% -- replaced with /var/log/ceph/ceph-client.admin.99382.log.gz 2024-09-21T07:55:40.480 INFO:teuthology.orchestra.run.smithi039.stderr: --verbose -- /var/log/ceph/ceph-client.admin.95523.log 2024-09-21T07:55:40.480 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.70107.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.70107.log.gz 2024-09-21T07:55:40.480 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102398.log 2024-09-21T07:55:40.482 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.95523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98397.log 91.1% 2024-09-21T07:55:40.482 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.95523.log.gz 2024-09-21T07:55:40.482 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.102398.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.102398.log.gz 2024-09-21T07:55:40.483 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45471.log 2024-09-21T07:55:40.484 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.98397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49872.log 2024-09-21T07:55:40.484 INFO:teuthology.orchestra.run.smithi039.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.98397.log.gz 2024-09-21T07:55:40.485 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.45471.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45471.log.gz 2024-09-21T07:55:40.486 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.49872.log: /var/log/ceph/ceph-client.admin.106422.log 2024-09-21T07:55:40.487 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49872.log.gz 2024-09-21T07:55:40.487 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52831.log 2024-09-21T07:55:40.488 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.106422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92132.log 2024-09-21T07:55:40.488 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.106422.log.gz 2024-09-21T07:55:40.489 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79137.log 2024-09-21T07:55:40.489 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52831.log.gz 2024-09-21T07:55:40.490 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.92132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112566.log 2024-09-21T07:55:40.490 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.92132.log.gz 2024-09-21T07:55:40.491 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.79137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94409.log 2024-09-21T07:55:40.491 INFO:teuthology.orchestra.run.smithi039.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.79137.log.gz 2024-09-21T07:55:40.494 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.112566.log: gzip -5 87.4% -- replaced with /var/log/ceph/ceph-client.admin.112566.log.gz 2024-09-21T07:55:40.494 INFO:teuthology.orchestra.run.smithi039.stderr: --verbose -- /var/log/ceph/ceph-client.admin.98541.log 2024-09-21T07:55:40.494 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.94409.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.94409.log.gz 2024-09-21T07:55:40.495 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55857.log 2024-09-21T07:55:40.496 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.98541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43928.log 2024-09-21T07:55:40.496 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.98541.log.gz 2024-09-21T07:55:40.498 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.55857.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55857.log.gz 2024-09-21T07:55:40.498 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.43928.log: -- /var/log/ceph/ceph-client.admin.94762.log 2024-09-21T07:55:40.498 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43928.log.gz 2024-09-21T07:55:40.498 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.45129.log 2024-09-21T07:55:40.500 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.94762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74578.log 2024-09-21T07:55:40.500 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.94762.log.gz 2024-09-21T07:55:40.500 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.2.45129.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.45129.log.gz 2024-09-21T07:55:40.501 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57961.log 2024-09-21T07:55:40.502 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.74578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103711.log 2024-09-21T07:55:40.502 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.74578.log.gz 2024-09-21T07:55:40.503 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.57961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78824.log 2024-09-21T07:55:40.504 INFO:teuthology.orchestra.run.smithi039.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.57961.log.gz 2024-09-21T07:55:40.505 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.103711.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.103711.log.gz 2024-09-21T07:55:40.506 INFO:teuthology.orchestra.run.smithi039.stderr:gzip/var/log/ceph/ceph-client.admin.78824.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.110579.log 2024-09-21T07:55:40.506 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.78824.log.gz 2024-09-21T07:55:40.507 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103341.log 2024-09-21T07:55:40.508 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.110579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51849.log 2024-09-21T07:55:40.508 INFO:teuthology.orchestra.run.smithi039.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.110579.log.gz 2024-09-21T07:55:40.509 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.103341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49565.log 2024-09-21T07:55:40.509 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.103341.log.gz 2024-09-21T07:55:40.510 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68850.log 2024-09-21T07:55:40.511 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51849.log.gz 2024-09-21T07:55:40.511 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49565.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49565.log.gz 2024-09-21T07:55:40.512 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64549.log 2024-09-21T07:55:40.513 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103059.log 2024-09-21T07:55:40.513 INFO:teuthology.orchestra.run.smithi039.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.68850.log.gz 2024-09-21T07:55:40.515 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.64549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65621.log 2024-09-21T07:55:40.515 INFO:teuthology.orchestra.run.smithi039.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.64549.log.gz/var/log/ceph/ceph-client.admin.103059.log: 2024-09-21T07:55:40.516 INFO:teuthology.orchestra.run.smithi039.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.103059.log.gz 2024-09-21T07:55:40.517 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91018.log 2024-09-21T07:55:40.517 INFO:teuthology.orchestra.run.smithi148.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-21T07:55:40.517 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.65621.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65621.log.gz 2024-09-21T07:55:40.518 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102975.log 2024-09-21T07:55:40.519 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.91018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91060.log 2024-09-21T07:55:40.519 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.91018.log.gz 2024-09-21T07:55:40.520 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.102975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79662.log 2024-09-21T07:55:40.520 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.102975.log.gz 2024-09-21T07:55:40.521 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.91060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94026.log 2024-09-21T07:55:40.522 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.91060.log.gz 2024-09-21T07:55:40.522 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.79662.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.79662.log.gz 2024-09-21T07:55:40.524 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.94026.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.94026.log.gz 2024-09-21T07:55:40.524 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59826.log 2024-09-21T07:55:40.525 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67253.log 2024-09-21T07:55:40.527 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.59826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51769.log 2024-09-21T07:55:40.527 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59826.log.gz 2024-09-21T07:55:40.529 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.67253.log: 91.0%/var/log/ceph/ceph-client.admin.51769.log: -- replaced with /var/log/ceph/ceph-client.admin.67253.log.gz 2024-09-21T07:55:40.529 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 87.4% --verbose -- replaced with /var/log/ceph/ceph-client.admin.51769.log.gz -- 2024-09-21T07:55:40.529 INFO:teuthology.orchestra.run.smithi039.stderr: /var/log/ceph/ceph-client.admin.60859.log 2024-09-21T07:55:40.530 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108573.log 2024-09-21T07:55:40.532 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86412.log 2024-09-21T07:55:40.532 INFO:teuthology.orchestra.run.smithi039.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60859.log.gz 2024-09-21T07:55:40.533 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.108573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98044.log 2024-09-21T07:55:40.533 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.108573.log.gz 2024-09-21T07:55:40.534 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.86412.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.86412.log.gz 2024-09-21T07:55:40.535 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.95244.log 2024-09-21T07:55:40.536 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.98044.log: gzip -5 --verbose -- 90.9% -- replaced with /var/log/ceph/ceph-client.admin.98044.log.gz 2024-09-21T07:55:40.536 INFO:teuthology.orchestra.run.smithi039.stderr: /var/log/ceph/ceph-client.admin.99737.log 2024-09-21T07:55:40.536 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.0.95244.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.95244.log.gz 2024-09-21T07:55:40.537 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60673.log 2024-09-21T07:55:40.539 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.99737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43689.log 2024-09-21T07:55:40.539 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.99737.log.gz 2024-09-21T07:55:40.540 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100487.log 2024-09-21T07:55:40.540 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60673.log.gz 2024-09-21T07:55:40.541 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43689.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43689.log.gz 2024-09-21T07:55:40.541 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94365.log 2024-09-21T07:55:40.542 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.100487.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.100487.log.gz 2024-09-21T07:55:40.543 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90495.log 2024-09-21T07:55:40.544 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.94365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106254.log 2024-09-21T07:55:40.544 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.94365.log.gz 2024-09-21T07:55:40.545 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.90495.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.90495.log.gz 2024-09-21T07:55:40.546 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41969.log 2024-09-21T07:55:40.546 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.106254.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.106254.log.gz 2024-09-21T07:55:40.547 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68047.log 2024-09-21T07:55:40.548 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.41969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42147.log 2024-09-21T07:55:40.548 INFO:teuthology.orchestra.run.smithi039.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.41969.log.gz 2024-09-21T07:55:40.549 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68047.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.68047.log.gz 2024-09-21T07:55:40.550 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106530.log 2024-09-21T07:55:40.550 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42147.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42147.log.gz 2024-09-21T07:55:40.551 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57763.log 2024-09-21T07:55:40.552 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.106530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80545.log 2024-09-21T07:55:40.552 INFO:teuthology.orchestra.run.smithi039.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.106530.log.gz 2024-09-21T07:55:40.553 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.57763.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.57763.log.gz 2024-09-21T07:55:40.553 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46709.log 2024-09-21T07:55:40.554 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.80545.log: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46962.log 2024-09-21T07:55:40.555 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.80545.log.gz 2024-09-21T07:55:40.556 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106212.log 2024-09-21T07:55:40.556 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46709.log.gz 2024-09-21T07:55:40.557 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46962.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46962.log.gz 2024-09-21T07:55:40.558 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45345.log 2024-09-21T07:55:40.558 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.106212.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.106212.log.gz 2024-09-21T07:55:40.558 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35974.log 2024-09-21T07:55:40.560 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.45345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108511.log 2024-09-21T07:55:40.560 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45345.log.gz 2024-09-21T07:55:40.561 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53846.log 2024-09-21T07:55:40.561 INFO:teuthology.orchestra.run.smithi039.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.35974.log.gz 2024-09-21T07:55:40.563 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.108511.log: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53034.log 2024-09-21T07:55:40.563 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.108511.log.gz 2024-09-21T07:55:40.564 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53846.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53846.log.gz 2024-09-21T07:55:40.564 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43353.log 2024-09-21T07:55:40.565 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53034.log: 83.7% -- replaced with /var/log/ceph/ceph-client.admin.53034.log.gz 2024-09-21T07:55:40.565 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74494.log 2024-09-21T07:55:40.567 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43353.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.29610.log 2024-09-21T07:55:40.567 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43353.log.gz 2024-09-21T07:55:40.568 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.74494.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.74494.log.gz 2024-09-21T07:55:40.569 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51342.log 2024-09-21T07:55:40.569 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph.tmp-client.admin.29610.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.29610.log.gz 2024-09-21T07:55:40.570 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93732.log 2024-09-21T07:55:40.571 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64615.log 2024-09-21T07:55:40.571 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51342.log.gz 2024-09-21T07:55:40.572 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.93732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85004.log 2024-09-21T07:55:40.573 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.93732.log.gz 2024-09-21T07:55:40.574 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.64615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74368.log 2024-09-21T07:55:40.574 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.64615.log.gz 2024-09-21T07:55:40.575 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.85004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53575.log 2024-09-21T07:55:40.575 INFO:teuthology.orchestra.run.smithi039.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.85004.log.gz 2024-09-21T07:55:40.576 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.74368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58516.log 2024-09-21T07:55:40.576 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.74368.log.gz 2024-09-21T07:55:40.577 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96118.log 2024-09-21T07:55:40.578 INFO:teuthology.orchestra.run.smithi039.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53575.log.gz 2024-09-21T07:55:40.579 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.58516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55815.log 2024-09-21T07:55:40.579 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58516.log.gz 2024-09-21T07:55:40.580 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.96118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85765.log 2024-09-21T07:55:40.580 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.96118.log.gz 2024-09-21T07:55:40.581 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.55815.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55815.log.gz 2024-09-21T07:55:40.582 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79095.log 2024-09-21T07:55:40.582 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.85765.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.85765.log.gz 2024-09-21T07:55:40.583 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50040.log 2024-09-21T07:55:40.584 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.79095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51071.log 2024-09-21T07:55:40.584 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.79095.log.gz 2024-09-21T07:55:40.585 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.50040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94678.log 2024-09-21T07:55:40.585 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50040.log.gz 2024-09-21T07:55:40.587 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46079.log 2024-09-21T07:55:40.587 INFO:teuthology.orchestra.run.smithi039.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51071.log.gz 2024-09-21T07:55:40.588 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.94678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104372.log 2024-09-21T07:55:40.588 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.94678.log.gz 2024-09-21T07:55:40.589 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49616.log 2024-09-21T07:55:40.589 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46079.log.gz 2024-09-21T07:55:40.590 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.104372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70416.log 2024-09-21T07:55:40.591 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.104372.log.gz 2024-09-21T07:55:40.592 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63050.log 2024-09-21T07:55:40.592 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49616.log.gz 2024-09-21T07:55:40.593 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.70416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52116.log 2024-09-21T07:55:40.593 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.70416.log.gz 2024-09-21T07:55:40.594 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63050.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.63050.log.gz 2024-09-21T07:55:40.595 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114881.log 2024-09-21T07:55:40.596 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59925.log 2024-09-21T07:55:40.596 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52116.log.gz 2024-09-21T07:55:40.597 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.114881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115301.log 2024-09-21T07:55:40.597 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.114881.log.gz 2024-09-21T07:55:40.598 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.59925.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59925.log.gz 2024-09-21T07:55:40.599 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.115301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73589.log 2024-09-21T07:55:40.599 INFO:teuthology.orchestra.run.smithi039.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.115301.log.gz 2024-09-21T07:55:40.600 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73716.log 2024-09-21T07:55:40.601 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.73589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45263.log 2024-09-21T07:55:40.602 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.73589.log.gz 2024-09-21T07:55:40.602 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.73716.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.73716.log.gz 2024-09-21T07:55:40.603 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96328.log 2024-09-21T07:55:40.604 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.45263.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45263.log.gz 2024-09-21T07:55:40.604 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43676.log 2024-09-21T07:55:40.606 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.96328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85214.log 2024-09-21T07:55:40.606 INFO:teuthology.orchestra.run.smithi039.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.96328.log.gz 2024-09-21T07:55:40.607 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43676.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43676.log.gz 2024-09-21T07:55:40.607 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.85214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65318.log 2024-09-21T07:55:40.608 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.85214.log.gz 2024-09-21T07:55:40.608 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85046.log 2024-09-21T07:55:40.610 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.65318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98002.log 2024-09-21T07:55:40.610 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.65318.log.gz 2024-09-21T07:55:40.611 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.85046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57158.log 2024-09-21T07:55:40.611 INFO:teuthology.orchestra.run.smithi039.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.85046.log.gz 2024-09-21T07:55:40.613 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.98002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103413.log 2024-09-21T07:55:40.613 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.98002.log.gz 2024-09-21T07:55:40.614 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.57158.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57158.log.gz 2024-09-21T07:55:40.614 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112341.log 2024-09-21T07:55:40.615 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.103413.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.103413.log.gz 2024-09-21T07:55:40.616 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101534.log 2024-09-21T07:55:40.617 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.112341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109947.log 2024-09-21T07:55:40.617 INFO:teuthology.orchestra.run.smithi039.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.112341.log.gz 2024-09-21T07:55:40.618 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.101534.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.101534.log.gz 2024-09-21T07:55:40.619 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114063.log 2024-09-21T07:55:40.620 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.109947.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.109947.log.gz 2024-09-21T07:55:40.620 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63448.log 2024-09-21T07:55:40.621 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.114063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60757.log 2024-09-21T07:55:40.621 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.114063.log.gz 2024-09-21T07:55:40.622 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63448.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.63448.log.gz 2024-09-21T07:55:40.623 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105522.log 2024-09-21T07:55:40.624 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60757.log: gzip 87.1% -- replaced with /var/log/ceph/ceph-client.admin.60757.log.gz -5 2024-09-21T07:55:40.625 INFO:teuthology.orchestra.run.smithi039.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48679.log 2024-09-21T07:55:40.626 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.105522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93690.log 2024-09-21T07:55:40.626 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.105522.log.gz 2024-09-21T07:55:40.626 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111602.log 2024-09-21T07:55:40.627 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48679.log.gz 2024-09-21T07:55:40.628 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.93690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56761.log 2024-09-21T07:55:40.628 INFO:teuthology.orchestra.run.smithi039.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.93690.log.gz 2024-09-21T07:55:40.629 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.111602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90327.log 2024-09-21T07:55:40.629 INFO:teuthology.orchestra.run.smithi039.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.111602.log.gz 2024-09-21T07:55:40.630 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61568.log 2024-09-21T07:55:40.630 INFO:teuthology.orchestra.run.smithi039.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56761.log.gz 2024-09-21T07:55:40.631 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.90327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91102.log 2024-09-21T07:55:40.631 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.90327.log.gz 2024-09-21T07:55:40.633 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.61568.log: gzip -5 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61568.log.gz 2024-09-21T07:55:40.633 INFO:teuthology.orchestra.run.smithi039.stderr: --verbose -- /var/log/ceph/ceph-client.admin.85680.log 2024-09-21T07:55:40.634 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.91102.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.91102.log.gz 2024-09-21T07:55:40.634 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100122.log 2024-09-21T07:55:40.635 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.85680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79929.log 2024-09-21T07:55:40.635 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.85680.log.gz 2024-09-21T07:55:40.637 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.100122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107428.log 2024-09-21T07:55:40.637 INFO:teuthology.orchestra.run.smithi039.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.100122.log.gz 2024-09-21T07:55:40.638 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.79929.log: gzip 87.0% -- replaced with /var/log/ceph/ceph-client.admin.79929.log.gz 2024-09-21T07:55:40.638 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.92419.log 2024-09-21T07:55:40.639 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.107428.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.107428.log.gz 2024-09-21T07:55:40.639 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42668.log 2024-09-21T07:55:40.641 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.92419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96846.log 2024-09-21T07:55:40.641 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.92419.log.gz 2024-09-21T07:55:40.642 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107736.log 2024-09-21T07:55:40.642 INFO:teuthology.orchestra.run.smithi039.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.42668.log.gz 2024-09-21T07:55:40.643 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.96846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36276.log 2024-09-21T07:55:40.643 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.96846.log.gz 2024-09-21T07:55:40.644 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.107736.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.107736.log.gz 2024-09-21T07:55:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47399.log 2024-09-21T07:55:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36276.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.36276.log.gz 2024-09-21T07:55:40.646 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53407.log 2024-09-21T07:55:40.647 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65402.log 2024-09-21T07:55:40.647 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47399.log.gz 2024-09-21T07:55:40.648 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103777.log 2024-09-21T07:55:40.648 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53407.log.gz 2024-09-21T07:55:40.650 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.65402.log: gzip -5 87.2% --verbose -- /var/log/ceph/ceph-client.admin.104588.log 2024-09-21T07:55:40.650 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65402.log.gz 2024-09-21T07:55:40.651 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.103777.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.103777.log.gz 2024-09-21T07:55:40.651 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112926.log 2024-09-21T07:55:40.652 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.104588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60294.log 2024-09-21T07:55:40.653 INFO:teuthology.orchestra.run.smithi039.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.104588.log.gz 2024-09-21T07:55:40.654 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.112926.log: gzip -5 --verbose 87.8% -- /var/log/ceph/ceph-client.admin.74326.log 2024-09-21T07:55:40.654 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.112926.log.gz 2024-09-21T07:55:40.654 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60294.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60294.log.gz 2024-09-21T07:55:40.655 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80353.log 2024-09-21T07:55:40.656 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.74326.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.74326.log.gz 2024-09-21T07:55:40.657 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.80353.log: -- /var/log/ceph/ceph-client.admin.106170.log 2024-09-21T07:55:40.658 INFO:teuthology.orchestra.run.smithi039.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.80353.log.gz 2024-09-21T07:55:40.658 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54056.log 2024-09-21T07:55:40.660 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.106170.log: gzip -5 --verbose 87.1% -- replaced with /var/log/ceph/ceph-client.admin.106170.log.gz 2024-09-21T07:55:40.660 INFO:teuthology.orchestra.run.smithi039.stderr: -- /var/log/ceph/ceph-client.admin.112776.log 2024-09-21T07:55:40.661 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54056.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54056.log.gz 2024-09-21T07:55:40.661 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106850.log 2024-09-21T07:55:40.662 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.112776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92024.log 2024-09-21T07:55:40.663 INFO:teuthology.orchestra.run.smithi039.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.112776.log.gz 2024-09-21T07:55:40.664 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.106850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71145.log 2024-09-21T07:55:40.664 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.106850.log.gz 2024-09-21T07:55:40.665 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.92024.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.92024.log.gz 2024-09-21T07:55:40.665 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49746.log 2024-09-21T07:55:40.666 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.71145.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-21T07:55:40.666 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71145.log.gz 2024-09-21T07:55:40.668 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49746.log: gzip -5 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49746.log.gz 2024-09-21T07:55:40.668 INFO:teuthology.orchestra.run.smithi039.stderr: --verbose -- /var/log/ceph/ceph-client.admin.58558.log 2024-09-21T07:55:40.669 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53491.log 2024-09-21T07:55:40.670 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.58558.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.58558.log.gz 2024-09-21T07:55:40.671 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42319.log/var/log/ceph/ceph-client.admin.53491.log: 2024-09-21T07:55:40.672 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53491.log.gz 2024-09-21T07:55:40.672 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80017.log 2024-09-21T07:55:40.674 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42319.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42319.log.gz 2024-09-21T07:55:40.675 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97357.log 2024-09-21T07:55:40.675 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.80017.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.80017.log.gz 2024-09-21T07:55:40.676 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46613.log 2024-09-21T07:55:40.677 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.97357.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.97357.log.gz 2024-09-21T07:55:40.678 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99527.log 2024-09-21T07:55:40.679 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46613.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46613.log.gz 2024-09-21T07:55:40.680 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112818.log 2024-09-21T07:55:40.680 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.99527.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.99527.log.gz 2024-09-21T07:55:40.681 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52414.log 2024-09-21T07:55:40.683 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.112818.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.112818.log.gz 2024-09-21T07:55:40.683 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68571.log 2024-09-21T07:55:40.684 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52414.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52414.log.gz 2024-09-21T07:55:40.684 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72298.log 2024-09-21T07:55:40.686 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68571.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68571.log.gz 2024-09-21T07:55:40.687 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85853.log 2024-09-21T07:55:40.688 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.72298.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.72298.log.gz 2024-09-21T07:55:40.688 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66902.log 2024-09-21T07:55:40.689 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.85853.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.85853.log.gz 2024-09-21T07:55:40.690 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103645.log 2024-09-21T07:55:40.691 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.66902.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.66902.log.gz 2024-09-21T07:55:40.692 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68168.log 2024-09-21T07:55:40.693 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.103645.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.103645.log.gz 2024-09-21T07:55:40.694 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68168.log: gzip -5 --verbose -- 87.3% -- replaced with /var/log/ceph/ceph-client.admin.68168.log.gz 2024-09-21T07:55:40.694 INFO:teuthology.orchestra.run.smithi039.stderr: /var/log/ceph/ceph-mds.c.log 2024-09-21T07:55:40.696 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54708.log 2024-09-21T07:55:40.699 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-mds.c.log: /var/log/ceph/ceph-client.admin.54708.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54708.log.gz 2024-09-21T07:55:40.699 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97918.log 2024-09-21T07:55:40.702 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.97918.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.97918.log.gz 2024-09-21T07:55:40.702 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100659.log 2024-09-21T07:55:40.705 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.100659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60444.log 2024-09-21T07:55:40.706 INFO:teuthology.orchestra.run.smithi039.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.100659.log.gz 2024-09-21T07:55:40.706 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65464.log 2024-09-21T07:55:40.708 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.65464.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65464.log.gz 2024-09-21T07:55:40.710 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60444.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.60444.log.gz 2024-09-21T07:55:40.711 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114779.log 2024-09-21T07:55:40.714 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.114779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42752.log 2024-09-21T07:55:40.715 INFO:teuthology.orchestra.run.smithi039.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.114779.log.gz 2024-09-21T07:55:40.716 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46289.log 2024-09-21T07:55:40.720 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42752.log: /var/log/ceph/ceph-client.admin.46289.log: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109902.log 2024-09-21T07:55:40.722 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.109902.log: -- replaced with /var/log/ceph/ceph-client.admin.42752.log.gz 2024-09-21T07:55:40.725 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102170.log 2024-09-21T07:55:40.725 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.46289.log.gz 2024-09-21T07:55:40.727 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.109902.log.gz 2024-09-21T07:55:40.727 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50615.log 2024-09-21T07:55:40.729 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.50615.log: /var/log/ceph/ceph-client.admin.102170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63885.log 2024-09-21T07:55:40.730 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% 85.2% -- replaced with /var/log/ceph/ceph-client.admin.102170.log.gz 2024-09-21T07:55:40.731 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50615.log.gz 2024-09-21T07:55:40.731 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102294.log 2024-09-21T07:55:40.732 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63885.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.63885.log.gz 2024-09-21T07:55:40.734 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51155.log 2024-09-21T07:55:40.735 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.102294.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.102294.log.gz 2024-09-21T07:55:40.739 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51155.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51155.log.gz 2024-09-21T07:55:40.739 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110249.log 2024-09-21T07:55:40.741 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.110249.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.110249.log.gz 2024-09-21T07:55:40.741 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99653.log 2024-09-21T07:55:40.744 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.99653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107676.log 2024-09-21T07:55:40.745 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.99653.log.gz 2024-09-21T07:55:40.747 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64283.log 2024-09-21T07:55:40.751 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.107676.log: /var/log/ceph/ceph-client.admin.64283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90621.log 2024-09-21T07:55:40.751 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.107676.log.gz 2024-09-21T07:55:40.752 INFO:teuthology.orchestra.run.smithi039.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.64283.log.gz 2024-09-21T07:55:40.753 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66720.log 2024-09-21T07:55:40.758 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.90621.log: /var/log/ceph/ceph-client.admin.66720.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.90621.log.gz 2024-09-21T07:55:40.758 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72996.log 2024-09-21T07:55:40.760 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.66720.log.gz 2024-09-21T07:55:40.760 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45995.log 2024-09-21T07:55:40.764 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.45995.log: /var/log/ceph/ceph-client.admin.72996.log: 87.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43329.log 2024-09-21T07:55:40.764 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.72996.log.gz 2024-09-21T07:55:40.765 INFO:teuthology.orchestra.run.smithi039.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45995.log.gz 2024-09-21T07:55:40.767 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90892.log 2024-09-21T07:55:40.770 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.90892.log: /var/log/ceph/ceph-client.admin.43329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112297.log 2024-09-21T07:55:40.772 INFO:teuthology.orchestra.run.smithi039.stderr: 87.3% 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43329.log.gz 2024-09-21T07:55:40.772 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.90892.log.gz 2024-09-21T07:55:40.774 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.112297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105230.log 2024-09-21T07:55:40.776 INFO:teuthology.orchestra.run.smithi039.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.112297.log.gz 2024-09-21T07:55:40.777 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54354.log 2024-09-21T07:55:40.780 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54354.log: /var/log/ceph/ceph-client.admin.105230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111074.log 2024-09-21T07:55:40.783 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98439.log 2024-09-21T07:55:40.783 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.105230.log.gz 2024-09-21T07:55:40.783 INFO:teuthology.orchestra.run.smithi039.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54354.log.gz 2024-09-21T07:55:40.785 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.111074.log: /var/log/ceph/ceph-client.admin.98439.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.111074.log.gz 2024-09-21T07:55:40.785 INFO:teuthology.orchestra.run.smithi039.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.98439.log.gz 2024-09-21T07:55:40.786 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55372.log 2024-09-21T07:55:40.788 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.55372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53365.log 2024-09-21T07:55:40.789 INFO:teuthology.orchestra.run.smithi039.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55372.log.gz 2024-09-21T07:55:40.791 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55773.log 2024-09-21T07:55:40.795 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53365.log: /var/log/ceph/ceph-client.admin.55773.log: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71187.log 2024-09-21T07:55:40.795 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53365.log.gz 2024-09-21T07:55:40.796 INFO:teuthology.orchestra.run.smithi039.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55773.log.gz 2024-09-21T07:55:40.798 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74452.log 2024-09-21T07:55:40.802 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.71187.log: /var/log/ceph/ceph-client.admin.74452.log: 87.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47693.log 2024-09-21T07:55:40.802 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.71187.log.gz 2024-09-21T07:55:40.804 INFO:teuthology.orchestra.run.smithi039.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.74452.log.gz 2024-09-21T07:55:40.804 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72062.log 2024-09-21T07:55:40.806 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47693.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.47693.log.gz 2024-09-21T07:55:40.808 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.72062.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.72062.log.gz 2024-09-21T07:55:40.809 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68529.log 2024-09-21T07:55:40.812 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105648.log 2024-09-21T07:55:40.814 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.105648.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.105648.log.gz 2024-09-21T07:55:40.815 INFO:teuthology.orchestra.run.smithi039.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.68529.log.gz 2024-09-21T07:55:41.156 INFO:teuthology.orchestra.run.smithi039.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-21T07:55:41.375 INFO:teuthology.orchestra.run.smithi148.stderr: 94.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-21T07:55:41.608 INFO:teuthology.orchestra.run.smithi148.stderr: 94.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-21T07:55:41.656 INFO:teuthology.orchestra.run.smithi039.stderr: 94.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-21T07:55:41.755 INFO:teuthology.orchestra.run.smithi039.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-21T07:55:41.925 INFO:teuthology.orchestra.run.smithi148.stderr: 95.3% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-21T07:55:41.957 INFO:teuthology.orchestra.run.smithi148.stderr: 94.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-21T07:55:41.960 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-09-21T07:55:41.961 INFO:teuthology.orchestra.run.smithi148.stderr:real 0m2.493s 2024-09-21T07:55:41.961 INFO:teuthology.orchestra.run.smithi148.stderr:user 0m10.232s 2024-09-21T07:55:41.961 INFO:teuthology.orchestra.run.smithi148.stderr:sys 0m1.009s 2024-09-21T07:55:41.986 INFO:teuthology.orchestra.run.smithi039.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-21T07:55:42.319 INFO:teuthology.orchestra.run.smithi039.stderr: 94.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-21T07:55:42.378 INFO:teuthology.orchestra.run.smithi039.stderr: 95.2% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-21T07:55:43.344 INFO:teuthology.orchestra.run.smithi039.stderr: 94.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-21T07:55:43.346 INFO:teuthology.orchestra.run.smithi039.stderr: 2024-09-21T07:55:43.347 INFO:teuthology.orchestra.run.smithi039.stderr:real 0m3.883s 2024-09-21T07:55:43.347 INFO:teuthology.orchestra.run.smithi039.stderr:user 0m12.937s 2024-09-21T07:55:43.347 INFO:teuthology.orchestra.run.smithi039.stderr:sys 0m3.651s 2024-09-21T07:55:43.348 INFO:tasks.ceph:Archiving logs... 2024-09-21T07:55:43.348 DEBUG:teuthology.misc:Transferring archived files from smithi039:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913602/remote/smithi039/log 2024-09-21T07:55:43.349 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T07:55:45.588 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913602/remote/smithi148/log 2024-09-21T07:55:45.589 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T07:55:46.974 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T07:55:46.986 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T07:55:46.987 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T07:55:46.991 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T07:55:47.108 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T07:55:47.116 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T07:55:47.117 DEBUG:teuthology.orchestra.run.smithi039:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T07:55:47.121 DEBUG:teuthology.orchestra.run.smithi148:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T07:55:47.152 INFO:teuthology.orchestra.run.smithi039.stderr:bash: line 1: ntpq: command not found 2024-09-21T07:55:47.156 INFO:teuthology.orchestra.run.smithi148.stderr:bash: line 1: ntpq: command not found 2024-09-21T07:55:47.160 INFO:teuthology.orchestra.run.smithi039.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T07:55:47.160 INFO:teuthology.orchestra.run.smithi039.stdout:=============================================================================== 2024-09-21T07:55:47.160 INFO:teuthology.orchestra.run.smithi039.stdout:^* hv01.front.sepia.ceph.com 3 6 377 40 +50us[ +49us] +/- 48ms 2024-09-21T07:55:47.160 INFO:teuthology.orchestra.run.smithi039.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 44 -143us[ -144us] +/- 57ms 2024-09-21T07:55:47.160 INFO:teuthology.orchestra.run.smithi039.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 43 +144us[ +144us] +/- 77ms 2024-09-21T07:55:47.160 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T07:55:47.164 INFO:teuthology.orchestra.run.smithi148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T07:55:47.164 INFO:teuthology.orchestra.run.smithi148.stdout:=============================================================================== 2024-09-21T07:55:47.164 INFO:teuthology.orchestra.run.smithi148.stdout:^* hv01.front.sepia.ceph.com 3 6 377 42 +44us[ +46us] +/- 48ms 2024-09-21T07:55:47.164 INFO:teuthology.orchestra.run.smithi148.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 42 -154us[ -154us] +/- 57ms 2024-09-21T07:55:47.164 INFO:teuthology.orchestra.run.smithi148.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 43 +154us[ +156us] +/- 77ms 2024-09-21T07:55:47.164 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T07:55:47.165 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T07:55:47.174 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T07:55:47.174 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T07:55:47.185 DEBUG:teuthology.orchestra.run.smithi039:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T07:55:47.367 DEBUG:teuthology.orchestra.run.smithi148:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T07:55:47.489 DEBUG:teuthology.orchestra.run.smithi039:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-21T07:55:47.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:55:47.553 DEBUG:teuthology.orchestra.run.smithi148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-21T07:55:47.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:55:47.614 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T07:55:47.623 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T07:55:47.639 INFO:teuthology.task.internal:Duration was 1821.487407 seconds 2024-09-21T07:55:47.639 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T07:55:47.647 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T07:55:47.647 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T07:55:47.651 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T07:55:47.735 INFO:teuthology.orchestra.run.smithi039.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T07:55:47.741 INFO:teuthology.orchestra.run.smithi148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T07:55:48.143 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T07:55:48.143 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi039.front.sepia.ceph.com 2024-09-21T07:55:48.143 DEBUG:teuthology.orchestra.run.smithi039:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T07:55:48.193 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi148.front.sepia.ceph.com 2024-09-21T07:55:48.193 DEBUG:teuthology.orchestra.run.smithi148:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T07:55:48.244 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T07:55:48.244 DEBUG:teuthology.orchestra.run.smithi039:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T07:55:48.248 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T07:55:48.309 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T07:55:48.309 DEBUG:teuthology.orchestra.run.smithi039:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T07:55:48.313 DEBUG:teuthology.orchestra.run.smithi148:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T07:55:48.526 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T07:55:48.536 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T07:55:48.536 DEBUG:teuthology.orchestra.run.smithi039:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T07:55:48.592 DEBUG:teuthology.orchestra.run.smithi148:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T07:55:48.648 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T07:55:48.658 DEBUG:teuthology.orchestra.run.smithi039:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T07:55:48.662 DEBUG:teuthology.orchestra.run.smithi148:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T07:55:48.729 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern = core 2024-09-21T07:55:48.732 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = core 2024-09-21T07:55:48.762 DEBUG:teuthology.orchestra.run.smithi039:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T07:55:48.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:55:48.795 DEBUG:teuthology.orchestra.run.smithi148:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T07:55:48.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:55:48.829 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T07:55:48.839 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T07:55:48.840 DEBUG:teuthology.misc:Transferring archived files from smithi039:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913602/remote/smithi039 2024-09-21T07:55:48.840 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T07:55:48.908 DEBUG:teuthology.misc:Transferring archived files from smithi148:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913602/remote/smithi148 2024-09-21T07:55:48.908 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T07:55:48.971 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T07:55:48.971 DEBUG:teuthology.orchestra.run.smithi039:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T07:55:48.975 DEBUG:teuthology.orchestra.run.smithi148:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T07:55:49.009 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T07:55:49.021 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T07:55:49.021 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T07:55:49.030 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T07:55:49.030 DEBUG:teuthology.orchestra.run.smithi039:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T07:55:49.052 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T07:55:49.082 INFO:teuthology.orchestra.run.smithi039.stdout: 393550 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 07:55 /home/ubuntu/cephtest 2024-09-21T07:55:49.087 INFO:teuthology.orchestra.run.smithi148.stdout: 393233 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 07:55 /home/ubuntu/cephtest 2024-09-21T07:55:49.089 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T07:55:49.100 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T07:55:49.159 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a client.1-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/snapshots} duration: 1821.487407207489 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-09-21T07:55:49.160 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T07:55:49.249 INFO:teuthology.run:pass